2024-09-14T06:18:06.776 INFO:root:teuthology version: 1.2.2.dev2+g3752d38 2024-09-14T06:18:06.793 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-09-14T06:18:06.871 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/teuthology-2024-09-10_20:16:17-rbd-main-distro-default-smithi/7900381 branch: main description: rbd/thrash/{base/install clusters/{fixed-2 openstack} conf/{disable-pool-app} msgr-failures/few objectstore/bluestore-comp-snappy supported-random-distro$/{centos_latest} thrashers/default thrashosds-health workloads/rbd_fsx_deep_copy} email: ceph-qa@ceph.com first_in_suite: false job_id: '7900381' kernel: kdb: 1 sha1: distro last_in_suite: false machine_type: smithi name: teuthology-2024-09-10_20:16:17-rbd-main-distro-default-smithi no_nested_subset: false openstack: - volumes: count: 4 size: 10 - machine: cpus: 1 disk: 40 ram: 8000 volumes: count: 4 size: 30 os_type: centos os_version: 9.stream overrides: admin_socket: branch: main ceph: conf: global: mon client directed command retry: 5 mon warn on pool no app: false ms inject socket failures: 5000 mgr: debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 osd: bluestore block size: 96636764160 bluestore compression algorithm: snappy bluestore compression mode: aggressive bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug ms: 1 debug osd: 20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd blocked scrub grace period: 3600 osd failsafe full ratio: 0.95 osd max markdown count: 1000 osd objectstore: bluestore osd shutdown pgref assert: true flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - \(OSD_SLOW_PING_TIME - but it is still running - objects unfound and apparently lost - overall HEALTH_ - \(OSDMAP_FLAGS\) - \(OSD_ - \(PG_ - \(POOL_ - \(CACHE_POOL_ - \(SMALLER_PGP_NUM\) - \(OBJECT_ - SLOW_OPS - \(REQUEST_SLOW\) - \(TOO_FEW_PGS\) - slow request - timeout on replica - late reservation from - MON_DOWN - OSDMAP_FLAGS - OSD_DOWN - PG_DEGRADED - PG_AVAILABILITY - POOL_APP_NOT_ENABLED - mons down - mon down - out of quorum - noscrub - nodeep-scrub sha1: 7f24e589328ae9e639c0009d93340e5b235d0a47 ceph-deploy: conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} install: ceph: flavor: default sha1: 7f24e589328ae9e639c0009d93340e5b235d0a47 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: main sha1: 0d61375b8ab53be4c233f5b93e463edfdff9316a owner: scheduled_teuthology@teuthology priority: 950 repo: https://git.ceph.com/ceph.git roles: - - mon.a - mon.c - mgr.y - osd.0 - osd.1 - osd.2 - osd.3 - client.0 - node-exporter.a - - mon.b - mgr.x - osd.4 - osd.5 - osd.6 - osd.7 - client.1 - prometheus.a - node-exporter.b seed: 5839 sha1: 7f24e589328ae9e639c0009d93340e5b235d0a47 sleep_before_teardown: 0 subset: 99/128 suite: rbd suite_branch: main suite_path: /home/teuthworker/src/git.ceph.com_ceph_0d61375b8ab53be4c233f5b93e463edfdff9316a/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph.git suite_sha1: 0d61375b8ab53be4c233f5b93e463edfdff9316a targets: smithi094.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBMq5orWEJUtY0478KVTlybB6zawRp9ZyFFP3FnqhItrepeTUH6nq/xoX4gF0E++iaEjoDM+Xx0iVas3Gsbg/8p8= smithi138.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBP1hR8CCordjszdEK541OFTVIl0tYp1j7Fan1xPdPvI/fL3IlnOf08uS5vEuocEtpp8IGyv/Ud1buZ/DaYyrs6Y= tasks: - install: null - ceph: null - thrashosds: timeout: 1200 - rbd_fsx: clients: - client.0 deep_copy: true ops: 6000 teuthology: fragments_dropped: [] meta: {} postmerge: [] teuthology_branch: main teuthology_sha1: 3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a timestamp: 2024-09-10_20:16:17 tube: smithi user: teuthology verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.3401856 2024-09-14T06:18:06.871 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph_0d61375b8ab53be4c233f5b93e463edfdff9316a/qa; will attempt to use it 2024-09-14T06:18:06.872 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph_0d61375b8ab53be4c233f5b93e463edfdff9316a/qa/tasks 2024-09-14T06:18:06.872 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-09-14T06:18:06.873 INFO:teuthology.task.internal:Checking packages... 2024-09-14T06:18:06.894 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash '7f24e589328ae9e639c0009d93340e5b235d0a47' 2024-09-14T06:18:06.894 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-09-14T06:18:06.894 INFO:teuthology.packaging:ref: None 2024-09-14T06:18:06.894 INFO:teuthology.packaging:tag: None 2024-09-14T06:18:06.894 INFO:teuthology.packaging:branch: main 2024-09-14T06:18:06.895 INFO:teuthology.packaging:sha1: 7f24e589328ae9e639c0009d93340e5b235d0a47 2024-09-14T06:18:06.895 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=main 2024-09-14T06:18:07.147 INFO:teuthology.task.internal:Found packages for ceph version 19.3.0-4926.g1c941a66 2024-09-14T06:18:07.149 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-09-14T06:18:07.155 INFO:teuthology.task.internal:no buildpackages task found 2024-09-14T06:18:07.156 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-09-14T06:18:07.226 INFO:teuthology.task.internal:Saving configuration 2024-09-14T06:18:07.239 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-09-14T06:18:07.252 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-09-14T06:18:07.279 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi094.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2024-09-10_20:16:17-rbd-main-distro-default-smithi/7900381', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9.stream', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-09-14 06:13:17.332408', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBMq5orWEJUtY0478KVTlybB6zawRp9ZyFFP3FnqhItrepeTUH6nq/xoX4gF0E++iaEjoDM+Xx0iVas3Gsbg/8p8='} 2024-09-14T06:18:07.304 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi138.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2024-09-10_20:16:17-rbd-main-distro-default-smithi/7900381', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9.stream', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-09-14 06:13:17.330921', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBP1hR8CCordjszdEK541OFTVIl0tYp1j7Fan1xPdPvI/fL3IlnOf08uS5vEuocEtpp8IGyv/Ud1buZ/DaYyrs6Y='} 2024-09-14T06:18:07.304 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-09-14T06:18:07.311 INFO:teuthology.task.internal:roles: ubuntu@smithi094.front.sepia.ceph.com - ['mon.a', 'mon.c', 'mgr.y', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'node-exporter.a'] 2024-09-14T06:18:07.311 INFO:teuthology.task.internal:roles: ubuntu@smithi138.front.sepia.ceph.com - ['mon.b', 'mgr.x', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'prometheus.a', 'node-exporter.b'] 2024-09-14T06:18:07.311 INFO:teuthology.run_tasks:Running task console_log... 2024-09-14T06:18:07.371 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f68504b7400>, signals=[15]) 2024-09-14T06:18:07.371 INFO:teuthology.run_tasks:Running task internal.connect... 2024-09-14T06:18:07.378 INFO:teuthology.task.internal:Opening connections... 2024-09-14T06:18:07.378 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi094.front.sepia.ceph.com 2024-09-14T06:18:07.379 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi094.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-14T06:18:07.453 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi138.front.sepia.ceph.com 2024-09-14T06:18:07.454 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi138.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-14T06:18:07.526 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-09-14T06:18:07.533 DEBUG:teuthology.orchestra.run.smithi094:> uname -m 2024-09-14T06:18:07.550 INFO:teuthology.orchestra.run.smithi094.stdout:x86_64 2024-09-14T06:18:07.551 DEBUG:teuthology.orchestra.run.smithi094:> cat /etc/os-release 2024-09-14T06:18:07.609 INFO:teuthology.orchestra.run.smithi094.stdout:NAME="CentOS Stream" 2024-09-14T06:18:07.609 INFO:teuthology.orchestra.run.smithi094.stdout:VERSION="9" 2024-09-14T06:18:07.609 INFO:teuthology.orchestra.run.smithi094.stdout:ID="centos" 2024-09-14T06:18:07.609 INFO:teuthology.orchestra.run.smithi094.stdout:ID_LIKE="rhel fedora" 2024-09-14T06:18:07.609 INFO:teuthology.orchestra.run.smithi094.stdout:VERSION_ID="9" 2024-09-14T06:18:07.610 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM_ID="platform:el9" 2024-09-14T06:18:07.610 INFO:teuthology.orchestra.run.smithi094.stdout:PRETTY_NAME="CentOS Stream 9" 2024-09-14T06:18:07.610 INFO:teuthology.orchestra.run.smithi094.stdout:ANSI_COLOR="0;31" 2024-09-14T06:18:07.610 INFO:teuthology.orchestra.run.smithi094.stdout:LOGO="fedora-logo-icon" 2024-09-14T06:18:07.610 INFO:teuthology.orchestra.run.smithi094.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-09-14T06:18:07.611 INFO:teuthology.orchestra.run.smithi094.stdout:HOME_URL="https://centos.org/" 2024-09-14T06:18:07.611 INFO:teuthology.orchestra.run.smithi094.stdout:BUG_REPORT_URL="https://issues.redhat.com/" 2024-09-14T06:18:07.611 INFO:teuthology.orchestra.run.smithi094.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-09-14T06:18:07.611 INFO:teuthology.orchestra.run.smithi094.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-09-14T06:18:07.612 INFO:teuthology.lock.ops:Updating smithi094.front.sepia.ceph.com on lock server 2024-09-14T06:18:07.633 DEBUG:teuthology.orchestra.run.smithi138:> uname -m 2024-09-14T06:18:07.652 INFO:teuthology.orchestra.run.smithi138.stdout:x86_64 2024-09-14T06:18:07.652 DEBUG:teuthology.orchestra.run.smithi138:> cat /etc/os-release 2024-09-14T06:18:07.710 INFO:teuthology.orchestra.run.smithi138.stdout:NAME="CentOS Stream" 2024-09-14T06:18:07.710 INFO:teuthology.orchestra.run.smithi138.stdout:VERSION="9" 2024-09-14T06:18:07.710 INFO:teuthology.orchestra.run.smithi138.stdout:ID="centos" 2024-09-14T06:18:07.710 INFO:teuthology.orchestra.run.smithi138.stdout:ID_LIKE="rhel fedora" 2024-09-14T06:18:07.710 INFO:teuthology.orchestra.run.smithi138.stdout:VERSION_ID="9" 2024-09-14T06:18:07.711 INFO:teuthology.orchestra.run.smithi138.stdout:PLATFORM_ID="platform:el9" 2024-09-14T06:18:07.711 INFO:teuthology.orchestra.run.smithi138.stdout:PRETTY_NAME="CentOS Stream 9" 2024-09-14T06:18:07.711 INFO:teuthology.orchestra.run.smithi138.stdout:ANSI_COLOR="0;31" 2024-09-14T06:18:07.711 INFO:teuthology.orchestra.run.smithi138.stdout:LOGO="fedora-logo-icon" 2024-09-14T06:18:07.711 INFO:teuthology.orchestra.run.smithi138.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-09-14T06:18:07.711 INFO:teuthology.orchestra.run.smithi138.stdout:HOME_URL="https://centos.org/" 2024-09-14T06:18:07.711 INFO:teuthology.orchestra.run.smithi138.stdout:BUG_REPORT_URL="https://issues.redhat.com/" 2024-09-14T06:18:07.711 INFO:teuthology.orchestra.run.smithi138.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-09-14T06:18:07.711 INFO:teuthology.orchestra.run.smithi138.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-09-14T06:18:07.711 INFO:teuthology.lock.ops:Updating smithi138.front.sepia.ceph.com on lock server 2024-09-14T06:18:07.732 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-09-14T06:18:07.742 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-09-14T06:18:07.747 INFO:teuthology.task.internal:Checking for old test directory... 2024-09-14T06:18:07.747 DEBUG:teuthology.orchestra.run.smithi094:> test '!' -e /home/ubuntu/cephtest 2024-09-14T06:18:07.749 DEBUG:teuthology.orchestra.run.smithi138:> test '!' -e /home/ubuntu/cephtest 2024-09-14T06:18:07.767 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-09-14T06:18:07.773 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-09-14T06:18:07.773 DEBUG:teuthology.orchestra.run.smithi094:> test -z $(ls -A /var/lib/ceph) 2024-09-14T06:18:07.807 DEBUG:teuthology.orchestra.run.smithi138:> test -z $(ls -A /var/lib/ceph) 2024-09-14T06:18:07.905 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-09-14T06:18:07.962 INFO:teuthology.run_tasks:Running task kernel... 2024-09-14T06:18:07.976 INFO:teuthology.task.kernel:normalize config orig: {'kdb': 1, 'sha1': 'distro'} 2024-09-14T06:18:07.976 INFO:teuthology.task.kernel:config {'mon.a': {'kdb': 1, 'sha1': 'distro'}, 'mon.b': {'kdb': 1, 'sha1': 'distro'}}, timeout 300 2024-09-14T06:18:07.977 DEBUG:teuthology.orchestra.run.smithi094:> test -f /run/.containerenv -o -f /.dockerenv 2024-09-14T06:18:07.977 DEBUG:teuthology.orchestra.run.smithi138:> test -f /run/.containerenv -o -f /.dockerenv 2024-09-14T06:18:07.994 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-14T06:18:07.994 DEBUG:teuthology.orchestra.run.smithi094:> uname -r 2024-09-14T06:18:07.995 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-14T06:18:07.995 DEBUG:teuthology.orchestra.run.smithi138:> uname -r 2024-09-14T06:18:08.053 INFO:teuthology.orchestra.run.smithi094.stdout:5.14.0-503.el9.x86_64 2024-09-14T06:18:08.053 INFO:teuthology.task.kernel:Running kernel on smithi094: 5.14.0-503.el9.x86_64 2024-09-14T06:18:08.053 DEBUG:teuthology.orchestra.run.smithi094:> sudo yum install -y kernel 2024-09-14T06:18:08.054 INFO:teuthology.orchestra.run.smithi138.stdout:5.14.0-503.el9.x86_64 2024-09-14T06:18:08.054 INFO:teuthology.task.kernel:Running kernel on smithi138: 5.14.0-503.el9.x86_64 2024-09-14T06:18:08.054 DEBUG:teuthology.orchestra.run.smithi138:> sudo yum install -y kernel 2024-09-14T06:18:10.435 INFO:teuthology.orchestra.run.smithi094.stdout:CentOS Stream 9 - BaseOS 15 MB/s | 8.2 MB 00:00 2024-09-14T06:18:26.338 INFO:teuthology.orchestra.run.smithi138.stdout:CentOS Stream 9 - BaseOS 512 kB/s | 8.2 MB 00:16 2024-09-14T06:18:29.200 INFO:teuthology.orchestra.run.smithi138.stdout:CentOS Stream 9 - AppStream 12 MB/s | 20 MB 00:01 2024-09-14T06:18:30.700 INFO:teuthology.orchestra.run.smithi094.stdout:CentOS Stream 9 - AppStream 1.1 MB/s | 20 MB 00:19 2024-09-14T06:18:33.648 INFO:teuthology.orchestra.run.smithi138.stdout:CentOS Stream 9 - CRB 10 MB/s | 6.5 MB 00:00 2024-09-14T06:18:35.173 INFO:teuthology.orchestra.run.smithi138.stdout:CentOS Stream 9 - Extras packages 59 kB/s | 19 kB 00:00 2024-09-14T06:18:35.426 INFO:teuthology.orchestra.run.smithi094.stdout:CentOS Stream 9 - CRB 7.0 MB/s | 6.5 MB 00:00 2024-09-14T06:18:36.119 INFO:teuthology.orchestra.run.smithi138.stdout:Extra Packages for Enterprise Linux 28 MB/s | 23 MB 00:00 2024-09-14T06:18:37.102 INFO:teuthology.orchestra.run.smithi094.stdout:CentOS Stream 9 - Extras packages 40 kB/s | 19 kB 00:00 2024-09-14T06:18:37.890 INFO:teuthology.orchestra.run.smithi094.stdout:Extra Packages for Enterprise Linux 34 MB/s | 23 MB 00:00 2024-09-14T06:18:41.625 INFO:teuthology.orchestra.run.smithi138.stdout:lab-extras 34 kB/s | 1.7 kB 00:00 2024-09-14T06:18:43.295 INFO:teuthology.orchestra.run.smithi094.stdout:lab-extras 29 kB/s | 1.7 kB 00:00 2024-09-14T06:18:43.623 INFO:teuthology.orchestra.run.smithi138.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-09-14T06:18:43.623 INFO:teuthology.orchestra.run.smithi138.stdout:Package kernel-5.14.0-503.el9.x86_64 is already installed. 2024-09-14T06:18:43.712 INFO:teuthology.orchestra.run.smithi138.stdout:Dependencies resolved. 2024-09-14T06:18:43.719 INFO:teuthology.orchestra.run.smithi138.stdout:================================================================================ 2024-09-14T06:18:43.719 INFO:teuthology.orchestra.run.smithi138.stdout: Package Architecture Version Repository Size 2024-09-14T06:18:43.720 INFO:teuthology.orchestra.run.smithi138.stdout:================================================================================ 2024-09-14T06:18:43.720 INFO:teuthology.orchestra.run.smithi138.stdout:Installing: 2024-09-14T06:18:43.720 INFO:teuthology.orchestra.run.smithi138.stdout: kernel x86_64 5.14.0-505.el9 baseos 23 k 2024-09-14T06:18:43.720 INFO:teuthology.orchestra.run.smithi138.stdout:Installing dependencies: 2024-09-14T06:18:43.720 INFO:teuthology.orchestra.run.smithi138.stdout: kernel-core x86_64 5.14.0-505.el9 baseos 16 M 2024-09-14T06:18:43.720 INFO:teuthology.orchestra.run.smithi138.stdout: kernel-modules x86_64 5.14.0-505.el9 baseos 35 M 2024-09-14T06:18:43.720 INFO:teuthology.orchestra.run.smithi138.stdout: kernel-modules-core x86_64 5.14.0-505.el9 baseos 29 M 2024-09-14T06:18:43.720 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-09-14T06:18:43.720 INFO:teuthology.orchestra.run.smithi138.stdout:Transaction Summary 2024-09-14T06:18:43.720 INFO:teuthology.orchestra.run.smithi138.stdout:================================================================================ 2024-09-14T06:18:43.720 INFO:teuthology.orchestra.run.smithi138.stdout:Install 4 Packages 2024-09-14T06:18:43.720 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-09-14T06:18:43.720 INFO:teuthology.orchestra.run.smithi138.stdout:Total download size: 79 M 2024-09-14T06:18:43.720 INFO:teuthology.orchestra.run.smithi138.stdout:Installed size: 126 M 2024-09-14T06:18:43.721 INFO:teuthology.orchestra.run.smithi138.stdout:Downloading Packages: 2024-09-14T06:18:43.935 INFO:teuthology.orchestra.run.smithi138.stdout:(1/4): kernel-5.14.0-505.el9.x86_64.rpm 260 kB/s | 23 kB 00:00 2024-09-14T06:18:44.302 INFO:teuthology.orchestra.run.smithi138.stdout:(2/4): kernel-core-5.14.0-505.el9.x86_64.rpm 34 MB/s | 16 MB 00:00 2024-09-14T06:18:44.727 INFO:teuthology.orchestra.run.smithi138.stdout:(3/4): kernel-modules-5.14.0-505.el9.x86_64.rpm 39 MB/s | 35 MB 00:00 2024-09-14T06:18:45.194 INFO:teuthology.orchestra.run.smithi138.stdout:(4/4): kernel-modules-core-5.14.0-505.el9.x86_6 23 MB/s | 29 MB 00:01 2024-09-14T06:18:45.194 INFO:teuthology.orchestra.run.smithi138.stdout:-------------------------------------------------------------------------------- 2024-09-14T06:18:45.194 INFO:teuthology.orchestra.run.smithi138.stdout:Total 53 MB/s | 79 MB 00:01 2024-09-14T06:18:45.318 INFO:teuthology.orchestra.run.smithi094.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-09-14T06:18:45.319 INFO:teuthology.orchestra.run.smithi094.stdout:Package kernel-5.14.0-503.el9.x86_64 is already installed. 2024-09-14T06:18:45.406 INFO:teuthology.orchestra.run.smithi094.stdout:Dependencies resolved. 2024-09-14T06:18:45.412 INFO:teuthology.orchestra.run.smithi094.stdout:================================================================================ 2024-09-14T06:18:45.412 INFO:teuthology.orchestra.run.smithi094.stdout: Package Architecture Version Repository Size 2024-09-14T06:18:45.413 INFO:teuthology.orchestra.run.smithi094.stdout:================================================================================ 2024-09-14T06:18:45.413 INFO:teuthology.orchestra.run.smithi094.stdout:Installing: 2024-09-14T06:18:45.413 INFO:teuthology.orchestra.run.smithi094.stdout: kernel x86_64 5.14.0-505.el9 baseos 23 k 2024-09-14T06:18:45.413 INFO:teuthology.orchestra.run.smithi094.stdout:Installing dependencies: 2024-09-14T06:18:45.413 INFO:teuthology.orchestra.run.smithi094.stdout: kernel-core x86_64 5.14.0-505.el9 baseos 16 M 2024-09-14T06:18:45.413 INFO:teuthology.orchestra.run.smithi094.stdout: kernel-modules x86_64 5.14.0-505.el9 baseos 35 M 2024-09-14T06:18:45.413 INFO:teuthology.orchestra.run.smithi094.stdout: kernel-modules-core x86_64 5.14.0-505.el9 baseos 29 M 2024-09-14T06:18:45.413 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-14T06:18:45.413 INFO:teuthology.orchestra.run.smithi094.stdout:Transaction Summary 2024-09-14T06:18:45.413 INFO:teuthology.orchestra.run.smithi094.stdout:================================================================================ 2024-09-14T06:18:45.413 INFO:teuthology.orchestra.run.smithi094.stdout:Install 4 Packages 2024-09-14T06:18:45.413 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-14T06:18:45.414 INFO:teuthology.orchestra.run.smithi094.stdout:Total download size: 79 M 2024-09-14T06:18:45.414 INFO:teuthology.orchestra.run.smithi094.stdout:Installed size: 126 M 2024-09-14T06:18:45.414 INFO:teuthology.orchestra.run.smithi094.stdout:Downloading Packages: 2024-09-14T06:18:45.872 INFO:teuthology.orchestra.run.smithi138.stdout:Running transaction check 2024-09-14T06:18:45.997 INFO:teuthology.orchestra.run.smithi094.stdout:(1/4): kernel-5.14.0-505.el9.x86_64.rpm 43 kB/s | 23 kB 00:00 2024-09-14T06:18:46.247 INFO:teuthology.orchestra.run.smithi138.stdout:Transaction check succeeded. 2024-09-14T06:18:46.247 INFO:teuthology.orchestra.run.smithi138.stdout:Running transaction test 2024-09-14T06:18:46.980 INFO:teuthology.orchestra.run.smithi094.stdout:(2/4): kernel-core-5.14.0-505.el9.x86_64.rpm 10 MB/s | 16 MB 00:01 2024-09-14T06:18:47.194 INFO:teuthology.orchestra.run.smithi138.stdout:Transaction test succeeded. 2024-09-14T06:18:47.194 INFO:teuthology.orchestra.run.smithi138.stdout:Running transaction 2024-09-14T06:18:47.580 INFO:teuthology.orchestra.run.smithi094.stdout:(3/4): kernel-modules-5.14.0-505.el9.x86_64.rpm 16 MB/s | 35 MB 00:02 2024-09-14T06:18:48.297 INFO:teuthology.orchestra.run.smithi094.stdout:(4/4): kernel-modules-core-5.14.0-505.el9.x86_6 12 MB/s | 29 MB 00:02 2024-09-14T06:18:48.298 INFO:teuthology.orchestra.run.smithi094.stdout:-------------------------------------------------------------------------------- 2024-09-14T06:18:48.298 INFO:teuthology.orchestra.run.smithi094.stdout:Total 27 MB/s | 79 MB 00:02 2024-09-14T06:18:48.986 INFO:teuthology.orchestra.run.smithi094.stdout:Running transaction check 2024-09-14T06:18:49.344 INFO:teuthology.orchestra.run.smithi094.stdout:Transaction check succeeded. 2024-09-14T06:18:49.344 INFO:teuthology.orchestra.run.smithi094.stdout:Running transaction test 2024-09-14T06:18:49.620 INFO:teuthology.orchestra.run.smithi138.stdout: Preparing : 1/1 2024-09-14T06:18:50.316 INFO:teuthology.orchestra.run.smithi094.stdout:Transaction test succeeded. 2024-09-14T06:18:50.316 INFO:teuthology.orchestra.run.smithi094.stdout:Running transaction 2024-09-14T06:18:50.337 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : kernel-modules-core-5.14.0-505.el9.x86_64 1/4 2024-09-14T06:18:50.413 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : kernel-core-5.14.0-505.el9.x86_64 2/4 2024-09-14T06:18:52.757 INFO:teuthology.orchestra.run.smithi094.stdout: Preparing : 1/1 2024-09-14T06:18:53.211 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: kernel-core-5.14.0-505.el9.x86_64 2/4 2024-09-14T06:18:53.364 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : kernel-modules-5.14.0-505.el9.x86_64 3/4 2024-09-14T06:18:53.442 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : kernel-modules-core-5.14.0-505.el9.x86_64 1/4 2024-09-14T06:18:53.538 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : kernel-core-5.14.0-505.el9.x86_64 2/4 2024-09-14T06:18:56.326 INFO:teuthology.orchestra.run.smithi094.stdout: Running scriptlet: kernel-core-5.14.0-505.el9.x86_64 2/4 2024-09-14T06:18:56.464 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : kernel-modules-5.14.0-505.el9.x86_64 3/4 2024-09-14T06:18:58.175 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: kernel-modules-5.14.0-505.el9.x86_64 3/4 2024-09-14T06:18:58.331 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : kernel-5.14.0-505.el9.x86_64 4/4 2024-09-14T06:19:01.317 INFO:teuthology.orchestra.run.smithi094.stdout: Running scriptlet: kernel-modules-5.14.0-505.el9.x86_64 3/4 2024-09-14T06:19:01.438 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : kernel-5.14.0-505.el9.x86_64 4/4 2024-09-14T06:19:03.154 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: kernel-modules-core-5.14.0-505.el9.x86_64 4/4 2024-09-14T06:19:06.265 INFO:teuthology.orchestra.run.smithi094.stdout: Running scriptlet: kernel-modules-core-5.14.0-505.el9.x86_64 4/4 2024-09-14T06:19:40.980 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: kernel-core-5.14.0-505.el9.x86_64 4/4 2024-09-14T06:19:41.019 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: kernel-modules-5.14.0-505.el9.x86_64 4/4 2024-09-14T06:19:41.403 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: kernel-5.14.0-505.el9.x86_64 4/4 2024-09-14T06:19:41.403 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : kernel-5.14.0-505.el9.x86_64 1/4 2024-09-14T06:19:41.403 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : kernel-core-5.14.0-505.el9.x86_64 2/4 2024-09-14T06:19:41.404 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : kernel-modules-5.14.0-505.el9.x86_64 3/4 2024-09-14T06:19:42.206 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : kernel-modules-core-5.14.0-505.el9.x86_64 4/4 2024-09-14T06:19:42.207 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-09-14T06:19:42.207 INFO:teuthology.orchestra.run.smithi138.stdout:Installed: 2024-09-14T06:19:42.207 INFO:teuthology.orchestra.run.smithi138.stdout: kernel-5.14.0-505.el9.x86_64 2024-09-14T06:19:42.207 INFO:teuthology.orchestra.run.smithi138.stdout: kernel-core-5.14.0-505.el9.x86_64 2024-09-14T06:19:42.207 INFO:teuthology.orchestra.run.smithi138.stdout: kernel-modules-5.14.0-505.el9.x86_64 2024-09-14T06:19:42.207 INFO:teuthology.orchestra.run.smithi138.stdout: kernel-modules-core-5.14.0-505.el9.x86_64 2024-09-14T06:19:42.207 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-09-14T06:19:42.207 INFO:teuthology.orchestra.run.smithi138.stdout:Complete! 2024-09-14T06:19:42.516 DEBUG:teuthology.orchestra.run.smithi138:> rpm -q kernel | sort -rV | head -n 1 2024-09-14T06:19:42.638 INFO:teuthology.orchestra.run.smithi138.stdout:kernel-5.14.0-505.el9.x86_64 2024-09-14T06:19:42.639 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-505.el9.x86_64 2024-09-14T06:19:42.639 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-503.el9.x86_64 Expected: 5.14.0-505.el9.x86_64 2024-09-14T06:19:42.639 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-09-14T06:19:42.639 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-09-14T06:19:42.639 INFO:teuthology.task.kernel:Installing distro kernel on mon.b... 2024-09-14T06:19:42.640 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi138.front.sepia.ceph.com, path=None, version=distro) 2024-09-14T06:19:42.640 DEBUG:teuthology.orchestra.run.smithi138:> rpm -q kernel | sort -rV | head -n 1 2024-09-14T06:19:42.698 INFO:teuthology.orchestra.run.smithi138.stdout:kernel-5.14.0-505.el9.x86_64 2024-09-14T06:19:42.698 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-505.el9.x86_64 2024-09-14T06:19:42.698 DEBUG:teuthology.orchestra.run.smithi138:> sudo rpm -qi grub2-tools 2024-09-14T06:19:42.739 INFO:teuthology.orchestra.run.smithi138.stdout:Name : grub2-tools 2024-09-14T06:19:42.739 INFO:teuthology.orchestra.run.smithi138.stdout:Epoch : 1 2024-09-14T06:19:42.740 INFO:teuthology.orchestra.run.smithi138.stdout:Version : 2.06 2024-09-14T06:19:42.740 INFO:teuthology.orchestra.run.smithi138.stdout:Release : 82.el9 2024-09-14T06:19:42.740 INFO:teuthology.orchestra.run.smithi138.stdout:Architecture: x86_64 2024-09-14T06:19:42.740 INFO:teuthology.orchestra.run.smithi138.stdout:Install Date: Wed 28 Aug 2024 09:06:27 PM UTC 2024-09-14T06:19:42.740 INFO:teuthology.orchestra.run.smithi138.stdout:Group : Unspecified 2024-09-14T06:19:42.740 INFO:teuthology.orchestra.run.smithi138.stdout:Size : 8274670 2024-09-14T06:19:42.740 INFO:teuthology.orchestra.run.smithi138.stdout:License : GPLv3+ 2024-09-14T06:19:42.740 INFO:teuthology.orchestra.run.smithi138.stdout:Signature : RSA/SHA256, Mon 01 Jul 2024 07:46:42 AM UTC, Key ID 05b555b38483c65d 2024-09-14T06:19:42.740 INFO:teuthology.orchestra.run.smithi138.stdout:Source RPM : grub2-2.06-82.el9.src.rpm 2024-09-14T06:19:42.740 INFO:teuthology.orchestra.run.smithi138.stdout:Build Date : Fri 28 Jun 2024 09:24:44 AM UTC 2024-09-14T06:19:42.740 INFO:teuthology.orchestra.run.smithi138.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-09-14T06:19:42.741 INFO:teuthology.orchestra.run.smithi138.stdout:Packager : builder@centos.org 2024-09-14T06:19:42.741 INFO:teuthology.orchestra.run.smithi138.stdout:Vendor : CentOS 2024-09-14T06:19:42.741 INFO:teuthology.orchestra.run.smithi138.stdout:URL : http://www.gnu.org/software/grub/ 2024-09-14T06:19:42.741 INFO:teuthology.orchestra.run.smithi138.stdout:Summary : Support tools for GRUB. 2024-09-14T06:19:42.741 INFO:teuthology.orchestra.run.smithi138.stdout:Description : 2024-09-14T06:19:42.741 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-09-14T06:19:42.741 INFO:teuthology.orchestra.run.smithi138.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-09-14T06:19:42.741 INFO:teuthology.orchestra.run.smithi138.stdout:customizable bootloader with modular architecture. It supports a rich 2024-09-14T06:19:42.741 INFO:teuthology.orchestra.run.smithi138.stdout:variety of kernel formats, file systems, computer architectures and 2024-09-14T06:19:42.741 INFO:teuthology.orchestra.run.smithi138.stdout:hardware devices. 2024-09-14T06:19:42.741 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-09-14T06:19:42.741 INFO:teuthology.orchestra.run.smithi138.stdout:This subpackage provides tools for support of all platforms. 2024-09-14T06:19:42.742 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-09-14T06:19:42.743 INFO:teuthology.task.kernel:Updating grub on smithi138 to boot 5.14.0-505.el9.x86_64 2024-09-14T06:19:42.743 DEBUG:teuthology.orchestra.run.smithi138:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-09-14T06:19:43.448 INFO:teuthology.orchestra.run.smithi138.stderr:Generating grub configuration file ... 2024-09-14T06:19:44.839 INFO:teuthology.orchestra.run.smithi138.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-09-14T06:19:44.846 INFO:teuthology.orchestra.run.smithi094.stdout: Running scriptlet: kernel-core-5.14.0-505.el9.x86_64 4/4 2024-09-14T06:19:44.871 INFO:teuthology.orchestra.run.smithi138.stderr:done 2024-09-14T06:19:44.874 DEBUG:teuthology.orchestra.run.smithi138:> mktemp 2024-09-14T06:19:44.889 INFO:teuthology.orchestra.run.smithi138.stdout:/tmp/tmp.9v50DFKw8T 2024-09-14T06:19:44.889 DEBUG:teuthology.orchestra.run.smithi138:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.9v50DFKw8T 2024-09-14T06:19:44.890 INFO:teuthology.orchestra.run.smithi094.stdout: Running scriptlet: kernel-modules-5.14.0-505.el9.x86_64 4/4 2024-09-14T06:19:44.955 DEBUG:teuthology.orchestra.run.smithi138:> sudo chmod 0666 /tmp/tmp.9v50DFKw8T 2024-09-14T06:19:45.096 DEBUG:teuthology.orchestra.remote:smithi138:/tmp/tmp.9v50DFKw8T is 6KB 2024-09-14T06:19:45.116 DEBUG:teuthology.orchestra.run.smithi138:> rm -fr /tmp/tmp.9v50DFKw8T 2024-09-14T06:19:45.131 DEBUG:teuthology.orchestra.run.smithi138:> sudo /bin/ls /boot/loader/entries || true 2024-09-14T06:19:45.194 INFO:teuthology.orchestra.run.smithi138.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-0-rescue.conf 2024-09-14T06:19:45.194 INFO:teuthology.orchestra.run.smithi138.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-239.el9.x86_64.conf 2024-09-14T06:19:45.194 INFO:teuthology.orchestra.run.smithi138.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-503.el9.x86_64.conf 2024-09-14T06:19:45.194 INFO:teuthology.orchestra.run.smithi138.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-505.el9.x86_64.conf 2024-09-14T06:19:45.196 DEBUG:teuthology.orchestra.run.smithi138:> sudo grub2-set-default fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-505.el9.x86_64 2024-09-14T06:19:45.329 INFO:teuthology.orchestra.run.smithi094.stdout: Running scriptlet: kernel-5.14.0-505.el9.x86_64 4/4 2024-09-14T06:19:45.329 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : kernel-5.14.0-505.el9.x86_64 1/4 2024-09-14T06:19:45.329 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : kernel-core-5.14.0-505.el9.x86_64 2/4 2024-09-14T06:19:45.329 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : kernel-modules-5.14.0-505.el9.x86_64 3/4 2024-09-14T06:19:45.365 DEBUG:teuthology.orchestra.run.smithi138:> sudo shutdown -r now 2024-09-14T06:19:46.220 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : kernel-modules-core-5.14.0-505.el9.x86_64 4/4 2024-09-14T06:19:46.220 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-14T06:19:46.220 INFO:teuthology.orchestra.run.smithi094.stdout:Installed: 2024-09-14T06:19:46.220 INFO:teuthology.orchestra.run.smithi094.stdout: kernel-5.14.0-505.el9.x86_64 2024-09-14T06:19:46.220 INFO:teuthology.orchestra.run.smithi094.stdout: kernel-core-5.14.0-505.el9.x86_64 2024-09-14T06:19:46.220 INFO:teuthology.orchestra.run.smithi094.stdout: kernel-modules-5.14.0-505.el9.x86_64 2024-09-14T06:19:46.220 INFO:teuthology.orchestra.run.smithi094.stdout: kernel-modules-core-5.14.0-505.el9.x86_64 2024-09-14T06:19:46.220 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-14T06:19:46.221 INFO:teuthology.orchestra.run.smithi094.stdout:Complete! 2024-09-14T06:19:46.638 DEBUG:teuthology.orchestra.run.smithi094:> rpm -q kernel | sort -rV | head -n 1 2024-09-14T06:19:46.713 INFO:teuthology.orchestra.run.smithi094.stdout:kernel-5.14.0-505.el9.x86_64 2024-09-14T06:19:46.714 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-505.el9.x86_64 2024-09-14T06:19:46.714 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-503.el9.x86_64 Expected: 5.14.0-505.el9.x86_64 2024-09-14T06:19:46.714 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-09-14T06:19:46.714 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-09-14T06:19:46.714 INFO:teuthology.task.kernel:Installing distro kernel on mon.a... 2024-09-14T06:19:46.714 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi094.front.sepia.ceph.com, path=None, version=distro) 2024-09-14T06:19:46.714 DEBUG:teuthology.orchestra.run.smithi094:> rpm -q kernel | sort -rV | head -n 1 2024-09-14T06:19:46.773 INFO:teuthology.orchestra.run.smithi094.stdout:kernel-5.14.0-505.el9.x86_64 2024-09-14T06:19:46.776 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-505.el9.x86_64 2024-09-14T06:19:46.777 DEBUG:teuthology.orchestra.run.smithi094:> sudo rpm -qi grub2-tools 2024-09-14T06:19:46.808 INFO:teuthology.orchestra.run.smithi094.stdout:Name : grub2-tools 2024-09-14T06:19:46.808 INFO:teuthology.orchestra.run.smithi094.stdout:Epoch : 1 2024-09-14T06:19:46.808 INFO:teuthology.orchestra.run.smithi094.stdout:Version : 2.06 2024-09-14T06:19:46.808 INFO:teuthology.orchestra.run.smithi094.stdout:Release : 82.el9 2024-09-14T06:19:46.808 INFO:teuthology.orchestra.run.smithi094.stdout:Architecture: x86_64 2024-09-14T06:19:46.808 INFO:teuthology.orchestra.run.smithi094.stdout:Install Date: Wed 28 Aug 2024 09:06:27 PM UTC 2024-09-14T06:19:46.808 INFO:teuthology.orchestra.run.smithi094.stdout:Group : Unspecified 2024-09-14T06:19:46.808 INFO:teuthology.orchestra.run.smithi094.stdout:Size : 8274670 2024-09-14T06:19:46.808 INFO:teuthology.orchestra.run.smithi094.stdout:License : GPLv3+ 2024-09-14T06:19:46.809 INFO:teuthology.orchestra.run.smithi094.stdout:Signature : RSA/SHA256, Mon 01 Jul 2024 07:46:42 AM UTC, Key ID 05b555b38483c65d 2024-09-14T06:19:46.809 INFO:teuthology.orchestra.run.smithi094.stdout:Source RPM : grub2-2.06-82.el9.src.rpm 2024-09-14T06:19:46.809 INFO:teuthology.orchestra.run.smithi094.stdout:Build Date : Fri 28 Jun 2024 09:24:44 AM UTC 2024-09-14T06:19:46.809 INFO:teuthology.orchestra.run.smithi094.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-09-14T06:19:46.809 INFO:teuthology.orchestra.run.smithi094.stdout:Packager : builder@centos.org 2024-09-14T06:19:46.809 INFO:teuthology.orchestra.run.smithi094.stdout:Vendor : CentOS 2024-09-14T06:19:46.809 INFO:teuthology.orchestra.run.smithi094.stdout:URL : http://www.gnu.org/software/grub/ 2024-09-14T06:19:46.809 INFO:teuthology.orchestra.run.smithi094.stdout:Summary : Support tools for GRUB. 2024-09-14T06:19:46.809 INFO:teuthology.orchestra.run.smithi094.stdout:Description : 2024-09-14T06:19:46.809 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-14T06:19:46.809 INFO:teuthology.orchestra.run.smithi094.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-09-14T06:19:46.809 INFO:teuthology.orchestra.run.smithi094.stdout:customizable bootloader with modular architecture. It supports a rich 2024-09-14T06:19:46.809 INFO:teuthology.orchestra.run.smithi094.stdout:variety of kernel formats, file systems, computer architectures and 2024-09-14T06:19:46.810 INFO:teuthology.orchestra.run.smithi094.stdout:hardware devices. 2024-09-14T06:19:46.810 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-14T06:19:46.810 INFO:teuthology.orchestra.run.smithi094.stdout:This subpackage provides tools for support of all platforms. 2024-09-14T06:19:46.810 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-09-14T06:19:46.810 INFO:teuthology.task.kernel:Updating grub on smithi094 to boot 5.14.0-505.el9.x86_64 2024-09-14T06:19:46.811 DEBUG:teuthology.orchestra.run.smithi094:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-09-14T06:19:47.434 INFO:teuthology.orchestra.run.smithi094.stderr:Generating grub configuration file ... 2024-09-14T06:19:48.776 INFO:teuthology.orchestra.run.smithi094.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-09-14T06:19:48.805 INFO:teuthology.orchestra.run.smithi094.stderr:done 2024-09-14T06:19:48.807 DEBUG:teuthology.orchestra.run.smithi094:> mktemp 2024-09-14T06:19:48.823 INFO:teuthology.orchestra.run.smithi094.stdout:/tmp/tmp.QpjdZ9K7bR 2024-09-14T06:19:48.823 DEBUG:teuthology.orchestra.run.smithi094:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.QpjdZ9K7bR 2024-09-14T06:19:48.890 DEBUG:teuthology.orchestra.run.smithi094:> sudo chmod 0666 /tmp/tmp.QpjdZ9K7bR 2024-09-14T06:19:49.028 DEBUG:teuthology.orchestra.remote:smithi094:/tmp/tmp.QpjdZ9K7bR is 6KB 2024-09-14T06:19:49.076 DEBUG:teuthology.orchestra.run.smithi094:> rm -fr /tmp/tmp.QpjdZ9K7bR 2024-09-14T06:19:49.090 DEBUG:teuthology.orchestra.run.smithi094:> sudo /bin/ls /boot/loader/entries || true 2024-09-14T06:19:49.154 INFO:teuthology.orchestra.run.smithi094.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-0-rescue.conf 2024-09-14T06:19:49.154 INFO:teuthology.orchestra.run.smithi094.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-239.el9.x86_64.conf 2024-09-14T06:19:49.154 INFO:teuthology.orchestra.run.smithi094.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-503.el9.x86_64.conf 2024-09-14T06:19:49.154 INFO:teuthology.orchestra.run.smithi094.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-505.el9.x86_64.conf 2024-09-14T06:19:49.155 DEBUG:teuthology.orchestra.run.smithi094:> sudo grub2-set-default fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-505.el9.x86_64 2024-09-14T06:19:49.297 DEBUG:teuthology.orchestra.run.smithi094:> sudo shutdown -r now 2024-09-14T06:20:15.393 INFO:teuthology.task.kernel:Checking client mon.b for new kernel version... 2024-09-14T06:20:15.394 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi138.front.sepia.ceph.com' 2024-09-14T06:20:15.395 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi138.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-14T06:20:19.304 INFO:teuthology.task.kernel:Checking client mon.a for new kernel version... 2024-09-14T06:20:19.305 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi094.front.sepia.ceph.com' 2024-09-14T06:20:19.305 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi094.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-14T06:20:33.729 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.138 2024-09-14T06:20:37.821 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.94 2024-09-14T06:20:42.734 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi138.front.sepia.ceph.com' 2024-09-14T06:20:42.735 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi138.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-14T06:20:45.789 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.138 2024-09-14T06:20:46.823 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi094.front.sepia.ceph.com' 2024-09-14T06:20:46.824 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi094.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-14T06:20:57.801 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi138.front.sepia.ceph.com' 2024-09-14T06:20:57.802 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi138.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-14T06:21:05.217 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.94 2024-09-14T06:21:17.222 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi094.front.sepia.ceph.com' 2024-09-14T06:21:17.223 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi094.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-14T06:21:20.285 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.94 2024-09-14T06:21:29.799 DEBUG:teuthology.orchestra.run.smithi138:> true 2024-09-14T06:21:30.127 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi138.front.sepia.ceph.com' 2024-09-14T06:21:30.128 INFO:teuthology.task.kernel:Checking kernel version of mon.b, want "5.14.0-505.el9.x86_64"... 2024-09-14T06:21:30.128 DEBUG:teuthology.orchestra.run.smithi138:> uname -r 2024-09-14T06:21:30.143 INFO:teuthology.orchestra.run.smithi138.stdout:5.14.0-505.el9.x86_64 2024-09-14T06:21:30.143 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-505.el9.x86_64 vs 5.14.0-505.el9.x86_64 2024-09-14T06:21:30.143 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-09-14T06:21:30.143 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-09-14T06:21:31.144 INFO:teuthology.task.kernel:Enabling kdb on mon.b... 2024-09-14T06:21:31.144 DEBUG:teuthology.orchestra.run.smithi138:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-09-14T06:21:31.273 INFO:teuthology.orchestra.run.smithi138.stdout:ttyS1 2024-09-14T06:21:31.295 DEBUG:teuthology.parallel:result is None 2024-09-14T06:21:35.286 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi094.front.sepia.ceph.com' 2024-09-14T06:21:35.287 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi094.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-14T06:21:35.767 DEBUG:teuthology.orchestra.run.smithi094:> true 2024-09-14T06:21:36.119 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi094.front.sepia.ceph.com' 2024-09-14T06:21:36.119 INFO:teuthology.task.kernel:Checking kernel version of mon.a, want "5.14.0-505.el9.x86_64"... 2024-09-14T06:21:36.120 DEBUG:teuthology.orchestra.run.smithi094:> uname -r 2024-09-14T06:21:36.136 INFO:teuthology.orchestra.run.smithi094.stdout:5.14.0-505.el9.x86_64 2024-09-14T06:21:36.136 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-505.el9.x86_64 vs 5.14.0-505.el9.x86_64 2024-09-14T06:21:36.136 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-09-14T06:21:36.136 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-09-14T06:21:37.137 INFO:teuthology.task.kernel:Enabling kdb on mon.a... 2024-09-14T06:21:37.137 DEBUG:teuthology.orchestra.run.smithi094:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-09-14T06:21:37.219 INFO:teuthology.orchestra.run.smithi094.stdout:ttyS1 2024-09-14T06:21:37.237 DEBUG:teuthology.parallel:result is None 2024-09-14T06:21:37.238 INFO:teuthology.run_tasks:Running task internal.base... 2024-09-14T06:21:37.245 INFO:teuthology.task.internal:Creating test directory... 2024-09-14T06:21:37.245 DEBUG:teuthology.orchestra.run.smithi094:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-09-14T06:21:37.247 DEBUG:teuthology.orchestra.run.smithi138:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-09-14T06:21:37.267 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-09-14T06:21:37.273 INFO:teuthology.run_tasks:Running task internal.archive... 2024-09-14T06:21:37.281 INFO:teuthology.task.internal:Creating archive directory... 2024-09-14T06:21:37.281 DEBUG:teuthology.orchestra.run.smithi094:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-09-14T06:21:37.306 DEBUG:teuthology.orchestra.run.smithi138:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-09-14T06:21:37.357 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-09-14T06:21:37.363 INFO:teuthology.task.internal:Enabling coredump saving... 2024-09-14T06:21:37.364 DEBUG:teuthology.orchestra.run.smithi094:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-09-14T06:21:37.366 DEBUG:teuthology.orchestra.run.smithi138:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-09-14T06:21:37.401 INFO:teuthology.orchestra.run.smithi094.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-14T06:21:37.414 INFO:teuthology.orchestra.run.smithi094.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-14T06:21:37.422 INFO:teuthology.orchestra.run.smithi138.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-14T06:21:37.434 INFO:teuthology.orchestra.run.smithi138.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-14T06:21:37.435 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-09-14T06:21:37.442 INFO:teuthology.task.internal:Configuring sudo... 2024-09-14T06:21:37.443 DEBUG:teuthology.orchestra.run.smithi094:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-09-14T06:21:37.458 DEBUG:teuthology.orchestra.run.smithi138:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-09-14T06:21:37.503 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-09-14T06:21:37.512 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-09-14T06:21:37.512 DEBUG:teuthology.orchestra.run.smithi094:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-09-14T06:21:37.529 DEBUG:teuthology.orchestra.run.smithi138:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-09-14T06:21:37.560 DEBUG:teuthology.orchestra.run.smithi094:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-14T06:21:37.630 DEBUG:teuthology.orchestra.run.smithi094:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-14T06:21:37.677 DEBUG:teuthology.orchestra.run.smithi094:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-14T06:21:37.763 DEBUG:teuthology.orchestra.run.smithi094:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-14T06:21:37.803 DEBUG:teuthology.orchestra.run.smithi094:> set -ex 2024-09-14T06:21:37.803 DEBUG:teuthology.orchestra.run.smithi094:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-09-14T06:21:37.868 DEBUG:teuthology.orchestra.run.smithi138:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-14T06:21:37.912 DEBUG:teuthology.orchestra.run.smithi138:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-14T06:21:37.968 DEBUG:teuthology.orchestra.run.smithi138:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-14T06:21:38.052 DEBUG:teuthology.orchestra.run.smithi138:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-14T06:21:38.094 DEBUG:teuthology.orchestra.run.smithi138:> set -ex 2024-09-14T06:21:38.094 DEBUG:teuthology.orchestra.run.smithi138:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-09-14T06:21:38.161 DEBUG:teuthology.orchestra.run.smithi094:> sudo service rsyslog restart 2024-09-14T06:21:38.165 DEBUG:teuthology.orchestra.run.smithi138:> sudo service rsyslog restart 2024-09-14T06:21:38.202 INFO:teuthology.orchestra.run.smithi094.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-14T06:21:38.250 INFO:teuthology.orchestra.run.smithi138.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-14T06:21:38.762 INFO:teuthology.run_tasks:Running task internal.timer... 2024-09-14T06:21:38.770 INFO:teuthology.task.internal:Starting timer... 2024-09-14T06:21:38.770 INFO:teuthology.run_tasks:Running task pcp... 2024-09-14T06:21:38.801 INFO:teuthology.run_tasks:Running task selinux... 2024-09-14T06:21:38.809 DEBUG:teuthology.orchestra.run.smithi094:> sudo service auditd rotate 2024-09-14T06:21:38.856 INFO:teuthology.orchestra.run.smithi094.stdout:Rotating logs: 2024-09-14T06:21:38.858 DEBUG:teuthology.orchestra.run.smithi138:> sudo service auditd rotate 2024-09-14T06:21:38.923 INFO:teuthology.orchestra.run.smithi138.stdout:Rotating logs: 2024-09-14T06:21:38.925 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-09-14T06:21:38.925 DEBUG:teuthology.orchestra.run.smithi094:> /usr/sbin/getenforce 2024-09-14T06:21:38.948 INFO:teuthology.orchestra.run.smithi094.stdout:Permissive 2024-09-14T06:21:38.948 DEBUG:teuthology.orchestra.run.smithi138:> /usr/sbin/getenforce 2024-09-14T06:21:38.968 INFO:teuthology.orchestra.run.smithi138.stdout:Permissive 2024-09-14T06:21:38.968 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi094.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi138.front.sepia.ceph.com': 'permissive'} 2024-09-14T06:21:38.968 DEBUG:teuthology.orchestra.run.smithi094:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 2024-09-14T06:21:39.016 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-14T06:21:39.017 DEBUG:teuthology.orchestra.run.smithi138:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 2024-09-14T06:21:39.044 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-14T06:21:39.045 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-09-14T06:21:39.045 DEBUG:teuthology.orchestra.run.smithi094:> sudo /usr/sbin/setenforce permissive 2024-09-14T06:21:39.095 DEBUG:teuthology.orchestra.run.smithi138:> sudo /usr/sbin/setenforce permissive 2024-09-14T06:21:39.134 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-09-14T06:21:39.143 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-09-14T06:21:39.155 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-09-14T06:21:39.156 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi094.front.sepia.ceph.com,smithi138.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-09-14T06:25:21.873 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi094.front.sepia.ceph.com'), Remote(name='ubuntu@smithi138.front.sepia.ceph.com')] 2024-09-14T06:25:21.874 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi094.front.sepia.ceph.com' 2024-09-14T06:25:21.876 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi094.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-14T06:25:21.954 DEBUG:teuthology.orchestra.run.smithi094:> true 2024-09-14T06:25:22.031 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi094.front.sepia.ceph.com' 2024-09-14T06:25:22.032 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi138.front.sepia.ceph.com' 2024-09-14T06:25:22.032 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi138.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-14T06:25:22.105 DEBUG:teuthology.orchestra.run.smithi138:> true 2024-09-14T06:25:22.188 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi138.front.sepia.ceph.com' 2024-09-14T06:25:22.189 INFO:teuthology.run_tasks:Running task clock... 2024-09-14T06:25:22.200 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-09-14T06:25:22.201 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-09-14T06:25:22.201 DEBUG:teuthology.orchestra.run.smithi094:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-14T06:25:22.203 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-09-14T06:25:22.204 DEBUG:teuthology.orchestra.run.smithi138:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-14T06:25:22.230 INFO:teuthology.orchestra.run.smithi094.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-09-14T06:25:22.244 INFO:teuthology.orchestra.run.smithi094.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-09-14T06:25:22.259 INFO:teuthology.orchestra.run.smithi138.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-09-14T06:25:22.267 INFO:teuthology.orchestra.run.smithi094.stderr:sudo: ntpd: command not found 2024-09-14T06:25:22.273 INFO:teuthology.orchestra.run.smithi138.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-09-14T06:25:22.278 INFO:teuthology.orchestra.run.smithi094.stdout:506 Cannot talk to daemon 2024-09-14T06:25:22.292 INFO:teuthology.orchestra.run.smithi094.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-09-14T06:25:22.298 INFO:teuthology.orchestra.run.smithi138.stderr:sudo: ntpd: command not found 2024-09-14T06:25:22.306 INFO:teuthology.orchestra.run.smithi094.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-09-14T06:25:22.309 INFO:teuthology.orchestra.run.smithi138.stdout:506 Cannot talk to daemon 2024-09-14T06:25:22.324 INFO:teuthology.orchestra.run.smithi138.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-09-14T06:25:22.338 INFO:teuthology.orchestra.run.smithi138.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-09-14T06:25:22.365 INFO:teuthology.orchestra.run.smithi094.stderr:bash: line 1: ntpq: command not found 2024-09-14T06:25:22.369 INFO:teuthology.orchestra.run.smithi094.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-14T06:25:22.369 INFO:teuthology.orchestra.run.smithi094.stdout:=============================================================================== 2024-09-14T06:25:22.369 INFO:teuthology.orchestra.run.smithi094.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-14T06:25:22.369 INFO:teuthology.orchestra.run.smithi094.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-14T06:25:22.369 INFO:teuthology.orchestra.run.smithi094.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-14T06:25:22.369 INFO:teuthology.orchestra.run.smithi094.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-14T06:25:22.390 INFO:teuthology.orchestra.run.smithi138.stderr:bash: line 1: ntpq: command not found 2024-09-14T06:25:22.395 INFO:teuthology.orchestra.run.smithi138.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-14T06:25:22.395 INFO:teuthology.orchestra.run.smithi138.stdout:=============================================================================== 2024-09-14T06:25:22.395 INFO:teuthology.orchestra.run.smithi138.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-14T06:25:22.395 INFO:teuthology.orchestra.run.smithi138.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-14T06:25:22.395 INFO:teuthology.orchestra.run.smithi138.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-14T06:25:22.395 INFO:teuthology.orchestra.run.smithi138.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-14T06:25:22.395 INFO:teuthology.run_tasks:Running task install... 2024-09-14T06:25:22.407 DEBUG:teuthology.task.install:project ceph 2024-09-14T06:25:22.407 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': '7f24e589328ae9e639c0009d93340e5b235d0a47'}} 2024-09-14T06:25:22.407 DEBUG:teuthology.task.install:config {'flavor': 'default', 'sha1': '7f24e589328ae9e639c0009d93340e5b235d0a47'} 2024-09-14T06:25:22.407 INFO:teuthology.task.install:Using flavor: default 2024-09-14T06:25:22.413 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-09-14T06:25:22.413 INFO:teuthology.task.install:extra packages: [] 2024-09-14T06:25:22.414 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': [], 'extra_system_packages': [], 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '7f24e589328ae9e639c0009d93340e5b235d0a47', 'tag': None, 'wait_for_package': False} 2024-09-14T06:25:22.415 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=7f24e589328ae9e639c0009d93340e5b235d0a47 2024-09-14T06:25:22.417 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': [], 'extra_system_packages': [], 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '7f24e589328ae9e639c0009d93340e5b235d0a47', 'tag': None, 'wait_for_package': False} 2024-09-14T06:25:22.417 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=7f24e589328ae9e639c0009d93340e5b235d0a47 2024-09-14T06:25:22.562 INFO:teuthology.task.install.rpm:Pulling from https://1.chacra.ceph.com/r/ceph/main/7f24e589328ae9e639c0009d93340e5b235d0a47/centos/9/flavors/default/ 2024-09-14T06:25:22.562 INFO:teuthology.task.install.rpm:Package version is 19.3.0-4775.g7f24e589 2024-09-14T06:25:22.570 INFO:teuthology.task.install.rpm:Pulling from https://1.chacra.ceph.com/r/ceph/main/7f24e589328ae9e639c0009d93340e5b235d0a47/centos/9/flavors/default/ 2024-09-14T06:25:22.570 INFO:teuthology.task.install.rpm:Package version is 19.3.0-4775.g7f24e589 2024-09-14T06:25:22.672 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://1.chacra.ceph.com/r/ceph/main/7f24e589328ae9e639c0009d93340e5b235d0a47/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://1.chacra.ceph.com/r/ceph/main/7f24e589328ae9e639c0009d93340e5b235d0a47/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://1.chacra.ceph.com/r/ceph/main/7f24e589328ae9e639c0009d93340e5b235d0a47/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-09-14T06:25:22.672 DEBUG:teuthology.orchestra.run.smithi094:> set -ex 2024-09-14T06:25:22.672 DEBUG:teuthology.orchestra.run.smithi094:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-09-14T06:25:22.679 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://1.chacra.ceph.com/r/ceph/main/7f24e589328ae9e639c0009d93340e5b235d0a47/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://1.chacra.ceph.com/r/ceph/main/7f24e589328ae9e639c0009d93340e5b235d0a47/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://1.chacra.ceph.com/r/ceph/main/7f24e589328ae9e639c0009d93340e5b235d0a47/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-09-14T06:25:22.679 DEBUG:teuthology.orchestra.run.smithi138:> set -ex 2024-09-14T06:25:22.679 DEBUG:teuthology.orchestra.run.smithi138:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-09-14T06:25:22.699 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd on remote rpm x86_64 2024-09-14T06:25:22.699 DEBUG:teuthology.orchestra.run.smithi094:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/7f24e589328ae9e639c0009d93340e5b235d0a47/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-09-14T06:25:22.705 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd on remote rpm x86_64 2024-09-14T06:25:22.705 DEBUG:teuthology.orchestra.run.smithi138:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/7f24e589328ae9e639c0009d93340e5b235d0a47/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-09-14T06:25:22.765 DEBUG:teuthology.orchestra.run.smithi094:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-09-14T06:25:22.770 DEBUG:teuthology.orchestra.run.smithi138:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-09-14T06:25:22.840 DEBUG:teuthology.orchestra.run.smithi094:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-09-14T06:25:22.845 DEBUG:teuthology.orchestra.run.smithi138:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-09-14T06:25:22.904 INFO:teuthology.orchestra.run.smithi094.stdout:check_obsoletes = 1 2024-09-14T06:25:22.906 DEBUG:teuthology.orchestra.run.smithi094:> sudo yum clean all 2024-09-14T06:25:22.909 INFO:teuthology.orchestra.run.smithi138.stdout:check_obsoletes = 1 2024-09-14T06:25:22.911 DEBUG:teuthology.orchestra.run.smithi138:> sudo yum clean all 2024-09-14T06:25:23.128 INFO:teuthology.orchestra.run.smithi094.stdout:45 files removed 2024-09-14T06:25:23.134 INFO:teuthology.orchestra.run.smithi138.stdout:45 files removed 2024-09-14T06:25:23.155 DEBUG:teuthology.orchestra.run.smithi094:> sudo yum -y install ceph-radosgw 2024-09-14T06:25:23.161 DEBUG:teuthology.orchestra.run.smithi138:> sudo yum -y install ceph-radosgw 2024-09-14T06:25:23.704 INFO:teuthology.orchestra.run.smithi094.stdout:ceph packages for x86_64 317 kB/s | 84 kB 00:00 2024-09-14T06:25:23.715 INFO:teuthology.orchestra.run.smithi138.stdout:ceph packages for x86_64 308 kB/s | 84 kB 00:00 2024-09-14T06:25:23.996 INFO:teuthology.orchestra.run.smithi094.stdout:ceph noarch packages 97 kB/s | 24 kB 00:00 2024-09-14T06:25:23.999 INFO:teuthology.orchestra.run.smithi138.stdout:ceph noarch packages 101 kB/s | 24 kB 00:00 2024-09-14T06:25:24.271 INFO:teuthology.orchestra.run.smithi094.stdout:ceph source packages 8.1 kB/s | 1.9 kB 00:00 2024-09-14T06:25:24.275 INFO:teuthology.orchestra.run.smithi138.stdout:ceph source packages 8.1 kB/s | 1.9 kB 00:00 2024-09-14T06:25:27.400 INFO:teuthology.orchestra.run.smithi138.stdout:CentOS Stream 9 - BaseOS 2.7 MB/s | 8.2 MB 00:03 2024-09-14T06:25:28.172 INFO:teuthology.orchestra.run.smithi094.stdout:CentOS Stream 9 - BaseOS 2.1 MB/s | 8.2 MB 00:03 2024-09-14T06:25:30.065 INFO:teuthology.orchestra.run.smithi094.stdout:CentOS Stream 9 - AppStream 26 MB/s | 20 MB 00:00 2024-09-14T06:25:34.541 INFO:teuthology.orchestra.run.smithi094.stdout:CentOS Stream 9 - CRB 10 MB/s | 6.5 MB 00:00 2024-09-14T06:25:35.925 INFO:teuthology.orchestra.run.smithi094.stdout:CentOS Stream 9 - Extras packages 132 kB/s | 19 kB 00:00 2024-09-14T06:25:36.871 INFO:teuthology.orchestra.run.smithi094.stdout:Extra Packages for Enterprise Linux 28 MB/s | 23 MB 00:00 2024-09-14T06:25:42.302 INFO:teuthology.orchestra.run.smithi094.stdout:lab-extras 30 kB/s | 1.7 kB 00:00 2024-09-14T06:25:43.100 INFO:teuthology.orchestra.run.smithi138.stdout:CentOS Stream 9 - AppStream 1.4 MB/s | 20 MB 00:14 2024-09-14T06:25:44.670 INFO:teuthology.orchestra.run.smithi094.stdout:Dependencies resolved. 2024-09-14T06:25:44.687 INFO:teuthology.orchestra.run.smithi094.stdout:================================================================================ 2024-09-14T06:25:44.687 INFO:teuthology.orchestra.run.smithi094.stdout: Package Arch Version Repository Size 2024-09-14T06:25:44.687 INFO:teuthology.orchestra.run.smithi094.stdout:================================================================================ 2024-09-14T06:25:44.688 INFO:teuthology.orchestra.run.smithi094.stdout:Installing: 2024-09-14T06:25:44.688 INFO:teuthology.orchestra.run.smithi094.stdout: ceph-radosgw x86_64 2:19.3.0-4775.g7f24e589.el9 ceph 11 M 2024-09-14T06:25:44.688 INFO:teuthology.orchestra.run.smithi094.stdout:Upgrading: 2024-09-14T06:25:44.688 INFO:teuthology.orchestra.run.smithi094.stdout: librados2 x86_64 2:19.3.0-4775.g7f24e589.el9 ceph 3.5 M 2024-09-14T06:25:44.688 INFO:teuthology.orchestra.run.smithi094.stdout: librbd1 x86_64 2:19.3.0-4775.g7f24e589.el9 ceph 3.2 M 2024-09-14T06:25:44.688 INFO:teuthology.orchestra.run.smithi094.stdout:Installing dependencies: 2024-09-14T06:25:44.688 INFO:teuthology.orchestra.run.smithi094.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-09-14T06:25:44.688 INFO:teuthology.orchestra.run.smithi094.stdout: ceph-base x86_64 2:19.3.0-4775.g7f24e589.el9 ceph 5.5 M 2024-09-14T06:25:44.688 INFO:teuthology.orchestra.run.smithi094.stdout: ceph-common x86_64 2:19.3.0-4775.g7f24e589.el9 ceph 22 M 2024-09-14T06:25:44.688 INFO:teuthology.orchestra.run.smithi094.stdout: ceph-selinux x86_64 2:19.3.0-4775.g7f24e589.el9 ceph 25 k 2024-09-14T06:25:44.688 INFO:teuthology.orchestra.run.smithi094.stdout: gperftools-libs x86_64 2.9.1-3.el9 epel 308 k 2024-09-14T06:25:44.688 INFO:teuthology.orchestra.run.smithi094.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-09-14T06:25:44.689 INFO:teuthology.orchestra.run.smithi094.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-09-14T06:25:44.689 INFO:teuthology.orchestra.run.smithi094.stdout: libcephfs2 x86_64 2:19.3.0-4775.g7f24e589.el9 ceph 755 k 2024-09-14T06:25:44.689 INFO:teuthology.orchestra.run.smithi094.stdout: libnbd x86_64 1.20.2-2.el9 appstream 171 k 2024-09-14T06:25:44.689 INFO:teuthology.orchestra.run.smithi094.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-09-14T06:25:44.689 INFO:teuthology.orchestra.run.smithi094.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-09-14T06:25:44.689 INFO:teuthology.orchestra.run.smithi094.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-09-14T06:25:44.689 INFO:teuthology.orchestra.run.smithi094.stdout: libradosstriper1 x86_64 2:19.3.0-4775.g7f24e589.el9 ceph 508 k 2024-09-14T06:25:44.689 INFO:teuthology.orchestra.run.smithi094.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-09-14T06:25:44.689 INFO:teuthology.orchestra.run.smithi094.stdout: librgw2 x86_64 2:19.3.0-4775.g7f24e589.el9 ceph 5.5 M 2024-09-14T06:25:44.689 INFO:teuthology.orchestra.run.smithi094.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-09-14T06:25:44.689 INFO:teuthology.orchestra.run.smithi094.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-09-14T06:25:44.689 INFO:teuthology.orchestra.run.smithi094.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-09-14T06:25:44.689 INFO:teuthology.orchestra.run.smithi094.stdout: python3-ceph-argparse x86_64 2:19.3.0-4775.g7f24e589.el9 ceph 45 k 2024-09-14T06:25:44.689 INFO:teuthology.orchestra.run.smithi094.stdout: python3-ceph-common x86_64 2:19.3.0-4775.g7f24e589.el9 ceph 143 k 2024-09-14T06:25:44.690 INFO:teuthology.orchestra.run.smithi094.stdout: python3-cephfs x86_64 2:19.3.0-4775.g7f24e589.el9 ceph 161 k 2024-09-14T06:25:44.690 INFO:teuthology.orchestra.run.smithi094.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-09-14T06:25:44.690 INFO:teuthology.orchestra.run.smithi094.stdout: python3-rados x86_64 2:19.3.0-4775.g7f24e589.el9 ceph 321 k 2024-09-14T06:25:44.690 INFO:teuthology.orchestra.run.smithi094.stdout: python3-rbd x86_64 2:19.3.0-4775.g7f24e589.el9 ceph 306 k 2024-09-14T06:25:44.690 INFO:teuthology.orchestra.run.smithi094.stdout: python3-rgw x86_64 2:19.3.0-4775.g7f24e589.el9 ceph 99 k 2024-09-14T06:25:44.690 INFO:teuthology.orchestra.run.smithi094.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-09-14T06:25:44.690 INFO:teuthology.orchestra.run.smithi094.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-09-14T06:25:44.690 INFO:teuthology.orchestra.run.smithi094.stdout: re2 x86_64 1:20211101-20.el9 epel 191 k 2024-09-14T06:25:44.690 INFO:teuthology.orchestra.run.smithi094.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-09-14T06:25:44.690 INFO:teuthology.orchestra.run.smithi094.stdout:Installing weak dependencies: 2024-09-14T06:25:44.690 INFO:teuthology.orchestra.run.smithi094.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-09-14T06:25:44.690 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-14T06:25:44.690 INFO:teuthology.orchestra.run.smithi094.stdout:Transaction Summary 2024-09-14T06:25:44.691 INFO:teuthology.orchestra.run.smithi094.stdout:================================================================================ 2024-09-14T06:25:44.691 INFO:teuthology.orchestra.run.smithi094.stdout:Install 31 Packages 2024-09-14T06:25:44.691 INFO:teuthology.orchestra.run.smithi094.stdout:Upgrade 2 Packages 2024-09-14T06:25:44.691 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-14T06:25:44.691 INFO:teuthology.orchestra.run.smithi094.stdout:Total download size: 62 M 2024-09-14T06:25:44.691 INFO:teuthology.orchestra.run.smithi094.stdout:Downloading Packages: 2024-09-14T06:25:45.212 INFO:teuthology.orchestra.run.smithi094.stdout:(1/33): ceph-base-19.3.0-4775.g7f24e589.el9.x86 12 MB/s | 5.5 MB 00:00 2024-09-14T06:25:45.279 INFO:teuthology.orchestra.run.smithi094.stdout:(2/33): ceph-selinux-19.3.0-4775.g7f24e589.el9. 378 kB/s | 25 kB 00:00 2024-09-14T06:25:45.362 INFO:teuthology.orchestra.run.smithi094.stdout:(3/33): libcephfs2-19.3.0-4775.g7f24e589.el9.x8 8.9 MB/s | 755 kB 00:00 2024-09-14T06:25:45.446 INFO:teuthology.orchestra.run.smithi094.stdout:(4/33): libradosstriper1-19.3.0-4775.g7f24e589. 6.0 MB/s | 508 kB 00:00 2024-09-14T06:25:45.580 INFO:teuthology.orchestra.run.smithi094.stdout:(5/33): ceph-radosgw-19.3.0-4775.g7f24e589.el9. 14 MB/s | 11 MB 00:00 2024-09-14T06:25:45.647 INFO:teuthology.orchestra.run.smithi094.stdout:(6/33): python3-ceph-argparse-19.3.0-4775.g7f24 678 kB/s | 45 kB 00:00 2024-09-14T06:25:45.739 INFO:teuthology.orchestra.run.smithi094.stdout:(7/33): librgw2-19.3.0-4775.g7f24e589.el9.x86_6 19 MB/s | 5.5 MB 00:00 2024-09-14T06:25:45.764 INFO:teuthology.orchestra.run.smithi094.stdout:(8/33): python3-ceph-common-19.3.0-4775.g7f24e5 1.2 MB/s | 143 kB 00:00 2024-09-14T06:25:45.789 INFO:teuthology.orchestra.run.smithi094.stdout:(9/33): python3-cephfs-19.3.0-4775.g7f24e589.el 3.1 MB/s | 161 kB 00:00 2024-09-14T06:25:45.815 INFO:teuthology.orchestra.run.smithi094.stdout:(10/33): python3-rados-19.3.0-4775.g7f24e589.el 6.2 MB/s | 321 kB 00:00 2024-09-14T06:25:45.848 INFO:teuthology.orchestra.run.smithi094.stdout:(11/33): python3-rbd-19.3.0-4775.g7f24e589.el9. 5.1 MB/s | 306 kB 00:00 2024-09-14T06:25:45.874 INFO:teuthology.orchestra.run.smithi094.stdout:(12/33): python3-rgw-19.3.0-4775.g7f24e589.el9. 1.7 MB/s | 99 kB 00:00 2024-09-14T06:25:46.124 INFO:teuthology.orchestra.run.smithi094.stdout:(13/33): ceph-common-19.3.0-4775.g7f24e589.el9. 16 MB/s | 22 MB 00:01 2024-09-14T06:25:46.191 INFO:teuthology.orchestra.run.smithi094.stdout:(14/33): boost-program-options-1.75.0-8.el9.x86 312 kB/s | 107 kB 00:00 2024-09-14T06:25:46.233 INFO:teuthology.orchestra.run.smithi094.stdout:(15/33): libnbd-1.20.2-2.el9.x86_64.rpm 475 kB/s | 171 kB 00:00 2024-09-14T06:25:46.258 INFO:teuthology.orchestra.run.smithi094.stdout:(16/33): librabbitmq-0.11.0-7.el9.x86_64.rpm 676 kB/s | 45 kB 00:00 2024-09-14T06:25:46.292 INFO:teuthology.orchestra.run.smithi094.stdout:(17/33): libpmemobj-1.12.1-1.el9.x86_64.rpm 958 kB/s | 160 kB 00:00 2024-09-14T06:25:46.334 INFO:teuthology.orchestra.run.smithi094.stdout:(18/33): python3-prettytable-0.7.2-27.el9.noarc 1.0 MB/s | 42 kB 00:00 2024-09-14T06:25:46.409 INFO:teuthology.orchestra.run.smithi094.stdout:(19/33): qatlib-24.02.0-1.el9.x86_64.rpm 2.9 MB/s | 221 kB 00:00 2024-09-14T06:25:46.442 INFO:teuthology.orchestra.run.smithi094.stdout:(20/33): lttng-ust-2.12.0-6.el9.x86_64.rpm 1.5 MB/s | 292 kB 00:00 2024-09-14T06:25:46.468 INFO:teuthology.orchestra.run.smithi094.stdout:(21/33): qatlib-service-24.02.0-1.el9.x86_64.rp 614 kB/s | 36 kB 00:00 2024-09-14T06:25:46.501 INFO:teuthology.orchestra.run.smithi094.stdout:(22/33): qatzip-libs-1.2.0-1.el9.x86_64.rpm 809 kB/s | 47 kB 00:00 2024-09-14T06:25:46.527 INFO:teuthology.orchestra.run.smithi094.stdout:(23/33): librdkafka-1.6.1-102.el9.x86_64.rpm 2.2 MB/s | 662 kB 00:00 2024-09-14T06:25:46.569 INFO:teuthology.orchestra.run.smithi094.stdout:(24/33): gperftools-libs-2.9.1-3.el9.x86_64.rpm 3.0 MB/s | 308 kB 00:00 2024-09-14T06:25:46.594 INFO:teuthology.orchestra.run.smithi094.stdout:(25/33): liboath-2.6.7-2.el9.x86_64.rpm 2.1 MB/s | 49 kB 00:00 2024-09-14T06:25:46.619 INFO:teuthology.orchestra.run.smithi094.stdout:(26/33): libarrow-doc-9.0.0-11.el9.noarch.rpm 306 kB/s | 26 kB 00:00 2024-09-14T06:25:46.645 INFO:teuthology.orchestra.run.smithi094.stdout:(27/33): libunwind-1.6.2-1.el9.x86_64.rpm 1.3 MB/s | 67 kB 00:00 2024-09-14T06:25:46.678 INFO:teuthology.orchestra.run.smithi094.stdout:(28/33): re2-20211101-20.el9.x86_64.rpm 5.6 MB/s | 191 kB 00:00 2024-09-14T06:25:46.729 INFO:teuthology.orchestra.run.smithi094.stdout:(29/33): parquet-libs-9.0.0-11.el9.x86_64.rpm 7.5 MB/s | 839 kB 00:00 2024-09-14T06:25:46.787 INFO:teuthology.orchestra.run.smithi094.stdout:(30/33): thrift-0.15.0-2.el9.x86_64.rpm 15 MB/s | 1.6 MB 00:00 2024-09-14T06:25:46.862 INFO:teuthology.orchestra.run.smithi094.stdout:(31/33): libarrow-9.0.0-11.el9.x86_64.rpm 12 MB/s | 4.4 MB 00:00 2024-09-14T06:25:47.005 INFO:teuthology.orchestra.run.smithi094.stdout:(32/33): librados2-19.3.0-4775.g7f24e589.el9.x8 13 MB/s | 3.5 MB 00:00 2024-09-14T06:25:47.063 INFO:teuthology.orchestra.run.smithi094.stdout:(33/33): librbd1-19.3.0-4775.g7f24e589.el9.x86_ 12 MB/s | 3.2 MB 00:00 2024-09-14T06:25:47.067 INFO:teuthology.orchestra.run.smithi094.stdout:-------------------------------------------------------------------------------- 2024-09-14T06:25:47.068 INFO:teuthology.orchestra.run.smithi094.stdout:Total 26 MB/s | 62 MB 00:02 2024-09-14T06:25:47.177 INFO:teuthology.orchestra.run.smithi094.stdout:Running transaction check 2024-09-14T06:25:47.213 INFO:teuthology.orchestra.run.smithi094.stdout:Transaction check succeeded. 2024-09-14T06:25:47.213 INFO:teuthology.orchestra.run.smithi094.stdout:Running transaction test 2024-09-14T06:25:47.884 INFO:teuthology.orchestra.run.smithi094.stdout:Transaction test succeeded. 2024-09-14T06:25:47.885 INFO:teuthology.orchestra.run.smithi094.stdout:Running transaction 2024-09-14T06:25:48.726 INFO:teuthology.orchestra.run.smithi138.stdout:CentOS Stream 9 - CRB 3.5 MB/s | 6.5 MB 00:01 2024-09-14T06:25:49.388 INFO:teuthology.orchestra.run.smithi094.stdout: Preparing : 1/1 2024-09-14T06:25:49.479 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 1/35 2024-09-14T06:25:49.526 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 2/35 2024-09-14T06:25:49.552 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 3/35 2024-09-14T06:25:49.575 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 4/35 2024-09-14T06:25:49.603 INFO:teuthology.orchestra.run.smithi094.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 4/35 2024-09-14T06:25:49.794 INFO:teuthology.orchestra.run.smithi094.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 5/35 2024-09-14T06:25:49.855 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 5/35 2024-09-14T06:25:50.110 INFO:teuthology.orchestra.run.smithi138.stdout:CentOS Stream 9 - Extras packages 112 kB/s | 19 kB 00:00 2024-09-14T06:25:50.246 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : thrift-0.15.0-2.el9.x86_64 6/35 2024-09-14T06:25:50.337 INFO:teuthology.orchestra.run.smithi094.stdout: Upgrading : librados2-2:19.3.0-4775.g7f24e589.el9.x86_64 7/35 2024-09-14T06:25:50.432 INFO:teuthology.orchestra.run.smithi094.stdout: Running scriptlet: librados2-2:19.3.0-4775.g7f24e589.el9.x86_64 7/35 2024-09-14T06:25:50.527 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-rados-2:19.3.0-4775.g7f24e589.el9.x86_64 8/35 2024-09-14T06:25:50.556 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : libcephfs2-2:19.3.0-4775.g7f24e589.el9.x86_64 9/35 2024-09-14T06:25:50.619 INFO:teuthology.orchestra.run.smithi094.stdout: Running scriptlet: libcephfs2-2:19.3.0-4775.g7f24e589.el9.x86_64 9/35 2024-09-14T06:25:50.655 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : liboath-2.6.7-2.el9.x86_64 10/35 2024-09-14T06:25:50.699 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-ceph-argparse-2:19.3.0-4775.g7f24e589.el9. 11/35 2024-09-14T06:25:50.768 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-cephfs-2:19.3.0-4775.g7f24e589.el9.x86_64 12/35 2024-09-14T06:25:50.792 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : libradosstriper1-2:19.3.0-4775.g7f24e589.el9.x86_6 13/35 2024-09-14T06:25:50.856 INFO:teuthology.orchestra.run.smithi094.stdout: Running scriptlet: libradosstriper1-2:19.3.0-4775.g7f24e589.el9.x86_6 13/35 2024-09-14T06:25:50.891 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 14/35 2024-09-14T06:25:50.981 INFO:teuthology.orchestra.run.smithi138.stdout:Extra Packages for Enterprise Linux 30 MB/s | 23 MB 00:00 2024-09-14T06:25:50.998 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : re2-1:20211101-20.el9.x86_64 15/35 2024-09-14T06:25:51.033 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 16/35 2024-09-14T06:25:51.083 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 17/35 2024-09-14T06:25:51.123 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : gperftools-libs-2.9.1-3.el9.x86_64 18/35 2024-09-14T06:25:51.161 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 19/35 2024-09-14T06:25:51.202 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 20/35 2024-09-14T06:25:51.240 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 21/35 2024-09-14T06:25:51.556 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : libnbd-1.20.2-2.el9.x86_64 22/35 2024-09-14T06:25:51.599 INFO:teuthology.orchestra.run.smithi094.stdout: Upgrading : librbd1-2:19.3.0-4775.g7f24e589.el9.x86_64 23/35 2024-09-14T06:25:51.689 INFO:teuthology.orchestra.run.smithi094.stdout: Running scriptlet: librbd1-2:19.3.0-4775.g7f24e589.el9.x86_64 23/35 2024-09-14T06:25:51.721 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-rbd-2:19.3.0-4775.g7f24e589.el9.x86_64 24/35 2024-09-14T06:25:51.765 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 25/35 2024-09-14T06:25:52.264 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 26/35 2024-09-14T06:25:52.293 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : librgw2-2:19.3.0-4775.g7f24e589.el9.x86_64 27/35 2024-09-14T06:25:52.374 INFO:teuthology.orchestra.run.smithi094.stdout: Running scriptlet: librgw2-2:19.3.0-4775.g7f24e589.el9.x86_64 27/35 2024-09-14T06:25:52.452 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-rgw-2:19.3.0-4775.g7f24e589.el9.x86_64 28/35 2024-09-14T06:25:52.496 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-ceph-common-2:19.3.0-4775.g7f24e589.el9.x8 29/35 2024-09-14T06:25:54.839 INFO:teuthology.orchestra.run.smithi094.stdout: Running scriptlet: ceph-common-2:19.3.0-4775.g7f24e589.el9.x86_64 30/35 2024-09-14T06:25:54.876 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : ceph-common-2:19.3.0-4775.g7f24e589.el9.x86_64 30/35 2024-09-14T06:25:55.455 INFO:teuthology.orchestra.run.smithi094.stdout: Running scriptlet: ceph-common-2:19.3.0-4775.g7f24e589.el9.x86_64 30/35 2024-09-14T06:25:55.489 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : ceph-base-2:19.3.0-4775.g7f24e589.el9.x86_64 31/35 2024-09-14T06:25:55.585 INFO:teuthology.orchestra.run.smithi094.stdout: Running scriptlet: ceph-base-2:19.3.0-4775.g7f24e589.el9.x86_64 31/35 2024-09-14T06:25:55.585 INFO:teuthology.orchestra.run.smithi094.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-09-14T06:25:55.585 INFO:teuthology.orchestra.run.smithi094.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-09-14T06:25:55.585 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-14T06:25:55.623 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : ceph-selinux-2:19.3.0-4775.g7f24e589.el9.x86_64 32/35 2024-09-14T06:25:56.354 INFO:teuthology.orchestra.run.smithi138.stdout:lab-extras 30 kB/s | 1.7 kB 00:00 2024-09-14T06:25:58.781 INFO:teuthology.orchestra.run.smithi138.stdout:Dependencies resolved. 2024-09-14T06:25:58.792 INFO:teuthology.orchestra.run.smithi138.stdout:================================================================================ 2024-09-14T06:25:58.792 INFO:teuthology.orchestra.run.smithi138.stdout: Package Arch Version Repository Size 2024-09-14T06:25:58.792 INFO:teuthology.orchestra.run.smithi138.stdout:================================================================================ 2024-09-14T06:25:58.793 INFO:teuthology.orchestra.run.smithi138.stdout:Installing: 2024-09-14T06:25:58.793 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-radosgw x86_64 2:19.3.0-4775.g7f24e589.el9 ceph 11 M 2024-09-14T06:25:58.793 INFO:teuthology.orchestra.run.smithi138.stdout:Upgrading: 2024-09-14T06:25:58.793 INFO:teuthology.orchestra.run.smithi138.stdout: librados2 x86_64 2:19.3.0-4775.g7f24e589.el9 ceph 3.5 M 2024-09-14T06:25:58.793 INFO:teuthology.orchestra.run.smithi138.stdout: librbd1 x86_64 2:19.3.0-4775.g7f24e589.el9 ceph 3.2 M 2024-09-14T06:25:58.793 INFO:teuthology.orchestra.run.smithi138.stdout:Installing dependencies: 2024-09-14T06:25:58.793 INFO:teuthology.orchestra.run.smithi138.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-09-14T06:25:58.793 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-base x86_64 2:19.3.0-4775.g7f24e589.el9 ceph 5.5 M 2024-09-14T06:25:58.793 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-common x86_64 2:19.3.0-4775.g7f24e589.el9 ceph 22 M 2024-09-14T06:25:58.793 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-selinux x86_64 2:19.3.0-4775.g7f24e589.el9 ceph 25 k 2024-09-14T06:25:58.793 INFO:teuthology.orchestra.run.smithi138.stdout: gperftools-libs x86_64 2.9.1-3.el9 epel 308 k 2024-09-14T06:25:58.793 INFO:teuthology.orchestra.run.smithi138.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-09-14T06:25:58.794 INFO:teuthology.orchestra.run.smithi138.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-09-14T06:25:58.794 INFO:teuthology.orchestra.run.smithi138.stdout: libcephfs2 x86_64 2:19.3.0-4775.g7f24e589.el9 ceph 755 k 2024-09-14T06:25:58.794 INFO:teuthology.orchestra.run.smithi138.stdout: libnbd x86_64 1.20.2-2.el9 appstream 171 k 2024-09-14T06:25:58.794 INFO:teuthology.orchestra.run.smithi138.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-09-14T06:25:58.794 INFO:teuthology.orchestra.run.smithi138.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-09-14T06:25:58.794 INFO:teuthology.orchestra.run.smithi138.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-09-14T06:25:58.794 INFO:teuthology.orchestra.run.smithi138.stdout: libradosstriper1 x86_64 2:19.3.0-4775.g7f24e589.el9 ceph 508 k 2024-09-14T06:25:58.794 INFO:teuthology.orchestra.run.smithi138.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-09-14T06:25:58.794 INFO:teuthology.orchestra.run.smithi138.stdout: librgw2 x86_64 2:19.3.0-4775.g7f24e589.el9 ceph 5.5 M 2024-09-14T06:25:58.794 INFO:teuthology.orchestra.run.smithi138.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-09-14T06:25:58.794 INFO:teuthology.orchestra.run.smithi138.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-09-14T06:25:58.794 INFO:teuthology.orchestra.run.smithi138.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-09-14T06:25:58.794 INFO:teuthology.orchestra.run.smithi138.stdout: python3-ceph-argparse x86_64 2:19.3.0-4775.g7f24e589.el9 ceph 45 k 2024-09-14T06:25:58.794 INFO:teuthology.orchestra.run.smithi138.stdout: python3-ceph-common x86_64 2:19.3.0-4775.g7f24e589.el9 ceph 143 k 2024-09-14T06:25:58.795 INFO:teuthology.orchestra.run.smithi138.stdout: python3-cephfs x86_64 2:19.3.0-4775.g7f24e589.el9 ceph 161 k 2024-09-14T06:25:58.795 INFO:teuthology.orchestra.run.smithi138.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-09-14T06:25:58.795 INFO:teuthology.orchestra.run.smithi138.stdout: python3-rados x86_64 2:19.3.0-4775.g7f24e589.el9 ceph 321 k 2024-09-14T06:25:58.795 INFO:teuthology.orchestra.run.smithi138.stdout: python3-rbd x86_64 2:19.3.0-4775.g7f24e589.el9 ceph 306 k 2024-09-14T06:25:58.795 INFO:teuthology.orchestra.run.smithi138.stdout: python3-rgw x86_64 2:19.3.0-4775.g7f24e589.el9 ceph 99 k 2024-09-14T06:25:58.795 INFO:teuthology.orchestra.run.smithi138.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-09-14T06:25:58.795 INFO:teuthology.orchestra.run.smithi138.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-09-14T06:25:58.795 INFO:teuthology.orchestra.run.smithi138.stdout: re2 x86_64 1:20211101-20.el9 epel 191 k 2024-09-14T06:25:58.795 INFO:teuthology.orchestra.run.smithi138.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-09-14T06:25:58.795 INFO:teuthology.orchestra.run.smithi138.stdout:Installing weak dependencies: 2024-09-14T06:25:58.795 INFO:teuthology.orchestra.run.smithi138.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-09-14T06:25:58.795 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-09-14T06:25:58.795 INFO:teuthology.orchestra.run.smithi138.stdout:Transaction Summary 2024-09-14T06:25:58.795 INFO:teuthology.orchestra.run.smithi138.stdout:================================================================================ 2024-09-14T06:25:58.795 INFO:teuthology.orchestra.run.smithi138.stdout:Install 31 Packages 2024-09-14T06:25:58.796 INFO:teuthology.orchestra.run.smithi138.stdout:Upgrade 2 Packages 2024-09-14T06:25:58.796 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-09-14T06:25:58.796 INFO:teuthology.orchestra.run.smithi138.stdout:Total download size: 62 M 2024-09-14T06:25:58.796 INFO:teuthology.orchestra.run.smithi138.stdout:Downloading Packages: 2024-09-14T06:25:59.640 INFO:teuthology.orchestra.run.smithi138.stdout:(1/33): ceph-base-19.3.0-4775.g7f24e589.el9.x86 8.8 MB/s | 5.5 MB 00:00 2024-09-14T06:25:59.698 INFO:teuthology.orchestra.run.smithi138.stdout:(2/33): ceph-selinux-19.3.0-4775.g7f24e589.el9. 433 kB/s | 25 kB 00:00 2024-09-14T06:25:59.898 INFO:teuthology.orchestra.run.smithi138.stdout:(3/33): ceph-radosgw-19.3.0-4775.g7f24e589.el9. 12 MB/s | 11 MB 00:00 2024-09-14T06:25:59.924 INFO:teuthology.orchestra.run.smithi138.stdout:(4/33): libcephfs2-19.3.0-4775.g7f24e589.el9.x8 3.3 MB/s | 755 kB 00:00 2024-09-14T06:25:59.949 INFO:teuthology.orchestra.run.smithi138.stdout:(5/33): libradosstriper1-19.3.0-4775.g7f24e589. 9.9 MB/s | 508 kB 00:00 2024-09-14T06:26:00.008 INFO:teuthology.orchestra.run.smithi138.stdout:(6/33): python3-ceph-argparse-19.3.0-4775.g7f24 771 kB/s | 45 kB 00:00 2024-09-14T06:26:00.242 INFO:teuthology.orchestra.run.smithi138.stdout:(7/33): ceph-common-19.3.0-4775.g7f24e589.el9.x 18 MB/s | 22 MB 00:01 2024-09-14T06:26:00.267 INFO:teuthology.orchestra.run.smithi138.stdout:(8/33): python3-ceph-common-19.3.0-4775.g7f24e5 553 kB/s | 143 kB 00:00 2024-09-14T06:26:00.314 INFO:teuthology.orchestra.run.smithi138.stdout:(9/33): python3-cephfs-19.3.0-4775.g7f24e589.el 2.2 MB/s | 161 kB 00:00 2024-09-14T06:26:00.342 INFO:teuthology.orchestra.run.smithi138.stdout:(10/33): python3-rados-19.3.0-4775.g7f24e589.el 4.2 MB/s | 321 kB 00:00 2024-09-14T06:26:00.368 INFO:teuthology.orchestra.run.smithi138.stdout:(11/33): python3-rbd-19.3.0-4775.g7f24e589.el9. 5.6 MB/s | 306 kB 00:00 2024-09-14T06:26:00.393 INFO:teuthology.orchestra.run.smithi138.stdout:(12/33): python3-rgw-19.3.0-4775.g7f24e589.el9. 1.9 MB/s | 99 kB 00:00 2024-09-14T06:26:00.485 INFO:teuthology.orchestra.run.smithi138.stdout:(13/33): librgw2-19.3.0-4775.g7f24e589.el9.x86_ 9.8 MB/s | 5.5 MB 00:00 2024-09-14T06:26:00.527 INFO:teuthology.orchestra.run.smithi138.stdout:(14/33): boost-program-options-1.75.0-8.el9.x86 671 kB/s | 107 kB 00:00 2024-09-14T06:26:00.594 INFO:teuthology.orchestra.run.smithi138.stdout:(15/33): libnbd-1.20.2-2.el9.x86_64.rpm 849 kB/s | 171 kB 00:00 2024-09-14T06:26:00.620 INFO:teuthology.orchestra.run.smithi138.stdout:(16/33): librabbitmq-0.11.0-7.el9.x86_64.rpm 491 kB/s | 45 kB 00:00 2024-09-14T06:26:00.645 INFO:teuthology.orchestra.run.smithi138.stdout:(17/33): libpmemobj-1.12.1-1.el9.x86_64.rpm 1.0 MB/s | 160 kB 00:00 2024-09-14T06:26:00.687 INFO:teuthology.orchestra.run.smithi138.stdout:(18/33): librdkafka-1.6.1-102.el9.x86_64.rpm 7.0 MB/s | 662 kB 00:00 2024-09-14T06:26:00.712 INFO:teuthology.orchestra.run.smithi138.stdout:(19/33): python3-prettytable-0.7.2-27.el9.noarc 631 kB/s | 42 kB 00:00 2024-09-14T06:26:00.737 INFO:teuthology.orchestra.run.smithi138.stdout:(20/33): lttng-ust-2.12.0-6.el9.x86_64.rpm 2.4 MB/s | 292 kB 00:00 2024-09-14T06:26:00.763 INFO:teuthology.orchestra.run.smithi138.stdout:(21/33): qatlib-24.02.0-1.el9.x86_64.rpm 2.8 MB/s | 221 kB 00:00 2024-09-14T06:26:00.788 INFO:teuthology.orchestra.run.smithi138.stdout:(22/33): qatlib-service-24.02.0-1.el9.x86_64.rp 474 kB/s | 36 kB 00:00 2024-09-14T06:26:00.830 INFO:teuthology.orchestra.run.smithi138.stdout:(23/33): qatzip-libs-1.2.0-1.el9.x86_64.rpm 512 kB/s | 47 kB 00:00 2024-09-14T06:26:00.864 INFO:teuthology.orchestra.run.smithi138.stdout:(24/33): gperftools-libs-2.9.1-3.el9.x86_64.rpm 3.0 MB/s | 308 kB 00:00 2024-09-14T06:26:00.889 INFO:teuthology.orchestra.run.smithi138.stdout:(25/33): liboath-2.6.7-2.el9.x86_64.rpm 2.1 MB/s | 49 kB 00:00 2024-09-14T06:26:00.915 INFO:teuthology.orchestra.run.smithi138.stdout:(26/33): libarrow-doc-9.0.0-11.el9.noarch.rpm 315 kB/s | 26 kB 00:00 2024-09-14T06:26:00.940 INFO:teuthology.orchestra.run.smithi138.stdout:(27/33): libunwind-1.6.2-1.el9.x86_64.rpm 1.3 MB/s | 67 kB 00:00 2024-09-14T06:26:00.974 INFO:teuthology.orchestra.run.smithi138.stdout:(28/33): re2-20211101-20.el9.x86_64.rpm 5.6 MB/s | 191 kB 00:00 2024-09-14T06:26:01.007 INFO:teuthology.orchestra.run.smithi138.stdout:(29/33): parquet-libs-9.0.0-11.el9.x86_64.rpm 8.9 MB/s | 839 kB 00:00 2024-09-14T06:26:01.058 INFO:teuthology.orchestra.run.smithi138.stdout:(30/33): thrift-0.15.0-2.el9.x86_64.rpm 19 MB/s | 1.6 MB 00:00 2024-09-14T06:26:01.177 INFO:teuthology.orchestra.run.smithi138.stdout:(31/33): libarrow-9.0.0-11.el9.x86_64.rpm 11 MB/s | 4.4 MB 00:00 2024-09-14T06:26:01.459 INFO:teuthology.orchestra.run.smithi138.stdout:(32/33): librados2-19.3.0-4775.g7f24e589.el9.x8 7.7 MB/s | 3.5 MB 00:00 2024-09-14T06:26:01.517 INFO:teuthology.orchestra.run.smithi138.stdout:(33/33): librbd1-19.3.0-4775.g7f24e589.el9.x86_ 7.1 MB/s | 3.2 MB 00:00 2024-09-14T06:26:01.522 INFO:teuthology.orchestra.run.smithi138.stdout:-------------------------------------------------------------------------------- 2024-09-14T06:26:01.523 INFO:teuthology.orchestra.run.smithi138.stdout:Total 23 MB/s | 62 MB 00:02 2024-09-14T06:26:01.633 INFO:teuthology.orchestra.run.smithi138.stdout:Running transaction check 2024-09-14T06:26:01.669 INFO:teuthology.orchestra.run.smithi138.stdout:Transaction check succeeded. 2024-09-14T06:26:01.670 INFO:teuthology.orchestra.run.smithi138.stdout:Running transaction test 2024-09-14T06:26:02.357 INFO:teuthology.orchestra.run.smithi138.stdout:Transaction test succeeded. 2024-09-14T06:26:02.357 INFO:teuthology.orchestra.run.smithi138.stdout:Running transaction 2024-09-14T06:26:03.907 INFO:teuthology.orchestra.run.smithi138.stdout: Preparing : 1/1 2024-09-14T06:26:03.985 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 1/35 2024-09-14T06:26:04.014 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 2/35 2024-09-14T06:26:04.040 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 3/35 2024-09-14T06:26:04.062 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 4/35 2024-09-14T06:26:04.084 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 4/35 2024-09-14T06:26:04.298 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 5/35 2024-09-14T06:26:04.352 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 5/35 2024-09-14T06:26:04.739 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : thrift-0.15.0-2.el9.x86_64 6/35 2024-09-14T06:26:04.809 INFO:teuthology.orchestra.run.smithi138.stdout: Upgrading : librados2-2:19.3.0-4775.g7f24e589.el9.x86_64 7/35 2024-09-14T06:26:04.910 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: librados2-2:19.3.0-4775.g7f24e589.el9.x86_64 7/35 2024-09-14T06:26:05.009 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-rados-2:19.3.0-4775.g7f24e589.el9.x86_64 8/35 2024-09-14T06:26:05.045 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : libcephfs2-2:19.3.0-4775.g7f24e589.el9.x86_64 9/35 2024-09-14T06:26:05.108 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: libcephfs2-2:19.3.0-4775.g7f24e589.el9.x86_64 9/35 2024-09-14T06:26:05.148 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : liboath-2.6.7-2.el9.x86_64 10/35 2024-09-14T06:26:05.189 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-ceph-argparse-2:19.3.0-4775.g7f24e589.el9. 11/35 2024-09-14T06:26:05.259 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-cephfs-2:19.3.0-4775.g7f24e589.el9.x86_64 12/35 2024-09-14T06:26:05.280 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : libradosstriper1-2:19.3.0-4775.g7f24e589.el9.x86_6 13/35 2024-09-14T06:26:05.345 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: libradosstriper1-2:19.3.0-4775.g7f24e589.el9.x86_6 13/35 2024-09-14T06:26:05.380 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 14/35 2024-09-14T06:26:05.499 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : re2-1:20211101-20.el9.x86_64 15/35 2024-09-14T06:26:05.531 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 16/35 2024-09-14T06:26:05.572 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 17/35 2024-09-14T06:26:05.589 INFO:teuthology.orchestra.run.smithi094.stdout: Running scriptlet: ceph-selinux-2:19.3.0-4775.g7f24e589.el9.x86_64 32/35 2024-09-14T06:26:05.589 INFO:teuthology.orchestra.run.smithi094.stdout:skipping the directory /sys 2024-09-14T06:26:05.589 INFO:teuthology.orchestra.run.smithi094.stdout:skipping the directory /proc 2024-09-14T06:26:05.589 INFO:teuthology.orchestra.run.smithi094.stdout:skipping the directory /mnt 2024-09-14T06:26:05.590 INFO:teuthology.orchestra.run.smithi094.stdout:skipping the directory /var/tmp 2024-09-14T06:26:05.590 INFO:teuthology.orchestra.run.smithi094.stdout:skipping the directory /home 2024-09-14T06:26:05.590 INFO:teuthology.orchestra.run.smithi094.stdout:skipping the directory /root 2024-09-14T06:26:05.590 INFO:teuthology.orchestra.run.smithi094.stdout:skipping the directory /tmp 2024-09-14T06:26:05.590 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-14T06:26:05.605 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : gperftools-libs-2.9.1-3.el9.x86_64 18/35 2024-09-14T06:26:05.634 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 19/35 2024-09-14T06:26:05.691 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 20/35 2024-09-14T06:26:05.730 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 21/35 2024-09-14T06:26:06.048 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : libnbd-1.20.2-2.el9.x86_64 22/35 2024-09-14T06:26:06.098 INFO:teuthology.orchestra.run.smithi138.stdout: Upgrading : librbd1-2:19.3.0-4775.g7f24e589.el9.x86_64 23/35 2024-09-14T06:26:06.199 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: librbd1-2:19.3.0-4775.g7f24e589.el9.x86_64 23/35 2024-09-14T06:26:06.233 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-rbd-2:19.3.0-4775.g7f24e589.el9.x86_64 24/35 2024-09-14T06:26:06.272 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 25/35 2024-09-14T06:26:06.621 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : ceph-radosgw-2:19.3.0-4775.g7f24e589.el9.x86_64 33/35 2024-09-14T06:26:06.646 INFO:teuthology.orchestra.run.smithi094.stdout: Running scriptlet: ceph-radosgw-2:19.3.0-4775.g7f24e589.el9.x86_64 33/35 2024-09-14T06:26:06.646 INFO:teuthology.orchestra.run.smithi094.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-14T06:26:06.646 INFO:teuthology.orchestra.run.smithi094.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-09-14T06:26:06.646 INFO:teuthology.orchestra.run.smithi094.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-14T06:26:06.647 INFO:teuthology.orchestra.run.smithi094.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-14T06:26:06.647 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-14T06:26:06.648 INFO:teuthology.orchestra.run.smithi094.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 34/35 2024-09-14T06:26:06.740 INFO:teuthology.orchestra.run.smithi094.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 34/35 2024-09-14T06:26:06.741 INFO:teuthology.orchestra.run.smithi094.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 35/35 2024-09-14T06:26:06.765 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 26/35 2024-09-14T06:26:06.793 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : librgw2-2:19.3.0-4775.g7f24e589.el9.x86_64 27/35 2024-09-14T06:26:06.856 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: librgw2-2:19.3.0-4775.g7f24e589.el9.x86_64 27/35 2024-09-14T06:26:06.934 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-rgw-2:19.3.0-4775.g7f24e589.el9.x86_64 28/35 2024-09-14T06:26:06.962 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-ceph-common-2:19.3.0-4775.g7f24e589.el9.x8 29/35 2024-09-14T06:26:07.949 INFO:teuthology.orchestra.run.smithi094.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 35/35 2024-09-14T06:26:07.949 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : ceph-base-2:19.3.0-4775.g7f24e589.el9.x86_64 1/35 2024-09-14T06:26:07.949 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : ceph-common-2:19.3.0-4775.g7f24e589.el9.x86_64 2/35 2024-09-14T06:26:07.950 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : ceph-radosgw-2:19.3.0-4775.g7f24e589.el9.x86_64 3/35 2024-09-14T06:26:07.950 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : ceph-selinux-2:19.3.0-4775.g7f24e589.el9.x86_64 4/35 2024-09-14T06:26:07.950 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : libcephfs2-2:19.3.0-4775.g7f24e589.el9.x86_64 5/35 2024-09-14T06:26:07.950 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : libradosstriper1-2:19.3.0-4775.g7f24e589.el9.x86_6 6/35 2024-09-14T06:26:07.950 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : librgw2-2:19.3.0-4775.g7f24e589.el9.x86_64 7/35 2024-09-14T06:26:07.950 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-ceph-argparse-2:19.3.0-4775.g7f24e589.el9. 8/35 2024-09-14T06:26:07.950 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-ceph-common-2:19.3.0-4775.g7f24e589.el9.x8 9/35 2024-09-14T06:26:07.950 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-cephfs-2:19.3.0-4775.g7f24e589.el9.x86_64 10/35 2024-09-14T06:26:07.950 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-rados-2:19.3.0-4775.g7f24e589.el9.x86_64 11/35 2024-09-14T06:26:07.950 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-rbd-2:19.3.0-4775.g7f24e589.el9.x86_64 12/35 2024-09-14T06:26:07.950 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-rgw-2:19.3.0-4775.g7f24e589.el9.x86_64 13/35 2024-09-14T06:26:07.950 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 14/35 2024-09-14T06:26:07.950 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : libnbd-1.20.2-2.el9.x86_64 15/35 2024-09-14T06:26:07.951 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 16/35 2024-09-14T06:26:07.951 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 17/35 2024-09-14T06:26:07.951 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 18/35 2024-09-14T06:26:07.951 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 19/35 2024-09-14T06:26:07.951 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 20/35 2024-09-14T06:26:07.951 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 21/35 2024-09-14T06:26:07.951 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 22/35 2024-09-14T06:26:07.951 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 23/35 2024-09-14T06:26:07.951 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : gperftools-libs-2.9.1-3.el9.x86_64 24/35 2024-09-14T06:26:07.951 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 25/35 2024-09-14T06:26:07.951 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 26/35 2024-09-14T06:26:07.951 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 27/35 2024-09-14T06:26:07.951 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 28/35 2024-09-14T06:26:07.952 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 29/35 2024-09-14T06:26:07.952 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : re2-1:20211101-20.el9.x86_64 30/35 2024-09-14T06:26:07.952 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 31/35 2024-09-14T06:26:07.952 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : librados2-2:19.3.0-4775.g7f24e589.el9.x86_64 32/35 2024-09-14T06:26:07.952 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 33/35 2024-09-14T06:26:07.952 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : librbd1-2:19.3.0-4775.g7f24e589.el9.x86_64 34/35 2024-09-14T06:26:08.951 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 35/35 2024-09-14T06:26:08.951 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-14T06:26:08.951 INFO:teuthology.orchestra.run.smithi094.stdout:Upgraded: 2024-09-14T06:26:08.951 INFO:teuthology.orchestra.run.smithi094.stdout: librados2-2:19.3.0-4775.g7f24e589.el9.x86_64 2024-09-14T06:26:08.951 INFO:teuthology.orchestra.run.smithi094.stdout: librbd1-2:19.3.0-4775.g7f24e589.el9.x86_64 2024-09-14T06:26:08.951 INFO:teuthology.orchestra.run.smithi094.stdout:Installed: 2024-09-14T06:26:08.952 INFO:teuthology.orchestra.run.smithi094.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-09-14T06:26:08.952 INFO:teuthology.orchestra.run.smithi094.stdout: ceph-base-2:19.3.0-4775.g7f24e589.el9.x86_64 2024-09-14T06:26:08.952 INFO:teuthology.orchestra.run.smithi094.stdout: ceph-common-2:19.3.0-4775.g7f24e589.el9.x86_64 2024-09-14T06:26:08.952 INFO:teuthology.orchestra.run.smithi094.stdout: ceph-radosgw-2:19.3.0-4775.g7f24e589.el9.x86_64 2024-09-14T06:26:08.952 INFO:teuthology.orchestra.run.smithi094.stdout: ceph-selinux-2:19.3.0-4775.g7f24e589.el9.x86_64 2024-09-14T06:26:08.952 INFO:teuthology.orchestra.run.smithi094.stdout: gperftools-libs-2.9.1-3.el9.x86_64 2024-09-14T06:26:08.952 INFO:teuthology.orchestra.run.smithi094.stdout: libarrow-9.0.0-11.el9.x86_64 2024-09-14T06:26:08.952 INFO:teuthology.orchestra.run.smithi094.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-09-14T06:26:08.952 INFO:teuthology.orchestra.run.smithi094.stdout: libcephfs2-2:19.3.0-4775.g7f24e589.el9.x86_64 2024-09-14T06:26:08.952 INFO:teuthology.orchestra.run.smithi094.stdout: libnbd-1.20.2-2.el9.x86_64 2024-09-14T06:26:08.952 INFO:teuthology.orchestra.run.smithi094.stdout: liboath-2.6.7-2.el9.x86_64 2024-09-14T06:26:08.952 INFO:teuthology.orchestra.run.smithi094.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-09-14T06:26:08.952 INFO:teuthology.orchestra.run.smithi094.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-09-14T06:26:08.953 INFO:teuthology.orchestra.run.smithi094.stdout: libradosstriper1-2:19.3.0-4775.g7f24e589.el9.x86_64 2024-09-14T06:26:08.953 INFO:teuthology.orchestra.run.smithi094.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-09-14T06:26:08.953 INFO:teuthology.orchestra.run.smithi094.stdout: librgw2-2:19.3.0-4775.g7f24e589.el9.x86_64 2024-09-14T06:26:08.953 INFO:teuthology.orchestra.run.smithi094.stdout: libunwind-1.6.2-1.el9.x86_64 2024-09-14T06:26:08.953 INFO:teuthology.orchestra.run.smithi094.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-09-14T06:26:08.953 INFO:teuthology.orchestra.run.smithi094.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-09-14T06:26:08.953 INFO:teuthology.orchestra.run.smithi094.stdout: python3-ceph-argparse-2:19.3.0-4775.g7f24e589.el9.x86_64 2024-09-14T06:26:08.953 INFO:teuthology.orchestra.run.smithi094.stdout: python3-ceph-common-2:19.3.0-4775.g7f24e589.el9.x86_64 2024-09-14T06:26:08.953 INFO:teuthology.orchestra.run.smithi094.stdout: python3-cephfs-2:19.3.0-4775.g7f24e589.el9.x86_64 2024-09-14T06:26:08.953 INFO:teuthology.orchestra.run.smithi094.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-09-14T06:26:08.953 INFO:teuthology.orchestra.run.smithi094.stdout: python3-rados-2:19.3.0-4775.g7f24e589.el9.x86_64 2024-09-14T06:26:08.953 INFO:teuthology.orchestra.run.smithi094.stdout: python3-rbd-2:19.3.0-4775.g7f24e589.el9.x86_64 2024-09-14T06:26:08.953 INFO:teuthology.orchestra.run.smithi094.stdout: python3-rgw-2:19.3.0-4775.g7f24e589.el9.x86_64 2024-09-14T06:26:08.953 INFO:teuthology.orchestra.run.smithi094.stdout: qatlib-24.02.0-1.el9.x86_64 2024-09-14T06:26:08.954 INFO:teuthology.orchestra.run.smithi094.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-09-14T06:26:08.954 INFO:teuthology.orchestra.run.smithi094.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-09-14T06:26:08.954 INFO:teuthology.orchestra.run.smithi094.stdout: re2-1:20211101-20.el9.x86_64 2024-09-14T06:26:08.954 INFO:teuthology.orchestra.run.smithi094.stdout: thrift-0.15.0-2.el9.x86_64 2024-09-14T06:26:08.954 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-14T06:26:08.954 INFO:teuthology.orchestra.run.smithi094.stdout:Complete! 2024-09-14T06:26:09.324 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: ceph-common-2:19.3.0-4775.g7f24e589.el9.x86_64 30/35 2024-09-14T06:26:09.334 DEBUG:teuthology.orchestra.run.smithi094:> sudo yum -y install ceph-test 2024-09-14T06:26:09.360 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : ceph-common-2:19.3.0-4775.g7f24e589.el9.x86_64 30/35 2024-09-14T06:26:09.904 INFO:teuthology.orchestra.run.smithi094.stdout:Last metadata expiration check: 0:00:27 ago on Sat 14 Sep 2024 06:25:42 AM UTC. 2024-09-14T06:26:09.926 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: ceph-common-2:19.3.0-4775.g7f24e589.el9.x86_64 30/35 2024-09-14T06:26:09.964 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : ceph-base-2:19.3.0-4775.g7f24e589.el9.x86_64 31/35 2024-09-14T06:26:10.037 INFO:teuthology.orchestra.run.smithi094.stdout:Dependencies resolved. 2024-09-14T06:26:10.037 INFO:teuthology.orchestra.run.smithi094.stdout:================================================================================ 2024-09-14T06:26:10.037 INFO:teuthology.orchestra.run.smithi094.stdout: Package Arch Version Repository Size 2024-09-14T06:26:10.038 INFO:teuthology.orchestra.run.smithi094.stdout:================================================================================ 2024-09-14T06:26:10.038 INFO:teuthology.orchestra.run.smithi094.stdout:Installing: 2024-09-14T06:26:10.038 INFO:teuthology.orchestra.run.smithi094.stdout: ceph-test x86_64 2:19.3.0-4775.g7f24e589.el9 ceph 50 M 2024-09-14T06:26:10.038 INFO:teuthology.orchestra.run.smithi094.stdout:Installing dependencies: 2024-09-14T06:26:10.038 INFO:teuthology.orchestra.run.smithi094.stdout: libcephsqlite x86_64 2:19.3.0-4775.g7f24e589.el9 ceph 162 k 2024-09-14T06:26:10.038 INFO:teuthology.orchestra.run.smithi094.stdout: socat x86_64 1.7.4.1-6.el9 appstream 304 k 2024-09-14T06:26:10.038 INFO:teuthology.orchestra.run.smithi094.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-09-14T06:26:10.038 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-14T06:26:10.038 INFO:teuthology.orchestra.run.smithi094.stdout:Transaction Summary 2024-09-14T06:26:10.038 INFO:teuthology.orchestra.run.smithi094.stdout:================================================================================ 2024-09-14T06:26:10.038 INFO:teuthology.orchestra.run.smithi094.stdout:Install 4 Packages 2024-09-14T06:26:10.039 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-14T06:26:10.039 INFO:teuthology.orchestra.run.smithi094.stdout:Total download size: 51 M 2024-09-14T06:26:10.039 INFO:teuthology.orchestra.run.smithi094.stdout:Installed size: 217 M 2024-09-14T06:26:10.039 INFO:teuthology.orchestra.run.smithi094.stdout:Downloading Packages: 2024-09-14T06:26:10.054 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: ceph-base-2:19.3.0-4775.g7f24e589.el9.x86_64 31/35 2024-09-14T06:26:10.054 INFO:teuthology.orchestra.run.smithi138.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-09-14T06:26:10.054 INFO:teuthology.orchestra.run.smithi138.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-09-14T06:26:10.054 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-09-14T06:26:10.082 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : ceph-selinux-2:19.3.0-4775.g7f24e589.el9.x86_64 32/35 2024-09-14T06:26:10.271 INFO:teuthology.orchestra.run.smithi094.stdout:(1/4): libcephsqlite-19.3.0-4775.g7f24e589.el9. 767 kB/s | 162 kB 00:00 2024-09-14T06:26:10.296 INFO:teuthology.orchestra.run.smithi094.stdout:(2/4): socat-1.7.4.1-6.el9.x86_64.rpm 1.3 MB/s | 304 kB 00:00 2024-09-14T06:26:10.405 INFO:teuthology.orchestra.run.smithi094.stdout:(3/4): xmlstarlet-1.6.1-20.el9.x86_64.rpm 477 kB/s | 64 kB 00:00 2024-09-14T06:26:11.664 INFO:teuthology.orchestra.run.smithi094.stdout:(4/4): ceph-test-19.3.0-4775.g7f24e589.el9.x86_ 31 MB/s | 50 MB 00:01 2024-09-14T06:26:11.666 INFO:teuthology.orchestra.run.smithi094.stdout:-------------------------------------------------------------------------------- 2024-09-14T06:26:11.666 INFO:teuthology.orchestra.run.smithi094.stdout:Total 31 MB/s | 51 MB 00:01 2024-09-14T06:26:11.692 INFO:teuthology.orchestra.run.smithi094.stdout:Running transaction check 2024-09-14T06:26:11.712 INFO:teuthology.orchestra.run.smithi094.stdout:Transaction check succeeded. 2024-09-14T06:26:11.712 INFO:teuthology.orchestra.run.smithi094.stdout:Running transaction test 2024-09-14T06:26:12.106 INFO:teuthology.orchestra.run.smithi094.stdout:Transaction test succeeded. 2024-09-14T06:26:12.106 INFO:teuthology.orchestra.run.smithi094.stdout:Running transaction 2024-09-14T06:26:12.641 INFO:teuthology.orchestra.run.smithi094.stdout: Preparing : 1/1 2024-09-14T06:26:12.723 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 1/4 2024-09-14T06:26:12.764 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : socat-1.7.4.1-6.el9.x86_64 2/4 2024-09-14T06:26:12.791 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : libcephsqlite-2:19.3.0-4775.g7f24e589.el9.x86_64 3/4 2024-09-14T06:26:17.718 INFO:teuthology.orchestra.run.smithi094.stdout: Running scriptlet: libcephsqlite-2:19.3.0-4775.g7f24e589.el9.x86_64 3/4 2024-09-14T06:26:17.797 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : ceph-test-2:19.3.0-4775.g7f24e589.el9.x86_64 4/4 2024-09-14T06:26:20.056 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: ceph-selinux-2:19.3.0-4775.g7f24e589.el9.x86_64 32/35 2024-09-14T06:26:20.056 INFO:teuthology.orchestra.run.smithi138.stdout:skipping the directory /sys 2024-09-14T06:26:20.056 INFO:teuthology.orchestra.run.smithi138.stdout:skipping the directory /proc 2024-09-14T06:26:20.056 INFO:teuthology.orchestra.run.smithi138.stdout:skipping the directory /mnt 2024-09-14T06:26:20.056 INFO:teuthology.orchestra.run.smithi138.stdout:skipping the directory /var/tmp 2024-09-14T06:26:20.057 INFO:teuthology.orchestra.run.smithi138.stdout:skipping the directory /home 2024-09-14T06:26:20.057 INFO:teuthology.orchestra.run.smithi138.stdout:skipping the directory /root 2024-09-14T06:26:20.057 INFO:teuthology.orchestra.run.smithi138.stdout:skipping the directory /tmp 2024-09-14T06:26:20.057 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-09-14T06:26:20.393 INFO:teuthology.orchestra.run.smithi094.stdout: Running scriptlet: ceph-test-2:19.3.0-4775.g7f24e589.el9.x86_64 4/4 2024-09-14T06:26:20.394 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : ceph-test-2:19.3.0-4775.g7f24e589.el9.x86_64 1/4 2024-09-14T06:26:20.394 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : libcephsqlite-2:19.3.0-4775.g7f24e589.el9.x86_64 2/4 2024-09-14T06:26:20.394 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : socat-1.7.4.1-6.el9.x86_64 3/4 2024-09-14T06:26:20.728 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 4/4 2024-09-14T06:26:20.729 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-14T06:26:20.729 INFO:teuthology.orchestra.run.smithi094.stdout:Installed: 2024-09-14T06:26:20.729 INFO:teuthology.orchestra.run.smithi094.stdout: ceph-test-2:19.3.0-4775.g7f24e589.el9.x86_64 2024-09-14T06:26:20.729 INFO:teuthology.orchestra.run.smithi094.stdout: libcephsqlite-2:19.3.0-4775.g7f24e589.el9.x86_64 2024-09-14T06:26:20.729 INFO:teuthology.orchestra.run.smithi094.stdout: socat-1.7.4.1-6.el9.x86_64 2024-09-14T06:26:20.729 INFO:teuthology.orchestra.run.smithi094.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-09-14T06:26:20.729 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-14T06:26:20.729 INFO:teuthology.orchestra.run.smithi094.stdout:Complete! 2024-09-14T06:26:20.992 DEBUG:teuthology.orchestra.run.smithi094:> sudo yum -y install ceph 2024-09-14T06:26:21.050 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : ceph-radosgw-2:19.3.0-4775.g7f24e589.el9.x86_64 33/35 2024-09-14T06:26:21.076 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: ceph-radosgw-2:19.3.0-4775.g7f24e589.el9.x86_64 33/35 2024-09-14T06:26:21.076 INFO:teuthology.orchestra.run.smithi138.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-14T06:26:21.077 INFO:teuthology.orchestra.run.smithi138.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-09-14T06:26:21.077 INFO:teuthology.orchestra.run.smithi138.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-14T06:26:21.077 INFO:teuthology.orchestra.run.smithi138.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-14T06:26:21.077 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-09-14T06:26:21.078 INFO:teuthology.orchestra.run.smithi138.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 34/35 2024-09-14T06:26:21.160 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 34/35 2024-09-14T06:26:21.160 INFO:teuthology.orchestra.run.smithi138.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 35/35 2024-09-14T06:26:21.564 INFO:teuthology.orchestra.run.smithi094.stdout:Last metadata expiration check: 0:00:39 ago on Sat 14 Sep 2024 06:25:42 AM UTC. 2024-09-14T06:26:21.704 INFO:teuthology.orchestra.run.smithi094.stdout:Dependencies resolved. 2024-09-14T06:26:21.707 INFO:teuthology.orchestra.run.smithi094.stdout:================================================================================ 2024-09-14T06:26:21.707 INFO:teuthology.orchestra.run.smithi094.stdout: Package Arch Version Repository Size 2024-09-14T06:26:21.707 INFO:teuthology.orchestra.run.smithi094.stdout:================================================================================ 2024-09-14T06:26:21.707 INFO:teuthology.orchestra.run.smithi094.stdout:Installing: 2024-09-14T06:26:21.707 INFO:teuthology.orchestra.run.smithi094.stdout: ceph x86_64 2:19.3.0-4775.g7f24e589.el9 ceph 6.6 k 2024-09-14T06:26:21.707 INFO:teuthology.orchestra.run.smithi094.stdout:Installing dependencies: 2024-09-14T06:26:21.707 INFO:teuthology.orchestra.run.smithi094.stdout: ceph-mds x86_64 2:19.3.0-4775.g7f24e589.el9 ceph 2.4 M 2024-09-14T06:26:21.707 INFO:teuthology.orchestra.run.smithi094.stdout: ceph-mgr x86_64 2:19.3.0-4775.g7f24e589.el9 ceph 1.5 M 2024-09-14T06:26:21.707 INFO:teuthology.orchestra.run.smithi094.stdout: ceph-mgr-modules-core noarch 2:19.3.0-4775.g7f24e589.el9 ceph-noarch 291 k 2024-09-14T06:26:21.707 INFO:teuthology.orchestra.run.smithi094.stdout: ceph-mon x86_64 2:19.3.0-4775.g7f24e589.el9 ceph 4.8 M 2024-09-14T06:26:21.707 INFO:teuthology.orchestra.run.smithi094.stdout: ceph-osd x86_64 2:19.3.0-4775.g7f24e589.el9 ceph 17 M 2024-09-14T06:26:21.708 INFO:teuthology.orchestra.run.smithi094.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-09-14T06:26:21.708 INFO:teuthology.orchestra.run.smithi094.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-09-14T06:26:21.708 INFO:teuthology.orchestra.run.smithi094.stdout: luarocks noarch 3.9.2-5.el9 epel 151 k 2024-09-14T06:26:21.708 INFO:teuthology.orchestra.run.smithi094.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-09-14T06:26:21.708 INFO:teuthology.orchestra.run.smithi094.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-09-14T06:26:21.708 INFO:teuthology.orchestra.run.smithi094.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-09-14T06:26:21.708 INFO:teuthology.orchestra.run.smithi094.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-09-14T06:26:21.708 INFO:teuthology.orchestra.run.smithi094.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-09-14T06:26:21.708 INFO:teuthology.orchestra.run.smithi094.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-09-14T06:26:21.708 INFO:teuthology.orchestra.run.smithi094.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-09-14T06:26:21.708 INFO:teuthology.orchestra.run.smithi094.stdout: python3-jaraco-collections 2024-09-14T06:26:21.708 INFO:teuthology.orchestra.run.smithi094.stdout: noarch 3.0.0-8.el9 epel 23 k 2024-09-14T06:26:21.708 INFO:teuthology.orchestra.run.smithi094.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-09-14T06:26:21.709 INFO:teuthology.orchestra.run.smithi094.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-09-14T06:26:21.709 INFO:teuthology.orchestra.run.smithi094.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-09-14T06:26:21.709 INFO:teuthology.orchestra.run.smithi094.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-09-14T06:26:21.709 INFO:teuthology.orchestra.run.smithi094.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-09-14T06:26:21.709 INFO:teuthology.orchestra.run.smithi094.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-09-14T06:26:21.709 INFO:teuthology.orchestra.run.smithi094.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-09-14T06:26:21.709 INFO:teuthology.orchestra.run.smithi094.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-09-14T06:26:21.709 INFO:teuthology.orchestra.run.smithi094.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-09-14T06:26:21.709 INFO:teuthology.orchestra.run.smithi094.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-09-14T06:26:21.709 INFO:teuthology.orchestra.run.smithi094.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-09-14T06:26:21.709 INFO:teuthology.orchestra.run.smithi094.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-09-14T06:26:21.709 INFO:teuthology.orchestra.run.smithi094.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-09-14T06:26:21.709 INFO:teuthology.orchestra.run.smithi094.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-09-14T06:26:21.710 INFO:teuthology.orchestra.run.smithi094.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-09-14T06:26:21.710 INFO:teuthology.orchestra.run.smithi094.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-09-14T06:26:21.710 INFO:teuthology.orchestra.run.smithi094.stdout: python3-urllib3 noarch 1.26.5-6.el9 baseos 215 k 2024-09-14T06:26:21.710 INFO:teuthology.orchestra.run.smithi094.stdout: python3-webob noarch 1.8.8-2.el9 epel 230 k 2024-09-14T06:26:21.710 INFO:teuthology.orchestra.run.smithi094.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-09-14T06:26:21.710 INFO:teuthology.orchestra.run.smithi094.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-09-14T06:26:21.710 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-14T06:26:21.710 INFO:teuthology.orchestra.run.smithi094.stdout:Transaction Summary 2024-09-14T06:26:21.710 INFO:teuthology.orchestra.run.smithi094.stdout:================================================================================ 2024-09-14T06:26:21.710 INFO:teuthology.orchestra.run.smithi094.stdout:Install 37 Packages 2024-09-14T06:26:21.710 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-14T06:26:21.711 INFO:teuthology.orchestra.run.smithi094.stdout:Total download size: 31 M 2024-09-14T06:26:21.711 INFO:teuthology.orchestra.run.smithi094.stdout:Installed size: 109 M 2024-09-14T06:26:21.711 INFO:teuthology.orchestra.run.smithi094.stdout:Downloading Packages: 2024-09-14T06:26:22.401 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 35/35 2024-09-14T06:26:22.401 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : ceph-base-2:19.3.0-4775.g7f24e589.el9.x86_64 1/35 2024-09-14T06:26:22.401 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : ceph-common-2:19.3.0-4775.g7f24e589.el9.x86_64 2/35 2024-09-14T06:26:22.401 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : ceph-radosgw-2:19.3.0-4775.g7f24e589.el9.x86_64 3/35 2024-09-14T06:26:22.401 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : ceph-selinux-2:19.3.0-4775.g7f24e589.el9.x86_64 4/35 2024-09-14T06:26:22.401 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : libcephfs2-2:19.3.0-4775.g7f24e589.el9.x86_64 5/35 2024-09-14T06:26:22.401 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : libradosstriper1-2:19.3.0-4775.g7f24e589.el9.x86_6 6/35 2024-09-14T06:26:22.401 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : librgw2-2:19.3.0-4775.g7f24e589.el9.x86_64 7/35 2024-09-14T06:26:22.401 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-ceph-argparse-2:19.3.0-4775.g7f24e589.el9. 8/35 2024-09-14T06:26:22.402 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-ceph-common-2:19.3.0-4775.g7f24e589.el9.x8 9/35 2024-09-14T06:26:22.402 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-cephfs-2:19.3.0-4775.g7f24e589.el9.x86_64 10/35 2024-09-14T06:26:22.402 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-rados-2:19.3.0-4775.g7f24e589.el9.x86_64 11/35 2024-09-14T06:26:22.402 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-rbd-2:19.3.0-4775.g7f24e589.el9.x86_64 12/35 2024-09-14T06:26:22.402 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-rgw-2:19.3.0-4775.g7f24e589.el9.x86_64 13/35 2024-09-14T06:26:22.402 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 14/35 2024-09-14T06:26:22.402 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : libnbd-1.20.2-2.el9.x86_64 15/35 2024-09-14T06:26:22.402 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 16/35 2024-09-14T06:26:22.402 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 17/35 2024-09-14T06:26:22.402 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 18/35 2024-09-14T06:26:22.402 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 19/35 2024-09-14T06:26:22.402 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 20/35 2024-09-14T06:26:22.402 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 21/35 2024-09-14T06:26:22.402 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 22/35 2024-09-14T06:26:22.402 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 23/35 2024-09-14T06:26:22.403 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : gperftools-libs-2.9.1-3.el9.x86_64 24/35 2024-09-14T06:26:22.403 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 25/35 2024-09-14T06:26:22.403 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 26/35 2024-09-14T06:26:22.403 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 27/35 2024-09-14T06:26:22.404 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 28/35 2024-09-14T06:26:22.404 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 29/35 2024-09-14T06:26:22.404 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : re2-1:20211101-20.el9.x86_64 30/35 2024-09-14T06:26:22.404 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 31/35 2024-09-14T06:26:22.404 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : librados2-2:19.3.0-4775.g7f24e589.el9.x86_64 32/35 2024-09-14T06:26:22.404 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 33/35 2024-09-14T06:26:22.404 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : librbd1-2:19.3.0-4775.g7f24e589.el9.x86_64 34/35 2024-09-14T06:26:22.508 INFO:teuthology.orchestra.run.smithi094.stdout:(1/37): ceph-19.3.0-4775.g7f24e589.el9.x86_64.r 50 kB/s | 6.6 kB 00:00 2024-09-14T06:26:22.670 INFO:teuthology.orchestra.run.smithi094.stdout:(2/37): ceph-mgr-19.3.0-4775.g7f24e589.el9.x86_ 5.2 MB/s | 1.5 MB 00:00 2024-09-14T06:26:22.729 INFO:teuthology.orchestra.run.smithi094.stdout:(3/37): ceph-mds-19.3.0-4775.g7f24e589.el9.x86_ 6.8 MB/s | 2.4 MB 00:00 2024-09-14T06:26:22.829 INFO:teuthology.orchestra.run.smithi094.stdout:(4/37): ceph-mon-19.3.0-4775.g7f24e589.el9.x86_ 15 MB/s | 4.8 MB 00:00 2024-09-14T06:26:22.854 INFO:teuthology.orchestra.run.smithi094.stdout:(5/37): ceph-mgr-modules-core-19.3.0-4775.g7f24 2.3 MB/s | 291 kB 00:00 2024-09-14T06:26:23.255 INFO:teuthology.orchestra.run.smithi094.stdout:(6/37): ceph-osd-19.3.0-4775.g7f24e589.el9.x86_ 29 MB/s | 17 MB 00:00 2024-09-14T06:26:23.465 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 35/35 2024-09-14T06:26:23.465 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-09-14T06:26:23.465 INFO:teuthology.orchestra.run.smithi138.stdout:Upgraded: 2024-09-14T06:26:23.465 INFO:teuthology.orchestra.run.smithi138.stdout: librados2-2:19.3.0-4775.g7f24e589.el9.x86_64 2024-09-14T06:26:23.465 INFO:teuthology.orchestra.run.smithi138.stdout: librbd1-2:19.3.0-4775.g7f24e589.el9.x86_64 2024-09-14T06:26:23.465 INFO:teuthology.orchestra.run.smithi138.stdout:Installed: 2024-09-14T06:26:23.465 INFO:teuthology.orchestra.run.smithi138.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-09-14T06:26:23.465 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-base-2:19.3.0-4775.g7f24e589.el9.x86_64 2024-09-14T06:26:23.465 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-common-2:19.3.0-4775.g7f24e589.el9.x86_64 2024-09-14T06:26:23.466 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-radosgw-2:19.3.0-4775.g7f24e589.el9.x86_64 2024-09-14T06:26:23.466 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-selinux-2:19.3.0-4775.g7f24e589.el9.x86_64 2024-09-14T06:26:23.466 INFO:teuthology.orchestra.run.smithi138.stdout: gperftools-libs-2.9.1-3.el9.x86_64 2024-09-14T06:26:23.466 INFO:teuthology.orchestra.run.smithi138.stdout: libarrow-9.0.0-11.el9.x86_64 2024-09-14T06:26:23.466 INFO:teuthology.orchestra.run.smithi138.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-09-14T06:26:23.466 INFO:teuthology.orchestra.run.smithi138.stdout: libcephfs2-2:19.3.0-4775.g7f24e589.el9.x86_64 2024-09-14T06:26:23.466 INFO:teuthology.orchestra.run.smithi138.stdout: libnbd-1.20.2-2.el9.x86_64 2024-09-14T06:26:23.466 INFO:teuthology.orchestra.run.smithi138.stdout: liboath-2.6.7-2.el9.x86_64 2024-09-14T06:26:23.466 INFO:teuthology.orchestra.run.smithi138.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-09-14T06:26:23.466 INFO:teuthology.orchestra.run.smithi138.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-09-14T06:26:23.466 INFO:teuthology.orchestra.run.smithi138.stdout: libradosstriper1-2:19.3.0-4775.g7f24e589.el9.x86_64 2024-09-14T06:26:23.466 INFO:teuthology.orchestra.run.smithi138.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-09-14T06:26:23.466 INFO:teuthology.orchestra.run.smithi138.stdout: librgw2-2:19.3.0-4775.g7f24e589.el9.x86_64 2024-09-14T06:26:23.466 INFO:teuthology.orchestra.run.smithi138.stdout: libunwind-1.6.2-1.el9.x86_64 2024-09-14T06:26:23.467 INFO:teuthology.orchestra.run.smithi138.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-09-14T06:26:23.467 INFO:teuthology.orchestra.run.smithi138.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-09-14T06:26:23.467 INFO:teuthology.orchestra.run.smithi138.stdout: python3-ceph-argparse-2:19.3.0-4775.g7f24e589.el9.x86_64 2024-09-14T06:26:23.467 INFO:teuthology.orchestra.run.smithi138.stdout: python3-ceph-common-2:19.3.0-4775.g7f24e589.el9.x86_64 2024-09-14T06:26:23.467 INFO:teuthology.orchestra.run.smithi138.stdout: python3-cephfs-2:19.3.0-4775.g7f24e589.el9.x86_64 2024-09-14T06:26:23.467 INFO:teuthology.orchestra.run.smithi138.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-09-14T06:26:23.467 INFO:teuthology.orchestra.run.smithi138.stdout: python3-rados-2:19.3.0-4775.g7f24e589.el9.x86_64 2024-09-14T06:26:23.467 INFO:teuthology.orchestra.run.smithi138.stdout: python3-rbd-2:19.3.0-4775.g7f24e589.el9.x86_64 2024-09-14T06:26:23.467 INFO:teuthology.orchestra.run.smithi138.stdout: python3-rgw-2:19.3.0-4775.g7f24e589.el9.x86_64 2024-09-14T06:26:23.467 INFO:teuthology.orchestra.run.smithi138.stdout: qatlib-24.02.0-1.el9.x86_64 2024-09-14T06:26:23.467 INFO:teuthology.orchestra.run.smithi138.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-09-14T06:26:23.467 INFO:teuthology.orchestra.run.smithi138.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-09-14T06:26:23.467 INFO:teuthology.orchestra.run.smithi138.stdout: re2-1:20211101-20.el9.x86_64 2024-09-14T06:26:23.468 INFO:teuthology.orchestra.run.smithi138.stdout: thrift-0.15.0-2.el9.x86_64 2024-09-14T06:26:23.468 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-09-14T06:26:23.468 INFO:teuthology.orchestra.run.smithi138.stdout:Complete! 2024-09-14T06:26:23.764 INFO:teuthology.orchestra.run.smithi094.stdout:(7/37): python3-cffi-1.14.5-5.el9.x86_64.rpm 271 kB/s | 253 kB 00:00 2024-09-14T06:26:23.803 DEBUG:teuthology.orchestra.run.smithi138:> sudo yum -y install ceph-test 2024-09-14T06:26:23.906 INFO:teuthology.orchestra.run.smithi094.stdout:(8/37): python3-cryptography-36.0.1-4.el9.x86_6 1.2 MB/s | 1.2 MB 00:01 2024-09-14T06:26:23.931 INFO:teuthology.orchestra.run.smithi094.stdout:(9/37): python3-ply-3.11-14.el9.noarch.rpm 158 kB/s | 106 kB 00:00 2024-09-14T06:26:24.374 INFO:teuthology.orchestra.run.smithi138.stdout:Last metadata expiration check: 0:00:28 ago on Sat 14 Sep 2024 06:25:56 AM UTC. 2024-09-14T06:26:24.473 INFO:teuthology.orchestra.run.smithi094.stdout:(10/37): python3-pycparser-2.20-6.el9.noarch.rp 191 kB/s | 135 kB 00:00 2024-09-14T06:26:24.506 INFO:teuthology.orchestra.run.smithi138.stdout:Dependencies resolved. 2024-09-14T06:26:24.507 INFO:teuthology.orchestra.run.smithi138.stdout:================================================================================ 2024-09-14T06:26:24.507 INFO:teuthology.orchestra.run.smithi138.stdout: Package Arch Version Repository Size 2024-09-14T06:26:24.507 INFO:teuthology.orchestra.run.smithi138.stdout:================================================================================ 2024-09-14T06:26:24.507 INFO:teuthology.orchestra.run.smithi138.stdout:Installing: 2024-09-14T06:26:24.507 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-test x86_64 2:19.3.0-4775.g7f24e589.el9 ceph 50 M 2024-09-14T06:26:24.507 INFO:teuthology.orchestra.run.smithi138.stdout:Installing dependencies: 2024-09-14T06:26:24.507 INFO:teuthology.orchestra.run.smithi138.stdout: libcephsqlite x86_64 2:19.3.0-4775.g7f24e589.el9 ceph 162 k 2024-09-14T06:26:24.508 INFO:teuthology.orchestra.run.smithi138.stdout: socat x86_64 1.7.4.1-6.el9 appstream 304 k 2024-09-14T06:26:24.508 INFO:teuthology.orchestra.run.smithi138.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-09-14T06:26:24.508 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-09-14T06:26:24.508 INFO:teuthology.orchestra.run.smithi138.stdout:Transaction Summary 2024-09-14T06:26:24.508 INFO:teuthology.orchestra.run.smithi138.stdout:================================================================================ 2024-09-14T06:26:24.508 INFO:teuthology.orchestra.run.smithi138.stdout:Install 4 Packages 2024-09-14T06:26:24.508 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-09-14T06:26:24.508 INFO:teuthology.orchestra.run.smithi138.stdout:Total download size: 51 M 2024-09-14T06:26:24.509 INFO:teuthology.orchestra.run.smithi138.stdout:Installed size: 217 M 2024-09-14T06:26:24.509 INFO:teuthology.orchestra.run.smithi138.stdout:Downloading Packages: 2024-09-14T06:26:24.582 INFO:teuthology.orchestra.run.smithi094.stdout:(11/37): python3-requests-2.25.1-8.el9.noarch.r 185 kB/s | 125 kB 00:00 2024-09-14T06:26:24.607 INFO:teuthology.orchestra.run.smithi094.stdout:(12/37): lua-5.4.4-4.el9.x86_64.rpm 1.4 MB/s | 188 kB 00:00 2024-09-14T06:26:24.649 INFO:teuthology.orchestra.run.smithi094.stdout:(13/37): python3-markupsafe-1.1.1-12.el9.x86_64 835 kB/s | 35 kB 00:00 2024-09-14T06:26:24.691 INFO:teuthology.orchestra.run.smithi094.stdout:(14/37): python3-urllib3-1.26.5-6.el9.noarch.rp 283 kB/s | 215 kB 00:00 2024-09-14T06:26:24.716 INFO:teuthology.orchestra.run.smithi094.stdout:(15/37): python3-packaging-20.9-5.el9.noarch.rp 1.1 MB/s | 77 kB 00:00 2024-09-14T06:26:24.739 INFO:teuthology.orchestra.run.smithi138.stdout:(1/4): libcephsqlite-19.3.0-4775.g7f24e589.el9. 773 kB/s | 162 kB 00:00 2024-09-14T06:26:24.741 INFO:teuthology.orchestra.run.smithi094.stdout:(16/37): python3-mako-1.1.4-6.el9.noarch.rpm 1.1 MB/s | 172 kB 00:00 2024-09-14T06:26:24.766 INFO:teuthology.orchestra.run.smithi094.stdout:(17/37): python3-toml-0.10.2-6.el9.noarch.rpm 829 kB/s | 42 kB 00:00 2024-09-14T06:26:24.800 INFO:teuthology.orchestra.run.smithi094.stdout:(18/37): python3-pytz-2021.1-5.el9.noarch.rpm 469 kB/s | 51 kB 00:00 2024-09-14T06:26:24.834 INFO:teuthology.orchestra.run.smithi094.stdout:(19/37): luarocks-3.9.2-5.el9.noarch.rpm 2.2 MB/s | 151 kB 00:00 2024-09-14T06:26:24.859 INFO:teuthology.orchestra.run.smithi094.stdout:(20/37): lua-devel-5.4.4-4.el9.x86_64.rpm 190 kB/s | 22 kB 00:00 2024-09-14T06:26:24.884 INFO:teuthology.orchestra.run.smithi094.stdout:(21/37): python3-bcrypt-3.2.2-1.el9.x86_64.rpm 529 kB/s | 43 kB 00:00 2024-09-14T06:26:24.918 INFO:teuthology.orchestra.run.smithi094.stdout:(22/37): python3-jaraco-8.2.1-3.el9.noarch.rpm 414 kB/s | 11 kB 00:00 2024-09-14T06:26:24.943 INFO:teuthology.orchestra.run.smithi094.stdout:(23/37): python3-cheroot-8.6.0-4.el9.noarch.rpm 1.6 MB/s | 172 kB 00:00 2024-09-14T06:26:24.969 INFO:teuthology.orchestra.run.smithi094.stdout:(24/37): python3-jaraco-classes-3.2.1-5.el9.noa 364 kB/s | 18 kB 00:00 2024-09-14T06:26:24.994 INFO:teuthology.orchestra.run.smithi094.stdout:(25/37): python3-jaraco-collections-3.0.0-8.el9 460 kB/s | 23 kB 00:00 2024-09-14T06:26:25.019 INFO:teuthology.orchestra.run.smithi094.stdout:(26/37): python3-jaraco-functools-3.5.0-2.el9.n 388 kB/s | 19 kB 00:00 2024-09-14T06:26:25.044 INFO:teuthology.orchestra.run.smithi094.stdout:(27/37): python3-jaraco-text-3.2.0-6.el9.noarch 392 kB/s | 20 kB 00:00 2024-09-14T06:26:25.070 INFO:teuthology.orchestra.run.smithi094.stdout:(28/37): python3-logutils-0.3.5-21.el9.noarch.r 922 kB/s | 46 kB 00:00 2024-09-14T06:26:25.095 INFO:teuthology.orchestra.run.smithi094.stdout:(29/37): python3-more-itertools-8.12.0-2.el9.no 1.5 MB/s | 79 kB 00:00 2024-09-14T06:26:25.114 INFO:teuthology.orchestra.run.smithi138.stdout:(2/4): xmlstarlet-1.6.1-20.el9.x86_64.rpm 170 kB/s | 64 kB 00:00 2024-09-14T06:26:25.120 INFO:teuthology.orchestra.run.smithi094.stdout:(30/37): python3-portend-3.1.0-2.el9.noarch.rpm 658 kB/s | 16 kB 00:00 2024-09-14T06:26:25.146 INFO:teuthology.orchestra.run.smithi094.stdout:(31/37): python3-cherrypy-18.6.1-2.el9.noarch.r 1.2 MB/s | 358 kB 00:00 2024-09-14T06:26:25.164 INFO:teuthology.orchestra.run.smithi138.stdout:(3/4): socat-1.7.4.1-6.el9.x86_64.rpm 479 kB/s | 304 kB 00:00 2024-09-14T06:26:25.171 INFO:teuthology.orchestra.run.smithi094.stdout:(32/37): python3-pyOpenSSL-21.0.0-1.el9.noarch. 1.7 MB/s | 90 kB 00:00 2024-09-14T06:26:25.196 INFO:teuthology.orchestra.run.smithi094.stdout:(33/37): python3-tempora-5.0.0-2.el9.noarch.rpm 714 kB/s | 36 kB 00:00 2024-09-14T06:26:25.221 INFO:teuthology.orchestra.run.smithi094.stdout:(34/37): python3-pecan-1.4.2-3.el9.noarch.rpm 1.8 MB/s | 272 kB 00:00 2024-09-14T06:26:25.247 INFO:teuthology.orchestra.run.smithi094.stdout:(35/37): python3-zc-lockfile-2.0-10.el9.noarch. 797 kB/s | 20 kB 00:00 2024-09-14T06:26:25.272 INFO:teuthology.orchestra.run.smithi094.stdout:(36/37): python3-webob-1.8.8-2.el9.noarch.rpm 3.0 MB/s | 230 kB 00:00 2024-09-14T06:26:25.306 INFO:teuthology.orchestra.run.smithi094.stdout:(37/37): python3-werkzeug-2.0.3-3.el9.1.noarch. 3.8 MB/s | 427 kB 00:00 2024-09-14T06:26:25.311 INFO:teuthology.orchestra.run.smithi094.stdout:-------------------------------------------------------------------------------- 2024-09-14T06:26:25.311 INFO:teuthology.orchestra.run.smithi094.stdout:Total 8.5 MB/s | 31 MB 00:03 2024-09-14T06:26:25.477 INFO:teuthology.orchestra.run.smithi094.stdout:Running transaction check 2024-09-14T06:26:25.519 INFO:teuthology.orchestra.run.smithi094.stdout:Transaction check succeeded. 2024-09-14T06:26:25.519 INFO:teuthology.orchestra.run.smithi094.stdout:Running transaction test 2024-09-14T06:26:26.004 INFO:teuthology.orchestra.run.smithi094.stdout:Transaction test succeeded. 2024-09-14T06:26:26.004 INFO:teuthology.orchestra.run.smithi094.stdout:Running transaction 2024-09-14T06:26:26.098 INFO:teuthology.orchestra.run.smithi138.stdout:(4/4): ceph-test-19.3.0-4775.g7f24e589.el9.x86_ 32 MB/s | 50 MB 00:01 2024-09-14T06:26:26.100 INFO:teuthology.orchestra.run.smithi138.stdout:-------------------------------------------------------------------------------- 2024-09-14T06:26:26.100 INFO:teuthology.orchestra.run.smithi138.stdout:Total 32 MB/s | 51 MB 00:01 2024-09-14T06:26:26.128 INFO:teuthology.orchestra.run.smithi138.stdout:Running transaction check 2024-09-14T06:26:26.148 INFO:teuthology.orchestra.run.smithi138.stdout:Transaction check succeeded. 2024-09-14T06:26:26.149 INFO:teuthology.orchestra.run.smithi138.stdout:Running transaction test 2024-09-14T06:26:26.547 INFO:teuthology.orchestra.run.smithi138.stdout:Transaction test succeeded. 2024-09-14T06:26:26.547 INFO:teuthology.orchestra.run.smithi138.stdout:Running transaction 2024-09-14T06:26:27.100 INFO:teuthology.orchestra.run.smithi138.stdout: Preparing : 1/1 2024-09-14T06:26:27.191 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 1/4 2024-09-14T06:26:27.231 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : socat-1.7.4.1-6.el9.x86_64 2/4 2024-09-14T06:26:27.259 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : libcephsqlite-2:19.3.0-4775.g7f24e589.el9.x86_64 3/4 2024-09-14T06:26:27.412 INFO:teuthology.orchestra.run.smithi094.stdout: Preparing : 1/1 2024-09-14T06:26:27.475 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 1/37 2024-09-14T06:26:27.517 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 2/37 2024-09-14T06:26:27.545 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : lua-5.4.4-4.el9.x86_64 3/37 2024-09-14T06:26:27.617 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 4/37 2024-09-14T06:26:27.678 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : luarocks-3.9.2-5.el9.noarch 5/37 2024-09-14T06:26:27.719 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 6/37 2024-09-14T06:26:27.819 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 7/37 2024-09-14T06:26:27.905 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 8/37 2024-09-14T06:26:27.945 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-webob-1.8.8-2.el9.noarch 9/37 2024-09-14T06:26:27.981 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 10/37 2024-09-14T06:26:28.011 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 11/37 2024-09-14T06:26:28.045 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 12/37 2024-09-14T06:26:28.076 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 13/37 2024-09-14T06:26:28.125 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 14/37 2024-09-14T06:26:28.160 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 15/37 2024-09-14T06:26:28.195 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 16/37 2024-09-14T06:26:28.251 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 17/37 2024-09-14T06:26:28.292 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-packaging-20.9-5.el9.noarch 18/37 2024-09-14T06:26:28.372 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 19/37 2024-09-14T06:26:28.509 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 20/37 2024-09-14T06:26:28.592 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 21/37 2024-09-14T06:26:28.645 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-urllib3-1.26.5-6.el9.noarch 22/37 2024-09-14T06:26:28.684 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 23/37 2024-09-14T06:26:28.732 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-ply-3.11-14.el9.noarch 24/37 2024-09-14T06:26:28.796 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 25/37 2024-09-14T06:26:28.989 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 26/37 2024-09-14T06:26:29.038 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 27/37 2024-09-14T06:26:29.108 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 28/37 2024-09-14T06:26:29.196 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 29/37 2024-09-14T06:26:29.229 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 30/37 2024-09-14T06:26:29.369 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 31/37 2024-09-14T06:26:29.536 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : ceph-mgr-modules-core-2:19.3.0-4775.g7f24e589.el9. 32/37 2024-09-14T06:26:29.565 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : ceph-mgr-2:19.3.0-4775.g7f24e589.el9.x86_64 33/37 2024-09-14T06:26:29.592 INFO:teuthology.orchestra.run.smithi094.stdout: Running scriptlet: ceph-mgr-2:19.3.0-4775.g7f24e589.el9.x86_64 33/37 2024-09-14T06:26:29.592 INFO:teuthology.orchestra.run.smithi094.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-14T06:26:29.592 INFO:teuthology.orchestra.run.smithi094.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-09-14T06:26:29.592 INFO:teuthology.orchestra.run.smithi094.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-14T06:26:29.592 INFO:teuthology.orchestra.run.smithi094.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-14T06:26:29.592 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-14T06:26:31.049 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : ceph-osd-2:19.3.0-4775.g7f24e589.el9.x86_64 34/37 2024-09-14T06:26:31.076 INFO:teuthology.orchestra.run.smithi094.stdout: Running scriptlet: ceph-osd-2:19.3.0-4775.g7f24e589.el9.x86_64 34/37 2024-09-14T06:26:31.076 INFO:teuthology.orchestra.run.smithi094.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-14T06:26:31.076 INFO:teuthology.orchestra.run.smithi094.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-09-14T06:26:31.077 INFO:teuthology.orchestra.run.smithi094.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-14T06:26:31.077 INFO:teuthology.orchestra.run.smithi094.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-14T06:26:31.077 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-14T06:26:31.525 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : ceph-mon-2:19.3.0-4775.g7f24e589.el9.x86_64 35/37 2024-09-14T06:26:31.549 INFO:teuthology.orchestra.run.smithi094.stdout: Running scriptlet: ceph-mon-2:19.3.0-4775.g7f24e589.el9.x86_64 35/37 2024-09-14T06:26:31.549 INFO:teuthology.orchestra.run.smithi094.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-14T06:26:31.549 INFO:teuthology.orchestra.run.smithi094.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-09-14T06:26:31.549 INFO:teuthology.orchestra.run.smithi094.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-14T06:26:31.549 INFO:teuthology.orchestra.run.smithi094.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-14T06:26:31.549 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-14T06:26:31.776 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : ceph-mds-2:19.3.0-4775.g7f24e589.el9.x86_64 36/37 2024-09-14T06:26:31.799 INFO:teuthology.orchestra.run.smithi094.stdout: Running scriptlet: ceph-mds-2:19.3.0-4775.g7f24e589.el9.x86_64 36/37 2024-09-14T06:26:31.800 INFO:teuthology.orchestra.run.smithi094.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-14T06:26:31.800 INFO:teuthology.orchestra.run.smithi094.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-09-14T06:26:31.800 INFO:teuthology.orchestra.run.smithi094.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-14T06:26:31.800 INFO:teuthology.orchestra.run.smithi094.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-14T06:26:31.800 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-14T06:26:31.872 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : ceph-2:19.3.0-4775.g7f24e589.el9.x86_64 37/37 2024-09-14T06:26:32.029 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: libcephsqlite-2:19.3.0-4775.g7f24e589.el9.x86_64 3/4 2024-09-14T06:26:32.109 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : ceph-test-2:19.3.0-4775.g7f24e589.el9.x86_64 4/4 2024-09-14T06:26:33.776 INFO:teuthology.orchestra.run.smithi094.stdout: Running scriptlet: ceph-2:19.3.0-4775.g7f24e589.el9.x86_64 37/37 2024-09-14T06:26:33.776 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : ceph-2:19.3.0-4775.g7f24e589.el9.x86_64 1/37 2024-09-14T06:26:33.776 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : ceph-mds-2:19.3.0-4775.g7f24e589.el9.x86_64 2/37 2024-09-14T06:26:33.776 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : ceph-mgr-2:19.3.0-4775.g7f24e589.el9.x86_64 3/37 2024-09-14T06:26:33.776 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : ceph-mon-2:19.3.0-4775.g7f24e589.el9.x86_64 4/37 2024-09-14T06:26:33.776 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : ceph-osd-2:19.3.0-4775.g7f24e589.el9.x86_64 5/37 2024-09-14T06:26:33.776 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : ceph-mgr-modules-core-2:19.3.0-4775.g7f24e589.el9. 6/37 2024-09-14T06:26:33.776 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 7/37 2024-09-14T06:26:33.777 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 8/37 2024-09-14T06:26:33.777 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-ply-3.11-14.el9.noarch 9/37 2024-09-14T06:26:33.777 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 10/37 2024-09-14T06:26:33.777 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 11/37 2024-09-14T06:26:33.777 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-urllib3-1.26.5-6.el9.noarch 12/37 2024-09-14T06:26:33.777 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : lua-5.4.4-4.el9.x86_64 13/37 2024-09-14T06:26:33.777 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 14/37 2024-09-14T06:26:33.777 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 15/37 2024-09-14T06:26:33.777 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 16/37 2024-09-14T06:26:33.777 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 17/37 2024-09-14T06:26:33.777 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 18/37 2024-09-14T06:26:33.777 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 19/37 2024-09-14T06:26:33.777 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : luarocks-3.9.2-5.el9.noarch 20/37 2024-09-14T06:26:33.777 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 21/37 2024-09-14T06:26:33.778 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 22/37 2024-09-14T06:26:33.778 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 23/37 2024-09-14T06:26:33.778 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 24/37 2024-09-14T06:26:33.778 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 25/37 2024-09-14T06:26:33.778 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 26/37 2024-09-14T06:26:33.778 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 27/37 2024-09-14T06:26:33.779 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 28/37 2024-09-14T06:26:33.779 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 29/37 2024-09-14T06:26:33.779 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 30/37 2024-09-14T06:26:33.779 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 31/37 2024-09-14T06:26:33.779 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 32/37 2024-09-14T06:26:33.779 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 33/37 2024-09-14T06:26:33.779 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 34/37 2024-09-14T06:26:33.779 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-webob-1.8.8-2.el9.noarch 35/37 2024-09-14T06:26:33.780 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 36/37 2024-09-14T06:26:34.537 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: ceph-test-2:19.3.0-4775.g7f24e589.el9.x86_64 4/4 2024-09-14T06:26:34.537 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : ceph-test-2:19.3.0-4775.g7f24e589.el9.x86_64 1/4 2024-09-14T06:26:34.537 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : libcephsqlite-2:19.3.0-4775.g7f24e589.el9.x86_64 2/4 2024-09-14T06:26:34.537 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : socat-1.7.4.1-6.el9.x86_64 3/4 2024-09-14T06:26:34.753 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 37/37 2024-09-14T06:26:34.753 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-14T06:26:34.753 INFO:teuthology.orchestra.run.smithi094.stdout:Installed: 2024-09-14T06:26:34.753 INFO:teuthology.orchestra.run.smithi094.stdout: ceph-2:19.3.0-4775.g7f24e589.el9.x86_64 2024-09-14T06:26:34.753 INFO:teuthology.orchestra.run.smithi094.stdout: ceph-mds-2:19.3.0-4775.g7f24e589.el9.x86_64 2024-09-14T06:26:34.753 INFO:teuthology.orchestra.run.smithi094.stdout: ceph-mgr-2:19.3.0-4775.g7f24e589.el9.x86_64 2024-09-14T06:26:34.753 INFO:teuthology.orchestra.run.smithi094.stdout: ceph-mgr-modules-core-2:19.3.0-4775.g7f24e589.el9.noarch 2024-09-14T06:26:34.754 INFO:teuthology.orchestra.run.smithi094.stdout: ceph-mon-2:19.3.0-4775.g7f24e589.el9.x86_64 2024-09-14T06:26:34.754 INFO:teuthology.orchestra.run.smithi094.stdout: ceph-osd-2:19.3.0-4775.g7f24e589.el9.x86_64 2024-09-14T06:26:34.754 INFO:teuthology.orchestra.run.smithi094.stdout: lua-5.4.4-4.el9.x86_64 2024-09-14T06:26:34.754 INFO:teuthology.orchestra.run.smithi094.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-09-14T06:26:34.754 INFO:teuthology.orchestra.run.smithi094.stdout: luarocks-3.9.2-5.el9.noarch 2024-09-14T06:26:34.754 INFO:teuthology.orchestra.run.smithi094.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-09-14T06:26:34.754 INFO:teuthology.orchestra.run.smithi094.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-09-14T06:26:34.754 INFO:teuthology.orchestra.run.smithi094.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-09-14T06:26:34.754 INFO:teuthology.orchestra.run.smithi094.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-09-14T06:26:34.754 INFO:teuthology.orchestra.run.smithi094.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-09-14T06:26:34.754 INFO:teuthology.orchestra.run.smithi094.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-09-14T06:26:34.754 INFO:teuthology.orchestra.run.smithi094.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-09-14T06:26:34.754 INFO:teuthology.orchestra.run.smithi094.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-09-14T06:26:34.754 INFO:teuthology.orchestra.run.smithi094.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-09-14T06:26:34.755 INFO:teuthology.orchestra.run.smithi094.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-09-14T06:26:34.755 INFO:teuthology.orchestra.run.smithi094.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-09-14T06:26:34.755 INFO:teuthology.orchestra.run.smithi094.stdout: python3-mako-1.1.4-6.el9.noarch 2024-09-14T06:26:34.755 INFO:teuthology.orchestra.run.smithi094.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-09-14T06:26:34.755 INFO:teuthology.orchestra.run.smithi094.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-09-14T06:26:34.755 INFO:teuthology.orchestra.run.smithi094.stdout: python3-packaging-20.9-5.el9.noarch 2024-09-14T06:26:34.755 INFO:teuthology.orchestra.run.smithi094.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-09-14T06:26:34.755 INFO:teuthology.orchestra.run.smithi094.stdout: python3-ply-3.11-14.el9.noarch 2024-09-14T06:26:34.755 INFO:teuthology.orchestra.run.smithi094.stdout: python3-portend-3.1.0-2.el9.noarch 2024-09-14T06:26:34.755 INFO:teuthology.orchestra.run.smithi094.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-09-14T06:26:34.755 INFO:teuthology.orchestra.run.smithi094.stdout: python3-pycparser-2.20-6.el9.noarch 2024-09-14T06:26:34.755 INFO:teuthology.orchestra.run.smithi094.stdout: python3-pytz-2021.1-5.el9.noarch 2024-09-14T06:26:34.755 INFO:teuthology.orchestra.run.smithi094.stdout: python3-requests-2.25.1-8.el9.noarch 2024-09-14T06:26:34.756 INFO:teuthology.orchestra.run.smithi094.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-09-14T06:26:34.756 INFO:teuthology.orchestra.run.smithi094.stdout: python3-toml-0.10.2-6.el9.noarch 2024-09-14T06:26:34.756 INFO:teuthology.orchestra.run.smithi094.stdout: python3-urllib3-1.26.5-6.el9.noarch 2024-09-14T06:26:34.756 INFO:teuthology.orchestra.run.smithi094.stdout: python3-webob-1.8.8-2.el9.noarch 2024-09-14T06:26:34.756 INFO:teuthology.orchestra.run.smithi094.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-09-14T06:26:34.756 INFO:teuthology.orchestra.run.smithi094.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-09-14T06:26:34.756 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-14T06:26:34.756 INFO:teuthology.orchestra.run.smithi094.stdout:Complete! 2024-09-14T06:26:34.898 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 4/4 2024-09-14T06:26:34.898 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-09-14T06:26:34.898 INFO:teuthology.orchestra.run.smithi138.stdout:Installed: 2024-09-14T06:26:34.899 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-test-2:19.3.0-4775.g7f24e589.el9.x86_64 2024-09-14T06:26:34.899 INFO:teuthology.orchestra.run.smithi138.stdout: libcephsqlite-2:19.3.0-4775.g7f24e589.el9.x86_64 2024-09-14T06:26:34.899 INFO:teuthology.orchestra.run.smithi138.stdout: socat-1.7.4.1-6.el9.x86_64 2024-09-14T06:26:34.899 INFO:teuthology.orchestra.run.smithi138.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-09-14T06:26:34.899 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-09-14T06:26:34.899 INFO:teuthology.orchestra.run.smithi138.stdout:Complete! 2024-09-14T06:26:35.133 DEBUG:teuthology.orchestra.run.smithi094:> sudo yum -y install ceph-base 2024-09-14T06:26:35.140 DEBUG:teuthology.orchestra.run.smithi138:> sudo yum -y install ceph 2024-09-14T06:26:35.697 INFO:teuthology.orchestra.run.smithi094.stdout:Last metadata expiration check: 0:00:53 ago on Sat 14 Sep 2024 06:25:42 AM UTC. 2024-09-14T06:26:35.710 INFO:teuthology.orchestra.run.smithi138.stdout:Last metadata expiration check: 0:00:39 ago on Sat 14 Sep 2024 06:25:56 AM UTC. 2024-09-14T06:26:35.787 INFO:teuthology.orchestra.run.smithi094.stdout:Package ceph-base-2:19.3.0-4775.g7f24e589.el9.x86_64 is already installed. 2024-09-14T06:26:35.829 INFO:teuthology.orchestra.run.smithi094.stdout:Dependencies resolved. 2024-09-14T06:26:35.830 INFO:teuthology.orchestra.run.smithi094.stdout:Nothing to do. 2024-09-14T06:26:35.830 INFO:teuthology.orchestra.run.smithi094.stdout:Complete! 2024-09-14T06:26:35.864 INFO:teuthology.orchestra.run.smithi138.stdout:Dependencies resolved. 2024-09-14T06:26:35.867 INFO:teuthology.orchestra.run.smithi138.stdout:================================================================================ 2024-09-14T06:26:35.867 INFO:teuthology.orchestra.run.smithi138.stdout: Package Arch Version Repository Size 2024-09-14T06:26:35.867 INFO:teuthology.orchestra.run.smithi138.stdout:================================================================================ 2024-09-14T06:26:35.867 INFO:teuthology.orchestra.run.smithi138.stdout:Installing: 2024-09-14T06:26:35.867 INFO:teuthology.orchestra.run.smithi138.stdout: ceph x86_64 2:19.3.0-4775.g7f24e589.el9 ceph 6.6 k 2024-09-14T06:26:35.867 INFO:teuthology.orchestra.run.smithi138.stdout:Installing dependencies: 2024-09-14T06:26:35.867 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-mds x86_64 2:19.3.0-4775.g7f24e589.el9 ceph 2.4 M 2024-09-14T06:26:35.867 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-mgr x86_64 2:19.3.0-4775.g7f24e589.el9 ceph 1.5 M 2024-09-14T06:26:35.867 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-mgr-modules-core noarch 2:19.3.0-4775.g7f24e589.el9 ceph-noarch 291 k 2024-09-14T06:26:35.867 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-mon x86_64 2:19.3.0-4775.g7f24e589.el9 ceph 4.8 M 2024-09-14T06:26:35.867 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-osd x86_64 2:19.3.0-4775.g7f24e589.el9 ceph 17 M 2024-09-14T06:26:35.868 INFO:teuthology.orchestra.run.smithi138.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-09-14T06:26:35.868 INFO:teuthology.orchestra.run.smithi138.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-09-14T06:26:35.868 INFO:teuthology.orchestra.run.smithi138.stdout: luarocks noarch 3.9.2-5.el9 epel 151 k 2024-09-14T06:26:35.868 INFO:teuthology.orchestra.run.smithi138.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-09-14T06:26:35.868 INFO:teuthology.orchestra.run.smithi138.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-09-14T06:26:35.868 INFO:teuthology.orchestra.run.smithi138.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-09-14T06:26:35.868 INFO:teuthology.orchestra.run.smithi138.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-09-14T06:26:35.868 INFO:teuthology.orchestra.run.smithi138.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-09-14T06:26:35.868 INFO:teuthology.orchestra.run.smithi138.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-09-14T06:26:35.868 INFO:teuthology.orchestra.run.smithi138.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-09-14T06:26:35.868 INFO:teuthology.orchestra.run.smithi138.stdout: python3-jaraco-collections 2024-09-14T06:26:35.868 INFO:teuthology.orchestra.run.smithi138.stdout: noarch 3.0.0-8.el9 epel 23 k 2024-09-14T06:26:35.868 INFO:teuthology.orchestra.run.smithi138.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-09-14T06:26:35.869 INFO:teuthology.orchestra.run.smithi138.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-09-14T06:26:35.869 INFO:teuthology.orchestra.run.smithi138.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-09-14T06:26:35.869 INFO:teuthology.orchestra.run.smithi138.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-09-14T06:26:35.869 INFO:teuthology.orchestra.run.smithi138.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-09-14T06:26:35.869 INFO:teuthology.orchestra.run.smithi138.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-09-14T06:26:35.869 INFO:teuthology.orchestra.run.smithi138.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-09-14T06:26:35.869 INFO:teuthology.orchestra.run.smithi138.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-09-14T06:26:35.869 INFO:teuthology.orchestra.run.smithi138.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-09-14T06:26:35.869 INFO:teuthology.orchestra.run.smithi138.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-09-14T06:26:35.869 INFO:teuthology.orchestra.run.smithi138.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-09-14T06:26:35.869 INFO:teuthology.orchestra.run.smithi138.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-09-14T06:26:35.869 INFO:teuthology.orchestra.run.smithi138.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-09-14T06:26:35.869 INFO:teuthology.orchestra.run.smithi138.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-09-14T06:26:35.870 INFO:teuthology.orchestra.run.smithi138.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-09-14T06:26:35.870 INFO:teuthology.orchestra.run.smithi138.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-09-14T06:26:35.870 INFO:teuthology.orchestra.run.smithi138.stdout: python3-urllib3 noarch 1.26.5-6.el9 baseos 215 k 2024-09-14T06:26:35.870 INFO:teuthology.orchestra.run.smithi138.stdout: python3-webob noarch 1.8.8-2.el9 epel 230 k 2024-09-14T06:26:35.870 INFO:teuthology.orchestra.run.smithi138.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-09-14T06:26:35.870 INFO:teuthology.orchestra.run.smithi138.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-09-14T06:26:35.870 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-09-14T06:26:35.870 INFO:teuthology.orchestra.run.smithi138.stdout:Transaction Summary 2024-09-14T06:26:35.870 INFO:teuthology.orchestra.run.smithi138.stdout:================================================================================ 2024-09-14T06:26:35.870 INFO:teuthology.orchestra.run.smithi138.stdout:Install 37 Packages 2024-09-14T06:26:35.870 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-09-14T06:26:35.871 INFO:teuthology.orchestra.run.smithi138.stdout:Total download size: 31 M 2024-09-14T06:26:35.871 INFO:teuthology.orchestra.run.smithi138.stdout:Installed size: 109 M 2024-09-14T06:26:35.871 INFO:teuthology.orchestra.run.smithi138.stdout:Downloading Packages: 2024-09-14T06:26:35.888 DEBUG:teuthology.orchestra.run.smithi094:> sudo yum -y install cephadm 2024-09-14T06:26:36.246 INFO:teuthology.orchestra.run.smithi138.stdout:(1/37): ceph-19.3.0-4775.g7f24e589.el9.x86_64.r 46 kB/s | 6.6 kB 00:00 2024-09-14T06:26:36.404 INFO:teuthology.orchestra.run.smithi138.stdout:(2/37): ceph-mgr-19.3.0-4775.g7f24e589.el9.x86_ 5.1 MB/s | 1.5 MB 00:00 2024-09-14T06:26:36.450 INFO:teuthology.orchestra.run.smithi094.stdout:Last metadata expiration check: 0:00:54 ago on Sat 14 Sep 2024 06:25:42 AM UTC. 2024-09-14T06:26:36.454 INFO:teuthology.orchestra.run.smithi138.stdout:(3/37): ceph-mds-19.3.0-4775.g7f24e589.el9.x86_ 6.8 MB/s | 2.4 MB 00:00 2024-09-14T06:26:36.530 INFO:teuthology.orchestra.run.smithi138.stdout:(4/37): ceph-mgr-modules-core-19.3.0-4775.g7f24 3.8 MB/s | 291 kB 00:00 2024-09-14T06:26:36.582 INFO:teuthology.orchestra.run.smithi094.stdout:Dependencies resolved. 2024-09-14T06:26:36.582 INFO:teuthology.orchestra.run.smithi094.stdout:================================================================================ 2024-09-14T06:26:36.582 INFO:teuthology.orchestra.run.smithi094.stdout: Package Arch Version Repository Size 2024-09-14T06:26:36.582 INFO:teuthology.orchestra.run.smithi094.stdout:================================================================================ 2024-09-14T06:26:36.582 INFO:teuthology.orchestra.run.smithi094.stdout:Installing: 2024-09-14T06:26:36.583 INFO:teuthology.orchestra.run.smithi094.stdout: cephadm noarch 2:19.3.0-4775.g7f24e589.el9 ceph-noarch 783 k 2024-09-14T06:26:36.583 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-14T06:26:36.583 INFO:teuthology.orchestra.run.smithi094.stdout:Transaction Summary 2024-09-14T06:26:36.583 INFO:teuthology.orchestra.run.smithi094.stdout:================================================================================ 2024-09-14T06:26:36.583 INFO:teuthology.orchestra.run.smithi094.stdout:Install 1 Package 2024-09-14T06:26:36.583 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-14T06:26:36.583 INFO:teuthology.orchestra.run.smithi094.stdout:Total download size: 783 k 2024-09-14T06:26:36.584 INFO:teuthology.orchestra.run.smithi094.stdout:Installed size: 789 k 2024-09-14T06:26:36.584 INFO:teuthology.orchestra.run.smithi094.stdout:Downloading Packages: 2024-09-14T06:26:36.984 INFO:teuthology.orchestra.run.smithi094.stdout:cephadm-19.3.0-4775.g7f24e589.el9.noarch.rpm 1.9 MB/s | 783 kB 00:00 2024-09-14T06:26:36.985 INFO:teuthology.orchestra.run.smithi094.stdout:-------------------------------------------------------------------------------- 2024-09-14T06:26:36.985 INFO:teuthology.orchestra.run.smithi094.stdout:Total 1.9 MB/s | 783 kB 00:00 2024-09-14T06:26:36.986 INFO:teuthology.orchestra.run.smithi094.stdout:Running transaction check 2024-09-14T06:26:36.991 INFO:teuthology.orchestra.run.smithi094.stdout:Transaction check succeeded. 2024-09-14T06:26:36.991 INFO:teuthology.orchestra.run.smithi094.stdout:Running transaction test 2024-09-14T06:26:36.997 INFO:teuthology.orchestra.run.smithi094.stdout:Transaction test succeeded. 2024-09-14T06:26:36.998 INFO:teuthology.orchestra.run.smithi138.stdout:(5/37): ceph-osd-19.3.0-4775.g7f24e589.el9.x86_ 29 MB/s | 17 MB 00:00 2024-09-14T06:26:36.998 INFO:teuthology.orchestra.run.smithi094.stdout:Running transaction 2024-09-14T06:26:37.071 INFO:teuthology.orchestra.run.smithi094.stdout: Preparing : 1/1 2024-09-14T06:26:37.081 INFO:teuthology.orchestra.run.smithi138.stdout:(6/37): ceph-mon-19.3.0-4775.g7f24e589.el9.x86_ 5.7 MB/s | 4.8 MB 00:00 2024-09-14T06:26:37.473 INFO:teuthology.orchestra.run.smithi138.stdout:(7/37): python3-cffi-1.14.5-5.el9.x86_64.rpm 269 kB/s | 253 kB 00:00 2024-09-14T06:26:37.535 INFO:teuthology.orchestra.run.smithi094.stdout: Running scriptlet: cephadm-2:19.3.0-4775.g7f24e589.el9.noarch 1/1 2024-09-14T06:26:37.630 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : cephadm-2:19.3.0-4775.g7f24e589.el9.noarch 1/1 2024-09-14T06:26:37.773 INFO:teuthology.orchestra.run.smithi138.stdout:(8/37): python3-ply-3.11-14.el9.noarch.rpm 154 kB/s | 106 kB 00:00 2024-09-14T06:26:37.976 INFO:teuthology.orchestra.run.smithi094.stdout: Running scriptlet: cephadm-2:19.3.0-4775.g7f24e589.el9.noarch 1/1 2024-09-14T06:26:38.015 INFO:teuthology.orchestra.run.smithi138.stdout:(9/37): python3-cryptography-36.0.1-4.el9.x86_6 1.2 MB/s | 1.2 MB 00:01 2024-09-14T06:26:38.182 INFO:teuthology.orchestra.run.smithi138.stdout:(10/37): python3-pycparser-2.20-6.el9.noarch.rp 191 kB/s | 135 kB 00:00 2024-09-14T06:26:38.230 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : cephadm-2:19.3.0-4775.g7f24e589.el9.noarch 1/1 2024-09-14T06:26:38.231 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-14T06:26:38.231 INFO:teuthology.orchestra.run.smithi094.stdout:Installed: 2024-09-14T06:26:38.231 INFO:teuthology.orchestra.run.smithi094.stdout: cephadm-2:19.3.0-4775.g7f24e589.el9.noarch 2024-09-14T06:26:38.231 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-14T06:26:38.231 INFO:teuthology.orchestra.run.smithi094.stdout:Complete! 2024-09-14T06:26:38.349 INFO:teuthology.orchestra.run.smithi138.stdout:(11/37): lua-5.4.4-4.el9.x86_64.rpm 1.1 MB/s | 188 kB 00:00 2024-09-14T06:26:38.419 DEBUG:teuthology.orchestra.run.smithi094:> sudo yum -y install ceph-immutable-object-cache 2024-09-14T06:26:38.433 INFO:teuthology.orchestra.run.smithi138.stdout:(12/37): python3-mako-1.1.4-6.el9.noarch.rpm 2.0 MB/s | 172 kB 00:00 2024-09-14T06:26:38.483 INFO:teuthology.orchestra.run.smithi138.stdout:(13/37): python3-markupsafe-1.1.1-12.el9.x86_64 696 kB/s | 35 kB 00:00 2024-09-14T06:26:38.508 INFO:teuthology.orchestra.run.smithi138.stdout:(14/37): python3-requests-2.25.1-8.el9.noarch.r 170 kB/s | 125 kB 00:00 2024-09-14T06:26:38.542 INFO:teuthology.orchestra.run.smithi138.stdout:(15/37): python3-packaging-20.9-5.el9.noarch.rp 1.3 MB/s | 77 kB 00:00 2024-09-14T06:26:38.592 INFO:teuthology.orchestra.run.smithi138.stdout:(16/37): python3-toml-0.10.2-6.el9.noarch.rpm 835 kB/s | 42 kB 00:00 2024-09-14T06:26:38.618 INFO:teuthology.orchestra.run.smithi138.stdout:(17/37): python3-pytz-2021.1-5.el9.noarch.rpm 469 kB/s | 51 kB 00:00 2024-09-14T06:26:38.660 INFO:teuthology.orchestra.run.smithi138.stdout:(18/37): luarocks-3.9.2-5.el9.noarch.rpm 3.5 MB/s | 151 kB 00:00 2024-09-14T06:26:38.685 INFO:teuthology.orchestra.run.smithi138.stdout:(19/37): lua-devel-5.4.4-4.el9.x86_64.rpm 241 kB/s | 22 kB 00:00 2024-09-14T06:26:38.710 INFO:teuthology.orchestra.run.smithi138.stdout:(20/37): python3-bcrypt-3.2.2-1.el9.x86_64.rpm 863 kB/s | 43 kB 00:00 2024-09-14T06:26:38.744 INFO:teuthology.orchestra.run.smithi138.stdout:(21/37): python3-cherrypy-18.6.1-2.el9.noarch.r 14 MB/s | 358 kB 00:00 2024-09-14T06:26:38.769 INFO:teuthology.orchestra.run.smithi138.stdout:(22/37): python3-jaraco-8.2.1-3.el9.noarch.rpm 460 kB/s | 11 kB 00:00 2024-09-14T06:26:38.795 INFO:teuthology.orchestra.run.smithi138.stdout:(23/37): python3-jaraco-classes-3.2.1-5.el9.noa 706 kB/s | 18 kB 00:00 2024-09-14T06:26:38.820 INFO:teuthology.orchestra.run.smithi138.stdout:(24/37): python3-urllib3-1.26.5-6.el9.noarch.rp 269 kB/s | 215 kB 00:00 2024-09-14T06:26:38.846 INFO:teuthology.orchestra.run.smithi138.stdout:(25/37): python3-jaraco-collections-3.0.0-8.el9 460 kB/s | 23 kB 00:00 2024-09-14T06:26:38.879 INFO:teuthology.orchestra.run.smithi138.stdout:(26/37): python3-cheroot-8.6.0-4.el9.noarch.rpm 886 kB/s | 172 kB 00:00 2024-09-14T06:26:38.909 INFO:teuthology.orchestra.run.smithi138.stdout:(27/37): python3-jaraco-text-3.2.0-6.el9.noarch 355 kB/s | 20 kB 00:00 2024-09-14T06:26:38.934 INFO:teuthology.orchestra.run.smithi138.stdout:(28/37): python3-jaraco-functools-3.5.0-2.el9.n 171 kB/s | 19 kB 00:00 2024-09-14T06:26:38.960 INFO:teuthology.orchestra.run.smithi138.stdout:(29/37): python3-logutils-0.3.5-21.el9.noarch.r 593 kB/s | 46 kB 00:00 2024-09-14T06:26:38.977 INFO:teuthology.orchestra.run.smithi138.stdout:(30/37): python3-more-itertools-8.12.0-2.el9.no 1.1 MB/s | 79 kB 00:00 2024-09-14T06:26:38.995 INFO:teuthology.orchestra.run.smithi094.stdout:Last metadata expiration check: 0:00:56 ago on Sat 14 Sep 2024 06:25:42 AM UTC. 2024-09-14T06:26:39.002 INFO:teuthology.orchestra.run.smithi138.stdout:(31/37): python3-portend-3.1.0-2.el9.noarch.rpm 390 kB/s | 16 kB 00:00 2024-09-14T06:26:39.027 INFO:teuthology.orchestra.run.smithi138.stdout:(32/37): python3-pyOpenSSL-21.0.0-1.el9.noarch. 1.7 MB/s | 90 kB 00:00 2024-09-14T06:26:39.053 INFO:teuthology.orchestra.run.smithi138.stdout:(33/37): python3-tempora-5.0.0-2.el9.noarch.rpm 711 kB/s | 36 kB 00:00 2024-09-14T06:26:39.078 INFO:teuthology.orchestra.run.smithi138.stdout:(34/37): python3-pecan-1.4.2-3.el9.noarch.rpm 1.9 MB/s | 272 kB 00:00 2024-09-14T06:26:39.103 INFO:teuthology.orchestra.run.smithi138.stdout:(35/37): python3-webob-1.8.8-2.el9.noarch.rpm 3.0 MB/s | 230 kB 00:00 2024-09-14T06:26:39.129 INFO:teuthology.orchestra.run.smithi138.stdout:(36/37): python3-zc-lockfile-2.0-10.el9.noarch. 397 kB/s | 20 kB 00:00 2024-09-14T06:26:39.129 INFO:teuthology.orchestra.run.smithi094.stdout:Dependencies resolved. 2024-09-14T06:26:39.129 INFO:teuthology.orchestra.run.smithi094.stdout:================================================================================ 2024-09-14T06:26:39.129 INFO:teuthology.orchestra.run.smithi094.stdout: Package Arch Version Repo Size 2024-09-14T06:26:39.130 INFO:teuthology.orchestra.run.smithi094.stdout:================================================================================ 2024-09-14T06:26:39.130 INFO:teuthology.orchestra.run.smithi094.stdout:Installing: 2024-09-14T06:26:39.130 INFO:teuthology.orchestra.run.smithi094.stdout: ceph-immutable-object-cache x86_64 2:19.3.0-4775.g7f24e589.el9 ceph 145 k 2024-09-14T06:26:39.130 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-14T06:26:39.130 INFO:teuthology.orchestra.run.smithi094.stdout:Transaction Summary 2024-09-14T06:26:39.130 INFO:teuthology.orchestra.run.smithi094.stdout:================================================================================ 2024-09-14T06:26:39.130 INFO:teuthology.orchestra.run.smithi094.stdout:Install 1 Package 2024-09-14T06:26:39.130 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-14T06:26:39.131 INFO:teuthology.orchestra.run.smithi094.stdout:Total download size: 145 k 2024-09-14T06:26:39.131 INFO:teuthology.orchestra.run.smithi094.stdout:Installed size: 432 k 2024-09-14T06:26:39.131 INFO:teuthology.orchestra.run.smithi094.stdout:Downloading Packages: 2024-09-14T06:26:39.154 INFO:teuthology.orchestra.run.smithi138.stdout:(37/37): python3-werkzeug-2.0.3-3.el9.1.noarch. 4.1 MB/s | 427 kB 00:00 2024-09-14T06:26:39.159 INFO:teuthology.orchestra.run.smithi138.stdout:-------------------------------------------------------------------------------- 2024-09-14T06:26:39.159 INFO:teuthology.orchestra.run.smithi138.stdout:Total 9.3 MB/s | 31 MB 00:03 2024-09-14T06:26:39.311 INFO:teuthology.orchestra.run.smithi138.stdout:Running transaction check 2024-09-14T06:26:39.332 INFO:teuthology.orchestra.run.smithi094.stdout:ceph-immutable-object-cache-19.3.0-4775.g7f24e5 720 kB/s | 145 kB 00:00 2024-09-14T06:26:39.333 INFO:teuthology.orchestra.run.smithi094.stdout:-------------------------------------------------------------------------------- 2024-09-14T06:26:39.334 INFO:teuthology.orchestra.run.smithi094.stdout:Total 713 kB/s | 145 kB 00:00 2024-09-14T06:26:39.334 INFO:teuthology.orchestra.run.smithi094.stdout:Running transaction check 2024-09-14T06:26:39.341 INFO:teuthology.orchestra.run.smithi094.stdout:Transaction check succeeded. 2024-09-14T06:26:39.341 INFO:teuthology.orchestra.run.smithi094.stdout:Running transaction test 2024-09-14T06:26:39.350 INFO:teuthology.orchestra.run.smithi138.stdout:Transaction check succeeded. 2024-09-14T06:26:39.350 INFO:teuthology.orchestra.run.smithi138.stdout:Running transaction test 2024-09-14T06:26:39.398 INFO:teuthology.orchestra.run.smithi094.stdout:Transaction test succeeded. 2024-09-14T06:26:39.398 INFO:teuthology.orchestra.run.smithi094.stdout:Running transaction 2024-09-14T06:26:39.555 INFO:teuthology.orchestra.run.smithi094.stdout: Preparing : 1/1 2024-09-14T06:26:39.633 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : ceph-immutable-object-cache-2:19.3.0-4775.g7f24e589. 1/1 2024-09-14T06:26:39.659 INFO:teuthology.orchestra.run.smithi094.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.3.0-4775.g7f24e589. 1/1 2024-09-14T06:26:39.659 INFO:teuthology.orchestra.run.smithi094.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-14T06:26:39.659 INFO:teuthology.orchestra.run.smithi094.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-09-14T06:26:39.659 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-14T06:26:39.812 INFO:teuthology.orchestra.run.smithi138.stdout:Transaction test succeeded. 2024-09-14T06:26:39.812 INFO:teuthology.orchestra.run.smithi138.stdout:Running transaction 2024-09-14T06:26:40.601 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : ceph-immutable-object-cache-2:19.3.0-4775.g7f24e589. 1/1 2024-09-14T06:26:40.601 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-14T06:26:40.601 INFO:teuthology.orchestra.run.smithi094.stdout:Installed: 2024-09-14T06:26:40.601 INFO:teuthology.orchestra.run.smithi094.stdout: ceph-immutable-object-cache-2:19.3.0-4775.g7f24e589.el9.x86_64 2024-09-14T06:26:40.601 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-14T06:26:40.601 INFO:teuthology.orchestra.run.smithi094.stdout:Complete! 2024-09-14T06:26:40.835 DEBUG:teuthology.orchestra.run.smithi094:> sudo yum -y install ceph-mgr 2024-09-14T06:26:41.174 INFO:teuthology.orchestra.run.smithi138.stdout: Preparing : 1/1 2024-09-14T06:26:41.241 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 1/37 2024-09-14T06:26:41.273 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 2/37 2024-09-14T06:26:41.310 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : lua-5.4.4-4.el9.x86_64 3/37 2024-09-14T06:26:41.384 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 4/37 2024-09-14T06:26:41.407 INFO:teuthology.orchestra.run.smithi094.stdout:Last metadata expiration check: 0:00:59 ago on Sat 14 Sep 2024 06:25:42 AM UTC. 2024-09-14T06:26:41.426 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : luarocks-3.9.2-5.el9.noarch 5/37 2024-09-14T06:26:41.459 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 6/37 2024-09-14T06:26:41.502 INFO:teuthology.orchestra.run.smithi094.stdout:Package ceph-mgr-2:19.3.0-4775.g7f24e589.el9.x86_64 is already installed. 2024-09-14T06:26:41.544 INFO:teuthology.orchestra.run.smithi094.stdout:Dependencies resolved. 2024-09-14T06:26:41.545 INFO:teuthology.orchestra.run.smithi094.stdout:Nothing to do. 2024-09-14T06:26:41.545 INFO:teuthology.orchestra.run.smithi094.stdout:Complete! 2024-09-14T06:26:41.551 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 7/37 2024-09-14T06:26:41.607 DEBUG:teuthology.orchestra.run.smithi094:> sudo yum -y install ceph-mgr-dashboard 2024-09-14T06:26:41.630 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 8/37 2024-09-14T06:26:41.679 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-webob-1.8.8-2.el9.noarch 9/37 2024-09-14T06:26:41.714 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 10/37 2024-09-14T06:26:41.744 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 11/37 2024-09-14T06:26:41.779 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 12/37 2024-09-14T06:26:41.812 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 13/37 2024-09-14T06:26:41.870 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 14/37 2024-09-14T06:26:41.913 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 15/37 2024-09-14T06:26:41.956 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 16/37 2024-09-14T06:26:42.005 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 17/37 2024-09-14T06:26:42.037 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-packaging-20.9-5.el9.noarch 18/37 2024-09-14T06:26:42.111 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 19/37 2024-09-14T06:26:42.173 INFO:teuthology.orchestra.run.smithi094.stdout:Last metadata expiration check: 0:01:00 ago on Sat 14 Sep 2024 06:25:42 AM UTC. 2024-09-14T06:26:42.259 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 20/37 2024-09-14T06:26:42.306 INFO:teuthology.orchestra.run.smithi094.stdout:Dependencies resolved. 2024-09-14T06:26:42.307 INFO:teuthology.orchestra.run.smithi094.stdout:================================================================================ 2024-09-14T06:26:42.307 INFO:teuthology.orchestra.run.smithi094.stdout: Package Arch Version Repository Size 2024-09-14T06:26:42.307 INFO:teuthology.orchestra.run.smithi094.stdout:================================================================================ 2024-09-14T06:26:42.307 INFO:teuthology.orchestra.run.smithi094.stdout:Installing: 2024-09-14T06:26:42.308 INFO:teuthology.orchestra.run.smithi094.stdout: ceph-mgr-dashboard noarch 2:19.3.0-4775.g7f24e589.el9 ceph-noarch 14 M 2024-09-14T06:26:42.308 INFO:teuthology.orchestra.run.smithi094.stdout:Installing dependencies: 2024-09-14T06:26:42.308 INFO:teuthology.orchestra.run.smithi094.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-09-14T06:26:42.308 INFO:teuthology.orchestra.run.smithi094.stdout: ceph-grafana-dashboards noarch 2:19.3.0-4775.g7f24e589.el9 ceph-noarch 34 k 2024-09-14T06:26:42.308 INFO:teuthology.orchestra.run.smithi094.stdout: ceph-prometheus-alerts noarch 2:19.3.0-4775.g7f24e589.el9 ceph-noarch 16 k 2024-09-14T06:26:42.308 INFO:teuthology.orchestra.run.smithi094.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-09-14T06:26:42.308 INFO:teuthology.orchestra.run.smithi094.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-09-14T06:26:42.308 INFO:teuthology.orchestra.run.smithi094.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-09-14T06:26:42.308 INFO:teuthology.orchestra.run.smithi094.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-09-14T06:26:42.308 INFO:teuthology.orchestra.run.smithi094.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-09-14T06:26:42.308 INFO:teuthology.orchestra.run.smithi094.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-09-14T06:26:42.308 INFO:teuthology.orchestra.run.smithi094.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-09-14T06:26:42.309 INFO:teuthology.orchestra.run.smithi094.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-09-14T06:26:42.309 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-14T06:26:42.309 INFO:teuthology.orchestra.run.smithi094.stdout:Transaction Summary 2024-09-14T06:26:42.309 INFO:teuthology.orchestra.run.smithi094.stdout:================================================================================ 2024-09-14T06:26:42.309 INFO:teuthology.orchestra.run.smithi094.stdout:Install 12 Packages 2024-09-14T06:26:42.309 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-14T06:26:42.309 INFO:teuthology.orchestra.run.smithi094.stdout:Total download size: 19 M 2024-09-14T06:26:42.310 INFO:teuthology.orchestra.run.smithi094.stdout:Installed size: 159 M 2024-09-14T06:26:42.310 INFO:teuthology.orchestra.run.smithi094.stdout:Downloading Packages: 2024-09-14T06:26:42.339 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 21/37 2024-09-14T06:26:42.389 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-urllib3-1.26.5-6.el9.noarch 22/37 2024-09-14T06:26:42.435 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 23/37 2024-09-14T06:26:42.485 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-ply-3.11-14.el9.noarch 24/37 2024-09-14T06:26:42.541 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 25/37 2024-09-14T06:26:42.729 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 26/37 2024-09-14T06:26:42.782 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 27/37 2024-09-14T06:26:42.862 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 28/37 2024-09-14T06:26:42.903 INFO:teuthology.orchestra.run.smithi094.stdout:(1/12): ceph-prometheus-alerts-19.3.0-4775.g7f2 104 kB/s | 16 kB 00:00 2024-09-14T06:26:42.928 INFO:teuthology.orchestra.run.smithi094.stdout:(2/12): ceph-grafana-dashboards-19.3.0-4775.g7f 186 kB/s | 34 kB 00:00 2024-09-14T06:26:42.958 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 29/37 2024-09-14T06:26:42.998 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 30/37 2024-09-14T06:26:43.071 INFO:teuthology.orchestra.run.smithi094.stdout:(3/12): python3-protobuf-3.14.0-13.el9.noarch.r 1.9 MB/s | 269 kB 00:00 2024-09-14T06:26:43.104 INFO:teuthology.orchestra.run.smithi094.stdout:(4/12): protobuf-3.14.0-13.el9.x86_64.rpm 5.0 MB/s | 1.0 MB 00:00 2024-09-14T06:26:43.130 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 31/37 2024-09-14T06:26:43.154 INFO:teuthology.orchestra.run.smithi094.stdout:(5/12): abseil-cpp-20211102.0-3.el9.x86_64.rpm 11 MB/s | 548 kB 00:00 2024-09-14T06:26:43.179 INFO:teuthology.orchestra.run.smithi094.stdout:(6/12): grpc-data-1.46.7-10.el9.noarch.rpm 782 kB/s | 19 kB 00:00 2024-09-14T06:26:43.247 INFO:teuthology.orchestra.run.smithi094.stdout:(7/12): python3-grpcio-1.46.7-10.el9.x86_64.rpm 31 MB/s | 2.0 MB 00:00 2024-09-14T06:26:43.280 INFO:teuthology.orchestra.run.smithi094.stdout:(8/12): python3-grpcio-tools-1.46.7-10.el9.x86_ 4.2 MB/s | 144 kB 00:00 2024-09-14T06:26:43.304 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : ceph-mgr-modules-core-2:19.3.0-4775.g7f24e589.el9. 32/37 2024-09-14T06:26:43.305 INFO:teuthology.orchestra.run.smithi094.stdout:(9/12): python3-repoze-lru-0.7-16.el9.noarch.rp 1.2 MB/s | 31 kB 00:00 2024-09-14T06:26:43.331 INFO:teuthology.orchestra.run.smithi094.stdout:(10/12): python3-routes-2.5.1-5.el9.noarch.rpm 7.4 MB/s | 188 kB 00:00 2024-09-14T06:26:43.334 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : ceph-mgr-2:19.3.0-4775.g7f24e589.el9.x86_64 33/37 2024-09-14T06:26:43.361 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: ceph-mgr-2:19.3.0-4775.g7f24e589.el9.x86_64 33/37 2024-09-14T06:26:43.361 INFO:teuthology.orchestra.run.smithi138.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-14T06:26:43.361 INFO:teuthology.orchestra.run.smithi138.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-09-14T06:26:43.361 INFO:teuthology.orchestra.run.smithi138.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-14T06:26:43.361 INFO:teuthology.orchestra.run.smithi138.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-14T06:26:43.361 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-09-14T06:26:43.481 INFO:teuthology.orchestra.run.smithi094.stdout:(11/12): ceph-mgr-dashboard-19.3.0-4775.g7f24e5 19 MB/s | 14 MB 00:00 2024-09-14T06:26:43.948 INFO:teuthology.orchestra.run.smithi094.stdout:(12/12): protobuf-compiler-3.14.0-13.el9.x86_64 984 kB/s | 863 kB 00:00 2024-09-14T06:26:43.952 INFO:teuthology.orchestra.run.smithi094.stdout:-------------------------------------------------------------------------------- 2024-09-14T06:26:43.952 INFO:teuthology.orchestra.run.smithi094.stdout:Total 12 MB/s | 19 MB 00:01 2024-09-14T06:26:44.003 INFO:teuthology.orchestra.run.smithi094.stdout:Running transaction check 2024-09-14T06:26:44.023 INFO:teuthology.orchestra.run.smithi094.stdout:Transaction check succeeded. 2024-09-14T06:26:44.023 INFO:teuthology.orchestra.run.smithi094.stdout:Running transaction test 2024-09-14T06:26:44.300 INFO:teuthology.orchestra.run.smithi094.stdout:Transaction test succeeded. 2024-09-14T06:26:44.300 INFO:teuthology.orchestra.run.smithi094.stdout:Running transaction 2024-09-14T06:26:44.819 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : ceph-osd-2:19.3.0-4775.g7f24e589.el9.x86_64 34/37 2024-09-14T06:26:44.846 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: ceph-osd-2:19.3.0-4775.g7f24e589.el9.x86_64 34/37 2024-09-14T06:26:44.846 INFO:teuthology.orchestra.run.smithi138.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-14T06:26:44.847 INFO:teuthology.orchestra.run.smithi138.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-09-14T06:26:44.847 INFO:teuthology.orchestra.run.smithi138.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-14T06:26:44.847 INFO:teuthology.orchestra.run.smithi138.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-14T06:26:44.847 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-09-14T06:26:44.895 INFO:teuthology.orchestra.run.smithi094.stdout: Preparing : 1/1 2024-09-14T06:26:44.982 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 1/12 2024-09-14T06:26:45.017 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 2/12 2024-09-14T06:26:45.073 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 3/12 2024-09-14T06:26:45.103 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 4/12 2024-09-14T06:26:45.211 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 5/12 2024-09-14T06:26:45.295 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : ceph-mon-2:19.3.0-4775.g7f24e589.el9.x86_64 35/37 2024-09-14T06:26:45.319 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: ceph-mon-2:19.3.0-4775.g7f24e589.el9.x86_64 35/37 2024-09-14T06:26:45.319 INFO:teuthology.orchestra.run.smithi138.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-14T06:26:45.319 INFO:teuthology.orchestra.run.smithi138.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-09-14T06:26:45.319 INFO:teuthology.orchestra.run.smithi138.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-14T06:26:45.319 INFO:teuthology.orchestra.run.smithi138.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-14T06:26:45.320 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-09-14T06:26:45.338 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 6/12 2024-09-14T06:26:45.423 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 7/12 2024-09-14T06:26:45.474 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 8/12 2024-09-14T06:26:45.511 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 9/12 2024-09-14T06:26:45.543 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : ceph-prometheus-alerts-2:19.3.0-4775.g7f24e589.el9 10/12 2024-09-14T06:26:45.545 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : ceph-mds-2:19.3.0-4775.g7f24e589.el9.x86_64 36/37 2024-09-14T06:26:45.570 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: ceph-mds-2:19.3.0-4775.g7f24e589.el9.x86_64 36/37 2024-09-14T06:26:45.570 INFO:teuthology.orchestra.run.smithi138.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-14T06:26:45.570 INFO:teuthology.orchestra.run.smithi138.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-09-14T06:26:45.570 INFO:teuthology.orchestra.run.smithi138.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-14T06:26:45.571 INFO:teuthology.orchestra.run.smithi138.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-14T06:26:45.571 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-09-14T06:26:45.635 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : ceph-2:19.3.0-4775.g7f24e589.el9.x86_64 37/37 2024-09-14T06:26:47.443 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: ceph-2:19.3.0-4775.g7f24e589.el9.x86_64 37/37 2024-09-14T06:26:47.444 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : ceph-2:19.3.0-4775.g7f24e589.el9.x86_64 1/37 2024-09-14T06:26:47.444 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : ceph-mds-2:19.3.0-4775.g7f24e589.el9.x86_64 2/37 2024-09-14T06:26:47.444 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : ceph-mgr-2:19.3.0-4775.g7f24e589.el9.x86_64 3/37 2024-09-14T06:26:47.444 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : ceph-mon-2:19.3.0-4775.g7f24e589.el9.x86_64 4/37 2024-09-14T06:26:47.444 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : ceph-osd-2:19.3.0-4775.g7f24e589.el9.x86_64 5/37 2024-09-14T06:26:47.444 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : ceph-mgr-modules-core-2:19.3.0-4775.g7f24e589.el9. 6/37 2024-09-14T06:26:47.444 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 7/37 2024-09-14T06:26:47.444 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 8/37 2024-09-14T06:26:47.444 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-ply-3.11-14.el9.noarch 9/37 2024-09-14T06:26:47.444 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 10/37 2024-09-14T06:26:47.444 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 11/37 2024-09-14T06:26:47.445 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-urllib3-1.26.5-6.el9.noarch 12/37 2024-09-14T06:26:47.445 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : lua-5.4.4-4.el9.x86_64 13/37 2024-09-14T06:26:47.445 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 14/37 2024-09-14T06:26:47.445 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 15/37 2024-09-14T06:26:47.445 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 16/37 2024-09-14T06:26:47.445 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 17/37 2024-09-14T06:26:47.445 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 18/37 2024-09-14T06:26:47.445 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 19/37 2024-09-14T06:26:47.445 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : luarocks-3.9.2-5.el9.noarch 20/37 2024-09-14T06:26:47.445 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 21/37 2024-09-14T06:26:47.446 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 22/37 2024-09-14T06:26:47.446 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 23/37 2024-09-14T06:26:47.446 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 24/37 2024-09-14T06:26:47.446 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 25/37 2024-09-14T06:26:47.447 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 26/37 2024-09-14T06:26:47.447 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 27/37 2024-09-14T06:26:47.447 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 28/37 2024-09-14T06:26:47.447 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 29/37 2024-09-14T06:26:47.447 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 30/37 2024-09-14T06:26:47.448 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 31/37 2024-09-14T06:26:47.448 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 32/37 2024-09-14T06:26:47.448 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 33/37 2024-09-14T06:26:47.448 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 34/37 2024-09-14T06:26:47.448 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-webob-1.8.8-2.el9.noarch 35/37 2024-09-14T06:26:47.448 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 36/37 2024-09-14T06:26:48.326 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 37/37 2024-09-14T06:26:48.326 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-09-14T06:26:48.326 INFO:teuthology.orchestra.run.smithi138.stdout:Installed: 2024-09-14T06:26:48.326 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-2:19.3.0-4775.g7f24e589.el9.x86_64 2024-09-14T06:26:48.326 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-mds-2:19.3.0-4775.g7f24e589.el9.x86_64 2024-09-14T06:26:48.326 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-mgr-2:19.3.0-4775.g7f24e589.el9.x86_64 2024-09-14T06:26:48.327 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-mgr-modules-core-2:19.3.0-4775.g7f24e589.el9.noarch 2024-09-14T06:26:48.327 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-mon-2:19.3.0-4775.g7f24e589.el9.x86_64 2024-09-14T06:26:48.327 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-osd-2:19.3.0-4775.g7f24e589.el9.x86_64 2024-09-14T06:26:48.327 INFO:teuthology.orchestra.run.smithi138.stdout: lua-5.4.4-4.el9.x86_64 2024-09-14T06:26:48.327 INFO:teuthology.orchestra.run.smithi138.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-09-14T06:26:48.327 INFO:teuthology.orchestra.run.smithi138.stdout: luarocks-3.9.2-5.el9.noarch 2024-09-14T06:26:48.327 INFO:teuthology.orchestra.run.smithi138.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-09-14T06:26:48.327 INFO:teuthology.orchestra.run.smithi138.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-09-14T06:26:48.327 INFO:teuthology.orchestra.run.smithi138.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-09-14T06:26:48.327 INFO:teuthology.orchestra.run.smithi138.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-09-14T06:26:48.327 INFO:teuthology.orchestra.run.smithi138.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-09-14T06:26:48.327 INFO:teuthology.orchestra.run.smithi138.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-09-14T06:26:48.327 INFO:teuthology.orchestra.run.smithi138.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-09-14T06:26:48.328 INFO:teuthology.orchestra.run.smithi138.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-09-14T06:26:48.328 INFO:teuthology.orchestra.run.smithi138.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-09-14T06:26:48.328 INFO:teuthology.orchestra.run.smithi138.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-09-14T06:26:48.328 INFO:teuthology.orchestra.run.smithi138.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-09-14T06:26:48.328 INFO:teuthology.orchestra.run.smithi138.stdout: python3-mako-1.1.4-6.el9.noarch 2024-09-14T06:26:48.328 INFO:teuthology.orchestra.run.smithi138.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-09-14T06:26:48.328 INFO:teuthology.orchestra.run.smithi138.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-09-14T06:26:48.328 INFO:teuthology.orchestra.run.smithi138.stdout: python3-packaging-20.9-5.el9.noarch 2024-09-14T06:26:48.328 INFO:teuthology.orchestra.run.smithi138.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-09-14T06:26:48.328 INFO:teuthology.orchestra.run.smithi138.stdout: python3-ply-3.11-14.el9.noarch 2024-09-14T06:26:48.328 INFO:teuthology.orchestra.run.smithi138.stdout: python3-portend-3.1.0-2.el9.noarch 2024-09-14T06:26:48.328 INFO:teuthology.orchestra.run.smithi138.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-09-14T06:26:48.328 INFO:teuthology.orchestra.run.smithi138.stdout: python3-pycparser-2.20-6.el9.noarch 2024-09-14T06:26:48.329 INFO:teuthology.orchestra.run.smithi138.stdout: python3-pytz-2021.1-5.el9.noarch 2024-09-14T06:26:48.329 INFO:teuthology.orchestra.run.smithi138.stdout: python3-requests-2.25.1-8.el9.noarch 2024-09-14T06:26:48.329 INFO:teuthology.orchestra.run.smithi138.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-09-14T06:26:48.329 INFO:teuthology.orchestra.run.smithi138.stdout: python3-toml-0.10.2-6.el9.noarch 2024-09-14T06:26:48.329 INFO:teuthology.orchestra.run.smithi138.stdout: python3-urllib3-1.26.5-6.el9.noarch 2024-09-14T06:26:48.329 INFO:teuthology.orchestra.run.smithi138.stdout: python3-webob-1.8.8-2.el9.noarch 2024-09-14T06:26:48.329 INFO:teuthology.orchestra.run.smithi138.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-09-14T06:26:48.329 INFO:teuthology.orchestra.run.smithi138.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-09-14T06:26:48.329 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-09-14T06:26:48.329 INFO:teuthology.orchestra.run.smithi138.stdout:Complete! 2024-09-14T06:26:48.665 DEBUG:teuthology.orchestra.run.smithi138:> sudo yum -y install ceph-base 2024-09-14T06:26:48.742 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : ceph-grafana-dashboards-2:19.3.0-4775.g7f24e589.el 11/12 2024-09-14T06:26:48.799 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : ceph-mgr-dashboard-2:19.3.0-4775.g7f24e589.el9.noa 12/12 2024-09-14T06:26:49.239 INFO:teuthology.orchestra.run.smithi138.stdout:Last metadata expiration check: 0:00:53 ago on Sat 14 Sep 2024 06:25:56 AM UTC. 2024-09-14T06:26:49.330 INFO:teuthology.orchestra.run.smithi138.stdout:Package ceph-base-2:19.3.0-4775.g7f24e589.el9.x86_64 is already installed. 2024-09-14T06:26:49.373 INFO:teuthology.orchestra.run.smithi138.stdout:Dependencies resolved. 2024-09-14T06:26:49.374 INFO:teuthology.orchestra.run.smithi138.stdout:Nothing to do. 2024-09-14T06:26:49.374 INFO:teuthology.orchestra.run.smithi138.stdout:Complete! 2024-09-14T06:26:49.439 DEBUG:teuthology.orchestra.run.smithi138:> sudo yum -y install cephadm 2024-09-14T06:26:50.002 INFO:teuthology.orchestra.run.smithi138.stdout:Last metadata expiration check: 0:00:54 ago on Sat 14 Sep 2024 06:25:56 AM UTC. 2024-09-14T06:26:50.134 INFO:teuthology.orchestra.run.smithi138.stdout:Dependencies resolved. 2024-09-14T06:26:50.134 INFO:teuthology.orchestra.run.smithi138.stdout:================================================================================ 2024-09-14T06:26:50.135 INFO:teuthology.orchestra.run.smithi138.stdout: Package Arch Version Repository Size 2024-09-14T06:26:50.135 INFO:teuthology.orchestra.run.smithi138.stdout:================================================================================ 2024-09-14T06:26:50.135 INFO:teuthology.orchestra.run.smithi138.stdout:Installing: 2024-09-14T06:26:50.135 INFO:teuthology.orchestra.run.smithi138.stdout: cephadm noarch 2:19.3.0-4775.g7f24e589.el9 ceph-noarch 783 k 2024-09-14T06:26:50.135 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-09-14T06:26:50.135 INFO:teuthology.orchestra.run.smithi138.stdout:Transaction Summary 2024-09-14T06:26:50.135 INFO:teuthology.orchestra.run.smithi138.stdout:================================================================================ 2024-09-14T06:26:50.135 INFO:teuthology.orchestra.run.smithi138.stdout:Install 1 Package 2024-09-14T06:26:50.135 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-09-14T06:26:50.135 INFO:teuthology.orchestra.run.smithi138.stdout:Total download size: 783 k 2024-09-14T06:26:50.135 INFO:teuthology.orchestra.run.smithi138.stdout:Installed size: 789 k 2024-09-14T06:26:50.135 INFO:teuthology.orchestra.run.smithi138.stdout:Downloading Packages: 2024-09-14T06:26:50.423 INFO:teuthology.orchestra.run.smithi138.stdout:cephadm-19.3.0-4775.g7f24e589.el9.noarch.rpm 2.7 MB/s | 783 kB 00:00 2024-09-14T06:26:50.424 INFO:teuthology.orchestra.run.smithi138.stdout:-------------------------------------------------------------------------------- 2024-09-14T06:26:50.424 INFO:teuthology.orchestra.run.smithi138.stdout:Total 2.6 MB/s | 783 kB 00:00 2024-09-14T06:26:50.424 INFO:teuthology.orchestra.run.smithi138.stdout:Running transaction check 2024-09-14T06:26:50.429 INFO:teuthology.orchestra.run.smithi138.stdout:Transaction check succeeded. 2024-09-14T06:26:50.430 INFO:teuthology.orchestra.run.smithi138.stdout:Running transaction test 2024-09-14T06:26:50.436 INFO:teuthology.orchestra.run.smithi138.stdout:Transaction test succeeded. 2024-09-14T06:26:50.436 INFO:teuthology.orchestra.run.smithi138.stdout:Running transaction 2024-09-14T06:26:50.506 INFO:teuthology.orchestra.run.smithi138.stdout: Preparing : 1/1 2024-09-14T06:26:50.880 INFO:teuthology.orchestra.run.smithi094.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.3.0-4775.g7f24e589.el9.noa 12/12 2024-09-14T06:26:50.880 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : ceph-grafana-dashboards-2:19.3.0-4775.g7f24e589.el 1/12 2024-09-14T06:26:50.880 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : ceph-mgr-dashboard-2:19.3.0-4775.g7f24e589.el9.noa 2/12 2024-09-14T06:26:50.880 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : ceph-prometheus-alerts-2:19.3.0-4775.g7f24e589.el9 3/12 2024-09-14T06:26:50.880 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 4/12 2024-09-14T06:26:50.880 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 5/12 2024-09-14T06:26:50.880 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 6/12 2024-09-14T06:26:50.880 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 7/12 2024-09-14T06:26:50.880 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 8/12 2024-09-14T06:26:50.880 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 9/12 2024-09-14T06:26:50.881 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 10/12 2024-09-14T06:26:50.881 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 11/12 2024-09-14T06:26:50.950 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: cephadm-2:19.3.0-4775.g7f24e589.el9.noarch 1/1 2024-09-14T06:26:51.037 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : cephadm-2:19.3.0-4775.g7f24e589.el9.noarch 1/1 2024-09-14T06:26:51.212 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 12/12 2024-09-14T06:26:51.212 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-14T06:26:51.212 INFO:teuthology.orchestra.run.smithi094.stdout:Installed: 2024-09-14T06:26:51.212 INFO:teuthology.orchestra.run.smithi094.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-09-14T06:26:51.213 INFO:teuthology.orchestra.run.smithi094.stdout: ceph-grafana-dashboards-2:19.3.0-4775.g7f24e589.el9.noarch 2024-09-14T06:26:51.213 INFO:teuthology.orchestra.run.smithi094.stdout: ceph-mgr-dashboard-2:19.3.0-4775.g7f24e589.el9.noarch 2024-09-14T06:26:51.213 INFO:teuthology.orchestra.run.smithi094.stdout: ceph-prometheus-alerts-2:19.3.0-4775.g7f24e589.el9.noarch 2024-09-14T06:26:51.213 INFO:teuthology.orchestra.run.smithi094.stdout: grpc-data-1.46.7-10.el9.noarch 2024-09-14T06:26:51.213 INFO:teuthology.orchestra.run.smithi094.stdout: protobuf-3.14.0-13.el9.x86_64 2024-09-14T06:26:51.213 INFO:teuthology.orchestra.run.smithi094.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-09-14T06:26:51.213 INFO:teuthology.orchestra.run.smithi094.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-09-14T06:26:51.213 INFO:teuthology.orchestra.run.smithi094.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-09-14T06:26:51.213 INFO:teuthology.orchestra.run.smithi094.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-09-14T06:26:51.213 INFO:teuthology.orchestra.run.smithi094.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-09-14T06:26:51.213 INFO:teuthology.orchestra.run.smithi094.stdout: python3-routes-2.5.1-5.el9.noarch 2024-09-14T06:26:51.213 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-14T06:26:51.213 INFO:teuthology.orchestra.run.smithi094.stdout:Complete! 2024-09-14T06:26:51.467 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: cephadm-2:19.3.0-4775.g7f24e589.el9.noarch 1/1 2024-09-14T06:26:51.468 DEBUG:teuthology.orchestra.run.smithi094:> sudo yum -y install ceph-mgr-diskprediction-local 2024-09-14T06:26:51.736 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : cephadm-2:19.3.0-4775.g7f24e589.el9.noarch 1/1 2024-09-14T06:26:51.736 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-09-14T06:26:51.736 INFO:teuthology.orchestra.run.smithi138.stdout:Installed: 2024-09-14T06:26:51.736 INFO:teuthology.orchestra.run.smithi138.stdout: cephadm-2:19.3.0-4775.g7f24e589.el9.noarch 2024-09-14T06:26:51.736 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-09-14T06:26:51.736 INFO:teuthology.orchestra.run.smithi138.stdout:Complete! 2024-09-14T06:26:51.955 DEBUG:teuthology.orchestra.run.smithi138:> sudo yum -y install ceph-immutable-object-cache 2024-09-14T06:26:52.042 INFO:teuthology.orchestra.run.smithi094.stdout:Last metadata expiration check: 0:01:10 ago on Sat 14 Sep 2024 06:25:42 AM UTC. 2024-09-14T06:26:52.185 INFO:teuthology.orchestra.run.smithi094.stdout:Dependencies resolved. 2024-09-14T06:26:52.186 INFO:teuthology.orchestra.run.smithi094.stdout:======================================================================================= 2024-09-14T06:26:52.186 INFO:teuthology.orchestra.run.smithi094.stdout: Package Arch Version Repository Size 2024-09-14T06:26:52.186 INFO:teuthology.orchestra.run.smithi094.stdout:======================================================================================= 2024-09-14T06:26:52.186 INFO:teuthology.orchestra.run.smithi094.stdout:Installing: 2024-09-14T06:26:52.186 INFO:teuthology.orchestra.run.smithi094.stdout: ceph-mgr-diskprediction-local noarch 2:19.3.0-4775.g7f24e589.el9 ceph-noarch 7.4 M 2024-09-14T06:26:52.186 INFO:teuthology.orchestra.run.smithi094.stdout:Installing dependencies: 2024-09-14T06:26:52.187 INFO:teuthology.orchestra.run.smithi094.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-09-14T06:26:52.187 INFO:teuthology.orchestra.run.smithi094.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-09-14T06:26:52.187 INFO:teuthology.orchestra.run.smithi094.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-09-14T06:26:52.187 INFO:teuthology.orchestra.run.smithi094.stdout: libgfortran x86_64 11.5.0-2.el9 baseos 801 k 2024-09-14T06:26:52.187 INFO:teuthology.orchestra.run.smithi094.stdout: libquadmath x86_64 11.5.0-2.el9 baseos 191 k 2024-09-14T06:26:52.187 INFO:teuthology.orchestra.run.smithi094.stdout: openblas x86_64 0.3.26-2.el9 appstream 38 k 2024-09-14T06:26:52.187 INFO:teuthology.orchestra.run.smithi094.stdout: openblas-openmp x86_64 0.3.26-2.el9 appstream 4.9 M 2024-09-14T06:26:52.187 INFO:teuthology.orchestra.run.smithi094.stdout: python3-devel x86_64 3.9.19-8.el9 appstream 245 k 2024-09-14T06:26:52.187 INFO:teuthology.orchestra.run.smithi094.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-09-14T06:26:52.187 INFO:teuthology.orchestra.run.smithi094.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-09-14T06:26:52.187 INFO:teuthology.orchestra.run.smithi094.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-09-14T06:26:52.187 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-14T06:26:52.187 INFO:teuthology.orchestra.run.smithi094.stdout:Transaction Summary 2024-09-14T06:26:52.188 INFO:teuthology.orchestra.run.smithi094.stdout:======================================================================================= 2024-09-14T06:26:52.188 INFO:teuthology.orchestra.run.smithi094.stdout:Install 12 Packages 2024-09-14T06:26:52.188 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-14T06:26:52.188 INFO:teuthology.orchestra.run.smithi094.stdout:Total download size: 38 M 2024-09-14T06:26:52.188 INFO:teuthology.orchestra.run.smithi094.stdout:Installed size: 212 M 2024-09-14T06:26:52.188 INFO:teuthology.orchestra.run.smithi094.stdout:Downloading Packages: 2024-09-14T06:26:52.523 INFO:teuthology.orchestra.run.smithi138.stdout:Last metadata expiration check: 0:00:56 ago on Sat 14 Sep 2024 06:25:56 AM UTC. 2024-09-14T06:26:52.658 INFO:teuthology.orchestra.run.smithi138.stdout:Dependencies resolved. 2024-09-14T06:26:52.659 INFO:teuthology.orchestra.run.smithi138.stdout:================================================================================ 2024-09-14T06:26:52.659 INFO:teuthology.orchestra.run.smithi138.stdout: Package Arch Version Repo Size 2024-09-14T06:26:52.659 INFO:teuthology.orchestra.run.smithi138.stdout:================================================================================ 2024-09-14T06:26:52.659 INFO:teuthology.orchestra.run.smithi138.stdout:Installing: 2024-09-14T06:26:52.659 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-immutable-object-cache x86_64 2:19.3.0-4775.g7f24e589.el9 ceph 145 k 2024-09-14T06:26:52.659 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-09-14T06:26:52.659 INFO:teuthology.orchestra.run.smithi138.stdout:Transaction Summary 2024-09-14T06:26:52.659 INFO:teuthology.orchestra.run.smithi138.stdout:================================================================================ 2024-09-14T06:26:52.659 INFO:teuthology.orchestra.run.smithi138.stdout:Install 1 Package 2024-09-14T06:26:52.659 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-09-14T06:26:52.660 INFO:teuthology.orchestra.run.smithi138.stdout:Total download size: 145 k 2024-09-14T06:26:52.660 INFO:teuthology.orchestra.run.smithi138.stdout:Installed size: 432 k 2024-09-14T06:26:52.660 INFO:teuthology.orchestra.run.smithi138.stdout:Downloading Packages: 2024-09-14T06:26:52.757 INFO:teuthology.orchestra.run.smithi094.stdout:(1/12): ceph-mgr-diskprediction-local-19.3.0-47 17 MB/s | 7.4 MB 00:00 2024-09-14T06:26:52.859 INFO:teuthology.orchestra.run.smithi138.stdout:ceph-immutable-object-cache-19.3.0-4775.g7f24e5 728 kB/s | 145 kB 00:00 2024-09-14T06:26:52.859 INFO:teuthology.orchestra.run.smithi138.stdout:-------------------------------------------------------------------------------- 2024-09-14T06:26:52.859 INFO:teuthology.orchestra.run.smithi138.stdout:Total 722 kB/s | 145 kB 00:00 2024-09-14T06:26:52.860 INFO:teuthology.orchestra.run.smithi138.stdout:Running transaction check 2024-09-14T06:26:52.867 INFO:teuthology.orchestra.run.smithi138.stdout:Transaction check succeeded. 2024-09-14T06:26:52.867 INFO:teuthology.orchestra.run.smithi138.stdout:Running transaction test 2024-09-14T06:26:52.924 INFO:teuthology.orchestra.run.smithi138.stdout:Transaction test succeeded. 2024-09-14T06:26:52.925 INFO:teuthology.orchestra.run.smithi138.stdout:Running transaction 2024-09-14T06:26:53.084 INFO:teuthology.orchestra.run.smithi138.stdout: Preparing : 1/1 2024-09-14T06:26:53.124 INFO:teuthology.orchestra.run.smithi094.stdout:(2/12): libquadmath-11.5.0-2.el9.x86_64.rpm 236 kB/s | 191 kB 00:00 2024-09-14T06:26:53.177 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : ceph-immutable-object-cache-2:19.3.0-4775.g7f24e589. 1/1 2024-09-14T06:26:53.202 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.3.0-4775.g7f24e589. 1/1 2024-09-14T06:26:53.202 INFO:teuthology.orchestra.run.smithi138.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-14T06:26:53.202 INFO:teuthology.orchestra.run.smithi138.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-09-14T06:26:53.202 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-09-14T06:26:53.283 INFO:teuthology.orchestra.run.smithi094.stdout:(3/12): libgfortran-11.5.0-2.el9.x86_64.rpm 827 kB/s | 801 kB 00:00 2024-09-14T06:26:53.316 INFO:teuthology.orchestra.run.smithi094.stdout:(4/12): flexiblas-3.0.4-8.el9.x86_64.rpm 58 kB/s | 33 kB 00:00 2024-09-14T06:26:53.759 INFO:teuthology.orchestra.run.smithi094.stdout:(5/12): flexiblas-openblas-openmp-3.0.4-8.el9.x 38 kB/s | 18 kB 00:00 2024-09-14T06:26:53.867 INFO:teuthology.orchestra.run.smithi094.stdout:(6/12): openblas-0.3.26-2.el9.x86_64.rpm 70 kB/s | 38 kB 00:00 2024-09-14T06:26:54.130 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : ceph-immutable-object-cache-2:19.3.0-4775.g7f24e589. 1/1 2024-09-14T06:26:54.130 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-09-14T06:26:54.130 INFO:teuthology.orchestra.run.smithi138.stdout:Installed: 2024-09-14T06:26:54.130 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-immutable-object-cache-2:19.3.0-4775.g7f24e589.el9.x86_64 2024-09-14T06:26:54.130 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-09-14T06:26:54.130 INFO:teuthology.orchestra.run.smithi138.stdout:Complete! 2024-09-14T06:26:54.176 INFO:teuthology.orchestra.run.smithi094.stdout:(7/12): flexiblas-netlib-3.0.4-8.el9.x86_64.rpm 2.8 MB/s | 3.0 MB 00:01 2024-09-14T06:26:54.376 DEBUG:teuthology.orchestra.run.smithi138:> sudo yum -y install ceph-mgr 2024-09-14T06:26:54.643 INFO:teuthology.orchestra.run.smithi094.stdout:(8/12): python3-devel-3.9.19-8.el9.x86_64.rpm 316 kB/s | 245 kB 00:00 2024-09-14T06:26:54.945 INFO:teuthology.orchestra.run.smithi138.stdout:Last metadata expiration check: 0:00:58 ago on Sat 14 Sep 2024 06:25:56 AM UTC. 2024-09-14T06:26:55.001 INFO:teuthology.orchestra.run.smithi094.stdout:(9/12): openblas-openmp-0.3.26-2.el9.x86_64.rpm 4.0 MB/s | 4.9 MB 00:01 2024-09-14T06:26:55.041 INFO:teuthology.orchestra.run.smithi138.stdout:Package ceph-mgr-2:19.3.0-4775.g7f24e589.el9.x86_64 is already installed. 2024-09-14T06:26:55.082 INFO:teuthology.orchestra.run.smithi138.stdout:Dependencies resolved. 2024-09-14T06:26:55.083 INFO:teuthology.orchestra.run.smithi138.stdout:Nothing to do. 2024-09-14T06:26:55.083 INFO:teuthology.orchestra.run.smithi138.stdout:Complete! 2024-09-14T06:26:55.144 DEBUG:teuthology.orchestra.run.smithi138:> sudo yum -y install ceph-mgr-dashboard 2024-09-14T06:26:55.327 INFO:teuthology.orchestra.run.smithi094.stdout:(10/12): python3-numpy-1.20.1-5.el9.x86_64.rpm 4.4 MB/s | 5.1 MB 00:01 2024-09-14T06:26:55.477 INFO:teuthology.orchestra.run.smithi094.stdout:(11/12): python3-numpy-f2py-1.20.1-5.el9.x86_64 301 kB/s | 250 kB 00:00 2024-09-14T06:26:55.707 INFO:teuthology.orchestra.run.smithi138.stdout:Last metadata expiration check: 0:00:59 ago on Sat 14 Sep 2024 06:25:56 AM UTC. 2024-09-14T06:26:55.839 INFO:teuthology.orchestra.run.smithi138.stdout:Dependencies resolved. 2024-09-14T06:26:55.841 INFO:teuthology.orchestra.run.smithi138.stdout:================================================================================ 2024-09-14T06:26:55.841 INFO:teuthology.orchestra.run.smithi138.stdout: Package Arch Version Repository Size 2024-09-14T06:26:55.841 INFO:teuthology.orchestra.run.smithi138.stdout:================================================================================ 2024-09-14T06:26:55.841 INFO:teuthology.orchestra.run.smithi138.stdout:Installing: 2024-09-14T06:26:55.841 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-mgr-dashboard noarch 2:19.3.0-4775.g7f24e589.el9 ceph-noarch 14 M 2024-09-14T06:26:55.841 INFO:teuthology.orchestra.run.smithi138.stdout:Installing dependencies: 2024-09-14T06:26:55.841 INFO:teuthology.orchestra.run.smithi138.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-09-14T06:26:55.841 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-grafana-dashboards noarch 2:19.3.0-4775.g7f24e589.el9 ceph-noarch 34 k 2024-09-14T06:26:55.841 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-prometheus-alerts noarch 2:19.3.0-4775.g7f24e589.el9 ceph-noarch 16 k 2024-09-14T06:26:55.841 INFO:teuthology.orchestra.run.smithi138.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-09-14T06:26:55.842 INFO:teuthology.orchestra.run.smithi138.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-09-14T06:26:55.842 INFO:teuthology.orchestra.run.smithi138.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-09-14T06:26:55.842 INFO:teuthology.orchestra.run.smithi138.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-09-14T06:26:55.842 INFO:teuthology.orchestra.run.smithi138.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-09-14T06:26:55.842 INFO:teuthology.orchestra.run.smithi138.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-09-14T06:26:55.842 INFO:teuthology.orchestra.run.smithi138.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-09-14T06:26:55.842 INFO:teuthology.orchestra.run.smithi138.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-09-14T06:26:55.842 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-09-14T06:26:55.842 INFO:teuthology.orchestra.run.smithi138.stdout:Transaction Summary 2024-09-14T06:26:55.842 INFO:teuthology.orchestra.run.smithi138.stdout:================================================================================ 2024-09-14T06:26:55.842 INFO:teuthology.orchestra.run.smithi138.stdout:Install 12 Packages 2024-09-14T06:26:55.842 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-09-14T06:26:55.843 INFO:teuthology.orchestra.run.smithi138.stdout:Total download size: 19 M 2024-09-14T06:26:55.843 INFO:teuthology.orchestra.run.smithi138.stdout:Installed size: 159 M 2024-09-14T06:26:55.843 INFO:teuthology.orchestra.run.smithi138.stdout:Downloading Packages: 2024-09-14T06:26:56.141 INFO:teuthology.orchestra.run.smithi138.stdout:(1/12): ceph-grafana-dashboards-19.3.0-4775.g7f 229 kB/s | 34 kB 00:00 2024-09-14T06:26:56.166 INFO:teuthology.orchestra.run.smithi138.stdout:(2/12): ceph-prometheus-alerts-19.3.0-4775.g7f2 94 kB/s | 16 kB 00:00 2024-09-14T06:26:56.300 INFO:teuthology.orchestra.run.smithi138.stdout:(3/12): python3-protobuf-3.14.0-13.el9.noarch.r 2.0 MB/s | 269 kB 00:00 2024-09-14T06:26:56.350 INFO:teuthology.orchestra.run.smithi138.stdout:(4/12): protobuf-3.14.0-13.el9.x86_64.rpm 4.8 MB/s | 1.0 MB 00:00 2024-09-14T06:26:56.401 INFO:teuthology.orchestra.run.smithi138.stdout:(5/12): abseil-cpp-20211102.0-3.el9.x86_64.rpm 11 MB/s | 548 kB 00:00 2024-09-14T06:26:56.418 INFO:teuthology.orchestra.run.smithi138.stdout:(6/12): grpc-data-1.46.7-10.el9.noarch.rpm 1.1 MB/s | 19 kB 00:00 2024-09-14T06:26:56.477 INFO:teuthology.orchestra.run.smithi094.stdout:(12/12): python3-scipy-1.6.2-8.el9.x86_64.rpm 11 MB/s | 16 MB 00:01 2024-09-14T06:26:56.479 INFO:teuthology.orchestra.run.smithi094.stdout:-------------------------------------------------------------------------------- 2024-09-14T06:26:56.479 INFO:teuthology.orchestra.run.smithi094.stdout:Total 8.9 MB/s | 38 MB 00:04 2024-09-14T06:26:56.485 INFO:teuthology.orchestra.run.smithi138.stdout:(7/12): python3-grpcio-1.46.7-10.el9.x86_64.rpm 30 MB/s | 2.0 MB 00:00 2024-09-14T06:26:56.502 INFO:teuthology.orchestra.run.smithi138.stdout:(8/12): python3-grpcio-tools-1.46.7-10.el9.x86_ 8.6 MB/s | 144 kB 00:00 2024-09-14T06:26:56.527 INFO:teuthology.orchestra.run.smithi138.stdout:(9/12): python3-repoze-lru-0.7-16.el9.noarch.rp 1.2 MB/s | 31 kB 00:00 2024-09-14T06:26:56.552 INFO:teuthology.orchestra.run.smithi138.stdout:(10/12): python3-routes-2.5.1-5.el9.noarch.rpm 7.3 MB/s | 188 kB 00:00 2024-09-14T06:26:56.728 INFO:teuthology.orchestra.run.smithi138.stdout:(11/12): ceph-mgr-dashboard-19.3.0-4775.g7f24e5 19 MB/s | 14 MB 00:00 2024-09-14T06:26:56.837 INFO:teuthology.orchestra.run.smithi094.stdout:Running transaction check 2024-09-14T06:26:56.862 INFO:teuthology.orchestra.run.smithi094.stdout:Transaction check succeeded. 2024-09-14T06:26:56.862 INFO:teuthology.orchestra.run.smithi094.stdout:Running transaction test 2024-09-14T06:26:57.302 INFO:teuthology.orchestra.run.smithi094.stdout:Transaction test succeeded. 2024-09-14T06:26:57.303 INFO:teuthology.orchestra.run.smithi094.stdout:Running transaction 2024-09-14T06:26:57.336 INFO:teuthology.orchestra.run.smithi138.stdout:(12/12): protobuf-compiler-3.14.0-13.el9.x86_64 833 kB/s | 863 kB 00:01 2024-09-14T06:26:57.341 INFO:teuthology.orchestra.run.smithi138.stdout:-------------------------------------------------------------------------------- 2024-09-14T06:26:57.341 INFO:teuthology.orchestra.run.smithi138.stdout:Total 13 MB/s | 19 MB 00:01 2024-09-14T06:26:57.393 INFO:teuthology.orchestra.run.smithi138.stdout:Running transaction check 2024-09-14T06:26:57.413 INFO:teuthology.orchestra.run.smithi138.stdout:Transaction check succeeded. 2024-09-14T06:26:57.413 INFO:teuthology.orchestra.run.smithi138.stdout:Running transaction test 2024-09-14T06:26:57.674 INFO:teuthology.orchestra.run.smithi138.stdout:Transaction test succeeded. 2024-09-14T06:26:57.675 INFO:teuthology.orchestra.run.smithi138.stdout:Running transaction 2024-09-14T06:26:58.080 INFO:teuthology.orchestra.run.smithi094.stdout: Preparing : 1/1 2024-09-14T06:26:58.161 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 1/12 2024-09-14T06:26:58.200 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : libquadmath-11.5.0-2.el9.x86_64 2/12 2024-09-14T06:26:58.264 INFO:teuthology.orchestra.run.smithi138.stdout: Preparing : 1/1 2024-09-14T06:26:58.300 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : libgfortran-11.5.0-2.el9.x86_64 3/12 2024-09-14T06:26:58.335 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-devel-3.9.19-8.el9.x86_64 4/12 2024-09-14T06:26:58.345 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 1/12 2024-09-14T06:26:58.389 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 2/12 2024-09-14T06:26:58.436 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 3/12 2024-09-14T06:26:58.466 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 4/12 2024-09-14T06:26:58.521 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : openblas-0.3.26-2.el9.x86_64 5/12 2024-09-14T06:26:58.550 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : openblas-openmp-0.3.26-2.el9.x86_64 6/12 2024-09-14T06:26:58.580 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 5/12 2024-09-14T06:26:58.635 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 7/12 2024-09-14T06:26:58.702 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 6/12 2024-09-14T06:26:58.796 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 7/12 2024-09-14T06:26:58.848 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 8/12 2024-09-14T06:26:58.901 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 9/12 2024-09-14T06:26:58.933 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : ceph-prometheus-alerts-2:19.3.0-4775.g7f24e589.el9 10/12 2024-09-14T06:26:59.295 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 8/12 2024-09-14T06:26:59.414 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 9/12 2024-09-14T06:27:00.651 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 10/12 2024-09-14T06:27:01.698 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 11/12 2024-09-14T06:27:01.726 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : ceph-mgr-diskprediction-local-2:19.3.0-4775.g7f24e 12/12 2024-09-14T06:27:02.140 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : ceph-grafana-dashboards-2:19.3.0-4775.g7f24e589.el 11/12 2024-09-14T06:27:02.206 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : ceph-mgr-dashboard-2:19.3.0-4775.g7f24e589.el9.noa 12/12 2024-09-14T06:27:04.209 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.3.0-4775.g7f24e589.el9.noa 12/12 2024-09-14T06:27:04.210 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : ceph-grafana-dashboards-2:19.3.0-4775.g7f24e589.el 1/12 2024-09-14T06:27:04.210 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : ceph-mgr-dashboard-2:19.3.0-4775.g7f24e589.el9.noa 2/12 2024-09-14T06:27:04.210 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : ceph-prometheus-alerts-2:19.3.0-4775.g7f24e589.el9 3/12 2024-09-14T06:27:04.210 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 4/12 2024-09-14T06:27:04.210 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 5/12 2024-09-14T06:27:04.210 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 6/12 2024-09-14T06:27:04.210 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 7/12 2024-09-14T06:27:04.210 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 8/12 2024-09-14T06:27:04.210 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 9/12 2024-09-14T06:27:04.210 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 10/12 2024-09-14T06:27:04.211 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 11/12 2024-09-14T06:27:04.443 INFO:teuthology.orchestra.run.smithi094.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.3.0-4775.g7f24e 12/12 2024-09-14T06:27:04.443 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.3.0-4775.g7f24e 1/12 2024-09-14T06:27:04.443 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : libgfortran-11.5.0-2.el9.x86_64 2/12 2024-09-14T06:27:04.443 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : libquadmath-11.5.0-2.el9.x86_64 3/12 2024-09-14T06:27:04.443 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 4/12 2024-09-14T06:27:04.443 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 5/12 2024-09-14T06:27:04.444 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 6/12 2024-09-14T06:27:04.444 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : openblas-0.3.26-2.el9.x86_64 7/12 2024-09-14T06:27:04.444 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : openblas-openmp-0.3.26-2.el9.x86_64 8/12 2024-09-14T06:27:04.444 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-devel-3.9.19-8.el9.x86_64 9/12 2024-09-14T06:27:04.444 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 10/12 2024-09-14T06:27:04.444 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 11/12 2024-09-14T06:27:04.525 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 12/12 2024-09-14T06:27:04.525 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-09-14T06:27:04.525 INFO:teuthology.orchestra.run.smithi138.stdout:Installed: 2024-09-14T06:27:04.525 INFO:teuthology.orchestra.run.smithi138.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-09-14T06:27:04.525 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-grafana-dashboards-2:19.3.0-4775.g7f24e589.el9.noarch 2024-09-14T06:27:04.525 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-mgr-dashboard-2:19.3.0-4775.g7f24e589.el9.noarch 2024-09-14T06:27:04.525 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-prometheus-alerts-2:19.3.0-4775.g7f24e589.el9.noarch 2024-09-14T06:27:04.525 INFO:teuthology.orchestra.run.smithi138.stdout: grpc-data-1.46.7-10.el9.noarch 2024-09-14T06:27:04.526 INFO:teuthology.orchestra.run.smithi138.stdout: protobuf-3.14.0-13.el9.x86_64 2024-09-14T06:27:04.526 INFO:teuthology.orchestra.run.smithi138.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-09-14T06:27:04.526 INFO:teuthology.orchestra.run.smithi138.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-09-14T06:27:04.526 INFO:teuthology.orchestra.run.smithi138.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-09-14T06:27:04.526 INFO:teuthology.orchestra.run.smithi138.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-09-14T06:27:04.526 INFO:teuthology.orchestra.run.smithi138.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-09-14T06:27:04.526 INFO:teuthology.orchestra.run.smithi138.stdout: python3-routes-2.5.1-5.el9.noarch 2024-09-14T06:27:04.526 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-09-14T06:27:04.526 INFO:teuthology.orchestra.run.smithi138.stdout:Complete! 2024-09-14T06:27:04.769 DEBUG:teuthology.orchestra.run.smithi138:> sudo yum -y install ceph-mgr-diskprediction-local 2024-09-14T06:27:04.885 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 12/12 2024-09-14T06:27:04.885 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-14T06:27:04.885 INFO:teuthology.orchestra.run.smithi094.stdout:Installed: 2024-09-14T06:27:04.885 INFO:teuthology.orchestra.run.smithi094.stdout: ceph-mgr-diskprediction-local-2:19.3.0-4775.g7f24e589.el9.noarch 2024-09-14T06:27:04.886 INFO:teuthology.orchestra.run.smithi094.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-09-14T06:27:04.886 INFO:teuthology.orchestra.run.smithi094.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-09-14T06:27:04.886 INFO:teuthology.orchestra.run.smithi094.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-09-14T06:27:04.886 INFO:teuthology.orchestra.run.smithi094.stdout: libgfortran-11.5.0-2.el9.x86_64 2024-09-14T06:27:04.886 INFO:teuthology.orchestra.run.smithi094.stdout: libquadmath-11.5.0-2.el9.x86_64 2024-09-14T06:27:04.886 INFO:teuthology.orchestra.run.smithi094.stdout: openblas-0.3.26-2.el9.x86_64 2024-09-14T06:27:04.886 INFO:teuthology.orchestra.run.smithi094.stdout: openblas-openmp-0.3.26-2.el9.x86_64 2024-09-14T06:27:04.886 INFO:teuthology.orchestra.run.smithi094.stdout: python3-devel-3.9.19-8.el9.x86_64 2024-09-14T06:27:04.886 INFO:teuthology.orchestra.run.smithi094.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-09-14T06:27:04.886 INFO:teuthology.orchestra.run.smithi094.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-09-14T06:27:04.886 INFO:teuthology.orchestra.run.smithi094.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-09-14T06:27:04.886 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-14T06:27:04.886 INFO:teuthology.orchestra.run.smithi094.stdout:Complete! 2024-09-14T06:27:05.245 DEBUG:teuthology.orchestra.run.smithi094:> sudo yum -y install ceph-mgr-rook 2024-09-14T06:27:05.344 INFO:teuthology.orchestra.run.smithi138.stdout:Last metadata expiration check: 0:01:09 ago on Sat 14 Sep 2024 06:25:56 AM UTC. 2024-09-14T06:27:05.485 INFO:teuthology.orchestra.run.smithi138.stdout:Dependencies resolved. 2024-09-14T06:27:05.486 INFO:teuthology.orchestra.run.smithi138.stdout:======================================================================================= 2024-09-14T06:27:05.486 INFO:teuthology.orchestra.run.smithi138.stdout: Package Arch Version Repository Size 2024-09-14T06:27:05.486 INFO:teuthology.orchestra.run.smithi138.stdout:======================================================================================= 2024-09-14T06:27:05.486 INFO:teuthology.orchestra.run.smithi138.stdout:Installing: 2024-09-14T06:27:05.486 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-mgr-diskprediction-local noarch 2:19.3.0-4775.g7f24e589.el9 ceph-noarch 7.4 M 2024-09-14T06:27:05.487 INFO:teuthology.orchestra.run.smithi138.stdout:Installing dependencies: 2024-09-14T06:27:05.487 INFO:teuthology.orchestra.run.smithi138.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-09-14T06:27:05.487 INFO:teuthology.orchestra.run.smithi138.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-09-14T06:27:05.487 INFO:teuthology.orchestra.run.smithi138.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-09-14T06:27:05.487 INFO:teuthology.orchestra.run.smithi138.stdout: libgfortran x86_64 11.5.0-2.el9 baseos 801 k 2024-09-14T06:27:05.487 INFO:teuthology.orchestra.run.smithi138.stdout: libquadmath x86_64 11.5.0-2.el9 baseos 191 k 2024-09-14T06:27:05.487 INFO:teuthology.orchestra.run.smithi138.stdout: openblas x86_64 0.3.26-2.el9 appstream 38 k 2024-09-14T06:27:05.487 INFO:teuthology.orchestra.run.smithi138.stdout: openblas-openmp x86_64 0.3.26-2.el9 appstream 4.9 M 2024-09-14T06:27:05.487 INFO:teuthology.orchestra.run.smithi138.stdout: python3-devel x86_64 3.9.19-8.el9 appstream 245 k 2024-09-14T06:27:05.487 INFO:teuthology.orchestra.run.smithi138.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-09-14T06:27:05.487 INFO:teuthology.orchestra.run.smithi138.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-09-14T06:27:05.487 INFO:teuthology.orchestra.run.smithi138.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-09-14T06:27:05.487 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-09-14T06:27:05.488 INFO:teuthology.orchestra.run.smithi138.stdout:Transaction Summary 2024-09-14T06:27:05.488 INFO:teuthology.orchestra.run.smithi138.stdout:======================================================================================= 2024-09-14T06:27:05.488 INFO:teuthology.orchestra.run.smithi138.stdout:Install 12 Packages 2024-09-14T06:27:05.488 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-09-14T06:27:05.488 INFO:teuthology.orchestra.run.smithi138.stdout:Total download size: 38 M 2024-09-14T06:27:05.488 INFO:teuthology.orchestra.run.smithi138.stdout:Installed size: 212 M 2024-09-14T06:27:05.488 INFO:teuthology.orchestra.run.smithi138.stdout:Downloading Packages: 2024-09-14T06:27:05.818 INFO:teuthology.orchestra.run.smithi094.stdout:Last metadata expiration check: 0:01:23 ago on Sat 14 Sep 2024 06:25:42 AM UTC. 2024-09-14T06:27:05.954 INFO:teuthology.orchestra.run.smithi094.stdout:Dependencies resolved. 2024-09-14T06:27:05.955 INFO:teuthology.orchestra.run.smithi094.stdout:================================================================================ 2024-09-14T06:27:05.955 INFO:teuthology.orchestra.run.smithi094.stdout: Package Arch Version Repository Size 2024-09-14T06:27:05.955 INFO:teuthology.orchestra.run.smithi094.stdout:================================================================================ 2024-09-14T06:27:05.956 INFO:teuthology.orchestra.run.smithi094.stdout:Installing: 2024-09-14T06:27:05.956 INFO:teuthology.orchestra.run.smithi094.stdout: ceph-mgr-rook noarch 2:19.3.0-4775.g7f24e589.el9 ceph-noarch 49 k 2024-09-14T06:27:05.956 INFO:teuthology.orchestra.run.smithi094.stdout:Installing dependencies: 2024-09-14T06:27:05.956 INFO:teuthology.orchestra.run.smithi094.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-09-14T06:27:05.956 INFO:teuthology.orchestra.run.smithi094.stdout: python3-certifi noarch 2023.05.07-4.el9 epel 14 k 2024-09-14T06:27:05.956 INFO:teuthology.orchestra.run.smithi094.stdout: python3-google-auth noarch 1:2.34.0-1.el9 epel 220 k 2024-09-14T06:27:05.956 INFO:teuthology.orchestra.run.smithi094.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-09-14T06:27:05.956 INFO:teuthology.orchestra.run.smithi094.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-09-14T06:27:05.956 INFO:teuthology.orchestra.run.smithi094.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-09-14T06:27:05.956 INFO:teuthology.orchestra.run.smithi094.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-09-14T06:27:05.956 INFO:teuthology.orchestra.run.smithi094.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-09-14T06:27:05.957 INFO:teuthology.orchestra.run.smithi094.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-09-14T06:27:05.957 INFO:teuthology.orchestra.run.smithi094.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-09-14T06:27:05.957 INFO:teuthology.orchestra.run.smithi094.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-09-14T06:27:05.957 INFO:teuthology.orchestra.run.smithi094.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-09-14T06:27:05.957 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-14T06:27:05.957 INFO:teuthology.orchestra.run.smithi094.stdout:Transaction Summary 2024-09-14T06:27:05.957 INFO:teuthology.orchestra.run.smithi094.stdout:================================================================================ 2024-09-14T06:27:05.957 INFO:teuthology.orchestra.run.smithi094.stdout:Install 13 Packages 2024-09-14T06:27:05.957 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-14T06:27:05.958 INFO:teuthology.orchestra.run.smithi094.stdout:Total download size: 2.2 M 2024-09-14T06:27:05.958 INFO:teuthology.orchestra.run.smithi094.stdout:Installed size: 26 M 2024-09-14T06:27:05.958 INFO:teuthology.orchestra.run.smithi094.stdout:Downloading Packages: 2024-09-14T06:27:06.091 INFO:teuthology.orchestra.run.smithi138.stdout:(1/12): ceph-mgr-diskprediction-local-19.3.0-47 17 MB/s | 7.4 MB 00:00 2024-09-14T06:27:06.174 INFO:teuthology.orchestra.run.smithi138.stdout:(2/12): flexiblas-3.0.4-8.el9.x86_64.rpm 394 kB/s | 33 kB 00:00 2024-09-14T06:27:06.244 INFO:teuthology.orchestra.run.smithi094.stdout:(1/13): python3-jsonpointer-2.0-4.el9.noarch.rp 219 kB/s | 19 kB 00:00 2024-09-14T06:27:06.270 INFO:teuthology.orchestra.run.smithi094.stdout:(2/13): python3-jsonpatch-1.21-16.el9.noarch.rp 227 kB/s | 26 kB 00:00 2024-09-14T06:27:06.337 INFO:teuthology.orchestra.run.smithi094.stdout:(3/13): python3-pyasn1-0.4.8-6.el9.noarch.rpm 2.3 MB/s | 159 kB 00:00 2024-09-14T06:27:06.362 INFO:teuthology.orchestra.run.smithi094.stdout:(4/13): ceph-mgr-rook-19.3.0-4775.g7f24e589.el9 240 kB/s | 49 kB 00:00 2024-09-14T06:27:06.367 INFO:teuthology.orchestra.run.smithi138.stdout:(3/12): flexiblas-netlib-3.0.4-8.el9.x86_64.rpm 16 MB/s | 3.0 MB 00:00 2024-09-14T06:27:06.396 INFO:teuthology.orchestra.run.smithi094.stdout:(5/13): python3-oauthlib-3.1.1-5.el9.noarch.rpm 1.4 MB/s | 222 kB 00:00 2024-09-14T06:27:06.408 INFO:teuthology.orchestra.run.smithi138.stdout:(4/12): flexiblas-openblas-openmp-3.0.4-8.el9.x 440 kB/s | 18 kB 00:00 2024-09-14T06:27:06.421 INFO:teuthology.orchestra.run.smithi094.stdout:(6/13): python3-requests-oauthlib-1.3.0-12.el9. 913 kB/s | 54 kB 00:00 2024-09-14T06:27:06.446 INFO:teuthology.orchestra.run.smithi094.stdout:(7/13): python3-pyasn1-modules-0.4.8-6.el9.noar 2.5 MB/s | 279 kB 00:00 2024-09-14T06:27:06.451 INFO:teuthology.orchestra.run.smithi138.stdout:(5/12): openblas-0.3.26-2.el9.x86_64.rpm 918 kB/s | 38 kB 00:00 2024-09-14T06:27:06.484 INFO:teuthology.orchestra.run.smithi138.stdout:(6/12): libquadmath-11.5.0-2.el9.x86_64.rpm 229 kB/s | 191 kB 00:00 2024-09-14T06:27:06.496 INFO:teuthology.orchestra.run.smithi094.stdout:(8/13): python3-cachetools-4.2.4-1.el9.noarch.r 320 kB/s | 32 kB 00:00 2024-09-14T06:27:06.522 INFO:teuthology.orchestra.run.smithi094.stdout:(9/13): python3-certifi-2023.05.07-4.el9.noarch 140 kB/s | 14 kB 00:00 2024-09-14T06:27:06.547 INFO:teuthology.orchestra.run.smithi094.stdout:(10/13): python3-google-auth-2.34.0-1.el9.noarc 2.5 MB/s | 220 kB 00:00 2024-09-14T06:27:06.581 INFO:teuthology.orchestra.run.smithi094.stdout:(11/13): python3-rsa-4.9-2.el9.noarch.rpm 1.0 MB/s | 59 kB 00:00 2024-09-14T06:27:06.585 INFO:teuthology.orchestra.run.smithi138.stdout:(7/12): openblas-openmp-0.3.26-2.el9.x86_64.rpm 37 MB/s | 4.9 MB 00:00 2024-09-14T06:27:06.606 INFO:teuthology.orchestra.run.smithi094.stdout:(12/13): python3-websocket-client-1.2.3-2.el9.n 1.5 MB/s | 90 kB 00:00 2024-09-14T06:27:06.627 INFO:teuthology.orchestra.run.smithi138.stdout:(8/12): libgfortran-11.5.0-2.el9.x86_64.rpm 821 kB/s | 801 kB 00:00 2024-09-14T06:27:06.648 INFO:teuthology.orchestra.run.smithi094.stdout:(13/13): python3-kubernetes-26.1.0-2.el9.noarch 6.8 MB/s | 1.0 MB 00:00 2024-09-14T06:27:06.650 INFO:teuthology.orchestra.run.smithi094.stdout:-------------------------------------------------------------------------------- 2024-09-14T06:27:06.650 INFO:teuthology.orchestra.run.smithi094.stdout:Total 3.2 MB/s | 2.2 MB 00:00 2024-09-14T06:27:06.712 INFO:teuthology.orchestra.run.smithi094.stdout:Running transaction check 2024-09-14T06:27:06.727 INFO:teuthology.orchestra.run.smithi094.stdout:Transaction check succeeded. 2024-09-14T06:27:06.728 INFO:teuthology.orchestra.run.smithi094.stdout:Running transaction test 2024-09-14T06:27:06.749 INFO:teuthology.orchestra.run.smithi138.stdout:(9/12): python3-numpy-1.20.1-5.el9.x86_64.rpm 31 MB/s | 5.1 MB 00:00 2024-09-14T06:27:06.786 INFO:teuthology.orchestra.run.smithi138.stdout:(10/12): python3-devel-3.9.19-8.el9.x86_64.rpm 813 kB/s | 245 kB 00:00 2024-09-14T06:27:06.817 INFO:teuthology.orchestra.run.smithi094.stdout:Transaction test succeeded. 2024-09-14T06:27:06.818 INFO:teuthology.orchestra.run.smithi094.stdout:Running transaction 2024-09-14T06:27:06.844 INFO:teuthology.orchestra.run.smithi138.stdout:(11/12): python3-numpy-f2py-1.20.1-5.el9.x86_64 1.1 MB/s | 250 kB 00:00 2024-09-14T06:27:07.053 INFO:teuthology.orchestra.run.smithi138.stdout:(12/12): python3-scipy-1.6.2-8.el9.x86_64.rpm 54 MB/s | 16 MB 00:00 2024-09-14T06:27:07.054 INFO:teuthology.orchestra.run.smithi138.stdout:-------------------------------------------------------------------------------- 2024-09-14T06:27:07.054 INFO:teuthology.orchestra.run.smithi138.stdout:Total 24 MB/s | 38 MB 00:01 2024-09-14T06:27:07.342 INFO:teuthology.orchestra.run.smithi094.stdout: Preparing : 1/1 2024-09-14T06:27:07.407 INFO:teuthology.orchestra.run.smithi138.stdout:Running transaction check 2024-09-14T06:27:07.434 INFO:teuthology.orchestra.run.smithi138.stdout:Transaction check succeeded. 2024-09-14T06:27:07.434 INFO:teuthology.orchestra.run.smithi138.stdout:Running transaction test 2024-09-14T06:27:07.521 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 1/13 2024-09-14T06:27:07.574 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 2/13 2024-09-14T06:27:07.631 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-rsa-4.9-2.el9.noarch 3/13 2024-09-14T06:27:07.667 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 4/13 2024-09-14T06:27:07.709 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-certifi-2023.05.07-4.el9.noarch 5/13 2024-09-14T06:27:07.811 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 6/13 2024-09-14T06:27:07.854 INFO:teuthology.orchestra.run.smithi138.stdout:Transaction test succeeded. 2024-09-14T06:27:07.855 INFO:teuthology.orchestra.run.smithi138.stdout:Running transaction 2024-09-14T06:27:07.920 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-google-auth-1:2.34.0-1.el9.noarch 7/13 2024-09-14T06:27:07.977 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 8/13 2024-09-14T06:27:08.634 INFO:teuthology.orchestra.run.smithi138.stdout: Preparing : 1/1 2024-09-14T06:27:08.702 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 9/13 2024-09-14T06:27:08.708 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 1/12 2024-09-14T06:27:08.750 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 10/13 2024-09-14T06:27:08.751 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : libquadmath-11.5.0-2.el9.x86_64 2/12 2024-09-14T06:27:08.783 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 11/13 2024-09-14T06:27:08.832 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 12/13 2024-09-14T06:27:08.855 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : libgfortran-11.5.0-2.el9.x86_64 3/12 2024-09-14T06:27:08.862 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : ceph-mgr-rook-2:19.3.0-4775.g7f24e589.el9.noarch 13/13 2024-09-14T06:27:08.903 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-devel-3.9.19-8.el9.x86_64 4/12 2024-09-14T06:27:09.090 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : openblas-0.3.26-2.el9.x86_64 5/12 2024-09-14T06:27:09.127 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : openblas-openmp-0.3.26-2.el9.x86_64 6/12 2024-09-14T06:27:09.204 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 7/12 2024-09-14T06:27:09.701 INFO:teuthology.orchestra.run.smithi094.stdout: Running scriptlet: ceph-mgr-rook-2:19.3.0-4775.g7f24e589.el9.noarch 13/13 2024-09-14T06:27:09.701 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : ceph-mgr-rook-2:19.3.0-4775.g7f24e589.el9.noarch 1/13 2024-09-14T06:27:09.701 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 2/13 2024-09-14T06:27:09.702 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 3/13 2024-09-14T06:27:09.702 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 4/13 2024-09-14T06:27:09.702 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 5/13 2024-09-14T06:27:09.702 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 6/13 2024-09-14T06:27:09.702 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 7/13 2024-09-14T06:27:09.702 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 8/13 2024-09-14T06:27:09.702 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-certifi-2023.05.07-4.el9.noarch 9/13 2024-09-14T06:27:09.702 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-google-auth-1:2.34.0-1.el9.noarch 10/13 2024-09-14T06:27:09.702 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 11/13 2024-09-14T06:27:09.702 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 12/13 2024-09-14T06:27:09.873 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 8/12 2024-09-14T06:27:10.001 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 9/12 2024-09-14T06:27:10.044 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 13/13 2024-09-14T06:27:10.044 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-14T06:27:10.045 INFO:teuthology.orchestra.run.smithi094.stdout:Installed: 2024-09-14T06:27:10.045 INFO:teuthology.orchestra.run.smithi094.stdout: ceph-mgr-rook-2:19.3.0-4775.g7f24e589.el9.noarch 2024-09-14T06:27:10.045 INFO:teuthology.orchestra.run.smithi094.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-09-14T06:27:10.045 INFO:teuthology.orchestra.run.smithi094.stdout: python3-certifi-2023.05.07-4.el9.noarch 2024-09-14T06:27:10.045 INFO:teuthology.orchestra.run.smithi094.stdout: python3-google-auth-1:2.34.0-1.el9.noarch 2024-09-14T06:27:10.045 INFO:teuthology.orchestra.run.smithi094.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-09-14T06:27:10.045 INFO:teuthology.orchestra.run.smithi094.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-09-14T06:27:10.045 INFO:teuthology.orchestra.run.smithi094.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-09-14T06:27:10.045 INFO:teuthology.orchestra.run.smithi094.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-09-14T06:27:10.045 INFO:teuthology.orchestra.run.smithi094.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-09-14T06:27:10.045 INFO:teuthology.orchestra.run.smithi094.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-09-14T06:27:10.045 INFO:teuthology.orchestra.run.smithi094.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-09-14T06:27:10.045 INFO:teuthology.orchestra.run.smithi094.stdout: python3-rsa-4.9-2.el9.noarch 2024-09-14T06:27:10.046 INFO:teuthology.orchestra.run.smithi094.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-09-14T06:27:10.046 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-14T06:27:10.046 INFO:teuthology.orchestra.run.smithi094.stdout:Complete! 2024-09-14T06:27:10.300 DEBUG:teuthology.orchestra.run.smithi094:> sudo yum -y install ceph-mgr-cephadm 2024-09-14T06:27:10.862 INFO:teuthology.orchestra.run.smithi094.stdout:Last metadata expiration check: 0:01:28 ago on Sat 14 Sep 2024 06:25:42 AM UTC. 2024-09-14T06:27:10.995 INFO:teuthology.orchestra.run.smithi094.stdout:Dependencies resolved. 2024-09-14T06:27:10.996 INFO:teuthology.orchestra.run.smithi094.stdout:================================================================================ 2024-09-14T06:27:10.996 INFO:teuthology.orchestra.run.smithi094.stdout: Package Arch Version Repository Size 2024-09-14T06:27:10.996 INFO:teuthology.orchestra.run.smithi094.stdout:================================================================================ 2024-09-14T06:27:10.996 INFO:teuthology.orchestra.run.smithi094.stdout:Installing: 2024-09-14T06:27:10.996 INFO:teuthology.orchestra.run.smithi094.stdout: ceph-mgr-cephadm noarch 2:19.3.0-4775.g7f24e589.el9 ceph-noarch 154 k 2024-09-14T06:27:10.996 INFO:teuthology.orchestra.run.smithi094.stdout:Installing dependencies: 2024-09-14T06:27:10.996 INFO:teuthology.orchestra.run.smithi094.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-09-14T06:27:10.996 INFO:teuthology.orchestra.run.smithi094.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-09-14T06:27:10.996 INFO:teuthology.orchestra.run.smithi094.stdout: python3-jinja2 noarch 2.11.3-6.el9 appstream 248 k 2024-09-14T06:27:10.997 INFO:teuthology.orchestra.run.smithi094.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-09-14T06:27:10.997 INFO:teuthology.orchestra.run.smithi094.stdout: python3-typing-extensions noarch 4.12.2-2.el9 epel 74 k 2024-09-14T06:27:10.997 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-14T06:27:10.997 INFO:teuthology.orchestra.run.smithi094.stdout:Transaction Summary 2024-09-14T06:27:10.997 INFO:teuthology.orchestra.run.smithi094.stdout:================================================================================ 2024-09-14T06:27:10.997 INFO:teuthology.orchestra.run.smithi094.stdout:Install 6 Packages 2024-09-14T06:27:10.997 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-14T06:27:10.997 INFO:teuthology.orchestra.run.smithi094.stdout:Total download size: 7.0 M 2024-09-14T06:27:10.997 INFO:teuthology.orchestra.run.smithi094.stdout:Installed size: 33 M 2024-09-14T06:27:10.998 INFO:teuthology.orchestra.run.smithi094.stdout:Downloading Packages: 2024-09-14T06:27:11.248 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 10/12 2024-09-14T06:27:11.418 INFO:teuthology.orchestra.run.smithi094.stdout:(1/6): python3-jinja2-2.11.3-6.el9.noarch.rpm 1.8 MB/s | 248 kB 00:00 2024-09-14T06:27:11.468 INFO:teuthology.orchestra.run.smithi094.stdout:(2/6): python3-asyncssh-2.13.2-5.el9.noarch.rpm 11 MB/s | 548 kB 00:00 2024-09-14T06:27:11.493 INFO:teuthology.orchestra.run.smithi094.stdout:(3/6): python3-natsort-7.1.1-5.el9.noarch.rpm 2.3 MB/s | 58 kB 00:00 2024-09-14T06:27:11.519 INFO:teuthology.orchestra.run.smithi094.stdout:(4/6): ceph-mgr-cephadm-19.3.0-4775.g7f24e589.e 645 kB/s | 154 kB 00:00 2024-09-14T06:27:11.544 INFO:teuthology.orchestra.run.smithi094.stdout:(5/6): python3-typing-extensions-4.12.2-2.el9.n 1.4 MB/s | 74 kB 00:00 2024-09-14T06:27:11.653 INFO:teuthology.orchestra.run.smithi094.stdout:(6/6): python3-babel-2.9.1-2.el9.noarch.rpm 16 MB/s | 6.0 MB 00:00 2024-09-14T06:27:11.655 INFO:teuthology.orchestra.run.smithi094.stdout:-------------------------------------------------------------------------------- 2024-09-14T06:27:11.655 INFO:teuthology.orchestra.run.smithi094.stdout:Total 11 MB/s | 7.0 MB 00:00 2024-09-14T06:27:11.726 INFO:teuthology.orchestra.run.smithi094.stdout:Running transaction check 2024-09-14T06:27:11.737 INFO:teuthology.orchestra.run.smithi094.stdout:Transaction check succeeded. 2024-09-14T06:27:11.737 INFO:teuthology.orchestra.run.smithi094.stdout:Running transaction test 2024-09-14T06:27:11.837 INFO:teuthology.orchestra.run.smithi094.stdout:Transaction test succeeded. 2024-09-14T06:27:11.838 INFO:teuthology.orchestra.run.smithi094.stdout:Running transaction 2024-09-14T06:27:12.146 INFO:teuthology.orchestra.run.smithi094.stdout: Preparing : 1/1 2024-09-14T06:27:12.300 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 11/12 2024-09-14T06:27:12.323 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : ceph-mgr-diskprediction-local-2:19.3.0-4775.g7f24e 12/12 2024-09-14T06:27:12.327 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-typing-extensions-4.12.2-2.el9.noarch 1/6 2024-09-14T06:27:12.379 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 2/6 2024-09-14T06:27:12.903 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 3/6 2024-09-14T06:27:12.982 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 4/6 2024-09-14T06:27:13.051 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-jinja2-2.11.3-6.el9.noarch 5/6 2024-09-14T06:27:13.084 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : ceph-mgr-cephadm-2:19.3.0-4775.g7f24e589.el9.noarch 6/6 2024-09-14T06:27:13.865 INFO:teuthology.orchestra.run.smithi094.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.3.0-4775.g7f24e589.el9.noarch 6/6 2024-09-14T06:27:13.865 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : ceph-mgr-cephadm-2:19.3.0-4775.g7f24e589.el9.noarch 1/6 2024-09-14T06:27:13.865 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 2/6 2024-09-14T06:27:13.865 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-jinja2-2.11.3-6.el9.noarch 3/6 2024-09-14T06:27:13.865 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 4/6 2024-09-14T06:27:13.865 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 5/6 2024-09-14T06:27:14.135 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-typing-extensions-4.12.2-2.el9.noarch 6/6 2024-09-14T06:27:14.135 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-14T06:27:14.135 INFO:teuthology.orchestra.run.smithi094.stdout:Installed: 2024-09-14T06:27:14.135 INFO:teuthology.orchestra.run.smithi094.stdout: ceph-mgr-cephadm-2:19.3.0-4775.g7f24e589.el9.noarch 2024-09-14T06:27:14.135 INFO:teuthology.orchestra.run.smithi094.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-09-14T06:27:14.135 INFO:teuthology.orchestra.run.smithi094.stdout: python3-babel-2.9.1-2.el9.noarch 2024-09-14T06:27:14.135 INFO:teuthology.orchestra.run.smithi094.stdout: python3-jinja2-2.11.3-6.el9.noarch 2024-09-14T06:27:14.135 INFO:teuthology.orchestra.run.smithi094.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-09-14T06:27:14.136 INFO:teuthology.orchestra.run.smithi094.stdout: python3-typing-extensions-4.12.2-2.el9.noarch 2024-09-14T06:27:14.136 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-14T06:27:14.136 INFO:teuthology.orchestra.run.smithi094.stdout:Complete! 2024-09-14T06:27:14.403 DEBUG:teuthology.orchestra.run.smithi094:> sudo yum -y install ceph-fuse 2024-09-14T06:27:14.967 INFO:teuthology.orchestra.run.smithi094.stdout:Last metadata expiration check: 0:01:32 ago on Sat 14 Sep 2024 06:25:42 AM UTC. 2024-09-14T06:27:15.032 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.3.0-4775.g7f24e 12/12 2024-09-14T06:27:15.032 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.3.0-4775.g7f24e 1/12 2024-09-14T06:27:15.032 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : libgfortran-11.5.0-2.el9.x86_64 2/12 2024-09-14T06:27:15.033 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : libquadmath-11.5.0-2.el9.x86_64 3/12 2024-09-14T06:27:15.033 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 4/12 2024-09-14T06:27:15.033 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 5/12 2024-09-14T06:27:15.033 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 6/12 2024-09-14T06:27:15.033 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : openblas-0.3.26-2.el9.x86_64 7/12 2024-09-14T06:27:15.033 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : openblas-openmp-0.3.26-2.el9.x86_64 8/12 2024-09-14T06:27:15.034 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-devel-3.9.19-8.el9.x86_64 9/12 2024-09-14T06:27:15.034 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 10/12 2024-09-14T06:27:15.034 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 11/12 2024-09-14T06:27:15.102 INFO:teuthology.orchestra.run.smithi094.stdout:Dependencies resolved. 2024-09-14T06:27:15.102 INFO:teuthology.orchestra.run.smithi094.stdout:================================================================================ 2024-09-14T06:27:15.102 INFO:teuthology.orchestra.run.smithi094.stdout: Package Arch Version Repository Size 2024-09-14T06:27:15.102 INFO:teuthology.orchestra.run.smithi094.stdout:================================================================================ 2024-09-14T06:27:15.102 INFO:teuthology.orchestra.run.smithi094.stdout:Installing: 2024-09-14T06:27:15.103 INFO:teuthology.orchestra.run.smithi094.stdout: ceph-fuse x86_64 2:19.3.0-4775.g7f24e589.el9 ceph 900 k 2024-09-14T06:27:15.103 INFO:teuthology.orchestra.run.smithi094.stdout:Installing dependencies: 2024-09-14T06:27:15.103 INFO:teuthology.orchestra.run.smithi094.stdout: fuse x86_64 2.9.9-16.el9 baseos 80 k 2024-09-14T06:27:15.103 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-14T06:27:15.103 INFO:teuthology.orchestra.run.smithi094.stdout:Transaction Summary 2024-09-14T06:27:15.103 INFO:teuthology.orchestra.run.smithi094.stdout:================================================================================ 2024-09-14T06:27:15.103 INFO:teuthology.orchestra.run.smithi094.stdout:Install 2 Packages 2024-09-14T06:27:15.103 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-14T06:27:15.104 INFO:teuthology.orchestra.run.smithi094.stdout:Total download size: 979 k 2024-09-14T06:27:15.104 INFO:teuthology.orchestra.run.smithi094.stdout:Installed size: 2.9 M 2024-09-14T06:27:15.104 INFO:teuthology.orchestra.run.smithi094.stdout:Downloading Packages: 2024-09-14T06:27:15.460 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 12/12 2024-09-14T06:27:15.460 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-09-14T06:27:15.460 INFO:teuthology.orchestra.run.smithi138.stdout:Installed: 2024-09-14T06:27:15.460 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-mgr-diskprediction-local-2:19.3.0-4775.g7f24e589.el9.noarch 2024-09-14T06:27:15.460 INFO:teuthology.orchestra.run.smithi138.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-09-14T06:27:15.461 INFO:teuthology.orchestra.run.smithi138.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-09-14T06:27:15.461 INFO:teuthology.orchestra.run.smithi138.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-09-14T06:27:15.461 INFO:teuthology.orchestra.run.smithi138.stdout: libgfortran-11.5.0-2.el9.x86_64 2024-09-14T06:27:15.461 INFO:teuthology.orchestra.run.smithi138.stdout: libquadmath-11.5.0-2.el9.x86_64 2024-09-14T06:27:15.461 INFO:teuthology.orchestra.run.smithi138.stdout: openblas-0.3.26-2.el9.x86_64 2024-09-14T06:27:15.461 INFO:teuthology.orchestra.run.smithi138.stdout: openblas-openmp-0.3.26-2.el9.x86_64 2024-09-14T06:27:15.461 INFO:teuthology.orchestra.run.smithi138.stdout: python3-devel-3.9.19-8.el9.x86_64 2024-09-14T06:27:15.461 INFO:teuthology.orchestra.run.smithi138.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-09-14T06:27:15.461 INFO:teuthology.orchestra.run.smithi138.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-09-14T06:27:15.461 INFO:teuthology.orchestra.run.smithi138.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-09-14T06:27:15.461 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-09-14T06:27:15.461 INFO:teuthology.orchestra.run.smithi138.stdout:Complete! 2024-09-14T06:27:15.539 INFO:teuthology.orchestra.run.smithi094.stdout:(1/2): fuse-2.9.9-16.el9.x86_64.rpm 511 kB/s | 80 kB 00:00 2024-09-14T06:27:15.681 INFO:teuthology.orchestra.run.smithi094.stdout:(2/2): ceph-fuse-19.3.0-4775.g7f24e589.el9.x86_ 2.9 MB/s | 900 kB 00:00 2024-09-14T06:27:15.681 INFO:teuthology.orchestra.run.smithi094.stdout:-------------------------------------------------------------------------------- 2024-09-14T06:27:15.682 INFO:teuthology.orchestra.run.smithi094.stdout:Total 1.7 MB/s | 979 kB 00:00 2024-09-14T06:27:15.693 INFO:teuthology.orchestra.run.smithi094.stdout:Running transaction check 2024-09-14T06:27:15.716 INFO:teuthology.orchestra.run.smithi094.stdout:Transaction check succeeded. 2024-09-14T06:27:15.716 INFO:teuthology.orchestra.run.smithi094.stdout:Running transaction test 2024-09-14T06:27:15.796 INFO:teuthology.orchestra.run.smithi094.stdout:Transaction test succeeded. 2024-09-14T06:27:15.796 INFO:teuthology.orchestra.run.smithi094.stdout:Running transaction 2024-09-14T06:27:15.805 DEBUG:teuthology.orchestra.run.smithi138:> sudo yum -y install ceph-mgr-rook 2024-09-14T06:27:15.981 INFO:teuthology.orchestra.run.smithi094.stdout: Preparing : 1/1 2024-09-14T06:27:16.117 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : fuse-2.9.9-16.el9.x86_64 1/2 2024-09-14T06:27:16.189 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : ceph-fuse-2:19.3.0-4775.g7f24e589.el9.x86_64 2/2 2024-09-14T06:27:16.386 INFO:teuthology.orchestra.run.smithi138.stdout:Last metadata expiration check: 0:01:20 ago on Sat 14 Sep 2024 06:25:56 AM UTC. 2024-09-14T06:27:16.522 INFO:teuthology.orchestra.run.smithi138.stdout:Dependencies resolved. 2024-09-14T06:27:16.523 INFO:teuthology.orchestra.run.smithi138.stdout:================================================================================ 2024-09-14T06:27:16.524 INFO:teuthology.orchestra.run.smithi138.stdout: Package Arch Version Repository Size 2024-09-14T06:27:16.524 INFO:teuthology.orchestra.run.smithi138.stdout:================================================================================ 2024-09-14T06:27:16.524 INFO:teuthology.orchestra.run.smithi138.stdout:Installing: 2024-09-14T06:27:16.524 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-mgr-rook noarch 2:19.3.0-4775.g7f24e589.el9 ceph-noarch 49 k 2024-09-14T06:27:16.524 INFO:teuthology.orchestra.run.smithi138.stdout:Installing dependencies: 2024-09-14T06:27:16.524 INFO:teuthology.orchestra.run.smithi138.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-09-14T06:27:16.524 INFO:teuthology.orchestra.run.smithi138.stdout: python3-certifi noarch 2023.05.07-4.el9 epel 14 k 2024-09-14T06:27:16.524 INFO:teuthology.orchestra.run.smithi138.stdout: python3-google-auth noarch 1:2.34.0-1.el9 epel 220 k 2024-09-14T06:27:16.524 INFO:teuthology.orchestra.run.smithi138.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-09-14T06:27:16.524 INFO:teuthology.orchestra.run.smithi138.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-09-14T06:27:16.524 INFO:teuthology.orchestra.run.smithi138.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-09-14T06:27:16.524 INFO:teuthology.orchestra.run.smithi138.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-09-14T06:27:16.524 INFO:teuthology.orchestra.run.smithi138.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-09-14T06:27:16.525 INFO:teuthology.orchestra.run.smithi138.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-09-14T06:27:16.525 INFO:teuthology.orchestra.run.smithi138.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-09-14T06:27:16.525 INFO:teuthology.orchestra.run.smithi138.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-09-14T06:27:16.525 INFO:teuthology.orchestra.run.smithi138.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-09-14T06:27:16.525 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-09-14T06:27:16.525 INFO:teuthology.orchestra.run.smithi138.stdout:Transaction Summary 2024-09-14T06:27:16.525 INFO:teuthology.orchestra.run.smithi138.stdout:================================================================================ 2024-09-14T06:27:16.525 INFO:teuthology.orchestra.run.smithi138.stdout:Install 13 Packages 2024-09-14T06:27:16.525 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-09-14T06:27:16.526 INFO:teuthology.orchestra.run.smithi138.stdout:Total download size: 2.2 M 2024-09-14T06:27:16.526 INFO:teuthology.orchestra.run.smithi138.stdout:Installed size: 26 M 2024-09-14T06:27:16.526 INFO:teuthology.orchestra.run.smithi138.stdout:Downloading Packages: 2024-09-14T06:27:16.879 INFO:teuthology.orchestra.run.smithi094.stdout: Running scriptlet: ceph-fuse-2:19.3.0-4775.g7f24e589.el9.x86_64 2/2 2024-09-14T06:27:16.879 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : ceph-fuse-2:19.3.0-4775.g7f24e589.el9.x86_64 1/2 2024-09-14T06:27:17.029 INFO:teuthology.orchestra.run.smithi138.stdout:(1/13): ceph-mgr-rook-19.3.0-4775.g7f24e589.el9 292 kB/s | 49 kB 00:00 2024-09-14T06:27:17.141 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : fuse-2.9.9-16.el9.x86_64 2/2 2024-09-14T06:27:17.141 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-14T06:27:17.141 INFO:teuthology.orchestra.run.smithi094.stdout:Installed: 2024-09-14T06:27:17.141 INFO:teuthology.orchestra.run.smithi094.stdout: ceph-fuse-2:19.3.0-4775.g7f24e589.el9.x86_64 fuse-2.9.9-16.el9.x86_64 2024-09-14T06:27:17.141 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-14T06:27:17.141 INFO:teuthology.orchestra.run.smithi094.stdout:Complete! 2024-09-14T06:27:17.146 INFO:teuthology.orchestra.run.smithi138.stdout:(2/13): python3-jsonpatch-1.21-16.el9.noarch.rp 90 kB/s | 26 kB 00:00 2024-09-14T06:27:17.171 INFO:teuthology.orchestra.run.smithi138.stdout:(3/13): python3-jsonpointer-2.0-4.el9.noarch.rp 62 kB/s | 19 kB 00:00 2024-09-14T06:27:17.417 DEBUG:teuthology.orchestra.run.smithi094:> sudo yum -y install ceph-volume 2024-09-14T06:27:17.521 INFO:teuthology.orchestra.run.smithi138.stdout:(4/13): python3-pyasn1-0.4.8-6.el9.noarch.rpm 423 kB/s | 159 kB 00:00 2024-09-14T06:27:17.580 INFO:teuthology.orchestra.run.smithi138.stdout:(5/13): python3-oauthlib-3.1.1-5.el9.noarch.rpm 403 kB/s | 222 kB 00:00 2024-09-14T06:27:17.622 INFO:teuthology.orchestra.run.smithi138.stdout:(6/13): python3-cachetools-4.2.4-1.el9.noarch.r 774 kB/s | 32 kB 00:00 2024-09-14T06:27:17.647 INFO:teuthology.orchestra.run.smithi138.stdout:(7/13): python3-requests-oauthlib-1.3.0-12.el9. 427 kB/s | 54 kB 00:00 2024-09-14T06:27:17.672 INFO:teuthology.orchestra.run.smithi138.stdout:(8/13): python3-certifi-2023.05.07-4.el9.noarch 281 kB/s | 14 kB 00:00 2024-09-14T06:27:17.715 INFO:teuthology.orchestra.run.smithi138.stdout:(9/13): python3-pyasn1-modules-0.4.8-6.el9.noar 513 kB/s | 279 kB 00:00 2024-09-14T06:27:17.756 INFO:teuthology.orchestra.run.smithi138.stdout:(10/13): python3-google-auth-2.34.0-1.el9.noarc 2.0 MB/s | 220 kB 00:00 2024-09-14T06:27:17.782 INFO:teuthology.orchestra.run.smithi138.stdout:(11/13): python3-rsa-4.9-2.el9.noarch.rpm 909 kB/s | 59 kB 00:00 2024-09-14T06:27:17.807 INFO:teuthology.orchestra.run.smithi138.stdout:(12/13): python3-websocket-client-1.2.3-2.el9.n 1.8 MB/s | 90 kB 00:00 2024-09-14T06:27:17.840 INFO:teuthology.orchestra.run.smithi138.stdout:(13/13): python3-kubernetes-26.1.0-2.el9.noarch 6.4 MB/s | 1.0 MB 00:00 2024-09-14T06:27:17.844 INFO:teuthology.orchestra.run.smithi138.stdout:-------------------------------------------------------------------------------- 2024-09-14T06:27:17.844 INFO:teuthology.orchestra.run.smithi138.stdout:Total 1.7 MB/s | 2.2 MB 00:01 2024-09-14T06:27:17.916 INFO:teuthology.orchestra.run.smithi138.stdout:Running transaction check 2024-09-14T06:27:17.932 INFO:teuthology.orchestra.run.smithi138.stdout:Transaction check succeeded. 2024-09-14T06:27:17.932 INFO:teuthology.orchestra.run.smithi138.stdout:Running transaction test 2024-09-14T06:27:17.988 INFO:teuthology.orchestra.run.smithi094.stdout:Last metadata expiration check: 0:01:35 ago on Sat 14 Sep 2024 06:25:42 AM UTC. 2024-09-14T06:27:18.022 INFO:teuthology.orchestra.run.smithi138.stdout:Transaction test succeeded. 2024-09-14T06:27:18.024 INFO:teuthology.orchestra.run.smithi138.stdout:Running transaction 2024-09-14T06:27:18.128 INFO:teuthology.orchestra.run.smithi094.stdout:Dependencies resolved. 2024-09-14T06:27:18.128 INFO:teuthology.orchestra.run.smithi094.stdout:================================================================================ 2024-09-14T06:27:18.129 INFO:teuthology.orchestra.run.smithi094.stdout: Package Arch Version Repository Size 2024-09-14T06:27:18.129 INFO:teuthology.orchestra.run.smithi094.stdout:================================================================================ 2024-09-14T06:27:18.129 INFO:teuthology.orchestra.run.smithi094.stdout:Installing: 2024-09-14T06:27:18.129 INFO:teuthology.orchestra.run.smithi094.stdout: ceph-volume noarch 2:19.3.0-4775.g7f24e589.el9 ceph-noarch 285 k 2024-09-14T06:27:18.129 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-14T06:27:18.129 INFO:teuthology.orchestra.run.smithi094.stdout:Transaction Summary 2024-09-14T06:27:18.129 INFO:teuthology.orchestra.run.smithi094.stdout:================================================================================ 2024-09-14T06:27:18.129 INFO:teuthology.orchestra.run.smithi094.stdout:Install 1 Package 2024-09-14T06:27:18.129 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-14T06:27:18.129 INFO:teuthology.orchestra.run.smithi094.stdout:Total download size: 285 k 2024-09-14T06:27:18.129 INFO:teuthology.orchestra.run.smithi094.stdout:Installed size: 1.3 M 2024-09-14T06:27:18.129 INFO:teuthology.orchestra.run.smithi094.stdout:Downloading Packages: 2024-09-14T06:27:18.337 INFO:teuthology.orchestra.run.smithi094.stdout:ceph-volume-19.3.0-4775.g7f24e589.el9.noarch.rp 1.3 MB/s | 285 kB 00:00 2024-09-14T06:27:18.337 INFO:teuthology.orchestra.run.smithi094.stdout:-------------------------------------------------------------------------------- 2024-09-14T06:27:18.338 INFO:teuthology.orchestra.run.smithi094.stdout:Total 1.3 MB/s | 285 kB 00:00 2024-09-14T06:27:18.338 INFO:teuthology.orchestra.run.smithi094.stdout:Running transaction check 2024-09-14T06:27:18.346 INFO:teuthology.orchestra.run.smithi094.stdout:Transaction check succeeded. 2024-09-14T06:27:18.346 INFO:teuthology.orchestra.run.smithi094.stdout:Running transaction test 2024-09-14T06:27:18.374 INFO:teuthology.orchestra.run.smithi094.stdout:Transaction test succeeded. 2024-09-14T06:27:18.374 INFO:teuthology.orchestra.run.smithi094.stdout:Running transaction 2024-09-14T06:27:18.525 INFO:teuthology.orchestra.run.smithi138.stdout: Preparing : 1/1 2024-09-14T06:27:18.597 INFO:teuthology.orchestra.run.smithi094.stdout: Preparing : 1/1 2024-09-14T06:27:18.664 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : ceph-volume-2:19.3.0-4775.g7f24e589.el9.noarch 1/1 2024-09-14T06:27:18.680 INFO:teuthology.orchestra.run.smithi094.stdout: Running scriptlet: ceph-volume-2:19.3.0-4775.g7f24e589.el9.noarch 1/1 2024-09-14T06:27:18.681 INFO:teuthology.orchestra.run.smithi094.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-14T06:27:18.681 INFO:teuthology.orchestra.run.smithi094.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-09-14T06:27:18.681 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-14T06:27:18.712 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 1/13 2024-09-14T06:27:18.767 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 2/13 2024-09-14T06:27:18.822 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-rsa-4.9-2.el9.noarch 3/13 2024-09-14T06:27:18.868 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 4/13 2024-09-14T06:27:18.918 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-certifi-2023.05.07-4.el9.noarch 5/13 2024-09-14T06:27:19.037 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 6/13 2024-09-14T06:27:19.138 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-google-auth-1:2.34.0-1.el9.noarch 7/13 2024-09-14T06:27:19.187 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 8/13 2024-09-14T06:27:19.692 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : ceph-volume-2:19.3.0-4775.g7f24e589.el9.noarch 1/1 2024-09-14T06:27:19.692 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-14T06:27:19.692 INFO:teuthology.orchestra.run.smithi094.stdout:Installed: 2024-09-14T06:27:19.692 INFO:teuthology.orchestra.run.smithi094.stdout: ceph-volume-2:19.3.0-4775.g7f24e589.el9.noarch 2024-09-14T06:27:19.692 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-14T06:27:19.692 INFO:teuthology.orchestra.run.smithi094.stdout:Complete! 2024-09-14T06:27:19.909 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 9/13 2024-09-14T06:27:19.961 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 10/13 2024-09-14T06:27:19.970 DEBUG:teuthology.orchestra.run.smithi094:> sudo yum -y install librados-devel 2024-09-14T06:27:20.003 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 11/13 2024-09-14T06:27:20.051 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 12/13 2024-09-14T06:27:20.073 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : ceph-mgr-rook-2:19.3.0-4775.g7f24e589.el9.noarch 13/13 2024-09-14T06:27:20.555 INFO:teuthology.orchestra.run.smithi094.stdout:Last metadata expiration check: 0:01:38 ago on Sat 14 Sep 2024 06:25:42 AM UTC. 2024-09-14T06:27:20.690 INFO:teuthology.orchestra.run.smithi094.stdout:Dependencies resolved. 2024-09-14T06:27:20.691 INFO:teuthology.orchestra.run.smithi094.stdout:================================================================================ 2024-09-14T06:27:20.692 INFO:teuthology.orchestra.run.smithi094.stdout: Package Arch Version Repo Size 2024-09-14T06:27:20.692 INFO:teuthology.orchestra.run.smithi094.stdout:================================================================================ 2024-09-14T06:27:20.692 INFO:teuthology.orchestra.run.smithi094.stdout:Installing: 2024-09-14T06:27:20.692 INFO:teuthology.orchestra.run.smithi094.stdout: librados-devel x86_64 2:19.3.0-4775.g7f24e589.el9 ceph 126 k 2024-09-14T06:27:20.692 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-14T06:27:20.692 INFO:teuthology.orchestra.run.smithi094.stdout:Transaction Summary 2024-09-14T06:27:20.693 INFO:teuthology.orchestra.run.smithi094.stdout:================================================================================ 2024-09-14T06:27:20.693 INFO:teuthology.orchestra.run.smithi094.stdout:Install 1 Package 2024-09-14T06:27:20.693 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-14T06:27:20.693 INFO:teuthology.orchestra.run.smithi094.stdout:Total download size: 126 k 2024-09-14T06:27:20.693 INFO:teuthology.orchestra.run.smithi094.stdout:Installed size: 456 k 2024-09-14T06:27:20.693 INFO:teuthology.orchestra.run.smithi094.stdout:Downloading Packages: 2024-09-14T06:27:20.841 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: ceph-mgr-rook-2:19.3.0-4775.g7f24e589.el9.noarch 13/13 2024-09-14T06:27:20.841 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : ceph-mgr-rook-2:19.3.0-4775.g7f24e589.el9.noarch 1/13 2024-09-14T06:27:20.842 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 2/13 2024-09-14T06:27:20.842 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 3/13 2024-09-14T06:27:20.842 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 4/13 2024-09-14T06:27:20.842 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 5/13 2024-09-14T06:27:20.842 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 6/13 2024-09-14T06:27:20.842 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 7/13 2024-09-14T06:27:20.842 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 8/13 2024-09-14T06:27:20.842 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-certifi-2023.05.07-4.el9.noarch 9/13 2024-09-14T06:27:20.842 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-google-auth-1:2.34.0-1.el9.noarch 10/13 2024-09-14T06:27:20.842 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 11/13 2024-09-14T06:27:20.842 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 12/13 2024-09-14T06:27:20.902 INFO:teuthology.orchestra.run.smithi094.stdout:librados-devel-19.3.0-4775.g7f24e589.el9.x86_64 602 kB/s | 126 kB 00:00 2024-09-14T06:27:20.902 INFO:teuthology.orchestra.run.smithi094.stdout:-------------------------------------------------------------------------------- 2024-09-14T06:27:20.902 INFO:teuthology.orchestra.run.smithi094.stdout:Total 599 kB/s | 126 kB 00:00 2024-09-14T06:27:20.903 INFO:teuthology.orchestra.run.smithi094.stdout:Running transaction check 2024-09-14T06:27:20.908 INFO:teuthology.orchestra.run.smithi094.stdout:Transaction check succeeded. 2024-09-14T06:27:20.908 INFO:teuthology.orchestra.run.smithi094.stdout:Running transaction test 2024-09-14T06:27:20.971 INFO:teuthology.orchestra.run.smithi094.stdout:Transaction test succeeded. 2024-09-14T06:27:20.972 INFO:teuthology.orchestra.run.smithi094.stdout:Running transaction 2024-09-14T06:27:21.121 INFO:teuthology.orchestra.run.smithi094.stdout: Preparing : 1/1 2024-09-14T06:27:21.165 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 13/13 2024-09-14T06:27:21.165 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-09-14T06:27:21.166 INFO:teuthology.orchestra.run.smithi138.stdout:Installed: 2024-09-14T06:27:21.166 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-mgr-rook-2:19.3.0-4775.g7f24e589.el9.noarch 2024-09-14T06:27:21.166 INFO:teuthology.orchestra.run.smithi138.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-09-14T06:27:21.166 INFO:teuthology.orchestra.run.smithi138.stdout: python3-certifi-2023.05.07-4.el9.noarch 2024-09-14T06:27:21.166 INFO:teuthology.orchestra.run.smithi138.stdout: python3-google-auth-1:2.34.0-1.el9.noarch 2024-09-14T06:27:21.166 INFO:teuthology.orchestra.run.smithi138.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-09-14T06:27:21.166 INFO:teuthology.orchestra.run.smithi138.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-09-14T06:27:21.166 INFO:teuthology.orchestra.run.smithi138.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-09-14T06:27:21.166 INFO:teuthology.orchestra.run.smithi138.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-09-14T06:27:21.166 INFO:teuthology.orchestra.run.smithi138.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-09-14T06:27:21.166 INFO:teuthology.orchestra.run.smithi138.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-09-14T06:27:21.166 INFO:teuthology.orchestra.run.smithi138.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-09-14T06:27:21.166 INFO:teuthology.orchestra.run.smithi138.stdout: python3-rsa-4.9-2.el9.noarch 2024-09-14T06:27:21.166 INFO:teuthology.orchestra.run.smithi138.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-09-14T06:27:21.167 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-09-14T06:27:21.167 INFO:teuthology.orchestra.run.smithi138.stdout:Complete! 2024-09-14T06:27:21.225 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : librados-devel-2:19.3.0-4775.g7f24e589.el9.x86_64 1/1 2024-09-14T06:27:21.426 DEBUG:teuthology.orchestra.run.smithi138:> sudo yum -y install ceph-mgr-cephadm 2024-09-14T06:27:21.667 INFO:teuthology.orchestra.run.smithi094.stdout: Running scriptlet: librados-devel-2:19.3.0-4775.g7f24e589.el9.x86_64 1/1 2024-09-14T06:27:21.946 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : librados-devel-2:19.3.0-4775.g7f24e589.el9.x86_64 1/1 2024-09-14T06:27:21.946 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-14T06:27:21.946 INFO:teuthology.orchestra.run.smithi094.stdout:Installed: 2024-09-14T06:27:21.946 INFO:teuthology.orchestra.run.smithi094.stdout: librados-devel-2:19.3.0-4775.g7f24e589.el9.x86_64 2024-09-14T06:27:21.946 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-14T06:27:21.946 INFO:teuthology.orchestra.run.smithi094.stdout:Complete! 2024-09-14T06:27:21.986 INFO:teuthology.orchestra.run.smithi138.stdout:Last metadata expiration check: 0:01:25 ago on Sat 14 Sep 2024 06:25:56 AM UTC. 2024-09-14T06:27:22.119 INFO:teuthology.orchestra.run.smithi138.stdout:Dependencies resolved. 2024-09-14T06:27:22.120 INFO:teuthology.orchestra.run.smithi138.stdout:================================================================================ 2024-09-14T06:27:22.120 INFO:teuthology.orchestra.run.smithi138.stdout: Package Arch Version Repository Size 2024-09-14T06:27:22.120 INFO:teuthology.orchestra.run.smithi138.stdout:================================================================================ 2024-09-14T06:27:22.120 INFO:teuthology.orchestra.run.smithi138.stdout:Installing: 2024-09-14T06:27:22.120 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-mgr-cephadm noarch 2:19.3.0-4775.g7f24e589.el9 ceph-noarch 154 k 2024-09-14T06:27:22.120 INFO:teuthology.orchestra.run.smithi138.stdout:Installing dependencies: 2024-09-14T06:27:22.120 INFO:teuthology.orchestra.run.smithi138.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-09-14T06:27:22.120 INFO:teuthology.orchestra.run.smithi138.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-09-14T06:27:22.120 INFO:teuthology.orchestra.run.smithi138.stdout: python3-jinja2 noarch 2.11.3-6.el9 appstream 248 k 2024-09-14T06:27:22.121 INFO:teuthology.orchestra.run.smithi138.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-09-14T06:27:22.121 INFO:teuthology.orchestra.run.smithi138.stdout: python3-typing-extensions noarch 4.12.2-2.el9 epel 74 k 2024-09-14T06:27:22.121 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-09-14T06:27:22.121 INFO:teuthology.orchestra.run.smithi138.stdout:Transaction Summary 2024-09-14T06:27:22.121 INFO:teuthology.orchestra.run.smithi138.stdout:================================================================================ 2024-09-14T06:27:22.121 INFO:teuthology.orchestra.run.smithi138.stdout:Install 6 Packages 2024-09-14T06:27:22.121 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-09-14T06:27:22.121 INFO:teuthology.orchestra.run.smithi138.stdout:Total download size: 7.0 M 2024-09-14T06:27:22.122 INFO:teuthology.orchestra.run.smithi138.stdout:Installed size: 33 M 2024-09-14T06:27:22.122 INFO:teuthology.orchestra.run.smithi138.stdout:Downloading Packages: 2024-09-14T06:27:22.205 DEBUG:teuthology.orchestra.run.smithi094:> sudo yum -y install libcephfs2 2024-09-14T06:27:22.655 INFO:teuthology.orchestra.run.smithi138.stdout:(1/6): python3-jinja2-2.11.3-6.el9.noarch.rpm 2.0 MB/s | 248 kB 00:00 2024-09-14T06:27:22.705 INFO:teuthology.orchestra.run.smithi138.stdout:(2/6): python3-asyncssh-2.13.2-5.el9.noarch.rpm 11 MB/s | 548 kB 00:00 2024-09-14T06:27:22.731 INFO:teuthology.orchestra.run.smithi138.stdout:(3/6): python3-natsort-7.1.1-5.el9.noarch.rpm 2.2 MB/s | 58 kB 00:00 2024-09-14T06:27:22.756 INFO:teuthology.orchestra.run.smithi138.stdout:(4/6): ceph-mgr-cephadm-19.3.0-4775.g7f24e589.e 697 kB/s | 154 kB 00:00 2024-09-14T06:27:22.781 INFO:teuthology.orchestra.run.smithi138.stdout:(5/6): python3-typing-extensions-4.12.2-2.el9.n 1.4 MB/s | 74 kB 00:00 2024-09-14T06:27:22.794 INFO:teuthology.orchestra.run.smithi094.stdout:Last metadata expiration check: 0:01:40 ago on Sat 14 Sep 2024 06:25:42 AM UTC. 2024-09-14T06:27:22.890 INFO:teuthology.orchestra.run.smithi094.stdout:Package libcephfs2-2:19.3.0-4775.g7f24e589.el9.x86_64 is already installed. 2024-09-14T06:27:22.936 INFO:teuthology.orchestra.run.smithi094.stdout:Dependencies resolved. 2024-09-14T06:27:22.937 INFO:teuthology.orchestra.run.smithi094.stdout:Nothing to do. 2024-09-14T06:27:22.937 INFO:teuthology.orchestra.run.smithi094.stdout:Complete! 2024-09-14T06:27:22.999 DEBUG:teuthology.orchestra.run.smithi094:> sudo yum -y install libcephfs-devel 2024-09-14T06:27:23.569 INFO:teuthology.orchestra.run.smithi094.stdout:Last metadata expiration check: 0:01:41 ago on Sat 14 Sep 2024 06:25:42 AM UTC. 2024-09-14T06:27:23.703 INFO:teuthology.orchestra.run.smithi094.stdout:Dependencies resolved. 2024-09-14T06:27:23.704 INFO:teuthology.orchestra.run.smithi094.stdout:================================================================================ 2024-09-14T06:27:23.704 INFO:teuthology.orchestra.run.smithi094.stdout: Package Arch Version Repo Size 2024-09-14T06:27:23.704 INFO:teuthology.orchestra.run.smithi094.stdout:================================================================================ 2024-09-14T06:27:23.705 INFO:teuthology.orchestra.run.smithi094.stdout:Installing: 2024-09-14T06:27:23.705 INFO:teuthology.orchestra.run.smithi094.stdout: libcephfs-devel x86_64 2:19.3.0-4775.g7f24e589.el9 ceph 32 k 2024-09-14T06:27:23.705 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-14T06:27:23.705 INFO:teuthology.orchestra.run.smithi094.stdout:Transaction Summary 2024-09-14T06:27:23.705 INFO:teuthology.orchestra.run.smithi094.stdout:================================================================================ 2024-09-14T06:27:23.705 INFO:teuthology.orchestra.run.smithi094.stdout:Install 1 Package 2024-09-14T06:27:23.705 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-14T06:27:23.705 INFO:teuthology.orchestra.run.smithi094.stdout:Total download size: 32 k 2024-09-14T06:27:23.705 INFO:teuthology.orchestra.run.smithi094.stdout:Installed size: 142 k 2024-09-14T06:27:23.705 INFO:teuthology.orchestra.run.smithi094.stdout:Downloading Packages: 2024-09-14T06:27:23.898 INFO:teuthology.orchestra.run.smithi094.stdout:libcephfs-devel-19.3.0-4775.g7f24e589.el9.x86_6 164 kB/s | 32 kB 00:00 2024-09-14T06:27:23.899 INFO:teuthology.orchestra.run.smithi094.stdout:-------------------------------------------------------------------------------- 2024-09-14T06:27:23.899 INFO:teuthology.orchestra.run.smithi094.stdout:Total 163 kB/s | 32 kB 00:00 2024-09-14T06:27:23.899 INFO:teuthology.orchestra.run.smithi094.stdout:Running transaction check 2024-09-14T06:27:23.903 INFO:teuthology.orchestra.run.smithi094.stdout:Transaction check succeeded. 2024-09-14T06:27:23.903 INFO:teuthology.orchestra.run.smithi094.stdout:Running transaction test 2024-09-14T06:27:23.920 INFO:teuthology.orchestra.run.smithi094.stdout:Transaction test succeeded. 2024-09-14T06:27:23.920 INFO:teuthology.orchestra.run.smithi094.stdout:Running transaction 2024-09-14T06:27:23.973 INFO:teuthology.orchestra.run.smithi138.stdout:(6/6): python3-babel-2.9.1-2.el9.noarch.rpm 4.1 MB/s | 6.0 MB 00:01 2024-09-14T06:27:23.976 INFO:teuthology.orchestra.run.smithi138.stdout:-------------------------------------------------------------------------------- 2024-09-14T06:27:23.976 INFO:teuthology.orchestra.run.smithi138.stdout:Total 3.8 MB/s | 7.0 MB 00:01 2024-09-14T06:27:24.002 INFO:teuthology.orchestra.run.smithi094.stdout: Preparing : 1/1 2024-09-14T06:27:24.049 INFO:teuthology.orchestra.run.smithi138.stdout:Running transaction check 2024-09-14T06:27:24.061 INFO:teuthology.orchestra.run.smithi138.stdout:Transaction check succeeded. 2024-09-14T06:27:24.062 INFO:teuthology.orchestra.run.smithi138.stdout:Running transaction test 2024-09-14T06:27:24.138 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : libcephfs-devel-2:19.3.0-4775.g7f24e589.el9.x86_64 1/1 2024-09-14T06:27:24.161 INFO:teuthology.orchestra.run.smithi138.stdout:Transaction test succeeded. 2024-09-14T06:27:24.163 INFO:teuthology.orchestra.run.smithi138.stdout:Running transaction 2024-09-14T06:27:24.464 INFO:teuthology.orchestra.run.smithi138.stdout: Preparing : 1/1 2024-09-14T06:27:24.555 INFO:teuthology.orchestra.run.smithi094.stdout: Running scriptlet: libcephfs-devel-2:19.3.0-4775.g7f24e589.el9.x86_64 1/1 2024-09-14T06:27:24.638 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-typing-extensions-4.12.2-2.el9.noarch 1/6 2024-09-14T06:27:24.692 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 2/6 2024-09-14T06:27:24.798 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : libcephfs-devel-2:19.3.0-4775.g7f24e589.el9.x86_64 1/1 2024-09-14T06:27:24.798 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-14T06:27:24.798 INFO:teuthology.orchestra.run.smithi094.stdout:Installed: 2024-09-14T06:27:24.798 INFO:teuthology.orchestra.run.smithi094.stdout: libcephfs-devel-2:19.3.0-4775.g7f24e589.el9.x86_64 2024-09-14T06:27:24.798 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-14T06:27:24.798 INFO:teuthology.orchestra.run.smithi094.stdout:Complete! 2024-09-14T06:27:24.978 DEBUG:teuthology.orchestra.run.smithi094:> sudo yum -y install librados2 2024-09-14T06:27:25.215 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 3/6 2024-09-14T06:27:25.312 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 4/6 2024-09-14T06:27:25.388 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-jinja2-2.11.3-6.el9.noarch 5/6 2024-09-14T06:27:25.414 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : ceph-mgr-cephadm-2:19.3.0-4775.g7f24e589.el9.noarch 6/6 2024-09-14T06:27:25.545 INFO:teuthology.orchestra.run.smithi094.stdout:Last metadata expiration check: 0:01:43 ago on Sat 14 Sep 2024 06:25:42 AM UTC. 2024-09-14T06:27:25.636 INFO:teuthology.orchestra.run.smithi094.stdout:Package librados2-2:19.3.0-4775.g7f24e589.el9.x86_64 is already installed. 2024-09-14T06:27:25.680 INFO:teuthology.orchestra.run.smithi094.stdout:Dependencies resolved. 2024-09-14T06:27:25.681 INFO:teuthology.orchestra.run.smithi094.stdout:Nothing to do. 2024-09-14T06:27:25.681 INFO:teuthology.orchestra.run.smithi094.stdout:Complete! 2024-09-14T06:27:25.743 DEBUG:teuthology.orchestra.run.smithi094:> sudo yum -y install librbd1 2024-09-14T06:27:26.183 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.3.0-4775.g7f24e589.el9.noarch 6/6 2024-09-14T06:27:26.183 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : ceph-mgr-cephadm-2:19.3.0-4775.g7f24e589.el9.noarch 1/6 2024-09-14T06:27:26.183 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 2/6 2024-09-14T06:27:26.183 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-jinja2-2.11.3-6.el9.noarch 3/6 2024-09-14T06:27:26.183 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 4/6 2024-09-14T06:27:26.183 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 5/6 2024-09-14T06:27:26.309 INFO:teuthology.orchestra.run.smithi094.stdout:Last metadata expiration check: 0:01:44 ago on Sat 14 Sep 2024 06:25:42 AM UTC. 2024-09-14T06:27:26.399 INFO:teuthology.orchestra.run.smithi094.stdout:Package librbd1-2:19.3.0-4775.g7f24e589.el9.x86_64 is already installed. 2024-09-14T06:27:26.442 INFO:teuthology.orchestra.run.smithi094.stdout:Dependencies resolved. 2024-09-14T06:27:26.444 INFO:teuthology.orchestra.run.smithi094.stdout:Nothing to do. 2024-09-14T06:27:26.444 INFO:teuthology.orchestra.run.smithi094.stdout:Complete! 2024-09-14T06:27:26.465 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-typing-extensions-4.12.2-2.el9.noarch 6/6 2024-09-14T06:27:26.465 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-09-14T06:27:26.465 INFO:teuthology.orchestra.run.smithi138.stdout:Installed: 2024-09-14T06:27:26.465 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-mgr-cephadm-2:19.3.0-4775.g7f24e589.el9.noarch 2024-09-14T06:27:26.465 INFO:teuthology.orchestra.run.smithi138.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-09-14T06:27:26.465 INFO:teuthology.orchestra.run.smithi138.stdout: python3-babel-2.9.1-2.el9.noarch 2024-09-14T06:27:26.465 INFO:teuthology.orchestra.run.smithi138.stdout: python3-jinja2-2.11.3-6.el9.noarch 2024-09-14T06:27:26.465 INFO:teuthology.orchestra.run.smithi138.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-09-14T06:27:26.465 INFO:teuthology.orchestra.run.smithi138.stdout: python3-typing-extensions-4.12.2-2.el9.noarch 2024-09-14T06:27:26.465 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-09-14T06:27:26.465 INFO:teuthology.orchestra.run.smithi138.stdout:Complete! 2024-09-14T06:27:26.511 DEBUG:teuthology.orchestra.run.smithi094:> sudo yum -y install python3-rados 2024-09-14T06:27:26.756 DEBUG:teuthology.orchestra.run.smithi138:> sudo yum -y install ceph-fuse 2024-09-14T06:27:27.074 INFO:teuthology.orchestra.run.smithi094.stdout:Last metadata expiration check: 0:01:45 ago on Sat 14 Sep 2024 06:25:42 AM UTC. 2024-09-14T06:27:27.165 INFO:teuthology.orchestra.run.smithi094.stdout:Package python3-rados-2:19.3.0-4775.g7f24e589.el9.x86_64 is already installed. 2024-09-14T06:27:27.207 INFO:teuthology.orchestra.run.smithi094.stdout:Dependencies resolved. 2024-09-14T06:27:27.208 INFO:teuthology.orchestra.run.smithi094.stdout:Nothing to do. 2024-09-14T06:27:27.209 INFO:teuthology.orchestra.run.smithi094.stdout:Complete! 2024-09-14T06:27:27.265 DEBUG:teuthology.orchestra.run.smithi094:> sudo yum -y install python3-rgw 2024-09-14T06:27:27.330 INFO:teuthology.orchestra.run.smithi138.stdout:Last metadata expiration check: 0:01:31 ago on Sat 14 Sep 2024 06:25:56 AM UTC. 2024-09-14T06:27:27.476 INFO:teuthology.orchestra.run.smithi138.stdout:Dependencies resolved. 2024-09-14T06:27:27.477 INFO:teuthology.orchestra.run.smithi138.stdout:================================================================================ 2024-09-14T06:27:27.477 INFO:teuthology.orchestra.run.smithi138.stdout: Package Arch Version Repository Size 2024-09-14T06:27:27.477 INFO:teuthology.orchestra.run.smithi138.stdout:================================================================================ 2024-09-14T06:27:27.477 INFO:teuthology.orchestra.run.smithi138.stdout:Installing: 2024-09-14T06:27:27.477 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-fuse x86_64 2:19.3.0-4775.g7f24e589.el9 ceph 900 k 2024-09-14T06:27:27.477 INFO:teuthology.orchestra.run.smithi138.stdout:Installing dependencies: 2024-09-14T06:27:27.477 INFO:teuthology.orchestra.run.smithi138.stdout: fuse x86_64 2.9.9-16.el9 baseos 80 k 2024-09-14T06:27:27.477 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-09-14T06:27:27.477 INFO:teuthology.orchestra.run.smithi138.stdout:Transaction Summary 2024-09-14T06:27:27.477 INFO:teuthology.orchestra.run.smithi138.stdout:================================================================================ 2024-09-14T06:27:27.477 INFO:teuthology.orchestra.run.smithi138.stdout:Install 2 Packages 2024-09-14T06:27:27.478 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-09-14T06:27:27.478 INFO:teuthology.orchestra.run.smithi138.stdout:Total download size: 979 k 2024-09-14T06:27:27.478 INFO:teuthology.orchestra.run.smithi138.stdout:Installed size: 2.9 M 2024-09-14T06:27:27.478 INFO:teuthology.orchestra.run.smithi138.stdout:Downloading Packages: 2024-09-14T06:27:27.588 INFO:teuthology.orchestra.run.smithi138.stdout:(1/2): fuse-2.9.9-16.el9.x86_64.rpm 920 kB/s | 80 kB 00:00 2024-09-14T06:27:27.778 INFO:teuthology.orchestra.run.smithi138.stdout:(2/2): ceph-fuse-19.3.0-4775.g7f24e589.el9.x86_ 3.2 MB/s | 900 kB 00:00 2024-09-14T06:27:27.779 INFO:teuthology.orchestra.run.smithi138.stdout:-------------------------------------------------------------------------------- 2024-09-14T06:27:27.779 INFO:teuthology.orchestra.run.smithi138.stdout:Total 3.2 MB/s | 979 kB 00:00 2024-09-14T06:27:27.790 INFO:teuthology.orchestra.run.smithi138.stdout:Running transaction check 2024-09-14T06:27:27.813 INFO:teuthology.orchestra.run.smithi138.stdout:Transaction check succeeded. 2024-09-14T06:27:27.813 INFO:teuthology.orchestra.run.smithi138.stdout:Running transaction test 2024-09-14T06:27:27.826 INFO:teuthology.orchestra.run.smithi094.stdout:Last metadata expiration check: 0:01:45 ago on Sat 14 Sep 2024 06:25:42 AM UTC. 2024-09-14T06:27:27.893 INFO:teuthology.orchestra.run.smithi138.stdout:Transaction test succeeded. 2024-09-14T06:27:27.893 INFO:teuthology.orchestra.run.smithi138.stdout:Running transaction 2024-09-14T06:27:27.915 INFO:teuthology.orchestra.run.smithi094.stdout:Package python3-rgw-2:19.3.0-4775.g7f24e589.el9.x86_64 is already installed. 2024-09-14T06:27:27.958 INFO:teuthology.orchestra.run.smithi094.stdout:Dependencies resolved. 2024-09-14T06:27:27.959 INFO:teuthology.orchestra.run.smithi094.stdout:Nothing to do. 2024-09-14T06:27:27.959 INFO:teuthology.orchestra.run.smithi094.stdout:Complete! 2024-09-14T06:27:28.022 DEBUG:teuthology.orchestra.run.smithi094:> sudo yum -y install python3-cephfs 2024-09-14T06:27:28.117 INFO:teuthology.orchestra.run.smithi138.stdout: Preparing : 1/1 2024-09-14T06:27:28.239 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : fuse-2.9.9-16.el9.x86_64 1/2 2024-09-14T06:27:28.303 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : ceph-fuse-2:19.3.0-4775.g7f24e589.el9.x86_64 2/2 2024-09-14T06:27:28.592 INFO:teuthology.orchestra.run.smithi094.stdout:Last metadata expiration check: 0:01:46 ago on Sat 14 Sep 2024 06:25:42 AM UTC. 2024-09-14T06:27:28.684 INFO:teuthology.orchestra.run.smithi094.stdout:Package python3-cephfs-2:19.3.0-4775.g7f24e589.el9.x86_64 is already installed. 2024-09-14T06:27:28.727 INFO:teuthology.orchestra.run.smithi094.stdout:Dependencies resolved. 2024-09-14T06:27:28.728 INFO:teuthology.orchestra.run.smithi094.stdout:Nothing to do. 2024-09-14T06:27:28.728 INFO:teuthology.orchestra.run.smithi094.stdout:Complete! 2024-09-14T06:27:28.789 DEBUG:teuthology.orchestra.run.smithi094:> sudo yum -y install python3-rbd 2024-09-14T06:27:28.976 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: ceph-fuse-2:19.3.0-4775.g7f24e589.el9.x86_64 2/2 2024-09-14T06:27:28.976 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : ceph-fuse-2:19.3.0-4775.g7f24e589.el9.x86_64 1/2 2024-09-14T06:27:29.264 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : fuse-2.9.9-16.el9.x86_64 2/2 2024-09-14T06:27:29.264 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-09-14T06:27:29.264 INFO:teuthology.orchestra.run.smithi138.stdout:Installed: 2024-09-14T06:27:29.264 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-fuse-2:19.3.0-4775.g7f24e589.el9.x86_64 fuse-2.9.9-16.el9.x86_64 2024-09-14T06:27:29.264 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-09-14T06:27:29.264 INFO:teuthology.orchestra.run.smithi138.stdout:Complete! 2024-09-14T06:27:29.350 INFO:teuthology.orchestra.run.smithi094.stdout:Last metadata expiration check: 0:01:47 ago on Sat 14 Sep 2024 06:25:42 AM UTC. 2024-09-14T06:27:29.439 INFO:teuthology.orchestra.run.smithi094.stdout:Package python3-rbd-2:19.3.0-4775.g7f24e589.el9.x86_64 is already installed. 2024-09-14T06:27:29.482 INFO:teuthology.orchestra.run.smithi094.stdout:Dependencies resolved. 2024-09-14T06:27:29.483 INFO:teuthology.orchestra.run.smithi094.stdout:Nothing to do. 2024-09-14T06:27:29.483 INFO:teuthology.orchestra.run.smithi094.stdout:Complete! 2024-09-14T06:27:29.547 DEBUG:teuthology.orchestra.run.smithi094:> sudo yum -y install rbd-fuse 2024-09-14T06:27:29.573 DEBUG:teuthology.orchestra.run.smithi138:> sudo yum -y install ceph-volume 2024-09-14T06:27:30.113 INFO:teuthology.orchestra.run.smithi094.stdout:Last metadata expiration check: 0:01:48 ago on Sat 14 Sep 2024 06:25:42 AM UTC. 2024-09-14T06:27:30.141 INFO:teuthology.orchestra.run.smithi138.stdout:Last metadata expiration check: 0:01:34 ago on Sat 14 Sep 2024 06:25:56 AM UTC. 2024-09-14T06:27:30.246 INFO:teuthology.orchestra.run.smithi094.stdout:Dependencies resolved. 2024-09-14T06:27:30.247 INFO:teuthology.orchestra.run.smithi094.stdout:================================================================================ 2024-09-14T06:27:30.247 INFO:teuthology.orchestra.run.smithi094.stdout: Package Architecture Version Repository Size 2024-09-14T06:27:30.247 INFO:teuthology.orchestra.run.smithi094.stdout:================================================================================ 2024-09-14T06:27:30.247 INFO:teuthology.orchestra.run.smithi094.stdout:Installing: 2024-09-14T06:27:30.247 INFO:teuthology.orchestra.run.smithi094.stdout: rbd-fuse x86_64 2:19.3.0-4775.g7f24e589.el9 ceph 85 k 2024-09-14T06:27:30.247 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-14T06:27:30.247 INFO:teuthology.orchestra.run.smithi094.stdout:Transaction Summary 2024-09-14T06:27:30.247 INFO:teuthology.orchestra.run.smithi094.stdout:================================================================================ 2024-09-14T06:27:30.247 INFO:teuthology.orchestra.run.smithi094.stdout:Install 1 Package 2024-09-14T06:27:30.247 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-14T06:27:30.247 INFO:teuthology.orchestra.run.smithi094.stdout:Total download size: 85 k 2024-09-14T06:27:30.247 INFO:teuthology.orchestra.run.smithi094.stdout:Installed size: 231 k 2024-09-14T06:27:30.248 INFO:teuthology.orchestra.run.smithi094.stdout:Downloading Packages: 2024-09-14T06:27:30.275 INFO:teuthology.orchestra.run.smithi138.stdout:Dependencies resolved. 2024-09-14T06:27:30.275 INFO:teuthology.orchestra.run.smithi138.stdout:================================================================================ 2024-09-14T06:27:30.276 INFO:teuthology.orchestra.run.smithi138.stdout: Package Arch Version Repository Size 2024-09-14T06:27:30.276 INFO:teuthology.orchestra.run.smithi138.stdout:================================================================================ 2024-09-14T06:27:30.276 INFO:teuthology.orchestra.run.smithi138.stdout:Installing: 2024-09-14T06:27:30.276 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-volume noarch 2:19.3.0-4775.g7f24e589.el9 ceph-noarch 285 k 2024-09-14T06:27:30.276 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-09-14T06:27:30.276 INFO:teuthology.orchestra.run.smithi138.stdout:Transaction Summary 2024-09-14T06:27:30.276 INFO:teuthology.orchestra.run.smithi138.stdout:================================================================================ 2024-09-14T06:27:30.276 INFO:teuthology.orchestra.run.smithi138.stdout:Install 1 Package 2024-09-14T06:27:30.276 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-09-14T06:27:30.276 INFO:teuthology.orchestra.run.smithi138.stdout:Total download size: 285 k 2024-09-14T06:27:30.276 INFO:teuthology.orchestra.run.smithi138.stdout:Installed size: 1.3 M 2024-09-14T06:27:30.276 INFO:teuthology.orchestra.run.smithi138.stdout:Downloading Packages: 2024-09-14T06:27:30.435 INFO:teuthology.orchestra.run.smithi094.stdout:rbd-fuse-19.3.0-4775.g7f24e589.el9.x86_64.rpm 452 kB/s | 85 kB 00:00 2024-09-14T06:27:30.436 INFO:teuthology.orchestra.run.smithi094.stdout:-------------------------------------------------------------------------------- 2024-09-14T06:27:30.436 INFO:teuthology.orchestra.run.smithi094.stdout:Total 449 kB/s | 85 kB 00:00 2024-09-14T06:27:30.436 INFO:teuthology.orchestra.run.smithi094.stdout:Running transaction check 2024-09-14T06:27:30.443 INFO:teuthology.orchestra.run.smithi094.stdout:Transaction check succeeded. 2024-09-14T06:27:30.443 INFO:teuthology.orchestra.run.smithi094.stdout:Running transaction test 2024-09-14T06:27:30.507 INFO:teuthology.orchestra.run.smithi094.stdout:Transaction test succeeded. 2024-09-14T06:27:30.507 INFO:teuthology.orchestra.run.smithi094.stdout:Running transaction 2024-09-14T06:27:30.510 INFO:teuthology.orchestra.run.smithi138.stdout:ceph-volume-19.3.0-4775.g7f24e589.el9.noarch.rp 1.2 MB/s | 285 kB 00:00 2024-09-14T06:27:30.511 INFO:teuthology.orchestra.run.smithi138.stdout:-------------------------------------------------------------------------------- 2024-09-14T06:27:30.511 INFO:teuthology.orchestra.run.smithi138.stdout:Total 1.2 MB/s | 285 kB 00:00 2024-09-14T06:27:30.511 INFO:teuthology.orchestra.run.smithi138.stdout:Running transaction check 2024-09-14T06:27:30.520 INFO:teuthology.orchestra.run.smithi138.stdout:Transaction check succeeded. 2024-09-14T06:27:30.520 INFO:teuthology.orchestra.run.smithi138.stdout:Running transaction test 2024-09-14T06:27:30.552 INFO:teuthology.orchestra.run.smithi138.stdout:Transaction test succeeded. 2024-09-14T06:27:30.552 INFO:teuthology.orchestra.run.smithi138.stdout:Running transaction 2024-09-14T06:27:30.648 INFO:teuthology.orchestra.run.smithi094.stdout: Preparing : 1/1 2024-09-14T06:27:30.766 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : rbd-fuse-2:19.3.0-4775.g7f24e589.el9.x86_64 1/1 2024-09-14T06:27:30.773 INFO:teuthology.orchestra.run.smithi138.stdout: Preparing : 1/1 2024-09-14T06:27:30.838 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : ceph-volume-2:19.3.0-4775.g7f24e589.el9.noarch 1/1 2024-09-14T06:27:30.853 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: ceph-volume-2:19.3.0-4775.g7f24e589.el9.noarch 1/1 2024-09-14T06:27:30.853 INFO:teuthology.orchestra.run.smithi138.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-14T06:27:30.853 INFO:teuthology.orchestra.run.smithi138.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-09-14T06:27:30.853 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-09-14T06:27:31.158 INFO:teuthology.orchestra.run.smithi094.stdout: Running scriptlet: rbd-fuse-2:19.3.0-4775.g7f24e589.el9.x86_64 1/1 2024-09-14T06:27:31.405 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : rbd-fuse-2:19.3.0-4775.g7f24e589.el9.x86_64 1/1 2024-09-14T06:27:31.405 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-14T06:27:31.405 INFO:teuthology.orchestra.run.smithi094.stdout:Installed: 2024-09-14T06:27:31.405 INFO:teuthology.orchestra.run.smithi094.stdout: rbd-fuse-2:19.3.0-4775.g7f24e589.el9.x86_64 2024-09-14T06:27:31.405 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-14T06:27:31.405 INFO:teuthology.orchestra.run.smithi094.stdout:Complete! 2024-09-14T06:27:31.669 DEBUG:teuthology.orchestra.run.smithi094:> sudo yum -y install rbd-mirror 2024-09-14T06:27:31.864 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : ceph-volume-2:19.3.0-4775.g7f24e589.el9.noarch 1/1 2024-09-14T06:27:31.864 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-09-14T06:27:31.864 INFO:teuthology.orchestra.run.smithi138.stdout:Installed: 2024-09-14T06:27:31.864 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-volume-2:19.3.0-4775.g7f24e589.el9.noarch 2024-09-14T06:27:31.865 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-09-14T06:27:31.865 INFO:teuthology.orchestra.run.smithi138.stdout:Complete! 2024-09-14T06:27:32.114 DEBUG:teuthology.orchestra.run.smithi138:> sudo yum -y install librados-devel 2024-09-14T06:27:32.243 INFO:teuthology.orchestra.run.smithi094.stdout:Last metadata expiration check: 0:01:50 ago on Sat 14 Sep 2024 06:25:42 AM UTC. 2024-09-14T06:27:32.380 INFO:teuthology.orchestra.run.smithi094.stdout:Dependencies resolved. 2024-09-14T06:27:32.381 INFO:teuthology.orchestra.run.smithi094.stdout:================================================================================ 2024-09-14T06:27:32.381 INFO:teuthology.orchestra.run.smithi094.stdout: Package Arch Version Repo Size 2024-09-14T06:27:32.381 INFO:teuthology.orchestra.run.smithi094.stdout:================================================================================ 2024-09-14T06:27:32.381 INFO:teuthology.orchestra.run.smithi094.stdout:Installing: 2024-09-14T06:27:32.381 INFO:teuthology.orchestra.run.smithi094.stdout: rbd-mirror x86_64 2:19.3.0-4775.g7f24e589.el9 ceph 3.2 M 2024-09-14T06:27:32.381 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-14T06:27:32.381 INFO:teuthology.orchestra.run.smithi094.stdout:Transaction Summary 2024-09-14T06:27:32.381 INFO:teuthology.orchestra.run.smithi094.stdout:================================================================================ 2024-09-14T06:27:32.381 INFO:teuthology.orchestra.run.smithi094.stdout:Install 1 Package 2024-09-14T06:27:32.381 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-14T06:27:32.382 INFO:teuthology.orchestra.run.smithi094.stdout:Total download size: 3.2 M 2024-09-14T06:27:32.382 INFO:teuthology.orchestra.run.smithi094.stdout:Installed size: 14 M 2024-09-14T06:27:32.382 INFO:teuthology.orchestra.run.smithi094.stdout:Downloading Packages: 2024-09-14T06:27:32.701 INFO:teuthology.orchestra.run.smithi138.stdout:Last metadata expiration check: 0:01:36 ago on Sat 14 Sep 2024 06:25:56 AM UTC. 2024-09-14T06:27:32.715 INFO:teuthology.orchestra.run.smithi094.stdout:rbd-mirror-19.3.0-4775.g7f24e589.el9.x86_64.rpm 9.5 MB/s | 3.2 MB 00:00 2024-09-14T06:27:32.715 INFO:teuthology.orchestra.run.smithi094.stdout:-------------------------------------------------------------------------------- 2024-09-14T06:27:32.716 INFO:teuthology.orchestra.run.smithi094.stdout:Total 9.5 MB/s | 3.2 MB 00:00 2024-09-14T06:27:32.716 INFO:teuthology.orchestra.run.smithi094.stdout:Running transaction check 2024-09-14T06:27:32.725 INFO:teuthology.orchestra.run.smithi094.stdout:Transaction check succeeded. 2024-09-14T06:27:32.725 INFO:teuthology.orchestra.run.smithi094.stdout:Running transaction test 2024-09-14T06:27:32.809 INFO:teuthology.orchestra.run.smithi094.stdout:Transaction test succeeded. 2024-09-14T06:27:32.809 INFO:teuthology.orchestra.run.smithi094.stdout:Running transaction 2024-09-14T06:27:32.840 INFO:teuthology.orchestra.run.smithi138.stdout:Dependencies resolved. 2024-09-14T06:27:32.841 INFO:teuthology.orchestra.run.smithi138.stdout:================================================================================ 2024-09-14T06:27:32.841 INFO:teuthology.orchestra.run.smithi138.stdout: Package Arch Version Repo Size 2024-09-14T06:27:32.841 INFO:teuthology.orchestra.run.smithi138.stdout:================================================================================ 2024-09-14T06:27:32.841 INFO:teuthology.orchestra.run.smithi138.stdout:Installing: 2024-09-14T06:27:32.841 INFO:teuthology.orchestra.run.smithi138.stdout: librados-devel x86_64 2:19.3.0-4775.g7f24e589.el9 ceph 126 k 2024-09-14T06:27:32.841 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-09-14T06:27:32.841 INFO:teuthology.orchestra.run.smithi138.stdout:Transaction Summary 2024-09-14T06:27:32.841 INFO:teuthology.orchestra.run.smithi138.stdout:================================================================================ 2024-09-14T06:27:32.841 INFO:teuthology.orchestra.run.smithi138.stdout:Install 1 Package 2024-09-14T06:27:32.842 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-09-14T06:27:32.842 INFO:teuthology.orchestra.run.smithi138.stdout:Total download size: 126 k 2024-09-14T06:27:32.842 INFO:teuthology.orchestra.run.smithi138.stdout:Installed size: 456 k 2024-09-14T06:27:32.842 INFO:teuthology.orchestra.run.smithi138.stdout:Downloading Packages: 2024-09-14T06:27:33.045 INFO:teuthology.orchestra.run.smithi138.stdout:librados-devel-19.3.0-4775.g7f24e589.el9.x86_64 625 kB/s | 126 kB 00:00 2024-09-14T06:27:33.045 INFO:teuthology.orchestra.run.smithi138.stdout:-------------------------------------------------------------------------------- 2024-09-14T06:27:33.045 INFO:teuthology.orchestra.run.smithi138.stdout:Total 621 kB/s | 126 kB 00:00 2024-09-14T06:27:33.046 INFO:teuthology.orchestra.run.smithi138.stdout:Running transaction check 2024-09-14T06:27:33.051 INFO:teuthology.orchestra.run.smithi138.stdout:Transaction check succeeded. 2024-09-14T06:27:33.051 INFO:teuthology.orchestra.run.smithi138.stdout:Running transaction test 2024-09-14T06:27:33.116 INFO:teuthology.orchestra.run.smithi138.stdout:Transaction test succeeded. 2024-09-14T06:27:33.116 INFO:teuthology.orchestra.run.smithi138.stdout:Running transaction 2024-09-14T06:27:33.250 INFO:teuthology.orchestra.run.smithi094.stdout: Preparing : 1/1 2024-09-14T06:27:33.265 INFO:teuthology.orchestra.run.smithi138.stdout: Preparing : 1/1 2024-09-14T06:27:33.316 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : rbd-mirror-2:19.3.0-4775.g7f24e589.el9.x86_64 1/1 2024-09-14T06:27:33.341 INFO:teuthology.orchestra.run.smithi094.stdout: Running scriptlet: rbd-mirror-2:19.3.0-4775.g7f24e589.el9.x86_64 1/1 2024-09-14T06:27:33.341 INFO:teuthology.orchestra.run.smithi094.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-14T06:27:33.341 INFO:teuthology.orchestra.run.smithi094.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-09-14T06:27:33.341 INFO:teuthology.orchestra.run.smithi094.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-14T06:27:33.341 INFO:teuthology.orchestra.run.smithi094.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-14T06:27:33.341 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-14T06:27:33.358 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : librados-devel-2:19.3.0-4775.g7f24e589.el9.x86_64 1/1 2024-09-14T06:27:33.786 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: librados-devel-2:19.3.0-4775.g7f24e589.el9.x86_64 1/1 2024-09-14T06:27:34.038 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : librados-devel-2:19.3.0-4775.g7f24e589.el9.x86_64 1/1 2024-09-14T06:27:34.039 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-09-14T06:27:34.039 INFO:teuthology.orchestra.run.smithi138.stdout:Installed: 2024-09-14T06:27:34.039 INFO:teuthology.orchestra.run.smithi138.stdout: librados-devel-2:19.3.0-4775.g7f24e589.el9.x86_64 2024-09-14T06:27:34.039 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-09-14T06:27:34.039 INFO:teuthology.orchestra.run.smithi138.stdout:Complete! 2024-09-14T06:27:34.298 DEBUG:teuthology.orchestra.run.smithi138:> sudo yum -y install libcephfs2 2024-09-14T06:27:34.446 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : rbd-mirror-2:19.3.0-4775.g7f24e589.el9.x86_64 1/1 2024-09-14T06:27:34.446 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-14T06:27:34.446 INFO:teuthology.orchestra.run.smithi094.stdout:Installed: 2024-09-14T06:27:34.446 INFO:teuthology.orchestra.run.smithi094.stdout: rbd-mirror-2:19.3.0-4775.g7f24e589.el9.x86_64 2024-09-14T06:27:34.447 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-14T06:27:34.447 INFO:teuthology.orchestra.run.smithi094.stdout:Complete! 2024-09-14T06:27:34.715 DEBUG:teuthology.orchestra.run.smithi094:> sudo yum -y install rbd-nbd 2024-09-14T06:27:34.880 INFO:teuthology.orchestra.run.smithi138.stdout:Last metadata expiration check: 0:01:38 ago on Sat 14 Sep 2024 06:25:56 AM UTC. 2024-09-14T06:27:34.973 INFO:teuthology.orchestra.run.smithi138.stdout:Package libcephfs2-2:19.3.0-4775.g7f24e589.el9.x86_64 is already installed. 2024-09-14T06:27:35.018 INFO:teuthology.orchestra.run.smithi138.stdout:Dependencies resolved. 2024-09-14T06:27:35.019 INFO:teuthology.orchestra.run.smithi138.stdout:Nothing to do. 2024-09-14T06:27:35.019 INFO:teuthology.orchestra.run.smithi138.stdout:Complete! 2024-09-14T06:27:35.081 DEBUG:teuthology.orchestra.run.smithi138:> sudo yum -y install libcephfs-devel 2024-09-14T06:27:35.296 INFO:teuthology.orchestra.run.smithi094.stdout:Last metadata expiration check: 0:01:53 ago on Sat 14 Sep 2024 06:25:42 AM UTC. 2024-09-14T06:27:35.431 INFO:teuthology.orchestra.run.smithi094.stdout:Dependencies resolved. 2024-09-14T06:27:35.431 INFO:teuthology.orchestra.run.smithi094.stdout:================================================================================ 2024-09-14T06:27:35.432 INFO:teuthology.orchestra.run.smithi094.stdout: Package Architecture Version Repository Size 2024-09-14T06:27:35.432 INFO:teuthology.orchestra.run.smithi094.stdout:================================================================================ 2024-09-14T06:27:35.432 INFO:teuthology.orchestra.run.smithi094.stdout:Installing: 2024-09-14T06:27:35.432 INFO:teuthology.orchestra.run.smithi094.stdout: rbd-nbd x86_64 2:19.3.0-4775.g7f24e589.el9 ceph 171 k 2024-09-14T06:27:35.432 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-14T06:27:35.432 INFO:teuthology.orchestra.run.smithi094.stdout:Transaction Summary 2024-09-14T06:27:35.432 INFO:teuthology.orchestra.run.smithi094.stdout:================================================================================ 2024-09-14T06:27:35.432 INFO:teuthology.orchestra.run.smithi094.stdout:Install 1 Package 2024-09-14T06:27:35.432 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-14T06:27:35.432 INFO:teuthology.orchestra.run.smithi094.stdout:Total download size: 171 k 2024-09-14T06:27:35.432 INFO:teuthology.orchestra.run.smithi094.stdout:Installed size: 490 k 2024-09-14T06:27:35.432 INFO:teuthology.orchestra.run.smithi094.stdout:Downloading Packages: 2024-09-14T06:27:35.638 INFO:teuthology.orchestra.run.smithi094.stdout:rbd-nbd-19.3.0-4775.g7f24e589.el9.x86_64.rpm 833 kB/s | 171 kB 00:00 2024-09-14T06:27:35.638 INFO:teuthology.orchestra.run.smithi094.stdout:-------------------------------------------------------------------------------- 2024-09-14T06:27:35.639 INFO:teuthology.orchestra.run.smithi094.stdout:Total 827 kB/s | 171 kB 00:00 2024-09-14T06:27:35.639 INFO:teuthology.orchestra.run.smithi094.stdout:Running transaction check 2024-09-14T06:27:35.646 INFO:teuthology.orchestra.run.smithi094.stdout:Transaction check succeeded. 2024-09-14T06:27:35.646 INFO:teuthology.orchestra.run.smithi094.stdout:Running transaction test 2024-09-14T06:27:35.673 INFO:teuthology.orchestra.run.smithi138.stdout:Last metadata expiration check: 0:01:39 ago on Sat 14 Sep 2024 06:25:56 AM UTC. 2024-09-14T06:27:35.711 INFO:teuthology.orchestra.run.smithi094.stdout:Transaction test succeeded. 2024-09-14T06:27:35.711 INFO:teuthology.orchestra.run.smithi094.stdout:Running transaction 2024-09-14T06:27:35.817 INFO:teuthology.orchestra.run.smithi138.stdout:Dependencies resolved. 2024-09-14T06:27:35.817 INFO:teuthology.orchestra.run.smithi138.stdout:================================================================================ 2024-09-14T06:27:35.818 INFO:teuthology.orchestra.run.smithi138.stdout: Package Arch Version Repo Size 2024-09-14T06:27:35.818 INFO:teuthology.orchestra.run.smithi138.stdout:================================================================================ 2024-09-14T06:27:35.818 INFO:teuthology.orchestra.run.smithi138.stdout:Installing: 2024-09-14T06:27:35.818 INFO:teuthology.orchestra.run.smithi138.stdout: libcephfs-devel x86_64 2:19.3.0-4775.g7f24e589.el9 ceph 32 k 2024-09-14T06:27:35.818 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-09-14T06:27:35.818 INFO:teuthology.orchestra.run.smithi138.stdout:Transaction Summary 2024-09-14T06:27:35.818 INFO:teuthology.orchestra.run.smithi138.stdout:================================================================================ 2024-09-14T06:27:35.818 INFO:teuthology.orchestra.run.smithi138.stdout:Install 1 Package 2024-09-14T06:27:35.818 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-09-14T06:27:35.818 INFO:teuthology.orchestra.run.smithi138.stdout:Total download size: 32 k 2024-09-14T06:27:35.818 INFO:teuthology.orchestra.run.smithi138.stdout:Installed size: 142 k 2024-09-14T06:27:35.818 INFO:teuthology.orchestra.run.smithi138.stdout:Downloading Packages: 2024-09-14T06:27:35.876 INFO:teuthology.orchestra.run.smithi094.stdout: Preparing : 1/1 2024-09-14T06:27:35.968 INFO:teuthology.orchestra.run.smithi138.stdout:libcephfs-devel-19.3.0-4775.g7f24e589.el9.x86_6 213 kB/s | 32 kB 00:00 2024-09-14T06:27:35.969 INFO:teuthology.orchestra.run.smithi138.stdout:-------------------------------------------------------------------------------- 2024-09-14T06:27:35.969 INFO:teuthology.orchestra.run.smithi138.stdout:Total 210 kB/s | 32 kB 00:00 2024-09-14T06:27:35.969 INFO:teuthology.orchestra.run.smithi138.stdout:Running transaction check 2024-09-14T06:27:35.973 INFO:teuthology.orchestra.run.smithi138.stdout:Transaction check succeeded. 2024-09-14T06:27:35.973 INFO:teuthology.orchestra.run.smithi138.stdout:Running transaction test 2024-09-14T06:27:35.985 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : rbd-nbd-2:19.3.0-4775.g7f24e589.el9.x86_64 1/1 2024-09-14T06:27:35.998 INFO:teuthology.orchestra.run.smithi138.stdout:Transaction test succeeded. 2024-09-14T06:27:35.998 INFO:teuthology.orchestra.run.smithi138.stdout:Running transaction 2024-09-14T06:27:36.090 INFO:teuthology.orchestra.run.smithi138.stdout: Preparing : 1/1 2024-09-14T06:27:36.208 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : libcephfs-devel-2:19.3.0-4775.g7f24e589.el9.x86_64 1/1 2024-09-14T06:27:36.420 INFO:teuthology.orchestra.run.smithi094.stdout: Running scriptlet: rbd-nbd-2:19.3.0-4775.g7f24e589.el9.x86_64 1/1 2024-09-14T06:27:36.611 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: libcephfs-devel-2:19.3.0-4775.g7f24e589.el9.x86_64 1/1 2024-09-14T06:27:36.680 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : rbd-nbd-2:19.3.0-4775.g7f24e589.el9.x86_64 1/1 2024-09-14T06:27:36.680 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-14T06:27:36.680 INFO:teuthology.orchestra.run.smithi094.stdout:Installed: 2024-09-14T06:27:36.680 INFO:teuthology.orchestra.run.smithi094.stdout: rbd-nbd-2:19.3.0-4775.g7f24e589.el9.x86_64 2024-09-14T06:27:36.680 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-14T06:27:36.680 INFO:teuthology.orchestra.run.smithi094.stdout:Complete! 2024-09-14T06:27:36.844 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : libcephfs-devel-2:19.3.0-4775.g7f24e589.el9.x86_64 1/1 2024-09-14T06:27:36.845 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-09-14T06:27:36.845 INFO:teuthology.orchestra.run.smithi138.stdout:Installed: 2024-09-14T06:27:36.845 INFO:teuthology.orchestra.run.smithi138.stdout: libcephfs-devel-2:19.3.0-4775.g7f24e589.el9.x86_64 2024-09-14T06:27:36.845 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-09-14T06:27:36.845 INFO:teuthology.orchestra.run.smithi138.stdout:Complete! 2024-09-14T06:27:36.943 DEBUG:teuthology.parallel:result is None 2024-09-14T06:27:37.025 DEBUG:teuthology.orchestra.run.smithi138:> sudo yum -y install librados2 2024-09-14T06:27:37.590 INFO:teuthology.orchestra.run.smithi138.stdout:Last metadata expiration check: 0:01:41 ago on Sat 14 Sep 2024 06:25:56 AM UTC. 2024-09-14T06:27:37.680 INFO:teuthology.orchestra.run.smithi138.stdout:Package librados2-2:19.3.0-4775.g7f24e589.el9.x86_64 is already installed. 2024-09-14T06:27:37.722 INFO:teuthology.orchestra.run.smithi138.stdout:Dependencies resolved. 2024-09-14T06:27:37.723 INFO:teuthology.orchestra.run.smithi138.stdout:Nothing to do. 2024-09-14T06:27:37.723 INFO:teuthology.orchestra.run.smithi138.stdout:Complete! 2024-09-14T06:27:37.786 DEBUG:teuthology.orchestra.run.smithi138:> sudo yum -y install librbd1 2024-09-14T06:27:38.350 INFO:teuthology.orchestra.run.smithi138.stdout:Last metadata expiration check: 0:01:42 ago on Sat 14 Sep 2024 06:25:56 AM UTC. 2024-09-14T06:27:38.440 INFO:teuthology.orchestra.run.smithi138.stdout:Package librbd1-2:19.3.0-4775.g7f24e589.el9.x86_64 is already installed. 2024-09-14T06:27:38.484 INFO:teuthology.orchestra.run.smithi138.stdout:Dependencies resolved. 2024-09-14T06:27:38.485 INFO:teuthology.orchestra.run.smithi138.stdout:Nothing to do. 2024-09-14T06:27:38.485 INFO:teuthology.orchestra.run.smithi138.stdout:Complete! 2024-09-14T06:27:38.545 DEBUG:teuthology.orchestra.run.smithi138:> sudo yum -y install python3-rados 2024-09-14T06:27:39.112 INFO:teuthology.orchestra.run.smithi138.stdout:Last metadata expiration check: 0:01:43 ago on Sat 14 Sep 2024 06:25:56 AM UTC. 2024-09-14T06:27:39.203 INFO:teuthology.orchestra.run.smithi138.stdout:Package python3-rados-2:19.3.0-4775.g7f24e589.el9.x86_64 is already installed. 2024-09-14T06:27:39.247 INFO:teuthology.orchestra.run.smithi138.stdout:Dependencies resolved. 2024-09-14T06:27:39.248 INFO:teuthology.orchestra.run.smithi138.stdout:Nothing to do. 2024-09-14T06:27:39.248 INFO:teuthology.orchestra.run.smithi138.stdout:Complete! 2024-09-14T06:27:39.311 DEBUG:teuthology.orchestra.run.smithi138:> sudo yum -y install python3-rgw 2024-09-14T06:27:39.875 INFO:teuthology.orchestra.run.smithi138.stdout:Last metadata expiration check: 0:01:43 ago on Sat 14 Sep 2024 06:25:56 AM UTC. 2024-09-14T06:27:39.964 INFO:teuthology.orchestra.run.smithi138.stdout:Package python3-rgw-2:19.3.0-4775.g7f24e589.el9.x86_64 is already installed. 2024-09-14T06:27:40.007 INFO:teuthology.orchestra.run.smithi138.stdout:Dependencies resolved. 2024-09-14T06:27:40.009 INFO:teuthology.orchestra.run.smithi138.stdout:Nothing to do. 2024-09-14T06:27:40.009 INFO:teuthology.orchestra.run.smithi138.stdout:Complete! 2024-09-14T06:27:40.070 DEBUG:teuthology.orchestra.run.smithi138:> sudo yum -y install python3-cephfs 2024-09-14T06:27:40.642 INFO:teuthology.orchestra.run.smithi138.stdout:Last metadata expiration check: 0:01:44 ago on Sat 14 Sep 2024 06:25:56 AM UTC. 2024-09-14T06:27:40.733 INFO:teuthology.orchestra.run.smithi138.stdout:Package python3-cephfs-2:19.3.0-4775.g7f24e589.el9.x86_64 is already installed. 2024-09-14T06:27:40.776 INFO:teuthology.orchestra.run.smithi138.stdout:Dependencies resolved. 2024-09-14T06:27:40.777 INFO:teuthology.orchestra.run.smithi138.stdout:Nothing to do. 2024-09-14T06:27:40.777 INFO:teuthology.orchestra.run.smithi138.stdout:Complete! 2024-09-14T06:27:40.836 DEBUG:teuthology.orchestra.run.smithi138:> sudo yum -y install python3-rbd 2024-09-14T06:27:41.399 INFO:teuthology.orchestra.run.smithi138.stdout:Last metadata expiration check: 0:01:45 ago on Sat 14 Sep 2024 06:25:56 AM UTC. 2024-09-14T06:27:41.489 INFO:teuthology.orchestra.run.smithi138.stdout:Package python3-rbd-2:19.3.0-4775.g7f24e589.el9.x86_64 is already installed. 2024-09-14T06:27:41.532 INFO:teuthology.orchestra.run.smithi138.stdout:Dependencies resolved. 2024-09-14T06:27:41.533 INFO:teuthology.orchestra.run.smithi138.stdout:Nothing to do. 2024-09-14T06:27:41.533 INFO:teuthology.orchestra.run.smithi138.stdout:Complete! 2024-09-14T06:27:41.595 DEBUG:teuthology.orchestra.run.smithi138:> sudo yum -y install rbd-fuse 2024-09-14T06:27:42.160 INFO:teuthology.orchestra.run.smithi138.stdout:Last metadata expiration check: 0:01:46 ago on Sat 14 Sep 2024 06:25:56 AM UTC. 2024-09-14T06:27:42.294 INFO:teuthology.orchestra.run.smithi138.stdout:Dependencies resolved. 2024-09-14T06:27:42.294 INFO:teuthology.orchestra.run.smithi138.stdout:================================================================================ 2024-09-14T06:27:42.295 INFO:teuthology.orchestra.run.smithi138.stdout: Package Architecture Version Repository Size 2024-09-14T06:27:42.295 INFO:teuthology.orchestra.run.smithi138.stdout:================================================================================ 2024-09-14T06:27:42.295 INFO:teuthology.orchestra.run.smithi138.stdout:Installing: 2024-09-14T06:27:42.295 INFO:teuthology.orchestra.run.smithi138.stdout: rbd-fuse x86_64 2:19.3.0-4775.g7f24e589.el9 ceph 85 k 2024-09-14T06:27:42.295 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-09-14T06:27:42.295 INFO:teuthology.orchestra.run.smithi138.stdout:Transaction Summary 2024-09-14T06:27:42.295 INFO:teuthology.orchestra.run.smithi138.stdout:================================================================================ 2024-09-14T06:27:42.295 INFO:teuthology.orchestra.run.smithi138.stdout:Install 1 Package 2024-09-14T06:27:42.295 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-09-14T06:27:42.295 INFO:teuthology.orchestra.run.smithi138.stdout:Total download size: 85 k 2024-09-14T06:27:42.295 INFO:teuthology.orchestra.run.smithi138.stdout:Installed size: 231 k 2024-09-14T06:27:42.295 INFO:teuthology.orchestra.run.smithi138.stdout:Downloading Packages: 2024-09-14T06:27:42.488 INFO:teuthology.orchestra.run.smithi138.stdout:rbd-fuse-19.3.0-4775.g7f24e589.el9.x86_64.rpm 440 kB/s | 85 kB 00:00 2024-09-14T06:27:42.489 INFO:teuthology.orchestra.run.smithi138.stdout:-------------------------------------------------------------------------------- 2024-09-14T06:27:42.489 INFO:teuthology.orchestra.run.smithi138.stdout:Total 437 kB/s | 85 kB 00:00 2024-09-14T06:27:42.489 INFO:teuthology.orchestra.run.smithi138.stdout:Running transaction check 2024-09-14T06:27:42.496 INFO:teuthology.orchestra.run.smithi138.stdout:Transaction check succeeded. 2024-09-14T06:27:42.497 INFO:teuthology.orchestra.run.smithi138.stdout:Running transaction test 2024-09-14T06:27:42.560 INFO:teuthology.orchestra.run.smithi138.stdout:Transaction test succeeded. 2024-09-14T06:27:42.560 INFO:teuthology.orchestra.run.smithi138.stdout:Running transaction 2024-09-14T06:27:42.700 INFO:teuthology.orchestra.run.smithi138.stdout: Preparing : 1/1 2024-09-14T06:27:42.811 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : rbd-fuse-2:19.3.0-4775.g7f24e589.el9.x86_64 1/1 2024-09-14T06:27:43.221 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: rbd-fuse-2:19.3.0-4775.g7f24e589.el9.x86_64 1/1 2024-09-14T06:27:43.478 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : rbd-fuse-2:19.3.0-4775.g7f24e589.el9.x86_64 1/1 2024-09-14T06:27:43.478 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-09-14T06:27:43.479 INFO:teuthology.orchestra.run.smithi138.stdout:Installed: 2024-09-14T06:27:43.479 INFO:teuthology.orchestra.run.smithi138.stdout: rbd-fuse-2:19.3.0-4775.g7f24e589.el9.x86_64 2024-09-14T06:27:43.479 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-09-14T06:27:43.479 INFO:teuthology.orchestra.run.smithi138.stdout:Complete! 2024-09-14T06:27:43.739 DEBUG:teuthology.orchestra.run.smithi138:> sudo yum -y install rbd-mirror 2024-09-14T06:27:44.311 INFO:teuthology.orchestra.run.smithi138.stdout:Last metadata expiration check: 0:01:48 ago on Sat 14 Sep 2024 06:25:56 AM UTC. 2024-09-14T06:27:44.450 INFO:teuthology.orchestra.run.smithi138.stdout:Dependencies resolved. 2024-09-14T06:27:44.450 INFO:teuthology.orchestra.run.smithi138.stdout:================================================================================ 2024-09-14T06:27:44.451 INFO:teuthology.orchestra.run.smithi138.stdout: Package Arch Version Repo Size 2024-09-14T06:27:44.451 INFO:teuthology.orchestra.run.smithi138.stdout:================================================================================ 2024-09-14T06:27:44.451 INFO:teuthology.orchestra.run.smithi138.stdout:Installing: 2024-09-14T06:27:44.451 INFO:teuthology.orchestra.run.smithi138.stdout: rbd-mirror x86_64 2:19.3.0-4775.g7f24e589.el9 ceph 3.2 M 2024-09-14T06:27:44.451 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-09-14T06:27:44.451 INFO:teuthology.orchestra.run.smithi138.stdout:Transaction Summary 2024-09-14T06:27:44.451 INFO:teuthology.orchestra.run.smithi138.stdout:================================================================================ 2024-09-14T06:27:44.451 INFO:teuthology.orchestra.run.smithi138.stdout:Install 1 Package 2024-09-14T06:27:44.451 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-09-14T06:27:44.451 INFO:teuthology.orchestra.run.smithi138.stdout:Total download size: 3.2 M 2024-09-14T06:27:44.451 INFO:teuthology.orchestra.run.smithi138.stdout:Installed size: 14 M 2024-09-14T06:27:44.451 INFO:teuthology.orchestra.run.smithi138.stdout:Downloading Packages: 2024-09-14T06:27:44.803 INFO:teuthology.orchestra.run.smithi138.stdout:rbd-mirror-19.3.0-4775.g7f24e589.el9.x86_64.rpm 9.0 MB/s | 3.2 MB 00:00 2024-09-14T06:27:44.804 INFO:teuthology.orchestra.run.smithi138.stdout:-------------------------------------------------------------------------------- 2024-09-14T06:27:44.804 INFO:teuthology.orchestra.run.smithi138.stdout:Total 9.0 MB/s | 3.2 MB 00:00 2024-09-14T06:27:44.804 INFO:teuthology.orchestra.run.smithi138.stdout:Running transaction check 2024-09-14T06:27:44.813 INFO:teuthology.orchestra.run.smithi138.stdout:Transaction check succeeded. 2024-09-14T06:27:44.813 INFO:teuthology.orchestra.run.smithi138.stdout:Running transaction test 2024-09-14T06:27:44.895 INFO:teuthology.orchestra.run.smithi138.stdout:Transaction test succeeded. 2024-09-14T06:27:44.895 INFO:teuthology.orchestra.run.smithi138.stdout:Running transaction 2024-09-14T06:27:45.336 INFO:teuthology.orchestra.run.smithi138.stdout: Preparing : 1/1 2024-09-14T06:27:45.395 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : rbd-mirror-2:19.3.0-4775.g7f24e589.el9.x86_64 1/1 2024-09-14T06:27:45.420 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: rbd-mirror-2:19.3.0-4775.g7f24e589.el9.x86_64 1/1 2024-09-14T06:27:45.420 INFO:teuthology.orchestra.run.smithi138.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-14T06:27:45.421 INFO:teuthology.orchestra.run.smithi138.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-09-14T06:27:45.421 INFO:teuthology.orchestra.run.smithi138.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-14T06:27:45.421 INFO:teuthology.orchestra.run.smithi138.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-14T06:27:45.421 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-09-14T06:27:46.512 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : rbd-mirror-2:19.3.0-4775.g7f24e589.el9.x86_64 1/1 2024-09-14T06:27:46.512 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-09-14T06:27:46.512 INFO:teuthology.orchestra.run.smithi138.stdout:Installed: 2024-09-14T06:27:46.512 INFO:teuthology.orchestra.run.smithi138.stdout: rbd-mirror-2:19.3.0-4775.g7f24e589.el9.x86_64 2024-09-14T06:27:46.512 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-09-14T06:27:46.513 INFO:teuthology.orchestra.run.smithi138.stdout:Complete! 2024-09-14T06:27:46.738 DEBUG:teuthology.orchestra.run.smithi138:> sudo yum -y install rbd-nbd 2024-09-14T06:27:47.320 INFO:teuthology.orchestra.run.smithi138.stdout:Last metadata expiration check: 0:01:51 ago on Sat 14 Sep 2024 06:25:56 AM UTC. 2024-09-14T06:27:47.458 INFO:teuthology.orchestra.run.smithi138.stdout:Dependencies resolved. 2024-09-14T06:27:47.459 INFO:teuthology.orchestra.run.smithi138.stdout:================================================================================ 2024-09-14T06:27:47.459 INFO:teuthology.orchestra.run.smithi138.stdout: Package Architecture Version Repository Size 2024-09-14T06:27:47.459 INFO:teuthology.orchestra.run.smithi138.stdout:================================================================================ 2024-09-14T06:27:47.459 INFO:teuthology.orchestra.run.smithi138.stdout:Installing: 2024-09-14T06:27:47.459 INFO:teuthology.orchestra.run.smithi138.stdout: rbd-nbd x86_64 2:19.3.0-4775.g7f24e589.el9 ceph 171 k 2024-09-14T06:27:47.459 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-09-14T06:27:47.459 INFO:teuthology.orchestra.run.smithi138.stdout:Transaction Summary 2024-09-14T06:27:47.459 INFO:teuthology.orchestra.run.smithi138.stdout:================================================================================ 2024-09-14T06:27:47.459 INFO:teuthology.orchestra.run.smithi138.stdout:Install 1 Package 2024-09-14T06:27:47.459 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-09-14T06:27:47.459 INFO:teuthology.orchestra.run.smithi138.stdout:Total download size: 171 k 2024-09-14T06:27:47.460 INFO:teuthology.orchestra.run.smithi138.stdout:Installed size: 490 k 2024-09-14T06:27:47.460 INFO:teuthology.orchestra.run.smithi138.stdout:Downloading Packages: 2024-09-14T06:27:47.823 INFO:teuthology.orchestra.run.smithi138.stdout:rbd-nbd-19.3.0-4775.g7f24e589.el9.x86_64.rpm 471 kB/s | 171 kB 00:00 2024-09-14T06:27:47.824 INFO:teuthology.orchestra.run.smithi138.stdout:-------------------------------------------------------------------------------- 2024-09-14T06:27:47.824 INFO:teuthology.orchestra.run.smithi138.stdout:Total 468 kB/s | 171 kB 00:00 2024-09-14T06:27:47.824 INFO:teuthology.orchestra.run.smithi138.stdout:Running transaction check 2024-09-14T06:27:47.831 INFO:teuthology.orchestra.run.smithi138.stdout:Transaction check succeeded. 2024-09-14T06:27:47.831 INFO:teuthology.orchestra.run.smithi138.stdout:Running transaction test 2024-09-14T06:27:47.896 INFO:teuthology.orchestra.run.smithi138.stdout:Transaction test succeeded. 2024-09-14T06:27:47.896 INFO:teuthology.orchestra.run.smithi138.stdout:Running transaction 2024-09-14T06:27:48.065 INFO:teuthology.orchestra.run.smithi138.stdout: Preparing : 1/1 2024-09-14T06:27:48.162 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : rbd-nbd-2:19.3.0-4775.g7f24e589.el9.x86_64 1/1 2024-09-14T06:27:48.579 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: rbd-nbd-2:19.3.0-4775.g7f24e589.el9.x86_64 1/1 2024-09-14T06:27:48.870 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : rbd-nbd-2:19.3.0-4775.g7f24e589.el9.x86_64 1/1 2024-09-14T06:27:48.870 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-09-14T06:27:48.870 INFO:teuthology.orchestra.run.smithi138.stdout:Installed: 2024-09-14T06:27:48.870 INFO:teuthology.orchestra.run.smithi138.stdout: rbd-nbd-2:19.3.0-4775.g7f24e589.el9.x86_64 2024-09-14T06:27:48.871 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-09-14T06:27:48.871 INFO:teuthology.orchestra.run.smithi138.stdout:Complete! 2024-09-14T06:27:49.091 DEBUG:teuthology.parallel:result is None 2024-09-14T06:27:49.092 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=7f24e589328ae9e639c0009d93340e5b235d0a47 2024-09-14T06:27:49.240 DEBUG:teuthology.orchestra.run.smithi094:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-09-14T06:27:49.266 INFO:teuthology.orchestra.run.smithi094.stdout:19.3.0-4775.g7f24e589.el9 2024-09-14T06:27:49.266 INFO:teuthology.packaging:The installed version of ceph is 19.3.0-4775.g7f24e589.el9 2024-09-14T06:27:49.267 INFO:teuthology.task.install:The correct ceph version 19.3.0-4775.g7f24e589 is installed. 2024-09-14T06:27:49.268 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=7f24e589328ae9e639c0009d93340e5b235d0a47 2024-09-14T06:27:49.408 DEBUG:teuthology.orchestra.run.smithi138:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-09-14T06:27:49.429 INFO:teuthology.orchestra.run.smithi138.stdout:19.3.0-4775.g7f24e589.el9 2024-09-14T06:27:49.429 INFO:teuthology.packaging:The installed version of ceph is 19.3.0-4775.g7f24e589.el9 2024-09-14T06:27:49.429 INFO:teuthology.task.install:The correct ceph version 19.3.0-4775.g7f24e589 is installed. 2024-09-14T06:27:49.431 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-09-14T06:27:49.431 DEBUG:teuthology.orchestra.run.smithi094:> set -ex 2024-09-14T06:27:49.431 DEBUG:teuthology.orchestra.run.smithi094:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-09-14T06:27:49.462 DEBUG:teuthology.orchestra.run.smithi138:> set -ex 2024-09-14T06:27:49.463 DEBUG:teuthology.orchestra.run.smithi138:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-09-14T06:27:49.493 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-09-14T06:27:49.494 DEBUG:teuthology.orchestra.run.smithi094:> set -ex 2024-09-14T06:27:49.494 DEBUG:teuthology.orchestra.run.smithi094:> sudo dd of=/usr/bin/daemon-helper 2024-09-14T06:27:49.533 DEBUG:teuthology.orchestra.run.smithi094:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-09-14T06:27:49.601 DEBUG:teuthology.orchestra.run.smithi138:> set -ex 2024-09-14T06:27:49.601 DEBUG:teuthology.orchestra.run.smithi138:> sudo dd of=/usr/bin/daemon-helper 2024-09-14T06:27:49.625 DEBUG:teuthology.orchestra.run.smithi138:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-09-14T06:27:49.690 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-09-14T06:27:49.690 DEBUG:teuthology.orchestra.run.smithi094:> set -ex 2024-09-14T06:27:49.690 DEBUG:teuthology.orchestra.run.smithi094:> sudo dd of=/usr/bin/adjust-ulimits 2024-09-14T06:27:49.719 DEBUG:teuthology.orchestra.run.smithi094:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-09-14T06:27:49.787 DEBUG:teuthology.orchestra.run.smithi138:> set -ex 2024-09-14T06:27:49.788 DEBUG:teuthology.orchestra.run.smithi138:> sudo dd of=/usr/bin/adjust-ulimits 2024-09-14T06:27:49.818 DEBUG:teuthology.orchestra.run.smithi138:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-09-14T06:27:49.883 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-09-14T06:27:49.883 DEBUG:teuthology.orchestra.run.smithi094:> set -ex 2024-09-14T06:27:49.883 DEBUG:teuthology.orchestra.run.smithi094:> sudo dd of=/usr/bin/stdin-killer 2024-09-14T06:27:49.911 DEBUG:teuthology.orchestra.run.smithi094:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-09-14T06:27:49.978 DEBUG:teuthology.orchestra.run.smithi138:> set -ex 2024-09-14T06:27:49.978 DEBUG:teuthology.orchestra.run.smithi138:> sudo dd of=/usr/bin/stdin-killer 2024-09-14T06:27:50.002 DEBUG:teuthology.orchestra.run.smithi138:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-09-14T06:27:50.068 INFO:teuthology.run_tasks:Running task ceph... 2024-09-14T06:27:50.146 INFO:tasks.ceph:Making ceph log dir writeable by non-root... 2024-09-14T06:27:50.147 DEBUG:teuthology.orchestra.run.smithi094:> sudo chmod 777 /var/log/ceph 2024-09-14T06:27:50.149 DEBUG:teuthology.orchestra.run.smithi138:> sudo chmod 777 /var/log/ceph 2024-09-14T06:27:50.174 INFO:tasks.ceph:Disabling ceph logrotate... 2024-09-14T06:27:50.174 DEBUG:teuthology.orchestra.run.smithi094:> sudo rm -f -- /etc/logrotate.d/ceph 2024-09-14T06:27:50.217 DEBUG:teuthology.orchestra.run.smithi138:> sudo rm -f -- /etc/logrotate.d/ceph 2024-09-14T06:27:50.241 INFO:tasks.ceph:Creating extra log directories... 2024-09-14T06:27:50.241 DEBUG:teuthology.orchestra.run.smithi094:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-09-14T06:27:50.283 DEBUG:teuthology.orchestra.run.smithi138:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-09-14T06:27:50.317 INFO:tasks.ceph:Creating ceph cluster ceph... 2024-09-14T06:27:50.317 INFO:tasks.ceph:config {'conf': {'global': {'mon client directed command retry': 5, 'mon warn on pool no app': False, 'ms inject socket failures': 5000}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20}, 'osd': {'bluestore block size': 96636764160, 'bluestore compression algorithm': 'snappy', 'bluestore compression mode': 'aggressive', 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd blocked scrub grace period': 3600, 'osd failsafe full ratio': 0.95, 'osd max markdown count': 1000, 'osd objectstore': 'bluestore', 'osd shutdown pgref assert': True}}, 'fs': 'xfs', 'mkfs_options': None, 'mount_options': None, 'skip_mgr_daemons': False, 'log_ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(OSD_SLOW_PING_TIME', 'but it is still running', 'objects unfound and apparently lost', 'overall HEALTH_', '\\(OSDMAP_FLAGS\\)', '\\(OSD_', '\\(PG_', '\\(POOL_', '\\(CACHE_POOL_', '\\(SMALLER_PGP_NUM\\)', '\\(OBJECT_', 'SLOW_OPS', '\\(REQUEST_SLOW\\)', '\\(TOO_FEW_PGS\\)', 'slow request', 'timeout on replica', 'late reservation from', 'MON_DOWN', 'OSDMAP_FLAGS', 'OSD_DOWN', 'PG_DEGRADED', 'PG_AVAILABILITY', 'POOL_APP_NOT_ENABLED', 'mons down', 'mon down', 'out of quorum', 'noscrub', 'nodeep-scrub'], 'cpu_profile': set(), 'cluster': 'ceph', 'mon_bind_msgr2': True, 'mon_bind_addrvec': True} 2024-09-14T06:27:50.317 INFO:tasks.ceph:ctx.config {'archive_path': '/home/teuthworker/archive/teuthology-2024-09-10_20:16:17-rbd-main-distro-default-smithi/7900381', 'branch': 'main', 'description': 'rbd/thrash/{base/install clusters/{fixed-2 openstack} conf/{disable-pool-app} msgr-failures/few objectstore/bluestore-comp-snappy supported-random-distro$/{centos_latest} thrashers/default thrashosds-health workloads/rbd_fsx_deep_copy}', 'email': 'ceph-qa@ceph.com', 'first_in_suite': False, 'job_id': '7900381', 'kernel': {'kdb': 1, 'sha1': 'distro'}, 'last_in_suite': False, 'machine_type': 'smithi', 'name': 'teuthology-2024-09-10_20:16:17-rbd-main-distro-default-smithi', 'no_nested_subset': False, 'openstack': [{'volumes': {'count': 4, 'size': 10}}, {'machine': {'cpus': 1, 'disk': 40, 'ram': 8000}, 'volumes': {'count': 4, 'size': 30}}], 'os_type': 'centos', 'os_version': '9.stream', 'overrides': {'admin_socket': {'branch': 'main'}, 'ceph': {'conf': {'global': {'mon client directed command retry': 5, 'mon warn on pool no app': False, 'ms inject socket failures': 5000}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20}, 'osd': {'bluestore block size': 96636764160, 'bluestore compression algorithm': 'snappy', 'bluestore compression mode': 'aggressive', 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd blocked scrub grace period': 3600, 'osd failsafe full ratio': 0.95, 'osd max markdown count': 1000, 'osd objectstore': 'bluestore', 'osd shutdown pgref assert': True}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(OSD_SLOW_PING_TIME', 'but it is still running', 'objects unfound and apparently lost', 'overall HEALTH_', '\\(OSDMAP_FLAGS\\)', '\\(OSD_', '\\(PG_', '\\(POOL_', '\\(CACHE_POOL_', '\\(SMALLER_PGP_NUM\\)', '\\(OBJECT_', 'SLOW_OPS', '\\(REQUEST_SLOW\\)', '\\(TOO_FEW_PGS\\)', 'slow request', 'timeout on replica', 'late reservation from', 'MON_DOWN', 'OSDMAP_FLAGS', 'OSD_DOWN', 'PG_DEGRADED', 'PG_AVAILABILITY', 'POOL_APP_NOT_ENABLED', 'mons down', 'mon down', 'out of quorum', 'noscrub', 'nodeep-scrub'], 'sha1': '7f24e589328ae9e639c0009d93340e5b235d0a47'}, 'ceph-deploy': {'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {}}}, 'install': {'ceph': {'flavor': 'default', 'sha1': '7f24e589328ae9e639c0009d93340e5b235d0a47'}}, 'thrashosds': {'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5}, 'workunit': {'branch': 'main', 'sha1': '0d61375b8ab53be4c233f5b93e463edfdff9316a'}}, 'owner': 'scheduled_teuthology@teuthology', 'priority': 950, 'repo': 'https://git.ceph.com/ceph.git', 'roles': [['mon.a', 'mon.c', 'mgr.y', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'node-exporter.a'], ['mon.b', 'mgr.x', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'prometheus.a', 'node-exporter.b']], 'seed': 5839, 'sha1': '7f24e589328ae9e639c0009d93340e5b235d0a47', 'sleep_before_teardown': 0, 'subset': '99/128', 'suite': 'rbd', 'suite_branch': 'main', 'suite_path': '/home/teuthworker/src/git.ceph.com_ceph_0d61375b8ab53be4c233f5b93e463edfdff9316a/qa', 'suite_relpath': 'qa', 'suite_repo': 'https://git.ceph.com/ceph.git', 'suite_sha1': '0d61375b8ab53be4c233f5b93e463edfdff9316a', 'targets': {'smithi094.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBMq5orWEJUtY0478KVTlybB6zawRp9ZyFFP3FnqhItrepeTUH6nq/xoX4gF0E++iaEjoDM+Xx0iVas3Gsbg/8p8=', 'smithi138.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBP1hR8CCordjszdEK541OFTVIl0tYp1j7Fan1xPdPvI/fL3IlnOf08uS5vEuocEtpp8IGyv/Ud1buZ/DaYyrs6Y='}, 'tasks': [{'internal.check_packages': None}, {'internal.buildpackages_prep': None}, {'internal.save_config': None}, {'internal.check_lock': None}, {'internal.add_remotes': None}, {'console_log': None}, {'internal.connect': None}, {'internal.push_inventory': None}, {'internal.serialize_remote_roles': None}, {'internal.check_conflict': None}, {'internal.check_ceph_data': None}, {'internal.vm_setup': None}, {'kernel': {'kdb': 1, 'sha1': 'distro'}}, {'internal.base': None}, {'internal.archive_upload': None}, {'internal.archive': None}, {'internal.coredump': None}, {'internal.sudo': None}, {'internal.syslog': None}, {'internal.timer': None}, {'pcp': None}, {'selinux': None}, {'ansible.cephlab': None}, {'clock': None}, {'install': None}, {'ceph': None}, {'thrashosds': {'timeout': 1200}}, {'rbd_fsx': {'clients': ['client.0'], 'deep_copy': True, 'ops': 6000}}], 'teuthology': {'fragments_dropped': [], 'meta': {}, 'postmerge': []}, 'teuthology_branch': 'main', 'teuthology_sha1': '3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a', 'timestamp': '2024-09-10_20:16:17', 'tube': 'smithi', 'user': 'teuthology', 'verbose': False, 'worker_log': '/home/teuthworker/archive/worker_logs/dispatcher.smithi.3401856'} 2024-09-14T06:27:50.318 DEBUG:teuthology.orchestra.run.smithi094:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-09-14T06:27:50.349 DEBUG:teuthology.orchestra.run.smithi138:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-09-14T06:27:50.365 DEBUG:teuthology.orchestra.run.smithi094:> sudo install -d -m0777 -- /var/run/ceph 2024-09-14T06:27:50.405 DEBUG:teuthology.orchestra.run.smithi138:> sudo install -d -m0777 -- /var/run/ceph 2024-09-14T06:27:50.429 DEBUG:teuthology.orchestra.run.smithi094:> set -ex 2024-09-14T06:27:50.429 DEBUG:teuthology.orchestra.run.smithi094:> dd if=/scratch_devs of=/dev/stdout 2024-09-14T06:27:50.485 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-09-14T06:27:50.485 DEBUG:teuthology.orchestra.run.smithi094:> stat /dev/vg_nvme/lv_1 2024-09-14T06:27:50.540 INFO:teuthology.orchestra.run.smithi094.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-09-14T06:27:50.540 INFO:teuthology.orchestra.run.smithi094.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-14T06:27:50.540 INFO:teuthology.orchestra.run.smithi094.stdout:Device: 5h/5d Inode: 667 Links: 1 2024-09-14T06:27:50.540 INFO:teuthology.orchestra.run.smithi094.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-14T06:27:50.540 INFO:teuthology.orchestra.run.smithi094.stdout:Context: system_u:object_r:device_t:s0 2024-09-14T06:27:50.540 INFO:teuthology.orchestra.run.smithi094.stdout:Access: 2024-09-14 06:27:33.598877974 +0000 2024-09-14T06:27:50.540 INFO:teuthology.orchestra.run.smithi094.stdout:Modify: 2024-09-14 06:25:02.990184906 +0000 2024-09-14T06:27:50.541 INFO:teuthology.orchestra.run.smithi094.stdout:Change: 2024-09-14 06:25:02.990184906 +0000 2024-09-14T06:27:50.541 INFO:teuthology.orchestra.run.smithi094.stdout: Birth: 2024-09-14 06:25:02.990184906 +0000 2024-09-14T06:27:50.541 DEBUG:teuthology.orchestra.run.smithi094:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-09-14T06:27:50.603 INFO:teuthology.orchestra.run.smithi094.stderr:1+0 records in 2024-09-14T06:27:50.603 INFO:teuthology.orchestra.run.smithi094.stderr:1+0 records out 2024-09-14T06:27:50.603 INFO:teuthology.orchestra.run.smithi094.stderr:512 bytes copied, 9.7849e-05 s, 5.2 MB/s 2024-09-14T06:27:50.605 DEBUG:teuthology.orchestra.run.smithi094:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-09-14T06:27:50.661 DEBUG:teuthology.orchestra.run.smithi094:> stat /dev/vg_nvme/lv_2 2024-09-14T06:27:50.717 INFO:teuthology.orchestra.run.smithi094.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-09-14T06:27:50.717 INFO:teuthology.orchestra.run.smithi094.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-14T06:27:50.717 INFO:teuthology.orchestra.run.smithi094.stdout:Device: 5h/5d Inode: 687 Links: 1 2024-09-14T06:27:50.717 INFO:teuthology.orchestra.run.smithi094.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-14T06:27:50.717 INFO:teuthology.orchestra.run.smithi094.stdout:Context: system_u:object_r:device_t:s0 2024-09-14T06:27:50.718 INFO:teuthology.orchestra.run.smithi094.stdout:Access: 2024-09-14 06:27:33.599877957 +0000 2024-09-14T06:27:50.718 INFO:teuthology.orchestra.run.smithi094.stdout:Modify: 2024-09-14 06:25:03.311179833 +0000 2024-09-14T06:27:50.718 INFO:teuthology.orchestra.run.smithi094.stdout:Change: 2024-09-14 06:25:03.311179833 +0000 2024-09-14T06:27:50.718 INFO:teuthology.orchestra.run.smithi094.stdout: Birth: 2024-09-14 06:25:03.311179833 +0000 2024-09-14T06:27:50.718 DEBUG:teuthology.orchestra.run.smithi094:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-09-14T06:27:50.780 INFO:teuthology.orchestra.run.smithi094.stderr:1+0 records in 2024-09-14T06:27:50.780 INFO:teuthology.orchestra.run.smithi094.stderr:1+0 records out 2024-09-14T06:27:50.780 INFO:teuthology.orchestra.run.smithi094.stderr:512 bytes copied, 8.1901e-05 s, 6.3 MB/s 2024-09-14T06:27:50.781 DEBUG:teuthology.orchestra.run.smithi094:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-09-14T06:27:50.837 DEBUG:teuthology.orchestra.run.smithi094:> stat /dev/vg_nvme/lv_3 2024-09-14T06:27:50.892 INFO:teuthology.orchestra.run.smithi094.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-09-14T06:27:50.892 INFO:teuthology.orchestra.run.smithi094.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-14T06:27:50.892 INFO:teuthology.orchestra.run.smithi094.stdout:Device: 5h/5d Inode: 703 Links: 1 2024-09-14T06:27:50.892 INFO:teuthology.orchestra.run.smithi094.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-14T06:27:50.892 INFO:teuthology.orchestra.run.smithi094.stdout:Context: system_u:object_r:device_t:s0 2024-09-14T06:27:50.892 INFO:teuthology.orchestra.run.smithi094.stdout:Access: 2024-09-14 06:27:33.599877957 +0000 2024-09-14T06:27:50.892 INFO:teuthology.orchestra.run.smithi094.stdout:Modify: 2024-09-14 06:25:03.631174775 +0000 2024-09-14T06:27:50.892 INFO:teuthology.orchestra.run.smithi094.stdout:Change: 2024-09-14 06:25:03.631174775 +0000 2024-09-14T06:27:50.892 INFO:teuthology.orchestra.run.smithi094.stdout: Birth: 2024-09-14 06:25:03.631174775 +0000 2024-09-14T06:27:50.893 DEBUG:teuthology.orchestra.run.smithi094:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-09-14T06:27:50.955 INFO:teuthology.orchestra.run.smithi094.stderr:1+0 records in 2024-09-14T06:27:50.955 INFO:teuthology.orchestra.run.smithi094.stderr:1+0 records out 2024-09-14T06:27:50.955 INFO:teuthology.orchestra.run.smithi094.stderr:512 bytes copied, 0.000115757 s, 4.4 MB/s 2024-09-14T06:27:50.956 DEBUG:teuthology.orchestra.run.smithi094:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-09-14T06:27:51.012 DEBUG:teuthology.orchestra.run.smithi094:> stat /dev/vg_nvme/lv_4 2024-09-14T06:27:51.067 INFO:teuthology.orchestra.run.smithi094.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-09-14T06:27:51.067 INFO:teuthology.orchestra.run.smithi094.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-14T06:27:51.067 INFO:teuthology.orchestra.run.smithi094.stdout:Device: 5h/5d Inode: 725 Links: 1 2024-09-14T06:27:51.067 INFO:teuthology.orchestra.run.smithi094.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-14T06:27:51.067 INFO:teuthology.orchestra.run.smithi094.stdout:Context: system_u:object_r:device_t:s0 2024-09-14T06:27:51.067 INFO:teuthology.orchestra.run.smithi094.stdout:Access: 2024-09-14 06:27:33.599877957 +0000 2024-09-14T06:27:51.067 INFO:teuthology.orchestra.run.smithi094.stdout:Modify: 2024-09-14 06:25:03.946169796 +0000 2024-09-14T06:27:51.067 INFO:teuthology.orchestra.run.smithi094.stdout:Change: 2024-09-14 06:25:03.946169796 +0000 2024-09-14T06:27:51.067 INFO:teuthology.orchestra.run.smithi094.stdout: Birth: 2024-09-14 06:25:03.946169796 +0000 2024-09-14T06:27:51.068 DEBUG:teuthology.orchestra.run.smithi094:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-09-14T06:27:51.130 INFO:teuthology.orchestra.run.smithi094.stderr:1+0 records in 2024-09-14T06:27:51.130 INFO:teuthology.orchestra.run.smithi094.stderr:1+0 records out 2024-09-14T06:27:51.130 INFO:teuthology.orchestra.run.smithi094.stderr:512 bytes copied, 0.000100259 s, 5.1 MB/s 2024-09-14T06:27:51.131 DEBUG:teuthology.orchestra.run.smithi094:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-09-14T06:27:51.188 INFO:tasks.ceph:osd dev map: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-09-14T06:27:51.188 DEBUG:teuthology.orchestra.run.smithi138:> set -ex 2024-09-14T06:27:51.188 DEBUG:teuthology.orchestra.run.smithi138:> dd if=/scratch_devs of=/dev/stdout 2024-09-14T06:27:51.205 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-09-14T06:27:51.205 DEBUG:teuthology.orchestra.run.smithi138:> stat /dev/vg_nvme/lv_1 2024-09-14T06:27:51.261 INFO:teuthology.orchestra.run.smithi138.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-09-14T06:27:51.261 INFO:teuthology.orchestra.run.smithi138.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-14T06:27:51.262 INFO:teuthology.orchestra.run.smithi138.stdout:Device: 5h/5d Inode: 666 Links: 1 2024-09-14T06:27:51.262 INFO:teuthology.orchestra.run.smithi138.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-14T06:27:51.262 INFO:teuthology.orchestra.run.smithi138.stdout:Context: system_u:object_r:device_t:s0 2024-09-14T06:27:51.262 INFO:teuthology.orchestra.run.smithi138.stdout:Access: 2024-09-14 06:27:45.681589282 +0000 2024-09-14T06:27:51.262 INFO:teuthology.orchestra.run.smithi138.stdout:Modify: 2024-09-14 06:25:01.003746214 +0000 2024-09-14T06:27:51.262 INFO:teuthology.orchestra.run.smithi138.stdout:Change: 2024-09-14 06:25:01.003746214 +0000 2024-09-14T06:27:51.262 INFO:teuthology.orchestra.run.smithi138.stdout: Birth: 2024-09-14 06:25:01.003746214 +0000 2024-09-14T06:27:51.262 DEBUG:teuthology.orchestra.run.smithi138:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-09-14T06:27:51.325 INFO:teuthology.orchestra.run.smithi138.stderr:1+0 records in 2024-09-14T06:27:51.326 INFO:teuthology.orchestra.run.smithi138.stderr:1+0 records out 2024-09-14T06:27:51.326 INFO:teuthology.orchestra.run.smithi138.stderr:512 bytes copied, 0.000210406 s, 2.4 MB/s 2024-09-14T06:27:51.327 DEBUG:teuthology.orchestra.run.smithi138:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-09-14T06:27:51.384 DEBUG:teuthology.orchestra.run.smithi138:> stat /dev/vg_nvme/lv_2 2024-09-14T06:27:51.440 INFO:teuthology.orchestra.run.smithi138.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-09-14T06:27:51.440 INFO:teuthology.orchestra.run.smithi138.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-14T06:27:51.440 INFO:teuthology.orchestra.run.smithi138.stdout:Device: 5h/5d Inode: 686 Links: 1 2024-09-14T06:27:51.440 INFO:teuthology.orchestra.run.smithi138.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-14T06:27:51.440 INFO:teuthology.orchestra.run.smithi138.stdout:Context: system_u:object_r:device_t:s0 2024-09-14T06:27:51.440 INFO:teuthology.orchestra.run.smithi138.stdout:Access: 2024-09-14 06:27:45.681589282 +0000 2024-09-14T06:27:51.440 INFO:teuthology.orchestra.run.smithi138.stdout:Modify: 2024-09-14 06:25:01.345738900 +0000 2024-09-14T06:27:51.440 INFO:teuthology.orchestra.run.smithi138.stdout:Change: 2024-09-14 06:25:01.345738900 +0000 2024-09-14T06:27:51.440 INFO:teuthology.orchestra.run.smithi138.stdout: Birth: 2024-09-14 06:25:01.345738900 +0000 2024-09-14T06:27:51.440 DEBUG:teuthology.orchestra.run.smithi138:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-09-14T06:27:51.504 INFO:teuthology.orchestra.run.smithi138.stderr:1+0 records in 2024-09-14T06:27:51.504 INFO:teuthology.orchestra.run.smithi138.stderr:1+0 records out 2024-09-14T06:27:51.505 INFO:teuthology.orchestra.run.smithi138.stderr:512 bytes copied, 0.00019229 s, 2.7 MB/s 2024-09-14T06:27:51.506 DEBUG:teuthology.orchestra.run.smithi138:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-09-14T06:27:51.562 DEBUG:teuthology.orchestra.run.smithi138:> stat /dev/vg_nvme/lv_3 2024-09-14T06:27:51.618 INFO:teuthology.orchestra.run.smithi138.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-09-14T06:27:51.618 INFO:teuthology.orchestra.run.smithi138.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-14T06:27:51.618 INFO:teuthology.orchestra.run.smithi138.stdout:Device: 5h/5d Inode: 703 Links: 1 2024-09-14T06:27:51.619 INFO:teuthology.orchestra.run.smithi138.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-14T06:27:51.619 INFO:teuthology.orchestra.run.smithi138.stdout:Context: system_u:object_r:device_t:s0 2024-09-14T06:27:51.619 INFO:teuthology.orchestra.run.smithi138.stdout:Access: 2024-09-14 06:27:45.682589261 +0000 2024-09-14T06:27:51.619 INFO:teuthology.orchestra.run.smithi138.stdout:Modify: 2024-09-14 06:25:01.665732055 +0000 2024-09-14T06:27:51.619 INFO:teuthology.orchestra.run.smithi138.stdout:Change: 2024-09-14 06:25:01.665732055 +0000 2024-09-14T06:27:51.619 INFO:teuthology.orchestra.run.smithi138.stdout: Birth: 2024-09-14 06:25:01.665732055 +0000 2024-09-14T06:27:51.619 DEBUG:teuthology.orchestra.run.smithi138:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-09-14T06:27:51.681 INFO:teuthology.orchestra.run.smithi138.stderr:1+0 records in 2024-09-14T06:27:51.681 INFO:teuthology.orchestra.run.smithi138.stderr:1+0 records out 2024-09-14T06:27:51.681 INFO:teuthology.orchestra.run.smithi138.stderr:512 bytes copied, 0.000186133 s, 2.8 MB/s 2024-09-14T06:27:51.682 DEBUG:teuthology.orchestra.run.smithi138:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-09-14T06:27:51.739 DEBUG:teuthology.orchestra.run.smithi138:> stat /dev/vg_nvme/lv_4 2024-09-14T06:27:51.795 INFO:teuthology.orchestra.run.smithi138.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-09-14T06:27:51.795 INFO:teuthology.orchestra.run.smithi138.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-14T06:27:51.795 INFO:teuthology.orchestra.run.smithi138.stdout:Device: 5h/5d Inode: 725 Links: 1 2024-09-14T06:27:51.795 INFO:teuthology.orchestra.run.smithi138.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-14T06:27:51.795 INFO:teuthology.orchestra.run.smithi138.stdout:Context: system_u:object_r:device_t:s0 2024-09-14T06:27:51.795 INFO:teuthology.orchestra.run.smithi138.stdout:Access: 2024-09-14 06:27:45.682589261 +0000 2024-09-14T06:27:51.795 INFO:teuthology.orchestra.run.smithi138.stdout:Modify: 2024-09-14 06:25:01.993725040 +0000 2024-09-14T06:27:51.795 INFO:teuthology.orchestra.run.smithi138.stdout:Change: 2024-09-14 06:25:01.993725040 +0000 2024-09-14T06:27:51.796 INFO:teuthology.orchestra.run.smithi138.stdout: Birth: 2024-09-14 06:25:01.993725040 +0000 2024-09-14T06:27:51.796 DEBUG:teuthology.orchestra.run.smithi138:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-09-14T06:27:51.858 INFO:teuthology.orchestra.run.smithi138.stderr:1+0 records in 2024-09-14T06:27:51.858 INFO:teuthology.orchestra.run.smithi138.stderr:1+0 records out 2024-09-14T06:27:51.858 INFO:teuthology.orchestra.run.smithi138.stderr:512 bytes copied, 0.000196164 s, 2.6 MB/s 2024-09-14T06:27:51.860 DEBUG:teuthology.orchestra.run.smithi138:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-09-14T06:27:51.916 INFO:tasks.ceph:osd dev map: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-09-14T06:27:51.916 INFO:tasks.ceph:remote_to_roles_to_devs: {Remote(name='ubuntu@smithi094.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi138.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2024-09-14T06:27:51.916 INFO:tasks.ceph:Generating config... 2024-09-14T06:27:51.918 INFO:tasks.ceph:[global] mon client directed command retry = 5 2024-09-14T06:27:51.918 INFO:tasks.ceph:[global] mon warn on pool no app = False 2024-09-14T06:27:51.918 INFO:tasks.ceph:[global] ms inject socket failures = 5000 2024-09-14T06:27:51.918 INFO:tasks.ceph:[mgr] debug mgr = 20 2024-09-14T06:27:51.918 INFO:tasks.ceph:[mgr] debug ms = 1 2024-09-14T06:27:51.918 INFO:tasks.ceph:[mon] debug mon = 20 2024-09-14T06:27:51.918 INFO:tasks.ceph:[mon] debug ms = 1 2024-09-14T06:27:51.918 INFO:tasks.ceph:[mon] debug paxos = 20 2024-09-14T06:27:51.918 INFO:tasks.ceph:[osd] bluestore block size = 96636764160 2024-09-14T06:27:51.918 INFO:tasks.ceph:[osd] bluestore compression algorithm = snappy 2024-09-14T06:27:51.918 INFO:tasks.ceph:[osd] bluestore compression mode = aggressive 2024-09-14T06:27:51.918 INFO:tasks.ceph:[osd] bluestore fsck on mount = True 2024-09-14T06:27:51.918 INFO:tasks.ceph:[osd] debug bluefs = 1/20 2024-09-14T06:27:51.919 INFO:tasks.ceph:[osd] debug bluestore = 1/20 2024-09-14T06:27:51.919 INFO:tasks.ceph:[osd] debug ms = 1 2024-09-14T06:27:51.919 INFO:tasks.ceph:[osd] debug osd = 20 2024-09-14T06:27:51.919 INFO:tasks.ceph:[osd] debug rocksdb = 4/10 2024-09-14T06:27:51.919 INFO:tasks.ceph:[osd] mon osd backfillfull_ratio = 0.85 2024-09-14T06:27:51.919 INFO:tasks.ceph:[osd] mon osd full ratio = 0.9 2024-09-14T06:27:51.919 INFO:tasks.ceph:[osd] mon osd nearfull ratio = 0.8 2024-09-14T06:27:51.919 INFO:tasks.ceph:[osd] osd blocked scrub grace period = 3600 2024-09-14T06:27:51.919 INFO:tasks.ceph:[osd] osd failsafe full ratio = 0.95 2024-09-14T06:27:51.919 INFO:tasks.ceph:[osd] osd max markdown count = 1000 2024-09-14T06:27:51.919 INFO:tasks.ceph:[osd] osd objectstore = bluestore 2024-09-14T06:27:51.919 INFO:tasks.ceph:[osd] osd shutdown pgref assert = True 2024-09-14T06:27:51.919 INFO:tasks.ceph:Setting up mon.a... 2024-09-14T06:27:51.920 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring /etc/ceph/ceph.keyring 2024-09-14T06:27:51.966 INFO:teuthology.orchestra.run.smithi094.stdout:creating /etc/ceph/ceph.keyring 2024-09-14T06:27:51.969 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=mon. /etc/ceph/ceph.keyring 2024-09-14T06:27:52.011 DEBUG:teuthology.orchestra.run.smithi094:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-09-14T06:27:52.075 DEBUG:tasks.ceph:Ceph mon addresses: [('mon.a', '172.21.15.94'), ('mon.c', '[v2:172.21.15.94:3301,v1:172.21.15.94:6790]'), ('mon.b', '172.21.15.138')] 2024-09-14T06:27:52.076 DEBUG:tasks.ceph:writing out conf {'global': {'chdir': '', 'pid file': '/var/run/ceph/$cluster-$name.pid', 'auth supported': 'cephx', 'filestore xattr use omap': 'true', 'mon clock drift allowed': '1.000', 'osd crush chooseleaf type': '0', 'auth debug': 'true', 'ms die on old message': 'true', 'ms die on bug': 'true', 'mon max pg per osd': '10000', 'mon pg warn max object skew': '0', 'osd_pool_default_pg_autoscale_mode': 'off', 'osd pool default size': '2', 'mon osd allow primary affinity': 'true', 'mon osd allow pg remap': 'true', 'mon warn on legacy crush tunables': 'false', 'mon warn on crush straw calc version zero': 'false', 'mon warn on no sortbitwise': 'false', 'mon warn on osd down out interval zero': 'false', 'mon warn on too few osds': 'false', 'mon_warn_on_pool_pg_num_not_power_of_two': 'false', 'mon_warn_on_pool_no_redundancy': 'false', 'mon_allow_pool_size_one': 'true', 'osd pool default erasure code profile': 'plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd', 'osd default data pool replay window': '5', 'mon allow pool delete': 'true', 'mon cluster log file level': 'debug', 'debug asserts on shutdown': 'true', 'mon health detail to clog': 'false', 'mon host': '172.21.15.94,[v2:172.21.15.94:3301,v1:172.21.15.94:6790],172.21.15.138', 'mon client directed command retry': 5, 'mon warn on pool no app': False, 'ms inject socket failures': 5000}, 'osd': {'osd journal size': '100', 'osd scrub load threshold': '5.0', 'osd scrub max interval': '600', 'osd mclock profile': 'high_recovery_ops', 'osd recover clone overlap': 'true', 'osd recovery max chunk': '1048576', 'osd debug shutdown': 'true', 'osd debug op order': 'true', 'osd debug verify stray on activate': 'true', 'osd debug trim objects': 'true', 'osd open classes on start': 'true', 'osd debug pg log writeout': 'true', 'osd deep scrub update digest min age': '30', 'osd map max advance': '10', 'journal zero on create': 'true', 'filestore ondisk finisher threads': '3', 'filestore apply finisher threads': '3', 'bdev debug aio': 'true', 'osd debug misdirected ops': 'true', 'bluestore block size': 96636764160, 'bluestore compression algorithm': 'snappy', 'bluestore compression mode': 'aggressive', 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd blocked scrub grace period': 3600, 'osd failsafe full ratio': 0.95, 'osd max markdown count': 1000, 'osd objectstore': 'bluestore', 'osd shutdown pgref assert': True}, 'mgr': {'debug ms': 1, 'debug mgr': 20, 'debug mon': '20', 'debug auth': '20', 'mon reweight min pgs per osd': '4', 'mon reweight min bytes per osd': '10', 'mgr/telemetry/nag': 'false'}, 'mon': {'debug ms': 1, 'debug mon': 20, 'debug paxos': 20, 'debug auth': '20', 'mon data avail warn': '5', 'mon mgr mkfs grace': '240', 'mon reweight min pgs per osd': '4', 'mon osd reporter subtree level': 'osd', 'mon osd prime pg temp': 'true', 'mon reweight min bytes per osd': '10', 'auth mon ticket ttl': '660', 'auth service ticket ttl': '240', 'mon_warn_on_insecure_global_id_reclaim': 'false', 'mon_warn_on_insecure_global_id_reclaim_allowed': 'false', 'mon_down_mkfs_grace': '2m', 'mon_warn_on_filestore_osds': 'false'}, 'client': {'rgw cache enabled': 'true', 'rgw enable ops log': 'true', 'rgw enable usage log': 'true', 'log file': '/var/log/ceph/$cluster-$name.$pid.log', 'admin socket': '/var/run/ceph/$cluster-$name.$pid.asok'}, 'mon.a': {}, 'mon.c': {}, 'mon.b': {}} 2024-09-14T06:27:52.076 DEBUG:teuthology.orchestra.run.smithi094:> set -ex 2024-09-14T06:27:52.076 DEBUG:teuthology.orchestra.run.smithi094:> dd of=/home/ubuntu/cephtest/ceph.tmp.conf 2024-09-14T06:27:52.130 DEBUG:teuthology.orchestra.run.smithi094:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage monmaptool -c /home/ubuntu/cephtest/ceph.tmp.conf --create --clobber --enable-all-features --add a 172.21.15.94 --addv c '[v2:172.21.15.94:3301,v1:172.21.15.94:6790]' --add b 172.21.15.138 --print /home/ubuntu/cephtest/ceph.monmap 2024-09-14T06:27:52.216 INFO:teuthology.orchestra.run.smithi094.stdout:monmaptool: monmap file /home/ubuntu/cephtest/ceph.monmap 2024-09-14T06:27:52.216 INFO:teuthology.orchestra.run.smithi094.stdout:monmaptool: generated fsid 200ad3da-bd63-4e04-af74-efb5ca0af75e 2024-09-14T06:27:52.216 INFO:teuthology.orchestra.run.smithi094.stdout:setting min_mon_release = quincy 2024-09-14T06:27:52.216 INFO:teuthology.orchestra.run.smithi094.stdout:epoch 0 2024-09-14T06:27:52.216 INFO:teuthology.orchestra.run.smithi094.stdout:fsid 200ad3da-bd63-4e04-af74-efb5ca0af75e 2024-09-14T06:27:52.217 INFO:teuthology.orchestra.run.smithi094.stdout:last_changed 2024-09-14T06:27:52.215802+0000 2024-09-14T06:27:52.217 INFO:teuthology.orchestra.run.smithi094.stdout:created 2024-09-14T06:27:52.215802+0000 2024-09-14T06:27:52.217 INFO:teuthology.orchestra.run.smithi094.stdout:min_mon_release 17 (quincy) 2024-09-14T06:27:52.217 INFO:teuthology.orchestra.run.smithi094.stdout:election_strategy: 1 2024-09-14T06:27:52.217 INFO:teuthology.orchestra.run.smithi094.stdout:0: [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] mon.a 2024-09-14T06:27:52.217 INFO:teuthology.orchestra.run.smithi094.stdout:1: [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] mon.b 2024-09-14T06:27:52.217 INFO:teuthology.orchestra.run.smithi094.stdout:2: [v2:172.21.15.94:3301/0,v1:172.21.15.94:6790/0] mon.c 2024-09-14T06:27:52.217 INFO:teuthology.orchestra.run.smithi094.stdout:monmaptool: writing epoch 0 to /home/ubuntu/cephtest/ceph.monmap (3 monitors) 2024-09-14T06:27:52.218 DEBUG:teuthology.orchestra.run.smithi094:> rm -- /home/ubuntu/cephtest/ceph.tmp.conf 2024-09-14T06:27:52.233 INFO:tasks.ceph:Writing /etc/ceph/ceph.conf for FSID 200ad3da-bd63-4e04-af74-efb5ca0af75e... 2024-09-14T06:27:52.234 DEBUG:teuthology.orchestra.run.smithi094:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-09-14T06:27:52.275 DEBUG:teuthology.orchestra.run.smithi138:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-09-14T06:27:52.315 INFO:teuthology.orchestra.run.smithi094.stdout:[global] 2024-09-14T06:27:52.316 INFO:teuthology.orchestra.run.smithi094.stdout: chdir = "" 2024-09-14T06:27:52.316 INFO:teuthology.orchestra.run.smithi094.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-09-14T06:27:52.316 INFO:teuthology.orchestra.run.smithi094.stdout: auth supported = cephx 2024-09-14T06:27:52.316 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-14T06:27:52.316 INFO:teuthology.orchestra.run.smithi094.stdout: filestore xattr use omap = true 2024-09-14T06:27:52.316 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-14T06:27:52.316 INFO:teuthology.orchestra.run.smithi094.stdout: mon clock drift allowed = 1.000 2024-09-14T06:27:52.316 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-14T06:27:52.316 INFO:teuthology.orchestra.run.smithi094.stdout: osd crush chooseleaf type = 0 2024-09-14T06:27:52.316 INFO:teuthology.orchestra.run.smithi094.stdout: auth debug = true 2024-09-14T06:27:52.316 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-14T06:27:52.316 INFO:teuthology.orchestra.run.smithi094.stdout: ms die on old message = true 2024-09-14T06:27:52.316 INFO:teuthology.orchestra.run.smithi094.stdout: ms die on bug = true 2024-09-14T06:27:52.317 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-14T06:27:52.317 INFO:teuthology.orchestra.run.smithi094.stdout: mon max pg per osd = 10000 # >= luminous 2024-09-14T06:27:52.317 INFO:teuthology.orchestra.run.smithi094.stdout: mon pg warn max object skew = 0 2024-09-14T06:27:52.317 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-14T06:27:52.317 INFO:teuthology.orchestra.run.smithi094.stdout: # disable pg_autoscaler by default for new pools 2024-09-14T06:27:52.317 INFO:teuthology.orchestra.run.smithi094.stdout: osd_pool_default_pg_autoscale_mode = off 2024-09-14T06:27:52.317 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-14T06:27:52.317 INFO:teuthology.orchestra.run.smithi094.stdout: osd pool default size = 2 2024-09-14T06:27:52.317 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-14T06:27:52.317 INFO:teuthology.orchestra.run.smithi094.stdout: mon osd allow primary affinity = true 2024-09-14T06:27:52.317 INFO:teuthology.orchestra.run.smithi094.stdout: mon osd allow pg remap = true 2024-09-14T06:27:52.317 INFO:teuthology.orchestra.run.smithi094.stdout: mon warn on legacy crush tunables = false 2024-09-14T06:27:52.317 INFO:teuthology.orchestra.run.smithi094.stdout: mon warn on crush straw calc version zero = false 2024-09-14T06:27:52.318 INFO:teuthology.orchestra.run.smithi094.stdout: mon warn on no sortbitwise = false 2024-09-14T06:27:52.318 INFO:teuthology.orchestra.run.smithi094.stdout: mon warn on osd down out interval zero = false 2024-09-14T06:27:52.318 INFO:teuthology.orchestra.run.smithi094.stdout: mon warn on too few osds = false 2024-09-14T06:27:52.318 INFO:teuthology.orchestra.run.smithi094.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-09-14T06:27:52.318 INFO:teuthology.orchestra.run.smithi094.stdout: mon_warn_on_pool_no_redundancy = false 2024-09-14T06:27:52.318 INFO:teuthology.orchestra.run.smithi094.stdout: mon_allow_pool_size_one = true 2024-09-14T06:27:52.318 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-14T06:27:52.318 INFO:teuthology.orchestra.run.smithi094.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-09-14T06:27:52.318 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-14T06:27:52.318 INFO:teuthology.orchestra.run.smithi094.stdout: osd default data pool replay window = 5 2024-09-14T06:27:52.318 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-14T06:27:52.318 INFO:teuthology.orchestra.run.smithi094.stdout: mon allow pool delete = true 2024-09-14T06:27:52.318 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-14T06:27:52.318 INFO:teuthology.orchestra.run.smithi094.stdout: mon cluster log file level = debug 2024-09-14T06:27:52.319 INFO:teuthology.orchestra.run.smithi094.stdout: debug asserts on shutdown = true 2024-09-14T06:27:52.319 INFO:teuthology.orchestra.run.smithi094.stdout: mon health detail to clog = false 2024-09-14T06:27:52.319 INFO:teuthology.orchestra.run.smithi094.stdout: mon host = "172.21.15.94,[v2:172.21.15.94:3301,v1:172.21.15.94:6790],172.21.15.138" 2024-09-14T06:27:52.319 INFO:teuthology.orchestra.run.smithi094.stdout: mon client directed command retry = 5 2024-09-14T06:27:52.319 INFO:teuthology.orchestra.run.smithi094.stdout: mon warn on pool no app = False 2024-09-14T06:27:52.319 INFO:teuthology.orchestra.run.smithi094.stdout: ms inject socket failures = 5000 2024-09-14T06:27:52.319 INFO:teuthology.orchestra.run.smithi094.stdout: fsid = 200ad3da-bd63-4e04-af74-efb5ca0af75e 2024-09-14T06:27:52.319 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-14T06:27:52.319 INFO:teuthology.orchestra.run.smithi094.stdout:[osd] 2024-09-14T06:27:52.319 INFO:teuthology.orchestra.run.smithi094.stdout: osd journal size = 100 2024-09-14T06:27:52.319 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-14T06:27:52.319 INFO:teuthology.orchestra.run.smithi094.stdout: osd scrub load threshold = 5.0 2024-09-14T06:27:52.319 INFO:teuthology.orchestra.run.smithi094.stdout: osd scrub max interval = 600 2024-09-14T06:27:52.319 INFO:teuthology.orchestra.run.smithi094.stdout: osd mclock profile = high_recovery_ops 2024-09-14T06:27:52.320 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-14T06:27:52.320 INFO:teuthology.orchestra.run.smithi094.stdout: osd recover clone overlap = true 2024-09-14T06:27:52.320 INFO:teuthology.orchestra.run.smithi094.stdout: osd recovery max chunk = 1048576 2024-09-14T06:27:52.320 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-14T06:27:52.320 INFO:teuthology.orchestra.run.smithi094.stdout: osd debug shutdown = true 2024-09-14T06:27:52.320 INFO:teuthology.orchestra.run.smithi094.stdout: osd debug op order = true 2024-09-14T06:27:52.320 INFO:teuthology.orchestra.run.smithi094.stdout: osd debug verify stray on activate = true 2024-09-14T06:27:52.320 INFO:teuthology.orchestra.run.smithi094.stdout: osd debug trim objects = true 2024-09-14T06:27:52.320 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-14T06:27:52.320 INFO:teuthology.orchestra.run.smithi094.stdout: osd open classes on start = true 2024-09-14T06:27:52.320 INFO:teuthology.orchestra.run.smithi094.stdout: osd debug pg log writeout = true 2024-09-14T06:27:52.320 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-14T06:27:52.320 INFO:teuthology.orchestra.run.smithi094.stdout: osd deep scrub update digest min age = 30 2024-09-14T06:27:52.320 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-14T06:27:52.321 INFO:teuthology.orchestra.run.smithi094.stdout: osd map max advance = 10 2024-09-14T06:27:52.321 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-14T06:27:52.321 INFO:teuthology.orchestra.run.smithi094.stdout: journal zero on create = true 2024-09-14T06:27:52.321 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-14T06:27:52.321 INFO:teuthology.orchestra.run.smithi094.stdout: filestore ondisk finisher threads = 3 2024-09-14T06:27:52.321 INFO:teuthology.orchestra.run.smithi094.stdout: filestore apply finisher threads = 3 2024-09-14T06:27:52.321 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-14T06:27:52.321 INFO:teuthology.orchestra.run.smithi094.stdout: bdev debug aio = true 2024-09-14T06:27:52.321 INFO:teuthology.orchestra.run.smithi094.stdout: osd debug misdirected ops = true 2024-09-14T06:27:52.321 INFO:teuthology.orchestra.run.smithi094.stdout: bluestore block size = 96636764160 2024-09-14T06:27:52.321 INFO:teuthology.orchestra.run.smithi094.stdout: bluestore compression algorithm = snappy 2024-09-14T06:27:52.321 INFO:teuthology.orchestra.run.smithi094.stdout: bluestore compression mode = aggressive 2024-09-14T06:27:52.321 INFO:teuthology.orchestra.run.smithi094.stdout: bluestore fsck on mount = True 2024-09-14T06:27:52.321 INFO:teuthology.orchestra.run.smithi094.stdout: debug bluefs = 1/20 2024-09-14T06:27:52.322 INFO:teuthology.orchestra.run.smithi094.stdout: debug bluestore = 1/20 2024-09-14T06:27:52.322 INFO:teuthology.orchestra.run.smithi094.stdout: debug ms = 1 2024-09-14T06:27:52.322 INFO:teuthology.orchestra.run.smithi094.stdout: debug osd = 20 2024-09-14T06:27:52.322 INFO:teuthology.orchestra.run.smithi094.stdout: debug rocksdb = 4/10 2024-09-14T06:27:52.322 INFO:teuthology.orchestra.run.smithi094.stdout: mon osd backfillfull_ratio = 0.85 2024-09-14T06:27:52.322 INFO:teuthology.orchestra.run.smithi094.stdout: mon osd full ratio = 0.9 2024-09-14T06:27:52.322 INFO:teuthology.orchestra.run.smithi094.stdout: mon osd nearfull ratio = 0.8 2024-09-14T06:27:52.322 INFO:teuthology.orchestra.run.smithi094.stdout: osd blocked scrub grace period = 3600 2024-09-14T06:27:52.322 INFO:teuthology.orchestra.run.smithi094.stdout: osd failsafe full ratio = 0.95 2024-09-14T06:27:52.322 INFO:teuthology.orchestra.run.smithi094.stdout: osd max markdown count = 1000 2024-09-14T06:27:52.322 INFO:teuthology.orchestra.run.smithi094.stdout: osd objectstore = bluestore 2024-09-14T06:27:52.322 INFO:teuthology.orchestra.run.smithi094.stdout: osd shutdown pgref assert = True 2024-09-14T06:27:52.322 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-14T06:27:52.322 INFO:teuthology.orchestra.run.smithi094.stdout:[mgr] 2024-09-14T06:27:52.323 INFO:teuthology.orchestra.run.smithi094.stdout: debug ms = 1 2024-09-14T06:27:52.323 INFO:teuthology.orchestra.run.smithi094.stdout: debug mgr = 20 2024-09-14T06:27:52.323 INFO:teuthology.orchestra.run.smithi094.stdout: debug mon = 20 2024-09-14T06:27:52.323 INFO:teuthology.orchestra.run.smithi094.stdout: debug auth = 20 2024-09-14T06:27:52.323 INFO:teuthology.orchestra.run.smithi094.stdout: mon reweight min pgs per osd = 4 2024-09-14T06:27:52.323 INFO:teuthology.orchestra.run.smithi094.stdout: mon reweight min bytes per osd = 10 2024-09-14T06:27:52.323 INFO:teuthology.orchestra.run.smithi094.stdout: mgr/telemetry/nag = false 2024-09-14T06:27:52.323 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-14T06:27:52.323 INFO:teuthology.orchestra.run.smithi094.stdout:[mon] 2024-09-14T06:27:52.323 INFO:teuthology.orchestra.run.smithi094.stdout: debug ms = 1 2024-09-14T06:27:52.323 INFO:teuthology.orchestra.run.smithi094.stdout: debug mon = 20 2024-09-14T06:27:52.323 INFO:teuthology.orchestra.run.smithi094.stdout: debug paxos = 20 2024-09-14T06:27:52.323 INFO:teuthology.orchestra.run.smithi094.stdout: debug auth = 20 2024-09-14T06:27:52.323 INFO:teuthology.orchestra.run.smithi094.stdout: mon data avail warn = 5 2024-09-14T06:27:52.324 INFO:teuthology.orchestra.run.smithi094.stdout: mon mgr mkfs grace = 240 2024-09-14T06:27:52.324 INFO:teuthology.orchestra.run.smithi094.stdout: mon reweight min pgs per osd = 4 2024-09-14T06:27:52.324 INFO:teuthology.orchestra.run.smithi094.stdout: mon osd reporter subtree level = osd 2024-09-14T06:27:52.324 INFO:teuthology.orchestra.run.smithi094.stdout: mon osd prime pg temp = true 2024-09-14T06:27:52.324 INFO:teuthology.orchestra.run.smithi094.stdout: mon reweight min bytes per osd = 10 2024-09-14T06:27:52.324 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-14T06:27:52.324 INFO:teuthology.orchestra.run.smithi094.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-09-14T06:27:52.324 INFO:teuthology.orchestra.run.smithi094.stdout: auth mon ticket ttl = 660 # 11m 2024-09-14T06:27:52.324 INFO:teuthology.orchestra.run.smithi094.stdout: auth service ticket ttl = 240 # 4m 2024-09-14T06:27:52.324 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-14T06:27:52.324 INFO:teuthology.orchestra.run.smithi094.stdout: # don't complain about insecure global_id in the test suite 2024-09-14T06:27:52.324 INFO:teuthology.orchestra.run.smithi094.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-09-14T06:27:52.324 INFO:teuthology.orchestra.run.smithi094.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-09-14T06:27:52.324 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-14T06:27:52.325 INFO:teuthology.orchestra.run.smithi094.stdout: # 1m isn't quite enough 2024-09-14T06:27:52.325 INFO:teuthology.orchestra.run.smithi094.stdout: mon_down_mkfs_grace = 2m 2024-09-14T06:27:52.325 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-14T06:27:52.325 INFO:teuthology.orchestra.run.smithi094.stdout: mon_warn_on_filestore_osds = false 2024-09-14T06:27:52.325 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-14T06:27:52.325 INFO:teuthology.orchestra.run.smithi094.stdout:[client] 2024-09-14T06:27:52.325 INFO:teuthology.orchestra.run.smithi094.stdout: rgw cache enabled = true 2024-09-14T06:27:52.325 INFO:teuthology.orchestra.run.smithi094.stdout: rgw enable ops log = true 2024-09-14T06:27:52.325 INFO:teuthology.orchestra.run.smithi094.stdout: rgw enable usage log = true 2024-09-14T06:27:52.325 INFO:teuthology.orchestra.run.smithi094.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-09-14T06:27:52.325 INFO:teuthology.orchestra.run.smithi094.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-09-14T06:27:52.325 INFO:teuthology.orchestra.run.smithi094.stdout:[mon.a] 2024-09-14T06:27:52.325 INFO:teuthology.orchestra.run.smithi094.stdout:[mon.c] 2024-09-14T06:27:52.325 INFO:teuthology.orchestra.run.smithi094.stdout:[mon.b] 2024-09-14T06:27:52.326 INFO:teuthology.orchestra.run.smithi138.stdout:[global] 2024-09-14T06:27:52.326 INFO:teuthology.orchestra.run.smithi138.stdout: chdir = "" 2024-09-14T06:27:52.326 INFO:teuthology.orchestra.run.smithi138.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-09-14T06:27:52.326 INFO:teuthology.orchestra.run.smithi138.stdout: auth supported = cephx 2024-09-14T06:27:52.326 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-09-14T06:27:52.326 INFO:teuthology.orchestra.run.smithi138.stdout: filestore xattr use omap = true 2024-09-14T06:27:52.326 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-09-14T06:27:52.326 INFO:teuthology.orchestra.run.smithi138.stdout: mon clock drift allowed = 1.000 2024-09-14T06:27:52.327 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-09-14T06:27:52.327 INFO:teuthology.orchestra.run.smithi138.stdout: osd crush chooseleaf type = 0 2024-09-14T06:27:52.327 INFO:teuthology.orchestra.run.smithi138.stdout: auth debug = true 2024-09-14T06:27:52.327 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-09-14T06:27:52.327 INFO:teuthology.orchestra.run.smithi138.stdout: ms die on old message = true 2024-09-14T06:27:52.327 INFO:teuthology.orchestra.run.smithi138.stdout: ms die on bug = true 2024-09-14T06:27:52.327 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-09-14T06:27:52.327 INFO:teuthology.orchestra.run.smithi138.stdout: mon max pg per osd = 10000 # >= luminous 2024-09-14T06:27:52.327 INFO:teuthology.orchestra.run.smithi138.stdout: mon pg warn max object skew = 0 2024-09-14T06:27:52.327 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-09-14T06:27:52.327 INFO:teuthology.orchestra.run.smithi138.stdout: # disable pg_autoscaler by default for new pools 2024-09-14T06:27:52.327 INFO:teuthology.orchestra.run.smithi138.stdout: osd_pool_default_pg_autoscale_mode = off 2024-09-14T06:27:52.327 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-09-14T06:27:52.327 INFO:teuthology.orchestra.run.smithi138.stdout: osd pool default size = 2 2024-09-14T06:27:52.327 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-09-14T06:27:52.328 INFO:teuthology.orchestra.run.smithi138.stdout: mon osd allow primary affinity = true 2024-09-14T06:27:52.328 INFO:teuthology.orchestra.run.smithi138.stdout: mon osd allow pg remap = true 2024-09-14T06:27:52.328 INFO:teuthology.orchestra.run.smithi138.stdout: mon warn on legacy crush tunables = false 2024-09-14T06:27:52.328 INFO:teuthology.orchestra.run.smithi138.stdout: mon warn on crush straw calc version zero = false 2024-09-14T06:27:52.328 INFO:teuthology.orchestra.run.smithi138.stdout: mon warn on no sortbitwise = false 2024-09-14T06:27:52.328 INFO:teuthology.orchestra.run.smithi138.stdout: mon warn on osd down out interval zero = false 2024-09-14T06:27:52.328 INFO:teuthology.orchestra.run.smithi138.stdout: mon warn on too few osds = false 2024-09-14T06:27:52.328 INFO:teuthology.orchestra.run.smithi138.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-09-14T06:27:52.328 INFO:teuthology.orchestra.run.smithi138.stdout: mon_warn_on_pool_no_redundancy = false 2024-09-14T06:27:52.328 INFO:teuthology.orchestra.run.smithi138.stdout: mon_allow_pool_size_one = true 2024-09-14T06:27:52.328 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-09-14T06:27:52.328 INFO:teuthology.orchestra.run.smithi138.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-09-14T06:27:52.328 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-09-14T06:27:52.328 INFO:teuthology.orchestra.run.smithi138.stdout: osd default data pool replay window = 5 2024-09-14T06:27:52.329 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-09-14T06:27:52.329 INFO:teuthology.orchestra.run.smithi138.stdout: mon allow pool delete = true 2024-09-14T06:27:52.329 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-09-14T06:27:52.329 INFO:teuthology.orchestra.run.smithi138.stdout: mon cluster log file level = debug 2024-09-14T06:27:52.329 INFO:teuthology.orchestra.run.smithi138.stdout: debug asserts on shutdown = true 2024-09-14T06:27:52.329 INFO:teuthology.orchestra.run.smithi138.stdout: mon health detail to clog = false 2024-09-14T06:27:52.329 INFO:teuthology.orchestra.run.smithi138.stdout: mon host = "172.21.15.94,[v2:172.21.15.94:3301,v1:172.21.15.94:6790],172.21.15.138" 2024-09-14T06:27:52.329 INFO:teuthology.orchestra.run.smithi138.stdout: mon client directed command retry = 5 2024-09-14T06:27:52.329 INFO:teuthology.orchestra.run.smithi138.stdout: mon warn on pool no app = False 2024-09-14T06:27:52.329 INFO:teuthology.orchestra.run.smithi138.stdout: ms inject socket failures = 5000 2024-09-14T06:27:52.329 INFO:teuthology.orchestra.run.smithi138.stdout: fsid = 200ad3da-bd63-4e04-af74-efb5ca0af75e 2024-09-14T06:27:52.329 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-09-14T06:27:52.329 INFO:teuthology.orchestra.run.smithi138.stdout:[osd] 2024-09-14T06:27:52.330 INFO:teuthology.orchestra.run.smithi138.stdout: osd journal size = 100 2024-09-14T06:27:52.330 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-09-14T06:27:52.330 INFO:teuthology.orchestra.run.smithi138.stdout: osd scrub load threshold = 5.0 2024-09-14T06:27:52.330 INFO:teuthology.orchestra.run.smithi138.stdout: osd scrub max interval = 600 2024-09-14T06:27:52.330 INFO:teuthology.orchestra.run.smithi138.stdout: osd mclock profile = high_recovery_ops 2024-09-14T06:27:52.330 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-09-14T06:27:52.330 INFO:teuthology.orchestra.run.smithi138.stdout: osd recover clone overlap = true 2024-09-14T06:27:52.330 INFO:teuthology.orchestra.run.smithi138.stdout: osd recovery max chunk = 1048576 2024-09-14T06:27:52.330 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-09-14T06:27:52.330 INFO:teuthology.orchestra.run.smithi138.stdout: osd debug shutdown = true 2024-09-14T06:27:52.330 INFO:teuthology.orchestra.run.smithi138.stdout: osd debug op order = true 2024-09-14T06:27:52.330 INFO:teuthology.orchestra.run.smithi138.stdout: osd debug verify stray on activate = true 2024-09-14T06:27:52.330 INFO:teuthology.orchestra.run.smithi138.stdout: osd debug trim objects = true 2024-09-14T06:27:52.330 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-09-14T06:27:52.330 INFO:teuthology.orchestra.run.smithi138.stdout: osd open classes on start = true 2024-09-14T06:27:52.331 INFO:teuthology.orchestra.run.smithi138.stdout: osd debug pg log writeout = true 2024-09-14T06:27:52.331 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-09-14T06:27:52.331 INFO:teuthology.orchestra.run.smithi138.stdout: osd deep scrub update digest min age = 30 2024-09-14T06:27:52.331 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-09-14T06:27:52.331 INFO:teuthology.orchestra.run.smithi138.stdout: osd map max advance = 10 2024-09-14T06:27:52.331 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-09-14T06:27:52.331 INFO:teuthology.orchestra.run.smithi138.stdout: journal zero on create = true 2024-09-14T06:27:52.331 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-09-14T06:27:52.331 INFO:teuthology.orchestra.run.smithi138.stdout: filestore ondisk finisher threads = 3 2024-09-14T06:27:52.331 INFO:teuthology.orchestra.run.smithi138.stdout: filestore apply finisher threads = 3 2024-09-14T06:27:52.331 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-09-14T06:27:52.331 INFO:teuthology.orchestra.run.smithi138.stdout: bdev debug aio = true 2024-09-14T06:27:52.331 INFO:teuthology.orchestra.run.smithi138.stdout: osd debug misdirected ops = true 2024-09-14T06:27:52.331 INFO:teuthology.orchestra.run.smithi138.stdout: bluestore block size = 96636764160 2024-09-14T06:27:52.332 INFO:teuthology.orchestra.run.smithi138.stdout: bluestore compression algorithm = snappy 2024-09-14T06:27:52.332 INFO:teuthology.orchestra.run.smithi138.stdout: bluestore compression mode = aggressive 2024-09-14T06:27:52.332 INFO:teuthology.orchestra.run.smithi138.stdout: bluestore fsck on mount = True 2024-09-14T06:27:52.332 INFO:teuthology.orchestra.run.smithi138.stdout: debug bluefs = 1/20 2024-09-14T06:27:52.332 INFO:teuthology.orchestra.run.smithi138.stdout: debug bluestore = 1/20 2024-09-14T06:27:52.332 INFO:teuthology.orchestra.run.smithi138.stdout: debug ms = 1 2024-09-14T06:27:52.332 INFO:teuthology.orchestra.run.smithi138.stdout: debug osd = 20 2024-09-14T06:27:52.332 INFO:teuthology.orchestra.run.smithi138.stdout: debug rocksdb = 4/10 2024-09-14T06:27:52.332 INFO:teuthology.orchestra.run.smithi138.stdout: mon osd backfillfull_ratio = 0.85 2024-09-14T06:27:52.332 INFO:teuthology.orchestra.run.smithi138.stdout: mon osd full ratio = 0.9 2024-09-14T06:27:52.332 INFO:teuthology.orchestra.run.smithi138.stdout: mon osd nearfull ratio = 0.8 2024-09-14T06:27:52.332 INFO:teuthology.orchestra.run.smithi138.stdout: osd blocked scrub grace period = 3600 2024-09-14T06:27:52.333 INFO:teuthology.orchestra.run.smithi138.stdout: osd failsafe full ratio = 0.95 2024-09-14T06:27:52.333 INFO:teuthology.orchestra.run.smithi138.stdout: osd max markdown count = 1000 2024-09-14T06:27:52.333 INFO:teuthology.orchestra.run.smithi138.stdout: osd objectstore = bluestore 2024-09-14T06:27:52.333 INFO:teuthology.orchestra.run.smithi138.stdout: osd shutdown pgref assert = True 2024-09-14T06:27:52.333 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-09-14T06:27:52.333 INFO:teuthology.orchestra.run.smithi138.stdout:[mgr] 2024-09-14T06:27:52.333 INFO:teuthology.orchestra.run.smithi138.stdout: debug ms = 1 2024-09-14T06:27:52.333 INFO:teuthology.orchestra.run.smithi138.stdout: debug mgr = 20 2024-09-14T06:27:52.333 INFO:teuthology.orchestra.run.smithi138.stdout: debug mon = 20 2024-09-14T06:27:52.333 INFO:teuthology.orchestra.run.smithi138.stdout: debug auth = 20 2024-09-14T06:27:52.333 INFO:teuthology.orchestra.run.smithi138.stdout: mon reweight min pgs per osd = 4 2024-09-14T06:27:52.333 INFO:teuthology.orchestra.run.smithi138.stdout: mon reweight min bytes per osd = 10 2024-09-14T06:27:52.333 INFO:teuthology.orchestra.run.smithi138.stdout: mgr/telemetry/nag = false 2024-09-14T06:27:52.333 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-09-14T06:27:52.333 INFO:teuthology.orchestra.run.smithi138.stdout:[mon] 2024-09-14T06:27:52.334 INFO:teuthology.orchestra.run.smithi138.stdout: debug ms = 1 2024-09-14T06:27:52.334 INFO:teuthology.orchestra.run.smithi138.stdout: debug mon = 20 2024-09-14T06:27:52.334 INFO:teuthology.orchestra.run.smithi138.stdout: debug paxos = 20 2024-09-14T06:27:52.334 INFO:teuthology.orchestra.run.smithi138.stdout: debug auth = 20 2024-09-14T06:27:52.334 INFO:teuthology.orchestra.run.smithi138.stdout: mon data avail warn = 5 2024-09-14T06:27:52.334 INFO:teuthology.orchestra.run.smithi138.stdout: mon mgr mkfs grace = 240 2024-09-14T06:27:52.334 INFO:teuthology.orchestra.run.smithi138.stdout: mon reweight min pgs per osd = 4 2024-09-14T06:27:52.334 INFO:teuthology.orchestra.run.smithi138.stdout: mon osd reporter subtree level = osd 2024-09-14T06:27:52.334 INFO:teuthology.orchestra.run.smithi138.stdout: mon osd prime pg temp = true 2024-09-14T06:27:52.334 INFO:teuthology.orchestra.run.smithi138.stdout: mon reweight min bytes per osd = 10 2024-09-14T06:27:52.334 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-09-14T06:27:52.334 INFO:teuthology.orchestra.run.smithi138.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-09-14T06:27:52.334 INFO:teuthology.orchestra.run.smithi138.stdout: auth mon ticket ttl = 660 # 11m 2024-09-14T06:27:52.334 INFO:teuthology.orchestra.run.smithi138.stdout: auth service ticket ttl = 240 # 4m 2024-09-14T06:27:52.334 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-09-14T06:27:52.335 INFO:teuthology.orchestra.run.smithi138.stdout: # don't complain about insecure global_id in the test suite 2024-09-14T06:27:52.335 INFO:teuthology.orchestra.run.smithi138.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-09-14T06:27:52.335 INFO:teuthology.orchestra.run.smithi138.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-09-14T06:27:52.335 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-09-14T06:27:52.335 INFO:teuthology.orchestra.run.smithi138.stdout: # 1m isn't quite enough 2024-09-14T06:27:52.335 INFO:teuthology.orchestra.run.smithi138.stdout: mon_down_mkfs_grace = 2m 2024-09-14T06:27:52.335 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-09-14T06:27:52.335 INFO:teuthology.orchestra.run.smithi138.stdout: mon_warn_on_filestore_osds = false 2024-09-14T06:27:52.335 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-09-14T06:27:52.335 INFO:teuthology.orchestra.run.smithi138.stdout:[client] 2024-09-14T06:27:52.335 INFO:teuthology.orchestra.run.smithi138.stdout: rgw cache enabled = true 2024-09-14T06:27:52.335 INFO:teuthology.orchestra.run.smithi138.stdout: rgw enable ops log = true 2024-09-14T06:27:52.335 INFO:teuthology.orchestra.run.smithi138.stdout: rgw enable usage log = true 2024-09-14T06:27:52.335 INFO:teuthology.orchestra.run.smithi138.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-09-14T06:27:52.336 INFO:teuthology.orchestra.run.smithi138.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-09-14T06:27:52.336 INFO:teuthology.orchestra.run.smithi138.stdout:[mon.a] 2024-09-14T06:27:52.336 INFO:teuthology.orchestra.run.smithi138.stdout:[mon.c] 2024-09-14T06:27:52.336 INFO:teuthology.orchestra.run.smithi138.stdout:[mon.b] 2024-09-14T06:27:52.337 INFO:tasks.ceph:Creating admin key on mon.a... 2024-09-14T06:27:52.337 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=client.admin --cap mon 'allow *' --cap osd 'allow *' --cap mds 'allow *' --cap mgr 'allow *' /etc/ceph/ceph.keyring 2024-09-14T06:27:52.431 INFO:tasks.ceph:Copying monmap to all nodes... 2024-09-14T06:27:52.431 DEBUG:teuthology.orchestra.run.smithi094:> set -ex 2024-09-14T06:27:52.431 DEBUG:teuthology.orchestra.run.smithi094:> dd if=/etc/ceph/ceph.keyring of=/dev/stdout 2024-09-14T06:27:52.446 DEBUG:teuthology.orchestra.run.smithi094:> set -ex 2024-09-14T06:27:52.447 DEBUG:teuthology.orchestra.run.smithi094:> dd if=/home/ubuntu/cephtest/ceph.monmap of=/dev/stdout 2024-09-14T06:27:52.502 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi094.front.sepia.ceph.com 2024-09-14T06:27:52.502 DEBUG:teuthology.orchestra.run.smithi094:> set -ex 2024-09-14T06:27:52.502 DEBUG:teuthology.orchestra.run.smithi094:> sudo dd of=/etc/ceph/ceph.keyring 2024-09-14T06:27:52.502 DEBUG:teuthology.orchestra.run.smithi094:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-09-14T06:27:52.576 DEBUG:teuthology.orchestra.run.smithi094:> set -ex 2024-09-14T06:27:52.576 DEBUG:teuthology.orchestra.run.smithi094:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-09-14T06:27:52.632 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi138.front.sepia.ceph.com 2024-09-14T06:27:52.632 DEBUG:teuthology.orchestra.run.smithi138:> set -ex 2024-09-14T06:27:52.632 DEBUG:teuthology.orchestra.run.smithi138:> sudo dd of=/etc/ceph/ceph.keyring 2024-09-14T06:27:52.632 DEBUG:teuthology.orchestra.run.smithi138:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-09-14T06:27:52.667 DEBUG:teuthology.orchestra.run.smithi138:> set -ex 2024-09-14T06:27:52.667 DEBUG:teuthology.orchestra.run.smithi138:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-09-14T06:27:52.722 INFO:tasks.ceph:Setting up mon nodes... 2024-09-14T06:27:52.722 INFO:tasks.ceph:Setting up mgr nodes... 2024-09-14T06:27:52.722 DEBUG:teuthology.orchestra.run.smithi094:> sudo mkdir -p /var/lib/ceph/mgr/ceph-y && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.y /var/lib/ceph/mgr/ceph-y/keyring 2024-09-14T06:27:52.773 INFO:teuthology.orchestra.run.smithi094.stdout:creating /var/lib/ceph/mgr/ceph-y/keyring 2024-09-14T06:27:52.776 DEBUG:teuthology.orchestra.run.smithi138:> sudo mkdir -p /var/lib/ceph/mgr/ceph-x && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.x /var/lib/ceph/mgr/ceph-x/keyring 2024-09-14T06:27:52.844 INFO:teuthology.orchestra.run.smithi138.stdout:creating /var/lib/ceph/mgr/ceph-x/keyring 2024-09-14T06:27:52.847 INFO:tasks.ceph:Setting up mds nodes... 2024-09-14T06:27:52.847 INFO:tasks.ceph_client:Setting up client nodes... 2024-09-14T06:27:52.847 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.0 /etc/ceph/ceph.client.0.keyring && sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-09-14T06:27:52.888 INFO:teuthology.orchestra.run.smithi094.stdout:creating /etc/ceph/ceph.client.0.keyring 2024-09-14T06:27:52.901 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.1 /etc/ceph/ceph.client.1.keyring && sudo chmod 0644 /etc/ceph/ceph.client.1.keyring 2024-09-14T06:27:52.942 INFO:teuthology.orchestra.run.smithi138.stdout:creating /etc/ceph/ceph.client.1.keyring 2024-09-14T06:27:52.954 INFO:tasks.ceph:Running mkfs on osd nodes... 2024-09-14T06:27:52.955 INFO:tasks.ceph:ctx.disk_config.remote_to_roles_to_dev: {Remote(name='ubuntu@smithi094.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi138.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2024-09-14T06:27:52.955 DEBUG:teuthology.orchestra.run.smithi094:> sudo mkdir -p /var/lib/ceph/osd/ceph-0 2024-09-14T06:27:52.979 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-09-14T06:27:52.979 INFO:tasks.ceph:role: osd.0 2024-09-14T06:27:52.979 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi094.front.sepia.ceph.com 2024-09-14T06:27:52.979 DEBUG:teuthology.orchestra.run.smithi094:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-09-14T06:27:53.043 INFO:teuthology.orchestra.run.smithi094.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-09-14T06:27:53.043 INFO:teuthology.orchestra.run.smithi094.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-14T06:27:53.043 INFO:teuthology.orchestra.run.smithi094.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-14T06:27:53.044 INFO:teuthology.orchestra.run.smithi094.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-09-14T06:27:53.044 INFO:teuthology.orchestra.run.smithi094.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-14T06:27:53.044 INFO:teuthology.orchestra.run.smithi094.stdout: = sunit=0 swidth=0 blks 2024-09-14T06:27:53.044 INFO:teuthology.orchestra.run.smithi094.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-14T06:27:53.044 INFO:teuthology.orchestra.run.smithi094.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-09-14T06:27:53.044 INFO:teuthology.orchestra.run.smithi094.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-14T06:27:53.044 INFO:teuthology.orchestra.run.smithi094.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-14T06:27:53.082 INFO:teuthology.orchestra.run.smithi094.stdout:Discarding blocks...Done. 2024-09-14T06:27:53.083 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi094.front.sepia.ceph.com -o noatime 2024-09-14T06:27:53.083 DEBUG:teuthology.orchestra.run.smithi094:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-0 2024-09-14T06:27:53.158 DEBUG:teuthology.orchestra.run.smithi094:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-0 2024-09-14T06:27:53.227 DEBUG:teuthology.orchestra.run.smithi094:> sudo mkdir -p /var/lib/ceph/osd/ceph-1 2024-09-14T06:27:53.291 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-09-14T06:27:53.292 INFO:tasks.ceph:role: osd.1 2024-09-14T06:27:53.292 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi094.front.sepia.ceph.com 2024-09-14T06:27:53.292 DEBUG:teuthology.orchestra.run.smithi094:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-09-14T06:27:53.356 INFO:teuthology.orchestra.run.smithi094.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-09-14T06:27:53.356 INFO:teuthology.orchestra.run.smithi094.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-14T06:27:53.356 INFO:teuthology.orchestra.run.smithi094.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-14T06:27:53.356 INFO:teuthology.orchestra.run.smithi094.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-09-14T06:27:53.356 INFO:teuthology.orchestra.run.smithi094.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-14T06:27:53.356 INFO:teuthology.orchestra.run.smithi094.stdout: = sunit=0 swidth=0 blks 2024-09-14T06:27:53.356 INFO:teuthology.orchestra.run.smithi094.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-14T06:27:53.356 INFO:teuthology.orchestra.run.smithi094.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-09-14T06:27:53.356 INFO:teuthology.orchestra.run.smithi094.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-14T06:27:53.356 INFO:teuthology.orchestra.run.smithi094.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-14T06:27:53.397 INFO:teuthology.orchestra.run.smithi094.stdout:Discarding blocks...Done. 2024-09-14T06:27:53.399 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi094.front.sepia.ceph.com -o noatime 2024-09-14T06:27:53.399 DEBUG:teuthology.orchestra.run.smithi094:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-1 2024-09-14T06:27:53.469 DEBUG:teuthology.orchestra.run.smithi094:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-1 2024-09-14T06:27:53.538 DEBUG:teuthology.orchestra.run.smithi094:> sudo mkdir -p /var/lib/ceph/osd/ceph-2 2024-09-14T06:27:53.603 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-09-14T06:27:53.603 INFO:tasks.ceph:role: osd.2 2024-09-14T06:27:53.603 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi094.front.sepia.ceph.com 2024-09-14T06:27:53.603 DEBUG:teuthology.orchestra.run.smithi094:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-09-14T06:27:53.667 INFO:teuthology.orchestra.run.smithi094.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-09-14T06:27:53.667 INFO:teuthology.orchestra.run.smithi094.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-14T06:27:53.667 INFO:teuthology.orchestra.run.smithi094.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-14T06:27:53.667 INFO:teuthology.orchestra.run.smithi094.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-09-14T06:27:53.667 INFO:teuthology.orchestra.run.smithi094.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-14T06:27:53.667 INFO:teuthology.orchestra.run.smithi094.stdout: = sunit=0 swidth=0 blks 2024-09-14T06:27:53.668 INFO:teuthology.orchestra.run.smithi094.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-14T06:27:53.668 INFO:teuthology.orchestra.run.smithi094.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-09-14T06:27:53.668 INFO:teuthology.orchestra.run.smithi094.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-14T06:27:53.668 INFO:teuthology.orchestra.run.smithi094.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-14T06:27:53.709 INFO:teuthology.orchestra.run.smithi094.stdout:Discarding blocks...Done. 2024-09-14T06:27:53.711 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi094.front.sepia.ceph.com -o noatime 2024-09-14T06:27:53.712 DEBUG:teuthology.orchestra.run.smithi094:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-2 2024-09-14T06:27:53.781 DEBUG:teuthology.orchestra.run.smithi094:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-2 2024-09-14T06:27:53.851 DEBUG:teuthology.orchestra.run.smithi094:> sudo mkdir -p /var/lib/ceph/osd/ceph-3 2024-09-14T06:27:53.915 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-09-14T06:27:53.916 INFO:tasks.ceph:role: osd.3 2024-09-14T06:27:53.916 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi094.front.sepia.ceph.com 2024-09-14T06:27:53.916 DEBUG:teuthology.orchestra.run.smithi094:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2024-09-14T06:27:53.980 INFO:teuthology.orchestra.run.smithi094.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2024-09-14T06:27:53.981 INFO:teuthology.orchestra.run.smithi094.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-14T06:27:53.981 INFO:teuthology.orchestra.run.smithi094.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-14T06:27:53.981 INFO:teuthology.orchestra.run.smithi094.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-09-14T06:27:53.981 INFO:teuthology.orchestra.run.smithi094.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-14T06:27:53.981 INFO:teuthology.orchestra.run.smithi094.stdout: = sunit=0 swidth=0 blks 2024-09-14T06:27:53.981 INFO:teuthology.orchestra.run.smithi094.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-14T06:27:53.981 INFO:teuthology.orchestra.run.smithi094.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-09-14T06:27:53.981 INFO:teuthology.orchestra.run.smithi094.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-14T06:27:53.981 INFO:teuthology.orchestra.run.smithi094.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-14T06:27:54.023 INFO:teuthology.orchestra.run.smithi094.stdout:Discarding blocks...Done. 2024-09-14T06:27:54.025 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi094.front.sepia.ceph.com -o noatime 2024-09-14T06:27:54.025 DEBUG:teuthology.orchestra.run.smithi094:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-3 2024-09-14T06:27:54.094 DEBUG:teuthology.orchestra.run.smithi094:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-3 2024-09-14T06:27:54.166 DEBUG:teuthology.orchestra.run.smithi094:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 0 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-14T06:27:54.280 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-14T06:27:54.278+0000 7f3fde81c640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-0/keyring: can't open /var/lib/ceph/osd/ceph-0/keyring: (2) No such file or directory 2024-09-14T06:27:54.280 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-14T06:27:54.279+0000 7f3fde81c640 -1 created new key in keyring /var/lib/ceph/osd/ceph-0/keyring 2024-09-14T06:27:54.280 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-14T06:27:54.279+0000 7f3fde81c640 -1 bdev(0x558c4ab41800 /var/lib/ceph/osd/ceph-0/block) open stat got: (1) Operation not permitted 2024-09-14T06:27:54.280 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-14T06:27:54.279+0000 7f3fde81c640 -1 bluestore(/var/lib/ceph/osd/ceph-0) _read_fsid unparsable uuid 2024-09-14T06:27:57.432 DEBUG:teuthology.orchestra.run.smithi094:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2024-09-14T06:27:57.462 DEBUG:teuthology.orchestra.run.smithi094:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 1 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-14T06:27:57.553 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-14T06:27:57.551+0000 7f21ed80d640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-1/keyring: can't open /var/lib/ceph/osd/ceph-1/keyring: (2) No such file or directory 2024-09-14T06:27:57.553 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-14T06:27:57.552+0000 7f21ed80d640 -1 created new key in keyring /var/lib/ceph/osd/ceph-1/keyring 2024-09-14T06:27:57.553 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-14T06:27:57.552+0000 7f21ed80d640 -1 bdev(0x5593cff59800 /var/lib/ceph/osd/ceph-1/block) open stat got: (1) Operation not permitted 2024-09-14T06:27:57.554 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-14T06:27:57.552+0000 7f21ed80d640 -1 bluestore(/var/lib/ceph/osd/ceph-1) _read_fsid unparsable uuid 2024-09-14T06:28:00.462 DEBUG:teuthology.orchestra.run.smithi094:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2024-09-14T06:28:00.488 DEBUG:teuthology.orchestra.run.smithi094:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 2 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-14T06:28:00.576 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-14T06:28:00.574+0000 7f5f43f02640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-2/keyring: can't open /var/lib/ceph/osd/ceph-2/keyring: (2) No such file or directory 2024-09-14T06:28:00.576 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-14T06:28:00.574+0000 7f5f43f02640 -1 created new key in keyring /var/lib/ceph/osd/ceph-2/keyring 2024-09-14T06:28:00.576 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-14T06:28:00.574+0000 7f5f43f02640 -1 bdev(0x55e941d73800 /var/lib/ceph/osd/ceph-2/block) open stat got: (1) Operation not permitted 2024-09-14T06:28:00.576 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-14T06:28:00.574+0000 7f5f43f02640 -1 bluestore(/var/lib/ceph/osd/ceph-2) _read_fsid unparsable uuid 2024-09-14T06:28:03.510 DEBUG:teuthology.orchestra.run.smithi094:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2024-09-14T06:28:03.536 DEBUG:teuthology.orchestra.run.smithi094:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 3 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-14T06:28:03.621 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-14T06:28:03.620+0000 7f4a961db640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-3/keyring: can't open /var/lib/ceph/osd/ceph-3/keyring: (2) No such file or directory 2024-09-14T06:28:03.622 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-14T06:28:03.620+0000 7f4a961db640 -1 created new key in keyring /var/lib/ceph/osd/ceph-3/keyring 2024-09-14T06:28:03.622 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-14T06:28:03.620+0000 7f4a961db640 -1 bdev(0x55f790433800 /var/lib/ceph/osd/ceph-3/block) open stat got: (1) Operation not permitted 2024-09-14T06:28:03.622 INFO:teuthology.orchestra.run.smithi094.stderr:2024-09-14T06:28:03.620+0000 7f4a961db640 -1 bluestore(/var/lib/ceph/osd/ceph-3) _read_fsid unparsable uuid 2024-09-14T06:28:06.774 DEBUG:teuthology.orchestra.run.smithi094:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2024-09-14T06:28:06.804 DEBUG:teuthology.orchestra.run.smithi138:> sudo mkdir -p /var/lib/ceph/osd/ceph-4 2024-09-14T06:28:06.834 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-09-14T06:28:06.834 INFO:tasks.ceph:role: osd.4 2024-09-14T06:28:06.834 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi138.front.sepia.ceph.com 2024-09-14T06:28:06.834 DEBUG:teuthology.orchestra.run.smithi138:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-09-14T06:28:06.910 INFO:teuthology.orchestra.run.smithi138.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-09-14T06:28:06.910 INFO:teuthology.orchestra.run.smithi138.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-14T06:28:06.910 INFO:teuthology.orchestra.run.smithi138.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-14T06:28:06.910 INFO:teuthology.orchestra.run.smithi138.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-09-14T06:28:06.910 INFO:teuthology.orchestra.run.smithi138.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-14T06:28:06.910 INFO:teuthology.orchestra.run.smithi138.stdout: = sunit=0 swidth=0 blks 2024-09-14T06:28:06.910 INFO:teuthology.orchestra.run.smithi138.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-14T06:28:06.911 INFO:teuthology.orchestra.run.smithi138.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-09-14T06:28:06.911 INFO:teuthology.orchestra.run.smithi138.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-14T06:28:06.911 INFO:teuthology.orchestra.run.smithi138.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-14T06:28:07.407 INFO:teuthology.orchestra.run.smithi138.stdout:Discarding blocks...Done. 2024-09-14T06:28:07.409 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi138.front.sepia.ceph.com -o noatime 2024-09-14T06:28:07.409 DEBUG:teuthology.orchestra.run.smithi138:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-4 2024-09-14T06:28:07.533 DEBUG:teuthology.orchestra.run.smithi138:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-4 2024-09-14T06:28:07.563 DEBUG:teuthology.orchestra.run.smithi138:> sudo mkdir -p /var/lib/ceph/osd/ceph-5 2024-09-14T06:28:07.629 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-09-14T06:28:07.630 INFO:tasks.ceph:role: osd.5 2024-09-14T06:28:07.630 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi138.front.sepia.ceph.com 2024-09-14T06:28:07.630 DEBUG:teuthology.orchestra.run.smithi138:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-09-14T06:28:07.703 INFO:teuthology.orchestra.run.smithi138.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-09-14T06:28:07.703 INFO:teuthology.orchestra.run.smithi138.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-14T06:28:07.703 INFO:teuthology.orchestra.run.smithi138.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-14T06:28:07.703 INFO:teuthology.orchestra.run.smithi138.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-09-14T06:28:07.704 INFO:teuthology.orchestra.run.smithi138.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-14T06:28:07.704 INFO:teuthology.orchestra.run.smithi138.stdout: = sunit=0 swidth=0 blks 2024-09-14T06:28:07.704 INFO:teuthology.orchestra.run.smithi138.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-14T06:28:07.704 INFO:teuthology.orchestra.run.smithi138.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-09-14T06:28:07.704 INFO:teuthology.orchestra.run.smithi138.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-14T06:28:07.704 INFO:teuthology.orchestra.run.smithi138.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-14T06:28:08.158 INFO:teuthology.orchestra.run.smithi138.stdout:Discarding blocks...Done. 2024-09-14T06:28:08.160 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi138.front.sepia.ceph.com -o noatime 2024-09-14T06:28:08.160 DEBUG:teuthology.orchestra.run.smithi138:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-5 2024-09-14T06:28:08.242 DEBUG:teuthology.orchestra.run.smithi138:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-5 2024-09-14T06:28:08.311 DEBUG:teuthology.orchestra.run.smithi138:> sudo mkdir -p /var/lib/ceph/osd/ceph-6 2024-09-14T06:28:08.375 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-09-14T06:28:08.376 INFO:tasks.ceph:role: osd.6 2024-09-14T06:28:08.376 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi138.front.sepia.ceph.com 2024-09-14T06:28:08.376 DEBUG:teuthology.orchestra.run.smithi138:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-09-14T06:28:08.449 INFO:teuthology.orchestra.run.smithi138.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-09-14T06:28:08.449 INFO:teuthology.orchestra.run.smithi138.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-14T06:28:08.449 INFO:teuthology.orchestra.run.smithi138.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-14T06:28:08.449 INFO:teuthology.orchestra.run.smithi138.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-09-14T06:28:08.449 INFO:teuthology.orchestra.run.smithi138.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-14T06:28:08.450 INFO:teuthology.orchestra.run.smithi138.stdout: = sunit=0 swidth=0 blks 2024-09-14T06:28:08.450 INFO:teuthology.orchestra.run.smithi138.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-14T06:28:08.450 INFO:teuthology.orchestra.run.smithi138.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-09-14T06:28:08.450 INFO:teuthology.orchestra.run.smithi138.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-14T06:28:08.450 INFO:teuthology.orchestra.run.smithi138.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-14T06:28:08.962 INFO:teuthology.orchestra.run.smithi138.stdout:Discarding blocks...Done. 2024-09-14T06:28:08.964 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi138.front.sepia.ceph.com -o noatime 2024-09-14T06:28:08.964 DEBUG:teuthology.orchestra.run.smithi138:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-6 2024-09-14T06:28:09.106 DEBUG:teuthology.orchestra.run.smithi138:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-6 2024-09-14T06:28:09.137 DEBUG:teuthology.orchestra.run.smithi138:> sudo mkdir -p /var/lib/ceph/osd/ceph-7 2024-09-14T06:28:09.203 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-09-14T06:28:09.203 INFO:tasks.ceph:role: osd.7 2024-09-14T06:28:09.203 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi138.front.sepia.ceph.com 2024-09-14T06:28:09.203 DEBUG:teuthology.orchestra.run.smithi138:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2024-09-14T06:28:09.276 INFO:teuthology.orchestra.run.smithi138.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2024-09-14T06:28:09.276 INFO:teuthology.orchestra.run.smithi138.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-14T06:28:09.276 INFO:teuthology.orchestra.run.smithi138.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-14T06:28:09.276 INFO:teuthology.orchestra.run.smithi138.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-09-14T06:28:09.276 INFO:teuthology.orchestra.run.smithi138.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-14T06:28:09.276 INFO:teuthology.orchestra.run.smithi138.stdout: = sunit=0 swidth=0 blks 2024-09-14T06:28:09.276 INFO:teuthology.orchestra.run.smithi138.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-14T06:28:09.277 INFO:teuthology.orchestra.run.smithi138.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-09-14T06:28:09.277 INFO:teuthology.orchestra.run.smithi138.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-14T06:28:09.277 INFO:teuthology.orchestra.run.smithi138.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-14T06:28:09.788 INFO:teuthology.orchestra.run.smithi138.stdout:Discarding blocks...Done. 2024-09-14T06:28:09.791 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi138.front.sepia.ceph.com -o noatime 2024-09-14T06:28:09.791 DEBUG:teuthology.orchestra.run.smithi138:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-7 2024-09-14T06:28:09.884 DEBUG:teuthology.orchestra.run.smithi138:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-7 2024-09-14T06:28:09.916 DEBUG:teuthology.orchestra.run.smithi138:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 4 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-14T06:28:10.047 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-14T06:28:10.046+0000 7f3635dd4640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-4/keyring: can't open /var/lib/ceph/osd/ceph-4/keyring: (2) No such file or directory 2024-09-14T06:28:10.047 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-14T06:28:10.046+0000 7f3635dd4640 -1 created new key in keyring /var/lib/ceph/osd/ceph-4/keyring 2024-09-14T06:28:10.047 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-14T06:28:10.046+0000 7f3635dd4640 -1 bdev(0x55eecd5cb800 /var/lib/ceph/osd/ceph-4/block) open stat got: (1) Operation not permitted 2024-09-14T06:28:10.048 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-14T06:28:10.046+0000 7f3635dd4640 -1 bluestore(/var/lib/ceph/osd/ceph-4) _read_fsid unparsable uuid 2024-09-14T06:28:13.701 DEBUG:teuthology.orchestra.run.smithi138:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2024-09-14T06:28:13.731 DEBUG:teuthology.orchestra.run.smithi138:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 5 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-14T06:28:13.821 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-14T06:28:13.818+0000 7fcc70e32640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-5/keyring: can't open /var/lib/ceph/osd/ceph-5/keyring: (2) No such file or directory 2024-09-14T06:28:13.821 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-14T06:28:13.819+0000 7fcc70e32640 -1 created new key in keyring /var/lib/ceph/osd/ceph-5/keyring 2024-09-14T06:28:13.821 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-14T06:28:13.819+0000 7fcc70e32640 -1 bdev(0x559b9b40d800 /var/lib/ceph/osd/ceph-5/block) open stat got: (1) Operation not permitted 2024-09-14T06:28:13.821 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-14T06:28:13.819+0000 7fcc70e32640 -1 bluestore(/var/lib/ceph/osd/ceph-5) _read_fsid unparsable uuid 2024-09-14T06:28:16.727 DEBUG:teuthology.orchestra.run.smithi138:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2024-09-14T06:28:16.757 DEBUG:teuthology.orchestra.run.smithi138:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 6 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-14T06:28:16.851 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-14T06:28:16.849+0000 7f24b7e1d640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-6/keyring: can't open /var/lib/ceph/osd/ceph-6/keyring: (2) No such file or directory 2024-09-14T06:28:16.852 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-14T06:28:16.849+0000 7f24b7e1d640 -1 created new key in keyring /var/lib/ceph/osd/ceph-6/keyring 2024-09-14T06:28:16.852 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-14T06:28:16.849+0000 7f24b7e1d640 -1 bdev(0x5642e0869800 /var/lib/ceph/osd/ceph-6/block) open stat got: (1) Operation not permitted 2024-09-14T06:28:16.852 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-14T06:28:16.849+0000 7f24b7e1d640 -1 bluestore(/var/lib/ceph/osd/ceph-6) _read_fsid unparsable uuid 2024-09-14T06:28:20.013 DEBUG:teuthology.orchestra.run.smithi138:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-6 2024-09-14T06:28:20.041 DEBUG:teuthology.orchestra.run.smithi138:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 7 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-14T06:28:20.130 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-14T06:28:20.128+0000 7f3f70415640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-7/keyring: can't open /var/lib/ceph/osd/ceph-7/keyring: (2) No such file or directory 2024-09-14T06:28:20.130 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-14T06:28:20.128+0000 7f3f70415640 -1 created new key in keyring /var/lib/ceph/osd/ceph-7/keyring 2024-09-14T06:28:20.130 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-14T06:28:20.129+0000 7f3f70415640 -1 bdev(0x5599e2db3800 /var/lib/ceph/osd/ceph-7/block) open stat got: (1) Operation not permitted 2024-09-14T06:28:20.130 INFO:teuthology.orchestra.run.smithi138.stderr:2024-09-14T06:28:20.129+0000 7f3f70415640 -1 bluestore(/var/lib/ceph/osd/ceph-7) _read_fsid unparsable uuid 2024-09-14T06:28:23.523 DEBUG:teuthology.orchestra.run.smithi138:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-7 2024-09-14T06:28:23.550 INFO:tasks.ceph:Reading keys from all nodes... 2024-09-14T06:28:23.550 DEBUG:teuthology.orchestra.run.smithi094:> set -ex 2024-09-14T06:28:23.550 DEBUG:teuthology.orchestra.run.smithi094:> sudo dd if=/var/lib/ceph/mgr/ceph-y/keyring of=/dev/stdout 2024-09-14T06:28:23.580 DEBUG:teuthology.orchestra.run.smithi094:> set -ex 2024-09-14T06:28:23.580 DEBUG:teuthology.orchestra.run.smithi094:> sudo dd if=/var/lib/ceph/osd/ceph-0/keyring of=/dev/stdout 2024-09-14T06:28:23.650 DEBUG:teuthology.orchestra.run.smithi094:> set -ex 2024-09-14T06:28:23.650 DEBUG:teuthology.orchestra.run.smithi094:> sudo dd if=/var/lib/ceph/osd/ceph-1/keyring of=/dev/stdout 2024-09-14T06:28:23.718 DEBUG:teuthology.orchestra.run.smithi094:> set -ex 2024-09-14T06:28:23.718 DEBUG:teuthology.orchestra.run.smithi094:> sudo dd if=/var/lib/ceph/osd/ceph-2/keyring of=/dev/stdout 2024-09-14T06:28:23.785 DEBUG:teuthology.orchestra.run.smithi094:> set -ex 2024-09-14T06:28:23.785 DEBUG:teuthology.orchestra.run.smithi094:> sudo dd if=/var/lib/ceph/osd/ceph-3/keyring of=/dev/stdout 2024-09-14T06:28:23.851 DEBUG:teuthology.orchestra.run.smithi138:> set -ex 2024-09-14T06:28:23.851 DEBUG:teuthology.orchestra.run.smithi138:> sudo dd if=/var/lib/ceph/mgr/ceph-x/keyring of=/dev/stdout 2024-09-14T06:28:23.879 DEBUG:teuthology.orchestra.run.smithi138:> set -ex 2024-09-14T06:28:23.879 DEBUG:teuthology.orchestra.run.smithi138:> sudo dd if=/var/lib/ceph/osd/ceph-4/keyring of=/dev/stdout 2024-09-14T06:28:23.946 DEBUG:teuthology.orchestra.run.smithi138:> set -ex 2024-09-14T06:28:23.947 DEBUG:teuthology.orchestra.run.smithi138:> sudo dd if=/var/lib/ceph/osd/ceph-5/keyring of=/dev/stdout 2024-09-14T06:28:24.013 DEBUG:teuthology.orchestra.run.smithi138:> set -ex 2024-09-14T06:28:24.013 DEBUG:teuthology.orchestra.run.smithi138:> sudo dd if=/var/lib/ceph/osd/ceph-6/keyring of=/dev/stdout 2024-09-14T06:28:24.079 DEBUG:teuthology.orchestra.run.smithi138:> set -ex 2024-09-14T06:28:24.079 DEBUG:teuthology.orchestra.run.smithi138:> sudo dd if=/var/lib/ceph/osd/ceph-7/keyring of=/dev/stdout 2024-09-14T06:28:24.144 DEBUG:teuthology.orchestra.run.smithi094:> set -ex 2024-09-14T06:28:24.144 DEBUG:teuthology.orchestra.run.smithi094:> dd if=/etc/ceph/ceph.client.0.keyring of=/dev/stdout 2024-09-14T06:28:24.162 DEBUG:teuthology.orchestra.run.smithi138:> set -ex 2024-09-14T06:28:24.162 DEBUG:teuthology.orchestra.run.smithi138:> dd if=/etc/ceph/ceph.client.1.keyring of=/dev/stdout 2024-09-14T06:28:24.200 INFO:tasks.ceph:Adding keys to all mons... 2024-09-14T06:28:24.200 DEBUG:teuthology.orchestra.run.smithi094:> sudo tee -a /etc/ceph/ceph.keyring 2024-09-14T06:28:24.204 DEBUG:teuthology.orchestra.run.smithi138:> sudo tee -a /etc/ceph/ceph.keyring 2024-09-14T06:28:24.243 INFO:teuthology.orchestra.run.smithi094.stdout:[mgr.y] 2024-09-14T06:28:24.243 INFO:teuthology.orchestra.run.smithi094.stdout: key = AQDoLOVmvMcaLhAASJzELBFGAQbdUTdyz2Qgog== 2024-09-14T06:28:24.244 INFO:teuthology.orchestra.run.smithi094.stdout:[osd.0] 2024-09-14T06:28:24.244 INFO:teuthology.orchestra.run.smithi094.stdout: key = AQDqLOVmTpSrEBAAaO3aRWDmwbMzS2r/SjEbGg== 2024-09-14T06:28:24.244 INFO:teuthology.orchestra.run.smithi094.stdout:[osd.1] 2024-09-14T06:28:24.244 INFO:teuthology.orchestra.run.smithi094.stdout: key = AQDtLOVmsOHxIBAA75uGwAn54GG7TiLEfjI0LQ== 2024-09-14T06:28:24.244 INFO:teuthology.orchestra.run.smithi094.stdout:[osd.2] 2024-09-14T06:28:24.244 INFO:teuthology.orchestra.run.smithi094.stdout: key = AQDwLOVmXqVLIhAAfDnZx5cjpuqqokuVsJ3a9A== 2024-09-14T06:28:24.244 INFO:teuthology.orchestra.run.smithi094.stdout:[osd.3] 2024-09-14T06:28:24.244 INFO:teuthology.orchestra.run.smithi094.stdout: key = AQDzLOVmersIJRAA0CEZm3uDnXipFiErnX8ayg== 2024-09-14T06:28:24.244 INFO:teuthology.orchestra.run.smithi094.stdout:[mgr.x] 2024-09-14T06:28:24.244 INFO:teuthology.orchestra.run.smithi094.stdout: key = AQDoLOVmnmZRMhAADx/Ay4fohb1IW7LtUr6EMg== 2024-09-14T06:28:24.244 INFO:teuthology.orchestra.run.smithi094.stdout:[osd.4] 2024-09-14T06:28:24.244 INFO:teuthology.orchestra.run.smithi094.stdout: key = AQD6LOVmSjvOAhAAms6ONMl5ykTdFwFFLNLtog== 2024-09-14T06:28:24.245 INFO:teuthology.orchestra.run.smithi094.stdout:[osd.5] 2024-09-14T06:28:24.245 INFO:teuthology.orchestra.run.smithi094.stdout: key = AQD9LOVmhn/uMBAA50tVYdS31BJI5bLX/i2y/A== 2024-09-14T06:28:24.245 INFO:teuthology.orchestra.run.smithi094.stdout:[osd.6] 2024-09-14T06:28:24.245 INFO:teuthology.orchestra.run.smithi094.stdout: key = AQAALeVmBta7MhAAIJi81w37kEwoD4OVBEgGSw== 2024-09-14T06:28:24.245 INFO:teuthology.orchestra.run.smithi094.stdout:[osd.7] 2024-09-14T06:28:24.245 INFO:teuthology.orchestra.run.smithi094.stdout: key = AQAELeVmtkS6BxAAb3kUqJghFvIfDB9+KF6NAQ== 2024-09-14T06:28:24.245 INFO:teuthology.orchestra.run.smithi094.stdout:[client.0] 2024-09-14T06:28:24.245 INFO:teuthology.orchestra.run.smithi094.stdout: key = AQDoLOVmC4nvNBAAV2skm88gr42J/RIx+o+7NA== 2024-09-14T06:28:24.245 INFO:teuthology.orchestra.run.smithi094.stdout:[client.1] 2024-09-14T06:28:24.245 INFO:teuthology.orchestra.run.smithi094.stdout: key = AQDoLOVmLiEoOBAA3pXibLrCWy2ZoGPD217Axw== 2024-09-14T06:28:24.263 INFO:teuthology.orchestra.run.smithi138.stdout:[mgr.y] 2024-09-14T06:28:24.263 INFO:teuthology.orchestra.run.smithi138.stdout: key = AQDoLOVmvMcaLhAASJzELBFGAQbdUTdyz2Qgog== 2024-09-14T06:28:24.264 INFO:teuthology.orchestra.run.smithi138.stdout:[osd.0] 2024-09-14T06:28:24.264 INFO:teuthology.orchestra.run.smithi138.stdout: key = AQDqLOVmTpSrEBAAaO3aRWDmwbMzS2r/SjEbGg== 2024-09-14T06:28:24.264 INFO:teuthology.orchestra.run.smithi138.stdout:[osd.1] 2024-09-14T06:28:24.264 INFO:teuthology.orchestra.run.smithi138.stdout: key = AQDtLOVmsOHxIBAA75uGwAn54GG7TiLEfjI0LQ== 2024-09-14T06:28:24.264 INFO:teuthology.orchestra.run.smithi138.stdout:[osd.2] 2024-09-14T06:28:24.264 INFO:teuthology.orchestra.run.smithi138.stdout: key = AQDwLOVmXqVLIhAAfDnZx5cjpuqqokuVsJ3a9A== 2024-09-14T06:28:24.264 INFO:teuthology.orchestra.run.smithi138.stdout:[osd.3] 2024-09-14T06:28:24.264 INFO:teuthology.orchestra.run.smithi138.stdout: key = AQDzLOVmersIJRAA0CEZm3uDnXipFiErnX8ayg== 2024-09-14T06:28:24.264 INFO:teuthology.orchestra.run.smithi138.stdout:[mgr.x] 2024-09-14T06:28:24.264 INFO:teuthology.orchestra.run.smithi138.stdout: key = AQDoLOVmnmZRMhAADx/Ay4fohb1IW7LtUr6EMg== 2024-09-14T06:28:24.264 INFO:teuthology.orchestra.run.smithi138.stdout:[osd.4] 2024-09-14T06:28:24.264 INFO:teuthology.orchestra.run.smithi138.stdout: key = AQD6LOVmSjvOAhAAms6ONMl5ykTdFwFFLNLtog== 2024-09-14T06:28:24.264 INFO:teuthology.orchestra.run.smithi138.stdout:[osd.5] 2024-09-14T06:28:24.265 INFO:teuthology.orchestra.run.smithi138.stdout: key = AQD9LOVmhn/uMBAA50tVYdS31BJI5bLX/i2y/A== 2024-09-14T06:28:24.265 INFO:teuthology.orchestra.run.smithi138.stdout:[osd.6] 2024-09-14T06:28:24.265 INFO:teuthology.orchestra.run.smithi138.stdout: key = AQAALeVmBta7MhAAIJi81w37kEwoD4OVBEgGSw== 2024-09-14T06:28:24.265 INFO:teuthology.orchestra.run.smithi138.stdout:[osd.7] 2024-09-14T06:28:24.265 INFO:teuthology.orchestra.run.smithi138.stdout: key = AQAELeVmtkS6BxAAb3kUqJghFvIfDB9+KF6NAQ== 2024-09-14T06:28:24.265 INFO:teuthology.orchestra.run.smithi138.stdout:[client.0] 2024-09-14T06:28:24.265 INFO:teuthology.orchestra.run.smithi138.stdout: key = AQDoLOVmC4nvNBAAV2skm88gr42J/RIx+o+7NA== 2024-09-14T06:28:24.265 INFO:teuthology.orchestra.run.smithi138.stdout:[client.1] 2024-09-14T06:28:24.265 INFO:teuthology.orchestra.run.smithi138.stdout: key = AQDoLOVmLiEoOBAA3pXibLrCWy2ZoGPD217Axw== 2024-09-14T06:28:24.266 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-09-14T06:28:24.289 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-09-14T06:28:24.357 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-14T06:28:24.359 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-14T06:28:24.410 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-14T06:28:24.413 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-14T06:28:24.463 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-14T06:28:24.466 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-14T06:28:24.516 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-14T06:28:24.519 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-14T06:28:24.570 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-09-14T06:28:24.572 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-09-14T06:28:24.623 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-14T06:28:24.626 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-14T06:28:24.676 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-14T06:28:24.678 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-14T06:28:24.729 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-14T06:28:24.731 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-14T06:28:24.782 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-14T06:28:24.784 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-14T06:28:24.835 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-09-14T06:28:24.837 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-09-14T06:28:24.891 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-09-14T06:28:24.893 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-09-14T06:28:24.947 INFO:tasks.ceph:Running mkfs on mon nodes... 2024-09-14T06:28:24.947 DEBUG:teuthology.orchestra.run.smithi094:> sudo mkdir -p /var/lib/ceph/mon/ceph-a 2024-09-14T06:28:24.972 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i a --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-09-14T06:28:25.068 DEBUG:teuthology.orchestra.run.smithi094:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-a 2024-09-14T06:28:25.093 DEBUG:teuthology.orchestra.run.smithi094:> sudo mkdir -p /var/lib/ceph/mon/ceph-c 2024-09-14T06:28:25.157 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i c --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-09-14T06:28:25.253 DEBUG:teuthology.orchestra.run.smithi094:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-c 2024-09-14T06:28:25.278 DEBUG:teuthology.orchestra.run.smithi138:> sudo mkdir -p /var/lib/ceph/mon/ceph-b 2024-09-14T06:28:25.304 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i b --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-09-14T06:28:25.400 DEBUG:teuthology.orchestra.run.smithi138:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-b 2024-09-14T06:28:25.425 DEBUG:teuthology.orchestra.run.smithi094:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-09-14T06:28:25.427 DEBUG:teuthology.orchestra.run.smithi138:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-09-14T06:28:25.480 INFO:tasks.ceph:Starting mon daemons in cluster ceph... 2024-09-14T06:28:25.481 INFO:tasks.ceph.mon.a:Restarting daemon 2024-09-14T06:28:25.481 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i a 2024-09-14T06:28:25.483 INFO:tasks.ceph.mon.a:Started 2024-09-14T06:28:25.483 INFO:tasks.ceph.mon.c:Restarting daemon 2024-09-14T06:28:25.483 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i c 2024-09-14T06:28:25.485 INFO:tasks.ceph.mon.c:Started 2024-09-14T06:28:25.485 INFO:tasks.ceph.mon.b:Restarting daemon 2024-09-14T06:28:25.485 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i b 2024-09-14T06:28:25.522 INFO:tasks.ceph.mon.b:Started 2024-09-14T06:28:25.523 INFO:tasks.ceph:Starting mgr daemons in cluster ceph... 2024-09-14T06:28:25.523 INFO:tasks.ceph.mgr.y:Restarting daemon 2024-09-14T06:28:25.523 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i y 2024-09-14T06:28:25.525 INFO:tasks.ceph.mgr.y:Started 2024-09-14T06:28:25.526 INFO:tasks.ceph.mgr.x:Restarting daemon 2024-09-14T06:28:25.526 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i x 2024-09-14T06:28:25.528 INFO:tasks.ceph.mgr.x:Started 2024-09-14T06:28:25.528 DEBUG:tasks.ceph:set 0 configs 2024-09-14T06:28:25.528 INFO:tasks.ceph:Setting crush tunables to default 2024-09-14T06:28:25.528 DEBUG:teuthology.orchestra.run.smithi094:> sudo ceph --cluster ceph osd crush tunables default 2024-09-14T06:28:31.707 INFO:teuthology.orchestra.run.smithi094.stderr:adjusted tunables profile to default 2024-09-14T06:28:31.719 INFO:tasks.ceph:check_enable_crimson: False 2024-09-14T06:28:31.719 INFO:tasks.ceph:Starting osd daemons in cluster ceph... 2024-09-14T06:28:31.719 DEBUG:teuthology.orchestra.run.smithi094:> set -ex 2024-09-14T06:28:31.719 DEBUG:teuthology.orchestra.run.smithi094:> sudo dd if=/var/lib/ceph/osd/ceph-0/fsid of=/dev/stdout 2024-09-14T06:28:31.745 DEBUG:teuthology.orchestra.run.smithi094:> set -ex 2024-09-14T06:28:31.745 DEBUG:teuthology.orchestra.run.smithi094:> sudo dd if=/var/lib/ceph/osd/ceph-1/fsid of=/dev/stdout 2024-09-14T06:28:31.809 DEBUG:teuthology.orchestra.run.smithi094:> set -ex 2024-09-14T06:28:31.810 DEBUG:teuthology.orchestra.run.smithi094:> sudo dd if=/var/lib/ceph/osd/ceph-2/fsid of=/dev/stdout 2024-09-14T06:28:31.874 DEBUG:teuthology.orchestra.run.smithi094:> set -ex 2024-09-14T06:28:31.874 DEBUG:teuthology.orchestra.run.smithi094:> sudo dd if=/var/lib/ceph/osd/ceph-3/fsid of=/dev/stdout 2024-09-14T06:28:31.938 DEBUG:teuthology.orchestra.run.smithi138:> set -ex 2024-09-14T06:28:31.939 DEBUG:teuthology.orchestra.run.smithi138:> sudo dd if=/var/lib/ceph/osd/ceph-4/fsid of=/dev/stdout 2024-09-14T06:28:31.963 DEBUG:teuthology.orchestra.run.smithi138:> set -ex 2024-09-14T06:28:31.964 DEBUG:teuthology.orchestra.run.smithi138:> sudo dd if=/var/lib/ceph/osd/ceph-5/fsid of=/dev/stdout 2024-09-14T06:28:32.029 DEBUG:teuthology.orchestra.run.smithi138:> set -ex 2024-09-14T06:28:32.029 DEBUG:teuthology.orchestra.run.smithi138:> sudo dd if=/var/lib/ceph/osd/ceph-6/fsid of=/dev/stdout 2024-09-14T06:28:32.093 DEBUG:teuthology.orchestra.run.smithi138:> set -ex 2024-09-14T06:28:32.093 DEBUG:teuthology.orchestra.run.smithi138:> sudo dd if=/var/lib/ceph/osd/ceph-7/fsid of=/dev/stdout 2024-09-14T06:28:32.157 DEBUG:teuthology.orchestra.run.smithi138:> sudo ceph --cluster ceph osd new 8b22a249-b842-406a-a02c-c0f31f57b635 0 2024-09-14T06:28:32.378 INFO:teuthology.orchestra.run.smithi138.stdout:0 2024-09-14T06:28:32.391 DEBUG:teuthology.orchestra.run.smithi138:> sudo ceph --cluster ceph osd new 02251a03-84a0-414d-a980-fac4fd043cb6 1 2024-09-14T06:28:32.574 INFO:teuthology.orchestra.run.smithi138.stdout:1 2024-09-14T06:28:32.586 DEBUG:teuthology.orchestra.run.smithi138:> sudo ceph --cluster ceph osd new d45bf470-bb29-4739-8294-cd122b2b4102 2 2024-09-14T06:28:32.764 INFO:teuthology.orchestra.run.smithi138.stdout:2 2024-09-14T06:28:32.777 DEBUG:teuthology.orchestra.run.smithi138:> sudo ceph --cluster ceph osd new 456ec2d9-c438-4243-b931-335704f661eb 3 2024-09-14T06:28:32.961 INFO:teuthology.orchestra.run.smithi138.stdout:3 2024-09-14T06:28:32.972 DEBUG:teuthology.orchestra.run.smithi138:> sudo ceph --cluster ceph osd new 2dab948e-c6e4-4437-bf17-017d1ac6821c 4 2024-09-14T06:28:33.150 INFO:teuthology.orchestra.run.smithi138.stdout:4 2024-09-14T06:28:33.162 DEBUG:teuthology.orchestra.run.smithi138:> sudo ceph --cluster ceph osd new 8caf3aa3-ad87-4d33-b82e-4c15195176fd 5 2024-09-14T06:28:33.345 INFO:teuthology.orchestra.run.smithi138.stdout:5 2024-09-14T06:28:33.356 DEBUG:teuthology.orchestra.run.smithi138:> sudo ceph --cluster ceph osd new e55d11fd-433d-4462-93a7-d5e84560221b 6 2024-09-14T06:28:33.535 INFO:teuthology.orchestra.run.smithi138.stdout:6 2024-09-14T06:28:33.546 DEBUG:teuthology.orchestra.run.smithi138:> sudo ceph --cluster ceph osd new ba14ba69-2c6c-4d09-81e2-2593dc3a7ea9 7 2024-09-14T06:28:33.727 INFO:teuthology.orchestra.run.smithi138.stdout:7 2024-09-14T06:28:33.738 INFO:tasks.ceph.osd.0:Restarting daemon 2024-09-14T06:28:33.738 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 0 2024-09-14T06:28:33.741 INFO:tasks.ceph.osd.0:Started 2024-09-14T06:28:33.741 INFO:tasks.ceph.osd.1:Restarting daemon 2024-09-14T06:28:33.741 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 1 2024-09-14T06:28:33.743 INFO:tasks.ceph.osd.1:Started 2024-09-14T06:28:33.743 INFO:tasks.ceph.osd.2:Restarting daemon 2024-09-14T06:28:33.743 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 2 2024-09-14T06:28:33.745 INFO:tasks.ceph.osd.2:Started 2024-09-14T06:28:33.745 INFO:tasks.ceph.osd.3:Restarting daemon 2024-09-14T06:28:33.745 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 3 2024-09-14T06:28:33.746 INFO:tasks.ceph.osd.3:Started 2024-09-14T06:28:33.747 INFO:tasks.ceph.osd.4:Restarting daemon 2024-09-14T06:28:33.747 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 4 2024-09-14T06:28:33.749 INFO:tasks.ceph.osd.4:Started 2024-09-14T06:28:33.749 INFO:tasks.ceph.osd.5:Restarting daemon 2024-09-14T06:28:33.749 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 5 2024-09-14T06:28:33.751 INFO:tasks.ceph.osd.5:Started 2024-09-14T06:28:33.751 INFO:tasks.ceph.osd.6:Restarting daemon 2024-09-14T06:28:33.751 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 6 2024-09-14T06:28:33.753 INFO:tasks.ceph.osd.6:Started 2024-09-14T06:28:33.753 INFO:tasks.ceph.osd.7:Restarting daemon 2024-09-14T06:28:33.753 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 7 2024-09-14T06:28:33.755 INFO:tasks.ceph.osd.7:Started 2024-09-14T06:28:33.756 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-14T06:28:33.948 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-14T06:28:33.948 INFO:teuthology.orchestra.run.smithi094.stdout:{"epoch":10,"fsid":"200ad3da-bd63-4e04-af74-efb5ca0af75e","created":"2024-09-14T06:28:30.782046+0000","modified":"2024-09-14T06:28:33.724176+0000","last_up_change":"0.000000","last_in_change":"2024-09-14T06:28:33.724176+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"8b22a249-b842-406a-a02c-c0f31f57b635","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"02251a03-84a0-414d-a980-fac4fd043cb6","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"d45bf470-bb29-4739-8294-cd122b2b4102","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"456ec2d9-c438-4243-b931-335704f661eb","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"2dab948e-c6e4-4437-bf17-017d1ac6821c","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"8caf3aa3-ad87-4d33-b82e-4c15195176fd","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"e55d11fd-433d-4462-93a7-d5e84560221b","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"ba14ba69-2c6c-4d09-81e2-2593dc3a7ea9","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-14T06:28:33.959 INFO:tasks.ceph.ceph_manager.ceph:[] 2024-09-14T06:28:33.959 INFO:tasks.ceph:Waiting for OSDs to come up 2024-09-14T06:28:34.712 INFO:tasks.ceph.osd.3.smithi094.stderr:2024-09-14T06:28:34.710+0000 7feeee044640 -1 Falling back to public interface 2024-09-14T06:28:34.765 DEBUG:teuthology.orchestra.run.smithi094:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-09-14T06:28:34.926 INFO:teuthology.misc.health.smithi094.stdout: 2024-09-14T06:28:34.927 INFO:teuthology.misc.health.smithi094.stdout:{"epoch":10,"fsid":"200ad3da-bd63-4e04-af74-efb5ca0af75e","created":"2024-09-14T06:28:30.782046+0000","modified":"2024-09-14T06:28:33.724176+0000","last_up_change":"0.000000","last_in_change":"2024-09-14T06:28:33.724176+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"8b22a249-b842-406a-a02c-c0f31f57b635","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"02251a03-84a0-414d-a980-fac4fd043cb6","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"d45bf470-bb29-4739-8294-cd122b2b4102","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"456ec2d9-c438-4243-b931-335704f661eb","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"2dab948e-c6e4-4437-bf17-017d1ac6821c","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"8caf3aa3-ad87-4d33-b82e-4c15195176fd","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"e55d11fd-433d-4462-93a7-d5e84560221b","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"ba14ba69-2c6c-4d09-81e2-2593dc3a7ea9","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-14T06:28:34.936 DEBUG:teuthology.misc:0 of 8 OSDs are up 2024-09-14T06:28:35.195 INFO:tasks.ceph.osd.2.smithi094.stderr:2024-09-14T06:28:35.193+0000 7faf35e0c640 -1 Falling back to public interface 2024-09-14T06:28:35.200 INFO:tasks.ceph.osd.1.smithi094.stderr:2024-09-14T06:28:35.197+0000 7f32914d6640 -1 Falling back to public interface 2024-09-14T06:28:35.218 INFO:tasks.ceph.osd.7.smithi138.stderr:2024-09-14T06:28:35.217+0000 7f3c6d81d640 -1 Falling back to public interface 2024-09-14T06:28:35.438 INFO:tasks.ceph.osd.0.smithi094.stderr:2024-09-14T06:28:35.436+0000 7f07cca0f640 -1 Falling back to public interface 2024-09-14T06:28:35.441 INFO:tasks.ceph.osd.4.smithi138.stderr:2024-09-14T06:28:35.440+0000 7f6c68424640 -1 Falling back to public interface 2024-09-14T06:28:35.446 INFO:tasks.ceph.osd.6.smithi138.stderr:2024-09-14T06:28:35.445+0000 7f8cdb8a5640 -1 Falling back to public interface 2024-09-14T06:28:35.724 INFO:tasks.ceph.osd.5.smithi138.stderr:2024-09-14T06:28:35.722+0000 7fe13aa1a640 -1 Falling back to public interface 2024-09-14T06:28:36.285 INFO:tasks.ceph.mgr.x.smithi138.stderr:2024-09-14T06:28:36.284+0000 7f40e7725640 -1 mgr.server handle_report got status from non-daemon mon.c 2024-09-14T06:28:36.285 INFO:tasks.ceph.mgr.x.smithi138.stderr:2024-09-14T06:28:36.284+0000 7f40e7725640 -1 mgr.server handle_report got status from non-daemon mon.b 2024-09-14T06:28:38.261 INFO:tasks.ceph.osd.3.smithi094.stderr:2024-09-14T06:28:38.260+0000 7feeee044640 -1 osd.3 0 log_to_monitors true 2024-09-14T06:28:38.943 INFO:tasks.ceph.osd.0.smithi094.stderr:2024-09-14T06:28:38.941+0000 7f07cca0f640 -1 osd.0 0 log_to_monitors true 2024-09-14T06:28:39.000 INFO:tasks.ceph.osd.2.smithi094.stderr:2024-09-14T06:28:38.999+0000 7faf35e0c640 -1 osd.2 0 log_to_monitors true 2024-09-14T06:28:39.234 INFO:tasks.ceph.osd.4.smithi138.stderr:2024-09-14T06:28:39.233+0000 7f6c68424640 -1 osd.4 0 log_to_monitors true 2024-09-14T06:28:39.301 INFO:tasks.ceph.osd.7.smithi138.stderr:2024-09-14T06:28:39.300+0000 7f3c6d81d640 -1 osd.7 0 log_to_monitors true 2024-09-14T06:28:39.525 INFO:tasks.ceph.osd.1.smithi094.stderr:2024-09-14T06:28:39.524+0000 7f32914d6640 -1 osd.1 0 log_to_monitors true 2024-09-14T06:28:39.562 INFO:tasks.ceph.osd.6.smithi138.stderr:2024-09-14T06:28:39.561+0000 7f8cdb8a5640 -1 osd.6 0 log_to_monitors true 2024-09-14T06:28:39.774 INFO:tasks.ceph.osd.5.smithi138.stderr:2024-09-14T06:28:39.773+0000 7fe13aa1a640 -1 osd.5 0 log_to_monitors true 2024-09-14T06:28:40.506 INFO:tasks.ceph.osd.3.smithi094.stderr:2024-09-14T06:28:40.505+0000 7feeea7e7640 -1 osd.3 0 waiting for initial osdmap 2024-09-14T06:28:40.515 INFO:tasks.ceph.osd.3.smithi094.stderr:2024-09-14T06:28:40.514+0000 7feee1df2640 -1 osd.3 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-14T06:28:41.744 DEBUG:teuthology.orchestra.run.smithi094:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-09-14T06:28:41.785 INFO:tasks.ceph.osd.0.smithi094.stderr:2024-09-14T06:28:41.784+0000 7f07c831e640 -1 osd.0 0 waiting for initial osdmap 2024-09-14T06:28:41.794 INFO:tasks.ceph.osd.2.smithi094.stderr:2024-09-14T06:28:41.793+0000 7faf31686640 -1 osd.2 0 waiting for initial osdmap 2024-09-14T06:28:41.800 INFO:tasks.ceph.osd.0.smithi094.stderr:2024-09-14T06:28:41.799+0000 7f07c013d640 -1 osd.0 13 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-14T06:28:41.807 INFO:tasks.ceph.osd.2.smithi094.stderr:2024-09-14T06:28:41.806+0000 7faf294a5640 -1 osd.2 13 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-14T06:28:41.813 INFO:tasks.ceph.osd.1.smithi094.stderr:2024-09-14T06:28:41.811+0000 7f328d465640 -1 osd.1 0 waiting for initial osdmap 2024-09-14T06:28:41.824 INFO:tasks.ceph.osd.1.smithi094.stderr:2024-09-14T06:28:41.823+0000 7f3285284640 -1 osd.1 13 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-14T06:28:41.917 INFO:tasks.ceph.osd.5.smithi138.stderr:2024-09-14T06:28:41.916+0000 7fe13645a640 -1 osd.5 0 waiting for initial osdmap 2024-09-14T06:28:41.927 INFO:tasks.ceph.osd.6.smithi138.stderr:2024-09-14T06:28:41.926+0000 7f8cd8048640 -1 osd.6 0 waiting for initial osdmap 2024-09-14T06:28:41.928 INFO:tasks.ceph.osd.5.smithi138.stderr:2024-09-14T06:28:41.927+0000 7fe12e279640 -1 osd.5 14 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-14T06:28:41.942 INFO:tasks.ceph.osd.6.smithi138.stderr:2024-09-14T06:28:41.941+0000 7f8ccf653640 -1 osd.6 14 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-14T06:28:41.964 INFO:tasks.ceph.osd.7.smithi138.stderr:2024-09-14T06:28:41.963+0000 7f3c69208640 -1 osd.7 0 waiting for initial osdmap 2024-09-14T06:28:41.972 INFO:tasks.ceph.osd.4.smithi138.stderr:2024-09-14T06:28:41.971+0000 7f6c643b3640 -1 osd.4 0 waiting for initial osdmap 2024-09-14T06:28:41.974 INFO:tasks.ceph.osd.7.smithi138.stderr:2024-09-14T06:28:41.973+0000 7f3c61027640 -1 osd.7 14 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-14T06:28:41.987 INFO:tasks.ceph.osd.4.smithi138.stderr:2024-09-14T06:28:41.986+0000 7f6c5c1d2640 -1 osd.4 14 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-14T06:28:42.046 INFO:teuthology.misc.health.smithi094.stdout: 2024-09-14T06:28:42.046 INFO:teuthology.misc.health.smithi094.stdout:{"epoch":14,"fsid":"200ad3da-bd63-4e04-af74-efb5ca0af75e","created":"2024-09-14T06:28:30.782046+0000","modified":"2024-09-14T06:28:41.823485+0000","last_up_change":"2024-09-14T06:28:41.823485+0000","last_in_change":"2024-09-14T06:28:33.724176+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"8b22a249-b842-406a-a02c-c0f31f57b635","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6824","nonce":1809564923},{"type":"v1","addr":"172.21.15.94:6825","nonce":1809564923}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6826","nonce":1809564923},{"type":"v1","addr":"172.21.15.94:6827","nonce":1809564923}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6830","nonce":1809564923},{"type":"v1","addr":"172.21.15.94:6831","nonce":1809564923}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6828","nonce":1809564923},{"type":"v1","addr":"172.21.15.94:6829","nonce":1809564923}]},"public_addr":"172.21.15.94:6825/1809564923","cluster_addr":"172.21.15.94:6827/1809564923","heartbeat_back_addr":"172.21.15.94:6831/1809564923","heartbeat_front_addr":"172.21.15.94:6829/1809564923","state":["exists","up"]},{"osd":1,"uuid":"02251a03-84a0-414d-a980-fac4fd043cb6","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"d45bf470-bb29-4739-8294-cd122b2b4102","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6808","nonce":1063204782},{"type":"v1","addr":"172.21.15.94:6809","nonce":1063204782}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6810","nonce":1063204782},{"type":"v1","addr":"172.21.15.94:6811","nonce":1063204782}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6814","nonce":1063204782},{"type":"v1","addr":"172.21.15.94:6815","nonce":1063204782}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6812","nonce":1063204782},{"type":"v1","addr":"172.21.15.94:6813","nonce":1063204782}]},"public_addr":"172.21.15.94:6809/1063204782","cluster_addr":"172.21.15.94:6811/1063204782","heartbeat_back_addr":"172.21.15.94:6815/1063204782","heartbeat_front_addr":"172.21.15.94:6813/1063204782","state":["exists","up"]},{"osd":3,"uuid":"456ec2d9-c438-4243-b931-335704f661eb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6800","nonce":3382046754},{"type":"v1","addr":"172.21.15.94:6801","nonce":3382046754}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6802","nonce":3382046754},{"type":"v1","addr":"172.21.15.94:6803","nonce":3382046754}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6806","nonce":3382046754},{"type":"v1","addr":"172.21.15.94:6807","nonce":3382046754}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6804","nonce":3382046754},{"type":"v1","addr":"172.21.15.94:6805","nonce":3382046754}]},"public_addr":"172.21.15.94:6801/3382046754","cluster_addr":"172.21.15.94:6803/3382046754","heartbeat_back_addr":"172.21.15.94:6807/3382046754","heartbeat_front_addr":"172.21.15.94:6805/3382046754","state":["exists","up"]},{"osd":4,"uuid":"2dab948e-c6e4-4437-bf17-017d1ac6821c","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"8caf3aa3-ad87-4d33-b82e-4c15195176fd","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"e55d11fd-433d-4462-93a7-d5e84560221b","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"ba14ba69-2c6c-4d09-81e2-2593dc3a7ea9","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-14T06:28:39.217944+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-14T06:28:42.057 DEBUG:teuthology.misc:3 of 8 OSDs are up 2024-09-14T06:28:48.865 DEBUG:teuthology.orchestra.run.smithi094:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-09-14T06:28:49.154 INFO:teuthology.misc.health.smithi094.stdout: 2024-09-14T06:28:49.154 INFO:teuthology.misc.health.smithi094.stdout:{"epoch":18,"fsid":"200ad3da-bd63-4e04-af74-efb5ca0af75e","created":"2024-09-14T06:28:30.782046+0000","modified":"2024-09-14T06:28:45.836625+0000","last_up_change":"2024-09-14T06:28:42.827239+0000","last_in_change":"2024-09-14T06:28:33.724176+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-14T06:28:43.295990+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"8b22a249-b842-406a-a02c-c0f31f57b635","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6824","nonce":1809564923},{"type":"v1","addr":"172.21.15.94:6825","nonce":1809564923}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6826","nonce":1809564923},{"type":"v1","addr":"172.21.15.94:6827","nonce":1809564923}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6830","nonce":1809564923},{"type":"v1","addr":"172.21.15.94:6831","nonce":1809564923}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6828","nonce":1809564923},{"type":"v1","addr":"172.21.15.94:6829","nonce":1809564923}]},"public_addr":"172.21.15.94:6825/1809564923","cluster_addr":"172.21.15.94:6827/1809564923","heartbeat_back_addr":"172.21.15.94:6831/1809564923","heartbeat_front_addr":"172.21.15.94:6829/1809564923","state":["exists","up"]},{"osd":1,"uuid":"02251a03-84a0-414d-a980-fac4fd043cb6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6816","nonce":736402652},{"type":"v1","addr":"172.21.15.94:6817","nonce":736402652}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6818","nonce":736402652},{"type":"v1","addr":"172.21.15.94:6819","nonce":736402652}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6822","nonce":736402652},{"type":"v1","addr":"172.21.15.94:6823","nonce":736402652}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6820","nonce":736402652},{"type":"v1","addr":"172.21.15.94:6821","nonce":736402652}]},"public_addr":"172.21.15.94:6817/736402652","cluster_addr":"172.21.15.94:6819/736402652","heartbeat_back_addr":"172.21.15.94:6823/736402652","heartbeat_front_addr":"172.21.15.94:6821/736402652","state":["exists","up"]},{"osd":2,"uuid":"d45bf470-bb29-4739-8294-cd122b2b4102","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6808","nonce":1063204782},{"type":"v1","addr":"172.21.15.94:6809","nonce":1063204782}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6810","nonce":1063204782},{"type":"v1","addr":"172.21.15.94:6811","nonce":1063204782}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6814","nonce":1063204782},{"type":"v1","addr":"172.21.15.94:6815","nonce":1063204782}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6812","nonce":1063204782},{"type":"v1","addr":"172.21.15.94:6813","nonce":1063204782}]},"public_addr":"172.21.15.94:6809/1063204782","cluster_addr":"172.21.15.94:6811/1063204782","heartbeat_back_addr":"172.21.15.94:6815/1063204782","heartbeat_front_addr":"172.21.15.94:6813/1063204782","state":["exists","up"]},{"osd":3,"uuid":"456ec2d9-c438-4243-b931-335704f661eb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6800","nonce":3382046754},{"type":"v1","addr":"172.21.15.94:6801","nonce":3382046754}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6802","nonce":3382046754},{"type":"v1","addr":"172.21.15.94:6803","nonce":3382046754}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6806","nonce":3382046754},{"type":"v1","addr":"172.21.15.94:6807","nonce":3382046754}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6804","nonce":3382046754},{"type":"v1","addr":"172.21.15.94:6805","nonce":3382046754}]},"public_addr":"172.21.15.94:6801/3382046754","cluster_addr":"172.21.15.94:6803/3382046754","heartbeat_back_addr":"172.21.15.94:6807/3382046754","heartbeat_front_addr":"172.21.15.94:6805/3382046754","state":["exists","up"]},{"osd":4,"uuid":"2dab948e-c6e4-4437-bf17-017d1ac6821c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6810","nonce":501726729},{"type":"v1","addr":"172.21.15.138:6811","nonce":501726729}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6812","nonce":501726729},{"type":"v1","addr":"172.21.15.138:6813","nonce":501726729}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6816","nonce":501726729},{"type":"v1","addr":"172.21.15.138:6817","nonce":501726729}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6814","nonce":501726729},{"type":"v1","addr":"172.21.15.138:6815","nonce":501726729}]},"public_addr":"172.21.15.138:6811/501726729","cluster_addr":"172.21.15.138:6813/501726729","heartbeat_back_addr":"172.21.15.138:6817/501726729","heartbeat_front_addr":"172.21.15.138:6815/501726729","state":["exists","up"]},{"osd":5,"uuid":"8caf3aa3-ad87-4d33-b82e-4c15195176fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6826","nonce":1599292735},{"type":"v1","addr":"172.21.15.138:6827","nonce":1599292735}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6828","nonce":1599292735},{"type":"v1","addr":"172.21.15.138:6829","nonce":1599292735}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6832","nonce":1599292735},{"type":"v1","addr":"172.21.15.138:6833","nonce":1599292735}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6830","nonce":1599292735},{"type":"v1","addr":"172.21.15.138:6831","nonce":1599292735}]},"public_addr":"172.21.15.138:6827/1599292735","cluster_addr":"172.21.15.138:6829/1599292735","heartbeat_back_addr":"172.21.15.138:6833/1599292735","heartbeat_front_addr":"172.21.15.138:6831/1599292735","state":["exists","up"]},{"osd":6,"uuid":"e55d11fd-433d-4462-93a7-d5e84560221b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6818","nonce":163531479},{"type":"v1","addr":"172.21.15.138:6819","nonce":163531479}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6820","nonce":163531479},{"type":"v1","addr":"172.21.15.138:6821","nonce":163531479}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6824","nonce":163531479},{"type":"v1","addr":"172.21.15.138:6825","nonce":163531479}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6822","nonce":163531479},{"type":"v1","addr":"172.21.15.138:6823","nonce":163531479}]},"public_addr":"172.21.15.138:6819/163531479","cluster_addr":"172.21.15.138:6821/163531479","heartbeat_back_addr":"172.21.15.138:6825/163531479","heartbeat_front_addr":"172.21.15.138:6823/163531479","state":["exists","up"]},{"osd":7,"uuid":"ba14ba69-2c6c-4d09-81e2-2593dc3a7ea9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6800","nonce":218507235},{"type":"v1","addr":"172.21.15.138:6801","nonce":218507235}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6802","nonce":218507235},{"type":"v1","addr":"172.21.15.138:6803","nonce":218507235}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6806","nonce":218507235},{"type":"v1","addr":"172.21.15.138:6807","nonce":218507235}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6804","nonce":218507235},{"type":"v1","addr":"172.21.15.138:6805","nonce":218507235}]},"public_addr":"172.21.15.138:6801/218507235","cluster_addr":"172.21.15.138:6803/218507235","heartbeat_back_addr":"172.21.15.138:6807/218507235","heartbeat_front_addr":"172.21.15.138:6805/218507235","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-14T06:28:39.906639+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-14T06:28:40.541279+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-14T06:28:40.025350+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-14T06:28:39.217944+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-14T06:28:40.185898+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-14T06:28:40.783504+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-14T06:28:40.527305+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-14T06:28:40.349676+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-14T06:28:49.165 DEBUG:teuthology.misc:8 of 8 OSDs are up 2024-09-14T06:28:49.166 INFO:tasks.ceph:Creating RBD pool 2024-09-14T06:28:49.166 DEBUG:teuthology.orchestra.run.smithi094:> sudo ceph --cluster ceph osd pool create rbd 8 2024-09-14T06:28:50.210 INFO:teuthology.orchestra.run.smithi094.stderr:pool 'rbd' created 2024-09-14T06:28:50.225 DEBUG:teuthology.orchestra.run.smithi094:> rbd --cluster ceph pool init rbd 2024-09-14T06:28:53.241 INFO:tasks.ceph:Starting mds daemons in cluster ceph... 2024-09-14T06:28:53.243 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config log 1 --format=json 2024-09-14T06:28:53.243 INFO:tasks.daemonwatchdog.daemon_watchdog:watchdog starting 2024-09-14T06:28:53.534 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-14T06:28:53.546 INFO:teuthology.orchestra.run.smithi094.stdout:[{"version":9,"timestamp":"2024-09-14T06:28:41.973123+0000","name":"","changes":[{"name":"osd.4/osd_mclock_max_capacity_iops_ssd","new_value":"4562.332868"}]}] 2024-09-14T06:28:53.547 INFO:tasks.ceph_manager:config epoch is 9 2024-09-14T06:28:53.547 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-09-14T06:28:53.547 INFO:tasks.ceph.ceph_manager.ceph:waiting for mgr available 2024-09-14T06:28:53.547 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json 2024-09-14T06:28:53.879 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-14T06:28:53.889 INFO:teuthology.orchestra.run.smithi094.stdout:{"epoch":5,"flags":0,"active_gid":4102,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6808","nonce":2403047907},{"type":"v1","addr":"172.21.15.138:6809","nonce":2403047907}]},"active_addr":"172.21.15.138:6809/2403047907","active_change":"2024-09-14T06:28:35.272821+0000","active_mgr_features":4540719139924082687,"available":true,"standbys":[{"gid":4107,"name":"y","mgr_features":4540719139924082687,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to, use commas to separate multiple","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.27.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:10.4.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_nginx":{"name":"container_image_nginx","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nginx:1.26.1","min":"","max":"","enum_allowed":[],"desc":"Nginx container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.7.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.17","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_oauth2_proxy":{"name":"container_image_oauth2_proxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/oauth2-proxy/oauth2-proxy:v7.6.0","min":"","max":"","enum_allowed":[],"desc":"oauth2-proxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.51.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_requests":{"name":"max_requests","type":"int","level":"advanced","flags":0,"default_value":"500","min":"","max":"","enum_allowed":[],"desc":"Maximum number of requests to keep in memory. When new request comes in, the oldest request will be removed if the number of requests exceeds the max request number.if un-finished request is removed, error message will be logged in the ceph-mgr log.","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"secondary_zone_period_retry_limit":{"name":"secondary_zone_period_retry_limit","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"RGW module period update retry limit for secondary site","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"smb","can_run":true,"error_string":"","module_options":{"internal_store_backend":{"name":"internal_store_backend","type":"str","level":"dev","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"set internal store backend. for develoment and testing only","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"update_orchestration":{"name":"update_orchestration","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically update orchestration when smb resources are changed","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to, use commas to separate multiple","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.27.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:10.4.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_nginx":{"name":"container_image_nginx","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nginx:1.26.1","min":"","max":"","enum_allowed":[],"desc":"Nginx container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.7.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.17","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_oauth2_proxy":{"name":"container_image_oauth2_proxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/oauth2-proxy/oauth2-proxy:v7.6.0","min":"","max":"","enum_allowed":[],"desc":"oauth2-proxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.51.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_requests":{"name":"max_requests","type":"int","level":"advanced","flags":0,"default_value":"500","min":"","max":"","enum_allowed":[],"desc":"Maximum number of requests to keep in memory. When new request comes in, the oldest request will be removed if the number of requests exceeds the max request number.if un-finished request is removed, error message will be logged in the ceph-mgr log.","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"secondary_zone_period_retry_limit":{"name":"secondary_zone_period_retry_limit","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"RGW module period update retry limit for secondary site","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"smb","can_run":true,"error_string":"","module_options":{"internal_store_backend":{"name":"internal_store_backend","type":"str","level":"dev","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"set internal store backend. for develoment and testing only","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"update_orchestration":{"name":"update_orchestration","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically update orchestration when smb resources are changed","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"squid":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":0,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.138:0","nonce":2587944639}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.138:0","nonce":3795168327}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.138:0","nonce":432035839}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.138:0","nonce":2661375497}]}]} 2024-09-14T06:28:53.893 INFO:tasks.ceph.ceph_manager.ceph:mgr available! 2024-09-14T06:28:53.893 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2024-09-14T06:28:53.893 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-14T06:28:54.179 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-14T06:28:54.179 INFO:teuthology.orchestra.run.smithi094.stdout:{"epoch":22,"fsid":"200ad3da-bd63-4e04-af74-efb5ca0af75e","created":"2024-09-14T06:28:30.782046+0000","modified":"2024-09-14T06:28:53.222080+0000","last_up_change":"2024-09-14T06:28:42.827239+0000","last_in_change":"2024-09-14T06:28:33.724176+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-14T06:28:43.295990+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2024-09-14T06:28:49.451714+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":2,"snap_epoch":22,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"8b22a249-b842-406a-a02c-c0f31f57b635","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6824","nonce":1809564923},{"type":"v1","addr":"172.21.15.94:6825","nonce":1809564923}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6826","nonce":1809564923},{"type":"v1","addr":"172.21.15.94:6827","nonce":1809564923}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6830","nonce":1809564923},{"type":"v1","addr":"172.21.15.94:6831","nonce":1809564923}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6828","nonce":1809564923},{"type":"v1","addr":"172.21.15.94:6829","nonce":1809564923}]},"public_addr":"172.21.15.94:6825/1809564923","cluster_addr":"172.21.15.94:6827/1809564923","heartbeat_back_addr":"172.21.15.94:6831/1809564923","heartbeat_front_addr":"172.21.15.94:6829/1809564923","state":["exists","up"]},{"osd":1,"uuid":"02251a03-84a0-414d-a980-fac4fd043cb6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6816","nonce":736402652},{"type":"v1","addr":"172.21.15.94:6817","nonce":736402652}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6818","nonce":736402652},{"type":"v1","addr":"172.21.15.94:6819","nonce":736402652}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6822","nonce":736402652},{"type":"v1","addr":"172.21.15.94:6823","nonce":736402652}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6820","nonce":736402652},{"type":"v1","addr":"172.21.15.94:6821","nonce":736402652}]},"public_addr":"172.21.15.94:6817/736402652","cluster_addr":"172.21.15.94:6819/736402652","heartbeat_back_addr":"172.21.15.94:6823/736402652","heartbeat_front_addr":"172.21.15.94:6821/736402652","state":["exists","up"]},{"osd":2,"uuid":"d45bf470-bb29-4739-8294-cd122b2b4102","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6808","nonce":1063204782},{"type":"v1","addr":"172.21.15.94:6809","nonce":1063204782}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6810","nonce":1063204782},{"type":"v1","addr":"172.21.15.94:6811","nonce":1063204782}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6814","nonce":1063204782},{"type":"v1","addr":"172.21.15.94:6815","nonce":1063204782}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6812","nonce":1063204782},{"type":"v1","addr":"172.21.15.94:6813","nonce":1063204782}]},"public_addr":"172.21.15.94:6809/1063204782","cluster_addr":"172.21.15.94:6811/1063204782","heartbeat_back_addr":"172.21.15.94:6815/1063204782","heartbeat_front_addr":"172.21.15.94:6813/1063204782","state":["exists","up"]},{"osd":3,"uuid":"456ec2d9-c438-4243-b931-335704f661eb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6800","nonce":3382046754},{"type":"v1","addr":"172.21.15.94:6801","nonce":3382046754}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6802","nonce":3382046754},{"type":"v1","addr":"172.21.15.94:6803","nonce":3382046754}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6806","nonce":3382046754},{"type":"v1","addr":"172.21.15.94:6807","nonce":3382046754}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6804","nonce":3382046754},{"type":"v1","addr":"172.21.15.94:6805","nonce":3382046754}]},"public_addr":"172.21.15.94:6801/3382046754","cluster_addr":"172.21.15.94:6803/3382046754","heartbeat_back_addr":"172.21.15.94:6807/3382046754","heartbeat_front_addr":"172.21.15.94:6805/3382046754","state":["exists","up"]},{"osd":4,"uuid":"2dab948e-c6e4-4437-bf17-017d1ac6821c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6810","nonce":501726729},{"type":"v1","addr":"172.21.15.138:6811","nonce":501726729}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6812","nonce":501726729},{"type":"v1","addr":"172.21.15.138:6813","nonce":501726729}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6816","nonce":501726729},{"type":"v1","addr":"172.21.15.138:6817","nonce":501726729}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6814","nonce":501726729},{"type":"v1","addr":"172.21.15.138:6815","nonce":501726729}]},"public_addr":"172.21.15.138:6811/501726729","cluster_addr":"172.21.15.138:6813/501726729","heartbeat_back_addr":"172.21.15.138:6817/501726729","heartbeat_front_addr":"172.21.15.138:6815/501726729","state":["exists","up"]},{"osd":5,"uuid":"8caf3aa3-ad87-4d33-b82e-4c15195176fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6826","nonce":1599292735},{"type":"v1","addr":"172.21.15.138:6827","nonce":1599292735}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6828","nonce":1599292735},{"type":"v1","addr":"172.21.15.138:6829","nonce":1599292735}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6832","nonce":1599292735},{"type":"v1","addr":"172.21.15.138:6833","nonce":1599292735}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6830","nonce":1599292735},{"type":"v1","addr":"172.21.15.138:6831","nonce":1599292735}]},"public_addr":"172.21.15.138:6827/1599292735","cluster_addr":"172.21.15.138:6829/1599292735","heartbeat_back_addr":"172.21.15.138:6833/1599292735","heartbeat_front_addr":"172.21.15.138:6831/1599292735","state":["exists","up"]},{"osd":6,"uuid":"e55d11fd-433d-4462-93a7-d5e84560221b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6818","nonce":163531479},{"type":"v1","addr":"172.21.15.138:6819","nonce":163531479}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6820","nonce":163531479},{"type":"v1","addr":"172.21.15.138:6821","nonce":163531479}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6824","nonce":163531479},{"type":"v1","addr":"172.21.15.138:6825","nonce":163531479}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6822","nonce":163531479},{"type":"v1","addr":"172.21.15.138:6823","nonce":163531479}]},"public_addr":"172.21.15.138:6819/163531479","cluster_addr":"172.21.15.138:6821/163531479","heartbeat_back_addr":"172.21.15.138:6825/163531479","heartbeat_front_addr":"172.21.15.138:6823/163531479","state":["exists","up"]},{"osd":7,"uuid":"ba14ba69-2c6c-4d09-81e2-2593dc3a7ea9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6800","nonce":218507235},{"type":"v1","addr":"172.21.15.138:6801","nonce":218507235}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6802","nonce":218507235},{"type":"v1","addr":"172.21.15.138:6803","nonce":218507235}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6806","nonce":218507235},{"type":"v1","addr":"172.21.15.138:6807","nonce":218507235}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6804","nonce":218507235},{"type":"v1","addr":"172.21.15.138:6805","nonce":218507235}]},"public_addr":"172.21.15.138:6801/218507235","cluster_addr":"172.21.15.138:6803/218507235","heartbeat_back_addr":"172.21.15.138:6807/218507235","heartbeat_front_addr":"172.21.15.138:6805/218507235","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-14T06:28:39.906639+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-14T06:28:40.541279+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-14T06:28:40.025350+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-14T06:28:39.217944+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-14T06:28:40.185898+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-14T06:28:40.783504+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-14T06:28:40.527305+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-14T06:28:40.349676+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":2,"snaps":[{"begin":2,"length":1}]}],"new_removed_snaps":[{"pool":2,"snaps":[{"begin":2,"length":1}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-14T06:28:54.192 INFO:tasks.ceph.ceph_manager.ceph:all up! 2024-09-14T06:28:54.192 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-14T06:28:54.477 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-14T06:28:54.477 INFO:teuthology.orchestra.run.smithi094.stdout:{"epoch":22,"fsid":"200ad3da-bd63-4e04-af74-efb5ca0af75e","created":"2024-09-14T06:28:30.782046+0000","modified":"2024-09-14T06:28:53.222080+0000","last_up_change":"2024-09-14T06:28:42.827239+0000","last_in_change":"2024-09-14T06:28:33.724176+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-14T06:28:43.295990+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2024-09-14T06:28:49.451714+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":2,"snap_epoch":22,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"8b22a249-b842-406a-a02c-c0f31f57b635","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6824","nonce":1809564923},{"type":"v1","addr":"172.21.15.94:6825","nonce":1809564923}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6826","nonce":1809564923},{"type":"v1","addr":"172.21.15.94:6827","nonce":1809564923}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6830","nonce":1809564923},{"type":"v1","addr":"172.21.15.94:6831","nonce":1809564923}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6828","nonce":1809564923},{"type":"v1","addr":"172.21.15.94:6829","nonce":1809564923}]},"public_addr":"172.21.15.94:6825/1809564923","cluster_addr":"172.21.15.94:6827/1809564923","heartbeat_back_addr":"172.21.15.94:6831/1809564923","heartbeat_front_addr":"172.21.15.94:6829/1809564923","state":["exists","up"]},{"osd":1,"uuid":"02251a03-84a0-414d-a980-fac4fd043cb6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6816","nonce":736402652},{"type":"v1","addr":"172.21.15.94:6817","nonce":736402652}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6818","nonce":736402652},{"type":"v1","addr":"172.21.15.94:6819","nonce":736402652}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6822","nonce":736402652},{"type":"v1","addr":"172.21.15.94:6823","nonce":736402652}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6820","nonce":736402652},{"type":"v1","addr":"172.21.15.94:6821","nonce":736402652}]},"public_addr":"172.21.15.94:6817/736402652","cluster_addr":"172.21.15.94:6819/736402652","heartbeat_back_addr":"172.21.15.94:6823/736402652","heartbeat_front_addr":"172.21.15.94:6821/736402652","state":["exists","up"]},{"osd":2,"uuid":"d45bf470-bb29-4739-8294-cd122b2b4102","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6808","nonce":1063204782},{"type":"v1","addr":"172.21.15.94:6809","nonce":1063204782}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6810","nonce":1063204782},{"type":"v1","addr":"172.21.15.94:6811","nonce":1063204782}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6814","nonce":1063204782},{"type":"v1","addr":"172.21.15.94:6815","nonce":1063204782}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6812","nonce":1063204782},{"type":"v1","addr":"172.21.15.94:6813","nonce":1063204782}]},"public_addr":"172.21.15.94:6809/1063204782","cluster_addr":"172.21.15.94:6811/1063204782","heartbeat_back_addr":"172.21.15.94:6815/1063204782","heartbeat_front_addr":"172.21.15.94:6813/1063204782","state":["exists","up"]},{"osd":3,"uuid":"456ec2d9-c438-4243-b931-335704f661eb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6800","nonce":3382046754},{"type":"v1","addr":"172.21.15.94:6801","nonce":3382046754}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6802","nonce":3382046754},{"type":"v1","addr":"172.21.15.94:6803","nonce":3382046754}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6806","nonce":3382046754},{"type":"v1","addr":"172.21.15.94:6807","nonce":3382046754}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6804","nonce":3382046754},{"type":"v1","addr":"172.21.15.94:6805","nonce":3382046754}]},"public_addr":"172.21.15.94:6801/3382046754","cluster_addr":"172.21.15.94:6803/3382046754","heartbeat_back_addr":"172.21.15.94:6807/3382046754","heartbeat_front_addr":"172.21.15.94:6805/3382046754","state":["exists","up"]},{"osd":4,"uuid":"2dab948e-c6e4-4437-bf17-017d1ac6821c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6810","nonce":501726729},{"type":"v1","addr":"172.21.15.138:6811","nonce":501726729}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6812","nonce":501726729},{"type":"v1","addr":"172.21.15.138:6813","nonce":501726729}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6816","nonce":501726729},{"type":"v1","addr":"172.21.15.138:6817","nonce":501726729}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6814","nonce":501726729},{"type":"v1","addr":"172.21.15.138:6815","nonce":501726729}]},"public_addr":"172.21.15.138:6811/501726729","cluster_addr":"172.21.15.138:6813/501726729","heartbeat_back_addr":"172.21.15.138:6817/501726729","heartbeat_front_addr":"172.21.15.138:6815/501726729","state":["exists","up"]},{"osd":5,"uuid":"8caf3aa3-ad87-4d33-b82e-4c15195176fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6826","nonce":1599292735},{"type":"v1","addr":"172.21.15.138:6827","nonce":1599292735}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6828","nonce":1599292735},{"type":"v1","addr":"172.21.15.138:6829","nonce":1599292735}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6832","nonce":1599292735},{"type":"v1","addr":"172.21.15.138:6833","nonce":1599292735}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6830","nonce":1599292735},{"type":"v1","addr":"172.21.15.138:6831","nonce":1599292735}]},"public_addr":"172.21.15.138:6827/1599292735","cluster_addr":"172.21.15.138:6829/1599292735","heartbeat_back_addr":"172.21.15.138:6833/1599292735","heartbeat_front_addr":"172.21.15.138:6831/1599292735","state":["exists","up"]},{"osd":6,"uuid":"e55d11fd-433d-4462-93a7-d5e84560221b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6818","nonce":163531479},{"type":"v1","addr":"172.21.15.138:6819","nonce":163531479}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6820","nonce":163531479},{"type":"v1","addr":"172.21.15.138:6821","nonce":163531479}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6824","nonce":163531479},{"type":"v1","addr":"172.21.15.138:6825","nonce":163531479}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6822","nonce":163531479},{"type":"v1","addr":"172.21.15.138:6823","nonce":163531479}]},"public_addr":"172.21.15.138:6819/163531479","cluster_addr":"172.21.15.138:6821/163531479","heartbeat_back_addr":"172.21.15.138:6825/163531479","heartbeat_front_addr":"172.21.15.138:6823/163531479","state":["exists","up"]},{"osd":7,"uuid":"ba14ba69-2c6c-4d09-81e2-2593dc3a7ea9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6800","nonce":218507235},{"type":"v1","addr":"172.21.15.138:6801","nonce":218507235}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6802","nonce":218507235},{"type":"v1","addr":"172.21.15.138:6803","nonce":218507235}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6806","nonce":218507235},{"type":"v1","addr":"172.21.15.138:6807","nonce":218507235}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6804","nonce":218507235},{"type":"v1","addr":"172.21.15.138:6805","nonce":218507235}]},"public_addr":"172.21.15.138:6801/218507235","cluster_addr":"172.21.15.138:6803/218507235","heartbeat_back_addr":"172.21.15.138:6807/218507235","heartbeat_front_addr":"172.21.15.138:6805/218507235","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-14T06:28:39.906639+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-14T06:28:40.541279+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-14T06:28:40.025350+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-14T06:28:39.217944+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-14T06:28:40.185898+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-14T06:28:40.783504+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-14T06:28:40.527305+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-14T06:28:40.349676+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":2,"snaps":[{"begin":2,"length":1}]}],"new_removed_snaps":[{"pool":2,"snaps":[{"begin":2,"length":1}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-14T06:28:54.490 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2024-09-14T06:28:54.490 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2024-09-14T06:28:54.491 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2024-09-14T06:28:54.491 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 flush_pg_stats 2024-09-14T06:28:54.492 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 flush_pg_stats 2024-09-14T06:28:54.492 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 flush_pg_stats 2024-09-14T06:28:54.492 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 flush_pg_stats 2024-09-14T06:28:54.493 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 flush_pg_stats 2024-09-14T06:28:54.728 INFO:teuthology.orchestra.run.smithi094.stdout:64424509445 2024-09-14T06:28:54.729 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2024-09-14T06:28:54.730 INFO:teuthology.orchestra.run.smithi094.stdout:60129542148 2024-09-14T06:28:54.731 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-09-14T06:28:54.734 INFO:teuthology.orchestra.run.smithi094.stdout:64424509446 2024-09-14T06:28:54.734 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-09-14T06:28:54.735 INFO:teuthology.orchestra.run.smithi094.stdout:64424509446 2024-09-14T06:28:54.735 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-09-14T06:28:54.736 INFO:teuthology.orchestra.run.smithi094.stdout:55834574853 2024-09-14T06:28:54.736 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-09-14T06:28:54.737 INFO:teuthology.orchestra.run.smithi094.stdout:64424509445 2024-09-14T06:28:54.737 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-09-14T06:28:54.738 INFO:teuthology.orchestra.run.smithi094.stdout:60129542149 2024-09-14T06:28:54.739 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-09-14T06:28:54.740 INFO:teuthology.orchestra.run.smithi094.stdout:64424509444 2024-09-14T06:28:54.740 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2024-09-14T06:28:55.297 INFO:teuthology.orchestra.run.smithi094.stdout:60129542147 2024-09-14T06:28:55.298 INFO:teuthology.orchestra.run.smithi094.stdout:64424509443 2024-09-14T06:28:55.320 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509445 got 64424509443 for osd.7 2024-09-14T06:28:55.321 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542148 got 60129542147 for osd.0 2024-09-14T06:28:55.344 INFO:teuthology.orchestra.run.smithi094.stdout:64424509444 2024-09-14T06:28:55.349 INFO:teuthology.orchestra.run.smithi094.stdout:64424509443 2024-09-14T06:28:55.359 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509446 got 64424509444 for osd.5 2024-09-14T06:28:55.363 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509445 got 64424509443 for osd.4 2024-09-14T06:28:55.387 INFO:teuthology.orchestra.run.smithi094.stdout:60129542148 2024-09-14T06:28:55.397 INFO:teuthology.orchestra.run.smithi094.stdout:64424509444 2024-09-14T06:28:55.397 INFO:teuthology.orchestra.run.smithi094.stdout:64424509442 2024-09-14T06:28:55.400 INFO:teuthology.orchestra.run.smithi094.stdout:55834574852 2024-09-14T06:28:55.404 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542149 got 60129542148 for osd.2 2024-09-14T06:28:55.412 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509446 got 64424509444 for osd.1 2024-09-14T06:28:55.417 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509444 got 64424509442 for osd.6 2024-09-14T06:28:55.418 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574853 got 55834574852 for osd.3 2024-09-14T06:28:56.321 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2024-09-14T06:28:56.322 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-09-14T06:28:56.360 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-09-14T06:28:56.364 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-09-14T06:28:56.405 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-09-14T06:28:56.412 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-09-14T06:28:56.418 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2024-09-14T06:28:56.419 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-09-14T06:28:56.853 INFO:teuthology.orchestra.run.smithi094.stdout:64424509445 2024-09-14T06:28:56.853 INFO:teuthology.orchestra.run.smithi094.stdout:60129542148 2024-09-14T06:28:56.871 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509445 got 64424509445 for osd.7 2024-09-14T06:28:56.871 DEBUG:teuthology.parallel:result is None 2024-09-14T06:28:56.874 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542148 got 60129542148 for osd.0 2024-09-14T06:28:56.875 DEBUG:teuthology.parallel:result is None 2024-09-14T06:28:56.894 INFO:teuthology.orchestra.run.smithi094.stdout:64424509445 2024-09-14T06:28:56.897 INFO:teuthology.orchestra.run.smithi094.stdout:64424509446 2024-09-14T06:28:56.908 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509445 got 64424509445 for osd.4 2024-09-14T06:28:56.908 DEBUG:teuthology.parallel:result is None 2024-09-14T06:28:56.917 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509446 got 64424509446 for osd.5 2024-09-14T06:28:56.917 DEBUG:teuthology.parallel:result is None 2024-09-14T06:28:56.933 INFO:teuthology.orchestra.run.smithi094.stdout:60129542149 2024-09-14T06:28:56.947 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542149 got 60129542149 for osd.2 2024-09-14T06:28:56.947 DEBUG:teuthology.parallel:result is None 2024-09-14T06:28:56.984 INFO:teuthology.orchestra.run.smithi094.stdout:64424509446 2024-09-14T06:28:56.994 INFO:teuthology.orchestra.run.smithi094.stdout:55834574853 2024-09-14T06:28:56.994 INFO:teuthology.orchestra.run.smithi094.stdout:64424509444 2024-09-14T06:28:56.997 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509446 got 64424509446 for osd.1 2024-09-14T06:28:56.997 DEBUG:teuthology.parallel:result is None 2024-09-14T06:28:57.011 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574853 got 55834574853 for osd.3 2024-09-14T06:28:57.012 DEBUG:teuthology.parallel:result is None 2024-09-14T06:28:57.012 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509444 got 64424509444 for osd.6 2024-09-14T06:28:57.012 DEBUG:teuthology.parallel:result is None 2024-09-14T06:28:57.013 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-14T06:28:57.013 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-14T06:28:57.335 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-14T06:28:57.335 INFO:teuthology.orchestra.run.smithi094.stderr:dumped all 2024-09-14T06:28:57.347 INFO:teuthology.orchestra.run.smithi094.stdout:{"pg_ready":true,"pg_map":{"version":24,"stamp":"2024-09-14T06:28:57.281756+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":103,"num_write_kb":1630,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":75,"ondisk_log_size":75,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":7,"kb":754974720,"kb_used":234764,"kb_used_data":1864,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754739956,"statfs":{"total":773094113280,"available":772853714944,"internally_reserved":0,"allocated":1908736,"data_stored":1781462,"data_compressed":65166,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":12751,"internal_metadata":238276145},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.070051"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":20,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-14T06:28:53.625193+0000","last_change":"2024-09-14T06:28:53.625193+0000","last_active":"2024-09-14T06:28:53.625193+0000","last_peered":"2024-09-14T06:28:53.625193+0000","last_clean":"2024-09-14T06:28:53.625193+0000","last_became_active":"2024-09-14T06:28:51.223535+0000","last_became_peered":"2024-09-14T06:28:51.223535+0000","last_unstale":"2024-09-14T06:28:53.625193+0000","last_undegraded":"2024-09-14T06:28:53.625193+0000","last_fullsized":"2024-09-14T06:28:53.625193+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:28:50.205350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:28:50.205350+0000","last_clean_scrub_stamp":"2024-09-14T06:28:50.205350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T18:14:22.822245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00052660300000000005,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.6","version":"0'0","reported_seq":20,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-14T06:28:53.226122+0000","last_change":"2024-09-14T06:28:53.226122+0000","last_active":"2024-09-14T06:28:53.226122+0000","last_peered":"2024-09-14T06:28:53.226122+0000","last_clean":"2024-09-14T06:28:53.226122+0000","last_became_active":"2024-09-14T06:28:51.217205+0000","last_became_peered":"2024-09-14T06:28:51.217205+0000","last_unstale":"2024-09-14T06:28:53.226122+0000","last_undegraded":"2024-09-14T06:28:53.226122+0000","last_fullsized":"2024-09-14T06:28:53.226122+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:28:50.205350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:28:50.205350+0000","last_clean_scrub_stamp":"2024-09-14T06:28:50.205350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T15:29:19.563302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065018200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.5","version":"0'0","reported_seq":20,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-14T06:28:53.640035+0000","last_change":"2024-09-14T06:28:53.640035+0000","last_active":"2024-09-14T06:28:53.640035+0000","last_peered":"2024-09-14T06:28:53.640035+0000","last_clean":"2024-09-14T06:28:53.640035+0000","last_became_active":"2024-09-14T06:28:51.308058+0000","last_became_peered":"2024-09-14T06:28:51.308058+0000","last_unstale":"2024-09-14T06:28:53.640035+0000","last_undegraded":"2024-09-14T06:28:53.640035+0000","last_fullsized":"2024-09-14T06:28:53.640035+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:28:50.205350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:28:50.205350+0000","last_clean_scrub_stamp":"2024-09-14T06:28:50.205350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T13:35:48.544057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00037824000000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.4","version":"0'0","reported_seq":20,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-14T06:28:53.226072+0000","last_change":"2024-09-14T06:28:53.226072+0000","last_active":"2024-09-14T06:28:53.226072+0000","last_peered":"2024-09-14T06:28:53.226072+0000","last_clean":"2024-09-14T06:28:53.226072+0000","last_became_active":"2024-09-14T06:28:51.308072+0000","last_became_peered":"2024-09-14T06:28:51.308072+0000","last_unstale":"2024-09-14T06:28:53.226072+0000","last_undegraded":"2024-09-14T06:28:53.226072+0000","last_fullsized":"2024-09-14T06:28:53.226072+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:28:50.205350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:28:50.205350+0000","last_clean_scrub_stamp":"2024-09-14T06:28:50.205350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T13:15:09.052328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066197399999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.2","version":"22'2","reported_seq":22,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-14T06:28:53.235878+0000","last_change":"2024-09-14T06:28:53.231428+0000","last_active":"2024-09-14T06:28:53.235878+0000","last_peered":"2024-09-14T06:28:53.235878+0000","last_clean":"2024-09-14T06:28:53.235878+0000","last_became_active":"2024-09-14T06:28:51.222937+0000","last_became_peered":"2024-09-14T06:28:51.222937+0000","last_unstale":"2024-09-14T06:28:53.235878+0000","last_undegraded":"2024-09-14T06:28:53.235878+0000","last_fullsized":"2024-09-14T06:28:53.235878+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:28:50.205350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:28:50.205350+0000","last_clean_scrub_stamp":"2024-09-14T06:28:50.205350+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T14:27:37.233976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010072970000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.1","version":"0'0","reported_seq":20,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-14T06:28:53.225992+0000","last_change":"2024-09-14T06:28:53.226264+0000","last_active":"2024-09-14T06:28:53.225992+0000","last_peered":"2024-09-14T06:28:53.225992+0000","last_clean":"2024-09-14T06:28:53.225992+0000","last_became_active":"2024-09-14T06:28:51.223252+0000","last_became_peered":"2024-09-14T06:28:51.223252+0000","last_unstale":"2024-09-14T06:28:53.225992+0000","last_undegraded":"2024-09-14T06:28:53.225992+0000","last_fullsized":"2024-09-14T06:28:53.225992+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:28:50.205350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:28:50.205350+0000","last_clean_scrub_stamp":"2024-09-14T06:28:50.205350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T11:56:04.373414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053403100000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.0","version":"0'0","reported_seq":20,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-14T06:28:53.640096+0000","last_change":"2024-09-14T06:28:53.640096+0000","last_active":"2024-09-14T06:28:53.640096+0000","last_peered":"2024-09-14T06:28:53.640096+0000","last_clean":"2024-09-14T06:28:53.640096+0000","last_became_active":"2024-09-14T06:28:51.223069+0000","last_became_peered":"2024-09-14T06:28:51.223069+0000","last_unstale":"2024-09-14T06:28:53.640096+0000","last_undegraded":"2024-09-14T06:28:53.640096+0000","last_fullsized":"2024-09-14T06:28:53.640096+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:28:50.205350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:28:50.205350+0000","last_clean_scrub_stamp":"2024-09-14T06:28:50.205350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T15:13:00.471509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00043714600000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.3","version":"20'1","reported_seq":21,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-14T06:28:53.230933+0000","last_change":"2024-09-14T06:28:53.230933+0000","last_active":"2024-09-14T06:28:53.230933+0000","last_peered":"2024-09-14T06:28:53.230933+0000","last_clean":"2024-09-14T06:28:53.230933+0000","last_became_active":"2024-09-14T06:28:51.222874+0000","last_became_peered":"2024-09-14T06:28:51.222874+0000","last_unstale":"2024-09-14T06:28:53.230933+0000","last_undegraded":"2024-09-14T06:28:53.230933+0000","last_fullsized":"2024-09-14T06:28:53.230933+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:28:50.205350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:28:50.205350+0000","last_clean_scrub_stamp":"2024-09-14T06:28:50.205350+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T12:04:25.190171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050808899999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.0","version":"17'72","reported_seq":111,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-14T06:28:53.639580+0000","last_change":"2024-09-14T06:28:44.839885+0000","last_active":"2024-09-14T06:28:53.639580+0000","last_peered":"2024-09-14T06:28:53.639580+0000","last_clean":"2024-09-14T06:28:53.639580+0000","last_became_active":"2024-09-14T06:28:44.839573+0000","last_became_peered":"2024-09-14T06:28:44.839573+0000","last_unstale":"2024-09-14T06:28:53.639580+0000","last_undegraded":"2024-09-14T06:28:53.639580+0000","last_fullsized":"2024-09-14T06:28:53.639580+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:28:43.828494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:28:43.828494+0000","last_clean_scrub_stamp":"2024-09-14T06:28:43.828494+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T14:41:25.787293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":131072,"data_stored":1180736,"data_compressed":65166,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509445,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27308,"kb_used_data":280,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344532,"statfs":{"total":96636764160,"available":96608800768,"internally_reserved":0,"allocated":286720,"data_stored":665454,"data_compressed":32583,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509444,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27244,"kb_used_data":216,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344596,"statfs":{"total":96636764160,"available":96608866304,"internally_reserved":0,"allocated":221184,"data_stored":75086,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509446,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27256,"kb_used_data":220,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344584,"statfs":{"total":96636764160,"available":96608854016,"internally_reserved":0,"allocated":225280,"data_stored":75105,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509445,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27252,"kb_used_data":216,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344588,"statfs":{"total":96636764160,"available":96608858112,"internally_reserved":0,"allocated":221184,"data_stored":75086,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509446,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31416,"kb_used_data":220,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340424,"statfs":{"total":96636764160,"available":96604594176,"internally_reserved":0,"allocated":225280,"data_stored":75105,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":14,"seq":60129542149,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31408,"kb_used_data":216,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340432,"statfs":{"total":96636764160,"available":96604602368,"internally_reserved":0,"allocated":221184,"data_stored":75086,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542148,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31472,"kb_used_data":280,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340368,"statfs":{"total":96636764160,"available":96604536832,"internally_reserved":0,"allocated":286720,"data_stored":665454,"data_compressed":32583,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574854,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31408,"kb_used_data":216,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340432,"statfs":{"total":96636764160,"available":96604602368,"internally_reserved":0,"allocated":221184,"data_stored":75086,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32583,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32583,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-14T06:28:57.349 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-14T06:28:57.627 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-14T06:28:57.628 INFO:teuthology.orchestra.run.smithi094.stderr:dumped all 2024-09-14T06:28:57.639 INFO:teuthology.orchestra.run.smithi094.stdout:{"pg_ready":true,"pg_map":{"version":24,"stamp":"2024-09-14T06:28:57.281756+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":103,"num_write_kb":1630,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":75,"ondisk_log_size":75,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":7,"kb":754974720,"kb_used":234764,"kb_used_data":1864,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754739956,"statfs":{"total":773094113280,"available":772853714944,"internally_reserved":0,"allocated":1908736,"data_stored":1781462,"data_compressed":65166,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":12751,"internal_metadata":238276145},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.070051"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":20,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-14T06:28:53.625193+0000","last_change":"2024-09-14T06:28:53.625193+0000","last_active":"2024-09-14T06:28:53.625193+0000","last_peered":"2024-09-14T06:28:53.625193+0000","last_clean":"2024-09-14T06:28:53.625193+0000","last_became_active":"2024-09-14T06:28:51.223535+0000","last_became_peered":"2024-09-14T06:28:51.223535+0000","last_unstale":"2024-09-14T06:28:53.625193+0000","last_undegraded":"2024-09-14T06:28:53.625193+0000","last_fullsized":"2024-09-14T06:28:53.625193+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:28:50.205350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:28:50.205350+0000","last_clean_scrub_stamp":"2024-09-14T06:28:50.205350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T18:14:22.822245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00052660300000000005,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.6","version":"0'0","reported_seq":20,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-14T06:28:53.226122+0000","last_change":"2024-09-14T06:28:53.226122+0000","last_active":"2024-09-14T06:28:53.226122+0000","last_peered":"2024-09-14T06:28:53.226122+0000","last_clean":"2024-09-14T06:28:53.226122+0000","last_became_active":"2024-09-14T06:28:51.217205+0000","last_became_peered":"2024-09-14T06:28:51.217205+0000","last_unstale":"2024-09-14T06:28:53.226122+0000","last_undegraded":"2024-09-14T06:28:53.226122+0000","last_fullsized":"2024-09-14T06:28:53.226122+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:28:50.205350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:28:50.205350+0000","last_clean_scrub_stamp":"2024-09-14T06:28:50.205350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T15:29:19.563302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065018200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.5","version":"0'0","reported_seq":20,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-14T06:28:53.640035+0000","last_change":"2024-09-14T06:28:53.640035+0000","last_active":"2024-09-14T06:28:53.640035+0000","last_peered":"2024-09-14T06:28:53.640035+0000","last_clean":"2024-09-14T06:28:53.640035+0000","last_became_active":"2024-09-14T06:28:51.308058+0000","last_became_peered":"2024-09-14T06:28:51.308058+0000","last_unstale":"2024-09-14T06:28:53.640035+0000","last_undegraded":"2024-09-14T06:28:53.640035+0000","last_fullsized":"2024-09-14T06:28:53.640035+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:28:50.205350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:28:50.205350+0000","last_clean_scrub_stamp":"2024-09-14T06:28:50.205350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T13:35:48.544057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00037824000000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.4","version":"0'0","reported_seq":20,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-14T06:28:53.226072+0000","last_change":"2024-09-14T06:28:53.226072+0000","last_active":"2024-09-14T06:28:53.226072+0000","last_peered":"2024-09-14T06:28:53.226072+0000","last_clean":"2024-09-14T06:28:53.226072+0000","last_became_active":"2024-09-14T06:28:51.308072+0000","last_became_peered":"2024-09-14T06:28:51.308072+0000","last_unstale":"2024-09-14T06:28:53.226072+0000","last_undegraded":"2024-09-14T06:28:53.226072+0000","last_fullsized":"2024-09-14T06:28:53.226072+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:28:50.205350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:28:50.205350+0000","last_clean_scrub_stamp":"2024-09-14T06:28:50.205350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T13:15:09.052328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066197399999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.2","version":"22'2","reported_seq":22,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-14T06:28:53.235878+0000","last_change":"2024-09-14T06:28:53.231428+0000","last_active":"2024-09-14T06:28:53.235878+0000","last_peered":"2024-09-14T06:28:53.235878+0000","last_clean":"2024-09-14T06:28:53.235878+0000","last_became_active":"2024-09-14T06:28:51.222937+0000","last_became_peered":"2024-09-14T06:28:51.222937+0000","last_unstale":"2024-09-14T06:28:53.235878+0000","last_undegraded":"2024-09-14T06:28:53.235878+0000","last_fullsized":"2024-09-14T06:28:53.235878+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:28:50.205350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:28:50.205350+0000","last_clean_scrub_stamp":"2024-09-14T06:28:50.205350+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T14:27:37.233976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010072970000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.1","version":"0'0","reported_seq":20,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-14T06:28:53.225992+0000","last_change":"2024-09-14T06:28:53.226264+0000","last_active":"2024-09-14T06:28:53.225992+0000","last_peered":"2024-09-14T06:28:53.225992+0000","last_clean":"2024-09-14T06:28:53.225992+0000","last_became_active":"2024-09-14T06:28:51.223252+0000","last_became_peered":"2024-09-14T06:28:51.223252+0000","last_unstale":"2024-09-14T06:28:53.225992+0000","last_undegraded":"2024-09-14T06:28:53.225992+0000","last_fullsized":"2024-09-14T06:28:53.225992+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:28:50.205350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:28:50.205350+0000","last_clean_scrub_stamp":"2024-09-14T06:28:50.205350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T11:56:04.373414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053403100000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.0","version":"0'0","reported_seq":20,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-14T06:28:53.640096+0000","last_change":"2024-09-14T06:28:53.640096+0000","last_active":"2024-09-14T06:28:53.640096+0000","last_peered":"2024-09-14T06:28:53.640096+0000","last_clean":"2024-09-14T06:28:53.640096+0000","last_became_active":"2024-09-14T06:28:51.223069+0000","last_became_peered":"2024-09-14T06:28:51.223069+0000","last_unstale":"2024-09-14T06:28:53.640096+0000","last_undegraded":"2024-09-14T06:28:53.640096+0000","last_fullsized":"2024-09-14T06:28:53.640096+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:28:50.205350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:28:50.205350+0000","last_clean_scrub_stamp":"2024-09-14T06:28:50.205350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T15:13:00.471509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00043714600000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.3","version":"20'1","reported_seq":21,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-14T06:28:53.230933+0000","last_change":"2024-09-14T06:28:53.230933+0000","last_active":"2024-09-14T06:28:53.230933+0000","last_peered":"2024-09-14T06:28:53.230933+0000","last_clean":"2024-09-14T06:28:53.230933+0000","last_became_active":"2024-09-14T06:28:51.222874+0000","last_became_peered":"2024-09-14T06:28:51.222874+0000","last_unstale":"2024-09-14T06:28:53.230933+0000","last_undegraded":"2024-09-14T06:28:53.230933+0000","last_fullsized":"2024-09-14T06:28:53.230933+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:28:50.205350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:28:50.205350+0000","last_clean_scrub_stamp":"2024-09-14T06:28:50.205350+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T12:04:25.190171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050808899999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.0","version":"17'72","reported_seq":111,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-14T06:28:53.639580+0000","last_change":"2024-09-14T06:28:44.839885+0000","last_active":"2024-09-14T06:28:53.639580+0000","last_peered":"2024-09-14T06:28:53.639580+0000","last_clean":"2024-09-14T06:28:53.639580+0000","last_became_active":"2024-09-14T06:28:44.839573+0000","last_became_peered":"2024-09-14T06:28:44.839573+0000","last_unstale":"2024-09-14T06:28:53.639580+0000","last_undegraded":"2024-09-14T06:28:53.639580+0000","last_fullsized":"2024-09-14T06:28:53.639580+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:28:43.828494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:28:43.828494+0000","last_clean_scrub_stamp":"2024-09-14T06:28:43.828494+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T14:41:25.787293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":131072,"data_stored":1180736,"data_compressed":65166,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509445,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27308,"kb_used_data":280,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344532,"statfs":{"total":96636764160,"available":96608800768,"internally_reserved":0,"allocated":286720,"data_stored":665454,"data_compressed":32583,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509444,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27244,"kb_used_data":216,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344596,"statfs":{"total":96636764160,"available":96608866304,"internally_reserved":0,"allocated":221184,"data_stored":75086,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509446,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27256,"kb_used_data":220,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344584,"statfs":{"total":96636764160,"available":96608854016,"internally_reserved":0,"allocated":225280,"data_stored":75105,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509445,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27252,"kb_used_data":216,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344588,"statfs":{"total":96636764160,"available":96608858112,"internally_reserved":0,"allocated":221184,"data_stored":75086,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509446,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31416,"kb_used_data":220,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340424,"statfs":{"total":96636764160,"available":96604594176,"internally_reserved":0,"allocated":225280,"data_stored":75105,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":14,"seq":60129542149,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31408,"kb_used_data":216,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340432,"statfs":{"total":96636764160,"available":96604602368,"internally_reserved":0,"allocated":221184,"data_stored":75086,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542148,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31472,"kb_used_data":280,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340368,"statfs":{"total":96636764160,"available":96604536832,"internally_reserved":0,"allocated":286720,"data_stored":665454,"data_compressed":32583,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574854,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31408,"kb_used_data":216,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340432,"statfs":{"total":96636764160,"available":96604602368,"internally_reserved":0,"allocated":221184,"data_stored":75086,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32583,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32583,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-14T06:28:57.640 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-14T06:28:57.641 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-09-14T06:28:57.641 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy 2024-09-14T06:28:57.641 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-09-14T06:28:57.959 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-14T06:28:57.959 INFO:teuthology.orchestra.run.smithi094.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-09-14T06:28:57.972 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy done 2024-09-14T06:28:57.972 INFO:teuthology.run_tasks:Running task thrashosds... 2024-09-14T06:28:57.983 INFO:tasks.thrashosds:config is {'timeout': 1200, 'sighup_delay': 0.1, 'optrack_toggle_delay': 2.0, 'dump_ops_enable': 'true', 'noscrub_toggle_delay': 2.0, 'random_eio': 0.0} 2024-09-14T06:28:57.984 INFO:tasks.thrashosds:overrides is {'admin_socket': {'branch': 'main'}, 'ceph': {'conf': {'global': {'mon client directed command retry': 5, 'mon warn on pool no app': False, 'ms inject socket failures': 5000}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20}, 'osd': {'bluestore block size': 96636764160, 'bluestore compression algorithm': 'snappy', 'bluestore compression mode': 'aggressive', 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd blocked scrub grace period': 3600, 'osd failsafe full ratio': 0.95, 'osd max markdown count': 1000, 'osd objectstore': 'bluestore', 'osd shutdown pgref assert': True}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(OSD_SLOW_PING_TIME', 'but it is still running', 'objects unfound and apparently lost', 'overall HEALTH_', '\\(OSDMAP_FLAGS\\)', '\\(OSD_', '\\(PG_', '\\(POOL_', '\\(CACHE_POOL_', '\\(SMALLER_PGP_NUM\\)', '\\(OBJECT_', 'SLOW_OPS', '\\(REQUEST_SLOW\\)', '\\(TOO_FEW_PGS\\)', 'slow request', 'timeout on replica', 'late reservation from', 'MON_DOWN', 'OSDMAP_FLAGS', 'OSD_DOWN', 'PG_DEGRADED', 'PG_AVAILABILITY', 'POOL_APP_NOT_ENABLED', 'mons down', 'mon down', 'out of quorum', 'noscrub', 'nodeep-scrub'], 'sha1': '7f24e589328ae9e639c0009d93340e5b235d0a47'}, 'ceph-deploy': {'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {}}}, 'install': {'ceph': {'flavor': 'default', 'sha1': '7f24e589328ae9e639c0009d93340e5b235d0a47'}}, 'thrashosds': {'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5}, 'workunit': {'branch': 'main', 'sha1': '0d61375b8ab53be4c233f5b93e463edfdff9316a'}} 2024-09-14T06:28:57.984 INFO:tasks.thrashosds:config is {'timeout': 1200, 'sighup_delay': 0.1, 'optrack_toggle_delay': 2.0, 'dump_ops_enable': 'true', 'noscrub_toggle_delay': 2.0, 'random_eio': 0.0, 'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5} 2024-09-14T06:28:57.984 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr_debug_aggressive_pg_num_changes true 2024-09-14T06:28:58.290 INFO:tasks.thrashosds:Beginning thrashosds... 2024-09-14T06:28:58.290 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-14T06:28:58.290 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-14T06:28:58.568 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-14T06:28:58.569 INFO:teuthology.orchestra.run.smithi094.stderr:dumped all 2024-09-14T06:28:58.580 INFO:teuthology.orchestra.run.smithi094.stdout:{"pg_ready":true,"pg_map":{"version":24,"stamp":"2024-09-14T06:28:57.281756+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":103,"num_write_kb":1630,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":75,"ondisk_log_size":75,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":7,"kb":754974720,"kb_used":234764,"kb_used_data":1864,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754739956,"statfs":{"total":773094113280,"available":772853714944,"internally_reserved":0,"allocated":1908736,"data_stored":1781462,"data_compressed":65166,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":12751,"internal_metadata":238276145},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.070051"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":20,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-14T06:28:53.625193+0000","last_change":"2024-09-14T06:28:53.625193+0000","last_active":"2024-09-14T06:28:53.625193+0000","last_peered":"2024-09-14T06:28:53.625193+0000","last_clean":"2024-09-14T06:28:53.625193+0000","last_became_active":"2024-09-14T06:28:51.223535+0000","last_became_peered":"2024-09-14T06:28:51.223535+0000","last_unstale":"2024-09-14T06:28:53.625193+0000","last_undegraded":"2024-09-14T06:28:53.625193+0000","last_fullsized":"2024-09-14T06:28:53.625193+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:28:50.205350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:28:50.205350+0000","last_clean_scrub_stamp":"2024-09-14T06:28:50.205350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T18:14:22.822245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00052660300000000005,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.6","version":"0'0","reported_seq":20,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-14T06:28:53.226122+0000","last_change":"2024-09-14T06:28:53.226122+0000","last_active":"2024-09-14T06:28:53.226122+0000","last_peered":"2024-09-14T06:28:53.226122+0000","last_clean":"2024-09-14T06:28:53.226122+0000","last_became_active":"2024-09-14T06:28:51.217205+0000","last_became_peered":"2024-09-14T06:28:51.217205+0000","last_unstale":"2024-09-14T06:28:53.226122+0000","last_undegraded":"2024-09-14T06:28:53.226122+0000","last_fullsized":"2024-09-14T06:28:53.226122+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:28:50.205350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:28:50.205350+0000","last_clean_scrub_stamp":"2024-09-14T06:28:50.205350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T15:29:19.563302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065018200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.5","version":"0'0","reported_seq":20,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-14T06:28:53.640035+0000","last_change":"2024-09-14T06:28:53.640035+0000","last_active":"2024-09-14T06:28:53.640035+0000","last_peered":"2024-09-14T06:28:53.640035+0000","last_clean":"2024-09-14T06:28:53.640035+0000","last_became_active":"2024-09-14T06:28:51.308058+0000","last_became_peered":"2024-09-14T06:28:51.308058+0000","last_unstale":"2024-09-14T06:28:53.640035+0000","last_undegraded":"2024-09-14T06:28:53.640035+0000","last_fullsized":"2024-09-14T06:28:53.640035+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:28:50.205350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:28:50.205350+0000","last_clean_scrub_stamp":"2024-09-14T06:28:50.205350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T13:35:48.544057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00037824000000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.4","version":"0'0","reported_seq":20,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-14T06:28:53.226072+0000","last_change":"2024-09-14T06:28:53.226072+0000","last_active":"2024-09-14T06:28:53.226072+0000","last_peered":"2024-09-14T06:28:53.226072+0000","last_clean":"2024-09-14T06:28:53.226072+0000","last_became_active":"2024-09-14T06:28:51.308072+0000","last_became_peered":"2024-09-14T06:28:51.308072+0000","last_unstale":"2024-09-14T06:28:53.226072+0000","last_undegraded":"2024-09-14T06:28:53.226072+0000","last_fullsized":"2024-09-14T06:28:53.226072+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:28:50.205350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:28:50.205350+0000","last_clean_scrub_stamp":"2024-09-14T06:28:50.205350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T13:15:09.052328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066197399999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.2","version":"22'2","reported_seq":22,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-14T06:28:53.235878+0000","last_change":"2024-09-14T06:28:53.231428+0000","last_active":"2024-09-14T06:28:53.235878+0000","last_peered":"2024-09-14T06:28:53.235878+0000","last_clean":"2024-09-14T06:28:53.235878+0000","last_became_active":"2024-09-14T06:28:51.222937+0000","last_became_peered":"2024-09-14T06:28:51.222937+0000","last_unstale":"2024-09-14T06:28:53.235878+0000","last_undegraded":"2024-09-14T06:28:53.235878+0000","last_fullsized":"2024-09-14T06:28:53.235878+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:28:50.205350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:28:50.205350+0000","last_clean_scrub_stamp":"2024-09-14T06:28:50.205350+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T14:27:37.233976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010072970000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.1","version":"0'0","reported_seq":20,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-14T06:28:53.225992+0000","last_change":"2024-09-14T06:28:53.226264+0000","last_active":"2024-09-14T06:28:53.225992+0000","last_peered":"2024-09-14T06:28:53.225992+0000","last_clean":"2024-09-14T06:28:53.225992+0000","last_became_active":"2024-09-14T06:28:51.223252+0000","last_became_peered":"2024-09-14T06:28:51.223252+0000","last_unstale":"2024-09-14T06:28:53.225992+0000","last_undegraded":"2024-09-14T06:28:53.225992+0000","last_fullsized":"2024-09-14T06:28:53.225992+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:28:50.205350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:28:50.205350+0000","last_clean_scrub_stamp":"2024-09-14T06:28:50.205350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T11:56:04.373414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053403100000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.0","version":"0'0","reported_seq":20,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-14T06:28:53.640096+0000","last_change":"2024-09-14T06:28:53.640096+0000","last_active":"2024-09-14T06:28:53.640096+0000","last_peered":"2024-09-14T06:28:53.640096+0000","last_clean":"2024-09-14T06:28:53.640096+0000","last_became_active":"2024-09-14T06:28:51.223069+0000","last_became_peered":"2024-09-14T06:28:51.223069+0000","last_unstale":"2024-09-14T06:28:53.640096+0000","last_undegraded":"2024-09-14T06:28:53.640096+0000","last_fullsized":"2024-09-14T06:28:53.640096+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:28:50.205350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:28:50.205350+0000","last_clean_scrub_stamp":"2024-09-14T06:28:50.205350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T15:13:00.471509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00043714600000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.3","version":"20'1","reported_seq":21,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-14T06:28:53.230933+0000","last_change":"2024-09-14T06:28:53.230933+0000","last_active":"2024-09-14T06:28:53.230933+0000","last_peered":"2024-09-14T06:28:53.230933+0000","last_clean":"2024-09-14T06:28:53.230933+0000","last_became_active":"2024-09-14T06:28:51.222874+0000","last_became_peered":"2024-09-14T06:28:51.222874+0000","last_unstale":"2024-09-14T06:28:53.230933+0000","last_undegraded":"2024-09-14T06:28:53.230933+0000","last_fullsized":"2024-09-14T06:28:53.230933+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:28:50.205350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:28:50.205350+0000","last_clean_scrub_stamp":"2024-09-14T06:28:50.205350+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T12:04:25.190171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050808899999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.0","version":"17'72","reported_seq":111,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-14T06:28:53.639580+0000","last_change":"2024-09-14T06:28:44.839885+0000","last_active":"2024-09-14T06:28:53.639580+0000","last_peered":"2024-09-14T06:28:53.639580+0000","last_clean":"2024-09-14T06:28:53.639580+0000","last_became_active":"2024-09-14T06:28:44.839573+0000","last_became_peered":"2024-09-14T06:28:44.839573+0000","last_unstale":"2024-09-14T06:28:53.639580+0000","last_undegraded":"2024-09-14T06:28:53.639580+0000","last_fullsized":"2024-09-14T06:28:53.639580+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:28:43.828494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:28:43.828494+0000","last_clean_scrub_stamp":"2024-09-14T06:28:43.828494+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T14:41:25.787293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":131072,"data_stored":1180736,"data_compressed":65166,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509445,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27308,"kb_used_data":280,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344532,"statfs":{"total":96636764160,"available":96608800768,"internally_reserved":0,"allocated":286720,"data_stored":665454,"data_compressed":32583,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509444,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27244,"kb_used_data":216,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344596,"statfs":{"total":96636764160,"available":96608866304,"internally_reserved":0,"allocated":221184,"data_stored":75086,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509446,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27256,"kb_used_data":220,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344584,"statfs":{"total":96636764160,"available":96608854016,"internally_reserved":0,"allocated":225280,"data_stored":75105,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509445,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27252,"kb_used_data":216,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344588,"statfs":{"total":96636764160,"available":96608858112,"internally_reserved":0,"allocated":221184,"data_stored":75086,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509446,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31416,"kb_used_data":220,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340424,"statfs":{"total":96636764160,"available":96604594176,"internally_reserved":0,"allocated":225280,"data_stored":75105,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":14,"seq":60129542149,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31408,"kb_used_data":216,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340432,"statfs":{"total":96636764160,"available":96604602368,"internally_reserved":0,"allocated":221184,"data_stored":75086,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542148,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31472,"kb_used_data":280,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340368,"statfs":{"total":96636764160,"available":96604536832,"internally_reserved":0,"allocated":286720,"data_stored":665454,"data_compressed":32583,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574854,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31408,"kb_used_data":216,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340432,"statfs":{"total":96636764160,"available":96604602368,"internally_reserved":0,"allocated":221184,"data_stored":75086,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32583,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32583,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-14T06:28:58.581 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-14T06:28:58.860 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-14T06:28:58.860 INFO:teuthology.orchestra.run.smithi094.stderr:dumped all 2024-09-14T06:28:58.872 INFO:teuthology.orchestra.run.smithi094.stdout:{"pg_ready":true,"pg_map":{"version":24,"stamp":"2024-09-14T06:28:57.281756+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":103,"num_write_kb":1630,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":75,"ondisk_log_size":75,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":7,"kb":754974720,"kb_used":234764,"kb_used_data":1864,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754739956,"statfs":{"total":773094113280,"available":772853714944,"internally_reserved":0,"allocated":1908736,"data_stored":1781462,"data_compressed":65166,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":12751,"internal_metadata":238276145},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.070051"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":20,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-14T06:28:53.625193+0000","last_change":"2024-09-14T06:28:53.625193+0000","last_active":"2024-09-14T06:28:53.625193+0000","last_peered":"2024-09-14T06:28:53.625193+0000","last_clean":"2024-09-14T06:28:53.625193+0000","last_became_active":"2024-09-14T06:28:51.223535+0000","last_became_peered":"2024-09-14T06:28:51.223535+0000","last_unstale":"2024-09-14T06:28:53.625193+0000","last_undegraded":"2024-09-14T06:28:53.625193+0000","last_fullsized":"2024-09-14T06:28:53.625193+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:28:50.205350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:28:50.205350+0000","last_clean_scrub_stamp":"2024-09-14T06:28:50.205350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T18:14:22.822245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00052660300000000005,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.6","version":"0'0","reported_seq":20,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-14T06:28:53.226122+0000","last_change":"2024-09-14T06:28:53.226122+0000","last_active":"2024-09-14T06:28:53.226122+0000","last_peered":"2024-09-14T06:28:53.226122+0000","last_clean":"2024-09-14T06:28:53.226122+0000","last_became_active":"2024-09-14T06:28:51.217205+0000","last_became_peered":"2024-09-14T06:28:51.217205+0000","last_unstale":"2024-09-14T06:28:53.226122+0000","last_undegraded":"2024-09-14T06:28:53.226122+0000","last_fullsized":"2024-09-14T06:28:53.226122+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:28:50.205350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:28:50.205350+0000","last_clean_scrub_stamp":"2024-09-14T06:28:50.205350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T15:29:19.563302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065018200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.5","version":"0'0","reported_seq":20,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-14T06:28:53.640035+0000","last_change":"2024-09-14T06:28:53.640035+0000","last_active":"2024-09-14T06:28:53.640035+0000","last_peered":"2024-09-14T06:28:53.640035+0000","last_clean":"2024-09-14T06:28:53.640035+0000","last_became_active":"2024-09-14T06:28:51.308058+0000","last_became_peered":"2024-09-14T06:28:51.308058+0000","last_unstale":"2024-09-14T06:28:53.640035+0000","last_undegraded":"2024-09-14T06:28:53.640035+0000","last_fullsized":"2024-09-14T06:28:53.640035+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:28:50.205350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:28:50.205350+0000","last_clean_scrub_stamp":"2024-09-14T06:28:50.205350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T13:35:48.544057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00037824000000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.4","version":"0'0","reported_seq":20,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-14T06:28:53.226072+0000","last_change":"2024-09-14T06:28:53.226072+0000","last_active":"2024-09-14T06:28:53.226072+0000","last_peered":"2024-09-14T06:28:53.226072+0000","last_clean":"2024-09-14T06:28:53.226072+0000","last_became_active":"2024-09-14T06:28:51.308072+0000","last_became_peered":"2024-09-14T06:28:51.308072+0000","last_unstale":"2024-09-14T06:28:53.226072+0000","last_undegraded":"2024-09-14T06:28:53.226072+0000","last_fullsized":"2024-09-14T06:28:53.226072+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:28:50.205350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:28:50.205350+0000","last_clean_scrub_stamp":"2024-09-14T06:28:50.205350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T13:15:09.052328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066197399999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.2","version":"22'2","reported_seq":22,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-14T06:28:53.235878+0000","last_change":"2024-09-14T06:28:53.231428+0000","last_active":"2024-09-14T06:28:53.235878+0000","last_peered":"2024-09-14T06:28:53.235878+0000","last_clean":"2024-09-14T06:28:53.235878+0000","last_became_active":"2024-09-14T06:28:51.222937+0000","last_became_peered":"2024-09-14T06:28:51.222937+0000","last_unstale":"2024-09-14T06:28:53.235878+0000","last_undegraded":"2024-09-14T06:28:53.235878+0000","last_fullsized":"2024-09-14T06:28:53.235878+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:28:50.205350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:28:50.205350+0000","last_clean_scrub_stamp":"2024-09-14T06:28:50.205350+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T14:27:37.233976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010072970000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.1","version":"0'0","reported_seq":20,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-14T06:28:53.225992+0000","last_change":"2024-09-14T06:28:53.226264+0000","last_active":"2024-09-14T06:28:53.225992+0000","last_peered":"2024-09-14T06:28:53.225992+0000","last_clean":"2024-09-14T06:28:53.225992+0000","last_became_active":"2024-09-14T06:28:51.223252+0000","last_became_peered":"2024-09-14T06:28:51.223252+0000","last_unstale":"2024-09-14T06:28:53.225992+0000","last_undegraded":"2024-09-14T06:28:53.225992+0000","last_fullsized":"2024-09-14T06:28:53.225992+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:28:50.205350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:28:50.205350+0000","last_clean_scrub_stamp":"2024-09-14T06:28:50.205350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T11:56:04.373414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053403100000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.0","version":"0'0","reported_seq":20,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-14T06:28:53.640096+0000","last_change":"2024-09-14T06:28:53.640096+0000","last_active":"2024-09-14T06:28:53.640096+0000","last_peered":"2024-09-14T06:28:53.640096+0000","last_clean":"2024-09-14T06:28:53.640096+0000","last_became_active":"2024-09-14T06:28:51.223069+0000","last_became_peered":"2024-09-14T06:28:51.223069+0000","last_unstale":"2024-09-14T06:28:53.640096+0000","last_undegraded":"2024-09-14T06:28:53.640096+0000","last_fullsized":"2024-09-14T06:28:53.640096+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:28:50.205350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:28:50.205350+0000","last_clean_scrub_stamp":"2024-09-14T06:28:50.205350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T15:13:00.471509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00043714600000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.3","version":"20'1","reported_seq":21,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-14T06:28:53.230933+0000","last_change":"2024-09-14T06:28:53.230933+0000","last_active":"2024-09-14T06:28:53.230933+0000","last_peered":"2024-09-14T06:28:53.230933+0000","last_clean":"2024-09-14T06:28:53.230933+0000","last_became_active":"2024-09-14T06:28:51.222874+0000","last_became_peered":"2024-09-14T06:28:51.222874+0000","last_unstale":"2024-09-14T06:28:53.230933+0000","last_undegraded":"2024-09-14T06:28:53.230933+0000","last_fullsized":"2024-09-14T06:28:53.230933+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:28:50.205350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:28:50.205350+0000","last_clean_scrub_stamp":"2024-09-14T06:28:50.205350+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T12:04:25.190171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050808899999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.0","version":"17'72","reported_seq":111,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-14T06:28:53.639580+0000","last_change":"2024-09-14T06:28:44.839885+0000","last_active":"2024-09-14T06:28:53.639580+0000","last_peered":"2024-09-14T06:28:53.639580+0000","last_clean":"2024-09-14T06:28:53.639580+0000","last_became_active":"2024-09-14T06:28:44.839573+0000","last_became_peered":"2024-09-14T06:28:44.839573+0000","last_unstale":"2024-09-14T06:28:53.639580+0000","last_undegraded":"2024-09-14T06:28:53.639580+0000","last_fullsized":"2024-09-14T06:28:53.639580+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:28:43.828494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:28:43.828494+0000","last_clean_scrub_stamp":"2024-09-14T06:28:43.828494+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T14:41:25.787293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":131072,"data_stored":1180736,"data_compressed":65166,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509445,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27308,"kb_used_data":280,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344532,"statfs":{"total":96636764160,"available":96608800768,"internally_reserved":0,"allocated":286720,"data_stored":665454,"data_compressed":32583,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509444,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27244,"kb_used_data":216,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344596,"statfs":{"total":96636764160,"available":96608866304,"internally_reserved":0,"allocated":221184,"data_stored":75086,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509446,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27256,"kb_used_data":220,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344584,"statfs":{"total":96636764160,"available":96608854016,"internally_reserved":0,"allocated":225280,"data_stored":75105,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509445,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27252,"kb_used_data":216,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344588,"statfs":{"total":96636764160,"available":96608858112,"internally_reserved":0,"allocated":221184,"data_stored":75086,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509446,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31416,"kb_used_data":220,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340424,"statfs":{"total":96636764160,"available":96604594176,"internally_reserved":0,"allocated":225280,"data_stored":75105,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":14,"seq":60129542149,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31408,"kb_used_data":216,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340432,"statfs":{"total":96636764160,"available":96604602368,"internally_reserved":0,"allocated":221184,"data_stored":75086,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542148,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31472,"kb_used_data":280,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340368,"statfs":{"total":96636764160,"available":96604536832,"internally_reserved":0,"allocated":286720,"data_stored":665454,"data_compressed":32583,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574854,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31408,"kb_used_data":216,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340432,"statfs":{"total":96636764160,"available":96604602368,"internally_reserved":0,"allocated":221184,"data_stored":75086,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32583,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32583,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-14T06:28:58.873 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-14T06:28:58.873 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump 2024-09-14T06:28:59.154 INFO:teuthology.orchestra.run.smithi094.stdout:epoch 22 2024-09-14T06:28:59.154 INFO:teuthology.orchestra.run.smithi094.stdout:fsid 200ad3da-bd63-4e04-af74-efb5ca0af75e 2024-09-14T06:28:59.154 INFO:teuthology.orchestra.run.smithi094.stdout:created 2024-09-14T06:28:30.782046+0000 2024-09-14T06:28:59.154 INFO:teuthology.orchestra.run.smithi094.stdout:modified 2024-09-14T06:28:53.222080+0000 2024-09-14T06:28:59.154 INFO:teuthology.orchestra.run.smithi094.stdout:flags sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit 2024-09-14T06:28:59.154 INFO:teuthology.orchestra.run.smithi094.stdout:crush_version 5 2024-09-14T06:28:59.154 INFO:teuthology.orchestra.run.smithi094.stdout:full_ratio 0.95 2024-09-14T06:28:59.154 INFO:teuthology.orchestra.run.smithi094.stdout:backfillfull_ratio 0.9 2024-09-14T06:28:59.154 INFO:teuthology.orchestra.run.smithi094.stdout:nearfull_ratio 0.85 2024-09-14T06:28:59.154 INFO:teuthology.orchestra.run.smithi094.stdout:require_min_compat_client luminous 2024-09-14T06:28:59.155 INFO:teuthology.orchestra.run.smithi094.stdout:min_compat_client jewel 2024-09-14T06:28:59.155 INFO:teuthology.orchestra.run.smithi094.stdout:require_osd_release squid 2024-09-14T06:28:59.155 INFO:teuthology.orchestra.run.smithi094.stdout:stretch_mode_enabled false 2024-09-14T06:28:59.155 INFO:teuthology.orchestra.run.smithi094.stdout:pool 1 '.mgr' replicated size 2 min_size 1 crush_rule 0 object_hash rjenkins pg_num 1 pgp_num 1 autoscale_mode off last_change 18 flags hashpspool stripe_width 0 pg_num_max 32 pg_num_min 1 application mgr read_balance_score 8.00 2024-09-14T06:28:59.155 INFO:teuthology.orchestra.run.smithi094.stdout:pool 2 'rbd' replicated size 2 min_size 1 crush_rule 0 object_hash rjenkins pg_num 8 pgp_num 8 autoscale_mode off last_change 22 flags hashpspool,selfmanaged_snaps stripe_width 0 application rbd read_balance_score 1.99 2024-09-14T06:28:59.155 INFO:teuthology.orchestra.run.smithi094.stdout: removed_snaps_queue [2~1] 2024-09-14T06:28:59.155 INFO:teuthology.orchestra.run.smithi094.stdout:max_osd 8 2024-09-14T06:28:59.155 INFO:teuthology.orchestra.run.smithi094.stdout:osd.0 up in weight 1 up_from 14 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:172.21.15.94:6824/1809564923,v1:172.21.15.94:6825/1809564923] [v2:172.21.15.94:6826/1809564923,v1:172.21.15.94:6827/1809564923] exists,up 8b22a249-b842-406a-a02c-c0f31f57b635 2024-09-14T06:28:59.155 INFO:teuthology.orchestra.run.smithi094.stdout:osd.1 up in weight 1 up_from 15 up_thru 19 down_at 0 last_clean_interval [0,0) [v2:172.21.15.94:6816/736402652,v1:172.21.15.94:6817/736402652] [v2:172.21.15.94:6818/736402652,v1:172.21.15.94:6819/736402652] exists,up 02251a03-84a0-414d-a980-fac4fd043cb6 2024-09-14T06:28:59.155 INFO:teuthology.orchestra.run.smithi094.stdout:osd.2 up in weight 1 up_from 14 up_thru 19 down_at 0 last_clean_interval [0,0) [v2:172.21.15.94:6808/1063204782,v1:172.21.15.94:6809/1063204782] [v2:172.21.15.94:6810/1063204782,v1:172.21.15.94:6811/1063204782] exists,up d45bf470-bb29-4739-8294-cd122b2b4102 2024-09-14T06:28:59.155 INFO:teuthology.orchestra.run.smithi094.stdout:osd.3 up in weight 1 up_from 13 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:172.21.15.94:6800/3382046754,v1:172.21.15.94:6801/3382046754] [v2:172.21.15.94:6802/3382046754,v1:172.21.15.94:6803/3382046754] exists,up 456ec2d9-c438-4243-b931-335704f661eb 2024-09-14T06:28:59.155 INFO:teuthology.orchestra.run.smithi094.stdout:osd.4 up in weight 1 up_from 15 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:172.21.15.138:6810/501726729,v1:172.21.15.138:6811/501726729] [v2:172.21.15.138:6812/501726729,v1:172.21.15.138:6813/501726729] exists,up 2dab948e-c6e4-4437-bf17-017d1ac6821c 2024-09-14T06:28:59.155 INFO:teuthology.orchestra.run.smithi094.stdout:osd.5 up in weight 1 up_from 15 up_thru 19 down_at 0 last_clean_interval [0,0) [v2:172.21.15.138:6826/1599292735,v1:172.21.15.138:6827/1599292735] [v2:172.21.15.138:6828/1599292735,v1:172.21.15.138:6829/1599292735] exists,up 8caf3aa3-ad87-4d33-b82e-4c15195176fd 2024-09-14T06:28:59.156 INFO:teuthology.orchestra.run.smithi094.stdout:osd.6 up in weight 1 up_from 15 up_thru 19 down_at 0 last_clean_interval [0,0) [v2:172.21.15.138:6818/163531479,v1:172.21.15.138:6819/163531479] [v2:172.21.15.138:6820/163531479,v1:172.21.15.138:6821/163531479] exists,up e55d11fd-433d-4462-93a7-d5e84560221b 2024-09-14T06:28:59.156 INFO:teuthology.orchestra.run.smithi094.stdout:osd.7 up in weight 1 up_from 15 up_thru 19 down_at 0 last_clean_interval [0,0) [v2:172.21.15.138:6800/218507235,v1:172.21.15.138:6801/218507235] [v2:172.21.15.138:6802/218507235,v1:172.21.15.138:6803/218507235] exists,up ba14ba69-2c6c-4d09-81e2-2593dc3a7ea9 2024-09-14T06:28:59.166 INFO:tasks.ceph.ceph_manager.ceph:['osd.0 up in weight 1 up_from 14 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:172.21.15.94:6824/1809564923,v1:172.21.15.94:6825/1809564923] [v2:172.21.15.94:6826/1809564923,v1:172.21.15.94:6827/1809564923] exists,up 8b22a249-b842-406a-a02c-c0f31f57b635', 'osd.1 up in weight 1 up_from 15 up_thru 19 down_at 0 last_clean_interval [0,0) [v2:172.21.15.94:6816/736402652,v1:172.21.15.94:6817/736402652] [v2:172.21.15.94:6818/736402652,v1:172.21.15.94:6819/736402652] exists,up 02251a03-84a0-414d-a980-fac4fd043cb6', 'osd.2 up in weight 1 up_from 14 up_thru 19 down_at 0 last_clean_interval [0,0) [v2:172.21.15.94:6808/1063204782,v1:172.21.15.94:6809/1063204782] [v2:172.21.15.94:6810/1063204782,v1:172.21.15.94:6811/1063204782] exists,up d45bf470-bb29-4739-8294-cd122b2b4102', 'osd.3 up in weight 1 up_from 13 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:172.21.15.94:6800/3382046754,v1:172.21.15.94:6801/3382046754] [v2:172.21.15.94:6802/3382046754,v1:172.21.15.94:6803/3382046754] exists,up 456ec2d9-c438-4243-b931-335704f661eb', 'osd.4 up in weight 1 up_from 15 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:172.21.15.138:6810/501726729,v1:172.21.15.138:6811/501726729] [v2:172.21.15.138:6812/501726729,v1:172.21.15.138:6813/501726729] exists,up 2dab948e-c6e4-4437-bf17-017d1ac6821c', 'osd.5 up in weight 1 up_from 15 up_thru 19 down_at 0 last_clean_interval [0,0) [v2:172.21.15.138:6826/1599292735,v1:172.21.15.138:6827/1599292735] [v2:172.21.15.138:6828/1599292735,v1:172.21.15.138:6829/1599292735] exists,up 8caf3aa3-ad87-4d33-b82e-4c15195176fd', 'osd.6 up in weight 1 up_from 15 up_thru 19 down_at 0 last_clean_interval [0,0) [v2:172.21.15.138:6818/163531479,v1:172.21.15.138:6819/163531479] [v2:172.21.15.138:6820/163531479,v1:172.21.15.138:6821/163531479] exists,up e55d11fd-433d-4462-93a7-d5e84560221b', 'osd.7 up in weight 1 up_from 15 up_thru 19 down_at 0 last_clean_interval [0,0) [v2:172.21.15.138:6800/218507235,v1:172.21.15.138:6801/218507235] [v2:172.21.15.138:6802/218507235,v1:172.21.15.138:6803/218507235] exists,up ba14ba69-2c6c-4d09-81e2-2593dc3a7ea9'] 2024-09-14T06:28:59.167 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mon.a.asok config show 2024-09-14T06:28:59.264 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-14T06:28:59.265 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "mon.a", 2024-09-14T06:28:59.265 INFO:teuthology.orchestra.run.smithi094.stdout: "cluster": "ceph", 2024-09-14T06:28:59.265 INFO:teuthology.orchestra.run.smithi094.stdout: "admin_socket": "/var/run/ceph/ceph-mon.a.asok", 2024-09-14T06:28:59.265 INFO:teuthology.orchestra.run.smithi094.stdout: "admin_socket_mode": "", 2024-09-14T06:28:59.265 INFO:teuthology.orchestra.run.smithi094.stdout: "auth_allow_insecure_global_id_reclaim": "true", 2024-09-14T06:28:59.265 INFO:teuthology.orchestra.run.smithi094.stdout: "auth_client_required": "cephx, none", 2024-09-14T06:28:59.265 INFO:teuthology.orchestra.run.smithi094.stdout: "auth_cluster_required": "cephx", 2024-09-14T06:28:59.265 INFO:teuthology.orchestra.run.smithi094.stdout: "auth_debug": "true", 2024-09-14T06:28:59.265 INFO:teuthology.orchestra.run.smithi094.stdout: "auth_expose_insecure_global_id_reclaim": "true", 2024-09-14T06:28:59.265 INFO:teuthology.orchestra.run.smithi094.stdout: "auth_mon_ticket_ttl": "660.000000", 2024-09-14T06:28:59.265 INFO:teuthology.orchestra.run.smithi094.stdout: "auth_service_required": "cephx", 2024-09-14T06:28:59.265 INFO:teuthology.orchestra.run.smithi094.stdout: "auth_service_ticket_ttl": "240.000000", 2024-09-14T06:28:59.265 INFO:teuthology.orchestra.run.smithi094.stdout: "auth_supported": "cephx", 2024-09-14T06:28:59.265 INFO:teuthology.orchestra.run.smithi094.stdout: "bdev_aio": "true", 2024-09-14T06:28:59.266 INFO:teuthology.orchestra.run.smithi094.stdout: "bdev_aio_max_queue_depth": "1024", 2024-09-14T06:28:59.266 INFO:teuthology.orchestra.run.smithi094.stdout: "bdev_aio_poll_ms": "250", 2024-09-14T06:28:59.266 INFO:teuthology.orchestra.run.smithi094.stdout: "bdev_aio_reap_max": "16", 2024-09-14T06:28:59.266 INFO:teuthology.orchestra.run.smithi094.stdout: "bdev_async_discard_threads": "0", 2024-09-14T06:28:59.266 INFO:teuthology.orchestra.run.smithi094.stdout: "bdev_block_size": "4096", 2024-09-14T06:28:59.266 INFO:teuthology.orchestra.run.smithi094.stdout: "bdev_debug_aio": "false", 2024-09-14T06:28:59.266 INFO:teuthology.orchestra.run.smithi094.stdout: "bdev_debug_aio_log_age": "5.000000", 2024-09-14T06:28:59.266 INFO:teuthology.orchestra.run.smithi094.stdout: "bdev_debug_aio_suicide_timeout": "60.000000", 2024-09-14T06:28:59.266 INFO:teuthology.orchestra.run.smithi094.stdout: "bdev_debug_inflight_ios": "false", 2024-09-14T06:28:59.266 INFO:teuthology.orchestra.run.smithi094.stdout: "bdev_enable_discard": "false", 2024-09-14T06:28:59.266 INFO:teuthology.orchestra.run.smithi094.stdout: "bdev_flock_retry": "3", 2024-09-14T06:28:59.266 INFO:teuthology.orchestra.run.smithi094.stdout: "bdev_flock_retry_interval": "0.100000", 2024-09-14T06:28:59.266 INFO:teuthology.orchestra.run.smithi094.stdout: "bdev_inject_crash": "0", 2024-09-14T06:28:59.266 INFO:teuthology.orchestra.run.smithi094.stdout: "bdev_inject_crash_flush_delay": "2", 2024-09-14T06:28:59.266 INFO:teuthology.orchestra.run.smithi094.stdout: "bdev_ioring": "false", 2024-09-14T06:28:59.267 INFO:teuthology.orchestra.run.smithi094.stdout: "bdev_ioring_hipri": "false", 2024-09-14T06:28:59.267 INFO:teuthology.orchestra.run.smithi094.stdout: "bdev_ioring_sqthread_poll": "false", 2024-09-14T06:28:59.267 INFO:teuthology.orchestra.run.smithi094.stdout: "bdev_nvme_unbind_from_kernel": "false", 2024-09-14T06:28:59.267 INFO:teuthology.orchestra.run.smithi094.stdout: "bdev_read_buffer_alignment": "4096", 2024-09-14T06:28:59.267 INFO:teuthology.orchestra.run.smithi094.stdout: "bdev_read_preallocated_huge_buffers": "", 2024-09-14T06:28:59.267 INFO:teuthology.orchestra.run.smithi094.stdout: "bdev_stalled_read_warn_lifetime": "86400", 2024-09-14T06:28:59.267 INFO:teuthology.orchestra.run.smithi094.stdout: "bdev_stalled_read_warn_threshold": "1", 2024-09-14T06:28:59.267 INFO:teuthology.orchestra.run.smithi094.stdout: "bdev_type": "", 2024-09-14T06:28:59.267 INFO:teuthology.orchestra.run.smithi094.stdout: "bluefs_alloc_size": "1048576", 2024-09-14T06:28:59.267 INFO:teuthology.orchestra.run.smithi094.stdout: "bluefs_allocator": "hybrid", 2024-09-14T06:28:59.267 INFO:teuthology.orchestra.run.smithi094.stdout: "bluefs_buffered_io": "true", 2024-09-14T06:28:59.267 INFO:teuthology.orchestra.run.smithi094.stdout: "bluefs_check_for_zeros": "false", 2024-09-14T06:28:59.267 INFO:teuthology.orchestra.run.smithi094.stdout: "bluefs_check_volume_selector_often": "false", 2024-09-14T06:28:59.267 INFO:teuthology.orchestra.run.smithi094.stdout: "bluefs_check_volume_selector_on_umount": "false", 2024-09-14T06:28:59.267 INFO:teuthology.orchestra.run.smithi094.stdout: "bluefs_compact_log_sync": "false", 2024-09-14T06:28:59.268 INFO:teuthology.orchestra.run.smithi094.stdout: "bluefs_failed_shared_alloc_cooldown": "600.000000", 2024-09-14T06:28:59.268 INFO:teuthology.orchestra.run.smithi094.stdout: "bluefs_log_compact_min_ratio": "5.000000", 2024-09-14T06:28:59.268 INFO:teuthology.orchestra.run.smithi094.stdout: "bluefs_log_compact_min_size": "16777216", 2024-09-14T06:28:59.268 INFO:teuthology.orchestra.run.smithi094.stdout: "bluefs_log_replay_check_allocations": "true", 2024-09-14T06:28:59.268 INFO:teuthology.orchestra.run.smithi094.stdout: "bluefs_max_log_runway": "4194304", 2024-09-14T06:28:59.268 INFO:teuthology.orchestra.run.smithi094.stdout: "bluefs_max_prefetch": "1048576", 2024-09-14T06:28:59.268 INFO:teuthology.orchestra.run.smithi094.stdout: "bluefs_min_flush_size": "524288", 2024-09-14T06:28:59.268 INFO:teuthology.orchestra.run.smithi094.stdout: "bluefs_min_log_runway": "1048576", 2024-09-14T06:28:59.268 INFO:teuthology.orchestra.run.smithi094.stdout: "bluefs_replay_recovery": "false", 2024-09-14T06:28:59.268 INFO:teuthology.orchestra.run.smithi094.stdout: "bluefs_replay_recovery_disable_compact": "false", 2024-09-14T06:28:59.268 INFO:teuthology.orchestra.run.smithi094.stdout: "bluefs_shared_alloc_size": "65536", 2024-09-14T06:28:59.268 INFO:teuthology.orchestra.run.smithi094.stdout: "bluefs_sync_write": "false", 2024-09-14T06:28:59.268 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_2q_cache_kin_ratio": "0.500000", 2024-09-14T06:28:59.269 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_2q_cache_kout_ratio": "0.500000", 2024-09-14T06:28:59.269 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_alloc_stats_dump_interval": "86400.000000", 2024-09-14T06:28:59.269 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_allocation_from_file": "true", 2024-09-14T06:28:59.269 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_allocator": "hybrid", 2024-09-14T06:28:59.269 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_async_db_compaction": "true", 2024-09-14T06:28:59.269 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_avl_alloc_bf_free_pct": "4", 2024-09-14T06:28:59.269 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_avl_alloc_bf_threshold": "131072", 2024-09-14T06:28:59.269 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_avl_alloc_ff_max_search_bytes": "16777216", 2024-09-14T06:28:59.269 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_avl_alloc_ff_max_search_count": "100", 2024-09-14T06:28:59.269 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_bdev_label_multi": "true", 2024-09-14T06:28:59.269 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_bdev_label_multi_upgrade": "false", 2024-09-14T06:28:59.269 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_bdev_label_require_all": "true", 2024-09-14T06:28:59.269 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_bitmapallocator_blocks_per_zone": "1024", 2024-09-14T06:28:59.270 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_bitmapallocator_span_size": "1024", 2024-09-14T06:28:59.270 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_blobid_prealloc": "10240", 2024-09-14T06:28:59.270 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_block_create": "true", 2024-09-14T06:28:59.270 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_block_db_create": "false", 2024-09-14T06:28:59.270 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_block_db_path": "", 2024-09-14T06:28:59.270 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_block_db_size": "0", 2024-09-14T06:28:59.270 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_block_path": "", 2024-09-14T06:28:59.270 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_block_preallocate_file": "false", 2024-09-14T06:28:59.270 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_block_size": "107374182400", 2024-09-14T06:28:59.270 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_block_wal_create": "false", 2024-09-14T06:28:59.270 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_block_wal_path": "", 2024-09-14T06:28:59.270 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_block_wal_size": "100663296", 2024-09-14T06:28:59.270 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_bluefs": "true", 2024-09-14T06:28:59.270 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_bluefs_alloc_failure_dump_interval": "0.000000", 2024-09-14T06:28:59.271 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_bluefs_env_mirror": "false", 2024-09-14T06:28:59.271 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_bluefs_max_free": "10737418240", 2024-09-14T06:28:59.271 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_btree2_alloc_weight_factor": "2.000000", 2024-09-14T06:28:59.271 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_cache_age_bin_interval": "1.000000", 2024-09-14T06:28:59.271 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_cache_age_bins_data": "1 2 6 24 120 720 0 0 0 0", 2024-09-14T06:28:59.271 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_cache_age_bins_kv": "1 2 6 24 120 720 0 0 0 0", 2024-09-14T06:28:59.271 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_cache_age_bins_kv_onode": "0 0 0 0 0 0 0 0 0 720", 2024-09-14T06:28:59.271 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_cache_age_bins_meta": "1 2 6 24 120 720 0 0 0 0", 2024-09-14T06:28:59.271 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_cache_autotune": "true", 2024-09-14T06:28:59.271 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_cache_autotune_interval": "5.000000", 2024-09-14T06:28:59.271 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_cache_kv_onode_ratio": "0.040000", 2024-09-14T06:28:59.271 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_cache_kv_ratio": "0.450000", 2024-09-14T06:28:59.271 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_cache_meta_ratio": "0.450000", 2024-09-14T06:28:59.271 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_cache_size": "0", 2024-09-14T06:28:59.271 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_cache_size_hdd": "1073741824", 2024-09-14T06:28:59.272 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_cache_size_ssd": "3221225472", 2024-09-14T06:28:59.272 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_cache_trim_interval": "0.050000", 2024-09-14T06:28:59.272 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_cache_trim_max_skip_pinned": "1000", 2024-09-14T06:28:59.272 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_cache_type": "2q", 2024-09-14T06:28:59.272 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_cleaner_sleep_interval": "5.000000", 2024-09-14T06:28:59.272 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_clone_cow": "true", 2024-09-14T06:28:59.272 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_compression_algorithm": "snappy", 2024-09-14T06:28:59.272 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_compression_max_blob_size": "0", 2024-09-14T06:28:59.272 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_compression_max_blob_size_hdd": "65536", 2024-09-14T06:28:59.272 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_compression_max_blob_size_ssd": "65536", 2024-09-14T06:28:59.272 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_compression_min_blob_size": "0", 2024-09-14T06:28:59.272 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_compression_min_blob_size_hdd": "8192", 2024-09-14T06:28:59.272 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_compression_min_blob_size_ssd": "65536", 2024-09-14T06:28:59.272 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_compression_mode": "none", 2024-09-14T06:28:59.273 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_compression_required_ratio": "0.875000", 2024-09-14T06:28:59.273 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_csum_type": "crc32c", 2024-09-14T06:28:59.273 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_debug_enforce_min_alloc_size": "0", 2024-09-14T06:28:59.273 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_debug_enforce_settings": "default", 2024-09-14T06:28:59.273 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_debug_freelist": "false", 2024-09-14T06:28:59.273 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_debug_fsck_abort": "false", 2024-09-14T06:28:59.273 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_debug_inject_allocation_from_file_failure": "0.000000", 2024-09-14T06:28:59.273 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_debug_inject_csum_err_probability": "0.000000", 2024-09-14T06:28:59.273 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_debug_inject_read_err": "false", 2024-09-14T06:28:59.273 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_debug_legacy_omap": "false", 2024-09-14T06:28:59.273 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_debug_no_reuse_blocks": "false", 2024-09-14T06:28:59.273 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_debug_omit_block_device_write": "false", 2024-09-14T06:28:59.273 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_debug_omit_kv_commit": "false", 2024-09-14T06:28:59.273 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_debug_permit_any_bdev_label": "false", 2024-09-14T06:28:59.274 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_debug_prefragment_max": "1048576", 2024-09-14T06:28:59.274 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_debug_random_read_err": "0.000000", 2024-09-14T06:28:59.274 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_debug_randomize_serial_transaction": "0", 2024-09-14T06:28:59.274 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_debug_small_allocations": "0", 2024-09-14T06:28:59.274 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_debug_too_many_blobs_threshold": "24576", 2024-09-14T06:28:59.274 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_default_buffered_read": "true", 2024-09-14T06:28:59.274 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_default_buffered_write": "false", 2024-09-14T06:28:59.274 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_deferred_batch_ops": "0", 2024-09-14T06:28:59.274 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_deferred_batch_ops_hdd": "64", 2024-09-14T06:28:59.274 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_deferred_batch_ops_ssd": "16", 2024-09-14T06:28:59.274 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_elastic_shared_blobs": "true", 2024-09-14T06:28:59.274 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_extent_map_inline_shard_prealloc_size": "256", 2024-09-14T06:28:59.274 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_extent_map_shard_max_size": "1200", 2024-09-14T06:28:59.275 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_extent_map_shard_min_size": "150", 2024-09-14T06:28:59.275 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_extent_map_shard_target_size": "500", 2024-09-14T06:28:59.275 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_extent_map_shard_target_size_slop": "0.200000", 2024-09-14T06:28:59.275 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_fail_eio": "false", 2024-09-14T06:28:59.275 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_freelist_blocks_per_key": "128", 2024-09-14T06:28:59.275 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_fsck_error_on_no_per_pg_omap": "false", 2024-09-14T06:28:59.275 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_fsck_error_on_no_per_pool_omap": "false", 2024-09-14T06:28:59.275 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_fsck_error_on_no_per_pool_stats": "false", 2024-09-14T06:28:59.275 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_fsck_on_mkfs": "true", 2024-09-14T06:28:59.275 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_fsck_on_mkfs_deep": "false", 2024-09-14T06:28:59.275 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_fsck_on_mount": "false", 2024-09-14T06:28:59.275 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_fsck_on_mount_deep": "false", 2024-09-14T06:28:59.275 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_fsck_on_umount": "false", 2024-09-14T06:28:59.275 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_fsck_on_umount_deep": "false", 2024-09-14T06:28:59.276 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_fsck_quick_fix_on_mount": "false", 2024-09-14T06:28:59.276 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_fsck_quick_fix_threads": "2", 2024-09-14T06:28:59.276 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_fsck_read_bytes_cap": "67108864", 2024-09-14T06:28:59.276 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_fsck_shared_blob_tracker_size": "0.031250", 2024-09-14T06:28:59.276 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_gc_enable_blob_threshold": "0", 2024-09-14T06:28:59.276 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_gc_enable_total_threshold": "0", 2024-09-14T06:28:59.276 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_hybrid_alloc_mem_cap": "67108864", 2024-09-14T06:28:59.276 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_ignore_data_csum": "false", 2024-09-14T06:28:59.276 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_kv_sync_util_logging_s": "10.000000", 2024-09-14T06:28:59.276 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_kvbackend": "rocksdb", 2024-09-14T06:28:59.276 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_log_collection_list_age": "60.000000", 2024-09-14T06:28:59.276 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_log_omap_iterator_age": "5.000000", 2024-09-14T06:28:59.276 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_log_op_age": "5.000000", 2024-09-14T06:28:59.276 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_max_alloc_size": "0", 2024-09-14T06:28:59.276 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_max_blob_size": "0", 2024-09-14T06:28:59.277 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_max_blob_size_hdd": "65536", 2024-09-14T06:28:59.277 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_max_blob_size_ssd": "65536", 2024-09-14T06:28:59.277 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_max_defer_interval": "3.000000", 2024-09-14T06:28:59.277 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_max_deferred_txc": "32", 2024-09-14T06:28:59.277 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_min_alloc_size": "0", 2024-09-14T06:28:59.277 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_min_alloc_size_hdd": "4096", 2024-09-14T06:28:59.277 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_min_alloc_size_ssd": "4096", 2024-09-14T06:28:59.277 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_nid_prealloc": "1024", 2024-09-14T06:28:59.277 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_prefer_deferred_size": "0", 2024-09-14T06:28:59.277 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_prefer_deferred_size_hdd": "65536", 2024-09-14T06:28:59.277 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_prefer_deferred_size_ssd": "0", 2024-09-14T06:28:59.277 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_qfsck_on_mount": "true", 2024-09-14T06:28:59.277 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_retry_disk_reads": "3", 2024-09-14T06:28:59.277 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_rocksdb_cf": "true", 2024-09-14T06:28:59.278 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_rocksdb_cfs": "m(3) p(3,0-12) O(3,0-13)=block_cache={type=binned_lru} L=min_write_buffer_number_to_merge=32 P=min_write_buffer_number_to_merge=32", 2024-09-14T06:28:59.278 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_rocksdb_options": "compression=kLZ4Compression,max_write_buffer_number=64,min_write_buffer_number_to_merge=6,compaction_style=kCompactionStyleLevel,write_buffer_size=16777216,max_background_jobs=4,level0_file_num_compaction_trigger=8,max_bytes_for_level_base=1073741824,max_bytes_for_level_multiplier=8,compaction_readahead_size=2MB,max_total_wal_size=1073741824,writable_file_max_buffer_size=0", 2024-09-14T06:28:59.278 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_rocksdb_options_annex": "", 2024-09-14T06:28:59.278 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_slow_ops_warn_lifetime": "86400", 2024-09-14T06:28:59.278 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_slow_ops_warn_threshold": "1", 2024-09-14T06:28:59.278 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_spdk_coremask": "0x1", 2024-09-14T06:28:59.278 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_spdk_io_sleep": "5", 2024-09-14T06:28:59.278 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_spdk_max_io_completion": "0", 2024-09-14T06:28:59.278 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_spdk_mem": "512", 2024-09-14T06:28:59.278 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_sync_submit_transaction": "false", 2024-09-14T06:28:59.278 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_throttle_bytes": "67108864", 2024-09-14T06:28:59.278 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_throttle_cost_per_io": "0", 2024-09-14T06:28:59.278 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_throttle_cost_per_io_hdd": "670000", 2024-09-14T06:28:59.278 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_throttle_cost_per_io_ssd": "4000", 2024-09-14T06:28:59.278 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_throttle_deferred_bytes": "134217728", 2024-09-14T06:28:59.279 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_throttle_trace_rate": "0.000000", 2024-09-14T06:28:59.279 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_tracing": "false", 2024-09-14T06:28:59.279 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_use_optimal_io_size_for_min_alloc_size": "false", 2024-09-14T06:28:59.279 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_volume_selection_policy": "use_some_extra", 2024-09-14T06:28:59.279 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_volume_selection_reserved": "0", 2024-09-14T06:28:59.279 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_volume_selection_reserved_factor": "2.000000", 2024-09-14T06:28:59.279 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_warn_on_bluefs_spillover": "true", 2024-09-14T06:28:59.279 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_warn_on_legacy_statfs": "true", 2024-09-14T06:28:59.279 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_warn_on_no_per_pg_omap": "false", 2024-09-14T06:28:59.279 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_warn_on_no_per_pool_omap": "true", 2024-09-14T06:28:59.279 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_warn_on_spurious_read_errors": "true", 2024-09-14T06:28:59.279 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_write_v2": "false", 2024-09-14T06:28:59.279 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_write_v2_random": "false", 2024-09-14T06:28:59.279 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_zero_block_detection": "false", 2024-09-14T06:28:59.279 INFO:teuthology.orchestra.run.smithi094.stdout: "cephadm_path": "/usr/sbin/cephadm", 2024-09-14T06:28:59.280 INFO:teuthology.orchestra.run.smithi094.stdout: "cephfs_mirror_action_update_interval": "2", 2024-09-14T06:28:59.280 INFO:teuthology.orchestra.run.smithi094.stdout: "cephfs_mirror_directory_scan_interval": "10", 2024-09-14T06:28:59.280 INFO:teuthology.orchestra.run.smithi094.stdout: "cephfs_mirror_max_concurrent_directory_syncs": "3", 2024-09-14T06:28:59.280 INFO:teuthology.orchestra.run.smithi094.stdout: "cephfs_mirror_max_consecutive_failures_per_directory": "10", 2024-09-14T06:28:59.280 INFO:teuthology.orchestra.run.smithi094.stdout: "cephfs_mirror_max_snapshot_sync_per_cycle": "3", 2024-09-14T06:28:59.280 INFO:teuthology.orchestra.run.smithi094.stdout: "cephfs_mirror_mount_timeout": "10", 2024-09-14T06:28:59.280 INFO:teuthology.orchestra.run.smithi094.stdout: "cephfs_mirror_perf_stats_prio": "5", 2024-09-14T06:28:59.280 INFO:teuthology.orchestra.run.smithi094.stdout: "cephfs_mirror_restart_mirror_on_blocklist_interval": "30", 2024-09-14T06:28:59.280 INFO:teuthology.orchestra.run.smithi094.stdout: "cephfs_mirror_restart_mirror_on_failure_interval": "20", 2024-09-14T06:28:59.280 INFO:teuthology.orchestra.run.smithi094.stdout: "cephfs_mirror_retry_failed_directories_interval": "60", 2024-09-14T06:28:59.280 INFO:teuthology.orchestra.run.smithi094.stdout: "cephsqlite_blocklist_dead_locker": "true", 2024-09-14T06:28:59.280 INFO:teuthology.orchestra.run.smithi094.stdout: "cephsqlite_lock_renewal_interval": "2000", 2024-09-14T06:28:59.280 INFO:teuthology.orchestra.run.smithi094.stdout: "cephsqlite_lock_renewal_timeout": "30000", 2024-09-14T06:28:59.281 INFO:teuthology.orchestra.run.smithi094.stdout: "cephx_cluster_require_signatures": "false", 2024-09-14T06:28:59.281 INFO:teuthology.orchestra.run.smithi094.stdout: "cephx_cluster_require_version": "2", 2024-09-14T06:28:59.281 INFO:teuthology.orchestra.run.smithi094.stdout: "cephx_require_signatures": "false", 2024-09-14T06:28:59.281 INFO:teuthology.orchestra.run.smithi094.stdout: "cephx_require_version": "2", 2024-09-14T06:28:59.281 INFO:teuthology.orchestra.run.smithi094.stdout: "cephx_service_require_signatures": "false", 2024-09-14T06:28:59.281 INFO:teuthology.orchestra.run.smithi094.stdout: "cephx_service_require_version": "2", 2024-09-14T06:28:59.281 INFO:teuthology.orchestra.run.smithi094.stdout: "cephx_sign_messages": "true", 2024-09-14T06:28:59.281 INFO:teuthology.orchestra.run.smithi094.stdout: "chdir": "", 2024-09-14T06:28:59.281 INFO:teuthology.orchestra.run.smithi094.stdout: "client_acl_type": "", 2024-09-14T06:28:59.281 INFO:teuthology.orchestra.run.smithi094.stdout: "client_asio_thread_count": "2", 2024-09-14T06:28:59.281 INFO:teuthology.orchestra.run.smithi094.stdout: "client_cache_mid": "0.750000", 2024-09-14T06:28:59.281 INFO:teuthology.orchestra.run.smithi094.stdout: "client_cache_size": "16384", 2024-09-14T06:28:59.281 INFO:teuthology.orchestra.run.smithi094.stdout: "client_caps_release_delay": "5", 2024-09-14T06:28:59.281 INFO:teuthology.orchestra.run.smithi094.stdout: "client_check_pool_perm": "true", 2024-09-14T06:28:59.282 INFO:teuthology.orchestra.run.smithi094.stdout: "client_collect_and_send_global_metrics": "false", 2024-09-14T06:28:59.282 INFO:teuthology.orchestra.run.smithi094.stdout: "client_debug_force_sync_read": "false", 2024-09-14T06:28:59.282 INFO:teuthology.orchestra.run.smithi094.stdout: "client_debug_getattr_caps": "false", 2024-09-14T06:28:59.282 INFO:teuthology.orchestra.run.smithi094.stdout: "client_debug_inject_features": "", 2024-09-14T06:28:59.282 INFO:teuthology.orchestra.run.smithi094.stdout: "client_debug_inject_tick_delay": "0", 2024-09-14T06:28:59.282 INFO:teuthology.orchestra.run.smithi094.stdout: "client_die_on_failed_dentry_invalidate": "true", 2024-09-14T06:28:59.282 INFO:teuthology.orchestra.run.smithi094.stdout: "client_die_on_failed_remount": "false", 2024-09-14T06:28:59.282 INFO:teuthology.orchestra.run.smithi094.stdout: "client_dirsize_rbytes": "true", 2024-09-14T06:28:59.282 INFO:teuthology.orchestra.run.smithi094.stdout: "client_force_lazyio": "false", 2024-09-14T06:28:59.282 INFO:teuthology.orchestra.run.smithi094.stdout: "client_fs": "", 2024-09-14T06:28:59.282 INFO:teuthology.orchestra.run.smithi094.stdout: "client_inject_fixed_oldest_tid": "false", 2024-09-14T06:28:59.282 INFO:teuthology.orchestra.run.smithi094.stdout: "client_inject_release_failure": "false", 2024-09-14T06:28:59.282 INFO:teuthology.orchestra.run.smithi094.stdout: "client_max_inline_size": "4096", 2024-09-14T06:28:59.282 INFO:teuthology.orchestra.run.smithi094.stdout: "client_max_retries_on_remount_failure": "5", 2024-09-14T06:28:59.282 INFO:teuthology.orchestra.run.smithi094.stdout: "client_mds_namespace": "", 2024-09-14T06:28:59.283 INFO:teuthology.orchestra.run.smithi094.stdout: "client_metadata": "", 2024-09-14T06:28:59.283 INFO:teuthology.orchestra.run.smithi094.stdout: "client_mount_gid": "-1", 2024-09-14T06:28:59.283 INFO:teuthology.orchestra.run.smithi094.stdout: "client_mount_timeout": "300", 2024-09-14T06:28:59.283 INFO:teuthology.orchestra.run.smithi094.stdout: "client_mount_uid": "-1", 2024-09-14T06:28:59.283 INFO:teuthology.orchestra.run.smithi094.stdout: "client_mountpoint": "/", 2024-09-14T06:28:59.283 INFO:teuthology.orchestra.run.smithi094.stdout: "client_notify_timeout": "10", 2024-09-14T06:28:59.283 INFO:teuthology.orchestra.run.smithi094.stdout: "client_oc": "true", 2024-09-14T06:28:59.283 INFO:teuthology.orchestra.run.smithi094.stdout: "client_oc_max_dirty": "104857600", 2024-09-14T06:28:59.283 INFO:teuthology.orchestra.run.smithi094.stdout: "client_oc_max_dirty_age": "5.000000", 2024-09-14T06:28:59.283 INFO:teuthology.orchestra.run.smithi094.stdout: "client_oc_max_objects": "1000", 2024-09-14T06:28:59.283 INFO:teuthology.orchestra.run.smithi094.stdout: "client_oc_size": "209715200", 2024-09-14T06:28:59.283 INFO:teuthology.orchestra.run.smithi094.stdout: "client_oc_target_dirty": "8388608", 2024-09-14T06:28:59.283 INFO:teuthology.orchestra.run.smithi094.stdout: "client_permissions": "true", 2024-09-14T06:28:59.284 INFO:teuthology.orchestra.run.smithi094.stdout: "client_quota": "true", 2024-09-14T06:28:59.284 INFO:teuthology.orchestra.run.smithi094.stdout: "client_quota_df": "true", 2024-09-14T06:28:59.284 INFO:teuthology.orchestra.run.smithi094.stdout: "client_readahead_max_bytes": "0", 2024-09-14T06:28:59.284 INFO:teuthology.orchestra.run.smithi094.stdout: "client_readahead_max_periods": "4", 2024-09-14T06:28:59.284 INFO:teuthology.orchestra.run.smithi094.stdout: "client_readahead_min": "131072", 2024-09-14T06:28:59.284 INFO:teuthology.orchestra.run.smithi094.stdout: "client_reconnect_stale": "false", 2024-09-14T06:28:59.284 INFO:teuthology.orchestra.run.smithi094.stdout: "client_shutdown_timeout": "30", 2024-09-14T06:28:59.284 INFO:teuthology.orchestra.run.smithi094.stdout: "client_snapdir": ".snap", 2024-09-14T06:28:59.284 INFO:teuthology.orchestra.run.smithi094.stdout: "client_tick_interval": "1", 2024-09-14T06:28:59.284 INFO:teuthology.orchestra.run.smithi094.stdout: "client_trace": "", 2024-09-14T06:28:59.284 INFO:teuthology.orchestra.run.smithi094.stdout: "client_try_dentry_invalidate": "false", 2024-09-14T06:28:59.284 INFO:teuthology.orchestra.run.smithi094.stdout: "client_use_faked_inos": "false", 2024-09-14T06:28:59.284 INFO:teuthology.orchestra.run.smithi094.stdout: "client_use_random_mds": "false", 2024-09-14T06:28:59.285 INFO:teuthology.orchestra.run.smithi094.stdout: "clog_to_graylog": "false", 2024-09-14T06:28:59.285 INFO:teuthology.orchestra.run.smithi094.stdout: "clog_to_graylog_host": "127.0.0.1", 2024-09-14T06:28:59.285 INFO:teuthology.orchestra.run.smithi094.stdout: "clog_to_graylog_port": "12201", 2024-09-14T06:28:59.285 INFO:teuthology.orchestra.run.smithi094.stdout: "clog_to_monitors": "default=true", 2024-09-14T06:28:59.285 INFO:teuthology.orchestra.run.smithi094.stdout: "clog_to_syslog": "false", 2024-09-14T06:28:59.285 INFO:teuthology.orchestra.run.smithi094.stdout: "clog_to_syslog_facility": "default=daemon audit=local0", 2024-09-14T06:28:59.285 INFO:teuthology.orchestra.run.smithi094.stdout: "clog_to_syslog_level": "info", 2024-09-14T06:28:59.285 INFO:teuthology.orchestra.run.smithi094.stdout: "cluster_addr": "-", 2024-09-14T06:28:59.285 INFO:teuthology.orchestra.run.smithi094.stdout: "cluster_network": "", 2024-09-14T06:28:59.285 INFO:teuthology.orchestra.run.smithi094.stdout: "cluster_network_interface": "", 2024-09-14T06:28:59.285 INFO:teuthology.orchestra.run.smithi094.stdout: "compressor_zlib_isal": "false", 2024-09-14T06:28:59.285 INFO:teuthology.orchestra.run.smithi094.stdout: "compressor_zlib_level": "5", 2024-09-14T06:28:59.285 INFO:teuthology.orchestra.run.smithi094.stdout: "compressor_zlib_winsize": "-15", 2024-09-14T06:28:59.285 INFO:teuthology.orchestra.run.smithi094.stdout: "compressor_zstd_level": "1", 2024-09-14T06:28:59.286 INFO:teuthology.orchestra.run.smithi094.stdout: "container_image": "docker.io/ceph/daemon-base:latest-master-devel", 2024-09-14T06:28:59.286 INFO:teuthology.orchestra.run.smithi094.stdout: "crash_dir": "/var/lib/ceph/crash", 2024-09-14T06:28:59.286 INFO:teuthology.orchestra.run.smithi094.stdout: "crimson_alien_op_num_threads": "6", 2024-09-14T06:28:59.286 INFO:teuthology.orchestra.run.smithi094.stdout: "crimson_alien_thread_cpu_cores": "", 2024-09-14T06:28:59.286 INFO:teuthology.orchestra.run.smithi094.stdout: "crimson_osd_obc_lru_size": "512", 2024-09-14T06:28:59.286 INFO:teuthology.orchestra.run.smithi094.stdout: "crimson_osd_scheduler_concurrency": "0", 2024-09-14T06:28:59.286 INFO:teuthology.orchestra.run.smithi094.stdout: "crimson_osd_stat_interval": "0", 2024-09-14T06:28:59.286 INFO:teuthology.orchestra.run.smithi094.stdout: "crimson_seastar_cpu_cores": "", 2024-09-14T06:28:59.286 INFO:teuthology.orchestra.run.smithi094.stdout: "crimson_seastar_num_threads": "0", 2024-09-14T06:28:59.286 INFO:teuthology.orchestra.run.smithi094.stdout: "crush_location": "", 2024-09-14T06:28:59.286 INFO:teuthology.orchestra.run.smithi094.stdout: "crush_location_hook": "", 2024-09-14T06:28:59.286 INFO:teuthology.orchestra.run.smithi094.stdout: "crush_location_hook_timeout": "10", 2024-09-14T06:28:59.286 INFO:teuthology.orchestra.run.smithi094.stdout: "daemonize": "false", 2024-09-14T06:28:59.286 INFO:teuthology.orchestra.run.smithi094.stdout: "daos_pool": "tank", 2024-09-14T06:28:59.287 INFO:teuthology.orchestra.run.smithi094.stdout: "dbstore_config_uri": "file:/var/lib/ceph/radosgw/dbstore-config.db", 2024-09-14T06:28:59.287 INFO:teuthology.orchestra.run.smithi094.stdout: "dbstore_db_dir": "/var/lib/ceph/radosgw", 2024-09-14T06:28:59.287 INFO:teuthology.orchestra.run.smithi094.stdout: "dbstore_db_name_prefix": "dbstore", 2024-09-14T06:28:59.287 INFO:teuthology.orchestra.run.smithi094.stdout: "debug_alienstore": "0/5", 2024-09-14T06:28:59.287 INFO:teuthology.orchestra.run.smithi094.stdout: "debug_allow_any_pool_priority": "false", 2024-09-14T06:28:59.287 INFO:teuthology.orchestra.run.smithi094.stdout: "debug_asok": "1/5", 2024-09-14T06:28:59.287 INFO:teuthology.orchestra.run.smithi094.stdout: "debug_asok_assert_abort": "false", 2024-09-14T06:28:59.287 INFO:teuthology.orchestra.run.smithi094.stdout: "debug_asserts_on_shutdown": "true", 2024-09-14T06:28:59.287 INFO:teuthology.orchestra.run.smithi094.stdout: "debug_auth": "20/20", 2024-09-14T06:28:59.287 INFO:teuthology.orchestra.run.smithi094.stdout: "debug_bdev": "1/3", 2024-09-14T06:28:59.287 INFO:teuthology.orchestra.run.smithi094.stdout: "debug_bluefs": "1/5", 2024-09-14T06:28:59.287 INFO:teuthology.orchestra.run.smithi094.stdout: "debug_bluestore": "1/5", 2024-09-14T06:28:59.287 INFO:teuthology.orchestra.run.smithi094.stdout: "debug_buffer": "0/1", 2024-09-14T06:28:59.287 INFO:teuthology.orchestra.run.smithi094.stdout: "debug_ceph_dedup": "0/5", 2024-09-14T06:28:59.287 INFO:teuthology.orchestra.run.smithi094.stdout: "debug_ceph_exporter": "1/5", 2024-09-14T06:28:59.288 INFO:teuthology.orchestra.run.smithi094.stdout: "debug_cephfs_mirror": "0/5", 2024-09-14T06:28:59.288 INFO:teuthology.orchestra.run.smithi094.stdout: "debug_cephsqlite": "0/5", 2024-09-14T06:28:59.288 INFO:teuthology.orchestra.run.smithi094.stdout: "debug_client": "0/5", 2024-09-14T06:28:59.288 INFO:teuthology.orchestra.run.smithi094.stdout: "debug_compressor": "1/5", 2024-09-14T06:28:59.288 INFO:teuthology.orchestra.run.smithi094.stdout: "debug_context": "0/1", 2024-09-14T06:28:59.288 INFO:teuthology.orchestra.run.smithi094.stdout: "debug_crimson_interrupt": "0/5", 2024-09-14T06:28:59.288 INFO:teuthology.orchestra.run.smithi094.stdout: "debug_crush": "1/1", 2024-09-14T06:28:59.288 INFO:teuthology.orchestra.run.smithi094.stdout: "debug_crypto": "1/5", 2024-09-14T06:28:59.288 INFO:teuthology.orchestra.run.smithi094.stdout: "debug_cyanstore": "0/5", 2024-09-14T06:28:59.288 INFO:teuthology.orchestra.run.smithi094.stdout: "debug_deliberately_leak_memory": "false", 2024-09-14T06:28:59.288 INFO:teuthology.orchestra.run.smithi094.stdout: "debug_disable_randomized_ping": "false", 2024-09-14T06:28:59.288 INFO:teuthology.orchestra.run.smithi094.stdout: "debug_dpdk": "1/5", 2024-09-14T06:28:59.288 INFO:teuthology.orchestra.run.smithi094.stdout: "debug_eventtrace": "1/5", 2024-09-14T06:28:59.288 INFO:teuthology.orchestra.run.smithi094.stdout: "debug_filer": "0/1", 2024-09-14T06:28:59.288 INFO:teuthology.orchestra.run.smithi094.stdout: "debug_filestore": "1/3", 2024-09-14T06:28:59.289 INFO:teuthology.orchestra.run.smithi094.stdout: "debug_finisher": "1/1", 2024-09-14T06:28:59.289 INFO:teuthology.orchestra.run.smithi094.stdout: "debug_fuse": "1/5", 2024-09-14T06:28:59.289 INFO:teuthology.orchestra.run.smithi094.stdout: "debug_heartbeat_testing_span": "0", 2024-09-14T06:28:59.289 INFO:teuthology.orchestra.run.smithi094.stdout: "debug_heartbeatmap": "1/5", 2024-09-14T06:28:59.289 INFO:teuthology.orchestra.run.smithi094.stdout: "debug_immutable_obj_cache": "0/5", 2024-09-14T06:28:59.289 INFO:teuthology.orchestra.run.smithi094.stdout: "debug_javaclient": "1/5", 2024-09-14T06:28:59.289 INFO:teuthology.orchestra.run.smithi094.stdout: "debug_journal": "1/3", 2024-09-14T06:28:59.289 INFO:teuthology.orchestra.run.smithi094.stdout: "debug_journaler": "0/5", 2024-09-14T06:28:59.289 INFO:teuthology.orchestra.run.smithi094.stdout: "debug_kstore": "1/5", 2024-09-14T06:28:59.290 INFO:teuthology.orchestra.run.smithi094.stdout: "debug_lockdep": "0/1", 2024-09-14T06:28:59.290 INFO:teuthology.orchestra.run.smithi094.stdout: "debug_mclock": "1/5", 2024-09-14T06:28:59.290 INFO:teuthology.orchestra.run.smithi094.stdout: "debug_mds": "1/5", 2024-09-14T06:28:59.290 INFO:teuthology.orchestra.run.smithi094.stdout: "debug_mds_balancer": "1/5", 2024-09-14T06:28:59.290 INFO:teuthology.orchestra.run.smithi094.stdout: "debug_mds_locker": "1/5", 2024-09-14T06:28:59.290 INFO:teuthology.orchestra.run.smithi094.stdout: "debug_mds_log": "1/5", 2024-09-14T06:28:59.290 INFO:teuthology.orchestra.run.smithi094.stdout: "debug_mds_log_expire": "1/5", 2024-09-14T06:28:59.290 INFO:teuthology.orchestra.run.smithi094.stdout: "debug_mds_migrator": "1/5", 2024-09-14T06:28:59.290 INFO:teuthology.orchestra.run.smithi094.stdout: "debug_mds_quiesce": "3/5", 2024-09-14T06:28:59.290 INFO:teuthology.orchestra.run.smithi094.stdout: "debug_memstore": "1/5", 2024-09-14T06:28:59.290 INFO:teuthology.orchestra.run.smithi094.stdout: "debug_mgr": "2/5", 2024-09-14T06:28:59.290 INFO:teuthology.orchestra.run.smithi094.stdout: "debug_mgrc": "1/5", 2024-09-14T06:28:59.290 INFO:teuthology.orchestra.run.smithi094.stdout: "debug_mon": "20/20", 2024-09-14T06:28:59.290 INFO:teuthology.orchestra.run.smithi094.stdout: "debug_monc": "0/10", 2024-09-14T06:28:59.291 INFO:teuthology.orchestra.run.smithi094.stdout: "debug_ms": "1/1", 2024-09-14T06:28:59.291 INFO:teuthology.orchestra.run.smithi094.stdout: "debug_none": "0/5", 2024-09-14T06:28:59.291 INFO:teuthology.orchestra.run.smithi094.stdout: "debug_objclass": "0/5", 2024-09-14T06:28:59.291 INFO:teuthology.orchestra.run.smithi094.stdout: "debug_objectcacher": "0/5", 2024-09-14T06:28:59.291 INFO:teuthology.orchestra.run.smithi094.stdout: "debug_objecter": "0/1", 2024-09-14T06:28:59.291 INFO:teuthology.orchestra.run.smithi094.stdout: "debug_optracker": "0/5", 2024-09-14T06:28:59.291 INFO:teuthology.orchestra.run.smithi094.stdout: "debug_osd": "1/5", 2024-09-14T06:28:59.291 INFO:teuthology.orchestra.run.smithi094.stdout: "debug_paxos": "20/20", 2024-09-14T06:28:59.291 INFO:teuthology.orchestra.run.smithi094.stdout: "debug_perfcounter": "1/5", 2024-09-14T06:28:59.291 INFO:teuthology.orchestra.run.smithi094.stdout: "debug_prioritycache": "1/5", 2024-09-14T06:28:59.291 INFO:teuthology.orchestra.run.smithi094.stdout: "debug_rados": "0/5", 2024-09-14T06:28:59.291 INFO:teuthology.orchestra.run.smithi094.stdout: "debug_rbd": "0/5", 2024-09-14T06:28:59.291 INFO:teuthology.orchestra.run.smithi094.stdout: "debug_rbd_mirror": "0/5", 2024-09-14T06:28:59.291 INFO:teuthology.orchestra.run.smithi094.stdout: "debug_rbd_pwl": "0/5", 2024-09-14T06:28:59.292 INFO:teuthology.orchestra.run.smithi094.stdout: "debug_rbd_replay": "0/5", 2024-09-14T06:28:59.292 INFO:teuthology.orchestra.run.smithi094.stdout: "debug_refs": "0/0", 2024-09-14T06:28:59.292 INFO:teuthology.orchestra.run.smithi094.stdout: "debug_reserver": "1/1", 2024-09-14T06:28:59.292 INFO:teuthology.orchestra.run.smithi094.stdout: "debug_rgw": "1/5", 2024-09-14T06:28:59.292 INFO:teuthology.orchestra.run.smithi094.stdout: "debug_rgw_access": "1/5", 2024-09-14T06:28:59.292 INFO:teuthology.orchestra.run.smithi094.stdout: "debug_rgw_datacache": "1/5", 2024-09-14T06:28:59.292 INFO:teuthology.orchestra.run.smithi094.stdout: "debug_rgw_dbstore": "1/5", 2024-09-14T06:28:59.292 INFO:teuthology.orchestra.run.smithi094.stdout: "debug_rgw_flight": "1/5", 2024-09-14T06:28:59.292 INFO:teuthology.orchestra.run.smithi094.stdout: "debug_rgw_lifecycle": "1/5", 2024-09-14T06:28:59.292 INFO:teuthology.orchestra.run.smithi094.stdout: "debug_rgw_notification": "1/5", 2024-09-14T06:28:59.292 INFO:teuthology.orchestra.run.smithi094.stdout: "debug_rgw_sync": "1/5", 2024-09-14T06:28:59.292 INFO:teuthology.orchestra.run.smithi094.stdout: "debug_rocksdb": "4/5", 2024-09-14T06:28:59.292 INFO:teuthology.orchestra.run.smithi094.stdout: "debug_seastore": "0/5", 2024-09-14T06:28:59.292 INFO:teuthology.orchestra.run.smithi094.stdout: "debug_seastore_backref": "0/5", 2024-09-14T06:28:59.293 INFO:teuthology.orchestra.run.smithi094.stdout: "debug_seastore_cache": "0/5", 2024-09-14T06:28:59.293 INFO:teuthology.orchestra.run.smithi094.stdout: "debug_seastore_cleaner": "0/5", 2024-09-14T06:28:59.293 INFO:teuthology.orchestra.run.smithi094.stdout: "debug_seastore_device": "0/5", 2024-09-14T06:28:59.293 INFO:teuthology.orchestra.run.smithi094.stdout: "debug_seastore_epm": "0/5", 2024-09-14T06:28:59.293 INFO:teuthology.orchestra.run.smithi094.stdout: "debug_seastore_fixedkv_tree": "0/5", 2024-09-14T06:28:59.293 INFO:teuthology.orchestra.run.smithi094.stdout: "debug_seastore_journal": "0/5", 2024-09-14T06:28:59.293 INFO:teuthology.orchestra.run.smithi094.stdout: "debug_seastore_lba": "0/5", 2024-09-14T06:28:59.293 INFO:teuthology.orchestra.run.smithi094.stdout: "debug_seastore_odata": "0/5", 2024-09-14T06:28:59.293 INFO:teuthology.orchestra.run.smithi094.stdout: "debug_seastore_omap": "0/5", 2024-09-14T06:28:59.293 INFO:teuthology.orchestra.run.smithi094.stdout: "debug_seastore_onode": "0/5", 2024-09-14T06:28:59.293 INFO:teuthology.orchestra.run.smithi094.stdout: "debug_seastore_t": "0/5", 2024-09-14T06:28:59.293 INFO:teuthology.orchestra.run.smithi094.stdout: "debug_seastore_tm": "0/5", 2024-09-14T06:28:59.293 INFO:teuthology.orchestra.run.smithi094.stdout: "debug_striper": "0/1", 2024-09-14T06:28:59.294 INFO:teuthology.orchestra.run.smithi094.stdout: "debug_test": "0/5", 2024-09-14T06:28:59.294 INFO:teuthology.orchestra.run.smithi094.stdout: "debug_throttle": "1/1", 2024-09-14T06:28:59.294 INFO:teuthology.orchestra.run.smithi094.stdout: "debug_timer": "0/1", 2024-09-14T06:28:59.294 INFO:teuthology.orchestra.run.smithi094.stdout: "debug_tp": "0/5", 2024-09-14T06:28:59.294 INFO:teuthology.orchestra.run.smithi094.stdout: "debug_trace": "1/5", 2024-09-14T06:28:59.294 INFO:teuthology.orchestra.run.smithi094.stdout: "defer_client_eviction_on_laggy_osds": "false", 2024-09-14T06:28:59.294 INFO:teuthology.orchestra.run.smithi094.stdout: "device_failure_prediction_mode": "none", 2024-09-14T06:28:59.294 INFO:teuthology.orchestra.run.smithi094.stdout: "enable_experimental_unrecoverable_data_corrupting_features": "", 2024-09-14T06:28:59.294 INFO:teuthology.orchestra.run.smithi094.stdout: "erasure_code_dir": "/usr/lib64/ceph/erasure-code", 2024-09-14T06:28:59.294 INFO:teuthology.orchestra.run.smithi094.stdout: "err_to_graylog": "false", 2024-09-14T06:28:59.294 INFO:teuthology.orchestra.run.smithi094.stdout: "err_to_journald": "false", 2024-09-14T06:28:59.294 INFO:teuthology.orchestra.run.smithi094.stdout: "err_to_stderr": "true", 2024-09-14T06:28:59.294 INFO:teuthology.orchestra.run.smithi094.stdout: "err_to_syslog": "false", 2024-09-14T06:28:59.294 INFO:teuthology.orchestra.run.smithi094.stdout: "event_tracing": "false", 2024-09-14T06:28:59.294 INFO:teuthology.orchestra.run.smithi094.stdout: "exporter_addr": "0.0.0.0", 2024-09-14T06:28:59.295 INFO:teuthology.orchestra.run.smithi094.stdout: "exporter_cert_file": "", 2024-09-14T06:28:59.295 INFO:teuthology.orchestra.run.smithi094.stdout: "exporter_http_port": "9926", 2024-09-14T06:28:59.295 INFO:teuthology.orchestra.run.smithi094.stdout: "exporter_key_file": "", 2024-09-14T06:28:59.295 INFO:teuthology.orchestra.run.smithi094.stdout: "exporter_prio_limit": "5", 2024-09-14T06:28:59.295 INFO:teuthology.orchestra.run.smithi094.stdout: "exporter_sock_dir": "/var/run/ceph/", 2024-09-14T06:28:59.295 INFO:teuthology.orchestra.run.smithi094.stdout: "exporter_sort_metrics": "true", 2024-09-14T06:28:59.295 INFO:teuthology.orchestra.run.smithi094.stdout: "exporter_stats_period": "5", 2024-09-14T06:28:59.295 INFO:teuthology.orchestra.run.smithi094.stdout: "fake_statfs_for_testing": "0", 2024-09-14T06:28:59.295 INFO:teuthology.orchestra.run.smithi094.stdout: "fatal_signal_handlers": "true", 2024-09-14T06:28:59.295 INFO:teuthology.orchestra.run.smithi094.stdout: "filer_max_purge_ops": "10", 2024-09-14T06:28:59.295 INFO:teuthology.orchestra.run.smithi094.stdout: "filer_max_truncate_ops": "128", 2024-09-14T06:28:59.295 INFO:teuthology.orchestra.run.smithi094.stdout: "filestore_apply_finisher_threads": "1", 2024-09-14T06:28:59.295 INFO:teuthology.orchestra.run.smithi094.stdout: "filestore_blackhole": "false", 2024-09-14T06:28:59.295 INFO:teuthology.orchestra.run.smithi094.stdout: "filestore_btrfs_clone_range": "true", 2024-09-14T06:28:59.296 INFO:teuthology.orchestra.run.smithi094.stdout: "filestore_btrfs_snap": "true", 2024-09-14T06:28:59.296 INFO:teuthology.orchestra.run.smithi094.stdout: "filestore_caller_concurrency": "10", 2024-09-14T06:28:59.296 INFO:teuthology.orchestra.run.smithi094.stdout: "filestore_collect_device_partition_information": "true", 2024-09-14T06:28:59.296 INFO:teuthology.orchestra.run.smithi094.stdout: "filestore_commit_timeout": "600.000000", 2024-09-14T06:28:59.296 INFO:teuthology.orchestra.run.smithi094.stdout: "filestore_debug_inject_read_err": "false", 2024-09-14T06:28:59.296 INFO:teuthology.orchestra.run.smithi094.stdout: "filestore_debug_omap_check": "false", 2024-09-14T06:28:59.296 INFO:teuthology.orchestra.run.smithi094.stdout: "filestore_debug_verify_split": "false", 2024-09-14T06:28:59.296 INFO:teuthology.orchestra.run.smithi094.stdout: "filestore_dump_file": "", 2024-09-14T06:28:59.296 INFO:teuthology.orchestra.run.smithi094.stdout: "filestore_expected_throughput_bytes": "209715200.000000", 2024-09-14T06:28:59.296 INFO:teuthology.orchestra.run.smithi094.stdout: "filestore_expected_throughput_ops": "200.000000", 2024-09-14T06:28:59.296 INFO:teuthology.orchestra.run.smithi094.stdout: "filestore_fadvise": "true", 2024-09-14T06:28:59.296 INFO:teuthology.orchestra.run.smithi094.stdout: "filestore_fail_eio": "true", 2024-09-14T06:28:59.296 INFO:teuthology.orchestra.run.smithi094.stdout: "filestore_fd_cache_shards": "16", 2024-09-14T06:28:59.297 INFO:teuthology.orchestra.run.smithi094.stdout: "filestore_fd_cache_size": "128", 2024-09-14T06:28:59.297 INFO:teuthology.orchestra.run.smithi094.stdout: "filestore_fiemap": "false", 2024-09-14T06:28:59.297 INFO:teuthology.orchestra.run.smithi094.stdout: "filestore_fiemap_threshold": "4096", 2024-09-14T06:28:59.297 INFO:teuthology.orchestra.run.smithi094.stdout: "filestore_fsync_flushes_journal_data": "false", 2024-09-14T06:28:59.297 INFO:teuthology.orchestra.run.smithi094.stdout: "filestore_index_retry_probability": "0.000000", 2024-09-14T06:28:59.297 INFO:teuthology.orchestra.run.smithi094.stdout: "filestore_inject_stall": "0", 2024-09-14T06:28:59.297 INFO:teuthology.orchestra.run.smithi094.stdout: "filestore_journal_parallel": "false", 2024-09-14T06:28:59.297 INFO:teuthology.orchestra.run.smithi094.stdout: "filestore_journal_trailing": "false", 2024-09-14T06:28:59.297 INFO:teuthology.orchestra.run.smithi094.stdout: "filestore_journal_writeahead": "false", 2024-09-14T06:28:59.297 INFO:teuthology.orchestra.run.smithi094.stdout: "filestore_kill_at": "0", 2024-09-14T06:28:59.297 INFO:teuthology.orchestra.run.smithi094.stdout: "filestore_max_alloc_hint_size": "1048576", 2024-09-14T06:28:59.297 INFO:teuthology.orchestra.run.smithi094.stdout: "filestore_max_inline_xattr_size": "0", 2024-09-14T06:28:59.298 INFO:teuthology.orchestra.run.smithi094.stdout: "filestore_max_inline_xattr_size_btrfs": "2048", 2024-09-14T06:28:59.298 INFO:teuthology.orchestra.run.smithi094.stdout: "filestore_max_inline_xattr_size_other": "512", 2024-09-14T06:28:59.298 INFO:teuthology.orchestra.run.smithi094.stdout: "filestore_max_inline_xattr_size_xfs": "65536", 2024-09-14T06:28:59.298 INFO:teuthology.orchestra.run.smithi094.stdout: "filestore_max_inline_xattrs": "0", 2024-09-14T06:28:59.298 INFO:teuthology.orchestra.run.smithi094.stdout: "filestore_max_inline_xattrs_btrfs": "10", 2024-09-14T06:28:59.298 INFO:teuthology.orchestra.run.smithi094.stdout: "filestore_max_inline_xattrs_other": "2", 2024-09-14T06:28:59.298 INFO:teuthology.orchestra.run.smithi094.stdout: "filestore_max_inline_xattrs_xfs": "10", 2024-09-14T06:28:59.298 INFO:teuthology.orchestra.run.smithi094.stdout: "filestore_max_sync_interval": "5.000000", 2024-09-14T06:28:59.298 INFO:teuthology.orchestra.run.smithi094.stdout: "filestore_max_xattr_value_size": "0", 2024-09-14T06:28:59.298 INFO:teuthology.orchestra.run.smithi094.stdout: "filestore_max_xattr_value_size_btrfs": "65536", 2024-09-14T06:28:59.298 INFO:teuthology.orchestra.run.smithi094.stdout: "filestore_max_xattr_value_size_other": "1024", 2024-09-14T06:28:59.298 INFO:teuthology.orchestra.run.smithi094.stdout: "filestore_max_xattr_value_size_xfs": "65536", 2024-09-14T06:28:59.299 INFO:teuthology.orchestra.run.smithi094.stdout: "filestore_merge_threshold": "-10", 2024-09-14T06:28:59.299 INFO:teuthology.orchestra.run.smithi094.stdout: "filestore_min_sync_interval": "0.010000", 2024-09-14T06:28:59.299 INFO:teuthology.orchestra.run.smithi094.stdout: "filestore_odsync_write": "false", 2024-09-14T06:28:59.299 INFO:teuthology.orchestra.run.smithi094.stdout: "filestore_omap_backend": "rocksdb", 2024-09-14T06:28:59.299 INFO:teuthology.orchestra.run.smithi094.stdout: "filestore_omap_backend_path": "", 2024-09-14T06:28:59.299 INFO:teuthology.orchestra.run.smithi094.stdout: "filestore_omap_header_cache_size": "1024", 2024-09-14T06:28:59.299 INFO:teuthology.orchestra.run.smithi094.stdout: "filestore_ondisk_finisher_threads": "1", 2024-09-14T06:28:59.299 INFO:teuthology.orchestra.run.smithi094.stdout: "filestore_op_thread_suicide_timeout": "180", 2024-09-14T06:28:59.299 INFO:teuthology.orchestra.run.smithi094.stdout: "filestore_op_thread_timeout": "60", 2024-09-14T06:28:59.299 INFO:teuthology.orchestra.run.smithi094.stdout: "filestore_op_threads": "2", 2024-09-14T06:28:59.299 INFO:teuthology.orchestra.run.smithi094.stdout: "filestore_punch_hole": "false", 2024-09-14T06:28:59.299 INFO:teuthology.orchestra.run.smithi094.stdout: "filestore_queue_high_delay_multiple": "0.000000", 2024-09-14T06:28:59.299 INFO:teuthology.orchestra.run.smithi094.stdout: "filestore_queue_high_delay_multiple_bytes": "0.000000", 2024-09-14T06:28:59.299 INFO:teuthology.orchestra.run.smithi094.stdout: "filestore_queue_high_delay_multiple_ops": "0.000000", 2024-09-14T06:28:59.300 INFO:teuthology.orchestra.run.smithi094.stdout: "filestore_queue_high_threshhold": "0.900000", 2024-09-14T06:28:59.300 INFO:teuthology.orchestra.run.smithi094.stdout: "filestore_queue_low_threshhold": "0.300000", 2024-09-14T06:28:59.300 INFO:teuthology.orchestra.run.smithi094.stdout: "filestore_queue_max_bytes": "104857600", 2024-09-14T06:28:59.300 INFO:teuthology.orchestra.run.smithi094.stdout: "filestore_queue_max_delay_multiple": "0.000000", 2024-09-14T06:28:59.300 INFO:teuthology.orchestra.run.smithi094.stdout: "filestore_queue_max_delay_multiple_bytes": "0.000000", 2024-09-14T06:28:59.300 INFO:teuthology.orchestra.run.smithi094.stdout: "filestore_queue_max_delay_multiple_ops": "0.000000", 2024-09-14T06:28:59.300 INFO:teuthology.orchestra.run.smithi094.stdout: "filestore_queue_max_ops": "50", 2024-09-14T06:28:59.300 INFO:teuthology.orchestra.run.smithi094.stdout: "filestore_rocksdb_options": "max_background_jobs=10,compaction_readahead_size=2097152,compression=kNoCompression", 2024-09-14T06:28:59.300 INFO:teuthology.orchestra.run.smithi094.stdout: "filestore_seek_data_hole": "false", 2024-09-14T06:28:59.300 INFO:teuthology.orchestra.run.smithi094.stdout: "filestore_sloppy_crc": "false", 2024-09-14T06:28:59.300 INFO:teuthology.orchestra.run.smithi094.stdout: "filestore_sloppy_crc_block_size": "65536", 2024-09-14T06:28:59.300 INFO:teuthology.orchestra.run.smithi094.stdout: "filestore_splice": "false", 2024-09-14T06:28:59.300 INFO:teuthology.orchestra.run.smithi094.stdout: "filestore_split_multiple": "2", 2024-09-14T06:28:59.300 INFO:teuthology.orchestra.run.smithi094.stdout: "filestore_split_rand_factor": "20", 2024-09-14T06:28:59.301 INFO:teuthology.orchestra.run.smithi094.stdout: "filestore_update_to": "1000", 2024-09-14T06:28:59.301 INFO:teuthology.orchestra.run.smithi094.stdout: "filestore_wbthrottle_btrfs_bytes_hard_limit": "419430400", 2024-09-14T06:28:59.301 INFO:teuthology.orchestra.run.smithi094.stdout: "filestore_wbthrottle_btrfs_bytes_start_flusher": "41943040", 2024-09-14T06:28:59.301 INFO:teuthology.orchestra.run.smithi094.stdout: "filestore_wbthrottle_btrfs_inodes_hard_limit": "5000", 2024-09-14T06:28:59.301 INFO:teuthology.orchestra.run.smithi094.stdout: "filestore_wbthrottle_btrfs_inodes_start_flusher": "500", 2024-09-14T06:28:59.301 INFO:teuthology.orchestra.run.smithi094.stdout: "filestore_wbthrottle_btrfs_ios_hard_limit": "5000", 2024-09-14T06:28:59.301 INFO:teuthology.orchestra.run.smithi094.stdout: "filestore_wbthrottle_btrfs_ios_start_flusher": "500", 2024-09-14T06:28:59.301 INFO:teuthology.orchestra.run.smithi094.stdout: "filestore_wbthrottle_enable": "true", 2024-09-14T06:28:59.301 INFO:teuthology.orchestra.run.smithi094.stdout: "filestore_wbthrottle_xfs_bytes_hard_limit": "419430400", 2024-09-14T06:28:59.301 INFO:teuthology.orchestra.run.smithi094.stdout: "filestore_wbthrottle_xfs_bytes_start_flusher": "41943040", 2024-09-14T06:28:59.301 INFO:teuthology.orchestra.run.smithi094.stdout: "filestore_wbthrottle_xfs_inodes_hard_limit": "5000", 2024-09-14T06:28:59.301 INFO:teuthology.orchestra.run.smithi094.stdout: "filestore_wbthrottle_xfs_inodes_start_flusher": "500", 2024-09-14T06:28:59.301 INFO:teuthology.orchestra.run.smithi094.stdout: "filestore_wbthrottle_xfs_ios_hard_limit": "5000", 2024-09-14T06:28:59.302 INFO:teuthology.orchestra.run.smithi094.stdout: "filestore_wbthrottle_xfs_ios_start_flusher": "500", 2024-09-14T06:28:59.302 INFO:teuthology.orchestra.run.smithi094.stdout: "filestore_xfs_extsize": "false", 2024-09-14T06:28:59.302 INFO:teuthology.orchestra.run.smithi094.stdout: "filestore_zfs_snap": "false", 2024-09-14T06:28:59.302 INFO:teuthology.orchestra.run.smithi094.stdout: "fio_dir": "/tmp/fio", 2024-09-14T06:28:59.302 INFO:teuthology.orchestra.run.smithi094.stdout: "fsid": "200ad3da-bd63-4e04-af74-efb5ca0af75e", 2024-09-14T06:28:59.302 INFO:teuthology.orchestra.run.smithi094.stdout: "fuse_allow_other": "true", 2024-09-14T06:28:59.302 INFO:teuthology.orchestra.run.smithi094.stdout: "fuse_atomic_o_trunc": "true", 2024-09-14T06:28:59.302 INFO:teuthology.orchestra.run.smithi094.stdout: "fuse_big_writes": "true", 2024-09-14T06:28:59.302 INFO:teuthology.orchestra.run.smithi094.stdout: "fuse_debug": "false", 2024-09-14T06:28:59.302 INFO:teuthology.orchestra.run.smithi094.stdout: "fuse_default_permissions": "false", 2024-09-14T06:28:59.302 INFO:teuthology.orchestra.run.smithi094.stdout: "fuse_disable_pagecache": "false", 2024-09-14T06:28:59.302 INFO:teuthology.orchestra.run.smithi094.stdout: "fuse_max_write": "0", 2024-09-14T06:28:59.302 INFO:teuthology.orchestra.run.smithi094.stdout: "fuse_multithreaded": "true", 2024-09-14T06:28:59.302 INFO:teuthology.orchestra.run.smithi094.stdout: "fuse_require_active_mds": "true", 2024-09-14T06:28:59.302 INFO:teuthology.orchestra.run.smithi094.stdout: "fuse_set_user_groups": "true", 2024-09-14T06:28:59.303 INFO:teuthology.orchestra.run.smithi094.stdout: "fuse_splice_move": "true", 2024-09-14T06:28:59.303 INFO:teuthology.orchestra.run.smithi094.stdout: "fuse_splice_read": "true", 2024-09-14T06:28:59.303 INFO:teuthology.orchestra.run.smithi094.stdout: "fuse_splice_write": "true", 2024-09-14T06:28:59.303 INFO:teuthology.orchestra.run.smithi094.stdout: "fuse_syncfs_on_mksnap": "true", 2024-09-14T06:28:59.303 INFO:teuthology.orchestra.run.smithi094.stdout: "fuse_use_invalidate_cb": "true", 2024-09-14T06:28:59.303 INFO:teuthology.orchestra.run.smithi094.stdout: "gss_ktab_client_file": "/var/lib/ceph/mon.a/gss_client_mon.a.ktab", 2024-09-14T06:28:59.303 INFO:teuthology.orchestra.run.smithi094.stdout: "gss_target_name": "ceph", 2024-09-14T06:28:59.303 INFO:teuthology.orchestra.run.smithi094.stdout: "heartbeat_file": "", 2024-09-14T06:28:59.303 INFO:teuthology.orchestra.run.smithi094.stdout: "heartbeat_inject_failure": "0", 2024-09-14T06:28:59.303 INFO:teuthology.orchestra.run.smithi094.stdout: "heartbeat_interval": "5", 2024-09-14T06:28:59.303 INFO:teuthology.orchestra.run.smithi094.stdout: "host": "smithi094", 2024-09-14T06:28:59.303 INFO:teuthology.orchestra.run.smithi094.stdout: "immutable_object_cache_client_dedicated_thread_num": "2", 2024-09-14T06:28:59.303 INFO:teuthology.orchestra.run.smithi094.stdout: "immutable_object_cache_max_inflight_ops": "128", 2024-09-14T06:28:59.303 INFO:teuthology.orchestra.run.smithi094.stdout: "immutable_object_cache_max_size": "1073741824", 2024-09-14T06:28:59.304 INFO:teuthology.orchestra.run.smithi094.stdout: "immutable_object_cache_path": "/tmp/ceph_immutable_object_cache", 2024-09-14T06:28:59.304 INFO:teuthology.orchestra.run.smithi094.stdout: "immutable_object_cache_qos_bps_burst": "0", 2024-09-14T06:28:59.304 INFO:teuthology.orchestra.run.smithi094.stdout: "immutable_object_cache_qos_bps_burst_seconds": "1", 2024-09-14T06:28:59.304 INFO:teuthology.orchestra.run.smithi094.stdout: "immutable_object_cache_qos_bps_limit": "0", 2024-09-14T06:28:59.304 INFO:teuthology.orchestra.run.smithi094.stdout: "immutable_object_cache_qos_iops_burst": "0", 2024-09-14T06:28:59.304 INFO:teuthology.orchestra.run.smithi094.stdout: "immutable_object_cache_qos_iops_burst_seconds": "1", 2024-09-14T06:28:59.304 INFO:teuthology.orchestra.run.smithi094.stdout: "immutable_object_cache_qos_iops_limit": "0", 2024-09-14T06:28:59.304 INFO:teuthology.orchestra.run.smithi094.stdout: "immutable_object_cache_qos_schedule_tick_min": "50", 2024-09-14T06:28:59.304 INFO:teuthology.orchestra.run.smithi094.stdout: "immutable_object_cache_sock": "/var/run/ceph/immutable_object_cache_sock", 2024-09-14T06:28:59.304 INFO:teuthology.orchestra.run.smithi094.stdout: "immutable_object_cache_watermark": "0.900000", 2024-09-14T06:28:59.304 INFO:teuthology.orchestra.run.smithi094.stdout: "inject_early_sigterm": "false", 2024-09-14T06:28:59.304 INFO:teuthology.orchestra.run.smithi094.stdout: "jaeger_agent_port": "6799", 2024-09-14T06:28:59.304 INFO:teuthology.orchestra.run.smithi094.stdout: "jaeger_tracing_enable": "false", 2024-09-14T06:28:59.304 INFO:teuthology.orchestra.run.smithi094.stdout: "journal_aio": "true", 2024-09-14T06:28:59.305 INFO:teuthology.orchestra.run.smithi094.stdout: "journal_align_min_size": "65536", 2024-09-14T06:28:59.305 INFO:teuthology.orchestra.run.smithi094.stdout: "journal_block_align": "true", 2024-09-14T06:28:59.305 INFO:teuthology.orchestra.run.smithi094.stdout: "journal_block_size": "4096", 2024-09-14T06:28:59.305 INFO:teuthology.orchestra.run.smithi094.stdout: "journal_dio": "true", 2024-09-14T06:28:59.305 INFO:teuthology.orchestra.run.smithi094.stdout: "journal_discard": "false", 2024-09-14T06:28:59.305 INFO:teuthology.orchestra.run.smithi094.stdout: "journal_force_aio": "false", 2024-09-14T06:28:59.305 INFO:teuthology.orchestra.run.smithi094.stdout: "journal_ignore_corruption": "false", 2024-09-14T06:28:59.305 INFO:teuthology.orchestra.run.smithi094.stdout: "journal_max_write_bytes": "10485760", 2024-09-14T06:28:59.305 INFO:teuthology.orchestra.run.smithi094.stdout: "journal_max_write_entries": "100", 2024-09-14T06:28:59.305 INFO:teuthology.orchestra.run.smithi094.stdout: "journal_replay_from": "0", 2024-09-14T06:28:59.305 INFO:teuthology.orchestra.run.smithi094.stdout: "journal_throttle_high_multiple": "0.000000", 2024-09-14T06:28:59.305 INFO:teuthology.orchestra.run.smithi094.stdout: "journal_throttle_high_threshhold": "0.900000", 2024-09-14T06:28:59.305 INFO:teuthology.orchestra.run.smithi094.stdout: "journal_throttle_low_threshhold": "0.600000", 2024-09-14T06:28:59.305 INFO:teuthology.orchestra.run.smithi094.stdout: "journal_throttle_max_multiple": "0.000000", 2024-09-14T06:28:59.305 INFO:teuthology.orchestra.run.smithi094.stdout: "journal_write_header_frequency": "0", 2024-09-14T06:28:59.306 INFO:teuthology.orchestra.run.smithi094.stdout: "journal_zero_on_create": "false", 2024-09-14T06:28:59.306 INFO:teuthology.orchestra.run.smithi094.stdout: "journaler_prefetch_periods": "10", 2024-09-14T06:28:59.306 INFO:teuthology.orchestra.run.smithi094.stdout: "journaler_prezero_periods": "5", 2024-09-14T06:28:59.306 INFO:teuthology.orchestra.run.smithi094.stdout: "journaler_write_head_interval": "15", 2024-09-14T06:28:59.306 INFO:teuthology.orchestra.run.smithi094.stdout: "key": "", 2024-09-14T06:28:59.306 INFO:teuthology.orchestra.run.smithi094.stdout: "keyfile": "", 2024-09-14T06:28:59.306 INFO:teuthology.orchestra.run.smithi094.stdout: "keyring": "/var/lib/ceph/mon/ceph-a/keyring", 2024-09-14T06:28:59.306 INFO:teuthology.orchestra.run.smithi094.stdout: "kstore_backend": "rocksdb", 2024-09-14T06:28:59.306 INFO:teuthology.orchestra.run.smithi094.stdout: "kstore_default_stripe_size": "65536", 2024-09-14T06:28:59.306 INFO:teuthology.orchestra.run.smithi094.stdout: "kstore_fsck_on_mount": "false", 2024-09-14T06:28:59.306 INFO:teuthology.orchestra.run.smithi094.stdout: "kstore_fsck_on_mount_deep": "true", 2024-09-14T06:28:59.306 INFO:teuthology.orchestra.run.smithi094.stdout: "kstore_max_bytes": "67108864", 2024-09-14T06:28:59.306 INFO:teuthology.orchestra.run.smithi094.stdout: "kstore_max_ops": "512", 2024-09-14T06:28:59.307 INFO:teuthology.orchestra.run.smithi094.stdout: "kstore_nid_prealloc": "1024", 2024-09-14T06:28:59.307 INFO:teuthology.orchestra.run.smithi094.stdout: "kstore_onode_map_size": "1024", 2024-09-14T06:28:59.307 INFO:teuthology.orchestra.run.smithi094.stdout: "kstore_rocksdb_options": "compression=kNoCompression", 2024-09-14T06:28:59.307 INFO:teuthology.orchestra.run.smithi094.stdout: "kstore_sync_submit_transaction": "false", 2024-09-14T06:28:59.307 INFO:teuthology.orchestra.run.smithi094.stdout: "kstore_sync_transaction": "false", 2024-09-14T06:28:59.307 INFO:teuthology.orchestra.run.smithi094.stdout: "librados_thread_count": "2", 2024-09-14T06:28:59.307 INFO:teuthology.orchestra.run.smithi094.stdout: "lockdep": "false", 2024-09-14T06:28:59.307 INFO:teuthology.orchestra.run.smithi094.stdout: "lockdep_force_backtrace": "false", 2024-09-14T06:28:59.307 INFO:teuthology.orchestra.run.smithi094.stdout: "log_coarse_timestamps": "true", 2024-09-14T06:28:59.307 INFO:teuthology.orchestra.run.smithi094.stdout: "log_file": "/var/log/ceph/ceph-mon.a.log", 2024-09-14T06:28:59.307 INFO:teuthology.orchestra.run.smithi094.stdout: "log_flush_on_exit": "false", 2024-09-14T06:28:59.307 INFO:teuthology.orchestra.run.smithi094.stdout: "log_graylog_host": "127.0.0.1", 2024-09-14T06:28:59.307 INFO:teuthology.orchestra.run.smithi094.stdout: "log_graylog_port": "12201", 2024-09-14T06:28:59.307 INFO:teuthology.orchestra.run.smithi094.stdout: "log_max_new": "1000", 2024-09-14T06:28:59.308 INFO:teuthology.orchestra.run.smithi094.stdout: "log_max_recent": "10000", 2024-09-14T06:28:59.308 INFO:teuthology.orchestra.run.smithi094.stdout: "log_stderr_prefix": "", 2024-09-14T06:28:59.308 INFO:teuthology.orchestra.run.smithi094.stdout: "log_stop_at_utilization": "0.970000", 2024-09-14T06:28:59.308 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_file": "true", 2024-09-14T06:28:59.308 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_graylog": "false", 2024-09-14T06:28:59.308 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_journald": "false", 2024-09-14T06:28:59.308 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_stderr": "false", 2024-09-14T06:28:59.308 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_syslog": "false", 2024-09-14T06:28:59.308 INFO:teuthology.orchestra.run.smithi094.stdout: "max_rotating_auth_attempts": "10", 2024-09-14T06:28:59.308 INFO:teuthology.orchestra.run.smithi094.stdout: "mds_abort_on_newly_corrupt_dentry": "true", 2024-09-14T06:28:59.308 INFO:teuthology.orchestra.run.smithi094.stdout: "mds_action_on_write_error": "1", 2024-09-14T06:28:59.308 INFO:teuthology.orchestra.run.smithi094.stdout: "mds_alternate_name_max": "8192", 2024-09-14T06:28:59.308 INFO:teuthology.orchestra.run.smithi094.stdout: "mds_asio_thread_count": "2", 2024-09-14T06:28:59.308 INFO:teuthology.orchestra.run.smithi094.stdout: "mds_bal_export_pin": "true", 2024-09-14T06:28:59.309 INFO:teuthology.orchestra.run.smithi094.stdout: "mds_bal_fragment_dirs": "true", 2024-09-14T06:28:59.309 INFO:teuthology.orchestra.run.smithi094.stdout: "mds_bal_fragment_fast_factor": "1.500000", 2024-09-14T06:28:59.309 INFO:teuthology.orchestra.run.smithi094.stdout: "mds_bal_fragment_interval": "5", 2024-09-14T06:28:59.309 INFO:teuthology.orchestra.run.smithi094.stdout: "mds_bal_fragment_size_max": "100000", 2024-09-14T06:28:59.309 INFO:teuthology.orchestra.run.smithi094.stdout: "mds_bal_idle_threshold": "0.000000", 2024-09-14T06:28:59.309 INFO:teuthology.orchestra.run.smithi094.stdout: "mds_bal_interval": "10", 2024-09-14T06:28:59.309 INFO:teuthology.orchestra.run.smithi094.stdout: "mds_bal_max": "-1", 2024-09-14T06:28:59.309 INFO:teuthology.orchestra.run.smithi094.stdout: "mds_bal_max_until": "-1", 2024-09-14T06:28:59.309 INFO:teuthology.orchestra.run.smithi094.stdout: "mds_bal_merge_size": "50", 2024-09-14T06:28:59.309 INFO:teuthology.orchestra.run.smithi094.stdout: "mds_bal_midchunk": "0.300000", 2024-09-14T06:28:59.309 INFO:teuthology.orchestra.run.smithi094.stdout: "mds_bal_min_rebalance": "0.100000", 2024-09-14T06:28:59.309 INFO:teuthology.orchestra.run.smithi094.stdout: "mds_bal_min_start": "0.200000", 2024-09-14T06:28:59.309 INFO:teuthology.orchestra.run.smithi094.stdout: "mds_bal_minchunk": "0.001000", 2024-09-14T06:28:59.310 INFO:teuthology.orchestra.run.smithi094.stdout: "mds_bal_mode": "0", 2024-09-14T06:28:59.310 INFO:teuthology.orchestra.run.smithi094.stdout: "mds_bal_need_max": "1.200000", 2024-09-14T06:28:59.310 INFO:teuthology.orchestra.run.smithi094.stdout: "mds_bal_need_min": "0.800000", 2024-09-14T06:28:59.310 INFO:teuthology.orchestra.run.smithi094.stdout: "mds_bal_overload_epochs": "2", 2024-09-14T06:28:59.310 INFO:teuthology.orchestra.run.smithi094.stdout: "mds_bal_replicate_threshold": "8000.000000", 2024-09-14T06:28:59.310 INFO:teuthology.orchestra.run.smithi094.stdout: "mds_bal_sample_interval": "3.000000", 2024-09-14T06:28:59.310 INFO:teuthology.orchestra.run.smithi094.stdout: "mds_bal_split_bits": "3", 2024-09-14T06:28:59.310 INFO:teuthology.orchestra.run.smithi094.stdout: "mds_bal_split_rd": "25000.000000", 2024-09-14T06:28:59.310 INFO:teuthology.orchestra.run.smithi094.stdout: "mds_bal_split_size": "10000", 2024-09-14T06:28:59.310 INFO:teuthology.orchestra.run.smithi094.stdout: "mds_bal_split_wr": "10000.000000", 2024-09-14T06:28:59.310 INFO:teuthology.orchestra.run.smithi094.stdout: "mds_bal_target_decay": "10.000000", 2024-09-14T06:28:59.310 INFO:teuthology.orchestra.run.smithi094.stdout: "mds_bal_unreplicate_threshold": "0.000000", 2024-09-14T06:28:59.310 INFO:teuthology.orchestra.run.smithi094.stdout: "mds_beacon_grace": "15.000000", 2024-09-14T06:28:59.310 INFO:teuthology.orchestra.run.smithi094.stdout: "mds_beacon_interval": "4.000000", 2024-09-14T06:28:59.311 INFO:teuthology.orchestra.run.smithi094.stdout: "mds_beacon_mon_down_grace": "60", 2024-09-14T06:28:59.311 INFO:teuthology.orchestra.run.smithi094.stdout: "mds_cache_memory_limit": "4294967296", 2024-09-14T06:28:59.311 INFO:teuthology.orchestra.run.smithi094.stdout: "mds_cache_mid": "0.700000", 2024-09-14T06:28:59.311 INFO:teuthology.orchestra.run.smithi094.stdout: "mds_cache_quiesce_decay_rate": "1.000000", 2024-09-14T06:28:59.311 INFO:teuthology.orchestra.run.smithi094.stdout: "mds_cache_quiesce_delay": "0", 2024-09-14T06:28:59.311 INFO:teuthology.orchestra.run.smithi094.stdout: "mds_cache_quiesce_sleep": "200", 2024-09-14T06:28:59.311 INFO:teuthology.orchestra.run.smithi094.stdout: "mds_cache_quiesce_splitauth": "true", 2024-09-14T06:28:59.311 INFO:teuthology.orchestra.run.smithi094.stdout: "mds_cache_quiesce_threshold": "524288", 2024-09-14T06:28:59.311 INFO:teuthology.orchestra.run.smithi094.stdout: "mds_cache_release_free_interval": "10", 2024-09-14T06:28:59.311 INFO:teuthology.orchestra.run.smithi094.stdout: "mds_cache_reservation": "0.050000", 2024-09-14T06:28:59.311 INFO:teuthology.orchestra.run.smithi094.stdout: "mds_cache_trim_decay_rate": "1.000000", 2024-09-14T06:28:59.311 INFO:teuthology.orchestra.run.smithi094.stdout: "mds_cache_trim_interval": "1", 2024-09-14T06:28:59.311 INFO:teuthology.orchestra.run.smithi094.stdout: "mds_cache_trim_threshold": "262144", 2024-09-14T06:28:59.311 INFO:teuthology.orchestra.run.smithi094.stdout: "mds_cap_acquisition_throttle_retry_request_timeout": "0.500000", 2024-09-14T06:28:59.312 INFO:teuthology.orchestra.run.smithi094.stdout: "mds_cap_revoke_eviction_timeout": "0.000000", 2024-09-14T06:28:59.312 INFO:teuthology.orchestra.run.smithi094.stdout: "mds_client_delegate_inos_pct": "50", 2024-09-14T06:28:59.312 INFO:teuthology.orchestra.run.smithi094.stdout: "mds_client_prealloc_inos": "1000", 2024-09-14T06:28:59.312 INFO:teuthology.orchestra.run.smithi094.stdout: "mds_client_writeable_range_max_inc_objs": "1024", 2024-09-14T06:28:59.312 INFO:teuthology.orchestra.run.smithi094.stdout: "mds_connect_bootstrapping": "false", 2024-09-14T06:28:59.312 INFO:teuthology.orchestra.run.smithi094.stdout: "mds_damage_table_max_entries": "10000", 2024-09-14T06:28:59.312 INFO:teuthology.orchestra.run.smithi094.stdout: "mds_data": "/var/lib/ceph/mds/ceph-a", 2024-09-14T06:28:59.312 INFO:teuthology.orchestra.run.smithi094.stdout: "mds_debug_auth_pins": "false", 2024-09-14T06:28:59.312 INFO:teuthology.orchestra.run.smithi094.stdout: "mds_debug_frag": "false", 2024-09-14T06:28:59.312 INFO:teuthology.orchestra.run.smithi094.stdout: "mds_debug_scatterstat": "false", 2024-09-14T06:28:59.312 INFO:teuthology.orchestra.run.smithi094.stdout: "mds_debug_subtrees": "false", 2024-09-14T06:28:59.312 INFO:teuthology.orchestra.run.smithi094.stdout: "mds_decay_halflife": "5.000000", 2024-09-14T06:28:59.312 INFO:teuthology.orchestra.run.smithi094.stdout: "mds_default_dir_hash": "2", 2024-09-14T06:28:59.312 INFO:teuthology.orchestra.run.smithi094.stdout: "mds_defer_session_stale": "true", 2024-09-14T06:28:59.313 INFO:teuthology.orchestra.run.smithi094.stdout: "mds_deny_all_reconnect": "false", 2024-09-14T06:28:59.313 INFO:teuthology.orchestra.run.smithi094.stdout: "mds_dir_keys_per_op": "16384", 2024-09-14T06:28:59.313 INFO:teuthology.orchestra.run.smithi094.stdout: "mds_dir_max_commit_size": "10", 2024-09-14T06:28:59.313 INFO:teuthology.orchestra.run.smithi094.stdout: "mds_dir_max_entries": "0", 2024-09-14T06:28:59.313 INFO:teuthology.orchestra.run.smithi094.stdout: "mds_dir_prefetch": "true", 2024-09-14T06:28:59.313 INFO:teuthology.orchestra.run.smithi094.stdout: "mds_dirstat_min_interval": "1.000000", 2024-09-14T06:28:59.313 INFO:teuthology.orchestra.run.smithi094.stdout: "mds_dump_cache_after_rejoin": "false", 2024-09-14T06:28:59.313 INFO:teuthology.orchestra.run.smithi094.stdout: "mds_dump_cache_on_map": "false", 2024-09-14T06:28:59.313 INFO:teuthology.orchestra.run.smithi094.stdout: "mds_dump_cache_threshold_file": "0", 2024-09-14T06:28:59.313 INFO:teuthology.orchestra.run.smithi094.stdout: "mds_dump_cache_threshold_formatter": "1073741824", 2024-09-14T06:28:59.313 INFO:teuthology.orchestra.run.smithi094.stdout: "mds_early_reply": "true", 2024-09-14T06:28:59.313 INFO:teuthology.orchestra.run.smithi094.stdout: "mds_enable_op_tracker": "true", 2024-09-14T06:28:59.313 INFO:teuthology.orchestra.run.smithi094.stdout: "mds_enforce_unique_name": "true", 2024-09-14T06:28:59.314 INFO:teuthology.orchestra.run.smithi094.stdout: "mds_export_ephemeral_distributed": "true", 2024-09-14T06:28:59.314 INFO:teuthology.orchestra.run.smithi094.stdout: "mds_export_ephemeral_distributed_factor": "2.000000", 2024-09-14T06:28:59.314 INFO:teuthology.orchestra.run.smithi094.stdout: "mds_export_ephemeral_random": "true", 2024-09-14T06:28:59.314 INFO:teuthology.orchestra.run.smithi094.stdout: "mds_export_ephemeral_random_max": "0.010000", 2024-09-14T06:28:59.314 INFO:teuthology.orchestra.run.smithi094.stdout: "mds_extraordinary_events_dump_interval": "0", 2024-09-14T06:28:59.314 INFO:teuthology.orchestra.run.smithi094.stdout: "mds_forward_all_requests_to_auth": "false", 2024-09-14T06:28:59.314 INFO:teuthology.orchestra.run.smithi094.stdout: "mds_freeze_tree_timeout": "30.000000", 2024-09-14T06:28:59.314 INFO:teuthology.orchestra.run.smithi094.stdout: "mds_fscrypt_last_block_max_size": "4096", 2024-09-14T06:28:59.314 INFO:teuthology.orchestra.run.smithi094.stdout: "mds_go_bad_corrupt_dentry": "true", 2024-09-14T06:28:59.314 INFO:teuthology.orchestra.run.smithi094.stdout: "mds_hack_allow_loading_invalid_metadata": "false", 2024-09-14T06:28:59.314 INFO:teuthology.orchestra.run.smithi094.stdout: "mds_health_cache_threshold": "1.500000", 2024-09-14T06:28:59.314 INFO:teuthology.orchestra.run.smithi094.stdout: "mds_health_summarize_threshold": "10", 2024-09-14T06:28:59.314 INFO:teuthology.orchestra.run.smithi094.stdout: "mds_heartbeat_grace": "15.000000", 2024-09-14T06:28:59.314 INFO:teuthology.orchestra.run.smithi094.stdout: "mds_heartbeat_reset_grace": "1000", 2024-09-14T06:28:59.314 INFO:teuthology.orchestra.run.smithi094.stdout: "mds_inject_health_dummy": "false", 2024-09-14T06:28:59.315 INFO:teuthology.orchestra.run.smithi094.stdout: "mds_inject_journal_corrupt_dentry_first": "0.000000", 2024-09-14T06:28:59.315 INFO:teuthology.orchestra.run.smithi094.stdout: "mds_inject_migrator_session_race": "false", 2024-09-14T06:28:59.315 INFO:teuthology.orchestra.run.smithi094.stdout: "mds_inject_rename_corrupt_dentry_first": "0.000000", 2024-09-14T06:28:59.315 INFO:teuthology.orchestra.run.smithi094.stdout: "mds_inject_skip_replaying_inotable": "false", 2024-09-14T06:28:59.315 INFO:teuthology.orchestra.run.smithi094.stdout: "mds_inject_traceless_reply_probability": "0.000000", 2024-09-14T06:28:59.315 INFO:teuthology.orchestra.run.smithi094.stdout: "mds_join_fs": "", 2024-09-14T06:28:59.315 INFO:teuthology.orchestra.run.smithi094.stdout: "mds_journal_format": "1", 2024-09-14T06:28:59.315 INFO:teuthology.orchestra.run.smithi094.stdout: "mds_kill_after_journal_logs_flushed": "false", 2024-09-14T06:28:59.315 INFO:teuthology.orchestra.run.smithi094.stdout: "mds_kill_create_at": "0", 2024-09-14T06:28:59.315 INFO:teuthology.orchestra.run.smithi094.stdout: "mds_kill_dirfrag_at": "0", 2024-09-14T06:28:59.315 INFO:teuthology.orchestra.run.smithi094.stdout: "mds_kill_export_at": "0", 2024-09-14T06:28:59.315 INFO:teuthology.orchestra.run.smithi094.stdout: "mds_kill_import_at": "0", 2024-09-14T06:28:59.315 INFO:teuthology.orchestra.run.smithi094.stdout: "mds_kill_journal_at": "0", 2024-09-14T06:28:59.315 INFO:teuthology.orchestra.run.smithi094.stdout: "mds_kill_journal_expire_at": "0", 2024-09-14T06:28:59.316 INFO:teuthology.orchestra.run.smithi094.stdout: "mds_kill_journal_replay_at": "0", 2024-09-14T06:28:59.316 INFO:teuthology.orchestra.run.smithi094.stdout: "mds_kill_link_at": "0", 2024-09-14T06:28:59.316 INFO:teuthology.orchestra.run.smithi094.stdout: "mds_kill_mdstable_at": "0", 2024-09-14T06:28:59.316 INFO:teuthology.orchestra.run.smithi094.stdout: "mds_kill_openc_at": "0", 2024-09-14T06:28:59.316 INFO:teuthology.orchestra.run.smithi094.stdout: "mds_kill_rename_at": "0", 2024-09-14T06:28:59.316 INFO:teuthology.orchestra.run.smithi094.stdout: "mds_kill_shutdown_at": "0", 2024-09-14T06:28:59.316 INFO:teuthology.orchestra.run.smithi094.stdout: "mds_log_event_large_threshold": "524288", 2024-09-14T06:28:59.316 INFO:teuthology.orchestra.run.smithi094.stdout: "mds_log_events_per_segment": "1024", 2024-09-14T06:28:59.316 INFO:teuthology.orchestra.run.smithi094.stdout: "mds_log_major_segment_event_ratio": "12", 2024-09-14T06:28:59.316 INFO:teuthology.orchestra.run.smithi094.stdout: "mds_log_max_events": "-1", 2024-09-14T06:28:59.316 INFO:teuthology.orchestra.run.smithi094.stdout: "mds_log_max_segments": "128", 2024-09-14T06:28:59.316 INFO:teuthology.orchestra.run.smithi094.stdout: "mds_log_pause": "false", 2024-09-14T06:28:59.316 INFO:teuthology.orchestra.run.smithi094.stdout: "mds_log_segment_size": "0", 2024-09-14T06:28:59.316 INFO:teuthology.orchestra.run.smithi094.stdout: "mds_log_skip_corrupt_events": "false", 2024-09-14T06:28:59.317 INFO:teuthology.orchestra.run.smithi094.stdout: "mds_log_skip_unbounded_events": "false", 2024-09-14T06:28:59.317 INFO:teuthology.orchestra.run.smithi094.stdout: "mds_log_trim_decay_rate": "1.000000", 2024-09-14T06:28:59.317 INFO:teuthology.orchestra.run.smithi094.stdout: "mds_log_trim_threshold": "128", 2024-09-14T06:28:59.317 INFO:teuthology.orchestra.run.smithi094.stdout: "mds_log_trim_upkeep_interval": "1000", 2024-09-14T06:28:59.317 INFO:teuthology.orchestra.run.smithi094.stdout: "mds_log_warn_factor": "2.000000", 2024-09-14T06:28:59.317 INFO:teuthology.orchestra.run.smithi094.stdout: "mds_max_caps_per_client": "1048576", 2024-09-14T06:28:59.317 INFO:teuthology.orchestra.run.smithi094.stdout: "mds_max_completed_flushes": "100000", 2024-09-14T06:28:59.317 INFO:teuthology.orchestra.run.smithi094.stdout: "mds_max_completed_requests": "100000", 2024-09-14T06:28:59.317 INFO:teuthology.orchestra.run.smithi094.stdout: "mds_max_export_size": "20971520", 2024-09-14T06:28:59.317 INFO:teuthology.orchestra.run.smithi094.stdout: "mds_max_file_recover": "32", 2024-09-14T06:28:59.317 INFO:teuthology.orchestra.run.smithi094.stdout: "mds_max_purge_files": "64", 2024-09-14T06:28:59.317 INFO:teuthology.orchestra.run.smithi094.stdout: "mds_max_purge_ops": "8192", 2024-09-14T06:28:59.317 INFO:teuthology.orchestra.run.smithi094.stdout: "mds_max_purge_ops_per_pg": "0.500000", 2024-09-14T06:28:59.318 INFO:teuthology.orchestra.run.smithi094.stdout: "mds_max_scrub_ops_in_progress": "5", 2024-09-14T06:28:59.318 INFO:teuthology.orchestra.run.smithi094.stdout: "mds_max_snaps_per_dir": "100", 2024-09-14T06:28:59.318 INFO:teuthology.orchestra.run.smithi094.stdout: "mds_metrics_update_interval": "2", 2024-09-14T06:28:59.318 INFO:teuthology.orchestra.run.smithi094.stdout: "mds_min_caps_per_client": "100", 2024-09-14T06:28:59.318 INFO:teuthology.orchestra.run.smithi094.stdout: "mds_min_caps_working_set": "10000", 2024-09-14T06:28:59.318 INFO:teuthology.orchestra.run.smithi094.stdout: "mds_mon_shutdown_timeout": "5.000000", 2024-09-14T06:28:59.318 INFO:teuthology.orchestra.run.smithi094.stdout: "mds_numa_node": "-1", 2024-09-14T06:28:59.318 INFO:teuthology.orchestra.run.smithi094.stdout: "mds_oft_prefetch_dirfrags": "false", 2024-09-14T06:28:59.318 INFO:teuthology.orchestra.run.smithi094.stdout: "mds_op_complaint_time": "30.000000", 2024-09-14T06:28:59.318 INFO:teuthology.orchestra.run.smithi094.stdout: "mds_op_history_duration": "600", 2024-09-14T06:28:59.318 INFO:teuthology.orchestra.run.smithi094.stdout: "mds_op_history_size": "20", 2024-09-14T06:28:59.318 INFO:teuthology.orchestra.run.smithi094.stdout: "mds_op_history_slow_op_size": "20", 2024-09-14T06:28:59.318 INFO:teuthology.orchestra.run.smithi094.stdout: "mds_op_history_slow_op_threshold": "10", 2024-09-14T06:28:59.318 INFO:teuthology.orchestra.run.smithi094.stdout: "mds_op_log_threshold": "5", 2024-09-14T06:28:59.319 INFO:teuthology.orchestra.run.smithi094.stdout: "mds_ping_grace": "15", 2024-09-14T06:28:59.319 INFO:teuthology.orchestra.run.smithi094.stdout: "mds_ping_interval": "5", 2024-09-14T06:28:59.319 INFO:teuthology.orchestra.run.smithi094.stdout: "mds_purge_queue_busy_flush_period": "1.000000", 2024-09-14T06:28:59.319 INFO:teuthology.orchestra.run.smithi094.stdout: "mds_recall_global_max_decay_threshold": "131072", 2024-09-14T06:28:59.319 INFO:teuthology.orchestra.run.smithi094.stdout: "mds_recall_max_caps": "30000", 2024-09-14T06:28:59.319 INFO:teuthology.orchestra.run.smithi094.stdout: "mds_recall_max_decay_rate": "1.500000", 2024-09-14T06:28:59.319 INFO:teuthology.orchestra.run.smithi094.stdout: "mds_recall_max_decay_threshold": "131072", 2024-09-14T06:28:59.319 INFO:teuthology.orchestra.run.smithi094.stdout: "mds_recall_warning_decay_rate": "60.000000", 2024-09-14T06:28:59.319 INFO:teuthology.orchestra.run.smithi094.stdout: "mds_recall_warning_threshold": "262144", 2024-09-14T06:28:59.319 INFO:teuthology.orchestra.run.smithi094.stdout: "mds_reconnect_timeout": "45.000000", 2024-09-14T06:28:59.319 INFO:teuthology.orchestra.run.smithi094.stdout: "mds_replay_interval": "1.000000", 2024-09-14T06:28:59.319 INFO:teuthology.orchestra.run.smithi094.stdout: "mds_replay_unsafe_with_closed_session": "false", 2024-09-14T06:28:59.319 INFO:teuthology.orchestra.run.smithi094.stdout: "mds_request_load_average_decay_rate": "60.000000", 2024-09-14T06:28:59.319 INFO:teuthology.orchestra.run.smithi094.stdout: "mds_root_ino_gid": "0", 2024-09-14T06:28:59.320 INFO:teuthology.orchestra.run.smithi094.stdout: "mds_root_ino_uid": "0", 2024-09-14T06:28:59.320 INFO:teuthology.orchestra.run.smithi094.stdout: "mds_scatter_nudge_interval": "5.000000", 2024-09-14T06:28:59.320 INFO:teuthology.orchestra.run.smithi094.stdout: "mds_scrub_stats_review_period": "1", 2024-09-14T06:28:59.320 INFO:teuthology.orchestra.run.smithi094.stdout: "mds_server_dispatch_client_request_delay": "0", 2024-09-14T06:28:59.320 INFO:teuthology.orchestra.run.smithi094.stdout: "mds_server_dispatch_killpoint_random": "0.000000", 2024-09-14T06:28:59.320 INFO:teuthology.orchestra.run.smithi094.stdout: "mds_session_blocklist_on_evict": "true", 2024-09-14T06:28:59.320 INFO:teuthology.orchestra.run.smithi094.stdout: "mds_session_blocklist_on_timeout": "true", 2024-09-14T06:28:59.320 INFO:teuthology.orchestra.run.smithi094.stdout: "mds_session_cache_liveness_decay_rate": "300.000000", 2024-09-14T06:28:59.320 INFO:teuthology.orchestra.run.smithi094.stdout: "mds_session_cache_liveness_magnitude": "10", 2024-09-14T06:28:59.320 INFO:teuthology.orchestra.run.smithi094.stdout: "mds_session_cap_acquisition_decay_rate": "30.000000", 2024-09-14T06:28:59.320 INFO:teuthology.orchestra.run.smithi094.stdout: "mds_session_cap_acquisition_throttle": "100000", 2024-09-14T06:28:59.320 INFO:teuthology.orchestra.run.smithi094.stdout: "mds_session_max_caps_throttle_ratio": "1.100000", 2024-09-14T06:28:59.320 INFO:teuthology.orchestra.run.smithi094.stdout: "mds_session_metadata_threshold": "16777216", 2024-09-14T06:28:59.320 INFO:teuthology.orchestra.run.smithi094.stdout: "mds_sessionmap_keys_per_op": "1024", 2024-09-14T06:28:59.320 INFO:teuthology.orchestra.run.smithi094.stdout: "mds_shutdown_check": "0", 2024-09-14T06:28:59.321 INFO:teuthology.orchestra.run.smithi094.stdout: "mds_skip_ino": "0", 2024-09-14T06:28:59.321 INFO:teuthology.orchestra.run.smithi094.stdout: "mds_sleep_rank_change": "0.000000", 2024-09-14T06:28:59.321 INFO:teuthology.orchestra.run.smithi094.stdout: "mds_snap_max_uid": "4294967294", 2024-09-14T06:28:59.321 INFO:teuthology.orchestra.run.smithi094.stdout: "mds_snap_min_uid": "0", 2024-09-14T06:28:59.321 INFO:teuthology.orchestra.run.smithi094.stdout: "mds_snap_rstat": "false", 2024-09-14T06:28:59.321 INFO:teuthology.orchestra.run.smithi094.stdout: "mds_standby_replay_damaged": "false", 2024-09-14T06:28:59.321 INFO:teuthology.orchestra.run.smithi094.stdout: "mds_symlink_recovery": "true", 2024-09-14T06:28:59.321 INFO:teuthology.orchestra.run.smithi094.stdout: "mds_task_status_update_interval": "2.000000", 2024-09-14T06:28:59.321 INFO:teuthology.orchestra.run.smithi094.stdout: "mds_thrash_exports": "0", 2024-09-14T06:28:59.321 INFO:teuthology.orchestra.run.smithi094.stdout: "mds_thrash_fragments": "0", 2024-09-14T06:28:59.321 INFO:teuthology.orchestra.run.smithi094.stdout: "mds_tick_interval": "5.000000", 2024-09-14T06:28:59.321 INFO:teuthology.orchestra.run.smithi094.stdout: "mds_valgrind_exit": "false", 2024-09-14T06:28:59.321 INFO:teuthology.orchestra.run.smithi094.stdout: "mds_verify_backtrace": "1", 2024-09-14T06:28:59.321 INFO:teuthology.orchestra.run.smithi094.stdout: "mds_verify_scatter": "false", 2024-09-14T06:28:59.322 INFO:teuthology.orchestra.run.smithi094.stdout: "mds_wipe_ino_prealloc": "false", 2024-09-14T06:28:59.322 INFO:teuthology.orchestra.run.smithi094.stdout: "mds_wipe_sessions": "false", 2024-09-14T06:28:59.322 INFO:teuthology.orchestra.run.smithi094.stdout: "mempool_debug": "false", 2024-09-14T06:28:59.322 INFO:teuthology.orchestra.run.smithi094.stdout: "memstore_debug_omit_block_device_write": "false", 2024-09-14T06:28:59.322 INFO:teuthology.orchestra.run.smithi094.stdout: "memstore_device_bytes": "1073741824", 2024-09-14T06:28:59.322 INFO:teuthology.orchestra.run.smithi094.stdout: "memstore_page_set": "false", 2024-09-14T06:28:59.322 INFO:teuthology.orchestra.run.smithi094.stdout: "memstore_page_size": "65536", 2024-09-14T06:28:59.322 INFO:teuthology.orchestra.run.smithi094.stdout: "mgr_client_bytes": "134217728", 2024-09-14T06:28:59.322 INFO:teuthology.orchestra.run.smithi094.stdout: "mgr_client_messages": "512", 2024-09-14T06:28:59.322 INFO:teuthology.orchestra.run.smithi094.stdout: "mgr_client_service_daemon_unregister_timeout": "1.000000", 2024-09-14T06:28:59.322 INFO:teuthology.orchestra.run.smithi094.stdout: "mgr_connect_retry_interval": "1.000000", 2024-09-14T06:28:59.322 INFO:teuthology.orchestra.run.smithi094.stdout: "mgr_data": "/var/lib/ceph/mgr/ceph-a", 2024-09-14T06:28:59.322 INFO:teuthology.orchestra.run.smithi094.stdout: "mgr_debug_aggressive_pg_num_changes": "false", 2024-09-14T06:28:59.322 INFO:teuthology.orchestra.run.smithi094.stdout: "mgr_disabled_modules": "", 2024-09-14T06:28:59.323 INFO:teuthology.orchestra.run.smithi094.stdout: "mgr_enable_op_tracker": "true", 2024-09-14T06:28:59.323 INFO:teuthology.orchestra.run.smithi094.stdout: "mgr_initial_modules": "restful iostat nfs", 2024-09-14T06:28:59.323 INFO:teuthology.orchestra.run.smithi094.stdout: "mgr_max_pg_creating": "1024", 2024-09-14T06:28:59.323 INFO:teuthology.orchestra.run.smithi094.stdout: "mgr_max_pg_num_change": "128", 2024-09-14T06:28:59.323 INFO:teuthology.orchestra.run.smithi094.stdout: "mgr_mds_bytes": "134217728", 2024-09-14T06:28:59.323 INFO:teuthology.orchestra.run.smithi094.stdout: "mgr_mds_messages": "128", 2024-09-14T06:28:59.323 INFO:teuthology.orchestra.run.smithi094.stdout: "mgr_module_path": "/usr/share/ceph/mgr", 2024-09-14T06:28:59.323 INFO:teuthology.orchestra.run.smithi094.stdout: "mgr_mon_bytes": "134217728", 2024-09-14T06:28:59.323 INFO:teuthology.orchestra.run.smithi094.stdout: "mgr_mon_messages": "128", 2024-09-14T06:28:59.323 INFO:teuthology.orchestra.run.smithi094.stdout: "mgr_num_op_tracker_shard": "32", 2024-09-14T06:28:59.323 INFO:teuthology.orchestra.run.smithi094.stdout: "mgr_op_complaint_time": "30.000000", 2024-09-14T06:28:59.323 INFO:teuthology.orchestra.run.smithi094.stdout: "mgr_op_history_duration": "600", 2024-09-14T06:28:59.323 INFO:teuthology.orchestra.run.smithi094.stdout: "mgr_op_history_size": "20", 2024-09-14T06:28:59.323 INFO:teuthology.orchestra.run.smithi094.stdout: "mgr_op_history_slow_op_size": "20", 2024-09-14T06:28:59.324 INFO:teuthology.orchestra.run.smithi094.stdout: "mgr_op_history_slow_op_threshold": "10.000000", 2024-09-14T06:28:59.324 INFO:teuthology.orchestra.run.smithi094.stdout: "mgr_op_log_threshold": "5", 2024-09-14T06:28:59.324 INFO:teuthology.orchestra.run.smithi094.stdout: "mgr_osd_bytes": "536870912", 2024-09-14T06:28:59.324 INFO:teuthology.orchestra.run.smithi094.stdout: "mgr_osd_messages": "8192", 2024-09-14T06:28:59.324 INFO:teuthology.orchestra.run.smithi094.stdout: "mgr_pool": "true", 2024-09-14T06:28:59.324 INFO:teuthology.orchestra.run.smithi094.stdout: "mgr_service_beacon_grace": "60.000000", 2024-09-14T06:28:59.324 INFO:teuthology.orchestra.run.smithi094.stdout: "mgr_standby_modules": "true", 2024-09-14T06:28:59.324 INFO:teuthology.orchestra.run.smithi094.stdout: "mgr_stats_period": "5", 2024-09-14T06:28:59.324 INFO:teuthology.orchestra.run.smithi094.stdout: "mgr_stats_threshold": "5", 2024-09-14T06:28:59.324 INFO:teuthology.orchestra.run.smithi094.stdout: "mgr_tick_period": "2", 2024-09-14T06:28:59.324 INFO:teuthology.orchestra.run.smithi094.stdout: "mgr_ttl_cache_expire_seconds": "0", 2024-09-14T06:28:59.324 INFO:teuthology.orchestra.run.smithi094.stdout: "mon_accept_timeout_factor": "2.000000", 2024-09-14T06:28:59.324 INFO:teuthology.orchestra.run.smithi094.stdout: "mon_allow_pool_delete": "true", 2024-09-14T06:28:59.324 INFO:teuthology.orchestra.run.smithi094.stdout: "mon_allow_pool_size_one": "true", 2024-09-14T06:28:59.325 INFO:teuthology.orchestra.run.smithi094.stdout: "mon_auth_validate_all_caps": "true", 2024-09-14T06:28:59.325 INFO:teuthology.orchestra.run.smithi094.stdout: "mon_cache_target_full_warn_ratio": "0.660000", 2024-09-14T06:28:59.325 INFO:teuthology.orchestra.run.smithi094.stdout: "mon_clean_pg_upmaps_per_chunk": "256", 2024-09-14T06:28:59.325 INFO:teuthology.orchestra.run.smithi094.stdout: "mon_client_bytes": "104857600", 2024-09-14T06:28:59.325 INFO:teuthology.orchestra.run.smithi094.stdout: "mon_client_directed_command_retry": "5", 2024-09-14T06:28:59.325 INFO:teuthology.orchestra.run.smithi094.stdout: "mon_client_hunt_interval": "3.000000", 2024-09-14T06:28:59.325 INFO:teuthology.orchestra.run.smithi094.stdout: "mon_client_hunt_interval_backoff": "1.500000", 2024-09-14T06:28:59.325 INFO:teuthology.orchestra.run.smithi094.stdout: "mon_client_hunt_interval_max_multiple": "10.000000", 2024-09-14T06:28:59.325 INFO:teuthology.orchestra.run.smithi094.stdout: "mon_client_hunt_interval_min_multiple": "1.000000", 2024-09-14T06:28:59.325 INFO:teuthology.orchestra.run.smithi094.stdout: "mon_client_hunt_parallel": "3", 2024-09-14T06:28:59.325 INFO:teuthology.orchestra.run.smithi094.stdout: "mon_client_log_interval": "1.000000", 2024-09-14T06:28:59.325 INFO:teuthology.orchestra.run.smithi094.stdout: "mon_client_max_log_entries_per_message": "1000", 2024-09-14T06:28:59.325 INFO:teuthology.orchestra.run.smithi094.stdout: "mon_client_ping_interval": "10.000000", 2024-09-14T06:28:59.325 INFO:teuthology.orchestra.run.smithi094.stdout: "mon_client_ping_timeout": "30.000000", 2024-09-14T06:28:59.326 INFO:teuthology.orchestra.run.smithi094.stdout: "mon_client_target_rank": "-1", 2024-09-14T06:28:59.326 INFO:teuthology.orchestra.run.smithi094.stdout: "mon_clock_drift_allowed": "1.000000", 2024-09-14T06:28:59.326 INFO:teuthology.orchestra.run.smithi094.stdout: "mon_clock_drift_warn_backoff": "5.000000", 2024-09-14T06:28:59.326 INFO:teuthology.orchestra.run.smithi094.stdout: "mon_cluster_log_file": "default=/var/log/ceph/ceph.$channel.log cluster=/var/log/ceph/ceph.log", 2024-09-14T06:28:59.326 INFO:teuthology.orchestra.run.smithi094.stdout: "mon_cluster_log_level": "debug", 2024-09-14T06:28:59.326 INFO:teuthology.orchestra.run.smithi094.stdout: "mon_cluster_log_to_file": "true", 2024-09-14T06:28:59.326 INFO:teuthology.orchestra.run.smithi094.stdout: "mon_cluster_log_to_graylog": "false", 2024-09-14T06:28:59.326 INFO:teuthology.orchestra.run.smithi094.stdout: "mon_cluster_log_to_graylog_host": "127.0.0.1", 2024-09-14T06:28:59.326 INFO:teuthology.orchestra.run.smithi094.stdout: "mon_cluster_log_to_graylog_port": "12201", 2024-09-14T06:28:59.326 INFO:teuthology.orchestra.run.smithi094.stdout: "mon_cluster_log_to_journald": "false", 2024-09-14T06:28:59.326 INFO:teuthology.orchestra.run.smithi094.stdout: "mon_cluster_log_to_stderr": "false", 2024-09-14T06:28:59.326 INFO:teuthology.orchestra.run.smithi094.stdout: "mon_cluster_log_to_syslog": "default=false", 2024-09-14T06:28:59.326 INFO:teuthology.orchestra.run.smithi094.stdout: "mon_cluster_log_to_syslog_facility": "daemon", 2024-09-14T06:28:59.326 INFO:teuthology.orchestra.run.smithi094.stdout: "mon_compact_on_bootstrap": "false", 2024-09-14T06:28:59.327 INFO:teuthology.orchestra.run.smithi094.stdout: "mon_compact_on_start": "false", 2024-09-14T06:28:59.327 INFO:teuthology.orchestra.run.smithi094.stdout: "mon_compact_on_trim": "true", 2024-09-14T06:28:59.327 INFO:teuthology.orchestra.run.smithi094.stdout: "mon_con_tracker_persist_interval": "10", 2024-09-14T06:28:59.327 INFO:teuthology.orchestra.run.smithi094.stdout: "mon_con_tracker_score_halflife": "43200", 2024-09-14T06:28:59.327 INFO:teuthology.orchestra.run.smithi094.stdout: "mon_config_key_max_entry_size": "65536", 2024-09-14T06:28:59.327 INFO:teuthology.orchestra.run.smithi094.stdout: "mon_cpu_threads": "4", 2024-09-14T06:28:59.327 INFO:teuthology.orchestra.run.smithi094.stdout: "mon_crush_min_required_version": "hammer", 2024-09-14T06:28:59.327 INFO:teuthology.orchestra.run.smithi094.stdout: "mon_daemon_bytes": "419430400", 2024-09-14T06:28:59.327 INFO:teuthology.orchestra.run.smithi094.stdout: "mon_data": "/var/lib/ceph/mon/ceph-a", 2024-09-14T06:28:59.327 INFO:teuthology.orchestra.run.smithi094.stdout: "mon_data_avail_crit": "5", 2024-09-14T06:28:59.327 INFO:teuthology.orchestra.run.smithi094.stdout: "mon_data_avail_warn": "5", 2024-09-14T06:28:59.327 INFO:teuthology.orchestra.run.smithi094.stdout: "mon_data_size_warn": "16106127360", 2024-09-14T06:28:59.327 INFO:teuthology.orchestra.run.smithi094.stdout: "mon_debug_block_osdmap_trim": "false", 2024-09-14T06:28:59.327 INFO:teuthology.orchestra.run.smithi094.stdout: "mon_debug_deprecated_as_obsolete": "false", 2024-09-14T06:28:59.328 INFO:teuthology.orchestra.run.smithi094.stdout: "mon_debug_dump_json": "false", 2024-09-14T06:28:59.328 INFO:teuthology.orchestra.run.smithi094.stdout: "mon_debug_dump_location": "/var/log/ceph/ceph-mon.a.tdump", 2024-09-14T06:28:59.328 INFO:teuthology.orchestra.run.smithi094.stdout: "mon_debug_dump_transactions": "false", 2024-09-14T06:28:59.328 INFO:teuthology.orchestra.run.smithi094.stdout: "mon_debug_extra_checks": "false", 2024-09-14T06:28:59.328 INFO:teuthology.orchestra.run.smithi094.stdout: "mon_debug_no_initial_persistent_features": "false", 2024-09-14T06:28:59.328 INFO:teuthology.orchestra.run.smithi094.stdout: "mon_debug_no_require_bluestore_for_ec_overwrites": "false", 2024-09-14T06:28:59.328 INFO:teuthology.orchestra.run.smithi094.stdout: "mon_debug_no_require_reef": "false", 2024-09-14T06:28:59.328 INFO:teuthology.orchestra.run.smithi094.stdout: "mon_debug_no_require_squid": "false", 2024-09-14T06:28:59.328 INFO:teuthology.orchestra.run.smithi094.stdout: "mon_debug_unsafe_allow_tier_with_nonempty_snaps": "false", 2024-09-14T06:28:59.328 INFO:teuthology.orchestra.run.smithi094.stdout: "mon_delta_reset_interval": "10.000000", 2024-09-14T06:28:59.328 INFO:teuthology.orchestra.run.smithi094.stdout: "mon_dns_srv_name": "ceph-mon", 2024-09-14T06:28:59.328 INFO:teuthology.orchestra.run.smithi094.stdout: "mon_down_mkfs_grace": "120", 2024-09-14T06:28:59.328 INFO:teuthology.orchestra.run.smithi094.stdout: "mon_down_uptime_grace": "60", 2024-09-14T06:28:59.328 INFO:teuthology.orchestra.run.smithi094.stdout: "mon_election_default_strategy": "1", 2024-09-14T06:28:59.329 INFO:teuthology.orchestra.run.smithi094.stdout: "mon_election_timeout": "5.000000", 2024-09-14T06:28:59.329 INFO:teuthology.orchestra.run.smithi094.stdout: "mon_elector_ignore_propose_margin": "0.000500", 2024-09-14T06:28:59.329 INFO:teuthology.orchestra.run.smithi094.stdout: "mon_elector_ping_divisor": "2", 2024-09-14T06:28:59.329 INFO:teuthology.orchestra.run.smithi094.stdout: "mon_elector_ping_timeout": "2.000000", 2024-09-14T06:28:59.329 INFO:teuthology.orchestra.run.smithi094.stdout: "mon_enable_op_tracker": "true", 2024-09-14T06:28:59.329 INFO:teuthology.orchestra.run.smithi094.stdout: "mon_fake_pool_delete": "false", 2024-09-14T06:28:59.329 INFO:teuthology.orchestra.run.smithi094.stdout: "mon_force_quorum_join": "false", 2024-09-14T06:28:59.329 INFO:teuthology.orchestra.run.smithi094.stdout: "mon_fsmap_prune_threshold": "300", 2024-09-14T06:28:59.329 INFO:teuthology.orchestra.run.smithi094.stdout: "mon_globalid_prealloc": "10000", 2024-09-14T06:28:59.329 INFO:teuthology.orchestra.run.smithi094.stdout: "mon_health_detail_to_clog": "false", 2024-09-14T06:28:59.329 INFO:teuthology.orchestra.run.smithi094.stdout: "mon_health_log_update_period": "5", 2024-09-14T06:28:59.329 INFO:teuthology.orchestra.run.smithi094.stdout: "mon_health_max_detail": "50", 2024-09-14T06:28:59.329 INFO:teuthology.orchestra.run.smithi094.stdout: "mon_health_to_clog": "true", 2024-09-14T06:28:59.329 INFO:teuthology.orchestra.run.smithi094.stdout: "mon_health_to_clog_interval": "600", 2024-09-14T06:28:59.330 INFO:teuthology.orchestra.run.smithi094.stdout: "mon_health_to_clog_tick_interval": "60.000000", 2024-09-14T06:28:59.330 INFO:teuthology.orchestra.run.smithi094.stdout: "mon_host": "172.21.15.94,[v2:172.21.15.94:3301,v1:172.21.15.94:6790],172.21.15.138", 2024-09-14T06:28:59.330 INFO:teuthology.orchestra.run.smithi094.stdout: "mon_host_override": "", 2024-09-14T06:28:59.330 INFO:teuthology.orchestra.run.smithi094.stdout: "mon_initial_members": "", 2024-09-14T06:28:59.330 INFO:teuthology.orchestra.run.smithi094.stdout: "mon_inject_pg_merge_bounce_probability": "0.000000", 2024-09-14T06:28:59.330 INFO:teuthology.orchestra.run.smithi094.stdout: "mon_inject_sync_get_chunk_delay": "0.000000", 2024-09-14T06:28:59.330 INFO:teuthology.orchestra.run.smithi094.stdout: "mon_inject_transaction_delay_max": "10.000000", 2024-09-14T06:28:59.330 INFO:teuthology.orchestra.run.smithi094.stdout: "mon_inject_transaction_delay_probability": "0.000000", 2024-09-14T06:28:59.330 INFO:teuthology.orchestra.run.smithi094.stdout: "mon_keyvaluedb": "rocksdb", 2024-09-14T06:28:59.330 INFO:teuthology.orchestra.run.smithi094.stdout: "mon_lease": "5.000000", 2024-09-14T06:28:59.330 INFO:teuthology.orchestra.run.smithi094.stdout: "mon_lease_ack_timeout_factor": "2.000000", 2024-09-14T06:28:59.330 INFO:teuthology.orchestra.run.smithi094.stdout: "mon_lease_renew_interval_factor": "0.600000", 2024-09-14T06:28:59.330 INFO:teuthology.orchestra.run.smithi094.stdout: "mon_log_full_interval": "50", 2024-09-14T06:28:59.330 INFO:teuthology.orchestra.run.smithi094.stdout: "mon_log_max": "10000", 2024-09-14T06:28:59.331 INFO:teuthology.orchestra.run.smithi094.stdout: "mon_log_max_summary": "50", 2024-09-14T06:28:59.331 INFO:teuthology.orchestra.run.smithi094.stdout: "mon_max_log_entries_per_event": "4096", 2024-09-14T06:28:59.331 INFO:teuthology.orchestra.run.smithi094.stdout: "mon_max_log_epochs": "500", 2024-09-14T06:28:59.331 INFO:teuthology.orchestra.run.smithi094.stdout: "mon_max_mdsmap_epochs": "500", 2024-09-14T06:28:59.331 INFO:teuthology.orchestra.run.smithi094.stdout: "mon_max_mgrmap_epochs": "500", 2024-09-14T06:28:59.331 INFO:teuthology.orchestra.run.smithi094.stdout: "mon_max_nvmeof_epochs": "500", 2024-09-14T06:28:59.331 INFO:teuthology.orchestra.run.smithi094.stdout: "mon_max_osd": "10000", 2024-09-14T06:28:59.331 INFO:teuthology.orchestra.run.smithi094.stdout: "mon_max_pg_per_osd": "10000", 2024-09-14T06:28:59.331 INFO:teuthology.orchestra.run.smithi094.stdout: "mon_max_pool_pg_num": "65536", 2024-09-14T06:28:59.331 INFO:teuthology.orchestra.run.smithi094.stdout: "mon_max_snap_prune_per_epoch": "100", 2024-09-14T06:28:59.331 INFO:teuthology.orchestra.run.smithi094.stdout: "mon_mds_blocklist_interval": "86400.000000", 2024-09-14T06:28:59.331 INFO:teuthology.orchestra.run.smithi094.stdout: "mon_mds_force_trim_to": "0", 2024-09-14T06:28:59.331 INFO:teuthology.orchestra.run.smithi094.stdout: "mon_mds_skip_sanity": "false", 2024-09-14T06:28:59.331 INFO:teuthology.orchestra.run.smithi094.stdout: "mon_memory_autotune": "true", 2024-09-14T06:28:59.331 INFO:teuthology.orchestra.run.smithi094.stdout: "mon_memory_target": "2147483648", 2024-09-14T06:28:59.332 INFO:teuthology.orchestra.run.smithi094.stdout: "mon_mgr_beacon_grace": "30", 2024-09-14T06:28:59.332 INFO:teuthology.orchestra.run.smithi094.stdout: "mon_mgr_blocklist_interval": "86400.000000", 2024-09-14T06:28:59.332 INFO:teuthology.orchestra.run.smithi094.stdout: "mon_mgr_digest_period": "5", 2024-09-14T06:28:59.332 INFO:teuthology.orchestra.run.smithi094.stdout: "mon_mgr_inactive_grace": "60", 2024-09-14T06:28:59.332 INFO:teuthology.orchestra.run.smithi094.stdout: "mon_mgr_mkfs_grace": "240", 2024-09-14T06:28:59.332 INFO:teuthology.orchestra.run.smithi094.stdout: "mon_mgr_proxy_client_bytes_ratio": "0.300000", 2024-09-14T06:28:59.332 INFO:teuthology.orchestra.run.smithi094.stdout: "mon_min_osdmap_epochs": "500", 2024-09-14T06:28:59.332 INFO:teuthology.orchestra.run.smithi094.stdout: "mon_nvmeofgw_beacon_grace": "10", 2024-09-14T06:28:59.332 INFO:teuthology.orchestra.run.smithi094.stdout: "mon_nvmeofgw_set_group_id_retry": "1000", 2024-09-14T06:28:59.332 INFO:teuthology.orchestra.run.smithi094.stdout: "mon_op_complaint_time": "30", 2024-09-14T06:28:59.332 INFO:teuthology.orchestra.run.smithi094.stdout: "mon_op_history_duration": "600", 2024-09-14T06:28:59.332 INFO:teuthology.orchestra.run.smithi094.stdout: "mon_op_history_size": "20", 2024-09-14T06:28:59.332 INFO:teuthology.orchestra.run.smithi094.stdout: "mon_op_history_slow_op_size": "20", 2024-09-14T06:28:59.332 INFO:teuthology.orchestra.run.smithi094.stdout: "mon_op_history_slow_op_threshold": "10", 2024-09-14T06:28:59.333 INFO:teuthology.orchestra.run.smithi094.stdout: "mon_op_log_threshold": "5", 2024-09-14T06:28:59.333 INFO:teuthology.orchestra.run.smithi094.stdout: "mon_osd_adjust_down_out_interval": "true", 2024-09-14T06:28:59.333 INFO:teuthology.orchestra.run.smithi094.stdout: "mon_osd_adjust_heartbeat_grace": "true", 2024-09-14T06:28:59.333 INFO:teuthology.orchestra.run.smithi094.stdout: "mon_osd_auto_mark_auto_out_in": "true", 2024-09-14T06:28:59.333 INFO:teuthology.orchestra.run.smithi094.stdout: "mon_osd_auto_mark_in": "false", 2024-09-14T06:28:59.333 INFO:teuthology.orchestra.run.smithi094.stdout: "mon_osd_auto_mark_new_in": "true", 2024-09-14T06:28:59.333 INFO:teuthology.orchestra.run.smithi094.stdout: "mon_osd_backfillfull_ratio": "0.900000", 2024-09-14T06:28:59.333 INFO:teuthology.orchestra.run.smithi094.stdout: "mon_osd_blocklist_default_expire": "3600.000000", 2024-09-14T06:28:59.333 INFO:teuthology.orchestra.run.smithi094.stdout: "mon_osd_cache_size": "500", 2024-09-14T06:28:59.333 INFO:teuthology.orchestra.run.smithi094.stdout: "mon_osd_cache_size_min": "134217728", 2024-09-14T06:28:59.333 INFO:teuthology.orchestra.run.smithi094.stdout: "mon_osd_crush_smoke_test": "true", 2024-09-14T06:28:59.333 INFO:teuthology.orchestra.run.smithi094.stdout: "mon_osd_destroyed_out_interval": "600", 2024-09-14T06:28:59.333 INFO:teuthology.orchestra.run.smithi094.stdout: "mon_osd_down_out_interval": "600", 2024-09-14T06:28:59.334 INFO:teuthology.orchestra.run.smithi094.stdout: "mon_osd_down_out_subtree_limit": "rack", 2024-09-14T06:28:59.334 INFO:teuthology.orchestra.run.smithi094.stdout: "mon_osd_err_op_age_ratio": "128.000000", 2024-09-14T06:28:59.334 INFO:teuthology.orchestra.run.smithi094.stdout: "mon_osd_force_trim_to": "0", 2024-09-14T06:28:59.334 INFO:teuthology.orchestra.run.smithi094.stdout: "mon_osd_full_ratio": "0.950000", 2024-09-14T06:28:59.334 INFO:teuthology.orchestra.run.smithi094.stdout: "mon_osd_initial_require_min_compat_client": "luminous", 2024-09-14T06:28:59.334 INFO:teuthology.orchestra.run.smithi094.stdout: "mon_osd_laggy_halflife": "3600", 2024-09-14T06:28:59.334 INFO:teuthology.orchestra.run.smithi094.stdout: "mon_osd_laggy_max_interval": "300", 2024-09-14T06:28:59.334 INFO:teuthology.orchestra.run.smithi094.stdout: "mon_osd_laggy_weight": "0.300000", 2024-09-14T06:28:59.334 INFO:teuthology.orchestra.run.smithi094.stdout: "mon_osd_mapping_pgs_per_chunk": "4096", 2024-09-14T06:28:59.334 INFO:teuthology.orchestra.run.smithi094.stdout: "mon_osd_max_initial_pgs": "1024", 2024-09-14T06:28:59.334 INFO:teuthology.orchestra.run.smithi094.stdout: "mon_osd_min_down_reporters": "2", 2024-09-14T06:28:59.334 INFO:teuthology.orchestra.run.smithi094.stdout: "mon_osd_min_in_ratio": "0.750000", 2024-09-14T06:28:59.334 INFO:teuthology.orchestra.run.smithi094.stdout: "mon_osd_min_up_ratio": "0.300000", 2024-09-14T06:28:59.334 INFO:teuthology.orchestra.run.smithi094.stdout: "mon_osd_nearfull_ratio": "0.850000", 2024-09-14T06:28:59.335 INFO:teuthology.orchestra.run.smithi094.stdout: "mon_osd_prime_pg_temp": "true", 2024-09-14T06:28:59.335 INFO:teuthology.orchestra.run.smithi094.stdout: "mon_osd_prime_pg_temp_max_estimate": "0.250000", 2024-09-14T06:28:59.335 INFO:teuthology.orchestra.run.smithi094.stdout: "mon_osd_prime_pg_temp_max_time": "0.500000", 2024-09-14T06:28:59.335 INFO:teuthology.orchestra.run.smithi094.stdout: "mon_osd_report_timeout": "900", 2024-09-14T06:28:59.335 INFO:teuthology.orchestra.run.smithi094.stdout: "mon_osd_reporter_subtree_level": "osd", 2024-09-14T06:28:59.335 INFO:teuthology.orchestra.run.smithi094.stdout: "mon_osd_snap_trim_queue_warn_on": "32768", 2024-09-14T06:28:59.335 INFO:teuthology.orchestra.run.smithi094.stdout: "mon_osd_warn_num_repaired": "10", 2024-09-14T06:28:59.335 INFO:teuthology.orchestra.run.smithi094.stdout: "mon_osd_warn_op_age": "32.000000", 2024-09-14T06:28:59.335 INFO:teuthology.orchestra.run.smithi094.stdout: "mon_osdmap_full_prune_enabled": "true", 2024-09-14T06:28:59.335 INFO:teuthology.orchestra.run.smithi094.stdout: "mon_osdmap_full_prune_interval": "10", 2024-09-14T06:28:59.335 INFO:teuthology.orchestra.run.smithi094.stdout: "mon_osdmap_full_prune_min": "10000", 2024-09-14T06:28:59.335 INFO:teuthology.orchestra.run.smithi094.stdout: "mon_osdmap_full_prune_txsize": "100", 2024-09-14T06:28:59.335 INFO:teuthology.orchestra.run.smithi094.stdout: "mon_pg_check_down_all_threshold": "0.500000", 2024-09-14T06:28:59.336 INFO:teuthology.orchestra.run.smithi094.stdout: "mon_pg_stuck_threshold": "60", 2024-09-14T06:28:59.336 INFO:teuthology.orchestra.run.smithi094.stdout: "mon_pg_warn_max_object_skew": "0.000000", 2024-09-14T06:28:59.336 INFO:teuthology.orchestra.run.smithi094.stdout: "mon_pg_warn_min_objects": "10000", 2024-09-14T06:28:59.336 INFO:teuthology.orchestra.run.smithi094.stdout: "mon_pg_warn_min_per_osd": "0", 2024-09-14T06:28:59.336 INFO:teuthology.orchestra.run.smithi094.stdout: "mon_pg_warn_min_pool_objects": "1000", 2024-09-14T06:28:59.336 INFO:teuthology.orchestra.run.smithi094.stdout: "mon_pool_quota_crit_threshold": "0", 2024-09-14T06:28:59.336 INFO:teuthology.orchestra.run.smithi094.stdout: "mon_pool_quota_warn_threshold": "0", 2024-09-14T06:28:59.336 INFO:teuthology.orchestra.run.smithi094.stdout: "mon_probe_timeout": "2.000000", 2024-09-14T06:28:59.336 INFO:teuthology.orchestra.run.smithi094.stdout: "mon_reweight_max_change": "0.050000", 2024-09-14T06:28:59.336 INFO:teuthology.orchestra.run.smithi094.stdout: "mon_reweight_max_osds": "4", 2024-09-14T06:28:59.336 INFO:teuthology.orchestra.run.smithi094.stdout: "mon_reweight_min_bytes_per_osd": "10", 2024-09-14T06:28:59.336 INFO:teuthology.orchestra.run.smithi094.stdout: "mon_reweight_min_pgs_per_osd": "4", 2024-09-14T06:28:59.336 INFO:teuthology.orchestra.run.smithi094.stdout: "mon_rocksdb_options": "write_buffer_size=33554432,compression=kNoCompression,level_compaction_dynamic_level_bytes=true", 2024-09-14T06:28:59.336 INFO:teuthology.orchestra.run.smithi094.stdout: "mon_scrub_inject_crc_mismatch": "0.000000", 2024-09-14T06:28:59.337 INFO:teuthology.orchestra.run.smithi094.stdout: "mon_scrub_inject_missing_keys": "0.000000", 2024-09-14T06:28:59.337 INFO:teuthology.orchestra.run.smithi094.stdout: "mon_scrub_interval": "86400", 2024-09-14T06:28:59.337 INFO:teuthology.orchestra.run.smithi094.stdout: "mon_scrub_max_keys": "100", 2024-09-14T06:28:59.337 INFO:teuthology.orchestra.run.smithi094.stdout: "mon_scrub_timeout": "300", 2024-09-14T06:28:59.337 INFO:teuthology.orchestra.run.smithi094.stdout: "mon_session_timeout": "300", 2024-09-14T06:28:59.337 INFO:teuthology.orchestra.run.smithi094.stdout: "mon_smart_report_timeout": "5", 2024-09-14T06:28:59.337 INFO:teuthology.orchestra.run.smithi094.stdout: "mon_stat_smooth_intervals": "6", 2024-09-14T06:28:59.337 INFO:teuthology.orchestra.run.smithi094.stdout: "mon_stretch_cluster_recovery_ratio": "0.600000", 2024-09-14T06:28:59.337 INFO:teuthology.orchestra.run.smithi094.stdout: "mon_stretch_pool_min_size": "2", 2024-09-14T06:28:59.337 INFO:teuthology.orchestra.run.smithi094.stdout: "mon_stretch_pool_size": "4", 2024-09-14T06:28:59.337 INFO:teuthology.orchestra.run.smithi094.stdout: "mon_stretch_recovery_min_wait": "15.000000", 2024-09-14T06:28:59.337 INFO:teuthology.orchestra.run.smithi094.stdout: "mon_subscribe_interval": "86400.000000", 2024-09-14T06:28:59.337 INFO:teuthology.orchestra.run.smithi094.stdout: "mon_sync_debug": "false", 2024-09-14T06:28:59.337 INFO:teuthology.orchestra.run.smithi094.stdout: "mon_sync_max_payload_keys": "2000", 2024-09-14T06:28:59.338 INFO:teuthology.orchestra.run.smithi094.stdout: "mon_sync_max_payload_size": "1048576", 2024-09-14T06:28:59.338 INFO:teuthology.orchestra.run.smithi094.stdout: "mon_sync_provider_kill_at": "0", 2024-09-14T06:28:59.338 INFO:teuthology.orchestra.run.smithi094.stdout: "mon_sync_requester_kill_at": "0", 2024-09-14T06:28:59.338 INFO:teuthology.orchestra.run.smithi094.stdout: "mon_sync_timeout": "60.000000", 2024-09-14T06:28:59.338 INFO:teuthology.orchestra.run.smithi094.stdout: "mon_target_pg_per_osd": "100", 2024-09-14T06:28:59.338 INFO:teuthology.orchestra.run.smithi094.stdout: "mon_tick_interval": "5", 2024-09-14T06:28:59.338 INFO:teuthology.orchestra.run.smithi094.stdout: "mon_timecheck_interval": "300.000000", 2024-09-14T06:28:59.338 INFO:teuthology.orchestra.run.smithi094.stdout: "mon_timecheck_skew_interval": "30.000000", 2024-09-14T06:28:59.338 INFO:teuthology.orchestra.run.smithi094.stdout: "mon_use_min_delay_socket": "false", 2024-09-14T06:28:59.338 INFO:teuthology.orchestra.run.smithi094.stdout: "mon_warn_older_version_delay": "604800", 2024-09-14T06:28:59.338 INFO:teuthology.orchestra.run.smithi094.stdout: "mon_warn_on_cache_pools_without_hit_sets": "true", 2024-09-14T06:28:59.338 INFO:teuthology.orchestra.run.smithi094.stdout: "mon_warn_on_crush_straw_calc_version_zero": "false", 2024-09-14T06:28:59.338 INFO:teuthology.orchestra.run.smithi094.stdout: "mon_warn_on_degraded_stretch_mode": "true", 2024-09-14T06:28:59.338 INFO:teuthology.orchestra.run.smithi094.stdout: "mon_warn_on_filestore_osds": "false", 2024-09-14T06:28:59.339 INFO:teuthology.orchestra.run.smithi094.stdout: "mon_warn_on_insecure_global_id_reclaim": "false", 2024-09-14T06:28:59.339 INFO:teuthology.orchestra.run.smithi094.stdout: "mon_warn_on_insecure_global_id_reclaim_allowed": "false", 2024-09-14T06:28:59.339 INFO:teuthology.orchestra.run.smithi094.stdout: "mon_warn_on_legacy_crush_tunables": "false", 2024-09-14T06:28:59.339 INFO:teuthology.orchestra.run.smithi094.stdout: "mon_warn_on_misplaced": "false", 2024-09-14T06:28:59.339 INFO:teuthology.orchestra.run.smithi094.stdout: "mon_warn_on_msgr2_not_enabled": "true", 2024-09-14T06:28:59.339 INFO:teuthology.orchestra.run.smithi094.stdout: "mon_warn_on_older_version": "true", 2024-09-14T06:28:59.339 INFO:teuthology.orchestra.run.smithi094.stdout: "mon_warn_on_osd_down_out_interval_zero": "false", 2024-09-14T06:28:59.339 INFO:teuthology.orchestra.run.smithi094.stdout: "mon_warn_on_pool_no_app": "false", 2024-09-14T06:28:59.339 INFO:teuthology.orchestra.run.smithi094.stdout: "mon_warn_on_pool_no_redundancy": "false", 2024-09-14T06:28:59.339 INFO:teuthology.orchestra.run.smithi094.stdout: "mon_warn_on_pool_pg_num_not_power_of_two": "false", 2024-09-14T06:28:59.339 INFO:teuthology.orchestra.run.smithi094.stdout: "mon_warn_on_slow_ping_ratio": "0.050000", 2024-09-14T06:28:59.339 INFO:teuthology.orchestra.run.smithi094.stdout: "mon_warn_on_slow_ping_time": "0.000000", 2024-09-14T06:28:59.339 INFO:teuthology.orchestra.run.smithi094.stdout: "mon_warn_on_too_few_osds": "false", 2024-09-14T06:28:59.339 INFO:teuthology.orchestra.run.smithi094.stdout: "mon_warn_pg_not_deep_scrubbed_ratio": "0.750000", 2024-09-14T06:28:59.340 INFO:teuthology.orchestra.run.smithi094.stdout: "mon_warn_pg_not_scrubbed_ratio": "0.500000", 2024-09-14T06:28:59.340 INFO:teuthology.orchestra.run.smithi094.stdout: "monmap": "", 2024-09-14T06:28:59.340 INFO:teuthology.orchestra.run.smithi094.stdout: "motr_admin_endpoint": "192.168.180.182@tcp:12345:4:1", 2024-09-14T06:28:59.340 INFO:teuthology.orchestra.run.smithi094.stdout: "motr_admin_fid": "0x7200000000000001:0x0", 2024-09-14T06:28:59.340 INFO:teuthology.orchestra.run.smithi094.stdout: "motr_ha_endpoint": "192.168.180.182@tcp:12345:1:1", 2024-09-14T06:28:59.340 INFO:teuthology.orchestra.run.smithi094.stdout: "motr_my_endpoint": "192.168.180.182@tcp:12345:4:1", 2024-09-14T06:28:59.340 INFO:teuthology.orchestra.run.smithi094.stdout: "motr_my_fid": "0x7200000000000001:0x0", 2024-09-14T06:28:59.340 INFO:teuthology.orchestra.run.smithi094.stdout: "motr_profile_fid": "0x7000000000000001:0x0", 2024-09-14T06:28:59.340 INFO:teuthology.orchestra.run.smithi094.stdout: "motr_tracing_enabled": "false", 2024-09-14T06:28:59.340 INFO:teuthology.orchestra.run.smithi094.stdout: "ms_async_op_threads": "3", 2024-09-14T06:28:59.340 INFO:teuthology.orchestra.run.smithi094.stdout: "ms_async_rdma_buffer_size": "131072", 2024-09-14T06:28:59.340 INFO:teuthology.orchestra.run.smithi094.stdout: "ms_async_rdma_cm": "false", 2024-09-14T06:28:59.340 INFO:teuthology.orchestra.run.smithi094.stdout: "ms_async_rdma_device_name": "", 2024-09-14T06:28:59.341 INFO:teuthology.orchestra.run.smithi094.stdout: "ms_async_rdma_dscp": "96", 2024-09-14T06:28:59.341 INFO:teuthology.orchestra.run.smithi094.stdout: "ms_async_rdma_enable_hugepage": "false", 2024-09-14T06:28:59.341 INFO:teuthology.orchestra.run.smithi094.stdout: "ms_async_rdma_gid_idx": "0", 2024-09-14T06:28:59.341 INFO:teuthology.orchestra.run.smithi094.stdout: "ms_async_rdma_local_gid": "", 2024-09-14T06:28:59.341 INFO:teuthology.orchestra.run.smithi094.stdout: "ms_async_rdma_polling_us": "1000", 2024-09-14T06:28:59.341 INFO:teuthology.orchestra.run.smithi094.stdout: "ms_async_rdma_port_num": "1", 2024-09-14T06:28:59.341 INFO:teuthology.orchestra.run.smithi094.stdout: "ms_async_rdma_receive_buffers": "32768", 2024-09-14T06:28:59.341 INFO:teuthology.orchestra.run.smithi094.stdout: "ms_async_rdma_receive_queue_len": "4096", 2024-09-14T06:28:59.341 INFO:teuthology.orchestra.run.smithi094.stdout: "ms_async_rdma_roce_ver": "1", 2024-09-14T06:28:59.341 INFO:teuthology.orchestra.run.smithi094.stdout: "ms_async_rdma_send_buffers": "1024", 2024-09-14T06:28:59.341 INFO:teuthology.orchestra.run.smithi094.stdout: "ms_async_rdma_sl": "3", 2024-09-14T06:28:59.341 INFO:teuthology.orchestra.run.smithi094.stdout: "ms_async_rdma_support_srq": "true", 2024-09-14T06:28:59.341 INFO:teuthology.orchestra.run.smithi094.stdout: "ms_async_rdma_type": "ib", 2024-09-14T06:28:59.342 INFO:teuthology.orchestra.run.smithi094.stdout: "ms_async_reap_threshold": "5", 2024-09-14T06:28:59.342 INFO:teuthology.orchestra.run.smithi094.stdout: "ms_bind_before_connect": "false", 2024-09-14T06:28:59.342 INFO:teuthology.orchestra.run.smithi094.stdout: "ms_bind_ipv4": "true", 2024-09-14T06:28:59.342 INFO:teuthology.orchestra.run.smithi094.stdout: "ms_bind_ipv6": "false", 2024-09-14T06:28:59.342 INFO:teuthology.orchestra.run.smithi094.stdout: "ms_bind_msgr1": "true", 2024-09-14T06:28:59.342 INFO:teuthology.orchestra.run.smithi094.stdout: "ms_bind_msgr2": "true", 2024-09-14T06:28:59.342 INFO:teuthology.orchestra.run.smithi094.stdout: "ms_bind_port_max": "7568", 2024-09-14T06:28:59.342 INFO:teuthology.orchestra.run.smithi094.stdout: "ms_bind_port_min": "6800", 2024-09-14T06:28:59.342 INFO:teuthology.orchestra.run.smithi094.stdout: "ms_bind_prefer_ipv4": "false", 2024-09-14T06:28:59.342 INFO:teuthology.orchestra.run.smithi094.stdout: "ms_bind_retry_count": "3", 2024-09-14T06:28:59.342 INFO:teuthology.orchestra.run.smithi094.stdout: "ms_bind_retry_delay": "5", 2024-09-14T06:28:59.342 INFO:teuthology.orchestra.run.smithi094.stdout: "ms_blackhole_client": "false", 2024-09-14T06:28:59.342 INFO:teuthology.orchestra.run.smithi094.stdout: "ms_blackhole_mds": "false", 2024-09-14T06:28:59.342 INFO:teuthology.orchestra.run.smithi094.stdout: "ms_blackhole_mgr": "false", 2024-09-14T06:28:59.342 INFO:teuthology.orchestra.run.smithi094.stdout: "ms_blackhole_mon": "false", 2024-09-14T06:28:59.343 INFO:teuthology.orchestra.run.smithi094.stdout: "ms_blackhole_osd": "false", 2024-09-14T06:28:59.343 INFO:teuthology.orchestra.run.smithi094.stdout: "ms_client_mode": "crc secure", 2024-09-14T06:28:59.343 INFO:teuthology.orchestra.run.smithi094.stdout: "ms_client_throttle_retry_time_interval": "5000", 2024-09-14T06:28:59.343 INFO:teuthology.orchestra.run.smithi094.stdout: "ms_cluster_mode": "crc secure", 2024-09-14T06:28:59.343 INFO:teuthology.orchestra.run.smithi094.stdout: "ms_cluster_type": "", 2024-09-14T06:28:59.343 INFO:teuthology.orchestra.run.smithi094.stdout: "ms_compress_secure": "false", 2024-09-14T06:28:59.343 INFO:teuthology.orchestra.run.smithi094.stdout: "ms_connection_idle_timeout": "900", 2024-09-14T06:28:59.343 INFO:teuthology.orchestra.run.smithi094.stdout: "ms_connection_ready_timeout": "10", 2024-09-14T06:28:59.343 INFO:teuthology.orchestra.run.smithi094.stdout: "ms_crc_data": "true", 2024-09-14T06:28:59.343 INFO:teuthology.orchestra.run.smithi094.stdout: "ms_crc_header": "true", 2024-09-14T06:28:59.343 INFO:teuthology.orchestra.run.smithi094.stdout: "ms_die_on_bad_msg": "false", 2024-09-14T06:28:59.343 INFO:teuthology.orchestra.run.smithi094.stdout: "ms_die_on_bug": "true", 2024-09-14T06:28:59.343 INFO:teuthology.orchestra.run.smithi094.stdout: "ms_die_on_old_message": "true", 2024-09-14T06:28:59.344 INFO:teuthology.orchestra.run.smithi094.stdout: "ms_die_on_skipped_message": "false", 2024-09-14T06:28:59.344 INFO:teuthology.orchestra.run.smithi094.stdout: "ms_die_on_unhandled_msg": "false", 2024-09-14T06:28:59.344 INFO:teuthology.orchestra.run.smithi094.stdout: "ms_dispatch_throttle_bytes": "104857600", 2024-09-14T06:28:59.344 INFO:teuthology.orchestra.run.smithi094.stdout: "ms_dpdk_coremask": "0xF", 2024-09-14T06:28:59.344 INFO:teuthology.orchestra.run.smithi094.stdout: "ms_dpdk_debug_allow_loopback": "false", 2024-09-14T06:28:59.344 INFO:teuthology.orchestra.run.smithi094.stdout: "ms_dpdk_devs_allowlist": "", 2024-09-14T06:28:59.344 INFO:teuthology.orchestra.run.smithi094.stdout: "ms_dpdk_enable_tso": "true", 2024-09-14T06:28:59.344 INFO:teuthology.orchestra.run.smithi094.stdout: "ms_dpdk_gateway_ipv4_addr": "", 2024-09-14T06:28:59.344 INFO:teuthology.orchestra.run.smithi094.stdout: "ms_dpdk_host_ipv4_addr": "", 2024-09-14T06:28:59.344 INFO:teuthology.orchestra.run.smithi094.stdout: "ms_dpdk_hugepages": "", 2024-09-14T06:28:59.344 INFO:teuthology.orchestra.run.smithi094.stdout: "ms_dpdk_hw_flow_control": "true", 2024-09-14T06:28:59.344 INFO:teuthology.orchestra.run.smithi094.stdout: "ms_dpdk_hw_queue_weight": "1.000000", 2024-09-14T06:28:59.344 INFO:teuthology.orchestra.run.smithi094.stdout: "ms_dpdk_lro": "true", 2024-09-14T06:28:59.344 INFO:teuthology.orchestra.run.smithi094.stdout: "ms_dpdk_memory_channel": "4", 2024-09-14T06:28:59.345 INFO:teuthology.orchestra.run.smithi094.stdout: "ms_dpdk_netmask_ipv4_addr": "", 2024-09-14T06:28:59.345 INFO:teuthology.orchestra.run.smithi094.stdout: "ms_dpdk_pmd": "", 2024-09-14T06:28:59.345 INFO:teuthology.orchestra.run.smithi094.stdout: "ms_dpdk_port_id": "0", 2024-09-14T06:28:59.345 INFO:teuthology.orchestra.run.smithi094.stdout: "ms_dpdk_rx_buffer_count_per_core": "8192", 2024-09-14T06:28:59.345 INFO:teuthology.orchestra.run.smithi094.stdout: "ms_dump_corrupt_message_level": "1", 2024-09-14T06:28:59.345 INFO:teuthology.orchestra.run.smithi094.stdout: "ms_dump_on_send": "false", 2024-09-14T06:28:59.345 INFO:teuthology.orchestra.run.smithi094.stdout: "ms_initial_backoff": "0.200000", 2024-09-14T06:28:59.345 INFO:teuthology.orchestra.run.smithi094.stdout: "ms_inject_delay_max": "1.000000", 2024-09-14T06:28:59.345 INFO:teuthology.orchestra.run.smithi094.stdout: "ms_inject_delay_probability": "0.000000", 2024-09-14T06:28:59.345 INFO:teuthology.orchestra.run.smithi094.stdout: "ms_inject_delay_type": "", 2024-09-14T06:28:59.345 INFO:teuthology.orchestra.run.smithi094.stdout: "ms_inject_internal_delays": "0.000000", 2024-09-14T06:28:59.345 INFO:teuthology.orchestra.run.smithi094.stdout: "ms_inject_network_congestion": "0", 2024-09-14T06:28:59.345 INFO:teuthology.orchestra.run.smithi094.stdout: "ms_inject_socket_failures": "5000", 2024-09-14T06:28:59.345 INFO:teuthology.orchestra.run.smithi094.stdout: "ms_learn_addr_from_peer": "true", 2024-09-14T06:28:59.346 INFO:teuthology.orchestra.run.smithi094.stdout: "ms_max_accept_failures": "4", 2024-09-14T06:28:59.346 INFO:teuthology.orchestra.run.smithi094.stdout: "ms_max_backoff": "15.000000", 2024-09-14T06:28:59.346 INFO:teuthology.orchestra.run.smithi094.stdout: "ms_mon_client_mode": "secure crc", 2024-09-14T06:28:59.346 INFO:teuthology.orchestra.run.smithi094.stdout: "ms_mon_cluster_mode": "secure crc", 2024-09-14T06:28:59.346 INFO:teuthology.orchestra.run.smithi094.stdout: "ms_mon_service_mode": "secure crc", 2024-09-14T06:28:59.346 INFO:teuthology.orchestra.run.smithi094.stdout: "ms_osd_compress_min_size": "1024", 2024-09-14T06:28:59.346 INFO:teuthology.orchestra.run.smithi094.stdout: "ms_osd_compress_mode": "none", 2024-09-14T06:28:59.346 INFO:teuthology.orchestra.run.smithi094.stdout: "ms_osd_compression_algorithm": "snappy", 2024-09-14T06:28:59.346 INFO:teuthology.orchestra.run.smithi094.stdout: "ms_pq_max_tokens_per_priority": "16777216", 2024-09-14T06:28:59.346 INFO:teuthology.orchestra.run.smithi094.stdout: "ms_pq_min_cost": "65536", 2024-09-14T06:28:59.346 INFO:teuthology.orchestra.run.smithi094.stdout: "ms_public_type": "", 2024-09-14T06:28:59.346 INFO:teuthology.orchestra.run.smithi094.stdout: "ms_service_mode": "crc secure", 2024-09-14T06:28:59.346 INFO:teuthology.orchestra.run.smithi094.stdout: "ms_tcp_listen_backlog": "512", 2024-09-14T06:28:59.346 INFO:teuthology.orchestra.run.smithi094.stdout: "ms_tcp_nodelay": "true", 2024-09-14T06:28:59.346 INFO:teuthology.orchestra.run.smithi094.stdout: "ms_tcp_prefetch_max_size": "4096", 2024-09-14T06:28:59.347 INFO:teuthology.orchestra.run.smithi094.stdout: "ms_tcp_rcvbuf": "0", 2024-09-14T06:28:59.347 INFO:teuthology.orchestra.run.smithi094.stdout: "ms_time_events_min_wait_interval": "1000", 2024-09-14T06:28:59.347 INFO:teuthology.orchestra.run.smithi094.stdout: "ms_type": "async+posix", 2024-09-14T06:28:59.347 INFO:teuthology.orchestra.run.smithi094.stdout: "no_config_file": "false", 2024-09-14T06:28:59.347 INFO:teuthology.orchestra.run.smithi094.stdout: "nvmeof_mon_client_disconnect_panic": "100", 2024-09-14T06:28:59.347 INFO:teuthology.orchestra.run.smithi094.stdout: "nvmeof_mon_client_tick_period": "2", 2024-09-14T06:28:59.347 INFO:teuthology.orchestra.run.smithi094.stdout: "objecter_completion_locks_per_session": "32", 2024-09-14T06:28:59.347 INFO:teuthology.orchestra.run.smithi094.stdout: "objecter_debug_inject_relock_delay": "false", 2024-09-14T06:28:59.347 INFO:teuthology.orchestra.run.smithi094.stdout: "objecter_inflight_op_bytes": "104857600", 2024-09-14T06:28:59.347 INFO:teuthology.orchestra.run.smithi094.stdout: "objecter_inflight_ops": "1024", 2024-09-14T06:28:59.347 INFO:teuthology.orchestra.run.smithi094.stdout: "objecter_inject_no_watch_ping": "false", 2024-09-14T06:28:59.347 INFO:teuthology.orchestra.run.smithi094.stdout: "objecter_retry_writes_after_first_reply": "false", 2024-09-14T06:28:59.347 INFO:teuthology.orchestra.run.smithi094.stdout: "objecter_tick_interval": "5.000000", 2024-09-14T06:28:59.347 INFO:teuthology.orchestra.run.smithi094.stdout: "objecter_timeout": "10.000000", 2024-09-14T06:28:59.348 INFO:teuthology.orchestra.run.smithi094.stdout: "objectstore_blackhole": "false", 2024-09-14T06:28:59.348 INFO:teuthology.orchestra.run.smithi094.stdout: "objectstore_debug_throw_on_failed_txc": "false", 2024-09-14T06:28:59.348 INFO:teuthology.orchestra.run.smithi094.stdout: "openssl_engine_opts": "", 2024-09-14T06:28:59.348 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_agent_delay_time": "5.000000", 2024-09-14T06:28:59.348 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_agent_hist_halflife": "1000", 2024-09-14T06:28:59.348 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_agent_max_low_ops": "2", 2024-09-14T06:28:59.348 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_agent_max_ops": "4", 2024-09-14T06:28:59.348 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_agent_min_evict_effort": "0.100000", 2024-09-14T06:28:59.348 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_agent_quantize_effort": "0.100000", 2024-09-14T06:28:59.348 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_agent_slop": "0.020000", 2024-09-14T06:28:59.349 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_aggregated_slow_ops_logging": "true", 2024-09-14T06:28:59.349 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_allow_recovery_below_min_size": "true", 2024-09-14T06:28:59.349 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_asio_thread_count": "2", 2024-09-14T06:28:59.349 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_async_recovery_min_cost": "100", 2024-09-14T06:28:59.349 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_auto_mark_unfound_lost": "false", 2024-09-14T06:28:59.349 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_backfill_retry_interval": "30.000000", 2024-09-14T06:28:59.349 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_backfill_scan_max": "512", 2024-09-14T06:28:59.349 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_backfill_scan_min": "64", 2024-09-14T06:28:59.349 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_backoff_on_degraded": "false", 2024-09-14T06:28:59.349 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_backoff_on_peering": "false", 2024-09-14T06:28:59.349 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_backoff_on_unfound": "true", 2024-09-14T06:28:59.349 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_beacon_report_interval": "300", 2024-09-14T06:28:59.349 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_bench_duration": "30", 2024-09-14T06:28:59.349 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_bench_large_size_max_throughput": "104857600", 2024-09-14T06:28:59.350 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_bench_max_block_size": "67108864", 2024-09-14T06:28:59.350 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_bench_small_size_max_iops": "100", 2024-09-14T06:28:59.350 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_blkin_trace_all": "false", 2024-09-14T06:28:59.350 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_blocked_scrub_grace_period": "120", 2024-09-14T06:28:59.350 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_calc_pg_upmaps_aggressively": "true", 2024-09-14T06:28:59.350 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_calc_pg_upmaps_aggressively_fast": "true", 2024-09-14T06:28:59.350 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_calc_pg_upmaps_local_fallback_retries": "100", 2024-09-14T06:28:59.350 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_check_for_log_corruption": "false", 2024-09-14T06:28:59.350 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_check_max_object_name_len_on_startup": "true", 2024-09-14T06:28:59.350 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_class_default_list": "cephfs hello journal lock log numops otp rbd refcount rgw rgw_gc timeindex user version cas cmpomap queue 2pc_queue fifo", 2024-09-14T06:28:59.350 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_class_dir": "/usr/lib64/rados-classes", 2024-09-14T06:28:59.350 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_class_load_list": "cephfs hello journal lock log numops otp rbd refcount rgw rgw_gc timeindex user version cas cmpomap queue 2pc_queue fifo", 2024-09-14T06:28:59.350 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_class_update_on_start": "true", 2024-09-14T06:28:59.351 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_client_message_cap": "256", 2024-09-14T06:28:59.351 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_client_message_size_cap": "524288000", 2024-09-14T06:28:59.351 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_client_op_priority": "63", 2024-09-14T06:28:59.351 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_client_watch_timeout": "30", 2024-09-14T06:28:59.351 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_command_max_records": "256", 2024-09-14T06:28:59.351 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_compact_on_start": "false", 2024-09-14T06:28:59.351 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_copyfrom_max_chunk": "8388608", 2024-09-14T06:28:59.351 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_crush_chooseleaf_type": "0", 2024-09-14T06:28:59.351 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_crush_initial_weight": "-1.000000", 2024-09-14T06:28:59.351 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_crush_update_on_start": "true", 2024-09-14T06:28:59.351 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_crush_update_weight_set": "true", 2024-09-14T06:28:59.351 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_data": "/var/lib/ceph/osd/ceph-a", 2024-09-14T06:28:59.351 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_debug_crash_on_ignored_backoff": "false", 2024-09-14T06:28:59.352 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_debug_deep_scrub_sleep": "0.000000", 2024-09-14T06:28:59.352 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_debug_drop_ping_duration": "0", 2024-09-14T06:28:59.352 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_debug_drop_ping_probability": "0.000000", 2024-09-14T06:28:59.352 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_debug_feed_pullee": "-1", 2024-09-14T06:28:59.352 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_debug_inject_copyfrom_error": "false", 2024-09-14T06:28:59.352 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_debug_inject_dispatch_delay_duration": "0.100000", 2024-09-14T06:28:59.352 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_debug_inject_dispatch_delay_probability": "0.000000", 2024-09-14T06:28:59.352 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_debug_misdirected_ops": "false", 2024-09-14T06:28:59.352 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_debug_no_acting_change": "false", 2024-09-14T06:28:59.352 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_debug_no_purge_strays": "false", 2024-09-14T06:28:59.352 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_debug_op_order": "false", 2024-09-14T06:28:59.352 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_debug_pg_log_writeout": "false", 2024-09-14T06:28:59.352 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_debug_pretend_recovery_active": "false", 2024-09-14T06:28:59.352 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_debug_random_push_read_error": "0.000000", 2024-09-14T06:28:59.353 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_debug_reject_backfill_probability": "0.000000", 2024-09-14T06:28:59.353 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_debug_shutdown": "false", 2024-09-14T06:28:59.353 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_debug_skip_full_check_in_backfill_reservation": "false", 2024-09-14T06:28:59.353 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_debug_skip_full_check_in_recovery": "false", 2024-09-14T06:28:59.353 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_debug_trim_objects": "false", 2024-09-14T06:28:59.353 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_debug_verify_cached_snaps": "false", 2024-09-14T06:28:59.353 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_debug_verify_missing_on_start": "false", 2024-09-14T06:28:59.353 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_debug_verify_snaps": "false", 2024-09-14T06:28:59.353 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_debug_verify_stray_on_activate": "false", 2024-09-14T06:28:59.353 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_deep_scrub_interval": "604800.000000", 2024-09-14T06:28:59.353 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_deep_scrub_keys": "1024", 2024-09-14T06:28:59.353 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_deep_scrub_large_omap_object_key_threshold": "200000", 2024-09-14T06:28:59.353 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_deep_scrub_large_omap_object_value_sum_threshold": "1073741824", 2024-09-14T06:28:59.353 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_deep_scrub_randomize_ratio": "0.150000", 2024-09-14T06:28:59.354 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_deep_scrub_stride": "524288", 2024-09-14T06:28:59.354 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_deep_scrub_update_digest_min_age": "7200", 2024-09-14T06:28:59.354 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_default_data_pool_replay_window": "5", 2024-09-14T06:28:59.354 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_default_notify_timeout": "30", 2024-09-14T06:28:59.354 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_delete_sleep": "0.000000", 2024-09-14T06:28:59.354 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_delete_sleep_hdd": "5.000000", 2024-09-14T06:28:59.354 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_delete_sleep_hybrid": "1.000000", 2024-09-14T06:28:59.354 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_delete_sleep_ssd": "1.000000", 2024-09-14T06:28:59.354 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_discard_disconnected_ops": "true", 2024-09-14T06:28:59.354 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_ec_partial_reads": "true", 2024-09-14T06:28:59.354 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_enable_op_tracker": "true", 2024-09-14T06:28:59.354 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_erasure_code_plugins": "jerasure lrc", 2024-09-14T06:28:59.354 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_extblkdev_plugins": "vdo", 2024-09-14T06:28:59.354 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_failsafe_full_ratio": "0.970000", 2024-09-14T06:28:59.354 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_fast_fail_on_connection_refused": "true", 2024-09-14T06:28:59.355 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_fast_info": "true", 2024-09-14T06:28:59.355 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_fast_shutdown": "true", 2024-09-14T06:28:59.355 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_fast_shutdown_notify_mon": "true", 2024-09-14T06:28:59.355 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_fast_shutdown_timeout": "15", 2024-09-14T06:28:59.355 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_find_best_info_ignore_history_les": "false", 2024-09-14T06:28:59.355 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_force_auth_primary_missing_objects": "100", 2024-09-14T06:28:59.355 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_force_recovery_pg_log_entries_factor": "1.300000", 2024-09-14T06:28:59.355 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_function_tracing": "false", 2024-09-14T06:28:59.355 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_heartbeat_grace": "20", 2024-09-14T06:28:59.355 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_heartbeat_interval": "6", 2024-09-14T06:28:59.355 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_heartbeat_min_healthy_ratio": "0.330000", 2024-09-14T06:28:59.355 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_heartbeat_min_peers": "10", 2024-09-14T06:28:59.355 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_heartbeat_min_size": "2000", 2024-09-14T06:28:59.355 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_heartbeat_stale": "600", 2024-09-14T06:28:59.356 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_heartbeat_use_min_delay_socket": "false", 2024-09-14T06:28:59.356 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_hit_set_max_size": "100000", 2024-09-14T06:28:59.356 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_hit_set_min_size": "1000", 2024-09-14T06:28:59.356 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_hit_set_namespace": ".ceph-internal", 2024-09-14T06:28:59.356 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_ignore_stale_divergent_priors": "false", 2024-09-14T06:28:59.356 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_inject_bad_map_crc_probability": "0.000000", 2024-09-14T06:28:59.356 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_inject_failure_on_pg_removal": "false", 2024-09-14T06:28:59.356 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_journal": "/var/lib/ceph/osd/ceph-a/journal", 2024-09-14T06:28:59.356 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_journal_flush_on_shutdown": "true", 2024-09-14T06:28:59.356 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_journal_size": "5120", 2024-09-14T06:28:59.356 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_kill_backfill_at": "0", 2024-09-14T06:28:59.356 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_loop_before_reset_tphandle": "64", 2024-09-14T06:28:59.356 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_map_cache_size": "50", 2024-09-14T06:28:59.357 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_map_dedup": "true", 2024-09-14T06:28:59.357 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_map_message_max": "40", 2024-09-14T06:28:59.357 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_map_message_max_bytes": "10485760", 2024-09-14T06:28:59.357 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_map_share_max_epochs": "40", 2024-09-14T06:28:59.357 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_max_attr_name_len": "100", 2024-09-14T06:28:59.357 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_max_attr_size": "0", 2024-09-14T06:28:59.357 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_max_backfills": "1", 2024-09-14T06:28:59.357 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_max_markdown_count": "5", 2024-09-14T06:28:59.357 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_max_markdown_period": "600", 2024-09-14T06:28:59.357 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_max_object_name_len": "2048", 2024-09-14T06:28:59.357 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_max_object_namespace_len": "256", 2024-09-14T06:28:59.357 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_max_object_size": "134217728", 2024-09-14T06:28:59.357 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_max_omap_bytes_per_request": "1073741824", 2024-09-14T06:28:59.357 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_max_omap_entries_per_request": "1024", 2024-09-14T06:28:59.358 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_max_pg_blocked_by": "16", 2024-09-14T06:28:59.358 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_max_pg_log_entries": "10000", 2024-09-14T06:28:59.358 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_max_pg_per_osd_hard_ratio": "3.000000", 2024-09-14T06:28:59.358 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_max_pgls": "1024", 2024-09-14T06:28:59.358 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_max_push_cost": "8388608", 2024-09-14T06:28:59.358 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_max_push_objects": "10", 2024-09-14T06:28:59.358 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_max_scrubs": "3", 2024-09-14T06:28:59.358 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_max_snap_prune_intervals_per_epoch": "512", 2024-09-14T06:28:59.358 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_max_trimming_pgs": "2", 2024-09-14T06:28:59.358 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_max_write_op_reply_len": "64", 2024-09-14T06:28:59.358 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_max_write_size": "90", 2024-09-14T06:28:59.358 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_mclock_force_run_benchmark_on_init": "false", 2024-09-14T06:28:59.358 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_mclock_iops_capacity_threshold_hdd": "500.000000", 2024-09-14T06:28:59.358 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_mclock_iops_capacity_threshold_ssd": "80000.000000", 2024-09-14T06:28:59.359 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_mclock_max_capacity_iops_hdd": "315.000000", 2024-09-14T06:28:59.359 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_mclock_max_capacity_iops_ssd": "21500.000000", 2024-09-14T06:28:59.359 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_mclock_max_sequential_bandwidth_hdd": "157286400", 2024-09-14T06:28:59.359 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_mclock_max_sequential_bandwidth_ssd": "1258291200", 2024-09-14T06:28:59.359 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_mclock_override_recovery_settings": "false", 2024-09-14T06:28:59.359 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_mclock_profile": "balanced", 2024-09-14T06:28:59.359 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_mclock_scheduler_anticipation_timeout": "0.000000", 2024-09-14T06:28:59.359 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_mclock_scheduler_background_best_effort_lim": "0.000000", 2024-09-14T06:28:59.359 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_mclock_scheduler_background_best_effort_res": "0.000000", 2024-09-14T06:28:59.359 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_mclock_scheduler_background_best_effort_wgt": "1", 2024-09-14T06:28:59.359 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_mclock_scheduler_background_recovery_lim": "0.000000", 2024-09-14T06:28:59.359 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_mclock_scheduler_background_recovery_res": "0.000000", 2024-09-14T06:28:59.359 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_mclock_scheduler_background_recovery_wgt": "1", 2024-09-14T06:28:59.359 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_mclock_scheduler_client_lim": "0.000000", 2024-09-14T06:28:59.360 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_mclock_scheduler_client_res": "0.000000", 2024-09-14T06:28:59.360 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_mclock_scheduler_client_wgt": "1", 2024-09-14T06:28:59.360 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_mclock_skip_benchmark": "false", 2024-09-14T06:28:59.360 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_memory_base": "805306368", 2024-09-14T06:28:59.360 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_memory_cache_min": "134217728", 2024-09-14T06:28:59.360 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_memory_cache_resize_interval": "1.000000", 2024-09-14T06:28:59.360 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_memory_expected_fragmentation": "0.150000", 2024-09-14T06:28:59.360 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_memory_target": "4294967296", 2024-09-14T06:28:59.360 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_memory_target_autotune": "false", 2024-09-14T06:28:59.360 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_memory_target_cgroup_limit_ratio": "0.800000", 2024-09-14T06:28:59.360 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_min_pg_log_entries": "250", 2024-09-14T06:28:59.360 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_min_recovery_priority": "0", 2024-09-14T06:28:59.360 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_mon_heartbeat_interval": "30", 2024-09-14T06:28:59.360 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_mon_heartbeat_stat_stale": "3600", 2024-09-14T06:28:59.361 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_mon_report_interval": "5", 2024-09-14T06:28:59.361 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_mon_report_max_in_flight": "2", 2024-09-14T06:28:59.361 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_mon_shutdown_timeout": "5.000000", 2024-09-14T06:28:59.361 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_num_cache_shards": "32", 2024-09-14T06:28:59.361 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_num_op_tracker_shard": "32", 2024-09-14T06:28:59.361 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_numa_auto_affinity": "true", 2024-09-14T06:28:59.361 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_numa_node": "-1", 2024-09-14T06:28:59.361 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_numa_prefer_iface": "true", 2024-09-14T06:28:59.361 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_object_clean_region_max_num_intervals": "10", 2024-09-14T06:28:59.361 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_objecter_finishers": "1", 2024-09-14T06:28:59.361 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_objectstore": "bluestore", 2024-09-14T06:28:59.361 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_objectstore_fuse": "false", 2024-09-14T06:28:59.361 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_objectstore_ideal_list_max": "64", 2024-09-14T06:28:59.362 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_objectstore_tracing": "false", 2024-09-14T06:28:59.362 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_op_complaint_time": "30.000000", 2024-09-14T06:28:59.362 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_op_history_duration": "600", 2024-09-14T06:28:59.362 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_op_history_size": "20", 2024-09-14T06:28:59.362 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_op_history_slow_op_size": "20", 2024-09-14T06:28:59.362 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_op_history_slow_op_threshold": "10.000000", 2024-09-14T06:28:59.362 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_op_log_threshold": "5", 2024-09-14T06:28:59.362 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_op_num_shards": "0", 2024-09-14T06:28:59.362 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_op_num_shards_hdd": "5", 2024-09-14T06:28:59.362 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_op_num_shards_ssd": "8", 2024-09-14T06:28:59.362 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_op_num_threads_per_shard": "0", 2024-09-14T06:28:59.362 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_op_num_threads_per_shard_hdd": "1", 2024-09-14T06:28:59.362 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_op_num_threads_per_shard_ssd": "2", 2024-09-14T06:28:59.362 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_op_pq_max_tokens_per_priority": "4194304", 2024-09-14T06:28:59.362 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_op_pq_min_cost": "65536", 2024-09-14T06:28:59.362 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_op_queue": "mclock_scheduler", 2024-09-14T06:28:59.363 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_op_queue_cut_off": "high", 2024-09-14T06:28:59.363 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_op_thread_suicide_timeout": "150", 2024-09-14T06:28:59.363 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_op_thread_timeout": "15", 2024-09-14T06:28:59.363 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_open_classes_on_start": "true", 2024-09-14T06:28:59.363 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_os_flags": "0", 2024-09-14T06:28:59.363 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_peering_op_priority": "255", 2024-09-14T06:28:59.363 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_pg_delete_cost": "1048576", 2024-09-14T06:28:59.363 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_pg_delete_priority": "5", 2024-09-14T06:28:59.363 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_pg_epoch_max_lag_factor": "2.000000", 2024-09-14T06:28:59.363 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_pg_epoch_persisted_max_stale": "40", 2024-09-14T06:28:59.363 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_pg_log_dups_tracked": "3000", 2024-09-14T06:28:59.363 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_pg_log_trim_max": "10000", 2024-09-14T06:28:59.363 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_pg_log_trim_min": "100", 2024-09-14T06:28:59.363 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_pg_max_concurrent_snap_trims": "2", 2024-09-14T06:28:59.364 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_pg_object_context_cache_count": "64", 2024-09-14T06:28:59.364 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_pg_stat_report_interval_max_epochs": "500", 2024-09-14T06:28:59.364 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_pg_stat_report_interval_max_seconds": "5", 2024-09-14T06:28:59.364 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_pool_default_cache_max_evict_check_size": "10", 2024-09-14T06:28:59.364 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_pool_default_cache_min_evict_age": "0", 2024-09-14T06:28:59.364 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_pool_default_cache_min_flush_age": "0", 2024-09-14T06:28:59.364 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_pool_default_cache_target_dirty_high_ratio": "0.600000", 2024-09-14T06:28:59.364 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_pool_default_cache_target_dirty_ratio": "0.400000", 2024-09-14T06:28:59.364 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_pool_default_cache_target_full_ratio": "0.800000", 2024-09-14T06:28:59.364 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_pool_default_crimson": "false", 2024-09-14T06:28:59.364 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_pool_default_crush_rule": "-1", 2024-09-14T06:28:59.364 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_pool_default_ec_fast_read": "false", 2024-09-14T06:28:59.364 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_pool_default_erasure_code_profile": "plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd", 2024-09-14T06:28:59.365 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_pool_default_flag_bulk": "false", 2024-09-14T06:28:59.365 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_pool_default_flag_hashpspool": "true", 2024-09-14T06:28:59.365 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_pool_default_flag_nodelete": "false", 2024-09-14T06:28:59.365 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_pool_default_flag_nopgchange": "false", 2024-09-14T06:28:59.365 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_pool_default_flag_nosizechange": "false", 2024-09-14T06:28:59.365 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_pool_default_flags": "0", 2024-09-14T06:28:59.365 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_pool_default_hit_set_bloom_fpp": "0.050000", 2024-09-14T06:28:59.365 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_pool_default_min_size": "0", 2024-09-14T06:28:59.365 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_pool_default_pg_autoscale_mode": "off", 2024-09-14T06:28:59.365 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_pool_default_pg_num": "32", 2024-09-14T06:28:59.365 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_pool_default_pgp_num": "0", 2024-09-14T06:28:59.365 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_pool_default_read_lease_ratio": "0.800000", 2024-09-14T06:28:59.365 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_pool_default_read_ratio": "70", 2024-09-14T06:28:59.365 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_pool_default_size": "2", 2024-09-14T06:28:59.366 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_pool_default_type": "replicated", 2024-09-14T06:28:59.366 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_pool_erasure_code_stripe_unit": "4096", 2024-09-14T06:28:59.366 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_pool_use_gmt_hitset": "true", 2024-09-14T06:28:59.366 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_push_per_object_cost": "1000", 2024-09-14T06:28:59.366 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_read_ec_check_for_errors": "false", 2024-09-14T06:28:59.366 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_recover_clone_overlap": "true", 2024-09-14T06:28:59.366 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_recover_clone_overlap_limit": "10", 2024-09-14T06:28:59.366 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_recovery_cost": "20971520", 2024-09-14T06:28:59.366 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_recovery_delay_start": "0.000000", 2024-09-14T06:28:59.366 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_recovery_max_active": "0", 2024-09-14T06:28:59.366 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_recovery_max_active_hdd": "3", 2024-09-14T06:28:59.366 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_recovery_max_active_ssd": "10", 2024-09-14T06:28:59.366 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_recovery_max_chunk": "8388608", 2024-09-14T06:28:59.367 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_recovery_max_omap_entries_per_chunk": "8096", 2024-09-14T06:28:59.367 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_recovery_max_single_start": "1", 2024-09-14T06:28:59.367 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_recovery_op_priority": "3", 2024-09-14T06:28:59.367 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_recovery_op_warn_multiple": "16", 2024-09-14T06:28:59.367 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_recovery_priority": "5", 2024-09-14T06:28:59.367 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_recovery_retry_interval": "30.000000", 2024-09-14T06:28:59.367 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_recovery_sleep": "0.000000", 2024-09-14T06:28:59.367 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_recovery_sleep_hdd": "0.100000", 2024-09-14T06:28:59.367 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_recovery_sleep_hybrid": "0.025000", 2024-09-14T06:28:59.367 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_recovery_sleep_ssd": "0.000000", 2024-09-14T06:28:59.367 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_repair_during_recovery": "false", 2024-09-14T06:28:59.367 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_requested_scrub_priority": "120", 2024-09-14T06:28:59.367 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_rocksdb_iterator_bounds_enabled": "true", 2024-09-14T06:28:59.368 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_rollback_to_cluster_snap": "", 2024-09-14T06:28:59.368 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_scrub_auto_repair": "false", 2024-09-14T06:28:59.368 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_scrub_auto_repair_num_errors": "5", 2024-09-14T06:28:59.368 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_scrub_backoff_ratio": "0.660000", 2024-09-14T06:28:59.368 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_scrub_begin_hour": "0", 2024-09-14T06:28:59.368 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_scrub_begin_week_day": "0", 2024-09-14T06:28:59.368 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_scrub_chunk_max": "25", 2024-09-14T06:28:59.368 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_scrub_chunk_min": "5", 2024-09-14T06:28:59.368 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_scrub_cost": "52428800", 2024-09-14T06:28:59.368 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_scrub_disable_reservation_queuing": "false", 2024-09-14T06:28:59.368 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_scrub_during_recovery": "false", 2024-09-14T06:28:59.368 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_scrub_end_hour": "0", 2024-09-14T06:28:59.368 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_scrub_end_week_day": "0", 2024-09-14T06:28:59.368 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_scrub_event_cost": "4096", 2024-09-14T06:28:59.369 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_scrub_extended_sleep": "0.000000", 2024-09-14T06:28:59.369 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_scrub_interval_randomize_ratio": "0.500000", 2024-09-14T06:28:59.369 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_scrub_invalid_stats": "true", 2024-09-14T06:28:59.369 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_scrub_load_threshold": "0.500000", 2024-09-14T06:28:59.369 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_scrub_max_interval": "604800.000000", 2024-09-14T06:28:59.369 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_scrub_max_preemptions": "5", 2024-09-14T06:28:59.369 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_scrub_min_interval": "86400.000000", 2024-09-14T06:28:59.369 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_scrub_priority": "5", 2024-09-14T06:28:59.369 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_scrub_sleep": "0.000000", 2024-09-14T06:28:59.369 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_shallow_scrub_chunk_max": "100", 2024-09-14T06:28:59.369 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_shallow_scrub_chunk_min": "50", 2024-09-14T06:28:59.369 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_shutdown_pgref_assert": "false", 2024-09-14T06:28:59.369 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_skip_check_past_interval_bounds": "false", 2024-09-14T06:28:59.369 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_skip_data_digest": "false", 2024-09-14T06:28:59.370 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_smart_report_timeout": "5", 2024-09-14T06:28:59.370 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_snap_trim_cost": "1048576", 2024-09-14T06:28:59.370 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_snap_trim_priority": "5", 2024-09-14T06:28:59.370 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_snap_trim_sleep": "0.000000", 2024-09-14T06:28:59.370 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_snap_trim_sleep_hdd": "5.000000", 2024-09-14T06:28:59.370 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_snap_trim_sleep_hybrid": "2.000000", 2024-09-14T06:28:59.370 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_snap_trim_sleep_ssd": "0.000000", 2024-09-14T06:28:59.370 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_stats_update_period_not_scrubbing": "120", 2024-09-14T06:28:59.370 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_stats_update_period_scrubbing": "15", 2024-09-14T06:28:59.370 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_target_pg_log_entries_per_osd": "300000", 2024-09-14T06:28:59.370 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_target_transaction_size": "30", 2024-09-14T06:28:59.370 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_tier_default_cache_hit_set_count": "4", 2024-09-14T06:28:59.370 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_tier_default_cache_hit_set_grade_decay_rate": "20", 2024-09-14T06:28:59.370 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_tier_default_cache_hit_set_period": "1200", 2024-09-14T06:28:59.371 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_tier_default_cache_hit_set_search_last_n": "1", 2024-09-14T06:28:59.371 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_tier_default_cache_hit_set_type": "bloom", 2024-09-14T06:28:59.371 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_tier_default_cache_min_read_recency_for_promote": "1", 2024-09-14T06:28:59.371 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_tier_default_cache_min_write_recency_for_promote": "1", 2024-09-14T06:28:59.371 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_tier_default_cache_mode": "writeback", 2024-09-14T06:28:59.371 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_tier_promote_max_bytes_sec": "5242880", 2024-09-14T06:28:59.371 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_tier_promote_max_objects_sec": "25", 2024-09-14T06:28:59.371 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_tracing": "false", 2024-09-14T06:28:59.371 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_use_stale_snap": "false", 2024-09-14T06:28:59.371 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_uuid": "00000000-0000-0000-0000-000000000000", 2024-09-14T06:28:59.371 INFO:teuthology.orchestra.run.smithi094.stdout: "osdc_blkin_trace_all": "false", 2024-09-14T06:28:59.371 INFO:teuthology.orchestra.run.smithi094.stdout: "paxos_kill_at": "0", 2024-09-14T06:28:59.371 INFO:teuthology.orchestra.run.smithi094.stdout: "paxos_max_join_drift": "10", 2024-09-14T06:28:59.371 INFO:teuthology.orchestra.run.smithi094.stdout: "paxos_min": "500", 2024-09-14T06:28:59.372 INFO:teuthology.orchestra.run.smithi094.stdout: "paxos_min_wait": "0.050000", 2024-09-14T06:28:59.372 INFO:teuthology.orchestra.run.smithi094.stdout: "paxos_propose_interval": "1.000000", 2024-09-14T06:28:59.372 INFO:teuthology.orchestra.run.smithi094.stdout: "paxos_service_trim_max": "500", 2024-09-14T06:28:59.372 INFO:teuthology.orchestra.run.smithi094.stdout: "paxos_service_trim_max_multiplier": "20", 2024-09-14T06:28:59.372 INFO:teuthology.orchestra.run.smithi094.stdout: "paxos_service_trim_min": "250", 2024-09-14T06:28:59.372 INFO:teuthology.orchestra.run.smithi094.stdout: "paxos_stash_full_interval": "25", 2024-09-14T06:28:59.372 INFO:teuthology.orchestra.run.smithi094.stdout: "paxos_trim_max": "500", 2024-09-14T06:28:59.372 INFO:teuthology.orchestra.run.smithi094.stdout: "paxos_trim_min": "250", 2024-09-14T06:28:59.372 INFO:teuthology.orchestra.run.smithi094.stdout: "perf": "true", 2024-09-14T06:28:59.372 INFO:teuthology.orchestra.run.smithi094.stdout: "pid_file": "/var/run/ceph/ceph-mon.a.pid", 2024-09-14T06:28:59.372 INFO:teuthology.orchestra.run.smithi094.stdout: "plugin_crypto_accelerator": "crypto_isal", 2024-09-14T06:28:59.372 INFO:teuthology.orchestra.run.smithi094.stdout: "plugin_dir": "/usr/lib64/ceph", 2024-09-14T06:28:59.372 INFO:teuthology.orchestra.run.smithi094.stdout: "public_addr": "-", 2024-09-14T06:28:59.372 INFO:teuthology.orchestra.run.smithi094.stdout: "public_addrv": "", 2024-09-14T06:28:59.373 INFO:teuthology.orchestra.run.smithi094.stdout: "public_bind_addr": "-", 2024-09-14T06:28:59.373 INFO:teuthology.orchestra.run.smithi094.stdout: "public_network": "", 2024-09-14T06:28:59.373 INFO:teuthology.orchestra.run.smithi094.stdout: "public_network_interface": "", 2024-09-14T06:28:59.373 INFO:teuthology.orchestra.run.smithi094.stdout: "qat_compressor_busy_polling": "false", 2024-09-14T06:28:59.373 INFO:teuthology.orchestra.run.smithi094.stdout: "qat_compressor_enabled": "false", 2024-09-14T06:28:59.373 INFO:teuthology.orchestra.run.smithi094.stdout: "qat_compressor_session_max_number": "256", 2024-09-14T06:28:59.373 INFO:teuthology.orchestra.run.smithi094.stdout: "rados_mon_op_timeout": "0", 2024-09-14T06:28:59.373 INFO:teuthology.orchestra.run.smithi094.stdout: "rados_osd_op_timeout": "0", 2024-09-14T06:28:59.373 INFO:teuthology.orchestra.run.smithi094.stdout: "rados_tracing": "false", 2024-09-14T06:28:59.373 INFO:teuthology.orchestra.run.smithi094.stdout: "rbd_atime_update_interval": "60", 2024-09-14T06:28:59.373 INFO:teuthology.orchestra.run.smithi094.stdout: "rbd_auto_exclusive_lock_until_manual_request": "true", 2024-09-14T06:28:59.373 INFO:teuthology.orchestra.run.smithi094.stdout: "rbd_balance_parent_reads": "false", 2024-09-14T06:28:59.373 INFO:teuthology.orchestra.run.smithi094.stdout: "rbd_balance_snap_reads": "false", 2024-09-14T06:28:59.373 INFO:teuthology.orchestra.run.smithi094.stdout: "rbd_blkin_trace_all": "false", 2024-09-14T06:28:59.374 INFO:teuthology.orchestra.run.smithi094.stdout: "rbd_blocklist_expire_seconds": "0", 2024-09-14T06:28:59.374 INFO:teuthology.orchestra.run.smithi094.stdout: "rbd_blocklist_on_break_lock": "true", 2024-09-14T06:28:59.374 INFO:teuthology.orchestra.run.smithi094.stdout: "rbd_cache": "true", 2024-09-14T06:28:59.374 INFO:teuthology.orchestra.run.smithi094.stdout: "rbd_cache_block_writes_upfront": "false", 2024-09-14T06:28:59.374 INFO:teuthology.orchestra.run.smithi094.stdout: "rbd_cache_max_dirty": "25165824", 2024-09-14T06:28:59.374 INFO:teuthology.orchestra.run.smithi094.stdout: "rbd_cache_max_dirty_age": "1.000000", 2024-09-14T06:28:59.374 INFO:teuthology.orchestra.run.smithi094.stdout: "rbd_cache_max_dirty_object": "0", 2024-09-14T06:28:59.374 INFO:teuthology.orchestra.run.smithi094.stdout: "rbd_cache_policy": "writearound", 2024-09-14T06:28:59.374 INFO:teuthology.orchestra.run.smithi094.stdout: "rbd_cache_size": "33554432", 2024-09-14T06:28:59.374 INFO:teuthology.orchestra.run.smithi094.stdout: "rbd_cache_target_dirty": "16777216", 2024-09-14T06:28:59.374 INFO:teuthology.orchestra.run.smithi094.stdout: "rbd_cache_writethrough_until_flush": "true", 2024-09-14T06:28:59.374 INFO:teuthology.orchestra.run.smithi094.stdout: "rbd_clone_copy_on_read": "false", 2024-09-14T06:28:59.374 INFO:teuthology.orchestra.run.smithi094.stdout: "rbd_compression_hint": "none", 2024-09-14T06:28:59.374 INFO:teuthology.orchestra.run.smithi094.stdout: "rbd_concurrent_management_ops": "10", 2024-09-14T06:28:59.375 INFO:teuthology.orchestra.run.smithi094.stdout: "rbd_config_pool_override_update_timestamp": "0", 2024-09-14T06:28:59.375 INFO:teuthology.orchestra.run.smithi094.stdout: "rbd_default_clone_format": "auto", 2024-09-14T06:28:59.375 INFO:teuthology.orchestra.run.smithi094.stdout: "rbd_default_data_pool": "", 2024-09-14T06:28:59.375 INFO:teuthology.orchestra.run.smithi094.stdout: "rbd_default_features": "61", 2024-09-14T06:28:59.375 INFO:teuthology.orchestra.run.smithi094.stdout: "rbd_default_format": "2", 2024-09-14T06:28:59.375 INFO:teuthology.orchestra.run.smithi094.stdout: "rbd_default_map_options": "", 2024-09-14T06:28:59.375 INFO:teuthology.orchestra.run.smithi094.stdout: "rbd_default_order": "22", 2024-09-14T06:28:59.375 INFO:teuthology.orchestra.run.smithi094.stdout: "rbd_default_pool": "rbd", 2024-09-14T06:28:59.375 INFO:teuthology.orchestra.run.smithi094.stdout: "rbd_default_snapshot_quiesce_mode": "required", 2024-09-14T06:28:59.375 INFO:teuthology.orchestra.run.smithi094.stdout: "rbd_default_stripe_count": "0", 2024-09-14T06:28:59.375 INFO:teuthology.orchestra.run.smithi094.stdout: "rbd_default_stripe_unit": "0", 2024-09-14T06:28:59.375 INFO:teuthology.orchestra.run.smithi094.stdout: "rbd_disable_zero_copy_writes": "true", 2024-09-14T06:28:59.375 INFO:teuthology.orchestra.run.smithi094.stdout: "rbd_discard_granularity_bytes": "65536", 2024-09-14T06:28:59.375 INFO:teuthology.orchestra.run.smithi094.stdout: "rbd_discard_on_zeroed_write_same": "true", 2024-09-14T06:28:59.376 INFO:teuthology.orchestra.run.smithi094.stdout: "rbd_enable_alloc_hint": "true", 2024-09-14T06:28:59.376 INFO:teuthology.orchestra.run.smithi094.stdout: "rbd_invalidate_object_map_on_timeout": "true", 2024-09-14T06:28:59.376 INFO:teuthology.orchestra.run.smithi094.stdout: "rbd_io_scheduler": "simple", 2024-09-14T06:28:59.376 INFO:teuthology.orchestra.run.smithi094.stdout: "rbd_io_scheduler_simple_max_delay": "0", 2024-09-14T06:28:59.376 INFO:teuthology.orchestra.run.smithi094.stdout: "rbd_journal_commit_age": "5.000000", 2024-09-14T06:28:59.376 INFO:teuthology.orchestra.run.smithi094.stdout: "rbd_journal_max_concurrent_object_sets": "0", 2024-09-14T06:28:59.376 INFO:teuthology.orchestra.run.smithi094.stdout: "rbd_journal_max_payload_bytes": "16384", 2024-09-14T06:28:59.376 INFO:teuthology.orchestra.run.smithi094.stdout: "rbd_journal_object_flush_age": "0.000000", 2024-09-14T06:28:59.376 INFO:teuthology.orchestra.run.smithi094.stdout: "rbd_journal_object_flush_bytes": "1048576", 2024-09-14T06:28:59.376 INFO:teuthology.orchestra.run.smithi094.stdout: "rbd_journal_object_flush_interval": "0", 2024-09-14T06:28:59.376 INFO:teuthology.orchestra.run.smithi094.stdout: "rbd_journal_object_max_in_flight_appends": "0", 2024-09-14T06:28:59.376 INFO:teuthology.orchestra.run.smithi094.stdout: "rbd_journal_object_writethrough_until_flush": "true", 2024-09-14T06:28:59.376 INFO:teuthology.orchestra.run.smithi094.stdout: "rbd_journal_order": "24", 2024-09-14T06:28:59.377 INFO:teuthology.orchestra.run.smithi094.stdout: "rbd_journal_pool": "", 2024-09-14T06:28:59.377 INFO:teuthology.orchestra.run.smithi094.stdout: "rbd_journal_splay_width": "4", 2024-09-14T06:28:59.377 INFO:teuthology.orchestra.run.smithi094.stdout: "rbd_localize_parent_reads": "false", 2024-09-14T06:28:59.377 INFO:teuthology.orchestra.run.smithi094.stdout: "rbd_localize_snap_reads": "false", 2024-09-14T06:28:59.377 INFO:teuthology.orchestra.run.smithi094.stdout: "rbd_mirror_concurrent_image_deletions": "1", 2024-09-14T06:28:59.377 INFO:teuthology.orchestra.run.smithi094.stdout: "rbd_mirror_concurrent_image_syncs": "5", 2024-09-14T06:28:59.377 INFO:teuthology.orchestra.run.smithi094.stdout: "rbd_mirror_delete_retry_interval": "30.000000", 2024-09-14T06:28:59.377 INFO:teuthology.orchestra.run.smithi094.stdout: "rbd_mirror_image_perf_stats_prio": "5", 2024-09-14T06:28:59.377 INFO:teuthology.orchestra.run.smithi094.stdout: "rbd_mirror_image_policy_migration_throttle": "300", 2024-09-14T06:28:59.377 INFO:teuthology.orchestra.run.smithi094.stdout: "rbd_mirror_image_policy_rebalance_timeout": "0.000000", 2024-09-14T06:28:59.377 INFO:teuthology.orchestra.run.smithi094.stdout: "rbd_mirror_image_policy_type": "simple", 2024-09-14T06:28:59.377 INFO:teuthology.orchestra.run.smithi094.stdout: "rbd_mirror_image_policy_update_throttle_interval": "1.000000", 2024-09-14T06:28:59.377 INFO:teuthology.orchestra.run.smithi094.stdout: "rbd_mirror_image_state_check_interval": "30", 2024-09-14T06:28:59.377 INFO:teuthology.orchestra.run.smithi094.stdout: "rbd_mirror_journal_commit_age": "5.000000", 2024-09-14T06:28:59.378 INFO:teuthology.orchestra.run.smithi094.stdout: "rbd_mirror_journal_poll_age": "5.000000", 2024-09-14T06:28:59.378 INFO:teuthology.orchestra.run.smithi094.stdout: "rbd_mirror_leader_heartbeat_interval": "5", 2024-09-14T06:28:59.378 INFO:teuthology.orchestra.run.smithi094.stdout: "rbd_mirror_leader_max_acquire_attempts_before_break": "3", 2024-09-14T06:28:59.378 INFO:teuthology.orchestra.run.smithi094.stdout: "rbd_mirror_leader_max_missed_heartbeats": "2", 2024-09-14T06:28:59.378 INFO:teuthology.orchestra.run.smithi094.stdout: "rbd_mirror_memory_autotune": "true", 2024-09-14T06:28:59.378 INFO:teuthology.orchestra.run.smithi094.stdout: "rbd_mirror_memory_base": "805306368", 2024-09-14T06:28:59.378 INFO:teuthology.orchestra.run.smithi094.stdout: "rbd_mirror_memory_cache_autotune_interval": "30.000000", 2024-09-14T06:28:59.378 INFO:teuthology.orchestra.run.smithi094.stdout: "rbd_mirror_memory_cache_min": "134217728", 2024-09-14T06:28:59.378 INFO:teuthology.orchestra.run.smithi094.stdout: "rbd_mirror_memory_cache_resize_interval": "5.000000", 2024-09-14T06:28:59.378 INFO:teuthology.orchestra.run.smithi094.stdout: "rbd_mirror_memory_expected_fragmentation": "0.150000", 2024-09-14T06:28:59.378 INFO:teuthology.orchestra.run.smithi094.stdout: "rbd_mirror_memory_target": "4294967296", 2024-09-14T06:28:59.378 INFO:teuthology.orchestra.run.smithi094.stdout: "rbd_mirror_perf_stats_prio": "5", 2024-09-14T06:28:59.378 INFO:teuthology.orchestra.run.smithi094.stdout: "rbd_mirror_pool_replayers_refresh_interval": "30", 2024-09-14T06:28:59.378 INFO:teuthology.orchestra.run.smithi094.stdout: "rbd_mirror_sync_point_update_age": "30.000000", 2024-09-14T06:28:59.379 INFO:teuthology.orchestra.run.smithi094.stdout: "rbd_mirroring_delete_delay": "0", 2024-09-14T06:28:59.379 INFO:teuthology.orchestra.run.smithi094.stdout: "rbd_mirroring_max_mirroring_snapshots": "5", 2024-09-14T06:28:59.379 INFO:teuthology.orchestra.run.smithi094.stdout: "rbd_mirroring_replay_delay": "0", 2024-09-14T06:28:59.379 INFO:teuthology.orchestra.run.smithi094.stdout: "rbd_mirroring_resync_after_disconnect": "false", 2024-09-14T06:28:59.379 INFO:teuthology.orchestra.run.smithi094.stdout: "rbd_move_parent_to_trash_on_remove": "false", 2024-09-14T06:28:59.379 INFO:teuthology.orchestra.run.smithi094.stdout: "rbd_move_to_trash_on_remove": "false", 2024-09-14T06:28:59.379 INFO:teuthology.orchestra.run.smithi094.stdout: "rbd_move_to_trash_on_remove_expire_seconds": "0", 2024-09-14T06:28:59.379 INFO:teuthology.orchestra.run.smithi094.stdout: "rbd_mtime_update_interval": "60", 2024-09-14T06:28:59.379 INFO:teuthology.orchestra.run.smithi094.stdout: "rbd_non_blocking_aio": "true", 2024-09-14T06:28:59.379 INFO:teuthology.orchestra.run.smithi094.stdout: "rbd_op_thread_timeout": "60", 2024-09-14T06:28:59.379 INFO:teuthology.orchestra.run.smithi094.stdout: "rbd_op_threads": "1", 2024-09-14T06:28:59.379 INFO:teuthology.orchestra.run.smithi094.stdout: "rbd_parent_cache_enabled": "false", 2024-09-14T06:28:59.379 INFO:teuthology.orchestra.run.smithi094.stdout: "rbd_persistent_cache_mode": "disabled", 2024-09-14T06:28:59.379 INFO:teuthology.orchestra.run.smithi094.stdout: "rbd_persistent_cache_path": "/tmp", 2024-09-14T06:28:59.379 INFO:teuthology.orchestra.run.smithi094.stdout: "rbd_persistent_cache_size": "1073741824", 2024-09-14T06:28:59.380 INFO:teuthology.orchestra.run.smithi094.stdout: "rbd_plugins": "", 2024-09-14T06:28:59.380 INFO:teuthology.orchestra.run.smithi094.stdout: "rbd_qos_bps_burst": "0", 2024-09-14T06:28:59.380 INFO:teuthology.orchestra.run.smithi094.stdout: "rbd_qos_bps_burst_seconds": "1", 2024-09-14T06:28:59.380 INFO:teuthology.orchestra.run.smithi094.stdout: "rbd_qos_bps_limit": "0", 2024-09-14T06:28:59.380 INFO:teuthology.orchestra.run.smithi094.stdout: "rbd_qos_exclude_ops": "0", 2024-09-14T06:28:59.380 INFO:teuthology.orchestra.run.smithi094.stdout: "rbd_qos_iops_burst": "0", 2024-09-14T06:28:59.380 INFO:teuthology.orchestra.run.smithi094.stdout: "rbd_qos_iops_burst_seconds": "1", 2024-09-14T06:28:59.380 INFO:teuthology.orchestra.run.smithi094.stdout: "rbd_qos_iops_limit": "0", 2024-09-14T06:28:59.380 INFO:teuthology.orchestra.run.smithi094.stdout: "rbd_qos_read_bps_burst": "0", 2024-09-14T06:28:59.380 INFO:teuthology.orchestra.run.smithi094.stdout: "rbd_qos_read_bps_burst_seconds": "1", 2024-09-14T06:28:59.380 INFO:teuthology.orchestra.run.smithi094.stdout: "rbd_qos_read_bps_limit": "0", 2024-09-14T06:28:59.380 INFO:teuthology.orchestra.run.smithi094.stdout: "rbd_qos_read_iops_burst": "0", 2024-09-14T06:28:59.380 INFO:teuthology.orchestra.run.smithi094.stdout: "rbd_qos_read_iops_burst_seconds": "1", 2024-09-14T06:28:59.381 INFO:teuthology.orchestra.run.smithi094.stdout: "rbd_qos_read_iops_limit": "0", 2024-09-14T06:28:59.381 INFO:teuthology.orchestra.run.smithi094.stdout: "rbd_qos_schedule_tick_min": "50", 2024-09-14T06:28:59.381 INFO:teuthology.orchestra.run.smithi094.stdout: "rbd_qos_write_bps_burst": "0", 2024-09-14T06:28:59.381 INFO:teuthology.orchestra.run.smithi094.stdout: "rbd_qos_write_bps_burst_seconds": "1", 2024-09-14T06:28:59.381 INFO:teuthology.orchestra.run.smithi094.stdout: "rbd_qos_write_bps_limit": "0", 2024-09-14T06:28:59.381 INFO:teuthology.orchestra.run.smithi094.stdout: "rbd_qos_write_iops_burst": "0", 2024-09-14T06:28:59.381 INFO:teuthology.orchestra.run.smithi094.stdout: "rbd_qos_write_iops_burst_seconds": "1", 2024-09-14T06:28:59.381 INFO:teuthology.orchestra.run.smithi094.stdout: "rbd_qos_write_iops_limit": "0", 2024-09-14T06:28:59.381 INFO:teuthology.orchestra.run.smithi094.stdout: "rbd_quiesce_notification_attempts": "10", 2024-09-14T06:28:59.381 INFO:teuthology.orchestra.run.smithi094.stdout: "rbd_read_from_replica_policy": "default", 2024-09-14T06:28:59.381 INFO:teuthology.orchestra.run.smithi094.stdout: "rbd_readahead_disable_after_bytes": "52428800", 2024-09-14T06:28:59.381 INFO:teuthology.orchestra.run.smithi094.stdout: "rbd_readahead_max_bytes": "524288", 2024-09-14T06:28:59.381 INFO:teuthology.orchestra.run.smithi094.stdout: "rbd_readahead_trigger_requests": "10", 2024-09-14T06:28:59.381 INFO:teuthology.orchestra.run.smithi094.stdout: "rbd_request_timed_out_seconds": "30", 2024-09-14T06:28:59.382 INFO:teuthology.orchestra.run.smithi094.stdout: "rbd_skip_partial_discard": "true", 2024-09-14T06:28:59.382 INFO:teuthology.orchestra.run.smithi094.stdout: "rbd_sparse_read_threshold_bytes": "65536", 2024-09-14T06:28:59.382 INFO:teuthology.orchestra.run.smithi094.stdout: "rbd_tracing": "false", 2024-09-14T06:28:59.382 INFO:teuthology.orchestra.run.smithi094.stdout: "rbd_validate_names": "true", 2024-09-14T06:28:59.382 INFO:teuthology.orchestra.run.smithi094.stdout: "rbd_validate_pool": "true", 2024-09-14T06:28:59.382 INFO:teuthology.orchestra.run.smithi094.stdout: "restapi_base_url": "", 2024-09-14T06:28:59.382 INFO:teuthology.orchestra.run.smithi094.stdout: "restapi_log_level": "", 2024-09-14T06:28:59.382 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_account_default_quota_max_objects": "-1", 2024-09-14T06:28:59.382 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_account_default_quota_max_size": "-1", 2024-09-14T06:28:59.382 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_acl_grants_max_num": "100", 2024-09-14T06:28:59.382 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_admin_entry": "admin", 2024-09-14T06:28:59.382 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_allow_notification_secrets_in_cleartext": "false", 2024-09-14T06:28:59.382 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_asio_assert_yielding": "false", 2024-09-14T06:28:59.382 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_backend_store": "rados", 2024-09-14T06:28:59.383 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_barbican_url": "", 2024-09-14T06:28:59.383 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_beast_enable_async": "true", 2024-09-14T06:28:59.383 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_bucket_counters_cache": "false", 2024-09-14T06:28:59.383 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_bucket_counters_cache_size": "10000", 2024-09-14T06:28:59.383 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_bucket_default_quota_max_objects": "-1", 2024-09-14T06:28:59.383 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_bucket_default_quota_max_size": "-1", 2024-09-14T06:28:59.383 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_bucket_index_max_aio": "128", 2024-09-14T06:28:59.383 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_bucket_index_transaction_instrumentation": "false", 2024-09-14T06:28:59.383 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_bucket_quota_cache_size": "10000", 2024-09-14T06:28:59.383 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_bucket_quota_ttl": "600", 2024-09-14T06:28:59.383 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_bucket_sync_spawn_window": "20", 2024-09-14T06:28:59.383 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_cache_enabled": "true", 2024-09-14T06:28:59.383 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_cache_expiry_interval": "900", 2024-09-14T06:28:59.383 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_cache_lru_size": "25000", 2024-09-14T06:28:59.384 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_config_store": "rados", 2024-09-14T06:28:59.384 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_content_length_compat": "false", 2024-09-14T06:28:59.384 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_copy_obj_progress": "true", 2024-09-14T06:28:59.384 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_copy_obj_progress_every_bytes": "1048576", 2024-09-14T06:28:59.384 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_cors_rules_max_num": "100", 2024-09-14T06:28:59.384 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_cross_domain_policy": "", 2024-09-14T06:28:59.384 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_crypt_default_encryption_key": "", 2024-09-14T06:28:59.384 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_crypt_kmip_addr": "", 2024-09-14T06:28:59.384 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_crypt_kmip_ca_path": "", 2024-09-14T06:28:59.384 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_crypt_kmip_client_cert": "", 2024-09-14T06:28:59.384 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_crypt_kmip_client_key": "", 2024-09-14T06:28:59.384 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_crypt_kmip_kms_key_template": "", 2024-09-14T06:28:59.384 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_crypt_kmip_password": "", 2024-09-14T06:28:59.384 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_crypt_kmip_s3_key_template": "$keyid", 2024-09-14T06:28:59.385 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_crypt_kmip_username": "", 2024-09-14T06:28:59.385 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_crypt_require_ssl": "true", 2024-09-14T06:28:59.385 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_crypt_s3_kms_backend": "barbican", 2024-09-14T06:28:59.385 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_crypt_s3_kms_encryption_keys": "", 2024-09-14T06:28:59.385 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_crypt_sse_s3_backend": "vault", 2024-09-14T06:28:59.385 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_crypt_sse_s3_key_template": "%bucket_id", 2024-09-14T06:28:59.385 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_crypt_sse_s3_vault_addr": "", 2024-09-14T06:28:59.385 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_crypt_sse_s3_vault_auth": "token", 2024-09-14T06:28:59.385 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_crypt_sse_s3_vault_namespace": "", 2024-09-14T06:28:59.385 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_crypt_sse_s3_vault_prefix": "", 2024-09-14T06:28:59.385 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_crypt_sse_s3_vault_secret_engine": "transit", 2024-09-14T06:28:59.385 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_crypt_sse_s3_vault_ssl_cacert": "", 2024-09-14T06:28:59.385 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_crypt_sse_s3_vault_ssl_clientcert": "", 2024-09-14T06:28:59.385 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_crypt_sse_s3_vault_ssl_clientkey": "", 2024-09-14T06:28:59.386 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_crypt_sse_s3_vault_token_file": "", 2024-09-14T06:28:59.386 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_crypt_sse_s3_vault_verify_ssl": "true", 2024-09-14T06:28:59.386 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_crypt_suppress_logs": "true", 2024-09-14T06:28:59.386 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_crypt_vault_addr": "", 2024-09-14T06:28:59.386 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_crypt_vault_auth": "token", 2024-09-14T06:28:59.386 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_crypt_vault_namespace": "", 2024-09-14T06:28:59.386 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_crypt_vault_prefix": "", 2024-09-14T06:28:59.386 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_crypt_vault_secret_engine": "transit", 2024-09-14T06:28:59.386 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_crypt_vault_ssl_cacert": "", 2024-09-14T06:28:59.386 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_crypt_vault_ssl_clientcert": "", 2024-09-14T06:28:59.386 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_crypt_vault_ssl_clientkey": "", 2024-09-14T06:28:59.386 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_crypt_vault_token_file": "", 2024-09-14T06:28:59.386 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_crypt_vault_verify_ssl": "true", 2024-09-14T06:28:59.386 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_curl_buffersize": "524288", 2024-09-14T06:28:59.386 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_curl_low_speed_limit": "1024", 2024-09-14T06:28:59.387 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_curl_low_speed_time": "300", 2024-09-14T06:28:59.387 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_curl_tcp_keepalive": "0", 2024-09-14T06:28:59.387 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_curl_wait_timeout_ms": "1000", 2024-09-14T06:28:59.387 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_d3n_l1_datacache_persistent_path": "/tmp/rgw_datacache/", 2024-09-14T06:28:59.387 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_d3n_l1_datacache_size": "1073741824", 2024-09-14T06:28:59.387 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_d3n_l1_evict_cache_on_start": "true", 2024-09-14T06:28:59.387 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_d3n_l1_eviction_policy": "lru", 2024-09-14T06:28:59.387 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_d3n_l1_fadvise": "4", 2024-09-14T06:28:59.387 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_d3n_l1_local_datacache_enabled": "false", 2024-09-14T06:28:59.387 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_d3n_libaio_aio_num": "64", 2024-09-14T06:28:59.387 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_d3n_libaio_aio_threads": "20", 2024-09-14T06:28:59.387 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_d4n_address": "127.0.0.1:6379", 2024-09-14T06:28:59.387 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_d4n_l1_datacache_address": "127.0.0.1:6379", 2024-09-14T06:28:59.387 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_d4n_l1_datacache_persistent_path": "/tmp/rgw_d4n_datacache/", 2024-09-14T06:28:59.388 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_d4n_l1_datacache_size": "1073741824", 2024-09-14T06:28:59.388 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_d4n_l1_evict_cache_on_start": "true", 2024-09-14T06:28:59.388 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_d4n_l1_fadvise": "4", 2024-09-14T06:28:59.388 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_d4n_libaio_aio_num": "64", 2024-09-14T06:28:59.388 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_d4n_libaio_aio_threads": "20", 2024-09-14T06:28:59.388 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_data": "/var/lib/ceph/radosgw/ceph-a", 2024-09-14T06:28:59.388 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_data_log_changes_size": "1000", 2024-09-14T06:28:59.388 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_data_log_num_shards": "128", 2024-09-14T06:28:59.388 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_data_log_obj_prefix": "data_log", 2024-09-14T06:28:59.388 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_data_log_window": "30", 2024-09-14T06:28:59.388 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_data_notify_interval_msec": "0", 2024-09-14T06:28:59.388 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_data_sync_poll_interval": "20", 2024-09-14T06:28:59.388 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_data_sync_spawn_window": "20", 2024-09-14T06:28:59.389 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_debug_inject_latency_bi_unlink": "0", 2024-09-14T06:28:59.389 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_debug_inject_olh_cancel_modification_err": "false", 2024-09-14T06:28:59.389 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_debug_inject_set_olh_err": "0", 2024-09-14T06:28:59.389 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_default_data_log_backing": "fifo", 2024-09-14T06:28:59.389 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_default_realm_info_oid": "default.realm", 2024-09-14T06:28:59.389 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_default_region_info_oid": "default.region", 2024-09-14T06:28:59.389 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_default_zone_info_oid": "default.zone", 2024-09-14T06:28:59.389 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_default_zonegroup_info_oid": "default.zonegroup", 2024-09-14T06:28:59.389 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_defer_to_bucket_acls": "", 2024-09-14T06:28:59.389 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_delete_multi_obj_max_num": "1000", 2024-09-14T06:28:59.389 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_disable_s3select": "false", 2024-09-14T06:28:59.389 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_dmclock_admin_lim": "0.000000", 2024-09-14T06:28:59.392 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_dmclock_admin_res": "100.000000", 2024-09-14T06:28:59.392 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_dmclock_admin_wgt": "100.000000", 2024-09-14T06:28:59.392 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_dmclock_auth_lim": "0.000000", 2024-09-14T06:28:59.393 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_dmclock_auth_res": "200.000000", 2024-09-14T06:28:59.393 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_dmclock_auth_wgt": "100.000000", 2024-09-14T06:28:59.393 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_dmclock_data_lim": "0.000000", 2024-09-14T06:28:59.393 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_dmclock_data_res": "500.000000", 2024-09-14T06:28:59.393 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_dmclock_data_wgt": "500.000000", 2024-09-14T06:28:59.393 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_dmclock_metadata_lim": "0.000000", 2024-09-14T06:28:59.393 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_dmclock_metadata_res": "500.000000", 2024-09-14T06:28:59.393 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_dmclock_metadata_wgt": "500.000000", 2024-09-14T06:28:59.393 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_dns_name": "", 2024-09-14T06:28:59.393 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_dns_s3website_name": "", 2024-09-14T06:28:59.393 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_dynamic_resharding": "true", 2024-09-14T06:28:59.393 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_dynamic_resharding_may_reduce": "true", 2024-09-14T06:28:59.393 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_dynamic_resharding_reduction_wait": "120", 2024-09-14T06:28:59.393 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_enable_apis": "s3, s3website, swift, swift_auth, admin, sts, iam, notifications", 2024-09-14T06:28:59.394 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_enable_gc_threads": "true", 2024-09-14T06:28:59.394 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_enable_lc_threads": "true", 2024-09-14T06:28:59.394 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_enable_mdsearch": "true", 2024-09-14T06:28:59.394 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_enable_ops_log": "false", 2024-09-14T06:28:59.394 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_enable_quota_threads": "true", 2024-09-14T06:28:59.394 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_enable_static_website": "false", 2024-09-14T06:28:59.394 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_enable_usage_log": "false", 2024-09-14T06:28:59.394 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_enforce_swift_acls": "true", 2024-09-14T06:28:59.394 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_exit_timeout_secs": "120", 2024-09-14T06:28:59.394 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_expose_bucket": "false", 2024-09-14T06:28:59.394 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_extended_http_attrs": "", 2024-09-14T06:28:59.394 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_filter": "none", 2024-09-14T06:28:59.394 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_frontend_defaults": "beast ssl_certificate=config://rgw/cert/$realm/$zone.crt ssl_private_key=config://rgw/cert/$realm/$zone.key", 2024-09-14T06:28:59.394 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_frontends": "beast port=7480", 2024-09-14T06:28:59.395 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_gc_max_concurrent_io": "10", 2024-09-14T06:28:59.395 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_gc_max_deferred": "50", 2024-09-14T06:28:59.395 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_gc_max_deferred_entries_size": "3072", 2024-09-14T06:28:59.395 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_gc_max_objs": "32", 2024-09-14T06:28:59.395 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_gc_max_queue_size": "134213632", 2024-09-14T06:28:59.395 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_gc_max_trim_chunk": "16", 2024-09-14T06:28:59.395 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_gc_obj_min_wait": "7200", 2024-09-14T06:28:59.395 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_gc_processor_max_time": "3600", 2024-09-14T06:28:59.395 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_gc_processor_period": "3600", 2024-09-14T06:28:59.395 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_get_obj_max_req_size": "4194304", 2024-09-14T06:28:59.395 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_get_obj_window_size": "16777216", 2024-09-14T06:28:59.395 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_healthcheck_disabling_path": "", 2024-09-14T06:28:59.395 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_ignore_get_invalid_range": "false", 2024-09-14T06:28:59.395 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_init_timeout": "300", 2024-09-14T06:28:59.396 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_inject_notify_timeout_probability": "0.000000", 2024-09-14T06:28:59.396 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_json_config": "/var/lib/ceph/radosgw/config.json", 2024-09-14T06:28:59.396 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_kafka_connection_idle": "300", 2024-09-14T06:28:59.396 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_kafka_message_timeout": "5000", 2024-09-14T06:28:59.396 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_kafka_sleep_timeout": "10", 2024-09-14T06:28:59.396 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_keystone_accepted_admin_roles": "", 2024-09-14T06:28:59.396 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_keystone_accepted_reader_roles": "", 2024-09-14T06:28:59.396 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_keystone_accepted_roles": "Member, admin", 2024-09-14T06:28:59.396 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_keystone_admin_domain": "", 2024-09-14T06:28:59.396 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_keystone_admin_password": "", 2024-09-14T06:28:59.396 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_keystone_admin_password_path": "", 2024-09-14T06:28:59.396 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_keystone_admin_project": "", 2024-09-14T06:28:59.396 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_keystone_admin_tenant": "", 2024-09-14T06:28:59.396 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_keystone_admin_token": "", 2024-09-14T06:28:59.397 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_keystone_admin_token_path": "", 2024-09-14T06:28:59.397 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_keystone_admin_user": "", 2024-09-14T06:28:59.397 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_keystone_barbican_domain": "", 2024-09-14T06:28:59.397 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_keystone_barbican_password": "", 2024-09-14T06:28:59.397 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_keystone_barbican_project": "", 2024-09-14T06:28:59.397 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_keystone_barbican_tenant": "", 2024-09-14T06:28:59.397 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_keystone_barbican_user": "", 2024-09-14T06:28:59.397 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_keystone_expired_token_cache_expiration": "3600", 2024-09-14T06:28:59.397 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_keystone_implicit_tenants": "false", 2024-09-14T06:28:59.397 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_keystone_service_token_accepted_roles": "admin", 2024-09-14T06:28:59.397 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_keystone_service_token_enabled": "false", 2024-09-14T06:28:59.397 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_keystone_token_cache_size": "10000", 2024-09-14T06:28:59.397 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_keystone_url": "", 2024-09-14T06:28:59.398 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_keystone_verify_ssl": "true", 2024-09-14T06:28:59.398 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_lc_debug_interval": "-1", 2024-09-14T06:28:59.398 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_lc_lock_max_time": "90", 2024-09-14T06:28:59.398 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_lc_max_objs": "32", 2024-09-14T06:28:59.398 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_lc_max_rules": "1000", 2024-09-14T06:28:59.398 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_lc_max_worker": "3", 2024-09-14T06:28:59.398 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_lc_max_wp_worker": "3", 2024-09-14T06:28:59.398 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_lc_thread_delay": "0", 2024-09-14T06:28:59.398 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_ldap_binddn": "uid=admin,cn=users,dc=example,dc=com", 2024-09-14T06:28:59.398 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_ldap_dnattr": "uid", 2024-09-14T06:28:59.398 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_ldap_searchdn": "cn=users,cn=accounts,dc=example,dc=com", 2024-09-14T06:28:59.398 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_ldap_searchfilter": "", 2024-09-14T06:28:59.398 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_ldap_secret": "/etc/openldap/secret", 2024-09-14T06:28:59.398 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_ldap_uri": "", 2024-09-14T06:28:59.399 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_lfuda_sync_frequency": "60", 2024-09-14T06:28:59.399 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_lifecycle_work_time": "00:00-06:00", 2024-09-14T06:28:59.399 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_list_bucket_min_readahead": "1000", 2024-09-14T06:28:59.399 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_list_buckets_max_chunk": "1000", 2024-09-14T06:28:59.399 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_log_http_headers": "", 2024-09-14T06:28:59.399 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_log_nonexistent_bucket": "false", 2024-09-14T06:28:59.399 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_log_object_name": "%Y-%m-%d-%H-%i-%n", 2024-09-14T06:28:59.399 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_log_object_name_utc": "false", 2024-09-14T06:28:59.399 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_lua_max_memory_per_state": "128000", 2024-09-14T06:28:59.399 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_luarocks_location": "/tmp/rgw_luarocks/mon.a", 2024-09-14T06:28:59.399 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_max_attr_name_len": "0", 2024-09-14T06:28:59.399 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_max_attr_size": "0", 2024-09-14T06:28:59.399 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_max_attrs_num_in_req": "0", 2024-09-14T06:28:59.400 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_max_chunk_size": "4194304", 2024-09-14T06:28:59.400 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_max_concurrent_requests": "1024", 2024-09-14T06:28:59.400 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_max_copy_obj_concurrent_io": "10", 2024-09-14T06:28:59.400 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_max_dynamic_shards": "1999", 2024-09-14T06:28:59.400 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_max_listing_results": "1000", 2024-09-14T06:28:59.400 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_max_notify_retries": "10", 2024-09-14T06:28:59.400 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_max_objs_per_shard": "100000", 2024-09-14T06:28:59.400 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_max_put_param_size": "1048576", 2024-09-14T06:28:59.400 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_max_put_size": "5368709120", 2024-09-14T06:28:59.400 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_max_slo_entries": "1000", 2024-09-14T06:28:59.400 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_md_log_max_shards": "64", 2024-09-14T06:28:59.400 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_md_notify_interval_msec": "200", 2024-09-14T06:28:59.400 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_meta_sync_poll_interval": "20", 2024-09-14T06:28:59.400 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_meta_sync_spawn_window": "20", 2024-09-14T06:28:59.401 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_mime_types_file": "/etc/mime.types", 2024-09-14T06:28:59.401 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_mp_lock_max_time": "600", 2024-09-14T06:28:59.401 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_multi_obj_del_max_aio": "16", 2024-09-14T06:28:59.401 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_multipart_min_part_size": "5242880", 2024-09-14T06:28:59.401 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_multipart_part_upload_limit": "10000", 2024-09-14T06:28:59.401 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_nfs_fhcache_partitions": "3", 2024-09-14T06:28:59.401 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_nfs_fhcache_size": "2017", 2024-09-14T06:28:59.401 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_nfs_frontends": "rgw-nfs", 2024-09-14T06:28:59.401 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_nfs_lru_lane_hiwat": "911", 2024-09-14T06:28:59.401 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_nfs_lru_lanes": "5", 2024-09-14T06:28:59.401 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_nfs_max_gc": "300", 2024-09-14T06:28:59.401 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_nfs_namespace_expire_secs": "300", 2024-09-14T06:28:59.401 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_nfs_run_gc_threads": "false", 2024-09-14T06:28:59.402 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_nfs_run_lc_threads": "false", 2024-09-14T06:28:59.402 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_nfs_run_quota_threads": "false", 2024-09-14T06:28:59.402 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_nfs_run_sync_thread": "false", 2024-09-14T06:28:59.402 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_nfs_s3_fast_attrs": "false", 2024-09-14T06:28:59.402 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_nfs_write_completion_interval_s": "10", 2024-09-14T06:28:59.402 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_num_async_rados_threads": "32", 2024-09-14T06:28:59.402 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_num_control_oids": "8", 2024-09-14T06:28:59.402 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_numa_node": "-1", 2024-09-14T06:28:59.402 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_obj_stripe_size": "4194304", 2024-09-14T06:28:59.402 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_obj_tombstone_cache_size": "1000", 2024-09-14T06:28:59.402 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_objexp_chunk_size": "100", 2024-09-14T06:28:59.402 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_objexp_gc_interval": "600", 2024-09-14T06:28:59.402 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_objexp_hints_num_shards": "127", 2024-09-14T06:28:59.402 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_olh_pending_timeout_sec": "3600", 2024-09-14T06:28:59.403 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_op_thread_suicide_timeout": "0", 2024-09-14T06:28:59.403 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_op_thread_timeout": "600", 2024-09-14T06:28:59.403 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_op_tracing": "false", 2024-09-14T06:28:59.403 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_opa_token": "", 2024-09-14T06:28:59.403 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_opa_url": "", 2024-09-14T06:28:59.403 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_opa_verify_ssl": "true", 2024-09-14T06:28:59.403 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_ops_log_data_backlog": "5242880", 2024-09-14T06:28:59.403 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_ops_log_file_path": "/var/log/ceph/ops-log-ceph-mon.a.log", 2024-09-14T06:28:59.403 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_ops_log_rados": "false", 2024-09-14T06:28:59.403 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_ops_log_socket_path": "", 2024-09-14T06:28:59.403 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_override_bucket_index_max_shards": "0", 2024-09-14T06:28:59.403 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_pending_bucket_index_op_expiration": "120", 2024-09-14T06:28:59.403 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_period_latest_epoch_info_oid": ".latest_epoch", 2024-09-14T06:28:59.403 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_period_push_interval": "2.000000", 2024-09-14T06:28:59.404 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_period_push_interval_max": "30.000000", 2024-09-14T06:28:59.404 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_period_root_pool": ".rgw.root", 2024-09-14T06:28:59.404 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_policy_reject_invalid_principals": "true", 2024-09-14T06:28:59.404 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_posix_base_path": "/tmp/rgw_posix_driver", 2024-09-14T06:28:59.404 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_posix_cache_lanes": "3", 2024-09-14T06:28:59.404 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_posix_cache_lmdb_count": "3", 2024-09-14T06:28:59.404 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_posix_cache_max_buckets": "100", 2024-09-14T06:28:59.404 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_posix_cache_partitions": "3", 2024-09-14T06:28:59.404 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_posix_database_root": "/var/lib/ceph/radosgw", 2024-09-14T06:28:59.404 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_print_continue": "true", 2024-09-14T06:28:59.404 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_print_prohibited_content_length": "false", 2024-09-14T06:28:59.404 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_put_obj_max_window_size": "67108864", 2024-09-14T06:28:59.404 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_put_obj_min_window_size": "16777216", 2024-09-14T06:28:59.404 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_rados_pool_autoscale_bias": "4.000000", 2024-09-14T06:28:59.405 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_rados_pool_recovery_priority": "5", 2024-09-14T06:28:59.405 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_rados_tracing": "false", 2024-09-14T06:28:59.405 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_realm": "", 2024-09-14T06:28:59.405 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_realm_id": "", 2024-09-14T06:28:59.405 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_realm_root_pool": ".rgw.root", 2024-09-14T06:28:59.405 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_region": "", 2024-09-14T06:28:59.405 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_region_root_pool": ".rgw.root", 2024-09-14T06:28:59.405 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_relaxed_region_enforcement": "false", 2024-09-14T06:28:59.405 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_relaxed_s3_bucket_names": "false", 2024-09-14T06:28:59.405 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_remote_addr_param": "REMOTE_ADDR", 2024-09-14T06:28:59.405 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_request_uri": "", 2024-09-14T06:28:59.405 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_reshard_batch_size": "64", 2024-09-14T06:28:59.405 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_reshard_bucket_lock_duration": "360", 2024-09-14T06:28:59.406 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_reshard_debug_interval": "-1", 2024-09-14T06:28:59.406 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_reshard_max_aio": "128", 2024-09-14T06:28:59.406 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_reshard_num_logs": "16", 2024-09-14T06:28:59.406 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_reshard_progress_judge_interval": "120", 2024-09-14T06:28:59.406 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_reshard_progress_judge_ratio": "0.500000", 2024-09-14T06:28:59.406 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_reshard_thread_interval": "600", 2024-09-14T06:28:59.406 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_reshardlog_threshold": "30000", 2024-09-14T06:28:59.406 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_resolve_cname": "false", 2024-09-14T06:28:59.406 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_rest_getusage_op_compat": "false", 2024-09-14T06:28:59.406 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_run_sync_thread": "true", 2024-09-14T06:28:59.406 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_s3_auth_disable_signature_url": "false", 2024-09-14T06:28:59.406 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_s3_auth_order": "sts, external, local", 2024-09-14T06:28:59.406 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_s3_auth_use_keystone": "false", 2024-09-14T06:28:59.406 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_s3_auth_use_ldap": "false", 2024-09-14T06:28:59.407 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_s3_auth_use_rados": "true", 2024-09-14T06:28:59.407 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_s3_auth_use_sts": "false", 2024-09-14T06:28:59.407 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_s3_client_max_sig_ver": "-1", 2024-09-14T06:28:59.407 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_s3_success_create_obj_status": "0", 2024-09-14T06:28:59.407 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_safe_max_objects_per_shard": "102400", 2024-09-14T06:28:59.407 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_scheduler_type": "throttler", 2024-09-14T06:28:59.407 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_script_uri": "", 2024-09-14T06:28:59.407 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_service_provider_name": "", 2024-09-14T06:28:59.407 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_shard_warning_threshold": "90.000000", 2024-09-14T06:28:59.407 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_sts_client_id": "", 2024-09-14T06:28:59.407 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_sts_client_secret": "", 2024-09-14T06:28:59.407 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_sts_entry": "sts", 2024-09-14T06:28:59.407 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_sts_key": "sts", 2024-09-14T06:28:59.407 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_sts_max_session_duration": "43200", 2024-09-14T06:28:59.407 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_sts_min_session_duration": "900", 2024-09-14T06:28:59.408 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_sts_token_introspection_url": "", 2024-09-14T06:28:59.408 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_swift_account_in_url": "false", 2024-09-14T06:28:59.408 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_swift_auth_entry": "auth", 2024-09-14T06:28:59.408 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_swift_auth_url": "", 2024-09-14T06:28:59.408 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_swift_custom_header": "", 2024-09-14T06:28:59.408 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_swift_enforce_content_length": "false", 2024-09-14T06:28:59.408 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_swift_need_stats": "true", 2024-09-14T06:28:59.408 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_swift_tenant_name": "", 2024-09-14T06:28:59.408 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_swift_token_expiration": "86400", 2024-09-14T06:28:59.408 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_swift_url": "", 2024-09-14T06:28:59.408 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_swift_url_prefix": "swift", 2024-09-14T06:28:59.408 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_swift_versioning_enabled": "false", 2024-09-14T06:28:59.408 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_sync_data_full_inject_err_probability": "0.000000", 2024-09-14T06:28:59.409 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_sync_data_inject_err_probability": "0.000000", 2024-09-14T06:28:59.409 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_sync_lease_period": "120", 2024-09-14T06:28:59.409 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_sync_log_trim_concurrent_buckets": "4", 2024-09-14T06:28:59.409 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_sync_log_trim_interval": "1200", 2024-09-14T06:28:59.409 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_sync_log_trim_max_buckets": "16", 2024-09-14T06:28:59.409 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_sync_log_trim_min_cold_buckets": "4", 2024-09-14T06:28:59.409 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_sync_meta_inject_err_probability": "0.000000", 2024-09-14T06:28:59.409 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_sync_obj_etag_verify": "false", 2024-09-14T06:28:59.409 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_sync_trace_history_size": "4096", 2024-09-14T06:28:59.409 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_sync_trace_per_node_log_size": "32", 2024-09-14T06:28:59.409 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_sync_trace_servicemap_update_interval": "10", 2024-09-14T06:28:59.409 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_thread_pool_size": "512", 2024-09-14T06:28:59.409 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_topic_persistency_max_retries": "0", 2024-09-14T06:28:59.409 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_topic_persistency_sleep_duration": "0", 2024-09-14T06:28:59.410 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_topic_persistency_time_to_live": "0", 2024-09-14T06:28:59.410 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_topic_require_publish_policy": "false", 2024-09-14T06:28:59.410 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_torrent_comment": "", 2024-09-14T06:28:59.410 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_torrent_createby": "", 2024-09-14T06:28:59.410 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_torrent_encoding": "", 2024-09-14T06:28:59.410 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_torrent_flag": "false", 2024-09-14T06:28:59.410 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_torrent_max_size": "5368709120", 2024-09-14T06:28:59.410 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_torrent_origin": "", 2024-09-14T06:28:59.410 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_torrent_sha_unit": "524288", 2024-09-14T06:28:59.410 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_torrent_tracker": "", 2024-09-14T06:28:59.410 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_trust_forwarded_https": "false", 2024-09-14T06:28:59.410 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_usage_log_flush_threshold": "1024", 2024-09-14T06:28:59.410 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_usage_log_tick_interval": "30", 2024-09-14T06:28:59.410 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_usage_max_shards": "32", 2024-09-14T06:28:59.410 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_usage_max_user_shards": "1", 2024-09-14T06:28:59.411 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_use_opa_authz": "false", 2024-09-14T06:28:59.411 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_user_counters_cache": "false", 2024-09-14T06:28:59.411 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_user_counters_cache_size": "10000", 2024-09-14T06:28:59.411 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_user_default_quota_max_objects": "-1", 2024-09-14T06:28:59.411 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_user_default_quota_max_size": "-1", 2024-09-14T06:28:59.411 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_user_max_buckets": "1000", 2024-09-14T06:28:59.411 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_user_policies_max_num": "100", 2024-09-14T06:28:59.411 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_user_quota_bucket_sync_interval": "180", 2024-09-14T06:28:59.411 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_user_quota_sync_idle_users": "false", 2024-09-14T06:28:59.411 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_user_quota_sync_interval": "86400", 2024-09-14T06:28:59.411 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_user_quota_sync_wait_time": "86400", 2024-09-14T06:28:59.411 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_user_unique_email": "true", 2024-09-14T06:28:59.411 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_verify_ssl": "true", 2024-09-14T06:28:59.411 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_website_routing_rules_max_num": "50", 2024-09-14T06:28:59.412 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_zone": "", 2024-09-14T06:28:59.412 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_zone_id": "", 2024-09-14T06:28:59.412 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_zone_root_pool": ".rgw.root", 2024-09-14T06:28:59.412 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_zonegroup": "", 2024-09-14T06:28:59.412 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_zonegroup_id": "", 2024-09-14T06:28:59.412 INFO:teuthology.orchestra.run.smithi094.stdout: "rgw_zonegroup_root_pool": ".rgw.root", 2024-09-14T06:28:59.412 INFO:teuthology.orchestra.run.smithi094.stdout: "rgwlc_auto_session_clear": "true", 2024-09-14T06:28:59.412 INFO:teuthology.orchestra.run.smithi094.stdout: "rgwlc_skip_bucket_step": "false", 2024-09-14T06:28:59.412 INFO:teuthology.orchestra.run.smithi094.stdout: "rocksdb_block_size": "4096", 2024-09-14T06:28:59.412 INFO:teuthology.orchestra.run.smithi094.stdout: "rocksdb_bloom_bits_per_key": "20", 2024-09-14T06:28:59.412 INFO:teuthology.orchestra.run.smithi094.stdout: "rocksdb_cache_index_and_filter_blocks": "true", 2024-09-14T06:28:59.412 INFO:teuthology.orchestra.run.smithi094.stdout: "rocksdb_cache_index_and_filter_blocks_with_high_priority": "false", 2024-09-14T06:28:59.412 INFO:teuthology.orchestra.run.smithi094.stdout: "rocksdb_cache_row_ratio": "0.000000", 2024-09-14T06:28:59.412 INFO:teuthology.orchestra.run.smithi094.stdout: "rocksdb_cache_shard_bits": "4", 2024-09-14T06:28:59.413 INFO:teuthology.orchestra.run.smithi094.stdout: "rocksdb_cache_size": "536870912", 2024-09-14T06:28:59.413 INFO:teuthology.orchestra.run.smithi094.stdout: "rocksdb_cache_type": "binned_lru", 2024-09-14T06:28:59.413 INFO:teuthology.orchestra.run.smithi094.stdout: "rocksdb_cf_compact_on_deletion": "true", 2024-09-14T06:28:59.413 INFO:teuthology.orchestra.run.smithi094.stdout: "rocksdb_cf_compact_on_deletion_sliding_window": "32768", 2024-09-14T06:28:59.413 INFO:teuthology.orchestra.run.smithi094.stdout: "rocksdb_cf_compact_on_deletion_trigger": "16384", 2024-09-14T06:28:59.413 INFO:teuthology.orchestra.run.smithi094.stdout: "rocksdb_collect_compaction_stats": "false", 2024-09-14T06:28:59.413 INFO:teuthology.orchestra.run.smithi094.stdout: "rocksdb_collect_extended_stats": "false", 2024-09-14T06:28:59.413 INFO:teuthology.orchestra.run.smithi094.stdout: "rocksdb_collect_memory_stats": "false", 2024-09-14T06:28:59.413 INFO:teuthology.orchestra.run.smithi094.stdout: "rocksdb_delete_range_threshold": "1048576", 2024-09-14T06:28:59.413 INFO:teuthology.orchestra.run.smithi094.stdout: "rocksdb_index_type": "binary_search", 2024-09-14T06:28:59.413 INFO:teuthology.orchestra.run.smithi094.stdout: "rocksdb_log_to_ceph_log": "true", 2024-09-14T06:28:59.413 INFO:teuthology.orchestra.run.smithi094.stdout: "rocksdb_metadata_block_size": "4096", 2024-09-14T06:28:59.413 INFO:teuthology.orchestra.run.smithi094.stdout: "rocksdb_partition_filters": "false", 2024-09-14T06:28:59.413 INFO:teuthology.orchestra.run.smithi094.stdout: "rocksdb_perf": "false", 2024-09-14T06:28:59.414 INFO:teuthology.orchestra.run.smithi094.stdout: "rocksdb_pin_l0_filter_and_index_blocks_in_cache": "false", 2024-09-14T06:28:59.414 INFO:teuthology.orchestra.run.smithi094.stdout: "rotating_keys_bootstrap_timeout": "30", 2024-09-14T06:28:59.414 INFO:teuthology.orchestra.run.smithi094.stdout: "rotating_keys_renewal_timeout": "10", 2024-09-14T06:28:59.414 INFO:teuthology.orchestra.run.smithi094.stdout: "run_dir": "/var/run/ceph", 2024-09-14T06:28:59.414 INFO:teuthology.orchestra.run.smithi094.stdout: "seastore_block_create": "true", 2024-09-14T06:28:59.414 INFO:teuthology.orchestra.run.smithi094.stdout: "seastore_cache_lru_size": "67108864", 2024-09-14T06:28:59.414 INFO:teuthology.orchestra.run.smithi094.stdout: "seastore_cbjournal_size": "5368709120", 2024-09-14T06:28:59.414 INFO:teuthology.orchestra.run.smithi094.stdout: "seastore_data_delta_based_overwrite": "0", 2024-09-14T06:28:59.414 INFO:teuthology.orchestra.run.smithi094.stdout: "seastore_default_max_object_size": "16777216", 2024-09-14T06:28:59.414 INFO:teuthology.orchestra.run.smithi094.stdout: "seastore_default_object_metadata_reservation": "16777216", 2024-09-14T06:28:59.414 INFO:teuthology.orchestra.run.smithi094.stdout: "seastore_device_size": "53687091200", 2024-09-14T06:28:59.414 INFO:teuthology.orchestra.run.smithi094.stdout: "seastore_disable_end_to_end_data_protection": "true", 2024-09-14T06:28:59.414 INFO:teuthology.orchestra.run.smithi094.stdout: "seastore_full_integrity_check": "false", 2024-09-14T06:28:59.414 INFO:teuthology.orchestra.run.smithi094.stdout: "seastore_journal_batch_capacity": "16", 2024-09-14T06:28:59.414 INFO:teuthology.orchestra.run.smithi094.stdout: "seastore_journal_batch_flush_size": "16777216", 2024-09-14T06:28:59.415 INFO:teuthology.orchestra.run.smithi094.stdout: "seastore_journal_batch_preferred_fullness": "0.950000", 2024-09-14T06:28:59.415 INFO:teuthology.orchestra.run.smithi094.stdout: "seastore_journal_iodepth_limit": "5", 2024-09-14T06:28:59.415 INFO:teuthology.orchestra.run.smithi094.stdout: "seastore_main_device_type": "SSD", 2024-09-14T06:28:59.415 INFO:teuthology.orchestra.run.smithi094.stdout: "seastore_max_concurrent_transactions": "8", 2024-09-14T06:28:59.415 INFO:teuthology.orchestra.run.smithi094.stdout: "seastore_max_data_allocation_size": "32768", 2024-09-14T06:28:59.415 INFO:teuthology.orchestra.run.smithi094.stdout: "seastore_multiple_tiers_default_evict_ratio": "0.600000", 2024-09-14T06:28:59.415 INFO:teuthology.orchestra.run.smithi094.stdout: "seastore_multiple_tiers_fast_evict_ratio": "0.700000", 2024-09-14T06:28:59.415 INFO:teuthology.orchestra.run.smithi094.stdout: "seastore_multiple_tiers_stop_evict_ratio": "0.500000", 2024-09-14T06:28:59.415 INFO:teuthology.orchestra.run.smithi094.stdout: "seastore_obj_data_write_amplification": "1.250000", 2024-09-14T06:28:59.415 INFO:teuthology.orchestra.run.smithi094.stdout: "seastore_segment_size": "67108864", 2024-09-14T06:28:59.415 INFO:teuthology.orchestra.run.smithi094.stdout: "set_keepcaps": "false", 2024-09-14T06:28:59.415 INFO:teuthology.orchestra.run.smithi094.stdout: "setgroup": "", 2024-09-14T06:28:59.415 INFO:teuthology.orchestra.run.smithi094.stdout: "setuser": "", 2024-09-14T06:28:59.415 INFO:teuthology.orchestra.run.smithi094.stdout: "setuser_match_path": "", 2024-09-14T06:28:59.416 INFO:teuthology.orchestra.run.smithi094.stdout: "target_max_misplaced_ratio": "0.050000", 2024-09-14T06:28:59.416 INFO:teuthology.orchestra.run.smithi094.stdout: "thp": "false", 2024-09-14T06:28:59.416 INFO:teuthology.orchestra.run.smithi094.stdout: "threadpool_default_timeout": "60", 2024-09-14T06:28:59.416 INFO:teuthology.orchestra.run.smithi094.stdout: "threadpool_empty_queue_max_wait": "2", 2024-09-14T06:28:59.416 INFO:teuthology.orchestra.run.smithi094.stdout: "throttler_perf_counter": "true", 2024-09-14T06:28:59.416 INFO:teuthology.orchestra.run.smithi094.stdout: "tmp_dir": "/tmp", 2024-09-14T06:28:59.416 INFO:teuthology.orchestra.run.smithi094.stdout: "tmp_file_template": "/tmp/ceph-mon.a.XXXXXX", 2024-09-14T06:28:59.416 INFO:teuthology.orchestra.run.smithi094.stdout: "uadk_compressor_enabled": "false", 2024-09-14T06:28:59.416 INFO:teuthology.orchestra.run.smithi094.stdout: "uadk_wd_sync_ctx_num": "2" 2024-09-14T06:28:59.416 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-14T06:28:59.418 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'mon.*' injectargs --mon_osd_down_out_interval=0 2024-09-14T06:28:59.530 INFO:teuthology.orchestra.run.smithi094.stdout:mon.a: {} 2024-09-14T06:28:59.531 INFO:teuthology.orchestra.run.smithi094.stderr:mon.a: mon_osd_down_out_interval = '' (not observed, change may require restart) 2024-09-14T06:28:59.540 INFO:teuthology.orchestra.run.smithi094.stdout:mon.b: {} 2024-09-14T06:28:59.541 INFO:teuthology.orchestra.run.smithi094.stderr:mon.b: mon_osd_down_out_interval = '' (not observed, change may require restart) 2024-09-14T06:28:59.549 INFO:teuthology.orchestra.run.smithi094.stdout:mon.c: {} 2024-09-14T06:28:59.549 INFO:teuthology.orchestra.run.smithi094.stderr:mon.c: mon_osd_down_out_interval = '' (not observed, change may require restart) 2024-09-14T06:28:59.559 DEBUG:teuthology.orchestra.run.smithi138:> type ceph-objectstore-tool 2024-09-14T06:28:59.577 INFO:teuthology.orchestra.run.smithi138.stdout:ceph-objectstore-tool is /usr/bin/ceph-objectstore-tool 2024-09-14T06:28:59.577 DEBUG:teuthology.orchestra.run.smithi094:> type ceph-objectstore-tool 2024-09-14T06:28:59.592 INFO:teuthology.orchestra.run.smithi094.stdout:ceph-objectstore-tool is /usr/bin/ceph-objectstore-tool 2024-09-14T06:28:59.592 INFO:teuthology.run_tasks:Running task rbd_fsx... 2024-09-14T06:28:59.602 INFO:tasks.rbd_fsx:starting rbd_fsx... 2024-09-14T06:28:59.602 INFO:tasks.thrashosds.thrasher:starting do_thrash 2024-09-14T06:28:59.602 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-14T06:28:59.603 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-14T06:28:59.603 INFO:tasks.thrashosds.thrasher:thrash_pg_upmap 2024-09-14T06:28:59.603 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump -f json-pretty 2024-09-14T06:28:59.603 INFO:tasks.thrashosds.thrasher:starting do_sighup with a delay of 0.1 2024-09-14T06:28:59.603 INFO:tasks.thrashosds.thrasher:starting do_optrack_toggle with a delay of 2.0 2024-09-14T06:28:59.603 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-14T06:28:59.604 INFO:tasks.thrashosds.thrasher:starting do_dump_ops 2024-09-14T06:28:59.604 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-14T06:28:59.604 INFO:tasks.thrashosds.thrasher:starting do_noscrub_toggle with a delay of 2.0 2024-09-14T06:28:59.604 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-14T06:28:59.605 DEBUG:teuthology.orchestra.run.smithi094:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph_test_librbd_fsx --cluster ceph --id 0 -d -W -R -p 100 -P /home/ubuntu/cephtest/archive -r 1 -w 1 -t 1 -h 1 -l 250000000 -S 0 -N 6000 -g pool_client.0 image_client.0 2024-09-14T06:28:59.619 INFO:tasks.ceph.osd.7.smithi138.stderr:2024-09-14T06:28:59.618+0000 7f3c6a21d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33623) UID: 0 2024-09-14T06:28:59.671 INFO:teuthology.orchestra.run.smithi094.stdout:mapped writes DISABLED 2024-09-14T06:28:59.671 INFO:teuthology.orchestra.run.smithi094.stdout:mapped reads DISABLED 2024-09-14T06:28:59.671 INFO:teuthology.orchestra.run.smithi094.stdout:Seed set to 5394 2024-09-14T06:28:59.712 INFO:tasks.ceph.osd.1.smithi094.stderr:2024-09-14T06:28:59.711+0000 7f328e47a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33641) UID: 0 2024-09-14T06:28:59.744 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-14T06:28:59.745 INFO:teuthology.orchestra.run.smithi094.stdout: "ops": [], 2024-09-14T06:28:59.745 INFO:teuthology.orchestra.run.smithi094.stdout: "num_ops": 0 2024-09-14T06:28:59.745 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-14T06:28:59.755 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-14T06:28:59.796 INFO:teuthology.orchestra.run.smithi094.stdout:osd.0: {} 2024-09-14T06:28:59.796 INFO:teuthology.orchestra.run.smithi094.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-14T06:28:59.807 INFO:tasks.ceph.osd.5.smithi138.stderr:2024-09-14T06:28:59.806+0000 7fe13746f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33618) UID: 0 2024-09-14T06:28:59.814 INFO:teuthology.orchestra.run.smithi094.stdout:osd.1: {} 2024-09-14T06:28:59.815 INFO:teuthology.orchestra.run.smithi094.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-14T06:28:59.832 INFO:teuthology.orchestra.run.smithi094.stdout:osd.2: {} 2024-09-14T06:28:59.832 INFO:teuthology.orchestra.run.smithi094.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-14T06:28:59.863 INFO:teuthology.orchestra.run.smithi094.stdout:osd.3: {} 2024-09-14T06:28:59.864 INFO:teuthology.orchestra.run.smithi094.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-14T06:28:59.886 INFO:teuthology.orchestra.run.smithi094.stdout:osd.4: {} 2024-09-14T06:28:59.887 INFO:teuthology.orchestra.run.smithi094.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-14T06:28:59.904 INFO:teuthology.orchestra.run.smithi094.stdout:osd.5: {} 2024-09-14T06:28:59.904 INFO:teuthology.orchestra.run.smithi094.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-14T06:28:59.908 INFO:tasks.ceph.osd.5.smithi138.stderr:2024-09-14T06:28:59.907+0000 7fe13746f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33618) UID: 0 2024-09-14T06:28:59.916 INFO:teuthology.orchestra.run.smithi094.stdout:ERROR: (22) Invalid argument 2024-09-14T06:28:59.917 INFO:teuthology.orchestra.run.smithi094.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:28:59.921 INFO:teuthology.orchestra.run.smithi094.stdout:osd.6: {} 2024-09-14T06:28:59.921 INFO:teuthology.orchestra.run.smithi094.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-14T06:28:59.926 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-14T06:28:59.938 INFO:teuthology.orchestra.run.smithi094.stdout:osd.7: {} 2024-09-14T06:28:59.939 INFO:teuthology.orchestra.run.smithi094.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-14T06:28:59.948 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-14T06:28:59.948 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-14T06:28:59.948 INFO:teuthology.orchestra.run.smithi094.stdout: "epoch": 22, 2024-09-14T06:28:59.948 INFO:teuthology.orchestra.run.smithi094.stdout: "fsid": "200ad3da-bd63-4e04-af74-efb5ca0af75e", 2024-09-14T06:28:59.948 INFO:teuthology.orchestra.run.smithi094.stdout: "created": "2024-09-14T06:28:30.782046+0000", 2024-09-14T06:28:59.948 INFO:teuthology.orchestra.run.smithi094.stdout: "modified": "2024-09-14T06:28:53.222080+0000", 2024-09-14T06:28:59.948 INFO:teuthology.orchestra.run.smithi094.stdout: "last_up_change": "2024-09-14T06:28:42.827239+0000", 2024-09-14T06:28:59.948 INFO:teuthology.orchestra.run.smithi094.stdout: "last_in_change": "2024-09-14T06:28:33.724176+0000", 2024-09-14T06:28:59.948 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2024-09-14T06:28:59.948 INFO:teuthology.orchestra.run.smithi094.stdout: "flags_num": 5799936, 2024-09-14T06:28:59.949 INFO:teuthology.orchestra.run.smithi094.stdout: "flags_set": [ 2024-09-14T06:28:59.949 INFO:teuthology.orchestra.run.smithi094.stdout: "pglog_hardlimit", 2024-09-14T06:28:59.949 INFO:teuthology.orchestra.run.smithi094.stdout: "purged_snapdirs", 2024-09-14T06:28:59.949 INFO:teuthology.orchestra.run.smithi094.stdout: "recovery_deletes", 2024-09-14T06:28:59.949 INFO:teuthology.orchestra.run.smithi094.stdout: "sortbitwise" 2024-09-14T06:28:59.949 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-09-14T06:28:59.949 INFO:teuthology.orchestra.run.smithi094.stdout: "crush_version": 5, 2024-09-14T06:28:59.949 INFO:teuthology.orchestra.run.smithi094.stdout: "full_ratio": 0.94999998807907104, 2024-09-14T06:28:59.949 INFO:teuthology.orchestra.run.smithi094.stdout: "backfillfull_ratio": 0.89999997615814209, 2024-09-14T06:28:59.949 INFO:teuthology.orchestra.run.smithi094.stdout: "nearfull_ratio": 0.85000002384185791, 2024-09-14T06:28:59.949 INFO:teuthology.orchestra.run.smithi094.stdout: "cluster_snapshot": "", 2024-09-14T06:28:59.949 INFO:teuthology.orchestra.run.smithi094.stdout: "pool_max": 2, 2024-09-14T06:28:59.950 INFO:teuthology.orchestra.run.smithi094.stdout: "max_osd": 8, 2024-09-14T06:28:59.950 INFO:teuthology.orchestra.run.smithi094.stdout: "require_min_compat_client": "luminous", 2024-09-14T06:28:59.950 INFO:teuthology.orchestra.run.smithi094.stdout: "min_compat_client": "jewel", 2024-09-14T06:28:59.950 INFO:teuthology.orchestra.run.smithi094.stdout: "require_osd_release": "squid", 2024-09-14T06:28:59.950 INFO:teuthology.orchestra.run.smithi094.stdout: "allow_crimson": false, 2024-09-14T06:28:59.950 INFO:teuthology.orchestra.run.smithi094.stdout: "pools": [ 2024-09-14T06:28:59.950 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:28:59.950 INFO:teuthology.orchestra.run.smithi094.stdout: "pool": 1, 2024-09-14T06:28:59.950 INFO:teuthology.orchestra.run.smithi094.stdout: "pool_name": ".mgr", 2024-09-14T06:28:59.950 INFO:teuthology.orchestra.run.smithi094.stdout: "create_time": "2024-09-14T06:28:43.295990+0000", 2024-09-14T06:28:59.950 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-14T06:28:59.951 INFO:teuthology.orchestra.run.smithi094.stdout: "flags_names": "hashpspool", 2024-09-14T06:28:59.951 INFO:teuthology.orchestra.run.smithi094.stdout: "type": 1, 2024-09-14T06:28:59.951 INFO:teuthology.orchestra.run.smithi094.stdout: "size": 2, 2024-09-14T06:28:59.951 INFO:teuthology.orchestra.run.smithi094.stdout: "min_size": 1, 2024-09-14T06:28:59.951 INFO:teuthology.orchestra.run.smithi094.stdout: "crush_rule": 0, 2024-09-14T06:28:59.951 INFO:teuthology.orchestra.run.smithi094.stdout: "peering_crush_bucket_count": 0, 2024-09-14T06:28:59.951 INFO:teuthology.orchestra.run.smithi094.stdout: "peering_crush_bucket_target": 0, 2024-09-14T06:28:59.951 INFO:teuthology.orchestra.run.smithi094.stdout: "peering_crush_bucket_barrier": 0, 2024-09-14T06:28:59.951 INFO:teuthology.orchestra.run.smithi094.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-14T06:28:59.951 INFO:teuthology.orchestra.run.smithi094.stdout: "is_stretch_pool": false, 2024-09-14T06:28:59.952 INFO:teuthology.orchestra.run.smithi094.stdout: "object_hash": 2, 2024-09-14T06:28:59.952 INFO:teuthology.orchestra.run.smithi094.stdout: "pg_autoscale_mode": "off", 2024-09-14T06:28:59.952 INFO:teuthology.orchestra.run.smithi094.stdout: "pg_num": 1, 2024-09-14T06:28:59.952 INFO:teuthology.orchestra.run.smithi094.stdout: "pg_placement_num": 1, 2024-09-14T06:28:59.952 INFO:teuthology.orchestra.run.smithi094.stdout: "pg_placement_num_target": 1, 2024-09-14T06:28:59.952 INFO:teuthology.orchestra.run.smithi094.stdout: "pg_num_target": 1, 2024-09-14T06:28:59.952 INFO:teuthology.orchestra.run.smithi094.stdout: "pg_num_pending": 1, 2024-09-14T06:28:59.952 INFO:teuthology.orchestra.run.smithi094.stdout: "last_pg_merge_meta": { 2024-09-14T06:28:59.952 INFO:teuthology.orchestra.run.smithi094.stdout: "source_pgid": "0.0", 2024-09-14T06:28:59.952 INFO:teuthology.orchestra.run.smithi094.stdout: "ready_epoch": 0, 2024-09-14T06:28:59.953 INFO:teuthology.orchestra.run.smithi094.stdout: "last_epoch_started": 0, 2024-09-14T06:28:59.953 INFO:teuthology.orchestra.run.smithi094.stdout: "last_epoch_clean": 0, 2024-09-14T06:28:59.953 INFO:teuthology.orchestra.run.smithi094.stdout: "source_version": "0'0", 2024-09-14T06:28:59.953 INFO:teuthology.orchestra.run.smithi094.stdout: "target_version": "0'0" 2024-09-14T06:28:59.953 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:28:59.953 INFO:teuthology.orchestra.run.smithi094.stdout: "last_change": "18", 2024-09-14T06:28:59.953 INFO:teuthology.orchestra.run.smithi094.stdout: "last_force_op_resend": "0", 2024-09-14T06:28:59.953 INFO:teuthology.orchestra.run.smithi094.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-14T06:28:59.953 INFO:teuthology.orchestra.run.smithi094.stdout: "last_force_op_resend_preluminous": "0", 2024-09-14T06:28:59.953 INFO:teuthology.orchestra.run.smithi094.stdout: "auid": 0, 2024-09-14T06:28:59.953 INFO:teuthology.orchestra.run.smithi094.stdout: "snap_mode": "selfmanaged", 2024-09-14T06:28:59.953 INFO:teuthology.orchestra.run.smithi094.stdout: "snap_seq": 0, 2024-09-14T06:28:59.953 INFO:teuthology.orchestra.run.smithi094.stdout: "snap_epoch": 0, 2024-09-14T06:28:59.954 INFO:teuthology.orchestra.run.smithi094.stdout: "pool_snaps": [], 2024-09-14T06:28:59.954 INFO:teuthology.orchestra.run.smithi094.stdout: "removed_snaps": "[]", 2024-09-14T06:28:59.954 INFO:teuthology.orchestra.run.smithi094.stdout: "quota_max_bytes": 0, 2024-09-14T06:28:59.954 INFO:teuthology.orchestra.run.smithi094.stdout: "quota_max_objects": 0, 2024-09-14T06:28:59.954 INFO:teuthology.orchestra.run.smithi094.stdout: "tiers": [], 2024-09-14T06:28:59.954 INFO:teuthology.orchestra.run.smithi094.stdout: "tier_of": -1, 2024-09-14T06:28:59.954 INFO:teuthology.orchestra.run.smithi094.stdout: "read_tier": -1, 2024-09-14T06:28:59.954 INFO:teuthology.orchestra.run.smithi094.stdout: "write_tier": -1, 2024-09-14T06:28:59.954 INFO:teuthology.orchestra.run.smithi094.stdout: "cache_mode": "none", 2024-09-14T06:28:59.954 INFO:teuthology.orchestra.run.smithi094.stdout: "target_max_bytes": 0, 2024-09-14T06:28:59.954 INFO:teuthology.orchestra.run.smithi094.stdout: "target_max_objects": 0, 2024-09-14T06:28:59.954 INFO:teuthology.orchestra.run.smithi094.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-14T06:28:59.955 INFO:teuthology.orchestra.run.smithi094.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-14T06:28:59.955 INFO:teuthology.orchestra.run.smithi094.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-14T06:28:59.955 INFO:teuthology.orchestra.run.smithi094.stdout: "cache_min_flush_age": 0, 2024-09-14T06:28:59.955 INFO:teuthology.orchestra.run.smithi094.stdout: "cache_min_evict_age": 0, 2024-09-14T06:28:59.955 INFO:teuthology.orchestra.run.smithi094.stdout: "erasure_code_profile": "", 2024-09-14T06:28:59.955 INFO:teuthology.orchestra.run.smithi094.stdout: "hit_set_params": { 2024-09-14T06:28:59.955 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "none" 2024-09-14T06:28:59.955 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:28:59.955 INFO:teuthology.orchestra.run.smithi094.stdout: "hit_set_period": 0, 2024-09-14T06:28:59.955 INFO:teuthology.orchestra.run.smithi094.stdout: "hit_set_count": 0, 2024-09-14T06:28:59.955 INFO:teuthology.orchestra.run.smithi094.stdout: "use_gmt_hitset": true, 2024-09-14T06:28:59.955 INFO:teuthology.orchestra.run.smithi094.stdout: "min_read_recency_for_promote": 0, 2024-09-14T06:28:59.955 INFO:teuthology.orchestra.run.smithi094.stdout: "min_write_recency_for_promote": 0, 2024-09-14T06:28:59.956 INFO:teuthology.orchestra.run.smithi094.stdout: "hit_set_grade_decay_rate": 0, 2024-09-14T06:28:59.956 INFO:teuthology.orchestra.run.smithi094.stdout: "hit_set_search_last_n": 0, 2024-09-14T06:28:59.956 INFO:teuthology.orchestra.run.smithi094.stdout: "grade_table": [], 2024-09-14T06:28:59.956 INFO:teuthology.orchestra.run.smithi094.stdout: "stripe_width": 0, 2024-09-14T06:28:59.956 INFO:teuthology.orchestra.run.smithi094.stdout: "expected_num_objects": 0, 2024-09-14T06:28:59.956 INFO:teuthology.orchestra.run.smithi094.stdout: "fast_read": false, 2024-09-14T06:28:59.956 INFO:teuthology.orchestra.run.smithi094.stdout: "options": { 2024-09-14T06:28:59.956 INFO:teuthology.orchestra.run.smithi094.stdout: "pg_num_max": 32, 2024-09-14T06:28:59.956 INFO:teuthology.orchestra.run.smithi094.stdout: "pg_num_min": 1 2024-09-14T06:28:59.956 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:28:59.956 INFO:teuthology.orchestra.run.smithi094.stdout: "application_metadata": { 2024-09-14T06:28:59.956 INFO:teuthology.orchestra.run.smithi094.stdout: "mgr": {} 2024-09-14T06:28:59.957 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:28:59.957 INFO:teuthology.orchestra.run.smithi094.stdout: "read_balance": { 2024-09-14T06:28:59.957 INFO:teuthology.orchestra.run.smithi094.stdout: "score_type": "Fair distribution", 2024-09-14T06:28:59.957 INFO:teuthology.orchestra.run.smithi094.stdout: "score_acting": 8, 2024-09-14T06:28:59.957 INFO:teuthology.orchestra.run.smithi094.stdout: "score_stable": 8, 2024-09-14T06:28:59.957 INFO:teuthology.orchestra.run.smithi094.stdout: "optimal_score": 0.25, 2024-09-14T06:28:59.957 INFO:teuthology.orchestra.run.smithi094.stdout: "raw_score_acting": 2, 2024-09-14T06:28:59.957 INFO:teuthology.orchestra.run.smithi094.stdout: "raw_score_stable": 2, 2024-09-14T06:28:59.957 INFO:teuthology.orchestra.run.smithi094.stdout: "primary_affinity_weighted": 1, 2024-09-14T06:28:59.957 INFO:teuthology.orchestra.run.smithi094.stdout: "average_primary_affinity": 1, 2024-09-14T06:28:59.957 INFO:teuthology.orchestra.run.smithi094.stdout: "average_primary_affinity_weighted": 1 2024-09-14T06:28:59.957 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:28:59.958 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:28:59.958 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:28:59.958 INFO:teuthology.orchestra.run.smithi094.stdout: "pool": 2, 2024-09-14T06:28:59.958 INFO:teuthology.orchestra.run.smithi094.stdout: "pool_name": "rbd", 2024-09-14T06:28:59.958 INFO:teuthology.orchestra.run.smithi094.stdout: "create_time": "2024-09-14T06:28:49.451714+0000", 2024-09-14T06:28:59.958 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 8193, 2024-09-14T06:28:59.958 INFO:teuthology.orchestra.run.smithi094.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-09-14T06:28:59.958 INFO:teuthology.orchestra.run.smithi094.stdout: "type": 1, 2024-09-14T06:28:59.958 INFO:teuthology.orchestra.run.smithi094.stdout: "size": 2, 2024-09-14T06:28:59.959 INFO:teuthology.orchestra.run.smithi094.stdout: "min_size": 1, 2024-09-14T06:28:59.959 INFO:teuthology.orchestra.run.smithi094.stdout: "crush_rule": 0, 2024-09-14T06:28:59.959 INFO:teuthology.orchestra.run.smithi094.stdout: "peering_crush_bucket_count": 0, 2024-09-14T06:28:59.959 INFO:teuthology.orchestra.run.smithi094.stdout: "peering_crush_bucket_target": 0, 2024-09-14T06:28:59.959 INFO:teuthology.orchestra.run.smithi094.stdout: "peering_crush_bucket_barrier": 0, 2024-09-14T06:28:59.959 INFO:teuthology.orchestra.run.smithi094.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-14T06:28:59.959 INFO:teuthology.orchestra.run.smithi094.stdout: "is_stretch_pool": false, 2024-09-14T06:28:59.959 INFO:teuthology.orchestra.run.smithi094.stdout: "object_hash": 2, 2024-09-14T06:28:59.959 INFO:teuthology.orchestra.run.smithi094.stdout: "pg_autoscale_mode": "off", 2024-09-14T06:28:59.959 INFO:teuthology.orchestra.run.smithi094.stdout: "pg_num": 8, 2024-09-14T06:28:59.959 INFO:teuthology.orchestra.run.smithi094.stdout: "pg_placement_num": 8, 2024-09-14T06:28:59.960 INFO:teuthology.orchestra.run.smithi094.stdout: "pg_placement_num_target": 8, 2024-09-14T06:28:59.960 INFO:teuthology.orchestra.run.smithi094.stdout: "pg_num_target": 8, 2024-09-14T06:28:59.960 INFO:teuthology.orchestra.run.smithi094.stdout: "pg_num_pending": 8, 2024-09-14T06:28:59.960 INFO:teuthology.orchestra.run.smithi094.stdout: "last_pg_merge_meta": { 2024-09-14T06:28:59.960 INFO:teuthology.orchestra.run.smithi094.stdout: "source_pgid": "0.0", 2024-09-14T06:28:59.960 INFO:teuthology.orchestra.run.smithi094.stdout: "ready_epoch": 0, 2024-09-14T06:28:59.960 INFO:teuthology.orchestra.run.smithi094.stdout: "last_epoch_started": 0, 2024-09-14T06:28:59.960 INFO:teuthology.orchestra.run.smithi094.stdout: "last_epoch_clean": 0, 2024-09-14T06:28:59.960 INFO:teuthology.orchestra.run.smithi094.stdout: "source_version": "0'0", 2024-09-14T06:28:59.960 INFO:teuthology.orchestra.run.smithi094.stdout: "target_version": "0'0" 2024-09-14T06:28:59.960 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:28:59.960 INFO:teuthology.orchestra.run.smithi094.stdout: "last_change": "22", 2024-09-14T06:28:59.961 INFO:teuthology.orchestra.run.smithi094.stdout: "last_force_op_resend": "0", 2024-09-14T06:28:59.961 INFO:teuthology.orchestra.run.smithi094.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-14T06:28:59.961 INFO:teuthology.orchestra.run.smithi094.stdout: "last_force_op_resend_preluminous": "0", 2024-09-14T06:28:59.961 INFO:teuthology.orchestra.run.smithi094.stdout: "auid": 0, 2024-09-14T06:28:59.961 INFO:teuthology.orchestra.run.smithi094.stdout: "snap_mode": "selfmanaged", 2024-09-14T06:28:59.961 INFO:teuthology.orchestra.run.smithi094.stdout: "snap_seq": 2, 2024-09-14T06:28:59.961 INFO:teuthology.orchestra.run.smithi094.stdout: "snap_epoch": 22, 2024-09-14T06:28:59.961 INFO:teuthology.orchestra.run.smithi094.stdout: "pool_snaps": [], 2024-09-14T06:28:59.961 INFO:teuthology.orchestra.run.smithi094.stdout: "removed_snaps": "[]", 2024-09-14T06:28:59.961 INFO:teuthology.orchestra.run.smithi094.stdout: "quota_max_bytes": 0, 2024-09-14T06:28:59.961 INFO:teuthology.orchestra.run.smithi094.stdout: "quota_max_objects": 0, 2024-09-14T06:28:59.961 INFO:teuthology.orchestra.run.smithi094.stdout: "tiers": [], 2024-09-14T06:28:59.962 INFO:teuthology.orchestra.run.smithi094.stdout: "tier_of": -1, 2024-09-14T06:28:59.962 INFO:teuthology.orchestra.run.smithi094.stdout: "read_tier": -1, 2024-09-14T06:28:59.962 INFO:teuthology.orchestra.run.smithi094.stdout: "write_tier": -1, 2024-09-14T06:28:59.962 INFO:teuthology.orchestra.run.smithi094.stdout: "cache_mode": "none", 2024-09-14T06:28:59.962 INFO:teuthology.orchestra.run.smithi094.stdout: "target_max_bytes": 0, 2024-09-14T06:28:59.962 INFO:teuthology.orchestra.run.smithi094.stdout: "target_max_objects": 0, 2024-09-14T06:28:59.962 INFO:teuthology.orchestra.run.smithi094.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-14T06:28:59.962 INFO:teuthology.orchestra.run.smithi094.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-14T06:28:59.962 INFO:teuthology.orchestra.run.smithi094.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-14T06:28:59.962 INFO:teuthology.orchestra.run.smithi094.stdout: "cache_min_flush_age": 0, 2024-09-14T06:28:59.962 INFO:teuthology.orchestra.run.smithi094.stdout: "cache_min_evict_age": 0, 2024-09-14T06:28:59.962 INFO:teuthology.orchestra.run.smithi094.stdout: "erasure_code_profile": "", 2024-09-14T06:28:59.963 INFO:teuthology.orchestra.run.smithi094.stdout: "hit_set_params": { 2024-09-14T06:28:59.963 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "none" 2024-09-14T06:28:59.963 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:28:59.963 INFO:teuthology.orchestra.run.smithi094.stdout: "hit_set_period": 0, 2024-09-14T06:28:59.963 INFO:teuthology.orchestra.run.smithi094.stdout: "hit_set_count": 0, 2024-09-14T06:28:59.963 INFO:teuthology.orchestra.run.smithi094.stdout: "use_gmt_hitset": true, 2024-09-14T06:28:59.963 INFO:teuthology.orchestra.run.smithi094.stdout: "min_read_recency_for_promote": 0, 2024-09-14T06:28:59.963 INFO:teuthology.orchestra.run.smithi094.stdout: "min_write_recency_for_promote": 0, 2024-09-14T06:28:59.963 INFO:teuthology.orchestra.run.smithi094.stdout: "hit_set_grade_decay_rate": 0, 2024-09-14T06:28:59.963 INFO:teuthology.orchestra.run.smithi094.stdout: "hit_set_search_last_n": 0, 2024-09-14T06:28:59.963 INFO:teuthology.orchestra.run.smithi094.stdout: "grade_table": [], 2024-09-14T06:28:59.963 INFO:teuthology.orchestra.run.smithi094.stdout: "stripe_width": 0, 2024-09-14T06:28:59.963 INFO:teuthology.orchestra.run.smithi094.stdout: "expected_num_objects": 0, 2024-09-14T06:28:59.963 INFO:teuthology.orchestra.run.smithi094.stdout: "fast_read": false, 2024-09-14T06:28:59.963 INFO:teuthology.orchestra.run.smithi094.stdout: "options": {}, 2024-09-14T06:28:59.964 INFO:teuthology.orchestra.run.smithi094.stdout: "application_metadata": { 2024-09-14T06:28:59.964 INFO:teuthology.orchestra.run.smithi094.stdout: "rbd": {} 2024-09-14T06:28:59.964 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:28:59.964 INFO:teuthology.orchestra.run.smithi094.stdout: "read_balance": { 2024-09-14T06:28:59.964 INFO:teuthology.orchestra.run.smithi094.stdout: "score_type": "Fair distribution", 2024-09-14T06:28:59.964 INFO:teuthology.orchestra.run.smithi094.stdout: "score_acting": 1.9900000095367432, 2024-09-14T06:28:59.964 INFO:teuthology.orchestra.run.smithi094.stdout: "score_stable": 1.9900000095367432, 2024-09-14T06:28:59.964 INFO:teuthology.orchestra.run.smithi094.stdout: "optimal_score": 0.87999999523162842, 2024-09-14T06:28:59.964 INFO:teuthology.orchestra.run.smithi094.stdout: "raw_score_acting": 1.75, 2024-09-14T06:28:59.964 INFO:teuthology.orchestra.run.smithi094.stdout: "raw_score_stable": 1.75, 2024-09-14T06:28:59.964 INFO:teuthology.orchestra.run.smithi094.stdout: "primary_affinity_weighted": 1, 2024-09-14T06:28:59.964 INFO:teuthology.orchestra.run.smithi094.stdout: "average_primary_affinity": 1, 2024-09-14T06:28:59.964 INFO:teuthology.orchestra.run.smithi094.stdout: "average_primary_affinity_weighted": 1 2024-09-14T06:28:59.964 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:28:59.964 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:28:59.965 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-09-14T06:28:59.965 INFO:teuthology.orchestra.run.smithi094.stdout: "osds": [ 2024-09-14T06:28:59.965 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:28:59.965 INFO:teuthology.orchestra.run.smithi094.stdout: "osd": 0, 2024-09-14T06:28:59.965 INFO:teuthology.orchestra.run.smithi094.stdout: "uuid": "8b22a249-b842-406a-a02c-c0f31f57b635", 2024-09-14T06:28:59.965 INFO:teuthology.orchestra.run.smithi094.stdout: "up": 1, 2024-09-14T06:28:59.965 INFO:teuthology.orchestra.run.smithi094.stdout: "in": 1, 2024-09-14T06:28:59.965 INFO:teuthology.orchestra.run.smithi094.stdout: "weight": 1, 2024-09-14T06:28:59.965 INFO:teuthology.orchestra.run.smithi094.stdout: "primary_affinity": 1, 2024-09-14T06:28:59.965 INFO:teuthology.orchestra.run.smithi094.stdout: "last_clean_begin": 0, 2024-09-14T06:28:59.965 INFO:teuthology.orchestra.run.smithi094.stdout: "last_clean_end": 0, 2024-09-14T06:28:59.965 INFO:teuthology.orchestra.run.smithi094.stdout: "up_from": 14, 2024-09-14T06:28:59.965 INFO:teuthology.orchestra.run.smithi094.stdout: "up_thru": 0, 2024-09-14T06:28:59.965 INFO:teuthology.orchestra.run.smithi094.stdout: "down_at": 0, 2024-09-14T06:28:59.966 INFO:teuthology.orchestra.run.smithi094.stdout: "lost_at": 0, 2024-09-14T06:28:59.966 INFO:teuthology.orchestra.run.smithi094.stdout: "public_addrs": { 2024-09-14T06:28:59.966 INFO:teuthology.orchestra.run.smithi094.stdout: "addrvec": [ 2024-09-14T06:28:59.966 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:28:59.966 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v2", 2024-09-14T06:28:59.966 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.94:6824", 2024-09-14T06:28:59.966 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 1809564923 2024-09-14T06:28:59.966 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:28:59.966 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:28:59.966 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v1", 2024-09-14T06:28:59.966 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.94:6825", 2024-09-14T06:28:59.966 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 1809564923 2024-09-14T06:28:59.966 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:28:59.966 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:28:59.966 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:28:59.967 INFO:teuthology.orchestra.run.smithi094.stdout: "cluster_addrs": { 2024-09-14T06:28:59.967 INFO:teuthology.orchestra.run.smithi094.stdout: "addrvec": [ 2024-09-14T06:28:59.967 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:28:59.967 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v2", 2024-09-14T06:28:59.967 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.94:6826", 2024-09-14T06:28:59.967 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 1809564923 2024-09-14T06:28:59.967 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:28:59.967 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:28:59.967 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v1", 2024-09-14T06:28:59.967 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.94:6827", 2024-09-14T06:28:59.967 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 1809564923 2024-09-14T06:28:59.967 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:28:59.967 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:28:59.968 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:28:59.968 INFO:teuthology.orchestra.run.smithi094.stdout: "heartbeat_back_addrs": { 2024-09-14T06:28:59.968 INFO:teuthology.orchestra.run.smithi094.stdout: "addrvec": [ 2024-09-14T06:28:59.968 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:28:59.968 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v2", 2024-09-14T06:28:59.968 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.94:6830", 2024-09-14T06:28:59.968 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 1809564923 2024-09-14T06:28:59.968 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:28:59.968 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:28:59.968 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v1", 2024-09-14T06:28:59.968 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.94:6831", 2024-09-14T06:28:59.968 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 1809564923 2024-09-14T06:28:59.968 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:28:59.969 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:28:59.969 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:28:59.969 INFO:teuthology.orchestra.run.smithi094.stdout: "heartbeat_front_addrs": { 2024-09-14T06:28:59.969 INFO:teuthology.orchestra.run.smithi094.stdout: "addrvec": [ 2024-09-14T06:28:59.969 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:28:59.969 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v2", 2024-09-14T06:28:59.969 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.94:6828", 2024-09-14T06:28:59.969 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 1809564923 2024-09-14T06:28:59.969 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:28:59.969 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:28:59.969 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v1", 2024-09-14T06:28:59.969 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.94:6829", 2024-09-14T06:28:59.969 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 1809564923 2024-09-14T06:28:59.970 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:28:59.970 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:28:59.970 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:28:59.970 INFO:teuthology.orchestra.run.smithi094.stdout: "public_addr": "172.21.15.94:6825/1809564923", 2024-09-14T06:28:59.970 INFO:teuthology.orchestra.run.smithi094.stdout: "cluster_addr": "172.21.15.94:6827/1809564923", 2024-09-14T06:28:59.970 INFO:teuthology.orchestra.run.smithi094.stdout: "heartbeat_back_addr": "172.21.15.94:6831/1809564923", 2024-09-14T06:28:59.970 INFO:teuthology.orchestra.run.smithi094.stdout: "heartbeat_front_addr": "172.21.15.94:6829/1809564923", 2024-09-14T06:28:59.970 INFO:teuthology.orchestra.run.smithi094.stdout: "state": [ 2024-09-14T06:28:59.970 INFO:teuthology.orchestra.run.smithi094.stdout: "exists", 2024-09-14T06:28:59.970 INFO:teuthology.orchestra.run.smithi094.stdout: "up" 2024-09-14T06:28:59.970 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:28:59.970 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:28:59.970 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:28:59.970 INFO:teuthology.orchestra.run.smithi094.stdout: "osd": 1, 2024-09-14T06:28:59.971 INFO:teuthology.orchestra.run.smithi094.stdout: "uuid": "02251a03-84a0-414d-a980-fac4fd043cb6", 2024-09-14T06:28:59.971 INFO:teuthology.orchestra.run.smithi094.stdout: "up": 1, 2024-09-14T06:28:59.971 INFO:teuthology.orchestra.run.smithi094.stdout: "in": 1, 2024-09-14T06:28:59.971 INFO:teuthology.orchestra.run.smithi094.stdout: "weight": 1, 2024-09-14T06:28:59.971 INFO:teuthology.orchestra.run.smithi094.stdout: "primary_affinity": 1, 2024-09-14T06:28:59.971 INFO:teuthology.orchestra.run.smithi094.stdout: "last_clean_begin": 0, 2024-09-14T06:28:59.971 INFO:teuthology.orchestra.run.smithi094.stdout: "last_clean_end": 0, 2024-09-14T06:28:59.971 INFO:teuthology.orchestra.run.smithi094.stdout: "up_from": 15, 2024-09-14T06:28:59.971 INFO:teuthology.orchestra.run.smithi094.stdout: "up_thru": 19, 2024-09-14T06:28:59.971 INFO:teuthology.orchestra.run.smithi094.stdout: "down_at": 0, 2024-09-14T06:28:59.971 INFO:teuthology.orchestra.run.smithi094.stdout: "lost_at": 0, 2024-09-14T06:28:59.971 INFO:teuthology.orchestra.run.smithi094.stdout: "public_addrs": { 2024-09-14T06:28:59.972 INFO:teuthology.orchestra.run.smithi094.stdout: "addrvec": [ 2024-09-14T06:28:59.972 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:28:59.972 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v2", 2024-09-14T06:28:59.972 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.94:6816", 2024-09-14T06:28:59.972 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 736402652 2024-09-14T06:28:59.972 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:28:59.972 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:28:59.972 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v1", 2024-09-14T06:28:59.972 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.94:6817", 2024-09-14T06:28:59.972 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 736402652 2024-09-14T06:28:59.973 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:28:59.973 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:28:59.973 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:28:59.973 INFO:teuthology.orchestra.run.smithi094.stdout: "cluster_addrs": { 2024-09-14T06:28:59.973 INFO:teuthology.orchestra.run.smithi094.stdout: "addrvec": [ 2024-09-14T06:28:59.973 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:28:59.973 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v2", 2024-09-14T06:28:59.973 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.94:6818", 2024-09-14T06:28:59.973 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 736402652 2024-09-14T06:28:59.973 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:28:59.973 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:28:59.973 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v1", 2024-09-14T06:28:59.973 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.94:6819", 2024-09-14T06:28:59.974 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 736402652 2024-09-14T06:28:59.974 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:28:59.974 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:28:59.974 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:28:59.974 INFO:teuthology.orchestra.run.smithi094.stdout: "heartbeat_back_addrs": { 2024-09-14T06:28:59.974 INFO:teuthology.orchestra.run.smithi094.stdout: "addrvec": [ 2024-09-14T06:28:59.974 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:28:59.974 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v2", 2024-09-14T06:28:59.974 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.94:6822", 2024-09-14T06:28:59.974 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 736402652 2024-09-14T06:28:59.975 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:28:59.975 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:28:59.975 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v1", 2024-09-14T06:28:59.975 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.94:6823", 2024-09-14T06:28:59.975 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 736402652 2024-09-14T06:28:59.975 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:28:59.975 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:28:59.975 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:28:59.975 INFO:teuthology.orchestra.run.smithi094.stdout: "heartbeat_front_addrs": { 2024-09-14T06:28:59.975 INFO:teuthology.orchestra.run.smithi094.stdout: "addrvec": [ 2024-09-14T06:28:59.975 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:28:59.975 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v2", 2024-09-14T06:28:59.975 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.94:6820", 2024-09-14T06:28:59.975 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 736402652 2024-09-14T06:28:59.976 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:28:59.976 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:28:59.976 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v1", 2024-09-14T06:28:59.976 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.94:6821", 2024-09-14T06:28:59.976 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 736402652 2024-09-14T06:28:59.976 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:28:59.976 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:28:59.976 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:28:59.976 INFO:teuthology.orchestra.run.smithi094.stdout: "public_addr": "172.21.15.94:6817/736402652", 2024-09-14T06:28:59.976 INFO:teuthology.orchestra.run.smithi094.stdout: "cluster_addr": "172.21.15.94:6819/736402652", 2024-09-14T06:28:59.976 INFO:teuthology.orchestra.run.smithi094.stdout: "heartbeat_back_addr": "172.21.15.94:6823/736402652", 2024-09-14T06:28:59.976 INFO:teuthology.orchestra.run.smithi094.stdout: "heartbeat_front_addr": "172.21.15.94:6821/736402652", 2024-09-14T06:28:59.976 INFO:teuthology.orchestra.run.smithi094.stdout: "state": [ 2024-09-14T06:28:59.977 INFO:teuthology.orchestra.run.smithi094.stdout: "exists", 2024-09-14T06:28:59.977 INFO:teuthology.orchestra.run.smithi094.stdout: "up" 2024-09-14T06:28:59.977 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:28:59.977 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:28:59.977 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:28:59.977 INFO:teuthology.orchestra.run.smithi094.stdout: "osd": 2, 2024-09-14T06:28:59.977 INFO:teuthology.orchestra.run.smithi094.stdout: "uuid": "d45bf470-bb29-4739-8294-cd122b2b4102", 2024-09-14T06:28:59.977 INFO:teuthology.orchestra.run.smithi094.stdout: "up": 1, 2024-09-14T06:28:59.977 INFO:teuthology.orchestra.run.smithi094.stdout: "in": 1, 2024-09-14T06:28:59.977 INFO:teuthology.orchestra.run.smithi094.stdout: "weight": 1, 2024-09-14T06:28:59.977 INFO:teuthology.orchestra.run.smithi094.stdout: "primary_affinity": 1, 2024-09-14T06:28:59.977 INFO:teuthology.orchestra.run.smithi094.stdout: "last_clean_begin": 0, 2024-09-14T06:28:59.977 INFO:teuthology.orchestra.run.smithi094.stdout: "last_clean_end": 0, 2024-09-14T06:28:59.978 INFO:teuthology.orchestra.run.smithi094.stdout: "up_from": 14, 2024-09-14T06:28:59.978 INFO:teuthology.orchestra.run.smithi094.stdout: "up_thru": 19, 2024-09-14T06:28:59.978 INFO:teuthology.orchestra.run.smithi094.stdout: "down_at": 0, 2024-09-14T06:28:59.978 INFO:teuthology.orchestra.run.smithi094.stdout: "lost_at": 0, 2024-09-14T06:28:59.978 INFO:teuthology.orchestra.run.smithi094.stdout: "public_addrs": { 2024-09-14T06:28:59.978 INFO:teuthology.orchestra.run.smithi094.stdout: "addrvec": [ 2024-09-14T06:28:59.978 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:28:59.978 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v2", 2024-09-14T06:28:59.978 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.94:6808", 2024-09-14T06:28:59.978 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 1063204782 2024-09-14T06:28:59.978 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:28:59.978 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:28:59.978 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v1", 2024-09-14T06:28:59.979 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.94:6809", 2024-09-14T06:28:59.979 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 1063204782 2024-09-14T06:28:59.979 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:28:59.979 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:28:59.979 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:28:59.979 INFO:teuthology.orchestra.run.smithi094.stdout: "cluster_addrs": { 2024-09-14T06:28:59.979 INFO:teuthology.orchestra.run.smithi094.stdout: "addrvec": [ 2024-09-14T06:28:59.979 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:28:59.979 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v2", 2024-09-14T06:28:59.979 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.94:6810", 2024-09-14T06:28:59.979 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 1063204782 2024-09-14T06:28:59.979 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:28:59.979 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:28:59.980 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v1", 2024-09-14T06:28:59.980 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.94:6811", 2024-09-14T06:28:59.980 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 1063204782 2024-09-14T06:28:59.980 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:28:59.980 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:28:59.980 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:28:59.980 INFO:teuthology.orchestra.run.smithi094.stdout: "heartbeat_back_addrs": { 2024-09-14T06:28:59.980 INFO:teuthology.orchestra.run.smithi094.stdout: "addrvec": [ 2024-09-14T06:28:59.980 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:28:59.980 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v2", 2024-09-14T06:28:59.980 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.94:6814", 2024-09-14T06:28:59.980 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 1063204782 2024-09-14T06:28:59.980 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:28:59.980 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:28:59.981 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v1", 2024-09-14T06:28:59.981 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.94:6815", 2024-09-14T06:28:59.981 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 1063204782 2024-09-14T06:28:59.981 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:28:59.981 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:28:59.981 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:28:59.981 INFO:teuthology.orchestra.run.smithi094.stdout: "heartbeat_front_addrs": { 2024-09-14T06:28:59.981 INFO:teuthology.orchestra.run.smithi094.stdout: "addrvec": [ 2024-09-14T06:28:59.981 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:28:59.981 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v2", 2024-09-14T06:28:59.981 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.94:6812", 2024-09-14T06:28:59.981 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 1063204782 2024-09-14T06:28:59.981 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:28:59.982 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:28:59.982 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v1", 2024-09-14T06:28:59.982 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.94:6813", 2024-09-14T06:28:59.982 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 1063204782 2024-09-14T06:28:59.982 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:28:59.982 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:28:59.982 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:28:59.982 INFO:teuthology.orchestra.run.smithi094.stdout: "public_addr": "172.21.15.94:6809/1063204782", 2024-09-14T06:28:59.982 INFO:teuthology.orchestra.run.smithi094.stdout: "cluster_addr": "172.21.15.94:6811/1063204782", 2024-09-14T06:28:59.982 INFO:teuthology.orchestra.run.smithi094.stdout: "heartbeat_back_addr": "172.21.15.94:6815/1063204782", 2024-09-14T06:28:59.982 INFO:teuthology.orchestra.run.smithi094.stdout: "heartbeat_front_addr": "172.21.15.94:6813/1063204782", 2024-09-14T06:28:59.983 INFO:teuthology.orchestra.run.smithi094.stdout: "state": [ 2024-09-14T06:28:59.983 INFO:teuthology.orchestra.run.smithi094.stdout: "exists", 2024-09-14T06:28:59.983 INFO:teuthology.orchestra.run.smithi094.stdout: "up" 2024-09-14T06:28:59.983 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:28:59.983 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:28:59.983 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:28:59.983 INFO:teuthology.orchestra.run.smithi094.stdout: "osd": 3, 2024-09-14T06:28:59.983 INFO:teuthology.orchestra.run.smithi094.stdout: "uuid": "456ec2d9-c438-4243-b931-335704f661eb", 2024-09-14T06:28:59.983 INFO:teuthology.orchestra.run.smithi094.stdout: "up": 1, 2024-09-14T06:28:59.983 INFO:teuthology.orchestra.run.smithi094.stdout: "in": 1, 2024-09-14T06:28:59.983 INFO:teuthology.orchestra.run.smithi094.stdout: "weight": 1, 2024-09-14T06:28:59.983 INFO:teuthology.orchestra.run.smithi094.stdout: "primary_affinity": 1, 2024-09-14T06:28:59.983 INFO:teuthology.orchestra.run.smithi094.stdout: "last_clean_begin": 0, 2024-09-14T06:28:59.983 INFO:teuthology.orchestra.run.smithi094.stdout: "last_clean_end": 0, 2024-09-14T06:28:59.984 INFO:teuthology.orchestra.run.smithi094.stdout: "up_from": 13, 2024-09-14T06:28:59.984 INFO:teuthology.orchestra.run.smithi094.stdout: "up_thru": 0, 2024-09-14T06:28:59.984 INFO:teuthology.orchestra.run.smithi094.stdout: "down_at": 0, 2024-09-14T06:28:59.984 INFO:teuthology.orchestra.run.smithi094.stdout: "lost_at": 0, 2024-09-14T06:28:59.984 INFO:teuthology.orchestra.run.smithi094.stdout: "public_addrs": { 2024-09-14T06:28:59.984 INFO:teuthology.orchestra.run.smithi094.stdout: "addrvec": [ 2024-09-14T06:28:59.984 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:28:59.984 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v2", 2024-09-14T06:28:59.984 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.94:6800", 2024-09-14T06:28:59.984 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 3382046754 2024-09-14T06:28:59.984 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:28:59.984 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:28:59.984 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v1", 2024-09-14T06:28:59.985 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.94:6801", 2024-09-14T06:28:59.985 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 3382046754 2024-09-14T06:28:59.985 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:28:59.985 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:28:59.985 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:28:59.985 INFO:teuthology.orchestra.run.smithi094.stdout: "cluster_addrs": { 2024-09-14T06:28:59.985 INFO:teuthology.orchestra.run.smithi094.stdout: "addrvec": [ 2024-09-14T06:28:59.985 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:28:59.985 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v2", 2024-09-14T06:28:59.985 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.94:6802", 2024-09-14T06:28:59.986 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 3382046754 2024-09-14T06:28:59.986 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:28:59.986 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:28:59.986 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v1", 2024-09-14T06:28:59.986 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.94:6803", 2024-09-14T06:28:59.986 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 3382046754 2024-09-14T06:28:59.986 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:28:59.986 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:28:59.986 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:28:59.986 INFO:teuthology.orchestra.run.smithi094.stdout: "heartbeat_back_addrs": { 2024-09-14T06:28:59.986 INFO:teuthology.orchestra.run.smithi094.stdout: "addrvec": [ 2024-09-14T06:28:59.986 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:28:59.986 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v2", 2024-09-14T06:28:59.986 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.94:6806", 2024-09-14T06:28:59.987 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 3382046754 2024-09-14T06:28:59.987 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:28:59.987 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:28:59.987 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v1", 2024-09-14T06:28:59.987 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.94:6807", 2024-09-14T06:28:59.987 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 3382046754 2024-09-14T06:28:59.987 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:28:59.987 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:28:59.987 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:28:59.987 INFO:teuthology.orchestra.run.smithi094.stdout: "heartbeat_front_addrs": { 2024-09-14T06:28:59.987 INFO:teuthology.orchestra.run.smithi094.stdout: "addrvec": [ 2024-09-14T06:28:59.987 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:28:59.987 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v2", 2024-09-14T06:28:59.988 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.94:6804", 2024-09-14T06:28:59.988 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 3382046754 2024-09-14T06:28:59.988 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:28:59.988 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:28:59.988 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v1", 2024-09-14T06:28:59.988 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.94:6805", 2024-09-14T06:28:59.988 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 3382046754 2024-09-14T06:28:59.988 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:28:59.988 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:28:59.988 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:28:59.989 INFO:teuthology.orchestra.run.smithi094.stdout: "public_addr": "172.21.15.94:6801/3382046754", 2024-09-14T06:28:59.989 INFO:teuthology.orchestra.run.smithi094.stdout: "cluster_addr": "172.21.15.94:6803/3382046754", 2024-09-14T06:28:59.989 INFO:teuthology.orchestra.run.smithi094.stdout: "heartbeat_back_addr": "172.21.15.94:6807/3382046754", 2024-09-14T06:28:59.989 INFO:teuthology.orchestra.run.smithi094.stdout: "heartbeat_front_addr": "172.21.15.94:6805/3382046754", 2024-09-14T06:28:59.989 INFO:teuthology.orchestra.run.smithi094.stdout: "state": [ 2024-09-14T06:28:59.989 INFO:teuthology.orchestra.run.smithi094.stdout: "exists", 2024-09-14T06:28:59.989 INFO:teuthology.orchestra.run.smithi094.stdout: "up" 2024-09-14T06:28:59.989 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:28:59.989 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:28:59.989 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:28:59.989 INFO:teuthology.orchestra.run.smithi094.stdout: "osd": 4, 2024-09-14T06:28:59.989 INFO:teuthology.orchestra.run.smithi094.stdout: "uuid": "2dab948e-c6e4-4437-bf17-017d1ac6821c", 2024-09-14T06:28:59.989 INFO:teuthology.orchestra.run.smithi094.stdout: "up": 1, 2024-09-14T06:28:59.990 INFO:teuthology.orchestra.run.smithi094.stdout: "in": 1, 2024-09-14T06:28:59.990 INFO:teuthology.orchestra.run.smithi094.stdout: "weight": 1, 2024-09-14T06:28:59.990 INFO:teuthology.orchestra.run.smithi094.stdout: "primary_affinity": 1, 2024-09-14T06:28:59.990 INFO:teuthology.orchestra.run.smithi094.stdout: "last_clean_begin": 0, 2024-09-14T06:28:59.990 INFO:teuthology.orchestra.run.smithi094.stdout: "last_clean_end": 0, 2024-09-14T06:28:59.990 INFO:teuthology.orchestra.run.smithi094.stdout: "up_from": 15, 2024-09-14T06:28:59.990 INFO:teuthology.orchestra.run.smithi094.stdout: "up_thru": 0, 2024-09-14T06:28:59.990 INFO:teuthology.orchestra.run.smithi094.stdout: "down_at": 0, 2024-09-14T06:28:59.990 INFO:teuthology.orchestra.run.smithi094.stdout: "lost_at": 0, 2024-09-14T06:28:59.990 INFO:teuthology.orchestra.run.smithi094.stdout: "public_addrs": { 2024-09-14T06:28:59.990 INFO:teuthology.orchestra.run.smithi094.stdout: "addrvec": [ 2024-09-14T06:28:59.990 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:28:59.990 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v2", 2024-09-14T06:28:59.990 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.138:6810", 2024-09-14T06:28:59.990 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 501726729 2024-09-14T06:28:59.991 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:28:59.991 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:28:59.991 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v1", 2024-09-14T06:28:59.991 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.138:6811", 2024-09-14T06:28:59.991 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 501726729 2024-09-14T06:28:59.991 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:28:59.991 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:28:59.991 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:28:59.991 INFO:teuthology.orchestra.run.smithi094.stdout: "cluster_addrs": { 2024-09-14T06:28:59.991 INFO:teuthology.orchestra.run.smithi094.stdout: "addrvec": [ 2024-09-14T06:28:59.991 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:28:59.991 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v2", 2024-09-14T06:28:59.991 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.138:6812", 2024-09-14T06:28:59.991 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 501726729 2024-09-14T06:28:59.991 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:28:59.992 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:28:59.992 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v1", 2024-09-14T06:28:59.992 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.138:6813", 2024-09-14T06:28:59.992 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 501726729 2024-09-14T06:28:59.992 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:28:59.992 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:28:59.992 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:28:59.992 INFO:teuthology.orchestra.run.smithi094.stdout: "heartbeat_back_addrs": { 2024-09-14T06:28:59.992 INFO:teuthology.orchestra.run.smithi094.stdout: "addrvec": [ 2024-09-14T06:28:59.992 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:28:59.992 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v2", 2024-09-14T06:28:59.992 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.138:6816", 2024-09-14T06:28:59.992 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 501726729 2024-09-14T06:28:59.992 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:28:59.992 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:28:59.993 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v1", 2024-09-14T06:28:59.993 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.138:6817", 2024-09-14T06:28:59.993 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 501726729 2024-09-14T06:28:59.993 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:28:59.993 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:28:59.993 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:28:59.993 INFO:teuthology.orchestra.run.smithi094.stdout: "heartbeat_front_addrs": { 2024-09-14T06:28:59.993 INFO:teuthology.orchestra.run.smithi094.stdout: "addrvec": [ 2024-09-14T06:28:59.993 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:28:59.994 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v2", 2024-09-14T06:28:59.994 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.138:6814", 2024-09-14T06:28:59.994 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 501726729 2024-09-14T06:28:59.994 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:28:59.994 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:28:59.994 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v1", 2024-09-14T06:28:59.994 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.138:6815", 2024-09-14T06:28:59.994 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 501726729 2024-09-14T06:28:59.994 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:28:59.995 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:28:59.995 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:28:59.995 INFO:teuthology.orchestra.run.smithi094.stdout: "public_addr": "172.21.15.138:6811/501726729", 2024-09-14T06:28:59.995 INFO:teuthology.orchestra.run.smithi094.stdout: "cluster_addr": "172.21.15.138:6813/501726729", 2024-09-14T06:28:59.995 INFO:teuthology.orchestra.run.smithi094.stdout: "heartbeat_back_addr": "172.21.15.138:6817/501726729", 2024-09-14T06:28:59.995 INFO:teuthology.orchestra.run.smithi094.stdout: "heartbeat_front_addr": "172.21.15.138:6815/501726729", 2024-09-14T06:28:59.995 INFO:teuthology.orchestra.run.smithi094.stdout: "state": [ 2024-09-14T06:28:59.995 INFO:teuthology.orchestra.run.smithi094.stdout: "exists", 2024-09-14T06:28:59.995 INFO:teuthology.orchestra.run.smithi094.stdout: "up" 2024-09-14T06:28:59.995 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:28:59.995 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:28:59.995 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:28:59.995 INFO:teuthology.orchestra.run.smithi094.stdout: "osd": 5, 2024-09-14T06:28:59.996 INFO:teuthology.orchestra.run.smithi094.stdout: "uuid": "8caf3aa3-ad87-4d33-b82e-4c15195176fd", 2024-09-14T06:28:59.996 INFO:teuthology.orchestra.run.smithi094.stdout: "up": 1, 2024-09-14T06:28:59.996 INFO:teuthology.orchestra.run.smithi094.stdout: "in": 1, 2024-09-14T06:28:59.996 INFO:teuthology.orchestra.run.smithi094.stdout: "weight": 1, 2024-09-14T06:28:59.996 INFO:teuthology.orchestra.run.smithi094.stdout: "primary_affinity": 1, 2024-09-14T06:28:59.996 INFO:teuthology.orchestra.run.smithi094.stdout: "last_clean_begin": 0, 2024-09-14T06:28:59.996 INFO:teuthology.orchestra.run.smithi094.stdout: "last_clean_end": 0, 2024-09-14T06:28:59.996 INFO:teuthology.orchestra.run.smithi094.stdout: "up_from": 15, 2024-09-14T06:28:59.996 INFO:teuthology.orchestra.run.smithi094.stdout: "up_thru": 19, 2024-09-14T06:28:59.996 INFO:teuthology.orchestra.run.smithi094.stdout: "down_at": 0, 2024-09-14T06:28:59.996 INFO:teuthology.orchestra.run.smithi094.stdout: "lost_at": 0, 2024-09-14T06:28:59.996 INFO:teuthology.orchestra.run.smithi094.stdout: "public_addrs": { 2024-09-14T06:28:59.996 INFO:teuthology.orchestra.run.smithi094.stdout: "addrvec": [ 2024-09-14T06:28:59.997 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:28:59.997 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v2", 2024-09-14T06:28:59.997 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.138:6826", 2024-09-14T06:28:59.997 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 1599292735 2024-09-14T06:28:59.997 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:28:59.997 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:28:59.997 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v1", 2024-09-14T06:28:59.997 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.138:6827", 2024-09-14T06:28:59.997 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 1599292735 2024-09-14T06:28:59.997 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:28:59.997 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:28:59.997 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:28:59.997 INFO:teuthology.orchestra.run.smithi094.stdout: "cluster_addrs": { 2024-09-14T06:28:59.997 INFO:teuthology.orchestra.run.smithi094.stdout: "addrvec": [ 2024-09-14T06:28:59.998 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:28:59.998 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v2", 2024-09-14T06:28:59.998 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.138:6828", 2024-09-14T06:28:59.998 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 1599292735 2024-09-14T06:28:59.998 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:28:59.998 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:28:59.998 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v1", 2024-09-14T06:28:59.998 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.138:6829", 2024-09-14T06:28:59.998 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 1599292735 2024-09-14T06:28:59.998 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:28:59.998 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:28:59.998 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:28:59.998 INFO:teuthology.orchestra.run.smithi094.stdout: "heartbeat_back_addrs": { 2024-09-14T06:28:59.999 INFO:teuthology.orchestra.run.smithi094.stdout: "addrvec": [ 2024-09-14T06:28:59.999 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:28:59.999 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v2", 2024-09-14T06:28:59.999 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.138:6832", 2024-09-14T06:28:59.999 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 1599292735 2024-09-14T06:28:59.999 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:28:59.999 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:28:59.999 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v1", 2024-09-14T06:28:59.999 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.138:6833", 2024-09-14T06:28:59.999 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 1599292735 2024-09-14T06:28:59.999 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:28:59.999 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:28:59.999 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:28:59.999 INFO:teuthology.orchestra.run.smithi094.stdout: "heartbeat_front_addrs": { 2024-09-14T06:29:00.000 INFO:teuthology.orchestra.run.smithi094.stdout: "addrvec": [ 2024-09-14T06:29:00.000 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:00.000 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v2", 2024-09-14T06:29:00.000 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.138:6830", 2024-09-14T06:29:00.000 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 1599292735 2024-09-14T06:29:00.000 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:00.000 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:00.000 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v1", 2024-09-14T06:29:00.000 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.138:6831", 2024-09-14T06:29:00.000 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 1599292735 2024-09-14T06:29:00.000 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:00.000 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:00.000 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:00.001 INFO:teuthology.orchestra.run.smithi094.stdout: "public_addr": "172.21.15.138:6827/1599292735", 2024-09-14T06:29:00.001 INFO:teuthology.orchestra.run.smithi094.stdout: "cluster_addr": "172.21.15.138:6829/1599292735", 2024-09-14T06:29:00.001 INFO:teuthology.orchestra.run.smithi094.stdout: "heartbeat_back_addr": "172.21.15.138:6833/1599292735", 2024-09-14T06:29:00.001 INFO:teuthology.orchestra.run.smithi094.stdout: "heartbeat_front_addr": "172.21.15.138:6831/1599292735", 2024-09-14T06:29:00.001 INFO:teuthology.orchestra.run.smithi094.stdout: "state": [ 2024-09-14T06:29:00.001 INFO:teuthology.orchestra.run.smithi094.stdout: "exists", 2024-09-14T06:29:00.001 INFO:teuthology.orchestra.run.smithi094.stdout: "up" 2024-09-14T06:29:00.001 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:00.001 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:00.001 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:00.001 INFO:teuthology.orchestra.run.smithi094.stdout: "osd": 6, 2024-09-14T06:29:00.001 INFO:teuthology.orchestra.run.smithi094.stdout: "uuid": "e55d11fd-433d-4462-93a7-d5e84560221b", 2024-09-14T06:29:00.001 INFO:teuthology.orchestra.run.smithi094.stdout: "up": 1, 2024-09-14T06:29:00.002 INFO:teuthology.orchestra.run.smithi094.stdout: "in": 1, 2024-09-14T06:29:00.002 INFO:teuthology.orchestra.run.smithi094.stdout: "weight": 1, 2024-09-14T06:29:00.002 INFO:teuthology.orchestra.run.smithi094.stdout: "primary_affinity": 1, 2024-09-14T06:29:00.002 INFO:teuthology.orchestra.run.smithi094.stdout: "last_clean_begin": 0, 2024-09-14T06:29:00.002 INFO:teuthology.orchestra.run.smithi094.stdout: "last_clean_end": 0, 2024-09-14T06:29:00.002 INFO:teuthology.orchestra.run.smithi094.stdout: "up_from": 15, 2024-09-14T06:29:00.002 INFO:teuthology.orchestra.run.smithi094.stdout: "up_thru": 19, 2024-09-14T06:29:00.002 INFO:teuthology.orchestra.run.smithi094.stdout: "down_at": 0, 2024-09-14T06:29:00.002 INFO:teuthology.orchestra.run.smithi094.stdout: "lost_at": 0, 2024-09-14T06:29:00.002 INFO:teuthology.orchestra.run.smithi094.stdout: "public_addrs": { 2024-09-14T06:29:00.002 INFO:teuthology.orchestra.run.smithi094.stdout: "addrvec": [ 2024-09-14T06:29:00.002 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:00.002 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v2", 2024-09-14T06:29:00.002 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.138:6818", 2024-09-14T06:29:00.003 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 163531479 2024-09-14T06:29:00.003 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:00.003 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:00.003 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v1", 2024-09-14T06:29:00.003 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.138:6819", 2024-09-14T06:29:00.003 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 163531479 2024-09-14T06:29:00.003 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:00.003 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:00.003 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:00.003 INFO:teuthology.orchestra.run.smithi094.stdout: "cluster_addrs": { 2024-09-14T06:29:00.004 INFO:teuthology.orchestra.run.smithi094.stdout: "addrvec": [ 2024-09-14T06:29:00.004 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:00.004 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v2", 2024-09-14T06:29:00.004 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.138:6820", 2024-09-14T06:29:00.004 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 163531479 2024-09-14T06:29:00.004 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:00.004 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:00.004 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v1", 2024-09-14T06:29:00.004 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.138:6821", 2024-09-14T06:29:00.004 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 163531479 2024-09-14T06:29:00.004 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:00.004 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:00.004 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:00.004 INFO:teuthology.orchestra.run.smithi094.stdout: "heartbeat_back_addrs": { 2024-09-14T06:29:00.005 INFO:teuthology.orchestra.run.smithi094.stdout: "addrvec": [ 2024-09-14T06:29:00.005 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:00.005 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v2", 2024-09-14T06:29:00.005 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.138:6824", 2024-09-14T06:29:00.005 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 163531479 2024-09-14T06:29:00.005 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:00.005 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:00.005 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v1", 2024-09-14T06:29:00.005 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.138:6825", 2024-09-14T06:29:00.005 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 163531479 2024-09-14T06:29:00.005 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:00.005 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:00.005 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:00.005 INFO:teuthology.orchestra.run.smithi094.stdout: "heartbeat_front_addrs": { 2024-09-14T06:29:00.006 INFO:teuthology.orchestra.run.smithi094.stdout: "addrvec": [ 2024-09-14T06:29:00.006 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:00.006 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v2", 2024-09-14T06:29:00.006 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.138:6822", 2024-09-14T06:29:00.006 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 163531479 2024-09-14T06:29:00.006 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:00.006 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:00.006 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v1", 2024-09-14T06:29:00.006 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.138:6823", 2024-09-14T06:29:00.006 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 163531479 2024-09-14T06:29:00.006 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:00.006 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:00.006 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:00.006 INFO:teuthology.orchestra.run.smithi094.stdout: "public_addr": "172.21.15.138:6819/163531479", 2024-09-14T06:29:00.007 INFO:teuthology.orchestra.run.smithi094.stdout: "cluster_addr": "172.21.15.138:6821/163531479", 2024-09-14T06:29:00.007 INFO:teuthology.orchestra.run.smithi094.stdout: "heartbeat_back_addr": "172.21.15.138:6825/163531479", 2024-09-14T06:29:00.007 INFO:teuthology.orchestra.run.smithi094.stdout: "heartbeat_front_addr": "172.21.15.138:6823/163531479", 2024-09-14T06:29:00.007 INFO:teuthology.orchestra.run.smithi094.stdout: "state": [ 2024-09-14T06:29:00.007 INFO:teuthology.orchestra.run.smithi094.stdout: "exists", 2024-09-14T06:29:00.007 INFO:teuthology.orchestra.run.smithi094.stdout: "up" 2024-09-14T06:29:00.007 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:00.007 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:00.007 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:00.007 INFO:teuthology.orchestra.run.smithi094.stdout: "osd": 7, 2024-09-14T06:29:00.007 INFO:teuthology.orchestra.run.smithi094.stdout: "uuid": "ba14ba69-2c6c-4d09-81e2-2593dc3a7ea9", 2024-09-14T06:29:00.007 INFO:teuthology.orchestra.run.smithi094.stdout: "up": 1, 2024-09-14T06:29:00.007 INFO:teuthology.orchestra.run.smithi094.stdout: "in": 1, 2024-09-14T06:29:00.007 INFO:teuthology.orchestra.run.smithi094.stdout: "weight": 1, 2024-09-14T06:29:00.008 INFO:teuthology.orchestra.run.smithi094.stdout: "primary_affinity": 1, 2024-09-14T06:29:00.008 INFO:teuthology.orchestra.run.smithi094.stdout: "last_clean_begin": 0, 2024-09-14T06:29:00.008 INFO:teuthology.orchestra.run.smithi094.stdout: "last_clean_end": 0, 2024-09-14T06:29:00.008 INFO:teuthology.orchestra.run.smithi094.stdout: "up_from": 15, 2024-09-14T06:29:00.008 INFO:teuthology.orchestra.run.smithi094.stdout: "up_thru": 19, 2024-09-14T06:29:00.008 INFO:teuthology.orchestra.run.smithi094.stdout: "down_at": 0, 2024-09-14T06:29:00.008 INFO:teuthology.orchestra.run.smithi094.stdout: "lost_at": 0, 2024-09-14T06:29:00.008 INFO:teuthology.orchestra.run.smithi094.stdout: "public_addrs": { 2024-09-14T06:29:00.008 INFO:teuthology.orchestra.run.smithi094.stdout: "addrvec": [ 2024-09-14T06:29:00.008 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:00.008 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v2", 2024-09-14T06:29:00.009 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.138:6800", 2024-09-14T06:29:00.009 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 218507235 2024-09-14T06:29:00.009 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:00.009 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:00.009 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v1", 2024-09-14T06:29:00.009 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.138:6801", 2024-09-14T06:29:00.009 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 218507235 2024-09-14T06:29:00.009 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:00.009 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:00.009 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:00.009 INFO:teuthology.orchestra.run.smithi094.stdout: "cluster_addrs": { 2024-09-14T06:29:00.009 INFO:teuthology.orchestra.run.smithi094.stdout: "addrvec": [ 2024-09-14T06:29:00.009 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:00.010 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v2", 2024-09-14T06:29:00.010 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.138:6802", 2024-09-14T06:29:00.010 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 218507235 2024-09-14T06:29:00.010 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:00.010 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:00.010 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v1", 2024-09-14T06:29:00.010 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.138:6803", 2024-09-14T06:29:00.010 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 218507235 2024-09-14T06:29:00.010 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:00.010 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:00.010 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:00.010 INFO:teuthology.orchestra.run.smithi094.stdout: "heartbeat_back_addrs": { 2024-09-14T06:29:00.010 INFO:teuthology.orchestra.run.smithi094.stdout: "addrvec": [ 2024-09-14T06:29:00.010 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:00.011 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v2", 2024-09-14T06:29:00.011 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.138:6806", 2024-09-14T06:29:00.011 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 218507235 2024-09-14T06:29:00.011 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:00.011 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:00.011 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v1", 2024-09-14T06:29:00.011 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.138:6807", 2024-09-14T06:29:00.011 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 218507235 2024-09-14T06:29:00.011 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:00.011 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:00.011 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:00.011 INFO:teuthology.orchestra.run.smithi094.stdout: "heartbeat_front_addrs": { 2024-09-14T06:29:00.011 INFO:teuthology.orchestra.run.smithi094.stdout: "addrvec": [ 2024-09-14T06:29:00.012 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:00.012 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v2", 2024-09-14T06:29:00.012 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.138:6804", 2024-09-14T06:29:00.012 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 218507235 2024-09-14T06:29:00.012 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:00.012 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:00.012 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v1", 2024-09-14T06:29:00.012 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.138:6805", 2024-09-14T06:29:00.012 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 218507235 2024-09-14T06:29:00.012 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:00.012 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:00.012 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:00.012 INFO:teuthology.orchestra.run.smithi094.stdout: "public_addr": "172.21.15.138:6801/218507235", 2024-09-14T06:29:00.012 INFO:teuthology.orchestra.run.smithi094.stdout: "cluster_addr": "172.21.15.138:6803/218507235", 2024-09-14T06:29:00.013 INFO:teuthology.orchestra.run.smithi094.stdout: "heartbeat_back_addr": "172.21.15.138:6807/218507235", 2024-09-14T06:29:00.013 INFO:teuthology.orchestra.run.smithi094.stdout: "heartbeat_front_addr": "172.21.15.138:6805/218507235", 2024-09-14T06:29:00.013 INFO:teuthology.orchestra.run.smithi094.stdout: "state": [ 2024-09-14T06:29:00.013 INFO:teuthology.orchestra.run.smithi094.stdout: "exists", 2024-09-14T06:29:00.013 INFO:teuthology.orchestra.run.smithi094.stdout: "up" 2024-09-14T06:29:00.013 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:00.013 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:00.013 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-09-14T06:29:00.013 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_xinfo": [ 2024-09-14T06:29:00.013 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:00.013 INFO:teuthology.orchestra.run.smithi094.stdout: "osd": 0, 2024-09-14T06:29:00.013 INFO:teuthology.orchestra.run.smithi094.stdout: "down_stamp": "0.000000", 2024-09-14T06:29:00.013 INFO:teuthology.orchestra.run.smithi094.stdout: "laggy_probability": 0, 2024-09-14T06:29:00.014 INFO:teuthology.orchestra.run.smithi094.stdout: "laggy_interval": 0, 2024-09-14T06:29:00.014 INFO:teuthology.orchestra.run.smithi094.stdout: "features": 4540719139924082687, 2024-09-14T06:29:00.014 INFO:teuthology.orchestra.run.smithi094.stdout: "old_weight": 0, 2024-09-14T06:29:00.014 INFO:teuthology.orchestra.run.smithi094.stdout: "last_purged_snaps_scrub": "2024-09-14T06:28:39.906639+0000", 2024-09-14T06:29:00.014 INFO:teuthology.orchestra.run.smithi094.stdout: "dead_epoch": 0 2024-09-14T06:29:00.014 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:00.014 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:00.014 INFO:teuthology.orchestra.run.smithi094.stdout: "osd": 1, 2024-09-14T06:29:00.014 INFO:teuthology.orchestra.run.smithi094.stdout: "down_stamp": "0.000000", 2024-09-14T06:29:00.015 INFO:teuthology.orchestra.run.smithi094.stdout: "laggy_probability": 0, 2024-09-14T06:29:00.015 INFO:teuthology.orchestra.run.smithi094.stdout: "laggy_interval": 0, 2024-09-14T06:29:00.015 INFO:teuthology.orchestra.run.smithi094.stdout: "features": 4540719139924082687, 2024-09-14T06:29:00.015 INFO:teuthology.orchestra.run.smithi094.stdout: "old_weight": 0, 2024-09-14T06:29:00.015 INFO:teuthology.orchestra.run.smithi094.stdout: "last_purged_snaps_scrub": "2024-09-14T06:28:40.541279+0000", 2024-09-14T06:29:00.015 INFO:teuthology.orchestra.run.smithi094.stdout: "dead_epoch": 0 2024-09-14T06:29:00.015 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:00.015 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:00.015 INFO:teuthology.orchestra.run.smithi094.stdout: "osd": 2, 2024-09-14T06:29:00.015 INFO:teuthology.orchestra.run.smithi094.stdout: "down_stamp": "0.000000", 2024-09-14T06:29:00.015 INFO:teuthology.orchestra.run.smithi094.stdout: "laggy_probability": 0, 2024-09-14T06:29:00.015 INFO:teuthology.orchestra.run.smithi094.stdout: "laggy_interval": 0, 2024-09-14T06:29:00.015 INFO:teuthology.orchestra.run.smithi094.stdout: "features": 4540719139924082687, 2024-09-14T06:29:00.015 INFO:teuthology.orchestra.run.smithi094.stdout: "old_weight": 0, 2024-09-14T06:29:00.016 INFO:teuthology.orchestra.run.smithi094.stdout: "last_purged_snaps_scrub": "2024-09-14T06:28:40.025350+0000", 2024-09-14T06:29:00.016 INFO:teuthology.orchestra.run.smithi094.stdout: "dead_epoch": 0 2024-09-14T06:29:00.016 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:00.016 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:00.016 INFO:teuthology.orchestra.run.smithi094.stdout: "osd": 3, 2024-09-14T06:29:00.016 INFO:teuthology.orchestra.run.smithi094.stdout: "down_stamp": "0.000000", 2024-09-14T06:29:00.016 INFO:teuthology.orchestra.run.smithi094.stdout: "laggy_probability": 0, 2024-09-14T06:29:00.016 INFO:teuthology.orchestra.run.smithi094.stdout: "laggy_interval": 0, 2024-09-14T06:29:00.016 INFO:teuthology.orchestra.run.smithi094.stdout: "features": 4540719139924082687, 2024-09-14T06:29:00.016 INFO:teuthology.orchestra.run.smithi094.stdout: "old_weight": 0, 2024-09-14T06:29:00.016 INFO:teuthology.orchestra.run.smithi094.stdout: "last_purged_snaps_scrub": "2024-09-14T06:28:39.217944+0000", 2024-09-14T06:29:00.016 INFO:teuthology.orchestra.run.smithi094.stdout: "dead_epoch": 0 2024-09-14T06:29:00.016 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:00.016 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:00.017 INFO:teuthology.orchestra.run.smithi094.stdout: "osd": 4, 2024-09-14T06:29:00.017 INFO:teuthology.orchestra.run.smithi094.stdout: "down_stamp": "0.000000", 2024-09-14T06:29:00.017 INFO:teuthology.orchestra.run.smithi094.stdout: "laggy_probability": 0, 2024-09-14T06:29:00.017 INFO:teuthology.orchestra.run.smithi094.stdout: "laggy_interval": 0, 2024-09-14T06:29:00.017 INFO:teuthology.orchestra.run.smithi094.stdout: "features": 4540719139924082687, 2024-09-14T06:29:00.017 INFO:teuthology.orchestra.run.smithi094.stdout: "old_weight": 0, 2024-09-14T06:29:00.017 INFO:teuthology.orchestra.run.smithi094.stdout: "last_purged_snaps_scrub": "2024-09-14T06:28:40.185898+0000", 2024-09-14T06:29:00.017 INFO:teuthology.orchestra.run.smithi094.stdout: "dead_epoch": 0 2024-09-14T06:29:00.017 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:00.017 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:00.017 INFO:teuthology.orchestra.run.smithi094.stdout: "osd": 5, 2024-09-14T06:29:00.017 INFO:teuthology.orchestra.run.smithi094.stdout: "down_stamp": "0.000000", 2024-09-14T06:29:00.017 INFO:teuthology.orchestra.run.smithi094.stdout: "laggy_probability": 0, 2024-09-14T06:29:00.017 INFO:teuthology.orchestra.run.smithi094.stdout: "laggy_interval": 0, 2024-09-14T06:29:00.018 INFO:teuthology.orchestra.run.smithi094.stdout: "features": 4540719139924082687, 2024-09-14T06:29:00.018 INFO:teuthology.orchestra.run.smithi094.stdout: "old_weight": 0, 2024-09-14T06:29:00.018 INFO:teuthology.orchestra.run.smithi094.stdout: "last_purged_snaps_scrub": "2024-09-14T06:28:40.783504+0000", 2024-09-14T06:29:00.018 INFO:teuthology.orchestra.run.smithi094.stdout: "dead_epoch": 0 2024-09-14T06:29:00.018 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:00.018 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:00.018 INFO:teuthology.orchestra.run.smithi094.stdout: "osd": 6, 2024-09-14T06:29:00.018 INFO:teuthology.orchestra.run.smithi094.stdout: "down_stamp": "0.000000", 2024-09-14T06:29:00.018 INFO:teuthology.orchestra.run.smithi094.stdout: "laggy_probability": 0, 2024-09-14T06:29:00.019 INFO:teuthology.orchestra.run.smithi094.stdout: "laggy_interval": 0, 2024-09-14T06:29:00.019 INFO:teuthology.orchestra.run.smithi094.stdout: "features": 4540719139924082687, 2024-09-14T06:29:00.019 INFO:teuthology.orchestra.run.smithi094.stdout: "old_weight": 0, 2024-09-14T06:29:00.019 INFO:teuthology.orchestra.run.smithi094.stdout: "last_purged_snaps_scrub": "2024-09-14T06:28:40.527305+0000", 2024-09-14T06:29:00.019 INFO:teuthology.orchestra.run.smithi094.stdout: "dead_epoch": 0 2024-09-14T06:29:00.019 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:00.019 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:00.019 INFO:teuthology.orchestra.run.smithi094.stdout: "osd": 7, 2024-09-14T06:29:00.019 INFO:teuthology.orchestra.run.smithi094.stdout: "down_stamp": "0.000000", 2024-09-14T06:29:00.019 INFO:teuthology.orchestra.run.smithi094.stdout: "laggy_probability": 0, 2024-09-14T06:29:00.019 INFO:teuthology.orchestra.run.smithi094.stdout: "laggy_interval": 0, 2024-09-14T06:29:00.020 INFO:teuthology.orchestra.run.smithi094.stdout: "features": 4540719139924082687, 2024-09-14T06:29:00.020 INFO:teuthology.orchestra.run.smithi094.stdout: "old_weight": 0, 2024-09-14T06:29:00.020 INFO:teuthology.orchestra.run.smithi094.stdout: "last_purged_snaps_scrub": "2024-09-14T06:28:40.349676+0000", 2024-09-14T06:29:00.020 INFO:teuthology.orchestra.run.smithi094.stdout: "dead_epoch": 0 2024-09-14T06:29:00.020 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:00.020 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-09-14T06:29:00.020 INFO:teuthology.orchestra.run.smithi094.stdout: "pg_upmap": [], 2024-09-14T06:29:00.020 INFO:teuthology.orchestra.run.smithi094.stdout: "pg_upmap_items": [], 2024-09-14T06:29:00.020 INFO:teuthology.orchestra.run.smithi094.stdout: "pg_upmap_primaries": [], 2024-09-14T06:29:00.020 INFO:teuthology.orchestra.run.smithi094.stdout: "pg_temp": [], 2024-09-14T06:29:00.020 INFO:teuthology.orchestra.run.smithi094.stdout: "primary_temp": [], 2024-09-14T06:29:00.020 INFO:teuthology.orchestra.run.smithi094.stdout: "blocklist": {}, 2024-09-14T06:29:00.020 INFO:teuthology.orchestra.run.smithi094.stdout: "range_blocklist": {}, 2024-09-14T06:29:00.021 INFO:teuthology.orchestra.run.smithi094.stdout: "erasure_code_profiles": { 2024-09-14T06:29:00.021 INFO:teuthology.orchestra.run.smithi094.stdout: "default": { 2024-09-14T06:29:00.021 INFO:teuthology.orchestra.run.smithi094.stdout: "crush-failure-domain": "osd", 2024-09-14T06:29:00.021 INFO:teuthology.orchestra.run.smithi094.stdout: "k": "2", 2024-09-14T06:29:00.021 INFO:teuthology.orchestra.run.smithi094.stdout: "m": "1", 2024-09-14T06:29:00.021 INFO:teuthology.orchestra.run.smithi094.stdout: "plugin": "jerasure", 2024-09-14T06:29:00.021 INFO:teuthology.orchestra.run.smithi094.stdout: "technique": "reed_sol_van" 2024-09-14T06:29:00.021 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:00.021 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:00.021 INFO:teuthology.orchestra.run.smithi094.stdout: "removed_snaps_queue": [ 2024-09-14T06:29:00.021 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:00.021 INFO:teuthology.orchestra.run.smithi094.stdout: "pool": 2, 2024-09-14T06:29:00.021 INFO:teuthology.orchestra.run.smithi094.stdout: "snaps": [ 2024-09-14T06:29:00.021 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:00.022 INFO:teuthology.orchestra.run.smithi094.stdout: "begin": 2, 2024-09-14T06:29:00.022 INFO:teuthology.orchestra.run.smithi094.stdout: "length": 1 2024-09-14T06:29:00.022 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:00.022 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:00.022 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:00.022 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-09-14T06:29:00.022 INFO:teuthology.orchestra.run.smithi094.stdout: "new_removed_snaps": [ 2024-09-14T06:29:00.022 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:00.022 INFO:teuthology.orchestra.run.smithi094.stdout: "pool": 2, 2024-09-14T06:29:00.022 INFO:teuthology.orchestra.run.smithi094.stdout: "snaps": [ 2024-09-14T06:29:00.022 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:00.022 INFO:teuthology.orchestra.run.smithi094.stdout: "begin": 2, 2024-09-14T06:29:00.022 INFO:teuthology.orchestra.run.smithi094.stdout: "length": 1 2024-09-14T06:29:00.022 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:00.022 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:00.023 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:00.023 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-09-14T06:29:00.023 INFO:teuthology.orchestra.run.smithi094.stdout: "new_purged_snaps": [], 2024-09-14T06:29:00.023 INFO:teuthology.orchestra.run.smithi094.stdout: "crush_node_flags": {}, 2024-09-14T06:29:00.023 INFO:teuthology.orchestra.run.smithi094.stdout: "device_class_flags": {}, 2024-09-14T06:29:00.023 INFO:teuthology.orchestra.run.smithi094.stdout: "stretch_mode": { 2024-09-14T06:29:00.023 INFO:teuthology.orchestra.run.smithi094.stdout: "stretch_mode_enabled": false, 2024-09-14T06:29:00.023 INFO:teuthology.orchestra.run.smithi094.stdout: "stretch_bucket_count": 0, 2024-09-14T06:29:00.023 INFO:teuthology.orchestra.run.smithi094.stdout: "degraded_stretch_mode": 0, 2024-09-14T06:29:00.023 INFO:teuthology.orchestra.run.smithi094.stdout: "recovering_stretch_mode": 0, 2024-09-14T06:29:00.023 INFO:teuthology.orchestra.run.smithi094.stdout: "stretch_mode_bucket": 0 2024-09-14T06:29:00.023 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:00.023 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-14T06:29:00.026 INFO:tasks.thrashosds.thrasher:j is {'epoch': 22, 'fsid': '200ad3da-bd63-4e04-af74-efb5ca0af75e', 'created': '2024-09-14T06:28:30.782046+0000', 'modified': '2024-09-14T06:28:53.222080+0000', 'last_up_change': '2024-09-14T06:28:42.827239+0000', 'last_in_change': '2024-09-14T06:28:33.724176+0000', 'flags': 'sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit', 'flags_num': 5799936, 'flags_set': ['pglog_hardlimit', 'purged_snapdirs', 'recovery_deletes', 'sortbitwise'], 'crush_version': 5, 'full_ratio': 0.949999988079071, 'backfillfull_ratio': 0.8999999761581421, 'nearfull_ratio': 0.8500000238418579, 'cluster_snapshot': '', 'pool_max': 2, 'max_osd': 8, 'require_min_compat_client': 'luminous', 'min_compat_client': 'jewel', 'require_osd_release': 'squid', 'allow_crimson': False, 'pools': [{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-14T06:28:43.295990+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2024-09-14T06:28:49.451714+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 2, 'snap_epoch': 22, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.9900000095367432, 'score_stable': 1.9900000095367432, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}], 'osds': [{'osd': 0, 'uuid': '8b22a249-b842-406a-a02c-c0f31f57b635', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 0, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.94:6824', 'nonce': 1809564923}, {'type': 'v1', 'addr': '172.21.15.94:6825', 'nonce': 1809564923}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.94:6826', 'nonce': 1809564923}, {'type': 'v1', 'addr': '172.21.15.94:6827', 'nonce': 1809564923}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.94:6830', 'nonce': 1809564923}, {'type': 'v1', 'addr': '172.21.15.94:6831', 'nonce': 1809564923}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.94:6828', 'nonce': 1809564923}, {'type': 'v1', 'addr': '172.21.15.94:6829', 'nonce': 1809564923}]}, 'public_addr': '172.21.15.94:6825/1809564923', 'cluster_addr': '172.21.15.94:6827/1809564923', 'heartbeat_back_addr': '172.21.15.94:6831/1809564923', 'heartbeat_front_addr': '172.21.15.94:6829/1809564923', 'state': ['exists', 'up']}, {'osd': 1, 'uuid': '02251a03-84a0-414d-a980-fac4fd043cb6', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 19, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.94:6816', 'nonce': 736402652}, {'type': 'v1', 'addr': '172.21.15.94:6817', 'nonce': 736402652}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.94:6818', 'nonce': 736402652}, {'type': 'v1', 'addr': '172.21.15.94:6819', 'nonce': 736402652}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.94:6822', 'nonce': 736402652}, {'type': 'v1', 'addr': '172.21.15.94:6823', 'nonce': 736402652}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.94:6820', 'nonce': 736402652}, {'type': 'v1', 'addr': '172.21.15.94:6821', 'nonce': 736402652}]}, 'public_addr': '172.21.15.94:6817/736402652', 'cluster_addr': '172.21.15.94:6819/736402652', 'heartbeat_back_addr': '172.21.15.94:6823/736402652', 'heartbeat_front_addr': '172.21.15.94:6821/736402652', 'state': ['exists', 'up']}, {'osd': 2, 'uuid': 'd45bf470-bb29-4739-8294-cd122b2b4102', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 19, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.94:6808', 'nonce': 1063204782}, {'type': 'v1', 'addr': '172.21.15.94:6809', 'nonce': 1063204782}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.94:6810', 'nonce': 1063204782}, {'type': 'v1', 'addr': '172.21.15.94:6811', 'nonce': 1063204782}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.94:6814', 'nonce': 1063204782}, {'type': 'v1', 'addr': '172.21.15.94:6815', 'nonce': 1063204782}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.94:6812', 'nonce': 1063204782}, {'type': 'v1', 'addr': '172.21.15.94:6813', 'nonce': 1063204782}]}, 'public_addr': '172.21.15.94:6809/1063204782', 'cluster_addr': '172.21.15.94:6811/1063204782', 'heartbeat_back_addr': '172.21.15.94:6815/1063204782', 'heartbeat_front_addr': '172.21.15.94:6813/1063204782', 'state': ['exists', 'up']}, {'osd': 3, 'uuid': '456ec2d9-c438-4243-b931-335704f661eb', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 0, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.94:6800', 'nonce': 3382046754}, {'type': 'v1', 'addr': '172.21.15.94:6801', 'nonce': 3382046754}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.94:6802', 'nonce': 3382046754}, {'type': 'v1', 'addr': '172.21.15.94:6803', 'nonce': 3382046754}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.94:6806', 'nonce': 3382046754}, {'type': 'v1', 'addr': '172.21.15.94:6807', 'nonce': 3382046754}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.94:6804', 'nonce': 3382046754}, {'type': 'v1', 'addr': '172.21.15.94:6805', 'nonce': 3382046754}]}, 'public_addr': '172.21.15.94:6801/3382046754', 'cluster_addr': '172.21.15.94:6803/3382046754', 'heartbeat_back_addr': '172.21.15.94:6807/3382046754', 'heartbeat_front_addr': '172.21.15.94:6805/3382046754', 'state': ['exists', 'up']}, {'osd': 4, 'uuid': '2dab948e-c6e4-4437-bf17-017d1ac6821c', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 0, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.138:6810', 'nonce': 501726729}, {'type': 'v1', 'addr': '172.21.15.138:6811', 'nonce': 501726729}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.138:6812', 'nonce': 501726729}, {'type': 'v1', 'addr': '172.21.15.138:6813', 'nonce': 501726729}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.138:6816', 'nonce': 501726729}, {'type': 'v1', 'addr': '172.21.15.138:6817', 'nonce': 501726729}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.138:6814', 'nonce': 501726729}, {'type': 'v1', 'addr': '172.21.15.138:6815', 'nonce': 501726729}]}, 'public_addr': '172.21.15.138:6811/501726729', 'cluster_addr': '172.21.15.138:6813/501726729', 'heartbeat_back_addr': '172.21.15.138:6817/501726729', 'heartbeat_front_addr': '172.21.15.138:6815/501726729', 'state': ['exists', 'up']}, {'osd': 5, 'uuid': '8caf3aa3-ad87-4d33-b82e-4c15195176fd', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 19, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.138:6826', 'nonce': 1599292735}, {'type': 'v1', 'addr': '172.21.15.138:6827', 'nonce': 1599292735}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.138:6828', 'nonce': 1599292735}, {'type': 'v1', 'addr': '172.21.15.138:6829', 'nonce': 1599292735}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.138:6832', 'nonce': 1599292735}, {'type': 'v1', 'addr': '172.21.15.138:6833', 'nonce': 1599292735}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.138:6830', 'nonce': 1599292735}, {'type': 'v1', 'addr': '172.21.15.138:6831', 'nonce': 1599292735}]}, 'public_addr': '172.21.15.138:6827/1599292735', 'cluster_addr': '172.21.15.138:6829/1599292735', 'heartbeat_back_addr': '172.21.15.138:6833/1599292735', 'heartbeat_front_addr': '172.21.15.138:6831/1599292735', 'state': ['exists', 'up']}, {'osd': 6, 'uuid': 'e55d11fd-433d-4462-93a7-d5e84560221b', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 19, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.138:6818', 'nonce': 163531479}, {'type': 'v1', 'addr': '172.21.15.138:6819', 'nonce': 163531479}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.138:6820', 'nonce': 163531479}, {'type': 'v1', 'addr': '172.21.15.138:6821', 'nonce': 163531479}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.138:6824', 'nonce': 163531479}, {'type': 'v1', 'addr': '172.21.15.138:6825', 'nonce': 163531479}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.138:6822', 'nonce': 163531479}, {'type': 'v1', 'addr': '172.21.15.138:6823', 'nonce': 163531479}]}, 'public_addr': '172.21.15.138:6819/163531479', 'cluster_addr': '172.21.15.138:6821/163531479', 'heartbeat_back_addr': '172.21.15.138:6825/163531479', 'heartbeat_front_addr': '172.21.15.138:6823/163531479', 'state': ['exists', 'up']}, {'osd': 7, 'uuid': 'ba14ba69-2c6c-4d09-81e2-2593dc3a7ea9', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 19, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.138:6800', 'nonce': 218507235}, {'type': 'v1', 'addr': '172.21.15.138:6801', 'nonce': 218507235}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.138:6802', 'nonce': 218507235}, {'type': 'v1', 'addr': '172.21.15.138:6803', 'nonce': 218507235}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.138:6806', 'nonce': 218507235}, {'type': 'v1', 'addr': '172.21.15.138:6807', 'nonce': 218507235}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.138:6804', 'nonce': 218507235}, {'type': 'v1', 'addr': '172.21.15.138:6805', 'nonce': 218507235}]}, 'public_addr': '172.21.15.138:6801/218507235', 'cluster_addr': '172.21.15.138:6803/218507235', 'heartbeat_back_addr': '172.21.15.138:6807/218507235', 'heartbeat_front_addr': '172.21.15.138:6805/218507235', 'state': ['exists', 'up']}], 'osd_xinfo': [{'osd': 0, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-14T06:28:39.906639+0000', 'dead_epoch': 0}, {'osd': 1, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-14T06:28:40.541279+0000', 'dead_epoch': 0}, {'osd': 2, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-14T06:28:40.025350+0000', 'dead_epoch': 0}, {'osd': 3, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-14T06:28:39.217944+0000', 'dead_epoch': 0}, {'osd': 4, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-14T06:28:40.185898+0000', 'dead_epoch': 0}, {'osd': 5, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-14T06:28:40.783504+0000', 'dead_epoch': 0}, {'osd': 6, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-14T06:28:40.527305+0000', 'dead_epoch': 0}, {'osd': 7, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-14T06:28:40.349676+0000', 'dead_epoch': 0}], 'pg_upmap': [], 'pg_upmap_items': [], 'pg_upmap_primaries': [], 'pg_temp': [], 'primary_temp': [], 'blocklist': {}, 'range_blocklist': {}, 'erasure_code_profiles': {'default': {'crush-failure-domain': 'osd', 'k': '2', 'm': '1', 'plugin': 'jerasure', 'technique': 'reed_sol_van'}}, 'removed_snaps_queue': [{'pool': 2, 'snaps': [{'begin': 2, 'length': 1}]}], 'new_removed_snaps': [{'pool': 2, 'snaps': [{'begin': 2, 'length': 1}]}], 'new_purged_snaps': [], 'crush_node_flags': {}, 'device_class_flags': {}, 'stretch_mode': {'stretch_mode_enabled': False, 'stretch_bucket_count': 0, 'degraded_stretch_mode': 0, 'recovering_stretch_mode': 0, 'stretch_mode_bucket': 0}} 2024-09-14T06:29:00.026 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-14T06:29:00.027 INFO:tasks.ceph.osd.2.smithi094.stderr:2024-09-14T06:29:00.024+0000 7faf3269b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33649) UID: 0 2024-09-14T06:29:00.042 INFO:teuthology.orchestra.run.smithi094.stdout:ERROR: (22) Invalid argument 2024-09-14T06:29:00.043 INFO:teuthology.orchestra.run.smithi094.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:29:00.052 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-14T06:29:00.125 INFO:tasks.ceph.osd.4.smithi138.stderr:2024-09-14T06:29:00.125+0000 7f6c653c8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33615) UID: 0 2024-09-14T06:29:00.197 INFO:teuthology.orchestra.run.smithi094.stdout:ERROR: (22) Invalid argument 2024-09-14T06:29:00.197 INFO:teuthology.orchestra.run.smithi094.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:29:00.209 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-14T06:29:00.226 INFO:tasks.ceph.osd.5.smithi138.stderr:2024-09-14T06:29:00.225+0000 7fe13746f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33618) UID: 0 2024-09-14T06:29:00.281 INFO:teuthology.orchestra.run.smithi094.stderr:noscrub is set 2024-09-14T06:29:00.321 INFO:teuthology.orchestra.run.smithi094.stdout:ERROR: (22) Invalid argument 2024-09-14T06:29:00.321 INFO:teuthology.orchestra.run.smithi094.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:29:00.328 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-14T06:29:00.328 INFO:tasks.ceph.osd.6.smithi138.stderr:2024-09-14T06:29:00.327+0000 7f8cd8849640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33620) UID: 0 2024-09-14T06:29:00.343 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-14T06:29:00.344 INFO:teuthology.orchestra.run.smithi094.stderr:dumped all 2024-09-14T06:29:00.356 INFO:teuthology.orchestra.run.smithi094.stdout:{"pg_ready":true,"pg_map":{"version":26,"stamp":"2024-09-14T06:29:00.281680+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":103,"num_write_kb":1630,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":75,"ondisk_log_size":75,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":7,"kb":754974720,"kb_used":234764,"kb_used_data":1864,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754739956,"statfs":{"total":773094113280,"available":772853714944,"internally_reserved":0,"allocated":1908736,"data_stored":1781462,"data_compressed":65166,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":12751,"internal_metadata":238276145},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.058775"},"pg_stats":[{"pgid":"3.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-14T06:29:00.276180+0000","last_change":"2024-09-14T06:29:00.276180+0000","last_active":"2024-09-14T06:29:00.276180+0000","last_peered":"2024-09-14T06:29:00.276180+0000","last_clean":"2024-09-14T06:29:00.276180+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-14T06:29:00.276180+0000","last_undegraded":"2024-09-14T06:29:00.276180+0000","last_fullsized":"2024-09-14T06:29:00.276180+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-14T06:29:00.276180+0000","last_change":"2024-09-14T06:29:00.276180+0000","last_active":"2024-09-14T06:29:00.276180+0000","last_peered":"2024-09-14T06:29:00.276180+0000","last_clean":"2024-09-14T06:29:00.276180+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-14T06:29:00.276180+0000","last_undegraded":"2024-09-14T06:29:00.276180+0000","last_fullsized":"2024-09-14T06:29:00.276180+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-14T06:29:00.276180+0000","last_change":"2024-09-14T06:29:00.276180+0000","last_active":"2024-09-14T06:29:00.276180+0000","last_peered":"2024-09-14T06:29:00.276180+0000","last_clean":"2024-09-14T06:29:00.276180+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-14T06:29:00.276180+0000","last_undegraded":"2024-09-14T06:29:00.276180+0000","last_fullsized":"2024-09-14T06:29:00.276180+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-14T06:29:00.276180+0000","last_change":"2024-09-14T06:29:00.276180+0000","last_active":"2024-09-14T06:29:00.276180+0000","last_peered":"2024-09-14T06:29:00.276180+0000","last_clean":"2024-09-14T06:29:00.276180+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-14T06:29:00.276180+0000","last_undegraded":"2024-09-14T06:29:00.276180+0000","last_fullsized":"2024-09-14T06:29:00.276180+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-14T06:29:00.276180+0000","last_change":"2024-09-14T06:29:00.276180+0000","last_active":"2024-09-14T06:29:00.276180+0000","last_peered":"2024-09-14T06:29:00.276180+0000","last_clean":"2024-09-14T06:29:00.276180+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-14T06:29:00.276180+0000","last_undegraded":"2024-09-14T06:29:00.276180+0000","last_fullsized":"2024-09-14T06:29:00.276180+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-14T06:29:00.276180+0000","last_change":"2024-09-14T06:29:00.276180+0000","last_active":"2024-09-14T06:29:00.276180+0000","last_peered":"2024-09-14T06:29:00.276180+0000","last_clean":"2024-09-14T06:29:00.276180+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-14T06:29:00.276180+0000","last_undegraded":"2024-09-14T06:29:00.276180+0000","last_fullsized":"2024-09-14T06:29:00.276180+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-14T06:29:00.276180+0000","last_change":"2024-09-14T06:29:00.276180+0000","last_active":"2024-09-14T06:29:00.276180+0000","last_peered":"2024-09-14T06:29:00.276180+0000","last_clean":"2024-09-14T06:29:00.276180+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-14T06:29:00.276180+0000","last_undegraded":"2024-09-14T06:29:00.276180+0000","last_fullsized":"2024-09-14T06:29:00.276180+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-14T06:29:00.276180+0000","last_change":"2024-09-14T06:29:00.276180+0000","last_active":"2024-09-14T06:29:00.276180+0000","last_peered":"2024-09-14T06:29:00.276180+0000","last_clean":"2024-09-14T06:29:00.276180+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-14T06:29:00.276180+0000","last_undegraded":"2024-09-14T06:29:00.276180+0000","last_fullsized":"2024-09-14T06:29:00.276180+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-14T06:29:00.276180+0000","last_change":"2024-09-14T06:29:00.276180+0000","last_active":"2024-09-14T06:29:00.276180+0000","last_peered":"2024-09-14T06:29:00.276180+0000","last_clean":"2024-09-14T06:29:00.276180+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-14T06:29:00.276180+0000","last_undegraded":"2024-09-14T06:29:00.276180+0000","last_fullsized":"2024-09-14T06:29:00.276180+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-14T06:29:00.276180+0000","last_change":"2024-09-14T06:29:00.276180+0000","last_active":"2024-09-14T06:29:00.276180+0000","last_peered":"2024-09-14T06:29:00.276180+0000","last_clean":"2024-09-14T06:29:00.276180+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-14T06:29:00.276180+0000","last_undegraded":"2024-09-14T06:29:00.276180+0000","last_fullsized":"2024-09-14T06:29:00.276180+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-14T06:29:00.276180+0000","last_change":"2024-09-14T06:29:00.276180+0000","last_active":"2024-09-14T06:29:00.276180+0000","last_peered":"2024-09-14T06:29:00.276180+0000","last_clean":"2024-09-14T06:29:00.276180+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-14T06:29:00.276180+0000","last_undegraded":"2024-09-14T06:29:00.276180+0000","last_fullsized":"2024-09-14T06:29:00.276180+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-14T06:29:00.276180+0000","last_change":"2024-09-14T06:29:00.276180+0000","last_active":"2024-09-14T06:29:00.276180+0000","last_peered":"2024-09-14T06:29:00.276180+0000","last_clean":"2024-09-14T06:29:00.276180+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-14T06:29:00.276180+0000","last_undegraded":"2024-09-14T06:29:00.276180+0000","last_fullsized":"2024-09-14T06:29:00.276180+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":20,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-14T06:28:53.625193+0000","last_change":"2024-09-14T06:28:53.625193+0000","last_active":"2024-09-14T06:28:53.625193+0000","last_peered":"2024-09-14T06:28:53.625193+0000","last_clean":"2024-09-14T06:28:53.625193+0000","last_became_active":"2024-09-14T06:28:51.223535+0000","last_became_peered":"2024-09-14T06:28:51.223535+0000","last_unstale":"2024-09-14T06:28:53.625193+0000","last_undegraded":"2024-09-14T06:28:53.625193+0000","last_fullsized":"2024-09-14T06:28:53.625193+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:28:50.205350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:28:50.205350+0000","last_clean_scrub_stamp":"2024-09-14T06:28:50.205350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T18:14:22.822245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00052660300000000005,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"3.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-14T06:29:00.276180+0000","last_change":"2024-09-14T06:29:00.276180+0000","last_active":"2024-09-14T06:29:00.276180+0000","last_peered":"2024-09-14T06:29:00.276180+0000","last_clean":"2024-09-14T06:29:00.276180+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-14T06:29:00.276180+0000","last_undegraded":"2024-09-14T06:29:00.276180+0000","last_fullsized":"2024-09-14T06:29:00.276180+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":20,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-14T06:28:53.226122+0000","last_change":"2024-09-14T06:28:53.226122+0000","last_active":"2024-09-14T06:28:53.226122+0000","last_peered":"2024-09-14T06:28:53.226122+0000","last_clean":"2024-09-14T06:28:53.226122+0000","last_became_active":"2024-09-14T06:28:51.217205+0000","last_became_peered":"2024-09-14T06:28:51.217205+0000","last_unstale":"2024-09-14T06:28:53.226122+0000","last_undegraded":"2024-09-14T06:28:53.226122+0000","last_fullsized":"2024-09-14T06:28:53.226122+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:28:50.205350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:28:50.205350+0000","last_clean_scrub_stamp":"2024-09-14T06:28:50.205350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T15:29:19.563302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065018200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"3.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-14T06:29:00.276180+0000","last_change":"2024-09-14T06:29:00.276180+0000","last_active":"2024-09-14T06:29:00.276180+0000","last_peered":"2024-09-14T06:29:00.276180+0000","last_clean":"2024-09-14T06:29:00.276180+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-14T06:29:00.276180+0000","last_undegraded":"2024-09-14T06:29:00.276180+0000","last_fullsized":"2024-09-14T06:29:00.276180+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":20,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-14T06:28:53.640035+0000","last_change":"2024-09-14T06:28:53.640035+0000","last_active":"2024-09-14T06:28:53.640035+0000","last_peered":"2024-09-14T06:28:53.640035+0000","last_clean":"2024-09-14T06:28:53.640035+0000","last_became_active":"2024-09-14T06:28:51.308058+0000","last_became_peered":"2024-09-14T06:28:51.308058+0000","last_unstale":"2024-09-14T06:28:53.640035+0000","last_undegraded":"2024-09-14T06:28:53.640035+0000","last_fullsized":"2024-09-14T06:28:53.640035+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:28:50.205350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:28:50.205350+0000","last_clean_scrub_stamp":"2024-09-14T06:28:50.205350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T13:35:48.544057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00037824000000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"3.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-14T06:29:00.276180+0000","last_change":"2024-09-14T06:29:00.276180+0000","last_active":"2024-09-14T06:29:00.276180+0000","last_peered":"2024-09-14T06:29:00.276180+0000","last_clean":"2024-09-14T06:29:00.276180+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-14T06:29:00.276180+0000","last_undegraded":"2024-09-14T06:29:00.276180+0000","last_fullsized":"2024-09-14T06:29:00.276180+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":20,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-14T06:28:53.226072+0000","last_change":"2024-09-14T06:28:53.226072+0000","last_active":"2024-09-14T06:28:53.226072+0000","last_peered":"2024-09-14T06:28:53.226072+0000","last_clean":"2024-09-14T06:28:53.226072+0000","last_became_active":"2024-09-14T06:28:51.308072+0000","last_became_peered":"2024-09-14T06:28:51.308072+0000","last_unstale":"2024-09-14T06:28:53.226072+0000","last_undegraded":"2024-09-14T06:28:53.226072+0000","last_fullsized":"2024-09-14T06:28:53.226072+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:28:50.205350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:28:50.205350+0000","last_clean_scrub_stamp":"2024-09-14T06:28:50.205350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T13:15:09.052328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066197399999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"3.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-14T06:29:00.276180+0000","last_change":"2024-09-14T06:29:00.276180+0000","last_active":"2024-09-14T06:29:00.276180+0000","last_peered":"2024-09-14T06:29:00.276180+0000","last_clean":"2024-09-14T06:29:00.276180+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-14T06:29:00.276180+0000","last_undegraded":"2024-09-14T06:29:00.276180+0000","last_fullsized":"2024-09-14T06:29:00.276180+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-14T06:29:00.276180+0000","last_change":"2024-09-14T06:29:00.276180+0000","last_active":"2024-09-14T06:29:00.276180+0000","last_peered":"2024-09-14T06:29:00.276180+0000","last_clean":"2024-09-14T06:29:00.276180+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-14T06:29:00.276180+0000","last_undegraded":"2024-09-14T06:29:00.276180+0000","last_fullsized":"2024-09-14T06:29:00.276180+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.2","version":"22'2","reported_seq":22,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-14T06:28:53.235878+0000","last_change":"2024-09-14T06:28:53.231428+0000","last_active":"2024-09-14T06:28:53.235878+0000","last_peered":"2024-09-14T06:28:53.235878+0000","last_clean":"2024-09-14T06:28:53.235878+0000","last_became_active":"2024-09-14T06:28:51.222937+0000","last_became_peered":"2024-09-14T06:28:51.222937+0000","last_unstale":"2024-09-14T06:28:53.235878+0000","last_undegraded":"2024-09-14T06:28:53.235878+0000","last_fullsized":"2024-09-14T06:28:53.235878+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:28:50.205350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:28:50.205350+0000","last_clean_scrub_stamp":"2024-09-14T06:28:50.205350+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T14:27:37.233976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010072970000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"3.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-14T06:29:00.276180+0000","last_change":"2024-09-14T06:29:00.276180+0000","last_active":"2024-09-14T06:29:00.276180+0000","last_peered":"2024-09-14T06:29:00.276180+0000","last_clean":"2024-09-14T06:29:00.276180+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-14T06:29:00.276180+0000","last_undegraded":"2024-09-14T06:29:00.276180+0000","last_fullsized":"2024-09-14T06:29:00.276180+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":20,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-14T06:28:53.225992+0000","last_change":"2024-09-14T06:28:53.226264+0000","last_active":"2024-09-14T06:28:53.225992+0000","last_peered":"2024-09-14T06:28:53.225992+0000","last_clean":"2024-09-14T06:28:53.225992+0000","last_became_active":"2024-09-14T06:28:51.223252+0000","last_became_peered":"2024-09-14T06:28:51.223252+0000","last_unstale":"2024-09-14T06:28:53.225992+0000","last_undegraded":"2024-09-14T06:28:53.225992+0000","last_fullsized":"2024-09-14T06:28:53.225992+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:28:50.205350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:28:50.205350+0000","last_clean_scrub_stamp":"2024-09-14T06:28:50.205350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T11:56:04.373414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053403100000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"3.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-14T06:29:00.276180+0000","last_change":"2024-09-14T06:29:00.276180+0000","last_active":"2024-09-14T06:29:00.276180+0000","last_peered":"2024-09-14T06:29:00.276180+0000","last_clean":"2024-09-14T06:29:00.276180+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-14T06:29:00.276180+0000","last_undegraded":"2024-09-14T06:29:00.276180+0000","last_fullsized":"2024-09-14T06:29:00.276180+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":20,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-14T06:28:53.640096+0000","last_change":"2024-09-14T06:28:53.640096+0000","last_active":"2024-09-14T06:28:53.640096+0000","last_peered":"2024-09-14T06:28:53.640096+0000","last_clean":"2024-09-14T06:28:53.640096+0000","last_became_active":"2024-09-14T06:28:51.223069+0000","last_became_peered":"2024-09-14T06:28:51.223069+0000","last_unstale":"2024-09-14T06:28:53.640096+0000","last_undegraded":"2024-09-14T06:28:53.640096+0000","last_fullsized":"2024-09-14T06:28:53.640096+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:28:50.205350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:28:50.205350+0000","last_clean_scrub_stamp":"2024-09-14T06:28:50.205350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T15:13:00.471509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00043714600000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"3.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-14T06:29:00.276180+0000","last_change":"2024-09-14T06:29:00.276180+0000","last_active":"2024-09-14T06:29:00.276180+0000","last_peered":"2024-09-14T06:29:00.276180+0000","last_clean":"2024-09-14T06:29:00.276180+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-14T06:29:00.276180+0000","last_undegraded":"2024-09-14T06:29:00.276180+0000","last_fullsized":"2024-09-14T06:29:00.276180+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.3","version":"20'1","reported_seq":21,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-14T06:28:53.230933+0000","last_change":"2024-09-14T06:28:53.230933+0000","last_active":"2024-09-14T06:28:53.230933+0000","last_peered":"2024-09-14T06:28:53.230933+0000","last_clean":"2024-09-14T06:28:53.230933+0000","last_became_active":"2024-09-14T06:28:51.222874+0000","last_became_peered":"2024-09-14T06:28:51.222874+0000","last_unstale":"2024-09-14T06:28:53.230933+0000","last_undegraded":"2024-09-14T06:28:53.230933+0000","last_fullsized":"2024-09-14T06:28:53.230933+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:28:50.205350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:28:50.205350+0000","last_clean_scrub_stamp":"2024-09-14T06:28:50.205350+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T12:04:25.190171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050808899999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.0","version":"17'72","reported_seq":111,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-14T06:28:53.639580+0000","last_change":"2024-09-14T06:28:44.839885+0000","last_active":"2024-09-14T06:28:53.639580+0000","last_peered":"2024-09-14T06:28:53.639580+0000","last_clean":"2024-09-14T06:28:53.639580+0000","last_became_active":"2024-09-14T06:28:44.839573+0000","last_became_peered":"2024-09-14T06:28:44.839573+0000","last_unstale":"2024-09-14T06:28:53.639580+0000","last_undegraded":"2024-09-14T06:28:53.639580+0000","last_fullsized":"2024-09-14T06:28:53.639580+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:28:43.828494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:28:43.828494+0000","last_clean_scrub_stamp":"2024-09-14T06:28:43.828494+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T14:41:25.787293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-14T06:29:00.276180+0000","last_change":"2024-09-14T06:29:00.276180+0000","last_active":"2024-09-14T06:29:00.276180+0000","last_peered":"2024-09-14T06:29:00.276180+0000","last_clean":"2024-09-14T06:29:00.276180+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-14T06:29:00.276180+0000","last_undegraded":"2024-09-14T06:29:00.276180+0000","last_fullsized":"2024-09-14T06:29:00.276180+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-14T06:29:00.276180+0000","last_change":"2024-09-14T06:29:00.276180+0000","last_active":"2024-09-14T06:29:00.276180+0000","last_peered":"2024-09-14T06:29:00.276180+0000","last_clean":"2024-09-14T06:29:00.276180+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-14T06:29:00.276180+0000","last_undegraded":"2024-09-14T06:29:00.276180+0000","last_fullsized":"2024-09-14T06:29:00.276180+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-14T06:29:00.276180+0000","last_change":"2024-09-14T06:29:00.276180+0000","last_active":"2024-09-14T06:29:00.276180+0000","last_peered":"2024-09-14T06:29:00.276180+0000","last_clean":"2024-09-14T06:29:00.276180+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-14T06:29:00.276180+0000","last_undegraded":"2024-09-14T06:29:00.276180+0000","last_fullsized":"2024-09-14T06:29:00.276180+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-14T06:29:00.276180+0000","last_change":"2024-09-14T06:29:00.276180+0000","last_active":"2024-09-14T06:29:00.276180+0000","last_peered":"2024-09-14T06:29:00.276180+0000","last_clean":"2024-09-14T06:29:00.276180+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-14T06:29:00.276180+0000","last_undegraded":"2024-09-14T06:29:00.276180+0000","last_fullsized":"2024-09-14T06:29:00.276180+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-14T06:29:00.276180+0000","last_change":"2024-09-14T06:29:00.276180+0000","last_active":"2024-09-14T06:29:00.276180+0000","last_peered":"2024-09-14T06:29:00.276180+0000","last_clean":"2024-09-14T06:29:00.276180+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-14T06:29:00.276180+0000","last_undegraded":"2024-09-14T06:29:00.276180+0000","last_fullsized":"2024-09-14T06:29:00.276180+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-14T06:29:00.276180+0000","last_change":"2024-09-14T06:29:00.276180+0000","last_active":"2024-09-14T06:29:00.276180+0000","last_peered":"2024-09-14T06:29:00.276180+0000","last_clean":"2024-09-14T06:29:00.276180+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-14T06:29:00.276180+0000","last_undegraded":"2024-09-14T06:29:00.276180+0000","last_fullsized":"2024-09-14T06:29:00.276180+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-14T06:29:00.276180+0000","last_change":"2024-09-14T06:29:00.276180+0000","last_active":"2024-09-14T06:29:00.276180+0000","last_peered":"2024-09-14T06:29:00.276180+0000","last_clean":"2024-09-14T06:29:00.276180+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-14T06:29:00.276180+0000","last_undegraded":"2024-09-14T06:29:00.276180+0000","last_fullsized":"2024-09-14T06:29:00.276180+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-14T06:29:00.276180+0000","last_change":"2024-09-14T06:29:00.276180+0000","last_active":"2024-09-14T06:29:00.276180+0000","last_peered":"2024-09-14T06:29:00.276180+0000","last_clean":"2024-09-14T06:29:00.276180+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-14T06:29:00.276180+0000","last_undegraded":"2024-09-14T06:29:00.276180+0000","last_fullsized":"2024-09-14T06:29:00.276180+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-14T06:29:00.276180+0000","last_change":"2024-09-14T06:29:00.276180+0000","last_active":"2024-09-14T06:29:00.276180+0000","last_peered":"2024-09-14T06:29:00.276180+0000","last_clean":"2024-09-14T06:29:00.276180+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-14T06:29:00.276180+0000","last_undegraded":"2024-09-14T06:29:00.276180+0000","last_fullsized":"2024-09-14T06:29:00.276180+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-14T06:29:00.276180+0000","last_change":"2024-09-14T06:29:00.276180+0000","last_active":"2024-09-14T06:29:00.276180+0000","last_peered":"2024-09-14T06:29:00.276180+0000","last_clean":"2024-09-14T06:29:00.276180+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-14T06:29:00.276180+0000","last_undegraded":"2024-09-14T06:29:00.276180+0000","last_fullsized":"2024-09-14T06:29:00.276180+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-14T06:29:00.276180+0000","last_change":"2024-09-14T06:29:00.276180+0000","last_active":"2024-09-14T06:29:00.276180+0000","last_peered":"2024-09-14T06:29:00.276180+0000","last_clean":"2024-09-14T06:29:00.276180+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-14T06:29:00.276180+0000","last_undegraded":"2024-09-14T06:29:00.276180+0000","last_fullsized":"2024-09-14T06:29:00.276180+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-14T06:29:00.276180+0000","last_change":"2024-09-14T06:29:00.276180+0000","last_active":"2024-09-14T06:29:00.276180+0000","last_peered":"2024-09-14T06:29:00.276180+0000","last_clean":"2024-09-14T06:29:00.276180+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-14T06:29:00.276180+0000","last_undegraded":"2024-09-14T06:29:00.276180+0000","last_fullsized":"2024-09-14T06:29:00.276180+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":131072,"data_stored":1180736,"data_compressed":65166,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509446,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27308,"kb_used_data":280,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344532,"statfs":{"total":96636764160,"available":96608800768,"internally_reserved":0,"allocated":286720,"data_stored":665454,"data_compressed":32583,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509445,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27244,"kb_used_data":216,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344596,"statfs":{"total":96636764160,"available":96608866304,"internally_reserved":0,"allocated":221184,"data_stored":75086,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509447,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27256,"kb_used_data":220,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344584,"statfs":{"total":96636764160,"available":96608854016,"internally_reserved":0,"allocated":225280,"data_stored":75105,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509446,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27252,"kb_used_data":216,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344588,"statfs":{"total":96636764160,"available":96608858112,"internally_reserved":0,"allocated":221184,"data_stored":75086,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509447,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31416,"kb_used_data":220,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340424,"statfs":{"total":96636764160,"available":96604594176,"internally_reserved":0,"allocated":225280,"data_stored":75105,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":14,"seq":60129542150,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31408,"kb_used_data":216,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340432,"statfs":{"total":96636764160,"available":96604602368,"internally_reserved":0,"allocated":221184,"data_stored":75086,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542149,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31472,"kb_used_data":280,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340368,"statfs":{"total":96636764160,"available":96604536832,"internally_reserved":0,"allocated":286720,"data_stored":665454,"data_compressed":32583,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574854,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31408,"kb_used_data":216,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340432,"statfs":{"total":96636764160,"available":96604602368,"internally_reserved":0,"allocated":221184,"data_stored":75086,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32583,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32583,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-14T06:29:00.358 INFO:tasks.thrashosds.thrasher:No pools; doing nothing 2024-09-14T06:29:00.428 INFO:tasks.ceph.osd.4.smithi138.stderr:2024-09-14T06:29:00.428+0000 7f6c653c8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33615) UID: 0 2024-09-14T06:29:00.446 INFO:teuthology.orchestra.run.smithi094.stdout:ERROR: (22) Invalid argument 2024-09-14T06:29:00.446 INFO:teuthology.orchestra.run.smithi094.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:29:00.455 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-14T06:29:00.529 INFO:tasks.ceph.osd.6.smithi138.stderr:2024-09-14T06:29:00.528+0000 7f8cd8849640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33620) UID: 0 2024-09-14T06:29:00.559 INFO:teuthology.orchestra.run.smithi094.stdout:ERROR: (22) Invalid argument 2024-09-14T06:29:00.559 INFO:teuthology.orchestra.run.smithi094.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:29:00.567 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-14T06:29:00.630 INFO:tasks.ceph.osd.6.smithi138.stderr:2024-09-14T06:29:00.630+0000 7f8cd8849640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33620) UID: 0 2024-09-14T06:29:00.675 INFO:teuthology.orchestra.run.smithi094.stdout:ERROR: (22) Invalid argument 2024-09-14T06:29:00.675 INFO:teuthology.orchestra.run.smithi094.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:29:00.683 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-14T06:29:00.731 INFO:tasks.ceph.osd.2.smithi094.stderr:2024-09-14T06:29:00.730+0000 7faf3269b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33649) UID: 0 2024-09-14T06:29:00.786 INFO:teuthology.orchestra.run.smithi094.stdout:ERROR: (22) Invalid argument 2024-09-14T06:29:00.786 INFO:teuthology.orchestra.run.smithi094.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:29:00.795 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-14T06:29:00.836 INFO:tasks.ceph.osd.3.smithi094.stderr:2024-09-14T06:29:00.835+0000 7feeeafe8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33655) UID: 0 2024-09-14T06:29:00.933 INFO:tasks.ceph.osd.6.smithi138.stderr:2024-09-14T06:29:00.932+0000 7f8cd8849640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33620) UID: 0 2024-09-14T06:29:00.938 INFO:teuthology.orchestra.run.smithi094.stdout:ERROR: (22) Invalid argument 2024-09-14T06:29:00.938 INFO:teuthology.orchestra.run.smithi094.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:29:00.947 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-14T06:29:01.034 INFO:tasks.ceph.osd.2.smithi094.stderr:2024-09-14T06:29:01.033+0000 7faf3269b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33649) UID: 0 2024-09-14T06:29:01.049 INFO:teuthology.orchestra.run.smithi094.stdout:ERROR: (22) Invalid argument 2024-09-14T06:29:01.049 INFO:teuthology.orchestra.run.smithi094.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:29:01.057 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-14T06:29:01.142 INFO:tasks.ceph.osd.1.smithi094.stderr:2024-09-14T06:29:01.141+0000 7f328e47a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33641) UID: 0 2024-09-14T06:29:01.200 INFO:teuthology.orchestra.run.smithi094.stdout:ERROR: (22) Invalid argument 2024-09-14T06:29:01.201 INFO:teuthology.orchestra.run.smithi094.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:29:01.209 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-14T06:29:01.250 INFO:tasks.ceph.osd.3.smithi094.stderr:2024-09-14T06:29:01.249+0000 7feeeafe8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33655) UID: 0 2024-09-14T06:29:01.326 INFO:teuthology.orchestra.run.smithi138.stdout:ERROR: (22) Invalid argument 2024-09-14T06:29:01.326 INFO:teuthology.orchestra.run.smithi138.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:29:01.336 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-14T06:29:01.336 INFO:tasks.ceph.osd.6.smithi138.stderr:2024-09-14T06:29:01.334+0000 7f8cd8849640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33620) UID: 0 2024-09-14T06:29:01.437 INFO:tasks.ceph.osd.2.smithi094.stderr:2024-09-14T06:29:01.436+0000 7faf3269b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33649) UID: 0 2024-09-14T06:29:01.480 INFO:teuthology.orchestra.run.smithi138.stdout:ERROR: (22) Invalid argument 2024-09-14T06:29:01.480 INFO:teuthology.orchestra.run.smithi138.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:29:01.488 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-14T06:29:01.538 INFO:tasks.ceph.osd.6.smithi138.stderr:2024-09-14T06:29:01.536+0000 7f8cd8849640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33620) UID: 0 2024-09-14T06:29:01.592 INFO:teuthology.orchestra.run.smithi138.stdout:ERROR: (22) Invalid argument 2024-09-14T06:29:01.592 INFO:teuthology.orchestra.run.smithi138.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:29:01.601 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-14T06:29:01.642 INFO:tasks.ceph.osd.7.smithi138.stderr:2024-09-14T06:29:01.640+0000 7f3c6a21d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33623) UID: 0 2024-09-14T06:29:01.742 INFO:tasks.ceph.osd.5.smithi138.stderr:2024-09-14T06:29:01.739+0000 7fe13746f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33618) UID: 0 2024-09-14T06:29:01.743 INFO:teuthology.orchestra.run.smithi138.stdout:ERROR: (22) Invalid argument 2024-09-14T06:29:01.744 INFO:teuthology.orchestra.run.smithi138.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:29:01.752 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-14T06:29:01.845 INFO:tasks.ceph.osd.2.smithi094.stderr:2024-09-14T06:29:01.844+0000 7faf3269b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33649) UID: 0 2024-09-14T06:29:01.895 INFO:teuthology.orchestra.run.smithi138.stdout:ERROR: (22) Invalid argument 2024-09-14T06:29:01.895 INFO:teuthology.orchestra.run.smithi138.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:29:01.904 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-14T06:29:01.946 INFO:tasks.ceph.osd.3.smithi094.stderr:2024-09-14T06:29:01.945+0000 7feeeafe8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33655) UID: 0 2024-09-14T06:29:02.007 INFO:teuthology.orchestra.run.smithi138.stdout:ERROR: (22) Invalid argument 2024-09-14T06:29:02.007 INFO:teuthology.orchestra.run.smithi138.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:29:02.015 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-14T06:29:02.026 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-14T06:29:02.059 INFO:tasks.ceph.osd.4.smithi138.stderr:2024-09-14T06:29:02.057+0000 7f6c653c8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33615) UID: 0 2024-09-14T06:29:02.118 INFO:teuthology.orchestra.run.smithi138.stdout:ERROR: (22) Invalid argument 2024-09-14T06:29:02.119 INFO:teuthology.orchestra.run.smithi138.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:29:02.127 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-14T06:29:02.147 INFO:tasks.ceph.osd.2.smithi094.stderr:2024-09-14T06:29:02.146+0000 7faf3269b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33649) UID: 0 2024-09-14T06:29:02.151 INFO:teuthology.orchestra.run.smithi094.stdout:osd.0: {} 2024-09-14T06:29:02.152 INFO:teuthology.orchestra.run.smithi094.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-14T06:29:02.168 INFO:teuthology.orchestra.run.smithi094.stdout:osd.1: {} 2024-09-14T06:29:02.168 INFO:teuthology.orchestra.run.smithi094.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-14T06:29:02.184 INFO:teuthology.orchestra.run.smithi094.stdout:osd.2: {} 2024-09-14T06:29:02.184 INFO:teuthology.orchestra.run.smithi094.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-14T06:29:02.200 INFO:teuthology.orchestra.run.smithi094.stdout:osd.3: {} 2024-09-14T06:29:02.200 INFO:teuthology.orchestra.run.smithi094.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-14T06:29:02.216 INFO:teuthology.orchestra.run.smithi094.stdout:osd.4: {} 2024-09-14T06:29:02.216 INFO:teuthology.orchestra.run.smithi094.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-14T06:29:02.233 INFO:teuthology.orchestra.run.smithi094.stdout:osd.5: {} 2024-09-14T06:29:02.233 INFO:teuthology.orchestra.run.smithi094.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-14T06:29:02.248 INFO:tasks.ceph.osd.6.smithi138.stderr:2024-09-14T06:29:02.246+0000 7f8cd8849640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33620) UID: 0 2024-09-14T06:29:02.249 INFO:teuthology.orchestra.run.smithi094.stdout:osd.6: {} 2024-09-14T06:29:02.249 INFO:teuthology.orchestra.run.smithi094.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-14T06:29:02.265 INFO:teuthology.orchestra.run.smithi094.stdout:osd.7: {} 2024-09-14T06:29:02.265 INFO:teuthology.orchestra.run.smithi094.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-14T06:29:02.272 INFO:teuthology.orchestra.run.smithi138.stdout:{ 2024-09-14T06:29:02.272 INFO:teuthology.orchestra.run.smithi138.stdout: "ops": [], 2024-09-14T06:29:02.272 INFO:teuthology.orchestra.run.smithi138.stdout: "complaint_time": 30, 2024-09-14T06:29:02.272 INFO:teuthology.orchestra.run.smithi138.stdout: "num_blocked_ops": 0 2024-09-14T06:29:02.272 INFO:teuthology.orchestra.run.smithi138.stdout:} 2024-09-14T06:29:02.280 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-14T06:29:02.300 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-14T06:29:02.360 INFO:tasks.ceph.osd.0.smithi094.stderr:2024-09-14T06:29:02.359+0000 7f07c9333640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33636) UID: 0 2024-09-14T06:29:02.422 INFO:teuthology.orchestra.run.smithi138.stdout:{ 2024-09-14T06:29:02.423 INFO:teuthology.orchestra.run.smithi138.stdout: "size": 20, 2024-09-14T06:29:02.423 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 600, 2024-09-14T06:29:02.423 INFO:teuthology.orchestra.run.smithi138.stdout: "ops": [] 2024-09-14T06:29:02.423 INFO:teuthology.orchestra.run.smithi138.stdout:} 2024-09-14T06:29:02.431 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-14T06:29:02.475 INFO:tasks.ceph.osd.5.smithi138.stderr:2024-09-14T06:29:02.473+0000 7fe13746f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33618) UID: 0 2024-09-14T06:29:02.534 INFO:teuthology.orchestra.run.smithi138.stdout:{ 2024-09-14T06:29:02.534 INFO:teuthology.orchestra.run.smithi138.stdout: "ops": [], 2024-09-14T06:29:02.534 INFO:teuthology.orchestra.run.smithi138.stdout: "num_ops": 0 2024-09-14T06:29:02.534 INFO:teuthology.orchestra.run.smithi138.stdout:} 2024-09-14T06:29:02.542 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-14T06:29:02.584 INFO:tasks.ceph.osd.7.smithi138.stderr:2024-09-14T06:29:02.582+0000 7f3c6a21d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33623) UID: 0 2024-09-14T06:29:02.651 INFO:tasks.ceph.osd.0.smithi094.stderr:2024-09-14T06:29:02.650+0000 7f07c9333640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33636) UID: 0 2024-09-14T06:29:02.686 INFO:teuthology.orchestra.run.smithi138.stdout:{ 2024-09-14T06:29:02.686 INFO:teuthology.orchestra.run.smithi138.stdout: "ops": [], 2024-09-14T06:29:02.686 INFO:teuthology.orchestra.run.smithi138.stdout: "complaint_time": 30, 2024-09-14T06:29:02.686 INFO:teuthology.orchestra.run.smithi138.stdout: "num_blocked_ops": 0 2024-09-14T06:29:02.686 INFO:teuthology.orchestra.run.smithi138.stdout:} 2024-09-14T06:29:02.694 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-14T06:29:02.752 INFO:tasks.ceph.osd.7.smithi138.stderr:2024-09-14T06:29:02.750+0000 7f3c6a21d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33623) UID: 0 2024-09-14T06:29:02.800 INFO:teuthology.orchestra.run.smithi138.stdout:{ 2024-09-14T06:29:02.800 INFO:teuthology.orchestra.run.smithi138.stdout: "size": 20, 2024-09-14T06:29:02.800 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 600, 2024-09-14T06:29:02.800 INFO:teuthology.orchestra.run.smithi138.stdout: "ops": [ 2024-09-14T06:29:02.800 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:02.800 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4195.0:25 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-14T06:29:02.800 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:28:44.860134+0000", 2024-09-14T06:29:02.800 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 17.937303264000001, 2024-09-14T06:29:02.800 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.0076319070000000003, 2024-09-14T06:29:02.801 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:29:02.801 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:29:02.801 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:29:02.801 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:29:02.801 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4195", 2024-09-14T06:29:02.801 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.138:0/3795168327", 2024-09-14T06:29:02.801 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 25 2024-09-14T06:29:02.801 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:02.801 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:29:02.801 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:02.801 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:29:02.801 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:28:44.860134+0000", 2024-09-14T06:29:02.801 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:02.802 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:02.802 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:02.802 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:29:02.802 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:28:44.860134+0000", 2024-09-14T06:29:02.802 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:02.802 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:02.802 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:02.802 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:29:02.802 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:28:44.860135+0000", 2024-09-14T06:29:02.802 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.47e-07 2024-09-14T06:29:02.802 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:02.802 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:02.802 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:29:02.802 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:28:44.860140+0000", 2024-09-14T06:29:02.803 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.7790000000000001e-06 2024-09-14T06:29:02.803 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:02.803 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:02.803 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:29:02.803 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:28:44.860152+0000", 2024-09-14T06:29:02.803 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.199e-05 2024-09-14T06:29:02.803 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:02.803 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:02.803 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:29:02.803 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:28:44.860160+0000", 2024-09-14T06:29:02.803 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.8699999999999992e-06 2024-09-14T06:29:02.803 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:02.803 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:02.803 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:29:02.803 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:28:44.863273+0000", 2024-09-14T06:29:02.804 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.0031130369999999999 2024-09-14T06:29:02.804 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:02.804 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:02.804 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:29:02.804 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:28:44.863324+0000", 2024-09-14T06:29:02.804 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.0841000000000003e-05 2024-09-14T06:29:02.804 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:02.804 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:02.804 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "waiting for subops from 0", 2024-09-14T06:29:02.804 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:28:44.863407+0000", 2024-09-14T06:29:02.804 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.3171999999999996e-05 2024-09-14T06:29:02.804 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:02.804 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:02.805 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:29:02.805 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:28:44.864292+0000", 2024-09-14T06:29:02.805 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00088475200000000004 2024-09-14T06:29:02.805 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:02.805 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:02.805 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "op_commit", 2024-09-14T06:29:02.805 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:28:44.867706+0000", 2024-09-14T06:29:02.805 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.0034142280000000001 2024-09-14T06:29:02.805 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:02.805 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:02.805 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "commit_sent", 2024-09-14T06:29:02.806 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:28:44.867741+0000", 2024-09-14T06:29:02.806 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.4388999999999999e-05 2024-09-14T06:29:02.806 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:02.806 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:02.806 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:29:02.806 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:28:44.867766+0000", 2024-09-14T06:29:02.806 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.5001999999999999e-05 2024-09-14T06:29:02.806 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:02.806 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:29:02.806 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:02.806 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:02.806 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:02.806 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4195.0:45 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-14T06:29:02.806 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:28:45.178909+0000", 2024-09-14T06:29:02.806 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 17.618528091999998, 2024-09-14T06:29:02.807 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.0062892770000000002, 2024-09-14T06:29:02.807 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:29:02.807 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:29:02.807 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:29:02.807 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:29:02.807 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4195", 2024-09-14T06:29:02.807 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.138:0/3795168327", 2024-09-14T06:29:02.807 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 45 2024-09-14T06:29:02.807 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:02.807 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:29:02.807 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:02.807 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:29:02.807 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:28:45.178909+0000", 2024-09-14T06:29:02.807 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:02.807 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:02.808 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:02.808 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:29:02.808 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:28:45.178909+0000", 2024-09-14T06:29:02.808 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:02.808 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:02.808 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:02.808 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:29:02.808 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:28:45.178909+0000", 2024-09-14T06:29:02.808 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.58e-07 2024-09-14T06:29:02.808 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:02.808 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:02.808 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:29:02.808 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:28:45.178920+0000", 2024-09-14T06:29:02.808 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.0689999999999999e-05 2024-09-14T06:29:02.809 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:02.809 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:02.809 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:29:02.809 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:28:45.178927+0000", 2024-09-14T06:29:02.809 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.1400000000000002e-06 2024-09-14T06:29:02.809 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:02.809 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:02.809 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:29:02.809 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:28:45.178934+0000", 2024-09-14T06:29:02.809 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.4200000000000004e-06 2024-09-14T06:29:02.809 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:02.809 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:02.809 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:29:02.809 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:28:45.182798+0000", 2024-09-14T06:29:02.809 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.0038647619999999999 2024-09-14T06:29:02.810 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:02.810 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:02.810 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:29:02.810 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:28:45.182887+0000", 2024-09-14T06:29:02.810 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.8764000000000006e-05 2024-09-14T06:29:02.810 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:02.810 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:02.810 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "waiting for subops from 0", 2024-09-14T06:29:02.810 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:28:45.183034+0000", 2024-09-14T06:29:02.810 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00014712199999999999 2024-09-14T06:29:02.810 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:02.810 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:02.810 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:29:02.810 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:28:45.185120+0000", 2024-09-14T06:29:02.810 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.0020861669999999999 2024-09-14T06:29:02.811 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:02.811 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:02.811 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "op_commit", 2024-09-14T06:29:02.811 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:28:45.185130+0000", 2024-09-14T06:29:02.811 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.4939999999999992e-06 2024-09-14T06:29:02.811 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:02.811 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:02.811 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "commit_sent", 2024-09-14T06:29:02.811 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:28:45.185161+0000", 2024-09-14T06:29:02.811 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.0966999999999997e-05 2024-09-14T06:29:02.811 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:02.811 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:02.811 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:29:02.811 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:28:45.185198+0000", 2024-09-14T06:29:02.811 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.7292999999999997e-05 2024-09-14T06:29:02.812 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:02.812 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:29:02.812 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:02.812 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:02.812 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:02.812 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4195.0:46 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131600~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-14T06:29:02.812 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:28:45.178944+0000", 2024-09-14T06:29:02.812 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 17.618493072, 2024-09-14T06:29:02.812 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.0064278110000000003, 2024-09-14T06:29:02.812 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:29:02.812 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:29:02.812 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:29:02.812 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:29:02.812 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4195", 2024-09-14T06:29:02.812 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.138:0/3795168327", 2024-09-14T06:29:02.813 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 46 2024-09-14T06:29:02.813 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:02.813 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:29:02.813 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:02.813 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:29:02.813 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:28:45.178944+0000", 2024-09-14T06:29:02.813 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:02.813 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:02.813 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:02.813 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:29:02.813 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:28:45.178944+0000", 2024-09-14T06:29:02.813 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:02.814 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:02.814 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:02.814 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:29:02.814 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:28:45.178944+0000", 2024-09-14T06:29:02.814 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.1099999999999996e-07 2024-09-14T06:29:02.814 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:02.814 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:02.814 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:29:02.814 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:28:45.178950+0000", 2024-09-14T06:29:02.814 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.3190000000000002e-06 2024-09-14T06:29:02.814 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:02.814 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:02.814 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:29:02.814 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:28:45.178955+0000", 2024-09-14T06:29:02.814 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.9180000000000002e-06 2024-09-14T06:29:02.815 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:02.815 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:02.815 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:29:02.815 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:28:45.178959+0000", 2024-09-14T06:29:02.815 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.0740000000000003e-06 2024-09-14T06:29:02.815 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:02.815 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:02.815 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:29:02.815 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:28:45.183353+0000", 2024-09-14T06:29:02.815 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.0043943710000000002 2024-09-14T06:29:02.815 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:02.815 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:02.815 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:29:02.816 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:28:45.183412+0000", 2024-09-14T06:29:02.816 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.8464000000000001e-05 2024-09-14T06:29:02.816 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:02.816 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:02.816 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "waiting for subops from 0", 2024-09-14T06:29:02.816 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:28:45.183493+0000", 2024-09-14T06:29:02.816 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.1563000000000005e-05 2024-09-14T06:29:02.816 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:02.816 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:02.816 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "op_commit", 2024-09-14T06:29:02.816 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:28:45.185205+0000", 2024-09-14T06:29:02.816 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.0017123749999999999 2024-09-14T06:29:02.816 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:02.816 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:02.817 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:29:02.817 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:28:45.185273+0000", 2024-09-14T06:29:02.817 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.7175000000000007e-05 2024-09-14T06:29:02.817 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:02.817 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:02.817 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "commit_sent", 2024-09-14T06:29:02.817 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:28:45.185333+0000", 2024-09-14T06:29:02.818 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.0519000000000001e-05 2024-09-14T06:29:02.818 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:02.818 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:02.818 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:29:02.818 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:28:45.185372+0000", 2024-09-14T06:29:02.818 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.8522000000000001e-05 2024-09-14T06:29:02.818 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:02.819 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:29:02.819 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:02.819 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:02.819 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:02.819 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4195.0:47 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-14T06:29:02.819 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:28:45.178969+0000", 2024-09-14T06:29:02.820 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 17.618467939999999, 2024-09-14T06:29:02.820 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.0076346369999999997, 2024-09-14T06:29:02.820 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:29:02.820 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:29:02.820 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:29:02.820 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:29:02.821 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4195", 2024-09-14T06:29:02.821 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.138:0/3795168327", 2024-09-14T06:29:02.821 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 47 2024-09-14T06:29:02.821 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:02.821 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:29:02.821 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:02.821 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:29:02.822 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:28:45.178969+0000", 2024-09-14T06:29:02.822 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:02.822 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:02.822 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:02.822 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:29:02.822 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:28:45.178969+0000", 2024-09-14T06:29:02.822 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:02.823 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:02.823 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:02.823 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:29:02.823 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:28:45.178969+0000", 2024-09-14T06:29:02.823 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.8700000000000002e-07 2024-09-14T06:29:02.823 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:02.824 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:02.824 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:29:02.824 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:28:45.179037+0000", 2024-09-14T06:29:02.824 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.7750999999999996e-05 2024-09-14T06:29:02.824 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:02.824 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:02.824 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:29:02.825 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:28:45.179052+0000", 2024-09-14T06:29:02.825 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.5373e-05 2024-09-14T06:29:02.825 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:02.825 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:02.825 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:29:02.825 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:28:45.179057+0000", 2024-09-14T06:29:02.826 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.7609999999999998e-06 2024-09-14T06:29:02.826 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:02.826 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:02.826 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:29:02.826 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:28:45.183861+0000", 2024-09-14T06:29:02.826 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.0048036550000000004 2024-09-14T06:29:02.827 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:02.827 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:02.827 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:29:02.827 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:28:45.183942+0000", 2024-09-14T06:29:02.827 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.1219999999999998e-05 2024-09-14T06:29:02.827 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:02.827 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:02.828 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "waiting for subops from 0", 2024-09-14T06:29:02.828 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:28:45.184057+0000", 2024-09-14T06:29:02.828 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00011453 2024-09-14T06:29:02.828 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:02.828 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:02.828 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:29:02.829 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:28:45.185422+0000", 2024-09-14T06:29:02.829 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.0013656740000000001 2024-09-14T06:29:02.829 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:02.829 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:02.829 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "op_commit", 2024-09-14T06:29:02.829 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:28:45.186546+0000", 2024-09-14T06:29:02.830 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.001123642 2024-09-14T06:29:02.830 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:02.830 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:02.830 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "commit_sent", 2024-09-14T06:29:02.830 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:28:45.186576+0000", 2024-09-14T06:29:02.830 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.0546000000000002e-05 2024-09-14T06:29:02.831 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:02.831 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:02.831 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:29:02.831 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:28:45.186604+0000", 2024-09-14T06:29:02.831 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.7197999999999999e-05 2024-09-14T06:29:02.831 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:02.831 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:29:02.831 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:02.831 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:02.831 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:02.831 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4195.0:48 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-14T06:29:02.831 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:28:45.179087+0000", 2024-09-14T06:29:02.832 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 17.618350263, 2024-09-14T06:29:02.832 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.0076893559999999996, 2024-09-14T06:29:02.832 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:29:02.832 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:29:02.832 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:29:02.832 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:29:02.832 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4195", 2024-09-14T06:29:02.832 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.138:0/3795168327", 2024-09-14T06:29:02.832 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 48 2024-09-14T06:29:02.832 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:02.832 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:29:02.832 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:02.832 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:29:02.833 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:28:45.179087+0000", 2024-09-14T06:29:02.833 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:02.833 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:02.833 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:02.833 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:29:02.833 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:28:45.179087+0000", 2024-09-14T06:29:02.833 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:02.833 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:02.833 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:02.833 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:29:02.833 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:28:45.179087+0000", 2024-09-14T06:29:02.833 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.3399999999999999e-07 2024-09-14T06:29:02.833 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:02.833 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:02.834 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:29:02.834 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:28:45.179091+0000", 2024-09-14T06:29:02.834 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.366e-06 2024-09-14T06:29:02.834 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:02.834 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:02.834 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:29:02.834 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:28:45.179097+0000", 2024-09-14T06:29:02.834 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.7830000000000001e-06 2024-09-14T06:29:02.834 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:02.834 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:02.835 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:29:02.835 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:28:45.179102+0000", 2024-09-14T06:29:02.835 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.1529999999999999e-06 2024-09-14T06:29:02.835 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:02.835 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:02.835 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:29:02.835 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:28:45.184654+0000", 2024-09-14T06:29:02.836 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.0055525390000000004 2024-09-14T06:29:02.836 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:02.836 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:02.836 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:29:02.836 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:28:45.184726+0000", 2024-09-14T06:29:02.836 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.1717999999999993e-05 2024-09-14T06:29:02.837 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:02.837 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:02.837 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "waiting for subops from 0", 2024-09-14T06:29:02.837 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:28:45.184809+0000", 2024-09-14T06:29:02.837 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.3461999999999995e-05 2024-09-14T06:29:02.837 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:02.837 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:02.838 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:29:02.838 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:28:45.186653+0000", 2024-09-14T06:29:02.838 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.0018440010000000001 2024-09-14T06:29:02.838 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:02.838 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:02.838 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "op_commit", 2024-09-14T06:29:02.839 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:28:45.186722+0000", 2024-09-14T06:29:02.839 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.8388999999999999e-05 2024-09-14T06:29:02.839 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:02.839 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:02.839 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "commit_sent", 2024-09-14T06:29:02.840 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:28:45.186752+0000", 2024-09-14T06:29:02.840 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.0026e-05 2024-09-14T06:29:02.840 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:02.840 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:02.840 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:29:02.840 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:28:45.186776+0000", 2024-09-14T06:29:02.841 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.4385000000000001e-05 2024-09-14T06:29:02.841 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:02.841 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:29:02.841 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:02.841 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:02.841 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:02.842 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4195.0:49 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-14T06:29:02.842 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:28:45.179111+0000", 2024-09-14T06:29:02.842 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 17.618325682999998, 2024-09-14T06:29:02.842 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.0085059769999999996, 2024-09-14T06:29:02.842 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:29:02.842 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:29:02.842 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:29:02.843 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:29:02.843 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4195", 2024-09-14T06:29:02.843 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.138:0/3795168327", 2024-09-14T06:29:02.843 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 49 2024-09-14T06:29:02.843 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:02.843 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:29:02.843 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:02.844 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:29:02.844 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:28:45.179111+0000", 2024-09-14T06:29:02.844 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:02.844 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:02.844 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:02.844 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:29:02.845 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:28:45.179111+0000", 2024-09-14T06:29:02.845 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:02.845 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:02.845 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:02.845 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:29:02.845 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:28:45.179112+0000", 2024-09-14T06:29:02.846 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.5699999999999998e-07 2024-09-14T06:29:02.846 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:02.846 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:02.846 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:29:02.846 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:28:45.179115+0000", 2024-09-14T06:29:02.846 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.709e-06 2024-09-14T06:29:02.846 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:02.847 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:02.847 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:29:02.847 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:28:45.179120+0000", 2024-09-14T06:29:02.847 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.1790000000000001e-06 2024-09-14T06:29:02.847 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:02.848 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:02.848 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:29:02.848 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:28:45.179123+0000", 2024-09-14T06:29:02.848 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.1039999999999998e-06 2024-09-14T06:29:02.848 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:02.848 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:02.849 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:29:02.849 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:28:45.185469+0000", 2024-09-14T06:29:02.849 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.0063467009999999997 2024-09-14T06:29:02.849 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:02.849 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:02.850 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:29:02.850 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:28:45.185558+0000", 2024-09-14T06:29:02.850 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.8576e-05 2024-09-14T06:29:02.850 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:02.850 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:02.851 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "waiting for subops from 0", 2024-09-14T06:29:02.851 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:28:45.185653+0000", 2024-09-14T06:29:02.851 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.4672000000000004e-05 2024-09-14T06:29:02.851 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:02.851 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:02.852 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:29:02.852 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:28:45.186711+0000", 2024-09-14T06:29:02.852 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.001058699 2024-09-14T06:29:02.852 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:02.852 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:02.853 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "op_commit", 2024-09-14T06:29:02.853 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:28:45.187559+0000", 2024-09-14T06:29:02.853 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00084785100000000005 2024-09-14T06:29:02.853 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:02.853 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:02.854 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "commit_sent", 2024-09-14T06:29:02.854 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:28:45.187591+0000", 2024-09-14T06:29:02.854 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.1406999999999999e-05 2024-09-14T06:29:02.854 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:02.854 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:02.854 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:29:02.855 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:28:45.187617+0000", 2024-09-14T06:29:02.855 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.6721999999999999e-05 2024-09-14T06:29:02.855 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:02.855 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:29:02.855 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:02.855 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:02.856 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:02.856 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4195.0:50 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-14T06:29:02.856 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:28:45.179146+0000", 2024-09-14T06:29:02.856 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 17.618290749, 2024-09-14T06:29:02.856 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.0088678730000000001, 2024-09-14T06:29:02.857 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:29:02.857 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:29:02.857 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:29:02.857 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:29:02.857 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4195", 2024-09-14T06:29:02.857 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.138:0/3795168327", 2024-09-14T06:29:02.858 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 50 2024-09-14T06:29:02.858 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:02.858 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:29:02.858 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:02.858 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:29:02.859 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:28:45.179146+0000", 2024-09-14T06:29:02.859 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:02.859 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:02.859 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:02.859 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:29:02.860 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:28:45.179146+0000", 2024-09-14T06:29:02.860 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:02.860 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:02.860 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:02.860 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:29:02.860 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:28:45.179147+0000", 2024-09-14T06:29:02.861 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.0400000000000005e-07 2024-09-14T06:29:02.861 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:02.861 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:02.861 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:29:02.861 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:28:45.179185+0000", 2024-09-14T06:29:02.862 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.7876000000000001e-05 2024-09-14T06:29:02.862 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:02.862 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:02.862 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:29:02.863 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:28:45.179191+0000", 2024-09-14T06:29:02.863 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.2179999999999996e-06 2024-09-14T06:29:02.863 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:02.863 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:02.863 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:29:02.864 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:28:45.179195+0000", 2024-09-14T06:29:02.864 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.6330000000000002e-06 2024-09-14T06:29:02.864 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:02.864 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:02.864 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:29:02.864 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:28:45.185928+0000", 2024-09-14T06:29:02.864 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.0067329529999999999 2024-09-14T06:29:02.864 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:02.864 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:02.865 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:29:02.865 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:28:45.185991+0000", 2024-09-14T06:29:02.865 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.3590999999999995e-05 2024-09-14T06:29:02.865 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:02.865 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:02.865 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "waiting for subops from 0", 2024-09-14T06:29:02.865 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:28:45.186087+0000", 2024-09-14T06:29:02.865 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.6088000000000006e-05 2024-09-14T06:29:02.865 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:02.865 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:02.865 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:29:02.865 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:28:45.187646+0000", 2024-09-14T06:29:02.865 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.0015581569999999999 2024-09-14T06:29:02.865 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:02.865 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:02.866 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "op_commit", 2024-09-14T06:29:02.866 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:28:45.187945+0000", 2024-09-14T06:29:02.866 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00029935999999999998 2024-09-14T06:29:02.866 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:02.866 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:02.866 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "commit_sent", 2024-09-14T06:29:02.866 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:28:45.187982+0000", 2024-09-14T06:29:02.866 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.6939e-05 2024-09-14T06:29:02.866 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:02.866 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:02.866 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:29:02.866 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:28:45.188014+0000", 2024-09-14T06:29:02.866 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.2254e-05 2024-09-14T06:29:02.867 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:02.867 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:29:02.867 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:02.867 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:02.867 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:02.867 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4195.0:51 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-14T06:29:02.867 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:28:45.179213+0000", 2024-09-14T06:29:02.867 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 17.618223529000002, 2024-09-14T06:29:02.867 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.0096335610000000006, 2024-09-14T06:29:02.867 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:29:02.867 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:29:02.867 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:29:02.867 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:29:02.867 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4195", 2024-09-14T06:29:02.868 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.138:0/3795168327", 2024-09-14T06:29:02.868 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 51 2024-09-14T06:29:02.868 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:02.868 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:29:02.868 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:02.868 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:29:02.868 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:28:45.179213+0000", 2024-09-14T06:29:02.868 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:02.868 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:02.868 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:02.868 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:29:02.868 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:28:45.179213+0000", 2024-09-14T06:29:02.868 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:02.868 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:02.869 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:02.869 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:29:02.869 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:28:45.179214+0000", 2024-09-14T06:29:02.869 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.1099999999999996e-07 2024-09-14T06:29:02.869 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:02.869 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:02.869 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:29:02.869 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:28:45.179217+0000", 2024-09-14T06:29:02.869 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.1e-06 2024-09-14T06:29:02.869 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:02.869 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:02.869 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:29:02.869 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:28:45.179228+0000", 2024-09-14T06:29:02.869 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.1432e-05 2024-09-14T06:29:02.869 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:02.870 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:02.870 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:29:02.870 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:28:45.179241+0000", 2024-09-14T06:29:02.870 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.2605999999999999e-05 2024-09-14T06:29:02.870 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:02.870 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:02.870 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:29:02.870 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:28:45.186811+0000", 2024-09-14T06:29:02.870 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.0075703180000000004 2024-09-14T06:29:02.870 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:02.870 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:02.870 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:29:02.870 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:28:45.186871+0000", 2024-09-14T06:29:02.870 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.0040000000000001e-05 2024-09-14T06:29:02.870 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:02.871 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:02.871 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "waiting for subops from 0", 2024-09-14T06:29:02.871 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:28:45.186964+0000", 2024-09-14T06:29:02.871 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.2200000000000005e-05 2024-09-14T06:29:02.871 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:02.871 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:02.871 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:29:02.871 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:28:45.187765+0000", 2024-09-14T06:29:02.871 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00080108499999999999 2024-09-14T06:29:02.871 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:02.871 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:02.871 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "op_commit", 2024-09-14T06:29:02.871 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:28:45.188729+0000", 2024-09-14T06:29:02.871 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00096401799999999997 2024-09-14T06:29:02.871 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:02.872 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:02.872 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "commit_sent", 2024-09-14T06:29:02.872 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:28:45.188795+0000", 2024-09-14T06:29:02.872 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.6591e-05 2024-09-14T06:29:02.872 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:02.872 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:02.872 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:29:02.872 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:28:45.188847+0000", 2024-09-14T06:29:02.872 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.1659999999999997e-05 2024-09-14T06:29:02.872 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:02.872 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:29:02.872 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:02.872 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:02.872 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:02.873 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4195.0:52 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-14T06:29:02.873 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:28:45.179251+0000", 2024-09-14T06:29:02.873 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 17.618186289, 2024-09-14T06:29:02.873 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.0096800199999999993, 2024-09-14T06:29:02.873 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:29:02.873 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:29:02.873 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:29:02.873 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:29:02.873 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4195", 2024-09-14T06:29:02.873 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.138:0/3795168327", 2024-09-14T06:29:02.873 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 52 2024-09-14T06:29:02.873 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:02.873 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:29:02.873 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:02.874 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:29:02.874 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:28:45.179251+0000", 2024-09-14T06:29:02.874 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:02.874 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:02.874 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:02.874 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:29:02.874 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:28:45.179251+0000", 2024-09-14T06:29:02.874 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:02.874 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:02.874 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:02.874 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:29:02.874 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:28:45.179251+0000", 2024-09-14T06:29:02.874 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.3799999999999997e-07 2024-09-14T06:29:02.874 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:02.874 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:02.875 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:29:02.875 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:28:45.179263+0000", 2024-09-14T06:29:02.875 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.136e-05 2024-09-14T06:29:02.875 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:02.875 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:02.875 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:29:02.875 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:28:45.179274+0000", 2024-09-14T06:29:02.875 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.1891e-05 2024-09-14T06:29:02.875 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:02.875 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:02.875 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:29:02.875 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:28:45.179286+0000", 2024-09-14T06:29:02.875 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.1403000000000001e-05 2024-09-14T06:29:02.875 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:02.876 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:02.876 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:29:02.876 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:28:45.187249+0000", 2024-09-14T06:29:02.876 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.0079630099999999995 2024-09-14T06:29:02.876 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:02.876 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:02.876 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:29:02.876 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:28:45.187302+0000", 2024-09-14T06:29:02.876 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.3448000000000001e-05 2024-09-14T06:29:02.876 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:02.876 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:02.876 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "waiting for subops from 0", 2024-09-14T06:29:02.876 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:28:45.187379+0000", 2024-09-14T06:29:02.876 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.7057000000000004e-05 2024-09-14T06:29:02.877 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:02.877 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:02.877 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:29:02.877 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:28:45.188060+0000", 2024-09-14T06:29:02.877 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00068107599999999997 2024-09-14T06:29:02.877 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:02.877 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:02.877 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "op_commit", 2024-09-14T06:29:02.877 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:28:45.188857+0000", 2024-09-14T06:29:02.877 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00079684799999999996 2024-09-14T06:29:02.877 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:02.877 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:02.877 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "commit_sent", 2024-09-14T06:29:02.877 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:28:45.188897+0000", 2024-09-14T06:29:02.878 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.9994999999999999e-05 2024-09-14T06:29:02.878 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:02.878 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:02.878 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:29:02.878 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:28:45.188931+0000", 2024-09-14T06:29:02.878 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.3393999999999999e-05 2024-09-14T06:29:02.878 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:02.878 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:29:02.878 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:02.878 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:02.878 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:02.878 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4195.0:66 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 516~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-14T06:29:02.878 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:28:45.409173+0000", 2024-09-14T06:29:02.878 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 17.38826388, 2024-09-14T06:29:02.879 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.0073280919999999996, 2024-09-14T06:29:02.879 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:29:02.879 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:29:02.879 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:29:02.879 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:29:02.879 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4195", 2024-09-14T06:29:02.879 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.138:0/3795168327", 2024-09-14T06:29:02.879 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 66 2024-09-14T06:29:02.879 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:02.879 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:29:02.879 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:02.879 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:29:02.879 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:28:45.409173+0000", 2024-09-14T06:29:02.879 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:02.879 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:02.880 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:02.880 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:29:02.880 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:28:45.409173+0000", 2024-09-14T06:29:02.880 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:02.880 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:02.880 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:02.880 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:29:02.880 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:28:45.409174+0000", 2024-09-14T06:29:02.880 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.2e-07 2024-09-14T06:29:02.880 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:02.880 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:02.880 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:29:02.880 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:28:45.409207+0000", 2024-09-14T06:29:02.880 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.2923999999999999e-05 2024-09-14T06:29:02.881 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:02.881 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:02.881 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:29:02.881 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:28:45.409213+0000", 2024-09-14T06:29:02.881 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.0519999999999998e-06 2024-09-14T06:29:02.881 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:02.881 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:02.881 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:29:02.881 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:28:45.409216+0000", 2024-09-14T06:29:02.881 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.698e-06 2024-09-14T06:29:02.881 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:02.881 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:02.881 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:29:02.882 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:28:45.410167+0000", 2024-09-14T06:29:02.882 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00095031100000000002 2024-09-14T06:29:02.882 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:02.882 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:02.882 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:29:02.882 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:28:45.410218+0000", 2024-09-14T06:29:02.882 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.0951000000000001e-05 2024-09-14T06:29:02.882 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:02.882 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:02.882 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "waiting for subops from 0", 2024-09-14T06:29:02.882 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:28:45.410296+0000", 2024-09-14T06:29:02.882 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.8178999999999998e-05 2024-09-14T06:29:02.882 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:02.882 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:02.882 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:29:02.883 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:28:45.412148+0000", 2024-09-14T06:29:02.883 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.0018521060000000001 2024-09-14T06:29:02.883 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:02.883 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:02.883 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "op_commit", 2024-09-14T06:29:02.883 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:28:45.416447+0000", 2024-09-14T06:29:02.883 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.0042988130000000003 2024-09-14T06:29:02.883 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:02.883 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:02.883 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "commit_sent", 2024-09-14T06:29:02.883 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:28:45.416476+0000", 2024-09-14T06:29:02.883 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.9439999999999999e-05 2024-09-14T06:29:02.883 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:02.883 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:02.884 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:29:02.884 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:28:45.416501+0000", 2024-09-14T06:29:02.884 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.5097999999999999e-05 2024-09-14T06:29:02.884 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:02.884 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:29:02.884 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:02.884 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:02.884 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:02.884 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4195.0:67 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66052~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-14T06:29:02.884 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:28:45.409226+0000", 2024-09-14T06:29:02.884 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 17.388210986000001, 2024-09-14T06:29:02.884 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.0073353799999999999, 2024-09-14T06:29:02.884 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:29:02.884 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:29:02.885 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:29:02.885 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:29:02.885 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4195", 2024-09-14T06:29:02.885 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.138:0/3795168327", 2024-09-14T06:29:02.885 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 67 2024-09-14T06:29:02.885 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:02.885 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:29:02.885 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:02.885 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:29:02.885 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:28:45.409226+0000", 2024-09-14T06:29:02.885 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:02.885 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:02.885 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:02.886 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:29:02.886 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:28:45.409226+0000", 2024-09-14T06:29:02.886 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:02.886 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:02.886 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:02.886 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:29:02.886 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:28:45.409226+0000", 2024-09-14T06:29:02.886 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.9000000000000002e-07 2024-09-14T06:29:02.886 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:02.886 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:02.886 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:29:02.886 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:28:45.409229+0000", 2024-09-14T06:29:02.886 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.3089999999999998e-06 2024-09-14T06:29:02.886 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:02.887 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:02.887 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:29:02.887 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:28:45.409233+0000", 2024-09-14T06:29:02.887 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.0450000000000002e-06 2024-09-14T06:29:02.887 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:02.887 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:02.887 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:29:02.887 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:28:45.409236+0000", 2024-09-14T06:29:02.887 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.18e-06 2024-09-14T06:29:02.887 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:02.887 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:02.887 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:29:02.887 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:28:45.411620+0000", 2024-09-14T06:29:02.888 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.0023840179999999999 2024-09-14T06:29:02.888 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:02.888 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:02.888 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:29:02.888 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:28:45.411670+0000", 2024-09-14T06:29:02.888 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.0476000000000003e-05 2024-09-14T06:29:02.888 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:02.888 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:02.888 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "waiting for subops from 0", 2024-09-14T06:29:02.888 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:28:45.411745+0000", 2024-09-14T06:29:02.888 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.4884999999999999e-05 2024-09-14T06:29:02.888 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:02.888 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:02.888 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:29:02.888 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:28:45.413291+0000", 2024-09-14T06:29:02.889 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.0015459670000000001 2024-09-14T06:29:02.889 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:02.889 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:02.889 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "op_commit", 2024-09-14T06:29:02.889 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:28:45.416507+0000", 2024-09-14T06:29:02.889 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.003215987 2024-09-14T06:29:02.889 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:02.889 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:02.889 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "commit_sent", 2024-09-14T06:29:02.889 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:28:45.416530+0000", 2024-09-14T06:29:02.889 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.2288000000000001e-05 2024-09-14T06:29:02.889 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:02.889 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:02.890 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:29:02.890 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:28:45.416561+0000", 2024-09-14T06:29:02.890 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.1835000000000003e-05 2024-09-14T06:29:02.890 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:02.890 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:29:02.890 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:02.890 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:02.890 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:02.890 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4195.0:68 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66056~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-14T06:29:02.890 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:28:45.409458+0000", 2024-09-14T06:29:02.890 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 17.387979296000001, 2024-09-14T06:29:02.890 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.0071585260000000001, 2024-09-14T06:29:02.890 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:29:02.890 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:29:02.890 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:29:02.891 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:29:02.891 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4195", 2024-09-14T06:29:02.891 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.138:0/3795168327", 2024-09-14T06:29:02.891 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 68 2024-09-14T06:29:02.891 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:02.891 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:29:02.891 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:02.891 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:29:02.891 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:28:45.409458+0000", 2024-09-14T06:29:02.891 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:02.891 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:02.891 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:02.891 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:29:02.891 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:28:45.409458+0000", 2024-09-14T06:29:02.891 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:02.892 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:02.892 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:02.892 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:29:02.892 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:28:45.409458+0000", 2024-09-14T06:29:02.892 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.9599999999999999e-07 2024-09-14T06:29:02.892 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:02.892 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:02.892 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:29:02.892 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:28:45.409462+0000", 2024-09-14T06:29:02.892 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.3060000000000002e-06 2024-09-14T06:29:02.892 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:02.892 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:02.892 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:29:02.892 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:28:45.409469+0000", 2024-09-14T06:29:02.892 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.2490000000000004e-06 2024-09-14T06:29:02.893 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:02.893 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:02.893 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:29:02.893 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:28:45.409473+0000", 2024-09-14T06:29:02.893 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.0450000000000002e-06 2024-09-14T06:29:02.893 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:02.893 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:02.893 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:29:02.893 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:28:45.412181+0000", 2024-09-14T06:29:02.893 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.002708055 2024-09-14T06:29:02.893 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:02.893 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:02.893 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:29:02.894 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:28:45.412229+0000", 2024-09-14T06:29:02.894 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.7579e-05 2024-09-14T06:29:02.894 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:02.894 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:02.894 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "waiting for subops from 0", 2024-09-14T06:29:02.894 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:28:45.412305+0000", 2024-09-14T06:29:02.894 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.6909000000000005e-05 2024-09-14T06:29:02.894 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:02.894 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:02.894 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:29:02.894 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:28:45.414230+0000", 2024-09-14T06:29:02.894 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.001924123 2024-09-14T06:29:02.894 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:02.894 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:02.894 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "op_commit", 2024-09-14T06:29:02.895 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:28:45.416567+0000", 2024-09-14T06:29:02.895 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.0023376310000000002 2024-09-14T06:29:02.895 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:02.895 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:02.895 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "commit_sent", 2024-09-14T06:29:02.895 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:28:45.416598+0000", 2024-09-14T06:29:02.895 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.0463e-05 2024-09-14T06:29:02.895 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:02.895 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:02.895 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:29:02.895 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:28:45.416616+0000", 2024-09-14T06:29:02.895 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.857e-05 2024-09-14T06:29:02.895 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:02.895 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:29:02.896 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:02.896 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:02.896 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:02.896 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4195.0:69 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-14T06:29:02.896 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:28:45.409515+0000", 2024-09-14T06:29:02.896 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 17.387922035999999, 2024-09-14T06:29:02.896 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.0084731589999999992, 2024-09-14T06:29:02.896 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:29:02.896 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:29:02.896 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:29:02.896 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:29:02.896 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4195", 2024-09-14T06:29:02.896 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.138:0/3795168327", 2024-09-14T06:29:02.896 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 69 2024-09-14T06:29:02.897 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:02.897 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:29:02.897 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:02.897 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:29:02.897 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:28:45.409515+0000", 2024-09-14T06:29:02.897 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:02.897 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:02.897 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:02.897 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:29:02.897 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:28:45.409515+0000", 2024-09-14T06:29:02.897 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:02.897 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:02.897 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:02.897 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:29:02.897 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:28:45.409516+0000", 2024-09-14T06:29:02.898 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.4700000000000001e-07 2024-09-14T06:29:02.898 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:02.898 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:02.898 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:29:02.898 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:28:45.409558+0000", 2024-09-14T06:29:02.898 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.1866999999999997e-05 2024-09-14T06:29:02.898 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:02.898 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:02.898 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:29:02.898 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:28:45.409564+0000", 2024-09-14T06:29:02.898 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.1589999999999998e-06 2024-09-14T06:29:02.898 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:02.898 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:02.898 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:29:02.898 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:28:45.409567+0000", 2024-09-14T06:29:02.899 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.7060000000000002e-06 2024-09-14T06:29:02.899 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:02.899 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:02.899 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:29:02.899 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:28:45.412626+0000", 2024-09-14T06:29:02.899 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.0030587259999999999 2024-09-14T06:29:02.899 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:02.899 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:02.899 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:29:02.899 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:28:45.412674+0000", 2024-09-14T06:29:02.899 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.7450000000000001e-05 2024-09-14T06:29:02.899 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:02.899 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:02.899 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "waiting for subops from 0", 2024-09-14T06:29:02.899 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:28:45.412763+0000", 2024-09-14T06:29:02.899 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.8970999999999999e-05 2024-09-14T06:29:02.900 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:02.900 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:02.900 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:29:02.900 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:28:45.414275+0000", 2024-09-14T06:29:02.900 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.0015125099999999999 2024-09-14T06:29:02.900 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:02.900 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:02.900 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "op_commit", 2024-09-14T06:29:02.900 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:28:45.417925+0000", 2024-09-14T06:29:02.900 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.003649926 2024-09-14T06:29:02.900 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:02.900 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:02.900 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "commit_sent", 2024-09-14T06:29:02.900 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:28:45.417963+0000", 2024-09-14T06:29:02.901 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.8374000000000002e-05 2024-09-14T06:29:02.901 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:02.901 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:02.901 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:29:02.901 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:28:45.417988+0000", 2024-09-14T06:29:02.901 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.4723e-05 2024-09-14T06:29:02.901 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:02.901 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:29:02.901 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:02.901 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:02.901 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:02.901 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4195.0:70 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-14T06:29:02.901 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:28:45.409575+0000", 2024-09-14T06:29:02.902 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 17.387861959999999, 2024-09-14T06:29:02.902 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.0084578740000000006, 2024-09-14T06:29:02.902 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:29:02.902 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:29:02.902 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:29:02.902 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:29:02.902 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4195", 2024-09-14T06:29:02.902 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.138:0/3795168327", 2024-09-14T06:29:02.902 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 70 2024-09-14T06:29:02.902 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:02.902 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:29:02.902 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:02.902 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:29:02.902 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:28:45.409575+0000", 2024-09-14T06:29:02.902 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:02.902 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:02.903 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:02.903 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:29:02.903 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:28:45.409575+0000", 2024-09-14T06:29:02.903 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:02.903 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:02.903 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:02.903 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:29:02.903 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:28:45.409575+0000", 2024-09-14T06:29:02.903 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.3200000000000001e-07 2024-09-14T06:29:02.903 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:02.903 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:02.903 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:29:02.903 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:28:45.409578+0000", 2024-09-14T06:29:02.903 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.667e-06 2024-09-14T06:29:02.903 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:02.903 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:02.904 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:29:02.904 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:28:45.409586+0000", 2024-09-14T06:29:02.904 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.5530000000000004e-06 2024-09-14T06:29:02.904 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:02.904 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:02.904 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:29:02.904 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:28:45.409589+0000", 2024-09-14T06:29:02.904 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.2399999999999999e-06 2024-09-14T06:29:02.904 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:02.904 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:02.904 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:29:02.904 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:28:45.413324+0000", 2024-09-14T06:29:02.904 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.003735487 2024-09-14T06:29:02.904 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:02.905 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:02.905 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:29:02.905 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:28:45.413376+0000", 2024-09-14T06:29:02.905 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.1576000000000002e-05 2024-09-14T06:29:02.905 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:02.905 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:02.905 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "waiting for subops from 0", 2024-09-14T06:29:02.905 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:28:45.413468+0000", 2024-09-14T06:29:02.905 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.1867000000000007e-05 2024-09-14T06:29:02.905 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:02.905 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:02.905 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:29:02.905 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:28:45.415349+0000", 2024-09-14T06:29:02.905 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00188134 2024-09-14T06:29:02.906 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:02.906 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:02.906 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "op_commit", 2024-09-14T06:29:02.906 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:28:45.417994+0000", 2024-09-14T06:29:02.906 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.0026445079999999998 2024-09-14T06:29:02.906 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:02.906 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:02.906 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "commit_sent", 2024-09-14T06:29:02.906 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:28:45.418016+0000", 2024-09-14T06:29:02.906 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.2334999999999999e-05 2024-09-14T06:29:02.906 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:02.906 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:02.906 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:29:02.906 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:28:45.418033+0000", 2024-09-14T06:29:02.906 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.6969e-05 2024-09-14T06:29:02.907 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:02.907 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:29:02.907 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:02.907 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:02.907 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:02.907 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4195.0:71 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131600~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-14T06:29:02.907 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:28:45.409593+0000", 2024-09-14T06:29:02.907 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 17.38784398, 2024-09-14T06:29:02.907 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.0084860820000000007, 2024-09-14T06:29:02.907 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:29:02.907 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:29:02.907 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:29:02.907 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:29:02.908 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4195", 2024-09-14T06:29:02.908 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.138:0/3795168327", 2024-09-14T06:29:02.908 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 71 2024-09-14T06:29:02.908 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:02.908 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:29:02.908 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:02.908 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:29:02.908 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:28:45.409593+0000", 2024-09-14T06:29:02.908 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:02.908 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:02.908 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:02.908 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:29:02.908 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:28:45.409593+0000", 2024-09-14T06:29:02.908 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:02.908 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:02.909 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:02.909 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:29:02.909 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:28:45.409593+0000", 2024-09-14T06:29:02.909 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.29e-07 2024-09-14T06:29:02.909 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:02.909 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:02.909 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:29:02.909 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:28:45.409595+0000", 2024-09-14T06:29:02.909 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.0940000000000002e-06 2024-09-14T06:29:02.909 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:02.909 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:02.909 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:29:02.909 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:28:45.409600+0000", 2024-09-14T06:29:02.910 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.6929999999999998e-06 2024-09-14T06:29:02.910 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:02.910 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:02.910 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:29:02.910 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:28:45.409606+0000", 2024-09-14T06:29:02.910 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.7710000000000003e-06 2024-09-14T06:29:02.910 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:02.910 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:02.910 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:29:02.910 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:28:45.413730+0000", 2024-09-14T06:29:02.910 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.0041246700000000004 2024-09-14T06:29:02.910 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:02.910 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:02.910 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:29:02.911 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:28:45.413778+0000", 2024-09-14T06:29:02.911 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.7163000000000002e-05 2024-09-14T06:29:02.911 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:02.911 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:02.911 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "waiting for subops from 0", 2024-09-14T06:29:02.911 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:28:45.413854+0000", 2024-09-14T06:29:02.911 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.6477000000000007e-05 2024-09-14T06:29:02.911 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:02.911 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:02.911 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:29:02.911 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:28:45.415399+0000", 2024-09-14T06:29:02.911 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.001544964 2024-09-14T06:29:02.911 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:02.911 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:02.912 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "op_commit", 2024-09-14T06:29:02.912 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:28:45.418036+0000", 2024-09-14T06:29:02.912 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.0026369700000000002 2024-09-14T06:29:02.912 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:02.912 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:02.912 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "commit_sent", 2024-09-14T06:29:02.912 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:28:45.418056+0000", 2024-09-14T06:29:02.912 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.0146999999999999e-05 2024-09-14T06:29:02.912 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:02.912 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:02.912 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:29:02.912 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:28:45.418079+0000", 2024-09-14T06:29:02.912 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.2904e-05 2024-09-14T06:29:02.912 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:02.913 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:29:02.913 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:02.913 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:02.913 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:02.913 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4195.0:72 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-14T06:29:02.913 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:28:45.409617+0000", 2024-09-14T06:29:02.913 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 17.387819943, 2024-09-14T06:29:02.913 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.0085144520000000005, 2024-09-14T06:29:02.913 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:29:02.913 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:29:02.913 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:29:02.913 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:29:02.913 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4195", 2024-09-14T06:29:02.913 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.138:0/3795168327", 2024-09-14T06:29:02.914 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 72 2024-09-14T06:29:02.914 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:02.914 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:29:02.914 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:02.914 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:29:02.914 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:28:45.409617+0000", 2024-09-14T06:29:02.914 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:02.914 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:02.914 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:02.914 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:29:02.914 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:28:45.409617+0000", 2024-09-14T06:29:02.914 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:02.914 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:02.914 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:02.915 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:29:02.915 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:28:45.409617+0000", 2024-09-14T06:29:02.915 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.2500000000000001e-07 2024-09-14T06:29:02.915 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:02.915 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:02.915 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:29:02.915 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:28:45.409674+0000", 2024-09-14T06:29:02.915 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.6453999999999999e-05 2024-09-14T06:29:02.915 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:02.915 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:02.915 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:29:02.915 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:28:45.409683+0000", 2024-09-14T06:29:02.915 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.8210000000000008e-06 2024-09-14T06:29:02.915 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:02.915 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:02.915 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:29:02.916 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:28:45.409689+0000", 2024-09-14T06:29:02.916 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.2489999999999998e-06 2024-09-14T06:29:02.916 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:02.916 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:02.916 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:29:02.916 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:28:45.414296+0000", 2024-09-14T06:29:02.916 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.0046065280000000004 2024-09-14T06:29:02.916 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:02.916 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:02.916 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:29:02.916 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:28:45.414347+0000", 2024-09-14T06:29:02.916 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.1863999999999997e-05 2024-09-14T06:29:02.916 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:02.916 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:02.917 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "waiting for subops from 0", 2024-09-14T06:29:02.917 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:28:45.414432+0000", 2024-09-14T06:29:02.917 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.4379e-05 2024-09-14T06:29:02.917 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:02.917 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:02.917 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:29:02.917 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:28:45.416660+0000", 2024-09-14T06:29:02.917 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.0022279130000000002 2024-09-14T06:29:02.917 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:02.917 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:02.917 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "op_commit", 2024-09-14T06:29:02.917 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:28:45.418084+0000", 2024-09-14T06:29:02.917 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.0014245429999999999 2024-09-14T06:29:02.917 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:02.917 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:02.918 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "commit_sent", 2024-09-14T06:29:02.918 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:28:45.418113+0000", 2024-09-14T06:29:02.918 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.8960999999999999e-05 2024-09-14T06:29:02.918 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:02.918 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:02.918 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:29:02.918 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:28:45.418131+0000", 2024-09-14T06:29:02.918 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.8315e-05 2024-09-14T06:29:02.918 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:02.918 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:29:02.918 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:02.918 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:02.918 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:02.918 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4195.0:73 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-14T06:29:02.918 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:28:45.409702+0000", 2024-09-14T06:29:02.918 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 17.387735101000001, 2024-09-14T06:29:02.919 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.0084798410000000001, 2024-09-14T06:29:02.919 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:29:02.919 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:29:02.919 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:29:02.919 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:29:02.919 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4195", 2024-09-14T06:29:02.919 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.138:0/3795168327", 2024-09-14T06:29:02.919 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 73 2024-09-14T06:29:02.919 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:02.919 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:29:02.919 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:02.919 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:29:02.919 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:28:45.409702+0000", 2024-09-14T06:29:02.919 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:02.919 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:02.920 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:02.920 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:29:02.920 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:28:45.409702+0000", 2024-09-14T06:29:02.920 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:02.920 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:02.920 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:02.920 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:29:02.920 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:28:45.409703+0000", 2024-09-14T06:29:02.920 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.0699999999999996e-07 2024-09-14T06:29:02.920 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:02.920 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:02.920 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:29:02.920 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:28:45.409707+0000", 2024-09-14T06:29:02.920 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.0799999999999999e-06 2024-09-14T06:29:02.920 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:02.921 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:02.921 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:29:02.921 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:28:45.409714+0000", 2024-09-14T06:29:02.921 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.8680000000000001e-06 2024-09-14T06:29:02.921 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:02.921 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:02.921 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:29:02.921 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:28:45.409722+0000", 2024-09-14T06:29:02.921 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.8699999999999998e-06 2024-09-14T06:29:02.921 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:02.921 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:02.921 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:29:02.921 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:28:45.414888+0000", 2024-09-14T06:29:02.921 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.0051651420000000002 2024-09-14T06:29:02.922 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:02.922 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:02.922 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:29:02.922 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:28:45.414944+0000", 2024-09-14T06:29:02.922 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.5998000000000002e-05 2024-09-14T06:29:02.922 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:02.922 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:02.922 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "waiting for subops from 0", 2024-09-14T06:29:02.922 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:28:45.415021+0000", 2024-09-14T06:29:02.922 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.7367000000000007e-05 2024-09-14T06:29:02.922 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:02.922 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:02.922 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:29:02.922 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:28:45.416702+0000", 2024-09-14T06:29:02.922 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00168144 2024-09-14T06:29:02.922 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:02.923 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:02.923 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "op_commit", 2024-09-14T06:29:02.923 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:28:45.418136+0000", 2024-09-14T06:29:02.923 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00143318 2024-09-14T06:29:02.923 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:02.923 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:02.923 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "commit_sent", 2024-09-14T06:29:02.923 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:28:45.418157+0000", 2024-09-14T06:29:02.923 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.1571000000000001e-05 2024-09-14T06:29:02.923 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:02.923 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:02.923 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:29:02.923 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:28:45.418182+0000", 2024-09-14T06:29:02.923 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.4518000000000001e-05 2024-09-14T06:29:02.923 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:02.924 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:29:02.924 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:02.924 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:02.924 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:02.924 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4195.0:74 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-14T06:29:02.924 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:28:45.409740+0000", 2024-09-14T06:29:02.924 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 17.387696633000001, 2024-09-14T06:29:02.924 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.0087373780000000005, 2024-09-14T06:29:02.924 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:29:02.924 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:29:02.924 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:29:02.924 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:29:02.924 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4195", 2024-09-14T06:29:02.925 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.138:0/3795168327", 2024-09-14T06:29:02.925 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 74 2024-09-14T06:29:02.925 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:02.925 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:29:02.925 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:02.925 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:29:02.925 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:28:45.409740+0000", 2024-09-14T06:29:02.925 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:02.925 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:02.925 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:02.925 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:29:02.925 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:28:45.409740+0000", 2024-09-14T06:29:02.925 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:02.926 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:02.926 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:02.926 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:29:02.926 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:28:45.409741+0000", 2024-09-14T06:29:02.926 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.7000000000000004e-07 2024-09-14T06:29:02.926 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:02.926 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:02.926 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:29:02.926 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:28:45.409746+0000", 2024-09-14T06:29:02.926 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.8269999999999997e-06 2024-09-14T06:29:02.926 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:02.926 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:02.926 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:29:02.927 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:28:45.409756+0000", 2024-09-14T06:29:02.927 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.8660000000000001e-06 2024-09-14T06:29:02.927 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:02.927 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:02.927 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:29:02.927 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:28:45.409763+0000", 2024-09-14T06:29:02.927 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.2590000000000003e-06 2024-09-14T06:29:02.927 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:02.927 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:02.927 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:29:02.927 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:28:45.415429+0000", 2024-09-14T06:29:02.927 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.0056658050000000003 2024-09-14T06:29:02.927 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:02.928 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:02.928 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:29:02.928 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:28:45.415480+0000", 2024-09-14T06:29:02.928 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.1007999999999997e-05 2024-09-14T06:29:02.928 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:02.928 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:02.928 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "waiting for subops from 0", 2024-09-14T06:29:02.928 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:28:45.415562+0000", 2024-09-14T06:29:02.928 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.1860999999999994e-05 2024-09-14T06:29:02.928 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:02.928 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:02.928 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:29:02.928 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:28:45.416730+0000", 2024-09-14T06:29:02.929 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.001168271 2024-09-14T06:29:02.929 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:02.929 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:02.929 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "op_commit", 2024-09-14T06:29:02.929 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:28:45.418417+0000", 2024-09-14T06:29:02.929 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.0016873770000000001 2024-09-14T06:29:02.929 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:02.929 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:02.929 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "commit_sent", 2024-09-14T06:29:02.929 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:28:45.418456+0000", 2024-09-14T06:29:02.929 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.9209999999999999e-05 2024-09-14T06:29:02.929 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:02.929 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:02.929 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:29:02.930 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:28:45.418478+0000", 2024-09-14T06:29:02.930 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.1223999999999999e-05 2024-09-14T06:29:02.930 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:02.930 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:29:02.930 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:02.930 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:02.930 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:02.930 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4195.0:75 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-14T06:29:02.930 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:28:45.409774+0000", 2024-09-14T06:29:02.930 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 17.387662476999999, 2024-09-14T06:29:02.930 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.0087573210000000002, 2024-09-14T06:29:02.930 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:29:02.930 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:29:02.930 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:29:02.930 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:29:02.930 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4195", 2024-09-14T06:29:02.930 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.138:0/3795168327", 2024-09-14T06:29:02.931 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 75 2024-09-14T06:29:02.931 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:02.931 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:29:02.931 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:02.931 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:29:02.931 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:28:45.409774+0000", 2024-09-14T06:29:02.931 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:02.931 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:02.931 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:02.931 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:29:02.931 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:28:45.409774+0000", 2024-09-14T06:29:02.931 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:02.931 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:02.932 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:02.932 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:29:02.932 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:28:45.409775+0000", 2024-09-14T06:29:02.932 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.5499999999999998e-07 2024-09-14T06:29:02.932 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:02.932 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:02.932 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:29:02.932 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:28:45.409821+0000", 2024-09-14T06:29:02.932 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.6440000000000003e-05 2024-09-14T06:29:02.932 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:02.932 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:02.932 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:29:02.932 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:28:45.409839+0000", 2024-09-14T06:29:02.933 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.7039999999999999e-05 2024-09-14T06:29:02.933 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:02.933 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:02.933 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:29:02.933 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:28:45.409847+0000", 2024-09-14T06:29:02.933 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.3850000000000002e-06 2024-09-14T06:29:02.933 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:02.933 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:02.933 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:29:02.933 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:28:45.415820+0000", 2024-09-14T06:29:02.933 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.0059730139999999996 2024-09-14T06:29:02.933 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:02.933 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:02.933 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:29:02.934 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:28:45.415869+0000", 2024-09-14T06:29:02.934 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.8746999999999999e-05 2024-09-14T06:29:02.934 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:02.934 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:02.934 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "waiting for subops from 0", 2024-09-14T06:29:02.934 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:28:45.415951+0000", 2024-09-14T06:29:02.934 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.2126999999999995e-05 2024-09-14T06:29:02.934 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:02.934 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:02.934 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:29:02.934 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:28:45.417189+0000", 2024-09-14T06:29:02.934 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.0012384869999999999 2024-09-14T06:29:02.934 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:02.934 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:02.934 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "op_commit", 2024-09-14T06:29:02.935 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:28:45.418483+0000", 2024-09-14T06:29:02.935 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.001293275 2024-09-14T06:29:02.935 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:02.935 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:02.935 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "commit_sent", 2024-09-14T06:29:02.935 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:28:45.418512+0000", 2024-09-14T06:29:02.935 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.9203000000000001e-05 2024-09-14T06:29:02.935 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:02.935 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:02.935 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:29:02.935 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:28:45.418532+0000", 2024-09-14T06:29:02.935 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.0047999999999998e-05 2024-09-14T06:29:02.935 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:02.935 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:29:02.935 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:02.936 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:02.936 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:02.936 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4195.0:76 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-14T06:29:02.936 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:28:45.409859+0000", 2024-09-14T06:29:02.936 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 17.387577572000001, 2024-09-14T06:29:02.936 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.0087223149999999996, 2024-09-14T06:29:02.936 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:29:02.936 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:29:02.936 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:29:02.936 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:29:02.936 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4195", 2024-09-14T06:29:02.936 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.138:0/3795168327", 2024-09-14T06:29:02.936 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 76 2024-09-14T06:29:02.936 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:02.937 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:29:02.937 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:02.937 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:29:02.937 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:28:45.409859+0000", 2024-09-14T06:29:02.937 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:02.937 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:02.937 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:02.937 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:29:02.937 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:28:45.409859+0000", 2024-09-14T06:29:02.937 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:02.937 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:02.937 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:02.937 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:29:02.937 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:28:45.409860+0000", 2024-09-14T06:29:02.938 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.3499999999999995e-07 2024-09-14T06:29:02.938 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:02.938 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:02.938 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:29:02.938 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:28:45.409865+0000", 2024-09-14T06:29:02.938 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.6750000000000001e-06 2024-09-14T06:29:02.938 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:02.938 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:02.938 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:29:02.938 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:28:45.409873+0000", 2024-09-14T06:29:02.938 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.9060000000000002e-06 2024-09-14T06:29:02.938 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:02.938 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:02.938 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:29:02.938 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:28:45.409880+0000", 2024-09-14T06:29:02.938 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.7920000000000004e-06 2024-09-14T06:29:02.939 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:02.939 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:02.939 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:29:02.939 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:28:45.416748+0000", 2024-09-14T06:29:02.939 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.0068687189999999997 2024-09-14T06:29:02.939 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:02.939 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:02.939 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:29:02.939 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:28:45.416795+0000", 2024-09-14T06:29:02.939 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.7125e-05 2024-09-14T06:29:02.939 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:02.939 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:02.939 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "waiting for subops from 0", 2024-09-14T06:29:02.939 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:28:45.416871+0000", 2024-09-14T06:29:02.939 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.5903999999999997e-05 2024-09-14T06:29:02.939 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:02.940 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:02.940 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:29:02.940 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:28:45.417672+0000", 2024-09-14T06:29:02.940 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00080118799999999999 2024-09-14T06:29:02.940 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:02.940 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:02.940 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "op_commit", 2024-09-14T06:29:02.940 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:28:45.418536+0000", 2024-09-14T06:29:02.940 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00086370700000000002 2024-09-14T06:29:02.940 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:02.940 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:02.940 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "commit_sent", 2024-09-14T06:29:02.940 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:28:45.418564+0000", 2024-09-14T06:29:02.940 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.7580999999999999e-05 2024-09-14T06:29:02.940 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:02.941 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:02.941 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:29:02.941 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:28:45.418582+0000", 2024-09-14T06:29:02.941 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.7983e-05 2024-09-14T06:29:02.941 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:02.941 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:29:02.941 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:02.941 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:02.941 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:29:02.941 INFO:teuthology.orchestra.run.smithi138.stdout:} 2024-09-14T06:29:02.943 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-14T06:29:02.943 INFO:tasks.ceph.osd.3.smithi094.stderr:2024-09-14T06:29:02.941+0000 7feeeafe8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33655) UID: 0 2024-09-14T06:29:03.043 INFO:tasks.ceph.osd.4.smithi138.stderr:2024-09-14T06:29:03.042+0000 7f6c653c8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33615) UID: 0 2024-09-14T06:29:03.048 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-14T06:29:03.048 INFO:teuthology.orchestra.run.smithi094.stdout: "ops": [], 2024-09-14T06:29:03.048 INFO:teuthology.orchestra.run.smithi094.stdout: "num_ops": 0 2024-09-14T06:29:03.048 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-14T06:29:03.057 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-14T06:29:03.145 INFO:tasks.ceph.osd.3.smithi094.stderr:2024-09-14T06:29:03.144+0000 7feeeafe8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33655) UID: 0 2024-09-14T06:29:03.161 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-14T06:29:03.161 INFO:teuthology.orchestra.run.smithi094.stdout: "ops": [], 2024-09-14T06:29:03.161 INFO:teuthology.orchestra.run.smithi094.stdout: "complaint_time": 30, 2024-09-14T06:29:03.161 INFO:teuthology.orchestra.run.smithi094.stdout: "num_blocked_ops": 0 2024-09-14T06:29:03.162 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-14T06:29:03.171 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-14T06:29:03.255 INFO:tasks.ceph.osd.0.smithi094.stderr:2024-09-14T06:29:03.254+0000 7f07c9333640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33636) UID: 0 2024-09-14T06:29:03.319 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-14T06:29:03.319 INFO:teuthology.orchestra.run.smithi094.stdout: "size": 20, 2024-09-14T06:29:03.319 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 600, 2024-09-14T06:29:03.319 INFO:teuthology.orchestra.run.smithi094.stdout: "ops": [ 2024-09-14T06:29:03.319 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:03.319 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4195.0:14 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'5, mlcod=17'3)", 2024-09-14T06:29:03.320 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:28:44.853346+0000", 2024-09-14T06:29:03.320 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 18.463302018, 2024-09-14T06:29:03.320 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00084974400000000004, 2024-09-14T06:29:03.320 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:29:03.320 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:29:03.320 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:29:03.320 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:29:03.320 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:03.320 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:29:03.320 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:28:44.853346+0000", 2024-09-14T06:29:03.320 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:03.320 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:03.321 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:03.321 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:29:03.321 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:28:44.853346+0000", 2024-09-14T06:29:03.321 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:03.321 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:03.321 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:03.321 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:29:03.321 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:28:44.853348+0000", 2024-09-14T06:29:03.321 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.8330000000000001e-06 2024-09-14T06:29:03.321 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:03.321 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:03.321 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:29:03.321 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:28:44.853359+0000", 2024-09-14T06:29:03.321 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.163e-05 2024-09-14T06:29:03.322 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:03.322 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:03.322 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:29:03.322 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:28:44.853386+0000", 2024-09-14T06:29:03.322 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.6596000000000001e-05 2024-09-14T06:29:03.322 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:03.322 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:03.322 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:29:03.322 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:28:44.853401+0000", 2024-09-14T06:29:03.322 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.4742999999999999e-05 2024-09-14T06:29:03.322 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:03.323 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:03.323 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:29:03.323 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:28:44.853522+0000", 2024-09-14T06:29:03.323 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000121212 2024-09-14T06:29:03.323 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:03.323 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:03.323 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:29:03.323 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:28:44.853587+0000", 2024-09-14T06:29:03.323 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.5089e-05 2024-09-14T06:29:03.323 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:03.323 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:03.324 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-14T06:29:03.324 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:28:44.854142+0000", 2024-09-14T06:29:03.324 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00055446100000000002 2024-09-14T06:29:03.324 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:03.324 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:03.324 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:29:03.324 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:28:44.854196+0000", 2024-09-14T06:29:03.324 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.418e-05 2024-09-14T06:29:03.324 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:03.324 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:03.324 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:03.324 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:03.325 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:03.325 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4195.0:20 1.0 e17/16 1:30306672:devicehealth::main.db.0000000000000000:head v 17'10, mlcod=17'8)", 2024-09-14T06:29:03.325 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:28:44.859997+0000", 2024-09-14T06:29:03.325 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 18.456651378, 2024-09-14T06:29:03.325 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0012158620000000001, 2024-09-14T06:29:03.325 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:29:03.325 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:29:03.325 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:29:03.325 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:29:03.325 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:03.325 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:29:03.325 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:28:44.859997+0000", 2024-09-14T06:29:03.325 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:03.326 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:03.326 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:03.326 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:29:03.326 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:28:44.859997+0000", 2024-09-14T06:29:03.326 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:03.326 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:03.326 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:03.326 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:29:03.326 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:28:44.859997+0000", 2024-09-14T06:29:03.326 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.8599999999999998e-07 2024-09-14T06:29:03.326 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:03.326 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:03.326 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:29:03.326 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:28:44.860132+0000", 2024-09-14T06:29:03.327 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000135136 2024-09-14T06:29:03.327 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:03.327 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:03.327 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:29:03.327 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:28:44.860142+0000", 2024-09-14T06:29:03.327 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.0084999999999999e-05 2024-09-14T06:29:03.327 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:03.327 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:03.327 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:29:03.327 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:28:44.860165+0000", 2024-09-14T06:29:03.327 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.2745999999999999e-05 2024-09-14T06:29:03.327 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:03.327 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:03.327 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:29:03.328 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:28:44.860218+0000", 2024-09-14T06:29:03.328 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.2840000000000002e-05 2024-09-14T06:29:03.328 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:03.328 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:03.328 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:29:03.328 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:28:44.860239+0000", 2024-09-14T06:29:03.328 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.1310000000000001e-05 2024-09-14T06:29:03.328 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:03.328 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:03.328 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-14T06:29:03.328 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:28:44.861172+0000", 2024-09-14T06:29:03.328 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00093291999999999995 2024-09-14T06:29:03.328 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:03.329 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:03.329 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:29:03.329 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:28:44.861212+0000", 2024-09-14T06:29:03.329 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.0139000000000003e-05 2024-09-14T06:29:03.329 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:03.329 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:03.329 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:03.329 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:03.329 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:03.329 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4195.0:22 1.0 e17/16 1:30306672:devicehealth::main.db.0000000000000000:head v 17'12, mlcod=17'8)", 2024-09-14T06:29:03.329 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:28:44.861435+0000", 2024-09-14T06:29:03.329 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 18.455212538000001, 2024-09-14T06:29:03.329 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00090736099999999997, 2024-09-14T06:29:03.329 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:29:03.330 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:29:03.330 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:29:03.330 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:29:03.330 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:03.330 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:29:03.330 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:28:44.861435+0000", 2024-09-14T06:29:03.330 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:03.330 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:03.330 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:03.330 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:29:03.330 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:28:44.861435+0000", 2024-09-14T06:29:03.330 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:03.330 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:03.330 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:03.331 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:29:03.331 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:28:44.861436+0000", 2024-09-14T06:29:03.331 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.0399999999999995e-07 2024-09-14T06:29:03.331 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:03.331 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:03.331 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:29:03.331 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:28:44.861568+0000", 2024-09-14T06:29:03.331 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000132371 2024-09-14T06:29:03.331 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:03.331 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:03.331 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:29:03.331 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:28:44.861580+0000", 2024-09-14T06:29:03.331 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.1440999999999999e-05 2024-09-14T06:29:03.331 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:03.332 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:03.332 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:29:03.332 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:28:44.861585+0000", 2024-09-14T06:29:03.332 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.3199999999999999e-06 2024-09-14T06:29:03.332 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:03.332 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:03.332 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:29:03.332 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:28:44.861626+0000", 2024-09-14T06:29:03.332 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.0428e-05 2024-09-14T06:29:03.332 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:03.332 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:03.332 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:29:03.332 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:28:44.861643+0000", 2024-09-14T06:29:03.332 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.702e-05 2024-09-14T06:29:03.333 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:03.333 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:03.333 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-14T06:29:03.333 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:28:44.862306+0000", 2024-09-14T06:29:03.333 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000662837 2024-09-14T06:29:03.333 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:03.333 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:03.333 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:29:03.333 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:28:44.862343+0000", 2024-09-14T06:29:03.333 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.7240000000000003e-05 2024-09-14T06:29:03.333 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:03.333 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:03.333 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:03.333 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:03.333 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:03.334 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4195.0:23 1.0 e17/16 1:30306672:devicehealth::main.db.0000000000000000:head v 17'13, mlcod=17'8)", 2024-09-14T06:29:03.334 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:28:44.861832+0000", 2024-09-14T06:29:03.334 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 18.454815985, 2024-09-14T06:29:03.334 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00083773400000000003, 2024-09-14T06:29:03.334 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:29:03.334 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:29:03.334 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:29:03.334 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:29:03.334 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:03.334 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:29:03.334 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:28:44.861832+0000", 2024-09-14T06:29:03.334 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:03.334 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:03.335 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:03.335 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:29:03.335 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:28:44.861832+0000", 2024-09-14T06:29:03.335 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:03.335 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:03.335 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:03.335 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:29:03.335 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:28:44.861833+0000", 2024-09-14T06:29:03.335 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.8500000000000001e-07 2024-09-14T06:29:03.335 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:03.335 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:03.335 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:29:03.335 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:28:44.861893+0000", 2024-09-14T06:29:03.336 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.0132e-05 2024-09-14T06:29:03.336 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:03.336 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:03.336 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:29:03.336 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:28:44.861903+0000", 2024-09-14T06:29:03.336 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.8209999999999997e-06 2024-09-14T06:29:03.336 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:03.336 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:03.336 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:29:03.336 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:28:44.861914+0000", 2024-09-14T06:29:03.336 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.1445e-05 2024-09-14T06:29:03.336 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:03.336 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:03.336 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:29:03.337 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:28:44.861981+0000", 2024-09-14T06:29:03.337 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.6679000000000003e-05 2024-09-14T06:29:03.337 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:03.337 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:03.337 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:29:03.337 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:28:44.861999+0000", 2024-09-14T06:29:03.337 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.8573999999999999e-05 2024-09-14T06:29:03.337 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:03.337 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:03.337 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-14T06:29:03.337 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:28:44.862638+0000", 2024-09-14T06:29:03.337 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000638778 2024-09-14T06:29:03.337 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:03.338 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:03.338 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:29:03.338 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:28:44.862670+0000", 2024-09-14T06:29:03.338 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.1619999999999999e-05 2024-09-14T06:29:03.338 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:03.338 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:03.338 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:03.338 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:03.338 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:03.338 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4195.0:26 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'16, mlcod=17'14)", 2024-09-14T06:29:03.338 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:28:44.868168+0000", 2024-09-14T06:29:03.338 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 18.448480376999999, 2024-09-14T06:29:03.338 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00095204700000000003, 2024-09-14T06:29:03.338 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:29:03.339 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:29:03.339 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:29:03.339 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:29:03.339 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:03.339 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:29:03.339 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:28:44.868168+0000", 2024-09-14T06:29:03.339 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:03.339 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:03.339 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:03.339 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:29:03.339 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:28:44.868168+0000", 2024-09-14T06:29:03.339 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:03.339 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:03.339 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:03.340 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:29:03.340 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:28:44.868168+0000", 2024-09-14T06:29:03.340 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.2500000000000004e-07 2024-09-14T06:29:03.340 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:03.340 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:03.340 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:29:03.340 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:28:44.868174+0000", 2024-09-14T06:29:03.340 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.0279999999999997e-06 2024-09-14T06:29:03.340 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:03.340 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:03.340 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:29:03.340 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:28:44.868186+0000", 2024-09-14T06:29:03.340 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.1272e-05 2024-09-14T06:29:03.341 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:03.341 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:03.341 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:29:03.341 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:28:44.868209+0000", 2024-09-14T06:29:03.341 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.2969e-05 2024-09-14T06:29:03.341 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:03.341 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:03.341 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:29:03.341 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:28:44.868263+0000", 2024-09-14T06:29:03.341 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.4225999999999999e-05 2024-09-14T06:29:03.341 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:03.341 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:03.341 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:29:03.341 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:28:44.868285+0000", 2024-09-14T06:29:03.341 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.2566e-05 2024-09-14T06:29:03.342 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:03.342 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:03.342 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-14T06:29:03.342 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:28:44.869083+0000", 2024-09-14T06:29:03.342 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00079728199999999996 2024-09-14T06:29:03.342 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:03.342 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:03.342 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:29:03.342 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:28:44.869120+0000", 2024-09-14T06:29:03.342 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.6878999999999997e-05 2024-09-14T06:29:03.342 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:03.342 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:03.342 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:03.343 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:03.343 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:03.343 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4195.0:34 1.0 e17/16 1:30306672:devicehealth::main.db.0000000000000000:head v 17'20, mlcod=17'18)", 2024-09-14T06:29:03.343 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:28:45.175764+0000", 2024-09-14T06:29:03.343 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 18.140884077999999, 2024-09-14T06:29:03.343 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00093694399999999999, 2024-09-14T06:29:03.343 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:29:03.343 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:29:03.343 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:29:03.343 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:29:03.343 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:03.343 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:29:03.343 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:28:45.175764+0000", 2024-09-14T06:29:03.343 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:03.343 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:03.344 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:03.344 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:29:03.344 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:28:45.175764+0000", 2024-09-14T06:29:03.344 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:03.344 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:03.344 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:03.344 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:29:03.344 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:28:45.175767+0000", 2024-09-14T06:29:03.344 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.202e-06 2024-09-14T06:29:03.344 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:03.344 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:03.344 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:29:03.344 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:28:45.175790+0000", 2024-09-14T06:29:03.344 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.2946e-05 2024-09-14T06:29:03.345 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:03.345 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:03.345 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:29:03.345 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:28:45.175836+0000", 2024-09-14T06:29:03.345 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.5649999999999998e-05 2024-09-14T06:29:03.345 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:03.345 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:03.345 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:29:03.345 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:28:45.175852+0000", 2024-09-14T06:29:03.345 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.5843000000000001e-05 2024-09-14T06:29:03.345 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:03.345 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:03.345 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:29:03.346 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:28:45.175946+0000", 2024-09-14T06:29:03.346 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.4783000000000004e-05 2024-09-14T06:29:03.346 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:03.346 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:03.346 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:29:03.346 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:28:45.175974+0000", 2024-09-14T06:29:03.346 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.7330000000000001e-05 2024-09-14T06:29:03.346 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:03.346 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:03.346 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-14T06:29:03.346 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:28:45.176654+0000", 2024-09-14T06:29:03.346 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00068052399999999995 2024-09-14T06:29:03.346 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:03.346 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:03.347 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:29:03.347 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:28:45.176701+0000", 2024-09-14T06:29:03.347 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.6665999999999997e-05 2024-09-14T06:29:03.347 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:03.347 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:03.347 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:03.347 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:03.347 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:03.347 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4195.0:39 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'21, mlcod=17'19)", 2024-09-14T06:29:03.347 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:28:45.178796+0000", 2024-09-14T06:29:03.347 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 18.137851895000001, 2024-09-14T06:29:03.347 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00096248299999999998, 2024-09-14T06:29:03.347 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:29:03.347 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:29:03.347 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:29:03.348 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:29:03.348 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:03.348 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:29:03.348 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:28:45.178796+0000", 2024-09-14T06:29:03.348 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:03.348 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:03.348 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:03.348 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:29:03.348 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:28:45.178796+0000", 2024-09-14T06:29:03.348 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:03.348 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:03.348 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:03.348 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:29:03.349 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:28:45.178797+0000", 2024-09-14T06:29:03.349 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.7400000000000003e-07 2024-09-14T06:29:03.349 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:03.349 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:03.349 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:29:03.349 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:28:45.178829+0000", 2024-09-14T06:29:03.349 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.2546000000000003e-05 2024-09-14T06:29:03.349 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:03.349 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:03.349 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:29:03.349 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:28:45.178840+0000", 2024-09-14T06:29:03.349 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.0818999999999999e-05 2024-09-14T06:29:03.350 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:03.350 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:03.350 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:29:03.350 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:28:45.178846+0000", 2024-09-14T06:29:03.350 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.5779999999999998e-06 2024-09-14T06:29:03.350 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:03.350 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:03.350 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:29:03.350 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:28:45.178888+0000", 2024-09-14T06:29:03.350 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.2721e-05 2024-09-14T06:29:03.350 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:03.350 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:03.350 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:29:03.351 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:28:45.178907+0000", 2024-09-14T06:29:03.351 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.8700999999999999e-05 2024-09-14T06:29:03.351 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:03.351 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:03.351 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-14T06:29:03.351 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:28:45.179718+0000", 2024-09-14T06:29:03.351 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00081047499999999995 2024-09-14T06:29:03.351 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:03.351 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:03.351 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:29:03.351 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:28:45.179759+0000", 2024-09-14T06:29:03.351 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.0969e-05 2024-09-14T06:29:03.351 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:03.351 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:03.352 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:03.352 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:03.352 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:03.352 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4195.0:40 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'22, mlcod=17'19)", 2024-09-14T06:29:03.352 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:28:45.179376+0000", 2024-09-14T06:29:03.352 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 18.137271626, 2024-09-14T06:29:03.352 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.001227375, 2024-09-14T06:29:03.352 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:29:03.352 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:29:03.352 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:29:03.352 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:29:03.352 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:03.352 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:29:03.352 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:28:45.179376+0000", 2024-09-14T06:29:03.353 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:03.353 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:03.353 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:03.353 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:29:03.353 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:28:45.179376+0000", 2024-09-14T06:29:03.353 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:03.353 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:03.353 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:03.353 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:29:03.353 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:28:45.179377+0000", 2024-09-14T06:29:03.353 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.0599999999999999e-07 2024-09-14T06:29:03.353 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:03.353 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:03.354 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:29:03.354 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:28:45.179383+0000", 2024-09-14T06:29:03.354 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.964e-06 2024-09-14T06:29:03.354 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:03.354 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:03.354 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:29:03.354 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:28:45.179392+0000", 2024-09-14T06:29:03.354 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.0019999999999995e-06 2024-09-14T06:29:03.354 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:03.354 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:03.354 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:29:03.354 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:28:45.179398+0000", 2024-09-14T06:29:03.354 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.1240000000000002e-06 2024-09-14T06:29:03.354 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:03.355 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:03.355 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:29:03.355 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:28:45.179449+0000", 2024-09-14T06:29:03.355 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.0312000000000002e-05 2024-09-14T06:29:03.355 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:03.355 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:03.355 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:29:03.355 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:28:45.179473+0000", 2024-09-14T06:29:03.355 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.4621999999999999e-05 2024-09-14T06:29:03.355 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:03.355 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:03.355 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-14T06:29:03.355 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:28:45.180555+0000", 2024-09-14T06:29:03.355 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.001082237 2024-09-14T06:29:03.355 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:03.356 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:03.356 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:29:03.356 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:28:45.180604+0000", 2024-09-14T06:29:03.356 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.8208000000000003e-05 2024-09-14T06:29:03.356 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:03.356 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:03.356 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:03.356 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:03.356 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:03.356 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4195.0:41 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'23, mlcod=17'19)", 2024-09-14T06:29:03.356 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:28:45.179986+0000", 2024-09-14T06:29:03.356 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 18.136662221000002, 2024-09-14T06:29:03.356 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0011278600000000001, 2024-09-14T06:29:03.356 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:29:03.356 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:29:03.357 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:29:03.357 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:29:03.357 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:03.357 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:29:03.357 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:28:45.179986+0000", 2024-09-14T06:29:03.357 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:03.357 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:03.357 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:03.357 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:29:03.357 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:28:45.179986+0000", 2024-09-14T06:29:03.357 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:03.357 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:03.357 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:03.357 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:29:03.357 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:28:45.179987+0000", 2024-09-14T06:29:03.358 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.1800000000000004e-07 2024-09-14T06:29:03.358 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:03.358 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:03.358 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:29:03.358 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:28:45.180067+0000", 2024-09-14T06:29:03.358 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.0687999999999997e-05 2024-09-14T06:29:03.358 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:03.358 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:03.358 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:29:03.358 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:28:45.180080+0000", 2024-09-14T06:29:03.358 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.2295e-05 2024-09-14T06:29:03.358 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:03.358 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:03.358 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:29:03.359 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:28:45.180087+0000", 2024-09-14T06:29:03.359 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.0849999999999999e-06 2024-09-14T06:29:03.359 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:03.359 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:03.359 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:29:03.359 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:28:45.180130+0000", 2024-09-14T06:29:03.359 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.337e-05 2024-09-14T06:29:03.359 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:03.359 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:03.359 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:29:03.359 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:28:45.180149+0000", 2024-09-14T06:29:03.359 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.8950999999999999e-05 2024-09-14T06:29:03.359 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:03.359 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:03.360 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-14T06:29:03.360 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:28:45.181075+0000", 2024-09-14T06:29:03.360 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00092617199999999995 2024-09-14T06:29:03.360 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:03.360 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:03.360 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:29:03.360 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:28:45.181114+0000", 2024-09-14T06:29:03.360 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.8380999999999997e-05 2024-09-14T06:29:03.360 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:03.360 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:03.360 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:03.360 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:03.360 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:03.360 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4195.0:53 1.0 e17/16 1:30306672:devicehealth::main.db.0000000000000000:head v 17'35, mlcod=17'33)", 2024-09-14T06:29:03.361 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:28:45.189435+0000", 2024-09-14T06:29:03.361 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 18.127212737000001, 2024-09-14T06:29:03.361 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00097472699999999999, 2024-09-14T06:29:03.361 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:29:03.361 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:29:03.361 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:29:03.361 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:29:03.361 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:03.361 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:29:03.361 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:28:45.189435+0000", 2024-09-14T06:29:03.361 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:03.361 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:03.361 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:03.361 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:29:03.361 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:28:45.189435+0000", 2024-09-14T06:29:03.361 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:03.362 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:03.362 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:03.362 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:29:03.362 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:28:45.189436+0000", 2024-09-14T06:29:03.362 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.012e-06 2024-09-14T06:29:03.362 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:03.362 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:03.362 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:29:03.362 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:28:45.189492+0000", 2024-09-14T06:29:03.362 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.5831999999999997e-05 2024-09-14T06:29:03.362 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:03.362 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:03.362 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:29:03.362 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:28:45.189504+0000", 2024-09-14T06:29:03.362 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.1725999999999999e-05 2024-09-14T06:29:03.363 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:03.363 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:03.363 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:29:03.363 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:28:45.189509+0000", 2024-09-14T06:29:03.363 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.339e-06 2024-09-14T06:29:03.363 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:03.363 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:03.363 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:29:03.363 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:28:45.189557+0000", 2024-09-14T06:29:03.363 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.8151e-05 2024-09-14T06:29:03.364 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:03.364 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:03.364 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:29:03.364 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:28:45.189578+0000", 2024-09-14T06:29:03.364 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.071e-05 2024-09-14T06:29:03.364 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:03.364 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:03.364 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-14T06:29:03.364 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:28:45.190361+0000", 2024-09-14T06:29:03.364 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00078262600000000002 2024-09-14T06:29:03.364 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:03.364 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:03.365 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:29:03.365 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:28:45.190410+0000", 2024-09-14T06:29:03.365 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.9330999999999999e-05 2024-09-14T06:29:03.365 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:03.365 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:03.365 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:03.365 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:03.365 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:03.365 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4195.0:59 1.0 e17/16 1:30306672:devicehealth::main.db.0000000000000000:head v 17'41, mlcod=17'39)", 2024-09-14T06:29:03.365 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:28:45.406369+0000", 2024-09-14T06:29:03.365 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 17.910279109000001, 2024-09-14T06:29:03.365 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0010286469999999999, 2024-09-14T06:29:03.365 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:29:03.365 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:29:03.366 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:29:03.366 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:29:03.366 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:03.366 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:29:03.366 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:28:45.406369+0000", 2024-09-14T06:29:03.366 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:03.366 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:03.366 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:03.366 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:29:03.366 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:28:45.406369+0000", 2024-09-14T06:29:03.366 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:03.366 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:03.366 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:03.366 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:29:03.367 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:28:45.406371+0000", 2024-09-14T06:29:03.367 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.2929999999999999e-06 2024-09-14T06:29:03.367 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:03.367 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:03.367 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:29:03.367 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:28:45.406386+0000", 2024-09-14T06:29:03.367 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.4946999999999999e-05 2024-09-14T06:29:03.367 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:03.367 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:03.367 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:29:03.367 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:28:45.406415+0000", 2024-09-14T06:29:03.367 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.9116e-05 2024-09-14T06:29:03.367 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:03.367 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:03.368 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:29:03.368 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:28:45.406433+0000", 2024-09-14T06:29:03.368 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.7410000000000001e-05 2024-09-14T06:29:03.368 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:03.368 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:03.368 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:29:03.368 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:28:45.406577+0000", 2024-09-14T06:29:03.368 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000144034 2024-09-14T06:29:03.368 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:03.368 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:03.368 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:29:03.368 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:28:45.406640+0000", 2024-09-14T06:29:03.368 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.3398999999999994e-05 2024-09-14T06:29:03.368 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:03.369 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:03.369 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-14T06:29:03.369 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:28:45.407359+0000", 2024-09-14T06:29:03.369 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00071911700000000002 2024-09-14T06:29:03.369 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:03.369 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:03.369 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:29:03.369 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:28:45.407397+0000", 2024-09-14T06:29:03.369 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.8331000000000003e-05 2024-09-14T06:29:03.369 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:03.369 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:03.369 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:03.369 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:03.369 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:03.370 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4195.0:64 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'42, mlcod=17'40)", 2024-09-14T06:29:03.370 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:28:45.409373+0000", 2024-09-14T06:29:03.370 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 17.907275098, 2024-09-14T06:29:03.370 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00088970800000000001, 2024-09-14T06:29:03.370 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:29:03.370 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:29:03.370 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:29:03.370 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:29:03.370 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:03.370 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:29:03.370 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:28:45.409373+0000", 2024-09-14T06:29:03.370 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:03.370 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:03.371 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:03.371 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:29:03.371 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:28:45.409373+0000", 2024-09-14T06:29:03.371 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:03.371 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:03.371 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:03.371 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:29:03.371 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:28:45.409391+0000", 2024-09-14T06:29:03.371 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.8314000000000001e-05 2024-09-14T06:29:03.371 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:03.371 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:03.371 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:29:03.371 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:28:45.409396+0000", 2024-09-14T06:29:03.372 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.4279999999999998e-06 2024-09-14T06:29:03.372 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:03.372 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:03.372 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:29:03.372 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:28:45.409405+0000", 2024-09-14T06:29:03.372 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.7010000000000008e-06 2024-09-14T06:29:03.372 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:03.372 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:03.372 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:29:03.372 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:28:45.409410+0000", 2024-09-14T06:29:03.372 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.7190000000000001e-06 2024-09-14T06:29:03.372 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:03.372 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:03.372 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:29:03.372 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:28:45.409453+0000", 2024-09-14T06:29:03.373 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.3399000000000002e-05 2024-09-14T06:29:03.373 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:03.373 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:03.373 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:29:03.373 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:28:45.409470+0000", 2024-09-14T06:29:03.373 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.6659000000000001e-05 2024-09-14T06:29:03.373 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:03.373 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:03.373 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-14T06:29:03.373 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:28:45.410229+0000", 2024-09-14T06:29:03.373 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00075934899999999998 2024-09-14T06:29:03.373 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:03.373 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:03.374 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:29:03.374 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:28:45.410263+0000", 2024-09-14T06:29:03.374 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.3139000000000002e-05 2024-09-14T06:29:03.374 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:03.374 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:03.374 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:03.374 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:03.374 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:03.374 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4195.0:65 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'43, mlcod=17'40)", 2024-09-14T06:29:03.374 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:28:45.409898+0000", 2024-09-14T06:29:03.374 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 17.906749660999999, 2024-09-14T06:29:03.374 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00099306000000000004, 2024-09-14T06:29:03.374 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:29:03.374 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:29:03.375 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:29:03.375 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:29:03.375 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:03.375 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:29:03.375 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:28:45.409898+0000", 2024-09-14T06:29:03.375 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:03.375 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:03.375 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:03.375 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:29:03.375 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:28:45.409898+0000", 2024-09-14T06:29:03.375 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:03.375 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:03.375 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:03.376 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:29:03.376 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:28:45.409899+0000", 2024-09-14T06:29:03.376 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.3000000000000001e-07 2024-09-14T06:29:03.376 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:03.376 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:03.376 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:29:03.376 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:28:45.409921+0000", 2024-09-14T06:29:03.376 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.2566999999999999e-05 2024-09-14T06:29:03.376 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:03.376 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:03.376 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:29:03.376 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:28:45.409932+0000", 2024-09-14T06:29:03.376 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.0759e-05 2024-09-14T06:29:03.376 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:03.376 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:03.377 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:29:03.377 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:28:45.409938+0000", 2024-09-14T06:29:03.377 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.2829999999999998e-06 2024-09-14T06:29:03.377 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:03.377 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:03.377 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:29:03.377 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:28:45.409996+0000", 2024-09-14T06:29:03.377 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.8004000000000002e-05 2024-09-14T06:29:03.377 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:03.377 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:03.377 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:29:03.377 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:28:45.410014+0000", 2024-09-14T06:29:03.377 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.7790000000000001e-05 2024-09-14T06:29:03.377 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:03.378 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:03.378 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-14T06:29:03.378 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:28:45.410859+0000", 2024-09-14T06:29:03.378 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00084497000000000001 2024-09-14T06:29:03.378 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:03.378 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:03.378 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:29:03.378 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:28:45.410891+0000", 2024-09-14T06:29:03.378 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.2156999999999997e-05 2024-09-14T06:29:03.378 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:03.378 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:03.378 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:03.378 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:03.378 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:03.378 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4195.0:68 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'46, mlcod=17'41)", 2024-09-14T06:29:03.378 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:28:45.412374+0000", 2024-09-14T06:29:03.379 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 17.904273456999999, 2024-09-14T06:29:03.379 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00099083799999999996, 2024-09-14T06:29:03.379 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:29:03.379 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:29:03.379 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:29:03.379 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:29:03.379 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:03.379 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:29:03.379 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:28:45.412374+0000", 2024-09-14T06:29:03.379 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:03.379 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:03.379 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:03.379 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:29:03.379 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:28:45.412374+0000", 2024-09-14T06:29:03.379 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:03.380 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:03.380 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:03.380 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:29:03.380 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:28:45.412375+0000", 2024-09-14T06:29:03.380 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.6000000000000003e-07 2024-09-14T06:29:03.380 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:03.380 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:03.380 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:29:03.380 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:28:45.412380+0000", 2024-09-14T06:29:03.380 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.1880000000000001e-06 2024-09-14T06:29:03.380 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:03.380 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:03.380 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:29:03.381 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:28:45.412389+0000", 2024-09-14T06:29:03.381 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.8279999999999992e-06 2024-09-14T06:29:03.381 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:03.381 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:03.381 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:29:03.381 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:28:45.412394+0000", 2024-09-14T06:29:03.381 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.4730000000000002e-06 2024-09-14T06:29:03.381 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:03.381 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:03.381 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:29:03.381 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:28:45.412433+0000", 2024-09-14T06:29:03.381 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.9375999999999997e-05 2024-09-14T06:29:03.381 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:03.382 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:03.382 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:29:03.382 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:28:45.412451+0000", 2024-09-14T06:29:03.382 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.7462999999999999e-05 2024-09-14T06:29:03.382 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:03.382 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:03.382 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-14T06:29:03.382 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:28:45.413335+0000", 2024-09-14T06:29:03.382 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00088460899999999996 2024-09-14T06:29:03.382 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:03.382 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:03.382 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:29:03.382 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:28:45.413365+0000", 2024-09-14T06:29:03.383 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.0141000000000002e-05 2024-09-14T06:29:03.383 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:03.383 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:03.383 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:03.383 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:03.383 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:03.383 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4195.0:69 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'47, mlcod=17'41)", 2024-09-14T06:29:03.383 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:28:45.412825+0000", 2024-09-14T06:29:03.383 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 17.903823006, 2024-09-14T06:29:03.383 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000989415, 2024-09-14T06:29:03.383 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:29:03.383 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:29:03.383 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:29:03.383 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:29:03.384 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:03.384 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:29:03.384 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:28:45.412825+0000", 2024-09-14T06:29:03.384 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:03.384 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:03.384 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:03.384 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:29:03.384 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:28:45.412825+0000", 2024-09-14T06:29:03.384 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:03.384 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:03.384 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:03.384 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:29:03.384 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:28:45.412826+0000", 2024-09-14T06:29:03.385 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.61e-07 2024-09-14T06:29:03.385 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:03.385 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:03.385 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:29:03.385 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:28:45.412869+0000", 2024-09-14T06:29:03.385 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.3229999999999998e-05 2024-09-14T06:29:03.385 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:03.385 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:03.385 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:29:03.385 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:28:45.412880+0000", 2024-09-14T06:29:03.385 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.1266e-05 2024-09-14T06:29:03.385 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:03.385 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:03.386 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:29:03.386 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:28:45.412885+0000", 2024-09-14T06:29:03.386 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.3299999999999998e-06 2024-09-14T06:29:03.386 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:03.386 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:03.386 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:29:03.386 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:28:45.412930+0000", 2024-09-14T06:29:03.386 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.4910999999999997e-05 2024-09-14T06:29:03.386 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:03.386 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:03.386 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:29:03.386 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:28:45.412949+0000", 2024-09-14T06:29:03.387 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.8576e-05 2024-09-14T06:29:03.387 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:03.387 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:03.387 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-14T06:29:03.387 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:28:45.413776+0000", 2024-09-14T06:29:03.387 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00082660899999999996 2024-09-14T06:29:03.387 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:03.387 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:03.387 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:29:03.387 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:28:45.413814+0000", 2024-09-14T06:29:03.387 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.8831999999999997e-05 2024-09-14T06:29:03.387 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:03.387 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:03.387 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:03.388 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:03.388 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:03.388 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4195.0:72 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'50, mlcod=17'42)", 2024-09-14T06:29:03.388 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:28:45.414470+0000", 2024-09-14T06:29:03.388 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 17.902177525999999, 2024-09-14T06:29:03.388 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00091175000000000002, 2024-09-14T06:29:03.388 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:29:03.388 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:29:03.388 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:29:03.388 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:29:03.388 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:03.388 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:29:03.388 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:28:45.414470+0000", 2024-09-14T06:29:03.388 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:03.388 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:03.389 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:03.389 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:29:03.389 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:28:45.414470+0000", 2024-09-14T06:29:03.389 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:03.389 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:03.389 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:03.389 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:29:03.389 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:28:45.414471+0000", 2024-09-14T06:29:03.389 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.2500000000000001e-07 2024-09-14T06:29:03.389 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:03.389 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:03.389 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:29:03.389 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:28:45.414543+0000", 2024-09-14T06:29:03.390 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.1699000000000006e-05 2024-09-14T06:29:03.390 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:03.390 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:03.390 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:29:03.390 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:28:45.414550+0000", 2024-09-14T06:29:03.390 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.0110000000000003e-06 2024-09-14T06:29:03.390 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:03.390 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:03.390 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:29:03.390 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:28:45.414553+0000", 2024-09-14T06:29:03.390 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.4000000000000001e-06 2024-09-14T06:29:03.390 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:03.390 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:03.390 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:29:03.391 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:28:45.414587+0000", 2024-09-14T06:29:03.391 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.3630000000000002e-05 2024-09-14T06:29:03.391 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:03.391 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:03.391 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:29:03.391 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:28:45.414604+0000", 2024-09-14T06:29:03.391 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.7563000000000001e-05 2024-09-14T06:29:03.391 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:03.391 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:03.391 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-14T06:29:03.391 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:28:45.415350+0000", 2024-09-14T06:29:03.391 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00074571900000000005 2024-09-14T06:29:03.391 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:03.392 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:03.392 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:29:03.392 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:28:45.415382+0000", 2024-09-14T06:29:03.392 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.2302999999999999e-05 2024-09-14T06:29:03.392 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:03.392 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:03.392 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:03.392 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:03.392 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:03.392 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4195.0:78 1.0 e17/16 1:30306672:devicehealth::main.db.0000000000000000:head v 17'56, mlcod=17'53)", 2024-09-14T06:29:03.392 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:28:45.419490+0000", 2024-09-14T06:29:03.392 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 17.897158418, 2024-09-14T06:29:03.392 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00082279100000000002, 2024-09-14T06:29:03.392 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:29:03.393 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:29:03.393 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:29:03.393 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:29:03.393 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:03.393 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:29:03.393 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:28:45.419490+0000", 2024-09-14T06:29:03.393 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:03.393 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:03.393 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:03.393 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:29:03.393 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:28:45.419490+0000", 2024-09-14T06:29:03.393 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:03.393 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:03.393 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:03.394 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:29:03.394 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:28:45.419490+0000", 2024-09-14T06:29:03.394 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.13e-07 2024-09-14T06:29:03.394 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:03.394 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:03.394 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:29:03.394 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:28:45.419549+0000", 2024-09-14T06:29:03.394 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.8570000000000003e-05 2024-09-14T06:29:03.394 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:03.394 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:03.394 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:29:03.394 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:28:45.419557+0000", 2024-09-14T06:29:03.394 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.1589999999999993e-06 2024-09-14T06:29:03.394 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:03.394 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:03.395 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:29:03.395 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:28:45.419561+0000", 2024-09-14T06:29:03.395 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.1579999999999998e-06 2024-09-14T06:29:03.395 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:03.395 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:03.395 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:29:03.395 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:28:45.419595+0000", 2024-09-14T06:29:03.395 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.4295999999999998e-05 2024-09-14T06:29:03.395 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:03.395 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:03.395 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:29:03.395 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:28:45.419607+0000", 2024-09-14T06:29:03.395 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.1888999999999999e-05 2024-09-14T06:29:03.395 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:03.396 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:03.396 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-14T06:29:03.396 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:28:45.420286+0000", 2024-09-14T06:29:03.396 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00067916899999999997 2024-09-14T06:29:03.396 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:03.396 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:03.396 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:29:03.396 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:28:45.420312+0000", 2024-09-14T06:29:03.396 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.6037000000000002e-05 2024-09-14T06:29:03.396 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:03.396 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:03.396 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:03.396 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:03.396 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:03.397 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4195.0:90 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'64, mlcod=17'60)", 2024-09-14T06:29:03.397 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:28:45.689107+0000", 2024-09-14T06:29:03.397 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 17.627540839000002, 2024-09-14T06:29:03.397 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00084026000000000001, 2024-09-14T06:29:03.397 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:29:03.397 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:29:03.397 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:29:03.397 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:29:03.397 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:03.397 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:29:03.397 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:28:45.689107+0000", 2024-09-14T06:29:03.397 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:03.397 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:03.397 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:03.397 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:29:03.398 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:28:45.689107+0000", 2024-09-14T06:29:03.398 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:03.398 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:03.398 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:03.398 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:29:03.398 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:28:45.689108+0000", 2024-09-14T06:29:03.398 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.0999999999999999e-07 2024-09-14T06:29:03.398 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:03.398 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:03.398 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:29:03.398 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:28:45.689201+0000", 2024-09-14T06:29:03.398 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.3419999999999997e-05 2024-09-14T06:29:03.398 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:03.398 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:03.398 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:29:03.398 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:28:45.689207+0000", 2024-09-14T06:29:03.399 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.5910000000000003e-06 2024-09-14T06:29:03.399 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:03.399 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:03.399 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:29:03.399 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:28:45.689210+0000", 2024-09-14T06:29:03.399 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.8310000000000003e-06 2024-09-14T06:29:03.399 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:03.399 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:03.399 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:29:03.399 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:28:45.689239+0000", 2024-09-14T06:29:03.399 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.8807e-05 2024-09-14T06:29:03.399 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:03.399 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:03.399 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:29:03.399 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:28:45.689250+0000", 2024-09-14T06:29:03.400 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.1257000000000001e-05 2024-09-14T06:29:03.400 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:03.400 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:03.400 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-14T06:29:03.400 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:28:45.689925+0000", 2024-09-14T06:29:03.400 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00067502199999999995 2024-09-14T06:29:03.400 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:03.400 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:03.400 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:29:03.400 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:28:45.689947+0000", 2024-09-14T06:29:03.400 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.1922000000000001e-05 2024-09-14T06:29:03.400 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:03.400 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:03.400 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:03.400 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:03.401 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:03.401 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4195.0:92 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'66, mlcod=17'62)", 2024-09-14T06:29:03.401 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:28:45.690456+0000", 2024-09-14T06:29:03.401 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 17.626192406000001, 2024-09-14T06:29:03.401 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00089540400000000003, 2024-09-14T06:29:03.401 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:29:03.401 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:29:03.401 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:29:03.401 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:29:03.401 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:03.401 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:29:03.401 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:28:45.690456+0000", 2024-09-14T06:29:03.401 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:03.401 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:03.401 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:03.402 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:29:03.402 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:28:45.690456+0000", 2024-09-14T06:29:03.402 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:03.402 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:03.402 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:03.402 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:29:03.402 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:28:45.690456+0000", 2024-09-14T06:29:03.402 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.3500000000000002e-07 2024-09-14T06:29:03.402 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:03.402 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:03.402 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:29:03.402 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:28:45.690459+0000", 2024-09-14T06:29:03.402 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.1039999999999998e-06 2024-09-14T06:29:03.402 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:03.402 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:03.403 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:29:03.403 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:28:45.690465+0000", 2024-09-14T06:29:03.403 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.3879999999999997e-06 2024-09-14T06:29:03.403 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:03.403 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:03.403 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:29:03.403 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:28:45.690469+0000", 2024-09-14T06:29:03.403 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.4659999999999999e-06 2024-09-14T06:29:03.403 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:03.403 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:03.403 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:29:03.403 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:28:45.690501+0000", 2024-09-14T06:29:03.403 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.2319e-05 2024-09-14T06:29:03.403 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:03.403 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:03.403 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:29:03.404 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:28:45.690511+0000", 2024-09-14T06:29:03.404 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.8600000000000005e-06 2024-09-14T06:29:03.404 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:03.404 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:03.404 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-14T06:29:03.404 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:28:45.691325+0000", 2024-09-14T06:29:03.404 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00081355299999999995 2024-09-14T06:29:03.404 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:03.404 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:03.404 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:29:03.404 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:28:45.691351+0000", 2024-09-14T06:29:03.404 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.6279e-05 2024-09-14T06:29:03.404 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:03.404 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:03.404 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:03.405 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:03.405 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:03.405 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:11 3.1f 3:fa654f18:::rbd_id.image_client.0:head [create,call rbd.set_id in=9b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-14T06:29:03.405 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:02.586739+0000", 2024-09-14T06:29:03.405 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 0.72990891899999999, 2024-09-14T06:29:03.405 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.001178023, 2024-09-14T06:29:03.405 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:29:03.405 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:29:03.405 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:29:03.405 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:29:03.405 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:29:03.405 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:03.405 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 11 2024-09-14T06:29:03.406 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:03.406 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:29:03.406 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:03.406 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:29:03.406 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:02.586739+0000", 2024-09-14T06:29:03.406 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:03.406 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:03.406 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:03.406 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:29:03.406 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:02.586739+0000", 2024-09-14T06:29:03.406 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:03.406 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:03.406 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:03.406 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:29:03.406 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:02.586740+0000", 2024-09-14T06:29:03.406 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.238e-06 2024-09-14T06:29:03.407 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:03.407 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:03.407 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:29:03.407 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:02.586746+0000", 2024-09-14T06:29:03.407 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.8490000000000002e-06 2024-09-14T06:29:03.407 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:03.407 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:03.407 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:29:03.407 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:02.586758+0000", 2024-09-14T06:29:03.407 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.2164000000000001e-05 2024-09-14T06:29:03.407 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:03.407 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:03.407 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:29:03.407 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:02.586766+0000", 2024-09-14T06:29:03.408 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.6299999999999998e-06 2024-09-14T06:29:03.408 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:03.408 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:03.408 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:29:03.408 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:02.586812+0000", 2024-09-14T06:29:03.408 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.6075999999999997e-05 2024-09-14T06:29:03.408 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:03.408 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:03.408 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:29:03.408 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:02.586913+0000", 2024-09-14T06:29:03.408 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000100871 2024-09-14T06:29:03.408 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:03.408 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:03.409 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "waiting for subops from 5", 2024-09-14T06:29:03.409 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:02.587040+0000", 2024-09-14T06:29:03.409 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00012704 2024-09-14T06:29:03.409 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:03.409 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:03.409 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "op_commit", 2024-09-14T06:29:03.409 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:02.587612+0000", 2024-09-14T06:29:03.409 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00057181099999999998 2024-09-14T06:29:03.409 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:03.409 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:03.409 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:29:03.409 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:02.587856+0000", 2024-09-14T06:29:03.409 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00024470699999999999 2024-09-14T06:29:03.409 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:03.410 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:03.410 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-14T06:29:03.410 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:02.587889+0000", 2024-09-14T06:29:03.410 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.2737999999999997e-05 2024-09-14T06:29:03.410 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:03.410 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:03.410 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:29:03.410 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:02.587917+0000", 2024-09-14T06:29:03.410 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.7898999999999998e-05 2024-09-14T06:29:03.410 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:03.410 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:03.410 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:03.410 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:03.410 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:03.411 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-14T06:29:03.412 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-14T06:29:03.412 INFO:tasks.ceph.osd.7.smithi138.stderr:2024-09-14T06:29:03.410+0000 7f3c6a21d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33623) UID: 0 2024-09-14T06:29:03.513 INFO:tasks.ceph.osd.6.smithi138.stderr:2024-09-14T06:29:03.511+0000 7f8cd8849640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33620) UID: 0 2024-09-14T06:29:03.557 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-14T06:29:03.557 INFO:teuthology.orchestra.run.smithi094.stdout: "ops": [], 2024-09-14T06:29:03.557 INFO:teuthology.orchestra.run.smithi094.stdout: "num_ops": 0 2024-09-14T06:29:03.557 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-14T06:29:03.566 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-14T06:29:03.582 INFO:teuthology.orchestra.run.smithi094.stderr:nodeep-scrub is set 2024-09-14T06:29:03.638 INFO:tasks.ceph.osd.3.smithi094.stderr:2024-09-14T06:29:03.637+0000 7feeeafe8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33655) UID: 0 2024-09-14T06:29:03.673 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-14T06:29:03.674 INFO:teuthology.orchestra.run.smithi094.stdout: "ops": [], 2024-09-14T06:29:03.674 INFO:teuthology.orchestra.run.smithi094.stdout: "complaint_time": 30, 2024-09-14T06:29:03.674 INFO:teuthology.orchestra.run.smithi094.stdout: "num_blocked_ops": 0 2024-09-14T06:29:03.674 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-14T06:29:03.682 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-14T06:29:03.723 INFO:tasks.ceph.osd.3.smithi094.stderr:2024-09-14T06:29:03.722+0000 7feeeafe8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33655) UID: 0 2024-09-14T06:29:03.816 INFO:tasks.ceph.osd.0.smithi094.stderr:2024-09-14T06:29:03.815+0000 7f07c9333640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33636) UID: 0 2024-09-14T06:29:03.827 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-14T06:29:03.827 INFO:teuthology.orchestra.run.smithi094.stdout: "size": 20, 2024-09-14T06:29:03.827 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 600, 2024-09-14T06:29:03.828 INFO:teuthology.orchestra.run.smithi094.stdout: "ops": [ 2024-09-14T06:29:03.828 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:03.828 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4299.0:5 2.2 e21/19 2:5cea7035:::rbd_info:head v 21'1, mlcod=0'0)", 2024-09-14T06:29:03.828 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:28:52.233676+0000", 2024-09-14T06:29:03.828 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 11.592970108999999, 2024-09-14T06:29:03.828 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.001150833, 2024-09-14T06:29:03.828 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:29:03.828 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:29:03.828 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:29:03.828 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:29:03.828 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:03.828 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:29:03.828 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:28:52.233676+0000", 2024-09-14T06:29:03.829 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:03.829 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:03.829 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:03.829 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:29:03.829 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:28:52.233676+0000", 2024-09-14T06:29:03.829 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:03.829 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:03.829 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:03.829 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:29:03.829 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:28:52.233676+0000", 2024-09-14T06:29:03.829 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.8099999999999999e-07 2024-09-14T06:29:03.829 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:03.829 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:03.830 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:29:03.830 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:28:52.233703+0000", 2024-09-14T06:29:03.830 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.6798e-05 2024-09-14T06:29:03.830 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:03.830 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:03.830 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:29:03.830 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:28:52.233715+0000", 2024-09-14T06:29:03.830 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.2516e-05 2024-09-14T06:29:03.830 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:03.830 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:03.830 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:29:03.830 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:28:52.233723+0000", 2024-09-14T06:29:03.830 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.3580000000000003e-06 2024-09-14T06:29:03.830 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:03.831 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:03.831 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:29:03.831 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:28:52.233786+0000", 2024-09-14T06:29:03.831 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.2434999999999994e-05 2024-09-14T06:29:03.831 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:03.831 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:03.831 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:29:03.831 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:28:52.233827+0000", 2024-09-14T06:29:03.831 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.1344999999999998e-05 2024-09-14T06:29:03.831 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:03.831 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:03.831 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-14T06:29:03.831 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:28:52.234770+0000", 2024-09-14T06:29:03.831 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00094315000000000002 2024-09-14T06:29:03.831 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:03.832 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:03.832 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:29:03.832 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:28:52.234826+0000", 2024-09-14T06:29:03.832 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.605e-05 2024-09-14T06:29:03.832 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:03.832 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:03.832 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:03.832 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:03.832 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:03.832 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4299.0:7 2.2 e22/19 2:5cea7035:::rbd_info:head v 22'2, mlcod=0'0)", 2024-09-14T06:29:03.832 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:28:53.231020+0000", 2024-09-14T06:29:03.832 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 10.595625786999999, 2024-09-14T06:29:03.832 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00094937099999999996, 2024-09-14T06:29:03.832 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:29:03.832 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:29:03.833 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:29:03.833 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:29:03.833 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:03.833 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:29:03.833 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:28:53.231020+0000", 2024-09-14T06:29:03.833 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:03.833 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:03.833 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:03.833 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:29:03.833 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:28:53.231020+0000", 2024-09-14T06:29:03.833 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:03.833 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:03.833 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:03.833 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:29:03.834 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:28:53.231020+0000", 2024-09-14T06:29:03.834 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.4400000000000001e-07 2024-09-14T06:29:03.834 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:03.834 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:03.834 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:29:03.834 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:28:53.231029+0000", 2024-09-14T06:29:03.834 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.0429999999999996e-06 2024-09-14T06:29:03.834 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:03.834 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:03.834 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:29:03.834 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:28:53.231044+0000", 2024-09-14T06:29:03.834 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.4584e-05 2024-09-14T06:29:03.834 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:03.834 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:03.835 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:29:03.835 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:28:53.231052+0000", 2024-09-14T06:29:03.835 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.3189999999999995e-06 2024-09-14T06:29:03.835 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:03.835 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:03.835 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:29:03.835 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:28:53.231128+0000", 2024-09-14T06:29:03.835 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.5872999999999997e-05 2024-09-14T06:29:03.835 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:03.835 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:03.835 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:29:03.835 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:28:53.231156+0000", 2024-09-14T06:29:03.835 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.8452000000000001e-05 2024-09-14T06:29:03.835 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:03.836 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:03.836 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-14T06:29:03.836 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:28:53.231927+0000", 2024-09-14T06:29:03.836 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00077038900000000001 2024-09-14T06:29:03.836 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:03.836 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:03.836 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:29:03.836 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:28:53.231969+0000", 2024-09-14T06:29:03.836 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.2466999999999998e-05 2024-09-14T06:29:03.836 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:03.836 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:03.836 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:03.836 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:03.836 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:03.837 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4397.0:9 3.1a e26/23 3:5cea7035:::rbd_info:head v 26'2, mlcod=0'0)", 2024-09-14T06:29:03.837 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:02.579336+0000", 2024-09-14T06:29:03.837 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 1.247309529, 2024-09-14T06:29:03.837 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00085234600000000003, 2024-09-14T06:29:03.837 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:29:03.837 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:29:03.837 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:29:03.837 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:29:03.837 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:03.837 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:29:03.837 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:02.579336+0000", 2024-09-14T06:29:03.837 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:03.837 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:03.838 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:03.838 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:29:03.838 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:02.579336+0000", 2024-09-14T06:29:03.838 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:03.838 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:03.838 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:03.838 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:29:03.838 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:02.579336+0000", 2024-09-14T06:29:03.838 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.8099999999999999e-07 2024-09-14T06:29:03.838 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:03.838 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:03.838 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:29:03.838 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:02.579340+0000", 2024-09-14T06:29:03.838 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.687e-06 2024-09-14T06:29:03.838 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:03.839 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:03.839 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:29:03.839 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:02.579349+0000", 2024-09-14T06:29:03.839 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.3829999999999998e-06 2024-09-14T06:29:03.839 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:03.839 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:03.839 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:29:03.839 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:02.579357+0000", 2024-09-14T06:29:03.839 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.6690000000000006e-06 2024-09-14T06:29:03.839 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:03.839 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:03.839 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:29:03.839 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:02.579409+0000", 2024-09-14T06:29:03.839 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.1564999999999998e-05 2024-09-14T06:29:03.839 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:03.840 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:03.840 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:29:03.840 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:02.579432+0000", 2024-09-14T06:29:03.840 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.3079e-05 2024-09-14T06:29:03.840 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:03.840 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:03.840 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-14T06:29:03.840 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:02.580145+0000", 2024-09-14T06:29:03.840 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00071347000000000001 2024-09-14T06:29:03.840 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:03.840 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:03.840 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:29:03.840 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:02.580188+0000", 2024-09-14T06:29:03.841 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.3312000000000001e-05 2024-09-14T06:29:03.841 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:03.841 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:03.841 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:03.841 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:03.841 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:03.841 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:13 3.4 3:27567f25:::rbd_object_map.112d7adb82ff:head [call rbd.object_map_resize in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-14T06:29:03.841 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:02.589814+0000", 2024-09-14T06:29:03.841 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 1.2368317799999999, 2024-09-14T06:29:03.841 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0014947319999999999, 2024-09-14T06:29:03.841 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:29:03.841 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:29:03.841 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:29:03.841 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:29:03.842 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:29:03.842 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:03.842 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 13 2024-09-14T06:29:03.842 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:03.842 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:29:03.842 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:03.842 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:29:03.842 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:02.589814+0000", 2024-09-14T06:29:03.842 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:03.842 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:03.842 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:03.842 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:29:03.842 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:02.589814+0000", 2024-09-14T06:29:03.842 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:03.842 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:03.843 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:03.843 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:29:03.843 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:02.589815+0000", 2024-09-14T06:29:03.843 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.1599999999999999e-06 2024-09-14T06:29:03.843 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:03.843 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:03.843 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:29:03.843 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:02.589820+0000", 2024-09-14T06:29:03.843 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.8080000000000004e-06 2024-09-14T06:29:03.843 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:03.843 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:03.843 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:29:03.843 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:02.589828+0000", 2024-09-14T06:29:03.843 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.7340000000000008e-06 2024-09-14T06:29:03.844 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:03.844 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:03.844 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:29:03.844 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:02.589843+0000", 2024-09-14T06:29:03.844 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.5500000000000001e-05 2024-09-14T06:29:03.844 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:03.844 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:03.844 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:29:03.844 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:02.589889+0000", 2024-09-14T06:29:03.844 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.5831999999999998e-05 2024-09-14T06:29:03.844 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:03.844 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:03.844 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:29:03.845 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:02.589972+0000", 2024-09-14T06:29:03.845 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.3158999999999994e-05 2024-09-14T06:29:03.845 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:03.845 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:03.845 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "waiting for subops from 2", 2024-09-14T06:29:03.845 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:02.590091+0000", 2024-09-14T06:29:03.845 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000118935 2024-09-14T06:29:03.845 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:03.845 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:03.845 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "op_commit", 2024-09-14T06:29:03.845 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:02.590722+0000", 2024-09-14T06:29:03.845 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00063134300000000003 2024-09-14T06:29:03.845 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:03.845 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:03.846 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:29:03.846 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:02.591241+0000", 2024-09-14T06:29:03.846 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00051846900000000003 2024-09-14T06:29:03.846 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:03.846 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:03.846 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-14T06:29:03.846 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:02.591272+0000", 2024-09-14T06:29:03.846 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.1346000000000001e-05 2024-09-14T06:29:03.846 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:03.846 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:03.846 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:29:03.846 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:02.591309+0000", 2024-09-14T06:29:03.846 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.6446000000000003e-05 2024-09-14T06:29:03.846 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:03.847 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:03.847 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:03.847 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:03.847 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:03.847 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop_reply(client.4397.0:13 3.4 e25/23 ondisk, result = 0)", 2024-09-14T06:29:03.847 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:02.591171+0000", 2024-09-14T06:29:03.847 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 1.235474816, 2024-09-14T06:29:03.847 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000143535, 2024-09-14T06:29:03.847 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:29:03.847 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:29:03.847 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:29:03.847 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:29:03.847 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:03.847 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:29:03.847 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:02.591171+0000", 2024-09-14T06:29:03.847 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:03.848 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:03.848 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:03.848 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:29:03.848 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:02.591171+0000", 2024-09-14T06:29:03.848 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:03.848 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:03.848 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:03.848 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:29:03.848 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:02.591171+0000", 2024-09-14T06:29:03.848 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.4700000000000001e-07 2024-09-14T06:29:03.848 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:03.848 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:03.848 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:29:03.848 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:02.591175+0000", 2024-09-14T06:29:03.848 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.3119999999999998e-06 2024-09-14T06:29:03.849 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:03.849 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:03.849 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:29:03.849 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:02.591184+0000", 2024-09-14T06:29:03.849 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.8799999999999997e-06 2024-09-14T06:29:03.849 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:03.849 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:03.849 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:29:03.849 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:02.591189+0000", 2024-09-14T06:29:03.849 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.6949999999999997e-06 2024-09-14T06:29:03.849 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:03.849 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:03.849 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:29:03.849 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:02.591233+0000", 2024-09-14T06:29:03.850 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.3630000000000001e-05 2024-09-14T06:29:03.850 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:03.850 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:03.850 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:29:03.850 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:02.591238+0000", 2024-09-14T06:29:03.850 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.7489999999999998e-06 2024-09-14T06:29:03.850 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:03.850 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:03.850 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:29:03.850 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:02.591314+0000", 2024-09-14T06:29:03.850 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.6621999999999999e-05 2024-09-14T06:29:03.850 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:03.850 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:03.850 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:03.850 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:03.851 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:03.851 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-14T06:29:03.852 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-14T06:29:03.917 INFO:tasks.ceph.osd.5.smithi138.stderr:2024-09-14T06:29:03.915+0000 7fe13746f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33618) UID: 0 2024-09-14T06:29:03.997 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-14T06:29:03.997 INFO:teuthology.orchestra.run.smithi094.stdout: "ops": [], 2024-09-14T06:29:03.997 INFO:teuthology.orchestra.run.smithi094.stdout: "num_ops": 0 2024-09-14T06:29:03.997 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-14T06:29:04.006 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-14T06:29:04.018 INFO:tasks.ceph.osd.7.smithi138.stderr:2024-09-14T06:29:04.016+0000 7f3c6a21d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33623) UID: 0 2024-09-14T06:29:04.110 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-14T06:29:04.110 INFO:teuthology.orchestra.run.smithi094.stdout: "ops": [], 2024-09-14T06:29:04.110 INFO:teuthology.orchestra.run.smithi094.stdout: "complaint_time": 30, 2024-09-14T06:29:04.110 INFO:teuthology.orchestra.run.smithi094.stdout: "num_blocked_ops": 0 2024-09-14T06:29:04.111 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-14T06:29:04.119 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-14T06:29:04.120 INFO:tasks.ceph.osd.1.smithi094.stderr:2024-09-14T06:29:04.118+0000 7f328e47a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33641) UID: 0 2024-09-14T06:29:04.221 INFO:tasks.ceph.osd.7.smithi138.stderr:2024-09-14T06:29:04.219+0000 7f3c6a21d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33623) UID: 0 2024-09-14T06:29:04.269 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-14T06:29:04.269 INFO:teuthology.orchestra.run.smithi094.stdout: "size": 20, 2024-09-14T06:29:04.269 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 600, 2024-09-14T06:29:04.269 INFO:teuthology.orchestra.run.smithi094.stdout: "ops": [ 2024-09-14T06:29:04.269 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:04.269 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4299.0:1 2.3 e20/19 2:c4c92e5a:::rbd_trash:head v 20'1, mlcod=0'0)", 2024-09-14T06:29:04.269 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:28:51.223664+0000", 2024-09-14T06:29:04.270 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 13.044893884, 2024-09-14T06:29:04.270 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.001165588, 2024-09-14T06:29:04.270 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:29:04.270 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:29:04.270 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:29:04.270 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:29:04.270 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:04.270 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:29:04.270 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:28:51.223664+0000", 2024-09-14T06:29:04.270 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:04.270 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:04.270 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:04.271 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:29:04.271 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:28:51.223664+0000", 2024-09-14T06:29:04.271 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:04.271 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:04.271 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:04.271 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:29:04.271 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:28:51.223665+0000", 2024-09-14T06:29:04.271 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.3600000000000003e-07 2024-09-14T06:29:04.271 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:04.271 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:04.271 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:29:04.271 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:28:51.223672+0000", 2024-09-14T06:29:04.271 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.4740000000000002e-06 2024-09-14T06:29:04.271 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:04.272 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:04.272 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:29:04.272 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:28:51.223695+0000", 2024-09-14T06:29:04.272 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.3859e-05 2024-09-14T06:29:04.272 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:04.272 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:04.272 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:29:04.272 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:28:51.223715+0000", 2024-09-14T06:29:04.272 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.9406e-05 2024-09-14T06:29:04.272 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:04.272 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:04.272 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:29:04.272 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:28:51.223780+0000", 2024-09-14T06:29:04.272 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.4777999999999995e-05 2024-09-14T06:29:04.273 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:04.273 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:04.273 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:29:04.273 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:28:51.223817+0000", 2024-09-14T06:29:04.273 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.7660999999999997e-05 2024-09-14T06:29:04.273 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:04.274 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:04.274 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-14T06:29:04.274 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:28:51.224781+0000", 2024-09-14T06:29:04.274 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00096367800000000004 2024-09-14T06:29:04.274 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:04.274 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:04.274 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:29:04.274 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:28:51.224830+0000", 2024-09-14T06:29:04.274 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.9095999999999999e-05 2024-09-14T06:29:04.274 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:04.274 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:04.274 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:04.274 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:04.275 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:04.275 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4397.0:13 3.4 e26/23 3:27567f25:::rbd_object_map.112d7adb82ff:head v 26'1, mlcod=0'0)", 2024-09-14T06:29:04.275 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:02.590205+0000", 2024-09-14T06:29:04.275 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 1.678353129, 2024-09-14T06:29:04.275 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00092769300000000005, 2024-09-14T06:29:04.275 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:29:04.275 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:29:04.275 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:29:04.275 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:29:04.275 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:04.275 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:29:04.275 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:02.590205+0000", 2024-09-14T06:29:04.275 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:04.275 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:04.276 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:04.276 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:29:04.276 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:02.590205+0000", 2024-09-14T06:29:04.276 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:04.276 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:04.276 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:04.276 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:29:04.276 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:02.590205+0000", 2024-09-14T06:29:04.276 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.5699999999999999e-07 2024-09-14T06:29:04.276 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:04.276 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:04.276 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:29:04.276 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:02.590211+0000", 2024-09-14T06:29:04.276 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.2209999999999996e-06 2024-09-14T06:29:04.277 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:04.277 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:04.277 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:29:04.277 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:02.590220+0000", 2024-09-14T06:29:04.277 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.5359999999999998e-06 2024-09-14T06:29:04.277 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:04.277 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:04.277 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:29:04.277 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:02.590227+0000", 2024-09-14T06:29:04.277 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.4790000000000001e-06 2024-09-14T06:29:04.277 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:04.277 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:04.277 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:29:04.278 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:02.590286+0000", 2024-09-14T06:29:04.278 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.9948999999999998e-05 2024-09-14T06:29:04.278 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:04.278 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:04.278 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:29:04.278 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:02.590308+0000", 2024-09-14T06:29:04.278 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.1580999999999999e-05 2024-09-14T06:29:04.278 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:04.278 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:04.278 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-14T06:29:04.278 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:02.591104+0000", 2024-09-14T06:29:04.278 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00079564299999999996 2024-09-14T06:29:04.278 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:04.278 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:04.278 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:29:04.279 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:02.591133+0000", 2024-09-14T06:29:04.279 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.9127000000000001e-05 2024-09-14T06:29:04.279 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:04.279 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:04.279 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:04.279 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:04.279 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:04.279 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-14T06:29:04.279 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-14T06:29:04.281 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-14T06:29:04.289 INFO:teuthology.orchestra.run.smithi094.stdout:skipping zero size read 2024-09-14T06:29:04.302 INFO:teuthology.orchestra.run.smithi094.stdout:2 write 0x67131a1 thru 0x6719a1d (0x687d bytes) 2024-09-14T06:29:04.306 INFO:teuthology.orchestra.run.smithi094.stdout:5 read 0x188a8a3 thru 0x18998b7 (0xf015 bytes) 2024-09-14T06:29:04.306 INFO:teuthology.orchestra.run.smithi094.stdout:truncating to largest ever: 0xfd92a1 2024-09-14T06:29:04.313 INFO:teuthology.orchestra.run.smithi094.stdout:6 trunc from 0x6719a1e to 0xfd92a1 2024-09-14T06:29:04.321 INFO:tasks.ceph.osd.2.smithi094.stderr:2024-09-14T06:29:04.320+0000 7faf3269b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33649) UID: 0 2024-09-14T06:29:04.324 INFO:teuthology.orchestra.run.smithi094.stdout:7 punch from 0xaddebd to 0xaeb2dc, (0xd41f bytes) 2024-09-14T06:29:04.344 INFO:teuthology.orchestra.run.smithi094.stdout:8 write 0xbf3e3d2 thru 0xbf46f17 (0x8b46 bytes) 2024-09-14T06:29:04.347 INFO:teuthology.orchestra.run.smithi094.stdout:truncating to largest ever: 0x95584e9 2024-09-14T06:29:04.352 INFO:teuthology.orchestra.run.smithi094.stdout:9 trunc from 0xbf46f18 to 0x95584e9 2024-09-14T06:29:04.361 INFO:teuthology.orchestra.run.smithi094.stdout:11 punch from 0x568f4fb to 0x56982b0, (0x8db5 bytes) 2024-09-14T06:29:04.371 INFO:teuthology.orchestra.run.smithi094.stdout:12 write 0xd2e6961 thru 0xd2f068b (0x9d2b bytes) 2024-09-14T06:29:04.375 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-14T06:29:04.375 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-14T06:29:04.383 INFO:teuthology.orchestra.run.smithi094.stdout:20 trunc from 0xd2f068c to 0x6c037e9 2024-09-14T06:29:04.394 INFO:teuthology.orchestra.run.smithi094.stdout:21 writesame 0x18dc93a thru 0x18e9587 data_size 0x6(0xcc4e bytes) 2024-09-14T06:29:04.411 INFO:teuthology.orchestra.run.smithi094.stdout:22 write 0x7d8da02 thru 0x7d97b35 (0xa134 bytes) 2024-09-14T06:29:04.411 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-14T06:29:04.411 INFO:teuthology.orchestra.run.smithi094.stdout: "ops": [], 2024-09-14T06:29:04.411 INFO:teuthology.orchestra.run.smithi094.stdout: "num_ops": 0 2024-09-14T06:29:04.411 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-14T06:29:04.414 INFO:teuthology.orchestra.run.smithi094.stdout:23 punch from 0x6d243ca to 0x6d270c4, (0x2cfa bytes) 2024-09-14T06:29:04.414 INFO:teuthology.orchestra.run.smithi094.stdout:25 punch from 0x1a97287 to 0x1aa051c, (0x9295 bytes) 2024-09-14T06:29:04.415 INFO:teuthology.orchestra.run.smithi094.stdout:26 read 0x458732e thru 0x45961e7 (0xeeba bytes) 2024-09-14T06:29:04.422 INFO:tasks.ceph.osd.4.smithi138.stderr:2024-09-14T06:29:04.420+0000 7f6c653c8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33615) UID: 0 2024-09-14T06:29:04.423 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-14T06:29:04.423 INFO:teuthology.orchestra.run.smithi094.stdout:28 write 0x9e9fb02 thru 0x9ea7412 (0x7911 bytes) 2024-09-14T06:29:04.426 INFO:teuthology.orchestra.run.smithi094.stdout:29 writesame 0x7f48c5b thru 0x7f4f3e5 data_size 0x1(0x678b bytes) 2024-09-14T06:29:04.427 INFO:teuthology.orchestra.run.smithi094.stdout:osd.0: {} 2024-09-14T06:29:04.427 INFO:teuthology.orchestra.run.smithi094.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-14T06:29:04.427 INFO:teuthology.orchestra.run.smithi094.stdout:30 punch from 0x44160de to 0x44206c1, (0xa5e3 bytes) 2024-09-14T06:29:04.433 INFO:teuthology.orchestra.run.smithi094.stdout:31 write 0xa3c5818 thru 0xa3d1e21 (0xc60a bytes) 2024-09-14T06:29:04.437 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-14T06:29:04.437 INFO:teuthology.orchestra.run.smithi094.stdout:truncating to largest ever: 0x96972d0 2024-09-14T06:29:04.438 INFO:teuthology.orchestra.run.smithi094.stdout:34 trunc from 0xa3d1e22 to 0x96972d0 2024-09-14T06:29:04.448 INFO:teuthology.orchestra.run.smithi094.stdout:osd.1: {} 2024-09-14T06:29:04.448 INFO:teuthology.orchestra.run.smithi094.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-14T06:29:04.449 INFO:teuthology.orchestra.run.smithi094.stdout:truncating to largest ever: 0xae7316b 2024-09-14T06:29:04.451 INFO:teuthology.orchestra.run.smithi094.stdout:35 trunc from 0x96972d0 to 0xae7316b 2024-09-14T06:29:04.457 INFO:teuthology.orchestra.run.smithi094.stdout:36 punch from 0x12bc4d2 to 0x12bfb09, (0x3637 bytes) 2024-09-14T06:29:04.457 INFO:teuthology.orchestra.run.smithi094.stdout:40 read 0x89ebdeb thru 0x89ee9be (0x2bd4 bytes) 2024-09-14T06:29:04.457 INFO:teuthology.orchestra.run.smithi094.stdout:43 write 0x340e338 thru 0x3414aa7 (0x6770 bytes) 2024-09-14T06:29:04.460 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-14T06:29:04.460 INFO:teuthology.orchestra.run.smithi094.stdout:45 read 0x3d6b77d thru 0x3d79297 (0xdb1b bytes) 2024-09-14T06:29:04.461 INFO:teuthology.orchestra.run.smithi094.stdout:47 write 0x1d191f thru 0x1dd4cb (0xbbad bytes) 2024-09-14T06:29:04.464 INFO:teuthology.orchestra.run.smithi094.stdout:48 read 0x417bf0e thru 0x41809ee (0x4ae1 bytes) 2024-09-14T06:29:04.464 INFO:teuthology.orchestra.run.smithi094.stdout:51 write 0xa48271c thru 0xa485358 (0x2c3d bytes) 2024-09-14T06:29:04.467 INFO:teuthology.orchestra.run.smithi094.stdout:52 write 0x87ba3af thru 0x87c6ba4 (0xc7f6 bytes) 2024-09-14T06:29:04.468 INFO:teuthology.orchestra.run.smithi094.stdout:osd.2: {} 2024-09-14T06:29:04.469 INFO:teuthology.orchestra.run.smithi094.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-14T06:29:04.479 INFO:teuthology.orchestra.run.smithi094.stdout:56 writesame 0xcdf7d7d thru 0xcdff6ea data_size 0x63(0x796e bytes) 2024-09-14T06:29:04.479 INFO:teuthology.orchestra.run.smithi094.stdout:57 clone 1 order 19 su 65536 sc 4 2024-09-14T06:29:04.485 INFO:teuthology.orchestra.run.smithi094.stdout:osd.3: {} 2024-09-14T06:29:04.485 INFO:teuthology.orchestra.run.smithi094.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-14T06:29:04.502 INFO:teuthology.orchestra.run.smithi094.stdout:osd.4: {} 2024-09-14T06:29:04.502 INFO:teuthology.orchestra.run.smithi094.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-14T06:29:04.519 INFO:teuthology.orchestra.run.smithi094.stdout:osd.5: {} 2024-09-14T06:29:04.519 INFO:teuthology.orchestra.run.smithi094.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-14T06:29:04.522 INFO:tasks.ceph.osd.3.smithi094.stderr:2024-09-14T06:29:04.521+0000 7feeeafe8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33655) UID: 0 2024-09-14T06:29:04.535 INFO:teuthology.orchestra.run.smithi094.stdout:ERROR: (22) Invalid argument 2024-09-14T06:29:04.535 INFO:teuthology.orchestra.run.smithi094.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:29:04.537 INFO:teuthology.orchestra.run.smithi094.stdout:osd.6: {} 2024-09-14T06:29:04.537 INFO:teuthology.orchestra.run.smithi094.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-14T06:29:04.544 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-14T06:29:04.554 INFO:teuthology.orchestra.run.smithi094.stdout:osd.7: {} 2024-09-14T06:29:04.554 INFO:teuthology.orchestra.run.smithi094.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-14T06:29:04.624 INFO:tasks.ceph.osd.4.smithi138.stderr:2024-09-14T06:29:04.622+0000 7f6c653c8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33615) UID: 0 2024-09-14T06:29:04.661 INFO:teuthology.orchestra.run.smithi094.stdout:ERROR: (22) Invalid argument 2024-09-14T06:29:04.661 INFO:teuthology.orchestra.run.smithi094.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:29:04.670 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-14T06:29:04.724 INFO:tasks.ceph.osd.1.smithi094.stderr:2024-09-14T06:29:04.724+0000 7f328e47a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33641) UID: 0 2024-09-14T06:29:04.774 INFO:teuthology.orchestra.run.smithi138.stdout:ERROR: (22) Invalid argument 2024-09-14T06:29:04.774 INFO:teuthology.orchestra.run.smithi138.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:29:04.783 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-14T06:29:04.825 INFO:tasks.ceph.osd.2.smithi094.stderr:2024-09-14T06:29:04.825+0000 7faf3269b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33649) UID: 0 2024-09-14T06:29:04.886 INFO:teuthology.orchestra.run.smithi138.stdout:ERROR: (22) Invalid argument 2024-09-14T06:29:04.886 INFO:teuthology.orchestra.run.smithi138.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:29:04.894 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-14T06:29:04.927 INFO:tasks.ceph.osd.1.smithi094.stderr:2024-09-14T06:29:04.926+0000 7f328e47a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33641) UID: 0 2024-09-14T06:29:04.997 INFO:teuthology.orchestra.run.smithi138.stdout:ERROR: (22) Invalid argument 2024-09-14T06:29:04.998 INFO:teuthology.orchestra.run.smithi138.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:29:05.006 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-14T06:29:05.027 INFO:tasks.ceph.osd.0.smithi094.stderr:2024-09-14T06:29:05.027+0000 7f07c9333640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33636) UID: 0 2024-09-14T06:29:05.109 INFO:teuthology.orchestra.run.smithi138.stdout:ERROR: (22) Invalid argument 2024-09-14T06:29:05.110 INFO:teuthology.orchestra.run.smithi138.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:29:05.118 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-14T06:29:05.162 INFO:tasks.ceph.osd.4.smithi138.stderr:2024-09-14T06:29:05.161+0000 7f6c653c8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33615) UID: 0 2024-09-14T06:29:05.222 INFO:teuthology.orchestra.run.smithi138.stdout:ERROR: (22) Invalid argument 2024-09-14T06:29:05.222 INFO:teuthology.orchestra.run.smithi138.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:29:05.230 INFO:tasks.ceph.osd.3.smithi094.stderr:2024-09-14T06:29:05.228+0000 7feeeafe8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33655) UID: 0 2024-09-14T06:29:05.231 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-14T06:29:05.331 INFO:tasks.ceph.osd.1.smithi094.stderr:2024-09-14T06:29:05.330+0000 7f328e47a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33641) UID: 0 2024-09-14T06:29:05.359 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-14T06:29:05.359 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-14T06:29:05.359 INFO:tasks.thrashosds.thrasher:inject_pause on osd.0 2024-09-14T06:29:05.359 INFO:tasks.thrashosds.thrasher:Testing filestore_inject_stall pause injection for duration 3 2024-09-14T06:29:05.359 INFO:tasks.thrashosds.thrasher:Checking after 0, should_be_down=False 2024-09-14T06:29:05.360 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok config set filestore_inject_stall 3 2024-09-14T06:29:05.373 INFO:teuthology.orchestra.run.smithi138.stdout:ERROR: (22) Invalid argument 2024-09-14T06:29:05.373 INFO:teuthology.orchestra.run.smithi138.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:29:05.381 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-14T06:29:05.432 INFO:tasks.ceph.osd.3.smithi094.stderr:2024-09-14T06:29:05.431+0000 7feeeafe8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33655) UID: 0 2024-09-14T06:29:05.465 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-14T06:29:05.466 INFO:teuthology.orchestra.run.smithi094.stdout: "success": "filestore_inject_stall = '' (not observed, change may require restart) " 2024-09-14T06:29:05.466 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-14T06:29:05.484 INFO:teuthology.orchestra.run.smithi138.stdout:ERROR: (22) Invalid argument 2024-09-14T06:29:05.485 INFO:teuthology.orchestra.run.smithi138.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:29:05.493 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-14T06:29:05.537 INFO:tasks.ceph.osd.4.smithi138.stderr:2024-09-14T06:29:05.536+0000 7f6c653c8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33615) UID: 0 2024-09-14T06:29:05.598 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-14T06:29:05.598 INFO:teuthology.orchestra.run.smithi138.stdout:ERROR: (22) Invalid argument 2024-09-14T06:29:05.598 INFO:teuthology.orchestra.run.smithi138.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:29:05.608 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-14T06:29:05.650 INFO:tasks.ceph.osd.5.smithi138.stderr:2024-09-14T06:29:05.649+0000 7fe13746f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33618) UID: 0 2024-09-14T06:29:05.735 INFO:tasks.ceph.osd.0.smithi094.stderr:2024-09-14T06:29:05.734+0000 7f07c9333640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33636) UID: 0 2024-09-14T06:29:05.752 INFO:teuthology.orchestra.run.smithi138.stdout:ERROR: (22) Invalid argument 2024-09-14T06:29:05.752 INFO:teuthology.orchestra.run.smithi138.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:29:05.760 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-14T06:29:05.836 INFO:tasks.ceph.osd.5.smithi138.stderr:2024-09-14T06:29:05.835+0000 7fe13746f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33618) UID: 0 2024-09-14T06:29:05.864 INFO:teuthology.orchestra.run.smithi138.stdout:ERROR: (22) Invalid argument 2024-09-14T06:29:05.864 INFO:teuthology.orchestra.run.smithi138.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:29:05.872 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-14T06:29:05.937 INFO:tasks.ceph.osd.2.smithi094.stderr:2024-09-14T06:29:05.936+0000 7faf3269b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33649) UID: 0 2024-09-14T06:29:06.016 INFO:teuthology.orchestra.run.smithi138.stdout:ERROR: (22) Invalid argument 2024-09-14T06:29:06.016 INFO:teuthology.orchestra.run.smithi138.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:29:06.024 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-14T06:29:06.038 INFO:tasks.ceph.osd.0.smithi094.stderr:2024-09-14T06:29:06.037+0000 7f07c9333640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33636) UID: 0 2024-09-14T06:29:06.126 INFO:teuthology.orchestra.run.smithi138.stdout:ERROR: (22) Invalid argument 2024-09-14T06:29:06.126 INFO:teuthology.orchestra.run.smithi138.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:29:06.135 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-14T06:29:06.138 INFO:tasks.ceph.osd.7.smithi138.stderr:2024-09-14T06:29:06.137+0000 7f3c6a21d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33623) UID: 0 2024-09-14T06:29:06.239 INFO:teuthology.orchestra.run.smithi094.stdout:ERROR: (22) Invalid argument 2024-09-14T06:29:06.239 INFO:teuthology.orchestra.run.smithi094.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:29:06.239 INFO:tasks.ceph.osd.3.smithi094.stderr:2024-09-14T06:29:06.238+0000 7feeeafe8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33655) UID: 0 2024-09-14T06:29:06.248 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-14T06:29:06.340 INFO:tasks.ceph.osd.0.smithi094.stderr:2024-09-14T06:29:06.339+0000 7f07c9333640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33636) UID: 0 2024-09-14T06:29:06.391 INFO:teuthology.orchestra.run.smithi094.stdout:ERROR: (22) Invalid argument 2024-09-14T06:29:06.391 INFO:teuthology.orchestra.run.smithi094.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:29:06.399 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-14T06:29:06.441 INFO:tasks.ceph.osd.1.smithi094.stderr:2024-09-14T06:29:06.440+0000 7f328e47a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33641) UID: 0 2024-09-14T06:29:06.542 INFO:tasks.ceph.osd.4.smithi138.stderr:2024-09-14T06:29:06.541+0000 7f6c653c8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33615) UID: 0 2024-09-14T06:29:06.543 INFO:teuthology.orchestra.run.smithi094.stdout:ERROR: (22) Invalid argument 2024-09-14T06:29:06.543 INFO:teuthology.orchestra.run.smithi094.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:29:06.552 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-14T06:29:06.565 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-14T06:29:06.590 INFO:teuthology.orchestra.run.smithi094.stderr:noscrub is unset 2024-09-14T06:29:06.643 INFO:tasks.ceph.osd.7.smithi138.stderr:2024-09-14T06:29:06.642+0000 7f3c6a21d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33623) UID: 0 2024-09-14T06:29:06.670 INFO:teuthology.orchestra.run.smithi094.stdout:ERROR: (22) Invalid argument 2024-09-14T06:29:06.670 INFO:teuthology.orchestra.run.smithi094.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:29:06.680 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-14T06:29:06.737 INFO:teuthology.orchestra.run.smithi094.stdout:osd.0: {} 2024-09-14T06:29:06.738 INFO:teuthology.orchestra.run.smithi094.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-14T06:29:06.744 INFO:tasks.ceph.osd.3.smithi094.stderr:2024-09-14T06:29:06.743+0000 7feeeafe8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33655) UID: 0 2024-09-14T06:29:06.754 INFO:teuthology.orchestra.run.smithi094.stdout:59 writesame 0xd064efc thru 0xd068bc2 data_size 0x1(0x3cc7 bytes) 2024-09-14T06:29:06.754 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-14T06:29:06.754 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-14T06:29:06.754 INFO:teuthology.orchestra.run.smithi094.stdout:truncating to largest ever: 0xd3124be 2024-09-14T06:29:06.754 INFO:teuthology.orchestra.run.smithi094.stdout:62 trunc from 0xd068bc3 to 0xd3124be 2024-09-14T06:29:06.757 INFO:teuthology.orchestra.run.smithi094.stdout:osd.1: {} 2024-09-14T06:29:06.757 INFO:teuthology.orchestra.run.smithi094.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-14T06:29:06.766 INFO:teuthology.orchestra.run.smithi094.stdout:63 write 0xd956e89 thru 0xd9658ff (0xea77 bytes) 2024-09-14T06:29:06.771 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-14T06:29:06.771 INFO:teuthology.orchestra.run.smithi094.stdout:66 read 0xfdaa thru 0x1cc09 (0xce60 bytes) 2024-09-14T06:29:06.772 INFO:teuthology.orchestra.run.smithi094.stdout:68 write 0x5b93c53 thru 0x5b9f03c (0xb3ea bytes) 2024-09-14T06:29:06.775 INFO:teuthology.orchestra.run.smithi094.stdout:69 write 0x79344e5 thru 0x7939b72 (0x568e bytes) 2024-09-14T06:29:06.776 INFO:teuthology.orchestra.run.smithi094.stdout:osd.2: {} 2024-09-14T06:29:06.777 INFO:teuthology.orchestra.run.smithi094.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-14T06:29:06.778 INFO:teuthology.orchestra.run.smithi094.stdout:70 read 0xc1c091f thru 0xc1c1bf6 (0x12d8 bytes) 2024-09-14T06:29:06.778 INFO:teuthology.orchestra.run.smithi094.stdout:71 writesame 0x3f21a2e thru 0x3f3116d data_size 0x5c(0xf740 bytes) 2024-09-14T06:29:06.782 INFO:teuthology.orchestra.run.smithi094.stdout:73 read 0xb7651d3 thru 0xb773a95 (0xe8c3 bytes) 2024-09-14T06:29:06.782 INFO:teuthology.orchestra.run.smithi094.stdout:74 write 0xda0d75 thru 0xdaeb58 (0xdde4 bytes) 2024-09-14T06:29:06.785 INFO:teuthology.orchestra.run.smithi094.stdout:75 read 0x230c32e thru 0x231a928 (0xe5fb bytes) 2024-09-14T06:29:06.786 INFO:teuthology.orchestra.run.smithi094.stdout:76 punch from 0xa6b19f5 to 0xa6b4105, (0x2710 bytes) 2024-09-14T06:29:06.786 INFO:teuthology.orchestra.run.smithi094.stdout:77 punch from 0x8e495f3 to 0x8e4ddff, (0x480c bytes) 2024-09-14T06:29:06.792 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-14T06:29:06.792 INFO:teuthology.orchestra.run.smithi094.stdout: "ops": [], 2024-09-14T06:29:06.792 INFO:teuthology.orchestra.run.smithi094.stdout: "complaint_time": 30, 2024-09-14T06:29:06.792 INFO:teuthology.orchestra.run.smithi094.stdout: "num_blocked_ops": 0 2024-09-14T06:29:06.792 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-14T06:29:06.793 INFO:teuthology.orchestra.run.smithi094.stdout:79 trunc from 0xd965900 to 0x74800dc 2024-09-14T06:29:06.794 INFO:teuthology.orchestra.run.smithi094.stdout:osd.3: {} 2024-09-14T06:29:06.794 INFO:teuthology.orchestra.run.smithi094.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-14T06:29:06.803 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-14T06:29:06.811 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-14T06:29:06.811 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-14T06:29:06.811 INFO:teuthology.orchestra.run.smithi094.stdout:84 read 0x230ad30 thru 0x230fdd9 (0x50aa bytes) 2024-09-14T06:29:06.811 INFO:teuthology.orchestra.run.smithi094.stdout:85 read 0x4dabbe1 thru 0x4db8047 (0xc467 bytes) 2024-09-14T06:29:06.811 INFO:teuthology.orchestra.run.smithi094.stdout:86 write 0x5e3088 thru 0x5f123d (0xe1b6 bytes) 2024-09-14T06:29:06.814 INFO:teuthology.orchestra.run.smithi094.stdout:osd.4: {} 2024-09-14T06:29:06.814 INFO:teuthology.orchestra.run.smithi094.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-14T06:29:06.816 INFO:teuthology.orchestra.run.smithi094.stdout:88 punch from 0x713ea9f to 0x714aac5, (0xc026 bytes) 2024-09-14T06:29:06.816 INFO:teuthology.orchestra.run.smithi094.stdout:89 punch from 0x382c6a8 to 0x383c514, (0xfe6c bytes) 2024-09-14T06:29:06.816 INFO:teuthology.orchestra.run.smithi094.stdout:90 punch from 0x3bdec to 0x4afab, (0xf1bf bytes) 2024-09-14T06:29:06.817 INFO:teuthology.orchestra.run.smithi094.stdout:truncating to largest ever: 0xe361d80 2024-09-14T06:29:06.824 INFO:teuthology.orchestra.run.smithi094.stdout:91 trunc from 0x74800dc to 0xe361d80 2024-09-14T06:29:06.829 INFO:teuthology.orchestra.run.smithi094.stdout:92 writesame 0x13f941f thru 0x13f9735 data_size 0x1(0x317 bytes) 2024-09-14T06:29:06.832 INFO:teuthology.orchestra.run.smithi094.stdout:osd.5: {} 2024-09-14T06:29:06.832 INFO:teuthology.orchestra.run.smithi094.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-14T06:29:06.840 INFO:teuthology.orchestra.run.smithi094.stdout:93 trunc from 0xe361d80 to 0x87046a7 2024-09-14T06:29:06.845 INFO:tasks.ceph.osd.4.smithi138.stderr:2024-09-14T06:29:06.844+0000 7f6c653c8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33615) UID: 0 2024-09-14T06:29:06.846 INFO:teuthology.orchestra.run.smithi094.stdout:94 punch from 0x8119a62 to 0x81251b8, (0xb756 bytes) 2024-09-14T06:29:06.846 INFO:teuthology.orchestra.run.smithi094.stdout:97 read 0x2655434 thru 0x2663a51 (0xe61e bytes) 2024-09-14T06:29:06.852 INFO:teuthology.orchestra.run.smithi094.stdout:osd.6: {} 2024-09-14T06:29:06.853 INFO:teuthology.orchestra.run.smithi094.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-14T06:29:06.853 INFO:teuthology.orchestra.run.smithi094.stdout:98 trunc from 0x87046a7 to 0xd6be1bf 2024-09-14T06:29:06.871 INFO:teuthology.orchestra.run.smithi094.stdout:osd.7: {} 2024-09-14T06:29:06.871 INFO:teuthology.orchestra.run.smithi094.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-14T06:29:06.872 INFO:teuthology.orchestra.run.smithi094.stdout:99 trunc from 0xd6be1bf to 0x2f25cae 2024-09-14T06:29:06.881 INFO:teuthology.orchestra.run.smithi094.stdout:100 read 0x1543d8b thru 0x1549d5e (0x5fd4 bytes) 2024-09-14T06:29:06.886 INFO:teuthology.orchestra.run.smithi094.stdout:101 writesame 0x344c928 thru 0x344d918 data_size 0xb(0xff1 bytes) 2024-09-14T06:29:06.886 INFO:teuthology.orchestra.run.smithi094.stdout:103 read 0x1112357 thru 0x1118527 (0x61d1 bytes) 2024-09-14T06:29:06.887 INFO:teuthology.orchestra.run.smithi094.stdout:104 read 0x2b24122 thru 0x2b2a400 (0x62df bytes) 2024-09-14T06:29:06.894 INFO:teuthology.orchestra.run.smithi094.stdout:105 write 0x555a6f1 thru 0x55649de (0xa2ee bytes) 2024-09-14T06:29:06.897 INFO:teuthology.orchestra.run.smithi094.stdout:106 read 0xf36336 thru 0xf390f0 (0x2dbb bytes) 2024-09-14T06:29:06.901 INFO:teuthology.orchestra.run.smithi094.stdout:107 trunc from 0x55649df to 0x276169a 2024-09-14T06:29:06.911 INFO:teuthology.orchestra.run.smithi094.stdout:108 read 0x11e8a2d thru 0x11f4ff1 (0xc5c5 bytes) 2024-09-14T06:29:06.912 INFO:teuthology.orchestra.run.smithi094.stdout:109 punch from 0xa07b7b to 0xa106d4, (0x8b59 bytes) 2024-09-14T06:29:06.912 INFO:teuthology.orchestra.run.smithi094.stdout:110 read 0x263ae09 thru 0x263e9fe (0x3bf6 bytes) 2024-09-14T06:29:06.924 INFO:teuthology.orchestra.run.smithi094.stdout:111 trunc from 0x276169a to 0xce01aa9 2024-09-14T06:29:06.934 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-14T06:29:06.934 INFO:teuthology.orchestra.run.smithi094.stdout: "size": 20, 2024-09-14T06:29:06.934 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 600, 2024-09-14T06:29:06.934 INFO:teuthology.orchestra.run.smithi094.stdout: "ops": [ 2024-09-14T06:29:06.935 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:06.935 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:13 3.4 3:27567f25:::rbd_object_map.112d7adb82ff:head [call rbd.object_map_resize in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-14T06:29:06.935 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:02.589814+0000", 2024-09-14T06:29:06.935 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 4.3422357759999999, 2024-09-14T06:29:06.935 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0014947319999999999, 2024-09-14T06:29:06.935 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:29:06.935 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:29:06.935 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:29:06.935 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:29:06.935 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:29:06.935 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:06.935 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 13 2024-09-14T06:29:06.935 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:06.935 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:29:06.936 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:06.936 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:29:06.936 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:02.589814+0000", 2024-09-14T06:29:06.936 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:06.936 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:06.936 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:06.936 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:29:06.936 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:02.589814+0000", 2024-09-14T06:29:06.936 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:06.936 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:06.936 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:06.936 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:29:06.936 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:02.589815+0000", 2024-09-14T06:29:06.937 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.1599999999999999e-06 2024-09-14T06:29:06.937 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:06.937 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:06.937 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:29:06.937 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:02.589820+0000", 2024-09-14T06:29:06.937 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.8080000000000004e-06 2024-09-14T06:29:06.937 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:06.937 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:06.937 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:29:06.937 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:02.589828+0000", 2024-09-14T06:29:06.937 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.7340000000000008e-06 2024-09-14T06:29:06.937 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:06.938 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:06.938 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:29:06.938 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:02.589843+0000", 2024-09-14T06:29:06.938 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.5500000000000001e-05 2024-09-14T06:29:06.938 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:06.938 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:06.938 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:29:06.938 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:02.589889+0000", 2024-09-14T06:29:06.938 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.5831999999999998e-05 2024-09-14T06:29:06.938 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:06.938 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:06.938 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:29:06.938 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:02.589972+0000", 2024-09-14T06:29:06.938 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.3158999999999994e-05 2024-09-14T06:29:06.939 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:06.939 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:06.939 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "waiting for subops from 2", 2024-09-14T06:29:06.939 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:02.590091+0000", 2024-09-14T06:29:06.939 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000118935 2024-09-14T06:29:06.939 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:06.939 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:06.939 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "op_commit", 2024-09-14T06:29:06.939 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:02.590722+0000", 2024-09-14T06:29:06.939 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00063134300000000003 2024-09-14T06:29:06.939 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:06.939 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:06.939 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:29:06.940 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:02.591241+0000", 2024-09-14T06:29:06.940 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00051846900000000003 2024-09-14T06:29:06.940 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:06.940 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:06.940 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-14T06:29:06.940 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:02.591272+0000", 2024-09-14T06:29:06.940 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.1346000000000001e-05 2024-09-14T06:29:06.940 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:06.940 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:06.940 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:29:06.940 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:02.591309+0000", 2024-09-14T06:29:06.940 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.6446000000000003e-05 2024-09-14T06:29:06.940 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:06.940 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:06.941 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:06.941 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:06.941 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:06.941 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:28 3.4 3:27567f25:::rbd_object_map.112d7adb82ff:head [call lock.lock in=8b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-14T06:29:06.941 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:04.271783+0000", 2024-09-14T06:29:06.941 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 2.6602670220000002, 2024-09-14T06:29:06.941 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0029333810000000001, 2024-09-14T06:29:06.941 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:29:06.941 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:29:06.941 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:29:06.941 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:29:06.941 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:29:06.942 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:06.942 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 28 2024-09-14T06:29:06.942 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:06.942 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:29:06.942 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:06.942 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:29:06.942 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:04.271783+0000", 2024-09-14T06:29:06.942 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:06.942 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:06.942 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:06.942 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:29:06.942 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:04.271783+0000", 2024-09-14T06:29:06.942 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:06.942 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:06.943 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:06.943 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:29:06.943 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:04.271784+0000", 2024-09-14T06:29:06.943 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.635e-06 2024-09-14T06:29:06.943 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:06.943 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:06.943 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:29:06.943 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:04.271794+0000", 2024-09-14T06:29:06.943 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.2650000000000002e-06 2024-09-14T06:29:06.943 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:06.943 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:06.943 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:29:06.943 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:04.271817+0000", 2024-09-14T06:29:06.943 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.3385000000000001e-05 2024-09-14T06:29:06.944 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:06.944 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:06.944 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:29:06.944 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:04.271824+0000", 2024-09-14T06:29:06.944 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.0500000000000003e-06 2024-09-14T06:29:06.944 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:06.944 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:06.944 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:29:06.944 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:04.271889+0000", 2024-09-14T06:29:06.944 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.4659999999999994e-05 2024-09-14T06:29:06.944 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:06.944 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:06.944 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "op must wait for map", 2024-09-14T06:29:06.945 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:04.271896+0000", 2024-09-14T06:29:06.945 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.52e-06 2024-09-14T06:29:06.945 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:06.945 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:06.945 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:29:06.945 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:04.273576+0000", 2024-09-14T06:29:06.945 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.001680225 2024-09-14T06:29:06.945 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:06.945 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:06.945 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:29:06.945 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:04.273635+0000", 2024-09-14T06:29:06.945 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.8832000000000002e-05 2024-09-14T06:29:06.945 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:06.945 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:06.946 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "waiting for subops from 2", 2024-09-14T06:29:06.946 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:04.273765+0000", 2024-09-14T06:29:06.946 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000129713 2024-09-14T06:29:06.946 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:06.946 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:06.946 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "op_commit", 2024-09-14T06:29:06.946 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:04.274315+0000", 2024-09-14T06:29:06.946 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00054979099999999999 2024-09-14T06:29:06.946 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:06.946 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:06.946 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:29:06.946 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:04.274605+0000", 2024-09-14T06:29:06.946 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00028987899999999999 2024-09-14T06:29:06.947 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:06.947 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:06.947 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-14T06:29:06.947 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:04.274666+0000", 2024-09-14T06:29:06.947 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.1710000000000004e-05 2024-09-14T06:29:06.947 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:06.947 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:06.947 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:29:06.947 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:04.274716+0000", 2024-09-14T06:29:06.947 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.9716000000000003e-05 2024-09-14T06:29:06.947 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:06.947 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:06.947 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:06.947 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:06.948 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:06.948 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:57 3.4 3:27567f25:::rbd_object_map.112d7adb82ff:head [call lock.assert_locked in=17b,call rbd.object_map_resize in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-14T06:29:06.948 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:04.296659+0000", 2024-09-14T06:29:06.948 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 2.6353904200000002, 2024-09-14T06:29:06.948 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.001536857, 2024-09-14T06:29:06.948 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:29:06.948 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:29:06.948 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:29:06.948 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:29:06.948 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:29:06.948 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:06.948 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 57 2024-09-14T06:29:06.948 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:06.948 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:29:06.949 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:06.949 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:29:06.949 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:04.296659+0000", 2024-09-14T06:29:06.949 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:06.949 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:06.949 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:06.949 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:29:06.949 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:04.296659+0000", 2024-09-14T06:29:06.949 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:06.949 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:06.949 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:06.949 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:29:06.949 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:04.296661+0000", 2024-09-14T06:29:06.950 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.2010000000000002e-06 2024-09-14T06:29:06.950 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:06.950 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:06.950 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:29:06.950 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:04.296675+0000", 2024-09-14T06:29:06.950 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.364e-05 2024-09-14T06:29:06.950 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:06.950 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:06.950 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:29:06.950 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:04.296696+0000", 2024-09-14T06:29:06.950 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.0741e-05 2024-09-14T06:29:06.950 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:06.950 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:06.951 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:29:06.951 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:04.296706+0000", 2024-09-14T06:29:06.951 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.0458999999999999e-05 2024-09-14T06:29:06.951 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:06.951 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:06.951 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:29:06.951 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:04.296766+0000", 2024-09-14T06:29:06.951 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.0024e-05 2024-09-14T06:29:06.951 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:06.951 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:06.951 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:29:06.951 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:04.296858+0000", 2024-09-14T06:29:06.951 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.1298999999999994e-05 2024-09-14T06:29:06.951 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:06.952 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:06.952 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "waiting for subops from 2", 2024-09-14T06:29:06.952 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:04.297036+0000", 2024-09-14T06:29:06.952 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00017805800000000001 2024-09-14T06:29:06.952 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:06.952 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:06.952 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "op_commit", 2024-09-14T06:29:06.952 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:04.297837+0000", 2024-09-14T06:29:06.952 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00080139999999999996 2024-09-14T06:29:06.952 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:06.952 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:06.952 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:29:06.952 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:04.298117+0000", 2024-09-14T06:29:06.952 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00028043099999999999 2024-09-14T06:29:06.953 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:06.953 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:06.953 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-14T06:29:06.953 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:04.298149+0000", 2024-09-14T06:29:06.953 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.1794000000000001e-05 2024-09-14T06:29:06.953 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:06.953 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:06.953 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:29:06.953 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:04.298196+0000", 2024-09-14T06:29:06.953 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.6810000000000001e-05 2024-09-14T06:29:06.953 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:06.953 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:06.953 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:06.953 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:06.954 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:06.954 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:66 3.4 3:27567f25:::rbd_object_map.112d7adb82ff:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-14T06:29:06.954 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:04.302439+0000", 2024-09-14T06:29:06.954 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 2.6296105289999998, 2024-09-14T06:29:06.954 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0016486840000000001, 2024-09-14T06:29:06.954 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:29:06.954 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:29:06.954 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:29:06.954 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:29:06.954 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:29:06.954 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:06.954 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 66 2024-09-14T06:29:06.954 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:06.954 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:29:06.955 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:06.955 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:29:06.955 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:04.302439+0000", 2024-09-14T06:29:06.955 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:06.955 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:06.955 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:06.955 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:29:06.955 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:04.302439+0000", 2024-09-14T06:29:06.955 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:06.955 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:06.955 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:06.955 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:29:06.955 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:04.302441+0000", 2024-09-14T06:29:06.955 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.429e-06 2024-09-14T06:29:06.956 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:06.956 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:06.956 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:29:06.956 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:04.302448+0000", 2024-09-14T06:29:06.956 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.2030000000000003e-06 2024-09-14T06:29:06.956 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:06.956 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:06.956 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:29:06.956 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:04.302461+0000", 2024-09-14T06:29:06.956 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.3078000000000001e-05 2024-09-14T06:29:06.956 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:06.956 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:06.956 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:29:06.956 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:04.302470+0000", 2024-09-14T06:29:06.957 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.0920000000000004e-06 2024-09-14T06:29:06.957 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:06.957 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:06.957 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:29:06.957 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:04.302537+0000", 2024-09-14T06:29:06.957 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.6565999999999999e-05 2024-09-14T06:29:06.957 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:06.957 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:06.957 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:29:06.957 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:04.302643+0000", 2024-09-14T06:29:06.957 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000106938 2024-09-14T06:29:06.957 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:06.957 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:06.958 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "waiting for subops from 2", 2024-09-14T06:29:06.958 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:04.302996+0000", 2024-09-14T06:29:06.958 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000352727 2024-09-14T06:29:06.958 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:06.958 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:06.958 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "op_commit", 2024-09-14T06:29:06.958 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:04.303734+0000", 2024-09-14T06:29:06.958 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00073748900000000003 2024-09-14T06:29:06.958 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:06.958 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:06.958 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:29:06.958 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:04.304014+0000", 2024-09-14T06:29:06.958 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00028057100000000002 2024-09-14T06:29:06.958 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:06.958 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:06.959 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-14T06:29:06.959 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:04.304045+0000", 2024-09-14T06:29:06.959 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.1270999999999999e-05 2024-09-14T06:29:06.959 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:06.959 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:06.959 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:29:06.959 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:04.304088+0000", 2024-09-14T06:29:06.959 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.2320000000000001e-05 2024-09-14T06:29:06.959 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:06.959 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:06.959 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:06.959 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:06.959 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:06.960 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:68 3.4 3:27567f25:::rbd_object_map.112d7adb82ff:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-14T06:29:06.960 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:04.313380+0000", 2024-09-14T06:29:06.960 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 2.6186693619999999, 2024-09-14T06:29:06.960 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.001792709, 2024-09-14T06:29:06.960 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:29:06.960 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:29:06.960 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:29:06.960 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:29:06.960 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:29:06.960 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:06.961 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 68 2024-09-14T06:29:06.961 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:06.961 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:29:06.961 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:06.961 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:29:06.961 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:04.313380+0000", 2024-09-14T06:29:06.961 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:06.961 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:06.961 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:06.961 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:29:06.961 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:04.313380+0000", 2024-09-14T06:29:06.961 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:06.961 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:06.961 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:06.962 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:29:06.962 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:04.313383+0000", 2024-09-14T06:29:06.962 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.5890000000000001e-06 2024-09-14T06:29:06.962 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:06.962 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:06.962 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:29:06.962 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:04.313396+0000", 2024-09-14T06:29:06.962 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.326e-05 2024-09-14T06:29:06.962 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:06.962 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:06.962 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:29:06.962 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:04.313417+0000", 2024-09-14T06:29:06.962 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.0936000000000001e-05 2024-09-14T06:29:06.962 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:06.963 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:06.963 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:29:06.963 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:04.313437+0000", 2024-09-14T06:29:06.963 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.0163999999999999e-05 2024-09-14T06:29:06.963 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:06.963 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:06.963 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:29:06.963 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:04.313514+0000", 2024-09-14T06:29:06.963 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.6457000000000003e-05 2024-09-14T06:29:06.963 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:06.963 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:06.963 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:29:06.963 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:04.313630+0000", 2024-09-14T06:29:06.964 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000116347 2024-09-14T06:29:06.964 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:06.964 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:06.964 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "waiting for subops from 2", 2024-09-14T06:29:06.964 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:04.314035+0000", 2024-09-14T06:29:06.964 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00040450700000000002 2024-09-14T06:29:06.964 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:06.964 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:06.964 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "op_commit", 2024-09-14T06:29:06.964 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:04.314915+0000", 2024-09-14T06:29:06.965 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00088063699999999998 2024-09-14T06:29:06.965 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:06.965 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:06.965 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:29:06.965 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:04.315098+0000", 2024-09-14T06:29:06.965 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000182747 2024-09-14T06:29:06.965 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:06.965 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:06.965 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-14T06:29:06.965 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:04.315132+0000", 2024-09-14T06:29:06.965 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.4514999999999997e-05 2024-09-14T06:29:06.965 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:06.965 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:06.966 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:29:06.966 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:04.315173+0000", 2024-09-14T06:29:06.966 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.0550000000000003e-05 2024-09-14T06:29:06.966 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:06.966 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:06.966 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:06.966 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:06.966 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:06.966 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:70 3.4 3:27567f25:::rbd_object_map.112d7adb82ff:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-14T06:29:06.966 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:04.316905+0000", 2024-09-14T06:29:06.966 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 2.6151450380000001, 2024-09-14T06:29:06.966 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00141282, 2024-09-14T06:29:06.966 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:29:06.967 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:29:06.967 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:29:06.967 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:29:06.967 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:29:06.967 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:06.967 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 70 2024-09-14T06:29:06.967 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:06.967 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:29:06.967 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:06.967 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:29:06.967 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:04.316905+0000", 2024-09-14T06:29:06.967 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:06.967 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:06.967 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:06.968 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:29:06.968 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:04.316905+0000", 2024-09-14T06:29:06.968 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:06.968 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:06.968 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:06.968 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:29:06.968 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:04.316906+0000", 2024-09-14T06:29:06.968 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.78e-07 2024-09-14T06:29:06.968 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:06.968 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:06.968 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:29:06.968 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:04.316911+0000", 2024-09-14T06:29:06.968 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.169e-06 2024-09-14T06:29:06.969 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:06.969 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:06.969 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:29:06.969 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:04.316921+0000", 2024-09-14T06:29:06.969 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.0807e-05 2024-09-14T06:29:06.969 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:06.969 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:06.969 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:29:06.969 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:04.316927+0000", 2024-09-14T06:29:06.969 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.2490000000000001e-06 2024-09-14T06:29:06.969 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:06.969 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:06.969 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:29:06.970 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:04.316978+0000", 2024-09-14T06:29:06.970 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.0844999999999999e-05 2024-09-14T06:29:06.970 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:06.970 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:06.970 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:29:06.970 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:04.317062+0000", 2024-09-14T06:29:06.970 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.4438999999999996e-05 2024-09-14T06:29:06.970 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:06.970 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:06.970 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "waiting for subops from 2", 2024-09-14T06:29:06.970 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:04.317424+0000", 2024-09-14T06:29:06.970 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00036247599999999998 2024-09-14T06:29:06.970 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:06.970 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:06.971 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "op_commit", 2024-09-14T06:29:06.971 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:04.318205+0000", 2024-09-14T06:29:06.971 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000780338 2024-09-14T06:29:06.971 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:06.971 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:06.971 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:29:06.971 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:04.318254+0000", 2024-09-14T06:29:06.971 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.9119999999999997e-05 2024-09-14T06:29:06.971 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:06.971 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:06.971 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-14T06:29:06.971 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:04.318285+0000", 2024-09-14T06:29:06.971 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.0601000000000001e-05 2024-09-14T06:29:06.971 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:06.972 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:06.972 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:29:06.972 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:04.318317+0000", 2024-09-14T06:29:06.972 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.2898000000000002e-05 2024-09-14T06:29:06.972 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:06.972 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:06.972 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:06.972 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:06.972 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:06.972 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:81 3.4 3:27567f25:::rbd_object_map.112d7adb82ff:head [call lock.assert_locked in=17b,call rbd.object_map_resize in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-14T06:29:06.972 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:04.337520+0000", 2024-09-14T06:29:06.972 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 2.594529493, 2024-09-14T06:29:06.972 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.001379063, 2024-09-14T06:29:06.972 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:29:06.973 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:29:06.973 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:29:06.973 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:29:06.973 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:29:06.973 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:06.973 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 81 2024-09-14T06:29:06.973 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:06.973 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:29:06.973 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:06.973 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:29:06.973 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:04.337520+0000", 2024-09-14T06:29:06.974 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:06.974 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:06.974 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:06.974 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:29:06.974 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:04.337520+0000", 2024-09-14T06:29:06.974 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:06.974 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:06.974 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:06.974 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:29:06.974 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:04.337523+0000", 2024-09-14T06:29:06.974 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.1149999999999998e-06 2024-09-14T06:29:06.974 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:06.974 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:06.974 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:29:06.974 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:04.337536+0000", 2024-09-14T06:29:06.975 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.2999999999999999e-05 2024-09-14T06:29:06.975 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:06.975 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:06.975 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:29:06.975 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:04.337563+0000", 2024-09-14T06:29:06.975 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.6831000000000001e-05 2024-09-14T06:29:06.975 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:06.975 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:06.975 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:29:06.975 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:04.337591+0000", 2024-09-14T06:29:06.975 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.7747999999999999e-05 2024-09-14T06:29:06.975 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:06.975 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:06.975 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:29:06.976 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:04.337663+0000", 2024-09-14T06:29:06.976 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.2266000000000002e-05 2024-09-14T06:29:06.976 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:06.976 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:06.976 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:29:06.976 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:04.337740+0000", 2024-09-14T06:29:06.976 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.7212000000000005e-05 2024-09-14T06:29:06.976 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:06.976 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:06.976 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "waiting for subops from 2", 2024-09-14T06:29:06.976 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:04.337936+0000", 2024-09-14T06:29:06.976 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000196088 2024-09-14T06:29:06.976 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:06.976 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:06.977 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "op_commit", 2024-09-14T06:29:06.977 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:04.338669+0000", 2024-09-14T06:29:06.977 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00073243600000000005 2024-09-14T06:29:06.977 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:06.977 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:06.977 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:29:06.977 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:04.338833+0000", 2024-09-14T06:29:06.977 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00016430700000000001 2024-09-14T06:29:06.977 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:06.977 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:06.977 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-14T06:29:06.977 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:04.338865+0000", 2024-09-14T06:29:06.977 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.1327e-05 2024-09-14T06:29:06.978 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:06.978 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:06.978 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:29:06.978 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:04.338899+0000", 2024-09-14T06:29:06.978 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.4733000000000001e-05 2024-09-14T06:29:06.978 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:06.978 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:06.978 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:06.978 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:06.978 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:06.978 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:93 3.4 3:27567f25:::rbd_object_map.112d7adb82ff:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-14T06:29:06.978 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:04.350466+0000", 2024-09-14T06:29:06.978 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 2.5815841150000001, 2024-09-14T06:29:06.978 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0014315549999999999, 2024-09-14T06:29:06.979 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:29:06.979 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:29:06.979 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:29:06.979 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:29:06.979 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:29:06.979 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:06.979 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 93 2024-09-14T06:29:06.979 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:06.979 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:29:06.979 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:06.979 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:29:06.979 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:04.350466+0000", 2024-09-14T06:29:06.979 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:06.979 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:06.979 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:06.980 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:29:06.980 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:04.350466+0000", 2024-09-14T06:29:06.980 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:06.980 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:06.980 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:06.980 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:29:06.980 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:04.350468+0000", 2024-09-14T06:29:06.980 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.7530000000000002e-06 2024-09-14T06:29:06.980 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:06.980 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:06.980 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:29:06.980 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:04.350484+0000", 2024-09-14T06:29:06.980 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.6064000000000001e-05 2024-09-14T06:29:06.981 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:06.981 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:06.981 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:29:06.981 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:04.350509+0000", 2024-09-14T06:29:06.981 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.457e-05 2024-09-14T06:29:06.981 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:06.981 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:06.981 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:29:06.981 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:04.350522+0000", 2024-09-14T06:29:06.981 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.2938999999999999e-05 2024-09-14T06:29:06.981 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:06.981 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:06.981 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:29:06.982 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:04.350593+0000", 2024-09-14T06:29:06.982 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.1585e-05 2024-09-14T06:29:06.982 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:06.982 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:06.982 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:29:06.982 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:04.350662+0000", 2024-09-14T06:29:06.982 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.8221999999999998e-05 2024-09-14T06:29:06.982 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:06.982 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:06.982 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "waiting for subops from 2", 2024-09-14T06:29:06.982 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:04.350918+0000", 2024-09-14T06:29:06.982 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00025614099999999999 2024-09-14T06:29:06.982 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:06.983 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:06.983 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "op_commit", 2024-09-14T06:29:06.983 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:04.351629+0000", 2024-09-14T06:29:06.983 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00071124000000000003 2024-09-14T06:29:06.983 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:06.983 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:06.983 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:29:06.983 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:04.351839+0000", 2024-09-14T06:29:06.983 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000209533 2024-09-14T06:29:06.983 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:06.983 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:06.983 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-14T06:29:06.983 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:04.351868+0000", 2024-09-14T06:29:06.983 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.9708000000000001e-05 2024-09-14T06:29:06.984 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:06.984 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:06.984 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:29:06.984 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:04.351897+0000", 2024-09-14T06:29:06.984 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.8799999999999999e-05 2024-09-14T06:29:06.984 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:06.984 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:06.984 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:06.984 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:06.984 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:06.984 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:95 3.4 3:27567f25:::rbd_object_map.112d7adb82ff:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-14T06:29:06.984 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:04.353289+0000", 2024-09-14T06:29:06.984 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 2.5787606570000001, 2024-09-14T06:29:06.985 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0013676109999999999, 2024-09-14T06:29:06.985 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:29:06.985 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:29:06.985 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:29:06.985 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:29:06.985 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:29:06.985 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:06.985 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 95 2024-09-14T06:29:06.985 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:06.985 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:29:06.985 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:06.985 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:29:06.985 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:04.353289+0000", 2024-09-14T06:29:06.985 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:06.986 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:06.986 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:06.986 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:29:06.986 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:04.353289+0000", 2024-09-14T06:29:06.986 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:06.986 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:06.986 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:06.986 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:29:06.986 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:04.353290+0000", 2024-09-14T06:29:06.986 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.062e-06 2024-09-14T06:29:06.986 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:06.986 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:06.986 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:29:06.986 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:04.353295+0000", 2024-09-14T06:29:06.986 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.2249999999999999e-06 2024-09-14T06:29:06.987 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:06.987 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:06.987 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:29:06.987 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:04.353309+0000", 2024-09-14T06:29:06.987 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.399e-05 2024-09-14T06:29:06.987 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:06.987 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:06.987 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:29:06.987 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:04.353317+0000", 2024-09-14T06:29:06.987 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.9950000000000005e-06 2024-09-14T06:29:06.987 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:06.987 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:06.987 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:29:06.987 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:04.353362+0000", 2024-09-14T06:29:06.988 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.5124999999999999e-05 2024-09-14T06:29:06.988 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:06.988 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:06.988 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:29:06.988 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:04.353413+0000", 2024-09-14T06:29:06.988 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.0652999999999998e-05 2024-09-14T06:29:06.988 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:06.988 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:06.988 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "waiting for subops from 2", 2024-09-14T06:29:06.988 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:04.353612+0000", 2024-09-14T06:29:06.988 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00019920100000000001 2024-09-14T06:29:06.988 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:06.988 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:06.988 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "op_commit", 2024-09-14T06:29:06.989 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:04.354224+0000", 2024-09-14T06:29:06.989 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00061148699999999997 2024-09-14T06:29:06.989 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:06.989 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:06.989 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:29:06.989 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:04.354605+0000", 2024-09-14T06:29:06.989 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00038160100000000002 2024-09-14T06:29:06.989 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:06.989 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:06.989 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-14T06:29:06.989 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:04.354633+0000", 2024-09-14T06:29:06.989 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.7710000000000001e-05 2024-09-14T06:29:06.989 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:06.989 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:06.990 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:29:06.990 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:04.354657+0000", 2024-09-14T06:29:06.990 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.3561999999999999e-05 2024-09-14T06:29:06.990 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:06.990 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:06.990 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:06.990 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:06.990 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:06.990 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:105 3.4 3:27567f25:::rbd_object_map.112d7adb82ff:head [call lock.assert_locked in=17b,call rbd.object_map_resize in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-14T06:29:06.990 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:04.365730+0000", 2024-09-14T06:29:06.990 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 2.5663201070000001, 2024-09-14T06:29:06.990 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.001684264, 2024-09-14T06:29:06.990 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:29:06.990 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:29:06.991 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:29:06.991 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:29:06.991 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:29:06.991 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:06.991 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 105 2024-09-14T06:29:06.991 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:06.991 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:29:06.991 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:06.991 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:29:06.991 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:04.365730+0000", 2024-09-14T06:29:06.991 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:06.991 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:06.991 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:06.992 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:29:06.992 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:04.365730+0000", 2024-09-14T06:29:06.992 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:06.992 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:06.992 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:06.992 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:29:06.992 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:04.365732+0000", 2024-09-14T06:29:06.992 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.9069999999999999e-06 2024-09-14T06:29:06.992 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:06.992 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:06.992 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:29:06.992 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:04.365746+0000", 2024-09-14T06:29:06.992 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.3645999999999999e-05 2024-09-14T06:29:06.992 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:06.992 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:06.993 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:29:06.993 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:04.365772+0000", 2024-09-14T06:29:06.993 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.55e-05 2024-09-14T06:29:06.993 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:06.993 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:06.993 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:29:06.993 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:04.365785+0000", 2024-09-14T06:29:06.993 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.3876000000000001e-05 2024-09-14T06:29:06.993 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:06.993 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:06.993 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:29:06.993 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:04.365882+0000", 2024-09-14T06:29:06.993 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.6711000000000003e-05 2024-09-14T06:29:06.993 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:06.993 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:06.994 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:29:06.994 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:04.365987+0000", 2024-09-14T06:29:06.994 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00010477800000000001 2024-09-14T06:29:06.994 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:06.994 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:06.994 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "waiting for subops from 2", 2024-09-14T06:29:06.994 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:04.366285+0000", 2024-09-14T06:29:06.994 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00029762400000000002 2024-09-14T06:29:06.994 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:06.994 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:06.994 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "op_commit", 2024-09-14T06:29:06.994 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:04.367021+0000", 2024-09-14T06:29:06.994 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00073628099999999998 2024-09-14T06:29:06.995 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:06.995 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:06.995 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:29:06.995 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:04.367296+0000", 2024-09-14T06:29:06.995 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00027557100000000001 2024-09-14T06:29:06.995 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:06.995 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:06.995 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-14T06:29:06.995 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:04.367360+0000", 2024-09-14T06:29:06.995 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.3969999999999999e-05 2024-09-14T06:29:06.995 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:06.995 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:06.995 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:29:06.996 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:04.367414+0000", 2024-09-14T06:29:06.996 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.3399999999999997e-05 2024-09-14T06:29:06.996 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:06.996 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:06.996 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:06.996 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:06.996 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:06.996 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:114 3.4 3:27567f25:::rbd_object_map.112d7adb82ff:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-14T06:29:06.996 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:04.371716+0000", 2024-09-14T06:29:06.996 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 2.5603338500000001, 2024-09-14T06:29:06.996 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0018045750000000001, 2024-09-14T06:29:06.996 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:29:06.996 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:29:06.997 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:29:06.997 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:29:06.997 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:29:06.997 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:06.997 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 114 2024-09-14T06:29:06.997 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:06.997 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:29:06.997 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:06.997 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:29:06.997 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:04.371716+0000", 2024-09-14T06:29:06.997 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:06.998 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:06.998 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:06.998 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:29:06.998 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:04.371716+0000", 2024-09-14T06:29:06.998 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:06.998 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:06.998 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:06.998 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:29:06.998 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:04.371719+0000", 2024-09-14T06:29:06.998 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.6860000000000002e-06 2024-09-14T06:29:06.998 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:06.998 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:06.998 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:29:06.999 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:04.371729+0000", 2024-09-14T06:29:06.999 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.0688e-05 2024-09-14T06:29:06.999 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:06.999 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:06.999 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:29:06.999 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:04.371750+0000", 2024-09-14T06:29:06.999 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.0596000000000001e-05 2024-09-14T06:29:06.999 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:06.999 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:06.999 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:29:06.999 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:04.371769+0000", 2024-09-14T06:29:06.999 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.9252000000000001e-05 2024-09-14T06:29:06.999 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:06.999 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:07.000 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:29:07.000 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:04.371844+0000", 2024-09-14T06:29:07.000 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.4467000000000004e-05 2024-09-14T06:29:07.000 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:07.000 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:07.000 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:29:07.000 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:04.371945+0000", 2024-09-14T06:29:07.000 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00010165700000000001 2024-09-14T06:29:07.000 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:07.000 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:07.000 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "waiting for subops from 2", 2024-09-14T06:29:07.000 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:04.372331+0000", 2024-09-14T06:29:07.000 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00038578799999999997 2024-09-14T06:29:07.001 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:07.001 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:07.001 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "op_commit", 2024-09-14T06:29:07.001 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:04.373184+0000", 2024-09-14T06:29:07.001 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00085342700000000003 2024-09-14T06:29:07.001 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:07.001 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:07.001 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:29:07.001 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:04.373422+0000", 2024-09-14T06:29:07.001 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000237858 2024-09-14T06:29:07.001 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:07.001 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:07.001 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-14T06:29:07.001 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:04.373476+0000", 2024-09-14T06:29:07.002 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.3318e-05 2024-09-14T06:29:07.002 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:07.002 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:07.002 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:29:07.002 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:04.373520+0000", 2024-09-14T06:29:07.002 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.4838e-05 2024-09-14T06:29:07.002 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:07.002 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:07.002 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:07.002 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:07.002 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:07.002 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:116 3.4 3:27567f25:::rbd_object_map.112d7adb82ff:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-14T06:29:07.002 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:04.383445+0000", 2024-09-14T06:29:07.002 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 2.5486050049999998, 2024-09-14T06:29:07.003 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0017313129999999999, 2024-09-14T06:29:07.003 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:29:07.003 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:29:07.003 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:29:07.003 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:29:07.003 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:29:07.003 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:07.003 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 116 2024-09-14T06:29:07.003 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:07.003 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:29:07.003 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:07.003 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:29:07.003 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:04.383445+0000", 2024-09-14T06:29:07.003 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:07.003 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:07.004 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:07.004 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:29:07.004 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:04.383445+0000", 2024-09-14T06:29:07.004 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:07.004 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:07.004 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:07.004 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:29:07.004 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:04.383447+0000", 2024-09-14T06:29:07.004 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.058e-06 2024-09-14T06:29:07.004 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:07.004 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:07.004 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:29:07.004 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:04.383460+0000", 2024-09-14T06:29:07.004 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.2974e-05 2024-09-14T06:29:07.005 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:07.005 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:07.005 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:29:07.005 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:04.383480+0000", 2024-09-14T06:29:07.005 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.0689000000000001e-05 2024-09-14T06:29:07.005 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:07.005 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:07.005 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:29:07.005 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:04.383491+0000", 2024-09-14T06:29:07.005 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.0198e-05 2024-09-14T06:29:07.005 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:07.005 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:07.005 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:29:07.005 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:04.383562+0000", 2024-09-14T06:29:07.006 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.1638999999999996e-05 2024-09-14T06:29:07.006 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:07.006 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:07.006 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:29:07.006 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:04.383666+0000", 2024-09-14T06:29:07.006 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000103686 2024-09-14T06:29:07.006 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:07.006 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:07.006 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "waiting for subops from 2", 2024-09-14T06:29:07.006 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:04.384063+0000", 2024-09-14T06:29:07.006 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00039673600000000002 2024-09-14T06:29:07.006 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:07.006 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:07.006 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "op_commit", 2024-09-14T06:29:07.006 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:04.384929+0000", 2024-09-14T06:29:07.007 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00086629600000000001 2024-09-14T06:29:07.007 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:07.007 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:07.007 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:29:07.007 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:04.385104+0000", 2024-09-14T06:29:07.007 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00017465199999999999 2024-09-14T06:29:07.007 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:07.007 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:07.007 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-14T06:29:07.007 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:04.385137+0000", 2024-09-14T06:29:07.007 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.3646000000000003e-05 2024-09-14T06:29:07.007 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:07.007 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:07.007 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:29:07.008 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:04.385176+0000", 2024-09-14T06:29:07.008 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.8739000000000003e-05 2024-09-14T06:29:07.008 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:07.008 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:07.008 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:07.008 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:07.008 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:07.008 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:120 3.4 3:27567f25:::rbd_object_map.112d7adb82ff:head [call lock.assert_locked in=17b,call rbd.object_map_resize in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-14T06:29:07.008 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:04.390166+0000", 2024-09-14T06:29:07.008 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 2.5418835839999998, 2024-09-14T06:29:07.008 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.001374859, 2024-09-14T06:29:07.008 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:29:07.008 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:29:07.008 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:29:07.008 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:29:07.009 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:29:07.009 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:07.009 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 120 2024-09-14T06:29:07.009 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:07.009 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:29:07.009 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:07.009 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:29:07.009 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:04.390166+0000", 2024-09-14T06:29:07.009 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:07.009 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:07.009 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:07.009 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:29:07.009 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:04.390166+0000", 2024-09-14T06:29:07.009 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:07.010 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:07.010 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:07.010 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:29:07.010 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:04.390167+0000", 2024-09-14T06:29:07.010 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.2279999999999999e-06 2024-09-14T06:29:07.010 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:07.010 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:07.010 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:29:07.010 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:04.390172+0000", 2024-09-14T06:29:07.010 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.4390000000000002e-06 2024-09-14T06:29:07.010 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:07.010 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:07.011 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:29:07.011 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:04.390182+0000", 2024-09-14T06:29:07.011 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.0603e-05 2024-09-14T06:29:07.011 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:07.011 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:07.011 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:29:07.011 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:04.390188+0000", 2024-09-14T06:29:07.011 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.0390000000000001e-06 2024-09-14T06:29:07.011 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:07.011 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:07.011 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:29:07.011 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:04.390237+0000", 2024-09-14T06:29:07.012 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.8180999999999998e-05 2024-09-14T06:29:07.012 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:07.012 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:07.012 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:29:07.012 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:04.390323+0000", 2024-09-14T06:29:07.012 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.6372000000000006e-05 2024-09-14T06:29:07.012 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:07.012 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:07.012 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "waiting for subops from 2", 2024-09-14T06:29:07.013 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:04.390571+0000", 2024-09-14T06:29:07.013 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00024761 2024-09-14T06:29:07.013 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:07.013 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:07.013 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "op_commit", 2024-09-14T06:29:07.013 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:04.391199+0000", 2024-09-14T06:29:07.013 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00062887100000000003 2024-09-14T06:29:07.013 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:07.013 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:07.013 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:29:07.013 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:04.391443+0000", 2024-09-14T06:29:07.014 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000243172 2024-09-14T06:29:07.014 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:07.014 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:07.014 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-14T06:29:07.014 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:04.391495+0000", 2024-09-14T06:29:07.014 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.2240999999999997e-05 2024-09-14T06:29:07.014 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:07.014 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:07.014 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:29:07.014 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:04.391541+0000", 2024-09-14T06:29:07.014 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.6103000000000002e-05 2024-09-14T06:29:07.015 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:07.015 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:07.015 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:07.015 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:07.015 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:07.015 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:130 3.4 3:27567f25:::rbd_object_map.112d7adb82ff:head [call lock.assert_locked in=17b,call rbd.object_map_resize in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-14T06:29:07.015 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:04.404337+0000", 2024-09-14T06:29:07.015 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 2.5277124309999999, 2024-09-14T06:29:07.015 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0021083479999999999, 2024-09-14T06:29:07.015 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:29:07.015 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:29:07.016 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:29:07.016 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:29:07.016 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:29:07.016 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:07.016 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 130 2024-09-14T06:29:07.016 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:07.016 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:29:07.016 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:07.016 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:29:07.016 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:04.404337+0000", 2024-09-14T06:29:07.016 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:07.016 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:07.017 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:07.017 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:29:07.017 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:04.404337+0000", 2024-09-14T06:29:07.017 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:07.017 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:07.017 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:07.017 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:29:07.017 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:04.404340+0000", 2024-09-14T06:29:07.017 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.8159999999999998e-06 2024-09-14T06:29:07.017 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:07.017 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:07.017 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:29:07.018 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:04.404356+0000", 2024-09-14T06:29:07.018 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.5554e-05 2024-09-14T06:29:07.018 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:07.018 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:07.018 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:29:07.018 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:04.404380+0000", 2024-09-14T06:29:07.018 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.4068000000000001e-05 2024-09-14T06:29:07.018 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:07.018 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:07.018 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:29:07.018 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:04.404393+0000", 2024-09-14T06:29:07.018 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.3169999999999999e-05 2024-09-14T06:29:07.018 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:07.019 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:07.019 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:29:07.019 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:04.404475+0000", 2024-09-14T06:29:07.019 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.2280999999999994e-05 2024-09-14T06:29:07.019 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:07.019 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:07.019 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:29:07.019 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:04.404580+0000", 2024-09-14T06:29:07.019 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000104391 2024-09-14T06:29:07.019 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:07.019 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:07.019 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "waiting for subops from 2", 2024-09-14T06:29:07.019 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:04.404886+0000", 2024-09-14T06:29:07.020 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00030692000000000002 2024-09-14T06:29:07.020 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:07.020 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:07.020 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:29:07.020 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:04.406046+0000", 2024-09-14T06:29:07.020 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.001159705 2024-09-14T06:29:07.020 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:07.020 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:07.020 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "op_commit", 2024-09-14T06:29:07.020 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:04.406332+0000", 2024-09-14T06:29:07.020 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00028583500000000002 2024-09-14T06:29:07.020 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:07.021 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:07.021 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-14T06:29:07.021 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:04.406392+0000", 2024-09-14T06:29:07.021 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.9988000000000002e-05 2024-09-14T06:29:07.021 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:07.021 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:07.021 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:29:07.021 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:04.406446+0000", 2024-09-14T06:29:07.021 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.3619999999999998e-05 2024-09-14T06:29:07.021 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:07.021 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:07.021 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:07.021 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:07.021 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:07.022 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:141 3.4 3:27567f25:::rbd_object_map.112d7adb82ff:head [call lock.assert_locked in=17b,call rbd.object_map_resize in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-14T06:29:07.022 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:04.417460+0000", 2024-09-14T06:29:07.022 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 2.51458994, 2024-09-14T06:29:07.022 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.001413538, 2024-09-14T06:29:07.022 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:29:07.022 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:29:07.022 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:29:07.022 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:29:07.022 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:29:07.022 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:07.022 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 141 2024-09-14T06:29:07.022 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:07.022 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:29:07.023 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:07.023 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:29:07.023 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:04.417460+0000", 2024-09-14T06:29:07.023 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:07.023 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:07.023 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:07.023 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:29:07.023 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:04.417460+0000", 2024-09-14T06:29:07.023 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:07.023 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:07.023 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:07.023 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:29:07.023 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:04.417462+0000", 2024-09-14T06:29:07.024 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.1689999999999999e-06 2024-09-14T06:29:07.024 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:07.024 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:07.024 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:29:07.024 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:04.417475+0000", 2024-09-14T06:29:07.024 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.2982999999999999e-05 2024-09-14T06:29:07.024 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:07.024 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:07.024 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:29:07.025 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:04.417495+0000", 2024-09-14T06:29:07.025 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.0418e-05 2024-09-14T06:29:07.025 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:07.025 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:07.025 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:29:07.025 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:04.417505+0000", 2024-09-14T06:29:07.025 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.6779999999999995e-06 2024-09-14T06:29:07.025 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:07.025 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:07.025 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:29:07.025 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:04.417574+0000", 2024-09-14T06:29:07.025 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.8670000000000005e-05 2024-09-14T06:29:07.025 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:07.025 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:07.026 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:29:07.026 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:04.417644+0000", 2024-09-14T06:29:07.026 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.0622000000000002e-05 2024-09-14T06:29:07.026 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:07.026 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:07.026 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "waiting for subops from 2", 2024-09-14T06:29:07.026 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:04.417824+0000", 2024-09-14T06:29:07.026 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00017984300000000001 2024-09-14T06:29:07.026 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:07.026 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:07.026 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "op_commit", 2024-09-14T06:29:07.026 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:04.418487+0000", 2024-09-14T06:29:07.026 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00066268000000000002 2024-09-14T06:29:07.026 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:07.027 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:07.027 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:29:07.027 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:04.418809+0000", 2024-09-14T06:29:07.027 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00032176299999999998 2024-09-14T06:29:07.027 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:07.027 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:07.027 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-14T06:29:07.027 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:04.418842+0000", 2024-09-14T06:29:07.027 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.3416e-05 2024-09-14T06:29:07.027 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:07.027 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:07.027 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:29:07.027 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:04.418873+0000", 2024-09-14T06:29:07.027 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.1296e-05 2024-09-14T06:29:07.027 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:07.028 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:07.028 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:07.028 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:07.028 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:07.028 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:150 3.4 3:27567f25:::rbd_object_map.112d7adb82ff:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-14T06:29:07.028 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:04.423378+0000", 2024-09-14T06:29:07.028 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 2.5086717190000001, 2024-09-14T06:29:07.028 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.001404626, 2024-09-14T06:29:07.028 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:29:07.028 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:29:07.028 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:29:07.028 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:29:07.028 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:29:07.028 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:07.029 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 150 2024-09-14T06:29:07.029 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:07.029 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:29:07.029 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:07.029 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:29:07.029 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:04.423378+0000", 2024-09-14T06:29:07.029 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:07.029 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:07.029 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:07.029 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:29:07.029 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:04.423378+0000", 2024-09-14T06:29:07.029 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:07.029 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:07.029 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:07.030 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:29:07.030 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:04.423380+0000", 2024-09-14T06:29:07.030 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.7969999999999999e-06 2024-09-14T06:29:07.030 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:07.030 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:07.030 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:29:07.030 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:04.423387+0000", 2024-09-14T06:29:07.030 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.1910000000000003e-06 2024-09-14T06:29:07.030 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:07.030 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:07.030 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:29:07.030 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:04.423401+0000", 2024-09-14T06:29:07.030 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.4503e-05 2024-09-14T06:29:07.030 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:07.031 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:07.031 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:29:07.031 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:04.423409+0000", 2024-09-14T06:29:07.031 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.5970000000000003e-06 2024-09-14T06:29:07.031 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:07.031 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:07.031 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:29:07.031 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:04.423463+0000", 2024-09-14T06:29:07.031 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.3729e-05 2024-09-14T06:29:07.031 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:07.031 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:07.031 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:29:07.031 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:04.423528+0000", 2024-09-14T06:29:07.031 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.5359999999999998e-05 2024-09-14T06:29:07.032 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:07.032 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:07.032 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "waiting for subops from 2", 2024-09-14T06:29:07.032 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:04.423745+0000", 2024-09-14T06:29:07.032 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00021682099999999999 2024-09-14T06:29:07.032 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:07.032 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:07.032 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "op_commit", 2024-09-14T06:29:07.032 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:04.424398+0000", 2024-09-14T06:29:07.032 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00065297499999999997 2024-09-14T06:29:07.032 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:07.032 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:07.032 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:29:07.033 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:04.424681+0000", 2024-09-14T06:29:07.033 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00028282399999999999 2024-09-14T06:29:07.033 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:07.033 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:07.033 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-14T06:29:07.033 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:04.424736+0000", 2024-09-14T06:29:07.033 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.5359000000000003e-05 2024-09-14T06:29:07.033 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:07.033 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:07.033 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:29:07.033 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:04.424783+0000", 2024-09-14T06:29:07.033 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.6470000000000001e-05 2024-09-14T06:29:07.033 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:07.034 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:07.034 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:07.034 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:07.034 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:07.034 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:153 3.4 3:27567f25:::rbd_object_map.112d7adb82ff:head [call lock.assert_locked in=17b,call rbd.object_map_resize in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-14T06:29:07.034 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:04.428237+0000", 2024-09-14T06:29:07.034 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 2.5038125449999997, 2024-09-14T06:29:07.034 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.001518152, 2024-09-14T06:29:07.034 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:29:07.034 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:29:07.034 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:29:07.034 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:29:07.034 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:29:07.034 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:07.035 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 153 2024-09-14T06:29:07.035 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:07.035 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:29:07.035 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:07.035 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:29:07.035 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:04.428237+0000", 2024-09-14T06:29:07.035 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:07.035 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:07.035 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:07.035 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:29:07.035 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:04.428237+0000", 2024-09-14T06:29:07.035 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:07.035 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:07.035 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:07.036 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:29:07.036 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:04.428238+0000", 2024-09-14T06:29:07.036 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.3260000000000001e-06 2024-09-14T06:29:07.036 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:07.036 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:07.036 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:29:07.036 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:04.428247+0000", 2024-09-14T06:29:07.036 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.2539999999999992e-06 2024-09-14T06:29:07.036 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:07.036 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:07.036 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:29:07.036 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:04.428259+0000", 2024-09-14T06:29:07.036 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.2493e-05 2024-09-14T06:29:07.036 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:07.036 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:07.037 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:29:07.037 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:04.428267+0000", 2024-09-14T06:29:07.037 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.3849999999999996e-06 2024-09-14T06:29:07.037 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:07.037 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:07.037 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:29:07.037 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:04.428332+0000", 2024-09-14T06:29:07.037 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.5401000000000006e-05 2024-09-14T06:29:07.037 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:07.037 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:07.037 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:29:07.037 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:04.428427+0000", 2024-09-14T06:29:07.037 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.4927999999999996e-05 2024-09-14T06:29:07.037 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:07.038 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:07.038 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "waiting for subops from 2", 2024-09-14T06:29:07.038 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:04.428696+0000", 2024-09-14T06:29:07.038 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00026946899999999999 2024-09-14T06:29:07.038 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:07.038 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:07.038 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "op_commit", 2024-09-14T06:29:07.038 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:04.429457+0000", 2024-09-14T06:29:07.038 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00076069500000000003 2024-09-14T06:29:07.038 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:07.038 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:07.038 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:29:07.038 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:04.429654+0000", 2024-09-14T06:29:07.038 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00019732800000000001 2024-09-14T06:29:07.039 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:07.039 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:07.039 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-14T06:29:07.039 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:04.429709+0000", 2024-09-14T06:29:07.039 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.4372000000000001e-05 2024-09-14T06:29:07.039 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:07.039 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:07.039 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:29:07.039 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:04.429755+0000", 2024-09-14T06:29:07.039 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.6501000000000001e-05 2024-09-14T06:29:07.039 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:07.039 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:07.039 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:07.039 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:07.039 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:07.040 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:162 3.4 3:27567f25:::rbd_object_map.112d7adb82ff:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-14T06:29:07.040 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:04.433551+0000", 2024-09-14T06:29:07.040 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 2.4984989670000002, 2024-09-14T06:29:07.040 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0014139510000000001, 2024-09-14T06:29:07.040 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:29:07.040 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:29:07.040 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:29:07.040 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:29:07.040 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:29:07.040 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:07.040 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 162 2024-09-14T06:29:07.040 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:07.040 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:29:07.040 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:07.041 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:29:07.041 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:04.433551+0000", 2024-09-14T06:29:07.041 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:07.041 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:07.041 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:07.041 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:29:07.041 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:04.433551+0000", 2024-09-14T06:29:07.041 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:07.041 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:07.041 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:07.041 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:29:07.041 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:04.433552+0000", 2024-09-14T06:29:07.041 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.6020000000000001e-06 2024-09-14T06:29:07.041 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:07.042 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:07.042 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:29:07.042 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:04.433563+0000", 2024-09-14T06:29:07.042 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.0631e-05 2024-09-14T06:29:07.042 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:07.042 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:07.042 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:29:07.042 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:04.433578+0000", 2024-09-14T06:29:07.042 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.5311999999999998e-05 2024-09-14T06:29:07.042 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:07.042 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:07.042 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:29:07.042 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:04.433586+0000", 2024-09-14T06:29:07.042 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.5959999999999998e-06 2024-09-14T06:29:07.043 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:07.043 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:07.043 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:29:07.043 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:04.433638+0000", 2024-09-14T06:29:07.043 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.24e-05 2024-09-14T06:29:07.043 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:07.043 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:07.043 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:29:07.043 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:04.433709+0000", 2024-09-14T06:29:07.043 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.0407000000000005e-05 2024-09-14T06:29:07.043 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:07.043 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:07.043 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "waiting for subops from 2", 2024-09-14T06:29:07.043 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:04.433953+0000", 2024-09-14T06:29:07.044 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000244047 2024-09-14T06:29:07.044 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:07.044 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:07.044 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "op_commit", 2024-09-14T06:29:07.044 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:04.434744+0000", 2024-09-14T06:29:07.044 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00079113800000000004 2024-09-14T06:29:07.044 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:07.044 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:07.044 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:29:07.044 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:04.434893+0000", 2024-09-14T06:29:07.044 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00014878 2024-09-14T06:29:07.044 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:07.044 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:07.045 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-14T06:29:07.045 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:04.434924+0000", 2024-09-14T06:29:07.045 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.1736000000000002e-05 2024-09-14T06:29:07.045 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:07.045 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:07.045 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:29:07.045 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:04.434965+0000", 2024-09-14T06:29:07.045 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.0302000000000001e-05 2024-09-14T06:29:07.045 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:07.045 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:07.045 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:07.045 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:07.045 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:07.046 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:164 3.4 3:27567f25:::rbd_object_map.112d7adb82ff:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-14T06:29:07.046 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:04.438161+0000", 2024-09-14T06:29:07.046 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 2.493889061, 2024-09-14T06:29:07.046 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0019683679999999999, 2024-09-14T06:29:07.046 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:29:07.046 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:29:07.046 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:29:07.046 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:29:07.046 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:29:07.046 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:07.046 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 164 2024-09-14T06:29:07.046 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:07.046 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:29:07.046 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:07.046 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:29:07.047 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:04.438161+0000", 2024-09-14T06:29:07.047 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:07.047 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:07.047 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:07.047 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:29:07.047 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:04.438161+0000", 2024-09-14T06:29:07.047 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:07.047 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:07.047 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:07.047 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:29:07.047 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:04.438163+0000", 2024-09-14T06:29:07.047 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.34e-06 2024-09-14T06:29:07.047 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:07.047 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:07.048 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:29:07.048 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:04.438175+0000", 2024-09-14T06:29:07.048 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.1585e-05 2024-09-14T06:29:07.048 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:07.048 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:07.048 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:29:07.048 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:04.438195+0000", 2024-09-14T06:29:07.048 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.9981000000000001e-05 2024-09-14T06:29:07.048 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:07.048 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:07.048 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:29:07.048 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:04.438210+0000", 2024-09-14T06:29:07.048 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.5420000000000001e-05 2024-09-14T06:29:07.049 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:07.049 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:07.049 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:29:07.049 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:04.438298+0000", 2024-09-14T06:29:07.049 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.8378e-05 2024-09-14T06:29:07.049 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:07.049 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:07.049 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:29:07.049 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:04.438422+0000", 2024-09-14T06:29:07.049 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00012416099999999999 2024-09-14T06:29:07.049 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:07.049 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:07.049 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "waiting for subops from 2", 2024-09-14T06:29:07.049 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:04.438900+0000", 2024-09-14T06:29:07.050 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00047733500000000001 2024-09-14T06:29:07.050 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:07.050 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:07.050 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "op_commit", 2024-09-14T06:29:07.050 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:04.439772+0000", 2024-09-14T06:29:07.050 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00087208999999999995 2024-09-14T06:29:07.050 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:07.050 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:07.050 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:29:07.050 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:04.440028+0000", 2024-09-14T06:29:07.050 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00025594000000000002 2024-09-14T06:29:07.050 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:07.050 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:07.050 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-14T06:29:07.051 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:04.440088+0000", 2024-09-14T06:29:07.051 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.033e-05 2024-09-14T06:29:07.051 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:07.051 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:07.051 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:29:07.051 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:04.440129+0000", 2024-09-14T06:29:07.051 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.0808e-05 2024-09-14T06:29:07.051 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:07.051 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:07.051 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:07.051 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:07.051 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:07.051 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:167 3.4 3:27567f25:::rbd_object_map.112d7adb82ff:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-14T06:29:07.051 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:04.441627+0000", 2024-09-14T06:29:07.052 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 2.4904229330000001, 2024-09-14T06:29:07.052 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0016181349999999999, 2024-09-14T06:29:07.052 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:29:07.052 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:29:07.052 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:29:07.052 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:29:07.052 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:29:07.052 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:07.052 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 167 2024-09-14T06:29:07.052 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:07.052 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:29:07.052 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:07.052 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:29:07.053 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:04.441627+0000", 2024-09-14T06:29:07.053 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:07.053 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:07.053 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:07.053 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:29:07.053 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:04.441627+0000", 2024-09-14T06:29:07.053 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:07.053 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:07.053 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:07.053 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:29:07.053 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:04.441628+0000", 2024-09-14T06:29:07.053 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.048e-06 2024-09-14T06:29:07.053 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:07.054 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:07.054 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:29:07.054 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:04.441632+0000", 2024-09-14T06:29:07.054 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.3769999999999998e-06 2024-09-14T06:29:07.054 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:07.054 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:07.054 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:29:07.054 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:04.441642+0000", 2024-09-14T06:29:07.054 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.0142e-05 2024-09-14T06:29:07.054 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:07.054 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:07.054 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:29:07.054 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:04.441649+0000", 2024-09-14T06:29:07.054 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.3999999999999997e-06 2024-09-14T06:29:07.055 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:07.055 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:07.055 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:29:07.055 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:04.441704+0000", 2024-09-14T06:29:07.055 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.5538999999999998e-05 2024-09-14T06:29:07.055 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:07.055 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:07.055 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:29:07.055 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:04.441798+0000", 2024-09-14T06:29:07.055 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.3510000000000004e-05 2024-09-14T06:29:07.055 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:07.055 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:07.055 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "waiting for subops from 2", 2024-09-14T06:29:07.055 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:04.442222+0000", 2024-09-14T06:29:07.056 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00042416200000000001 2024-09-14T06:29:07.056 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:07.056 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:07.056 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "op_commit", 2024-09-14T06:29:07.056 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:04.442967+0000", 2024-09-14T06:29:07.056 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000744676 2024-09-14T06:29:07.056 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:07.056 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:07.056 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:29:07.056 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:04.443141+0000", 2024-09-14T06:29:07.056 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000174704 2024-09-14T06:29:07.056 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:07.056 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:07.056 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-14T06:29:07.057 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:04.443202+0000", 2024-09-14T06:29:07.057 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.0634999999999998e-05 2024-09-14T06:29:07.057 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:07.057 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:07.057 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:29:07.057 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:04.443245+0000", 2024-09-14T06:29:07.057 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.2942000000000003e-05 2024-09-14T06:29:07.057 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:07.057 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:07.057 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:07.057 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:07.057 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:07.057 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-14T06:29:07.059 INFO:tasks.ceph.osd.6.smithi138.stderr:2024-09-14T06:29:07.057+0000 7f8cd8849640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33620) UID: 0 2024-09-14T06:29:07.059 INFO:teuthology.orchestra.run.smithi094.stdout:112 trunc from 0xce01aa9 to 0x540a49e 2024-09-14T06:29:07.059 INFO:teuthology.orchestra.run.smithi094.stdout:113 trunc from 0x540a49e to 0x33eecf0 2024-09-14T06:29:07.060 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-14T06:29:07.062 INFO:teuthology.orchestra.run.smithi094.stdout:114 punch from 0x1986edb to 0x198ab4e, (0x3c73 bytes) 2024-09-14T06:29:07.062 INFO:teuthology.orchestra.run.smithi094.stdout:115 read 0x2edbb79 thru 0x2eeacc4 (0xf14c bytes) 2024-09-14T06:29:07.063 INFO:teuthology.orchestra.run.smithi094.stdout:116 writesame 0x9849866 thru 0x984c832 data_size 0x1(0x2fcd bytes) 2024-09-14T06:29:07.063 INFO:teuthology.orchestra.run.smithi094.stdout:118 writesame 0x4300df7 thru 0x43080f0 data_size 0x1(0x72fa bytes) 2024-09-14T06:29:07.063 INFO:teuthology.orchestra.run.smithi094.stdout:119 trunc from 0x984c833 to 0xce7a572 2024-09-14T06:29:07.063 INFO:teuthology.orchestra.run.smithi094.stdout:120 writesame 0xebf3151 thru 0xebf6665 data_size 0x1(0x3515 bytes) 2024-09-14T06:29:07.063 INFO:teuthology.orchestra.run.smithi094.stdout:122 write 0xa69c9e7 thru 0xa6a883e (0xbe58 bytes) 2024-09-14T06:29:07.063 INFO:teuthology.orchestra.run.smithi094.stdout:123 write 0x4cff5aa thru 0x4d0a7d6 (0xb22d bytes) 2024-09-14T06:29:07.063 INFO:teuthology.orchestra.run.smithi094.stdout:124 punch from 0x879110e to 0x879152c, (0x41e bytes) 2024-09-14T06:29:07.063 INFO:teuthology.orchestra.run.smithi094.stdout:125 read 0x76158cb thru 0x761677f (0xeb5 bytes) 2024-09-14T06:29:07.063 INFO:teuthology.orchestra.run.smithi094.stdout:126 read 0xd2d4ba thru 0xd2fecb (0x2a12 bytes) 2024-09-14T06:29:07.063 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-14T06:29:07.063 INFO:teuthology.orchestra.run.smithi094.stdout:129 punch from 0x6a12247 to 0x6a1c446, (0xa1ff bytes) 2024-09-14T06:29:07.063 INFO:teuthology.orchestra.run.smithi094.stdout:130 write 0x60e368d thru 0x60e6e4a (0x37be bytes) 2024-09-14T06:29:07.063 INFO:teuthology.orchestra.run.smithi094.stdout:131 read 0x5638a12 thru 0x5645bc4 (0xd1b3 bytes) 2024-09-14T06:29:07.064 INFO:teuthology.orchestra.run.smithi094.stdout:132 writesame 0x2b4dc46 thru 0x2b58cdb data_size 0x7(0xb096 bytes) 2024-09-14T06:29:07.064 INFO:teuthology.orchestra.run.smithi094.stdout:133 writesame 0x3d39d7b thru 0x3d40528 data_size 0x17(0x67ae bytes) 2024-09-14T06:29:07.064 INFO:teuthology.orchestra.run.smithi094.stdout:135 punch from 0x11eba96 to 0x11f14d5, (0x5a3f bytes) 2024-09-14T06:29:07.064 INFO:teuthology.orchestra.run.smithi094.stdout:136 read 0xd58d050 thru 0xd593e9f (0x6e50 bytes) 2024-09-14T06:29:07.064 INFO:teuthology.orchestra.run.smithi094.stdout:137 read 0xc1405cb thru 0xc14c6dc (0xc112 bytes) 2024-09-14T06:29:07.064 INFO:teuthology.orchestra.run.smithi094.stdout:138 read 0xcfb2245 thru 0xcfbd83b (0xb5f7 bytes) 2024-09-14T06:29:07.064 INFO:teuthology.orchestra.run.smithi094.stdout:139 punch from 0x4e8086f to 0x4e8fd00, (0xf491 bytes) 2024-09-14T06:29:07.064 INFO:teuthology.orchestra.run.smithi094.stdout:140 write 0x37cc4d0 thru 0x37d9bb2 (0xd6e3 bytes) 2024-09-14T06:29:07.064 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-14T06:29:07.064 INFO:teuthology.orchestra.run.smithi094.stdout:145 write 0x630c244 thru 0x63153ad (0x916a bytes) 2024-09-14T06:29:07.064 INFO:teuthology.orchestra.run.smithi094.stdout:146 read 0x79ed356 thru 0x79f470e (0x73b9 bytes) 2024-09-14T06:29:07.064 INFO:teuthology.orchestra.run.smithi094.stdout:147 punch from 0x71a7137 to 0x71b237e, (0xb247 bytes) 2024-09-14T06:29:07.064 INFO:teuthology.orchestra.run.smithi094.stdout:148 write 0x2ebafcf thru 0x2eca098 (0xf0ca bytes) 2024-09-14T06:29:07.064 INFO:teuthology.orchestra.run.smithi094.stdout:149 trunc from 0xebf6666 to 0x8a27deb 2024-09-14T06:29:07.066 INFO:teuthology.orchestra.run.smithi094.stdout:151 writesame 0x8d0ff04 thru 0x8d119f3 data_size 0x8(0x1af0 bytes) 2024-09-14T06:29:07.067 INFO:teuthology.orchestra.run.smithi094.stdout:152 write 0x8981b9b thru 0x898f820 (0xdc86 bytes) 2024-09-14T06:29:07.069 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-14T06:29:07.069 INFO:teuthology.orchestra.run.smithi094.stdout:155 read 0xcd0546 thru 0xcd9bc3 (0x967e bytes) 2024-09-14T06:29:07.070 INFO:teuthology.orchestra.run.smithi094.stdout:157 read 0x23e4c46 thru 0x23e9e91 (0x524c bytes) 2024-09-14T06:29:07.070 INFO:teuthology.orchestra.run.smithi094.stdout:158 read 0x4e1b812 thru 0x4e20875 (0x5064 bytes) 2024-09-14T06:29:07.070 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-14T06:29:07.070 INFO:teuthology.orchestra.run.smithi094.stdout:160 write 0x23e50b thru 0x24cb15 (0xe60b bytes) 2024-09-14T06:29:07.074 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-14T06:29:07.075 INFO:teuthology.orchestra.run.smithi094.stdout:162 punch from 0x7dec4a3 to 0x7df7e41, (0xb99e bytes) 2024-09-14T06:29:07.075 INFO:teuthology.orchestra.run.smithi094.stdout:163 write 0x1006b0b thru 0x1014f33 (0xe429 bytes) 2024-09-14T06:29:07.084 INFO:teuthology.orchestra.run.smithi094.stdout:165 write 0x9b3aa37 thru 0x9b46ca9 (0xc273 bytes) 2024-09-14T06:29:07.091 INFO:teuthology.orchestra.run.smithi094.stdout:166 writesame 0x57c921b thru 0x57d0512 data_size 0x34(0x72f8 bytes) 2024-09-14T06:29:07.097 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-14T06:29:07.097 INFO:teuthology.orchestra.run.smithi094.stdout:168 write 0x20d5461 thru 0x20dce69 (0x7a09 bytes) 2024-09-14T06:29:07.100 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-14T06:29:07.100 INFO:teuthology.orchestra.run.smithi094.stdout:170 punch from 0x993ac67 to 0x9942b79, (0x7f12 bytes) 2024-09-14T06:29:07.100 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-14T06:29:07.100 INFO:teuthology.orchestra.run.smithi094.stdout:172 read 0x3be3885 thru 0x3be5c2c (0x23a8 bytes) 2024-09-14T06:29:07.100 INFO:teuthology.orchestra.run.smithi094.stdout:173 write 0x876fc04 thru 0x877e038 (0xe435 bytes) 2024-09-14T06:29:07.112 INFO:teuthology.orchestra.run.smithi094.stdout:174 trunc from 0x9b46caa to 0x4ae936c 2024-09-14T06:29:07.125 INFO:teuthology.orchestra.run.smithi094.stdout:176 write 0x6af36a thru 0x6bd621 (0xe2b8 bytes) 2024-09-14T06:29:07.129 INFO:teuthology.orchestra.run.smithi094.stdout:179 write 0x483bc5c thru 0x48469a4 (0xad49 bytes) 2024-09-14T06:29:07.132 INFO:teuthology.orchestra.run.smithi094.stdout:180 read 0x4707ded thru 0x470a4c5 (0x26d9 bytes) 2024-09-14T06:29:07.144 INFO:teuthology.orchestra.run.smithi094.stdout:182 write 0xa49913d thru 0xa4a1ee2 (0x8da6 bytes) 2024-09-14T06:29:07.148 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-14T06:29:07.148 INFO:teuthology.orchestra.run.smithi094.stdout:185 read 0x8bd4d05 thru 0x8bdef7e (0xa27a bytes) 2024-09-14T06:29:07.148 INFO:teuthology.orchestra.run.smithi094.stdout:187 write 0x8b12efd thru 0x8b20f6c (0xe070 bytes) 2024-09-14T06:29:07.152 INFO:teuthology.orchestra.run.smithi094.stdout:192 read 0x7220e8 thru 0x72c579 (0xa492 bytes) 2024-09-14T06:29:07.157 INFO:teuthology.orchestra.run.smithi094.stdout:197 trunc from 0xa4a1ee3 to 0xe237427 2024-09-14T06:29:07.159 INFO:tasks.ceph.osd.4.smithi138.stderr:2024-09-14T06:29:07.158+0000 7f6c653c8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33615) UID: 0 2024-09-14T06:29:07.162 INFO:teuthology.orchestra.run.smithi094.stdout:198 read 0x9c2dbc1 thru 0x9c359e3 (0x7e23 bytes) 2024-09-14T06:29:07.162 INFO:teuthology.orchestra.run.smithi094.stdout:201 writesame 0x85793a thru 0x86655c data_size 0x1(0xec23 bytes) 2024-09-14T06:29:07.178 INFO:teuthology.orchestra.run.smithi094.stdout:202 write 0xc7d8b25 thru 0xc7e3e0b (0xb2e7 bytes) 2024-09-14T06:29:07.180 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-14T06:29:07.181 INFO:teuthology.orchestra.run.smithi094.stdout: "ops": [], 2024-09-14T06:29:07.181 INFO:teuthology.orchestra.run.smithi094.stdout: "num_ops": 0 2024-09-14T06:29:07.181 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-14T06:29:07.185 INFO:teuthology.orchestra.run.smithi094.stdout:205 clone 2 order 20 su 4096 sc 4 2024-09-14T06:29:07.190 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-14T06:29:07.260 INFO:tasks.ceph.osd.7.smithi138.stderr:2024-09-14T06:29:07.259+0000 7f3c6a21d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33623) UID: 0 2024-09-14T06:29:07.333 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-14T06:29:07.333 INFO:teuthology.orchestra.run.smithi094.stdout: "ops": [], 2024-09-14T06:29:07.333 INFO:teuthology.orchestra.run.smithi094.stdout: "complaint_time": 30, 2024-09-14T06:29:07.333 INFO:teuthology.orchestra.run.smithi094.stdout: "num_blocked_ops": 0 2024-09-14T06:29:07.333 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-14T06:29:07.341 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-14T06:29:07.361 INFO:tasks.ceph.osd.6.smithi138.stderr:2024-09-14T06:29:07.360+0000 7f8cd8849640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33620) UID: 0 2024-09-14T06:29:07.445 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-14T06:29:07.446 INFO:teuthology.orchestra.run.smithi094.stdout: "size": 20, 2024-09-14T06:29:07.446 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 600, 2024-09-14T06:29:07.446 INFO:teuthology.orchestra.run.smithi094.stdout: "ops": [ 2024-09-14T06:29:07.446 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:07.446 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4299.0:1 2.3 e20/19 2:c4c92e5a:::rbd_trash:head v 20'1, mlcod=0'0)", 2024-09-14T06:29:07.446 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:28:51.223664+0000", 2024-09-14T06:29:07.446 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 16.219647417000001, 2024-09-14T06:29:07.446 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.001165588, 2024-09-14T06:29:07.446 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:29:07.446 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:29:07.446 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:29:07.446 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:29:07.446 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:07.446 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:29:07.447 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:28:51.223664+0000", 2024-09-14T06:29:07.447 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:07.447 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:07.447 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:07.447 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:29:07.447 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:28:51.223664+0000", 2024-09-14T06:29:07.447 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:07.447 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:07.447 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:07.447 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:29:07.447 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:28:51.223665+0000", 2024-09-14T06:29:07.447 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.3600000000000003e-07 2024-09-14T06:29:07.447 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:07.448 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:07.448 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:29:07.448 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:28:51.223672+0000", 2024-09-14T06:29:07.448 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.4740000000000002e-06 2024-09-14T06:29:07.448 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:07.448 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:07.448 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:29:07.448 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:28:51.223695+0000", 2024-09-14T06:29:07.448 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.3859e-05 2024-09-14T06:29:07.448 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:07.448 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:07.448 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:29:07.448 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:28:51.223715+0000", 2024-09-14T06:29:07.449 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.9406e-05 2024-09-14T06:29:07.449 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:07.449 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:07.449 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:29:07.449 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:28:51.223780+0000", 2024-09-14T06:29:07.449 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.4777999999999995e-05 2024-09-14T06:29:07.449 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:07.449 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:07.449 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:29:07.449 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:28:51.223817+0000", 2024-09-14T06:29:07.449 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.7660999999999997e-05 2024-09-14T06:29:07.449 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:07.449 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:07.449 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-14T06:29:07.450 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:28:51.224781+0000", 2024-09-14T06:29:07.450 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00096367800000000004 2024-09-14T06:29:07.450 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:07.450 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:07.450 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:29:07.450 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:28:51.224830+0000", 2024-09-14T06:29:07.450 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.9095999999999999e-05 2024-09-14T06:29:07.450 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:07.450 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:07.450 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:07.450 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:07.450 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:07.450 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4397.0:13 3.4 e26/23 3:27567f25:::rbd_object_map.112d7adb82ff:head v 26'1, mlcod=0'0)", 2024-09-14T06:29:07.451 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:02.590205+0000", 2024-09-14T06:29:07.451 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 4.8531066620000001, 2024-09-14T06:29:07.451 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00092769300000000005, 2024-09-14T06:29:07.451 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:29:07.451 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:29:07.451 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:29:07.451 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:29:07.451 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:07.451 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:29:07.451 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:02.590205+0000", 2024-09-14T06:29:07.451 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:07.451 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:07.451 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:07.451 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:29:07.452 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:02.590205+0000", 2024-09-14T06:29:07.452 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:07.452 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:07.452 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:07.452 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:29:07.452 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:02.590205+0000", 2024-09-14T06:29:07.452 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.5699999999999999e-07 2024-09-14T06:29:07.452 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:07.452 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:07.452 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:29:07.452 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:02.590211+0000", 2024-09-14T06:29:07.452 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.2209999999999996e-06 2024-09-14T06:29:07.452 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:07.452 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:07.453 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:29:07.453 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:02.590220+0000", 2024-09-14T06:29:07.453 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.5359999999999998e-06 2024-09-14T06:29:07.453 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:07.453 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:07.453 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:29:07.453 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:02.590227+0000", 2024-09-14T06:29:07.453 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.4790000000000001e-06 2024-09-14T06:29:07.453 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:07.453 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:07.453 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:29:07.453 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:02.590286+0000", 2024-09-14T06:29:07.453 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.9948999999999998e-05 2024-09-14T06:29:07.454 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:07.454 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:07.454 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:29:07.454 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:02.590308+0000", 2024-09-14T06:29:07.454 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.1580999999999999e-05 2024-09-14T06:29:07.454 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:07.454 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:07.454 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-14T06:29:07.454 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:02.591104+0000", 2024-09-14T06:29:07.454 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00079564299999999996 2024-09-14T06:29:07.454 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:07.454 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:07.454 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:29:07.454 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:02.591133+0000", 2024-09-14T06:29:07.455 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.9127000000000001e-05 2024-09-14T06:29:07.455 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:07.455 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:07.455 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:07.455 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:07.455 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:07.455 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4397.0:57 3.4 e27/23 3:27567f25:::rbd_object_map.112d7adb82ff:head v 27'5, mlcod=27'3)", 2024-09-14T06:29:07.455 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:04.297136+0000", 2024-09-14T06:29:07.455 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 3.1461758569999998, 2024-09-14T06:29:07.455 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00086588099999999999, 2024-09-14T06:29:07.455 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:29:07.455 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:29:07.455 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:29:07.455 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:29:07.456 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:07.456 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:29:07.456 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:04.297136+0000", 2024-09-14T06:29:07.456 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:07.456 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:07.456 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:07.456 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:29:07.456 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:04.297136+0000", 2024-09-14T06:29:07.456 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:07.456 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:07.456 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:07.456 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:29:07.456 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:04.297137+0000", 2024-09-14T06:29:07.456 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.404e-06 2024-09-14T06:29:07.457 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:07.457 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:07.457 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:29:07.457 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:04.297159+0000", 2024-09-14T06:29:07.457 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.2005999999999999e-05 2024-09-14T06:29:07.457 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:07.457 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:07.457 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:29:07.457 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:04.297179+0000", 2024-09-14T06:29:07.457 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.963e-05 2024-09-14T06:29:07.457 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:07.457 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:07.457 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:29:07.458 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:04.297188+0000", 2024-09-14T06:29:07.458 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.0189999999999995e-06 2024-09-14T06:29:07.458 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:07.458 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:07.458 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:29:07.458 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:04.297255+0000", 2024-09-14T06:29:07.458 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.7112000000000004e-05 2024-09-14T06:29:07.458 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:07.458 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:07.458 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:29:07.458 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:04.297279+0000", 2024-09-14T06:29:07.458 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.4017000000000001e-05 2024-09-14T06:29:07.458 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:07.458 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:07.459 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-14T06:29:07.459 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:04.297966+0000", 2024-09-14T06:29:07.459 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00068650600000000003 2024-09-14T06:29:07.459 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:07.459 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:07.459 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:29:07.459 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:04.298002+0000", 2024-09-14T06:29:07.459 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.6186999999999998e-05 2024-09-14T06:29:07.459 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:07.459 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:07.459 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:07.459 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:07.459 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:07.459 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4397.0:66 3.4 e27/23 3:27567f25:::rbd_object_map.112d7adb82ff:head v 27'6, mlcod=27'4)", 2024-09-14T06:29:07.460 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:04.303066+0000", 2024-09-14T06:29:07.460 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 3.1402460400000001, 2024-09-14T06:29:07.460 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00083700699999999996, 2024-09-14T06:29:07.460 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:29:07.460 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:29:07.460 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:29:07.460 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:29:07.460 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:07.460 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:29:07.460 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:04.303066+0000", 2024-09-14T06:29:07.460 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:07.460 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:07.460 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:07.461 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:29:07.461 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:04.303066+0000", 2024-09-14T06:29:07.461 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:07.461 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:07.461 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:07.461 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:29:07.461 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:04.303067+0000", 2024-09-14T06:29:07.461 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.1880000000000001e-06 2024-09-14T06:29:07.461 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:07.461 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:07.461 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:29:07.461 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:04.303075+0000", 2024-09-14T06:29:07.461 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.1410000000000005e-06 2024-09-14T06:29:07.462 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:07.462 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:07.462 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:29:07.462 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:04.303091+0000", 2024-09-14T06:29:07.462 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.5928000000000001e-05 2024-09-14T06:29:07.462 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:07.462 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:07.462 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:29:07.462 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:04.303100+0000", 2024-09-14T06:29:07.462 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.2019999999999993e-06 2024-09-14T06:29:07.462 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:07.462 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:07.462 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:29:07.462 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:04.303168+0000", 2024-09-14T06:29:07.463 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.7754000000000002e-05 2024-09-14T06:29:07.463 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:07.463 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:07.463 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:29:07.463 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:04.303191+0000", 2024-09-14T06:29:07.463 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.3434e-05 2024-09-14T06:29:07.463 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:07.463 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:07.463 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-14T06:29:07.463 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:04.303872+0000", 2024-09-14T06:29:07.463 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00068037399999999997 2024-09-14T06:29:07.463 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:07.463 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:07.464 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:29:07.464 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:04.303903+0000", 2024-09-14T06:29:07.464 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.0985999999999998e-05 2024-09-14T06:29:07.464 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:07.464 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:07.464 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:07.464 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:07.464 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:07.464 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4397.0:68 3.4 e27/23 3:27567f25:::rbd_object_map.112d7adb82ff:head v 27'7, mlcod=27'5)", 2024-09-14T06:29:07.464 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:04.314113+0000", 2024-09-14T06:29:07.464 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 3.1291991800000001, 2024-09-14T06:29:07.465 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00086905299999999995, 2024-09-14T06:29:07.465 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:29:07.465 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:29:07.465 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:29:07.465 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:29:07.465 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:07.465 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:29:07.465 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:04.314113+0000", 2024-09-14T06:29:07.465 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:07.465 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:07.465 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:07.465 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:29:07.466 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:04.314113+0000", 2024-09-14T06:29:07.466 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:07.466 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:07.466 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:07.466 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:29:07.466 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:04.314114+0000", 2024-09-14T06:29:07.466 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.2780000000000001e-06 2024-09-14T06:29:07.466 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:07.466 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:07.466 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:29:07.466 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:04.314133+0000", 2024-09-14T06:29:07.466 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.95e-05 2024-09-14T06:29:07.466 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:07.466 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:07.467 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:29:07.467 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:04.314149+0000", 2024-09-14T06:29:07.467 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.5752000000000001e-05 2024-09-14T06:29:07.467 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:07.467 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:07.467 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:29:07.467 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:04.314163+0000", 2024-09-14T06:29:07.467 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.3910000000000001e-05 2024-09-14T06:29:07.467 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:07.467 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:07.467 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:29:07.467 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:04.314234+0000", 2024-09-14T06:29:07.467 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.0564999999999999e-05 2024-09-14T06:29:07.467 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:07.467 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:07.468 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:29:07.468 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:04.314258+0000", 2024-09-14T06:29:07.468 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.4040999999999999e-05 2024-09-14T06:29:07.468 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:07.468 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:07.468 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-14T06:29:07.468 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:04.314950+0000", 2024-09-14T06:29:07.468 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00069187399999999998 2024-09-14T06:29:07.468 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:07.468 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:07.468 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:29:07.469 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:04.314982+0000", 2024-09-14T06:29:07.469 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.2132999999999999e-05 2024-09-14T06:29:07.469 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:07.469 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:07.469 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:07.469 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:07.469 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:07.469 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4397.0:72 3.4 e27/23 3:27567f25:::rbd_object_map.112d7adb82ff:head v 27'9, mlcod=27'7)", 2024-09-14T06:29:07.469 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:04.320357+0000", 2024-09-14T06:29:07.469 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 3.1229547630000001, 2024-09-14T06:29:07.469 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00077947800000000001, 2024-09-14T06:29:07.469 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:29:07.469 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:29:07.469 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:29:07.470 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:29:07.470 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:07.470 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:29:07.470 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:04.320357+0000", 2024-09-14T06:29:07.470 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:07.470 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:07.470 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:07.470 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:29:07.470 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:04.320357+0000", 2024-09-14T06:29:07.470 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:07.470 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:07.470 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:07.470 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:29:07.470 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:04.320358+0000", 2024-09-14T06:29:07.471 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.9600000000000008e-07 2024-09-14T06:29:07.471 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:07.471 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:07.471 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:29:07.471 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:04.320366+0000", 2024-09-14T06:29:07.471 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.9920000000000007e-06 2024-09-14T06:29:07.471 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:07.471 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:07.471 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:29:07.471 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:04.320380+0000", 2024-09-14T06:29:07.471 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.4056000000000001e-05 2024-09-14T06:29:07.471 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:07.471 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:07.471 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:29:07.472 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:04.320388+0000", 2024-09-14T06:29:07.472 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.6660000000000008e-06 2024-09-14T06:29:07.472 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:07.472 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:07.472 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:29:07.472 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:04.320445+0000", 2024-09-14T06:29:07.472 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.7268000000000001e-05 2024-09-14T06:29:07.472 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:07.472 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:07.472 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:29:07.472 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:04.320465+0000", 2024-09-14T06:29:07.472 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.0355000000000001e-05 2024-09-14T06:29:07.472 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:07.472 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:07.472 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-14T06:29:07.473 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:04.321109+0000", 2024-09-14T06:29:07.473 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00064390699999999999 2024-09-14T06:29:07.473 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:07.473 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:07.473 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:29:07.473 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:04.321137+0000", 2024-09-14T06:29:07.473 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.7237999999999999e-05 2024-09-14T06:29:07.473 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:07.473 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:07.473 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:07.473 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:07.473 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:07.473 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4397.0:95 3.4 e27/23 3:27567f25:::rbd_object_map.112d7adb82ff:head v 27'13, mlcod=27'11)", 2024-09-14T06:29:07.473 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:04.353692+0000", 2024-09-14T06:29:07.474 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 3.0896194019999998, 2024-09-14T06:29:07.474 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00082120299999999995, 2024-09-14T06:29:07.474 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:29:07.474 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:29:07.474 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:29:07.474 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:29:07.474 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:07.474 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:29:07.474 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:04.353692+0000", 2024-09-14T06:29:07.474 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:07.474 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:07.474 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:07.474 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:29:07.474 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:04.353692+0000", 2024-09-14T06:29:07.475 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:07.475 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:07.475 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:07.475 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:29:07.475 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:04.353693+0000", 2024-09-14T06:29:07.475 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.23e-07 2024-09-14T06:29:07.475 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:07.475 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:07.475 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:29:07.475 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:04.353707+0000", 2024-09-14T06:29:07.475 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.3947999999999999e-05 2024-09-14T06:29:07.475 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:07.475 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:07.476 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:29:07.476 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:04.353717+0000", 2024-09-14T06:29:07.476 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.9809999999999998e-06 2024-09-14T06:29:07.476 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:07.476 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:07.476 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:29:07.476 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:04.353722+0000", 2024-09-14T06:29:07.476 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.7210000000000002e-06 2024-09-14T06:29:07.476 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:07.476 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:07.476 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:29:07.476 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:04.353778+0000", 2024-09-14T06:29:07.477 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.6675000000000002e-05 2024-09-14T06:29:07.477 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:07.477 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:07.477 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:29:07.477 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:04.353802+0000", 2024-09-14T06:29:07.477 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.3422e-05 2024-09-14T06:29:07.477 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:07.477 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:07.477 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-14T06:29:07.477 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:04.354486+0000", 2024-09-14T06:29:07.477 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00068423799999999999 2024-09-14T06:29:07.477 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:07.477 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:07.477 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:29:07.477 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:04.354514+0000", 2024-09-14T06:29:07.478 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.7495e-05 2024-09-14T06:29:07.478 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:07.478 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:07.478 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:07.478 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:07.478 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:07.478 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4397.0:114 3.4 e27/23 3:27567f25:::rbd_object_map.112d7adb82ff:head v 27'16, mlcod=27'14)", 2024-09-14T06:29:07.478 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:04.372433+0000", 2024-09-14T06:29:07.478 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 3.0708784800000002, 2024-09-14T06:29:07.478 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00088299699999999995, 2024-09-14T06:29:07.478 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:29:07.478 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:29:07.478 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:29:07.479 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:29:07.479 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:07.479 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:29:07.479 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:04.372433+0000", 2024-09-14T06:29:07.479 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:07.479 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:07.479 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:07.479 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:29:07.479 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:04.372433+0000", 2024-09-14T06:29:07.479 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:07.479 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:07.479 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:07.479 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:29:07.479 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:04.372435+0000", 2024-09-14T06:29:07.480 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.655e-06 2024-09-14T06:29:07.480 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:07.480 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:07.480 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:29:07.480 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:04.372443+0000", 2024-09-14T06:29:07.480 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.6809999999999998e-06 2024-09-14T06:29:07.480 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:07.480 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:07.480 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:29:07.480 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:04.372458+0000", 2024-09-14T06:29:07.480 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.4941e-05 2024-09-14T06:29:07.480 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:07.480 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:07.481 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:29:07.481 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:04.372475+0000", 2024-09-14T06:29:07.481 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.6886000000000001e-05 2024-09-14T06:29:07.481 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:07.481 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:07.481 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:29:07.481 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:04.372530+0000", 2024-09-14T06:29:07.481 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.507e-05 2024-09-14T06:29:07.481 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:07.481 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:07.481 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:29:07.481 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:04.372555+0000", 2024-09-14T06:29:07.481 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.5785000000000001e-05 2024-09-14T06:29:07.481 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:07.482 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:07.482 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-14T06:29:07.482 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:04.373275+0000", 2024-09-14T06:29:07.482 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00071917599999999997 2024-09-14T06:29:07.482 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:07.482 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:07.482 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:29:07.482 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:04.373316+0000", 2024-09-14T06:29:07.482 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.1802999999999999e-05 2024-09-14T06:29:07.482 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:07.482 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:07.482 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:07.482 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:07.482 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:07.483 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4397.0:116 3.4 e27/23 3:27567f25:::rbd_object_map.112d7adb82ff:head v 27'17, mlcod=27'15)", 2024-09-14T06:29:07.483 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:04.384169+0000", 2024-09-14T06:29:07.483 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 3.0591429699999999, 2024-09-14T06:29:07.483 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00083693600000000004, 2024-09-14T06:29:07.483 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:29:07.483 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:29:07.483 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:29:07.483 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:29:07.483 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:07.483 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:29:07.483 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:04.384169+0000", 2024-09-14T06:29:07.483 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:07.483 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:07.483 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:07.484 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:29:07.484 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:04.384169+0000", 2024-09-14T06:29:07.484 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:07.484 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:07.484 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:07.484 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:29:07.484 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:04.384170+0000", 2024-09-14T06:29:07.484 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.359e-06 2024-09-14T06:29:07.484 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:07.484 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:07.484 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:29:07.484 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:04.384179+0000", 2024-09-14T06:29:07.484 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.1919999999999994e-06 2024-09-14T06:29:07.485 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:07.485 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:07.485 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:29:07.485 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:04.384198+0000", 2024-09-14T06:29:07.485 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.8879999999999999e-05 2024-09-14T06:29:07.485 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:07.485 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:07.485 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:29:07.485 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:04.384208+0000", 2024-09-14T06:29:07.485 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.3300000000000005e-06 2024-09-14T06:29:07.485 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:07.485 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:07.485 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:29:07.486 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:04.384273+0000", 2024-09-14T06:29:07.486 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.5275000000000001e-05 2024-09-14T06:29:07.486 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:07.486 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:07.486 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:29:07.486 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:04.384291+0000", 2024-09-14T06:29:07.486 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.8139999999999999e-05 2024-09-14T06:29:07.486 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:07.486 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:07.486 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-14T06:29:07.486 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:04.384963+0000", 2024-09-14T06:29:07.486 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00067157199999999995 2024-09-14T06:29:07.486 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:07.486 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:07.487 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:29:07.487 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:04.385006+0000", 2024-09-14T06:29:07.487 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.3188000000000001e-05 2024-09-14T06:29:07.487 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:07.487 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:07.487 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:07.487 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:07.487 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:07.487 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4397.0:130 3.4 e27/23 3:27567f25:::rbd_object_map.112d7adb82ff:head v 27'21, mlcod=27'19)", 2024-09-14T06:29:07.487 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:04.405039+0000", 2024-09-14T06:29:07.487 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 3.0382723540000001, 2024-09-14T06:29:07.487 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000900945, 2024-09-14T06:29:07.487 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:29:07.487 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:29:07.488 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:29:07.488 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:29:07.488 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:07.488 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:29:07.488 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:04.405039+0000", 2024-09-14T06:29:07.488 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:07.488 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:07.488 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:07.488 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:29:07.488 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:04.405039+0000", 2024-09-14T06:29:07.488 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:07.488 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:07.488 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:07.489 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:29:07.489 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:04.405041+0000", 2024-09-14T06:29:07.489 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.516e-06 2024-09-14T06:29:07.489 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:07.489 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:07.489 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:29:07.489 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:04.405054+0000", 2024-09-14T06:29:07.489 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.3383999999999999e-05 2024-09-14T06:29:07.489 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:07.489 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:07.489 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:29:07.489 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:04.405073+0000", 2024-09-14T06:29:07.489 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.8598e-05 2024-09-14T06:29:07.490 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:07.490 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:07.490 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:29:07.490 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:04.405082+0000", 2024-09-14T06:29:07.490 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.4390000000000006e-06 2024-09-14T06:29:07.490 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:07.490 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:07.490 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:29:07.490 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:04.405164+0000", 2024-09-14T06:29:07.490 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.1358000000000003e-05 2024-09-14T06:29:07.490 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:07.490 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:07.490 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:29:07.490 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:04.405192+0000", 2024-09-14T06:29:07.491 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.8648000000000001e-05 2024-09-14T06:29:07.491 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:07.491 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:07.491 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-14T06:29:07.491 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:04.405897+0000", 2024-09-14T06:29:07.491 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00070438900000000004 2024-09-14T06:29:07.491 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:07.491 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:07.491 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:29:07.491 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:04.405940+0000", 2024-09-14T06:29:07.491 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.3612999999999997e-05 2024-09-14T06:29:07.491 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:07.491 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:07.491 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:07.492 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:07.492 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:07.492 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4397.0:141 3.4 e27/23 3:27567f25:::rbd_object_map.112d7adb82ff:head v 27'23, mlcod=27'21)", 2024-09-14T06:29:07.492 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:04.417903+0000", 2024-09-14T06:29:07.492 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 3.0254089849999999, 2024-09-14T06:29:07.492 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00080468100000000002, 2024-09-14T06:29:07.492 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:29:07.492 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:29:07.492 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:29:07.492 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:29:07.492 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:07.492 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:29:07.492 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:04.417903+0000", 2024-09-14T06:29:07.492 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:07.493 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:07.493 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:07.493 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:29:07.493 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:04.417903+0000", 2024-09-14T06:29:07.493 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:07.493 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:07.493 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:07.493 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:29:07.493 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:04.417904+0000", 2024-09-14T06:29:07.493 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.2100000000000001e-06 2024-09-14T06:29:07.493 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:07.493 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:07.493 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:29:07.493 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:04.417912+0000", 2024-09-14T06:29:07.494 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.7819999999999994e-06 2024-09-14T06:29:07.494 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:07.494 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:07.494 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:29:07.494 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:04.417927+0000", 2024-09-14T06:29:07.494 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.5537999999999999e-05 2024-09-14T06:29:07.494 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:07.494 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:07.494 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:29:07.494 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:04.417936+0000", 2024-09-14T06:29:07.494 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.5140000000000001e-06 2024-09-14T06:29:07.494 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:07.494 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:07.494 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:29:07.495 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:04.418003+0000", 2024-09-14T06:29:07.495 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.7365000000000003e-05 2024-09-14T06:29:07.495 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:07.495 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:07.495 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:29:07.495 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:04.418028+0000", 2024-09-14T06:29:07.495 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.4396000000000002e-05 2024-09-14T06:29:07.495 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:07.495 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:07.495 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-14T06:29:07.495 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:04.418654+0000", 2024-09-14T06:29:07.495 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00062674699999999998 2024-09-14T06:29:07.495 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:07.495 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:07.495 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:29:07.496 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:04.418708+0000", 2024-09-14T06:29:07.496 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.3128999999999999e-05 2024-09-14T06:29:07.496 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:07.496 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:07.496 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:07.496 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:07.496 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:07.496 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4397.0:164 3.4 e27/23 3:27567f25:::rbd_object_map.112d7adb82ff:head v 27'27, mlcod=27'25)", 2024-09-14T06:29:07.496 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:04.438987+0000", 2024-09-14T06:29:07.496 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 3.0043244609999999, 2024-09-14T06:29:07.496 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00093266400000000004, 2024-09-14T06:29:07.496 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:29:07.496 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:29:07.496 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:29:07.497 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:29:07.497 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:07.497 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:29:07.497 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:04.438987+0000", 2024-09-14T06:29:07.497 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:07.497 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:07.497 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:07.497 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:29:07.497 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:04.438987+0000", 2024-09-14T06:29:07.497 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:07.497 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:07.497 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:07.497 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:29:07.497 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:04.438989+0000", 2024-09-14T06:29:07.498 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.1829999999999999e-06 2024-09-14T06:29:07.498 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:07.498 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:07.498 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:29:07.498 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:04.438998+0000", 2024-09-14T06:29:07.498 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.5489999999999995e-06 2024-09-14T06:29:07.498 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:07.498 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:07.498 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:29:07.498 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:04.439014+0000", 2024-09-14T06:29:07.498 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.5659e-05 2024-09-14T06:29:07.498 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:07.498 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:07.498 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:29:07.499 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:04.439024+0000", 2024-09-14T06:29:07.499 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.0229e-05 2024-09-14T06:29:07.499 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:07.499 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:07.499 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:29:07.499 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:04.439085+0000", 2024-09-14T06:29:07.499 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.1507000000000006e-05 2024-09-14T06:29:07.499 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:07.499 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:07.499 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:29:07.499 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:04.439111+0000", 2024-09-14T06:29:07.499 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.5479000000000001e-05 2024-09-14T06:29:07.499 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:07.499 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:07.500 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-14T06:29:07.500 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:04.439872+0000", 2024-09-14T06:29:07.500 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00076089899999999999 2024-09-14T06:29:07.500 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:07.500 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:07.500 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:29:07.500 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:04.439920+0000", 2024-09-14T06:29:07.500 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.8158999999999997e-05 2024-09-14T06:29:07.500 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:07.500 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:07.500 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:07.500 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:07.500 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:07.500 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4397.0:177 3.4 e27/23 3:27567f25:::rbd_object_map.112d7adb82ff:head v 27'30, mlcod=27'28)", 2024-09-14T06:29:07.501 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:04.451688+0000", 2024-09-14T06:29:07.501 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 2.9916240589999998, 2024-09-14T06:29:07.501 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00082312599999999998, 2024-09-14T06:29:07.501 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:29:07.501 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:29:07.501 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:29:07.501 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:29:07.501 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:07.501 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:29:07.501 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:04.451688+0000", 2024-09-14T06:29:07.501 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:07.501 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:07.501 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:07.501 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:29:07.502 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:04.451688+0000", 2024-09-14T06:29:07.502 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:07.502 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:07.502 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:07.502 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:29:07.502 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:04.451690+0000", 2024-09-14T06:29:07.502 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.857e-06 2024-09-14T06:29:07.502 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:07.502 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:07.502 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:29:07.502 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:04.451699+0000", 2024-09-14T06:29:07.502 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.5910000000000002e-06 2024-09-14T06:29:07.502 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:07.502 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:07.503 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:29:07.503 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:04.451718+0000", 2024-09-14T06:29:07.503 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.8609000000000001e-05 2024-09-14T06:29:07.503 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:07.503 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:07.503 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:29:07.503 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:04.451730+0000", 2024-09-14T06:29:07.503 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.1776999999999999e-05 2024-09-14T06:29:07.503 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:07.503 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:07.503 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:29:07.503 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:04.451793+0000", 2024-09-14T06:29:07.503 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.3841000000000001e-05 2024-09-14T06:29:07.504 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:07.504 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:07.504 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:29:07.504 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:04.451823+0000", 2024-09-14T06:29:07.504 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.9241999999999999e-05 2024-09-14T06:29:07.504 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:07.504 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:07.504 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-14T06:29:07.504 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:04.452462+0000", 2024-09-14T06:29:07.504 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00063914699999999996 2024-09-14T06:29:07.504 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:07.504 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:07.504 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:29:07.504 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:04.452511+0000", 2024-09-14T06:29:07.505 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.9061999999999999e-05 2024-09-14T06:29:07.505 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:07.505 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:07.505 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:07.505 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:07.505 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:07.505 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4397.0:186 3.4 e27/23 3:27567f25:::rbd_object_map.112d7adb82ff:head v 27'31, mlcod=27'29)", 2024-09-14T06:29:07.505 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:04.457782+0000", 2024-09-14T06:29:07.505 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 2.985529326, 2024-09-14T06:29:07.505 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00083834100000000004, 2024-09-14T06:29:07.505 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:29:07.505 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:29:07.505 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:29:07.506 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:29:07.506 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:07.506 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:29:07.506 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:04.457782+0000", 2024-09-14T06:29:07.506 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:07.506 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:07.506 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:07.506 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:29:07.506 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:04.457782+0000", 2024-09-14T06:29:07.506 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:07.506 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:07.506 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:07.506 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:29:07.506 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:04.457783+0000", 2024-09-14T06:29:07.507 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.001e-06 2024-09-14T06:29:07.507 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:07.507 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:07.507 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:29:07.507 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:04.457801+0000", 2024-09-14T06:29:07.507 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.7119999999999999e-05 2024-09-14T06:29:07.507 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:07.507 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:07.507 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:29:07.507 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:04.457817+0000", 2024-09-14T06:29:07.507 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.6761000000000001e-05 2024-09-14T06:29:07.507 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:07.507 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:07.508 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:29:07.508 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:04.457826+0000", 2024-09-14T06:29:07.508 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.3059999999999998e-06 2024-09-14T06:29:07.508 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:07.508 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:07.508 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:29:07.508 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:04.457883+0000", 2024-09-14T06:29:07.508 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.7420000000000003e-05 2024-09-14T06:29:07.508 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:07.508 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:07.508 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:29:07.508 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:04.457906+0000", 2024-09-14T06:29:07.508 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.2643e-05 2024-09-14T06:29:07.509 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:07.509 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:07.509 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-14T06:29:07.509 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:04.458570+0000", 2024-09-14T06:29:07.509 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00066450099999999996 2024-09-14T06:29:07.509 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:07.509 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:07.509 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:29:07.509 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:04.458621+0000", 2024-09-14T06:29:07.509 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.0589e-05 2024-09-14T06:29:07.509 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:07.509 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:07.509 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:07.509 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:07.510 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:07.510 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4397.0:188 3.4 e27/23 3:27567f25:::rbd_object_map.112d7adb82ff:head v 27'32, mlcod=27'30)", 2024-09-14T06:29:07.510 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:04.461213+0000", 2024-09-14T06:29:07.510 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 2.9820986299999999, 2024-09-14T06:29:07.510 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00079458399999999998, 2024-09-14T06:29:07.510 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:29:07.510 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:29:07.510 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:29:07.510 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:29:07.510 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:07.510 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:29:07.510 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:04.461213+0000", 2024-09-14T06:29:07.510 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:07.510 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:07.511 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:07.511 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:29:07.511 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:04.461213+0000", 2024-09-14T06:29:07.511 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:07.511 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:07.511 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:07.511 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:29:07.511 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:04.461214+0000", 2024-09-14T06:29:07.511 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.1200000000000002e-07 2024-09-14T06:29:07.511 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:07.511 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:07.511 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:29:07.511 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:04.461219+0000", 2024-09-14T06:29:07.511 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.8980000000000004e-06 2024-09-14T06:29:07.512 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:07.512 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:07.512 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:29:07.512 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:04.461229+0000", 2024-09-14T06:29:07.512 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.0073e-05 2024-09-14T06:29:07.512 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:07.512 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:07.512 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:29:07.512 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:04.461237+0000", 2024-09-14T06:29:07.512 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.0819999999999999e-06 2024-09-14T06:29:07.512 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:07.512 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:07.512 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:29:07.512 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:04.461291+0000", 2024-09-14T06:29:07.513 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.3667e-05 2024-09-14T06:29:07.513 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:07.513 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:07.513 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:29:07.513 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:04.461311+0000", 2024-09-14T06:29:07.513 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.0259999999999999e-05 2024-09-14T06:29:07.513 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:07.513 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:07.513 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-14T06:29:07.513 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:04.461955+0000", 2024-09-14T06:29:07.513 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00064418199999999998 2024-09-14T06:29:07.513 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:07.513 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:07.513 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:29:07.514 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:04.462008+0000", 2024-09-14T06:29:07.514 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.2710000000000002e-05 2024-09-14T06:29:07.514 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:07.514 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:07.514 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:07.514 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:07.514 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:07.514 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4397.0:190 3.4 e27/23 3:27567f25:::rbd_object_map.112d7adb82ff:head v 27'33, mlcod=27'31)", 2024-09-14T06:29:07.514 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:04.464398+0000", 2024-09-14T06:29:07.514 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 2.978913774, 2024-09-14T06:29:07.514 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00081911799999999997, 2024-09-14T06:29:07.514 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:29:07.514 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:29:07.514 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:29:07.515 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:29:07.515 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:07.515 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:29:07.515 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:04.464398+0000", 2024-09-14T06:29:07.515 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:07.515 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:07.515 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:07.515 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:29:07.515 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:04.464398+0000", 2024-09-14T06:29:07.515 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:07.515 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:07.515 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:07.515 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:29:07.515 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:04.464399+0000", 2024-09-14T06:29:07.516 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.6499999999999998e-07 2024-09-14T06:29:07.516 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:07.516 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:07.516 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:29:07.516 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:04.464404+0000", 2024-09-14T06:29:07.516 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.1549999999999998e-06 2024-09-14T06:29:07.516 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:07.516 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:07.516 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:29:07.516 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:04.464414+0000", 2024-09-14T06:29:07.516 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.9650000000000003e-06 2024-09-14T06:29:07.516 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:07.516 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:07.517 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:29:07.517 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:04.464419+0000", 2024-09-14T06:29:07.517 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.685e-06 2024-09-14T06:29:07.517 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:07.517 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:07.517 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:29:07.517 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:04.464470+0000", 2024-09-14T06:29:07.517 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.1255000000000003e-05 2024-09-14T06:29:07.517 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:07.517 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:07.517 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:29:07.517 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:04.464491+0000", 2024-09-14T06:29:07.517 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.1308999999999999e-05 2024-09-14T06:29:07.517 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:07.518 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:07.518 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-14T06:29:07.518 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:04.465177+0000", 2024-09-14T06:29:07.518 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00068529500000000004 2024-09-14T06:29:07.518 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:07.518 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:07.518 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:29:07.518 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:04.465217+0000", 2024-09-14T06:29:07.518 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.0589000000000001e-05 2024-09-14T06:29:07.518 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:07.518 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:07.518 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:07.518 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:07.518 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:07.519 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4397.0:616 3.9 e30/23 3:908d0935:::rbd_data.112d7dfbd766.000000000000007e:head v 30'2, mlcod=0'0)", 2024-09-14T06:29:07.519 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:06.780526+0000", 2024-09-14T06:29:07.519 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 0.66278555100000003, 2024-09-14T06:29:07.519 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00091880900000000003, 2024-09-14T06:29:07.519 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:29:07.519 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:29:07.519 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:29:07.519 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:29:07.519 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:07.519 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:29:07.519 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:06.780526+0000", 2024-09-14T06:29:07.519 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:07.519 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:07.519 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:07.520 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:29:07.520 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:06.780526+0000", 2024-09-14T06:29:07.520 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:07.520 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:07.520 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:07.520 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:29:07.520 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:06.780527+0000", 2024-09-14T06:29:07.520 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.3799999999999998e-07 2024-09-14T06:29:07.520 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:07.520 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:07.520 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:29:07.520 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:06.780625+0000", 2024-09-14T06:29:07.520 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.8785999999999999e-05 2024-09-14T06:29:07.520 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:07.521 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:07.521 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:29:07.521 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:06.780640+0000", 2024-09-14T06:29:07.521 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.4363e-05 2024-09-14T06:29:07.521 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:07.521 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:07.521 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:29:07.521 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:06.780650+0000", 2024-09-14T06:29:07.521 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.0164e-05 2024-09-14T06:29:07.521 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:07.521 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:07.521 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:29:07.521 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:06.780702+0000", 2024-09-14T06:29:07.522 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.1992e-05 2024-09-14T06:29:07.522 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:07.522 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:07.522 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:29:07.522 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:06.780718+0000", 2024-09-14T06:29:07.522 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.6161e-05 2024-09-14T06:29:07.522 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:07.522 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:07.522 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-14T06:29:07.522 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:06.781404+0000", 2024-09-14T06:29:07.522 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000685526 2024-09-14T06:29:07.522 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:07.522 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:07.522 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:29:07.523 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:06.781445+0000", 2024-09-14T06:29:07.523 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.1479e-05 2024-09-14T06:29:07.523 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:07.523 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:07.523 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:07.523 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:07.523 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:07.523 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4397.0:792 3.9 e30/23 3:977d4c6a:::rbd_data.112d7dfbd766.000000000000014d:head v 30'4, mlcod=30'2)", 2024-09-14T06:29:07.523 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:06.998893+0000", 2024-09-14T06:29:07.523 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 0.444418863, 2024-09-14T06:29:07.523 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00089658700000000001, 2024-09-14T06:29:07.523 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:29:07.523 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:29:07.523 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:29:07.524 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:29:07.524 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:07.524 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:29:07.524 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:06.998893+0000", 2024-09-14T06:29:07.524 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:07.524 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:07.524 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:07.524 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:29:07.524 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:06.998893+0000", 2024-09-14T06:29:07.524 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:07.524 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:07.524 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:07.524 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:29:07.524 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:06.998894+0000", 2024-09-14T06:29:07.525 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.2550000000000001e-06 2024-09-14T06:29:07.525 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:07.525 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:07.525 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:29:07.525 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:06.998912+0000", 2024-09-14T06:29:07.525 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.7818000000000001e-05 2024-09-14T06:29:07.525 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:07.525 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:07.525 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:29:07.525 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:06.998926+0000", 2024-09-14T06:29:07.525 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.4368e-05 2024-09-14T06:29:07.525 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:07.525 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:07.525 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:29:07.526 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:06.998934+0000", 2024-09-14T06:29:07.526 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.6839999999999996e-06 2024-09-14T06:29:07.526 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:07.526 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:07.526 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:29:07.526 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:06.998981+0000", 2024-09-14T06:29:07.526 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.6995e-05 2024-09-14T06:29:07.526 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:07.526 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:07.526 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:29:07.526 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:06.998998+0000", 2024-09-14T06:29:07.526 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.6628000000000001e-05 2024-09-14T06:29:07.526 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:07.526 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:07.526 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-14T06:29:07.527 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:06.999757+0000", 2024-09-14T06:29:07.527 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00075925099999999996 2024-09-14T06:29:07.527 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:07.527 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:07.527 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:29:07.527 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:06.999790+0000", 2024-09-14T06:29:07.527 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.2588e-05 2024-09-14T06:29:07.527 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:07.527 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:07.527 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:07.527 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:07.527 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:07.527 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4397.0:803 3.9 e30/23 3:9762c2a9:::rbd_data.112d7dfbd766.0000000000000054:head v 30'5, mlcod=30'3)", 2024-09-14T06:29:07.527 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:07.013871+0000", 2024-09-14T06:29:07.528 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 0.42944112899999998, 2024-09-14T06:29:07.528 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000824587, 2024-09-14T06:29:07.528 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:29:07.528 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:29:07.528 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:29:07.528 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:29:07.528 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:07.528 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:29:07.528 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:07.013871+0000", 2024-09-14T06:29:07.528 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:07.528 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:07.528 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:07.528 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:29:07.529 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:07.013871+0000", 2024-09-14T06:29:07.529 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:07.529 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:07.529 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:07.529 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:29:07.529 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:07.013872+0000", 2024-09-14T06:29:07.529 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.7999999999999999e-06 2024-09-14T06:29:07.529 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:07.529 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:07.529 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:29:07.529 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:07.013896+0000", 2024-09-14T06:29:07.529 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.3178999999999999e-05 2024-09-14T06:29:07.529 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:07.530 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:07.530 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:29:07.530 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:07.013921+0000", 2024-09-14T06:29:07.530 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.4956e-05 2024-09-14T06:29:07.530 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:07.530 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:07.530 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:29:07.530 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:07.013946+0000", 2024-09-14T06:29:07.530 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.5046999999999999e-05 2024-09-14T06:29:07.530 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:07.530 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:07.530 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:29:07.530 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:07.014042+0000", 2024-09-14T06:29:07.530 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.5968e-05 2024-09-14T06:29:07.530 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:07.531 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:07.531 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:29:07.531 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:07.014080+0000", 2024-09-14T06:29:07.531 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.8843000000000001e-05 2024-09-14T06:29:07.531 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:07.531 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:07.531 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-14T06:29:07.531 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:07.014670+0000", 2024-09-14T06:29:07.531 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00058908699999999997 2024-09-14T06:29:07.531 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:07.531 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:07.531 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:29:07.531 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:07.014695+0000", 2024-09-14T06:29:07.531 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.5707e-05 2024-09-14T06:29:07.531 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:07.532 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:07.532 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:07.532 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:07.532 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:07.532 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4397.0:937 3.9 e30/23 3:92b37d3e:::rbd_data.112d7dfbd766.0000000000000012:head v 30'7, mlcod=30'5)", 2024-09-14T06:29:07.532 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:07.169523+0000", 2024-09-14T06:29:07.532 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 0.27378899499999998, 2024-09-14T06:29:07.532 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0013381339999999999, 2024-09-14T06:29:07.532 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:29:07.532 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:29:07.532 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:29:07.532 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:29:07.532 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:07.532 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:29:07.533 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:07.169523+0000", 2024-09-14T06:29:07.533 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:07.533 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:07.533 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:07.533 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:29:07.533 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:07.169523+0000", 2024-09-14T06:29:07.533 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:07.533 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:07.533 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:07.533 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:29:07.533 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:07.169524+0000", 2024-09-14T06:29:07.533 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.125e-06 2024-09-14T06:29:07.533 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:07.534 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:07.534 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:29:07.534 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:07.170106+0000", 2024-09-14T06:29:07.534 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00058239900000000005 2024-09-14T06:29:07.534 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:07.534 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:07.534 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:29:07.534 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:07.170121+0000", 2024-09-14T06:29:07.534 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.4851999999999999e-05 2024-09-14T06:29:07.534 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:07.534 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:07.534 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:29:07.534 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:07.170130+0000", 2024-09-14T06:29:07.535 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.9199999999999993e-06 2024-09-14T06:29:07.535 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:07.535 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:07.535 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:29:07.535 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:07.170182+0000", 2024-09-14T06:29:07.535 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.1669999999999998e-05 2024-09-14T06:29:07.535 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:07.535 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:07.535 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:29:07.535 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:07.170199+0000", 2024-09-14T06:29:07.535 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.7193999999999998e-05 2024-09-14T06:29:07.535 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:07.535 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:07.535 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-14T06:29:07.536 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:07.170816+0000", 2024-09-14T06:29:07.536 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00061745400000000003 2024-09-14T06:29:07.536 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:07.536 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:07.536 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:29:07.536 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:07.170861+0000", 2024-09-14T06:29:07.536 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.4520000000000001e-05 2024-09-14T06:29:07.536 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:07.536 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:07.536 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:07.536 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:07.536 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:07.536 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-14T06:29:07.538 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-14T06:29:07.538 INFO:tasks.ceph.osd.6.smithi138.stderr:2024-09-14T06:29:07.536+0000 7f8cd8849640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33620) UID: 0 2024-09-14T06:29:07.639 INFO:tasks.ceph.osd.5.smithi138.stderr:2024-09-14T06:29:07.638+0000 7fe13746f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33618) UID: 0 2024-09-14T06:29:07.647 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-14T06:29:07.648 INFO:teuthology.orchestra.run.smithi094.stdout: "ops": [], 2024-09-14T06:29:07.648 INFO:teuthology.orchestra.run.smithi094.stdout: "num_ops": 0 2024-09-14T06:29:07.648 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-14T06:29:07.657 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-14T06:29:07.740 INFO:tasks.ceph.osd.0.smithi094.stderr:2024-09-14T06:29:07.739+0000 7f07c9333640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33636) UID: 0 2024-09-14T06:29:07.758 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-14T06:29:07.758 INFO:teuthology.orchestra.run.smithi094.stdout: "ops": [], 2024-09-14T06:29:07.759 INFO:teuthology.orchestra.run.smithi094.stdout: "complaint_time": 30, 2024-09-14T06:29:07.759 INFO:teuthology.orchestra.run.smithi094.stdout: "num_blocked_ops": 0 2024-09-14T06:29:07.759 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-14T06:29:07.767 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-14T06:29:07.851 INFO:tasks.ceph.osd.2.smithi094.stderr:2024-09-14T06:29:07.850+0000 7faf3269b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33649) UID: 0 2024-09-14T06:29:07.912 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-14T06:29:07.912 INFO:teuthology.orchestra.run.smithi094.stdout: "size": 20, 2024-09-14T06:29:07.912 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 600, 2024-09-14T06:29:07.912 INFO:teuthology.orchestra.run.smithi094.stdout: "ops": [ 2024-09-14T06:29:07.912 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:07.912 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:115 3.2 3:427fbb68:::rbd_data.112d7adb82ff.0000000000000034:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3041633~40235] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-14T06:29:07.912 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:04.374132+0000", 2024-09-14T06:29:07.913 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 3.5356041989999998, 2024-09-14T06:29:07.913 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0014800220000000001, 2024-09-14T06:29:07.913 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:29:07.913 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:29:07.913 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:29:07.913 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:29:07.913 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:29:07.913 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:07.913 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 115 2024-09-14T06:29:07.913 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:07.913 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:29:07.913 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:07.913 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:29:07.914 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:04.374132+0000", 2024-09-14T06:29:07.914 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:07.914 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:07.914 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:07.914 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:29:07.914 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:04.374132+0000", 2024-09-14T06:29:07.914 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:07.914 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:07.914 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:07.914 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:29:07.914 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:04.374133+0000", 2024-09-14T06:29:07.914 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.175e-06 2024-09-14T06:29:07.914 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:07.914 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:07.915 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:29:07.915 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:04.374165+0000", 2024-09-14T06:29:07.915 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.2322e-05 2024-09-14T06:29:07.915 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:07.915 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:07.915 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:29:07.915 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:04.374174+0000", 2024-09-14T06:29:07.915 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.2789999999999998e-06 2024-09-14T06:29:07.915 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:07.915 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:07.915 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:29:07.915 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:04.374179+0000", 2024-09-14T06:29:07.915 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.8810000000000001e-06 2024-09-14T06:29:07.916 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:07.916 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:07.916 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:29:07.916 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:04.374237+0000", 2024-09-14T06:29:07.916 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.7260999999999999e-05 2024-09-14T06:29:07.916 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:07.916 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:07.916 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:29:07.916 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:04.374341+0000", 2024-09-14T06:29:07.916 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000104128 2024-09-14T06:29:07.916 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:07.916 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:07.916 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "waiting for subops from 5", 2024-09-14T06:29:07.917 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:04.374436+0000", 2024-09-14T06:29:07.917 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.467e-05 2024-09-14T06:29:07.917 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:07.917 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:07.917 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "op_commit", 2024-09-14T06:29:07.917 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:04.375211+0000", 2024-09-14T06:29:07.917 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00077517299999999999 2024-09-14T06:29:07.917 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:07.917 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:07.917 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:29:07.917 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:04.375521+0000", 2024-09-14T06:29:07.917 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00031072899999999999 2024-09-14T06:29:07.917 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:07.917 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:07.918 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-14T06:29:07.918 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:04.375570+0000", 2024-09-14T06:29:07.918 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.8687999999999999e-05 2024-09-14T06:29:07.918 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:07.918 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:07.918 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:29:07.918 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:04.375612+0000", 2024-09-14T06:29:07.918 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.1715999999999998e-05 2024-09-14T06:29:07.918 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:07.918 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:07.918 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:07.918 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:07.918 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:07.919 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:117 3.2 3:427fbb68:::rbd_data.112d7adb82ff.0000000000000034:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-14T06:29:07.919 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:04.385299+0000", 2024-09-14T06:29:07.919 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 3.5244373840000001, 2024-09-14T06:29:07.919 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.001611599, 2024-09-14T06:29:07.919 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:29:07.919 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:29:07.919 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:29:07.919 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:29:07.919 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:29:07.919 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:07.919 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 117 2024-09-14T06:29:07.919 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:07.919 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:29:07.919 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:07.920 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:29:07.920 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:04.385299+0000", 2024-09-14T06:29:07.920 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:07.920 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:07.920 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:07.920 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:29:07.920 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:04.385299+0000", 2024-09-14T06:29:07.920 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:07.920 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:07.920 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:07.920 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:29:07.920 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:04.385300+0000", 2024-09-14T06:29:07.920 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.5260000000000001e-06 2024-09-14T06:29:07.920 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:07.921 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:07.921 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:29:07.921 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:04.385308+0000", 2024-09-14T06:29:07.921 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.8250000000000005e-06 2024-09-14T06:29:07.921 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:07.921 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:07.921 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:29:07.921 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:04.385323+0000", 2024-09-14T06:29:07.921 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.5328e-05 2024-09-14T06:29:07.921 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:07.921 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:07.921 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:29:07.921 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:04.385331+0000", 2024-09-14T06:29:07.921 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.2810000000000003e-06 2024-09-14T06:29:07.922 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:07.922 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:07.922 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:29:07.922 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:04.385393+0000", 2024-09-14T06:29:07.922 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.2524e-05 2024-09-14T06:29:07.922 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:07.922 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:07.922 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:29:07.922 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:04.385478+0000", 2024-09-14T06:29:07.922 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.4544999999999998e-05 2024-09-14T06:29:07.922 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:07.922 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:07.922 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "waiting for subops from 5", 2024-09-14T06:29:07.922 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:04.385594+0000", 2024-09-14T06:29:07.923 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000115911 2024-09-14T06:29:07.923 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:07.923 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:07.923 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "op_commit", 2024-09-14T06:29:07.923 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:04.386097+0000", 2024-09-14T06:29:07.923 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00050349900000000003 2024-09-14T06:29:07.923 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:07.923 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:07.923 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:29:07.923 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:04.386812+0000", 2024-09-14T06:29:07.923 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00071502499999999999 2024-09-14T06:29:07.923 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:07.924 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:07.924 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-14T06:29:07.924 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:04.386867+0000", 2024-09-14T06:29:07.924 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.4657000000000002e-05 2024-09-14T06:29:07.924 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:07.924 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:07.924 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:29:07.924 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:04.386910+0000", 2024-09-14T06:29:07.924 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.3477999999999999e-05 2024-09-14T06:29:07.924 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:07.924 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:07.924 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:07.924 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:07.925 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:07.925 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:140 3.1e 3:7b8dac6a:::rbd_data.112d7adb82ff.000000000000001f:head [set-alloc-hint object_size 4194304 write_size 4194304,write 1628674~41268] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-14T06:29:07.925 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:04.412485+0000", 2024-09-14T06:29:07.925 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 3.4972513410000001, 2024-09-14T06:29:07.925 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0020336149999999999, 2024-09-14T06:29:07.925 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:29:07.925 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:29:07.925 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:29:07.925 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:29:07.925 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:29:07.925 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:07.925 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 140 2024-09-14T06:29:07.925 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:07.925 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:29:07.926 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:07.926 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:29:07.926 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:04.412485+0000", 2024-09-14T06:29:07.926 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:07.926 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:07.926 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:07.926 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:29:07.926 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:04.412485+0000", 2024-09-14T06:29:07.926 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:07.926 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:07.926 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:07.926 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:29:07.927 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:04.412487+0000", 2024-09-14T06:29:07.927 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.5689999999999999e-06 2024-09-14T06:29:07.927 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:07.927 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:07.927 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:29:07.927 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:04.412519+0000", 2024-09-14T06:29:07.927 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.1510999999999997e-05 2024-09-14T06:29:07.927 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:07.927 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:07.927 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:29:07.927 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:04.412534+0000", 2024-09-14T06:29:07.927 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.5690000000000001e-05 2024-09-14T06:29:07.927 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:07.928 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:07.928 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:29:07.928 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:04.412543+0000", 2024-09-14T06:29:07.928 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.1960000000000008e-06 2024-09-14T06:29:07.928 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:07.928 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:07.928 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:29:07.928 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:04.412594+0000", 2024-09-14T06:29:07.928 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.1492000000000001e-05 2024-09-14T06:29:07.928 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:07.928 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:07.928 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:29:07.928 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:04.412702+0000", 2024-09-14T06:29:07.928 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000108194 2024-09-14T06:29:07.929 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:07.929 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:07.929 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "waiting for subops from 6", 2024-09-14T06:29:07.929 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:04.412782+0000", 2024-09-14T06:29:07.929 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.9769000000000001e-05 2024-09-14T06:29:07.929 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:07.929 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:07.929 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "op_commit", 2024-09-14T06:29:07.929 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:04.413544+0000", 2024-09-14T06:29:07.929 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00076192899999999995 2024-09-14T06:29:07.929 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:07.929 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:07.929 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:29:07.929 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:04.414442+0000", 2024-09-14T06:29:07.930 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000897578 2024-09-14T06:29:07.930 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:07.930 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:07.930 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-14T06:29:07.930 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:04.414474+0000", 2024-09-14T06:29:07.930 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.2393e-05 2024-09-14T06:29:07.930 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:07.930 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:07.930 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:29:07.930 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:04.414518+0000", 2024-09-14T06:29:07.930 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.4294e-05 2024-09-14T06:29:07.930 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:07.930 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:07.930 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:07.931 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:07.931 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:07.931 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:151 3.2 3:42fc824e:::rbd_data.112d7adb82ff.0000000000000027:head [set-alloc-hint object_size 4194304 write_size 4194304,write 2751234~30993] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-14T06:29:07.931 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:04.424873+0000", 2024-09-14T06:29:07.931 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 3.484862787, 2024-09-14T06:29:07.931 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0014145360000000001, 2024-09-14T06:29:07.931 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:29:07.931 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:29:07.931 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:29:07.931 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:29:07.931 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:29:07.931 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:07.931 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 151 2024-09-14T06:29:07.931 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:07.932 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:29:07.932 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:07.932 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:29:07.932 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:04.424873+0000", 2024-09-14T06:29:07.932 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:07.932 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:07.932 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:07.932 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:29:07.932 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:04.424873+0000", 2024-09-14T06:29:07.932 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:07.932 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:07.932 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:07.932 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:29:07.933 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:04.424875+0000", 2024-09-14T06:29:07.933 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.79e-06 2024-09-14T06:29:07.933 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:07.933 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:07.933 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:29:07.933 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:04.424899+0000", 2024-09-14T06:29:07.933 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.3757999999999999e-05 2024-09-14T06:29:07.933 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:07.933 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:07.933 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:29:07.933 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:04.424913+0000", 2024-09-14T06:29:07.933 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.3896e-05 2024-09-14T06:29:07.933 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:07.933 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:07.933 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:29:07.934 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:04.424920+0000", 2024-09-14T06:29:07.934 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.5809999999999999e-06 2024-09-14T06:29:07.934 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:07.934 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:07.934 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:29:07.934 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:04.424976+0000", 2024-09-14T06:29:07.934 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.605e-05 2024-09-14T06:29:07.934 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:07.934 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:07.934 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:29:07.934 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:04.425085+0000", 2024-09-14T06:29:07.934 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000108686 2024-09-14T06:29:07.934 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:07.935 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:07.935 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "waiting for subops from 5", 2024-09-14T06:29:07.935 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:04.425170+0000", 2024-09-14T06:29:07.935 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.5212999999999999e-05 2024-09-14T06:29:07.935 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:07.935 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:07.935 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "op_commit", 2024-09-14T06:29:07.935 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:04.425856+0000", 2024-09-14T06:29:07.935 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00068545699999999999 2024-09-14T06:29:07.935 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:07.935 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:07.935 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:29:07.935 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:04.426229+0000", 2024-09-14T06:29:07.935 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00037311499999999998 2024-09-14T06:29:07.936 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:07.936 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:07.936 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-14T06:29:07.936 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:04.426261+0000", 2024-09-14T06:29:07.936 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.2421e-05 2024-09-14T06:29:07.936 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:07.936 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:07.936 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:29:07.936 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:04.426288+0000", 2024-09-14T06:29:07.936 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.6568999999999999e-05 2024-09-14T06:29:07.936 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:07.936 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:07.936 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:07.936 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:07.937 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:07.937 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:191 3.18 3:1bded3e9:::rbd_data.112d7adb82ff.0000000000000029:head [set-alloc-hint object_size 4194304 write_size 4194304,write 534300~11325] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-14T06:29:07.937 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:04.465498+0000", 2024-09-14T06:29:07.937 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 3.4442377620000002, 2024-09-14T06:29:07.937 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0014630769999999999, 2024-09-14T06:29:07.937 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:29:07.937 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:29:07.937 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:29:07.937 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:29:07.937 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:29:07.937 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:07.937 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 191 2024-09-14T06:29:07.937 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:07.937 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:29:07.938 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:07.938 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:29:07.938 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:04.465498+0000", 2024-09-14T06:29:07.938 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:07.938 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:07.938 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:07.938 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:29:07.938 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:04.465498+0000", 2024-09-14T06:29:07.938 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:07.938 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:07.938 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:07.938 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:29:07.938 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:04.465500+0000", 2024-09-14T06:29:07.938 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.1600000000000001e-06 2024-09-14T06:29:07.939 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:07.939 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:07.939 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:29:07.939 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:04.465528+0000", 2024-09-14T06:29:07.939 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.7886999999999999e-05 2024-09-14T06:29:07.939 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:07.939 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:07.939 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:29:07.939 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:04.465550+0000", 2024-09-14T06:29:07.939 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.1678000000000001e-05 2024-09-14T06:29:07.939 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:07.939 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:07.939 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:29:07.939 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:04.465566+0000", 2024-09-14T06:29:07.940 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.5977999999999998e-05 2024-09-14T06:29:07.940 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:07.940 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:07.940 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:29:07.940 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:04.465626+0000", 2024-09-14T06:29:07.940 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.0232999999999997e-05 2024-09-14T06:29:07.940 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:07.940 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:07.940 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:29:07.940 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:04.465727+0000", 2024-09-14T06:29:07.940 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000100558 2024-09-14T06:29:07.940 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:07.941 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:07.941 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "waiting for subops from 0", 2024-09-14T06:29:07.941 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:04.465811+0000", 2024-09-14T06:29:07.941 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.4617000000000007e-05 2024-09-14T06:29:07.941 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:07.941 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:07.941 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "op_commit", 2024-09-14T06:29:07.941 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:04.466531+0000", 2024-09-14T06:29:07.941 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00071950099999999999 2024-09-14T06:29:07.941 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:07.941 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:07.941 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:29:07.941 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:04.466903+0000", 2024-09-14T06:29:07.941 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00037260700000000001 2024-09-14T06:29:07.942 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:07.942 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:07.942 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-14T06:29:07.942 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:04.466935+0000", 2024-09-14T06:29:07.942 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.1265e-05 2024-09-14T06:29:07.942 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:07.942 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:07.942 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:29:07.942 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:04.466961+0000", 2024-09-14T06:29:07.942 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.6593000000000001e-05 2024-09-14T06:29:07.942 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:07.942 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:07.942 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:07.942 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:07.943 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:07.943 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:193 3.2 3:432dba90:::rbd_data.112d7adb82ff.0000000000000021:head [set-alloc-hint object_size 4194304 write_size 4194304,write 3908527~51190] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-14T06:29:07.943 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:04.468836+0000", 2024-09-14T06:29:07.943 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 3.440900466, 2024-09-14T06:29:07.943 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0016524350000000001, 2024-09-14T06:29:07.943 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:29:07.943 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:29:07.943 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:29:07.943 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:29:07.943 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:29:07.943 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:07.943 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 193 2024-09-14T06:29:07.943 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:07.943 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:29:07.944 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:07.944 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:29:07.944 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:04.468836+0000", 2024-09-14T06:29:07.944 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:07.944 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:07.944 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:07.944 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:29:07.944 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:04.468836+0000", 2024-09-14T06:29:07.944 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:07.944 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:07.944 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:07.944 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:29:07.944 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:04.468836+0000", 2024-09-14T06:29:07.945 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.7300000000000005e-07 2024-09-14T06:29:07.945 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:07.945 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:07.945 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:29:07.945 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:04.468863+0000", 2024-09-14T06:29:07.945 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.6274e-05 2024-09-14T06:29:07.945 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:07.945 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:07.945 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:29:07.945 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:04.468873+0000", 2024-09-14T06:29:07.945 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.0589e-05 2024-09-14T06:29:07.945 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:07.945 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:07.945 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:29:07.946 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:04.468879+0000", 2024-09-14T06:29:07.946 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.246e-06 2024-09-14T06:29:07.946 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:07.946 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:07.946 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:29:07.946 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:04.468929+0000", 2024-09-14T06:29:07.946 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.9554e-05 2024-09-14T06:29:07.946 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:07.946 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:07.946 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:29:07.946 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:04.469009+0000", 2024-09-14T06:29:07.946 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.0000999999999995e-05 2024-09-14T06:29:07.946 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:07.946 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:07.947 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "waiting for subops from 5", 2024-09-14T06:29:07.947 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:04.469102+0000", 2024-09-14T06:29:07.947 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.2967999999999995e-05 2024-09-14T06:29:07.947 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:07.947 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:07.947 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "op_commit", 2024-09-14T06:29:07.947 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:04.469786+0000", 2024-09-14T06:29:07.947 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00068363899999999999 2024-09-14T06:29:07.947 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:07.947 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:07.947 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:29:07.947 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:04.470425+0000", 2024-09-14T06:29:07.947 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00063908499999999996 2024-09-14T06:29:07.948 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:07.948 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:07.948 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-14T06:29:07.948 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:04.470454+0000", 2024-09-14T06:29:07.948 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.9459999999999999e-05 2024-09-14T06:29:07.948 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:07.948 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:07.948 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:29:07.948 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:04.470488+0000", 2024-09-14T06:29:07.948 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.3846000000000001e-05 2024-09-14T06:29:07.948 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:07.948 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:07.948 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:07.948 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:07.949 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:07.949 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:627 3.7 3:e2d97966:::rbd_data.112d7dfbd766.0000000000000109:head [delete] snapc 4=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-14T06:29:07.949 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:06.795047+0000", 2024-09-14T06:29:07.949 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 1.1146888210000001, 2024-09-14T06:29:07.949 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.001637112, 2024-09-14T06:29:07.949 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:29:07.949 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:29:07.949 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:29:07.949 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:29:07.949 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:29:07.949 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:07.949 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 627 2024-09-14T06:29:07.949 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:07.950 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:29:07.950 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:07.950 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:29:07.950 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:06.795047+0000", 2024-09-14T06:29:07.950 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:07.950 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:07.950 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:07.950 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:29:07.950 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:06.795047+0000", 2024-09-14T06:29:07.950 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:07.950 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:07.950 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:07.950 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:29:07.950 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:06.795049+0000", 2024-09-14T06:29:07.950 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.4249999999999999e-06 2024-09-14T06:29:07.951 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:07.951 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:07.951 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:29:07.951 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:06.795059+0000", 2024-09-14T06:29:07.951 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.0681e-05 2024-09-14T06:29:07.951 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:07.951 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:07.951 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:29:07.951 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:06.795076+0000", 2024-09-14T06:29:07.951 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.6307000000000002e-05 2024-09-14T06:29:07.951 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:07.951 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:07.951 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:29:07.951 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:06.795083+0000", 2024-09-14T06:29:07.952 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.1420000000000004e-06 2024-09-14T06:29:07.952 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:07.952 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:07.952 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:29:07.952 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:06.795141+0000", 2024-09-14T06:29:07.952 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.7796e-05 2024-09-14T06:29:07.952 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:07.952 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:07.952 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:29:07.952 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:06.795241+0000", 2024-09-14T06:29:07.952 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000100342 2024-09-14T06:29:07.952 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:07.952 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:07.953 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:29:07.953 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:06.796684+0000", 2024-09-14T06:29:07.953 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.001443419 2024-09-14T06:29:07.953 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:07.953 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:07.953 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:07.953 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:07.953 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:07.953 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:638 3.1e 3:7bdefa74:::rbd_data.112d7dfbd766.00000000000001b1:head [delete] snapc 4=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-14T06:29:07.953 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:06.799217+0000", 2024-09-14T06:29:07.953 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 1.110519187, 2024-09-14T06:29:07.953 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0022975029999999998, 2024-09-14T06:29:07.953 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:29:07.953 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:29:07.954 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:29:07.954 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:29:07.954 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:29:07.954 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:07.954 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 638 2024-09-14T06:29:07.954 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:07.954 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:29:07.954 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:07.954 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:29:07.954 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:06.799217+0000", 2024-09-14T06:29:07.954 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:07.954 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:07.954 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:07.954 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:29:07.955 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:06.799217+0000", 2024-09-14T06:29:07.955 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:07.955 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:07.955 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:07.955 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:29:07.955 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:06.799218+0000", 2024-09-14T06:29:07.955 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.4789999999999999e-06 2024-09-14T06:29:07.955 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:07.955 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:07.955 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:29:07.955 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:06.799236+0000", 2024-09-14T06:29:07.955 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.7796999999999999e-05 2024-09-14T06:29:07.956 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:07.956 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:07.956 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:29:07.956 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:06.799251+0000", 2024-09-14T06:29:07.956 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.472e-05 2024-09-14T06:29:07.956 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:07.956 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:07.956 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:29:07.956 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:06.799266+0000", 2024-09-14T06:29:07.956 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.5096000000000001e-05 2024-09-14T06:29:07.956 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:07.956 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:07.956 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:29:07.957 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:06.799328+0000", 2024-09-14T06:29:07.957 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.2550000000000003e-05 2024-09-14T06:29:07.957 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:07.957 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:07.957 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:29:07.957 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:06.799424+0000", 2024-09-14T06:29:07.957 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.5193999999999997e-05 2024-09-14T06:29:07.957 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:07.957 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:07.957 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "waiting for subops from 6", 2024-09-14T06:29:07.957 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:06.799526+0000", 2024-09-14T06:29:07.957 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00010205900000000001 2024-09-14T06:29:07.957 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:07.957 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:07.958 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "op_commit", 2024-09-14T06:29:07.958 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:06.800545+0000", 2024-09-14T06:29:07.958 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.001019355 2024-09-14T06:29:07.958 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:07.958 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:07.958 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:29:07.958 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:06.801415+0000", 2024-09-14T06:29:07.958 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00086940599999999998 2024-09-14T06:29:07.958 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:07.958 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:07.958 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-14T06:29:07.958 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:06.801465+0000", 2024-09-14T06:29:07.958 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.0866999999999999e-05 2024-09-14T06:29:07.959 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:07.959 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:07.959 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:29:07.959 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:06.801514+0000", 2024-09-14T06:29:07.959 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.8980000000000002e-05 2024-09-14T06:29:07.959 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:07.959 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:07.959 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:07.959 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:07.959 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:07.959 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:684 3.b 3:d0ae437e:::rbd_data.112d7dfbd766.0000000000000068:head [delete] snapc 4=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-14T06:29:07.959 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:06.873312+0000", 2024-09-14T06:29:07.959 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 1.0364242189999999, 2024-09-14T06:29:07.960 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.001595431, 2024-09-14T06:29:07.960 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:29:07.960 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:29:07.960 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:29:07.960 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:29:07.960 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:29:07.960 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:07.960 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 684 2024-09-14T06:29:07.960 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:07.960 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:29:07.960 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:07.960 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:29:07.960 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:06.873312+0000", 2024-09-14T06:29:07.961 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:07.961 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:07.961 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:07.961 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:29:07.961 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:06.873312+0000", 2024-09-14T06:29:07.961 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:07.961 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:07.961 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:07.961 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:29:07.961 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:06.873314+0000", 2024-09-14T06:29:07.961 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.5339999999999998e-06 2024-09-14T06:29:07.961 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:07.961 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:07.962 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:29:07.962 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:06.873325+0000", 2024-09-14T06:29:07.962 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.0202e-05 2024-09-14T06:29:07.962 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:07.962 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:07.962 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:29:07.962 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:06.873342+0000", 2024-09-14T06:29:07.962 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.7408999999999999e-05 2024-09-14T06:29:07.962 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:07.962 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:07.962 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:29:07.962 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:06.873354+0000", 2024-09-14T06:29:07.962 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.2072000000000001e-05 2024-09-14T06:29:07.962 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:07.963 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:07.963 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:29:07.963 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:06.873422+0000", 2024-09-14T06:29:07.963 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.8250999999999994e-05 2024-09-14T06:29:07.963 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:07.963 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:07.963 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:29:07.963 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:06.873530+0000", 2024-09-14T06:29:07.963 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000107666 2024-09-14T06:29:07.963 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:07.963 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:07.963 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "waiting for subops from 0", 2024-09-14T06:29:07.963 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:06.873671+0000", 2024-09-14T06:29:07.963 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00014134799999999999 2024-09-14T06:29:07.964 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:07.964 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:07.964 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "op_commit", 2024-09-14T06:29:07.964 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:06.874512+0000", 2024-09-14T06:29:07.964 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000840583 2024-09-14T06:29:07.964 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:07.964 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:07.964 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:29:07.964 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:06.874835+0000", 2024-09-14T06:29:07.964 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00032299199999999997 2024-09-14T06:29:07.964 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:07.964 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:07.964 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-14T06:29:07.965 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:06.874864+0000", 2024-09-14T06:29:07.965 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.8753000000000001e-05 2024-09-14T06:29:07.965 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:07.965 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:07.965 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:29:07.965 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:06.874907+0000", 2024-09-14T06:29:07.965 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.3621000000000001e-05 2024-09-14T06:29:07.965 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:07.965 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:07.965 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:07.965 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:07.965 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:07.965 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:796 3.7 3:e1d260bf:::rbd_data.112d7dfbd766.0000000000000098:head [set-alloc-hint object_size 524288 write_size 524288,write 262144~42967] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-14T06:29:07.966 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:07.002608+0000", 2024-09-14T06:29:07.966 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 0.90712767800000005, 2024-09-14T06:29:07.966 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0049985309999999996, 2024-09-14T06:29:07.966 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:29:07.966 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:29:07.966 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:29:07.966 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:29:07.966 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:29:07.966 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:07.966 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 796 2024-09-14T06:29:07.966 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:07.966 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:29:07.966 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:07.966 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:29:07.967 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:07.002608+0000", 2024-09-14T06:29:07.967 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:07.967 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:07.967 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:07.967 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:29:07.967 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:07.002608+0000", 2024-09-14T06:29:07.967 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:07.967 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:07.967 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:07.967 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:29:07.967 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:07.002610+0000", 2024-09-14T06:29:07.967 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.6980000000000001e-06 2024-09-14T06:29:07.967 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:07.967 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:07.967 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:29:07.968 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:07.002628+0000", 2024-09-14T06:29:07.968 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.7510999999999999e-05 2024-09-14T06:29:07.968 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:07.968 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:07.968 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:29:07.968 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:07.002643+0000", 2024-09-14T06:29:07.968 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.5517000000000001e-05 2024-09-14T06:29:07.968 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:07.968 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:07.968 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:29:07.968 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:07.002650+0000", 2024-09-14T06:29:07.968 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.0650000000000001e-06 2024-09-14T06:29:07.968 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:07.969 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:07.969 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:29:07.969 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:07.002704+0000", 2024-09-14T06:29:07.969 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.3579000000000003e-05 2024-09-14T06:29:07.969 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:07.969 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:07.969 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:29:07.969 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:07.002790+0000", 2024-09-14T06:29:07.969 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.6453999999999996e-05 2024-09-14T06:29:07.969 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:07.969 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:07.969 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "waiting for subops from 7", 2024-09-14T06:29:07.969 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:07.002883+0000", 2024-09-14T06:29:07.970 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.3178999999999997e-05 2024-09-14T06:29:07.970 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:07.970 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:07.970 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "op_commit", 2024-09-14T06:29:07.970 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:07.003603+0000", 2024-09-14T06:29:07.970 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00071951700000000003 2024-09-14T06:29:07.970 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:07.970 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:07.970 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:29:07.970 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:07.007544+0000", 2024-09-14T06:29:07.970 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0039407929999999997 2024-09-14T06:29:07.970 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:07.970 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:07.971 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-14T06:29:07.971 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:07.007579+0000", 2024-09-14T06:29:07.971 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.4924e-05 2024-09-14T06:29:07.971 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:07.971 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:07.971 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:29:07.971 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:07.007607+0000", 2024-09-14T06:29:07.971 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.8294e-05 2024-09-14T06:29:07.971 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:07.971 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:07.971 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:07.971 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:07.971 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:07.971 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:812 3.7 3:e68f0b27:::rbd_data.112d7dfbd766.000000000000006d:head [set-alloc-hint object_size 524288 write_size 524288,write 458752~39859] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-14T06:29:07.972 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:07.029172+0000", 2024-09-14T06:29:07.972 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 0.88056398400000002, 2024-09-14T06:29:07.972 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.002999041, 2024-09-14T06:29:07.972 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:29:07.972 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:29:07.972 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:29:07.972 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:29:07.972 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:29:07.972 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:07.972 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 812 2024-09-14T06:29:07.972 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:07.972 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:29:07.972 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:07.972 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:29:07.973 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:07.029172+0000", 2024-09-14T06:29:07.973 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:07.973 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:07.973 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:07.973 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:29:07.973 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:07.029172+0000", 2024-09-14T06:29:07.973 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:07.973 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:07.973 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:07.973 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:29:07.973 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:07.029173+0000", 2024-09-14T06:29:07.973 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.9100000000000003e-07 2024-09-14T06:29:07.973 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:07.974 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:07.974 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:29:07.974 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:07.029189+0000", 2024-09-14T06:29:07.974 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.6603999999999999e-05 2024-09-14T06:29:07.974 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:07.974 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:07.974 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:29:07.974 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:07.029201+0000", 2024-09-14T06:29:07.974 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.1096e-05 2024-09-14T06:29:07.974 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:07.974 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:07.974 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:29:07.974 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:07.029206+0000", 2024-09-14T06:29:07.975 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.0340000000000004e-06 2024-09-14T06:29:07.975 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:07.975 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:07.975 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:29:07.975 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:07.029249+0000", 2024-09-14T06:29:07.975 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.3723999999999997e-05 2024-09-14T06:29:07.975 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:07.975 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:07.975 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:29:07.975 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:07.029325+0000", 2024-09-14T06:29:07.975 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.6070999999999997e-05 2024-09-14T06:29:07.975 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:07.975 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:07.975 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "waiting for subops from 7", 2024-09-14T06:29:07.976 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:07.029406+0000", 2024-09-14T06:29:07.976 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.1036000000000001e-05 2024-09-14T06:29:07.976 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:07.976 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:07.976 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "op_commit", 2024-09-14T06:29:07.976 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:07.030112+0000", 2024-09-14T06:29:07.976 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00070564 2024-09-14T06:29:07.976 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:07.976 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:07.976 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:29:07.976 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:07.032116+0000", 2024-09-14T06:29:07.976 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0020035589999999998 2024-09-14T06:29:07.976 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:07.976 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:07.977 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-14T06:29:07.977 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:07.032146+0000", 2024-09-14T06:29:07.977 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.0895999999999997e-05 2024-09-14T06:29:07.977 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:07.977 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:07.977 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:29:07.977 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:07.032171+0000", 2024-09-14T06:29:07.977 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.459e-05 2024-09-14T06:29:07.977 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:07.977 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:07.977 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:07.977 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:07.977 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:07.978 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:815 3.1e 3:7e93f151:::rbd_data.112d7dfbd766.00000000000000c4:head [set-alloc-hint object_size 524288 write_size 524288,write 311876~15804] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-14T06:29:07.978 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:07.034373+0000", 2024-09-14T06:29:07.978 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 0.87536268900000003, 2024-09-14T06:29:07.978 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.002445001, 2024-09-14T06:29:07.978 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:29:07.978 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:29:07.978 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:29:07.978 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:29:07.978 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:29:07.978 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:07.978 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 815 2024-09-14T06:29:07.978 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:07.978 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:29:07.978 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:07.979 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:29:07.979 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:07.034373+0000", 2024-09-14T06:29:07.979 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:07.979 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:07.979 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:07.979 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:29:07.979 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:07.034373+0000", 2024-09-14T06:29:07.979 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:07.979 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:07.979 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:07.979 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:29:07.979 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:07.034374+0000", 2024-09-14T06:29:07.979 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.9999999999999996e-07 2024-09-14T06:29:07.979 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:07.980 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:07.980 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:29:07.980 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:07.034393+0000", 2024-09-14T06:29:07.980 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.925e-05 2024-09-14T06:29:07.980 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:07.980 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:07.980 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:29:07.980 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:07.034404+0000", 2024-09-14T06:29:07.980 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.1079e-05 2024-09-14T06:29:07.980 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:07.980 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:07.980 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:29:07.980 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:07.034410+0000", 2024-09-14T06:29:07.980 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.1100000000000002e-06 2024-09-14T06:29:07.981 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:07.981 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:07.981 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:29:07.981 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:07.034450+0000", 2024-09-14T06:29:07.981 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.0701000000000002e-05 2024-09-14T06:29:07.981 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:07.981 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:07.981 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:29:07.981 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:07.034533+0000", 2024-09-14T06:29:07.981 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.2397000000000004e-05 2024-09-14T06:29:07.981 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:07.981 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:07.981 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "waiting for subops from 6", 2024-09-14T06:29:07.981 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:07.034614+0000", 2024-09-14T06:29:07.982 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.1549000000000001e-05 2024-09-14T06:29:07.982 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:07.982 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:07.982 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "op_commit", 2024-09-14T06:29:07.982 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:07.035202+0000", 2024-09-14T06:29:07.982 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00058752100000000003 2024-09-14T06:29:07.982 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:07.982 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:07.982 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:29:07.982 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:07.036761+0000", 2024-09-14T06:29:07.982 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.001559665 2024-09-14T06:29:07.982 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:07.982 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:07.983 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-14T06:29:07.983 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:07.036791+0000", 2024-09-14T06:29:07.983 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.9643000000000001e-05 2024-09-14T06:29:07.983 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:07.983 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:07.983 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:29:07.983 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:07.036818+0000", 2024-09-14T06:29:07.983 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.7285999999999999e-05 2024-09-14T06:29:07.983 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:07.983 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:07.983 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:07.983 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:07.983 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:07.984 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:852 3.2 3:407f1726:::rbd_data.112d7dfbd766.0000000000000021:head [set-alloc-hint object_size 524288 write_size 524288,write 0~20276] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-14T06:29:07.984 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:07.076813+0000", 2024-09-14T06:29:07.984 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 0.83292298099999995, 2024-09-14T06:29:07.984 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0014094559999999999, 2024-09-14T06:29:07.984 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:29:07.984 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:29:07.984 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:29:07.984 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:29:07.984 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:29:07.984 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:07.984 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 852 2024-09-14T06:29:07.984 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:07.984 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:29:07.984 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:07.985 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:29:07.985 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:07.076813+0000", 2024-09-14T06:29:07.985 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:07.985 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:07.985 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:07.985 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:29:07.985 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:07.076813+0000", 2024-09-14T06:29:07.985 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:07.985 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:07.985 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:07.985 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:29:07.985 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:07.076815+0000", 2024-09-14T06:29:07.985 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.8199999999999999e-06 2024-09-14T06:29:07.986 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:07.986 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:07.986 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:29:07.986 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:07.076829+0000", 2024-09-14T06:29:07.986 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.4151999999999999e-05 2024-09-14T06:29:07.986 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:07.986 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:07.986 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:29:07.986 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:07.076843+0000", 2024-09-14T06:29:07.986 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.4034e-05 2024-09-14T06:29:07.986 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:07.986 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:07.986 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:29:07.986 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:07.076852+0000", 2024-09-14T06:29:07.987 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.7839999999999992e-06 2024-09-14T06:29:07.987 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:07.987 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:07.987 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:29:07.987 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:07.076916+0000", 2024-09-14T06:29:07.987 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.4530999999999996e-05 2024-09-14T06:29:07.987 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:07.987 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:07.987 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:29:07.987 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:07.077054+0000", 2024-09-14T06:29:07.987 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00013812699999999999 2024-09-14T06:29:07.987 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:07.987 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:07.988 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "waiting for subops from 5", 2024-09-14T06:29:07.988 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:07.077210+0000", 2024-09-14T06:29:07.988 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000155871 2024-09-14T06:29:07.988 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:07.988 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:07.988 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "op_commit", 2024-09-14T06:29:07.988 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:07.077959+0000", 2024-09-14T06:29:07.988 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00074852100000000004 2024-09-14T06:29:07.988 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:07.988 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:07.988 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:29:07.988 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:07.078157+0000", 2024-09-14T06:29:07.988 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00019829599999999999 2024-09-14T06:29:07.988 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:07.989 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:07.989 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-14T06:29:07.989 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:07.078192+0000", 2024-09-14T06:29:07.989 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.4922000000000002e-05 2024-09-14T06:29:07.989 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:07.989 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:07.989 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:29:07.989 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:07.078223+0000", 2024-09-14T06:29:07.989 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.0397999999999999e-05 2024-09-14T06:29:07.989 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:07.989 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:07.989 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:07.989 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:07.990 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:07.990 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:868 3.2 3:47d81400:::rbd_data.112d7dfbd766.00000000000000ac:head [set-alloc-hint object_size 524288 write_size 524288,write 496155~28133] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-14T06:29:07.990 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:07.093795+0000", 2024-09-14T06:29:07.990 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 0.81594113099999999, 2024-09-14T06:29:07.990 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0026291999999999999, 2024-09-14T06:29:07.990 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:29:07.990 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:29:07.990 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:29:07.990 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:29:07.990 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:29:07.990 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:07.990 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 868 2024-09-14T06:29:07.990 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:07.990 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:29:07.991 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:07.991 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:29:07.991 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:07.093795+0000", 2024-09-14T06:29:07.991 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:07.991 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:07.991 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:07.991 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:29:07.991 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:07.093795+0000", 2024-09-14T06:29:07.991 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:07.991 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:07.991 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:07.991 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:29:07.991 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:07.093797+0000", 2024-09-14T06:29:07.991 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.4080000000000001e-06 2024-09-14T06:29:07.992 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:07.992 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:07.992 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:29:07.992 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:07.093846+0000", 2024-09-14T06:29:07.992 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.8226000000000002e-05 2024-09-14T06:29:07.992 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:07.992 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:07.992 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:29:07.992 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:07.093865+0000", 2024-09-14T06:29:07.992 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.912e-05 2024-09-14T06:29:07.992 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:07.992 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:07.992 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:29:07.992 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:07.093873+0000", 2024-09-14T06:29:07.993 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.4740000000000005e-06 2024-09-14T06:29:07.993 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:07.993 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:07.993 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:29:07.993 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:07.093938+0000", 2024-09-14T06:29:07.993 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.4387999999999994e-05 2024-09-14T06:29:07.993 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:07.993 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:07.993 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:29:07.993 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:07.094043+0000", 2024-09-14T06:29:07.993 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000105215 2024-09-14T06:29:07.993 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:07.993 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:07.994 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "waiting for subops from 5", 2024-09-14T06:29:07.994 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:07.094140+0000", 2024-09-14T06:29:07.994 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.7647999999999997e-05 2024-09-14T06:29:07.994 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:07.994 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:07.994 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "op_commit", 2024-09-14T06:29:07.994 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:07.094908+0000", 2024-09-14T06:29:07.994 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00076767700000000003 2024-09-14T06:29:07.994 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:07.994 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:07.994 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:29:07.994 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:07.096320+0000", 2024-09-14T06:29:07.994 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0014116599999999999 2024-09-14T06:29:07.994 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:07.995 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:07.995 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-14T06:29:07.995 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:07.096376+0000", 2024-09-14T06:29:07.995 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.6153000000000003e-05 2024-09-14T06:29:07.995 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:07.995 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:07.995 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:29:07.995 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:07.096424+0000", 2024-09-14T06:29:07.995 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.8231e-05 2024-09-14T06:29:07.995 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:07.995 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:07.995 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:07.995 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:07.996 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:07.996 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:877 3.7 3:e1d260bf:::rbd_data.112d7dfbd766.0000000000000098:head [delete] snapc 4=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-14T06:29:07.996 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:07.113943+0000", 2024-09-14T06:29:07.996 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 0.79579274200000005, 2024-09-14T06:29:07.996 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0017770119999999999, 2024-09-14T06:29:07.996 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:29:07.996 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:29:07.996 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:29:07.996 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:29:07.996 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:29:07.996 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:07.996 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 877 2024-09-14T06:29:07.997 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:07.997 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:29:07.997 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:07.997 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:29:07.997 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:07.113943+0000", 2024-09-14T06:29:07.997 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:07.997 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:07.997 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:07.997 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:29:07.997 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:07.113943+0000", 2024-09-14T06:29:07.997 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:07.997 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:07.998 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:07.998 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:29:07.998 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:07.113946+0000", 2024-09-14T06:29:07.998 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.6110000000000001e-06 2024-09-14T06:29:07.998 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:07.998 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:07.998 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:29:07.998 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:07.113954+0000", 2024-09-14T06:29:07.998 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.7789999999999996e-06 2024-09-14T06:29:07.998 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:07.998 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:07.998 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:29:07.998 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:07.113975+0000", 2024-09-14T06:29:07.998 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.1290000000000001e-05 2024-09-14T06:29:07.999 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:07.999 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:07.999 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:29:07.999 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:07.113985+0000", 2024-09-14T06:29:07.999 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.5780000000000005e-06 2024-09-14T06:29:07.999 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:07.999 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:07.999 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:29:07.999 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:07.114058+0000", 2024-09-14T06:29:07.999 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.3380000000000006e-05 2024-09-14T06:29:07.999 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:07.999 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:07.999 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:29:07.999 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:07.114151+0000", 2024-09-14T06:29:08.000 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.3064000000000002e-05 2024-09-14T06:29:08.000 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:08.000 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:08.000 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "waiting for subops from 7", 2024-09-14T06:29:08.000 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:07.114273+0000", 2024-09-14T06:29:08.000 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000122113 2024-09-14T06:29:08.000 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:08.000 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:08.000 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "op_commit", 2024-09-14T06:29:08.000 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:07.115517+0000", 2024-09-14T06:29:08.000 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.001243535 2024-09-14T06:29:08.000 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:08.000 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:08.000 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:29:08.001 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:07.115611+0000", 2024-09-14T06:29:08.001 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.4820000000000004e-05 2024-09-14T06:29:08.001 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:08.001 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:08.001 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-14T06:29:08.001 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:07.115669+0000", 2024-09-14T06:29:08.001 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.7340999999999999e-05 2024-09-14T06:29:08.001 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:08.001 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:08.001 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:29:08.001 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:07.115720+0000", 2024-09-14T06:29:08.001 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.1501e-05 2024-09-14T06:29:08.001 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:08.002 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:08.002 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:08.002 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:08.002 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:08.002 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:879 3.2 3:47d81400:::rbd_data.112d7dfbd766.00000000000000ac:head [delete] snapc 4=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-14T06:29:08.002 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:07.114013+0000", 2024-09-14T06:29:08.002 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 0.79572316499999995, 2024-09-14T06:29:08.002 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.001940518, 2024-09-14T06:29:08.002 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:29:08.002 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:29:08.002 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:29:08.002 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:29:08.002 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:29:08.002 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:08.003 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 879 2024-09-14T06:29:08.003 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:08.003 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:29:08.003 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:08.003 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:29:08.003 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:07.114013+0000", 2024-09-14T06:29:08.003 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:08.003 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:08.003 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:08.003 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:29:08.003 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:07.114013+0000", 2024-09-14T06:29:08.003 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:08.003 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:08.003 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:08.004 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:29:08.004 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:07.114013+0000", 2024-09-14T06:29:08.004 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.5799999999999999e-07 2024-09-14T06:29:08.004 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:08.004 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:08.004 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:29:08.004 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:07.114016+0000", 2024-09-14T06:29:08.004 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.7030000000000002e-06 2024-09-14T06:29:08.004 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:08.004 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:08.004 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:29:08.004 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:07.114026+0000", 2024-09-14T06:29:08.004 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.6549999999999999e-06 2024-09-14T06:29:08.004 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:08.005 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:08.005 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:29:08.005 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:07.114031+0000", 2024-09-14T06:29:08.005 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.3900000000000001e-06 2024-09-14T06:29:08.005 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:08.005 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:08.005 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:29:08.005 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:07.114102+0000", 2024-09-14T06:29:08.005 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.1095999999999998e-05 2024-09-14T06:29:08.005 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:08.005 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:08.005 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:29:08.005 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:07.114166+0000", 2024-09-14T06:29:08.005 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.3411999999999995e-05 2024-09-14T06:29:08.006 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:08.006 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:08.006 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "waiting for subops from 5", 2024-09-14T06:29:08.006 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:07.114273+0000", 2024-09-14T06:29:08.006 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000107273 2024-09-14T06:29:08.006 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:08.006 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:08.006 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:29:08.006 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:07.115638+0000", 2024-09-14T06:29:08.006 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0013652079999999999 2024-09-14T06:29:08.006 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:08.006 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:08.006 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "op_commit", 2024-09-14T06:29:08.006 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:07.115852+0000", 2024-09-14T06:29:08.007 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00021393399999999999 2024-09-14T06:29:08.007 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:08.007 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:08.007 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-14T06:29:08.007 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:07.115908+0000", 2024-09-14T06:29:08.007 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.6243999999999999e-05 2024-09-14T06:29:08.007 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:08.007 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:08.007 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:29:08.007 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:07.115953+0000", 2024-09-14T06:29:08.007 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.5045e-05 2024-09-14T06:29:08.007 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:08.007 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:08.007 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:08.008 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:08.008 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:08.008 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:882 3.1e 3:7e93f151:::rbd_data.112d7dfbd766.00000000000000c4:head [delete] snapc 4=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-14T06:29:08.008 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:07.114046+0000", 2024-09-14T06:29:08.008 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 0.79568989300000004, 2024-09-14T06:29:08.008 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0039198699999999998, 2024-09-14T06:29:08.008 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:29:08.008 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:29:08.008 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:29:08.008 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:29:08.008 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:29:08.008 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:08.008 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 882 2024-09-14T06:29:08.009 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:08.009 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:29:08.009 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:08.009 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:29:08.009 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:07.114046+0000", 2024-09-14T06:29:08.009 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:08.009 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:08.009 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:08.009 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:29:08.009 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:07.114046+0000", 2024-09-14T06:29:08.009 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:08.009 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:08.009 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:08.009 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:29:08.010 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:07.114047+0000", 2024-09-14T06:29:08.010 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.2399999999999999e-07 2024-09-14T06:29:08.010 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:08.010 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:08.010 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:29:08.010 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:07.114049+0000", 2024-09-14T06:29:08.010 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.216e-06 2024-09-14T06:29:08.010 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:08.010 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:08.010 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:29:08.010 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:07.114055+0000", 2024-09-14T06:29:08.010 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.2369999999999997e-06 2024-09-14T06:29:08.010 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:08.010 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:08.011 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:29:08.011 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:07.114059+0000", 2024-09-14T06:29:08.011 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.3460000000000004e-06 2024-09-14T06:29:08.011 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:08.011 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:08.011 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:29:08.011 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:07.114616+0000", 2024-09-14T06:29:08.011 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00055674299999999995 2024-09-14T06:29:08.011 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:08.011 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:08.011 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:29:08.011 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:07.114694+0000", 2024-09-14T06:29:08.011 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.7951e-05 2024-09-14T06:29:08.012 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:08.012 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:08.012 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "waiting for subops from 6", 2024-09-14T06:29:08.012 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:07.114797+0000", 2024-09-14T06:29:08.012 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000102549 2024-09-14T06:29:08.012 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:08.012 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:08.012 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "op_commit", 2024-09-14T06:29:08.012 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:07.115865+0000", 2024-09-14T06:29:08.012 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.001068315 2024-09-14T06:29:08.012 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:08.012 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:08.012 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:29:08.012 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:07.117915+0000", 2024-09-14T06:29:08.013 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0020502070000000001 2024-09-14T06:29:08.013 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:08.013 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:08.013 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-14T06:29:08.013 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:07.117943+0000", 2024-09-14T06:29:08.013 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.8061000000000001e-05 2024-09-14T06:29:08.013 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:08.013 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:08.013 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:29:08.013 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:07.117966+0000", 2024-09-14T06:29:08.013 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.2821000000000001e-05 2024-09-14T06:29:08.013 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:08.013 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:08.014 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:08.014 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:08.014 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:08.014 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:884 3.b 3:d72ce592:::rbd_data.112d7dfbd766.000000000000010e:head [delete] snapc 4=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-14T06:29:08.014 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:07.114580+0000", 2024-09-14T06:29:08.014 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 0.79515556099999996, 2024-09-14T06:29:08.014 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0013752790000000001, 2024-09-14T06:29:08.014 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:29:08.014 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:29:08.014 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:29:08.014 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:29:08.014 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:29:08.014 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:08.014 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 884 2024-09-14T06:29:08.015 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:08.015 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:29:08.015 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:08.015 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:29:08.015 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:07.114580+0000", 2024-09-14T06:29:08.015 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:08.015 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:08.015 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:08.015 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:29:08.015 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:07.114580+0000", 2024-09-14T06:29:08.015 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:08.015 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:08.015 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:08.015 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:29:08.015 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:07.114581+0000", 2024-09-14T06:29:08.016 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.7000000000000004e-07 2024-09-14T06:29:08.016 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:08.016 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:08.016 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:29:08.016 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:07.114585+0000", 2024-09-14T06:29:08.016 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.1590000000000003e-06 2024-09-14T06:29:08.016 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:08.016 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:08.016 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:29:08.016 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:07.114594+0000", 2024-09-14T06:29:08.016 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.4239999999999993e-06 2024-09-14T06:29:08.016 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:08.016 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:08.017 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:29:08.017 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:07.114600+0000", 2024-09-14T06:29:08.017 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.3559999999999998e-06 2024-09-14T06:29:08.017 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:08.017 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:08.017 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:29:08.017 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:07.114664+0000", 2024-09-14T06:29:08.017 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.4257999999999994e-05 2024-09-14T06:29:08.017 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:08.017 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:08.017 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:29:08.017 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:07.114745+0000", 2024-09-14T06:29:08.017 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.1037000000000004e-05 2024-09-14T06:29:08.017 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:08.018 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:08.018 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "waiting for subops from 0", 2024-09-14T06:29:08.018 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:07.114861+0000", 2024-09-14T06:29:08.018 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000116045 2024-09-14T06:29:08.018 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:08.018 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:08.018 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:29:08.018 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:07.115700+0000", 2024-09-14T06:29:08.018 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00083815799999999998 2024-09-14T06:29:08.018 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:08.018 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:08.018 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "op_commit", 2024-09-14T06:29:08.018 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:07.115881+0000", 2024-09-14T06:29:08.019 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00018127400000000001 2024-09-14T06:29:08.019 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:08.019 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:08.019 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-14T06:29:08.019 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:07.115911+0000", 2024-09-14T06:29:08.019 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.9743e-05 2024-09-14T06:29:08.019 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:08.019 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:08.019 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:29:08.019 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:07.115956+0000", 2024-09-14T06:29:08.019 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.5154999999999997e-05 2024-09-14T06:29:08.019 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:08.019 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:08.020 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:08.020 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:08.020 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:08.020 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:936 3.8 3:17df59d8:::rbd_data.112d7dfbd766.0000000000000011:head [set-alloc-hint object_size 524288 write_size 524288,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-14T06:29:08.020 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:07.164453+0000", 2024-09-14T06:29:08.020 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 0.74528340500000001, 2024-09-14T06:29:08.020 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.013369270000000001, 2024-09-14T06:29:08.020 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:29:08.020 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:29:08.020 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:29:08.020 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:29:08.020 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:29:08.020 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:08.020 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 936 2024-09-14T06:29:08.021 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:08.021 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:29:08.021 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:08.021 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:29:08.021 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:07.164453+0000", 2024-09-14T06:29:08.021 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:08.021 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:08.021 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:08.021 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:29:08.021 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:07.164453+0000", 2024-09-14T06:29:08.021 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:08.022 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:08.022 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:08.022 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:29:08.022 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:07.164454+0000", 2024-09-14T06:29:08.022 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.8190000000000001e-06 2024-09-14T06:29:08.022 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:08.022 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:08.022 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:29:08.022 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:07.164463+0000", 2024-09-14T06:29:08.022 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.9900000000000003e-06 2024-09-14T06:29:08.022 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:08.022 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:08.022 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:29:08.022 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:07.164479+0000", 2024-09-14T06:29:08.022 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.5654e-05 2024-09-14T06:29:08.023 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:08.023 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:08.023 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:29:08.023 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:07.164487+0000", 2024-09-14T06:29:08.023 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.435e-06 2024-09-14T06:29:08.023 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:08.023 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:08.023 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:29:08.023 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:07.164537+0000", 2024-09-14T06:29:08.023 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.0175e-05 2024-09-14T06:29:08.023 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:08.023 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:08.023 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:29:08.024 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:07.164626+0000", 2024-09-14T06:29:08.024 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.9272999999999997e-05 2024-09-14T06:29:08.024 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:08.024 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:08.024 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "waiting for subops from 1", 2024-09-14T06:29:08.024 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:07.167461+0000", 2024-09-14T06:29:08.024 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0028350369999999999 2024-09-14T06:29:08.024 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:08.024 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:08.024 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "op_commit", 2024-09-14T06:29:08.024 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:07.177674+0000", 2024-09-14T06:29:08.024 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.01021267 2024-09-14T06:29:08.024 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:08.025 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:08.025 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:29:08.025 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:07.177728+0000", 2024-09-14T06:29:08.025 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.3829000000000002e-05 2024-09-14T06:29:08.025 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:08.025 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:08.025 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-14T06:29:08.025 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:07.177766+0000", 2024-09-14T06:29:08.025 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.8130000000000003e-05 2024-09-14T06:29:08.025 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:08.025 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:08.025 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:29:08.025 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:07.177822+0000", 2024-09-14T06:29:08.026 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.6258000000000002e-05 2024-09-14T06:29:08.026 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:08.026 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:08.026 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:08.026 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:08.026 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:08.026 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop_reply(client.4397.0:936 3.8 e30/23 ondisk, result = 0)", 2024-09-14T06:29:08.026 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:07.172840+0000", 2024-09-14T06:29:08.026 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 0.73689622600000004, 2024-09-14T06:29:08.026 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0049886009999999996, 2024-09-14T06:29:08.026 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:29:08.026 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:29:08.026 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:29:08.026 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:29:08.026 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:08.027 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:29:08.027 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:07.172840+0000", 2024-09-14T06:29:08.027 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:08.027 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:08.027 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:08.027 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:29:08.027 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:07.172840+0000", 2024-09-14T06:29:08.027 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:08.027 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:08.027 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:08.027 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:29:08.027 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:07.172841+0000", 2024-09-14T06:29:08.027 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.78e-07 2024-09-14T06:29:08.027 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:08.028 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:08.028 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:29:08.028 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:07.172845+0000", 2024-09-14T06:29:08.028 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.3420000000000001e-06 2024-09-14T06:29:08.028 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:08.028 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:08.028 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:29:08.028 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:07.172860+0000", 2024-09-14T06:29:08.028 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.4771e-05 2024-09-14T06:29:08.028 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:08.028 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:08.028 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:29:08.028 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:07.172868+0000", 2024-09-14T06:29:08.029 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.7670000000000003e-06 2024-09-14T06:29:08.029 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:08.029 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:08.029 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:29:08.029 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:07.177717+0000", 2024-09-14T06:29:08.029 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0048489570000000001 2024-09-14T06:29:08.029 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:08.029 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:08.029 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:29:08.029 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:07.177724+0000", 2024-09-14T06:29:08.029 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.08e-06 2024-09-14T06:29:08.029 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:08.029 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:08.030 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:29:08.030 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:07.177828+0000", 2024-09-14T06:29:08.030 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000104706 2024-09-14T06:29:08.030 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:08.030 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:08.030 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:08.030 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:08.030 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:08.030 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-14T06:29:08.031 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-14T06:29:08.032 INFO:tasks.ceph.osd.7.smithi138.stderr:2024-09-14T06:29:08.030+0000 7f3c6a21d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33623) UID: 0 2024-09-14T06:29:08.132 INFO:tasks.ceph.osd.4.smithi138.stderr:2024-09-14T06:29:08.131+0000 7f6c653c8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33615) UID: 0 2024-09-14T06:29:08.147 INFO:teuthology.orchestra.run.smithi138.stdout:{ 2024-09-14T06:29:08.147 INFO:teuthology.orchestra.run.smithi138.stdout: "ops": [], 2024-09-14T06:29:08.147 INFO:teuthology.orchestra.run.smithi138.stdout: "num_ops": 0 2024-09-14T06:29:08.147 INFO:teuthology.orchestra.run.smithi138.stdout:} 2024-09-14T06:29:08.157 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-14T06:29:08.233 INFO:tasks.ceph.osd.3.smithi094.stderr:2024-09-14T06:29:08.232+0000 7feeeafe8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33655) UID: 0 2024-09-14T06:29:08.302 INFO:teuthology.orchestra.run.smithi138.stdout:{ 2024-09-14T06:29:08.302 INFO:teuthology.orchestra.run.smithi138.stdout: "ops": [], 2024-09-14T06:29:08.302 INFO:teuthology.orchestra.run.smithi138.stdout: "complaint_time": 30, 2024-09-14T06:29:08.302 INFO:teuthology.orchestra.run.smithi138.stdout: "num_blocked_ops": 0 2024-09-14T06:29:08.302 INFO:teuthology.orchestra.run.smithi138.stdout:} 2024-09-14T06:29:08.310 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-14T06:29:08.354 INFO:tasks.ceph.osd.4.smithi138.stderr:2024-09-14T06:29:08.353+0000 7f6c653c8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33615) UID: 0 2024-09-14T06:29:08.415 INFO:teuthology.orchestra.run.smithi138.stdout:{ 2024-09-14T06:29:08.415 INFO:teuthology.orchestra.run.smithi138.stdout: "size": 20, 2024-09-14T06:29:08.415 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 600, 2024-09-14T06:29:08.415 INFO:teuthology.orchestra.run.smithi138.stdout: "ops": [ 2024-09-14T06:29:08.415 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:08.415 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:9 3.1a 3:5cea7035:::rbd_info:head [write 0~19] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-14T06:29:08.415 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:02.577373+0000", 2024-09-14T06:29:08.415 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 5.835726492, 2024-09-14T06:29:08.416 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.003029739, 2024-09-14T06:29:08.416 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:29:08.416 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:29:08.416 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:29:08.416 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:29:08.416 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:29:08.416 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:08.416 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 9 2024-09-14T06:29:08.416 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:08.416 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:29:08.416 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:08.416 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:29:08.416 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:02.577373+0000", 2024-09-14T06:29:08.416 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:08.417 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:08.417 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:08.417 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:29:08.417 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:02.577373+0000", 2024-09-14T06:29:08.417 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:08.417 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:08.417 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:08.417 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:29:08.417 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:02.577374+0000", 2024-09-14T06:29:08.417 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.271e-06 2024-09-14T06:29:08.417 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:08.417 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:08.417 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:29:08.417 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:02.577380+0000", 2024-09-14T06:29:08.418 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.0750000000000003e-06 2024-09-14T06:29:08.418 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:08.418 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:08.418 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:29:08.418 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:02.577391+0000", 2024-09-14T06:29:08.418 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.1273e-05 2024-09-14T06:29:08.418 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:08.418 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:08.418 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:29:08.418 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:02.577399+0000", 2024-09-14T06:29:08.418 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.6860000000000006e-06 2024-09-14T06:29:08.418 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:08.418 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:08.419 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:29:08.419 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:02.577447+0000", 2024-09-14T06:29:08.419 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.8488000000000001e-05 2024-09-14T06:29:08.419 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:08.419 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:08.419 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "op must wait for map", 2024-09-14T06:29:08.419 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:02.577452+0000", 2024-09-14T06:29:08.419 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.4800000000000003e-06 2024-09-14T06:29:08.419 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:08.419 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:08.419 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:29:08.419 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:02.579120+0000", 2024-09-14T06:29:08.419 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.0016679259999999999 2024-09-14T06:29:08.419 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:08.420 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:08.420 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:29:08.420 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:02.579174+0000", 2024-09-14T06:29:08.420 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.4357999999999997e-05 2024-09-14T06:29:08.420 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:08.420 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:08.420 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "waiting for subops from 1", 2024-09-14T06:29:08.420 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:02.579257+0000", 2024-09-14T06:29:08.420 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.3010999999999995e-05 2024-09-14T06:29:08.420 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:08.420 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:08.420 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "op_commit", 2024-09-14T06:29:08.420 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:02.580137+0000", 2024-09-14T06:29:08.421 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00087947600000000002 2024-09-14T06:29:08.421 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:08.421 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:08.421 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:29:08.421 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:02.580344+0000", 2024-09-14T06:29:08.421 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00020764700000000001 2024-09-14T06:29:08.421 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:08.421 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:08.421 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "commit_sent", 2024-09-14T06:29:08.421 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:02.580376+0000", 2024-09-14T06:29:08.421 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.1522000000000001e-05 2024-09-14T06:29:08.421 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:08.421 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:08.422 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:29:08.422 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:02.580402+0000", 2024-09-14T06:29:08.422 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.6526e-05 2024-09-14T06:29:08.422 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:08.422 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:29:08.422 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:08.422 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:08.422 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:08.422 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_repop(client.4397.0:10 3.1c e26/23 3:3831950c:::rbd_directory:head v 26'1, mlcod=0'0)", 2024-09-14T06:29:08.422 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:02.583620+0000", 2024-09-14T06:29:08.422 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 5.8294791510000001, 2024-09-14T06:29:08.422 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.0025782019999999999, 2024-09-14T06:29:08.422 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:29:08.422 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:29:08.423 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:29:08.423 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:29:08.423 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:08.423 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:29:08.423 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:02.583620+0000", 2024-09-14T06:29:08.423 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:08.423 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:08.423 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:08.423 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:29:08.423 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:02.583620+0000", 2024-09-14T06:29:08.423 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:08.423 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:08.423 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:08.423 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:29:08.424 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:02.583620+0000", 2024-09-14T06:29:08.424 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.0900000000000001e-07 2024-09-14T06:29:08.424 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:08.424 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:08.424 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:29:08.424 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:02.583623+0000", 2024-09-14T06:29:08.424 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.8949999999999998e-06 2024-09-14T06:29:08.424 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:08.424 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:08.424 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:29:08.424 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:02.583632+0000", 2024-09-14T06:29:08.424 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.7460000000000006e-06 2024-09-14T06:29:08.424 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:08.425 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:08.425 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:29:08.425 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:02.583637+0000", 2024-09-14T06:29:08.425 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.3299999999999998e-06 2024-09-14T06:29:08.425 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:08.425 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:08.425 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:29:08.425 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:02.583679+0000", 2024-09-14T06:29:08.425 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.1832999999999997e-05 2024-09-14T06:29:08.425 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:08.425 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:08.426 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:29:08.426 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:02.583690+0000", 2024-09-14T06:29:08.426 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.1481000000000001e-05 2024-09-14T06:29:08.426 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:08.426 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:08.426 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "commit_sent", 2024-09-14T06:29:08.426 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:02.586157+0000", 2024-09-14T06:29:08.426 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.0024669620000000001 2024-09-14T06:29:08.426 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:08.426 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:08.426 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:29:08.426 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:02.586198+0000", 2024-09-14T06:29:08.426 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.0745999999999999e-05 2024-09-14T06:29:08.427 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:08.427 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:29:08.427 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:08.427 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:08.427 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:08.427 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_repop(client.4397.0:724 3.11 e30/23 3:88d41b02:::rbd_object_map.112d7dfbd766:head v 30'40, mlcod=30'38)", 2024-09-14T06:29:08.427 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:06.904478+0000", 2024-09-14T06:29:08.427 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 1.5086212269999999, 2024-09-14T06:29:08.427 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.0022753019999999999, 2024-09-14T06:29:08.427 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:29:08.427 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:29:08.427 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:29:08.427 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:29:08.428 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:08.428 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:29:08.428 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:06.904478+0000", 2024-09-14T06:29:08.428 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:08.428 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:08.428 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:08.428 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:29:08.428 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:06.904478+0000", 2024-09-14T06:29:08.428 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:08.428 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:08.428 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:08.428 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:29:08.428 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:06.904478+0000", 2024-09-14T06:29:08.428 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.0699999999999997e-07 2024-09-14T06:29:08.428 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:08.429 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:08.429 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:29:08.429 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:06.904484+0000", 2024-09-14T06:29:08.429 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.7760000000000003e-06 2024-09-14T06:29:08.429 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:08.429 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:08.429 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:29:08.429 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:06.904493+0000", 2024-09-14T06:29:08.429 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.7630000000000006e-06 2024-09-14T06:29:08.429 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:08.429 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:08.429 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:29:08.429 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:06.904498+0000", 2024-09-14T06:29:08.430 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.4970000000000003e-06 2024-09-14T06:29:08.430 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:08.430 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:08.430 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:29:08.430 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:06.904534+0000", 2024-09-14T06:29:08.430 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.6977999999999997e-05 2024-09-14T06:29:08.430 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:08.430 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:08.430 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:29:08.430 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:06.904546+0000", 2024-09-14T06:29:08.430 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.1586000000000001e-05 2024-09-14T06:29:08.430 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:08.430 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:08.430 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "commit_sent", 2024-09-14T06:29:08.431 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:06.906712+0000", 2024-09-14T06:29:08.431 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.0021659280000000001 2024-09-14T06:29:08.431 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:08.431 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:08.431 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:29:08.431 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:06.906753+0000", 2024-09-14T06:29:08.431 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.1167000000000001e-05 2024-09-14T06:29:08.431 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:08.431 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:29:08.431 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:08.431 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:08.431 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:08.431 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_repop(client.4397.0:795 3.11 e30/23 3:88d41b02:::rbd_object_map.112d7dfbd766:head v 30'52, mlcod=30'49)", 2024-09-14T06:29:08.431 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:07.002026+0000", 2024-09-14T06:29:08.432 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 1.411073273, 2024-09-14T06:29:08.432 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.0026048780000000001, 2024-09-14T06:29:08.432 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:29:08.432 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:29:08.432 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:29:08.432 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:29:08.432 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:08.432 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:29:08.432 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:07.002026+0000", 2024-09-14T06:29:08.432 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:08.432 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:08.432 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:08.432 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:29:08.432 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:07.002026+0000", 2024-09-14T06:29:08.433 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:08.433 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:08.433 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:08.433 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:29:08.433 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:07.002026+0000", 2024-09-14T06:29:08.433 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.7999999999999995e-07 2024-09-14T06:29:08.433 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:08.433 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:08.433 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:29:08.433 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:07.002032+0000", 2024-09-14T06:29:08.433 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.6790000000000002e-06 2024-09-14T06:29:08.433 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:08.433 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:08.433 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:29:08.434 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:07.002041+0000", 2024-09-14T06:29:08.434 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.9129999999999992e-06 2024-09-14T06:29:08.434 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:08.434 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:08.434 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:29:08.434 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:07.002046+0000", 2024-09-14T06:29:08.434 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.1560000000000003e-06 2024-09-14T06:29:08.434 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:08.434 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:08.434 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:29:08.434 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:07.002089+0000", 2024-09-14T06:29:08.434 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.2549000000000002e-05 2024-09-14T06:29:08.434 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:08.434 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:08.434 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:29:08.435 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:07.002107+0000", 2024-09-14T06:29:08.435 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.7821000000000001e-05 2024-09-14T06:29:08.435 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:08.435 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:08.435 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "commit_sent", 2024-09-14T06:29:08.435 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:07.004597+0000", 2024-09-14T06:29:08.435 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.0024908500000000002 2024-09-14T06:29:08.435 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:08.435 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:08.435 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:29:08.435 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:07.004631+0000", 2024-09-14T06:29:08.435 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.3330000000000001e-05 2024-09-14T06:29:08.435 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:08.435 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:29:08.436 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:08.436 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:08.436 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:08.436 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_repop(client.4397.0:802 3.11 e30/23 3:88d41b02:::rbd_object_map.112d7dfbd766:head v 30'55, mlcod=30'52)", 2024-09-14T06:29:08.436 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:07.012350+0000", 2024-09-14T06:29:08.436 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 1.4007493100000001, 2024-09-14T06:29:08.436 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.0031356790000000002, 2024-09-14T06:29:08.436 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:29:08.436 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:29:08.436 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:29:08.436 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:29:08.436 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:08.436 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:29:08.436 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:07.012350+0000", 2024-09-14T06:29:08.436 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:08.437 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:08.437 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:08.437 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:29:08.437 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:07.012350+0000", 2024-09-14T06:29:08.437 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:08.437 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:08.437 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:08.437 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:29:08.437 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:07.012350+0000", 2024-09-14T06:29:08.437 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.4000000000000001e-07 2024-09-14T06:29:08.437 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:08.437 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:08.437 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:29:08.437 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:07.012356+0000", 2024-09-14T06:29:08.438 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.558e-06 2024-09-14T06:29:08.438 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:08.438 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:08.438 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:29:08.438 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:07.012367+0000", 2024-09-14T06:29:08.438 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.118e-05 2024-09-14T06:29:08.438 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:08.438 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:08.438 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:29:08.438 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:07.012373+0000", 2024-09-14T06:29:08.438 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.1290000000000001e-06 2024-09-14T06:29:08.438 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:08.438 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:08.438 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:29:08.439 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:07.012442+0000", 2024-09-14T06:29:08.439 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.8709999999999998e-05 2024-09-14T06:29:08.439 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:08.439 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:08.439 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:29:08.439 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:07.012455+0000", 2024-09-14T06:29:08.439 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.3362e-05 2024-09-14T06:29:08.439 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:08.439 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:08.440 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "commit_sent", 2024-09-14T06:29:08.440 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:07.015451+0000", 2024-09-14T06:29:08.440 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.002996117 2024-09-14T06:29:08.440 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:08.440 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:08.440 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:29:08.440 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:07.015485+0000", 2024-09-14T06:29:08.440 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.3983000000000003e-05 2024-09-14T06:29:08.440 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:08.440 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:29:08.440 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:08.440 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:08.440 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:08.441 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:803 3.9 3:9762c2a9:::rbd_data.112d7dfbd766.0000000000000054:head [set-alloc-hint object_size 524288 write_size 524288,write 384070~9146] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-14T06:29:08.441 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:07.013454+0000", 2024-09-14T06:29:08.441 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 1.399644927, 2024-09-14T06:29:08.441 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.0046503810000000003, 2024-09-14T06:29:08.441 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:29:08.441 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:29:08.441 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:29:08.441 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:29:08.441 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:29:08.441 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:08.441 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 803 2024-09-14T06:29:08.441 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:08.441 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:29:08.441 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:08.441 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:29:08.442 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:07.013454+0000", 2024-09-14T06:29:08.442 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:08.442 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:08.442 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:08.442 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:29:08.442 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:07.013454+0000", 2024-09-14T06:29:08.442 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:08.442 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:08.442 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:08.442 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:29:08.442 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:07.013455+0000", 2024-09-14T06:29:08.442 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.7100000000000001e-07 2024-09-14T06:29:08.442 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:08.442 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:08.442 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:29:08.443 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:07.013464+0000", 2024-09-14T06:29:08.443 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.3409999999999992e-06 2024-09-14T06:29:08.443 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:08.443 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:08.443 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:29:08.443 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:07.013474+0000", 2024-09-14T06:29:08.443 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.0128e-05 2024-09-14T06:29:08.443 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:08.443 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:08.443 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:29:08.443 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:07.013480+0000", 2024-09-14T06:29:08.443 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.3449999999999996e-06 2024-09-14T06:29:08.443 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:08.443 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:08.444 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:29:08.444 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:07.013526+0000", 2024-09-14T06:29:08.444 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.6115000000000001e-05 2024-09-14T06:29:08.444 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:08.444 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:08.444 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:29:08.444 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:07.013645+0000", 2024-09-14T06:29:08.444 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.000119429 2024-09-14T06:29:08.444 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:08.444 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:08.444 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "waiting for subops from 2", 2024-09-14T06:29:08.444 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:07.013782+0000", 2024-09-14T06:29:08.444 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.000136487 2024-09-14T06:29:08.445 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:08.445 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:08.445 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:29:08.445 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:07.014822+0000", 2024-09-14T06:29:08.445 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.0010406980000000001 2024-09-14T06:29:08.445 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:08.445 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:08.445 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "op_commit", 2024-09-14T06:29:08.445 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:07.018018+0000", 2024-09-14T06:29:08.445 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.0031957869999999998 2024-09-14T06:29:08.445 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:08.445 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:08.445 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "commit_sent", 2024-09-14T06:29:08.445 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:07.018070+0000", 2024-09-14T06:29:08.446 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.1712000000000002e-05 2024-09-14T06:29:08.446 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:08.446 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:08.446 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:29:08.446 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:07.018105+0000", 2024-09-14T06:29:08.446 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.4567999999999998e-05 2024-09-14T06:29:08.446 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:08.446 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:29:08.446 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:08.446 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:08.446 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:08.446 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_repop(client.4397.0:806 3.11 e30/23 3:88d41b02:::rbd_object_map.112d7dfbd766:head v 30'57, mlcod=30'54)", 2024-09-14T06:29:08.446 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:07.020299+0000", 2024-09-14T06:29:08.446 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 1.3928001970000001, 2024-09-14T06:29:08.447 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.0035814150000000001, 2024-09-14T06:29:08.447 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:29:08.447 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:29:08.447 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:29:08.447 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:29:08.447 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:08.447 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:29:08.447 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:07.020299+0000", 2024-09-14T06:29:08.447 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:08.447 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:08.447 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:08.447 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:29:08.447 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:07.020299+0000", 2024-09-14T06:29:08.447 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:08.448 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:08.448 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:08.448 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:29:08.448 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:07.020300+0000", 2024-09-14T06:29:08.448 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.0699999999999996e-07 2024-09-14T06:29:08.448 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:08.448 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:08.448 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:29:08.448 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:07.020305+0000", 2024-09-14T06:29:08.448 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.1909999999999999e-06 2024-09-14T06:29:08.448 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:08.448 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:08.448 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:29:08.449 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:07.020315+0000", 2024-09-14T06:29:08.449 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.0667e-05 2024-09-14T06:29:08.449 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:08.449 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:08.449 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:29:08.449 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:07.020322+0000", 2024-09-14T06:29:08.449 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.4439999999999996e-06 2024-09-14T06:29:08.449 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:08.449 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:08.449 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:29:08.449 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:07.020363+0000", 2024-09-14T06:29:08.449 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.1220000000000002e-05 2024-09-14T06:29:08.449 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:08.449 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:08.450 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:29:08.450 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:07.020376+0000", 2024-09-14T06:29:08.450 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.3010000000000001e-05 2024-09-14T06:29:08.450 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:08.450 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:08.450 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "commit_sent", 2024-09-14T06:29:08.450 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:07.023853+0000", 2024-09-14T06:29:08.450 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.0034766770000000001 2024-09-14T06:29:08.450 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:08.450 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:08.450 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:29:08.450 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:07.023880+0000", 2024-09-14T06:29:08.450 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.7498999999999999e-05 2024-09-14T06:29:08.450 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:08.450 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:29:08.451 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:08.451 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:08.451 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:08.451 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_repop(client.4397.0:814 3.11 e30/23 3:88d41b02:::rbd_object_map.112d7dfbd766:head v 30'62, mlcod=30'59)", 2024-09-14T06:29:08.451 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:07.033773+0000", 2024-09-14T06:29:08.451 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 1.3793260300000001, 2024-09-14T06:29:08.451 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.002658512, 2024-09-14T06:29:08.451 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:29:08.451 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:29:08.451 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:29:08.451 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:29:08.451 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:08.451 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:29:08.451 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:07.033773+0000", 2024-09-14T06:29:08.451 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:08.452 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:08.452 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:08.452 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:29:08.452 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:07.033773+0000", 2024-09-14T06:29:08.452 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:08.452 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:08.452 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:08.452 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:29:08.452 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:07.033774+0000", 2024-09-14T06:29:08.452 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.5499999999999998e-07 2024-09-14T06:29:08.452 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:08.452 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:08.452 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:29:08.453 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:07.033778+0000", 2024-09-14T06:29:08.453 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.5140000000000003e-06 2024-09-14T06:29:08.453 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:08.453 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:08.453 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:29:08.453 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:07.033789+0000", 2024-09-14T06:29:08.453 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.1173e-05 2024-09-14T06:29:08.453 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:08.453 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:08.453 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:29:08.453 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:07.033796+0000", 2024-09-14T06:29:08.453 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.8909999999999998e-06 2024-09-14T06:29:08.453 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:08.453 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:08.453 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:29:08.454 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:07.033831+0000", 2024-09-14T06:29:08.454 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.4879999999999998e-05 2024-09-14T06:29:08.454 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:08.454 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:08.454 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:29:08.454 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:07.033844+0000", 2024-09-14T06:29:08.454 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.2388999999999999e-05 2024-09-14T06:29:08.454 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:08.454 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:08.454 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "commit_sent", 2024-09-14T06:29:08.454 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:07.036399+0000", 2024-09-14T06:29:08.454 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.0025554589999999999 2024-09-14T06:29:08.454 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:08.454 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:08.454 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:29:08.454 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:07.036432+0000", 2024-09-14T06:29:08.455 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.2551000000000001e-05 2024-09-14T06:29:08.455 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:08.455 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:29:08.455 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:08.455 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:08.455 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:08.455 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_repop(client.4397.0:818 3.11 e30/23 3:88d41b02:::rbd_object_map.112d7dfbd766:head v 30'64, mlcod=30'61)", 2024-09-14T06:29:08.455 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:07.039127+0000", 2024-09-14T06:29:08.455 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 1.3739725149999999, 2024-09-14T06:29:08.455 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.0026504340000000001, 2024-09-14T06:29:08.455 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:29:08.455 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:29:08.455 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:29:08.455 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:29:08.455 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:08.456 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:29:08.456 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:07.039127+0000", 2024-09-14T06:29:08.456 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:08.456 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:08.456 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:08.456 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:29:08.456 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:07.039127+0000", 2024-09-14T06:29:08.456 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:08.456 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:08.456 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:08.457 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:29:08.457 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:07.039127+0000", 2024-09-14T06:29:08.457 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.9200000000000001e-07 2024-09-14T06:29:08.457 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:08.457 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:08.457 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:29:08.457 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:07.039131+0000", 2024-09-14T06:29:08.457 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.1509999999999997e-06 2024-09-14T06:29:08.457 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:08.457 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:08.457 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:29:08.457 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:07.039140+0000", 2024-09-14T06:29:08.457 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.2749999999999995e-06 2024-09-14T06:29:08.457 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:08.457 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:08.458 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:29:08.458 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:07.039145+0000", 2024-09-14T06:29:08.458 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.7019999999999998e-06 2024-09-14T06:29:08.458 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:08.458 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:08.458 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:29:08.458 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:07.039187+0000", 2024-09-14T06:29:08.458 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.1387999999999997e-05 2024-09-14T06:29:08.458 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:08.458 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:08.458 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:29:08.458 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:07.039202+0000", 2024-09-14T06:29:08.458 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.4874e-05 2024-09-14T06:29:08.459 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:08.459 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:08.459 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "commit_sent", 2024-09-14T06:29:08.459 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:07.041750+0000", 2024-09-14T06:29:08.459 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.0025479539999999998 2024-09-14T06:29:08.459 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:08.459 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:08.459 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:29:08.459 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:07.041777+0000", 2024-09-14T06:29:08.459 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.7498e-05 2024-09-14T06:29:08.459 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:08.459 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:29:08.459 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:08.459 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:08.460 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:08.460 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_repop(client.4397.0:847 3.11 e30/23 3:8db7a798:::rbd_data.112d7dfbd766.0000000000000004:head v 30'72, mlcod=30'70)", 2024-09-14T06:29:08.460 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:07.072332+0000", 2024-09-14T06:29:08.460 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 1.3407669360000001, 2024-09-14T06:29:08.460 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.0020673890000000002, 2024-09-14T06:29:08.460 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:29:08.460 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:29:08.460 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:29:08.460 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:29:08.460 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:08.460 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:29:08.460 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:07.072332+0000", 2024-09-14T06:29:08.460 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:08.461 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:08.461 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:08.461 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:29:08.461 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:07.072332+0000", 2024-09-14T06:29:08.461 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:08.461 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:08.461 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:08.461 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:29:08.461 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:07.072333+0000", 2024-09-14T06:29:08.461 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.6499999999999999e-07 2024-09-14T06:29:08.461 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:08.461 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:08.462 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:29:08.462 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:07.072364+0000", 2024-09-14T06:29:08.462 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.1761999999999998e-05 2024-09-14T06:29:08.462 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:08.462 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:08.462 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:29:08.462 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:07.072374+0000", 2024-09-14T06:29:08.462 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.4830000000000005e-06 2024-09-14T06:29:08.462 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:08.462 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:08.462 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:29:08.462 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:07.072379+0000", 2024-09-14T06:29:08.462 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.5789999999999997e-06 2024-09-14T06:29:08.463 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:08.463 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:08.463 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:29:08.463 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:07.072415+0000", 2024-09-14T06:29:08.463 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.6239999999999999e-05 2024-09-14T06:29:08.463 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:08.463 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:08.463 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:29:08.463 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:07.072428+0000", 2024-09-14T06:29:08.463 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.2771e-05 2024-09-14T06:29:08.463 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:08.463 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:08.463 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "commit_sent", 2024-09-14T06:29:08.463 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:07.074370+0000", 2024-09-14T06:29:08.464 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.001942219 2024-09-14T06:29:08.464 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:08.464 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:08.464 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:29:08.464 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:07.074400+0000", 2024-09-14T06:29:08.464 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.9770000000000001e-05 2024-09-14T06:29:08.464 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:08.464 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:29:08.464 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:08.464 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:08.464 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:08.464 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_repop(client.4397.0:865 3.e e30/23 3:761a4fa7:::rbd_data.112d7dfbd766.0000000000000137:head v 30'4, mlcod=30'2)", 2024-09-14T06:29:08.464 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:07.086959+0000", 2024-09-14T06:29:08.465 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 1.3261404429999999, 2024-09-14T06:29:08.465 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.003737322, 2024-09-14T06:29:08.465 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:29:08.465 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:29:08.465 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:29:08.465 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:29:08.465 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:08.465 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:29:08.465 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:07.086959+0000", 2024-09-14T06:29:08.465 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:08.465 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:08.465 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:08.465 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:29:08.466 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:07.086959+0000", 2024-09-14T06:29:08.466 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:08.466 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:08.466 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:08.466 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:29:08.466 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:07.086959+0000", 2024-09-14T06:29:08.466 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.9800000000000003e-07 2024-09-14T06:29:08.466 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:08.466 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:08.466 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:29:08.467 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:07.086971+0000", 2024-09-14T06:29:08.467 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.1922e-05 2024-09-14T06:29:08.467 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:08.467 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:08.467 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:29:08.467 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:07.086979+0000", 2024-09-14T06:29:08.467 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.1340000000000004e-06 2024-09-14T06:29:08.467 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:08.467 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:08.467 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:29:08.467 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:07.086984+0000", 2024-09-14T06:29:08.467 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.5600000000000004e-06 2024-09-14T06:29:08.467 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:08.468 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:08.468 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:29:08.468 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:07.087028+0000", 2024-09-14T06:29:08.468 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.4215000000000003e-05 2024-09-14T06:29:08.468 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:08.468 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:08.468 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:29:08.468 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:07.087042+0000", 2024-09-14T06:29:08.468 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.3577000000000001e-05 2024-09-14T06:29:08.469 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:08.469 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:08.469 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "commit_sent", 2024-09-14T06:29:08.469 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:07.090661+0000", 2024-09-14T06:29:08.469 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.003619386 2024-09-14T06:29:08.469 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:08.469 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:08.469 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:29:08.469 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:07.090696+0000", 2024-09-14T06:29:08.469 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.4929999999999999e-05 2024-09-14T06:29:08.469 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:08.469 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:29:08.469 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:08.469 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:08.470 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:08.470 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_repop(client.4397.0:867 3.11 e30/23 3:88d41b02:::rbd_object_map.112d7dfbd766:head v 30'79, mlcod=30'76)", 2024-09-14T06:29:08.470 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:07.091988+0000", 2024-09-14T06:29:08.470 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 1.3211109429999999, 2024-09-14T06:29:08.470 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.003967028, 2024-09-14T06:29:08.470 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:29:08.470 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:29:08.470 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:29:08.470 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:29:08.470 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:08.470 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:29:08.470 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:07.091988+0000", 2024-09-14T06:29:08.470 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:08.470 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:08.471 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:08.471 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:29:08.471 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:07.091988+0000", 2024-09-14T06:29:08.471 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:08.471 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:08.471 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:08.471 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:29:08.471 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:07.091989+0000", 2024-09-14T06:29:08.471 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.6100000000000001e-07 2024-09-14T06:29:08.471 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:08.471 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:08.471 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:29:08.471 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:07.091994+0000", 2024-09-14T06:29:08.471 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.8779999999999997e-06 2024-09-14T06:29:08.472 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:08.472 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:08.472 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:29:08.472 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:07.092003+0000", 2024-09-14T06:29:08.472 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.6420000000000002e-06 2024-09-14T06:29:08.472 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:08.472 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:08.472 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:29:08.472 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:07.092009+0000", 2024-09-14T06:29:08.472 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.677e-06 2024-09-14T06:29:08.472 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:08.472 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:08.472 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:29:08.473 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:07.092047+0000", 2024-09-14T06:29:08.473 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.7976999999999999e-05 2024-09-14T06:29:08.473 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:08.473 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:08.473 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:29:08.473 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:07.092058+0000", 2024-09-14T06:29:08.473 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.0893999999999999e-05 2024-09-14T06:29:08.473 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:08.473 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:08.473 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "commit_sent", 2024-09-14T06:29:08.473 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:07.095925+0000", 2024-09-14T06:29:08.473 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.0038674299999999998 2024-09-14T06:29:08.473 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:08.473 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:08.474 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:29:08.474 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:07.095955+0000", 2024-09-14T06:29:08.474 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.9969000000000001e-05 2024-09-14T06:29:08.474 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:08.474 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:29:08.474 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:08.474 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:08.474 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:08.474 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_repop(client.4397.0:873 3.11 e30/23 3:88d41b02:::rbd_object_map.112d7dfbd766:head v 30'82, mlcod=30'79)", 2024-09-14T06:29:08.474 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:07.101538+0000", 2024-09-14T06:29:08.474 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 1.3115606849999999, 2024-09-14T06:29:08.474 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.0021352459999999999, 2024-09-14T06:29:08.474 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:29:08.474 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:29:08.475 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:29:08.475 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:29:08.475 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:08.475 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:29:08.475 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:07.101538+0000", 2024-09-14T06:29:08.475 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:08.475 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:08.475 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:08.475 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:29:08.475 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:07.101538+0000", 2024-09-14T06:29:08.475 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:08.475 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:08.475 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:08.475 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:29:08.476 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:07.101539+0000", 2024-09-14T06:29:08.476 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.5000000000000002e-07 2024-09-14T06:29:08.476 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:08.476 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:08.476 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:29:08.476 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:07.101544+0000", 2024-09-14T06:29:08.476 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.2290000000000002e-06 2024-09-14T06:29:08.476 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:08.476 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:08.476 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:29:08.476 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:07.101555+0000", 2024-09-14T06:29:08.476 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.0677e-05 2024-09-14T06:29:08.476 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:08.476 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:08.477 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:29:08.477 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:07.101560+0000", 2024-09-14T06:29:08.477 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.9069999999999998e-06 2024-09-14T06:29:08.477 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:08.477 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:08.477 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:29:08.477 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:07.101606+0000", 2024-09-14T06:29:08.477 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.6193999999999999e-05 2024-09-14T06:29:08.477 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:08.477 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:08.477 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:29:08.477 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:07.101624+0000", 2024-09-14T06:29:08.477 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.7592999999999999e-05 2024-09-14T06:29:08.478 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:08.478 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:08.478 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "commit_sent", 2024-09-14T06:29:08.478 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:07.103632+0000", 2024-09-14T06:29:08.478 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.0020084999999999999 2024-09-14T06:29:08.478 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:08.478 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:08.478 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:29:08.478 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:07.103674+0000", 2024-09-14T06:29:08.478 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.1495999999999997e-05 2024-09-14T06:29:08.478 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:08.478 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:29:08.478 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:08.479 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:08.479 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:08.479 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:875 3.3 3:c1192472:::rbd_data.112d7dfbd766.000000000000010f:head [set-alloc-hint object_size 524288 write_size 524288,write 327680~57401] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-14T06:29:08.479 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:07.103994+0000", 2024-09-14T06:29:08.479 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 1.309104692, 2024-09-14T06:29:08.479 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.0021599380000000001, 2024-09-14T06:29:08.479 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:29:08.479 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:29:08.479 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:29:08.479 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:29:08.479 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:29:08.479 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:08.479 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 875 2024-09-14T06:29:08.479 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:08.479 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:29:08.480 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:08.480 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:29:08.480 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:07.103994+0000", 2024-09-14T06:29:08.480 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:08.480 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:08.480 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:08.480 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:29:08.480 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:07.103994+0000", 2024-09-14T06:29:08.480 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:08.480 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:08.480 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:08.480 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:29:08.480 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:07.103995+0000", 2024-09-14T06:29:08.481 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.6099999999999999e-07 2024-09-14T06:29:08.481 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:08.481 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:08.481 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:29:08.481 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:07.104032+0000", 2024-09-14T06:29:08.481 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.6927000000000001e-05 2024-09-14T06:29:08.481 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:08.481 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:08.481 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:29:08.481 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:07.104044+0000", 2024-09-14T06:29:08.481 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.1240000000000001e-05 2024-09-14T06:29:08.481 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:08.481 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:08.481 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:29:08.482 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:07.104049+0000", 2024-09-14T06:29:08.482 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.451e-06 2024-09-14T06:29:08.482 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:08.482 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:08.482 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:29:08.482 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:07.104092+0000", 2024-09-14T06:29:08.482 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.2716999999999998e-05 2024-09-14T06:29:08.482 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:08.482 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:08.482 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:29:08.482 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:07.104142+0000", 2024-09-14T06:29:08.482 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.0145999999999997e-05 2024-09-14T06:29:08.482 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:08.482 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:08.483 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "waiting for subops from 0", 2024-09-14T06:29:08.483 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:07.104233+0000", 2024-09-14T06:29:08.483 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.1434000000000006e-05 2024-09-14T06:29:08.483 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:08.483 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:08.483 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:29:08.483 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:07.105316+0000", 2024-09-14T06:29:08.483 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.0010828490000000001 2024-09-14T06:29:08.483 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:08.484 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:08.484 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "op_commit", 2024-09-14T06:29:08.484 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:07.106027+0000", 2024-09-14T06:29:08.484 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00071113399999999998 2024-09-14T06:29:08.484 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:08.484 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:08.484 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "commit_sent", 2024-09-14T06:29:08.484 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:07.106104+0000", 2024-09-14T06:29:08.484 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.6729000000000003e-05 2024-09-14T06:29:08.484 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:08.484 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:08.484 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:29:08.484 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:07.106154+0000", 2024-09-14T06:29:08.484 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.0349999999999997e-05 2024-09-14T06:29:08.485 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:08.485 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:29:08.485 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:08.485 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:08.485 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:08.485 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_repop(client.4397.0:887 3.f e30/23 3:f62ad09f:::rbd_data.112d7dfbd766.0000000000000134:head v 30'4, mlcod=27'2)", 2024-09-14T06:29:08.485 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:07.115713+0000", 2024-09-14T06:29:08.485 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 1.2973861170000001, 2024-09-14T06:29:08.485 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.0022341430000000001, 2024-09-14T06:29:08.485 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:29:08.485 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:29:08.485 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:29:08.485 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:29:08.486 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:08.486 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:29:08.486 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:07.115713+0000", 2024-09-14T06:29:08.486 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:08.486 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:08.486 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:08.486 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:29:08.486 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:07.115713+0000", 2024-09-14T06:29:08.486 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:08.486 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:08.486 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:08.486 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:29:08.486 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:07.115714+0000", 2024-09-14T06:29:08.486 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.0900000000000001e-07 2024-09-14T06:29:08.487 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:08.487 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:08.487 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:29:08.487 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:07.115719+0000", 2024-09-14T06:29:08.487 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.5729999999999998e-06 2024-09-14T06:29:08.487 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:08.487 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:08.487 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:29:08.487 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:07.115731+0000", 2024-09-14T06:29:08.487 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.1639e-05 2024-09-14T06:29:08.487 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:08.487 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:08.487 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:29:08.488 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:07.115737+0000", 2024-09-14T06:29:08.488 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.4300000000000003e-06 2024-09-14T06:29:08.488 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:08.488 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:08.488 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:29:08.488 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:07.115787+0000", 2024-09-14T06:29:08.488 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.0107e-05 2024-09-14T06:29:08.488 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:08.488 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:08.488 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:29:08.488 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:07.115809+0000", 2024-09-14T06:29:08.488 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.1963e-05 2024-09-14T06:29:08.488 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:08.489 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:08.489 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "commit_sent", 2024-09-14T06:29:08.489 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:07.117907+0000", 2024-09-14T06:29:08.489 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.0020980980000000001 2024-09-14T06:29:08.489 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:08.489 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:08.489 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:29:08.489 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:07.117947+0000", 2024-09-14T06:29:08.489 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.9724000000000001e-05 2024-09-14T06:29:08.489 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:08.489 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:29:08.489 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:08.489 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:08.490 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:08.490 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_repop(client.4397.0:888 3.e e30/23 3:761a4fa7:::rbd_data.112d7dfbd766.0000000000000137:head v 30'5, mlcod=30'3)", 2024-09-14T06:29:08.490 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:07.115792+0000", 2024-09-14T06:29:08.490 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 1.2973069960000001, 2024-09-14T06:29:08.490 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.0024282259999999999, 2024-09-14T06:29:08.490 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:29:08.490 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:29:08.490 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:29:08.490 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:29:08.490 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:08.490 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:29:08.490 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:07.115792+0000", 2024-09-14T06:29:08.490 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:08.490 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:08.491 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:08.491 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:29:08.491 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:07.115792+0000", 2024-09-14T06:29:08.491 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:08.491 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:08.491 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:08.491 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:29:08.491 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:07.115793+0000", 2024-09-14T06:29:08.491 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.1600000000000001e-07 2024-09-14T06:29:08.491 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:08.491 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:08.491 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:29:08.491 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:07.115797+0000", 2024-09-14T06:29:08.491 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.1500000000000001e-06 2024-09-14T06:29:08.492 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:08.492 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:08.492 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:29:08.492 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:07.115805+0000", 2024-09-14T06:29:08.492 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.9140000000000008e-06 2024-09-14T06:29:08.492 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:08.492 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:08.492 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:29:08.492 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:07.115810+0000", 2024-09-14T06:29:08.492 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.4589999999999997e-06 2024-09-14T06:29:08.492 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:08.492 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:08.492 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:29:08.492 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:07.115867+0000", 2024-09-14T06:29:08.492 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.6728999999999998e-05 2024-09-14T06:29:08.493 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:08.493 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:08.493 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:29:08.493 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:07.115882+0000", 2024-09-14T06:29:08.493 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.4837999999999999e-05 2024-09-14T06:29:08.493 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:08.493 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:08.493 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "commit_sent", 2024-09-14T06:29:08.493 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:07.118179+0000", 2024-09-14T06:29:08.493 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.002297515 2024-09-14T06:29:08.493 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:08.493 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:08.493 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:29:08.494 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:07.118220+0000", 2024-09-14T06:29:08.494 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.1105e-05 2024-09-14T06:29:08.494 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:08.494 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:29:08.494 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:08.494 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:08.494 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:08.494 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:937 3.9 3:92b37d3e:::rbd_data.112d7dfbd766.0000000000000012:head [set-alloc-hint object_size 524288 write_size 524288,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-14T06:29:08.494 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:07.164952+0000", 2024-09-14T06:29:08.494 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 1.248146778, 2024-09-14T06:29:08.494 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.011535095, 2024-09-14T06:29:08.494 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:29:08.494 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:29:08.494 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:29:08.494 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:29:08.494 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:29:08.495 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:08.495 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 937 2024-09-14T06:29:08.495 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:08.495 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:29:08.495 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:08.495 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:29:08.495 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:07.164952+0000", 2024-09-14T06:29:08.495 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:08.495 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:08.495 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:08.495 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:29:08.495 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:07.164952+0000", 2024-09-14T06:29:08.495 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:08.495 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:08.495 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:08.496 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:29:08.496 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:07.164953+0000", 2024-09-14T06:29:08.496 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.3300000000000001e-07 2024-09-14T06:29:08.496 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:08.496 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:08.496 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:29:08.496 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:07.164957+0000", 2024-09-14T06:29:08.496 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.912e-06 2024-09-14T06:29:08.496 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:08.496 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:08.496 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:29:08.496 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:07.164965+0000", 2024-09-14T06:29:08.496 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.2770000000000005e-06 2024-09-14T06:29:08.496 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:08.496 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:08.497 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:29:08.497 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:07.164972+0000", 2024-09-14T06:29:08.497 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.6819999999999997e-06 2024-09-14T06:29:08.497 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:08.497 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:08.497 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:29:08.497 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:07.165017+0000", 2024-09-14T06:29:08.497 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.5223999999999999e-05 2024-09-14T06:29:08.497 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:08.497 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:08.497 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:29:08.497 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:07.165106+0000", 2024-09-14T06:29:08.497 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.9239000000000004e-05 2024-09-14T06:29:08.497 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:08.498 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:08.498 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "waiting for subops from 2", 2024-09-14T06:29:08.498 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:07.167003+0000", 2024-09-14T06:29:08.498 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.0018969799999999999 2024-09-14T06:29:08.498 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:08.498 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:08.498 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:29:08.498 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:07.176420+0000", 2024-09-14T06:29:08.498 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.0094163990000000006 2024-09-14T06:29:08.498 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:08.498 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:08.498 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "op_commit", 2024-09-14T06:29:08.498 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:07.176430+0000", 2024-09-14T06:29:08.498 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.0558e-05 2024-09-14T06:29:08.498 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:08.498 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:08.499 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "commit_sent", 2024-09-14T06:29:08.499 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:07.176459+0000", 2024-09-14T06:29:08.499 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.8415000000000002e-05 2024-09-14T06:29:08.499 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:08.499 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:08.499 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:29:08.499 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:07.176487+0000", 2024-09-14T06:29:08.499 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.8676000000000001e-05 2024-09-14T06:29:08.499 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:08.499 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:29:08.499 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:08.499 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:08.499 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:08.499 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_repop_reply(client.4397.0:937 3.9 e30/23 ondisk, result = 0)", 2024-09-14T06:29:08.499 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:07.172779+0000", 2024-09-14T06:29:08.500 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 1.240320053, 2024-09-14T06:29:08.500 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.003645587, 2024-09-14T06:29:08.500 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:29:08.500 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:29:08.500 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:29:08.500 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:29:08.500 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:08.500 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:29:08.500 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:07.172779+0000", 2024-09-14T06:29:08.500 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:08.500 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:08.500 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:08.500 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:29:08.500 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:07.172779+0000", 2024-09-14T06:29:08.500 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:08.501 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:08.501 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:08.501 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:29:08.501 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:07.172780+0000", 2024-09-14T06:29:08.501 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.0800000000000004e-07 2024-09-14T06:29:08.501 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:08.501 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:08.501 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:29:08.501 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:07.172783+0000", 2024-09-14T06:29:08.501 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.579e-06 2024-09-14T06:29:08.501 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:08.501 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:08.501 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:29:08.502 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:07.172794+0000", 2024-09-14T06:29:08.502 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.0265000000000001e-05 2024-09-14T06:29:08.502 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:08.502 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:08.502 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:29:08.502 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:07.172799+0000", 2024-09-14T06:29:08.502 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.0679999999999996e-06 2024-09-14T06:29:08.502 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:08.502 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:08.502 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:29:08.502 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:07.176408+0000", 2024-09-14T06:29:08.502 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.0036096119999999999 2024-09-14T06:29:08.502 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:08.502 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:08.502 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:29:08.503 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:07.176415+0000", 2024-09-14T06:29:08.503 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.7530000000000004e-06 2024-09-14T06:29:08.503 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:08.503 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:08.503 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:29:08.503 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:07.176425+0000", 2024-09-14T06:29:08.503 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.6020000000000006e-06 2024-09-14T06:29:08.503 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:08.503 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:29:08.503 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:08.503 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:08.503 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:08.503 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_repop(client.4397.0:939 3.11 e30/23 3:88d41b02:::rbd_object_map.112d7dfbd766:head v 30'99, mlcod=30'96)", 2024-09-14T06:29:08.503 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:07.179458+0000", 2024-09-14T06:29:08.504 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 1.2336412240000001, 2024-09-14T06:29:08.504 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.002740803, 2024-09-14T06:29:08.504 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:29:08.504 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:29:08.504 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:29:08.504 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:29:08.504 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:08.504 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:29:08.504 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:07.179458+0000", 2024-09-14T06:29:08.504 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:08.504 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:08.504 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:08.504 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:29:08.504 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:07.179458+0000", 2024-09-14T06:29:08.504 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:08.505 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:08.505 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:08.505 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:29:08.505 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:07.179458+0000", 2024-09-14T06:29:08.505 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.4700000000000002e-07 2024-09-14T06:29:08.505 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:08.505 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:08.505 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:29:08.505 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:07.179462+0000", 2024-09-14T06:29:08.505 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.7519999999999998e-06 2024-09-14T06:29:08.505 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:08.505 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:08.505 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:29:08.506 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:07.179470+0000", 2024-09-14T06:29:08.506 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.3590000000000008e-06 2024-09-14T06:29:08.506 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:08.506 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:08.506 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:29:08.506 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:07.179475+0000", 2024-09-14T06:29:08.506 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.4619999999999998e-06 2024-09-14T06:29:08.506 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:08.506 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:08.506 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:29:08.506 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:07.179518+0000", 2024-09-14T06:29:08.506 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.2670999999999999e-05 2024-09-14T06:29:08.506 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:08.506 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:08.507 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:29:08.507 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:07.179532+0000", 2024-09-14T06:29:08.507 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.3943e-05 2024-09-14T06:29:08.507 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:08.507 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:08.507 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "commit_sent", 2024-09-14T06:29:08.507 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:07.182164+0000", 2024-09-14T06:29:08.507 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.0026327 2024-09-14T06:29:08.507 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:08.507 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:08.507 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:29:08.507 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:07.182199+0000", 2024-09-14T06:29:08.507 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.4468999999999998e-05 2024-09-14T06:29:08.507 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:08.507 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:29:08.508 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:08.508 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:08.508 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:08.508 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_repop(client.4397.0:941 3.e e30/23 3:76e48551:::rbd_data.112d7dfbd766.000000000000018e:head v 30'6, mlcod=30'4)", 2024-09-14T06:29:08.508 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:07.182799+0000", 2024-09-14T06:29:08.508 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 1.2303003889999999, 2024-09-14T06:29:08.508 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.0020029850000000001, 2024-09-14T06:29:08.508 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:29:08.508 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:29:08.508 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:29:08.508 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:29:08.508 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:08.508 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:29:08.508 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:07.182799+0000", 2024-09-14T06:29:08.509 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:08.509 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:08.509 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:08.509 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:29:08.509 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:07.182799+0000", 2024-09-14T06:29:08.509 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:08.509 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:08.509 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:08.509 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:29:08.509 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:07.182799+0000", 2024-09-14T06:29:08.509 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.8999999999999996e-07 2024-09-14T06:29:08.509 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:08.509 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:08.509 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:29:08.510 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:07.182816+0000", 2024-09-14T06:29:08.510 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.6884e-05 2024-09-14T06:29:08.510 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:08.510 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:08.510 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:29:08.510 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:07.182825+0000", 2024-09-14T06:29:08.510 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.5220000000000007e-06 2024-09-14T06:29:08.510 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:08.510 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:08.510 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:29:08.510 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:07.182831+0000", 2024-09-14T06:29:08.510 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.1399999999999997e-06 2024-09-14T06:29:08.510 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:08.510 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:08.511 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:29:08.511 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:07.182878+0000", 2024-09-14T06:29:08.511 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.761e-05 2024-09-14T06:29:08.511 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:08.511 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:08.511 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:29:08.511 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:07.182893+0000", 2024-09-14T06:29:08.511 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.4303e-05 2024-09-14T06:29:08.511 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:08.511 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:08.511 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "commit_sent", 2024-09-14T06:29:08.511 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:07.184771+0000", 2024-09-14T06:29:08.511 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.0018778530000000001 2024-09-14T06:29:08.511 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:08.512 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:08.512 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:29:08.512 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:07.184802+0000", 2024-09-14T06:29:08.512 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.1083e-05 2024-09-14T06:29:08.512 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:08.512 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:29:08.512 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:08.512 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:08.512 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:29:08.512 INFO:teuthology.orchestra.run.smithi138.stdout:} 2024-09-14T06:29:08.514 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-14T06:29:08.514 INFO:tasks.ceph.osd.4.smithi138.stderr:2024-09-14T06:29:08.512+0000 7f6c653c8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33615) UID: 0 2024-09-14T06:29:08.610 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-14T06:29:08.655 INFO:tasks.ceph.osd.3.smithi094.stderr:2024-09-14T06:29:08.654+0000 7feeeafe8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33655) UID: 0 2024-09-14T06:29:08.695 INFO:teuthology.orchestra.run.smithi138.stdout:{ 2024-09-14T06:29:08.695 INFO:teuthology.orchestra.run.smithi138.stdout: "ops": [ 2024-09-14T06:29:08.695 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:08.695 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:2350 3.1d 3:bf9aaf9c:::rbd_object_map.112dcdddac74.0000000000000006:head [call rbd.object_map_update in=38b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-09-14T06:29:08.695 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:08.689617+0000", 2024-09-14T06:29:08.695 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 0.0042171550000000002, 2024-09-14T06:29:08.695 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.0042431420000000001, 2024-09-14T06:29:08.696 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:29:08.696 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:29:08.696 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "waiting for sub ops", 2024-09-14T06:29:08.696 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:29:08.696 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:29:08.696 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:08.696 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 2350 2024-09-14T06:29:08.696 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:08.696 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:29:08.696 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:08.696 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:29:08.696 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.689617+0000", 2024-09-14T06:29:08.696 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:08.697 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:08.697 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:08.697 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:29:08.697 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.689617+0000", 2024-09-14T06:29:08.697 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:08.697 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:08.697 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:08.697 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:29:08.697 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.689617+0000", 2024-09-14T06:29:08.697 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.4600000000000003e-07 2024-09-14T06:29:08.697 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:08.697 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:08.697 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:29:08.698 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.689622+0000", 2024-09-14T06:29:08.698 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.5249999999999999e-06 2024-09-14T06:29:08.698 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:08.698 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:08.698 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:29:08.698 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.689634+0000", 2024-09-14T06:29:08.698 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.2249000000000001e-05 2024-09-14T06:29:08.698 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:08.698 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:08.698 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:29:08.698 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.689641+0000", 2024-09-14T06:29:08.698 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.8549999999999996e-06 2024-09-14T06:29:08.698 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:08.699 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:08.699 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:29:08.699 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.689711+0000", 2024-09-14T06:29:08.699 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.0118999999999997e-05 2024-09-14T06:29:08.699 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:08.699 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:08.699 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:29:08.699 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.689795+0000", 2024-09-14T06:29:08.699 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.3968000000000007e-05 2024-09-14T06:29:08.699 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:08.699 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:08.699 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "waiting for subops from 4", 2024-09-14T06:29:08.699 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.690126+0000", 2024-09-14T06:29:08.699 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00033108999999999998 2024-09-14T06:29:08.700 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:08.700 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:29:08.700 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:08.700 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:08.700 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:08.700 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:2397 3.1d 3:bf9aaf9c:::rbd_object_map.112dcdddac74.0000000000000006:head [call rbd.object_map_update in=38b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-09-14T06:29:08.700 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:08.690341+0000", 2024-09-14T06:29:08.700 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 0.0034927420000000001, 2024-09-14T06:29:08.700 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.0035973820000000001, 2024-09-14T06:29:08.700 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:29:08.700 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:29:08.700 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "waiting for sub ops", 2024-09-14T06:29:08.701 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:29:08.701 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:29:08.701 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:08.701 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 2397 2024-09-14T06:29:08.701 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:08.701 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:29:08.701 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:08.701 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:29:08.701 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.690341+0000", 2024-09-14T06:29:08.701 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:08.701 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:08.701 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:08.702 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:29:08.702 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.690341+0000", 2024-09-14T06:29:08.702 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:08.702 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:08.702 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:08.702 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:29:08.702 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.690342+0000", 2024-09-14T06:29:08.702 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.4099999999999997e-07 2024-09-14T06:29:08.702 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:08.702 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:08.702 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:29:08.702 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.690347+0000", 2024-09-14T06:29:08.702 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.9980000000000003e-06 2024-09-14T06:29:08.702 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:08.703 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:08.703 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:29:08.703 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.690358+0000", 2024-09-14T06:29:08.703 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.0662999999999999e-05 2024-09-14T06:29:08.703 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:08.703 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:08.703 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:29:08.703 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.690362+0000", 2024-09-14T06:29:08.703 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.8960000000000002e-06 2024-09-14T06:29:08.703 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:08.703 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:08.703 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:29:08.703 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.690484+0000", 2024-09-14T06:29:08.703 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00012132400000000001 2024-09-14T06:29:08.704 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:08.704 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:08.704 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:29:08.704 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.690569+0000", 2024-09-14T06:29:08.704 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.4753999999999995e-05 2024-09-14T06:29:08.704 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:08.704 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:08.704 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "waiting for subops from 4", 2024-09-14T06:29:08.704 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.690910+0000", 2024-09-14T06:29:08.704 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00034160199999999999 2024-09-14T06:29:08.704 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:08.704 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:29:08.704 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:08.705 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:08.705 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:08.705 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:2453 3.1d 3:bf9aaf9c:::rbd_object_map.112dcdddac74.0000000000000006:head [call rbd.object_map_update in=38b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-09-14T06:29:08.705 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:08.692778+0000", 2024-09-14T06:29:08.705 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 0.0010557120000000001, 2024-09-14T06:29:08.705 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.0012273850000000001, 2024-09-14T06:29:08.705 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:29:08.705 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:29:08.705 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "waiting for sub ops", 2024-09-14T06:29:08.705 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:29:08.705 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:29:08.705 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:08.705 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 2453 2024-09-14T06:29:08.705 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:08.706 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:29:08.706 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:08.706 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:29:08.706 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.692778+0000", 2024-09-14T06:29:08.706 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:08.706 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:08.706 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:08.706 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:29:08.706 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.692778+0000", 2024-09-14T06:29:08.706 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:08.706 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:08.706 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:08.706 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:29:08.706 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.692779+0000", 2024-09-14T06:29:08.707 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.5999999999999991e-07 2024-09-14T06:29:08.707 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:08.707 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:08.707 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:29:08.707 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.692784+0000", 2024-09-14T06:29:08.707 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.1599999999999997e-06 2024-09-14T06:29:08.707 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:08.707 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:08.707 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:29:08.707 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.692799+0000", 2024-09-14T06:29:08.707 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.5193e-05 2024-09-14T06:29:08.707 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:08.707 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:08.707 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:29:08.707 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.692807+0000", 2024-09-14T06:29:08.707 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.8169999999999999e-06 2024-09-14T06:29:08.708 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:08.708 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:08.708 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:29:08.708 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.692859+0000", 2024-09-14T06:29:08.708 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.2034999999999999e-05 2024-09-14T06:29:08.708 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:08.708 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:08.708 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:29:08.708 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.693021+0000", 2024-09-14T06:29:08.708 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00016214199999999999 2024-09-14T06:29:08.708 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:08.708 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:08.708 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "waiting for subops from 4", 2024-09-14T06:29:08.708 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.693364+0000", 2024-09-14T06:29:08.709 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00034234499999999998 2024-09-14T06:29:08.709 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:08.709 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:29:08.709 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:08.709 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:08.709 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:08.709 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:2455 3.1d 3:bf9aaf9c:::rbd_object_map.112dcdddac74.0000000000000006:head [call rbd.object_map_update in=38b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-09-14T06:29:08.709 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:08.693535+0000", 2024-09-14T06:29:08.709 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 0.00029876499999999999, 2024-09-14T06:29:08.709 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00054008799999999998, 2024-09-14T06:29:08.709 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:29:08.709 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:29:08.709 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:29:08.709 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:29:08.709 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:29:08.710 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:08.710 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 2455 2024-09-14T06:29:08.710 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:08.710 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:29:08.710 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:08.710 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:29:08.710 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.693535+0000", 2024-09-14T06:29:08.710 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:08.710 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:08.710 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:08.710 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:29:08.710 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.693535+0000", 2024-09-14T06:29:08.710 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:08.710 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:08.710 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:08.711 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:29:08.711 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.693536+0000", 2024-09-14T06:29:08.711 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.3399999999999998e-07 2024-09-14T06:29:08.711 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:08.711 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:08.711 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:29:08.711 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.693541+0000", 2024-09-14T06:29:08.711 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.9629999999999997e-06 2024-09-14T06:29:08.711 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:08.711 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:08.711 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:29:08.711 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.693551+0000", 2024-09-14T06:29:08.711 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.0642000000000001e-05 2024-09-14T06:29:08.711 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:08.711 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:08.711 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:29:08.712 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.693560+0000", 2024-09-14T06:29:08.712 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.2209999999999997e-06 2024-09-14T06:29:08.712 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:08.712 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:08.712 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:29:08.712 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.693859+0000", 2024-09-14T06:29:08.712 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00029938399999999998 2024-09-14T06:29:08.712 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:08.712 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:08.712 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:29:08.712 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.693970+0000", 2024-09-14T06:29:08.712 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00011136700000000001 2024-09-14T06:29:08.712 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:08.712 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:29:08.712 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:08.713 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:08.713 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:08.713 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:2456 3.1c 3:3ace1f47:::rbd_data.112dcdddac74.0000000000000021:head [write 16384~16384 [fadvise_sequential+fadvise_nocache] in=16384b,write 81920~16384 [fadvise_sequential+fadvise_nocache] in=16384b,write 147456~16384 [fadvise_sequential+fadvise_nocache] in=16384b,write 212992~12288 [fadvise_sequential+fadvise_nocache] in=12288b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-09-14T06:29:08.713 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:08.693956+0000", 2024-09-14T06:29:08.713 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 4294967295.9998779, 2024-09-14T06:29:08.713 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00019192100000000001, 2024-09-14T06:29:08.713 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:29:08.713 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:29:08.713 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "reached pg", 2024-09-14T06:29:08.713 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:29:08.713 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:29:08.713 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:08.713 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 2456 2024-09-14T06:29:08.713 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:08.714 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:29:08.714 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:08.714 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:29:08.714 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.693956+0000", 2024-09-14T06:29:08.714 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:08.714 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:08.714 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:08.714 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:29:08.714 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.693956+0000", 2024-09-14T06:29:08.714 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:08.714 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:08.714 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:08.714 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:29:08.714 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.693957+0000", 2024-09-14T06:29:08.714 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.6700000000000003e-07 2024-09-14T06:29:08.715 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:08.715 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:08.715 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:29:08.715 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.693991+0000", 2024-09-14T06:29:08.715 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.4691999999999999e-05 2024-09-14T06:29:08.715 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:08.715 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:08.715 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:29:08.715 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.694002+0000", 2024-09-14T06:29:08.715 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.0890000000000001e-05 2024-09-14T06:29:08.715 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:08.715 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:08.715 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:29:08.715 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.694010+0000", 2024-09-14T06:29:08.715 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.2860000000000002e-06 2024-09-14T06:29:08.715 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:08.716 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:08.716 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:29:08.716 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.694077+0000", 2024-09-14T06:29:08.716 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.7063000000000005e-05 2024-09-14T06:29:08.716 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:08.716 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:29:08.716 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:08.716 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:08.716 INFO:teuthology.orchestra.run.smithi138.stdout: ], 2024-09-14T06:29:08.716 INFO:teuthology.orchestra.run.smithi138.stdout: "num_ops": 5 2024-09-14T06:29:08.716 INFO:teuthology.orchestra.run.smithi138.stdout:} 2024-09-14T06:29:08.718 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-14T06:29:08.718 INFO:tasks.ceph.osd.6.smithi138.stderr:2024-09-14T06:29:08.716+0000 7f8cd8849640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33620) UID: 0 2024-09-14T06:29:08.818 INFO:tasks.ceph.osd.7.smithi138.stderr:2024-09-14T06:29:08.817+0000 7f3c6a21d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33623) UID: 0 2024-09-14T06:29:08.877 INFO:teuthology.orchestra.run.smithi138.stdout:{ 2024-09-14T06:29:08.877 INFO:teuthology.orchestra.run.smithi138.stdout: "ops": [], 2024-09-14T06:29:08.878 INFO:teuthology.orchestra.run.smithi138.stdout: "complaint_time": 30, 2024-09-14T06:29:08.878 INFO:teuthology.orchestra.run.smithi138.stdout: "num_blocked_ops": 0 2024-09-14T06:29:08.878 INFO:teuthology.orchestra.run.smithi138.stdout:} 2024-09-14T06:29:08.884 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-14T06:29:08.886 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-14T06:29:08.927 INFO:tasks.ceph.osd.7.smithi138.stderr:2024-09-14T06:29:08.926+0000 7f3c6a21d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33623) UID: 0 2024-09-14T06:29:09.005 INFO:teuthology.orchestra.run.smithi094.stdout:osd.0: {} 2024-09-14T06:29:09.005 INFO:teuthology.orchestra.run.smithi094.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-14T06:29:09.019 INFO:tasks.ceph.osd.1.smithi094.stderr:2024-09-14T06:29:09.018+0000 7f328e47a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33641) UID: 0 2024-09-14T06:29:09.021 INFO:teuthology.orchestra.run.smithi094.stdout:osd.1: {} 2024-09-14T06:29:09.022 INFO:teuthology.orchestra.run.smithi094.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-14T06:29:09.032 INFO:teuthology.orchestra.run.smithi138.stdout:{ 2024-09-14T06:29:09.032 INFO:teuthology.orchestra.run.smithi138.stdout: "size": 20, 2024-09-14T06:29:09.032 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 600, 2024-09-14T06:29:09.032 INFO:teuthology.orchestra.run.smithi138.stdout: "ops": [ 2024-09-14T06:29:09.032 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:09.032 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:3256 3.5 3:a6acff7e:::rbd_data.112d7dfbd766.0000000000000078:5 [read 110592~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-14T06:29:09.032 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:08.738879+0000", 2024-09-14T06:29:09.032 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 0.29137132900000001, 2024-09-14T06:29:09.033 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.011432067000000001, 2024-09-14T06:29:09.033 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:29:09.033 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:29:09.033 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:29:09.033 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:29:09.033 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:29:09.033 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:09.033 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 3256 2024-09-14T06:29:09.033 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:09.033 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:29:09.033 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:09.033 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:29:09.034 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.738879+0000", 2024-09-14T06:29:09.034 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:09.034 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:09.034 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:09.034 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:29:09.034 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.738879+0000", 2024-09-14T06:29:09.034 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:09.034 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:09.034 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:09.034 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:29:09.034 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.738879+0000", 2024-09-14T06:29:09.034 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.8099999999999998e-07 2024-09-14T06:29:09.034 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:09.034 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:09.034 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:29:09.035 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.738885+0000", 2024-09-14T06:29:09.035 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.3199999999999999e-06 2024-09-14T06:29:09.035 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:09.035 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:09.035 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:29:09.035 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.738891+0000", 2024-09-14T06:29:09.035 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.3149999999999997e-06 2024-09-14T06:29:09.035 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:09.035 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:09.035 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:29:09.035 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.738895+0000", 2024-09-14T06:29:09.035 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.478e-06 2024-09-14T06:29:09.035 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:09.036 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:09.036 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:29:09.036 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.750221+0000", 2024-09-14T06:29:09.036 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.011326166 2024-09-14T06:29:09.036 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:09.036 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:09.036 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:29:09.036 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.750260+0000", 2024-09-14T06:29:09.036 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.9239999999999997e-05 2024-09-14T06:29:09.036 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:09.036 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:09.036 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:29:09.037 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.750311+0000", 2024-09-14T06:29:09.037 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.1167e-05 2024-09-14T06:29:09.037 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:09.037 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:29:09.037 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:09.037 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:09.037 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:09.037 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:3257 3.5 3:a6acff7e:::rbd_data.112d7dfbd766.0000000000000078:5 [read 126976~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-14T06:29:09.037 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:08.738902+0000", 2024-09-14T06:29:09.037 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 0.29134863700000002, 2024-09-14T06:29:09.037 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.011528261999999999, 2024-09-14T06:29:09.037 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:29:09.037 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:29:09.038 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:29:09.038 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:29:09.038 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:29:09.038 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:09.038 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 3257 2024-09-14T06:29:09.038 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:09.038 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:29:09.038 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:09.038 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:29:09.038 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.738902+0000", 2024-09-14T06:29:09.038 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:09.038 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:09.038 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:09.039 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:29:09.039 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.738902+0000", 2024-09-14T06:29:09.039 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:09.039 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:09.039 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:09.039 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:29:09.039 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.738902+0000", 2024-09-14T06:29:09.039 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.1899999999999998e-07 2024-09-14T06:29:09.039 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:09.039 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:09.039 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:29:09.039 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.738904+0000", 2024-09-14T06:29:09.039 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.872e-06 2024-09-14T06:29:09.040 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:09.040 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:09.040 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:29:09.040 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.738910+0000", 2024-09-14T06:29:09.040 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.8749999999999997e-06 2024-09-14T06:29:09.040 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:09.040 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:09.040 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:29:09.040 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.738918+0000", 2024-09-14T06:29:09.040 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.2570000000000007e-06 2024-09-14T06:29:09.040 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:09.040 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:09.040 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:29:09.041 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.750331+0000", 2024-09-14T06:29:09.041 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.011412813000000001 2024-09-14T06:29:09.041 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:09.041 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:09.041 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:29:09.041 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.750373+0000", 2024-09-14T06:29:09.041 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.1754e-05 2024-09-14T06:29:09.041 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:09.042 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:09.042 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:29:09.042 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.750430+0000", 2024-09-14T06:29:09.042 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.7272000000000003e-05 2024-09-14T06:29:09.042 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:09.042 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:29:09.042 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:09.042 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:09.042 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:09.042 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:3258 3.5 3:a6acff7e:::rbd_data.112d7dfbd766.0000000000000078:5 [read 143360~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-14T06:29:09.042 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:08.738925+0000", 2024-09-14T06:29:09.042 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 0.29132493999999998, 2024-09-14T06:29:09.043 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.01161296, 2024-09-14T06:29:09.043 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:29:09.043 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:29:09.043 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:29:09.043 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:29:09.043 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:29:09.043 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:09.043 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 3258 2024-09-14T06:29:09.043 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:09.043 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:29:09.043 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:09.043 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:29:09.043 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.738925+0000", 2024-09-14T06:29:09.044 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:09.044 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:09.044 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:09.044 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:29:09.044 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.738925+0000", 2024-09-14T06:29:09.044 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:09.044 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:09.044 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:09.044 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:29:09.044 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.738926+0000", 2024-09-14T06:29:09.044 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.1100000000000001e-07 2024-09-14T06:29:09.044 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:09.044 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:09.045 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:29:09.045 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.738928+0000", 2024-09-14T06:29:09.045 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.0679999999999999e-06 2024-09-14T06:29:09.045 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:09.045 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:09.045 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:29:09.045 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.738934+0000", 2024-09-14T06:29:09.045 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.054e-06 2024-09-14T06:29:09.045 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:09.045 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:09.045 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:29:09.045 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.738938+0000", 2024-09-14T06:29:09.045 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.4429999999999996e-06 2024-09-14T06:29:09.046 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:09.046 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:09.046 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:29:09.046 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.750448+0000", 2024-09-14T06:29:09.046 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.011510016 2024-09-14T06:29:09.046 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:09.046 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:09.046 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:29:09.046 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.750488+0000", 2024-09-14T06:29:09.046 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.9402e-05 2024-09-14T06:29:09.046 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:09.046 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:09.046 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:29:09.046 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.750538+0000", 2024-09-14T06:29:09.047 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.0565999999999997e-05 2024-09-14T06:29:09.047 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:09.047 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:29:09.047 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:09.047 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:09.047 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:09.047 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:3259 3.5 3:a6acff7e:::rbd_data.112d7dfbd766.0000000000000078:5 [read 159744~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-14T06:29:09.047 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:08.738946+0000", 2024-09-14T06:29:09.047 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 0.29130392900000002, 2024-09-14T06:29:09.047 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.011711767999999999, 2024-09-14T06:29:09.047 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:29:09.047 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:29:09.048 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:29:09.048 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:29:09.048 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:29:09.048 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:09.048 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 3259 2024-09-14T06:29:09.048 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:09.048 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:29:09.048 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:09.048 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:29:09.048 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.738946+0000", 2024-09-14T06:29:09.048 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:09.048 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:09.048 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:09.048 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:29:09.049 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.738946+0000", 2024-09-14T06:29:09.049 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:09.049 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:09.049 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:09.049 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:29:09.049 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.738947+0000", 2024-09-14T06:29:09.049 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.5400000000000002e-07 2024-09-14T06:29:09.049 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:09.049 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:09.049 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:29:09.049 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.738949+0000", 2024-09-14T06:29:09.049 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.0260000000000002e-06 2024-09-14T06:29:09.049 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:09.050 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:09.050 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:29:09.050 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.738955+0000", 2024-09-14T06:29:09.050 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.3709999999999996e-06 2024-09-14T06:29:09.050 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:09.050 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:09.050 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:29:09.050 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.738960+0000", 2024-09-14T06:29:09.050 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.7369999999999997e-06 2024-09-14T06:29:09.050 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:09.050 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:09.050 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:29:09.050 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.750560+0000", 2024-09-14T06:29:09.051 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.011600103 2024-09-14T06:29:09.051 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:09.051 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:09.051 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:29:09.051 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.750605+0000", 2024-09-14T06:29:09.051 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.4799999999999998e-05 2024-09-14T06:29:09.051 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:09.051 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:09.051 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:29:09.051 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.750658+0000", 2024-09-14T06:29:09.051 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.3377000000000001e-05 2024-09-14T06:29:09.051 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:09.051 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:29:09.051 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:09.052 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:09.052 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:09.052 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:3260 3.5 3:a6acff7e:::rbd_data.112d7dfbd766.0000000000000078:5 [read 176128~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-14T06:29:09.052 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:08.738967+0000", 2024-09-14T06:29:09.052 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 0.291283128, 2024-09-14T06:29:09.052 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.011809811, 2024-09-14T06:29:09.052 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:29:09.052 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:29:09.052 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:29:09.052 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:29:09.052 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:29:09.052 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:09.053 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 3260 2024-09-14T06:29:09.053 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:09.053 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:29:09.053 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:09.053 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:29:09.053 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.738967+0000", 2024-09-14T06:29:09.053 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:09.053 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:09.053 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:09.053 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:29:09.053 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.738967+0000", 2024-09-14T06:29:09.053 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:09.053 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:09.054 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:09.054 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:29:09.054 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.738968+0000", 2024-09-14T06:29:09.054 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.2599999999999998e-07 2024-09-14T06:29:09.054 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:09.054 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:09.054 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:29:09.054 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.738970+0000", 2024-09-14T06:29:09.054 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.0729999999999999e-06 2024-09-14T06:29:09.054 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:09.054 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:09.054 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:29:09.054 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.738976+0000", 2024-09-14T06:29:09.054 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.6830000000000002e-06 2024-09-14T06:29:09.055 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:09.055 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:09.055 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:29:09.055 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.738981+0000", 2024-09-14T06:29:09.055 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.6299999999999997e-06 2024-09-14T06:29:09.055 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:09.055 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:09.055 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:29:09.055 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.750681+0000", 2024-09-14T06:29:09.055 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.011700093999999999 2024-09-14T06:29:09.055 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:09.055 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:09.055 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:29:09.056 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.750724+0000", 2024-09-14T06:29:09.056 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.295e-05 2024-09-14T06:29:09.056 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:09.056 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:09.056 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:29:09.056 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.750777+0000", 2024-09-14T06:29:09.056 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.2954999999999997e-05 2024-09-14T06:29:09.056 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:09.056 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:29:09.056 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:09.056 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:09.056 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:09.056 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:3261 3.5 3:a6acff7e:::rbd_data.112d7dfbd766.0000000000000078:5 [read 192512~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-14T06:29:09.057 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:08.738988+0000", 2024-09-14T06:29:09.057 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 0.29126275099999999, 2024-09-14T06:29:09.057 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.011907709000000001, 2024-09-14T06:29:09.057 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:29:09.057 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:29:09.057 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:29:09.057 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:29:09.057 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:29:09.057 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:09.057 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 3261 2024-09-14T06:29:09.057 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:09.057 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:29:09.057 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:09.058 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:29:09.058 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.738988+0000", 2024-09-14T06:29:09.058 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:09.058 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:09.058 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:09.058 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:29:09.058 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.738988+0000", 2024-09-14T06:29:09.058 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:09.058 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:09.058 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:09.058 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:29:09.059 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.738988+0000", 2024-09-14T06:29:09.059 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.9200000000000002e-07 2024-09-14T06:29:09.059 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:09.059 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:09.059 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:29:09.059 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.738995+0000", 2024-09-14T06:29:09.059 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.8399999999999997e-06 2024-09-14T06:29:09.059 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:09.059 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:09.059 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:29:09.059 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.739001+0000", 2024-09-14T06:29:09.060 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.2319999999999998e-06 2024-09-14T06:29:09.060 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:09.060 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:09.060 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:29:09.060 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.739007+0000", 2024-09-14T06:29:09.060 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.57e-06 2024-09-14T06:29:09.060 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:09.060 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:09.060 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:29:09.060 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.750798+0000", 2024-09-14T06:29:09.060 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.011790995 2024-09-14T06:29:09.060 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:09.060 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:09.061 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:29:09.061 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.750837+0000", 2024-09-14T06:29:09.061 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.9765999999999998e-05 2024-09-14T06:29:09.061 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:09.061 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:09.061 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:29:09.061 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.750895+0000", 2024-09-14T06:29:09.061 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.7914000000000002e-05 2024-09-14T06:29:09.061 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:09.061 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:29:09.061 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:09.061 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:09.061 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:09.061 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:3262 3.5 3:a6acff7e:::rbd_data.112d7dfbd766.0000000000000078:5 [read 208896~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-14T06:29:09.062 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:08.739013+0000", 2024-09-14T06:29:09.062 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 0.29123716199999999, 2024-09-14T06:29:09.062 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.012002934999999999, 2024-09-14T06:29:09.062 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:29:09.062 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:29:09.062 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:29:09.062 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:29:09.062 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:29:09.062 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:09.062 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 3262 2024-09-14T06:29:09.062 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:09.062 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:29:09.062 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:09.063 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:29:09.063 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.739013+0000", 2024-09-14T06:29:09.063 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:09.063 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:09.063 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:09.063 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:29:09.063 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.739013+0000", 2024-09-14T06:29:09.063 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:09.063 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:09.063 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:09.063 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:29:09.063 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.739014+0000", 2024-09-14T06:29:09.063 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.3099999999999998e-07 2024-09-14T06:29:09.064 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:09.064 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:09.064 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:29:09.064 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.739015+0000", 2024-09-14T06:29:09.064 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.818e-06 2024-09-14T06:29:09.064 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:09.064 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:09.064 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:29:09.064 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.739023+0000", 2024-09-14T06:29:09.064 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.2559999999999996e-06 2024-09-14T06:29:09.064 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:09.064 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:09.064 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:29:09.065 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.739028+0000", 2024-09-14T06:29:09.065 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.7870000000000001e-06 2024-09-14T06:29:09.065 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:09.065 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:09.065 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:29:09.065 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.750921+0000", 2024-09-14T06:29:09.065 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.011893470999999999 2024-09-14T06:29:09.065 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:09.065 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:09.065 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:29:09.065 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.750964+0000", 2024-09-14T06:29:09.065 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.3229000000000002e-05 2024-09-14T06:29:09.065 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:09.066 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:09.066 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:29:09.066 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.751016+0000", 2024-09-14T06:29:09.066 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.1943000000000001e-05 2024-09-14T06:29:09.066 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:09.066 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:29:09.066 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:09.066 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:09.066 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:09.066 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:3263 3.5 3:a6acff7e:::rbd_data.112d7dfbd766.0000000000000078:5 [read 225280~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-14T06:29:09.066 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:08.739032+0000", 2024-09-14T06:29:09.066 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 0.29121802099999999, 2024-09-14T06:29:09.067 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.012101318, 2024-09-14T06:29:09.067 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:29:09.067 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:29:09.067 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:29:09.067 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:29:09.067 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:29:09.067 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:09.067 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 3263 2024-09-14T06:29:09.067 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:09.067 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:29:09.067 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:09.067 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:29:09.067 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.739032+0000", 2024-09-14T06:29:09.067 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:09.068 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:09.068 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:09.068 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:29:09.068 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.739032+0000", 2024-09-14T06:29:09.068 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:09.068 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:09.068 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:09.068 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:29:09.068 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.739033+0000", 2024-09-14T06:29:09.068 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.5800000000000001e-07 2024-09-14T06:29:09.068 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:09.068 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:09.069 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:29:09.069 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.739034+0000", 2024-09-14T06:29:09.069 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.136e-06 2024-09-14T06:29:09.069 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:09.069 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:09.069 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:29:09.069 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.739037+0000", 2024-09-14T06:29:09.069 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.2370000000000001e-06 2024-09-14T06:29:09.069 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:09.069 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:09.069 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:29:09.069 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.739040+0000", 2024-09-14T06:29:09.070 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.3739999999999998e-06 2024-09-14T06:29:09.070 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:09.070 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:09.070 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:29:09.070 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.751036+0000", 2024-09-14T06:29:09.070 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.011995178 2024-09-14T06:29:09.070 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:09.070 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:09.070 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:29:09.070 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.751078+0000", 2024-09-14T06:29:09.071 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.2519999999999999e-05 2024-09-14T06:29:09.071 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:09.071 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:09.071 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:29:09.071 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.751134+0000", 2024-09-14T06:29:09.071 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.5615000000000002e-05 2024-09-14T06:29:09.071 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:09.071 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:29:09.071 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:09.071 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:09.071 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:09.071 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:3264 3.5 3:a6acff7e:::rbd_data.112d7dfbd766.0000000000000078:5 [read 241664~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-14T06:29:09.071 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:08.739043+0000", 2024-09-14T06:29:09.072 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 0.29120700799999999, 2024-09-14T06:29:09.072 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.012198746, 2024-09-14T06:29:09.072 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:29:09.072 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:29:09.072 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:29:09.072 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:29:09.072 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:29:09.072 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:09.072 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 3264 2024-09-14T06:29:09.072 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:09.072 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:29:09.072 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:09.072 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:29:09.073 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.739043+0000", 2024-09-14T06:29:09.073 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:09.073 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:09.073 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:09.073 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:29:09.073 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.739043+0000", 2024-09-14T06:29:09.073 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:09.073 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:09.073 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:09.073 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:29:09.073 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.739044+0000", 2024-09-14T06:29:09.073 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.28e-07 2024-09-14T06:29:09.073 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:09.073 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:09.074 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:29:09.074 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.739045+0000", 2024-09-14T06:29:09.074 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.78e-07 2024-09-14T06:29:09.074 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:09.074 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:09.074 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:29:09.074 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.739048+0000", 2024-09-14T06:29:09.074 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.9950000000000001e-06 2024-09-14T06:29:09.074 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:09.074 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:09.074 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:29:09.074 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.739051+0000", 2024-09-14T06:29:09.074 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.0340000000000001e-06 2024-09-14T06:29:09.075 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:09.075 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:09.075 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:29:09.075 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.751154+0000", 2024-09-14T06:29:09.075 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.012103541000000001 2024-09-14T06:29:09.075 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:09.075 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:09.075 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:29:09.075 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.751193+0000", 2024-09-14T06:29:09.075 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.9048000000000003e-05 2024-09-14T06:29:09.075 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:09.075 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:09.075 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:29:09.076 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.751242+0000", 2024-09-14T06:29:09.076 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.8921999999999997e-05 2024-09-14T06:29:09.076 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:09.076 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:29:09.076 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:09.076 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:09.076 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:09.076 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:3265 3.5 3:a6acff7e:::rbd_data.112d7dfbd766.0000000000000078:5 [read 258048~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-14T06:29:09.076 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:08.739053+0000", 2024-09-14T06:29:09.076 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 0.29119688799999999, 2024-09-14T06:29:09.076 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.01230612, 2024-09-14T06:29:09.076 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:29:09.076 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:29:09.076 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:29:09.077 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:29:09.077 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:29:09.077 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:09.077 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 3265 2024-09-14T06:29:09.077 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:09.077 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:29:09.077 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:09.077 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:29:09.077 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.739053+0000", 2024-09-14T06:29:09.077 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:09.077 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:09.077 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:09.077 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:29:09.077 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.739053+0000", 2024-09-14T06:29:09.078 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:09.078 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:09.078 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:09.078 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:29:09.078 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.739054+0000", 2024-09-14T06:29:09.078 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.3999999999999998e-07 2024-09-14T06:29:09.078 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:09.078 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:09.078 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:29:09.078 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.739055+0000", 2024-09-14T06:29:09.078 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.2099999999999995e-07 2024-09-14T06:29:09.078 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:09.078 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:09.078 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:29:09.078 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.739058+0000", 2024-09-14T06:29:09.079 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.9340000000000002e-06 2024-09-14T06:29:09.079 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:09.079 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:09.079 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:29:09.079 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.739060+0000", 2024-09-14T06:29:09.079 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.8030000000000001e-06 2024-09-14T06:29:09.079 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:09.079 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:09.079 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:29:09.079 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.751263+0000", 2024-09-14T06:29:09.079 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.012202821000000001 2024-09-14T06:29:09.079 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:09.079 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:09.079 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:29:09.080 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.751308+0000", 2024-09-14T06:29:09.080 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.4802999999999998e-05 2024-09-14T06:29:09.080 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:09.080 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:09.080 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:29:09.080 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.751360+0000", 2024-09-14T06:29:09.080 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.1598000000000003e-05 2024-09-14T06:29:09.080 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:09.080 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:29:09.080 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:09.080 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:09.080 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:09.080 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:3266 3.5 3:a6acff7e:::rbd_data.112d7dfbd766.0000000000000078:5 [read 274432~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-14T06:29:09.081 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:08.739063+0000", 2024-09-14T06:29:09.081 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 0.29118729799999998, 2024-09-14T06:29:09.081 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.012402339, 2024-09-14T06:29:09.081 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:29:09.081 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:29:09.081 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:29:09.081 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:29:09.081 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:29:09.081 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:09.081 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 3266 2024-09-14T06:29:09.081 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:09.081 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:29:09.081 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:09.081 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:29:09.082 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.739063+0000", 2024-09-14T06:29:09.082 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:09.082 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:09.082 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:09.082 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:29:09.082 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.739063+0000", 2024-09-14T06:29:09.082 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:09.082 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:09.082 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:09.082 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:29:09.082 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.739063+0000", 2024-09-14T06:29:09.082 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.2000000000000001e-07 2024-09-14T06:29:09.082 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:09.082 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:09.083 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:29:09.083 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.739065+0000", 2024-09-14T06:29:09.083 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.299e-06 2024-09-14T06:29:09.083 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:09.083 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:09.083 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:29:09.083 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.739069+0000", 2024-09-14T06:29:09.083 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.1309999999999999e-06 2024-09-14T06:29:09.083 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:09.083 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:09.083 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:29:09.083 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.739071+0000", 2024-09-14T06:29:09.083 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.4279999999999999e-06 2024-09-14T06:29:09.084 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:09.084 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:09.084 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:29:09.084 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.751381+0000", 2024-09-14T06:29:09.084 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.012309394 2024-09-14T06:29:09.084 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:09.084 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:09.084 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:29:09.084 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.751419+0000", 2024-09-14T06:29:09.084 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.8068000000000002e-05 2024-09-14T06:29:09.084 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:09.084 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:09.085 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:29:09.085 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.751465+0000", 2024-09-14T06:29:09.085 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.6798999999999997e-05 2024-09-14T06:29:09.085 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:09.085 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:29:09.085 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:09.085 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:09.085 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:09.085 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:3267 3.5 3:a6acff7e:::rbd_data.112d7dfbd766.0000000000000078:5 [read 290816~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-14T06:29:09.085 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:08.739074+0000", 2024-09-14T06:29:09.086 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 0.291176141, 2024-09-14T06:29:09.086 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.012508533, 2024-09-14T06:29:09.086 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:29:09.086 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:29:09.086 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:29:09.086 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:29:09.086 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:29:09.086 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:09.086 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 3267 2024-09-14T06:29:09.086 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:09.086 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:29:09.086 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:09.086 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:29:09.087 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.739074+0000", 2024-09-14T06:29:09.087 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:09.087 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:09.087 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:09.087 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:29:09.087 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.739074+0000", 2024-09-14T06:29:09.087 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:09.087 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:09.087 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:09.087 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:29:09.087 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.739074+0000", 2024-09-14T06:29:09.087 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.2000000000000001e-07 2024-09-14T06:29:09.087 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:09.088 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:09.088 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:29:09.088 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.739075+0000", 2024-09-14T06:29:09.088 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.2299999999999999e-07 2024-09-14T06:29:09.088 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:09.088 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:09.088 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:29:09.088 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.739079+0000", 2024-09-14T06:29:09.088 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.6500000000000002e-06 2024-09-14T06:29:09.088 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:09.088 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:09.088 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:29:09.088 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.739083+0000", 2024-09-14T06:29:09.089 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.856e-06 2024-09-14T06:29:09.089 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:09.089 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:09.089 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:29:09.089 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.751485+0000", 2024-09-14T06:29:09.089 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.012401772 2024-09-14T06:29:09.089 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:09.089 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:09.089 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:29:09.089 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.751529+0000", 2024-09-14T06:29:09.089 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.4542000000000001e-05 2024-09-14T06:29:09.089 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:09.089 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:09.089 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:29:09.090 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.751583+0000", 2024-09-14T06:29:09.090 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.3569999999999997e-05 2024-09-14T06:29:09.090 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:09.090 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:29:09.090 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:09.090 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:09.090 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:09.090 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:3268 3.5 3:a6acff7e:::rbd_data.112d7dfbd766.0000000000000078:5 [read 307200~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-14T06:29:09.090 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:08.739088+0000", 2024-09-14T06:29:09.090 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 0.291162113, 2024-09-14T06:29:09.090 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.012618495, 2024-09-14T06:29:09.090 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:29:09.090 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:29:09.090 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:29:09.090 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:29:09.090 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:29:09.091 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:09.091 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 3268 2024-09-14T06:29:09.091 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:09.091 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:29:09.091 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:09.091 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:29:09.091 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.739088+0000", 2024-09-14T06:29:09.091 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:09.091 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:09.091 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:09.091 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:29:09.091 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.739088+0000", 2024-09-14T06:29:09.091 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:09.092 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:09.092 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:09.092 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:29:09.092 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.739089+0000", 2024-09-14T06:29:09.092 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.4400000000000001e-07 2024-09-14T06:29:09.092 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:09.092 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:09.092 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:29:09.092 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.739090+0000", 2024-09-14T06:29:09.092 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.0669999999999999e-06 2024-09-14T06:29:09.092 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:09.092 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:09.092 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:29:09.093 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.739093+0000", 2024-09-14T06:29:09.093 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.3189999999999999e-06 2024-09-14T06:29:09.093 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:09.093 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:09.093 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:29:09.093 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.739095+0000", 2024-09-14T06:29:09.093 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.5799999999999999e-06 2024-09-14T06:29:09.093 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:09.093 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:09.093 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:29:09.093 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.751604+0000", 2024-09-14T06:29:09.093 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.012508543 2024-09-14T06:29:09.093 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:09.094 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:09.094 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:29:09.094 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.751655+0000", 2024-09-14T06:29:09.094 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.0872999999999999e-05 2024-09-14T06:29:09.094 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:09.094 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:09.094 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:29:09.094 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.751707+0000", 2024-09-14T06:29:09.094 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.1869000000000001e-05 2024-09-14T06:29:09.094 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:09.094 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:29:09.094 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:09.094 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:09.094 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:09.095 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:3269 3.5 3:a6acff7e:::rbd_data.112d7dfbd766.0000000000000078:5 [read 323584~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-14T06:29:09.095 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:08.739101+0000", 2024-09-14T06:29:09.095 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 0.29114962900000002, 2024-09-14T06:29:09.095 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.012722587, 2024-09-14T06:29:09.095 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:29:09.095 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:29:09.095 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:29:09.095 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:29:09.095 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:29:09.095 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:09.095 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 3269 2024-09-14T06:29:09.095 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:09.095 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:29:09.095 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:09.096 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:29:09.096 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.739101+0000", 2024-09-14T06:29:09.096 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:09.096 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:09.096 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:09.096 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:29:09.096 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.739101+0000", 2024-09-14T06:29:09.096 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:09.096 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:09.096 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:09.096 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:29:09.096 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.739101+0000", 2024-09-14T06:29:09.096 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.17e-07 2024-09-14T06:29:09.096 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:09.097 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:09.097 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:29:09.097 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.739102+0000", 2024-09-14T06:29:09.097 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.6899999999999996e-07 2024-09-14T06:29:09.097 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:09.097 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:09.097 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:29:09.097 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.739106+0000", 2024-09-14T06:29:09.097 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.2390000000000004e-06 2024-09-14T06:29:09.097 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:09.097 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:09.097 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:29:09.097 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.739109+0000", 2024-09-14T06:29:09.098 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.5239999999999999e-06 2024-09-14T06:29:09.098 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:09.098 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:09.098 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:29:09.098 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.751732+0000", 2024-09-14T06:29:09.098 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.012622858000000001 2024-09-14T06:29:09.098 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:09.098 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:09.098 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:29:09.098 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.751774+0000", 2024-09-14T06:29:09.098 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.2485999999999999e-05 2024-09-14T06:29:09.098 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:09.098 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:09.099 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:29:09.099 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.751823+0000", 2024-09-14T06:29:09.099 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.9293999999999999e-05 2024-09-14T06:29:09.099 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:09.099 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:29:09.099 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:09.099 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:09.099 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:09.099 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:3290 3.1d 3:bf9aaf9c:::rbd_object_map.112dcdddac74.0000000000000006:head [call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-09-14T06:29:09.099 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:08.739264+0000", 2024-09-14T06:29:09.099 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 0.29098648500000002, 2024-09-14T06:29:09.099 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.015385322, 2024-09-14T06:29:09.099 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:29:09.099 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:29:09.100 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:29:09.100 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:29:09.100 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:29:09.100 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:09.100 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 3290 2024-09-14T06:29:09.100 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:09.100 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:29:09.100 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:09.100 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:29:09.100 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.739264+0000", 2024-09-14T06:29:09.100 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:09.100 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:09.100 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:09.101 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:29:09.101 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.739264+0000", 2024-09-14T06:29:09.101 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:09.101 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:09.101 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:09.101 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:29:09.101 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.739265+0000", 2024-09-14T06:29:09.101 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.02e-07 2024-09-14T06:29:09.101 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:09.101 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:09.101 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:29:09.101 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.739269+0000", 2024-09-14T06:29:09.101 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.5990000000000004e-06 2024-09-14T06:29:09.101 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:09.102 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:09.102 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:29:09.102 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.739278+0000", 2024-09-14T06:29:09.102 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.8109999999999992e-06 2024-09-14T06:29:09.102 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:09.102 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:09.102 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:29:09.102 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.739285+0000", 2024-09-14T06:29:09.102 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.7159999999999997e-06 2024-09-14T06:29:09.102 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:09.102 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:09.102 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:29:09.102 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.751725+0000", 2024-09-14T06:29:09.102 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.012439597 2024-09-14T06:29:09.103 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:09.103 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:09.103 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:29:09.103 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.751771+0000", 2024-09-14T06:29:09.103 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.6353999999999997e-05 2024-09-14T06:29:09.103 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:09.103 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:09.103 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "waiting for subops from 4", 2024-09-14T06:29:09.103 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.751969+0000", 2024-09-14T06:29:09.103 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00019807700000000001 2024-09-14T06:29:09.103 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:09.103 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:09.103 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:29:09.103 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.753869+0000", 2024-09-14T06:29:09.104 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.001900335 2024-09-14T06:29:09.104 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:09.104 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:09.104 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "op_commit", 2024-09-14T06:29:09.104 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.754569+0000", 2024-09-14T06:29:09.104 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00069978200000000003 2024-09-14T06:29:09.104 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:09.104 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:09.104 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "commit_sent", 2024-09-14T06:29:09.104 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.754618+0000", 2024-09-14T06:29:09.104 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.9011000000000002e-05 2024-09-14T06:29:09.104 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:09.105 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:09.105 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:29:09.105 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.754649+0000", 2024-09-14T06:29:09.105 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.1137999999999999e-05 2024-09-14T06:29:09.105 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:09.105 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:29:09.105 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:09.105 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:09.105 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:09.105 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:3294 3.1d 3:bf9aaf9c:::rbd_object_map.112dcdddac74.0000000000000006:head [call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-09-14T06:29:09.105 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:08.740037+0000", 2024-09-14T06:29:09.105 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 0.29021335599999998, 2024-09-14T06:29:09.105 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.017846711000000001, 2024-09-14T06:29:09.106 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:29:09.106 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:29:09.106 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:29:09.106 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:29:09.106 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:29:09.106 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:09.106 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 3294 2024-09-14T06:29:09.106 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:09.106 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:29:09.106 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:09.106 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:29:09.106 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.740037+0000", 2024-09-14T06:29:09.106 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:09.106 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:09.107 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:09.107 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:29:09.107 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.740037+0000", 2024-09-14T06:29:09.107 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:09.107 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:09.107 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:09.107 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:29:09.107 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.740038+0000", 2024-09-14T06:29:09.107 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.1900000000000001e-07 2024-09-14T06:29:09.107 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:09.107 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:09.107 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:29:09.107 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.740043+0000", 2024-09-14T06:29:09.107 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.9649999999999999e-06 2024-09-14T06:29:09.107 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:09.108 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:09.108 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:29:09.108 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.740054+0000", 2024-09-14T06:29:09.108 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.1299e-05 2024-09-14T06:29:09.108 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:09.108 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:09.108 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:29:09.108 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.740061+0000", 2024-09-14T06:29:09.108 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.1250000000000004e-06 2024-09-14T06:29:09.108 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:09.108 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:09.108 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:29:09.108 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.752369+0000", 2024-09-14T06:29:09.108 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.012307603 2024-09-14T06:29:09.109 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:09.109 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:09.109 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:29:09.109 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.752453+0000", 2024-09-14T06:29:09.109 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.4557999999999999e-05 2024-09-14T06:29:09.109 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:09.109 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:09.109 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "waiting for subops from 4", 2024-09-14T06:29:09.109 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.752811+0000", 2024-09-14T06:29:09.109 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00035743 2024-09-14T06:29:09.109 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:09.109 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:09.109 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "op_commit", 2024-09-14T06:29:09.110 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.754655+0000", 2024-09-14T06:29:09.110 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00184362 2024-09-14T06:29:09.110 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:09.110 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:09.110 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:29:09.110 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.757784+0000", 2024-09-14T06:29:09.110 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.003129565 2024-09-14T06:29:09.110 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:09.110 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:09.110 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "commit_sent", 2024-09-14T06:29:09.110 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.757836+0000", 2024-09-14T06:29:09.110 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.1598000000000003e-05 2024-09-14T06:29:09.110 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:09.110 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:09.110 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:29:09.111 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.757884+0000", 2024-09-14T06:29:09.111 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.8028999999999997e-05 2024-09-14T06:29:09.111 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:09.111 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:29:09.111 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:09.111 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:09.111 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:09.111 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:3303 3.1d 3:bf9aaf9c:::rbd_object_map.112dcdddac74.0000000000000006:head [call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-09-14T06:29:09.111 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:08.743444+0000", 2024-09-14T06:29:09.111 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 0.28680591500000002, 2024-09-14T06:29:09.111 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.014637346000000001, 2024-09-14T06:29:09.111 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:29:09.111 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:29:09.111 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:29:09.111 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:29:09.112 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:29:09.112 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:09.112 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 3303 2024-09-14T06:29:09.112 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:09.112 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:29:09.112 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:09.112 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:29:09.112 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.743444+0000", 2024-09-14T06:29:09.112 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:09.112 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:09.112 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:09.112 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:29:09.112 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.743444+0000", 2024-09-14T06:29:09.112 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:09.113 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:09.113 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:09.113 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:29:09.113 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.743445+0000", 2024-09-14T06:29:09.113 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.540000000000001e-07 2024-09-14T06:29:09.113 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:09.113 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:09.113 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:29:09.113 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.743450+0000", 2024-09-14T06:29:09.113 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.9520000000000002e-06 2024-09-14T06:29:09.113 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:09.113 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:09.113 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:29:09.113 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.743462+0000", 2024-09-14T06:29:09.114 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.1674e-05 2024-09-14T06:29:09.114 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:09.114 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:09.114 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:29:09.114 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.743469+0000", 2024-09-14T06:29:09.114 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.4499999999999998e-06 2024-09-14T06:29:09.114 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:09.114 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:09.114 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:29:09.114 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.753195+0000", 2024-09-14T06:29:09.114 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.0097257260000000005 2024-09-14T06:29:09.114 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:09.114 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:09.114 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:29:09.115 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.753277+0000", 2024-09-14T06:29:09.115 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.1469000000000002e-05 2024-09-14T06:29:09.115 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:09.115 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:09.115 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "waiting for subops from 4", 2024-09-14T06:29:09.115 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.753482+0000", 2024-09-14T06:29:09.115 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00020495999999999999 2024-09-14T06:29:09.115 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:09.115 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:09.115 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:29:09.115 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.757930+0000", 2024-09-14T06:29:09.115 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.0044481640000000001 2024-09-14T06:29:09.115 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:09.116 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:09.116 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "op_commit", 2024-09-14T06:29:09.116 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.757992+0000", 2024-09-14T06:29:09.116 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.2648000000000001e-05 2024-09-14T06:29:09.116 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:09.116 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:09.116 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "commit_sent", 2024-09-14T06:29:09.116 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.758040+0000", 2024-09-14T06:29:09.116 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.7821000000000002e-05 2024-09-14T06:29:09.116 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:09.116 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:09.116 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:29:09.116 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.758082+0000", 2024-09-14T06:29:09.116 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.1527999999999999e-05 2024-09-14T06:29:09.116 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:09.117 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:29:09.117 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:09.117 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:09.117 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:09.117 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:3331 3.1d 3:bf9aaf9c:::rbd_object_map.112dcdddac74.0000000000000006:head [call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-09-14T06:29:09.117 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:08.745245+0000", 2024-09-14T06:29:09.117 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 0.28500563400000001, 2024-09-14T06:29:09.117 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.012916459999999999, 2024-09-14T06:29:09.117 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:29:09.117 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:29:09.117 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:29:09.117 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:29:09.117 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:29:09.118 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:09.118 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 3331 2024-09-14T06:29:09.118 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:09.118 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:29:09.118 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:09.118 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:29:09.118 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.745245+0000", 2024-09-14T06:29:09.118 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:09.118 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:09.118 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:09.118 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:29:09.118 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.745245+0000", 2024-09-14T06:29:09.118 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:09.119 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:09.119 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:09.119 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:29:09.119 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.745245+0000", 2024-09-14T06:29:09.119 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.5400000000000001e-07 2024-09-14T06:29:09.119 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:09.119 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:09.119 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:29:09.119 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.745250+0000", 2024-09-14T06:29:09.119 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.9230000000000001e-06 2024-09-14T06:29:09.119 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:09.119 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:09.119 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:29:09.119 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.745259+0000", 2024-09-14T06:29:09.120 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.5459999999999992e-06 2024-09-14T06:29:09.120 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:09.120 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:09.120 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:29:09.120 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.745264+0000", 2024-09-14T06:29:09.120 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.9609999999999996e-06 2024-09-14T06:29:09.120 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:09.120 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:09.120 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:29:09.120 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.753892+0000", 2024-09-14T06:29:09.120 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.0086279670000000003 2024-09-14T06:29:09.120 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:09.120 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:09.121 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:29:09.121 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.753948+0000", 2024-09-14T06:29:09.121 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.6036999999999999e-05 2024-09-14T06:29:09.121 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:09.121 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:09.121 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "waiting for subops from 4", 2024-09-14T06:29:09.121 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.754216+0000", 2024-09-14T06:29:09.121 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00026780199999999998 2024-09-14T06:29:09.121 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:09.121 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:09.121 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:29:09.121 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.757963+0000", 2024-09-14T06:29:09.121 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.0037473350000000001 2024-09-14T06:29:09.122 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:09.122 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:09.122 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "op_commit", 2024-09-14T06:29:09.122 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.758091+0000", 2024-09-14T06:29:09.122 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00012849000000000001 2024-09-14T06:29:09.122 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:09.122 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:09.122 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "commit_sent", 2024-09-14T06:29:09.122 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.758128+0000", 2024-09-14T06:29:09.122 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.6350999999999998e-05 2024-09-14T06:29:09.122 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:09.122 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:09.122 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:29:09.122 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.758161+0000", 2024-09-14T06:29:09.123 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.3494000000000002e-05 2024-09-14T06:29:09.123 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:09.123 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:29:09.123 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:09.123 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:09.123 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:09.123 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:3332 3.1d 3:bf9aaf9c:::rbd_object_map.112dcdddac74.0000000000000006:head [call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-09-14T06:29:09.123 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:08.746143+0000", 2024-09-14T06:29:09.123 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 0.28410756300000001, 2024-09-14T06:29:09.123 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.012083902000000001, 2024-09-14T06:29:09.123 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:29:09.123 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:29:09.123 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:29:09.123 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:29:09.124 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:29:09.124 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:09.124 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 3332 2024-09-14T06:29:09.124 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:09.124 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:29:09.124 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:09.124 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:29:09.124 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.746143+0000", 2024-09-14T06:29:09.124 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:09.124 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:09.124 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:09.124 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:29:09.124 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.746143+0000", 2024-09-14T06:29:09.125 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:09.125 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:09.125 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:09.125 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:29:09.125 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.746144+0000", 2024-09-14T06:29:09.125 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.5499999999999997e-07 2024-09-14T06:29:09.125 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:09.125 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:09.125 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:29:09.125 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.746149+0000", 2024-09-14T06:29:09.125 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.2039999999999996e-06 2024-09-14T06:29:09.125 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:09.125 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:09.126 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:29:09.126 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.746168+0000", 2024-09-14T06:29:09.126 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.8678999999999999e-05 2024-09-14T06:29:09.126 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:09.126 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:09.126 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:29:09.126 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.746176+0000", 2024-09-14T06:29:09.126 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.0439999999999996e-06 2024-09-14T06:29:09.126 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:09.126 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:09.126 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:29:09.126 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.754692+0000", 2024-09-14T06:29:09.126 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.0085166459999999992 2024-09-14T06:29:09.126 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:09.127 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:09.127 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:29:09.127 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.754746+0000", 2024-09-14T06:29:09.127 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.3375000000000003e-05 2024-09-14T06:29:09.127 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:09.127 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:09.127 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "waiting for subops from 4", 2024-09-14T06:29:09.127 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.754972+0000", 2024-09-14T06:29:09.127 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.000226586 2024-09-14T06:29:09.127 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:09.127 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:09.127 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:29:09.127 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.757985+0000", 2024-09-14T06:29:09.127 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.003012696 2024-09-14T06:29:09.128 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:09.128 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:09.128 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "op_commit", 2024-09-14T06:29:09.128 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.758171+0000", 2024-09-14T06:29:09.128 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00018605800000000001 2024-09-14T06:29:09.128 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:09.128 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:09.128 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "commit_sent", 2024-09-14T06:29:09.128 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.758195+0000", 2024-09-14T06:29:09.129 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.3591000000000002e-05 2024-09-14T06:29:09.129 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:09.129 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:09.129 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:29:09.129 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.758227+0000", 2024-09-14T06:29:09.129 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.2168000000000001e-05 2024-09-14T06:29:09.129 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:09.129 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:29:09.129 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:09.129 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:09.129 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:09.129 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:3334 3.1d 3:bf9aaf9c:::rbd_object_map.112dcdddac74.0000000000000006:head [call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-09-14T06:29:09.129 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:08.747923+0000", 2024-09-14T06:29:09.130 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 0.28232738899999998, 2024-09-14T06:29:09.130 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.012298584, 2024-09-14T06:29:09.130 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:29:09.130 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:29:09.130 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:29:09.130 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:29:09.130 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:29:09.130 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:09.130 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 3334 2024-09-14T06:29:09.130 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:09.130 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:29:09.130 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:09.130 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:29:09.130 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.747923+0000", 2024-09-14T06:29:09.131 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:09.131 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:09.131 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:09.131 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:29:09.131 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.747923+0000", 2024-09-14T06:29:09.131 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:09.131 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:09.131 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:09.131 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:29:09.131 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.747924+0000", 2024-09-14T06:29:09.131 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.2500000000000004e-07 2024-09-14T06:29:09.131 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:09.131 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:09.131 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:29:09.131 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.747929+0000", 2024-09-14T06:29:09.132 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.6909999999999996e-06 2024-09-14T06:29:09.132 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:09.132 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:09.132 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:29:09.132 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.747946+0000", 2024-09-14T06:29:09.132 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.7337e-05 2024-09-14T06:29:09.132 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:09.132 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:09.132 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:29:09.132 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.747954+0000", 2024-09-14T06:29:09.132 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.9959999999999993e-06 2024-09-14T06:29:09.132 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:09.132 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:09.132 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:29:09.132 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.756177+0000", 2024-09-14T06:29:09.133 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.0082233319999999999 2024-09-14T06:29:09.133 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:09.133 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:09.133 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:29:09.133 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.756266+0000", 2024-09-14T06:29:09.133 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.8252999999999997e-05 2024-09-14T06:29:09.133 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:09.133 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:09.133 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "waiting for subops from 4", 2024-09-14T06:29:09.133 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.756514+0000", 2024-09-14T06:29:09.133 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00024885599999999999 2024-09-14T06:29:09.133 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:09.133 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:09.134 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:29:09.134 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.759928+0000", 2024-09-14T06:29:09.134 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.0034139410000000002 2024-09-14T06:29:09.134 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:09.134 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:09.134 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "op_commit", 2024-09-14T06:29:09.134 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.760138+0000", 2024-09-14T06:29:09.134 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.000209545 2024-09-14T06:29:09.134 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:09.134 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:09.134 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "commit_sent", 2024-09-14T06:29:09.134 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.760180+0000", 2024-09-14T06:29:09.134 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.2212999999999997e-05 2024-09-14T06:29:09.135 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:09.135 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:09.135 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:29:09.135 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.760222+0000", 2024-09-14T06:29:09.135 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.1495000000000002e-05 2024-09-14T06:29:09.135 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:09.135 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:29:09.135 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:09.135 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:09.135 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:29:09.135 INFO:teuthology.orchestra.run.smithi138.stdout:} 2024-09-14T06:29:09.136 INFO:teuthology.orchestra.run.smithi094.stdout:osd.2: {} 2024-09-14T06:29:09.137 INFO:teuthology.orchestra.run.smithi094.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-14T06:29:09.137 INFO:teuthology.orchestra.run.smithi094.stdout:osd.3: {} 2024-09-14T06:29:09.138 INFO:teuthology.orchestra.run.smithi094.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-14T06:29:09.138 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-14T06:29:09.138 INFO:tasks.ceph.osd.5.smithi138.stderr:2024-09-14T06:29:09.135+0000 7fe13746f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33618) UID: 0 2024-09-14T06:29:09.139 INFO:teuthology.orchestra.run.smithi094.stdout:osd.4: {} 2024-09-14T06:29:09.139 INFO:teuthology.orchestra.run.smithi094.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-14T06:29:09.139 INFO:teuthology.orchestra.run.smithi094.stdout:osd.5: {} 2024-09-14T06:29:09.139 INFO:teuthology.orchestra.run.smithi094.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-14T06:29:09.140 INFO:teuthology.orchestra.run.smithi094.stdout:osd.6: {} 2024-09-14T06:29:09.140 INFO:teuthology.orchestra.run.smithi094.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-14T06:29:09.140 INFO:teuthology.orchestra.run.smithi094.stdout:osd.7: {} 2024-09-14T06:29:09.141 INFO:teuthology.orchestra.run.smithi094.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-14T06:29:09.238 INFO:tasks.ceph.osd.2.smithi094.stderr:2024-09-14T06:29:09.237+0000 7faf3269b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33649) UID: 0 2024-09-14T06:29:09.283 INFO:teuthology.orchestra.run.smithi138.stdout:ERROR: (22) Invalid argument 2024-09-14T06:29:09.283 INFO:teuthology.orchestra.run.smithi138.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:29:09.292 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-14T06:29:09.338 INFO:tasks.ceph.osd.2.smithi094.stderr:2024-09-14T06:29:09.338+0000 7faf3269b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33649) UID: 0 2024-09-14T06:29:09.393 INFO:teuthology.orchestra.run.smithi138.stdout:ERROR: (22) Invalid argument 2024-09-14T06:29:09.393 INFO:teuthology.orchestra.run.smithi138.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:29:09.401 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-14T06:29:09.439 INFO:tasks.ceph.osd.0.smithi094.stderr:2024-09-14T06:29:09.438+0000 7f07c9333640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33636) UID: 0 2024-09-14T06:29:09.503 INFO:teuthology.orchestra.run.smithi138.stdout:ERROR: (22) Invalid argument 2024-09-14T06:29:09.504 INFO:teuthology.orchestra.run.smithi138.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:29:09.512 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-14T06:29:09.555 INFO:tasks.ceph.osd.5.smithi138.stderr:2024-09-14T06:29:09.554+0000 7fe13746f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33618) UID: 0 2024-09-14T06:29:09.606 INFO:teuthology.orchestra.run.smithi094.stderr:nodeep-scrub is unset 2024-09-14T06:29:09.617 INFO:teuthology.orchestra.run.smithi138.stdout:ERROR: (22) Invalid argument 2024-09-14T06:29:09.617 INFO:teuthology.orchestra.run.smithi138.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:29:09.626 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-14T06:29:09.667 INFO:tasks.ceph.osd.7.smithi138.stderr:2024-09-14T06:29:09.666+0000 7f3c6a21d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33623) UID: 0 2024-09-14T06:29:09.740 INFO:tasks.ceph.osd.4.smithi138.stderr:2024-09-14T06:29:09.739+0000 7f6c653c8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33615) UID: 0 2024-09-14T06:29:09.753 INFO:teuthology.orchestra.run.smithi094.stdout:checking clone #0, image image_client.0 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent1 2024-09-14T06:29:09.771 INFO:teuthology.orchestra.run.smithi138.stdout:ERROR: (22) Invalid argument 2024-09-14T06:29:09.771 INFO:teuthology.orchestra.run.smithi138.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:29:09.779 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-14T06:29:09.841 INFO:tasks.ceph.osd.0.smithi094.stderr:2024-09-14T06:29:09.840+0000 7f07c9333640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33636) UID: 0 2024-09-14T06:29:09.922 INFO:teuthology.orchestra.run.smithi138.stdout:ERROR: (22) Invalid argument 2024-09-14T06:29:09.922 INFO:teuthology.orchestra.run.smithi138.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:29:09.930 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-14T06:29:09.942 INFO:tasks.ceph.osd.4.smithi138.stderr:2024-09-14T06:29:09.941+0000 7f6c653c8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33615) UID: 0 2024-09-14T06:29:09.993 INFO:teuthology.orchestra.run.smithi094.stdout:208 trunc from 0xe237427 to 0x48c3e9 2024-09-14T06:29:10.041 INFO:teuthology.orchestra.run.smithi094.stdout:209 writesame 0xbf28815 thru 0xbf2facf data_size 0x1(0x72bb bytes) 2024-09-14T06:29:10.041 INFO:teuthology.orchestra.run.smithi094.stdout:210 write 0x8295740 thru 0x829a9e4 (0x52a5 bytes) 2024-09-14T06:29:10.042 INFO:tasks.ceph.osd.4.smithi138.stderr:2024-09-14T06:29:10.041+0000 7f6c653c8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33615) UID: 0 2024-09-14T06:29:10.050 INFO:teuthology.orchestra.run.smithi094.stdout:ERROR: (22) Invalid argument 2024-09-14T06:29:10.050 INFO:teuthology.orchestra.run.smithi094.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:29:10.060 INFO:teuthology.orchestra.run.smithi094.stdout:211 trunc from 0xbf2fad0 to 0x1d33bfc 2024-09-14T06:29:10.069 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-14T06:29:10.070 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-14T06:29:10.078 INFO:teuthology.orchestra.run.smithi094.stdout:213 write 0x44d79e8 thru 0x44e0244 (0x885d bytes) 2024-09-14T06:29:10.086 INFO:teuthology.orchestra.run.smithi094.stdout:214 writesame 0x2245924 thru 0x22552bd data_size 0x2b(0xf99a bytes) 2024-09-14T06:29:10.102 INFO:teuthology.orchestra.run.smithi094.stdout:216 trunc from 0x44e0245 to 0x92029f0 2024-09-14T06:29:10.108 INFO:teuthology.orchestra.run.smithi094.stdout:218 read 0x1433f62 thru 0x1440c0b (0xccaa bytes) 2024-09-14T06:29:10.108 INFO:teuthology.orchestra.run.smithi094.stdout:219 read 0x450eaa3 thru 0x4513744 (0x4ca2 bytes) 2024-09-14T06:29:10.111 INFO:teuthology.orchestra.run.smithi094.stdout:220 trunc from 0x92029f0 to 0x6de6c37 2024-09-14T06:29:10.116 INFO:teuthology.orchestra.run.smithi094.stdout:222 write 0x6813618 thru 0x6813b03 (0x4ec bytes) 2024-09-14T06:29:10.131 INFO:teuthology.orchestra.run.smithi094.stdout:225 write 0xce7857d thru 0xce86bcc (0xe650 bytes) 2024-09-14T06:29:10.137 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-14T06:29:10.137 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-14T06:29:10.137 INFO:teuthology.orchestra.run.smithi094.stdout:232 read 0x804aa15 thru 0x804b1ac (0x798 bytes) 2024-09-14T06:29:10.137 INFO:teuthology.orchestra.run.smithi094.stdout:233 read 0x8a1c4de thru 0x8a1c7f4 (0x317 bytes) 2024-09-14T06:29:10.137 INFO:teuthology.orchestra.run.smithi094.stdout:234 read 0x338395d thru 0x338a026 (0x66ca bytes) 2024-09-14T06:29:10.137 INFO:teuthology.orchestra.run.smithi094.stdout:235 write 0x879010b thru 0x879f324 (0xf21a bytes) 2024-09-14T06:29:10.143 INFO:tasks.ceph.osd.4.smithi138.stderr:2024-09-14T06:29:10.142+0000 7f6c653c8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33615) UID: 0 2024-09-14T06:29:10.145 INFO:teuthology.orchestra.run.smithi094.stdout:236 read 0x7309c96 thru 0x730a720 (0xa8b bytes) 2024-09-14T06:29:10.146 INFO:teuthology.orchestra.run.smithi094.stdout:237 punch from 0x8c5ea99 to 0x8c676e8, (0x8c4f bytes) 2024-09-14T06:29:10.146 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-14T06:29:10.146 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-14T06:29:10.146 INFO:teuthology.orchestra.run.smithi094.stdout:241 writesame 0x8c28861 thru 0x8c37d49 data_size 0x1(0xf4e9 bytes) 2024-09-14T06:29:10.162 INFO:teuthology.orchestra.run.smithi094.stdout:242 write 0xa00a1a3 thru 0xa014b31 (0xa98f bytes) 2024-09-14T06:29:10.172 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-14T06:29:10.178 INFO:teuthology.orchestra.run.smithi094.stdout:244 write 0xd9d3abd thru 0xd9d6b09 (0x304d bytes) 2024-09-14T06:29:10.188 INFO:teuthology.orchestra.run.smithi094.stdout:ERROR: (22) Invalid argument 2024-09-14T06:29:10.188 INFO:teuthology.orchestra.run.smithi094.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:29:10.188 INFO:teuthology.orchestra.run.smithi094.stdout:245 writesame 0xcb32974 thru 0xcb3f8e9 data_size 0x5e(0xcf76 bytes) 2024-09-14T06:29:10.199 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-14T06:29:10.200 INFO:teuthology.orchestra.run.smithi094.stdout:246 punch from 0x27af339 to 0x27bd606, (0xe2cd bytes) 2024-09-14T06:29:10.200 INFO:teuthology.orchestra.run.smithi094.stdout:249 punch from 0x92ce0a9 to 0x92d0b8f, (0x2ae6 bytes) 2024-09-14T06:29:10.200 INFO:teuthology.orchestra.run.smithi094.stdout:250 read 0x404f20b thru 0x405e35f (0xf155 bytes) 2024-09-14T06:29:10.205 INFO:teuthology.orchestra.run.smithi094.stdout:254 write 0xebe56a8 thru 0xebf22e5 (0xcc3e bytes) 2024-09-14T06:29:10.214 INFO:teuthology.orchestra.run.smithi094.stdout:255 write 0xe786566 thru 0xe78a07a (0x3b15 bytes) 2024-09-14T06:29:10.223 INFO:teuthology.orchestra.run.smithi094.stdout:256 read 0x54bb624 thru 0x54c11fd (0x5bda bytes) 2024-09-14T06:29:10.223 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-14T06:29:10.223 INFO:teuthology.orchestra.run.smithi094.stdout:258 write 0xe592160 thru 0xe599616 (0x74b7 bytes) 2024-09-14T06:29:10.227 INFO:teuthology.orchestra.run.smithi094.stdout:259 punch from 0x54b6e7e to 0x54c2a01, (0xbb83 bytes) 2024-09-14T06:29:10.227 INFO:teuthology.orchestra.run.smithi094.stdout:260 writesame 0xd619a3e thru 0xd61e81e data_size 0x1(0x4de1 bytes) 2024-09-14T06:29:10.235 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-14T06:29:10.236 INFO:teuthology.orchestra.run.smithi094.stdout:262 read 0xdfaf33c thru 0xdfba791 (0xb456 bytes) 2024-09-14T06:29:10.236 INFO:teuthology.orchestra.run.smithi094.stdout:263 read 0x771754d thru 0x77267cf (0xf283 bytes) 2024-09-14T06:29:10.236 INFO:teuthology.orchestra.run.smithi094.stdout:267 punch from 0x58aaafa to 0x58aad56, (0x25c bytes) 2024-09-14T06:29:10.236 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-14T06:29:10.241 INFO:teuthology.orchestra.run.smithi094.stdout:273 trunc from 0xebf22e6 to 0x9991ed2 2024-09-14T06:29:10.244 INFO:tasks.ceph.osd.2.smithi094.stderr:2024-09-14T06:29:10.243+0000 7faf3269b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33649) UID: 0 2024-09-14T06:29:10.261 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-14T06:29:10.261 INFO:teuthology.orchestra.run.smithi094.stdout:275 trunc from 0x9991ed2 to 0x9a49a2a 2024-09-14T06:29:10.276 INFO:teuthology.orchestra.run.smithi094.stdout:276 write 0xdfa01f8 thru 0xdfa3087 (0x2e90 bytes) 2024-09-14T06:29:10.281 INFO:teuthology.orchestra.run.smithi094.stdout:278 write 0x1fc2555 thru 0x1fcf87b (0xd327 bytes) 2024-09-14T06:29:10.286 INFO:teuthology.orchestra.run.smithi094.stdout:279 read 0x1844c04 thru 0x184a511 (0x590e bytes) 2024-09-14T06:29:10.286 INFO:teuthology.orchestra.run.smithi094.stdout:280 write 0x4bd3482 thru 0x4bd36b4 (0x233 bytes) 2024-09-14T06:29:10.299 INFO:teuthology.orchestra.run.smithi094.stdout:281 trunc from 0xdfa3088 to 0x4d881d2 2024-09-14T06:29:10.312 INFO:teuthology.orchestra.run.smithi094.stdout:282 read 0x1dfe40d thru 0x1e09451 (0xb045 bytes) 2024-09-14T06:29:10.318 INFO:teuthology.orchestra.run.smithi094.stdout:283 trunc from 0x4d881d2 to 0x9dd096d 2024-09-14T06:29:10.323 INFO:teuthology.orchestra.run.smithi094.stdout:284 read 0x772cea8 thru 0x77395c4 (0xc71d bytes) 2024-09-14T06:29:10.324 INFO:teuthology.orchestra.run.smithi094.stdout:285 read 0x85e1138 thru 0x85e1316 (0x1df bytes) 2024-09-14T06:29:10.324 INFO:teuthology.orchestra.run.smithi094.stdout:287 write 0x679d9de thru 0x67a5b13 (0x8136 bytes) 2024-09-14T06:29:10.327 INFO:teuthology.orchestra.run.smithi094.stdout:ERROR: (22) Invalid argument 2024-09-14T06:29:10.327 INFO:teuthology.orchestra.run.smithi094.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:29:10.329 INFO:teuthology.orchestra.run.smithi094.stdout:288 punch from 0x6ad46f6 to 0x6adebec, (0xa4f6 bytes) 2024-09-14T06:29:10.329 INFO:teuthology.orchestra.run.smithi094.stdout:290 punch from 0x3ccb630 to 0x3cd3940, (0x8310 bytes) 2024-09-14T06:29:10.329 INFO:teuthology.orchestra.run.smithi094.stdout:291 write 0x1d1d071 thru 0x1d1dcf2 (0xc82 bytes) 2024-09-14T06:29:10.339 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-14T06:29:10.340 INFO:teuthology.orchestra.run.smithi094.stdout:292 trunc from 0x9dd096d to 0x1942957 2024-09-14T06:29:10.351 INFO:teuthology.orchestra.run.smithi094.stdout:294 punch from 0xf360ed to 0xf3999f, (0x38b2 bytes) 2024-09-14T06:29:10.351 INFO:teuthology.orchestra.run.smithi094.stdout:296 read 0x8887f3 thru 0x8973b0 (0xebbe bytes) 2024-09-14T06:29:10.352 INFO:tasks.ceph.osd.2.smithi094.stderr:2024-09-14T06:29:10.350+0000 7faf3269b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33649) UID: 0 2024-09-14T06:29:10.359 INFO:teuthology.orchestra.run.smithi094.stdout:297 write 0x41e5f49 thru 0x41ec3b7 (0x646f bytes) 2024-09-14T06:29:10.369 INFO:teuthology.orchestra.run.smithi094.stdout:299 trunc from 0x41ec3b8 to 0x7cad739 2024-09-14T06:29:10.374 INFO:teuthology.orchestra.run.smithi094.stdout:301 write 0x72abb23 thru 0x72ad1c7 (0x16a5 bytes) 2024-09-14T06:29:10.393 INFO:teuthology.orchestra.run.smithi094.stdout:302 write 0xec45050 thru 0xec4861e (0x35cf bytes) 2024-09-14T06:29:10.401 INFO:teuthology.orchestra.run.smithi094.stdout:303 write 0xb97af41 thru 0xb986374 (0xb434 bytes) 2024-09-14T06:29:10.408 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-14T06:29:10.408 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-14T06:29:10.408 INFO:teuthology.orchestra.run.smithi094.stdout:306 read 0x4752ae4 thru 0x475fad9 (0xcff6 bytes) 2024-09-14T06:29:10.408 INFO:teuthology.orchestra.run.smithi094.stdout:307 write 0xcf833d8 thru 0xcf84e29 (0x1a52 bytes) 2024-09-14T06:29:10.414 INFO:teuthology.orchestra.run.smithi094.stdout:309 write 0x7606a19 thru 0x76072a2 (0x88a bytes) 2024-09-14T06:29:10.417 INFO:teuthology.orchestra.run.smithi094.stdout:310 read 0x3426f94 thru 0x342e1a4 (0x7211 bytes) 2024-09-14T06:29:10.417 INFO:teuthology.orchestra.run.smithi094.stdout:311 write 0xa4ca2f6 thru 0xa4d200b (0x7d16 bytes) 2024-09-14T06:29:10.439 INFO:teuthology.orchestra.run.smithi094.stdout:313 trunc from 0xec4861f to 0x62ed48f 2024-09-14T06:29:10.445 INFO:tasks.ceph.osd.1.smithi094.stderr:2024-09-14T06:29:10.444+0000 7f328e47a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33641) UID: 0 2024-09-14T06:29:10.459 INFO:teuthology.orchestra.run.smithi094.stdout:314 read 0x3ecd0a5 thru 0x3ed72ed (0xa249 bytes) 2024-09-14T06:29:10.470 INFO:teuthology.orchestra.run.smithi094.stdout:ERROR: (22) Invalid argument 2024-09-14T06:29:10.470 INFO:teuthology.orchestra.run.smithi094.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:29:10.472 INFO:teuthology.orchestra.run.smithi094.stdout:315 write 0xca26d3d thru 0xca29a78 (0x2d3c bytes) 2024-09-14T06:29:10.475 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-14T06:29:10.475 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-14T06:29:10.475 INFO:tasks.thrashosds.thrasher:inject_pause on osd.4 2024-09-14T06:29:10.475 INFO:tasks.thrashosds.thrasher:Testing filestore_inject_stall pause injection for duration 3 2024-09-14T06:29:10.475 INFO:tasks.thrashosds.thrasher:Checking after 0, should_be_down=False 2024-09-14T06:29:10.475 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok config set filestore_inject_stall 3 2024-09-14T06:29:10.480 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-14T06:29:10.487 INFO:teuthology.orchestra.run.smithi094.stdout:316 read 0x43aabd thru 0x43ede5 (0x4329 bytes) 2024-09-14T06:29:10.492 INFO:teuthology.orchestra.run.smithi094.stdout:318 trunc from 0xca29a79 to 0x8857c8e 2024-09-14T06:29:10.505 INFO:teuthology.orchestra.run.smithi094.stdout:319 read 0x6544f7f thru 0x6549e2d (0x4eaf bytes) 2024-09-14T06:29:10.506 INFO:teuthology.orchestra.run.smithi094.stdout:320 writesame 0x86af754 thru 0x86b35df data_size 0x2(0x3e8c bytes) 2024-09-14T06:29:10.512 INFO:teuthology.orchestra.run.smithi094.stdout:321 punch from 0x4ee4dfb to 0x4eedc19, (0x8e1e bytes) 2024-09-14T06:29:10.512 INFO:teuthology.orchestra.run.smithi094.stdout:323 writesame 0x3b582cb thru 0x3b5bdf7 data_size 0x1(0x3b2d bytes) 2024-09-14T06:29:10.528 INFO:teuthology.orchestra.run.smithi094.stdout:325 trunc from 0x8857c8e to 0x3fb6ed 2024-09-14T06:29:10.546 INFO:teuthology.orchestra.run.smithi094.stdout:326 punch from 0x1136e3 to 0x1141f6, (0xb13 bytes) 2024-09-14T06:29:10.546 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-14T06:29:10.546 INFO:teuthology.orchestra.run.smithi094.stdout:328 read 0x36f951 thru 0x374e15 (0x54c5 bytes) 2024-09-14T06:29:10.546 INFO:tasks.ceph.osd.7.smithi138.stderr:2024-09-14T06:29:10.545+0000 7f3c6a21d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33623) UID: 0 2024-09-14T06:29:10.550 INFO:teuthology.orchestra.run.smithi094.stdout:329 trunc from 0x3fb6ed to 0x3614fb6 2024-09-14T06:29:10.565 INFO:teuthology.orchestra.run.smithi094.stdout:330 write 0x7d119fb thru 0x7d1af83 (0x9589 bytes) 2024-09-14T06:29:10.572 INFO:teuthology.orchestra.run.smithi094.stdout:332 read 0x51820a0 thru 0x5183628 (0x1589 bytes) 2024-09-14T06:29:10.572 INFO:teuthology.orchestra.run.smithi094.stdout:333 read 0xf628b7 thru 0xf6a9ad (0x80f7 bytes) 2024-09-14T06:29:10.572 INFO:teuthology.orchestra.run.smithi094.stdout:334 write 0x29c0500 thru 0x29c4013 (0x3b14 bytes) 2024-09-14T06:29:10.582 INFO:teuthology.orchestra.run.smithi094.stdout:335 clone 3 order 22 su 2097152 sc 15 2024-09-14T06:29:10.589 INFO:teuthology.orchestra.run.smithi138.stdout:{ 2024-09-14T06:29:10.589 INFO:teuthology.orchestra.run.smithi138.stdout: "success": "filestore_inject_stall = '' (not observed, change may require restart) " 2024-09-14T06:29:10.589 INFO:teuthology.orchestra.run.smithi138.stdout:} 2024-09-14T06:29:10.605 INFO:teuthology.orchestra.run.smithi094.stdout:ERROR: (22) Invalid argument 2024-09-14T06:29:10.605 INFO:teuthology.orchestra.run.smithi094.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:29:10.615 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-14T06:29:10.656 INFO:tasks.ceph.osd.2.smithi094.stderr:2024-09-14T06:29:10.655+0000 7faf3269b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33649) UID: 0 2024-09-14T06:29:10.748 INFO:tasks.ceph.osd.0.smithi094.stderr:2024-09-14T06:29:10.747+0000 7f07c9333640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33636) UID: 0 2024-09-14T06:29:10.758 INFO:teuthology.orchestra.run.smithi094.stdout:ERROR: (22) Invalid argument 2024-09-14T06:29:10.758 INFO:teuthology.orchestra.run.smithi094.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:29:10.767 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-14T06:29:10.850 INFO:tasks.ceph.osd.4.smithi138.stderr:2024-09-14T06:29:10.849+0000 7f6c653c8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33615) UID: 0 2024-09-14T06:29:10.911 INFO:teuthology.orchestra.run.smithi094.stdout:ERROR: (22) Invalid argument 2024-09-14T06:29:10.911 INFO:teuthology.orchestra.run.smithi094.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:29:10.919 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-14T06:29:10.951 INFO:tasks.ceph.osd.7.smithi138.stderr:2024-09-14T06:29:10.950+0000 7f3c6a21d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33623) UID: 0 2024-09-14T06:29:11.021 INFO:teuthology.orchestra.run.smithi094.stdout:ERROR: (22) Invalid argument 2024-09-14T06:29:11.021 INFO:teuthology.orchestra.run.smithi094.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:29:11.030 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-14T06:29:11.075 INFO:tasks.ceph.osd.0.smithi094.stderr:2024-09-14T06:29:11.074+0000 7f07c9333640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33636) UID: 0 2024-09-14T06:29:11.134 INFO:teuthology.orchestra.run.smithi094.stdout:ERROR: (22) Invalid argument 2024-09-14T06:29:11.134 INFO:teuthology.orchestra.run.smithi094.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:29:11.142 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-14T06:29:11.142 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-14T06:29:11.183 INFO:tasks.ceph.osd.3.smithi094.stderr:2024-09-14T06:29:11.182+0000 7feeeafe8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33655) UID: 0 2024-09-14T06:29:11.253 INFO:tasks.ceph.osd.1.smithi094.stderr:2024-09-14T06:29:11.252+0000 7f328e47a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33641) UID: 0 2024-09-14T06:29:11.289 INFO:teuthology.orchestra.run.smithi094.stdout:ERROR: (22) Invalid argument 2024-09-14T06:29:11.289 INFO:teuthology.orchestra.run.smithi094.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:29:11.298 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-14T06:29:11.304 INFO:teuthology.orchestra.run.smithi094.stdout:osd.0: {} 2024-09-14T06:29:11.305 INFO:teuthology.orchestra.run.smithi094.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-14T06:29:11.322 INFO:teuthology.orchestra.run.smithi094.stdout:osd.1: {} 2024-09-14T06:29:11.322 INFO:teuthology.orchestra.run.smithi094.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-14T06:29:11.338 INFO:teuthology.orchestra.run.smithi094.stdout:osd.2: {} 2024-09-14T06:29:11.338 INFO:teuthology.orchestra.run.smithi094.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-14T06:29:11.353 INFO:tasks.ceph.osd.0.smithi094.stderr:2024-09-14T06:29:11.353+0000 7f07c9333640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33636) UID: 0 2024-09-14T06:29:11.354 INFO:teuthology.orchestra.run.smithi094.stdout:osd.3: {} 2024-09-14T06:29:11.355 INFO:teuthology.orchestra.run.smithi094.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-14T06:29:11.372 INFO:teuthology.orchestra.run.smithi094.stdout:osd.4: {} 2024-09-14T06:29:11.372 INFO:teuthology.orchestra.run.smithi094.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-14T06:29:11.389 INFO:teuthology.orchestra.run.smithi094.stdout:osd.5: {} 2024-09-14T06:29:11.389 INFO:teuthology.orchestra.run.smithi094.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-14T06:29:11.406 INFO:teuthology.orchestra.run.smithi094.stdout:osd.6: {} 2024-09-14T06:29:11.406 INFO:teuthology.orchestra.run.smithi094.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-14T06:29:11.409 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-14T06:29:11.410 INFO:teuthology.orchestra.run.smithi094.stdout: "ops": [], 2024-09-14T06:29:11.410 INFO:teuthology.orchestra.run.smithi094.stdout: "complaint_time": 30, 2024-09-14T06:29:11.410 INFO:teuthology.orchestra.run.smithi094.stdout: "num_blocked_ops": 0 2024-09-14T06:29:11.410 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-14T06:29:11.419 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-14T06:29:11.424 INFO:teuthology.orchestra.run.smithi094.stdout:osd.7: {} 2024-09-14T06:29:11.424 INFO:teuthology.orchestra.run.smithi094.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-14T06:29:11.455 INFO:tasks.ceph.osd.6.smithi138.stderr:2024-09-14T06:29:11.454+0000 7f8cd8849640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33620) UID: 0 2024-09-14T06:29:11.527 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-14T06:29:11.528 INFO:teuthology.orchestra.run.smithi094.stdout: "size": 20, 2024-09-14T06:29:11.528 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 600, 2024-09-14T06:29:11.528 INFO:teuthology.orchestra.run.smithi094.stdout: "ops": [ 2024-09-14T06:29:11.528 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:11.528 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:936 3.8 3:17df59d8:::rbd_data.112d7dfbd766.0000000000000011:head [set-alloc-hint object_size 524288 write_size 524288,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-14T06:29:11.528 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:07.164453+0000", 2024-09-14T06:29:11.528 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 4.3612620639999999, 2024-09-14T06:29:11.528 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.013369270000000001, 2024-09-14T06:29:11.528 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:29:11.528 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:29:11.529 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:29:11.529 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:29:11.529 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:29:11.529 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:11.529 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 936 2024-09-14T06:29:11.529 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:11.529 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:29:11.529 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:11.529 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:29:11.529 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:07.164453+0000", 2024-09-14T06:29:11.529 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:11.529 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:11.529 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:11.530 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:29:11.530 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:07.164453+0000", 2024-09-14T06:29:11.530 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:11.530 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:11.530 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:11.530 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:29:11.530 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:07.164454+0000", 2024-09-14T06:29:11.530 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.8190000000000001e-06 2024-09-14T06:29:11.530 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:11.530 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:11.530 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:29:11.530 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:07.164463+0000", 2024-09-14T06:29:11.530 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.9900000000000003e-06 2024-09-14T06:29:11.530 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:11.530 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:11.531 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:29:11.531 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:07.164479+0000", 2024-09-14T06:29:11.531 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.5654e-05 2024-09-14T06:29:11.531 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:11.531 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:11.531 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:29:11.531 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:07.164487+0000", 2024-09-14T06:29:11.531 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.435e-06 2024-09-14T06:29:11.531 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:11.531 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:11.531 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:29:11.531 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:07.164537+0000", 2024-09-14T06:29:11.531 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.0175e-05 2024-09-14T06:29:11.531 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:11.531 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:11.532 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:29:11.532 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:07.164626+0000", 2024-09-14T06:29:11.532 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.9272999999999997e-05 2024-09-14T06:29:11.532 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:11.532 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:11.532 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "waiting for subops from 1", 2024-09-14T06:29:11.532 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:07.167461+0000", 2024-09-14T06:29:11.532 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0028350369999999999 2024-09-14T06:29:11.532 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:11.532 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:11.532 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "op_commit", 2024-09-14T06:29:11.532 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:07.177674+0000", 2024-09-14T06:29:11.532 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.01021267 2024-09-14T06:29:11.532 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:11.532 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:11.533 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:29:11.533 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:07.177728+0000", 2024-09-14T06:29:11.533 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.3829000000000002e-05 2024-09-14T06:29:11.533 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:11.533 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:11.533 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-14T06:29:11.533 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:07.177766+0000", 2024-09-14T06:29:11.533 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.8130000000000003e-05 2024-09-14T06:29:11.533 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:11.533 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:11.533 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:29:11.533 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:07.177822+0000", 2024-09-14T06:29:11.533 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.6258000000000002e-05 2024-09-14T06:29:11.533 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:11.534 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:11.534 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:11.534 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:11.534 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:11.534 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:2823 3.7 3:e68f0b27:::rbd_data.112d7dfbd766.000000000000006d:5 [read 237568~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-14T06:29:11.534 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:08.711240+0000", 2024-09-14T06:29:11.534 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 2.8144742790000001, 2024-09-14T06:29:11.534 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.011935549, 2024-09-14T06:29:11.534 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:29:11.534 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:29:11.534 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:29:11.534 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:29:11.534 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:29:11.534 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:11.534 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 2823 2024-09-14T06:29:11.534 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:11.535 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:29:11.535 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:11.535 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:29:11.535 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.711240+0000", 2024-09-14T06:29:11.535 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:11.535 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:11.535 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:11.535 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:29:11.535 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.711240+0000", 2024-09-14T06:29:11.535 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:11.535 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:11.535 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:11.535 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:29:11.535 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.711241+0000", 2024-09-14T06:29:11.535 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.7899999999999999e-07 2024-09-14T06:29:11.536 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:11.536 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:11.536 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:29:11.536 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.711243+0000", 2024-09-14T06:29:11.536 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.939e-06 2024-09-14T06:29:11.536 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:11.536 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:11.536 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:29:11.536 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.711252+0000", 2024-09-14T06:29:11.536 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.4360000000000008e-06 2024-09-14T06:29:11.536 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:11.536 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:11.536 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:29:11.536 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.711257+0000", 2024-09-14T06:29:11.536 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.8790000000000002e-06 2024-09-14T06:29:11.537 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:11.537 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:11.537 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:29:11.537 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.722967+0000", 2024-09-14T06:29:11.537 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.011709514000000001 2024-09-14T06:29:11.537 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:11.537 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:11.537 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:29:11.537 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.723039+0000", 2024-09-14T06:29:11.537 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.2460999999999996e-05 2024-09-14T06:29:11.537 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:11.537 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:11.537 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:29:11.537 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.723176+0000", 2024-09-14T06:29:11.538 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00013684100000000001 2024-09-14T06:29:11.538 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:11.538 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:11.538 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:11.538 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:11.538 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:11.538 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:2824 3.7 3:e68f0b27:::rbd_data.112d7dfbd766.000000000000006d:5 [read 253952~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-14T06:29:11.538 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:08.711278+0000", 2024-09-14T06:29:11.538 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 2.8144368019999999, 2024-09-14T06:29:11.538 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.012049709, 2024-09-14T06:29:11.538 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:29:11.538 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:29:11.538 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:29:11.538 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:29:11.538 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:29:11.539 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:11.539 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 2824 2024-09-14T06:29:11.539 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:11.539 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:29:11.539 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:11.539 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:29:11.539 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.711278+0000", 2024-09-14T06:29:11.539 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:11.539 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:11.539 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:11.539 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:29:11.539 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.711278+0000", 2024-09-14T06:29:11.539 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:11.539 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:11.540 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:11.540 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:29:11.540 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.711278+0000", 2024-09-14T06:29:11.540 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.2799999999999996e-07 2024-09-14T06:29:11.540 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:11.540 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:11.540 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:29:11.540 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.711281+0000", 2024-09-14T06:29:11.540 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.0839999999999999e-06 2024-09-14T06:29:11.540 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:11.540 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:11.540 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:29:11.540 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.711287+0000", 2024-09-14T06:29:11.541 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.6540000000000002e-06 2024-09-14T06:29:11.541 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:11.541 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:11.541 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:29:11.541 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.711292+0000", 2024-09-14T06:29:11.541 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.4320000000000001e-06 2024-09-14T06:29:11.541 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:11.541 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:11.541 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:29:11.541 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.723168+0000", 2024-09-14T06:29:11.541 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.011876183 2024-09-14T06:29:11.541 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:11.541 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:11.542 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:29:11.542 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.723235+0000", 2024-09-14T06:29:11.542 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.7199999999999994e-05 2024-09-14T06:29:11.542 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:11.542 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:11.542 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:29:11.542 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.723328+0000", 2024-09-14T06:29:11.542 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.2627999999999995e-05 2024-09-14T06:29:11.542 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:11.542 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:11.542 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:11.542 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:11.542 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:11.542 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:2825 3.7 3:e68f0b27:::rbd_data.112d7dfbd766.000000000000006d:5 [read 270336~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-14T06:29:11.543 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:08.711298+0000", 2024-09-14T06:29:11.543 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 2.8144164109999998, 2024-09-14T06:29:11.543 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.012245903000000001, 2024-09-14T06:29:11.543 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:29:11.543 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:29:11.543 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:29:11.543 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:29:11.543 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:29:11.543 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:11.543 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 2825 2024-09-14T06:29:11.543 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:11.543 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:29:11.543 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:11.543 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:29:11.543 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.711298+0000", 2024-09-14T06:29:11.543 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:11.544 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:11.544 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:11.544 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:29:11.544 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.711298+0000", 2024-09-14T06:29:11.544 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:11.544 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:11.544 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:11.544 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:29:11.544 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.711299+0000", 2024-09-14T06:29:11.544 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.4999999999999998e-07 2024-09-14T06:29:11.544 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:11.544 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:11.544 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:29:11.544 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.711305+0000", 2024-09-14T06:29:11.545 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.4899999999999997e-06 2024-09-14T06:29:11.545 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:11.545 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:11.545 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:29:11.545 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.711312+0000", 2024-09-14T06:29:11.545 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.9489999999999999e-06 2024-09-14T06:29:11.545 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:11.545 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:11.545 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:29:11.545 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.711317+0000", 2024-09-14T06:29:11.545 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.6e-06 2024-09-14T06:29:11.545 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:11.546 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:11.546 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:29:11.546 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.723370+0000", 2024-09-14T06:29:11.546 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.012052916 2024-09-14T06:29:11.546 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:11.546 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:11.546 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:29:11.546 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.723445+0000", 2024-09-14T06:29:11.546 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.5687999999999998e-05 2024-09-14T06:29:11.546 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:11.546 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:11.546 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:29:11.546 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.723544+0000", 2024-09-14T06:29:11.546 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.8809999999999998e-05 2024-09-14T06:29:11.547 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:11.547 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:11.547 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:11.547 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:11.547 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:11.547 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:2826 3.7 3:e68f0b27:::rbd_data.112d7dfbd766.000000000000006d:5 [read 286720~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-14T06:29:11.547 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:08.711324+0000", 2024-09-14T06:29:11.547 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 2.8143904800000001, 2024-09-14T06:29:11.547 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.012395302, 2024-09-14T06:29:11.547 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:29:11.547 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:29:11.547 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:29:11.547 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:29:11.548 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:29:11.548 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:11.548 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 2826 2024-09-14T06:29:11.548 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:11.548 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:29:11.548 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:11.548 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:29:11.548 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.711324+0000", 2024-09-14T06:29:11.548 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:11.548 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:11.548 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:11.548 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:29:11.548 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.711324+0000", 2024-09-14T06:29:11.548 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:11.549 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:11.549 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:11.549 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:29:11.549 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.711325+0000", 2024-09-14T06:29:11.549 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.9900000000000001e-07 2024-09-14T06:29:11.549 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:11.549 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:11.549 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:29:11.549 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.711332+0000", 2024-09-14T06:29:11.549 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.0110000000000003e-06 2024-09-14T06:29:11.549 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:11.549 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:11.549 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:29:11.550 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.711338+0000", 2024-09-14T06:29:11.550 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.2859999999999996e-06 2024-09-14T06:29:11.550 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:11.550 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:11.550 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:29:11.550 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.711343+0000", 2024-09-14T06:29:11.550 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.6480000000000002e-06 2024-09-14T06:29:11.550 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:11.550 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:11.550 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:29:11.550 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.723582+0000", 2024-09-14T06:29:11.550 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.012239641000000001 2024-09-14T06:29:11.550 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:11.551 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:11.551 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:29:11.551 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.723658+0000", 2024-09-14T06:29:11.551 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.5988000000000005e-05 2024-09-14T06:29:11.551 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:11.551 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:11.551 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:29:11.551 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.723720+0000", 2024-09-14T06:29:11.551 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.1328999999999995e-05 2024-09-14T06:29:11.551 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:11.551 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:11.551 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:11.551 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:11.551 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:11.552 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:2827 3.7 3:e68f0b27:::rbd_data.112d7dfbd766.000000000000006d:5 [read 303104~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-14T06:29:11.552 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:08.711349+0000", 2024-09-14T06:29:11.552 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 2.8143655110000001, 2024-09-14T06:29:11.552 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.01252516, 2024-09-14T06:29:11.552 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:29:11.552 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:29:11.552 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:29:11.552 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:29:11.552 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:29:11.552 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:11.552 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 2827 2024-09-14T06:29:11.552 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:11.553 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:29:11.553 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:11.553 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:29:11.553 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.711349+0000", 2024-09-14T06:29:11.553 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:11.553 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:11.553 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:11.553 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:29:11.553 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.711349+0000", 2024-09-14T06:29:11.553 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:11.553 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:11.553 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:11.553 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:29:11.554 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.711350+0000", 2024-09-14T06:29:11.554 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.7899999999999999e-07 2024-09-14T06:29:11.554 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:11.554 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:11.554 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:29:11.554 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.711351+0000", 2024-09-14T06:29:11.554 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.795e-06 2024-09-14T06:29:11.554 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:11.554 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:11.554 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:29:11.554 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.711357+0000", 2024-09-14T06:29:11.554 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.0050000000000001e-06 2024-09-14T06:29:11.554 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:11.554 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:11.554 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:29:11.555 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.711362+0000", 2024-09-14T06:29:11.555 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.4279999999999998e-06 2024-09-14T06:29:11.555 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:11.555 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:11.555 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:29:11.555 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.723749+0000", 2024-09-14T06:29:11.555 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.012387624 2024-09-14T06:29:11.555 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:11.555 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:11.555 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:29:11.555 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.723790+0000", 2024-09-14T06:29:11.555 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.0215e-05 2024-09-14T06:29:11.555 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:11.555 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:11.555 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:29:11.555 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.723874+0000", 2024-09-14T06:29:11.556 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.4714000000000002e-05 2024-09-14T06:29:11.556 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:11.556 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:11.556 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:11.556 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:11.556 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:11.556 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:2828 3.7 3:e68f0b27:::rbd_data.112d7dfbd766.000000000000006d:5 [read 319488~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-14T06:29:11.556 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:08.711368+0000", 2024-09-14T06:29:11.556 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 2.814346247, 2024-09-14T06:29:11.556 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.012743901, 2024-09-14T06:29:11.556 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:29:11.556 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:29:11.556 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:29:11.556 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:29:11.556 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:29:11.557 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:11.557 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 2828 2024-09-14T06:29:11.557 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:11.557 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:29:11.557 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:11.557 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:29:11.557 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.711368+0000", 2024-09-14T06:29:11.557 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:11.557 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:11.557 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:11.557 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:29:11.557 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.711368+0000", 2024-09-14T06:29:11.557 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:11.557 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:11.558 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:11.558 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:29:11.558 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.711369+0000", 2024-09-14T06:29:11.558 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.2399999999999999e-07 2024-09-14T06:29:11.558 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:11.558 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:11.558 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:29:11.558 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.711370+0000", 2024-09-14T06:29:11.558 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.688e-06 2024-09-14T06:29:11.558 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:11.558 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:11.558 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:29:11.558 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.711376+0000", 2024-09-14T06:29:11.558 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.8900000000000004e-06 2024-09-14T06:29:11.558 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:11.559 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:11.559 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:29:11.559 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.711381+0000", 2024-09-14T06:29:11.559 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.3390000000000003e-06 2024-09-14T06:29:11.559 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:11.559 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:11.559 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:29:11.559 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.723921+0000", 2024-09-14T06:29:11.559 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0125402 2024-09-14T06:29:11.559 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:11.559 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:11.559 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:29:11.559 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.724006+0000", 2024-09-14T06:29:11.559 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.5554000000000001e-05 2024-09-14T06:29:11.559 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:11.559 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:11.560 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:29:11.560 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.724112+0000", 2024-09-14T06:29:11.560 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000105906 2024-09-14T06:29:11.560 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:11.560 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:11.560 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:11.560 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:11.560 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:11.560 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:2829 3.7 3:e68f0b27:::rbd_data.112d7dfbd766.000000000000006d:5 [read 335872~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-14T06:29:11.560 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:08.711408+0000", 2024-09-14T06:29:11.560 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 2.8143063050000001, 2024-09-14T06:29:11.560 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.012901864000000001, 2024-09-14T06:29:11.560 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:29:11.560 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:29:11.561 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:29:11.561 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:29:11.561 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:29:11.561 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:11.561 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 2829 2024-09-14T06:29:11.561 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:11.561 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:29:11.561 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:11.561 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:29:11.561 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.711408+0000", 2024-09-14T06:29:11.561 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:11.561 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:11.561 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:11.561 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:29:11.562 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.711408+0000", 2024-09-14T06:29:11.562 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:11.562 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:11.562 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:11.562 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:29:11.562 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.711409+0000", 2024-09-14T06:29:11.562 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.3000000000000001e-07 2024-09-14T06:29:11.562 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:11.562 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:11.562 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:29:11.562 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.711411+0000", 2024-09-14T06:29:11.562 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.0930000000000001e-06 2024-09-14T06:29:11.562 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:11.562 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:11.562 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:29:11.563 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.711419+0000", 2024-09-14T06:29:11.563 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.9000000000000006e-06 2024-09-14T06:29:11.563 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:11.563 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:11.563 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:29:11.563 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.711426+0000", 2024-09-14T06:29:11.563 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.8539999999999999e-06 2024-09-14T06:29:11.563 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:11.563 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:11.563 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:29:11.563 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.724137+0000", 2024-09-14T06:29:11.563 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.012710993 2024-09-14T06:29:11.563 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:11.564 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:11.564 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:29:11.564 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.724215+0000", 2024-09-14T06:29:11.564 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.8475999999999999e-05 2024-09-14T06:29:11.564 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:11.564 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:11.564 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:29:11.564 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.724310+0000", 2024-09-14T06:29:11.564 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.5018000000000004e-05 2024-09-14T06:29:11.564 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:11.564 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:11.564 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:11.564 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:11.564 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:11.564 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:2830 3.7 3:e68f0b27:::rbd_data.112d7dfbd766.000000000000006d:5 [read 352256~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-14T06:29:11.565 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:08.711433+0000", 2024-09-14T06:29:11.565 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 2.814281528, 2024-09-14T06:29:11.565 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.013118222000000001, 2024-09-14T06:29:11.565 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:29:11.565 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:29:11.565 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:29:11.565 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:29:11.565 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:29:11.565 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:11.565 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 2830 2024-09-14T06:29:11.565 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:11.565 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:29:11.565 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:11.565 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:29:11.566 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.711433+0000", 2024-09-14T06:29:11.566 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:11.566 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:11.566 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:11.566 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:29:11.566 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.711433+0000", 2024-09-14T06:29:11.566 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:11.566 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:11.566 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:11.566 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:29:11.566 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.711434+0000", 2024-09-14T06:29:11.566 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.2599999999999998e-07 2024-09-14T06:29:11.566 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:11.566 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:11.566 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:29:11.566 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.711435+0000", 2024-09-14T06:29:11.567 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.8980000000000001e-06 2024-09-14T06:29:11.567 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:11.567 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:11.567 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:29:11.567 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.711441+0000", 2024-09-14T06:29:11.567 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.8270000000000003e-06 2024-09-14T06:29:11.567 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:11.567 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:11.567 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:29:11.567 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.711445+0000", 2024-09-14T06:29:11.567 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.1579999999999998e-06 2024-09-14T06:29:11.567 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:11.567 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:11.567 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:29:11.567 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.724358+0000", 2024-09-14T06:29:11.568 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.012912699999999999 2024-09-14T06:29:11.568 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:11.568 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:11.568 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:29:11.568 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.724450+0000", 2024-09-14T06:29:11.568 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.1531000000000002e-05 2024-09-14T06:29:11.568 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:11.568 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:11.568 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:29:11.568 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.724551+0000", 2024-09-14T06:29:11.568 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00010168200000000001 2024-09-14T06:29:11.568 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:11.568 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:11.568 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:11.569 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:11.569 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:11.569 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:2831 3.7 3:e68f0b27:::rbd_data.112d7dfbd766.000000000000006d:5 [read 368640~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-14T06:29:11.569 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:08.711451+0000", 2024-09-14T06:29:11.569 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 2.814263446, 2024-09-14T06:29:11.569 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.013397492, 2024-09-14T06:29:11.569 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:29:11.569 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:29:11.569 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:29:11.569 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:29:11.569 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:29:11.569 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:11.569 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 2831 2024-09-14T06:29:11.569 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:11.570 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:29:11.570 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:11.570 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:29:11.570 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.711451+0000", 2024-09-14T06:29:11.570 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:11.570 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:11.570 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:11.570 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:29:11.570 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.711451+0000", 2024-09-14T06:29:11.570 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:11.570 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:11.570 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:11.570 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:29:11.570 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.711452+0000", 2024-09-14T06:29:11.571 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.5900000000000003e-07 2024-09-14T06:29:11.571 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:11.571 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:11.571 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:29:11.571 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.711453+0000", 2024-09-14T06:29:11.571 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.5999999999999999e-06 2024-09-14T06:29:11.571 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:11.571 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:11.571 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:29:11.571 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.711461+0000", 2024-09-14T06:29:11.571 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.123e-06 2024-09-14T06:29:11.571 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:11.571 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:11.571 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:29:11.572 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.711469+0000", 2024-09-14T06:29:11.572 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.7919999999999993e-06 2024-09-14T06:29:11.572 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:11.572 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:11.572 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:29:11.572 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.724641+0000", 2024-09-14T06:29:11.572 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.013172095 2024-09-14T06:29:11.572 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:11.572 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:11.572 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:29:11.572 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.724730+0000", 2024-09-14T06:29:11.572 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.8497000000000003e-05 2024-09-14T06:29:11.572 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:11.572 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:11.573 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:29:11.573 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.724849+0000", 2024-09-14T06:29:11.573 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000119026 2024-09-14T06:29:11.573 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:11.573 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:11.573 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:11.573 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:11.573 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:11.573 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:2832 3.7 3:e68f0b27:::rbd_data.112d7dfbd766.000000000000006d:5 [read 385024~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-14T06:29:11.573 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:08.711483+0000", 2024-09-14T06:29:11.573 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 2.814231548, 2024-09-14T06:29:11.573 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.013595408, 2024-09-14T06:29:11.573 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:29:11.573 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:29:11.573 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:29:11.574 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:29:11.574 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:29:11.574 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:11.574 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 2832 2024-09-14T06:29:11.574 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:11.574 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:29:11.574 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:11.574 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:29:11.574 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.711483+0000", 2024-09-14T06:29:11.574 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:11.574 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:11.574 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:11.574 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:29:11.574 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.711483+0000", 2024-09-14T06:29:11.574 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:11.575 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:11.575 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:11.575 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:29:11.575 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.711484+0000", 2024-09-14T06:29:11.575 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.0399999999999996e-07 2024-09-14T06:29:11.575 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:11.575 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:11.575 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:29:11.575 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.711486+0000", 2024-09-14T06:29:11.575 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.0269999999999998e-06 2024-09-14T06:29:11.575 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:11.575 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:11.575 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:29:11.575 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.711492+0000", 2024-09-14T06:29:11.575 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.4489999999999996e-06 2024-09-14T06:29:11.576 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:11.576 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:11.576 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:29:11.576 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.711501+0000", 2024-09-14T06:29:11.576 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.3449999999999995e-06 2024-09-14T06:29:11.576 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:11.576 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:11.576 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:29:11.576 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.724897+0000", 2024-09-14T06:29:11.576 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.013395473999999999 2024-09-14T06:29:11.576 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:11.576 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:11.576 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:29:11.576 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.724974+0000", 2024-09-14T06:29:11.576 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.7158999999999998e-05 2024-09-14T06:29:11.577 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:11.577 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:11.577 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:29:11.577 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.725079+0000", 2024-09-14T06:29:11.577 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00010445 2024-09-14T06:29:11.577 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:11.577 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:11.577 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:11.577 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:11.577 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:11.577 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:2833 3.7 3:e68f0b27:::rbd_data.112d7dfbd766.000000000000006d:5 [read 401408~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-14T06:29:11.577 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:08.711508+0000", 2024-09-14T06:29:11.577 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 2.814206355, 2024-09-14T06:29:11.578 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.013818878, 2024-09-14T06:29:11.578 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:29:11.578 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:29:11.578 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:29:11.578 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:29:11.578 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:29:11.578 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:11.578 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 2833 2024-09-14T06:29:11.578 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:11.578 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:29:11.578 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:11.578 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:29:11.579 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.711508+0000", 2024-09-14T06:29:11.579 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:11.579 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:11.579 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:11.579 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:29:11.579 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.711508+0000", 2024-09-14T06:29:11.579 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:11.579 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:11.579 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:11.580 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:29:11.580 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.711509+0000", 2024-09-14T06:29:11.580 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.6899999999999998e-07 2024-09-14T06:29:11.580 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:11.580 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:11.580 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:29:11.580 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.711511+0000", 2024-09-14T06:29:11.580 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.9199999999999998e-06 2024-09-14T06:29:11.580 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:11.580 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:11.580 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:29:11.580 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.711517+0000", 2024-09-14T06:29:11.580 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.9719999999999997e-06 2024-09-14T06:29:11.581 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:11.581 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:11.581 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:29:11.581 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.711524+0000", 2024-09-14T06:29:11.581 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.8690000000000004e-06 2024-09-14T06:29:11.581 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:11.581 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:11.581 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:29:11.581 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.725129+0000", 2024-09-14T06:29:11.581 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.013604565000000001 2024-09-14T06:29:11.581 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:11.581 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:11.582 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:29:11.582 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.725212+0000", 2024-09-14T06:29:11.582 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.2492999999999998e-05 2024-09-14T06:29:11.582 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:11.582 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:11.582 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:29:11.582 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.725327+0000", 2024-09-14T06:29:11.582 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00011569 2024-09-14T06:29:11.582 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:11.582 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:11.582 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:11.582 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:11.582 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:11.582 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:2834 3.7 3:e68f0b27:::rbd_data.112d7dfbd766.000000000000006d:5 [read 417792~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-14T06:29:11.582 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:08.711531+0000", 2024-09-14T06:29:11.583 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 2.8141838699999999, 2024-09-14T06:29:11.583 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.014032723, 2024-09-14T06:29:11.583 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:29:11.583 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:29:11.583 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:29:11.583 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:29:11.583 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:29:11.583 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:11.583 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 2834 2024-09-14T06:29:11.583 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:11.583 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:29:11.583 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:11.583 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:29:11.583 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.711531+0000", 2024-09-14T06:29:11.584 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:11.584 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:11.584 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:11.584 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:29:11.584 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.711531+0000", 2024-09-14T06:29:11.584 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:11.584 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:11.584 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:11.584 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:29:11.585 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.711531+0000", 2024-09-14T06:29:11.585 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.7599999999999998e-07 2024-09-14T06:29:11.585 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:11.585 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:11.585 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:29:11.585 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.711533+0000", 2024-09-14T06:29:11.585 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.7540000000000001e-06 2024-09-14T06:29:11.585 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:11.585 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:11.585 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:29:11.585 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.711539+0000", 2024-09-14T06:29:11.585 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.7899999999999996e-06 2024-09-14T06:29:11.585 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:11.585 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:11.586 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:29:11.586 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.711548+0000", 2024-09-14T06:29:11.586 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.3039999999999994e-06 2024-09-14T06:29:11.586 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:11.586 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:11.586 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:29:11.586 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.725376+0000", 2024-09-14T06:29:11.586 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.013828343 2024-09-14T06:29:11.586 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:11.586 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:11.586 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:29:11.586 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.725459+0000", 2024-09-14T06:29:11.586 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.2888999999999999e-05 2024-09-14T06:29:11.586 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:11.586 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:11.587 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:29:11.587 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.725564+0000", 2024-09-14T06:29:11.587 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000104267 2024-09-14T06:29:11.587 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:11.587 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:11.587 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:11.587 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:11.587 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:11.587 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:2835 3.7 3:e68f0b27:::rbd_data.112d7dfbd766.000000000000006d:5 [read 434176~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-14T06:29:11.587 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:08.711556+0000", 2024-09-14T06:29:11.587 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 2.8141588950000003, 2024-09-14T06:29:11.587 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.014225475, 2024-09-14T06:29:11.587 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:29:11.587 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:29:11.587 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:29:11.587 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:29:11.588 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:29:11.588 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:11.588 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 2835 2024-09-14T06:29:11.588 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:11.588 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:29:11.588 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:11.588 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:29:11.588 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.711556+0000", 2024-09-14T06:29:11.588 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:11.588 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:11.588 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:11.588 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:29:11.588 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.711556+0000", 2024-09-14T06:29:11.588 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:11.589 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:11.589 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:11.589 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:29:11.589 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.711556+0000", 2024-09-14T06:29:11.589 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.2e-07 2024-09-14T06:29:11.589 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:11.589 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:11.589 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:29:11.589 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.711558+0000", 2024-09-14T06:29:11.589 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.781e-06 2024-09-14T06:29:11.589 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:11.589 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:11.589 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:29:11.589 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.711564+0000", 2024-09-14T06:29:11.589 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.0070000000000003e-06 2024-09-14T06:29:11.590 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:11.590 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:11.590 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:29:11.590 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.711569+0000", 2024-09-14T06:29:11.590 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.5329999999999996e-06 2024-09-14T06:29:11.590 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:11.590 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:11.590 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:29:11.590 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.725597+0000", 2024-09-14T06:29:11.590 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.014028357 2024-09-14T06:29:11.590 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:11.590 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:11.590 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:29:11.590 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.725671+0000", 2024-09-14T06:29:11.591 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.3623999999999998e-05 2024-09-14T06:29:11.591 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:11.591 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:11.591 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:29:11.591 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.725781+0000", 2024-09-14T06:29:11.591 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000110753 2024-09-14T06:29:11.591 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:11.591 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:11.591 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:11.591 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:11.591 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:11.591 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:2836 3.7 3:e68f0b27:::rbd_data.112d7dfbd766.000000000000006d:5 [read 450560~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-14T06:29:11.591 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:08.711574+0000", 2024-09-14T06:29:11.591 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 2.8141405750000001, 2024-09-14T06:29:11.591 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.014419141999999999, 2024-09-14T06:29:11.592 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:29:11.592 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:29:11.592 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:29:11.592 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:29:11.592 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:29:11.592 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:11.592 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 2836 2024-09-14T06:29:11.592 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:11.592 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:29:11.592 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:11.592 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:29:11.592 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.711574+0000", 2024-09-14T06:29:11.592 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:11.592 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:11.592 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:11.592 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:29:11.593 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.711574+0000", 2024-09-14T06:29:11.593 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:11.593 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:11.593 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:11.593 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:29:11.593 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.711574+0000", 2024-09-14T06:29:11.593 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.58e-07 2024-09-14T06:29:11.593 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:11.593 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:11.593 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:29:11.593 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.711576+0000", 2024-09-14T06:29:11.593 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.6500000000000001e-06 2024-09-14T06:29:11.593 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:11.593 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:11.593 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:29:11.594 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.711582+0000", 2024-09-14T06:29:11.594 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.6720000000000001e-06 2024-09-14T06:29:11.594 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:11.594 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:11.594 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:29:11.594 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.711586+0000", 2024-09-14T06:29:11.594 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.3680000000000004e-06 2024-09-14T06:29:11.594 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:11.594 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:11.594 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:29:11.594 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.725820+0000", 2024-09-14T06:29:11.594 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.014233934 2024-09-14T06:29:11.594 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:11.594 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:11.594 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:29:11.595 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.725891+0000", 2024-09-14T06:29:11.595 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.0609000000000001e-05 2024-09-14T06:29:11.595 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:11.595 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:11.595 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:29:11.595 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.725993+0000", 2024-09-14T06:29:11.595 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000102551 2024-09-14T06:29:11.595 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:11.595 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:11.596 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:11.596 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:11.596 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:11.596 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:2837 3.7 3:e68f0b27:::rbd_data.112d7dfbd766.000000000000006d:5 [read 466944~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-14T06:29:11.596 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:08.711592+0000", 2024-09-14T06:29:11.596 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 2.8141231480000002, 2024-09-14T06:29:11.596 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.014675145000000001, 2024-09-14T06:29:11.596 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:29:11.596 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:29:11.596 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:29:11.596 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:29:11.596 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:29:11.596 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:11.596 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 2837 2024-09-14T06:29:11.597 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:11.597 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:29:11.597 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:11.597 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:29:11.597 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.711592+0000", 2024-09-14T06:29:11.597 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:11.597 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:11.597 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:11.597 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:29:11.597 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.711592+0000", 2024-09-14T06:29:11.597 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:11.597 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:11.597 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:11.597 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:29:11.598 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.711592+0000", 2024-09-14T06:29:11.598 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.5199999999999997e-07 2024-09-14T06:29:11.598 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:11.598 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:11.598 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:29:11.598 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.711622+0000", 2024-09-14T06:29:11.598 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.0199999999999999e-05 2024-09-14T06:29:11.598 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:11.598 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:11.598 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:29:11.598 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.711629+0000", 2024-09-14T06:29:11.598 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.1049999999999997e-06 2024-09-14T06:29:11.598 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:11.598 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:11.599 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:29:11.599 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.711634+0000", 2024-09-14T06:29:11.599 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.826e-06 2024-09-14T06:29:11.599 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:11.599 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:11.599 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:29:11.599 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.726039+0000", 2024-09-14T06:29:11.599 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.014404682 2024-09-14T06:29:11.599 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:11.599 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:11.599 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:29:11.599 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.726122+0000", 2024-09-14T06:29:11.599 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.3133000000000005e-05 2024-09-14T06:29:11.600 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:11.600 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:11.600 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:29:11.600 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.726267+0000", 2024-09-14T06:29:11.600 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000144647 2024-09-14T06:29:11.600 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:11.600 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:11.600 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:11.600 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:11.600 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:11.600 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:2838 3.7 3:e68f0b27:::rbd_data.112d7dfbd766.000000000000006d:5 [read 483328~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-14T06:29:11.600 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:08.711641+0000", 2024-09-14T06:29:11.600 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 2.8140732110000002, 2024-09-14T06:29:11.601 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.014898524999999999, 2024-09-14T06:29:11.601 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:29:11.601 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:29:11.601 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:29:11.601 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:29:11.601 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:29:11.601 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:11.601 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 2838 2024-09-14T06:29:11.601 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:11.601 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:29:11.601 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:11.601 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:29:11.601 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.711641+0000", 2024-09-14T06:29:11.601 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:11.602 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:11.602 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:11.602 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:29:11.602 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.711641+0000", 2024-09-14T06:29:11.602 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:11.602 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:11.602 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:11.602 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:29:11.602 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.711642+0000", 2024-09-14T06:29:11.602 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.3200000000000005e-07 2024-09-14T06:29:11.602 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:11.602 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:11.602 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:29:11.602 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.711644+0000", 2024-09-14T06:29:11.602 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.827e-06 2024-09-14T06:29:11.603 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:11.603 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:11.603 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:29:11.603 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.711655+0000", 2024-09-14T06:29:11.603 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.1564e-05 2024-09-14T06:29:11.603 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:11.603 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:11.603 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:29:11.603 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.711661+0000", 2024-09-14T06:29:11.603 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.7599999999999999e-06 2024-09-14T06:29:11.603 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:11.603 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:11.603 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:29:11.603 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.726299+0000", 2024-09-14T06:29:11.603 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.014638138 2024-09-14T06:29:11.603 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:11.604 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:11.604 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:29:11.604 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.726380+0000", 2024-09-14T06:29:11.604 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.0756999999999999e-05 2024-09-14T06:29:11.604 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:11.604 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:11.604 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:29:11.604 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.726540+0000", 2024-09-14T06:29:11.604 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00015994699999999999 2024-09-14T06:29:11.604 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:11.604 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:11.604 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:11.604 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:11.604 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:11.605 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:2841 3.7 3:e5b3a0f9:::rbd_data.112d7dfbd766.0000000000000069:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-14T06:29:11.605 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:08.711698+0000", 2024-09-14T06:29:11.605 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 2.8140169020000001, 2024-09-14T06:29:11.605 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.014993504, 2024-09-14T06:29:11.605 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:29:11.605 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:29:11.605 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "reached pg", 2024-09-14T06:29:11.605 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:29:11.605 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:29:11.605 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:11.605 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 2841 2024-09-14T06:29:11.605 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:11.605 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:29:11.605 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:11.605 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:29:11.606 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.711698+0000", 2024-09-14T06:29:11.606 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:11.606 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:11.606 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:11.606 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:29:11.606 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.711698+0000", 2024-09-14T06:29:11.606 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:11.606 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:11.606 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:11.606 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:29:11.606 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.711698+0000", 2024-09-14T06:29:11.606 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.7e-07 2024-09-14T06:29:11.606 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:11.606 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:11.607 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:29:11.607 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.711700+0000", 2024-09-14T06:29:11.607 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.8199999999999999e-06 2024-09-14T06:29:11.607 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:11.607 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:11.607 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:29:11.607 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.711706+0000", 2024-09-14T06:29:11.607 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.8050000000000003e-06 2024-09-14T06:29:11.607 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:11.607 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:11.607 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:29:11.608 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.711710+0000", 2024-09-14T06:29:11.608 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.69e-06 2024-09-14T06:29:11.608 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:11.608 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:11.608 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:29:11.608 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.726608+0000", 2024-09-14T06:29:11.608 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.014897881999999999 2024-09-14T06:29:11.608 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:11.608 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:11.608 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:29:11.608 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.726691+0000", 2024-09-14T06:29:11.608 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.2936999999999996e-05 2024-09-14T06:29:11.608 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:11.608 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:11.609 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:11.609 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:11.609 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:11.609 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:2843 3.7 3:e750405e:::rbd_data.112d7dfbd766.000000000000006b:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-14T06:29:11.609 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:08.711718+0000", 2024-09-14T06:29:11.609 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 2.8139968070000001, 2024-09-14T06:29:11.609 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.01513749, 2024-09-14T06:29:11.609 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:29:11.609 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:29:11.609 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "reached pg", 2024-09-14T06:29:11.609 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:29:11.609 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:29:11.609 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:11.609 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 2843 2024-09-14T06:29:11.610 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:11.610 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:29:11.610 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:11.610 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:29:11.610 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.711718+0000", 2024-09-14T06:29:11.610 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:11.610 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:11.610 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:11.610 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:29:11.610 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.711718+0000", 2024-09-14T06:29:11.610 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:11.610 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:11.610 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:11.610 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:29:11.610 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.711718+0000", 2024-09-14T06:29:11.611 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.5900000000000003e-07 2024-09-14T06:29:11.611 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:11.611 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:11.611 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:29:11.611 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.711720+0000", 2024-09-14T06:29:11.611 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.6899999999999999e-06 2024-09-14T06:29:11.611 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:11.611 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:11.611 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:29:11.611 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.711726+0000", 2024-09-14T06:29:11.611 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.7819999999999999e-06 2024-09-14T06:29:11.611 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:11.612 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:11.612 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:29:11.612 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.711730+0000", 2024-09-14T06:29:11.612 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.2390000000000004e-06 2024-09-14T06:29:11.612 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:11.612 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:11.612 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:29:11.612 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.726739+0000", 2024-09-14T06:29:11.612 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.015009366999999999 2024-09-14T06:29:11.612 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:11.612 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:11.612 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:29:11.612 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.726855+0000", 2024-09-14T06:29:11.613 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00011605299999999999 2024-09-14T06:29:11.613 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:11.613 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:11.613 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:11.613 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:11.613 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:11.613 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:2845 3.7 3:e68f0b27:::rbd_data.112d7dfbd766.000000000000006d:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-14T06:29:11.613 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:08.711736+0000", 2024-09-14T06:29:11.613 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 2.813978358, 2024-09-14T06:29:11.613 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.015271547, 2024-09-14T06:29:11.613 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:29:11.613 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:29:11.613 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:29:11.614 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:29:11.614 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:29:11.614 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:11.614 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 2845 2024-09-14T06:29:11.614 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:11.614 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:29:11.614 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:11.614 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:29:11.614 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.711736+0000", 2024-09-14T06:29:11.614 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:11.614 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:11.614 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:11.614 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:29:11.614 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.711736+0000", 2024-09-14T06:29:11.615 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:11.615 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:11.615 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:11.615 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:29:11.615 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.711737+0000", 2024-09-14T06:29:11.615 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.3799999999999998e-07 2024-09-14T06:29:11.615 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:11.615 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:11.615 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:29:11.615 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.711738+0000", 2024-09-14T06:29:11.615 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.801e-06 2024-09-14T06:29:11.615 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:11.615 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:11.615 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:29:11.616 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.711744+0000", 2024-09-14T06:29:11.616 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.9190000000000004e-06 2024-09-14T06:29:11.616 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:11.616 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:11.616 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:29:11.616 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.711749+0000", 2024-09-14T06:29:11.616 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.4469999999999999e-06 2024-09-14T06:29:11.616 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:11.616 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:11.616 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:29:11.616 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.726889+0000", 2024-09-14T06:29:11.616 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.015140374999999999 2024-09-14T06:29:11.616 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:11.616 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:11.617 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:29:11.617 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.726930+0000", 2024-09-14T06:29:11.617 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.1239000000000003e-05 2024-09-14T06:29:11.617 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:11.617 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:11.617 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:29:11.617 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.727008+0000", 2024-09-14T06:29:11.617 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.7428000000000005e-05 2024-09-14T06:29:11.617 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:11.617 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:11.617 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:11.617 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:11.617 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:11.618 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-14T06:29:11.619 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-14T06:29:11.620 INFO:tasks.ceph.osd.6.smithi138.stderr:2024-09-14T06:29:11.617+0000 7f8cd8849640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33620) UID: 0 2024-09-14T06:29:11.624 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-14T06:29:11.720 INFO:tasks.ceph.osd.3.smithi094.stderr:2024-09-14T06:29:11.719+0000 7feeeafe8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33655) UID: 0 2024-09-14T06:29:11.728 INFO:teuthology.orchestra.run.smithi138.stdout:{ 2024-09-14T06:29:11.728 INFO:teuthology.orchestra.run.smithi138.stdout: "ops": [], 2024-09-14T06:29:11.729 INFO:teuthology.orchestra.run.smithi138.stdout: "num_ops": 0 2024-09-14T06:29:11.729 INFO:teuthology.orchestra.run.smithi138.stdout:} 2024-09-14T06:29:11.737 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-14T06:29:11.821 INFO:tasks.ceph.osd.2.smithi094.stderr:2024-09-14T06:29:11.820+0000 7faf3269b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33649) UID: 0 2024-09-14T06:29:11.839 INFO:teuthology.orchestra.run.smithi138.stdout:{ 2024-09-14T06:29:11.839 INFO:teuthology.orchestra.run.smithi138.stdout: "ops": [], 2024-09-14T06:29:11.839 INFO:teuthology.orchestra.run.smithi138.stdout: "complaint_time": 30, 2024-09-14T06:29:11.839 INFO:teuthology.orchestra.run.smithi138.stdout: "num_blocked_ops": 0 2024-09-14T06:29:11.839 INFO:teuthology.orchestra.run.smithi138.stdout:} 2024-09-14T06:29:11.848 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-14T06:29:11.921 INFO:tasks.ceph.osd.1.smithi094.stderr:2024-09-14T06:29:11.920+0000 7f328e47a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33641) UID: 0 2024-09-14T06:29:11.952 INFO:teuthology.orchestra.run.smithi138.stdout:{ 2024-09-14T06:29:11.952 INFO:teuthology.orchestra.run.smithi138.stdout: "size": 20, 2024-09-14T06:29:11.952 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 600, 2024-09-14T06:29:11.952 INFO:teuthology.orchestra.run.smithi138.stdout: "ops": [ 2024-09-14T06:29:11.952 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:11.952 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:1809 3.3 3:c0711257:::rbd_data.112d7dfbd766.000000000000001a:5 [read 126976~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-14T06:29:11.952 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:08.654606+0000", 2024-09-14T06:29:11.952 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 3.2955054929999998, 2024-09-14T06:29:11.952 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.019587192, 2024-09-14T06:29:11.952 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:29:11.952 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:29:11.952 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:29:11.953 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:29:11.953 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:29:11.953 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:11.953 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 1809 2024-09-14T06:29:11.953 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:11.953 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:29:11.953 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:11.953 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:29:11.953 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.654606+0000", 2024-09-14T06:29:11.953 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:11.953 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:11.953 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:11.954 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:29:11.954 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.654606+0000", 2024-09-14T06:29:11.954 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:11.954 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:11.954 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:11.954 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:29:11.954 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.654606+0000", 2024-09-14T06:29:11.954 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.34e-07 2024-09-14T06:29:11.954 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:11.954 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:11.954 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:29:11.954 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.654607+0000", 2024-09-14T06:29:11.954 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.3600000000000002e-07 2024-09-14T06:29:11.954 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:11.954 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:11.955 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:29:11.955 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.654610+0000", 2024-09-14T06:29:11.955 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.0299999999999998e-06 2024-09-14T06:29:11.955 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:11.955 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:11.955 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:29:11.955 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.654612+0000", 2024-09-14T06:29:11.955 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.3309999999999998e-06 2024-09-14T06:29:11.955 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:11.955 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:11.955 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:29:11.955 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.674056+0000", 2024-09-14T06:29:11.955 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.019443790999999998 2024-09-14T06:29:11.955 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:11.956 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:11.956 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:29:11.956 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.674121+0000", 2024-09-14T06:29:11.956 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.5029000000000004e-05 2024-09-14T06:29:11.956 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:11.956 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:11.956 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:29:11.956 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.674193+0000", 2024-09-14T06:29:11.956 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.1841000000000006e-05 2024-09-14T06:29:11.956 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:11.956 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:29:11.956 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:11.956 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:11.956 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:11.957 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:1810 3.3 3:c0711257:::rbd_data.112d7dfbd766.000000000000001a:5 [read 143360~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-14T06:29:11.957 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:08.654617+0000", 2024-09-14T06:29:11.957 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 3.2954947849999998, 2024-09-14T06:29:11.957 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.019784883, 2024-09-14T06:29:11.957 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:29:11.957 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:29:11.957 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:29:11.957 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:29:11.957 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:29:11.957 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:11.957 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 1810 2024-09-14T06:29:11.957 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:11.957 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:29:11.957 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:11.958 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:29:11.958 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.654617+0000", 2024-09-14T06:29:11.958 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:11.958 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:11.958 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:11.958 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:29:11.958 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.654617+0000", 2024-09-14T06:29:11.958 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:11.958 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:11.958 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:11.958 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:29:11.958 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.654617+0000", 2024-09-14T06:29:11.958 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.1e-07 2024-09-14T06:29:11.959 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:11.959 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:11.959 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:29:11.959 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.654618+0000", 2024-09-14T06:29:11.959 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.7000000000000003e-07 2024-09-14T06:29:11.959 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:11.959 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:11.959 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:29:11.959 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.654621+0000", 2024-09-14T06:29:11.959 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.9579999999999999e-06 2024-09-14T06:29:11.959 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:11.959 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:11.959 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:29:11.960 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.654623+0000", 2024-09-14T06:29:11.960 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.4210000000000002e-06 2024-09-14T06:29:11.960 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:11.960 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:11.960 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:29:11.960 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.674220+0000", 2024-09-14T06:29:11.960 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.019597063000000001 2024-09-14T06:29:11.960 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:11.960 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:11.960 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:29:11.960 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.674303+0000", 2024-09-14T06:29:11.960 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.2867999999999993e-05 2024-09-14T06:29:11.960 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:11.961 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:11.961 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:29:11.961 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.674402+0000", 2024-09-14T06:29:11.961 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.8492999999999994e-05 2024-09-14T06:29:11.961 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:11.961 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:29:11.961 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:11.961 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:11.961 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:11.961 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:1811 3.3 3:c0711257:::rbd_data.112d7dfbd766.000000000000001a:5 [read 159744~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-14T06:29:11.961 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:08.654627+0000", 2024-09-14T06:29:11.961 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 3.2954849290000001, 2024-09-14T06:29:11.961 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.020020645, 2024-09-14T06:29:11.961 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:29:11.962 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:29:11.962 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:29:11.962 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:29:11.962 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:29:11.962 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:11.962 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 1811 2024-09-14T06:29:11.962 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:11.962 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:29:11.962 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:11.962 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:29:11.962 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.654627+0000", 2024-09-14T06:29:11.962 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:11.962 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:11.962 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:11.963 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:29:11.963 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.654627+0000", 2024-09-14T06:29:11.963 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:11.963 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:11.963 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:11.963 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:29:11.963 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.654627+0000", 2024-09-14T06:29:11.963 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.8599999999999999e-07 2024-09-14T06:29:11.963 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:11.963 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:11.963 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:29:11.963 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.654628+0000", 2024-09-14T06:29:11.963 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.39e-06 2024-09-14T06:29:11.964 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:11.964 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:11.964 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:29:11.964 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.654633+0000", 2024-09-14T06:29:11.964 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.8810000000000004e-06 2024-09-14T06:29:11.964 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:11.964 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:11.964 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:29:11.964 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.654637+0000", 2024-09-14T06:29:11.964 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.1629999999999998e-06 2024-09-14T06:29:11.964 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:11.964 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:11.964 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:29:11.964 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.674468+0000", 2024-09-14T06:29:11.964 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.019830668999999999 2024-09-14T06:29:11.965 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:11.965 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:11.965 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:29:11.965 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.674544+0000", 2024-09-14T06:29:11.965 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.6438000000000002e-05 2024-09-14T06:29:11.965 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:11.965 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:11.965 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:29:11.965 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.674647+0000", 2024-09-14T06:29:11.965 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.000102818 2024-09-14T06:29:11.965 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:11.965 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:29:11.965 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:11.965 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:11.965 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:11.966 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:1812 3.3 3:c0711257:::rbd_data.112d7dfbd766.000000000000001a:5 [read 176128~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-14T06:29:11.966 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:08.654642+0000", 2024-09-14T06:29:11.966 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 3.2954697479999999, 2024-09-14T06:29:11.966 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.020225290999999999, 2024-09-14T06:29:11.966 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:29:11.966 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:29:11.966 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:29:11.966 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:29:11.966 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:29:11.966 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:11.966 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 1812 2024-09-14T06:29:11.966 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:11.966 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:29:11.966 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:11.966 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:29:11.967 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.654642+0000", 2024-09-14T06:29:11.967 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:11.967 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:11.967 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:11.967 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:29:11.967 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.654642+0000", 2024-09-14T06:29:11.967 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:11.967 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:11.967 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:11.967 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:29:11.967 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.654642+0000", 2024-09-14T06:29:11.967 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.35e-07 2024-09-14T06:29:11.967 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:11.967 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:11.967 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:29:11.968 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.654643+0000", 2024-09-14T06:29:11.968 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.1599999999999999e-06 2024-09-14T06:29:11.968 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:11.968 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:11.968 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:29:11.968 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.654646+0000", 2024-09-14T06:29:11.968 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.185e-06 2024-09-14T06:29:11.968 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:11.968 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:11.968 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:29:11.968 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.654649+0000", 2024-09-14T06:29:11.968 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.2339999999999999e-06 2024-09-14T06:29:11.968 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:11.968 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:11.968 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:29:11.969 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.674683+0000", 2024-09-14T06:29:11.969 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.02003344 2024-09-14T06:29:11.969 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:11.969 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:11.969 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:29:11.969 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.674773+0000", 2024-09-14T06:29:11.969 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.0227000000000002e-05 2024-09-14T06:29:11.969 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:11.969 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:11.969 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:29:11.969 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.674867+0000", 2024-09-14T06:29:11.969 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.3809999999999998e-05 2024-09-14T06:29:11.969 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:11.969 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:29:11.969 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:11.970 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:11.970 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:11.970 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:1813 3.3 3:c0711257:::rbd_data.112d7dfbd766.000000000000001a:5 [read 192512~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-14T06:29:11.970 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:08.654672+0000", 2024-09-14T06:29:11.970 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 3.2954389449999999, 2024-09-14T06:29:11.970 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.020374199999999999, 2024-09-14T06:29:11.970 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:29:11.970 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:29:11.970 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:29:11.970 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:29:11.970 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:29:11.970 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:11.970 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 1813 2024-09-14T06:29:11.970 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:11.970 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:29:11.971 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:11.971 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:29:11.971 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.654672+0000", 2024-09-14T06:29:11.971 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:11.971 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:11.971 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:11.971 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:29:11.971 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.654672+0000", 2024-09-14T06:29:11.971 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:11.971 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:11.971 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:11.971 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:29:11.971 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.654673+0000", 2024-09-14T06:29:11.971 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.1099999999999995e-07 2024-09-14T06:29:11.971 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:11.972 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:11.972 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:29:11.972 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.654675+0000", 2024-09-14T06:29:11.972 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.2249999999999999e-06 2024-09-14T06:29:11.972 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:11.972 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:11.972 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:29:11.972 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.654682+0000", 2024-09-14T06:29:11.972 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.781e-06 2024-09-14T06:29:11.972 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:11.972 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:11.972 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:29:11.972 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.654686+0000", 2024-09-14T06:29:11.972 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.6490000000000001e-06 2024-09-14T06:29:11.972 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:11.973 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:11.973 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:29:11.973 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.674904+0000", 2024-09-14T06:29:11.973 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.020218604000000001 2024-09-14T06:29:11.973 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:11.973 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:11.973 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:29:11.973 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.674959+0000", 2024-09-14T06:29:11.973 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.5004999999999999e-05 2024-09-14T06:29:11.973 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:11.973 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:11.973 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:29:11.973 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.675047+0000", 2024-09-14T06:29:11.973 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.7324999999999995e-05 2024-09-14T06:29:11.973 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:11.974 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:29:11.974 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:11.974 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:11.974 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:11.974 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:1814 3.3 3:c0711257:::rbd_data.112d7dfbd766.000000000000001a:5 [read 208896~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-14T06:29:11.974 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:08.654691+0000", 2024-09-14T06:29:11.974 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 3.2954204069999999, 2024-09-14T06:29:11.974 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.020493225, 2024-09-14T06:29:11.974 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:29:11.974 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:29:11.974 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:29:11.974 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:29:11.974 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:29:11.974 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:11.974 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 1814 2024-09-14T06:29:11.974 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:11.975 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:29:11.975 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:11.975 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:29:11.975 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.654691+0000", 2024-09-14T06:29:11.975 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:11.975 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:11.975 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:11.975 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:29:11.975 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.654691+0000", 2024-09-14T06:29:11.975 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:11.975 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:11.975 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:11.975 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:29:11.975 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.654691+0000", 2024-09-14T06:29:11.976 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.41e-07 2024-09-14T06:29:11.976 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:11.976 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:11.976 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:29:11.976 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.654693+0000", 2024-09-14T06:29:11.976 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.3060000000000001e-06 2024-09-14T06:29:11.976 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:11.976 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:11.976 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:29:11.976 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.654696+0000", 2024-09-14T06:29:11.976 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.2100000000000002e-06 2024-09-14T06:29:11.976 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:11.976 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:11.976 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:29:11.977 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.654700+0000", 2024-09-14T06:29:11.977 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.7749999999999999e-06 2024-09-14T06:29:11.977 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:11.977 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:11.977 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:29:11.977 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.675069+0000", 2024-09-14T06:29:11.977 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.020369585999999999 2024-09-14T06:29:11.977 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:11.977 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:11.977 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:29:11.977 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.675108+0000", 2024-09-14T06:29:11.977 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.8479999999999997e-05 2024-09-14T06:29:11.977 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:11.977 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:11.977 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:29:11.978 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.675184+0000", 2024-09-14T06:29:11.978 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.6526999999999994e-05 2024-09-14T06:29:11.978 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:11.978 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:29:11.978 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:11.978 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:11.978 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:11.978 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:1815 3.3 3:c0711257:::rbd_data.112d7dfbd766.000000000000001a:5 [read 225280~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-14T06:29:11.978 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:08.654703+0000", 2024-09-14T06:29:11.978 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 3.2954084290000001, 2024-09-14T06:29:11.978 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.020638215000000001, 2024-09-14T06:29:11.978 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:29:11.978 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:29:11.978 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:29:11.978 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:29:11.978 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:29:11.979 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:11.979 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 1815 2024-09-14T06:29:11.979 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:11.979 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:29:11.979 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:11.979 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:29:11.979 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.654703+0000", 2024-09-14T06:29:11.979 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:11.979 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:11.979 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:11.979 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:29:11.979 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.654703+0000", 2024-09-14T06:29:11.979 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:11.979 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:11.980 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:11.980 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:29:11.980 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.654703+0000", 2024-09-14T06:29:11.980 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.4400000000000001e-07 2024-09-14T06:29:11.980 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:11.980 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:11.980 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:29:11.980 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.654704+0000", 2024-09-14T06:29:11.980 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.4900000000000004e-07 2024-09-14T06:29:11.980 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:11.981 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:11.981 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:29:11.981 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.654707+0000", 2024-09-14T06:29:11.981 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.0359999999999998e-06 2024-09-14T06:29:11.981 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:11.981 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:11.981 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:29:11.981 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.654710+0000", 2024-09-14T06:29:11.981 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.4820000000000001e-06 2024-09-14T06:29:11.981 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:11.981 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:11.981 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:29:11.981 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.675212+0000", 2024-09-14T06:29:11.981 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.020502126999999998 2024-09-14T06:29:11.982 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:11.982 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:11.982 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:29:11.982 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.675256+0000", 2024-09-14T06:29:11.982 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.4659000000000001e-05 2024-09-14T06:29:11.982 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:11.982 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:11.982 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:29:11.982 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.675341+0000", 2024-09-14T06:29:11.982 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.4717999999999998e-05 2024-09-14T06:29:11.982 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:11.982 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:29:11.982 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:11.982 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:11.982 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:11.983 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:1816 3.3 3:c0711257:::rbd_data.112d7dfbd766.000000000000001a:5 [read 241664~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-14T06:29:11.983 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:08.654713+0000", 2024-09-14T06:29:11.983 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 3.2953985590000001, 2024-09-14T06:29:11.983 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.020816715999999999, 2024-09-14T06:29:11.983 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:29:11.983 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:29:11.983 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:29:11.983 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:29:11.983 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:29:11.983 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:11.983 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 1816 2024-09-14T06:29:11.983 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:11.983 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:29:11.983 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:11.984 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:29:11.984 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.654713+0000", 2024-09-14T06:29:11.984 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:11.984 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:11.984 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:11.984 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:29:11.984 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.654713+0000", 2024-09-14T06:29:11.984 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:11.984 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:11.984 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:11.984 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:29:11.984 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.654713+0000", 2024-09-14T06:29:11.984 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.22e-07 2024-09-14T06:29:11.984 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:11.985 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:11.985 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:29:11.985 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.654714+0000", 2024-09-14T06:29:11.985 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.9600000000000008e-07 2024-09-14T06:29:11.985 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:11.985 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:11.985 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:29:11.985 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.654717+0000", 2024-09-14T06:29:11.985 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.9749999999999999e-06 2024-09-14T06:29:11.985 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:11.985 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:11.985 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:29:11.985 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.654720+0000", 2024-09-14T06:29:11.985 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.4760000000000001e-06 2024-09-14T06:29:11.986 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:11.986 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:11.986 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:29:11.986 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.675404+0000", 2024-09-14T06:29:11.986 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.020684599000000001 2024-09-14T06:29:11.986 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:11.986 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:11.986 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:29:11.986 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.675475+0000", 2024-09-14T06:29:11.986 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.0753000000000005e-05 2024-09-14T06:29:11.986 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:11.986 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:11.986 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:29:11.986 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.675530+0000", 2024-09-14T06:29:11.986 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.4694999999999997e-05 2024-09-14T06:29:11.987 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:11.987 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:29:11.987 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:11.987 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:11.987 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:11.987 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:1817 3.3 3:c0711257:::rbd_data.112d7dfbd766.000000000000001a:5 [read 258048~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-14T06:29:11.987 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:08.654723+0000", 2024-09-14T06:29:11.987 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 3.2953887540000002, 2024-09-14T06:29:11.987 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.020951320999999998, 2024-09-14T06:29:11.987 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:29:11.987 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:29:11.987 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:29:11.987 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:29:11.987 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:29:11.987 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:11.988 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 1817 2024-09-14T06:29:11.988 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:11.988 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:29:11.988 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:11.988 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:29:11.988 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.654723+0000", 2024-09-14T06:29:11.988 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:11.988 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:11.988 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:11.988 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:29:11.988 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.654723+0000", 2024-09-14T06:29:11.988 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:11.988 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:11.988 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:11.988 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:29:11.988 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.654723+0000", 2024-09-14T06:29:11.989 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.16e-07 2024-09-14T06:29:11.989 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:11.989 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:11.989 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:29:11.989 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.654731+0000", 2024-09-14T06:29:11.989 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.8140000000000001e-06 2024-09-14T06:29:11.989 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:11.989 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:11.989 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:29:11.989 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.654734+0000", 2024-09-14T06:29:11.989 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.3569999999999998e-06 2024-09-14T06:29:11.989 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:11.989 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:11.989 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:29:11.989 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.654737+0000", 2024-09-14T06:29:11.990 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.683e-06 2024-09-14T06:29:11.990 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:11.990 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:11.990 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:29:11.990 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.675557+0000", 2024-09-14T06:29:11.990 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.020820490000000001 2024-09-14T06:29:11.990 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:11.990 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:11.990 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:29:11.990 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.675605+0000", 2024-09-14T06:29:11.990 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.7432999999999998e-05 2024-09-14T06:29:11.990 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:11.990 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:11.990 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:29:11.990 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.675674+0000", 2024-09-14T06:29:11.990 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.9327999999999998e-05 2024-09-14T06:29:11.991 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:11.991 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:29:11.991 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:11.991 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:11.991 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:11.991 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:1818 3.3 3:c0711257:::rbd_data.112d7dfbd766.000000000000001a:5 [read 274432~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-14T06:29:11.991 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:08.654740+0000", 2024-09-14T06:29:11.991 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 3.2953710470000002, 2024-09-14T06:29:11.991 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.021074558, 2024-09-14T06:29:11.991 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:29:11.991 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:29:11.991 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:29:11.991 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:29:11.991 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:29:11.991 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:11.991 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 1818 2024-09-14T06:29:11.992 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:11.992 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:29:11.992 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:11.992 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:29:11.992 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.654740+0000", 2024-09-14T06:29:11.992 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:11.992 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:11.992 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:11.992 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:29:11.992 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.654740+0000", 2024-09-14T06:29:11.992 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:11.992 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:11.992 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:11.992 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:29:11.993 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.654741+0000", 2024-09-14T06:29:11.993 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.8700000000000002e-07 2024-09-14T06:29:11.993 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:11.993 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:11.993 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:29:11.993 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.654742+0000", 2024-09-14T06:29:11.993 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.604e-06 2024-09-14T06:29:11.993 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:11.993 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:11.993 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:29:11.993 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.654745+0000", 2024-09-14T06:29:11.993 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.067e-06 2024-09-14T06:29:11.993 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:11.993 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:11.993 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:29:11.994 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.654748+0000", 2024-09-14T06:29:11.994 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.599e-06 2024-09-14T06:29:11.994 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:11.994 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:11.994 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:29:11.994 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.675700+0000", 2024-09-14T06:29:11.994 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.020951612000000001 2024-09-14T06:29:11.994 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:11.994 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:11.994 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:29:11.994 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.675738+0000", 2024-09-14T06:29:11.994 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.8300999999999998e-05 2024-09-14T06:29:11.994 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:11.995 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:11.995 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:29:11.995 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.675815+0000", 2024-09-14T06:29:11.995 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.7088000000000005e-05 2024-09-14T06:29:11.995 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:11.995 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:29:11.995 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:11.995 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:11.995 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:11.995 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:1819 3.3 3:c0711257:::rbd_data.112d7dfbd766.000000000000001a:5 [read 290816~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-14T06:29:11.995 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:08.654751+0000", 2024-09-14T06:29:11.995 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 3.2953603980000001, 2024-09-14T06:29:11.995 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.021190092000000001, 2024-09-14T06:29:11.996 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:29:11.996 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:29:11.996 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:29:11.996 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:29:11.996 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:29:11.996 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:11.996 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 1819 2024-09-14T06:29:11.996 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:11.996 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:29:11.996 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:11.996 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:29:11.996 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.654751+0000", 2024-09-14T06:29:11.997 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:11.997 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:11.997 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:11.997 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:29:11.997 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.654751+0000", 2024-09-14T06:29:11.997 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:11.997 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:11.997 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:11.997 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:29:11.997 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.654751+0000", 2024-09-14T06:29:11.997 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.2100000000000001e-07 2024-09-14T06:29:11.997 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:11.997 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:11.998 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:29:11.998 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.654752+0000", 2024-09-14T06:29:11.998 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.0500000000000002e-07 2024-09-14T06:29:11.998 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:11.998 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:11.998 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:29:11.998 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.654755+0000", 2024-09-14T06:29:11.998 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.9799999999999998e-06 2024-09-14T06:29:11.998 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:11.998 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:11.998 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:29:11.998 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.654758+0000", 2024-09-14T06:29:11.998 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.48e-06 2024-09-14T06:29:11.998 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:11.999 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:11.999 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:29:11.999 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.675841+0000", 2024-09-14T06:29:11.999 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.021083085000000001 2024-09-14T06:29:11.999 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:11.999 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:11.999 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:29:11.999 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.675879+0000", 2024-09-14T06:29:11.999 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.8429000000000001e-05 2024-09-14T06:29:11.999 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:11.999 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:11.999 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:29:11.999 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.675941+0000", 2024-09-14T06:29:11.999 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.1991999999999999e-05 2024-09-14T06:29:11.999 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:12.000 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:29:12.000 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:12.000 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:12.000 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:12.000 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:1820 3.3 3:c0711257:::rbd_data.112d7dfbd766.000000000000001a:5 [read 307200~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-14T06:29:12.000 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:08.654762+0000", 2024-09-14T06:29:12.000 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 3.2953491289999999, 2024-09-14T06:29:12.000 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.021318159, 2024-09-14T06:29:12.000 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:29:12.000 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:29:12.000 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:29:12.000 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:29:12.001 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:29:12.001 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:12.001 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 1820 2024-09-14T06:29:12.001 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:12.001 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:29:12.001 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:12.001 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:29:12.001 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.654762+0000", 2024-09-14T06:29:12.001 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:12.001 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:12.001 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:12.001 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:29:12.001 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.654762+0000", 2024-09-14T06:29:12.001 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:12.002 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:12.002 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:12.002 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:29:12.002 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.654763+0000", 2024-09-14T06:29:12.002 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.23e-07 2024-09-14T06:29:12.002 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:12.002 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:12.002 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:29:12.002 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.654764+0000", 2024-09-14T06:29:12.002 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.879999999999999e-07 2024-09-14T06:29:12.002 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:12.002 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:12.002 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:29:12.002 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.654766+0000", 2024-09-14T06:29:12.002 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.9500000000000001e-06 2024-09-14T06:29:12.002 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:12.003 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:12.003 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:29:12.003 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.654769+0000", 2024-09-14T06:29:12.003 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.5799999999999999e-06 2024-09-14T06:29:12.003 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:12.003 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:12.003 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:29:12.003 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.675972+0000", 2024-09-14T06:29:12.003 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.021202478 2024-09-14T06:29:12.003 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:12.003 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:12.003 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:29:12.003 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.676011+0000", 2024-09-14T06:29:12.003 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.9412999999999997e-05 2024-09-14T06:29:12.003 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:12.004 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:12.004 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:29:12.004 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.676080+0000", 2024-09-14T06:29:12.004 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.9527e-05 2024-09-14T06:29:12.004 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:12.004 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:29:12.004 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:12.004 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:12.004 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:12.004 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:1821 3.3 3:c0711257:::rbd_data.112d7dfbd766.000000000000001a:5 [read 323584~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-14T06:29:12.004 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:08.654772+0000", 2024-09-14T06:29:12.004 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 3.2953392450000001, 2024-09-14T06:29:12.004 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.021435718999999999, 2024-09-14T06:29:12.004 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:29:12.005 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:29:12.005 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:29:12.005 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:29:12.005 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:29:12.005 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:12.005 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 1821 2024-09-14T06:29:12.005 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:12.005 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:29:12.005 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:12.005 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:29:12.005 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.654772+0000", 2024-09-14T06:29:12.005 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:12.005 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:12.005 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:12.006 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:29:12.006 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.654772+0000", 2024-09-14T06:29:12.006 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:12.006 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:12.006 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:12.006 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:29:12.006 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.654772+0000", 2024-09-14T06:29:12.006 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.2399999999999999e-07 2024-09-14T06:29:12.006 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:12.006 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:12.006 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:29:12.006 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.654773+0000", 2024-09-14T06:29:12.006 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.6199999999999996e-07 2024-09-14T06:29:12.007 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:12.007 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:12.007 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:29:12.007 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.654776+0000", 2024-09-14T06:29:12.007 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.8899999999999999e-06 2024-09-14T06:29:12.007 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:12.007 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:12.007 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:29:12.007 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.654779+0000", 2024-09-14T06:29:12.007 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.5620000000000002e-06 2024-09-14T06:29:12.007 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:12.007 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:12.007 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:29:12.008 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.676107+0000", 2024-09-14T06:29:12.008 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.021328203 2024-09-14T06:29:12.008 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:12.008 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:12.008 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:29:12.008 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.676155+0000", 2024-09-14T06:29:12.008 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.8109999999999998e-05 2024-09-14T06:29:12.008 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:12.008 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:12.008 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:29:12.008 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.676208+0000", 2024-09-14T06:29:12.008 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.2868000000000003e-05 2024-09-14T06:29:12.008 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:12.009 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:29:12.009 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:12.009 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:12.009 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:12.009 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:1822 3.3 3:c0711257:::rbd_data.112d7dfbd766.000000000000001a:5 [read 339968~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-14T06:29:12.009 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:08.654782+0000", 2024-09-14T06:29:12.009 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 3.2953296500000002, 2024-09-14T06:29:12.009 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.021547572000000001, 2024-09-14T06:29:12.009 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:29:12.009 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:29:12.009 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:29:12.009 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:29:12.010 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:29:12.010 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:12.010 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 1822 2024-09-14T06:29:12.010 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:12.010 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:29:12.010 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:12.010 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:29:12.010 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.654782+0000", 2024-09-14T06:29:12.010 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:12.010 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:12.010 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:12.010 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:29:12.010 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.654782+0000", 2024-09-14T06:29:12.011 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:12.011 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:12.011 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:12.011 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:29:12.011 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.654782+0000", 2024-09-14T06:29:12.011 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.22e-07 2024-09-14T06:29:12.011 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:12.011 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:12.011 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:29:12.011 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.654783+0000", 2024-09-14T06:29:12.011 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.0699999999999996e-07 2024-09-14T06:29:12.011 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:12.011 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:12.012 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:29:12.012 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.654786+0000", 2024-09-14T06:29:12.012 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.88e-06 2024-09-14T06:29:12.012 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:12.012 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:12.012 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:29:12.012 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.654788+0000", 2024-09-14T06:29:12.012 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.464e-06 2024-09-14T06:29:12.012 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:12.012 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:12.012 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:29:12.012 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.676232+0000", 2024-09-14T06:29:12.013 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.021443502 2024-09-14T06:29:12.013 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:12.013 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:12.013 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:29:12.013 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.676271+0000", 2024-09-14T06:29:12.013 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.8871000000000001e-05 2024-09-14T06:29:12.013 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:12.013 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:12.013 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:29:12.013 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.676329+0000", 2024-09-14T06:29:12.013 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.8726e-05 2024-09-14T06:29:12.013 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:12.014 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:29:12.014 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:12.014 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:12.014 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:12.014 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:1823 3.3 3:c0711257:::rbd_data.112d7dfbd766.000000000000001a:5 [read 356352~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-14T06:29:12.014 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:08.654792+0000", 2024-09-14T06:29:12.014 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 3.2953189370000002, 2024-09-14T06:29:12.014 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.021659046000000001, 2024-09-14T06:29:12.014 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:29:12.014 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:29:12.014 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:29:12.014 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:29:12.014 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:29:12.015 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:12.015 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 1823 2024-09-14T06:29:12.015 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:12.015 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:29:12.015 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:12.015 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:29:12.015 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.654792+0000", 2024-09-14T06:29:12.015 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:12.015 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:12.015 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:12.015 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:29:12.015 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.654792+0000", 2024-09-14T06:29:12.015 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:12.015 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:12.015 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:12.016 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:29:12.016 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.654793+0000", 2024-09-14T06:29:12.016 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.11e-07 2024-09-14T06:29:12.016 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:12.016 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:12.016 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:29:12.016 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.654794+0000", 2024-09-14T06:29:12.016 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.1399999999999996e-07 2024-09-14T06:29:12.016 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:12.016 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:12.016 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:29:12.016 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.654796+0000", 2024-09-14T06:29:12.016 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.88e-06 2024-09-14T06:29:12.016 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:12.017 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:12.017 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:29:12.017 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.654799+0000", 2024-09-14T06:29:12.017 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.9289999999999998e-06 2024-09-14T06:29:12.017 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:12.017 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:12.017 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:29:12.017 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.676351+0000", 2024-09-14T06:29:12.017 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.021552136999999999 2024-09-14T06:29:12.017 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:12.017 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:12.017 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:29:12.017 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.676398+0000", 2024-09-14T06:29:12.017 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.6660999999999999e-05 2024-09-14T06:29:12.017 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:12.018 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:12.018 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:29:12.018 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.676452+0000", 2024-09-14T06:29:12.018 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.3414000000000001e-05 2024-09-14T06:29:12.018 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:12.018 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:29:12.018 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:12.018 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:12.018 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:12.018 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:1824 3.3 3:c0711257:::rbd_data.112d7dfbd766.000000000000001a:5 [read 372736~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-14T06:29:12.018 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:08.654802+0000", 2024-09-14T06:29:12.018 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 3.2953091030000001, 2024-09-14T06:29:12.018 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.021747074000000002, 2024-09-14T06:29:12.018 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:29:12.018 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:29:12.019 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:29:12.019 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:29:12.019 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:29:12.019 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:12.019 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 1824 2024-09-14T06:29:12.019 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:12.019 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:29:12.019 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:12.019 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:29:12.019 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.654802+0000", 2024-09-14T06:29:12.019 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:12.019 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:12.019 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:12.019 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:29:12.019 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.654802+0000", 2024-09-14T06:29:12.020 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:12.020 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:12.020 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:12.020 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:29:12.020 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.654803+0000", 2024-09-14T06:29:12.020 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.0699999999999999e-07 2024-09-14T06:29:12.020 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:12.020 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:12.020 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:29:12.020 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.654803+0000", 2024-09-14T06:29:12.020 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.3799999999999996e-07 2024-09-14T06:29:12.020 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:12.020 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:12.020 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:29:12.020 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.654807+0000", 2024-09-14T06:29:12.021 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.6650000000000001e-06 2024-09-14T06:29:12.021 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:12.021 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:12.021 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:29:12.021 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.654810+0000", 2024-09-14T06:29:12.021 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.5409999999999999e-06 2024-09-14T06:29:12.021 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:12.021 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:12.021 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:29:12.021 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.676464+0000", 2024-09-14T06:29:12.021 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.021654651 2024-09-14T06:29:12.021 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:12.021 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:12.021 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:29:12.022 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.676501+0000", 2024-09-14T06:29:12.022 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.7073000000000002e-05 2024-09-14T06:29:12.022 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:12.022 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:12.022 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:29:12.022 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.676549+0000", 2024-09-14T06:29:12.022 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.7998999999999999e-05 2024-09-14T06:29:12.022 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:12.022 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:29:12.022 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:12.022 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:12.022 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:12.022 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:1825 3.3 3:c0711257:::rbd_data.112d7dfbd766.000000000000001a:5 [read 389120~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-14T06:29:12.022 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:08.654813+0000", 2024-09-14T06:29:12.022 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 3.295298732, 2024-09-14T06:29:12.023 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.021865797999999999, 2024-09-14T06:29:12.023 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:29:12.023 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:29:12.023 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:29:12.023 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:29:12.023 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:29:12.023 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:12.023 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 1825 2024-09-14T06:29:12.023 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:12.023 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:29:12.023 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:12.023 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:29:12.023 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.654813+0000", 2024-09-14T06:29:12.023 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:12.023 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:12.024 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:12.024 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:29:12.024 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.654813+0000", 2024-09-14T06:29:12.024 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:12.024 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:12.024 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:12.024 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:29:12.024 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.654813+0000", 2024-09-14T06:29:12.024 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.11e-07 2024-09-14T06:29:12.024 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:12.024 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:12.024 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:29:12.024 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.654814+0000", 2024-09-14T06:29:12.025 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.511e-06 2024-09-14T06:29:12.025 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:12.025 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:12.025 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:29:12.025 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.654817+0000", 2024-09-14T06:29:12.025 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.0580000000000002e-06 2024-09-14T06:29:12.025 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:12.025 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:12.025 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:29:12.025 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.654820+0000", 2024-09-14T06:29:12.025 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.4530000000000001e-06 2024-09-14T06:29:12.025 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:12.025 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:12.026 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:29:12.026 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.676571+0000", 2024-09-14T06:29:12.026 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.021750698999999998 2024-09-14T06:29:12.026 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:12.026 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:12.026 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:29:12.026 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.676617+0000", 2024-09-14T06:29:12.026 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.6226000000000001e-05 2024-09-14T06:29:12.026 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:12.026 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:12.026 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:29:12.026 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.676678+0000", 2024-09-14T06:29:12.026 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.1639999999999999e-05 2024-09-14T06:29:12.026 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:12.027 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:29:12.027 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:12.027 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:12.027 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:12.027 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:1826 3.3 3:c0711257:::rbd_data.112d7dfbd766.000000000000001a:5 [read 405504~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-14T06:29:12.027 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:08.654823+0000", 2024-09-14T06:29:12.027 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 3.2952882959999998, 2024-09-14T06:29:12.027 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.022032890999999999, 2024-09-14T06:29:12.027 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:29:12.027 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:29:12.027 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:29:12.027 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:29:12.027 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:29:12.027 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:12.027 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 1826 2024-09-14T06:29:12.028 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:12.028 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:29:12.028 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:12.028 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:29:12.028 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.654823+0000", 2024-09-14T06:29:12.028 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:12.028 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:12.028 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:12.028 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:29:12.028 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.654823+0000", 2024-09-14T06:29:12.028 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:12.028 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:12.028 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:12.028 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:29:12.028 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.654823+0000", 2024-09-14T06:29:12.029 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.1299999999999999e-07 2024-09-14T06:29:12.029 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:12.029 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:12.029 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:29:12.029 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.654824+0000", 2024-09-14T06:29:12.029 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.6300000000000004e-07 2024-09-14T06:29:12.029 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:12.029 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:12.029 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:29:12.029 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.654827+0000", 2024-09-14T06:29:12.029 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.0230000000000001e-06 2024-09-14T06:29:12.029 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:12.029 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:12.029 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:29:12.030 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.654830+0000", 2024-09-14T06:29:12.030 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.7650000000000002e-06 2024-09-14T06:29:12.030 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:12.030 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:12.030 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:29:12.030 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.676702+0000", 2024-09-14T06:29:12.030 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.021871561000000001 2024-09-14T06:29:12.030 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:12.030 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:12.030 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:29:12.030 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.676737+0000", 2024-09-14T06:29:12.030 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.5506000000000002e-05 2024-09-14T06:29:12.030 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:12.031 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:12.031 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:29:12.031 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.676856+0000", 2024-09-14T06:29:12.031 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00011896 2024-09-14T06:29:12.031 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:12.031 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:29:12.031 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:12.031 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:12.031 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:12.031 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:1827 3.3 3:c0711257:::rbd_data.112d7dfbd766.000000000000001a:5 [read 421888~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-14T06:29:12.031 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:08.654833+0000", 2024-09-14T06:29:12.031 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 3.2952780179999999, 2024-09-14T06:29:12.031 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.022281015000000001, 2024-09-14T06:29:12.031 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:29:12.032 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:29:12.032 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:29:12.032 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:29:12.032 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:29:12.032 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:12.032 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 1827 2024-09-14T06:29:12.032 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:12.032 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:29:12.032 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:12.032 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:29:12.032 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.654833+0000", 2024-09-14T06:29:12.032 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:12.032 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:12.033 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:12.033 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:29:12.033 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.654833+0000", 2024-09-14T06:29:12.033 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:12.033 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:12.033 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:12.033 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:29:12.033 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.654834+0000", 2024-09-14T06:29:12.033 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.7e-07 2024-09-14T06:29:12.033 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:12.033 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:12.033 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:29:12.033 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.654835+0000", 2024-09-14T06:29:12.033 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.3900000000000003e-07 2024-09-14T06:29:12.034 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:12.034 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:12.034 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:29:12.034 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.654838+0000", 2024-09-14T06:29:12.034 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.909e-06 2024-09-14T06:29:12.034 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:12.034 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:12.034 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:29:12.034 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.654840+0000", 2024-09-14T06:29:12.034 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.4739999999999999e-06 2024-09-14T06:29:12.034 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:12.034 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:12.034 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:29:12.035 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.676880+0000", 2024-09-14T06:29:12.035 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.022039741000000002 2024-09-14T06:29:12.035 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:12.035 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:12.035 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:29:12.035 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.676926+0000", 2024-09-14T06:29:12.035 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.6344000000000002e-05 2024-09-14T06:29:12.035 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:12.035 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:12.035 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:29:12.035 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.677114+0000", 2024-09-14T06:29:12.035 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.000188238 2024-09-14T06:29:12.035 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:12.035 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:29:12.035 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:12.036 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:12.036 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:12.036 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:1828 3.3 3:c0711257:::rbd_data.112d7dfbd766.000000000000001a:5 [read 438272~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-14T06:29:12.036 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:08.654855+0000", 2024-09-14T06:29:12.036 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 3.2952568680000001, 2024-09-14T06:29:12.036 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.022438038, 2024-09-14T06:29:12.036 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:29:12.036 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:29:12.036 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:29:12.036 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:29:12.036 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:29:12.036 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:12.036 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 1828 2024-09-14T06:29:12.037 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:12.037 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:29:12.037 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:12.037 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:29:12.037 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.654855+0000", 2024-09-14T06:29:12.037 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:12.037 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:12.037 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:12.037 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:29:12.037 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.654855+0000", 2024-09-14T06:29:12.037 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:12.037 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:12.037 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:12.037 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:29:12.038 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.654855+0000", 2024-09-14T06:29:12.038 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.9499999999999998e-07 2024-09-14T06:29:12.038 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:12.038 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:12.038 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:29:12.038 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.654856+0000", 2024-09-14T06:29:12.038 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.403e-06 2024-09-14T06:29:12.038 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:12.038 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:12.038 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:29:12.038 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.654869+0000", 2024-09-14T06:29:12.038 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.2459999999999999e-05 2024-09-14T06:29:12.039 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:12.039 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:12.039 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:29:12.039 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.654872+0000", 2024-09-14T06:29:12.039 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.072e-06 2024-09-14T06:29:12.039 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:12.039 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:12.039 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:29:12.039 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.677127+0000", 2024-09-14T06:29:12.039 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.022255119 2024-09-14T06:29:12.039 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:12.039 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:12.039 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:29:12.040 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.677179+0000", 2024-09-14T06:29:12.040 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.1736000000000001e-05 2024-09-14T06:29:12.040 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:12.040 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:12.040 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:29:12.040 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.677293+0000", 2024-09-14T06:29:12.040 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.000113953 2024-09-14T06:29:12.040 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:12.040 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:29:12.040 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:12.040 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:12.040 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:29:12.040 INFO:teuthology.orchestra.run.smithi138.stdout:} 2024-09-14T06:29:12.042 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-14T06:29:12.042 INFO:tasks.ceph.osd.3.smithi094.stderr:2024-09-14T06:29:12.040+0000 7feeeafe8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33655) UID: 0 2024-09-14T06:29:12.142 INFO:tasks.ceph.osd.5.smithi138.stderr:2024-09-14T06:29:12.141+0000 7fe13746f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33618) UID: 0 2024-09-14T06:29:12.184 INFO:teuthology.orchestra.run.smithi138.stdout:{ 2024-09-14T06:29:12.184 INFO:teuthology.orchestra.run.smithi138.stdout: "ops": [], 2024-09-14T06:29:12.184 INFO:teuthology.orchestra.run.smithi138.stdout: "num_ops": 0 2024-09-14T06:29:12.184 INFO:teuthology.orchestra.run.smithi138.stdout:} 2024-09-14T06:29:12.184 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-14T06:29:12.244 INFO:tasks.ceph.osd.3.smithi094.stderr:2024-09-14T06:29:12.243+0000 7feeeafe8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33655) UID: 0 2024-09-14T06:29:12.327 INFO:teuthology.orchestra.run.smithi138.stdout:{ 2024-09-14T06:29:12.327 INFO:teuthology.orchestra.run.smithi138.stdout: "ops": [], 2024-09-14T06:29:12.327 INFO:teuthology.orchestra.run.smithi138.stdout: "complaint_time": 30, 2024-09-14T06:29:12.327 INFO:teuthology.orchestra.run.smithi138.stdout: "num_blocked_ops": 0 2024-09-14T06:29:12.327 INFO:teuthology.orchestra.run.smithi138.stdout:} 2024-09-14T06:29:12.336 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-14T06:29:12.379 INFO:tasks.ceph.osd.6.smithi138.stderr:2024-09-14T06:29:12.378+0000 7f8cd8849640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33620) UID: 0 2024-09-14T06:29:12.441 INFO:teuthology.orchestra.run.smithi138.stdout:{ 2024-09-14T06:29:12.441 INFO:teuthology.orchestra.run.smithi138.stdout: "size": 20, 2024-09-14T06:29:12.441 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 600, 2024-09-14T06:29:12.441 INFO:teuthology.orchestra.run.smithi138.stdout: "ops": [ 2024-09-14T06:29:12.441 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:12.441 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:3256 3.5 3:a6acff7e:::rbd_data.112d7dfbd766.0000000000000078:5 [read 110592~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-14T06:29:12.441 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:08.738879+0000", 2024-09-14T06:29:12.441 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 3.70042289, 2024-09-14T06:29:12.441 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.011432067000000001, 2024-09-14T06:29:12.442 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:29:12.442 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:29:12.442 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:29:12.442 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:29:12.442 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:29:12.442 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:12.442 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 3256 2024-09-14T06:29:12.442 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:12.442 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:29:12.442 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:12.442 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:29:12.442 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.738879+0000", 2024-09-14T06:29:12.442 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:12.443 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:12.443 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:12.443 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:29:12.443 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.738879+0000", 2024-09-14T06:29:12.443 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:12.443 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:12.443 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:12.443 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:29:12.443 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.738879+0000", 2024-09-14T06:29:12.443 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.8099999999999998e-07 2024-09-14T06:29:12.443 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:12.443 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:12.443 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:29:12.443 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.738885+0000", 2024-09-14T06:29:12.444 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.3199999999999999e-06 2024-09-14T06:29:12.444 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:12.444 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:12.444 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:29:12.444 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.738891+0000", 2024-09-14T06:29:12.444 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.3149999999999997e-06 2024-09-14T06:29:12.444 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:12.444 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:12.444 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:29:12.444 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.738895+0000", 2024-09-14T06:29:12.444 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.478e-06 2024-09-14T06:29:12.444 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:12.444 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:12.444 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:29:12.445 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.750221+0000", 2024-09-14T06:29:12.445 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.011326166 2024-09-14T06:29:12.445 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:12.445 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:12.445 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:29:12.445 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.750260+0000", 2024-09-14T06:29:12.445 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.9239999999999997e-05 2024-09-14T06:29:12.445 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:12.445 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:12.445 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:29:12.445 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.750311+0000", 2024-09-14T06:29:12.445 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.1167e-05 2024-09-14T06:29:12.445 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:12.446 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:29:12.446 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:12.446 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:12.446 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:12.446 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:3257 3.5 3:a6acff7e:::rbd_data.112d7dfbd766.0000000000000078:5 [read 126976~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-14T06:29:12.446 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:08.738902+0000", 2024-09-14T06:29:12.446 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 3.7004001980000001, 2024-09-14T06:29:12.446 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.011528261999999999, 2024-09-14T06:29:12.446 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:29:12.446 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:29:12.446 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:29:12.446 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:29:12.446 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:29:12.446 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:12.446 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 3257 2024-09-14T06:29:12.447 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:12.447 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:29:12.447 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:12.447 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:29:12.447 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.738902+0000", 2024-09-14T06:29:12.447 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:12.447 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:12.447 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:12.447 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:29:12.447 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.738902+0000", 2024-09-14T06:29:12.447 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:12.447 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:12.447 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:12.447 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:29:12.447 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.738902+0000", 2024-09-14T06:29:12.448 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.1899999999999998e-07 2024-09-14T06:29:12.448 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:12.448 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:12.448 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:29:12.448 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.738904+0000", 2024-09-14T06:29:12.448 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.872e-06 2024-09-14T06:29:12.448 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:12.448 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:12.448 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:29:12.448 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.738910+0000", 2024-09-14T06:29:12.448 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.8749999999999997e-06 2024-09-14T06:29:12.448 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:12.448 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:12.448 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:29:12.449 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.738918+0000", 2024-09-14T06:29:12.449 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.2570000000000007e-06 2024-09-14T06:29:12.449 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:12.449 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:12.449 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:29:12.449 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.750331+0000", 2024-09-14T06:29:12.449 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.011412813000000001 2024-09-14T06:29:12.449 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:12.449 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:12.449 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:29:12.449 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.750373+0000", 2024-09-14T06:29:12.449 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.1754e-05 2024-09-14T06:29:12.449 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:12.449 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:12.449 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:29:12.450 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.750430+0000", 2024-09-14T06:29:12.450 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.7272000000000003e-05 2024-09-14T06:29:12.450 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:12.450 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:29:12.450 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:12.450 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:12.450 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:12.450 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:3258 3.5 3:a6acff7e:::rbd_data.112d7dfbd766.0000000000000078:5 [read 143360~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-14T06:29:12.450 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:08.738925+0000", 2024-09-14T06:29:12.450 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 3.700376501, 2024-09-14T06:29:12.450 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.01161296, 2024-09-14T06:29:12.450 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:29:12.450 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:29:12.450 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:29:12.451 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:29:12.451 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:29:12.451 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:12.451 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 3258 2024-09-14T06:29:12.451 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:12.451 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:29:12.451 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:12.451 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:29:12.451 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.738925+0000", 2024-09-14T06:29:12.451 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:12.451 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:12.451 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:12.451 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:29:12.451 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.738925+0000", 2024-09-14T06:29:12.452 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:12.452 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:12.452 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:12.452 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:29:12.452 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.738926+0000", 2024-09-14T06:29:12.452 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.1100000000000001e-07 2024-09-14T06:29:12.452 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:12.452 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:12.452 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:29:12.452 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.738928+0000", 2024-09-14T06:29:12.452 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.0679999999999999e-06 2024-09-14T06:29:12.452 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:12.452 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:12.452 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:29:12.453 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.738934+0000", 2024-09-14T06:29:12.453 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.054e-06 2024-09-14T06:29:12.453 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:12.453 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:12.453 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:29:12.453 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.738938+0000", 2024-09-14T06:29:12.453 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.4429999999999996e-06 2024-09-14T06:29:12.453 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:12.453 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:12.453 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:29:12.453 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.750448+0000", 2024-09-14T06:29:12.453 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.011510016 2024-09-14T06:29:12.453 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:12.453 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:12.454 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:29:12.454 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.750488+0000", 2024-09-14T06:29:12.454 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.9402e-05 2024-09-14T06:29:12.454 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:12.454 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:12.454 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:29:12.454 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.750538+0000", 2024-09-14T06:29:12.454 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.0565999999999997e-05 2024-09-14T06:29:12.454 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:12.454 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:29:12.454 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:12.454 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:12.454 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:12.454 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:3259 3.5 3:a6acff7e:::rbd_data.112d7dfbd766.0000000000000078:5 [read 159744~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-14T06:29:12.455 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:08.738946+0000", 2024-09-14T06:29:12.455 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 3.7003554899999997, 2024-09-14T06:29:12.455 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.011711767999999999, 2024-09-14T06:29:12.455 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:29:12.455 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:29:12.455 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:29:12.455 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:29:12.455 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:29:12.455 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:12.455 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 3259 2024-09-14T06:29:12.455 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:12.455 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:29:12.455 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:12.455 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:29:12.456 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.738946+0000", 2024-09-14T06:29:12.456 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:12.456 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:12.456 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:12.456 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:29:12.456 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.738946+0000", 2024-09-14T06:29:12.456 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:12.456 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:12.456 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:12.456 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:29:12.456 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.738947+0000", 2024-09-14T06:29:12.456 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.5400000000000002e-07 2024-09-14T06:29:12.456 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:12.457 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:12.457 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:29:12.457 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.738949+0000", 2024-09-14T06:29:12.457 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.0260000000000002e-06 2024-09-14T06:29:12.457 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:12.457 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:12.457 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:29:12.457 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.738955+0000", 2024-09-14T06:29:12.457 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.3709999999999996e-06 2024-09-14T06:29:12.457 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:12.457 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:12.457 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:29:12.457 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.738960+0000", 2024-09-14T06:29:12.457 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.7369999999999997e-06 2024-09-14T06:29:12.458 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:12.458 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:12.458 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:29:12.458 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.750560+0000", 2024-09-14T06:29:12.458 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.011600103 2024-09-14T06:29:12.458 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:12.458 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:12.458 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:29:12.458 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.750605+0000", 2024-09-14T06:29:12.458 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.4799999999999998e-05 2024-09-14T06:29:12.458 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:12.458 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:12.458 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:29:12.458 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.750658+0000", 2024-09-14T06:29:12.458 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.3377000000000001e-05 2024-09-14T06:29:12.459 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:12.459 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:29:12.459 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:12.459 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:12.459 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:12.459 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:3260 3.5 3:a6acff7e:::rbd_data.112d7dfbd766.0000000000000078:5 [read 176128~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-14T06:29:12.459 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:08.738967+0000", 2024-09-14T06:29:12.459 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 3.700334689, 2024-09-14T06:29:12.459 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.011809811, 2024-09-14T06:29:12.459 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:29:12.459 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:29:12.459 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:29:12.459 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:29:12.459 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:29:12.460 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:12.460 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 3260 2024-09-14T06:29:12.460 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:12.460 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:29:12.460 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:12.460 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:29:12.460 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.738967+0000", 2024-09-14T06:29:12.460 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:12.460 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:12.460 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:12.460 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:29:12.460 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.738967+0000", 2024-09-14T06:29:12.461 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:12.461 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:12.461 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:12.461 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:29:12.461 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.738968+0000", 2024-09-14T06:29:12.461 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.2599999999999998e-07 2024-09-14T06:29:12.461 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:12.461 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:12.461 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:29:12.461 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.738970+0000", 2024-09-14T06:29:12.461 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.0729999999999999e-06 2024-09-14T06:29:12.461 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:12.461 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:12.461 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:29:12.462 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.738976+0000", 2024-09-14T06:29:12.462 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.6830000000000002e-06 2024-09-14T06:29:12.462 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:12.462 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:12.462 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:29:12.462 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.738981+0000", 2024-09-14T06:29:12.462 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.6299999999999997e-06 2024-09-14T06:29:12.462 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:12.462 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:12.462 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:29:12.462 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.750681+0000", 2024-09-14T06:29:12.462 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.011700093999999999 2024-09-14T06:29:12.462 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:12.462 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:12.463 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:29:12.463 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.750724+0000", 2024-09-14T06:29:12.463 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.295e-05 2024-09-14T06:29:12.463 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:12.463 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:12.463 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:29:12.463 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.750777+0000", 2024-09-14T06:29:12.463 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.2954999999999997e-05 2024-09-14T06:29:12.463 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:12.463 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:29:12.463 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:12.463 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:12.463 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:12.464 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:3261 3.5 3:a6acff7e:::rbd_data.112d7dfbd766.0000000000000078:5 [read 192512~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-14T06:29:12.464 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:08.738988+0000", 2024-09-14T06:29:12.464 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 3.7003143120000002, 2024-09-14T06:29:12.464 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.011907709000000001, 2024-09-14T06:29:12.464 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:29:12.464 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:29:12.464 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:29:12.464 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:29:12.464 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:29:12.464 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:12.464 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 3261 2024-09-14T06:29:12.464 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:12.464 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:29:12.464 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:12.465 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:29:12.465 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.738988+0000", 2024-09-14T06:29:12.465 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:12.465 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:12.465 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:12.465 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:29:12.465 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.738988+0000", 2024-09-14T06:29:12.465 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:12.465 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:12.465 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:12.465 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:29:12.465 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.738988+0000", 2024-09-14T06:29:12.465 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.9200000000000002e-07 2024-09-14T06:29:12.466 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:12.466 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:12.466 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:29:12.466 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.738995+0000", 2024-09-14T06:29:12.466 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.8399999999999997e-06 2024-09-14T06:29:12.466 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:12.466 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:12.466 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:29:12.466 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.739001+0000", 2024-09-14T06:29:12.466 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.2319999999999998e-06 2024-09-14T06:29:12.466 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:12.466 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:12.466 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:29:12.466 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.739007+0000", 2024-09-14T06:29:12.467 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.57e-06 2024-09-14T06:29:12.467 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:12.467 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:12.467 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:29:12.467 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.750798+0000", 2024-09-14T06:29:12.467 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.011790995 2024-09-14T06:29:12.467 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:12.467 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:12.467 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:29:12.467 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.750837+0000", 2024-09-14T06:29:12.467 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.9765999999999998e-05 2024-09-14T06:29:12.467 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:12.467 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:12.467 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:29:12.468 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.750895+0000", 2024-09-14T06:29:12.468 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.7914000000000002e-05 2024-09-14T06:29:12.468 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:12.468 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:29:12.468 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:12.468 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:12.468 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:12.468 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:3262 3.5 3:a6acff7e:::rbd_data.112d7dfbd766.0000000000000078:5 [read 208896~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-14T06:29:12.468 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:08.739013+0000", 2024-09-14T06:29:12.468 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 3.7002887229999999, 2024-09-14T06:29:12.468 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.012002934999999999, 2024-09-14T06:29:12.468 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:29:12.468 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:29:12.468 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:29:12.468 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:29:12.469 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:29:12.469 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:12.469 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 3262 2024-09-14T06:29:12.469 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:12.469 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:29:12.469 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:12.469 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:29:12.469 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.739013+0000", 2024-09-14T06:29:12.469 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:12.469 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:12.469 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:12.469 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:29:12.469 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.739013+0000", 2024-09-14T06:29:12.470 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:12.470 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:12.470 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:12.470 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:29:12.470 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.739014+0000", 2024-09-14T06:29:12.470 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.3099999999999998e-07 2024-09-14T06:29:12.470 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:12.470 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:12.470 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:29:12.470 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.739015+0000", 2024-09-14T06:29:12.470 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.818e-06 2024-09-14T06:29:12.470 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:12.470 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:12.470 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:29:12.471 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.739023+0000", 2024-09-14T06:29:12.471 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.2559999999999996e-06 2024-09-14T06:29:12.471 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:12.471 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:12.471 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:29:12.471 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.739028+0000", 2024-09-14T06:29:12.471 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.7870000000000001e-06 2024-09-14T06:29:12.471 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:12.471 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:12.471 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:29:12.471 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.750921+0000", 2024-09-14T06:29:12.471 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.011893470999999999 2024-09-14T06:29:12.472 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:12.472 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:12.472 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:29:12.472 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.750964+0000", 2024-09-14T06:29:12.472 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.3229000000000002e-05 2024-09-14T06:29:12.472 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:12.472 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:12.472 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:29:12.472 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.751016+0000", 2024-09-14T06:29:12.472 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.1943000000000001e-05 2024-09-14T06:29:12.472 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:12.472 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:29:12.472 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:12.473 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:12.473 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:12.473 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:3263 3.5 3:a6acff7e:::rbd_data.112d7dfbd766.0000000000000078:5 [read 225280~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-14T06:29:12.473 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:08.739032+0000", 2024-09-14T06:29:12.473 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 3.7002695819999998, 2024-09-14T06:29:12.473 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.012101318, 2024-09-14T06:29:12.473 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:29:12.473 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:29:12.473 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:29:12.473 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:29:12.473 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:29:12.473 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:12.473 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 3263 2024-09-14T06:29:12.473 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:12.474 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:29:12.474 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:12.474 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:29:12.474 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.739032+0000", 2024-09-14T06:29:12.474 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:12.474 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:12.474 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:12.474 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:29:12.474 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.739032+0000", 2024-09-14T06:29:12.474 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:12.474 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:12.474 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:12.474 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:29:12.474 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.739033+0000", 2024-09-14T06:29:12.475 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.5800000000000001e-07 2024-09-14T06:29:12.475 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:12.475 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:12.475 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:29:12.475 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.739034+0000", 2024-09-14T06:29:12.475 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.136e-06 2024-09-14T06:29:12.475 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:12.475 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:12.475 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:29:12.475 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.739037+0000", 2024-09-14T06:29:12.475 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.2370000000000001e-06 2024-09-14T06:29:12.475 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:12.475 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:12.475 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:29:12.475 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.739040+0000", 2024-09-14T06:29:12.475 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.3739999999999998e-06 2024-09-14T06:29:12.476 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:12.476 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:12.476 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:29:12.476 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.751036+0000", 2024-09-14T06:29:12.476 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.011995178 2024-09-14T06:29:12.476 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:12.476 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:12.476 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:29:12.476 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.751078+0000", 2024-09-14T06:29:12.476 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.2519999999999999e-05 2024-09-14T06:29:12.476 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:12.476 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:12.476 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:29:12.476 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.751134+0000", 2024-09-14T06:29:12.477 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.5615000000000002e-05 2024-09-14T06:29:12.477 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:12.477 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:29:12.477 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:12.477 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:12.477 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:12.477 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:3264 3.5 3:a6acff7e:::rbd_data.112d7dfbd766.0000000000000078:5 [read 241664~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-14T06:29:12.477 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:08.739043+0000", 2024-09-14T06:29:12.477 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 3.7002585689999998, 2024-09-14T06:29:12.477 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.012198746, 2024-09-14T06:29:12.477 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:29:12.477 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:29:12.477 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:29:12.477 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:29:12.477 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:29:12.478 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:12.478 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 3264 2024-09-14T06:29:12.478 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:12.478 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:29:12.478 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:12.478 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:29:12.478 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.739043+0000", 2024-09-14T06:29:12.478 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:12.478 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:12.478 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:12.478 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:29:12.478 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.739043+0000", 2024-09-14T06:29:12.478 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:12.478 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:12.478 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:12.479 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:29:12.479 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.739044+0000", 2024-09-14T06:29:12.479 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.28e-07 2024-09-14T06:29:12.479 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:12.479 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:12.479 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:29:12.479 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.739045+0000", 2024-09-14T06:29:12.479 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.78e-07 2024-09-14T06:29:12.479 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:12.479 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:12.479 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:29:12.479 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.739048+0000", 2024-09-14T06:29:12.479 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.9950000000000001e-06 2024-09-14T06:29:12.479 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:12.480 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:12.480 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:29:12.480 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.739051+0000", 2024-09-14T06:29:12.480 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.0340000000000001e-06 2024-09-14T06:29:12.480 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:12.480 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:12.480 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:29:12.480 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.751154+0000", 2024-09-14T06:29:12.480 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.012103541000000001 2024-09-14T06:29:12.480 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:12.480 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:12.480 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:29:12.480 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.751193+0000", 2024-09-14T06:29:12.480 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.9048000000000003e-05 2024-09-14T06:29:12.481 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:12.481 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:12.481 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:29:12.481 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.751242+0000", 2024-09-14T06:29:12.481 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.8921999999999997e-05 2024-09-14T06:29:12.481 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:12.481 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:29:12.481 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:12.481 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:12.481 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:12.481 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:3265 3.5 3:a6acff7e:::rbd_data.112d7dfbd766.0000000000000078:5 [read 258048~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-14T06:29:12.482 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:08.739053+0000", 2024-09-14T06:29:12.482 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 3.7002484490000001, 2024-09-14T06:29:12.482 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.01230612, 2024-09-14T06:29:12.482 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:29:12.482 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:29:12.482 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:29:12.482 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:29:12.482 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:29:12.482 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:12.482 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 3265 2024-09-14T06:29:12.482 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:12.482 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:29:12.482 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:12.482 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:29:12.482 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.739053+0000", 2024-09-14T06:29:12.482 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:12.483 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:12.483 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:12.483 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:29:12.483 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.739053+0000", 2024-09-14T06:29:12.483 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:12.483 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:12.483 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:12.483 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:29:12.483 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.739054+0000", 2024-09-14T06:29:12.483 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.3999999999999998e-07 2024-09-14T06:29:12.483 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:12.483 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:12.483 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:29:12.483 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.739055+0000", 2024-09-14T06:29:12.483 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.2099999999999995e-07 2024-09-14T06:29:12.484 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:12.484 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:12.484 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:29:12.484 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.739058+0000", 2024-09-14T06:29:12.484 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.9340000000000002e-06 2024-09-14T06:29:12.484 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:12.484 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:12.484 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:29:12.484 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.739060+0000", 2024-09-14T06:29:12.484 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.8030000000000001e-06 2024-09-14T06:29:12.484 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:12.484 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:12.484 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:29:12.484 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.751263+0000", 2024-09-14T06:29:12.484 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.012202821000000001 2024-09-14T06:29:12.485 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:12.485 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:12.485 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:29:12.485 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.751308+0000", 2024-09-14T06:29:12.485 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.4802999999999998e-05 2024-09-14T06:29:12.485 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:12.485 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:12.485 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:29:12.485 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.751360+0000", 2024-09-14T06:29:12.485 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.1598000000000003e-05 2024-09-14T06:29:12.485 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:12.485 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:29:12.485 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:12.485 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:12.485 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:12.485 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:3266 3.5 3:a6acff7e:::rbd_data.112d7dfbd766.0000000000000078:5 [read 274432~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-14T06:29:12.486 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:08.739063+0000", 2024-09-14T06:29:12.486 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 3.7002388590000002, 2024-09-14T06:29:12.486 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.012402339, 2024-09-14T06:29:12.486 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:29:12.486 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:29:12.486 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:29:12.486 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:29:12.486 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:29:12.486 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:12.486 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 3266 2024-09-14T06:29:12.486 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:12.486 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:29:12.486 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:12.486 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:29:12.487 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.739063+0000", 2024-09-14T06:29:12.487 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:12.487 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:12.487 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:12.487 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:29:12.487 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.739063+0000", 2024-09-14T06:29:12.487 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:12.487 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:12.487 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:12.487 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:29:12.487 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.739063+0000", 2024-09-14T06:29:12.487 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.2000000000000001e-07 2024-09-14T06:29:12.487 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:12.487 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:12.488 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:29:12.488 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.739065+0000", 2024-09-14T06:29:12.488 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.299e-06 2024-09-14T06:29:12.488 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:12.488 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:12.488 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:29:12.488 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.739069+0000", 2024-09-14T06:29:12.488 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.1309999999999999e-06 2024-09-14T06:29:12.488 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:12.488 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:12.488 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:29:12.488 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.739071+0000", 2024-09-14T06:29:12.488 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.4279999999999999e-06 2024-09-14T06:29:12.488 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:12.489 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:12.489 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:29:12.489 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.751381+0000", 2024-09-14T06:29:12.489 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.012309394 2024-09-14T06:29:12.489 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:12.489 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:12.489 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:29:12.489 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.751419+0000", 2024-09-14T06:29:12.489 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.8068000000000002e-05 2024-09-14T06:29:12.489 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:12.489 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:12.489 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:29:12.489 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.751465+0000", 2024-09-14T06:29:12.490 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.6798999999999997e-05 2024-09-14T06:29:12.490 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:12.490 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:29:12.490 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:12.490 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:12.490 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:12.490 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:3267 3.5 3:a6acff7e:::rbd_data.112d7dfbd766.0000000000000078:5 [read 290816~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-14T06:29:12.490 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:08.739074+0000", 2024-09-14T06:29:12.490 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 3.7002277019999998, 2024-09-14T06:29:12.490 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.012508533, 2024-09-14T06:29:12.490 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:29:12.490 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:29:12.490 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:29:12.491 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:29:12.491 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:29:12.491 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:12.491 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 3267 2024-09-14T06:29:12.491 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:12.491 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:29:12.491 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:12.491 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:29:12.491 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.739074+0000", 2024-09-14T06:29:12.491 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:12.491 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:12.491 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:12.491 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:29:12.491 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.739074+0000", 2024-09-14T06:29:12.491 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:12.492 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:12.492 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:12.492 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:29:12.492 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.739074+0000", 2024-09-14T06:29:12.492 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.2000000000000001e-07 2024-09-14T06:29:12.492 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:12.492 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:12.492 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:29:12.492 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.739075+0000", 2024-09-14T06:29:12.492 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.2299999999999999e-07 2024-09-14T06:29:12.492 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:12.492 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:12.492 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:29:12.492 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.739079+0000", 2024-09-14T06:29:12.493 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.6500000000000002e-06 2024-09-14T06:29:12.493 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:12.493 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:12.493 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:29:12.493 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.739083+0000", 2024-09-14T06:29:12.493 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.856e-06 2024-09-14T06:29:12.493 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:12.493 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:12.493 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:29:12.493 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.751485+0000", 2024-09-14T06:29:12.493 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.012401772 2024-09-14T06:29:12.493 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:12.493 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:12.494 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:29:12.494 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.751529+0000", 2024-09-14T06:29:12.494 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.4542000000000001e-05 2024-09-14T06:29:12.494 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:12.494 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:12.494 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:29:12.494 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.751583+0000", 2024-09-14T06:29:12.494 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.3569999999999997e-05 2024-09-14T06:29:12.494 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:12.494 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:29:12.494 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:12.494 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:12.494 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:12.494 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:3268 3.5 3:a6acff7e:::rbd_data.112d7dfbd766.0000000000000078:5 [read 307200~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-14T06:29:12.494 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:08.739088+0000", 2024-09-14T06:29:12.495 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 3.700213674, 2024-09-14T06:29:12.495 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.012618495, 2024-09-14T06:29:12.495 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:29:12.495 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:29:12.495 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:29:12.495 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:29:12.495 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:29:12.495 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:12.495 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 3268 2024-09-14T06:29:12.495 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:12.495 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:29:12.495 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:12.495 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:29:12.495 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.739088+0000", 2024-09-14T06:29:12.495 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:12.496 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:12.496 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:12.496 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:29:12.496 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.739088+0000", 2024-09-14T06:29:12.496 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:12.496 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:12.496 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:12.496 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:29:12.496 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.739089+0000", 2024-09-14T06:29:12.496 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.4400000000000001e-07 2024-09-14T06:29:12.496 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:12.496 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:12.496 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:29:12.496 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.739090+0000", 2024-09-14T06:29:12.496 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.0669999999999999e-06 2024-09-14T06:29:12.497 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:12.497 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:12.497 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:29:12.497 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.739093+0000", 2024-09-14T06:29:12.497 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.3189999999999999e-06 2024-09-14T06:29:12.497 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:12.497 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:12.497 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:29:12.497 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.739095+0000", 2024-09-14T06:29:12.497 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.5799999999999999e-06 2024-09-14T06:29:12.497 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:12.497 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:12.497 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:29:12.497 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.751604+0000", 2024-09-14T06:29:12.497 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.012508543 2024-09-14T06:29:12.498 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:12.498 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:12.498 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:29:12.498 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.751655+0000", 2024-09-14T06:29:12.498 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.0872999999999999e-05 2024-09-14T06:29:12.498 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:12.498 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:12.498 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:29:12.498 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.751707+0000", 2024-09-14T06:29:12.498 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.1869000000000001e-05 2024-09-14T06:29:12.498 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:12.498 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:29:12.499 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:12.499 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:12.499 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:12.499 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:3269 3.5 3:a6acff7e:::rbd_data.112d7dfbd766.0000000000000078:5 [read 323584~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-14T06:29:12.499 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:08.739101+0000", 2024-09-14T06:29:12.499 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 3.70020119, 2024-09-14T06:29:12.499 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.012722587, 2024-09-14T06:29:12.499 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:29:12.499 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:29:12.499 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:29:12.499 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:29:12.499 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:29:12.499 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:12.499 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 3269 2024-09-14T06:29:12.499 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:12.500 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:29:12.500 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:12.500 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:29:12.500 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.739101+0000", 2024-09-14T06:29:12.500 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:12.500 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:12.500 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:12.500 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:29:12.500 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.739101+0000", 2024-09-14T06:29:12.500 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:12.500 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:12.500 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:12.500 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:29:12.501 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.739101+0000", 2024-09-14T06:29:12.501 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.17e-07 2024-09-14T06:29:12.501 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:12.501 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:12.501 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:29:12.501 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.739102+0000", 2024-09-14T06:29:12.501 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.6899999999999996e-07 2024-09-14T06:29:12.501 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:12.501 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:12.501 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:29:12.501 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.739106+0000", 2024-09-14T06:29:12.501 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.2390000000000004e-06 2024-09-14T06:29:12.501 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:12.501 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:12.502 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:29:12.502 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.739109+0000", 2024-09-14T06:29:12.502 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.5239999999999999e-06 2024-09-14T06:29:12.502 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:12.502 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:12.502 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:29:12.502 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.751732+0000", 2024-09-14T06:29:12.502 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.012622858000000001 2024-09-14T06:29:12.502 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:12.502 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:12.502 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:29:12.502 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.751774+0000", 2024-09-14T06:29:12.502 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.2485999999999999e-05 2024-09-14T06:29:12.503 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:12.503 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:12.503 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:29:12.503 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.751823+0000", 2024-09-14T06:29:12.503 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.9293999999999999e-05 2024-09-14T06:29:12.503 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:12.503 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:29:12.503 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:12.503 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:12.503 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:12.503 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:3290 3.1d 3:bf9aaf9c:::rbd_object_map.112dcdddac74.0000000000000006:head [call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-09-14T06:29:12.503 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:08.739264+0000", 2024-09-14T06:29:12.503 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 3.700038046, 2024-09-14T06:29:12.503 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.015385322, 2024-09-14T06:29:12.503 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:29:12.504 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:29:12.504 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:29:12.504 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:29:12.504 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:29:12.504 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:12.504 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 3290 2024-09-14T06:29:12.504 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:12.504 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:29:12.504 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:12.504 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:29:12.504 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.739264+0000", 2024-09-14T06:29:12.504 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:12.504 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:12.504 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:12.505 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:29:12.505 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.739264+0000", 2024-09-14T06:29:12.505 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:12.505 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:12.505 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:12.505 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:29:12.505 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.739265+0000", 2024-09-14T06:29:12.505 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.02e-07 2024-09-14T06:29:12.505 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:12.505 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:12.508 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:29:12.508 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.739269+0000", 2024-09-14T06:29:12.508 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.5990000000000004e-06 2024-09-14T06:29:12.508 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:12.508 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:12.508 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:29:12.508 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.739278+0000", 2024-09-14T06:29:12.508 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.8109999999999992e-06 2024-09-14T06:29:12.509 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:12.509 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:12.509 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:29:12.509 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.739285+0000", 2024-09-14T06:29:12.509 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.7159999999999997e-06 2024-09-14T06:29:12.509 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:12.509 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:12.509 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:29:12.509 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.751725+0000", 2024-09-14T06:29:12.509 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.012439597 2024-09-14T06:29:12.509 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:12.509 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:12.509 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:29:12.509 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.751771+0000", 2024-09-14T06:29:12.510 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.6353999999999997e-05 2024-09-14T06:29:12.510 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:12.510 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:12.510 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "waiting for subops from 4", 2024-09-14T06:29:12.510 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.751969+0000", 2024-09-14T06:29:12.510 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00019807700000000001 2024-09-14T06:29:12.510 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:12.510 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:12.510 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:29:12.510 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.753869+0000", 2024-09-14T06:29:12.510 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.001900335 2024-09-14T06:29:12.510 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:12.510 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:12.510 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "op_commit", 2024-09-14T06:29:12.511 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.754569+0000", 2024-09-14T06:29:12.511 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00069978200000000003 2024-09-14T06:29:12.511 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:12.511 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:12.511 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "commit_sent", 2024-09-14T06:29:12.511 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.754618+0000", 2024-09-14T06:29:12.511 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.9011000000000002e-05 2024-09-14T06:29:12.511 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:12.511 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:12.511 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:29:12.511 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.754649+0000", 2024-09-14T06:29:12.511 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.1137999999999999e-05 2024-09-14T06:29:12.511 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:12.511 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:29:12.512 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:12.512 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:12.512 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:12.512 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:3294 3.1d 3:bf9aaf9c:::rbd_object_map.112dcdddac74.0000000000000006:head [call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-09-14T06:29:12.512 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:08.740037+0000", 2024-09-14T06:29:12.512 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 3.6992649169999998, 2024-09-14T06:29:12.512 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.017846711000000001, 2024-09-14T06:29:12.512 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:29:12.512 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:29:12.512 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:29:12.512 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:29:12.512 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:29:12.512 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:12.512 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 3294 2024-09-14T06:29:12.513 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:12.513 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:29:12.513 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:12.513 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:29:12.513 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.740037+0000", 2024-09-14T06:29:12.513 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:12.513 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:12.513 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:12.513 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:29:12.513 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.740037+0000", 2024-09-14T06:29:12.513 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:12.513 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:12.513 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:12.513 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:29:12.514 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.740038+0000", 2024-09-14T06:29:12.514 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.1900000000000001e-07 2024-09-14T06:29:12.514 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:12.514 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:12.514 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:29:12.514 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.740043+0000", 2024-09-14T06:29:12.514 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.9649999999999999e-06 2024-09-14T06:29:12.514 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:12.514 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:12.514 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:29:12.514 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.740054+0000", 2024-09-14T06:29:12.514 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.1299e-05 2024-09-14T06:29:12.514 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:12.514 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:12.514 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:29:12.515 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.740061+0000", 2024-09-14T06:29:12.515 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.1250000000000004e-06 2024-09-14T06:29:12.515 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:12.515 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:12.515 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:29:12.515 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.752369+0000", 2024-09-14T06:29:12.515 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.012307603 2024-09-14T06:29:12.515 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:12.515 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:12.515 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:29:12.515 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.752453+0000", 2024-09-14T06:29:12.515 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.4557999999999999e-05 2024-09-14T06:29:12.515 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:12.515 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:12.516 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "waiting for subops from 4", 2024-09-14T06:29:12.516 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.752811+0000", 2024-09-14T06:29:12.516 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00035743 2024-09-14T06:29:12.516 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:12.516 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:12.516 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "op_commit", 2024-09-14T06:29:12.516 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.754655+0000", 2024-09-14T06:29:12.516 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00184362 2024-09-14T06:29:12.516 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:12.516 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:12.516 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:29:12.516 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.757784+0000", 2024-09-14T06:29:12.516 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.003129565 2024-09-14T06:29:12.517 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:12.517 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:12.517 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "commit_sent", 2024-09-14T06:29:12.517 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.757836+0000", 2024-09-14T06:29:12.517 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.1598000000000003e-05 2024-09-14T06:29:12.517 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:12.517 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:12.517 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:29:12.517 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.757884+0000", 2024-09-14T06:29:12.517 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.8028999999999997e-05 2024-09-14T06:29:12.517 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:12.517 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:29:12.517 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:12.517 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:12.518 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:12.518 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:3303 3.1d 3:bf9aaf9c:::rbd_object_map.112dcdddac74.0000000000000006:head [call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-09-14T06:29:12.518 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:08.743444+0000", 2024-09-14T06:29:12.518 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 3.695857476, 2024-09-14T06:29:12.518 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.014637346000000001, 2024-09-14T06:29:12.518 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:29:12.518 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:29:12.518 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:29:12.518 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:29:12.518 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:29:12.518 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:12.518 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 3303 2024-09-14T06:29:12.518 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:12.518 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:29:12.518 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:12.519 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:29:12.519 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.743444+0000", 2024-09-14T06:29:12.519 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:12.519 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:12.519 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:12.519 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:29:12.519 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.743444+0000", 2024-09-14T06:29:12.519 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:12.519 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:12.519 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:12.519 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:29:12.519 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.743445+0000", 2024-09-14T06:29:12.519 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.540000000000001e-07 2024-09-14T06:29:12.519 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:12.519 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:12.520 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:29:12.520 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.743450+0000", 2024-09-14T06:29:12.520 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.9520000000000002e-06 2024-09-14T06:29:12.520 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:12.520 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:12.520 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:29:12.520 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.743462+0000", 2024-09-14T06:29:12.520 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.1674e-05 2024-09-14T06:29:12.520 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:12.520 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:12.520 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:29:12.520 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.743469+0000", 2024-09-14T06:29:12.520 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.4499999999999998e-06 2024-09-14T06:29:12.520 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:12.521 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:12.521 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:29:12.521 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.753195+0000", 2024-09-14T06:29:12.521 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.0097257260000000005 2024-09-14T06:29:12.521 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:12.521 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:12.521 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:29:12.521 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.753277+0000", 2024-09-14T06:29:12.521 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.1469000000000002e-05 2024-09-14T06:29:12.521 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:12.521 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:12.521 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "waiting for subops from 4", 2024-09-14T06:29:12.521 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.753482+0000", 2024-09-14T06:29:12.521 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00020495999999999999 2024-09-14T06:29:12.522 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:12.522 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:12.522 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:29:12.522 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.757930+0000", 2024-09-14T06:29:12.522 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.0044481640000000001 2024-09-14T06:29:12.522 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:12.522 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:12.522 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "op_commit", 2024-09-14T06:29:12.522 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.757992+0000", 2024-09-14T06:29:12.522 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.2648000000000001e-05 2024-09-14T06:29:12.522 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:12.522 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:12.522 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "commit_sent", 2024-09-14T06:29:12.522 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.758040+0000", 2024-09-14T06:29:12.522 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.7821000000000002e-05 2024-09-14T06:29:12.522 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:12.523 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:12.523 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:29:12.523 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.758082+0000", 2024-09-14T06:29:12.523 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.1527999999999999e-05 2024-09-14T06:29:12.523 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:12.523 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:29:12.523 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:12.523 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:12.523 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:12.523 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:3331 3.1d 3:bf9aaf9c:::rbd_object_map.112dcdddac74.0000000000000006:head [call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-09-14T06:29:12.523 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:08.745245+0000", 2024-09-14T06:29:12.523 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 3.6940571950000001, 2024-09-14T06:29:12.523 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.012916459999999999, 2024-09-14T06:29:12.523 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:29:12.524 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:29:12.524 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:29:12.524 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:29:12.524 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:29:12.524 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:12.524 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 3331 2024-09-14T06:29:12.524 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:12.524 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:29:12.524 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:12.524 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:29:12.524 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.745245+0000", 2024-09-14T06:29:12.524 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:12.524 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:12.524 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:12.524 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:29:12.524 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.745245+0000", 2024-09-14T06:29:12.525 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:12.525 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:12.525 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:12.525 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:29:12.525 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.745245+0000", 2024-09-14T06:29:12.525 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.5400000000000001e-07 2024-09-14T06:29:12.525 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:12.525 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:12.525 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:29:12.525 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.745250+0000", 2024-09-14T06:29:12.525 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.9230000000000001e-06 2024-09-14T06:29:12.525 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:12.525 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:12.525 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:29:12.525 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.745259+0000", 2024-09-14T06:29:12.526 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.5459999999999992e-06 2024-09-14T06:29:12.526 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:12.526 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:12.526 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:29:12.526 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.745264+0000", 2024-09-14T06:29:12.526 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.9609999999999996e-06 2024-09-14T06:29:12.526 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:12.526 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:12.526 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:29:12.526 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.753892+0000", 2024-09-14T06:29:12.526 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.0086279670000000003 2024-09-14T06:29:12.526 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:12.526 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:12.526 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:29:12.526 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.753948+0000", 2024-09-14T06:29:12.527 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.6036999999999999e-05 2024-09-14T06:29:12.527 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:12.527 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:12.527 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "waiting for subops from 4", 2024-09-14T06:29:12.527 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.754216+0000", 2024-09-14T06:29:12.527 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00026780199999999998 2024-09-14T06:29:12.527 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:12.527 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:12.527 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:29:12.527 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.757963+0000", 2024-09-14T06:29:12.527 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.0037473350000000001 2024-09-14T06:29:12.527 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:12.527 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:12.527 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "op_commit", 2024-09-14T06:29:12.527 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.758091+0000", 2024-09-14T06:29:12.527 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00012849000000000001 2024-09-14T06:29:12.528 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:12.528 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:12.528 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "commit_sent", 2024-09-14T06:29:12.528 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.758128+0000", 2024-09-14T06:29:12.528 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.6350999999999998e-05 2024-09-14T06:29:12.528 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:12.528 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:12.528 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:29:12.528 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.758161+0000", 2024-09-14T06:29:12.528 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.3494000000000002e-05 2024-09-14T06:29:12.528 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:12.528 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:29:12.528 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:12.528 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:12.528 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:12.529 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:3332 3.1d 3:bf9aaf9c:::rbd_object_map.112dcdddac74.0000000000000006:head [call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-09-14T06:29:12.529 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:08.746143+0000", 2024-09-14T06:29:12.529 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 3.6931591240000001, 2024-09-14T06:29:12.529 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.012083902000000001, 2024-09-14T06:29:12.529 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:29:12.529 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:29:12.529 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:29:12.529 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:29:12.529 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:29:12.529 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:12.529 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 3332 2024-09-14T06:29:12.529 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:12.529 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:29:12.529 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:12.530 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:29:12.530 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.746143+0000", 2024-09-14T06:29:12.530 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:12.530 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:12.530 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:12.530 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:29:12.530 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.746143+0000", 2024-09-14T06:29:12.530 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:12.530 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:12.530 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:12.530 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:29:12.530 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.746144+0000", 2024-09-14T06:29:12.530 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.5499999999999997e-07 2024-09-14T06:29:12.530 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:12.530 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:12.530 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:29:12.531 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.746149+0000", 2024-09-14T06:29:12.531 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.2039999999999996e-06 2024-09-14T06:29:12.531 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:12.531 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:12.531 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:29:12.531 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.746168+0000", 2024-09-14T06:29:12.531 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.8678999999999999e-05 2024-09-14T06:29:12.531 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:12.531 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:12.531 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:29:12.531 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.746176+0000", 2024-09-14T06:29:12.531 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.0439999999999996e-06 2024-09-14T06:29:12.531 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:12.531 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:12.531 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:29:12.532 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.754692+0000", 2024-09-14T06:29:12.532 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.0085166459999999992 2024-09-14T06:29:12.532 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:12.532 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:12.532 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:29:12.532 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.754746+0000", 2024-09-14T06:29:12.532 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.3375000000000003e-05 2024-09-14T06:29:12.532 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:12.532 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:12.532 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "waiting for subops from 4", 2024-09-14T06:29:12.532 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.754972+0000", 2024-09-14T06:29:12.532 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.000226586 2024-09-14T06:29:12.532 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:12.532 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:12.532 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:29:12.532 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.757985+0000", 2024-09-14T06:29:12.533 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.003012696 2024-09-14T06:29:12.533 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:12.533 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:12.533 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "op_commit", 2024-09-14T06:29:12.533 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.758171+0000", 2024-09-14T06:29:12.533 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00018605800000000001 2024-09-14T06:29:12.533 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:12.533 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:12.533 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "commit_sent", 2024-09-14T06:29:12.533 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.758195+0000", 2024-09-14T06:29:12.533 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.3591000000000002e-05 2024-09-14T06:29:12.533 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:12.533 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:12.533 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:29:12.533 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.758227+0000", 2024-09-14T06:29:12.534 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.2168000000000001e-05 2024-09-14T06:29:12.534 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:12.534 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:29:12.534 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:12.534 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:12.534 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:12.534 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:3334 3.1d 3:bf9aaf9c:::rbd_object_map.112dcdddac74.0000000000000006:head [call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-09-14T06:29:12.534 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:08.747923+0000", 2024-09-14T06:29:12.534 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 3.6913789499999998, 2024-09-14T06:29:12.534 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.012298584, 2024-09-14T06:29:12.534 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:29:12.534 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:29:12.534 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:29:12.534 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:29:12.535 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:29:12.535 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:12.535 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 3334 2024-09-14T06:29:12.535 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:12.535 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:29:12.535 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:12.535 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:29:12.535 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.747923+0000", 2024-09-14T06:29:12.535 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:12.535 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:12.535 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:12.535 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:29:12.535 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.747923+0000", 2024-09-14T06:29:12.535 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:12.535 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:12.536 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:12.536 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:29:12.536 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.747924+0000", 2024-09-14T06:29:12.536 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.2500000000000004e-07 2024-09-14T06:29:12.536 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:12.536 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:12.536 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:29:12.536 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.747929+0000", 2024-09-14T06:29:12.536 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.6909999999999996e-06 2024-09-14T06:29:12.536 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:12.536 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:12.536 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:29:12.536 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.747946+0000", 2024-09-14T06:29:12.536 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.7337e-05 2024-09-14T06:29:12.536 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:12.536 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:12.537 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:29:12.537 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.747954+0000", 2024-09-14T06:29:12.537 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.9959999999999993e-06 2024-09-14T06:29:12.537 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:12.537 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:12.537 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:29:12.537 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.756177+0000", 2024-09-14T06:29:12.537 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.0082233319999999999 2024-09-14T06:29:12.537 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:12.537 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:12.537 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:29:12.537 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.756266+0000", 2024-09-14T06:29:12.537 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.8252999999999997e-05 2024-09-14T06:29:12.537 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:12.537 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:12.538 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "waiting for subops from 4", 2024-09-14T06:29:12.538 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.756514+0000", 2024-09-14T06:29:12.538 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00024885599999999999 2024-09-14T06:29:12.538 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:12.538 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:12.538 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:29:12.538 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.759928+0000", 2024-09-14T06:29:12.538 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.0034139410000000002 2024-09-14T06:29:12.538 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:12.538 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:12.538 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "op_commit", 2024-09-14T06:29:12.538 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.760138+0000", 2024-09-14T06:29:12.538 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.000209545 2024-09-14T06:29:12.538 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:12.538 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:12.538 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "commit_sent", 2024-09-14T06:29:12.539 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.760180+0000", 2024-09-14T06:29:12.539 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.2212999999999997e-05 2024-09-14T06:29:12.539 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:12.539 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:12.539 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:29:12.539 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.760222+0000", 2024-09-14T06:29:12.539 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.1495000000000002e-05 2024-09-14T06:29:12.539 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:12.539 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:29:12.539 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:12.539 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:12.539 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:29:12.539 INFO:teuthology.orchestra.run.smithi138.stdout:} 2024-09-14T06:29:12.541 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-14T06:29:12.541 INFO:tasks.ceph.osd.1.smithi094.stderr:2024-09-14T06:29:12.540+0000 7f328e47a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33641) UID: 0 2024-09-14T06:29:12.617 INFO:teuthology.orchestra.run.smithi094.stderr:noscrub is set 2024-09-14T06:29:12.677 INFO:tasks.ceph.osd.2.smithi094.stderr:2024-09-14T06:29:12.676+0000 7faf3269b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33649) UID: 0 2024-09-14T06:29:12.696 INFO:teuthology.orchestra.run.smithi138.stdout:{ 2024-09-14T06:29:12.696 INFO:teuthology.orchestra.run.smithi138.stdout: "ops": [], 2024-09-14T06:29:12.696 INFO:teuthology.orchestra.run.smithi138.stdout: "num_ops": 0 2024-09-14T06:29:12.696 INFO:teuthology.orchestra.run.smithi138.stdout:} 2024-09-14T06:29:12.705 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-14T06:29:12.742 INFO:tasks.ceph.osd.2.smithi094.stderr:2024-09-14T06:29:12.741+0000 7faf3269b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33649) UID: 0 2024-09-14T06:29:12.806 INFO:teuthology.orchestra.run.smithi138.stdout:{ 2024-09-14T06:29:12.806 INFO:teuthology.orchestra.run.smithi138.stdout: "ops": [], 2024-09-14T06:29:12.806 INFO:teuthology.orchestra.run.smithi138.stdout: "complaint_time": 30, 2024-09-14T06:29:12.806 INFO:teuthology.orchestra.run.smithi138.stdout: "num_blocked_ops": 0 2024-09-14T06:29:12.807 INFO:teuthology.orchestra.run.smithi138.stdout:} 2024-09-14T06:29:12.815 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-14T06:29:12.858 INFO:tasks.ceph.osd.4.smithi138.stderr:2024-09-14T06:29:12.857+0000 7f6c653c8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33615) UID: 0 2024-09-14T06:29:12.918 INFO:teuthology.orchestra.run.smithi138.stdout:{ 2024-09-14T06:29:12.919 INFO:teuthology.orchestra.run.smithi138.stdout: "size": 20, 2024-09-14T06:29:12.919 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 600, 2024-09-14T06:29:12.919 INFO:teuthology.orchestra.run.smithi138.stdout: "ops": [ 2024-09-14T06:29:12.919 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:12.919 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:1484 3.a 3:57c28b0f:::rbd_data.112d7dfbd766.000000000000000a:5 [read 208896~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-14T06:29:12.919 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:08.628246+0000", 2024-09-14T06:29:12.919 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 4.2887809260000003, 2024-09-14T06:29:12.919 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.018096672000000001, 2024-09-14T06:29:12.919 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:29:12.919 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:29:12.919 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:29:12.919 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:29:12.919 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:29:12.920 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:12.920 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 1484 2024-09-14T06:29:12.920 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:12.920 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:29:12.920 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:12.920 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:29:12.920 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.628246+0000", 2024-09-14T06:29:12.920 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:12.920 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:12.920 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:12.920 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:29:12.920 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.628246+0000", 2024-09-14T06:29:12.921 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:12.921 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:12.921 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:12.921 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:29:12.921 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.628246+0000", 2024-09-14T06:29:12.921 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.79e-07 2024-09-14T06:29:12.921 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:12.921 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:12.921 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:29:12.921 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.628247+0000", 2024-09-14T06:29:12.921 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.5179999999999999e-06 2024-09-14T06:29:12.921 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:12.921 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:12.921 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:29:12.922 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.628252+0000", 2024-09-14T06:29:12.922 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.0139999999999998e-06 2024-09-14T06:29:12.922 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:12.922 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:12.922 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:29:12.922 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.628257+0000", 2024-09-14T06:29:12.922 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.2429999999999999e-06 2024-09-14T06:29:12.922 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:12.922 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:12.922 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:29:12.922 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.646180+0000", 2024-09-14T06:29:12.922 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.017923136999999999 2024-09-14T06:29:12.922 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:12.922 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:12.922 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:29:12.923 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.646251+0000", 2024-09-14T06:29:12.923 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.1285000000000006e-05 2024-09-14T06:29:12.923 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:12.923 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:12.923 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:29:12.923 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.646342+0000", 2024-09-14T06:29:12.923 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.1195999999999999e-05 2024-09-14T06:29:12.923 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:12.923 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:29:12.923 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:12.923 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:12.923 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:12.923 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:1485 3.a 3:57c28b0f:::rbd_data.112d7dfbd766.000000000000000a:5 [read 225280~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-14T06:29:12.923 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:08.628268+0000", 2024-09-14T06:29:12.924 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 4.288758445, 2024-09-14T06:29:12.924 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.018257005999999999, 2024-09-14T06:29:12.924 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:29:12.924 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:29:12.924 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:29:12.924 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:29:12.924 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:29:12.924 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:12.924 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 1485 2024-09-14T06:29:12.924 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:12.924 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:29:12.924 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:12.924 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:29:12.924 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.628268+0000", 2024-09-14T06:29:12.924 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:12.925 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:12.925 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:12.925 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:29:12.925 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.628268+0000", 2024-09-14T06:29:12.925 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:12.925 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:12.925 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:12.925 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:29:12.925 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.628269+0000", 2024-09-14T06:29:12.925 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.6800000000000001e-07 2024-09-14T06:29:12.925 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:12.925 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:12.925 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:29:12.925 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.628270+0000", 2024-09-14T06:29:12.926 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.8449999999999999e-06 2024-09-14T06:29:12.926 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:12.926 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:12.926 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:29:12.926 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.628276+0000", 2024-09-14T06:29:12.926 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.998e-06 2024-09-14T06:29:12.926 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:12.926 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:12.926 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:29:12.926 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.628281+0000", 2024-09-14T06:29:12.926 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.3120000000000004e-06 2024-09-14T06:29:12.926 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:12.926 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:12.927 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:29:12.927 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.646371+0000", 2024-09-14T06:29:12.927 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.018090694000000001 2024-09-14T06:29:12.927 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:12.927 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:12.927 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:29:12.927 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.646439+0000", 2024-09-14T06:29:12.927 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.7182999999999997e-05 2024-09-14T06:29:12.927 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:12.927 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:12.927 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:29:12.927 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.646525+0000", 2024-09-14T06:29:12.927 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.6506000000000002e-05 2024-09-14T06:29:12.927 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:12.928 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:29:12.928 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:12.928 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:12.928 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:12.928 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:1486 3.a 3:57c28b0f:::rbd_data.112d7dfbd766.000000000000000a:5 [read 241664~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-14T06:29:12.928 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:08.628288+0000", 2024-09-14T06:29:12.928 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 4.2887385590000004, 2024-09-14T06:29:12.928 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.018416895999999999, 2024-09-14T06:29:12.928 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:29:12.928 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:29:12.928 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:29:12.928 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:29:12.928 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:29:12.928 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:12.928 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 1486 2024-09-14T06:29:12.929 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:12.929 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:29:12.929 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:12.929 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:29:12.929 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.628288+0000", 2024-09-14T06:29:12.929 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:12.929 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:12.929 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:12.929 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:29:12.929 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.628288+0000", 2024-09-14T06:29:12.929 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:12.929 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:12.929 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:12.929 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:29:12.930 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.628288+0000", 2024-09-14T06:29:12.930 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.9499999999999998e-07 2024-09-14T06:29:12.930 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:12.930 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:12.930 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:29:12.930 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.628290+0000", 2024-09-14T06:29:12.930 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.6840000000000001e-06 2024-09-14T06:29:12.930 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:12.930 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:12.930 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:29:12.930 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.628295+0000", 2024-09-14T06:29:12.930 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.3059999999999997e-06 2024-09-14T06:29:12.930 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:12.930 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:12.931 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:29:12.931 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.628300+0000", 2024-09-14T06:29:12.931 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.2690000000000001e-06 2024-09-14T06:29:12.931 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:12.931 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:12.931 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:29:12.931 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.646553+0000", 2024-09-14T06:29:12.931 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.018253015000000001 2024-09-14T06:29:12.931 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:12.931 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:12.931 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:29:12.931 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.646626+0000", 2024-09-14T06:29:12.931 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.2939000000000001e-05 2024-09-14T06:29:12.931 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:12.931 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:12.931 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:29:12.932 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.646705+0000", 2024-09-14T06:29:12.932 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.9388000000000006e-05 2024-09-14T06:29:12.932 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:12.932 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:29:12.932 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:12.932 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:12.932 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:12.932 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:1487 3.a 3:57c28b0f:::rbd_data.112d7dfbd766.000000000000000a:5 [read 258048~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-14T06:29:12.932 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:08.628842+0000", 2024-09-14T06:29:12.932 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 4.288184115, 2024-09-14T06:29:12.932 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.018037741999999999, 2024-09-14T06:29:12.932 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:29:12.932 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:29:12.932 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:29:12.932 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:29:12.933 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:29:12.933 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:12.933 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 1487 2024-09-14T06:29:12.933 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:12.933 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:29:12.933 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:12.933 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:29:12.933 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.628842+0000", 2024-09-14T06:29:12.933 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:12.933 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:12.933 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:12.933 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:29:12.933 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.628842+0000", 2024-09-14T06:29:12.933 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:12.933 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:12.933 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:12.934 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:29:12.934 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.628843+0000", 2024-09-14T06:29:12.934 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.5600000000000005e-07 2024-09-14T06:29:12.934 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:12.934 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:12.934 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:29:12.934 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.628847+0000", 2024-09-14T06:29:12.934 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.394e-06 2024-09-14T06:29:12.934 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:12.934 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:12.934 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:29:12.934 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.628855+0000", 2024-09-14T06:29:12.934 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.0760000000000003e-06 2024-09-14T06:29:12.934 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:12.935 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:12.935 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:29:12.935 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.628868+0000", 2024-09-14T06:29:12.935 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.2860000000000001e-05 2024-09-14T06:29:12.935 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:12.935 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:12.935 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:29:12.935 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.646733+0000", 2024-09-14T06:29:12.935 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.017865387999999999 2024-09-14T06:29:12.935 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:12.935 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:12.935 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:29:12.935 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.646800+0000", 2024-09-14T06:29:12.935 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.6619999999999996e-05 2024-09-14T06:29:12.936 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:12.936 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:12.936 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:29:12.936 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.646880+0000", 2024-09-14T06:29:12.936 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.0648000000000004e-05 2024-09-14T06:29:12.936 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:12.936 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:29:12.936 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:12.936 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:12.936 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:12.936 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:1488 3.a 3:57c28b0f:::rbd_data.112d7dfbd766.000000000000000a:5 [read 274432~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-14T06:29:12.936 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:08.628878+0000", 2024-09-14T06:29:12.936 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 4.2881484969999999, 2024-09-14T06:29:12.936 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.018187624999999999, 2024-09-14T06:29:12.936 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:29:12.937 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:29:12.937 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:29:12.937 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:29:12.937 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:29:12.937 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:12.937 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 1488 2024-09-14T06:29:12.937 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:12.937 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:29:12.937 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:12.937 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:29:12.937 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.628878+0000", 2024-09-14T06:29:12.937 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:12.937 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:12.937 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:12.937 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:29:12.938 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.628878+0000", 2024-09-14T06:29:12.938 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:12.938 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:12.938 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:12.938 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:29:12.938 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.628879+0000", 2024-09-14T06:29:12.938 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.58e-07 2024-09-14T06:29:12.938 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:12.938 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:12.938 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:29:12.938 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.628881+0000", 2024-09-14T06:29:12.938 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.0389999999999999e-06 2024-09-14T06:29:12.938 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:12.938 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:12.938 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:29:12.939 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.628887+0000", 2024-09-14T06:29:12.939 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.9429999999999997e-06 2024-09-14T06:29:12.939 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:12.939 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:12.939 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:29:12.939 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.628891+0000", 2024-09-14T06:29:12.939 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.1309999999999999e-06 2024-09-14T06:29:12.939 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:12.939 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:12.939 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:29:12.939 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.646909+0000", 2024-09-14T06:29:12.939 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.018017920999999999 2024-09-14T06:29:12.939 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:12.939 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:12.939 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:29:12.939 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.646984+0000", 2024-09-14T06:29:12.940 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.4989999999999999e-05 2024-09-14T06:29:12.940 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:12.940 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:12.940 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:29:12.940 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.647066+0000", 2024-09-14T06:29:12.940 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.2143000000000003e-05 2024-09-14T06:29:12.940 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:12.940 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:29:12.940 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:12.940 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:12.940 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:12.940 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:1489 3.a 3:57c28b0f:::rbd_data.112d7dfbd766.000000000000000a:5 [read 290816~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-14T06:29:12.940 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:08.628897+0000", 2024-09-14T06:29:12.941 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 4.288130046, 2024-09-14T06:29:12.941 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.018341491000000001, 2024-09-14T06:29:12.941 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:29:12.941 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:29:12.941 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:29:12.941 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:29:12.941 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:29:12.941 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:12.941 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 1489 2024-09-14T06:29:12.941 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:12.941 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:29:12.941 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:12.941 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:29:12.941 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.628897+0000", 2024-09-14T06:29:12.942 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:12.942 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:12.942 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:12.942 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:29:12.942 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.628897+0000", 2024-09-14T06:29:12.942 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:12.942 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:12.942 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:12.942 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:29:12.942 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.628897+0000", 2024-09-14T06:29:12.942 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.6800000000000001e-07 2024-09-14T06:29:12.942 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:12.942 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:12.942 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:29:12.943 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.628899+0000", 2024-09-14T06:29:12.943 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.606e-06 2024-09-14T06:29:12.943 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:12.943 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:12.943 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:29:12.943 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.628904+0000", 2024-09-14T06:29:12.943 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.1839999999999998e-06 2024-09-14T06:29:12.943 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:12.943 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:12.943 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:29:12.943 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.628908+0000", 2024-09-14T06:29:12.943 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.0500000000000002e-06 2024-09-14T06:29:12.943 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:12.943 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:12.944 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:29:12.944 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.647093+0000", 2024-09-14T06:29:12.944 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.018184846000000001 2024-09-14T06:29:12.944 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:12.944 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:12.944 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:29:12.944 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.647158+0000", 2024-09-14T06:29:12.944 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.5068999999999997e-05 2024-09-14T06:29:12.944 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:12.944 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:12.944 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:29:12.945 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.647238+0000", 2024-09-14T06:29:12.945 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.0368e-05 2024-09-14T06:29:12.945 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:12.945 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:29:12.945 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:12.945 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:12.945 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:12.945 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:1490 3.a 3:57c28b0f:::rbd_data.112d7dfbd766.000000000000000a:5 [read 307200~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-14T06:29:12.945 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:08.628921+0000", 2024-09-14T06:29:12.945 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 4.2881060489999996, 2024-09-14T06:29:12.945 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.018575338, 2024-09-14T06:29:12.945 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:29:12.945 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:29:12.945 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:29:12.946 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:29:12.946 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:29:12.946 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:12.946 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 1490 2024-09-14T06:29:12.946 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:12.946 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:29:12.946 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:12.946 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:29:12.946 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.628921+0000", 2024-09-14T06:29:12.946 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:12.946 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:12.946 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:12.946 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:29:12.946 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.628921+0000", 2024-09-14T06:29:12.946 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:12.947 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:12.947 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:12.947 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:29:12.947 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.628921+0000", 2024-09-14T06:29:12.947 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.39e-07 2024-09-14T06:29:12.947 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:12.947 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:12.947 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:29:12.947 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.628928+0000", 2024-09-14T06:29:12.947 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.81e-06 2024-09-14T06:29:12.947 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:12.947 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:12.947 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:29:12.947 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.628934+0000", 2024-09-14T06:29:12.947 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.4649999999999999e-06 2024-09-14T06:29:12.947 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:12.948 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:12.948 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:29:12.948 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.628939+0000", 2024-09-14T06:29:12.948 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.3730000000000003e-06 2024-09-14T06:29:12.948 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:12.948 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:12.948 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:29:12.948 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.647265+0000", 2024-09-14T06:29:12.948 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.018326499999999999 2024-09-14T06:29:12.948 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:12.948 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:12.948 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:29:12.948 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.647381+0000", 2024-09-14T06:29:12.948 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.000115549 2024-09-14T06:29:12.948 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:12.949 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:12.949 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:29:12.949 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.647496+0000", 2024-09-14T06:29:12.949 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.000115202 2024-09-14T06:29:12.949 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:12.949 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:29:12.949 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:12.949 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:12.949 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:12.949 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:1491 3.a 3:57c28b0f:::rbd_data.112d7dfbd766.000000000000000a:5 [read 323584~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-14T06:29:12.949 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:08.628964+0000", 2024-09-14T06:29:12.949 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 4.2880629859999999, 2024-09-14T06:29:12.949 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.018753204999999998, 2024-09-14T06:29:12.949 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:29:12.949 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:29:12.949 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:29:12.950 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:29:12.950 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:29:12.950 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:12.950 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 1491 2024-09-14T06:29:12.950 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:12.950 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:29:12.950 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:12.950 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:29:12.950 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.628964+0000", 2024-09-14T06:29:12.950 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:12.950 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:12.950 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:12.950 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:29:12.950 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.628964+0000", 2024-09-14T06:29:12.951 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:12.951 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:12.951 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:12.951 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:29:12.951 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.628964+0000", 2024-09-14T06:29:12.951 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.3600000000000004e-07 2024-09-14T06:29:12.951 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:12.951 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:12.951 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:29:12.951 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.628966+0000", 2024-09-14T06:29:12.951 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.119e-06 2024-09-14T06:29:12.951 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:12.951 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:12.952 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:29:12.952 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.628972+0000", 2024-09-14T06:29:12.952 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.0929999999999999e-06 2024-09-14T06:29:12.952 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:12.952 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:12.952 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:29:12.952 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.628977+0000", 2024-09-14T06:29:12.952 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.5360000000000003e-06 2024-09-14T06:29:12.952 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:12.952 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:12.952 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:29:12.952 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.647527+0000", 2024-09-14T06:29:12.952 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.018550433000000002 2024-09-14T06:29:12.952 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:12.952 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:12.952 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:29:12.953 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.647614+0000", 2024-09-14T06:29:12.953 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.6757999999999998e-05 2024-09-14T06:29:12.953 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:12.953 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:12.953 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:29:12.953 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.647717+0000", 2024-09-14T06:29:12.953 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00010273 2024-09-14T06:29:12.953 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:12.953 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:29:12.953 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:12.953 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:12.953 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:12.953 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:1492 3.a 3:57c28b0f:::rbd_data.112d7dfbd766.000000000000000a:5 [read 339968~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-14T06:29:12.953 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:08.628983+0000", 2024-09-14T06:29:12.953 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 4.2880433030000003, 2024-09-14T06:29:12.954 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.018937253000000001, 2024-09-14T06:29:12.954 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:29:12.954 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:29:12.954 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:29:12.954 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:29:12.954 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:29:12.954 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:12.954 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 1492 2024-09-14T06:29:12.954 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:12.954 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:29:12.954 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:12.954 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:29:12.954 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.628983+0000", 2024-09-14T06:29:12.954 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:12.955 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:12.955 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:12.955 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:29:12.955 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.628983+0000", 2024-09-14T06:29:12.955 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:12.955 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:12.955 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:12.955 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:29:12.955 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.628984+0000", 2024-09-14T06:29:12.955 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.0100000000000001e-07 2024-09-14T06:29:12.955 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:12.955 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:12.955 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:29:12.955 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.628985+0000", 2024-09-14T06:29:12.955 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.561e-06 2024-09-14T06:29:12.956 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:12.956 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:12.956 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:29:12.956 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.628990+0000", 2024-09-14T06:29:12.956 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.1540000000000001e-06 2024-09-14T06:29:12.956 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:12.956 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:12.956 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:29:12.956 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.628994+0000", 2024-09-14T06:29:12.956 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.8890000000000004e-06 2024-09-14T06:29:12.956 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:12.956 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:12.956 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:29:12.956 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.647745+0000", 2024-09-14T06:29:12.957 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.018751213999999999 2024-09-14T06:29:12.957 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:12.957 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:12.957 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:29:12.957 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.647824+0000", 2024-09-14T06:29:12.957 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.8331999999999995e-05 2024-09-14T06:29:12.957 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:12.957 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:12.957 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:29:12.957 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.647921+0000", 2024-09-14T06:29:12.957 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.6801999999999999e-05 2024-09-14T06:29:12.957 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:12.958 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:29:12.958 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:12.958 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:12.958 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:12.958 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:1493 3.a 3:57c28b0f:::rbd_data.112d7dfbd766.000000000000000a:5 [read 356352~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-14T06:29:12.958 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:08.628999+0000", 2024-09-14T06:29:12.958 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 4.2880273879999997, 2024-09-14T06:29:12.958 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.019127128, 2024-09-14T06:29:12.958 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:29:12.958 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:29:12.958 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:29:12.958 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:29:12.958 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:29:12.958 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:12.959 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 1493 2024-09-14T06:29:12.959 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:12.959 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:29:12.959 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:12.959 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:29:12.959 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.628999+0000", 2024-09-14T06:29:12.959 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:12.959 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:12.959 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:12.959 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:29:12.959 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.628999+0000", 2024-09-14T06:29:12.959 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:12.959 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:12.959 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:12.960 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:29:12.960 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.629000+0000", 2024-09-14T06:29:12.960 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.2500000000000001e-07 2024-09-14T06:29:12.960 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:12.960 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:12.960 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:29:12.960 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.629001+0000", 2024-09-14T06:29:12.960 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.511e-06 2024-09-14T06:29:12.960 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:12.960 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:12.960 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:29:12.960 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.629006+0000", 2024-09-14T06:29:12.960 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.1420000000000001e-06 2024-09-14T06:29:12.961 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:12.961 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:12.961 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:29:12.961 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.629010+0000", 2024-09-14T06:29:12.961 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.8120000000000001e-06 2024-09-14T06:29:12.961 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:12.961 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:12.961 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:29:12.961 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.647950+0000", 2024-09-14T06:29:12.961 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.018939568 2024-09-14T06:29:12.961 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:12.961 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:12.962 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:29:12.962 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.648032+0000", 2024-09-14T06:29:12.962 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.2132000000000006e-05 2024-09-14T06:29:12.962 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:12.962 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:12.962 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:29:12.962 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.648126+0000", 2024-09-14T06:29:12.962 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.4637999999999998e-05 2024-09-14T06:29:12.962 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:12.962 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:29:12.962 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:12.962 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:12.962 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:12.962 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:1494 3.a 3:57c28b0f:::rbd_data.112d7dfbd766.000000000000000a:5 [read 372736~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-14T06:29:12.962 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:08.629017+0000", 2024-09-14T06:29:12.963 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 4.2880093270000001, 2024-09-14T06:29:12.963 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.019310414000000001, 2024-09-14T06:29:12.963 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:29:12.963 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:29:12.963 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:29:12.963 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:29:12.963 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:29:12.963 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:12.963 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 1494 2024-09-14T06:29:12.963 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:12.963 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:29:12.963 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:12.963 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:29:12.964 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.629017+0000", 2024-09-14T06:29:12.964 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:12.964 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:12.964 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:12.964 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:29:12.964 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.629017+0000", 2024-09-14T06:29:12.964 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:12.964 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:12.964 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:12.964 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:29:12.964 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.629018+0000", 2024-09-14T06:29:12.964 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.9000000000000002e-07 2024-09-14T06:29:12.964 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:12.964 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:12.965 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:29:12.965 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.629019+0000", 2024-09-14T06:29:12.965 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.5230000000000001e-06 2024-09-14T06:29:12.965 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:12.965 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:12.965 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:29:12.965 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.629024+0000", 2024-09-14T06:29:12.965 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.2299999999999999e-06 2024-09-14T06:29:12.965 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:12.965 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:12.965 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:29:12.965 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.629028+0000", 2024-09-14T06:29:12.965 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.7749999999999999e-06 2024-09-14T06:29:12.965 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:12.965 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:12.966 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:29:12.966 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.648156+0000", 2024-09-14T06:29:12.966 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.019128045999999999 2024-09-14T06:29:12.966 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:12.966 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:12.966 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:29:12.966 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.648238+0000", 2024-09-14T06:29:12.966 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.1877000000000002e-05 2024-09-14T06:29:12.966 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:12.966 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:12.966 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:29:12.966 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.648328+0000", 2024-09-14T06:29:12.966 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.9573000000000005e-05 2024-09-14T06:29:12.966 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:12.966 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:29:12.967 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:12.967 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:12.967 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:12.967 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:1495 3.a 3:57c28b0f:::rbd_data.112d7dfbd766.000000000000000a:5 [read 389120~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-14T06:29:12.967 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:08.629035+0000", 2024-09-14T06:29:12.967 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 4.2879913030000001, 2024-09-14T06:29:12.967 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.019486507, 2024-09-14T06:29:12.967 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:29:12.967 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:29:12.967 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:29:12.967 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:29:12.967 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:29:12.967 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:12.967 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 1495 2024-09-14T06:29:12.967 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:12.968 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:29:12.968 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:12.968 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:29:12.968 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.629035+0000", 2024-09-14T06:29:12.968 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:12.968 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:12.968 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:12.968 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:29:12.968 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.629035+0000", 2024-09-14T06:29:12.968 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:12.968 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:12.968 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:12.968 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:29:12.969 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.629036+0000", 2024-09-14T06:29:12.969 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.2000000000000001e-07 2024-09-14T06:29:12.969 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:12.969 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:12.969 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:29:12.969 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.629037+0000", 2024-09-14T06:29:12.969 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.4300000000000001e-06 2024-09-14T06:29:12.969 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:12.969 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:12.969 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:29:12.969 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.629043+0000", 2024-09-14T06:29:12.969 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.575e-06 2024-09-14T06:29:12.969 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:12.970 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:12.970 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:29:12.970 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.629047+0000", 2024-09-14T06:29:12.970 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.9179999999999996e-06 2024-09-14T06:29:12.970 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:12.970 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:12.970 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:29:12.970 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.648356+0000", 2024-09-14T06:29:12.970 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.019309862000000001 2024-09-14T06:29:12.970 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:12.970 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:12.970 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:29:12.970 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.648433+0000", 2024-09-14T06:29:12.970 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.6973999999999999e-05 2024-09-14T06:29:12.971 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:12.971 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:12.971 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:29:12.971 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.648522+0000", 2024-09-14T06:29:12.971 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.8428000000000001e-05 2024-09-14T06:29:12.971 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:12.971 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:29:12.971 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:12.971 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:12.971 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:12.971 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:1496 3.a 3:57c28b0f:::rbd_data.112d7dfbd766.000000000000000a:5 [read 405504~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-14T06:29:12.971 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:08.629086+0000", 2024-09-14T06:29:12.971 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 4.287940259, 2024-09-14T06:29:12.971 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.019618813999999998, 2024-09-14T06:29:12.971 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:29:12.972 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:29:12.972 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:29:12.972 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:29:12.972 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:29:12.972 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:12.972 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 1496 2024-09-14T06:29:12.972 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:12.972 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:29:12.972 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:12.972 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:29:12.972 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.629086+0000", 2024-09-14T06:29:12.972 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:12.972 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:12.972 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:12.972 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:29:12.973 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.629086+0000", 2024-09-14T06:29:12.973 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:12.973 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:12.973 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:12.973 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:29:12.973 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.629087+0000", 2024-09-14T06:29:12.973 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.1399999999999996e-07 2024-09-14T06:29:12.973 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:12.973 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:12.973 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:29:12.973 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.629100+0000", 2024-09-14T06:29:12.973 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.2748999999999999e-05 2024-09-14T06:29:12.973 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:12.973 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:12.974 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:29:12.974 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.629111+0000", 2024-09-14T06:29:12.974 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.1531e-05 2024-09-14T06:29:12.974 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:12.974 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:12.974 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:29:12.974 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.629131+0000", 2024-09-14T06:29:12.974 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.9341999999999999e-05 2024-09-14T06:29:12.974 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:12.974 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:12.974 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:29:12.974 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.648550+0000", 2024-09-14T06:29:12.974 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.019418916000000001 2024-09-14T06:29:12.974 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:12.975 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:12.975 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:29:12.975 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.648616+0000", 2024-09-14T06:29:12.975 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.6359000000000006e-05 2024-09-14T06:29:12.975 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:12.975 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:12.975 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:29:12.975 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.648705+0000", 2024-09-14T06:29:12.975 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.9103000000000004e-05 2024-09-14T06:29:12.975 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:12.975 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:29:12.975 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:12.975 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:12.975 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:12.975 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:1497 3.a 3:57c28b0f:::rbd_data.112d7dfbd766.000000000000000a:5 [read 421888~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-14T06:29:12.976 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:08.629145+0000", 2024-09-14T06:29:12.976 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 4.2878814810000003, 2024-09-14T06:29:12.976 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.019744725000000001, 2024-09-14T06:29:12.976 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:29:12.976 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:29:12.976 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:29:12.976 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:29:12.976 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:29:12.976 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:12.976 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 1497 2024-09-14T06:29:12.976 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:12.976 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:29:12.976 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:12.976 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:29:12.976 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.629145+0000", 2024-09-14T06:29:12.976 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:12.977 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:12.977 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:12.977 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:29:12.977 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.629145+0000", 2024-09-14T06:29:12.977 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:12.977 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:12.977 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:12.977 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:29:12.977 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.629146+0000", 2024-09-14T06:29:12.977 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.1699999999999999e-07 2024-09-14T06:29:12.977 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:12.977 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:12.977 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:29:12.977 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.629148+0000", 2024-09-14T06:29:12.977 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.024e-06 2024-09-14T06:29:12.978 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:12.978 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:12.978 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:29:12.978 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.629152+0000", 2024-09-14T06:29:12.978 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.5970000000000002e-06 2024-09-14T06:29:12.978 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:12.978 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:12.978 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:29:12.978 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.629157+0000", 2024-09-14T06:29:12.978 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.8119999999999998e-06 2024-09-14T06:29:12.978 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:12.978 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:12.978 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:29:12.978 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.648734+0000", 2024-09-14T06:29:12.978 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.019576935 2024-09-14T06:29:12.978 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:12.979 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:12.979 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:29:12.979 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.648801+0000", 2024-09-14T06:29:12.979 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.7305999999999996e-05 2024-09-14T06:29:12.979 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:12.979 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:12.979 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:29:12.979 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.648890+0000", 2024-09-14T06:29:12.979 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.8634000000000006e-05 2024-09-14T06:29:12.979 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:12.979 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:29:12.979 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:12.979 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:12.979 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:12.979 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:1498 3.a 3:57c28b0f:::rbd_data.112d7dfbd766.000000000000000a:5 [read 438272~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-14T06:29:12.980 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:08.629167+0000", 2024-09-14T06:29:12.980 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 4.287859417, 2024-09-14T06:29:12.980 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.019912138999999999, 2024-09-14T06:29:12.980 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:29:12.980 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:29:12.980 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:29:12.980 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:29:12.980 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:29:12.980 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:12.980 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 1498 2024-09-14T06:29:12.980 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:12.980 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:29:12.980 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:12.980 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:29:12.981 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.629167+0000", 2024-09-14T06:29:12.981 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:12.981 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:12.981 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:12.981 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:29:12.981 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.629167+0000", 2024-09-14T06:29:12.981 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:12.981 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:12.981 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:12.981 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:29:12.981 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.629168+0000", 2024-09-14T06:29:12.981 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.1300000000000001e-07 2024-09-14T06:29:12.981 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:12.981 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:12.981 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:29:12.982 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.629170+0000", 2024-09-14T06:29:12.982 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.7209999999999999e-06 2024-09-14T06:29:12.982 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:12.982 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:12.982 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:29:12.982 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.629174+0000", 2024-09-14T06:29:12.982 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.1459999999999998e-06 2024-09-14T06:29:12.982 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:12.982 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:12.982 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:29:12.982 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.629178+0000", 2024-09-14T06:29:12.982 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.8519999999999997e-06 2024-09-14T06:29:12.982 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:12.982 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:12.982 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:29:12.982 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.648924+0000", 2024-09-14T06:29:12.983 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.019746107999999998 2024-09-14T06:29:12.983 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:12.983 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:12.983 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:29:12.983 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.648995+0000", 2024-09-14T06:29:12.983 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.0809999999999995e-05 2024-09-14T06:29:12.983 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:12.983 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:12.983 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:29:12.983 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.649079+0000", 2024-09-14T06:29:12.983 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.4089000000000001e-05 2024-09-14T06:29:12.983 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:12.983 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:29:12.984 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:12.984 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:12.984 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:12.984 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:1499 3.a 3:57c28b0f:::rbd_data.112d7dfbd766.000000000000000a:5 [read 454656~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-14T06:29:12.984 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:08.629187+0000", 2024-09-14T06:29:12.984 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 4.2878394530000001, 2024-09-14T06:29:12.984 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.020072543000000002, 2024-09-14T06:29:12.984 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:29:12.984 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:29:12.984 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:29:12.984 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:29:12.984 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:29:12.984 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:12.984 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 1499 2024-09-14T06:29:12.985 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:12.985 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:29:12.985 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:12.985 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:29:12.985 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.629187+0000", 2024-09-14T06:29:12.985 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:12.985 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:12.985 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:12.985 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:29:12.985 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.629187+0000", 2024-09-14T06:29:12.985 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:12.985 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:12.985 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:12.986 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:29:12.986 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.629188+0000", 2024-09-14T06:29:12.986 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.89e-07 2024-09-14T06:29:12.986 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:12.986 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:12.986 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:29:12.986 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.629190+0000", 2024-09-14T06:29:12.986 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.0169999999999999e-06 2024-09-14T06:29:12.986 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:12.986 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:12.986 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:29:12.986 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.629195+0000", 2024-09-14T06:29:12.986 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.4149999999999998e-06 2024-09-14T06:29:12.986 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:12.987 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:12.987 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:29:12.987 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.629198+0000", 2024-09-14T06:29:12.987 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.605e-06 2024-09-14T06:29:12.987 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:12.987 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:12.987 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:29:12.987 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.649107+0000", 2024-09-14T06:29:12.987 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.019909339000000002 2024-09-14T06:29:12.987 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:12.987 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:12.987 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:29:12.987 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.649178+0000", 2024-09-14T06:29:12.987 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.0809999999999995e-05 2024-09-14T06:29:12.988 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:12.988 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:12.988 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:29:12.988 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.649260+0000", 2024-09-14T06:29:12.988 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.1967999999999999e-05 2024-09-14T06:29:12.988 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:12.988 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:29:12.988 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:12.988 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:12.988 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:12.988 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:1500 3.a 3:57c28b0f:::rbd_data.112d7dfbd766.000000000000000a:5 [read 471040~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-14T06:29:12.988 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:08.629204+0000", 2024-09-14T06:29:12.988 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 4.2878229609999998, 2024-09-14T06:29:12.989 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.020473080000000001, 2024-09-14T06:29:12.989 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:29:12.989 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:29:12.989 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:29:12.989 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:29:12.989 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:29:12.989 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:12.989 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 1500 2024-09-14T06:29:12.989 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:12.989 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:29:12.989 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:12.989 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:29:12.990 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.629204+0000", 2024-09-14T06:29:12.990 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:12.990 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:12.990 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:12.990 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:29:12.990 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.629204+0000", 2024-09-14T06:29:12.990 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:12.990 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:12.990 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:12.990 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:29:12.990 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.629204+0000", 2024-09-14T06:29:12.990 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.34e-07 2024-09-14T06:29:12.991 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:12.991 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:12.991 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:29:12.991 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.629206+0000", 2024-09-14T06:29:12.991 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.939e-06 2024-09-14T06:29:12.991 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:12.991 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:12.991 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:29:12.991 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.629212+0000", 2024-09-14T06:29:12.991 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.8669999999999999e-06 2024-09-14T06:29:12.991 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:12.991 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:12.991 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:29:12.991 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.629216+0000", 2024-09-14T06:29:12.992 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.3830000000000002e-06 2024-09-14T06:29:12.992 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:12.992 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:12.992 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:29:12.992 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.649288+0000", 2024-09-14T06:29:12.992 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.020072113999999999 2024-09-14T06:29:12.992 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:12.992 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:12.992 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:29:12.992 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.649355+0000", 2024-09-14T06:29:12.992 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.7218000000000006e-05 2024-09-14T06:29:12.992 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:12.992 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:12.993 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:29:12.993 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.649677+0000", 2024-09-14T06:29:12.993 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00032122500000000002 2024-09-14T06:29:12.993 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:12.993 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:29:12.993 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:12.993 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:12.993 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:12.993 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:1501 3.a 3:57c28b0f:::rbd_data.112d7dfbd766.000000000000000a:5 [read 487424~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-14T06:29:12.993 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:08.629223+0000", 2024-09-14T06:29:12.993 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 4.287803512, 2024-09-14T06:29:12.993 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.021370209000000001, 2024-09-14T06:29:12.993 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:29:12.993 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:29:12.993 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:29:12.994 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:29:12.994 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:29:12.994 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:12.994 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 1501 2024-09-14T06:29:12.994 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:12.994 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:29:12.994 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:12.994 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:29:12.994 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.629223+0000", 2024-09-14T06:29:12.994 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:12.994 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:12.994 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:12.994 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:29:12.994 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.629223+0000", 2024-09-14T06:29:12.994 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:12.995 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:12.995 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:12.995 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:29:12.995 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.629223+0000", 2024-09-14T06:29:12.995 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.9900000000000001e-07 2024-09-14T06:29:12.995 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:12.995 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:12.995 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:29:12.995 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.629231+0000", 2024-09-14T06:29:12.995 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.9179999999999994e-06 2024-09-14T06:29:12.995 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:12.995 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:12.995 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:29:12.995 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.629238+0000", 2024-09-14T06:29:12.995 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.421e-06 2024-09-14T06:29:12.996 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:12.996 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:12.996 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:29:12.996 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.629242+0000", 2024-09-14T06:29:12.996 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.5419999999999999e-06 2024-09-14T06:29:12.996 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:12.996 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:12.996 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:29:12.996 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.650284+0000", 2024-09-14T06:29:12.996 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.021041131000000001 2024-09-14T06:29:12.996 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:12.996 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:12.996 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:29:12.996 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.650361+0000", 2024-09-14T06:29:12.996 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.7787999999999995e-05 2024-09-14T06:29:12.997 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:12.997 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:12.997 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:29:12.997 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.650593+0000", 2024-09-14T06:29:12.997 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00023201 2024-09-14T06:29:12.997 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:12.997 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:29:12.997 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:12.997 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:12.997 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:12.997 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:1502 3.a 3:57c28b0f:::rbd_data.112d7dfbd766.000000000000000a:5 [read 503808~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-14T06:29:12.997 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:08.629249+0000", 2024-09-14T06:29:12.997 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 4.287777266, 2024-09-14T06:29:12.997 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.021713217, 2024-09-14T06:29:12.997 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:29:12.998 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:29:12.998 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:29:12.998 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:29:12.998 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:29:12.998 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:12.998 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 1502 2024-09-14T06:29:12.998 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:12.998 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:29:12.998 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:12.998 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:29:12.998 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.629249+0000", 2024-09-14T06:29:12.998 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:12.998 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:12.998 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:12.998 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:29:12.998 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.629249+0000", 2024-09-14T06:29:12.999 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:12.999 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:12.999 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:12.999 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:29:12.999 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.629250+0000", 2024-09-14T06:29:12.999 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.03e-07 2024-09-14T06:29:12.999 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:12.999 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:12.999 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:29:12.999 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.629252+0000", 2024-09-14T06:29:12.999 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.995e-06 2024-09-14T06:29:12.999 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:12.999 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:12.999 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:29:12.999 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.629257+0000", 2024-09-14T06:29:13.000 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.7010000000000001e-06 2024-09-14T06:29:13.000 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:13.000 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:13.000 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:29:13.000 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.629262+0000", 2024-09-14T06:29:13.000 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.16e-06 2024-09-14T06:29:13.000 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:13.000 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:13.000 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:29:13.000 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.650629+0000", 2024-09-14T06:29:13.001 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.021367684000000001 2024-09-14T06:29:13.001 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:13.001 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:13.001 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:29:13.001 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.650718+0000", 2024-09-14T06:29:13.001 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.8832000000000006e-05 2024-09-14T06:29:13.001 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:13.001 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:13.001 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:29:13.001 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.650963+0000", 2024-09-14T06:29:13.001 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00024444199999999999 2024-09-14T06:29:13.001 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:13.001 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:29:13.001 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:13.001 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:13.002 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:13.002 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:1503 3.a 3:57c28b0f:::rbd_data.112d7dfbd766.000000000000000a:5 [read 520192~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-14T06:29:13.002 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:08.629268+0000", 2024-09-14T06:29:13.002 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 4.2877587249999998, 2024-09-14T06:29:13.002 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.022244885999999998, 2024-09-14T06:29:13.002 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:29:13.002 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:29:13.002 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:29:13.002 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:29:13.002 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:29:13.002 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:13.002 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 1503 2024-09-14T06:29:13.002 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:13.002 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:29:13.002 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:13.002 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:29:13.003 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.629268+0000", 2024-09-14T06:29:13.003 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:13.003 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:13.003 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:13.003 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:29:13.003 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.629268+0000", 2024-09-14T06:29:13.003 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:13.003 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:13.003 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:13.003 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:29:13.003 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.629268+0000", 2024-09-14T06:29:13.003 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.6599999999999996e-07 2024-09-14T06:29:13.003 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:13.004 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:13.004 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:29:13.004 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.629270+0000", 2024-09-14T06:29:13.004 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.7129999999999999e-06 2024-09-14T06:29:13.004 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:13.004 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:13.004 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:29:13.004 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.629276+0000", 2024-09-14T06:29:13.004 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.8119999999999996e-06 2024-09-14T06:29:13.004 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:13.004 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:13.004 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:29:13.004 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.629280+0000", 2024-09-14T06:29:13.004 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.4920000000000004e-06 2024-09-14T06:29:13.004 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:13.005 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:13.005 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:29:13.005 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.651039+0000", 2024-09-14T06:29:13.005 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.021758116000000001 2024-09-14T06:29:13.005 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:13.005 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:13.005 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:29:13.005 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.651127+0000", 2024-09-14T06:29:13.005 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.7983000000000001e-05 2024-09-14T06:29:13.005 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:13.005 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:13.005 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:29:13.005 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.651513+0000", 2024-09-14T06:29:13.005 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00038620400000000002 2024-09-14T06:29:13.005 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:13.006 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:29:13.006 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:13.006 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:13.006 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:29:13.006 INFO:teuthology.orchestra.run.smithi138.stdout:} 2024-09-14T06:29:13.007 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-14T06:29:13.007 INFO:tasks.ceph.osd.1.smithi094.stderr:2024-09-14T06:29:13.006+0000 7f328e47a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33641) UID: 0 2024-09-14T06:29:13.108 INFO:tasks.ceph.osd.4.smithi138.stderr:2024-09-14T06:29:13.107+0000 7f6c653c8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33615) UID: 0 2024-09-14T06:29:13.150 INFO:teuthology.orchestra.run.smithi138.stdout:{ 2024-09-14T06:29:13.150 INFO:teuthology.orchestra.run.smithi138.stdout: "ops": [], 2024-09-14T06:29:13.150 INFO:teuthology.orchestra.run.smithi138.stdout: "num_ops": 0 2024-09-14T06:29:13.150 INFO:teuthology.orchestra.run.smithi138.stdout:} 2024-09-14T06:29:13.159 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-14T06:29:13.243 INFO:tasks.ceph.osd.5.smithi138.stderr:2024-09-14T06:29:13.242+0000 7fe13746f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33618) UID: 0 2024-09-14T06:29:13.302 INFO:teuthology.orchestra.run.smithi138.stdout:{ 2024-09-14T06:29:13.302 INFO:teuthology.orchestra.run.smithi138.stdout: "ops": [], 2024-09-14T06:29:13.302 INFO:teuthology.orchestra.run.smithi138.stdout: "complaint_time": 30, 2024-09-14T06:29:13.302 INFO:teuthology.orchestra.run.smithi138.stdout: "num_blocked_ops": 0 2024-09-14T06:29:13.302 INFO:teuthology.orchestra.run.smithi138.stdout:} 2024-09-14T06:29:13.310 INFO:tasks.ceph.osd.1.smithi094.stderr:2024-09-14T06:29:13.309+0000 7f328e47a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33641) UID: 0 2024-09-14T06:29:13.311 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-14T06:29:13.410 INFO:tasks.ceph.osd.6.smithi138.stderr:2024-09-14T06:29:13.409+0000 7f8cd8849640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33620) UID: 0 2024-09-14T06:29:13.436 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-14T06:29:13.455 INFO:teuthology.orchestra.run.smithi138.stdout:{ 2024-09-14T06:29:13.456 INFO:teuthology.orchestra.run.smithi138.stdout: "size": 20, 2024-09-14T06:29:13.456 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 600, 2024-09-14T06:29:13.456 INFO:teuthology.orchestra.run.smithi138.stdout: "ops": [ 2024-09-14T06:29:13.456 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:13.456 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:1566 3.15 3:accf422f:::rbd_data.112d7dfbd766.0000000000000001:5 [read 192512~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-14T06:29:13.456 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:08.630416+0000", 2024-09-14T06:29:13.456 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 4.823423494, 2024-09-14T06:29:13.456 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.021425026999999999, 2024-09-14T06:29:13.456 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:29:13.456 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:29:13.457 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:29:13.457 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:29:13.457 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:29:13.457 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:13.457 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 1566 2024-09-14T06:29:13.457 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:13.457 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:29:13.457 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:13.457 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:29:13.457 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.630416+0000", 2024-09-14T06:29:13.457 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:13.457 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:13.457 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:13.457 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:29:13.458 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.630416+0000", 2024-09-14T06:29:13.458 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:13.458 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:13.458 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:13.458 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:29:13.458 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.630417+0000", 2024-09-14T06:29:13.458 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.82e-07 2024-09-14T06:29:13.458 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:13.458 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:13.458 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:29:13.458 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.630419+0000", 2024-09-14T06:29:13.458 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.8059999999999999e-06 2024-09-14T06:29:13.458 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:13.459 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:13.459 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:29:13.459 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.630425+0000", 2024-09-14T06:29:13.459 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.3790000000000003e-06 2024-09-14T06:29:13.459 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:13.459 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:13.459 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:29:13.459 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.630431+0000", 2024-09-14T06:29:13.459 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.2570000000000004e-06 2024-09-14T06:29:13.459 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:13.459 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:13.459 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:29:13.459 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.651640+0000", 2024-09-14T06:29:13.459 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.021208320999999999 2024-09-14T06:29:13.460 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:13.460 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:13.460 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:29:13.460 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.651721+0000", 2024-09-14T06:29:13.460 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.1792000000000006e-05 2024-09-14T06:29:13.460 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:13.460 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:13.460 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:29:13.460 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.651841+0000", 2024-09-14T06:29:13.460 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00011989000000000001 2024-09-14T06:29:13.460 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:13.460 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:29:13.460 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:13.461 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:13.461 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:13.461 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:1567 3.15 3:accf422f:::rbd_data.112d7dfbd766.0000000000000001:5 [read 208896~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-14T06:29:13.461 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:08.630438+0000", 2024-09-14T06:29:13.461 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 4.8234021350000003, 2024-09-14T06:29:13.461 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.021638307999999998, 2024-09-14T06:29:13.461 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:29:13.461 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:29:13.461 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:29:13.461 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:29:13.461 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:29:13.461 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:13.461 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 1567 2024-09-14T06:29:13.462 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:13.462 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:29:13.462 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:13.462 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:29:13.462 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.630438+0000", 2024-09-14T06:29:13.462 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:13.462 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:13.462 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:13.462 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:29:13.462 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.630438+0000", 2024-09-14T06:29:13.462 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:13.462 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:13.462 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:13.462 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:29:13.463 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.630438+0000", 2024-09-14T06:29:13.463 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.5400000000000002e-07 2024-09-14T06:29:13.463 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:13.463 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:13.463 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:29:13.463 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.630440+0000", 2024-09-14T06:29:13.463 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.559e-06 2024-09-14T06:29:13.463 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:13.463 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:13.463 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:29:13.463 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.630446+0000", 2024-09-14T06:29:13.463 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.421e-06 2024-09-14T06:29:13.463 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:13.464 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:13.464 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:29:13.464 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.630451+0000", 2024-09-14T06:29:13.464 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.0849999999999996e-06 2024-09-14T06:29:13.464 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:13.464 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:13.464 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:29:13.464 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.651885+0000", 2024-09-14T06:29:13.464 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.021433773 2024-09-14T06:29:13.464 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:13.464 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:13.464 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:29:13.464 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.651976+0000", 2024-09-14T06:29:13.464 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.0897999999999996e-05 2024-09-14T06:29:13.465 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:13.465 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:13.465 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:29:13.465 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.652076+0000", 2024-09-14T06:29:13.465 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00010011800000000001 2024-09-14T06:29:13.465 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:13.465 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:29:13.465 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:13.465 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:13.465 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:13.465 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:1568 3.15 3:accf422f:::rbd_data.112d7dfbd766.0000000000000001:5 [read 225280~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-14T06:29:13.465 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:08.630458+0000", 2024-09-14T06:29:13.465 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 4.8233817859999997, 2024-09-14T06:29:13.466 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.021830307, 2024-09-14T06:29:13.466 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:29:13.466 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:29:13.466 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:29:13.466 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:29:13.466 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:29:13.466 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:13.466 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 1568 2024-09-14T06:29:13.466 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:13.466 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:29:13.466 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:13.466 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:29:13.466 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.630458+0000", 2024-09-14T06:29:13.467 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:13.467 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:13.467 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:13.467 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:29:13.467 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.630458+0000", 2024-09-14T06:29:13.467 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:13.467 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:13.467 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:13.467 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:29:13.467 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.630458+0000", 2024-09-14T06:29:13.467 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.7899999999999998e-07 2024-09-14T06:29:13.467 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:13.467 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:13.467 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:29:13.468 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.630465+0000", 2024-09-14T06:29:13.468 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.6109999999999999e-06 2024-09-14T06:29:13.468 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:13.468 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:13.468 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:29:13.468 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.630473+0000", 2024-09-14T06:29:13.468 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.2290000000000003e-06 2024-09-14T06:29:13.468 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:13.468 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:13.468 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:29:13.468 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.630479+0000", 2024-09-14T06:29:13.468 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.2249999999999999e-06 2024-09-14T06:29:13.468 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:13.468 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:13.468 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:29:13.469 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.652109+0000", 2024-09-14T06:29:13.469 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.021630527 2024-09-14T06:29:13.469 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:13.469 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:13.469 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:29:13.469 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.652187+0000", 2024-09-14T06:29:13.469 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.8300999999999995e-05 2024-09-14T06:29:13.469 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:13.469 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:13.469 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:29:13.469 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.652288+0000", 2024-09-14T06:29:13.469 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00010083500000000001 2024-09-14T06:29:13.469 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:13.469 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:29:13.470 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:13.470 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:13.470 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:13.470 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:1569 3.15 3:accf422f:::rbd_data.112d7dfbd766.0000000000000001:5 [read 241664~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-14T06:29:13.470 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:08.630493+0000", 2024-09-14T06:29:13.470 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 4.8233465320000004, 2024-09-14T06:29:13.470 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.022263345, 2024-09-14T06:29:13.470 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:29:13.470 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:29:13.470 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:29:13.470 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:29:13.470 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:29:13.470 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:13.471 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 1569 2024-09-14T06:29:13.471 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:13.471 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:29:13.471 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:13.471 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:29:13.471 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.630493+0000", 2024-09-14T06:29:13.471 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:13.471 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:13.471 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:13.471 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:29:13.471 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.630493+0000", 2024-09-14T06:29:13.471 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:13.471 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:13.472 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:13.472 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:29:13.472 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.630494+0000", 2024-09-14T06:29:13.472 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.0399999999999996e-07 2024-09-14T06:29:13.472 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:13.472 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:13.472 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:29:13.472 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.630496+0000", 2024-09-14T06:29:13.472 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.2749999999999998e-06 2024-09-14T06:29:13.472 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:13.472 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:13.472 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:29:13.472 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.630503+0000", 2024-09-14T06:29:13.472 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.1670000000000002e-06 2024-09-14T06:29:13.473 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:13.473 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:13.473 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:29:13.473 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.630509+0000", 2024-09-14T06:29:13.473 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.6300000000000003e-06 2024-09-14T06:29:13.473 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:13.473 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:13.473 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:29:13.473 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.652320+0000", 2024-09-14T06:29:13.473 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.021811047 2024-09-14T06:29:13.473 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:13.473 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:13.473 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:29:13.474 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.652397+0000", 2024-09-14T06:29:13.474 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.7347000000000003e-05 2024-09-14T06:29:13.474 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:13.474 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:13.474 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:29:13.474 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.652757+0000", 2024-09-14T06:29:13.474 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00035927499999999999 2024-09-14T06:29:13.474 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:13.474 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:29:13.474 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:13.474 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:13.474 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:13.474 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:1570 3.15 3:accf422f:::rbd_data.112d7dfbd766.0000000000000001:5 [read 258048~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-14T06:29:13.475 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:08.630517+0000", 2024-09-14T06:29:13.475 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 4.8233227750000003, 2024-09-14T06:29:13.475 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.022589267, 2024-09-14T06:29:13.475 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:29:13.475 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:29:13.475 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:29:13.475 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:29:13.475 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:29:13.475 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:13.475 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 1570 2024-09-14T06:29:13.475 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:13.475 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:29:13.475 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:13.476 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:29:13.476 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.630517+0000", 2024-09-14T06:29:13.476 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:13.476 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:13.476 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:13.476 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:29:13.476 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.630517+0000", 2024-09-14T06:29:13.476 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:13.476 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:13.476 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:13.476 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:29:13.476 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.630517+0000", 2024-09-14T06:29:13.476 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.4000000000000002e-07 2024-09-14T06:29:13.476 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:13.477 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:13.477 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:29:13.477 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.630524+0000", 2024-09-14T06:29:13.477 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.2829999999999998e-06 2024-09-14T06:29:13.477 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:13.477 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:13.477 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:29:13.477 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.630532+0000", 2024-09-14T06:29:13.477 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.9149999999999996e-06 2024-09-14T06:29:13.477 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:13.477 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:13.477 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:29:13.477 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.630537+0000", 2024-09-14T06:29:13.478 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.6500000000000001e-06 2024-09-14T06:29:13.478 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:13.478 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:13.478 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:29:13.478 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.652791+0000", 2024-09-14T06:29:13.478 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.022253586999999998 2024-09-14T06:29:13.478 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:13.478 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:13.478 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:29:13.478 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.652866+0000", 2024-09-14T06:29:13.478 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.5507999999999996e-05 2024-09-14T06:29:13.478 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:13.478 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:13.479 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:29:13.479 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.653106+0000", 2024-09-14T06:29:13.479 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.000239884 2024-09-14T06:29:13.479 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:13.479 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:29:13.479 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:13.479 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:13.479 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:13.479 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:1571 3.15 3:accf422f:::rbd_data.112d7dfbd766.0000000000000001:5 [read 274432~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-14T06:29:13.479 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:08.630544+0000", 2024-09-14T06:29:13.479 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 4.8232952239999998, 2024-09-14T06:29:13.479 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.023662969999999998, 2024-09-14T06:29:13.479 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:29:13.479 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:29:13.480 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:29:13.480 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:29:13.480 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:29:13.480 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:13.480 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 1571 2024-09-14T06:29:13.480 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:13.480 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:29:13.480 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:13.480 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:29:13.480 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.630544+0000", 2024-09-14T06:29:13.480 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:13.480 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:13.480 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:13.481 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:29:13.481 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.630544+0000", 2024-09-14T06:29:13.481 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:13.481 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:13.481 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:13.481 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:29:13.481 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.630545+0000", 2024-09-14T06:29:13.481 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.6100000000000001e-07 2024-09-14T06:29:13.481 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:13.481 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:13.481 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:29:13.481 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.630547+0000", 2024-09-14T06:29:13.481 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.7990000000000001e-06 2024-09-14T06:29:13.481 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:13.482 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:13.482 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:29:13.482 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.630554+0000", 2024-09-14T06:29:13.482 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.9110000000000004e-06 2024-09-14T06:29:13.482 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:13.482 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:13.482 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:29:13.482 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.630565+0000", 2024-09-14T06:29:13.482 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.1308000000000001e-05 2024-09-14T06:29:13.482 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:13.482 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:13.482 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:29:13.482 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.653226+0000", 2024-09-14T06:29:13.483 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.022661345999999999 2024-09-14T06:29:13.483 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:13.483 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:13.483 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:29:13.483 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.654105+0000", 2024-09-14T06:29:13.483 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00087874700000000004 2024-09-14T06:29:13.483 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:13.483 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:13.483 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:29:13.483 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.654207+0000", 2024-09-14T06:29:13.483 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.000102398 2024-09-14T06:29:13.483 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:13.483 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:29:13.483 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:13.484 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:13.484 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:13.484 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:1572 3.15 3:accf422f:::rbd_data.112d7dfbd766.0000000000000001:5 [read 290816~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-14T06:29:13.484 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:08.630573+0000", 2024-09-14T06:29:13.484 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 4.8232670869999996, 2024-09-14T06:29:13.484 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.023868055999999999, 2024-09-14T06:29:13.484 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:29:13.484 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:29:13.484 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:29:13.484 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:29:13.484 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:29:13.484 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:13.484 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 1572 2024-09-14T06:29:13.485 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:13.485 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:29:13.485 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:13.485 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:29:13.485 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.630573+0000", 2024-09-14T06:29:13.485 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:13.485 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:13.485 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:13.485 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:29:13.485 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.630573+0000", 2024-09-14T06:29:13.486 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:13.486 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:13.486 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:13.486 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:29:13.486 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.630573+0000", 2024-09-14T06:29:13.486 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.1300000000000001e-07 2024-09-14T06:29:13.486 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:13.486 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:13.486 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:29:13.486 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.630575+0000", 2024-09-14T06:29:13.486 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.9539999999999998e-06 2024-09-14T06:29:13.486 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:13.486 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:13.487 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:29:13.487 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.630583+0000", 2024-09-14T06:29:13.487 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.5619999999999998e-06 2024-09-14T06:29:13.487 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:13.487 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:13.487 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:29:13.487 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.630587+0000", 2024-09-14T06:29:13.487 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.8579999999999999e-06 2024-09-14T06:29:13.487 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:13.487 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:13.487 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:29:13.487 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.654254+0000", 2024-09-14T06:29:13.487 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.023666194000000002 2024-09-14T06:29:13.487 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:13.488 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:13.488 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:29:13.488 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.654341+0000", 2024-09-14T06:29:13.488 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.7713999999999994e-05 2024-09-14T06:29:13.488 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:13.488 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:13.488 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:29:13.488 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.654441+0000", 2024-09-14T06:29:13.488 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.9361e-05 2024-09-14T06:29:13.488 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:13.488 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:29:13.488 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:13.488 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:13.489 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:13.489 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:1573 3.15 3:accf422f:::rbd_data.112d7dfbd766.0000000000000001:5 [read 307200~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-14T06:29:13.489 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:08.630595+0000", 2024-09-14T06:29:13.489 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 4.8232451149999997, 2024-09-14T06:29:13.489 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.024070597999999999, 2024-09-14T06:29:13.489 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:29:13.489 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:29:13.489 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:29:13.489 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:29:13.489 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:29:13.489 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:13.489 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 1573 2024-09-14T06:29:13.489 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:13.489 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:29:13.490 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:13.490 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:29:13.490 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.630595+0000", 2024-09-14T06:29:13.490 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:13.490 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:13.490 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:13.490 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:29:13.490 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.630595+0000", 2024-09-14T06:29:13.490 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:13.490 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:13.490 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:13.490 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:29:13.491 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.630595+0000", 2024-09-14T06:29:13.491 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.7100000000000002e-07 2024-09-14T06:29:13.491 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:13.491 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:13.491 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:29:13.491 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.630597+0000", 2024-09-14T06:29:13.491 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.8300000000000001e-06 2024-09-14T06:29:13.491 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:13.491 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:13.491 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:29:13.491 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.630603+0000", 2024-09-14T06:29:13.491 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.2940000000000002e-06 2024-09-14T06:29:13.491 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:13.492 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:13.492 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:29:13.492 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.630613+0000", 2024-09-14T06:29:13.492 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.0023e-05 2024-09-14T06:29:13.492 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:13.492 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:13.492 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:29:13.492 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.654481+0000", 2024-09-14T06:29:13.492 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.023868034999999999 2024-09-14T06:29:13.492 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:13.492 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:13.492 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:29:13.493 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.654564+0000", 2024-09-14T06:29:13.493 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.2738000000000006e-05 2024-09-14T06:29:13.493 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:13.493 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:13.493 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:29:13.493 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.654665+0000", 2024-09-14T06:29:13.493 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00010110700000000001 2024-09-14T06:29:13.493 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:13.493 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:29:13.493 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:13.493 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:13.493 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:13.493 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:1574 3.15 3:accf422f:::rbd_data.112d7dfbd766.0000000000000001:5 [read 323584~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-14T06:29:13.494 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:08.630620+0000", 2024-09-14T06:29:13.494 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 4.8232192700000001, 2024-09-14T06:29:13.494 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.024280177, 2024-09-14T06:29:13.494 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:29:13.494 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:29:13.494 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:29:13.494 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:29:13.494 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:29:13.494 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:13.494 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 1574 2024-09-14T06:29:13.494 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:13.494 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:29:13.494 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:13.494 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:29:13.495 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.630620+0000", 2024-09-14T06:29:13.495 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:13.495 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:13.495 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:13.495 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:29:13.495 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.630620+0000", 2024-09-14T06:29:13.495 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:13.495 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:13.495 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:13.495 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:29:13.496 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.630621+0000", 2024-09-14T06:29:13.496 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.7000000000000005e-07 2024-09-14T06:29:13.496 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:13.496 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:13.496 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:29:13.496 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.630623+0000", 2024-09-14T06:29:13.496 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.9290000000000001e-06 2024-09-14T06:29:13.496 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:13.496 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:13.496 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:29:13.496 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.630630+0000", 2024-09-14T06:29:13.496 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.7059999999999998e-06 2024-09-14T06:29:13.496 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:13.497 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:13.497 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:29:13.497 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.630636+0000", 2024-09-14T06:29:13.497 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.9510000000000003e-06 2024-09-14T06:29:13.497 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:13.497 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:13.497 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:29:13.497 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.654709+0000", 2024-09-14T06:29:13.497 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.024073324 2024-09-14T06:29:13.497 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:13.497 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:13.497 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:29:13.497 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.654796+0000", 2024-09-14T06:29:13.498 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.6669999999999995e-05 2024-09-14T06:29:13.498 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:13.498 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:13.498 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:29:13.498 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.654901+0000", 2024-09-14T06:29:13.498 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.000105027 2024-09-14T06:29:13.498 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:13.498 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:29:13.498 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:13.498 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:13.498 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:13.498 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:1575 3.15 3:accf422f:::rbd_data.112d7dfbd766.0000000000000001:5 [read 339968~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-14T06:29:13.498 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:08.630645+0000", 2024-09-14T06:29:13.498 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 4.8231950399999999, 2024-09-14T06:29:13.499 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.024511650999999999, 2024-09-14T06:29:13.499 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:29:13.499 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:29:13.499 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:29:13.499 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:29:13.499 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:29:13.499 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:13.499 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 1575 2024-09-14T06:29:13.499 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:13.499 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:29:13.499 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:13.499 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:29:13.499 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.630645+0000", 2024-09-14T06:29:13.500 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:13.500 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:13.500 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:13.500 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:29:13.500 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.630645+0000", 2024-09-14T06:29:13.500 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:13.500 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:13.500 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:13.500 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:29:13.500 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.630645+0000", 2024-09-14T06:29:13.500 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.9400000000000005e-07 2024-09-14T06:29:13.500 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:13.500 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:13.500 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:29:13.501 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.630647+0000", 2024-09-14T06:29:13.501 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.9309999999999998e-06 2024-09-14T06:29:13.501 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:13.501 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:13.501 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:29:13.501 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.630654+0000", 2024-09-14T06:29:13.501 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.55e-06 2024-09-14T06:29:13.501 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:13.501 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:13.501 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:29:13.501 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.630659+0000", 2024-09-14T06:29:13.501 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.0139999999999998e-06 2024-09-14T06:29:13.501 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:13.502 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:13.502 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:29:13.502 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.654964+0000", 2024-09-14T06:29:13.502 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.024305651000000001 2024-09-14T06:29:13.502 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:13.502 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:13.502 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:29:13.502 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.655048+0000", 2024-09-14T06:29:13.503 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.3389999999999999e-05 2024-09-14T06:29:13.503 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:13.503 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:13.503 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:29:13.503 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.655156+0000", 2024-09-14T06:29:13.503 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.000108521 2024-09-14T06:29:13.503 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:13.503 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:29:13.503 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:13.503 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:13.503 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:13.503 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:1576 3.15 3:accf422f:::rbd_data.112d7dfbd766.0000000000000001:5 [read 356352~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-14T06:29:13.503 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:08.630666+0000", 2024-09-14T06:29:13.503 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 4.823174002, 2024-09-14T06:29:13.504 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.024703712999999999, 2024-09-14T06:29:13.504 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:29:13.504 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:29:13.504 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:29:13.504 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:29:13.504 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:29:13.504 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:13.504 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 1576 2024-09-14T06:29:13.504 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:13.504 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:29:13.504 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:13.504 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:29:13.504 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.630666+0000", 2024-09-14T06:29:13.505 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:13.505 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:13.505 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:13.505 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:29:13.505 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.630666+0000", 2024-09-14T06:29:13.505 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:13.505 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:13.505 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:13.505 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:29:13.505 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.630666+0000", 2024-09-14T06:29:13.505 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.8299999999999997e-07 2024-09-14T06:29:13.505 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:13.505 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:13.506 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:29:13.506 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.630668+0000", 2024-09-14T06:29:13.506 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.8789999999999999e-06 2024-09-14T06:29:13.506 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:13.506 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:13.506 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:29:13.506 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.630675+0000", 2024-09-14T06:29:13.506 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.2899999999999997e-06 2024-09-14T06:29:13.506 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:13.506 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:13.506 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:29:13.506 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.630681+0000", 2024-09-14T06:29:13.506 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.2730000000000002e-06 2024-09-14T06:29:13.506 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:13.507 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:13.507 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:29:13.507 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.655182+0000", 2024-09-14T06:29:13.507 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.024501228999999999 2024-09-14T06:29:13.507 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:13.507 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:13.507 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:29:13.507 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.655257+0000", 2024-09-14T06:29:13.507 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.4954000000000001e-05 2024-09-14T06:29:13.507 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:13.507 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:13.507 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:29:13.507 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.655369+0000", 2024-09-14T06:29:13.507 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.000112605 2024-09-14T06:29:13.507 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:13.508 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:29:13.508 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:13.508 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:13.508 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:13.508 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:1577 3.15 3:accf422f:::rbd_data.112d7dfbd766.0000000000000001:5 [read 372736~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-14T06:29:13.508 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:08.630688+0000", 2024-09-14T06:29:13.508 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 4.8231514899999999, 2024-09-14T06:29:13.508 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.024927538999999999, 2024-09-14T06:29:13.508 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:29:13.508 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:29:13.508 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:29:13.508 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:29:13.508 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:29:13.508 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:13.509 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 1577 2024-09-14T06:29:13.509 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:13.509 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:29:13.509 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:13.509 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:29:13.509 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.630688+0000", 2024-09-14T06:29:13.509 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:13.509 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:13.509 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:13.509 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:29:13.509 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.630688+0000", 2024-09-14T06:29:13.509 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:13.509 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:13.509 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:13.510 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:29:13.510 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.630689+0000", 2024-09-14T06:29:13.510 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.2199999999999999e-07 2024-09-14T06:29:13.510 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:13.510 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:13.510 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:29:13.510 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.630691+0000", 2024-09-14T06:29:13.510 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.9410000000000001e-06 2024-09-14T06:29:13.510 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:13.510 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:13.510 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:29:13.510 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.630699+0000", 2024-09-14T06:29:13.510 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.0250000000000002e-06 2024-09-14T06:29:13.510 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:13.510 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:13.511 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:29:13.511 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.630704+0000", 2024-09-14T06:29:13.511 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.4349999999999996e-06 2024-09-14T06:29:13.511 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:13.511 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:13.511 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:29:13.511 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.655406+0000", 2024-09-14T06:29:13.511 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.024702279000000001 2024-09-14T06:29:13.511 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:13.511 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:13.511 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:29:13.511 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.655483+0000", 2024-09-14T06:29:13.511 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.6915999999999993e-05 2024-09-14T06:29:13.511 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:13.512 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:13.512 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:29:13.512 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.655616+0000", 2024-09-14T06:29:13.512 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.000132521 2024-09-14T06:29:13.512 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:13.512 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:29:13.512 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:13.512 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:13.512 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:13.512 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:1578 3.15 3:accf422f:::rbd_data.112d7dfbd766.0000000000000001:5 [read 389120~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-14T06:29:13.512 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:08.630711+0000", 2024-09-14T06:29:13.512 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 4.8231290180000004, 2024-09-14T06:29:13.512 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.025111108, 2024-09-14T06:29:13.513 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:29:13.513 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:29:13.513 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:29:13.513 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:29:13.513 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:29:13.513 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:13.513 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 1578 2024-09-14T06:29:13.513 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:13.513 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:29:13.513 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:13.513 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:29:13.513 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.630711+0000", 2024-09-14T06:29:13.513 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:13.513 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:13.514 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:13.514 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:29:13.514 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.630711+0000", 2024-09-14T06:29:13.514 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:13.514 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:13.514 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:13.514 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:29:13.514 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.630711+0000", 2024-09-14T06:29:13.514 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.01e-07 2024-09-14T06:29:13.514 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:13.514 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:13.514 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:29:13.514 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.630713+0000", 2024-09-14T06:29:13.515 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.8199999999999999e-06 2024-09-14T06:29:13.515 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:13.515 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:13.515 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:29:13.515 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.630720+0000", 2024-09-14T06:29:13.515 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.3880000000000002e-06 2024-09-14T06:29:13.515 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:13.515 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:13.515 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:29:13.515 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.630726+0000", 2024-09-14T06:29:13.515 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.3480000000000003e-06 2024-09-14T06:29:13.515 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:13.515 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:13.516 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:29:13.516 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.655646+0000", 2024-09-14T06:29:13.516 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.024920312999999999 2024-09-14T06:29:13.516 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:13.516 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:13.516 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:29:13.516 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.655724+0000", 2024-09-14T06:29:13.516 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.7589000000000005e-05 2024-09-14T06:29:13.516 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:13.516 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:13.516 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:29:13.516 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.655822+0000", 2024-09-14T06:29:13.516 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.8249000000000001e-05 2024-09-14T06:29:13.517 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:13.517 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:29:13.517 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:13.517 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:13.517 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:13.517 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:1579 3.15 3:accf422f:::rbd_data.112d7dfbd766.0000000000000001:5 [read 405504~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-14T06:29:13.517 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:08.630732+0000", 2024-09-14T06:29:13.517 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 4.8231076999999996, 2024-09-14T06:29:13.517 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.025303169, 2024-09-14T06:29:13.517 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:29:13.517 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:29:13.517 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:29:13.517 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:29:13.517 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:29:13.518 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:13.518 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 1579 2024-09-14T06:29:13.518 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:13.518 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:29:13.518 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:13.518 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:29:13.518 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.630732+0000", 2024-09-14T06:29:13.518 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:13.518 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:13.518 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:13.518 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:29:13.518 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.630732+0000", 2024-09-14T06:29:13.518 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:13.518 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:13.519 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:13.519 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:29:13.519 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.630732+0000", 2024-09-14T06:29:13.519 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.5199999999999998e-07 2024-09-14T06:29:13.519 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:13.519 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:13.519 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:29:13.519 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.630741+0000", 2024-09-14T06:29:13.519 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.1549999999999996e-06 2024-09-14T06:29:13.519 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:13.519 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:13.519 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:29:13.519 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.630748+0000", 2024-09-14T06:29:13.520 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.6429999999999998e-06 2024-09-14T06:29:13.520 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:13.520 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:13.520 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:29:13.520 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.630753+0000", 2024-09-14T06:29:13.520 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.0579999999999997e-06 2024-09-14T06:29:13.520 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:13.520 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:13.520 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:29:13.520 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.655858+0000", 2024-09-14T06:29:13.520 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.025104774999999999 2024-09-14T06:29:13.520 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:13.520 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:13.521 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:29:13.521 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.655942+0000", 2024-09-14T06:29:13.521 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.4129999999999996e-05 2024-09-14T06:29:13.521 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:13.521 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:13.521 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:29:13.521 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.656035+0000", 2024-09-14T06:29:13.521 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.3055999999999998e-05 2024-09-14T06:29:13.521 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:13.521 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:29:13.521 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:13.521 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:13.521 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:13.522 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:1580 3.15 3:accf422f:::rbd_data.112d7dfbd766.0000000000000001:5 [read 421888~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-14T06:29:13.522 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:08.630762+0000", 2024-09-14T06:29:13.522 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 4.8230779219999995, 2024-09-14T06:29:13.522 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.025487388999999999, 2024-09-14T06:29:13.522 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:29:13.522 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:29:13.522 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:29:13.522 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:29:13.522 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:29:13.522 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:13.523 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 1580 2024-09-14T06:29:13.523 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:13.523 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:29:13.523 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:13.523 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:29:13.523 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.630762+0000", 2024-09-14T06:29:13.523 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:13.523 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:13.523 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:13.523 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:29:13.523 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.630762+0000", 2024-09-14T06:29:13.523 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:13.523 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:13.524 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:13.524 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:29:13.524 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.630762+0000", 2024-09-14T06:29:13.524 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.2300000000000002e-07 2024-09-14T06:29:13.524 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:13.524 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:13.524 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:29:13.524 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.630764+0000", 2024-09-14T06:29:13.524 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.99e-06 2024-09-14T06:29:13.524 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:13.524 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:13.524 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:29:13.524 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.630771+0000", 2024-09-14T06:29:13.524 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.9349999999999997e-06 2024-09-14T06:29:13.525 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:13.525 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:13.525 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:29:13.525 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.630776+0000", 2024-09-14T06:29:13.525 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.1710000000000001e-06 2024-09-14T06:29:13.525 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:13.525 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:13.525 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:29:13.525 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.656068+0000", 2024-09-14T06:29:13.525 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.025291905999999999 2024-09-14T06:29:13.525 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:13.525 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:13.525 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:29:13.526 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.656151+0000", 2024-09-14T06:29:13.526 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.3209999999999998e-05 2024-09-14T06:29:13.526 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:13.526 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:13.526 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:29:13.526 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.656249+0000", 2024-09-14T06:29:13.526 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.7754e-05 2024-09-14T06:29:13.526 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:13.526 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:29:13.526 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:13.526 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:13.526 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:13.526 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:1581 3.15 3:accf422f:::rbd_data.112d7dfbd766.0000000000000001:5 [read 438272~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-14T06:29:13.527 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:08.630802+0000", 2024-09-14T06:29:13.527 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 4.8230372719999997, 2024-09-14T06:29:13.527 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.025670063, 2024-09-14T06:29:13.527 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:29:13.527 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:29:13.527 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:29:13.527 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:29:13.527 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:29:13.527 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:13.527 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 1581 2024-09-14T06:29:13.527 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:13.527 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:29:13.527 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:13.527 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:29:13.528 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.630802+0000", 2024-09-14T06:29:13.528 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:13.528 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:13.528 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:13.528 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:29:13.528 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.630802+0000", 2024-09-14T06:29:13.528 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:13.528 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:13.528 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:13.528 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:29:13.528 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.630803+0000", 2024-09-14T06:29:13.528 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.3099999999999997e-07 2024-09-14T06:29:13.528 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:13.528 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:13.528 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:29:13.529 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.630806+0000", 2024-09-14T06:29:13.529 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.7319999999999998e-06 2024-09-14T06:29:13.529 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:13.529 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:13.529 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:29:13.529 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.630813+0000", 2024-09-14T06:29:13.529 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.0840000000000003e-06 2024-09-14T06:29:13.529 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:13.529 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:13.529 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:29:13.529 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.630826+0000", 2024-09-14T06:29:13.529 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.3104e-05 2024-09-14T06:29:13.529 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:13.529 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:13.530 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:29:13.530 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.656349+0000", 2024-09-14T06:29:13.530 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.025523113 2024-09-14T06:29:13.530 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:13.530 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:13.530 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:29:13.530 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.656400+0000", 2024-09-14T06:29:13.530 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.1156000000000003e-05 2024-09-14T06:29:13.530 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:13.530 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:13.530 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:29:13.530 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.656472+0000", 2024-09-14T06:29:13.530 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.2243000000000006e-05 2024-09-14T06:29:13.530 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:13.531 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:29:13.531 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:13.531 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:13.531 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:13.531 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:1582 3.15 3:accf422f:::rbd_data.112d7dfbd766.0000000000000001:5 [read 454656~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-14T06:29:13.531 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:08.630838+0000", 2024-09-14T06:29:13.531 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 4.8230012479999997, 2024-09-14T06:29:13.531 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.025956396999999999, 2024-09-14T06:29:13.531 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:29:13.531 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:29:13.531 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:29:13.531 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:29:13.531 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:29:13.531 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:13.532 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 1582 2024-09-14T06:29:13.532 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:13.532 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:29:13.532 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:13.532 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:29:13.532 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.630838+0000", 2024-09-14T06:29:13.532 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:13.532 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:13.532 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:13.532 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:29:13.532 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.630838+0000", 2024-09-14T06:29:13.532 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:13.532 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:13.532 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:13.533 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:29:13.533 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.630839+0000", 2024-09-14T06:29:13.533 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.8400000000000004e-07 2024-09-14T06:29:13.533 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:13.533 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:13.533 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:29:13.533 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.630846+0000", 2024-09-14T06:29:13.533 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.4599999999999997e-06 2024-09-14T06:29:13.533 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:13.533 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:13.533 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:29:13.533 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.630854+0000", 2024-09-14T06:29:13.533 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.3139999999999998e-06 2024-09-14T06:29:13.534 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:13.534 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:13.534 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:29:13.536 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.630865+0000", 2024-09-14T06:29:13.536 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.0988999999999999e-05 2024-09-14T06:29:13.536 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:13.536 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:13.536 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:29:13.536 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.656598+0000", 2024-09-14T06:29:13.536 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.025733335 2024-09-14T06:29:13.536 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:13.537 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:13.537 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:29:13.537 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.656682+0000", 2024-09-14T06:29:13.537 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.4000999999999998e-05 2024-09-14T06:29:13.537 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:13.537 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:13.537 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:29:13.537 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.656795+0000", 2024-09-14T06:29:13.537 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00011271400000000001 2024-09-14T06:29:13.537 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:13.537 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:29:13.537 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:13.537 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:13.538 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:13.538 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:1583 3.15 3:accf422f:::rbd_data.112d7dfbd766.0000000000000001:5 [read 471040~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-14T06:29:13.538 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:08.630873+0000", 2024-09-14T06:29:13.538 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 4.8229663800000004, 2024-09-14T06:29:13.538 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.026337467, 2024-09-14T06:29:13.538 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:29:13.538 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:29:13.538 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:29:13.538 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:29:13.538 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:29:13.538 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:13.538 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 1583 2024-09-14T06:29:13.538 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:13.538 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:29:13.539 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:13.539 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:29:13.539 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.630873+0000", 2024-09-14T06:29:13.539 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:13.539 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:13.539 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:13.539 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:29:13.539 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.630873+0000", 2024-09-14T06:29:13.539 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:13.539 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:13.539 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:13.539 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:29:13.539 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.630874+0000", 2024-09-14T06:29:13.540 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.2500000000000005e-07 2024-09-14T06:29:13.540 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:13.540 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:13.540 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:29:13.540 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.630876+0000", 2024-09-14T06:29:13.540 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.2689999999999998e-06 2024-09-14T06:29:13.540 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:13.540 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:13.540 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:29:13.540 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.630883+0000", 2024-09-14T06:29:13.540 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.1919999999999999e-06 2024-09-14T06:29:13.540 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:13.540 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:13.540 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:29:13.541 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.630891+0000", 2024-09-14T06:29:13.541 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.1579999999999999e-06 2024-09-14T06:29:13.541 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:13.541 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:13.541 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:29:13.541 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.656836+0000", 2024-09-14T06:29:13.541 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.025945134000000002 2024-09-14T06:29:13.541 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:13.541 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:13.541 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:29:13.541 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.656920+0000", 2024-09-14T06:29:13.541 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.4351999999999995e-05 2024-09-14T06:29:13.541 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:13.541 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:13.542 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:29:13.542 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.657211+0000", 2024-09-14T06:29:13.542 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.000290737 2024-09-14T06:29:13.542 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:13.542 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:29:13.542 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:13.542 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:13.542 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:13.542 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:1584 3.15 3:accf422f:::rbd_data.112d7dfbd766.0000000000000001:5 [read 487424~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-14T06:29:13.542 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:08.630897+0000", 2024-09-14T06:29:13.542 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 4.8229422839999998, 2024-09-14T06:29:13.542 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.026720516, 2024-09-14T06:29:13.542 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:29:13.542 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:29:13.542 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:29:13.543 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:29:13.543 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:29:13.543 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:13.543 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 1584 2024-09-14T06:29:13.543 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:13.543 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:29:13.543 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:13.543 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:29:13.543 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.630897+0000", 2024-09-14T06:29:13.543 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:13.543 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:13.543 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:13.543 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:29:13.543 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.630897+0000", 2024-09-14T06:29:13.543 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:13.544 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:13.544 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:13.544 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:29:13.544 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.630898+0000", 2024-09-14T06:29:13.544 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.9200000000000002e-07 2024-09-14T06:29:13.544 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:13.544 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:13.544 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:29:13.544 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.630900+0000", 2024-09-14T06:29:13.544 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.7269999999999999e-06 2024-09-14T06:29:13.544 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:13.544 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:13.544 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:29:13.545 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.630906+0000", 2024-09-14T06:29:13.545 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.5849999999999996e-06 2024-09-14T06:29:13.545 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:13.545 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:13.545 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:29:13.545 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.630919+0000", 2024-09-14T06:29:13.545 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.3098999999999999e-05 2024-09-14T06:29:13.545 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:13.545 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:13.545 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:29:13.545 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.657253+0000", 2024-09-14T06:29:13.545 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.026333895999999999 2024-09-14T06:29:13.545 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:13.546 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:13.546 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:29:13.546 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.657342+0000", 2024-09-14T06:29:13.546 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.9044999999999999e-05 2024-09-14T06:29:13.546 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:13.546 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:13.546 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:29:13.546 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.657618+0000", 2024-09-14T06:29:13.546 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00027577199999999997 2024-09-14T06:29:13.546 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:13.546 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:29:13.546 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:13.546 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:13.546 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:13.547 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:1585 3.15 3:accf422f:::rbd_data.112d7dfbd766.0000000000000001:5 [read 503808~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-14T06:29:13.547 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:08.630927+0000", 2024-09-14T06:29:13.547 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 4.8229122819999999, 2024-09-14T06:29:13.547 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.027110801, 2024-09-14T06:29:13.547 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:29:13.547 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:29:13.547 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:29:13.547 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:29:13.547 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:29:13.547 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:13.547 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 1585 2024-09-14T06:29:13.547 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:13.547 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:29:13.547 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:13.548 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:29:13.548 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.630927+0000", 2024-09-14T06:29:13.548 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:13.548 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:13.548 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:13.548 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:29:13.548 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.630927+0000", 2024-09-14T06:29:13.548 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:13.548 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:13.548 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:13.548 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:29:13.548 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.630928+0000", 2024-09-14T06:29:13.548 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.7800000000000001e-07 2024-09-14T06:29:13.549 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:13.549 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:13.549 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:29:13.549 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.630930+0000", 2024-09-14T06:29:13.549 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.0829999999999998e-06 2024-09-14T06:29:13.549 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:13.549 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:13.549 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:29:13.549 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.630937+0000", 2024-09-14T06:29:13.549 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.7179999999999999e-06 2024-09-14T06:29:13.549 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:13.549 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:13.549 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:29:13.550 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.630942+0000", 2024-09-14T06:29:13.550 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.2989999999999996e-06 2024-09-14T06:29:13.550 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:13.550 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:13.550 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:29:13.550 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.657673+0000", 2024-09-14T06:29:13.550 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.026731390000000001 2024-09-14T06:29:13.550 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:13.550 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:13.550 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:29:13.550 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.657754+0000", 2024-09-14T06:29:13.550 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.0580000000000004e-05 2024-09-14T06:29:13.550 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:13.550 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:13.551 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:29:13.551 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.658038+0000", 2024-09-14T06:29:13.551 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00028415299999999999 2024-09-14T06:29:13.551 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:13.551 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:29:13.551 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:13.551 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:13.551 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:29:13.551 INFO:teuthology.orchestra.run.smithi138.stdout:} 2024-09-14T06:29:13.552 INFO:tasks.ceph.osd.1.smithi094.stderr:2024-09-14T06:29:13.551+0000 7f328e47a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33641) UID: 0 2024-09-14T06:29:13.553 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-14T06:29:13.567 INFO:teuthology.orchestra.run.smithi094.stdout:osd.0: {} 2024-09-14T06:29:13.568 INFO:teuthology.orchestra.run.smithi094.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-14T06:29:13.586 INFO:teuthology.orchestra.run.smithi094.stdout:osd.1: {} 2024-09-14T06:29:13.587 INFO:teuthology.orchestra.run.smithi094.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-14T06:29:13.603 INFO:teuthology.orchestra.run.smithi094.stdout:osd.2: {} 2024-09-14T06:29:13.603 INFO:teuthology.orchestra.run.smithi094.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-14T06:29:13.619 INFO:teuthology.orchestra.run.smithi094.stdout:osd.3: {} 2024-09-14T06:29:13.619 INFO:teuthology.orchestra.run.smithi094.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-14T06:29:13.639 INFO:teuthology.orchestra.run.smithi094.stdout:osd.4: {} 2024-09-14T06:29:13.639 INFO:teuthology.orchestra.run.smithi094.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-14T06:29:13.654 INFO:tasks.ceph.osd.6.smithi138.stderr:2024-09-14T06:29:13.653+0000 7f8cd8849640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33620) UID: 0 2024-09-14T06:29:13.656 INFO:teuthology.orchestra.run.smithi094.stdout:osd.5: {} 2024-09-14T06:29:13.656 INFO:teuthology.orchestra.run.smithi094.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-14T06:29:13.673 INFO:teuthology.orchestra.run.smithi094.stdout:osd.6: {} 2024-09-14T06:29:13.674 INFO:teuthology.orchestra.run.smithi094.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-14T06:29:13.677 INFO:teuthology.orchestra.run.smithi094.stdout:ERROR: (22) Invalid argument 2024-09-14T06:29:13.677 INFO:teuthology.orchestra.run.smithi094.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:29:13.687 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-14T06:29:13.691 INFO:teuthology.orchestra.run.smithi094.stdout:osd.7: {} 2024-09-14T06:29:13.691 INFO:teuthology.orchestra.run.smithi094.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-14T06:29:13.740 INFO:teuthology.orchestra.run.smithi094.stdout:checking clone #1, image image_client.0-clone1 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent2 2024-09-14T06:29:13.754 INFO:tasks.ceph.osd.7.smithi138.stderr:2024-09-14T06:29:13.753+0000 7f3c6a21d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33623) UID: 0 2024-09-14T06:29:13.810 INFO:teuthology.orchestra.run.smithi094.stdout:ERROR: (22) Invalid argument 2024-09-14T06:29:13.810 INFO:teuthology.orchestra.run.smithi094.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:29:13.822 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-14T06:29:13.863 INFO:tasks.ceph.osd.3.smithi094.stderr:2024-09-14T06:29:13.862+0000 7feeeafe8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33655) UID: 0 2024-09-14T06:29:13.957 INFO:tasks.ceph.osd.2.smithi094.stderr:2024-09-14T06:29:13.956+0000 7faf3269b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33649) UID: 0 2024-09-14T06:29:13.969 INFO:teuthology.orchestra.run.smithi094.stdout:ERROR: (22) Invalid argument 2024-09-14T06:29:13.969 INFO:teuthology.orchestra.run.smithi094.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:29:13.978 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-14T06:29:13.997 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-14T06:29:14.005 INFO:teuthology.orchestra.run.smithi094.stdout:339 trunc from 0x7d1af84 to 0xe024628 2024-09-14T06:29:14.014 INFO:teuthology.orchestra.run.smithi094.stdout:340 read 0x6603cf9 thru 0x66121ed (0xe4f5 bytes) 2024-09-14T06:29:14.014 INFO:teuthology.orchestra.run.smithi094.stdout:341 clone 4 order 22 su 2097152 sc 11 2024-09-14T06:29:14.058 INFO:tasks.ceph.osd.0.smithi094.stderr:2024-09-14T06:29:14.057+0000 7f07c9333640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33636) UID: 0 2024-09-14T06:29:14.101 INFO:teuthology.orchestra.run.smithi094.stdout:ERROR: (22) Invalid argument 2024-09-14T06:29:14.101 INFO:teuthology.orchestra.run.smithi094.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:29:14.110 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-14T06:29:14.159 INFO:tasks.ceph.osd.7.smithi138.stderr:2024-09-14T06:29:14.158+0000 7f3c6a21d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33623) UID: 0 2024-09-14T06:29:14.254 INFO:teuthology.orchestra.run.smithi094.stdout:ERROR: (22) Invalid argument 2024-09-14T06:29:14.254 INFO:teuthology.orchestra.run.smithi094.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:29:14.261 INFO:tasks.ceph.osd.4.smithi138.stderr:2024-09-14T06:29:14.260+0000 7f6c653c8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33615) UID: 0 2024-09-14T06:29:14.263 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-14T06:29:14.362 INFO:tasks.ceph.osd.5.smithi138.stderr:2024-09-14T06:29:14.361+0000 7fe13746f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33618) UID: 0 2024-09-14T06:29:14.366 INFO:teuthology.orchestra.run.smithi094.stdout:ERROR: (22) Invalid argument 2024-09-14T06:29:14.366 INFO:teuthology.orchestra.run.smithi094.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:29:14.374 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-14T06:29:14.463 INFO:tasks.ceph.osd.1.smithi094.stderr:2024-09-14T06:29:14.462+0000 7f328e47a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33641) UID: 0 2024-09-14T06:29:14.477 INFO:teuthology.orchestra.run.smithi094.stdout:ERROR: (22) Invalid argument 2024-09-14T06:29:14.477 INFO:teuthology.orchestra.run.smithi094.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:29:14.486 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-14T06:29:14.565 INFO:tasks.ceph.osd.5.smithi138.stderr:2024-09-14T06:29:14.564+0000 7fe13746f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33618) UID: 0 2024-09-14T06:29:14.631 INFO:teuthology.orchestra.run.smithi094.stdout:ERROR: (22) Invalid argument 2024-09-14T06:29:14.631 INFO:teuthology.orchestra.run.smithi094.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:29:14.637 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-14T06:29:14.641 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-14T06:29:14.666 INFO:tasks.ceph.osd.7.smithi138.stderr:2024-09-14T06:29:14.665+0000 7f3c6a21d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33623) UID: 0 2024-09-14T06:29:14.767 INFO:tasks.ceph.osd.1.smithi094.stderr:2024-09-14T06:29:14.766+0000 7f328e47a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33641) UID: 0 2024-09-14T06:29:14.787 INFO:teuthology.orchestra.run.smithi094.stdout:ERROR: (22) Invalid argument 2024-09-14T06:29:14.787 INFO:teuthology.orchestra.run.smithi094.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:29:14.796 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-14T06:29:14.869 INFO:tasks.ceph.osd.5.smithi138.stderr:2024-09-14T06:29:14.868+0000 7fe13746f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33618) UID: 0 2024-09-14T06:29:14.941 INFO:teuthology.orchestra.run.smithi094.stdout:ERROR: (22) Invalid argument 2024-09-14T06:29:14.941 INFO:teuthology.orchestra.run.smithi094.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:29:14.949 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-14T06:29:14.970 INFO:tasks.ceph.osd.6.smithi138.stderr:2024-09-14T06:29:14.969+0000 7f8cd8849640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33620) UID: 0 2024-09-14T06:29:15.052 INFO:teuthology.orchestra.run.smithi094.stdout:ERROR: (22) Invalid argument 2024-09-14T06:29:15.052 INFO:teuthology.orchestra.run.smithi094.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:29:15.060 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-14T06:29:15.070 INFO:tasks.ceph.osd.6.smithi138.stderr:2024-09-14T06:29:15.069+0000 7f8cd8849640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33620) UID: 0 2024-09-14T06:29:15.162 INFO:teuthology.orchestra.run.smithi094.stdout:ERROR: (22) Invalid argument 2024-09-14T06:29:15.162 INFO:teuthology.orchestra.run.smithi094.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:29:15.171 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-14T06:29:15.171 INFO:tasks.ceph.osd.3.smithi094.stderr:2024-09-14T06:29:15.169+0000 7feeeafe8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33655) UID: 0 2024-09-14T06:29:15.272 INFO:tasks.ceph.osd.0.smithi094.stderr:2024-09-14T06:29:15.271+0000 7f07c9333640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33636) UID: 0 2024-09-14T06:29:15.284 INFO:teuthology.orchestra.run.smithi138.stdout:ERROR: (22) Invalid argument 2024-09-14T06:29:15.284 INFO:teuthology.orchestra.run.smithi138.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:29:15.294 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-14T06:29:15.373 INFO:tasks.ceph.osd.5.smithi138.stderr:2024-09-14T06:29:15.372+0000 7fe13746f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33618) UID: 0 2024-09-14T06:29:15.398 INFO:teuthology.orchestra.run.smithi138.stdout:ERROR: (22) Invalid argument 2024-09-14T06:29:15.398 INFO:teuthology.orchestra.run.smithi138.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:29:15.408 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-14T06:29:15.474 INFO:tasks.ceph.osd.0.smithi094.stderr:2024-09-14T06:29:15.473+0000 7f07c9333640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33636) UID: 0 2024-09-14T06:29:15.550 INFO:teuthology.orchestra.run.smithi138.stdout:ERROR: (22) Invalid argument 2024-09-14T06:29:15.551 INFO:teuthology.orchestra.run.smithi138.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:29:15.560 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-14T06:29:15.570 INFO:teuthology.orchestra.run.smithi094.stderr:nodeep-scrub is set 2024-09-14T06:29:15.575 INFO:tasks.ceph.osd.0.smithi094.stderr:2024-09-14T06:29:15.574+0000 7f07c9333640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33636) UID: 0 2024-09-14T06:29:15.599 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-14T06:29:15.600 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-14T06:29:15.600 INFO:tasks.thrashosds.thrasher:inject_pause on osd.2 2024-09-14T06:29:15.600 INFO:tasks.thrashosds.thrasher:Testing filestore_inject_stall pause injection for duration 3 2024-09-14T06:29:15.600 INFO:tasks.thrashosds.thrasher:Checking after 0, should_be_down=False 2024-09-14T06:29:15.600 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok config set filestore_inject_stall 3 2024-09-14T06:29:15.673 INFO:teuthology.orchestra.run.smithi138.stdout:ERROR: (22) Invalid argument 2024-09-14T06:29:15.673 INFO:teuthology.orchestra.run.smithi138.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:29:15.675 INFO:tasks.ceph.osd.7.smithi138.stderr:2024-09-14T06:29:15.674+0000 7f3c6a21d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33623) UID: 0 2024-09-14T06:29:15.682 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-14T06:29:15.705 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-14T06:29:15.731 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-14T06:29:15.731 INFO:teuthology.orchestra.run.smithi094.stdout: "success": "filestore_inject_stall = '' (not observed, change may require restart) " 2024-09-14T06:29:15.731 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-14T06:29:15.782 INFO:tasks.ceph.osd.2.smithi094.stderr:2024-09-14T06:29:15.781+0000 7faf3269b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33649) UID: 0 2024-09-14T06:29:15.825 INFO:teuthology.orchestra.run.smithi094.stdout:osd.0: {} 2024-09-14T06:29:15.826 INFO:teuthology.orchestra.run.smithi094.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-14T06:29:15.826 INFO:teuthology.orchestra.run.smithi138.stdout:ERROR: (22) Invalid argument 2024-09-14T06:29:15.826 INFO:teuthology.orchestra.run.smithi138.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:29:15.834 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-14T06:29:15.842 INFO:teuthology.orchestra.run.smithi094.stdout:osd.1: {} 2024-09-14T06:29:15.842 INFO:teuthology.orchestra.run.smithi094.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-14T06:29:15.858 INFO:teuthology.orchestra.run.smithi094.stdout:osd.2: {} 2024-09-14T06:29:15.858 INFO:teuthology.orchestra.run.smithi094.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-14T06:29:15.874 INFO:teuthology.orchestra.run.smithi094.stdout:osd.3: {} 2024-09-14T06:29:15.874 INFO:teuthology.orchestra.run.smithi094.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-14T06:29:15.877 INFO:tasks.ceph.osd.0.smithi094.stderr:2024-09-14T06:29:15.876+0000 7f07c9333640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33636) UID: 0 2024-09-14T06:29:15.890 INFO:teuthology.orchestra.run.smithi094.stdout:osd.4: {} 2024-09-14T06:29:15.890 INFO:teuthology.orchestra.run.smithi094.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-14T06:29:15.907 INFO:teuthology.orchestra.run.smithi094.stdout:osd.5: {} 2024-09-14T06:29:15.907 INFO:teuthology.orchestra.run.smithi094.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-14T06:29:15.923 INFO:teuthology.orchestra.run.smithi094.stdout:osd.6: {} 2024-09-14T06:29:15.923 INFO:teuthology.orchestra.run.smithi094.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-14T06:29:15.939 INFO:teuthology.orchestra.run.smithi138.stdout:{ 2024-09-14T06:29:15.940 INFO:teuthology.orchestra.run.smithi138.stdout: "size": 20, 2024-09-14T06:29:15.940 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 600, 2024-09-14T06:29:15.940 INFO:teuthology.orchestra.run.smithi138.stdout: "ops": [ 2024-09-14T06:29:15.940 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:15.940 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:3256 3.5 3:a6acff7e:::rbd_data.112d7dfbd766.0000000000000078:5 [read 110592~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-14T06:29:15.940 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:08.738879+0000", 2024-09-14T06:29:15.940 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 7.1992545290000001, 2024-09-14T06:29:15.940 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.011432067000000001, 2024-09-14T06:29:15.940 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:29:15.940 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:29:15.940 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:29:15.941 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:29:15.941 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:29:15.941 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:15.941 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 3256 2024-09-14T06:29:15.941 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:15.941 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:29:15.941 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:15.941 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:29:15.941 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.738879+0000", 2024-09-14T06:29:15.941 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:15.941 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:15.941 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:15.941 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:29:15.942 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.738879+0000", 2024-09-14T06:29:15.942 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:15.942 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:15.942 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:15.942 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:29:15.942 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.738879+0000", 2024-09-14T06:29:15.942 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.8099999999999998e-07 2024-09-14T06:29:15.942 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:15.942 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:15.942 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:29:15.942 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.738885+0000", 2024-09-14T06:29:15.942 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.3199999999999999e-06 2024-09-14T06:29:15.942 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:15.942 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:15.943 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:29:15.943 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.738891+0000", 2024-09-14T06:29:15.943 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.3149999999999997e-06 2024-09-14T06:29:15.943 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:15.943 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:15.943 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:29:15.943 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.738895+0000", 2024-09-14T06:29:15.943 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.478e-06 2024-09-14T06:29:15.943 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:15.943 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:15.943 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:29:15.943 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.750221+0000", 2024-09-14T06:29:15.943 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.011326166 2024-09-14T06:29:15.943 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:15.944 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:15.944 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:29:15.944 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.750260+0000", 2024-09-14T06:29:15.944 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.9239999999999997e-05 2024-09-14T06:29:15.944 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:15.944 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:15.944 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:29:15.944 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.750311+0000", 2024-09-14T06:29:15.944 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.1167e-05 2024-09-14T06:29:15.944 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:15.944 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:29:15.944 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:15.944 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:15.945 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:15.945 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:3257 3.5 3:a6acff7e:::rbd_data.112d7dfbd766.0000000000000078:5 [read 126976~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-14T06:29:15.945 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:08.738902+0000", 2024-09-14T06:29:15.945 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 7.1992318370000001, 2024-09-14T06:29:15.945 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.011528261999999999, 2024-09-14T06:29:15.945 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:29:15.945 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:29:15.945 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:29:15.945 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:29:15.945 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:29:15.945 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:15.945 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 3257 2024-09-14T06:29:15.945 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:15.945 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:29:15.946 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:15.946 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:29:15.946 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.738902+0000", 2024-09-14T06:29:15.946 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:15.946 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:15.946 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:15.946 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:29:15.946 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.738902+0000", 2024-09-14T06:29:15.946 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:15.946 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:15.946 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:15.946 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:29:15.946 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.738902+0000", 2024-09-14T06:29:15.946 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.1899999999999998e-07 2024-09-14T06:29:15.946 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:15.947 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:15.947 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:29:15.947 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.738904+0000", 2024-09-14T06:29:15.947 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.872e-06 2024-09-14T06:29:15.947 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:15.947 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:15.947 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:29:15.947 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.738910+0000", 2024-09-14T06:29:15.947 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.8749999999999997e-06 2024-09-14T06:29:15.947 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:15.947 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:15.947 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:29:15.947 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.738918+0000", 2024-09-14T06:29:15.947 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.2570000000000007e-06 2024-09-14T06:29:15.947 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:15.948 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:15.948 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:29:15.948 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.750331+0000", 2024-09-14T06:29:15.948 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.011412813000000001 2024-09-14T06:29:15.948 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:15.948 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:15.948 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:29:15.948 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.750373+0000", 2024-09-14T06:29:15.948 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.1754e-05 2024-09-14T06:29:15.948 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:15.948 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:15.948 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:29:15.948 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.750430+0000", 2024-09-14T06:29:15.948 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.7272000000000003e-05 2024-09-14T06:29:15.949 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:15.949 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:29:15.949 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:15.949 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:15.949 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:15.949 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:3258 3.5 3:a6acff7e:::rbd_data.112d7dfbd766.0000000000000078:5 [read 143360~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-14T06:29:15.949 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:08.738925+0000", 2024-09-14T06:29:15.949 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 7.1992081399999996, 2024-09-14T06:29:15.949 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.01161296, 2024-09-14T06:29:15.949 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:29:15.949 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:29:15.949 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:29:15.949 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:29:15.949 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:29:15.950 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:15.950 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 3258 2024-09-14T06:29:15.950 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:15.950 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:29:15.950 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:15.950 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:29:15.950 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.738925+0000", 2024-09-14T06:29:15.950 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:15.950 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:15.950 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:15.950 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:29:15.950 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.738925+0000", 2024-09-14T06:29:15.950 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:15.951 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:15.951 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:15.951 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:29:15.951 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.738926+0000", 2024-09-14T06:29:15.951 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.1100000000000001e-07 2024-09-14T06:29:15.951 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:15.951 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:15.951 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:29:15.951 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.738928+0000", 2024-09-14T06:29:15.951 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.0679999999999999e-06 2024-09-14T06:29:15.951 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:15.951 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:15.951 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:29:15.951 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.738934+0000", 2024-09-14T06:29:15.951 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.054e-06 2024-09-14T06:29:15.952 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:15.952 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:15.952 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:29:15.952 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.738938+0000", 2024-09-14T06:29:15.952 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.4429999999999996e-06 2024-09-14T06:29:15.952 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:15.952 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:15.952 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:29:15.952 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.750448+0000", 2024-09-14T06:29:15.952 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.011510016 2024-09-14T06:29:15.952 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:15.952 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:15.952 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:29:15.952 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.750488+0000", 2024-09-14T06:29:15.953 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.9402e-05 2024-09-14T06:29:15.953 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:15.953 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:15.953 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:29:15.953 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.750538+0000", 2024-09-14T06:29:15.953 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.0565999999999997e-05 2024-09-14T06:29:15.953 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:15.953 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:29:15.953 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:15.953 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:15.953 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:15.953 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:3259 3.5 3:a6acff7e:::rbd_data.112d7dfbd766.0000000000000078:5 [read 159744~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-14T06:29:15.953 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:08.738946+0000", 2024-09-14T06:29:15.954 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 7.1991871290000002, 2024-09-14T06:29:15.954 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.011711767999999999, 2024-09-14T06:29:15.954 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:29:15.954 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:29:15.954 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:29:15.954 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:29:15.954 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:29:15.954 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:15.954 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 3259 2024-09-14T06:29:15.954 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:15.954 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:29:15.954 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:15.954 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:29:15.955 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.738946+0000", 2024-09-14T06:29:15.955 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:15.955 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:15.955 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:15.955 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:29:15.955 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.738946+0000", 2024-09-14T06:29:15.955 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:15.955 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:15.955 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:15.955 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:29:15.956 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.738947+0000", 2024-09-14T06:29:15.956 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.5400000000000002e-07 2024-09-14T06:29:15.956 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:15.956 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:15.956 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:29:15.956 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.738949+0000", 2024-09-14T06:29:15.956 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.0260000000000002e-06 2024-09-14T06:29:15.956 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:15.956 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:15.956 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:29:15.956 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.738955+0000", 2024-09-14T06:29:15.956 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.3709999999999996e-06 2024-09-14T06:29:15.956 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:15.956 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:15.957 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:29:15.957 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.738960+0000", 2024-09-14T06:29:15.957 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.7369999999999997e-06 2024-09-14T06:29:15.957 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:15.957 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:15.957 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:29:15.957 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.750560+0000", 2024-09-14T06:29:15.957 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.011600103 2024-09-14T06:29:15.957 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:15.957 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:15.957 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:29:15.957 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.750605+0000", 2024-09-14T06:29:15.957 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.4799999999999998e-05 2024-09-14T06:29:15.957 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:15.957 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:15.958 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:29:15.958 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.750658+0000", 2024-09-14T06:29:15.958 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.3377000000000001e-05 2024-09-14T06:29:15.958 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:15.958 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:29:15.958 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:15.958 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:15.958 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:15.958 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:3260 3.5 3:a6acff7e:::rbd_data.112d7dfbd766.0000000000000078:5 [read 176128~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-14T06:29:15.958 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:08.738967+0000", 2024-09-14T06:29:15.958 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 7.1991663280000004, 2024-09-14T06:29:15.958 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.011809811, 2024-09-14T06:29:15.958 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:29:15.958 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:29:15.959 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:29:15.959 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:29:15.959 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:29:15.959 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:15.959 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 3260 2024-09-14T06:29:15.959 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:15.959 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:29:15.959 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:15.959 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:29:15.959 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.738967+0000", 2024-09-14T06:29:15.959 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:15.959 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:15.959 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:15.960 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:29:15.960 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.738967+0000", 2024-09-14T06:29:15.960 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:15.960 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:15.960 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:15.960 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:29:15.960 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.738968+0000", 2024-09-14T06:29:15.960 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.2599999999999998e-07 2024-09-14T06:29:15.960 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:15.960 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:15.960 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:29:15.960 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.738970+0000", 2024-09-14T06:29:15.960 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.0729999999999999e-06 2024-09-14T06:29:15.960 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:15.961 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:15.961 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:29:15.961 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.738976+0000", 2024-09-14T06:29:15.961 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.6830000000000002e-06 2024-09-14T06:29:15.961 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:15.961 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:15.961 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:29:15.961 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.738981+0000", 2024-09-14T06:29:15.961 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.6299999999999997e-06 2024-09-14T06:29:15.961 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:15.961 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:15.961 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:29:15.961 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.750681+0000", 2024-09-14T06:29:15.961 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.011700093999999999 2024-09-14T06:29:15.961 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:15.962 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:15.962 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:29:15.962 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.750724+0000", 2024-09-14T06:29:15.962 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.295e-05 2024-09-14T06:29:15.962 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:15.962 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:15.962 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:29:15.962 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.750777+0000", 2024-09-14T06:29:15.962 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.2954999999999997e-05 2024-09-14T06:29:15.962 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:15.962 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:29:15.962 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:15.962 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:15.962 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:15.963 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:3261 3.5 3:a6acff7e:::rbd_data.112d7dfbd766.0000000000000078:5 [read 192512~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-14T06:29:15.963 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:08.738988+0000", 2024-09-14T06:29:15.963 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 7.1991459510000002, 2024-09-14T06:29:15.963 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.011907709000000001, 2024-09-14T06:29:15.963 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:29:15.963 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:29:15.963 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:29:15.963 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:29:15.963 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:29:15.963 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:15.963 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 3261 2024-09-14T06:29:15.963 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:15.963 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:29:15.963 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:15.964 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:29:15.964 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.738988+0000", 2024-09-14T06:29:15.964 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:15.964 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:15.964 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:15.964 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:29:15.964 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.738988+0000", 2024-09-14T06:29:15.964 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:15.964 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:15.964 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:15.964 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:29:15.964 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.738988+0000", 2024-09-14T06:29:15.964 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.9200000000000002e-07 2024-09-14T06:29:15.964 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:15.965 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:15.965 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:29:15.965 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.738995+0000", 2024-09-14T06:29:15.965 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.8399999999999997e-06 2024-09-14T06:29:15.965 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:15.965 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:15.965 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:29:15.965 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.739001+0000", 2024-09-14T06:29:15.965 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.2319999999999998e-06 2024-09-14T06:29:15.965 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:15.965 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:15.965 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:29:15.965 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.739007+0000", 2024-09-14T06:29:15.965 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.57e-06 2024-09-14T06:29:15.966 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:15.966 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:15.966 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:29:15.966 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.750798+0000", 2024-09-14T06:29:15.966 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.011790995 2024-09-14T06:29:15.966 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:15.966 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:15.966 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:29:15.966 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.750837+0000", 2024-09-14T06:29:15.966 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.9765999999999998e-05 2024-09-14T06:29:15.966 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:15.966 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:15.966 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:29:15.966 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.750895+0000", 2024-09-14T06:29:15.966 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.7914000000000002e-05 2024-09-14T06:29:15.967 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:15.967 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:29:15.967 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:15.967 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:15.967 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:15.967 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:3262 3.5 3:a6acff7e:::rbd_data.112d7dfbd766.0000000000000078:5 [read 208896~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-14T06:29:15.967 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:08.739013+0000", 2024-09-14T06:29:15.967 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 7.1991203620000004, 2024-09-14T06:29:15.967 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.012002934999999999, 2024-09-14T06:29:15.967 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:29:15.967 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:29:15.967 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:29:15.967 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:29:15.967 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:29:15.967 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:15.968 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 3262 2024-09-14T06:29:15.968 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:15.968 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:29:15.968 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:15.968 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:29:15.968 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.739013+0000", 2024-09-14T06:29:15.968 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:15.968 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:15.968 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:15.968 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:29:15.968 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.739013+0000", 2024-09-14T06:29:15.968 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:15.968 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:15.968 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:15.968 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:29:15.969 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.739014+0000", 2024-09-14T06:29:15.969 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.3099999999999998e-07 2024-09-14T06:29:15.969 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:15.969 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:15.969 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:29:15.969 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.739015+0000", 2024-09-14T06:29:15.969 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.818e-06 2024-09-14T06:29:15.969 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:15.969 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:15.969 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:29:15.969 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.739023+0000", 2024-09-14T06:29:15.969 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.2559999999999996e-06 2024-09-14T06:29:15.969 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:15.969 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:15.970 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:29:15.970 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.739028+0000", 2024-09-14T06:29:15.970 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.7870000000000001e-06 2024-09-14T06:29:15.970 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:15.970 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:15.970 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:29:15.970 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.750921+0000", 2024-09-14T06:29:15.970 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.011893470999999999 2024-09-14T06:29:15.970 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:15.970 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:15.970 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:29:15.970 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.750964+0000", 2024-09-14T06:29:15.970 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.3229000000000002e-05 2024-09-14T06:29:15.970 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:15.971 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:15.971 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:29:15.971 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.751016+0000", 2024-09-14T06:29:15.971 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.1943000000000001e-05 2024-09-14T06:29:15.971 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:15.971 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:29:15.971 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:15.971 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:15.971 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:15.971 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:3263 3.5 3:a6acff7e:::rbd_data.112d7dfbd766.0000000000000078:5 [read 225280~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-14T06:29:15.971 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:08.739032+0000", 2024-09-14T06:29:15.971 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 7.1991012210000003, 2024-09-14T06:29:15.971 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.012101318, 2024-09-14T06:29:15.971 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:29:15.972 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:29:15.972 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:29:15.972 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:29:15.972 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:29:15.972 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:15.972 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 3263 2024-09-14T06:29:15.972 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:15.972 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:29:15.972 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:15.972 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:29:15.972 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.739032+0000", 2024-09-14T06:29:15.972 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:15.972 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:15.972 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:15.973 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:29:15.973 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.739032+0000", 2024-09-14T06:29:15.973 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:15.973 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:15.973 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:15.973 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:29:15.973 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.739033+0000", 2024-09-14T06:29:15.973 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.5800000000000001e-07 2024-09-14T06:29:15.973 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:15.973 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:15.973 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:29:15.973 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.739034+0000", 2024-09-14T06:29:15.973 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.136e-06 2024-09-14T06:29:15.974 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:15.974 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:15.974 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:29:15.974 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.739037+0000", 2024-09-14T06:29:15.974 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.2370000000000001e-06 2024-09-14T06:29:15.974 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:15.974 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:15.974 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:29:15.974 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.739040+0000", 2024-09-14T06:29:15.974 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.3739999999999998e-06 2024-09-14T06:29:15.974 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:15.974 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:15.974 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:29:15.974 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.751036+0000", 2024-09-14T06:29:15.975 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.011995178 2024-09-14T06:29:15.975 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:15.975 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:15.975 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:29:15.975 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.751078+0000", 2024-09-14T06:29:15.975 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.2519999999999999e-05 2024-09-14T06:29:15.975 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:15.975 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:15.975 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:29:15.975 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.751134+0000", 2024-09-14T06:29:15.975 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.5615000000000002e-05 2024-09-14T06:29:15.975 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:15.975 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:29:15.975 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:15.976 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:15.976 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:15.976 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:3264 3.5 3:a6acff7e:::rbd_data.112d7dfbd766.0000000000000078:5 [read 241664~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-14T06:29:15.976 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:08.739043+0000", 2024-09-14T06:29:15.976 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 7.1990902080000003, 2024-09-14T06:29:15.976 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.012198746, 2024-09-14T06:29:15.976 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:29:15.976 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:29:15.976 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:29:15.976 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:29:15.976 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:29:15.976 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:15.976 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 3264 2024-09-14T06:29:15.977 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:15.977 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:29:15.977 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:15.977 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:29:15.977 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.739043+0000", 2024-09-14T06:29:15.977 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:15.977 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:15.977 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:15.977 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:29:15.977 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.739043+0000", 2024-09-14T06:29:15.977 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:15.977 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:15.977 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:15.977 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:29:15.978 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.739044+0000", 2024-09-14T06:29:15.978 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.28e-07 2024-09-14T06:29:15.978 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:15.978 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:15.978 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:29:15.978 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.739045+0000", 2024-09-14T06:29:15.978 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.78e-07 2024-09-14T06:29:15.978 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:15.978 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:15.978 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:29:15.978 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.739048+0000", 2024-09-14T06:29:15.978 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.9950000000000001e-06 2024-09-14T06:29:15.978 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:15.978 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:15.978 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:29:15.979 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.739051+0000", 2024-09-14T06:29:15.979 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.0340000000000001e-06 2024-09-14T06:29:15.979 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:15.979 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:15.979 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:29:15.979 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.751154+0000", 2024-09-14T06:29:15.979 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.012103541000000001 2024-09-14T06:29:15.979 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:15.979 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:15.979 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:29:15.979 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.751193+0000", 2024-09-14T06:29:15.979 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.9048000000000003e-05 2024-09-14T06:29:15.979 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:15.979 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:15.979 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:29:15.980 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.751242+0000", 2024-09-14T06:29:15.980 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.8921999999999997e-05 2024-09-14T06:29:15.980 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:15.980 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:29:15.980 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:15.980 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:15.980 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:15.980 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:3265 3.5 3:a6acff7e:::rbd_data.112d7dfbd766.0000000000000078:5 [read 258048~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-14T06:29:15.980 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:08.739053+0000", 2024-09-14T06:29:15.980 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 7.1990800879999997, 2024-09-14T06:29:15.980 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.01230612, 2024-09-14T06:29:15.980 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:29:15.980 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:29:15.980 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:29:15.981 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:29:15.981 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:29:15.981 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:15.981 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 3265 2024-09-14T06:29:15.981 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:15.981 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:29:15.981 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:15.981 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:29:15.981 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.739053+0000", 2024-09-14T06:29:15.981 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:15.981 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:15.981 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:15.981 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:29:15.981 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.739053+0000", 2024-09-14T06:29:15.982 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:15.982 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:15.982 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:15.982 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:29:15.982 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.739054+0000", 2024-09-14T06:29:15.982 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.3999999999999998e-07 2024-09-14T06:29:15.982 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:15.982 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:15.982 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:29:15.982 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.739055+0000", 2024-09-14T06:29:15.982 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.2099999999999995e-07 2024-09-14T06:29:15.982 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:15.982 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:15.982 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:29:15.983 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.739058+0000", 2024-09-14T06:29:15.983 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.9340000000000002e-06 2024-09-14T06:29:15.983 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:15.983 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:15.983 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:29:15.983 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.739060+0000", 2024-09-14T06:29:15.983 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.8030000000000001e-06 2024-09-14T06:29:15.983 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:15.983 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:15.983 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:29:15.983 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.751263+0000", 2024-09-14T06:29:15.983 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.012202821000000001 2024-09-14T06:29:15.983 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:15.983 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:15.983 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:29:15.984 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.751308+0000", 2024-09-14T06:29:15.984 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.4802999999999998e-05 2024-09-14T06:29:15.984 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:15.984 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:15.984 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:29:15.984 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.751360+0000", 2024-09-14T06:29:15.984 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.1598000000000003e-05 2024-09-14T06:29:15.984 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:15.984 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:29:15.984 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:15.984 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:15.984 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:15.984 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:3266 3.5 3:a6acff7e:::rbd_data.112d7dfbd766.0000000000000078:5 [read 274432~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-14T06:29:15.984 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:08.739063+0000", 2024-09-14T06:29:15.985 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 7.1990704980000002, 2024-09-14T06:29:15.985 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.012402339, 2024-09-14T06:29:15.985 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:29:15.985 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:29:15.985 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:29:15.985 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:29:15.985 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:29:15.985 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:15.985 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 3266 2024-09-14T06:29:15.985 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:15.985 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:29:15.985 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:15.985 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:29:15.985 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.739063+0000", 2024-09-14T06:29:15.985 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:15.986 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:15.986 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:15.986 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:29:15.986 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.739063+0000", 2024-09-14T06:29:15.986 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:15.986 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:15.986 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:15.986 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:29:15.986 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.739063+0000", 2024-09-14T06:29:15.986 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.2000000000000001e-07 2024-09-14T06:29:15.986 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:15.986 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:15.986 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:29:15.986 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.739065+0000", 2024-09-14T06:29:15.986 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.299e-06 2024-09-14T06:29:15.987 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:15.987 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:15.987 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:29:15.987 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.739069+0000", 2024-09-14T06:29:15.987 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.1309999999999999e-06 2024-09-14T06:29:15.987 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:15.987 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:15.987 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:29:15.987 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.739071+0000", 2024-09-14T06:29:15.987 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.4279999999999999e-06 2024-09-14T06:29:15.987 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:15.987 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:15.987 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:29:15.987 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.751381+0000", 2024-09-14T06:29:15.988 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.012309394 2024-09-14T06:29:15.988 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:15.988 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:15.988 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:29:15.988 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.751419+0000", 2024-09-14T06:29:15.988 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.8068000000000002e-05 2024-09-14T06:29:15.988 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:15.988 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:15.988 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:29:15.988 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.751465+0000", 2024-09-14T06:29:15.988 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.6798999999999997e-05 2024-09-14T06:29:15.988 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:15.988 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:29:15.988 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:15.989 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:15.989 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:15.989 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:3267 3.5 3:a6acff7e:::rbd_data.112d7dfbd766.0000000000000078:5 [read 290816~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-14T06:29:15.989 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:08.739074+0000", 2024-09-14T06:29:15.989 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 7.1990593409999999, 2024-09-14T06:29:15.989 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.012508533, 2024-09-14T06:29:15.989 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:29:15.989 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:29:15.989 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:29:15.989 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:29:15.989 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:29:15.989 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:15.989 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 3267 2024-09-14T06:29:15.990 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:15.990 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:29:15.990 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:15.990 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:29:15.990 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.739074+0000", 2024-09-14T06:29:15.990 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:15.990 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:15.990 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:15.990 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:29:15.990 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.739074+0000", 2024-09-14T06:29:15.990 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:15.990 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:15.990 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:15.990 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:29:15.991 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.739074+0000", 2024-09-14T06:29:15.991 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.2000000000000001e-07 2024-09-14T06:29:15.991 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:15.991 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:15.991 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:29:15.991 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.739075+0000", 2024-09-14T06:29:15.991 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.2299999999999999e-07 2024-09-14T06:29:15.991 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:15.991 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:15.991 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:29:15.991 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.739079+0000", 2024-09-14T06:29:15.991 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.6500000000000002e-06 2024-09-14T06:29:15.991 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:15.992 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:15.992 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:29:15.992 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.739083+0000", 2024-09-14T06:29:15.992 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.856e-06 2024-09-14T06:29:15.992 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:15.992 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:15.992 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:29:15.992 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.751485+0000", 2024-09-14T06:29:15.992 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.012401772 2024-09-14T06:29:15.992 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:15.992 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:15.992 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:29:15.992 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.751529+0000", 2024-09-14T06:29:15.993 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.4542000000000001e-05 2024-09-14T06:29:15.993 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:15.993 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:15.993 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:29:15.993 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.751583+0000", 2024-09-14T06:29:15.993 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.3569999999999997e-05 2024-09-14T06:29:15.993 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:15.993 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:29:15.993 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:15.993 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:15.993 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:15.993 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:3268 3.5 3:a6acff7e:::rbd_data.112d7dfbd766.0000000000000078:5 [read 307200~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-14T06:29:15.993 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:08.739088+0000", 2024-09-14T06:29:15.993 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 7.1990453130000001, 2024-09-14T06:29:15.993 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.012618495, 2024-09-14T06:29:15.994 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:29:15.994 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:29:15.994 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:29:15.994 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:29:15.994 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:29:15.994 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:15.994 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 3268 2024-09-14T06:29:15.994 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:15.994 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:29:15.994 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:15.994 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:29:15.994 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.739088+0000", 2024-09-14T06:29:15.994 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:15.994 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:15.994 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:15.994 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:29:15.995 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.739088+0000", 2024-09-14T06:29:15.995 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:15.995 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:15.995 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:15.995 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:29:15.995 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.739089+0000", 2024-09-14T06:29:15.995 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.4400000000000001e-07 2024-09-14T06:29:15.995 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:15.995 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:15.995 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:29:15.995 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.739090+0000", 2024-09-14T06:29:15.995 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.0669999999999999e-06 2024-09-14T06:29:15.995 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:15.995 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:15.995 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:29:15.996 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.739093+0000", 2024-09-14T06:29:15.996 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.3189999999999999e-06 2024-09-14T06:29:15.996 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:15.996 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:15.996 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:29:15.996 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.739095+0000", 2024-09-14T06:29:15.996 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.5799999999999999e-06 2024-09-14T06:29:15.996 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:15.996 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:15.996 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:29:15.996 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.751604+0000", 2024-09-14T06:29:15.996 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.012508543 2024-09-14T06:29:15.996 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:15.996 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:15.997 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:29:15.997 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.751655+0000", 2024-09-14T06:29:15.997 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.0872999999999999e-05 2024-09-14T06:29:15.997 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:15.997 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:15.997 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:29:15.997 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.751707+0000", 2024-09-14T06:29:15.997 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.1869000000000001e-05 2024-09-14T06:29:15.997 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:15.997 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:29:15.997 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:15.997 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:15.997 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:15.998 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:3269 3.5 3:a6acff7e:::rbd_data.112d7dfbd766.0000000000000078:5 [read 323584~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-14T06:29:15.998 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:08.739101+0000", 2024-09-14T06:29:15.998 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 7.1990328290000001, 2024-09-14T06:29:15.998 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.012722587, 2024-09-14T06:29:15.998 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:29:15.998 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:29:15.998 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:29:15.998 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:29:15.998 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:29:15.998 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:15.998 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 3269 2024-09-14T06:29:15.998 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:15.998 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:29:15.998 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:15.999 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:29:15.999 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.739101+0000", 2024-09-14T06:29:15.999 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:15.999 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:15.999 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:15.999 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:29:15.999 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.739101+0000", 2024-09-14T06:29:15.999 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:15.999 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:15.999 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:15.999 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:29:15.999 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.739101+0000", 2024-09-14T06:29:15.999 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.17e-07 2024-09-14T06:29:15.999 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:15.999 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:16.000 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:29:16.000 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.739102+0000", 2024-09-14T06:29:16.000 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.6899999999999996e-07 2024-09-14T06:29:16.000 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:16.000 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:16.000 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:29:16.000 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.739106+0000", 2024-09-14T06:29:16.000 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.2390000000000004e-06 2024-09-14T06:29:16.000 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:16.000 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:16.000 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:29:16.000 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.739109+0000", 2024-09-14T06:29:16.000 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.5239999999999999e-06 2024-09-14T06:29:16.001 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:16.001 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:16.001 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:29:16.001 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.751732+0000", 2024-09-14T06:29:16.001 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.012622858000000001 2024-09-14T06:29:16.001 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:16.001 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:16.001 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:29:16.001 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.751774+0000", 2024-09-14T06:29:16.001 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.2485999999999999e-05 2024-09-14T06:29:16.001 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:16.001 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:16.001 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:29:16.001 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.751823+0000", 2024-09-14T06:29:16.002 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.9293999999999999e-05 2024-09-14T06:29:16.002 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:16.002 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:29:16.002 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:16.002 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:16.002 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:16.002 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:3290 3.1d 3:bf9aaf9c:::rbd_object_map.112dcdddac74.0000000000000006:head [call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-09-14T06:29:16.005 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:08.739264+0000", 2024-09-14T06:29:16.005 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 7.198869685, 2024-09-14T06:29:16.005 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.015385322, 2024-09-14T06:29:16.005 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:29:16.005 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:29:16.005 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:29:16.005 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:29:16.005 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:29:16.006 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:16.006 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 3290 2024-09-14T06:29:16.006 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:16.006 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:29:16.006 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:16.006 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:29:16.006 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.739264+0000", 2024-09-14T06:29:16.006 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:16.006 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:16.006 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:16.006 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:29:16.006 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.739264+0000", 2024-09-14T06:29:16.006 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:16.006 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:16.006 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:16.007 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:29:16.007 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.739265+0000", 2024-09-14T06:29:16.007 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.02e-07 2024-09-14T06:29:16.007 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:16.007 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:16.007 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:29:16.007 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.739269+0000", 2024-09-14T06:29:16.007 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.5990000000000004e-06 2024-09-14T06:29:16.007 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:16.007 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:16.007 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:29:16.007 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.739278+0000", 2024-09-14T06:29:16.008 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.8109999999999992e-06 2024-09-14T06:29:16.008 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:16.008 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:16.008 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:29:16.008 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.739285+0000", 2024-09-14T06:29:16.008 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.7159999999999997e-06 2024-09-14T06:29:16.008 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:16.008 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:16.008 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:29:16.008 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.751725+0000", 2024-09-14T06:29:16.008 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.012439597 2024-09-14T06:29:16.008 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:16.008 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:16.009 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:29:16.009 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.751771+0000", 2024-09-14T06:29:16.009 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.6353999999999997e-05 2024-09-14T06:29:16.009 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:16.009 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:16.009 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "waiting for subops from 4", 2024-09-14T06:29:16.009 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.751969+0000", 2024-09-14T06:29:16.009 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00019807700000000001 2024-09-14T06:29:16.009 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:16.009 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:16.009 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:29:16.009 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.753869+0000", 2024-09-14T06:29:16.009 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.001900335 2024-09-14T06:29:16.009 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:16.009 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:16.010 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "op_commit", 2024-09-14T06:29:16.010 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.754569+0000", 2024-09-14T06:29:16.010 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00069978200000000003 2024-09-14T06:29:16.010 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:16.010 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:16.010 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "commit_sent", 2024-09-14T06:29:16.010 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.754618+0000", 2024-09-14T06:29:16.010 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.9011000000000002e-05 2024-09-14T06:29:16.010 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:16.010 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:16.010 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:29:16.010 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.754649+0000", 2024-09-14T06:29:16.010 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.1137999999999999e-05 2024-09-14T06:29:16.010 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:16.011 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:29:16.011 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:16.011 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:16.011 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:16.011 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:3294 3.1d 3:bf9aaf9c:::rbd_object_map.112dcdddac74.0000000000000006:head [call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-09-14T06:29:16.011 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:08.740037+0000", 2024-09-14T06:29:16.011 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 7.1980965560000003, 2024-09-14T06:29:16.011 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.017846711000000001, 2024-09-14T06:29:16.011 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:29:16.011 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:29:16.011 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:29:16.011 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:29:16.012 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:29:16.012 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:16.012 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 3294 2024-09-14T06:29:16.012 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:16.012 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:29:16.012 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:16.012 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:29:16.012 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.740037+0000", 2024-09-14T06:29:16.012 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:16.013 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:16.013 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:16.013 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:29:16.013 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.740037+0000", 2024-09-14T06:29:16.013 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:16.013 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:16.013 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:16.013 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:29:16.013 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.740038+0000", 2024-09-14T06:29:16.013 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.1900000000000001e-07 2024-09-14T06:29:16.013 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:16.013 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:16.013 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:29:16.013 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.740043+0000", 2024-09-14T06:29:16.014 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.9649999999999999e-06 2024-09-14T06:29:16.014 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:16.014 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:16.014 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:29:16.014 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.740054+0000", 2024-09-14T06:29:16.014 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.1299e-05 2024-09-14T06:29:16.014 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:16.014 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:16.014 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:29:16.014 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.740061+0000", 2024-09-14T06:29:16.014 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.1250000000000004e-06 2024-09-14T06:29:16.014 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:16.014 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:16.014 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:29:16.014 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.752369+0000", 2024-09-14T06:29:16.015 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.012307603 2024-09-14T06:29:16.015 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:16.015 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:16.015 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:29:16.015 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.752453+0000", 2024-09-14T06:29:16.015 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.4557999999999999e-05 2024-09-14T06:29:16.015 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:16.015 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:16.015 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "waiting for subops from 4", 2024-09-14T06:29:16.015 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.752811+0000", 2024-09-14T06:29:16.015 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00035743 2024-09-14T06:29:16.015 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:16.015 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:16.015 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "op_commit", 2024-09-14T06:29:16.016 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.754655+0000", 2024-09-14T06:29:16.016 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00184362 2024-09-14T06:29:16.016 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:16.016 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:16.016 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:29:16.016 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.757784+0000", 2024-09-14T06:29:16.016 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.003129565 2024-09-14T06:29:16.016 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:16.016 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:16.016 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "commit_sent", 2024-09-14T06:29:16.016 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.757836+0000", 2024-09-14T06:29:16.016 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.1598000000000003e-05 2024-09-14T06:29:16.016 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:16.017 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:16.017 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:29:16.017 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.757884+0000", 2024-09-14T06:29:16.017 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.8028999999999997e-05 2024-09-14T06:29:16.017 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:16.017 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:29:16.017 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:16.017 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:16.017 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:16.017 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:3303 3.1d 3:bf9aaf9c:::rbd_object_map.112dcdddac74.0000000000000006:head [call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-09-14T06:29:16.017 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:08.743444+0000", 2024-09-14T06:29:16.017 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 7.1946891150000001, 2024-09-14T06:29:16.017 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.014637346000000001, 2024-09-14T06:29:16.017 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:29:16.018 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:29:16.018 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:29:16.018 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:29:16.018 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:29:16.018 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:16.018 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 3303 2024-09-14T06:29:16.018 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:16.018 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:29:16.018 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:16.018 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:29:16.018 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.743444+0000", 2024-09-14T06:29:16.018 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:16.018 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:16.018 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:16.019 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:29:16.019 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.743444+0000", 2024-09-14T06:29:16.019 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:16.019 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:16.019 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:16.019 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:29:16.019 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.743445+0000", 2024-09-14T06:29:16.019 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.540000000000001e-07 2024-09-14T06:29:16.019 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:16.019 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:16.019 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:29:16.019 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.743450+0000", 2024-09-14T06:29:16.019 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.9520000000000002e-06 2024-09-14T06:29:16.020 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:16.020 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:16.020 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:29:16.020 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.743462+0000", 2024-09-14T06:29:16.020 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.1674e-05 2024-09-14T06:29:16.020 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:16.020 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:16.020 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:29:16.020 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.743469+0000", 2024-09-14T06:29:16.020 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.4499999999999998e-06 2024-09-14T06:29:16.020 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:16.020 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:16.020 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:29:16.020 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.753195+0000", 2024-09-14T06:29:16.021 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.0097257260000000005 2024-09-14T06:29:16.021 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:16.021 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:16.021 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:29:16.021 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.753277+0000", 2024-09-14T06:29:16.021 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.1469000000000002e-05 2024-09-14T06:29:16.021 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:16.021 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:16.021 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "waiting for subops from 4", 2024-09-14T06:29:16.021 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.753482+0000", 2024-09-14T06:29:16.021 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00020495999999999999 2024-09-14T06:29:16.021 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:16.021 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:16.021 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:29:16.022 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.757930+0000", 2024-09-14T06:29:16.022 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.0044481640000000001 2024-09-14T06:29:16.022 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:16.022 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:16.022 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "op_commit", 2024-09-14T06:29:16.022 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.757992+0000", 2024-09-14T06:29:16.022 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.2648000000000001e-05 2024-09-14T06:29:16.022 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:16.022 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:16.022 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "commit_sent", 2024-09-14T06:29:16.022 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.758040+0000", 2024-09-14T06:29:16.022 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.7821000000000002e-05 2024-09-14T06:29:16.022 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:16.022 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:16.023 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:29:16.023 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.758082+0000", 2024-09-14T06:29:16.023 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.1527999999999999e-05 2024-09-14T06:29:16.023 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:16.023 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:29:16.023 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:16.023 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:16.023 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:16.023 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:3331 3.1d 3:bf9aaf9c:::rbd_object_map.112dcdddac74.0000000000000006:head [call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-09-14T06:29:16.023 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:08.745245+0000", 2024-09-14T06:29:16.023 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 7.1928888339999997, 2024-09-14T06:29:16.023 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.012916459999999999, 2024-09-14T06:29:16.023 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:29:16.023 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:29:16.023 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:29:16.023 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:29:16.023 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:29:16.024 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:16.024 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 3331 2024-09-14T06:29:16.024 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:16.024 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:29:16.024 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:16.024 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:29:16.024 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.745245+0000", 2024-09-14T06:29:16.024 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:16.024 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:16.024 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:16.024 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:29:16.024 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.745245+0000", 2024-09-14T06:29:16.024 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:16.024 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:16.024 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:16.024 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:29:16.025 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.745245+0000", 2024-09-14T06:29:16.025 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.5400000000000001e-07 2024-09-14T06:29:16.025 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:16.025 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:16.025 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:29:16.025 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.745250+0000", 2024-09-14T06:29:16.025 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.9230000000000001e-06 2024-09-14T06:29:16.025 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:16.025 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:16.025 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:29:16.025 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.745259+0000", 2024-09-14T06:29:16.025 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.5459999999999992e-06 2024-09-14T06:29:16.025 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:16.025 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:16.025 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:29:16.026 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.745264+0000", 2024-09-14T06:29:16.026 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.9609999999999996e-06 2024-09-14T06:29:16.026 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:16.026 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:16.026 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:29:16.026 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.753892+0000", 2024-09-14T06:29:16.026 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.0086279670000000003 2024-09-14T06:29:16.026 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:16.026 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:16.026 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:29:16.026 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.753948+0000", 2024-09-14T06:29:16.026 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.6036999999999999e-05 2024-09-14T06:29:16.026 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:16.026 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:16.026 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "waiting for subops from 4", 2024-09-14T06:29:16.027 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.754216+0000", 2024-09-14T06:29:16.027 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00026780199999999998 2024-09-14T06:29:16.027 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:16.027 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:16.027 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:29:16.027 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.757963+0000", 2024-09-14T06:29:16.027 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.0037473350000000001 2024-09-14T06:29:16.027 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:16.027 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:16.027 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "op_commit", 2024-09-14T06:29:16.027 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.758091+0000", 2024-09-14T06:29:16.027 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00012849000000000001 2024-09-14T06:29:16.027 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:16.027 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:16.027 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "commit_sent", 2024-09-14T06:29:16.027 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.758128+0000", 2024-09-14T06:29:16.027 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.6350999999999998e-05 2024-09-14T06:29:16.028 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:16.028 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:16.028 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:29:16.028 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.758161+0000", 2024-09-14T06:29:16.028 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.3494000000000002e-05 2024-09-14T06:29:16.028 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:16.028 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:29:16.028 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:16.028 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:16.028 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:16.028 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:3332 3.1d 3:bf9aaf9c:::rbd_object_map.112dcdddac74.0000000000000006:head [call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-09-14T06:29:16.028 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:08.746143+0000", 2024-09-14T06:29:16.028 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 7.1919907629999997, 2024-09-14T06:29:16.029 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.012083902000000001, 2024-09-14T06:29:16.029 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:29:16.029 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:29:16.029 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:29:16.029 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:29:16.029 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:29:16.029 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:16.029 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 3332 2024-09-14T06:29:16.029 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:16.029 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:29:16.029 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:16.029 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:29:16.029 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.746143+0000", 2024-09-14T06:29:16.030 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:16.030 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:16.030 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:16.030 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:29:16.030 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.746143+0000", 2024-09-14T06:29:16.030 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:16.030 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:16.030 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:16.030 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:29:16.030 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.746144+0000", 2024-09-14T06:29:16.030 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.5499999999999997e-07 2024-09-14T06:29:16.030 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:16.030 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:16.030 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:29:16.030 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.746149+0000", 2024-09-14T06:29:16.031 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.2039999999999996e-06 2024-09-14T06:29:16.031 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:16.031 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:16.031 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:29:16.031 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.746168+0000", 2024-09-14T06:29:16.031 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.8678999999999999e-05 2024-09-14T06:29:16.031 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:16.031 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:16.031 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:29:16.031 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.746176+0000", 2024-09-14T06:29:16.031 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.0439999999999996e-06 2024-09-14T06:29:16.031 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:16.031 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:16.031 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:29:16.031 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.754692+0000", 2024-09-14T06:29:16.032 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.0085166459999999992 2024-09-14T06:29:16.032 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:16.032 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:16.032 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:29:16.032 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.754746+0000", 2024-09-14T06:29:16.032 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.3375000000000003e-05 2024-09-14T06:29:16.032 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:16.032 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:16.032 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "waiting for subops from 4", 2024-09-14T06:29:16.032 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.754972+0000", 2024-09-14T06:29:16.032 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.000226586 2024-09-14T06:29:16.032 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:16.032 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:16.032 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:29:16.032 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.757985+0000", 2024-09-14T06:29:16.032 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.003012696 2024-09-14T06:29:16.033 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:16.033 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:16.033 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "op_commit", 2024-09-14T06:29:16.033 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.758171+0000", 2024-09-14T06:29:16.033 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00018605800000000001 2024-09-14T06:29:16.033 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:16.033 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:16.033 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "commit_sent", 2024-09-14T06:29:16.033 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.758195+0000", 2024-09-14T06:29:16.033 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.3591000000000002e-05 2024-09-14T06:29:16.033 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:16.033 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:16.033 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:29:16.033 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.758227+0000", 2024-09-14T06:29:16.033 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.2168000000000001e-05 2024-09-14T06:29:16.034 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:16.034 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:29:16.034 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:16.034 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:16.034 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:16.034 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:3334 3.1d 3:bf9aaf9c:::rbd_object_map.112dcdddac74.0000000000000006:head [call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-09-14T06:29:16.034 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:08.747923+0000", 2024-09-14T06:29:16.034 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 7.1902105890000003, 2024-09-14T06:29:16.034 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.012298584, 2024-09-14T06:29:16.034 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:29:16.034 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:29:16.034 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:29:16.034 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:29:16.034 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:29:16.034 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:16.034 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 3334 2024-09-14T06:29:16.035 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:16.035 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:29:16.035 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:16.035 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:29:16.035 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.747923+0000", 2024-09-14T06:29:16.035 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:16.035 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:16.035 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:16.035 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:29:16.035 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.747923+0000", 2024-09-14T06:29:16.035 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:16.035 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:16.035 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:16.035 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:29:16.035 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.747924+0000", 2024-09-14T06:29:16.035 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.2500000000000004e-07 2024-09-14T06:29:16.035 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:16.036 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:16.036 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:29:16.036 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.747929+0000", 2024-09-14T06:29:16.036 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.6909999999999996e-06 2024-09-14T06:29:16.036 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:16.036 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:16.036 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:29:16.036 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.747946+0000", 2024-09-14T06:29:16.036 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.7337e-05 2024-09-14T06:29:16.036 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:16.036 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:16.036 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:29:16.036 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.747954+0000", 2024-09-14T06:29:16.036 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.9959999999999993e-06 2024-09-14T06:29:16.036 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:16.037 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:16.037 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:29:16.037 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.756177+0000", 2024-09-14T06:29:16.037 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.0082233319999999999 2024-09-14T06:29:16.037 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:16.037 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:16.037 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:29:16.037 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.756266+0000", 2024-09-14T06:29:16.037 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.8252999999999997e-05 2024-09-14T06:29:16.037 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:16.037 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:16.037 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "waiting for subops from 4", 2024-09-14T06:29:16.037 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.756514+0000", 2024-09-14T06:29:16.037 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00024885599999999999 2024-09-14T06:29:16.037 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:16.038 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:16.038 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:29:16.038 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.759928+0000", 2024-09-14T06:29:16.038 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.0034139410000000002 2024-09-14T06:29:16.038 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:16.038 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:16.038 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "op_commit", 2024-09-14T06:29:16.038 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.760138+0000", 2024-09-14T06:29:16.038 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.000209545 2024-09-14T06:29:16.038 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:16.038 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:16.038 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "commit_sent", 2024-09-14T06:29:16.038 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.760180+0000", 2024-09-14T06:29:16.039 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.2212999999999997e-05 2024-09-14T06:29:16.039 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:16.039 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:16.039 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:29:16.039 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.760222+0000", 2024-09-14T06:29:16.039 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.1495000000000002e-05 2024-09-14T06:29:16.039 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:16.039 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:29:16.039 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:16.039 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:16.039 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:29:16.039 INFO:teuthology.orchestra.run.smithi138.stdout:} 2024-09-14T06:29:16.040 INFO:teuthology.orchestra.run.smithi094.stdout:osd.7: {} 2024-09-14T06:29:16.042 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-14T06:29:16.042 INFO:tasks.ceph.osd.6.smithi138.stderr:2024-09-14T06:29:16.039+0000 7f8cd8849640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33620) UID: 0 2024-09-14T06:29:16.042 INFO:teuthology.orchestra.run.smithi094.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-14T06:29:16.141 INFO:tasks.ceph.osd.3.smithi094.stderr:2024-09-14T06:29:16.140+0000 7feeeafe8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33655) UID: 0 2024-09-14T06:29:16.185 INFO:teuthology.orchestra.run.smithi138.stdout:{ 2024-09-14T06:29:16.185 INFO:teuthology.orchestra.run.smithi138.stdout: "ops": [], 2024-09-14T06:29:16.185 INFO:teuthology.orchestra.run.smithi138.stdout: "num_ops": 0 2024-09-14T06:29:16.185 INFO:teuthology.orchestra.run.smithi138.stdout:} 2024-09-14T06:29:16.194 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-14T06:29:16.242 INFO:tasks.ceph.osd.1.smithi094.stderr:2024-09-14T06:29:16.241+0000 7f328e47a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33641) UID: 0 2024-09-14T06:29:16.297 INFO:teuthology.orchestra.run.smithi138.stdout:{ 2024-09-14T06:29:16.297 INFO:teuthology.orchestra.run.smithi138.stdout: "ops": [], 2024-09-14T06:29:16.297 INFO:teuthology.orchestra.run.smithi138.stdout: "complaint_time": 30, 2024-09-14T06:29:16.297 INFO:teuthology.orchestra.run.smithi138.stdout: "num_blocked_ops": 0 2024-09-14T06:29:16.297 INFO:teuthology.orchestra.run.smithi138.stdout:} 2024-09-14T06:29:16.305 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-14T06:29:16.343 INFO:tasks.ceph.osd.3.smithi094.stderr:2024-09-14T06:29:16.342+0000 7feeeafe8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33655) UID: 0 2024-09-14T06:29:16.409 INFO:teuthology.orchestra.run.smithi138.stdout:{ 2024-09-14T06:29:16.410 INFO:teuthology.orchestra.run.smithi138.stdout: "size": 20, 2024-09-14T06:29:16.410 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 600, 2024-09-14T06:29:16.410 INFO:teuthology.orchestra.run.smithi138.stdout: "ops": [ 2024-09-14T06:29:16.410 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:16.410 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:1484 3.a 3:57c28b0f:::rbd_data.112d7dfbd766.000000000000000a:5 [read 208896~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-14T06:29:16.410 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:08.628246+0000", 2024-09-14T06:29:16.410 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 7.7797915159999995, 2024-09-14T06:29:16.410 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.018096672000000001, 2024-09-14T06:29:16.410 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:29:16.410 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:29:16.410 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:29:16.410 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:29:16.411 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:29:16.411 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:16.411 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 1484 2024-09-14T06:29:16.411 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:16.411 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:29:16.411 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:16.411 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:29:16.411 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.628246+0000", 2024-09-14T06:29:16.411 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:16.411 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:16.411 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:16.411 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:29:16.411 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.628246+0000", 2024-09-14T06:29:16.412 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:16.412 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:16.412 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:16.412 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:29:16.412 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.628246+0000", 2024-09-14T06:29:16.412 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.79e-07 2024-09-14T06:29:16.412 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:16.412 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:16.412 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:29:16.412 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.628247+0000", 2024-09-14T06:29:16.412 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.5179999999999999e-06 2024-09-14T06:29:16.412 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:16.412 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:16.412 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:29:16.412 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.628252+0000", 2024-09-14T06:29:16.413 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.0139999999999998e-06 2024-09-14T06:29:16.413 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:16.416 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:16.416 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:29:16.416 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.628257+0000", 2024-09-14T06:29:16.416 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.2429999999999999e-06 2024-09-14T06:29:16.417 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:16.417 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:16.417 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:29:16.417 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.646180+0000", 2024-09-14T06:29:16.417 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.017923136999999999 2024-09-14T06:29:16.417 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:16.417 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:16.417 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:29:16.417 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.646251+0000", 2024-09-14T06:29:16.417 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.1285000000000006e-05 2024-09-14T06:29:16.417 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:16.417 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:16.417 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:29:16.418 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.646342+0000", 2024-09-14T06:29:16.418 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.1195999999999999e-05 2024-09-14T06:29:16.418 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:16.418 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:29:16.418 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:16.418 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:16.418 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:16.418 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:1485 3.a 3:57c28b0f:::rbd_data.112d7dfbd766.000000000000000a:5 [read 225280~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-14T06:29:16.418 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:08.628268+0000", 2024-09-14T06:29:16.418 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 7.7797690350000002, 2024-09-14T06:29:16.418 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.018257005999999999, 2024-09-14T06:29:16.418 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:29:16.418 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:29:16.418 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:29:16.419 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:29:16.419 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:29:16.419 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:16.419 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 1485 2024-09-14T06:29:16.419 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:16.419 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:29:16.419 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:16.419 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:29:16.419 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.628268+0000", 2024-09-14T06:29:16.419 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:16.419 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:16.419 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:16.419 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:29:16.419 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.628268+0000", 2024-09-14T06:29:16.420 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:16.420 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:16.420 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:16.420 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:29:16.420 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.628269+0000", 2024-09-14T06:29:16.420 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.6800000000000001e-07 2024-09-14T06:29:16.420 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:16.420 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:16.420 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:29:16.420 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.628270+0000", 2024-09-14T06:29:16.420 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.8449999999999999e-06 2024-09-14T06:29:16.420 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:16.420 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:16.420 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:29:16.421 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.628276+0000", 2024-09-14T06:29:16.421 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.998e-06 2024-09-14T06:29:16.421 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:16.421 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:16.421 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:29:16.421 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.628281+0000", 2024-09-14T06:29:16.421 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.3120000000000004e-06 2024-09-14T06:29:16.421 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:16.421 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:16.421 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:29:16.421 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.646371+0000", 2024-09-14T06:29:16.421 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.018090694000000001 2024-09-14T06:29:16.421 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:16.421 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:16.422 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:29:16.422 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.646439+0000", 2024-09-14T06:29:16.422 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.7182999999999997e-05 2024-09-14T06:29:16.422 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:16.422 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:16.422 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:29:16.422 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.646525+0000", 2024-09-14T06:29:16.422 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.6506000000000002e-05 2024-09-14T06:29:16.422 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:16.422 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:29:16.422 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:16.422 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:16.422 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:16.422 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:1486 3.a 3:57c28b0f:::rbd_data.112d7dfbd766.000000000000000a:5 [read 241664~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-14T06:29:16.422 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:08.628288+0000", 2024-09-14T06:29:16.423 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 7.7797491489999997, 2024-09-14T06:29:16.423 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.018416895999999999, 2024-09-14T06:29:16.423 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:29:16.423 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:29:16.423 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:29:16.423 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:29:16.423 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:29:16.423 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:16.423 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 1486 2024-09-14T06:29:16.423 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:16.423 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:29:16.423 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:16.423 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:29:16.423 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.628288+0000", 2024-09-14T06:29:16.424 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:16.424 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:16.424 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:16.424 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:29:16.424 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.628288+0000", 2024-09-14T06:29:16.424 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:16.424 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:16.424 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:16.424 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:29:16.424 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.628288+0000", 2024-09-14T06:29:16.424 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.9499999999999998e-07 2024-09-14T06:29:16.424 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:16.424 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:16.424 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:29:16.425 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.628290+0000", 2024-09-14T06:29:16.425 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.6840000000000001e-06 2024-09-14T06:29:16.425 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:16.425 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:16.425 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:29:16.425 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.628295+0000", 2024-09-14T06:29:16.425 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.3059999999999997e-06 2024-09-14T06:29:16.425 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:16.425 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:16.425 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:29:16.425 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.628300+0000", 2024-09-14T06:29:16.425 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.2690000000000001e-06 2024-09-14T06:29:16.425 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:16.425 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:16.426 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:29:16.426 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.646553+0000", 2024-09-14T06:29:16.426 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.018253015000000001 2024-09-14T06:29:16.426 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:16.426 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:16.426 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:29:16.426 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.646626+0000", 2024-09-14T06:29:16.426 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.2939000000000001e-05 2024-09-14T06:29:16.426 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:16.426 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:16.426 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:29:16.426 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.646705+0000", 2024-09-14T06:29:16.426 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.9388000000000006e-05 2024-09-14T06:29:16.426 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:16.426 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:29:16.427 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:16.427 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:16.427 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:16.427 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:1487 3.a 3:57c28b0f:::rbd_data.112d7dfbd766.000000000000000a:5 [read 258048~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-14T06:29:16.427 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:08.628842+0000", 2024-09-14T06:29:16.427 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 7.7791947050000001, 2024-09-14T06:29:16.427 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.018037741999999999, 2024-09-14T06:29:16.427 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:29:16.427 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:29:16.427 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:29:16.427 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:29:16.427 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:29:16.427 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:16.428 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 1487 2024-09-14T06:29:16.428 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:16.428 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:29:16.428 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:16.428 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:29:16.428 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.628842+0000", 2024-09-14T06:29:16.428 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:16.428 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:16.428 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:16.428 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:29:16.428 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.628842+0000", 2024-09-14T06:29:16.428 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:16.428 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:16.429 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:16.429 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:29:16.429 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.628843+0000", 2024-09-14T06:29:16.429 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.5600000000000005e-07 2024-09-14T06:29:16.429 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:16.429 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:16.429 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:29:16.429 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.628847+0000", 2024-09-14T06:29:16.429 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.394e-06 2024-09-14T06:29:16.429 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:16.429 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:16.429 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:29:16.429 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.628855+0000", 2024-09-14T06:29:16.429 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.0760000000000003e-06 2024-09-14T06:29:16.430 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:16.430 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:16.430 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:29:16.430 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.628868+0000", 2024-09-14T06:29:16.430 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.2860000000000001e-05 2024-09-14T06:29:16.430 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:16.430 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:16.430 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:29:16.430 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.646733+0000", 2024-09-14T06:29:16.430 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.017865387999999999 2024-09-14T06:29:16.430 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:16.430 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:16.430 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:29:16.430 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.646800+0000", 2024-09-14T06:29:16.431 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.6619999999999996e-05 2024-09-14T06:29:16.431 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:16.431 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:16.431 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:29:16.431 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.646880+0000", 2024-09-14T06:29:16.431 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.0648000000000004e-05 2024-09-14T06:29:16.431 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:16.431 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:29:16.431 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:16.431 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:16.431 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:16.431 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:1488 3.a 3:57c28b0f:::rbd_data.112d7dfbd766.000000000000000a:5 [read 274432~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-14T06:29:16.431 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:08.628878+0000", 2024-09-14T06:29:16.431 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 7.779159087, 2024-09-14T06:29:16.432 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.018187624999999999, 2024-09-14T06:29:16.432 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:29:16.432 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:29:16.432 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:29:16.432 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:29:16.432 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:29:16.432 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:16.432 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 1488 2024-09-14T06:29:16.432 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:16.432 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:29:16.432 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:16.432 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:29:16.432 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.628878+0000", 2024-09-14T06:29:16.432 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:16.432 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:16.433 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:16.433 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:29:16.433 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.628878+0000", 2024-09-14T06:29:16.433 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:16.433 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:16.433 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:16.433 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:29:16.433 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.628879+0000", 2024-09-14T06:29:16.433 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.58e-07 2024-09-14T06:29:16.433 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:16.433 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:16.433 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:29:16.433 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.628881+0000", 2024-09-14T06:29:16.434 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.0389999999999999e-06 2024-09-14T06:29:16.434 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:16.434 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:16.434 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:29:16.434 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.628887+0000", 2024-09-14T06:29:16.434 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.9429999999999997e-06 2024-09-14T06:29:16.434 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:16.434 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:16.434 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:29:16.434 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.628891+0000", 2024-09-14T06:29:16.434 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.1309999999999999e-06 2024-09-14T06:29:16.434 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:16.434 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:16.435 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:29:16.435 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.646909+0000", 2024-09-14T06:29:16.435 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.018017920999999999 2024-09-14T06:29:16.435 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:16.435 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:16.435 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:29:16.435 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.646984+0000", 2024-09-14T06:29:16.435 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.4989999999999999e-05 2024-09-14T06:29:16.435 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:16.435 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:16.435 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:29:16.436 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.647066+0000", 2024-09-14T06:29:16.436 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.2143000000000003e-05 2024-09-14T06:29:16.436 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:16.436 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:29:16.436 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:16.436 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:16.436 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:16.436 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:1489 3.a 3:57c28b0f:::rbd_data.112d7dfbd766.000000000000000a:5 [read 290816~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-14T06:29:16.436 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:08.628897+0000", 2024-09-14T06:29:16.436 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 7.7791406360000002, 2024-09-14T06:29:16.436 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.018341491000000001, 2024-09-14T06:29:16.436 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:29:16.436 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:29:16.436 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:29:16.436 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:29:16.437 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:29:16.437 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:16.437 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 1489 2024-09-14T06:29:16.437 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:16.437 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:29:16.437 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:16.437 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:29:16.437 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.628897+0000", 2024-09-14T06:29:16.437 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:16.437 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:16.437 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:16.437 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:29:16.437 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.628897+0000", 2024-09-14T06:29:16.437 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:16.437 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:16.437 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:16.438 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:29:16.438 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.628897+0000", 2024-09-14T06:29:16.438 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.6800000000000001e-07 2024-09-14T06:29:16.438 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:16.438 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:16.438 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:29:16.438 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.628899+0000", 2024-09-14T06:29:16.438 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.606e-06 2024-09-14T06:29:16.438 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:16.438 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:16.438 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:29:16.438 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.628904+0000", 2024-09-14T06:29:16.438 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.1839999999999998e-06 2024-09-14T06:29:16.438 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:16.438 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:16.439 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:29:16.439 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.628908+0000", 2024-09-14T06:29:16.439 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.0500000000000002e-06 2024-09-14T06:29:16.439 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:16.439 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:16.439 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:29:16.439 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.647093+0000", 2024-09-14T06:29:16.439 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.018184846000000001 2024-09-14T06:29:16.439 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:16.439 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:16.439 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:29:16.439 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.647158+0000", 2024-09-14T06:29:16.439 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.5068999999999997e-05 2024-09-14T06:29:16.439 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:16.439 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:16.439 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:29:16.440 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.647238+0000", 2024-09-14T06:29:16.440 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.0368e-05 2024-09-14T06:29:16.440 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:16.440 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:29:16.440 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:16.440 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:16.440 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:16.440 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:1490 3.a 3:57c28b0f:::rbd_data.112d7dfbd766.000000000000000a:5 [read 307200~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-14T06:29:16.440 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:08.628921+0000", 2024-09-14T06:29:16.440 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 7.7791166389999997, 2024-09-14T06:29:16.440 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.018575338, 2024-09-14T06:29:16.440 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:29:16.440 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:29:16.440 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:29:16.440 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:29:16.441 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:29:16.441 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:16.441 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 1490 2024-09-14T06:29:16.441 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:16.441 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:29:16.441 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:16.441 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:29:16.441 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.628921+0000", 2024-09-14T06:29:16.441 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:16.441 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:16.441 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:16.441 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:29:16.441 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.628921+0000", 2024-09-14T06:29:16.442 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:16.442 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:16.442 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:16.442 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:29:16.442 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.628921+0000", 2024-09-14T06:29:16.442 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.39e-07 2024-09-14T06:29:16.442 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:16.442 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:16.442 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:29:16.442 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.628928+0000", 2024-09-14T06:29:16.442 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.81e-06 2024-09-14T06:29:16.442 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:16.442 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:16.442 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:29:16.443 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.628934+0000", 2024-09-14T06:29:16.443 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.4649999999999999e-06 2024-09-14T06:29:16.443 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:16.443 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:16.443 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:29:16.443 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.628939+0000", 2024-09-14T06:29:16.443 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.3730000000000003e-06 2024-09-14T06:29:16.443 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:16.443 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:16.443 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:29:16.443 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.647265+0000", 2024-09-14T06:29:16.443 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.018326499999999999 2024-09-14T06:29:16.443 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:16.443 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:16.444 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:29:16.444 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.647381+0000", 2024-09-14T06:29:16.444 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.000115549 2024-09-14T06:29:16.444 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:16.444 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:16.444 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:29:16.444 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.647496+0000", 2024-09-14T06:29:16.444 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.000115202 2024-09-14T06:29:16.444 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:16.444 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:29:16.444 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:16.444 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:16.444 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:16.445 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:1491 3.a 3:57c28b0f:::rbd_data.112d7dfbd766.000000000000000a:5 [read 323584~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-14T06:29:16.445 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:08.628964+0000", 2024-09-14T06:29:16.445 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 7.779073576, 2024-09-14T06:29:16.445 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.018753204999999998, 2024-09-14T06:29:16.445 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:29:16.445 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:29:16.445 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:29:16.445 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:29:16.445 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:29:16.445 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:16.446 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 1491 2024-09-14T06:29:16.446 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:16.446 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:29:16.446 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:16.446 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:29:16.446 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.628964+0000", 2024-09-14T06:29:16.446 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:16.446 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:16.446 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:16.446 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:29:16.446 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.628964+0000", 2024-09-14T06:29:16.446 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:16.446 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:16.446 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:16.446 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:29:16.447 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.628964+0000", 2024-09-14T06:29:16.447 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.3600000000000004e-07 2024-09-14T06:29:16.447 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:16.447 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:16.447 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:29:16.447 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.628966+0000", 2024-09-14T06:29:16.447 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.119e-06 2024-09-14T06:29:16.447 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:16.447 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:16.447 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:29:16.447 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.628972+0000", 2024-09-14T06:29:16.447 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.0929999999999999e-06 2024-09-14T06:29:16.447 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:16.447 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:16.447 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:29:16.448 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.628977+0000", 2024-09-14T06:29:16.448 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.5360000000000003e-06 2024-09-14T06:29:16.448 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:16.448 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:16.448 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:29:16.448 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.647527+0000", 2024-09-14T06:29:16.448 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.018550433000000002 2024-09-14T06:29:16.448 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:16.448 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:16.448 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:29:16.448 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.647614+0000", 2024-09-14T06:29:16.448 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.6757999999999998e-05 2024-09-14T06:29:16.448 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:16.448 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:16.448 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:29:16.449 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.647717+0000", 2024-09-14T06:29:16.449 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00010273 2024-09-14T06:29:16.449 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:16.449 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:29:16.449 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:16.449 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:16.449 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:16.449 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:1492 3.a 3:57c28b0f:::rbd_data.112d7dfbd766.000000000000000a:5 [read 339968~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-14T06:29:16.449 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:08.628983+0000", 2024-09-14T06:29:16.449 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 7.7790538930000004, 2024-09-14T06:29:16.449 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.018937253000000001, 2024-09-14T06:29:16.449 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:29:16.449 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:29:16.450 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:29:16.450 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:29:16.450 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:29:16.450 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:16.450 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 1492 2024-09-14T06:29:16.450 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:16.450 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:29:16.450 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:16.450 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:29:16.450 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.628983+0000", 2024-09-14T06:29:16.450 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:16.450 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:16.450 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:16.450 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:29:16.450 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.628983+0000", 2024-09-14T06:29:16.451 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:16.451 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:16.451 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:16.451 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:29:16.451 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.628984+0000", 2024-09-14T06:29:16.451 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.0100000000000001e-07 2024-09-14T06:29:16.451 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:16.451 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:16.451 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:29:16.451 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.628985+0000", 2024-09-14T06:29:16.451 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.561e-06 2024-09-14T06:29:16.451 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:16.451 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:16.451 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:29:16.452 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.628990+0000", 2024-09-14T06:29:16.452 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.1540000000000001e-06 2024-09-14T06:29:16.452 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:16.452 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:16.452 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:29:16.452 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.628994+0000", 2024-09-14T06:29:16.452 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.8890000000000004e-06 2024-09-14T06:29:16.452 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:16.452 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:16.452 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:29:16.452 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.647745+0000", 2024-09-14T06:29:16.452 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.018751213999999999 2024-09-14T06:29:16.452 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:16.452 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:16.453 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:29:16.453 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.647824+0000", 2024-09-14T06:29:16.453 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.8331999999999995e-05 2024-09-14T06:29:16.453 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:16.453 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:16.453 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:29:16.453 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.647921+0000", 2024-09-14T06:29:16.453 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.6801999999999999e-05 2024-09-14T06:29:16.453 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:16.453 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:29:16.453 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:16.453 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:16.453 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:16.453 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:1493 3.a 3:57c28b0f:::rbd_data.112d7dfbd766.000000000000000a:5 [read 356352~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-14T06:29:16.454 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:08.628999+0000", 2024-09-14T06:29:16.454 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 7.7790379779999999, 2024-09-14T06:29:16.454 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.019127128, 2024-09-14T06:29:16.454 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:29:16.454 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:29:16.454 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:29:16.454 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:29:16.454 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:29:16.454 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:16.454 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 1493 2024-09-14T06:29:16.454 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:16.454 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:29:16.454 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:16.454 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:29:16.455 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.628999+0000", 2024-09-14T06:29:16.455 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:16.455 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:16.455 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:16.455 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:29:16.455 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.628999+0000", 2024-09-14T06:29:16.455 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:16.455 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:16.455 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:16.455 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:29:16.455 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.629000+0000", 2024-09-14T06:29:16.455 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.2500000000000001e-07 2024-09-14T06:29:16.455 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:16.455 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:16.455 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:29:16.456 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.629001+0000", 2024-09-14T06:29:16.456 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.511e-06 2024-09-14T06:29:16.456 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:16.456 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:16.456 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:29:16.456 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.629006+0000", 2024-09-14T06:29:16.456 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.1420000000000001e-06 2024-09-14T06:29:16.456 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:16.456 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:16.456 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:29:16.456 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.629010+0000", 2024-09-14T06:29:16.456 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.8120000000000001e-06 2024-09-14T06:29:16.456 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:16.457 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:16.457 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:29:16.457 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.647950+0000", 2024-09-14T06:29:16.457 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.018939568 2024-09-14T06:29:16.457 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:16.457 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:16.457 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:29:16.457 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.648032+0000", 2024-09-14T06:29:16.457 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.2132000000000006e-05 2024-09-14T06:29:16.457 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:16.457 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:16.457 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:29:16.457 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.648126+0000", 2024-09-14T06:29:16.457 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.4637999999999998e-05 2024-09-14T06:29:16.458 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:16.458 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:29:16.458 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:16.458 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:16.458 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:16.458 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:1494 3.a 3:57c28b0f:::rbd_data.112d7dfbd766.000000000000000a:5 [read 372736~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-14T06:29:16.458 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:08.629017+0000", 2024-09-14T06:29:16.458 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 7.7790199170000003, 2024-09-14T06:29:16.458 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.019310414000000001, 2024-09-14T06:29:16.458 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:29:16.458 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:29:16.458 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:29:16.458 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:29:16.459 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:29:16.459 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:16.459 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 1494 2024-09-14T06:29:16.459 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:16.459 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:29:16.459 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:16.459 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:29:16.459 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.629017+0000", 2024-09-14T06:29:16.459 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:16.459 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:16.459 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:16.459 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:29:16.459 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.629017+0000", 2024-09-14T06:29:16.459 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:16.459 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:16.460 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:16.460 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:29:16.460 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.629018+0000", 2024-09-14T06:29:16.460 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.9000000000000002e-07 2024-09-14T06:29:16.460 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:16.460 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:16.460 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:29:16.460 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.629019+0000", 2024-09-14T06:29:16.460 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.5230000000000001e-06 2024-09-14T06:29:16.460 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:16.460 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:16.460 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:29:16.460 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.629024+0000", 2024-09-14T06:29:16.460 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.2299999999999999e-06 2024-09-14T06:29:16.461 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:16.461 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:16.461 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:29:16.461 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.629028+0000", 2024-09-14T06:29:16.461 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.7749999999999999e-06 2024-09-14T06:29:16.461 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:16.461 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:16.461 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:29:16.461 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.648156+0000", 2024-09-14T06:29:16.461 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.019128045999999999 2024-09-14T06:29:16.461 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:16.461 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:16.461 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:29:16.461 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.648238+0000", 2024-09-14T06:29:16.461 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.1877000000000002e-05 2024-09-14T06:29:16.462 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:16.462 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:16.462 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:29:16.462 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.648328+0000", 2024-09-14T06:29:16.462 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.9573000000000005e-05 2024-09-14T06:29:16.462 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:16.462 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:29:16.462 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:16.462 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:16.462 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:16.462 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:1495 3.a 3:57c28b0f:::rbd_data.112d7dfbd766.000000000000000a:5 [read 389120~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-14T06:29:16.462 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:08.629035+0000", 2024-09-14T06:29:16.462 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 7.7790018930000002, 2024-09-14T06:29:16.462 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.019486507, 2024-09-14T06:29:16.463 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:29:16.463 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:29:16.463 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:29:16.463 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:29:16.463 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:29:16.463 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:16.463 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 1495 2024-09-14T06:29:16.463 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:16.463 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:29:16.463 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:16.463 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:29:16.463 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.629035+0000", 2024-09-14T06:29:16.463 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:16.463 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:16.463 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:16.463 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:29:16.464 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.629035+0000", 2024-09-14T06:29:16.464 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:16.464 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:16.464 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:16.464 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:29:16.464 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.629036+0000", 2024-09-14T06:29:16.464 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.2000000000000001e-07 2024-09-14T06:29:16.464 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:16.464 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:16.464 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:29:16.464 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.629037+0000", 2024-09-14T06:29:16.464 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.4300000000000001e-06 2024-09-14T06:29:16.464 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:16.464 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:16.464 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:29:16.465 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.629043+0000", 2024-09-14T06:29:16.465 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.575e-06 2024-09-14T06:29:16.465 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:16.465 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:16.465 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:29:16.465 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.629047+0000", 2024-09-14T06:29:16.465 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.9179999999999996e-06 2024-09-14T06:29:16.465 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:16.465 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:16.465 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:29:16.465 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.648356+0000", 2024-09-14T06:29:16.465 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.019309862000000001 2024-09-14T06:29:16.465 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:16.465 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:16.465 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:29:16.466 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.648433+0000", 2024-09-14T06:29:16.466 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.6973999999999999e-05 2024-09-14T06:29:16.466 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:16.466 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:16.466 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:29:16.466 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.648522+0000", 2024-09-14T06:29:16.466 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.8428000000000001e-05 2024-09-14T06:29:16.466 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:16.466 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:29:16.466 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:16.466 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:16.466 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:16.466 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:1496 3.a 3:57c28b0f:::rbd_data.112d7dfbd766.000000000000000a:5 [read 405504~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-14T06:29:16.466 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:08.629086+0000", 2024-09-14T06:29:16.466 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 7.7789508490000001, 2024-09-14T06:29:16.467 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.019618813999999998, 2024-09-14T06:29:16.467 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:29:16.467 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:29:16.467 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:29:16.467 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:29:16.467 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:29:16.467 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:16.467 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 1496 2024-09-14T06:29:16.467 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:16.467 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:29:16.467 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:16.467 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:29:16.467 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.629086+0000", 2024-09-14T06:29:16.467 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:16.467 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:16.467 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:16.468 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:29:16.468 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.629086+0000", 2024-09-14T06:29:16.468 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:16.468 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:16.468 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:16.468 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:29:16.468 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.629087+0000", 2024-09-14T06:29:16.468 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.1399999999999996e-07 2024-09-14T06:29:16.468 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:16.468 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:16.468 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:29:16.468 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.629100+0000", 2024-09-14T06:29:16.468 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.2748999999999999e-05 2024-09-14T06:29:16.468 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:16.469 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:16.469 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:29:16.469 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.629111+0000", 2024-09-14T06:29:16.469 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.1531e-05 2024-09-14T06:29:16.469 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:16.469 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:16.469 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:29:16.469 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.629131+0000", 2024-09-14T06:29:16.469 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.9341999999999999e-05 2024-09-14T06:29:16.469 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:16.469 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:16.469 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:29:16.469 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.648550+0000", 2024-09-14T06:29:16.469 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.019418916000000001 2024-09-14T06:29:16.470 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:16.470 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:16.470 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:29:16.470 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.648616+0000", 2024-09-14T06:29:16.470 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.6359000000000006e-05 2024-09-14T06:29:16.470 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:16.470 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:16.470 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:29:16.470 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.648705+0000", 2024-09-14T06:29:16.470 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.9103000000000004e-05 2024-09-14T06:29:16.470 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:16.470 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:29:16.470 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:16.470 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:16.471 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:16.471 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:1497 3.a 3:57c28b0f:::rbd_data.112d7dfbd766.000000000000000a:5 [read 421888~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-14T06:29:16.471 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:08.629145+0000", 2024-09-14T06:29:16.471 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 7.7788920709999996, 2024-09-14T06:29:16.471 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.019744725000000001, 2024-09-14T06:29:16.471 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:29:16.471 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:29:16.471 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:29:16.471 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:29:16.471 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:29:16.471 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:16.471 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 1497 2024-09-14T06:29:16.471 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:16.471 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:29:16.472 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:16.472 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:29:16.472 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.629145+0000", 2024-09-14T06:29:16.472 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:16.472 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:16.472 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:16.472 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:29:16.472 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.629145+0000", 2024-09-14T06:29:16.472 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:16.472 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:16.472 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:16.472 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:29:16.472 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.629146+0000", 2024-09-14T06:29:16.472 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.1699999999999999e-07 2024-09-14T06:29:16.473 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:16.473 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:16.473 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:29:16.473 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.629148+0000", 2024-09-14T06:29:16.473 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.024e-06 2024-09-14T06:29:16.473 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:16.473 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:16.473 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:29:16.473 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.629152+0000", 2024-09-14T06:29:16.473 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.5970000000000002e-06 2024-09-14T06:29:16.473 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:16.473 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:16.473 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:29:16.473 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.629157+0000", 2024-09-14T06:29:16.474 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.8119999999999998e-06 2024-09-14T06:29:16.474 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:16.474 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:16.474 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:29:16.474 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.648734+0000", 2024-09-14T06:29:16.474 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.019576935 2024-09-14T06:29:16.474 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:16.474 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:16.474 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:29:16.474 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.648801+0000", 2024-09-14T06:29:16.474 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.7305999999999996e-05 2024-09-14T06:29:16.474 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:16.474 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:16.474 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:29:16.474 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.648890+0000", 2024-09-14T06:29:16.475 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.8634000000000006e-05 2024-09-14T06:29:16.475 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:16.475 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:29:16.475 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:16.475 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:16.475 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:16.475 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:1498 3.a 3:57c28b0f:::rbd_data.112d7dfbd766.000000000000000a:5 [read 438272~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-14T06:29:16.475 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:08.629167+0000", 2024-09-14T06:29:16.475 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 7.7788700070000001, 2024-09-14T06:29:16.475 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.019912138999999999, 2024-09-14T06:29:16.475 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:29:16.475 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:29:16.475 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:29:16.475 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:29:16.476 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:29:16.476 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:16.476 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 1498 2024-09-14T06:29:16.476 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:16.476 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:29:16.476 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:16.476 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:29:16.476 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.629167+0000", 2024-09-14T06:29:16.476 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:16.476 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:16.476 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:16.476 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:29:16.476 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.629167+0000", 2024-09-14T06:29:16.476 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:16.477 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:16.477 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:16.477 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:29:16.477 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.629168+0000", 2024-09-14T06:29:16.477 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.1300000000000001e-07 2024-09-14T06:29:16.477 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:16.477 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:16.477 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:29:16.477 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.629170+0000", 2024-09-14T06:29:16.477 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.7209999999999999e-06 2024-09-14T06:29:16.477 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:16.477 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:16.477 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:29:16.477 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.629174+0000", 2024-09-14T06:29:16.478 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.1459999999999998e-06 2024-09-14T06:29:16.478 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:16.478 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:16.478 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:29:16.478 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.629178+0000", 2024-09-14T06:29:16.478 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.8519999999999997e-06 2024-09-14T06:29:16.478 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:16.478 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:16.478 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:29:16.478 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.648924+0000", 2024-09-14T06:29:16.478 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.019746107999999998 2024-09-14T06:29:16.478 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:16.478 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:16.478 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:29:16.479 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.648995+0000", 2024-09-14T06:29:16.479 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.0809999999999995e-05 2024-09-14T06:29:16.479 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:16.479 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:16.479 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:29:16.479 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.649079+0000", 2024-09-14T06:29:16.479 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.4089000000000001e-05 2024-09-14T06:29:16.479 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:16.479 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:29:16.479 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:16.479 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:16.479 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:16.479 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:1499 3.a 3:57c28b0f:::rbd_data.112d7dfbd766.000000000000000a:5 [read 454656~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-14T06:29:16.479 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:08.629187+0000", 2024-09-14T06:29:16.479 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 7.7788500430000003, 2024-09-14T06:29:16.480 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.020072543000000002, 2024-09-14T06:29:16.480 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:29:16.480 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:29:16.480 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:29:16.480 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:29:16.480 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:29:16.480 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:16.480 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 1499 2024-09-14T06:29:16.480 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:16.480 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:29:16.480 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:16.480 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:29:16.480 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.629187+0000", 2024-09-14T06:29:16.480 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:16.481 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:16.481 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:16.481 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:29:16.481 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.629187+0000", 2024-09-14T06:29:16.481 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:16.481 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:16.481 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:16.481 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:29:16.481 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.629188+0000", 2024-09-14T06:29:16.481 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.89e-07 2024-09-14T06:29:16.481 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:16.481 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:16.481 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:29:16.481 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.629190+0000", 2024-09-14T06:29:16.482 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.0169999999999999e-06 2024-09-14T06:29:16.482 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:16.482 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:16.482 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:29:16.482 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.629195+0000", 2024-09-14T06:29:16.482 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.4149999999999998e-06 2024-09-14T06:29:16.482 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:16.482 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:16.482 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:29:16.482 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.629198+0000", 2024-09-14T06:29:16.482 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.605e-06 2024-09-14T06:29:16.482 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:16.482 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:16.482 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:29:16.482 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.649107+0000", 2024-09-14T06:29:16.482 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.019909339000000002 2024-09-14T06:29:16.483 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:16.483 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:16.483 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:29:16.483 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.649178+0000", 2024-09-14T06:29:16.483 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.0809999999999995e-05 2024-09-14T06:29:16.483 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:16.483 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:16.483 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:29:16.483 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.649260+0000", 2024-09-14T06:29:16.483 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.1967999999999999e-05 2024-09-14T06:29:16.483 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:16.483 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:29:16.483 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:16.483 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:16.483 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:16.484 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:1500 3.a 3:57c28b0f:::rbd_data.112d7dfbd766.000000000000000a:5 [read 471040~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-14T06:29:16.484 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:08.629204+0000", 2024-09-14T06:29:16.484 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 7.778833551, 2024-09-14T06:29:16.484 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.020473080000000001, 2024-09-14T06:29:16.484 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:29:16.484 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:29:16.484 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:29:16.484 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:29:16.484 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:29:16.484 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:16.484 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 1500 2024-09-14T06:29:16.484 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:16.484 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:29:16.484 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:16.484 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:29:16.485 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.629204+0000", 2024-09-14T06:29:16.485 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:16.485 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:16.485 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:16.485 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:29:16.485 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.629204+0000", 2024-09-14T06:29:16.485 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:16.485 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:16.485 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:16.485 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:29:16.485 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.629204+0000", 2024-09-14T06:29:16.485 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.34e-07 2024-09-14T06:29:16.485 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:16.485 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:16.486 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:29:16.486 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.629206+0000", 2024-09-14T06:29:16.486 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.939e-06 2024-09-14T06:29:16.486 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:16.486 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:16.486 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:29:16.486 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.629212+0000", 2024-09-14T06:29:16.486 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.8669999999999999e-06 2024-09-14T06:29:16.486 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:16.486 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:16.486 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:29:16.486 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.629216+0000", 2024-09-14T06:29:16.486 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.3830000000000002e-06 2024-09-14T06:29:16.487 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:16.487 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:16.487 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:29:16.487 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.649288+0000", 2024-09-14T06:29:16.487 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.020072113999999999 2024-09-14T06:29:16.487 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:16.487 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:16.487 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:29:16.487 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.649355+0000", 2024-09-14T06:29:16.487 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.7218000000000006e-05 2024-09-14T06:29:16.487 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:16.487 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:16.487 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:29:16.487 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.649677+0000", 2024-09-14T06:29:16.487 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00032122500000000002 2024-09-14T06:29:16.487 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:16.488 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:29:16.488 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:16.488 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:16.488 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:16.488 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:1501 3.a 3:57c28b0f:::rbd_data.112d7dfbd766.000000000000000a:5 [read 487424~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-14T06:29:16.488 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:08.629223+0000", 2024-09-14T06:29:16.488 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 7.7788141020000001, 2024-09-14T06:29:16.488 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.021370209000000001, 2024-09-14T06:29:16.488 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:29:16.488 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:29:16.488 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:29:16.488 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:29:16.488 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:29:16.488 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:16.488 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 1501 2024-09-14T06:29:16.489 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:16.489 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:29:16.489 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:16.489 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:29:16.489 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.629223+0000", 2024-09-14T06:29:16.489 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:16.489 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:16.489 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:16.489 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:29:16.489 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.629223+0000", 2024-09-14T06:29:16.489 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:16.489 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:16.489 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:16.489 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:29:16.490 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.629223+0000", 2024-09-14T06:29:16.490 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.9900000000000001e-07 2024-09-14T06:29:16.490 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:16.490 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:16.490 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:29:16.490 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.629231+0000", 2024-09-14T06:29:16.490 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.9179999999999994e-06 2024-09-14T06:29:16.490 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:16.490 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:16.490 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:29:16.490 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.629238+0000", 2024-09-14T06:29:16.490 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.421e-06 2024-09-14T06:29:16.490 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:16.490 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:16.490 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:29:16.490 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.629242+0000", 2024-09-14T06:29:16.491 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.5419999999999999e-06 2024-09-14T06:29:16.491 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:16.491 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:16.491 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:29:16.491 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.650284+0000", 2024-09-14T06:29:16.491 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.021041131000000001 2024-09-14T06:29:16.491 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:16.491 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:16.491 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:29:16.491 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.650361+0000", 2024-09-14T06:29:16.491 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.7787999999999995e-05 2024-09-14T06:29:16.491 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:16.491 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:16.491 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:29:16.491 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.650593+0000", 2024-09-14T06:29:16.491 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00023201 2024-09-14T06:29:16.492 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:16.492 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:29:16.492 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:16.492 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:16.492 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:16.492 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:1502 3.a 3:57c28b0f:::rbd_data.112d7dfbd766.000000000000000a:5 [read 503808~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-14T06:29:16.492 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:08.629249+0000", 2024-09-14T06:29:16.492 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 7.7787878560000001, 2024-09-14T06:29:16.492 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.021713217, 2024-09-14T06:29:16.492 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:29:16.492 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:29:16.492 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:29:16.492 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:29:16.492 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:29:16.493 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:16.493 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 1502 2024-09-14T06:29:16.493 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:16.493 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:29:16.493 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:16.493 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:29:16.493 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.629249+0000", 2024-09-14T06:29:16.493 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:16.493 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:16.493 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:16.493 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:29:16.493 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.629249+0000", 2024-09-14T06:29:16.493 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:16.493 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:16.494 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:16.494 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:29:16.494 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.629250+0000", 2024-09-14T06:29:16.494 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.03e-07 2024-09-14T06:29:16.494 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:16.494 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:16.494 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:29:16.494 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.629252+0000", 2024-09-14T06:29:16.494 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.995e-06 2024-09-14T06:29:16.494 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:16.494 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:16.495 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:29:16.495 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.629257+0000", 2024-09-14T06:29:16.495 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.7010000000000001e-06 2024-09-14T06:29:16.495 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:16.495 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:16.495 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:29:16.495 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.629262+0000", 2024-09-14T06:29:16.495 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.16e-06 2024-09-14T06:29:16.495 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:16.495 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:16.495 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:29:16.495 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.650629+0000", 2024-09-14T06:29:16.495 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.021367684000000001 2024-09-14T06:29:16.495 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:16.495 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:16.496 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:29:16.496 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.650718+0000", 2024-09-14T06:29:16.496 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.8832000000000006e-05 2024-09-14T06:29:16.496 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:16.496 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:16.496 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:29:16.496 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.650963+0000", 2024-09-14T06:29:16.496 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00024444199999999999 2024-09-14T06:29:16.496 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:16.496 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:29:16.496 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:16.496 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:16.496 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:16.496 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:1503 3.a 3:57c28b0f:::rbd_data.112d7dfbd766.000000000000000a:5 [read 520192~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-14T06:29:16.496 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:08.629268+0000", 2024-09-14T06:29:16.496 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 7.7787693149999999, 2024-09-14T06:29:16.497 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.022244885999999998, 2024-09-14T06:29:16.497 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:29:16.497 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:29:16.497 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:29:16.497 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:29:16.497 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:29:16.497 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:16.497 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 1503 2024-09-14T06:29:16.497 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:16.497 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:29:16.497 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:16.497 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:29:16.497 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.629268+0000", 2024-09-14T06:29:16.497 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:16.497 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:16.498 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:16.498 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:29:16.498 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.629268+0000", 2024-09-14T06:29:16.498 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:16.498 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:16.498 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:16.498 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:29:16.498 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.629268+0000", 2024-09-14T06:29:16.498 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.6599999999999996e-07 2024-09-14T06:29:16.498 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:16.498 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:16.498 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:29:16.498 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.629270+0000", 2024-09-14T06:29:16.498 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.7129999999999999e-06 2024-09-14T06:29:16.498 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:16.498 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:16.499 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:29:16.499 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.629276+0000", 2024-09-14T06:29:16.499 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.8119999999999996e-06 2024-09-14T06:29:16.499 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:16.499 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:16.499 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:29:16.499 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.629280+0000", 2024-09-14T06:29:16.499 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.4920000000000004e-06 2024-09-14T06:29:16.499 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:16.499 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:16.499 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:29:16.499 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.651039+0000", 2024-09-14T06:29:16.499 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.021758116000000001 2024-09-14T06:29:16.499 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:16.500 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:16.500 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:29:16.500 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.651127+0000", 2024-09-14T06:29:16.500 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.7983000000000001e-05 2024-09-14T06:29:16.500 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:16.500 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:16.500 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:29:16.500 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.651513+0000", 2024-09-14T06:29:16.500 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00038620400000000002 2024-09-14T06:29:16.500 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:16.500 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:29:16.500 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:16.500 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:16.500 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:29:16.500 INFO:teuthology.orchestra.run.smithi138.stdout:} 2024-09-14T06:29:16.502 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-14T06:29:16.502 INFO:tasks.ceph.osd.0.smithi094.stderr:2024-09-14T06:29:16.501+0000 7f07c9333640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33636) UID: 0 2024-09-14T06:29:16.603 INFO:tasks.ceph.osd.2.smithi094.stderr:2024-09-14T06:29:16.602+0000 7faf3269b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33649) UID: 0 2024-09-14T06:29:16.609 INFO:teuthology.orchestra.run.smithi138.stdout:{ 2024-09-14T06:29:16.609 INFO:teuthology.orchestra.run.smithi138.stdout: "ops": [], 2024-09-14T06:29:16.609 INFO:teuthology.orchestra.run.smithi138.stdout: "num_ops": 0 2024-09-14T06:29:16.609 INFO:teuthology.orchestra.run.smithi138.stdout:} 2024-09-14T06:29:16.618 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-14T06:29:16.704 INFO:tasks.ceph.osd.3.smithi094.stderr:2024-09-14T06:29:16.703+0000 7feeeafe8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33655) UID: 0 2024-09-14T06:29:16.723 INFO:teuthology.orchestra.run.smithi138.stdout:{ 2024-09-14T06:29:16.723 INFO:teuthology.orchestra.run.smithi138.stdout: "ops": [], 2024-09-14T06:29:16.724 INFO:teuthology.orchestra.run.smithi138.stdout: "complaint_time": 30, 2024-09-14T06:29:16.724 INFO:teuthology.orchestra.run.smithi138.stdout: "num_blocked_ops": 0 2024-09-14T06:29:16.724 INFO:teuthology.orchestra.run.smithi138.stdout:} 2024-09-14T06:29:16.729 INFO:teuthology.orchestra.run.smithi094.stdout:checking clone #2, image image_client.0-clone2 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent3 2024-09-14T06:29:16.734 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-14T06:29:16.804 INFO:tasks.ceph.osd.1.smithi094.stderr:2024-09-14T06:29:16.803+0000 7f328e47a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33641) UID: 0 2024-09-14T06:29:16.839 INFO:teuthology.orchestra.run.smithi138.stdout:{ 2024-09-14T06:29:16.840 INFO:teuthology.orchestra.run.smithi138.stdout: "size": 20, 2024-09-14T06:29:16.840 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 600, 2024-09-14T06:29:16.840 INFO:teuthology.orchestra.run.smithi138.stdout: "ops": [ 2024-09-14T06:29:16.840 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:16.840 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:1566 3.15 3:accf422f:::rbd_data.112d7dfbd766.0000000000000001:5 [read 192512~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-14T06:29:16.840 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:08.630416+0000", 2024-09-14T06:29:16.840 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 8.2075326559999997, 2024-09-14T06:29:16.840 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.021425026999999999, 2024-09-14T06:29:16.840 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:29:16.840 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:29:16.840 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:29:16.840 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:29:16.841 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:29:16.841 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:16.841 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 1566 2024-09-14T06:29:16.841 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:16.841 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:29:16.841 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:16.841 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:29:16.841 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.630416+0000", 2024-09-14T06:29:16.841 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:16.841 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:16.841 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:16.841 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:29:16.841 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.630416+0000", 2024-09-14T06:29:16.842 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:16.842 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:16.842 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:16.842 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:29:16.842 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.630417+0000", 2024-09-14T06:29:16.842 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.82e-07 2024-09-14T06:29:16.842 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:16.842 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:16.842 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:29:16.842 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.630419+0000", 2024-09-14T06:29:16.842 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.8059999999999999e-06 2024-09-14T06:29:16.842 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:16.842 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:16.842 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:29:16.842 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.630425+0000", 2024-09-14T06:29:16.843 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.3790000000000003e-06 2024-09-14T06:29:16.843 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:16.843 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:16.843 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:29:16.843 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.630431+0000", 2024-09-14T06:29:16.843 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.2570000000000004e-06 2024-09-14T06:29:16.843 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:16.843 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:16.843 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:29:16.843 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.651640+0000", 2024-09-14T06:29:16.843 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.021208320999999999 2024-09-14T06:29:16.843 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:16.843 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:16.844 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:29:16.844 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.651721+0000", 2024-09-14T06:29:16.844 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.1792000000000006e-05 2024-09-14T06:29:16.844 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:16.844 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:16.844 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:29:16.844 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.651841+0000", 2024-09-14T06:29:16.844 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00011989000000000001 2024-09-14T06:29:16.844 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:16.844 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:29:16.844 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:16.844 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:16.844 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:16.845 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:1567 3.15 3:accf422f:::rbd_data.112d7dfbd766.0000000000000001:5 [read 208896~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-14T06:29:16.845 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:08.630438+0000", 2024-09-14T06:29:16.845 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 8.2075112969999999, 2024-09-14T06:29:16.845 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.021638307999999998, 2024-09-14T06:29:16.845 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:29:16.845 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:29:16.845 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:29:16.845 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:29:16.845 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:29:16.845 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:16.845 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 1567 2024-09-14T06:29:16.845 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:16.845 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:29:16.845 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:16.846 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:29:16.846 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.630438+0000", 2024-09-14T06:29:16.846 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:16.846 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:16.846 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:16.846 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:29:16.846 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.630438+0000", 2024-09-14T06:29:16.846 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:16.846 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:16.846 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:16.846 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:29:16.846 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.630438+0000", 2024-09-14T06:29:16.846 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.5400000000000002e-07 2024-09-14T06:29:16.846 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:16.846 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:16.847 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:29:16.847 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.630440+0000", 2024-09-14T06:29:16.847 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.559e-06 2024-09-14T06:29:16.847 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:16.847 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:16.847 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:29:16.847 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.630446+0000", 2024-09-14T06:29:16.847 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.421e-06 2024-09-14T06:29:16.847 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:16.847 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:16.847 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:29:16.847 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.630451+0000", 2024-09-14T06:29:16.847 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.0849999999999996e-06 2024-09-14T06:29:16.848 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:16.848 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:16.848 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:29:16.848 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.651885+0000", 2024-09-14T06:29:16.848 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.021433773 2024-09-14T06:29:16.848 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:16.848 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:16.848 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:29:16.848 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.651976+0000", 2024-09-14T06:29:16.848 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.0897999999999996e-05 2024-09-14T06:29:16.848 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:16.848 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:16.848 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:29:16.848 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.652076+0000", 2024-09-14T06:29:16.849 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00010011800000000001 2024-09-14T06:29:16.849 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:16.849 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:29:16.849 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:16.849 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:16.849 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:16.849 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:1568 3.15 3:accf422f:::rbd_data.112d7dfbd766.0000000000000001:5 [read 225280~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-14T06:29:16.849 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:08.630458+0000", 2024-09-14T06:29:16.849 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 8.2074909480000002, 2024-09-14T06:29:16.849 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.021830307, 2024-09-14T06:29:16.849 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:29:16.849 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:29:16.850 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:29:16.850 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:29:16.850 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:29:16.850 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:16.850 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 1568 2024-09-14T06:29:16.850 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:16.850 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:29:16.850 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:16.850 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:29:16.850 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.630458+0000", 2024-09-14T06:29:16.850 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:16.850 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:16.850 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:16.850 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:29:16.851 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.630458+0000", 2024-09-14T06:29:16.851 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:16.851 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:16.851 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:16.851 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:29:16.851 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.630458+0000", 2024-09-14T06:29:16.851 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.7899999999999998e-07 2024-09-14T06:29:16.851 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:16.851 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:16.851 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:29:16.851 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.630465+0000", 2024-09-14T06:29:16.851 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.6109999999999999e-06 2024-09-14T06:29:16.851 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:16.852 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:16.852 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:29:16.852 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.630473+0000", 2024-09-14T06:29:16.852 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.2290000000000003e-06 2024-09-14T06:29:16.852 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:16.852 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:16.852 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:29:16.852 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.630479+0000", 2024-09-14T06:29:16.852 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.2249999999999999e-06 2024-09-14T06:29:16.852 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:16.852 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:16.852 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:29:16.852 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.652109+0000", 2024-09-14T06:29:16.852 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.021630527 2024-09-14T06:29:16.853 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:16.853 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:16.853 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:29:16.853 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.652187+0000", 2024-09-14T06:29:16.853 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.8300999999999995e-05 2024-09-14T06:29:16.853 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:16.853 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:16.853 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:29:16.853 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.652288+0000", 2024-09-14T06:29:16.853 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00010083500000000001 2024-09-14T06:29:16.853 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:16.853 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:29:16.853 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:16.854 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:16.854 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:16.854 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:1569 3.15 3:accf422f:::rbd_data.112d7dfbd766.0000000000000001:5 [read 241664~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-14T06:29:16.854 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:08.630493+0000", 2024-09-14T06:29:16.854 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 8.2074556940000001, 2024-09-14T06:29:16.854 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.022263345, 2024-09-14T06:29:16.854 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:29:16.854 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:29:16.854 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:29:16.854 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:29:16.854 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:29:16.854 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:16.854 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 1569 2024-09-14T06:29:16.854 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:16.854 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:29:16.855 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:16.855 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:29:16.855 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.630493+0000", 2024-09-14T06:29:16.855 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:16.855 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:16.855 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:16.855 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:29:16.855 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.630493+0000", 2024-09-14T06:29:16.855 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:16.855 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:16.855 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:16.855 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:29:16.855 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.630494+0000", 2024-09-14T06:29:16.856 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.0399999999999996e-07 2024-09-14T06:29:16.856 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:16.856 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:16.856 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:29:16.856 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.630496+0000", 2024-09-14T06:29:16.856 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.2749999999999998e-06 2024-09-14T06:29:16.856 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:16.856 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:16.856 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:29:16.856 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.630503+0000", 2024-09-14T06:29:16.856 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.1670000000000002e-06 2024-09-14T06:29:16.856 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:16.856 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:16.857 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:29:16.857 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.630509+0000", 2024-09-14T06:29:16.857 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.6300000000000003e-06 2024-09-14T06:29:16.857 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:16.857 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:16.857 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:29:16.857 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.652320+0000", 2024-09-14T06:29:16.857 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.021811047 2024-09-14T06:29:16.857 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:16.858 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:16.858 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:29:16.858 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.652397+0000", 2024-09-14T06:29:16.858 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.7347000000000003e-05 2024-09-14T06:29:16.858 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:16.858 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:16.858 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:29:16.858 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.652757+0000", 2024-09-14T06:29:16.858 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00035927499999999999 2024-09-14T06:29:16.858 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:16.858 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:29:16.858 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:16.859 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:16.859 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:16.859 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:1570 3.15 3:accf422f:::rbd_data.112d7dfbd766.0000000000000001:5 [read 258048~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-14T06:29:16.859 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:08.630517+0000", 2024-09-14T06:29:16.859 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 8.2074319370000008, 2024-09-14T06:29:16.859 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.022589267, 2024-09-14T06:29:16.859 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:29:16.859 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:29:16.859 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:29:16.859 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:29:16.859 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:29:16.859 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:16.859 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 1570 2024-09-14T06:29:16.860 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:16.860 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:29:16.860 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:16.860 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:29:16.860 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.630517+0000", 2024-09-14T06:29:16.860 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:16.860 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:16.860 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:16.860 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:29:16.860 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.630517+0000", 2024-09-14T06:29:16.860 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:16.860 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:16.860 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:16.860 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:29:16.861 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.630517+0000", 2024-09-14T06:29:16.861 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.4000000000000002e-07 2024-09-14T06:29:16.861 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:16.861 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:16.861 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:29:16.861 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.630524+0000", 2024-09-14T06:29:16.861 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.2829999999999998e-06 2024-09-14T06:29:16.861 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:16.861 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:16.861 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:29:16.861 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.630532+0000", 2024-09-14T06:29:16.861 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.9149999999999996e-06 2024-09-14T06:29:16.861 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:16.861 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:16.862 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:29:16.862 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.630537+0000", 2024-09-14T06:29:16.862 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.6500000000000001e-06 2024-09-14T06:29:16.862 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:16.862 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:16.862 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:29:16.862 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.652791+0000", 2024-09-14T06:29:16.862 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.022253586999999998 2024-09-14T06:29:16.862 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:16.862 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:16.862 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:29:16.862 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.652866+0000", 2024-09-14T06:29:16.862 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.5507999999999996e-05 2024-09-14T06:29:16.862 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:16.862 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:16.862 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:29:16.863 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.653106+0000", 2024-09-14T06:29:16.863 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.000239884 2024-09-14T06:29:16.863 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:16.863 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:29:16.863 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:16.863 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:16.863 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:16.863 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:1571 3.15 3:accf422f:::rbd_data.112d7dfbd766.0000000000000001:5 [read 274432~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-14T06:29:16.863 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:08.630544+0000", 2024-09-14T06:29:16.863 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 8.2074043860000003, 2024-09-14T06:29:16.863 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.023662969999999998, 2024-09-14T06:29:16.863 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:29:16.863 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:29:16.863 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:29:16.864 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:29:16.864 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:29:16.864 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:16.864 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 1571 2024-09-14T06:29:16.864 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:16.864 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:29:16.864 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:16.864 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:29:16.864 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.630544+0000", 2024-09-14T06:29:16.864 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:16.864 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:16.864 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:16.864 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:29:16.864 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.630544+0000", 2024-09-14T06:29:16.865 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:16.865 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:16.865 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:16.865 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:29:16.865 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.630545+0000", 2024-09-14T06:29:16.865 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.6100000000000001e-07 2024-09-14T06:29:16.865 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:16.865 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:16.865 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:29:16.865 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.630547+0000", 2024-09-14T06:29:16.865 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.7990000000000001e-06 2024-09-14T06:29:16.865 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:16.865 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:16.865 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:29:16.866 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.630554+0000", 2024-09-14T06:29:16.866 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.9110000000000004e-06 2024-09-14T06:29:16.866 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:16.866 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:16.866 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:29:16.866 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.630565+0000", 2024-09-14T06:29:16.866 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.1308000000000001e-05 2024-09-14T06:29:16.866 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:16.866 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:16.866 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:29:16.866 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.653226+0000", 2024-09-14T06:29:16.866 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.022661345999999999 2024-09-14T06:29:16.866 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:16.867 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:16.867 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:29:16.867 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.654105+0000", 2024-09-14T06:29:16.867 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00087874700000000004 2024-09-14T06:29:16.867 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:16.867 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:16.867 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:29:16.867 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.654207+0000", 2024-09-14T06:29:16.867 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.000102398 2024-09-14T06:29:16.867 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:16.867 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:29:16.867 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:16.867 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:16.867 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:16.867 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:1572 3.15 3:accf422f:::rbd_data.112d7dfbd766.0000000000000001:5 [read 290816~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-14T06:29:16.868 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:08.630573+0000", 2024-09-14T06:29:16.868 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 8.2073762489999993, 2024-09-14T06:29:16.868 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.023868055999999999, 2024-09-14T06:29:16.868 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:29:16.868 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:29:16.868 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:29:16.868 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:29:16.868 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:29:16.868 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:16.868 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 1572 2024-09-14T06:29:16.868 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:16.868 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:29:16.868 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:16.869 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:29:16.869 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.630573+0000", 2024-09-14T06:29:16.869 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:16.869 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:16.869 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:16.869 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:29:16.869 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.630573+0000", 2024-09-14T06:29:16.869 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:16.869 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:16.869 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:16.869 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:29:16.869 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.630573+0000", 2024-09-14T06:29:16.869 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.1300000000000001e-07 2024-09-14T06:29:16.870 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:16.870 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:16.870 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:29:16.870 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.630575+0000", 2024-09-14T06:29:16.870 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.9539999999999998e-06 2024-09-14T06:29:16.870 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:16.870 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:16.870 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:29:16.870 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.630583+0000", 2024-09-14T06:29:16.870 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.5619999999999998e-06 2024-09-14T06:29:16.870 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:16.870 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:16.870 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:29:16.870 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.630587+0000", 2024-09-14T06:29:16.871 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.8579999999999999e-06 2024-09-14T06:29:16.871 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:16.871 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:16.871 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:29:16.871 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.654254+0000", 2024-09-14T06:29:16.871 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.023666194000000002 2024-09-14T06:29:16.871 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:16.871 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:16.871 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:29:16.871 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.654341+0000", 2024-09-14T06:29:16.871 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.7713999999999994e-05 2024-09-14T06:29:16.871 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:16.871 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:16.872 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:29:16.872 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.654441+0000", 2024-09-14T06:29:16.872 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.9361e-05 2024-09-14T06:29:16.872 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:16.872 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:29:16.872 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:16.872 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:16.872 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:16.872 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:1573 3.15 3:accf422f:::rbd_data.112d7dfbd766.0000000000000001:5 [read 307200~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-14T06:29:16.872 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:08.630595+0000", 2024-09-14T06:29:16.872 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 8.2073542770000003, 2024-09-14T06:29:16.872 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.024070597999999999, 2024-09-14T06:29:16.872 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:29:16.873 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:29:16.873 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:29:16.873 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:29:16.873 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:29:16.873 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:16.873 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 1573 2024-09-14T06:29:16.873 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:16.873 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:29:16.873 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:16.873 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:29:16.873 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.630595+0000", 2024-09-14T06:29:16.873 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:16.874 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:16.874 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:16.874 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:29:16.874 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.630595+0000", 2024-09-14T06:29:16.874 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:16.874 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:16.874 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:16.874 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:29:16.874 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.630595+0000", 2024-09-14T06:29:16.874 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.7100000000000002e-07 2024-09-14T06:29:16.874 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:16.874 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:16.875 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:29:16.875 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.630597+0000", 2024-09-14T06:29:16.875 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.8300000000000001e-06 2024-09-14T06:29:16.875 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:16.875 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:16.875 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:29:16.875 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.630603+0000", 2024-09-14T06:29:16.875 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.2940000000000002e-06 2024-09-14T06:29:16.875 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:16.875 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:16.875 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:29:16.875 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.630613+0000", 2024-09-14T06:29:16.875 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.0023e-05 2024-09-14T06:29:16.875 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:16.875 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:16.876 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:29:16.876 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.654481+0000", 2024-09-14T06:29:16.876 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.023868034999999999 2024-09-14T06:29:16.876 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:16.876 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:16.876 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:29:16.876 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.654564+0000", 2024-09-14T06:29:16.876 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.2738000000000006e-05 2024-09-14T06:29:16.876 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:16.876 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:16.876 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:29:16.876 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.654665+0000", 2024-09-14T06:29:16.876 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00010110700000000001 2024-09-14T06:29:16.876 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:16.877 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:29:16.877 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:16.877 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:16.877 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:16.877 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:1574 3.15 3:accf422f:::rbd_data.112d7dfbd766.0000000000000001:5 [read 323584~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-14T06:29:16.877 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:08.630620+0000", 2024-09-14T06:29:16.877 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 8.2073284320000006, 2024-09-14T06:29:16.877 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.024280177, 2024-09-14T06:29:16.877 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:29:16.877 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:29:16.877 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:29:16.877 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:29:16.877 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:29:16.878 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:16.878 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 1574 2024-09-14T06:29:16.878 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:16.878 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:29:16.878 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:16.878 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:29:16.878 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.630620+0000", 2024-09-14T06:29:16.878 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:16.878 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:16.878 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:16.878 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:29:16.878 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.630620+0000", 2024-09-14T06:29:16.878 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:16.878 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:16.878 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:16.878 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:29:16.879 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.630621+0000", 2024-09-14T06:29:16.879 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.7000000000000005e-07 2024-09-14T06:29:16.879 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:16.879 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:16.879 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:29:16.879 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.630623+0000", 2024-09-14T06:29:16.879 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.9290000000000001e-06 2024-09-14T06:29:16.879 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:16.879 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:16.879 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:29:16.879 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.630630+0000", 2024-09-14T06:29:16.879 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.7059999999999998e-06 2024-09-14T06:29:16.879 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:16.880 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:16.880 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:29:16.880 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.630636+0000", 2024-09-14T06:29:16.880 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.9510000000000003e-06 2024-09-14T06:29:16.880 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:16.880 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:16.880 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:29:16.880 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.654709+0000", 2024-09-14T06:29:16.880 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.024073324 2024-09-14T06:29:16.880 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:16.880 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:16.880 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:29:16.880 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.654796+0000", 2024-09-14T06:29:16.880 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.6669999999999995e-05 2024-09-14T06:29:16.881 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:16.881 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:16.881 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:29:16.881 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.654901+0000", 2024-09-14T06:29:16.881 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.000105027 2024-09-14T06:29:16.881 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:16.881 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:29:16.881 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:16.881 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:16.881 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:16.881 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:1575 3.15 3:accf422f:::rbd_data.112d7dfbd766.0000000000000001:5 [read 339968~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-14T06:29:16.881 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:08.630645+0000", 2024-09-14T06:29:16.881 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 8.2073042019999995, 2024-09-14T06:29:16.881 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.024511650999999999, 2024-09-14T06:29:16.882 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:29:16.882 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:29:16.882 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:29:16.882 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:29:16.882 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:29:16.882 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:16.882 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 1575 2024-09-14T06:29:16.882 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:16.882 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:29:16.882 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:16.882 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:29:16.882 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.630645+0000", 2024-09-14T06:29:16.882 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:16.882 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:16.883 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:16.883 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:29:16.883 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.630645+0000", 2024-09-14T06:29:16.883 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:16.883 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:16.883 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:16.883 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:29:16.883 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.630645+0000", 2024-09-14T06:29:16.883 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.9400000000000005e-07 2024-09-14T06:29:16.883 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:16.883 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:16.883 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:29:16.883 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.630647+0000", 2024-09-14T06:29:16.884 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.9309999999999998e-06 2024-09-14T06:29:16.884 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:16.884 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:16.884 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:29:16.884 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.630654+0000", 2024-09-14T06:29:16.884 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.55e-06 2024-09-14T06:29:16.884 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:16.884 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:16.884 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:29:16.884 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.630659+0000", 2024-09-14T06:29:16.884 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.0139999999999998e-06 2024-09-14T06:29:16.884 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:16.884 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:16.884 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:29:16.885 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.654964+0000", 2024-09-14T06:29:16.885 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.024305651000000001 2024-09-14T06:29:16.885 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:16.885 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:16.885 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:29:16.885 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.655048+0000", 2024-09-14T06:29:16.885 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.3389999999999999e-05 2024-09-14T06:29:16.885 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:16.885 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:16.885 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:29:16.885 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.655156+0000", 2024-09-14T06:29:16.885 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.000108521 2024-09-14T06:29:16.885 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:16.885 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:29:16.886 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:16.886 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:16.886 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:16.886 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:1576 3.15 3:accf422f:::rbd_data.112d7dfbd766.0000000000000001:5 [read 356352~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-14T06:29:16.886 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:08.630666+0000", 2024-09-14T06:29:16.886 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 8.2072831639999997, 2024-09-14T06:29:16.886 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.024703712999999999, 2024-09-14T06:29:16.886 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:29:16.886 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:29:16.886 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:29:16.886 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:29:16.886 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:29:16.886 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:16.886 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 1576 2024-09-14T06:29:16.887 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:16.887 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:29:16.887 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:16.887 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:29:16.887 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.630666+0000", 2024-09-14T06:29:16.887 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:16.887 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:16.887 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:16.887 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:29:16.887 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.630666+0000", 2024-09-14T06:29:16.887 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:16.887 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:16.887 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:16.888 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:29:16.888 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.630666+0000", 2024-09-14T06:29:16.888 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.8299999999999997e-07 2024-09-14T06:29:16.888 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:16.888 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:16.888 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:29:16.888 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.630668+0000", 2024-09-14T06:29:16.888 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.8789999999999999e-06 2024-09-14T06:29:16.888 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:16.888 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:16.888 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:29:16.888 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.630675+0000", 2024-09-14T06:29:16.888 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.2899999999999997e-06 2024-09-14T06:29:16.888 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:16.889 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:16.889 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:29:16.889 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.630681+0000", 2024-09-14T06:29:16.889 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.2730000000000002e-06 2024-09-14T06:29:16.889 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:16.889 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:16.889 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:29:16.889 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.655182+0000", 2024-09-14T06:29:16.889 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.024501228999999999 2024-09-14T06:29:16.889 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:16.889 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:16.889 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:29:16.889 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.655257+0000", 2024-09-14T06:29:16.889 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.4954000000000001e-05 2024-09-14T06:29:16.890 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:16.890 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:16.890 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:29:16.890 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.655369+0000", 2024-09-14T06:29:16.890 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.000112605 2024-09-14T06:29:16.890 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:16.890 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:29:16.890 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:16.890 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:16.890 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:16.890 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:1577 3.15 3:accf422f:::rbd_data.112d7dfbd766.0000000000000001:5 [read 372736~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-14T06:29:16.890 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:08.630688+0000", 2024-09-14T06:29:16.890 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 8.2072606520000004, 2024-09-14T06:29:16.891 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.024927538999999999, 2024-09-14T06:29:16.891 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:29:16.891 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:29:16.891 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:29:16.891 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:29:16.891 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:29:16.891 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:16.891 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 1577 2024-09-14T06:29:16.891 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:16.891 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:29:16.891 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:16.891 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:29:16.891 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.630688+0000", 2024-09-14T06:29:16.891 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:16.892 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:16.892 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:16.892 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:29:16.892 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.630688+0000", 2024-09-14T06:29:16.892 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:16.892 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:16.892 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:16.892 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:29:16.892 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.630689+0000", 2024-09-14T06:29:16.892 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.2199999999999999e-07 2024-09-14T06:29:16.892 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:16.892 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:16.892 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:29:16.892 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.630691+0000", 2024-09-14T06:29:16.893 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.9410000000000001e-06 2024-09-14T06:29:16.893 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:16.893 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:16.893 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:29:16.893 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.630699+0000", 2024-09-14T06:29:16.893 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.0250000000000002e-06 2024-09-14T06:29:16.893 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:16.893 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:16.893 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:29:16.893 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.630704+0000", 2024-09-14T06:29:16.893 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.4349999999999996e-06 2024-09-14T06:29:16.893 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:16.893 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:16.894 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:29:16.894 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.655406+0000", 2024-09-14T06:29:16.894 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.024702279000000001 2024-09-14T06:29:16.894 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:16.894 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:16.894 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:29:16.894 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.655483+0000", 2024-09-14T06:29:16.894 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.6915999999999993e-05 2024-09-14T06:29:16.894 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:16.894 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:16.894 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:29:16.894 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.655616+0000", 2024-09-14T06:29:16.894 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.000132521 2024-09-14T06:29:16.894 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:16.894 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:29:16.895 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:16.895 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:16.895 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:16.895 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:1578 3.15 3:accf422f:::rbd_data.112d7dfbd766.0000000000000001:5 [read 389120~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-14T06:29:16.895 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:08.630711+0000", 2024-09-14T06:29:16.895 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 8.2072381799999992, 2024-09-14T06:29:16.895 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.025111108, 2024-09-14T06:29:16.895 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:29:16.895 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:29:16.895 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:29:16.895 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:29:16.895 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:29:16.895 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:16.895 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 1578 2024-09-14T06:29:16.896 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:16.896 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:29:16.896 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:16.896 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:29:16.896 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.630711+0000", 2024-09-14T06:29:16.896 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:16.896 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:16.896 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:16.896 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:29:16.896 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.630711+0000", 2024-09-14T06:29:16.896 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:16.896 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:16.896 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:16.896 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:29:16.897 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.630711+0000", 2024-09-14T06:29:16.897 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.01e-07 2024-09-14T06:29:16.897 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:16.897 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:16.897 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:29:16.897 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.630713+0000", 2024-09-14T06:29:16.897 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.8199999999999999e-06 2024-09-14T06:29:16.897 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:16.897 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:16.897 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:29:16.897 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.630720+0000", 2024-09-14T06:29:16.897 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.3880000000000002e-06 2024-09-14T06:29:16.897 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:16.898 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:16.898 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:29:16.898 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.630726+0000", 2024-09-14T06:29:16.898 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.3480000000000003e-06 2024-09-14T06:29:16.898 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:16.898 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:16.898 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:29:16.898 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.655646+0000", 2024-09-14T06:29:16.898 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.024920312999999999 2024-09-14T06:29:16.898 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:16.898 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:16.898 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:29:16.898 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.655724+0000", 2024-09-14T06:29:16.898 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.7589000000000005e-05 2024-09-14T06:29:16.899 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:16.899 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:16.899 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:29:16.899 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.655822+0000", 2024-09-14T06:29:16.899 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.8249000000000001e-05 2024-09-14T06:29:16.899 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:16.899 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:29:16.899 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:16.899 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:16.899 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:16.899 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:1579 3.15 3:accf422f:::rbd_data.112d7dfbd766.0000000000000001:5 [read 405504~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-14T06:29:16.899 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:08.630732+0000", 2024-09-14T06:29:16.899 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 8.2072168619999992, 2024-09-14T06:29:16.900 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.025303169, 2024-09-14T06:29:16.900 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:29:16.900 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:29:16.900 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:29:16.900 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:29:16.900 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:29:16.900 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:16.900 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 1579 2024-09-14T06:29:16.900 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:16.900 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:29:16.900 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:16.900 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:29:16.900 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.630732+0000", 2024-09-14T06:29:16.900 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:16.901 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:16.901 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:16.901 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:29:16.901 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.630732+0000", 2024-09-14T06:29:16.901 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:16.901 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:16.901 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:16.901 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:29:16.901 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.630732+0000", 2024-09-14T06:29:16.901 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.5199999999999998e-07 2024-09-14T06:29:16.901 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:16.901 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:16.901 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:29:16.902 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.630741+0000", 2024-09-14T06:29:16.902 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.1549999999999996e-06 2024-09-14T06:29:16.902 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:16.902 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:16.902 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:29:16.902 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.630748+0000", 2024-09-14T06:29:16.902 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.6429999999999998e-06 2024-09-14T06:29:16.902 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:16.902 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:16.902 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:29:16.902 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.630753+0000", 2024-09-14T06:29:16.902 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.0579999999999997e-06 2024-09-14T06:29:16.902 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:16.902 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:16.903 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:29:16.903 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.655858+0000", 2024-09-14T06:29:16.903 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.025104774999999999 2024-09-14T06:29:16.903 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:16.903 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:16.903 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:29:16.903 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.655942+0000", 2024-09-14T06:29:16.903 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.4129999999999996e-05 2024-09-14T06:29:16.903 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:16.903 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:16.903 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:29:16.903 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.656035+0000", 2024-09-14T06:29:16.903 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.3055999999999998e-05 2024-09-14T06:29:16.903 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:16.904 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:29:16.904 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:16.904 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:16.904 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:16.904 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:1580 3.15 3:accf422f:::rbd_data.112d7dfbd766.0000000000000001:5 [read 421888~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-14T06:29:16.904 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:08.630762+0000", 2024-09-14T06:29:16.904 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 8.2071870839999992, 2024-09-14T06:29:16.904 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.025487388999999999, 2024-09-14T06:29:16.904 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:29:16.904 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:29:16.904 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:29:16.904 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:29:16.904 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:29:16.904 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:16.905 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 1580 2024-09-14T06:29:16.905 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:16.905 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:29:16.905 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:16.905 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:29:16.905 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.630762+0000", 2024-09-14T06:29:16.905 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:16.905 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:16.905 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:16.905 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:29:16.905 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.630762+0000", 2024-09-14T06:29:16.905 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:16.906 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:16.906 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:16.906 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:29:16.906 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.630762+0000", 2024-09-14T06:29:16.906 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.2300000000000002e-07 2024-09-14T06:29:16.906 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:16.906 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:16.906 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:29:16.906 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.630764+0000", 2024-09-14T06:29:16.906 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.99e-06 2024-09-14T06:29:16.906 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:16.906 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:16.906 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:29:16.907 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.630771+0000", 2024-09-14T06:29:16.907 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.9349999999999997e-06 2024-09-14T06:29:16.907 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:16.907 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:16.907 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:29:16.907 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.630776+0000", 2024-09-14T06:29:16.907 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.1710000000000001e-06 2024-09-14T06:29:16.907 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:16.907 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:16.907 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:29:16.907 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.656068+0000", 2024-09-14T06:29:16.907 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.025291905999999999 2024-09-14T06:29:16.907 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:16.907 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:16.907 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:29:16.908 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.656151+0000", 2024-09-14T06:29:16.908 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.3209999999999998e-05 2024-09-14T06:29:16.908 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:16.908 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:16.908 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:29:16.908 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.656249+0000", 2024-09-14T06:29:16.908 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.7754e-05 2024-09-14T06:29:16.908 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:16.908 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:29:16.908 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:16.908 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:16.908 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:16.908 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:1581 3.15 3:accf422f:::rbd_data.112d7dfbd766.0000000000000001:5 [read 438272~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-14T06:29:16.909 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:08.630802+0000", 2024-09-14T06:29:16.909 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 8.2071464340000002, 2024-09-14T06:29:16.909 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.025670063, 2024-09-14T06:29:16.909 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:29:16.909 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:29:16.909 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:29:16.909 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:29:16.909 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:29:16.909 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:16.909 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 1581 2024-09-14T06:29:16.909 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:16.909 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:29:16.909 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:16.909 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:29:16.910 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.630802+0000", 2024-09-14T06:29:16.910 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:16.910 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:16.910 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:16.910 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:29:16.910 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.630802+0000", 2024-09-14T06:29:16.910 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:16.910 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:16.910 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:16.910 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:29:16.910 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.630803+0000", 2024-09-14T06:29:16.910 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.3099999999999997e-07 2024-09-14T06:29:16.910 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:16.910 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:16.910 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:29:16.911 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.630806+0000", 2024-09-14T06:29:16.911 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.7319999999999998e-06 2024-09-14T06:29:16.911 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:16.911 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:16.911 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:29:16.911 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.630813+0000", 2024-09-14T06:29:16.911 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.0840000000000003e-06 2024-09-14T06:29:16.911 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:16.911 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:16.911 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:29:16.911 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.630826+0000", 2024-09-14T06:29:16.911 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.3104e-05 2024-09-14T06:29:16.911 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:16.911 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:16.912 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:29:16.912 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.656349+0000", 2024-09-14T06:29:16.912 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.025523113 2024-09-14T06:29:16.912 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:16.912 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:16.912 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:29:16.912 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.656400+0000", 2024-09-14T06:29:16.912 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.1156000000000003e-05 2024-09-14T06:29:16.912 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:16.912 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:16.912 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:29:16.912 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.656472+0000", 2024-09-14T06:29:16.913 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.2243000000000006e-05 2024-09-14T06:29:16.913 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:16.913 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:29:16.913 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:16.913 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:16.913 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:16.913 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:1582 3.15 3:accf422f:::rbd_data.112d7dfbd766.0000000000000001:5 [read 454656~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-14T06:29:16.913 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:08.630838+0000", 2024-09-14T06:29:16.913 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 8.2071104100000003, 2024-09-14T06:29:16.913 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.025956396999999999, 2024-09-14T06:29:16.913 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:29:16.913 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:29:16.914 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:29:16.914 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:29:16.914 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:29:16.914 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:16.914 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 1582 2024-09-14T06:29:16.914 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:16.914 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:29:16.914 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:16.914 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:29:16.914 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.630838+0000", 2024-09-14T06:29:16.914 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:16.914 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:16.914 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:16.914 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:29:16.915 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.630838+0000", 2024-09-14T06:29:16.915 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:16.915 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:16.915 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:16.915 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:29:16.915 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.630839+0000", 2024-09-14T06:29:16.915 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.8400000000000004e-07 2024-09-14T06:29:16.915 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:16.915 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:16.915 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:29:16.915 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.630846+0000", 2024-09-14T06:29:16.915 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.4599999999999997e-06 2024-09-14T06:29:16.915 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:16.916 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:16.916 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:29:16.916 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.630854+0000", 2024-09-14T06:29:16.916 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.3139999999999998e-06 2024-09-14T06:29:16.916 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:16.916 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:16.916 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:29:16.916 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.630865+0000", 2024-09-14T06:29:16.916 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.0988999999999999e-05 2024-09-14T06:29:16.916 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:16.916 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:16.916 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:29:16.916 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.656598+0000", 2024-09-14T06:29:16.916 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.025733335 2024-09-14T06:29:16.917 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:16.917 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:16.917 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:29:16.917 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.656682+0000", 2024-09-14T06:29:16.917 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.4000999999999998e-05 2024-09-14T06:29:16.917 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:16.917 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:16.917 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:29:16.917 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.656795+0000", 2024-09-14T06:29:16.917 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00011271400000000001 2024-09-14T06:29:16.917 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:16.917 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:29:16.917 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:16.918 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:16.918 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:16.918 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:1583 3.15 3:accf422f:::rbd_data.112d7dfbd766.0000000000000001:5 [read 471040~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-14T06:29:16.918 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:08.630873+0000", 2024-09-14T06:29:16.918 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 8.2070755420000001, 2024-09-14T06:29:16.918 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.026337467, 2024-09-14T06:29:16.918 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:29:16.918 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:29:16.918 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:29:16.918 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:29:16.919 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:29:16.919 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:16.919 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 1583 2024-09-14T06:29:16.919 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:16.919 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:29:16.919 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:16.919 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:29:16.919 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.630873+0000", 2024-09-14T06:29:16.919 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:16.919 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:16.919 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:16.919 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:29:16.920 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.630873+0000", 2024-09-14T06:29:16.920 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:16.920 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:16.920 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:16.920 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:29:16.920 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.630874+0000", 2024-09-14T06:29:16.920 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.2500000000000005e-07 2024-09-14T06:29:16.920 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:16.920 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:16.920 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:29:16.920 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.630876+0000", 2024-09-14T06:29:16.920 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.2689999999999998e-06 2024-09-14T06:29:16.920 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:16.920 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:16.921 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:29:16.921 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.630883+0000", 2024-09-14T06:29:16.921 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.1919999999999999e-06 2024-09-14T06:29:16.921 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:16.921 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:16.921 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:29:16.921 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.630891+0000", 2024-09-14T06:29:16.921 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.1579999999999999e-06 2024-09-14T06:29:16.921 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:16.921 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:16.921 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:29:16.921 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.656836+0000", 2024-09-14T06:29:16.921 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.025945134000000002 2024-09-14T06:29:16.922 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:16.922 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:16.922 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:29:16.922 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.656920+0000", 2024-09-14T06:29:16.922 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.4351999999999995e-05 2024-09-14T06:29:16.922 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:16.922 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:16.922 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:29:16.922 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.657211+0000", 2024-09-14T06:29:16.922 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.000290737 2024-09-14T06:29:16.922 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:16.922 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:29:16.923 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:16.923 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:16.923 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:16.923 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:1584 3.15 3:accf422f:::rbd_data.112d7dfbd766.0000000000000001:5 [read 487424~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-14T06:29:16.923 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:08.630897+0000", 2024-09-14T06:29:16.923 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 8.2070514459999995, 2024-09-14T06:29:16.923 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.026720516, 2024-09-14T06:29:16.923 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:29:16.923 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:29:16.923 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:29:16.923 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:29:16.923 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:29:16.923 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:16.923 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 1584 2024-09-14T06:29:16.923 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:16.924 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:29:16.924 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:16.924 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:29:16.924 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.630897+0000", 2024-09-14T06:29:16.924 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:16.924 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:16.924 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:16.924 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:29:16.924 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.630897+0000", 2024-09-14T06:29:16.924 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:16.924 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:16.924 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:16.924 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:29:16.924 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.630898+0000", 2024-09-14T06:29:16.925 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.9200000000000002e-07 2024-09-14T06:29:16.925 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:16.925 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:16.925 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:29:16.925 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.630900+0000", 2024-09-14T06:29:16.925 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.7269999999999999e-06 2024-09-14T06:29:16.925 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:16.925 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:16.925 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:29:16.925 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.630906+0000", 2024-09-14T06:29:16.925 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.5849999999999996e-06 2024-09-14T06:29:16.925 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:16.925 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:16.926 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:29:16.926 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.630919+0000", 2024-09-14T06:29:16.926 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.3098999999999999e-05 2024-09-14T06:29:16.926 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:16.926 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:16.926 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:29:16.926 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.657253+0000", 2024-09-14T06:29:16.926 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.026333895999999999 2024-09-14T06:29:16.926 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:16.926 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:16.926 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:29:16.926 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.657342+0000", 2024-09-14T06:29:16.926 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.9044999999999999e-05 2024-09-14T06:29:16.927 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:16.927 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:16.927 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:29:16.927 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.657618+0000", 2024-09-14T06:29:16.927 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00027577199999999997 2024-09-14T06:29:16.927 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:16.927 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:29:16.927 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:16.927 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:16.927 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:16.927 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:1585 3.15 3:accf422f:::rbd_data.112d7dfbd766.0000000000000001:5 [read 503808~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-14T06:29:16.927 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:08.630927+0000", 2024-09-14T06:29:16.927 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 8.2070214440000004, 2024-09-14T06:29:16.927 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.027110801, 2024-09-14T06:29:16.928 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:29:16.928 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:29:16.928 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:29:16.928 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:29:16.928 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:29:16.928 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:16.928 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 1585 2024-09-14T06:29:16.928 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:16.928 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:29:16.928 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:16.928 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:29:16.928 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.630927+0000", 2024-09-14T06:29:16.928 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:16.928 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:16.929 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:16.929 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:29:16.929 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.630927+0000", 2024-09-14T06:29:16.929 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:16.929 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:16.929 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:16.929 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:29:16.929 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.630928+0000", 2024-09-14T06:29:16.929 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.7800000000000001e-07 2024-09-14T06:29:16.929 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:16.929 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:16.929 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:29:16.929 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.630930+0000", 2024-09-14T06:29:16.929 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.0829999999999998e-06 2024-09-14T06:29:16.930 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:16.930 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:16.930 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:29:16.930 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.630937+0000", 2024-09-14T06:29:16.930 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.7179999999999999e-06 2024-09-14T06:29:16.930 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:16.930 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:16.930 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:29:16.930 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.630942+0000", 2024-09-14T06:29:16.931 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.2989999999999996e-06 2024-09-14T06:29:16.931 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:16.931 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:16.931 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:29:16.931 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.657673+0000", 2024-09-14T06:29:16.931 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.026731390000000001 2024-09-14T06:29:16.931 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:16.931 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:16.931 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:29:16.931 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.657754+0000", 2024-09-14T06:29:16.931 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.0580000000000004e-05 2024-09-14T06:29:16.931 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:16.931 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:16.931 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:29:16.932 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.658038+0000", 2024-09-14T06:29:16.932 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00028415299999999999 2024-09-14T06:29:16.932 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:16.932 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:29:16.932 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:16.932 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:16.932 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:29:16.932 INFO:teuthology.orchestra.run.smithi138.stdout:} 2024-09-14T06:29:16.934 INFO:tasks.ceph.osd.6.smithi138.stderr:2024-09-14T06:29:16.932+0000 7f8cd8849640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33620) UID: 0 2024-09-14T06:29:16.934 INFO:teuthology.orchestra.run.smithi094.stdout:343 punch from 0x50742a4 to 0x507c31a, (0x8076 bytes) 2024-09-14T06:29:16.934 INFO:teuthology.orchestra.run.smithi094.stdout:344 writesame 0xae1fb2d thru 0xae294e1 data_size 0x13(0x99b5 bytes) 2024-09-14T06:29:16.934 INFO:teuthology.orchestra.run.smithi094.stdout:345 write 0xc9d7427 thru 0xc9de929 (0x7503 bytes) 2024-09-14T06:29:16.935 INFO:teuthology.orchestra.run.smithi094.stdout:348 trunc from 0xe024628 to 0x80037fc 2024-09-14T06:29:16.935 INFO:teuthology.orchestra.run.smithi094.stdout:349 write 0xca26860 thru 0xca30073 (0x9814 bytes) 2024-09-14T06:29:16.935 INFO:teuthology.orchestra.run.smithi094.stdout:350 read 0x291aad0 thru 0x29236a4 (0x8bd5 bytes) 2024-09-14T06:29:16.935 INFO:teuthology.orchestra.run.smithi094.stdout:352 write 0xdc35db8 thru 0xdc43a6a (0xdcb3 bytes) 2024-09-14T06:29:16.935 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-14T06:29:16.939 INFO:teuthology.orchestra.run.smithi094.stdout:353 trunc from 0xdc43a6b to 0x7bb8b5a 2024-09-14T06:29:16.951 INFO:teuthology.orchestra.run.smithi094.stdout:354 read 0x15f26a8 thru 0x1600dd6 (0xe72f bytes) 2024-09-14T06:29:16.951 INFO:teuthology.orchestra.run.smithi094.stdout:355 punch from 0xb87370 to 0xb90b3e, (0x97ce bytes) 2024-09-14T06:29:16.951 INFO:teuthology.orchestra.run.smithi094.stdout:356 writesame 0x45b6c75 thru 0x45c46e4 data_size 0xc(0xda70 bytes) 2024-09-14T06:29:16.955 INFO:teuthology.orchestra.run.smithi094.stdout:357 read 0x7181a79 thru 0x7182b2d (0x10b5 bytes) 2024-09-14T06:29:16.965 INFO:teuthology.orchestra.run.smithi094.stdout:358 write 0xa3b7084 thru 0xa3c152c (0xa4a9 bytes) 2024-09-14T06:29:16.969 INFO:teuthology.orchestra.run.smithi094.stdout:359 punch from 0x9f61f4 to 0x9fc3c4, (0x61d0 bytes) 2024-09-14T06:29:16.977 INFO:teuthology.orchestra.run.smithi094.stdout:360 write 0xbeddcba thru 0xbeea503 (0xc84a bytes) 2024-09-14T06:29:16.980 INFO:teuthology.orchestra.run.smithi094.stdout:361 read 0x86ac026 thru 0x86b04d0 (0x44ab bytes) 2024-09-14T06:29:16.980 INFO:teuthology.orchestra.run.smithi094.stdout:364 writesame 0xa42f57 thru 0xa45b2a data_size 0x55(0x2bd4 bytes) 2024-09-14T06:29:16.983 INFO:teuthology.orchestra.run.smithi094.stdout:365 read 0xf6146a thru 0xf6734c (0x5ee3 bytes) 2024-09-14T06:29:16.983 INFO:teuthology.orchestra.run.smithi094.stdout:366 write 0x7b76079 thru 0x7b7cf3c (0x6ec4 bytes) 2024-09-14T06:29:16.986 INFO:teuthology.orchestra.run.smithi094.stdout:367 write 0xa0d7945 thru 0xa0dc899 (0x4f55 bytes) 2024-09-14T06:29:16.989 INFO:teuthology.orchestra.run.smithi094.stdout:368 punch from 0x4b64dbb to 0x4b6628c, (0x14d1 bytes) 2024-09-14T06:29:16.989 INFO:teuthology.orchestra.run.smithi094.stdout:369 read 0x8eb6bd7 thru 0x8ec4f08 (0xe332 bytes) 2024-09-14T06:29:16.989 INFO:teuthology.orchestra.run.smithi094.stdout:371 write 0x3bdcb43 thru 0x3be63f0 (0x98ae bytes) 2024-09-14T06:29:16.992 INFO:teuthology.orchestra.run.smithi094.stdout:372 read 0x15dd6bf thru 0x15e3a93 (0x63d5 bytes) 2024-09-14T06:29:16.992 INFO:teuthology.orchestra.run.smithi094.stdout:373 read 0x9f43627 thru 0x9f4c6b4 (0x908e bytes) 2024-09-14T06:29:16.992 INFO:teuthology.orchestra.run.smithi094.stdout:374 read 0xb4f432b thru 0xb4fb426 (0x70fc bytes) 2024-09-14T06:29:16.992 INFO:teuthology.orchestra.run.smithi094.stdout:375 read 0x129973 thru 0x1395e6 (0xfc74 bytes) 2024-09-14T06:29:16.993 INFO:teuthology.orchestra.run.smithi094.stdout:377 punch from 0x2162575 to 0x2166a99, (0x4524 bytes) 2024-09-14T06:29:16.993 INFO:teuthology.orchestra.run.smithi094.stdout:378 read 0x1ec72f7 thru 0x1eca611 (0x331b bytes) 2024-09-14T06:29:16.994 INFO:teuthology.orchestra.run.smithi094.stdout:381 trunc from 0xbeea504 to 0xcd1cc85 2024-09-14T06:29:16.998 INFO:teuthology.orchestra.run.smithi094.stdout:382 read 0x2f2e416 thru 0x2f39bbb (0xb7a6 bytes) 2024-09-14T06:29:16.999 INFO:teuthology.orchestra.run.smithi094.stdout:384 write 0x916641e thru 0x916b07b (0x4c5e bytes) 2024-09-14T06:29:17.002 INFO:teuthology.orchestra.run.smithi094.stdout:385 write 0x5c5e267 thru 0x5c6b487 (0xd221 bytes) 2024-09-14T06:29:17.005 INFO:teuthology.orchestra.run.smithi094.stdout:386 write 0x889c0ba thru 0x88a3f92 (0x7ed9 bytes) 2024-09-14T06:29:17.007 INFO:teuthology.orchestra.run.smithi094.stdout:387 read 0x15cadeb thru 0x15d6e08 (0xc01e bytes) 2024-09-14T06:29:17.008 INFO:teuthology.orchestra.run.smithi094.stdout:388 write 0x1a8dc9f thru 0x1a94977 (0x6cd9 bytes) 2024-09-14T06:29:17.010 INFO:teuthology.orchestra.run.smithi094.stdout:389 trunc from 0xcd1cc85 to 0xce6f3e2 2024-09-14T06:29:17.014 INFO:teuthology.orchestra.run.smithi094.stdout:392 clone 5 order 23 su 131072 sc 15 2024-09-14T06:29:17.034 INFO:tasks.ceph.osd.2.smithi094.stderr:2024-09-14T06:29:17.033+0000 7faf3269b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33649) UID: 0 2024-09-14T06:29:17.043 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-14T06:29:17.044 INFO:teuthology.orchestra.run.smithi094.stdout: "ops": [], 2024-09-14T06:29:17.044 INFO:teuthology.orchestra.run.smithi094.stdout: "num_ops": 0 2024-09-14T06:29:17.044 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-14T06:29:17.052 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-14T06:29:17.135 INFO:tasks.ceph.osd.7.smithi138.stderr:2024-09-14T06:29:17.134+0000 7f3c6a21d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33623) UID: 0 2024-09-14T06:29:17.195 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-14T06:29:17.195 INFO:teuthology.orchestra.run.smithi094.stdout: "ops": [], 2024-09-14T06:29:17.195 INFO:teuthology.orchestra.run.smithi094.stdout: "complaint_time": 30, 2024-09-14T06:29:17.195 INFO:teuthology.orchestra.run.smithi094.stdout: "num_blocked_ops": 0 2024-09-14T06:29:17.195 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-14T06:29:17.203 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-14T06:29:17.247 INFO:tasks.ceph.osd.2.smithi094.stderr:2024-09-14T06:29:17.246+0000 7faf3269b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33649) UID: 0 2024-09-14T06:29:17.308 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-14T06:29:17.308 INFO:teuthology.orchestra.run.smithi094.stdout: "size": 20, 2024-09-14T06:29:17.308 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 600, 2024-09-14T06:29:17.308 INFO:teuthology.orchestra.run.smithi094.stdout: "ops": [ 2024-09-14T06:29:17.308 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:17.308 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:2175 3.6 3:637aae64:::rbd_data.112d7dfbd766.0000000000000032:5 [read 143360~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-14T06:29:17.309 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:08.676224+0000", 2024-09-14T06:29:17.309 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 8.6301443070000001, 2024-09-14T06:29:17.309 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.015113018000000001, 2024-09-14T06:29:17.309 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:29:17.309 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:29:17.309 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:29:17.309 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:29:17.309 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:29:17.309 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:17.309 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 2175 2024-09-14T06:29:17.309 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:17.310 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:29:17.310 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:17.310 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:29:17.310 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.676224+0000", 2024-09-14T06:29:17.310 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:17.310 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:17.310 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:17.310 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:29:17.310 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.676224+0000", 2024-09-14T06:29:17.310 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:17.310 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:17.310 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:17.310 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:29:17.311 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.676224+0000", 2024-09-14T06:29:17.311 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.6100000000000002e-07 2024-09-14T06:29:17.311 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:17.311 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:17.311 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:29:17.311 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.676226+0000", 2024-09-14T06:29:17.311 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.64e-06 2024-09-14T06:29:17.311 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:17.311 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:17.311 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:29:17.311 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.676232+0000", 2024-09-14T06:29:17.311 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.0820000000000004e-06 2024-09-14T06:29:17.311 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:17.311 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:17.312 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:29:17.312 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.676237+0000", 2024-09-14T06:29:17.312 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.7870000000000001e-06 2024-09-14T06:29:17.312 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:17.312 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:17.312 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:29:17.312 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.691140+0000", 2024-09-14T06:29:17.312 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.01490286 2024-09-14T06:29:17.312 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:17.312 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:17.312 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:29:17.313 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.691235+0000", 2024-09-14T06:29:17.313 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.5437000000000001e-05 2024-09-14T06:29:17.313 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:17.313 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:17.313 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:29:17.313 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.691337+0000", 2024-09-14T06:29:17.313 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000101851 2024-09-14T06:29:17.313 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:17.313 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:17.313 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:17.313 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:17.313 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:17.313 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:2176 3.6 3:637aae64:::rbd_data.112d7dfbd766.0000000000000032:5 [read 159744~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-14T06:29:17.313 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:08.676243+0000", 2024-09-14T06:29:17.314 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 8.6301254620000005, 2024-09-14T06:29:17.314 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.015261653999999999, 2024-09-14T06:29:17.314 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:29:17.314 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:29:17.314 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:29:17.314 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:29:17.314 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:29:17.314 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:17.314 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 2176 2024-09-14T06:29:17.314 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:17.314 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:29:17.314 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:17.314 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:29:17.314 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.676243+0000", 2024-09-14T06:29:17.315 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:17.315 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:17.315 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:17.315 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:29:17.315 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.676243+0000", 2024-09-14T06:29:17.315 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:17.315 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:17.315 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:17.315 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:29:17.315 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.676243+0000", 2024-09-14T06:29:17.315 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.8799999999999998e-07 2024-09-14T06:29:17.315 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:17.315 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:17.316 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:29:17.316 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.676245+0000", 2024-09-14T06:29:17.316 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.9470000000000002e-06 2024-09-14T06:29:17.316 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:17.316 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:17.316 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:29:17.316 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.676251+0000", 2024-09-14T06:29:17.316 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.1759999999999998e-06 2024-09-14T06:29:17.316 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:17.316 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:17.316 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:29:17.316 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.676256+0000", 2024-09-14T06:29:17.316 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.5260000000000004e-06 2024-09-14T06:29:17.316 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:17.317 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:17.317 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:29:17.317 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.691388+0000", 2024-09-14T06:29:17.317 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.015132563 2024-09-14T06:29:17.317 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:17.317 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:17.317 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:29:17.317 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.691432+0000", 2024-09-14T06:29:17.317 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.3197e-05 2024-09-14T06:29:17.317 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:17.317 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:17.317 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:29:17.317 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.691504+0000", 2024-09-14T06:29:17.317 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.2856999999999997e-05 2024-09-14T06:29:17.318 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:17.318 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:17.318 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:17.318 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:17.318 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:17.318 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:2177 3.6 3:637aae64:::rbd_data.112d7dfbd766.0000000000000032:5 [read 176128~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-14T06:29:17.318 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:08.676262+0000", 2024-09-14T06:29:17.318 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 8.6301063720000002, 2024-09-14T06:29:17.318 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.015436462, 2024-09-14T06:29:17.318 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:29:17.318 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:29:17.318 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:29:17.318 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:29:17.318 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:29:17.318 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:17.319 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 2177 2024-09-14T06:29:17.319 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:17.319 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:29:17.319 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:17.319 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:29:17.319 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.676262+0000", 2024-09-14T06:29:17.319 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:17.319 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:17.319 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:17.319 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:29:17.320 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.676262+0000", 2024-09-14T06:29:17.320 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:17.320 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:17.320 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:17.320 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:29:17.320 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.676262+0000", 2024-09-14T06:29:17.320 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.5600000000000001e-07 2024-09-14T06:29:17.320 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:17.320 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:17.320 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:29:17.320 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.676264+0000", 2024-09-14T06:29:17.320 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.8929999999999999e-06 2024-09-14T06:29:17.320 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:17.321 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:17.321 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:29:17.321 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.676272+0000", 2024-09-14T06:29:17.321 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.3599999999999996e-06 2024-09-14T06:29:17.321 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:17.321 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:17.321 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:29:17.321 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.676277+0000", 2024-09-14T06:29:17.321 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.899e-06 2024-09-14T06:29:17.321 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:17.321 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:17.321 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:29:17.321 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.691533+0000", 2024-09-14T06:29:17.321 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.015255258000000001 2024-09-14T06:29:17.322 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:17.322 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:17.322 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:29:17.322 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.691600+0000", 2024-09-14T06:29:17.322 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.7361999999999996e-05 2024-09-14T06:29:17.322 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:17.322 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:17.322 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:29:17.322 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.691698+0000", 2024-09-14T06:29:17.322 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.8333999999999997e-05 2024-09-14T06:29:17.322 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:17.322 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:17.322 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:17.322 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:17.323 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:17.323 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:2178 3.6 3:637aae64:::rbd_data.112d7dfbd766.0000000000000032:5 [read 192512~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-14T06:29:17.323 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:08.676284+0000", 2024-09-14T06:29:17.323 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 8.6300846609999997, 2024-09-14T06:29:17.323 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.015609618, 2024-09-14T06:29:17.323 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:29:17.323 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:29:17.323 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:29:17.323 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:29:17.323 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:29:17.323 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:17.323 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 2178 2024-09-14T06:29:17.323 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:17.324 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:29:17.324 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:17.324 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:29:17.324 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.676284+0000", 2024-09-14T06:29:17.324 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:17.324 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:17.324 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:17.324 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:29:17.324 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.676284+0000", 2024-09-14T06:29:17.324 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:17.324 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:17.324 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:17.324 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:29:17.325 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.676284+0000", 2024-09-14T06:29:17.325 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.9000000000000002e-07 2024-09-14T06:29:17.325 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:17.325 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:17.325 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:29:17.325 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.676287+0000", 2024-09-14T06:29:17.325 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.0359999999999998e-06 2024-09-14T06:29:17.325 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:17.325 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:17.325 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:29:17.325 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.676293+0000", 2024-09-14T06:29:17.325 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.8470000000000001e-06 2024-09-14T06:29:17.325 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:17.325 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:17.326 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:29:17.326 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.676298+0000", 2024-09-14T06:29:17.326 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.3009999999999997e-06 2024-09-14T06:29:17.326 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:17.326 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:17.326 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:29:17.326 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.691730+0000", 2024-09-14T06:29:17.326 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.015431706 2024-09-14T06:29:17.326 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:17.326 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:17.326 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:29:17.326 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.691813+0000", 2024-09-14T06:29:17.326 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.2738999999999995e-05 2024-09-14T06:29:17.326 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:17.327 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:17.327 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:29:17.327 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.691893+0000", 2024-09-14T06:29:17.327 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.0599000000000005e-05 2024-09-14T06:29:17.327 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:17.327 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:17.327 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:17.327 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:17.327 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:17.327 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:2179 3.6 3:637aae64:::rbd_data.112d7dfbd766.0000000000000032:5 [read 208896~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-14T06:29:17.327 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:08.676324+0000", 2024-09-14T06:29:17.327 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 8.6300441360000004, 2024-09-14T06:29:17.328 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.015825689, 2024-09-14T06:29:17.328 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:29:17.328 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:29:17.328 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:29:17.328 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:29:17.328 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:29:17.328 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:17.328 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 2179 2024-09-14T06:29:17.328 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:17.328 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:29:17.328 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:17.328 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:29:17.328 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.676324+0000", 2024-09-14T06:29:17.328 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:17.329 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:17.329 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:17.329 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:29:17.329 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.676324+0000", 2024-09-14T06:29:17.329 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:17.329 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:17.329 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:17.329 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:29:17.329 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.676325+0000", 2024-09-14T06:29:17.329 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.2099999999999996e-07 2024-09-14T06:29:17.329 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:17.329 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:17.329 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:29:17.329 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.676327+0000", 2024-09-14T06:29:17.330 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.3309999999999998e-06 2024-09-14T06:29:17.330 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:17.330 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:17.330 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:29:17.330 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.676334+0000", 2024-09-14T06:29:17.330 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.7819999999999996e-06 2024-09-14T06:29:17.330 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:17.330 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:17.330 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:29:17.330 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.676339+0000", 2024-09-14T06:29:17.330 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.6760000000000004e-06 2024-09-14T06:29:17.330 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:17.330 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:17.330 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:29:17.331 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.691922+0000", 2024-09-14T06:29:17.331 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.015582554 2024-09-14T06:29:17.331 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:17.331 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:17.331 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:29:17.331 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.691988+0000", 2024-09-14T06:29:17.331 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.5568999999999995e-05 2024-09-14T06:29:17.331 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:17.331 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:17.331 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:29:17.331 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.692150+0000", 2024-09-14T06:29:17.331 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000162156 2024-09-14T06:29:17.331 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:17.331 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:17.332 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:17.332 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:17.332 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:17.332 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:2180 3.6 3:637aae64:::rbd_data.112d7dfbd766.0000000000000032:5 [read 225280~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-14T06:29:17.332 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:08.676346+0000", 2024-09-14T06:29:17.332 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 8.6300217840000002, 2024-09-14T06:29:17.332 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.016174806999999999, 2024-09-14T06:29:17.332 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:29:17.332 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:29:17.332 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:29:17.332 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:29:17.332 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:29:17.332 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:17.333 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 2180 2024-09-14T06:29:17.333 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:17.333 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:29:17.333 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:17.333 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:29:17.333 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.676346+0000", 2024-09-14T06:29:17.333 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:17.333 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:17.333 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:17.333 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:29:17.333 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.676346+0000", 2024-09-14T06:29:17.334 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:17.334 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:17.334 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:17.334 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:29:17.334 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.676347+0000", 2024-09-14T06:29:17.334 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.9700000000000002e-07 2024-09-14T06:29:17.334 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:17.334 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:17.334 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:29:17.335 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.676349+0000", 2024-09-14T06:29:17.335 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.142e-06 2024-09-14T06:29:17.335 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:17.335 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:17.335 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:29:17.335 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.676355+0000", 2024-09-14T06:29:17.335 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.1929999999999998e-06 2024-09-14T06:29:17.335 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:17.335 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:17.335 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:29:17.335 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.676362+0000", 2024-09-14T06:29:17.335 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.3089999999999998e-06 2024-09-14T06:29:17.335 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:17.335 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:17.336 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:29:17.336 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.692187+0000", 2024-09-14T06:29:17.336 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.015824530999999999 2024-09-14T06:29:17.336 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:17.336 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:17.336 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:29:17.336 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.692287+0000", 2024-09-14T06:29:17.336 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.9884999999999997e-05 2024-09-14T06:29:17.336 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:17.336 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:17.336 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:29:17.336 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.692521+0000", 2024-09-14T06:29:17.336 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00023435 2024-09-14T06:29:17.337 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:17.337 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:17.337 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:17.337 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:17.337 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:17.337 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:3743 3.1f 3:fa5dbf8a:::rbd_data.112d7dfbd766.000000000000018d:5 [read 303104~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-14T06:29:17.337 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:08.774918+0000", 2024-09-14T06:29:17.337 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 8.5314501870000008, 2024-09-14T06:29:17.337 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.015121839, 2024-09-14T06:29:17.337 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:29:17.337 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:29:17.337 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:29:17.338 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:29:17.338 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:29:17.338 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:17.338 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 3743 2024-09-14T06:29:17.338 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:17.338 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:29:17.338 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:17.338 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:29:17.338 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.774918+0000", 2024-09-14T06:29:17.338 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:17.338 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:17.338 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:17.339 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:29:17.339 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.774918+0000", 2024-09-14T06:29:17.339 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:17.339 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:17.339 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:17.339 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:29:17.339 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.774918+0000", 2024-09-14T06:29:17.339 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.11e-07 2024-09-14T06:29:17.339 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:17.339 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:17.339 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:29:17.339 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.774919+0000", 2024-09-14T06:29:17.339 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.7100000000000011e-07 2024-09-14T06:29:17.339 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:17.340 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:17.340 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:29:17.340 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.774923+0000", 2024-09-14T06:29:17.340 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.625e-06 2024-09-14T06:29:17.340 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:17.340 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:17.340 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:29:17.340 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.774925+0000", 2024-09-14T06:29:17.340 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.6220000000000001e-06 2024-09-14T06:29:17.340 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:17.340 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:17.340 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:29:17.340 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.789865+0000", 2024-09-14T06:29:17.341 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.014939233999999999 2024-09-14T06:29:17.341 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:17.341 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:17.341 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:29:17.341 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.789939+0000", 2024-09-14T06:29:17.341 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.3848000000000002e-05 2024-09-14T06:29:17.341 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:17.341 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:17.341 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:29:17.341 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.790040+0000", 2024-09-14T06:29:17.341 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000101328 2024-09-14T06:29:17.341 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:17.341 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:17.342 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:17.342 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:17.342 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:17.342 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:3744 3.1f 3:fa5dbf8a:::rbd_data.112d7dfbd766.000000000000018d:5 [read 319488~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-14T06:29:17.342 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:08.774929+0000", 2024-09-14T06:29:17.342 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 8.5314395950000002, 2024-09-14T06:29:17.342 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.015307502000000001, 2024-09-14T06:29:17.342 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:29:17.342 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:29:17.342 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:29:17.342 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:29:17.342 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:29:17.342 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:17.342 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 3744 2024-09-14T06:29:17.343 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:17.343 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:29:17.343 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:17.343 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:29:17.343 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.774929+0000", 2024-09-14T06:29:17.343 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:17.343 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:17.343 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:17.343 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:29:17.343 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.774929+0000", 2024-09-14T06:29:17.343 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:17.343 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:17.343 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:17.343 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:29:17.343 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.774929+0000", 2024-09-14T06:29:17.344 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.0800000000000001e-07 2024-09-14T06:29:17.344 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:17.344 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:17.344 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:29:17.344 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.774930+0000", 2024-09-14T06:29:17.344 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.482e-06 2024-09-14T06:29:17.344 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:17.344 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:17.344 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:29:17.344 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.774934+0000", 2024-09-14T06:29:17.344 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.8489999999999999e-06 2024-09-14T06:29:17.344 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:17.344 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:17.344 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:29:17.345 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.774937+0000", 2024-09-14T06:29:17.345 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.5799999999999999e-06 2024-09-14T06:29:17.345 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:17.345 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:17.345 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:29:17.345 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.790069+0000", 2024-09-14T06:29:17.345 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.015132546 2024-09-14T06:29:17.345 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:17.345 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:17.345 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:29:17.345 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.790149+0000", 2024-09-14T06:29:17.345 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.9374000000000003e-05 2024-09-14T06:29:17.345 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:17.345 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:17.346 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:29:17.346 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.790236+0000", 2024-09-14T06:29:17.346 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.7462999999999999e-05 2024-09-14T06:29:17.346 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:17.346 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:17.346 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:17.346 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:17.346 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:17.346 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:3745 3.1f 3:fa5dbf8a:::rbd_data.112d7dfbd766.000000000000018d:5 [read 335872~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-14T06:29:17.346 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:08.774940+0000", 2024-09-14T06:29:17.346 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 8.5314283660000001, 2024-09-14T06:29:17.346 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.015484926, 2024-09-14T06:29:17.346 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:29:17.346 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:29:17.346 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:29:17.347 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:29:17.347 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:29:17.347 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:17.347 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 3745 2024-09-14T06:29:17.347 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:17.347 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:29:17.347 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:17.347 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:29:17.347 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.774940+0000", 2024-09-14T06:29:17.347 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:17.347 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:17.347 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:17.347 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:29:17.348 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.774940+0000", 2024-09-14T06:29:17.348 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:17.348 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:17.348 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:17.348 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:29:17.348 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.774940+0000", 2024-09-14T06:29:17.348 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.11e-07 2024-09-14T06:29:17.348 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:17.348 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:17.348 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:29:17.348 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.774941+0000", 2024-09-14T06:29:17.348 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.0500000000000002e-07 2024-09-14T06:29:17.348 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:17.349 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:17.349 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:29:17.349 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.774945+0000", 2024-09-14T06:29:17.349 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.7400000000000002e-06 2024-09-14T06:29:17.349 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:17.349 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:17.349 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:29:17.349 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.774948+0000", 2024-09-14T06:29:17.349 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.0740000000000001e-06 2024-09-14T06:29:17.349 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:17.349 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:17.349 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:29:17.349 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.790264+0000", 2024-09-14T06:29:17.349 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.015316553 2024-09-14T06:29:17.349 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:17.350 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:17.350 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:29:17.350 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.790336+0000", 2024-09-14T06:29:17.350 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.1722000000000002e-05 2024-09-14T06:29:17.350 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:17.350 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:17.350 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:29:17.350 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.790425+0000", 2024-09-14T06:29:17.350 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.8721000000000007e-05 2024-09-14T06:29:17.350 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:17.350 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:17.350 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:17.350 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:17.350 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:17.351 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:3746 3.1f 3:fa5dbf8a:::rbd_data.112d7dfbd766.000000000000018d:5 [read 352256~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-14T06:29:17.351 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:08.774952+0000", 2024-09-14T06:29:17.351 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 8.5314159519999997, 2024-09-14T06:29:17.351 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.015662054000000002, 2024-09-14T06:29:17.351 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:29:17.351 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:29:17.351 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:29:17.351 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:29:17.351 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:29:17.351 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:17.351 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 3746 2024-09-14T06:29:17.351 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:17.351 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:29:17.351 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:17.351 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:29:17.352 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.774952+0000", 2024-09-14T06:29:17.352 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:17.352 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:17.352 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:17.352 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:29:17.352 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.774952+0000", 2024-09-14T06:29:17.352 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:17.352 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:17.352 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:17.352 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:29:17.352 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.774952+0000", 2024-09-14T06:29:17.352 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.1299999999999999e-07 2024-09-14T06:29:17.352 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:17.353 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:17.353 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:29:17.353 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.774953+0000", 2024-09-14T06:29:17.353 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.8100000000000001e-07 2024-09-14T06:29:17.353 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:17.353 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:17.353 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:29:17.353 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.774957+0000", 2024-09-14T06:29:17.353 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.8060000000000001e-06 2024-09-14T06:29:17.353 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:17.353 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:17.353 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:29:17.353 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.774960+0000", 2024-09-14T06:29:17.353 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.633e-06 2024-09-14T06:29:17.354 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:17.354 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:17.354 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:29:17.354 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.790454+0000", 2024-09-14T06:29:17.354 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.015494588 2024-09-14T06:29:17.354 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:17.354 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:17.354 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:29:17.354 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.790525+0000", 2024-09-14T06:29:17.354 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.0869000000000002e-05 2024-09-14T06:29:17.354 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:17.354 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:17.354 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:29:17.354 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.790614+0000", 2024-09-14T06:29:17.355 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.8963999999999997e-05 2024-09-14T06:29:17.355 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:17.355 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:17.355 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:17.355 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:17.355 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:17.355 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:3747 3.1f 3:fa5dbf8a:::rbd_data.112d7dfbd766.000000000000018d:5 [read 368640~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-14T06:29:17.355 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:08.774964+0000", 2024-09-14T06:29:17.355 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 8.5314039309999998, 2024-09-14T06:29:17.355 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.016009292000000001, 2024-09-14T06:29:17.355 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:29:17.355 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:29:17.355 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:29:17.355 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:29:17.355 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:29:17.356 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:17.356 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 3747 2024-09-14T06:29:17.356 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:17.356 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:29:17.356 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:17.356 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:29:17.356 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.774964+0000", 2024-09-14T06:29:17.356 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:17.356 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:17.356 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:17.356 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:29:17.356 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.774964+0000", 2024-09-14T06:29:17.356 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:17.357 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:17.357 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:17.357 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:29:17.357 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.774965+0000", 2024-09-14T06:29:17.357 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.11e-07 2024-09-14T06:29:17.357 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:17.357 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:17.357 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:29:17.357 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.774966+0000", 2024-09-14T06:29:17.357 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.1349999999999999e-06 2024-09-14T06:29:17.357 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:17.357 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:17.357 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:29:17.357 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.774969+0000", 2024-09-14T06:29:17.357 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.755e-06 2024-09-14T06:29:17.358 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:17.358 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:17.358 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:29:17.358 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.774972+0000", 2024-09-14T06:29:17.358 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.4420000000000001e-06 2024-09-14T06:29:17.358 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:17.358 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:17.358 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:29:17.358 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.790816+0000", 2024-09-14T06:29:17.358 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.015843768000000001 2024-09-14T06:29:17.358 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:17.358 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:17.358 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:29:17.359 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.790885+0000", 2024-09-14T06:29:17.359 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.9080999999999998e-05 2024-09-14T06:29:17.359 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:17.359 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:17.359 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:29:17.359 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.790974+0000", 2024-09-14T06:29:17.359 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.8900000000000006e-05 2024-09-14T06:29:17.359 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:17.359 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:17.359 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:17.359 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:17.359 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:17.359 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:3748 3.1f 3:fa5dbf8a:::rbd_data.112d7dfbd766.000000000000018d:5 [read 385024~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-14T06:29:17.359 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:08.774976+0000", 2024-09-14T06:29:17.360 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 8.5313922459999993, 2024-09-14T06:29:17.360 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.016156594999999999, 2024-09-14T06:29:17.360 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:29:17.360 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:29:17.360 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:29:17.360 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:29:17.360 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:29:17.360 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:17.360 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 3748 2024-09-14T06:29:17.360 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:17.360 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:29:17.360 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:17.360 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:29:17.361 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.774976+0000", 2024-09-14T06:29:17.361 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:17.361 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:17.361 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:17.361 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:29:17.361 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.774976+0000", 2024-09-14T06:29:17.361 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:17.361 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:17.361 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:17.361 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:29:17.361 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.774976+0000", 2024-09-14T06:29:17.361 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.1799999999999999e-07 2024-09-14T06:29:17.361 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:17.361 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:17.362 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:29:17.362 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.774977+0000", 2024-09-14T06:29:17.362 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.0279999999999999e-06 2024-09-14T06:29:17.362 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:17.362 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:17.362 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:29:17.362 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.774981+0000", 2024-09-14T06:29:17.362 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.7179999999999998e-06 2024-09-14T06:29:17.362 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:17.362 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:17.362 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:29:17.362 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.774984+0000", 2024-09-14T06:29:17.362 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.7530000000000002e-06 2024-09-14T06:29:17.363 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:17.363 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:17.363 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:29:17.363 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.791013+0000", 2024-09-14T06:29:17.363 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.016029024999999999 2024-09-14T06:29:17.363 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:17.363 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:17.363 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:29:17.363 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.791062+0000", 2024-09-14T06:29:17.363 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.8902e-05 2024-09-14T06:29:17.363 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:17.363 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:17.363 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:29:17.363 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.791133+0000", 2024-09-14T06:29:17.364 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.0951000000000006e-05 2024-09-14T06:29:17.364 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:17.364 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:17.364 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:17.364 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:17.364 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:17.364 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:3749 3.1f 3:fa5dbf8a:::rbd_data.112d7dfbd766.000000000000018d:5 [read 401408~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-14T06:29:17.364 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:08.774987+0000", 2024-09-14T06:29:17.364 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 8.5313814909999994, 2024-09-14T06:29:17.364 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.016330092000000001, 2024-09-14T06:29:17.364 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:29:17.364 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:29:17.364 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:29:17.364 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:29:17.364 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:29:17.365 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:17.365 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 3749 2024-09-14T06:29:17.365 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:17.365 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:29:17.365 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:17.365 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:29:17.365 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.774987+0000", 2024-09-14T06:29:17.365 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:17.365 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:17.365 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:17.365 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:29:17.365 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.774987+0000", 2024-09-14T06:29:17.365 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:17.365 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:17.366 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:17.366 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:29:17.366 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.774987+0000", 2024-09-14T06:29:17.366 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.0599999999999999e-07 2024-09-14T06:29:17.366 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:17.366 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:17.366 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:29:17.366 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.774988+0000", 2024-09-14T06:29:17.366 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.7900000000000007e-07 2024-09-14T06:29:17.366 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:17.366 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:17.366 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:29:17.366 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.774992+0000", 2024-09-14T06:29:17.367 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.6540000000000001e-06 2024-09-14T06:29:17.367 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:17.367 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:17.367 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:29:17.367 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.774994+0000", 2024-09-14T06:29:17.367 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.5950000000000001e-06 2024-09-14T06:29:17.367 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:17.367 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:17.367 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:29:17.367 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.791166+0000", 2024-09-14T06:29:17.367 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.016172083 2024-09-14T06:29:17.367 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:17.367 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:17.367 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:29:17.368 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.791221+0000", 2024-09-14T06:29:17.368 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.4843000000000003e-05 2024-09-14T06:29:17.368 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:17.368 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:17.368 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:29:17.368 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.791317+0000", 2024-09-14T06:29:17.368 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.5731999999999998e-05 2024-09-14T06:29:17.368 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:17.368 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:17.368 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:17.368 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:17.368 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:17.368 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:3750 3.1f 3:fa5dbf8a:::rbd_data.112d7dfbd766.000000000000018d:5 [read 417792~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-14T06:29:17.368 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:08.774997+0000", 2024-09-14T06:29:17.369 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 8.5313711320000003, 2024-09-14T06:29:17.369 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.016504642, 2024-09-14T06:29:17.369 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:29:17.369 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:29:17.369 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:29:17.369 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:29:17.369 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:29:17.369 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:17.369 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 3750 2024-09-14T06:29:17.369 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:17.369 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:29:17.369 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:17.369 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:29:17.369 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.774997+0000", 2024-09-14T06:29:17.370 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:17.370 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:17.370 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:17.370 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:29:17.370 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.774997+0000", 2024-09-14T06:29:17.370 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:17.370 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:17.370 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:17.370 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:29:17.370 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.774997+0000", 2024-09-14T06:29:17.370 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.1500000000000001e-07 2024-09-14T06:29:17.370 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:17.370 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:17.370 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:29:17.370 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.774998+0000", 2024-09-14T06:29:17.371 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.6599999999999994e-07 2024-09-14T06:29:17.371 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:17.371 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:17.371 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:29:17.371 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.775002+0000", 2024-09-14T06:29:17.371 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.659e-06 2024-09-14T06:29:17.371 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:17.371 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:17.371 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:29:17.371 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.775005+0000", 2024-09-14T06:29:17.371 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.6299999999999998e-06 2024-09-14T06:29:17.371 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:17.371 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:17.371 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:29:17.372 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.791363+0000", 2024-09-14T06:29:17.372 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.016358707 2024-09-14T06:29:17.372 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:17.372 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:17.372 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:29:17.372 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.791421+0000", 2024-09-14T06:29:17.372 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.7816999999999999e-05 2024-09-14T06:29:17.372 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:17.372 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:17.372 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:29:17.372 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.791502+0000", 2024-09-14T06:29:17.372 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.0648000000000004e-05 2024-09-14T06:29:17.372 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:17.372 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:17.373 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:17.373 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:17.373 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:17.373 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:3751 3.1f 3:fa5dbf8a:::rbd_data.112d7dfbd766.000000000000018d:5 [read 434176~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-14T06:29:17.373 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:08.775010+0000", 2024-09-14T06:29:17.373 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 8.5313583959999999, 2024-09-14T06:29:17.373 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.016656712000000001, 2024-09-14T06:29:17.373 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:29:17.373 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:29:17.373 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:29:17.373 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:29:17.373 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:29:17.373 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:17.373 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 3751 2024-09-14T06:29:17.374 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:17.374 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:29:17.374 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:17.374 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:29:17.374 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.775010+0000", 2024-09-14T06:29:17.374 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:17.374 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:17.374 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:17.374 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:29:17.374 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.775010+0000", 2024-09-14T06:29:17.374 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:17.374 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:17.374 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:17.375 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:29:17.375 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.775010+0000", 2024-09-14T06:29:17.375 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.0800000000000001e-07 2024-09-14T06:29:17.375 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:17.375 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:17.375 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:29:17.375 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.775012+0000", 2024-09-14T06:29:17.375 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.528e-06 2024-09-14T06:29:17.375 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:17.375 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:17.375 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:29:17.375 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.775015+0000", 2024-09-14T06:29:17.375 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.8709999999999999e-06 2024-09-14T06:29:17.375 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:17.376 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:17.376 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:29:17.376 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.775018+0000", 2024-09-14T06:29:17.376 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.57e-06 2024-09-14T06:29:17.376 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:17.376 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:17.376 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:29:17.376 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.791547+0000", 2024-09-14T06:29:17.376 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.016529419 2024-09-14T06:29:17.376 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:17.376 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:17.376 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:29:17.376 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.791594+0000", 2024-09-14T06:29:17.376 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.6159000000000003e-05 2024-09-14T06:29:17.377 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:17.377 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:17.377 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:29:17.377 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.791667+0000", 2024-09-14T06:29:17.377 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.2957e-05 2024-09-14T06:29:17.377 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:17.377 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:17.377 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:17.377 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:17.377 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:17.377 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:3752 3.1f 3:fa5dbf8a:::rbd_data.112d7dfbd766.000000000000018d:5 [read 450560~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-14T06:29:17.377 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:08.775021+0000", 2024-09-14T06:29:17.377 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 8.5313471019999998, 2024-09-14T06:29:17.377 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.016807249, 2024-09-14T06:29:17.378 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:29:17.378 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:29:17.378 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:29:17.378 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:29:17.378 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:29:17.378 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:17.378 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 3752 2024-09-14T06:29:17.378 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:17.378 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:29:17.378 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:17.378 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:29:17.378 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.775021+0000", 2024-09-14T06:29:17.378 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:17.378 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:17.379 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:17.379 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:29:17.379 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.775021+0000", 2024-09-14T06:29:17.379 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:17.379 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:17.379 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:17.379 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:29:17.379 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.775021+0000", 2024-09-14T06:29:17.379 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.11e-07 2024-09-14T06:29:17.379 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:17.379 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:17.379 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:29:17.379 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.775022+0000", 2024-09-14T06:29:17.379 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.64e-07 2024-09-14T06:29:17.380 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:17.380 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:17.380 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:29:17.380 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.775026+0000", 2024-09-14T06:29:17.380 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.7869999999999999e-06 2024-09-14T06:29:17.380 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:17.380 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:17.380 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:29:17.380 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.775029+0000", 2024-09-14T06:29:17.380 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.5330000000000001e-06 2024-09-14T06:29:17.380 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:17.380 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:17.380 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:29:17.381 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.791706+0000", 2024-09-14T06:29:17.381 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.016677208999999998 2024-09-14T06:29:17.381 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:17.381 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:17.381 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:29:17.381 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.791753+0000", 2024-09-14T06:29:17.381 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.7655999999999999e-05 2024-09-14T06:29:17.381 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:17.381 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:17.381 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:29:17.381 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.791828+0000", 2024-09-14T06:29:17.381 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.4888999999999994e-05 2024-09-14T06:29:17.381 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:17.382 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:17.382 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:17.382 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:17.382 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:17.382 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:3753 3.1f 3:fa5dbf8a:::rbd_data.112d7dfbd766.000000000000018d:5 [read 466944~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-14T06:29:17.382 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:08.775032+0000", 2024-09-14T06:29:17.382 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 8.5313365270000006, 2024-09-14T06:29:17.382 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.016938284000000001, 2024-09-14T06:29:17.382 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:29:17.382 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:29:17.382 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:29:17.382 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:29:17.382 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:29:17.382 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:17.383 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 3753 2024-09-14T06:29:17.383 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:17.383 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:29:17.383 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:17.383 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:29:17.383 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.775032+0000", 2024-09-14T06:29:17.383 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:17.383 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:17.383 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:17.383 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:29:17.383 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.775032+0000", 2024-09-14T06:29:17.383 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:17.383 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:17.384 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:17.384 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:29:17.384 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.775032+0000", 2024-09-14T06:29:17.384 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.0699999999999999e-07 2024-09-14T06:29:17.384 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:17.384 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:17.384 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:29:17.384 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.775033+0000", 2024-09-14T06:29:17.384 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.47e-07 2024-09-14T06:29:17.384 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:17.384 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:17.384 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:29:17.384 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.775036+0000", 2024-09-14T06:29:17.385 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.6349999999999999e-06 2024-09-14T06:29:17.385 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:17.385 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:17.385 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:29:17.385 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.775039+0000", 2024-09-14T06:29:17.385 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.571e-06 2024-09-14T06:29:17.385 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:17.385 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:17.385 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:29:17.385 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.791863+0000", 2024-09-14T06:29:17.385 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.016824163999999999 2024-09-14T06:29:17.385 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:17.385 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:17.385 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:29:17.386 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.791907+0000", 2024-09-14T06:29:17.386 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.3538000000000002e-05 2024-09-14T06:29:17.386 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:17.386 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:17.386 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:29:17.386 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.791970+0000", 2024-09-14T06:29:17.386 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.3221999999999999e-05 2024-09-14T06:29:17.386 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:17.386 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:17.386 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:17.386 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:17.386 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:17.386 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:3754 3.1f 3:fa5dbf8a:::rbd_data.112d7dfbd766.000000000000018d:5 [read 483328~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-14T06:29:17.386 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:08.775042+0000", 2024-09-14T06:29:17.386 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 8.5313261400000009, 2024-09-14T06:29:17.387 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.017038721, 2024-09-14T06:29:17.387 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:29:17.387 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:29:17.387 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:29:17.387 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:29:17.387 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:29:17.387 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:17.387 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 3754 2024-09-14T06:29:17.387 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:17.387 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:29:17.387 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:17.387 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:29:17.387 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.775042+0000", 2024-09-14T06:29:17.387 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:17.388 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:17.388 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:17.388 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:29:17.388 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.775042+0000", 2024-09-14T06:29:17.388 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:17.388 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:17.388 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:17.388 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:29:17.388 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.775042+0000", 2024-09-14T06:29:17.388 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.1299999999999999e-07 2024-09-14T06:29:17.388 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:17.388 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:17.388 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:29:17.389 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.775050+0000", 2024-09-14T06:29:17.389 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.0350000000000001e-06 2024-09-14T06:29:17.389 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:17.389 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:17.389 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:29:17.389 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.775055+0000", 2024-09-14T06:29:17.389 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.34e-06 2024-09-14T06:29:17.389 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:17.389 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:17.389 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:29:17.389 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.775057+0000", 2024-09-14T06:29:17.389 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.6800000000000002e-06 2024-09-14T06:29:17.390 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:17.390 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:17.390 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:29:17.390 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.791988+0000", 2024-09-14T06:29:17.390 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.016930776000000002 2024-09-14T06:29:17.390 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:17.390 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:17.390 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:29:17.390 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.792029+0000", 2024-09-14T06:29:17.390 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.0439999999999999e-05 2024-09-14T06:29:17.390 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:17.390 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:17.390 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:29:17.390 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.792081+0000", 2024-09-14T06:29:17.391 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.2237000000000001e-05 2024-09-14T06:29:17.391 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:17.391 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:17.391 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:17.391 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:17.391 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:17.391 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:3755 3.1f 3:fa5dbf8a:::rbd_data.112d7dfbd766.000000000000018d:5 [read 499712~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-14T06:29:17.391 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:08.775063+0000", 2024-09-14T06:29:17.391 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 8.5313050669999999, 2024-09-14T06:29:17.391 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.017183832999999999, 2024-09-14T06:29:17.391 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:29:17.391 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:29:17.392 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:29:17.392 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:29:17.392 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:29:17.392 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:17.392 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 3755 2024-09-14T06:29:17.392 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:17.392 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:29:17.392 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:17.392 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:29:17.392 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.775063+0000", 2024-09-14T06:29:17.392 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:17.392 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:17.392 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:17.393 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:29:17.393 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.775063+0000", 2024-09-14T06:29:17.393 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:17.393 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:17.393 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:17.393 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:29:17.393 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.775063+0000", 2024-09-14T06:29:17.393 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.8000000000000002e-07 2024-09-14T06:29:17.393 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:17.393 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:17.393 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:29:17.393 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.775065+0000", 2024-09-14T06:29:17.393 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.1480000000000001e-06 2024-09-14T06:29:17.393 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:17.394 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:17.394 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:29:17.394 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.775068+0000", 2024-09-14T06:29:17.394 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.811e-06 2024-09-14T06:29:17.394 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:17.394 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:17.394 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:29:17.394 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.775071+0000", 2024-09-14T06:29:17.394 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.5900000000000002e-06 2024-09-14T06:29:17.394 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:17.394 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:17.394 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:29:17.395 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.792098+0000", 2024-09-14T06:29:17.395 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.017027490999999999 2024-09-14T06:29:17.395 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:17.395 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:17.395 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:29:17.395 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.792138+0000", 2024-09-14T06:29:17.395 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.9999000000000001e-05 2024-09-14T06:29:17.395 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:17.395 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:17.395 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:29:17.395 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.792247+0000", 2024-09-14T06:29:17.395 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000108514 2024-09-14T06:29:17.395 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:17.395 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:17.396 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:17.396 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:17.396 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:17.396 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:3756 3.1f 3:fa5dbf8a:::rbd_data.112d7dfbd766.000000000000018d:5 [read 516096~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-14T06:29:17.396 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:08.775074+0000", 2024-09-14T06:29:17.396 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 8.5312942409999994, 2024-09-14T06:29:17.396 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.017330607000000001, 2024-09-14T06:29:17.396 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:29:17.396 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:29:17.396 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:29:17.396 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:29:17.396 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:29:17.396 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:17.396 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 3756 2024-09-14T06:29:17.397 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:17.397 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:29:17.397 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:17.397 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:29:17.397 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.775074+0000", 2024-09-14T06:29:17.397 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:17.397 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:17.397 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:17.397 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:29:17.397 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.775074+0000", 2024-09-14T06:29:17.397 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:17.397 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:17.397 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:17.398 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:29:17.398 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.775074+0000", 2024-09-14T06:29:17.398 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.1199999999999999e-07 2024-09-14T06:29:17.398 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:17.398 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:17.398 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:29:17.398 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.775075+0000", 2024-09-14T06:29:17.398 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.26e-06 2024-09-14T06:29:17.398 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:17.399 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:17.399 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:29:17.399 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.775079+0000", 2024-09-14T06:29:17.399 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.6689999999999999e-06 2024-09-14T06:29:17.399 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:17.399 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:17.399 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:29:17.399 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.775082+0000", 2024-09-14T06:29:17.399 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.3999999999999999e-06 2024-09-14T06:29:17.399 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:17.399 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:17.399 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:29:17.399 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.792267+0000", 2024-09-14T06:29:17.399 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.01718548 2024-09-14T06:29:17.399 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:17.400 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:17.400 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:29:17.400 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.792308+0000", 2024-09-14T06:29:17.400 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.0877000000000002e-05 2024-09-14T06:29:17.400 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:17.400 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:17.400 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:29:17.400 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.792405+0000", 2024-09-14T06:29:17.400 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.6708999999999999e-05 2024-09-14T06:29:17.400 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:17.400 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:17.400 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:17.400 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:17.400 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:17.401 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-14T06:29:17.402 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-14T06:29:17.402 INFO:tasks.ceph.osd.1.smithi094.stderr:2024-09-14T06:29:17.401+0000 7f328e47a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33641) UID: 0 2024-09-14T06:29:17.503 INFO:tasks.ceph.osd.1.smithi094.stderr:2024-09-14T06:29:17.502+0000 7f328e47a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33641) UID: 0 2024-09-14T06:29:17.547 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-14T06:29:17.547 INFO:teuthology.orchestra.run.smithi094.stdout: "ops": [], 2024-09-14T06:29:17.547 INFO:teuthology.orchestra.run.smithi094.stdout: "num_ops": 0 2024-09-14T06:29:17.547 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-14T06:29:17.556 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-14T06:29:17.587 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-14T06:29:17.640 INFO:tasks.ceph.osd.3.smithi094.stderr:2024-09-14T06:29:17.639+0000 7feeeafe8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33655) UID: 0 2024-09-14T06:29:17.704 INFO:tasks.ceph.osd.6.smithi138.stderr:2024-09-14T06:29:17.703+0000 7f8cd8849640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33620) UID: 0 2024-09-14T06:29:17.709 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-14T06:29:17.709 INFO:teuthology.orchestra.run.smithi094.stdout: "ops": [], 2024-09-14T06:29:17.709 INFO:teuthology.orchestra.run.smithi094.stdout: "complaint_time": 30, 2024-09-14T06:29:17.709 INFO:teuthology.orchestra.run.smithi094.stdout: "num_blocked_ops": 0 2024-09-14T06:29:17.709 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-14T06:29:17.719 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-14T06:29:17.805 INFO:tasks.ceph.osd.2.smithi094.stderr:2024-09-14T06:29:17.805+0000 7faf3269b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33649) UID: 0 2024-09-14T06:29:17.867 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-14T06:29:17.867 INFO:teuthology.orchestra.run.smithi094.stdout: "size": 20, 2024-09-14T06:29:17.867 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 600, 2024-09-14T06:29:17.867 INFO:teuthology.orchestra.run.smithi094.stdout: "ops": [ 2024-09-14T06:29:17.867 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:17.867 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:28 3.4 3:27567f25:::rbd_object_map.112d7adb82ff:head [call lock.lock in=8b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-14T06:29:17.867 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:04.271783+0000", 2024-09-14T06:29:17.867 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 13.593088956000001, 2024-09-14T06:29:17.867 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0029333810000000001, 2024-09-14T06:29:17.868 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:29:17.868 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:29:17.868 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:29:17.868 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:29:17.868 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:29:17.868 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:17.868 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 28 2024-09-14T06:29:17.868 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:17.868 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:29:17.868 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:17.868 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:29:17.869 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:04.271783+0000", 2024-09-14T06:29:17.869 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:17.869 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:17.869 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:17.869 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:29:17.869 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:04.271783+0000", 2024-09-14T06:29:17.869 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:17.869 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:17.869 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:17.869 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:29:17.869 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:04.271784+0000", 2024-09-14T06:29:17.869 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.635e-06 2024-09-14T06:29:17.869 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:17.869 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:17.870 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:29:17.870 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:04.271794+0000", 2024-09-14T06:29:17.870 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.2650000000000002e-06 2024-09-14T06:29:17.870 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:17.870 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:17.870 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:29:17.870 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:04.271817+0000", 2024-09-14T06:29:17.871 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.3385000000000001e-05 2024-09-14T06:29:17.871 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:17.871 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:17.871 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:29:17.871 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:04.271824+0000", 2024-09-14T06:29:17.871 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.0500000000000003e-06 2024-09-14T06:29:17.871 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:17.871 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:17.871 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:29:17.872 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:04.271889+0000", 2024-09-14T06:29:17.872 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.4659999999999994e-05 2024-09-14T06:29:17.872 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:17.872 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:17.872 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "op must wait for map", 2024-09-14T06:29:17.872 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:04.271896+0000", 2024-09-14T06:29:17.872 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.52e-06 2024-09-14T06:29:17.872 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:17.872 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:17.872 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:29:17.872 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:04.273576+0000", 2024-09-14T06:29:17.872 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.001680225 2024-09-14T06:29:17.872 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:17.873 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:17.873 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:29:17.873 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:04.273635+0000", 2024-09-14T06:29:17.873 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.8832000000000002e-05 2024-09-14T06:29:17.873 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:17.873 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:17.873 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "waiting for subops from 2", 2024-09-14T06:29:17.873 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:04.273765+0000", 2024-09-14T06:29:17.873 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000129713 2024-09-14T06:29:17.873 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:17.873 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:17.873 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "op_commit", 2024-09-14T06:29:17.873 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:04.274315+0000", 2024-09-14T06:29:17.873 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00054979099999999999 2024-09-14T06:29:17.874 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:17.874 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:17.874 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:29:17.874 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:04.274605+0000", 2024-09-14T06:29:17.874 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00028987899999999999 2024-09-14T06:29:17.874 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:17.874 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:17.874 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-14T06:29:17.874 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:04.274666+0000", 2024-09-14T06:29:17.874 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.1710000000000004e-05 2024-09-14T06:29:17.874 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:17.874 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:17.874 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:29:17.875 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:04.274716+0000", 2024-09-14T06:29:17.875 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.9716000000000003e-05 2024-09-14T06:29:17.875 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:17.875 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:17.875 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:17.875 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:17.875 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:17.875 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:1848 3.4 3:27620a21:::rbd_data.112dcdddac74.000000000000000b:head [write 16384~32768 [fadvise_sequential+fadvise_nocache],write 81920~20480 [fadvise_sequential+fadvise_nocache]] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-09-14T06:29:17.875 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:08.657998+0000", 2024-09-14T06:29:17.875 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 9.2068740899999995, 2024-09-14T06:29:17.875 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0028814520000000001, 2024-09-14T06:29:17.875 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:29:17.875 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:29:17.875 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:29:17.876 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:29:17.876 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:29:17.876 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:17.876 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 1848 2024-09-14T06:29:17.876 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:17.876 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:29:17.876 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:17.876 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:29:17.876 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.657998+0000", 2024-09-14T06:29:17.877 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:17.877 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:17.877 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:17.877 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:29:17.877 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.657998+0000", 2024-09-14T06:29:17.877 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:17.877 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:17.877 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:17.877 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:29:17.877 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.657998+0000", 2024-09-14T06:29:17.877 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.9500000000000001e-07 2024-09-14T06:29:17.877 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:17.877 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:17.877 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:29:17.878 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.658028+0000", 2024-09-14T06:29:17.878 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.9471999999999998e-05 2024-09-14T06:29:17.878 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:17.878 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:17.878 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:29:17.878 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.658044+0000", 2024-09-14T06:29:17.878 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.6089000000000001e-05 2024-09-14T06:29:17.878 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:17.878 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:17.878 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:29:17.878 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.658055+0000", 2024-09-14T06:29:17.878 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.061e-05 2024-09-14T06:29:17.878 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:17.878 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:17.879 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:29:17.879 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.658116+0000", 2024-09-14T06:29:17.879 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.1333000000000004e-05 2024-09-14T06:29:17.879 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:17.879 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:17.879 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:29:17.879 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.658264+0000", 2024-09-14T06:29:17.879 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000148481 2024-09-14T06:29:17.879 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:17.879 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:17.879 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "waiting for subops from 2", 2024-09-14T06:29:17.879 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.658433+0000", 2024-09-14T06:29:17.879 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00016825 2024-09-14T06:29:17.879 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:17.879 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:17.880 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "op_commit", 2024-09-14T06:29:17.880 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.659373+0000", 2024-09-14T06:29:17.880 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00093997799999999995 2024-09-14T06:29:17.880 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:17.880 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:17.880 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:29:17.880 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.660755+0000", 2024-09-14T06:29:17.880 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0013822940000000001 2024-09-14T06:29:17.880 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:17.880 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:17.880 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-14T06:29:17.880 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.660829+0000", 2024-09-14T06:29:17.880 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.4369999999999994e-05 2024-09-14T06:29:17.880 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:17.881 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:17.881 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:29:17.881 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.660879+0000", 2024-09-14T06:29:17.881 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.9679999999999999e-05 2024-09-14T06:29:17.881 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:17.881 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:17.881 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:17.881 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:17.881 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:17.882 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:2199 3.0 3:05f3e980:::rbd_data.112dcdddac74.000000000000000f:head [write 32768~16384 [fadvise_sequential+fadvise_nocache],write 98304~16384 [fadvise_sequential+fadvise_nocache],write 163840~16384 [fadvise_sequential+fadvise_nocache],write 229376~16384 [fadvise_sequential+fadvise_nocache],write 294912~16384 [fadvise_sequential+fadvise_nocache],write 360448~16384 [fadvise_sequential+fadvise_nocache],write 425984~12288 [fadvise_sequential+fadvise_nocache]] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-09-14T06:29:17.882 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:08.679514+0000", 2024-09-14T06:29:17.882 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 9.1853571990000003, 2024-09-14T06:29:17.882 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0029599959999999999, 2024-09-14T06:29:17.882 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:29:17.882 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:29:17.882 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:29:17.882 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:29:17.883 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:29:17.883 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:17.883 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 2199 2024-09-14T06:29:17.883 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:17.883 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:29:17.883 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:17.883 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:29:17.883 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.679514+0000", 2024-09-14T06:29:17.883 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:17.883 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:17.883 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:17.883 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:29:17.884 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.679514+0000", 2024-09-14T06:29:17.884 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:17.884 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:17.884 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:17.884 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:29:17.884 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.679516+0000", 2024-09-14T06:29:17.884 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.7230000000000001e-06 2024-09-14T06:29:17.884 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:17.884 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:17.884 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:29:17.884 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.679576+0000", 2024-09-14T06:29:17.884 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.9734000000000001e-05 2024-09-14T06:29:17.884 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:17.884 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:17.885 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:29:17.885 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.679588+0000", 2024-09-14T06:29:17.885 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.2284e-05 2024-09-14T06:29:17.885 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:17.885 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:17.885 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:29:17.885 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.679596+0000", 2024-09-14T06:29:17.885 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.1480000000000006e-06 2024-09-14T06:29:17.885 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:17.885 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:17.885 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:29:17.886 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.679687+0000", 2024-09-14T06:29:17.886 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.0776e-05 2024-09-14T06:29:17.886 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:17.886 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:17.886 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:29:17.886 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.679854+0000", 2024-09-14T06:29:17.886 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00016668100000000001 2024-09-14T06:29:17.886 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:17.886 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:17.886 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "waiting for subops from 2", 2024-09-14T06:29:17.886 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.680112+0000", 2024-09-14T06:29:17.886 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00025867700000000002 2024-09-14T06:29:17.886 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:17.886 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:17.887 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "op_commit", 2024-09-14T06:29:17.887 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.681502+0000", 2024-09-14T06:29:17.887 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0013895260000000001 2024-09-14T06:29:17.887 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:17.887 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:17.887 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:29:17.887 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.682357+0000", 2024-09-14T06:29:17.887 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00085512900000000005 2024-09-14T06:29:17.887 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:17.887 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:17.887 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-14T06:29:17.887 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.682428+0000", 2024-09-14T06:29:17.887 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.0735000000000006e-05 2024-09-14T06:29:17.887 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:17.887 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:17.888 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:29:17.888 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.682474+0000", 2024-09-14T06:29:17.888 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.6582999999999998e-05 2024-09-14T06:29:17.888 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:17.888 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:17.888 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:17.888 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:17.888 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:17.888 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:2513 3.0 3:06f82ba7:::rbd_data.112dcdddac74.0000000000000028:head [write 524288~32768 [fadvise_sequential+fadvise_nocache],write 589824~32768 [fadvise_sequential+fadvise_nocache],write 655360~32768 [fadvise_sequential+fadvise_nocache],write 720896~32768 [fadvise_sequential+fadvise_nocache],write 786432~32768 [fadvise_sequential+fadvise_nocache],write 851968~27868 [fadvise_sequential+fadvise_nocache]] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-09-14T06:29:17.888 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:08.701645+0000", 2024-09-14T06:29:17.888 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 9.1632265060000009, 2024-09-14T06:29:17.888 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0034338799999999998, 2024-09-14T06:29:17.888 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:29:17.889 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:29:17.889 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:29:17.889 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:29:17.889 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:29:17.889 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:17.889 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 2513 2024-09-14T06:29:17.889 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:17.889 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:29:17.889 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:17.889 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:29:17.889 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.701645+0000", 2024-09-14T06:29:17.889 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:17.889 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:17.889 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:17.890 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:29:17.890 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.701645+0000", 2024-09-14T06:29:17.890 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:17.890 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:17.890 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:17.890 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:29:17.890 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.701646+0000", 2024-09-14T06:29:17.890 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.54e-07 2024-09-14T06:29:17.890 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:17.890 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:17.890 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:29:17.890 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.701708+0000", 2024-09-14T06:29:17.890 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.2290000000000002e-05 2024-09-14T06:29:17.890 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:17.890 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:17.891 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:29:17.891 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.701718+0000", 2024-09-14T06:29:17.891 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.6590000000000002e-06 2024-09-14T06:29:17.891 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:17.891 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:17.891 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:29:17.891 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.701725+0000", 2024-09-14T06:29:17.891 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.9059999999999996e-06 2024-09-14T06:29:17.891 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:17.891 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:17.891 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:29:17.891 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.701784+0000", 2024-09-14T06:29:17.891 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.9429000000000003e-05 2024-09-14T06:29:17.891 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:17.891 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:17.892 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:29:17.892 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.701917+0000", 2024-09-14T06:29:17.892 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00013305700000000001 2024-09-14T06:29:17.892 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:17.892 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:17.892 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "waiting for subops from 2", 2024-09-14T06:29:17.892 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.702237+0000", 2024-09-14T06:29:17.892 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00032004100000000002 2024-09-14T06:29:17.892 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:17.892 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:17.892 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "op_commit", 2024-09-14T06:29:17.892 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.704494+0000", 2024-09-14T06:29:17.892 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0022567949999999998 2024-09-14T06:29:17.892 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:17.893 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:17.893 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:29:17.893 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.705007+0000", 2024-09-14T06:29:17.893 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00051250199999999997 2024-09-14T06:29:17.893 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:17.893 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:17.893 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-14T06:29:17.893 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.705049+0000", 2024-09-14T06:29:17.893 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.1967e-05 2024-09-14T06:29:17.893 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:17.893 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:17.893 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:29:17.893 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.705079+0000", 2024-09-14T06:29:17.893 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.0380000000000001e-05 2024-09-14T06:29:17.894 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:17.894 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:17.894 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:17.894 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:17.894 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:17.894 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:2539 3.19 3:99040d63:::rbd_data.112d7dfbd766.000000000000005f:5 [read 167936~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-14T06:29:17.894 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:08.703090+0000", 2024-09-14T06:29:17.894 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 9.1617818030000002, 2024-09-14T06:29:17.894 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.002622138, 2024-09-14T06:29:17.894 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:29:17.894 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:29:17.894 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:29:17.895 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:29:17.895 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:29:17.895 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:17.895 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 2539 2024-09-14T06:29:17.895 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:17.895 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:29:17.895 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:17.895 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:29:17.895 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.703090+0000", 2024-09-14T06:29:17.895 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:17.895 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:17.895 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:17.895 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:29:17.895 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.703090+0000", 2024-09-14T06:29:17.895 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:17.896 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:17.896 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:17.896 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:29:17.896 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.703090+0000", 2024-09-14T06:29:17.896 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.6899999999999998e-07 2024-09-14T06:29:17.896 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:17.896 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:17.896 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:29:17.896 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.703092+0000", 2024-09-14T06:29:17.896 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.9099999999999999e-06 2024-09-14T06:29:17.896 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:17.896 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:17.896 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:29:17.897 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.703098+0000", 2024-09-14T06:29:17.897 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.6119999999999998e-06 2024-09-14T06:29:17.897 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:17.897 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:17.897 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:29:17.897 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.703102+0000", 2024-09-14T06:29:17.897 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.78e-06 2024-09-14T06:29:17.897 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:17.897 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:17.897 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:29:17.897 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.705527+0000", 2024-09-14T06:29:17.897 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.002424079 2024-09-14T06:29:17.897 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:17.897 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:17.898 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:29:17.898 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.705615+0000", 2024-09-14T06:29:17.898 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.8213000000000004e-05 2024-09-14T06:29:17.898 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:17.898 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:17.898 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:29:17.898 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.705712+0000", 2024-09-14T06:29:17.898 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.7175000000000004e-05 2024-09-14T06:29:17.898 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:17.898 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:17.898 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:17.898 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:17.898 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:17.898 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:2540 3.19 3:99040d63:::rbd_data.112d7dfbd766.000000000000005f:5 [read 184320~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-14T06:29:17.898 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:08.703109+0000", 2024-09-14T06:29:17.899 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 9.1617628769999992, 2024-09-14T06:29:17.899 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.002871976, 2024-09-14T06:29:17.899 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:29:17.899 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:29:17.899 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:29:17.899 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:29:17.899 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:29:17.899 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:17.899 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 2540 2024-09-14T06:29:17.899 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:17.899 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:29:17.899 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:17.899 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:29:17.899 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.703109+0000", 2024-09-14T06:29:17.900 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:17.900 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:17.900 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:17.900 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:29:17.900 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.703109+0000", 2024-09-14T06:29:17.900 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:17.900 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:17.900 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:17.900 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:29:17.900 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.703109+0000", 2024-09-14T06:29:17.900 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.7099999999999997e-07 2024-09-14T06:29:17.900 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:17.900 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:17.900 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:29:17.901 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.703124+0000", 2024-09-14T06:29:17.901 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.5157000000000001e-05 2024-09-14T06:29:17.901 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:17.901 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:17.901 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:29:17.901 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.703214+0000", 2024-09-14T06:29:17.901 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.9658000000000001e-05 2024-09-14T06:29:17.901 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:17.901 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:17.901 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:29:17.901 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.703403+0000", 2024-09-14T06:29:17.901 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00018862699999999999 2024-09-14T06:29:17.901 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:17.901 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:17.902 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:29:17.902 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.705742+0000", 2024-09-14T06:29:17.902 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0023398500000000001 2024-09-14T06:29:17.902 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:17.902 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:17.902 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:29:17.902 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.705825+0000", 2024-09-14T06:29:17.902 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.2766e-05 2024-09-14T06:29:17.902 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:17.902 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:17.902 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:29:17.902 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.705981+0000", 2024-09-14T06:29:17.902 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00015554699999999999 2024-09-14T06:29:17.902 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:17.903 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:17.903 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:17.903 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:17.903 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:17.903 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:2559 3.19 3:99040d63:::rbd_data.112d7dfbd766.000000000000005f:5 [read 40960~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-14T06:29:17.903 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:08.704001+0000", 2024-09-14T06:29:17.903 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 9.1608702599999994, 2024-09-14T06:29:17.903 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.002611969, 2024-09-14T06:29:17.903 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:29:17.903 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:29:17.903 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:29:17.903 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:29:17.903 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:29:17.904 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:17.904 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 2559 2024-09-14T06:29:17.904 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:17.904 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:29:17.904 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:17.904 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:29:17.904 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.704001+0000", 2024-09-14T06:29:17.904 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:17.904 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:17.904 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:17.904 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:29:17.904 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.704001+0000", 2024-09-14T06:29:17.904 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:17.904 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:17.905 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:17.905 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:29:17.905 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.704002+0000", 2024-09-14T06:29:17.905 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.6800000000000001e-07 2024-09-14T06:29:17.905 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:17.905 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:17.905 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:29:17.905 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.704003+0000", 2024-09-14T06:29:17.905 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.7910000000000001e-06 2024-09-14T06:29:17.905 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:17.905 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:17.905 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:29:17.905 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.704009+0000", 2024-09-14T06:29:17.905 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.7150000000000003e-06 2024-09-14T06:29:17.906 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:17.906 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:17.906 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:29:17.906 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.704017+0000", 2024-09-14T06:29:17.906 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.4039999999999998e-06 2024-09-14T06:29:17.906 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:17.906 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:17.906 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:29:17.906 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.706451+0000", 2024-09-14T06:29:17.906 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.002434758 2024-09-14T06:29:17.906 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:17.906 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:17.907 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:29:17.907 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.706522+0000", 2024-09-14T06:29:17.907 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.1038000000000007e-05 2024-09-14T06:29:17.907 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:17.907 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:17.907 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:29:17.907 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.706613+0000", 2024-09-14T06:29:17.907 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.0895000000000003e-05 2024-09-14T06:29:17.907 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:17.907 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:17.907 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:17.907 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:17.908 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:17.908 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:2560 3.19 3:99040d63:::rbd_data.112d7dfbd766.000000000000005f:5 [read 57344~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-14T06:29:17.908 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:08.704029+0000", 2024-09-14T06:29:17.908 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 9.1608421960000008, 2024-09-14T06:29:17.908 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.002744776, 2024-09-14T06:29:17.908 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:29:17.908 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:29:17.908 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:29:17.908 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:29:17.908 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:29:17.908 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:17.908 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 2560 2024-09-14T06:29:17.908 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:17.909 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:29:17.909 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:17.909 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:29:17.909 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.704029+0000", 2024-09-14T06:29:17.909 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:17.909 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:17.909 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:17.909 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:29:17.909 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.704029+0000", 2024-09-14T06:29:17.909 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:17.909 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:17.909 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:17.909 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:29:17.909 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.704030+0000", 2024-09-14T06:29:17.909 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.1399999999999997e-07 2024-09-14T06:29:17.910 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:17.910 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:17.910 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:29:17.910 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.704033+0000", 2024-09-14T06:29:17.910 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.7199999999999998e-06 2024-09-14T06:29:17.910 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:17.910 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:17.910 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:29:17.910 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.704038+0000", 2024-09-14T06:29:17.911 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.925e-06 2024-09-14T06:29:17.911 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:17.911 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:17.911 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:29:17.911 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.704043+0000", 2024-09-14T06:29:17.911 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.9239999999999998e-06 2024-09-14T06:29:17.911 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:17.911 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:17.911 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:29:17.911 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.706648+0000", 2024-09-14T06:29:17.911 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.002604285 2024-09-14T06:29:17.911 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:17.911 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:17.911 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:29:17.912 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.706690+0000", 2024-09-14T06:29:17.912 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.278e-05 2024-09-14T06:29:17.912 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:17.912 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:17.912 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:29:17.912 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.706774+0000", 2024-09-14T06:29:17.912 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.3727999999999995e-05 2024-09-14T06:29:17.912 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:17.912 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:17.912 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:17.912 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:17.912 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:17.912 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:2561 3.19 3:99040d63:::rbd_data.112d7dfbd766.000000000000005f:5 [read 73728~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-14T06:29:17.913 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:08.704050+0000", 2024-09-14T06:29:17.913 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 9.1608218969999999, 2024-09-14T06:29:17.913 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0029452300000000001, 2024-09-14T06:29:17.913 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:29:17.913 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:29:17.913 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:29:17.913 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:29:17.913 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:29:17.913 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:17.913 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 2561 2024-09-14T06:29:17.913 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:17.913 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:29:17.913 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:17.913 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:29:17.914 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.704050+0000", 2024-09-14T06:29:17.914 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:17.914 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:17.914 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:17.914 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:29:17.914 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.704050+0000", 2024-09-14T06:29:17.914 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:17.914 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:17.914 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:17.914 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:29:17.914 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.704050+0000", 2024-09-14T06:29:17.914 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.4400000000000001e-07 2024-09-14T06:29:17.914 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:17.915 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:17.915 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:29:17.915 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.704052+0000", 2024-09-14T06:29:17.915 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.9290000000000001e-06 2024-09-14T06:29:17.915 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:17.915 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:17.915 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:29:17.915 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.704072+0000", 2024-09-14T06:29:17.915 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.9650999999999999e-05 2024-09-14T06:29:17.915 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:17.915 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:17.915 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:29:17.915 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.704077+0000", 2024-09-14T06:29:17.915 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.7799999999999997e-06 2024-09-14T06:29:17.915 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:17.916 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:17.916 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:29:17.916 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.706827+0000", 2024-09-14T06:29:17.916 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0027497989999999998 2024-09-14T06:29:17.916 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:17.916 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:17.916 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:29:17.916 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.706901+0000", 2024-09-14T06:29:17.916 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.4208999999999994e-05 2024-09-14T06:29:17.916 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:17.916 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:17.917 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:29:17.917 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.706995+0000", 2024-09-14T06:29:17.917 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.3517999999999995e-05 2024-09-14T06:29:17.917 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:17.917 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:17.917 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:17.917 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:17.917 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:17.917 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:2562 3.19 3:99040d63:::rbd_data.112d7dfbd766.000000000000005f:5 [read 90112~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-14T06:29:17.917 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:08.704086+0000", 2024-09-14T06:29:17.917 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 9.1607859969999996, 2024-09-14T06:29:17.917 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0031048159999999998, 2024-09-14T06:29:17.917 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:29:17.917 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:29:17.918 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:29:17.918 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:29:17.918 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:29:17.918 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:17.918 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 2562 2024-09-14T06:29:17.918 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:17.918 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:29:17.918 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:17.918 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:29:17.918 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.704086+0000", 2024-09-14T06:29:17.918 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:17.918 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:17.918 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:17.918 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:29:17.919 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.704086+0000", 2024-09-14T06:29:17.919 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:17.919 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:17.919 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:17.919 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:29:17.919 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.704086+0000", 2024-09-14T06:29:17.919 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.8999999999999997e-07 2024-09-14T06:29:17.919 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:17.919 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:17.919 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:29:17.919 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.704089+0000", 2024-09-14T06:29:17.919 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.413e-06 2024-09-14T06:29:17.919 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:17.919 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:17.920 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:29:17.920 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.704094+0000", 2024-09-14T06:29:17.920 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.8119999999999996e-06 2024-09-14T06:29:17.920 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:17.920 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:17.920 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:29:17.920 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.704099+0000", 2024-09-14T06:29:17.920 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.7639999999999996e-06 2024-09-14T06:29:17.920 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:17.920 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:17.920 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:29:17.920 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.707026+0000", 2024-09-14T06:29:17.920 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0029271750000000002 2024-09-14T06:29:17.921 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:17.921 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:17.921 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:29:17.921 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.707097+0000", 2024-09-14T06:29:17.921 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.1052999999999999e-05 2024-09-14T06:29:17.921 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:17.921 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:17.921 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:29:17.921 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.707190+0000", 2024-09-14T06:29:17.921 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.3109000000000006e-05 2024-09-14T06:29:17.922 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:17.922 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:17.922 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:17.922 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:17.922 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:17.922 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:2563 3.19 3:99040d63:::rbd_data.112d7dfbd766.000000000000005f:5 [read 106496~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-14T06:29:17.922 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:08.704177+0000", 2024-09-14T06:29:17.922 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 9.1606949689999997, 2024-09-14T06:29:17.922 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0031916900000000001, 2024-09-14T06:29:17.922 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:29:17.922 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:29:17.922 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:29:17.923 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:29:17.923 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:29:17.923 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:17.923 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 2563 2024-09-14T06:29:17.923 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:17.923 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:29:17.923 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:17.923 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:29:17.923 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.704177+0000", 2024-09-14T06:29:17.923 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:17.923 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:17.923 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:17.923 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:29:17.923 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.704177+0000", 2024-09-14T06:29:17.924 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:17.924 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:17.924 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:17.924 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:29:17.924 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.704177+0000", 2024-09-14T06:29:17.924 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.4499999999999997e-07 2024-09-14T06:29:17.924 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:17.924 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:17.924 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:29:17.924 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.704180+0000", 2024-09-14T06:29:17.924 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.1439999999999999e-06 2024-09-14T06:29:17.924 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:17.924 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:17.924 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:29:17.925 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.704189+0000", 2024-09-14T06:29:17.925 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.2009999999999999e-06 2024-09-14T06:29:17.925 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:17.925 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:17.925 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:29:17.925 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.704194+0000", 2024-09-14T06:29:17.925 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.8690000000000001e-06 2024-09-14T06:29:17.925 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:17.925 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:17.925 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:29:17.925 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.707218+0000", 2024-09-14T06:29:17.925 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0030240029999999999 2024-09-14T06:29:17.925 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:17.926 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:17.926 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:29:17.926 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.707287+0000", 2024-09-14T06:29:17.926 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.8299000000000005e-05 2024-09-14T06:29:17.926 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:17.926 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:17.926 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:29:17.926 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.707368+0000", 2024-09-14T06:29:17.926 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.1528999999999998e-05 2024-09-14T06:29:17.926 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:17.926 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:17.926 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:17.926 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:17.927 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:17.927 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:2564 3.19 3:99040d63:::rbd_data.112d7dfbd766.000000000000005f:5 [read 122880~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-14T06:29:17.927 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:08.704210+0000", 2024-09-14T06:29:17.927 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 9.1606619719999998, 2024-09-14T06:29:17.927 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0033551369999999998, 2024-09-14T06:29:17.927 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:29:17.927 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:29:17.927 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:29:17.927 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:29:17.927 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:29:17.927 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:17.927 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 2564 2024-09-14T06:29:17.927 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:17.928 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:29:17.928 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:17.928 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:29:17.928 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.704210+0000", 2024-09-14T06:29:17.928 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:17.928 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:17.928 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:17.928 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:29:17.928 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.704210+0000", 2024-09-14T06:29:17.928 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:17.928 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:17.928 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:17.928 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:29:17.929 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.704210+0000", 2024-09-14T06:29:17.929 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.3799999999999998e-07 2024-09-14T06:29:17.929 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:17.929 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:17.929 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:29:17.929 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.704212+0000", 2024-09-14T06:29:17.929 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.2010000000000002e-06 2024-09-14T06:29:17.929 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:17.929 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:17.929 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:29:17.929 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.704218+0000", 2024-09-14T06:29:17.929 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.1700000000000002e-06 2024-09-14T06:29:17.930 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:17.930 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:17.930 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:29:17.930 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.704223+0000", 2024-09-14T06:29:17.930 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.9069999999999998e-06 2024-09-14T06:29:17.930 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:17.930 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:17.930 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:29:17.930 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.707400+0000", 2024-09-14T06:29:17.930 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0031764419999999998 2024-09-14T06:29:17.930 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:17.930 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:17.930 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:29:17.930 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.707473+0000", 2024-09-14T06:29:17.931 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.3205000000000001e-05 2024-09-14T06:29:17.931 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:17.931 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:17.931 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:29:17.931 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.707565+0000", 2024-09-14T06:29:17.931 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.1774000000000006e-05 2024-09-14T06:29:17.931 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:17.931 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:17.931 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:17.931 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:17.931 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:17.931 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:2565 3.19 3:99040d63:::rbd_data.112d7dfbd766.000000000000005f:5 [read 139264~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-14T06:29:17.931 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:08.704230+0000", 2024-09-14T06:29:17.931 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 9.1606419700000004, 2024-09-14T06:29:17.932 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.003537073, 2024-09-14T06:29:17.932 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:29:17.932 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:29:17.932 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:29:17.932 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:29:17.932 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:29:17.932 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:17.932 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 2565 2024-09-14T06:29:17.932 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:17.932 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:29:17.932 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:17.932 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:29:17.932 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.704230+0000", 2024-09-14T06:29:17.933 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:17.933 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:17.933 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:17.933 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:29:17.933 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.704230+0000", 2024-09-14T06:29:17.933 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:17.933 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:17.933 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:17.933 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:29:17.933 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.704230+0000", 2024-09-14T06:29:17.933 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.0900000000000002e-07 2024-09-14T06:29:17.933 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:17.933 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:17.933 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:29:17.934 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.704232+0000", 2024-09-14T06:29:17.934 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.6279999999999999e-06 2024-09-14T06:29:17.934 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:17.934 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:17.934 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:29:17.934 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.704237+0000", 2024-09-14T06:29:17.934 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.327e-06 2024-09-14T06:29:17.934 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:17.934 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:17.934 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:29:17.934 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.704242+0000", 2024-09-14T06:29:17.934 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.6330000000000004e-06 2024-09-14T06:29:17.934 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:17.935 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:17.935 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:29:17.935 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.707596+0000", 2024-09-14T06:29:17.935 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.003354788 2024-09-14T06:29:17.935 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:17.935 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:17.935 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:29:17.935 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.707667+0000", 2024-09-14T06:29:17.935 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.0623000000000005e-05 2024-09-14T06:29:17.935 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:17.935 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:17.935 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:29:17.936 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.707767+0000", 2024-09-14T06:29:17.936 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.9665000000000002e-05 2024-09-14T06:29:17.936 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:17.936 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:17.936 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:17.936 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:17.936 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:17.936 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:2566 3.19 3:99040d63:::rbd_data.112d7dfbd766.000000000000005f:5 [read 155648~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-14T06:29:17.936 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:08.704248+0000", 2024-09-14T06:29:17.936 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 9.1606234709999992, 2024-09-14T06:29:17.936 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0037151810000000001, 2024-09-14T06:29:17.936 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:29:17.936 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:29:17.937 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:29:17.937 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:29:17.937 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:29:17.937 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:17.937 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 2566 2024-09-14T06:29:17.937 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:17.937 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:29:17.937 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:17.937 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:29:17.937 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.704248+0000", 2024-09-14T06:29:17.937 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:17.937 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:17.937 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:17.938 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:29:17.938 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.704248+0000", 2024-09-14T06:29:17.938 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:17.938 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:17.938 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:17.938 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:29:17.938 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.704249+0000", 2024-09-14T06:29:17.938 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.7599999999999998e-07 2024-09-14T06:29:17.938 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:17.938 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:17.938 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:29:17.938 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.704252+0000", 2024-09-14T06:29:17.938 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.411e-06 2024-09-14T06:29:17.939 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:17.939 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:17.939 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:29:17.939 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.704257+0000", 2024-09-14T06:29:17.939 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.5369999999999997e-06 2024-09-14T06:29:17.939 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:17.939 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:17.939 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:29:17.939 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.704263+0000", 2024-09-14T06:29:17.939 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.4820000000000002e-06 2024-09-14T06:29:17.939 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:17.939 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:17.939 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:29:17.940 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.707797+0000", 2024-09-14T06:29:17.940 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.003533722 2024-09-14T06:29:17.940 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:17.940 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:17.940 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:29:17.940 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.707869+0000", 2024-09-14T06:29:17.940 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.1997999999999997e-05 2024-09-14T06:29:17.940 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:17.940 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:17.940 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:29:17.940 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.707963+0000", 2024-09-14T06:29:17.940 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.4654999999999994e-05 2024-09-14T06:29:17.940 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:17.941 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:17.941 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:17.941 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:17.941 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:17.941 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:2567 3.19 3:99040d63:::rbd_data.112d7dfbd766.000000000000005f:5 [read 172032~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-14T06:29:17.941 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:08.704269+0000", 2024-09-14T06:29:17.941 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 9.1606024329999993, 2024-09-14T06:29:17.941 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00397446, 2024-09-14T06:29:17.941 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:29:17.941 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:29:17.941 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:29:17.941 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:29:17.941 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:29:17.941 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:17.941 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 2567 2024-09-14T06:29:17.942 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:17.942 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:29:17.942 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:17.942 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:29:17.942 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.704269+0000", 2024-09-14T06:29:17.942 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:17.942 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:17.942 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:17.942 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:29:17.942 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.704269+0000", 2024-09-14T06:29:17.942 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:17.942 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:17.942 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:17.942 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:29:17.942 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.704270+0000", 2024-09-14T06:29:17.942 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.77e-07 2024-09-14T06:29:17.943 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:17.943 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:17.943 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:29:17.943 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.704271+0000", 2024-09-14T06:29:17.943 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.7829999999999999e-06 2024-09-14T06:29:17.943 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:17.943 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:17.943 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:29:17.943 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.704319+0000", 2024-09-14T06:29:17.943 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.8109999999999998e-05 2024-09-14T06:29:17.943 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:17.943 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:17.943 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:29:17.943 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.704325+0000", 2024-09-14T06:29:17.944 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.7810000000000002e-06 2024-09-14T06:29:17.944 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:17.944 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:17.944 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:29:17.944 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.707993+0000", 2024-09-14T06:29:17.944 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0036678710000000001 2024-09-14T06:29:17.944 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:17.944 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:17.944 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:29:17.944 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.708057+0000", 2024-09-14T06:29:17.944 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.4302999999999997e-05 2024-09-14T06:29:17.944 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:17.944 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:17.944 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:29:17.945 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.708244+0000", 2024-09-14T06:29:17.945 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00018623500000000001 2024-09-14T06:29:17.945 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:17.945 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:17.945 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:17.945 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:17.945 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:17.945 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:2568 3.19 3:99040d63:::rbd_data.112d7dfbd766.000000000000005f:5 [read 188416~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-14T06:29:17.945 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:08.704335+0000", 2024-09-14T06:29:17.945 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 9.1605369139999997, 2024-09-14T06:29:17.945 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.004144778, 2024-09-14T06:29:17.945 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:29:17.945 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:29:17.945 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:29:17.946 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:29:17.946 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:29:17.946 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:17.946 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 2568 2024-09-14T06:29:17.946 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:17.946 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:29:17.946 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:17.946 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:29:17.946 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.704335+0000", 2024-09-14T06:29:17.946 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:17.946 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:17.946 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:17.946 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:29:17.946 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.704335+0000", 2024-09-14T06:29:17.946 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:17.947 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:17.947 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:17.947 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:29:17.947 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.704335+0000", 2024-09-14T06:29:17.947 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.6199999999999998e-07 2024-09-14T06:29:17.947 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:17.947 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:17.947 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:29:17.947 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.704342+0000", 2024-09-14T06:29:17.947 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.5169999999999996e-06 2024-09-14T06:29:17.947 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:17.947 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:17.947 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:29:17.947 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.704348+0000", 2024-09-14T06:29:17.947 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.003e-06 2024-09-14T06:29:17.948 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:17.948 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:17.948 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:29:17.948 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.704353+0000", 2024-09-14T06:29:17.948 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.0699999999999997e-06 2024-09-14T06:29:17.948 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:17.948 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:17.948 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:29:17.948 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.708273+0000", 2024-09-14T06:29:17.948 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0039204009999999996 2024-09-14T06:29:17.948 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:17.948 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:17.948 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:29:17.948 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.708367+0000", 2024-09-14T06:29:17.949 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.3987999999999995e-05 2024-09-14T06:29:17.949 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:17.949 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:17.949 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:29:17.949 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.708479+0000", 2024-09-14T06:29:17.949 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000112237 2024-09-14T06:29:17.949 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:17.949 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:17.949 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:17.949 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:17.949 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:17.949 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:2573 3.19 3:9f664035:::rbd_data.112d7dfbd766.000000000000005d:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-14T06:29:17.949 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:08.704360+0000", 2024-09-14T06:29:17.949 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 9.1605115749999992, 2024-09-14T06:29:17.950 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.004261595, 2024-09-14T06:29:17.950 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:29:17.950 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:29:17.950 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "reached pg", 2024-09-14T06:29:17.950 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:29:17.950 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:29:17.950 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:17.950 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 2573 2024-09-14T06:29:17.950 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:17.950 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:29:17.950 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:17.950 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:29:17.950 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.704360+0000", 2024-09-14T06:29:17.950 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:17.951 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:17.951 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:17.951 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:29:17.951 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.704360+0000", 2024-09-14T06:29:17.951 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:17.951 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:17.951 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:17.951 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:29:17.951 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.704360+0000", 2024-09-14T06:29:17.951 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.63e-07 2024-09-14T06:29:17.951 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:17.951 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:17.951 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:29:17.951 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.704362+0000", 2024-09-14T06:29:17.952 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.748e-06 2024-09-14T06:29:17.952 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:17.952 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:17.952 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:29:17.952 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.704369+0000", 2024-09-14T06:29:17.952 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.0319999999999998e-06 2024-09-14T06:29:17.952 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:17.952 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:17.952 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:29:17.952 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.704404+0000", 2024-09-14T06:29:17.952 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.5057e-05 2024-09-14T06:29:17.952 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:17.953 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:17.953 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:29:17.953 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.708524+0000", 2024-09-14T06:29:17.953 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0041199280000000001 2024-09-14T06:29:17.953 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:17.953 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:17.953 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:29:17.953 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.708622+0000", 2024-09-14T06:29:17.953 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.7367000000000005e-05 2024-09-14T06:29:17.953 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:17.953 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:17.953 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:17.953 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:17.953 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:17.953 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:2575 3.19 3:99040d63:::rbd_data.112d7dfbd766.000000000000005f:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-14T06:29:17.954 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:08.704414+0000", 2024-09-14T06:29:17.954 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 9.1604573449999993, 2024-09-14T06:29:17.954 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0043794460000000004, 2024-09-14T06:29:17.954 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:29:17.954 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:29:17.954 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:29:17.954 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:29:17.954 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:29:17.954 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:17.954 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 2575 2024-09-14T06:29:17.954 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:17.954 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:29:17.954 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:17.954 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:29:17.955 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.704414+0000", 2024-09-14T06:29:17.955 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:17.955 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:17.955 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:17.955 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:29:17.955 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.704414+0000", 2024-09-14T06:29:17.955 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:17.955 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:17.955 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:17.955 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:29:17.955 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.704415+0000", 2024-09-14T06:29:17.955 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.7000000000000005e-07 2024-09-14T06:29:17.955 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:17.955 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:17.956 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:29:17.956 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.704417+0000", 2024-09-14T06:29:17.956 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.503e-06 2024-09-14T06:29:17.956 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:17.956 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:17.956 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:29:17.956 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.704469+0000", 2024-09-14T06:29:17.956 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.1492000000000001e-05 2024-09-14T06:29:17.956 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:17.956 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:17.956 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:29:17.956 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.704474+0000", 2024-09-14T06:29:17.956 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.5929999999999996e-06 2024-09-14T06:29:17.956 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:17.957 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:17.957 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:29:17.957 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.708654+0000", 2024-09-14T06:29:17.957 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0041792640000000002 2024-09-14T06:29:17.957 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:17.957 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:17.957 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:29:17.957 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.708725+0000", 2024-09-14T06:29:17.957 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.1755000000000007e-05 2024-09-14T06:29:17.957 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:17.957 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:17.957 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:29:17.957 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.708794+0000", 2024-09-14T06:29:17.957 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.8269000000000007e-05 2024-09-14T06:29:17.958 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:17.958 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:17.958 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:17.958 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:17.958 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:17.958 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:3928 3.19 3:9e9fa1be:::rbd_data.112dcdddac74.00000000000000c6:head [write 540672~32768 [fadvise_sequential+fadvise_nocache],write 606208~32768 [fadvise_sequential+fadvise_nocache],write 671744~32768 [fadvise_sequential+fadvise_nocache],write 737280~32768 [fadvise_sequential+fadvise_nocache],write 802816~32768 [fadvise_sequential+fadvise_nocache],write 868352~32768 [fadvise_sequential+fadvise_nocache],write 933888~32768 [fadvise_sequential+fadvise_nocache],write 999424~20480 [fadvise_sequential+fadvise_nocache]] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-09-14T06:29:17.958 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:08.794629+0000", 2024-09-14T06:29:17.958 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 9.0702421550000007, 2024-09-14T06:29:17.958 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.002453144, 2024-09-14T06:29:17.958 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:29:17.958 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:29:17.958 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:29:17.958 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:29:17.958 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:29:17.959 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:17.959 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 3928 2024-09-14T06:29:17.959 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:17.959 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:29:17.959 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:17.959 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:29:17.959 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.794629+0000", 2024-09-14T06:29:17.959 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:17.959 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:17.959 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:17.959 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:29:17.959 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.794629+0000", 2024-09-14T06:29:17.959 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:17.959 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:17.959 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:17.960 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:29:17.960 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.794631+0000", 2024-09-14T06:29:17.960 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.8360000000000001e-06 2024-09-14T06:29:17.960 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:17.960 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:17.960 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:29:17.960 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.794707+0000", 2024-09-14T06:29:17.960 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.5288000000000002e-05 2024-09-14T06:29:17.960 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:17.960 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:17.960 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:29:17.960 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.794721+0000", 2024-09-14T06:29:17.960 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.4671999999999999e-05 2024-09-14T06:29:17.960 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:17.961 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:17.961 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:29:17.961 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.794730+0000", 2024-09-14T06:29:17.961 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.4440000000000008e-06 2024-09-14T06:29:17.961 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:17.961 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:17.961 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:29:17.961 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.794795+0000", 2024-09-14T06:29:17.961 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.5158000000000002e-05 2024-09-14T06:29:17.961 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:17.961 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:17.961 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:29:17.961 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.794972+0000", 2024-09-14T06:29:17.961 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00017739899999999999 2024-09-14T06:29:17.961 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:17.962 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:17.962 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "waiting for subops from 3", 2024-09-14T06:29:17.962 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.795285+0000", 2024-09-14T06:29:17.962 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000312589 2024-09-14T06:29:17.962 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:17.962 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:17.962 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "op_commit", 2024-09-14T06:29:17.962 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.796713+0000", 2024-09-14T06:29:17.962 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.001428559 2024-09-14T06:29:17.962 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:17.962 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:17.962 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:29:17.963 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.796954+0000", 2024-09-14T06:29:17.963 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000240597 2024-09-14T06:29:17.963 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:17.963 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:17.963 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-14T06:29:17.963 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.797034+0000", 2024-09-14T06:29:17.963 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.9835999999999999e-05 2024-09-14T06:29:17.963 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:17.963 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:17.963 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:29:17.963 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.797083+0000", 2024-09-14T06:29:17.963 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.8766e-05 2024-09-14T06:29:17.963 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:17.963 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:17.964 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:17.964 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:17.964 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:17.964 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:4781 3.4 3:26a26a4b:::rbd_data.112dcdddac74.000000000000001c:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:29:17.964 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:12.628861+0000", 2024-09-14T06:29:17.964 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 5.2360104810000001, 2024-09-14T06:29:17.964 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.002795479, 2024-09-14T06:29:17.964 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:29:17.964 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:29:17.964 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "reached pg", 2024-09-14T06:29:17.964 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:29:17.964 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:29:17.964 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:17.965 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 4781 2024-09-14T06:29:17.965 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:17.965 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:29:17.965 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:17.965 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:29:17.965 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:12.628861+0000", 2024-09-14T06:29:17.965 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:17.965 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:17.965 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:17.965 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:29:17.965 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:12.628861+0000", 2024-09-14T06:29:17.965 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:17.965 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:17.965 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:17.966 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:29:17.966 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:12.628864+0000", 2024-09-14T06:29:17.966 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.481e-06 2024-09-14T06:29:17.966 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:17.966 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:17.966 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:29:17.966 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:12.628875+0000", 2024-09-14T06:29:17.966 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.1484000000000001e-05 2024-09-14T06:29:17.966 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:17.966 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:17.966 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:29:17.966 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:12.628898+0000", 2024-09-14T06:29:17.966 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.2507e-05 2024-09-14T06:29:17.966 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:17.966 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:17.967 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:29:17.967 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:12.628910+0000", 2024-09-14T06:29:17.967 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.1946999999999999e-05 2024-09-14T06:29:17.967 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:17.967 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:17.967 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:29:17.967 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:12.628999+0000", 2024-09-14T06:29:17.967 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.9954e-05 2024-09-14T06:29:17.967 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:17.967 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:17.967 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "op must wait for map", 2024-09-14T06:29:17.967 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:12.629008+0000", 2024-09-14T06:29:17.967 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.6160000000000002e-06 2024-09-14T06:29:17.967 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:17.968 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:17.968 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:29:17.968 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:12.631545+0000", 2024-09-14T06:29:17.968 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0025371640000000002 2024-09-14T06:29:17.968 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:17.968 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:17.968 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:29:17.968 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:12.631657+0000", 2024-09-14T06:29:17.968 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000111326 2024-09-14T06:29:17.968 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:17.968 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:17.968 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:17.968 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:17.968 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:17.968 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-14T06:29:17.970 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-14T06:29:17.970 INFO:tasks.ceph.osd.2.smithi094.stderr:2024-09-14T06:29:17.969+0000 7faf3269b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33649) UID: 0 2024-09-14T06:29:18.043 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-14T06:29:18.097 INFO:tasks.ceph.osd.0.smithi094.stderr:2024-09-14T06:29:18.096+0000 7f07c9333640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33636) UID: 0 2024-09-14T06:29:18.115 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-14T06:29:18.115 INFO:teuthology.orchestra.run.smithi094.stdout: "ops": [], 2024-09-14T06:29:18.115 INFO:teuthology.orchestra.run.smithi094.stdout: "num_ops": 0 2024-09-14T06:29:18.115 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-14T06:29:18.124 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-14T06:29:18.174 INFO:teuthology.orchestra.run.smithi094.stdout:osd.0: {} 2024-09-14T06:29:18.174 INFO:teuthology.orchestra.run.smithi094.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-14T06:29:18.175 INFO:tasks.ceph.osd.3.smithi094.stderr:2024-09-14T06:29:18.173+0000 7feeeafe8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33655) UID: 0 2024-09-14T06:29:18.191 INFO:teuthology.orchestra.run.smithi094.stdout:osd.1: {} 2024-09-14T06:29:18.191 INFO:teuthology.orchestra.run.smithi094.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-14T06:29:18.208 INFO:teuthology.orchestra.run.smithi094.stdout:osd.2: {} 2024-09-14T06:29:18.208 INFO:teuthology.orchestra.run.smithi094.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-14T06:29:18.225 INFO:teuthology.orchestra.run.smithi094.stdout:osd.3: {} 2024-09-14T06:29:18.225 INFO:teuthology.orchestra.run.smithi094.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-14T06:29:18.243 INFO:teuthology.orchestra.run.smithi094.stdout:osd.4: {} 2024-09-14T06:29:18.243 INFO:teuthology.orchestra.run.smithi094.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-14T06:29:18.260 INFO:teuthology.orchestra.run.smithi094.stdout:osd.5: {} 2024-09-14T06:29:18.261 INFO:teuthology.orchestra.run.smithi094.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-14T06:29:18.271 INFO:teuthology.orchestra.run.smithi094.stdout:ERROR: (22) Invalid argument 2024-09-14T06:29:18.271 INFO:teuthology.orchestra.run.smithi094.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:29:18.272 INFO:tasks.ceph.osd.2.smithi094.stderr:2024-09-14T06:29:18.271+0000 7faf3269b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33649) UID: 0 2024-09-14T06:29:18.277 INFO:teuthology.orchestra.run.smithi094.stdout:osd.6: {} 2024-09-14T06:29:18.277 INFO:teuthology.orchestra.run.smithi094.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-14T06:29:18.280 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-14T06:29:18.294 INFO:teuthology.orchestra.run.smithi094.stdout:osd.7: {} 2024-09-14T06:29:18.295 INFO:teuthology.orchestra.run.smithi094.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-14T06:29:18.373 INFO:tasks.ceph.osd.6.smithi138.stderr:2024-09-14T06:29:18.372+0000 7f8cd8849640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33620) UID: 0 2024-09-14T06:29:18.397 INFO:teuthology.orchestra.run.smithi094.stdout:ERROR: (22) Invalid argument 2024-09-14T06:29:18.397 INFO:teuthology.orchestra.run.smithi094.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:29:18.407 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-14T06:29:18.474 INFO:tasks.ceph.osd.4.smithi138.stderr:2024-09-14T06:29:18.473+0000 7f6c653c8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33615) UID: 0 2024-09-14T06:29:18.509 INFO:teuthology.orchestra.run.smithi094.stdout:ERROR: (22) Invalid argument 2024-09-14T06:29:18.509 INFO:teuthology.orchestra.run.smithi094.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:29:18.518 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-14T06:29:18.575 INFO:tasks.ceph.osd.0.smithi094.stderr:2024-09-14T06:29:18.574+0000 7f07c9333640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33636) UID: 0 2024-09-14T06:29:18.621 INFO:teuthology.orchestra.run.smithi094.stdout:ERROR: (22) Invalid argument 2024-09-14T06:29:18.621 INFO:teuthology.orchestra.run.smithi094.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:29:18.629 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-14T06:29:18.641 INFO:teuthology.orchestra.run.smithi094.stderr:noscrub is unset 2024-09-14T06:29:18.699 INFO:tasks.ceph.osd.2.smithi094.stderr:2024-09-14T06:29:18.698+0000 7faf3269b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33649) UID: 0 2024-09-14T06:29:18.767 INFO:teuthology.orchestra.run.smithi094.stdout:ERROR: (22) Invalid argument 2024-09-14T06:29:18.768 INFO:teuthology.orchestra.run.smithi094.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:29:18.777 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-14T06:29:18.777 INFO:tasks.ceph.osd.1.smithi094.stderr:2024-09-14T06:29:18.775+0000 7f328e47a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33641) UID: 0 2024-09-14T06:29:18.877 INFO:tasks.ceph.osd.5.smithi138.stderr:2024-09-14T06:29:18.876+0000 7fe13746f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33618) UID: 0 2024-09-14T06:29:18.882 INFO:teuthology.orchestra.run.smithi138.stdout:ERROR: (22) Invalid argument 2024-09-14T06:29:18.882 INFO:teuthology.orchestra.run.smithi138.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:29:18.891 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-14T06:29:18.978 INFO:tasks.ceph.osd.0.smithi094.stderr:2024-09-14T06:29:18.977+0000 7f07c9333640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33636) UID: 0 2024-09-14T06:29:19.034 INFO:teuthology.orchestra.run.smithi138.stdout:ERROR: (22) Invalid argument 2024-09-14T06:29:19.034 INFO:teuthology.orchestra.run.smithi138.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:29:19.042 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-14T06:29:19.086 INFO:tasks.ceph.osd.6.smithi138.stderr:2024-09-14T06:29:19.085+0000 7f8cd8849640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33620) UID: 0 2024-09-14T06:29:19.144 INFO:teuthology.orchestra.run.smithi138.stdout:ERROR: (22) Invalid argument 2024-09-14T06:29:19.144 INFO:teuthology.orchestra.run.smithi138.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:29:19.152 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-14T06:29:19.193 INFO:tasks.ceph.osd.6.smithi138.stderr:2024-09-14T06:29:19.192+0000 7f8cd8849640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33620) UID: 0 2024-09-14T06:29:19.282 INFO:tasks.ceph.osd.6.smithi138.stderr:2024-09-14T06:29:19.281+0000 7f8cd8849640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33620) UID: 0 2024-09-14T06:29:19.296 INFO:teuthology.orchestra.run.smithi138.stdout:ERROR: (22) Invalid argument 2024-09-14T06:29:19.296 INFO:teuthology.orchestra.run.smithi138.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:29:19.305 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-14T06:29:19.389 INFO:tasks.ceph.osd.5.smithi138.stderr:2024-09-14T06:29:19.388+0000 7fe13746f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33618) UID: 0 2024-09-14T06:29:19.448 INFO:teuthology.orchestra.run.smithi138.stdout:ERROR: (22) Invalid argument 2024-09-14T06:29:19.448 INFO:teuthology.orchestra.run.smithi138.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:29:19.457 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-14T06:29:19.498 INFO:tasks.ceph.osd.7.smithi138.stderr:2024-09-14T06:29:19.497+0000 7f3c6a21d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33623) UID: 0 2024-09-14T06:29:19.584 INFO:tasks.ceph.osd.5.smithi138.stderr:2024-09-14T06:29:19.583+0000 7fe13746f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33618) UID: 0 2024-09-14T06:29:19.600 INFO:teuthology.orchestra.run.smithi138.stdout:ERROR: (22) Invalid argument 2024-09-14T06:29:19.601 INFO:teuthology.orchestra.run.smithi138.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:29:19.609 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-14T06:29:19.685 INFO:tasks.ceph.osd.3.smithi094.stderr:2024-09-14T06:29:19.684+0000 7feeeafe8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33655) UID: 0 2024-09-14T06:29:19.756 INFO:teuthology.orchestra.run.smithi138.stdout:ERROR: (22) Invalid argument 2024-09-14T06:29:19.757 INFO:teuthology.orchestra.run.smithi138.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:29:19.765 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-14T06:29:19.786 INFO:teuthology.orchestra.run.smithi094.stdout:checking clone #3, image image_client.0-clone3 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent4 2024-09-14T06:29:19.786 INFO:tasks.ceph.osd.3.smithi094.stderr:2024-09-14T06:29:19.785+0000 7feeeafe8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33655) UID: 0 2024-09-14T06:29:19.870 INFO:teuthology.orchestra.run.smithi138.stdout:ERROR: (22) Invalid argument 2024-09-14T06:29:19.870 INFO:teuthology.orchestra.run.smithi138.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:29:19.879 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-14T06:29:19.923 INFO:tasks.ceph.osd.6.smithi138.stderr:2024-09-14T06:29:19.922+0000 7f8cd8849640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33620) UID: 0 2024-09-14T06:29:19.982 INFO:teuthology.orchestra.run.smithi138.stdout:ERROR: (22) Invalid argument 2024-09-14T06:29:19.982 INFO:teuthology.orchestra.run.smithi138.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:29:19.988 INFO:tasks.ceph.osd.4.smithi138.stderr:2024-09-14T06:29:19.987+0000 7f6c653c8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33615) UID: 0 2024-09-14T06:29:19.990 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-14T06:29:20.023 INFO:teuthology.orchestra.run.smithi094.stdout:393 read 0x668b984 thru 0x66905d3 (0x4c50 bytes) 2024-09-14T06:29:20.034 INFO:teuthology.orchestra.run.smithi094.stdout:394 write 0xdf1f6a9 thru 0xdf2110e (0x1a66 bytes) 2024-09-14T06:29:20.037 INFO:teuthology.orchestra.run.smithi094.stdout:395 read 0xaa8fae9 thru 0xaa906bd (0xbd5 bytes) 2024-09-14T06:29:20.037 INFO:teuthology.orchestra.run.smithi094.stdout:396 read 0xbb5e1ca thru 0xbb67364 (0x919b bytes) 2024-09-14T06:29:20.037 INFO:teuthology.orchestra.run.smithi094.stdout:397 read 0x98a6258 thru 0x98b0280 (0xa029 bytes) 2024-09-14T06:29:20.038 INFO:teuthology.orchestra.run.smithi094.stdout:398 write 0xcdbbb99 thru 0xcdc35c1 (0x7a29 bytes) 2024-09-14T06:29:20.041 INFO:teuthology.orchestra.run.smithi094.stdout:399 read 0xd53812b thru 0xd542b9c (0xaa72 bytes) 2024-09-14T06:29:20.041 INFO:teuthology.orchestra.run.smithi094.stdout:401 write 0x62fd38d thru 0x62fdad1 (0x745 bytes) 2024-09-14T06:29:20.055 INFO:teuthology.orchestra.run.smithi094.stdout:404 trunc from 0xdf2110f to 0x425eea6 2024-09-14T06:29:20.079 INFO:teuthology.orchestra.run.smithi094.stdout:405 read 0x2e5854e thru 0x2e6221e (0x9cd1 bytes) 2024-09-14T06:29:20.079 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-14T06:29:20.079 INFO:teuthology.orchestra.run.smithi094.stdout:407 read 0x34dcd23 thru 0x34e43c7 (0x76a5 bytes) 2024-09-14T06:29:20.080 INFO:teuthology.orchestra.run.smithi094.stdout:409 write 0x2a2bce5 thru 0x2a35368 (0x9684 bytes) 2024-09-14T06:29:20.081 INFO:teuthology.orchestra.run.smithi094.stdout:410 read 0x6c96dc thru 0x6ce48d (0x4db2 bytes) 2024-09-14T06:29:20.082 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-14T06:29:20.082 INFO:teuthology.orchestra.run.smithi094.stdout:412 read 0x24c3be3 thru 0x24c7535 (0x3953 bytes) 2024-09-14T06:29:20.082 INFO:teuthology.orchestra.run.smithi094.stdout:413 write 0x19f187b thru 0x19fb2c1 (0x9a47 bytes) 2024-09-14T06:29:20.089 INFO:tasks.ceph.osd.0.smithi094.stderr:2024-09-14T06:29:20.088+0000 7f07c9333640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33636) UID: 0 2024-09-14T06:29:20.092 INFO:teuthology.orchestra.run.smithi094.stdout:415 trunc from 0x425eea6 to 0xb492463 2024-09-14T06:29:20.097 INFO:teuthology.orchestra.run.smithi094.stdout:418 read 0x19bdc4f thru 0x19c8bfa (0xafac bytes) 2024-09-14T06:29:20.102 INFO:teuthology.orchestra.run.smithi094.stdout:420 trunc from 0xb492463 to 0x7c63ba9 2024-09-14T06:29:20.107 INFO:teuthology.orchestra.run.smithi094.stdout:421 punch from 0x4b9ee2c to 0x4ba507b, (0x624f bytes) 2024-09-14T06:29:20.107 INFO:teuthology.orchestra.run.smithi094.stdout:422 write 0x2fe18fc thru 0x2feeb6a (0xd26f bytes) 2024-09-14T06:29:20.109 INFO:teuthology.orchestra.run.smithi094.stdout:423 read 0x46c3bc5 thru 0x46c4e4a (0x1286 bytes) 2024-09-14T06:29:20.109 INFO:teuthology.orchestra.run.smithi094.stdout:424 writesame 0x2cdfd4e thru 0x2ceb6dc data_size 0x1(0xb98f bytes) 2024-09-14T06:29:20.135 INFO:teuthology.orchestra.run.smithi094.stdout:425 trunc from 0x7c63ba9 to 0xb61ccbe 2024-09-14T06:29:20.140 INFO:teuthology.orchestra.run.smithi094.stdout:426 write 0x97085ad thru 0x9713a1e (0xb472 bytes) 2024-09-14T06:29:20.140 INFO:teuthology.orchestra.run.smithi138.stdout:ERROR: (22) Invalid argument 2024-09-14T06:29:20.141 INFO:teuthology.orchestra.run.smithi138.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:29:20.149 INFO:teuthology.orchestra.run.smithi094.stdout:427 write 0xbeb5b7b thru 0xbebcd7a (0x7200 bytes) 2024-09-14T06:29:20.150 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-14T06:29:20.160 INFO:teuthology.orchestra.run.smithi094.stdout:428 writesame 0xe42c938 thru 0xe436f2d data_size 0x61(0xa5f6 bytes) 2024-09-14T06:29:20.161 INFO:teuthology.orchestra.run.smithi094.stdout:429 writesame 0x37be969 thru 0x37c8c45 data_size 0x1(0xa2dd bytes) 2024-09-14T06:29:20.179 INFO:teuthology.orchestra.run.smithi094.stdout:430 writesame 0x3ac9cea thru 0x3ad57c8 data_size 0x1(0xbadf bytes) 2024-09-14T06:29:20.193 INFO:tasks.ceph.osd.5.smithi138.stderr:2024-09-14T06:29:20.192+0000 7fe13746f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33618) UID: 0 2024-09-14T06:29:20.200 INFO:teuthology.orchestra.run.smithi094.stdout:432 writesame 0x4315dc9 thru 0x43253cc data_size 0x2(0xf604 bytes) 2024-09-14T06:29:20.213 INFO:teuthology.orchestra.run.smithi094.stdout:433 writesame 0xcf6ef1b thru 0xcf747ff data_size 0x1(0x58e5 bytes) 2024-09-14T06:29:20.229 INFO:teuthology.orchestra.run.smithi094.stdout:434 read 0x73c2e81 thru 0x73cca98 (0x9c18 bytes) 2024-09-14T06:29:20.230 INFO:teuthology.orchestra.run.smithi094.stdout:436 write 0x94317ba thru 0x9440574 (0xedbb bytes) 2024-09-14T06:29:20.239 INFO:teuthology.orchestra.run.smithi094.stdout:438 trunc from 0xe436f2e to 0x9e3b4da 2024-09-14T06:29:20.245 INFO:teuthology.orchestra.run.smithi094.stdout:439 read 0x2adc03e thru 0x2ae7eb5 (0xbe78 bytes) 2024-09-14T06:29:20.251 INFO:teuthology.orchestra.run.smithi094.stdout:440 trunc from 0x9e3b4da to 0x593c7d2 2024-09-14T06:29:20.262 INFO:teuthology.orchestra.run.smithi138.stdout:ERROR: (22) Invalid argument 2024-09-14T06:29:20.262 INFO:teuthology.orchestra.run.smithi138.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:29:20.265 INFO:teuthology.orchestra.run.smithi094.stdout:442 punch from 0x27f91b0 to 0x27fe657, (0x54a7 bytes) 2024-09-14T06:29:20.265 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-14T06:29:20.271 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-14T06:29:20.272 INFO:teuthology.orchestra.run.smithi094.stdout:444 write 0x6ba2e34 thru 0x6bb1a49 (0xec16 bytes) 2024-09-14T06:29:20.273 INFO:teuthology.orchestra.run.smithi094.stdout:445 writesame 0x834986 thru 0x83db02 data_size 0x41(0x917d bytes) 2024-09-14T06:29:20.274 INFO:teuthology.orchestra.run.smithi094.stdout:447 write 0x19f7f3f thru 0x1a04b46 (0xcc08 bytes) 2024-09-14T06:29:20.276 INFO:teuthology.orchestra.run.smithi094.stdout:448 read 0x1510aee thru 0x1513539 (0x2a4c bytes) 2024-09-14T06:29:20.287 INFO:teuthology.orchestra.run.smithi094.stdout:449 write 0xafee723 thru 0xaff9727 (0xb005 bytes) 2024-09-14T06:29:20.291 INFO:tasks.ceph.osd.3.smithi094.stderr:2024-09-14T06:29:20.290+0000 7feeeafe8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33655) UID: 0 2024-09-14T06:29:20.297 INFO:teuthology.orchestra.run.smithi094.stdout:450 trunc from 0xaff9728 to 0x4c174f0 2024-09-14T06:29:20.305 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-14T06:29:20.308 INFO:teuthology.orchestra.run.smithi094.stdout:451 read 0x273ebbc thru 0x27475b9 (0x89fe bytes) 2024-09-14T06:29:20.309 INFO:teuthology.orchestra.run.smithi094.stdout:452 read 0x744126 thru 0x753474 (0xf34f bytes) 2024-09-14T06:29:20.309 INFO:teuthology.orchestra.run.smithi094.stdout:453 punch from 0x2179a73 to 0x21827ef, (0x8d7c bytes) 2024-09-14T06:29:20.309 INFO:teuthology.orchestra.run.smithi094.stdout:454 read 0x2c7efee thru 0x2c7f0e3 (0xf6 bytes) 2024-09-14T06:29:20.310 INFO:teuthology.orchestra.run.smithi094.stdout:458 read 0x1eaeadd thru 0x1ebd7c8 (0xecec bytes) 2024-09-14T06:29:20.318 INFO:teuthology.orchestra.run.smithi094.stdout:460 trunc from 0x4c174f0 to 0xbb4e473 2024-09-14T06:29:20.331 INFO:teuthology.orchestra.run.smithi094.stdout:461 trunc from 0xbb4e473 to 0x5e6eee0 2024-09-14T06:29:20.339 INFO:teuthology.orchestra.run.smithi094.stdout:462 write 0x4c776e5 thru 0x4c7c005 (0x4921 bytes) 2024-09-14T06:29:20.356 INFO:teuthology.orchestra.run.smithi094.stdout:463 write 0xe71e791 thru 0xe72783b (0x90ab bytes) 2024-09-14T06:29:20.360 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-14T06:29:20.360 INFO:teuthology.orchestra.run.smithi094.stdout:465 read 0xd95f873 thru 0xd96358e (0x3d1c bytes) 2024-09-14T06:29:20.360 INFO:teuthology.orchestra.run.smithi094.stdout:466 write 0xd735558 thru 0xd736ab9 (0x1562 bytes) 2024-09-14T06:29:20.362 INFO:teuthology.orchestra.run.smithi094.stdout:469 punch from 0xc73d6f4 to 0xc745c88, (0x8594 bytes) 2024-09-14T06:29:20.362 INFO:teuthology.orchestra.run.smithi094.stdout:470 punch from 0x393c0fa to 0x393cd1b, (0xc21 bytes) 2024-09-14T06:29:20.378 INFO:teuthology.orchestra.run.smithi094.stdout:471 trunc from 0xe72783c to 0x1617ab7 2024-09-14T06:29:20.392 INFO:tasks.ceph.osd.5.smithi138.stderr:2024-09-14T06:29:20.391+0000 7fe13746f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33618) UID: 0 2024-09-14T06:29:20.407 INFO:teuthology.orchestra.run.smithi094.stdout:472 write 0xa52b27a thru 0xa52ba0c (0x793 bytes) 2024-09-14T06:29:20.413 INFO:teuthology.orchestra.run.smithi094.stdout:473 trunc from 0xa52ba0d to 0xcf46899 2024-09-14T06:29:20.418 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-14T06:29:20.422 INFO:teuthology.orchestra.run.smithi138.stdout:ERROR: (22) Invalid argument 2024-09-14T06:29:20.423 INFO:teuthology.orchestra.run.smithi138.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:29:20.423 INFO:teuthology.orchestra.run.smithi094.stdout:475 writesame 0xd14edfc thru 0xd1533d5 data_size 0x1(0x45da bytes) 2024-09-14T06:29:20.424 INFO:teuthology.orchestra.run.smithi094.stdout:476 write 0x12c131f thru 0x12d0a94 (0xf776 bytes) 2024-09-14T06:29:20.432 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-14T06:29:20.437 INFO:teuthology.orchestra.run.smithi094.stdout:479 trunc from 0xd1533d6 to 0x32fca1e 2024-09-14T06:29:20.446 INFO:teuthology.orchestra.run.smithi094.stdout:osd.0: {} 2024-09-14T06:29:20.446 INFO:teuthology.orchestra.run.smithi094.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-14T06:29:20.452 INFO:teuthology.orchestra.run.smithi094.stdout:481 read 0x109d6dd thru 0x10ab13e (0xda62 bytes) 2024-09-14T06:29:20.465 INFO:teuthology.orchestra.run.smithi094.stdout:483 write 0x8120789 thru 0x812d879 (0xd0f1 bytes) 2024-09-14T06:29:20.467 INFO:teuthology.orchestra.run.smithi094.stdout:osd.1: {} 2024-09-14T06:29:20.467 INFO:teuthology.orchestra.run.smithi094.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-14T06:29:20.470 INFO:teuthology.orchestra.run.smithi094.stdout:484 read 0x5633321 thru 0x563c700 (0x93e0 bytes) 2024-09-14T06:29:20.470 INFO:teuthology.orchestra.run.smithi094.stdout:485 write 0x6ca58da thru 0x6cab72e (0x5e55 bytes) 2024-09-14T06:29:20.471 INFO:teuthology.orchestra.run.smithi094.stdout:486 writesame 0x1a8f68d thru 0x1a9f533 data_size 0x43(0xfea7 bytes) 2024-09-14T06:29:20.481 INFO:teuthology.orchestra.run.smithi094.stdout:487 write 0x99a3828 thru 0x99b1739 (0xdf12 bytes) 2024-09-14T06:29:20.484 INFO:teuthology.orchestra.run.smithi094.stdout:489 trunc from 0x99b173a to 0x8efe9cd 2024-09-14T06:29:20.486 INFO:teuthology.orchestra.run.smithi094.stdout:osd.2: {} 2024-09-14T06:29:20.486 INFO:teuthology.orchestra.run.smithi094.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-14T06:29:20.491 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-14T06:29:20.491 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-14T06:29:20.493 INFO:tasks.ceph.osd.5.smithi138.stderr:2024-09-14T06:29:20.492+0000 7fe13746f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33618) UID: 0 2024-09-14T06:29:20.501 INFO:teuthology.orchestra.run.smithi094.stdout:494 write 0xcea7bcd thru 0xceabe57 (0x428b bytes) 2024-09-14T06:29:20.504 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-14T06:29:20.505 INFO:teuthology.orchestra.run.smithi094.stdout:498 read 0x312920b thru 0x3135d63 (0xcb59 bytes) 2024-09-14T06:29:20.505 INFO:teuthology.orchestra.run.smithi094.stdout:502 write 0x552ab7f thru 0x5536986 (0xbe08 bytes) 2024-09-14T06:29:20.506 INFO:teuthology.orchestra.run.smithi094.stdout:503 write 0xf919e thru 0x10035a (0x71bd bytes) 2024-09-14T06:29:20.508 INFO:teuthology.orchestra.run.smithi094.stdout:504 write 0x1c01be6 thru 0x1c0fe12 (0xe22d bytes) 2024-09-14T06:29:20.510 INFO:teuthology.orchestra.run.smithi094.stdout:505 read 0x311dc1c thru 0x3125354 (0x7739 bytes) 2024-09-14T06:29:20.511 INFO:teuthology.orchestra.run.smithi094.stdout:truncating to largest ever: 0xe86c0b7 2024-09-14T06:29:20.513 INFO:teuthology.orchestra.run.smithi094.stdout:506 trunc from 0xceabe58 to 0xe86c0b7 2024-09-14T06:29:20.518 INFO:teuthology.orchestra.run.smithi094.stdout:osd.3: {} 2024-09-14T06:29:20.519 INFO:teuthology.orchestra.run.smithi094.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-14T06:29:20.519 INFO:teuthology.orchestra.run.smithi094.stdout:507 read 0x3b4879e thru 0x3b51d29 (0x958c bytes) 2024-09-14T06:29:20.520 INFO:teuthology.orchestra.run.smithi094.stdout:508 punch from 0x670035a to 0x670bc58, (0xb8fe bytes) 2024-09-14T06:29:20.528 INFO:teuthology.orchestra.run.smithi094.stdout:510 trunc from 0xe86c0b7 to 0x92517eb 2024-09-14T06:29:20.534 INFO:teuthology.orchestra.run.smithi094.stdout:511 read 0x31f2b4d thru 0x31f3f6d (0x1421 bytes) 2024-09-14T06:29:20.537 INFO:teuthology.orchestra.run.smithi094.stdout:osd.4: {} 2024-09-14T06:29:20.537 INFO:teuthology.orchestra.run.smithi094.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-14T06:29:20.543 INFO:teuthology.orchestra.run.smithi094.stdout:512 write 0xb6f5831 thru 0xb703516 (0xdce6 bytes) 2024-09-14T06:29:20.547 INFO:teuthology.orchestra.run.smithi094.stdout:513 read 0x2960c59 thru 0x296af05 (0xa2ad bytes) 2024-09-14T06:29:20.547 INFO:teuthology.orchestra.run.smithi094.stdout:515 write 0x2c247f3 thru 0x2c2797e (0x318c bytes) 2024-09-14T06:29:20.548 INFO:teuthology.orchestra.run.smithi094.stdout:516 write 0x83cd120 thru 0x83cd30d (0x1ee bytes) 2024-09-14T06:29:20.551 INFO:teuthology.orchestra.run.smithi094.stdout:517 punch from 0x4f0399 to 0x4f1c67, (0x18ce bytes) 2024-09-14T06:29:20.553 INFO:teuthology.orchestra.run.smithi094.stdout:518 trunc from 0xb703517 to 0x9f91f86 2024-09-14T06:29:20.564 INFO:teuthology.orchestra.run.smithi094.stdout:519 trunc from 0x9f91f86 to 0x65d822f 2024-09-14T06:29:20.569 INFO:teuthology.orchestra.run.smithi094.stdout:osd.5: {} 2024-09-14T06:29:20.569 INFO:teuthology.orchestra.run.smithi094.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-14T06:29:20.584 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-14T06:29:20.584 INFO:teuthology.orchestra.run.smithi094.stdout: "ops": [], 2024-09-14T06:29:20.584 INFO:teuthology.orchestra.run.smithi094.stdout: "num_ops": 0 2024-09-14T06:29:20.584 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-14T06:29:20.591 INFO:teuthology.orchestra.run.smithi094.stdout:osd.6: {} 2024-09-14T06:29:20.591 INFO:teuthology.orchestra.run.smithi094.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-14T06:29:20.594 INFO:tasks.ceph.osd.0.smithi094.stderr:2024-09-14T06:29:20.593+0000 7f07c9333640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33636) UID: 0 2024-09-14T06:29:20.597 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-14T06:29:20.598 INFO:teuthology.orchestra.run.smithi094.stdout:520 writesame 0xe439617 thru 0xe43ad48 data_size 0x1(0x1732 bytes) 2024-09-14T06:29:20.598 INFO:teuthology.orchestra.run.smithi094.stdout:521 punch from 0x4887b49 to 0x48968b6, (0xed6d bytes) 2024-09-14T06:29:20.598 INFO:teuthology.orchestra.run.smithi094.stdout:truncating to largest ever: 0xea106a4 2024-09-14T06:29:20.598 INFO:teuthology.orchestra.run.smithi094.stdout:522 trunc from 0xe43ad49 to 0xea106a4 2024-09-14T06:29:20.603 INFO:teuthology.orchestra.run.smithi094.stdout:523 read 0xcd30b46 thru 0xcd3b4a8 (0xa963 bytes) 2024-09-14T06:29:20.603 INFO:teuthology.orchestra.run.smithi094.stdout:525 write 0x4bcc7a8 thru 0x4bd944f (0xcca8 bytes) 2024-09-14T06:29:20.605 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-14T06:29:20.605 INFO:teuthology.orchestra.run.smithi094.stdout:528 punch from 0x4e69c2 to 0x4f4f74, (0xe5b2 bytes) 2024-09-14T06:29:20.605 INFO:teuthology.orchestra.run.smithi094.stdout:530 clone 6 order 21 su 1048576 sc 3 2024-09-14T06:29:20.611 INFO:teuthology.orchestra.run.smithi094.stdout:osd.7: {} 2024-09-14T06:29:20.611 INFO:teuthology.orchestra.run.smithi094.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-14T06:29:20.659 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-14T06:29:20.696 INFO:tasks.ceph.osd.7.smithi138.stderr:2024-09-14T06:29:20.695+0000 7f3c6a21d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33623) UID: 0 2024-09-14T06:29:20.710 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-14T06:29:20.710 INFO:teuthology.orchestra.run.smithi094.stdout: "ops": [], 2024-09-14T06:29:20.710 INFO:teuthology.orchestra.run.smithi094.stdout: "complaint_time": 30, 2024-09-14T06:29:20.710 INFO:teuthology.orchestra.run.smithi094.stdout: "num_blocked_ops": 0 2024-09-14T06:29:20.710 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-14T06:29:20.719 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-14T06:29:20.741 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-14T06:29:20.741 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-14T06:29:20.741 INFO:tasks.thrashosds.thrasher:reset_purged_snaps_last 2024-09-14T06:29:20.741 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 reset_purged_snaps_last 2024-09-14T06:29:20.806 INFO:tasks.ceph.osd.0.smithi094.stderr:2024-09-14T06:29:20.805+0000 7f07c9333640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33636) UID: 0 2024-09-14T06:29:20.829 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-14T06:29:20.830 INFO:teuthology.orchestra.run.smithi094.stdout: "size": 20, 2024-09-14T06:29:20.830 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 600, 2024-09-14T06:29:20.830 INFO:teuthology.orchestra.run.smithi094.stdout: "ops": [ 2024-09-14T06:29:20.830 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:20.830 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:2175 3.6 3:637aae64:::rbd_data.112d7dfbd766.0000000000000032:5 [read 143360~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-14T06:29:20.830 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:08.676224+0000", 2024-09-14T06:29:20.830 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 12.150746967, 2024-09-14T06:29:20.830 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.015113018000000001, 2024-09-14T06:29:20.830 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:29:20.830 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:29:20.830 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:29:20.830 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:29:20.830 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:29:20.830 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:20.831 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 2175 2024-09-14T06:29:20.831 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:20.831 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:29:20.831 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:20.831 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:29:20.831 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.676224+0000", 2024-09-14T06:29:20.831 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:20.831 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:20.831 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:20.831 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:29:20.831 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.676224+0000", 2024-09-14T06:29:20.831 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:20.831 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:20.831 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:20.831 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:29:20.832 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.676224+0000", 2024-09-14T06:29:20.832 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.6100000000000002e-07 2024-09-14T06:29:20.832 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:20.832 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:20.832 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:29:20.832 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.676226+0000", 2024-09-14T06:29:20.832 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.64e-06 2024-09-14T06:29:20.832 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:20.832 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:20.832 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:29:20.832 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.676232+0000", 2024-09-14T06:29:20.833 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.0820000000000004e-06 2024-09-14T06:29:20.833 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:20.833 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:20.833 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:29:20.833 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.676237+0000", 2024-09-14T06:29:20.833 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.7870000000000001e-06 2024-09-14T06:29:20.833 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:20.833 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:20.833 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:29:20.833 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.691140+0000", 2024-09-14T06:29:20.833 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.01490286 2024-09-14T06:29:20.833 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:20.833 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:20.833 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:29:20.833 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.691235+0000", 2024-09-14T06:29:20.834 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.5437000000000001e-05 2024-09-14T06:29:20.834 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:20.834 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:20.834 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:29:20.834 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.691337+0000", 2024-09-14T06:29:20.834 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000101851 2024-09-14T06:29:20.834 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:20.834 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:20.834 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:20.834 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:20.834 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:20.834 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:2176 3.6 3:637aae64:::rbd_data.112d7dfbd766.0000000000000032:5 [read 159744~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-14T06:29:20.834 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:08.676243+0000", 2024-09-14T06:29:20.834 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 12.150728122, 2024-09-14T06:29:20.834 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.015261653999999999, 2024-09-14T06:29:20.835 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:29:20.835 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:29:20.835 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:29:20.835 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:29:20.835 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:29:20.835 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:20.835 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 2176 2024-09-14T06:29:20.835 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:20.835 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:29:20.835 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:20.835 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:29:20.835 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.676243+0000", 2024-09-14T06:29:20.835 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:20.835 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:20.836 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:20.836 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:29:20.836 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.676243+0000", 2024-09-14T06:29:20.836 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:20.836 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:20.836 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:20.836 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:29:20.836 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.676243+0000", 2024-09-14T06:29:20.836 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.8799999999999998e-07 2024-09-14T06:29:20.836 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:20.836 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:20.836 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:29:20.836 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.676245+0000", 2024-09-14T06:29:20.836 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.9470000000000002e-06 2024-09-14T06:29:20.837 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:20.837 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:20.837 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:29:20.837 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.676251+0000", 2024-09-14T06:29:20.837 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.1759999999999998e-06 2024-09-14T06:29:20.837 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:20.837 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:20.837 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:29:20.837 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.676256+0000", 2024-09-14T06:29:20.837 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.5260000000000004e-06 2024-09-14T06:29:20.837 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:20.837 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:20.837 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:29:20.837 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.691388+0000", 2024-09-14T06:29:20.837 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.015132563 2024-09-14T06:29:20.837 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:20.838 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:20.838 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:29:20.838 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.691432+0000", 2024-09-14T06:29:20.838 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.3197e-05 2024-09-14T06:29:20.838 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:20.838 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:20.838 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:29:20.838 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.691504+0000", 2024-09-14T06:29:20.838 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.2856999999999997e-05 2024-09-14T06:29:20.838 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:20.838 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:20.838 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:20.838 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:20.838 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:20.838 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:2177 3.6 3:637aae64:::rbd_data.112d7dfbd766.0000000000000032:5 [read 176128~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-14T06:29:20.839 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:08.676262+0000", 2024-09-14T06:29:20.839 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 12.150709032, 2024-09-14T06:29:20.839 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.015436462, 2024-09-14T06:29:20.839 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:29:20.839 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:29:20.839 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:29:20.839 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:29:20.839 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:29:20.839 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:20.839 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 2177 2024-09-14T06:29:20.839 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:20.839 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:29:20.839 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:20.840 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:29:20.840 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.676262+0000", 2024-09-14T06:29:20.840 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:20.840 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:20.840 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:20.840 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:29:20.840 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.676262+0000", 2024-09-14T06:29:20.840 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:20.840 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:20.840 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:20.840 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:29:20.840 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.676262+0000", 2024-09-14T06:29:20.840 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.5600000000000001e-07 2024-09-14T06:29:20.841 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:20.841 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:20.841 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:29:20.841 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.676264+0000", 2024-09-14T06:29:20.841 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.8929999999999999e-06 2024-09-14T06:29:20.841 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:20.841 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:20.841 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:29:20.841 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.676272+0000", 2024-09-14T06:29:20.841 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.3599999999999996e-06 2024-09-14T06:29:20.841 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:20.841 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:20.841 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:29:20.842 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.676277+0000", 2024-09-14T06:29:20.842 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.899e-06 2024-09-14T06:29:20.842 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:20.842 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:20.842 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:29:20.842 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.691533+0000", 2024-09-14T06:29:20.842 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.015255258000000001 2024-09-14T06:29:20.842 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:20.842 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:20.842 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:29:20.842 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.691600+0000", 2024-09-14T06:29:20.842 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.7361999999999996e-05 2024-09-14T06:29:20.842 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:20.842 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:20.842 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:29:20.843 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.691698+0000", 2024-09-14T06:29:20.843 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.8333999999999997e-05 2024-09-14T06:29:20.843 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:20.843 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:20.843 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:20.843 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:20.843 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:20.843 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:2178 3.6 3:637aae64:::rbd_data.112d7dfbd766.0000000000000032:5 [read 192512~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-14T06:29:20.843 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:08.676284+0000", 2024-09-14T06:29:20.843 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 12.150687320999999, 2024-09-14T06:29:20.843 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.015609618, 2024-09-14T06:29:20.843 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:29:20.843 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:29:20.843 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:29:20.843 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:29:20.843 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:29:20.844 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:20.844 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 2178 2024-09-14T06:29:20.844 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:20.844 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:29:20.844 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:20.844 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:29:20.844 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.676284+0000", 2024-09-14T06:29:20.844 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:20.844 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:20.844 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:20.844 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:29:20.844 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.676284+0000", 2024-09-14T06:29:20.844 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:20.844 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:20.844 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:20.844 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:29:20.845 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.676284+0000", 2024-09-14T06:29:20.845 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.9000000000000002e-07 2024-09-14T06:29:20.845 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:20.845 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:20.845 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:29:20.845 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.676287+0000", 2024-09-14T06:29:20.845 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.0359999999999998e-06 2024-09-14T06:29:20.845 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:20.845 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:20.845 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:29:20.845 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.676293+0000", 2024-09-14T06:29:20.845 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.8470000000000001e-06 2024-09-14T06:29:20.845 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:20.845 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:20.845 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:29:20.845 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.676298+0000", 2024-09-14T06:29:20.846 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.3009999999999997e-06 2024-09-14T06:29:20.846 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:20.846 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:20.846 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:29:20.846 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.691730+0000", 2024-09-14T06:29:20.846 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.015431706 2024-09-14T06:29:20.846 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:20.846 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:20.846 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:29:20.846 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.691813+0000", 2024-09-14T06:29:20.846 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.2738999999999995e-05 2024-09-14T06:29:20.846 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:20.846 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:20.846 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:29:20.846 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.691893+0000", 2024-09-14T06:29:20.846 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.0599000000000005e-05 2024-09-14T06:29:20.847 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:20.847 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:20.847 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:20.847 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:20.847 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:20.847 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:2179 3.6 3:637aae64:::rbd_data.112d7dfbd766.0000000000000032:5 [read 208896~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-14T06:29:20.847 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:08.676324+0000", 2024-09-14T06:29:20.847 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 12.150646796, 2024-09-14T06:29:20.847 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.015825689, 2024-09-14T06:29:20.847 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:29:20.847 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:29:20.847 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:29:20.847 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:29:20.847 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:29:20.847 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:20.847 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 2179 2024-09-14T06:29:20.848 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:20.848 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:29:20.848 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:20.848 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:29:20.848 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.676324+0000", 2024-09-14T06:29:20.848 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:20.848 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:20.848 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:20.848 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:29:20.848 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.676324+0000", 2024-09-14T06:29:20.848 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:20.848 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:20.848 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:20.848 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:29:20.848 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.676325+0000", 2024-09-14T06:29:20.848 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.2099999999999996e-07 2024-09-14T06:29:20.849 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:20.849 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:20.849 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:29:20.849 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.676327+0000", 2024-09-14T06:29:20.849 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.3309999999999998e-06 2024-09-14T06:29:20.849 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:20.849 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:20.849 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:29:20.849 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.676334+0000", 2024-09-14T06:29:20.849 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.7819999999999996e-06 2024-09-14T06:29:20.849 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:20.849 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:20.849 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:29:20.849 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.676339+0000", 2024-09-14T06:29:20.849 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.6760000000000004e-06 2024-09-14T06:29:20.850 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:20.850 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:20.850 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:29:20.850 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.691922+0000", 2024-09-14T06:29:20.850 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.015582554 2024-09-14T06:29:20.850 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:20.850 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:20.850 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:29:20.850 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.691988+0000", 2024-09-14T06:29:20.850 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.5568999999999995e-05 2024-09-14T06:29:20.850 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:20.850 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:20.850 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:29:20.850 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.692150+0000", 2024-09-14T06:29:20.850 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000162156 2024-09-14T06:29:20.851 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:20.851 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:20.851 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:20.851 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:20.851 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:20.851 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:2180 3.6 3:637aae64:::rbd_data.112d7dfbd766.0000000000000032:5 [read 225280~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-14T06:29:20.851 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:08.676346+0000", 2024-09-14T06:29:20.851 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 12.150624444, 2024-09-14T06:29:20.851 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.016174806999999999, 2024-09-14T06:29:20.851 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:29:20.851 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:29:20.851 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:29:20.851 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:29:20.851 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:29:20.852 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:20.852 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 2180 2024-09-14T06:29:20.852 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:20.852 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:29:20.852 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:20.852 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:29:20.852 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.676346+0000", 2024-09-14T06:29:20.852 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:20.852 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:20.852 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:20.852 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:29:20.852 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.676346+0000", 2024-09-14T06:29:20.852 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:20.852 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:20.852 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:20.853 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:29:20.853 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.676347+0000", 2024-09-14T06:29:20.853 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.9700000000000002e-07 2024-09-14T06:29:20.853 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:20.853 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:20.853 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:29:20.853 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.676349+0000", 2024-09-14T06:29:20.853 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.142e-06 2024-09-14T06:29:20.853 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:20.853 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:20.853 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:29:20.853 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.676355+0000", 2024-09-14T06:29:20.853 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.1929999999999998e-06 2024-09-14T06:29:20.854 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:20.854 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:20.854 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:29:20.854 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.676362+0000", 2024-09-14T06:29:20.854 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.3089999999999998e-06 2024-09-14T06:29:20.854 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:20.854 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:20.854 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:29:20.854 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.692187+0000", 2024-09-14T06:29:20.854 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.015824530999999999 2024-09-14T06:29:20.854 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:20.854 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:20.854 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:29:20.854 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.692287+0000", 2024-09-14T06:29:20.854 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.9884999999999997e-05 2024-09-14T06:29:20.855 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:20.855 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:20.855 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:29:20.855 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.692521+0000", 2024-09-14T06:29:20.855 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00023435 2024-09-14T06:29:20.855 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:20.855 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:20.855 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:20.855 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:20.855 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:20.855 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:3743 3.1f 3:fa5dbf8a:::rbd_data.112d7dfbd766.000000000000018d:5 [read 303104~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-14T06:29:20.855 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:08.774918+0000", 2024-09-14T06:29:20.855 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 12.052052847000001, 2024-09-14T06:29:20.855 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.015121839, 2024-09-14T06:29:20.856 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:29:20.856 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:29:20.856 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:29:20.856 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:29:20.856 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:29:20.856 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:20.856 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 3743 2024-09-14T06:29:20.856 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:20.856 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:29:20.856 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:20.856 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:29:20.856 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.774918+0000", 2024-09-14T06:29:20.856 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:20.856 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:20.856 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:20.857 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:29:20.857 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.774918+0000", 2024-09-14T06:29:20.857 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:20.857 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:20.857 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:20.857 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:29:20.857 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.774918+0000", 2024-09-14T06:29:20.857 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.11e-07 2024-09-14T06:29:20.857 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:20.857 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:20.857 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:29:20.857 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.774919+0000", 2024-09-14T06:29:20.857 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.7100000000000011e-07 2024-09-14T06:29:20.857 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:20.857 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:20.857 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:29:20.858 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.774923+0000", 2024-09-14T06:29:20.858 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.625e-06 2024-09-14T06:29:20.858 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:20.858 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:20.858 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:29:20.858 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.774925+0000", 2024-09-14T06:29:20.858 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.6220000000000001e-06 2024-09-14T06:29:20.858 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:20.858 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:20.858 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:29:20.858 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.789865+0000", 2024-09-14T06:29:20.858 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.014939233999999999 2024-09-14T06:29:20.858 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:20.858 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:20.858 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:29:20.858 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.789939+0000", 2024-09-14T06:29:20.859 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.3848000000000002e-05 2024-09-14T06:29:20.859 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:20.859 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:20.859 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:29:20.859 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.790040+0000", 2024-09-14T06:29:20.859 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000101328 2024-09-14T06:29:20.859 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:20.859 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:20.859 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:20.859 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:20.859 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:20.859 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:3744 3.1f 3:fa5dbf8a:::rbd_data.112d7dfbd766.000000000000018d:5 [read 319488~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-14T06:29:20.859 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:08.774929+0000", 2024-09-14T06:29:20.859 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 12.052042255, 2024-09-14T06:29:20.859 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.015307502000000001, 2024-09-14T06:29:20.859 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:29:20.860 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:29:20.860 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:29:20.860 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:29:20.860 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:29:20.860 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:20.860 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 3744 2024-09-14T06:29:20.860 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:20.860 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:29:20.860 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:20.860 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:29:20.860 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.774929+0000", 2024-09-14T06:29:20.860 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:20.861 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:20.861 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:20.861 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:29:20.861 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.774929+0000", 2024-09-14T06:29:20.861 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:20.861 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:20.861 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:20.861 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:29:20.861 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.774929+0000", 2024-09-14T06:29:20.861 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.0800000000000001e-07 2024-09-14T06:29:20.861 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:20.862 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:20.862 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:29:20.862 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.774930+0000", 2024-09-14T06:29:20.862 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.482e-06 2024-09-14T06:29:20.862 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:20.862 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:20.862 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:29:20.862 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.774934+0000", 2024-09-14T06:29:20.862 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.8489999999999999e-06 2024-09-14T06:29:20.862 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:20.862 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:20.862 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:29:20.862 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.774937+0000", 2024-09-14T06:29:20.862 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.5799999999999999e-06 2024-09-14T06:29:20.863 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:20.863 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:20.863 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:29:20.863 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.790069+0000", 2024-09-14T06:29:20.863 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.015132546 2024-09-14T06:29:20.863 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:20.863 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:20.863 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:29:20.863 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.790149+0000", 2024-09-14T06:29:20.863 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.9374000000000003e-05 2024-09-14T06:29:20.863 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:20.863 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:20.863 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:29:20.864 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.790236+0000", 2024-09-14T06:29:20.864 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.7462999999999999e-05 2024-09-14T06:29:20.864 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:20.864 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:20.864 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:20.864 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:20.864 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:20.864 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:3745 3.1f 3:fa5dbf8a:::rbd_data.112d7dfbd766.000000000000018d:5 [read 335872~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-14T06:29:20.864 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:08.774940+0000", 2024-09-14T06:29:20.864 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 12.052031026, 2024-09-14T06:29:20.864 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.015484926, 2024-09-14T06:29:20.864 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:29:20.864 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:29:20.865 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:29:20.865 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:29:20.865 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:29:20.865 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:20.865 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 3745 2024-09-14T06:29:20.865 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:20.865 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:29:20.865 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:20.865 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:29:20.865 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.774940+0000", 2024-09-14T06:29:20.865 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:20.865 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:20.865 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:20.866 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:29:20.866 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.774940+0000", 2024-09-14T06:29:20.866 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:20.866 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:20.866 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:20.866 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:29:20.866 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.774940+0000", 2024-09-14T06:29:20.866 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.11e-07 2024-09-14T06:29:20.866 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:20.866 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:20.866 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:29:20.866 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.774941+0000", 2024-09-14T06:29:20.866 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.0500000000000002e-07 2024-09-14T06:29:20.866 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:20.867 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:20.867 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:29:20.867 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.774945+0000", 2024-09-14T06:29:20.867 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.7400000000000002e-06 2024-09-14T06:29:20.867 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:20.867 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:20.867 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:29:20.867 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.774948+0000", 2024-09-14T06:29:20.867 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.0740000000000001e-06 2024-09-14T06:29:20.867 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:20.867 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:20.867 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:29:20.867 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.790264+0000", 2024-09-14T06:29:20.867 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.015316553 2024-09-14T06:29:20.868 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:20.868 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:20.868 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:29:20.868 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.790336+0000", 2024-09-14T06:29:20.868 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.1722000000000002e-05 2024-09-14T06:29:20.868 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:20.868 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:20.868 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:29:20.868 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.790425+0000", 2024-09-14T06:29:20.868 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.8721000000000007e-05 2024-09-14T06:29:20.868 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:20.868 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:20.868 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:20.868 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:20.868 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:20.869 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:3746 3.1f 3:fa5dbf8a:::rbd_data.112d7dfbd766.000000000000018d:5 [read 352256~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-14T06:29:20.869 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:08.774952+0000", 2024-09-14T06:29:20.869 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 12.052018611999999, 2024-09-14T06:29:20.869 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.015662054000000002, 2024-09-14T06:29:20.869 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:29:20.869 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:29:20.869 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:29:20.869 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:29:20.869 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:29:20.869 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:20.869 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 3746 2024-09-14T06:29:20.869 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:20.869 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:29:20.869 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:20.869 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:29:20.870 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.774952+0000", 2024-09-14T06:29:20.870 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:20.870 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:20.870 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:20.870 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:29:20.870 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.774952+0000", 2024-09-14T06:29:20.870 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:20.870 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:20.870 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:20.870 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:29:20.870 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.774952+0000", 2024-09-14T06:29:20.870 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.1299999999999999e-07 2024-09-14T06:29:20.870 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:20.871 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:20.871 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:29:20.871 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.774953+0000", 2024-09-14T06:29:20.871 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.8100000000000001e-07 2024-09-14T06:29:20.871 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:20.871 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:20.871 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:29:20.871 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.774957+0000", 2024-09-14T06:29:20.871 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.8060000000000001e-06 2024-09-14T06:29:20.871 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:20.871 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:20.871 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:29:20.871 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.774960+0000", 2024-09-14T06:29:20.871 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.633e-06 2024-09-14T06:29:20.871 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:20.872 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:20.872 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:29:20.872 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.790454+0000", 2024-09-14T06:29:20.872 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.015494588 2024-09-14T06:29:20.872 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:20.872 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:20.872 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:29:20.872 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.790525+0000", 2024-09-14T06:29:20.872 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.0869000000000002e-05 2024-09-14T06:29:20.872 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:20.872 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:20.872 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:29:20.872 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.790614+0000", 2024-09-14T06:29:20.872 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.8963999999999997e-05 2024-09-14T06:29:20.873 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:20.873 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:20.873 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:20.873 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:20.873 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:20.873 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:3747 3.1f 3:fa5dbf8a:::rbd_data.112d7dfbd766.000000000000018d:5 [read 368640~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-14T06:29:20.873 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:08.774964+0000", 2024-09-14T06:29:20.873 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 12.052006591, 2024-09-14T06:29:20.873 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.016009292000000001, 2024-09-14T06:29:20.873 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:29:20.873 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:29:20.873 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:29:20.873 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:29:20.874 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:29:20.874 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:20.874 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 3747 2024-09-14T06:29:20.874 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:20.874 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:29:20.874 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:20.874 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:29:20.874 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.774964+0000", 2024-09-14T06:29:20.874 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:20.874 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:20.874 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:20.874 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:29:20.874 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.774964+0000", 2024-09-14T06:29:20.874 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:20.875 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:20.875 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:20.875 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:29:20.875 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.774965+0000", 2024-09-14T06:29:20.875 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.11e-07 2024-09-14T06:29:20.875 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:20.875 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:20.875 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:29:20.875 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.774966+0000", 2024-09-14T06:29:20.875 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.1349999999999999e-06 2024-09-14T06:29:20.875 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:20.875 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:20.875 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:29:20.875 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.774969+0000", 2024-09-14T06:29:20.876 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.755e-06 2024-09-14T06:29:20.876 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:20.876 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:20.876 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:29:20.876 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.774972+0000", 2024-09-14T06:29:20.876 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.4420000000000001e-06 2024-09-14T06:29:20.876 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:20.876 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:20.876 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:29:20.876 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.790816+0000", 2024-09-14T06:29:20.876 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.015843768000000001 2024-09-14T06:29:20.876 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:20.876 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:20.876 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:29:20.877 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.790885+0000", 2024-09-14T06:29:20.877 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.9080999999999998e-05 2024-09-14T06:29:20.877 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:20.877 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:20.877 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:29:20.877 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.790974+0000", 2024-09-14T06:29:20.877 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.8900000000000006e-05 2024-09-14T06:29:20.877 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:20.877 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:20.877 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:20.877 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:20.877 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:20.877 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:3748 3.1f 3:fa5dbf8a:::rbd_data.112d7dfbd766.000000000000018d:5 [read 385024~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-14T06:29:20.877 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:08.774976+0000", 2024-09-14T06:29:20.877 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 12.051994905999999, 2024-09-14T06:29:20.878 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.016156594999999999, 2024-09-14T06:29:20.878 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:29:20.878 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:29:20.878 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:29:20.878 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:29:20.878 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:29:20.878 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:20.878 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 3748 2024-09-14T06:29:20.878 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:20.878 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:29:20.878 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:20.878 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:29:20.878 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.774976+0000", 2024-09-14T06:29:20.878 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:20.878 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:20.879 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:20.879 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:29:20.879 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.774976+0000", 2024-09-14T06:29:20.879 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:20.879 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:20.879 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:20.879 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:29:20.879 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.774976+0000", 2024-09-14T06:29:20.879 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.1799999999999999e-07 2024-09-14T06:29:20.879 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:20.879 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:20.879 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:29:20.879 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.774977+0000", 2024-09-14T06:29:20.879 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.0279999999999999e-06 2024-09-14T06:29:20.879 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:20.879 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:20.880 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:29:20.880 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.774981+0000", 2024-09-14T06:29:20.880 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.7179999999999998e-06 2024-09-14T06:29:20.880 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:20.880 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:20.880 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:29:20.880 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.774984+0000", 2024-09-14T06:29:20.880 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.7530000000000002e-06 2024-09-14T06:29:20.880 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:20.880 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:20.880 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:29:20.880 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.791013+0000", 2024-09-14T06:29:20.880 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.016029024999999999 2024-09-14T06:29:20.880 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:20.881 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:20.881 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:29:20.881 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.791062+0000", 2024-09-14T06:29:20.881 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.8902e-05 2024-09-14T06:29:20.881 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:20.881 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:20.881 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:29:20.881 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.791133+0000", 2024-09-14T06:29:20.881 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.0951000000000006e-05 2024-09-14T06:29:20.881 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:20.881 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:20.881 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:20.881 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:20.881 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:20.882 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:3749 3.1f 3:fa5dbf8a:::rbd_data.112d7dfbd766.000000000000018d:5 [read 401408~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-14T06:29:20.882 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:08.774987+0000", 2024-09-14T06:29:20.882 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 12.051984150999999, 2024-09-14T06:29:20.882 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.016330092000000001, 2024-09-14T06:29:20.882 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:29:20.882 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:29:20.882 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:29:20.882 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:29:20.882 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:29:20.882 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:20.882 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 3749 2024-09-14T06:29:20.882 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:20.882 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:29:20.882 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:20.882 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:29:20.883 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.774987+0000", 2024-09-14T06:29:20.883 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:20.883 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:20.883 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:20.883 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:29:20.883 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.774987+0000", 2024-09-14T06:29:20.883 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:20.883 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:20.883 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:20.883 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:29:20.883 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.774987+0000", 2024-09-14T06:29:20.883 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.0599999999999999e-07 2024-09-14T06:29:20.883 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:20.883 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:20.883 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:29:20.884 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.774988+0000", 2024-09-14T06:29:20.884 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.7900000000000007e-07 2024-09-14T06:29:20.884 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:20.884 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:20.884 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:29:20.884 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.774992+0000", 2024-09-14T06:29:20.884 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.6540000000000001e-06 2024-09-14T06:29:20.884 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:20.884 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:20.884 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:29:20.884 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.774994+0000", 2024-09-14T06:29:20.884 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.5950000000000001e-06 2024-09-14T06:29:20.884 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:20.884 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:20.884 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:29:20.885 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.791166+0000", 2024-09-14T06:29:20.885 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.016172083 2024-09-14T06:29:20.885 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:20.885 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:20.885 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:29:20.885 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.791221+0000", 2024-09-14T06:29:20.885 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.4843000000000003e-05 2024-09-14T06:29:20.885 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:20.885 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:20.885 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:29:20.885 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.791317+0000", 2024-09-14T06:29:20.885 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.5731999999999998e-05 2024-09-14T06:29:20.885 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:20.885 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:20.886 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:20.886 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:20.886 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:20.886 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:3750 3.1f 3:fa5dbf8a:::rbd_data.112d7dfbd766.000000000000018d:5 [read 417792~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-14T06:29:20.886 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:08.774997+0000", 2024-09-14T06:29:20.886 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 12.051973792, 2024-09-14T06:29:20.886 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.016504642, 2024-09-14T06:29:20.886 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:29:20.886 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:29:20.886 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:29:20.886 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:29:20.886 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:29:20.886 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:20.886 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 3750 2024-09-14T06:29:20.886 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:20.887 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:29:20.887 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:20.887 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:29:20.887 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.774997+0000", 2024-09-14T06:29:20.887 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:20.887 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:20.887 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:20.887 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:29:20.887 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.774997+0000", 2024-09-14T06:29:20.887 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:20.887 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:20.887 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:20.887 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:29:20.887 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.774997+0000", 2024-09-14T06:29:20.888 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.1500000000000001e-07 2024-09-14T06:29:20.888 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:20.888 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:20.888 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:29:20.888 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.774998+0000", 2024-09-14T06:29:20.888 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.6599999999999994e-07 2024-09-14T06:29:20.888 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:20.888 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:20.888 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:29:20.888 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.775002+0000", 2024-09-14T06:29:20.888 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.659e-06 2024-09-14T06:29:20.888 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:20.888 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:20.888 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:29:20.889 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.775005+0000", 2024-09-14T06:29:20.889 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.6299999999999998e-06 2024-09-14T06:29:20.889 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:20.889 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:20.889 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:29:20.889 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.791363+0000", 2024-09-14T06:29:20.889 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.016358707 2024-09-14T06:29:20.889 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:20.889 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:20.889 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:29:20.889 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.791421+0000", 2024-09-14T06:29:20.889 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.7816999999999999e-05 2024-09-14T06:29:20.889 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:20.889 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:20.890 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:29:20.890 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.791502+0000", 2024-09-14T06:29:20.890 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.0648000000000004e-05 2024-09-14T06:29:20.890 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:20.890 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:20.890 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:20.890 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:20.890 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:20.890 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:3751 3.1f 3:fa5dbf8a:::rbd_data.112d7dfbd766.000000000000018d:5 [read 434176~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-14T06:29:20.890 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:08.775010+0000", 2024-09-14T06:29:20.890 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 12.051961056, 2024-09-14T06:29:20.890 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.016656712000000001, 2024-09-14T06:29:20.890 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:29:20.890 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:29:20.890 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:29:20.891 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:29:20.891 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:29:20.891 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:20.891 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 3751 2024-09-14T06:29:20.891 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:20.891 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:29:20.891 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:20.891 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:29:20.891 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.775010+0000", 2024-09-14T06:29:20.891 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:20.891 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:20.891 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:20.891 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:29:20.891 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.775010+0000", 2024-09-14T06:29:20.891 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:20.892 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:20.892 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:20.892 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:29:20.892 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.775010+0000", 2024-09-14T06:29:20.892 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.0800000000000001e-07 2024-09-14T06:29:20.892 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:20.892 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:20.892 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:29:20.892 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.775012+0000", 2024-09-14T06:29:20.892 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.528e-06 2024-09-14T06:29:20.892 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:20.892 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:20.892 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:29:20.892 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.775015+0000", 2024-09-14T06:29:20.893 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.8709999999999999e-06 2024-09-14T06:29:20.893 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:20.893 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:20.893 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:29:20.893 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.775018+0000", 2024-09-14T06:29:20.893 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.57e-06 2024-09-14T06:29:20.893 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:20.893 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:20.893 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:29:20.893 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.791547+0000", 2024-09-14T06:29:20.893 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.016529419 2024-09-14T06:29:20.893 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:20.893 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:20.893 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:29:20.894 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.791594+0000", 2024-09-14T06:29:20.894 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.6159000000000003e-05 2024-09-14T06:29:20.894 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:20.894 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:20.894 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:29:20.894 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.791667+0000", 2024-09-14T06:29:20.894 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.2957e-05 2024-09-14T06:29:20.894 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:20.894 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:20.894 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:20.894 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:20.894 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:20.894 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:3752 3.1f 3:fa5dbf8a:::rbd_data.112d7dfbd766.000000000000018d:5 [read 450560~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-14T06:29:20.894 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:08.775021+0000", 2024-09-14T06:29:20.894 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 12.051949762, 2024-09-14T06:29:20.895 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.016807249, 2024-09-14T06:29:20.895 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:29:20.895 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:29:20.895 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:29:20.895 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:29:20.895 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:29:20.895 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:20.895 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 3752 2024-09-14T06:29:20.895 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:20.895 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:29:20.895 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:20.895 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:29:20.895 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.775021+0000", 2024-09-14T06:29:20.895 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:20.895 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:20.895 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:20.896 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:29:20.896 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.775021+0000", 2024-09-14T06:29:20.896 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:20.896 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:20.896 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:20.896 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:29:20.896 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.775021+0000", 2024-09-14T06:29:20.896 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.11e-07 2024-09-14T06:29:20.896 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:20.896 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:20.896 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:29:20.896 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.775022+0000", 2024-09-14T06:29:20.896 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.64e-07 2024-09-14T06:29:20.896 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:20.896 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:20.897 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:29:20.897 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.775026+0000", 2024-09-14T06:29:20.897 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.7869999999999999e-06 2024-09-14T06:29:20.897 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:20.897 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:20.897 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:29:20.897 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.775029+0000", 2024-09-14T06:29:20.897 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.5330000000000001e-06 2024-09-14T06:29:20.897 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:20.897 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:20.897 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:29:20.897 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.791706+0000", 2024-09-14T06:29:20.897 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.016677208999999998 2024-09-14T06:29:20.897 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:20.897 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:20.897 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:29:20.898 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.791753+0000", 2024-09-14T06:29:20.898 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.7655999999999999e-05 2024-09-14T06:29:20.898 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:20.898 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:20.898 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:29:20.898 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.791828+0000", 2024-09-14T06:29:20.898 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.4888999999999994e-05 2024-09-14T06:29:20.898 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:20.898 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:20.898 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:20.898 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:20.898 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:20.898 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:3753 3.1f 3:fa5dbf8a:::rbd_data.112d7dfbd766.000000000000018d:5 [read 466944~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-14T06:29:20.898 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:08.775032+0000", 2024-09-14T06:29:20.898 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 12.051939187, 2024-09-14T06:29:20.898 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.016938284000000001, 2024-09-14T06:29:20.899 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:29:20.899 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:29:20.899 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:29:20.899 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:29:20.899 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:29:20.899 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:20.899 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 3753 2024-09-14T06:29:20.899 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:20.899 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:29:20.899 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:20.899 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:29:20.899 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.775032+0000", 2024-09-14T06:29:20.899 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:20.899 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:20.900 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:20.900 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:29:20.900 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.775032+0000", 2024-09-14T06:29:20.900 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:20.900 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:20.900 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:20.900 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:29:20.900 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.775032+0000", 2024-09-14T06:29:20.900 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.0699999999999999e-07 2024-09-14T06:29:20.900 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:20.900 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:20.900 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:29:20.900 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.775033+0000", 2024-09-14T06:29:20.900 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.47e-07 2024-09-14T06:29:20.901 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:20.901 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:20.901 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:29:20.901 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.775036+0000", 2024-09-14T06:29:20.901 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.6349999999999999e-06 2024-09-14T06:29:20.901 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:20.901 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:20.901 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:29:20.901 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.775039+0000", 2024-09-14T06:29:20.901 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.571e-06 2024-09-14T06:29:20.901 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:20.901 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:20.901 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:29:20.901 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.791863+0000", 2024-09-14T06:29:20.901 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.016824163999999999 2024-09-14T06:29:20.902 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:20.902 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:20.902 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:29:20.902 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.791907+0000", 2024-09-14T06:29:20.902 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.3538000000000002e-05 2024-09-14T06:29:20.902 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:20.902 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:20.902 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:29:20.902 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.791970+0000", 2024-09-14T06:29:20.902 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.3221999999999999e-05 2024-09-14T06:29:20.902 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:20.902 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:20.902 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:20.903 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:20.903 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:20.903 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:3754 3.1f 3:fa5dbf8a:::rbd_data.112d7dfbd766.000000000000018d:5 [read 483328~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-14T06:29:20.903 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:08.775042+0000", 2024-09-14T06:29:20.903 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 12.051928800000001, 2024-09-14T06:29:20.903 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.017038721, 2024-09-14T06:29:20.903 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:29:20.903 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:29:20.903 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:29:20.903 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:29:20.903 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:29:20.903 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:20.903 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 3754 2024-09-14T06:29:20.903 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:20.903 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:29:20.903 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:20.904 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:29:20.904 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.775042+0000", 2024-09-14T06:29:20.904 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:20.904 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:20.904 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:20.904 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:29:20.904 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.775042+0000", 2024-09-14T06:29:20.904 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:20.904 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:20.904 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:20.904 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:29:20.904 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.775042+0000", 2024-09-14T06:29:20.904 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.1299999999999999e-07 2024-09-14T06:29:20.904 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:20.905 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:20.905 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:29:20.905 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.775050+0000", 2024-09-14T06:29:20.905 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.0350000000000001e-06 2024-09-14T06:29:20.905 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:20.905 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:20.905 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:29:20.905 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.775055+0000", 2024-09-14T06:29:20.905 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.34e-06 2024-09-14T06:29:20.905 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:20.905 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:20.905 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:29:20.905 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.775057+0000", 2024-09-14T06:29:20.905 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.6800000000000002e-06 2024-09-14T06:29:20.906 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:20.906 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:20.906 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:29:20.906 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.791988+0000", 2024-09-14T06:29:20.906 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.016930776000000002 2024-09-14T06:29:20.906 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:20.906 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:20.906 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:29:20.906 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.792029+0000", 2024-09-14T06:29:20.906 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.0439999999999999e-05 2024-09-14T06:29:20.906 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:20.906 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:20.906 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:29:20.906 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.792081+0000", 2024-09-14T06:29:20.907 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.2237000000000001e-05 2024-09-14T06:29:20.907 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:20.907 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:20.907 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:20.907 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:20.907 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:20.907 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:3755 3.1f 3:fa5dbf8a:::rbd_data.112d7dfbd766.000000000000018d:5 [read 499712~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-14T06:29:20.907 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:08.775063+0000", 2024-09-14T06:29:20.907 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 12.051907727, 2024-09-14T06:29:20.907 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.017183832999999999, 2024-09-14T06:29:20.907 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:29:20.907 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:29:20.907 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:29:20.907 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:29:20.907 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:29:20.907 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:20.908 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 3755 2024-09-14T06:29:20.908 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:20.908 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:29:20.908 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:20.908 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:29:20.908 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.775063+0000", 2024-09-14T06:29:20.908 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:20.908 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:20.908 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:20.908 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:29:20.908 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.775063+0000", 2024-09-14T06:29:20.908 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:20.908 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:20.908 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:20.908 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:29:20.908 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.775063+0000", 2024-09-14T06:29:20.909 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.8000000000000002e-07 2024-09-14T06:29:20.909 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:20.909 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:20.909 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:29:20.909 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.775065+0000", 2024-09-14T06:29:20.909 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.1480000000000001e-06 2024-09-14T06:29:20.909 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:20.909 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:20.909 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:29:20.909 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.775068+0000", 2024-09-14T06:29:20.909 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.811e-06 2024-09-14T06:29:20.909 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:20.909 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:20.909 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:29:20.909 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.775071+0000", 2024-09-14T06:29:20.910 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.5900000000000002e-06 2024-09-14T06:29:20.910 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:20.910 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:20.910 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:29:20.910 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.792098+0000", 2024-09-14T06:29:20.910 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.017027490999999999 2024-09-14T06:29:20.910 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:20.910 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:20.910 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:29:20.910 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.792138+0000", 2024-09-14T06:29:20.910 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.9999000000000001e-05 2024-09-14T06:29:20.910 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:20.910 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:20.910 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:29:20.911 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.792247+0000", 2024-09-14T06:29:20.911 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000108514 2024-09-14T06:29:20.911 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:20.911 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:20.911 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:20.911 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:20.911 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:20.911 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:3756 3.1f 3:fa5dbf8a:::rbd_data.112d7dfbd766.000000000000018d:5 [read 516096~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-14T06:29:20.911 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:08.775074+0000", 2024-09-14T06:29:20.911 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 12.051896900999999, 2024-09-14T06:29:20.911 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.017330607000000001, 2024-09-14T06:29:20.911 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:29:20.911 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:29:20.911 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:29:20.911 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:29:20.911 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:29:20.912 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:20.912 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 3756 2024-09-14T06:29:20.912 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:20.912 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:29:20.912 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:20.912 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:29:20.912 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.775074+0000", 2024-09-14T06:29:20.912 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:20.912 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:20.912 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:20.912 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:29:20.912 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.775074+0000", 2024-09-14T06:29:20.912 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:20.912 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:20.912 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:20.913 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:29:20.913 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.775074+0000", 2024-09-14T06:29:20.913 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.1199999999999999e-07 2024-09-14T06:29:20.913 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:20.913 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:20.913 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:29:20.913 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.775075+0000", 2024-09-14T06:29:20.913 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.26e-06 2024-09-14T06:29:20.913 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:20.913 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:20.913 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:29:20.913 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.775079+0000", 2024-09-14T06:29:20.913 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.6689999999999999e-06 2024-09-14T06:29:20.913 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:20.914 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:20.914 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:29:20.914 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.775082+0000", 2024-09-14T06:29:20.914 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.3999999999999999e-06 2024-09-14T06:29:20.914 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:20.914 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:20.914 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:29:20.914 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.792267+0000", 2024-09-14T06:29:20.914 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.01718548 2024-09-14T06:29:20.914 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:20.914 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:20.914 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:29:20.914 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.792308+0000", 2024-09-14T06:29:20.914 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.0877000000000002e-05 2024-09-14T06:29:20.914 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:20.915 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:20.915 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:29:20.915 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.792405+0000", 2024-09-14T06:29:20.915 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.6708999999999999e-05 2024-09-14T06:29:20.915 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:20.915 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:20.915 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:20.915 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:20.915 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:20.915 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-14T06:29:20.917 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-14T06:29:20.918 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 reset_purged_snaps_last 2024-09-14T06:29:20.918 INFO:tasks.ceph.osd.1.smithi094.stderr:2024-09-14T06:29:20.915+0000 7f328e47a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33641) UID: 0 2024-09-14T06:29:21.017 INFO:tasks.ceph.osd.6.smithi138.stderr:2024-09-14T06:29:21.016+0000 7f8cd8849640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33620) UID: 0 2024-09-14T06:29:21.063 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-14T06:29:21.063 INFO:teuthology.orchestra.run.smithi094.stdout: "ops": [], 2024-09-14T06:29:21.063 INFO:teuthology.orchestra.run.smithi094.stdout: "num_ops": 0 2024-09-14T06:29:21.063 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-14T06:29:21.072 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-14T06:29:21.094 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 reset_purged_snaps_last 2024-09-14T06:29:21.118 INFO:tasks.ceph.osd.6.smithi138.stderr:2024-09-14T06:29:21.117+0000 7f8cd8849640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33620) UID: 0 2024-09-14T06:29:21.177 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-14T06:29:21.178 INFO:teuthology.orchestra.run.smithi094.stdout: "ops": [], 2024-09-14T06:29:21.178 INFO:teuthology.orchestra.run.smithi094.stdout: "complaint_time": 30, 2024-09-14T06:29:21.178 INFO:teuthology.orchestra.run.smithi094.stdout: "num_blocked_ops": 0 2024-09-14T06:29:21.178 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-14T06:29:21.187 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-14T06:29:21.219 INFO:tasks.ceph.osd.4.smithi138.stderr:2024-09-14T06:29:21.218+0000 7f6c653c8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33615) UID: 0 2024-09-14T06:29:21.272 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 reset_purged_snaps_last 2024-09-14T06:29:21.320 INFO:tasks.ceph.osd.2.smithi094.stderr:2024-09-14T06:29:21.319+0000 7faf3269b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33649) UID: 0 2024-09-14T06:29:21.337 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-14T06:29:21.337 INFO:teuthology.orchestra.run.smithi094.stdout: "size": 20, 2024-09-14T06:29:21.337 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 600, 2024-09-14T06:29:21.337 INFO:teuthology.orchestra.run.smithi094.stdout: "ops": [ 2024-09-14T06:29:21.337 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:21.337 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:28 3.4 3:27567f25:::rbd_object_map.112d7adb82ff:head [call lock.lock in=8b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-14T06:29:21.337 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:04.271783+0000", 2024-09-14T06:29:21.337 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 17.06251804, 2024-09-14T06:29:21.337 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0029333810000000001, 2024-09-14T06:29:21.337 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:29:21.337 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:29:21.338 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:29:21.338 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:29:21.338 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:29:21.338 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:21.338 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 28 2024-09-14T06:29:21.338 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:21.338 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:29:21.338 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:21.338 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:29:21.338 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:04.271783+0000", 2024-09-14T06:29:21.338 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:21.338 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:21.339 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:21.339 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:29:21.339 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:04.271783+0000", 2024-09-14T06:29:21.339 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:21.339 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:21.339 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:21.339 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:29:21.339 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:04.271784+0000", 2024-09-14T06:29:21.339 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.635e-06 2024-09-14T06:29:21.339 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:21.339 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:21.339 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:29:21.340 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:04.271794+0000", 2024-09-14T06:29:21.340 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.2650000000000002e-06 2024-09-14T06:29:21.340 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:21.340 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:21.340 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:29:21.340 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:04.271817+0000", 2024-09-14T06:29:21.340 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.3385000000000001e-05 2024-09-14T06:29:21.340 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:21.340 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:21.340 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:29:21.340 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:04.271824+0000", 2024-09-14T06:29:21.340 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.0500000000000003e-06 2024-09-14T06:29:21.340 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:21.341 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:21.341 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:29:21.344 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:04.271889+0000", 2024-09-14T06:29:21.344 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.4659999999999994e-05 2024-09-14T06:29:21.344 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:21.344 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:21.344 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "op must wait for map", 2024-09-14T06:29:21.345 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:04.271896+0000", 2024-09-14T06:29:21.345 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.52e-06 2024-09-14T06:29:21.345 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:21.345 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:21.345 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:29:21.345 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:04.273576+0000", 2024-09-14T06:29:21.345 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.001680225 2024-09-14T06:29:21.345 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:21.345 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:21.345 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:29:21.345 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:04.273635+0000", 2024-09-14T06:29:21.345 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.8832000000000002e-05 2024-09-14T06:29:21.345 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:21.346 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:21.346 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "waiting for subops from 2", 2024-09-14T06:29:21.346 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:04.273765+0000", 2024-09-14T06:29:21.346 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000129713 2024-09-14T06:29:21.346 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:21.346 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:21.346 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "op_commit", 2024-09-14T06:29:21.346 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:04.274315+0000", 2024-09-14T06:29:21.346 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00054979099999999999 2024-09-14T06:29:21.346 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:21.346 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:21.346 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:29:21.346 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:04.274605+0000", 2024-09-14T06:29:21.347 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00028987899999999999 2024-09-14T06:29:21.347 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:21.347 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:21.347 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-14T06:29:21.347 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:04.274666+0000", 2024-09-14T06:29:21.347 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.1710000000000004e-05 2024-09-14T06:29:21.347 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:21.347 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:21.347 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:29:21.347 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:04.274716+0000", 2024-09-14T06:29:21.347 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.9716000000000003e-05 2024-09-14T06:29:21.347 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:21.347 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:21.348 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:21.348 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:21.348 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:21.348 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:1848 3.4 3:27620a21:::rbd_data.112dcdddac74.000000000000000b:head [write 16384~32768 [fadvise_sequential+fadvise_nocache],write 81920~20480 [fadvise_sequential+fadvise_nocache]] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-09-14T06:29:21.348 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:08.657998+0000", 2024-09-14T06:29:21.348 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 12.676303173999999, 2024-09-14T06:29:21.348 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0028814520000000001, 2024-09-14T06:29:21.348 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:29:21.348 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:29:21.348 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:29:21.348 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:29:21.348 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:29:21.348 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:21.349 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 1848 2024-09-14T06:29:21.349 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:21.349 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:29:21.349 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:21.349 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:29:21.349 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.657998+0000", 2024-09-14T06:29:21.349 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:21.349 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:21.349 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:21.349 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:29:21.349 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.657998+0000", 2024-09-14T06:29:21.349 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:21.349 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:21.349 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:21.350 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:29:21.350 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.657998+0000", 2024-09-14T06:29:21.350 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.9500000000000001e-07 2024-09-14T06:29:21.350 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:21.350 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:21.350 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:29:21.350 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.658028+0000", 2024-09-14T06:29:21.350 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.9471999999999998e-05 2024-09-14T06:29:21.350 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:21.350 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:21.350 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:29:21.350 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.658044+0000", 2024-09-14T06:29:21.350 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.6089000000000001e-05 2024-09-14T06:29:21.350 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:21.351 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:21.351 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:29:21.351 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.658055+0000", 2024-09-14T06:29:21.351 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.061e-05 2024-09-14T06:29:21.351 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:21.351 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:21.351 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:29:21.351 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.658116+0000", 2024-09-14T06:29:21.351 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.1333000000000004e-05 2024-09-14T06:29:21.351 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:21.351 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:21.351 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:29:21.351 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.658264+0000", 2024-09-14T06:29:21.351 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000148481 2024-09-14T06:29:21.351 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:21.352 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:21.352 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "waiting for subops from 2", 2024-09-14T06:29:21.352 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.658433+0000", 2024-09-14T06:29:21.352 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00016825 2024-09-14T06:29:21.352 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:21.352 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:21.352 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "op_commit", 2024-09-14T06:29:21.352 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.659373+0000", 2024-09-14T06:29:21.352 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00093997799999999995 2024-09-14T06:29:21.352 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:21.352 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:21.352 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:29:21.352 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.660755+0000", 2024-09-14T06:29:21.352 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0013822940000000001 2024-09-14T06:29:21.353 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:21.353 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:21.353 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-14T06:29:21.353 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.660829+0000", 2024-09-14T06:29:21.353 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.4369999999999994e-05 2024-09-14T06:29:21.353 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:21.353 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:21.353 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:29:21.353 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.660879+0000", 2024-09-14T06:29:21.353 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.9679999999999999e-05 2024-09-14T06:29:21.353 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:21.353 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:21.353 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:21.354 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:21.354 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:21.354 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:2199 3.0 3:05f3e980:::rbd_data.112dcdddac74.000000000000000f:head [write 32768~16384 [fadvise_sequential+fadvise_nocache],write 98304~16384 [fadvise_sequential+fadvise_nocache],write 163840~16384 [fadvise_sequential+fadvise_nocache],write 229376~16384 [fadvise_sequential+fadvise_nocache],write 294912~16384 [fadvise_sequential+fadvise_nocache],write 360448~16384 [fadvise_sequential+fadvise_nocache],write 425984~12288 [fadvise_sequential+fadvise_nocache]] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-09-14T06:29:21.354 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:08.679514+0000", 2024-09-14T06:29:21.354 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 12.654786283, 2024-09-14T06:29:21.354 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0029599959999999999, 2024-09-14T06:29:21.354 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:29:21.354 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:29:21.354 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:29:21.354 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:29:21.354 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:29:21.354 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:21.354 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 2199 2024-09-14T06:29:21.354 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:21.355 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:29:21.355 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:21.355 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:29:21.355 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.679514+0000", 2024-09-14T06:29:21.355 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:21.355 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:21.355 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:21.355 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:29:21.355 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.679514+0000", 2024-09-14T06:29:21.355 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:21.355 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:21.355 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:21.356 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:29:21.356 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.679516+0000", 2024-09-14T06:29:21.356 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.7230000000000001e-06 2024-09-14T06:29:21.356 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:21.356 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:21.356 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:29:21.356 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.679576+0000", 2024-09-14T06:29:21.356 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.9734000000000001e-05 2024-09-14T06:29:21.356 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:21.357 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:21.357 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:29:21.357 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.679588+0000", 2024-09-14T06:29:21.357 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.2284e-05 2024-09-14T06:29:21.357 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:21.357 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:21.357 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:29:21.357 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.679596+0000", 2024-09-14T06:29:21.357 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.1480000000000006e-06 2024-09-14T06:29:21.357 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:21.357 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:21.357 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:29:21.357 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.679687+0000", 2024-09-14T06:29:21.357 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.0776e-05 2024-09-14T06:29:21.358 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:21.358 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:21.358 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:29:21.358 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.679854+0000", 2024-09-14T06:29:21.358 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00016668100000000001 2024-09-14T06:29:21.358 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:21.358 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:21.358 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "waiting for subops from 2", 2024-09-14T06:29:21.358 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.680112+0000", 2024-09-14T06:29:21.358 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00025867700000000002 2024-09-14T06:29:21.358 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:21.358 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:21.358 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "op_commit", 2024-09-14T06:29:21.358 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.681502+0000", 2024-09-14T06:29:21.358 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0013895260000000001 2024-09-14T06:29:21.358 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:21.359 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:21.359 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:29:21.359 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.682357+0000", 2024-09-14T06:29:21.359 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00085512900000000005 2024-09-14T06:29:21.359 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:21.359 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:21.359 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-14T06:29:21.359 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.682428+0000", 2024-09-14T06:29:21.359 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.0735000000000006e-05 2024-09-14T06:29:21.359 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:21.359 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:21.359 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:29:21.359 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.682474+0000", 2024-09-14T06:29:21.359 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.6582999999999998e-05 2024-09-14T06:29:21.359 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:21.360 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:21.360 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:21.360 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:21.360 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:21.360 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:2513 3.0 3:06f82ba7:::rbd_data.112dcdddac74.0000000000000028:head [write 524288~32768 [fadvise_sequential+fadvise_nocache],write 589824~32768 [fadvise_sequential+fadvise_nocache],write 655360~32768 [fadvise_sequential+fadvise_nocache],write 720896~32768 [fadvise_sequential+fadvise_nocache],write 786432~32768 [fadvise_sequential+fadvise_nocache],write 851968~27868 [fadvise_sequential+fadvise_nocache]] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-09-14T06:29:21.360 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:08.701645+0000", 2024-09-14T06:29:21.360 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 12.632655590000001, 2024-09-14T06:29:21.360 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0034338799999999998, 2024-09-14T06:29:21.360 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:29:21.360 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:29:21.360 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:29:21.360 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:29:21.360 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:29:21.360 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:21.361 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 2513 2024-09-14T06:29:21.361 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:21.361 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:29:21.361 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:21.361 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:29:21.361 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.701645+0000", 2024-09-14T06:29:21.361 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:21.361 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:21.361 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:21.361 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:29:21.361 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.701645+0000", 2024-09-14T06:29:21.361 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:21.361 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:21.361 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:21.362 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:29:21.362 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.701646+0000", 2024-09-14T06:29:21.362 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.54e-07 2024-09-14T06:29:21.362 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:21.362 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:21.362 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:29:21.362 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.701708+0000", 2024-09-14T06:29:21.362 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.2290000000000002e-05 2024-09-14T06:29:21.362 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:21.362 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:21.362 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:29:21.362 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.701718+0000", 2024-09-14T06:29:21.362 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.6590000000000002e-06 2024-09-14T06:29:21.362 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:21.362 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:21.362 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:29:21.363 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.701725+0000", 2024-09-14T06:29:21.363 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.9059999999999996e-06 2024-09-14T06:29:21.363 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:21.363 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:21.363 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:29:21.363 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.701784+0000", 2024-09-14T06:29:21.363 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.9429000000000003e-05 2024-09-14T06:29:21.363 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:21.363 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:21.363 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:29:21.363 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.701917+0000", 2024-09-14T06:29:21.363 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00013305700000000001 2024-09-14T06:29:21.363 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:21.363 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:21.363 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "waiting for subops from 2", 2024-09-14T06:29:21.364 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.702237+0000", 2024-09-14T06:29:21.364 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00032004100000000002 2024-09-14T06:29:21.364 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:21.364 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:21.364 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "op_commit", 2024-09-14T06:29:21.364 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.704494+0000", 2024-09-14T06:29:21.364 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0022567949999999998 2024-09-14T06:29:21.364 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:21.364 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:21.364 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:29:21.364 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.705007+0000", 2024-09-14T06:29:21.364 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00051250199999999997 2024-09-14T06:29:21.364 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:21.365 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:21.365 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-14T06:29:21.365 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.705049+0000", 2024-09-14T06:29:21.365 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.1967e-05 2024-09-14T06:29:21.365 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:21.365 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:21.365 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:29:21.365 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.705079+0000", 2024-09-14T06:29:21.365 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.0380000000000001e-05 2024-09-14T06:29:21.365 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:21.365 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:21.365 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:21.365 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:21.365 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:21.366 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:2539 3.19 3:99040d63:::rbd_data.112d7dfbd766.000000000000005f:5 [read 167936~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-14T06:29:21.366 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:08.703090+0000", 2024-09-14T06:29:21.366 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 12.631210887, 2024-09-14T06:29:21.366 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.002622138, 2024-09-14T06:29:21.366 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:29:21.366 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:29:21.366 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:29:21.366 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:29:21.366 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:29:21.366 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:21.366 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 2539 2024-09-14T06:29:21.366 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:21.366 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:29:21.367 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:21.367 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:29:21.367 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.703090+0000", 2024-09-14T06:29:21.367 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:21.367 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:21.367 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:21.367 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:29:21.367 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.703090+0000", 2024-09-14T06:29:21.367 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:21.367 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:21.367 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:21.368 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:29:21.368 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.703090+0000", 2024-09-14T06:29:21.368 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.6899999999999998e-07 2024-09-14T06:29:21.368 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:21.368 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:21.368 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:29:21.368 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.703092+0000", 2024-09-14T06:29:21.368 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.9099999999999999e-06 2024-09-14T06:29:21.368 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:21.368 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:21.368 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:29:21.368 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.703098+0000", 2024-09-14T06:29:21.368 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.6119999999999998e-06 2024-09-14T06:29:21.368 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:21.369 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:21.369 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:29:21.369 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.703102+0000", 2024-09-14T06:29:21.369 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.78e-06 2024-09-14T06:29:21.369 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:21.369 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:21.369 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:29:21.369 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.705527+0000", 2024-09-14T06:29:21.369 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.002424079 2024-09-14T06:29:21.369 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:21.369 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:21.369 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:29:21.369 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.705615+0000", 2024-09-14T06:29:21.370 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.8213000000000004e-05 2024-09-14T06:29:21.370 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:21.370 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:21.370 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:29:21.370 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.705712+0000", 2024-09-14T06:29:21.370 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.7175000000000004e-05 2024-09-14T06:29:21.370 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:21.370 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:21.370 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:21.370 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:21.370 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:21.370 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:2540 3.19 3:99040d63:::rbd_data.112d7dfbd766.000000000000005f:5 [read 184320~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-14T06:29:21.371 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:08.703109+0000", 2024-09-14T06:29:21.371 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 12.631191961000001, 2024-09-14T06:29:21.371 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.002871976, 2024-09-14T06:29:21.371 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:29:21.371 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:29:21.371 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:29:21.371 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:29:21.371 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:29:21.371 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:21.371 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 2540 2024-09-14T06:29:21.371 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:21.371 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:29:21.372 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:21.372 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:29:21.372 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.703109+0000", 2024-09-14T06:29:21.372 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:21.372 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:21.372 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:21.372 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:29:21.372 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.703109+0000", 2024-09-14T06:29:21.372 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:21.372 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:21.372 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:21.372 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:29:21.373 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.703109+0000", 2024-09-14T06:29:21.373 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.7099999999999997e-07 2024-09-14T06:29:21.373 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:21.373 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:21.373 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:29:21.373 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.703124+0000", 2024-09-14T06:29:21.373 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.5157000000000001e-05 2024-09-14T06:29:21.373 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:21.373 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:21.373 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:29:21.373 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.703214+0000", 2024-09-14T06:29:21.373 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.9658000000000001e-05 2024-09-14T06:29:21.374 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:21.374 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:21.374 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:29:21.374 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.703403+0000", 2024-09-14T06:29:21.374 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00018862699999999999 2024-09-14T06:29:21.374 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:21.374 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:21.375 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:29:21.375 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.705742+0000", 2024-09-14T06:29:21.375 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0023398500000000001 2024-09-14T06:29:21.375 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:21.375 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:21.375 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:29:21.376 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.705825+0000", 2024-09-14T06:29:21.376 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.2766e-05 2024-09-14T06:29:21.376 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:21.376 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:21.376 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:29:21.376 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.705981+0000", 2024-09-14T06:29:21.377 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00015554699999999999 2024-09-14T06:29:21.377 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:21.377 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:21.377 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:21.377 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:21.377 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:21.377 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:2559 3.19 3:99040d63:::rbd_data.112d7dfbd766.000000000000005f:5 [read 40960~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-14T06:29:21.378 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:08.704001+0000", 2024-09-14T06:29:21.378 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 12.630299344000001, 2024-09-14T06:29:21.378 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.002611969, 2024-09-14T06:29:21.378 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:29:21.378 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:29:21.378 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:29:21.378 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:29:21.378 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:29:21.378 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:21.379 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 2559 2024-09-14T06:29:21.379 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:21.379 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:29:21.379 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:21.379 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:29:21.379 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.704001+0000", 2024-09-14T06:29:21.379 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:21.379 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:21.380 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:21.380 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:29:21.380 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.704001+0000", 2024-09-14T06:29:21.380 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:21.380 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:21.380 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:21.380 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:29:21.380 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.704002+0000", 2024-09-14T06:29:21.381 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.6800000000000001e-07 2024-09-14T06:29:21.381 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:21.381 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:21.381 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:29:21.381 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.704003+0000", 2024-09-14T06:29:21.381 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.7910000000000001e-06 2024-09-14T06:29:21.381 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:21.381 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:21.382 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:29:21.382 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.704009+0000", 2024-09-14T06:29:21.382 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.7150000000000003e-06 2024-09-14T06:29:21.382 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:21.382 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:21.382 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:29:21.382 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.704017+0000", 2024-09-14T06:29:21.382 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.4039999999999998e-06 2024-09-14T06:29:21.382 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:21.383 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:21.383 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:29:21.383 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.706451+0000", 2024-09-14T06:29:21.383 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.002434758 2024-09-14T06:29:21.383 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:21.383 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:21.383 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:29:21.383 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.706522+0000", 2024-09-14T06:29:21.383 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.1038000000000007e-05 2024-09-14T06:29:21.383 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:21.384 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:21.384 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:29:21.384 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.706613+0000", 2024-09-14T06:29:21.384 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.0895000000000003e-05 2024-09-14T06:29:21.384 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:21.384 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:21.384 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:21.384 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:21.384 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:21.384 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:2560 3.19 3:99040d63:::rbd_data.112d7dfbd766.000000000000005f:5 [read 57344~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-14T06:29:21.384 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:08.704029+0000", 2024-09-14T06:29:21.384 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 12.630271280000001, 2024-09-14T06:29:21.384 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.002744776, 2024-09-14T06:29:21.384 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:29:21.385 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:29:21.385 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:29:21.385 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:29:21.385 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:29:21.385 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:21.385 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 2560 2024-09-14T06:29:21.385 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:21.385 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:29:21.385 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:21.385 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:29:21.385 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.704029+0000", 2024-09-14T06:29:21.385 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:21.385 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:21.386 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:21.386 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:29:21.386 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.704029+0000", 2024-09-14T06:29:21.386 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:21.386 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:21.386 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:21.386 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:29:21.386 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.704030+0000", 2024-09-14T06:29:21.386 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.1399999999999997e-07 2024-09-14T06:29:21.386 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:21.386 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:21.386 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:29:21.386 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.704033+0000", 2024-09-14T06:29:21.386 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.7199999999999998e-06 2024-09-14T06:29:21.386 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:21.387 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:21.387 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:29:21.387 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.704038+0000", 2024-09-14T06:29:21.387 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.925e-06 2024-09-14T06:29:21.387 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:21.387 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:21.387 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:29:21.387 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.704043+0000", 2024-09-14T06:29:21.387 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.9239999999999998e-06 2024-09-14T06:29:21.387 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:21.387 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:21.387 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:29:21.387 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.706648+0000", 2024-09-14T06:29:21.387 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.002604285 2024-09-14T06:29:21.387 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:21.387 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:21.388 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:29:21.388 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.706690+0000", 2024-09-14T06:29:21.388 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.278e-05 2024-09-14T06:29:21.388 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:21.388 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:21.388 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:29:21.388 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.706774+0000", 2024-09-14T06:29:21.388 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.3727999999999995e-05 2024-09-14T06:29:21.388 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:21.388 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:21.388 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:21.388 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:21.388 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:21.388 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:2561 3.19 3:99040d63:::rbd_data.112d7dfbd766.000000000000005f:5 [read 73728~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-14T06:29:21.388 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:08.704050+0000", 2024-09-14T06:29:21.389 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 12.630250981, 2024-09-14T06:29:21.389 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0029452300000000001, 2024-09-14T06:29:21.389 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:29:21.389 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:29:21.389 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:29:21.389 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:29:21.389 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:29:21.389 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:21.389 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 2561 2024-09-14T06:29:21.389 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:21.389 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:29:21.389 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:21.389 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:29:21.389 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.704050+0000", 2024-09-14T06:29:21.390 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:21.390 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:21.390 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:21.390 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:29:21.390 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.704050+0000", 2024-09-14T06:29:21.390 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:21.390 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:21.390 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:21.390 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:29:21.390 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.704050+0000", 2024-09-14T06:29:21.390 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.4400000000000001e-07 2024-09-14T06:29:21.390 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:21.390 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:21.390 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:29:21.390 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.704052+0000", 2024-09-14T06:29:21.390 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.9290000000000001e-06 2024-09-14T06:29:21.391 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:21.391 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:21.391 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:29:21.391 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.704072+0000", 2024-09-14T06:29:21.391 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.9650999999999999e-05 2024-09-14T06:29:21.391 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:21.391 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:21.391 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:29:21.391 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.704077+0000", 2024-09-14T06:29:21.391 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.7799999999999997e-06 2024-09-14T06:29:21.391 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:21.391 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:21.391 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:29:21.391 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.706827+0000", 2024-09-14T06:29:21.391 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0027497989999999998 2024-09-14T06:29:21.392 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:21.392 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:21.392 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:29:21.392 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.706901+0000", 2024-09-14T06:29:21.392 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.4208999999999994e-05 2024-09-14T06:29:21.392 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:21.392 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:21.392 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:29:21.392 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.706995+0000", 2024-09-14T06:29:21.392 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.3517999999999995e-05 2024-09-14T06:29:21.392 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:21.392 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:21.392 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:21.392 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:21.392 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:21.392 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:2562 3.19 3:99040d63:::rbd_data.112d7dfbd766.000000000000005f:5 [read 90112~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-14T06:29:21.393 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:08.704086+0000", 2024-09-14T06:29:21.393 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 12.630215080999999, 2024-09-14T06:29:21.393 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0031048159999999998, 2024-09-14T06:29:21.393 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:29:21.393 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:29:21.393 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:29:21.393 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:29:21.393 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:29:21.393 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:21.393 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 2562 2024-09-14T06:29:21.393 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:21.393 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:29:21.394 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:21.394 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:29:21.394 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.704086+0000", 2024-09-14T06:29:21.394 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:21.394 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:21.394 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:21.394 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:29:21.394 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.704086+0000", 2024-09-14T06:29:21.394 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:21.394 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:21.394 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:21.394 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:29:21.394 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.704086+0000", 2024-09-14T06:29:21.394 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.8999999999999997e-07 2024-09-14T06:29:21.394 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:21.395 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:21.395 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:29:21.395 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.704089+0000", 2024-09-14T06:29:21.395 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.413e-06 2024-09-14T06:29:21.395 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:21.395 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:21.395 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:29:21.395 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.704094+0000", 2024-09-14T06:29:21.395 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.8119999999999996e-06 2024-09-14T06:29:21.395 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:21.395 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:21.396 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:29:21.396 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.704099+0000", 2024-09-14T06:29:21.396 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.7639999999999996e-06 2024-09-14T06:29:21.396 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:21.396 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:21.396 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:29:21.396 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.707026+0000", 2024-09-14T06:29:21.396 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0029271750000000002 2024-09-14T06:29:21.396 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:21.396 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:21.396 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:29:21.396 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.707097+0000", 2024-09-14T06:29:21.396 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.1052999999999999e-05 2024-09-14T06:29:21.397 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:21.397 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:21.397 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:29:21.397 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.707190+0000", 2024-09-14T06:29:21.397 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.3109000000000006e-05 2024-09-14T06:29:21.397 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:21.397 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:21.397 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:21.397 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:21.397 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:21.397 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:2563 3.19 3:99040d63:::rbd_data.112d7dfbd766.000000000000005f:5 [read 106496~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-14T06:29:21.397 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:08.704177+0000", 2024-09-14T06:29:21.398 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 12.630124052999999, 2024-09-14T06:29:21.398 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0031916900000000001, 2024-09-14T06:29:21.398 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:29:21.398 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:29:21.398 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:29:21.398 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:29:21.398 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:29:21.398 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:21.398 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 2563 2024-09-14T06:29:21.398 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:21.398 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:29:21.399 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:21.399 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:29:21.399 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.704177+0000", 2024-09-14T06:29:21.399 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:21.399 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:21.399 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:21.399 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:29:21.399 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.704177+0000", 2024-09-14T06:29:21.399 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:21.399 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:21.399 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:21.399 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:29:21.399 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.704177+0000", 2024-09-14T06:29:21.399 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.4499999999999997e-07 2024-09-14T06:29:21.399 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:21.400 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:21.400 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:29:21.400 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.704180+0000", 2024-09-14T06:29:21.400 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.1439999999999999e-06 2024-09-14T06:29:21.400 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:21.400 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:21.400 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:29:21.400 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.704189+0000", 2024-09-14T06:29:21.400 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.2009999999999999e-06 2024-09-14T06:29:21.400 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:21.400 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:21.400 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:29:21.400 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.704194+0000", 2024-09-14T06:29:21.400 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.8690000000000001e-06 2024-09-14T06:29:21.400 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:21.401 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:21.401 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:29:21.401 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.707218+0000", 2024-09-14T06:29:21.401 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0030240029999999999 2024-09-14T06:29:21.401 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:21.401 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:21.401 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:29:21.401 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.707287+0000", 2024-09-14T06:29:21.401 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.8299000000000005e-05 2024-09-14T06:29:21.401 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:21.401 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:21.401 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:29:21.401 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.707368+0000", 2024-09-14T06:29:21.402 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.1528999999999998e-05 2024-09-14T06:29:21.402 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:21.402 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:21.402 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:21.402 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:21.402 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:21.402 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:2564 3.19 3:99040d63:::rbd_data.112d7dfbd766.000000000000005f:5 [read 122880~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-14T06:29:21.402 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:08.704210+0000", 2024-09-14T06:29:21.402 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 12.630091055999999, 2024-09-14T06:29:21.402 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0033551369999999998, 2024-09-14T06:29:21.402 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:29:21.402 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:29:21.402 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:29:21.402 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:29:21.402 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:29:21.403 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:21.403 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 2564 2024-09-14T06:29:21.403 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:21.403 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:29:21.403 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:21.403 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:29:21.403 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.704210+0000", 2024-09-14T06:29:21.403 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:21.403 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:21.403 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:21.403 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:29:21.403 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.704210+0000", 2024-09-14T06:29:21.403 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:21.403 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:21.403 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:21.404 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:29:21.404 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.704210+0000", 2024-09-14T06:29:21.404 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.3799999999999998e-07 2024-09-14T06:29:21.404 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:21.404 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:21.404 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:29:21.404 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.704212+0000", 2024-09-14T06:29:21.405 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.2010000000000002e-06 2024-09-14T06:29:21.405 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:21.405 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:21.405 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:29:21.405 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.704218+0000", 2024-09-14T06:29:21.405 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.1700000000000002e-06 2024-09-14T06:29:21.405 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:21.405 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:21.405 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:29:21.405 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.704223+0000", 2024-09-14T06:29:21.405 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.9069999999999998e-06 2024-09-14T06:29:21.405 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:21.405 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:21.406 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:29:21.406 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.707400+0000", 2024-09-14T06:29:21.406 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0031764419999999998 2024-09-14T06:29:21.406 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:21.406 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:21.406 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:29:21.406 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.707473+0000", 2024-09-14T06:29:21.406 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.3205000000000001e-05 2024-09-14T06:29:21.406 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:21.406 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:21.406 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:29:21.406 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.707565+0000", 2024-09-14T06:29:21.406 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.1774000000000006e-05 2024-09-14T06:29:21.406 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:21.406 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:21.407 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:21.407 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:21.407 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:21.407 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:2565 3.19 3:99040d63:::rbd_data.112d7dfbd766.000000000000005f:5 [read 139264~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-14T06:29:21.407 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:08.704230+0000", 2024-09-14T06:29:21.407 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 12.630071054, 2024-09-14T06:29:21.407 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.003537073, 2024-09-14T06:29:21.407 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:29:21.407 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:29:21.407 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:29:21.407 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:29:21.407 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:29:21.407 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:21.407 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 2565 2024-09-14T06:29:21.407 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:21.407 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:29:21.408 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:21.408 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:29:21.408 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.704230+0000", 2024-09-14T06:29:21.408 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:21.408 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:21.408 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:21.408 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:29:21.408 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.704230+0000", 2024-09-14T06:29:21.408 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:21.408 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:21.408 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:21.409 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:29:21.409 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.704230+0000", 2024-09-14T06:29:21.409 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.0900000000000002e-07 2024-09-14T06:29:21.409 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:21.409 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:21.409 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:29:21.409 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.704232+0000", 2024-09-14T06:29:21.409 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.6279999999999999e-06 2024-09-14T06:29:21.409 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:21.409 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:21.409 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:29:21.409 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.704237+0000", 2024-09-14T06:29:21.409 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.327e-06 2024-09-14T06:29:21.409 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:21.410 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:21.410 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:29:21.410 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.704242+0000", 2024-09-14T06:29:21.410 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.6330000000000004e-06 2024-09-14T06:29:21.410 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:21.410 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:21.410 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:29:21.410 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.707596+0000", 2024-09-14T06:29:21.410 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.003354788 2024-09-14T06:29:21.410 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:21.410 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:21.410 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:29:21.410 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.707667+0000", 2024-09-14T06:29:21.410 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.0623000000000005e-05 2024-09-14T06:29:21.410 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:21.411 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:21.411 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:29:21.411 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.707767+0000", 2024-09-14T06:29:21.411 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.9665000000000002e-05 2024-09-14T06:29:21.411 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:21.411 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:21.411 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:21.411 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:21.411 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:21.411 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:2566 3.19 3:99040d63:::rbd_data.112d7dfbd766.000000000000005f:5 [read 155648~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-14T06:29:21.411 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:08.704248+0000", 2024-09-14T06:29:21.412 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 12.630052555000001, 2024-09-14T06:29:21.412 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0037151810000000001, 2024-09-14T06:29:21.412 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:29:21.412 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:29:21.412 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:29:21.412 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:29:21.412 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:29:21.412 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:21.412 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 2566 2024-09-14T06:29:21.412 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:21.412 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:29:21.412 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:21.412 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:29:21.412 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.704248+0000", 2024-09-14T06:29:21.412 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:21.413 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:21.413 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:21.413 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:29:21.413 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.704248+0000", 2024-09-14T06:29:21.413 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:21.413 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:21.413 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:21.413 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:29:21.413 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.704249+0000", 2024-09-14T06:29:21.413 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.7599999999999998e-07 2024-09-14T06:29:21.413 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:21.413 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:21.413 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:29:21.414 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.704252+0000", 2024-09-14T06:29:21.414 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.411e-06 2024-09-14T06:29:21.414 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:21.414 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:21.414 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:29:21.414 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.704257+0000", 2024-09-14T06:29:21.414 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.5369999999999997e-06 2024-09-14T06:29:21.414 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:21.414 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:21.414 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:29:21.414 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.704263+0000", 2024-09-14T06:29:21.414 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.4820000000000002e-06 2024-09-14T06:29:21.414 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:21.414 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:21.415 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:29:21.415 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.707797+0000", 2024-09-14T06:29:21.415 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.003533722 2024-09-14T06:29:21.415 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:21.415 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:21.415 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:29:21.415 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.707869+0000", 2024-09-14T06:29:21.415 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.1997999999999997e-05 2024-09-14T06:29:21.415 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:21.415 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:21.415 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:29:21.415 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.707963+0000", 2024-09-14T06:29:21.415 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.4654999999999994e-05 2024-09-14T06:29:21.416 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:21.416 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:21.416 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:21.416 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:21.416 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:21.416 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:2567 3.19 3:99040d63:::rbd_data.112d7dfbd766.000000000000005f:5 [read 172032~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-14T06:29:21.416 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:08.704269+0000", 2024-09-14T06:29:21.416 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 12.630031517000001, 2024-09-14T06:29:21.416 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00397446, 2024-09-14T06:29:21.416 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:29:21.416 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:29:21.416 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:29:21.417 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:29:21.417 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:29:21.417 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:21.417 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 2567 2024-09-14T06:29:21.417 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:21.417 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:29:21.417 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:21.417 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:29:21.417 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.704269+0000", 2024-09-14T06:29:21.417 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:21.417 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:21.417 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:21.417 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:29:21.417 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.704269+0000", 2024-09-14T06:29:21.418 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:21.418 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:21.418 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:21.418 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:29:21.418 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.704270+0000", 2024-09-14T06:29:21.418 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.77e-07 2024-09-14T06:29:21.418 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:21.418 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:21.418 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:29:21.418 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.704271+0000", 2024-09-14T06:29:21.418 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.7829999999999999e-06 2024-09-14T06:29:21.418 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:21.419 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:21.419 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:29:21.419 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.704319+0000", 2024-09-14T06:29:21.419 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.8109999999999998e-05 2024-09-14T06:29:21.419 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:21.420 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:21.420 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:29:21.420 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.704325+0000", 2024-09-14T06:29:21.420 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.7810000000000002e-06 2024-09-14T06:29:21.420 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:21.420 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:21.420 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:29:21.420 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.707993+0000", 2024-09-14T06:29:21.421 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0036678710000000001 2024-09-14T06:29:21.421 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:21.421 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:21.421 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:29:21.421 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.708057+0000", 2024-09-14T06:29:21.421 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.4302999999999997e-05 2024-09-14T06:29:21.421 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:21.421 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:21.421 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:29:21.421 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.708244+0000", 2024-09-14T06:29:21.421 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00018623500000000001 2024-09-14T06:29:21.421 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:21.422 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:21.422 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:21.422 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:21.422 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:21.422 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:2568 3.19 3:99040d63:::rbd_data.112d7dfbd766.000000000000005f:5 [read 188416~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-14T06:29:21.422 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:08.704335+0000", 2024-09-14T06:29:21.422 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 12.629965997999999, 2024-09-14T06:29:21.422 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.004144778, 2024-09-14T06:29:21.422 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:29:21.422 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:29:21.422 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:29:21.422 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:29:21.422 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:29:21.422 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:21.423 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 2568 2024-09-14T06:29:21.423 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:21.423 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:29:21.423 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:21.423 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:29:21.423 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.704335+0000", 2024-09-14T06:29:21.423 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:21.423 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:21.423 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:21.423 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:29:21.423 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.704335+0000", 2024-09-14T06:29:21.423 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:21.423 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:21.423 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:21.423 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:29:21.424 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.704335+0000", 2024-09-14T06:29:21.424 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.6199999999999998e-07 2024-09-14T06:29:21.424 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:21.424 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:21.424 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:29:21.424 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.704342+0000", 2024-09-14T06:29:21.424 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.5169999999999996e-06 2024-09-14T06:29:21.424 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:21.424 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:21.424 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:29:21.424 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.704348+0000", 2024-09-14T06:29:21.424 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.003e-06 2024-09-14T06:29:21.424 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:21.425 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:21.425 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:29:21.425 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.704353+0000", 2024-09-14T06:29:21.425 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.0699999999999997e-06 2024-09-14T06:29:21.425 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:21.425 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:21.425 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:29:21.425 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.708273+0000", 2024-09-14T06:29:21.425 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0039204009999999996 2024-09-14T06:29:21.425 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:21.425 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:21.425 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:29:21.425 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.708367+0000", 2024-09-14T06:29:21.426 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.3987999999999995e-05 2024-09-14T06:29:21.426 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:21.426 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:21.426 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:29:21.426 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.708479+0000", 2024-09-14T06:29:21.426 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000112237 2024-09-14T06:29:21.426 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:21.426 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:21.426 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:21.426 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:21.426 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:21.426 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:2573 3.19 3:9f664035:::rbd_data.112d7dfbd766.000000000000005d:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-14T06:29:21.426 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:08.704360+0000", 2024-09-14T06:29:21.426 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 12.629940659000001, 2024-09-14T06:29:21.427 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.004261595, 2024-09-14T06:29:21.427 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:29:21.427 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:29:21.427 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "reached pg", 2024-09-14T06:29:21.427 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:29:21.427 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:29:21.427 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:21.427 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 2573 2024-09-14T06:29:21.427 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:21.427 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:29:21.427 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:21.427 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:29:21.427 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.704360+0000", 2024-09-14T06:29:21.427 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:21.428 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:21.428 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:21.428 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:29:21.428 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.704360+0000", 2024-09-14T06:29:21.428 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:21.428 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:21.428 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:21.428 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:29:21.428 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.704360+0000", 2024-09-14T06:29:21.429 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.63e-07 2024-09-14T06:29:21.429 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:21.429 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:21.429 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:29:21.429 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.704362+0000", 2024-09-14T06:29:21.429 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.748e-06 2024-09-14T06:29:21.429 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:21.429 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:21.429 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:29:21.429 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.704369+0000", 2024-09-14T06:29:21.429 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.0319999999999998e-06 2024-09-14T06:29:21.429 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:21.429 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:21.430 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:29:21.430 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.704404+0000", 2024-09-14T06:29:21.430 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.5057e-05 2024-09-14T06:29:21.430 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:21.430 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:21.430 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:29:21.430 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.708524+0000", 2024-09-14T06:29:21.430 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0041199280000000001 2024-09-14T06:29:21.430 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:21.430 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:21.430 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:29:21.430 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.708622+0000", 2024-09-14T06:29:21.431 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.7367000000000005e-05 2024-09-14T06:29:21.431 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:21.431 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:21.431 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:21.431 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:21.432 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:21.432 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:2575 3.19 3:99040d63:::rbd_data.112d7dfbd766.000000000000005f:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-14T06:29:21.432 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:08.704414+0000", 2024-09-14T06:29:21.432 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 12.629886429000001, 2024-09-14T06:29:21.433 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0043794460000000004, 2024-09-14T06:29:21.433 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:29:21.433 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:29:21.433 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:29:21.433 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:29:21.433 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:29:21.434 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:21.434 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 2575 2024-09-14T06:29:21.434 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:21.434 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:29:21.435 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:21.435 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:29:21.435 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.704414+0000", 2024-09-14T06:29:21.435 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:21.436 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:21.436 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:21.436 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:29:21.436 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.704414+0000", 2024-09-14T06:29:21.436 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:21.436 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:21.437 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:21.437 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:29:21.437 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.704415+0000", 2024-09-14T06:29:21.437 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.7000000000000005e-07 2024-09-14T06:29:21.437 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:21.437 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:21.438 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:29:21.438 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.704417+0000", 2024-09-14T06:29:21.438 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.503e-06 2024-09-14T06:29:21.438 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:21.438 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:21.438 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:29:21.438 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.704469+0000", 2024-09-14T06:29:21.438 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.1492000000000001e-05 2024-09-14T06:29:21.439 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:21.439 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:21.439 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:29:21.439 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.704474+0000", 2024-09-14T06:29:21.439 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.5929999999999996e-06 2024-09-14T06:29:21.440 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:21.440 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:21.441 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:29:21.441 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.708654+0000", 2024-09-14T06:29:21.441 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0041792640000000002 2024-09-14T06:29:21.441 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:21.441 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:21.441 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:29:21.442 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.708725+0000", 2024-09-14T06:29:21.442 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.1755000000000007e-05 2024-09-14T06:29:21.442 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:21.442 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:21.442 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:29:21.443 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.708794+0000", 2024-09-14T06:29:21.443 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.8269000000000007e-05 2024-09-14T06:29:21.443 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:21.443 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:21.443 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:21.443 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:21.444 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:21.444 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:3928 3.19 3:9e9fa1be:::rbd_data.112dcdddac74.00000000000000c6:head [write 540672~32768 [fadvise_sequential+fadvise_nocache],write 606208~32768 [fadvise_sequential+fadvise_nocache],write 671744~32768 [fadvise_sequential+fadvise_nocache],write 737280~32768 [fadvise_sequential+fadvise_nocache],write 802816~32768 [fadvise_sequential+fadvise_nocache],write 868352~32768 [fadvise_sequential+fadvise_nocache],write 933888~32768 [fadvise_sequential+fadvise_nocache],write 999424~20480 [fadvise_sequential+fadvise_nocache]] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-09-14T06:29:21.444 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:08.794629+0000", 2024-09-14T06:29:21.444 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 12.539671239, 2024-09-14T06:29:21.444 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.002453144, 2024-09-14T06:29:21.444 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:29:21.444 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:29:21.445 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:29:21.445 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:29:21.445 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:29:21.445 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:21.445 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 3928 2024-09-14T06:29:21.445 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:21.445 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:29:21.445 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:21.446 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:29:21.446 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.794629+0000", 2024-09-14T06:29:21.446 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:21.446 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:21.446 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:21.446 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:29:21.446 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.794629+0000", 2024-09-14T06:29:21.446 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:21.447 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:21.447 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:21.447 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:29:21.447 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.794631+0000", 2024-09-14T06:29:21.447 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.8360000000000001e-06 2024-09-14T06:29:21.447 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:21.447 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:21.447 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:29:21.447 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.794707+0000", 2024-09-14T06:29:21.448 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.5288000000000002e-05 2024-09-14T06:29:21.448 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:21.448 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:21.448 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:29:21.448 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.794721+0000", 2024-09-14T06:29:21.448 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.4671999999999999e-05 2024-09-14T06:29:21.448 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:21.449 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:21.449 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:29:21.449 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.794730+0000", 2024-09-14T06:29:21.449 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.4440000000000008e-06 2024-09-14T06:29:21.449 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:21.449 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:21.450 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:29:21.450 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.794795+0000", 2024-09-14T06:29:21.450 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.5158000000000002e-05 2024-09-14T06:29:21.450 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:21.450 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:21.450 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:29:21.451 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.794972+0000", 2024-09-14T06:29:21.451 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00017739899999999999 2024-09-14T06:29:21.451 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:21.451 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:21.451 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "waiting for subops from 3", 2024-09-14T06:29:21.451 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.795285+0000", 2024-09-14T06:29:21.452 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000312589 2024-09-14T06:29:21.452 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:21.452 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:21.452 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "op_commit", 2024-09-14T06:29:21.452 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.796713+0000", 2024-09-14T06:29:21.452 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.001428559 2024-09-14T06:29:21.452 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:21.452 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:21.452 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:29:21.452 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.796954+0000", 2024-09-14T06:29:21.452 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000240597 2024-09-14T06:29:21.453 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:21.453 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:21.453 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-14T06:29:21.453 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.797034+0000", 2024-09-14T06:29:21.453 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.9835999999999999e-05 2024-09-14T06:29:21.453 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:21.453 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:21.453 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:29:21.453 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.797083+0000", 2024-09-14T06:29:21.453 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.8766e-05 2024-09-14T06:29:21.453 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:21.453 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:21.453 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:21.454 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:21.454 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:21.454 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:4781 3.4 3:26a26a4b:::rbd_data.112dcdddac74.000000000000001c:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-14T06:29:21.454 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:12.628861+0000", 2024-09-14T06:29:21.454 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 8.7054395650000007, 2024-09-14T06:29:21.454 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.002795479, 2024-09-14T06:29:21.454 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:29:21.454 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:29:21.454 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "reached pg", 2024-09-14T06:29:21.454 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:29:21.454 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:29:21.454 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:21.454 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 4781 2024-09-14T06:29:21.454 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:21.455 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:29:21.455 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:21.455 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:29:21.455 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:12.628861+0000", 2024-09-14T06:29:21.455 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:21.455 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:21.455 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:21.455 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:29:21.455 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:12.628861+0000", 2024-09-14T06:29:21.455 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:21.455 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:21.455 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:21.456 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:29:21.456 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:12.628864+0000", 2024-09-14T06:29:21.456 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.481e-06 2024-09-14T06:29:21.456 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:21.456 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:21.456 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:29:21.456 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:12.628875+0000", 2024-09-14T06:29:21.456 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.1484000000000001e-05 2024-09-14T06:29:21.456 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:21.456 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:21.456 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:29:21.456 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:12.628898+0000", 2024-09-14T06:29:21.456 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.2507e-05 2024-09-14T06:29:21.457 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:21.457 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:21.457 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:29:21.457 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:12.628910+0000", 2024-09-14T06:29:21.457 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.1946999999999999e-05 2024-09-14T06:29:21.457 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:21.457 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:21.457 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:29:21.457 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:12.628999+0000", 2024-09-14T06:29:21.457 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.9954e-05 2024-09-14T06:29:21.457 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:21.457 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:21.457 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "op must wait for map", 2024-09-14T06:29:21.457 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:12.629008+0000", 2024-09-14T06:29:21.458 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.6160000000000002e-06 2024-09-14T06:29:21.458 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:21.458 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:21.458 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:29:21.458 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:12.631545+0000", 2024-09-14T06:29:21.458 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0025371640000000002 2024-09-14T06:29:21.458 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:21.458 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:21.458 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:29:21.458 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:12.631657+0000", 2024-09-14T06:29:21.458 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000111326 2024-09-14T06:29:21.458 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:21.458 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:21.458 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:21.458 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:21.459 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:21.459 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-14T06:29:21.460 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-14T06:29:21.461 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 reset_purged_snaps_last 2024-09-14T06:29:21.462 INFO:tasks.ceph.osd.5.smithi138.stderr:2024-09-14T06:29:21.459+0000 7fe13746f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33618) UID: 0 2024-09-14T06:29:21.561 INFO:tasks.ceph.osd.6.smithi138.stderr:2024-09-14T06:29:21.560+0000 7f8cd8849640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33620) UID: 0 2024-09-14T06:29:21.577 INFO:teuthology.orchestra.run.smithi094.stderr:nodeep-scrub is unset 2024-09-14T06:29:21.617 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-14T06:29:21.618 INFO:teuthology.orchestra.run.smithi094.stdout: "ops": [], 2024-09-14T06:29:21.618 INFO:teuthology.orchestra.run.smithi094.stdout: "num_ops": 0 2024-09-14T06:29:21.618 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-14T06:29:21.627 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-14T06:29:21.647 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 reset_purged_snaps_last 2024-09-14T06:29:21.689 INFO:tasks.ceph.osd.2.smithi094.stderr:2024-09-14T06:29:21.688+0000 7faf3269b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33649) UID: 0 2024-09-14T06:29:21.763 INFO:tasks.ceph.osd.1.smithi094.stderr:2024-09-14T06:29:21.762+0000 7f328e47a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33641) UID: 0 2024-09-14T06:29:21.793 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-14T06:29:21.793 INFO:teuthology.orchestra.run.smithi094.stdout: "ops": [], 2024-09-14T06:29:21.793 INFO:teuthology.orchestra.run.smithi094.stdout: "complaint_time": 30, 2024-09-14T06:29:21.793 INFO:teuthology.orchestra.run.smithi094.stdout: "num_blocked_ops": 0 2024-09-14T06:29:21.793 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-14T06:29:21.802 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-14T06:29:21.824 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 reset_purged_snaps_last 2024-09-14T06:29:21.863 INFO:tasks.ceph.osd.7.smithi138.stderr:2024-09-14T06:29:21.862+0000 7f3c6a21d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33623) UID: 0 2024-09-14T06:29:21.964 INFO:tasks.ceph.osd.6.smithi138.stderr:2024-09-14T06:29:21.963+0000 7f8cd8849640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33620) UID: 0 2024-09-14T06:29:21.972 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-14T06:29:21.972 INFO:teuthology.orchestra.run.smithi094.stdout: "size": 20, 2024-09-14T06:29:21.972 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 600, 2024-09-14T06:29:21.973 INFO:teuthology.orchestra.run.smithi094.stdout: "ops": [ 2024-09-14T06:29:21.973 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:21.973 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4299.0:1 2.3 e20/19 2:c4c92e5a:::rbd_trash:head v 20'1, mlcod=0'0)", 2024-09-14T06:29:21.973 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:28:51.223664+0000", 2024-09-14T06:29:21.973 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 30.745958270999999, 2024-09-14T06:29:21.973 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.001165588, 2024-09-14T06:29:21.973 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:29:21.973 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:29:21.973 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:29:21.973 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:29:21.974 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:21.974 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:29:21.974 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:28:51.223664+0000", 2024-09-14T06:29:21.974 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:21.974 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:21.974 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:21.974 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:29:21.974 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:28:51.223664+0000", 2024-09-14T06:29:21.974 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:21.974 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:21.975 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:21.975 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:29:21.975 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:28:51.223665+0000", 2024-09-14T06:29:21.975 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.3600000000000003e-07 2024-09-14T06:29:21.975 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:21.975 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:21.975 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:29:21.975 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:28:51.223672+0000", 2024-09-14T06:29:21.975 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.4740000000000002e-06 2024-09-14T06:29:21.975 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:21.975 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:21.975 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:29:21.975 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:28:51.223695+0000", 2024-09-14T06:29:21.975 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.3859e-05 2024-09-14T06:29:21.976 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:21.976 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:21.976 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:29:21.976 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:28:51.223715+0000", 2024-09-14T06:29:21.976 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.9406e-05 2024-09-14T06:29:21.976 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:21.976 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:21.976 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:29:21.976 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:28:51.223780+0000", 2024-09-14T06:29:21.976 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.4777999999999995e-05 2024-09-14T06:29:21.976 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:21.976 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:21.976 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:29:21.977 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:28:51.223817+0000", 2024-09-14T06:29:21.977 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.7660999999999997e-05 2024-09-14T06:29:21.977 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:21.977 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:21.977 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-14T06:29:21.977 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:28:51.224781+0000", 2024-09-14T06:29:21.977 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00096367800000000004 2024-09-14T06:29:21.977 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:21.977 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:21.978 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:29:21.978 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:28:51.224830+0000", 2024-09-14T06:29:21.978 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.9095999999999999e-05 2024-09-14T06:29:21.978 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:21.978 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:21.978 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:21.978 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:21.978 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:21.978 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4397.0:13 3.4 e26/23 3:27567f25:::rbd_object_map.112d7adb82ff:head v 26'1, mlcod=0'0)", 2024-09-14T06:29:21.978 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:02.590205+0000", 2024-09-14T06:29:21.978 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 19.379417516, 2024-09-14T06:29:21.978 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00092769300000000005, 2024-09-14T06:29:21.978 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:29:21.978 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:29:21.979 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:29:21.979 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:29:21.979 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:21.979 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:29:21.979 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:02.590205+0000", 2024-09-14T06:29:21.979 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:21.979 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:21.979 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:21.979 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:29:21.979 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:02.590205+0000", 2024-09-14T06:29:21.979 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:21.979 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:21.979 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:21.979 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:29:21.979 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:02.590205+0000", 2024-09-14T06:29:21.980 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.5699999999999999e-07 2024-09-14T06:29:21.980 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:21.980 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:21.980 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:29:21.980 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:02.590211+0000", 2024-09-14T06:29:21.980 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.2209999999999996e-06 2024-09-14T06:29:21.980 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:21.980 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:21.980 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:29:21.980 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:02.590220+0000", 2024-09-14T06:29:21.980 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.5359999999999998e-06 2024-09-14T06:29:21.980 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:21.980 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:21.981 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:29:21.981 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:02.590227+0000", 2024-09-14T06:29:21.981 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.4790000000000001e-06 2024-09-14T06:29:21.981 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:21.981 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:21.981 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:29:21.981 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:02.590286+0000", 2024-09-14T06:29:21.981 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.9948999999999998e-05 2024-09-14T06:29:21.981 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:21.982 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:21.982 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:29:21.982 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:02.590308+0000", 2024-09-14T06:29:21.982 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.1580999999999999e-05 2024-09-14T06:29:21.982 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:21.982 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:21.982 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-14T06:29:21.982 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:02.591104+0000", 2024-09-14T06:29:21.982 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00079564299999999996 2024-09-14T06:29:21.982 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:21.982 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:21.982 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:29:21.982 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:02.591133+0000", 2024-09-14T06:29:21.983 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.9127000000000001e-05 2024-09-14T06:29:21.983 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:21.983 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:21.983 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:21.983 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:21.983 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:21.983 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4397.0:57 3.4 e27/23 3:27567f25:::rbd_object_map.112d7adb82ff:head v 27'5, mlcod=27'3)", 2024-09-14T06:29:21.983 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:04.297136+0000", 2024-09-14T06:29:21.983 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 17.672486711000001, 2024-09-14T06:29:21.983 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00086588099999999999, 2024-09-14T06:29:21.984 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:29:21.984 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:29:21.984 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:29:21.984 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:29:21.984 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:21.984 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:29:21.984 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:04.297136+0000", 2024-09-14T06:29:21.984 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:21.984 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:21.984 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:21.984 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:29:21.984 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:04.297136+0000", 2024-09-14T06:29:21.984 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:21.985 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:21.985 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:21.985 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:29:21.985 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:04.297137+0000", 2024-09-14T06:29:21.985 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.404e-06 2024-09-14T06:29:21.985 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:21.985 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:21.985 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:29:21.985 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:04.297159+0000", 2024-09-14T06:29:21.985 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.2005999999999999e-05 2024-09-14T06:29:21.985 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:21.985 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:21.985 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:29:21.985 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:04.297179+0000", 2024-09-14T06:29:21.986 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.963e-05 2024-09-14T06:29:21.986 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:21.986 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:21.986 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:29:21.986 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:04.297188+0000", 2024-09-14T06:29:21.986 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.0189999999999995e-06 2024-09-14T06:29:21.986 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:21.986 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:21.986 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:29:21.986 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:04.297255+0000", 2024-09-14T06:29:21.986 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.7112000000000004e-05 2024-09-14T06:29:21.986 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:21.986 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:21.986 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:29:21.987 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:04.297279+0000", 2024-09-14T06:29:21.987 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.4017000000000001e-05 2024-09-14T06:29:21.987 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:21.987 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:21.987 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-14T06:29:21.987 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:04.297966+0000", 2024-09-14T06:29:21.987 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00068650600000000003 2024-09-14T06:29:21.987 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:21.987 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:21.987 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:29:21.987 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:04.298002+0000", 2024-09-14T06:29:21.987 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.6186999999999998e-05 2024-09-14T06:29:21.987 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:21.987 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:21.987 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:21.988 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:21.988 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:21.988 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4397.0:66 3.4 e27/23 3:27567f25:::rbd_object_map.112d7adb82ff:head v 27'6, mlcod=27'4)", 2024-09-14T06:29:21.988 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:04.303066+0000", 2024-09-14T06:29:21.988 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 17.666556893999999, 2024-09-14T06:29:21.988 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00083700699999999996, 2024-09-14T06:29:21.988 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:29:21.988 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:29:21.988 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:29:21.988 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:29:21.988 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:21.988 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:29:21.989 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:04.303066+0000", 2024-09-14T06:29:21.989 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:21.989 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:21.989 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:21.989 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:29:21.989 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:04.303066+0000", 2024-09-14T06:29:21.989 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:21.989 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:21.989 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:21.989 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:29:21.989 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:04.303067+0000", 2024-09-14T06:29:21.989 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.1880000000000001e-06 2024-09-14T06:29:21.989 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:21.989 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:21.990 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:29:21.990 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:04.303075+0000", 2024-09-14T06:29:21.990 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.1410000000000005e-06 2024-09-14T06:29:21.990 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:21.990 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:21.990 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:29:21.990 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:04.303091+0000", 2024-09-14T06:29:21.990 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.5928000000000001e-05 2024-09-14T06:29:21.990 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:21.990 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:21.990 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:29:21.990 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:04.303100+0000", 2024-09-14T06:29:21.990 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.2019999999999993e-06 2024-09-14T06:29:21.990 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:21.991 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:21.991 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:29:21.991 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:04.303168+0000", 2024-09-14T06:29:21.991 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.7754000000000002e-05 2024-09-14T06:29:21.991 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:21.991 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:21.991 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:29:21.991 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:04.303191+0000", 2024-09-14T06:29:21.991 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.3434e-05 2024-09-14T06:29:21.991 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:21.991 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:21.991 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-14T06:29:21.991 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:04.303872+0000", 2024-09-14T06:29:21.992 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00068037399999999997 2024-09-14T06:29:21.992 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:21.992 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:21.993 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:29:21.993 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:04.303903+0000", 2024-09-14T06:29:21.993 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.0985999999999998e-05 2024-09-14T06:29:21.993 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:21.993 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:21.994 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:21.994 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:21.994 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:21.994 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4397.0:68 3.4 e27/23 3:27567f25:::rbd_object_map.112d7adb82ff:head v 27'7, mlcod=27'5)", 2024-09-14T06:29:21.994 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:04.314113+0000", 2024-09-14T06:29:21.994 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 17.655510033999999, 2024-09-14T06:29:21.994 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00086905299999999995, 2024-09-14T06:29:21.994 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:29:21.995 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:29:21.995 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:29:21.995 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:29:21.995 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:21.995 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:29:21.995 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:04.314113+0000", 2024-09-14T06:29:21.995 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:21.995 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:21.995 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:21.995 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:29:21.995 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:04.314113+0000", 2024-09-14T06:29:21.995 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:21.995 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:21.996 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:21.996 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:29:21.996 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:04.314114+0000", 2024-09-14T06:29:21.996 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.2780000000000001e-06 2024-09-14T06:29:21.996 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:21.996 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:21.996 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:29:21.996 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:04.314133+0000", 2024-09-14T06:29:21.996 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.95e-05 2024-09-14T06:29:21.996 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:21.996 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:21.996 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:29:21.996 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:04.314149+0000", 2024-09-14T06:29:21.996 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.5752000000000001e-05 2024-09-14T06:29:21.996 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:21.997 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:21.997 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:29:21.997 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:04.314163+0000", 2024-09-14T06:29:21.997 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.3910000000000001e-05 2024-09-14T06:29:21.997 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:21.997 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:21.997 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:29:21.997 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:04.314234+0000", 2024-09-14T06:29:21.997 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.0564999999999999e-05 2024-09-14T06:29:21.997 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:21.997 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:21.998 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:29:21.998 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:04.314258+0000", 2024-09-14T06:29:21.998 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.4040999999999999e-05 2024-09-14T06:29:21.998 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:21.998 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:21.998 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-14T06:29:21.998 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:04.314950+0000", 2024-09-14T06:29:21.998 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00069187399999999998 2024-09-14T06:29:21.998 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:21.998 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:21.998 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:29:21.998 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:04.314982+0000", 2024-09-14T06:29:21.998 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.2132999999999999e-05 2024-09-14T06:29:21.998 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:21.998 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:21.999 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:21.999 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:21.999 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:21.999 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4397.0:114 3.4 e27/23 3:27567f25:::rbd_object_map.112d7adb82ff:head v 27'16, mlcod=27'14)", 2024-09-14T06:29:21.999 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:04.372433+0000", 2024-09-14T06:29:21.999 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 17.597189333999999, 2024-09-14T06:29:21.999 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00088299699999999995, 2024-09-14T06:29:21.999 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:29:21.999 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:29:21.999 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:29:21.999 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:29:21.999 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:21.999 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:29:21.999 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:04.372433+0000", 2024-09-14T06:29:21.999 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:22.000 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:22.000 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:22.000 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:29:22.000 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:04.372433+0000", 2024-09-14T06:29:22.000 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:22.000 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:22.000 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:22.000 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:29:22.000 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:04.372435+0000", 2024-09-14T06:29:22.000 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.655e-06 2024-09-14T06:29:22.000 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:22.000 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:22.000 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:29:22.000 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:04.372443+0000", 2024-09-14T06:29:22.000 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.6809999999999998e-06 2024-09-14T06:29:22.001 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:22.001 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:22.001 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:29:22.001 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:04.372458+0000", 2024-09-14T06:29:22.001 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.4941e-05 2024-09-14T06:29:22.001 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:22.001 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:22.001 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:29:22.001 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:04.372475+0000", 2024-09-14T06:29:22.001 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.6886000000000001e-05 2024-09-14T06:29:22.001 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:22.001 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:22.001 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:29:22.001 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:04.372530+0000", 2024-09-14T06:29:22.002 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.507e-05 2024-09-14T06:29:22.002 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:22.002 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:22.002 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:29:22.002 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:04.372555+0000", 2024-09-14T06:29:22.002 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.5785000000000001e-05 2024-09-14T06:29:22.002 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:22.002 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:22.002 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-14T06:29:22.002 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:04.373275+0000", 2024-09-14T06:29:22.002 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00071917599999999997 2024-09-14T06:29:22.002 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:22.002 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:22.002 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:29:22.002 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:04.373316+0000", 2024-09-14T06:29:22.003 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.1802999999999999e-05 2024-09-14T06:29:22.003 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:22.003 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:22.003 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:22.003 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:22.003 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:22.003 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4397.0:116 3.4 e27/23 3:27567f25:::rbd_object_map.112d7adb82ff:head v 27'17, mlcod=27'15)", 2024-09-14T06:29:22.003 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:04.384169+0000", 2024-09-14T06:29:22.003 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 17.585453823999998, 2024-09-14T06:29:22.003 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00083693600000000004, 2024-09-14T06:29:22.003 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:29:22.003 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:29:22.003 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:29:22.003 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:29:22.003 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:22.003 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:29:22.004 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:04.384169+0000", 2024-09-14T06:29:22.004 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:22.004 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:22.004 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:22.004 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:29:22.004 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:04.384169+0000", 2024-09-14T06:29:22.004 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:22.004 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:22.004 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:22.004 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:29:22.004 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:04.384170+0000", 2024-09-14T06:29:22.004 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.359e-06 2024-09-14T06:29:22.005 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:22.005 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:22.005 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:29:22.005 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:04.384179+0000", 2024-09-14T06:29:22.005 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.1919999999999994e-06 2024-09-14T06:29:22.006 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:22.006 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:22.006 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:29:22.006 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:04.384198+0000", 2024-09-14T06:29:22.006 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.8879999999999999e-05 2024-09-14T06:29:22.006 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:22.007 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:22.007 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:29:22.007 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:04.384208+0000", 2024-09-14T06:29:22.007 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.3300000000000005e-06 2024-09-14T06:29:22.007 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:22.007 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:22.008 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:29:22.008 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:04.384273+0000", 2024-09-14T06:29:22.008 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.5275000000000001e-05 2024-09-14T06:29:22.008 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:22.008 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:22.008 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:29:22.009 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:04.384291+0000", 2024-09-14T06:29:22.009 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.8139999999999999e-05 2024-09-14T06:29:22.009 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:22.009 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:22.009 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-14T06:29:22.009 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:04.384963+0000", 2024-09-14T06:29:22.009 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00067157199999999995 2024-09-14T06:29:22.010 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:22.010 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:22.010 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:29:22.010 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:04.385006+0000", 2024-09-14T06:29:22.010 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.3188000000000001e-05 2024-09-14T06:29:22.010 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:22.010 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:22.011 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:22.011 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:22.011 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:22.011 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4397.0:130 3.4 e27/23 3:27567f25:::rbd_object_map.112d7adb82ff:head v 27'21, mlcod=27'19)", 2024-09-14T06:29:22.011 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:04.405039+0000", 2024-09-14T06:29:22.011 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 17.564583207999998, 2024-09-14T06:29:22.012 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000900945, 2024-09-14T06:29:22.012 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:29:22.012 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:29:22.012 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:29:22.012 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:29:22.012 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:22.012 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:29:22.012 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:04.405039+0000", 2024-09-14T06:29:22.013 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:22.013 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:22.013 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:22.013 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:29:22.013 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:04.405039+0000", 2024-09-14T06:29:22.013 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:22.013 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:22.014 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:22.014 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:29:22.014 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:04.405041+0000", 2024-09-14T06:29:22.014 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.516e-06 2024-09-14T06:29:22.014 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:22.014 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:22.014 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:29:22.015 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:04.405054+0000", 2024-09-14T06:29:22.015 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.3383999999999999e-05 2024-09-14T06:29:22.015 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:22.015 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:22.015 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:29:22.015 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:04.405073+0000", 2024-09-14T06:29:22.016 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.8598e-05 2024-09-14T06:29:22.016 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:22.016 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:22.016 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:29:22.016 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:04.405082+0000", 2024-09-14T06:29:22.016 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.4390000000000006e-06 2024-09-14T06:29:22.017 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:22.017 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:22.017 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:29:22.017 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:04.405164+0000", 2024-09-14T06:29:22.017 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.1358000000000003e-05 2024-09-14T06:29:22.018 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:22.018 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:22.018 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:29:22.018 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:04.405192+0000", 2024-09-14T06:29:22.018 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.8648000000000001e-05 2024-09-14T06:29:22.018 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:22.019 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:22.019 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-14T06:29:22.019 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:04.405897+0000", 2024-09-14T06:29:22.019 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00070438900000000004 2024-09-14T06:29:22.019 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:22.020 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:22.020 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:29:22.020 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:04.405940+0000", 2024-09-14T06:29:22.020 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.3612999999999997e-05 2024-09-14T06:29:22.020 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:22.020 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:22.021 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:22.021 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:22.021 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:22.021 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4397.0:164 3.4 e27/23 3:27567f25:::rbd_object_map.112d7adb82ff:head v 27'27, mlcod=27'25)", 2024-09-14T06:29:22.021 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:04.438987+0000", 2024-09-14T06:29:22.021 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 17.530635315000001, 2024-09-14T06:29:22.021 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00093266400000000004, 2024-09-14T06:29:22.021 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:29:22.022 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:29:22.022 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:29:22.022 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:29:22.022 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:22.022 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:29:22.022 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:04.438987+0000", 2024-09-14T06:29:22.022 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:22.022 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:22.022 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:22.023 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:29:22.023 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:04.438987+0000", 2024-09-14T06:29:22.023 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:22.023 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:22.023 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:22.023 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:29:22.023 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:04.438989+0000", 2024-09-14T06:29:22.023 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.1829999999999999e-06 2024-09-14T06:29:22.024 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:22.024 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:22.024 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:29:22.024 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:04.438998+0000", 2024-09-14T06:29:22.024 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.5489999999999995e-06 2024-09-14T06:29:22.024 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:22.024 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:22.025 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:29:22.025 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:04.439014+0000", 2024-09-14T06:29:22.025 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.5659e-05 2024-09-14T06:29:22.025 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:22.025 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:22.025 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:29:22.025 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:04.439024+0000", 2024-09-14T06:29:22.026 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.0229e-05 2024-09-14T06:29:22.026 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:22.026 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:22.026 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:29:22.026 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:04.439085+0000", 2024-09-14T06:29:22.026 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.1507000000000006e-05 2024-09-14T06:29:22.026 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:22.027 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:22.027 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:29:22.027 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:04.439111+0000", 2024-09-14T06:29:22.027 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.5479000000000001e-05 2024-09-14T06:29:22.027 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:22.027 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:22.027 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-14T06:29:22.027 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:04.439872+0000", 2024-09-14T06:29:22.027 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00076089899999999999 2024-09-14T06:29:22.027 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:22.027 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:22.027 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:29:22.027 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:04.439920+0000", 2024-09-14T06:29:22.027 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.8158999999999997e-05 2024-09-14T06:29:22.028 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:22.028 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:22.028 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:22.028 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:22.028 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:22.028 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4397.0:186 3.4 e27/23 3:27567f25:::rbd_object_map.112d7adb82ff:head v 27'31, mlcod=27'29)", 2024-09-14T06:29:22.028 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:04.457782+0000", 2024-09-14T06:29:22.028 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 17.51184018, 2024-09-14T06:29:22.028 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00083834100000000004, 2024-09-14T06:29:22.028 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:29:22.028 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:29:22.028 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:29:22.028 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:29:22.029 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:22.029 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:29:22.029 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:04.457782+0000", 2024-09-14T06:29:22.029 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:22.029 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:22.029 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:22.029 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:29:22.029 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:04.457782+0000", 2024-09-14T06:29:22.029 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:22.029 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:22.029 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:22.029 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:29:22.029 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:04.457783+0000", 2024-09-14T06:29:22.029 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.001e-06 2024-09-14T06:29:22.030 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:22.030 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:22.030 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:29:22.030 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:04.457801+0000", 2024-09-14T06:29:22.030 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.7119999999999999e-05 2024-09-14T06:29:22.030 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:22.030 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:22.030 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:29:22.030 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:04.457817+0000", 2024-09-14T06:29:22.030 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.6761000000000001e-05 2024-09-14T06:29:22.030 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:22.030 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:22.031 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:29:22.031 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:04.457826+0000", 2024-09-14T06:29:22.031 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.3059999999999998e-06 2024-09-14T06:29:22.031 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:22.031 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:22.031 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:29:22.031 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:04.457883+0000", 2024-09-14T06:29:22.031 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.7420000000000003e-05 2024-09-14T06:29:22.031 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:22.031 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:22.031 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:29:22.031 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:04.457906+0000", 2024-09-14T06:29:22.032 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.2643e-05 2024-09-14T06:29:22.032 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:22.032 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:22.032 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-14T06:29:22.032 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:04.458570+0000", 2024-09-14T06:29:22.032 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00066450099999999996 2024-09-14T06:29:22.032 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:22.032 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:22.032 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:29:22.032 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:04.458621+0000", 2024-09-14T06:29:22.032 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.0589e-05 2024-09-14T06:29:22.032 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:22.033 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:22.033 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:22.033 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:22.033 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:22.033 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4397.0:616 3.9 e30/23 3:908d0935:::rbd_data.112d7dfbd766.000000000000007e:head v 30'2, mlcod=0'0)", 2024-09-14T06:29:22.033 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:06.780526+0000", 2024-09-14T06:29:22.033 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 15.189096405000001, 2024-09-14T06:29:22.033 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00091880900000000003, 2024-09-14T06:29:22.033 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:29:22.033 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:29:22.033 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:29:22.033 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:29:22.033 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:22.033 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:29:22.034 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:06.780526+0000", 2024-09-14T06:29:22.034 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:22.034 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:22.034 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:22.034 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:29:22.034 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:06.780526+0000", 2024-09-14T06:29:22.034 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:22.034 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:22.034 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:22.034 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:29:22.034 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:06.780527+0000", 2024-09-14T06:29:22.034 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.3799999999999998e-07 2024-09-14T06:29:22.035 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:22.035 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:22.035 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:29:22.035 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:06.780625+0000", 2024-09-14T06:29:22.035 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.8785999999999999e-05 2024-09-14T06:29:22.035 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:22.035 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:22.035 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:29:22.035 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:06.780640+0000", 2024-09-14T06:29:22.035 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.4363e-05 2024-09-14T06:29:22.035 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:22.035 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:22.035 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:29:22.035 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:06.780650+0000", 2024-09-14T06:29:22.036 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.0164e-05 2024-09-14T06:29:22.036 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:22.036 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:22.036 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:29:22.036 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:06.780702+0000", 2024-09-14T06:29:22.036 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.1992e-05 2024-09-14T06:29:22.036 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:22.036 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:22.037 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:29:22.037 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:06.780718+0000", 2024-09-14T06:29:22.037 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.6161e-05 2024-09-14T06:29:22.037 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:22.037 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:22.037 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-14T06:29:22.037 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:06.781404+0000", 2024-09-14T06:29:22.037 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000685526 2024-09-14T06:29:22.038 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:22.038 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:22.038 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:29:22.038 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:06.781445+0000", 2024-09-14T06:29:22.038 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.1479e-05 2024-09-14T06:29:22.038 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:22.038 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:22.038 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:22.038 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:22.038 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:22.038 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4397.0:792 3.9 e30/23 3:977d4c6a:::rbd_data.112d7dfbd766.000000000000014d:head v 30'4, mlcod=30'2)", 2024-09-14T06:29:22.038 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:06.998893+0000", 2024-09-14T06:29:22.038 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 14.970729716999999, 2024-09-14T06:29:22.039 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00089658700000000001, 2024-09-14T06:29:22.039 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:29:22.039 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:29:22.039 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:29:22.039 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:29:22.039 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:22.039 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:29:22.039 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:06.998893+0000", 2024-09-14T06:29:22.039 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:22.039 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:22.039 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:22.039 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:29:22.039 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:06.998893+0000", 2024-09-14T06:29:22.039 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:22.039 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:22.040 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:22.040 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:29:22.040 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:06.998894+0000", 2024-09-14T06:29:22.040 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.2550000000000001e-06 2024-09-14T06:29:22.040 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:22.040 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:22.040 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:29:22.040 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:06.998912+0000", 2024-09-14T06:29:22.040 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.7818000000000001e-05 2024-09-14T06:29:22.040 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:22.040 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:22.040 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:29:22.040 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:06.998926+0000", 2024-09-14T06:29:22.041 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.4368e-05 2024-09-14T06:29:22.041 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:22.041 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:22.041 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:29:22.041 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:06.998934+0000", 2024-09-14T06:29:22.041 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.6839999999999996e-06 2024-09-14T06:29:22.041 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:22.041 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:22.041 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:29:22.041 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:06.998981+0000", 2024-09-14T06:29:22.041 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.6995e-05 2024-09-14T06:29:22.041 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:22.041 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:22.042 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:29:22.042 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:06.998998+0000", 2024-09-14T06:29:22.042 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.6628000000000001e-05 2024-09-14T06:29:22.042 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:22.042 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:22.042 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-14T06:29:22.042 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:06.999757+0000", 2024-09-14T06:29:22.042 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00075925099999999996 2024-09-14T06:29:22.042 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:22.042 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:22.042 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:29:22.042 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:06.999790+0000", 2024-09-14T06:29:22.042 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.2588e-05 2024-09-14T06:29:22.042 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:22.043 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:22.043 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:22.043 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:22.043 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:22.043 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4397.0:937 3.9 e30/23 3:92b37d3e:::rbd_data.112d7dfbd766.0000000000000012:head v 30'7, mlcod=30'5)", 2024-09-14T06:29:22.043 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:07.169523+0000", 2024-09-14T06:29:22.043 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 14.800099849, 2024-09-14T06:29:22.043 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0013381339999999999, 2024-09-14T06:29:22.043 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:29:22.043 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:29:22.043 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:29:22.043 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:29:22.043 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:22.043 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:29:22.044 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:07.169523+0000", 2024-09-14T06:29:22.044 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:22.044 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:22.044 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:22.044 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:29:22.044 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:07.169523+0000", 2024-09-14T06:29:22.044 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:22.044 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:22.044 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:22.044 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:29:22.044 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:07.169524+0000", 2024-09-14T06:29:22.044 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.125e-06 2024-09-14T06:29:22.044 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:22.044 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:22.045 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:29:22.045 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:07.170106+0000", 2024-09-14T06:29:22.045 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00058239900000000005 2024-09-14T06:29:22.045 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:22.045 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:22.045 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:29:22.045 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:07.170121+0000", 2024-09-14T06:29:22.045 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.4851999999999999e-05 2024-09-14T06:29:22.045 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:22.045 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:22.045 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:29:22.045 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:07.170130+0000", 2024-09-14T06:29:22.045 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.9199999999999993e-06 2024-09-14T06:29:22.046 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:22.046 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:22.046 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:29:22.046 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:07.170182+0000", 2024-09-14T06:29:22.046 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.1669999999999998e-05 2024-09-14T06:29:22.046 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:22.046 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:22.046 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:29:22.046 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:07.170199+0000", 2024-09-14T06:29:22.046 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.7193999999999998e-05 2024-09-14T06:29:22.046 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:22.046 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:22.046 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-14T06:29:22.046 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:07.170816+0000", 2024-09-14T06:29:22.047 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00061745400000000003 2024-09-14T06:29:22.047 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:22.047 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:22.047 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:29:22.047 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:07.170861+0000", 2024-09-14T06:29:22.047 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.4520000000000001e-05 2024-09-14T06:29:22.047 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:22.047 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:22.047 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:22.047 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:22.047 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:22.047 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4397.0:1634 3.0 e32/23 3:01b99234:::rbd_data.112dcdddac74.0000000000000009:head v 32'2, mlcod=0'0)", 2024-09-14T06:29:22.047 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:08.644163+0000", 2024-09-14T06:29:22.048 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 13.325460049, 2024-09-14T06:29:22.048 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0010247730000000001, 2024-09-14T06:29:22.048 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:29:22.048 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:29:22.048 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:29:22.048 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:29:22.048 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:22.048 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:29:22.048 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.644163+0000", 2024-09-14T06:29:22.048 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:22.048 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:22.048 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:22.048 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:29:22.048 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.644163+0000", 2024-09-14T06:29:22.048 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:22.049 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:22.049 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:22.049 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:29:22.049 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.644163+0000", 2024-09-14T06:29:22.049 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.7800000000000002e-07 2024-09-14T06:29:22.049 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:22.049 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:22.049 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:29:22.049 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.644210+0000", 2024-09-14T06:29:22.049 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.7240000000000002e-05 2024-09-14T06:29:22.049 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:22.049 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:22.049 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:29:22.050 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.644245+0000", 2024-09-14T06:29:22.050 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.4288000000000001e-05 2024-09-14T06:29:22.050 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:22.050 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:22.050 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:29:22.050 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.644254+0000", 2024-09-14T06:29:22.050 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.3719999999999994e-06 2024-09-14T06:29:22.050 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:22.050 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:22.050 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:29:22.050 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.644338+0000", 2024-09-14T06:29:22.050 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.3892000000000002e-05 2024-09-14T06:29:22.050 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:22.050 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:22.051 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:29:22.051 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.644364+0000", 2024-09-14T06:29:22.051 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.5805000000000001e-05 2024-09-14T06:29:22.051 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:22.051 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:22.051 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-14T06:29:22.051 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.645151+0000", 2024-09-14T06:29:22.051 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00078698199999999998 2024-09-14T06:29:22.051 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:22.051 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:22.051 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:29:22.051 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.645187+0000", 2024-09-14T06:29:22.051 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.6816000000000001e-05 2024-09-14T06:29:22.052 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:22.052 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:22.052 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:22.052 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:22.052 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:22.052 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4397.0:1848 3.4 e32/23 3:27620a21:::rbd_data.112dcdddac74.000000000000000b:head v 32'40, mlcod=30'38)", 2024-09-14T06:29:22.052 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:08.658649+0000", 2024-09-14T06:29:22.052 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 13.310973476000001, 2024-09-14T06:29:22.052 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.001862867, 2024-09-14T06:29:22.052 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:29:22.052 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:29:22.053 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:29:22.053 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:29:22.053 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:22.053 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:29:22.053 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.658649+0000", 2024-09-14T06:29:22.053 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:22.053 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:22.053 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:22.053 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:29:22.053 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.658649+0000", 2024-09-14T06:29:22.053 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:22.053 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:22.053 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:22.054 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:29:22.054 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.658651+0000", 2024-09-14T06:29:22.054 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.5650000000000001e-06 2024-09-14T06:29:22.054 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:22.054 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:22.054 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:29:22.054 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.658709+0000", 2024-09-14T06:29:22.054 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.8442000000000001e-05 2024-09-14T06:29:22.054 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:22.054 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:22.054 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:29:22.054 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.658730+0000", 2024-09-14T06:29:22.054 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.0509999999999998e-05 2024-09-14T06:29:22.054 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:22.055 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:22.055 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:29:22.055 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.658740+0000", 2024-09-14T06:29:22.055 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.8619999999999998e-06 2024-09-14T06:29:22.055 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:22.055 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:22.055 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:29:22.055 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.658813+0000", 2024-09-14T06:29:22.055 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.3578000000000006e-05 2024-09-14T06:29:22.055 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:22.055 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:22.055 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:29:22.055 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.658836+0000", 2024-09-14T06:29:22.055 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.2677e-05 2024-09-14T06:29:22.056 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:22.056 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:22.056 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-14T06:29:22.056 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.660462+0000", 2024-09-14T06:29:22.056 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00162637 2024-09-14T06:29:22.056 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:22.056 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:22.056 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:29:22.056 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.660512+0000", 2024-09-14T06:29:22.056 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.9863e-05 2024-09-14T06:29:22.056 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:22.056 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:22.056 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:22.056 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:22.057 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:22.057 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4397.0:1941 3.9 e32/23 3:92dc473f:::rbd_data.112dcdddac74.000000000000000c:head v 32'8, mlcod=30'6)", 2024-09-14T06:29:22.057 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:08.664245+0000", 2024-09-14T06:29:22.057 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 13.30537816, 2024-09-14T06:29:22.057 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.001294746, 2024-09-14T06:29:22.057 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:29:22.057 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:29:22.057 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:29:22.057 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:29:22.057 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:22.057 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:29:22.057 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.664245+0000", 2024-09-14T06:29:22.057 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:22.057 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:22.058 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:22.058 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:29:22.058 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.664245+0000", 2024-09-14T06:29:22.058 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:22.058 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:22.058 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:22.058 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:29:22.058 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.664245+0000", 2024-09-14T06:29:22.058 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.0999999999999999e-07 2024-09-14T06:29:22.058 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:22.058 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:22.058 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:29:22.058 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.664408+0000", 2024-09-14T06:29:22.058 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00016308800000000001 2024-09-14T06:29:22.059 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:22.059 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:22.059 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:29:22.059 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.664421+0000", 2024-09-14T06:29:22.059 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.3097e-05 2024-09-14T06:29:22.059 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:22.059 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:22.059 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:29:22.059 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.664429+0000", 2024-09-14T06:29:22.059 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.943e-06 2024-09-14T06:29:22.059 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:22.059 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:22.060 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:29:22.060 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.664511+0000", 2024-09-14T06:29:22.060 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.1945000000000002e-05 2024-09-14T06:29:22.060 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:22.060 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:22.060 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:29:22.061 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.664532+0000", 2024-09-14T06:29:22.061 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.0931999999999999e-05 2024-09-14T06:29:22.061 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:22.061 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:22.061 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-14T06:29:22.061 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.665490+0000", 2024-09-14T06:29:22.062 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00095764400000000001 2024-09-14T06:29:22.062 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:22.062 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:22.062 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:29:22.062 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.665539+0000", 2024-09-14T06:29:22.062 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.9687e-05 2024-09-14T06:29:22.063 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:22.063 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:22.063 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:22.063 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:22.063 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:22.063 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4397.0:2199 3.0 e32/23 3:05f3e980:::rbd_data.112dcdddac74.000000000000000f:head v 32'3, mlcod=31'1)", 2024-09-14T06:29:22.064 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:08.680236+0000", 2024-09-14T06:29:22.064 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 13.289387111, 2024-09-14T06:29:22.064 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0019068990000000001, 2024-09-14T06:29:22.064 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:29:22.065 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:29:22.065 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:29:22.065 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:29:22.065 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:22.065 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:29:22.065 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.680236+0000", 2024-09-14T06:29:22.065 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:22.065 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:22.065 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:22.065 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:29:22.065 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.680236+0000", 2024-09-14T06:29:22.065 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:22.066 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:22.066 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:22.066 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:29:22.066 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.680237+0000", 2024-09-14T06:29:22.066 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.4980000000000001e-06 2024-09-14T06:29:22.066 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:22.066 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:22.066 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:29:22.066 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.680324+0000", 2024-09-14T06:29:22.066 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.7310000000000003e-05 2024-09-14T06:29:22.066 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:22.066 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:22.067 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:29:22.067 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.680354+0000", 2024-09-14T06:29:22.067 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.97e-05 2024-09-14T06:29:22.067 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:22.067 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:22.067 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:29:22.067 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.680366+0000", 2024-09-14T06:29:22.068 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.2048999999999999e-05 2024-09-14T06:29:22.068 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:22.068 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:22.068 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:29:22.068 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.680439+0000", 2024-09-14T06:29:22.068 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.2910000000000005e-05 2024-09-14T06:29:22.068 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:22.068 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:22.068 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:29:22.068 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.680465+0000", 2024-09-14T06:29:22.068 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.5812999999999998e-05 2024-09-14T06:29:22.068 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:22.068 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:22.069 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-14T06:29:22.069 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.682094+0000", 2024-09-14T06:29:22.069 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.001628876 2024-09-14T06:29:22.069 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:22.069 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:22.069 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:29:22.069 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.682142+0000", 2024-09-14T06:29:22.069 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.8742999999999997e-05 2024-09-14T06:29:22.069 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:22.069 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:22.069 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:22.069 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:22.069 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:22.070 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4397.0:2513 3.0 e32/23 3:06f82ba7:::rbd_data.112dcdddac74.0000000000000028:head v 32'4, mlcod=32'2)", 2024-09-14T06:29:22.070 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:08.703547+0000", 2024-09-14T06:29:22.070 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 13.266075507, 2024-09-14T06:29:22.070 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0013041859999999999, 2024-09-14T06:29:22.070 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:29:22.070 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:29:22.070 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:29:22.070 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:29:22.071 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:22.071 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:29:22.071 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.703547+0000", 2024-09-14T06:29:22.071 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:22.071 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:22.071 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:22.072 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:29:22.072 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.703547+0000", 2024-09-14T06:29:22.072 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:22.072 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:22.072 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:22.072 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:29:22.072 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.703549+0000", 2024-09-14T06:29:22.073 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.39e-06 2024-09-14T06:29:22.073 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:22.073 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:22.073 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:29:22.073 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.703666+0000", 2024-09-14T06:29:22.073 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000117314 2024-09-14T06:29:22.073 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:22.074 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:22.074 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:29:22.074 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.703685+0000", 2024-09-14T06:29:22.074 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.8984999999999999e-05 2024-09-14T06:29:22.074 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:22.074 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:22.074 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:29:22.074 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.703697+0000", 2024-09-14T06:29:22.075 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.2405000000000001e-05 2024-09-14T06:29:22.075 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:22.075 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:22.075 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:29:22.075 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.703765+0000", 2024-09-14T06:29:22.075 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.7595000000000006e-05 2024-09-14T06:29:22.075 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:22.076 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:22.076 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:29:22.076 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.703788+0000", 2024-09-14T06:29:22.076 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.3300000000000001e-05 2024-09-14T06:29:22.076 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:22.077 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:22.077 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-14T06:29:22.077 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.704801+0000", 2024-09-14T06:29:22.077 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0010130339999999999 2024-09-14T06:29:22.077 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:22.077 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:22.078 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:29:22.078 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.704851+0000", 2024-09-14T06:29:22.078 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.0163000000000001e-05 2024-09-14T06:29:22.078 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:22.078 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:22.078 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:22.078 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:22.078 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:22.078 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4397.0:2885 3.9 e32/23 3:974aae1b:::rbd_data.112dcdddac74.000000000000002d:head v 32'9, mlcod=30'7)", 2024-09-14T06:29:22.079 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:08.717100+0000", 2024-09-14T06:29:22.079 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 13.252522893, 2024-09-14T06:29:22.079 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0012528229999999999, 2024-09-14T06:29:22.079 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:29:22.079 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:29:22.079 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:29:22.079 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:29:22.079 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:22.079 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:29:22.079 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.717100+0000", 2024-09-14T06:29:22.080 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:22.080 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:22.080 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:22.080 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:29:22.080 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.717100+0000", 2024-09-14T06:29:22.080 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:22.080 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:22.080 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:22.080 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:29:22.080 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.717101+0000", 2024-09-14T06:29:22.080 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.2929999999999999e-06 2024-09-14T06:29:22.080 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:22.080 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:22.081 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:29:22.081 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.717270+0000", 2024-09-14T06:29:22.081 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00016929799999999999 2024-09-14T06:29:22.081 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:22.081 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:22.081 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:29:22.081 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.717283+0000", 2024-09-14T06:29:22.081 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.2483999999999999e-05 2024-09-14T06:29:22.081 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:22.081 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:22.081 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:29:22.081 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.717292+0000", 2024-09-14T06:29:22.081 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.7220000000000005e-06 2024-09-14T06:29:22.081 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:22.082 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:22.082 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:29:22.082 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.717356+0000", 2024-09-14T06:29:22.082 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.4055999999999998e-05 2024-09-14T06:29:22.082 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:22.082 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:22.082 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:29:22.082 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.717383+0000", 2024-09-14T06:29:22.082 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.7274999999999999e-05 2024-09-14T06:29:22.082 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:22.082 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:22.082 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-14T06:29:22.082 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.718318+0000", 2024-09-14T06:29:22.082 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00093470100000000002 2024-09-14T06:29:22.083 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:22.083 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:22.083 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:29:22.083 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.718353+0000", 2024-09-14T06:29:22.083 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.4993999999999997e-05 2024-09-14T06:29:22.083 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:22.083 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:22.083 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:22.083 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:22.083 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:22.083 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4397.0:3292 3.9 e32/23 3:91dbce57:::rbd_data.112dcdddac74.0000000000000036:head v 32'10, mlcod=32'8)", 2024-09-14T06:29:22.083 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:08.740506+0000", 2024-09-14T06:29:22.083 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 13.229116399, 2024-09-14T06:29:22.084 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.001864166, 2024-09-14T06:29:22.084 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:29:22.084 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:29:22.084 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:29:22.084 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:29:22.084 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:22.084 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:29:22.084 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.740506+0000", 2024-09-14T06:29:22.084 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:22.084 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:22.084 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:22.084 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:29:22.084 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.740506+0000", 2024-09-14T06:29:22.084 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:22.085 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:22.085 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:22.085 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:29:22.085 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.740514+0000", 2024-09-14T06:29:22.085 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.2109999999999998e-06 2024-09-14T06:29:22.085 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:22.085 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:22.085 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:29:22.085 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.740745+0000", 2024-09-14T06:29:22.085 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000230194 2024-09-14T06:29:22.085 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:22.085 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:22.085 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:29:22.086 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.740764+0000", 2024-09-14T06:29:22.086 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.9330999999999998e-05 2024-09-14T06:29:22.086 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:22.086 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:22.086 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:29:22.086 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.740774+0000", 2024-09-14T06:29:22.086 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.6260000000000007e-06 2024-09-14T06:29:22.086 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:22.086 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:22.086 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:29:22.086 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.740839+0000", 2024-09-14T06:29:22.086 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.5314000000000005e-05 2024-09-14T06:29:22.086 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:22.086 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:22.086 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:29:22.087 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.740860+0000", 2024-09-14T06:29:22.087 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.0868000000000001e-05 2024-09-14T06:29:22.087 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:22.087 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:22.087 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-14T06:29:22.087 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.742327+0000", 2024-09-14T06:29:22.087 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0014672909999999999 2024-09-14T06:29:22.087 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:22.087 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:22.087 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:29:22.087 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.742370+0000", 2024-09-14T06:29:22.087 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.3331000000000002e-05 2024-09-14T06:29:22.087 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:22.087 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:22.088 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:22.088 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:22.088 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:22.088 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-14T06:29:22.089 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-14T06:29:22.090 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 reset_purged_snaps_last 2024-09-14T06:29:22.091 INFO:tasks.ceph.osd.1.smithi094.stderr:2024-09-14T06:29:22.088+0000 7f328e47a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33641) UID: 0 2024-09-14T06:29:22.190 INFO:tasks.ceph.osd.6.smithi138.stderr:2024-09-14T06:29:22.188+0000 7f8cd8849640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33620) UID: 0 2024-09-14T06:29:22.237 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-14T06:29:22.237 INFO:teuthology.orchestra.run.smithi094.stdout: "ops": [], 2024-09-14T06:29:22.238 INFO:teuthology.orchestra.run.smithi094.stdout: "num_ops": 0 2024-09-14T06:29:22.238 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-14T06:29:22.248 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-14T06:29:22.290 INFO:tasks.ceph.osd.6.smithi138.stderr:2024-09-14T06:29:22.289+0000 7f8cd8849640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33620) UID: 0 2024-09-14T06:29:22.352 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-14T06:29:22.352 INFO:teuthology.orchestra.run.smithi094.stdout: "ops": [], 2024-09-14T06:29:22.352 INFO:teuthology.orchestra.run.smithi094.stdout: "complaint_time": 30, 2024-09-14T06:29:22.352 INFO:teuthology.orchestra.run.smithi094.stdout: "num_blocked_ops": 0 2024-09-14T06:29:22.352 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-14T06:29:22.361 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-14T06:29:22.391 INFO:tasks.ceph.osd.6.smithi138.stderr:2024-09-14T06:29:22.389+0000 7f8cd8849640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33620) UID: 0 2024-09-14T06:29:22.464 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-14T06:29:22.465 INFO:teuthology.orchestra.run.smithi094.stdout: "size": 20, 2024-09-14T06:29:22.465 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 600, 2024-09-14T06:29:22.465 INFO:teuthology.orchestra.run.smithi094.stdout: "ops": [ 2024-09-14T06:29:22.465 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:22.465 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:936 3.8 3:17df59d8:::rbd_data.112d7dfbd766.0000000000000011:head [set-alloc-hint object_size 524288 write_size 524288,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-14T06:29:22.465 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:07.164453+0000", 2024-09-14T06:29:22.465 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 15.29848282, 2024-09-14T06:29:22.465 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.013369270000000001, 2024-09-14T06:29:22.465 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:29:22.465 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:29:22.465 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:29:22.465 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:29:22.466 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:29:22.466 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:22.466 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 936 2024-09-14T06:29:22.466 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:22.466 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:29:22.466 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:22.466 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:29:22.466 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:07.164453+0000", 2024-09-14T06:29:22.466 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:22.466 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:22.466 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:22.466 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:29:22.466 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:07.164453+0000", 2024-09-14T06:29:22.467 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:22.467 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:22.467 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:22.467 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:29:22.467 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:07.164454+0000", 2024-09-14T06:29:22.467 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.8190000000000001e-06 2024-09-14T06:29:22.467 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:22.467 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:22.467 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:29:22.467 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:07.164463+0000", 2024-09-14T06:29:22.467 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.9900000000000003e-06 2024-09-14T06:29:22.467 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:22.467 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:22.467 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:29:22.467 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:07.164479+0000", 2024-09-14T06:29:22.468 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.5654e-05 2024-09-14T06:29:22.468 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:22.468 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:22.468 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:29:22.468 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:07.164487+0000", 2024-09-14T06:29:22.468 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.435e-06 2024-09-14T06:29:22.468 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:22.468 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:22.469 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:29:22.469 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:07.164537+0000", 2024-09-14T06:29:22.469 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.0175e-05 2024-09-14T06:29:22.469 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:22.469 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:22.469 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:29:22.469 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:07.164626+0000", 2024-09-14T06:29:22.469 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.9272999999999997e-05 2024-09-14T06:29:22.469 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:22.469 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:22.470 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "waiting for subops from 1", 2024-09-14T06:29:22.470 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:07.167461+0000", 2024-09-14T06:29:22.470 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0028350369999999999 2024-09-14T06:29:22.470 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:22.470 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:22.470 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "op_commit", 2024-09-14T06:29:22.470 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:07.177674+0000", 2024-09-14T06:29:22.470 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.01021267 2024-09-14T06:29:22.470 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:22.470 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:22.470 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:29:22.470 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:07.177728+0000", 2024-09-14T06:29:22.470 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.3829000000000002e-05 2024-09-14T06:29:22.470 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:22.471 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:22.471 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-14T06:29:22.471 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:07.177766+0000", 2024-09-14T06:29:22.471 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.8130000000000003e-05 2024-09-14T06:29:22.471 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:22.471 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:22.471 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:29:22.471 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:07.177822+0000", 2024-09-14T06:29:22.471 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.6258000000000002e-05 2024-09-14T06:29:22.471 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:22.471 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:22.471 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:22.471 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:22.471 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:22.472 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:2823 3.7 3:e68f0b27:::rbd_data.112d7dfbd766.000000000000006d:5 [read 237568~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-14T06:29:22.472 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:08.711240+0000", 2024-09-14T06:29:22.472 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 13.751695035000001, 2024-09-14T06:29:22.472 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.011935549, 2024-09-14T06:29:22.472 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:29:22.472 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:29:22.472 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:29:22.472 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:29:22.472 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:29:22.472 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:22.472 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 2823 2024-09-14T06:29:22.472 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:22.472 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:29:22.472 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:22.473 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:29:22.473 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.711240+0000", 2024-09-14T06:29:22.473 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:22.473 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:22.473 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:22.473 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:29:22.473 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.711240+0000", 2024-09-14T06:29:22.473 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:22.473 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:22.473 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:22.473 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:29:22.473 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.711241+0000", 2024-09-14T06:29:22.473 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.7899999999999999e-07 2024-09-14T06:29:22.473 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:22.474 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:22.474 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:29:22.474 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.711243+0000", 2024-09-14T06:29:22.474 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.939e-06 2024-09-14T06:29:22.474 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:22.474 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:22.474 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:29:22.474 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.711252+0000", 2024-09-14T06:29:22.474 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.4360000000000008e-06 2024-09-14T06:29:22.474 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:22.474 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:22.474 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:29:22.474 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.711257+0000", 2024-09-14T06:29:22.474 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.8790000000000002e-06 2024-09-14T06:29:22.475 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:22.475 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:22.475 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:29:22.475 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.722967+0000", 2024-09-14T06:29:22.475 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.011709514000000001 2024-09-14T06:29:22.475 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:22.475 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:22.475 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:29:22.475 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.723039+0000", 2024-09-14T06:29:22.475 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.2460999999999996e-05 2024-09-14T06:29:22.475 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:22.475 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:22.475 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:29:22.475 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.723176+0000", 2024-09-14T06:29:22.475 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00013684100000000001 2024-09-14T06:29:22.476 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:22.476 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:22.476 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:22.476 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:22.476 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:22.476 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:2824 3.7 3:e68f0b27:::rbd_data.112d7dfbd766.000000000000006d:5 [read 253952~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-14T06:29:22.476 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:08.711278+0000", 2024-09-14T06:29:22.476 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 13.751657558, 2024-09-14T06:29:22.476 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.012049709, 2024-09-14T06:29:22.476 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:29:22.476 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:29:22.476 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:29:22.476 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:29:22.477 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:29:22.477 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:22.477 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 2824 2024-09-14T06:29:22.477 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:22.477 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:29:22.477 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:22.477 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:29:22.477 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.711278+0000", 2024-09-14T06:29:22.477 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:22.477 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:22.477 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:22.477 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:29:22.477 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.711278+0000", 2024-09-14T06:29:22.477 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:22.477 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:22.478 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:22.478 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:29:22.478 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.711278+0000", 2024-09-14T06:29:22.478 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.2799999999999996e-07 2024-09-14T06:29:22.478 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:22.478 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:22.478 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:29:22.478 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.711281+0000", 2024-09-14T06:29:22.478 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.0839999999999999e-06 2024-09-14T06:29:22.478 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:22.478 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:22.478 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:29:22.478 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.711287+0000", 2024-09-14T06:29:22.478 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.6540000000000002e-06 2024-09-14T06:29:22.479 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:22.479 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:22.479 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:29:22.479 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.711292+0000", 2024-09-14T06:29:22.479 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.4320000000000001e-06 2024-09-14T06:29:22.479 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:22.479 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:22.479 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:29:22.479 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.723168+0000", 2024-09-14T06:29:22.480 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.011876183 2024-09-14T06:29:22.480 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:22.480 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:22.480 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:29:22.480 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.723235+0000", 2024-09-14T06:29:22.480 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.7199999999999994e-05 2024-09-14T06:29:22.480 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:22.480 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:22.480 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:29:22.480 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.723328+0000", 2024-09-14T06:29:22.480 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.2627999999999995e-05 2024-09-14T06:29:22.480 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:22.480 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:22.481 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:22.481 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:22.481 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:22.481 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:2825 3.7 3:e68f0b27:::rbd_data.112d7dfbd766.000000000000006d:5 [read 270336~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-14T06:29:22.481 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:08.711298+0000", 2024-09-14T06:29:22.481 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 13.751637167, 2024-09-14T06:29:22.481 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.012245903000000001, 2024-09-14T06:29:22.481 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:29:22.481 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:29:22.481 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:29:22.481 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:29:22.481 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:29:22.481 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:22.481 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 2825 2024-09-14T06:29:22.482 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:22.482 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:29:22.482 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:22.482 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:29:22.482 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.711298+0000", 2024-09-14T06:29:22.482 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:22.482 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:22.482 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:22.482 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:29:22.482 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.711298+0000", 2024-09-14T06:29:22.482 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:22.482 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:22.482 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:22.482 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:29:22.483 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.711299+0000", 2024-09-14T06:29:22.483 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.4999999999999998e-07 2024-09-14T06:29:22.483 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:22.483 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:22.483 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:29:22.483 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.711305+0000", 2024-09-14T06:29:22.483 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.4899999999999997e-06 2024-09-14T06:29:22.483 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:22.483 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:22.483 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:29:22.483 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.711312+0000", 2024-09-14T06:29:22.483 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.9489999999999999e-06 2024-09-14T06:29:22.483 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:22.483 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:22.484 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:29:22.484 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.711317+0000", 2024-09-14T06:29:22.484 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.6e-06 2024-09-14T06:29:22.484 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:22.484 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:22.484 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:29:22.484 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.723370+0000", 2024-09-14T06:29:22.484 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.012052916 2024-09-14T06:29:22.484 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:22.484 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:22.484 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:29:22.484 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.723445+0000", 2024-09-14T06:29:22.484 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.5687999999999998e-05 2024-09-14T06:29:22.484 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:22.485 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:22.485 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:29:22.485 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.723544+0000", 2024-09-14T06:29:22.485 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.8809999999999998e-05 2024-09-14T06:29:22.485 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:22.485 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:22.485 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:22.485 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:22.485 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:22.485 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:2826 3.7 3:e68f0b27:::rbd_data.112d7dfbd766.000000000000006d:5 [read 286720~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-14T06:29:22.485 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:08.711324+0000", 2024-09-14T06:29:22.485 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 13.751611236, 2024-09-14T06:29:22.485 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.012395302, 2024-09-14T06:29:22.486 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:29:22.486 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:29:22.486 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:29:22.486 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:29:22.486 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:29:22.486 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:22.486 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 2826 2024-09-14T06:29:22.486 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:22.486 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:29:22.486 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:22.486 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:29:22.486 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.711324+0000", 2024-09-14T06:29:22.486 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:22.486 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:22.487 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:22.487 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:29:22.487 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.711324+0000", 2024-09-14T06:29:22.487 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:22.487 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:22.487 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:22.487 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:29:22.487 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.711325+0000", 2024-09-14T06:29:22.487 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.9900000000000001e-07 2024-09-14T06:29:22.487 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:22.487 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:22.487 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:29:22.487 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.711332+0000", 2024-09-14T06:29:22.487 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.0110000000000003e-06 2024-09-14T06:29:22.488 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:22.488 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:22.488 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:29:22.488 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.711338+0000", 2024-09-14T06:29:22.488 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.2859999999999996e-06 2024-09-14T06:29:22.488 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:22.488 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:22.488 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:29:22.488 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.711343+0000", 2024-09-14T06:29:22.488 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.6480000000000002e-06 2024-09-14T06:29:22.488 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:22.488 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:22.488 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:29:22.489 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.723582+0000", 2024-09-14T06:29:22.489 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.012239641000000001 2024-09-14T06:29:22.489 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:22.489 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:22.489 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:29:22.490 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.723658+0000", 2024-09-14T06:29:22.490 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.5988000000000005e-05 2024-09-14T06:29:22.490 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:22.490 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:22.490 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:29:22.490 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.723720+0000", 2024-09-14T06:29:22.490 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.1328999999999995e-05 2024-09-14T06:29:22.490 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:22.490 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:22.490 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:22.490 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:22.490 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:22.491 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:2827 3.7 3:e68f0b27:::rbd_data.112d7dfbd766.000000000000006d:5 [read 303104~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-14T06:29:22.491 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:08.711349+0000", 2024-09-14T06:29:22.491 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 13.751586267, 2024-09-14T06:29:22.491 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.01252516, 2024-09-14T06:29:22.491 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:29:22.491 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:29:22.491 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:29:22.491 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:29:22.491 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:29:22.491 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:22.491 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 2827 2024-09-14T06:29:22.491 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:22.491 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:29:22.491 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:22.492 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:29:22.492 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.711349+0000", 2024-09-14T06:29:22.492 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:22.492 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:22.492 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:22.492 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:29:22.492 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.711349+0000", 2024-09-14T06:29:22.492 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:22.492 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:22.492 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:22.492 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:29:22.492 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.711350+0000", 2024-09-14T06:29:22.492 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.7899999999999999e-07 2024-09-14T06:29:22.493 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:22.493 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:22.493 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:29:22.493 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.711351+0000", 2024-09-14T06:29:22.493 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.795e-06 2024-09-14T06:29:22.493 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:22.493 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:22.493 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:29:22.493 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.711357+0000", 2024-09-14T06:29:22.493 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.0050000000000001e-06 2024-09-14T06:29:22.493 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:22.493 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:22.493 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:29:22.493 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.711362+0000", 2024-09-14T06:29:22.494 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.4279999999999998e-06 2024-09-14T06:29:22.494 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:22.494 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:22.494 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:29:22.494 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.723749+0000", 2024-09-14T06:29:22.494 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.012387624 2024-09-14T06:29:22.494 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:22.497 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:22.497 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:29:22.497 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.723790+0000", 2024-09-14T06:29:22.497 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.0215e-05 2024-09-14T06:29:22.497 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:22.497 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:22.497 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:29:22.497 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.723874+0000", 2024-09-14T06:29:22.498 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.4714000000000002e-05 2024-09-14T06:29:22.498 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:22.498 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:22.498 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:22.498 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:22.498 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:22.498 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:2828 3.7 3:e68f0b27:::rbd_data.112d7dfbd766.000000000000006d:5 [read 319488~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-14T06:29:22.498 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:08.711368+0000", 2024-09-14T06:29:22.498 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 13.751567003, 2024-09-14T06:29:22.498 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.012743901, 2024-09-14T06:29:22.498 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:29:22.498 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:29:22.498 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:29:22.498 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:29:22.499 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:29:22.499 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:22.499 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 2828 2024-09-14T06:29:22.499 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:22.499 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:29:22.499 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:22.499 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:29:22.499 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.711368+0000", 2024-09-14T06:29:22.499 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:22.499 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:22.499 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:22.499 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:29:22.500 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.711368+0000", 2024-09-14T06:29:22.500 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:22.500 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:22.500 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:22.500 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:29:22.500 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.711369+0000", 2024-09-14T06:29:22.500 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.2399999999999999e-07 2024-09-14T06:29:22.500 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:22.500 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:22.500 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:29:22.500 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.711370+0000", 2024-09-14T06:29:22.500 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.688e-06 2024-09-14T06:29:22.500 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:22.501 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:22.501 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:29:22.501 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.711376+0000", 2024-09-14T06:29:22.501 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.8900000000000004e-06 2024-09-14T06:29:22.501 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:22.501 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:22.501 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:29:22.501 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.711381+0000", 2024-09-14T06:29:22.501 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.3390000000000003e-06 2024-09-14T06:29:22.501 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:22.501 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:22.501 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:29:22.502 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.723921+0000", 2024-09-14T06:29:22.502 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0125402 2024-09-14T06:29:22.502 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:22.502 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:22.502 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:29:22.502 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.724006+0000", 2024-09-14T06:29:22.502 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.5554000000000001e-05 2024-09-14T06:29:22.502 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:22.502 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:22.502 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:29:22.502 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.724112+0000", 2024-09-14T06:29:22.502 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000105906 2024-09-14T06:29:22.502 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:22.503 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:22.503 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:22.503 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:22.503 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:22.503 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:2829 3.7 3:e68f0b27:::rbd_data.112d7dfbd766.000000000000006d:5 [read 335872~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-14T06:29:22.503 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:08.711408+0000", 2024-09-14T06:29:22.503 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 13.751527061000001, 2024-09-14T06:29:22.503 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.012901864000000001, 2024-09-14T06:29:22.503 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:29:22.503 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:29:22.503 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:29:22.503 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:29:22.503 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:29:22.504 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:22.504 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 2829 2024-09-14T06:29:22.504 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:22.504 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:29:22.504 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:22.504 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:29:22.504 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.711408+0000", 2024-09-14T06:29:22.504 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:22.504 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:22.504 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:22.504 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:29:22.504 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.711408+0000", 2024-09-14T06:29:22.504 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:22.504 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:22.505 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:22.505 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:29:22.505 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.711409+0000", 2024-09-14T06:29:22.505 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.3000000000000001e-07 2024-09-14T06:29:22.505 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:22.505 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:22.505 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:29:22.505 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.711411+0000", 2024-09-14T06:29:22.505 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.0930000000000001e-06 2024-09-14T06:29:22.505 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:22.505 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:22.505 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:29:22.505 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.711419+0000", 2024-09-14T06:29:22.505 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.9000000000000006e-06 2024-09-14T06:29:22.506 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:22.506 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:22.506 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:29:22.506 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.711426+0000", 2024-09-14T06:29:22.506 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.8539999999999999e-06 2024-09-14T06:29:22.506 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:22.506 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:22.506 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:29:22.506 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.724137+0000", 2024-09-14T06:29:22.506 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.012710993 2024-09-14T06:29:22.506 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:22.506 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:22.506 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:29:22.507 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.724215+0000", 2024-09-14T06:29:22.507 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.8475999999999999e-05 2024-09-14T06:29:22.507 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:22.507 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:22.507 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:29:22.507 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.724310+0000", 2024-09-14T06:29:22.507 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.5018000000000004e-05 2024-09-14T06:29:22.507 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:22.507 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:22.507 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:22.507 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:22.507 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:22.507 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:2830 3.7 3:e68f0b27:::rbd_data.112d7dfbd766.000000000000006d:5 [read 352256~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-14T06:29:22.507 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:08.711433+0000", 2024-09-14T06:29:22.508 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 13.751502284000001, 2024-09-14T06:29:22.508 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.013118222000000001, 2024-09-14T06:29:22.508 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:29:22.508 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:29:22.508 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:29:22.508 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:29:22.508 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:29:22.508 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:22.508 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 2830 2024-09-14T06:29:22.509 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:22.509 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:29:22.509 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:22.509 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:29:22.509 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.711433+0000", 2024-09-14T06:29:22.509 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:22.509 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:22.509 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:22.509 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:29:22.509 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.711433+0000", 2024-09-14T06:29:22.509 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:22.509 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:22.510 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:22.510 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:29:22.510 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.711434+0000", 2024-09-14T06:29:22.510 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.2599999999999998e-07 2024-09-14T06:29:22.510 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:22.510 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:22.510 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:29:22.510 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.711435+0000", 2024-09-14T06:29:22.510 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.8980000000000001e-06 2024-09-14T06:29:22.510 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:22.510 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:22.510 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:29:22.510 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.711441+0000", 2024-09-14T06:29:22.511 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.8270000000000003e-06 2024-09-14T06:29:22.511 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:22.511 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:22.511 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:29:22.511 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.711445+0000", 2024-09-14T06:29:22.511 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.1579999999999998e-06 2024-09-14T06:29:22.511 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:22.511 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:22.511 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:29:22.511 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.724358+0000", 2024-09-14T06:29:22.511 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.012912699999999999 2024-09-14T06:29:22.511 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:22.511 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:22.511 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:29:22.512 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.724450+0000", 2024-09-14T06:29:22.512 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.1531000000000002e-05 2024-09-14T06:29:22.512 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:22.512 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:22.512 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:29:22.512 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.724551+0000", 2024-09-14T06:29:22.512 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00010168200000000001 2024-09-14T06:29:22.512 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:22.512 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:22.512 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:22.512 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:22.512 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:22.512 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:2831 3.7 3:e68f0b27:::rbd_data.112d7dfbd766.000000000000006d:5 [read 368640~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-14T06:29:22.513 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:08.711451+0000", 2024-09-14T06:29:22.513 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 13.751484202, 2024-09-14T06:29:22.513 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.013397492, 2024-09-14T06:29:22.513 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:29:22.513 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:29:22.513 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:29:22.513 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:29:22.513 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:29:22.513 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:22.513 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 2831 2024-09-14T06:29:22.513 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:22.513 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:29:22.513 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:22.513 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:29:22.513 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.711451+0000", 2024-09-14T06:29:22.514 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:22.514 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:22.514 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:22.514 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:29:22.514 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.711451+0000", 2024-09-14T06:29:22.514 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:22.514 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:22.514 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:22.514 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:29:22.514 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.711452+0000", 2024-09-14T06:29:22.514 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.5900000000000003e-07 2024-09-14T06:29:22.514 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:22.514 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:22.514 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:29:22.515 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.711453+0000", 2024-09-14T06:29:22.515 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.5999999999999999e-06 2024-09-14T06:29:22.515 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:22.515 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:22.515 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:29:22.515 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.711461+0000", 2024-09-14T06:29:22.515 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.123e-06 2024-09-14T06:29:22.515 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:22.515 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:22.515 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:29:22.515 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.711469+0000", 2024-09-14T06:29:22.515 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.7919999999999993e-06 2024-09-14T06:29:22.515 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:22.515 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:22.515 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:29:22.515 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.724641+0000", 2024-09-14T06:29:22.516 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.013172095 2024-09-14T06:29:22.516 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:22.516 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:22.516 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:29:22.516 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.724730+0000", 2024-09-14T06:29:22.516 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.8497000000000003e-05 2024-09-14T06:29:22.516 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:22.516 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:22.516 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:29:22.516 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.724849+0000", 2024-09-14T06:29:22.516 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000119026 2024-09-14T06:29:22.516 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:22.516 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:22.516 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:22.516 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:22.517 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:22.517 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:2832 3.7 3:e68f0b27:::rbd_data.112d7dfbd766.000000000000006d:5 [read 385024~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-14T06:29:22.517 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:08.711483+0000", 2024-09-14T06:29:22.517 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 13.751452304000001, 2024-09-14T06:29:22.517 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.013595408, 2024-09-14T06:29:22.517 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:29:22.517 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:29:22.517 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:29:22.517 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:29:22.517 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:29:22.517 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:22.518 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 2832 2024-09-14T06:29:22.518 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:22.518 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:29:22.518 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:22.518 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:29:22.518 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.711483+0000", 2024-09-14T06:29:22.518 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:22.518 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:22.518 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:22.518 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:29:22.518 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.711483+0000", 2024-09-14T06:29:22.518 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:22.518 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:22.518 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:22.519 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:29:22.519 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.711484+0000", 2024-09-14T06:29:22.519 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.0399999999999996e-07 2024-09-14T06:29:22.519 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:22.519 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:22.519 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:29:22.519 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.711486+0000", 2024-09-14T06:29:22.519 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.0269999999999998e-06 2024-09-14T06:29:22.519 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:22.519 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:22.519 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:29:22.519 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.711492+0000", 2024-09-14T06:29:22.519 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.4489999999999996e-06 2024-09-14T06:29:22.519 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:22.520 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:22.520 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:29:22.520 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.711501+0000", 2024-09-14T06:29:22.520 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.3449999999999995e-06 2024-09-14T06:29:22.520 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:22.520 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:22.520 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:29:22.520 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.724897+0000", 2024-09-14T06:29:22.520 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.013395473999999999 2024-09-14T06:29:22.520 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:22.520 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:22.520 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:29:22.520 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.724974+0000", 2024-09-14T06:29:22.520 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.7158999999999998e-05 2024-09-14T06:29:22.521 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:22.521 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:22.521 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:29:22.521 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.725079+0000", 2024-09-14T06:29:22.521 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00010445 2024-09-14T06:29:22.521 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:22.521 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:22.521 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:22.521 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:22.521 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:22.521 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:2833 3.7 3:e68f0b27:::rbd_data.112d7dfbd766.000000000000006d:5 [read 401408~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-14T06:29:22.521 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:08.711508+0000", 2024-09-14T06:29:22.522 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 13.751427111, 2024-09-14T06:29:22.522 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.013818878, 2024-09-14T06:29:22.522 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:29:22.522 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:29:22.522 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:29:22.522 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:29:22.522 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:29:22.522 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:22.522 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 2833 2024-09-14T06:29:22.522 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:22.522 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:29:22.522 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:22.522 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:29:22.522 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.711508+0000", 2024-09-14T06:29:22.522 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:22.523 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:22.523 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:22.523 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:29:22.523 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.711508+0000", 2024-09-14T06:29:22.523 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:22.523 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:22.523 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:22.523 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:29:22.523 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.711509+0000", 2024-09-14T06:29:22.523 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.6899999999999998e-07 2024-09-14T06:29:22.523 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:22.523 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:22.523 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:29:22.523 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.711511+0000", 2024-09-14T06:29:22.523 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.9199999999999998e-06 2024-09-14T06:29:22.524 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:22.524 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:22.524 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:29:22.524 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.711517+0000", 2024-09-14T06:29:22.524 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.9719999999999997e-06 2024-09-14T06:29:22.524 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:22.524 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:22.524 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:29:22.524 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.711524+0000", 2024-09-14T06:29:22.524 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.8690000000000004e-06 2024-09-14T06:29:22.524 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:22.524 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:22.524 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:29:22.524 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.725129+0000", 2024-09-14T06:29:22.524 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.013604565000000001 2024-09-14T06:29:22.525 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:22.525 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:22.525 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:29:22.525 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.725212+0000", 2024-09-14T06:29:22.525 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.2492999999999998e-05 2024-09-14T06:29:22.525 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:22.525 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:22.525 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:29:22.525 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.725327+0000", 2024-09-14T06:29:22.525 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00011569 2024-09-14T06:29:22.525 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:22.525 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:22.525 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:22.525 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:22.526 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:22.526 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:2834 3.7 3:e68f0b27:::rbd_data.112d7dfbd766.000000000000006d:5 [read 417792~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-14T06:29:22.526 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:08.711531+0000", 2024-09-14T06:29:22.526 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 13.751404625999999, 2024-09-14T06:29:22.526 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.014032723, 2024-09-14T06:29:22.526 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:29:22.526 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:29:22.526 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:29:22.526 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:29:22.526 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:29:22.526 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:22.526 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 2834 2024-09-14T06:29:22.526 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:22.526 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:29:22.527 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:22.527 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:29:22.527 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.711531+0000", 2024-09-14T06:29:22.527 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:22.527 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:22.527 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:22.527 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:29:22.527 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.711531+0000", 2024-09-14T06:29:22.527 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:22.527 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:22.527 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:22.527 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:29:22.527 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.711531+0000", 2024-09-14T06:29:22.527 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.7599999999999998e-07 2024-09-14T06:29:22.528 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:22.528 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:22.528 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:29:22.528 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.711533+0000", 2024-09-14T06:29:22.528 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.7540000000000001e-06 2024-09-14T06:29:22.528 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:22.528 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:22.528 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:29:22.528 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.711539+0000", 2024-09-14T06:29:22.528 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.7899999999999996e-06 2024-09-14T06:29:22.528 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:22.528 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:22.528 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:29:22.528 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.711548+0000", 2024-09-14T06:29:22.528 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.3039999999999994e-06 2024-09-14T06:29:22.529 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:22.529 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:22.529 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:29:22.529 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.725376+0000", 2024-09-14T06:29:22.529 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.013828343 2024-09-14T06:29:22.529 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:22.529 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:22.529 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:29:22.529 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.725459+0000", 2024-09-14T06:29:22.529 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.2888999999999999e-05 2024-09-14T06:29:22.529 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:22.529 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:22.529 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:29:22.529 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.725564+0000", 2024-09-14T06:29:22.529 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000104267 2024-09-14T06:29:22.529 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:22.530 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:22.530 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:22.530 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:22.530 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:22.530 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:2835 3.7 3:e68f0b27:::rbd_data.112d7dfbd766.000000000000006d:5 [read 434176~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-14T06:29:22.530 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:08.711556+0000", 2024-09-14T06:29:22.530 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 13.751379651000001, 2024-09-14T06:29:22.530 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.014225475, 2024-09-14T06:29:22.530 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:29:22.530 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:29:22.530 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:29:22.530 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:29:22.530 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:29:22.530 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:22.530 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 2835 2024-09-14T06:29:22.531 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:22.531 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:29:22.531 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:22.531 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:29:22.531 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.711556+0000", 2024-09-14T06:29:22.531 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:22.531 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:22.531 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:22.531 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:29:22.531 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.711556+0000", 2024-09-14T06:29:22.531 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:22.531 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:22.531 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:22.531 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:29:22.531 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.711556+0000", 2024-09-14T06:29:22.532 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.2e-07 2024-09-14T06:29:22.532 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:22.532 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:22.532 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:29:22.532 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.711558+0000", 2024-09-14T06:29:22.532 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.781e-06 2024-09-14T06:29:22.532 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:22.532 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:22.532 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:29:22.532 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.711564+0000", 2024-09-14T06:29:22.532 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.0070000000000003e-06 2024-09-14T06:29:22.532 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:22.532 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:22.532 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:29:22.533 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.711569+0000", 2024-09-14T06:29:22.533 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.5329999999999996e-06 2024-09-14T06:29:22.533 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:22.533 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:22.533 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:29:22.533 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.725597+0000", 2024-09-14T06:29:22.533 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.014028357 2024-09-14T06:29:22.533 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:22.533 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:22.533 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:29:22.533 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.725671+0000", 2024-09-14T06:29:22.534 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.3623999999999998e-05 2024-09-14T06:29:22.534 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:22.534 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:22.534 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:29:22.534 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.725781+0000", 2024-09-14T06:29:22.534 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000110753 2024-09-14T06:29:22.534 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:22.534 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:22.534 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:22.534 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:22.534 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:22.534 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:2836 3.7 3:e68f0b27:::rbd_data.112d7dfbd766.000000000000006d:5 [read 450560~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-14T06:29:22.534 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:08.711574+0000", 2024-09-14T06:29:22.534 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 13.751361331, 2024-09-14T06:29:22.534 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.014419141999999999, 2024-09-14T06:29:22.534 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:29:22.535 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:29:22.535 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:29:22.535 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:29:22.535 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:29:22.535 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:22.535 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 2836 2024-09-14T06:29:22.535 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:22.535 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:29:22.535 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:22.535 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:29:22.535 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.711574+0000", 2024-09-14T06:29:22.535 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:22.535 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:22.535 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:22.535 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:29:22.535 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.711574+0000", 2024-09-14T06:29:22.536 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:22.536 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:22.536 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:22.536 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:29:22.536 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.711574+0000", 2024-09-14T06:29:22.536 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.58e-07 2024-09-14T06:29:22.536 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:22.536 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:22.536 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:29:22.536 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.711576+0000", 2024-09-14T06:29:22.536 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.6500000000000001e-06 2024-09-14T06:29:22.536 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:22.536 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:22.537 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:29:22.537 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.711582+0000", 2024-09-14T06:29:22.537 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.6720000000000001e-06 2024-09-14T06:29:22.537 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:22.537 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:22.537 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:29:22.537 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.711586+0000", 2024-09-14T06:29:22.537 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.3680000000000004e-06 2024-09-14T06:29:22.537 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:22.537 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:22.537 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:29:22.537 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.725820+0000", 2024-09-14T06:29:22.537 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.014233934 2024-09-14T06:29:22.537 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:22.538 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:22.538 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:29:22.538 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.725891+0000", 2024-09-14T06:29:22.538 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.0609000000000001e-05 2024-09-14T06:29:22.538 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:22.538 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:22.538 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:29:22.538 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.725993+0000", 2024-09-14T06:29:22.538 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000102551 2024-09-14T06:29:22.538 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:22.538 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:22.539 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:22.539 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:22.539 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:22.539 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:2837 3.7 3:e68f0b27:::rbd_data.112d7dfbd766.000000000000006d:5 [read 466944~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-14T06:29:22.539 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:08.711592+0000", 2024-09-14T06:29:22.539 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 13.751343904000001, 2024-09-14T06:29:22.539 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.014675145000000001, 2024-09-14T06:29:22.539 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:29:22.540 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:29:22.540 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:29:22.540 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:29:22.540 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:29:22.540 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:22.540 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 2837 2024-09-14T06:29:22.540 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:22.540 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:29:22.540 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:22.540 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:29:22.540 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.711592+0000", 2024-09-14T06:29:22.540 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:22.541 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:22.541 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:22.541 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:29:22.541 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.711592+0000", 2024-09-14T06:29:22.541 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:22.541 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:22.541 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:22.541 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:29:22.541 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.711592+0000", 2024-09-14T06:29:22.541 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.5199999999999997e-07 2024-09-14T06:29:22.541 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:22.541 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:22.541 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:29:22.542 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.711622+0000", 2024-09-14T06:29:22.542 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.0199999999999999e-05 2024-09-14T06:29:22.542 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:22.542 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:22.542 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:29:22.542 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.711629+0000", 2024-09-14T06:29:22.542 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.1049999999999997e-06 2024-09-14T06:29:22.542 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:22.542 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:22.542 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:29:22.542 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.711634+0000", 2024-09-14T06:29:22.542 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.826e-06 2024-09-14T06:29:22.542 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:22.542 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:22.542 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:29:22.543 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.726039+0000", 2024-09-14T06:29:22.543 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.014404682 2024-09-14T06:29:22.543 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:22.543 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:22.543 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:29:22.543 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.726122+0000", 2024-09-14T06:29:22.543 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.3133000000000005e-05 2024-09-14T06:29:22.543 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:22.543 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:22.543 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:29:22.543 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.726267+0000", 2024-09-14T06:29:22.543 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000144647 2024-09-14T06:29:22.543 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:22.543 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:22.544 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:22.544 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:22.544 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:22.544 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:2838 3.7 3:e68f0b27:::rbd_data.112d7dfbd766.000000000000006d:5 [read 483328~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-14T06:29:22.544 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:08.711641+0000", 2024-09-14T06:29:22.544 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 13.751293967000001, 2024-09-14T06:29:22.544 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.014898524999999999, 2024-09-14T06:29:22.544 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:29:22.544 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:29:22.544 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:29:22.544 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:29:22.544 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:29:22.544 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:22.545 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 2838 2024-09-14T06:29:22.545 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:22.545 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:29:22.545 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:22.545 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:29:22.545 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.711641+0000", 2024-09-14T06:29:22.545 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:22.545 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:22.545 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:22.545 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:29:22.545 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.711641+0000", 2024-09-14T06:29:22.545 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:22.545 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:22.546 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:22.546 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:29:22.546 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.711642+0000", 2024-09-14T06:29:22.546 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.3200000000000005e-07 2024-09-14T06:29:22.546 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:22.546 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:22.546 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:29:22.546 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.711644+0000", 2024-09-14T06:29:22.546 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.827e-06 2024-09-14T06:29:22.546 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:22.546 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:22.546 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:29:22.546 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.711655+0000", 2024-09-14T06:29:22.546 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.1564e-05 2024-09-14T06:29:22.546 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:22.547 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:22.547 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:29:22.547 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.711661+0000", 2024-09-14T06:29:22.547 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.7599999999999999e-06 2024-09-14T06:29:22.547 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:22.547 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:22.547 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:29:22.547 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.726299+0000", 2024-09-14T06:29:22.547 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.014638138 2024-09-14T06:29:22.547 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:22.547 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:22.547 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:29:22.547 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.726380+0000", 2024-09-14T06:29:22.547 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.0756999999999999e-05 2024-09-14T06:29:22.548 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:22.548 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:22.548 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:29:22.548 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.726540+0000", 2024-09-14T06:29:22.548 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00015994699999999999 2024-09-14T06:29:22.548 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:22.548 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:22.548 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:22.548 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:22.548 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:22.548 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:2841 3.7 3:e5b3a0f9:::rbd_data.112d7dfbd766.0000000000000069:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-14T06:29:22.548 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:08.711698+0000", 2024-09-14T06:29:22.548 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 13.751237658000001, 2024-09-14T06:29:22.548 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.014993504, 2024-09-14T06:29:22.548 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:29:22.549 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:29:22.549 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "reached pg", 2024-09-14T06:29:22.549 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:29:22.549 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:29:22.549 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:22.549 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 2841 2024-09-14T06:29:22.549 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:22.549 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:29:22.549 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:22.549 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:29:22.549 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.711698+0000", 2024-09-14T06:29:22.549 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:22.549 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:22.550 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:22.550 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:29:22.550 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.711698+0000", 2024-09-14T06:29:22.550 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:22.550 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:22.550 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:22.550 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:29:22.550 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.711698+0000", 2024-09-14T06:29:22.550 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.7e-07 2024-09-14T06:29:22.550 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:22.550 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:22.550 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:29:22.551 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.711700+0000", 2024-09-14T06:29:22.551 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.8199999999999999e-06 2024-09-14T06:29:22.551 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:22.551 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:22.551 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:29:22.551 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.711706+0000", 2024-09-14T06:29:22.551 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.8050000000000003e-06 2024-09-14T06:29:22.551 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:22.551 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:22.551 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:29:22.551 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.711710+0000", 2024-09-14T06:29:22.551 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.69e-06 2024-09-14T06:29:22.551 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:22.551 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:22.551 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:29:22.551 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.726608+0000", 2024-09-14T06:29:22.552 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.014897881999999999 2024-09-14T06:29:22.552 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:22.552 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:22.552 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:29:22.552 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.726691+0000", 2024-09-14T06:29:22.552 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.2936999999999996e-05 2024-09-14T06:29:22.552 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:22.552 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:22.552 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:22.552 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:22.552 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:22.552 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:2843 3.7 3:e750405e:::rbd_data.112d7dfbd766.000000000000006b:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-14T06:29:22.552 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:08.711718+0000", 2024-09-14T06:29:22.552 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 13.751217563000001, 2024-09-14T06:29:22.552 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.01513749, 2024-09-14T06:29:22.553 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:29:22.553 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:29:22.553 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "reached pg", 2024-09-14T06:29:22.553 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:29:22.553 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:29:22.553 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:22.553 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 2843 2024-09-14T06:29:22.553 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:22.553 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:29:22.553 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:22.553 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:29:22.553 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.711718+0000", 2024-09-14T06:29:22.553 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:22.553 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:22.553 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:22.553 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:29:22.554 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.711718+0000", 2024-09-14T06:29:22.554 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:22.554 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:22.554 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:22.554 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:29:22.554 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.711718+0000", 2024-09-14T06:29:22.554 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.5900000000000003e-07 2024-09-14T06:29:22.554 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:22.554 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:22.554 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:29:22.554 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.711720+0000", 2024-09-14T06:29:22.554 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.6899999999999999e-06 2024-09-14T06:29:22.554 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:22.554 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:22.555 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:29:22.555 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.711726+0000", 2024-09-14T06:29:22.555 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.7819999999999999e-06 2024-09-14T06:29:22.555 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:22.555 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:22.555 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:29:22.555 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.711730+0000", 2024-09-14T06:29:22.555 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.2390000000000004e-06 2024-09-14T06:29:22.555 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:22.555 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:22.555 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:29:22.555 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.726739+0000", 2024-09-14T06:29:22.555 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.015009366999999999 2024-09-14T06:29:22.555 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:22.555 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:22.556 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:29:22.556 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.726855+0000", 2024-09-14T06:29:22.556 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00011605299999999999 2024-09-14T06:29:22.556 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:22.556 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:22.556 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:22.556 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:22.556 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:22.556 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:2845 3.7 3:e68f0b27:::rbd_data.112d7dfbd766.000000000000006d:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-14T06:29:22.556 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:08.711736+0000", 2024-09-14T06:29:22.556 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 13.751199114, 2024-09-14T06:29:22.556 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.015271547, 2024-09-14T06:29:22.556 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:29:22.556 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:29:22.556 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:29:22.556 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:29:22.557 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:29:22.557 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:22.557 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 2845 2024-09-14T06:29:22.557 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:22.557 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:29:22.557 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:22.557 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:29:22.557 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.711736+0000", 2024-09-14T06:29:22.557 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:22.557 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:22.557 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:22.557 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:29:22.557 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.711736+0000", 2024-09-14T06:29:22.557 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:22.557 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:22.558 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:22.558 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:29:22.558 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.711737+0000", 2024-09-14T06:29:22.558 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.3799999999999998e-07 2024-09-14T06:29:22.558 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:22.558 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:22.558 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:29:22.558 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.711738+0000", 2024-09-14T06:29:22.558 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.801e-06 2024-09-14T06:29:22.558 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:22.558 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:22.558 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:29:22.558 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.711744+0000", 2024-09-14T06:29:22.558 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.9190000000000004e-06 2024-09-14T06:29:22.558 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:22.558 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:22.559 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:29:22.559 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.711749+0000", 2024-09-14T06:29:22.559 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.4469999999999999e-06 2024-09-14T06:29:22.559 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:22.559 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:22.559 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:29:22.559 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.726889+0000", 2024-09-14T06:29:22.559 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.015140374999999999 2024-09-14T06:29:22.559 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:22.559 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:22.559 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:29:22.559 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.726930+0000", 2024-09-14T06:29:22.559 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.1239000000000003e-05 2024-09-14T06:29:22.559 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:22.559 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:22.559 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:29:22.560 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.727008+0000", 2024-09-14T06:29:22.560 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.7428000000000005e-05 2024-09-14T06:29:22.560 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:22.560 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:22.560 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:22.560 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:22.560 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:22.560 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-14T06:29:22.562 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-14T06:29:22.562 INFO:tasks.ceph.osd.5.smithi138.stderr:2024-09-14T06:29:22.560+0000 7fe13746f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33618) UID: 0 2024-09-14T06:29:22.623 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-14T06:29:22.667 INFO:tasks.ceph.osd.1.smithi094.stderr:2024-09-14T06:29:22.666+0000 7f328e47a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33641) UID: 0 2024-09-14T06:29:22.717 INFO:teuthology.orchestra.run.smithi138.stdout:{ 2024-09-14T06:29:22.717 INFO:teuthology.orchestra.run.smithi138.stdout: "ops": [], 2024-09-14T06:29:22.717 INFO:teuthology.orchestra.run.smithi138.stdout: "num_ops": 0 2024-09-14T06:29:22.717 INFO:teuthology.orchestra.run.smithi138.stdout:} 2024-09-14T06:29:22.726 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-14T06:29:22.759 INFO:teuthology.orchestra.run.smithi094.stdout:osd.0: {} 2024-09-14T06:29:22.759 INFO:teuthology.orchestra.run.smithi094.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-14T06:29:22.763 INFO:tasks.ceph.osd.1.smithi094.stderr:2024-09-14T06:29:22.762+0000 7f328e47a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33641) UID: 0 2024-09-14T06:29:22.775 INFO:teuthology.orchestra.run.smithi094.stdout:osd.1: {} 2024-09-14T06:29:22.776 INFO:teuthology.orchestra.run.smithi094.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-14T06:29:22.791 INFO:teuthology.orchestra.run.smithi094.stdout:osd.2: {} 2024-09-14T06:29:22.792 INFO:teuthology.orchestra.run.smithi094.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-14T06:29:22.829 INFO:teuthology.orchestra.run.smithi138.stdout:{ 2024-09-14T06:29:22.829 INFO:teuthology.orchestra.run.smithi138.stdout: "ops": [], 2024-09-14T06:29:22.829 INFO:teuthology.orchestra.run.smithi138.stdout: "complaint_time": 30, 2024-09-14T06:29:22.829 INFO:teuthology.orchestra.run.smithi138.stdout: "num_blocked_ops": 0 2024-09-14T06:29:22.829 INFO:teuthology.orchestra.run.smithi138.stdout:} 2024-09-14T06:29:22.838 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-14T06:29:22.864 INFO:tasks.ceph.osd.3.smithi094.stderr:2024-09-14T06:29:22.863+0000 7feeeafe8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33655) UID: 0 2024-09-14T06:29:22.941 INFO:teuthology.orchestra.run.smithi138.stdout:{ 2024-09-14T06:29:22.941 INFO:teuthology.orchestra.run.smithi138.stdout: "size": 20, 2024-09-14T06:29:22.941 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 600, 2024-09-14T06:29:22.941 INFO:teuthology.orchestra.run.smithi138.stdout: "ops": [ 2024-09-14T06:29:22.941 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:22.941 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:1809 3.3 3:c0711257:::rbd_data.112d7dfbd766.000000000000001a:5 [read 126976~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-14T06:29:22.941 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:08.654606+0000", 2024-09-14T06:29:22.942 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 14.284689888999999, 2024-09-14T06:29:22.942 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.019587192, 2024-09-14T06:29:22.942 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:29:22.942 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:29:22.942 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:29:22.942 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:29:22.942 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:29:22.942 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:22.942 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 1809 2024-09-14T06:29:22.942 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:22.942 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:29:22.942 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:22.942 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:29:22.943 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.654606+0000", 2024-09-14T06:29:22.943 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:22.943 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:22.943 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:22.943 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:29:22.943 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.654606+0000", 2024-09-14T06:29:22.943 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:22.943 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:22.943 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:22.943 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:29:22.943 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.654606+0000", 2024-09-14T06:29:22.943 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.34e-07 2024-09-14T06:29:22.943 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:22.943 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:22.943 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:29:22.944 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.654607+0000", 2024-09-14T06:29:22.944 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.3600000000000002e-07 2024-09-14T06:29:22.944 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:22.944 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:22.944 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:29:22.944 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.654610+0000", 2024-09-14T06:29:22.944 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.0299999999999998e-06 2024-09-14T06:29:22.944 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:22.944 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:22.944 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:29:22.944 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.654612+0000", 2024-09-14T06:29:22.944 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.3309999999999998e-06 2024-09-14T06:29:22.944 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:22.945 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:22.945 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:29:22.945 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.674056+0000", 2024-09-14T06:29:22.945 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.019443790999999998 2024-09-14T06:29:22.945 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:22.945 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:22.945 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:29:22.945 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.674121+0000", 2024-09-14T06:29:22.945 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.5029000000000004e-05 2024-09-14T06:29:22.945 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:22.945 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:22.945 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:29:22.945 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.674193+0000", 2024-09-14T06:29:22.945 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.1841000000000006e-05 2024-09-14T06:29:22.945 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:22.946 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:29:22.946 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:22.946 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:22.946 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:22.946 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:1810 3.3 3:c0711257:::rbd_data.112d7dfbd766.000000000000001a:5 [read 143360~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-14T06:29:22.946 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:08.654617+0000", 2024-09-14T06:29:22.946 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 14.284679181, 2024-09-14T06:29:22.946 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.019784883, 2024-09-14T06:29:22.946 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:29:22.946 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:29:22.946 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:29:22.946 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:29:22.947 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:29:22.947 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:22.947 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 1810 2024-09-14T06:29:22.947 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:22.947 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:29:22.947 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:22.947 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:29:22.947 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.654617+0000", 2024-09-14T06:29:22.947 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:22.947 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:22.947 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:22.947 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:29:22.947 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.654617+0000", 2024-09-14T06:29:22.947 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:22.947 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:22.947 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:22.948 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:29:22.948 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.654617+0000", 2024-09-14T06:29:22.948 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.1e-07 2024-09-14T06:29:22.948 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:22.948 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:22.948 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:29:22.948 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.654618+0000", 2024-09-14T06:29:22.948 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.7000000000000003e-07 2024-09-14T06:29:22.948 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:22.948 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:22.948 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:29:22.948 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.654621+0000", 2024-09-14T06:29:22.948 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.9579999999999999e-06 2024-09-14T06:29:22.948 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:22.949 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:22.949 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:29:22.949 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.654623+0000", 2024-09-14T06:29:22.949 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.4210000000000002e-06 2024-09-14T06:29:22.949 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:22.949 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:22.949 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:29:22.949 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.674220+0000", 2024-09-14T06:29:22.949 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.019597063000000001 2024-09-14T06:29:22.949 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:22.949 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:22.949 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:29:22.950 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.674303+0000", 2024-09-14T06:29:22.950 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.2867999999999993e-05 2024-09-14T06:29:22.950 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:22.950 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:22.950 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:29:22.950 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.674402+0000", 2024-09-14T06:29:22.950 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.8492999999999994e-05 2024-09-14T06:29:22.950 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:22.950 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:29:22.950 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:22.950 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:22.950 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:22.950 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:1811 3.3 3:c0711257:::rbd_data.112d7dfbd766.000000000000001a:5 [read 159744~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-14T06:29:22.950 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:08.654627+0000", 2024-09-14T06:29:22.951 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 14.284669324999999, 2024-09-14T06:29:22.951 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.020020645, 2024-09-14T06:29:22.951 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:29:22.951 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:29:22.951 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:29:22.951 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:29:22.951 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:29:22.951 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:22.951 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 1811 2024-09-14T06:29:22.951 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:22.951 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:29:22.952 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:22.952 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:29:22.952 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.654627+0000", 2024-09-14T06:29:22.952 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:22.952 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:22.952 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:22.952 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:29:22.952 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.654627+0000", 2024-09-14T06:29:22.952 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:22.952 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:22.952 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:22.952 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:29:22.952 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.654627+0000", 2024-09-14T06:29:22.953 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.8599999999999999e-07 2024-09-14T06:29:22.953 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:22.953 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:22.953 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:29:22.953 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.654628+0000", 2024-09-14T06:29:22.953 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.39e-06 2024-09-14T06:29:22.953 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:22.953 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:22.953 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:29:22.953 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.654633+0000", 2024-09-14T06:29:22.953 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.8810000000000004e-06 2024-09-14T06:29:22.953 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:22.953 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:22.953 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:29:22.954 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.654637+0000", 2024-09-14T06:29:22.954 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.1629999999999998e-06 2024-09-14T06:29:22.954 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:22.954 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:22.954 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:29:22.954 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.674468+0000", 2024-09-14T06:29:22.954 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.019830668999999999 2024-09-14T06:29:22.954 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:22.954 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:22.954 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:29:22.954 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.674544+0000", 2024-09-14T06:29:22.954 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.6438000000000002e-05 2024-09-14T06:29:22.954 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:22.954 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:22.954 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:29:22.954 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.674647+0000", 2024-09-14T06:29:22.955 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.000102818 2024-09-14T06:29:22.955 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:22.955 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:29:22.955 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:22.955 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:22.955 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:22.955 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:1812 3.3 3:c0711257:::rbd_data.112d7dfbd766.000000000000001a:5 [read 176128~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-14T06:29:22.955 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:08.654642+0000", 2024-09-14T06:29:22.955 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 14.284654143999999, 2024-09-14T06:29:22.955 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.020225290999999999, 2024-09-14T06:29:22.955 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:29:22.955 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:29:22.955 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:29:22.955 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:29:22.955 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:29:22.956 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:22.956 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 1812 2024-09-14T06:29:22.956 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:22.956 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:29:22.956 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:22.956 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:29:22.956 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.654642+0000", 2024-09-14T06:29:22.956 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:22.956 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:22.956 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:22.956 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:29:22.956 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.654642+0000", 2024-09-14T06:29:22.956 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:22.956 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:22.957 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:22.957 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:29:22.957 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.654642+0000", 2024-09-14T06:29:22.957 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.35e-07 2024-09-14T06:29:22.957 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:22.957 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:22.957 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:29:22.957 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.654643+0000", 2024-09-14T06:29:22.957 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.1599999999999999e-06 2024-09-14T06:29:22.957 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:22.957 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:22.957 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:29:22.957 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.654646+0000", 2024-09-14T06:29:22.958 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.185e-06 2024-09-14T06:29:22.958 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:22.958 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:22.958 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:29:22.958 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.654649+0000", 2024-09-14T06:29:22.958 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.2339999999999999e-06 2024-09-14T06:29:22.958 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:22.958 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:22.958 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:29:22.958 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.674683+0000", 2024-09-14T06:29:22.958 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.02003344 2024-09-14T06:29:22.958 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:22.958 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:22.959 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:29:22.959 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.674773+0000", 2024-09-14T06:29:22.959 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.0227000000000002e-05 2024-09-14T06:29:22.959 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:22.959 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:22.959 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:29:22.959 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.674867+0000", 2024-09-14T06:29:22.959 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.3809999999999998e-05 2024-09-14T06:29:22.959 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:22.959 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:29:22.959 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:22.959 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:22.959 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:22.959 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:1813 3.3 3:c0711257:::rbd_data.112d7dfbd766.000000000000001a:5 [read 192512~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-14T06:29:22.959 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:08.654672+0000", 2024-09-14T06:29:22.960 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 14.284623341, 2024-09-14T06:29:22.960 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.020374199999999999, 2024-09-14T06:29:22.960 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:29:22.960 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:29:22.960 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:29:22.960 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:29:22.960 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:29:22.960 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:22.960 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 1813 2024-09-14T06:29:22.960 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:22.960 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:29:22.960 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:22.960 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:29:22.961 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.654672+0000", 2024-09-14T06:29:22.961 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:22.961 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:22.961 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:22.961 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:29:22.961 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.654672+0000", 2024-09-14T06:29:22.961 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:22.961 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:22.961 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:22.961 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:29:22.961 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.654673+0000", 2024-09-14T06:29:22.961 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.1099999999999995e-07 2024-09-14T06:29:22.961 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:22.962 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:22.962 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:29:22.962 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.654675+0000", 2024-09-14T06:29:22.962 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.2249999999999999e-06 2024-09-14T06:29:22.962 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:22.962 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:22.962 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:29:22.962 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.654682+0000", 2024-09-14T06:29:22.962 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.781e-06 2024-09-14T06:29:22.962 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:22.962 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:22.962 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:29:22.962 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.654686+0000", 2024-09-14T06:29:22.962 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.6490000000000001e-06 2024-09-14T06:29:22.963 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:22.963 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:22.963 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:29:22.963 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.674904+0000", 2024-09-14T06:29:22.963 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.020218604000000001 2024-09-14T06:29:22.963 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:22.963 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:22.963 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:29:22.963 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.674959+0000", 2024-09-14T06:29:22.963 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.5004999999999999e-05 2024-09-14T06:29:22.963 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:22.963 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:22.963 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:29:22.963 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.675047+0000", 2024-09-14T06:29:22.964 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.7324999999999995e-05 2024-09-14T06:29:22.964 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:22.964 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:29:22.964 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:22.964 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:22.964 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:22.964 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:1814 3.3 3:c0711257:::rbd_data.112d7dfbd766.000000000000001a:5 [read 208896~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-14T06:29:22.964 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:08.654691+0000", 2024-09-14T06:29:22.964 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 14.284604803000001, 2024-09-14T06:29:22.964 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.020493225, 2024-09-14T06:29:22.964 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:29:22.964 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:29:22.964 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:29:22.964 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:29:22.965 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:29:22.965 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:22.965 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 1814 2024-09-14T06:29:22.965 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:22.965 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:29:22.965 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:22.965 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:29:22.965 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.654691+0000", 2024-09-14T06:29:22.965 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:22.965 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:22.965 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:22.965 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:29:22.965 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.654691+0000", 2024-09-14T06:29:22.965 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:22.966 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:22.966 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:22.966 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:29:22.966 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.654691+0000", 2024-09-14T06:29:22.966 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.41e-07 2024-09-14T06:29:22.966 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:22.966 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:22.966 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:29:22.966 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.654693+0000", 2024-09-14T06:29:22.966 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.3060000000000001e-06 2024-09-14T06:29:22.966 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:22.966 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:22.966 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:29:22.966 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.654696+0000", 2024-09-14T06:29:22.966 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.2100000000000002e-06 2024-09-14T06:29:22.967 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:22.967 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:22.967 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:29:22.967 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.654700+0000", 2024-09-14T06:29:22.967 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.7749999999999999e-06 2024-09-14T06:29:22.967 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:22.967 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:22.967 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:29:22.967 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.675069+0000", 2024-09-14T06:29:22.967 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.020369585999999999 2024-09-14T06:29:22.967 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:22.967 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:22.967 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:29:22.967 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.675108+0000", 2024-09-14T06:29:22.967 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.8479999999999997e-05 2024-09-14T06:29:22.968 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:22.968 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:22.968 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:29:22.968 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.675184+0000", 2024-09-14T06:29:22.968 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.6526999999999994e-05 2024-09-14T06:29:22.968 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:22.968 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:29:22.968 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:22.968 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:22.968 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:22.968 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:1815 3.3 3:c0711257:::rbd_data.112d7dfbd766.000000000000001a:5 [read 225280~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-14T06:29:22.968 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:08.654703+0000", 2024-09-14T06:29:22.968 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 14.284592825000001, 2024-09-14T06:29:22.968 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.020638215000000001, 2024-09-14T06:29:22.968 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:29:22.969 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:29:22.969 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:29:22.969 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:29:22.969 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:29:22.969 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:22.969 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 1815 2024-09-14T06:29:22.969 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:22.969 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:29:22.969 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:22.969 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:29:22.969 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.654703+0000", 2024-09-14T06:29:22.969 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:22.969 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:22.970 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:22.970 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:29:22.970 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.654703+0000", 2024-09-14T06:29:22.970 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:22.970 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:22.970 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:22.970 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:29:22.970 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.654703+0000", 2024-09-14T06:29:22.970 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.4400000000000001e-07 2024-09-14T06:29:22.970 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:22.970 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:22.970 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:29:22.970 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.654704+0000", 2024-09-14T06:29:22.971 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.4900000000000004e-07 2024-09-14T06:29:22.971 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:22.971 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:22.971 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:29:22.971 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.654707+0000", 2024-09-14T06:29:22.971 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.0359999999999998e-06 2024-09-14T06:29:22.971 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:22.971 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:22.971 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:29:22.971 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.654710+0000", 2024-09-14T06:29:22.971 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.4820000000000001e-06 2024-09-14T06:29:22.971 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:22.971 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:22.972 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:29:22.972 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.675212+0000", 2024-09-14T06:29:22.972 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.020502126999999998 2024-09-14T06:29:22.972 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:22.972 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:22.972 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:29:22.972 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.675256+0000", 2024-09-14T06:29:22.972 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.4659000000000001e-05 2024-09-14T06:29:22.972 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:22.972 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:22.972 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:29:22.972 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.675341+0000", 2024-09-14T06:29:22.972 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.4717999999999998e-05 2024-09-14T06:29:22.972 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:22.973 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:29:22.973 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:22.973 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:22.973 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:22.973 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:1816 3.3 3:c0711257:::rbd_data.112d7dfbd766.000000000000001a:5 [read 241664~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-14T06:29:22.973 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:08.654713+0000", 2024-09-14T06:29:22.973 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 14.284582954999999, 2024-09-14T06:29:22.973 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.020816715999999999, 2024-09-14T06:29:22.973 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:29:22.973 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:29:22.973 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:29:22.973 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:29:22.973 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:29:22.974 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:22.974 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 1816 2024-09-14T06:29:22.974 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:22.974 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:29:22.974 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:22.974 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:29:22.974 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.654713+0000", 2024-09-14T06:29:22.974 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:22.974 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:22.974 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:22.974 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:29:22.974 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.654713+0000", 2024-09-14T06:29:22.974 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:22.974 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:22.975 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:22.975 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:29:22.975 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.654713+0000", 2024-09-14T06:29:22.975 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.22e-07 2024-09-14T06:29:22.975 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:22.975 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:22.975 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:29:22.975 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.654714+0000", 2024-09-14T06:29:22.975 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.9600000000000008e-07 2024-09-14T06:29:22.975 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:22.975 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:22.975 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:29:22.975 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.654717+0000", 2024-09-14T06:29:22.976 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.9749999999999999e-06 2024-09-14T06:29:22.976 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:22.976 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:22.976 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:29:22.976 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.654720+0000", 2024-09-14T06:29:22.976 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.4760000000000001e-06 2024-09-14T06:29:22.976 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:22.976 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:22.976 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:29:22.976 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.675404+0000", 2024-09-14T06:29:22.976 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.020684599000000001 2024-09-14T06:29:22.976 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:22.976 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:22.977 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:29:22.977 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.675475+0000", 2024-09-14T06:29:22.977 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.0753000000000005e-05 2024-09-14T06:29:22.977 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:22.977 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:22.977 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:29:22.977 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.675530+0000", 2024-09-14T06:29:22.977 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.4694999999999997e-05 2024-09-14T06:29:22.977 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:22.977 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:29:22.977 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:22.977 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:22.977 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:22.977 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:1817 3.3 3:c0711257:::rbd_data.112d7dfbd766.000000000000001a:5 [read 258048~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-14T06:29:22.978 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:08.654723+0000", 2024-09-14T06:29:22.978 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 14.28457315, 2024-09-14T06:29:22.978 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.020951320999999998, 2024-09-14T06:29:22.978 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:29:22.978 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:29:22.978 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:29:22.978 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:29:22.978 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:29:22.978 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:22.978 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 1817 2024-09-14T06:29:22.978 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:22.978 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:29:22.978 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:22.978 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:29:22.979 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.654723+0000", 2024-09-14T06:29:22.979 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:22.979 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:22.979 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:22.979 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:29:22.979 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.654723+0000", 2024-09-14T06:29:22.979 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:22.979 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:22.979 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:22.979 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:29:22.979 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.654723+0000", 2024-09-14T06:29:22.979 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.16e-07 2024-09-14T06:29:22.979 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:22.979 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:22.980 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:29:22.980 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.654731+0000", 2024-09-14T06:29:22.980 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.8140000000000001e-06 2024-09-14T06:29:22.980 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:22.980 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:22.980 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:29:22.980 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.654734+0000", 2024-09-14T06:29:22.980 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.3569999999999998e-06 2024-09-14T06:29:22.980 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:22.980 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:22.980 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:29:22.980 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.654737+0000", 2024-09-14T06:29:22.980 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.683e-06 2024-09-14T06:29:22.980 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:22.981 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:22.981 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:29:22.981 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.675557+0000", 2024-09-14T06:29:22.981 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.020820490000000001 2024-09-14T06:29:22.981 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:22.981 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:22.981 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:29:22.981 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.675605+0000", 2024-09-14T06:29:22.981 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.7432999999999998e-05 2024-09-14T06:29:22.981 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:22.981 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:22.981 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:29:22.981 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.675674+0000", 2024-09-14T06:29:22.981 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.9327999999999998e-05 2024-09-14T06:29:22.982 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:22.982 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:29:22.982 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:22.982 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:22.982 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:22.982 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:1818 3.3 3:c0711257:::rbd_data.112d7dfbd766.000000000000001a:5 [read 274432~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-14T06:29:22.982 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:08.654740+0000", 2024-09-14T06:29:22.982 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 14.284555443, 2024-09-14T06:29:22.982 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.021074558, 2024-09-14T06:29:22.982 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:29:22.982 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:29:22.982 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:29:22.982 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:29:22.982 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:29:22.982 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:22.983 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 1818 2024-09-14T06:29:22.983 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:22.983 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:29:22.983 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:22.983 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:29:22.983 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.654740+0000", 2024-09-14T06:29:22.983 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:22.983 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:22.983 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:22.983 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:29:22.983 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.654740+0000", 2024-09-14T06:29:22.983 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:22.983 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:22.983 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:22.984 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:29:22.984 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.654741+0000", 2024-09-14T06:29:22.984 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.8700000000000002e-07 2024-09-14T06:29:22.984 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:22.984 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:22.984 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:29:22.984 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.654742+0000", 2024-09-14T06:29:22.984 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.604e-06 2024-09-14T06:29:22.984 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:22.984 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:22.984 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:29:22.984 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.654745+0000", 2024-09-14T06:29:22.984 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.067e-06 2024-09-14T06:29:22.984 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:22.985 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:22.985 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:29:22.985 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.654748+0000", 2024-09-14T06:29:22.985 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.599e-06 2024-09-14T06:29:22.985 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:22.985 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:22.985 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:29:22.985 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.675700+0000", 2024-09-14T06:29:22.985 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.020951612000000001 2024-09-14T06:29:22.985 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:22.985 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:22.985 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:29:22.985 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.675738+0000", 2024-09-14T06:29:22.985 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.8300999999999998e-05 2024-09-14T06:29:22.986 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:22.986 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:22.986 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:29:22.986 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.675815+0000", 2024-09-14T06:29:22.986 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.7088000000000005e-05 2024-09-14T06:29:22.986 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:22.986 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:29:22.986 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:22.986 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:22.986 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:22.986 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:1819 3.3 3:c0711257:::rbd_data.112d7dfbd766.000000000000001a:5 [read 290816~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-14T06:29:22.986 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:08.654751+0000", 2024-09-14T06:29:22.986 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 14.284544794, 2024-09-14T06:29:22.986 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.021190092000000001, 2024-09-14T06:29:22.986 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:29:22.988 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:29:22.988 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:29:22.988 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:29:22.988 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:29:22.988 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:22.988 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 1819 2024-09-14T06:29:22.988 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:22.988 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:29:22.988 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:22.989 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:29:22.989 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.654751+0000", 2024-09-14T06:29:22.989 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:22.989 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:22.989 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:22.989 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:29:22.989 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.654751+0000", 2024-09-14T06:29:22.989 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:22.989 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:22.989 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:22.989 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:29:22.989 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.654751+0000", 2024-09-14T06:29:22.989 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.2100000000000001e-07 2024-09-14T06:29:22.989 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:22.990 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:22.990 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:29:22.990 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.654752+0000", 2024-09-14T06:29:22.990 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.0500000000000002e-07 2024-09-14T06:29:22.990 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:22.990 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:22.990 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:29:22.990 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.654755+0000", 2024-09-14T06:29:22.990 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.9799999999999998e-06 2024-09-14T06:29:22.990 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:22.990 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:22.990 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:29:22.990 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.654758+0000", 2024-09-14T06:29:22.990 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.48e-06 2024-09-14T06:29:22.990 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:22.991 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:22.991 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:29:22.991 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.675841+0000", 2024-09-14T06:29:22.991 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.021083085000000001 2024-09-14T06:29:22.991 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:22.991 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:22.991 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:29:22.991 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.675879+0000", 2024-09-14T06:29:22.991 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.8429000000000001e-05 2024-09-14T06:29:22.991 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:22.991 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:22.991 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:29:22.991 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.675941+0000", 2024-09-14T06:29:22.992 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.1991999999999999e-05 2024-09-14T06:29:22.992 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:22.992 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:29:22.992 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:22.992 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:22.992 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:22.992 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:1820 3.3 3:c0711257:::rbd_data.112d7dfbd766.000000000000001a:5 [read 307200~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-14T06:29:22.992 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:08.654762+0000", 2024-09-14T06:29:22.992 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 14.284533525000001, 2024-09-14T06:29:22.992 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.021318159, 2024-09-14T06:29:22.992 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:29:22.992 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:29:22.992 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:29:22.992 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:29:22.993 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:29:22.993 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:22.993 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 1820 2024-09-14T06:29:22.993 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:22.993 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:29:22.993 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:22.993 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:29:22.993 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.654762+0000", 2024-09-14T06:29:22.993 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:22.993 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:22.993 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:22.993 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:29:22.993 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.654762+0000", 2024-09-14T06:29:22.994 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:22.994 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:22.994 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:22.994 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:29:22.994 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.654763+0000", 2024-09-14T06:29:22.994 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.23e-07 2024-09-14T06:29:22.994 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:22.994 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:22.994 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:29:22.994 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.654764+0000", 2024-09-14T06:29:22.994 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.879999999999999e-07 2024-09-14T06:29:22.994 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:22.994 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:22.994 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:29:22.995 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.654766+0000", 2024-09-14T06:29:22.995 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.9500000000000001e-06 2024-09-14T06:29:22.995 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:22.995 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:22.995 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:29:22.995 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.654769+0000", 2024-09-14T06:29:22.995 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.5799999999999999e-06 2024-09-14T06:29:22.995 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:22.995 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:22.995 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:29:22.995 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.675972+0000", 2024-09-14T06:29:22.995 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.021202478 2024-09-14T06:29:22.995 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:22.995 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:22.996 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:29:22.996 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.676011+0000", 2024-09-14T06:29:22.996 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.9412999999999997e-05 2024-09-14T06:29:22.996 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:22.996 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:22.996 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:29:22.996 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.676080+0000", 2024-09-14T06:29:22.996 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.9527e-05 2024-09-14T06:29:22.996 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:22.996 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:29:22.996 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:22.996 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:22.996 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:22.996 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:1821 3.3 3:c0711257:::rbd_data.112d7dfbd766.000000000000001a:5 [read 323584~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-14T06:29:22.997 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:08.654772+0000", 2024-09-14T06:29:22.997 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 14.284523641, 2024-09-14T06:29:22.997 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.021435718999999999, 2024-09-14T06:29:22.997 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:29:22.997 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:29:22.997 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:29:22.997 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:29:22.997 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:29:22.997 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:22.997 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 1821 2024-09-14T06:29:22.997 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:22.997 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:29:22.997 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:22.997 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:29:22.998 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.654772+0000", 2024-09-14T06:29:22.998 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:22.998 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:22.998 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:22.998 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:29:22.998 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.654772+0000", 2024-09-14T06:29:22.998 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:22.998 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:22.998 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:22.998 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:29:22.998 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.654772+0000", 2024-09-14T06:29:22.998 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.2399999999999999e-07 2024-09-14T06:29:22.998 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:22.998 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:22.999 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:29:22.999 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.654773+0000", 2024-09-14T06:29:22.999 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.6199999999999996e-07 2024-09-14T06:29:22.999 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:22.999 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:22.999 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:29:22.999 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.654776+0000", 2024-09-14T06:29:22.999 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.8899999999999999e-06 2024-09-14T06:29:22.999 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:22.999 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:22.999 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:29:22.999 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.654779+0000", 2024-09-14T06:29:22.999 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.5620000000000002e-06 2024-09-14T06:29:23.000 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:23.000 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:23.000 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:29:23.000 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.676107+0000", 2024-09-14T06:29:23.000 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.021328203 2024-09-14T06:29:23.000 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:23.000 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:23.000 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:29:23.000 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.676155+0000", 2024-09-14T06:29:23.000 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.8109999999999998e-05 2024-09-14T06:29:23.000 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:23.000 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:23.000 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:29:23.000 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.676208+0000", 2024-09-14T06:29:23.000 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.2868000000000003e-05 2024-09-14T06:29:23.001 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:23.001 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:29:23.001 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:23.001 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:23.001 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:23.001 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:1822 3.3 3:c0711257:::rbd_data.112d7dfbd766.000000000000001a:5 [read 339968~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-14T06:29:23.001 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:08.654782+0000", 2024-09-14T06:29:23.001 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 14.284514046, 2024-09-14T06:29:23.001 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.021547572000000001, 2024-09-14T06:29:23.001 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:29:23.001 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:29:23.001 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:29:23.001 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:29:23.002 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:29:23.002 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:23.002 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 1822 2024-09-14T06:29:23.002 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:23.002 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:29:23.002 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:23.002 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:29:23.002 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.654782+0000", 2024-09-14T06:29:23.002 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:23.002 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:23.002 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:23.002 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:29:23.002 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.654782+0000", 2024-09-14T06:29:23.003 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:23.003 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:23.003 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:23.003 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:29:23.003 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.654782+0000", 2024-09-14T06:29:23.003 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.22e-07 2024-09-14T06:29:23.003 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:23.003 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:23.003 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:29:23.003 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.654783+0000", 2024-09-14T06:29:23.003 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.0699999999999996e-07 2024-09-14T06:29:23.003 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:23.003 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:23.003 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:29:23.003 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.654786+0000", 2024-09-14T06:29:23.004 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.88e-06 2024-09-14T06:29:23.004 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:23.004 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:23.004 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:29:23.004 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.654788+0000", 2024-09-14T06:29:23.004 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.464e-06 2024-09-14T06:29:23.004 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:23.004 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:23.004 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:29:23.004 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.676232+0000", 2024-09-14T06:29:23.004 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.021443502 2024-09-14T06:29:23.004 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:23.004 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:23.004 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:29:23.005 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.676271+0000", 2024-09-14T06:29:23.005 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.8871000000000001e-05 2024-09-14T06:29:23.005 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:23.005 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:23.005 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:29:23.005 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.676329+0000", 2024-09-14T06:29:23.005 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.8726e-05 2024-09-14T06:29:23.005 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:23.005 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:29:23.005 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:23.005 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:23.005 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:23.005 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:1823 3.3 3:c0711257:::rbd_data.112d7dfbd766.000000000000001a:5 [read 356352~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-14T06:29:23.005 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:08.654792+0000", 2024-09-14T06:29:23.006 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 14.284503333, 2024-09-14T06:29:23.006 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.021659046000000001, 2024-09-14T06:29:23.006 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:29:23.006 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:29:23.006 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:29:23.006 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:29:23.006 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:29:23.006 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:23.006 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 1823 2024-09-14T06:29:23.006 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:23.006 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:29:23.006 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:23.006 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:29:23.006 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.654792+0000", 2024-09-14T06:29:23.006 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:23.007 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:23.007 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:23.007 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:29:23.007 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.654792+0000", 2024-09-14T06:29:23.007 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:23.007 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:23.007 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:23.007 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:29:23.007 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.654793+0000", 2024-09-14T06:29:23.007 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.11e-07 2024-09-14T06:29:23.007 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:23.007 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:23.007 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:29:23.007 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.654794+0000", 2024-09-14T06:29:23.008 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.1399999999999996e-07 2024-09-14T06:29:23.008 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:23.008 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:23.008 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:29:23.008 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.654796+0000", 2024-09-14T06:29:23.008 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.88e-06 2024-09-14T06:29:23.008 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:23.008 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:23.008 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:29:23.008 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.654799+0000", 2024-09-14T06:29:23.008 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.9289999999999998e-06 2024-09-14T06:29:23.008 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:23.008 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:23.009 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:29:23.009 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.676351+0000", 2024-09-14T06:29:23.009 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.021552136999999999 2024-09-14T06:29:23.009 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:23.009 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:23.009 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:29:23.009 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.676398+0000", 2024-09-14T06:29:23.009 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.6660999999999999e-05 2024-09-14T06:29:23.009 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:23.009 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:23.009 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:29:23.009 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.676452+0000", 2024-09-14T06:29:23.009 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.3414000000000001e-05 2024-09-14T06:29:23.009 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:23.010 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:29:23.010 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:23.010 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:23.010 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:23.010 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:1824 3.3 3:c0711257:::rbd_data.112d7dfbd766.000000000000001a:5 [read 372736~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-14T06:29:23.010 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:08.654802+0000", 2024-09-14T06:29:23.010 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 14.284493499, 2024-09-14T06:29:23.010 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.021747074000000002, 2024-09-14T06:29:23.010 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:29:23.010 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:29:23.010 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:29:23.010 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:29:23.010 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:29:23.010 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:23.011 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 1824 2024-09-14T06:29:23.011 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:23.011 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:29:23.011 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:23.011 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:29:23.011 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.654802+0000", 2024-09-14T06:29:23.011 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:23.011 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:23.011 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:23.011 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:29:23.011 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.654802+0000", 2024-09-14T06:29:23.011 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:23.011 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:23.011 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:23.011 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:29:23.012 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.654803+0000", 2024-09-14T06:29:23.012 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.0699999999999999e-07 2024-09-14T06:29:23.012 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:23.012 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:23.012 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:29:23.012 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.654803+0000", 2024-09-14T06:29:23.012 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.3799999999999996e-07 2024-09-14T06:29:23.012 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:23.012 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:23.012 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:29:23.012 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.654807+0000", 2024-09-14T06:29:23.012 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.6650000000000001e-06 2024-09-14T06:29:23.012 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:23.012 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:23.013 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:29:23.013 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.654810+0000", 2024-09-14T06:29:23.013 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.5409999999999999e-06 2024-09-14T06:29:23.013 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:23.013 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:23.013 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:29:23.013 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.676464+0000", 2024-09-14T06:29:23.013 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.021654651 2024-09-14T06:29:23.013 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:23.013 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:23.013 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:29:23.013 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.676501+0000", 2024-09-14T06:29:23.013 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.7073000000000002e-05 2024-09-14T06:29:23.013 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:23.014 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:23.014 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:29:23.014 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.676549+0000", 2024-09-14T06:29:23.014 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.7998999999999999e-05 2024-09-14T06:29:23.014 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:23.014 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:29:23.014 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:23.014 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:23.014 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:23.014 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:1825 3.3 3:c0711257:::rbd_data.112d7dfbd766.000000000000001a:5 [read 389120~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-14T06:29:23.014 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:08.654813+0000", 2024-09-14T06:29:23.014 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 14.284483128, 2024-09-14T06:29:23.014 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.021865797999999999, 2024-09-14T06:29:23.014 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:29:23.014 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:29:23.015 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:29:23.015 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:29:23.015 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:29:23.015 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:23.015 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 1825 2024-09-14T06:29:23.015 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:23.015 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:29:23.015 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:23.015 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:29:23.015 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.654813+0000", 2024-09-14T06:29:23.015 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:23.015 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:23.015 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:23.015 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:29:23.016 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.654813+0000", 2024-09-14T06:29:23.016 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:23.016 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:23.016 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:23.016 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:29:23.016 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.654813+0000", 2024-09-14T06:29:23.016 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.11e-07 2024-09-14T06:29:23.016 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:23.016 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:23.016 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:29:23.016 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.654814+0000", 2024-09-14T06:29:23.016 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.511e-06 2024-09-14T06:29:23.016 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:23.016 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:23.017 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:29:23.017 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.654817+0000", 2024-09-14T06:29:23.017 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.0580000000000002e-06 2024-09-14T06:29:23.017 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:23.017 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:23.017 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:29:23.017 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.654820+0000", 2024-09-14T06:29:23.017 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.4530000000000001e-06 2024-09-14T06:29:23.017 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:23.017 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:23.017 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:29:23.017 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.676571+0000", 2024-09-14T06:29:23.017 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.021750698999999998 2024-09-14T06:29:23.017 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:23.017 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:23.018 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:29:23.018 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.676617+0000", 2024-09-14T06:29:23.018 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.6226000000000001e-05 2024-09-14T06:29:23.018 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:23.018 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:23.018 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:29:23.018 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.676678+0000", 2024-09-14T06:29:23.018 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.1639999999999999e-05 2024-09-14T06:29:23.018 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:23.018 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:29:23.018 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:23.018 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:23.018 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:23.018 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:1826 3.3 3:c0711257:::rbd_data.112d7dfbd766.000000000000001a:5 [read 405504~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-14T06:29:23.019 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:08.654823+0000", 2024-09-14T06:29:23.019 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 14.284472692, 2024-09-14T06:29:23.019 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.022032890999999999, 2024-09-14T06:29:23.019 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:29:23.019 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:29:23.019 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:29:23.019 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:29:23.019 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:29:23.019 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:23.019 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 1826 2024-09-14T06:29:23.019 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:23.019 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:29:23.019 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:23.019 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:29:23.020 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.654823+0000", 2024-09-14T06:29:23.020 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:23.020 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:23.020 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:23.020 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:29:23.020 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.654823+0000", 2024-09-14T06:29:23.020 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:23.020 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:23.020 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:23.020 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:29:23.020 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.654823+0000", 2024-09-14T06:29:23.020 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.1299999999999999e-07 2024-09-14T06:29:23.020 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:23.021 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:23.021 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:29:23.021 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.654824+0000", 2024-09-14T06:29:23.021 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.6300000000000004e-07 2024-09-14T06:29:23.021 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:23.021 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:23.021 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:29:23.021 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.654827+0000", 2024-09-14T06:29:23.021 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.0230000000000001e-06 2024-09-14T06:29:23.021 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:23.021 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:23.021 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:29:23.021 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.654830+0000", 2024-09-14T06:29:23.021 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.7650000000000002e-06 2024-09-14T06:29:23.021 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:23.022 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:23.022 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:29:23.022 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.676702+0000", 2024-09-14T06:29:23.022 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.021871561000000001 2024-09-14T06:29:23.022 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:23.022 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:23.022 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:29:23.022 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.676737+0000", 2024-09-14T06:29:23.022 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.5506000000000002e-05 2024-09-14T06:29:23.022 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:23.022 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:23.022 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:29:23.022 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.676856+0000", 2024-09-14T06:29:23.023 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00011896 2024-09-14T06:29:23.023 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:23.023 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:29:23.023 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:23.023 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:23.023 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:23.023 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:1827 3.3 3:c0711257:::rbd_data.112d7dfbd766.000000000000001a:5 [read 421888~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-14T06:29:23.023 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:08.654833+0000", 2024-09-14T06:29:23.023 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 14.284462414, 2024-09-14T06:29:23.023 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.022281015000000001, 2024-09-14T06:29:23.023 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:29:23.023 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:29:23.023 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:29:23.023 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:29:23.024 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:29:23.024 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:23.024 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 1827 2024-09-14T06:29:23.024 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:23.024 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:29:23.024 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:23.024 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:29:23.024 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.654833+0000", 2024-09-14T06:29:23.024 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:23.024 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:23.024 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:23.024 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:29:23.025 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.654833+0000", 2024-09-14T06:29:23.025 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:23.025 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:23.025 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:23.025 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:29:23.025 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.654834+0000", 2024-09-14T06:29:23.025 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.7e-07 2024-09-14T06:29:23.025 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:23.025 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:23.025 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:29:23.025 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.654835+0000", 2024-09-14T06:29:23.025 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.3900000000000003e-07 2024-09-14T06:29:23.025 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:23.025 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:23.026 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:29:23.026 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.654838+0000", 2024-09-14T06:29:23.026 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.909e-06 2024-09-14T06:29:23.026 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:23.026 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:23.026 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:29:23.026 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.654840+0000", 2024-09-14T06:29:23.026 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.4739999999999999e-06 2024-09-14T06:29:23.026 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:23.026 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:23.026 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:29:23.026 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.676880+0000", 2024-09-14T06:29:23.026 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.022039741000000002 2024-09-14T06:29:23.026 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:23.027 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:23.027 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:29:23.027 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.676926+0000", 2024-09-14T06:29:23.027 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.6344000000000002e-05 2024-09-14T06:29:23.027 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:23.027 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:23.027 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:29:23.027 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.677114+0000", 2024-09-14T06:29:23.027 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.000188238 2024-09-14T06:29:23.027 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:23.027 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:29:23.027 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:23.027 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:23.027 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:23.028 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:1828 3.3 3:c0711257:::rbd_data.112d7dfbd766.000000000000001a:5 [read 438272~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-14T06:29:23.028 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:08.654855+0000", 2024-09-14T06:29:23.028 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 14.284441264, 2024-09-14T06:29:23.028 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.022438038, 2024-09-14T06:29:23.028 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:29:23.028 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:29:23.028 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:29:23.028 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:29:23.028 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:29:23.028 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:23.028 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 1828 2024-09-14T06:29:23.028 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:23.028 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:29:23.028 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:23.029 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:29:23.029 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.654855+0000", 2024-09-14T06:29:23.029 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:23.029 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:23.029 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:23.029 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:29:23.029 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.654855+0000", 2024-09-14T06:29:23.029 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:23.029 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:23.029 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:23.029 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:29:23.029 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.654855+0000", 2024-09-14T06:29:23.029 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.9499999999999998e-07 2024-09-14T06:29:23.030 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:23.030 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:23.030 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:29:23.030 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.654856+0000", 2024-09-14T06:29:23.030 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.403e-06 2024-09-14T06:29:23.030 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:23.030 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:23.030 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:29:23.030 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.654869+0000", 2024-09-14T06:29:23.030 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.2459999999999999e-05 2024-09-14T06:29:23.030 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:23.030 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:23.030 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:29:23.030 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.654872+0000", 2024-09-14T06:29:23.030 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.072e-06 2024-09-14T06:29:23.031 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:23.031 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:23.031 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:29:23.031 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.677127+0000", 2024-09-14T06:29:23.031 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.022255119 2024-09-14T06:29:23.031 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:23.031 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:23.031 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:29:23.031 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.677179+0000", 2024-09-14T06:29:23.031 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.1736000000000001e-05 2024-09-14T06:29:23.031 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:23.031 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:23.031 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:29:23.031 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.677293+0000", 2024-09-14T06:29:23.031 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.000113953 2024-09-14T06:29:23.032 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:23.032 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:29:23.032 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:23.032 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:23.032 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:29:23.032 INFO:teuthology.orchestra.run.smithi138.stdout:} 2024-09-14T06:29:23.033 INFO:teuthology.orchestra.run.smithi094.stdout:osd.3: {} 2024-09-14T06:29:23.033 INFO:teuthology.orchestra.run.smithi094.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-14T06:29:23.034 INFO:teuthology.orchestra.run.smithi094.stdout:osd.4: {} 2024-09-14T06:29:23.034 INFO:teuthology.orchestra.run.smithi094.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-14T06:29:23.034 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-14T06:29:23.035 INFO:tasks.ceph.osd.4.smithi138.stderr:2024-09-14T06:29:23.032+0000 7f6c653c8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33615) UID: 0 2024-09-14T06:29:23.042 INFO:teuthology.orchestra.run.smithi094.stdout:osd.5: {} 2024-09-14T06:29:23.042 INFO:teuthology.orchestra.run.smithi094.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-14T06:29:23.058 INFO:teuthology.orchestra.run.smithi094.stdout:osd.6: {} 2024-09-14T06:29:23.059 INFO:teuthology.orchestra.run.smithi094.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-14T06:29:23.076 INFO:teuthology.orchestra.run.smithi094.stdout:osd.7: {} 2024-09-14T06:29:23.076 INFO:teuthology.orchestra.run.smithi094.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-14T06:29:23.134 INFO:tasks.ceph.osd.4.smithi138.stderr:2024-09-14T06:29:23.133+0000 7f6c653c8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33615) UID: 0 2024-09-14T06:29:23.178 INFO:teuthology.orchestra.run.smithi138.stdout:ERROR: (22) Invalid argument 2024-09-14T06:29:23.178 INFO:teuthology.orchestra.run.smithi138.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:29:23.186 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-14T06:29:23.235 INFO:tasks.ceph.osd.0.smithi094.stderr:2024-09-14T06:29:23.235+0000 7f07c9333640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33636) UID: 0 2024-09-14T06:29:23.330 INFO:teuthology.orchestra.run.smithi138.stdout:ERROR: (22) Invalid argument 2024-09-14T06:29:23.330 INFO:teuthology.orchestra.run.smithi138.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:29:23.336 INFO:tasks.ceph.osd.7.smithi138.stderr:2024-09-14T06:29:23.335+0000 7f3c6a21d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33623) UID: 0 2024-09-14T06:29:23.338 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-14T06:29:23.437 INFO:tasks.ceph.osd.4.smithi138.stderr:2024-09-14T06:29:23.436+0000 7f6c653c8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33615) UID: 0 2024-09-14T06:29:23.481 INFO:teuthology.orchestra.run.smithi138.stdout:ERROR: (22) Invalid argument 2024-09-14T06:29:23.481 INFO:teuthology.orchestra.run.smithi138.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:29:23.490 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-14T06:29:23.539 INFO:tasks.ceph.osd.3.smithi094.stderr:2024-09-14T06:29:23.538+0000 7feeeafe8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33655) UID: 0 2024-09-14T06:29:23.593 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-14T06:29:23.633 INFO:teuthology.orchestra.run.smithi138.stdout:ERROR: (22) Invalid argument 2024-09-14T06:29:23.634 INFO:teuthology.orchestra.run.smithi138.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:29:23.639 INFO:tasks.ceph.osd.2.smithi094.stderr:2024-09-14T06:29:23.638+0000 7faf3269b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33649) UID: 0 2024-09-14T06:29:23.642 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-14T06:29:23.740 INFO:tasks.ceph.osd.7.smithi138.stderr:2024-09-14T06:29:23.739+0000 7f3c6a21d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33623) UID: 0 2024-09-14T06:29:23.747 INFO:teuthology.orchestra.run.smithi138.stdout:ERROR: (22) Invalid argument 2024-09-14T06:29:23.747 INFO:teuthology.orchestra.run.smithi138.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:29:23.755 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-14T06:29:23.811 INFO:teuthology.orchestra.run.smithi094.stdout:checking clone #4, image image_client.0-clone4 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent5 2024-09-14T06:29:23.841 INFO:tasks.ceph.osd.4.smithi138.stderr:2024-09-14T06:29:23.840+0000 7f6c653c8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33615) UID: 0 2024-09-14T06:29:23.900 INFO:teuthology.orchestra.run.smithi138.stdout:ERROR: (22) Invalid argument 2024-09-14T06:29:23.900 INFO:teuthology.orchestra.run.smithi138.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:29:23.909 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-14T06:29:23.950 INFO:tasks.ceph.osd.6.smithi138.stderr:2024-09-14T06:29:23.949+0000 7f8cd8849640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33620) UID: 0 2024-09-14T06:29:24.040 INFO:teuthology.orchestra.run.smithi094.stdout:531 trunc from 0xea106a4 to 0x7ea1aad 2024-09-14T06:29:24.044 INFO:tasks.ceph.osd.0.smithi094.stderr:2024-09-14T06:29:24.043+0000 7f07c9333640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33636) UID: 0 2024-09-14T06:29:24.049 INFO:teuthology.orchestra.run.smithi094.stdout:532 read 0x1423c1 thru 0x1446fe (0x233e bytes) 2024-09-14T06:29:24.052 INFO:teuthology.orchestra.run.smithi138.stdout:ERROR: (22) Invalid argument 2024-09-14T06:29:24.053 INFO:teuthology.orchestra.run.smithi138.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:29:24.058 INFO:teuthology.orchestra.run.smithi094.stdout:534 write 0xa292447 thru 0xa298926 (0x64e0 bytes) 2024-09-14T06:29:24.061 INFO:teuthology.orchestra.run.smithi094.stdout:535 write 0x7c5706d thru 0x7c60637 (0x95cb bytes) 2024-09-14T06:29:24.062 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-14T06:29:24.063 INFO:teuthology.orchestra.run.smithi094.stdout:536 write 0x944e8d8 thru 0x944f0da (0x803 bytes) 2024-09-14T06:29:24.066 INFO:teuthology.orchestra.run.smithi094.stdout:537 punch from 0xa18c546 to 0xa18c8d3, (0x38d bytes) 2024-09-14T06:29:24.075 INFO:teuthology.orchestra.run.smithi094.stdout:538 write 0xd3065b3 thru 0xd30e733 (0x8181 bytes) 2024-09-14T06:29:24.078 INFO:teuthology.orchestra.run.smithi094.stdout:541 read 0x3593882 thru 0x359856c (0x4ceb bytes) 2024-09-14T06:29:24.081 INFO:teuthology.orchestra.run.smithi094.stdout:542 trunc from 0xd30e734 to 0xb3c71a2 2024-09-14T06:29:24.090 INFO:teuthology.orchestra.run.smithi094.stdout:543 clone 7 order 18 su 1024 sc 10 2024-09-14T06:29:24.145 INFO:tasks.ceph.osd.0.smithi094.stderr:2024-09-14T06:29:24.144+0000 7f07c9333640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33636) UID: 0 2024-09-14T06:29:24.164 INFO:teuthology.orchestra.run.smithi138.stdout:ERROR: (22) Invalid argument 2024-09-14T06:29:24.164 INFO:teuthology.orchestra.run.smithi138.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:29:24.173 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-14T06:29:24.246 INFO:tasks.ceph.osd.0.smithi094.stderr:2024-09-14T06:29:24.245+0000 7f07c9333640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33636) UID: 0 2024-09-14T06:29:24.276 INFO:teuthology.orchestra.run.smithi138.stdout:ERROR: (22) Invalid argument 2024-09-14T06:29:24.276 INFO:teuthology.orchestra.run.smithi138.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:29:24.285 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-14T06:29:24.347 INFO:tasks.ceph.osd.3.smithi094.stderr:2024-09-14T06:29:24.346+0000 7feeeafe8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33655) UID: 0 2024-09-14T06:29:24.389 INFO:teuthology.orchestra.run.smithi094.stdout:ERROR: (22) Invalid argument 2024-09-14T06:29:24.389 INFO:teuthology.orchestra.run.smithi094.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:29:24.397 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-14T06:29:24.482 INFO:tasks.ceph.osd.3.smithi094.stderr:2024-09-14T06:29:24.481+0000 7feeeafe8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33655) UID: 0 2024-09-14T06:29:24.540 INFO:teuthology.orchestra.run.smithi094.stdout:ERROR: (22) Invalid argument 2024-09-14T06:29:24.541 INFO:teuthology.orchestra.run.smithi094.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:29:24.549 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-14T06:29:24.550 INFO:tasks.ceph.osd.0.smithi094.stderr:2024-09-14T06:29:24.548+0000 7f07c9333640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33636) UID: 0 2024-09-14T06:29:24.650 INFO:tasks.ceph.osd.3.smithi094.stderr:2024-09-14T06:29:24.648+0000 7feeeafe8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33655) UID: 0 2024-09-14T06:29:24.656 INFO:teuthology.orchestra.run.smithi094.stderr:noscrub is set 2024-09-14T06:29:24.700 INFO:teuthology.orchestra.run.smithi094.stdout:ERROR: (22) Invalid argument 2024-09-14T06:29:24.700 INFO:teuthology.orchestra.run.smithi094.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:29:24.710 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-14T06:29:24.751 INFO:tasks.ceph.osd.5.smithi138.stderr:2024-09-14T06:29:24.749+0000 7fe13746f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33618) UID: 0 2024-09-14T06:29:24.852 INFO:tasks.ceph.osd.6.smithi138.stderr:2024-09-14T06:29:24.851+0000 7f8cd8849640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33620) UID: 0 2024-09-14T06:29:24.854 INFO:teuthology.orchestra.run.smithi094.stdout:ERROR: (22) Invalid argument 2024-09-14T06:29:24.854 INFO:teuthology.orchestra.run.smithi094.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:29:24.862 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-14T06:29:24.953 INFO:tasks.ceph.osd.0.smithi094.stderr:2024-09-14T06:29:24.952+0000 7f07c9333640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33636) UID: 0 2024-09-14T06:29:24.964 INFO:teuthology.orchestra.run.smithi094.stdout:ERROR: (22) Invalid argument 2024-09-14T06:29:24.965 INFO:teuthology.orchestra.run.smithi094.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:29:24.973 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-14T06:29:25.058 INFO:tasks.ceph.osd.3.smithi094.stderr:2024-09-14T06:29:25.057+0000 7feeeafe8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33655) UID: 0 2024-09-14T06:29:25.087 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-14T06:29:25.119 INFO:teuthology.orchestra.run.smithi094.stdout:ERROR: (22) Invalid argument 2024-09-14T06:29:25.119 INFO:teuthology.orchestra.run.smithi094.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:29:25.128 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-14T06:29:25.170 INFO:tasks.ceph.osd.0.smithi094.stderr:2024-09-14T06:29:25.169+0000 7f07c9333640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33636) UID: 0 2024-09-14T06:29:25.210 INFO:teuthology.orchestra.run.smithi094.stdout:osd.0: {} 2024-09-14T06:29:25.211 INFO:teuthology.orchestra.run.smithi094.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-14T06:29:25.227 INFO:teuthology.orchestra.run.smithi094.stdout:osd.1: {} 2024-09-14T06:29:25.228 INFO:teuthology.orchestra.run.smithi094.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-14T06:29:25.244 INFO:teuthology.orchestra.run.smithi094.stdout:osd.2: {} 2024-09-14T06:29:25.244 INFO:teuthology.orchestra.run.smithi094.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-14T06:29:25.255 INFO:tasks.ceph.osd.4.smithi138.stderr:2024-09-14T06:29:25.254+0000 7f6c653c8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33615) UID: 0 2024-09-14T06:29:25.260 INFO:teuthology.orchestra.run.smithi094.stdout:osd.3: {} 2024-09-14T06:29:25.260 INFO:teuthology.orchestra.run.smithi094.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-14T06:29:25.276 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-14T06:29:25.276 INFO:teuthology.orchestra.run.smithi094.stdout: "ops": [], 2024-09-14T06:29:25.276 INFO:teuthology.orchestra.run.smithi094.stdout: "num_ops": 0 2024-09-14T06:29:25.276 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-14T06:29:25.277 INFO:teuthology.orchestra.run.smithi094.stdout:osd.4: {} 2024-09-14T06:29:25.277 INFO:teuthology.orchestra.run.smithi094.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-14T06:29:25.285 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-14T06:29:25.294 INFO:teuthology.orchestra.run.smithi094.stdout:osd.5: {} 2024-09-14T06:29:25.295 INFO:teuthology.orchestra.run.smithi094.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-14T06:29:25.312 INFO:teuthology.orchestra.run.smithi094.stdout:osd.6: {} 2024-09-14T06:29:25.312 INFO:teuthology.orchestra.run.smithi094.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-14T06:29:25.329 INFO:teuthology.orchestra.run.smithi094.stdout:osd.7: {} 2024-09-14T06:29:25.329 INFO:teuthology.orchestra.run.smithi094.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-14T06:29:25.385 INFO:tasks.ceph.osd.1.smithi094.stderr:2024-09-14T06:29:25.384+0000 7f328e47a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33641) UID: 0 2024-09-14T06:29:25.403 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-14T06:29:25.404 INFO:teuthology.orchestra.run.smithi094.stdout: "ops": [], 2024-09-14T06:29:25.404 INFO:teuthology.orchestra.run.smithi094.stdout: "complaint_time": 30, 2024-09-14T06:29:25.404 INFO:teuthology.orchestra.run.smithi094.stdout: "num_blocked_ops": 0 2024-09-14T06:29:25.404 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-14T06:29:25.413 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-14T06:29:25.497 INFO:tasks.ceph.osd.2.smithi094.stderr:2024-09-14T06:29:25.496+0000 7faf3269b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33649) UID: 0 2024-09-14T06:29:25.558 INFO:tasks.ceph.osd.5.smithi138.stderr:2024-09-14T06:29:25.557+0000 7fe13746f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33618) UID: 0 2024-09-14T06:29:25.559 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-14T06:29:25.559 INFO:teuthology.orchestra.run.smithi094.stdout: "size": 20, 2024-09-14T06:29:25.559 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 600, 2024-09-14T06:29:25.559 INFO:teuthology.orchestra.run.smithi094.stdout: "ops": [ 2024-09-14T06:29:25.559 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:25.559 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4299.0:1 2.3 e20/19 2:c4c92e5a:::rbd_trash:head v 20'1, mlcod=0'0)", 2024-09-14T06:29:25.560 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:28:51.223664+0000", 2024-09-14T06:29:25.560 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 34.333077394999997, 2024-09-14T06:29:25.560 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.001165588, 2024-09-14T06:29:25.560 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:29:25.560 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:29:25.560 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:29:25.560 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:29:25.560 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:25.560 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:29:25.560 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:28:51.223664+0000", 2024-09-14T06:29:25.560 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:25.560 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:25.561 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:25.561 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:29:25.561 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:28:51.223664+0000", 2024-09-14T06:29:25.561 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:25.561 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:25.561 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:25.561 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:29:25.561 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:28:51.223665+0000", 2024-09-14T06:29:25.561 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.3600000000000003e-07 2024-09-14T06:29:25.561 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:25.561 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:25.562 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:29:25.562 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:28:51.223672+0000", 2024-09-14T06:29:25.562 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.4740000000000002e-06 2024-09-14T06:29:25.562 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:25.562 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:25.562 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:29:25.562 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:28:51.223695+0000", 2024-09-14T06:29:25.562 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.3859e-05 2024-09-14T06:29:25.562 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:25.562 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:25.562 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:29:25.562 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:28:51.223715+0000", 2024-09-14T06:29:25.562 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.9406e-05 2024-09-14T06:29:25.562 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:25.563 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:25.563 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:29:25.563 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:28:51.223780+0000", 2024-09-14T06:29:25.563 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.4777999999999995e-05 2024-09-14T06:29:25.563 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:25.563 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:25.563 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:29:25.563 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:28:51.223817+0000", 2024-09-14T06:29:25.563 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.7660999999999997e-05 2024-09-14T06:29:25.563 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:25.563 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:25.563 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-14T06:29:25.564 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:28:51.224781+0000", 2024-09-14T06:29:25.564 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00096367800000000004 2024-09-14T06:29:25.564 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:25.564 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:25.564 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:29:25.564 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:28:51.224830+0000", 2024-09-14T06:29:25.564 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.9095999999999999e-05 2024-09-14T06:29:25.564 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:25.564 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:25.564 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:25.565 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:25.565 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:25.565 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4397.0:13 3.4 e26/23 3:27567f25:::rbd_object_map.112d7adb82ff:head v 26'1, mlcod=0'0)", 2024-09-14T06:29:25.565 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:02.590205+0000", 2024-09-14T06:29:25.565 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 22.966536640000001, 2024-09-14T06:29:25.565 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00092769300000000005, 2024-09-14T06:29:25.565 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:29:25.565 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:29:25.565 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:29:25.565 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:29:25.565 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:25.565 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:29:25.565 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:02.590205+0000", 2024-09-14T06:29:25.565 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:25.566 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:25.566 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:25.566 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:29:25.566 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:02.590205+0000", 2024-09-14T06:29:25.566 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:25.566 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:25.566 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:25.566 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:29:25.566 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:02.590205+0000", 2024-09-14T06:29:25.566 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.5699999999999999e-07 2024-09-14T06:29:25.566 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:25.566 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:25.566 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:29:25.566 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:02.590211+0000", 2024-09-14T06:29:25.567 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.2209999999999996e-06 2024-09-14T06:29:25.567 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:25.567 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:25.567 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:29:25.567 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:02.590220+0000", 2024-09-14T06:29:25.567 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.5359999999999998e-06 2024-09-14T06:29:25.567 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:25.567 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:25.567 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:29:25.567 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:02.590227+0000", 2024-09-14T06:29:25.567 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.4790000000000001e-06 2024-09-14T06:29:25.567 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:25.567 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:25.567 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:29:25.568 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:02.590286+0000", 2024-09-14T06:29:25.568 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.9948999999999998e-05 2024-09-14T06:29:25.568 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:25.568 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:25.568 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:29:25.568 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:02.590308+0000", 2024-09-14T06:29:25.568 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.1580999999999999e-05 2024-09-14T06:29:25.568 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:25.568 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:25.568 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-14T06:29:25.568 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:02.591104+0000", 2024-09-14T06:29:25.568 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00079564299999999996 2024-09-14T06:29:25.569 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:25.569 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:25.569 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:29:25.569 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:02.591133+0000", 2024-09-14T06:29:25.569 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.9127000000000001e-05 2024-09-14T06:29:25.569 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:25.569 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:25.569 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:25.569 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:25.569 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:25.569 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4397.0:57 3.4 e27/23 3:27567f25:::rbd_object_map.112d7adb82ff:head v 27'5, mlcod=27'3)", 2024-09-14T06:29:25.569 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:04.297136+0000", 2024-09-14T06:29:25.569 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 21.259605834999999, 2024-09-14T06:29:25.570 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00086588099999999999, 2024-09-14T06:29:25.570 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:29:25.570 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:29:25.570 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:29:25.570 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:29:25.570 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:25.570 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:29:25.570 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:04.297136+0000", 2024-09-14T06:29:25.570 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:25.570 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:25.570 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:25.570 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:29:25.570 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:04.297136+0000", 2024-09-14T06:29:25.570 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:25.571 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:25.571 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:25.571 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:29:25.571 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:04.297137+0000", 2024-09-14T06:29:25.571 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.404e-06 2024-09-14T06:29:25.571 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:25.571 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:25.571 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:29:25.572 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:04.297159+0000", 2024-09-14T06:29:25.572 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.2005999999999999e-05 2024-09-14T06:29:25.572 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:25.572 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:25.572 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:29:25.572 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:04.297179+0000", 2024-09-14T06:29:25.572 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.963e-05 2024-09-14T06:29:25.572 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:25.572 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:25.572 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:29:25.572 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:04.297188+0000", 2024-09-14T06:29:25.572 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.0189999999999995e-06 2024-09-14T06:29:25.572 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:25.572 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:25.573 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:29:25.573 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:04.297255+0000", 2024-09-14T06:29:25.573 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.7112000000000004e-05 2024-09-14T06:29:25.573 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:25.573 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:25.573 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:29:25.573 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:04.297279+0000", 2024-09-14T06:29:25.573 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.4017000000000001e-05 2024-09-14T06:29:25.573 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:25.573 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:25.573 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-14T06:29:25.573 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:04.297966+0000", 2024-09-14T06:29:25.573 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00068650600000000003 2024-09-14T06:29:25.573 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:25.574 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:25.574 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:29:25.574 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:04.298002+0000", 2024-09-14T06:29:25.574 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.6186999999999998e-05 2024-09-14T06:29:25.574 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:25.574 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:25.574 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:25.574 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:25.574 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:25.574 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4397.0:66 3.4 e27/23 3:27567f25:::rbd_object_map.112d7adb82ff:head v 27'6, mlcod=27'4)", 2024-09-14T06:29:25.574 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:04.303066+0000", 2024-09-14T06:29:25.574 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 21.253676018, 2024-09-14T06:29:25.574 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00083700699999999996, 2024-09-14T06:29:25.574 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:29:25.574 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:29:25.575 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:29:25.575 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:29:25.575 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:25.575 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:29:25.575 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:04.303066+0000", 2024-09-14T06:29:25.575 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:25.575 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:25.575 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:25.575 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:29:25.575 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:04.303066+0000", 2024-09-14T06:29:25.575 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:25.575 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:25.575 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:25.576 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:29:25.576 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:04.303067+0000", 2024-09-14T06:29:25.576 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.1880000000000001e-06 2024-09-14T06:29:25.576 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:25.576 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:25.576 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:29:25.576 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:04.303075+0000", 2024-09-14T06:29:25.576 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.1410000000000005e-06 2024-09-14T06:29:25.576 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:25.576 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:25.577 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:29:25.577 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:04.303091+0000", 2024-09-14T06:29:25.577 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.5928000000000001e-05 2024-09-14T06:29:25.577 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:25.577 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:25.577 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:29:25.577 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:04.303100+0000", 2024-09-14T06:29:25.577 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.2019999999999993e-06 2024-09-14T06:29:25.577 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:25.577 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:25.577 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:29:25.577 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:04.303168+0000", 2024-09-14T06:29:25.577 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.7754000000000002e-05 2024-09-14T06:29:25.578 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:25.578 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:25.578 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:29:25.578 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:04.303191+0000", 2024-09-14T06:29:25.578 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.3434e-05 2024-09-14T06:29:25.578 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:25.578 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:25.578 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-14T06:29:25.578 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:04.303872+0000", 2024-09-14T06:29:25.578 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00068037399999999997 2024-09-14T06:29:25.578 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:25.578 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:25.578 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:29:25.578 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:04.303903+0000", 2024-09-14T06:29:25.578 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.0985999999999998e-05 2024-09-14T06:29:25.579 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:25.579 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:25.579 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:25.579 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:25.579 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:25.579 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4397.0:68 3.4 e27/23 3:27567f25:::rbd_object_map.112d7adb82ff:head v 27'7, mlcod=27'5)", 2024-09-14T06:29:25.579 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:04.314113+0000", 2024-09-14T06:29:25.579 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 21.242629158, 2024-09-14T06:29:25.579 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00086905299999999995, 2024-09-14T06:29:25.579 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:29:25.579 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:29:25.579 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:29:25.579 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:29:25.579 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:25.580 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:29:25.580 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:04.314113+0000", 2024-09-14T06:29:25.580 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:25.580 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:25.580 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:25.580 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:29:25.580 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:04.314113+0000", 2024-09-14T06:29:25.580 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:25.580 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:25.580 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:25.580 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:29:25.581 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:04.314114+0000", 2024-09-14T06:29:25.581 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.2780000000000001e-06 2024-09-14T06:29:25.581 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:25.581 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:25.581 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:29:25.581 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:04.314133+0000", 2024-09-14T06:29:25.581 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.95e-05 2024-09-14T06:29:25.581 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:25.581 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:25.581 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:29:25.581 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:04.314149+0000", 2024-09-14T06:29:25.581 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.5752000000000001e-05 2024-09-14T06:29:25.581 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:25.582 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:25.582 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:29:25.582 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:04.314163+0000", 2024-09-14T06:29:25.582 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.3910000000000001e-05 2024-09-14T06:29:25.582 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:25.582 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:25.582 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:29:25.582 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:04.314234+0000", 2024-09-14T06:29:25.582 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.0564999999999999e-05 2024-09-14T06:29:25.582 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:25.582 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:25.582 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:29:25.583 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:04.314258+0000", 2024-09-14T06:29:25.583 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.4040999999999999e-05 2024-09-14T06:29:25.583 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:25.583 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:25.583 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-14T06:29:25.583 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:04.314950+0000", 2024-09-14T06:29:25.583 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00069187399999999998 2024-09-14T06:29:25.583 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:25.583 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:25.583 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:29:25.583 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:04.314982+0000", 2024-09-14T06:29:25.583 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.2132999999999999e-05 2024-09-14T06:29:25.584 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:25.584 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:25.584 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:25.584 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:25.584 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:25.584 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4397.0:114 3.4 e27/23 3:27567f25:::rbd_object_map.112d7adb82ff:head v 27'16, mlcod=27'14)", 2024-09-14T06:29:25.584 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:04.372433+0000", 2024-09-14T06:29:25.584 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 21.184308458, 2024-09-14T06:29:25.584 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00088299699999999995, 2024-09-14T06:29:25.585 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:29:25.585 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:29:25.585 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:29:25.585 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:29:25.585 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:25.585 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:29:25.585 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:04.372433+0000", 2024-09-14T06:29:25.585 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:25.585 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:25.585 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:25.585 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:29:25.585 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:04.372433+0000", 2024-09-14T06:29:25.585 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:25.586 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:25.586 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:25.586 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:29:25.586 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:04.372435+0000", 2024-09-14T06:29:25.586 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.655e-06 2024-09-14T06:29:25.586 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:25.586 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:25.586 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:29:25.586 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:04.372443+0000", 2024-09-14T06:29:25.586 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.6809999999999998e-06 2024-09-14T06:29:25.586 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:25.586 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:25.586 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:29:25.586 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:04.372458+0000", 2024-09-14T06:29:25.587 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.4941e-05 2024-09-14T06:29:25.587 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:25.587 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:25.587 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:29:25.587 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:04.372475+0000", 2024-09-14T06:29:25.587 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.6886000000000001e-05 2024-09-14T06:29:25.587 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:25.587 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:25.587 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:29:25.587 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:04.372530+0000", 2024-09-14T06:29:25.587 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.507e-05 2024-09-14T06:29:25.587 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:25.587 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:25.587 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:29:25.588 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:04.372555+0000", 2024-09-14T06:29:25.588 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.5785000000000001e-05 2024-09-14T06:29:25.588 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:25.588 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:25.588 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-14T06:29:25.588 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:04.373275+0000", 2024-09-14T06:29:25.588 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00071917599999999997 2024-09-14T06:29:25.588 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:25.588 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:25.589 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:29:25.589 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:04.373316+0000", 2024-09-14T06:29:25.589 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.1802999999999999e-05 2024-09-14T06:29:25.589 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:25.589 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:25.589 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:25.589 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:25.589 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:25.589 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4397.0:130 3.4 e27/23 3:27567f25:::rbd_object_map.112d7adb82ff:head v 27'21, mlcod=27'19)", 2024-09-14T06:29:25.589 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:04.405039+0000", 2024-09-14T06:29:25.589 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 21.151702331999999, 2024-09-14T06:29:25.589 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000900945, 2024-09-14T06:29:25.589 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:29:25.589 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:29:25.590 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:29:25.590 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:29:25.590 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:25.590 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:29:25.590 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:04.405039+0000", 2024-09-14T06:29:25.590 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:25.590 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:25.590 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:25.590 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:29:25.590 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:04.405039+0000", 2024-09-14T06:29:25.591 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:25.591 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:25.591 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:25.591 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:29:25.591 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:04.405041+0000", 2024-09-14T06:29:25.591 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.516e-06 2024-09-14T06:29:25.591 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:25.591 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:25.591 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:29:25.591 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:04.405054+0000", 2024-09-14T06:29:25.591 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.3383999999999999e-05 2024-09-14T06:29:25.592 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:25.592 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:25.592 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:29:25.592 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:04.405073+0000", 2024-09-14T06:29:25.592 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.8598e-05 2024-09-14T06:29:25.592 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:25.592 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:25.592 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:29:25.592 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:04.405082+0000", 2024-09-14T06:29:25.592 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.4390000000000006e-06 2024-09-14T06:29:25.592 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:25.592 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:25.592 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:29:25.592 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:04.405164+0000", 2024-09-14T06:29:25.593 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.1358000000000003e-05 2024-09-14T06:29:25.593 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:25.593 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:25.593 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:29:25.593 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:04.405192+0000", 2024-09-14T06:29:25.593 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.8648000000000001e-05 2024-09-14T06:29:25.593 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:25.593 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:25.593 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-14T06:29:25.593 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:04.405897+0000", 2024-09-14T06:29:25.593 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00070438900000000004 2024-09-14T06:29:25.594 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:25.594 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:25.594 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:29:25.594 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:04.405940+0000", 2024-09-14T06:29:25.594 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.3612999999999997e-05 2024-09-14T06:29:25.594 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:25.594 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:25.594 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:25.594 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:25.594 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:25.594 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4397.0:164 3.4 e27/23 3:27567f25:::rbd_object_map.112d7adb82ff:head v 27'27, mlcod=27'25)", 2024-09-14T06:29:25.594 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:04.438987+0000", 2024-09-14T06:29:25.594 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 21.117754438999999, 2024-09-14T06:29:25.594 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00093266400000000004, 2024-09-14T06:29:25.595 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:29:25.595 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:29:25.595 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:29:25.595 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:29:25.595 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:25.595 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:29:25.595 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:04.438987+0000", 2024-09-14T06:29:25.595 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:25.595 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:25.595 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:25.595 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:29:25.595 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:04.438987+0000", 2024-09-14T06:29:25.595 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:25.595 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:25.596 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:25.596 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:29:25.596 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:04.438989+0000", 2024-09-14T06:29:25.596 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.1829999999999999e-06 2024-09-14T06:29:25.596 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:25.596 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:25.596 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:29:25.596 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:04.438998+0000", 2024-09-14T06:29:25.596 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.5489999999999995e-06 2024-09-14T06:29:25.596 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:25.596 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:25.596 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:29:25.596 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:04.439014+0000", 2024-09-14T06:29:25.597 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.5659e-05 2024-09-14T06:29:25.597 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:25.597 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:25.597 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:29:25.597 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:04.439024+0000", 2024-09-14T06:29:25.597 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.0229e-05 2024-09-14T06:29:25.597 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:25.597 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:25.597 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:29:25.597 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:04.439085+0000", 2024-09-14T06:29:25.597 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.1507000000000006e-05 2024-09-14T06:29:25.597 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:25.597 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:25.597 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:29:25.598 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:04.439111+0000", 2024-09-14T06:29:25.598 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.5479000000000001e-05 2024-09-14T06:29:25.598 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:25.598 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:25.598 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-14T06:29:25.598 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:04.439872+0000", 2024-09-14T06:29:25.598 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00076089899999999999 2024-09-14T06:29:25.598 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:25.598 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:25.598 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:29:25.598 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:04.439920+0000", 2024-09-14T06:29:25.598 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.8158999999999997e-05 2024-09-14T06:29:25.598 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:25.598 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:25.599 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:25.599 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:25.599 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:25.599 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4397.0:186 3.4 e27/23 3:27567f25:::rbd_object_map.112d7adb82ff:head v 27'31, mlcod=27'29)", 2024-09-14T06:29:25.599 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:04.457782+0000", 2024-09-14T06:29:25.599 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 21.098959304000001, 2024-09-14T06:29:25.599 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00083834100000000004, 2024-09-14T06:29:25.599 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:29:25.599 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:29:25.599 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:29:25.599 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:29:25.599 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:25.599 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:29:25.600 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:04.457782+0000", 2024-09-14T06:29:25.600 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:25.600 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:25.600 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:25.600 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:29:25.600 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:04.457782+0000", 2024-09-14T06:29:25.600 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:25.600 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:25.600 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:25.600 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:29:25.600 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:04.457783+0000", 2024-09-14T06:29:25.600 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.001e-06 2024-09-14T06:29:25.600 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:25.601 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:25.601 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:29:25.601 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:04.457801+0000", 2024-09-14T06:29:25.601 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.7119999999999999e-05 2024-09-14T06:29:25.601 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:25.601 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:25.601 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:29:25.601 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:04.457817+0000", 2024-09-14T06:29:25.601 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.6761000000000001e-05 2024-09-14T06:29:25.601 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:25.601 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:25.602 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:29:25.602 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:04.457826+0000", 2024-09-14T06:29:25.602 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.3059999999999998e-06 2024-09-14T06:29:25.602 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:25.602 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:25.602 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:29:25.602 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:04.457883+0000", 2024-09-14T06:29:25.602 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.7420000000000003e-05 2024-09-14T06:29:25.602 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:25.602 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:25.602 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:29:25.603 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:04.457906+0000", 2024-09-14T06:29:25.603 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.2643e-05 2024-09-14T06:29:25.603 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:25.603 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:25.603 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-14T06:29:25.603 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:04.458570+0000", 2024-09-14T06:29:25.603 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00066450099999999996 2024-09-14T06:29:25.603 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:25.603 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:25.603 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:29:25.603 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:04.458621+0000", 2024-09-14T06:29:25.603 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.0589e-05 2024-09-14T06:29:25.603 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:25.603 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:25.604 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:25.604 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:25.604 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:25.604 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4397.0:616 3.9 e30/23 3:908d0935:::rbd_data.112d7dfbd766.000000000000007e:head v 30'2, mlcod=0'0)", 2024-09-14T06:29:25.604 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:06.780526+0000", 2024-09-14T06:29:25.604 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 18.776215529000002, 2024-09-14T06:29:25.604 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00091880900000000003, 2024-09-14T06:29:25.604 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:29:25.604 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:29:25.604 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:29:25.604 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:29:25.604 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:25.604 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:29:25.605 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:06.780526+0000", 2024-09-14T06:29:25.605 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:25.605 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:25.605 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:25.605 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:29:25.605 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:06.780526+0000", 2024-09-14T06:29:25.605 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:25.605 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:25.605 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:25.605 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:29:25.605 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:06.780527+0000", 2024-09-14T06:29:25.605 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.3799999999999998e-07 2024-09-14T06:29:25.606 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:25.606 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:25.606 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:29:25.606 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:06.780625+0000", 2024-09-14T06:29:25.606 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.8785999999999999e-05 2024-09-14T06:29:25.606 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:25.606 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:25.606 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:29:25.606 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:06.780640+0000", 2024-09-14T06:29:25.606 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.4363e-05 2024-09-14T06:29:25.606 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:25.606 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:25.606 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:29:25.607 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:06.780650+0000", 2024-09-14T06:29:25.607 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.0164e-05 2024-09-14T06:29:25.607 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:25.607 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:25.607 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:29:25.607 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:06.780702+0000", 2024-09-14T06:29:25.607 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.1992e-05 2024-09-14T06:29:25.607 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:25.607 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:25.607 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:29:25.607 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:06.780718+0000", 2024-09-14T06:29:25.607 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.6161e-05 2024-09-14T06:29:25.607 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:25.607 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:25.607 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-14T06:29:25.608 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:06.781404+0000", 2024-09-14T06:29:25.608 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000685526 2024-09-14T06:29:25.608 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:25.608 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:25.608 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:29:25.608 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:06.781445+0000", 2024-09-14T06:29:25.608 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.1479e-05 2024-09-14T06:29:25.608 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:25.608 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:25.608 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:25.608 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:25.608 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:25.608 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4397.0:792 3.9 e30/23 3:977d4c6a:::rbd_data.112d7dfbd766.000000000000014d:head v 30'4, mlcod=30'2)", 2024-09-14T06:29:25.609 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:06.998893+0000", 2024-09-14T06:29:25.609 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 18.557848840999998, 2024-09-14T06:29:25.609 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00089658700000000001, 2024-09-14T06:29:25.609 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:29:25.609 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:29:25.609 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:29:25.609 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:29:25.609 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:25.609 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:29:25.609 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:06.998893+0000", 2024-09-14T06:29:25.609 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:25.609 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:25.609 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:25.609 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:29:25.610 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:06.998893+0000", 2024-09-14T06:29:25.610 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:25.610 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:25.610 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:25.610 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:29:25.610 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:06.998894+0000", 2024-09-14T06:29:25.610 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.2550000000000001e-06 2024-09-14T06:29:25.610 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:25.610 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:25.610 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:29:25.610 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:06.998912+0000", 2024-09-14T06:29:25.610 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.7818000000000001e-05 2024-09-14T06:29:25.610 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:25.610 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:25.611 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:29:25.611 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:06.998926+0000", 2024-09-14T06:29:25.611 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.4368e-05 2024-09-14T06:29:25.611 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:25.611 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:25.611 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:29:25.611 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:06.998934+0000", 2024-09-14T06:29:25.611 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.6839999999999996e-06 2024-09-14T06:29:25.611 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:25.611 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:25.611 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:29:25.611 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:06.998981+0000", 2024-09-14T06:29:25.611 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.6995e-05 2024-09-14T06:29:25.611 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:25.612 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:25.612 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:29:25.612 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:06.998998+0000", 2024-09-14T06:29:25.612 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.6628000000000001e-05 2024-09-14T06:29:25.612 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:25.612 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:25.612 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-14T06:29:25.612 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:06.999757+0000", 2024-09-14T06:29:25.612 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00075925099999999996 2024-09-14T06:29:25.612 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:25.612 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:25.612 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:29:25.613 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:06.999790+0000", 2024-09-14T06:29:25.613 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.2588e-05 2024-09-14T06:29:25.613 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:25.613 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:25.613 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:25.613 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:25.613 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:25.613 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4397.0:937 3.9 e30/23 3:92b37d3e:::rbd_data.112d7dfbd766.0000000000000012:head v 30'7, mlcod=30'5)", 2024-09-14T06:29:25.613 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:07.169523+0000", 2024-09-14T06:29:25.613 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 18.387218973, 2024-09-14T06:29:25.613 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0013381339999999999, 2024-09-14T06:29:25.614 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:29:25.614 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:29:25.614 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:29:25.614 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:29:25.614 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:25.614 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:29:25.614 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:07.169523+0000", 2024-09-14T06:29:25.614 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:25.614 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:25.614 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:25.614 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:29:25.614 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:07.169523+0000", 2024-09-14T06:29:25.614 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:25.614 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:25.615 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:25.615 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:29:25.615 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:07.169524+0000", 2024-09-14T06:29:25.615 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.125e-06 2024-09-14T06:29:25.615 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:25.615 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:25.615 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:29:25.615 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:07.170106+0000", 2024-09-14T06:29:25.615 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00058239900000000005 2024-09-14T06:29:25.615 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:25.615 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:25.616 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:29:25.616 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:07.170121+0000", 2024-09-14T06:29:25.616 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.4851999999999999e-05 2024-09-14T06:29:25.616 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:25.616 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:25.616 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:29:25.616 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:07.170130+0000", 2024-09-14T06:29:25.616 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.9199999999999993e-06 2024-09-14T06:29:25.616 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:25.616 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:25.616 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:29:25.616 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:07.170182+0000", 2024-09-14T06:29:25.616 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.1669999999999998e-05 2024-09-14T06:29:25.616 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:25.617 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:25.617 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:29:25.617 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:07.170199+0000", 2024-09-14T06:29:25.617 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.7193999999999998e-05 2024-09-14T06:29:25.617 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:25.617 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:25.617 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-14T06:29:25.617 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:07.170816+0000", 2024-09-14T06:29:25.617 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00061745400000000003 2024-09-14T06:29:25.617 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:25.617 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:25.617 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:29:25.617 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:07.170861+0000", 2024-09-14T06:29:25.618 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.4520000000000001e-05 2024-09-14T06:29:25.618 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:25.618 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:25.618 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:25.618 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:25.618 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:25.618 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4397.0:1634 3.0 e32/23 3:01b99234:::rbd_data.112dcdddac74.0000000000000009:head v 32'2, mlcod=0'0)", 2024-09-14T06:29:25.618 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:08.644163+0000", 2024-09-14T06:29:25.618 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 16.912579173000001, 2024-09-14T06:29:25.618 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0010247730000000001, 2024-09-14T06:29:25.618 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:29:25.618 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:29:25.618 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:29:25.618 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:29:25.619 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:25.619 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:29:25.619 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.644163+0000", 2024-09-14T06:29:25.619 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:25.619 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:25.619 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:25.619 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:29:25.619 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.644163+0000", 2024-09-14T06:29:25.619 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:25.619 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:25.619 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:25.619 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:29:25.620 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.644163+0000", 2024-09-14T06:29:25.620 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.7800000000000002e-07 2024-09-14T06:29:25.620 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:25.620 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:25.620 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:29:25.620 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.644210+0000", 2024-09-14T06:29:25.620 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.7240000000000002e-05 2024-09-14T06:29:25.620 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:25.620 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:25.620 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:29:25.620 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.644245+0000", 2024-09-14T06:29:25.621 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.4288000000000001e-05 2024-09-14T06:29:25.621 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:25.621 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:25.621 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:29:25.621 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.644254+0000", 2024-09-14T06:29:25.621 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.3719999999999994e-06 2024-09-14T06:29:25.621 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:25.621 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:25.621 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:29:25.621 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.644338+0000", 2024-09-14T06:29:25.621 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.3892000000000002e-05 2024-09-14T06:29:25.622 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:25.622 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:25.622 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:29:25.622 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.644364+0000", 2024-09-14T06:29:25.622 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.5805000000000001e-05 2024-09-14T06:29:25.622 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:25.622 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:25.622 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-14T06:29:25.622 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.645151+0000", 2024-09-14T06:29:25.622 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00078698199999999998 2024-09-14T06:29:25.622 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:25.622 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:25.622 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:29:25.622 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.645187+0000", 2024-09-14T06:29:25.623 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.6816000000000001e-05 2024-09-14T06:29:25.623 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:25.623 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:25.623 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:25.623 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:25.623 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:25.623 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4397.0:1848 3.4 e32/23 3:27620a21:::rbd_data.112dcdddac74.000000000000000b:head v 32'40, mlcod=30'38)", 2024-09-14T06:29:25.623 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:08.658649+0000", 2024-09-14T06:29:25.623 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 16.898092599999998, 2024-09-14T06:29:25.623 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.001862867, 2024-09-14T06:29:25.623 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:29:25.623 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:29:25.623 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:29:25.623 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:29:25.623 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:25.624 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:29:25.624 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.658649+0000", 2024-09-14T06:29:25.624 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:25.624 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:25.624 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:25.624 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:29:25.624 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.658649+0000", 2024-09-14T06:29:25.624 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:25.624 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:25.624 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:25.624 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:29:25.624 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.658651+0000", 2024-09-14T06:29:25.624 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.5650000000000001e-06 2024-09-14T06:29:25.624 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:25.625 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:25.625 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:29:25.625 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.658709+0000", 2024-09-14T06:29:25.625 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.8442000000000001e-05 2024-09-14T06:29:25.625 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:25.625 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:25.625 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:29:25.625 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.658730+0000", 2024-09-14T06:29:25.625 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.0509999999999998e-05 2024-09-14T06:29:25.625 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:25.625 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:25.625 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:29:25.625 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.658740+0000", 2024-09-14T06:29:25.626 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.8619999999999998e-06 2024-09-14T06:29:25.626 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:25.626 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:25.626 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:29:25.626 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.658813+0000", 2024-09-14T06:29:25.626 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.3578000000000006e-05 2024-09-14T06:29:25.626 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:25.626 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:25.626 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:29:25.626 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.658836+0000", 2024-09-14T06:29:25.626 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.2677e-05 2024-09-14T06:29:25.626 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:25.626 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:25.627 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-14T06:29:25.627 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.660462+0000", 2024-09-14T06:29:25.627 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00162637 2024-09-14T06:29:25.627 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:25.627 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:25.627 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:29:25.627 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.660512+0000", 2024-09-14T06:29:25.627 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.9863e-05 2024-09-14T06:29:25.627 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:25.627 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:25.627 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:25.627 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:25.628 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:25.628 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4397.0:1941 3.9 e32/23 3:92dc473f:::rbd_data.112dcdddac74.000000000000000c:head v 32'8, mlcod=30'6)", 2024-09-14T06:29:25.628 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:08.664245+0000", 2024-09-14T06:29:25.628 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 16.892497284000001, 2024-09-14T06:29:25.628 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.001294746, 2024-09-14T06:29:25.628 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:29:25.628 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:29:25.628 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:29:25.628 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:29:25.628 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:25.628 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:29:25.628 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.664245+0000", 2024-09-14T06:29:25.628 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:25.629 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:25.629 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:25.629 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:29:25.629 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.664245+0000", 2024-09-14T06:29:25.629 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:25.629 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:25.629 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:25.629 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:29:25.629 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.664245+0000", 2024-09-14T06:29:25.629 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.0999999999999999e-07 2024-09-14T06:29:25.629 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:25.629 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:25.629 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:29:25.630 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.664408+0000", 2024-09-14T06:29:25.630 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00016308800000000001 2024-09-14T06:29:25.630 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:25.630 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:25.630 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:29:25.630 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.664421+0000", 2024-09-14T06:29:25.630 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.3097e-05 2024-09-14T06:29:25.630 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:25.630 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:25.630 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:29:25.630 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.664429+0000", 2024-09-14T06:29:25.630 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.943e-06 2024-09-14T06:29:25.630 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:25.630 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:25.631 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:29:25.631 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.664511+0000", 2024-09-14T06:29:25.631 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.1945000000000002e-05 2024-09-14T06:29:25.631 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:25.631 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:25.631 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:29:25.631 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.664532+0000", 2024-09-14T06:29:25.631 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.0931999999999999e-05 2024-09-14T06:29:25.631 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:25.631 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:25.631 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-14T06:29:25.631 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.665490+0000", 2024-09-14T06:29:25.631 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00095764400000000001 2024-09-14T06:29:25.631 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:25.632 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:25.632 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:29:25.632 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.665539+0000", 2024-09-14T06:29:25.632 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.9687e-05 2024-09-14T06:29:25.632 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:25.632 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:25.632 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:25.632 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:25.632 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:25.632 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4397.0:2199 3.0 e32/23 3:05f3e980:::rbd_data.112dcdddac74.000000000000000f:head v 32'3, mlcod=31'1)", 2024-09-14T06:29:25.632 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:08.680236+0000", 2024-09-14T06:29:25.632 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 16.876506235000001, 2024-09-14T06:29:25.633 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0019068990000000001, 2024-09-14T06:29:25.633 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:29:25.633 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:29:25.633 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:29:25.633 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:29:25.633 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:25.633 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:29:25.633 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.680236+0000", 2024-09-14T06:29:25.633 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:25.633 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:25.633 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:25.633 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:29:25.633 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.680236+0000", 2024-09-14T06:29:25.633 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:25.634 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:25.634 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:25.634 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:29:25.634 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.680237+0000", 2024-09-14T06:29:25.634 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.4980000000000001e-06 2024-09-14T06:29:25.634 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:25.634 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:25.634 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:29:25.634 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.680324+0000", 2024-09-14T06:29:25.634 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.7310000000000003e-05 2024-09-14T06:29:25.634 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:25.634 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:25.635 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:29:25.635 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.680354+0000", 2024-09-14T06:29:25.635 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.97e-05 2024-09-14T06:29:25.635 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:25.635 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:25.635 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:29:25.635 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.680366+0000", 2024-09-14T06:29:25.635 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.2048999999999999e-05 2024-09-14T06:29:25.635 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:25.635 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:25.635 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:29:25.635 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.680439+0000", 2024-09-14T06:29:25.635 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.2910000000000005e-05 2024-09-14T06:29:25.635 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:25.636 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:25.636 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:29:25.636 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.680465+0000", 2024-09-14T06:29:25.636 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.5812999999999998e-05 2024-09-14T06:29:25.636 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:25.636 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:25.636 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-14T06:29:25.636 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.682094+0000", 2024-09-14T06:29:25.636 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.001628876 2024-09-14T06:29:25.636 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:25.636 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:25.637 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:29:25.637 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.682142+0000", 2024-09-14T06:29:25.637 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.8742999999999997e-05 2024-09-14T06:29:25.637 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:25.637 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:25.637 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:25.637 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:25.637 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:25.637 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4397.0:2513 3.0 e32/23 3:06f82ba7:::rbd_data.112dcdddac74.0000000000000028:head v 32'4, mlcod=32'2)", 2024-09-14T06:29:25.637 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:08.703547+0000", 2024-09-14T06:29:25.637 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 16.853194631000001, 2024-09-14T06:29:25.637 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0013041859999999999, 2024-09-14T06:29:25.638 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:29:25.638 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:29:25.638 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:29:25.638 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:29:25.638 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:25.638 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:29:25.638 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.703547+0000", 2024-09-14T06:29:25.638 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:25.638 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:25.638 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:25.638 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:29:25.638 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.703547+0000", 2024-09-14T06:29:25.638 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:25.638 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:25.639 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:25.639 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:29:25.639 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.703549+0000", 2024-09-14T06:29:25.639 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.39e-06 2024-09-14T06:29:25.639 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:25.639 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:25.639 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:29:25.639 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.703666+0000", 2024-09-14T06:29:25.639 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000117314 2024-09-14T06:29:25.639 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:25.639 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:25.639 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:29:25.639 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.703685+0000", 2024-09-14T06:29:25.639 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.8984999999999999e-05 2024-09-14T06:29:25.640 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:25.640 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:25.640 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:29:25.640 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.703697+0000", 2024-09-14T06:29:25.640 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.2405000000000001e-05 2024-09-14T06:29:25.640 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:25.640 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:25.640 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:29:25.640 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.703765+0000", 2024-09-14T06:29:25.640 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.7595000000000006e-05 2024-09-14T06:29:25.640 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:25.640 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:25.640 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:29:25.641 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.703788+0000", 2024-09-14T06:29:25.641 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.3300000000000001e-05 2024-09-14T06:29:25.641 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:25.644 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:25.644 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-14T06:29:25.644 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.704801+0000", 2024-09-14T06:29:25.644 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0010130339999999999 2024-09-14T06:29:25.644 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:25.644 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:25.645 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:29:25.645 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.704851+0000", 2024-09-14T06:29:25.645 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.0163000000000001e-05 2024-09-14T06:29:25.645 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:25.645 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:25.645 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:25.645 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:25.645 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:25.645 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4397.0:2885 3.9 e32/23 3:974aae1b:::rbd_data.112dcdddac74.000000000000002d:head v 32'9, mlcod=30'7)", 2024-09-14T06:29:25.645 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:08.717100+0000", 2024-09-14T06:29:25.646 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 16.839642016999999, 2024-09-14T06:29:25.646 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0012528229999999999, 2024-09-14T06:29:25.646 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:29:25.646 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:29:25.646 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:29:25.646 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:29:25.646 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:25.646 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:29:25.647 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.717100+0000", 2024-09-14T06:29:25.647 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:25.647 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:25.647 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:25.647 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:29:25.647 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.717100+0000", 2024-09-14T06:29:25.647 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:25.647 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:25.647 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:25.647 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:29:25.647 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.717101+0000", 2024-09-14T06:29:25.647 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.2929999999999999e-06 2024-09-14T06:29:25.648 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:25.648 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:25.648 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:29:25.648 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.717270+0000", 2024-09-14T06:29:25.648 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00016929799999999999 2024-09-14T06:29:25.648 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:25.648 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:25.648 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:29:25.648 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.717283+0000", 2024-09-14T06:29:25.648 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.2483999999999999e-05 2024-09-14T06:29:25.648 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:25.648 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:25.649 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:29:25.649 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.717292+0000", 2024-09-14T06:29:25.649 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.7220000000000005e-06 2024-09-14T06:29:25.649 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:25.649 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:25.649 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:29:25.649 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.717356+0000", 2024-09-14T06:29:25.649 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.4055999999999998e-05 2024-09-14T06:29:25.649 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:25.649 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:25.650 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:29:25.650 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.717383+0000", 2024-09-14T06:29:25.650 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.7274999999999999e-05 2024-09-14T06:29:25.650 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:25.650 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:25.650 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-14T06:29:25.650 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.718318+0000", 2024-09-14T06:29:25.650 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00093470100000000002 2024-09-14T06:29:25.650 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:25.650 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:25.651 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:29:25.651 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.718353+0000", 2024-09-14T06:29:25.651 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.4993999999999997e-05 2024-09-14T06:29:25.651 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:25.651 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:25.651 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:25.651 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:25.651 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:25.651 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4397.0:3292 3.9 e32/23 3:91dbce57:::rbd_data.112dcdddac74.0000000000000036:head v 32'10, mlcod=32'8)", 2024-09-14T06:29:25.651 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:08.740506+0000", 2024-09-14T06:29:25.651 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 16.816235523, 2024-09-14T06:29:25.651 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.001864166, 2024-09-14T06:29:25.651 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:29:25.652 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:29:25.652 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:29:25.652 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:29:25.652 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:25.652 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:29:25.652 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.740506+0000", 2024-09-14T06:29:25.652 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:25.652 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:25.652 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:25.652 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:29:25.652 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.740506+0000", 2024-09-14T06:29:25.652 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:25.652 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:25.652 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:25.653 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:29:25.653 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.740514+0000", 2024-09-14T06:29:25.653 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.2109999999999998e-06 2024-09-14T06:29:25.653 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:25.653 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:25.653 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:29:25.653 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.740745+0000", 2024-09-14T06:29:25.653 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000230194 2024-09-14T06:29:25.653 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:25.653 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:25.653 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:29:25.653 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.740764+0000", 2024-09-14T06:29:25.653 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.9330999999999998e-05 2024-09-14T06:29:25.653 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:25.653 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:25.654 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:29:25.654 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.740774+0000", 2024-09-14T06:29:25.654 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.6260000000000007e-06 2024-09-14T06:29:25.654 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:25.654 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:25.654 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:29:25.654 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.740839+0000", 2024-09-14T06:29:25.654 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.5314000000000005e-05 2024-09-14T06:29:25.654 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:25.654 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:25.654 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:29:25.654 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.740860+0000", 2024-09-14T06:29:25.654 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.0868000000000001e-05 2024-09-14T06:29:25.654 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:25.655 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:25.655 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-14T06:29:25.655 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.742327+0000", 2024-09-14T06:29:25.655 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0014672909999999999 2024-09-14T06:29:25.655 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:25.655 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:25.655 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:29:25.655 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.742370+0000", 2024-09-14T06:29:25.655 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.3331000000000002e-05 2024-09-14T06:29:25.655 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:25.655 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:25.655 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:25.655 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:25.656 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:25.656 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4397.0:8527 3.0 e46/23 3:05c39535:::rbd_data.112d92027590.0000000000000009:head v 46'11, mlcod=33'9)", 2024-09-14T06:29:25.656 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:22.617134+0000", 2024-09-14T06:29:25.656 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 2.939607804, 2024-09-14T06:29:25.656 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0010939770000000001, 2024-09-14T06:29:25.656 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:29:25.656 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:29:25.656 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:29:25.656 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:29:25.656 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:25.656 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:29:25.656 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:22.617134+0000", 2024-09-14T06:29:25.656 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:25.656 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:25.657 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:25.657 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:29:25.657 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:22.617134+0000", 2024-09-14T06:29:25.657 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:25.657 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:25.657 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:25.657 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:29:25.657 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:22.617134+0000", 2024-09-14T06:29:25.657 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.89e-07 2024-09-14T06:29:25.657 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:25.657 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:25.657 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:29:25.657 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:22.617189+0000", 2024-09-14T06:29:25.658 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.4415e-05 2024-09-14T06:29:25.658 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:25.658 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:25.658 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:29:25.658 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:22.617197+0000", 2024-09-14T06:29:25.658 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.5750000000000001e-06 2024-09-14T06:29:25.658 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:25.658 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:25.658 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:29:25.658 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:22.617205+0000", 2024-09-14T06:29:25.658 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.6029999999999999e-06 2024-09-14T06:29:25.658 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:25.658 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:25.659 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:29:25.659 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:22.617268+0000", 2024-09-14T06:29:25.659 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.2568000000000001e-05 2024-09-14T06:29:25.659 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:25.659 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:25.659 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:29:25.659 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:22.617303+0000", 2024-09-14T06:29:25.659 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.5259000000000002e-05 2024-09-14T06:29:25.659 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:25.659 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:25.659 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-14T06:29:25.659 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:22.618188+0000", 2024-09-14T06:29:25.659 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00088557900000000005 2024-09-14T06:29:25.659 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:25.659 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:25.660 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:29:25.660 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:22.618228+0000", 2024-09-14T06:29:25.660 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.9489000000000001e-05 2024-09-14T06:29:25.660 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:25.660 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:25.660 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:25.660 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:25.660 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:25.660 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-14T06:29:25.662 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-14T06:29:25.662 INFO:tasks.ceph.osd.3.smithi094.stderr:2024-09-14T06:29:25.660+0000 7feeeafe8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33655) UID: 0 2024-09-14T06:29:25.762 INFO:tasks.ceph.osd.0.smithi094.stderr:2024-09-14T06:29:25.761+0000 7f07c9333640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33636) UID: 0 2024-09-14T06:29:25.847 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-14T06:29:25.847 INFO:teuthology.orchestra.run.smithi094.stdout: "ops": [ 2024-09-14T06:29:25.847 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:25.847 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:12566 3.1e 3.302ef9fe (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:29:25.847 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:25.836288+0000", 2024-09-14T06:29:25.847 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 0.0017516840000000001, 2024-09-14T06:29:25.847 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0017658319999999999, 2024-09-14T06:29:25.848 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:29:25.848 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:29:25.848 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "queued for pg", 2024-09-14T06:29:25.848 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:29:25.848 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:29:25.848 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:25.849 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 12566 2024-09-14T06:29:25.849 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:25.849 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:29:25.849 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:25.849 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:29:25.849 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.836288+0000", 2024-09-14T06:29:25.849 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:25.849 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:25.849 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:25.849 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:29:25.849 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.836288+0000", 2024-09-14T06:29:25.849 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:25.849 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:25.850 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:25.850 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:29:25.850 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.836289+0000", 2024-09-14T06:29:25.850 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.5699999999999998e-07 2024-09-14T06:29:25.850 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:25.850 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:25.850 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:29:25.850 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.836290+0000", 2024-09-14T06:29:25.850 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.5009999999999999e-06 2024-09-14T06:29:25.850 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:25.850 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:25.850 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:29:25.850 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.836294+0000", 2024-09-14T06:29:25.850 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.0180000000000003e-06 2024-09-14T06:29:25.851 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:25.851 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:25.851 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:29:25.851 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.836297+0000", 2024-09-14T06:29:25.851 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.9129999999999999e-06 2024-09-14T06:29:25.851 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:25.851 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:25.851 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:25.852 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:25.852 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:25.852 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:12598 3.1e 3.302ef9fe (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:29:25.852 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:25.836791+0000", 2024-09-14T06:29:25.852 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 0.001248381, 2024-09-14T06:29:25.852 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0012936079999999999, 2024-09-14T06:29:25.852 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:29:25.852 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:29:25.852 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "queued for pg", 2024-09-14T06:29:25.852 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:29:25.852 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:29:25.852 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:25.853 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 12598 2024-09-14T06:29:25.853 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:25.853 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:29:25.853 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:25.853 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:29:25.853 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.836791+0000", 2024-09-14T06:29:25.853 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:25.853 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:25.853 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:25.853 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:29:25.853 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.836791+0000", 2024-09-14T06:29:25.853 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:25.853 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:25.854 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:25.854 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:29:25.854 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.836792+0000", 2024-09-14T06:29:25.854 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.0200000000000001e-07 2024-09-14T06:29:25.854 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:25.854 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:25.854 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:29:25.854 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.836793+0000", 2024-09-14T06:29:25.854 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.0729999999999999e-06 2024-09-14T06:29:25.854 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:25.854 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:25.854 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:29:25.854 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.836796+0000", 2024-09-14T06:29:25.854 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.1149999999999998e-06 2024-09-14T06:29:25.855 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:25.855 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:25.855 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:29:25.855 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.836798+0000", 2024-09-14T06:29:25.855 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.576e-06 2024-09-14T06:29:25.855 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:25.855 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:25.855 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:25.855 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:25.855 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:25.855 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:12630 3.1e 3.302ef9fe (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:29:25.855 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:25.837369+0000", 2024-09-14T06:29:25.855 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 0.00067103200000000001, 2024-09-14T06:29:25.855 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00074072700000000005, 2024-09-14T06:29:25.856 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:29:25.856 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:29:25.856 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "queued for pg", 2024-09-14T06:29:25.856 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:29:25.856 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:29:25.856 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:25.856 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 12630 2024-09-14T06:29:25.856 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:25.856 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:29:25.856 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:25.857 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:29:25.857 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.837369+0000", 2024-09-14T06:29:25.857 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:25.857 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:25.857 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:25.857 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:29:25.857 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.837369+0000", 2024-09-14T06:29:25.857 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:25.857 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:25.857 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:25.857 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:29:25.857 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.837369+0000", 2024-09-14T06:29:25.857 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.1e-07 2024-09-14T06:29:25.857 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:25.858 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:25.858 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:29:25.858 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.837370+0000", 2024-09-14T06:29:25.858 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.004e-06 2024-09-14T06:29:25.858 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:25.858 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:25.858 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:29:25.858 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.837373+0000", 2024-09-14T06:29:25.858 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.977e-06 2024-09-14T06:29:25.858 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:25.858 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:25.858 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:29:25.858 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.837375+0000", 2024-09-14T06:29:25.858 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.39e-06 2024-09-14T06:29:25.859 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:25.859 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:25.859 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:25.859 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:25.859 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:25.859 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:12662 3.1e 3.302ef9fe (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:29:25.859 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:25.837938+0000", 2024-09-14T06:29:25.859 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 0.00010199700000000001, 2024-09-14T06:29:25.859 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00019494099999999999, 2024-09-14T06:29:25.859 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:29:25.859 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:29:25.859 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "queued for pg", 2024-09-14T06:29:25.859 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:29:25.859 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:29:25.860 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:25.860 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 12662 2024-09-14T06:29:25.860 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:25.860 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:29:25.860 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:25.860 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:29:25.860 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.837938+0000", 2024-09-14T06:29:25.860 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:25.860 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:25.860 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:25.860 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:29:25.861 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.837938+0000", 2024-09-14T06:29:25.861 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:25.861 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:25.861 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:25.861 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:29:25.861 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.837938+0000", 2024-09-14T06:29:25.861 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.9900000000000001e-07 2024-09-14T06:29:25.861 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:25.861 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:25.861 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:29:25.861 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.837941+0000", 2024-09-14T06:29:25.861 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.4499999999999998e-06 2024-09-14T06:29:25.862 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:25.862 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:25.862 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:29:25.862 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.837946+0000", 2024-09-14T06:29:25.862 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.7049999999999996e-06 2024-09-14T06:29:25.862 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:25.862 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:25.862 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:29:25.862 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.837952+0000", 2024-09-14T06:29:25.862 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.631e-06 2024-09-14T06:29:25.862 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:25.862 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:25.862 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:25.863 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:25.863 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:25.863 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:12567 3.1e 3.302ef9fe (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:29:25.863 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:25.836302+0000", 2024-09-14T06:29:25.863 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 0.001738135, 2024-09-14T06:29:25.863 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.001854692, 2024-09-14T06:29:25.863 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:29:25.863 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:29:25.863 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "queued for pg", 2024-09-14T06:29:25.863 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:29:25.863 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:29:25.863 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:25.863 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 12567 2024-09-14T06:29:25.864 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:25.864 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:29:25.864 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:25.864 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:29:25.864 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.836302+0000", 2024-09-14T06:29:25.864 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:25.864 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:25.864 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:25.864 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:29:25.864 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.836302+0000", 2024-09-14T06:29:25.864 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:25.864 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:25.864 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:25.865 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:29:25.865 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.836302+0000", 2024-09-14T06:29:25.865 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.6899999999999999e-07 2024-09-14T06:29:25.865 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:25.865 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:25.865 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:29:25.865 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.836303+0000", 2024-09-14T06:29:25.865 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.0139999999999999e-06 2024-09-14T06:29:25.865 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:25.865 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:25.865 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:29:25.865 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.836307+0000", 2024-09-14T06:29:25.865 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.6059999999999999e-06 2024-09-14T06:29:25.865 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:25.866 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:25.866 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:29:25.866 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.836310+0000", 2024-09-14T06:29:25.866 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.4929999999999998e-06 2024-09-14T06:29:25.866 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:25.866 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:25.866 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:25.866 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:25.866 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:25.866 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:12599 3.1e 3.302ef9fe (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:29:25.866 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:25.836802+0000", 2024-09-14T06:29:25.866 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 0.0012383229999999999, 2024-09-14T06:29:25.867 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0013805790000000001, 2024-09-14T06:29:25.867 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:29:25.867 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:29:25.867 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "queued for pg", 2024-09-14T06:29:25.867 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:29:25.867 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:29:25.867 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:25.867 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 12599 2024-09-14T06:29:25.867 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:25.867 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:29:25.867 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:25.867 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:29:25.867 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.836802+0000", 2024-09-14T06:29:25.868 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:25.868 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:25.868 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:25.868 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:29:25.868 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.836802+0000", 2024-09-14T06:29:25.868 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:25.868 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:25.868 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:25.868 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:29:25.868 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.836802+0000", 2024-09-14T06:29:25.869 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.0200000000000001e-07 2024-09-14T06:29:25.869 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:25.869 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:25.869 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:29:25.869 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.836803+0000", 2024-09-14T06:29:25.869 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.85e-07 2024-09-14T06:29:25.869 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:25.869 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:25.869 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:29:25.869 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.836810+0000", 2024-09-14T06:29:25.869 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.4889999999999998e-06 2024-09-14T06:29:25.869 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:25.869 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:25.870 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:29:25.870 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.836813+0000", 2024-09-14T06:29:25.870 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.6869999999999999e-06 2024-09-14T06:29:25.870 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:25.870 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:25.870 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:25.870 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:25.870 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:25.870 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:12631 3.1e 3.302ef9fe (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:29:25.870 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:25.837379+0000", 2024-09-14T06:29:25.870 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 0.00066131199999999995, 2024-09-14T06:29:25.870 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00082787599999999996, 2024-09-14T06:29:25.870 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:29:25.870 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:29:25.871 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "queued for pg", 2024-09-14T06:29:25.871 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:29:25.871 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:29:25.871 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:25.871 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 12631 2024-09-14T06:29:25.871 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:25.871 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:29:25.871 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:25.871 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:29:25.871 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.837379+0000", 2024-09-14T06:29:25.871 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:25.871 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:25.871 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:25.871 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:29:25.871 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.837379+0000", 2024-09-14T06:29:25.872 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:25.872 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:25.872 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:25.872 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:29:25.872 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.837379+0000", 2024-09-14T06:29:25.872 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.0200000000000001e-07 2024-09-14T06:29:25.872 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:25.872 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:25.872 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:29:25.872 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.837380+0000", 2024-09-14T06:29:25.872 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.4449999999999999e-06 2024-09-14T06:29:25.873 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:25.873 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:25.873 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:29:25.873 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.837384+0000", 2024-09-14T06:29:25.873 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.8160000000000004e-06 2024-09-14T06:29:25.873 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:25.873 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:25.873 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:29:25.873 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.837386+0000", 2024-09-14T06:29:25.873 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.5179999999999999e-06 2024-09-14T06:29:25.873 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:25.873 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:25.873 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:25.874 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:25.874 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:25.874 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:12663 3.1e 3.302ef9fe (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:29:25.874 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:25.837958+0000", 2024-09-14T06:29:25.874 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 8.1726999999999999e-05, 2024-09-14T06:29:25.874 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00027208000000000001, 2024-09-14T06:29:25.874 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:29:25.874 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:29:25.874 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "queued for pg", 2024-09-14T06:29:25.874 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:29:25.875 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:29:25.875 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:25.875 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 12663 2024-09-14T06:29:25.875 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:25.875 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:29:25.875 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:25.875 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:29:25.875 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.837958+0000", 2024-09-14T06:29:25.875 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:25.876 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:25.876 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:25.876 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:29:25.876 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.837958+0000", 2024-09-14T06:29:25.876 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:25.876 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:25.876 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:25.876 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:29:25.877 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.837958+0000", 2024-09-14T06:29:25.877 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.7899999999999999e-07 2024-09-14T06:29:25.877 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:25.877 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:25.877 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:29:25.877 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.837960+0000", 2024-09-14T06:29:25.877 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.767e-06 2024-09-14T06:29:25.877 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:25.877 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:25.878 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:29:25.878 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.837967+0000", 2024-09-14T06:29:25.878 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.4999999999999996e-06 2024-09-14T06:29:25.878 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:25.878 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:25.878 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:29:25.878 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.837973+0000", 2024-09-14T06:29:25.878 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.8930000000000002e-06 2024-09-14T06:29:25.879 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:25.879 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:25.879 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:25.879 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:25.879 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:25.879 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:12568 3.1e 3.302ef9fe (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:29:25.879 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:25.836314+0000", 2024-09-14T06:29:25.879 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 0.0017260960000000001, 2024-09-14T06:29:25.880 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0019398340000000001, 2024-09-14T06:29:25.880 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:29:25.880 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:29:25.880 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "queued for pg", 2024-09-14T06:29:25.880 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:29:25.880 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:29:25.880 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:25.881 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 12568 2024-09-14T06:29:25.881 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:25.881 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:29:25.881 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:25.881 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:29:25.881 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.836314+0000", 2024-09-14T06:29:25.881 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:25.881 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:25.881 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:25.882 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:29:25.882 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.836314+0000", 2024-09-14T06:29:25.882 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:25.882 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:25.882 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:25.882 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:29:25.882 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.836314+0000", 2024-09-14T06:29:25.883 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.39e-07 2024-09-14T06:29:25.883 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:25.883 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:25.883 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:29:25.883 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.836315+0000", 2024-09-14T06:29:25.883 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.3799999999999996e-07 2024-09-14T06:29:25.883 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:25.883 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:25.884 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:29:25.884 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.836319+0000", 2024-09-14T06:29:25.884 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.5030000000000002e-06 2024-09-14T06:29:25.884 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:25.884 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:25.884 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:29:25.884 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.836322+0000", 2024-09-14T06:29:25.884 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.0139999999999999e-06 2024-09-14T06:29:25.885 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:25.885 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:25.885 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:25.885 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:25.885 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:25.885 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:12600 3.1e 3.302ef9fe (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:29:25.885 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:25.836816+0000", 2024-09-14T06:29:25.885 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 0.0012234539999999999, 2024-09-14T06:29:25.886 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00146174, 2024-09-14T06:29:25.886 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:29:25.886 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:29:25.886 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "queued for pg", 2024-09-14T06:29:25.886 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:29:25.886 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:29:25.886 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:25.886 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 12600 2024-09-14T06:29:25.886 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:25.887 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:29:25.887 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:25.887 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:29:25.887 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.836816+0000", 2024-09-14T06:29:25.887 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:25.887 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:25.887 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:25.887 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:29:25.887 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.836816+0000", 2024-09-14T06:29:25.887 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:25.887 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:25.887 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:25.887 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:29:25.888 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.836817+0000", 2024-09-14T06:29:25.888 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.23e-07 2024-09-14T06:29:25.888 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:25.888 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:25.888 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:29:25.888 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.836818+0000", 2024-09-14T06:29:25.888 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.017e-06 2024-09-14T06:29:25.888 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:25.888 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:25.888 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:29:25.888 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.836822+0000", 2024-09-14T06:29:25.888 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.0720000000000001e-06 2024-09-14T06:29:25.888 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:25.889 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:25.889 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:29:25.889 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.836824+0000", 2024-09-14T06:29:25.889 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.6010000000000002e-06 2024-09-14T06:29:25.889 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:25.889 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:25.889 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:25.889 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:25.889 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:25.889 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:12632 3.1e 3.302ef9fe (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:29:25.889 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:25.837390+0000", 2024-09-14T06:29:25.890 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 0.00065008799999999995, 2024-09-14T06:29:25.890 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000911136, 2024-09-14T06:29:25.890 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:29:25.890 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:29:25.890 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "queued for pg", 2024-09-14T06:29:25.890 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:29:25.890 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:29:25.890 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:25.890 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 12632 2024-09-14T06:29:25.890 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:25.890 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:29:25.890 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:25.890 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:29:25.891 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.837390+0000", 2024-09-14T06:29:25.891 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:25.891 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:25.891 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:25.891 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:29:25.891 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.837390+0000", 2024-09-14T06:29:25.891 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:25.891 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:25.891 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:25.892 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:29:25.892 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.837390+0000", 2024-09-14T06:29:25.892 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.05e-07 2024-09-14T06:29:25.892 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:25.892 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:25.892 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:29:25.892 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.837391+0000", 2024-09-14T06:29:25.892 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.2500000000000004e-07 2024-09-14T06:29:25.892 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:25.892 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:25.893 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:29:25.893 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.837396+0000", 2024-09-14T06:29:25.893 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.5289999999999999e-06 2024-09-14T06:29:25.893 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:25.893 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:25.893 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:29:25.893 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.837402+0000", 2024-09-14T06:29:25.893 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.9800000000000003e-06 2024-09-14T06:29:25.893 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:25.893 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:25.893 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:25.893 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:25.893 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:25.894 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:12664 3.1e 3.302ef9fe (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:29:25.894 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:25.837981+0000", 2024-09-14T06:29:25.894 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 5.8811000000000003e-05, 2024-09-14T06:29:25.894 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000342729, 2024-09-14T06:29:25.894 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:29:25.894 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:29:25.894 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "queued for pg", 2024-09-14T06:29:25.894 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:29:25.894 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:29:25.894 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:25.894 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 12664 2024-09-14T06:29:25.894 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:25.894 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:29:25.894 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:25.894 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:29:25.895 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.837981+0000", 2024-09-14T06:29:25.895 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:25.895 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:25.895 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:25.895 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:29:25.895 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.837981+0000", 2024-09-14T06:29:25.895 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:25.895 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:25.895 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:25.895 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:29:25.895 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.837981+0000", 2024-09-14T06:29:25.895 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.27e-07 2024-09-14T06:29:25.896 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:25.896 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:25.896 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:29:25.896 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.837983+0000", 2024-09-14T06:29:25.896 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.75e-06 2024-09-14T06:29:25.896 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:25.896 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:25.896 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:29:25.896 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.837991+0000", 2024-09-14T06:29:25.896 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.7700000000000001e-06 2024-09-14T06:29:25.896 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:25.896 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:25.896 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:29:25.896 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.837998+0000", 2024-09-14T06:29:25.896 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.6460000000000004e-06 2024-09-14T06:29:25.897 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:25.897 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:25.897 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:25.897 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:25.897 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:25.897 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:12569 3.1e 3.302ef9fe (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:29:25.897 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:25.836325+0000", 2024-09-14T06:29:25.897 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 0.001714546, 2024-09-14T06:29:25.897 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0020212379999999999, 2024-09-14T06:29:25.897 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:29:25.897 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:29:25.897 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "queued for pg", 2024-09-14T06:29:25.897 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:29:25.897 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:29:25.897 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:25.898 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 12569 2024-09-14T06:29:25.898 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:25.898 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:29:25.898 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:25.898 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:29:25.898 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.836325+0000", 2024-09-14T06:29:25.898 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:25.898 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:25.898 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:25.898 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:29:25.898 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.836325+0000", 2024-09-14T06:29:25.898 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:25.898 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:25.898 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:25.898 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:29:25.899 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.836325+0000", 2024-09-14T06:29:25.899 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.0900000000000001e-07 2024-09-14T06:29:25.899 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:25.899 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:25.899 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:29:25.899 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.836326+0000", 2024-09-14T06:29:25.899 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.09e-07 2024-09-14T06:29:25.899 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:25.899 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:25.899 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:29:25.899 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.836330+0000", 2024-09-14T06:29:25.899 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.9099999999999998e-06 2024-09-14T06:29:25.899 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:25.899 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:25.899 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:29:25.900 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.836333+0000", 2024-09-14T06:29:25.900 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.526e-06 2024-09-14T06:29:25.900 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:25.900 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:25.900 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:25.900 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:25.900 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:25.900 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:12601 3.1e 3.302ef9fe (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:29:25.900 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:25.836828+0000", 2024-09-14T06:29:25.900 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 0.001212294, 2024-09-14T06:29:25.900 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0015419159999999999, 2024-09-14T06:29:25.900 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:29:25.900 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:29:25.900 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "queued for pg", 2024-09-14T06:29:25.900 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:29:25.900 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:29:25.901 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:25.901 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 12601 2024-09-14T06:29:25.901 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:25.901 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:29:25.901 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:25.901 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:29:25.901 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.836828+0000", 2024-09-14T06:29:25.901 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:25.901 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:25.901 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:25.901 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:29:25.901 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.836828+0000", 2024-09-14T06:29:25.901 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:25.902 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:25.902 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:25.902 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:29:25.902 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.836828+0000", 2024-09-14T06:29:25.902 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.1e-07 2024-09-14T06:29:25.902 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:25.902 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:25.902 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:29:25.902 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.836831+0000", 2024-09-14T06:29:25.902 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.411e-06 2024-09-14T06:29:25.902 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:25.902 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:25.902 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:29:25.903 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.836836+0000", 2024-09-14T06:29:25.903 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.2580000000000003e-06 2024-09-14T06:29:25.903 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:25.903 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:25.903 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:29:25.903 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.836840+0000", 2024-09-14T06:29:25.903 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.6279999999999998e-06 2024-09-14T06:29:25.903 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:25.903 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:25.903 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:25.903 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:25.903 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:25.903 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:12633 3.1e 3.302ef9fe (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:29:25.903 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:25.837407+0000", 2024-09-14T06:29:25.903 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 0.00063271499999999995, 2024-09-14T06:29:25.904 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00098492999999999996, 2024-09-14T06:29:25.904 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:29:25.904 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:29:25.904 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "queued for pg", 2024-09-14T06:29:25.904 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:29:25.904 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:29:25.904 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:25.904 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 12633 2024-09-14T06:29:25.904 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:25.904 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:29:25.904 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:25.904 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:29:25.905 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.837407+0000", 2024-09-14T06:29:25.905 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:25.905 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:25.905 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:25.905 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:29:25.905 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.837407+0000", 2024-09-14T06:29:25.905 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:25.905 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:25.905 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:25.905 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:29:25.905 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.837407+0000", 2024-09-14T06:29:25.905 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.35e-07 2024-09-14T06:29:25.906 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:25.906 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:25.906 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:29:25.906 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.837408+0000", 2024-09-14T06:29:25.906 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.0809999999999999e-06 2024-09-14T06:29:25.906 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:25.906 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:25.906 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:29:25.906 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.837412+0000", 2024-09-14T06:29:25.906 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.0099999999999997e-06 2024-09-14T06:29:25.906 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:25.906 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:25.906 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:29:25.906 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.837415+0000", 2024-09-14T06:29:25.906 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.5359999999999999e-06 2024-09-14T06:29:25.907 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:25.907 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:25.907 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:25.907 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:25.907 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:25.907 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:12665 3.1e 3.302ef9fe (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:29:25.907 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:25.838005+0000", 2024-09-14T06:29:25.907 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 3.4409999999999998e-05, 2024-09-14T06:29:25.907 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00040923100000000003, 2024-09-14T06:29:25.907 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:29:25.907 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:29:25.907 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "queued for pg", 2024-09-14T06:29:25.907 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:29:25.907 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:29:25.908 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:25.908 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 12665 2024-09-14T06:29:25.908 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:25.908 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:29:25.908 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:25.908 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:29:25.908 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.838005+0000", 2024-09-14T06:29:25.908 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:25.908 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:25.908 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:25.909 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:29:25.909 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.838005+0000", 2024-09-14T06:29:25.909 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:25.909 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:25.909 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:25.909 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:29:25.909 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.838006+0000", 2024-09-14T06:29:25.909 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.4200000000000002e-07 2024-09-14T06:29:25.909 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:25.910 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:25.910 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:29:25.910 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.838008+0000", 2024-09-14T06:29:25.910 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.8530000000000001e-06 2024-09-14T06:29:25.910 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:25.910 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:25.910 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:29:25.910 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.838014+0000", 2024-09-14T06:29:25.910 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.0059999999999998e-06 2024-09-14T06:29:25.910 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:25.910 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:25.910 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:29:25.910 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.838020+0000", 2024-09-14T06:29:25.910 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.72e-06 2024-09-14T06:29:25.910 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:25.911 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:25.911 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:25.911 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:25.911 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:25.911 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:12570 3.1e 3.302ef9fe (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:29:25.911 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:25.836336+0000", 2024-09-14T06:29:25.911 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 0.001703799, 2024-09-14T06:29:25.911 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0021015489999999999, 2024-09-14T06:29:25.911 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:29:25.911 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:29:25.911 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "queued for pg", 2024-09-14T06:29:25.911 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:29:25.911 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:29:25.911 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:25.911 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 12570 2024-09-14T06:29:25.912 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:25.912 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:29:25.912 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:25.912 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:29:25.912 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.836336+0000", 2024-09-14T06:29:25.912 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:25.912 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:25.912 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:25.912 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:29:25.912 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.836336+0000", 2024-09-14T06:29:25.912 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:25.912 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:25.912 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:25.912 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:29:25.913 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.836336+0000", 2024-09-14T06:29:25.913 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.0200000000000001e-07 2024-09-14T06:29:25.913 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:25.913 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:25.913 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:29:25.913 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.836337+0000", 2024-09-14T06:29:25.913 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.4799999999999997e-07 2024-09-14T06:29:25.913 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:25.913 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:25.913 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:29:25.913 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.836341+0000", 2024-09-14T06:29:25.913 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.473e-06 2024-09-14T06:29:25.914 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:25.914 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:25.914 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:29:25.914 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.836343+0000", 2024-09-14T06:29:25.914 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.4880000000000001e-06 2024-09-14T06:29:25.914 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:25.914 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:25.914 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:25.914 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:25.914 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:25.914 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:12602 3.1e 3.302ef9fe (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:29:25.914 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:25.836844+0000", 2024-09-14T06:29:25.914 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 0.001196304, 2024-09-14T06:29:25.914 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.001622339, 2024-09-14T06:29:25.914 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:29:25.915 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:29:25.915 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "queued for pg", 2024-09-14T06:29:25.915 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:29:25.915 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:29:25.915 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:25.915 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 12602 2024-09-14T06:29:25.915 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:25.915 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:29:25.915 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:25.915 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:29:25.915 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.836844+0000", 2024-09-14T06:29:25.915 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:25.915 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:25.915 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:25.915 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:29:25.915 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.836844+0000", 2024-09-14T06:29:25.916 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:25.916 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:25.916 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:25.916 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:29:25.916 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.836844+0000", 2024-09-14T06:29:25.916 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.2999999999999999e-07 2024-09-14T06:29:25.916 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:25.916 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:25.917 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:29:25.917 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.836845+0000", 2024-09-14T06:29:25.917 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.003e-06 2024-09-14T06:29:25.917 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:25.917 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:25.917 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:29:25.917 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.836848+0000", 2024-09-14T06:29:25.917 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.4359999999999998e-06 2024-09-14T06:29:25.917 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:25.917 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:25.917 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:29:25.917 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.836856+0000", 2024-09-14T06:29:25.917 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.6699999999999994e-06 2024-09-14T06:29:25.917 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:25.918 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:25.918 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:25.918 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:25.918 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:25.918 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:12634 3.1e 3.302ef9fe (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:29:25.918 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:25.837418+0000", 2024-09-14T06:29:25.918 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 0.00062155100000000003, 2024-09-14T06:29:25.918 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.001070571, 2024-09-14T06:29:25.918 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:29:25.918 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:29:25.918 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "queued for pg", 2024-09-14T06:29:25.918 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:29:25.918 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:29:25.918 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:25.918 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 12634 2024-09-14T06:29:25.919 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:25.919 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:29:25.919 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:25.919 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:29:25.919 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.837418+0000", 2024-09-14T06:29:25.919 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:25.919 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:25.919 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:25.919 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:29:25.919 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.837418+0000", 2024-09-14T06:29:25.919 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:25.919 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:25.919 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:25.919 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:29:25.919 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.837419+0000", 2024-09-14T06:29:25.920 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.2700000000000001e-07 2024-09-14T06:29:25.920 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:25.920 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:25.920 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:29:25.920 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.837419+0000", 2024-09-14T06:29:25.920 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.7300000000000004e-07 2024-09-14T06:29:25.920 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:25.920 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:25.920 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:29:25.920 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.837427+0000", 2024-09-14T06:29:25.920 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.1019999999999999e-06 2024-09-14T06:29:25.920 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:25.920 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:25.921 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:29:25.921 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.837433+0000", 2024-09-14T06:29:25.921 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.3679999999999998e-06 2024-09-14T06:29:25.921 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:25.921 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:25.921 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:25.921 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:25.921 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:25.921 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:12666 3.1e 3.302ef9fe (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:29:25.921 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:25.838027+0000", 2024-09-14T06:29:25.921 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 1.2958e-05, 2024-09-14T06:29:25.922 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00048491000000000002, 2024-09-14T06:29:25.922 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:29:25.922 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:29:25.922 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "queued for pg", 2024-09-14T06:29:25.922 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:29:25.922 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:29:25.922 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:25.922 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 12666 2024-09-14T06:29:25.922 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:25.922 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:29:25.922 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:25.923 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:29:25.923 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.838027+0000", 2024-09-14T06:29:25.923 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:25.923 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:25.923 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:25.923 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:29:25.923 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.838027+0000", 2024-09-14T06:29:25.923 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:25.923 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:25.923 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:25.923 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:29:25.923 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.838027+0000", 2024-09-14T06:29:25.923 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.8799999999999998e-07 2024-09-14T06:29:25.923 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:25.923 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:25.924 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:29:25.924 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.838029+0000", 2024-09-14T06:29:25.924 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.852e-06 2024-09-14T06:29:25.924 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:25.924 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:25.924 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:29:25.924 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.838035+0000", 2024-09-14T06:29:25.924 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.7200000000000003e-06 2024-09-14T06:29:25.924 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:25.924 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:25.924 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:29:25.924 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.838040+0000", 2024-09-14T06:29:25.924 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.8180000000000003e-06 2024-09-14T06:29:25.924 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:25.925 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:25.925 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:25.925 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:25.925 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:25.925 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:12571 3.1e 3.302ef9fe (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:29:25.925 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:25.836346+0000", 2024-09-14T06:29:25.925 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 0.0016935730000000001, 2024-09-14T06:29:25.925 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.002188532, 2024-09-14T06:29:25.925 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:29:25.925 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:29:25.925 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "queued for pg", 2024-09-14T06:29:25.925 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:29:25.926 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:29:25.926 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:25.926 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 12571 2024-09-14T06:29:25.926 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:25.926 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:29:25.926 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:25.926 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:29:25.926 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.836346+0000", 2024-09-14T06:29:25.926 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:25.926 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:25.926 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:25.926 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:29:25.926 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.836346+0000", 2024-09-14T06:29:25.926 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:25.926 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:25.927 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:25.927 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:29:25.927 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.836346+0000", 2024-09-14T06:29:25.927 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.0800000000000001e-07 2024-09-14T06:29:25.927 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:25.927 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:25.927 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:29:25.927 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.836348+0000", 2024-09-14T06:29:25.927 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.457e-06 2024-09-14T06:29:25.927 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:25.927 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:25.927 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:29:25.927 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.836352+0000", 2024-09-14T06:29:25.927 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.9940000000000002e-06 2024-09-14T06:29:25.928 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:25.928 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:25.928 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:29:25.928 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.836354+0000", 2024-09-14T06:29:25.928 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.554e-06 2024-09-14T06:29:25.928 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:25.928 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:25.928 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:25.928 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:25.928 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:25.928 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:12603 3.1e 3.302ef9fe (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:29:25.928 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:25.836859+0000", 2024-09-14T06:29:25.928 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 0.0011804070000000001, 2024-09-14T06:29:25.928 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0016980649999999999, 2024-09-14T06:29:25.928 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:29:25.929 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:29:25.929 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "queued for pg", 2024-09-14T06:29:25.929 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:29:25.929 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:29:25.929 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:25.929 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 12603 2024-09-14T06:29:25.929 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:25.929 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:29:25.929 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:25.929 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:29:25.929 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.836859+0000", 2024-09-14T06:29:25.929 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:25.930 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:25.930 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:25.930 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:29:25.930 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.836859+0000", 2024-09-14T06:29:25.930 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:25.930 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:25.930 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:25.930 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:29:25.930 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.836860+0000", 2024-09-14T06:29:25.930 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.2100000000000001e-07 2024-09-14T06:29:25.930 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:25.930 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:25.930 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:29:25.930 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.836861+0000", 2024-09-14T06:29:25.930 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.0499999999999999e-06 2024-09-14T06:29:25.931 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:25.931 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:25.931 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:29:25.931 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.836868+0000", 2024-09-14T06:29:25.931 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.2439999999999996e-06 2024-09-14T06:29:25.931 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:25.931 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:25.931 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:29:25.931 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.836871+0000", 2024-09-14T06:29:25.931 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.0249999999999998e-06 2024-09-14T06:29:25.931 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:25.931 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:25.931 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:25.931 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:25.931 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:25.932 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:12635 3.1e 3.302ef9fe (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:29:25.932 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:25.837456+0000", 2024-09-14T06:29:25.932 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 0.00058430299999999999, 2024-09-14T06:29:25.932 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0011246419999999999, 2024-09-14T06:29:25.932 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:29:25.932 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:29:25.932 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "queued for pg", 2024-09-14T06:29:25.932 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:29:25.932 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:29:25.932 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:25.932 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 12635 2024-09-14T06:29:25.932 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:25.932 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:29:25.932 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:25.933 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:29:25.933 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.837456+0000", 2024-09-14T06:29:25.933 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:25.933 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:25.933 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:25.933 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:29:25.933 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.837456+0000", 2024-09-14T06:29:25.933 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:25.933 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:25.933 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:25.933 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:29:25.933 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.837456+0000", 2024-09-14T06:29:25.933 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.4300000000000003e-07 2024-09-14T06:29:25.933 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:25.934 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:25.934 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:29:25.934 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.837458+0000", 2024-09-14T06:29:25.934 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.0150000000000002e-06 2024-09-14T06:29:25.934 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:25.934 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:25.934 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:29:25.934 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.837465+0000", 2024-09-14T06:29:25.934 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.9909999999999996e-06 2024-09-14T06:29:25.934 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:25.934 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:25.934 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:29:25.934 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.837470+0000", 2024-09-14T06:29:25.934 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.6439999999999999e-06 2024-09-14T06:29:25.935 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:25.935 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:25.935 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:25.935 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:25.935 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:25.935 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:12667 3.1e 3.302ef9fe (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:29:25.935 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:25.838046+0000", 2024-09-14T06:29:25.935 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 4294967295.9999938, 2024-09-14T06:29:25.935 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00055772799999999996, 2024-09-14T06:29:25.935 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:29:25.935 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:29:25.935 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "queued for pg", 2024-09-14T06:29:25.936 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:29:25.936 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:29:25.936 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:25.936 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 12667 2024-09-14T06:29:25.936 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:25.936 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:29:25.936 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:25.936 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:29:25.936 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.838046+0000", 2024-09-14T06:29:25.936 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:25.936 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:25.936 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:25.936 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:29:25.937 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.838046+0000", 2024-09-14T06:29:25.937 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:25.937 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:25.937 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:25.937 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:29:25.937 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.838046+0000", 2024-09-14T06:29:25.937 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.22e-07 2024-09-14T06:29:25.937 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:25.937 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:25.937 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:29:25.937 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.838048+0000", 2024-09-14T06:29:25.937 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.7260000000000001e-06 2024-09-14T06:29:25.937 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:25.937 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:25.938 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:29:25.938 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.838053+0000", 2024-09-14T06:29:25.938 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.1510000000000003e-06 2024-09-14T06:29:25.938 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:25.938 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:25.938 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:29:25.938 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.838059+0000", 2024-09-14T06:29:25.938 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.4179999999999996e-06 2024-09-14T06:29:25.938 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:25.938 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:25.938 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:25.938 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:25.938 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:25.938 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:12572 3.1e 3.302ef9fe (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:29:25.939 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:25.836359+0000", 2024-09-14T06:29:25.939 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 0.0016812800000000001, 2024-09-14T06:29:25.939 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.002269339, 2024-09-14T06:29:25.939 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:29:25.939 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:29:25.939 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "queued for pg", 2024-09-14T06:29:25.939 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:29:25.939 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:29:25.939 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:25.939 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 12572 2024-09-14T06:29:25.939 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:25.939 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:29:25.939 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:25.939 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:29:25.939 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.836359+0000", 2024-09-14T06:29:25.940 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:25.940 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:25.940 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:25.940 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:29:25.940 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.836359+0000", 2024-09-14T06:29:25.940 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:25.940 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:25.940 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:25.940 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:29:25.940 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.836359+0000", 2024-09-14T06:29:25.940 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.1e-07 2024-09-14T06:29:25.941 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:25.941 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:25.941 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:29:25.941 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.836360+0000", 2024-09-14T06:29:25.941 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.3500000000000005e-07 2024-09-14T06:29:25.941 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:25.941 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:25.941 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:29:25.941 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.836363+0000", 2024-09-14T06:29:25.941 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.4510000000000001e-06 2024-09-14T06:29:25.941 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:25.941 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:25.941 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:29:25.942 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.836366+0000", 2024-09-14T06:29:25.942 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.5339999999999998e-06 2024-09-14T06:29:25.942 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:25.942 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:25.942 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:25.942 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:25.942 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:25.942 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:12604 3.1e 3.302ef9fe (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:29:25.942 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:25.836885+0000", 2024-09-14T06:29:25.942 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 0.0011546060000000001, 2024-09-14T06:29:25.942 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0017654089999999999, 2024-09-14T06:29:25.942 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:29:25.942 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:29:25.942 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "queued for pg", 2024-09-14T06:29:25.942 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:29:25.943 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:29:25.943 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:25.943 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 12604 2024-09-14T06:29:25.943 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:25.943 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:29:25.943 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:25.943 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:29:25.943 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.836885+0000", 2024-09-14T06:29:25.943 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:25.943 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:25.943 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:25.943 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:29:25.943 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.836885+0000", 2024-09-14T06:29:25.943 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:25.943 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:25.944 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:25.944 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:29:25.944 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.836885+0000", 2024-09-14T06:29:25.944 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.29e-07 2024-09-14T06:29:25.944 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:25.944 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:25.944 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:29:25.944 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.836887+0000", 2024-09-14T06:29:25.944 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.5740000000000001e-06 2024-09-14T06:29:25.944 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:25.944 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:25.944 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:29:25.944 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.836891+0000", 2024-09-14T06:29:25.944 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.2119999999999997e-06 2024-09-14T06:29:25.944 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:25.945 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:25.945 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:29:25.945 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.836894+0000", 2024-09-14T06:29:25.945 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.587e-06 2024-09-14T06:29:25.945 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:25.945 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:25.945 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:25.945 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:25.945 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:25.945 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:12636 3.1e 3.302ef9fe (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:29:25.945 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:25.837476+0000", 2024-09-14T06:29:25.945 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 0.00056345600000000005, 2024-09-14T06:29:25.945 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0011970520000000001, 2024-09-14T06:29:25.945 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:29:25.945 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:29:25.946 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "queued for pg", 2024-09-14T06:29:25.946 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:29:25.946 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:29:25.946 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:25.946 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 12636 2024-09-14T06:29:25.946 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:25.946 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:29:25.946 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:25.946 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:29:25.946 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.837476+0000", 2024-09-14T06:29:25.946 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:25.946 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:25.947 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:25.947 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:29:25.947 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.837476+0000", 2024-09-14T06:29:25.947 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:25.947 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:25.947 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:25.947 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:29:25.947 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.837477+0000", 2024-09-14T06:29:25.947 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.84e-07 2024-09-14T06:29:25.947 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:25.947 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:25.948 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:29:25.948 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.837479+0000", 2024-09-14T06:29:25.948 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.9180000000000001e-06 2024-09-14T06:29:25.948 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:25.948 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:25.948 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:29:25.948 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.837484+0000", 2024-09-14T06:29:25.948 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.5230000000000003e-06 2024-09-14T06:29:25.948 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:25.948 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:25.948 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:29:25.948 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.837490+0000", 2024-09-14T06:29:25.949 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.6869999999999999e-06 2024-09-14T06:29:25.949 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:25.949 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:25.949 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:25.949 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:25.949 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:25.949 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:12668 3.1e 3.302ef9fe (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:29:25.949 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:25.838067+0000", 2024-09-14T06:29:25.949 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 4294967295.9999723, 2024-09-14T06:29:25.949 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00062931700000000001, 2024-09-14T06:29:25.949 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:29:25.949 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:29:25.949 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "queued for pg", 2024-09-14T06:29:25.950 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:29:25.950 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:29:25.950 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:25.950 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 12668 2024-09-14T06:29:25.950 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:25.950 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:29:25.950 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:25.950 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:29:25.950 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.838067+0000", 2024-09-14T06:29:25.950 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:25.950 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:25.950 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:25.950 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:29:25.950 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.838067+0000", 2024-09-14T06:29:25.951 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:25.951 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:25.951 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:25.951 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:29:25.951 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.838068+0000", 2024-09-14T06:29:25.951 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.3999999999999997e-07 2024-09-14T06:29:25.951 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:25.951 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:25.951 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:29:25.951 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.838070+0000", 2024-09-14T06:29:25.951 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.8929999999999999e-06 2024-09-14T06:29:25.952 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:25.952 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:25.952 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:29:25.952 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.838080+0000", 2024-09-14T06:29:25.952 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.8320000000000001e-06 2024-09-14T06:29:25.952 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:25.952 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:25.952 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:29:25.952 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.838088+0000", 2024-09-14T06:29:25.952 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.2840000000000006e-06 2024-09-14T06:29:25.952 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:25.952 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:25.952 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:25.952 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:25.953 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:25.953 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:12573 3.1e 3.302ef9fe (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:29:25.953 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:25.836369+0000", 2024-09-14T06:29:25.953 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 0.0016710589999999999, 2024-09-14T06:29:25.953 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0023516349999999999, 2024-09-14T06:29:25.953 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:29:25.953 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:29:25.953 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "queued for pg", 2024-09-14T06:29:25.953 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:29:25.953 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:29:25.953 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:25.953 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 12573 2024-09-14T06:29:25.953 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:25.953 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:29:25.954 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:25.954 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:29:25.954 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.836369+0000", 2024-09-14T06:29:25.954 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:25.954 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:25.954 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:25.954 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:29:25.954 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.836369+0000", 2024-09-14T06:29:25.954 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:25.954 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:25.954 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:25.954 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:29:25.954 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.836369+0000", 2024-09-14T06:29:25.954 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.0900000000000001e-07 2024-09-14T06:29:25.954 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:25.955 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:25.955 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:29:25.955 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.836370+0000", 2024-09-14T06:29:25.955 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.44e-06 2024-09-14T06:29:25.955 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:25.955 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:25.955 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:29:25.955 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.836375+0000", 2024-09-14T06:29:25.955 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.3869999999999997e-06 2024-09-14T06:29:25.955 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:25.955 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:25.955 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:29:25.955 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.836377+0000", 2024-09-14T06:29:25.955 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.4770000000000002e-06 2024-09-14T06:29:25.955 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:25.956 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:25.956 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:25.956 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:25.956 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:25.956 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:12605 3.1e 3.302ef9fe (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:29:25.956 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:25.836898+0000", 2024-09-14T06:29:25.956 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 0.0011418139999999999, 2024-09-14T06:29:25.956 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0018455399999999999, 2024-09-14T06:29:25.956 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:29:25.956 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:29:25.956 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "queued for pg", 2024-09-14T06:29:25.956 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:29:25.956 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:29:25.956 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:25.956 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 12605 2024-09-14T06:29:25.957 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:25.957 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:29:25.957 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:25.957 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:29:25.957 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.836898+0000", 2024-09-14T06:29:25.957 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:25.957 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:25.957 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:25.957 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:29:25.957 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.836898+0000", 2024-09-14T06:29:25.957 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:25.957 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:25.957 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:25.957 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:29:25.958 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.836898+0000", 2024-09-14T06:29:25.958 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.03e-07 2024-09-14T06:29:25.958 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:25.958 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:25.958 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:29:25.958 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.836899+0000", 2024-09-14T06:29:25.958 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.6899999999999996e-07 2024-09-14T06:29:25.958 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:25.958 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:25.958 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:29:25.958 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.836902+0000", 2024-09-14T06:29:25.958 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.038e-06 2024-09-14T06:29:25.958 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:25.958 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:25.958 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:29:25.959 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.836905+0000", 2024-09-14T06:29:25.959 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.791e-06 2024-09-14T06:29:25.959 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:25.959 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:25.959 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:25.959 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:25.959 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:25.959 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:12637 3.1e 3.302ef9fe (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:29:25.959 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:25.837496+0000", 2024-09-14T06:29:25.959 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 0.00054358, 2024-09-14T06:29:25.959 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.001271627, 2024-09-14T06:29:25.959 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:29:25.959 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:29:25.959 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "queued for pg", 2024-09-14T06:29:25.959 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:29:25.960 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:29:25.960 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:25.960 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 12637 2024-09-14T06:29:25.960 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:25.960 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:29:25.960 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:25.960 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:29:25.960 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.837496+0000", 2024-09-14T06:29:25.961 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:25.961 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:25.961 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:25.961 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:29:25.961 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.837496+0000", 2024-09-14T06:29:25.961 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:25.961 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:25.961 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:25.961 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:29:25.962 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.837497+0000", 2024-09-14T06:29:25.962 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.7500000000000001e-07 2024-09-14T06:29:25.962 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:25.962 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:25.962 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:29:25.962 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.837501+0000", 2024-09-14T06:29:25.962 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.7210000000000002e-06 2024-09-14T06:29:25.962 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:25.962 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:25.962 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:29:25.962 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.837508+0000", 2024-09-14T06:29:25.962 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.46e-06 2024-09-14T06:29:25.962 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:25.962 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:25.962 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:29:25.962 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.837512+0000", 2024-09-14T06:29:25.963 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.5669999999999996e-06 2024-09-14T06:29:25.963 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:25.963 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:25.963 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:25.963 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:25.963 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:25.963 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:12669 3.1e 3.302ef9fe (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:29:25.963 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:25.838114+0000", 2024-09-14T06:29:25.963 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 4294967295.9999261, 2024-09-14T06:29:25.963 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00067772999999999998, 2024-09-14T06:29:25.963 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:29:25.963 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:29:25.963 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "queued for pg", 2024-09-14T06:29:25.963 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:29:25.963 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:29:25.964 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:25.964 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 12669 2024-09-14T06:29:25.964 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:25.964 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:29:25.964 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:25.964 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:29:25.964 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.838114+0000", 2024-09-14T06:29:25.964 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:25.964 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:25.964 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:25.964 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:29:25.964 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.838114+0000", 2024-09-14T06:29:25.964 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:25.964 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:25.965 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:25.965 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:29:25.965 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.838114+0000", 2024-09-14T06:29:25.965 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.4999999999999998e-07 2024-09-14T06:29:25.965 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:25.965 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:25.965 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:29:25.965 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.838116+0000", 2024-09-14T06:29:25.965 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.6589999999999999e-06 2024-09-14T06:29:25.965 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:25.965 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:25.965 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:29:25.965 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.838120+0000", 2024-09-14T06:29:25.965 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.0999999999999997e-06 2024-09-14T06:29:25.965 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:25.966 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:25.966 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:29:25.966 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.838123+0000", 2024-09-14T06:29:25.966 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.8770000000000002e-06 2024-09-14T06:29:25.966 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:25.966 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:25.966 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:25.966 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:25.966 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:25.966 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:12574 3.1e 3.302ef9fe (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:29:25.966 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:25.836381+0000", 2024-09-14T06:29:25.966 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 0.0016592740000000001, 2024-09-14T06:29:25.966 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0024492400000000001, 2024-09-14T06:29:25.966 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:29:25.967 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:29:25.967 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "queued for pg", 2024-09-14T06:29:25.967 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:29:25.967 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:29:25.967 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:25.967 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 12574 2024-09-14T06:29:25.967 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:25.967 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:29:25.967 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:25.967 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:29:25.967 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.836381+0000", 2024-09-14T06:29:25.968 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:25.968 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:25.968 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:25.968 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:29:25.968 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.836381+0000", 2024-09-14T06:29:25.968 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:25.968 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:25.968 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:25.968 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:29:25.968 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.836381+0000", 2024-09-14T06:29:25.968 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.04e-07 2024-09-14T06:29:25.968 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:25.968 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:25.968 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:29:25.969 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.836382+0000", 2024-09-14T06:29:25.969 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.1699999999999997e-07 2024-09-14T06:29:25.969 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:25.969 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:25.969 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:29:25.969 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.836385+0000", 2024-09-14T06:29:25.969 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.959e-06 2024-09-14T06:29:25.969 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:25.969 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:25.969 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:29:25.969 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.836388+0000", 2024-09-14T06:29:25.969 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.999e-06 2024-09-14T06:29:25.969 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:25.969 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:25.970 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:25.970 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:25.970 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:25.970 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:12606 3.1e 3.302ef9fe (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:29:25.970 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:25.836909+0000", 2024-09-14T06:29:25.970 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 0.0011312570000000001, 2024-09-14T06:29:25.970 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.001944753, 2024-09-14T06:29:25.970 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:29:25.970 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:29:25.970 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "queued for pg", 2024-09-14T06:29:25.970 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:29:25.970 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:29:25.970 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:25.971 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 12606 2024-09-14T06:29:25.971 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:25.971 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:29:25.971 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:25.971 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:29:25.971 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.836909+0000", 2024-09-14T06:29:25.971 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:25.971 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:25.971 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:25.971 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:29:25.971 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.836909+0000", 2024-09-14T06:29:25.971 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:25.971 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:25.971 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:25.971 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:29:25.971 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.836909+0000", 2024-09-14T06:29:25.972 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.04e-07 2024-09-14T06:29:25.972 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:25.972 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:25.972 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:29:25.972 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.836912+0000", 2024-09-14T06:29:25.972 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.6909999999999999e-06 2024-09-14T06:29:25.972 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:25.972 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:25.972 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:29:25.972 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.836918+0000", 2024-09-14T06:29:25.972 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.728e-06 2024-09-14T06:29:25.972 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:25.972 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:25.972 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:29:25.972 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.836921+0000", 2024-09-14T06:29:25.972 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.6070000000000002e-06 2024-09-14T06:29:25.973 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:25.973 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:25.973 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:25.973 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:25.973 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:25.973 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:12638 3.1e 3.302ef9fe (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:29:25.973 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:25.837520+0000", 2024-09-14T06:29:25.973 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 0.00052008299999999996, 2024-09-14T06:29:25.973 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0013771510000000001, 2024-09-14T06:29:25.973 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:29:25.973 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:29:25.973 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "queued for pg", 2024-09-14T06:29:25.973 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:29:25.974 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:29:25.974 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:25.974 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 12638 2024-09-14T06:29:25.974 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:25.974 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:29:25.974 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:25.974 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:29:25.974 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.837520+0000", 2024-09-14T06:29:25.974 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:25.974 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:25.974 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:25.974 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:29:25.974 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.837520+0000", 2024-09-14T06:29:25.974 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:25.975 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:25.975 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:25.975 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:29:25.975 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.837520+0000", 2024-09-14T06:29:25.975 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.3099999999999999e-07 2024-09-14T06:29:25.975 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:25.975 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:25.975 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:29:25.975 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.837522+0000", 2024-09-14T06:29:25.975 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.8199999999999999e-06 2024-09-14T06:29:25.975 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:25.975 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:25.975 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:29:25.975 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.837529+0000", 2024-09-14T06:29:25.975 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.6699999999999997e-06 2024-09-14T06:29:25.976 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:25.976 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:25.976 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:29:25.976 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.837538+0000", 2024-09-14T06:29:25.976 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.5349999999999993e-06 2024-09-14T06:29:25.976 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:25.976 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:25.976 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:25.976 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:25.976 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:25.976 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:12670 3.1e 3.302ef9fe (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:29:25.976 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:25.838127+0000", 2024-09-14T06:29:25.976 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 4294967295.9999132, 2024-09-14T06:29:25.976 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00079450500000000004, 2024-09-14T06:29:25.976 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:29:25.977 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:29:25.977 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "queued for pg", 2024-09-14T06:29:25.977 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:29:25.977 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:29:25.977 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:25.977 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 12670 2024-09-14T06:29:25.977 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:25.977 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:29:25.977 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:25.977 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:29:25.978 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.838127+0000", 2024-09-14T06:29:25.978 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:25.978 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:25.978 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:25.978 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:29:25.978 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.838127+0000", 2024-09-14T06:29:25.978 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:25.978 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:25.978 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:25.978 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:29:25.978 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.838127+0000", 2024-09-14T06:29:25.978 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.1e-07 2024-09-14T06:29:25.978 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:25.978 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:25.978 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:29:25.979 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.838128+0000", 2024-09-14T06:29:25.979 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.2800000000000005e-07 2024-09-14T06:29:25.979 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:25.979 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:25.979 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:29:25.979 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.838132+0000", 2024-09-14T06:29:25.979 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.9400000000000004e-06 2024-09-14T06:29:25.979 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:25.979 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:25.979 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:29:25.979 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.838135+0000", 2024-09-14T06:29:25.979 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.7530000000000002e-06 2024-09-14T06:29:25.979 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:25.979 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:25.979 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:25.980 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:25.980 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:25.980 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:12575 3.1e 3.302ef9fe (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:29:25.980 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:25.836391+0000", 2024-09-14T06:29:25.980 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 0.0016491959999999999, 2024-09-14T06:29:25.980 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0025544700000000001, 2024-09-14T06:29:25.980 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:29:25.980 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:29:25.980 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "queued for pg", 2024-09-14T06:29:25.980 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:29:25.980 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:29:25.980 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:25.980 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 12575 2024-09-14T06:29:25.981 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:25.981 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:29:25.981 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:25.981 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:29:25.981 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.836391+0000", 2024-09-14T06:29:25.981 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:25.981 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:25.981 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:25.981 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:29:25.981 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.836391+0000", 2024-09-14T06:29:25.981 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:25.981 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:25.981 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:25.981 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:29:25.982 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.836391+0000", 2024-09-14T06:29:25.982 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.0200000000000001e-07 2024-09-14T06:29:25.982 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:25.982 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:25.982 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:29:25.982 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.836392+0000", 2024-09-14T06:29:25.982 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.5499999999999996e-07 2024-09-14T06:29:25.982 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:25.982 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:25.982 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:29:25.982 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.836395+0000", 2024-09-14T06:29:25.982 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.915e-06 2024-09-14T06:29:25.982 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:25.982 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:25.983 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:29:25.983 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.836397+0000", 2024-09-14T06:29:25.983 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.4509999999999999e-06 2024-09-14T06:29:25.983 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:25.983 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:25.983 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:25.983 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:25.983 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:25.983 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:12607 3.1e 3.302ef9fe (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:29:25.983 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:25.836924+0000", 2024-09-14T06:29:25.983 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 0.001115551, 2024-09-14T06:29:25.983 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0020453839999999999, 2024-09-14T06:29:25.983 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:29:25.983 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:29:25.984 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "queued for pg", 2024-09-14T06:29:25.984 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:29:25.984 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:29:25.984 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:25.984 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 12607 2024-09-14T06:29:25.984 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:25.984 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:29:25.984 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:25.984 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:29:25.984 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.836924+0000", 2024-09-14T06:29:25.984 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:25.984 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:25.984 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:25.984 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:29:25.985 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.836924+0000", 2024-09-14T06:29:25.985 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:25.985 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:25.985 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:25.985 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:29:25.985 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.836924+0000", 2024-09-14T06:29:25.985 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.0800000000000001e-07 2024-09-14T06:29:25.985 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:25.985 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:25.985 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:29:25.985 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.836925+0000", 2024-09-14T06:29:25.986 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.5600000000000004e-07 2024-09-14T06:29:25.986 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:25.986 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:25.986 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:29:25.986 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.836928+0000", 2024-09-14T06:29:25.986 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.005e-06 2024-09-14T06:29:25.986 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:25.986 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:25.986 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:29:25.986 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.836931+0000", 2024-09-14T06:29:25.986 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.424e-06 2024-09-14T06:29:25.987 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:25.987 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:25.987 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:25.987 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:25.987 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:25.987 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:12639 3.1e 3.302ef9fe (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:29:25.987 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:25.837545+0000", 2024-09-14T06:29:25.987 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 0.00049503800000000005, 2024-09-14T06:29:25.987 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0014475390000000001, 2024-09-14T06:29:25.987 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:29:25.987 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:29:25.987 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "queued for pg", 2024-09-14T06:29:25.987 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:29:25.987 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:29:25.987 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:25.988 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 12639 2024-09-14T06:29:25.988 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:25.988 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:29:25.988 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:25.988 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:29:25.988 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.837545+0000", 2024-09-14T06:29:25.988 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:25.988 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:25.988 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:25.988 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:29:25.988 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.837545+0000", 2024-09-14T06:29:25.989 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:25.989 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:25.989 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:25.989 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:29:25.989 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.837545+0000", 2024-09-14T06:29:25.989 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.1600000000000002e-07 2024-09-14T06:29:25.989 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:25.989 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:25.989 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:29:25.989 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.837548+0000", 2024-09-14T06:29:25.990 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.379e-06 2024-09-14T06:29:25.990 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:25.990 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:25.990 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:29:25.990 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.837554+0000", 2024-09-14T06:29:25.990 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.8809999999999998e-06 2024-09-14T06:29:25.990 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:25.990 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:25.990 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:29:25.990 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.837559+0000", 2024-09-14T06:29:25.990 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.4349999999999999e-06 2024-09-14T06:29:25.990 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:25.990 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:25.991 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:25.991 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:25.991 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:25.991 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:12671 3.1e 3.302ef9fe (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:29:25.991 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:25.838138+0000", 2024-09-14T06:29:25.991 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 4294967295.9999018, 2024-09-14T06:29:25.991 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000879585, 2024-09-14T06:29:25.991 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:29:25.991 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:29:25.991 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "queued for pg", 2024-09-14T06:29:25.991 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:29:25.991 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:29:25.991 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:25.992 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 12671 2024-09-14T06:29:25.992 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:25.992 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:29:25.992 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:25.992 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:29:25.992 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.838138+0000", 2024-09-14T06:29:25.992 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:25.992 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:25.993 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:25.993 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:29:25.993 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.838138+0000", 2024-09-14T06:29:25.993 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:25.993 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:25.993 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:25.993 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:29:25.993 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.838138+0000", 2024-09-14T06:29:25.993 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.3099999999999999e-07 2024-09-14T06:29:25.993 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:25.993 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:25.993 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:29:25.993 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.838139+0000", 2024-09-14T06:29:25.994 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.8000000000000004e-07 2024-09-14T06:29:25.994 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:25.994 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:25.994 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:29:25.994 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.838142+0000", 2024-09-14T06:29:25.994 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.0010000000000002e-06 2024-09-14T06:29:25.994 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:25.994 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:25.994 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:29:25.994 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.838144+0000", 2024-09-14T06:29:25.994 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.5399999999999998e-06 2024-09-14T06:29:25.994 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:25.994 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:25.995 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:25.995 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:25.995 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:25.995 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:12576 3.1e 3.302ef9fe (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:29:25.995 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:25.836400+0000", 2024-09-14T06:29:25.995 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 0.0016396259999999999, 2024-09-14T06:29:25.995 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.002641081, 2024-09-14T06:29:25.995 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:29:25.995 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:29:25.995 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "queued for pg", 2024-09-14T06:29:25.995 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:29:25.996 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:29:25.996 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:25.996 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 12576 2024-09-14T06:29:25.996 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:25.996 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:29:25.996 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:25.996 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:29:25.996 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.836400+0000", 2024-09-14T06:29:25.996 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:25.996 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:25.996 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:25.996 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:29:25.996 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.836400+0000", 2024-09-14T06:29:25.996 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:25.997 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:25.997 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:25.997 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:29:25.997 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.836400+0000", 2024-09-14T06:29:25.997 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.0599999999999999e-07 2024-09-14T06:29:25.997 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:25.997 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:25.997 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:29:25.997 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.836401+0000", 2024-09-14T06:29:25.997 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.5499999999999997e-07 2024-09-14T06:29:25.997 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:25.997 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:25.997 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:29:25.997 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.836405+0000", 2024-09-14T06:29:25.998 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.8249999999999998e-06 2024-09-14T06:29:25.998 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:25.998 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:25.998 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:29:25.998 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.836409+0000", 2024-09-14T06:29:25.998 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.0409999999999999e-06 2024-09-14T06:29:25.998 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:25.998 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:25.998 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:25.998 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:25.998 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:25.999 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:12608 3.1e 3.302ef9fe (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:29:25.999 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:25.836934+0000", 2024-09-14T06:29:25.999 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 0.0011059329999999999, 2024-09-14T06:29:25.999 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0021317480000000001, 2024-09-14T06:29:25.999 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:29:25.999 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:29:25.999 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "queued for pg", 2024-09-14T06:29:25.999 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:29:25.999 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:29:25.999 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:25.999 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 12608 2024-09-14T06:29:25.999 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:25.999 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:29:26.000 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.000 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:29:26.000 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.836934+0000", 2024-09-14T06:29:26.000 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:26.000 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.000 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.000 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:29:26.000 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.836934+0000", 2024-09-14T06:29:26.000 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:26.000 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.000 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.000 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:29:26.000 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.836934+0000", 2024-09-14T06:29:26.000 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.0599999999999999e-07 2024-09-14T06:29:26.001 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.001 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.001 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:29:26.001 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.836935+0000", 2024-09-14T06:29:26.001 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.09e-07 2024-09-14T06:29:26.001 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.001 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.001 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:29:26.001 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.836938+0000", 2024-09-14T06:29:26.001 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.0139999999999999e-06 2024-09-14T06:29:26.001 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.001 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.001 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:29:26.002 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.836941+0000", 2024-09-14T06:29:26.002 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.9359999999999999e-06 2024-09-14T06:29:26.002 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:26.002 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:26.002 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:26.002 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.002 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.002 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:12640 3.1e 3.302ef9fe (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:29:26.002 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:25.837566+0000", 2024-09-14T06:29:26.002 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 0.000473881, 2024-09-14T06:29:26.002 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0015254120000000001, 2024-09-14T06:29:26.002 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:29:26.002 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:29:26.002 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "queued for pg", 2024-09-14T06:29:26.003 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:29:26.003 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:29:26.003 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:26.003 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 12640 2024-09-14T06:29:26.003 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.003 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:29:26.003 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.003 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:29:26.003 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.837566+0000", 2024-09-14T06:29:26.003 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:26.003 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.003 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.003 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:29:26.003 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.837566+0000", 2024-09-14T06:29:26.003 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:26.003 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.004 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.004 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:29:26.004 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.837566+0000", 2024-09-14T06:29:26.004 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.8000000000000001e-07 2024-09-14T06:29:26.004 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.004 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.004 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:29:26.004 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.837568+0000", 2024-09-14T06:29:26.004 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.778e-06 2024-09-14T06:29:26.004 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.004 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.004 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:29:26.005 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.837574+0000", 2024-09-14T06:29:26.005 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.4589999999999997e-06 2024-09-14T06:29:26.005 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.005 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.005 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:29:26.005 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.837578+0000", 2024-09-14T06:29:26.005 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.6260000000000003e-06 2024-09-14T06:29:26.005 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:26.005 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:26.005 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:26.005 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.006 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.006 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:12672 3.1e 3.302ef9fe (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:29:26.006 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:25.838147+0000", 2024-09-14T06:29:26.006 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 4294967295.9998922, 2024-09-14T06:29:26.006 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00097050900000000004, 2024-09-14T06:29:26.006 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:29:26.006 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:29:26.006 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "queued for pg", 2024-09-14T06:29:26.006 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:29:26.006 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:29:26.006 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:26.006 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 12672 2024-09-14T06:29:26.007 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.007 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:29:26.007 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.007 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:29:26.007 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.838147+0000", 2024-09-14T06:29:26.007 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:26.007 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.007 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.007 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:29:26.007 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.838147+0000", 2024-09-14T06:29:26.007 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:26.007 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.007 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.007 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:29:26.007 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.838148+0000", 2024-09-14T06:29:26.008 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.1400000000000001e-07 2024-09-14T06:29:26.008 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.008 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.008 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:29:26.008 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.838152+0000", 2024-09-14T06:29:26.008 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.5430000000000004e-06 2024-09-14T06:29:26.008 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.008 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.008 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:29:26.008 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.838156+0000", 2024-09-14T06:29:26.008 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.9509999999999999e-06 2024-09-14T06:29:26.008 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.008 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.009 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:29:26.009 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.838159+0000", 2024-09-14T06:29:26.009 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.655e-06 2024-09-14T06:29:26.009 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:26.009 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:26.009 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:26.009 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.009 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.009 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:12577 3.1e 3.302ef9fe (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:29:26.009 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:25.836412+0000", 2024-09-14T06:29:26.009 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 0.0016276750000000001, 2024-09-14T06:29:26.009 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.002729665, 2024-09-14T06:29:26.009 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:29:26.009 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:29:26.010 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "queued for pg", 2024-09-14T06:29:26.010 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:29:26.010 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:29:26.010 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:26.010 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 12577 2024-09-14T06:29:26.010 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.010 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:29:26.010 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.010 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:29:26.010 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.836412+0000", 2024-09-14T06:29:26.011 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:26.011 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.011 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.011 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:29:26.011 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.836412+0000", 2024-09-14T06:29:26.011 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:26.011 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.011 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.011 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:29:26.011 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.836412+0000", 2024-09-14T06:29:26.011 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.0900000000000001e-07 2024-09-14T06:29:26.011 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.011 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.011 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:29:26.011 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.836416+0000", 2024-09-14T06:29:26.012 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.2119999999999999e-06 2024-09-14T06:29:26.012 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.012 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.012 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:29:26.012 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.836420+0000", 2024-09-14T06:29:26.012 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.2690000000000001e-06 2024-09-14T06:29:26.012 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.012 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.012 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:29:26.012 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.836422+0000", 2024-09-14T06:29:26.012 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.6139999999999999e-06 2024-09-14T06:29:26.012 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:26.012 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:26.012 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:26.012 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.013 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.013 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:12609 3.1e 3.302ef9fe (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:29:26.013 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:25.836946+0000", 2024-09-14T06:29:26.013 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 0.0010941290000000001, 2024-09-14T06:29:26.013 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0022206000000000001, 2024-09-14T06:29:26.013 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:29:26.013 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:29:26.013 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "queued for pg", 2024-09-14T06:29:26.013 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:29:26.013 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:29:26.013 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:26.013 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 12609 2024-09-14T06:29:26.014 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.014 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:29:26.014 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.014 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:29:26.014 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.836946+0000", 2024-09-14T06:29:26.014 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:26.014 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.014 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.014 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:29:26.014 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.836946+0000", 2024-09-14T06:29:26.014 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:26.014 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.014 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.014 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:29:26.014 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.836946+0000", 2024-09-14T06:29:26.015 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.13e-07 2024-09-14T06:29:26.015 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.015 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.015 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:29:26.015 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.836948+0000", 2024-09-14T06:29:26.015 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.3790000000000001e-06 2024-09-14T06:29:26.015 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.015 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.015 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:29:26.015 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.836956+0000", 2024-09-14T06:29:26.015 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.1890000000000007e-06 2024-09-14T06:29:26.015 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.015 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.015 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:29:26.015 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.836960+0000", 2024-09-14T06:29:26.016 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.9339999999999999e-06 2024-09-14T06:29:26.016 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:26.016 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:26.016 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:26.016 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.016 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.016 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:12641 3.1e 3.302ef9fe (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:29:26.016 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:25.837584+0000", 2024-09-14T06:29:26.016 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 0.00045572700000000001, 2024-09-14T06:29:26.016 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.001604769, 2024-09-14T06:29:26.016 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:29:26.017 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:29:26.017 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "queued for pg", 2024-09-14T06:29:26.017 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:29:26.017 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:29:26.017 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:26.017 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 12641 2024-09-14T06:29:26.017 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.017 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:29:26.017 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.017 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:29:26.017 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.837584+0000", 2024-09-14T06:29:26.017 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:26.017 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.017 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.018 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:29:26.018 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.837584+0000", 2024-09-14T06:29:26.018 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:26.018 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.018 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.018 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:29:26.018 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.837584+0000", 2024-09-14T06:29:26.018 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.53e-07 2024-09-14T06:29:26.018 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.018 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.018 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:29:26.018 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.837586+0000", 2024-09-14T06:29:26.018 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.7129999999999999e-06 2024-09-14T06:29:26.018 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.018 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.019 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:29:26.019 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.837591+0000", 2024-09-14T06:29:26.019 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.2290000000000002e-06 2024-09-14T06:29:26.019 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.019 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.019 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:29:26.019 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.837599+0000", 2024-09-14T06:29:26.019 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.61e-06 2024-09-14T06:29:26.019 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:26.019 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:26.019 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:26.019 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.019 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.020 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:12673 3.1e 3.302ef9fe (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:29:26.020 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:25.838162+0000", 2024-09-14T06:29:26.020 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 4294967295.9998775, 2024-09-14T06:29:26.020 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.001051462, 2024-09-14T06:29:26.020 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:29:26.020 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:29:26.020 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "queued for pg", 2024-09-14T06:29:26.020 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:29:26.020 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:29:26.020 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:26.020 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 12673 2024-09-14T06:29:26.020 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.020 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:29:26.020 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.020 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:29:26.021 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.838162+0000", 2024-09-14T06:29:26.021 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:26.021 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.021 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.021 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:29:26.021 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.838162+0000", 2024-09-14T06:29:26.021 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:26.021 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.021 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.021 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:29:26.021 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.838162+0000", 2024-09-14T06:29:26.021 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.1899999999999999e-07 2024-09-14T06:29:26.021 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.021 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.021 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:29:26.022 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.838163+0000", 2024-09-14T06:29:26.022 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.0249999999999999e-06 2024-09-14T06:29:26.022 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.022 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.022 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:29:26.022 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.838167+0000", 2024-09-14T06:29:26.022 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.5360000000000001e-06 2024-09-14T06:29:26.022 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.022 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.023 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:29:26.023 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.838170+0000", 2024-09-14T06:29:26.023 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.649e-06 2024-09-14T06:29:26.023 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:26.023 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:26.023 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:26.023 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.023 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.023 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:12578 3.1e 3.302ef9fe (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:29:26.023 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:25.836426+0000", 2024-09-14T06:29:26.023 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 0.001613878, 2024-09-14T06:29:26.023 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0028114820000000001, 2024-09-14T06:29:26.023 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:29:26.023 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:29:26.024 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "queued for pg", 2024-09-14T06:29:26.024 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:29:26.024 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:29:26.024 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:26.024 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 12578 2024-09-14T06:29:26.024 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.024 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:29:26.024 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.024 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:29:26.024 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.836426+0000", 2024-09-14T06:29:26.024 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:26.024 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.024 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.024 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:29:26.024 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.836426+0000", 2024-09-14T06:29:26.025 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:26.025 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.025 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.025 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:29:26.025 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.836426+0000", 2024-09-14T06:29:26.025 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.3999999999999998e-07 2024-09-14T06:29:26.025 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.025 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.025 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:29:26.025 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.836427+0000", 2024-09-14T06:29:26.025 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.0580000000000001e-06 2024-09-14T06:29:26.025 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.025 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.025 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:29:26.025 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.836430+0000", 2024-09-14T06:29:26.026 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.9739999999999998e-06 2024-09-14T06:29:26.026 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.026 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.026 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:29:26.026 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.836433+0000", 2024-09-14T06:29:26.026 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.712e-06 2024-09-14T06:29:26.026 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:26.026 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:26.026 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:26.026 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.026 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.026 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:12610 3.1e 3.302ef9fe (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:29:26.027 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:25.836973+0000", 2024-09-14T06:29:26.027 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 0.0010668240000000001, 2024-09-14T06:29:26.027 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.002288703, 2024-09-14T06:29:26.027 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:29:26.027 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:29:26.027 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "queued for pg", 2024-09-14T06:29:26.027 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:29:26.027 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:29:26.027 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:26.027 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 12610 2024-09-14T06:29:26.027 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.027 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:29:26.027 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.027 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:29:26.027 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.836973+0000", 2024-09-14T06:29:26.028 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:26.028 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.028 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.028 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:29:26.028 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.836973+0000", 2024-09-14T06:29:26.028 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:26.028 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.028 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.028 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:29:26.028 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.836973+0000", 2024-09-14T06:29:26.028 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.4700000000000002e-07 2024-09-14T06:29:26.028 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.028 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.028 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:29:26.029 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.836975+0000", 2024-09-14T06:29:26.029 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.849e-06 2024-09-14T06:29:26.029 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.029 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.029 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:29:26.029 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.836981+0000", 2024-09-14T06:29:26.029 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.0639999999999999e-06 2024-09-14T06:29:26.029 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.029 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.029 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:29:26.029 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.836990+0000", 2024-09-14T06:29:26.030 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.0219999999999993e-06 2024-09-14T06:29:26.030 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:26.030 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:26.030 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:26.030 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.030 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.030 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:12642 3.1e 3.302ef9fe (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:29:26.030 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:25.837614+0000", 2024-09-14T06:29:26.030 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 0.00042626300000000002, 2024-09-14T06:29:26.030 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0016709520000000001, 2024-09-14T06:29:26.030 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:29:26.030 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:29:26.030 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "queued for pg", 2024-09-14T06:29:26.030 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:29:26.030 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:29:26.030 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:26.030 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 12642 2024-09-14T06:29:26.031 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.031 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:29:26.031 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.031 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:29:26.031 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.837614+0000", 2024-09-14T06:29:26.031 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:26.031 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.031 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.031 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:29:26.031 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.837614+0000", 2024-09-14T06:29:26.031 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:26.031 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.031 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.031 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:29:26.031 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.837614+0000", 2024-09-14T06:29:26.031 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.9900000000000001e-07 2024-09-14T06:29:26.032 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.032 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.032 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:29:26.032 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.837616+0000", 2024-09-14T06:29:26.032 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.1210000000000001e-06 2024-09-14T06:29:26.032 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.032 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.032 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:29:26.032 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.837622+0000", 2024-09-14T06:29:26.032 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.3289999999999999e-06 2024-09-14T06:29:26.032 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.032 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.032 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:29:26.032 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.837631+0000", 2024-09-14T06:29:26.033 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.6670000000000002e-06 2024-09-14T06:29:26.033 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:26.033 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:26.033 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:26.033 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.033 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.033 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:12674 3.1e 3.302ef9fe (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:29:26.033 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:25.838173+0000", 2024-09-14T06:29:26.033 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 4294967295.999867, 2024-09-14T06:29:26.033 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.001136656, 2024-09-14T06:29:26.033 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:29:26.033 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:29:26.033 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "queued for pg", 2024-09-14T06:29:26.033 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:29:26.034 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:29:26.034 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:26.034 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 12674 2024-09-14T06:29:26.034 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.034 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:29:26.034 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.034 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:29:26.034 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.838173+0000", 2024-09-14T06:29:26.034 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:26.034 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.034 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.034 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:29:26.034 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.838173+0000", 2024-09-14T06:29:26.034 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:26.034 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.035 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.035 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:29:26.035 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.838173+0000", 2024-09-14T06:29:26.035 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.0800000000000001e-07 2024-09-14T06:29:26.035 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.035 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.035 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:29:26.035 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.838174+0000", 2024-09-14T06:29:26.035 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.3099999999999996e-07 2024-09-14T06:29:26.035 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.035 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.035 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:29:26.035 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.838177+0000", 2024-09-14T06:29:26.035 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.9560000000000002e-06 2024-09-14T06:29:26.035 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.035 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.036 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:29:26.036 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.838179+0000", 2024-09-14T06:29:26.036 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.4700000000000001e-06 2024-09-14T06:29:26.036 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:26.036 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:26.036 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:26.036 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.036 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.036 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:12579 3.1e 3.302ef9fe (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:29:26.037 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:25.836436+0000", 2024-09-14T06:29:26.037 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 0.001603745, 2024-09-14T06:29:26.037 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.002897092, 2024-09-14T06:29:26.037 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:29:26.037 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:29:26.037 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "queued for pg", 2024-09-14T06:29:26.037 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:29:26.037 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:29:26.037 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:26.037 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 12579 2024-09-14T06:29:26.037 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.037 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:29:26.037 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.038 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:29:26.038 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.836436+0000", 2024-09-14T06:29:26.038 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:26.038 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.038 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.038 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:29:26.038 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.836436+0000", 2024-09-14T06:29:26.038 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:26.038 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.038 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.038 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:29:26.038 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.836436+0000", 2024-09-14T06:29:26.038 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.04e-07 2024-09-14T06:29:26.039 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.039 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.039 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:29:26.039 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.836437+0000", 2024-09-14T06:29:26.039 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.2200000000000002e-07 2024-09-14T06:29:26.039 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.039 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.039 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:29:26.039 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.836441+0000", 2024-09-14T06:29:26.039 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.8340000000000001e-06 2024-09-14T06:29:26.039 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.039 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.039 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:29:26.039 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.836444+0000", 2024-09-14T06:29:26.039 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.6450000000000001e-06 2024-09-14T06:29:26.040 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:26.040 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:26.040 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:26.040 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.040 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.040 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:12611 3.1e 3.302ef9fe (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:29:26.040 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:25.836998+0000", 2024-09-14T06:29:26.040 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 0.0010416690000000001, 2024-09-14T06:29:26.040 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0023579450000000002, 2024-09-14T06:29:26.040 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:29:26.040 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:29:26.040 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "queued for pg", 2024-09-14T06:29:26.040 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:29:26.041 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:29:26.041 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:26.041 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 12611 2024-09-14T06:29:26.041 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.041 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:29:26.041 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.041 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:29:26.041 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.836998+0000", 2024-09-14T06:29:26.041 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:26.041 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.041 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.041 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:29:26.041 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.836998+0000", 2024-09-14T06:29:26.041 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:26.041 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.042 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.042 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:29:26.042 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.836999+0000", 2024-09-14T06:29:26.042 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.2199999999999999e-07 2024-09-14T06:29:26.042 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.042 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.042 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:29:26.042 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.837000+0000", 2024-09-14T06:29:26.042 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.79e-06 2024-09-14T06:29:26.042 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.042 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.042 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:29:26.042 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.837010+0000", 2024-09-14T06:29:26.042 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.8980000000000008e-06 2024-09-14T06:29:26.042 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.043 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.043 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:29:26.043 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.837023+0000", 2024-09-14T06:29:26.043 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.2942000000000001e-05 2024-09-14T06:29:26.043 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:26.043 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:26.043 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:26.043 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.043 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.043 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:12643 3.1e 3.302ef9fe (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:29:26.043 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:25.837662+0000", 2024-09-14T06:29:26.043 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 0.00037832399999999998, 2024-09-14T06:29:26.043 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.001718785, 2024-09-14T06:29:26.043 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:29:26.043 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:29:26.043 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "queued for pg", 2024-09-14T06:29:26.044 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:29:26.044 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:29:26.044 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:26.044 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 12643 2024-09-14T06:29:26.044 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.044 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:29:26.044 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.044 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:29:26.044 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.837662+0000", 2024-09-14T06:29:26.044 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:26.044 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.044 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.044 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:29:26.044 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.837662+0000", 2024-09-14T06:29:26.044 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:26.044 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.045 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.045 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:29:26.045 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.837662+0000", 2024-09-14T06:29:26.045 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.41e-07 2024-09-14T06:29:26.045 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.045 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.045 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:29:26.045 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.837664+0000", 2024-09-14T06:29:26.045 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.73e-06 2024-09-14T06:29:26.045 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.045 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.045 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:29:26.045 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.837671+0000", 2024-09-14T06:29:26.045 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.8089999999999993e-06 2024-09-14T06:29:26.046 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.046 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.046 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:29:26.046 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.837675+0000", 2024-09-14T06:29:26.046 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.8190000000000002e-06 2024-09-14T06:29:26.046 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:26.046 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:26.046 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:26.046 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.046 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.046 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:12675 3.1e 3.302ef9fe (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:29:26.046 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:25.838182+0000", 2024-09-14T06:29:26.046 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 4294967295.9998574, 2024-09-14T06:29:26.046 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00122255, 2024-09-14T06:29:26.047 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:29:26.047 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:29:26.047 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "queued for pg", 2024-09-14T06:29:26.047 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:29:26.047 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:29:26.047 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:26.047 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 12675 2024-09-14T06:29:26.047 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.047 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:29:26.047 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.048 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:29:26.048 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.838182+0000", 2024-09-14T06:29:26.048 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:26.048 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.048 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.048 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:29:26.048 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.838182+0000", 2024-09-14T06:29:26.048 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:26.048 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.048 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.048 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:29:26.048 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.838182+0000", 2024-09-14T06:29:26.049 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.0900000000000001e-07 2024-09-14T06:29:26.049 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.049 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.049 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:29:26.049 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.838183+0000", 2024-09-14T06:29:26.049 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.2900000000000002e-07 2024-09-14T06:29:26.049 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.049 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.049 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:29:26.049 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.838186+0000", 2024-09-14T06:29:26.049 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.9040000000000001e-06 2024-09-14T06:29:26.049 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.049 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.050 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:29:26.050 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.838189+0000", 2024-09-14T06:29:26.050 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.886e-06 2024-09-14T06:29:26.050 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:26.050 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:26.050 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:26.050 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.050 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.050 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:12548 3.1e 3:7f9f740c:::rbd_data.112d92027590.0000000000000004:f [read 374784~1024 [fadvise_nocache] out=1024b] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:29:26.050 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:25.830557+0000", 2024-09-14T06:29:26.050 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 0.0074827590000000003, 2024-09-14T06:29:26.050 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0088773740000000004, 2024-09-14T06:29:26.050 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:29:26.050 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:29:26.050 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:29:26.051 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:29:26.051 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:29:26.051 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:26.051 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 12548 2024-09-14T06:29:26.051 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.051 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:29:26.051 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.051 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:29:26.051 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.830557+0000", 2024-09-14T06:29:26.051 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:26.051 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.051 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.052 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:29:26.052 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.830557+0000", 2024-09-14T06:29:26.052 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:26.052 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.052 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.052 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:29:26.052 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.830557+0000", 2024-09-14T06:29:26.052 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.3099999999999999e-07 2024-09-14T06:29:26.052 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.052 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.052 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:29:26.052 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.830558+0000", 2024-09-14T06:29:26.053 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.2200000000000002e-07 2024-09-14T06:29:26.053 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.053 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.053 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:29:26.053 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.830561+0000", 2024-09-14T06:29:26.053 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.9100000000000001e-06 2024-09-14T06:29:26.053 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.053 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.053 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:29:26.053 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.830564+0000", 2024-09-14T06:29:26.053 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.497e-06 2024-09-14T06:29:26.053 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.053 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.054 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:29:26.054 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.839350+0000", 2024-09-14T06:29:26.054 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0087865070000000007 2024-09-14T06:29:26.054 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.054 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.054 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:29:26.054 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.839396+0000", 2024-09-14T06:29:26.054 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.5676000000000001e-05 2024-09-14T06:29:26.054 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:26.054 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:26.054 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:26.054 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.054 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.054 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:12580 3.1e 3.302ef9fe (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:29:26.054 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:25.836447+0000", 2024-09-14T06:29:26.054 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 0.001593045, 2024-09-14T06:29:26.055 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.003015378, 2024-09-14T06:29:26.055 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:29:26.055 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:29:26.055 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "queued for pg", 2024-09-14T06:29:26.055 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:29:26.055 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:29:26.055 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:26.055 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 12580 2024-09-14T06:29:26.055 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.055 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:29:26.055 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.055 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:29:26.055 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.836447+0000", 2024-09-14T06:29:26.055 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:26.055 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.056 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.056 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:29:26.056 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.836447+0000", 2024-09-14T06:29:26.056 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:26.056 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.056 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.056 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:29:26.056 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.836447+0000", 2024-09-14T06:29:26.056 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.0699999999999999e-07 2024-09-14T06:29:26.056 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.056 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.056 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:29:26.056 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.836448+0000", 2024-09-14T06:29:26.056 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.6099999999999999e-07 2024-09-14T06:29:26.056 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.056 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.057 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:29:26.057 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.836454+0000", 2024-09-14T06:29:26.057 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.5080000000000002e-06 2024-09-14T06:29:26.057 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.057 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.057 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:29:26.057 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.836457+0000", 2024-09-14T06:29:26.057 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.7020000000000001e-06 2024-09-14T06:29:26.057 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:26.057 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:26.057 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:26.057 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.057 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.058 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:12612 3.1e 3.302ef9fe (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:29:26.058 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:25.837033+0000", 2024-09-14T06:29:26.058 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 0.0010066439999999999, 2024-09-14T06:29:26.058 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.002453401, 2024-09-14T06:29:26.058 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:29:26.058 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:29:26.058 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "queued for pg", 2024-09-14T06:29:26.058 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:29:26.058 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:29:26.058 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:26.058 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 12612 2024-09-14T06:29:26.058 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.058 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:29:26.058 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.059 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:29:26.059 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.837033+0000", 2024-09-14T06:29:26.059 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:26.059 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.059 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.059 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:29:26.059 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.837033+0000", 2024-09-14T06:29:26.059 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:26.059 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.059 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.059 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:29:26.059 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.837034+0000", 2024-09-14T06:29:26.059 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.4799999999999999e-07 2024-09-14T06:29:26.059 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.059 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.060 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:29:26.060 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.837036+0000", 2024-09-14T06:29:26.060 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.0329999999999998e-06 2024-09-14T06:29:26.060 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.060 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.060 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:29:26.060 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.837044+0000", 2024-09-14T06:29:26.060 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.0350000000000001e-06 2024-09-14T06:29:26.060 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.060 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.060 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:29:26.060 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.837049+0000", 2024-09-14T06:29:26.060 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.8040000000000001e-06 2024-09-14T06:29:26.060 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:26.060 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:26.060 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:26.061 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.061 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.061 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:12644 3.1e 3.302ef9fe (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:29:26.061 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:25.837681+0000", 2024-09-14T06:29:26.061 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 0.000358845, 2024-09-14T06:29:26.061 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0018281720000000001, 2024-09-14T06:29:26.061 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:29:26.061 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:29:26.062 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "queued for pg", 2024-09-14T06:29:26.062 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:29:26.062 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:29:26.062 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:26.062 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 12644 2024-09-14T06:29:26.063 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.063 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:29:26.063 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.063 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:29:26.063 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.837681+0000", 2024-09-14T06:29:26.063 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:26.063 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.063 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.063 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:29:26.063 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.837681+0000", 2024-09-14T06:29:26.063 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:26.063 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.063 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.063 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:29:26.063 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.837681+0000", 2024-09-14T06:29:26.064 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.41e-07 2024-09-14T06:29:26.064 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.064 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.064 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:29:26.064 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.837682+0000", 2024-09-14T06:29:26.064 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.2190000000000001e-06 2024-09-14T06:29:26.064 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.064 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.064 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:29:26.064 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.837687+0000", 2024-09-14T06:29:26.064 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.0820000000000001e-06 2024-09-14T06:29:26.064 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.064 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.064 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:29:26.065 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.837689+0000", 2024-09-14T06:29:26.065 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.3970000000000001e-06 2024-09-14T06:29:26.065 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:26.065 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:26.065 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:26.065 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.065 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.065 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:12676 3.1e 3.302ef9fe (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:29:26.065 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:25.838192+0000", 2024-09-14T06:29:26.065 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 4294967295.9998479, 2024-09-14T06:29:26.065 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.001342876, 2024-09-14T06:29:26.065 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:29:26.065 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:29:26.065 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "queued for pg", 2024-09-14T06:29:26.066 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:29:26.066 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:29:26.066 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:26.066 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 12676 2024-09-14T06:29:26.066 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.066 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:29:26.066 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.066 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:29:26.066 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.838192+0000", 2024-09-14T06:29:26.066 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:26.066 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.066 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.066 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:29:26.066 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.838192+0000", 2024-09-14T06:29:26.066 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:26.067 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.067 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.067 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:29:26.067 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.838192+0000", 2024-09-14T06:29:26.067 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.0800000000000001e-07 2024-09-14T06:29:26.067 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.067 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.067 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:29:26.067 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.838193+0000", 2024-09-14T06:29:26.067 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.85e-07 2024-09-14T06:29:26.067 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.067 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.068 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:29:26.068 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.838196+0000", 2024-09-14T06:29:26.068 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.8870000000000001e-06 2024-09-14T06:29:26.068 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.068 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.068 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:29:26.068 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.838198+0000", 2024-09-14T06:29:26.068 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.3989999999999998e-06 2024-09-14T06:29:26.068 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:26.068 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:26.068 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:26.068 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.068 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.068 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:12549 3.1e 3:7f9f740c:::rbd_data.112d92027590.0000000000000004:f [read 385024~1024 [fadvise_nocache] out=1024b] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:29:26.069 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:25.830572+0000", 2024-09-14T06:29:26.069 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 0.0074677850000000002, 2024-09-14T06:29:26.069 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0089896669999999998, 2024-09-14T06:29:26.069 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:29:26.069 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:29:26.069 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:29:26.069 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:29:26.069 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:29:26.069 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:26.069 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 12549 2024-09-14T06:29:26.069 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.069 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:29:26.069 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.069 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:29:26.070 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.830572+0000", 2024-09-14T06:29:26.070 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:26.070 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.070 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.070 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:29:26.070 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.830572+0000", 2024-09-14T06:29:26.070 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:26.070 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.070 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.070 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:29:26.070 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.830572+0000", 2024-09-14T06:29:26.070 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.5899999999999998e-07 2024-09-14T06:29:26.070 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.070 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.071 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:29:26.071 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.830574+0000", 2024-09-14T06:29:26.071 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.2950000000000001e-06 2024-09-14T06:29:26.071 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.071 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.071 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:29:26.071 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.830579+0000", 2024-09-14T06:29:26.071 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.0710000000000002e-06 2024-09-14T06:29:26.071 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.071 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.071 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:29:26.071 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.830582+0000", 2024-09-14T06:29:26.071 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.982e-06 2024-09-14T06:29:26.071 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.071 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.071 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:29:26.072 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.839474+0000", 2024-09-14T06:29:26.072 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0088924079999999992 2024-09-14T06:29:26.072 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.072 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.072 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:29:26.072 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.839514+0000", 2024-09-14T06:29:26.072 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.9987999999999997e-05 2024-09-14T06:29:26.072 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:26.072 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:26.072 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:26.072 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.072 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.072 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:12581 3.1e 3.302ef9fe (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:29:26.072 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:25.836461+0000", 2024-09-14T06:29:26.072 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 0.0015790349999999999, 2024-09-14T06:29:26.073 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0031306160000000001, 2024-09-14T06:29:26.073 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:29:26.073 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:29:26.073 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "queued for pg", 2024-09-14T06:29:26.073 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:29:26.073 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:29:26.073 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:26.073 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 12581 2024-09-14T06:29:26.073 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.073 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:29:26.073 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.073 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:29:26.073 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.836461+0000", 2024-09-14T06:29:26.073 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:26.073 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.073 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.074 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:29:26.074 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.836461+0000", 2024-09-14T06:29:26.074 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:26.074 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.074 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.074 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:29:26.074 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.836461+0000", 2024-09-14T06:29:26.074 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.05e-07 2024-09-14T06:29:26.074 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.074 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.074 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:29:26.074 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.836462+0000", 2024-09-14T06:29:26.074 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.4249999999999999e-06 2024-09-14T06:29:26.074 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.074 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.075 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:29:26.075 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.836467+0000", 2024-09-14T06:29:26.075 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.1779999999999996e-06 2024-09-14T06:29:26.075 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.075 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.075 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:29:26.075 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.836469+0000", 2024-09-14T06:29:26.075 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.5679999999999998e-06 2024-09-14T06:29:26.075 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:26.075 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:26.075 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:26.075 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.075 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.075 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:12613 3.1e 3.302ef9fe (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:29:26.075 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:25.837056+0000", 2024-09-14T06:29:26.075 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 0.000983996, 2024-09-14T06:29:26.076 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.002558243, 2024-09-14T06:29:26.076 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:29:26.076 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:29:26.076 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "queued for pg", 2024-09-14T06:29:26.076 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:29:26.076 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:29:26.076 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:26.076 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 12613 2024-09-14T06:29:26.076 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.076 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:29:26.076 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.076 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:29:26.076 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.837056+0000", 2024-09-14T06:29:26.076 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:26.076 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.077 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.077 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:29:26.077 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.837056+0000", 2024-09-14T06:29:26.077 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:26.077 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.077 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.077 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:29:26.077 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.837056+0000", 2024-09-14T06:29:26.077 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.7599999999999998e-07 2024-09-14T06:29:26.077 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.077 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.077 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:29:26.077 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.837061+0000", 2024-09-14T06:29:26.078 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.8770000000000001e-06 2024-09-14T06:29:26.078 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.078 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.078 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:29:26.078 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.837071+0000", 2024-09-14T06:29:26.078 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.0188e-05 2024-09-14T06:29:26.078 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.078 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.078 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:29:26.078 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.837076+0000", 2024-09-14T06:29:26.078 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.8049999999999997e-06 2024-09-14T06:29:26.078 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:26.078 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:26.079 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:26.079 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.079 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.079 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:12645 3.1e 3.302ef9fe (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:29:26.079 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:25.837692+0000", 2024-09-14T06:29:26.079 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 0.00034752399999999999, 2024-09-14T06:29:26.079 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.001945748, 2024-09-14T06:29:26.079 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:29:26.079 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:29:26.079 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "queued for pg", 2024-09-14T06:29:26.079 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:29:26.079 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:29:26.079 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:26.079 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 12645 2024-09-14T06:29:26.080 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.080 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:29:26.080 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.080 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:29:26.080 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.837692+0000", 2024-09-14T06:29:26.080 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:26.080 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.080 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.080 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:29:26.080 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.837692+0000", 2024-09-14T06:29:26.080 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:26.080 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.080 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.080 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:29:26.080 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.837693+0000", 2024-09-14T06:29:26.081 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.6800000000000002e-07 2024-09-14T06:29:26.081 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.081 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.081 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:29:26.081 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.837694+0000", 2024-09-14T06:29:26.081 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.9999999999999995e-07 2024-09-14T06:29:26.081 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.081 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.081 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:29:26.081 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.837697+0000", 2024-09-14T06:29:26.081 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.937e-06 2024-09-14T06:29:26.081 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.081 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.081 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:29:26.081 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.837699+0000", 2024-09-14T06:29:26.082 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.3709999999999998e-06 2024-09-14T06:29:26.082 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:26.082 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:26.082 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:26.082 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.082 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.082 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:12677 3.1e 3.302ef9fe (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:29:26.082 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:25.838202+0000", 2024-09-14T06:29:26.082 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 4294967295.9998384, 2024-09-14T06:29:26.082 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.001459697, 2024-09-14T06:29:26.082 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:29:26.082 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:29:26.082 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "queued for pg", 2024-09-14T06:29:26.082 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:29:26.082 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:29:26.082 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:26.083 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 12677 2024-09-14T06:29:26.083 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.083 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:29:26.083 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.083 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:29:26.083 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.838202+0000", 2024-09-14T06:29:26.083 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:26.083 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.083 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.083 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:29:26.083 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.838202+0000", 2024-09-14T06:29:26.083 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:26.083 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.083 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.084 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:29:26.084 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.838202+0000", 2024-09-14T06:29:26.084 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.04e-07 2024-09-14T06:29:26.084 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.084 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.084 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:29:26.084 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.838203+0000", 2024-09-14T06:29:26.084 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.2600000000000001e-07 2024-09-14T06:29:26.084 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.084 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.084 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:29:26.084 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.838206+0000", 2024-09-14T06:29:26.084 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.0070000000000002e-06 2024-09-14T06:29:26.084 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.085 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.085 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:29:26.085 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.838208+0000", 2024-09-14T06:29:26.085 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.5129999999999999e-06 2024-09-14T06:29:26.085 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:26.085 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:26.085 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:26.085 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.085 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.085 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:12550 3.1e 3:7f9f740c:::rbd_data.112d92027590.0000000000000004:f [read 395264~1024 [fadvise_nocache] out=1024b] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:29:26.085 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:25.830585+0000", 2024-09-14T06:29:26.085 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 0.0074543869999999998, 2024-09-14T06:29:26.085 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0090975899999999991, 2024-09-14T06:29:26.085 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:29:26.086 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:29:26.086 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:29:26.086 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:29:26.086 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:29:26.086 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:26.086 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 12550 2024-09-14T06:29:26.086 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.086 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:29:26.086 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.086 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:29:26.086 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.830585+0000", 2024-09-14T06:29:26.087 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:26.087 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.087 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.087 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:29:26.087 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.830585+0000", 2024-09-14T06:29:26.087 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:26.087 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.087 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.087 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:29:26.087 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.830586+0000", 2024-09-14T06:29:26.087 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.28e-07 2024-09-14T06:29:26.087 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.088 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.088 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:29:26.088 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.830587+0000", 2024-09-14T06:29:26.088 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.249e-06 2024-09-14T06:29:26.088 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.088 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.088 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:29:26.088 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.830590+0000", 2024-09-14T06:29:26.088 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.0290000000000001e-06 2024-09-14T06:29:26.088 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.088 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.088 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:29:26.089 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.830593+0000", 2024-09-14T06:29:26.089 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.57e-06 2024-09-14T06:29:26.089 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.089 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.089 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:29:26.089 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.839586+0000", 2024-09-14T06:29:26.089 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0089930989999999992 2024-09-14T06:29:26.089 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.089 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.089 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:29:26.090 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.839632+0000", 2024-09-14T06:29:26.090 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.5948999999999997e-05 2024-09-14T06:29:26.090 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.090 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.090 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:29:26.090 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.839683+0000", 2024-09-14T06:29:26.090 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.1465999999999998e-05 2024-09-14T06:29:26.090 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:26.090 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:26.090 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:26.090 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.090 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.090 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:12582 3.1e 3.302ef9fe (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:29:26.091 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:25.836472+0000", 2024-09-14T06:29:26.091 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 0.001567375, 2024-09-14T06:29:26.091 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0032478490000000001, 2024-09-14T06:29:26.091 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:29:26.091 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:29:26.091 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "queued for pg", 2024-09-14T06:29:26.091 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:29:26.091 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:29:26.091 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:26.091 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 12582 2024-09-14T06:29:26.091 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.091 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:29:26.091 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.091 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:29:26.091 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.836472+0000", 2024-09-14T06:29:26.092 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:26.092 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.092 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.092 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:29:26.092 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.836472+0000", 2024-09-14T06:29:26.092 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:26.092 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.092 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.092 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:29:26.092 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.836473+0000", 2024-09-14T06:29:26.092 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.3999999999999998e-07 2024-09-14T06:29:26.093 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.093 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.093 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:29:26.093 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.836474+0000", 2024-09-14T06:29:26.093 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.8899999999999998e-07 2024-09-14T06:29:26.093 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.093 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.093 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:29:26.093 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.836481+0000", 2024-09-14T06:29:26.093 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.939e-06 2024-09-14T06:29:26.093 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.093 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.093 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:29:26.093 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.836487+0000", 2024-09-14T06:29:26.094 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.0499999999999997e-06 2024-09-14T06:29:26.094 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:26.094 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:26.094 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:26.094 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.094 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.094 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:12614 3.1e 3.302ef9fe (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:29:26.094 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:25.837089+0000", 2024-09-14T06:29:26.094 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 0.00095109600000000002, 2024-09-14T06:29:26.094 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0026689510000000001, 2024-09-14T06:29:26.094 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:29:26.094 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:29:26.094 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "queued for pg", 2024-09-14T06:29:26.094 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:29:26.094 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:29:26.095 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:26.095 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 12614 2024-09-14T06:29:26.095 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.095 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:29:26.095 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.095 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:29:26.095 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.837089+0000", 2024-09-14T06:29:26.095 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:26.095 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.095 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.095 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:29:26.095 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.837089+0000", 2024-09-14T06:29:26.095 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:26.095 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.095 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.095 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:29:26.096 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.837089+0000", 2024-09-14T06:29:26.096 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.0200000000000003e-07 2024-09-14T06:29:26.096 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.096 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.096 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:29:26.096 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.837091+0000", 2024-09-14T06:29:26.096 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.968e-06 2024-09-14T06:29:26.096 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.096 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.096 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:29:26.096 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.837099+0000", 2024-09-14T06:29:26.096 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.8169999999999999e-06 2024-09-14T06:29:26.096 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.096 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.096 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:29:26.096 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.837104+0000", 2024-09-14T06:29:26.097 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.1109999999999999e-06 2024-09-14T06:29:26.097 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:26.097 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:26.097 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:26.097 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.097 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.097 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:12646 3.1e 3.302ef9fe (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:29:26.097 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:25.837702+0000", 2024-09-14T06:29:26.097 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 0.00033778900000000001, 2024-09-14T06:29:26.097 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.002078473, 2024-09-14T06:29:26.097 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:29:26.097 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:29:26.097 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "queued for pg", 2024-09-14T06:29:26.097 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:29:26.097 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:29:26.098 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:26.098 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 12646 2024-09-14T06:29:26.098 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.098 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:29:26.098 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.098 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:29:26.098 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.837702+0000", 2024-09-14T06:29:26.098 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:26.098 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.098 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.098 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:29:26.098 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.837702+0000", 2024-09-14T06:29:26.098 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:26.099 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.099 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.099 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:29:26.099 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.837702+0000", 2024-09-14T06:29:26.099 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.11e-07 2024-09-14T06:29:26.099 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.099 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.099 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:29:26.099 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.837704+0000", 2024-09-14T06:29:26.099 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.438e-06 2024-09-14T06:29:26.099 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.099 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.099 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:29:26.099 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.837711+0000", 2024-09-14T06:29:26.099 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.0670000000000003e-06 2024-09-14T06:29:26.100 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.100 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.100 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:29:26.100 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.837716+0000", 2024-09-14T06:29:26.100 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.5249999999999996e-06 2024-09-14T06:29:26.100 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:26.100 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:26.100 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:26.100 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.100 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.100 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:12678 3.1e 3.302ef9fe (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:29:26.100 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:25.838211+0000", 2024-09-14T06:29:26.100 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 4294967295.9998288, 2024-09-14T06:29:26.100 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0015958649999999999, 2024-09-14T06:29:26.101 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:29:26.101 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:29:26.101 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "queued for pg", 2024-09-14T06:29:26.101 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:29:26.101 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:29:26.101 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:26.101 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 12678 2024-09-14T06:29:26.101 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.101 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:29:26.101 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.101 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:29:26.101 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.838211+0000", 2024-09-14T06:29:26.101 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:26.101 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.101 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.102 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:29:26.102 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.838211+0000", 2024-09-14T06:29:26.102 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:26.102 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.102 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.102 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:29:26.102 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.838211+0000", 2024-09-14T06:29:26.102 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.0599999999999999e-07 2024-09-14T06:29:26.102 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.102 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.102 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:29:26.102 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.838212+0000", 2024-09-14T06:29:26.102 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.5300000000000002e-07 2024-09-14T06:29:26.102 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.102 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.103 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:29:26.103 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.838215+0000", 2024-09-14T06:29:26.103 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.8940000000000002e-06 2024-09-14T06:29:26.103 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.103 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.103 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:29:26.103 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.838218+0000", 2024-09-14T06:29:26.103 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.5639999999999999e-06 2024-09-14T06:29:26.103 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:26.103 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:26.103 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:26.103 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.103 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.103 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:12583 3.1e 3.302ef9fe (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:29:26.104 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:25.836493+0000", 2024-09-14T06:29:26.104 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 0.0015470919999999999, 2024-09-14T06:29:26.104 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0033382160000000002, 2024-09-14T06:29:26.104 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:29:26.104 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:29:26.104 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "queued for pg", 2024-09-14T06:29:26.104 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:29:26.104 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:29:26.104 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:26.104 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 12583 2024-09-14T06:29:26.104 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.104 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:29:26.104 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.104 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:29:26.105 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.836493+0000", 2024-09-14T06:29:26.105 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:26.105 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.105 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.105 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:29:26.105 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.836493+0000", 2024-09-14T06:29:26.105 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:26.105 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.105 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.105 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:29:26.105 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.836493+0000", 2024-09-14T06:29:26.105 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.06e-07 2024-09-14T06:29:26.105 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.105 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.106 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:29:26.106 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.836495+0000", 2024-09-14T06:29:26.106 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.838e-06 2024-09-14T06:29:26.106 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.106 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.106 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:29:26.106 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.836501+0000", 2024-09-14T06:29:26.106 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.682e-06 2024-09-14T06:29:26.106 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.106 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.106 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:29:26.106 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.836505+0000", 2024-09-14T06:29:26.106 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.6369999999999998e-06 2024-09-14T06:29:26.106 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:26.107 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:26.107 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:26.107 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.107 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.107 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:12615 3.1e 3.302ef9fe (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:29:26.107 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:25.837111+0000", 2024-09-14T06:29:26.107 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 0.00092905400000000001, 2024-09-14T06:29:26.107 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0027430850000000001, 2024-09-14T06:29:26.107 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:29:26.107 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:29:26.107 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "queued for pg", 2024-09-14T06:29:26.107 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:29:26.107 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:29:26.107 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:26.108 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 12615 2024-09-14T06:29:26.108 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.108 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:29:26.108 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.108 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:29:26.108 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.837111+0000", 2024-09-14T06:29:26.108 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:26.108 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.108 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.108 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:29:26.108 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.837111+0000", 2024-09-14T06:29:26.108 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:26.109 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.109 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.109 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:29:26.109 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.837111+0000", 2024-09-14T06:29:26.109 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.9400000000000001e-07 2024-09-14T06:29:26.109 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.109 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.109 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:29:26.109 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.837113+0000", 2024-09-14T06:29:26.109 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.821e-06 2024-09-14T06:29:26.109 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.109 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.109 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:29:26.109 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.837119+0000", 2024-09-14T06:29:26.110 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.6099999999999997e-06 2024-09-14T06:29:26.110 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.110 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.110 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:29:26.110 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.837123+0000", 2024-09-14T06:29:26.110 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.1629999999999998e-06 2024-09-14T06:29:26.110 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:26.110 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:26.110 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:26.110 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.110 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.110 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:12647 3.1e 3.302ef9fe (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:29:26.110 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:25.837720+0000", 2024-09-14T06:29:26.111 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 0.00031972899999999999, 2024-09-14T06:29:26.111 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0021562529999999999, 2024-09-14T06:29:26.111 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:29:26.111 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:29:26.111 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "queued for pg", 2024-09-14T06:29:26.111 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:29:26.111 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:29:26.111 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:26.111 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 12647 2024-09-14T06:29:26.111 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.111 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:29:26.111 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.112 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:29:26.112 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.837720+0000", 2024-09-14T06:29:26.112 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:26.112 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.112 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.112 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:29:26.112 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.837720+0000", 2024-09-14T06:29:26.112 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:26.112 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.112 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.112 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:29:26.112 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.837720+0000", 2024-09-14T06:29:26.112 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.1299999999999999e-07 2024-09-14T06:29:26.112 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.113 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.113 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:29:26.113 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.837721+0000", 2024-09-14T06:29:26.113 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.0330000000000001e-06 2024-09-14T06:29:26.113 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.113 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.113 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:29:26.113 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.837724+0000", 2024-09-14T06:29:26.113 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.0639999999999998e-06 2024-09-14T06:29:26.113 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.113 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.113 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:29:26.113 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.837727+0000", 2024-09-14T06:29:26.113 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.571e-06 2024-09-14T06:29:26.113 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:26.114 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:26.114 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:26.114 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.114 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.114 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:12679 3.1e 3.302ef9fe (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:29:26.114 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:25.838222+0000", 2024-09-14T06:29:26.114 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 4294967295.9998183, 2024-09-14T06:29:26.114 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0016780269999999999, 2024-09-14T06:29:26.114 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:29:26.114 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:29:26.114 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "queued for pg", 2024-09-14T06:29:26.114 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:29:26.114 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:29:26.114 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:26.115 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 12679 2024-09-14T06:29:26.115 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.115 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:29:26.115 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.115 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:29:26.115 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.838222+0000", 2024-09-14T06:29:26.115 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:26.115 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.115 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.115 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:29:26.115 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.838222+0000", 2024-09-14T06:29:26.115 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:26.115 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.115 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.115 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:29:26.116 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.838222+0000", 2024-09-14T06:29:26.116 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.6899999999999998e-07 2024-09-14T06:29:26.116 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.116 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.116 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:29:26.116 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.838224+0000", 2024-09-14T06:29:26.116 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.0080000000000001e-06 2024-09-14T06:29:26.116 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.116 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.116 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:29:26.116 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.838228+0000", 2024-09-14T06:29:26.116 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.963e-06 2024-09-14T06:29:26.116 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.116 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.116 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:29:26.117 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.838232+0000", 2024-09-14T06:29:26.117 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.5000000000000001e-06 2024-09-14T06:29:26.117 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:26.117 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:26.117 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:26.117 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.117 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.117 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:12584 3.1e 3.302ef9fe (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:29:26.117 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:25.836511+0000", 2024-09-14T06:29:26.117 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 0.001528569, 2024-09-14T06:29:26.117 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.003411791, 2024-09-14T06:29:26.117 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:29:26.117 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:29:26.117 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "queued for pg", 2024-09-14T06:29:26.118 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:29:26.118 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:29:26.118 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:26.118 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 12584 2024-09-14T06:29:26.118 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.118 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:29:26.118 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.118 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:29:26.118 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.836511+0000", 2024-09-14T06:29:26.118 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:26.118 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.118 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.118 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:29:26.119 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.836511+0000", 2024-09-14T06:29:26.119 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:26.119 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.119 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.119 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:29:26.119 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.836512+0000", 2024-09-14T06:29:26.119 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.5999999999999999e-07 2024-09-14T06:29:26.119 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.119 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.119 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:29:26.119 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.836514+0000", 2024-09-14T06:29:26.119 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.2929999999999999e-06 2024-09-14T06:29:26.119 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.119 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.120 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:29:26.120 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.836519+0000", 2024-09-14T06:29:26.120 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.2950000000000001e-06 2024-09-14T06:29:26.120 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.120 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.120 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:29:26.120 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.836525+0000", 2024-09-14T06:29:26.120 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.7019999999999998e-06 2024-09-14T06:29:26.120 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:26.120 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:26.120 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:26.120 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.120 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.121 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:12616 3.1e 3.302ef9fe (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:29:26.121 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:25.837130+0000", 2024-09-14T06:29:26.121 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 0.00090938900000000003, 2024-09-14T06:29:26.121 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.002815587, 2024-09-14T06:29:26.121 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:29:26.121 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:29:26.121 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "queued for pg", 2024-09-14T06:29:26.121 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:29:26.121 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:29:26.121 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:26.121 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 12616 2024-09-14T06:29:26.121 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.121 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:29:26.121 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.122 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:29:26.122 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.837130+0000", 2024-09-14T06:29:26.122 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:26.122 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.122 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.122 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:29:26.122 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.837130+0000", 2024-09-14T06:29:26.122 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:26.122 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.122 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.122 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:29:26.122 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.837131+0000", 2024-09-14T06:29:26.122 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.58e-07 2024-09-14T06:29:26.122 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.123 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.123 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:29:26.123 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.837133+0000", 2024-09-14T06:29:26.123 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.714e-06 2024-09-14T06:29:26.123 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.123 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.123 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:29:26.123 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.837143+0000", 2024-09-14T06:29:26.123 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.0108999999999999e-05 2024-09-14T06:29:26.123 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.123 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.124 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:29:26.124 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.837148+0000", 2024-09-14T06:29:26.124 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.8309999999999997e-06 2024-09-14T06:29:26.124 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:26.124 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:26.124 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:26.124 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.124 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.124 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:12648 3.1e 3.302ef9fe (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:29:26.124 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:25.837730+0000", 2024-09-14T06:29:26.124 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 0.00030980500000000002, 2024-09-14T06:29:26.124 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.002239432, 2024-09-14T06:29:26.124 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:29:26.124 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:29:26.125 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "queued for pg", 2024-09-14T06:29:26.125 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:29:26.125 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:29:26.125 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:26.125 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 12648 2024-09-14T06:29:26.125 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.125 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:29:26.125 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.125 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:29:26.125 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.837730+0000", 2024-09-14T06:29:26.125 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:26.125 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.125 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.125 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:29:26.125 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.837730+0000", 2024-09-14T06:29:26.126 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:26.126 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.126 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.126 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:29:26.126 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.837730+0000", 2024-09-14T06:29:26.126 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.0800000000000001e-07 2024-09-14T06:29:26.126 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.126 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.126 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:29:26.126 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.837731+0000", 2024-09-14T06:29:26.126 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.78e-07 2024-09-14T06:29:26.126 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.126 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.126 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:29:26.126 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.837734+0000", 2024-09-14T06:29:26.126 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.965e-06 2024-09-14T06:29:26.127 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.127 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.127 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:29:26.127 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.837737+0000", 2024-09-14T06:29:26.127 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.475e-06 2024-09-14T06:29:26.127 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:26.127 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:26.127 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:26.127 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.127 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.127 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:12680 3.1e 3.302ef9fe (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:29:26.127 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:25.838244+0000", 2024-09-14T06:29:26.127 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 4294967295.9997959, 2024-09-14T06:29:26.127 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.001749308, 2024-09-14T06:29:26.127 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:29:26.128 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:29:26.128 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "queued for pg", 2024-09-14T06:29:26.128 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:29:26.128 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:29:26.128 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:26.128 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 12680 2024-09-14T06:29:26.128 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.128 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:29:26.128 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.128 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:29:26.128 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.838244+0000", 2024-09-14T06:29:26.128 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:26.129 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.129 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.129 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:29:26.129 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.838244+0000", 2024-09-14T06:29:26.129 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:26.129 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.129 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.129 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:29:26.129 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.838244+0000", 2024-09-14T06:29:26.129 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.4000000000000002e-07 2024-09-14T06:29:26.129 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.129 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.129 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:29:26.130 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.838246+0000", 2024-09-14T06:29:26.130 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.77e-06 2024-09-14T06:29:26.130 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.130 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.130 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:29:26.130 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.838252+0000", 2024-09-14T06:29:26.130 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.4060000000000004e-06 2024-09-14T06:29:26.130 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.130 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.130 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:29:26.130 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.838258+0000", 2024-09-14T06:29:26.130 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.1589999999999998e-06 2024-09-14T06:29:26.130 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:26.130 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:26.130 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:26.131 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.131 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.131 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:12553 3.1e 3:7f9f740c:::rbd_data.112d92027590.0000000000000004:f [read 425984~1024 [fadvise_nocache] out=1024b] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:29:26.131 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:25.830659+0000", 2024-09-14T06:29:26.131 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 0.0073812080000000002, 2024-09-14T06:29:26.131 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0093608939999999998, 2024-09-14T06:29:26.131 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:29:26.131 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:29:26.131 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:29:26.131 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:29:26.131 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:29:26.131 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:26.131 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 12553 2024-09-14T06:29:26.131 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.131 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:29:26.131 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.132 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:29:26.132 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.830659+0000", 2024-09-14T06:29:26.132 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:26.132 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.132 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.132 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:29:26.132 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.830659+0000", 2024-09-14T06:29:26.132 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:26.132 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.132 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.132 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:29:26.132 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.830775+0000", 2024-09-14T06:29:26.132 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000116377 2024-09-14T06:29:26.132 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.132 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.133 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:29:26.133 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.830791+0000", 2024-09-14T06:29:26.133 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.6112000000000001e-05 2024-09-14T06:29:26.133 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.133 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.133 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:29:26.133 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.830796+0000", 2024-09-14T06:29:26.133 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.9300000000000002e-06 2024-09-14T06:29:26.133 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.133 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.133 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:29:26.133 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.830799+0000", 2024-09-14T06:29:26.133 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.1870000000000001e-06 2024-09-14T06:29:26.133 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.133 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.134 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:29:26.134 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.839932+0000", 2024-09-14T06:29:26.134 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0091324720000000009 2024-09-14T06:29:26.134 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.134 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.134 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:29:26.134 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.839981+0000", 2024-09-14T06:29:26.134 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.9261999999999997e-05 2024-09-14T06:29:26.134 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:26.134 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:26.134 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:26.134 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.134 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.134 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:12585 3.1e 3.302ef9fe (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:29:26.134 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:25.836532+0000", 2024-09-14T06:29:26.134 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 0.0015075659999999999, 2024-09-14T06:29:26.135 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0035145570000000002, 2024-09-14T06:29:26.135 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:29:26.135 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:29:26.135 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "queued for pg", 2024-09-14T06:29:26.135 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:29:26.135 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:29:26.135 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:26.135 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 12585 2024-09-14T06:29:26.135 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.135 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:29:26.135 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.135 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:29:26.135 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.836532+0000", 2024-09-14T06:29:26.135 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:26.136 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.136 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.136 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:29:26.136 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.836532+0000", 2024-09-14T06:29:26.136 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:26.136 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.136 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.136 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:29:26.136 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.836533+0000", 2024-09-14T06:29:26.136 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.4700000000000002e-07 2024-09-14T06:29:26.137 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.137 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.137 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:29:26.137 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.836534+0000", 2024-09-14T06:29:26.137 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.8109999999999999e-06 2024-09-14T06:29:26.137 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.137 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.137 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:29:26.137 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.836543+0000", 2024-09-14T06:29:26.137 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.4859999999999997e-06 2024-09-14T06:29:26.137 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.137 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.137 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:29:26.137 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.836549+0000", 2024-09-14T06:29:26.138 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.7250000000000002e-06 2024-09-14T06:29:26.138 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:26.138 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:26.138 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:26.138 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.138 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.138 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:12617 3.1e 3.302ef9fe (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:29:26.138 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:25.837156+0000", 2024-09-14T06:29:26.138 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 0.00088371800000000003, 2024-09-14T06:29:26.138 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0029133459999999998, 2024-09-14T06:29:26.138 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:29:26.138 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:29:26.138 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "queued for pg", 2024-09-14T06:29:26.138 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:29:26.138 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:29:26.138 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:26.139 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 12617 2024-09-14T06:29:26.139 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.139 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:29:26.139 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.139 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:29:26.139 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.837156+0000", 2024-09-14T06:29:26.139 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:26.139 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.139 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.139 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:29:26.139 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.837156+0000", 2024-09-14T06:29:26.139 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:26.139 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.140 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.140 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:29:26.140 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.837156+0000", 2024-09-14T06:29:26.140 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.8099999999999998e-07 2024-09-14T06:29:26.140 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.140 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.140 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:29:26.140 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.837164+0000", 2024-09-14T06:29:26.140 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.3710000000000002e-06 2024-09-14T06:29:26.140 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.140 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.140 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:29:26.140 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.837174+0000", 2024-09-14T06:29:26.140 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.8079999999999999e-06 2024-09-14T06:29:26.140 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.141 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.141 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:29:26.141 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.837182+0000", 2024-09-14T06:29:26.141 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.4379999999999995e-06 2024-09-14T06:29:26.141 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:26.141 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:26.141 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:26.141 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.141 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.141 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:12649 3.1e 3.302ef9fe (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:29:26.141 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:25.837742+0000", 2024-09-14T06:29:26.141 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 0.00029770600000000001, 2024-09-14T06:29:26.141 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.002350221, 2024-09-14T06:29:26.141 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:29:26.142 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:29:26.142 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "queued for pg", 2024-09-14T06:29:26.142 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:29:26.142 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:29:26.142 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:26.142 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 12649 2024-09-14T06:29:26.142 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.142 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:29:26.142 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.142 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:29:26.142 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.837742+0000", 2024-09-14T06:29:26.142 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:26.142 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.142 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.142 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:29:26.143 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.837742+0000", 2024-09-14T06:29:26.143 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:26.143 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.143 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.143 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:29:26.143 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.837742+0000", 2024-09-14T06:29:26.143 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.4600000000000001e-07 2024-09-14T06:29:26.143 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.143 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.143 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:29:26.143 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.837744+0000", 2024-09-14T06:29:26.143 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.176e-06 2024-09-14T06:29:26.143 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.143 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.143 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:29:26.143 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.837747+0000", 2024-09-14T06:29:26.144 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.4800000000000001e-06 2024-09-14T06:29:26.144 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.144 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.144 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:29:26.144 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.837750+0000", 2024-09-14T06:29:26.144 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.5500000000000001e-06 2024-09-14T06:29:26.144 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:26.144 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:26.144 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:26.144 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.144 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.144 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:12681 3.1e 3.302ef9fe (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:29:26.144 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:25.838264+0000", 2024-09-14T06:29:26.144 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 4294967295.9997759, 2024-09-14T06:29:26.144 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0018522370000000001, 2024-09-14T06:29:26.145 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:29:26.145 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:29:26.145 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "queued for pg", 2024-09-14T06:29:26.145 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:29:26.145 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:29:26.145 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:26.145 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 12681 2024-09-14T06:29:26.145 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.145 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:29:26.145 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.145 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:29:26.145 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.838264+0000", 2024-09-14T06:29:26.145 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:26.145 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.145 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.145 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:29:26.146 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.838264+0000", 2024-09-14T06:29:26.146 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:26.146 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.146 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.146 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:29:26.146 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.838265+0000", 2024-09-14T06:29:26.146 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.5299999999999999e-07 2024-09-14T06:29:26.146 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.146 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.146 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:29:26.146 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.838266+0000", 2024-09-14T06:29:26.146 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.6670000000000001e-06 2024-09-14T06:29:26.146 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.146 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.146 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:29:26.146 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.838272+0000", 2024-09-14T06:29:26.147 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.4349999999999996e-06 2024-09-14T06:29:26.147 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.147 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.147 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:29:26.147 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.838276+0000", 2024-09-14T06:29:26.147 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.6890000000000003e-06 2024-09-14T06:29:26.147 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:26.147 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:26.147 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:26.147 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.147 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.148 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:12554 3.1e 3:7f9f740c:::rbd_data.112d92027590.0000000000000004:f [read 436224~1024 [fadvise_nocache] out=1024b] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:29:26.148 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:25.830814+0000", 2024-09-14T06:29:26.148 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 0.0072260140000000002, 2024-09-14T06:29:26.148 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0093288099999999999, 2024-09-14T06:29:26.148 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:29:26.148 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:29:26.148 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:29:26.148 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:29:26.148 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:29:26.148 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:26.148 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 12554 2024-09-14T06:29:26.148 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.149 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:29:26.149 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.149 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:29:26.149 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.830814+0000", 2024-09-14T06:29:26.149 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:26.149 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.149 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.149 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:29:26.149 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.830814+0000", 2024-09-14T06:29:26.149 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:26.150 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.150 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.150 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:29:26.150 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.830814+0000", 2024-09-14T06:29:26.150 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.7099999999999997e-07 2024-09-14T06:29:26.150 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.150 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.150 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:29:26.150 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.830816+0000", 2024-09-14T06:29:26.150 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.418e-06 2024-09-14T06:29:26.150 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.150 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.150 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:29:26.150 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.830821+0000", 2024-09-14T06:29:26.151 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.5539999999999997e-06 2024-09-14T06:29:26.151 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.151 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.151 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:29:26.151 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.830824+0000", 2024-09-14T06:29:26.151 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.8219999999999998e-06 2024-09-14T06:29:26.151 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.151 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.151 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:29:26.151 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.840061+0000", 2024-09-14T06:29:26.151 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0092365780000000005 2024-09-14T06:29:26.151 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.151 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.151 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:29:26.152 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.840099+0000", 2024-09-14T06:29:26.152 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.8665999999999999e-05 2024-09-14T06:29:26.152 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:26.152 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:26.152 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:26.152 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.152 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.152 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:12586 3.1e 3.302ef9fe (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:29:26.152 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:25.836555+0000", 2024-09-14T06:29:26.152 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 0.001484469, 2024-09-14T06:29:26.153 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0036157120000000001, 2024-09-14T06:29:26.153 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:29:26.153 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:29:26.153 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "queued for pg", 2024-09-14T06:29:26.153 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:29:26.153 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:29:26.153 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:26.153 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 12586 2024-09-14T06:29:26.153 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.153 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:29:26.153 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.153 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:29:26.153 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.836555+0000", 2024-09-14T06:29:26.154 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:26.154 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.154 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.154 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:29:26.154 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.836555+0000", 2024-09-14T06:29:26.154 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:26.154 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.154 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.154 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:29:26.154 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.836556+0000", 2024-09-14T06:29:26.154 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.72e-07 2024-09-14T06:29:26.154 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.154 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.154 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:29:26.155 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.836557+0000", 2024-09-14T06:29:26.155 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.7570000000000001e-06 2024-09-14T06:29:26.155 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.155 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.155 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:29:26.155 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.836565+0000", 2024-09-14T06:29:26.155 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.0539999999999997e-06 2024-09-14T06:29:26.155 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.155 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.155 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:29:26.155 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.836569+0000", 2024-09-14T06:29:26.155 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.656e-06 2024-09-14T06:29:26.155 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:26.155 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:26.156 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:26.156 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.156 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.156 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:12618 3.1e 3.302ef9fe (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:29:26.156 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:25.837218+0000", 2024-09-14T06:29:26.156 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 0.00082198799999999995, 2024-09-14T06:29:26.156 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0029758509999999998, 2024-09-14T06:29:26.156 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:29:26.156 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:29:26.156 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "queued for pg", 2024-09-14T06:29:26.156 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:29:26.156 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:29:26.156 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:26.156 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 12618 2024-09-14T06:29:26.157 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.157 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:29:26.157 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.157 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:29:26.157 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.837218+0000", 2024-09-14T06:29:26.157 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:26.157 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.157 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.157 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:29:26.157 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.837218+0000", 2024-09-14T06:29:26.157 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:26.157 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.157 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.158 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:29:26.158 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.837218+0000", 2024-09-14T06:29:26.158 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.3500000000000002e-07 2024-09-14T06:29:26.158 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.158 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.158 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:29:26.158 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.837220+0000", 2024-09-14T06:29:26.158 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.7439999999999999e-06 2024-09-14T06:29:26.158 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.158 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.158 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:29:26.158 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.837228+0000", 2024-09-14T06:29:26.158 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.4800000000000004e-06 2024-09-14T06:29:26.158 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.159 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.159 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:29:26.159 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.837231+0000", 2024-09-14T06:29:26.159 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.0180000000000002e-06 2024-09-14T06:29:26.159 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:26.159 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:26.159 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:26.159 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.159 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.159 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:12650 3.1e 3.302ef9fe (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:29:26.159 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:25.837753+0000", 2024-09-14T06:29:26.159 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 0.00028690999999999998, 2024-09-14T06:29:26.159 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0024634290000000001, 2024-09-14T06:29:26.160 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:29:26.160 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:29:26.160 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "queued for pg", 2024-09-14T06:29:26.160 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:29:26.160 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:29:26.160 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:26.160 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 12650 2024-09-14T06:29:26.160 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.160 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:29:26.160 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.160 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:29:26.160 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.837753+0000", 2024-09-14T06:29:26.160 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:26.161 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.161 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.161 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:29:26.161 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.837753+0000", 2024-09-14T06:29:26.161 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:26.161 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.161 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.161 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:29:26.161 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.837753+0000", 2024-09-14T06:29:26.161 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.0900000000000001e-07 2024-09-14T06:29:26.161 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.161 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.161 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:29:26.162 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.837755+0000", 2024-09-14T06:29:26.162 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.432e-06 2024-09-14T06:29:26.162 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.162 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.162 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:29:26.162 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.837758+0000", 2024-09-14T06:29:26.162 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.0699999999999998e-06 2024-09-14T06:29:26.162 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.162 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.162 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:29:26.162 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.837760+0000", 2024-09-14T06:29:26.162 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.5119999999999998e-06 2024-09-14T06:29:26.163 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:26.163 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:26.163 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:26.163 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.163 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.163 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:12682 3.1e 3.302ef9fe (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:29:26.163 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:25.838285+0000", 2024-09-14T06:29:26.163 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 4294967295.9997544, 2024-09-14T06:29:26.163 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0019547610000000002, 2024-09-14T06:29:26.163 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:29:26.163 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:29:26.163 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "queued for pg", 2024-09-14T06:29:26.163 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:29:26.163 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:29:26.163 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:26.164 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 12682 2024-09-14T06:29:26.164 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.164 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:29:26.164 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.164 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:29:26.164 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.838285+0000", 2024-09-14T06:29:26.164 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:26.164 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.164 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.164 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:29:26.164 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.838285+0000", 2024-09-14T06:29:26.164 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:26.164 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.164 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.164 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:29:26.165 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.838286+0000", 2024-09-14T06:29:26.165 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.8400000000000005e-07 2024-09-14T06:29:26.165 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.165 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.165 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:29:26.165 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.838287+0000", 2024-09-14T06:29:26.165 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.705e-06 2024-09-14T06:29:26.165 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.165 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.165 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:29:26.165 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.838300+0000", 2024-09-14T06:29:26.165 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.3042e-05 2024-09-14T06:29:26.165 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.166 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.166 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:29:26.166 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.838314+0000", 2024-09-14T06:29:26.166 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.3181e-05 2024-09-14T06:29:26.166 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:26.166 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:26.166 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:26.166 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.166 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.166 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:12555 3.1e 3:7f9f740c:::rbd_data.112d92027590.0000000000000004:f [read 446464~1024 [fadvise_nocache] out=1024b] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:29:26.166 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:25.830852+0000", 2024-09-14T06:29:26.166 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 0.0071882639999999998, 2024-09-14T06:29:26.166 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0094122489999999993, 2024-09-14T06:29:26.166 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:29:26.167 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:29:26.167 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:29:26.167 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:29:26.167 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:29:26.167 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:26.167 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 12555 2024-09-14T06:29:26.167 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.167 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:29:26.167 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.167 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:29:26.167 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.830852+0000", 2024-09-14T06:29:26.167 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:26.167 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.167 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.167 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:29:26.168 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.830852+0000", 2024-09-14T06:29:26.168 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:26.168 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.168 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.168 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:29:26.168 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.830852+0000", 2024-09-14T06:29:26.168 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.6000000000000004e-07 2024-09-14T06:29:26.168 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.168 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.168 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:29:26.168 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.830854+0000", 2024-09-14T06:29:26.168 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.9530000000000002e-06 2024-09-14T06:29:26.168 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.169 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.169 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:29:26.169 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.830860+0000", 2024-09-14T06:29:26.169 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.015e-06 2024-09-14T06:29:26.169 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.169 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.169 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:29:26.169 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.830865+0000", 2024-09-14T06:29:26.169 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.6040000000000003e-06 2024-09-14T06:29:26.169 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.169 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.169 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:29:26.169 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.840171+0000", 2024-09-14T06:29:26.169 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0093061210000000005 2024-09-14T06:29:26.170 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.170 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.170 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:29:26.170 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.840212+0000", 2024-09-14T06:29:26.170 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.1149999999999997e-05 2024-09-14T06:29:26.170 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.171 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.171 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:29:26.171 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.840264+0000", 2024-09-14T06:29:26.171 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.1845999999999998e-05 2024-09-14T06:29:26.171 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:26.171 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:26.172 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:26.172 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.172 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.172 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:12587 3.1e 3.302ef9fe (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:29:26.172 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:25.836579+0000", 2024-09-14T06:29:26.172 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 0.001460409, 2024-09-14T06:29:26.173 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0037174410000000001, 2024-09-14T06:29:26.173 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:29:26.173 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:29:26.173 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "queued for pg", 2024-09-14T06:29:26.173 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:29:26.173 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:29:26.174 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:26.174 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 12587 2024-09-14T06:29:26.174 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.174 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:29:26.174 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.174 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:29:26.174 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.836579+0000", 2024-09-14T06:29:26.175 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:26.175 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.175 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.175 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:29:26.175 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.836579+0000", 2024-09-14T06:29:26.175 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:26.175 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.176 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.176 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:29:26.176 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.836580+0000", 2024-09-14T06:29:26.176 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.89e-07 2024-09-14T06:29:26.176 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.177 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.177 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:29:26.177 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.836582+0000", 2024-09-14T06:29:26.177 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.7010000000000001e-06 2024-09-14T06:29:26.177 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.177 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.178 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:29:26.178 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.836589+0000", 2024-09-14T06:29:26.178 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.4610000000000002e-06 2024-09-14T06:29:26.178 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.178 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.178 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:29:26.179 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.836596+0000", 2024-09-14T06:29:26.179 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.7349999999999999e-06 2024-09-14T06:29:26.179 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:26.179 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:26.179 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:26.179 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.180 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.180 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:12619 3.1e 3.302ef9fe (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:29:26.180 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:25.837235+0000", 2024-09-14T06:29:26.180 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 0.00080505599999999996, 2024-09-14T06:29:26.180 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0030855380000000001, 2024-09-14T06:29:26.180 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:29:26.181 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:29:26.181 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "queued for pg", 2024-09-14T06:29:26.181 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:29:26.181 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:29:26.181 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:26.182 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 12619 2024-09-14T06:29:26.182 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.182 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:29:26.182 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.182 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:29:26.182 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.837235+0000", 2024-09-14T06:29:26.183 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:26.183 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.183 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.183 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:29:26.183 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.837235+0000", 2024-09-14T06:29:26.183 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:26.184 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.184 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.184 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:29:26.184 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.837235+0000", 2024-09-14T06:29:26.184 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.1500000000000001e-07 2024-09-14T06:29:26.184 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.185 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.185 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:29:26.185 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.837236+0000", 2024-09-14T06:29:26.185 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.173e-06 2024-09-14T06:29:26.185 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.185 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.186 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:29:26.186 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.837240+0000", 2024-09-14T06:29:26.186 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.2259999999999999e-06 2024-09-14T06:29:26.186 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.186 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.186 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:29:26.187 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.837243+0000", 2024-09-14T06:29:26.187 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.481e-06 2024-09-14T06:29:26.187 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:26.187 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:26.187 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:26.187 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.187 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.187 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:12651 3.1e 3.302ef9fe (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:29:26.188 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:25.837763+0000", 2024-09-14T06:29:26.188 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 0.00027667000000000002, 2024-09-14T06:29:26.188 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0025799130000000001, 2024-09-14T06:29:26.188 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:29:26.188 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:29:26.188 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "queued for pg", 2024-09-14T06:29:26.189 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:29:26.189 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:29:26.189 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:26.189 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 12651 2024-09-14T06:29:26.189 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.189 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:29:26.189 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.190 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:29:26.190 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.837763+0000", 2024-09-14T06:29:26.190 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:26.190 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.190 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.190 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:29:26.190 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.837763+0000", 2024-09-14T06:29:26.190 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:26.190 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.190 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.190 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:29:26.190 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.837763+0000", 2024-09-14T06:29:26.190 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.29e-07 2024-09-14T06:29:26.190 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.191 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.191 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:29:26.191 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.837764+0000", 2024-09-14T06:29:26.191 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.0299999999999998e-07 2024-09-14T06:29:26.191 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.191 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.191 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:29:26.191 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.837767+0000", 2024-09-14T06:29:26.191 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.8899999999999999e-06 2024-09-14T06:29:26.191 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.191 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.191 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:29:26.191 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.837769+0000", 2024-09-14T06:29:26.192 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.4080000000000001e-06 2024-09-14T06:29:26.192 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:26.192 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:26.192 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:26.192 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.192 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.192 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:12683 3.1e 3.302ef9fe (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:29:26.192 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:25.838321+0000", 2024-09-14T06:29:26.192 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 4294967295.9997187, 2024-09-14T06:29:26.192 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.002045172, 2024-09-14T06:29:26.192 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:29:26.192 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:29:26.192 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "queued for pg", 2024-09-14T06:29:26.192 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:29:26.193 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:29:26.193 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:26.193 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 12683 2024-09-14T06:29:26.193 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.193 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:29:26.193 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.193 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:29:26.193 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.838321+0000", 2024-09-14T06:29:26.193 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:26.193 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.193 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.193 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:29:26.194 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.838321+0000", 2024-09-14T06:29:26.194 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:26.194 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.194 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.194 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:29:26.194 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.838322+0000", 2024-09-14T06:29:26.194 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.2399999999999999e-07 2024-09-14T06:29:26.194 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.194 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.194 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:29:26.194 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.838329+0000", 2024-09-14T06:29:26.194 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.1339999999999998e-06 2024-09-14T06:29:26.194 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.194 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.195 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:29:26.195 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.838336+0000", 2024-09-14T06:29:26.195 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.5140000000000004e-06 2024-09-14T06:29:26.195 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.195 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.195 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:29:26.195 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.838344+0000", 2024-09-14T06:29:26.195 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.8380000000000002e-06 2024-09-14T06:29:26.195 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:26.195 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:26.195 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:26.195 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.195 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.195 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:12588 3.1e 3.302ef9fe (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:29:26.196 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:25.836605+0000", 2024-09-14T06:29:26.196 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 0.001434619, 2024-09-14T06:29:26.196 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0037854550000000001, 2024-09-14T06:29:26.196 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:29:26.196 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:29:26.196 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "queued for pg", 2024-09-14T06:29:26.196 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:29:26.196 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:29:26.196 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:26.196 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 12588 2024-09-14T06:29:26.196 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.196 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:29:26.196 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.196 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:29:26.197 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.836605+0000", 2024-09-14T06:29:26.197 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:26.197 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.197 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.197 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:29:26.197 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.836605+0000", 2024-09-14T06:29:26.197 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:26.197 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.197 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.197 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:29:26.197 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.836606+0000", 2024-09-14T06:29:26.197 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.1100000000000001e-07 2024-09-14T06:29:26.197 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.198 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.198 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:29:26.198 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.836608+0000", 2024-09-14T06:29:26.198 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.9360000000000002e-06 2024-09-14T06:29:26.198 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.198 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.198 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:29:26.198 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.836613+0000", 2024-09-14T06:29:26.198 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.9290000000000003e-06 2024-09-14T06:29:26.198 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.198 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.198 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:29:26.198 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.836618+0000", 2024-09-14T06:29:26.198 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.6360000000000002e-06 2024-09-14T06:29:26.199 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:26.199 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:26.199 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:26.199 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.199 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.199 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:12620 3.1e 3.302ef9fe (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:29:26.199 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:25.837246+0000", 2024-09-14T06:29:26.199 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 0.00079375599999999995, 2024-09-14T06:29:26.199 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0031672919999999999, 2024-09-14T06:29:26.199 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:29:26.199 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:29:26.199 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "queued for pg", 2024-09-14T06:29:26.199 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:29:26.199 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:29:26.200 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:26.200 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 12620 2024-09-14T06:29:26.200 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.200 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:29:26.200 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.200 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:29:26.200 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.837246+0000", 2024-09-14T06:29:26.200 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:26.200 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.200 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.200 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:29:26.200 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.837246+0000", 2024-09-14T06:29:26.200 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:26.200 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.201 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.201 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:29:26.201 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.837246+0000", 2024-09-14T06:29:26.201 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.1600000000000002e-07 2024-09-14T06:29:26.201 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.201 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.201 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:29:26.201 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.837247+0000", 2024-09-14T06:29:26.201 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.037e-06 2024-09-14T06:29:26.201 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.201 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.201 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:29:26.201 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.837251+0000", 2024-09-14T06:29:26.201 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.5039999999999998e-06 2024-09-14T06:29:26.202 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.202 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.202 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:29:26.202 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.837257+0000", 2024-09-14T06:29:26.202 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.2809999999999997e-06 2024-09-14T06:29:26.202 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:26.202 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:26.202 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:26.202 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.202 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.202 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:12652 3.1e 3.302ef9fe (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:29:26.202 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:25.837772+0000", 2024-09-14T06:29:26.202 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 0.00026735099999999998, 2024-09-14T06:29:26.202 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0026638339999999999, 2024-09-14T06:29:26.203 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:29:26.203 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:29:26.203 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "queued for pg", 2024-09-14T06:29:26.203 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:29:26.203 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:29:26.203 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:26.203 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 12652 2024-09-14T06:29:26.203 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.203 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:29:26.203 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.203 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:29:26.203 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.837772+0000", 2024-09-14T06:29:26.203 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:26.203 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.203 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.204 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:29:26.204 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.837772+0000", 2024-09-14T06:29:26.204 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:26.204 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.204 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.204 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:29:26.204 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.837773+0000", 2024-09-14T06:29:26.204 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.1500000000000001e-07 2024-09-14T06:29:26.204 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.204 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.204 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:29:26.205 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.837774+0000", 2024-09-14T06:29:26.205 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.6700000000000002e-07 2024-09-14T06:29:26.205 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.205 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.205 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:29:26.205 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.837780+0000", 2024-09-14T06:29:26.205 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.579e-06 2024-09-14T06:29:26.205 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.205 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.205 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:29:26.205 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.837783+0000", 2024-09-14T06:29:26.205 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.6630000000000002e-06 2024-09-14T06:29:26.205 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:26.205 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:26.206 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:26.206 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.206 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.206 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:12684 3.1e 3.302ef9fe (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:29:26.206 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:25.838352+0000", 2024-09-14T06:29:26.206 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 4294967295.9996877, 2024-09-14T06:29:26.206 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0021074259999999999, 2024-09-14T06:29:26.206 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:29:26.206 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:29:26.206 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "queued for pg", 2024-09-14T06:29:26.206 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:29:26.206 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:29:26.206 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:26.206 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 12684 2024-09-14T06:29:26.206 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.207 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:29:26.207 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.207 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:29:26.207 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.838352+0000", 2024-09-14T06:29:26.207 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:26.207 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.207 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.207 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:29:26.207 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.838352+0000", 2024-09-14T06:29:26.207 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:26.207 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.207 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.207 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:29:26.207 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.838353+0000", 2024-09-14T06:29:26.208 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.96e-07 2024-09-14T06:29:26.208 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.208 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.208 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:29:26.208 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.838357+0000", 2024-09-14T06:29:26.208 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.6770000000000003e-06 2024-09-14T06:29:26.208 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.208 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.208 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:29:26.208 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.838363+0000", 2024-09-14T06:29:26.208 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.3149999999999997e-06 2024-09-14T06:29:26.208 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.208 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.208 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:29:26.209 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.838368+0000", 2024-09-14T06:29:26.209 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.5909999999999998e-06 2024-09-14T06:29:26.209 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:26.209 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:26.209 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:26.209 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.209 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.209 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:12589 3.1e 3.302ef9fe (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:29:26.209 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:25.836624+0000", 2024-09-14T06:29:26.209 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 0.001415578, 2024-09-14T06:29:26.209 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0038587679999999998, 2024-09-14T06:29:26.209 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:29:26.209 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:29:26.209 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "queued for pg", 2024-09-14T06:29:26.210 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:29:26.210 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:29:26.210 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:26.210 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 12589 2024-09-14T06:29:26.210 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.210 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:29:26.210 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.210 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:29:26.210 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.836624+0000", 2024-09-14T06:29:26.210 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:26.210 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.210 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.210 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:29:26.210 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.836624+0000", 2024-09-14T06:29:26.210 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:26.211 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.211 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.211 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:29:26.211 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.836625+0000", 2024-09-14T06:29:26.211 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.4999999999999998e-07 2024-09-14T06:29:26.211 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.211 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.211 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:29:26.211 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.836629+0000", 2024-09-14T06:29:26.211 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.6700000000000002e-06 2024-09-14T06:29:26.211 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.211 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.211 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:29:26.211 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.836635+0000", 2024-09-14T06:29:26.211 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.4700000000000001e-06 2024-09-14T06:29:26.212 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.212 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.212 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:29:26.212 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.836640+0000", 2024-09-14T06:29:26.212 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.6470000000000003e-06 2024-09-14T06:29:26.212 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:26.212 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:26.212 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:26.212 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.212 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.212 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:12621 3.1e 3.302ef9fe (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:29:26.212 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:25.837261+0000", 2024-09-14T06:29:26.212 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 0.00077919800000000004, 2024-09-14T06:29:26.212 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0032447259999999999, 2024-09-14T06:29:26.213 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:29:26.213 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:29:26.213 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "queued for pg", 2024-09-14T06:29:26.213 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:29:26.213 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:29:26.213 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:26.213 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 12621 2024-09-14T06:29:26.213 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.213 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:29:26.213 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.213 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:29:26.213 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.837261+0000", 2024-09-14T06:29:26.213 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:26.213 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.213 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.214 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:29:26.214 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.837261+0000", 2024-09-14T06:29:26.214 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:26.214 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.214 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.214 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:29:26.214 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.837261+0000", 2024-09-14T06:29:26.214 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.0900000000000001e-07 2024-09-14T06:29:26.214 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.214 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.214 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:29:26.214 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.837262+0000", 2024-09-14T06:29:26.214 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.057e-06 2024-09-14T06:29:26.214 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.215 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.215 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:29:26.215 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.837267+0000", 2024-09-14T06:29:26.215 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.9710000000000003e-06 2024-09-14T06:29:26.215 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.215 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.215 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:29:26.215 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.837269+0000", 2024-09-14T06:29:26.215 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.531e-06 2024-09-14T06:29:26.215 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:26.215 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:26.215 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:26.215 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.215 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.216 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:12653 3.1e 3.302ef9fe (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:29:26.216 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:25.837787+0000", 2024-09-14T06:29:26.216 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 0.00025303599999999999, 2024-09-14T06:29:26.216 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.002740925, 2024-09-14T06:29:26.216 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:29:26.216 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:29:26.216 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "queued for pg", 2024-09-14T06:29:26.216 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:29:26.216 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:29:26.216 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:26.216 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 12653 2024-09-14T06:29:26.216 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.216 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:29:26.216 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.217 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:29:26.217 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.837787+0000", 2024-09-14T06:29:26.217 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:26.217 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.217 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.217 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:29:26.217 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.837787+0000", 2024-09-14T06:29:26.217 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:26.217 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.217 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.217 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:29:26.217 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.837787+0000", 2024-09-14T06:29:26.217 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.16e-07 2024-09-14T06:29:26.217 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.218 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.218 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:29:26.218 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.837788+0000", 2024-09-14T06:29:26.218 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.0049999999999999e-06 2024-09-14T06:29:26.218 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.218 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.218 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:29:26.218 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.837792+0000", 2024-09-14T06:29:26.218 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.9310000000000001e-06 2024-09-14T06:29:26.218 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.218 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.218 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:29:26.218 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.837795+0000", 2024-09-14T06:29:26.218 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.5969999999999999e-06 2024-09-14T06:29:26.219 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:26.219 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:26.219 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:26.219 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.219 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.219 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:12686 3.1e 3.302ef9fe (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:29:26.219 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:25.838376+0000", 2024-09-14T06:29:26.219 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 4294967295.9996638, 2024-09-14T06:29:26.219 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.002174956, 2024-09-14T06:29:26.219 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:29:26.219 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:29:26.219 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "queued for pg", 2024-09-14T06:29:26.219 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:29:26.219 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:29:26.219 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:26.220 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 12686 2024-09-14T06:29:26.220 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.220 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:29:26.220 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.220 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:29:26.220 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.838376+0000", 2024-09-14T06:29:26.220 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:26.220 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.220 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.221 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:29:26.221 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.838376+0000", 2024-09-14T06:29:26.221 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:26.221 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.221 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.221 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:29:26.221 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.838376+0000", 2024-09-14T06:29:26.221 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.9400000000000001e-07 2024-09-14T06:29:26.222 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.222 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.222 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:29:26.222 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.838378+0000", 2024-09-14T06:29:26.222 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.7689999999999999e-06 2024-09-14T06:29:26.222 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.222 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.222 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:29:26.222 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.838384+0000", 2024-09-14T06:29:26.223 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.4729999999999999e-06 2024-09-14T06:29:26.223 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.223 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.223 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:29:26.223 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.838394+0000", 2024-09-14T06:29:26.223 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.0164e-05 2024-09-14T06:29:26.223 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:26.223 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:26.223 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:26.224 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.224 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.224 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:12558 3.1e 3:7f9f740c:::rbd_data.112d92027590.0000000000000004:f [read 477184~1024 [fadvise_nocache] out=1024b] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:29:26.224 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:25.830983+0000", 2024-09-14T06:29:26.224 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 0.0070564870000000002, 2024-09-14T06:29:26.224 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0095937929999999998, 2024-09-14T06:29:26.224 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:29:26.224 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:29:26.225 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:29:26.225 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:29:26.225 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:29:26.225 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:26.225 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 12558 2024-09-14T06:29:26.225 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.225 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:29:26.225 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.225 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:29:26.226 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.830983+0000", 2024-09-14T06:29:26.226 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:26.226 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.226 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.226 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:29:26.226 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.830983+0000", 2024-09-14T06:29:26.226 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:26.226 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.226 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.227 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:29:26.227 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.830984+0000", 2024-09-14T06:29:26.227 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.15e-07 2024-09-14T06:29:26.227 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.227 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.227 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:29:26.227 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.830985+0000", 2024-09-14T06:29:26.227 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.564e-06 2024-09-14T06:29:26.228 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.228 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.228 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:29:26.228 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.830993+0000", 2024-09-14T06:29:26.228 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.8930000000000005e-06 2024-09-14T06:29:26.228 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.228 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.228 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:29:26.228 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.830999+0000", 2024-09-14T06:29:26.229 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.1970000000000001e-06 2024-09-14T06:29:26.229 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.229 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.229 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:29:26.229 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.840504+0000", 2024-09-14T06:29:26.229 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.009504696 2024-09-14T06:29:26.229 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.229 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.229 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:29:26.230 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.840548+0000", 2024-09-14T06:29:26.230 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.4035000000000001e-05 2024-09-14T06:29:26.230 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:26.230 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:26.230 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:26.230 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.230 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.230 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:12590 3.1e 3.302ef9fe (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:29:26.230 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:25.836647+0000", 2024-09-14T06:29:26.231 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 0.0013929509999999999, 2024-09-14T06:29:26.231 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0039577070000000004, 2024-09-14T06:29:26.231 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:29:26.231 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:29:26.231 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "queued for pg", 2024-09-14T06:29:26.231 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:29:26.231 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:29:26.231 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:26.231 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 12590 2024-09-14T06:29:26.232 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.232 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:29:26.232 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.232 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:29:26.232 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.836647+0000", 2024-09-14T06:29:26.232 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:26.232 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.232 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.232 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:29:26.232 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.836647+0000", 2024-09-14T06:29:26.232 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:26.232 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.232 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.232 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:29:26.233 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.836647+0000", 2024-09-14T06:29:26.233 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.6899999999999998e-07 2024-09-14T06:29:26.233 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.233 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.233 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:29:26.233 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.836649+0000", 2024-09-14T06:29:26.233 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.7489999999999999e-06 2024-09-14T06:29:26.233 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.233 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.233 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:29:26.233 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.836655+0000", 2024-09-14T06:29:26.233 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.5310000000000001e-06 2024-09-14T06:29:26.233 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.233 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.233 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:29:26.234 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.836660+0000", 2024-09-14T06:29:26.234 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.6980000000000003e-06 2024-09-14T06:29:26.234 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:26.234 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:26.234 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:26.234 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.234 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.234 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:12622 3.1e 3.302ef9fe (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:29:26.234 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:25.837273+0000", 2024-09-14T06:29:26.234 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 0.00076727999999999996, 2024-09-14T06:29:26.234 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0033545110000000001, 2024-09-14T06:29:26.234 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:29:26.234 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:29:26.234 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "queued for pg", 2024-09-14T06:29:26.234 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:29:26.235 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:29:26.235 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:26.235 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 12622 2024-09-14T06:29:26.235 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.235 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:29:26.235 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.235 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:29:26.235 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.837273+0000", 2024-09-14T06:29:26.235 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:26.235 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.235 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.236 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:29:26.236 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.837273+0000", 2024-09-14T06:29:26.236 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:26.236 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.236 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.236 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:29:26.236 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.837273+0000", 2024-09-14T06:29:26.236 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.1199999999999999e-07 2024-09-14T06:29:26.236 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.236 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.236 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:29:26.236 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.837274+0000", 2024-09-14T06:29:26.236 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.6599999999999994e-07 2024-09-14T06:29:26.237 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.237 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.237 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:29:26.237 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.837277+0000", 2024-09-14T06:29:26.237 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.0029999999999999e-06 2024-09-14T06:29:26.237 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.237 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.237 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:29:26.237 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.837279+0000", 2024-09-14T06:29:26.237 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.407e-06 2024-09-14T06:29:26.237 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:26.237 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:26.237 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:26.238 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.238 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.238 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:12654 3.1e 3.302ef9fe (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:29:26.238 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:25.837798+0000", 2024-09-14T06:29:26.238 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 0.00024212499999999999, 2024-09-14T06:29:26.238 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0028519800000000001, 2024-09-14T06:29:26.238 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:29:26.238 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:29:26.238 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "queued for pg", 2024-09-14T06:29:26.238 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:29:26.238 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:29:26.238 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:26.239 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 12654 2024-09-14T06:29:26.239 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.239 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:29:26.239 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.239 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:29:26.239 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.837798+0000", 2024-09-14T06:29:26.239 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:26.239 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.239 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.239 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:29:26.239 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.837798+0000", 2024-09-14T06:29:26.239 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:26.239 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.239 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.240 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:29:26.240 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.837798+0000", 2024-09-14T06:29:26.240 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.0800000000000001e-07 2024-09-14T06:29:26.240 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.240 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.240 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:29:26.240 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.837799+0000", 2024-09-14T06:29:26.240 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.2600000000000001e-07 2024-09-14T06:29:26.240 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.240 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.240 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:29:26.240 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.837803+0000", 2024-09-14T06:29:26.240 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.9700000000000001e-06 2024-09-14T06:29:26.240 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.240 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.241 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:29:26.241 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.837810+0000", 2024-09-14T06:29:26.241 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.4800000000000004e-06 2024-09-14T06:29:26.241 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:26.241 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:26.241 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:26.241 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.241 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.241 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:12687 3.1e 3.302ef9fe (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:29:26.241 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:25.838408+0000", 2024-09-14T06:29:26.241 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 4294967295.9996314, 2024-09-14T06:29:26.241 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0022647380000000001, 2024-09-14T06:29:26.241 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:29:26.241 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:29:26.242 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "queued for pg", 2024-09-14T06:29:26.242 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:29:26.242 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:29:26.242 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:26.242 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 12687 2024-09-14T06:29:26.242 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.242 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:29:26.242 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.242 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:29:26.242 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.838408+0000", 2024-09-14T06:29:26.242 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:26.242 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.242 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.242 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:29:26.243 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.838408+0000", 2024-09-14T06:29:26.243 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:26.243 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.243 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.243 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:29:26.243 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.838409+0000", 2024-09-14T06:29:26.243 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.2199999999999999e-07 2024-09-14T06:29:26.243 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.243 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.243 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:29:26.243 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.838411+0000", 2024-09-14T06:29:26.243 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.024e-06 2024-09-14T06:29:26.243 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.243 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.243 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:29:26.243 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.838417+0000", 2024-09-14T06:29:26.244 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.0249999999999999e-06 2024-09-14T06:29:26.244 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.244 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.244 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:29:26.244 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.838421+0000", 2024-09-14T06:29:26.244 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.7609999999999998e-06 2024-09-14T06:29:26.244 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:26.244 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:26.244 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:26.244 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.244 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.244 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:12559 3.1e 3:7f9f740c:::rbd_data.112d92027590.0000000000000004:f [read 487424~1024 [fadvise_nocache] out=1024b] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:29:26.244 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:25.831004+0000", 2024-09-14T06:29:26.244 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 0.0070359480000000002, 2024-09-14T06:29:26.245 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0096950990000000004, 2024-09-14T06:29:26.245 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:29:26.245 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:29:26.245 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:29:26.245 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:29:26.245 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:29:26.245 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:26.245 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 12559 2024-09-14T06:29:26.245 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.245 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:29:26.245 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.245 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:29:26.245 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.831004+0000", 2024-09-14T06:29:26.245 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:26.245 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.246 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.246 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:29:26.246 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.831004+0000", 2024-09-14T06:29:26.246 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:26.246 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.246 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.246 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:29:26.246 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.831004+0000", 2024-09-14T06:29:26.246 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.1400000000000001e-07 2024-09-14T06:29:26.246 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.246 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.246 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:29:26.246 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.831005+0000", 2024-09-14T06:29:26.246 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.265e-06 2024-09-14T06:29:26.246 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.247 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.247 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:29:26.247 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.831014+0000", 2024-09-14T06:29:26.247 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.1200000000000008e-06 2024-09-14T06:29:26.247 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.247 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.247 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:29:26.247 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.831019+0000", 2024-09-14T06:29:26.247 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.8539999999999996e-06 2024-09-14T06:29:26.247 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.247 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.247 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:29:26.247 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.840630+0000", 2024-09-14T06:29:26.247 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0096110009999999992 2024-09-14T06:29:26.248 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.248 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.248 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:29:26.248 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.840669+0000", 2024-09-14T06:29:26.248 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.8915000000000003e-05 2024-09-14T06:29:26.248 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:26.248 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:26.248 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:26.248 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.248 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.248 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:12591 3.1e 3.302ef9fe (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:29:26.249 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:25.836666+0000", 2024-09-14T06:29:26.249 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 0.001373684, 2024-09-14T06:29:26.249 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0040612419999999996, 2024-09-14T06:29:26.249 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:29:26.249 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:29:26.249 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "queued for pg", 2024-09-14T06:29:26.249 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:29:26.249 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:29:26.249 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:26.249 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 12591 2024-09-14T06:29:26.250 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.250 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:29:26.250 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.250 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:29:26.250 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.836666+0000", 2024-09-14T06:29:26.250 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:26.250 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.250 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.250 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:29:26.250 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.836666+0000", 2024-09-14T06:29:26.250 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:26.250 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.250 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.251 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:29:26.251 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.836666+0000", 2024-09-14T06:29:26.251 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.4999999999999998e-07 2024-09-14T06:29:26.251 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.251 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.251 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:29:26.251 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.836668+0000", 2024-09-14T06:29:26.251 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.705e-06 2024-09-14T06:29:26.251 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.251 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.251 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:29:26.251 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.836675+0000", 2024-09-14T06:29:26.251 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.725e-06 2024-09-14T06:29:26.251 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.251 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.252 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:29:26.252 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.836679+0000", 2024-09-14T06:29:26.252 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.3800000000000004e-06 2024-09-14T06:29:26.252 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:26.252 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:26.252 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:26.252 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.252 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.252 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:12623 3.1e 3.302ef9fe (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:29:26.252 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:25.837282+0000", 2024-09-14T06:29:26.252 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 0.00075766199999999998, 2024-09-14T06:29:26.253 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.003467918, 2024-09-14T06:29:26.253 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:29:26.253 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:29:26.253 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "queued for pg", 2024-09-14T06:29:26.253 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:29:26.253 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:29:26.253 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:26.253 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 12623 2024-09-14T06:29:26.253 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.253 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:29:26.253 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.253 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:29:26.253 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.837282+0000", 2024-09-14T06:29:26.253 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:26.254 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.254 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.254 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:29:26.254 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.837282+0000", 2024-09-14T06:29:26.254 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:26.254 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.254 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.254 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:29:26.254 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.837282+0000", 2024-09-14T06:29:26.254 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.0800000000000001e-07 2024-09-14T06:29:26.254 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.254 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.254 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:29:26.254 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.837283+0000", 2024-09-14T06:29:26.255 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.0100000000000003e-07 2024-09-14T06:29:26.255 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.255 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.255 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:29:26.255 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.837286+0000", 2024-09-14T06:29:26.255 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.8849999999999999e-06 2024-09-14T06:29:26.255 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.255 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.255 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:29:26.255 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.837288+0000", 2024-09-14T06:29:26.255 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.328e-06 2024-09-14T06:29:26.255 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:26.255 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:26.255 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:26.255 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.256 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.256 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:12655 3.1e 3.302ef9fe (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:29:26.256 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:25.837818+0000", 2024-09-14T06:29:26.256 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 0.000221852, 2024-09-14T06:29:26.256 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.002955545, 2024-09-14T06:29:26.256 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:29:26.256 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:29:26.256 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "queued for pg", 2024-09-14T06:29:26.256 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:29:26.256 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:29:26.256 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:26.256 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 12655 2024-09-14T06:29:26.256 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.256 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:29:26.257 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.257 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:29:26.257 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.837818+0000", 2024-09-14T06:29:26.257 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:26.257 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.257 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.257 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:29:26.257 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.837818+0000", 2024-09-14T06:29:26.257 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:26.257 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.257 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.257 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:29:26.257 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.837818+0000", 2024-09-14T06:29:26.258 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.41e-07 2024-09-14T06:29:26.258 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.258 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.258 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:29:26.258 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.837820+0000", 2024-09-14T06:29:26.258 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.4050000000000001e-06 2024-09-14T06:29:26.258 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.258 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.258 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:29:26.258 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.837824+0000", 2024-09-14T06:29:26.258 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.3599999999999998e-06 2024-09-14T06:29:26.258 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.258 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.258 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:29:26.258 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.837827+0000", 2024-09-14T06:29:26.259 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.588e-06 2024-09-14T06:29:26.259 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:26.259 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:26.259 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:26.259 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.259 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.259 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:12688 3.1e 3.302ef9fe (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:29:26.259 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:25.838430+0000", 2024-09-14T06:29:26.259 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 4294967295.9996099, 2024-09-14T06:29:26.259 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.002366974, 2024-09-14T06:29:26.259 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:29:26.259 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:29:26.259 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "queued for pg", 2024-09-14T06:29:26.259 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:29:26.259 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:29:26.259 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:26.260 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 12688 2024-09-14T06:29:26.260 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.260 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:29:26.260 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.260 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:29:26.260 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.838430+0000", 2024-09-14T06:29:26.260 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:26.260 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.260 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.260 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:29:26.260 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.838430+0000", 2024-09-14T06:29:26.260 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:26.260 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.260 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.260 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:29:26.261 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.838430+0000", 2024-09-14T06:29:26.261 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.96e-07 2024-09-14T06:29:26.261 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.261 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.261 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:29:26.261 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.838432+0000", 2024-09-14T06:29:26.261 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.672e-06 2024-09-14T06:29:26.261 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.261 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.261 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:29:26.261 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.838438+0000", 2024-09-14T06:29:26.261 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.2430000000000002e-06 2024-09-14T06:29:26.261 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.261 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.261 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:29:26.262 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.838443+0000", 2024-09-14T06:29:26.262 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.6070000000000001e-06 2024-09-14T06:29:26.262 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:26.262 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:26.262 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:26.262 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.262 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.262 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:12560 3.1e 3:7f9f740c:::rbd_data.112d92027590.0000000000000004:f [read 497664~1024 [fadvise_nocache] out=1024b] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:29:26.262 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:25.831033+0000", 2024-09-14T06:29:26.262 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 0.0070071839999999996, 2024-09-14T06:29:26.262 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0097928319999999996, 2024-09-14T06:29:26.262 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:29:26.262 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:29:26.262 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:29:26.263 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:29:26.263 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:29:26.263 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:26.263 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 12560 2024-09-14T06:29:26.263 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.263 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:29:26.263 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.263 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:29:26.263 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.831033+0000", 2024-09-14T06:29:26.263 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:26.263 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.263 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.263 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:29:26.263 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.831033+0000", 2024-09-14T06:29:26.263 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:26.264 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.264 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.264 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:29:26.264 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.831033+0000", 2024-09-14T06:29:26.264 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.2799999999999996e-07 2024-09-14T06:29:26.264 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.264 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.264 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:29:26.264 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.831035+0000", 2024-09-14T06:29:26.264 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.0940000000000002e-06 2024-09-14T06:29:26.264 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.264 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.264 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:29:26.264 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.831041+0000", 2024-09-14T06:29:26.265 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.8900000000000004e-06 2024-09-14T06:29:26.265 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.265 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.265 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:29:26.265 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.831046+0000", 2024-09-14T06:29:26.265 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.4050000000000002e-06 2024-09-14T06:29:26.265 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.265 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.265 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:29:26.265 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.840732+0000", 2024-09-14T06:29:26.265 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0096864970000000005 2024-09-14T06:29:26.265 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.265 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.265 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:29:26.266 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.840774+0000", 2024-09-14T06:29:26.266 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.1556e-05 2024-09-14T06:29:26.266 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:26.266 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:26.266 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:26.266 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.266 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.266 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:12592 3.1e 3.302ef9fe (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:29:26.266 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:25.836686+0000", 2024-09-14T06:29:26.266 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 0.0013537, 2024-09-14T06:29:26.266 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0041681929999999997, 2024-09-14T06:29:26.266 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:29:26.266 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:29:26.266 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "queued for pg", 2024-09-14T06:29:26.266 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:29:26.266 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:29:26.267 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:26.267 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 12592 2024-09-14T06:29:26.267 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.267 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:29:26.267 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.267 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:29:26.267 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.836686+0000", 2024-09-14T06:29:26.267 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:26.267 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.267 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.267 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:29:26.267 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.836686+0000", 2024-09-14T06:29:26.267 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:26.267 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.268 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.268 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:29:26.268 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.836686+0000", 2024-09-14T06:29:26.268 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.4700000000000002e-07 2024-09-14T06:29:26.268 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.268 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.268 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:29:26.268 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.836689+0000", 2024-09-14T06:29:26.268 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.2919999999999998e-06 2024-09-14T06:29:26.268 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.268 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.268 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:29:26.268 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.836695+0000", 2024-09-14T06:29:26.269 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.1990000000000003e-06 2024-09-14T06:29:26.269 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.269 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.269 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:29:26.269 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.836700+0000", 2024-09-14T06:29:26.269 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.2129999999999999e-06 2024-09-14T06:29:26.269 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:26.269 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:26.269 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:26.269 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.269 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.269 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:12624 3.1e 3.302ef9fe (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:29:26.269 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:25.837291+0000", 2024-09-14T06:29:26.270 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 0.00074834999999999995, 2024-09-14T06:29:26.270 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0035853650000000001, 2024-09-14T06:29:26.270 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:29:26.270 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:29:26.270 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "queued for pg", 2024-09-14T06:29:26.270 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:29:26.270 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:29:26.270 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:26.270 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 12624 2024-09-14T06:29:26.270 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.270 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:29:26.270 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.271 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:29:26.271 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.837291+0000", 2024-09-14T06:29:26.271 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:26.271 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.271 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.271 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:29:26.271 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.837291+0000", 2024-09-14T06:29:26.271 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:26.271 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.271 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.271 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:29:26.271 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.837292+0000", 2024-09-14T06:29:26.271 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.0599999999999999e-07 2024-09-14T06:29:26.272 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.272 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.272 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:29:26.272 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.837293+0000", 2024-09-14T06:29:26.272 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.71e-07 2024-09-14T06:29:26.272 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.272 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.272 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:29:26.272 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.837296+0000", 2024-09-14T06:29:26.272 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.033e-06 2024-09-14T06:29:26.272 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.272 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.272 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:29:26.273 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.837298+0000", 2024-09-14T06:29:26.273 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.3590000000000002e-06 2024-09-14T06:29:26.273 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:26.273 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:26.273 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:26.273 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.273 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.273 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:12656 3.1e 3.302ef9fe (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:29:26.273 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:25.837830+0000", 2024-09-14T06:29:26.273 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 0.00020989199999999999, 2024-09-14T06:29:26.273 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0030731500000000002, 2024-09-14T06:29:26.273 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:29:26.273 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:29:26.273 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "queued for pg", 2024-09-14T06:29:26.274 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:29:26.274 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:29:26.274 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:26.274 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 12656 2024-09-14T06:29:26.274 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.274 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:29:26.274 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.274 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:29:26.274 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.837830+0000", 2024-09-14T06:29:26.274 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:26.274 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.274 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.274 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:29:26.274 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.837830+0000", 2024-09-14T06:29:26.274 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:26.275 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.275 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.275 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:29:26.275 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.837830+0000", 2024-09-14T06:29:26.275 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.0699999999999999e-07 2024-09-14T06:29:26.275 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.275 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.275 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:29:26.275 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.837831+0000", 2024-09-14T06:29:26.275 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.0079999999999999e-06 2024-09-14T06:29:26.275 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.275 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.275 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:29:26.275 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.837837+0000", 2024-09-14T06:29:26.276 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.9900000000000002e-06 2024-09-14T06:29:26.276 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.276 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.276 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:29:26.276 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.837840+0000", 2024-09-14T06:29:26.276 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.531e-06 2024-09-14T06:29:26.276 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:26.276 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:26.276 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:26.276 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.276 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.276 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:12689 3.1e 3.302ef9fe (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:29:26.276 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:25.838448+0000", 2024-09-14T06:29:26.276 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 4294967295.9995918, 2024-09-14T06:29:26.276 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0024785110000000001, 2024-09-14T06:29:26.276 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:29:26.277 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:29:26.277 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "queued for pg", 2024-09-14T06:29:26.277 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:29:26.277 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:29:26.277 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:26.277 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 12689 2024-09-14T06:29:26.277 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.277 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:29:26.277 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.277 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:29:26.277 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.838448+0000", 2024-09-14T06:29:26.277 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:26.277 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.277 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.277 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:29:26.277 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.838448+0000", 2024-09-14T06:29:26.278 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:26.278 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.278 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.278 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:29:26.278 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.838448+0000", 2024-09-14T06:29:26.278 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.67e-07 2024-09-14T06:29:26.278 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.278 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.278 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:29:26.278 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.838449+0000", 2024-09-14T06:29:26.278 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.1400000000000001e-06 2024-09-14T06:29:26.278 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.278 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.278 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:29:26.278 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.838454+0000", 2024-09-14T06:29:26.279 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.33e-06 2024-09-14T06:29:26.279 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.279 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.279 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:29:26.279 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.838456+0000", 2024-09-14T06:29:26.279 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.6529999999999998e-06 2024-09-14T06:29:26.279 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:26.279 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:26.279 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:26.279 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.279 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.279 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:12561 3.1e 3:7f9f740c:::rbd_data.112d92027590.0000000000000004:f [read 507904~1024 [fadvise_nocache] out=1024b] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:29:26.279 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:25.831060+0000", 2024-09-14T06:29:26.279 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 0.0069793449999999996, 2024-09-14T06:29:26.279 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0098889979999999995, 2024-09-14T06:29:26.280 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:29:26.280 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:29:26.280 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:29:26.280 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:29:26.280 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:29:26.280 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:26.280 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 12561 2024-09-14T06:29:26.280 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.280 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:29:26.280 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.280 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:29:26.280 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.831060+0000", 2024-09-14T06:29:26.280 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:26.280 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.281 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.281 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:29:26.281 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.831060+0000", 2024-09-14T06:29:26.281 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:26.281 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.281 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.281 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:29:26.281 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.831061+0000", 2024-09-14T06:29:26.281 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.72e-07 2024-09-14T06:29:26.281 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.281 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.281 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:29:26.281 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.831062+0000", 2024-09-14T06:29:26.281 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.494e-06 2024-09-14T06:29:26.282 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.282 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.282 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:29:26.282 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.831070+0000", 2024-09-14T06:29:26.282 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.5399999999999998e-06 2024-09-14T06:29:26.282 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.282 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.282 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:29:26.282 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.831075+0000", 2024-09-14T06:29:26.282 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.3229999999999997e-06 2024-09-14T06:29:26.282 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.282 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.282 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:29:26.282 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.840857+0000", 2024-09-14T06:29:26.282 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0097823219999999995 2024-09-14T06:29:26.283 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.283 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.283 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:29:26.283 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.840897+0000", 2024-09-14T06:29:26.283 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.9996000000000001e-05 2024-09-14T06:29:26.283 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.283 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.283 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:29:26.283 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.840949+0000", 2024-09-14T06:29:26.283 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.2051e-05 2024-09-14T06:29:26.283 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:26.283 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:26.284 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:26.284 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.284 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.284 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:12593 3.1e 3.302ef9fe (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:29:26.284 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:25.836722+0000", 2024-09-14T06:29:26.284 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 0.001318023, 2024-09-14T06:29:26.284 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0042620560000000002, 2024-09-14T06:29:26.284 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:29:26.284 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:29:26.285 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "queued for pg", 2024-09-14T06:29:26.285 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:29:26.285 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:29:26.285 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:26.285 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 12593 2024-09-14T06:29:26.285 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.285 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:29:26.285 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.285 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:29:26.285 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.836722+0000", 2024-09-14T06:29:26.285 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:26.285 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.285 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.286 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:29:26.286 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.836722+0000", 2024-09-14T06:29:26.286 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:26.286 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.286 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.286 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:29:26.286 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.836722+0000", 2024-09-14T06:29:26.286 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.2500000000000001e-07 2024-09-14T06:29:26.286 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.286 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.286 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:29:26.286 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.836724+0000", 2024-09-14T06:29:26.286 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.5689999999999999e-06 2024-09-14T06:29:26.286 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.287 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.287 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:29:26.287 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.836729+0000", 2024-09-14T06:29:26.287 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.0760000000000002e-06 2024-09-14T06:29:26.287 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.287 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.287 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:29:26.287 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.836732+0000", 2024-09-14T06:29:26.287 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.6899999999999998e-06 2024-09-14T06:29:26.287 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:26.287 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:26.287 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:26.287 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.287 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.287 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:12625 3.1e 3.302ef9fe (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:29:26.287 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:25.837301+0000", 2024-09-14T06:29:26.288 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 0.00073892100000000002, 2024-09-14T06:29:26.288 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.003705542, 2024-09-14T06:29:26.288 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:29:26.288 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:29:26.288 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "queued for pg", 2024-09-14T06:29:26.288 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:29:26.288 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:29:26.288 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:26.288 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 12625 2024-09-14T06:29:26.288 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.288 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:29:26.288 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.288 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:29:26.288 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.837301+0000", 2024-09-14T06:29:26.289 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:26.289 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.289 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.289 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:29:26.289 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.837301+0000", 2024-09-14T06:29:26.289 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:26.289 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.289 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.289 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:29:26.289 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.837301+0000", 2024-09-14T06:29:26.289 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.0800000000000001e-07 2024-09-14T06:29:26.289 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.290 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.290 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:29:26.290 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.837305+0000", 2024-09-14T06:29:26.290 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.8539999999999999e-06 2024-09-14T06:29:26.290 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.290 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.290 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:29:26.290 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.837308+0000", 2024-09-14T06:29:26.290 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.2669999999999998e-06 2024-09-14T06:29:26.290 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.290 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.290 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:29:26.291 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.837311+0000", 2024-09-14T06:29:26.291 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.3989999999999998e-06 2024-09-14T06:29:26.291 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:26.291 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:26.291 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:26.291 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.291 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.291 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:12657 3.1e 3.302ef9fe (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:29:26.291 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:25.837843+0000", 2024-09-14T06:29:26.291 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 0.000197147, 2024-09-14T06:29:26.291 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0031863690000000001, 2024-09-14T06:29:26.291 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:29:26.291 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:29:26.291 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "queued for pg", 2024-09-14T06:29:26.292 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:29:26.292 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:29:26.292 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:26.292 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 12657 2024-09-14T06:29:26.292 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.292 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:29:26.292 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.292 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:29:26.292 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.837843+0000", 2024-09-14T06:29:26.292 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:26.292 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.292 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.292 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:29:26.293 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.837843+0000", 2024-09-14T06:29:26.293 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:26.293 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.293 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.293 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:29:26.293 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.837843+0000", 2024-09-14T06:29:26.293 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.05e-07 2024-09-14T06:29:26.293 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.293 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.293 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:29:26.293 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.837844+0000", 2024-09-14T06:29:26.293 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.2099999999999995e-07 2024-09-14T06:29:26.293 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.293 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.294 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:29:26.294 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.837848+0000", 2024-09-14T06:29:26.294 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.704e-06 2024-09-14T06:29:26.294 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.294 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.294 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:29:26.294 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.837850+0000", 2024-09-14T06:29:26.294 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.6790000000000001e-06 2024-09-14T06:29:26.294 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:26.294 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:26.294 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:26.294 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.294 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.294 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:12690 3.1e 3.302ef9fe (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:29:26.295 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:25.838460+0000", 2024-09-14T06:29:26.295 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 4294967295.9995804, 2024-09-14T06:29:26.295 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0025927200000000002, 2024-09-14T06:29:26.295 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:29:26.295 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:29:26.295 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "queued for pg", 2024-09-14T06:29:26.295 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:29:26.295 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:29:26.295 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:26.295 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 12690 2024-09-14T06:29:26.295 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.295 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:29:26.295 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.295 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:29:26.295 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.838460+0000", 2024-09-14T06:29:26.296 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:26.296 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.296 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.296 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:29:26.296 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.838460+0000", 2024-09-14T06:29:26.296 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:26.296 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.296 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.296 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:29:26.296 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.838460+0000", 2024-09-14T06:29:26.296 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.15e-07 2024-09-14T06:29:26.296 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.296 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.296 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:29:26.296 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.838461+0000", 2024-09-14T06:29:26.297 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.7600000000000006e-07 2024-09-14T06:29:26.297 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.297 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.297 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:29:26.297 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.838464+0000", 2024-09-14T06:29:26.297 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.0180000000000002e-06 2024-09-14T06:29:26.297 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.297 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.297 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:29:26.297 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.838466+0000", 2024-09-14T06:29:26.297 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.5110000000000002e-06 2024-09-14T06:29:26.297 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:26.297 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:26.297 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:26.298 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.298 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.298 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:12562 3.1e 3:7f9f740c:::rbd_data.112d92027590.0000000000000004:f [read 518144~1024 [fadvise_nocache] out=1024b] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:29:26.298 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:25.831079+0000", 2024-09-14T06:29:26.298 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 0.0069605200000000004, 2024-09-14T06:29:26.298 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0099946630000000009, 2024-09-14T06:29:26.298 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:29:26.298 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:29:26.298 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:29:26.298 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:29:26.298 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:29:26.298 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:26.298 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 12562 2024-09-14T06:29:26.298 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.298 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:29:26.298 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.299 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:29:26.299 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.831079+0000", 2024-09-14T06:29:26.299 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:26.299 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.299 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.299 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:29:26.299 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.831079+0000", 2024-09-14T06:29:26.299 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:26.299 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.299 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.299 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:29:26.299 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.831080+0000", 2024-09-14T06:29:26.299 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.2000000000000001e-07 2024-09-14T06:29:26.299 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.299 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.300 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:29:26.300 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.831089+0000", 2024-09-14T06:29:26.300 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.2159999999999995e-06 2024-09-14T06:29:26.300 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.300 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.300 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:29:26.300 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.831092+0000", 2024-09-14T06:29:26.300 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.7479999999999999e-06 2024-09-14T06:29:26.300 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.300 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.300 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:29:26.300 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.831098+0000", 2024-09-14T06:29:26.300 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.5999999999999997e-06 2024-09-14T06:29:26.300 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.300 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.301 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:29:26.301 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.840975+0000", 2024-09-14T06:29:26.301 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0098767060000000007 2024-09-14T06:29:26.301 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.301 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.301 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:29:26.301 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.841023+0000", 2024-09-14T06:29:26.301 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.7765000000000001e-05 2024-09-14T06:29:26.301 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.301 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.301 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:29:26.301 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.841074+0000", 2024-09-14T06:29:26.301 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.1408e-05 2024-09-14T06:29:26.302 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:26.302 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:26.302 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:26.302 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.302 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.302 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:12594 3.1e 3.302ef9fe (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:29:26.302 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:25.836737+0000", 2024-09-14T06:29:26.302 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 0.0013033210000000001, 2024-09-14T06:29:26.302 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0043723299999999998, 2024-09-14T06:29:26.302 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:29:26.302 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:29:26.302 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "queued for pg", 2024-09-14T06:29:26.302 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:29:26.302 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:29:26.302 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:26.302 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 12594 2024-09-14T06:29:26.303 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.303 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:29:26.303 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.303 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:29:26.303 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.836737+0000", 2024-09-14T06:29:26.303 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:26.303 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.303 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.303 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:29:26.303 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.836737+0000", 2024-09-14T06:29:26.303 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:26.303 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.303 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.303 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:29:26.304 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.836737+0000", 2024-09-14T06:29:26.304 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.16e-07 2024-09-14T06:29:26.304 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.304 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.304 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:29:26.304 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.836738+0000", 2024-09-14T06:29:26.304 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.0160000000000001e-06 2024-09-14T06:29:26.304 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.304 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.304 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:29:26.304 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.836741+0000", 2024-09-14T06:29:26.304 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.5480000000000002e-06 2024-09-14T06:29:26.305 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.305 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.305 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:29:26.305 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.836744+0000", 2024-09-14T06:29:26.305 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.5509999999999998e-06 2024-09-14T06:29:26.305 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:26.305 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:26.305 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:26.305 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.305 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.305 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:12626 3.1e 3.302ef9fe (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:29:26.305 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:25.837314+0000", 2024-09-14T06:29:26.305 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 0.00072557399999999999, 2024-09-14T06:29:26.306 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.003817693, 2024-09-14T06:29:26.306 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:29:26.306 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:29:26.306 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "queued for pg", 2024-09-14T06:29:26.306 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:29:26.306 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:29:26.306 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:26.306 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 12626 2024-09-14T06:29:26.306 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.306 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:29:26.306 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.306 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:29:26.306 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.837314+0000", 2024-09-14T06:29:26.307 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:26.307 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.307 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.307 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:29:26.307 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.837314+0000", 2024-09-14T06:29:26.307 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:26.307 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.307 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.307 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:29:26.307 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.837314+0000", 2024-09-14T06:29:26.307 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.1400000000000001e-07 2024-09-14T06:29:26.307 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.307 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.307 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:29:26.308 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.837315+0000", 2024-09-14T06:29:26.308 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.3900000000000003e-07 2024-09-14T06:29:26.308 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.308 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.308 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:29:26.308 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.837318+0000", 2024-09-14T06:29:26.308 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.9529999999999999e-06 2024-09-14T06:29:26.308 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.308 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.308 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:29:26.308 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.837321+0000", 2024-09-14T06:29:26.308 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.328e-06 2024-09-14T06:29:26.308 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:26.309 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:26.309 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:26.309 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.309 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.309 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:12658 3.1e 3.302ef9fe (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:29:26.309 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:25.837853+0000", 2024-09-14T06:29:26.309 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 0.000186455, 2024-09-14T06:29:26.309 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0033016500000000002, 2024-09-14T06:29:26.309 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:29:26.309 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:29:26.309 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "queued for pg", 2024-09-14T06:29:26.310 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:29:26.310 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:29:26.310 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:26.310 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 12658 2024-09-14T06:29:26.310 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.310 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:29:26.310 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.310 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:29:26.310 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.837853+0000", 2024-09-14T06:29:26.310 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:26.310 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.310 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.310 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:29:26.310 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.837853+0000", 2024-09-14T06:29:26.311 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:26.311 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.311 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.311 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:29:26.311 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.837854+0000", 2024-09-14T06:29:26.311 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.0800000000000001e-07 2024-09-14T06:29:26.311 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.311 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.311 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:29:26.311 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.837855+0000", 2024-09-14T06:29:26.311 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.4900000000000004e-07 2024-09-14T06:29:26.311 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.311 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.311 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:29:26.312 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.837857+0000", 2024-09-14T06:29:26.312 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.8909999999999999e-06 2024-09-14T06:29:26.312 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.312 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.312 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:29:26.312 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.837860+0000", 2024-09-14T06:29:26.312 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.2859999999999998e-06 2024-09-14T06:29:26.312 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:26.312 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:26.312 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:26.312 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.312 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.312 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:12691 3.1e 3.302ef9fe (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:29:26.312 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:25.838470+0000", 2024-09-14T06:29:26.313 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 4294967295.9995699, 2024-09-14T06:29:26.313 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0027084359999999998, 2024-09-14T06:29:26.313 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:29:26.313 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:29:26.313 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "queued for pg", 2024-09-14T06:29:26.313 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:29:26.313 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:29:26.313 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:26.313 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 12691 2024-09-14T06:29:26.313 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.313 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:29:26.313 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.313 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:29:26.313 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.838470+0000", 2024-09-14T06:29:26.314 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:26.314 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.314 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.314 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:29:26.314 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.838470+0000", 2024-09-14T06:29:26.314 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:26.314 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.314 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.314 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:29:26.314 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.838470+0000", 2024-09-14T06:29:26.314 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.2000000000000001e-07 2024-09-14T06:29:26.314 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.314 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.314 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:29:26.314 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.838471+0000", 2024-09-14T06:29:26.315 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.0079999999999999e-06 2024-09-14T06:29:26.315 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.315 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.315 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:29:26.315 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.838474+0000", 2024-09-14T06:29:26.315 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.9780000000000001e-06 2024-09-14T06:29:26.315 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.315 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.315 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:29:26.315 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.838476+0000", 2024-09-14T06:29:26.315 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.52e-06 2024-09-14T06:29:26.315 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:26.315 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:26.315 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:26.315 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.315 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.316 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:12595 3.1e 3.302ef9fe (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:29:26.316 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:25.836747+0000", 2024-09-14T06:29:26.316 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 0.0012927360000000001, 2024-09-14T06:29:26.316 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0044555150000000002, 2024-09-14T06:29:26.316 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:29:26.316 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:29:26.316 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "queued for pg", 2024-09-14T06:29:26.316 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:29:26.316 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:29:26.316 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:26.316 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 12595 2024-09-14T06:29:26.316 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.316 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:29:26.316 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.317 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:29:26.317 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.836747+0000", 2024-09-14T06:29:26.317 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:26.317 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.317 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.317 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:29:26.317 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.836747+0000", 2024-09-14T06:29:26.317 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:26.317 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.317 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.317 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:29:26.317 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.836747+0000", 2024-09-14T06:29:26.317 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.0600000000000001e-07 2024-09-14T06:29:26.317 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.318 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.318 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:29:26.318 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.836749+0000", 2024-09-14T06:29:26.318 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.155e-06 2024-09-14T06:29:26.318 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.318 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.318 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:29:26.318 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.836755+0000", 2024-09-14T06:29:26.318 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.6889999999999998e-06 2024-09-14T06:29:26.318 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.318 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.318 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:29:26.318 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.836758+0000", 2024-09-14T06:29:26.318 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.7860000000000001e-06 2024-09-14T06:29:26.319 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:26.319 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:26.319 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:26.319 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.319 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.319 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:12627 3.1e 3.302ef9fe (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:29:26.319 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:25.837324+0000", 2024-09-14T06:29:26.319 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 0.00071611099999999996, 2024-09-14T06:29:26.319 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.003901524, 2024-09-14T06:29:26.319 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:29:26.319 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:29:26.319 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "queued for pg", 2024-09-14T06:29:26.319 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:29:26.319 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:29:26.319 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:26.319 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 12627 2024-09-14T06:29:26.320 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.320 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:29:26.320 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.320 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:29:26.320 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.837324+0000", 2024-09-14T06:29:26.320 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:26.320 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.320 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.320 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:29:26.320 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.837324+0000", 2024-09-14T06:29:26.320 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:26.320 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.320 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.320 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:29:26.321 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.837324+0000", 2024-09-14T06:29:26.321 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.0900000000000001e-07 2024-09-14T06:29:26.321 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.321 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.321 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:29:26.321 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.837325+0000", 2024-09-14T06:29:26.321 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.9500000000000001e-07 2024-09-14T06:29:26.321 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.321 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.321 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:29:26.321 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.837328+0000", 2024-09-14T06:29:26.321 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.9359999999999999e-06 2024-09-14T06:29:26.321 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.321 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.322 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:29:26.322 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.837330+0000", 2024-09-14T06:29:26.322 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.2709999999999999e-06 2024-09-14T06:29:26.322 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:26.322 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:26.322 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:26.322 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.322 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.322 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:12659 3.1e 3.302ef9fe (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:29:26.322 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:25.837863+0000", 2024-09-14T06:29:26.322 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 0.00017710800000000001, 2024-09-14T06:29:26.322 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0033851699999999998, 2024-09-14T06:29:26.322 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:29:26.322 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:29:26.322 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "queued for pg", 2024-09-14T06:29:26.322 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:29:26.323 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:29:26.323 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:26.323 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 12659 2024-09-14T06:29:26.323 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.323 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:29:26.323 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.323 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:29:26.323 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.837863+0000", 2024-09-14T06:29:26.323 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:26.323 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.323 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.323 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:29:26.323 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.837863+0000", 2024-09-14T06:29:26.323 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:26.323 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.324 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.324 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:29:26.324 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.837863+0000", 2024-09-14T06:29:26.324 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.29e-07 2024-09-14T06:29:26.324 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.324 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.324 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:29:26.324 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.837864+0000", 2024-09-14T06:29:26.324 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.6700000000000002e-07 2024-09-14T06:29:26.324 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.324 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.324 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:29:26.324 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.837867+0000", 2024-09-14T06:29:26.325 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.6739999999999999e-06 2024-09-14T06:29:26.325 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.325 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.325 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:29:26.325 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.837873+0000", 2024-09-14T06:29:26.325 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.3870000000000003e-06 2024-09-14T06:29:26.325 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:26.325 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:26.325 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:26.325 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.325 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.325 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:12692 3.1e 3.302ef9fe (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:29:26.325 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:25.838479+0000", 2024-09-14T06:29:26.325 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 4294967295.9995604, 2024-09-14T06:29:26.325 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0027916249999999998, 2024-09-14T06:29:26.326 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:29:26.326 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:29:26.326 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "queued for pg", 2024-09-14T06:29:26.326 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:29:26.326 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:29:26.326 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:26.326 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 12692 2024-09-14T06:29:26.326 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.326 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:29:26.326 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.326 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:29:26.326 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.838479+0000", 2024-09-14T06:29:26.326 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:26.326 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.326 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.326 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:29:26.327 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.838479+0000", 2024-09-14T06:29:26.327 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:26.327 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.327 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.327 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:29:26.327 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.838480+0000", 2024-09-14T06:29:26.327 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.05e-07 2024-09-14T06:29:26.327 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.327 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.327 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:29:26.327 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.838481+0000", 2024-09-14T06:29:26.327 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.9299999999999996e-07 2024-09-14T06:29:26.327 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.327 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.327 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:29:26.328 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.838483+0000", 2024-09-14T06:29:26.328 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.937e-06 2024-09-14T06:29:26.328 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.328 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.328 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:29:26.328 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.838486+0000", 2024-09-14T06:29:26.328 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.3719999999999999e-06 2024-09-14T06:29:26.328 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:26.328 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:26.328 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:26.328 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.328 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.328 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:12596 3.1e 3.302ef9fe (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:29:26.328 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:25.836763+0000", 2024-09-14T06:29:26.329 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 0.0012766399999999999, 2024-09-14T06:29:26.329 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.004531569, 2024-09-14T06:29:26.329 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:29:26.329 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:29:26.329 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "queued for pg", 2024-09-14T06:29:26.329 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:29:26.329 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:29:26.329 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:26.329 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 12596 2024-09-14T06:29:26.329 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.329 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:29:26.329 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.329 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:29:26.329 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.836763+0000", 2024-09-14T06:29:26.330 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:26.330 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.330 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.330 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:29:26.330 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.836763+0000", 2024-09-14T06:29:26.330 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:26.330 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.330 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.330 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:29:26.330 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.836763+0000", 2024-09-14T06:29:26.330 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.4600000000000001e-07 2024-09-14T06:29:26.330 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.330 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.330 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:29:26.330 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.836765+0000", 2024-09-14T06:29:26.331 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.172e-06 2024-09-14T06:29:26.331 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.331 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.331 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:29:26.331 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.836768+0000", 2024-09-14T06:29:26.331 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.2389999999999998e-06 2024-09-14T06:29:26.331 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.331 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.331 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:29:26.331 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.836771+0000", 2024-09-14T06:29:26.331 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.9399999999999998e-06 2024-09-14T06:29:26.331 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:26.331 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:26.331 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:26.332 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.332 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.332 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:12628 3.1e 3.302ef9fe (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:29:26.332 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:25.837333+0000", 2024-09-14T06:29:26.332 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 0.00070676099999999998, 2024-09-14T06:29:26.332 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0039843789999999997, 2024-09-14T06:29:26.332 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:29:26.332 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:29:26.332 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "queued for pg", 2024-09-14T06:29:26.332 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:29:26.332 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:29:26.332 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:26.333 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 12628 2024-09-14T06:29:26.333 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.333 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:29:26.333 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.333 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:29:26.333 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.837333+0000", 2024-09-14T06:29:26.333 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:26.333 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.333 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.333 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:29:26.333 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.837333+0000", 2024-09-14T06:29:26.333 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:26.333 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.333 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.334 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:29:26.334 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.837333+0000", 2024-09-14T06:29:26.334 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.0699999999999999e-07 2024-09-14T06:29:26.334 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.334 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.334 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:29:26.334 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.837337+0000", 2024-09-14T06:29:26.334 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.0500000000000002e-06 2024-09-14T06:29:26.334 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.334 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.334 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:29:26.334 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.837341+0000", 2024-09-14T06:29:26.334 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.3759999999999999e-06 2024-09-14T06:29:26.334 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.334 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.335 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:29:26.335 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.837349+0000", 2024-09-14T06:29:26.335 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.4449999999999996e-06 2024-09-14T06:29:26.335 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:26.335 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:26.335 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:26.335 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.335 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.335 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:12660 3.1e 3.302ef9fe (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:29:26.335 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:25.837891+0000", 2024-09-14T06:29:26.335 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 0.00014836800000000001, 2024-09-14T06:29:26.335 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0035469960000000002, 2024-09-14T06:29:26.335 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:29:26.335 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:29:26.336 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "queued for pg", 2024-09-14T06:29:26.336 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:29:26.336 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:29:26.336 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:26.336 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 12660 2024-09-14T06:29:26.336 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.336 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:29:26.336 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.336 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:29:26.336 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.837891+0000", 2024-09-14T06:29:26.337 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:26.337 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.337 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.337 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:29:26.337 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.837891+0000", 2024-09-14T06:29:26.337 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:26.337 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.337 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.337 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:29:26.337 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.837892+0000", 2024-09-14T06:29:26.337 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.5700000000000002e-07 2024-09-14T06:29:26.338 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.338 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.338 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:29:26.338 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.837896+0000", 2024-09-14T06:29:26.338 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.454e-06 2024-09-14T06:29:26.338 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.338 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.338 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:29:26.338 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.837902+0000", 2024-09-14T06:29:26.338 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.6879999999999996e-06 2024-09-14T06:29:26.338 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.338 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.338 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:29:26.338 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.837907+0000", 2024-09-14T06:29:26.339 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.7670000000000003e-06 2024-09-14T06:29:26.339 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:26.339 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:26.339 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:26.339 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.339 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.339 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:12693 3.1e 3.302ef9fe (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:29:26.339 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:25.838490+0000", 2024-09-14T06:29:26.339 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 4294967295.9995503, 2024-09-14T06:29:26.339 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0029733509999999999, 2024-09-14T06:29:26.339 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:29:26.339 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:29:26.339 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "queued for pg", 2024-09-14T06:29:26.339 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:29:26.340 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:29:26.340 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:26.340 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 12693 2024-09-14T06:29:26.340 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.340 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:29:26.340 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.340 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:29:26.340 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.838490+0000", 2024-09-14T06:29:26.340 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:26.340 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.340 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.340 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:29:26.340 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.838490+0000", 2024-09-14T06:29:26.340 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:26.341 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.341 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.341 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:29:26.341 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.838490+0000", 2024-09-14T06:29:26.341 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.28e-07 2024-09-14T06:29:26.341 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.341 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.341 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:29:26.341 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.838491+0000", 2024-09-14T06:29:26.341 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.6199999999999996e-07 2024-09-14T06:29:26.341 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.342 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.342 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:29:26.342 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.838494+0000", 2024-09-14T06:29:26.342 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.0079999999999998e-06 2024-09-14T06:29:26.342 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.342 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.342 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:29:26.342 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.838496+0000", 2024-09-14T06:29:26.342 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.401e-06 2024-09-14T06:29:26.343 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:26.343 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:26.343 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:26.343 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.343 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.343 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:12597 3.1e 3.302ef9fe (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:29:26.344 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:25.836774+0000", 2024-09-14T06:29:26.344 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 0.001265382, 2024-09-14T06:29:26.344 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0047137760000000003, 2024-09-14T06:29:26.344 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:29:26.344 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:29:26.344 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "queued for pg", 2024-09-14T06:29:26.344 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:29:26.345 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:29:26.345 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:26.345 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 12597 2024-09-14T06:29:26.345 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.345 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:29:26.345 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.346 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:29:26.346 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.836774+0000", 2024-09-14T06:29:26.346 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:26.346 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.346 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.346 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:29:26.347 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.836774+0000", 2024-09-14T06:29:26.347 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:26.347 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.347 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.347 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:29:26.347 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.836775+0000", 2024-09-14T06:29:26.348 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.1e-07 2024-09-14T06:29:26.348 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.348 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.348 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:29:26.348 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.836776+0000", 2024-09-14T06:29:26.348 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.3600000000000002e-07 2024-09-14T06:29:26.349 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.349 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.349 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:29:26.349 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.836781+0000", 2024-09-14T06:29:26.349 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.7810000000000002e-06 2024-09-14T06:29:26.349 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.350 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.350 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:29:26.350 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.836787+0000", 2024-09-14T06:29:26.350 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.8640000000000001e-06 2024-09-14T06:29:26.350 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:26.350 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:26.351 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:26.351 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.351 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.351 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:12629 3.1e 3.302ef9fe (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:29:26.351 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:25.837353+0000", 2024-09-14T06:29:26.351 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 0.00068644500000000005, 2024-09-14T06:29:26.352 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0041590079999999996, 2024-09-14T06:29:26.352 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:29:26.352 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:29:26.352 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "queued for pg", 2024-09-14T06:29:26.352 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:29:26.353 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:29:26.353 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:26.353 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 12629 2024-09-14T06:29:26.353 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.353 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:29:26.353 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.354 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:29:26.354 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.837353+0000", 2024-09-14T06:29:26.354 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:26.354 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.354 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.354 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:29:26.355 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.837353+0000", 2024-09-14T06:29:26.355 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:26.355 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.355 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.355 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:29:26.355 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.837354+0000", 2024-09-14T06:29:26.356 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.17e-07 2024-09-14T06:29:26.356 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.356 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.356 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:29:26.356 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.837355+0000", 2024-09-14T06:29:26.356 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.049e-06 2024-09-14T06:29:26.357 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.357 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.357 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:29:26.357 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.837359+0000", 2024-09-14T06:29:26.357 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.968e-06 2024-09-14T06:29:26.357 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.358 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.358 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:29:26.358 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.837365+0000", 2024-09-14T06:29:26.358 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.6109999999999999e-06 2024-09-14T06:29:26.358 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:26.358 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:26.359 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:26.359 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.359 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.359 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:12661 3.1e 3.302ef9fe (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:29:26.359 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:25.837915+0000", 2024-09-14T06:29:26.360 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 0.00012464799999999999, 2024-09-14T06:29:26.360 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0036213450000000002, 2024-09-14T06:29:26.360 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:29:26.360 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:29:26.360 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "queued for pg", 2024-09-14T06:29:26.360 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:29:26.361 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:29:26.361 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:26.361 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 12661 2024-09-14T06:29:26.361 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.361 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:29:26.361 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.361 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:29:26.362 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.837915+0000", 2024-09-14T06:29:26.362 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:26.362 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.362 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.362 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:29:26.363 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.837915+0000", 2024-09-14T06:29:26.363 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:26.363 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.363 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.363 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:29:26.363 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.837916+0000", 2024-09-14T06:29:26.363 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.6199999999999999e-07 2024-09-14T06:29:26.364 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.364 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.364 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:29:26.364 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.837918+0000", 2024-09-14T06:29:26.364 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.5490000000000001e-06 2024-09-14T06:29:26.365 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.365 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.365 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:29:26.365 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.837924+0000", 2024-09-14T06:29:26.365 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.0549999999999996e-06 2024-09-14T06:29:26.365 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.366 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.366 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:29:26.366 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.837930+0000", 2024-09-14T06:29:26.366 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.1249999999999998e-06 2024-09-14T06:29:26.366 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:26.366 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:26.367 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:26.367 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.367 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.367 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:12694 3.1e 3.302ef9fe (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:29:26.367 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:25.838499+0000", 2024-09-14T06:29:26.367 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 4294967295.9995408, 2024-09-14T06:29:26.368 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0030611029999999999, 2024-09-14T06:29:26.368 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:29:26.368 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:29:26.368 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "queued for pg", 2024-09-14T06:29:26.368 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:29:26.368 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:29:26.368 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:26.369 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 12694 2024-09-14T06:29:26.369 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.369 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:29:26.369 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.369 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:29:26.369 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.838499+0000", 2024-09-14T06:29:26.370 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:26.370 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.370 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.370 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:29:26.370 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.838499+0000", 2024-09-14T06:29:26.370 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:26.370 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.370 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.370 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:29:26.370 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.838499+0000", 2024-09-14T06:29:26.371 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.0100000000000001e-07 2024-09-14T06:29:26.371 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.371 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.371 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:29:26.371 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.838500+0000", 2024-09-14T06:29:26.371 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.2999999999999999e-07 2024-09-14T06:29:26.371 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.371 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.371 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:29:26.371 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.838503+0000", 2024-09-14T06:29:26.371 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.9510000000000002e-06 2024-09-14T06:29:26.371 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.371 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.371 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:29:26.371 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.838506+0000", 2024-09-14T06:29:26.372 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.351e-06 2024-09-14T06:29:26.372 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:26.372 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:26.372 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:26.372 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:26.372 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-09-14T06:29:26.372 INFO:teuthology.orchestra.run.smithi094.stdout: "num_ops": 139 2024-09-14T06:29:26.372 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-14T06:29:26.374 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-14T06:29:26.374 INFO:tasks.ceph.osd.1.smithi094.stderr:2024-09-14T06:29:26.372+0000 7f328e47a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33641) UID: 0 2024-09-14T06:29:26.474 INFO:tasks.ceph.osd.5.smithi138.stderr:2024-09-14T06:29:26.473+0000 7fe13746f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33618) UID: 0 2024-09-14T06:29:26.517 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-14T06:29:26.517 INFO:teuthology.orchestra.run.smithi094.stdout: "ops": [], 2024-09-14T06:29:26.517 INFO:teuthology.orchestra.run.smithi094.stdout: "complaint_time": 30, 2024-09-14T06:29:26.517 INFO:teuthology.orchestra.run.smithi094.stdout: "num_blocked_ops": 0 2024-09-14T06:29:26.517 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-14T06:29:26.529 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-14T06:29:26.575 INFO:tasks.ceph.osd.3.smithi094.stderr:2024-09-14T06:29:26.574+0000 7feeeafe8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33655) UID: 0 2024-09-14T06:29:26.642 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-14T06:29:26.642 INFO:teuthology.orchestra.run.smithi094.stdout: "size": 20, 2024-09-14T06:29:26.642 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 600, 2024-09-14T06:29:26.642 INFO:teuthology.orchestra.run.smithi094.stdout: "ops": [ 2024-09-14T06:29:26.643 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.643 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:15608 3.b 3:d24f5f7d:::rbd_data.112d92027590.000000000000000d:f [read 852992~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:29:26.643 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:26.142492+0000", 2024-09-14T06:29:26.643 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 0.49688807600000001, 2024-09-14T06:29:26.643 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.048597006999999998, 2024-09-14T06:29:26.643 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:29:26.644 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:29:26.644 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:29:26.644 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:29:26.644 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:29:26.644 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:26.645 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 15608 2024-09-14T06:29:26.645 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.645 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:29:26.645 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.645 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:29:26.645 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.142492+0000", 2024-09-14T06:29:26.646 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:26.646 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.646 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.646 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:29:26.646 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.142492+0000", 2024-09-14T06:29:26.646 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:26.646 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.647 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.647 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:29:26.647 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.142568+0000", 2024-09-14T06:29:26.647 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.6209000000000002e-05 2024-09-14T06:29:26.647 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.647 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.647 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:29:26.648 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.142571+0000", 2024-09-14T06:29:26.648 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.157e-06 2024-09-14T06:29:26.648 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.648 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.648 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:29:26.648 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.142579+0000", 2024-09-14T06:29:26.649 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.6879999999999999e-06 2024-09-14T06:29:26.649 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.649 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.649 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:29:26.649 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.142585+0000", 2024-09-14T06:29:26.649 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.8780000000000003e-06 2024-09-14T06:29:26.650 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.650 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.650 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:29:26.650 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.190912+0000", 2024-09-14T06:29:26.650 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.048327579000000002 2024-09-14T06:29:26.650 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.651 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.651 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:29:26.651 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.190990+0000", 2024-09-14T06:29:26.651 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.8119000000000002e-05 2024-09-14T06:29:26.651 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.651 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.652 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:29:26.652 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.191089+0000", 2024-09-14T06:29:26.652 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.8376999999999997e-05 2024-09-14T06:29:26.652 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:26.652 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:26.653 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:26.653 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.653 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.653 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:15609 3.b 3:d24f5f7d:::rbd_data.112d92027590.000000000000000d:f [read 863232~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:29:26.653 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:26.142596+0000", 2024-09-14T06:29:26.653 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 0.49678430600000001, 2024-09-14T06:29:26.654 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.048641772999999999, 2024-09-14T06:29:26.654 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:29:26.654 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:29:26.654 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:29:26.654 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:29:26.654 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:29:26.655 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:26.655 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 15609 2024-09-14T06:29:26.655 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.655 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:29:26.655 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.655 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:29:26.656 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.142596+0000", 2024-09-14T06:29:26.656 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:26.656 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.656 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.656 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:29:26.656 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.142596+0000", 2024-09-14T06:29:26.657 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:26.657 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.657 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.657 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:29:26.657 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.142741+0000", 2024-09-14T06:29:26.657 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00014539199999999999 2024-09-14T06:29:26.658 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.658 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.658 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:29:26.658 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.142745+0000", 2024-09-14T06:29:26.658 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.1470000000000003e-06 2024-09-14T06:29:26.659 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.659 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.659 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:29:26.659 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.142754+0000", 2024-09-14T06:29:26.659 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.1330000000000005e-06 2024-09-14T06:29:26.659 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.660 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.660 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:29:26.660 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.142761+0000", 2024-09-14T06:29:26.660 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.63e-06 2024-09-14T06:29:26.660 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.661 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.661 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:29:26.661 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.191127+0000", 2024-09-14T06:29:26.661 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.048366251999999998 2024-09-14T06:29:26.661 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.661 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.662 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:29:26.662 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.191178+0000", 2024-09-14T06:29:26.662 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.1359000000000001e-05 2024-09-14T06:29:26.662 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.662 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.662 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:29:26.663 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.191237+0000", 2024-09-14T06:29:26.663 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.8860000000000002e-05 2024-09-14T06:29:26.663 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:26.663 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:26.663 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:26.663 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.664 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.664 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:15610 3.b 3:d24f5f7d:::rbd_data.112d92027590.000000000000000d:f [read 873472~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:29:26.664 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:26.142773+0000", 2024-09-14T06:29:26.664 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 0.49660683, 2024-09-14T06:29:26.664 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.048677076999999999, 2024-09-14T06:29:26.664 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:29:26.665 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:29:26.665 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:29:26.665 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:29:26.665 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:29:26.665 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:26.666 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 15610 2024-09-14T06:29:26.666 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.666 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:29:26.666 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.666 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:29:26.666 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.142773+0000", 2024-09-14T06:29:26.667 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:26.667 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.667 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.667 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:29:26.667 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.142773+0000", 2024-09-14T06:29:26.667 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:26.668 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.668 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.668 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:29:26.668 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.142855+0000", 2024-09-14T06:29:26.668 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.2180000000000003e-05 2024-09-14T06:29:26.668 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.669 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.669 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:29:26.669 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.142859+0000", 2024-09-14T06:29:26.669 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.5810000000000001e-06 2024-09-14T06:29:26.669 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.670 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.670 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:29:26.670 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.142868+0000", 2024-09-14T06:29:26.670 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.8489999999999995e-06 2024-09-14T06:29:26.670 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.671 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.671 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:29:26.671 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.142874+0000", 2024-09-14T06:29:26.671 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.7750000000000004e-06 2024-09-14T06:29:26.671 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.671 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.672 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:29:26.672 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.191272+0000", 2024-09-14T06:29:26.672 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.048397430999999998 2024-09-14T06:29:26.672 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.672 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.672 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:29:26.673 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.191351+0000", 2024-09-14T06:29:26.673 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.9480000000000005e-05 2024-09-14T06:29:26.673 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.673 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.673 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:29:26.673 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.191450+0000", 2024-09-14T06:29:26.674 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.8781000000000002e-05 2024-09-14T06:29:26.674 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:26.674 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:26.674 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:26.674 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.674 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.674 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:15611 3.b 3:d24f5f7d:::rbd_data.112d92027590.000000000000000d:f [read 883712~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:29:26.674 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:26.142885+0000", 2024-09-14T06:29:26.674 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 0.496494881, 2024-09-14T06:29:26.675 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.048706685, 2024-09-14T06:29:26.675 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:29:26.675 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:29:26.675 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:29:26.675 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:29:26.675 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:29:26.675 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:26.675 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 15611 2024-09-14T06:29:26.675 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.675 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:29:26.675 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.675 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:29:26.675 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.142885+0000", 2024-09-14T06:29:26.675 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:26.675 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.676 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.676 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:29:26.676 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.142885+0000", 2024-09-14T06:29:26.676 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:26.676 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.676 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.676 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:29:26.676 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.142989+0000", 2024-09-14T06:29:26.676 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00010394 2024-09-14T06:29:26.676 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.676 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.676 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:29:26.676 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.142992+0000", 2024-09-14T06:29:26.677 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.534e-06 2024-09-14T06:29:26.677 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.677 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.677 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:29:26.677 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.143001+0000", 2024-09-14T06:29:26.677 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.9050000000000003e-06 2024-09-14T06:29:26.677 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.677 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.677 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:29:26.677 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.143008+0000", 2024-09-14T06:29:26.677 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.8900000000000001e-06 2024-09-14T06:29:26.677 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.678 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.678 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:29:26.678 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.191489+0000", 2024-09-14T06:29:26.678 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.048480489000000002 2024-09-14T06:29:26.678 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.678 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.678 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:29:26.679 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.191532+0000", 2024-09-14T06:29:26.679 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.3244999999999997e-05 2024-09-14T06:29:26.679 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.679 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.679 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:29:26.679 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.191592+0000", 2024-09-14T06:29:26.679 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.9682000000000002e-05 2024-09-14T06:29:26.679 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:26.679 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:26.679 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:26.679 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.679 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.679 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:15612 3.b 3:d24f5f7d:::rbd_data.112d92027590.000000000000000d:f [read 893952~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:29:26.680 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:26.143019+0000", 2024-09-14T06:29:26.680 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 0.49636119000000001, 2024-09-14T06:29:26.680 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.048778560999999998, 2024-09-14T06:29:26.680 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:29:26.680 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:29:26.680 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:29:26.680 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:29:26.680 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:29:26.680 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:26.680 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 15612 2024-09-14T06:29:26.680 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.680 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:29:26.680 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.680 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:29:26.681 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.143019+0000", 2024-09-14T06:29:26.681 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:26.681 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.681 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.681 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:29:26.681 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.143019+0000", 2024-09-14T06:29:26.681 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:26.681 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.681 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.681 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:29:26.681 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.143236+0000", 2024-09-14T06:29:26.681 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000217517 2024-09-14T06:29:26.681 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.682 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.682 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:29:26.682 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.143241+0000", 2024-09-14T06:29:26.682 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.7779999999999998e-06 2024-09-14T06:29:26.682 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.682 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.682 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:29:26.682 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.143248+0000", 2024-09-14T06:29:26.682 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.7100000000000001e-06 2024-09-14T06:29:26.682 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.682 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.682 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:29:26.682 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.143254+0000", 2024-09-14T06:29:26.682 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.8950000000000003e-06 2024-09-14T06:29:26.683 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.683 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.683 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:29:26.683 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.191626+0000", 2024-09-14T06:29:26.683 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.048372747000000001 2024-09-14T06:29:26.683 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.683 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.683 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:29:26.683 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.191703+0000", 2024-09-14T06:29:26.683 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.6246000000000001e-05 2024-09-14T06:29:26.683 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.684 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.684 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:29:26.684 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.191797+0000", 2024-09-14T06:29:26.684 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.4667999999999996e-05 2024-09-14T06:29:26.684 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:26.684 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:26.684 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:26.684 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.684 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.684 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:15613 3.b 3:d24f5f7d:::rbd_data.112d92027590.000000000000000d:f [read 904192~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:29:26.684 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:26.143264+0000", 2024-09-14T06:29:26.685 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 0.49611543699999999, 2024-09-14T06:29:26.685 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.048672880000000002, 2024-09-14T06:29:26.685 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:29:26.685 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:29:26.685 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:29:26.685 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:29:26.685 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:29:26.685 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:26.685 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 15613 2024-09-14T06:29:26.685 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.685 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:29:26.685 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.685 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:29:26.685 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.143264+0000", 2024-09-14T06:29:26.686 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:26.686 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.686 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.686 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:29:26.686 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.143264+0000", 2024-09-14T06:29:26.686 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:26.686 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.686 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.686 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:29:26.686 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.143557+0000", 2024-09-14T06:29:26.686 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000292572 2024-09-14T06:29:26.686 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.686 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.686 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:29:26.687 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.143560+0000", 2024-09-14T06:29:26.687 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.4690000000000002e-06 2024-09-14T06:29:26.687 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.687 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.687 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:29:26.687 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.143572+0000", 2024-09-14T06:29:26.687 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.1457000000000001e-05 2024-09-14T06:29:26.687 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.687 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.687 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:29:26.687 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.143578+0000", 2024-09-14T06:29:26.687 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.5520000000000001e-06 2024-09-14T06:29:26.687 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.687 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.688 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:29:26.688 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.191838+0000", 2024-09-14T06:29:26.688 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.048259947999999997 2024-09-14T06:29:26.688 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.688 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.688 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:29:26.688 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.191881+0000", 2024-09-14T06:29:26.688 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.3058e-05 2024-09-14T06:29:26.688 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.688 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.688 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:29:26.688 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.191937+0000", 2024-09-14T06:29:26.688 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.5823999999999999e-05 2024-09-14T06:29:26.688 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:26.689 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:26.689 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:26.689 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.689 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.689 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:15614 3.b 3:d24f5f7d:::rbd_data.112d92027590.000000000000000d:f [read 914432~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:29:26.689 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:26.143597+0000", 2024-09-14T06:29:26.689 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 0.49578286199999999, 2024-09-14T06:29:26.689 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.048548172000000001, 2024-09-14T06:29:26.689 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:29:26.689 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:29:26.689 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:29:26.689 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:29:26.689 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:29:26.690 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:26.690 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 15614 2024-09-14T06:29:26.690 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.690 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:29:26.690 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.690 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:29:26.690 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.143597+0000", 2024-09-14T06:29:26.690 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:26.690 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.690 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.690 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:29:26.690 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.143597+0000", 2024-09-14T06:29:26.690 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:26.690 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.691 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.691 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:29:26.691 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.143598+0000", 2024-09-14T06:29:26.691 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.1099999999999996e-07 2024-09-14T06:29:26.691 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.691 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.691 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:29:26.691 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.143600+0000", 2024-09-14T06:29:26.691 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.4210000000000002e-06 2024-09-14T06:29:26.691 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.691 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.692 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:29:26.692 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.143608+0000", 2024-09-14T06:29:26.692 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.2379999999999997e-06 2024-09-14T06:29:26.692 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.692 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.692 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:29:26.692 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.143613+0000", 2024-09-14T06:29:26.692 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.1909999999999999e-06 2024-09-14T06:29:26.692 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.692 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.692 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:29:26.692 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.191970+0000", 2024-09-14T06:29:26.692 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.048356757 2024-09-14T06:29:26.692 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.692 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.693 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:29:26.693 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.192049+0000", 2024-09-14T06:29:26.693 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.9252000000000006e-05 2024-09-14T06:29:26.693 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.693 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.693 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:29:26.693 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.192145+0000", 2024-09-14T06:29:26.693 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.5802000000000002e-05 2024-09-14T06:29:26.693 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:26.693 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:26.693 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:26.693 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.693 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.694 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:15615 3.b 3:d24f5f7d:::rbd_data.112d92027590.000000000000000d:f [read 924672~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:29:26.694 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:26.143621+0000", 2024-09-14T06:29:26.694 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 0.49575837900000003, 2024-09-14T06:29:26.694 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.04866728, 2024-09-14T06:29:26.694 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:29:26.694 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:29:26.694 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:29:26.694 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:29:26.694 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:29:26.694 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:26.695 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 15615 2024-09-14T06:29:26.695 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.695 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:29:26.695 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.695 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:29:26.695 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.143621+0000", 2024-09-14T06:29:26.695 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:26.695 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.695 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.695 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:29:26.695 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.143621+0000", 2024-09-14T06:29:26.695 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:26.696 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.696 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.696 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:29:26.696 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.145193+0000", 2024-09-14T06:29:26.696 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0015719480000000001 2024-09-14T06:29:26.696 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.696 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.696 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:29:26.696 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.145198+0000", 2024-09-14T06:29:26.696 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.8770000000000001e-06 2024-09-14T06:29:26.696 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.696 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.696 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:29:26.697 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.145207+0000", 2024-09-14T06:29:26.697 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.0620000000000007e-06 2024-09-14T06:29:26.697 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.697 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.697 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:29:26.697 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.145214+0000", 2024-09-14T06:29:26.697 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.156e-06 2024-09-14T06:29:26.697 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.697 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.697 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:29:26.697 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.192180+0000", 2024-09-14T06:29:26.697 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.046966624999999998 2024-09-14T06:29:26.697 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.698 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.698 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:29:26.698 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.192231+0000", 2024-09-14T06:29:26.698 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.0754999999999998e-05 2024-09-14T06:29:26.698 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.698 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.698 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:29:26.698 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.192289+0000", 2024-09-14T06:29:26.698 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.7856999999999998e-05 2024-09-14T06:29:26.698 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:26.698 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:26.698 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:26.698 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.698 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.699 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:15704 3.b 3:d24f5f7d:::rbd_data.112d92027590.000000000000000d:f [read 137216~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:29:26.699 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:26.148323+0000", 2024-09-14T06:29:26.699 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 0.49105685100000002, 2024-09-14T06:29:26.699 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.048538987999999998, 2024-09-14T06:29:26.699 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:29:26.699 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:29:26.699 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:29:26.699 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:29:26.699 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:29:26.699 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:26.699 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 15704 2024-09-14T06:29:26.699 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.699 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:29:26.700 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.700 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:29:26.700 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.148323+0000", 2024-09-14T06:29:26.700 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:26.700 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.700 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.700 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:29:26.700 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.148323+0000", 2024-09-14T06:29:26.700 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:26.700 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.700 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.700 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:29:26.700 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.148448+0000", 2024-09-14T06:29:26.700 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00012475999999999999 2024-09-14T06:29:26.701 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.701 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.701 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:29:26.701 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.148451+0000", 2024-09-14T06:29:26.701 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.5370000000000002e-06 2024-09-14T06:29:26.701 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.701 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.701 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:29:26.701 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.148460+0000", 2024-09-14T06:29:26.702 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.2949999999999994e-06 2024-09-14T06:29:26.702 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.702 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.702 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:29:26.702 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.148465+0000", 2024-09-14T06:29:26.702 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.7570000000000001e-06 2024-09-14T06:29:26.703 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.703 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.703 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:29:26.703 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.196699+0000", 2024-09-14T06:29:26.703 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.048233542999999997 2024-09-14T06:29:26.703 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.704 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.704 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:29:26.704 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.196766+0000", 2024-09-14T06:29:26.704 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.7259000000000001e-05 2024-09-14T06:29:26.704 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.704 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.704 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:29:26.705 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.196862+0000", 2024-09-14T06:29:26.705 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.5836999999999997e-05 2024-09-14T06:29:26.705 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:26.705 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:26.705 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:26.705 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.705 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.705 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:15705 3.b 3:d24f5f7d:::rbd_data.112d92027590.000000000000000d:f [read 147456~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:29:26.706 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:26.148475+0000", 2024-09-14T06:29:26.706 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 0.49090499100000001, 2024-09-14T06:29:26.706 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.048589192000000003, 2024-09-14T06:29:26.706 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:29:26.706 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:29:26.706 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:29:26.706 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:29:26.707 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:29:26.707 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:26.707 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 15705 2024-09-14T06:29:26.707 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.707 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:29:26.707 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.707 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:29:26.708 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.148475+0000", 2024-09-14T06:29:26.708 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:26.708 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.708 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.708 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:29:26.708 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.148475+0000", 2024-09-14T06:29:26.709 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:26.709 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.709 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.709 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:29:26.709 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.148565+0000", 2024-09-14T06:29:26.709 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.9876000000000005e-05 2024-09-14T06:29:26.709 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.710 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.710 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:29:26.710 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.148568+0000", 2024-09-14T06:29:26.710 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.2210000000000002e-06 2024-09-14T06:29:26.710 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.710 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.710 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:29:26.710 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.148580+0000", 2024-09-14T06:29:26.710 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.2442e-05 2024-09-14T06:29:26.710 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.710 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.710 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:29:26.711 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.148590+0000", 2024-09-14T06:29:26.711 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.0860000000000008e-06 2024-09-14T06:29:26.711 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.711 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.711 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:29:26.711 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.196893+0000", 2024-09-14T06:29:26.711 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.048303328999999999 2024-09-14T06:29:26.711 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.711 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.711 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:29:26.711 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.196968+0000", 2024-09-14T06:29:26.711 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.4690000000000005e-05 2024-09-14T06:29:26.711 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.711 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.712 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:29:26.712 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.197064+0000", 2024-09-14T06:29:26.712 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.6547999999999998e-05 2024-09-14T06:29:26.712 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:26.712 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:26.712 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:26.712 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.712 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.712 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:15706 3.b 3:d24f5f7d:::rbd_data.112d92027590.000000000000000d:f [read 157696~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:29:26.712 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:26.148603+0000", 2024-09-14T06:29:26.712 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 0.49077679800000001, 2024-09-14T06:29:26.712 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.048652025000000002, 2024-09-14T06:29:26.712 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:29:26.713 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:29:26.713 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:29:26.713 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:29:26.713 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:29:26.713 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:26.713 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 15706 2024-09-14T06:29:26.713 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.713 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:29:26.713 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.713 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:29:26.713 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.148603+0000", 2024-09-14T06:29:26.713 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:26.713 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.714 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.714 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:29:26.714 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.148603+0000", 2024-09-14T06:29:26.714 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:26.714 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.714 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.714 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:29:26.714 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.148791+0000", 2024-09-14T06:29:26.714 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00018775700000000001 2024-09-14T06:29:26.714 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.714 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.714 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:29:26.714 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.148794+0000", 2024-09-14T06:29:26.715 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.9550000000000001e-06 2024-09-14T06:29:26.715 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.715 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.715 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:29:26.715 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.148802+0000", 2024-09-14T06:29:26.715 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.7789999999999996e-06 2024-09-14T06:29:26.715 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.715 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.715 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:29:26.715 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.148814+0000", 2024-09-14T06:29:26.715 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.2886e-05 2024-09-14T06:29:26.715 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.715 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.715 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:29:26.716 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.197104+0000", 2024-09-14T06:29:26.716 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.048289683 2024-09-14T06:29:26.716 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.716 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.716 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:29:26.716 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.197171+0000", 2024-09-14T06:29:26.716 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.7137000000000005e-05 2024-09-14T06:29:26.716 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.716 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.716 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:29:26.716 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.197255+0000", 2024-09-14T06:29:26.716 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.3827999999999998e-05 2024-09-14T06:29:26.716 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:26.716 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:26.717 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:26.717 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.717 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.717 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:15707 3.b 3:d24f5f7d:::rbd_data.112d92027590.000000000000000d:f [read 167936~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:29:26.717 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:26.148826+0000", 2024-09-14T06:29:26.717 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 0.49055419500000003, 2024-09-14T06:29:26.717 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.048578723999999997, 2024-09-14T06:29:26.717 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:29:26.717 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:29:26.717 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:29:26.717 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:29:26.717 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:29:26.718 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:26.718 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 15707 2024-09-14T06:29:26.718 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.718 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:29:26.718 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.718 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:29:26.718 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.148826+0000", 2024-09-14T06:29:26.718 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:26.718 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.718 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.718 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:29:26.718 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.148826+0000", 2024-09-14T06:29:26.718 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:26.718 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.719 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.719 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:29:26.719 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.149053+0000", 2024-09-14T06:29:26.719 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00022741100000000001 2024-09-14T06:29:26.719 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.719 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.719 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:29:26.719 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.149060+0000", 2024-09-14T06:29:26.719 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.8040000000000004e-06 2024-09-14T06:29:26.719 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.719 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.719 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:29:26.719 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.149069+0000", 2024-09-14T06:29:26.719 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.8189999999999998e-06 2024-09-14T06:29:26.720 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.720 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.720 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:29:26.720 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.149075+0000", 2024-09-14T06:29:26.720 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.2079999999999997e-06 2024-09-14T06:29:26.720 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.720 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.720 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:29:26.720 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.197299+0000", 2024-09-14T06:29:26.720 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.048223749000000003 2024-09-14T06:29:26.720 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.720 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.720 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:29:26.721 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.197343+0000", 2024-09-14T06:29:26.721 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.4721000000000001e-05 2024-09-14T06:29:26.721 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.721 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.721 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:29:26.721 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.197404+0000", 2024-09-14T06:29:26.721 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.1011999999999998e-05 2024-09-14T06:29:26.721 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:26.721 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:26.721 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:26.721 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.721 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.721 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:15708 3.b 3:d24f5f7d:::rbd_data.112d92027590.000000000000000d:f [read 178176~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:29:26.722 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:26.149084+0000", 2024-09-14T06:29:26.722 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 0.49029557000000001, 2024-09-14T06:29:26.722 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.048528769999999999, 2024-09-14T06:29:26.722 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:29:26.722 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:29:26.722 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:29:26.722 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:29:26.722 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:29:26.722 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:26.722 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 15708 2024-09-14T06:29:26.722 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.722 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:29:26.722 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.722 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:29:26.722 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.149084+0000", 2024-09-14T06:29:26.723 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:26.723 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.723 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.723 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:29:26.723 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.149084+0000", 2024-09-14T06:29:26.723 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:26.723 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.723 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.723 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:29:26.723 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.150587+0000", 2024-09-14T06:29:26.723 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.001503187 2024-09-14T06:29:26.723 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.723 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.724 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:29:26.724 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.150592+0000", 2024-09-14T06:29:26.724 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.07e-06 2024-09-14T06:29:26.724 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.724 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.724 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:29:26.724 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.150602+0000", 2024-09-14T06:29:26.724 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.0584999999999999e-05 2024-09-14T06:29:26.724 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.724 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.724 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:29:26.724 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.150609+0000", 2024-09-14T06:29:26.725 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.8560000000000001e-06 2024-09-14T06:29:26.725 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.725 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.725 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:29:26.725 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.197446+0000", 2024-09-14T06:29:26.725 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.046837328999999997 2024-09-14T06:29:26.725 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.725 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.725 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:29:26.725 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.197530+0000", 2024-09-14T06:29:26.725 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.3493999999999997e-05 2024-09-14T06:29:26.725 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.725 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.726 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:29:26.726 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.197613+0000", 2024-09-14T06:29:26.726 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.3249000000000002e-05 2024-09-14T06:29:26.726 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:26.726 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:26.726 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:26.726 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.726 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.726 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:15736 3.b 3:d24f5f7d:::rbd_data.112d92027590.000000000000000d:f [read 464896~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:29:26.726 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:26.154778+0000", 2024-09-14T06:29:26.726 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 0.48460155500000002, 2024-09-14T06:29:26.726 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.048624703999999998, 2024-09-14T06:29:26.726 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:29:26.726 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:29:26.726 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:29:26.727 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:29:26.727 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:29:26.727 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:26.727 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 15736 2024-09-14T06:29:26.727 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.727 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:29:26.727 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.727 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:29:26.727 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.154778+0000", 2024-09-14T06:29:26.727 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:26.727 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.727 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.727 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:29:26.728 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.154778+0000", 2024-09-14T06:29:26.728 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:26.728 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.728 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.728 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:29:26.728 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.154847+0000", 2024-09-14T06:29:26.728 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.8374999999999995e-05 2024-09-14T06:29:26.728 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.728 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.728 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:29:26.728 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.154851+0000", 2024-09-14T06:29:26.728 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.4830000000000001e-06 2024-09-14T06:29:26.728 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.728 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.729 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:29:26.729 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.154863+0000", 2024-09-14T06:29:26.729 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.188e-05 2024-09-14T06:29:26.729 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.729 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.729 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:29:26.729 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.154870+0000", 2024-09-14T06:29:26.729 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.8159999999999996e-06 2024-09-14T06:29:26.729 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.729 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.729 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:29:26.729 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.203218+0000", 2024-09-14T06:29:26.729 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.048348440999999999 2024-09-14T06:29:26.729 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.730 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.730 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:29:26.730 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.203302+0000", 2024-09-14T06:29:26.730 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.3764999999999995e-05 2024-09-14T06:29:26.730 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.730 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.730 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:29:26.730 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.203403+0000", 2024-09-14T06:29:26.730 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000100944 2024-09-14T06:29:26.730 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:26.730 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:26.730 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:26.730 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.730 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.731 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:15737 3.b 3:d24f5f7d:::rbd_data.112d92027590.000000000000000d:f [read 475136~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:29:26.731 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:26.154883+0000", 2024-09-14T06:29:26.731 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 0.484497342, 2024-09-14T06:29:26.731 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.048784932000000003, 2024-09-14T06:29:26.731 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:29:26.731 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:29:26.731 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:29:26.731 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:29:26.731 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:29:26.731 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:26.731 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 15737 2024-09-14T06:29:26.731 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.731 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:29:26.732 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.732 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:29:26.732 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.154883+0000", 2024-09-14T06:29:26.732 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:26.732 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.732 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.732 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:29:26.732 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.154883+0000", 2024-09-14T06:29:26.732 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:26.732 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.732 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.732 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:29:26.732 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.155040+0000", 2024-09-14T06:29:26.732 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000157236 2024-09-14T06:29:26.733 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.733 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.733 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:29:26.733 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.155044+0000", 2024-09-14T06:29:26.733 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.1330000000000001e-06 2024-09-14T06:29:26.733 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.733 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.733 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:29:26.733 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.155053+0000", 2024-09-14T06:29:26.733 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.8869999999999998e-06 2024-09-14T06:29:26.733 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.733 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.733 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:29:26.733 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.155058+0000", 2024-09-14T06:29:26.734 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.3940000000000004e-06 2024-09-14T06:29:26.734 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.734 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.734 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:29:26.734 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.203478+0000", 2024-09-14T06:29:26.734 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.048420073000000001 2024-09-14T06:29:26.734 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.734 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.734 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:29:26.734 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.203566+0000", 2024-09-14T06:29:26.734 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.7620999999999993e-05 2024-09-14T06:29:26.734 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.734 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.734 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:29:26.734 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.203667+0000", 2024-09-14T06:29:26.735 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000101588 2024-09-14T06:29:26.735 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:26.735 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:26.735 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:26.735 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.735 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.735 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:15738 3.b 3:d24f5f7d:::rbd_data.112d92027590.000000000000000d:f [read 485376~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:29:26.735 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:26.155069+0000", 2024-09-14T06:29:26.735 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 0.48431041200000002, 2024-09-14T06:29:26.735 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.048810049000000001, 2024-09-14T06:29:26.735 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:29:26.735 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:29:26.735 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:29:26.736 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:29:26.736 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:29:26.736 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:26.736 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 15738 2024-09-14T06:29:26.736 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.736 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:29:26.736 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.736 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:29:26.736 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.155069+0000", 2024-09-14T06:29:26.736 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:26.736 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.736 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.736 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:29:26.736 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.155069+0000", 2024-09-14T06:29:26.737 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:26.737 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.737 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.737 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:29:26.737 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.155278+0000", 2024-09-14T06:29:26.737 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00020823400000000001 2024-09-14T06:29:26.737 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.737 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.737 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:29:26.737 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.155282+0000", 2024-09-14T06:29:26.737 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.4730000000000002e-06 2024-09-14T06:29:26.737 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.737 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.738 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:29:26.738 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.155294+0000", 2024-09-14T06:29:26.738 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.1347e-05 2024-09-14T06:29:26.738 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.738 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.738 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:29:26.738 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.155302+0000", 2024-09-14T06:29:26.738 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.7339999999999997e-06 2024-09-14T06:29:26.738 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.738 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.738 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:29:26.738 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.203701+0000", 2024-09-14T06:29:26.738 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.048399144999999998 2024-09-14T06:29:26.738 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.739 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.739 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:29:26.739 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.203778+0000", 2024-09-14T06:29:26.739 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.6169999999999997e-05 2024-09-14T06:29:26.739 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.739 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.739 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:29:26.739 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.203880+0000", 2024-09-14T06:29:26.739 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000101946 2024-09-14T06:29:26.739 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:26.739 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:26.739 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:26.739 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.739 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.739 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:15739 3.b 3:d24f5f7d:::rbd_data.112d92027590.000000000000000d:f [read 495616~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:29:26.740 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:26.155331+0000", 2024-09-14T06:29:26.740 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 0.48404889600000001, 2024-09-14T06:29:26.740 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.048708649999999999, 2024-09-14T06:29:26.740 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:29:26.740 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:29:26.740 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:29:26.740 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:29:26.740 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:29:26.740 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:26.740 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 15739 2024-09-14T06:29:26.740 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.740 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:29:26.740 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.741 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:29:26.741 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.155331+0000", 2024-09-14T06:29:26.741 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:26.741 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.741 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.741 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:29:26.741 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.155331+0000", 2024-09-14T06:29:26.741 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:26.741 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.741 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.741 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:29:26.741 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.155526+0000", 2024-09-14T06:29:26.741 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00019500299999999999 2024-09-14T06:29:26.741 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.742 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.742 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:29:26.742 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.155532+0000", 2024-09-14T06:29:26.742 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.6620000000000002e-06 2024-09-14T06:29:26.742 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.742 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.742 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:29:26.742 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.155541+0000", 2024-09-14T06:29:26.742 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.2240000000000001e-06 2024-09-14T06:29:26.742 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.742 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.742 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:29:26.742 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.155547+0000", 2024-09-14T06:29:26.743 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.4559999999999997e-06 2024-09-14T06:29:26.743 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.743 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.743 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:29:26.743 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.203917+0000", 2024-09-14T06:29:26.743 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.048370017000000001 2024-09-14T06:29:26.743 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.743 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.743 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:29:26.743 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.203970+0000", 2024-09-14T06:29:26.743 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.2308999999999997e-05 2024-09-14T06:29:26.743 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.743 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.743 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:29:26.744 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.204040+0000", 2024-09-14T06:29:26.744 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.9979000000000002e-05 2024-09-14T06:29:26.744 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:26.744 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:26.744 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:26.744 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.744 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.744 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:15740 3.b 3:d24f5f7d:::rbd_data.112d92027590.000000000000000d:f [read 505856~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:29:26.744 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:26.155558+0000", 2024-09-14T06:29:26.744 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 0.48382177199999998, 2024-09-14T06:29:26.744 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.048618724000000002, 2024-09-14T06:29:26.744 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:29:26.744 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:29:26.744 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:29:26.745 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:29:26.745 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:29:26.745 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:26.745 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 15740 2024-09-14T06:29:26.745 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.745 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:29:26.745 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.745 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:29:26.745 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.155558+0000", 2024-09-14T06:29:26.745 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:26.745 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.745 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.745 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:29:26.746 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.155558+0000", 2024-09-14T06:29:26.746 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:26.746 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.746 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.746 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:29:26.746 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.155766+0000", 2024-09-14T06:29:26.746 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00020751999999999999 2024-09-14T06:29:26.746 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.746 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.746 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:29:26.746 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.155769+0000", 2024-09-14T06:29:26.746 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.6500000000000002e-06 2024-09-14T06:29:26.746 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.746 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.747 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:29:26.747 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.155779+0000", 2024-09-14T06:29:26.747 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.0179e-05 2024-09-14T06:29:26.747 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.747 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.747 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:29:26.747 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.155785+0000", 2024-09-14T06:29:26.747 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.9120000000000003e-06 2024-09-14T06:29:26.747 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.747 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.747 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:29:26.747 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.204072+0000", 2024-09-14T06:29:26.747 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.048286463000000002 2024-09-14T06:29:26.747 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.747 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.748 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:29:26.748 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.204119+0000", 2024-09-14T06:29:26.748 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.7023e-05 2024-09-14T06:29:26.748 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.748 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.748 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:29:26.748 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.204177+0000", 2024-09-14T06:29:26.748 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.7976999999999997e-05 2024-09-14T06:29:26.748 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:26.748 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:26.748 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:26.748 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.748 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.748 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:15742 3.b 3:d24f5f7d:::rbd_data.112d92027590.000000000000000d:f [read 526336~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:29:26.749 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:26.156014+0000", 2024-09-14T06:29:26.749 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 0.483366187, 2024-09-14T06:29:26.749 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.048548612999999997, 2024-09-14T06:29:26.749 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:29:26.749 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:29:26.749 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:29:26.749 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:29:26.749 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:29:26.749 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:26.749 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 15742 2024-09-14T06:29:26.749 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.749 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:29:26.749 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.749 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:29:26.750 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.156014+0000", 2024-09-14T06:29:26.750 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:26.750 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.750 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.750 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:29:26.750 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.156014+0000", 2024-09-14T06:29:26.750 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:26.750 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.750 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.750 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:29:26.750 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.156246+0000", 2024-09-14T06:29:26.750 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00023262700000000001 2024-09-14T06:29:26.750 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.750 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.750 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:29:26.751 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.156251+0000", 2024-09-14T06:29:26.751 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.0209999999999999e-06 2024-09-14T06:29:26.751 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.751 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.751 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:29:26.751 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.156262+0000", 2024-09-14T06:29:26.751 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.0168e-05 2024-09-14T06:29:26.751 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.751 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.751 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:29:26.751 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.156269+0000", 2024-09-14T06:29:26.751 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.0999999999999998e-06 2024-09-14T06:29:26.751 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.752 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.752 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:29:26.752 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.204363+0000", 2024-09-14T06:29:26.752 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.048094455000000001 2024-09-14T06:29:26.752 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.752 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.752 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:29:26.752 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.204452+0000", 2024-09-14T06:29:26.752 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.8899000000000004e-05 2024-09-14T06:29:26.752 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.752 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.752 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:29:26.752 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.204562+0000", 2024-09-14T06:29:26.752 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000110343 2024-09-14T06:29:26.753 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:26.753 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:26.753 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:26.753 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.753 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.753 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:15746 3.b 3:d24f5f7d:::rbd_data.112d92027590.000000000000000d:f [read 567296~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:29:26.753 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:26.156734+0000", 2024-09-14T06:29:26.753 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 0.48264569699999998, 2024-09-14T06:29:26.753 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.048564930999999999, 2024-09-14T06:29:26.753 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:29:26.753 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:29:26.753 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:29:26.753 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:29:26.753 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:29:26.754 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:26.754 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 15746 2024-09-14T06:29:26.754 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.754 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:29:26.754 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.754 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:29:26.754 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.156734+0000", 2024-09-14T06:29:26.754 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:26.754 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.754 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.754 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:29:26.754 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.156734+0000", 2024-09-14T06:29:26.755 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:26.755 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.755 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.755 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:29:26.755 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.156952+0000", 2024-09-14T06:29:26.755 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000217844 2024-09-14T06:29:26.755 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.755 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.755 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:29:26.755 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.156956+0000", 2024-09-14T06:29:26.755 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.0790000000000002e-06 2024-09-14T06:29:26.755 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.755 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.755 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:29:26.756 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.156966+0000", 2024-09-14T06:29:26.756 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.4150000000000005e-06 2024-09-14T06:29:26.756 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.756 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.756 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:29:26.756 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.156972+0000", 2024-09-14T06:29:26.756 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.365e-06 2024-09-14T06:29:26.756 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.756 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.756 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:29:26.756 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.205128+0000", 2024-09-14T06:29:26.756 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.048155833000000002 2024-09-14T06:29:26.756 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.756 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.757 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:29:26.757 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.205205+0000", 2024-09-14T06:29:26.757 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.7250999999999996e-05 2024-09-14T06:29:26.757 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:26.757 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:26.757 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:29:26.757 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.205299+0000", 2024-09-14T06:29:26.757 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.4143999999999999e-05 2024-09-14T06:29:26.757 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:26.757 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:26.757 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:26.757 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:26.757 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:26.757 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-14T06:29:26.758 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-14T06:29:26.760 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-14T06:29:26.760 INFO:tasks.ceph.osd.5.smithi138.stderr:2024-09-14T06:29:26.758+0000 7fe13746f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33618) UID: 0 2024-09-14T06:29:26.860 INFO:tasks.ceph.osd.3.smithi094.stderr:2024-09-14T06:29:26.859+0000 7feeeafe8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33655) UID: 0 2024-09-14T06:29:26.862 INFO:teuthology.orchestra.run.smithi138.stdout:{ 2024-09-14T06:29:26.862 INFO:teuthology.orchestra.run.smithi138.stdout: "ops": [], 2024-09-14T06:29:26.862 INFO:teuthology.orchestra.run.smithi138.stdout: "num_ops": 0 2024-09-14T06:29:26.862 INFO:teuthology.orchestra.run.smithi138.stdout:} 2024-09-14T06:29:26.871 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-14T06:29:26.961 INFO:tasks.ceph.osd.2.smithi094.stderr:2024-09-14T06:29:26.960+0000 7faf3269b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33649) UID: 0 2024-09-14T06:29:26.988 INFO:teuthology.orchestra.run.smithi138.stdout:{ 2024-09-14T06:29:26.988 INFO:teuthology.orchestra.run.smithi138.stdout: "ops": [], 2024-09-14T06:29:26.988 INFO:teuthology.orchestra.run.smithi138.stdout: "complaint_time": 30, 2024-09-14T06:29:26.988 INFO:teuthology.orchestra.run.smithi138.stdout: "num_blocked_ops": 0 2024-09-14T06:29:26.988 INFO:teuthology.orchestra.run.smithi138.stdout:} 2024-09-14T06:29:27.000 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-14T06:29:27.061 INFO:tasks.ceph.osd.3.smithi094.stderr:2024-09-14T06:29:27.060+0000 7feeeafe8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33655) UID: 0 2024-09-14T06:29:27.140 INFO:teuthology.orchestra.run.smithi138.stdout:{ 2024-09-14T06:29:27.140 INFO:teuthology.orchestra.run.smithi138.stdout: "size": 20, 2024-09-14T06:29:27.140 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 600, 2024-09-14T06:29:27.141 INFO:teuthology.orchestra.run.smithi138.stdout: "ops": [ 2024-09-14T06:29:27.141 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:27.141 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:19328 3.3 3:c34e5e51:::rbd_data.112d92027590.0000000000000024:f [read 1926144~1024 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:29:27.141 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:27.059768+0000", 2024-09-14T06:29:27.141 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 0.078553873999999996, 2024-09-14T06:29:27.141 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046369147999999999, 2024-09-14T06:29:27.141 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:29:27.141 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:29:27.141 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:29:27.141 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:29:27.141 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:29:27.142 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:27.142 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 19328 2024-09-14T06:29:27.142 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:27.142 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:29:27.142 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:27.142 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:29:27.142 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.059768+0000", 2024-09-14T06:29:27.142 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:27.142 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:27.142 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:27.142 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:29:27.143 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.059768+0000", 2024-09-14T06:29:27.143 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:27.143 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:27.143 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:27.143 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:29:27.143 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.059874+0000", 2024-09-14T06:29:27.143 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00010551400000000001 2024-09-14T06:29:27.143 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:27.143 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:27.143 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:29:27.143 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.059877+0000", 2024-09-14T06:29:27.143 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.8279999999999999e-06 2024-09-14T06:29:27.143 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:27.144 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:27.144 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:29:27.144 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.059884+0000", 2024-09-14T06:29:27.144 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.6389999999999992e-06 2024-09-14T06:29:27.144 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:27.144 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:27.144 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:29:27.144 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.059890+0000", 2024-09-14T06:29:27.144 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.49e-06 2024-09-14T06:29:27.144 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:27.144 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:27.145 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:29:27.145 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.105804+0000", 2024-09-14T06:29:27.145 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.045914242000000001 2024-09-14T06:29:27.145 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:27.145 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:27.145 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:29:27.145 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.105886+0000", 2024-09-14T06:29:27.145 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.2305999999999994e-05 2024-09-14T06:29:27.145 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:27.145 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:27.145 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:29:27.145 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.106138+0000", 2024-09-14T06:29:27.145 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.000251129 2024-09-14T06:29:27.145 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:27.146 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:29:27.146 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:27.146 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:27.146 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:27.146 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:19329 3.3 3:c34e5e51:::rbd_data.112d92027590.0000000000000024:f [read 1936384~1024 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:29:27.146 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:27.059899+0000", 2024-09-14T06:29:27.146 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 0.078423351000000002, 2024-09-14T06:29:27.146 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046572122000000001, 2024-09-14T06:29:27.146 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:29:27.146 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:29:27.146 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:29:27.146 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:29:27.146 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:29:27.146 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:27.147 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 19329 2024-09-14T06:29:27.147 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:27.147 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:29:27.147 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:27.147 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:29:27.147 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.059899+0000", 2024-09-14T06:29:27.147 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:27.147 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:27.147 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:27.147 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:29:27.147 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.059899+0000", 2024-09-14T06:29:27.147 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:27.148 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:27.148 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:27.148 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:29:27.148 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062501+0000", 2024-09-14T06:29:27.148 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.0026016419999999999 2024-09-14T06:29:27.148 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:27.148 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:27.148 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:29:27.148 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062514+0000", 2024-09-14T06:29:27.148 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.2947e-05 2024-09-14T06:29:27.148 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:27.148 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:27.148 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:29:27.149 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062523+0000", 2024-09-14T06:29:27.149 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.5519999999999993e-06 2024-09-14T06:29:27.149 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:27.149 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:27.149 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:29:27.149 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062534+0000", 2024-09-14T06:29:27.149 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.1416e-05 2024-09-14T06:29:27.150 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:27.150 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:27.150 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:29:27.150 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.106179+0000", 2024-09-14T06:29:27.150 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.043644049999999997 2024-09-14T06:29:27.150 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:27.150 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:27.150 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:29:27.150 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.106262+0000", 2024-09-14T06:29:27.150 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.3178999999999998e-05 2024-09-14T06:29:27.150 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:27.150 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:27.150 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:29:27.150 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.106471+0000", 2024-09-14T06:29:27.150 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00020933599999999999 2024-09-14T06:29:27.151 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:27.151 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:29:27.151 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:27.151 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:27.151 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:27.151 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:19351 3.3 3:c34e5e51:::rbd_data.112d92027590.0000000000000024:f [read 1159168~1024 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:29:27.151 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:27.062766+0000", 2024-09-14T06:29:27.151 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 0.075556126000000001, 2024-09-14T06:29:27.151 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046099454999999998, 2024-09-14T06:29:27.151 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:29:27.151 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:29:27.151 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:29:27.151 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:29:27.151 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:29:27.151 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:27.152 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 19351 2024-09-14T06:29:27.152 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:27.152 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:29:27.152 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:27.152 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:29:27.152 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062766+0000", 2024-09-14T06:29:27.152 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:27.152 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:27.152 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:27.152 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:29:27.152 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062766+0000", 2024-09-14T06:29:27.152 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:27.152 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:27.152 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:27.153 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:29:27.153 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062766+0000", 2024-09-14T06:29:27.153 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.6e-07 2024-09-14T06:29:27.153 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:27.153 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:27.154 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:29:27.154 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062767+0000", 2024-09-14T06:29:27.154 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.0070000000000001e-06 2024-09-14T06:29:27.154 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:27.154 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:27.154 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:29:27.154 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062785+0000", 2024-09-14T06:29:27.154 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.7884999999999999e-05 2024-09-14T06:29:27.154 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:27.154 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:27.155 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:29:27.155 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062790+0000", 2024-09-14T06:29:27.155 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.0900000000000004e-06 2024-09-14T06:29:27.155 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:27.155 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:27.155 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:29:27.155 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.108735+0000", 2024-09-14T06:29:27.155 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.045944411999999997 2024-09-14T06:29:27.156 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:27.156 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:27.156 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:29:27.156 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.108809+0000", 2024-09-14T06:29:27.156 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.4157000000000002e-05 2024-09-14T06:29:27.156 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:27.156 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:27.156 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:29:27.156 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.108866+0000", 2024-09-14T06:29:27.157 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.6644000000000002e-05 2024-09-14T06:29:27.157 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:27.157 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:29:27.157 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:27.157 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:27.157 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:27.157 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:19352 3.3 3:c34e5e51:::rbd_data.112d92027590.0000000000000024:f [read 1169408~1024 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:29:27.157 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:27.062798+0000", 2024-09-14T06:29:27.157 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 0.075524715000000006, 2024-09-14T06:29:27.157 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046255900000000003, 2024-09-14T06:29:27.158 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:29:27.158 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:29:27.158 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:29:27.158 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:29:27.158 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:29:27.158 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:27.158 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 19352 2024-09-14T06:29:27.158 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:27.158 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:29:27.158 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:27.158 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:29:27.158 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062798+0000", 2024-09-14T06:29:27.158 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:27.158 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:27.159 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:27.159 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:29:27.159 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062798+0000", 2024-09-14T06:29:27.159 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:27.159 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:27.159 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:27.159 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:29:27.159 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062798+0000", 2024-09-14T06:29:27.159 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.37e-07 2024-09-14T06:29:27.159 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:27.159 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:27.159 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:29:27.159 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062800+0000", 2024-09-14T06:29:27.159 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.0269999999999998e-06 2024-09-14T06:29:27.160 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:27.160 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:27.160 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:29:27.160 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062806+0000", 2024-09-14T06:29:27.160 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.6690000000000003e-06 2024-09-14T06:29:27.160 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:27.160 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:27.160 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:29:27.160 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062810+0000", 2024-09-14T06:29:27.160 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.3449999999999999e-06 2024-09-14T06:29:27.160 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:27.160 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:27.160 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:29:27.161 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.108892+0000", 2024-09-14T06:29:27.161 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046081715000000002 2024-09-14T06:29:27.161 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:27.161 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:27.161 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:29:27.161 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.108940+0000", 2024-09-14T06:29:27.161 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.7815999999999998e-05 2024-09-14T06:29:27.161 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:27.161 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:27.161 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:29:27.161 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.109053+0000", 2024-09-14T06:29:27.161 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.000113791 2024-09-14T06:29:27.161 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:27.161 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:29:27.162 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:27.162 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:27.162 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:27.162 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:19353 3.3 3:c34e5e51:::rbd_data.112d92027590.0000000000000024:f [read 1179648~1024 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:29:27.162 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:27.062816+0000", 2024-09-14T06:29:27.162 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 0.075506195999999998, 2024-09-14T06:29:27.162 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046481256999999998, 2024-09-14T06:29:27.162 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:29:27.162 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:29:27.162 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:29:27.162 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:29:27.162 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:29:27.162 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:27.162 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 19353 2024-09-14T06:29:27.163 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:27.163 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:29:27.163 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:27.163 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:29:27.163 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062816+0000", 2024-09-14T06:29:27.163 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:27.163 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:27.163 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:27.163 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:29:27.163 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062816+0000", 2024-09-14T06:29:27.163 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:27.163 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:27.163 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:27.163 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:29:27.164 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062816+0000", 2024-09-14T06:29:27.164 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.2899999999999999e-07 2024-09-14T06:29:27.164 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:27.164 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:27.164 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:29:27.164 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062818+0000", 2024-09-14T06:29:27.164 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.699e-06 2024-09-14T06:29:27.164 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:27.164 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:27.164 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:29:27.164 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062825+0000", 2024-09-14T06:29:27.164 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.878e-06 2024-09-14T06:29:27.164 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:27.165 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:27.165 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:29:27.165 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062830+0000", 2024-09-14T06:29:27.165 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.5410000000000002e-06 2024-09-14T06:29:27.165 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:27.165 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:27.165 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:29:27.165 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.109096+0000", 2024-09-14T06:29:27.165 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046266517 2024-09-14T06:29:27.165 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:27.166 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:27.166 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:29:27.166 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.109190+0000", 2024-09-14T06:29:27.166 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.4000999999999997e-05 2024-09-14T06:29:27.166 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:27.166 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:27.166 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:29:27.166 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.109297+0000", 2024-09-14T06:29:27.166 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.000107292 2024-09-14T06:29:27.166 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:27.166 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:29:27.166 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:27.166 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:27.167 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:27.167 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:19354 3.3 3:c34e5e51:::rbd_data.112d92027590.0000000000000024:f [read 1189888~1024 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:29:27.167 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:27.062842+0000", 2024-09-14T06:29:27.167 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 0.075480741000000004, 2024-09-14T06:29:27.167 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046594257, 2024-09-14T06:29:27.167 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:29:27.167 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:29:27.167 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:29:27.167 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:29:27.167 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:29:27.167 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:27.167 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 19354 2024-09-14T06:29:27.167 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:27.167 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:29:27.167 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:27.168 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:29:27.168 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062842+0000", 2024-09-14T06:29:27.168 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:27.168 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:27.168 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:27.168 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:29:27.168 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062842+0000", 2024-09-14T06:29:27.168 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:27.168 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:27.168 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:27.168 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:29:27.168 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062842+0000", 2024-09-14T06:29:27.168 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.7399999999999999e-07 2024-09-14T06:29:27.169 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:27.169 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:27.169 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:29:27.169 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062859+0000", 2024-09-14T06:29:27.169 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.7010000000000001e-05 2024-09-14T06:29:27.169 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:27.169 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:27.169 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:29:27.169 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062866+0000", 2024-09-14T06:29:27.169 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.9990000000000002e-06 2024-09-14T06:29:27.169 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:27.169 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:27.169 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:29:27.169 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062873+0000", 2024-09-14T06:29:27.169 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.8569999999999997e-06 2024-09-14T06:29:27.170 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:27.170 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:27.170 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:29:27.170 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.109335+0000", 2024-09-14T06:29:27.170 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046461943999999998 2024-09-14T06:29:27.170 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:27.171 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:27.171 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:29:27.171 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.109377+0000", 2024-09-14T06:29:27.171 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.2558999999999997e-05 2024-09-14T06:29:27.171 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:27.171 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:27.171 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:29:27.171 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.109436+0000", 2024-09-14T06:29:27.171 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.8514000000000003e-05 2024-09-14T06:29:27.171 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:27.171 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:29:27.171 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:27.171 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:27.171 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:27.171 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:19355 3.3 3:c34e5e51:::rbd_data.112d92027590.0000000000000024:f [read 1200128~1024 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:29:27.172 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:27.062884+0000", 2024-09-14T06:29:27.172 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 0.075438067999999997, 2024-09-14T06:29:27.172 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046681650999999998, 2024-09-14T06:29:27.172 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:29:27.172 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:29:27.172 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:29:27.172 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:29:27.172 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:29:27.172 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:27.172 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 19355 2024-09-14T06:29:27.172 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:27.172 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:29:27.173 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:27.173 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:29:27.173 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062884+0000", 2024-09-14T06:29:27.173 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:27.173 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:27.173 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:27.173 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:29:27.173 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062884+0000", 2024-09-14T06:29:27.173 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:27.173 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:27.173 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:27.173 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:29:27.173 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062885+0000", 2024-09-14T06:29:27.174 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.8000000000000001e-07 2024-09-14T06:29:27.174 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:27.174 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:27.174 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:29:27.174 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062886+0000", 2024-09-14T06:29:27.174 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.8920000000000001e-06 2024-09-14T06:29:27.174 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:27.174 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:27.174 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:29:27.174 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062892+0000", 2024-09-14T06:29:27.174 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.84e-06 2024-09-14T06:29:27.174 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:27.174 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:27.174 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:29:27.175 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062899+0000", 2024-09-14T06:29:27.175 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.5339999999999996e-06 2024-09-14T06:29:27.175 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:27.175 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:27.175 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:29:27.175 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.109466+0000", 2024-09-14T06:29:27.175 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046566919999999998 2024-09-14T06:29:27.175 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:27.175 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:27.175 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:29:27.175 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.109510+0000", 2024-09-14T06:29:27.175 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.4153000000000002e-05 2024-09-14T06:29:27.175 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:27.175 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:27.176 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:29:27.176 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.109566+0000", 2024-09-14T06:29:27.176 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.5931999999999999e-05 2024-09-14T06:29:27.176 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:27.176 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:29:27.176 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:27.176 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:27.176 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:27.176 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:19356 3.3 3:c34e5e51:::rbd_data.112d92027590.0000000000000024:f [read 1210368~1024 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:29:27.177 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:27.062908+0000", 2024-09-14T06:29:27.177 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 0.075414630999999996, 2024-09-14T06:29:27.177 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046807498000000003, 2024-09-14T06:29:27.177 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:29:27.177 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:29:27.177 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:29:27.177 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:29:27.177 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:29:27.177 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:27.177 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 19356 2024-09-14T06:29:27.177 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:27.177 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:29:27.178 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:27.178 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:29:27.178 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062908+0000", 2024-09-14T06:29:27.178 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:27.178 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:27.178 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:27.178 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:29:27.178 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062908+0000", 2024-09-14T06:29:27.178 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:27.178 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:27.178 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:27.178 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:29:27.178 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062908+0000", 2024-09-14T06:29:27.179 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.5699999999999998e-07 2024-09-14T06:29:27.179 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:27.179 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:27.179 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:29:27.179 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062910+0000", 2024-09-14T06:29:27.179 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.7710000000000001e-06 2024-09-14T06:29:27.179 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:27.179 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:27.179 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:29:27.180 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062920+0000", 2024-09-14T06:29:27.180 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.0654e-05 2024-09-14T06:29:27.180 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:27.180 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:27.180 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:29:27.180 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062931+0000", 2024-09-14T06:29:27.181 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.075e-05 2024-09-14T06:29:27.181 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:27.181 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:27.181 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:29:27.181 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.109595+0000", 2024-09-14T06:29:27.181 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046663713000000002 2024-09-14T06:29:27.182 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:27.182 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:27.182 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:29:27.182 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.109632+0000", 2024-09-14T06:29:27.182 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.7505000000000001e-05 2024-09-14T06:29:27.182 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:27.182 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:27.182 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:29:27.182 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.109715+0000", 2024-09-14T06:29:27.182 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.2748000000000001e-05 2024-09-14T06:29:27.182 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:27.182 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:29:27.182 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:27.183 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:27.183 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:27.183 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:19357 3.3 3:c34e5e51:::rbd_data.112d92027590.0000000000000024:f [read 1220608~1024 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:29:27.183 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:27.062939+0000", 2024-09-14T06:29:27.183 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 0.075383183000000006, 2024-09-14T06:29:27.183 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046920508, 2024-09-14T06:29:27.183 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:29:27.183 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:29:27.183 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:29:27.183 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:29:27.183 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:29:27.183 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:27.183 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 19357 2024-09-14T06:29:27.183 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:27.184 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:29:27.184 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:27.184 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:29:27.184 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062939+0000", 2024-09-14T06:29:27.184 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:27.184 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:27.184 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:27.184 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:29:27.184 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062939+0000", 2024-09-14T06:29:27.184 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:27.184 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:27.184 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:27.184 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:29:27.185 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062940+0000", 2024-09-14T06:29:27.185 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.32e-07 2024-09-14T06:29:27.185 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:27.185 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:27.185 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:29:27.185 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062942+0000", 2024-09-14T06:29:27.185 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.1919999999999999e-06 2024-09-14T06:29:27.185 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:27.185 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:27.185 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:29:27.185 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062948+0000", 2024-09-14T06:29:27.185 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.1149999999999999e-06 2024-09-14T06:29:27.185 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:27.186 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:27.186 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:29:27.186 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062960+0000", 2024-09-14T06:29:27.186 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.1895e-05 2024-09-14T06:29:27.186 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:27.186 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:27.186 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:29:27.187 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.109704+0000", 2024-09-14T06:29:27.187 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046744584999999998 2024-09-14T06:29:27.187 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:27.187 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:27.187 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:29:27.187 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.109777+0000", 2024-09-14T06:29:27.187 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.2282999999999999e-05 2024-09-14T06:29:27.187 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:27.187 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:27.187 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:29:27.187 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.109860+0000", 2024-09-14T06:29:27.187 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.3005999999999998e-05 2024-09-14T06:29:27.187 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:27.187 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:29:27.188 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:27.188 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:27.188 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:27.188 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:19358 3.3 3:c34e5e51:::rbd_data.112d92027590.0000000000000024:f [read 1230848~1024 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:29:27.188 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:27.062967+0000", 2024-09-14T06:29:27.188 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 0.075355553000000006, 2024-09-14T06:29:27.188 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047144471, 2024-09-14T06:29:27.188 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:29:27.188 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:29:27.188 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:29:27.188 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:29:27.188 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:29:27.188 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:27.188 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 19358 2024-09-14T06:29:27.189 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:27.189 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:29:27.189 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:27.189 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:29:27.189 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062967+0000", 2024-09-14T06:29:27.189 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:27.189 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:27.189 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:27.189 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:29:27.189 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062967+0000", 2024-09-14T06:29:27.189 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:27.189 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:27.189 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:27.190 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:29:27.190 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062967+0000", 2024-09-14T06:29:27.190 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.89e-07 2024-09-14T06:29:27.190 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:27.190 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:27.190 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:29:27.190 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062969+0000", 2024-09-14T06:29:27.190 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.9929999999999998e-06 2024-09-14T06:29:27.190 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:27.190 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:27.190 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:29:27.190 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062975+0000", 2024-09-14T06:29:27.190 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.9030000000000001e-06 2024-09-14T06:29:27.190 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:27.190 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:27.191 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:29:27.191 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062979+0000", 2024-09-14T06:29:27.191 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.2250000000000002e-06 2024-09-14T06:29:27.191 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:27.191 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:27.191 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:29:27.191 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.109934+0000", 2024-09-14T06:29:27.191 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046954410000000002 2024-09-14T06:29:27.191 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:27.191 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:27.191 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:29:27.191 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.110007+0000", 2024-09-14T06:29:27.191 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.3829000000000001e-05 2024-09-14T06:29:27.191 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:27.192 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:27.192 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:29:27.192 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.110111+0000", 2024-09-14T06:29:27.192 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00010372199999999999 2024-09-14T06:29:27.192 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:27.192 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:29:27.192 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:27.192 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:27.192 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:27.192 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:19359 3.3 3:c34e5e51:::rbd_data.112d92027590.0000000000000024:f [read 1241088~1024 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:29:27.192 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:27.062987+0000", 2024-09-14T06:29:27.192 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 0.075335463000000005, 2024-09-14T06:29:27.192 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047297182, 2024-09-14T06:29:27.193 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:29:27.193 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:29:27.193 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:29:27.193 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:29:27.193 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:29:27.193 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:27.193 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 19359 2024-09-14T06:29:27.193 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:27.193 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:29:27.193 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:27.193 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:29:27.193 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062987+0000", 2024-09-14T06:29:27.193 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:27.193 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:27.194 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:27.194 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:29:27.194 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062987+0000", 2024-09-14T06:29:27.194 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:27.194 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:27.194 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:27.194 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:29:27.194 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062987+0000", 2024-09-14T06:29:27.194 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.2500000000000001e-07 2024-09-14T06:29:27.194 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:27.194 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:27.194 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:29:27.194 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062989+0000", 2024-09-14T06:29:27.194 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.6080000000000001e-06 2024-09-14T06:29:27.194 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:27.195 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:27.195 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:29:27.195 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062994+0000", 2024-09-14T06:29:27.195 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.5520000000000004e-06 2024-09-14T06:29:27.195 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:27.195 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:27.195 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:29:27.195 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062999+0000", 2024-09-14T06:29:27.195 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.3930000000000001e-06 2024-09-14T06:29:27.195 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:27.195 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:27.195 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:29:27.195 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.110142+0000", 2024-09-14T06:29:27.195 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047143154 2024-09-14T06:29:27.196 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:27.196 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:27.196 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:29:27.196 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.110219+0000", 2024-09-14T06:29:27.196 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.6737999999999996e-05 2024-09-14T06:29:27.196 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:27.196 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:27.196 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:29:27.196 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.110284+0000", 2024-09-14T06:29:27.196 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.5412000000000003e-05 2024-09-14T06:29:27.196 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:27.196 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:29:27.197 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:27.197 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:27.197 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:27.197 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:19360 3.3 3:c34e5e51:::rbd_data.112d92027590.0000000000000024:f [read 1251328~1024 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:29:27.197 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:27.063017+0000", 2024-09-14T06:29:27.197 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 0.075305001999999996, 2024-09-14T06:29:27.197 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047453823999999999, 2024-09-14T06:29:27.197 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:29:27.197 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:29:27.197 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:29:27.198 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:29:27.198 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:29:27.198 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:27.198 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 19360 2024-09-14T06:29:27.198 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:27.198 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:29:27.198 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:27.198 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:29:27.198 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.063017+0000", 2024-09-14T06:29:27.198 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:27.198 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:27.198 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:27.199 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:29:27.199 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.063017+0000", 2024-09-14T06:29:27.199 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:27.199 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:27.199 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:27.199 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:29:27.199 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.064977+0000", 2024-09-14T06:29:27.199 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.0019597080000000001 2024-09-14T06:29:27.199 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:27.199 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:27.199 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:29:27.199 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.064987+0000", 2024-09-14T06:29:27.200 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.9620000000000005e-06 2024-09-14T06:29:27.200 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:27.200 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:27.200 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:29:27.200 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065000+0000", 2024-09-14T06:29:27.200 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.3173000000000001e-05 2024-09-14T06:29:27.200 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:27.200 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:27.200 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:29:27.200 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065005+0000", 2024-09-14T06:29:27.200 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.9960000000000001e-06 2024-09-14T06:29:27.201 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:27.201 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:27.201 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:29:27.201 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.110314+0000", 2024-09-14T06:29:27.201 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.04530899 2024-09-14T06:29:27.201 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:27.201 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:27.201 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:29:27.201 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.110398+0000", 2024-09-14T06:29:27.201 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.3539000000000001e-05 2024-09-14T06:29:27.201 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:27.201 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:27.201 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:29:27.202 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.110471+0000", 2024-09-14T06:29:27.202 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.3455999999999996e-05 2024-09-14T06:29:27.202 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:27.202 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:29:27.202 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:27.202 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:27.202 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:27.202 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:19365 3.3 3:c34e5e51:::rbd_data.112d92027590.0000000000000024:f [read 1302528~1024 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:29:27.202 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:27.065058+0000", 2024-09-14T06:29:27.202 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 0.073264332000000001, 2024-09-14T06:29:27.202 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046105526000000001, 2024-09-14T06:29:27.202 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:29:27.203 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:29:27.203 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:29:27.203 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:29:27.203 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:29:27.203 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:27.203 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 19365 2024-09-14T06:29:27.203 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:27.203 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:29:27.203 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:27.203 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:29:27.203 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065058+0000", 2024-09-14T06:29:27.203 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:27.204 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:27.204 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:27.204 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:29:27.204 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065058+0000", 2024-09-14T06:29:27.204 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:27.204 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:27.204 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:27.204 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:29:27.204 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065058+0000", 2024-09-14T06:29:27.204 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.1899999999999999e-07 2024-09-14T06:29:27.204 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:27.205 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:27.205 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:29:27.205 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065059+0000", 2024-09-14T06:29:27.205 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.1299999999999998e-07 2024-09-14T06:29:27.205 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:27.205 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:27.205 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:29:27.205 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065063+0000", 2024-09-14T06:29:27.205 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.1099999999999996e-06 2024-09-14T06:29:27.205 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:27.205 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:27.205 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:29:27.206 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065070+0000", 2024-09-14T06:29:27.206 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.973e-06 2024-09-14T06:29:27.206 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:27.206 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:27.206 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:29:27.206 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.111014+0000", 2024-09-14T06:29:27.206 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.045943373000000003 2024-09-14T06:29:27.206 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:27.206 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:27.206 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:29:27.206 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.111073+0000", 2024-09-14T06:29:27.206 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.8987999999999998e-05 2024-09-14T06:29:27.207 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:27.207 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:27.207 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:29:27.207 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.111163+0000", 2024-09-14T06:29:27.207 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.0950000000000002e-05 2024-09-14T06:29:27.207 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:27.207 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:29:27.207 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:27.207 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:27.207 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:27.207 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:19366 3.3 3:c34e5e51:::rbd_data.112d92027590.0000000000000024:f [read 1312768~1024 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:29:27.207 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:27.065074+0000", 2024-09-14T06:29:27.208 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 0.073248482000000004, 2024-09-14T06:29:27.208 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046289858000000003, 2024-09-14T06:29:27.208 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:29:27.208 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:29:27.208 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:29:27.208 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:29:27.208 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:29:27.208 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:27.208 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 19366 2024-09-14T06:29:27.208 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:27.208 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:29:27.209 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:27.209 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:29:27.209 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065074+0000", 2024-09-14T06:29:27.209 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:27.209 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:27.209 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:27.209 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:29:27.209 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065074+0000", 2024-09-14T06:29:27.209 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:27.209 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:27.209 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:27.210 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:29:27.210 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065074+0000", 2024-09-14T06:29:27.210 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.3900000000000001e-07 2024-09-14T06:29:27.210 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:27.210 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:27.210 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:29:27.210 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065078+0000", 2024-09-14T06:29:27.210 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.3229999999999999e-06 2024-09-14T06:29:27.210 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:27.210 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:27.210 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:29:27.210 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065082+0000", 2024-09-14T06:29:27.210 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.8299999999999998e-06 2024-09-14T06:29:27.211 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:27.211 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:27.211 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:29:27.211 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065085+0000", 2024-09-14T06:29:27.211 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.8660000000000002e-06 2024-09-14T06:29:27.211 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:27.211 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:27.211 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:29:27.211 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.111270+0000", 2024-09-14T06:29:27.211 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046185035999999999 2024-09-14T06:29:27.211 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:27.211 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:27.212 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:29:27.212 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.111311+0000", 2024-09-14T06:29:27.212 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.0935e-05 2024-09-14T06:29:27.212 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:27.212 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:27.212 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:29:27.212 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.111364+0000", 2024-09-14T06:29:27.212 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.2629e-05 2024-09-14T06:29:27.212 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:27.212 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:29:27.212 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:27.212 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:27.213 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:27.213 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:19367 3.3 3:c34e5e51:::rbd_data.112d92027590.0000000000000024:f [read 1323008~1024 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:29:27.213 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:27.065115+0000", 2024-09-14T06:29:27.213 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 0.073206920999999994, 2024-09-14T06:29:27.213 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046407003000000002, 2024-09-14T06:29:27.213 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:29:27.213 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:29:27.213 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:29:27.213 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:29:27.213 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:29:27.213 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:27.213 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 19367 2024-09-14T06:29:27.214 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:27.214 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:29:27.214 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:27.214 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:29:27.214 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065115+0000", 2024-09-14T06:29:27.214 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:27.214 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:27.214 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:27.214 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:29:27.214 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065115+0000", 2024-09-14T06:29:27.215 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:27.215 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:27.215 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:27.215 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:29:27.215 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065116+0000", 2024-09-14T06:29:27.215 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.4999999999999998e-07 2024-09-14T06:29:27.215 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:27.215 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:27.215 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:29:27.215 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065117+0000", 2024-09-14T06:29:27.215 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.189e-06 2024-09-14T06:29:27.215 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:27.216 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:27.216 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:29:27.216 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065122+0000", 2024-09-14T06:29:27.216 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.3709999999999999e-06 2024-09-14T06:29:27.216 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:27.216 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:27.216 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:29:27.216 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065125+0000", 2024-09-14T06:29:27.216 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.8940000000000002e-06 2024-09-14T06:29:27.216 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:27.216 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:27.217 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:29:27.217 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.111402+0000", 2024-09-14T06:29:27.217 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046276645999999998 2024-09-14T06:29:27.217 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:27.217 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:27.217 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:29:27.217 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.111445+0000", 2024-09-14T06:29:27.217 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.3288000000000003e-05 2024-09-14T06:29:27.217 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:27.217 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:27.217 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:29:27.217 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.111522+0000", 2024-09-14T06:29:27.218 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.7164999999999997e-05 2024-09-14T06:29:27.218 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:27.218 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:29:27.218 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:27.218 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:27.218 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:27.218 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:19368 3.3 3:c34e5e51:::rbd_data.112d92027590.0000000000000024:f [read 1333248~1024 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:29:27.218 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:27.065130+0000", 2024-09-14T06:29:27.218 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 0.073192630999999994, 2024-09-14T06:29:27.218 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046507275000000001, 2024-09-14T06:29:27.218 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:29:27.218 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:29:27.219 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:29:27.219 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:29:27.219 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:29:27.219 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:27.219 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 19368 2024-09-14T06:29:27.219 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:27.219 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:29:27.219 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:27.219 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:29:27.219 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065130+0000", 2024-09-14T06:29:27.219 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:27.219 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:27.220 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:27.220 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:29:27.220 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065130+0000", 2024-09-14T06:29:27.220 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:27.220 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:27.220 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:27.220 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:29:27.220 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065130+0000", 2024-09-14T06:29:27.220 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.0800000000000001e-07 2024-09-14T06:29:27.220 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:27.220 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:27.220 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:29:27.221 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065131+0000", 2024-09-14T06:29:27.221 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.8700000000000004e-07 2024-09-14T06:29:27.221 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:27.221 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:27.221 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:29:27.221 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065134+0000", 2024-09-14T06:29:27.221 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.1039999999999998e-06 2024-09-14T06:29:27.221 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:27.221 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:27.221 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:29:27.221 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065136+0000", 2024-09-14T06:29:27.221 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.3949999999999999e-06 2024-09-14T06:29:27.222 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:27.222 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:27.222 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:29:27.222 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.111541+0000", 2024-09-14T06:29:27.222 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046404206000000003 2024-09-14T06:29:27.222 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:27.222 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:27.222 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:29:27.222 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.111582+0000", 2024-09-14T06:29:27.222 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.1051999999999999e-05 2024-09-14T06:29:27.222 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:27.222 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:27.223 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:29:27.223 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.111637+0000", 2024-09-14T06:29:27.223 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.5322999999999999e-05 2024-09-14T06:29:27.223 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:27.223 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:29:27.223 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:27.223 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:27.223 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:27.223 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:19369 3.3 3:c34e5e51:::rbd_data.112d92027590.0000000000000024:f [read 1343488~1024 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:29:27.223 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:27.065139+0000", 2024-09-14T06:29:27.223 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 0.073182813999999999, 2024-09-14T06:29:27.223 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046682169000000003, 2024-09-14T06:29:27.224 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:29:27.224 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:29:27.224 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:29:27.224 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:29:27.224 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:29:27.224 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:27.224 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 19369 2024-09-14T06:29:27.224 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:27.224 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:29:27.224 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:27.224 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:29:27.224 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065139+0000", 2024-09-14T06:29:27.225 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:27.225 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:27.225 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:27.225 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:29:27.225 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065139+0000", 2024-09-14T06:29:27.225 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:27.225 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:27.225 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:27.225 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:29:27.225 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065140+0000", 2024-09-14T06:29:27.225 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.8299999999999998e-07 2024-09-14T06:29:27.226 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:27.226 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:27.226 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:29:27.226 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065141+0000", 2024-09-14T06:29:27.226 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.3900000000000003e-07 2024-09-14T06:29:27.226 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:27.226 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:27.226 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:29:27.226 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065144+0000", 2024-09-14T06:29:27.226 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.7409999999999998e-06 2024-09-14T06:29:27.226 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:27.226 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:27.227 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:29:27.227 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065147+0000", 2024-09-14T06:29:27.227 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.4150000000000002e-06 2024-09-14T06:29:27.227 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:27.227 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:27.227 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:29:27.227 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.111673+0000", 2024-09-14T06:29:27.227 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046526004000000003 2024-09-14T06:29:27.227 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:27.227 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:27.227 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:29:27.227 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.111749+0000", 2024-09-14T06:29:27.228 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.5741999999999994e-05 2024-09-14T06:29:27.228 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:27.228 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:27.228 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:29:27.228 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.111822+0000", 2024-09-14T06:29:27.228 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.3045000000000003e-05 2024-09-14T06:29:27.228 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:27.228 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:29:27.228 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:27.228 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:27.228 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:27.228 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:19370 3.3 3:c34e5e51:::rbd_data.112d92027590.0000000000000024:f [read 1353728~1024 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:29:27.229 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:27.065150+0000", 2024-09-14T06:29:27.229 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 0.073172084999999998, 2024-09-14T06:29:27.229 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046897556999999999, 2024-09-14T06:29:27.229 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:29:27.229 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:29:27.229 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:29:27.229 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:29:27.229 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:29:27.229 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:27.229 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 19370 2024-09-14T06:29:27.229 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:27.230 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:29:27.230 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:27.230 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:29:27.230 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065150+0000", 2024-09-14T06:29:27.230 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:27.230 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:27.230 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:27.230 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:29:27.230 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065150+0000", 2024-09-14T06:29:27.230 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:27.230 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:27.230 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:27.231 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:29:27.231 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065150+0000", 2024-09-14T06:29:27.231 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.0200000000000001e-07 2024-09-14T06:29:27.231 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:27.231 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:27.231 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:29:27.231 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065151+0000", 2024-09-14T06:29:27.231 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.1100000000000004e-07 2024-09-14T06:29:27.231 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:27.231 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:27.231 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:29:27.231 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065154+0000", 2024-09-14T06:29:27.232 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.9639999999999999e-06 2024-09-14T06:29:27.232 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:27.232 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:27.232 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:29:27.232 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065157+0000", 2024-09-14T06:29:27.232 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.4260000000000002e-06 2024-09-14T06:29:27.232 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:27.232 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:27.232 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:29:27.232 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.111861+0000", 2024-09-14T06:29:27.232 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046704268 2024-09-14T06:29:27.233 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:27.233 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:27.233 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:29:27.233 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.111939+0000", 2024-09-14T06:29:27.233 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.8231000000000004e-05 2024-09-14T06:29:27.233 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:27.233 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:27.233 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:29:27.234 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.112048+0000", 2024-09-14T06:29:27.234 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00010855500000000001 2024-09-14T06:29:27.234 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:27.234 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:29:27.234 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:27.234 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:27.234 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:27.234 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:19371 3.3 3:c34e5e51:::rbd_data.112d92027590.0000000000000024:f [read 1363968~1024 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:29:27.234 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:27.065161+0000", 2024-09-14T06:29:27.234 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 0.073161434999999997, 2024-09-14T06:29:27.234 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047021944000000003, 2024-09-14T06:29:27.234 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:29:27.235 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:29:27.235 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:29:27.235 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:29:27.235 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:29:27.235 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:27.235 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 19371 2024-09-14T06:29:27.235 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:27.235 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:29:27.235 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:27.235 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:29:27.235 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065161+0000", 2024-09-14T06:29:27.235 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:27.236 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:27.236 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:27.236 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:29:27.236 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065161+0000", 2024-09-14T06:29:27.236 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:27.236 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:27.236 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:27.236 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:29:27.236 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065161+0000", 2024-09-14T06:29:27.236 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.0200000000000001e-07 2024-09-14T06:29:27.236 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:27.236 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:27.237 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:29:27.237 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065166+0000", 2024-09-14T06:29:27.237 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.9869999999999999e-06 2024-09-14T06:29:27.237 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:27.237 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:27.237 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:29:27.237 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065169+0000", 2024-09-14T06:29:27.237 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.3799999999999998e-06 2024-09-14T06:29:27.237 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:27.237 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:27.237 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:29:27.237 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065172+0000", 2024-09-14T06:29:27.238 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.4830000000000002e-06 2024-09-14T06:29:27.238 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:27.238 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:27.238 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:29:27.238 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.112071+0000", 2024-09-14T06:29:27.238 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046899529000000002 2024-09-14T06:29:27.238 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:27.238 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:27.238 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:29:27.238 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.112128+0000", 2024-09-14T06:29:27.238 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.6341999999999997e-05 2024-09-14T06:29:27.238 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:27.238 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:27.239 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:29:27.239 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.112183+0000", 2024-09-14T06:29:27.239 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.5021000000000001e-05 2024-09-14T06:29:27.239 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:27.239 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:29:27.239 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:27.239 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:27.239 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:27.239 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:19372 3.3 3:c34e5e51:::rbd_data.112d92027590.0000000000000024:f [read 1374208~1024 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:29:27.239 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:27.065176+0000", 2024-09-14T06:29:27.239 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 0.073146285000000005, 2024-09-14T06:29:27.239 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047188889999999997, 2024-09-14T06:29:27.239 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:29:27.240 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:29:27.240 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:29:27.240 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:29:27.240 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:29:27.240 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:27.240 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 19372 2024-09-14T06:29:27.240 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:27.240 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:29:27.241 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:27.241 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:29:27.241 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065176+0000", 2024-09-14T06:29:27.241 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:27.241 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:27.241 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:27.241 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:29:27.241 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065176+0000", 2024-09-14T06:29:27.241 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:27.241 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:27.241 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:27.241 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:29:27.242 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.071117+0000", 2024-09-14T06:29:27.242 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.0059404990000000001 2024-09-14T06:29:27.242 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:27.242 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:27.242 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:29:27.242 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.071120+0000", 2024-09-14T06:29:27.242 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.4340000000000001e-06 2024-09-14T06:29:27.242 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:27.242 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:27.242 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:29:27.242 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.071129+0000", 2024-09-14T06:29:27.242 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.8389999999999996e-06 2024-09-14T06:29:27.243 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:27.243 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:27.243 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:29:27.243 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.071134+0000", 2024-09-14T06:29:27.243 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.412e-06 2024-09-14T06:29:27.243 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:27.243 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:27.243 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:29:27.243 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.112221+0000", 2024-09-14T06:29:27.243 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.041086473999999998 2024-09-14T06:29:27.243 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:27.243 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:27.244 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:29:27.244 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.112299+0000", 2024-09-14T06:29:27.244 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.8746999999999997e-05 2024-09-14T06:29:27.244 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:27.244 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:27.244 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:29:27.244 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.112365+0000", 2024-09-14T06:29:27.244 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.5485000000000001e-05 2024-09-14T06:29:27.244 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:27.244 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:29:27.244 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:27.245 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:27.245 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:29:27.245 INFO:teuthology.orchestra.run.smithi138.stdout:} 2024-09-14T06:29:27.246 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-14T06:29:27.247 INFO:tasks.ceph.osd.0.smithi094.stderr:2024-09-14T06:29:27.246+0000 7f07c9333640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33636) UID: 0 2024-09-14T06:29:27.271 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-14T06:29:27.272 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-14T06:29:27.272 INFO:tasks.thrashosds.thrasher:inject_pause on osd.5 2024-09-14T06:29:27.272 INFO:tasks.thrashosds.thrasher:Testing filestore_inject_stall pause injection for duration 3 2024-09-14T06:29:27.272 INFO:tasks.thrashosds.thrasher:Checking after 0, should_be_down=False 2024-09-14T06:29:27.272 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok config set filestore_inject_stall 3 2024-09-14T06:29:27.344 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-14T06:29:27.346 INFO:tasks.ceph.osd.7.smithi138.stderr:2024-09-14T06:29:27.345+0000 7f3c6a21d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33623) UID: 0 2024-09-14T06:29:27.353 INFO:teuthology.orchestra.run.smithi138.stdout:{ 2024-09-14T06:29:27.354 INFO:teuthology.orchestra.run.smithi138.stdout: "ops": [], 2024-09-14T06:29:27.354 INFO:teuthology.orchestra.run.smithi138.stdout: "num_ops": 0 2024-09-14T06:29:27.354 INFO:teuthology.orchestra.run.smithi138.stdout:} 2024-09-14T06:29:27.363 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-14T06:29:27.396 INFO:teuthology.orchestra.run.smithi138.stdout:{ 2024-09-14T06:29:27.397 INFO:teuthology.orchestra.run.smithi138.stdout: "success": "filestore_inject_stall = '' (not observed, change may require restart) " 2024-09-14T06:29:27.397 INFO:teuthology.orchestra.run.smithi138.stdout:} 2024-09-14T06:29:27.447 INFO:tasks.ceph.osd.0.smithi094.stderr:2024-09-14T06:29:27.445+0000 7f07c9333640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33636) UID: 0 2024-09-14T06:29:27.489 INFO:teuthology.orchestra.run.smithi094.stdout:osd.0: {} 2024-09-14T06:29:27.489 INFO:teuthology.orchestra.run.smithi094.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-14T06:29:27.502 INFO:teuthology.orchestra.run.smithi138.stdout:{ 2024-09-14T06:29:27.572 INFO:teuthology.orchestra.run.smithi138.stdout: "ops": [], 2024-09-14T06:29:27.572 INFO:teuthology.orchestra.run.smithi138.stdout: "complaint_time": 30, 2024-09-14T06:29:27.572 INFO:teuthology.orchestra.run.smithi138.stdout: "num_blocked_ops": 0 2024-09-14T06:29:27.572 INFO:teuthology.orchestra.run.smithi138.stdout:} 2024-09-14T06:29:27.573 INFO:teuthology.orchestra.run.smithi094.stdout:osd.1: {} 2024-09-14T06:29:27.573 INFO:teuthology.orchestra.run.smithi094.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-14T06:29:27.574 INFO:teuthology.orchestra.run.smithi094.stdout:osd.2: {} 2024-09-14T06:29:27.574 INFO:teuthology.orchestra.run.smithi094.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-14T06:29:27.575 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-14T06:29:27.575 INFO:tasks.ceph.osd.4.smithi138.stderr:2024-09-14T06:29:27.572+0000 7f6c653c8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33615) UID: 0 2024-09-14T06:29:27.575 INFO:teuthology.orchestra.run.smithi094.stdout:osd.3: {} 2024-09-14T06:29:27.576 INFO:teuthology.orchestra.run.smithi094.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-14T06:29:27.582 INFO:teuthology.orchestra.run.smithi094.stdout:osd.4: {} 2024-09-14T06:29:27.583 INFO:teuthology.orchestra.run.smithi094.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-14T06:29:27.607 INFO:teuthology.orchestra.run.smithi094.stdout:osd.5: {} 2024-09-14T06:29:27.637 INFO:teuthology.orchestra.run.smithi094.stdout:osd.6: {} 2024-09-14T06:29:27.638 INFO:teuthology.orchestra.run.smithi094.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-14T06:29:27.655 INFO:teuthology.orchestra.run.smithi094.stdout:osd.7: {} 2024-09-14T06:29:27.655 INFO:teuthology.orchestra.run.smithi094.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-14T06:29:27.670 INFO:teuthology.orchestra.run.smithi094.stderr:nodeep-scrub is set 2024-09-14T06:29:27.674 INFO:tasks.ceph.osd.6.smithi138.stderr:2024-09-14T06:29:27.673+0000 7f8cd8849640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33620) UID: 0 2024-09-14T06:29:27.721 INFO:teuthology.orchestra.run.smithi138.stdout:ERROR: (22) Invalid argument 2024-09-14T06:29:27.721 INFO:teuthology.orchestra.run.smithi138.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:29:27.731 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-14T06:29:27.775 INFO:tasks.ceph.osd.0.smithi094.stderr:2024-09-14T06:29:27.773+0000 7f07c9333640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33636) UID: 0 2024-09-14T06:29:27.876 INFO:tasks.ceph.osd.1.smithi094.stderr:2024-09-14T06:29:27.874+0000 7f328e47a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33641) UID: 0 2024-09-14T06:29:27.885 INFO:teuthology.orchestra.run.smithi138.stdout:ERROR: (22) Invalid argument 2024-09-14T06:29:27.885 INFO:teuthology.orchestra.run.smithi138.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:29:27.895 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-14T06:29:27.976 INFO:tasks.ceph.osd.4.smithi138.stderr:2024-09-14T06:29:27.975+0000 7f6c653c8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33615) UID: 0 2024-09-14T06:29:27.997 INFO:teuthology.orchestra.run.smithi138.stdout:ERROR: (22) Invalid argument 2024-09-14T06:29:27.998 INFO:teuthology.orchestra.run.smithi138.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:29:28.006 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-14T06:29:28.078 INFO:tasks.ceph.osd.3.smithi094.stderr:2024-09-14T06:29:28.076+0000 7feeeafe8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33655) UID: 0 2024-09-14T06:29:28.149 INFO:teuthology.orchestra.run.smithi138.stdout:ERROR: (22) Invalid argument 2024-09-14T06:29:28.149 INFO:teuthology.orchestra.run.smithi138.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:29:28.158 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-14T06:29:28.202 INFO:tasks.ceph.osd.6.smithi138.stderr:2024-09-14T06:29:28.201+0000 7f8cd8849640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33620) UID: 0 2024-09-14T06:29:28.260 INFO:teuthology.orchestra.run.smithi138.stdout:ERROR: (22) Invalid argument 2024-09-14T06:29:28.260 INFO:teuthology.orchestra.run.smithi138.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:29:28.268 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-14T06:29:28.279 INFO:tasks.ceph.osd.1.smithi094.stderr:2024-09-14T06:29:28.277+0000 7f328e47a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33641) UID: 0 2024-09-14T06:29:28.380 INFO:tasks.ceph.osd.2.smithi094.stderr:2024-09-14T06:29:28.379+0000 7faf3269b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33649) UID: 0 2024-09-14T06:29:28.410 INFO:teuthology.orchestra.run.smithi138.stdout:ERROR: (22) Invalid argument 2024-09-14T06:29:28.410 INFO:teuthology.orchestra.run.smithi138.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:29:28.419 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-14T06:29:28.481 INFO:tasks.ceph.osd.5.smithi138.stderr:2024-09-14T06:29:28.480+0000 7fe13746f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33618) UID: 0 2024-09-14T06:29:28.522 INFO:teuthology.orchestra.run.smithi138.stdout:ERROR: (22) Invalid argument 2024-09-14T06:29:28.522 INFO:teuthology.orchestra.run.smithi138.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:29:28.531 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-14T06:29:28.583 INFO:tasks.ceph.osd.2.smithi094.stderr:2024-09-14T06:29:28.581+0000 7faf3269b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33649) UID: 0 2024-09-14T06:29:28.641 INFO:teuthology.orchestra.run.smithi094.stdout:ERROR: (22) Invalid argument 2024-09-14T06:29:28.641 INFO:teuthology.orchestra.run.smithi094.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:29:28.650 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-14T06:29:28.684 INFO:tasks.ceph.osd.5.smithi138.stderr:2024-09-14T06:29:28.682+0000 7fe13746f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33618) UID: 0 2024-09-14T06:29:28.785 INFO:tasks.ceph.osd.1.smithi094.stderr:2024-09-14T06:29:28.783+0000 7f328e47a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33641) UID: 0 2024-09-14T06:29:28.796 INFO:teuthology.orchestra.run.smithi094.stdout:checking clone #5, image image_client.0-clone5 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent6 2024-09-14T06:29:28.803 INFO:teuthology.orchestra.run.smithi094.stdout:ERROR: (22) Invalid argument 2024-09-14T06:29:28.803 INFO:teuthology.orchestra.run.smithi094.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:29:28.815 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-14T06:29:28.899 INFO:tasks.ceph.osd.0.smithi094.stderr:2024-09-14T06:29:28.897+0000 7f07c9333640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33636) UID: 0 2024-09-14T06:29:28.966 INFO:teuthology.orchestra.run.smithi094.stdout:ERROR: (22) Invalid argument 2024-09-14T06:29:28.966 INFO:teuthology.orchestra.run.smithi094.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:29:28.975 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-14T06:29:28.987 INFO:tasks.ceph.osd.5.smithi138.stderr:2024-09-14T06:29:28.986+0000 7fe13746f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33618) UID: 0 2024-09-14T06:29:29.053 INFO:teuthology.orchestra.run.smithi094.stdout:544 clone 8 order 24 su 1048576 sc 4 2024-09-14T06:29:29.088 INFO:tasks.ceph.osd.6.smithi138.stderr:2024-09-14T06:29:29.087+0000 7f8cd8849640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33620) UID: 0 2024-09-14T06:29:29.119 INFO:teuthology.orchestra.run.smithi094.stdout:ERROR: (22) Invalid argument 2024-09-14T06:29:29.120 INFO:teuthology.orchestra.run.smithi094.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:29:29.128 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-14T06:29:29.189 INFO:tasks.ceph.osd.4.smithi138.stderr:2024-09-14T06:29:29.188+0000 7f6c653c8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33615) UID: 0 2024-09-14T06:29:29.271 INFO:teuthology.orchestra.run.smithi094.stdout:ERROR: (22) Invalid argument 2024-09-14T06:29:29.271 INFO:teuthology.orchestra.run.smithi094.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:29:29.279 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-14T06:29:29.289 INFO:tasks.ceph.osd.4.smithi138.stderr:2024-09-14T06:29:29.288+0000 7f6c653c8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33615) UID: 0 2024-09-14T06:29:29.382 INFO:teuthology.orchestra.run.smithi094.stdout:ERROR: (22) Invalid argument 2024-09-14T06:29:29.382 INFO:teuthology.orchestra.run.smithi094.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:29:29.390 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-14T06:29:29.391 INFO:tasks.ceph.osd.5.smithi138.stderr:2024-09-14T06:29:29.389+0000 7fe13746f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33618) UID: 0 2024-09-14T06:29:29.491 INFO:tasks.ceph.osd.4.smithi138.stderr:2024-09-14T06:29:29.491+0000 7f6c653c8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33615) UID: 0 2024-09-14T06:29:29.493 INFO:teuthology.orchestra.run.smithi094.stdout:ERROR: (22) Invalid argument 2024-09-14T06:29:29.493 INFO:teuthology.orchestra.run.smithi094.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:29:29.503 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-14T06:29:29.593 INFO:tasks.ceph.osd.6.smithi138.stderr:2024-09-14T06:29:29.592+0000 7f8cd8849640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33620) UID: 0 2024-09-14T06:29:29.605 INFO:teuthology.orchestra.run.smithi094.stdout:ERROR: (22) Invalid argument 2024-09-14T06:29:29.605 INFO:teuthology.orchestra.run.smithi094.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:29:29.614 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-14T06:29:29.671 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-14T06:29:29.687 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-14T06:29:29.693 INFO:tasks.ceph.osd.7.smithi138.stderr:2024-09-14T06:29:29.692+0000 7f3c6a21d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33623) UID: 0 2024-09-14T06:29:29.725 INFO:teuthology.orchestra.run.smithi094.stdout:ERROR: (22) Invalid argument 2024-09-14T06:29:29.726 INFO:teuthology.orchestra.run.smithi094.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:29:29.735 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-14T06:29:29.794 INFO:tasks.ceph.osd.7.smithi138.stderr:2024-09-14T06:29:29.793+0000 7f3c6a21d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33623) UID: 0 2024-09-14T06:29:29.803 INFO:teuthology.orchestra.run.smithi094.stdout:osd.0: {} 2024-09-14T06:29:29.803 INFO:teuthology.orchestra.run.smithi094.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-14T06:29:29.821 INFO:teuthology.orchestra.run.smithi094.stdout:osd.1: {} 2024-09-14T06:29:29.821 INFO:teuthology.orchestra.run.smithi094.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-14T06:29:29.853 INFO:teuthology.orchestra.run.smithi094.stdout:osd.2: {} 2024-09-14T06:29:29.853 INFO:teuthology.orchestra.run.smithi094.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-14T06:29:29.873 INFO:teuthology.orchestra.run.smithi094.stdout:osd.3: {} 2024-09-14T06:29:29.873 INFO:teuthology.orchestra.run.smithi094.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-14T06:29:29.890 INFO:teuthology.orchestra.run.smithi094.stdout:osd.4: {} 2024-09-14T06:29:29.890 INFO:teuthology.orchestra.run.smithi094.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-14T06:29:29.895 INFO:tasks.ceph.osd.4.smithi138.stderr:2024-09-14T06:29:29.894+0000 7f6c653c8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33615) UID: 0 2024-09-14T06:29:29.898 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-14T06:29:29.898 INFO:teuthology.orchestra.run.smithi094.stdout: "ops": [], 2024-09-14T06:29:29.898 INFO:teuthology.orchestra.run.smithi094.stdout: "num_ops": 0 2024-09-14T06:29:29.898 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-14T06:29:29.907 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-14T06:29:29.908 INFO:teuthology.orchestra.run.smithi094.stdout:osd.5: {} 2024-09-14T06:29:29.908 INFO:teuthology.orchestra.run.smithi094.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-14T06:29:29.924 INFO:teuthology.orchestra.run.smithi094.stdout:osd.6: {} 2024-09-14T06:29:29.924 INFO:teuthology.orchestra.run.smithi094.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-14T06:29:29.940 INFO:teuthology.orchestra.run.smithi094.stdout:osd.7: {} 2024-09-14T06:29:29.940 INFO:teuthology.orchestra.run.smithi094.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-14T06:29:29.996 INFO:tasks.ceph.osd.0.smithi094.stderr:2024-09-14T06:29:29.995+0000 7f07c9333640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33636) UID: 0 2024-09-14T06:29:30.028 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-14T06:29:30.028 INFO:teuthology.orchestra.run.smithi094.stdout: "ops": [], 2024-09-14T06:29:30.028 INFO:teuthology.orchestra.run.smithi094.stdout: "complaint_time": 30, 2024-09-14T06:29:30.029 INFO:teuthology.orchestra.run.smithi094.stdout: "num_blocked_ops": 0 2024-09-14T06:29:30.029 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-14T06:29:30.037 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-14T06:29:30.098 INFO:tasks.ceph.osd.5.smithi138.stderr:2024-09-14T06:29:30.097+0000 7fe13746f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33618) UID: 0 2024-09-14T06:29:30.182 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-14T06:29:30.182 INFO:teuthology.orchestra.run.smithi094.stdout: "size": 20, 2024-09-14T06:29:30.182 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 600, 2024-09-14T06:29:30.182 INFO:teuthology.orchestra.run.smithi094.stdout: "ops": [ 2024-09-14T06:29:30.183 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:30.183 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:15608 3.b 3:d24f5f7d:::rbd_data.112d92027590.000000000000000d:f [read 852992~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:29:30.183 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:26.142492+0000", 2024-09-14T06:29:30.183 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 4.0379874710000001, 2024-09-14T06:29:30.183 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.048597006999999998, 2024-09-14T06:29:30.183 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:29:30.183 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:29:30.183 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:29:30.183 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:29:30.183 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:29:30.183 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:30.183 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 15608 2024-09-14T06:29:30.184 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:30.184 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:29:30.184 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:30.184 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:29:30.184 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.142492+0000", 2024-09-14T06:29:30.184 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:30.184 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:30.184 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:30.184 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:29:30.184 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.142492+0000", 2024-09-14T06:29:30.184 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:30.184 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:30.184 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:30.184 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:29:30.185 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.142568+0000", 2024-09-14T06:29:30.185 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.6209000000000002e-05 2024-09-14T06:29:30.185 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:30.185 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:30.185 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:29:30.185 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.142571+0000", 2024-09-14T06:29:30.185 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.157e-06 2024-09-14T06:29:30.185 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:30.185 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:30.185 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:29:30.185 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.142579+0000", 2024-09-14T06:29:30.185 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.6879999999999999e-06 2024-09-14T06:29:30.185 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:30.185 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:30.186 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:29:30.186 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.142585+0000", 2024-09-14T06:29:30.186 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.8780000000000003e-06 2024-09-14T06:29:30.186 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:30.186 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:30.186 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:29:30.186 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.190912+0000", 2024-09-14T06:29:30.186 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.048327579000000002 2024-09-14T06:29:30.186 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:30.186 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:30.186 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:29:30.186 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.190990+0000", 2024-09-14T06:29:30.186 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.8119000000000002e-05 2024-09-14T06:29:30.186 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:30.187 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:30.187 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:29:30.187 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.191089+0000", 2024-09-14T06:29:30.187 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.8376999999999997e-05 2024-09-14T06:29:30.187 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:30.187 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:30.187 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:30.187 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:30.187 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:30.187 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:15609 3.b 3:d24f5f7d:::rbd_data.112d92027590.000000000000000d:f [read 863232~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:29:30.187 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:26.142596+0000", 2024-09-14T06:29:30.187 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 4.0378837010000002, 2024-09-14T06:29:30.187 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.048641772999999999, 2024-09-14T06:29:30.187 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:29:30.188 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:29:30.188 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:29:30.188 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:29:30.188 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:29:30.188 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:30.188 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 15609 2024-09-14T06:29:30.188 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:30.188 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:29:30.188 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:30.188 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:29:30.188 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.142596+0000", 2024-09-14T06:29:30.188 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:30.188 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:30.189 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:30.189 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:29:30.189 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.142596+0000", 2024-09-14T06:29:30.189 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:30.189 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:30.189 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:30.189 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:29:30.189 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.142741+0000", 2024-09-14T06:29:30.189 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00014539199999999999 2024-09-14T06:29:30.189 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:30.189 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:30.189 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:29:30.189 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.142745+0000", 2024-09-14T06:29:30.190 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.1470000000000003e-06 2024-09-14T06:29:30.190 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:30.190 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:30.190 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:29:30.190 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.142754+0000", 2024-09-14T06:29:30.190 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.1330000000000005e-06 2024-09-14T06:29:30.190 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:30.190 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:30.190 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:29:30.190 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.142761+0000", 2024-09-14T06:29:30.190 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.63e-06 2024-09-14T06:29:30.190 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:30.190 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:30.190 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:29:30.191 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.191127+0000", 2024-09-14T06:29:30.191 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.048366251999999998 2024-09-14T06:29:30.191 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:30.191 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:30.191 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:29:30.191 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.191178+0000", 2024-09-14T06:29:30.191 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.1359000000000001e-05 2024-09-14T06:29:30.191 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:30.191 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:30.191 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:29:30.191 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.191237+0000", 2024-09-14T06:29:30.191 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.8860000000000002e-05 2024-09-14T06:29:30.191 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:30.191 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:30.191 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:30.192 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:30.192 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:30.192 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:15610 3.b 3:d24f5f7d:::rbd_data.112d92027590.000000000000000d:f [read 873472~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:29:30.192 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:26.142773+0000", 2024-09-14T06:29:30.192 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 4.037706225, 2024-09-14T06:29:30.192 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.048677076999999999, 2024-09-14T06:29:30.192 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:29:30.192 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:29:30.192 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:29:30.192 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:29:30.192 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:29:30.192 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:30.192 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 15610 2024-09-14T06:29:30.192 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:30.193 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:29:30.193 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:30.193 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:29:30.193 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.142773+0000", 2024-09-14T06:29:30.193 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:30.193 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:30.193 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:30.193 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:29:30.193 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.142773+0000", 2024-09-14T06:29:30.193 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:30.193 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:30.193 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:30.193 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:29:30.193 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.142855+0000", 2024-09-14T06:29:30.194 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.2180000000000003e-05 2024-09-14T06:29:30.194 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:30.194 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:30.194 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:29:30.194 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.142859+0000", 2024-09-14T06:29:30.194 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.5810000000000001e-06 2024-09-14T06:29:30.194 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:30.194 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:30.194 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:29:30.194 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.142868+0000", 2024-09-14T06:29:30.194 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.8489999999999995e-06 2024-09-14T06:29:30.194 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:30.194 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:30.195 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:29:30.195 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.142874+0000", 2024-09-14T06:29:30.195 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.7750000000000004e-06 2024-09-14T06:29:30.195 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:30.195 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:30.195 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:29:30.195 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.191272+0000", 2024-09-14T06:29:30.195 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.048397430999999998 2024-09-14T06:29:30.195 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:30.195 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:30.195 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:29:30.195 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.191351+0000", 2024-09-14T06:29:30.195 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.9480000000000005e-05 2024-09-14T06:29:30.195 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:30.196 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:30.196 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:29:30.196 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.191450+0000", 2024-09-14T06:29:30.196 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.8781000000000002e-05 2024-09-14T06:29:30.196 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:30.196 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:30.196 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:30.196 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:30.196 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:30.196 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:15611 3.b 3:d24f5f7d:::rbd_data.112d92027590.000000000000000d:f [read 883712~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:29:30.196 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:26.142885+0000", 2024-09-14T06:29:30.196 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 4.0375942760000001, 2024-09-14T06:29:30.196 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.048706685, 2024-09-14T06:29:30.197 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:29:30.197 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:29:30.197 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:29:30.197 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:29:30.197 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:29:30.197 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:30.197 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 15611 2024-09-14T06:29:30.197 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:30.197 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:29:30.197 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:30.197 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:29:30.197 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.142885+0000", 2024-09-14T06:29:30.197 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:30.197 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:30.198 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:30.198 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:29:30.198 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.142885+0000", 2024-09-14T06:29:30.198 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:30.198 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:30.198 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:30.198 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:29:30.198 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.142989+0000", 2024-09-14T06:29:30.198 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00010394 2024-09-14T06:29:30.198 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:30.198 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:30.198 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:29:30.198 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.142992+0000", 2024-09-14T06:29:30.199 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.534e-06 2024-09-14T06:29:30.199 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:30.199 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:30.199 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:29:30.199 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.143001+0000", 2024-09-14T06:29:30.199 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.9050000000000003e-06 2024-09-14T06:29:30.199 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:30.199 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:30.199 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:29:30.199 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.143008+0000", 2024-09-14T06:29:30.199 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.8900000000000001e-06 2024-09-14T06:29:30.199 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:30.199 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:30.199 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:29:30.199 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.191489+0000", 2024-09-14T06:29:30.200 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.048480489000000002 2024-09-14T06:29:30.200 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:30.200 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:30.200 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:29:30.200 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.191532+0000", 2024-09-14T06:29:30.200 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.3244999999999997e-05 2024-09-14T06:29:30.200 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:30.200 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:30.200 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:29:30.200 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.191592+0000", 2024-09-14T06:29:30.200 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.9682000000000002e-05 2024-09-14T06:29:30.200 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:30.200 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:30.201 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:30.201 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:30.201 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:30.201 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:15612 3.b 3:d24f5f7d:::rbd_data.112d92027590.000000000000000d:f [read 893952~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:29:30.201 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:26.143019+0000", 2024-09-14T06:29:30.201 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 4.0374605849999998, 2024-09-14T06:29:30.201 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.048778560999999998, 2024-09-14T06:29:30.201 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:29:30.201 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:29:30.201 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:29:30.201 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:29:30.201 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:29:30.201 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:30.201 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 15612 2024-09-14T06:29:30.202 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:30.202 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:29:30.202 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:30.202 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:29:30.202 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.143019+0000", 2024-09-14T06:29:30.202 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:30.202 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:30.202 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:30.202 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:29:30.202 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.143019+0000", 2024-09-14T06:29:30.202 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:30.202 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:30.202 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:30.202 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:29:30.202 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.143236+0000", 2024-09-14T06:29:30.203 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000217517 2024-09-14T06:29:30.203 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:30.203 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:30.203 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:29:30.203 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.143241+0000", 2024-09-14T06:29:30.203 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.7779999999999998e-06 2024-09-14T06:29:30.203 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:30.203 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:30.203 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:29:30.203 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.143248+0000", 2024-09-14T06:29:30.203 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.7100000000000001e-06 2024-09-14T06:29:30.203 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:30.203 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:30.203 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:29:30.204 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.143254+0000", 2024-09-14T06:29:30.204 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.8950000000000003e-06 2024-09-14T06:29:30.204 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:30.204 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:30.204 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:29:30.204 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.191626+0000", 2024-09-14T06:29:30.204 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.048372747000000001 2024-09-14T06:29:30.204 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:30.204 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:30.204 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:29:30.204 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.191703+0000", 2024-09-14T06:29:30.204 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.6246000000000001e-05 2024-09-14T06:29:30.204 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:30.205 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:30.205 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:29:30.205 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.191797+0000", 2024-09-14T06:29:30.205 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.4667999999999996e-05 2024-09-14T06:29:30.205 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:30.205 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:30.205 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:30.205 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:30.205 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:30.205 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:15613 3.b 3:d24f5f7d:::rbd_data.112d92027590.000000000000000d:f [read 904192~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:29:30.205 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:26.143264+0000", 2024-09-14T06:29:30.205 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 4.0372148320000001, 2024-09-14T06:29:30.206 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.048672880000000002, 2024-09-14T06:29:30.206 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:29:30.206 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:29:30.206 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:29:30.206 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:29:30.206 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:29:30.206 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:30.206 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 15613 2024-09-14T06:29:30.206 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:30.206 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:29:30.206 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:30.206 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:29:30.206 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.143264+0000", 2024-09-14T06:29:30.206 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:30.207 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:30.207 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:30.207 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:29:30.207 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.143264+0000", 2024-09-14T06:29:30.207 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:30.207 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:30.207 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:30.207 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:29:30.207 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.143557+0000", 2024-09-14T06:29:30.207 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000292572 2024-09-14T06:29:30.207 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:30.207 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:30.207 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:29:30.207 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.143560+0000", 2024-09-14T06:29:30.208 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.4690000000000002e-06 2024-09-14T06:29:30.208 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:30.208 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:30.208 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:29:30.208 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.143572+0000", 2024-09-14T06:29:30.208 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.1457000000000001e-05 2024-09-14T06:29:30.208 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:30.208 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:30.208 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:29:30.208 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.143578+0000", 2024-09-14T06:29:30.208 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.5520000000000001e-06 2024-09-14T06:29:30.208 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:30.208 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:30.208 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:29:30.209 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.191838+0000", 2024-09-14T06:29:30.209 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.048259947999999997 2024-09-14T06:29:30.209 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:30.209 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:30.209 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:29:30.209 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.191881+0000", 2024-09-14T06:29:30.209 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.3058e-05 2024-09-14T06:29:30.209 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:30.209 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:30.209 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:29:30.209 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.191937+0000", 2024-09-14T06:29:30.209 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.5823999999999999e-05 2024-09-14T06:29:30.209 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:30.210 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:30.210 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:30.210 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:30.210 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:30.210 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:15614 3.b 3:d24f5f7d:::rbd_data.112d92027590.000000000000000d:f [read 914432~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:29:30.210 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:26.143597+0000", 2024-09-14T06:29:30.210 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 4.0368822570000003, 2024-09-14T06:29:30.210 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.048548172000000001, 2024-09-14T06:29:30.210 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:29:30.210 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:29:30.210 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:29:30.210 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:29:30.210 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:29:30.210 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:30.211 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 15614 2024-09-14T06:29:30.211 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:30.211 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:29:30.211 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:30.211 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:29:30.211 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.143597+0000", 2024-09-14T06:29:30.211 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:30.211 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:30.211 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:30.211 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:29:30.211 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.143597+0000", 2024-09-14T06:29:30.211 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:30.211 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:30.211 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:30.212 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:29:30.212 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.143598+0000", 2024-09-14T06:29:30.212 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.1099999999999996e-07 2024-09-14T06:29:30.212 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:30.212 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:30.212 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:29:30.212 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.143600+0000", 2024-09-14T06:29:30.212 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.4210000000000002e-06 2024-09-14T06:29:30.212 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:30.212 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:30.212 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:29:30.212 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.143608+0000", 2024-09-14T06:29:30.212 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.2379999999999997e-06 2024-09-14T06:29:30.213 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:30.213 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:30.213 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:29:30.213 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.143613+0000", 2024-09-14T06:29:30.213 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.1909999999999999e-06 2024-09-14T06:29:30.213 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:30.213 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:30.213 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:29:30.213 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.191970+0000", 2024-09-14T06:29:30.213 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.048356757 2024-09-14T06:29:30.213 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:30.213 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:30.213 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:29:30.214 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.192049+0000", 2024-09-14T06:29:30.214 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.9252000000000006e-05 2024-09-14T06:29:30.214 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:30.214 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:30.214 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:29:30.214 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.192145+0000", 2024-09-14T06:29:30.214 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.5802000000000002e-05 2024-09-14T06:29:30.214 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:30.214 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:30.214 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:30.214 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:30.215 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:30.215 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:15615 3.b 3:d24f5f7d:::rbd_data.112d92027590.000000000000000d:f [read 924672~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:29:30.215 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:26.143621+0000", 2024-09-14T06:29:30.215 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 4.0368577739999996, 2024-09-14T06:29:30.215 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.04866728, 2024-09-14T06:29:30.215 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:29:30.215 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:29:30.215 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:29:30.215 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:29:30.215 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:29:30.215 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:30.215 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 15615 2024-09-14T06:29:30.215 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:30.215 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:29:30.216 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:30.216 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:29:30.216 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.143621+0000", 2024-09-14T06:29:30.216 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:30.216 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:30.216 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:30.216 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:29:30.216 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.143621+0000", 2024-09-14T06:29:30.216 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:30.216 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:30.216 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:30.216 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:29:30.216 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.145193+0000", 2024-09-14T06:29:30.217 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0015719480000000001 2024-09-14T06:29:30.217 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:30.217 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:30.217 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:29:30.217 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.145198+0000", 2024-09-14T06:29:30.217 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.8770000000000001e-06 2024-09-14T06:29:30.217 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:30.217 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:30.217 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:29:30.217 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.145207+0000", 2024-09-14T06:29:30.217 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.0620000000000007e-06 2024-09-14T06:29:30.217 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:30.217 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:30.217 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:29:30.218 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.145214+0000", 2024-09-14T06:29:30.218 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.156e-06 2024-09-14T06:29:30.218 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:30.218 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:30.218 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:29:30.218 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.192180+0000", 2024-09-14T06:29:30.218 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.046966624999999998 2024-09-14T06:29:30.218 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:30.218 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:30.218 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:29:30.218 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.192231+0000", 2024-09-14T06:29:30.218 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.0754999999999998e-05 2024-09-14T06:29:30.218 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:30.219 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:30.219 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:29:30.219 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.192289+0000", 2024-09-14T06:29:30.219 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.7856999999999998e-05 2024-09-14T06:29:30.219 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:30.219 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:30.219 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:30.219 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:30.219 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:30.219 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:15704 3.b 3:d24f5f7d:::rbd_data.112d92027590.000000000000000d:f [read 137216~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:29:30.219 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:26.148323+0000", 2024-09-14T06:29:30.219 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 4.0321562459999996, 2024-09-14T06:29:30.219 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.048538987999999998, 2024-09-14T06:29:30.219 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:29:30.219 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:29:30.220 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:29:30.220 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:29:30.220 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:29:30.220 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:30.220 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 15704 2024-09-14T06:29:30.220 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:30.220 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:29:30.220 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:30.220 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:29:30.220 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.148323+0000", 2024-09-14T06:29:30.220 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:30.221 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:30.221 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:30.221 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:29:30.221 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.148323+0000", 2024-09-14T06:29:30.221 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:30.221 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:30.221 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:30.221 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:29:30.221 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.148448+0000", 2024-09-14T06:29:30.221 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00012475999999999999 2024-09-14T06:29:30.221 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:30.221 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:30.221 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:29:30.222 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.148451+0000", 2024-09-14T06:29:30.222 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.5370000000000002e-06 2024-09-14T06:29:30.222 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:30.222 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:30.222 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:29:30.222 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.148460+0000", 2024-09-14T06:29:30.222 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.2949999999999994e-06 2024-09-14T06:29:30.222 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:30.222 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:30.222 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:29:30.222 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.148465+0000", 2024-09-14T06:29:30.222 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.7570000000000001e-06 2024-09-14T06:29:30.223 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:30.223 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:30.223 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:29:30.223 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.196699+0000", 2024-09-14T06:29:30.223 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.048233542999999997 2024-09-14T06:29:30.223 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:30.223 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:30.223 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:29:30.223 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.196766+0000", 2024-09-14T06:29:30.223 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.7259000000000001e-05 2024-09-14T06:29:30.223 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:30.223 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:30.223 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:29:30.223 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.196862+0000", 2024-09-14T06:29:30.223 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.5836999999999997e-05 2024-09-14T06:29:30.224 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:30.224 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:30.224 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:30.224 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:30.224 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:30.224 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:15705 3.b 3:d24f5f7d:::rbd_data.112d92027590.000000000000000d:f [read 147456~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:29:30.224 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:26.148475+0000", 2024-09-14T06:29:30.224 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 4.0320043859999997, 2024-09-14T06:29:30.224 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.048589192000000003, 2024-09-14T06:29:30.224 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:29:30.224 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:29:30.224 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:29:30.224 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:29:30.224 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:29:30.225 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:30.225 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 15705 2024-09-14T06:29:30.225 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:30.225 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:29:30.225 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:30.225 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:29:30.225 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.148475+0000", 2024-09-14T06:29:30.225 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:30.225 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:30.225 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:30.225 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:29:30.225 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.148475+0000", 2024-09-14T06:29:30.225 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:30.225 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:30.225 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:30.226 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:29:30.226 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.148565+0000", 2024-09-14T06:29:30.226 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.9876000000000005e-05 2024-09-14T06:29:30.226 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:30.226 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:30.226 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:29:30.226 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.148568+0000", 2024-09-14T06:29:30.226 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.2210000000000002e-06 2024-09-14T06:29:30.226 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:30.226 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:30.226 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:29:30.226 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.148580+0000", 2024-09-14T06:29:30.226 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.2442e-05 2024-09-14T06:29:30.226 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:30.227 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:30.227 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:29:30.227 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.148590+0000", 2024-09-14T06:29:30.227 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.0860000000000008e-06 2024-09-14T06:29:30.227 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:30.227 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:30.227 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:29:30.227 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.196893+0000", 2024-09-14T06:29:30.227 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.048303328999999999 2024-09-14T06:29:30.227 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:30.227 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:30.227 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:29:30.227 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.196968+0000", 2024-09-14T06:29:30.228 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.4690000000000005e-05 2024-09-14T06:29:30.228 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:30.228 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:30.228 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:29:30.228 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.197064+0000", 2024-09-14T06:29:30.228 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.6547999999999998e-05 2024-09-14T06:29:30.228 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:30.228 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:30.228 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:30.228 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:30.228 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:30.228 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:15706 3.b 3:d24f5f7d:::rbd_data.112d92027590.000000000000000d:f [read 157696~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:29:30.228 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:26.148603+0000", 2024-09-14T06:29:30.228 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 4.0318761930000004, 2024-09-14T06:29:30.228 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.048652025000000002, 2024-09-14T06:29:30.229 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:29:30.229 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:29:30.229 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:29:30.229 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:29:30.229 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:29:30.229 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:30.229 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 15706 2024-09-14T06:29:30.229 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:30.229 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:29:30.229 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:30.229 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:29:30.229 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.148603+0000", 2024-09-14T06:29:30.229 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:30.230 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:30.230 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:30.230 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:29:30.230 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.148603+0000", 2024-09-14T06:29:30.230 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:30.230 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:30.230 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:30.230 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:29:30.230 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.148791+0000", 2024-09-14T06:29:30.230 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00018775700000000001 2024-09-14T06:29:30.230 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:30.230 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:30.230 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:29:30.230 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.148794+0000", 2024-09-14T06:29:30.231 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.9550000000000001e-06 2024-09-14T06:29:30.231 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:30.231 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:30.231 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:29:30.231 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.148802+0000", 2024-09-14T06:29:30.231 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.7789999999999996e-06 2024-09-14T06:29:30.231 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:30.231 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:30.231 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:29:30.231 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.148814+0000", 2024-09-14T06:29:30.231 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.2886e-05 2024-09-14T06:29:30.231 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:30.231 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:30.231 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:29:30.232 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.197104+0000", 2024-09-14T06:29:30.232 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.048289683 2024-09-14T06:29:30.232 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:30.232 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:30.232 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:29:30.232 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.197171+0000", 2024-09-14T06:29:30.232 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.7137000000000005e-05 2024-09-14T06:29:30.232 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:30.232 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:30.232 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:29:30.232 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.197255+0000", 2024-09-14T06:29:30.232 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.3827999999999998e-05 2024-09-14T06:29:30.232 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:30.232 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:30.233 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:30.233 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:30.233 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:30.233 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:15707 3.b 3:d24f5f7d:::rbd_data.112d92027590.000000000000000d:f [read 167936~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:29:30.233 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:26.148826+0000", 2024-09-14T06:29:30.233 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 4.0316535900000003, 2024-09-14T06:29:30.233 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.048578723999999997, 2024-09-14T06:29:30.233 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:29:30.233 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:29:30.233 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:29:30.233 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:29:30.233 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:29:30.233 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:30.234 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 15707 2024-09-14T06:29:30.234 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:30.234 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:29:30.234 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:30.234 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:29:30.234 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.148826+0000", 2024-09-14T06:29:30.234 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:30.234 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:30.234 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:30.234 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:29:30.234 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.148826+0000", 2024-09-14T06:29:30.234 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:30.234 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:30.234 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:30.234 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:29:30.234 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.149053+0000", 2024-09-14T06:29:30.235 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00022741100000000001 2024-09-14T06:29:30.235 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:30.235 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:30.235 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:29:30.235 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.149060+0000", 2024-09-14T06:29:30.235 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.8040000000000004e-06 2024-09-14T06:29:30.235 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:30.235 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:30.235 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:29:30.235 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.149069+0000", 2024-09-14T06:29:30.235 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.8189999999999998e-06 2024-09-14T06:29:30.235 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:30.235 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:30.235 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:29:30.236 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.149075+0000", 2024-09-14T06:29:30.236 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.2079999999999997e-06 2024-09-14T06:29:30.236 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:30.236 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:30.236 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:29:30.236 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.197299+0000", 2024-09-14T06:29:30.236 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.048223749000000003 2024-09-14T06:29:30.236 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:30.236 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:30.236 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:29:30.236 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.197343+0000", 2024-09-14T06:29:30.236 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.4721000000000001e-05 2024-09-14T06:29:30.236 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:30.237 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:30.237 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:29:30.237 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.197404+0000", 2024-09-14T06:29:30.237 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.1011999999999998e-05 2024-09-14T06:29:30.237 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:30.237 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:30.237 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:30.237 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:30.237 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:30.237 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:15708 3.b 3:d24f5f7d:::rbd_data.112d92027590.000000000000000d:f [read 178176~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:29:30.237 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:26.149084+0000", 2024-09-14T06:29:30.237 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 4.0313949649999996, 2024-09-14T06:29:30.237 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.048528769999999999, 2024-09-14T06:29:30.238 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:29:30.238 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:29:30.238 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:29:30.238 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:29:30.238 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:29:30.238 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:30.238 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 15708 2024-09-14T06:29:30.238 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:30.238 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:29:30.238 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:30.238 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:29:30.238 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.149084+0000", 2024-09-14T06:29:30.239 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:30.239 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:30.239 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:30.239 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:29:30.239 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.149084+0000", 2024-09-14T06:29:30.239 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:30.239 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:30.239 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:30.239 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:29:30.239 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.150587+0000", 2024-09-14T06:29:30.239 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.001503187 2024-09-14T06:29:30.239 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:30.239 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:30.239 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:29:30.240 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.150592+0000", 2024-09-14T06:29:30.240 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.07e-06 2024-09-14T06:29:30.240 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:30.240 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:30.240 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:29:30.240 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.150602+0000", 2024-09-14T06:29:30.240 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.0584999999999999e-05 2024-09-14T06:29:30.240 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:30.240 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:30.240 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:29:30.240 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.150609+0000", 2024-09-14T06:29:30.240 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.8560000000000001e-06 2024-09-14T06:29:30.240 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:30.240 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:30.241 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:29:30.241 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.197446+0000", 2024-09-14T06:29:30.241 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.046837328999999997 2024-09-14T06:29:30.241 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:30.241 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:30.241 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:29:30.241 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.197530+0000", 2024-09-14T06:29:30.241 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.3493999999999997e-05 2024-09-14T06:29:30.241 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:30.241 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:30.241 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:29:30.241 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.197613+0000", 2024-09-14T06:29:30.241 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.3249000000000002e-05 2024-09-14T06:29:30.241 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:30.242 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:30.242 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:30.242 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:30.242 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:30.242 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:15736 3.b 3:d24f5f7d:::rbd_data.112d92027590.000000000000000d:f [read 464896~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:29:30.242 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:26.154778+0000", 2024-09-14T06:29:30.242 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 4.0257009500000001, 2024-09-14T06:29:30.242 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.048624703999999998, 2024-09-14T06:29:30.242 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:29:30.242 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:29:30.242 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:29:30.242 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:29:30.242 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:29:30.242 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:30.242 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 15736 2024-09-14T06:29:30.242 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:30.243 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:29:30.243 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:30.243 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:29:30.243 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.154778+0000", 2024-09-14T06:29:30.243 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:30.243 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:30.243 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:30.243 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:29:30.243 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.154778+0000", 2024-09-14T06:29:30.243 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:30.243 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:30.243 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:30.243 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:29:30.243 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.154847+0000", 2024-09-14T06:29:30.243 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.8374999999999995e-05 2024-09-14T06:29:30.243 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:30.244 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:30.244 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:29:30.244 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.154851+0000", 2024-09-14T06:29:30.244 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.4830000000000001e-06 2024-09-14T06:29:30.244 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:30.244 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:30.244 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:29:30.244 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.154863+0000", 2024-09-14T06:29:30.244 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.188e-05 2024-09-14T06:29:30.244 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:30.244 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:30.244 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:29:30.244 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.154870+0000", 2024-09-14T06:29:30.244 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.8159999999999996e-06 2024-09-14T06:29:30.244 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:30.245 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:30.245 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:29:30.245 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.203218+0000", 2024-09-14T06:29:30.245 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.048348440999999999 2024-09-14T06:29:30.245 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:30.245 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:30.245 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:29:30.245 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.203302+0000", 2024-09-14T06:29:30.245 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.3764999999999995e-05 2024-09-14T06:29:30.245 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:30.245 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:30.245 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:29:30.245 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.203403+0000", 2024-09-14T06:29:30.245 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000100944 2024-09-14T06:29:30.246 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:30.246 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:30.246 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:30.246 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:30.246 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:30.246 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:15737 3.b 3:d24f5f7d:::rbd_data.112d92027590.000000000000000d:f [read 475136~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:29:30.246 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:26.154883+0000", 2024-09-14T06:29:30.246 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 4.0255967369999999, 2024-09-14T06:29:30.246 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.048784932000000003, 2024-09-14T06:29:30.246 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:29:30.246 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:29:30.246 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:29:30.246 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:29:30.246 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:29:30.246 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:30.247 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 15737 2024-09-14T06:29:30.247 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:30.247 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:29:30.247 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:30.247 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:29:30.247 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.154883+0000", 2024-09-14T06:29:30.247 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:30.247 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:30.247 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:30.247 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:29:30.247 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.154883+0000", 2024-09-14T06:29:30.247 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:30.247 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:30.248 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:30.248 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:29:30.248 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.155040+0000", 2024-09-14T06:29:30.248 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000157236 2024-09-14T06:29:30.248 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:30.248 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:30.248 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:29:30.248 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.155044+0000", 2024-09-14T06:29:30.248 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.1330000000000001e-06 2024-09-14T06:29:30.248 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:30.248 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:30.248 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:29:30.248 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.155053+0000", 2024-09-14T06:29:30.248 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.8869999999999998e-06 2024-09-14T06:29:30.248 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:30.249 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:30.249 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:29:30.249 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.155058+0000", 2024-09-14T06:29:30.249 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.3940000000000004e-06 2024-09-14T06:29:30.249 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:30.249 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:30.249 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:29:30.249 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.203478+0000", 2024-09-14T06:29:30.249 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.048420073000000001 2024-09-14T06:29:30.249 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:30.249 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:30.249 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:29:30.249 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.203566+0000", 2024-09-14T06:29:30.249 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.7620999999999993e-05 2024-09-14T06:29:30.249 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:30.250 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:30.250 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:29:30.250 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.203667+0000", 2024-09-14T06:29:30.250 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000101588 2024-09-14T06:29:30.250 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:30.250 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:30.250 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:30.250 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:30.250 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:30.250 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:15738 3.b 3:d24f5f7d:::rbd_data.112d92027590.000000000000000d:f [read 485376~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:29:30.250 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:26.155069+0000", 2024-09-14T06:29:30.250 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 4.025409807, 2024-09-14T06:29:30.250 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.048810049000000001, 2024-09-14T06:29:30.250 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:29:30.250 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:29:30.250 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:29:30.251 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:29:30.251 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:29:30.251 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:30.251 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 15738 2024-09-14T06:29:30.251 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:30.251 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:29:30.251 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:30.251 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:29:30.251 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.155069+0000", 2024-09-14T06:29:30.251 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:30.251 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:30.251 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:30.251 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:29:30.251 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.155069+0000", 2024-09-14T06:29:30.251 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:30.252 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:30.252 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:30.252 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:29:30.252 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.155278+0000", 2024-09-14T06:29:30.252 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00020823400000000001 2024-09-14T06:29:30.252 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:30.252 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:30.252 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:29:30.252 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.155282+0000", 2024-09-14T06:29:30.252 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.4730000000000002e-06 2024-09-14T06:29:30.252 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:30.252 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:30.252 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:29:30.252 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.155294+0000", 2024-09-14T06:29:30.253 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.1347e-05 2024-09-14T06:29:30.253 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:30.253 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:30.253 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:29:30.253 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.155302+0000", 2024-09-14T06:29:30.253 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.7339999999999997e-06 2024-09-14T06:29:30.253 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:30.253 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:30.253 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:29:30.253 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.203701+0000", 2024-09-14T06:29:30.253 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.048399144999999998 2024-09-14T06:29:30.253 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:30.253 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:30.254 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:29:30.254 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.203778+0000", 2024-09-14T06:29:30.254 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.6169999999999997e-05 2024-09-14T06:29:30.254 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:30.254 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:30.254 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:29:30.254 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.203880+0000", 2024-09-14T06:29:30.254 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000101946 2024-09-14T06:29:30.254 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:30.254 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:30.254 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:30.254 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:30.254 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:30.254 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:15739 3.b 3:d24f5f7d:::rbd_data.112d92027590.000000000000000d:f [read 495616~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:29:30.254 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:26.155331+0000", 2024-09-14T06:29:30.255 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 4.0251482909999998, 2024-09-14T06:29:30.255 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.048708649999999999, 2024-09-14T06:29:30.255 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:29:30.255 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:29:30.255 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:29:30.255 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:29:30.255 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:29:30.255 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:30.255 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 15739 2024-09-14T06:29:30.255 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:30.255 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:29:30.256 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:30.256 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:29:30.256 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.155331+0000", 2024-09-14T06:29:30.256 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:30.256 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:30.256 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:30.256 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:29:30.256 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.155331+0000", 2024-09-14T06:29:30.256 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:30.256 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:30.256 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:30.256 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:29:30.256 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.155526+0000", 2024-09-14T06:29:30.256 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00019500299999999999 2024-09-14T06:29:30.256 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:30.257 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:30.257 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:29:30.257 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.155532+0000", 2024-09-14T06:29:30.257 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.6620000000000002e-06 2024-09-14T06:29:30.257 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:30.257 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:30.257 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:29:30.257 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.155541+0000", 2024-09-14T06:29:30.257 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.2240000000000001e-06 2024-09-14T06:29:30.257 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:30.257 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:30.257 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:29:30.257 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.155547+0000", 2024-09-14T06:29:30.257 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.4559999999999997e-06 2024-09-14T06:29:30.258 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:30.258 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:30.258 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:29:30.258 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.203917+0000", 2024-09-14T06:29:30.258 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.048370017000000001 2024-09-14T06:29:30.258 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:30.258 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:30.258 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:29:30.258 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.203970+0000", 2024-09-14T06:29:30.258 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.2308999999999997e-05 2024-09-14T06:29:30.258 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:30.258 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:30.259 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:29:30.259 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.204040+0000", 2024-09-14T06:29:30.259 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.9979000000000002e-05 2024-09-14T06:29:30.259 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:30.259 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:30.259 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:30.259 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:30.259 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:30.259 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:15740 3.b 3:d24f5f7d:::rbd_data.112d92027590.000000000000000d:f [read 505856~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:29:30.259 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:26.155558+0000", 2024-09-14T06:29:30.259 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 4.0249211669999996, 2024-09-14T06:29:30.259 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.048618724000000002, 2024-09-14T06:29:30.259 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:29:30.259 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:29:30.259 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:29:30.260 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:29:30.260 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:29:30.260 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:30.260 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 15740 2024-09-14T06:29:30.260 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:30.260 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:29:30.260 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:30.260 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:29:30.260 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.155558+0000", 2024-09-14T06:29:30.260 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:30.260 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:30.260 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:30.260 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:29:30.260 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.155558+0000", 2024-09-14T06:29:30.261 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:30.261 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:30.261 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:30.261 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:29:30.261 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.155766+0000", 2024-09-14T06:29:30.261 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00020751999999999999 2024-09-14T06:29:30.261 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:30.261 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:30.261 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:29:30.261 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.155769+0000", 2024-09-14T06:29:30.261 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.6500000000000002e-06 2024-09-14T06:29:30.261 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:30.261 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:30.261 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:29:30.262 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.155779+0000", 2024-09-14T06:29:30.262 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.0179e-05 2024-09-14T06:29:30.262 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:30.262 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:30.262 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:29:30.262 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.155785+0000", 2024-09-14T06:29:30.262 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.9120000000000003e-06 2024-09-14T06:29:30.262 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:30.262 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:30.262 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:29:30.262 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.204072+0000", 2024-09-14T06:29:30.262 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.048286463000000002 2024-09-14T06:29:30.262 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:30.262 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:30.263 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:29:30.263 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.204119+0000", 2024-09-14T06:29:30.263 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.7023e-05 2024-09-14T06:29:30.263 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:30.263 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:30.263 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:29:30.263 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.204177+0000", 2024-09-14T06:29:30.263 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.7976999999999997e-05 2024-09-14T06:29:30.263 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:30.263 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:30.263 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:30.263 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:30.263 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:30.264 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:15742 3.b 3:d24f5f7d:::rbd_data.112d92027590.000000000000000d:f [read 526336~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:29:30.264 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:26.156014+0000", 2024-09-14T06:29:30.264 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 4.0244655820000004, 2024-09-14T06:29:30.264 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.048548612999999997, 2024-09-14T06:29:30.264 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:29:30.264 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:29:30.264 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:29:30.264 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:29:30.264 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:29:30.264 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:30.264 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 15742 2024-09-14T06:29:30.264 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:30.264 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:29:30.264 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:30.265 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:29:30.265 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.156014+0000", 2024-09-14T06:29:30.265 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:30.265 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:30.265 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:30.265 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:29:30.265 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.156014+0000", 2024-09-14T06:29:30.265 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:30.265 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:30.265 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:30.265 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:29:30.265 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.156246+0000", 2024-09-14T06:29:30.265 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00023262700000000001 2024-09-14T06:29:30.265 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:30.265 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:30.266 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:29:30.266 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.156251+0000", 2024-09-14T06:29:30.266 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.0209999999999999e-06 2024-09-14T06:29:30.266 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:30.266 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:30.266 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:29:30.266 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.156262+0000", 2024-09-14T06:29:30.266 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.0168e-05 2024-09-14T06:29:30.266 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:30.266 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:30.266 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:29:30.266 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.156269+0000", 2024-09-14T06:29:30.266 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.0999999999999998e-06 2024-09-14T06:29:30.267 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:30.267 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:30.267 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:29:30.267 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.204363+0000", 2024-09-14T06:29:30.267 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.048094455000000001 2024-09-14T06:29:30.267 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:30.267 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:30.267 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:29:30.267 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.204452+0000", 2024-09-14T06:29:30.267 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.8899000000000004e-05 2024-09-14T06:29:30.267 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:30.267 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:30.268 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:29:30.268 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.204562+0000", 2024-09-14T06:29:30.268 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000110343 2024-09-14T06:29:30.268 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:30.268 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:30.268 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:30.268 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:30.268 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:30.268 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:15746 3.b 3:d24f5f7d:::rbd_data.112d92027590.000000000000000d:f [read 567296~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:29:30.268 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:26.156734+0000", 2024-09-14T06:29:30.268 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 4.0237450920000004, 2024-09-14T06:29:30.268 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.048564930999999999, 2024-09-14T06:29:30.268 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:29:30.269 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:29:30.269 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:29:30.269 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:29:30.269 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:29:30.269 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:30.269 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 15746 2024-09-14T06:29:30.269 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:30.269 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:29:30.269 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:30.269 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:29:30.269 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.156734+0000", 2024-09-14T06:29:30.269 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:30.269 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:30.269 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:30.270 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:29:30.270 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.156734+0000", 2024-09-14T06:29:30.270 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:30.270 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:30.270 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:30.270 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:29:30.270 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.156952+0000", 2024-09-14T06:29:30.270 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000217844 2024-09-14T06:29:30.270 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:30.270 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:30.270 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:29:30.270 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.156956+0000", 2024-09-14T06:29:30.270 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.0790000000000002e-06 2024-09-14T06:29:30.270 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:30.271 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:30.271 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:29:30.271 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.156966+0000", 2024-09-14T06:29:30.271 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.4150000000000005e-06 2024-09-14T06:29:30.271 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:30.271 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:30.271 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:29:30.271 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.156972+0000", 2024-09-14T06:29:30.271 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.365e-06 2024-09-14T06:29:30.271 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:30.271 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:30.271 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:29:30.271 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.205128+0000", 2024-09-14T06:29:30.271 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.048155833000000002 2024-09-14T06:29:30.271 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:30.272 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:30.272 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:29:30.272 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.205205+0000", 2024-09-14T06:29:30.272 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.7250999999999996e-05 2024-09-14T06:29:30.272 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:30.272 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:30.272 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:29:30.272 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.205299+0000", 2024-09-14T06:29:30.272 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.4143999999999999e-05 2024-09-14T06:29:30.272 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:30.272 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:30.272 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:30.272 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:30.272 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:30.273 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-14T06:29:30.274 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-14T06:29:30.274 INFO:tasks.ceph.osd.1.smithi094.stderr:2024-09-14T06:29:30.272+0000 7f328e47a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33641) UID: 0 2024-09-14T06:29:30.375 INFO:tasks.ceph.osd.7.smithi138.stderr:2024-09-14T06:29:30.374+0000 7f3c6a21d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33623) UID: 0 2024-09-14T06:29:30.381 INFO:teuthology.orchestra.run.smithi138.stdout:{ 2024-09-14T06:29:30.381 INFO:teuthology.orchestra.run.smithi138.stdout: "ops": [], 2024-09-14T06:29:30.382 INFO:teuthology.orchestra.run.smithi138.stdout: "num_ops": 0 2024-09-14T06:29:30.382 INFO:teuthology.orchestra.run.smithi138.stdout:} 2024-09-14T06:29:30.390 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-14T06:29:30.476 INFO:tasks.ceph.osd.7.smithi138.stderr:2024-09-14T06:29:30.475+0000 7f3c6a21d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33623) UID: 0 2024-09-14T06:29:30.533 INFO:teuthology.orchestra.run.smithi138.stdout:{ 2024-09-14T06:29:30.533 INFO:teuthology.orchestra.run.smithi138.stdout: "ops": [], 2024-09-14T06:29:30.533 INFO:teuthology.orchestra.run.smithi138.stdout: "complaint_time": 30, 2024-09-14T06:29:30.533 INFO:teuthology.orchestra.run.smithi138.stdout: "num_blocked_ops": 0 2024-09-14T06:29:30.533 INFO:teuthology.orchestra.run.smithi138.stdout:} 2024-09-14T06:29:30.541 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-14T06:29:30.574 INFO:teuthology.orchestra.run.smithi094.stderr:noscrub is unset 2024-09-14T06:29:30.583 INFO:tasks.ceph.osd.7.smithi138.stderr:2024-09-14T06:29:30.582+0000 7f3c6a21d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33623) UID: 0 2024-09-14T06:29:30.678 INFO:tasks.ceph.osd.5.smithi138.stderr:2024-09-14T06:29:30.677+0000 7fe13746f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33618) UID: 0 2024-09-14T06:29:30.718 INFO:teuthology.orchestra.run.smithi138.stdout:{ 2024-09-14T06:29:30.718 INFO:teuthology.orchestra.run.smithi138.stdout: "size": 20, 2024-09-14T06:29:30.718 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 600, 2024-09-14T06:29:30.719 INFO:teuthology.orchestra.run.smithi138.stdout: "ops": [ 2024-09-14T06:29:30.719 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:30.719 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:19328 3.3 3:c34e5e51:::rbd_data.112d92027590.0000000000000024:f [read 1926144~1024 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:29:30.719 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:27.059768+0000", 2024-09-14T06:29:30.719 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 3.6564906740000001, 2024-09-14T06:29:30.719 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046369147999999999, 2024-09-14T06:29:30.719 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:29:30.719 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:29:30.719 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:29:30.719 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:29:30.719 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:29:30.719 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:30.719 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 19328 2024-09-14T06:29:30.720 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:30.720 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:29:30.720 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:30.720 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:29:30.720 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.059768+0000", 2024-09-14T06:29:30.720 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:30.720 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:30.720 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:30.720 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:29:30.720 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.059768+0000", 2024-09-14T06:29:30.720 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:30.720 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:30.721 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:30.721 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:29:30.721 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.059874+0000", 2024-09-14T06:29:30.721 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00010551400000000001 2024-09-14T06:29:30.721 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:30.721 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:30.721 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:29:30.721 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.059877+0000", 2024-09-14T06:29:30.721 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.8279999999999999e-06 2024-09-14T06:29:30.721 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:30.721 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:30.721 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:29:30.721 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.059884+0000", 2024-09-14T06:29:30.722 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.6389999999999992e-06 2024-09-14T06:29:30.722 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:30.722 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:30.722 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:29:30.722 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.059890+0000", 2024-09-14T06:29:30.722 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.49e-06 2024-09-14T06:29:30.722 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:30.722 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:30.722 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:29:30.722 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.105804+0000", 2024-09-14T06:29:30.722 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.045914242000000001 2024-09-14T06:29:30.722 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:30.722 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:30.722 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:29:30.723 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.105886+0000", 2024-09-14T06:29:30.723 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.2305999999999994e-05 2024-09-14T06:29:30.723 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:30.723 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:30.723 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:29:30.723 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.106138+0000", 2024-09-14T06:29:30.723 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.000251129 2024-09-14T06:29:30.723 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:30.723 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:29:30.723 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:30.723 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:30.723 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:30.723 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:19329 3.3 3:c34e5e51:::rbd_data.112d92027590.0000000000000024:f [read 1936384~1024 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:29:30.723 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:27.059899+0000", 2024-09-14T06:29:30.724 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 3.6563601509999999, 2024-09-14T06:29:30.724 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046572122000000001, 2024-09-14T06:29:30.724 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:29:30.724 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:29:30.724 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:29:30.724 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:29:30.724 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:29:30.724 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:30.724 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 19329 2024-09-14T06:29:30.724 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:30.724 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:29:30.724 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:30.724 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:29:30.724 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.059899+0000", 2024-09-14T06:29:30.725 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:30.725 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:30.725 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:30.725 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:29:30.725 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.059899+0000", 2024-09-14T06:29:30.725 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:30.725 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:30.725 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:30.725 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:29:30.725 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062501+0000", 2024-09-14T06:29:30.725 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.0026016419999999999 2024-09-14T06:29:30.725 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:30.725 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:30.725 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:29:30.726 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062514+0000", 2024-09-14T06:29:30.726 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.2947e-05 2024-09-14T06:29:30.726 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:30.726 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:30.726 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:29:30.726 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062523+0000", 2024-09-14T06:29:30.726 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.5519999999999993e-06 2024-09-14T06:29:30.726 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:30.726 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:30.726 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:29:30.726 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062534+0000", 2024-09-14T06:29:30.726 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.1416e-05 2024-09-14T06:29:30.726 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:30.727 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:30.727 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:29:30.727 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.106179+0000", 2024-09-14T06:29:30.727 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.043644049999999997 2024-09-14T06:29:30.727 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:30.727 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:30.727 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:29:30.727 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.106262+0000", 2024-09-14T06:29:30.727 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.3178999999999998e-05 2024-09-14T06:29:30.727 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:30.727 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:30.727 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:29:30.727 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.106471+0000", 2024-09-14T06:29:30.728 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00020933599999999999 2024-09-14T06:29:30.728 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:30.728 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:29:30.728 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:30.728 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:30.728 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:30.728 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:19351 3.3 3:c34e5e51:::rbd_data.112d92027590.0000000000000024:f [read 1159168~1024 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:29:30.728 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:27.062766+0000", 2024-09-14T06:29:30.728 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 3.6534929260000002, 2024-09-14T06:29:30.728 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046099454999999998, 2024-09-14T06:29:30.728 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:29:30.728 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:29:30.728 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:29:30.728 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:29:30.729 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:29:30.729 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:30.729 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 19351 2024-09-14T06:29:30.729 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:30.729 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:29:30.729 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:30.729 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:29:30.729 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062766+0000", 2024-09-14T06:29:30.729 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:30.729 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:30.729 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:30.729 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:29:30.730 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062766+0000", 2024-09-14T06:29:30.730 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:30.730 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:30.730 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:30.730 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:29:30.730 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062766+0000", 2024-09-14T06:29:30.730 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.6e-07 2024-09-14T06:29:30.730 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:30.730 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:30.730 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:29:30.730 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062767+0000", 2024-09-14T06:29:30.730 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.0070000000000001e-06 2024-09-14T06:29:30.731 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:30.731 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:30.731 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:29:30.731 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062785+0000", 2024-09-14T06:29:30.731 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.7884999999999999e-05 2024-09-14T06:29:30.731 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:30.731 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:30.731 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:29:30.731 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062790+0000", 2024-09-14T06:29:30.731 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.0900000000000004e-06 2024-09-14T06:29:30.731 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:30.731 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:30.731 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:29:30.732 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.108735+0000", 2024-09-14T06:29:30.732 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.045944411999999997 2024-09-14T06:29:30.732 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:30.732 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:30.732 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:29:30.732 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.108809+0000", 2024-09-14T06:29:30.732 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.4157000000000002e-05 2024-09-14T06:29:30.732 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:30.732 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:30.732 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:29:30.732 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.108866+0000", 2024-09-14T06:29:30.732 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.6644000000000002e-05 2024-09-14T06:29:30.732 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:30.733 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:29:30.733 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:30.733 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:30.733 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:30.733 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:19352 3.3 3:c34e5e51:::rbd_data.112d92027590.0000000000000024:f [read 1169408~1024 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:29:30.733 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:27.062798+0000", 2024-09-14T06:29:30.733 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 3.653461515, 2024-09-14T06:29:30.733 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046255900000000003, 2024-09-14T06:29:30.733 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:29:30.733 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:29:30.733 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:29:30.733 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:29:30.733 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:29:30.734 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:30.734 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 19352 2024-09-14T06:29:30.734 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:30.734 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:29:30.734 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:30.734 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:29:30.734 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062798+0000", 2024-09-14T06:29:30.734 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:30.734 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:30.734 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:30.734 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:29:30.734 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062798+0000", 2024-09-14T06:29:30.734 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:30.734 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:30.735 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:30.735 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:29:30.735 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062798+0000", 2024-09-14T06:29:30.735 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.37e-07 2024-09-14T06:29:30.735 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:30.735 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:30.735 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:29:30.735 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062800+0000", 2024-09-14T06:29:30.735 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.0269999999999998e-06 2024-09-14T06:29:30.735 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:30.735 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:30.735 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:29:30.735 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062806+0000", 2024-09-14T06:29:30.735 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.6690000000000003e-06 2024-09-14T06:29:30.735 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:30.736 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:30.736 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:29:30.736 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062810+0000", 2024-09-14T06:29:30.736 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.3449999999999999e-06 2024-09-14T06:29:30.736 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:30.736 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:30.736 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:29:30.736 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.108892+0000", 2024-09-14T06:29:30.736 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046081715000000002 2024-09-14T06:29:30.736 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:30.736 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:30.736 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:29:30.736 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.108940+0000", 2024-09-14T06:29:30.736 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.7815999999999998e-05 2024-09-14T06:29:30.736 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:30.736 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:30.737 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:29:30.737 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.109053+0000", 2024-09-14T06:29:30.737 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.000113791 2024-09-14T06:29:30.737 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:30.737 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:29:30.737 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:30.737 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:30.737 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:30.737 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:19353 3.3 3:c34e5e51:::rbd_data.112d92027590.0000000000000024:f [read 1179648~1024 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:29:30.737 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:27.062816+0000", 2024-09-14T06:29:30.737 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 3.6534429959999999, 2024-09-14T06:29:30.737 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046481256999999998, 2024-09-14T06:29:30.737 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:29:30.737 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:29:30.737 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:29:30.738 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:29:30.738 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:29:30.738 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:30.738 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 19353 2024-09-14T06:29:30.738 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:30.738 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:29:30.738 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:30.738 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:29:30.738 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062816+0000", 2024-09-14T06:29:30.738 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:30.738 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:30.738 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:30.738 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:29:30.738 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062816+0000", 2024-09-14T06:29:30.738 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:30.739 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:30.739 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:30.739 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:29:30.739 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062816+0000", 2024-09-14T06:29:30.739 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.2899999999999999e-07 2024-09-14T06:29:30.739 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:30.739 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:30.739 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:29:30.739 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062818+0000", 2024-09-14T06:29:30.739 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.699e-06 2024-09-14T06:29:30.739 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:30.739 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:30.739 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:29:30.739 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062825+0000", 2024-09-14T06:29:30.739 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.878e-06 2024-09-14T06:29:30.740 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:30.740 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:30.740 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:29:30.740 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062830+0000", 2024-09-14T06:29:30.740 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.5410000000000002e-06 2024-09-14T06:29:30.740 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:30.740 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:30.740 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:29:30.740 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.109096+0000", 2024-09-14T06:29:30.740 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046266517 2024-09-14T06:29:30.740 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:30.740 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:30.740 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:29:30.741 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.109190+0000", 2024-09-14T06:29:30.741 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.4000999999999997e-05 2024-09-14T06:29:30.741 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:30.741 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:30.741 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:29:30.741 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.109297+0000", 2024-09-14T06:29:30.741 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.000107292 2024-09-14T06:29:30.741 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:30.741 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:29:30.741 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:30.741 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:30.741 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:30.741 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:19354 3.3 3:c34e5e51:::rbd_data.112d92027590.0000000000000024:f [read 1189888~1024 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:29:30.741 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:27.062842+0000", 2024-09-14T06:29:30.741 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 3.653417541, 2024-09-14T06:29:30.742 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046594257, 2024-09-14T06:29:30.742 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:29:30.742 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:29:30.742 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:29:30.742 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:29:30.742 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:29:30.742 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:30.742 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 19354 2024-09-14T06:29:30.742 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:30.742 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:29:30.742 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:30.742 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:29:30.742 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062842+0000", 2024-09-14T06:29:30.742 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:30.742 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:30.742 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:30.743 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:29:30.743 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062842+0000", 2024-09-14T06:29:30.743 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:30.743 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:30.743 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:30.743 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:29:30.743 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062842+0000", 2024-09-14T06:29:30.743 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.7399999999999999e-07 2024-09-14T06:29:30.743 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:30.743 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:30.743 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:29:30.743 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062859+0000", 2024-09-14T06:29:30.743 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.7010000000000001e-05 2024-09-14T06:29:30.744 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:30.744 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:30.744 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:29:30.744 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062866+0000", 2024-09-14T06:29:30.744 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.9990000000000002e-06 2024-09-14T06:29:30.744 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:30.744 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:30.744 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:29:30.744 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062873+0000", 2024-09-14T06:29:30.744 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.8569999999999997e-06 2024-09-14T06:29:30.744 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:30.744 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:30.744 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:29:30.745 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.109335+0000", 2024-09-14T06:29:30.745 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046461943999999998 2024-09-14T06:29:30.745 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:30.745 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:30.745 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:29:30.745 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.109377+0000", 2024-09-14T06:29:30.745 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.2558999999999997e-05 2024-09-14T06:29:30.745 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:30.745 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:30.745 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:29:30.745 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.109436+0000", 2024-09-14T06:29:30.745 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.8514000000000003e-05 2024-09-14T06:29:30.746 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:30.746 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:29:30.746 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:30.746 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:30.746 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:30.746 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:19355 3.3 3:c34e5e51:::rbd_data.112d92027590.0000000000000024:f [read 1200128~1024 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:29:30.746 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:27.062884+0000", 2024-09-14T06:29:30.746 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 3.6533748680000002, 2024-09-14T06:29:30.746 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046681650999999998, 2024-09-14T06:29:30.746 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:29:30.746 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:29:30.746 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:29:30.746 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:29:30.747 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:29:30.747 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:30.747 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 19355 2024-09-14T06:29:30.747 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:30.747 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:29:30.747 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:30.747 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:29:30.747 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062884+0000", 2024-09-14T06:29:30.747 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:30.747 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:30.747 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:30.747 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:29:30.748 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062884+0000", 2024-09-14T06:29:30.748 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:30.748 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:30.748 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:30.748 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:29:30.748 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062885+0000", 2024-09-14T06:29:30.748 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.8000000000000001e-07 2024-09-14T06:29:30.748 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:30.748 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:30.748 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:29:30.748 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062886+0000", 2024-09-14T06:29:30.749 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.8920000000000001e-06 2024-09-14T06:29:30.749 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:30.749 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:30.749 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:29:30.749 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062892+0000", 2024-09-14T06:29:30.749 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.84e-06 2024-09-14T06:29:30.749 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:30.749 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:30.749 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:29:30.749 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062899+0000", 2024-09-14T06:29:30.749 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.5339999999999996e-06 2024-09-14T06:29:30.749 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:30.750 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:30.750 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:29:30.750 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.109466+0000", 2024-09-14T06:29:30.750 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046566919999999998 2024-09-14T06:29:30.750 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:30.750 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:30.750 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:29:30.750 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.109510+0000", 2024-09-14T06:29:30.750 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.4153000000000002e-05 2024-09-14T06:29:30.750 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:30.750 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:30.751 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:29:30.751 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.109566+0000", 2024-09-14T06:29:30.751 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.5931999999999999e-05 2024-09-14T06:29:30.751 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:30.751 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:29:30.751 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:30.751 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:30.751 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:30.751 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:19356 3.3 3:c34e5e51:::rbd_data.112d92027590.0000000000000024:f [read 1210368~1024 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:29:30.751 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:27.062908+0000", 2024-09-14T06:29:30.751 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 3.6533514309999999, 2024-09-14T06:29:30.751 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046807498000000003, 2024-09-14T06:29:30.751 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:29:30.752 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:29:30.752 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:29:30.752 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:29:30.752 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:29:30.752 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:30.752 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 19356 2024-09-14T06:29:30.752 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:30.752 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:29:30.752 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:30.752 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:29:30.752 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062908+0000", 2024-09-14T06:29:30.752 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:30.752 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:30.752 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:30.752 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:29:30.753 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062908+0000", 2024-09-14T06:29:30.753 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:30.753 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:30.753 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:30.753 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:29:30.753 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062908+0000", 2024-09-14T06:29:30.753 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.5699999999999998e-07 2024-09-14T06:29:30.753 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:30.753 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:30.753 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:29:30.753 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062910+0000", 2024-09-14T06:29:30.753 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.7710000000000001e-06 2024-09-14T06:29:30.753 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:30.753 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:30.754 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:29:30.754 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062920+0000", 2024-09-14T06:29:30.754 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.0654e-05 2024-09-14T06:29:30.754 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:30.754 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:30.754 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:29:30.754 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062931+0000", 2024-09-14T06:29:30.754 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.075e-05 2024-09-14T06:29:30.754 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:30.754 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:30.754 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:29:30.754 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.109595+0000", 2024-09-14T06:29:30.754 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046663713000000002 2024-09-14T06:29:30.754 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:30.754 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:30.755 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:29:30.755 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.109632+0000", 2024-09-14T06:29:30.755 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.7505000000000001e-05 2024-09-14T06:29:30.755 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:30.755 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:30.755 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:29:30.755 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.109715+0000", 2024-09-14T06:29:30.755 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.2748000000000001e-05 2024-09-14T06:29:30.755 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:30.755 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:29:30.755 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:30.755 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:30.755 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:30.755 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:19357 3.3 3:c34e5e51:::rbd_data.112d92027590.0000000000000024:f [read 1220608~1024 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:29:30.755 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:27.062939+0000", 2024-09-14T06:29:30.756 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 3.6533199830000003, 2024-09-14T06:29:30.756 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046920508, 2024-09-14T06:29:30.756 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:29:30.756 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:29:30.756 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:29:30.756 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:29:30.756 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:29:30.756 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:30.756 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 19357 2024-09-14T06:29:30.756 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:30.756 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:29:30.756 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:30.756 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:29:30.757 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062939+0000", 2024-09-14T06:29:30.757 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:30.757 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:30.757 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:30.757 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:29:30.757 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062939+0000", 2024-09-14T06:29:30.757 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:30.757 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:30.757 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:30.757 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:29:30.757 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062940+0000", 2024-09-14T06:29:30.757 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.32e-07 2024-09-14T06:29:30.757 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:30.758 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:30.758 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:29:30.758 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062942+0000", 2024-09-14T06:29:30.758 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.1919999999999999e-06 2024-09-14T06:29:30.758 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:30.758 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:30.758 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:29:30.758 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062948+0000", 2024-09-14T06:29:30.758 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.1149999999999999e-06 2024-09-14T06:29:30.758 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:30.758 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:30.758 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:29:30.758 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062960+0000", 2024-09-14T06:29:30.758 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.1895e-05 2024-09-14T06:29:30.759 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:30.759 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:30.759 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:29:30.759 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.109704+0000", 2024-09-14T06:29:30.759 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046744584999999998 2024-09-14T06:29:30.759 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:30.759 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:30.759 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:29:30.759 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.109777+0000", 2024-09-14T06:29:30.759 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.2282999999999999e-05 2024-09-14T06:29:30.759 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:30.759 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:30.759 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:29:30.759 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.109860+0000", 2024-09-14T06:29:30.759 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.3005999999999998e-05 2024-09-14T06:29:30.759 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:30.760 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:29:30.760 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:30.760 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:30.760 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:30.760 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:19358 3.3 3:c34e5e51:::rbd_data.112d92027590.0000000000000024:f [read 1230848~1024 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:29:30.760 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:27.062967+0000", 2024-09-14T06:29:30.760 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 3.6532923529999999, 2024-09-14T06:29:30.760 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047144471, 2024-09-14T06:29:30.760 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:29:30.760 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:29:30.760 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:29:30.760 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:29:30.760 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:29:30.760 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:30.760 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 19358 2024-09-14T06:29:30.761 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:30.761 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:29:30.761 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:30.761 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:29:30.761 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062967+0000", 2024-09-14T06:29:30.761 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:30.761 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:30.761 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:30.761 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:29:30.761 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062967+0000", 2024-09-14T06:29:30.761 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:30.761 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:30.761 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:30.762 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:29:30.762 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062967+0000", 2024-09-14T06:29:30.762 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.89e-07 2024-09-14T06:29:30.762 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:30.762 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:30.762 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:29:30.762 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062969+0000", 2024-09-14T06:29:30.762 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.9929999999999998e-06 2024-09-14T06:29:30.762 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:30.762 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:30.762 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:29:30.762 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062975+0000", 2024-09-14T06:29:30.762 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.9030000000000001e-06 2024-09-14T06:29:30.763 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:30.763 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:30.763 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:29:30.763 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062979+0000", 2024-09-14T06:29:30.763 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.2250000000000002e-06 2024-09-14T06:29:30.763 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:30.763 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:30.763 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:29:30.763 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.109934+0000", 2024-09-14T06:29:30.763 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046954410000000002 2024-09-14T06:29:30.763 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:30.763 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:30.763 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:29:30.763 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.110007+0000", 2024-09-14T06:29:30.763 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.3829000000000001e-05 2024-09-14T06:29:30.763 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:30.764 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:30.764 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:29:30.764 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.110111+0000", 2024-09-14T06:29:30.764 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00010372199999999999 2024-09-14T06:29:30.764 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:30.764 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:29:30.764 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:30.764 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:30.764 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:30.764 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:19359 3.3 3:c34e5e51:::rbd_data.112d92027590.0000000000000024:f [read 1241088~1024 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:29:30.764 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:27.062987+0000", 2024-09-14T06:29:30.764 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 3.6532722629999999, 2024-09-14T06:29:30.764 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047297182, 2024-09-14T06:29:30.764 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:29:30.765 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:29:30.765 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:29:30.765 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:29:30.765 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:29:30.765 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:30.765 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 19359 2024-09-14T06:29:30.765 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:30.765 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:29:30.765 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:30.765 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:29:30.765 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062987+0000", 2024-09-14T06:29:30.765 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:30.766 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:30.766 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:30.766 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:29:30.766 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062987+0000", 2024-09-14T06:29:30.766 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:30.766 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:30.766 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:30.766 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:29:30.766 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062987+0000", 2024-09-14T06:29:30.766 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.2500000000000001e-07 2024-09-14T06:29:30.766 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:30.766 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:30.766 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:29:30.766 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062989+0000", 2024-09-14T06:29:30.766 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.6080000000000001e-06 2024-09-14T06:29:30.767 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:30.767 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:30.767 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:29:30.767 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062994+0000", 2024-09-14T06:29:30.767 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.5520000000000004e-06 2024-09-14T06:29:30.767 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:30.767 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:30.767 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:29:30.767 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062999+0000", 2024-09-14T06:29:30.767 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.3930000000000001e-06 2024-09-14T06:29:30.767 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:30.767 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:30.767 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:29:30.767 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.110142+0000", 2024-09-14T06:29:30.767 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047143154 2024-09-14T06:29:30.768 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:30.768 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:30.768 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:29:30.768 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.110219+0000", 2024-09-14T06:29:30.768 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.6737999999999996e-05 2024-09-14T06:29:30.768 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:30.768 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:30.768 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:29:30.768 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.110284+0000", 2024-09-14T06:29:30.768 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.5412000000000003e-05 2024-09-14T06:29:30.768 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:30.768 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:29:30.768 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:30.768 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:30.768 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:30.769 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:19360 3.3 3:c34e5e51:::rbd_data.112d92027590.0000000000000024:f [read 1251328~1024 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:29:30.769 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:27.063017+0000", 2024-09-14T06:29:30.769 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 3.6532418020000001, 2024-09-14T06:29:30.769 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047453823999999999, 2024-09-14T06:29:30.769 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:29:30.769 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:29:30.769 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:29:30.769 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:29:30.769 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:29:30.769 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:30.769 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 19360 2024-09-14T06:29:30.769 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:30.769 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:29:30.769 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:30.770 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:29:30.770 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.063017+0000", 2024-09-14T06:29:30.770 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:30.770 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:30.770 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:30.770 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:29:30.770 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.063017+0000", 2024-09-14T06:29:30.770 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:30.770 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:30.770 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:30.770 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:29:30.770 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.064977+0000", 2024-09-14T06:29:30.770 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.0019597080000000001 2024-09-14T06:29:30.771 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:30.771 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:30.771 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:29:30.771 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.064987+0000", 2024-09-14T06:29:30.771 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.9620000000000005e-06 2024-09-14T06:29:30.771 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:30.771 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:30.771 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:29:30.771 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065000+0000", 2024-09-14T06:29:30.771 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.3173000000000001e-05 2024-09-14T06:29:30.771 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:30.771 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:30.771 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:29:30.771 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065005+0000", 2024-09-14T06:29:30.771 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.9960000000000001e-06 2024-09-14T06:29:30.771 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:30.772 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:30.772 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:29:30.772 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.110314+0000", 2024-09-14T06:29:30.772 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.04530899 2024-09-14T06:29:30.772 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:30.772 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:30.772 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:29:30.772 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.110398+0000", 2024-09-14T06:29:30.772 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.3539000000000001e-05 2024-09-14T06:29:30.772 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:30.772 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:30.772 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:29:30.772 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.110471+0000", 2024-09-14T06:29:30.773 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.3455999999999996e-05 2024-09-14T06:29:30.773 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:30.773 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:29:30.773 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:30.773 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:30.773 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:30.773 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:19365 3.3 3:c34e5e51:::rbd_data.112d92027590.0000000000000024:f [read 1302528~1024 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:29:30.773 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:27.065058+0000", 2024-09-14T06:29:30.773 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 3.6512011319999997, 2024-09-14T06:29:30.773 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046105526000000001, 2024-09-14T06:29:30.773 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:29:30.773 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:29:30.774 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:29:30.774 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:29:30.774 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:29:30.774 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:30.774 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 19365 2024-09-14T06:29:30.774 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:30.774 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:29:30.774 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:30.774 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:29:30.774 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065058+0000", 2024-09-14T06:29:30.774 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:30.774 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:30.774 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:30.774 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:29:30.774 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065058+0000", 2024-09-14T06:29:30.775 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:30.775 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:30.775 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:30.775 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:29:30.775 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065058+0000", 2024-09-14T06:29:30.775 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.1899999999999999e-07 2024-09-14T06:29:30.775 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:30.775 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:30.775 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:29:30.775 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065059+0000", 2024-09-14T06:29:30.775 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.1299999999999998e-07 2024-09-14T06:29:30.775 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:30.775 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:30.775 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:29:30.775 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065063+0000", 2024-09-14T06:29:30.775 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.1099999999999996e-06 2024-09-14T06:29:30.776 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:30.776 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:30.776 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:29:30.776 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065070+0000", 2024-09-14T06:29:30.776 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.973e-06 2024-09-14T06:29:30.776 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:30.776 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:30.776 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:29:30.776 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.111014+0000", 2024-09-14T06:29:30.776 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.045943373000000003 2024-09-14T06:29:30.776 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:30.776 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:30.776 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:29:30.776 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.111073+0000", 2024-09-14T06:29:30.777 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.8987999999999998e-05 2024-09-14T06:29:30.777 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:30.777 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:30.777 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:29:30.777 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.111163+0000", 2024-09-14T06:29:30.777 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.0950000000000002e-05 2024-09-14T06:29:30.777 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:30.777 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:29:30.777 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:30.777 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:30.777 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:30.777 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:19366 3.3 3:c34e5e51:::rbd_data.112d92027590.0000000000000024:f [read 1312768~1024 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:29:30.777 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:27.065074+0000", 2024-09-14T06:29:30.777 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 3.6511852820000001, 2024-09-14T06:29:30.777 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046289858000000003, 2024-09-14T06:29:30.777 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:29:30.778 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:29:30.778 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:29:30.778 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:29:30.778 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:29:30.778 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:30.778 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 19366 2024-09-14T06:29:30.778 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:30.778 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:29:30.778 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:30.778 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:29:30.778 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065074+0000", 2024-09-14T06:29:30.778 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:30.778 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:30.779 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:30.779 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:29:30.779 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065074+0000", 2024-09-14T06:29:30.779 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:30.779 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:30.779 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:30.779 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:29:30.779 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065074+0000", 2024-09-14T06:29:30.779 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.3900000000000001e-07 2024-09-14T06:29:30.779 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:30.779 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:30.779 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:29:30.780 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065078+0000", 2024-09-14T06:29:30.780 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.3229999999999999e-06 2024-09-14T06:29:30.780 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:30.780 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:30.780 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:29:30.780 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065082+0000", 2024-09-14T06:29:30.780 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.8299999999999998e-06 2024-09-14T06:29:30.780 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:30.780 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:30.780 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:29:30.780 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065085+0000", 2024-09-14T06:29:30.781 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.8660000000000002e-06 2024-09-14T06:29:30.781 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:30.781 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:30.781 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:29:30.781 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.111270+0000", 2024-09-14T06:29:30.781 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046185035999999999 2024-09-14T06:29:30.781 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:30.781 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:30.781 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:29:30.781 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.111311+0000", 2024-09-14T06:29:30.781 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.0935e-05 2024-09-14T06:29:30.781 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:30.781 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:30.781 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:29:30.781 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.111364+0000", 2024-09-14T06:29:30.782 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.2629e-05 2024-09-14T06:29:30.782 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:30.782 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:29:30.782 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:30.782 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:30.782 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:30.782 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:19367 3.3 3:c34e5e51:::rbd_data.112d92027590.0000000000000024:f [read 1323008~1024 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:29:30.782 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:27.065115+0000", 2024-09-14T06:29:30.782 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 3.651143721, 2024-09-14T06:29:30.782 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046407003000000002, 2024-09-14T06:29:30.782 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:29:30.782 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:29:30.782 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:29:30.782 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:29:30.783 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:29:30.783 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:30.783 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 19367 2024-09-14T06:29:30.783 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:30.783 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:29:30.783 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:30.783 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:29:30.783 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065115+0000", 2024-09-14T06:29:30.783 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:30.783 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:30.783 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:30.783 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:29:30.783 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065115+0000", 2024-09-14T06:29:30.783 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:30.783 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:30.783 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:30.784 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:29:30.784 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065116+0000", 2024-09-14T06:29:30.784 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.4999999999999998e-07 2024-09-14T06:29:30.784 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:30.784 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:30.784 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:29:30.784 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065117+0000", 2024-09-14T06:29:30.784 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.189e-06 2024-09-14T06:29:30.784 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:30.784 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:30.784 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:29:30.784 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065122+0000", 2024-09-14T06:29:30.784 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.3709999999999999e-06 2024-09-14T06:29:30.784 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:30.784 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:30.785 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:29:30.785 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065125+0000", 2024-09-14T06:29:30.785 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.8940000000000002e-06 2024-09-14T06:29:30.785 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:30.785 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:30.785 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:29:30.785 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.111402+0000", 2024-09-14T06:29:30.785 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046276645999999998 2024-09-14T06:29:30.785 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:30.785 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:30.785 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:29:30.785 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.111445+0000", 2024-09-14T06:29:30.785 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.3288000000000003e-05 2024-09-14T06:29:30.785 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:30.785 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:30.786 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:29:30.786 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.111522+0000", 2024-09-14T06:29:30.786 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.7164999999999997e-05 2024-09-14T06:29:30.786 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:30.786 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:29:30.786 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:30.786 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:30.786 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:30.786 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:19368 3.3 3:c34e5e51:::rbd_data.112d92027590.0000000000000024:f [read 1333248~1024 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:29:30.786 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:27.065130+0000", 2024-09-14T06:29:30.786 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 3.6511294310000002, 2024-09-14T06:29:30.786 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046507275000000001, 2024-09-14T06:29:30.786 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:29:30.786 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:29:30.786 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:29:30.787 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:29:30.787 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:29:30.787 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:30.787 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 19368 2024-09-14T06:29:30.787 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:30.787 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:29:30.787 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:30.787 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:29:30.787 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065130+0000", 2024-09-14T06:29:30.787 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:30.787 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:30.787 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:30.787 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:29:30.787 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065130+0000", 2024-09-14T06:29:30.787 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:30.788 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:30.788 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:30.788 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:29:30.788 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065130+0000", 2024-09-14T06:29:30.788 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.0800000000000001e-07 2024-09-14T06:29:30.788 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:30.788 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:30.788 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:29:30.788 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065131+0000", 2024-09-14T06:29:30.788 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.8700000000000004e-07 2024-09-14T06:29:30.788 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:30.788 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:30.788 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:29:30.788 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065134+0000", 2024-09-14T06:29:30.789 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.1039999999999998e-06 2024-09-14T06:29:30.789 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:30.789 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:30.789 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:29:30.789 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065136+0000", 2024-09-14T06:29:30.789 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.3949999999999999e-06 2024-09-14T06:29:30.789 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:30.789 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:30.789 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:29:30.789 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.111541+0000", 2024-09-14T06:29:30.789 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046404206000000003 2024-09-14T06:29:30.789 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:30.789 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:30.790 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:29:30.790 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.111582+0000", 2024-09-14T06:29:30.790 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.1051999999999999e-05 2024-09-14T06:29:30.790 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:30.790 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:30.790 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:29:30.790 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.111637+0000", 2024-09-14T06:29:30.790 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.5322999999999999e-05 2024-09-14T06:29:30.790 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:30.790 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:29:30.790 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:30.790 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:30.790 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:30.791 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:19369 3.3 3:c34e5e51:::rbd_data.112d92027590.0000000000000024:f [read 1343488~1024 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:29:30.791 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:27.065139+0000", 2024-09-14T06:29:30.791 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 3.6511196139999997, 2024-09-14T06:29:30.791 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046682169000000003, 2024-09-14T06:29:30.791 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:29:30.791 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:29:30.791 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:29:30.791 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:29:30.791 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:29:30.791 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:30.791 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 19369 2024-09-14T06:29:30.791 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:30.791 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:29:30.791 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:30.791 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:29:30.792 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065139+0000", 2024-09-14T06:29:30.792 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:30.792 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:30.792 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:30.792 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:29:30.792 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065139+0000", 2024-09-14T06:29:30.792 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:30.792 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:30.792 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:30.792 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:29:30.792 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065140+0000", 2024-09-14T06:29:30.792 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.8299999999999998e-07 2024-09-14T06:29:30.792 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:30.793 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:30.793 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:29:30.793 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065141+0000", 2024-09-14T06:29:30.793 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.3900000000000003e-07 2024-09-14T06:29:30.793 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:30.793 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:30.793 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:29:30.793 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065144+0000", 2024-09-14T06:29:30.793 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.7409999999999998e-06 2024-09-14T06:29:30.793 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:30.793 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:30.793 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:29:30.793 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065147+0000", 2024-09-14T06:29:30.793 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.4150000000000002e-06 2024-09-14T06:29:30.793 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:30.794 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:30.794 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:29:30.794 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.111673+0000", 2024-09-14T06:29:30.794 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046526004000000003 2024-09-14T06:29:30.794 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:30.794 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:30.794 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:29:30.794 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.111749+0000", 2024-09-14T06:29:30.794 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.5741999999999994e-05 2024-09-14T06:29:30.794 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:30.794 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:30.794 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:29:30.794 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.111822+0000", 2024-09-14T06:29:30.794 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.3045000000000003e-05 2024-09-14T06:29:30.795 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:30.795 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:29:30.795 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:30.795 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:30.795 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:30.795 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:19370 3.3 3:c34e5e51:::rbd_data.112d92027590.0000000000000024:f [read 1353728~1024 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:29:30.795 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:27.065150+0000", 2024-09-14T06:29:30.795 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 3.6511088850000002, 2024-09-14T06:29:30.795 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046897556999999999, 2024-09-14T06:29:30.795 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:29:30.795 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:29:30.795 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:29:30.795 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:29:30.795 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:29:30.796 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:30.796 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 19370 2024-09-14T06:29:30.796 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:30.796 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:29:30.796 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:30.796 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:29:30.796 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065150+0000", 2024-09-14T06:29:30.796 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:30.796 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:30.796 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:30.796 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:29:30.796 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065150+0000", 2024-09-14T06:29:30.796 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:30.796 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:30.796 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:30.797 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:29:30.797 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065150+0000", 2024-09-14T06:29:30.797 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.0200000000000001e-07 2024-09-14T06:29:30.797 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:30.797 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:30.797 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:29:30.797 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065151+0000", 2024-09-14T06:29:30.797 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.1100000000000004e-07 2024-09-14T06:29:30.797 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:30.797 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:30.798 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:29:30.798 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065154+0000", 2024-09-14T06:29:30.798 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.9639999999999999e-06 2024-09-14T06:29:30.798 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:30.798 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:30.798 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:29:30.798 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065157+0000", 2024-09-14T06:29:30.798 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.4260000000000002e-06 2024-09-14T06:29:30.798 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:30.798 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:30.798 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:29:30.798 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.111861+0000", 2024-09-14T06:29:30.798 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046704268 2024-09-14T06:29:30.798 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:30.798 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:30.798 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:29:30.799 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.111939+0000", 2024-09-14T06:29:30.799 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.8231000000000004e-05 2024-09-14T06:29:30.799 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:30.799 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:30.799 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:29:30.799 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.112048+0000", 2024-09-14T06:29:30.799 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00010855500000000001 2024-09-14T06:29:30.799 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:30.799 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:29:30.799 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:30.799 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:30.799 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:30.799 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:19371 3.3 3:c34e5e51:::rbd_data.112d92027590.0000000000000024:f [read 1363968~1024 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:29:30.799 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:27.065161+0000", 2024-09-14T06:29:30.799 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 3.6510982350000001, 2024-09-14T06:29:30.799 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047021944000000003, 2024-09-14T06:29:30.800 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:29:30.800 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:29:30.800 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:29:30.800 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:29:30.800 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:29:30.800 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:30.800 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 19371 2024-09-14T06:29:30.800 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:30.800 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:29:30.800 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:30.801 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:29:30.801 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065161+0000", 2024-09-14T06:29:30.801 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:30.801 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:30.801 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:30.801 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:29:30.801 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065161+0000", 2024-09-14T06:29:30.801 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:30.801 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:30.801 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:30.801 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:29:30.801 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065161+0000", 2024-09-14T06:29:30.802 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.0200000000000001e-07 2024-09-14T06:29:30.802 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:30.802 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:30.802 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:29:30.802 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065166+0000", 2024-09-14T06:29:30.802 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.9869999999999999e-06 2024-09-14T06:29:30.802 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:30.802 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:30.802 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:29:30.802 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065169+0000", 2024-09-14T06:29:30.802 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.3799999999999998e-06 2024-09-14T06:29:30.802 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:30.802 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:30.803 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:29:30.803 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065172+0000", 2024-09-14T06:29:30.803 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.4830000000000002e-06 2024-09-14T06:29:30.803 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:30.803 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:30.803 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:29:30.803 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.112071+0000", 2024-09-14T06:29:30.803 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046899529000000002 2024-09-14T06:29:30.803 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:30.803 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:30.803 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:29:30.803 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.112128+0000", 2024-09-14T06:29:30.803 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.6341999999999997e-05 2024-09-14T06:29:30.804 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:30.804 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:30.804 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:29:30.804 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.112183+0000", 2024-09-14T06:29:30.804 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.5021000000000001e-05 2024-09-14T06:29:30.804 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:30.804 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:29:30.804 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:30.804 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:30.804 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:30.804 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:19372 3.3 3:c34e5e51:::rbd_data.112d92027590.0000000000000024:f [read 1374208~1024 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:29:30.804 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:27.065176+0000", 2024-09-14T06:29:30.804 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 3.6510830849999998, 2024-09-14T06:29:30.804 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047188889999999997, 2024-09-14T06:29:30.805 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:29:30.805 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:29:30.805 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:29:30.805 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:29:30.805 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:29:30.805 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:30.805 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 19372 2024-09-14T06:29:30.805 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:30.805 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:29:30.805 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:30.805 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:29:30.805 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065176+0000", 2024-09-14T06:29:30.805 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:30.806 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:30.806 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:30.806 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:29:30.806 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065176+0000", 2024-09-14T06:29:30.806 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:30.806 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:30.806 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:30.806 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:29:30.806 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.071117+0000", 2024-09-14T06:29:30.806 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.0059404990000000001 2024-09-14T06:29:30.806 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:30.806 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:30.806 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:29:30.806 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.071120+0000", 2024-09-14T06:29:30.806 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.4340000000000001e-06 2024-09-14T06:29:30.807 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:30.807 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:30.807 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:29:30.807 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.071129+0000", 2024-09-14T06:29:30.807 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.8389999999999996e-06 2024-09-14T06:29:30.807 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:30.807 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:30.807 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:29:30.807 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.071134+0000", 2024-09-14T06:29:30.807 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.412e-06 2024-09-14T06:29:30.807 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:30.807 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:30.807 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:29:30.807 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.112221+0000", 2024-09-14T06:29:30.807 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.041086473999999998 2024-09-14T06:29:30.808 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:30.808 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:30.808 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:29:30.808 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.112299+0000", 2024-09-14T06:29:30.808 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.8746999999999997e-05 2024-09-14T06:29:30.808 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:30.808 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:30.808 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:29:30.808 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.112365+0000", 2024-09-14T06:29:30.808 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.5485000000000001e-05 2024-09-14T06:29:30.808 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:30.808 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:29:30.808 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:30.808 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:30.809 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:29:30.809 INFO:teuthology.orchestra.run.smithi138.stdout:} 2024-09-14T06:29:30.810 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-14T06:29:30.810 INFO:tasks.ceph.osd.4.smithi138.stderr:2024-09-14T06:29:30.809+0000 7f6c653c8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33615) UID: 0 2024-09-14T06:29:30.911 INFO:tasks.ceph.osd.7.smithi138.stderr:2024-09-14T06:29:30.910+0000 7f3c6a21d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33623) UID: 0 2024-09-14T06:29:30.953 INFO:teuthology.orchestra.run.smithi138.stdout:{ 2024-09-14T06:29:30.953 INFO:teuthology.orchestra.run.smithi138.stdout: "ops": [], 2024-09-14T06:29:30.953 INFO:teuthology.orchestra.run.smithi138.stdout: "num_ops": 0 2024-09-14T06:29:30.953 INFO:teuthology.orchestra.run.smithi138.stdout:} 2024-09-14T06:29:30.961 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-14T06:29:31.045 INFO:tasks.ceph.osd.6.smithi138.stderr:2024-09-14T06:29:31.044+0000 7f8cd8849640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33620) UID: 0 2024-09-14T06:29:31.105 INFO:teuthology.orchestra.run.smithi138.stdout:{ 2024-09-14T06:29:31.105 INFO:teuthology.orchestra.run.smithi138.stdout: "ops": [], 2024-09-14T06:29:31.105 INFO:teuthology.orchestra.run.smithi138.stdout: "complaint_time": 30, 2024-09-14T06:29:31.105 INFO:teuthology.orchestra.run.smithi138.stdout: "num_blocked_ops": 0 2024-09-14T06:29:31.105 INFO:teuthology.orchestra.run.smithi138.stdout:} 2024-09-14T06:29:31.114 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-14T06:29:31.114 INFO:tasks.ceph.osd.7.smithi138.stderr:2024-09-14T06:29:31.112+0000 7f3c6a21d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33623) UID: 0 2024-09-14T06:29:31.214 INFO:tasks.ceph.osd.5.smithi138.stderr:2024-09-14T06:29:31.213+0000 7fe13746f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33618) UID: 0 2024-09-14T06:29:31.260 INFO:teuthology.orchestra.run.smithi138.stdout:{ 2024-09-14T06:29:31.260 INFO:teuthology.orchestra.run.smithi138.stdout: "size": 20, 2024-09-14T06:29:31.260 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 600, 2024-09-14T06:29:31.260 INFO:teuthology.orchestra.run.smithi138.stdout: "ops": [ 2024-09-14T06:29:31.260 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:31.260 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:11453 3.1d 3:bcad7303:::rbd_data.112d92027590.0000000000000001:f [read 94208~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:29:31.260 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:25.654375+0000", 2024-09-14T06:29:31.261 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 5.6039552490000002, 2024-09-14T06:29:31.261 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052449318000000002, 2024-09-14T06:29:31.261 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:29:31.261 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:29:31.261 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:29:31.261 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:29:31.261 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:29:31.261 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:31.261 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 11453 2024-09-14T06:29:31.261 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:31.261 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:29:31.261 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:31.261 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:29:31.261 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654375+0000", 2024-09-14T06:29:31.262 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:31.262 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:31.262 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:31.262 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:29:31.262 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654375+0000", 2024-09-14T06:29:31.262 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:31.262 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:31.262 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:31.262 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:29:31.262 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654496+0000", 2024-09-14T06:29:31.262 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00012081 2024-09-14T06:29:31.262 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:31.262 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:31.262 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:29:31.262 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654497+0000", 2024-09-14T06:29:31.263 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.8789999999999999e-06 2024-09-14T06:29:31.263 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:31.263 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:31.263 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:29:31.263 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654501+0000", 2024-09-14T06:29:31.263 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.8999999999999999e-06 2024-09-14T06:29:31.263 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:31.263 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:31.263 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:29:31.263 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654505+0000", 2024-09-14T06:29:31.263 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.2229999999999999e-06 2024-09-14T06:29:31.263 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:31.263 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:31.263 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:29:31.263 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.705917+0000", 2024-09-14T06:29:31.263 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.051412501999999999 2024-09-14T06:29:31.264 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:31.264 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:31.264 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:29:31.264 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.705999+0000", 2024-09-14T06:29:31.264 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.1551999999999994e-05 2024-09-14T06:29:31.264 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:31.264 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:31.264 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:29:31.264 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.706824+0000", 2024-09-14T06:29:31.264 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00082545199999999996 2024-09-14T06:29:31.264 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:31.264 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:29:31.264 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:31.264 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:31.264 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:31.265 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:11454 3.1d 3:bcad7303:::rbd_data.112d92027590.0000000000000001:f [read 104448~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:29:31.265 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:25.654510+0000", 2024-09-14T06:29:31.265 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 5.6038204589999996, 2024-09-14T06:29:31.265 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052610713000000003, 2024-09-14T06:29:31.265 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:29:31.265 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:29:31.265 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:29:31.265 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:29:31.265 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:29:31.265 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:31.265 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 11454 2024-09-14T06:29:31.265 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:31.265 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:29:31.265 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:31.266 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:29:31.266 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654510+0000", 2024-09-14T06:29:31.266 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:31.266 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:31.266 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:31.266 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:29:31.266 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654510+0000", 2024-09-14T06:29:31.266 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:31.266 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:31.266 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:31.266 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:29:31.266 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654825+0000", 2024-09-14T06:29:31.266 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00031522400000000002 2024-09-14T06:29:31.266 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:31.266 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:31.266 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:29:31.267 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654827+0000", 2024-09-14T06:29:31.267 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.9709999999999998e-06 2024-09-14T06:29:31.267 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:31.267 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:31.267 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:29:31.267 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654832+0000", 2024-09-14T06:29:31.267 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.4099999999999999e-06 2024-09-14T06:29:31.267 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:31.267 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:31.267 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:29:31.268 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654835+0000", 2024-09-14T06:29:31.268 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.14e-06 2024-09-14T06:29:31.268 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:31.268 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:31.268 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:29:31.268 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.706851+0000", 2024-09-14T06:29:31.268 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052015761000000001 2024-09-14T06:29:31.268 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:31.268 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:31.268 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:29:31.268 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.706934+0000", 2024-09-14T06:29:31.268 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.2484999999999994e-05 2024-09-14T06:29:31.268 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:31.268 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:31.269 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:29:31.269 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.707120+0000", 2024-09-14T06:29:31.269 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00018672200000000001 2024-09-14T06:29:31.269 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:31.269 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:29:31.269 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:31.269 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:31.269 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:31.269 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:11455 3.1d 3:bcad7303:::rbd_data.112d92027590.0000000000000001:f [read 114688~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:29:31.269 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:25.654841+0000", 2024-09-14T06:29:31.269 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 5.6034891299999998, 2024-09-14T06:29:31.269 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052579898, 2024-09-14T06:29:31.269 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:29:31.269 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:29:31.270 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:29:31.270 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:29:31.270 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:29:31.270 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:31.270 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 11455 2024-09-14T06:29:31.270 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:31.270 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:29:31.270 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:31.270 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:29:31.270 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654841+0000", 2024-09-14T06:29:31.270 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:31.270 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:31.270 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:31.270 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:29:31.270 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654841+0000", 2024-09-14T06:29:31.270 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:31.271 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:31.271 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:31.271 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:29:31.271 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654841+0000", 2024-09-14T06:29:31.271 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.7300000000000002e-07 2024-09-14T06:29:31.271 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:31.271 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:31.271 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:29:31.272 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654845+0000", 2024-09-14T06:29:31.272 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.0520000000000003e-06 2024-09-14T06:29:31.272 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:31.272 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:31.272 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:29:31.272 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654849+0000", 2024-09-14T06:29:31.272 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.0459999999999999e-06 2024-09-14T06:29:31.272 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:31.272 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:31.272 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:29:31.272 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654852+0000", 2024-09-14T06:29:31.272 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.9830000000000001e-06 2024-09-14T06:29:31.272 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:31.273 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:31.273 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:29:31.273 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.707159+0000", 2024-09-14T06:29:31.273 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052306826000000001 2024-09-14T06:29:31.273 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:31.273 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:31.273 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:29:31.273 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.707242+0000", 2024-09-14T06:29:31.273 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.2595000000000005e-05 2024-09-14T06:29:31.273 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:31.273 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:31.273 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:29:31.273 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.707421+0000", 2024-09-14T06:29:31.273 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.000179123 2024-09-14T06:29:31.273 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:31.274 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:29:31.274 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:31.274 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:31.274 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:31.274 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:11456 3.1d 3:bcad7303:::rbd_data.112d92027590.0000000000000001:f [read 124928~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:29:31.274 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:25.654856+0000", 2024-09-14T06:29:31.274 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 5.6034742839999998, 2024-09-14T06:29:31.274 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052801580000000001, 2024-09-14T06:29:31.274 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:29:31.274 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:29:31.274 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:29:31.274 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:29:31.274 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:29:31.274 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:31.275 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 11456 2024-09-14T06:29:31.275 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:31.275 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:29:31.275 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:31.275 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:29:31.275 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654856+0000", 2024-09-14T06:29:31.275 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:31.275 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:31.275 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:31.275 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:29:31.275 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654856+0000", 2024-09-14T06:29:31.275 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:31.275 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:31.275 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:31.275 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:29:31.276 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654856+0000", 2024-09-14T06:29:31.276 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.29e-07 2024-09-14T06:29:31.276 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:31.276 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:31.276 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:29:31.276 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654857+0000", 2024-09-14T06:29:31.276 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.187e-06 2024-09-14T06:29:31.276 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:31.276 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:31.276 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:29:31.276 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654861+0000", 2024-09-14T06:29:31.276 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.3850000000000002e-06 2024-09-14T06:29:31.276 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:31.276 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:31.277 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:29:31.277 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654863+0000", 2024-09-14T06:29:31.277 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.9500000000000001e-06 2024-09-14T06:29:31.277 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:31.277 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:31.277 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:29:31.277 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.707456+0000", 2024-09-14T06:29:31.277 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052592434 2024-09-14T06:29:31.277 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:31.277 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:31.277 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:29:31.277 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.707516+0000", 2024-09-14T06:29:31.277 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.0167999999999997e-05 2024-09-14T06:29:31.277 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:31.277 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:31.278 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:29:31.278 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.707657+0000", 2024-09-14T06:29:31.278 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00014122700000000001 2024-09-14T06:29:31.278 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:31.278 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:29:31.278 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:31.278 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:31.278 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:31.278 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:11457 3.1d 3:bcad7303:::rbd_data.112d92027590.0000000000000001:f [read 135168~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:29:31.278 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:25.654867+0000", 2024-09-14T06:29:31.278 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 5.6034633989999998, 2024-09-14T06:29:31.278 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052921902, 2024-09-14T06:29:31.278 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:29:31.278 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:29:31.278 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:29:31.278 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:29:31.279 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:29:31.279 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:31.279 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 11457 2024-09-14T06:29:31.279 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:31.279 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:29:31.279 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:31.279 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:29:31.279 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654867+0000", 2024-09-14T06:29:31.279 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:31.279 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:31.279 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:31.279 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:29:31.279 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654867+0000", 2024-09-14T06:29:31.279 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:31.279 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:31.280 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:31.280 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:29:31.280 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654972+0000", 2024-09-14T06:29:31.280 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00010584 2024-09-14T06:29:31.280 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:31.280 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:31.280 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:29:31.280 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654975+0000", 2024-09-14T06:29:31.280 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.1169999999999998e-06 2024-09-14T06:29:31.280 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:31.280 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:31.280 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:29:31.280 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654979+0000", 2024-09-14T06:29:31.280 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.1960000000000001e-06 2024-09-14T06:29:31.280 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:31.281 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:31.281 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:29:31.281 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654982+0000", 2024-09-14T06:29:31.281 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.2990000000000001e-06 2024-09-14T06:29:31.281 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:31.281 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:31.281 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:29:31.281 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.707684+0000", 2024-09-14T06:29:31.281 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052701979000000003 2024-09-14T06:29:31.281 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:31.281 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:31.282 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:29:31.282 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.707728+0000", 2024-09-14T06:29:31.282 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.3529999999999998e-05 2024-09-14T06:29:31.282 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:31.282 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:31.282 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:29:31.282 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.707789+0000", 2024-09-14T06:29:31.282 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.0940999999999998e-05 2024-09-14T06:29:31.282 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:31.282 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:29:31.282 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:31.282 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:31.282 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:31.282 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:11458 3.1d 3:bcad7303:::rbd_data.112d92027590.0000000000000001:f [read 145408~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:29:31.282 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:25.654987+0000", 2024-09-14T06:29:31.283 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 5.6033430009999998, 2024-09-14T06:29:31.283 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.05291655, 2024-09-14T06:29:31.283 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:29:31.283 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:29:31.283 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:29:31.283 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:29:31.283 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:29:31.283 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:31.283 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 11458 2024-09-14T06:29:31.283 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:31.283 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:29:31.283 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:31.283 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:29:31.283 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654987+0000", 2024-09-14T06:29:31.283 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:31.284 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:31.284 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:31.284 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:29:31.284 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654987+0000", 2024-09-14T06:29:31.284 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:31.284 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:31.284 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:31.284 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:29:31.284 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655077+0000", 2024-09-14T06:29:31.284 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.0253000000000005e-05 2024-09-14T06:29:31.284 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:31.284 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:31.284 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:29:31.284 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655079+0000", 2024-09-14T06:29:31.284 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.9989999999999998e-06 2024-09-14T06:29:31.285 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:31.285 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:31.285 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:29:31.285 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655083+0000", 2024-09-14T06:29:31.285 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.9280000000000003e-06 2024-09-14T06:29:31.285 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:31.285 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:31.285 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:29:31.285 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655086+0000", 2024-09-14T06:29:31.285 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.056e-06 2024-09-14T06:29:31.285 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:31.285 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:31.286 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:29:31.286 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.707805+0000", 2024-09-14T06:29:31.286 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052718627999999997 2024-09-14T06:29:31.286 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:31.286 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:31.286 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:29:31.286 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.707848+0000", 2024-09-14T06:29:31.286 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.2728999999999997e-05 2024-09-14T06:29:31.286 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:31.286 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:31.286 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:29:31.286 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.707904+0000", 2024-09-14T06:29:31.286 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.5957e-05 2024-09-14T06:29:31.286 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:31.286 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:29:31.286 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:31.287 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:31.287 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:31.287 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:11459 3.1d 3:bcad7303:::rbd_data.112d92027590.0000000000000001:f [read 155648~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:29:31.287 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:25.655091+0000", 2024-09-14T06:29:31.287 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 5.6032389929999997, 2024-09-14T06:29:31.287 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052944242000000002, 2024-09-14T06:29:31.287 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:29:31.287 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:29:31.287 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:29:31.287 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:29:31.287 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:29:31.287 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:31.287 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 11459 2024-09-14T06:29:31.287 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:31.287 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:29:31.288 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:31.288 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:29:31.288 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655091+0000", 2024-09-14T06:29:31.288 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:31.288 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:31.288 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:31.288 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:29:31.288 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655091+0000", 2024-09-14T06:29:31.288 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:31.288 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:31.288 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:31.288 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:29:31.288 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655237+0000", 2024-09-14T06:29:31.288 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00014570100000000001 2024-09-14T06:29:31.289 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:31.289 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:31.289 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:29:31.289 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655239+0000", 2024-09-14T06:29:31.289 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.8670000000000001e-06 2024-09-14T06:29:31.289 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:31.289 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:31.289 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:29:31.289 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655243+0000", 2024-09-14T06:29:31.289 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.9829999999999998e-06 2024-09-14T06:29:31.289 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:31.289 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:31.289 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:29:31.289 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655246+0000", 2024-09-14T06:29:31.289 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.1540000000000002e-06 2024-09-14T06:29:31.290 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:31.290 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:31.290 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:29:31.290 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.707934+0000", 2024-09-14T06:29:31.290 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052688286000000001 2024-09-14T06:29:31.290 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:31.290 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:31.290 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:29:31.290 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.707977+0000", 2024-09-14T06:29:31.290 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.3291999999999998e-05 2024-09-14T06:29:31.290 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:31.290 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:31.290 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:29:31.291 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.708035+0000", 2024-09-14T06:29:31.291 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.7958999999999999e-05 2024-09-14T06:29:31.291 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:31.291 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:29:31.291 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:31.291 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:31.291 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:31.291 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:11460 3.1d 3:bcad7303:::rbd_data.112d92027590.0000000000000001:f [read 165888~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:29:31.291 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:25.655250+0000", 2024-09-14T06:29:31.291 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 5.603079696, 2024-09-14T06:29:31.291 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052910698999999999, 2024-09-14T06:29:31.291 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:29:31.291 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:29:31.291 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:29:31.291 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:29:31.292 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:29:31.292 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:31.292 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 11460 2024-09-14T06:29:31.292 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:31.292 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:29:31.292 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:31.292 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:29:31.292 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655250+0000", 2024-09-14T06:29:31.292 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:31.292 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:31.292 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:31.292 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:29:31.292 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655250+0000", 2024-09-14T06:29:31.292 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:31.293 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:31.293 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:31.293 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:29:31.293 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655356+0000", 2024-09-14T06:29:31.293 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00010535900000000001 2024-09-14T06:29:31.293 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:31.293 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:31.293 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:29:31.293 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655357+0000", 2024-09-14T06:29:31.293 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.8080000000000001e-06 2024-09-14T06:29:31.293 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:31.293 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:31.293 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:29:31.294 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655361+0000", 2024-09-14T06:29:31.294 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.861e-06 2024-09-14T06:29:31.294 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:31.294 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:31.294 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:29:31.294 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655364+0000", 2024-09-14T06:29:31.294 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.1089999999999998e-06 2024-09-14T06:29:31.294 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:31.294 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:31.294 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:29:31.294 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.708065+0000", 2024-09-14T06:29:31.294 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052700595000000003 2024-09-14T06:29:31.294 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:31.294 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:31.294 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:29:31.295 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.708108+0000", 2024-09-14T06:29:31.295 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.2484999999999997e-05 2024-09-14T06:29:31.295 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:31.295 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:31.295 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:29:31.295 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.708161+0000", 2024-09-14T06:29:31.295 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.3482000000000001e-05 2024-09-14T06:29:31.295 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:31.295 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:29:31.295 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:31.295 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:31.295 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:31.295 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:11461 3.1d 3:bcad7303:::rbd_data.112d92027590.0000000000000001:f [read 176128~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:29:31.295 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:25.655370+0000", 2024-09-14T06:29:31.295 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 5.602960414, 2024-09-14T06:29:31.296 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052915141999999998, 2024-09-14T06:29:31.296 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:29:31.296 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:29:31.296 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:29:31.296 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:29:31.296 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:29:31.296 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:31.296 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 11461 2024-09-14T06:29:31.296 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:31.296 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:29:31.296 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:31.296 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:29:31.296 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655370+0000", 2024-09-14T06:29:31.296 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:31.297 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:31.297 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:31.297 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:29:31.297 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655370+0000", 2024-09-14T06:29:31.297 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:31.297 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:31.297 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:31.297 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:29:31.297 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655394+0000", 2024-09-14T06:29:31.297 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.4553999999999999e-05 2024-09-14T06:29:31.297 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:31.297 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:31.297 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:29:31.297 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655396+0000", 2024-09-14T06:29:31.298 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.8950000000000001e-06 2024-09-14T06:29:31.298 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:31.298 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:31.298 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:29:31.298 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655401+0000", 2024-09-14T06:29:31.298 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.0089999999999998e-06 2024-09-14T06:29:31.298 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:31.298 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:31.298 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:29:31.298 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655404+0000", 2024-09-14T06:29:31.298 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.3400000000000002e-06 2024-09-14T06:29:31.299 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:31.299 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:31.299 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:29:31.299 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.708186+0000", 2024-09-14T06:29:31.299 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052781726000000001 2024-09-14T06:29:31.299 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:31.299 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:31.299 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:29:31.299 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.708229+0000", 2024-09-14T06:29:31.299 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.2670999999999999e-05 2024-09-14T06:29:31.299 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:31.299 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:31.299 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:29:31.299 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.708285+0000", 2024-09-14T06:29:31.300 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.5946999999999998e-05 2024-09-14T06:29:31.300 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:31.300 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:29:31.300 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:31.300 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:31.300 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:31.300 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:11462 3.1d 3:bcad7303:::rbd_data.112d92027590.0000000000000001:f [read 186368~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:29:31.300 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:25.655410+0000", 2024-09-14T06:29:31.300 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 5.6029199360000002, 2024-09-14T06:29:31.300 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052995258000000003, 2024-09-14T06:29:31.300 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:29:31.300 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:29:31.301 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:29:31.301 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:29:31.301 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:29:31.301 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:31.301 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 11462 2024-09-14T06:29:31.301 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:31.301 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:29:31.301 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:31.301 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:29:31.301 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655410+0000", 2024-09-14T06:29:31.301 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:31.301 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:31.302 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:31.302 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:29:31.302 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655410+0000", 2024-09-14T06:29:31.302 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:31.302 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:31.302 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:31.302 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:29:31.302 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655625+0000", 2024-09-14T06:29:31.302 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00021442600000000001 2024-09-14T06:29:31.302 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:31.302 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:31.302 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:29:31.302 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655626+0000", 2024-09-14T06:29:31.303 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.815e-06 2024-09-14T06:29:31.303 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:31.303 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:31.303 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:29:31.303 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655631+0000", 2024-09-14T06:29:31.303 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.651e-06 2024-09-14T06:29:31.303 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:31.303 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:31.303 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:29:31.303 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655634+0000", 2024-09-14T06:29:31.303 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.2449999999999998e-06 2024-09-14T06:29:31.303 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:31.303 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:31.303 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:29:31.304 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.708312+0000", 2024-09-14T06:29:31.304 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052677412 2024-09-14T06:29:31.304 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:31.304 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:31.304 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:29:31.304 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.708352+0000", 2024-09-14T06:29:31.304 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.0729999999999998e-05 2024-09-14T06:29:31.304 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:31.304 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:31.304 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:29:31.304 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.708405+0000", 2024-09-14T06:29:31.304 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.2979000000000002e-05 2024-09-14T06:29:31.304 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:31.304 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:29:31.305 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:31.305 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:31.305 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:31.305 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:11463 3.1d 3:bcad7303:::rbd_data.112d92027590.0000000000000001:f [read 196608~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:29:31.305 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:25.655639+0000", 2024-09-14T06:29:31.305 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 5.6026907900000005, 2024-09-14T06:29:31.305 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052888859000000003, 2024-09-14T06:29:31.305 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:29:31.305 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:29:31.305 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:29:31.305 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:29:31.305 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:29:31.305 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:31.306 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 11463 2024-09-14T06:29:31.306 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:31.306 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:29:31.306 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:31.306 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:29:31.306 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655639+0000", 2024-09-14T06:29:31.306 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:31.306 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:31.306 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:31.306 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:29:31.306 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655639+0000", 2024-09-14T06:29:31.306 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:31.306 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:31.306 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:31.306 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:29:31.307 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655640+0000", 2024-09-14T06:29:31.307 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.08e-07 2024-09-14T06:29:31.307 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:31.307 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:31.307 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:29:31.307 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655641+0000", 2024-09-14T06:29:31.307 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.156e-06 2024-09-14T06:29:31.307 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:31.307 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:31.307 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:29:31.307 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655648+0000", 2024-09-14T06:29:31.307 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.8460000000000002e-06 2024-09-14T06:29:31.307 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:31.307 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:31.307 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:29:31.308 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655651+0000", 2024-09-14T06:29:31.308 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.0079999999999998e-06 2024-09-14T06:29:31.308 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:31.308 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:31.308 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:29:31.308 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.708429+0000", 2024-09-14T06:29:31.308 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052778459 2024-09-14T06:29:31.308 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:31.308 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:31.308 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:29:31.308 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.708473+0000", 2024-09-14T06:29:31.308 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.3782999999999997e-05 2024-09-14T06:29:31.308 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:31.308 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:31.309 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:29:31.309 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.708528+0000", 2024-09-14T06:29:31.309 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.5198999999999998e-05 2024-09-14T06:29:31.309 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:31.309 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:29:31.309 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:31.309 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:31.309 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:31.309 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:11464 3.1d 3:bcad7303:::rbd_data.112d92027590.0000000000000001:f [read 206848~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:29:31.309 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:25.655654+0000", 2024-09-14T06:29:31.309 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 5.6026759210000003, 2024-09-14T06:29:31.309 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052981845999999999, 2024-09-14T06:29:31.309 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:29:31.310 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:29:31.310 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:29:31.310 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:29:31.310 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:29:31.310 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:31.310 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 11464 2024-09-14T06:29:31.310 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:31.310 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:29:31.310 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:31.310 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:29:31.311 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655654+0000", 2024-09-14T06:29:31.311 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:31.311 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:31.311 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:31.311 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:29:31.311 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655654+0000", 2024-09-14T06:29:31.311 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:31.311 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:31.311 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:31.311 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:29:31.311 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655946+0000", 2024-09-14T06:29:31.311 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00029148200000000003 2024-09-14T06:29:31.311 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:31.311 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:31.312 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:29:31.312 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655948+0000", 2024-09-14T06:29:31.312 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.973e-06 2024-09-14T06:29:31.312 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:31.312 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:31.312 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:29:31.312 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655953+0000", 2024-09-14T06:29:31.312 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.1240000000000004e-06 2024-09-14T06:29:31.312 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:31.312 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:31.312 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:29:31.312 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655956+0000", 2024-09-14T06:29:31.312 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.315e-06 2024-09-14T06:29:31.313 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:31.313 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:31.313 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:29:31.313 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.708543+0000", 2024-09-14T06:29:31.313 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052587416999999997 2024-09-14T06:29:31.313 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:31.313 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:31.313 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:29:31.313 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.708584+0000", 2024-09-14T06:29:31.313 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.1004000000000002e-05 2024-09-14T06:29:31.313 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:31.313 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:31.313 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:29:31.313 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.708636+0000", 2024-09-14T06:29:31.313 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.1530999999999998e-05 2024-09-14T06:29:31.314 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:31.314 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:29:31.314 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:31.314 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:31.314 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:31.314 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:11465 3.1d 3:bcad7303:::rbd_data.112d92027590.0000000000000001:f [read 217088~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:29:31.314 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:25.655961+0000", 2024-09-14T06:29:31.314 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 5.6023687689999999, 2024-09-14T06:29:31.314 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052798682999999999, 2024-09-14T06:29:31.314 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:29:31.314 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:29:31.314 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:29:31.314 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:29:31.314 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:29:31.314 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:31.314 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 11465 2024-09-14T06:29:31.315 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:31.315 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:29:31.315 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:31.315 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:29:31.315 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655961+0000", 2024-09-14T06:29:31.315 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:31.315 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:31.315 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:31.315 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:29:31.315 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655961+0000", 2024-09-14T06:29:31.315 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:31.315 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:31.315 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:31.315 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:29:31.316 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.656114+0000", 2024-09-14T06:29:31.316 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.000152745 2024-09-14T06:29:31.316 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:31.316 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:31.316 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:29:31.316 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.656116+0000", 2024-09-14T06:29:31.316 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.804e-06 2024-09-14T06:29:31.316 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:31.316 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:31.316 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:29:31.316 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.656120+0000", 2024-09-14T06:29:31.316 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.4700000000000004e-06 2024-09-14T06:29:31.316 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:31.316 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:31.316 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:29:31.317 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.656123+0000", 2024-09-14T06:29:31.317 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.1269999999999999e-06 2024-09-14T06:29:31.317 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:31.317 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:31.317 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:29:31.317 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.708660+0000", 2024-09-14T06:29:31.317 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052536919000000001 2024-09-14T06:29:31.317 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:31.317 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:31.317 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:29:31.317 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.708703+0000", 2024-09-14T06:29:31.317 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.3077000000000001e-05 2024-09-14T06:29:31.317 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:31.317 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:31.318 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:29:31.318 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.708760+0000", 2024-09-14T06:29:31.318 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.6541e-05 2024-09-14T06:29:31.318 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:31.318 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:29:31.318 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:31.318 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:31.318 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:31.318 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:11466 3.1d 3:bcad7303:::rbd_data.112d92027590.0000000000000001:f [read 227328~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:29:31.318 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:25.656129+0000", 2024-09-14T06:29:31.318 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 5.6022013810000004, 2024-09-14T06:29:31.318 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052750987999999999, 2024-09-14T06:29:31.318 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:29:31.318 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:29:31.318 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:29:31.318 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:29:31.319 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:29:31.319 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:31.319 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 11466 2024-09-14T06:29:31.319 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:31.319 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:29:31.319 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:31.319 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:29:31.319 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.656129+0000", 2024-09-14T06:29:31.319 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:31.319 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:31.319 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:31.319 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:29:31.319 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.656129+0000", 2024-09-14T06:29:31.319 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:31.320 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:31.320 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:31.320 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:29:31.320 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.656347+0000", 2024-09-14T06:29:31.320 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.000218054 2024-09-14T06:29:31.320 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:31.320 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:31.320 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:29:31.320 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.656349+0000", 2024-09-14T06:29:31.320 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.9199999999999998e-06 2024-09-14T06:29:31.320 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:31.320 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:31.320 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:29:31.321 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.656353+0000", 2024-09-14T06:29:31.321 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.9210000000000002e-06 2024-09-14T06:29:31.321 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:31.321 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:31.321 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:29:31.321 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.656356+0000", 2024-09-14T06:29:31.321 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.7670000000000001e-06 2024-09-14T06:29:31.321 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:31.321 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:31.321 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:29:31.321 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.708787+0000", 2024-09-14T06:29:31.322 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052430840999999999 2024-09-14T06:29:31.322 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:31.322 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:31.322 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:29:31.322 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.708827+0000", 2024-09-14T06:29:31.322 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.9538e-05 2024-09-14T06:29:31.322 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:31.322 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:31.322 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:29:31.322 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.708880+0000", 2024-09-14T06:29:31.322 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.2947e-05 2024-09-14T06:29:31.322 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:31.322 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:29:31.322 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:31.323 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:31.323 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:31.323 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:11467 3.1d 3:bcad7303:::rbd_data.112d92027590.0000000000000001:f [read 237568~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:29:31.323 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:25.656361+0000", 2024-09-14T06:29:31.323 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 5.6019687329999996, 2024-09-14T06:29:31.323 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052643898000000001, 2024-09-14T06:29:31.323 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:29:31.323 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:29:31.323 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:29:31.323 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:29:31.323 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:29:31.323 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:31.323 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 11467 2024-09-14T06:29:31.324 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:31.324 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:29:31.324 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:31.324 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:29:31.324 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.656361+0000", 2024-09-14T06:29:31.324 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:31.324 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:31.324 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:31.324 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:29:31.324 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.656361+0000", 2024-09-14T06:29:31.324 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:31.324 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:31.324 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:31.324 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:29:31.325 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.656855+0000", 2024-09-14T06:29:31.325 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00049372000000000001 2024-09-14T06:29:31.325 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:31.325 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:31.325 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:29:31.325 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.656859+0000", 2024-09-14T06:29:31.325 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.1300000000000003e-06 2024-09-14T06:29:31.325 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:31.325 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:31.325 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:29:31.325 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.656863+0000", 2024-09-14T06:29:31.325 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.2629999999999997e-06 2024-09-14T06:29:31.325 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:31.325 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:31.326 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:29:31.326 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.656867+0000", 2024-09-14T06:29:31.326 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.3220000000000001e-06 2024-09-14T06:29:31.326 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:31.326 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:31.326 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:29:31.326 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.708903+0000", 2024-09-14T06:29:31.326 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052035855999999998 2024-09-14T06:29:31.326 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:31.326 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:31.326 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:29:31.326 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.708949+0000", 2024-09-14T06:29:31.326 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.6276999999999998e-05 2024-09-14T06:29:31.326 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:31.326 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:31.327 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:29:31.327 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.709005+0000", 2024-09-14T06:29:31.327 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.6329999999999998e-05 2024-09-14T06:29:31.327 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:31.327 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:29:31.327 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:31.327 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:31.327 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:31.327 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:11479 3.1d 3:bcad7303:::rbd_data.112d92027590.0000000000000001:f [read 360448~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:29:31.327 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:25.658388+0000", 2024-09-14T06:29:31.327 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 5.5999419000000001, 2024-09-14T06:29:31.327 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052408920999999997, 2024-09-14T06:29:31.327 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:29:31.327 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:29:31.328 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:29:31.328 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:29:31.328 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:29:31.328 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:31.328 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 11479 2024-09-14T06:29:31.328 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:31.328 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:29:31.328 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:31.328 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:29:31.328 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.658388+0000", 2024-09-14T06:29:31.328 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:31.328 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:31.328 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:31.328 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:29:31.329 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.658388+0000", 2024-09-14T06:29:31.329 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:31.329 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:31.329 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:31.329 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:29:31.329 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.658534+0000", 2024-09-14T06:29:31.329 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.000145613 2024-09-14T06:29:31.329 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:31.329 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:31.329 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:29:31.329 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.658540+0000", 2024-09-14T06:29:31.329 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.6680000000000004e-06 2024-09-14T06:29:31.329 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:31.329 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:31.330 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:29:31.330 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.658548+0000", 2024-09-14T06:29:31.330 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.7980000000000006e-06 2024-09-14T06:29:31.330 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:31.330 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:31.330 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:29:31.330 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.658554+0000", 2024-09-14T06:29:31.330 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.6869999999999999e-06 2024-09-14T06:29:31.330 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:31.330 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:31.330 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:29:31.330 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.710624+0000", 2024-09-14T06:29:31.330 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052070567999999998 2024-09-14T06:29:31.330 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:31.331 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:31.331 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:29:31.331 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.710666+0000", 2024-09-14T06:29:31.331 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.1467000000000001e-05 2024-09-14T06:29:31.331 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:31.331 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:31.331 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:29:31.331 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.710797+0000", 2024-09-14T06:29:31.331 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00013112 2024-09-14T06:29:31.331 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:31.331 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:29:31.331 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:31.331 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:31.331 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:31.332 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:11480 3.1d 3:bcad7303:::rbd_data.112d92027590.0000000000000001:f [read 370688~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:29:31.332 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:25.658563+0000", 2024-09-14T06:29:31.332 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 5.599766969, 2024-09-14T06:29:31.332 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052429164, 2024-09-14T06:29:31.332 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:29:31.332 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:29:31.332 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:29:31.332 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:29:31.332 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:29:31.332 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:31.332 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 11480 2024-09-14T06:29:31.332 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:31.332 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:29:31.333 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:31.333 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:29:31.333 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.658563+0000", 2024-09-14T06:29:31.333 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:31.333 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:31.333 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:31.333 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:29:31.333 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.658563+0000", 2024-09-14T06:29:31.333 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:31.333 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:31.333 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:31.333 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:29:31.333 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.658658+0000", 2024-09-14T06:29:31.334 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.5346999999999994e-05 2024-09-14T06:29:31.334 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:31.334 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:31.334 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:29:31.334 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.658662+0000", 2024-09-14T06:29:31.334 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.14e-06 2024-09-14T06:29:31.334 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:31.334 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:31.334 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:29:31.334 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.658669+0000", 2024-09-14T06:29:31.334 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.7379999999999994e-06 2024-09-14T06:29:31.334 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:31.334 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:31.334 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:29:31.335 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.658675+0000", 2024-09-14T06:29:31.335 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.5430000000000001e-06 2024-09-14T06:29:31.335 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:31.335 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:31.335 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:29:31.335 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.710824+0000", 2024-09-14T06:29:31.335 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052148847999999998 2024-09-14T06:29:31.335 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:31.335 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:31.335 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:29:31.335 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.710864+0000", 2024-09-14T06:29:31.335 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.9925999999999997e-05 2024-09-14T06:29:31.335 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:31.335 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:31.335 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:29:31.336 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.710992+0000", 2024-09-14T06:29:31.336 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.000128622 2024-09-14T06:29:31.336 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:31.336 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:29:31.336 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:31.336 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:31.336 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:31.336 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:11481 3.1d 3:bcad7303:::rbd_data.112d92027590.0000000000000001:f [read 380928~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:29:31.336 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:25.658684+0000", 2024-09-14T06:29:31.336 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 5.5996458990000004, 2024-09-14T06:29:31.336 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052508565, 2024-09-14T06:29:31.336 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:29:31.336 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:29:31.336 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:29:31.336 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:29:31.336 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:29:31.337 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:31.337 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 11481 2024-09-14T06:29:31.337 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:31.337 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:29:31.337 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:31.337 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:29:31.337 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.658684+0000", 2024-09-14T06:29:31.337 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:31.337 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:31.337 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:31.337 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:29:31.337 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.658684+0000", 2024-09-14T06:29:31.337 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:31.337 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:31.337 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:31.338 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:29:31.338 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.658715+0000", 2024-09-14T06:29:31.338 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.0703000000000001e-05 2024-09-14T06:29:31.338 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:31.338 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:31.338 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:29:31.338 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.658718+0000", 2024-09-14T06:29:31.338 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.0709999999999999e-06 2024-09-14T06:29:31.338 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:31.338 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:31.338 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:29:31.338 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.658725+0000", 2024-09-14T06:29:31.338 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.2339999999999997e-06 2024-09-14T06:29:31.339 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:31.339 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:31.339 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:29:31.339 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.658731+0000", 2024-09-14T06:29:31.339 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.6450000000000002e-06 2024-09-14T06:29:31.339 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:31.339 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:31.339 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:29:31.339 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.711020+0000", 2024-09-14T06:29:31.339 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052288886 2024-09-14T06:29:31.339 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:31.339 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:31.339 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:29:31.339 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.711061+0000", 2024-09-14T06:29:31.339 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.1084000000000002e-05 2024-09-14T06:29:31.339 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:31.340 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:31.340 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:29:31.340 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.711193+0000", 2024-09-14T06:29:31.340 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00013194199999999999 2024-09-14T06:29:31.340 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:31.340 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:29:31.340 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:31.340 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:31.340 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:31.340 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:11482 3.1d 3:bcad7303:::rbd_data.112d92027590.0000000000000001:f [read 391168~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:29:31.340 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:25.658740+0000", 2024-09-14T06:29:31.340 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 5.5995904339999996, 2024-09-14T06:29:31.340 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052637015000000002, 2024-09-14T06:29:31.340 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:29:31.341 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:29:31.341 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:29:31.341 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:29:31.341 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:29:31.341 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:31.341 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 11482 2024-09-14T06:29:31.341 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:31.341 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:29:31.341 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:31.341 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:29:31.341 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.658740+0000", 2024-09-14T06:29:31.341 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:31.341 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:31.341 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:31.341 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:29:31.342 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.658740+0000", 2024-09-14T06:29:31.342 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:31.342 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:31.342 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:31.342 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:29:31.342 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.658880+0000", 2024-09-14T06:29:31.342 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00014036100000000001 2024-09-14T06:29:31.342 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:31.342 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:31.342 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:29:31.342 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.658883+0000", 2024-09-14T06:29:31.342 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.3969999999999998e-06 2024-09-14T06:29:31.342 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:31.342 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:31.343 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:29:31.343 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.658890+0000", 2024-09-14T06:29:31.343 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.1500000000000002e-06 2024-09-14T06:29:31.343 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:31.343 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:31.343 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:29:31.343 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.658897+0000", 2024-09-14T06:29:31.343 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.1260000000000003e-06 2024-09-14T06:29:31.343 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:31.343 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:31.343 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:29:31.343 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.711207+0000", 2024-09-14T06:29:31.343 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052310777000000003 2024-09-14T06:29:31.344 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:31.344 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:31.344 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:29:31.344 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.711249+0000", 2024-09-14T06:29:31.344 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.1786999999999998e-05 2024-09-14T06:29:31.344 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:31.344 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:31.344 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:29:31.344 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.711377+0000", 2024-09-14T06:29:31.344 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.000127417 2024-09-14T06:29:31.344 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:31.344 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:29:31.344 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:31.345 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:31.345 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:31.345 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:11483 3.1d 3:bcad7303:::rbd_data.112d92027590.0000000000000001:f [read 401408~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:29:31.345 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:25.658906+0000", 2024-09-14T06:29:31.345 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 5.5994236500000003, 2024-09-14T06:29:31.345 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052595109000000001, 2024-09-14T06:29:31.345 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:29:31.345 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:29:31.345 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:29:31.345 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:29:31.345 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:29:31.346 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:31.346 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 11483 2024-09-14T06:29:31.346 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:31.346 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:29:31.346 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:31.346 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:29:31.346 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.658906+0000", 2024-09-14T06:29:31.346 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:31.346 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:31.346 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:31.346 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:29:31.346 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.658906+0000", 2024-09-14T06:29:31.347 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:31.347 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:31.347 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:31.347 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:29:31.347 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.664985+0000", 2024-09-14T06:29:31.347 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.0060787250000000001 2024-09-14T06:29:31.347 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:31.347 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:31.347 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:29:31.347 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.664991+0000", 2024-09-14T06:29:31.347 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.7030000000000003e-06 2024-09-14T06:29:31.347 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:31.347 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:31.347 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:29:31.347 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.664999+0000", 2024-09-14T06:29:31.348 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.4430000000000003e-06 2024-09-14T06:29:31.348 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:31.348 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:31.348 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:29:31.348 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.665006+0000", 2024-09-14T06:29:31.348 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.0840000000000003e-06 2024-09-14T06:29:31.348 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:31.348 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:31.348 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:29:31.348 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.711402+0000", 2024-09-14T06:29:31.348 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046395714999999997 2024-09-14T06:29:31.348 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:31.348 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:31.348 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:29:31.348 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.711443+0000", 2024-09-14T06:29:31.349 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.0911999999999997e-05 2024-09-14T06:29:31.349 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:31.349 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:31.349 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:29:31.349 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.711501+0000", 2024-09-14T06:29:31.349 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.8526999999999997e-05 2024-09-14T06:29:31.349 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:31.349 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:29:31.349 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:31.349 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:31.349 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:29:31.349 INFO:teuthology.orchestra.run.smithi138.stdout:} 2024-09-14T06:29:31.350 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-14T06:29:31.351 INFO:tasks.ceph.osd.0.smithi094.stderr:2024-09-14T06:29:31.348+0000 7f07c9333640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33636) UID: 0 2024-09-14T06:29:31.451 INFO:tasks.ceph.osd.3.smithi094.stderr:2024-09-14T06:29:31.449+0000 7feeeafe8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33655) UID: 0 2024-09-14T06:29:31.493 INFO:teuthology.orchestra.run.smithi138.stdout:{ 2024-09-14T06:29:31.494 INFO:teuthology.orchestra.run.smithi138.stdout: "ops": [], 2024-09-14T06:29:31.494 INFO:teuthology.orchestra.run.smithi138.stdout: "num_ops": 0 2024-09-14T06:29:31.494 INFO:teuthology.orchestra.run.smithi138.stdout:} 2024-09-14T06:29:31.502 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-14T06:29:31.552 INFO:tasks.ceph.osd.3.smithi094.stderr:2024-09-14T06:29:31.551+0000 7feeeafe8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33655) UID: 0 2024-09-14T06:29:31.604 INFO:teuthology.orchestra.run.smithi138.stdout:{ 2024-09-14T06:29:31.604 INFO:teuthology.orchestra.run.smithi138.stdout: "ops": [], 2024-09-14T06:29:31.604 INFO:teuthology.orchestra.run.smithi138.stdout: "complaint_time": 30, 2024-09-14T06:29:31.604 INFO:teuthology.orchestra.run.smithi138.stdout: "num_blocked_ops": 0 2024-09-14T06:29:31.604 INFO:teuthology.orchestra.run.smithi138.stdout:} 2024-09-14T06:29:31.612 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-14T06:29:31.652 INFO:tasks.ceph.osd.2.smithi094.stderr:2024-09-14T06:29:31.651+0000 7faf3269b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33649) UID: 0 2024-09-14T06:29:31.719 INFO:teuthology.orchestra.run.smithi138.stdout:{ 2024-09-14T06:29:31.719 INFO:teuthology.orchestra.run.smithi138.stdout: "size": 20, 2024-09-14T06:29:31.719 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 600, 2024-09-14T06:29:31.720 INFO:teuthology.orchestra.run.smithi138.stdout: "ops": [ 2024-09-14T06:29:31.720 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:31.720 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:1484 3.a 3:57c28b0f:::rbd_data.112d7dfbd766.000000000000000a:5 [read 208896~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-14T06:29:31.720 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:08.628246+0000", 2024-09-14T06:29:31.720 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 23.089421210000001, 2024-09-14T06:29:31.720 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.018096672000000001, 2024-09-14T06:29:31.720 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:29:31.720 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:29:31.720 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:29:31.720 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:29:31.720 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:29:31.720 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:31.720 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 1484 2024-09-14T06:29:31.721 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:31.721 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:29:31.721 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:31.721 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:29:31.721 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.628246+0000", 2024-09-14T06:29:31.721 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:31.721 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:31.721 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:31.721 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:29:31.721 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.628246+0000", 2024-09-14T06:29:31.721 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:31.721 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:31.721 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:31.722 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:29:31.722 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.628246+0000", 2024-09-14T06:29:31.722 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.79e-07 2024-09-14T06:29:31.722 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:31.722 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:31.722 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:29:31.722 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.628247+0000", 2024-09-14T06:29:31.722 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.5179999999999999e-06 2024-09-14T06:29:31.722 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:31.722 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:31.722 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:29:31.722 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.628252+0000", 2024-09-14T06:29:31.722 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.0139999999999998e-06 2024-09-14T06:29:31.722 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:31.722 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:31.723 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:29:31.723 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.628257+0000", 2024-09-14T06:29:31.723 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.2429999999999999e-06 2024-09-14T06:29:31.723 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:31.723 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:31.723 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:29:31.723 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.646180+0000", 2024-09-14T06:29:31.723 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.017923136999999999 2024-09-14T06:29:31.723 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:31.723 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:31.723 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:29:31.723 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.646251+0000", 2024-09-14T06:29:31.723 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.1285000000000006e-05 2024-09-14T06:29:31.723 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:31.724 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:31.724 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:29:31.724 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.646342+0000", 2024-09-14T06:29:31.724 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.1195999999999999e-05 2024-09-14T06:29:31.724 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:31.724 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:29:31.724 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:31.724 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:31.724 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:31.724 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:1485 3.a 3:57c28b0f:::rbd_data.112d7dfbd766.000000000000000a:5 [read 225280~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-14T06:29:31.724 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:08.628268+0000", 2024-09-14T06:29:31.725 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 23.089398728999999, 2024-09-14T06:29:31.725 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.018257005999999999, 2024-09-14T06:29:31.725 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:29:31.725 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:29:31.725 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:29:31.725 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:29:31.725 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:29:31.725 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:31.725 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 1485 2024-09-14T06:29:31.725 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:31.725 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:29:31.725 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:31.725 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:29:31.726 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.628268+0000", 2024-09-14T06:29:31.726 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:31.726 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:31.726 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:31.726 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:29:31.726 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.628268+0000", 2024-09-14T06:29:31.726 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:31.726 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:31.726 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:31.726 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:29:31.726 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.628269+0000", 2024-09-14T06:29:31.726 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.6800000000000001e-07 2024-09-14T06:29:31.726 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:31.726 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:31.727 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:29:31.727 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.628270+0000", 2024-09-14T06:29:31.727 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.8449999999999999e-06 2024-09-14T06:29:31.727 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:31.727 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:31.727 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:29:31.727 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.628276+0000", 2024-09-14T06:29:31.727 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.998e-06 2024-09-14T06:29:31.727 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:31.727 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:31.727 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:29:31.727 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.628281+0000", 2024-09-14T06:29:31.727 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.3120000000000004e-06 2024-09-14T06:29:31.728 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:31.728 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:31.728 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:29:31.728 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.646371+0000", 2024-09-14T06:29:31.728 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.018090694000000001 2024-09-14T06:29:31.728 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:31.728 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:31.728 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:29:31.728 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.646439+0000", 2024-09-14T06:29:31.728 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.7182999999999997e-05 2024-09-14T06:29:31.728 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:31.728 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:31.728 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:29:31.728 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.646525+0000", 2024-09-14T06:29:31.729 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.6506000000000002e-05 2024-09-14T06:29:31.729 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:31.729 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:29:31.729 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:31.729 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:31.729 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:31.729 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:1486 3.a 3:57c28b0f:::rbd_data.112d7dfbd766.000000000000000a:5 [read 241664~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-14T06:29:31.729 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:08.628288+0000", 2024-09-14T06:29:31.729 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 23.089378842999999, 2024-09-14T06:29:31.729 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.018416895999999999, 2024-09-14T06:29:31.729 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:29:31.729 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:29:31.729 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:29:31.729 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:29:31.730 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:29:31.730 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:31.730 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 1486 2024-09-14T06:29:31.730 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:31.730 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:29:31.730 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:31.730 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:29:31.730 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.628288+0000", 2024-09-14T06:29:31.730 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:31.730 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:31.730 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:31.730 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:29:31.730 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.628288+0000", 2024-09-14T06:29:31.730 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:31.731 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:31.731 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:31.731 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:29:31.731 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.628288+0000", 2024-09-14T06:29:31.731 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.9499999999999998e-07 2024-09-14T06:29:31.731 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:31.731 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:31.731 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:29:31.731 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.628290+0000", 2024-09-14T06:29:31.731 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.6840000000000001e-06 2024-09-14T06:29:31.731 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:31.731 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:31.731 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:29:31.732 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.628295+0000", 2024-09-14T06:29:31.732 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.3059999999999997e-06 2024-09-14T06:29:31.732 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:31.732 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:31.732 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:29:31.732 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.628300+0000", 2024-09-14T06:29:31.732 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.2690000000000001e-06 2024-09-14T06:29:31.732 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:31.732 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:31.732 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:29:31.733 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.646553+0000", 2024-09-14T06:29:31.733 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.018253015000000001 2024-09-14T06:29:31.733 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:31.733 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:31.733 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:29:31.733 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.646626+0000", 2024-09-14T06:29:31.733 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.2939000000000001e-05 2024-09-14T06:29:31.733 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:31.733 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:31.733 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:29:31.733 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.646705+0000", 2024-09-14T06:29:31.733 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.9388000000000006e-05 2024-09-14T06:29:31.733 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:31.733 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:29:31.734 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:31.734 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:31.734 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:31.734 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:1487 3.a 3:57c28b0f:::rbd_data.112d7dfbd766.000000000000000a:5 [read 258048~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-14T06:29:31.734 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:08.628842+0000", 2024-09-14T06:29:31.734 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 23.088824399, 2024-09-14T06:29:31.734 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.018037741999999999, 2024-09-14T06:29:31.734 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:29:31.734 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:29:31.734 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:29:31.734 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:29:31.734 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:29:31.734 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:31.734 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 1487 2024-09-14T06:29:31.735 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:31.735 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:29:31.735 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:31.735 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:29:31.735 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.628842+0000", 2024-09-14T06:29:31.735 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:31.735 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:31.735 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:31.735 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:29:31.735 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.628842+0000", 2024-09-14T06:29:31.735 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:31.735 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:31.735 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:31.735 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:29:31.736 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.628843+0000", 2024-09-14T06:29:31.736 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.5600000000000005e-07 2024-09-14T06:29:31.736 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:31.736 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:31.736 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:29:31.736 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.628847+0000", 2024-09-14T06:29:31.736 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.394e-06 2024-09-14T06:29:31.736 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:31.736 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:31.736 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:29:31.736 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.628855+0000", 2024-09-14T06:29:31.736 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.0760000000000003e-06 2024-09-14T06:29:31.736 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:31.736 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:31.737 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:29:31.737 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.628868+0000", 2024-09-14T06:29:31.737 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.2860000000000001e-05 2024-09-14T06:29:31.737 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:31.737 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:31.737 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:29:31.737 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.646733+0000", 2024-09-14T06:29:31.737 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.017865387999999999 2024-09-14T06:29:31.737 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:31.737 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:31.737 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:29:31.737 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.646800+0000", 2024-09-14T06:29:31.737 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.6619999999999996e-05 2024-09-14T06:29:31.737 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:31.738 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:31.738 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:29:31.738 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.646880+0000", 2024-09-14T06:29:31.738 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.0648000000000004e-05 2024-09-14T06:29:31.738 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:31.738 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:29:31.738 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:31.738 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:31.738 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:31.738 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:1488 3.a 3:57c28b0f:::rbd_data.112d7dfbd766.000000000000000a:5 [read 274432~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-14T06:29:31.738 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:08.628878+0000", 2024-09-14T06:29:31.738 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 23.088788781000002, 2024-09-14T06:29:31.738 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.018187624999999999, 2024-09-14T06:29:31.739 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:29:31.739 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:29:31.739 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:29:31.739 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:29:31.739 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:29:31.739 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:31.739 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 1488 2024-09-14T06:29:31.739 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:31.739 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:29:31.739 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:31.739 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:29:31.739 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.628878+0000", 2024-09-14T06:29:31.739 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:31.739 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:31.740 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:31.740 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:29:31.740 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.628878+0000", 2024-09-14T06:29:31.740 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:31.740 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:31.740 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:31.740 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:29:31.740 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.628879+0000", 2024-09-14T06:29:31.740 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.58e-07 2024-09-14T06:29:31.740 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:31.740 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:31.740 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:29:31.741 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.628881+0000", 2024-09-14T06:29:31.741 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.0389999999999999e-06 2024-09-14T06:29:31.741 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:31.741 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:31.741 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:29:31.741 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.628887+0000", 2024-09-14T06:29:31.741 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.9429999999999997e-06 2024-09-14T06:29:31.741 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:31.741 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:31.741 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:29:31.741 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.628891+0000", 2024-09-14T06:29:31.741 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.1309999999999999e-06 2024-09-14T06:29:31.741 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:31.742 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:31.742 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:29:31.742 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.646909+0000", 2024-09-14T06:29:31.742 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.018017920999999999 2024-09-14T06:29:31.742 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:31.742 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:31.742 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:29:31.742 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.646984+0000", 2024-09-14T06:29:31.742 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.4989999999999999e-05 2024-09-14T06:29:31.742 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:31.742 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:31.742 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:29:31.742 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.647066+0000", 2024-09-14T06:29:31.742 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.2143000000000003e-05 2024-09-14T06:29:31.743 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:31.743 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:29:31.743 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:31.743 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:31.743 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:31.743 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:1489 3.a 3:57c28b0f:::rbd_data.112d7dfbd766.000000000000000a:5 [read 290816~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-14T06:29:31.743 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:08.628897+0000", 2024-09-14T06:29:31.743 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 23.088770329999999, 2024-09-14T06:29:31.743 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.018341491000000001, 2024-09-14T06:29:31.743 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:29:31.743 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:29:31.743 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:29:31.743 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:29:31.743 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:29:31.744 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:31.744 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 1489 2024-09-14T06:29:31.744 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:31.744 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:29:31.744 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:31.744 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:29:31.744 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.628897+0000", 2024-09-14T06:29:31.744 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:31.744 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:31.744 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:31.745 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:29:31.745 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.628897+0000", 2024-09-14T06:29:31.745 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:31.745 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:31.745 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:31.745 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:29:31.745 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.628897+0000", 2024-09-14T06:29:31.745 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.6800000000000001e-07 2024-09-14T06:29:31.745 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:31.745 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:31.745 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:29:31.745 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.628899+0000", 2024-09-14T06:29:31.745 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.606e-06 2024-09-14T06:29:31.745 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:31.746 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:31.746 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:29:31.746 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.628904+0000", 2024-09-14T06:29:31.746 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.1839999999999998e-06 2024-09-14T06:29:31.746 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:31.746 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:31.746 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:29:31.746 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.628908+0000", 2024-09-14T06:29:31.746 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.0500000000000002e-06 2024-09-14T06:29:31.746 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:31.746 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:31.746 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:29:31.746 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.647093+0000", 2024-09-14T06:29:31.746 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.018184846000000001 2024-09-14T06:29:31.747 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:31.747 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:31.747 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:29:31.747 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.647158+0000", 2024-09-14T06:29:31.747 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.5068999999999997e-05 2024-09-14T06:29:31.747 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:31.747 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:31.747 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:29:31.747 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.647238+0000", 2024-09-14T06:29:31.747 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.0368e-05 2024-09-14T06:29:31.747 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:31.748 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:29:31.748 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:31.748 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:31.748 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:31.748 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:1490 3.a 3:57c28b0f:::rbd_data.112d7dfbd766.000000000000000a:5 [read 307200~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-14T06:29:31.748 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:08.628921+0000", 2024-09-14T06:29:31.748 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 23.088746333, 2024-09-14T06:29:31.748 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.018575338, 2024-09-14T06:29:31.748 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:29:31.748 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:29:31.748 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:29:31.749 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:29:31.749 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:29:31.749 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:31.749 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 1490 2024-09-14T06:29:31.749 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:31.749 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:29:31.749 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:31.749 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:29:31.749 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.628921+0000", 2024-09-14T06:29:31.749 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:31.749 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:31.749 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:31.749 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:29:31.749 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.628921+0000", 2024-09-14T06:29:31.750 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:31.750 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:31.750 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:31.750 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:29:31.750 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.628921+0000", 2024-09-14T06:29:31.750 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.39e-07 2024-09-14T06:29:31.750 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:31.750 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:31.750 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:29:31.750 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.628928+0000", 2024-09-14T06:29:31.750 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.81e-06 2024-09-14T06:29:31.750 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:31.750 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:31.750 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:29:31.751 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.628934+0000", 2024-09-14T06:29:31.751 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.4649999999999999e-06 2024-09-14T06:29:31.751 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:31.751 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:31.751 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:29:31.751 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.628939+0000", 2024-09-14T06:29:31.751 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.3730000000000003e-06 2024-09-14T06:29:31.751 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:31.751 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:31.751 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:29:31.751 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.647265+0000", 2024-09-14T06:29:31.751 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.018326499999999999 2024-09-14T06:29:31.751 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:31.751 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:31.751 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:29:31.752 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.647381+0000", 2024-09-14T06:29:31.752 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.000115549 2024-09-14T06:29:31.752 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:31.752 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:31.752 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:29:31.752 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.647496+0000", 2024-09-14T06:29:31.752 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.000115202 2024-09-14T06:29:31.752 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:31.752 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:29:31.752 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:31.752 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:31.752 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:31.752 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:1491 3.a 3:57c28b0f:::rbd_data.112d7dfbd766.000000000000000a:5 [read 323584~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-14T06:29:31.753 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:08.628964+0000", 2024-09-14T06:29:31.753 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 23.08870327, 2024-09-14T06:29:31.753 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.018753204999999998, 2024-09-14T06:29:31.753 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:29:31.753 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:29:31.753 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:29:31.753 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:29:31.753 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:29:31.753 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:31.753 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 1491 2024-09-14T06:29:31.753 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:31.753 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:29:31.753 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:31.754 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:29:31.754 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.628964+0000", 2024-09-14T06:29:31.754 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:31.754 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:31.754 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:31.754 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:29:31.754 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.628964+0000", 2024-09-14T06:29:31.754 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:31.754 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:31.754 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:31.754 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:29:31.754 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.628964+0000", 2024-09-14T06:29:31.754 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.3600000000000004e-07 2024-09-14T06:29:31.754 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:31.754 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:31.755 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:29:31.755 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.628966+0000", 2024-09-14T06:29:31.755 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.119e-06 2024-09-14T06:29:31.755 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:31.755 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:31.755 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:29:31.755 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.628972+0000", 2024-09-14T06:29:31.755 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.0929999999999999e-06 2024-09-14T06:29:31.755 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:31.755 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:31.755 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:29:31.755 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.628977+0000", 2024-09-14T06:29:31.755 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.5360000000000003e-06 2024-09-14T06:29:31.755 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:31.756 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:31.756 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:29:31.756 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.647527+0000", 2024-09-14T06:29:31.756 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.018550433000000002 2024-09-14T06:29:31.756 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:31.756 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:31.756 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:29:31.756 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.647614+0000", 2024-09-14T06:29:31.756 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.6757999999999998e-05 2024-09-14T06:29:31.756 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:31.756 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:31.756 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:29:31.756 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.647717+0000", 2024-09-14T06:29:31.757 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00010273 2024-09-14T06:29:31.757 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:31.757 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:29:31.757 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:31.757 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:31.757 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:31.757 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:1492 3.a 3:57c28b0f:::rbd_data.112d7dfbd766.000000000000000a:5 [read 339968~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-14T06:29:31.757 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:08.628983+0000", 2024-09-14T06:29:31.757 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 23.088683586999998, 2024-09-14T06:29:31.757 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.018937253000000001, 2024-09-14T06:29:31.757 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:29:31.757 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:29:31.758 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:29:31.758 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:29:31.758 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:29:31.758 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:31.758 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 1492 2024-09-14T06:29:31.758 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:31.758 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:29:31.758 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:31.758 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:29:31.758 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.628983+0000", 2024-09-14T06:29:31.758 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:31.758 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:31.759 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:31.759 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:29:31.759 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.628983+0000", 2024-09-14T06:29:31.759 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:31.759 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:31.759 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:31.759 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:29:31.759 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.628984+0000", 2024-09-14T06:29:31.759 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.0100000000000001e-07 2024-09-14T06:29:31.759 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:31.759 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:31.760 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:29:31.760 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.628985+0000", 2024-09-14T06:29:31.760 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.561e-06 2024-09-14T06:29:31.760 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:31.760 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:31.760 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:29:31.760 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.628990+0000", 2024-09-14T06:29:31.760 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.1540000000000001e-06 2024-09-14T06:29:31.760 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:31.760 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:31.760 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:29:31.760 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.628994+0000", 2024-09-14T06:29:31.760 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.8890000000000004e-06 2024-09-14T06:29:31.761 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:31.761 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:31.761 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:29:31.761 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.647745+0000", 2024-09-14T06:29:31.761 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.018751213999999999 2024-09-14T06:29:31.761 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:31.761 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:31.761 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:29:31.761 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.647824+0000", 2024-09-14T06:29:31.761 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.8331999999999995e-05 2024-09-14T06:29:31.761 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:31.761 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:31.761 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:29:31.761 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.647921+0000", 2024-09-14T06:29:31.762 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.6801999999999999e-05 2024-09-14T06:29:31.762 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:31.762 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:29:31.762 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:31.762 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:31.762 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:31.762 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:1493 3.a 3:57c28b0f:::rbd_data.112d7dfbd766.000000000000000a:5 [read 356352~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-14T06:29:31.762 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:08.628999+0000", 2024-09-14T06:29:31.762 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 23.088667672, 2024-09-14T06:29:31.762 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.019127128, 2024-09-14T06:29:31.762 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:29:31.762 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:29:31.762 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:29:31.762 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:29:31.762 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:29:31.763 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:31.763 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 1493 2024-09-14T06:29:31.763 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:31.763 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:29:31.763 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:31.763 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:29:31.763 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.628999+0000", 2024-09-14T06:29:31.763 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:31.763 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:31.763 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:31.763 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:29:31.763 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.628999+0000", 2024-09-14T06:29:31.763 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:31.764 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:31.764 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:31.764 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:29:31.764 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.629000+0000", 2024-09-14T06:29:31.764 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.2500000000000001e-07 2024-09-14T06:29:31.764 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:31.764 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:31.764 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:29:31.764 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.629001+0000", 2024-09-14T06:29:31.764 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.511e-06 2024-09-14T06:29:31.764 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:31.764 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:31.764 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:29:31.764 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.629006+0000", 2024-09-14T06:29:31.765 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.1420000000000001e-06 2024-09-14T06:29:31.765 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:31.765 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:31.765 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:29:31.765 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.629010+0000", 2024-09-14T06:29:31.765 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.8120000000000001e-06 2024-09-14T06:29:31.765 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:31.765 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:31.765 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:29:31.765 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.647950+0000", 2024-09-14T06:29:31.765 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.018939568 2024-09-14T06:29:31.765 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:31.765 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:31.765 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:29:31.766 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.648032+0000", 2024-09-14T06:29:31.766 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.2132000000000006e-05 2024-09-14T06:29:31.766 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:31.766 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:31.766 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:29:31.766 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.648126+0000", 2024-09-14T06:29:31.766 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.4637999999999998e-05 2024-09-14T06:29:31.766 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:31.766 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:29:31.766 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:31.766 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:31.766 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:31.766 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:1494 3.a 3:57c28b0f:::rbd_data.112d7dfbd766.000000000000000a:5 [read 372736~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-14T06:29:31.766 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:08.629017+0000", 2024-09-14T06:29:31.766 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 23.088649611000001, 2024-09-14T06:29:31.767 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.019310414000000001, 2024-09-14T06:29:31.767 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:29:31.767 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:29:31.767 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:29:31.767 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:29:31.767 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:29:31.767 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:31.767 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 1494 2024-09-14T06:29:31.767 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:31.767 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:29:31.767 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:31.767 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:29:31.767 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.629017+0000", 2024-09-14T06:29:31.767 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:31.767 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:31.768 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:31.768 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:29:31.768 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.629017+0000", 2024-09-14T06:29:31.768 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:31.768 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:31.768 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:31.768 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:29:31.768 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.629018+0000", 2024-09-14T06:29:31.768 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.9000000000000002e-07 2024-09-14T06:29:31.768 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:31.768 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:31.768 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:29:31.769 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.629019+0000", 2024-09-14T06:29:31.769 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.5230000000000001e-06 2024-09-14T06:29:31.769 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:31.769 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:31.769 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:29:31.769 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.629024+0000", 2024-09-14T06:29:31.769 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.2299999999999999e-06 2024-09-14T06:29:31.769 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:31.770 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:31.770 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:29:31.770 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.629028+0000", 2024-09-14T06:29:31.770 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.7749999999999999e-06 2024-09-14T06:29:31.770 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:31.770 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:31.770 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:29:31.770 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.648156+0000", 2024-09-14T06:29:31.770 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.019128045999999999 2024-09-14T06:29:31.770 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:31.771 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:31.771 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:29:31.771 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.648238+0000", 2024-09-14T06:29:31.771 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.1877000000000002e-05 2024-09-14T06:29:31.771 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:31.771 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:31.771 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:29:31.771 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.648328+0000", 2024-09-14T06:29:31.771 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.9573000000000005e-05 2024-09-14T06:29:31.771 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:31.771 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:29:31.771 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:31.771 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:31.771 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:31.772 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:1495 3.a 3:57c28b0f:::rbd_data.112d7dfbd766.000000000000000a:5 [read 389120~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-14T06:29:31.772 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:08.629035+0000", 2024-09-14T06:29:31.772 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 23.088631586999998, 2024-09-14T06:29:31.772 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.019486507, 2024-09-14T06:29:31.772 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:29:31.772 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:29:31.772 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:29:31.772 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:29:31.772 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:29:31.772 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:31.772 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 1495 2024-09-14T06:29:31.773 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:31.773 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:29:31.773 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:31.773 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:29:31.773 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.629035+0000", 2024-09-14T06:29:31.773 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:31.773 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:31.773 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:31.774 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:29:31.774 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.629035+0000", 2024-09-14T06:29:31.774 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:31.774 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:31.774 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:31.774 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:29:31.774 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.629036+0000", 2024-09-14T06:29:31.774 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.2000000000000001e-07 2024-09-14T06:29:31.775 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:31.775 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:31.775 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:29:31.775 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.629037+0000", 2024-09-14T06:29:31.775 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.4300000000000001e-06 2024-09-14T06:29:31.775 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:31.775 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:31.775 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:29:31.775 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.629043+0000", 2024-09-14T06:29:31.775 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.575e-06 2024-09-14T06:29:31.776 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:31.776 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:31.776 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:29:31.776 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.629047+0000", 2024-09-14T06:29:31.776 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.9179999999999996e-06 2024-09-14T06:29:31.776 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:31.776 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:31.776 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:29:31.777 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.648356+0000", 2024-09-14T06:29:31.777 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.019309862000000001 2024-09-14T06:29:31.777 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:31.777 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:31.777 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:29:31.777 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.648433+0000", 2024-09-14T06:29:31.777 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.6973999999999999e-05 2024-09-14T06:29:31.777 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:31.777 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:31.778 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:29:31.778 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.648522+0000", 2024-09-14T06:29:31.778 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.8428000000000001e-05 2024-09-14T06:29:31.778 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:31.778 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:29:31.778 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:31.778 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:31.778 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:31.778 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:1496 3.a 3:57c28b0f:::rbd_data.112d7dfbd766.000000000000000a:5 [read 405504~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-14T06:29:31.779 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:08.629086+0000", 2024-09-14T06:29:31.779 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 23.088580542999999, 2024-09-14T06:29:31.779 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.019618813999999998, 2024-09-14T06:29:31.779 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:29:31.779 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:29:31.779 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:29:31.779 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:29:31.779 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:29:31.780 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:31.780 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 1496 2024-09-14T06:29:31.780 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:31.780 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:29:31.780 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:31.780 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:29:31.780 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.629086+0000", 2024-09-14T06:29:31.780 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:31.780 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:31.781 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:31.781 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:29:31.781 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.629086+0000", 2024-09-14T06:29:31.781 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:31.781 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:31.781 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:31.781 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:29:31.781 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.629087+0000", 2024-09-14T06:29:31.782 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.1399999999999996e-07 2024-09-14T06:29:31.782 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:31.782 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:31.782 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:29:31.782 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.629100+0000", 2024-09-14T06:29:31.782 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.2748999999999999e-05 2024-09-14T06:29:31.782 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:31.782 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:31.783 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:29:31.783 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.629111+0000", 2024-09-14T06:29:31.783 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.1531e-05 2024-09-14T06:29:31.783 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:31.783 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:31.783 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:29:31.783 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.629131+0000", 2024-09-14T06:29:31.783 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.9341999999999999e-05 2024-09-14T06:29:31.783 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:31.784 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:31.784 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:29:31.784 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.648550+0000", 2024-09-14T06:29:31.784 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.019418916000000001 2024-09-14T06:29:31.784 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:31.784 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:31.784 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:29:31.784 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.648616+0000", 2024-09-14T06:29:31.784 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.6359000000000006e-05 2024-09-14T06:29:31.784 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:31.784 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:31.784 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:29:31.785 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.648705+0000", 2024-09-14T06:29:31.785 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.9103000000000004e-05 2024-09-14T06:29:31.785 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:31.785 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:29:31.785 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:31.785 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:31.785 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:31.785 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:1497 3.a 3:57c28b0f:::rbd_data.112d7dfbd766.000000000000000a:5 [read 421888~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-14T06:29:31.785 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:08.629145+0000", 2024-09-14T06:29:31.785 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 23.088521764999999, 2024-09-14T06:29:31.785 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.019744725000000001, 2024-09-14T06:29:31.785 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:29:31.785 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:29:31.785 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:29:31.786 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:29:31.786 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:29:31.786 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:31.786 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 1497 2024-09-14T06:29:31.786 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:31.786 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:29:31.786 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:31.786 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:29:31.786 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.629145+0000", 2024-09-14T06:29:31.786 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:31.786 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:31.786 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:31.786 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:29:31.786 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.629145+0000", 2024-09-14T06:29:31.786 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:31.786 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:31.787 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:31.787 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:29:31.787 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.629146+0000", 2024-09-14T06:29:31.787 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.1699999999999999e-07 2024-09-14T06:29:31.787 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:31.787 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:31.787 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:29:31.787 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.629148+0000", 2024-09-14T06:29:31.787 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.024e-06 2024-09-14T06:29:31.787 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:31.787 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:31.787 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:29:31.787 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.629152+0000", 2024-09-14T06:29:31.787 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.5970000000000002e-06 2024-09-14T06:29:31.787 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:31.787 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:31.788 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:29:31.788 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.629157+0000", 2024-09-14T06:29:31.788 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.8119999999999998e-06 2024-09-14T06:29:31.788 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:31.788 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:31.788 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:29:31.788 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.648734+0000", 2024-09-14T06:29:31.788 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.019576935 2024-09-14T06:29:31.788 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:31.788 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:31.788 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:29:31.788 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.648801+0000", 2024-09-14T06:29:31.789 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.7305999999999996e-05 2024-09-14T06:29:31.789 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:31.789 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:31.789 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:29:31.789 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.648890+0000", 2024-09-14T06:29:31.789 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.8634000000000006e-05 2024-09-14T06:29:31.789 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:31.789 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:29:31.789 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:31.789 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:31.789 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:31.789 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:1498 3.a 3:57c28b0f:::rbd_data.112d7dfbd766.000000000000000a:5 [read 438272~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-14T06:29:31.789 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:08.629167+0000", 2024-09-14T06:29:31.789 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 23.088499701, 2024-09-14T06:29:31.789 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.019912138999999999, 2024-09-14T06:29:31.790 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:29:31.790 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:29:31.790 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:29:31.790 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:29:31.790 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:29:31.790 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:31.790 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 1498 2024-09-14T06:29:31.790 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:31.790 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:29:31.790 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:31.790 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:29:31.790 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.629167+0000", 2024-09-14T06:29:31.790 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:31.790 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:31.791 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:31.791 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:29:31.791 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.629167+0000", 2024-09-14T06:29:31.791 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:31.791 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:31.791 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:31.791 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:29:31.791 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.629168+0000", 2024-09-14T06:29:31.791 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.1300000000000001e-07 2024-09-14T06:29:31.791 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:31.791 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:31.792 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:29:31.792 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.629170+0000", 2024-09-14T06:29:31.792 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.7209999999999999e-06 2024-09-14T06:29:31.792 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:31.792 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:31.792 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:29:31.792 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.629174+0000", 2024-09-14T06:29:31.792 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.1459999999999998e-06 2024-09-14T06:29:31.792 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:31.792 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:31.792 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:29:31.792 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.629178+0000", 2024-09-14T06:29:31.792 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.8519999999999997e-06 2024-09-14T06:29:31.793 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:31.793 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:31.793 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:29:31.793 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.648924+0000", 2024-09-14T06:29:31.793 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.019746107999999998 2024-09-14T06:29:31.793 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:31.793 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:31.793 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:29:31.793 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.648995+0000", 2024-09-14T06:29:31.793 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.0809999999999995e-05 2024-09-14T06:29:31.793 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:31.793 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:31.793 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:29:31.793 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.649079+0000", 2024-09-14T06:29:31.794 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.4089000000000001e-05 2024-09-14T06:29:31.794 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:31.794 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:29:31.794 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:31.794 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:31.794 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:31.794 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:1499 3.a 3:57c28b0f:::rbd_data.112d7dfbd766.000000000000000a:5 [read 454656~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-14T06:29:31.794 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:08.629187+0000", 2024-09-14T06:29:31.794 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 23.088479737, 2024-09-14T06:29:31.794 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.020072543000000002, 2024-09-14T06:29:31.794 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:29:31.794 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:29:31.794 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:29:31.795 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:29:31.795 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:29:31.795 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:31.795 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 1499 2024-09-14T06:29:31.795 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:31.795 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:29:31.795 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:31.795 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:29:31.795 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.629187+0000", 2024-09-14T06:29:31.795 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:31.795 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:31.795 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:31.795 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:29:31.795 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.629187+0000", 2024-09-14T06:29:31.795 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:31.795 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:31.796 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:31.796 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:29:31.796 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.629188+0000", 2024-09-14T06:29:31.796 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.89e-07 2024-09-14T06:29:31.796 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:31.796 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:31.796 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:29:31.796 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.629190+0000", 2024-09-14T06:29:31.796 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.0169999999999999e-06 2024-09-14T06:29:31.796 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:31.796 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:31.796 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:29:31.796 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.629195+0000", 2024-09-14T06:29:31.797 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.4149999999999998e-06 2024-09-14T06:29:31.797 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:31.797 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:31.797 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:29:31.797 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.629198+0000", 2024-09-14T06:29:31.797 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.605e-06 2024-09-14T06:29:31.797 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:31.798 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:31.798 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:29:31.798 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.649107+0000", 2024-09-14T06:29:31.798 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.019909339000000002 2024-09-14T06:29:31.798 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:31.798 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:31.798 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:29:31.798 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.649178+0000", 2024-09-14T06:29:31.798 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.0809999999999995e-05 2024-09-14T06:29:31.798 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:31.798 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:31.798 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:29:31.799 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.649260+0000", 2024-09-14T06:29:31.799 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.1967999999999999e-05 2024-09-14T06:29:31.799 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:31.799 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:29:31.799 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:31.799 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:31.799 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:31.799 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:1500 3.a 3:57c28b0f:::rbd_data.112d7dfbd766.000000000000000a:5 [read 471040~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-14T06:29:31.799 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:08.629204+0000", 2024-09-14T06:29:31.799 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 23.088463245, 2024-09-14T06:29:31.799 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.020473080000000001, 2024-09-14T06:29:31.799 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:29:31.799 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:29:31.800 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:29:31.800 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:29:31.800 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:29:31.800 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:31.800 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 1500 2024-09-14T06:29:31.800 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:31.800 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:29:31.800 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:31.800 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:29:31.800 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.629204+0000", 2024-09-14T06:29:31.800 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:31.800 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:31.800 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:31.800 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:29:31.801 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.629204+0000", 2024-09-14T06:29:31.801 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:31.801 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:31.801 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:31.801 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:29:31.801 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.629204+0000", 2024-09-14T06:29:31.801 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.34e-07 2024-09-14T06:29:31.801 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:31.801 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:31.801 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:29:31.801 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.629206+0000", 2024-09-14T06:29:31.801 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.939e-06 2024-09-14T06:29:31.801 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:31.801 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:31.802 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:29:31.802 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.629212+0000", 2024-09-14T06:29:31.802 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.8669999999999999e-06 2024-09-14T06:29:31.802 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:31.802 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:31.802 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:29:31.802 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.629216+0000", 2024-09-14T06:29:31.802 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.3830000000000002e-06 2024-09-14T06:29:31.802 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:31.802 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:31.802 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:29:31.802 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.649288+0000", 2024-09-14T06:29:31.802 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.020072113999999999 2024-09-14T06:29:31.802 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:31.803 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:31.803 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:29:31.803 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.649355+0000", 2024-09-14T06:29:31.803 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.7218000000000006e-05 2024-09-14T06:29:31.803 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:31.803 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:31.803 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:29:31.803 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.649677+0000", 2024-09-14T06:29:31.803 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00032122500000000002 2024-09-14T06:29:31.803 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:31.803 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:29:31.803 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:31.803 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:31.803 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:31.804 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:1501 3.a 3:57c28b0f:::rbd_data.112d7dfbd766.000000000000000a:5 [read 487424~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-14T06:29:31.804 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:08.629223+0000", 2024-09-14T06:29:31.804 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 23.088443796, 2024-09-14T06:29:31.804 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.021370209000000001, 2024-09-14T06:29:31.804 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:29:31.804 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:29:31.804 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:29:31.804 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:29:31.805 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:29:31.805 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:31.805 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 1501 2024-09-14T06:29:31.805 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:31.805 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:29:31.805 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:31.805 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:29:31.805 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.629223+0000", 2024-09-14T06:29:31.805 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:31.806 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:31.806 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:31.806 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:29:31.806 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.629223+0000", 2024-09-14T06:29:31.806 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:31.806 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:31.806 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:31.806 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:29:31.806 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.629223+0000", 2024-09-14T06:29:31.806 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.9900000000000001e-07 2024-09-14T06:29:31.806 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:31.806 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:31.806 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:29:31.806 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.629231+0000", 2024-09-14T06:29:31.807 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.9179999999999994e-06 2024-09-14T06:29:31.807 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:31.807 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:31.807 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:29:31.807 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.629238+0000", 2024-09-14T06:29:31.807 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.421e-06 2024-09-14T06:29:31.807 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:31.807 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:31.807 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:29:31.807 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.629242+0000", 2024-09-14T06:29:31.807 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.5419999999999999e-06 2024-09-14T06:29:31.807 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:31.807 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:31.807 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:29:31.807 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.650284+0000", 2024-09-14T06:29:31.808 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.021041131000000001 2024-09-14T06:29:31.808 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:31.808 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:31.808 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:29:31.808 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.650361+0000", 2024-09-14T06:29:31.808 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.7787999999999995e-05 2024-09-14T06:29:31.808 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:31.808 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:31.808 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:29:31.808 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.650593+0000", 2024-09-14T06:29:31.808 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00023201 2024-09-14T06:29:31.808 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:31.808 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:29:31.809 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:31.809 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:31.809 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:31.809 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:1502 3.a 3:57c28b0f:::rbd_data.112d7dfbd766.000000000000000a:5 [read 503808~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-14T06:29:31.809 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:08.629249+0000", 2024-09-14T06:29:31.809 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 23.088417549999999, 2024-09-14T06:29:31.809 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.021713217, 2024-09-14T06:29:31.809 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:29:31.809 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:29:31.809 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:29:31.809 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:29:31.809 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:29:31.809 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:31.810 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 1502 2024-09-14T06:29:31.810 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:31.810 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:29:31.810 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:31.810 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:29:31.810 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.629249+0000", 2024-09-14T06:29:31.810 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:31.810 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:31.810 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:31.810 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:29:31.810 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.629249+0000", 2024-09-14T06:29:31.810 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:31.810 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:31.810 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:31.811 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:29:31.811 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.629250+0000", 2024-09-14T06:29:31.811 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.03e-07 2024-09-14T06:29:31.811 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:31.811 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:31.811 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:29:31.811 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.629252+0000", 2024-09-14T06:29:31.811 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.995e-06 2024-09-14T06:29:31.811 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:31.811 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:31.811 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:29:31.812 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.629257+0000", 2024-09-14T06:29:31.812 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.7010000000000001e-06 2024-09-14T06:29:31.812 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:31.812 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:31.812 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:29:31.812 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.629262+0000", 2024-09-14T06:29:31.812 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.16e-06 2024-09-14T06:29:31.812 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:31.812 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:31.812 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:29:31.812 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.650629+0000", 2024-09-14T06:29:31.813 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.021367684000000001 2024-09-14T06:29:31.813 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:31.813 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:31.813 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:29:31.813 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.650718+0000", 2024-09-14T06:29:31.813 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.8832000000000006e-05 2024-09-14T06:29:31.813 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:31.813 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:31.813 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:29:31.813 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.650963+0000", 2024-09-14T06:29:31.813 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00024444199999999999 2024-09-14T06:29:31.813 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:31.813 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:29:31.814 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:31.814 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:31.814 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:31.814 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:1503 3.a 3:57c28b0f:::rbd_data.112d7dfbd766.000000000000000a:5 [read 520192~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-14T06:29:31.814 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:08.629268+0000", 2024-09-14T06:29:31.814 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 23.088399009, 2024-09-14T06:29:31.814 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.022244885999999998, 2024-09-14T06:29:31.814 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:29:31.814 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:29:31.814 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:29:31.814 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:29:31.814 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:29:31.814 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:31.814 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 1503 2024-09-14T06:29:31.815 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:31.815 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:29:31.815 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:31.815 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:29:31.815 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.629268+0000", 2024-09-14T06:29:31.815 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:31.815 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:31.815 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:31.815 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:29:31.815 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.629268+0000", 2024-09-14T06:29:31.815 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:31.815 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:31.815 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:31.815 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:29:31.816 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.629268+0000", 2024-09-14T06:29:31.816 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.6599999999999996e-07 2024-09-14T06:29:31.816 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:31.816 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:31.816 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:29:31.816 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.629270+0000", 2024-09-14T06:29:31.816 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.7129999999999999e-06 2024-09-14T06:29:31.816 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:31.816 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:31.816 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:29:31.816 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.629276+0000", 2024-09-14T06:29:31.816 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.8119999999999996e-06 2024-09-14T06:29:31.817 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:31.817 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:31.817 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:29:31.817 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.629280+0000", 2024-09-14T06:29:31.817 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.4920000000000004e-06 2024-09-14T06:29:31.817 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:31.817 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:31.817 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:29:31.817 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.651039+0000", 2024-09-14T06:29:31.817 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.021758116000000001 2024-09-14T06:29:31.817 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:31.817 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:31.817 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:29:31.818 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.651127+0000", 2024-09-14T06:29:31.818 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.7983000000000001e-05 2024-09-14T06:29:31.818 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:31.818 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:31.818 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:29:31.818 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.651513+0000", 2024-09-14T06:29:31.818 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00038620400000000002 2024-09-14T06:29:31.818 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:31.818 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:29:31.818 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:31.818 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:31.818 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:29:31.818 INFO:teuthology.orchestra.run.smithi138.stdout:} 2024-09-14T06:29:31.820 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-14T06:29:31.820 INFO:tasks.ceph.osd.2.smithi094.stderr:2024-09-14T06:29:31.817+0000 7faf3269b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33649) UID: 0 2024-09-14T06:29:31.823 INFO:teuthology.orchestra.run.smithi094.stdout:checking clone #6, image image_client.0-clone6 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent7 2024-09-14T06:29:31.920 INFO:tasks.ceph.osd.3.smithi094.stderr:2024-09-14T06:29:31.918+0000 7feeeafe8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33655) UID: 0 2024-09-14T06:29:31.924 INFO:teuthology.orchestra.run.smithi138.stdout:{ 2024-09-14T06:29:31.925 INFO:teuthology.orchestra.run.smithi138.stdout: "ops": [], 2024-09-14T06:29:31.925 INFO:teuthology.orchestra.run.smithi138.stdout: "num_ops": 0 2024-09-14T06:29:31.925 INFO:teuthology.orchestra.run.smithi138.stdout:} 2024-09-14T06:29:31.934 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-14T06:29:31.952 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-14T06:29:32.021 INFO:tasks.ceph.osd.1.smithi094.stderr:2024-09-14T06:29:32.019+0000 7f328e47a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33641) UID: 0 2024-09-14T06:29:32.032 INFO:teuthology.orchestra.run.smithi094.stdout:545 write 0xdd5c65b thru 0xdd5e885 (0x222b bytes) 2024-09-14T06:29:32.035 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-14T06:29:32.035 INFO:teuthology.orchestra.run.smithi094.stdout:548 write 0x7c8f890 thru 0x7c9cf51 (0xd6c2 bytes) 2024-09-14T06:29:32.037 INFO:teuthology.orchestra.run.smithi094.stdout:551 writesame 0xac7b817 thru 0xac7cdab data_size 0xd(0x1595 bytes) 2024-09-14T06:29:32.038 INFO:teuthology.orchestra.run.smithi094.stdout:553 read 0x62702a9 thru 0x627c170 (0xbec8 bytes) 2024-09-14T06:29:32.039 INFO:teuthology.orchestra.run.smithi094.stdout:554 punch from 0x43c5a8b to 0x43cc7d9, (0x6d4e bytes) 2024-09-14T06:29:32.039 INFO:teuthology.orchestra.run.smithi094.stdout:555 write 0xc15b6dc thru 0xc167bad (0xc4d2 bytes) 2024-09-14T06:29:32.039 INFO:teuthology.orchestra.run.smithi138.stdout:{ 2024-09-14T06:29:32.039 INFO:teuthology.orchestra.run.smithi138.stdout: "ops": [], 2024-09-14T06:29:32.039 INFO:teuthology.orchestra.run.smithi138.stdout: "complaint_time": 30, 2024-09-14T06:29:32.039 INFO:teuthology.orchestra.run.smithi138.stdout: "num_blocked_ops": 0 2024-09-14T06:29:32.039 INFO:teuthology.orchestra.run.smithi138.stdout:} 2024-09-14T06:29:32.040 INFO:teuthology.orchestra.run.smithi094.stdout:556 write 0x56d6533 thru 0x56e30e6 (0xcbb4 bytes) 2024-09-14T06:29:32.042 INFO:teuthology.orchestra.run.smithi094.stdout:557 punch from 0x7e22f83 to 0x7e2341c, (0x499 bytes) 2024-09-14T06:29:32.042 INFO:teuthology.orchestra.run.smithi094.stdout:560 write 0xb2fe4ab thru 0xb303b91 (0x56e7 bytes) 2024-09-14T06:29:32.043 INFO:teuthology.orchestra.run.smithi094.stdout:562 write 0x30eca98 thru 0x30f0e22 (0x438b bytes) 2024-09-14T06:29:32.044 INFO:teuthology.orchestra.run.smithi094.stdout:564 read 0x6b591b5 thru 0x6b622e0 (0x912c bytes) 2024-09-14T06:29:32.045 INFO:teuthology.orchestra.run.smithi094.stdout:567 read 0xbcf5ea6 thru 0xbcf9cb5 (0x3e10 bytes) 2024-09-14T06:29:32.045 INFO:teuthology.orchestra.run.smithi094.stdout:568 writesame 0xb758491 thru 0xb75e73e data_size 0x11(0x62ae bytes) 2024-09-14T06:29:32.047 INFO:teuthology.orchestra.run.smithi094.stdout:570 read 0x3c3f39 thru 0x3d03fa (0xc4c2 bytes) 2024-09-14T06:29:32.047 INFO:teuthology.orchestra.run.smithi094.stdout:572 punch from 0x432c4df to 0x4332599, (0x60ba bytes) 2024-09-14T06:29:32.048 INFO:teuthology.orchestra.run.smithi094.stdout:573 read 0x9710aca thru 0x971f202 (0xe739 bytes) 2024-09-14T06:29:32.048 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-14T06:29:32.049 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-14T06:29:32.049 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-14T06:29:32.049 INFO:teuthology.orchestra.run.smithi094.stdout:578 writesame 0x29738cb thru 0x298286a data_size 0x6c(0xefa0 bytes) 2024-09-14T06:29:32.049 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-14T06:29:32.050 INFO:teuthology.orchestra.run.smithi094.stdout:579 write 0x1fbe40 thru 0x2041e9 (0x83aa bytes) 2024-09-14T06:29:32.058 INFO:teuthology.orchestra.run.smithi094.stdout:580 write 0xe84af91 thru 0xe850d63 (0x5dd3 bytes) 2024-09-14T06:29:32.061 INFO:teuthology.orchestra.run.smithi094.stdout:582 write 0x240cefc thru 0x2410981 (0x3a86 bytes) 2024-09-14T06:29:32.062 INFO:teuthology.orchestra.run.smithi094.stdout:584 clone 9 order 21 su 16384 sc 10 2024-09-14T06:29:32.078 INFO:teuthology.orchestra.run.smithi094.stdout:osd.0: {} 2024-09-14T06:29:32.079 INFO:teuthology.orchestra.run.smithi094.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-14T06:29:32.095 INFO:teuthology.orchestra.run.smithi094.stdout:osd.1: {} 2024-09-14T06:29:32.095 INFO:teuthology.orchestra.run.smithi094.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-14T06:29:32.111 INFO:teuthology.orchestra.run.smithi094.stdout:osd.2: {} 2024-09-14T06:29:32.112 INFO:teuthology.orchestra.run.smithi094.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-14T06:29:32.122 INFO:tasks.ceph.osd.7.smithi138.stderr:2024-09-14T06:29:32.121+0000 7f3c6a21d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33623) UID: 0 2024-09-14T06:29:32.127 INFO:teuthology.orchestra.run.smithi094.stdout:osd.3: {} 2024-09-14T06:29:32.127 INFO:teuthology.orchestra.run.smithi094.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-14T06:29:32.144 INFO:teuthology.orchestra.run.smithi094.stdout:osd.4: {} 2024-09-14T06:29:32.145 INFO:teuthology.orchestra.run.smithi094.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-14T06:29:32.154 INFO:teuthology.orchestra.run.smithi138.stdout:{ 2024-09-14T06:29:32.155 INFO:teuthology.orchestra.run.smithi138.stdout: "size": 20, 2024-09-14T06:29:32.155 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 600, 2024-09-14T06:29:32.155 INFO:teuthology.orchestra.run.smithi138.stdout: "ops": [ 2024-09-14T06:29:32.155 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:32.155 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:1575 3.15 3:accf422f:::rbd_data.112d7dfbd766.0000000000000001:5 [read 339968~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-14T06:29:32.155 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:08.630645+0000", 2024-09-14T06:29:32.155 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 23.522586388000001, 2024-09-14T06:29:32.155 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.024511650999999999, 2024-09-14T06:29:32.155 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:29:32.155 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:29:32.155 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:29:32.155 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:29:32.155 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:29:32.156 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:32.156 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 1575 2024-09-14T06:29:32.156 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:32.156 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:29:32.156 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:32.156 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:29:32.156 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.630645+0000", 2024-09-14T06:29:32.156 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:32.156 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:32.156 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:32.156 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:29:32.156 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.630645+0000", 2024-09-14T06:29:32.156 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:32.156 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:32.157 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:32.157 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:29:32.157 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.630645+0000", 2024-09-14T06:29:32.157 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.9400000000000005e-07 2024-09-14T06:29:32.157 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:32.157 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:32.157 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:29:32.157 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.630647+0000", 2024-09-14T06:29:32.157 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.9309999999999998e-06 2024-09-14T06:29:32.157 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:32.157 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:32.157 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:29:32.157 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.630654+0000", 2024-09-14T06:29:32.157 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.55e-06 2024-09-14T06:29:32.158 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:32.158 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:32.158 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:29:32.158 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.630659+0000", 2024-09-14T06:29:32.158 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.0139999999999998e-06 2024-09-14T06:29:32.158 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:32.158 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:32.158 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:29:32.158 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.654964+0000", 2024-09-14T06:29:32.158 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.024305651000000001 2024-09-14T06:29:32.158 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:32.158 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:32.158 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:29:32.158 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.655048+0000", 2024-09-14T06:29:32.159 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.3389999999999999e-05 2024-09-14T06:29:32.159 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:32.159 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:32.159 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:29:32.159 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.655156+0000", 2024-09-14T06:29:32.159 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.000108521 2024-09-14T06:29:32.159 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:32.159 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:29:32.159 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:32.159 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:32.159 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:32.159 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:1576 3.15 3:accf422f:::rbd_data.112d7dfbd766.0000000000000001:5 [read 356352~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-14T06:29:32.159 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:08.630666+0000", 2024-09-14T06:29:32.160 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 23.522565350000001, 2024-09-14T06:29:32.160 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.024703712999999999, 2024-09-14T06:29:32.160 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:29:32.160 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:29:32.160 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:29:32.160 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:29:32.160 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:29:32.160 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:32.160 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 1576 2024-09-14T06:29:32.160 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:32.160 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:29:32.160 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:32.160 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:29:32.160 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.630666+0000", 2024-09-14T06:29:32.161 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:32.161 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:32.161 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:32.161 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:29:32.161 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.630666+0000", 2024-09-14T06:29:32.161 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:32.161 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:32.161 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:32.161 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:29:32.161 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.630666+0000", 2024-09-14T06:29:32.161 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.8299999999999997e-07 2024-09-14T06:29:32.161 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:32.161 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:32.162 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:29:32.162 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.630668+0000", 2024-09-14T06:29:32.162 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.8789999999999999e-06 2024-09-14T06:29:32.162 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:32.162 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:32.162 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:29:32.162 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.630675+0000", 2024-09-14T06:29:32.162 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.2899999999999997e-06 2024-09-14T06:29:32.162 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:32.162 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:32.162 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:29:32.162 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.630681+0000", 2024-09-14T06:29:32.162 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.2730000000000002e-06 2024-09-14T06:29:32.162 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:32.162 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:32.162 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:29:32.163 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.655182+0000", 2024-09-14T06:29:32.163 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.024501228999999999 2024-09-14T06:29:32.163 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:32.163 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:32.163 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:29:32.163 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.655257+0000", 2024-09-14T06:29:32.163 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.4954000000000001e-05 2024-09-14T06:29:32.163 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:32.163 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:32.163 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:29:32.163 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.655369+0000", 2024-09-14T06:29:32.163 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.000112605 2024-09-14T06:29:32.163 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:32.163 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:29:32.164 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:32.164 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:32.164 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:32.164 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:1577 3.15 3:accf422f:::rbd_data.112d7dfbd766.0000000000000001:5 [read 372736~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-14T06:29:32.164 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:08.630688+0000", 2024-09-14T06:29:32.164 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 23.522542838, 2024-09-14T06:29:32.164 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.024927538999999999, 2024-09-14T06:29:32.164 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:29:32.164 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:29:32.164 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:29:32.164 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:29:32.164 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:29:32.164 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:32.164 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 1577 2024-09-14T06:29:32.165 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:32.165 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:29:32.165 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:32.165 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:29:32.165 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.630688+0000", 2024-09-14T06:29:32.165 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:32.165 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:32.165 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:32.165 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:29:32.165 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.630688+0000", 2024-09-14T06:29:32.165 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:32.165 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:32.165 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:32.165 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:29:32.166 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.630689+0000", 2024-09-14T06:29:32.166 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.2199999999999999e-07 2024-09-14T06:29:32.166 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:32.166 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:32.166 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:29:32.166 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.630691+0000", 2024-09-14T06:29:32.166 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.9410000000000001e-06 2024-09-14T06:29:32.166 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:32.166 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:32.166 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:29:32.166 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.630699+0000", 2024-09-14T06:29:32.166 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.0250000000000002e-06 2024-09-14T06:29:32.166 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:32.166 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:32.166 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:29:32.167 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.630704+0000", 2024-09-14T06:29:32.167 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.4349999999999996e-06 2024-09-14T06:29:32.167 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:32.167 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:32.167 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:29:32.167 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.655406+0000", 2024-09-14T06:29:32.167 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.024702279000000001 2024-09-14T06:29:32.167 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:32.167 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:32.167 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:29:32.167 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.655483+0000", 2024-09-14T06:29:32.167 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.6915999999999993e-05 2024-09-14T06:29:32.167 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:32.167 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:32.168 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:29:32.168 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.655616+0000", 2024-09-14T06:29:32.168 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.000132521 2024-09-14T06:29:32.168 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:32.168 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:29:32.168 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:32.168 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:32.168 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:32.168 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:1578 3.15 3:accf422f:::rbd_data.112d7dfbd766.0000000000000001:5 [read 389120~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-14T06:29:32.168 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:08.630711+0000", 2024-09-14T06:29:32.168 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 23.522520365999998, 2024-09-14T06:29:32.168 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.025111108, 2024-09-14T06:29:32.168 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:29:32.168 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:29:32.169 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:29:32.169 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:29:32.169 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:29:32.169 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:32.169 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 1578 2024-09-14T06:29:32.169 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:32.169 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:29:32.169 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:32.169 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:29:32.169 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.630711+0000", 2024-09-14T06:29:32.169 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:32.169 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:32.169 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:32.170 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:29:32.170 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.630711+0000", 2024-09-14T06:29:32.170 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:32.170 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:32.170 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:32.170 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:29:32.170 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.630711+0000", 2024-09-14T06:29:32.170 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.01e-07 2024-09-14T06:29:32.170 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:32.170 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:32.170 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:29:32.170 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.630713+0000", 2024-09-14T06:29:32.170 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.8199999999999999e-06 2024-09-14T06:29:32.170 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:32.171 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:32.171 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:29:32.171 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.630720+0000", 2024-09-14T06:29:32.171 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.3880000000000002e-06 2024-09-14T06:29:32.171 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:32.171 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:32.171 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:29:32.171 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.630726+0000", 2024-09-14T06:29:32.171 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.3480000000000003e-06 2024-09-14T06:29:32.171 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:32.171 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:32.171 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:29:32.171 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.655646+0000", 2024-09-14T06:29:32.171 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.024920312999999999 2024-09-14T06:29:32.172 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:32.172 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:32.172 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:29:32.172 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.655724+0000", 2024-09-14T06:29:32.172 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.7589000000000005e-05 2024-09-14T06:29:32.172 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:32.172 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:32.172 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:29:32.172 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.655822+0000", 2024-09-14T06:29:32.172 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.8249000000000001e-05 2024-09-14T06:29:32.172 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:32.172 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:29:32.172 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:32.172 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:32.173 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:32.173 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:1579 3.15 3:accf422f:::rbd_data.112d7dfbd766.0000000000000001:5 [read 405504~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-14T06:29:32.173 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:08.630732+0000", 2024-09-14T06:29:32.173 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 23.522499048, 2024-09-14T06:29:32.173 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.025303169, 2024-09-14T06:29:32.173 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:29:32.173 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:29:32.173 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:29:32.173 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:29:32.173 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:29:32.173 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:32.173 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 1579 2024-09-14T06:29:32.173 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:32.174 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:29:32.174 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:32.174 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:29:32.174 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.630732+0000", 2024-09-14T06:29:32.174 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:32.174 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:32.174 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:32.174 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:29:32.174 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.630732+0000", 2024-09-14T06:29:32.174 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:32.174 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:32.174 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:32.174 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:29:32.175 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.630732+0000", 2024-09-14T06:29:32.175 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.5199999999999998e-07 2024-09-14T06:29:32.175 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:32.175 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:32.175 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:29:32.175 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.630741+0000", 2024-09-14T06:29:32.175 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.1549999999999996e-06 2024-09-14T06:29:32.175 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:32.175 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:32.175 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:29:32.175 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.630748+0000", 2024-09-14T06:29:32.175 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.6429999999999998e-06 2024-09-14T06:29:32.175 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:32.176 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:32.176 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:29:32.176 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.630753+0000", 2024-09-14T06:29:32.176 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.0579999999999997e-06 2024-09-14T06:29:32.176 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:32.176 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:32.176 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:29:32.176 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.655858+0000", 2024-09-14T06:29:32.176 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.025104774999999999 2024-09-14T06:29:32.176 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:32.176 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:32.176 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:29:32.176 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.655942+0000", 2024-09-14T06:29:32.176 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.4129999999999996e-05 2024-09-14T06:29:32.177 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:32.177 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:32.177 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:29:32.177 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.656035+0000", 2024-09-14T06:29:32.177 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.3055999999999998e-05 2024-09-14T06:29:32.177 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:32.177 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:29:32.177 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:32.177 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:32.177 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:32.177 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:1580 3.15 3:accf422f:::rbd_data.112d7dfbd766.0000000000000001:5 [read 421888~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-14T06:29:32.177 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:08.630762+0000", 2024-09-14T06:29:32.177 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 23.522469269999998, 2024-09-14T06:29:32.177 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.025487388999999999, 2024-09-14T06:29:32.178 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:29:32.178 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:29:32.178 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:29:32.178 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:29:32.178 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:29:32.178 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:32.178 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 1580 2024-09-14T06:29:32.178 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:32.178 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:29:32.178 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:32.178 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:29:32.178 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.630762+0000", 2024-09-14T06:29:32.178 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:32.178 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:32.178 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:32.178 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:29:32.179 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.630762+0000", 2024-09-14T06:29:32.179 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:32.179 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:32.179 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:32.179 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:29:32.179 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.630762+0000", 2024-09-14T06:29:32.179 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.2300000000000002e-07 2024-09-14T06:29:32.179 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:32.179 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:32.179 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:29:32.179 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.630764+0000", 2024-09-14T06:29:32.179 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.99e-06 2024-09-14T06:29:32.179 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:32.179 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:32.179 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:29:32.179 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.630771+0000", 2024-09-14T06:29:32.180 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.9349999999999997e-06 2024-09-14T06:29:32.180 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:32.180 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:32.180 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:29:32.180 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.630776+0000", 2024-09-14T06:29:32.180 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.1710000000000001e-06 2024-09-14T06:29:32.180 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:32.180 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:32.180 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:29:32.180 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.656068+0000", 2024-09-14T06:29:32.180 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.025291905999999999 2024-09-14T06:29:32.180 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:32.180 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:32.180 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:29:32.181 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.656151+0000", 2024-09-14T06:29:32.181 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.3209999999999998e-05 2024-09-14T06:29:32.181 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:32.181 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:32.181 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:29:32.181 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.656249+0000", 2024-09-14T06:29:32.181 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.7754e-05 2024-09-14T06:29:32.181 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:32.181 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:29:32.181 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:32.181 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:32.181 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:32.181 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:1581 3.15 3:accf422f:::rbd_data.112d7dfbd766.0000000000000001:5 [read 438272~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-14T06:29:32.181 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:08.630802+0000", 2024-09-14T06:29:32.181 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 23.522428619999999, 2024-09-14T06:29:32.182 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.025670063, 2024-09-14T06:29:32.182 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:29:32.182 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:29:32.182 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:29:32.182 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:29:32.182 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:29:32.182 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:32.182 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 1581 2024-09-14T06:29:32.182 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:32.182 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:29:32.182 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:32.182 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:29:32.182 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.630802+0000", 2024-09-14T06:29:32.182 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:32.182 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:32.183 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:32.183 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:29:32.183 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.630802+0000", 2024-09-14T06:29:32.183 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:32.183 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:32.183 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:32.183 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:29:32.183 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.630803+0000", 2024-09-14T06:29:32.183 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.3099999999999997e-07 2024-09-14T06:29:32.183 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:32.183 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:32.183 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:29:32.183 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.630806+0000", 2024-09-14T06:29:32.183 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.7319999999999998e-06 2024-09-14T06:29:32.183 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:32.183 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:32.184 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:29:32.184 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.630813+0000", 2024-09-14T06:29:32.184 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.0840000000000003e-06 2024-09-14T06:29:32.184 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:32.184 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:32.184 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:29:32.184 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.630826+0000", 2024-09-14T06:29:32.184 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.3104e-05 2024-09-14T06:29:32.184 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:32.184 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:32.184 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:29:32.184 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.656349+0000", 2024-09-14T06:29:32.184 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.025523113 2024-09-14T06:29:32.184 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:32.185 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:32.185 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:29:32.185 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.656400+0000", 2024-09-14T06:29:32.185 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.1156000000000003e-05 2024-09-14T06:29:32.185 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:32.185 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:32.185 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:29:32.185 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.656472+0000", 2024-09-14T06:29:32.185 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.2243000000000006e-05 2024-09-14T06:29:32.185 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:32.185 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:29:32.185 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:32.186 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:32.186 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:32.186 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:1582 3.15 3:accf422f:::rbd_data.112d7dfbd766.0000000000000001:5 [read 454656~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-14T06:29:32.186 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:08.630838+0000", 2024-09-14T06:29:32.186 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 23.522392596, 2024-09-14T06:29:32.186 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.025956396999999999, 2024-09-14T06:29:32.186 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:29:32.186 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:29:32.186 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:29:32.186 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:29:32.186 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:29:32.186 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:32.186 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 1582 2024-09-14T06:29:32.186 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:32.186 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:29:32.187 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:32.187 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:29:32.187 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.630838+0000", 2024-09-14T06:29:32.187 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:32.187 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:32.187 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:32.187 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:29:32.187 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.630838+0000", 2024-09-14T06:29:32.187 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:32.187 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:32.187 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:32.187 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:29:32.187 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.630839+0000", 2024-09-14T06:29:32.187 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.8400000000000004e-07 2024-09-14T06:29:32.188 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:32.188 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:32.188 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:29:32.188 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.630846+0000", 2024-09-14T06:29:32.188 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.4599999999999997e-06 2024-09-14T06:29:32.188 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:32.188 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:32.188 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:29:32.188 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.630854+0000", 2024-09-14T06:29:32.188 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.3139999999999998e-06 2024-09-14T06:29:32.188 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:32.188 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:32.188 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:29:32.188 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.630865+0000", 2024-09-14T06:29:32.189 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.0988999999999999e-05 2024-09-14T06:29:32.189 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:32.189 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:32.189 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:29:32.189 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.656598+0000", 2024-09-14T06:29:32.189 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.025733335 2024-09-14T06:29:32.189 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:32.189 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:32.189 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:29:32.189 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.656682+0000", 2024-09-14T06:29:32.189 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.4000999999999998e-05 2024-09-14T06:29:32.189 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:32.189 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:32.189 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:29:32.190 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.656795+0000", 2024-09-14T06:29:32.190 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00011271400000000001 2024-09-14T06:29:32.190 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:32.190 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:29:32.190 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:32.190 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:32.190 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:32.190 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:1583 3.15 3:accf422f:::rbd_data.112d7dfbd766.0000000000000001:5 [read 471040~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-14T06:29:32.190 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:08.630873+0000", 2024-09-14T06:29:32.190 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 23.522357727999999, 2024-09-14T06:29:32.190 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.026337467, 2024-09-14T06:29:32.190 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:29:32.191 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:29:32.191 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:29:32.191 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:29:32.191 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:29:32.191 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:32.191 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 1583 2024-09-14T06:29:32.191 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:32.191 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:29:32.191 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:32.191 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:29:32.191 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.630873+0000", 2024-09-14T06:29:32.191 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:32.191 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:32.192 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:32.192 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:29:32.192 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.630873+0000", 2024-09-14T06:29:32.192 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:32.192 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:32.192 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:32.192 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:29:32.192 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.630874+0000", 2024-09-14T06:29:32.192 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.2500000000000005e-07 2024-09-14T06:29:32.192 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:32.192 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:32.192 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:29:32.192 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.630876+0000", 2024-09-14T06:29:32.193 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.2689999999999998e-06 2024-09-14T06:29:32.193 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:32.193 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:32.193 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:29:32.193 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.630883+0000", 2024-09-14T06:29:32.193 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.1919999999999999e-06 2024-09-14T06:29:32.193 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:32.193 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:32.193 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:29:32.193 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.630891+0000", 2024-09-14T06:29:32.193 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.1579999999999999e-06 2024-09-14T06:29:32.193 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:32.194 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:32.194 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:29:32.194 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.656836+0000", 2024-09-14T06:29:32.194 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.025945134000000002 2024-09-14T06:29:32.194 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:32.194 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:32.194 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:29:32.194 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.656920+0000", 2024-09-14T06:29:32.194 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.4351999999999995e-05 2024-09-14T06:29:32.194 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:32.194 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:32.194 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:29:32.195 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.657211+0000", 2024-09-14T06:29:32.195 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.000290737 2024-09-14T06:29:32.195 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:32.195 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:29:32.195 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:32.195 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:32.195 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:32.195 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:1584 3.15 3:accf422f:::rbd_data.112d7dfbd766.0000000000000001:5 [read 487424~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-14T06:29:32.195 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:08.630897+0000", 2024-09-14T06:29:32.195 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 23.522333631999999, 2024-09-14T06:29:32.195 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.026720516, 2024-09-14T06:29:32.195 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:29:32.195 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:29:32.195 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:29:32.196 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:29:32.196 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:29:32.196 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:32.196 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 1584 2024-09-14T06:29:32.196 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:32.196 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:29:32.196 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:32.196 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:29:32.196 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.630897+0000", 2024-09-14T06:29:32.196 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:32.196 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:32.196 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:32.196 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:29:32.196 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.630897+0000", 2024-09-14T06:29:32.197 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:32.197 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:32.197 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:32.197 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:29:32.197 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.630898+0000", 2024-09-14T06:29:32.197 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.9200000000000002e-07 2024-09-14T06:29:32.197 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:32.197 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:32.197 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:29:32.197 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.630900+0000", 2024-09-14T06:29:32.197 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.7269999999999999e-06 2024-09-14T06:29:32.197 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:32.197 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:32.197 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:29:32.198 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.630906+0000", 2024-09-14T06:29:32.198 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.5849999999999996e-06 2024-09-14T06:29:32.198 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:32.198 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:32.198 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:29:32.198 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.630919+0000", 2024-09-14T06:29:32.198 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.3098999999999999e-05 2024-09-14T06:29:32.198 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:32.198 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:32.198 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:29:32.198 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.657253+0000", 2024-09-14T06:29:32.198 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.026333895999999999 2024-09-14T06:29:32.198 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:32.198 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:32.199 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:29:32.199 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.657342+0000", 2024-09-14T06:29:32.199 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.9044999999999999e-05 2024-09-14T06:29:32.199 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:32.199 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:32.199 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:29:32.199 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.657618+0000", 2024-09-14T06:29:32.199 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00027577199999999997 2024-09-14T06:29:32.199 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:32.199 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:29:32.199 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:32.199 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:32.199 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:32.200 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:1585 3.15 3:accf422f:::rbd_data.112d7dfbd766.0000000000000001:5 [read 503808~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-14T06:29:32.200 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:08.630927+0000", 2024-09-14T06:29:32.200 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 23.52230363, 2024-09-14T06:29:32.200 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.027110801, 2024-09-14T06:29:32.200 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:29:32.200 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:29:32.200 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:29:32.200 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:29:32.200 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:29:32.200 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:32.200 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 1585 2024-09-14T06:29:32.200 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:32.200 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:29:32.201 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:32.201 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:29:32.201 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.630927+0000", 2024-09-14T06:29:32.201 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:32.201 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:32.201 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:32.201 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:29:32.201 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.630927+0000", 2024-09-14T06:29:32.201 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:32.201 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:32.201 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:32.201 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:29:32.201 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.630928+0000", 2024-09-14T06:29:32.201 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.7800000000000001e-07 2024-09-14T06:29:32.201 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:32.202 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:32.202 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:29:32.202 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.630930+0000", 2024-09-14T06:29:32.202 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.0829999999999998e-06 2024-09-14T06:29:32.202 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:32.202 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:32.202 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:29:32.202 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.630937+0000", 2024-09-14T06:29:32.202 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.7179999999999999e-06 2024-09-14T06:29:32.202 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:32.202 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:32.202 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:29:32.202 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.630942+0000", 2024-09-14T06:29:32.202 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.2989999999999996e-06 2024-09-14T06:29:32.202 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:32.203 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:32.203 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:29:32.203 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.657673+0000", 2024-09-14T06:29:32.203 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.026731390000000001 2024-09-14T06:29:32.203 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:32.203 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:32.203 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:29:32.203 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.657754+0000", 2024-09-14T06:29:32.203 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.0580000000000004e-05 2024-09-14T06:29:32.203 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:32.203 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:32.203 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:29:32.203 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.658038+0000", 2024-09-14T06:29:32.203 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00028415299999999999 2024-09-14T06:29:32.204 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:32.204 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:29:32.204 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:32.204 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:32.204 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:32.204 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:17151 3.11 3:8b9b9188:::rbd_data.112d92027590.0000000000000017:f [read 74752~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:29:32.204 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:26.344466+0000", 2024-09-14T06:29:32.204 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 5.8087654530000004, 2024-09-14T06:29:32.204 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.024431092000000001, 2024-09-14T06:29:32.204 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:29:32.204 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:29:32.205 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:29:32.205 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:29:32.205 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:29:32.205 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:32.205 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 17151 2024-09-14T06:29:32.205 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:32.205 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:29:32.205 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:32.205 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:29:32.205 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:26.344466+0000", 2024-09-14T06:29:32.205 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:32.205 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:32.205 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:32.205 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:29:32.206 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:26.344466+0000", 2024-09-14T06:29:32.206 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:32.206 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:32.206 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:32.206 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:29:32.206 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:26.344466+0000", 2024-09-14T06:29:32.206 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.2099999999999998e-07 2024-09-14T06:29:32.206 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:32.206 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:32.206 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:29:32.206 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:26.344468+0000", 2024-09-14T06:29:32.206 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.793e-06 2024-09-14T06:29:32.206 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:32.206 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:32.206 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:29:32.207 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:26.344473+0000", 2024-09-14T06:29:32.207 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.4559999999999999e-06 2024-09-14T06:29:32.207 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:32.207 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:32.207 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:29:32.207 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:26.344480+0000", 2024-09-14T06:29:32.207 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.5339999999999996e-06 2024-09-14T06:29:32.207 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:32.207 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:32.207 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:29:32.207 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:26.368772+0000", 2024-09-14T06:29:32.207 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.024292218000000001 2024-09-14T06:29:32.207 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:32.207 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:32.207 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:29:32.208 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:26.368821+0000", 2024-09-14T06:29:32.208 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.8819000000000001e-05 2024-09-14T06:29:32.208 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:32.208 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:32.208 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:29:32.208 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:26.368897+0000", 2024-09-14T06:29:32.208 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.5951000000000005e-05 2024-09-14T06:29:32.208 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:32.208 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:29:32.208 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:32.208 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:32.208 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:32.208 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:17152 3.11 3:8b9b9188:::rbd_data.112d92027590.0000000000000017:f [read 84992~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:29:32.208 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:26.344487+0000", 2024-09-14T06:29:32.209 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 5.8087436950000004, 2024-09-14T06:29:32.209 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.024548099, 2024-09-14T06:29:32.209 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:29:32.209 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:29:32.209 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:29:32.209 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:29:32.209 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:29:32.209 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:32.209 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 17152 2024-09-14T06:29:32.209 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:32.209 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:29:32.209 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:32.209 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:29:32.209 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:26.344487+0000", 2024-09-14T06:29:32.209 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:32.210 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:32.210 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:32.210 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:29:32.210 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:26.344487+0000", 2024-09-14T06:29:32.210 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:32.210 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:32.210 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:32.210 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:29:32.210 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:26.344488+0000", 2024-09-14T06:29:32.210 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.3700000000000001e-07 2024-09-14T06:29:32.210 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:32.210 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:32.210 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:29:32.210 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:26.344490+0000", 2024-09-14T06:29:32.211 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.9800000000000001e-06 2024-09-14T06:29:32.211 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:32.211 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:32.211 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:29:32.211 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:26.344495+0000", 2024-09-14T06:29:32.211 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.6609999999999997e-06 2024-09-14T06:29:32.211 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:32.211 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:32.211 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:29:32.211 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:26.344502+0000", 2024-09-14T06:29:32.211 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.618e-06 2024-09-14T06:29:32.211 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:32.211 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:32.211 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:29:32.212 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:26.368935+0000", 2024-09-14T06:29:32.212 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.024433264999999999 2024-09-14T06:29:32.212 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:32.212 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:32.212 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:29:32.212 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:26.368979+0000", 2024-09-14T06:29:32.212 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.3949999999999998e-05 2024-09-14T06:29:32.212 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:32.212 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:32.212 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:29:32.212 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:26.369035+0000", 2024-09-14T06:29:32.212 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.6288e-05 2024-09-14T06:29:32.212 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:32.212 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:29:32.213 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:32.213 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:32.213 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:32.213 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:17153 3.11 3:8b9b9188:::rbd_data.112d92027590.0000000000000017:f [read 95232~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:29:32.213 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:26.344516+0000", 2024-09-14T06:29:32.213 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 5.8087154200000004, 2024-09-14T06:29:32.213 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.024640502000000002, 2024-09-14T06:29:32.213 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:29:32.213 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:29:32.213 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:29:32.213 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:29:32.213 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:29:32.213 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:32.213 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 17153 2024-09-14T06:29:32.214 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:32.214 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:29:32.214 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:32.214 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:29:32.214 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:26.344516+0000", 2024-09-14T06:29:32.214 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:32.214 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:32.214 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:32.214 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:29:32.214 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:26.344516+0000", 2024-09-14T06:29:32.214 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:32.214 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:32.214 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:32.214 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:29:32.214 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:26.344516+0000", 2024-09-14T06:29:32.215 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.9000000000000002e-07 2024-09-14T06:29:32.215 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:32.215 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:32.215 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:29:32.215 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:26.344518+0000", 2024-09-14T06:29:32.215 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.2230000000000001e-06 2024-09-14T06:29:32.215 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:32.215 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:32.215 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:29:32.215 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:26.344525+0000", 2024-09-14T06:29:32.215 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.1099999999999997e-06 2024-09-14T06:29:32.215 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:32.215 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:32.216 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:29:32.216 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:26.344531+0000", 2024-09-14T06:29:32.216 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.1430000000000003e-06 2024-09-14T06:29:32.216 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:32.216 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:32.216 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:29:32.216 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:26.369062+0000", 2024-09-14T06:29:32.216 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.024530086999999999 2024-09-14T06:29:32.216 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:32.216 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:32.216 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:29:32.216 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:26.369106+0000", 2024-09-14T06:29:32.216 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.4700000000000002e-05 2024-09-14T06:29:32.216 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:32.217 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:32.217 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:29:32.217 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:26.369156+0000", 2024-09-14T06:29:32.217 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.9849000000000003e-05 2024-09-14T06:29:32.217 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:32.217 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:29:32.217 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:32.217 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:32.217 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:32.217 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:17154 3.11 3:8b9b9188:::rbd_data.112d92027590.0000000000000017:f [read 105472~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:29:32.217 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:26.344556+0000", 2024-09-14T06:29:32.217 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 5.80867501, 2024-09-14T06:29:32.217 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.024718575, 2024-09-14T06:29:32.217 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:29:32.218 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:29:32.218 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:29:32.218 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:29:32.218 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:29:32.218 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:32.218 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 17154 2024-09-14T06:29:32.218 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:32.218 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:29:32.218 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:32.218 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:29:32.218 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:26.344556+0000", 2024-09-14T06:29:32.218 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:32.218 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:32.218 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:32.218 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:29:32.219 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:26.344556+0000", 2024-09-14T06:29:32.219 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:32.219 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:32.219 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:32.219 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:29:32.219 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:26.344556+0000", 2024-09-14T06:29:32.219 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.1600000000000002e-07 2024-09-14T06:29:32.219 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:32.219 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:32.219 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:29:32.219 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:26.344558+0000", 2024-09-14T06:29:32.219 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.514e-06 2024-09-14T06:29:32.219 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:32.220 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:32.220 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:29:32.220 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:26.344564+0000", 2024-09-14T06:29:32.220 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.7960000000000001e-06 2024-09-14T06:29:32.220 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:32.220 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:32.220 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:29:32.220 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:26.344568+0000", 2024-09-14T06:29:32.220 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.1150000000000004e-06 2024-09-14T06:29:32.220 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:32.220 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:32.220 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:29:32.220 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:26.369177+0000", 2024-09-14T06:29:32.220 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.024609529000000002 2024-09-14T06:29:32.221 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:32.221 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:32.221 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:29:32.221 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:26.369217+0000", 2024-09-14T06:29:32.221 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.9459000000000003e-05 2024-09-14T06:29:32.221 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:32.221 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:32.221 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:29:32.221 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:26.369275+0000", 2024-09-14T06:29:32.221 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.7745999999999999e-05 2024-09-14T06:29:32.221 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:32.221 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:29:32.221 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:32.221 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:32.222 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:32.222 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:17155 3.11 3:8b9b9188:::rbd_data.112d92027590.0000000000000017:f [read 115712~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:29:32.222 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:26.344575+0000", 2024-09-14T06:29:32.222 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 5.8086562659999998, 2024-09-14T06:29:32.222 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.024838444000000001, 2024-09-14T06:29:32.222 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:29:32.222 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:29:32.222 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:29:32.222 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:29:32.222 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:29:32.222 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:32.222 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 17155 2024-09-14T06:29:32.222 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:32.222 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:29:32.223 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:32.223 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:29:32.223 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:26.344575+0000", 2024-09-14T06:29:32.223 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:32.223 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:32.223 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:32.223 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:29:32.223 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:26.344575+0000", 2024-09-14T06:29:32.223 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:32.223 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:32.223 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:32.223 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:29:32.223 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:26.344575+0000", 2024-09-14T06:29:32.224 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.4600000000000001e-07 2024-09-14T06:29:32.224 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:32.224 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:32.224 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:29:32.224 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:26.344576+0000", 2024-09-14T06:29:32.224 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.1039999999999999e-06 2024-09-14T06:29:32.224 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:32.224 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:32.224 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:29:32.224 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:26.344579+0000", 2024-09-14T06:29:32.224 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.207e-06 2024-09-14T06:29:32.224 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:32.224 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:32.225 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:29:32.225 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:26.344582+0000", 2024-09-14T06:29:32.225 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.9239999999999999e-06 2024-09-14T06:29:32.225 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:32.225 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:32.225 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:29:32.225 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:26.369311+0000", 2024-09-14T06:29:32.225 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.024728488999999999 2024-09-14T06:29:32.225 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:32.225 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:32.225 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:29:32.225 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:26.369357+0000", 2024-09-14T06:29:32.225 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.5837999999999997e-05 2024-09-14T06:29:32.226 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:32.226 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:32.226 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:29:32.226 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:26.369413+0000", 2024-09-14T06:29:32.226 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.6635999999999998e-05 2024-09-14T06:29:32.226 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:32.226 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:29:32.226 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:32.226 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:32.226 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:32.226 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:17156 3.11 3:8b9b9188:::rbd_data.112d92027590.0000000000000017:f [read 125952~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:29:32.226 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:26.344585+0000", 2024-09-14T06:29:32.226 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 5.8086455470000002, 2024-09-14T06:29:32.226 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.024929532000000001, 2024-09-14T06:29:32.227 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:29:32.227 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:29:32.227 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:29:32.227 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:29:32.227 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:29:32.227 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:32.227 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 17156 2024-09-14T06:29:32.227 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:32.227 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:29:32.227 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:32.227 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:29:32.227 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:26.344585+0000", 2024-09-14T06:29:32.227 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:32.227 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:32.228 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:32.228 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:29:32.228 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:26.344585+0000", 2024-09-14T06:29:32.228 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:32.228 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:32.228 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:32.228 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:29:32.228 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:26.344586+0000", 2024-09-14T06:29:32.228 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.0599999999999999e-07 2024-09-14T06:29:32.228 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:32.228 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:32.228 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:29:32.228 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:26.344587+0000", 2024-09-14T06:29:32.228 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.0249999999999999e-06 2024-09-14T06:29:32.229 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:32.229 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:32.229 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:29:32.229 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:26.344590+0000", 2024-09-14T06:29:32.229 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.0910000000000001e-06 2024-09-14T06:29:32.229 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:32.229 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:32.229 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:29:32.229 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:26.344593+0000", 2024-09-14T06:29:32.229 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.4199999999999999e-06 2024-09-14T06:29:32.229 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:32.229 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:32.229 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:29:32.229 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:26.369428+0000", 2024-09-14T06:29:32.230 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.024834927 2024-09-14T06:29:32.230 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:32.230 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:32.230 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:29:32.230 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:26.369465+0000", 2024-09-14T06:29:32.230 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.7092999999999999e-05 2024-09-14T06:29:32.230 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:32.230 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:32.230 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:29:32.230 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:26.369515+0000", 2024-09-14T06:29:32.230 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.977e-05 2024-09-14T06:29:32.230 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:32.230 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:29:32.230 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:32.231 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:32.231 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:32.231 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:17157 3.11 3:8b9b9188:::rbd_data.112d92027590.0000000000000017:f [read 136192~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:29:32.231 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:26.344596+0000", 2024-09-14T06:29:32.231 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 5.8086347260000002, 2024-09-14T06:29:32.231 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.025164066999999998, 2024-09-14T06:29:32.231 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:29:32.231 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:29:32.231 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:29:32.231 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:29:32.231 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:29:32.231 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:32.231 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 17157 2024-09-14T06:29:32.231 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:32.232 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:29:32.232 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:32.232 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:29:32.232 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:26.344596+0000", 2024-09-14T06:29:32.232 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:32.232 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:32.232 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:32.232 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:29:32.232 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:26.344596+0000", 2024-09-14T06:29:32.232 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:32.232 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:32.232 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:32.232 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:29:32.232 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:26.344597+0000", 2024-09-14T06:29:32.232 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.0599999999999999e-07 2024-09-14T06:29:32.233 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:32.233 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:32.233 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:29:32.233 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:26.344598+0000", 2024-09-14T06:29:32.233 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.145e-06 2024-09-14T06:29:32.233 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:32.233 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:32.233 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:29:32.233 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:26.344601+0000", 2024-09-14T06:29:32.233 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.9809999999999999e-06 2024-09-14T06:29:32.233 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:32.233 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:32.233 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:29:32.233 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:26.344604+0000", 2024-09-14T06:29:32.234 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.0750000000000002e-06 2024-09-14T06:29:32.234 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:32.234 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:32.234 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:29:32.234 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:26.369571+0000", 2024-09-14T06:29:32.234 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.024967728000000002 2024-09-14T06:29:32.234 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:32.234 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:32.234 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:29:32.234 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:26.369653+0000", 2024-09-14T06:29:32.234 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.1312999999999999e-05 2024-09-14T06:29:32.234 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:32.234 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:32.234 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:29:32.235 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:26.369760+0000", 2024-09-14T06:29:32.235 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.000107619 2024-09-14T06:29:32.235 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:32.235 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:29:32.235 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:32.235 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:32.235 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:32.235 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:17158 3.11 3:8b9b9188:::rbd_data.112d92027590.0000000000000017:f [read 146432~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:29:32.235 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:26.344607+0000", 2024-09-14T06:29:32.235 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 5.8086243299999998, 2024-09-14T06:29:32.235 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.025345412000000001, 2024-09-14T06:29:32.235 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:29:32.235 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:29:32.235 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:29:32.236 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:29:32.236 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:29:32.236 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:32.236 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 17158 2024-09-14T06:29:32.236 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:32.236 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:29:32.236 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:32.236 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:29:32.236 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:26.344607+0000", 2024-09-14T06:29:32.236 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:32.236 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:32.236 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:32.236 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:29:32.236 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:26.344607+0000", 2024-09-14T06:29:32.237 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:32.237 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:32.237 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:32.237 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:29:32.237 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:26.344607+0000", 2024-09-14T06:29:32.237 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.05e-07 2024-09-14T06:29:32.237 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:32.237 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:32.237 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:29:32.237 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:26.344609+0000", 2024-09-14T06:29:32.237 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.734e-06 2024-09-14T06:29:32.237 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:32.237 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:32.237 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:29:32.237 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:26.344613+0000", 2024-09-14T06:29:32.238 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.6430000000000003e-06 2024-09-14T06:29:32.238 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:32.238 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:32.238 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:29:32.238 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:26.344616+0000", 2024-09-14T06:29:32.238 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.5799999999999999e-06 2024-09-14T06:29:32.238 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:32.238 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:32.238 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:29:32.238 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:26.369805+0000", 2024-09-14T06:29:32.238 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.025189238999999999 2024-09-14T06:29:32.238 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:32.238 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:32.238 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:29:32.238 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:26.369877+0000", 2024-09-14T06:29:32.238 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.1717000000000005e-05 2024-09-14T06:29:32.239 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:32.239 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:32.239 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:29:32.239 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:26.369952+0000", 2024-09-14T06:29:32.239 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.5294000000000001e-05 2024-09-14T06:29:32.239 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:32.239 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:29:32.239 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:32.239 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:32.239 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:32.239 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:17159 3.11 3:8b9b9188:::rbd_data.112d92027590.0000000000000017:f [read 156672~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:29:32.239 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:26.344619+0000", 2024-09-14T06:29:32.239 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 5.8086121259999999, 2024-09-14T06:29:32.239 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.025592119999999999, 2024-09-14T06:29:32.240 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:29:32.240 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:29:32.240 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:29:32.240 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:29:32.240 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:29:32.240 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:32.240 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 17159 2024-09-14T06:29:32.240 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:32.240 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:29:32.240 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:32.240 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:29:32.240 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:26.344619+0000", 2024-09-14T06:29:32.240 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:32.240 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:32.240 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:32.241 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:29:32.241 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:26.344619+0000", 2024-09-14T06:29:32.241 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:32.241 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:32.241 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:32.241 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:29:32.241 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:26.344619+0000", 2024-09-14T06:29:32.241 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.41e-07 2024-09-14T06:29:32.241 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:32.241 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:32.241 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:29:32.241 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:26.344620+0000", 2024-09-14T06:29:32.241 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.113e-06 2024-09-14T06:29:32.241 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:32.242 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:32.242 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:29:32.242 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:26.344623+0000", 2024-09-14T06:29:32.242 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.072e-06 2024-09-14T06:29:32.242 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:32.242 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:32.242 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:29:32.242 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:26.344626+0000", 2024-09-14T06:29:32.242 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.503e-06 2024-09-14T06:29:32.242 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:32.242 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:32.242 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:29:32.242 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:26.369987+0000", 2024-09-14T06:29:32.242 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.025361338000000001 2024-09-14T06:29:32.242 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:32.242 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:32.243 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:29:32.243 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:26.370078+0000", 2024-09-14T06:29:32.243 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.0662999999999996e-05 2024-09-14T06:29:32.243 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:32.243 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:32.243 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:29:32.243 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:26.370211+0000", 2024-09-14T06:29:32.243 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00013318999999999999 2024-09-14T06:29:32.243 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:32.243 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:29:32.243 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:32.243 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:32.243 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:29:32.243 INFO:teuthology.orchestra.run.smithi138.stdout:} 2024-09-14T06:29:32.244 INFO:teuthology.orchestra.run.smithi094.stdout:osd.5: {} 2024-09-14T06:29:32.245 INFO:teuthology.orchestra.run.smithi094.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-14T06:29:32.245 INFO:teuthology.orchestra.run.smithi094.stdout:osd.6: {} 2024-09-14T06:29:32.246 INFO:teuthology.orchestra.run.smithi094.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-14T06:29:32.246 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-14T06:29:32.247 INFO:teuthology.orchestra.run.smithi094.stdout:osd.7: {} 2024-09-14T06:29:32.248 INFO:teuthology.orchestra.run.smithi094.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-14T06:29:32.248 INFO:tasks.ceph.osd.3.smithi094.stderr:2024-09-14T06:29:32.243+0000 7feeeafe8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33655) UID: 0 2024-09-14T06:29:32.345 INFO:tasks.ceph.osd.2.smithi094.stderr:2024-09-14T06:29:32.343+0000 7faf3269b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33649) UID: 0 2024-09-14T06:29:32.390 INFO:teuthology.orchestra.run.smithi094.stdout:ERROR: (22) Invalid argument 2024-09-14T06:29:32.390 INFO:teuthology.orchestra.run.smithi094.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:29:32.399 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-14T06:29:32.406 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-14T06:29:32.406 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-14T06:29:32.406 INFO:tasks.thrashosds.thrasher:inject_pause on osd.6 2024-09-14T06:29:32.407 INFO:tasks.thrashosds.thrasher:Testing filestore_inject_stall pause injection for duration 3 2024-09-14T06:29:32.407 INFO:tasks.thrashosds.thrasher:Checking after 0, should_be_down=False 2024-09-14T06:29:32.407 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok config set filestore_inject_stall 3 2024-09-14T06:29:32.451 INFO:tasks.ceph.osd.6.smithi138.stderr:2024-09-14T06:29:32.450+0000 7f8cd8849640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33620) UID: 0 2024-09-14T06:29:32.509 INFO:teuthology.orchestra.run.smithi138.stdout:{ 2024-09-14T06:29:32.509 INFO:teuthology.orchestra.run.smithi138.stdout: "success": "filestore_inject_stall = '' (not observed, change may require restart) " 2024-09-14T06:29:32.509 INFO:teuthology.orchestra.run.smithi138.stdout:} 2024-09-14T06:29:32.541 INFO:teuthology.orchestra.run.smithi094.stdout:ERROR: (22) Invalid argument 2024-09-14T06:29:32.541 INFO:teuthology.orchestra.run.smithi094.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:29:32.550 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-14T06:29:32.558 INFO:tasks.ceph.osd.6.smithi138.stderr:2024-09-14T06:29:32.557+0000 7f8cd8849640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33620) UID: 0 2024-09-14T06:29:32.591 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-14T06:29:32.648 INFO:tasks.ceph.osd.2.smithi094.stderr:2024-09-14T06:29:32.646+0000 7faf3269b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33649) UID: 0 2024-09-14T06:29:32.656 INFO:teuthology.orchestra.run.smithi094.stdout:ERROR: (22) Invalid argument 2024-09-14T06:29:32.656 INFO:teuthology.orchestra.run.smithi094.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:29:32.665 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-14T06:29:32.748 INFO:tasks.ceph.osd.5.smithi138.stderr:2024-09-14T06:29:32.747+0000 7fe13746f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33618) UID: 0 2024-09-14T06:29:32.812 INFO:teuthology.orchestra.run.smithi094.stdout:ERROR: (22) Invalid argument 2024-09-14T06:29:32.813 INFO:teuthology.orchestra.run.smithi094.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:29:32.821 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-14T06:29:32.865 INFO:tasks.ceph.osd.0.smithi094.stderr:2024-09-14T06:29:32.863+0000 7f07c9333640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33636) UID: 0 2024-09-14T06:29:32.924 INFO:teuthology.orchestra.run.smithi094.stdout:ERROR: (22) Invalid argument 2024-09-14T06:29:32.925 INFO:teuthology.orchestra.run.smithi094.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:29:32.933 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-14T06:29:32.974 INFO:tasks.ceph.osd.3.smithi094.stderr:2024-09-14T06:29:32.972+0000 7feeeafe8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33655) UID: 0 2024-09-14T06:29:33.050 INFO:tasks.ceph.osd.2.smithi094.stderr:2024-09-14T06:29:33.048+0000 7faf3269b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33649) UID: 0 2024-09-14T06:29:33.075 INFO:teuthology.orchestra.run.smithi094.stdout:ERROR: (22) Invalid argument 2024-09-14T06:29:33.075 INFO:teuthology.orchestra.run.smithi094.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:29:33.083 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-14T06:29:33.151 INFO:tasks.ceph.osd.7.smithi138.stderr:2024-09-14T06:29:33.150+0000 7f3c6a21d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33623) UID: 0 2024-09-14T06:29:33.227 INFO:teuthology.orchestra.run.smithi094.stdout:ERROR: (22) Invalid argument 2024-09-14T06:29:33.227 INFO:teuthology.orchestra.run.smithi094.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:29:33.235 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-14T06:29:33.279 INFO:tasks.ceph.osd.1.smithi094.stderr:2024-09-14T06:29:33.277+0000 7f328e47a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33641) UID: 0 2024-09-14T06:29:33.338 INFO:teuthology.orchestra.run.smithi094.stdout:ERROR: (22) Invalid argument 2024-09-14T06:29:33.338 INFO:teuthology.orchestra.run.smithi094.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:29:33.347 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-14T06:29:33.388 INFO:tasks.ceph.osd.3.smithi094.stderr:2024-09-14T06:29:33.386+0000 7feeeafe8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33655) UID: 0 2024-09-14T06:29:33.454 INFO:tasks.ceph.osd.5.smithi138.stderr:2024-09-14T06:29:33.453+0000 7fe13746f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33618) UID: 0 2024-09-14T06:29:33.491 INFO:teuthology.orchestra.run.smithi094.stdout:ERROR: (22) Invalid argument 2024-09-14T06:29:33.491 INFO:teuthology.orchestra.run.smithi094.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:29:33.500 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-14T06:29:33.556 INFO:tasks.ceph.osd.6.smithi138.stderr:2024-09-14T06:29:33.554+0000 7f8cd8849640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33620) UID: 0 2024-09-14T06:29:33.602 INFO:teuthology.orchestra.run.smithi094.stdout:ERROR: (22) Invalid argument 2024-09-14T06:29:33.603 INFO:teuthology.orchestra.run.smithi094.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:29:33.611 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-14T06:29:33.656 INFO:tasks.ceph.osd.6.smithi138.stderr:2024-09-14T06:29:33.656+0000 7f8cd8849640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33620) UID: 0 2024-09-14T06:29:33.699 INFO:teuthology.orchestra.run.smithi094.stderr:nodeep-scrub is unset 2024-09-14T06:29:33.722 INFO:teuthology.orchestra.run.smithi094.stdout:ERROR: (22) Invalid argument 2024-09-14T06:29:33.722 INFO:teuthology.orchestra.run.smithi094.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:29:33.737 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-14T06:29:33.779 INFO:tasks.ceph.osd.3.smithi094.stderr:2024-09-14T06:29:33.777+0000 7feeeafe8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33655) UID: 0 2024-09-14T06:29:33.858 INFO:tasks.ceph.osd.3.smithi094.stderr:2024-09-14T06:29:33.856+0000 7feeeafe8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33655) UID: 0 2024-09-14T06:29:33.890 INFO:teuthology.orchestra.run.smithi094.stdout:ERROR: (22) Invalid argument 2024-09-14T06:29:33.890 INFO:teuthology.orchestra.run.smithi094.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:29:33.898 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-14T06:29:33.959 INFO:tasks.ceph.osd.5.smithi138.stderr:2024-09-14T06:29:33.958+0000 7fe13746f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33618) UID: 0 2024-09-14T06:29:34.011 INFO:teuthology.orchestra.run.smithi138.stdout:ERROR: (22) Invalid argument 2024-09-14T06:29:34.011 INFO:teuthology.orchestra.run.smithi138.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:29:34.022 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-14T06:29:34.063 INFO:tasks.ceph.osd.4.smithi138.stderr:2024-09-14T06:29:34.062+0000 7f6c653c8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33615) UID: 0 2024-09-14T06:29:34.161 INFO:tasks.ceph.osd.5.smithi138.stderr:2024-09-14T06:29:34.160+0000 7fe13746f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33618) UID: 0 2024-09-14T06:29:34.194 INFO:teuthology.orchestra.run.smithi138.stdout:ERROR: (22) Invalid argument 2024-09-14T06:29:34.194 INFO:teuthology.orchestra.run.smithi138.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:29:34.206 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-14T06:29:34.248 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-14T06:29:34.293 INFO:tasks.ceph.osd.0.smithi094.stderr:2024-09-14T06:29:34.291+0000 7f07c9333640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33636) UID: 0 2024-09-14T06:29:34.364 INFO:tasks.ceph.osd.1.smithi094.stderr:2024-09-14T06:29:34.362+0000 7f328e47a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33641) UID: 0 2024-09-14T06:29:34.389 INFO:teuthology.orchestra.run.smithi138.stdout:ERROR: (22) Invalid argument 2024-09-14T06:29:34.389 INFO:teuthology.orchestra.run.smithi138.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:29:34.400 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-14T06:29:34.448 INFO:teuthology.orchestra.run.smithi094.stdout:osd.0: {} 2024-09-14T06:29:34.448 INFO:teuthology.orchestra.run.smithi094.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-14T06:29:34.463 INFO:tasks.ceph.osd.5.smithi138.stderr:2024-09-14T06:29:34.462+0000 7fe13746f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33618) UID: 0 2024-09-14T06:29:34.473 INFO:teuthology.orchestra.run.smithi094.stdout:osd.1: {} 2024-09-14T06:29:34.473 INFO:teuthology.orchestra.run.smithi094.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-14T06:29:34.500 INFO:teuthology.orchestra.run.smithi094.stdout:osd.2: {} 2024-09-14T06:29:34.500 INFO:teuthology.orchestra.run.smithi094.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-14T06:29:34.526 INFO:teuthology.orchestra.run.smithi094.stdout:osd.3: {} 2024-09-14T06:29:34.527 INFO:teuthology.orchestra.run.smithi094.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-14T06:29:34.552 INFO:teuthology.orchestra.run.smithi138.stdout:ERROR: (22) Invalid argument 2024-09-14T06:29:34.552 INFO:teuthology.orchestra.run.smithi138.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:29:34.556 INFO:teuthology.orchestra.run.smithi094.stdout:osd.4: {} 2024-09-14T06:29:34.556 INFO:teuthology.orchestra.run.smithi094.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-14T06:29:34.563 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-14T06:29:34.564 INFO:tasks.ceph.osd.0.smithi094.stderr:2024-09-14T06:29:34.562+0000 7f07c9333640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33636) UID: 0 2024-09-14T06:29:34.582 INFO:teuthology.orchestra.run.smithi094.stdout:osd.5: {} 2024-09-14T06:29:34.582 INFO:teuthology.orchestra.run.smithi094.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-14T06:29:34.607 INFO:teuthology.orchestra.run.smithi094.stdout:osd.6: {} 2024-09-14T06:29:34.607 INFO:teuthology.orchestra.run.smithi094.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-14T06:29:34.632 INFO:teuthology.orchestra.run.smithi094.stdout:osd.7: {} 2024-09-14T06:29:34.632 INFO:teuthology.orchestra.run.smithi094.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-14T06:29:34.687 INFO:tasks.ceph.osd.1.smithi094.stderr:2024-09-14T06:29:34.685+0000 7f328e47a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33641) UID: 0 2024-09-14T06:29:34.716 INFO:teuthology.orchestra.run.smithi138.stdout:{ 2024-09-14T06:29:34.716 INFO:teuthology.orchestra.run.smithi138.stdout: "ops": [], 2024-09-14T06:29:34.716 INFO:teuthology.orchestra.run.smithi138.stdout: "complaint_time": 30, 2024-09-14T06:29:34.716 INFO:teuthology.orchestra.run.smithi138.stdout: "num_blocked_ops": 0 2024-09-14T06:29:34.716 INFO:teuthology.orchestra.run.smithi138.stdout:} 2024-09-14T06:29:34.724 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-14T06:29:34.766 INFO:tasks.ceph.osd.0.smithi094.stderr:2024-09-14T06:29:34.764+0000 7f07c9333640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33636) UID: 0 2024-09-14T06:29:34.830 INFO:teuthology.orchestra.run.smithi138.stdout:{ 2024-09-14T06:29:34.830 INFO:teuthology.orchestra.run.smithi138.stdout: "size": 20, 2024-09-14T06:29:34.830 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 600, 2024-09-14T06:29:34.830 INFO:teuthology.orchestra.run.smithi138.stdout: "ops": [ 2024-09-14T06:29:34.830 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:34.830 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:11453 3.1d 3:bcad7303:::rbd_data.112d92027590.0000000000000001:f [read 94208~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:29:34.830 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:25.654375+0000", 2024-09-14T06:29:34.830 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 9.1729591149999994, 2024-09-14T06:29:34.830 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052449318000000002, 2024-09-14T06:29:34.831 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:29:34.831 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:29:34.831 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:29:34.831 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:29:34.831 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:29:34.831 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:34.831 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 11453 2024-09-14T06:29:34.831 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:34.831 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:29:34.831 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:34.831 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:29:34.831 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654375+0000", 2024-09-14T06:29:34.832 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:34.832 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:34.832 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:34.832 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:29:34.832 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654375+0000", 2024-09-14T06:29:34.832 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:34.832 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:34.832 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:34.833 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:29:34.833 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654496+0000", 2024-09-14T06:29:34.833 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00012081 2024-09-14T06:29:34.833 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:34.833 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:34.833 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:29:34.833 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654497+0000", 2024-09-14T06:29:34.833 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.8789999999999999e-06 2024-09-14T06:29:34.833 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:34.833 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:34.833 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:29:34.833 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654501+0000", 2024-09-14T06:29:34.833 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.8999999999999999e-06 2024-09-14T06:29:34.833 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:34.834 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:34.834 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:29:34.834 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654505+0000", 2024-09-14T06:29:34.834 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.2229999999999999e-06 2024-09-14T06:29:34.834 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:34.834 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:34.834 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:29:34.834 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.705917+0000", 2024-09-14T06:29:34.834 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.051412501999999999 2024-09-14T06:29:34.834 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:34.834 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:34.834 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:29:34.834 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.705999+0000", 2024-09-14T06:29:34.834 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.1551999999999994e-05 2024-09-14T06:29:34.834 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:34.835 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:34.835 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:29:34.835 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.706824+0000", 2024-09-14T06:29:34.835 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00082545199999999996 2024-09-14T06:29:34.835 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:34.835 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:29:34.835 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:34.835 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:34.835 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:34.835 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:11454 3.1d 3:bcad7303:::rbd_data.112d92027590.0000000000000001:f [read 104448~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:29:34.835 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:25.654510+0000", 2024-09-14T06:29:34.835 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 9.1728243250000006, 2024-09-14T06:29:34.835 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052610713000000003, 2024-09-14T06:29:34.835 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:29:34.835 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:29:34.836 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:29:34.836 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:29:34.836 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:29:34.836 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:34.836 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 11454 2024-09-14T06:29:34.836 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:34.836 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:29:34.836 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:34.836 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:29:34.836 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654510+0000", 2024-09-14T06:29:34.836 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:34.836 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:34.836 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:34.836 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:29:34.836 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654510+0000", 2024-09-14T06:29:34.837 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:34.837 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:34.837 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:34.837 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:29:34.837 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654825+0000", 2024-09-14T06:29:34.837 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00031522400000000002 2024-09-14T06:29:34.837 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:34.837 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:34.837 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:29:34.837 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654827+0000", 2024-09-14T06:29:34.837 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.9709999999999998e-06 2024-09-14T06:29:34.838 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:34.838 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:34.838 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:29:34.838 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654832+0000", 2024-09-14T06:29:34.838 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.4099999999999999e-06 2024-09-14T06:29:34.838 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:34.838 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:34.838 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:29:34.838 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654835+0000", 2024-09-14T06:29:34.838 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.14e-06 2024-09-14T06:29:34.838 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:34.838 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:34.838 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:29:34.838 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.706851+0000", 2024-09-14T06:29:34.839 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052015761000000001 2024-09-14T06:29:34.839 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:34.839 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:34.839 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:29:34.839 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.706934+0000", 2024-09-14T06:29:34.839 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.2484999999999994e-05 2024-09-14T06:29:34.839 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:34.839 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:34.839 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:29:34.839 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.707120+0000", 2024-09-14T06:29:34.839 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00018672200000000001 2024-09-14T06:29:34.839 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:34.839 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:29:34.839 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:34.839 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:34.839 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:34.840 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:11455 3.1d 3:bcad7303:::rbd_data.112d92027590.0000000000000001:f [read 114688~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:29:34.840 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:25.654841+0000", 2024-09-14T06:29:34.840 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 9.1724929960000008, 2024-09-14T06:29:34.840 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052579898, 2024-09-14T06:29:34.840 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:29:34.840 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:29:34.840 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:29:34.840 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:29:34.840 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:29:34.840 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:34.840 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 11455 2024-09-14T06:29:34.841 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:34.841 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:29:34.841 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:34.841 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:29:34.841 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654841+0000", 2024-09-14T06:29:34.841 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:34.841 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:34.841 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:34.841 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:29:34.841 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654841+0000", 2024-09-14T06:29:34.841 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:34.841 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:34.841 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:34.841 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:29:34.841 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654841+0000", 2024-09-14T06:29:34.842 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.7300000000000002e-07 2024-09-14T06:29:34.842 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:34.842 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:34.842 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:29:34.842 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654845+0000", 2024-09-14T06:29:34.842 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.0520000000000003e-06 2024-09-14T06:29:34.842 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:34.842 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:34.842 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:29:34.842 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654849+0000", 2024-09-14T06:29:34.842 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.0459999999999999e-06 2024-09-14T06:29:34.843 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:34.843 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:34.843 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:29:34.843 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654852+0000", 2024-09-14T06:29:34.843 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.9830000000000001e-06 2024-09-14T06:29:34.843 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:34.843 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:34.843 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:29:34.843 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.707159+0000", 2024-09-14T06:29:34.843 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052306826000000001 2024-09-14T06:29:34.843 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:34.843 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:34.843 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:29:34.844 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.707242+0000", 2024-09-14T06:29:34.844 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.2595000000000005e-05 2024-09-14T06:29:34.844 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:34.844 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:34.844 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:29:34.844 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.707421+0000", 2024-09-14T06:29:34.844 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.000179123 2024-09-14T06:29:34.844 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:34.844 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:29:34.844 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:34.844 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:34.844 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:34.844 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:11456 3.1d 3:bcad7303:::rbd_data.112d92027590.0000000000000001:f [read 124928~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:29:34.845 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:25.654856+0000", 2024-09-14T06:29:34.845 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 9.1724781499999999, 2024-09-14T06:29:34.845 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052801580000000001, 2024-09-14T06:29:34.845 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:29:34.845 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:29:34.845 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:29:34.845 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:29:34.846 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:29:34.846 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:34.846 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 11456 2024-09-14T06:29:34.846 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:34.846 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:29:34.846 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:34.846 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:29:34.846 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654856+0000", 2024-09-14T06:29:34.846 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:34.846 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:34.846 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:34.846 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:29:34.847 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654856+0000", 2024-09-14T06:29:34.847 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:34.847 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:34.847 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:34.847 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:29:34.847 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654856+0000", 2024-09-14T06:29:34.847 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.29e-07 2024-09-14T06:29:34.847 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:34.847 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:34.848 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:29:34.848 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654857+0000", 2024-09-14T06:29:34.848 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.187e-06 2024-09-14T06:29:34.848 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:34.848 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:34.848 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:29:34.848 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654861+0000", 2024-09-14T06:29:34.848 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.3850000000000002e-06 2024-09-14T06:29:34.848 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:34.848 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:34.848 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:29:34.849 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654863+0000", 2024-09-14T06:29:34.849 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.9500000000000001e-06 2024-09-14T06:29:34.849 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:34.849 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:34.849 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:29:34.849 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.707456+0000", 2024-09-14T06:29:34.849 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052592434 2024-09-14T06:29:34.849 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:34.849 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:34.849 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:29:34.849 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.707516+0000", 2024-09-14T06:29:34.850 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.0167999999999997e-05 2024-09-14T06:29:34.850 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:34.850 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:34.850 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:29:34.850 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.707657+0000", 2024-09-14T06:29:34.850 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00014122700000000001 2024-09-14T06:29:34.850 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:34.850 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:29:34.850 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:34.850 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:34.850 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:34.850 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:11457 3.1d 3:bcad7303:::rbd_data.112d92027590.0000000000000001:f [read 135168~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:29:34.850 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:25.654867+0000", 2024-09-14T06:29:34.851 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 9.1724672649999999, 2024-09-14T06:29:34.851 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052921902, 2024-09-14T06:29:34.851 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:29:34.851 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:29:34.851 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:29:34.851 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:29:34.851 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:29:34.851 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:34.851 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 11457 2024-09-14T06:29:34.851 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:34.851 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:29:34.851 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:34.851 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:29:34.852 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654867+0000", 2024-09-14T06:29:34.852 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:34.852 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:34.852 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:34.852 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:29:34.852 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654867+0000", 2024-09-14T06:29:34.852 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:34.852 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:34.852 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:34.852 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:29:34.852 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654972+0000", 2024-09-14T06:29:34.853 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00010584 2024-09-14T06:29:34.853 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:34.853 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:34.853 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:29:34.853 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654975+0000", 2024-09-14T06:29:34.853 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.1169999999999998e-06 2024-09-14T06:29:34.853 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:34.853 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:34.853 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:29:34.853 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654979+0000", 2024-09-14T06:29:34.853 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.1960000000000001e-06 2024-09-14T06:29:34.853 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:34.853 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:34.854 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:29:34.854 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654982+0000", 2024-09-14T06:29:34.854 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.2990000000000001e-06 2024-09-14T06:29:34.854 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:34.854 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:34.854 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:29:34.854 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.707684+0000", 2024-09-14T06:29:34.854 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052701979000000003 2024-09-14T06:29:34.854 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:34.854 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:34.854 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:29:34.854 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.707728+0000", 2024-09-14T06:29:34.854 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.3529999999999998e-05 2024-09-14T06:29:34.854 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:34.855 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:34.855 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:29:34.855 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.707789+0000", 2024-09-14T06:29:34.855 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.0940999999999998e-05 2024-09-14T06:29:34.855 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:34.855 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:29:34.855 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:34.855 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:34.855 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:34.855 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:11458 3.1d 3:bcad7303:::rbd_data.112d92027590.0000000000000001:f [read 145408~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:29:34.855 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:25.654987+0000", 2024-09-14T06:29:34.855 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 9.1723468669999999, 2024-09-14T06:29:34.855 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.05291655, 2024-09-14T06:29:34.855 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:29:34.856 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:29:34.856 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:29:34.856 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:29:34.856 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:29:34.856 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:34.856 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 11458 2024-09-14T06:29:34.856 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:34.856 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:29:34.856 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:34.856 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:29:34.856 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654987+0000", 2024-09-14T06:29:34.856 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:34.856 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:34.857 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:34.857 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:29:34.857 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654987+0000", 2024-09-14T06:29:34.857 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:34.857 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:34.857 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:34.857 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:29:34.857 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655077+0000", 2024-09-14T06:29:34.857 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.0253000000000005e-05 2024-09-14T06:29:34.858 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:34.858 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:34.858 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:29:34.858 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655079+0000", 2024-09-14T06:29:34.858 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.9989999999999998e-06 2024-09-14T06:29:34.858 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:34.858 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:34.858 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:29:34.858 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655083+0000", 2024-09-14T06:29:34.858 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.9280000000000003e-06 2024-09-14T06:29:34.858 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:34.858 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:34.858 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:29:34.859 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655086+0000", 2024-09-14T06:29:34.859 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.056e-06 2024-09-14T06:29:34.859 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:34.859 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:34.859 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:29:34.859 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.707805+0000", 2024-09-14T06:29:34.859 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052718627999999997 2024-09-14T06:29:34.859 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:34.859 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:34.859 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:29:34.859 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.707848+0000", 2024-09-14T06:29:34.859 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.2728999999999997e-05 2024-09-14T06:29:34.859 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:34.859 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:34.859 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:29:34.860 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.707904+0000", 2024-09-14T06:29:34.860 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.5957e-05 2024-09-14T06:29:34.860 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:34.860 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:29:34.860 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:34.860 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:34.860 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:34.860 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:11459 3.1d 3:bcad7303:::rbd_data.112d92027590.0000000000000001:f [read 155648~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:29:34.860 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:25.655091+0000", 2024-09-14T06:29:34.860 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 9.1722428590000007, 2024-09-14T06:29:34.860 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052944242000000002, 2024-09-14T06:29:34.860 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:29:34.860 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:29:34.861 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:29:34.861 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:29:34.861 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:29:34.861 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:34.861 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 11459 2024-09-14T06:29:34.861 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:34.861 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:29:34.861 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:34.861 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:29:34.861 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655091+0000", 2024-09-14T06:29:34.861 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:34.861 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:34.862 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:34.862 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:29:34.862 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655091+0000", 2024-09-14T06:29:34.862 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:34.862 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:34.862 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:34.862 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:29:34.862 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655237+0000", 2024-09-14T06:29:34.862 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00014570100000000001 2024-09-14T06:29:34.862 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:34.863 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:34.863 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:29:34.863 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655239+0000", 2024-09-14T06:29:34.863 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.8670000000000001e-06 2024-09-14T06:29:34.863 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:34.863 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:34.863 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:29:34.863 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655243+0000", 2024-09-14T06:29:34.863 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.9829999999999998e-06 2024-09-14T06:29:34.863 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:34.864 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:34.864 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:29:34.864 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655246+0000", 2024-09-14T06:29:34.864 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.1540000000000002e-06 2024-09-14T06:29:34.864 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:34.864 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:34.864 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:29:34.864 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.707934+0000", 2024-09-14T06:29:34.864 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052688286000000001 2024-09-14T06:29:34.864 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:34.865 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:34.865 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:29:34.865 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.707977+0000", 2024-09-14T06:29:34.865 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.3291999999999998e-05 2024-09-14T06:29:34.865 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:34.865 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:34.865 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:29:34.865 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.708035+0000", 2024-09-14T06:29:34.865 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.7958999999999999e-05 2024-09-14T06:29:34.865 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:34.865 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:29:34.865 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:34.865 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:34.865 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:34.866 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:11460 3.1d 3:bcad7303:::rbd_data.112d92027590.0000000000000001:f [read 165888~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:29:34.866 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:25.655250+0000", 2024-09-14T06:29:34.866 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 9.1720835619999992, 2024-09-14T06:29:34.866 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052910698999999999, 2024-09-14T06:29:34.866 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:29:34.866 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:29:34.866 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:29:34.866 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:29:34.866 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:29:34.866 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:34.866 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 11460 2024-09-14T06:29:34.866 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:34.867 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:29:34.867 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:34.867 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:29:34.867 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655250+0000", 2024-09-14T06:29:34.867 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:34.867 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:34.867 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:34.867 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:29:34.867 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655250+0000", 2024-09-14T06:29:34.867 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:34.867 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:34.867 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:34.868 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:29:34.868 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655356+0000", 2024-09-14T06:29:34.868 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00010535900000000001 2024-09-14T06:29:34.868 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:34.868 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:34.868 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:29:34.868 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655357+0000", 2024-09-14T06:29:34.868 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.8080000000000001e-06 2024-09-14T06:29:34.868 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:34.868 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:34.868 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:29:34.869 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655361+0000", 2024-09-14T06:29:34.869 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.861e-06 2024-09-14T06:29:34.869 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:34.869 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:34.869 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:29:34.869 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655364+0000", 2024-09-14T06:29:34.869 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.1089999999999998e-06 2024-09-14T06:29:34.869 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:34.869 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:34.869 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:29:34.869 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.708065+0000", 2024-09-14T06:29:34.869 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052700595000000003 2024-09-14T06:29:34.870 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:34.870 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:34.870 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:29:34.870 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.708108+0000", 2024-09-14T06:29:34.870 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.2484999999999997e-05 2024-09-14T06:29:34.870 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:34.870 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:34.870 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:29:34.870 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.708161+0000", 2024-09-14T06:29:34.870 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.3482000000000001e-05 2024-09-14T06:29:34.870 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:34.870 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:29:34.871 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:34.871 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:34.871 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:34.871 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:11461 3.1d 3:bcad7303:::rbd_data.112d92027590.0000000000000001:f [read 176128~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:29:34.871 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:25.655370+0000", 2024-09-14T06:29:34.871 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 9.1719642799999992, 2024-09-14T06:29:34.871 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052915141999999998, 2024-09-14T06:29:34.871 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:29:34.871 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:29:34.871 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:29:34.871 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:29:34.871 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:29:34.871 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:34.872 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 11461 2024-09-14T06:29:34.872 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:34.872 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:29:34.872 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:34.872 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:29:34.872 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655370+0000", 2024-09-14T06:29:34.872 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:34.872 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:34.872 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:34.872 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:29:34.872 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655370+0000", 2024-09-14T06:29:34.872 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:34.872 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:34.872 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:34.872 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:29:34.873 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655394+0000", 2024-09-14T06:29:34.873 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.4553999999999999e-05 2024-09-14T06:29:34.873 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:34.873 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:34.873 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:29:34.873 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655396+0000", 2024-09-14T06:29:34.873 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.8950000000000001e-06 2024-09-14T06:29:34.873 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:34.873 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:34.873 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:29:34.873 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655401+0000", 2024-09-14T06:29:34.873 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.0089999999999998e-06 2024-09-14T06:29:34.873 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:34.873 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:34.874 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:29:34.874 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655404+0000", 2024-09-14T06:29:34.874 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.3400000000000002e-06 2024-09-14T06:29:34.874 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:34.874 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:34.874 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:29:34.874 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.708186+0000", 2024-09-14T06:29:34.874 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052781726000000001 2024-09-14T06:29:34.874 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:34.874 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:34.874 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:29:34.874 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.708229+0000", 2024-09-14T06:29:34.874 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.2670999999999999e-05 2024-09-14T06:29:34.874 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:34.874 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:34.875 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:29:34.875 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.708285+0000", 2024-09-14T06:29:34.875 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.5946999999999998e-05 2024-09-14T06:29:34.875 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:34.875 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:29:34.875 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:34.875 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:34.875 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:34.875 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:11462 3.1d 3:bcad7303:::rbd_data.112d92027590.0000000000000001:f [read 186368~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:29:34.875 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:25.655410+0000", 2024-09-14T06:29:34.875 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 9.1719238020000002, 2024-09-14T06:29:34.875 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052995258000000003, 2024-09-14T06:29:34.875 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:29:34.876 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:29:34.876 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:29:34.876 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:29:34.876 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:29:34.876 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:34.876 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 11462 2024-09-14T06:29:34.876 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:34.876 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:29:34.876 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:34.876 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:29:34.876 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655410+0000", 2024-09-14T06:29:34.876 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:34.876 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:34.876 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:34.876 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:29:34.877 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655410+0000", 2024-09-14T06:29:34.877 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:34.877 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:34.877 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:34.877 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:29:34.877 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655625+0000", 2024-09-14T06:29:34.877 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00021442600000000001 2024-09-14T06:29:34.877 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:34.877 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:34.877 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:29:34.877 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655626+0000", 2024-09-14T06:29:34.877 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.815e-06 2024-09-14T06:29:34.877 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:34.877 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:34.878 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:29:34.878 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655631+0000", 2024-09-14T06:29:34.878 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.651e-06 2024-09-14T06:29:34.878 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:34.878 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:34.878 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:29:34.878 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655634+0000", 2024-09-14T06:29:34.878 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.2449999999999998e-06 2024-09-14T06:29:34.878 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:34.878 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:34.878 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:29:34.878 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.708312+0000", 2024-09-14T06:29:34.878 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052677412 2024-09-14T06:29:34.878 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:34.879 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:34.879 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:29:34.879 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.708352+0000", 2024-09-14T06:29:34.879 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.0729999999999998e-05 2024-09-14T06:29:34.879 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:34.879 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:34.879 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:29:34.879 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.708405+0000", 2024-09-14T06:29:34.879 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.2979000000000002e-05 2024-09-14T06:29:34.879 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:34.879 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:29:34.879 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:34.879 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:34.879 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:34.880 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:11463 3.1d 3:bcad7303:::rbd_data.112d92027590.0000000000000001:f [read 196608~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:29:34.880 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:25.655639+0000", 2024-09-14T06:29:34.880 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 9.1716946559999997, 2024-09-14T06:29:34.880 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052888859000000003, 2024-09-14T06:29:34.880 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:29:34.880 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:29:34.880 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:29:34.880 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:29:34.880 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:29:34.880 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:34.880 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 11463 2024-09-14T06:29:34.880 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:34.881 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:29:34.881 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:34.881 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:29:34.881 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655639+0000", 2024-09-14T06:29:34.881 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:34.881 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:34.881 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:34.881 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:29:34.881 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655639+0000", 2024-09-14T06:29:34.881 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:34.881 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:34.881 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:34.881 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:29:34.882 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655640+0000", 2024-09-14T06:29:34.882 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.08e-07 2024-09-14T06:29:34.882 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:34.882 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:34.882 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:29:34.882 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655641+0000", 2024-09-14T06:29:34.882 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.156e-06 2024-09-14T06:29:34.882 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:34.882 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:34.882 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:29:34.882 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655648+0000", 2024-09-14T06:29:34.882 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.8460000000000002e-06 2024-09-14T06:29:34.882 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:34.883 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:34.883 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:29:34.883 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655651+0000", 2024-09-14T06:29:34.883 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.0079999999999998e-06 2024-09-14T06:29:34.883 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:34.883 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:34.883 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:29:34.883 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.708429+0000", 2024-09-14T06:29:34.883 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052778459 2024-09-14T06:29:34.884 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:34.884 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:34.884 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:29:34.884 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.708473+0000", 2024-09-14T06:29:34.884 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.3782999999999997e-05 2024-09-14T06:29:34.884 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:34.884 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:34.884 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:29:34.884 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.708528+0000", 2024-09-14T06:29:34.884 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.5198999999999998e-05 2024-09-14T06:29:34.884 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:34.884 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:29:34.885 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:34.885 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:34.885 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:34.885 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:11464 3.1d 3:bcad7303:::rbd_data.112d92027590.0000000000000001:f [read 206848~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:29:34.885 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:25.655654+0000", 2024-09-14T06:29:34.885 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 9.1716797870000004, 2024-09-14T06:29:34.885 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052981845999999999, 2024-09-14T06:29:34.885 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:29:34.885 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:29:34.885 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:29:34.886 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:29:34.886 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:29:34.886 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:34.886 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 11464 2024-09-14T06:29:34.886 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:34.886 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:29:34.886 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:34.886 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:29:34.886 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655654+0000", 2024-09-14T06:29:34.886 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:34.886 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:34.886 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:34.887 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:29:34.887 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655654+0000", 2024-09-14T06:29:34.887 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:34.887 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:34.887 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:34.887 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:29:34.887 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655946+0000", 2024-09-14T06:29:34.887 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00029148200000000003 2024-09-14T06:29:34.887 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:34.887 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:34.887 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:29:34.887 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655948+0000", 2024-09-14T06:29:34.888 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.973e-06 2024-09-14T06:29:34.888 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:34.888 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:34.888 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:29:34.888 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655953+0000", 2024-09-14T06:29:34.888 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.1240000000000004e-06 2024-09-14T06:29:34.888 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:34.888 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:34.888 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:29:34.888 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655956+0000", 2024-09-14T06:29:34.888 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.315e-06 2024-09-14T06:29:34.889 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:34.889 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:34.889 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:29:34.889 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.708543+0000", 2024-09-14T06:29:34.889 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052587416999999997 2024-09-14T06:29:34.889 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:34.889 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:34.889 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:29:34.889 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.708584+0000", 2024-09-14T06:29:34.889 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.1004000000000002e-05 2024-09-14T06:29:34.889 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:34.889 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:34.889 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:29:34.889 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.708636+0000", 2024-09-14T06:29:34.890 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.1530999999999998e-05 2024-09-14T06:29:34.890 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:34.890 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:29:34.890 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:34.890 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:34.890 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:34.890 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:11465 3.1d 3:bcad7303:::rbd_data.112d92027590.0000000000000001:f [read 217088~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:29:34.890 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:25.655961+0000", 2024-09-14T06:29:34.890 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 9.1713726350000009, 2024-09-14T06:29:34.890 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052798682999999999, 2024-09-14T06:29:34.890 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:29:34.890 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:29:34.890 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:29:34.890 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:29:34.890 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:29:34.890 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:34.891 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 11465 2024-09-14T06:29:34.891 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:34.891 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:29:34.891 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:34.891 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:29:34.891 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655961+0000", 2024-09-14T06:29:34.891 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:34.891 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:34.891 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:34.891 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:29:34.891 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655961+0000", 2024-09-14T06:29:34.892 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:34.892 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:34.892 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:34.892 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:29:34.892 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.656114+0000", 2024-09-14T06:29:34.892 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.000152745 2024-09-14T06:29:34.892 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:34.892 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:34.892 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:29:34.892 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.656116+0000", 2024-09-14T06:29:34.892 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.804e-06 2024-09-14T06:29:34.893 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:34.893 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:34.893 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:29:34.893 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.656120+0000", 2024-09-14T06:29:34.893 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.4700000000000004e-06 2024-09-14T06:29:34.893 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:34.893 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:34.893 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:29:34.893 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.656123+0000", 2024-09-14T06:29:34.893 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.1269999999999999e-06 2024-09-14T06:29:34.893 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:34.893 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:34.893 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:29:34.894 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.708660+0000", 2024-09-14T06:29:34.894 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052536919000000001 2024-09-14T06:29:34.894 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:34.894 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:34.894 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:29:34.894 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.708703+0000", 2024-09-14T06:29:34.894 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.3077000000000001e-05 2024-09-14T06:29:34.895 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:34.895 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:34.895 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:29:34.895 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.708760+0000", 2024-09-14T06:29:34.895 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.6541e-05 2024-09-14T06:29:34.895 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:34.895 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:29:34.895 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:34.895 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:34.895 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:34.895 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:11466 3.1d 3:bcad7303:::rbd_data.112d92027590.0000000000000001:f [read 227328~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:29:34.895 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:25.656129+0000", 2024-09-14T06:29:34.895 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 9.1712052469999996, 2024-09-14T06:29:34.895 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052750987999999999, 2024-09-14T06:29:34.896 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:29:34.896 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:29:34.896 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:29:34.896 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:29:34.896 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:29:34.896 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:34.896 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 11466 2024-09-14T06:29:34.896 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:34.896 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:29:34.896 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:34.896 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:29:34.896 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.656129+0000", 2024-09-14T06:29:34.896 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:34.896 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:34.896 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:34.897 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:29:34.897 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.656129+0000", 2024-09-14T06:29:34.897 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:34.897 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:34.897 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:34.897 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:29:34.897 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.656347+0000", 2024-09-14T06:29:34.897 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.000218054 2024-09-14T06:29:34.897 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:34.897 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:34.897 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:29:34.897 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.656349+0000", 2024-09-14T06:29:34.897 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.9199999999999998e-06 2024-09-14T06:29:34.897 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:34.898 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:34.898 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:29:34.898 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.656353+0000", 2024-09-14T06:29:34.898 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.9210000000000002e-06 2024-09-14T06:29:34.898 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:34.898 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:34.898 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:29:34.898 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.656356+0000", 2024-09-14T06:29:34.898 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.7670000000000001e-06 2024-09-14T06:29:34.898 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:34.898 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:34.898 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:29:34.898 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.708787+0000", 2024-09-14T06:29:34.898 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052430840999999999 2024-09-14T06:29:34.898 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:34.899 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:34.899 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:29:34.899 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.708827+0000", 2024-09-14T06:29:34.899 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.9538e-05 2024-09-14T06:29:34.899 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:34.899 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:34.899 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:29:34.899 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.708880+0000", 2024-09-14T06:29:34.899 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.2947e-05 2024-09-14T06:29:34.899 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:34.899 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:29:34.899 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:34.899 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:34.899 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:34.899 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:11467 3.1d 3:bcad7303:::rbd_data.112d92027590.0000000000000001:f [read 237568~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:29:34.899 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:25.656361+0000", 2024-09-14T06:29:34.900 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 9.1709725990000006, 2024-09-14T06:29:34.900 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052643898000000001, 2024-09-14T06:29:34.900 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:29:34.900 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:29:34.900 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:29:34.900 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:29:34.900 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:29:34.900 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:34.900 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 11467 2024-09-14T06:29:34.900 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:34.900 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:29:34.900 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:34.900 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:29:34.900 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.656361+0000", 2024-09-14T06:29:34.900 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:34.901 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:34.901 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:34.901 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:29:34.901 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.656361+0000", 2024-09-14T06:29:34.901 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:34.901 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:34.901 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:34.901 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:29:34.901 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.656855+0000", 2024-09-14T06:29:34.901 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00049372000000000001 2024-09-14T06:29:34.901 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:34.901 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:34.901 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:29:34.901 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.656859+0000", 2024-09-14T06:29:34.901 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.1300000000000003e-06 2024-09-14T06:29:34.901 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:34.902 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:34.902 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:29:34.902 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.656863+0000", 2024-09-14T06:29:34.902 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.2629999999999997e-06 2024-09-14T06:29:34.902 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:34.902 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:34.902 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:29:34.902 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.656867+0000", 2024-09-14T06:29:34.902 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.3220000000000001e-06 2024-09-14T06:29:34.902 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:34.902 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:34.902 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:29:34.902 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.708903+0000", 2024-09-14T06:29:34.902 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052035855999999998 2024-09-14T06:29:34.902 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:34.903 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:34.903 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:29:34.903 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.708949+0000", 2024-09-14T06:29:34.903 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.6276999999999998e-05 2024-09-14T06:29:34.903 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:34.903 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:34.903 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:29:34.903 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.709005+0000", 2024-09-14T06:29:34.903 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.6329999999999998e-05 2024-09-14T06:29:34.903 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:34.903 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:29:34.903 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:34.903 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:34.903 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:34.903 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:11479 3.1d 3:bcad7303:::rbd_data.112d92027590.0000000000000001:f [read 360448~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:29:34.904 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:25.658388+0000", 2024-09-14T06:29:34.904 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 9.1689457660000002, 2024-09-14T06:29:34.904 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052408920999999997, 2024-09-14T06:29:34.904 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:29:34.904 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:29:34.904 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:29:34.904 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:29:34.904 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:29:34.904 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:34.904 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 11479 2024-09-14T06:29:34.904 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:34.904 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:29:34.904 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:34.905 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:29:34.905 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.658388+0000", 2024-09-14T06:29:34.905 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:34.905 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:34.905 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:34.905 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:29:34.905 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.658388+0000", 2024-09-14T06:29:34.905 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:34.905 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:34.905 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:34.905 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:29:34.905 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.658534+0000", 2024-09-14T06:29:34.905 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.000145613 2024-09-14T06:29:34.905 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:34.905 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:34.906 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:29:34.906 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.658540+0000", 2024-09-14T06:29:34.906 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.6680000000000004e-06 2024-09-14T06:29:34.906 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:34.906 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:34.906 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:29:34.906 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.658548+0000", 2024-09-14T06:29:34.906 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.7980000000000006e-06 2024-09-14T06:29:34.906 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:34.906 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:34.906 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:29:34.906 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.658554+0000", 2024-09-14T06:29:34.906 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.6869999999999999e-06 2024-09-14T06:29:34.907 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:34.907 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:34.907 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:29:34.907 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.710624+0000", 2024-09-14T06:29:34.907 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052070567999999998 2024-09-14T06:29:34.907 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:34.907 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:34.907 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:29:34.907 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.710666+0000", 2024-09-14T06:29:34.908 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.1467000000000001e-05 2024-09-14T06:29:34.908 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:34.908 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:34.908 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:29:34.908 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.710797+0000", 2024-09-14T06:29:34.908 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00013112 2024-09-14T06:29:34.908 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:34.908 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:29:34.908 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:34.908 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:34.908 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:34.909 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:11480 3.1d 3:bcad7303:::rbd_data.112d92027590.0000000000000001:f [read 370688~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:29:34.909 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:25.658563+0000", 2024-09-14T06:29:34.909 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 9.1687708350000001, 2024-09-14T06:29:34.909 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052429164, 2024-09-14T06:29:34.909 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:29:34.909 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:29:34.909 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:29:34.909 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:29:34.909 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:29:34.909 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:34.909 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 11480 2024-09-14T06:29:34.909 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:34.909 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:29:34.910 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:34.910 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:29:34.910 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.658563+0000", 2024-09-14T06:29:34.910 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:34.910 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:34.910 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:34.910 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:29:34.910 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.658563+0000", 2024-09-14T06:29:34.910 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:34.910 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:34.910 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:34.910 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:29:34.910 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.658658+0000", 2024-09-14T06:29:34.910 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.5346999999999994e-05 2024-09-14T06:29:34.910 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:34.911 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:34.911 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:29:34.911 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.658662+0000", 2024-09-14T06:29:34.911 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.14e-06 2024-09-14T06:29:34.911 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:34.911 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:34.911 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:29:34.911 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.658669+0000", 2024-09-14T06:29:34.911 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.7379999999999994e-06 2024-09-14T06:29:34.911 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:34.911 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:34.911 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:29:34.911 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.658675+0000", 2024-09-14T06:29:34.911 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.5430000000000001e-06 2024-09-14T06:29:34.912 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:34.912 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:34.912 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:29:34.912 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.710824+0000", 2024-09-14T06:29:34.912 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052148847999999998 2024-09-14T06:29:34.912 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:34.912 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:34.912 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:29:34.912 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.710864+0000", 2024-09-14T06:29:34.912 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.9925999999999997e-05 2024-09-14T06:29:34.912 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:34.912 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:34.912 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:29:34.912 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.710992+0000", 2024-09-14T06:29:34.912 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.000128622 2024-09-14T06:29:34.912 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:34.913 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:29:34.913 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:34.913 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:34.913 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:34.913 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:11481 3.1d 3:bcad7303:::rbd_data.112d92027590.0000000000000001:f [read 380928~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:29:34.913 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:25.658684+0000", 2024-09-14T06:29:34.913 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 9.1686497649999996, 2024-09-14T06:29:34.913 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052508565, 2024-09-14T06:29:34.913 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:29:34.913 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:29:34.913 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:29:34.913 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:29:34.913 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:29:34.913 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:34.914 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 11481 2024-09-14T06:29:34.914 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:34.914 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:29:34.914 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:34.914 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:29:34.914 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.658684+0000", 2024-09-14T06:29:34.914 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:34.914 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:34.914 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:34.914 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:29:34.914 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.658684+0000", 2024-09-14T06:29:34.914 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:34.914 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:34.914 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:34.914 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:29:34.915 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.658715+0000", 2024-09-14T06:29:34.915 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.0703000000000001e-05 2024-09-14T06:29:34.915 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:34.915 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:34.915 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:29:34.915 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.658718+0000", 2024-09-14T06:29:34.915 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.0709999999999999e-06 2024-09-14T06:29:34.915 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:34.915 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:34.916 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:29:34.916 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.658725+0000", 2024-09-14T06:29:34.916 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.2339999999999997e-06 2024-09-14T06:29:34.916 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:34.916 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:34.916 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:29:34.916 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.658731+0000", 2024-09-14T06:29:34.916 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.6450000000000002e-06 2024-09-14T06:29:34.916 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:34.916 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:34.916 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:29:34.916 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.711020+0000", 2024-09-14T06:29:34.916 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052288886 2024-09-14T06:29:34.916 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:34.916 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:34.917 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:29:34.917 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.711061+0000", 2024-09-14T06:29:34.917 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.1084000000000002e-05 2024-09-14T06:29:34.917 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:34.917 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:34.917 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:29:34.917 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.711193+0000", 2024-09-14T06:29:34.917 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00013194199999999999 2024-09-14T06:29:34.917 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:34.917 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:29:34.917 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:34.917 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:34.917 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:34.917 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:11482 3.1d 3:bcad7303:::rbd_data.112d92027590.0000000000000001:f [read 391168~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:29:34.918 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:25.658740+0000", 2024-09-14T06:29:34.918 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 9.1685943000000005, 2024-09-14T06:29:34.918 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052637015000000002, 2024-09-14T06:29:34.918 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:29:34.918 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:29:34.918 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:29:34.918 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:29:34.918 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:29:34.918 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:34.918 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 11482 2024-09-14T06:29:34.918 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:34.918 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:29:34.918 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:34.919 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:29:34.919 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.658740+0000", 2024-09-14T06:29:34.919 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:34.919 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:34.919 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:34.919 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:29:34.919 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.658740+0000", 2024-09-14T06:29:34.919 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:34.919 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:34.919 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:34.919 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:29:34.919 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.658880+0000", 2024-09-14T06:29:34.919 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00014036100000000001 2024-09-14T06:29:34.919 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:34.920 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:34.920 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:29:34.920 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.658883+0000", 2024-09-14T06:29:34.920 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.3969999999999998e-06 2024-09-14T06:29:34.920 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:34.920 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:34.920 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:29:34.920 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.658890+0000", 2024-09-14T06:29:34.920 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.1500000000000002e-06 2024-09-14T06:29:34.920 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:34.920 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:34.920 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:29:34.920 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.658897+0000", 2024-09-14T06:29:34.920 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.1260000000000003e-06 2024-09-14T06:29:34.920 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:34.921 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:34.921 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:29:34.921 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.711207+0000", 2024-09-14T06:29:34.921 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052310777000000003 2024-09-14T06:29:34.921 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:34.921 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:34.921 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:29:34.921 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.711249+0000", 2024-09-14T06:29:34.921 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.1786999999999998e-05 2024-09-14T06:29:34.921 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:34.921 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:34.921 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:29:34.921 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.711377+0000", 2024-09-14T06:29:34.921 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.000127417 2024-09-14T06:29:34.922 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:34.922 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:29:34.922 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:34.922 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:34.922 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:34.922 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:11483 3.1d 3:bcad7303:::rbd_data.112d92027590.0000000000000001:f [read 401408~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:29:34.922 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:25.658906+0000", 2024-09-14T06:29:34.922 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 9.1684275159999995, 2024-09-14T06:29:34.922 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052595109000000001, 2024-09-14T06:29:34.922 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:29:34.922 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:29:34.922 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:29:34.923 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:29:34.923 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:29:34.923 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:34.923 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 11483 2024-09-14T06:29:34.923 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:34.923 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:29:34.923 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:34.923 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:29:34.923 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.658906+0000", 2024-09-14T06:29:34.923 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:34.923 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:34.923 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:34.923 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:29:34.923 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.658906+0000", 2024-09-14T06:29:34.924 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:34.924 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:34.924 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:34.924 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:29:34.924 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.664985+0000", 2024-09-14T06:29:34.924 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.0060787250000000001 2024-09-14T06:29:34.924 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:34.924 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:34.924 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:29:34.924 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.664991+0000", 2024-09-14T06:29:34.924 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.7030000000000003e-06 2024-09-14T06:29:34.924 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:34.924 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:34.924 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:29:34.925 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.664999+0000", 2024-09-14T06:29:34.925 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.4430000000000003e-06 2024-09-14T06:29:34.925 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:34.925 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:34.925 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:29:34.925 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.665006+0000", 2024-09-14T06:29:34.925 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.0840000000000003e-06 2024-09-14T06:29:34.926 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:34.926 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:34.926 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:29:34.926 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.711402+0000", 2024-09-14T06:29:34.926 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046395714999999997 2024-09-14T06:29:34.926 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:34.926 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:34.926 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:29:34.926 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.711443+0000", 2024-09-14T06:29:34.926 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.0911999999999997e-05 2024-09-14T06:29:34.926 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:34.926 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:34.926 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:29:34.927 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.711501+0000", 2024-09-14T06:29:34.927 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.8526999999999997e-05 2024-09-14T06:29:34.927 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:34.927 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:29:34.927 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:34.927 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:34.927 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:29:34.927 INFO:teuthology.orchestra.run.smithi138.stdout:} 2024-09-14T06:29:34.928 INFO:teuthology.orchestra.run.smithi094.stdout:checking clone #7, image image_client.0-clone7 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent8 2024-09-14T06:29:34.929 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-14T06:29:34.929 INFO:tasks.ceph.osd.1.smithi094.stderr:2024-09-14T06:29:34.926+0000 7f328e47a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33641) UID: 0 2024-09-14T06:29:35.029 INFO:tasks.ceph.osd.3.smithi094.stderr:2024-09-14T06:29:35.027+0000 7feeeafe8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33655) UID: 0 2024-09-14T06:29:35.032 INFO:teuthology.orchestra.run.smithi138.stdout:{ 2024-09-14T06:29:35.033 INFO:teuthology.orchestra.run.smithi138.stdout: "ops": [], 2024-09-14T06:29:35.033 INFO:teuthology.orchestra.run.smithi138.stdout: "num_ops": 0 2024-09-14T06:29:35.033 INFO:teuthology.orchestra.run.smithi138.stdout:} 2024-09-14T06:29:35.042 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-14T06:29:35.115 INFO:teuthology.orchestra.run.smithi094.stdout:587 trunc from 0xe850d64 to 0x96e8fc 2024-09-14T06:29:35.129 INFO:tasks.ceph.osd.5.smithi138.stderr:2024-09-14T06:29:35.128+0000 7fe13746f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33618) UID: 0 2024-09-14T06:29:35.157 INFO:teuthology.orchestra.run.smithi138.stdout:{ 2024-09-14T06:29:35.158 INFO:teuthology.orchestra.run.smithi138.stdout: "ops": [], 2024-09-14T06:29:35.158 INFO:teuthology.orchestra.run.smithi138.stdout: "complaint_time": 30, 2024-09-14T06:29:35.158 INFO:teuthology.orchestra.run.smithi138.stdout: "num_blocked_ops": 0 2024-09-14T06:29:35.158 INFO:teuthology.orchestra.run.smithi138.stdout:} 2024-09-14T06:29:35.166 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-14T06:29:35.251 INFO:tasks.ceph.osd.5.smithi138.stderr:2024-09-14T06:29:35.250+0000 7fe13746f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33618) UID: 0 2024-09-14T06:29:35.313 INFO:teuthology.orchestra.run.smithi138.stdout:{ 2024-09-14T06:29:35.314 INFO:teuthology.orchestra.run.smithi138.stdout: "size": 20, 2024-09-14T06:29:35.314 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 600, 2024-09-14T06:29:35.314 INFO:teuthology.orchestra.run.smithi138.stdout: "ops": [ 2024-09-14T06:29:35.314 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:35.314 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:1484 3.a 3:57c28b0f:::rbd_data.112d7dfbd766.000000000000000a:5 [read 208896~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-14T06:29:35.314 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:08.628246+0000", 2024-09-14T06:29:35.314 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 26.683271496, 2024-09-14T06:29:35.314 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.018096672000000001, 2024-09-14T06:29:35.314 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:29:35.314 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:29:35.314 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:29:35.314 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:29:35.315 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:29:35.315 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:35.315 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 1484 2024-09-14T06:29:35.315 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:35.315 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:29:35.315 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:35.315 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:29:35.315 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.628246+0000", 2024-09-14T06:29:35.315 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:35.315 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:35.315 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:35.315 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:29:35.315 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.628246+0000", 2024-09-14T06:29:35.316 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:35.316 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:35.316 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:35.316 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:29:35.316 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.628246+0000", 2024-09-14T06:29:35.316 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.79e-07 2024-09-14T06:29:35.316 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:35.316 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:35.316 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:29:35.316 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.628247+0000", 2024-09-14T06:29:35.316 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.5179999999999999e-06 2024-09-14T06:29:35.316 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:35.316 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:35.316 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:29:35.317 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.628252+0000", 2024-09-14T06:29:35.317 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.0139999999999998e-06 2024-09-14T06:29:35.317 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:35.317 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:35.317 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:29:35.317 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.628257+0000", 2024-09-14T06:29:35.317 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.2429999999999999e-06 2024-09-14T06:29:35.317 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:35.317 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:35.317 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:29:35.317 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.646180+0000", 2024-09-14T06:29:35.317 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.017923136999999999 2024-09-14T06:29:35.318 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:35.318 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:35.318 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:29:35.318 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.646251+0000", 2024-09-14T06:29:35.318 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.1285000000000006e-05 2024-09-14T06:29:35.318 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:35.318 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:35.318 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:29:35.318 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.646342+0000", 2024-09-14T06:29:35.318 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.1195999999999999e-05 2024-09-14T06:29:35.318 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:35.318 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:29:35.318 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:35.318 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:35.318 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:35.319 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:1485 3.a 3:57c28b0f:::rbd_data.112d7dfbd766.000000000000000a:5 [read 225280~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-14T06:29:35.319 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:08.628268+0000", 2024-09-14T06:29:35.319 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 26.683249015000001, 2024-09-14T06:29:35.319 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.018257005999999999, 2024-09-14T06:29:35.319 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:29:35.319 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:29:35.319 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:29:35.319 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:29:35.319 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:29:35.319 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:35.319 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 1485 2024-09-14T06:29:35.319 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:35.319 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:29:35.319 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:35.320 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:29:35.320 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.628268+0000", 2024-09-14T06:29:35.320 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:35.320 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:35.320 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:35.320 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:29:35.320 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.628268+0000", 2024-09-14T06:29:35.320 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:35.320 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:35.321 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:35.321 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:29:35.321 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.628269+0000", 2024-09-14T06:29:35.321 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.6800000000000001e-07 2024-09-14T06:29:35.321 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:35.321 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:35.321 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:29:35.321 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.628270+0000", 2024-09-14T06:29:35.321 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.8449999999999999e-06 2024-09-14T06:29:35.321 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:35.321 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:35.321 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:29:35.321 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.628276+0000", 2024-09-14T06:29:35.321 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.998e-06 2024-09-14T06:29:35.322 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:35.322 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:35.322 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:29:35.322 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.628281+0000", 2024-09-14T06:29:35.322 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.3120000000000004e-06 2024-09-14T06:29:35.322 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:35.322 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:35.322 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:29:35.322 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.646371+0000", 2024-09-14T06:29:35.322 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.018090694000000001 2024-09-14T06:29:35.322 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:35.322 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:35.322 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:29:35.322 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.646439+0000", 2024-09-14T06:29:35.323 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.7182999999999997e-05 2024-09-14T06:29:35.323 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:35.323 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:35.323 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:29:35.323 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.646525+0000", 2024-09-14T06:29:35.323 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.6506000000000002e-05 2024-09-14T06:29:35.323 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:35.323 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:29:35.323 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:35.323 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:35.323 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:35.323 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:1486 3.a 3:57c28b0f:::rbd_data.112d7dfbd766.000000000000000a:5 [read 241664~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-14T06:29:35.323 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:08.628288+0000", 2024-09-14T06:29:35.323 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 26.683229129000001, 2024-09-14T06:29:35.324 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.018416895999999999, 2024-09-14T06:29:35.324 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:29:35.324 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:29:35.324 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:29:35.324 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:29:35.324 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:29:35.324 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:35.324 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 1486 2024-09-14T06:29:35.324 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:35.324 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:29:35.324 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:35.324 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:29:35.324 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.628288+0000", 2024-09-14T06:29:35.325 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:35.325 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:35.325 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:35.325 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:29:35.325 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.628288+0000", 2024-09-14T06:29:35.325 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:35.325 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:35.325 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:35.325 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:29:35.325 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.628288+0000", 2024-09-14T06:29:35.325 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.9499999999999998e-07 2024-09-14T06:29:35.325 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:35.325 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:35.325 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:29:35.326 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.628290+0000", 2024-09-14T06:29:35.326 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.6840000000000001e-06 2024-09-14T06:29:35.326 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:35.326 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:35.326 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:29:35.326 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.628295+0000", 2024-09-14T06:29:35.326 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.3059999999999997e-06 2024-09-14T06:29:35.326 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:35.326 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:35.326 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:29:35.326 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.628300+0000", 2024-09-14T06:29:35.326 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.2690000000000001e-06 2024-09-14T06:29:35.326 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:35.326 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:35.326 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:29:35.327 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.646553+0000", 2024-09-14T06:29:35.327 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.018253015000000001 2024-09-14T06:29:35.327 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:35.327 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:35.327 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:29:35.327 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.646626+0000", 2024-09-14T06:29:35.327 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.2939000000000001e-05 2024-09-14T06:29:35.327 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:35.327 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:35.327 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:29:35.327 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.646705+0000", 2024-09-14T06:29:35.327 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.9388000000000006e-05 2024-09-14T06:29:35.327 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:35.327 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:29:35.327 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:35.328 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:35.328 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:35.328 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:1487 3.a 3:57c28b0f:::rbd_data.112d7dfbd766.000000000000000a:5 [read 258048~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-14T06:29:35.328 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:08.628842+0000", 2024-09-14T06:29:35.328 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 26.682674684999999, 2024-09-14T06:29:35.328 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.018037741999999999, 2024-09-14T06:29:35.328 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:29:35.328 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:29:35.328 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:29:35.328 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:29:35.328 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:29:35.328 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:35.328 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 1487 2024-09-14T06:29:35.329 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:35.329 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:29:35.329 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:35.329 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:29:35.329 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.628842+0000", 2024-09-14T06:29:35.329 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:35.329 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:35.329 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:35.329 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:29:35.329 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.628842+0000", 2024-09-14T06:29:35.329 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:35.329 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:35.330 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:35.330 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:29:35.330 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.628843+0000", 2024-09-14T06:29:35.330 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.5600000000000005e-07 2024-09-14T06:29:35.330 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:35.330 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:35.330 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:29:35.330 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.628847+0000", 2024-09-14T06:29:35.330 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.394e-06 2024-09-14T06:29:35.330 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:35.330 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:35.330 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:29:35.330 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.628855+0000", 2024-09-14T06:29:35.330 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.0760000000000003e-06 2024-09-14T06:29:35.331 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:35.331 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:35.331 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:29:35.331 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.628868+0000", 2024-09-14T06:29:35.331 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.2860000000000001e-05 2024-09-14T06:29:35.331 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:35.331 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:35.331 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:29:35.331 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.646733+0000", 2024-09-14T06:29:35.331 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.017865387999999999 2024-09-14T06:29:35.331 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:35.331 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:35.331 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:29:35.331 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.646800+0000", 2024-09-14T06:29:35.331 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.6619999999999996e-05 2024-09-14T06:29:35.332 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:35.332 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:35.332 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:29:35.332 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.646880+0000", 2024-09-14T06:29:35.332 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.0648000000000004e-05 2024-09-14T06:29:35.332 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:35.332 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:29:35.332 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:35.332 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:35.332 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:35.332 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:1488 3.a 3:57c28b0f:::rbd_data.112d7dfbd766.000000000000000a:5 [read 274432~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-14T06:29:35.332 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:08.628878+0000", 2024-09-14T06:29:35.332 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 26.682639067, 2024-09-14T06:29:35.332 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.018187624999999999, 2024-09-14T06:29:35.332 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:29:35.333 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:29:35.333 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:29:35.333 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:29:35.333 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:29:35.333 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:35.333 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 1488 2024-09-14T06:29:35.333 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:35.333 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:29:35.333 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:35.333 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:29:35.333 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.628878+0000", 2024-09-14T06:29:35.333 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:35.333 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:35.334 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:35.334 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:29:35.334 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.628878+0000", 2024-09-14T06:29:35.334 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:35.334 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:35.334 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:35.334 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:29:35.334 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.628879+0000", 2024-09-14T06:29:35.334 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.58e-07 2024-09-14T06:29:35.334 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:35.334 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:35.334 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:29:35.334 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.628881+0000", 2024-09-14T06:29:35.334 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.0389999999999999e-06 2024-09-14T06:29:35.334 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:35.335 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:35.335 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:29:35.335 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.628887+0000", 2024-09-14T06:29:35.335 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.9429999999999997e-06 2024-09-14T06:29:35.335 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:35.335 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:35.335 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:29:35.335 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.628891+0000", 2024-09-14T06:29:35.335 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.1309999999999999e-06 2024-09-14T06:29:35.335 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:35.335 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:35.335 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:29:35.335 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.646909+0000", 2024-09-14T06:29:35.335 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.018017920999999999 2024-09-14T06:29:35.335 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:35.335 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:35.336 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:29:35.336 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.646984+0000", 2024-09-14T06:29:35.336 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.4989999999999999e-05 2024-09-14T06:29:35.336 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:35.336 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:35.336 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:29:35.336 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.647066+0000", 2024-09-14T06:29:35.336 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.2143000000000003e-05 2024-09-14T06:29:35.336 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:35.336 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:29:35.336 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:35.336 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:35.337 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:35.337 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:1489 3.a 3:57c28b0f:::rbd_data.112d7dfbd766.000000000000000a:5 [read 290816~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-14T06:29:35.337 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:08.628897+0000", 2024-09-14T06:29:35.337 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 26.682620616000001, 2024-09-14T06:29:35.337 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.018341491000000001, 2024-09-14T06:29:35.337 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:29:35.337 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:29:35.337 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:29:35.337 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:29:35.337 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:29:35.337 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:35.338 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 1489 2024-09-14T06:29:35.338 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:35.338 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:29:35.338 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:35.338 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:29:35.338 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.628897+0000", 2024-09-14T06:29:35.338 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:35.338 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:35.338 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:35.338 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:29:35.338 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.628897+0000", 2024-09-14T06:29:35.338 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:35.338 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:35.339 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:35.339 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:29:35.339 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.628897+0000", 2024-09-14T06:29:35.339 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.6800000000000001e-07 2024-09-14T06:29:35.339 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:35.339 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:35.339 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:29:35.339 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.628899+0000", 2024-09-14T06:29:35.339 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.606e-06 2024-09-14T06:29:35.339 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:35.339 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:35.339 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:29:35.339 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.628904+0000", 2024-09-14T06:29:35.339 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.1839999999999998e-06 2024-09-14T06:29:35.339 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:35.339 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:35.340 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:29:35.340 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.628908+0000", 2024-09-14T06:29:35.340 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.0500000000000002e-06 2024-09-14T06:29:35.340 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:35.340 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:35.340 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:29:35.340 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.647093+0000", 2024-09-14T06:29:35.340 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.018184846000000001 2024-09-14T06:29:35.340 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:35.340 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:35.340 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:29:35.340 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.647158+0000", 2024-09-14T06:29:35.340 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.5068999999999997e-05 2024-09-14T06:29:35.341 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:35.341 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:35.341 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:29:35.341 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.647238+0000", 2024-09-14T06:29:35.341 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.0368e-05 2024-09-14T06:29:35.341 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:35.341 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:29:35.341 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:35.341 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:35.341 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:35.341 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:1490 3.a 3:57c28b0f:::rbd_data.112d7dfbd766.000000000000000a:5 [read 307200~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-14T06:29:35.341 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:08.628921+0000", 2024-09-14T06:29:35.342 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 26.682596619000002, 2024-09-14T06:29:35.342 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.018575338, 2024-09-14T06:29:35.342 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:29:35.342 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:29:35.342 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:29:35.342 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:29:35.342 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:29:35.342 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:35.342 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 1490 2024-09-14T06:29:35.342 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:35.342 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:29:35.342 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:35.342 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:29:35.343 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.628921+0000", 2024-09-14T06:29:35.343 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:35.343 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:35.343 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:35.343 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:29:35.343 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.628921+0000", 2024-09-14T06:29:35.343 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:35.343 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:35.343 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:35.343 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:29:35.343 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.628921+0000", 2024-09-14T06:29:35.344 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.39e-07 2024-09-14T06:29:35.344 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:35.344 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:35.344 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:29:35.344 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.628928+0000", 2024-09-14T06:29:35.344 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.81e-06 2024-09-14T06:29:35.344 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:35.344 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:35.344 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:29:35.344 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.628934+0000", 2024-09-14T06:29:35.344 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.4649999999999999e-06 2024-09-14T06:29:35.345 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:35.345 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:35.345 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:29:35.345 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.628939+0000", 2024-09-14T06:29:35.345 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.3730000000000003e-06 2024-09-14T06:29:35.345 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:35.345 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:35.345 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:29:35.345 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.647265+0000", 2024-09-14T06:29:35.346 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.018326499999999999 2024-09-14T06:29:35.346 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:35.346 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:35.346 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:29:35.346 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.647381+0000", 2024-09-14T06:29:35.346 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.000115549 2024-09-14T06:29:35.346 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:35.346 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:35.346 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:29:35.346 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.647496+0000", 2024-09-14T06:29:35.346 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.000115202 2024-09-14T06:29:35.346 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:35.346 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:29:35.346 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:35.346 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:35.347 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:35.347 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:1491 3.a 3:57c28b0f:::rbd_data.112d7dfbd766.000000000000000a:5 [read 323584~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-14T06:29:35.347 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:08.628964+0000", 2024-09-14T06:29:35.347 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 26.682553555999998, 2024-09-14T06:29:35.347 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.018753204999999998, 2024-09-14T06:29:35.347 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:29:35.347 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:29:35.347 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:29:35.347 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:29:35.347 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:29:35.347 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:35.347 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 1491 2024-09-14T06:29:35.348 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:35.348 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:29:35.348 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:35.348 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:29:35.348 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.628964+0000", 2024-09-14T06:29:35.348 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:35.348 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:35.348 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:35.348 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:29:35.348 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.628964+0000", 2024-09-14T06:29:35.348 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:35.348 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:35.348 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:35.348 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:29:35.349 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.628964+0000", 2024-09-14T06:29:35.349 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.3600000000000004e-07 2024-09-14T06:29:35.349 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:35.349 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:35.349 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:29:35.349 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.628966+0000", 2024-09-14T06:29:35.349 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.119e-06 2024-09-14T06:29:35.349 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:35.349 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:35.349 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:29:35.349 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.628972+0000", 2024-09-14T06:29:35.350 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.0929999999999999e-06 2024-09-14T06:29:35.350 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:35.350 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:35.350 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:29:35.350 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.628977+0000", 2024-09-14T06:29:35.350 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.5360000000000003e-06 2024-09-14T06:29:35.350 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:35.350 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:35.350 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:29:35.350 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.647527+0000", 2024-09-14T06:29:35.350 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.018550433000000002 2024-09-14T06:29:35.350 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:35.350 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:35.351 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:29:35.351 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.647614+0000", 2024-09-14T06:29:35.351 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.6757999999999998e-05 2024-09-14T06:29:35.351 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:35.351 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:35.351 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:29:35.351 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.647717+0000", 2024-09-14T06:29:35.351 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00010273 2024-09-14T06:29:35.351 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:35.351 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:29:35.351 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:35.351 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:35.351 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:35.351 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:1492 3.a 3:57c28b0f:::rbd_data.112d7dfbd766.000000000000000a:5 [read 339968~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-14T06:29:35.352 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:08.628983+0000", 2024-09-14T06:29:35.352 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 26.682533873000001, 2024-09-14T06:29:35.352 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.018937253000000001, 2024-09-14T06:29:35.352 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:29:35.352 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:29:35.352 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:29:35.352 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:29:35.352 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:29:35.352 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:35.352 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 1492 2024-09-14T06:29:35.352 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:35.352 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:29:35.352 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:35.353 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:29:35.353 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.628983+0000", 2024-09-14T06:29:35.353 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:35.353 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:35.353 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:35.353 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:29:35.353 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.628983+0000", 2024-09-14T06:29:35.353 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:35.353 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:35.353 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:35.353 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:29:35.353 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.628984+0000", 2024-09-14T06:29:35.354 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.0100000000000001e-07 2024-09-14T06:29:35.354 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:35.354 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:35.354 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:29:35.354 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.628985+0000", 2024-09-14T06:29:35.354 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.561e-06 2024-09-14T06:29:35.354 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:35.354 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:35.354 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:29:35.354 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.628990+0000", 2024-09-14T06:29:35.354 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.1540000000000001e-06 2024-09-14T06:29:35.355 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:35.355 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:35.355 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:29:35.355 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.628994+0000", 2024-09-14T06:29:35.355 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.8890000000000004e-06 2024-09-14T06:29:35.355 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:35.355 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:35.355 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:29:35.355 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.647745+0000", 2024-09-14T06:29:35.355 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.018751213999999999 2024-09-14T06:29:35.355 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:35.355 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:35.355 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:29:35.356 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.647824+0000", 2024-09-14T06:29:35.356 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.8331999999999995e-05 2024-09-14T06:29:35.356 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:35.356 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:35.356 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:29:35.356 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.647921+0000", 2024-09-14T06:29:35.356 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.6801999999999999e-05 2024-09-14T06:29:35.356 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:35.356 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:29:35.356 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:35.356 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:35.356 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:35.356 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:1493 3.a 3:57c28b0f:::rbd_data.112d7dfbd766.000000000000000a:5 [read 356352~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-14T06:29:35.357 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:08.628999+0000", 2024-09-14T06:29:35.357 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 26.682517957999998, 2024-09-14T06:29:35.357 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.019127128, 2024-09-14T06:29:35.357 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:29:35.357 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:29:35.357 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:29:35.357 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:29:35.357 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:29:35.357 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:35.357 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 1493 2024-09-14T06:29:35.357 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:35.358 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:29:35.358 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:35.358 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:29:35.358 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.628999+0000", 2024-09-14T06:29:35.358 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:35.358 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:35.358 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:35.358 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:29:35.358 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.628999+0000", 2024-09-14T06:29:35.358 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:35.358 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:35.358 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:35.359 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:29:35.359 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.629000+0000", 2024-09-14T06:29:35.359 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.2500000000000001e-07 2024-09-14T06:29:35.359 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:35.359 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:35.359 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:29:35.359 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.629001+0000", 2024-09-14T06:29:35.359 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.511e-06 2024-09-14T06:29:35.359 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:35.359 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:35.359 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:29:35.359 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.629006+0000", 2024-09-14T06:29:35.359 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.1420000000000001e-06 2024-09-14T06:29:35.360 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:35.360 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:35.360 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:29:35.360 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.629010+0000", 2024-09-14T06:29:35.360 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.8120000000000001e-06 2024-09-14T06:29:35.360 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:35.360 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:35.360 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:29:35.360 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.647950+0000", 2024-09-14T06:29:35.360 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.018939568 2024-09-14T06:29:35.360 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:35.360 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:35.360 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:29:35.360 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.648032+0000", 2024-09-14T06:29:35.360 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.2132000000000006e-05 2024-09-14T06:29:35.361 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:35.361 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:35.361 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:29:35.361 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.648126+0000", 2024-09-14T06:29:35.361 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.4637999999999998e-05 2024-09-14T06:29:35.361 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:35.361 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:29:35.361 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:35.361 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:35.361 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:35.361 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:1494 3.a 3:57c28b0f:::rbd_data.112d7dfbd766.000000000000000a:5 [read 372736~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-14T06:29:35.361 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:08.629017+0000", 2024-09-14T06:29:35.361 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 26.682499897, 2024-09-14T06:29:35.361 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.019310414000000001, 2024-09-14T06:29:35.362 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:29:35.362 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:29:35.362 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:29:35.362 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:29:35.362 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:29:35.362 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:35.362 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 1494 2024-09-14T06:29:35.362 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:35.362 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:29:35.362 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:35.362 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:29:35.362 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.629017+0000", 2024-09-14T06:29:35.362 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:35.362 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:35.362 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:35.362 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:29:35.363 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.629017+0000", 2024-09-14T06:29:35.363 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:35.363 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:35.363 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:35.363 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:29:35.363 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.629018+0000", 2024-09-14T06:29:35.363 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.9000000000000002e-07 2024-09-14T06:29:35.363 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:35.363 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:35.363 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:29:35.363 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.629019+0000", 2024-09-14T06:29:35.363 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.5230000000000001e-06 2024-09-14T06:29:35.363 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:35.364 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:35.364 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:29:35.364 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.629024+0000", 2024-09-14T06:29:35.364 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.2299999999999999e-06 2024-09-14T06:29:35.364 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:35.364 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:35.364 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:29:35.364 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.629028+0000", 2024-09-14T06:29:35.364 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.7749999999999999e-06 2024-09-14T06:29:35.364 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:35.364 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:35.365 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:29:35.365 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.648156+0000", 2024-09-14T06:29:35.365 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.019128045999999999 2024-09-14T06:29:35.365 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:35.365 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:35.365 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:29:35.365 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.648238+0000", 2024-09-14T06:29:35.365 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.1877000000000002e-05 2024-09-14T06:29:35.365 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:35.365 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:35.365 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:29:35.365 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.648328+0000", 2024-09-14T06:29:35.366 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.9573000000000005e-05 2024-09-14T06:29:35.366 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:35.366 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:29:35.366 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:35.366 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:35.366 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:35.366 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:1495 3.a 3:57c28b0f:::rbd_data.112d7dfbd766.000000000000000a:5 [read 389120~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-14T06:29:35.366 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:08.629035+0000", 2024-09-14T06:29:35.366 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 26.682481873, 2024-09-14T06:29:35.366 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.019486507, 2024-09-14T06:29:35.366 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:29:35.366 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:29:35.366 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:29:35.366 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:29:35.366 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:29:35.367 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:35.367 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 1495 2024-09-14T06:29:35.367 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:35.367 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:29:35.367 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:35.367 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:29:35.367 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.629035+0000", 2024-09-14T06:29:35.367 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:35.367 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:35.367 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:35.367 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:29:35.367 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.629035+0000", 2024-09-14T06:29:35.367 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:35.367 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:35.368 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:35.368 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:29:35.368 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.629036+0000", 2024-09-14T06:29:35.368 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.2000000000000001e-07 2024-09-14T06:29:35.368 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:35.368 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:35.368 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:29:35.368 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.629037+0000", 2024-09-14T06:29:35.368 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.4300000000000001e-06 2024-09-14T06:29:35.368 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:35.368 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:35.368 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:29:35.368 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.629043+0000", 2024-09-14T06:29:35.368 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.575e-06 2024-09-14T06:29:35.369 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:35.369 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:35.369 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:29:35.369 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.629047+0000", 2024-09-14T06:29:35.369 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.9179999999999996e-06 2024-09-14T06:29:35.369 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:35.369 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:35.369 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:29:35.369 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.648356+0000", 2024-09-14T06:29:35.369 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.019309862000000001 2024-09-14T06:29:35.369 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:35.369 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:35.369 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:29:35.369 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.648433+0000", 2024-09-14T06:29:35.370 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.6973999999999999e-05 2024-09-14T06:29:35.370 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:35.370 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:35.370 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:29:35.370 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.648522+0000", 2024-09-14T06:29:35.370 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.8428000000000001e-05 2024-09-14T06:29:35.370 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:35.370 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:29:35.370 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:35.370 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:35.370 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:35.371 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:1496 3.a 3:57c28b0f:::rbd_data.112d7dfbd766.000000000000000a:5 [read 405504~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-14T06:29:35.371 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:08.629086+0000", 2024-09-14T06:29:35.371 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 26.682430829000001, 2024-09-14T06:29:35.371 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.019618813999999998, 2024-09-14T06:29:35.371 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:29:35.371 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:29:35.371 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:29:35.371 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:29:35.371 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:29:35.371 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:35.371 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 1496 2024-09-14T06:29:35.371 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:35.371 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:29:35.371 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:35.372 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:29:35.372 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.629086+0000", 2024-09-14T06:29:35.372 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:35.372 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:35.372 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:35.372 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:29:35.372 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.629086+0000", 2024-09-14T06:29:35.372 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:35.372 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:35.372 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:35.372 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:29:35.372 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.629087+0000", 2024-09-14T06:29:35.373 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.1399999999999996e-07 2024-09-14T06:29:35.373 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:35.373 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:35.373 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:29:35.373 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.629100+0000", 2024-09-14T06:29:35.373 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.2748999999999999e-05 2024-09-14T06:29:35.373 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:35.373 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:35.373 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:29:35.373 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.629111+0000", 2024-09-14T06:29:35.373 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.1531e-05 2024-09-14T06:29:35.373 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:35.373 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:35.373 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:29:35.374 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.629131+0000", 2024-09-14T06:29:35.374 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.9341999999999999e-05 2024-09-14T06:29:35.374 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:35.374 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:35.374 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:29:35.374 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.648550+0000", 2024-09-14T06:29:35.374 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.019418916000000001 2024-09-14T06:29:35.374 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:35.374 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:35.374 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:29:35.374 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.648616+0000", 2024-09-14T06:29:35.374 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.6359000000000006e-05 2024-09-14T06:29:35.374 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:35.374 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:35.374 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:29:35.375 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.648705+0000", 2024-09-14T06:29:35.375 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.9103000000000004e-05 2024-09-14T06:29:35.375 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:35.375 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:29:35.375 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:35.375 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:35.375 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:35.375 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:1497 3.a 3:57c28b0f:::rbd_data.112d7dfbd766.000000000000000a:5 [read 421888~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-14T06:29:35.375 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:08.629145+0000", 2024-09-14T06:29:35.375 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 26.682372051000002, 2024-09-14T06:29:35.375 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.019744725000000001, 2024-09-14T06:29:35.375 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:29:35.376 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:29:35.376 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:29:35.376 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:29:35.376 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:29:35.376 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:35.376 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 1497 2024-09-14T06:29:35.376 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:35.376 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:29:35.376 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:35.376 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:29:35.376 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.629145+0000", 2024-09-14T06:29:35.377 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:35.377 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:35.377 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:35.377 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:29:35.377 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.629145+0000", 2024-09-14T06:29:35.377 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:35.377 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:35.377 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:35.377 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:29:35.377 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.629146+0000", 2024-09-14T06:29:35.377 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.1699999999999999e-07 2024-09-14T06:29:35.377 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:35.377 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:35.378 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:29:35.378 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.629148+0000", 2024-09-14T06:29:35.378 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.024e-06 2024-09-14T06:29:35.378 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:35.378 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:35.378 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:29:35.378 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.629152+0000", 2024-09-14T06:29:35.378 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.5970000000000002e-06 2024-09-14T06:29:35.378 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:35.378 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:35.378 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:29:35.378 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.629157+0000", 2024-09-14T06:29:35.378 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.8119999999999998e-06 2024-09-14T06:29:35.378 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:35.379 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:35.379 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:29:35.379 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.648734+0000", 2024-09-14T06:29:35.379 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.019576935 2024-09-14T06:29:35.379 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:35.379 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:35.379 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:29:35.379 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.648801+0000", 2024-09-14T06:29:35.379 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.7305999999999996e-05 2024-09-14T06:29:35.379 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:35.380 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:35.380 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:29:35.380 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.648890+0000", 2024-09-14T06:29:35.380 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.8634000000000006e-05 2024-09-14T06:29:35.380 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:35.380 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:29:35.380 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:35.380 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:35.380 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:35.380 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:1498 3.a 3:57c28b0f:::rbd_data.112d7dfbd766.000000000000000a:5 [read 438272~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-14T06:29:35.380 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:08.629167+0000", 2024-09-14T06:29:35.381 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 26.682349986999998, 2024-09-14T06:29:35.381 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.019912138999999999, 2024-09-14T06:29:35.381 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:29:35.381 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:29:35.381 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:29:35.381 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:29:35.381 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:29:35.381 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:35.381 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 1498 2024-09-14T06:29:35.381 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:35.381 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:29:35.381 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:35.382 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:29:35.382 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.629167+0000", 2024-09-14T06:29:35.382 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:35.382 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:35.382 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:35.382 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:29:35.382 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.629167+0000", 2024-09-14T06:29:35.382 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:35.382 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:35.382 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:35.382 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:29:35.383 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.629168+0000", 2024-09-14T06:29:35.383 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.1300000000000001e-07 2024-09-14T06:29:35.383 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:35.383 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:35.383 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:29:35.383 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.629170+0000", 2024-09-14T06:29:35.383 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.7209999999999999e-06 2024-09-14T06:29:35.383 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:35.383 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:35.383 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:29:35.383 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.629174+0000", 2024-09-14T06:29:35.383 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.1459999999999998e-06 2024-09-14T06:29:35.383 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:35.383 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:35.383 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:29:35.384 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.629178+0000", 2024-09-14T06:29:35.384 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.8519999999999997e-06 2024-09-14T06:29:35.384 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:35.384 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:35.384 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:29:35.384 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.648924+0000", 2024-09-14T06:29:35.384 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.019746107999999998 2024-09-14T06:29:35.384 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:35.384 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:35.384 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:29:35.384 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.648995+0000", 2024-09-14T06:29:35.384 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.0809999999999995e-05 2024-09-14T06:29:35.384 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:35.385 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:35.385 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:29:35.385 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.649079+0000", 2024-09-14T06:29:35.385 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.4089000000000001e-05 2024-09-14T06:29:35.385 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:35.385 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:29:35.385 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:35.385 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:35.385 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:35.385 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:1499 3.a 3:57c28b0f:::rbd_data.112d7dfbd766.000000000000000a:5 [read 454656~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-14T06:29:35.386 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:08.629187+0000", 2024-09-14T06:29:35.386 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 26.682330022999999, 2024-09-14T06:29:35.386 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.020072543000000002, 2024-09-14T06:29:35.386 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:29:35.386 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:29:35.386 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:29:35.386 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:29:35.386 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:29:35.386 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:35.386 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 1499 2024-09-14T06:29:35.387 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:35.387 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:29:35.387 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:35.387 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:29:35.387 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.629187+0000", 2024-09-14T06:29:35.387 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:35.387 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:35.387 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:35.387 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:29:35.387 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.629187+0000", 2024-09-14T06:29:35.388 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:35.388 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:35.388 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:35.388 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:29:35.388 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.629188+0000", 2024-09-14T06:29:35.388 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.89e-07 2024-09-14T06:29:35.388 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:35.389 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:35.389 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:29:35.389 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.629190+0000", 2024-09-14T06:29:35.389 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.0169999999999999e-06 2024-09-14T06:29:35.389 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:35.389 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:35.390 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:29:35.390 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.629195+0000", 2024-09-14T06:29:35.390 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.4149999999999998e-06 2024-09-14T06:29:35.390 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:35.390 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:35.390 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:29:35.390 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.629198+0000", 2024-09-14T06:29:35.390 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.605e-06 2024-09-14T06:29:35.390 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:35.390 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:35.390 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:29:35.390 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.649107+0000", 2024-09-14T06:29:35.391 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.019909339000000002 2024-09-14T06:29:35.391 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:35.391 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:35.391 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:29:35.391 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.649178+0000", 2024-09-14T06:29:35.391 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.0809999999999995e-05 2024-09-14T06:29:35.391 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:35.391 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:35.391 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:29:35.391 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.649260+0000", 2024-09-14T06:29:35.391 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.1967999999999999e-05 2024-09-14T06:29:35.391 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:35.391 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:29:35.391 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:35.392 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:35.392 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:35.392 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:1500 3.a 3:57c28b0f:::rbd_data.112d7dfbd766.000000000000000a:5 [read 471040~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-14T06:29:35.392 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:08.629204+0000", 2024-09-14T06:29:35.392 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 26.682313530999998, 2024-09-14T06:29:35.392 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.020473080000000001, 2024-09-14T06:29:35.392 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:29:35.392 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:29:35.392 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:29:35.392 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:29:35.392 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:29:35.392 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:35.392 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 1500 2024-09-14T06:29:35.392 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:35.393 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:29:35.393 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:35.393 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:29:35.393 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.629204+0000", 2024-09-14T06:29:35.393 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:35.393 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:35.393 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:35.393 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:29:35.393 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.629204+0000", 2024-09-14T06:29:35.393 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:35.393 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:35.393 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:35.393 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:29:35.393 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.629204+0000", 2024-09-14T06:29:35.394 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.34e-07 2024-09-14T06:29:35.394 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:35.394 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:35.394 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:29:35.394 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.629206+0000", 2024-09-14T06:29:35.394 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.939e-06 2024-09-14T06:29:35.394 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:35.394 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:35.394 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:29:35.394 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.629212+0000", 2024-09-14T06:29:35.394 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.8669999999999999e-06 2024-09-14T06:29:35.394 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:35.394 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:35.394 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:29:35.395 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.629216+0000", 2024-09-14T06:29:35.395 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.3830000000000002e-06 2024-09-14T06:29:35.395 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:35.395 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:35.395 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:29:35.395 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.649288+0000", 2024-09-14T06:29:35.395 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.020072113999999999 2024-09-14T06:29:35.395 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:35.395 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:35.395 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:29:35.395 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.649355+0000", 2024-09-14T06:29:35.395 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.7218000000000006e-05 2024-09-14T06:29:35.395 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:35.396 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:35.396 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:29:35.396 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.649677+0000", 2024-09-14T06:29:35.396 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00032122500000000002 2024-09-14T06:29:35.396 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:35.396 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:29:35.396 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:35.396 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:35.396 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:35.396 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:1501 3.a 3:57c28b0f:::rbd_data.112d7dfbd766.000000000000000a:5 [read 487424~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-14T06:29:35.397 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:08.629223+0000", 2024-09-14T06:29:35.397 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 26.682294081999999, 2024-09-14T06:29:35.397 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.021370209000000001, 2024-09-14T06:29:35.397 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:29:35.397 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:29:35.397 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:29:35.397 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:29:35.397 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:29:35.397 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:35.397 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 1501 2024-09-14T06:29:35.397 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:35.397 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:29:35.397 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:35.397 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:29:35.398 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.629223+0000", 2024-09-14T06:29:35.398 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:35.398 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:35.398 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:35.398 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:29:35.398 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.629223+0000", 2024-09-14T06:29:35.398 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:35.398 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:35.398 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:35.398 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:29:35.398 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.629223+0000", 2024-09-14T06:29:35.398 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.9900000000000001e-07 2024-09-14T06:29:35.398 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:35.398 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:35.399 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:29:35.399 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.629231+0000", 2024-09-14T06:29:35.399 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.9179999999999994e-06 2024-09-14T06:29:35.399 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:35.399 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:35.399 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:29:35.399 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.629238+0000", 2024-09-14T06:29:35.399 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.421e-06 2024-09-14T06:29:35.399 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:35.399 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:35.399 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:29:35.399 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.629242+0000", 2024-09-14T06:29:35.399 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.5419999999999999e-06 2024-09-14T06:29:35.399 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:35.400 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:35.400 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:29:35.400 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.650284+0000", 2024-09-14T06:29:35.400 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.021041131000000001 2024-09-14T06:29:35.400 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:35.400 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:35.400 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:29:35.400 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.650361+0000", 2024-09-14T06:29:35.400 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.7787999999999995e-05 2024-09-14T06:29:35.400 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:35.400 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:35.400 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:29:35.400 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.650593+0000", 2024-09-14T06:29:35.401 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00023201 2024-09-14T06:29:35.401 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:35.401 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:29:35.401 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:35.401 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:35.401 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:35.401 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:1502 3.a 3:57c28b0f:::rbd_data.112d7dfbd766.000000000000000a:5 [read 503808~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-14T06:29:35.401 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:08.629249+0000", 2024-09-14T06:29:35.401 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 26.682267836000001, 2024-09-14T06:29:35.401 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.021713217, 2024-09-14T06:29:35.401 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:29:35.401 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:29:35.401 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:29:35.401 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:29:35.402 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:29:35.402 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:35.402 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 1502 2024-09-14T06:29:35.402 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:35.402 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:29:35.402 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:35.402 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:29:35.402 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.629249+0000", 2024-09-14T06:29:35.402 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:35.402 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:35.402 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:35.403 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:29:35.403 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.629249+0000", 2024-09-14T06:29:35.403 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:35.403 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:35.403 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:35.403 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:29:35.403 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.629250+0000", 2024-09-14T06:29:35.403 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.03e-07 2024-09-14T06:29:35.403 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:35.403 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:35.403 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:29:35.403 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.629252+0000", 2024-09-14T06:29:35.403 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.995e-06 2024-09-14T06:29:35.403 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:35.403 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:35.404 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:29:35.404 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.629257+0000", 2024-09-14T06:29:35.404 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.7010000000000001e-06 2024-09-14T06:29:35.404 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:35.404 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:35.404 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:29:35.404 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.629262+0000", 2024-09-14T06:29:35.404 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.16e-06 2024-09-14T06:29:35.404 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:35.404 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:35.404 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:29:35.404 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.650629+0000", 2024-09-14T06:29:35.404 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.021367684000000001 2024-09-14T06:29:35.405 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:35.405 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:35.405 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:29:35.405 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.650718+0000", 2024-09-14T06:29:35.405 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.8832000000000006e-05 2024-09-14T06:29:35.405 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:35.405 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:35.405 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:29:35.405 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.650963+0000", 2024-09-14T06:29:35.405 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00024444199999999999 2024-09-14T06:29:35.405 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:35.405 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:29:35.405 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:35.406 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:35.406 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:35.406 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:1503 3.a 3:57c28b0f:::rbd_data.112d7dfbd766.000000000000000a:5 [read 520192~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-14T06:29:35.406 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:08.629268+0000", 2024-09-14T06:29:35.406 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 26.682249294999998, 2024-09-14T06:29:35.406 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.022244885999999998, 2024-09-14T06:29:35.406 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:29:35.406 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:29:35.406 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:29:35.406 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:29:35.406 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:29:35.406 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:35.406 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 1503 2024-09-14T06:29:35.406 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:35.407 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:29:35.407 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:35.407 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:29:35.407 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.629268+0000", 2024-09-14T06:29:35.407 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:35.407 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:35.407 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:35.407 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:29:35.407 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.629268+0000", 2024-09-14T06:29:35.407 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:35.407 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:35.407 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:35.407 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:29:35.407 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.629268+0000", 2024-09-14T06:29:35.408 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.6599999999999996e-07 2024-09-14T06:29:35.408 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:35.408 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:35.408 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:29:35.408 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.629270+0000", 2024-09-14T06:29:35.408 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.7129999999999999e-06 2024-09-14T06:29:35.408 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:35.408 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:35.408 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:29:35.408 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.629276+0000", 2024-09-14T06:29:35.409 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.8119999999999996e-06 2024-09-14T06:29:35.409 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:35.409 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:35.409 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:29:35.409 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.629280+0000", 2024-09-14T06:29:35.409 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.4920000000000004e-06 2024-09-14T06:29:35.409 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:35.409 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:35.409 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:29:35.410 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.651039+0000", 2024-09-14T06:29:35.410 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.021758116000000001 2024-09-14T06:29:35.410 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:35.410 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:35.410 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:29:35.410 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.651127+0000", 2024-09-14T06:29:35.410 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.7983000000000001e-05 2024-09-14T06:29:35.410 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:35.410 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:35.410 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:29:35.410 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.651513+0000", 2024-09-14T06:29:35.410 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00038620400000000002 2024-09-14T06:29:35.410 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:35.411 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:29:35.411 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:35.411 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:35.411 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:29:35.411 INFO:teuthology.orchestra.run.smithi138.stdout:} 2024-09-14T06:29:35.412 INFO:teuthology.orchestra.run.smithi094.stdout:588 write 0xa322d84 thru 0xa331194 (0xe411 bytes) 2024-09-14T06:29:35.412 INFO:teuthology.orchestra.run.smithi094.stdout:589 write 0x8b61e61 thru 0x8b69e59 (0x7ff9 bytes) 2024-09-14T06:29:35.413 INFO:teuthology.orchestra.run.smithi094.stdout:590 write 0x82d8c83 thru 0x82e0ecd (0x824b bytes) 2024-09-14T06:29:35.413 INFO:teuthology.orchestra.run.smithi094.stdout:591 write 0xcabf7b2 thru 0xcacba01 (0xc250 bytes) 2024-09-14T06:29:35.413 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-14T06:29:35.414 INFO:tasks.ceph.osd.4.smithi138.stderr:2024-09-14T06:29:35.411+0000 7f6c653c8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33615) UID: 0 2024-09-14T06:29:35.414 INFO:teuthology.orchestra.run.smithi094.stdout:593 trunc from 0xcacba02 to 0x2ef4410 2024-09-14T06:29:35.427 INFO:teuthology.orchestra.run.smithi094.stdout:595 trunc from 0x2ef4410 to 0x599e234 2024-09-14T06:29:35.432 INFO:teuthology.orchestra.run.smithi094.stdout:596 punch from 0x31b40b9 to 0x31bee82, (0xadc9 bytes) 2024-09-14T06:29:35.435 INFO:teuthology.orchestra.run.smithi094.stdout:597 trunc from 0x599e234 to 0x82e6d26 2024-09-14T06:29:35.440 INFO:teuthology.orchestra.run.smithi094.stdout:598 read 0x346ff2f thru 0x34700b5 (0x187 bytes) 2024-09-14T06:29:35.440 INFO:teuthology.orchestra.run.smithi094.stdout:599 write 0x63e8b70 thru 0x63e9f47 (0x13d8 bytes) 2024-09-14T06:29:35.447 INFO:teuthology.orchestra.run.smithi094.stdout:600 trunc from 0x82e6d26 to 0x544046c 2024-09-14T06:29:35.462 INFO:teuthology.orchestra.run.smithi094.stdout:602 writesame 0x85ead1f thru 0x85ed448 data_size 0x9(0x272a bytes) 2024-09-14T06:29:35.462 INFO:teuthology.orchestra.run.smithi094.stdout:603 write 0xa910af thru 0xa9a7f8 (0x974a bytes) 2024-09-14T06:29:35.464 INFO:teuthology.orchestra.run.smithi094.stdout:604 write 0x236d575 thru 0x2372150 (0x4bdc bytes) 2024-09-14T06:29:35.475 INFO:teuthology.orchestra.run.smithi094.stdout:606 trunc from 0x85ed449 to 0x21a39ac 2024-09-14T06:29:35.486 INFO:teuthology.orchestra.run.smithi094.stdout:608 punch from 0x4ea926 to 0x4f6e59, (0xc533 bytes) 2024-09-14T06:29:35.486 INFO:teuthology.orchestra.run.smithi094.stdout:609 punch from 0xebbd10 to 0xec68d9, (0xabc9 bytes) 2024-09-14T06:29:35.486 INFO:teuthology.orchestra.run.smithi094.stdout:610 read 0x62a0ec thru 0x62f2e7 (0x51fc bytes) 2024-09-14T06:29:35.498 INFO:teuthology.orchestra.run.smithi094.stdout:611 writesame 0x74a76e9 thru 0x74a9607 data_size 0x1(0x1f1f bytes) 2024-09-14T06:29:35.498 INFO:teuthology.orchestra.run.smithi094.stdout:612 read 0x3e4f677 thru 0x3e5b6eb (0xc075 bytes) 2024-09-14T06:29:35.498 INFO:teuthology.orchestra.run.smithi094.stdout:614 write 0x25ba7b8 thru 0x25c6855 (0xc09e bytes) 2024-09-14T06:29:35.503 INFO:teuthology.orchestra.run.smithi094.stdout:615 read 0x5a7f1c8 thru 0x5a8e95b (0xf794 bytes) 2024-09-14T06:29:35.503 INFO:teuthology.orchestra.run.smithi094.stdout:616 punch from 0x67534c2 to 0x675785a, (0x4398 bytes) 2024-09-14T06:29:35.503 INFO:teuthology.orchestra.run.smithi094.stdout:617 punch from 0x534193e to 0x5349760, (0x7e22 bytes) 2024-09-14T06:29:35.513 INFO:tasks.ceph.osd.3.smithi094.stderr:2024-09-14T06:29:35.511+0000 7feeeafe8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33655) UID: 0 2024-09-14T06:29:35.516 INFO:teuthology.orchestra.run.smithi094.stdout:618 write 0xd84c94d thru 0xd853d4a (0x73fe bytes) 2024-09-14T06:29:35.519 INFO:teuthology.orchestra.run.smithi094.stdout:619 writesame 0x786549 thru 0x78cae9 data_size 0x1(0x65a1 bytes) 2024-09-14T06:29:35.527 INFO:teuthology.orchestra.run.smithi094.stdout:620 punch from 0xcfb0f28 to 0xcfb4210, (0x32e8 bytes) 2024-09-14T06:29:35.527 INFO:teuthology.orchestra.run.smithi094.stdout:622 read 0x485e9d4 thru 0x486bf5b (0xd588 bytes) 2024-09-14T06:29:35.527 INFO:teuthology.orchestra.run.smithi094.stdout:623 write 0x4d7e6f thru 0x4e2b48 (0xacda bytes) 2024-09-14T06:29:35.530 INFO:teuthology.orchestra.run.smithi094.stdout:624 trunc from 0xd853d4b to 0xe845ce6 2024-09-14T06:29:35.536 INFO:teuthology.orchestra.run.smithi094.stdout:626 writesame 0xaddcbb3 thru 0xade4e77 data_size 0x1(0x82c5 bytes) 2024-09-14T06:29:35.546 INFO:teuthology.orchestra.run.smithi094.stdout:627 read 0x794745c thru 0x794ba3e (0x45e3 bytes) 2024-09-14T06:29:35.547 INFO:teuthology.orchestra.run.smithi094.stdout:630 read 0xcd10a5 thru 0xcdbacb (0xaa27 bytes) 2024-09-14T06:29:35.547 INFO:teuthology.orchestra.run.smithi094.stdout:631 write 0x199a6f2 thru 0x199da26 (0x3335 bytes) 2024-09-14T06:29:35.550 INFO:teuthology.orchestra.run.smithi094.stdout:632 read 0xe6925ed thru 0xe693b8b (0x159f bytes) 2024-09-14T06:29:35.550 INFO:teuthology.orchestra.run.smithi094.stdout:634 writesame 0x9e3a784 thru 0x9e42100 data_size 0x7(0x797d bytes) 2024-09-14T06:29:35.555 INFO:teuthology.orchestra.run.smithi094.stdout:636 read 0xc3d923f thru 0xc3e0513 (0x72d5 bytes) 2024-09-14T06:29:35.555 INFO:teuthology.orchestra.run.smithi094.stdout:637 write 0x4d6b0f7 thru 0x4d6f976 (0x4880 bytes) 2024-09-14T06:29:35.559 INFO:teuthology.orchestra.run.smithi094.stdout:639 punch from 0x2a73994 to 0x2a82f3b, (0xf5a7 bytes) 2024-09-14T06:29:35.559 INFO:teuthology.orchestra.run.smithi094.stdout:640 punch from 0xbdfb10f to 0xbe056fe, (0xa5ef bytes) 2024-09-14T06:29:35.562 INFO:teuthology.orchestra.run.smithi094.stdout:641 trunc from 0xe845ce6 to 0xbaed641 2024-09-14T06:29:35.568 INFO:teuthology.orchestra.run.smithi138.stdout:{ 2024-09-14T06:29:35.568 INFO:teuthology.orchestra.run.smithi138.stdout: "ops": [], 2024-09-14T06:29:35.568 INFO:teuthology.orchestra.run.smithi138.stdout: "num_ops": 0 2024-09-14T06:29:35.568 INFO:teuthology.orchestra.run.smithi138.stdout:} 2024-09-14T06:29:35.570 INFO:teuthology.orchestra.run.smithi094.stdout:642 read 0xbe1638 thru 0xbee61f (0xcfe8 bytes) 2024-09-14T06:29:35.571 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-14T06:29:35.571 INFO:teuthology.orchestra.run.smithi094.stdout:645 write 0x1360c98 thru 0x136ab10 (0x9e79 bytes) 2024-09-14T06:29:35.573 INFO:teuthology.orchestra.run.smithi094.stdout:646 read 0x43c7524 thru 0x43c8b04 (0x15e1 bytes) 2024-09-14T06:29:35.573 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-14T06:29:35.573 INFO:teuthology.orchestra.run.smithi094.stdout:648 punch from 0x3959a9a to 0x3965fb5, (0xc51b bytes) 2024-09-14T06:29:35.573 INFO:teuthology.orchestra.run.smithi094.stdout:650 writesame 0x480b6bf thru 0x481b3fc data_size 0xf(0xfd3e bytes) 2024-09-14T06:29:35.578 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-14T06:29:35.580 INFO:teuthology.orchestra.run.smithi094.stdout:651 punch from 0x612159f to 0x612bac3, (0xa524 bytes) 2024-09-14T06:29:35.580 INFO:teuthology.orchestra.run.smithi094.stdout:652 write 0x5af5679 thru 0x5af8e10 (0x3798 bytes) 2024-09-14T06:29:35.581 INFO:teuthology.orchestra.run.smithi094.stdout:653 punch from 0x864fa31 to 0x8651abb, (0x208a bytes) 2024-09-14T06:29:35.582 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-14T06:29:35.582 INFO:teuthology.orchestra.run.smithi094.stdout:656 punch from 0xdae1b5 to 0xdba763, (0xc5ae bytes) 2024-09-14T06:29:35.582 INFO:teuthology.orchestra.run.smithi094.stdout:657 write 0x9c49eee thru 0x9c4b62c (0x173f bytes) 2024-09-14T06:29:35.584 INFO:teuthology.orchestra.run.smithi094.stdout:659 writesame 0xd0edb6 thru 0xd1d051 data_size 0x2(0xe29c bytes) 2024-09-14T06:29:35.590 INFO:teuthology.orchestra.run.smithi094.stdout:662 punch from 0x53a2794 to 0x53aa14e, (0x79ba bytes) 2024-09-14T06:29:35.597 INFO:teuthology.orchestra.run.smithi094.stdout:665 trunc from 0xbaed641 to 0x5378cbb 2024-09-14T06:29:35.609 INFO:teuthology.orchestra.run.smithi094.stdout:667 writesame 0x2d2b471 thru 0x2d2f468 data_size 0x59(0x3ff8 bytes) 2024-09-14T06:29:35.615 INFO:tasks.ceph.osd.3.smithi094.stderr:2024-09-14T06:29:35.613+0000 7feeeafe8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33655) UID: 0 2024-09-14T06:29:35.623 INFO:teuthology.orchestra.run.smithi094.stdout:668 writesame 0x93600c6 thru 0x9363bdf data_size 0x22(0x3b1a bytes) 2024-09-14T06:29:35.623 INFO:teuthology.orchestra.run.smithi094.stdout:669 write 0x6578ffb thru 0x65812a7 (0x82ad bytes) 2024-09-14T06:29:35.635 INFO:teuthology.orchestra.run.smithi094.stdout:670 write 0xb172346 thru 0xb177f56 (0x5c11 bytes) 2024-09-14T06:29:35.638 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-14T06:29:35.639 INFO:teuthology.orchestra.run.smithi094.stdout:672 write 0x75bd99f thru 0x75be76b (0xdcd bytes) 2024-09-14T06:29:35.643 INFO:teuthology.orchestra.run.smithi094.stdout:674 trunc from 0xb177f57 to 0xe4803ff 2024-09-14T06:29:35.649 INFO:teuthology.orchestra.run.smithi094.stdout:675 writesame 0x243ef09 thru 0x244169c data_size 0x22(0x2794 bytes) 2024-09-14T06:29:35.652 INFO:teuthology.orchestra.run.smithi094.stdout:676 read 0xe119af6 thru 0xe12102d (0x7538 bytes) 2024-09-14T06:29:35.652 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-14T06:29:35.652 INFO:teuthology.orchestra.run.smithi094.stdout:678 read 0x8adf9ee thru 0x8aec445 (0xca58 bytes) 2024-09-14T06:29:35.666 INFO:teuthology.orchestra.run.smithi094.stdout:679 trunc from 0xe4803ff to 0x267ee01 2024-09-14T06:29:35.679 INFO:teuthology.orchestra.run.smithi094.stdout:681 punch from 0x20bc558 to 0x20c6ecc, (0xa974 bytes) 2024-09-14T06:29:35.689 INFO:teuthology.orchestra.run.smithi138.stdout:{ 2024-09-14T06:29:35.689 INFO:teuthology.orchestra.run.smithi138.stdout: "ops": [], 2024-09-14T06:29:35.689 INFO:teuthology.orchestra.run.smithi138.stdout: "complaint_time": 30, 2024-09-14T06:29:35.689 INFO:teuthology.orchestra.run.smithi138.stdout: "num_blocked_ops": 0 2024-09-14T06:29:35.690 INFO:teuthology.orchestra.run.smithi138.stdout:} 2024-09-14T06:29:35.697 INFO:teuthology.orchestra.run.smithi094.stdout:683 write 0xcc3495a thru 0xcc37897 (0x2f3e bytes) 2024-09-14T06:29:35.698 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-14T06:29:35.699 INFO:teuthology.orchestra.run.smithi094.stdout:685 writesame 0x13c3b76 thru 0x13c918e data_size 0x1f(0x5619 bytes) 2024-09-14T06:29:35.701 INFO:teuthology.orchestra.run.smithi094.stdout:686 punch from 0x998873 to 0x99f7cd, (0x6f5a bytes) 2024-09-14T06:29:35.701 INFO:teuthology.orchestra.run.smithi094.stdout:687 punch from 0x7de0e66 to 0x7de24a7, (0x1641 bytes) 2024-09-14T06:29:35.701 INFO:teuthology.orchestra.run.smithi094.stdout:689 write 0x5a59ccd thru 0x5a60932 (0x6c66 bytes) 2024-09-14T06:29:35.707 INFO:teuthology.orchestra.run.smithi094.stdout:690 read 0x7b6dd4 thru 0x7bccba (0x5ee7 bytes) 2024-09-14T06:29:35.708 INFO:teuthology.orchestra.run.smithi094.stdout:692 write 0x2113be0 thru 0x211792b (0x3d4c bytes) 2024-09-14T06:29:35.710 INFO:teuthology.orchestra.run.smithi094.stdout:693 read 0x78c62ab thru 0x78d4a4d (0xe7a3 bytes) 2024-09-14T06:29:35.710 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-14T06:29:35.711 INFO:teuthology.orchestra.run.smithi094.stdout:696 trunc from 0xcc37898 to 0xdc40215 2024-09-14T06:29:35.714 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-14T06:29:35.715 INFO:tasks.ceph.osd.3.smithi094.stderr:2024-09-14T06:29:35.713+0000 7feeeafe8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33655) UID: 0 2024-09-14T06:29:35.717 INFO:teuthology.orchestra.run.smithi094.stdout:697 writesame 0x590d316 thru 0x590ea5e data_size 0x1(0x1749 bytes) 2024-09-14T06:29:35.721 INFO:teuthology.orchestra.run.smithi094.stdout:699 read 0xc4b8ce0 thru 0xc4c65dc (0xd8fd bytes) 2024-09-14T06:29:35.721 INFO:teuthology.orchestra.run.smithi094.stdout:700 write 0x9ecbe5b thru 0x9eccf0d (0x10b3 bytes) 2024-09-14T06:29:35.724 INFO:teuthology.orchestra.run.smithi094.stdout:701 write 0x82d2f66 thru 0x82e1e19 (0xeeb4 bytes) 2024-09-14T06:29:35.730 INFO:teuthology.orchestra.run.smithi094.stdout:702 write 0x2c41515 thru 0x2c4198d (0x479 bytes) 2024-09-14T06:29:35.733 INFO:teuthology.orchestra.run.smithi094.stdout:703 writesame 0xa872b73 thru 0xa87c1e8 data_size 0x1(0x9676 bytes) 2024-09-14T06:29:35.745 INFO:teuthology.orchestra.run.smithi094.stdout:704 punch from 0x8c97efc to 0x8c9d069, (0x516d bytes) 2024-09-14T06:29:35.745 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-14T06:29:35.745 INFO:teuthology.orchestra.run.smithi094.stdout:707 punch from 0x88104f7 to 0x881c9b1, (0xc4ba bytes) 2024-09-14T06:29:35.746 INFO:teuthology.orchestra.run.smithi094.stdout:708 read 0x5bcaa44 thru 0x5bd89ac (0xdf69 bytes) 2024-09-14T06:29:35.746 INFO:teuthology.orchestra.run.smithi094.stdout:710 write 0x9b172cb thru 0x9b1b00a (0x3d40 bytes) 2024-09-14T06:29:35.749 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-14T06:29:35.749 INFO:teuthology.orchestra.run.smithi094.stdout:714 write 0x912b386 thru 0x912d76f (0x23ea bytes) 2024-09-14T06:29:35.752 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-14T06:29:35.752 INFO:teuthology.orchestra.run.smithi094.stdout:716 read 0x9ca9279 thru 0x9cb0085 (0x6e0d bytes) 2024-09-14T06:29:35.752 INFO:teuthology.orchestra.run.smithi094.stdout:717 clone 10 order 22 su 1048576 sc 11 2024-09-14T06:29:35.807 INFO:teuthology.orchestra.run.smithi138.stdout:{ 2024-09-14T06:29:35.807 INFO:teuthology.orchestra.run.smithi138.stdout: "size": 20, 2024-09-14T06:29:35.807 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 600, 2024-09-14T06:29:35.807 INFO:teuthology.orchestra.run.smithi138.stdout: "ops": [ 2024-09-14T06:29:35.807 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:35.807 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:1575 3.15 3:accf422f:::rbd_data.112d7dfbd766.0000000000000001:5 [read 339968~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-14T06:29:35.807 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:08.630645+0000", 2024-09-14T06:29:35.807 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 27.1746859, 2024-09-14T06:29:35.808 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.024511650999999999, 2024-09-14T06:29:35.808 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:29:35.808 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:29:35.808 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:29:35.808 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:29:35.808 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:29:35.808 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:35.808 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 1575 2024-09-14T06:29:35.808 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:35.808 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:29:35.808 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:35.808 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:29:35.809 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.630645+0000", 2024-09-14T06:29:35.809 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:35.809 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:35.809 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:35.809 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:29:35.809 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.630645+0000", 2024-09-14T06:29:35.809 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:35.809 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:35.809 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:35.809 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:29:35.810 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.630645+0000", 2024-09-14T06:29:35.810 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.9400000000000005e-07 2024-09-14T06:29:35.810 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:35.810 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:35.810 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:29:35.810 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.630647+0000", 2024-09-14T06:29:35.810 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.9309999999999998e-06 2024-09-14T06:29:35.810 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:35.810 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:35.810 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:29:35.811 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.630654+0000", 2024-09-14T06:29:35.811 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.55e-06 2024-09-14T06:29:35.811 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:35.811 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:35.811 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:29:35.811 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.630659+0000", 2024-09-14T06:29:35.811 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.0139999999999998e-06 2024-09-14T06:29:35.811 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:35.811 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:35.811 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:29:35.811 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.654964+0000", 2024-09-14T06:29:35.812 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.024305651000000001 2024-09-14T06:29:35.812 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:35.812 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:35.812 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:29:35.812 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.655048+0000", 2024-09-14T06:29:35.812 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.3389999999999999e-05 2024-09-14T06:29:35.812 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:35.812 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:35.812 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:29:35.812 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.655156+0000", 2024-09-14T06:29:35.813 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.000108521 2024-09-14T06:29:35.813 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:35.813 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:29:35.813 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:35.813 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:35.813 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:35.813 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:1576 3.15 3:accf422f:::rbd_data.112d7dfbd766.0000000000000001:5 [read 356352~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-14T06:29:35.813 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:08.630666+0000", 2024-09-14T06:29:35.813 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 27.174664862, 2024-09-14T06:29:35.813 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.024703712999999999, 2024-09-14T06:29:35.814 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:29:35.814 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:29:35.814 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:29:35.814 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:29:35.814 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:29:35.814 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:35.814 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 1576 2024-09-14T06:29:35.814 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:35.814 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:29:35.814 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:35.814 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:29:35.815 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.630666+0000", 2024-09-14T06:29:35.815 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:35.815 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:35.815 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:35.815 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:29:35.815 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.630666+0000", 2024-09-14T06:29:35.815 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:35.815 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:35.815 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:35.815 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:29:35.815 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.630666+0000", 2024-09-14T06:29:35.815 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.8299999999999997e-07 2024-09-14T06:29:35.816 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:35.816 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:35.816 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:29:35.816 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.630668+0000", 2024-09-14T06:29:35.816 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.8789999999999999e-06 2024-09-14T06:29:35.816 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:35.816 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:35.816 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:29:35.816 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.630675+0000", 2024-09-14T06:29:35.816 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.2899999999999997e-06 2024-09-14T06:29:35.816 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:35.817 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:35.817 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:29:35.817 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.630681+0000", 2024-09-14T06:29:35.817 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.2730000000000002e-06 2024-09-14T06:29:35.817 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:35.817 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:35.817 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:29:35.817 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.655182+0000", 2024-09-14T06:29:35.817 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.024501228999999999 2024-09-14T06:29:35.817 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:35.817 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:35.817 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:29:35.818 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.655257+0000", 2024-09-14T06:29:35.818 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.4954000000000001e-05 2024-09-14T06:29:35.818 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:35.818 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:35.818 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:29:35.818 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.655369+0000", 2024-09-14T06:29:35.818 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.000112605 2024-09-14T06:29:35.818 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:35.818 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:29:35.818 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:35.818 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:35.818 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:35.818 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:1577 3.15 3:accf422f:::rbd_data.112d7dfbd766.0000000000000001:5 [read 372736~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-14T06:29:35.819 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:08.630688+0000", 2024-09-14T06:29:35.819 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 27.174642349999999, 2024-09-14T06:29:35.819 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.024927538999999999, 2024-09-14T06:29:35.819 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:29:35.819 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:29:35.819 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:29:35.819 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:29:35.819 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:29:35.819 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:35.819 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 1577 2024-09-14T06:29:35.819 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:35.819 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:29:35.819 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:35.820 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:29:35.820 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.630688+0000", 2024-09-14T06:29:35.820 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:35.820 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:35.820 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:35.820 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:29:35.820 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.630688+0000", 2024-09-14T06:29:35.820 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:35.820 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:35.820 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:35.820 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:29:35.820 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.630689+0000", 2024-09-14T06:29:35.821 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.2199999999999999e-07 2024-09-14T06:29:35.821 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:35.821 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:35.821 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:29:35.821 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.630691+0000", 2024-09-14T06:29:35.821 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.9410000000000001e-06 2024-09-14T06:29:35.821 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:35.821 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:35.821 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:29:35.821 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.630699+0000", 2024-09-14T06:29:35.821 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.0250000000000002e-06 2024-09-14T06:29:35.821 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:35.821 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:35.822 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:29:35.822 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.630704+0000", 2024-09-14T06:29:35.822 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.4349999999999996e-06 2024-09-14T06:29:35.822 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:35.822 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:35.822 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:29:35.822 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.655406+0000", 2024-09-14T06:29:35.822 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.024702279000000001 2024-09-14T06:29:35.822 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:35.822 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:35.822 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:29:35.822 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.655483+0000", 2024-09-14T06:29:35.822 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.6915999999999993e-05 2024-09-14T06:29:35.822 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:35.823 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:35.823 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:29:35.823 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.655616+0000", 2024-09-14T06:29:35.823 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.000132521 2024-09-14T06:29:35.823 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:35.823 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:29:35.823 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:35.823 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:35.823 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:35.823 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:1578 3.15 3:accf422f:::rbd_data.112d7dfbd766.0000000000000001:5 [read 389120~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-14T06:29:35.823 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:08.630711+0000", 2024-09-14T06:29:35.823 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 27.174619878000001, 2024-09-14T06:29:35.823 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.025111108, 2024-09-14T06:29:35.824 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:29:35.824 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:29:35.824 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:29:35.824 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:29:35.824 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:29:35.824 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:35.824 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 1578 2024-09-14T06:29:35.824 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:35.824 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:29:35.824 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:35.824 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:29:35.824 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.630711+0000", 2024-09-14T06:29:35.825 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:35.825 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:35.825 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:35.825 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:29:35.825 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.630711+0000", 2024-09-14T06:29:35.825 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:35.825 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:35.825 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:35.825 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:29:35.825 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.630711+0000", 2024-09-14T06:29:35.825 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.01e-07 2024-09-14T06:29:35.825 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:35.826 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:35.826 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:29:35.826 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.630713+0000", 2024-09-14T06:29:35.826 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.8199999999999999e-06 2024-09-14T06:29:35.826 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:35.826 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:35.826 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:29:35.826 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.630720+0000", 2024-09-14T06:29:35.826 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.3880000000000002e-06 2024-09-14T06:29:35.826 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:35.826 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:35.826 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:29:35.827 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.630726+0000", 2024-09-14T06:29:35.827 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.3480000000000003e-06 2024-09-14T06:29:35.827 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:35.827 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:35.827 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:29:35.827 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.655646+0000", 2024-09-14T06:29:35.827 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.024920312999999999 2024-09-14T06:29:35.827 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:35.827 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:35.827 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:29:35.827 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.655724+0000", 2024-09-14T06:29:35.827 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.7589000000000005e-05 2024-09-14T06:29:35.827 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:35.828 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:35.828 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:29:35.828 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.655822+0000", 2024-09-14T06:29:35.828 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.8249000000000001e-05 2024-09-14T06:29:35.828 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:35.828 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:29:35.828 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:35.828 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:35.828 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:35.828 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:1579 3.15 3:accf422f:::rbd_data.112d7dfbd766.0000000000000001:5 [read 405504~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-14T06:29:35.828 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:08.630732+0000", 2024-09-14T06:29:35.829 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 27.17459856, 2024-09-14T06:29:35.829 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.025303169, 2024-09-14T06:29:35.829 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:29:35.829 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:29:35.829 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:29:35.829 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:29:35.829 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:29:35.829 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:35.829 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 1579 2024-09-14T06:29:35.829 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:35.829 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:29:35.829 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:35.830 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:29:35.830 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.630732+0000", 2024-09-14T06:29:35.830 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:35.830 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:35.830 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:35.830 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:29:35.830 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.630732+0000", 2024-09-14T06:29:35.830 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:35.830 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:35.830 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:35.830 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:29:35.830 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.630732+0000", 2024-09-14T06:29:35.831 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.5199999999999998e-07 2024-09-14T06:29:35.831 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:35.831 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:35.831 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:29:35.831 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.630741+0000", 2024-09-14T06:29:35.831 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.1549999999999996e-06 2024-09-14T06:29:35.831 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:35.831 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:35.831 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:29:35.831 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.630748+0000", 2024-09-14T06:29:35.831 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.6429999999999998e-06 2024-09-14T06:29:35.831 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:35.832 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:35.832 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:29:35.832 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.630753+0000", 2024-09-14T06:29:35.832 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.0579999999999997e-06 2024-09-14T06:29:35.832 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:35.832 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:35.832 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:29:35.832 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.655858+0000", 2024-09-14T06:29:35.832 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.025104774999999999 2024-09-14T06:29:35.832 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:35.832 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:35.832 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:29:35.833 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.655942+0000", 2024-09-14T06:29:35.833 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.4129999999999996e-05 2024-09-14T06:29:35.833 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:35.833 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:35.833 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:29:35.833 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.656035+0000", 2024-09-14T06:29:35.833 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.3055999999999998e-05 2024-09-14T06:29:35.833 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:35.833 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:29:35.833 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:35.833 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:35.833 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:35.834 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:1580 3.15 3:accf422f:::rbd_data.112d7dfbd766.0000000000000001:5 [read 421888~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-14T06:29:35.834 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:08.630762+0000", 2024-09-14T06:29:35.834 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 27.174568782000001, 2024-09-14T06:29:35.834 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.025487388999999999, 2024-09-14T06:29:35.834 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:29:35.834 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:29:35.834 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:29:35.834 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:29:35.834 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:29:35.834 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:35.834 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 1580 2024-09-14T06:29:35.834 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:35.834 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:29:35.835 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:35.835 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:29:35.835 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.630762+0000", 2024-09-14T06:29:35.835 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:35.835 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:35.835 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:35.835 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:29:35.835 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.630762+0000", 2024-09-14T06:29:35.835 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:35.835 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:35.835 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:35.835 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:29:35.835 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.630762+0000", 2024-09-14T06:29:35.836 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.2300000000000002e-07 2024-09-14T06:29:35.836 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:35.836 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:35.836 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:29:35.836 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.630764+0000", 2024-09-14T06:29:35.836 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.99e-06 2024-09-14T06:29:35.836 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:35.836 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:35.836 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:29:35.836 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.630771+0000", 2024-09-14T06:29:35.836 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.9349999999999997e-06 2024-09-14T06:29:35.836 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:35.837 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:35.837 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:29:35.837 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.630776+0000", 2024-09-14T06:29:35.837 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.1710000000000001e-06 2024-09-14T06:29:35.837 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:35.837 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:35.837 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:29:35.837 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.656068+0000", 2024-09-14T06:29:35.837 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.025291905999999999 2024-09-14T06:29:35.837 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:35.837 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:35.837 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:29:35.837 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.656151+0000", 2024-09-14T06:29:35.838 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.3209999999999998e-05 2024-09-14T06:29:35.838 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:35.838 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:35.838 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:29:35.838 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.656249+0000", 2024-09-14T06:29:35.838 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.7754e-05 2024-09-14T06:29:35.838 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:35.838 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:29:35.838 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:35.838 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:35.838 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:35.838 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:1581 3.15 3:accf422f:::rbd_data.112d7dfbd766.0000000000000001:5 [read 438272~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-14T06:29:35.838 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:08.630802+0000", 2024-09-14T06:29:35.839 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 27.174528131999999, 2024-09-14T06:29:35.839 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.025670063, 2024-09-14T06:29:35.839 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:29:35.839 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:29:35.839 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:29:35.839 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:29:35.839 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:29:35.839 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:35.839 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 1581 2024-09-14T06:29:35.839 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:35.839 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:29:35.839 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:35.839 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:29:35.839 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.630802+0000", 2024-09-14T06:29:35.840 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:35.840 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:35.840 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:35.840 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:29:35.840 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.630802+0000", 2024-09-14T06:29:35.840 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:35.840 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:35.840 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:35.840 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:29:35.840 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.630803+0000", 2024-09-14T06:29:35.840 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.3099999999999997e-07 2024-09-14T06:29:35.840 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:35.841 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:35.841 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:29:35.841 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.630806+0000", 2024-09-14T06:29:35.841 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.7319999999999998e-06 2024-09-14T06:29:35.841 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:35.841 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:35.841 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:29:35.841 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.630813+0000", 2024-09-14T06:29:35.841 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.0840000000000003e-06 2024-09-14T06:29:35.841 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:35.841 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:35.841 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:29:35.842 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.630826+0000", 2024-09-14T06:29:35.842 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.3104e-05 2024-09-14T06:29:35.842 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:35.842 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:35.842 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:29:35.842 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.656349+0000", 2024-09-14T06:29:35.842 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.025523113 2024-09-14T06:29:35.842 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:35.842 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:35.842 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:29:35.842 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.656400+0000", 2024-09-14T06:29:35.842 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.1156000000000003e-05 2024-09-14T06:29:35.843 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:35.843 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:35.843 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:29:35.843 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.656472+0000", 2024-09-14T06:29:35.843 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.2243000000000006e-05 2024-09-14T06:29:35.843 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:35.843 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:29:35.843 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:35.843 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:35.843 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:35.843 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:1582 3.15 3:accf422f:::rbd_data.112d7dfbd766.0000000000000001:5 [read 454656~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-14T06:29:35.843 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:08.630838+0000", 2024-09-14T06:29:35.844 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 27.174492107999999, 2024-09-14T06:29:35.844 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.025956396999999999, 2024-09-14T06:29:35.844 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:29:35.844 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:29:35.844 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:29:35.844 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:29:35.844 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:29:35.844 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:35.844 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 1582 2024-09-14T06:29:35.844 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:35.844 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:29:35.844 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:35.845 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:29:35.845 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.630838+0000", 2024-09-14T06:29:35.845 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:35.845 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:35.845 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:35.845 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:29:35.845 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.630838+0000", 2024-09-14T06:29:35.845 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:35.845 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:35.845 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:35.845 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:29:35.846 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.630839+0000", 2024-09-14T06:29:35.846 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.8400000000000004e-07 2024-09-14T06:29:35.846 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:35.846 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:35.846 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:29:35.846 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.630846+0000", 2024-09-14T06:29:35.846 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.4599999999999997e-06 2024-09-14T06:29:35.846 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:35.846 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:35.846 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:29:35.846 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.630854+0000", 2024-09-14T06:29:35.847 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.3139999999999998e-06 2024-09-14T06:29:35.847 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:35.847 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:35.847 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:29:35.847 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.630865+0000", 2024-09-14T06:29:35.847 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.0988999999999999e-05 2024-09-14T06:29:35.847 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:35.847 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:35.847 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:29:35.847 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.656598+0000", 2024-09-14T06:29:35.847 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.025733335 2024-09-14T06:29:35.847 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:35.848 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:35.848 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:29:35.848 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.656682+0000", 2024-09-14T06:29:35.848 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.4000999999999998e-05 2024-09-14T06:29:35.848 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:35.848 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:35.848 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:29:35.848 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.656795+0000", 2024-09-14T06:29:35.848 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00011271400000000001 2024-09-14T06:29:35.848 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:35.848 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:29:35.848 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:35.849 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:35.849 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:35.849 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:1583 3.15 3:accf422f:::rbd_data.112d7dfbd766.0000000000000001:5 [read 471040~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-14T06:29:35.849 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:08.630873+0000", 2024-09-14T06:29:35.849 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 27.174457239999999, 2024-09-14T06:29:35.849 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.026337467, 2024-09-14T06:29:35.849 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:29:35.849 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:29:35.849 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:29:35.849 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:29:35.849 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:29:35.849 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:35.850 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 1583 2024-09-14T06:29:35.850 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:35.850 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:29:35.850 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:35.850 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:29:35.850 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.630873+0000", 2024-09-14T06:29:35.850 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:35.850 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:35.850 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:35.850 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:29:35.850 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.630873+0000", 2024-09-14T06:29:35.850 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:35.851 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:35.851 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:35.851 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:29:35.851 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.630874+0000", 2024-09-14T06:29:35.851 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.2500000000000005e-07 2024-09-14T06:29:35.851 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:35.851 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:35.851 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:29:35.851 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.630876+0000", 2024-09-14T06:29:35.851 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.2689999999999998e-06 2024-09-14T06:29:35.851 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:35.851 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:35.852 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:29:35.852 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.630883+0000", 2024-09-14T06:29:35.852 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.1919999999999999e-06 2024-09-14T06:29:35.852 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:35.852 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:35.852 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:29:35.852 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.630891+0000", 2024-09-14T06:29:35.852 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.1579999999999999e-06 2024-09-14T06:29:35.852 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:35.852 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:35.852 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:29:35.852 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.656836+0000", 2024-09-14T06:29:35.853 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.025945134000000002 2024-09-14T06:29:35.853 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:35.853 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:35.853 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:29:35.853 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.656920+0000", 2024-09-14T06:29:35.853 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.4351999999999995e-05 2024-09-14T06:29:35.853 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:35.853 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:35.853 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:29:35.853 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.657211+0000", 2024-09-14T06:29:35.853 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.000290737 2024-09-14T06:29:35.853 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:35.854 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:29:35.854 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:35.854 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:35.854 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:35.854 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:1584 3.15 3:accf422f:::rbd_data.112d7dfbd766.0000000000000001:5 [read 487424~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-14T06:29:35.854 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:08.630897+0000", 2024-09-14T06:29:35.854 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 27.174433144000002, 2024-09-14T06:29:35.854 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.026720516, 2024-09-14T06:29:35.854 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:29:35.854 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:29:35.854 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:29:35.854 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:29:35.855 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:29:35.855 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:35.855 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 1584 2024-09-14T06:29:35.855 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:35.855 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:29:35.855 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:35.855 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:29:35.855 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.630897+0000", 2024-09-14T06:29:35.855 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:35.855 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:35.855 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:35.856 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:29:35.856 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.630897+0000", 2024-09-14T06:29:35.856 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:35.856 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:35.856 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:35.856 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:29:35.856 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.630898+0000", 2024-09-14T06:29:35.856 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.9200000000000002e-07 2024-09-14T06:29:35.856 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:35.856 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:35.856 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:29:35.857 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.630900+0000", 2024-09-14T06:29:35.857 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.7269999999999999e-06 2024-09-14T06:29:35.857 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:35.857 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:35.857 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:29:35.857 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.630906+0000", 2024-09-14T06:29:35.857 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.5849999999999996e-06 2024-09-14T06:29:35.857 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:35.857 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:35.857 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:29:35.857 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.630919+0000", 2024-09-14T06:29:35.858 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.3098999999999999e-05 2024-09-14T06:29:35.858 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:35.858 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:35.858 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:29:35.858 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.657253+0000", 2024-09-14T06:29:35.858 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.026333895999999999 2024-09-14T06:29:35.858 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:35.858 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:35.858 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:29:35.858 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.657342+0000", 2024-09-14T06:29:35.858 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.9044999999999999e-05 2024-09-14T06:29:35.858 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:35.858 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:35.859 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:29:35.859 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.657618+0000", 2024-09-14T06:29:35.859 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00027577199999999997 2024-09-14T06:29:35.859 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:35.859 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:29:35.859 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:35.859 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:35.859 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:35.859 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:1585 3.15 3:accf422f:::rbd_data.112d7dfbd766.0000000000000001:5 [read 503808~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-14T06:29:35.859 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:08.630927+0000", 2024-09-14T06:29:35.859 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 27.174403141999999, 2024-09-14T06:29:35.859 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.027110801, 2024-09-14T06:29:35.860 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:29:35.860 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:29:35.860 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:29:35.860 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:29:35.860 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:29:35.860 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:35.860 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 1585 2024-09-14T06:29:35.860 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:35.860 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:29:35.860 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:35.860 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:29:35.861 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.630927+0000", 2024-09-14T06:29:35.861 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:35.861 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:35.861 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:35.861 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:29:35.861 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.630927+0000", 2024-09-14T06:29:35.861 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:35.861 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:35.861 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:35.861 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:29:35.861 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.630928+0000", 2024-09-14T06:29:35.861 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.7800000000000001e-07 2024-09-14T06:29:35.862 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:35.862 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:35.862 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:29:35.862 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.630930+0000", 2024-09-14T06:29:35.862 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.0829999999999998e-06 2024-09-14T06:29:35.862 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:35.862 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:35.862 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:29:35.862 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.630937+0000", 2024-09-14T06:29:35.862 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.7179999999999999e-06 2024-09-14T06:29:35.862 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:35.862 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:35.863 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:29:35.863 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.630942+0000", 2024-09-14T06:29:35.863 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.2989999999999996e-06 2024-09-14T06:29:35.863 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:35.863 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:35.863 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:29:35.863 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.657673+0000", 2024-09-14T06:29:35.863 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.026731390000000001 2024-09-14T06:29:35.863 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:35.863 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:35.863 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:29:35.863 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.657754+0000", 2024-09-14T06:29:35.864 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.0580000000000004e-05 2024-09-14T06:29:35.864 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:35.864 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:35.864 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:29:35.864 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.658038+0000", 2024-09-14T06:29:35.864 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00028415299999999999 2024-09-14T06:29:35.864 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:35.864 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:29:35.864 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:35.864 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:35.864 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:35.864 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:17151 3.11 3:8b9b9188:::rbd_data.112d92027590.0000000000000017:f [read 74752~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:29:35.865 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:26.344466+0000", 2024-09-14T06:29:35.865 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 9.4608649650000007, 2024-09-14T06:29:35.865 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.024431092000000001, 2024-09-14T06:29:35.865 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:29:35.865 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:29:35.865 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:29:35.865 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:29:35.865 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:29:35.865 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:35.865 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 17151 2024-09-14T06:29:35.865 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:35.865 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:29:35.866 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:35.866 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:29:35.866 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:26.344466+0000", 2024-09-14T06:29:35.866 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:35.866 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:35.866 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:35.866 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:29:35.866 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:26.344466+0000", 2024-09-14T06:29:35.866 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:35.866 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:35.866 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:35.866 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:29:35.866 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:26.344466+0000", 2024-09-14T06:29:35.867 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.2099999999999998e-07 2024-09-14T06:29:35.867 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:35.867 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:35.867 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:29:35.867 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:26.344468+0000", 2024-09-14T06:29:35.867 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.793e-06 2024-09-14T06:29:35.867 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:35.867 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:35.867 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:29:35.867 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:26.344473+0000", 2024-09-14T06:29:35.867 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.4559999999999999e-06 2024-09-14T06:29:35.867 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:35.868 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:35.868 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:29:35.868 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:26.344480+0000", 2024-09-14T06:29:35.868 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.5339999999999996e-06 2024-09-14T06:29:35.868 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:35.868 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:35.868 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:29:35.868 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:26.368772+0000", 2024-09-14T06:29:35.868 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.024292218000000001 2024-09-14T06:29:35.868 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:35.868 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:35.868 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:29:35.869 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:26.368821+0000", 2024-09-14T06:29:35.869 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.8819000000000001e-05 2024-09-14T06:29:35.869 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:35.869 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:35.869 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:29:35.869 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:26.368897+0000", 2024-09-14T06:29:35.869 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.5951000000000005e-05 2024-09-14T06:29:35.869 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:35.869 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:29:35.869 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:35.869 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:35.869 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:35.870 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:17152 3.11 3:8b9b9188:::rbd_data.112d92027590.0000000000000017:f [read 84992~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:29:35.870 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:26.344487+0000", 2024-09-14T06:29:35.870 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 9.4608432069999999, 2024-09-14T06:29:35.870 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.024548099, 2024-09-14T06:29:35.870 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:29:35.870 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:29:35.870 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:29:35.870 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:29:35.870 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:29:35.870 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:35.870 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 17152 2024-09-14T06:29:35.870 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:35.870 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:29:35.871 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:35.871 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:29:35.871 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:26.344487+0000", 2024-09-14T06:29:35.871 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:35.871 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:35.871 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:35.871 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:29:35.871 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:26.344487+0000", 2024-09-14T06:29:35.871 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:35.871 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:35.871 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:35.871 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:29:35.872 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:26.344488+0000", 2024-09-14T06:29:35.872 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.3700000000000001e-07 2024-09-14T06:29:35.872 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:35.872 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:35.872 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:29:35.872 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:26.344490+0000", 2024-09-14T06:29:35.872 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.9800000000000001e-06 2024-09-14T06:29:35.872 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:35.872 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:35.872 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:29:35.872 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:26.344495+0000", 2024-09-14T06:29:35.872 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.6609999999999997e-06 2024-09-14T06:29:35.873 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:35.873 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:35.873 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:29:35.873 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:26.344502+0000", 2024-09-14T06:29:35.873 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.618e-06 2024-09-14T06:29:35.873 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:35.873 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:35.873 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:29:35.873 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:26.368935+0000", 2024-09-14T06:29:35.873 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.024433264999999999 2024-09-14T06:29:35.873 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:35.874 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:35.874 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:29:35.874 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:26.368979+0000", 2024-09-14T06:29:35.874 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.3949999999999998e-05 2024-09-14T06:29:35.874 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:35.874 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:35.874 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:29:35.874 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:26.369035+0000", 2024-09-14T06:29:35.874 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.6288e-05 2024-09-14T06:29:35.874 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:35.874 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:29:35.874 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:35.875 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:35.875 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:35.875 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:17153 3.11 3:8b9b9188:::rbd_data.112d92027590.0000000000000017:f [read 95232~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:29:35.875 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:26.344516+0000", 2024-09-14T06:29:35.875 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 9.4608149319999999, 2024-09-14T06:29:35.875 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.024640502000000002, 2024-09-14T06:29:35.875 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:29:35.875 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:29:35.875 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:29:35.875 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:29:35.875 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:29:35.875 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:35.876 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 17153 2024-09-14T06:29:35.876 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:35.876 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:29:35.876 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:35.876 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:29:35.876 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:26.344516+0000", 2024-09-14T06:29:35.876 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:35.876 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:35.876 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:35.876 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:29:35.876 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:26.344516+0000", 2024-09-14T06:29:35.876 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:35.877 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:35.877 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:35.877 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:29:35.877 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:26.344516+0000", 2024-09-14T06:29:35.877 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.9000000000000002e-07 2024-09-14T06:29:35.877 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:35.877 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:35.877 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:29:35.877 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:26.344518+0000", 2024-09-14T06:29:35.877 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.2230000000000001e-06 2024-09-14T06:29:35.877 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:35.877 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:35.878 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:29:35.878 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:26.344525+0000", 2024-09-14T06:29:35.878 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.1099999999999997e-06 2024-09-14T06:29:35.878 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:35.878 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:35.878 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:29:35.878 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:26.344531+0000", 2024-09-14T06:29:35.878 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.1430000000000003e-06 2024-09-14T06:29:35.878 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:35.878 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:35.878 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:29:35.878 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:26.369062+0000", 2024-09-14T06:29:35.878 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.024530086999999999 2024-09-14T06:29:35.879 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:35.879 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:35.879 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:29:35.879 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:26.369106+0000", 2024-09-14T06:29:35.879 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.4700000000000002e-05 2024-09-14T06:29:35.879 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:35.879 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:35.879 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:29:35.879 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:26.369156+0000", 2024-09-14T06:29:35.879 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.9849000000000003e-05 2024-09-14T06:29:35.879 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:35.879 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:29:35.879 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:35.880 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:35.880 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:35.880 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:17154 3.11 3:8b9b9188:::rbd_data.112d92027590.0000000000000017:f [read 105472~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:29:35.880 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:26.344556+0000", 2024-09-14T06:29:35.880 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 9.4607745219999995, 2024-09-14T06:29:35.880 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.024718575, 2024-09-14T06:29:35.880 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:29:35.880 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:29:35.880 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:29:35.880 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:29:35.881 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:29:35.881 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:35.881 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 17154 2024-09-14T06:29:35.881 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:35.881 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:29:35.881 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:35.881 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:29:35.881 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:26.344556+0000", 2024-09-14T06:29:35.881 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:35.881 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:35.881 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:35.881 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:29:35.882 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:26.344556+0000", 2024-09-14T06:29:35.882 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:35.882 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:35.882 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:35.882 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:29:35.882 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:26.344556+0000", 2024-09-14T06:29:35.882 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.1600000000000002e-07 2024-09-14T06:29:35.882 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:35.882 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:35.882 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:29:35.882 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:26.344558+0000", 2024-09-14T06:29:35.882 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.514e-06 2024-09-14T06:29:35.882 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:35.883 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:35.883 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:29:35.883 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:26.344564+0000", 2024-09-14T06:29:35.883 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.7960000000000001e-06 2024-09-14T06:29:35.883 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:35.883 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:35.883 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:29:35.883 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:26.344568+0000", 2024-09-14T06:29:35.883 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.1150000000000004e-06 2024-09-14T06:29:35.883 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:35.883 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:35.883 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:29:35.883 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:26.369177+0000", 2024-09-14T06:29:35.883 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.024609529000000002 2024-09-14T06:29:35.884 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:35.884 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:35.884 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:29:35.884 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:26.369217+0000", 2024-09-14T06:29:35.884 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.9459000000000003e-05 2024-09-14T06:29:35.884 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:35.884 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:35.884 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:29:35.884 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:26.369275+0000", 2024-09-14T06:29:35.884 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.7745999999999999e-05 2024-09-14T06:29:35.884 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:35.884 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:29:35.884 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:35.885 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:35.885 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:35.885 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:17155 3.11 3:8b9b9188:::rbd_data.112d92027590.0000000000000017:f [read 115712~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:29:35.885 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:26.344575+0000", 2024-09-14T06:29:35.885 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 9.4607557779999993, 2024-09-14T06:29:35.885 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.024838444000000001, 2024-09-14T06:29:35.885 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:29:35.885 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:29:35.885 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:29:35.885 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:29:35.885 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:29:35.885 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:35.886 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 17155 2024-09-14T06:29:35.886 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:35.886 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:29:35.886 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:35.886 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:29:35.886 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:26.344575+0000", 2024-09-14T06:29:35.886 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:35.886 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:35.886 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:35.886 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:29:35.886 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:26.344575+0000", 2024-09-14T06:29:35.886 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:35.886 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:35.886 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:35.887 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:29:35.887 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:26.344575+0000", 2024-09-14T06:29:35.887 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.4600000000000001e-07 2024-09-14T06:29:35.887 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:35.887 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:35.887 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:29:35.887 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:26.344576+0000", 2024-09-14T06:29:35.887 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.1039999999999999e-06 2024-09-14T06:29:35.887 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:35.887 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:35.887 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:29:35.887 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:26.344579+0000", 2024-09-14T06:29:35.887 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.207e-06 2024-09-14T06:29:35.888 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:35.888 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:35.888 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:29:35.888 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:26.344582+0000", 2024-09-14T06:29:35.888 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.9239999999999999e-06 2024-09-14T06:29:35.888 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:35.888 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:35.888 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:29:35.888 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:26.369311+0000", 2024-09-14T06:29:35.888 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.024728488999999999 2024-09-14T06:29:35.889 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:35.889 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:35.889 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:29:35.889 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:26.369357+0000", 2024-09-14T06:29:35.889 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.5837999999999997e-05 2024-09-14T06:29:35.889 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:35.889 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:35.889 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:29:35.889 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:26.369413+0000", 2024-09-14T06:29:35.889 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.6635999999999998e-05 2024-09-14T06:29:35.890 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:35.890 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:29:35.890 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:35.890 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:35.890 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:35.890 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:17156 3.11 3:8b9b9188:::rbd_data.112d92027590.0000000000000017:f [read 125952~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:29:35.890 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:26.344585+0000", 2024-09-14T06:29:35.890 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 9.4607450590000006, 2024-09-14T06:29:35.890 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.024929532000000001, 2024-09-14T06:29:35.890 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:29:35.890 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:29:35.890 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:29:35.890 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:29:35.891 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:29:35.891 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:35.891 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 17156 2024-09-14T06:29:35.891 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:35.891 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:29:35.891 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:35.891 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:29:35.891 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:26.344585+0000", 2024-09-14T06:29:35.891 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:35.891 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:35.891 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:35.891 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:29:35.891 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:26.344585+0000", 2024-09-14T06:29:35.892 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:35.892 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:35.892 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:35.892 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:29:35.892 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:26.344586+0000", 2024-09-14T06:29:35.892 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.0599999999999999e-07 2024-09-14T06:29:35.892 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:35.892 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:35.892 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:29:35.892 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:26.344587+0000", 2024-09-14T06:29:35.892 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.0249999999999999e-06 2024-09-14T06:29:35.892 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:35.893 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:35.893 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:29:35.893 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:26.344590+0000", 2024-09-14T06:29:35.893 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.0910000000000001e-06 2024-09-14T06:29:35.893 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:35.893 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:35.893 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:29:35.893 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:26.344593+0000", 2024-09-14T06:29:35.893 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.4199999999999999e-06 2024-09-14T06:29:35.893 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:35.893 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:35.893 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:29:35.894 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:26.369428+0000", 2024-09-14T06:29:35.894 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.024834927 2024-09-14T06:29:35.894 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:35.894 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:35.894 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:29:35.894 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:26.369465+0000", 2024-09-14T06:29:35.894 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.7092999999999999e-05 2024-09-14T06:29:35.894 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:35.894 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:35.894 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:29:35.894 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:26.369515+0000", 2024-09-14T06:29:35.894 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.977e-05 2024-09-14T06:29:35.894 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:35.895 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:29:35.895 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:35.895 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:35.895 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:35.895 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:17157 3.11 3:8b9b9188:::rbd_data.112d92027590.0000000000000017:f [read 136192~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:29:35.895 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:26.344596+0000", 2024-09-14T06:29:35.895 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 9.4607342380000006, 2024-09-14T06:29:35.895 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.025164066999999998, 2024-09-14T06:29:35.895 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:29:35.895 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:29:35.895 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:29:35.895 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:29:35.895 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:29:35.896 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:35.896 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 17157 2024-09-14T06:29:35.896 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:35.896 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:29:35.896 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:35.896 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:29:35.896 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:26.344596+0000", 2024-09-14T06:29:35.896 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:35.896 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:35.896 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:35.896 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:29:35.896 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:26.344596+0000", 2024-09-14T06:29:35.897 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:35.897 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:35.897 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:35.897 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:29:35.897 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:26.344597+0000", 2024-09-14T06:29:35.897 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.0599999999999999e-07 2024-09-14T06:29:35.897 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:35.897 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:35.897 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:29:35.897 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:26.344598+0000", 2024-09-14T06:29:35.897 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.145e-06 2024-09-14T06:29:35.897 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:35.898 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:35.898 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:29:35.898 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:26.344601+0000", 2024-09-14T06:29:35.898 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.9809999999999999e-06 2024-09-14T06:29:35.898 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:35.898 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:35.898 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:29:35.898 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:26.344604+0000", 2024-09-14T06:29:35.898 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.0750000000000002e-06 2024-09-14T06:29:35.898 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:35.898 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:35.899 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:29:35.899 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:26.369571+0000", 2024-09-14T06:29:35.899 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.024967728000000002 2024-09-14T06:29:35.899 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:35.899 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:35.899 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:29:35.899 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:26.369653+0000", 2024-09-14T06:29:35.899 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.1312999999999999e-05 2024-09-14T06:29:35.899 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:35.899 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:35.899 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:29:35.899 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:26.369760+0000", 2024-09-14T06:29:35.900 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.000107619 2024-09-14T06:29:35.900 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:35.900 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:29:35.900 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:35.900 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:35.900 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:35.900 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:17158 3.11 3:8b9b9188:::rbd_data.112d92027590.0000000000000017:f [read 146432~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:29:35.900 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:26.344607+0000", 2024-09-14T06:29:35.900 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 9.4607238420000002, 2024-09-14T06:29:35.900 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.025345412000000001, 2024-09-14T06:29:35.900 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:29:35.900 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:29:35.901 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:29:35.901 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:29:35.901 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:29:35.901 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:35.901 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 17158 2024-09-14T06:29:35.901 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:35.901 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:29:35.901 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:35.901 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:29:35.901 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:26.344607+0000", 2024-09-14T06:29:35.901 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:35.901 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:35.902 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:35.902 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:29:35.902 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:26.344607+0000", 2024-09-14T06:29:35.902 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:35.902 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:35.902 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:35.902 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:29:35.902 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:26.344607+0000", 2024-09-14T06:29:35.902 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.05e-07 2024-09-14T06:29:35.902 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:35.902 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:35.902 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:29:35.902 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:26.344609+0000", 2024-09-14T06:29:35.903 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.734e-06 2024-09-14T06:29:35.903 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:35.903 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:35.903 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:29:35.903 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:26.344613+0000", 2024-09-14T06:29:35.903 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.6430000000000003e-06 2024-09-14T06:29:35.903 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:35.903 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:35.903 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:29:35.903 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:26.344616+0000", 2024-09-14T06:29:35.903 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.5799999999999999e-06 2024-09-14T06:29:35.903 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:35.903 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:35.904 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:29:35.904 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:26.369805+0000", 2024-09-14T06:29:35.904 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.025189238999999999 2024-09-14T06:29:35.904 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:35.904 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:35.904 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:29:35.904 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:26.369877+0000", 2024-09-14T06:29:35.904 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.1717000000000005e-05 2024-09-14T06:29:35.904 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:35.904 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:35.904 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:29:35.904 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:26.369952+0000", 2024-09-14T06:29:35.905 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.5294000000000001e-05 2024-09-14T06:29:35.905 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:35.905 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:29:35.905 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:35.905 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:35.905 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:35.905 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:17159 3.11 3:8b9b9188:::rbd_data.112d92027590.0000000000000017:f [read 156672~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:29:35.905 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:26.344619+0000", 2024-09-14T06:29:35.905 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 9.4607116379999994, 2024-09-14T06:29:35.905 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.025592119999999999, 2024-09-14T06:29:35.905 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:29:35.906 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:29:35.906 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:29:35.906 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:29:35.906 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:29:35.906 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:35.906 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 17159 2024-09-14T06:29:35.906 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:35.906 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:29:35.906 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:35.906 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:29:35.907 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:26.344619+0000", 2024-09-14T06:29:35.907 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:35.907 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:35.907 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:35.907 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:29:35.907 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:26.344619+0000", 2024-09-14T06:29:35.907 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:35.907 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:35.907 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:35.907 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:29:35.907 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:26.344619+0000", 2024-09-14T06:29:35.907 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.41e-07 2024-09-14T06:29:35.908 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:35.908 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:35.908 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:29:35.908 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:26.344620+0000", 2024-09-14T06:29:35.908 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.113e-06 2024-09-14T06:29:35.908 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:35.908 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:35.908 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:29:35.908 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:26.344623+0000", 2024-09-14T06:29:35.908 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.072e-06 2024-09-14T06:29:35.908 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:35.909 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:35.909 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:29:35.909 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:26.344626+0000", 2024-09-14T06:29:35.909 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.503e-06 2024-09-14T06:29:35.909 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:35.909 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:35.909 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:29:35.909 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:26.369987+0000", 2024-09-14T06:29:35.909 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.025361338000000001 2024-09-14T06:29:35.909 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:35.909 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:35.910 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:29:35.910 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:26.370078+0000", 2024-09-14T06:29:35.910 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.0662999999999996e-05 2024-09-14T06:29:35.910 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:35.910 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:35.910 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:29:35.910 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:26.370211+0000", 2024-09-14T06:29:35.910 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00013318999999999999 2024-09-14T06:29:35.910 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:35.910 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:29:35.910 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:35.911 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:35.911 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:29:35.911 INFO:teuthology.orchestra.run.smithi138.stdout:} 2024-09-14T06:29:35.912 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-14T06:29:35.913 INFO:tasks.ceph.osd.3.smithi094.stderr:2024-09-14T06:29:35.910+0000 7feeeafe8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33655) UID: 0 2024-09-14T06:29:36.013 INFO:tasks.ceph.osd.4.smithi138.stderr:2024-09-14T06:29:36.012+0000 7f6c653c8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33615) UID: 0 2024-09-14T06:29:36.017 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-14T06:29:36.017 INFO:teuthology.orchestra.run.smithi094.stdout: "ops": [], 2024-09-14T06:29:36.018 INFO:teuthology.orchestra.run.smithi094.stdout: "num_ops": 0 2024-09-14T06:29:36.018 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-14T06:29:36.026 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-14T06:29:36.114 INFO:tasks.ceph.osd.0.smithi094.stderr:2024-09-14T06:29:36.112+0000 7f07c9333640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33636) UID: 0 2024-09-14T06:29:36.129 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-14T06:29:36.129 INFO:teuthology.orchestra.run.smithi094.stdout: "ops": [], 2024-09-14T06:29:36.129 INFO:teuthology.orchestra.run.smithi094.stdout: "complaint_time": 30, 2024-09-14T06:29:36.129 INFO:teuthology.orchestra.run.smithi094.stdout: "num_blocked_ops": 0 2024-09-14T06:29:36.129 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-14T06:29:36.137 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-14T06:29:36.215 INFO:tasks.ceph.osd.6.smithi138.stderr:2024-09-14T06:29:36.215+0000 7f8cd8849640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33620) UID: 0 2024-09-14T06:29:36.281 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-14T06:29:36.281 INFO:teuthology.orchestra.run.smithi094.stdout: "size": 20, 2024-09-14T06:29:36.281 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 600, 2024-09-14T06:29:36.281 INFO:teuthology.orchestra.run.smithi094.stdout: "ops": [ 2024-09-14T06:29:36.281 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:36.281 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:11860 3.12 3:4a837b73:::rbd_data.112d92027590.0000000000000000:f [read 735232~1024 [fadvise_nocache]] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:29:36.281 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:25.716255+0000", 2024-09-14T06:29:36.282 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 10.563077539, 2024-09-14T06:29:36.282 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.049569145000000002, 2024-09-14T06:29:36.282 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:29:36.282 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:29:36.282 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:29:36.282 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:29:36.282 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:29:36.282 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:36.282 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 11860 2024-09-14T06:29:36.282 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:36.282 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:29:36.282 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:36.282 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:29:36.283 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.716255+0000", 2024-09-14T06:29:36.283 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:36.283 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:36.283 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:36.283 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:29:36.283 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.716255+0000", 2024-09-14T06:29:36.283 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:36.283 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:36.283 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:36.283 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:29:36.283 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.716255+0000", 2024-09-14T06:29:36.283 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.1199999999999998e-07 2024-09-14T06:29:36.283 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:36.284 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:36.284 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:29:36.284 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.716257+0000", 2024-09-14T06:29:36.284 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.024e-06 2024-09-14T06:29:36.284 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:36.284 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:36.284 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:29:36.284 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.716265+0000", 2024-09-14T06:29:36.284 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.5259999999999996e-06 2024-09-14T06:29:36.284 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:36.284 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:36.284 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:29:36.285 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.716270+0000", 2024-09-14T06:29:36.285 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.0730000000000004e-06 2024-09-14T06:29:36.285 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:36.285 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:36.285 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:29:36.285 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.765618+0000", 2024-09-14T06:29:36.285 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.049348141999999998 2024-09-14T06:29:36.285 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:36.285 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:36.285 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:29:36.285 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.765703+0000", 2024-09-14T06:29:36.285 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.5594999999999996e-05 2024-09-14T06:29:36.286 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:36.286 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:36.286 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:29:36.286 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.765824+0000", 2024-09-14T06:29:36.286 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00012037299999999999 2024-09-14T06:29:36.286 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:36.286 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:36.286 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:36.286 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:36.286 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:36.286 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:11861 3.12 3:4a837b73:::rbd_data.112d92027590.0000000000000000:f [read 745472~1024 [fadvise_nocache]] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:29:36.286 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:25.716287+0000", 2024-09-14T06:29:36.286 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 10.563044718, 2024-09-14T06:29:36.287 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.049669320000000003, 2024-09-14T06:29:36.287 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:29:36.287 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:29:36.287 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:29:36.287 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:29:36.287 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:29:36.287 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:36.287 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 11861 2024-09-14T06:29:36.287 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:36.287 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:29:36.287 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:36.287 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:29:36.287 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.716287+0000", 2024-09-14T06:29:36.288 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:36.288 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:36.288 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:36.288 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:29:36.288 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.716287+0000", 2024-09-14T06:29:36.288 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:36.288 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:36.288 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:36.288 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:29:36.288 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.716288+0000", 2024-09-14T06:29:36.288 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.2099999999999998e-07 2024-09-14T06:29:36.289 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:36.289 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:36.289 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:29:36.289 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.716289+0000", 2024-09-14T06:29:36.289 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.5090000000000001e-06 2024-09-14T06:29:36.289 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:36.289 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:36.289 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:29:36.289 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.716296+0000", 2024-09-14T06:29:36.289 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.4930000000000003e-06 2024-09-14T06:29:36.289 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:36.289 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:36.289 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:29:36.289 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.716298+0000", 2024-09-14T06:29:36.290 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.6960000000000001e-06 2024-09-14T06:29:36.290 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:36.290 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:36.290 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:29:36.290 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.765856+0000", 2024-09-14T06:29:36.291 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.049557714000000003 2024-09-14T06:29:36.291 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:36.291 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:36.291 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:29:36.291 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.765900+0000", 2024-09-14T06:29:36.291 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.3708000000000002e-05 2024-09-14T06:29:36.291 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:36.291 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:36.291 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:29:36.291 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.765957+0000", 2024-09-14T06:29:36.291 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.6879000000000002e-05 2024-09-14T06:29:36.292 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:36.292 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:36.292 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:36.292 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:36.292 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:36.292 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:11862 3.12 3:4a837b73:::rbd_data.112d92027590.0000000000000000:f [read 755712~1024 [fadvise_nocache]] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:29:36.292 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:25.716302+0000", 2024-09-14T06:29:36.292 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 10.563030188999999, 2024-09-14T06:29:36.292 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.049889259999999998, 2024-09-14T06:29:36.292 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:29:36.292 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:29:36.292 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:29:36.293 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:29:36.293 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:29:36.293 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:36.293 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 11862 2024-09-14T06:29:36.293 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:36.293 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:29:36.293 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:36.293 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:29:36.293 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.716302+0000", 2024-09-14T06:29:36.293 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:36.293 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:36.293 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:36.294 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:29:36.294 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.716302+0000", 2024-09-14T06:29:36.294 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:36.294 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:36.294 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:36.294 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:29:36.294 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.716302+0000", 2024-09-14T06:29:36.294 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.1500000000000001e-07 2024-09-14T06:29:36.294 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:36.294 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:36.294 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:29:36.294 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.716306+0000", 2024-09-14T06:29:36.295 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.8840000000000004e-06 2024-09-14T06:29:36.295 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:36.295 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:36.295 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:29:36.295 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.716310+0000", 2024-09-14T06:29:36.295 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.6229999999999999e-06 2024-09-14T06:29:36.295 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:36.295 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:36.295 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:29:36.295 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.716317+0000", 2024-09-14T06:29:36.295 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.3869999999999997e-06 2024-09-14T06:29:36.295 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:36.296 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:36.296 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:29:36.296 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.765995+0000", 2024-09-14T06:29:36.296 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.049677683 2024-09-14T06:29:36.296 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:36.296 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:36.296 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:29:36.296 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.766080+0000", 2024-09-14T06:29:36.296 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.5296999999999993e-05 2024-09-14T06:29:36.296 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:36.296 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:36.296 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:29:36.297 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.766191+0000", 2024-09-14T06:29:36.297 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000111171 2024-09-14T06:29:36.297 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:36.297 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:36.297 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:36.297 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:36.297 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:36.297 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:11863 3.12 3:4a837b73:::rbd_data.112d92027590.0000000000000000:f [read 765952~1024 [fadvise_nocache]] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:29:36.297 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:25.716321+0000", 2024-09-14T06:29:36.297 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 10.563011586, 2024-09-14T06:29:36.297 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.050017788000000001, 2024-09-14T06:29:36.298 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:29:36.298 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:29:36.298 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:29:36.298 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:29:36.298 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:29:36.298 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:36.298 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 11863 2024-09-14T06:29:36.298 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:36.298 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:29:36.298 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:36.298 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:29:36.298 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.716321+0000", 2024-09-14T06:29:36.298 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:36.299 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:36.299 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:36.299 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:29:36.299 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.716321+0000", 2024-09-14T06:29:36.299 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:36.299 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:36.299 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:36.299 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:29:36.299 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.721995+0000", 2024-09-14T06:29:36.299 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0056740200000000001 2024-09-14T06:29:36.299 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:36.299 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:36.300 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:29:36.300 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.722001+0000", 2024-09-14T06:29:36.300 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.815e-06 2024-09-14T06:29:36.300 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:36.300 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:36.300 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:29:36.300 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.722014+0000", 2024-09-14T06:29:36.300 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.2371000000000001e-05 2024-09-14T06:29:36.300 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:36.300 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:36.300 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:29:36.300 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.722022+0000", 2024-09-14T06:29:36.301 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.2770000000000005e-06 2024-09-14T06:29:36.301 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:36.301 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:36.301 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:29:36.301 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.766234+0000", 2024-09-14T06:29:36.301 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.044211651999999997 2024-09-14T06:29:36.301 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:36.301 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:36.301 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:29:36.301 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.766280+0000", 2024-09-14T06:29:36.301 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.5871000000000002e-05 2024-09-14T06:29:36.302 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:36.302 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:36.302 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:29:36.302 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.766338+0000", 2024-09-14T06:29:36.302 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.8782000000000001e-05 2024-09-14T06:29:36.302 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:36.302 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:36.302 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:36.302 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:36.302 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:36.302 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:12016 3.12 3:4a837b73:::rbd_data.112d92027590.0000000000000000:f [read 9216~1024 [fadvise_nocache]] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:29:36.302 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:25.722677+0000", 2024-09-14T06:29:36.303 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 10.556655171999999, 2024-09-14T06:29:36.303 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.049707673000000001, 2024-09-14T06:29:36.303 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:29:36.303 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:29:36.303 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:29:36.303 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:29:36.303 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:29:36.303 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:36.303 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 12016 2024-09-14T06:29:36.303 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:36.303 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:29:36.303 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:36.304 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:29:36.304 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.722677+0000", 2024-09-14T06:29:36.304 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:36.304 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:36.304 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:36.304 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:29:36.304 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.722677+0000", 2024-09-14T06:29:36.304 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:36.304 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:36.304 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:36.304 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:29:36.305 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.722677+0000", 2024-09-14T06:29:36.305 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.4200000000000002e-07 2024-09-14T06:29:36.305 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:36.305 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:36.305 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:29:36.305 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.722679+0000", 2024-09-14T06:29:36.305 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.756e-06 2024-09-14T06:29:36.305 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:36.305 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:36.305 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:29:36.305 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.722689+0000", 2024-09-14T06:29:36.305 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.6010000000000001e-06 2024-09-14T06:29:36.305 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:36.306 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:36.306 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:29:36.306 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.722694+0000", 2024-09-14T06:29:36.306 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.9919999999999998e-06 2024-09-14T06:29:36.306 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:36.306 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:36.306 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:29:36.306 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.772184+0000", 2024-09-14T06:29:36.306 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.049490037000000001 2024-09-14T06:29:36.306 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:36.306 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:36.306 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:29:36.307 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.772272+0000", 2024-09-14T06:29:36.307 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.8553000000000004e-05 2024-09-14T06:29:36.307 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:36.307 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:36.307 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:29:36.307 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.772385+0000", 2024-09-14T06:29:36.307 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000112392 2024-09-14T06:29:36.307 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:36.307 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:36.307 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:36.307 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:36.307 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:36.308 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:12017 3.12 3:4a837b73:::rbd_data.112d92027590.0000000000000000:f [read 19456~1024 [fadvise_nocache]] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:29:36.308 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:25.722700+0000", 2024-09-14T06:29:36.308 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 10.556632241999999, 2024-09-14T06:29:36.308 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.049819216, 2024-09-14T06:29:36.308 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:29:36.308 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:29:36.308 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:29:36.308 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:29:36.308 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:29:36.308 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:36.308 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 12017 2024-09-14T06:29:36.308 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:36.309 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:29:36.309 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:36.309 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:29:36.309 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.722700+0000", 2024-09-14T06:29:36.309 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:36.309 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:36.309 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:36.309 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:29:36.309 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.722700+0000", 2024-09-14T06:29:36.309 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:36.309 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:36.309 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:36.310 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:29:36.310 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.722700+0000", 2024-09-14T06:29:36.310 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.9200000000000002e-07 2024-09-14T06:29:36.310 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:36.310 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:36.310 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:29:36.310 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.722702+0000", 2024-09-14T06:29:36.310 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.979e-06 2024-09-14T06:29:36.310 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:36.310 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:36.310 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:29:36.310 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.722709+0000", 2024-09-14T06:29:36.311 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.3260000000000001e-06 2024-09-14T06:29:36.311 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:36.311 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:36.311 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:29:36.311 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.722714+0000", 2024-09-14T06:29:36.311 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.9280000000000001e-06 2024-09-14T06:29:36.311 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:36.311 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:36.311 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:29:36.311 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.772418+0000", 2024-09-14T06:29:36.311 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.049704904000000001 2024-09-14T06:29:36.311 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:36.312 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:36.312 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:29:36.312 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.772462+0000", 2024-09-14T06:29:36.312 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.3509e-05 2024-09-14T06:29:36.312 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:36.312 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:36.312 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:29:36.312 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.772519+0000", 2024-09-14T06:29:36.312 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.7178e-05 2024-09-14T06:29:36.312 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:36.312 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:36.312 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:36.313 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:36.313 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:36.313 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:12018 3.12 3:4a837b73:::rbd_data.112d92027590.0000000000000000:f [read 29696~1024 [fadvise_nocache]] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:29:36.313 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:25.722728+0000", 2024-09-14T06:29:36.313 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 10.556604576, 2024-09-14T06:29:36.313 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.050040655000000003, 2024-09-14T06:29:36.313 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:29:36.313 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:29:36.313 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:29:36.313 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:29:36.313 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:29:36.313 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:36.313 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 12018 2024-09-14T06:29:36.314 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:36.314 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:29:36.314 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:36.314 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:29:36.314 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.722728+0000", 2024-09-14T06:29:36.314 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:36.314 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:36.314 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:36.314 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:29:36.314 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.722728+0000", 2024-09-14T06:29:36.314 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:36.314 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:36.314 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:36.315 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:29:36.315 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.722728+0000", 2024-09-14T06:29:36.315 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.6300000000000005e-07 2024-09-14T06:29:36.315 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:36.315 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:36.315 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:29:36.315 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.722731+0000", 2024-09-14T06:29:36.315 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.3020000000000002e-06 2024-09-14T06:29:36.315 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:36.315 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:36.315 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:29:36.315 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.722737+0000", 2024-09-14T06:29:36.316 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.4869999999999999e-06 2024-09-14T06:29:36.316 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:36.316 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:36.316 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:29:36.316 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.722742+0000", 2024-09-14T06:29:36.316 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.8080000000000004e-06 2024-09-14T06:29:36.316 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:36.316 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:36.316 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:29:36.316 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.772563+0000", 2024-09-14T06:29:36.316 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.049821217000000001 2024-09-14T06:29:36.317 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:36.317 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:36.317 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:29:36.317 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.772653+0000", 2024-09-14T06:29:36.317 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.0346000000000005e-05 2024-09-14T06:29:36.317 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:36.317 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:36.317 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:29:36.317 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.772768+0000", 2024-09-14T06:29:36.317 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000114832 2024-09-14T06:29:36.317 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:36.317 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:36.318 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:36.318 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:36.318 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:36.318 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:12019 3.12 3:4a837b73:::rbd_data.112d92027590.0000000000000000:f [read 39936~1024 [fadvise_nocache]] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:29:36.318 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:25.722749+0000", 2024-09-14T06:29:36.318 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 10.556583362, 2024-09-14T06:29:36.318 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.050265848000000002, 2024-09-14T06:29:36.318 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:29:36.318 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:29:36.318 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:29:36.318 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:29:36.318 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:29:36.318 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:36.319 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 12019 2024-09-14T06:29:36.319 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:36.319 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:29:36.319 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:36.319 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:29:36.319 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.722749+0000", 2024-09-14T06:29:36.319 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:36.319 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:36.319 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:36.319 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:29:36.320 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.722749+0000", 2024-09-14T06:29:36.320 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:36.320 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:36.320 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:36.320 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:29:36.320 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.722749+0000", 2024-09-14T06:29:36.320 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.39e-07 2024-09-14T06:29:36.320 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:36.320 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:36.320 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:29:36.320 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.722751+0000", 2024-09-14T06:29:36.321 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.8780000000000001e-06 2024-09-14T06:29:36.321 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:36.321 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:36.321 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:29:36.321 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.722757+0000", 2024-09-14T06:29:36.321 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.7289999999999997e-06 2024-09-14T06:29:36.321 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:36.321 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:36.321 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:29:36.321 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.722761+0000", 2024-09-14T06:29:36.321 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.685e-06 2024-09-14T06:29:36.321 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:36.322 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:36.322 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:29:36.322 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.772802+0000", 2024-09-14T06:29:36.322 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.050040544999999999 2024-09-14T06:29:36.322 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:36.322 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:36.322 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:29:36.322 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.772900+0000", 2024-09-14T06:29:36.322 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.7921e-05 2024-09-14T06:29:36.322 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:36.322 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:36.323 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:29:36.323 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.773015+0000", 2024-09-14T06:29:36.323 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000114751 2024-09-14T06:29:36.323 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:36.323 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:36.323 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:36.323 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:36.323 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:36.323 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:12020 3.12 3:4a837b73:::rbd_data.112d92027590.0000000000000000:f [read 50176~1024 [fadvise_nocache]] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:29:36.323 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:25.722767+0000", 2024-09-14T06:29:36.323 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 10.556565122, 2024-09-14T06:29:36.323 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.050402935000000003, 2024-09-14T06:29:36.324 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:29:36.324 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:29:36.324 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:29:36.324 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:29:36.324 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:29:36.324 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:36.324 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 12020 2024-09-14T06:29:36.324 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:36.324 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:29:36.324 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:36.324 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:29:36.324 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.722767+0000", 2024-09-14T06:29:36.325 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:36.325 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:36.325 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:36.325 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:29:36.325 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.722767+0000", 2024-09-14T06:29:36.325 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:36.325 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:36.325 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:36.325 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:29:36.325 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.722767+0000", 2024-09-14T06:29:36.325 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.46e-07 2024-09-14T06:29:36.326 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:36.326 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:36.326 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:29:36.326 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.722769+0000", 2024-09-14T06:29:36.326 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.6810000000000001e-06 2024-09-14T06:29:36.326 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:36.326 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:36.326 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:29:36.326 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.722774+0000", 2024-09-14T06:29:36.326 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.3689999999999998e-06 2024-09-14T06:29:36.326 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:36.326 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:36.326 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:29:36.327 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.722779+0000", 2024-09-14T06:29:36.327 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.7079999999999996e-06 2024-09-14T06:29:36.327 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:36.327 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:36.327 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:29:36.327 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.773055+0000", 2024-09-14T06:29:36.327 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.050276280999999999 2024-09-14T06:29:36.327 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:36.327 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:36.327 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:29:36.327 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.773106+0000", 2024-09-14T06:29:36.327 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.1088000000000003e-05 2024-09-14T06:29:36.328 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:36.328 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:36.328 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:29:36.328 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.773170+0000", 2024-09-14T06:29:36.328 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.3461999999999996e-05 2024-09-14T06:29:36.328 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:36.328 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:36.328 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:36.328 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:36.328 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:36.328 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:12021 3.12 3:4a837b73:::rbd_data.112d92027590.0000000000000000:f [read 60416~1024 [fadvise_nocache]] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:29:36.328 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:25.722785+0000", 2024-09-14T06:29:36.329 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 10.556547459000001, 2024-09-14T06:29:36.329 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.050634957000000001, 2024-09-14T06:29:36.329 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:29:36.329 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:29:36.329 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:29:36.329 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:29:36.329 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:29:36.329 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:36.329 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 12021 2024-09-14T06:29:36.329 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:36.329 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:29:36.329 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:36.330 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:29:36.330 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.722785+0000", 2024-09-14T06:29:36.330 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:36.330 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:36.330 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:36.330 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:29:36.330 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.722785+0000", 2024-09-14T06:29:36.330 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:36.330 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:36.330 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:36.330 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:29:36.331 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.722785+0000", 2024-09-14T06:29:36.331 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.53e-07 2024-09-14T06:29:36.331 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:36.331 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:36.331 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:29:36.331 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.722793+0000", 2024-09-14T06:29:36.331 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.7330000000000003e-06 2024-09-14T06:29:36.331 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:36.331 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:36.331 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:29:36.332 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.722801+0000", 2024-09-14T06:29:36.332 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.5550000000000003e-06 2024-09-14T06:29:36.332 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:36.332 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:36.332 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:29:36.332 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.722809+0000", 2024-09-14T06:29:36.332 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.0700000000000007e-06 2024-09-14T06:29:36.332 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:36.332 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:36.332 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:29:36.332 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.773222+0000", 2024-09-14T06:29:36.332 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.050412476999999997 2024-09-14T06:29:36.333 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:36.333 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:36.333 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:29:36.333 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.773308+0000", 2024-09-14T06:29:36.333 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.6174999999999994e-05 2024-09-14T06:29:36.333 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:36.333 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:36.333 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:29:36.333 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.773420+0000", 2024-09-14T06:29:36.333 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000111594 2024-09-14T06:29:36.334 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:36.334 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:36.334 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:36.334 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:36.334 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:36.334 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:12022 3.12 3:4a837b73:::rbd_data.112d92027590.0000000000000000:f [read 70656~1024 [fadvise_nocache]] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:29:36.334 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:25.722817+0000", 2024-09-14T06:29:36.334 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 10.556515596000001, 2024-09-14T06:29:36.334 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.050817018999999998, 2024-09-14T06:29:36.334 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:29:36.334 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:29:36.334 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:29:36.335 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:29:36.335 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:29:36.335 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:36.335 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 12022 2024-09-14T06:29:36.335 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:36.335 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:29:36.335 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:36.335 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:29:36.335 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.722817+0000", 2024-09-14T06:29:36.335 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:36.335 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:36.335 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:36.336 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:29:36.336 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.722817+0000", 2024-09-14T06:29:36.336 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:36.336 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:36.336 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:36.336 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:29:36.336 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.728894+0000", 2024-09-14T06:29:36.336 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0060776490000000001 2024-09-14T06:29:36.336 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:36.336 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:36.336 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:29:36.336 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.728905+0000", 2024-09-14T06:29:36.337 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.1181e-05 2024-09-14T06:29:36.337 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:36.337 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:36.337 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:29:36.337 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.728918+0000", 2024-09-14T06:29:36.337 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.2781e-05 2024-09-14T06:29:36.337 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:36.337 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:36.337 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:29:36.337 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.728935+0000", 2024-09-14T06:29:36.337 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.7314999999999999e-05 2024-09-14T06:29:36.337 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:36.338 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:36.338 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:29:36.338 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.773458+0000", 2024-09-14T06:29:36.338 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.044522300000000001 2024-09-14T06:29:36.338 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:36.338 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:36.338 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:29:36.338 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.773513+0000", 2024-09-14T06:29:36.338 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.4758e-05 2024-09-14T06:29:36.338 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:36.338 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:36.338 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:29:36.338 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.773634+0000", 2024-09-14T06:29:36.339 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000121035 2024-09-14T06:29:36.339 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:36.339 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:36.339 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:36.339 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:36.339 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:36.339 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:12058 3.12 3:4a837b73:::rbd_data.112d92027590.0000000000000000:f [read 439296~1024 [fadvise_nocache]] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:29:36.339 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:25.729608+0000", 2024-09-14T06:29:36.339 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 10.549724028, 2024-09-14T06:29:36.339 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.049566225999999998, 2024-09-14T06:29:36.339 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:29:36.339 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:29:36.339 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:29:36.339 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:29:36.340 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:29:36.340 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:36.340 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 12058 2024-09-14T06:29:36.340 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:36.340 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:29:36.340 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:36.340 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:29:36.340 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.729608+0000", 2024-09-14T06:29:36.340 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:36.340 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:36.340 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:36.340 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:29:36.341 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.729608+0000", 2024-09-14T06:29:36.341 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:36.341 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:36.341 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:36.341 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:29:36.341 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.729608+0000", 2024-09-14T06:29:36.341 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.0900000000000001e-07 2024-09-14T06:29:36.341 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:36.341 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:36.341 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:29:36.341 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.729612+0000", 2024-09-14T06:29:36.342 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.1790000000000001e-06 2024-09-14T06:29:36.342 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:36.342 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:36.342 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:29:36.342 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.729616+0000", 2024-09-14T06:29:36.342 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.2890000000000002e-06 2024-09-14T06:29:36.342 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:36.342 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:36.342 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:29:36.342 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.729624+0000", 2024-09-14T06:29:36.342 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.0120000000000005e-06 2024-09-14T06:29:36.342 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:36.342 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:36.342 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:29:36.343 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.779081+0000", 2024-09-14T06:29:36.343 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.049457138999999997 2024-09-14T06:29:36.343 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:36.343 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:36.343 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:29:36.343 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.779121+0000", 2024-09-14T06:29:36.343 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.9972000000000003e-05 2024-09-14T06:29:36.343 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:36.343 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:36.343 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:29:36.343 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.779174+0000", 2024-09-14T06:29:36.343 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.3426e-05 2024-09-14T06:29:36.344 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:36.344 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:36.344 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:36.344 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:36.344 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:36.344 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:12059 3.12 3:4a837b73:::rbd_data.112d92027590.0000000000000000:f [read 449536~1024 [fadvise_nocache]] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:29:36.344 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:25.729627+0000", 2024-09-14T06:29:36.344 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 10.54970479, 2024-09-14T06:29:36.344 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.049660990000000002, 2024-09-14T06:29:36.344 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:29:36.344 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:29:36.344 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:29:36.345 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:29:36.345 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:29:36.345 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:36.345 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 12059 2024-09-14T06:29:36.345 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:36.345 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:29:36.345 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:36.345 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:29:36.345 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.729627+0000", 2024-09-14T06:29:36.345 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:36.345 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:36.345 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:36.346 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:29:36.346 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.729627+0000", 2024-09-14T06:29:36.346 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:36.346 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:36.346 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:36.346 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:29:36.346 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.729628+0000", 2024-09-14T06:29:36.346 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.3200000000000001e-07 2024-09-14T06:29:36.346 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:36.346 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:36.346 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:29:36.346 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.729629+0000", 2024-09-14T06:29:36.347 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.0389999999999999e-06 2024-09-14T06:29:36.347 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:36.347 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:36.347 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:29:36.347 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.729632+0000", 2024-09-14T06:29:36.347 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.089e-06 2024-09-14T06:29:36.347 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:36.347 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:36.347 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:29:36.347 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.729634+0000", 2024-09-14T06:29:36.347 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.48e-06 2024-09-14T06:29:36.348 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:36.348 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:36.348 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:29:36.348 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.779198+0000", 2024-09-14T06:29:36.348 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.049564201000000002 2024-09-14T06:29:36.348 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:36.348 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:36.348 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:29:36.348 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.779234+0000", 2024-09-14T06:29:36.348 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.5618999999999999e-05 2024-09-14T06:29:36.348 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:36.348 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:36.349 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:29:36.349 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.779288+0000", 2024-09-14T06:29:36.349 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.4330000000000003e-05 2024-09-14T06:29:36.349 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:36.349 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:36.349 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:36.349 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:36.349 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:36.349 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:12060 3.12 3:4a837b73:::rbd_data.112d92027590.0000000000000000:f [read 459776~1024 [fadvise_nocache]] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:29:36.349 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:25.729637+0000", 2024-09-14T06:29:36.349 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 10.549694932, 2024-09-14T06:29:36.349 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.049769278, 2024-09-14T06:29:36.350 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:29:36.350 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:29:36.350 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:29:36.350 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:29:36.350 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:29:36.350 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:36.350 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 12060 2024-09-14T06:29:36.350 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:36.350 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:29:36.350 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:36.350 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:29:36.350 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.729637+0000", 2024-09-14T06:29:36.351 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:36.351 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:36.351 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:36.351 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:29:36.351 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.729637+0000", 2024-09-14T06:29:36.351 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:36.351 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:36.351 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:36.351 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:29:36.351 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.729637+0000", 2024-09-14T06:29:36.351 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.1e-07 2024-09-14T06:29:36.352 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:36.352 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:36.352 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:29:36.352 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.729638+0000", 2024-09-14T06:29:36.352 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.4499999999999995e-07 2024-09-14T06:29:36.352 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:36.352 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:36.352 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:29:36.352 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.729641+0000", 2024-09-14T06:29:36.352 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.8499999999999998e-06 2024-09-14T06:29:36.352 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:36.353 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:36.353 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:29:36.353 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.729643+0000", 2024-09-14T06:29:36.353 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.2649999999999999e-06 2024-09-14T06:29:36.353 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:36.353 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:36.353 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:29:36.353 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.779314+0000", 2024-09-14T06:29:36.353 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.049670181000000001 2024-09-14T06:29:36.353 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:36.353 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:36.354 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:29:36.354 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.779354+0000", 2024-09-14T06:29:36.354 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.0457000000000002e-05 2024-09-14T06:29:36.354 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:36.354 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:36.354 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:29:36.354 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.779406+0000", 2024-09-14T06:29:36.354 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.2370000000000002e-05 2024-09-14T06:29:36.354 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:36.354 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:36.354 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:36.355 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:36.355 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:36.355 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:12061 3.12 3:4a837b73:::rbd_data.112d92027590.0000000000000000:f [read 470016~1024 [fadvise_nocache]] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:29:36.355 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:25.729646+0000", 2024-09-14T06:29:36.355 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 10.549685800000001, 2024-09-14T06:29:36.355 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.049877289999999998, 2024-09-14T06:29:36.355 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:29:36.355 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:29:36.355 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:29:36.355 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:29:36.355 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:29:36.355 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:36.356 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 12061 2024-09-14T06:29:36.356 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:36.356 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:29:36.356 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:36.356 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:29:36.356 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.729646+0000", 2024-09-14T06:29:36.356 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:36.356 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:36.356 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:36.356 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:29:36.356 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.729646+0000", 2024-09-14T06:29:36.356 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:36.357 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:36.357 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:36.357 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:29:36.357 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.734882+0000", 2024-09-14T06:29:36.357 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0052353149999999999 2024-09-14T06:29:36.357 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:36.357 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:36.357 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:29:36.357 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.734889+0000", 2024-09-14T06:29:36.357 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.2339999999999997e-06 2024-09-14T06:29:36.357 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:36.357 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:36.358 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:29:36.358 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.734902+0000", 2024-09-14T06:29:36.358 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.3215e-05 2024-09-14T06:29:36.358 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:36.358 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:36.358 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:29:36.358 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.734911+0000", 2024-09-14T06:29:36.358 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.2539999999999998e-06 2024-09-14T06:29:36.358 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:36.358 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:36.358 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:29:36.358 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.779430+0000", 2024-09-14T06:29:36.358 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.044518886000000001 2024-09-14T06:29:36.359 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:36.359 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:36.359 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:29:36.359 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.779472+0000", 2024-09-14T06:29:36.359 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.1365000000000001e-05 2024-09-14T06:29:36.359 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:36.359 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:36.359 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:29:36.359 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.779524+0000", 2024-09-14T06:29:36.359 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.2021000000000002e-05 2024-09-14T06:29:36.359 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:36.360 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:36.360 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:36.360 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:36.360 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:36.360 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:16693 3.1f 3:fa270572:::rbd_data.112d92027590.000000000000000e:f [read 619520~1024 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:29:36.360 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:26.209219+0000", 2024-09-14T06:29:36.360 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 10.070112725, 2024-09-14T06:29:36.360 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.049691935, 2024-09-14T06:29:36.360 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:29:36.360 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:29:36.360 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:29:36.360 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:29:36.360 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:29:36.361 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:36.361 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 16693 2024-09-14T06:29:36.361 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:36.361 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:29:36.361 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:36.361 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:29:36.361 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.209219+0000", 2024-09-14T06:29:36.361 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:36.361 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:36.361 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:36.361 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:29:36.362 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.209219+0000", 2024-09-14T06:29:36.362 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:36.362 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:36.362 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:36.362 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:29:36.362 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.209220+0000", 2024-09-14T06:29:36.362 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.3599999999999999e-07 2024-09-14T06:29:36.362 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:36.362 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:36.362 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:29:36.362 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.209222+0000", 2024-09-14T06:29:36.362 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.012e-06 2024-09-14T06:29:36.363 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:36.363 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:36.363 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:29:36.363 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.209229+0000", 2024-09-14T06:29:36.363 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.5050000000000001e-06 2024-09-14T06:29:36.363 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:36.363 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:36.363 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:29:36.363 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.209250+0000", 2024-09-14T06:29:36.363 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.0262999999999999e-05 2024-09-14T06:29:36.364 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:36.364 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:36.364 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:29:36.364 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.258657+0000", 2024-09-14T06:29:36.364 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.049407581999999999 2024-09-14T06:29:36.364 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:36.364 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:36.364 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:29:36.364 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.258737+0000", 2024-09-14T06:29:36.364 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.9697999999999995e-05 2024-09-14T06:29:36.364 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:36.364 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:36.365 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:29:36.365 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.258911+0000", 2024-09-14T06:29:36.365 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000174539 2024-09-14T06:29:36.365 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:36.365 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:36.365 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:36.365 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:36.365 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:36.365 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:16694 3.1f 3:fa270572:::rbd_data.112d92027590.000000000000000e:f [read 629760~1024 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:29:36.365 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:26.209260+0000", 2024-09-14T06:29:36.366 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 10.070072371, 2024-09-14T06:29:36.366 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.049895688000000001, 2024-09-14T06:29:36.366 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:29:36.366 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:29:36.366 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:29:36.366 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:29:36.366 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:29:36.366 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:36.366 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 16694 2024-09-14T06:29:36.366 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:36.366 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:29:36.367 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:36.367 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:29:36.367 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.209260+0000", 2024-09-14T06:29:36.367 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:36.367 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:36.367 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:36.367 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:29:36.367 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.209260+0000", 2024-09-14T06:29:36.367 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:36.367 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:36.367 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:36.367 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:29:36.368 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.209260+0000", 2024-09-14T06:29:36.368 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.1899999999999998e-07 2024-09-14T06:29:36.368 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:36.368 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:36.368 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:29:36.368 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.209275+0000", 2024-09-14T06:29:36.368 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.501e-05 2024-09-14T06:29:36.368 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:36.368 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:36.368 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:29:36.368 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.209284+0000", 2024-09-14T06:29:36.369 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.7309999999999999e-06 2024-09-14T06:29:36.369 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:36.369 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:36.369 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:29:36.369 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.209290+0000", 2024-09-14T06:29:36.369 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.8810000000000001e-06 2024-09-14T06:29:36.369 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:36.369 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:36.369 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:29:36.369 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.258952+0000", 2024-09-14T06:29:36.370 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.049661702000000002 2024-09-14T06:29:36.370 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:36.370 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:36.370 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:29:36.370 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.258999+0000", 2024-09-14T06:29:36.370 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.7014000000000001e-05 2024-09-14T06:29:36.370 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:36.370 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:36.370 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:29:36.370 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.259155+0000", 2024-09-14T06:29:36.370 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00015693099999999999 2024-09-14T06:29:36.370 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:36.371 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:36.371 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:36.371 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:36.371 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:36.371 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:16695 3.1f 3:fa270572:::rbd_data.112d92027590.000000000000000e:f [read 640000~1024 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:29:36.371 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:26.209297+0000", 2024-09-14T06:29:36.371 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 10.070034762000001, 2024-09-14T06:29:36.371 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.050124426999999999, 2024-09-14T06:29:36.371 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:29:36.371 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:29:36.371 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:29:36.372 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:29:36.372 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:29:36.372 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:36.372 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 16695 2024-09-14T06:29:36.372 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:36.372 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:29:36.372 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:36.372 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:29:36.372 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.209297+0000", 2024-09-14T06:29:36.372 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:36.372 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:36.372 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:36.373 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:29:36.373 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.209297+0000", 2024-09-14T06:29:36.373 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:36.373 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:36.373 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:36.373 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:29:36.373 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.209298+0000", 2024-09-14T06:29:36.373 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.1399999999999997e-07 2024-09-14T06:29:36.373 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:36.374 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:36.374 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:29:36.374 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.209300+0000", 2024-09-14T06:29:36.374 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.041e-06 2024-09-14T06:29:36.374 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:36.374 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:36.374 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:29:36.374 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.209308+0000", 2024-09-14T06:29:36.374 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.9570000000000002e-06 2024-09-14T06:29:36.374 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:36.374 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:36.375 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:29:36.375 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.209313+0000", 2024-09-14T06:29:36.375 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.2190000000000003e-06 2024-09-14T06:29:36.375 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:36.375 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:36.375 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:29:36.375 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.259196+0000", 2024-09-14T06:29:36.375 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.049882889 2024-09-14T06:29:36.375 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:36.375 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:36.375 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:29:36.375 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.259274+0000", 2024-09-14T06:29:36.376 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.8516000000000005e-05 2024-09-14T06:29:36.376 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:36.376 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:36.376 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:29:36.376 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.259422+0000", 2024-09-14T06:29:36.376 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000147391 2024-09-14T06:29:36.376 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:36.376 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:36.376 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:36.376 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:36.376 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:36.376 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:16696 3.1f 3:fa270572:::rbd_data.112d92027590.000000000000000e:f [read 650240~1024 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:29:36.377 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:26.209320+0000", 2024-09-14T06:29:36.377 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 10.070012583, 2024-09-14T06:29:36.377 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.050363212999999997, 2024-09-14T06:29:36.377 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:29:36.377 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:29:36.377 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:29:36.377 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:29:36.377 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:29:36.377 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:36.377 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 16696 2024-09-14T06:29:36.377 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:36.378 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:29:36.378 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:36.378 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:29:36.378 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.209320+0000", 2024-09-14T06:29:36.378 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:36.378 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:36.378 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:36.378 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:29:36.378 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.209320+0000", 2024-09-14T06:29:36.378 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:36.378 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:36.378 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:36.378 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:29:36.379 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.209320+0000", 2024-09-14T06:29:36.379 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.8299999999999998e-07 2024-09-14T06:29:36.379 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:36.379 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:36.379 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:29:36.379 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.209322+0000", 2024-09-14T06:29:36.379 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.809e-06 2024-09-14T06:29:36.379 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:36.379 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:36.379 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:29:36.379 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.209329+0000", 2024-09-14T06:29:36.379 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.5809999999999999e-06 2024-09-14T06:29:36.380 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:36.380 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:36.380 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:29:36.380 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.209334+0000", 2024-09-14T06:29:36.380 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.1919999999999996e-06 2024-09-14T06:29:36.380 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:36.380 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:36.380 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:29:36.380 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.259452+0000", 2024-09-14T06:29:36.380 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.050117406000000003 2024-09-14T06:29:36.380 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:36.380 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:36.381 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:29:36.381 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.259527+0000", 2024-09-14T06:29:36.381 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.5260999999999997e-05 2024-09-14T06:29:36.381 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:36.381 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:36.381 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:29:36.381 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.259683+0000", 2024-09-14T06:29:36.381 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00015568099999999999 2024-09-14T06:29:36.381 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:36.381 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:36.381 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:36.381 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:36.381 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:36.382 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:16708 3.1f 3:fa270572:::rbd_data.112d92027590.000000000000000e:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:29:36.382 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:26.209635+0000", 2024-09-14T06:29:36.382 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 10.069697192, 2024-09-14T06:29:36.382 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.050216744000000001, 2024-09-14T06:29:36.382 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:29:36.382 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:29:36.382 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:29:36.382 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:29:36.382 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:29:36.382 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:36.382 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 16708 2024-09-14T06:29:36.382 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:36.382 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:29:36.383 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:36.383 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:29:36.383 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.209635+0000", 2024-09-14T06:29:36.383 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:36.383 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:36.383 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:36.383 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:29:36.383 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.209635+0000", 2024-09-14T06:29:36.383 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:36.383 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:36.383 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:36.383 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:29:36.383 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.209635+0000", 2024-09-14T06:29:36.384 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.9699999999999996e-07 2024-09-14T06:29:36.384 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:36.384 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:36.384 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:29:36.384 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.209638+0000", 2024-09-14T06:29:36.384 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.2699999999999999e-06 2024-09-14T06:29:36.384 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:36.384 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:36.384 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:29:36.384 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.209644+0000", 2024-09-14T06:29:36.384 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.2570000000000004e-06 2024-09-14T06:29:36.384 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:36.385 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:36.385 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:29:36.385 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.209649+0000", 2024-09-14T06:29:36.385 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.1850000000000003e-06 2024-09-14T06:29:36.385 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:36.385 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:36.385 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:29:36.385 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.259725+0000", 2024-09-14T06:29:36.385 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.050075762000000003 2024-09-14T06:29:36.385 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:36.385 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:36.385 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:29:36.386 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.259770+0000", 2024-09-14T06:29:36.386 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.5444000000000001e-05 2024-09-14T06:29:36.386 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:36.386 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:36.386 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:29:36.386 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.259852+0000", 2024-09-14T06:29:36.386 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.1328999999999993e-05 2024-09-14T06:29:36.386 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:36.386 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:36.386 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:36.386 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:36.386 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:36.386 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-14T06:29:36.388 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-14T06:29:36.388 INFO:tasks.ceph.osd.2.smithi094.stderr:2024-09-14T06:29:36.386+0000 7faf3269b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33649) UID: 0 2024-09-14T06:29:36.489 INFO:tasks.ceph.osd.3.smithi094.stderr:2024-09-14T06:29:36.487+0000 7feeeafe8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33655) UID: 0 2024-09-14T06:29:36.531 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-14T06:29:36.532 INFO:teuthology.orchestra.run.smithi094.stdout: "ops": [], 2024-09-14T06:29:36.532 INFO:teuthology.orchestra.run.smithi094.stdout: "num_ops": 0 2024-09-14T06:29:36.532 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-14T06:29:36.540 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-14T06:29:36.590 INFO:tasks.ceph.osd.6.smithi138.stderr:2024-09-14T06:29:36.589+0000 7f8cd8849640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33620) UID: 0 2024-09-14T06:29:36.647 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-14T06:29:36.685 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-14T06:29:36.685 INFO:teuthology.orchestra.run.smithi094.stdout: "ops": [], 2024-09-14T06:29:36.685 INFO:teuthology.orchestra.run.smithi094.stdout: "complaint_time": 30, 2024-09-14T06:29:36.686 INFO:teuthology.orchestra.run.smithi094.stdout: "num_blocked_ops": 0 2024-09-14T06:29:36.686 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-14T06:29:36.691 INFO:tasks.ceph.osd.0.smithi094.stderr:2024-09-14T06:29:36.689+0000 7f07c9333640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33636) UID: 0 2024-09-14T06:29:36.695 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-14T06:29:36.709 INFO:teuthology.orchestra.run.smithi094.stderr:noscrub is set 2024-09-14T06:29:36.776 INFO:teuthology.orchestra.run.smithi094.stdout:osd.0: {} 2024-09-14T06:29:36.776 INFO:teuthology.orchestra.run.smithi094.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-14T06:29:36.791 INFO:tasks.ceph.osd.0.smithi094.stderr:2024-09-14T06:29:36.789+0000 7f07c9333640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33636) UID: 0 2024-09-14T06:29:36.793 INFO:teuthology.orchestra.run.smithi094.stdout:osd.1: {} 2024-09-14T06:29:36.793 INFO:teuthology.orchestra.run.smithi094.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-14T06:29:36.809 INFO:teuthology.orchestra.run.smithi094.stdout:osd.2: {} 2024-09-14T06:29:36.810 INFO:teuthology.orchestra.run.smithi094.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-14T06:29:36.819 INFO:teuthology.orchestra.run.smithi094.stdout:ERROR: (22) Invalid argument 2024-09-14T06:29:36.819 INFO:teuthology.orchestra.run.smithi094.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:29:36.826 INFO:teuthology.orchestra.run.smithi094.stdout:osd.3: {} 2024-09-14T06:29:36.826 INFO:teuthology.orchestra.run.smithi094.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-14T06:29:36.828 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-14T06:29:36.843 INFO:teuthology.orchestra.run.smithi094.stdout:osd.4: {} 2024-09-14T06:29:36.843 INFO:teuthology.orchestra.run.smithi094.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-14T06:29:36.860 INFO:teuthology.orchestra.run.smithi094.stdout:osd.5: {} 2024-09-14T06:29:36.860 INFO:teuthology.orchestra.run.smithi094.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-14T06:29:36.876 INFO:teuthology.orchestra.run.smithi094.stdout:osd.6: {} 2024-09-14T06:29:36.877 INFO:teuthology.orchestra.run.smithi094.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-14T06:29:36.892 INFO:tasks.ceph.osd.5.smithi138.stderr:2024-09-14T06:29:36.891+0000 7fe13746f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33618) UID: 0 2024-09-14T06:29:36.893 INFO:teuthology.orchestra.run.smithi094.stdout:osd.7: {} 2024-09-14T06:29:36.894 INFO:teuthology.orchestra.run.smithi094.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-14T06:29:36.947 INFO:teuthology.orchestra.run.smithi094.stdout:ERROR: (22) Invalid argument 2024-09-14T06:29:36.947 INFO:teuthology.orchestra.run.smithi094.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:29:36.956 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-14T06:29:37.000 INFO:tasks.ceph.osd.2.smithi094.stderr:2024-09-14T06:29:36.998+0000 7faf3269b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33649) UID: 0 2024-09-14T06:29:37.058 INFO:teuthology.orchestra.run.smithi094.stdout:ERROR: (22) Invalid argument 2024-09-14T06:29:37.058 INFO:teuthology.orchestra.run.smithi094.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:29:37.067 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-14T06:29:37.108 INFO:tasks.ceph.osd.1.smithi094.stderr:2024-09-14T06:29:37.106+0000 7f328e47a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33641) UID: 0 2024-09-14T06:29:37.195 INFO:tasks.ceph.osd.1.smithi094.stderr:2024-09-14T06:29:37.193+0000 7f328e47a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33641) UID: 0 2024-09-14T06:29:37.210 INFO:teuthology.orchestra.run.smithi094.stdout:ERROR: (22) Invalid argument 2024-09-14T06:29:37.210 INFO:teuthology.orchestra.run.smithi094.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:29:37.218 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-14T06:29:37.296 INFO:tasks.ceph.osd.6.smithi138.stderr:2024-09-14T06:29:37.295+0000 7f8cd8849640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33620) UID: 0 2024-09-14T06:29:37.361 INFO:teuthology.orchestra.run.smithi094.stdout:ERROR: (22) Invalid argument 2024-09-14T06:29:37.362 INFO:teuthology.orchestra.run.smithi094.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:29:37.370 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-14T06:29:37.396 INFO:tasks.ceph.osd.5.smithi138.stderr:2024-09-14T06:29:37.395+0000 7fe13746f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33618) UID: 0 2024-09-14T06:29:37.474 INFO:teuthology.orchestra.run.smithi094.stdout:ERROR: (22) Invalid argument 2024-09-14T06:29:37.474 INFO:teuthology.orchestra.run.smithi094.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:29:37.482 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-14T06:29:37.497 INFO:tasks.ceph.osd.5.smithi138.stderr:2024-09-14T06:29:37.496+0000 7fe13746f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33618) UID: 0 2024-09-14T06:29:37.518 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-14T06:29:37.518 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-14T06:29:37.518 INFO:tasks.thrashosds.thrasher:reset_purged_snaps_last 2024-09-14T06:29:37.519 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 reset_purged_snaps_last 2024-09-14T06:29:37.587 INFO:teuthology.orchestra.run.smithi094.stdout:ERROR: (22) Invalid argument 2024-09-14T06:29:37.587 INFO:teuthology.orchestra.run.smithi094.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:29:37.596 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-14T06:29:37.597 INFO:tasks.ceph.osd.2.smithi094.stderr:2024-09-14T06:29:37.596+0000 7faf3269b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33649) UID: 0 2024-09-14T06:29:37.661 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 reset_purged_snaps_last 2024-09-14T06:29:37.698 INFO:tasks.ceph.osd.7.smithi138.stderr:2024-09-14T06:29:37.697+0000 7f3c6a21d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33623) UID: 0 2024-09-14T06:29:37.713 INFO:teuthology.orchestra.run.smithi138.stdout:ERROR: (22) Invalid argument 2024-09-14T06:29:37.713 INFO:teuthology.orchestra.run.smithi138.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:29:37.725 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-14T06:29:37.810 INFO:tasks.ceph.osd.4.smithi138.stderr:2024-09-14T06:29:37.809+0000 7f6c653c8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33615) UID: 0 2024-09-14T06:29:37.875 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 reset_purged_snaps_last 2024-09-14T06:29:37.890 INFO:teuthology.orchestra.run.smithi138.stdout:ERROR: (22) Invalid argument 2024-09-14T06:29:37.890 INFO:teuthology.orchestra.run.smithi138.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:29:37.900 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-14T06:29:37.919 INFO:tasks.ceph.osd.3.smithi094.stderr:2024-09-14T06:29:37.918+0000 7feeeafe8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33655) UID: 0 2024-09-14T06:29:38.000 INFO:tasks.ceph.osd.7.smithi138.stderr:2024-09-14T06:29:37.999+0000 7f3c6a21d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33623) UID: 0 2024-09-14T06:29:38.009 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 reset_purged_snaps_last 2024-09-14T06:29:38.044 INFO:teuthology.orchestra.run.smithi138.stdout:ERROR: (22) Invalid argument 2024-09-14T06:29:38.044 INFO:teuthology.orchestra.run.smithi138.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:29:38.052 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-14T06:29:38.136 INFO:tasks.ceph.osd.6.smithi138.stderr:2024-09-14T06:29:38.135+0000 7f8cd8849640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33620) UID: 0 2024-09-14T06:29:38.182 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 reset_purged_snaps_last 2024-09-14T06:29:38.195 INFO:teuthology.orchestra.run.smithi138.stdout:ERROR: (22) Invalid argument 2024-09-14T06:29:38.195 INFO:teuthology.orchestra.run.smithi138.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:29:38.202 INFO:tasks.ceph.osd.6.smithi138.stderr:2024-09-14T06:29:38.201+0000 7f8cd8849640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33620) UID: 0 2024-09-14T06:29:38.204 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-14T06:29:38.303 INFO:tasks.ceph.osd.6.smithi138.stderr:2024-09-14T06:29:38.302+0000 7f8cd8849640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33620) UID: 0 2024-09-14T06:29:38.314 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 reset_purged_snaps_last 2024-09-14T06:29:38.348 INFO:teuthology.orchestra.run.smithi138.stdout:ERROR: (22) Invalid argument 2024-09-14T06:29:38.348 INFO:teuthology.orchestra.run.smithi138.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:29:38.356 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-14T06:29:38.440 INFO:tasks.ceph.osd.6.smithi138.stderr:2024-09-14T06:29:38.439+0000 7f8cd8849640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33620) UID: 0 2024-09-14T06:29:38.447 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 reset_purged_snaps_last 2024-09-14T06:29:38.499 INFO:teuthology.orchestra.run.smithi138.stdout:ERROR: (22) Invalid argument 2024-09-14T06:29:38.499 INFO:teuthology.orchestra.run.smithi138.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:29:38.504 INFO:tasks.ceph.osd.5.smithi138.stderr:2024-09-14T06:29:38.503+0000 7fe13746f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33618) UID: 0 2024-09-14T06:29:38.507 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-14T06:29:38.581 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 reset_purged_snaps_last 2024-09-14T06:29:38.624 INFO:tasks.ceph.osd.3.smithi094.stderr:2024-09-14T06:29:38.623+0000 7feeeafe8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33655) UID: 0 2024-09-14T06:29:38.650 INFO:teuthology.orchestra.run.smithi138.stdout:ERROR: (22) Invalid argument 2024-09-14T06:29:38.650 INFO:teuthology.orchestra.run.smithi138.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:29:38.659 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-14T06:29:38.706 INFO:tasks.ceph.osd.3.smithi094.stderr:2024-09-14T06:29:38.704+0000 7feeeafe8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33655) UID: 0 2024-09-14T06:29:38.727 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-14T06:29:38.764 INFO:teuthology.orchestra.run.smithi138.stdout:ERROR: (22) Invalid argument 2024-09-14T06:29:38.764 INFO:teuthology.orchestra.run.smithi138.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:29:38.773 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-14T06:29:38.807 INFO:tasks.ceph.osd.3.smithi094.stderr:2024-09-14T06:29:38.805+0000 7feeeafe8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33655) UID: 0 2024-09-14T06:29:38.862 INFO:teuthology.orchestra.run.smithi094.stdout:checking clone #8, image image_client.0-clone8 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent9 2024-09-14T06:29:38.879 INFO:teuthology.orchestra.run.smithi138.stdout:ERROR: (22) Invalid argument 2024-09-14T06:29:38.879 INFO:teuthology.orchestra.run.smithi138.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:29:38.888 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-14T06:29:38.904 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-14T06:29:38.932 INFO:tasks.ceph.osd.7.smithi138.stderr:2024-09-14T06:29:38.931+0000 7f3c6a21d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33623) UID: 0 2024-09-14T06:29:38.992 INFO:teuthology.orchestra.run.smithi138.stdout:ERROR: (22) Invalid argument 2024-09-14T06:29:38.992 INFO:teuthology.orchestra.run.smithi138.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:29:39.001 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-14T06:29:39.038 INFO:teuthology.orchestra.run.smithi094.stdout:osd.0: {} 2024-09-14T06:29:39.039 INFO:teuthology.orchestra.run.smithi094.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-14T06:29:39.042 INFO:tasks.ceph.osd.6.smithi138.stderr:2024-09-14T06:29:39.041+0000 7f8cd8849640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33620) UID: 0 2024-09-14T06:29:39.056 INFO:teuthology.orchestra.run.smithi094.stdout:osd.1: {} 2024-09-14T06:29:39.056 INFO:teuthology.orchestra.run.smithi094.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-14T06:29:39.074 INFO:teuthology.orchestra.run.smithi094.stdout:osd.2: {} 2024-09-14T06:29:39.074 INFO:teuthology.orchestra.run.smithi094.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-14T06:29:39.091 INFO:teuthology.orchestra.run.smithi094.stdout:osd.3: {} 2024-09-14T06:29:39.091 INFO:teuthology.orchestra.run.smithi094.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-14T06:29:39.108 INFO:teuthology.orchestra.run.smithi094.stdout:osd.4: {} 2024-09-14T06:29:39.108 INFO:teuthology.orchestra.run.smithi094.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-14T06:29:39.109 INFO:tasks.ceph.osd.4.smithi138.stderr:2024-09-14T06:29:39.107+0000 7f6c653c8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33615) UID: 0 2024-09-14T06:29:39.124 INFO:teuthology.orchestra.run.smithi094.stdout:osd.5: {} 2024-09-14T06:29:39.125 INFO:teuthology.orchestra.run.smithi094.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-14T06:29:39.136 INFO:teuthology.orchestra.run.smithi094.stdout:718 trunc from 0xdc40215 to 0xac91d2e 2024-09-14T06:29:39.142 INFO:teuthology.orchestra.run.smithi094.stdout:osd.6: {} 2024-09-14T06:29:39.142 INFO:teuthology.orchestra.run.smithi094.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-14T06:29:39.145 INFO:teuthology.orchestra.run.smithi138.stdout:ERROR: (22) Invalid argument 2024-09-14T06:29:39.146 INFO:teuthology.orchestra.run.smithi138.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:29:39.155 INFO:teuthology.orchestra.run.smithi094.stdout:719 write 0x616f843 thru 0x617c923 (0xd0e1 bytes) 2024-09-14T06:29:39.156 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-14T06:29:39.157 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-14T06:29:39.161 INFO:teuthology.orchestra.run.smithi094.stdout:osd.7: {} 2024-09-14T06:29:39.161 INFO:teuthology.orchestra.run.smithi094.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-14T06:29:39.163 INFO:teuthology.orchestra.run.smithi094.stdout:722 write 0xb9de78e thru 0xb9e2630 (0x3ea3 bytes) 2024-09-14T06:29:39.167 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-14T06:29:39.175 INFO:teuthology.orchestra.run.smithi094.stdout:724 writesame 0xde37e3f thru 0xde3e0dd data_size 0x1(0x629f bytes) 2024-09-14T06:29:39.175 INFO:teuthology.orchestra.run.smithi094.stdout:725 punch from 0xa44af04 to 0xa459b3d, (0xec39 bytes) 2024-09-14T06:29:39.184 INFO:teuthology.orchestra.run.smithi094.stdout:726 trunc from 0xde3e0de to 0x6768860 2024-09-14T06:29:39.207 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-14T06:29:39.207 INFO:teuthology.orchestra.run.smithi094.stdout:728 punch from 0x2a0923c to 0x2a10704, (0x74c8 bytes) 2024-09-14T06:29:39.207 INFO:teuthology.orchestra.run.smithi094.stdout:729 read 0xf5643e thru 0xf62976 (0xc539 bytes) 2024-09-14T06:29:39.207 INFO:teuthology.orchestra.run.smithi094.stdout:730 trunc from 0x6768860 to 0x60cd004 2024-09-14T06:29:39.225 INFO:teuthology.orchestra.run.smithi094.stdout:731 write 0x6185473 thru 0x618900d (0x3b9b bytes) 2024-09-14T06:29:39.228 INFO:teuthology.orchestra.run.smithi094.stdout:732 write 0x304e6bf thru 0x3052ca0 (0x45e2 bytes) 2024-09-14T06:29:39.236 INFO:teuthology.orchestra.run.smithi094.stdout:733 writesame 0x648aa60 thru 0x648bd34 data_size 0x1(0x12d5 bytes) 2024-09-14T06:29:39.238 INFO:teuthology.orchestra.run.smithi094.stdout:734 punch from 0x1bbad58 to 0x1bbf1bf, (0x4467 bytes) 2024-09-14T06:29:39.241 INFO:tasks.ceph.osd.4.smithi138.stderr:2024-09-14T06:29:39.240+0000 7f6c653c8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33615) UID: 0 2024-09-14T06:29:39.250 INFO:teuthology.orchestra.run.smithi094.stdout:735 write 0xb813f3d thru 0xb8158ed (0x19b1 bytes) 2024-09-14T06:29:39.253 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-14T06:29:39.254 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-14T06:29:39.254 INFO:teuthology.orchestra.run.smithi094.stdout:742 read 0xb1c3c88 thru 0xb1c675a (0x2ad3 bytes) 2024-09-14T06:29:39.254 INFO:teuthology.orchestra.run.smithi094.stdout:743 read 0x7be07e3 thru 0x7be7379 (0x6b97 bytes) 2024-09-14T06:29:39.254 INFO:teuthology.orchestra.run.smithi094.stdout:748 punch from 0x2d1a0bb to 0x2d21509, (0x744e bytes) 2024-09-14T06:29:39.254 INFO:teuthology.orchestra.run.smithi094.stdout:750 writesame 0x62a3188 thru 0x62a6099 data_size 0x19(0x2f12 bytes) 2024-09-14T06:29:39.257 INFO:teuthology.orchestra.run.smithi094.stdout:751 write 0x213a610 thru 0x2144376 (0x9d67 bytes) 2024-09-14T06:29:39.258 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-14T06:29:39.259 INFO:teuthology.orchestra.run.smithi094.stdout:753 writesame 0x2a44610 thru 0x2a4a049 data_size 0x1(0x5a3a bytes) 2024-09-14T06:29:39.267 INFO:teuthology.orchestra.run.smithi094.stdout:754 read 0x15aa2f7 thru 0x15b6c4a (0xc954 bytes) 2024-09-14T06:29:39.268 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-14T06:29:39.268 INFO:teuthology.orchestra.run.smithi094.stdout:757 read 0x2175ed9 thru 0x2182131 (0xc259 bytes) 2024-09-14T06:29:39.269 INFO:teuthology.orchestra.run.smithi094.stdout:758 write 0xa67b259 thru 0xa6818ab (0x6653 bytes) 2024-09-14T06:29:39.279 INFO:teuthology.orchestra.run.smithi094.stdout:760 write 0xc340c50 thru 0xc34db06 (0xceb7 bytes) 2024-09-14T06:29:39.290 INFO:teuthology.orchestra.run.smithi094.stdout:763 write 0xe897292 thru 0xe89b25d (0x3fcc bytes) 2024-09-14T06:29:39.293 INFO:teuthology.orchestra.run.smithi094.stdout:764 writesame 0x666159f thru 0x666e406 data_size 0x14(0xce68 bytes) 2024-09-14T06:29:39.296 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-14T06:29:39.296 INFO:teuthology.orchestra.run.smithi094.stdout:767 write 0xadf06f4 thru 0xadf6779 (0x6086 bytes) 2024-09-14T06:29:39.299 INFO:teuthology.orchestra.run.smithi094.stdout:768 read 0xa60f146 thru 0xa61504b (0x5f06 bytes) 2024-09-14T06:29:39.300 INFO:teuthology.orchestra.run.smithi094.stdout:769 read 0x2d0facb thru 0x2d11f67 (0x249d bytes) 2024-09-14T06:29:39.300 INFO:teuthology.orchestra.run.smithi094.stdout:770 write 0x829fe85 thru 0x82a8130 (0x82ac bytes) 2024-09-14T06:29:39.301 INFO:teuthology.orchestra.run.smithi094.stdout:771 read 0x10c35d7 thru 0x10c43a2 (0xdcc bytes) 2024-09-14T06:29:39.301 INFO:teuthology.orchestra.run.smithi094.stdout:772 write 0x5334d91 thru 0x5344154 (0xf3c4 bytes) 2024-09-14T06:29:39.304 INFO:teuthology.orchestra.run.smithi094.stdout:773 write 0x635170a thru 0x635ce9c (0xb793 bytes) 2024-09-14T06:29:39.306 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-14T06:29:39.306 INFO:teuthology.orchestra.run.smithi094.stdout:775 read 0x4f8cbd8 thru 0x4f8eea1 (0x22ca bytes) 2024-09-14T06:29:39.309 INFO:tasks.ceph.osd.2.smithi094.stderr:2024-09-14T06:29:39.307+0000 7faf3269b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33649) UID: 0 2024-09-14T06:29:39.311 INFO:teuthology.orchestra.run.smithi138.stdout:{ 2024-09-14T06:29:39.311 INFO:teuthology.orchestra.run.smithi138.stdout: "size": 20, 2024-09-14T06:29:39.311 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 600, 2024-09-14T06:29:39.311 INFO:teuthology.orchestra.run.smithi138.stdout: "ops": [ 2024-09-14T06:29:39.311 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:39.311 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:1575 3.15 3:accf422f:::rbd_data.112d7dfbd766.0000000000000001:5 [read 339968~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-14T06:29:39.311 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:08.630645+0000", 2024-09-14T06:29:39.311 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 30.679013511000001, 2024-09-14T06:29:39.312 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.024511650999999999, 2024-09-14T06:29:39.312 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:29:39.312 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:29:39.312 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:29:39.312 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:29:39.312 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:29:39.312 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:39.312 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 1575 2024-09-14T06:29:39.312 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:39.312 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:29:39.312 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:39.312 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:29:39.312 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.630645+0000", 2024-09-14T06:29:39.312 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:39.313 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:39.313 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:39.313 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:29:39.313 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.630645+0000", 2024-09-14T06:29:39.313 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:39.313 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:39.313 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:39.313 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:29:39.313 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.630645+0000", 2024-09-14T06:29:39.313 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.9400000000000005e-07 2024-09-14T06:29:39.313 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:39.313 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:39.313 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:29:39.314 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.630647+0000", 2024-09-14T06:29:39.314 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.9309999999999998e-06 2024-09-14T06:29:39.314 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:39.314 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:39.314 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:29:39.314 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.630654+0000", 2024-09-14T06:29:39.314 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.55e-06 2024-09-14T06:29:39.314 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:39.314 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:39.314 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:29:39.314 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.630659+0000", 2024-09-14T06:29:39.314 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.0139999999999998e-06 2024-09-14T06:29:39.314 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:39.314 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:39.315 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:29:39.315 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.654964+0000", 2024-09-14T06:29:39.315 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.024305651000000001 2024-09-14T06:29:39.315 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:39.315 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:39.315 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:29:39.315 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.655048+0000", 2024-09-14T06:29:39.315 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.3389999999999999e-05 2024-09-14T06:29:39.315 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:39.315 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:39.315 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:29:39.315 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.655156+0000", 2024-09-14T06:29:39.315 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.000108521 2024-09-14T06:29:39.315 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:39.316 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:29:39.316 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:39.316 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:39.316 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:39.316 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:1576 3.15 3:accf422f:::rbd_data.112d7dfbd766.0000000000000001:5 [read 356352~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-14T06:29:39.316 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:08.630666+0000", 2024-09-14T06:29:39.316 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 30.678992473000001, 2024-09-14T06:29:39.316 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.024703712999999999, 2024-09-14T06:29:39.316 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:29:39.316 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:29:39.316 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:29:39.316 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:29:39.316 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:29:39.316 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:39.317 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 1576 2024-09-14T06:29:39.317 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:39.317 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:29:39.317 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:39.317 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:29:39.317 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.630666+0000", 2024-09-14T06:29:39.317 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:39.317 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:39.317 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:39.317 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:29:39.318 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.630666+0000", 2024-09-14T06:29:39.318 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:39.318 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:39.318 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:39.318 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:29:39.318 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.630666+0000", 2024-09-14T06:29:39.318 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.8299999999999997e-07 2024-09-14T06:29:39.318 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:39.318 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:39.318 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:29:39.318 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.630668+0000", 2024-09-14T06:29:39.318 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.8789999999999999e-06 2024-09-14T06:29:39.318 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:39.319 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:39.319 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:29:39.319 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.630675+0000", 2024-09-14T06:29:39.319 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.2899999999999997e-06 2024-09-14T06:29:39.319 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:39.319 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:39.319 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:29:39.319 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.630681+0000", 2024-09-14T06:29:39.319 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.2730000000000002e-06 2024-09-14T06:29:39.319 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:39.319 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:39.319 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:29:39.319 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.655182+0000", 2024-09-14T06:29:39.319 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.024501228999999999 2024-09-14T06:29:39.319 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:39.319 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:39.320 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:29:39.320 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.655257+0000", 2024-09-14T06:29:39.320 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.4954000000000001e-05 2024-09-14T06:29:39.320 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:39.320 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:39.320 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:29:39.320 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.655369+0000", 2024-09-14T06:29:39.320 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.000112605 2024-09-14T06:29:39.320 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:39.320 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:29:39.320 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:39.320 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:39.320 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:39.320 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:1577 3.15 3:accf422f:::rbd_data.112d7dfbd766.0000000000000001:5 [read 372736~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-14T06:29:39.320 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:08.630688+0000", 2024-09-14T06:29:39.321 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 30.678969961, 2024-09-14T06:29:39.321 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.024927538999999999, 2024-09-14T06:29:39.321 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:29:39.321 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:29:39.321 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:29:39.321 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:29:39.321 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:29:39.321 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:39.321 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 1577 2024-09-14T06:29:39.321 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:39.321 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:29:39.321 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:39.321 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:29:39.321 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.630688+0000", 2024-09-14T06:29:39.321 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:39.322 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:39.322 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:39.322 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:29:39.322 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.630688+0000", 2024-09-14T06:29:39.322 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:39.322 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:39.322 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:39.322 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:29:39.322 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.630689+0000", 2024-09-14T06:29:39.322 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.2199999999999999e-07 2024-09-14T06:29:39.322 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:39.322 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:39.322 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:29:39.322 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.630691+0000", 2024-09-14T06:29:39.322 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.9410000000000001e-06 2024-09-14T06:29:39.323 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:39.323 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:39.323 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:29:39.323 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.630699+0000", 2024-09-14T06:29:39.323 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.0250000000000002e-06 2024-09-14T06:29:39.323 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:39.323 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:39.323 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:29:39.323 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.630704+0000", 2024-09-14T06:29:39.323 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.4349999999999996e-06 2024-09-14T06:29:39.323 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:39.323 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:39.323 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:29:39.323 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.655406+0000", 2024-09-14T06:29:39.324 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.024702279000000001 2024-09-14T06:29:39.324 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:39.324 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:39.324 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:29:39.324 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.655483+0000", 2024-09-14T06:29:39.324 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.6915999999999993e-05 2024-09-14T06:29:39.324 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:39.324 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:39.324 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:29:39.324 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.655616+0000", 2024-09-14T06:29:39.324 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.000132521 2024-09-14T06:29:39.324 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:39.324 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:29:39.324 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:39.325 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:39.325 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:39.325 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:1578 3.15 3:accf422f:::rbd_data.112d7dfbd766.0000000000000001:5 [read 389120~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-14T06:29:39.325 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:08.630711+0000", 2024-09-14T06:29:39.325 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 30.678947488999999, 2024-09-14T06:29:39.325 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.025111108, 2024-09-14T06:29:39.325 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:29:39.325 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:29:39.325 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:29:39.325 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:29:39.325 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:29:39.325 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:39.325 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 1578 2024-09-14T06:29:39.325 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:39.326 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:29:39.326 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:39.326 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:29:39.326 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.630711+0000", 2024-09-14T06:29:39.326 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:39.326 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:39.326 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:39.326 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:29:39.326 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.630711+0000", 2024-09-14T06:29:39.326 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:39.326 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:39.326 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:39.326 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:29:39.327 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.630711+0000", 2024-09-14T06:29:39.327 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.01e-07 2024-09-14T06:29:39.327 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:39.327 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:39.327 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:29:39.327 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.630713+0000", 2024-09-14T06:29:39.327 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.8199999999999999e-06 2024-09-14T06:29:39.327 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:39.327 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:39.327 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:29:39.327 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.630720+0000", 2024-09-14T06:29:39.327 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.3880000000000002e-06 2024-09-14T06:29:39.328 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:39.328 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:39.328 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:29:39.328 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.630726+0000", 2024-09-14T06:29:39.328 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.3480000000000003e-06 2024-09-14T06:29:39.328 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:39.328 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:39.328 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:29:39.328 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.655646+0000", 2024-09-14T06:29:39.328 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.024920312999999999 2024-09-14T06:29:39.328 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:39.328 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:39.328 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:29:39.328 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.655724+0000", 2024-09-14T06:29:39.329 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.7589000000000005e-05 2024-09-14T06:29:39.329 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:39.329 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:39.329 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:29:39.329 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.655822+0000", 2024-09-14T06:29:39.329 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.8249000000000001e-05 2024-09-14T06:29:39.329 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:39.329 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:29:39.329 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:39.329 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:39.329 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:39.329 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:1579 3.15 3:accf422f:::rbd_data.112d7dfbd766.0000000000000001:5 [read 405504~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-14T06:29:39.329 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:08.630732+0000", 2024-09-14T06:29:39.329 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 30.678926171000001, 2024-09-14T06:29:39.330 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.025303169, 2024-09-14T06:29:39.330 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:29:39.330 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:29:39.330 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:29:39.330 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:29:39.330 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:29:39.330 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:39.330 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 1579 2024-09-14T06:29:39.330 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:39.330 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:29:39.330 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:39.330 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:29:39.330 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.630732+0000", 2024-09-14T06:29:39.331 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:39.331 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:39.331 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:39.331 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:29:39.331 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.630732+0000", 2024-09-14T06:29:39.331 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:39.331 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:39.331 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:39.331 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:29:39.331 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.630732+0000", 2024-09-14T06:29:39.331 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.5199999999999998e-07 2024-09-14T06:29:39.331 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:39.331 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:39.331 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:29:39.331 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.630741+0000", 2024-09-14T06:29:39.332 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.1549999999999996e-06 2024-09-14T06:29:39.332 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:39.332 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:39.332 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:29:39.332 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.630748+0000", 2024-09-14T06:29:39.332 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.6429999999999998e-06 2024-09-14T06:29:39.332 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:39.332 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:39.332 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:29:39.332 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.630753+0000", 2024-09-14T06:29:39.332 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.0579999999999997e-06 2024-09-14T06:29:39.332 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:39.332 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:39.332 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:29:39.333 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.655858+0000", 2024-09-14T06:29:39.333 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.025104774999999999 2024-09-14T06:29:39.333 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:39.333 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:39.333 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:29:39.333 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.655942+0000", 2024-09-14T06:29:39.333 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.4129999999999996e-05 2024-09-14T06:29:39.333 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:39.333 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:39.333 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:29:39.333 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.656035+0000", 2024-09-14T06:29:39.333 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.3055999999999998e-05 2024-09-14T06:29:39.333 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:39.334 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:29:39.334 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:39.334 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:39.334 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:39.334 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:1580 3.15 3:accf422f:::rbd_data.112d7dfbd766.0000000000000001:5 [read 421888~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-14T06:29:39.334 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:08.630762+0000", 2024-09-14T06:29:39.334 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 30.678896392999999, 2024-09-14T06:29:39.334 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.025487388999999999, 2024-09-14T06:29:39.334 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:29:39.334 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:29:39.334 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:29:39.334 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:29:39.334 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:29:39.335 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:39.335 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 1580 2024-09-14T06:29:39.335 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:39.335 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:29:39.335 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:39.335 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:29:39.335 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.630762+0000", 2024-09-14T06:29:39.335 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:39.335 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:39.335 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:39.335 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:29:39.335 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.630762+0000", 2024-09-14T06:29:39.335 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:39.335 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:39.335 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:39.335 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:29:39.336 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.630762+0000", 2024-09-14T06:29:39.336 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.2300000000000002e-07 2024-09-14T06:29:39.336 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:39.336 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:39.336 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:29:39.336 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.630764+0000", 2024-09-14T06:29:39.336 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.99e-06 2024-09-14T06:29:39.336 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:39.336 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:39.336 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:29:39.336 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.630771+0000", 2024-09-14T06:29:39.336 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.9349999999999997e-06 2024-09-14T06:29:39.336 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:39.336 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:39.337 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:29:39.337 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.630776+0000", 2024-09-14T06:29:39.337 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.1710000000000001e-06 2024-09-14T06:29:39.337 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:39.337 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:39.337 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:29:39.337 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.656068+0000", 2024-09-14T06:29:39.337 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.025291905999999999 2024-09-14T06:29:39.337 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:39.337 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:39.337 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:29:39.337 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.656151+0000", 2024-09-14T06:29:39.337 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.3209999999999998e-05 2024-09-14T06:29:39.337 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:39.338 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:39.338 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:29:39.338 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.656249+0000", 2024-09-14T06:29:39.338 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.7754e-05 2024-09-14T06:29:39.338 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:39.338 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:29:39.338 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:39.338 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:39.338 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:39.338 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:1581 3.15 3:accf422f:::rbd_data.112d7dfbd766.0000000000000001:5 [read 438272~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-14T06:29:39.338 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:08.630802+0000", 2024-09-14T06:29:39.338 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 30.678855743, 2024-09-14T06:29:39.338 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.025670063, 2024-09-14T06:29:39.339 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:29:39.339 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:29:39.339 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:29:39.339 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:29:39.339 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:29:39.339 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:39.339 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 1581 2024-09-14T06:29:39.339 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:39.339 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:29:39.339 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:39.340 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:29:39.340 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.630802+0000", 2024-09-14T06:29:39.340 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:39.340 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:39.340 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:39.340 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:29:39.340 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.630802+0000", 2024-09-14T06:29:39.340 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:39.340 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:39.340 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:39.340 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:29:39.340 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.630803+0000", 2024-09-14T06:29:39.340 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.3099999999999997e-07 2024-09-14T06:29:39.340 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:39.341 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:39.341 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:29:39.341 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.630806+0000", 2024-09-14T06:29:39.341 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.7319999999999998e-06 2024-09-14T06:29:39.341 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:39.341 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:39.341 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:29:39.341 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.630813+0000", 2024-09-14T06:29:39.341 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.0840000000000003e-06 2024-09-14T06:29:39.341 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:39.341 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:39.341 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:29:39.341 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.630826+0000", 2024-09-14T06:29:39.342 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.3104e-05 2024-09-14T06:29:39.342 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:39.342 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:39.342 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:29:39.342 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.656349+0000", 2024-09-14T06:29:39.342 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.025523113 2024-09-14T06:29:39.342 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:39.342 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:39.342 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:29:39.342 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.656400+0000", 2024-09-14T06:29:39.343 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.1156000000000003e-05 2024-09-14T06:29:39.343 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:39.343 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:39.343 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:29:39.343 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.656472+0000", 2024-09-14T06:29:39.343 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.2243000000000006e-05 2024-09-14T06:29:39.344 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:39.344 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:29:39.344 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:39.344 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:39.344 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:39.344 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:1582 3.15 3:accf422f:::rbd_data.112d7dfbd766.0000000000000001:5 [read 454656~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-14T06:29:39.344 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:08.630838+0000", 2024-09-14T06:29:39.345 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 30.678819719, 2024-09-14T06:29:39.345 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.025956396999999999, 2024-09-14T06:29:39.345 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:29:39.345 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:29:39.345 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:29:39.345 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:29:39.345 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:29:39.345 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:39.345 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 1582 2024-09-14T06:29:39.345 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:39.345 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:29:39.345 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:39.345 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:29:39.346 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.630838+0000", 2024-09-14T06:29:39.346 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:39.346 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:39.346 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:39.346 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:29:39.346 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.630838+0000", 2024-09-14T06:29:39.346 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:39.346 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:39.346 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:39.346 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:29:39.346 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.630839+0000", 2024-09-14T06:29:39.346 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.8400000000000004e-07 2024-09-14T06:29:39.346 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:39.347 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:39.347 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:29:39.347 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.630846+0000", 2024-09-14T06:29:39.347 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.4599999999999997e-06 2024-09-14T06:29:39.347 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:39.347 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:39.347 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:29:39.347 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.630854+0000", 2024-09-14T06:29:39.347 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.3139999999999998e-06 2024-09-14T06:29:39.347 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:39.347 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:39.347 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:29:39.347 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.630865+0000", 2024-09-14T06:29:39.347 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.0988999999999999e-05 2024-09-14T06:29:39.347 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:39.348 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:39.348 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:29:39.348 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.656598+0000", 2024-09-14T06:29:39.348 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.025733335 2024-09-14T06:29:39.348 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:39.348 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:39.348 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:29:39.348 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.656682+0000", 2024-09-14T06:29:39.348 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.4000999999999998e-05 2024-09-14T06:29:39.348 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:39.348 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:39.348 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:29:39.349 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.656795+0000", 2024-09-14T06:29:39.349 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00011271400000000001 2024-09-14T06:29:39.349 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:39.349 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:29:39.349 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:39.349 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:39.349 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:39.349 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:1583 3.15 3:accf422f:::rbd_data.112d7dfbd766.0000000000000001:5 [read 471040~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-14T06:29:39.349 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:08.630873+0000", 2024-09-14T06:29:39.349 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 30.678784851, 2024-09-14T06:29:39.349 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.026337467, 2024-09-14T06:29:39.349 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:29:39.349 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:29:39.350 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:29:39.350 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:29:39.350 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:29:39.350 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:39.350 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 1583 2024-09-14T06:29:39.350 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:39.350 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:29:39.350 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:39.350 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:29:39.350 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.630873+0000", 2024-09-14T06:29:39.351 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:39.351 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:39.351 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:39.351 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:29:39.351 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.630873+0000", 2024-09-14T06:29:39.351 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:39.352 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:39.352 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:39.352 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:29:39.352 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.630874+0000", 2024-09-14T06:29:39.352 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.2500000000000005e-07 2024-09-14T06:29:39.352 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:39.353 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:39.353 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:29:39.353 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.630876+0000", 2024-09-14T06:29:39.353 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.2689999999999998e-06 2024-09-14T06:29:39.353 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:39.353 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:39.353 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:29:39.354 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.630883+0000", 2024-09-14T06:29:39.354 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.1919999999999999e-06 2024-09-14T06:29:39.354 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:39.354 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:39.354 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:29:39.354 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.630891+0000", 2024-09-14T06:29:39.354 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.1579999999999999e-06 2024-09-14T06:29:39.354 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:39.355 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:39.355 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:29:39.355 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.656836+0000", 2024-09-14T06:29:39.355 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.025945134000000002 2024-09-14T06:29:39.355 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:39.355 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:39.355 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:29:39.356 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.656920+0000", 2024-09-14T06:29:39.356 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.4351999999999995e-05 2024-09-14T06:29:39.356 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:39.356 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:39.356 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:29:39.356 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.657211+0000", 2024-09-14T06:29:39.357 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.000290737 2024-09-14T06:29:39.357 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:39.357 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:29:39.357 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:39.357 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:39.357 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:39.357 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:1584 3.15 3:accf422f:::rbd_data.112d7dfbd766.0000000000000001:5 [read 487424~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-14T06:29:39.358 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:08.630897+0000", 2024-09-14T06:29:39.358 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 30.678760754999999, 2024-09-14T06:29:39.358 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.026720516, 2024-09-14T06:29:39.358 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:29:39.358 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:29:39.358 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:29:39.358 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:29:39.358 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:29:39.358 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:39.358 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 1584 2024-09-14T06:29:39.358 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:39.359 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:29:39.359 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:39.359 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:29:39.359 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.630897+0000", 2024-09-14T06:29:39.359 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:39.359 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:39.359 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:39.359 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:29:39.359 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.630897+0000", 2024-09-14T06:29:39.359 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:39.359 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:39.359 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:39.360 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:29:39.360 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.630898+0000", 2024-09-14T06:29:39.360 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.9200000000000002e-07 2024-09-14T06:29:39.360 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:39.360 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:39.360 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:29:39.360 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.630900+0000", 2024-09-14T06:29:39.360 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.7269999999999999e-06 2024-09-14T06:29:39.360 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:39.360 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:39.360 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:29:39.361 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.630906+0000", 2024-09-14T06:29:39.361 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.5849999999999996e-06 2024-09-14T06:29:39.361 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:39.361 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:39.361 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:29:39.361 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.630919+0000", 2024-09-14T06:29:39.361 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.3098999999999999e-05 2024-09-14T06:29:39.361 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:39.361 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:39.361 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:29:39.362 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.657253+0000", 2024-09-14T06:29:39.362 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.026333895999999999 2024-09-14T06:29:39.362 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:39.362 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:39.362 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:29:39.362 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.657342+0000", 2024-09-14T06:29:39.362 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.9044999999999999e-05 2024-09-14T06:29:39.362 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:39.362 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:39.362 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:29:39.362 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.657618+0000", 2024-09-14T06:29:39.362 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00027577199999999997 2024-09-14T06:29:39.362 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:39.362 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:29:39.363 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:39.363 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:39.363 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:39.363 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:1585 3.15 3:accf422f:::rbd_data.112d7dfbd766.0000000000000001:5 [read 503808~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-14T06:29:39.363 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:08.630927+0000", 2024-09-14T06:29:39.363 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 30.678730753, 2024-09-14T06:29:39.363 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.027110801, 2024-09-14T06:29:39.363 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:29:39.363 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:29:39.363 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:29:39.363 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:29:39.363 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:29:39.363 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:39.364 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 1585 2024-09-14T06:29:39.364 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:39.364 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:29:39.364 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:39.364 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:29:39.364 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.630927+0000", 2024-09-14T06:29:39.364 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:39.364 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:39.364 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:39.364 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:29:39.364 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.630927+0000", 2024-09-14T06:29:39.364 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:39.364 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:39.365 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:39.365 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:29:39.365 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.630928+0000", 2024-09-14T06:29:39.365 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.7800000000000001e-07 2024-09-14T06:29:39.365 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:39.365 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:39.365 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:29:39.365 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.630930+0000", 2024-09-14T06:29:39.365 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.0829999999999998e-06 2024-09-14T06:29:39.365 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:39.365 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:39.365 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:29:39.365 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.630937+0000", 2024-09-14T06:29:39.366 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.7179999999999999e-06 2024-09-14T06:29:39.366 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:39.366 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:39.366 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:29:39.366 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.630942+0000", 2024-09-14T06:29:39.366 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.2989999999999996e-06 2024-09-14T06:29:39.366 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:39.366 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:39.366 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:29:39.366 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.657673+0000", 2024-09-14T06:29:39.366 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.026731390000000001 2024-09-14T06:29:39.366 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:39.366 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:39.366 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:29:39.367 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.657754+0000", 2024-09-14T06:29:39.367 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.0580000000000004e-05 2024-09-14T06:29:39.367 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:39.367 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:39.367 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:29:39.367 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.658038+0000", 2024-09-14T06:29:39.367 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00028415299999999999 2024-09-14T06:29:39.367 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:39.367 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:29:39.367 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:39.367 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:39.367 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:39.368 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:17151 3.11 3:8b9b9188:::rbd_data.112d92027590.0000000000000017:f [read 74752~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:29:39.368 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:26.344466+0000", 2024-09-14T06:29:39.368 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 12.965192576, 2024-09-14T06:29:39.368 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.024431092000000001, 2024-09-14T06:29:39.368 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:29:39.368 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:29:39.368 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:29:39.368 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:29:39.368 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:29:39.368 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:39.368 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 17151 2024-09-14T06:29:39.368 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:39.368 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:29:39.368 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:39.368 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:29:39.369 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:26.344466+0000", 2024-09-14T06:29:39.369 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:39.369 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:39.369 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:39.369 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:29:39.369 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:26.344466+0000", 2024-09-14T06:29:39.369 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:39.369 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:39.369 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:39.369 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:29:39.369 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:26.344466+0000", 2024-09-14T06:29:39.369 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.2099999999999998e-07 2024-09-14T06:29:39.369 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:39.370 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:39.370 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:29:39.370 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:26.344468+0000", 2024-09-14T06:29:39.370 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.793e-06 2024-09-14T06:29:39.370 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:39.370 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:39.370 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:29:39.370 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:26.344473+0000", 2024-09-14T06:29:39.370 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.4559999999999999e-06 2024-09-14T06:29:39.370 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:39.370 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:39.370 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:29:39.370 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:26.344480+0000", 2024-09-14T06:29:39.371 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.5339999999999996e-06 2024-09-14T06:29:39.371 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:39.371 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:39.371 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:29:39.371 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:26.368772+0000", 2024-09-14T06:29:39.371 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.024292218000000001 2024-09-14T06:29:39.371 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:39.371 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:39.371 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:29:39.372 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:26.368821+0000", 2024-09-14T06:29:39.372 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.8819000000000001e-05 2024-09-14T06:29:39.372 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:39.372 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:39.372 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:29:39.372 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:26.368897+0000", 2024-09-14T06:29:39.372 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.5951000000000005e-05 2024-09-14T06:29:39.372 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:39.373 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:29:39.373 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:39.373 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:39.373 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:39.373 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:17152 3.11 3:8b9b9188:::rbd_data.112d92027590.0000000000000017:f [read 84992~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:29:39.373 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:26.344487+0000", 2024-09-14T06:29:39.373 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 12.965170818000001, 2024-09-14T06:29:39.373 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.024548099, 2024-09-14T06:29:39.373 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:29:39.373 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:29:39.373 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:29:39.373 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:29:39.373 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:29:39.374 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:39.374 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 17152 2024-09-14T06:29:39.374 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:39.374 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:29:39.374 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:39.374 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:29:39.374 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:26.344487+0000", 2024-09-14T06:29:39.374 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:39.374 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:39.374 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:39.374 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:29:39.374 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:26.344487+0000", 2024-09-14T06:29:39.374 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:39.375 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:39.375 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:39.375 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:29:39.375 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:26.344488+0000", 2024-09-14T06:29:39.375 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.3700000000000001e-07 2024-09-14T06:29:39.375 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:39.375 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:39.375 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:29:39.375 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:26.344490+0000", 2024-09-14T06:29:39.375 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.9800000000000001e-06 2024-09-14T06:29:39.375 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:39.375 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:39.375 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:29:39.376 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:26.344495+0000", 2024-09-14T06:29:39.376 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.6609999999999997e-06 2024-09-14T06:29:39.376 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:39.376 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:39.376 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:29:39.376 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:26.344502+0000", 2024-09-14T06:29:39.376 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.618e-06 2024-09-14T06:29:39.376 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:39.376 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:39.376 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:29:39.376 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:26.368935+0000", 2024-09-14T06:29:39.376 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.024433264999999999 2024-09-14T06:29:39.376 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:39.376 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:39.376 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:29:39.377 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:26.368979+0000", 2024-09-14T06:29:39.377 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.3949999999999998e-05 2024-09-14T06:29:39.377 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:39.377 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:39.377 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:29:39.377 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:26.369035+0000", 2024-09-14T06:29:39.377 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.6288e-05 2024-09-14T06:29:39.377 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:39.377 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:29:39.377 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:39.377 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:39.377 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:39.378 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:17153 3.11 3:8b9b9188:::rbd_data.112d92027590.0000000000000017:f [read 95232~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:29:39.378 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:26.344516+0000", 2024-09-14T06:29:39.378 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 12.965142543000001, 2024-09-14T06:29:39.378 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.024640502000000002, 2024-09-14T06:29:39.378 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:29:39.378 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:29:39.378 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:29:39.378 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:29:39.378 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:29:39.378 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:39.378 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 17153 2024-09-14T06:29:39.378 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:39.378 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:29:39.378 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:39.379 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:29:39.379 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:26.344516+0000", 2024-09-14T06:29:39.379 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:39.379 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:39.379 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:39.379 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:29:39.379 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:26.344516+0000", 2024-09-14T06:29:39.379 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:39.379 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:39.379 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:39.379 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:29:39.379 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:26.344516+0000", 2024-09-14T06:29:39.379 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.9000000000000002e-07 2024-09-14T06:29:39.379 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:39.379 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:39.380 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:29:39.380 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:26.344518+0000", 2024-09-14T06:29:39.380 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.2230000000000001e-06 2024-09-14T06:29:39.380 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:39.380 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:39.380 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:29:39.380 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:26.344525+0000", 2024-09-14T06:29:39.380 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.1099999999999997e-06 2024-09-14T06:29:39.380 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:39.380 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:39.380 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:29:39.381 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:26.344531+0000", 2024-09-14T06:29:39.381 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.1430000000000003e-06 2024-09-14T06:29:39.381 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:39.381 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:39.381 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:29:39.381 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:26.369062+0000", 2024-09-14T06:29:39.381 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.024530086999999999 2024-09-14T06:29:39.381 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:39.381 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:39.381 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:29:39.381 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:26.369106+0000", 2024-09-14T06:29:39.382 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.4700000000000002e-05 2024-09-14T06:29:39.382 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:39.382 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:39.382 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:29:39.382 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:26.369156+0000", 2024-09-14T06:29:39.382 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.9849000000000003e-05 2024-09-14T06:29:39.382 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:39.382 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:29:39.382 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:39.382 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:39.382 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:39.382 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:17154 3.11 3:8b9b9188:::rbd_data.112d92027590.0000000000000017:f [read 105472~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:29:39.382 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:26.344556+0000", 2024-09-14T06:29:39.382 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 12.965102133, 2024-09-14T06:29:39.382 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.024718575, 2024-09-14T06:29:39.383 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:29:39.383 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:29:39.383 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:29:39.383 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:29:39.383 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:29:39.383 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:39.383 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 17154 2024-09-14T06:29:39.383 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:39.383 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:29:39.383 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:39.383 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:29:39.383 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:26.344556+0000", 2024-09-14T06:29:39.383 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:39.383 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:39.384 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:39.384 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:29:39.384 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:26.344556+0000", 2024-09-14T06:29:39.384 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:39.384 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:39.384 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:39.384 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:29:39.384 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:26.344556+0000", 2024-09-14T06:29:39.384 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.1600000000000002e-07 2024-09-14T06:29:39.384 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:39.384 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:39.384 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:29:39.384 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:26.344558+0000", 2024-09-14T06:29:39.384 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.514e-06 2024-09-14T06:29:39.385 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:39.385 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:39.385 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:29:39.385 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:26.344564+0000", 2024-09-14T06:29:39.385 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.7960000000000001e-06 2024-09-14T06:29:39.385 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:39.385 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:39.385 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:29:39.385 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:26.344568+0000", 2024-09-14T06:29:39.385 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.1150000000000004e-06 2024-09-14T06:29:39.385 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:39.385 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:39.386 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:29:39.386 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:26.369177+0000", 2024-09-14T06:29:39.386 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.024609529000000002 2024-09-14T06:29:39.386 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:39.386 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:39.386 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:29:39.386 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:26.369217+0000", 2024-09-14T06:29:39.386 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.9459000000000003e-05 2024-09-14T06:29:39.386 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:39.386 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:39.386 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:29:39.386 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:26.369275+0000", 2024-09-14T06:29:39.386 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.7745999999999999e-05 2024-09-14T06:29:39.386 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:39.386 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:29:39.387 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:39.387 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:39.387 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:39.387 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:17155 3.11 3:8b9b9188:::rbd_data.112d92027590.0000000000000017:f [read 115712~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:29:39.387 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:26.344575+0000", 2024-09-14T06:29:39.387 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 12.965083389, 2024-09-14T06:29:39.387 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.024838444000000001, 2024-09-14T06:29:39.387 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:29:39.387 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:29:39.387 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:29:39.387 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:29:39.387 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:29:39.387 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:39.388 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 17155 2024-09-14T06:29:39.388 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:39.388 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:29:39.388 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:39.388 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:29:39.388 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:26.344575+0000", 2024-09-14T06:29:39.388 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:39.388 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:39.388 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:39.388 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:29:39.388 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:26.344575+0000", 2024-09-14T06:29:39.388 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:39.389 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:39.389 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:39.389 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:29:39.389 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:26.344575+0000", 2024-09-14T06:29:39.389 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.4600000000000001e-07 2024-09-14T06:29:39.389 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:39.389 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:39.389 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:29:39.389 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:26.344576+0000", 2024-09-14T06:29:39.389 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.1039999999999999e-06 2024-09-14T06:29:39.389 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:39.389 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:39.389 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:29:39.389 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:26.344579+0000", 2024-09-14T06:29:39.390 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.207e-06 2024-09-14T06:29:39.390 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:39.390 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:39.390 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:29:39.390 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:26.344582+0000", 2024-09-14T06:29:39.390 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.9239999999999999e-06 2024-09-14T06:29:39.390 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:39.390 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:39.390 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:29:39.390 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:26.369311+0000", 2024-09-14T06:29:39.390 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.024728488999999999 2024-09-14T06:29:39.390 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:39.390 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:39.390 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:29:39.390 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:26.369357+0000", 2024-09-14T06:29:39.391 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.5837999999999997e-05 2024-09-14T06:29:39.391 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:39.391 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:39.391 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:29:39.391 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:26.369413+0000", 2024-09-14T06:29:39.391 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.6635999999999998e-05 2024-09-14T06:29:39.391 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:39.391 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:29:39.391 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:39.391 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:39.391 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:39.391 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:17156 3.11 3:8b9b9188:::rbd_data.112d92027590.0000000000000017:f [read 125952~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:29:39.391 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:26.344585+0000", 2024-09-14T06:29:39.391 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 12.96507267, 2024-09-14T06:29:39.391 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.024929532000000001, 2024-09-14T06:29:39.391 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:29:39.392 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:29:39.392 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:29:39.392 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:29:39.392 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:29:39.392 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:39.392 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 17156 2024-09-14T06:29:39.392 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:39.392 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:29:39.392 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:39.392 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:29:39.392 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:26.344585+0000", 2024-09-14T06:29:39.392 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:39.392 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:39.392 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:39.392 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:29:39.393 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:26.344585+0000", 2024-09-14T06:29:39.393 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:39.393 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:39.393 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:39.393 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:29:39.393 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:26.344586+0000", 2024-09-14T06:29:39.393 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.0599999999999999e-07 2024-09-14T06:29:39.393 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:39.393 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:39.393 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:29:39.393 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:26.344587+0000", 2024-09-14T06:29:39.393 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.0249999999999999e-06 2024-09-14T06:29:39.393 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:39.393 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:39.394 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:29:39.394 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:26.344590+0000", 2024-09-14T06:29:39.394 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.0910000000000001e-06 2024-09-14T06:29:39.394 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:39.394 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:39.394 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:29:39.394 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:26.344593+0000", 2024-09-14T06:29:39.394 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.4199999999999999e-06 2024-09-14T06:29:39.394 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:39.394 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:39.394 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:29:39.394 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:26.369428+0000", 2024-09-14T06:29:39.395 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.024834927 2024-09-14T06:29:39.395 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:39.395 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:39.395 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:29:39.395 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:26.369465+0000", 2024-09-14T06:29:39.395 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.7092999999999999e-05 2024-09-14T06:29:39.395 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:39.395 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:39.395 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:29:39.396 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:26.369515+0000", 2024-09-14T06:29:39.396 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.977e-05 2024-09-14T06:29:39.396 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:39.396 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:29:39.396 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:39.396 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:39.396 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:39.396 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:17157 3.11 3:8b9b9188:::rbd_data.112d92027590.0000000000000017:f [read 136192~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:29:39.396 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:26.344596+0000", 2024-09-14T06:29:39.396 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 12.965061849, 2024-09-14T06:29:39.396 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.025164066999999998, 2024-09-14T06:29:39.396 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:29:39.396 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:29:39.396 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:29:39.397 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:29:39.397 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:29:39.397 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:39.397 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 17157 2024-09-14T06:29:39.397 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:39.397 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:29:39.397 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:39.397 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:29:39.397 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:26.344596+0000", 2024-09-14T06:29:39.397 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:39.397 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:39.397 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:39.397 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:29:39.397 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:26.344596+0000", 2024-09-14T06:29:39.398 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:39.398 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:39.398 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:39.398 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:29:39.398 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:26.344597+0000", 2024-09-14T06:29:39.398 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.0599999999999999e-07 2024-09-14T06:29:39.398 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:39.398 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:39.398 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:29:39.398 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:26.344598+0000", 2024-09-14T06:29:39.398 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.145e-06 2024-09-14T06:29:39.398 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:39.398 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:39.399 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:29:39.399 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:26.344601+0000", 2024-09-14T06:29:39.399 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.9809999999999999e-06 2024-09-14T06:29:39.399 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:39.399 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:39.399 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:29:39.399 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:26.344604+0000", 2024-09-14T06:29:39.399 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.0750000000000002e-06 2024-09-14T06:29:39.399 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:39.399 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:39.399 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:29:39.399 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:26.369571+0000", 2024-09-14T06:29:39.399 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.024967728000000002 2024-09-14T06:29:39.400 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:39.400 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:39.400 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:29:39.400 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:26.369653+0000", 2024-09-14T06:29:39.400 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.1312999999999999e-05 2024-09-14T06:29:39.400 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:39.400 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:39.400 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:29:39.400 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:26.369760+0000", 2024-09-14T06:29:39.400 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.000107619 2024-09-14T06:29:39.400 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:39.400 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:29:39.400 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:39.400 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:39.400 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:39.400 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:17158 3.11 3:8b9b9188:::rbd_data.112d92027590.0000000000000017:f [read 146432~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:29:39.401 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:26.344607+0000", 2024-09-14T06:29:39.401 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 12.965051452999999, 2024-09-14T06:29:39.401 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.025345412000000001, 2024-09-14T06:29:39.401 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:29:39.401 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:29:39.401 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:29:39.401 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:29:39.401 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:29:39.402 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:39.402 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 17158 2024-09-14T06:29:39.402 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:39.402 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:29:39.402 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:39.402 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:29:39.402 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:26.344607+0000", 2024-09-14T06:29:39.402 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:39.402 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:39.402 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:39.402 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:29:39.402 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:26.344607+0000", 2024-09-14T06:29:39.403 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:39.403 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:39.403 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:39.403 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:29:39.403 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:26.344607+0000", 2024-09-14T06:29:39.403 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.05e-07 2024-09-14T06:29:39.403 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:39.403 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:39.403 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:29:39.403 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:26.344609+0000", 2024-09-14T06:29:39.403 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.734e-06 2024-09-14T06:29:39.403 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:39.403 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:39.403 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:29:39.403 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:26.344613+0000", 2024-09-14T06:29:39.404 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.6430000000000003e-06 2024-09-14T06:29:39.404 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:39.404 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:39.404 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:29:39.404 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:26.344616+0000", 2024-09-14T06:29:39.404 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.5799999999999999e-06 2024-09-14T06:29:39.404 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:39.404 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:39.404 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:29:39.404 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:26.369805+0000", 2024-09-14T06:29:39.404 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.025189238999999999 2024-09-14T06:29:39.404 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:39.404 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:39.404 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:29:39.405 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:26.369877+0000", 2024-09-14T06:29:39.405 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.1717000000000005e-05 2024-09-14T06:29:39.405 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:39.405 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:39.405 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:29:39.405 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:26.369952+0000", 2024-09-14T06:29:39.405 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.5294000000000001e-05 2024-09-14T06:29:39.405 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:39.405 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:29:39.405 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:39.405 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:39.405 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:39.405 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:17159 3.11 3:8b9b9188:::rbd_data.112d92027590.0000000000000017:f [read 156672~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:29:39.406 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:26.344619+0000", 2024-09-14T06:29:39.406 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 12.965039249, 2024-09-14T06:29:39.406 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.025592119999999999, 2024-09-14T06:29:39.406 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:29:39.406 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:29:39.406 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:29:39.406 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:29:39.406 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:29:39.406 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:39.406 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 17159 2024-09-14T06:29:39.406 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:39.406 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:29:39.406 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:39.406 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:29:39.406 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:26.344619+0000", 2024-09-14T06:29:39.406 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:39.407 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:39.407 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:39.407 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:29:39.407 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:26.344619+0000", 2024-09-14T06:29:39.407 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:39.407 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:39.407 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:39.407 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:29:39.407 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:26.344619+0000", 2024-09-14T06:29:39.407 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.41e-07 2024-09-14T06:29:39.407 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:39.407 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:39.407 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:29:39.407 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:26.344620+0000", 2024-09-14T06:29:39.407 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.113e-06 2024-09-14T06:29:39.407 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:39.408 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:39.408 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:29:39.408 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:26.344623+0000", 2024-09-14T06:29:39.408 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.072e-06 2024-09-14T06:29:39.408 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:39.408 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:39.408 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:29:39.408 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:26.344626+0000", 2024-09-14T06:29:39.408 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.503e-06 2024-09-14T06:29:39.408 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:39.408 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:39.408 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:29:39.408 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:26.369987+0000", 2024-09-14T06:29:39.408 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.025361338000000001 2024-09-14T06:29:39.409 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:39.409 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:39.409 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:29:39.409 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:26.370078+0000", 2024-09-14T06:29:39.409 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.0662999999999996e-05 2024-09-14T06:29:39.409 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:39.409 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:39.409 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:29:39.409 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:26.370211+0000", 2024-09-14T06:29:39.409 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00013318999999999999 2024-09-14T06:29:39.409 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:39.410 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:29:39.410 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:39.410 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:39.410 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:29:39.410 INFO:teuthology.orchestra.run.smithi138.stdout:} 2024-09-14T06:29:39.412 INFO:tasks.ceph.osd.6.smithi138.stderr:2024-09-14T06:29:39.410+0000 7f8cd8849640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33620) UID: 0 2024-09-14T06:29:39.412 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-14T06:29:39.413 INFO:teuthology.orchestra.run.smithi094.stdout:776 trunc from 0xe89b25e to 0x20e200d 2024-09-14T06:29:39.413 INFO:teuthology.orchestra.run.smithi094.stdout:777 write 0x90b6ea2 thru 0x90c2cc2 (0xbe21 bytes) 2024-09-14T06:29:39.413 INFO:teuthology.orchestra.run.smithi094.stdout:778 write 0xab002ff thru 0xab02138 (0x1e3a bytes) 2024-09-14T06:29:39.413 INFO:teuthology.orchestra.run.smithi094.stdout:779 writesame 0xb4911a0 thru 0xb495baf data_size 0x50(0x4a10 bytes) 2024-09-14T06:29:39.413 INFO:teuthology.orchestra.run.smithi094.stdout:781 punch from 0x46a5023 to 0x46b33dd, (0xe3ba bytes) 2024-09-14T06:29:39.413 INFO:teuthology.orchestra.run.smithi094.stdout:782 writesame 0x6cc53b2 thru 0x6cd3ed2 data_size 0x1(0xeb21 bytes) 2024-09-14T06:29:39.413 INFO:teuthology.orchestra.run.smithi094.stdout:784 writesame 0xd9793f7 thru 0xd97de4d data_size 0x1(0x4a57 bytes) 2024-09-14T06:29:39.413 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-14T06:29:39.413 INFO:teuthology.orchestra.run.smithi094.stdout:786 writesame 0xa1c8588 thru 0xa1d38f6 data_size 0x1(0xb36f bytes) 2024-09-14T06:29:39.431 INFO:teuthology.orchestra.run.smithi094.stdout:787 read 0x6507a01 thru 0x6510958 (0x8f58 bytes) 2024-09-14T06:29:39.431 INFO:teuthology.orchestra.run.smithi094.stdout:788 read 0x7c6fc78 thru 0x7c7e635 (0xe9be bytes) 2024-09-14T06:29:39.431 INFO:teuthology.orchestra.run.smithi094.stdout:790 writesame 0x77f248b thru 0x77f2ab9 data_size 0x1(0x62f bytes) 2024-09-14T06:29:39.433 INFO:teuthology.orchestra.run.smithi094.stdout:791 read 0x3419f7d thru 0x341d5c8 (0x364c bytes) 2024-09-14T06:29:39.433 INFO:teuthology.orchestra.run.smithi094.stdout:792 writesame 0x90a6582 thru 0x90b265a data_size 0x1(0xc0d9 bytes) 2024-09-14T06:29:39.457 INFO:teuthology.orchestra.run.smithi094.stdout:795 write 0x72734f8 thru 0x7279b40 (0x6649 bytes) 2024-09-14T06:29:39.460 INFO:teuthology.orchestra.run.smithi094.stdout:797 punch from 0xbc4b3b5 to 0xbc52558, (0x71a3 bytes) 2024-09-14T06:29:39.468 INFO:teuthology.orchestra.run.smithi094.stdout:798 trunc from 0xd97de4e to 0x6fa7b44 2024-09-14T06:29:39.494 INFO:teuthology.orchestra.run.smithi094.stdout:799 writesame 0xcbe4cd2 thru 0xcbeb7c3 data_size 0x75(0x6af2 bytes) 2024-09-14T06:29:39.496 INFO:teuthology.orchestra.run.smithi094.stdout:800 trunc from 0xcbeb7c4 to 0xb34b060 2024-09-14T06:29:39.509 INFO:teuthology.orchestra.run.smithi094.stdout:801 writesame 0xc8d6681 thru 0xc8e30ff data_size 0x1(0xca7f bytes) 2024-09-14T06:29:39.512 INFO:teuthology.orchestra.run.smithi094.stdout:802 trunc from 0xc8e3100 to 0xa696c2d 2024-09-14T06:29:39.512 INFO:tasks.ceph.osd.3.smithi094.stderr:2024-09-14T06:29:39.511+0000 7feeeafe8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33655) UID: 0 2024-09-14T06:29:39.526 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-14T06:29:39.526 INFO:teuthology.orchestra.run.smithi094.stdout: "ops": [], 2024-09-14T06:29:39.526 INFO:teuthology.orchestra.run.smithi094.stdout: "num_ops": 0 2024-09-14T06:29:39.527 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-14T06:29:39.528 INFO:teuthology.orchestra.run.smithi094.stdout:803 trunc from 0xa696c2d to 0xd3f31b 2024-09-14T06:29:39.537 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-14T06:29:39.544 INFO:teuthology.orchestra.run.smithi094.stdout:804 punch from 0x1d9938 to 0x1e5ce3, (0xc3ab bytes) 2024-09-14T06:29:39.544 INFO:teuthology.orchestra.run.smithi094.stdout:805 punch from 0xaf05af to 0xaf867a, (0x80cb bytes) 2024-09-14T06:29:39.558 INFO:teuthology.orchestra.run.smithi094.stdout:806 trunc from 0xd3f31b to 0xcd5c670 2024-09-14T06:29:39.565 INFO:teuthology.orchestra.run.smithi094.stdout:808 writesame 0x2faa320 thru 0x2fab297 data_size 0x24(0xf78 bytes) 2024-09-14T06:29:39.568 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-14T06:29:39.568 INFO:teuthology.orchestra.run.smithi094.stdout:810 punch from 0xc53cf8c to 0xc54b154, (0xe1c8 bytes) 2024-09-14T06:29:39.568 INFO:teuthology.orchestra.run.smithi094.stdout:811 read 0x3188d9 thru 0x3280e0 (0xf808 bytes) 2024-09-14T06:29:39.575 INFO:teuthology.orchestra.run.smithi094.stdout:812 trunc from 0xcd5c670 to 0x75f3f1f 2024-09-14T06:29:39.581 INFO:teuthology.orchestra.run.smithi094.stdout:813 read 0xbd9f3d thru 0xbe840c (0xe4d0 bytes) 2024-09-14T06:29:39.595 INFO:teuthology.orchestra.run.smithi094.stdout:814 write 0xe85ae66 thru 0xe868a49 (0xdbe4 bytes) 2024-09-14T06:29:39.599 INFO:teuthology.orchestra.run.smithi094.stdout:815 read 0xa4400d3 thru 0xa44f685 (0xf5b3 bytes) 2024-09-14T06:29:39.599 INFO:teuthology.orchestra.run.smithi094.stdout:816 write 0x9ffa2aa thru 0xa004006 (0x9d5d bytes) 2024-09-14T06:29:39.603 INFO:teuthology.orchestra.run.smithi094.stdout:818 punch from 0x7096d3a to 0x70999d7, (0x2c9d bytes) 2024-09-14T06:29:39.603 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-14T06:29:39.603 INFO:teuthology.orchestra.run.smithi094.stdout:820 read 0x9f1fc52 thru 0x9f2bdd9 (0xc188 bytes) 2024-09-14T06:29:39.603 INFO:teuthology.orchestra.run.smithi094.stdout:822 read 0xc291017 thru 0xc2987fd (0x77e7 bytes) 2024-09-14T06:29:39.604 INFO:teuthology.orchestra.run.smithi094.stdout:823 write 0x6389f8d thru 0x6398944 (0xe9b8 bytes) 2024-09-14T06:29:39.613 INFO:teuthology.orchestra.run.smithi094.stdout:824 write 0xed9afec thru 0xeda8be4 (0xdbf9 bytes) 2024-09-14T06:29:39.613 INFO:tasks.ceph.osd.3.smithi094.stderr:2024-09-14T06:29:39.612+0000 7feeeafe8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33655) UID: 0 2024-09-14T06:29:39.626 INFO:teuthology.orchestra.run.smithi094.stdout:825 trunc from 0xeda8be5 to 0x6481b1c 2024-09-14T06:29:39.637 INFO:teuthology.orchestra.run.smithi094.stdout:827 punch from 0x4c8169b to 0x4c82395, (0xcfa bytes) 2024-09-14T06:29:39.652 INFO:teuthology.orchestra.run.smithi094.stdout:829 writesame 0xebbbba2 thru 0xebbfc17 data_size 0x2(0x4076 bytes) 2024-09-14T06:29:39.652 INFO:teuthology.orchestra.run.smithi094.stdout:830 punch from 0x5faeb3e to 0x5fb1a57, (0x2f19 bytes) 2024-09-14T06:29:39.652 INFO:teuthology.orchestra.run.smithi094.stdout:831 punch from 0x2419c73 to 0x2423776, (0x9b03 bytes) 2024-09-14T06:29:39.652 INFO:teuthology.orchestra.run.smithi094.stdout:832 read 0xc3183c6 thru 0xc32055a (0x8195 bytes) 2024-09-14T06:29:39.653 INFO:teuthology.orchestra.run.smithi094.stdout:834 writesame 0x1e1121c thru 0x1e145eb data_size 0x8(0x33d0 bytes) 2024-09-14T06:29:39.663 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-14T06:29:39.663 INFO:teuthology.orchestra.run.smithi094.stdout: "ops": [], 2024-09-14T06:29:39.663 INFO:teuthology.orchestra.run.smithi094.stdout: "complaint_time": 30, 2024-09-14T06:29:39.663 INFO:teuthology.orchestra.run.smithi094.stdout: "num_blocked_ops": 0 2024-09-14T06:29:39.663 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-14T06:29:39.668 INFO:teuthology.orchestra.run.smithi094.stdout:835 trunc from 0xebbfc18 to 0x35a629b 2024-09-14T06:29:39.674 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-14T06:29:39.679 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-14T06:29:39.679 INFO:teuthology.orchestra.run.smithi094.stdout:838 read 0x10e4d69 thru 0x10f021a (0xb4b2 bytes) 2024-09-14T06:29:39.679 INFO:teuthology.orchestra.run.smithi094.stdout:840 punch from 0x283add4 to 0x284a32b, (0xf557 bytes) 2024-09-14T06:29:39.692 INFO:teuthology.orchestra.run.smithi094.stdout:841 write 0xa09d103 thru 0xa0a707b (0x9f79 bytes) 2024-09-14T06:29:39.695 INFO:teuthology.orchestra.run.smithi094.stdout:843 read 0x6b3cba thru 0x6be01f (0xa366 bytes) 2024-09-14T06:29:39.707 INFO:teuthology.orchestra.run.smithi094.stdout:844 write 0xe1e47e1 thru 0xe1e5ecd (0x16ed bytes) 2024-09-14T06:29:39.710 INFO:teuthology.orchestra.run.smithi094.stdout:845 write 0xb31abfb thru 0xb329a90 (0xee96 bytes) 2024-09-14T06:29:39.713 INFO:teuthology.orchestra.run.smithi094.stdout:846 writesame 0x83f5c59 thru 0x83f7411 data_size 0x1(0x17b9 bytes) 2024-09-14T06:29:39.713 INFO:tasks.ceph.osd.5.smithi138.stderr:2024-09-14T06:29:39.712+0000 7fe13746f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33618) UID: 0 2024-09-14T06:29:39.718 INFO:teuthology.orchestra.run.smithi094.stdout:849 writesame 0xcdd6986 thru 0xcde5d31 data_size 0xa(0xf3ac bytes) 2024-09-14T06:29:39.720 INFO:teuthology.orchestra.run.smithi094.stderr:nodeep-scrub is set 2024-09-14T06:29:39.725 INFO:teuthology.orchestra.run.smithi094.stdout:850 punch from 0x90669cc to 0x90681aa, (0x17de bytes) 2024-09-14T06:29:39.725 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-14T06:29:39.725 INFO:teuthology.orchestra.run.smithi094.stdout:853 read 0x4c72687 thru 0x4c7d496 (0xae10 bytes) 2024-09-14T06:29:39.725 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-14T06:29:39.736 INFO:teuthology.orchestra.run.smithi094.stdout:856 trunc from 0xe1e5ece to 0x59a67b5 2024-09-14T06:29:39.751 INFO:teuthology.orchestra.run.smithi094.stdout:857 punch from 0x353b610 to 0x354b245, (0xfc35 bytes) 2024-09-14T06:29:39.751 INFO:teuthology.orchestra.run.smithi094.stdout:859 writesame 0x4db0888 thru 0x4db831b data_size 0x1e(0x7a94 bytes) 2024-09-14T06:29:39.754 INFO:teuthology.orchestra.run.smithi094.stdout:860 read 0x1b192ee thru 0x1b25549 (0xc25c bytes) 2024-09-14T06:29:39.754 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-14T06:29:39.754 INFO:teuthology.orchestra.run.smithi094.stdout:862 punch from 0x496db92 to 0x496f10f, (0x157d bytes) 2024-09-14T06:29:39.755 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-14T06:29:39.755 INFO:teuthology.orchestra.run.smithi094.stdout:864 read 0x590d5ed thru 0x590f47c (0x1e90 bytes) 2024-09-14T06:29:39.755 INFO:teuthology.orchestra.run.smithi094.stdout:865 read 0xa5360c thru 0xa55a01 (0x23f6 bytes) 2024-09-14T06:29:39.755 INFO:teuthology.orchestra.run.smithi094.stdout:867 writesame 0xfd8ccd thru 0xfe655a data_size 0x35(0xd88e bytes) 2024-09-14T06:29:39.757 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-14T06:29:39.761 INFO:teuthology.orchestra.run.smithi094.stdout:869 trunc from 0x59a67b5 to 0x988a310 2024-09-14T06:29:39.766 INFO:teuthology.orchestra.run.smithi094.stdout:874 writesame 0x3b7025a thru 0x3b71a95 data_size 0x2f(0x183c bytes) 2024-09-14T06:29:39.769 INFO:teuthology.orchestra.run.smithi094.stdout:875 write 0x1d22a6f thru 0x1d2a8b2 (0x7e44 bytes) 2024-09-14T06:29:39.771 INFO:teuthology.orchestra.run.smithi094.stdout:876 read 0x894e02d thru 0x894e364 (0x338 bytes) 2024-09-14T06:29:39.771 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-14T06:29:39.771 INFO:teuthology.orchestra.run.smithi094.stdout:879 read 0x6b83359 thru 0x6b89321 (0x5fc9 bytes) 2024-09-14T06:29:39.772 INFO:teuthology.orchestra.run.smithi094.stdout:880 punch from 0x11ff663 to 0x12083f4, (0x8d91 bytes) 2024-09-14T06:29:39.772 INFO:teuthology.orchestra.run.smithi094.stdout:881 punch from 0x310566 to 0x315215, (0x4caf bytes) 2024-09-14T06:29:39.776 INFO:teuthology.orchestra.run.smithi094.stdout:882 trunc from 0x988a310 to 0x51fdcc4 2024-09-14T06:29:39.783 INFO:teuthology.orchestra.run.smithi094.stdout:884 write 0x504c8cb thru 0x504e3af (0x1ae5 bytes) 2024-09-14T06:29:39.797 INFO:teuthology.orchestra.run.smithi094.stdout:885 write 0xc2df856 thru 0xc2e4867 (0x5012 bytes) 2024-09-14T06:29:39.800 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-14T06:29:39.803 INFO:teuthology.orchestra.run.smithi094.stdout:887 trunc from 0xc2e4868 to 0xe8628bc 2024-09-14T06:29:39.805 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-14T06:29:39.805 INFO:teuthology.orchestra.run.smithi094.stdout: "size": 20, 2024-09-14T06:29:39.805 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 600, 2024-09-14T06:29:39.805 INFO:teuthology.orchestra.run.smithi094.stdout: "ops": [ 2024-09-14T06:29:39.806 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:39.806 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:11860 3.12 3:4a837b73:::rbd_data.112d92027590.0000000000000000:f [read 735232~1024 [fadvise_nocache]] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:29:39.806 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:25.716255+0000", 2024-09-14T06:29:39.806 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 14.087322695999999, 2024-09-14T06:29:39.806 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.049569145000000002, 2024-09-14T06:29:39.806 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:29:39.806 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:29:39.806 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:29:39.806 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:29:39.806 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:29:39.806 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:39.806 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 11860 2024-09-14T06:29:39.806 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:39.806 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:29:39.807 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:39.807 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:29:39.807 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.716255+0000", 2024-09-14T06:29:39.807 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:39.807 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:39.807 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:39.807 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:29:39.807 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.716255+0000", 2024-09-14T06:29:39.807 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:39.807 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:39.807 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:39.807 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:29:39.807 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.716255+0000", 2024-09-14T06:29:39.807 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.1199999999999998e-07 2024-09-14T06:29:39.808 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:39.808 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:39.808 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:29:39.808 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.716257+0000", 2024-09-14T06:29:39.808 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.024e-06 2024-09-14T06:29:39.808 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:39.808 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:39.808 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:29:39.808 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.716265+0000", 2024-09-14T06:29:39.808 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.5259999999999996e-06 2024-09-14T06:29:39.808 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:39.808 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:39.809 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:29:39.809 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.716270+0000", 2024-09-14T06:29:39.809 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.0730000000000004e-06 2024-09-14T06:29:39.809 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:39.809 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:39.809 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:29:39.809 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.765618+0000", 2024-09-14T06:29:39.809 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.049348141999999998 2024-09-14T06:29:39.809 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:39.809 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:39.809 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:29:39.809 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.765703+0000", 2024-09-14T06:29:39.809 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.5594999999999996e-05 2024-09-14T06:29:39.809 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:39.810 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:39.810 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:29:39.810 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.765824+0000", 2024-09-14T06:29:39.810 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00012037299999999999 2024-09-14T06:29:39.810 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:39.810 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:39.810 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:39.810 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:39.810 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:39.810 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:11861 3.12 3:4a837b73:::rbd_data.112d92027590.0000000000000000:f [read 745472~1024 [fadvise_nocache]] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:29:39.810 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:25.716287+0000", 2024-09-14T06:29:39.810 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 14.087289875, 2024-09-14T06:29:39.810 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.049669320000000003, 2024-09-14T06:29:39.810 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:29:39.810 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:29:39.811 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:29:39.811 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:29:39.811 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:29:39.811 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:39.811 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 11861 2024-09-14T06:29:39.811 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:39.811 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:29:39.811 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:39.811 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:29:39.811 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.716287+0000", 2024-09-14T06:29:39.811 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:39.812 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:39.812 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:39.812 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:29:39.812 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.716287+0000", 2024-09-14T06:29:39.812 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:39.812 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:39.812 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:39.812 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:29:39.812 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.716288+0000", 2024-09-14T06:29:39.812 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.2099999999999998e-07 2024-09-14T06:29:39.812 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:39.812 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:39.812 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:29:39.813 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.716289+0000", 2024-09-14T06:29:39.813 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.5090000000000001e-06 2024-09-14T06:29:39.813 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:39.813 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:39.813 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:29:39.813 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.716296+0000", 2024-09-14T06:29:39.813 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.4930000000000003e-06 2024-09-14T06:29:39.813 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:39.813 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:39.814 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:29:39.814 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.716298+0000", 2024-09-14T06:29:39.814 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.6960000000000001e-06 2024-09-14T06:29:39.814 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:39.814 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:39.814 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:29:39.814 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.765856+0000", 2024-09-14T06:29:39.814 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.049557714000000003 2024-09-14T06:29:39.814 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:39.814 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:39.814 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:29:39.814 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.765900+0000", 2024-09-14T06:29:39.814 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.3708000000000002e-05 2024-09-14T06:29:39.814 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:39.815 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:39.815 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:29:39.815 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.765957+0000", 2024-09-14T06:29:39.815 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.6879000000000002e-05 2024-09-14T06:29:39.815 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:39.815 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:39.815 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:39.815 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:39.815 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:39.815 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:11862 3.12 3:4a837b73:::rbd_data.112d92027590.0000000000000000:f [read 755712~1024 [fadvise_nocache]] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:29:39.815 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:25.716302+0000", 2024-09-14T06:29:39.815 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 14.087275346, 2024-09-14T06:29:39.815 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.049889259999999998, 2024-09-14T06:29:39.815 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:29:39.816 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:29:39.816 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:29:39.816 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:29:39.816 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:29:39.816 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:39.816 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 11862 2024-09-14T06:29:39.816 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:39.816 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:29:39.816 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:39.816 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:29:39.816 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.716302+0000", 2024-09-14T06:29:39.816 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:39.816 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:39.816 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:39.816 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:29:39.817 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.716302+0000", 2024-09-14T06:29:39.817 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:39.817 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:39.817 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:39.817 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:29:39.817 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.716302+0000", 2024-09-14T06:29:39.817 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.1500000000000001e-07 2024-09-14T06:29:39.817 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:39.817 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:39.817 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:29:39.817 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.716306+0000", 2024-09-14T06:29:39.817 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.8840000000000004e-06 2024-09-14T06:29:39.817 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:39.817 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:39.818 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:29:39.818 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.716310+0000", 2024-09-14T06:29:39.818 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.6229999999999999e-06 2024-09-14T06:29:39.818 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:39.818 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:39.818 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:29:39.818 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.716317+0000", 2024-09-14T06:29:39.818 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.3869999999999997e-06 2024-09-14T06:29:39.818 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:39.818 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:39.818 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:29:39.818 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.765995+0000", 2024-09-14T06:29:39.818 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.049677683 2024-09-14T06:29:39.819 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:39.819 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:39.819 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:29:39.819 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.766080+0000", 2024-09-14T06:29:39.819 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.5296999999999993e-05 2024-09-14T06:29:39.819 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:39.819 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:39.819 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:29:39.819 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.766191+0000", 2024-09-14T06:29:39.819 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000111171 2024-09-14T06:29:39.819 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:39.819 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:39.820 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:39.820 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:39.820 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:39.820 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:11863 3.12 3:4a837b73:::rbd_data.112d92027590.0000000000000000:f [read 765952~1024 [fadvise_nocache]] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:29:39.820 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:25.716321+0000", 2024-09-14T06:29:39.820 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 14.087256742999999, 2024-09-14T06:29:39.820 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.050017788000000001, 2024-09-14T06:29:39.820 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:29:39.820 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:29:39.820 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:29:39.820 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:29:39.820 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:29:39.820 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:39.820 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 11863 2024-09-14T06:29:39.821 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:39.821 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:29:39.821 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:39.821 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:29:39.821 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.716321+0000", 2024-09-14T06:29:39.821 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:39.821 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:39.821 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:39.821 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:29:39.821 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.716321+0000", 2024-09-14T06:29:39.821 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:39.821 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:39.821 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:39.822 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:29:39.822 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.721995+0000", 2024-09-14T06:29:39.822 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0056740200000000001 2024-09-14T06:29:39.822 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:39.822 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:39.822 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:29:39.822 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.722001+0000", 2024-09-14T06:29:39.822 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.815e-06 2024-09-14T06:29:39.822 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:39.822 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:39.822 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:29:39.822 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.722014+0000", 2024-09-14T06:29:39.822 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.2371000000000001e-05 2024-09-14T06:29:39.822 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:39.823 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:39.823 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:29:39.823 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.722022+0000", 2024-09-14T06:29:39.823 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.2770000000000005e-06 2024-09-14T06:29:39.823 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:39.823 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:39.823 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:29:39.823 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.766234+0000", 2024-09-14T06:29:39.823 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.044211651999999997 2024-09-14T06:29:39.823 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:39.823 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:39.824 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:29:39.824 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.766280+0000", 2024-09-14T06:29:39.824 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.5871000000000002e-05 2024-09-14T06:29:39.824 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:39.824 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:39.824 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:29:39.824 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.766338+0000", 2024-09-14T06:29:39.824 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.8782000000000001e-05 2024-09-14T06:29:39.824 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:39.824 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:39.824 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:39.824 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:39.824 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:39.825 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:12016 3.12 3:4a837b73:::rbd_data.112d92027590.0000000000000000:f [read 9216~1024 [fadvise_nocache]] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:29:39.825 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:25.722677+0000", 2024-09-14T06:29:39.825 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 14.080900329, 2024-09-14T06:29:39.825 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.049707673000000001, 2024-09-14T06:29:39.825 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:29:39.825 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:29:39.825 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:29:39.825 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:29:39.825 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:29:39.825 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:39.825 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 12016 2024-09-14T06:29:39.825 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:39.825 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:29:39.825 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:39.826 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:29:39.826 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.722677+0000", 2024-09-14T06:29:39.826 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:39.826 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:39.826 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:39.826 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:29:39.826 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.722677+0000", 2024-09-14T06:29:39.826 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:39.826 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:39.826 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:39.826 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:29:39.826 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.722677+0000", 2024-09-14T06:29:39.826 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.4200000000000002e-07 2024-09-14T06:29:39.827 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:39.827 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:39.827 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:29:39.827 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.722679+0000", 2024-09-14T06:29:39.827 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.756e-06 2024-09-14T06:29:39.827 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:39.827 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:39.827 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:29:39.827 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.722689+0000", 2024-09-14T06:29:39.827 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.6010000000000001e-06 2024-09-14T06:29:39.827 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:39.827 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:39.827 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:29:39.828 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.722694+0000", 2024-09-14T06:29:39.828 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.9919999999999998e-06 2024-09-14T06:29:39.828 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:39.828 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:39.828 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:29:39.828 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.772184+0000", 2024-09-14T06:29:39.828 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.049490037000000001 2024-09-14T06:29:39.828 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:39.828 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:39.828 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:29:39.828 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.772272+0000", 2024-09-14T06:29:39.828 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.8553000000000004e-05 2024-09-14T06:29:39.828 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:39.828 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:39.829 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:29:39.829 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.772385+0000", 2024-09-14T06:29:39.829 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000112392 2024-09-14T06:29:39.829 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:39.829 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:39.829 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:39.829 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:39.829 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:39.829 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:12017 3.12 3:4a837b73:::rbd_data.112d92027590.0000000000000000:f [read 19456~1024 [fadvise_nocache]] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:29:39.829 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:25.722700+0000", 2024-09-14T06:29:39.829 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 14.080877399, 2024-09-14T06:29:39.829 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.049819216, 2024-09-14T06:29:39.829 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:29:39.829 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:29:39.830 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:29:39.830 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:29:39.830 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:29:39.830 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:39.830 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 12017 2024-09-14T06:29:39.830 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:39.830 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:29:39.830 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:39.830 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:29:39.830 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.722700+0000", 2024-09-14T06:29:39.830 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:39.830 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:39.830 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:39.830 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:29:39.830 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.722700+0000", 2024-09-14T06:29:39.831 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:39.831 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:39.831 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:39.831 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:29:39.831 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.722700+0000", 2024-09-14T06:29:39.831 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.9200000000000002e-07 2024-09-14T06:29:39.831 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:39.831 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:39.831 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:29:39.831 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.722702+0000", 2024-09-14T06:29:39.831 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.979e-06 2024-09-14T06:29:39.831 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:39.831 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:39.831 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:29:39.832 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.722709+0000", 2024-09-14T06:29:39.832 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.3260000000000001e-06 2024-09-14T06:29:39.832 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:39.832 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:39.832 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:29:39.832 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.722714+0000", 2024-09-14T06:29:39.832 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.9280000000000001e-06 2024-09-14T06:29:39.832 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:39.832 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:39.832 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:29:39.832 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.772418+0000", 2024-09-14T06:29:39.832 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.049704904000000001 2024-09-14T06:29:39.832 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:39.832 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:39.833 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:29:39.833 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.772462+0000", 2024-09-14T06:29:39.833 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.3509e-05 2024-09-14T06:29:39.833 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:39.833 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:39.833 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:29:39.833 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.772519+0000", 2024-09-14T06:29:39.833 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.7178e-05 2024-09-14T06:29:39.833 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:39.833 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:39.833 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:39.833 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:39.833 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:39.833 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:12018 3.12 3:4a837b73:::rbd_data.112d92027590.0000000000000000:f [read 29696~1024 [fadvise_nocache]] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:29:39.834 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:25.722728+0000", 2024-09-14T06:29:39.834 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 14.080849733000001, 2024-09-14T06:29:39.834 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.050040655000000003, 2024-09-14T06:29:39.834 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:29:39.834 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:29:39.834 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:29:39.834 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:29:39.834 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:29:39.834 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:39.834 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 12018 2024-09-14T06:29:39.835 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:39.835 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:29:39.835 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:39.835 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:29:39.835 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.722728+0000", 2024-09-14T06:29:39.835 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:39.835 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:39.835 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:39.835 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:29:39.835 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.722728+0000", 2024-09-14T06:29:39.835 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:39.835 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:39.835 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:39.836 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:29:39.836 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.722728+0000", 2024-09-14T06:29:39.836 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.6300000000000005e-07 2024-09-14T06:29:39.836 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:39.836 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:39.836 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:29:39.836 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.722731+0000", 2024-09-14T06:29:39.836 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.3020000000000002e-06 2024-09-14T06:29:39.836 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:39.836 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:39.836 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:29:39.836 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.722737+0000", 2024-09-14T06:29:39.836 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.4869999999999999e-06 2024-09-14T06:29:39.836 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:39.837 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:39.837 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:29:39.837 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.722742+0000", 2024-09-14T06:29:39.837 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.8080000000000004e-06 2024-09-14T06:29:39.837 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:39.837 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:39.837 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:29:39.837 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.772563+0000", 2024-09-14T06:29:39.837 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.049821217000000001 2024-09-14T06:29:39.837 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:39.837 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:39.838 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:29:39.838 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.772653+0000", 2024-09-14T06:29:39.838 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.0346000000000005e-05 2024-09-14T06:29:39.838 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:39.838 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:39.838 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:29:39.838 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.772768+0000", 2024-09-14T06:29:39.838 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000114832 2024-09-14T06:29:39.838 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:39.838 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:39.838 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:39.838 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:39.838 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:39.839 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:12019 3.12 3:4a837b73:::rbd_data.112d92027590.0000000000000000:f [read 39936~1024 [fadvise_nocache]] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:29:39.839 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:25.722749+0000", 2024-09-14T06:29:39.839 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 14.080828519000001, 2024-09-14T06:29:39.839 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.050265848000000002, 2024-09-14T06:29:39.839 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:29:39.839 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:29:39.839 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:29:39.839 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:29:39.839 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:29:39.839 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:39.839 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 12019 2024-09-14T06:29:39.839 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:39.839 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:29:39.839 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:39.840 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:29:39.840 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.722749+0000", 2024-09-14T06:29:39.840 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:39.840 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:39.840 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:39.840 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:29:39.840 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.722749+0000", 2024-09-14T06:29:39.840 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:39.840 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:39.840 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:39.840 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:29:39.840 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.722749+0000", 2024-09-14T06:29:39.840 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.39e-07 2024-09-14T06:29:39.840 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:39.841 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:39.841 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:29:39.841 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.722751+0000", 2024-09-14T06:29:39.841 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.8780000000000001e-06 2024-09-14T06:29:39.841 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:39.841 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:39.841 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:29:39.841 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.722757+0000", 2024-09-14T06:29:39.841 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.7289999999999997e-06 2024-09-14T06:29:39.841 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:39.841 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:39.841 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:29:39.841 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.722761+0000", 2024-09-14T06:29:39.842 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.685e-06 2024-09-14T06:29:39.842 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:39.842 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:39.842 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:29:39.842 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.772802+0000", 2024-09-14T06:29:39.842 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.050040544999999999 2024-09-14T06:29:39.842 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:39.842 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:39.842 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:29:39.842 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.772900+0000", 2024-09-14T06:29:39.842 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.7921e-05 2024-09-14T06:29:39.842 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:39.842 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:39.843 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:29:39.843 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.773015+0000", 2024-09-14T06:29:39.843 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000114751 2024-09-14T06:29:39.843 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:39.843 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:39.843 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:39.843 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:39.843 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:39.843 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:12020 3.12 3:4a837b73:::rbd_data.112d92027590.0000000000000000:f [read 50176~1024 [fadvise_nocache]] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:29:39.843 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:25.722767+0000", 2024-09-14T06:29:39.843 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 14.080810279, 2024-09-14T06:29:39.844 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.050402935000000003, 2024-09-14T06:29:39.844 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:29:39.844 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:29:39.844 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:29:39.844 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:29:39.844 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:29:39.844 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:39.844 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 12020 2024-09-14T06:29:39.845 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:39.845 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:29:39.845 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:39.845 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:29:39.845 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.722767+0000", 2024-09-14T06:29:39.845 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:39.845 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:39.845 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:39.845 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:29:39.845 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.722767+0000", 2024-09-14T06:29:39.845 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:39.845 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:39.845 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:39.845 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:29:39.846 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.722767+0000", 2024-09-14T06:29:39.846 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.46e-07 2024-09-14T06:29:39.846 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:39.846 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:39.846 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:29:39.846 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.722769+0000", 2024-09-14T06:29:39.846 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.6810000000000001e-06 2024-09-14T06:29:39.846 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:39.846 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:39.846 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:29:39.846 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.722774+0000", 2024-09-14T06:29:39.846 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.3689999999999998e-06 2024-09-14T06:29:39.846 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:39.846 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:39.846 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:29:39.847 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.722779+0000", 2024-09-14T06:29:39.847 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.7079999999999996e-06 2024-09-14T06:29:39.847 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:39.847 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:39.847 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:29:39.847 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.773055+0000", 2024-09-14T06:29:39.847 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.050276280999999999 2024-09-14T06:29:39.847 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:39.847 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:39.847 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:29:39.847 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.773106+0000", 2024-09-14T06:29:39.847 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.1088000000000003e-05 2024-09-14T06:29:39.847 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:39.848 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:39.848 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:29:39.848 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.773170+0000", 2024-09-14T06:29:39.848 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.3461999999999996e-05 2024-09-14T06:29:39.848 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:39.848 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:39.848 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:39.848 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:39.848 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:39.848 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:12021 3.12 3:4a837b73:::rbd_data.112d92027590.0000000000000000:f [read 60416~1024 [fadvise_nocache]] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:29:39.848 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:25.722785+0000", 2024-09-14T06:29:39.848 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 14.080792616, 2024-09-14T06:29:39.848 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.050634957000000001, 2024-09-14T06:29:39.848 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:29:39.849 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:29:39.849 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:29:39.849 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:29:39.849 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:29:39.849 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:39.849 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 12021 2024-09-14T06:29:39.849 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:39.849 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:29:39.849 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:39.849 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:29:39.849 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.722785+0000", 2024-09-14T06:29:39.849 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:39.849 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:39.849 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:39.850 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:29:39.850 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.722785+0000", 2024-09-14T06:29:39.850 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:39.850 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:39.850 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:39.850 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:29:39.850 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.722785+0000", 2024-09-14T06:29:39.850 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.53e-07 2024-09-14T06:29:39.850 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:39.850 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:39.850 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:29:39.850 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.722793+0000", 2024-09-14T06:29:39.850 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.7330000000000003e-06 2024-09-14T06:29:39.850 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:39.851 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:39.851 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:29:39.851 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.722801+0000", 2024-09-14T06:29:39.851 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.5550000000000003e-06 2024-09-14T06:29:39.851 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:39.851 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:39.851 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:29:39.851 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.722809+0000", 2024-09-14T06:29:39.851 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.0700000000000007e-06 2024-09-14T06:29:39.851 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:39.851 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:39.851 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:29:39.851 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.773222+0000", 2024-09-14T06:29:39.851 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.050412476999999997 2024-09-14T06:29:39.852 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:39.852 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:39.852 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:29:39.852 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.773308+0000", 2024-09-14T06:29:39.852 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.6174999999999994e-05 2024-09-14T06:29:39.852 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:39.852 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:39.852 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:29:39.853 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.773420+0000", 2024-09-14T06:29:39.853 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000111594 2024-09-14T06:29:39.853 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:39.853 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:39.853 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:39.853 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:39.853 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:39.853 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:12022 3.12 3:4a837b73:::rbd_data.112d92027590.0000000000000000:f [read 70656~1024 [fadvise_nocache]] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:29:39.853 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:25.722817+0000", 2024-09-14T06:29:39.853 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 14.080760753, 2024-09-14T06:29:39.854 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.050817018999999998, 2024-09-14T06:29:39.854 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:29:39.854 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:29:39.854 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:29:39.854 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:29:39.854 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:29:39.854 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:39.854 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 12022 2024-09-14T06:29:39.854 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:39.854 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:29:39.855 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:39.855 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:29:39.855 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.722817+0000", 2024-09-14T06:29:39.855 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:39.855 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:39.855 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:39.856 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:29:39.856 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.722817+0000", 2024-09-14T06:29:39.856 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:39.856 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:39.856 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:39.857 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:29:39.857 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.728894+0000", 2024-09-14T06:29:39.857 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0060776490000000001 2024-09-14T06:29:39.857 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:39.857 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:39.857 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:29:39.858 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.728905+0000", 2024-09-14T06:29:39.858 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.1181e-05 2024-09-14T06:29:39.858 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:39.858 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:39.858 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:29:39.858 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.728918+0000", 2024-09-14T06:29:39.858 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.2781e-05 2024-09-14T06:29:39.858 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:39.858 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:39.858 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:29:39.858 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.728935+0000", 2024-09-14T06:29:39.859 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.7314999999999999e-05 2024-09-14T06:29:39.859 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:39.859 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:39.859 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:29:39.859 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.773458+0000", 2024-09-14T06:29:39.859 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.044522300000000001 2024-09-14T06:29:39.859 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:39.860 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:39.860 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:29:39.860 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.773513+0000", 2024-09-14T06:29:39.860 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.4758e-05 2024-09-14T06:29:39.860 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:39.860 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:39.860 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:29:39.860 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.773634+0000", 2024-09-14T06:29:39.860 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000121035 2024-09-14T06:29:39.860 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:39.860 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:39.860 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:39.860 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:39.860 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:39.861 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:12058 3.12 3:4a837b73:::rbd_data.112d92027590.0000000000000000:f [read 439296~1024 [fadvise_nocache]] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:29:39.861 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:25.729608+0000", 2024-09-14T06:29:39.861 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 14.073969184999999, 2024-09-14T06:29:39.861 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.049566225999999998, 2024-09-14T06:29:39.861 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:29:39.861 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:29:39.861 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:29:39.861 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:29:39.861 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:29:39.861 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:39.861 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 12058 2024-09-14T06:29:39.861 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:39.861 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:29:39.861 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:39.862 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:29:39.862 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.729608+0000", 2024-09-14T06:29:39.862 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:39.862 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:39.862 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:39.862 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:29:39.862 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.729608+0000", 2024-09-14T06:29:39.862 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:39.862 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:39.862 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:39.862 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:29:39.862 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.729608+0000", 2024-09-14T06:29:39.862 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.0900000000000001e-07 2024-09-14T06:29:39.862 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:39.863 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:39.863 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:29:39.863 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.729612+0000", 2024-09-14T06:29:39.863 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.1790000000000001e-06 2024-09-14T06:29:39.863 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:39.863 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:39.863 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:29:39.863 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.729616+0000", 2024-09-14T06:29:39.863 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.2890000000000002e-06 2024-09-14T06:29:39.863 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:39.863 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:39.863 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:29:39.863 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.729624+0000", 2024-09-14T06:29:39.864 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.0120000000000005e-06 2024-09-14T06:29:39.864 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:39.864 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:39.864 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:29:39.864 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.779081+0000", 2024-09-14T06:29:39.864 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.049457138999999997 2024-09-14T06:29:39.864 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:39.864 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:39.864 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:29:39.864 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.779121+0000", 2024-09-14T06:29:39.864 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.9972000000000003e-05 2024-09-14T06:29:39.864 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:39.864 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:39.864 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:29:39.865 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.779174+0000", 2024-09-14T06:29:39.865 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.3426e-05 2024-09-14T06:29:39.865 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:39.865 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:39.865 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:39.865 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:39.865 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:39.865 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:12059 3.12 3:4a837b73:::rbd_data.112d92027590.0000000000000000:f [read 449536~1024 [fadvise_nocache]] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:29:39.865 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:25.729627+0000", 2024-09-14T06:29:39.865 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 14.073949946999999, 2024-09-14T06:29:39.865 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.049660990000000002, 2024-09-14T06:29:39.865 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:29:39.865 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:29:39.866 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:29:39.866 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:29:39.866 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:29:39.866 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:39.866 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 12059 2024-09-14T06:29:39.866 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:39.866 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:29:39.866 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:39.866 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:29:39.866 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.729627+0000", 2024-09-14T06:29:39.866 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:39.866 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:39.866 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:39.866 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:29:39.867 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.729627+0000", 2024-09-14T06:29:39.867 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:39.867 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:39.867 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:39.867 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:29:39.867 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.729628+0000", 2024-09-14T06:29:39.867 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.3200000000000001e-07 2024-09-14T06:29:39.867 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:39.867 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:39.867 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:29:39.867 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.729629+0000", 2024-09-14T06:29:39.867 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.0389999999999999e-06 2024-09-14T06:29:39.867 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:39.868 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:39.868 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:29:39.868 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.729632+0000", 2024-09-14T06:29:39.868 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.089e-06 2024-09-14T06:29:39.868 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:39.868 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:39.868 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:29:39.868 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.729634+0000", 2024-09-14T06:29:39.868 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.48e-06 2024-09-14T06:29:39.868 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:39.868 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:39.868 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:29:39.868 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.779198+0000", 2024-09-14T06:29:39.868 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.049564201000000002 2024-09-14T06:29:39.869 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:39.869 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:39.869 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:29:39.869 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.779234+0000", 2024-09-14T06:29:39.869 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.5618999999999999e-05 2024-09-14T06:29:39.869 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:39.869 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:39.869 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:29:39.869 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.779288+0000", 2024-09-14T06:29:39.869 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.4330000000000003e-05 2024-09-14T06:29:39.869 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:39.869 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:39.869 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:39.869 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:39.870 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:39.870 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:12060 3.12 3:4a837b73:::rbd_data.112d92027590.0000000000000000:f [read 459776~1024 [fadvise_nocache]] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:29:39.870 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:25.729637+0000", 2024-09-14T06:29:39.870 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 14.073940089000001, 2024-09-14T06:29:39.870 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.049769278, 2024-09-14T06:29:39.870 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:29:39.870 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:29:39.870 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:29:39.870 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:29:39.870 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:29:39.870 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:39.870 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 12060 2024-09-14T06:29:39.870 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:39.871 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:29:39.871 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:39.871 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:29:39.871 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.729637+0000", 2024-09-14T06:29:39.871 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:39.871 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:39.871 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:39.871 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:29:39.871 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.729637+0000", 2024-09-14T06:29:39.871 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:39.871 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:39.871 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:39.871 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:29:39.871 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.729637+0000", 2024-09-14T06:29:39.872 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.1e-07 2024-09-14T06:29:39.872 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:39.872 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:39.872 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:29:39.872 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.729638+0000", 2024-09-14T06:29:39.872 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.4499999999999995e-07 2024-09-14T06:29:39.872 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:39.872 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:39.872 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:29:39.872 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.729641+0000", 2024-09-14T06:29:39.872 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.8499999999999998e-06 2024-09-14T06:29:39.872 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:39.872 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:39.872 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:29:39.873 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.729643+0000", 2024-09-14T06:29:39.873 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.2649999999999999e-06 2024-09-14T06:29:39.873 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:39.873 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:39.873 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:29:39.873 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.779314+0000", 2024-09-14T06:29:39.873 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.049670181000000001 2024-09-14T06:29:39.873 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:39.873 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:39.873 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:29:39.873 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.779354+0000", 2024-09-14T06:29:39.873 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.0457000000000002e-05 2024-09-14T06:29:39.874 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:39.874 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:39.874 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:29:39.874 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.779406+0000", 2024-09-14T06:29:39.874 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.2370000000000002e-05 2024-09-14T06:29:39.874 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:39.874 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:39.874 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:39.874 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:39.875 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:39.875 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:12061 3.12 3:4a837b73:::rbd_data.112d92027590.0000000000000000:f [read 470016~1024 [fadvise_nocache]] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:29:39.875 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:25.729646+0000", 2024-09-14T06:29:39.875 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 14.073930957, 2024-09-14T06:29:39.875 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.049877289999999998, 2024-09-14T06:29:39.875 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:29:39.875 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:29:39.875 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:29:39.875 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:29:39.875 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:29:39.875 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:39.876 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 12061 2024-09-14T06:29:39.876 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:39.876 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:29:39.876 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:39.876 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:29:39.876 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.729646+0000", 2024-09-14T06:29:39.876 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:39.876 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:39.876 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:39.876 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:29:39.876 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.729646+0000", 2024-09-14T06:29:39.877 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:39.877 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:39.877 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:39.877 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:29:39.877 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.734882+0000", 2024-09-14T06:29:39.877 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0052353149999999999 2024-09-14T06:29:39.877 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:39.877 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:39.877 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:29:39.877 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.734889+0000", 2024-09-14T06:29:39.877 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.2339999999999997e-06 2024-09-14T06:29:39.877 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:39.877 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:39.877 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:29:39.877 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.734902+0000", 2024-09-14T06:29:39.878 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.3215e-05 2024-09-14T06:29:39.878 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:39.878 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:39.878 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:29:39.878 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.734911+0000", 2024-09-14T06:29:39.878 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.2539999999999998e-06 2024-09-14T06:29:39.878 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:39.878 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:39.878 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:29:39.878 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.779430+0000", 2024-09-14T06:29:39.878 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.044518886000000001 2024-09-14T06:29:39.878 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:39.878 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:39.878 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:29:39.879 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.779472+0000", 2024-09-14T06:29:39.879 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.1365000000000001e-05 2024-09-14T06:29:39.879 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:39.879 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:39.879 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:29:39.879 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.779524+0000", 2024-09-14T06:29:39.879 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.2021000000000002e-05 2024-09-14T06:29:39.879 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:39.879 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:39.879 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:39.879 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:39.879 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:39.879 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:16693 3.1f 3:fa270572:::rbd_data.112d92027590.000000000000000e:f [read 619520~1024 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:29:39.879 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:26.209219+0000", 2024-09-14T06:29:39.880 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 13.594357882000001, 2024-09-14T06:29:39.880 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.049691935, 2024-09-14T06:29:39.880 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:29:39.880 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:29:39.880 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:29:39.880 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:29:39.880 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:29:39.880 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:39.880 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 16693 2024-09-14T06:29:39.880 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:39.880 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:29:39.880 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:39.880 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:29:39.880 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.209219+0000", 2024-09-14T06:29:39.881 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:39.881 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:39.881 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:39.881 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:29:39.881 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.209219+0000", 2024-09-14T06:29:39.881 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:39.881 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:39.881 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:39.881 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:29:39.881 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.209220+0000", 2024-09-14T06:29:39.881 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.3599999999999999e-07 2024-09-14T06:29:39.881 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:39.881 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:39.881 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:29:39.882 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.209222+0000", 2024-09-14T06:29:39.882 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.012e-06 2024-09-14T06:29:39.882 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:39.882 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:39.882 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:29:39.882 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.209229+0000", 2024-09-14T06:29:39.882 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.5050000000000001e-06 2024-09-14T06:29:39.882 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:39.882 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:39.882 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:29:39.882 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.209250+0000", 2024-09-14T06:29:39.882 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.0262999999999999e-05 2024-09-14T06:29:39.882 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:39.882 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:39.883 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:29:39.883 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.258657+0000", 2024-09-14T06:29:39.883 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.049407581999999999 2024-09-14T06:29:39.883 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:39.883 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:39.883 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:29:39.883 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.258737+0000", 2024-09-14T06:29:39.883 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.9697999999999995e-05 2024-09-14T06:29:39.883 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:39.883 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:39.883 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:29:39.883 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.258911+0000", 2024-09-14T06:29:39.883 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000174539 2024-09-14T06:29:39.883 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:39.884 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:39.884 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:39.884 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:39.884 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:39.884 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:16694 3.1f 3:fa270572:::rbd_data.112d92027590.000000000000000e:f [read 629760~1024 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:29:39.884 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:26.209260+0000", 2024-09-14T06:29:39.884 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 13.594317527999999, 2024-09-14T06:29:39.884 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.049895688000000001, 2024-09-14T06:29:39.884 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:29:39.884 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:29:39.884 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:29:39.884 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:29:39.884 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:29:39.885 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:39.885 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 16694 2024-09-14T06:29:39.885 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:39.885 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:29:39.885 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:39.885 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:29:39.885 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.209260+0000", 2024-09-14T06:29:39.885 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:39.885 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:39.885 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:39.885 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:29:39.885 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.209260+0000", 2024-09-14T06:29:39.885 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:39.885 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:39.886 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:39.886 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:29:39.886 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.209260+0000", 2024-09-14T06:29:39.886 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.1899999999999998e-07 2024-09-14T06:29:39.886 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:39.886 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:39.886 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:29:39.886 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.209275+0000", 2024-09-14T06:29:39.886 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.501e-05 2024-09-14T06:29:39.886 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:39.886 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:39.887 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:29:39.887 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.209284+0000", 2024-09-14T06:29:39.887 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.7309999999999999e-06 2024-09-14T06:29:39.887 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:39.887 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:39.887 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:29:39.887 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.209290+0000", 2024-09-14T06:29:39.887 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.8810000000000001e-06 2024-09-14T06:29:39.887 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:39.887 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:39.887 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:29:39.887 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.258952+0000", 2024-09-14T06:29:39.887 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.049661702000000002 2024-09-14T06:29:39.888 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:39.888 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:39.888 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:29:39.888 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.258999+0000", 2024-09-14T06:29:39.888 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.7014000000000001e-05 2024-09-14T06:29:39.888 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:39.888 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:39.888 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:29:39.888 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.259155+0000", 2024-09-14T06:29:39.888 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00015693099999999999 2024-09-14T06:29:39.888 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:39.888 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:39.889 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:39.889 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:39.889 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:39.889 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:16695 3.1f 3:fa270572:::rbd_data.112d92027590.000000000000000e:f [read 640000~1024 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:29:39.889 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:26.209297+0000", 2024-09-14T06:29:39.889 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 13.594279919, 2024-09-14T06:29:39.889 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.050124426999999999, 2024-09-14T06:29:39.889 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:29:39.889 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:29:39.889 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:29:39.889 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:29:39.889 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:29:39.890 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:39.890 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 16695 2024-09-14T06:29:39.890 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:39.890 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:29:39.890 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:39.890 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:29:39.890 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.209297+0000", 2024-09-14T06:29:39.890 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:39.890 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:39.890 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:39.890 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:29:39.890 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.209297+0000", 2024-09-14T06:29:39.890 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:39.890 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:39.891 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:39.891 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:29:39.891 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.209298+0000", 2024-09-14T06:29:39.891 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.1399999999999997e-07 2024-09-14T06:29:39.891 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:39.891 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:39.891 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:29:39.891 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.209300+0000", 2024-09-14T06:29:39.891 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.041e-06 2024-09-14T06:29:39.891 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:39.891 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:39.891 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:29:39.891 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.209308+0000", 2024-09-14T06:29:39.891 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.9570000000000002e-06 2024-09-14T06:29:39.891 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:39.892 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:39.892 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:29:39.892 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.209313+0000", 2024-09-14T06:29:39.892 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.2190000000000003e-06 2024-09-14T06:29:39.892 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:39.892 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:39.892 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:29:39.892 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.259196+0000", 2024-09-14T06:29:39.892 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.049882889 2024-09-14T06:29:39.892 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:39.892 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:39.892 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:29:39.892 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.259274+0000", 2024-09-14T06:29:39.893 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.8516000000000005e-05 2024-09-14T06:29:39.893 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:39.893 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:39.893 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:29:39.893 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.259422+0000", 2024-09-14T06:29:39.893 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000147391 2024-09-14T06:29:39.893 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:39.893 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:39.893 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:39.893 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:39.893 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:39.893 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:16696 3.1f 3:fa270572:::rbd_data.112d92027590.000000000000000e:f [read 650240~1024 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:29:39.893 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:26.209320+0000", 2024-09-14T06:29:39.893 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 13.59425774, 2024-09-14T06:29:39.894 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.050363212999999997, 2024-09-14T06:29:39.894 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:29:39.894 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:29:39.894 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:29:39.894 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:29:39.894 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:29:39.894 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:39.894 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 16696 2024-09-14T06:29:39.894 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:39.894 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:29:39.894 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:39.894 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:29:39.894 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.209320+0000", 2024-09-14T06:29:39.895 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:39.895 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:39.895 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:39.895 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:29:39.895 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.209320+0000", 2024-09-14T06:29:39.895 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:39.895 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:39.895 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:39.895 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:29:39.895 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.209320+0000", 2024-09-14T06:29:39.895 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.8299999999999998e-07 2024-09-14T06:29:39.895 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:39.895 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:39.896 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:29:39.896 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.209322+0000", 2024-09-14T06:29:39.896 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.809e-06 2024-09-14T06:29:39.896 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:39.896 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:39.896 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:29:39.896 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.209329+0000", 2024-09-14T06:29:39.896 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.5809999999999999e-06 2024-09-14T06:29:39.896 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:39.896 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:39.896 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:29:39.896 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.209334+0000", 2024-09-14T06:29:39.896 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.1919999999999996e-06 2024-09-14T06:29:39.897 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:39.897 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:39.897 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:29:39.897 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.259452+0000", 2024-09-14T06:29:39.897 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.050117406000000003 2024-09-14T06:29:39.897 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:39.897 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:39.897 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:29:39.897 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.259527+0000", 2024-09-14T06:29:39.897 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.5260999999999997e-05 2024-09-14T06:29:39.897 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:39.897 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:39.897 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:29:39.897 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.259683+0000", 2024-09-14T06:29:39.897 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00015568099999999999 2024-09-14T06:29:39.898 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:39.898 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:39.898 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:39.898 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:39.898 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:39.898 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:16708 3.1f 3:fa270572:::rbd_data.112d92027590.000000000000000e:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:29:39.898 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:26.209635+0000", 2024-09-14T06:29:39.898 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 13.593942349000001, 2024-09-14T06:29:39.898 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.050216744000000001, 2024-09-14T06:29:39.898 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:29:39.898 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:29:39.898 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:29:39.898 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:29:39.898 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:29:39.899 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:39.899 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 16708 2024-09-14T06:29:39.899 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:39.899 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:29:39.899 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:39.899 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:29:39.899 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.209635+0000", 2024-09-14T06:29:39.899 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:39.899 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:39.900 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:39.900 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:29:39.900 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.209635+0000", 2024-09-14T06:29:39.900 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:39.900 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:39.900 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:39.900 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:29:39.900 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.209635+0000", 2024-09-14T06:29:39.900 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.9699999999999996e-07 2024-09-14T06:29:39.900 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:39.900 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:39.900 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:29:39.900 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.209638+0000", 2024-09-14T06:29:39.901 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.2699999999999999e-06 2024-09-14T06:29:39.901 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:39.901 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:39.901 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:29:39.901 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.209644+0000", 2024-09-14T06:29:39.901 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.2570000000000004e-06 2024-09-14T06:29:39.901 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:39.901 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:39.901 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:29:39.901 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.209649+0000", 2024-09-14T06:29:39.901 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.1850000000000003e-06 2024-09-14T06:29:39.901 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:39.901 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:39.901 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:29:39.902 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.259725+0000", 2024-09-14T06:29:39.902 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.050075762000000003 2024-09-14T06:29:39.902 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:39.902 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:39.902 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:29:39.902 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.259770+0000", 2024-09-14T06:29:39.902 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.5444000000000001e-05 2024-09-14T06:29:39.902 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:39.902 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:39.902 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:29:39.902 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.259852+0000", 2024-09-14T06:29:39.902 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.1328999999999993e-05 2024-09-14T06:29:39.902 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:39.903 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:39.903 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:39.903 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:39.903 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:39.903 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-14T06:29:39.904 INFO:teuthology.orchestra.run.smithi094.stdout:888 read 0xa891483 thru 0xa8968b5 (0x5433 bytes) 2024-09-14T06:29:39.905 INFO:teuthology.orchestra.run.smithi094.stdout:889 read 0xb47f3de thru 0xb47ff69 (0xb8c bytes) 2024-09-14T06:29:39.905 INFO:teuthology.orchestra.run.smithi094.stdout:891 writesame 0xda95bcd thru 0xda95f3f data_size 0x1(0x373 bytes) 2024-09-14T06:29:39.905 INFO:teuthology.orchestra.run.smithi094.stdout:892 trunc from 0xe8628bc to 0xd2537b2 2024-09-14T06:29:39.906 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-14T06:29:39.909 INFO:teuthology.orchestra.run.smithi094.stdout:893 write 0x9c2508c thru 0x9c30b33 (0xbaa8 bytes) 2024-09-14T06:29:39.909 INFO:teuthology.orchestra.run.smithi094.stdout:894 trunc from 0xd2537b2 to 0x93e86e6 2024-09-14T06:29:39.909 INFO:teuthology.orchestra.run.smithi094.stdout:895 writesame 0xcd90144 thru 0xcd92f2e data_size 0x1(0x2deb bytes) 2024-09-14T06:29:39.909 INFO:teuthology.orchestra.run.smithi094.stdout:897 writesame 0xe9810f9 thru 0xe9814d2 data_size 0x11(0x3da bytes) 2024-09-14T06:29:39.909 INFO:teuthology.orchestra.run.smithi094.stdout:898 punch from 0x367b36b to 0x367ed6b, (0x3a00 bytes) 2024-09-14T06:29:39.909 INFO:teuthology.orchestra.run.smithi094.stdout:899 trunc from 0xe9814d3 to 0xd457330 2024-09-14T06:29:39.909 INFO:teuthology.orchestra.run.smithi094.stdout:901 read 0x3b9509d thru 0x3b95744 (0x6a8 bytes) 2024-09-14T06:29:39.909 INFO:teuthology.orchestra.run.smithi094.stdout:902 punch from 0xb058cf7 to 0xb06160f, (0x8918 bytes) 2024-09-14T06:29:39.910 INFO:teuthology.orchestra.run.smithi094.stdout:903 trunc from 0xd457330 to 0xa637597 2024-09-14T06:29:39.910 INFO:teuthology.orchestra.run.smithi094.stdout:904 write 0x946e0f9 thru 0x94776d8 (0x95e0 bytes) 2024-09-14T06:29:39.910 INFO:teuthology.orchestra.run.smithi094.stdout:905 read 0x8e34428 thru 0x8e3b2ad (0x6e86 bytes) 2024-09-14T06:29:39.910 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-14T06:29:39.910 INFO:teuthology.orchestra.run.smithi094.stdout:907 write 0x4a34cc3 thru 0x4a44502 (0xf840 bytes) 2024-09-14T06:29:39.910 INFO:teuthology.orchestra.run.smithi094.stdout:909 read 0x8bb5215 thru 0x8bb74df (0x22cb bytes) 2024-09-14T06:29:39.910 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-14T06:29:39.910 INFO:teuthology.orchestra.run.smithi094.stdout:911 punch from 0x9c44411 to 0x9c4f4ba, (0xb0a9 bytes) 2024-09-14T06:29:39.910 INFO:teuthology.orchestra.run.smithi094.stdout:913 write 0xee30b1c thru 0xee347a1 (0x3c86 bytes) 2024-09-14T06:29:39.910 INFO:teuthology.orchestra.run.smithi094.stdout:916 write 0x5b6966c thru 0x5b790be (0xfa53 bytes) 2024-09-14T06:29:39.910 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-14T06:29:39.910 INFO:teuthology.orchestra.run.smithi094.stdout:918 trunc from 0xee347a2 to 0x992cf51 2024-09-14T06:29:39.910 INFO:tasks.ceph.osd.1.smithi094.stderr:2024-09-14T06:29:39.903+0000 7f328e47a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33641) UID: 0 2024-09-14T06:29:39.918 INFO:teuthology.orchestra.run.smithi094.stdout:920 write 0x9f3227d thru 0x9f3e4eb (0xc26f bytes) 2024-09-14T06:29:39.920 INFO:teuthology.orchestra.run.smithi094.stdout:922 read 0x14e84df thru 0x14f2055 (0x9b77 bytes) 2024-09-14T06:29:39.925 INFO:teuthology.orchestra.run.smithi094.stdout:924 write 0xa26b582 thru 0xa27988a (0xe309 bytes) 2024-09-14T06:29:39.929 INFO:teuthology.orchestra.run.smithi094.stdout:925 write 0x77948dd thru 0x779847f (0x3ba3 bytes) 2024-09-14T06:29:39.932 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-14T06:29:39.940 INFO:teuthology.orchestra.run.smithi094.stdout:927 write 0xd1e2b0d thru 0xd1e8545 (0x5a39 bytes) 2024-09-14T06:29:39.943 INFO:teuthology.orchestra.run.smithi094.stdout:928 read 0x4cbe75e thru 0x4cc4d62 (0x6605 bytes) 2024-09-14T06:29:39.944 INFO:teuthology.orchestra.run.smithi094.stdout:930 write 0x5efaf74 thru 0x5efc315 (0x13a2 bytes) 2024-09-14T06:29:39.946 INFO:teuthology.orchestra.run.smithi094.stdout:932 write 0xb2f462f thru 0xb2f8a8f (0x4461 bytes) 2024-09-14T06:29:39.949 INFO:teuthology.orchestra.run.smithi094.stdout:933 punch from 0x26cff62 to 0x26d34ad, (0x354b bytes) 2024-09-14T06:29:39.949 INFO:teuthology.orchestra.run.smithi094.stdout:935 write 0x1c67584 thru 0x1c6eeb6 (0x7933 bytes) 2024-09-14T06:29:39.950 INFO:teuthology.orchestra.run.smithi094.stdout:937 read 0x8fa6a0 thru 0x902aad (0x840e bytes) 2024-09-14T06:29:39.952 INFO:teuthology.orchestra.run.smithi094.stdout:938 trunc from 0xd1e8546 to 0xe31700f 2024-09-14T06:29:39.959 INFO:teuthology.orchestra.run.smithi094.stdout:942 trunc from 0xe31700f to 0xc1bfd95 2024-09-14T06:29:39.969 INFO:teuthology.orchestra.run.smithi094.stdout:943 write 0x2043e41 thru 0x2051a00 (0xdbc0 bytes) 2024-09-14T06:29:39.970 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-14T06:29:39.970 INFO:teuthology.orchestra.run.smithi094.stdout:946 read 0x3a14c91 thru 0x3a166ab (0x1a1b bytes) 2024-09-14T06:29:39.971 INFO:teuthology.orchestra.run.smithi094.stdout:947 punch from 0x8d856a8 to 0x8d8a783, (0x50db bytes) 2024-09-14T06:29:39.971 INFO:teuthology.orchestra.run.smithi094.stdout:948 read 0x3a3bde thru 0x3ac2e4 (0x8707 bytes) 2024-09-14T06:29:39.980 INFO:teuthology.orchestra.run.smithi094.stdout:949 trunc from 0xc1bfd95 to 0x4ce5089 2024-09-14T06:29:39.994 INFO:teuthology.orchestra.run.smithi094.stdout:950 trunc from 0x4ce5089 to 0x6adc8c8 2024-09-14T06:29:39.999 INFO:teuthology.orchestra.run.smithi094.stdout:951 write 0xa357a8 thru 0xa3a7ce (0x5027 bytes) 2024-09-14T06:29:40.000 INFO:teuthology.orchestra.run.smithi094.stdout:952 write 0x4d53846 thru 0x4d5de4a (0xa605 bytes) 2024-09-14T06:29:40.002 INFO:teuthology.orchestra.run.smithi094.stdout:953 writesame 0x3c5da1a thru 0x3c64cca data_size 0x1(0x72b1 bytes) 2024-09-14T06:29:40.005 INFO:tasks.ceph.osd.3.smithi094.stderr:2024-09-14T06:29:40.004+0000 7feeeafe8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33655) UID: 0 2024-09-14T06:29:40.017 INFO:teuthology.orchestra.run.smithi094.stdout:955 write 0x40d749c thru 0x40d93d6 (0x1f3b bytes) 2024-09-14T06:29:40.025 INFO:teuthology.orchestra.run.smithi094.stdout:958 trunc from 0x6adc8c8 to 0xadc9eaa 2024-09-14T06:29:40.030 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-14T06:29:40.030 INFO:teuthology.orchestra.run.smithi094.stdout:960 write 0x5daa648 thru 0x5db9eb8 (0xf871 bytes) 2024-09-14T06:29:40.030 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-14T06:29:40.031 INFO:teuthology.orchestra.run.smithi094.stdout: "ops": [], 2024-09-14T06:29:40.031 INFO:teuthology.orchestra.run.smithi094.stdout: "num_ops": 0 2024-09-14T06:29:40.031 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-14T06:29:40.034 INFO:teuthology.orchestra.run.smithi094.stdout:961 trunc from 0xadc9eaa to 0xa640c40 2024-09-14T06:29:40.040 INFO:teuthology.orchestra.run.smithi094.stdout:964 writesame 0x8e820b7 thru 0x8e82568 data_size 0x1(0x4b2 bytes) 2024-09-14T06:29:40.041 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-14T06:29:40.043 INFO:teuthology.orchestra.run.smithi094.stdout:966 read 0x4ec8ea4 thru 0x4ed518e (0xc2eb bytes) 2024-09-14T06:29:40.043 INFO:teuthology.orchestra.run.smithi094.stdout:967 read 0x7a5b76b thru 0x7a60012 (0x48a8 bytes) 2024-09-14T06:29:40.052 INFO:teuthology.orchestra.run.smithi094.stdout:968 write 0xe2635ab thru 0xe2649ec (0x1442 bytes) 2024-09-14T06:29:40.071 INFO:teuthology.orchestra.run.smithi094.stdout:969 trunc from 0xe2649ed to 0x47f2db 2024-09-14T06:29:40.106 INFO:tasks.ceph.osd.0.smithi094.stderr:2024-09-14T06:29:40.105+0000 7f07c9333640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33636) UID: 0 2024-09-14T06:29:40.113 INFO:teuthology.orchestra.run.smithi094.stdout:970 writesame 0xe3482bf thru 0xe350415 data_size 0x27(0x8157 bytes) 2024-09-14T06:29:40.113 INFO:teuthology.orchestra.run.smithi094.stdout:971 read 0x7e4438a thru 0x7e51496 (0xd10d bytes) 2024-09-14T06:29:40.113 INFO:teuthology.orchestra.run.smithi094.stdout:973 write 0x1e7d667 thru 0x1e7ef0f (0x18a9 bytes) 2024-09-14T06:29:40.115 INFO:teuthology.orchestra.run.smithi094.stdout:976 read 0x91787e7 thru 0x9183f31 (0xb74b bytes) 2024-09-14T06:29:40.115 INFO:teuthology.orchestra.run.smithi094.stdout:977 writesame 0x993bae4 thru 0x993d7c2 data_size 0x1(0x1cdf bytes) 2024-09-14T06:29:40.120 INFO:teuthology.orchestra.run.smithi094.stdout:978 write 0x4b7e0d1 thru 0x4b88842 (0xa772 bytes) 2024-09-14T06:29:40.124 INFO:teuthology.orchestra.run.smithi094.stdout:979 read 0x118bebc thru 0x11959e3 (0x9b28 bytes) 2024-09-14T06:29:40.124 INFO:teuthology.orchestra.run.smithi094.stdout:980 read 0x42d322d thru 0x42e16d9 (0xe4ad bytes) 2024-09-14T06:29:40.124 INFO:teuthology.orchestra.run.smithi094.stdout:982 write 0xb5a74d3 thru 0xb5af174 (0x7ca2 bytes) 2024-09-14T06:29:40.127 INFO:teuthology.orchestra.run.smithi094.stdout:983 read 0x35f9279 thru 0x3605b0c (0xc894 bytes) 2024-09-14T06:29:40.128 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-14T06:29:40.128 INFO:teuthology.orchestra.run.smithi094.stdout:985 writesame 0x9b06de5 thru 0x9b0f275 data_size 0x1(0x8491 bytes) 2024-09-14T06:29:40.146 INFO:teuthology.orchestra.run.smithi094.stdout:986 write 0x1de77a thru 0x1e3e05 (0x568c bytes) 2024-09-14T06:29:40.147 INFO:teuthology.orchestra.run.smithi094.stdout:987 write 0x2496245 thru 0x249c7ee (0x65aa bytes) 2024-09-14T06:29:40.149 INFO:teuthology.orchestra.run.smithi094.stdout:988 write 0x4f78b7b thru 0x4f7cf9a (0x4420 bytes) 2024-09-14T06:29:40.152 INFO:teuthology.orchestra.run.smithi094.stdout:989 read 0x893c5be thru 0x8944129 (0x7b6c bytes) 2024-09-14T06:29:40.152 INFO:teuthology.orchestra.run.smithi094.stdout:992 read 0x634ef0f thru 0x63594ab (0xa59d bytes) 2024-09-14T06:29:40.152 INFO:teuthology.orchestra.run.smithi094.stdout:truncating to largest ever: 0xedde2d8 2024-09-14T06:29:40.153 INFO:teuthology.orchestra.run.smithi094.stdout:993 trunc from 0xe350416 to 0xedde2d8 2024-09-14T06:29:40.159 INFO:teuthology.orchestra.run.smithi094.stdout:994 punch from 0x10ec5be to 0x10f3a93, (0x74d5 bytes) 2024-09-14T06:29:40.159 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-14T06:29:40.171 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-14T06:29:40.172 INFO:teuthology.orchestra.run.smithi094.stdout: "ops": [], 2024-09-14T06:29:40.172 INFO:teuthology.orchestra.run.smithi094.stdout: "complaint_time": 30, 2024-09-14T06:29:40.172 INFO:teuthology.orchestra.run.smithi094.stdout: "num_blocked_ops": 0 2024-09-14T06:29:40.172 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-14T06:29:40.173 INFO:teuthology.orchestra.run.smithi094.stdout:996 trunc from 0xedde2d8 to 0x42dbce5 2024-09-14T06:29:40.183 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-14T06:29:40.193 INFO:teuthology.orchestra.run.smithi094.stdout:997 write 0x6b01122 thru 0x6b109c0 (0xf89f bytes) 2024-09-14T06:29:40.203 INFO:teuthology.orchestra.run.smithi094.stdout:998 writesame 0x82e86e1 thru 0x82f5d1e data_size 0x63(0xd63e bytes) 2024-09-14T06:29:40.203 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-14T06:29:40.203 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-14T06:29:40.203 INFO:teuthology.orchestra.run.smithi094.stdout:1001 read 0x6b7d20b thru 0x6b8a0d2 (0xcec8 bytes) 2024-09-14T06:29:40.204 INFO:teuthology.orchestra.run.smithi094.stdout:1002 trunc from 0x82f5d1f to 0x91e0f10 2024-09-14T06:29:40.206 INFO:tasks.ceph.osd.0.smithi094.stderr:2024-09-14T06:29:40.205+0000 7f07c9333640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33636) UID: 0 2024-09-14T06:29:40.210 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-14T06:29:40.210 INFO:teuthology.orchestra.run.smithi094.stdout:1004 read 0x873843b thru 0x874073c (0x8302 bytes) 2024-09-14T06:29:40.210 INFO:teuthology.orchestra.run.smithi094.stdout:1005 punch from 0x64d01ce to 0x64d4669, (0x449b bytes) 2024-09-14T06:29:40.210 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-14T06:29:40.218 INFO:teuthology.orchestra.run.smithi094.stdout:1007 trunc from 0x91e0f10 to 0x26f2376 2024-09-14T06:29:40.228 INFO:teuthology.orchestra.run.smithi094.stdout:1009 punch from 0x1ff9a61 to 0x1ffc156, (0x26f5 bytes) 2024-09-14T06:29:40.238 INFO:teuthology.orchestra.run.smithi094.stdout:1010 write 0x5dc7b4c thru 0x5dd2a49 (0xaefe bytes) 2024-09-14T06:29:40.246 INFO:teuthology.orchestra.run.smithi094.stdout:1011 trunc from 0x5dd2a4a to 0x15972b2 2024-09-14T06:29:40.276 INFO:teuthology.orchestra.run.smithi094.stdout:1013 writesame 0xbcac7cb thru 0xbcb9529 data_size 0x19(0xcd5f bytes) 2024-09-14T06:29:40.276 INFO:teuthology.orchestra.run.smithi094.stdout:1015 read 0xf6e08a thru 0xf78185 (0xa0fc bytes) 2024-09-14T06:29:40.276 INFO:teuthology.orchestra.run.smithi094.stdout:1016 write 0xb60be7f thru 0xb60c109 (0x28b bytes) 2024-09-14T06:29:40.279 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-14T06:29:40.279 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-14T06:29:40.279 INFO:teuthology.orchestra.run.smithi094.stdout:1019 read 0x80ef03a thru 0x80fafec (0xbfb3 bytes) 2024-09-14T06:29:40.279 INFO:teuthology.orchestra.run.smithi094.stdout:1020 write 0x5c63f38 thru 0x5c6bd63 (0x7e2c bytes) 2024-09-14T06:29:40.282 INFO:teuthology.orchestra.run.smithi094.stdout:1021 read 0x65434d7 thru 0x6544c55 (0x177f bytes) 2024-09-14T06:29:40.282 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-14T06:29:40.282 INFO:teuthology.orchestra.run.smithi094.stdout:1025 write 0x282de51 thru 0x282ed9b (0xf4b bytes) 2024-09-14T06:29:40.285 INFO:teuthology.orchestra.run.smithi094.stdout:1026 read 0xa778363 thru 0xa787740 (0xf3de bytes) 2024-09-14T06:29:40.285 INFO:teuthology.orchestra.run.smithi094.stdout:1027 punch from 0x51c6eec to 0x51d51cb, (0xe2df bytes) 2024-09-14T06:29:40.285 INFO:teuthology.orchestra.run.smithi094.stdout:1028 write 0x6da1a92 thru 0x6dadd13 (0xc282 bytes) 2024-09-14T06:29:40.288 INFO:teuthology.orchestra.run.smithi094.stdout:1029 read 0x2d7cbf3 thru 0x2d8126b (0x4679 bytes) 2024-09-14T06:29:40.298 INFO:teuthology.orchestra.run.smithi094.stdout:1031 trunc from 0xbcb952a to 0x310e5ff 2024-09-14T06:29:40.307 INFO:tasks.ceph.osd.0.smithi094.stderr:2024-09-14T06:29:40.306+0000 7f07c9333640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33636) UID: 0 2024-09-14T06:29:40.308 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-14T06:29:40.308 INFO:teuthology.orchestra.run.smithi094.stdout: "size": 20, 2024-09-14T06:29:40.309 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 600, 2024-09-14T06:29:40.309 INFO:teuthology.orchestra.run.smithi094.stdout: "ops": [ 2024-09-14T06:29:40.309 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:40.309 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:14910 3.0 3:05c39535:::rbd_data.112d92027590.0000000000000009:f [read 713728~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:29:40.309 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:26.083696+0000", 2024-09-14T06:29:40.309 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 14.223285304999999, 2024-09-14T06:29:40.309 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.040571832000000002, 2024-09-14T06:29:40.309 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:29:40.309 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:29:40.309 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:29:40.309 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:29:40.309 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:29:40.309 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:40.309 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 14910 2024-09-14T06:29:40.310 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:40.310 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:29:40.310 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:40.310 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:29:40.310 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.083696+0000", 2024-09-14T06:29:40.310 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:40.310 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:40.310 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:40.310 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:29:40.310 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.083696+0000", 2024-09-14T06:29:40.310 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:40.310 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:40.310 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:40.310 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:29:40.311 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.083696+0000", 2024-09-14T06:29:40.311 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.0699999999999997e-07 2024-09-14T06:29:40.311 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:40.311 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:40.311 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:29:40.311 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.083698+0000", 2024-09-14T06:29:40.311 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.9429999999999999e-06 2024-09-14T06:29:40.311 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:40.311 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:40.311 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:29:40.311 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.083708+0000", 2024-09-14T06:29:40.311 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.6430000000000007e-06 2024-09-14T06:29:40.311 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:40.311 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:40.311 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:29:40.312 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.083719+0000", 2024-09-14T06:29:40.312 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.1017e-05 2024-09-14T06:29:40.312 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:40.312 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:40.312 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:29:40.312 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.124152+0000", 2024-09-14T06:29:40.312 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.040433104999999997 2024-09-14T06:29:40.312 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:40.312 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:40.312 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:29:40.312 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.124194+0000", 2024-09-14T06:29:40.312 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.1585999999999998e-05 2024-09-14T06:29:40.313 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:40.313 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:40.313 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:29:40.313 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.124268+0000", 2024-09-14T06:29:40.313 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.4030999999999996e-05 2024-09-14T06:29:40.313 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:40.313 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:40.313 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:40.313 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:40.313 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:40.313 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:14911 3.0 3:05c39535:::rbd_data.112d92027590.0000000000000009:f [read 723968~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:29:40.313 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:26.083729+0000", 2024-09-14T06:29:40.313 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 14.223252198000001, 2024-09-14T06:29:40.313 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.040746899000000003, 2024-09-14T06:29:40.314 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:29:40.314 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:29:40.314 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:29:40.314 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:29:40.314 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:29:40.314 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:40.314 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 14911 2024-09-14T06:29:40.314 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:40.314 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:29:40.314 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:40.314 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:29:40.314 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.083729+0000", 2024-09-14T06:29:40.315 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:40.315 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:40.315 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:40.315 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:29:40.315 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.083729+0000", 2024-09-14T06:29:40.315 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:40.315 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:40.315 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:40.315 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:29:40.315 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.083729+0000", 2024-09-14T06:29:40.316 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.7399999999999998e-07 2024-09-14T06:29:40.316 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:40.316 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:40.316 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:29:40.316 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.083731+0000", 2024-09-14T06:29:40.316 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.9709999999999998e-06 2024-09-14T06:29:40.316 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:40.316 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:40.316 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:29:40.316 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.083737+0000", 2024-09-14T06:29:40.316 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.801e-06 2024-09-14T06:29:40.316 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:40.316 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:40.316 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:29:40.317 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.083743+0000", 2024-09-14T06:29:40.317 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.9529999999999996e-06 2024-09-14T06:29:40.317 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:40.317 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:40.317 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:29:40.317 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.124306+0000", 2024-09-14T06:29:40.317 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.040562333999999999 2024-09-14T06:29:40.318 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:40.318 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:40.318 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:29:40.318 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.124384+0000", 2024-09-14T06:29:40.318 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.8110999999999998e-05 2024-09-14T06:29:40.318 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:40.319 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:40.319 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:29:40.319 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.124476+0000", 2024-09-14T06:29:40.319 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.2255000000000004e-05 2024-09-14T06:29:40.319 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:40.319 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:40.320 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:40.320 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:40.320 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:40.320 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:14912 3.0 3:05c39535:::rbd_data.112d92027590.0000000000000009:f [read 734208~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:29:40.320 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:26.083749+0000", 2024-09-14T06:29:40.320 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 14.223232060999999, 2024-09-14T06:29:40.320 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.040874964, 2024-09-14T06:29:40.320 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:29:40.320 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:29:40.320 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:29:40.321 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:29:40.321 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:29:40.321 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:40.321 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 14912 2024-09-14T06:29:40.321 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:40.321 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:29:40.321 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:40.321 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:29:40.322 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.083749+0000", 2024-09-14T06:29:40.322 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:40.322 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:40.322 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:40.322 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:29:40.322 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.083749+0000", 2024-09-14T06:29:40.322 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:40.322 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:40.322 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:40.322 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:29:40.322 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.083749+0000", 2024-09-14T06:29:40.323 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.58e-07 2024-09-14T06:29:40.323 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:40.323 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:40.323 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:29:40.323 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.083751+0000", 2024-09-14T06:29:40.323 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.7910000000000001e-06 2024-09-14T06:29:40.323 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:40.323 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:40.323 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:29:40.323 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.083758+0000", 2024-09-14T06:29:40.323 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.3799999999999999e-06 2024-09-14T06:29:40.323 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:40.323 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:40.323 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:29:40.323 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.083761+0000", 2024-09-14T06:29:40.323 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.8480000000000003e-06 2024-09-14T06:29:40.324 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:40.324 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:40.324 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:29:40.324 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.124514+0000", 2024-09-14T06:29:40.324 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.040752743000000001 2024-09-14T06:29:40.324 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:40.324 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:40.324 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:29:40.324 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.124554+0000", 2024-09-14T06:29:40.324 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.0209000000000001e-05 2024-09-14T06:29:40.324 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:40.324 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:40.324 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:29:40.325 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.124624+0000", 2024-09-14T06:29:40.325 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.9635000000000007e-05 2024-09-14T06:29:40.325 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:40.325 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:40.325 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:40.325 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:40.325 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:40.325 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:14913 3.0 3:05c39535:::rbd_data.112d92027590.0000000000000009:f [read 744448~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:29:40.325 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:26.083766+0000", 2024-09-14T06:29:40.325 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 14.223214763, 2024-09-14T06:29:40.325 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.041064829999999997, 2024-09-14T06:29:40.325 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:29:40.325 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:29:40.325 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:29:40.326 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:29:40.326 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:29:40.326 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:40.326 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 14913 2024-09-14T06:29:40.326 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:40.326 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:29:40.326 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:40.326 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:29:40.326 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.083766+0000", 2024-09-14T06:29:40.326 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:40.326 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:40.326 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:40.326 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:29:40.326 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.083766+0000", 2024-09-14T06:29:40.327 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:40.327 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:40.327 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:40.327 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:29:40.327 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.083767+0000", 2024-09-14T06:29:40.327 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.1399999999999998e-07 2024-09-14T06:29:40.327 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:40.327 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:40.327 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:29:40.327 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.083768+0000", 2024-09-14T06:29:40.327 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.1629999999999999e-06 2024-09-14T06:29:40.327 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:40.327 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:40.328 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:29:40.328 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.083771+0000", 2024-09-14T06:29:40.328 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.2030000000000001e-06 2024-09-14T06:29:40.328 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:40.328 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:40.328 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:29:40.328 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.083774+0000", 2024-09-14T06:29:40.328 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.627e-06 2024-09-14T06:29:40.328 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:40.328 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:40.328 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:29:40.328 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.124662+0000", 2024-09-14T06:29:40.328 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.040887973000000001 2024-09-14T06:29:40.328 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:40.329 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:40.329 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:29:40.329 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.124741+0000", 2024-09-14T06:29:40.329 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.8888999999999996e-05 2024-09-14T06:29:40.329 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:40.329 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:40.329 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:29:40.329 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.124831+0000", 2024-09-14T06:29:40.329 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.0661000000000005e-05 2024-09-14T06:29:40.329 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:40.329 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:40.329 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:40.329 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:40.329 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:40.329 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:14914 3.0 3:05c39535:::rbd_data.112d92027590.0000000000000009:f [read 754688~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:29:40.330 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:26.083777+0000", 2024-09-14T06:29:40.330 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 14.223204242, 2024-09-14T06:29:40.330 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.041197828999999998, 2024-09-14T06:29:40.330 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:29:40.330 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:29:40.330 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:29:40.330 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:29:40.330 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:29:40.330 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:40.330 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 14914 2024-09-14T06:29:40.330 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:40.330 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:29:40.330 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:40.331 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:29:40.331 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.083777+0000", 2024-09-14T06:29:40.331 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:40.331 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:40.331 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:40.331 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:29:40.331 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.083777+0000", 2024-09-14T06:29:40.331 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:40.331 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:40.331 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:40.331 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:29:40.331 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.083777+0000", 2024-09-14T06:29:40.331 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.0200000000000001e-07 2024-09-14T06:29:40.332 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:40.332 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:40.332 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:29:40.332 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.083778+0000", 2024-09-14T06:29:40.332 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.3699999999999999e-07 2024-09-14T06:29:40.332 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:40.332 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:40.332 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:29:40.332 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.083782+0000", 2024-09-14T06:29:40.332 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.563e-06 2024-09-14T06:29:40.332 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:40.332 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:40.332 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:29:40.332 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.083784+0000", 2024-09-14T06:29:40.333 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.508e-06 2024-09-14T06:29:40.333 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:40.333 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:40.333 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:29:40.333 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.124870+0000", 2024-09-14T06:29:40.333 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.041086077999999998 2024-09-14T06:29:40.333 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:40.333 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:40.333 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:29:40.333 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.124912+0000", 2024-09-14T06:29:40.333 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.1742000000000001e-05 2024-09-14T06:29:40.333 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:40.333 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:40.333 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:29:40.334 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.124975+0000", 2024-09-14T06:29:40.334 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.2799000000000006e-05 2024-09-14T06:29:40.334 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:40.334 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:40.334 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:40.334 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:40.334 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:40.334 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:14915 3.0 3:05c39535:::rbd_data.112d92027590.0000000000000009:f [read 764928~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:29:40.334 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:26.083787+0000", 2024-09-14T06:29:40.334 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 14.223193890999999, 2024-09-14T06:29:40.334 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.041359554999999999, 2024-09-14T06:29:40.334 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:29:40.334 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:29:40.334 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:29:40.334 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:29:40.334 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:29:40.335 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:40.335 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 14915 2024-09-14T06:29:40.335 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:40.335 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:29:40.335 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:40.335 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:29:40.335 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.083787+0000", 2024-09-14T06:29:40.335 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:40.335 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:40.335 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:40.335 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:29:40.336 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.083787+0000", 2024-09-14T06:29:40.336 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:40.336 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:40.336 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:40.336 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:29:40.336 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.083788+0000", 2024-09-14T06:29:40.336 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.2499999999999999e-07 2024-09-14T06:29:40.336 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:40.336 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:40.336 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:29:40.336 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.083788+0000", 2024-09-14T06:29:40.336 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.02e-07 2024-09-14T06:29:40.337 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:40.337 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:40.337 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:29:40.337 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.083791+0000", 2024-09-14T06:29:40.337 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.8849999999999999e-06 2024-09-14T06:29:40.337 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:40.337 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:40.337 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:29:40.337 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.083794+0000", 2024-09-14T06:29:40.337 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.587e-06 2024-09-14T06:29:40.337 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:40.337 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:40.337 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:29:40.338 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.125014+0000", 2024-09-14T06:29:40.338 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.041220075000000002 2024-09-14T06:29:40.338 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:40.338 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:40.338 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:29:40.338 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.125066+0000", 2024-09-14T06:29:40.338 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.1632000000000003e-05 2024-09-14T06:29:40.338 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:40.338 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:40.338 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:29:40.338 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.125147+0000", 2024-09-14T06:29:40.338 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.1248999999999994e-05 2024-09-14T06:29:40.338 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:40.338 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:40.338 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:40.339 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:40.339 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:40.339 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:14916 3.0 3:05c39535:::rbd_data.112d92027590.0000000000000009:f [read 775168~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:29:40.339 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:26.083797+0000", 2024-09-14T06:29:40.339 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 14.223184102999999, 2024-09-14T06:29:40.339 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.041475420999999998, 2024-09-14T06:29:40.339 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:29:40.339 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:29:40.339 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:29:40.339 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:29:40.339 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:29:40.339 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:40.339 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 14916 2024-09-14T06:29:40.339 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:40.339 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:29:40.340 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:40.340 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:29:40.340 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.083797+0000", 2024-09-14T06:29:40.340 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:40.340 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:40.340 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:40.340 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:29:40.340 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.083797+0000", 2024-09-14T06:29:40.340 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:40.340 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:40.340 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:40.340 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:29:40.340 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.083797+0000", 2024-09-14T06:29:40.340 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.2100000000000001e-07 2024-09-14T06:29:40.341 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:40.341 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:40.341 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:29:40.341 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.083812+0000", 2024-09-14T06:29:40.341 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.4630000000000001e-05 2024-09-14T06:29:40.341 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:40.341 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:40.341 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:29:40.341 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.083817+0000", 2024-09-14T06:29:40.341 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.0719999999999999e-06 2024-09-14T06:29:40.341 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:40.341 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:40.341 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:29:40.342 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.083820+0000", 2024-09-14T06:29:40.342 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.8150000000000002e-06 2024-09-14T06:29:40.342 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:40.342 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:40.342 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:29:40.342 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.125180+0000", 2024-09-14T06:29:40.342 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.041360643000000002 2024-09-14T06:29:40.342 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:40.342 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:40.342 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:29:40.342 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.125221+0000", 2024-09-14T06:29:40.342 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.0442999999999999e-05 2024-09-14T06:29:40.343 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:40.343 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:40.343 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:29:40.343 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.125272+0000", 2024-09-14T06:29:40.343 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.1597000000000001e-05 2024-09-14T06:29:40.343 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:40.343 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:40.343 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:40.343 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:40.343 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:40.343 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:14917 3.0 3:05c39535:::rbd_data.112d92027590.0000000000000009:f [read 785408~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:29:40.343 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:26.083824+0000", 2024-09-14T06:29:40.343 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 14.223157262999999, 2024-09-14T06:29:40.344 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.041594226999999998, 2024-09-14T06:29:40.344 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:29:40.344 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:29:40.344 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:29:40.344 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:29:40.344 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:29:40.344 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:40.344 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 14917 2024-09-14T06:29:40.344 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:40.344 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:29:40.344 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:40.344 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:29:40.344 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.083824+0000", 2024-09-14T06:29:40.344 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:40.345 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:40.345 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:40.345 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:29:40.345 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.083824+0000", 2024-09-14T06:29:40.345 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:40.345 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:40.345 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:40.345 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:29:40.345 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.083824+0000", 2024-09-14T06:29:40.345 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.23e-07 2024-09-14T06:29:40.345 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:40.345 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:40.345 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:29:40.345 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.083825+0000", 2024-09-14T06:29:40.346 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.0690000000000001e-06 2024-09-14T06:29:40.346 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:40.346 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:40.346 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:29:40.346 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.083828+0000", 2024-09-14T06:29:40.346 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.9840000000000001e-06 2024-09-14T06:29:40.346 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:40.346 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:40.346 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:29:40.346 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.083831+0000", 2024-09-14T06:29:40.346 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.4119999999999999e-06 2024-09-14T06:29:40.347 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:40.347 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:40.347 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:29:40.347 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.125303+0000", 2024-09-14T06:29:40.347 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.041472676999999999 2024-09-14T06:29:40.347 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:40.347 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:40.347 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:29:40.347 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.125358+0000", 2024-09-14T06:29:40.347 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.4654000000000002e-05 2024-09-14T06:29:40.347 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:40.347 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:40.347 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:29:40.347 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.125418+0000", 2024-09-14T06:29:40.347 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.0207999999999997e-05 2024-09-14T06:29:40.347 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:40.348 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:40.348 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:40.348 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:40.348 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:40.348 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:14918 3.0 3:05c39535:::rbd_data.112d92027590.0000000000000009:f [read 795648~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:29:40.348 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:26.083834+0000", 2024-09-14T06:29:40.348 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 14.223147554000001, 2024-09-14T06:29:40.348 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.041776287000000002, 2024-09-14T06:29:40.348 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:29:40.348 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:29:40.348 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:29:40.348 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:29:40.348 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:29:40.349 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:40.349 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 14918 2024-09-14T06:29:40.349 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:40.349 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:29:40.349 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:40.349 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:29:40.349 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.083834+0000", 2024-09-14T06:29:40.349 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:40.349 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:40.349 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:40.349 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:29:40.349 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.083834+0000", 2024-09-14T06:29:40.349 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:40.349 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:40.349 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:40.349 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:29:40.350 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.083834+0000", 2024-09-14T06:29:40.350 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.0100000000000001e-07 2024-09-14T06:29:40.350 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:40.350 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:40.350 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:29:40.350 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.083835+0000", 2024-09-14T06:29:40.350 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.3900000000000003e-07 2024-09-14T06:29:40.350 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:40.350 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:40.350 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:29:40.350 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.083838+0000", 2024-09-14T06:29:40.350 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.9830000000000001e-06 2024-09-14T06:29:40.350 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:40.350 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:40.351 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:29:40.351 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.083840+0000", 2024-09-14T06:29:40.351 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.4679999999999999e-06 2024-09-14T06:29:40.351 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:40.351 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:40.351 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:29:40.351 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.125455+0000", 2024-09-14T06:29:40.351 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.041614578999999999 2024-09-14T06:29:40.351 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:40.351 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:40.351 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:29:40.351 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.125496+0000", 2024-09-14T06:29:40.351 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.1047000000000002e-05 2024-09-14T06:29:40.352 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:40.352 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:40.352 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:29:40.352 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.125610+0000", 2024-09-14T06:29:40.352 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00011407 2024-09-14T06:29:40.352 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:40.352 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:40.353 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:40.353 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:40.353 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:40.353 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:14919 3.0 3:05c39535:::rbd_data.112d92027590.0000000000000009:f [read 805888~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:29:40.353 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:26.083843+0000", 2024-09-14T06:29:40.353 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 14.223137806, 2024-09-14T06:29:40.353 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.041992356000000002, 2024-09-14T06:29:40.353 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:29:40.353 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:29:40.353 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:29:40.353 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:29:40.353 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:29:40.353 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:40.353 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 14919 2024-09-14T06:29:40.353 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:40.354 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:29:40.354 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:40.354 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:29:40.354 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.083843+0000", 2024-09-14T06:29:40.354 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:40.354 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:40.354 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:40.354 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:29:40.354 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.083843+0000", 2024-09-14T06:29:40.354 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:40.355 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:40.355 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:40.355 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:29:40.355 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.083844+0000", 2024-09-14T06:29:40.355 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.3300000000000001e-07 2024-09-14T06:29:40.355 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:40.356 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:40.356 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:29:40.356 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.083845+0000", 2024-09-14T06:29:40.356 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.4729999999999999e-06 2024-09-14T06:29:40.356 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:40.357 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:40.357 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:29:40.357 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.083850+0000", 2024-09-14T06:29:40.357 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.0130000000000001e-06 2024-09-14T06:29:40.357 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:40.357 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:40.357 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:29:40.357 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.083854+0000", 2024-09-14T06:29:40.357 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.4759999999999998e-06 2024-09-14T06:29:40.357 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:40.358 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:40.358 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:29:40.358 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.125636+0000", 2024-09-14T06:29:40.358 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.041782683000000001 2024-09-14T06:29:40.358 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:40.358 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:40.358 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:29:40.358 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.125680+0000", 2024-09-14T06:29:40.359 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.3614e-05 2024-09-14T06:29:40.359 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:40.359 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:40.359 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:29:40.359 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.125836+0000", 2024-09-14T06:29:40.359 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000155864 2024-09-14T06:29:40.359 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:40.359 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:40.359 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:40.359 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:40.359 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:40.360 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:14920 3.0 3:05c39535:::rbd_data.112d92027590.0000000000000009:f [read 816128~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:29:40.360 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:26.083868+0000", 2024-09-14T06:29:40.360 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 14.223113144999999, 2024-09-14T06:29:40.360 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.042166771999999998, 2024-09-14T06:29:40.360 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:29:40.360 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:29:40.360 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:29:40.360 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:29:40.360 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:29:40.360 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:40.360 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 14920 2024-09-14T06:29:40.360 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:40.360 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:29:40.360 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:40.360 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:29:40.361 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.083868+0000", 2024-09-14T06:29:40.361 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:40.361 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:40.361 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:40.361 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:29:40.361 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.083868+0000", 2024-09-14T06:29:40.361 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:40.361 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:40.361 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:40.361 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:29:40.361 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.083869+0000", 2024-09-14T06:29:40.361 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.99e-07 2024-09-14T06:29:40.361 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:40.361 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:40.361 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:29:40.362 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.083870+0000", 2024-09-14T06:29:40.362 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.782e-06 2024-09-14T06:29:40.362 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:40.362 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:40.362 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:29:40.362 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.083876+0000", 2024-09-14T06:29:40.362 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.3809999999999998e-06 2024-09-14T06:29:40.362 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:40.362 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:40.362 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:29:40.362 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.083882+0000", 2024-09-14T06:29:40.362 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.9479999999999996e-06 2024-09-14T06:29:40.362 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:40.362 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:40.363 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:29:40.363 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.125878+0000", 2024-09-14T06:29:40.363 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.041996264999999998 2024-09-14T06:29:40.363 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:40.363 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:40.363 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:29:40.363 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.125929+0000", 2024-09-14T06:29:40.363 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.0498999999999999e-05 2024-09-14T06:29:40.363 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:40.363 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:40.363 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:29:40.363 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.126035+0000", 2024-09-14T06:29:40.363 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000106298 2024-09-14T06:29:40.363 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:40.364 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:40.364 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:40.364 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:40.364 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:40.364 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:14921 3.0 3:05c39535:::rbd_data.112d92027590.0000000000000009:f [read 826368~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:29:40.364 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:26.083889+0000", 2024-09-14T06:29:40.364 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 14.223092135, 2024-09-14T06:29:40.364 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.042435309999999997, 2024-09-14T06:29:40.364 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:29:40.364 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:29:40.364 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:29:40.364 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:29:40.364 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:29:40.364 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:40.365 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 14921 2024-09-14T06:29:40.365 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:40.365 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:29:40.365 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:40.365 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:29:40.365 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.083889+0000", 2024-09-14T06:29:40.365 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:40.365 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:40.365 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:40.365 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:29:40.365 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.083889+0000", 2024-09-14T06:29:40.365 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:40.365 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:40.365 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:40.366 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:29:40.366 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.086033+0000", 2024-09-14T06:29:40.366 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0021444179999999999 2024-09-14T06:29:40.366 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:40.366 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:40.366 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:29:40.366 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.086038+0000", 2024-09-14T06:29:40.366 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.0969999999999999e-06 2024-09-14T06:29:40.366 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:40.366 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:40.366 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:29:40.366 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.086046+0000", 2024-09-14T06:29:40.366 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.9440000000000005e-06 2024-09-14T06:29:40.366 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:40.366 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:40.366 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:29:40.367 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.086050+0000", 2024-09-14T06:29:40.367 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.69e-06 2024-09-14T06:29:40.367 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:40.367 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:40.367 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:29:40.367 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.126080+0000", 2024-09-14T06:29:40.367 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.040030237000000003 2024-09-14T06:29:40.367 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:40.367 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:40.367 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:29:40.367 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.126163+0000", 2024-09-14T06:29:40.367 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.2071999999999996e-05 2024-09-14T06:29:40.367 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:40.367 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:40.367 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:29:40.368 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.126324+0000", 2024-09-14T06:29:40.368 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00016185200000000001 2024-09-14T06:29:40.368 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:40.368 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:40.368 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:40.368 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:40.368 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:40.368 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:14922 3.0 3:05c39535:::rbd_data.112d92027590.0000000000000009:f [read 836608~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:29:40.368 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:26.086060+0000", 2024-09-14T06:29:40.368 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 14.220921419, 2024-09-14T06:29:40.368 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.040546885999999997, 2024-09-14T06:29:40.368 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:29:40.368 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:29:40.368 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:29:40.368 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:29:40.369 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:29:40.369 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:40.369 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 14922 2024-09-14T06:29:40.369 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:40.369 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:29:40.369 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:40.369 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:29:40.369 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.086060+0000", 2024-09-14T06:29:40.369 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:40.369 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:40.369 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:40.369 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:29:40.369 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.086060+0000", 2024-09-14T06:29:40.369 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:40.369 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:40.370 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:40.370 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:29:40.370 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.086060+0000", 2024-09-14T06:29:40.370 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.5499999999999999e-07 2024-09-14T06:29:40.370 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:40.370 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:40.370 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:29:40.370 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.086061+0000", 2024-09-14T06:29:40.370 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.299e-06 2024-09-14T06:29:40.370 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:40.370 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:40.370 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:29:40.370 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.086065+0000", 2024-09-14T06:29:40.370 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.687e-06 2024-09-14T06:29:40.370 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:40.371 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:40.371 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:29:40.371 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.086068+0000", 2024-09-14T06:29:40.371 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.8219999999999998e-06 2024-09-14T06:29:40.371 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:40.371 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:40.371 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:29:40.371 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.126372+0000", 2024-09-14T06:29:40.371 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.040303701999999997 2024-09-14T06:29:40.371 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:40.371 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:40.371 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:29:40.371 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.126459+0000", 2024-09-14T06:29:40.371 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.7037e-05 2024-09-14T06:29:40.371 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:40.372 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:40.372 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:29:40.372 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.126607+0000", 2024-09-14T06:29:40.372 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00014808399999999999 2024-09-14T06:29:40.372 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:40.372 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:40.372 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:40.372 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:40.372 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:40.372 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:14923 3.0 3:05c39535:::rbd_data.112d92027590.0000000000000009:f [read 846848~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:29:40.372 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:26.086072+0000", 2024-09-14T06:29:40.372 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 14.220909385000001, 2024-09-14T06:29:40.372 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.040715464999999999, 2024-09-14T06:29:40.372 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:29:40.372 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:29:40.373 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:29:40.373 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:29:40.373 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:29:40.373 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:40.373 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 14923 2024-09-14T06:29:40.373 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:40.373 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:29:40.373 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:40.373 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:29:40.373 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.086072+0000", 2024-09-14T06:29:40.373 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:40.373 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:40.373 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:40.373 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:29:40.373 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.086072+0000", 2024-09-14T06:29:40.373 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:40.374 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:40.374 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:40.374 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:29:40.374 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.086072+0000", 2024-09-14T06:29:40.374 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.0699999999999999e-07 2024-09-14T06:29:40.374 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:40.374 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:40.374 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:29:40.374 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.086073+0000", 2024-09-14T06:29:40.374 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.0899999999999999e-06 2024-09-14T06:29:40.374 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:40.374 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:40.374 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:29:40.374 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.086076+0000", 2024-09-14T06:29:40.375 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.157e-06 2024-09-14T06:29:40.375 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:40.375 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:40.375 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:29:40.375 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.086079+0000", 2024-09-14T06:29:40.375 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.5299999999999999e-06 2024-09-14T06:29:40.375 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:40.375 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:40.375 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:29:40.375 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.126633+0000", 2024-09-14T06:29:40.375 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.040553812000000002 2024-09-14T06:29:40.375 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:40.375 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:40.375 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:29:40.376 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.126685+0000", 2024-09-14T06:29:40.376 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.2853999999999999e-05 2024-09-14T06:29:40.376 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:40.376 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:40.376 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:29:40.376 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.126787+0000", 2024-09-14T06:29:40.376 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000101815 2024-09-14T06:29:40.376 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:40.376 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:40.376 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:40.376 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:40.376 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:40.376 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:14924 3.0 3:05c39535:::rbd_data.112d92027590.0000000000000009:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:29:40.377 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:26.086082+0000", 2024-09-14T06:29:40.377 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 14.220899156, 2024-09-14T06:29:40.377 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.040904439000000001, 2024-09-14T06:29:40.377 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:29:40.377 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:29:40.377 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:29:40.377 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:29:40.377 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:29:40.377 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:40.377 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 14924 2024-09-14T06:29:40.377 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:40.377 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:29:40.377 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:40.377 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:29:40.377 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.086082+0000", 2024-09-14T06:29:40.378 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:40.378 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:40.378 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:40.378 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:29:40.378 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.086082+0000", 2024-09-14T06:29:40.378 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:40.378 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:40.378 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:40.378 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:29:40.378 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.086082+0000", 2024-09-14T06:29:40.378 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.29e-07 2024-09-14T06:29:40.378 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:40.378 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:40.378 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:29:40.378 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.086083+0000", 2024-09-14T06:29:40.379 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.0410000000000001e-06 2024-09-14T06:29:40.379 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:40.379 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:40.379 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:29:40.379 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.086086+0000", 2024-09-14T06:29:40.379 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.179e-06 2024-09-14T06:29:40.379 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:40.379 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:40.379 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:29:40.379 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.086090+0000", 2024-09-14T06:29:40.380 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.0069999999999999e-06 2024-09-14T06:29:40.380 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:40.380 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:40.380 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:29:40.380 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.126832+0000", 2024-09-14T06:29:40.380 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.040741357999999998 2024-09-14T06:29:40.380 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:40.380 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:40.380 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:29:40.380 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.126906+0000", 2024-09-14T06:29:40.380 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.3935000000000003e-05 2024-09-14T06:29:40.380 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:40.380 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:40.380 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:29:40.381 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.126986+0000", 2024-09-14T06:29:40.381 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.0690000000000002e-05 2024-09-14T06:29:40.381 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:40.381 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:40.381 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:40.381 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:40.381 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:40.381 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:14927 3.0 3:05c39535:::rbd_data.112d92027590.0000000000000009:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:29:40.381 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:26.086130+0000", 2024-09-14T06:29:40.381 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 14.220850896, 2024-09-14T06:29:40.381 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.040995357000000003, 2024-09-14T06:29:40.381 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:29:40.382 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:29:40.382 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:29:40.382 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:29:40.382 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:29:40.382 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:40.382 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 14927 2024-09-14T06:29:40.382 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:40.382 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:29:40.382 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:40.382 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:29:40.382 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.086130+0000", 2024-09-14T06:29:40.382 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:40.382 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:40.382 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:40.382 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:29:40.382 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.086130+0000", 2024-09-14T06:29:40.383 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:40.383 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:40.383 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:40.383 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:29:40.383 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.086131+0000", 2024-09-14T06:29:40.383 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.2e-07 2024-09-14T06:29:40.383 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:40.383 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:40.383 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:29:40.383 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.086133+0000", 2024-09-14T06:29:40.383 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.3599999999999999e-06 2024-09-14T06:29:40.383 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:40.383 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:40.383 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:29:40.383 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.086139+0000", 2024-09-14T06:29:40.384 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.9970000000000004e-06 2024-09-14T06:29:40.384 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:40.384 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:40.384 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:29:40.384 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.086144+0000", 2024-09-14T06:29:40.384 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.3029999999999999e-06 2024-09-14T06:29:40.384 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:40.384 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:40.384 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:29:40.384 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.127032+0000", 2024-09-14T06:29:40.384 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.040887236 2024-09-14T06:29:40.384 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:40.384 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:40.384 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:29:40.384 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.127077+0000", 2024-09-14T06:29:40.385 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.5553000000000002e-05 2024-09-14T06:29:40.385 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:40.385 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:40.385 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:29:40.385 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.127126+0000", 2024-09-14T06:29:40.385 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.8488000000000001e-05 2024-09-14T06:29:40.385 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:40.385 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:40.385 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:40.385 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:40.385 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:40.385 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:14932 3.0 3:05c39535:::rbd_data.112d92027590.0000000000000009:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:29:40.385 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:26.086152+0000", 2024-09-14T06:29:40.386 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 14.220829137999999, 2024-09-14T06:29:40.386 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.041108513999999999, 2024-09-14T06:29:40.386 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:29:40.386 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:29:40.386 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:29:40.386 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:29:40.386 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:29:40.386 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:40.386 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 14932 2024-09-14T06:29:40.386 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:40.386 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:29:40.386 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:40.386 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:29:40.386 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.086152+0000", 2024-09-14T06:29:40.386 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:40.387 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:40.387 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:40.387 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:29:40.387 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.086152+0000", 2024-09-14T06:29:40.387 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:40.387 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:40.387 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:40.387 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:29:40.387 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.086152+0000", 2024-09-14T06:29:40.387 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.9000000000000002e-07 2024-09-14T06:29:40.387 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:40.387 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:40.387 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:29:40.387 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.086156+0000", 2024-09-14T06:29:40.387 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.3500000000000001e-06 2024-09-14T06:29:40.387 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:40.388 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:40.388 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:29:40.388 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.086162+0000", 2024-09-14T06:29:40.388 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.0120000000000002e-06 2024-09-14T06:29:40.388 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:40.388 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:40.388 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:29:40.388 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.086166+0000", 2024-09-14T06:29:40.388 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.6269999999999999e-06 2024-09-14T06:29:40.388 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:40.388 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:40.388 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:29:40.388 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.127162+0000", 2024-09-14T06:29:40.389 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.040995386000000002 2024-09-14T06:29:40.389 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:40.389 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:40.389 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:29:40.389 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.127214+0000", 2024-09-14T06:29:40.389 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.1860999999999997e-05 2024-09-14T06:29:40.389 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:40.389 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:40.389 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:29:40.389 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.127261+0000", 2024-09-14T06:29:40.389 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.6888000000000002e-05 2024-09-14T06:29:40.389 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:40.389 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:40.389 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:40.389 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:40.390 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:40.390 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:14937 3.0 3:05c39535:::rbd_data.112d92027590.0000000000000009:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:29:40.390 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:26.086173+0000", 2024-09-14T06:29:40.390 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 14.220808392, 2024-09-14T06:29:40.390 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.041239617999999999, 2024-09-14T06:29:40.390 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:29:40.390 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:29:40.390 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:29:40.390 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:29:40.390 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:29:40.390 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:40.390 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 14937 2024-09-14T06:29:40.390 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:40.390 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:29:40.391 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:40.391 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:29:40.391 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.086173+0000", 2024-09-14T06:29:40.391 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:40.391 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:40.391 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:40.391 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:29:40.391 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.086173+0000", 2024-09-14T06:29:40.391 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:40.391 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:40.391 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:40.391 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:29:40.391 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.086173+0000", 2024-09-14T06:29:40.391 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.5400000000000002e-07 2024-09-14T06:29:40.391 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:40.392 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:40.392 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:29:40.392 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.086175+0000", 2024-09-14T06:29:40.392 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.8479999999999999e-06 2024-09-14T06:29:40.392 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:40.392 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:40.392 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:29:40.392 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.086181+0000", 2024-09-14T06:29:40.392 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.8490000000000002e-06 2024-09-14T06:29:40.392 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:40.392 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:40.392 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:29:40.392 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.086185+0000", 2024-09-14T06:29:40.392 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.6190000000000002e-06 2024-09-14T06:29:40.392 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:40.393 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:40.393 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:29:40.393 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.127292+0000", 2024-09-14T06:29:40.393 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.041106119000000003 2024-09-14T06:29:40.393 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:40.393 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:40.393 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:29:40.393 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.127353+0000", 2024-09-14T06:29:40.393 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.1313000000000001e-05 2024-09-14T06:29:40.393 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:40.393 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:40.393 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:29:40.393 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.127412+0000", 2024-09-14T06:29:40.393 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.9515999999999997e-05 2024-09-14T06:29:40.394 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:40.394 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:40.394 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:40.394 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:40.394 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:40.394 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:14942 3.0 3:05c39535:::rbd_data.112d92027590.0000000000000009:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:29:40.394 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:26.086192+0000", 2024-09-14T06:29:40.394 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 14.220789532, 2024-09-14T06:29:40.394 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.041417974000000003, 2024-09-14T06:29:40.394 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:29:40.394 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:29:40.394 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:29:40.394 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:29:40.394 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:29:40.394 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:40.395 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 14942 2024-09-14T06:29:40.395 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:40.395 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:29:40.395 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:40.395 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:29:40.395 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.086192+0000", 2024-09-14T06:29:40.395 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:40.395 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:40.395 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:40.395 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:29:40.395 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.086192+0000", 2024-09-14T06:29:40.395 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:40.395 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:40.395 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:40.395 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:29:40.395 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.086192+0000", 2024-09-14T06:29:40.396 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.22e-07 2024-09-14T06:29:40.396 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:40.396 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:40.396 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:29:40.396 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.086199+0000", 2024-09-14T06:29:40.396 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.4789999999999999e-06 2024-09-14T06:29:40.396 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:40.396 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:40.396 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:29:40.396 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.086205+0000", 2024-09-14T06:29:40.397 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.0249999999999999e-06 2024-09-14T06:29:40.397 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:40.397 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:40.397 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:29:40.397 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.086210+0000", 2024-09-14T06:29:40.397 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.7400000000000004e-06 2024-09-14T06:29:40.397 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:40.397 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:40.397 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:29:40.397 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.127455+0000", 2024-09-14T06:29:40.397 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.041244691999999999 2024-09-14T06:29:40.397 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:40.397 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:40.398 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:29:40.398 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.127529+0000", 2024-09-14T06:29:40.398 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.4467000000000004e-05 2024-09-14T06:29:40.398 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:40.398 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:40.398 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:29:40.398 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.127610+0000", 2024-09-14T06:29:40.398 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.0248999999999997e-05 2024-09-14T06:29:40.398 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:40.398 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:40.398 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:40.398 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:40.398 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:40.398 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:14946 3.0 3:05c39535:::rbd_data.112d92027590.0000000000000009:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:29:40.399 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:26.086218+0000", 2024-09-14T06:29:40.399 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 14.220763602, 2024-09-14T06:29:40.399 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.041568755999999998, 2024-09-14T06:29:40.399 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:29:40.399 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:29:40.399 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:29:40.399 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:29:40.399 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:29:40.399 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:40.399 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 14946 2024-09-14T06:29:40.399 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:40.399 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:29:40.399 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:40.399 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:29:40.400 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.086218+0000", 2024-09-14T06:29:40.400 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:40.400 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:40.400 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:40.400 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:29:40.400 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.086218+0000", 2024-09-14T06:29:40.400 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:40.400 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:40.400 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:40.400 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:29:40.400 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.086218+0000", 2024-09-14T06:29:40.400 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.7399999999999999e-07 2024-09-14T06:29:40.400 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:40.400 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:40.400 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:29:40.401 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.086220+0000", 2024-09-14T06:29:40.401 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.9649999999999998e-06 2024-09-14T06:29:40.401 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:40.401 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:40.401 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:29:40.401 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.086226+0000", 2024-09-14T06:29:40.401 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.7130000000000002e-06 2024-09-14T06:29:40.401 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:40.401 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:40.401 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:29:40.401 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.086230+0000", 2024-09-14T06:29:40.401 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.5539999999999999e-06 2024-09-14T06:29:40.401 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:40.402 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:40.402 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:29:40.402 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.127640+0000", 2024-09-14T06:29:40.402 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.041409396000000001 2024-09-14T06:29:40.402 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:40.402 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:40.402 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:29:40.402 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.127710+0000", 2024-09-14T06:29:40.402 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.0333000000000006e-05 2024-09-14T06:29:40.403 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:40.403 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:40.403 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:29:40.403 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.127786+0000", 2024-09-14T06:29:40.403 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.6421000000000006e-05 2024-09-14T06:29:40.403 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:40.403 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:40.403 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:40.403 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:40.403 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:40.403 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-14T06:29:40.404 INFO:teuthology.orchestra.run.smithi094.stdout:1032 writesame 0x82a739 thru 0x83958c data_size 0x7(0xee54 bytes) 2024-09-14T06:29:40.405 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-14T06:29:40.407 INFO:teuthology.orchestra.run.smithi094.stdout:1034 writesame 0xdd45890 thru 0xdd49fa3 data_size 0x2(0x4714 bytes) 2024-09-14T06:29:40.407 INFO:teuthology.orchestra.run.smithi094.stdout:1035 write 0x2e4d5d1 thru 0x2e5530a (0x7d3a bytes) 2024-09-14T06:29:40.407 INFO:teuthology.orchestra.run.smithi094.stdout:1036 read 0x1d53e41 thru 0x1d5bcbb (0x7e7b bytes) 2024-09-14T06:29:40.408 INFO:teuthology.orchestra.run.smithi094.stdout:1037 read 0x163f06e thru 0x1646bc1 (0x7b54 bytes) 2024-09-14T06:29:40.408 INFO:teuthology.orchestra.run.smithi094.stdout:1040 read 0x8e34564 thru 0x8e3d29d (0x8d3a bytes) 2024-09-14T06:29:40.408 INFO:teuthology.orchestra.run.smithi094.stdout:1042 punch from 0x8a7e6a4 to 0x8a84edd, (0x6839 bytes) 2024-09-14T06:29:40.408 INFO:teuthology.orchestra.run.smithi094.stdout:1044 punch from 0xac5f0c to 0xad522b, (0xf31f bytes) 2024-09-14T06:29:40.408 INFO:teuthology.orchestra.run.smithi094.stdout:1045 trunc from 0xdd49fa4 to 0xe5d268c 2024-09-14T06:29:40.408 INFO:teuthology.orchestra.run.smithi094.stdout:1048 read 0x443ad84 thru 0x443f36b (0x45e8 bytes) 2024-09-14T06:29:40.408 INFO:teuthology.orchestra.run.smithi094.stdout:1049 read 0x9118271 thru 0x911c3af (0x413f bytes) 2024-09-14T06:29:40.409 INFO:teuthology.orchestra.run.smithi094.stdout:1050 writesame 0x36b1aff thru 0x36bad4b data_size 0x2b(0x924d bytes) 2024-09-14T06:29:40.409 INFO:teuthology.orchestra.run.smithi094.stdout:1052 trunc from 0xe5d268c to 0xaffd7a 2024-09-14T06:29:40.409 INFO:teuthology.orchestra.run.smithi094.stdout:1053 trunc from 0xaffd7a to 0xcc80eeb 2024-09-14T06:29:40.409 INFO:teuthology.orchestra.run.smithi094.stdout:1054 read 0xb799b7d thru 0xb79a0bc (0x540 bytes) 2024-09-14T06:29:40.409 INFO:teuthology.orchestra.run.smithi094.stdout:1055 punch from 0x562a562 to 0x562b4a4, (0xf42 bytes) 2024-09-14T06:29:40.409 INFO:teuthology.orchestra.run.smithi094.stdout:1056 writesame 0x5d110da thru 0x5d1598c data_size 0x1(0x48b3 bytes) 2024-09-14T06:29:40.409 INFO:teuthology.orchestra.run.smithi094.stdout:1057 read 0x7c8d4a9 thru 0x7c96db2 (0x990a bytes) 2024-09-14T06:29:40.409 INFO:teuthology.orchestra.run.smithi094.stdout:1058 read 0x8c6fcb2 thru 0x8c7b721 (0xba70 bytes) 2024-09-14T06:29:40.409 INFO:teuthology.orchestra.run.smithi094.stdout:1060 write 0x6dd62f6 thru 0x6dd9b1f (0x382a bytes) 2024-09-14T06:29:40.410 INFO:teuthology.orchestra.run.smithi094.stdout:1061 write 0x1d2c38b thru 0x1d31024 (0x4c9a bytes) 2024-09-14T06:29:40.410 INFO:teuthology.orchestra.run.smithi094.stdout:1063 punch from 0x77ba633 to 0x77c4ad1, (0xa49e bytes) 2024-09-14T06:29:40.410 INFO:teuthology.orchestra.run.smithi094.stdout:1064 punch from 0x5bd5b15 to 0x5bddf01, (0x83ec bytes) 2024-09-14T06:29:40.411 INFO:teuthology.orchestra.run.smithi094.stdout:1065 write 0xb907f2f thru 0xb90d186 (0x5258 bytes) 2024-09-14T06:29:40.411 INFO:tasks.ceph.osd.1.smithi094.stderr:2024-09-14T06:29:40.410+0000 7f328e47a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33641) UID: 0 2024-09-14T06:29:40.412 INFO:teuthology.orchestra.run.smithi094.stdout:1067 punch from 0xc2087c0 to 0xc20bb96, (0x33d6 bytes) 2024-09-14T06:29:40.422 INFO:teuthology.orchestra.run.smithi094.stdout:1068 trunc from 0xcc80eeb to 0x4446de4 2024-09-14T06:29:40.434 INFO:teuthology.orchestra.run.smithi094.stdout:1069 punch from 0x2ddf6ae to 0x2de3b82, (0x44d4 bytes) 2024-09-14T06:29:40.434 INFO:teuthology.orchestra.run.smithi094.stdout:1072 read 0x2750521 thru 0x275c259 (0xbd39 bytes) 2024-09-14T06:29:40.435 INFO:teuthology.orchestra.run.smithi094.stdout:1073 punch from 0x1a61af7 to 0x1a6a4a4, (0x89ad bytes) 2024-09-14T06:29:40.435 INFO:teuthology.orchestra.run.smithi094.stdout:1074 read 0x17a8261 thru 0x17afd13 (0x7ab3 bytes) 2024-09-14T06:29:40.435 INFO:teuthology.orchestra.run.smithi094.stdout:1075 write 0x3ca6bbf thru 0x3cacb06 (0x5f48 bytes) 2024-09-14T06:29:40.455 INFO:teuthology.orchestra.run.smithi094.stdout:1076 write 0xd98f7f1 thru 0xd9916ac (0x1ebc bytes) 2024-09-14T06:29:40.458 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-14T06:29:40.458 INFO:teuthology.orchestra.run.smithi094.stdout:1078 writesame 0x781ad13 thru 0x7820ecd data_size 0x1(0x61bb bytes) 2024-09-14T06:29:40.471 INFO:teuthology.orchestra.run.smithi094.stdout:1079 writesame 0x4d994c2 thru 0x4da325d data_size 0x1(0x9d9c bytes) 2024-09-14T06:29:40.493 INFO:teuthology.orchestra.run.smithi094.stdout:1080 read 0xb54b5af thru 0xb558b61 (0xd5b3 bytes) 2024-09-14T06:29:40.494 INFO:teuthology.orchestra.run.smithi094.stdout:1081 write 0xad3f0a9 thru 0xad497bb (0xa713 bytes) 2024-09-14T06:29:40.497 INFO:teuthology.orchestra.run.smithi094.stdout:1083 write 0xd3faaa2 thru 0xd3ff7dd (0x4d3c bytes) 2024-09-14T06:29:40.500 INFO:teuthology.orchestra.run.smithi094.stdout:1084 read 0xc4dcc9 thru 0xc5d058 (0xf390 bytes) 2024-09-14T06:29:40.500 INFO:teuthology.orchestra.run.smithi094.stdout:1086 writesame 0x8308bb6 thru 0x830b468 data_size 0x17(0x28b3 bytes) 2024-09-14T06:29:40.503 INFO:teuthology.orchestra.run.smithi094.stdout:1087 trunc from 0xd9916ad to 0xea29cf1 2024-09-14T06:29:40.509 INFO:teuthology.orchestra.run.smithi094.stdout:1089 read 0x302f5ed thru 0x30387a5 (0x91b9 bytes) 2024-09-14T06:29:40.509 INFO:teuthology.orchestra.run.smithi094.stdout:1090 read 0x8b3fd25 thru 0x8b4ca0a (0xcce6 bytes) 2024-09-14T06:29:40.509 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-14T06:29:40.509 INFO:teuthology.orchestra.run.smithi094.stdout:1092 write 0x97b52dd thru 0x97bf73e (0xa462 bytes) 2024-09-14T06:29:40.511 INFO:teuthology.orchestra.run.smithi094.stdout:1093 punch from 0x8e7c83b to 0x8e845f4, (0x7db9 bytes) 2024-09-14T06:29:40.511 INFO:teuthology.orchestra.run.smithi094.stdout:1094 writesame 0x6d0d768 thru 0x6d17684 data_size 0x4d(0x9f1d bytes) 2024-09-14T06:29:40.511 INFO:tasks.ceph.osd.6.smithi138.stderr:2024-09-14T06:29:40.510+0000 7f8cd8849640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33620) UID: 0 2024-09-14T06:29:40.511 INFO:teuthology.orchestra.run.smithi094.stdout:1095 writesame 0x6827779 thru 0x6835852 data_size 0x11(0xe0da bytes) 2024-09-14T06:29:40.515 INFO:teuthology.orchestra.run.smithi094.stdout:1096 write 0xc987a61 thru 0xc995c29 (0xe1c9 bytes) 2024-09-14T06:29:40.518 INFO:teuthology.orchestra.run.smithi094.stdout:1097 read 0x3b04187 thru 0x3b08865 (0x46df bytes) 2024-09-14T06:29:40.518 INFO:teuthology.orchestra.run.smithi094.stdout:1098 write 0x38892be thru 0x3892742 (0x9485 bytes) 2024-09-14T06:29:40.523 INFO:teuthology.orchestra.run.smithi094.stdout:1100 read 0x3478ee1 thru 0x347d610 (0x4730 bytes) 2024-09-14T06:29:40.523 INFO:teuthology.orchestra.run.smithi094.stdout:1101 trunc from 0xea29cf1 to 0xeb11444 2024-09-14T06:29:40.525 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-14T06:29:40.525 INFO:teuthology.orchestra.run.smithi094.stdout: "ops": [], 2024-09-14T06:29:40.525 INFO:teuthology.orchestra.run.smithi094.stdout: "num_ops": 0 2024-09-14T06:29:40.525 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-14T06:29:40.528 INFO:teuthology.orchestra.run.smithi094.stdout:1102 punch from 0x5297e6d to 0x529e445, (0x65d8 bytes) 2024-09-14T06:29:40.531 INFO:teuthology.orchestra.run.smithi094.stdout:1103 trunc from 0xeb11444 to 0xc77d67b 2024-09-14T06:29:40.536 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-14T06:29:40.537 INFO:teuthology.orchestra.run.smithi094.stdout:1105 writesame 0x5b74c57 thru 0x5b777fc data_size 0x25(0x2ba6 bytes) 2024-09-14T06:29:40.540 INFO:teuthology.orchestra.run.smithi094.stdout:1106 writesame 0x14d1458 thru 0x14d951a data_size 0x11(0x80c3 bytes) 2024-09-14T06:29:40.543 INFO:teuthology.orchestra.run.smithi094.stdout:1109 write 0xbe9d2ab thru 0xbeab325 (0xe07b bytes) 2024-09-14T06:29:40.552 INFO:teuthology.orchestra.run.smithi094.stdout:1110 write 0xe3d090d thru 0xe3d67b5 (0x5ea9 bytes) 2024-09-14T06:29:40.568 INFO:teuthology.orchestra.run.smithi094.stdout:1111 trunc from 0xe3d67b6 to 0x370101a 2024-09-14T06:29:40.585 INFO:teuthology.orchestra.run.smithi094.stdout:1112 writesame 0x848a18 thru 0x84f76e data_size 0x1(0x6d57 bytes) 2024-09-14T06:29:40.602 INFO:teuthology.orchestra.run.smithi094.stdout:1113 trunc from 0x370101a to 0x5b21f8 2024-09-14T06:29:40.612 INFO:tasks.ceph.osd.3.smithi094.stderr:2024-09-14T06:29:40.611+0000 7feeeafe8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33655) UID: 0 2024-09-14T06:29:40.623 INFO:teuthology.orchestra.run.smithi094.stdout:1114 write 0x2c52b6 thru 0x2c9f3d (0x4c88 bytes) 2024-09-14T06:29:40.625 INFO:teuthology.orchestra.run.smithi094.stdout:1119 read 0x30119c thru 0x302e94 (0x1cf9 bytes) 2024-09-14T06:29:40.638 INFO:teuthology.orchestra.run.smithi094.stdout:1120 write 0x6d9112c thru 0x6d9b7cb (0xa6a0 bytes) 2024-09-14T06:29:40.641 INFO:teuthology.orchestra.run.smithi094.stdout:1122 write 0x670cd4 thru 0x67721c (0x6549 bytes) 2024-09-14T06:29:40.644 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-14T06:29:40.647 INFO:teuthology.orchestra.run.smithi094.stdout:1124 trunc from 0x6d9b7cc to 0x3ec2aab 2024-09-14T06:29:40.652 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-14T06:29:40.652 INFO:teuthology.orchestra.run.smithi094.stdout: "ops": [], 2024-09-14T06:29:40.652 INFO:teuthology.orchestra.run.smithi094.stdout: "complaint_time": 30, 2024-09-14T06:29:40.652 INFO:teuthology.orchestra.run.smithi094.stdout: "num_blocked_ops": 0 2024-09-14T06:29:40.652 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-14T06:29:40.656 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-14T06:29:40.656 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-14T06:29:40.660 INFO:teuthology.orchestra.run.smithi094.stdout:1128 trunc from 0x3ec2aab to 0xa2d756 2024-09-14T06:29:40.662 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-14T06:29:40.678 INFO:teuthology.orchestra.run.smithi094.stdout:1129 trunc from 0xa2d756 to 0xab2a6bd 2024-09-14T06:29:40.684 INFO:teuthology.orchestra.run.smithi094.stdout:1130 read 0xa19a7ff thru 0xa19fcb2 (0x54b4 bytes) 2024-09-14T06:29:40.684 INFO:teuthology.orchestra.run.smithi094.stdout:1132 punch from 0xaa15f41 to 0xaa1d60c, (0x76cb bytes) 2024-09-14T06:29:40.684 INFO:teuthology.orchestra.run.smithi094.stdout:1133 read 0x39dab29 thru 0x39e971a (0xebf2 bytes) 2024-09-14T06:29:40.684 INFO:teuthology.orchestra.run.smithi094.stdout:1134 write 0xa5482 thru 0xa667d (0x11fc bytes) 2024-09-14T06:29:40.685 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-14T06:29:40.685 INFO:teuthology.orchestra.run.smithi094.stdout:1137 writesame 0x1b2b6b5 thru 0x1b31ead data_size 0x1(0x67f9 bytes) 2024-09-14T06:29:40.699 INFO:teuthology.orchestra.run.smithi094.stdout:1138 read 0x7f915f0 thru 0x7f9cf1b (0xb92c bytes) 2024-09-14T06:29:40.699 INFO:teuthology.orchestra.run.smithi094.stdout:1139 write 0x1209674 thru 0x1216017 (0xc9a4 bytes) 2024-09-14T06:29:40.702 INFO:teuthology.orchestra.run.smithi094.stdout:1140 punch from 0x8e347fa to 0x8e3f581, (0xad87 bytes) 2024-09-14T06:29:40.703 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-14T06:29:40.706 INFO:teuthology.orchestra.run.smithi094.stdout:1142 trunc from 0xab2a6bd to 0xdd07832 2024-09-14T06:29:40.711 INFO:teuthology.orchestra.run.smithi094.stdout:1143 writesame 0x44061b thru 0x441077 data_size 0x1(0xa5d bytes) 2024-09-14T06:29:40.712 INFO:tasks.ceph.osd.1.smithi094.stderr:2024-09-14T06:29:40.711+0000 7f328e47a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33641) UID: 0 2024-09-14T06:29:40.713 INFO:teuthology.orchestra.run.smithi094.stdout:1144 read 0xaa12893 thru 0xaa14df0 (0x255e bytes) 2024-09-14T06:29:40.714 INFO:teuthology.orchestra.run.smithi094.stdout:1146 write 0x7dbbe4 thru 0x7e219e (0x65bb bytes) 2024-09-14T06:29:40.715 INFO:teuthology.orchestra.run.smithi094.stdout:1147 write 0xb308b11 thru 0xb30d800 (0x4cf0 bytes) 2024-09-14T06:29:40.718 INFO:teuthology.orchestra.run.smithi094.stdout:1149 punch from 0x992d411 to 0x99310f3, (0x3ce2 bytes) 2024-09-14T06:29:40.724 INFO:teuthology.orchestra.run.smithi094.stdout:1150 trunc from 0xdd07832 to 0x91aee45 2024-09-14T06:29:40.735 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-14T06:29:40.735 INFO:teuthology.orchestra.run.smithi094.stdout:1155 write 0x8fdb41a thru 0x8fe4d5a (0x9941 bytes) 2024-09-14T06:29:40.747 INFO:teuthology.orchestra.run.smithi094.stdout:1157 write 0xbaa2bd7 thru 0xbaae9a1 (0xbdcb bytes) 2024-09-14T06:29:40.750 INFO:teuthology.orchestra.run.smithi094.stdout:1158 punch from 0x9afa003 to 0x9b05e44, (0xbe41 bytes) 2024-09-14T06:29:40.750 INFO:teuthology.orchestra.run.smithi094.stdout:1159 punch from 0x873d851 to 0x8747b8f, (0xa33e bytes) 2024-09-14T06:29:40.751 INFO:teuthology.orchestra.run.smithi094.stdout:1160 punch from 0xb202353 to 0xb203dc8, (0x1a75 bytes) 2024-09-14T06:29:40.751 INFO:teuthology.orchestra.run.smithi094.stdout:1161 read 0x8ab5b41 thru 0x8ab5e53 (0x313 bytes) 2024-09-14T06:29:40.751 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-14T06:29:40.751 INFO:teuthology.orchestra.run.smithi094.stdout:1163 writesame 0xb3fa8f3 thru 0xb4092b1 data_size 0x1(0xe9bf bytes) 2024-09-14T06:29:40.771 INFO:teuthology.orchestra.run.smithi094.stdout:1164 read 0x3898c46 thru 0x389fe08 (0x71c3 bytes) 2024-09-14T06:29:40.771 INFO:teuthology.orchestra.run.smithi094.stdout:1165 write 0x157e864 thru 0x158698e (0x812b bytes) 2024-09-14T06:29:40.775 INFO:teuthology.orchestra.run.smithi094.stdout:1166 write 0x12f43 thru 0x1f6be (0xc77c bytes) 2024-09-14T06:29:40.776 INFO:teuthology.orchestra.run.smithi094.stdout:1167 punch from 0x3f2affc to 0x3f2db59, (0x2b5d bytes) 2024-09-14T06:29:40.777 INFO:teuthology.orchestra.run.smithi094.stdout:1168 writesame 0x2b4b61d thru 0x2b501fd data_size 0x25(0x4be1 bytes) 2024-09-14T06:29:40.778 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-14T06:29:40.784 INFO:teuthology.orchestra.run.smithi094.stdout:1171 write 0xbdf8b47 thru 0xbdfefb9 (0x6473 bytes) 2024-09-14T06:29:40.787 INFO:teuthology.orchestra.run.smithi094.stdout:1172 read 0x12cae70 thru 0x12d98d0 (0xea61 bytes) 2024-09-14T06:29:40.787 INFO:teuthology.orchestra.run.smithi094.stdout:1173 read 0x7930f29 thru 0x7937cd0 (0x6da8 bytes) 2024-09-14T06:29:40.787 INFO:teuthology.orchestra.run.smithi094.stdout:1174 punch from 0x232d1 to 0x28c1e, (0x594d bytes) 2024-09-14T06:29:40.788 INFO:teuthology.orchestra.run.smithi094.stdout:1176 read 0x4434a7f thru 0x443560c (0xb8e bytes) 2024-09-14T06:29:40.788 INFO:teuthology.orchestra.run.smithi094.stdout:1177 write 0x45fab0b thru 0x45fbd75 (0x126b bytes) 2024-09-14T06:29:40.790 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-14T06:29:40.790 INFO:teuthology.orchestra.run.smithi094.stdout:1181 read 0x57e061e thru 0x57e38aa (0x328d bytes) 2024-09-14T06:29:40.795 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-14T06:29:40.795 INFO:teuthology.orchestra.run.smithi094.stdout: "size": 20, 2024-09-14T06:29:40.795 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 600, 2024-09-14T06:29:40.795 INFO:teuthology.orchestra.run.smithi094.stdout: "ops": [ 2024-09-14T06:29:40.795 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:40.795 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4299.0:1 2.3 e20/19 2:c4c92e5a:::rbd_trash:head v 20'1, mlcod=0'0)", 2024-09-14T06:29:40.795 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:28:51.223664+0000", 2024-09-14T06:29:40.795 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 49.569126609999998, 2024-09-14T06:29:40.796 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.001165588, 2024-09-14T06:29:40.796 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:29:40.796 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:29:40.796 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:29:40.796 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:29:40.796 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:40.796 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:29:40.796 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:28:51.223664+0000", 2024-09-14T06:29:40.796 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:40.796 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:40.796 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:40.797 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:29:40.797 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:28:51.223664+0000", 2024-09-14T06:29:40.797 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:40.797 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:40.797 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:40.797 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:29:40.797 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:28:51.223665+0000", 2024-09-14T06:29:40.798 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.3600000000000003e-07 2024-09-14T06:29:40.798 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:40.798 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:40.798 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:29:40.798 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:28:51.223672+0000", 2024-09-14T06:29:40.798 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.4740000000000002e-06 2024-09-14T06:29:40.798 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:40.798 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:40.798 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:29:40.798 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:28:51.223695+0000", 2024-09-14T06:29:40.798 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.3859e-05 2024-09-14T06:29:40.798 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:40.798 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:40.799 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:29:40.799 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:28:51.223715+0000", 2024-09-14T06:29:40.799 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.9406e-05 2024-09-14T06:29:40.799 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:40.799 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:40.799 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:29:40.799 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:28:51.223780+0000", 2024-09-14T06:29:40.799 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.4777999999999995e-05 2024-09-14T06:29:40.799 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:40.799 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:40.799 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:29:40.799 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:28:51.223817+0000", 2024-09-14T06:29:40.799 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.7660999999999997e-05 2024-09-14T06:29:40.799 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:40.800 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:40.800 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-14T06:29:40.800 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:28:51.224781+0000", 2024-09-14T06:29:40.800 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00096367800000000004 2024-09-14T06:29:40.800 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:40.800 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:40.800 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:29:40.800 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:28:51.224830+0000", 2024-09-14T06:29:40.800 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.9095999999999999e-05 2024-09-14T06:29:40.800 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:40.800 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:40.800 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:40.800 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:40.800 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:40.801 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4397.0:937 3.9 e30/23 3:92b37d3e:::rbd_data.112d7dfbd766.0000000000000012:head v 30'7, mlcod=30'5)", 2024-09-14T06:29:40.801 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:07.169523+0000", 2024-09-14T06:29:40.801 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 33.623268187999997, 2024-09-14T06:29:40.801 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0013381339999999999, 2024-09-14T06:29:40.801 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:29:40.801 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:29:40.801 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:29:40.801 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:29:40.801 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:40.801 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:29:40.801 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:07.169523+0000", 2024-09-14T06:29:40.801 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:40.801 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:40.802 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:40.802 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:29:40.802 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:07.169523+0000", 2024-09-14T06:29:40.802 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:40.802 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:40.802 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:40.802 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:29:40.802 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:07.169524+0000", 2024-09-14T06:29:40.802 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.125e-06 2024-09-14T06:29:40.802 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:40.802 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:40.802 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:29:40.802 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:07.170106+0000", 2024-09-14T06:29:40.803 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00058239900000000005 2024-09-14T06:29:40.803 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:40.803 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:40.803 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:29:40.803 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:07.170121+0000", 2024-09-14T06:29:40.803 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.4851999999999999e-05 2024-09-14T06:29:40.803 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:40.803 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:40.803 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:29:40.803 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:07.170130+0000", 2024-09-14T06:29:40.803 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.9199999999999993e-06 2024-09-14T06:29:40.803 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:40.803 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:40.803 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:29:40.803 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:07.170182+0000", 2024-09-14T06:29:40.804 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.1669999999999998e-05 2024-09-14T06:29:40.804 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:40.804 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:40.804 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:29:40.804 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:07.170199+0000", 2024-09-14T06:29:40.804 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.7193999999999998e-05 2024-09-14T06:29:40.804 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:40.804 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:40.804 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-14T06:29:40.804 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:07.170816+0000", 2024-09-14T06:29:40.804 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00061745400000000003 2024-09-14T06:29:40.804 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:40.804 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:40.804 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:29:40.805 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:07.170861+0000", 2024-09-14T06:29:40.805 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.4520000000000001e-05 2024-09-14T06:29:40.805 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:40.805 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:40.805 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:40.805 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:40.805 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:40.805 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4397.0:1848 3.4 e32/23 3:27620a21:::rbd_data.112dcdddac74.000000000000000b:head v 32'40, mlcod=30'38)", 2024-09-14T06:29:40.805 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:08.658649+0000", 2024-09-14T06:29:40.805 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 32.134141815, 2024-09-14T06:29:40.806 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.001862867, 2024-09-14T06:29:40.806 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:29:40.806 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:29:40.806 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:29:40.806 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:29:40.806 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:40.806 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:29:40.806 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.658649+0000", 2024-09-14T06:29:40.806 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:40.806 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:40.806 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:40.806 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:29:40.806 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.658649+0000", 2024-09-14T06:29:40.806 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:40.807 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:40.807 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:40.807 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:29:40.807 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.658651+0000", 2024-09-14T06:29:40.807 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.5650000000000001e-06 2024-09-14T06:29:40.807 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:40.807 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:40.807 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:29:40.807 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.658709+0000", 2024-09-14T06:29:40.807 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.8442000000000001e-05 2024-09-14T06:29:40.807 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:40.808 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:40.808 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:29:40.808 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.658730+0000", 2024-09-14T06:29:40.808 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.0509999999999998e-05 2024-09-14T06:29:40.808 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:40.808 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:40.808 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:29:40.808 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.658740+0000", 2024-09-14T06:29:40.808 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.8619999999999998e-06 2024-09-14T06:29:40.808 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:40.808 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:40.808 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:29:40.808 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.658813+0000", 2024-09-14T06:29:40.809 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.3578000000000006e-05 2024-09-14T06:29:40.809 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:40.809 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:40.809 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:29:40.809 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.658836+0000", 2024-09-14T06:29:40.809 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.2677e-05 2024-09-14T06:29:40.809 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:40.809 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:40.809 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-14T06:29:40.809 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.660462+0000", 2024-09-14T06:29:40.809 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00162637 2024-09-14T06:29:40.809 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:40.809 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:40.809 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:29:40.810 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.660512+0000", 2024-09-14T06:29:40.810 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.9863e-05 2024-09-14T06:29:40.810 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:40.810 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:40.810 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:40.810 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:40.810 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:40.810 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4397.0:1941 3.9 e32/23 3:92dc473f:::rbd_data.112dcdddac74.000000000000000c:head v 32'8, mlcod=30'6)", 2024-09-14T06:29:40.810 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:08.664245+0000", 2024-09-14T06:29:40.811 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 32.128546499000002, 2024-09-14T06:29:40.811 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.001294746, 2024-09-14T06:29:40.811 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:29:40.811 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:29:40.811 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:29:40.811 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:29:40.811 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:40.811 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:29:40.811 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.664245+0000", 2024-09-14T06:29:40.811 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:40.811 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:40.811 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:40.811 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:29:40.811 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.664245+0000", 2024-09-14T06:29:40.812 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:40.812 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:40.812 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:40.812 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:29:40.812 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.664245+0000", 2024-09-14T06:29:40.812 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.0999999999999999e-07 2024-09-14T06:29:40.812 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:40.812 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:40.812 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:29:40.812 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.664408+0000", 2024-09-14T06:29:40.813 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00016308800000000001 2024-09-14T06:29:40.813 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:40.813 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:40.813 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:29:40.813 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.664421+0000", 2024-09-14T06:29:40.814 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.3097e-05 2024-09-14T06:29:40.814 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:40.814 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:40.814 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:29:40.814 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.664429+0000", 2024-09-14T06:29:40.814 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.943e-06 2024-09-14T06:29:40.814 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:40.814 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:40.814 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:29:40.814 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.664511+0000", 2024-09-14T06:29:40.815 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.1945000000000002e-05 2024-09-14T06:29:40.815 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:40.815 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:40.815 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:29:40.815 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.664532+0000", 2024-09-14T06:29:40.816 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.0931999999999999e-05 2024-09-14T06:29:40.816 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:40.816 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:40.816 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-14T06:29:40.816 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.665490+0000", 2024-09-14T06:29:40.816 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00095764400000000001 2024-09-14T06:29:40.817 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:40.817 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:40.817 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:29:40.817 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.665539+0000", 2024-09-14T06:29:40.817 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.9687e-05 2024-09-14T06:29:40.818 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:40.818 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:40.818 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:40.818 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:40.818 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:40.818 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4397.0:2199 3.0 e32/23 3:05f3e980:::rbd_data.112dcdddac74.000000000000000f:head v 32'3, mlcod=31'1)", 2024-09-14T06:29:40.818 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:08.680236+0000", 2024-09-14T06:29:40.818 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 32.112555450000002, 2024-09-14T06:29:40.818 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0019068990000000001, 2024-09-14T06:29:40.818 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:29:40.818 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:29:40.818 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:29:40.819 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:29:40.819 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:40.819 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:29:40.819 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.680236+0000", 2024-09-14T06:29:40.819 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:40.820 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:40.820 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:40.820 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:29:40.820 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.680236+0000", 2024-09-14T06:29:40.820 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:40.820 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:40.820 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:40.820 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:29:40.820 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.680237+0000", 2024-09-14T06:29:40.820 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.4980000000000001e-06 2024-09-14T06:29:40.820 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:40.821 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:40.821 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:29:40.821 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.680324+0000", 2024-09-14T06:29:40.821 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.7310000000000003e-05 2024-09-14T06:29:40.821 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:40.822 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:40.822 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:29:40.822 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.680354+0000", 2024-09-14T06:29:40.822 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.97e-05 2024-09-14T06:29:40.822 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:40.822 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:40.822 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:29:40.822 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.680366+0000", 2024-09-14T06:29:40.822 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.2048999999999999e-05 2024-09-14T06:29:40.822 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:40.822 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:40.822 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:29:40.822 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.680439+0000", 2024-09-14T06:29:40.822 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.2910000000000005e-05 2024-09-14T06:29:40.823 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:40.823 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:40.823 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:29:40.823 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.680465+0000", 2024-09-14T06:29:40.823 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.5812999999999998e-05 2024-09-14T06:29:40.823 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:40.823 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:40.823 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-14T06:29:40.823 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.682094+0000", 2024-09-14T06:29:40.823 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.001628876 2024-09-14T06:29:40.823 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:40.823 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:40.823 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:29:40.823 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.682142+0000", 2024-09-14T06:29:40.823 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.8742999999999997e-05 2024-09-14T06:29:40.824 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:40.824 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:40.824 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:40.824 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:40.824 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:40.824 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4397.0:2513 3.0 e32/23 3:06f82ba7:::rbd_data.112dcdddac74.0000000000000028:head v 32'4, mlcod=32'2)", 2024-09-14T06:29:40.824 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:08.703547+0000", 2024-09-14T06:29:40.824 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 32.089243846000002, 2024-09-14T06:29:40.824 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0013041859999999999, 2024-09-14T06:29:40.824 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:29:40.824 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:29:40.824 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:29:40.824 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:29:40.824 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:40.824 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:29:40.825 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.703547+0000", 2024-09-14T06:29:40.825 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:40.825 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:40.825 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:40.825 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:29:40.825 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.703547+0000", 2024-09-14T06:29:40.825 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:40.825 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:40.825 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:40.825 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:29:40.825 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.703549+0000", 2024-09-14T06:29:40.825 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.39e-06 2024-09-14T06:29:40.825 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:40.825 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:40.826 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:29:40.826 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.703666+0000", 2024-09-14T06:29:40.826 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000117314 2024-09-14T06:29:40.826 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:40.826 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:40.826 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:29:40.826 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.703685+0000", 2024-09-14T06:29:40.826 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.8984999999999999e-05 2024-09-14T06:29:40.826 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:40.826 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:40.826 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:29:40.826 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.703697+0000", 2024-09-14T06:29:40.826 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.2405000000000001e-05 2024-09-14T06:29:40.826 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:40.826 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:40.827 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:29:40.827 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.703765+0000", 2024-09-14T06:29:40.827 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.7595000000000006e-05 2024-09-14T06:29:40.827 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:40.827 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:40.827 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:29:40.827 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.703788+0000", 2024-09-14T06:29:40.827 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.3300000000000001e-05 2024-09-14T06:29:40.827 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:40.827 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:40.827 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-14T06:29:40.827 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.704801+0000", 2024-09-14T06:29:40.827 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0010130339999999999 2024-09-14T06:29:40.827 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:40.827 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:40.828 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:29:40.828 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.704851+0000", 2024-09-14T06:29:40.828 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.0163000000000001e-05 2024-09-14T06:29:40.828 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:40.828 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:40.828 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:40.828 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:40.828 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:40.828 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4397.0:2885 3.9 e32/23 3:974aae1b:::rbd_data.112dcdddac74.000000000000002d:head v 32'9, mlcod=30'7)", 2024-09-14T06:29:40.828 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:08.717100+0000", 2024-09-14T06:29:40.828 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 32.075691231999997, 2024-09-14T06:29:40.828 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0012528229999999999, 2024-09-14T06:29:40.828 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:29:40.829 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:29:40.829 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:29:40.829 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:29:40.829 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:40.829 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:29:40.829 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.717100+0000", 2024-09-14T06:29:40.829 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:40.829 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:40.829 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:40.829 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:29:40.829 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.717100+0000", 2024-09-14T06:29:40.829 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:40.829 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:40.830 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:40.830 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:29:40.830 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.717101+0000", 2024-09-14T06:29:40.830 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.2929999999999999e-06 2024-09-14T06:29:40.830 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:40.830 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:40.830 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:29:40.830 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.717270+0000", 2024-09-14T06:29:40.830 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00016929799999999999 2024-09-14T06:29:40.830 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:40.830 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:40.830 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:29:40.830 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.717283+0000", 2024-09-14T06:29:40.831 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.2483999999999999e-05 2024-09-14T06:29:40.831 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:40.831 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:40.831 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:29:40.831 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.717292+0000", 2024-09-14T06:29:40.831 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.7220000000000005e-06 2024-09-14T06:29:40.831 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:40.831 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:40.831 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:29:40.831 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.717356+0000", 2024-09-14T06:29:40.831 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.4055999999999998e-05 2024-09-14T06:29:40.831 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:40.832 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:40.832 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:29:40.832 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.717383+0000", 2024-09-14T06:29:40.832 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.7274999999999999e-05 2024-09-14T06:29:40.832 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:40.832 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:40.832 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-14T06:29:40.832 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.718318+0000", 2024-09-14T06:29:40.832 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00093470100000000002 2024-09-14T06:29:40.832 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:40.832 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:40.833 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:29:40.833 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.718353+0000", 2024-09-14T06:29:40.833 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.4993999999999997e-05 2024-09-14T06:29:40.833 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:40.833 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:40.833 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:40.833 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:40.833 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:40.833 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4397.0:3292 3.9 e32/23 3:91dbce57:::rbd_data.112dcdddac74.0000000000000036:head v 32'10, mlcod=32'8)", 2024-09-14T06:29:40.833 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:08.740506+0000", 2024-09-14T06:29:40.833 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 32.052284737999997, 2024-09-14T06:29:40.833 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.001864166, 2024-09-14T06:29:40.833 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:29:40.834 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:29:40.834 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:29:40.834 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:29:40.834 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:40.834 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:29:40.834 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.740506+0000", 2024-09-14T06:29:40.834 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:40.834 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:40.834 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:40.834 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:29:40.834 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.740506+0000", 2024-09-14T06:29:40.834 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:40.834 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:40.835 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:40.835 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:29:40.835 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.740514+0000", 2024-09-14T06:29:40.835 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.2109999999999998e-06 2024-09-14T06:29:40.835 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:40.835 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:40.835 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:29:40.835 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.740745+0000", 2024-09-14T06:29:40.835 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000230194 2024-09-14T06:29:40.835 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:40.836 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:40.836 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:29:40.836 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.740764+0000", 2024-09-14T06:29:40.836 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.9330999999999998e-05 2024-09-14T06:29:40.836 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:40.836 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:40.836 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:29:40.836 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.740774+0000", 2024-09-14T06:29:40.836 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.6260000000000007e-06 2024-09-14T06:29:40.837 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:40.837 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:40.837 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:29:40.837 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.740839+0000", 2024-09-14T06:29:40.837 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.5314000000000005e-05 2024-09-14T06:29:40.837 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:40.837 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:40.837 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:29:40.837 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.740860+0000", 2024-09-14T06:29:40.837 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.0868000000000001e-05 2024-09-14T06:29:40.837 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:40.837 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:40.838 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-14T06:29:40.838 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.742327+0000", 2024-09-14T06:29:40.838 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0014672909999999999 2024-09-14T06:29:40.838 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:40.838 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:40.838 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:29:40.838 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.742370+0000", 2024-09-14T06:29:40.838 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.3331000000000002e-05 2024-09-14T06:29:40.838 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:40.838 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:40.838 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:40.838 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:40.838 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:40.838 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4397.0:8527 3.0 e46/23 3:05c39535:::rbd_data.112d92027590.0000000000000009:head v 46'11, mlcod=33'9)", 2024-09-14T06:29:40.839 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:22.617134+0000", 2024-09-14T06:29:40.839 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 18.175657018999999, 2024-09-14T06:29:40.839 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0010939770000000001, 2024-09-14T06:29:40.839 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:29:40.839 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:29:40.839 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:29:40.839 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:29:40.839 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:40.839 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:29:40.839 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:22.617134+0000", 2024-09-14T06:29:40.839 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:40.840 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:40.840 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:40.840 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:29:40.840 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:22.617134+0000", 2024-09-14T06:29:40.840 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:40.840 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:40.840 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:40.840 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:29:40.840 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:22.617134+0000", 2024-09-14T06:29:40.840 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.89e-07 2024-09-14T06:29:40.840 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:40.840 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:40.840 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:29:40.840 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:22.617189+0000", 2024-09-14T06:29:40.840 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.4415e-05 2024-09-14T06:29:40.841 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:40.841 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:40.841 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:29:40.841 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:22.617197+0000", 2024-09-14T06:29:40.841 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.5750000000000001e-06 2024-09-14T06:29:40.841 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:40.841 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:40.841 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:29:40.841 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:22.617205+0000", 2024-09-14T06:29:40.841 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.6029999999999999e-06 2024-09-14T06:29:40.841 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:40.842 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:40.842 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:29:40.842 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:22.617268+0000", 2024-09-14T06:29:40.842 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.2568000000000001e-05 2024-09-14T06:29:40.842 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:40.842 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:40.842 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:29:40.842 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:22.617303+0000", 2024-09-14T06:29:40.842 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.5259000000000002e-05 2024-09-14T06:29:40.842 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:40.842 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:40.843 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-14T06:29:40.843 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:22.618188+0000", 2024-09-14T06:29:40.843 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00088557900000000005 2024-09-14T06:29:40.843 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:40.843 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:40.843 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:29:40.843 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:22.618228+0000", 2024-09-14T06:29:40.843 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.9489000000000001e-05 2024-09-14T06:29:40.843 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:40.844 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:40.844 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:40.844 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:40.844 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:40.844 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4397.0:12132 3.4 e49/23 3:27ed714d:::rbd_data.112d8f603570.0000000000000003:head v 49'47, mlcod=33'45)", 2024-09-14T06:29:40.844 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:25.671949+0000", 2024-09-14T06:29:40.844 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 15.120842306, 2024-09-14T06:29:40.844 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00131298, 2024-09-14T06:29:40.844 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:29:40.844 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:29:40.845 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:29:40.845 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:29:40.845 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:40.845 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:29:40.845 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.671949+0000", 2024-09-14T06:29:40.845 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:40.845 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:40.845 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:40.845 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:29:40.845 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.671949+0000", 2024-09-14T06:29:40.845 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:40.845 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:40.845 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:40.846 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:29:40.846 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.671949+0000", 2024-09-14T06:29:40.846 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.0500000000000004e-07 2024-09-14T06:29:40.846 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:40.846 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:40.846 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:29:40.846 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.672087+0000", 2024-09-14T06:29:40.846 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00013826999999999999 2024-09-14T06:29:40.846 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:40.846 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:40.847 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:29:40.847 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.672098+0000", 2024-09-14T06:29:40.847 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.0753e-05 2024-09-14T06:29:40.847 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:40.847 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:40.847 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:29:40.847 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.672105+0000", 2024-09-14T06:29:40.847 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.613e-06 2024-09-14T06:29:40.847 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:40.847 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:40.847 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:29:40.848 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.672178+0000", 2024-09-14T06:29:40.848 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.3131000000000002e-05 2024-09-14T06:29:40.848 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:40.848 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:40.848 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:29:40.848 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.672202+0000", 2024-09-14T06:29:40.848 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.4301e-05 2024-09-14T06:29:40.848 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:40.848 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:40.849 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-14T06:29:40.849 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.673207+0000", 2024-09-14T06:29:40.849 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0010045779999999999 2024-09-14T06:29:40.849 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:40.849 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:40.849 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:29:40.849 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.673262+0000", 2024-09-14T06:29:40.849 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.4829e-05 2024-09-14T06:29:40.849 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:40.849 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:40.849 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:40.849 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:40.849 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:40.849 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4397.0:16705 3.9 e49/23 3:91acaef1:::rbd_data.112d8f603570.0000000000000067:head v 49'20, mlcod=49'18)", 2024-09-14T06:29:40.850 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:26.198172+0000", 2024-09-14T06:29:40.850 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 14.594619455, 2024-09-14T06:29:40.850 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00133445, 2024-09-14T06:29:40.850 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:29:40.850 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:29:40.850 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:29:40.850 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:29:40.850 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:40.850 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:29:40.850 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.198172+0000", 2024-09-14T06:29:40.851 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:40.851 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:40.851 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:40.851 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:29:40.851 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.198172+0000", 2024-09-14T06:29:40.851 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:40.851 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:40.851 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:40.851 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:29:40.851 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.198173+0000", 2024-09-14T06:29:40.851 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.4529999999999999e-06 2024-09-14T06:29:40.851 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:40.851 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:40.851 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:29:40.851 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.198318+0000", 2024-09-14T06:29:40.852 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00014472500000000001 2024-09-14T06:29:40.852 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:40.852 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:40.852 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:29:40.852 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.198333+0000", 2024-09-14T06:29:40.852 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.5420000000000001e-05 2024-09-14T06:29:40.852 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:40.852 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:40.852 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:29:40.852 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.198341+0000", 2024-09-14T06:29:40.852 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.2910000000000007e-06 2024-09-14T06:29:40.852 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:40.853 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:40.853 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:29:40.853 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.198403+0000", 2024-09-14T06:29:40.853 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.1985999999999999e-05 2024-09-14T06:29:40.853 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:40.853 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:40.853 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:29:40.853 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.198427+0000", 2024-09-14T06:29:40.853 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.3204e-05 2024-09-14T06:29:40.853 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:40.853 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:40.853 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-14T06:29:40.853 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.199458+0000", 2024-09-14T06:29:40.853 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0010313620000000001 2024-09-14T06:29:40.854 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:40.854 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:40.854 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:29:40.854 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.199506+0000", 2024-09-14T06:29:40.854 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.8009000000000001e-05 2024-09-14T06:29:40.854 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:40.854 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:40.854 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:40.854 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:40.854 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:40.854 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4397.0:21131 3.0 e49/23 3:04cbb647:::rbd_data.112d8f603570.00000000000001e2:head v 49'14, mlcod=49'12)", 2024-09-14T06:29:40.854 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:27.413180+0000", 2024-09-14T06:29:40.854 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 13.379611133999999, 2024-09-14T06:29:40.854 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0010618019999999999, 2024-09-14T06:29:40.855 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:29:40.855 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:29:40.855 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:29:40.855 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:29:40.855 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:40.855 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:29:40.855 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:27.413180+0000", 2024-09-14T06:29:40.855 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:40.855 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:40.855 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:40.855 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:29:40.855 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:27.413180+0000", 2024-09-14T06:29:40.855 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:40.855 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:40.856 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:40.856 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:29:40.856 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:27.413181+0000", 2024-09-14T06:29:40.856 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.279e-06 2024-09-14T06:29:40.856 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:40.856 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:40.856 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:29:40.856 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:27.413255+0000", 2024-09-14T06:29:40.856 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.3505999999999997e-05 2024-09-14T06:29:40.856 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:40.856 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:40.856 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:29:40.857 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:27.413272+0000", 2024-09-14T06:29:40.857 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.6973000000000002e-05 2024-09-14T06:29:40.857 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:40.857 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:40.857 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:29:40.857 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:27.413280+0000", 2024-09-14T06:29:40.857 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.9459999999999998e-06 2024-09-14T06:29:40.857 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:40.857 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:40.857 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:29:40.857 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:27.413348+0000", 2024-09-14T06:29:40.857 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.8490000000000003e-05 2024-09-14T06:29:40.857 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:40.858 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:40.858 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:29:40.858 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:27.413377+0000", 2024-09-14T06:29:40.858 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.9161000000000001e-05 2024-09-14T06:29:40.858 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:40.858 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:40.858 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-14T06:29:40.858 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:27.414190+0000", 2024-09-14T06:29:40.858 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00081261599999999995 2024-09-14T06:29:40.858 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:40.858 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:40.858 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:29:40.858 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:27.414242+0000", 2024-09-14T06:29:40.859 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.1830999999999999e-05 2024-09-14T06:29:40.859 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:40.859 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:40.859 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:40.859 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:40.859 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:40.859 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4397.0:26415 3.0 e53/23 3:05fd9be0:::rbd_data.112d38768680.0000000000000006:head v 53'21, mlcod=50'19)", 2024-09-14T06:29:40.859 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:30.721610+0000", 2024-09-14T06:29:40.859 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 10.071180669, 2024-09-14T06:29:40.859 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0021763160000000002, 2024-09-14T06:29:40.859 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:29:40.859 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:29:40.860 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:29:40.860 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:29:40.860 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:40.860 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:29:40.860 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:30.721610+0000", 2024-09-14T06:29:40.860 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:40.860 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:40.860 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:40.860 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:29:40.860 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:30.721610+0000", 2024-09-14T06:29:40.860 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:40.860 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:40.860 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:40.861 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:29:40.861 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:30.721611+0000", 2024-09-14T06:29:40.861 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.41e-07 2024-09-14T06:29:40.861 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:40.861 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:40.861 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:29:40.861 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:30.721905+0000", 2024-09-14T06:29:40.861 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00029394300000000001 2024-09-14T06:29:40.861 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:40.861 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:40.861 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:29:40.861 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:30.721921+0000", 2024-09-14T06:29:40.862 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.6609999999999999e-05 2024-09-14T06:29:40.862 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:40.862 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:40.862 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:29:40.862 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:30.721931+0000", 2024-09-14T06:29:40.862 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.6739999999999992e-06 2024-09-14T06:29:40.862 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:40.862 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:40.862 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:29:40.862 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:30.722001+0000", 2024-09-14T06:29:40.862 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.0549000000000005e-05 2024-09-14T06:29:40.862 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:40.862 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:40.862 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:29:40.862 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:30.722029+0000", 2024-09-14T06:29:40.863 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.7327000000000001e-05 2024-09-14T06:29:40.863 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:40.863 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:40.863 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-14T06:29:40.863 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:30.723727+0000", 2024-09-14T06:29:40.863 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0016977310000000001 2024-09-14T06:29:40.863 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:40.863 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:40.863 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:29:40.863 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:30.723787+0000", 2024-09-14T06:29:40.863 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.0140999999999999e-05 2024-09-14T06:29:40.863 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:40.863 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:40.863 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:40.863 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:40.864 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:40.864 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4397.0:36084 3.0 e56/23 3:05a48ef4:::rbd_data.112dc58639fa.0000000000000058:head v 56'29, mlcod=56'27)", 2024-09-14T06:29:40.864 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:34.508855+0000", 2024-09-14T06:29:40.864 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 6.2839359149999998, 2024-09-14T06:29:40.864 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0020801140000000001, 2024-09-14T06:29:40.864 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:29:40.864 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:29:40.864 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:29:40.864 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:29:40.864 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:40.864 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:29:40.864 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:34.508855+0000", 2024-09-14T06:29:40.864 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:40.864 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:40.865 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:40.865 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:29:40.865 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:34.508855+0000", 2024-09-14T06:29:40.865 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:40.865 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:40.865 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:40.865 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:29:40.865 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:34.508856+0000", 2024-09-14T06:29:40.865 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.1909999999999999e-06 2024-09-14T06:29:40.865 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:40.865 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:40.865 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:29:40.865 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:34.509203+0000", 2024-09-14T06:29:40.866 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00034625900000000002 2024-09-14T06:29:40.866 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:40.866 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:40.866 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:29:40.866 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:34.509218+0000", 2024-09-14T06:29:40.866 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.5091e-05 2024-09-14T06:29:40.866 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:40.866 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:40.866 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:29:40.866 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:34.509227+0000", 2024-09-14T06:29:40.866 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.0850000000000003e-06 2024-09-14T06:29:40.866 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:40.866 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:40.866 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:29:40.867 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:34.509311+0000", 2024-09-14T06:29:40.867 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.4436000000000003e-05 2024-09-14T06:29:40.867 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:40.867 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:40.867 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:29:40.867 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:34.509335+0000", 2024-09-14T06:29:40.867 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.3577000000000001e-05 2024-09-14T06:29:40.867 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:40.867 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:40.867 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-14T06:29:40.867 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:34.510890+0000", 2024-09-14T06:29:40.867 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0015554259999999999 2024-09-14T06:29:40.867 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:40.867 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:40.868 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:29:40.868 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:34.510935+0000", 2024-09-14T06:29:40.868 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.5049000000000002e-05 2024-09-14T06:29:40.868 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:40.868 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:40.868 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:40.868 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:40.868 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:40.868 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4397.0:36966 3.4 e56/23 3:22008318:::rbd_data.112dc58639fa.0000000000000066:head v 56'56, mlcod=50'54)", 2024-09-14T06:29:40.868 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:34.565614+0000", 2024-09-14T06:29:40.868 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 6.2271774789999998, 2024-09-14T06:29:40.868 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0018691929999999999, 2024-09-14T06:29:40.868 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:29:40.868 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:29:40.869 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:29:40.869 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:29:40.869 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:40.869 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:29:40.869 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:34.565614+0000", 2024-09-14T06:29:40.869 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:40.869 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:40.869 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:40.869 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:29:40.869 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:34.565614+0000", 2024-09-14T06:29:40.869 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:40.869 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:40.869 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:40.870 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:29:40.870 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:34.565615+0000", 2024-09-14T06:29:40.870 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.784e-06 2024-09-14T06:29:40.870 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:40.870 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:40.870 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:29:40.870 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:34.565919+0000", 2024-09-14T06:29:40.870 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00030351599999999999 2024-09-14T06:29:40.870 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:40.870 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:40.870 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:29:40.870 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:34.565933+0000", 2024-09-14T06:29:40.870 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.4465e-05 2024-09-14T06:29:40.870 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:40.870 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:40.871 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:29:40.871 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:34.565943+0000", 2024-09-14T06:29:40.871 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.7920000000000004e-06 2024-09-14T06:29:40.871 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:40.871 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:40.871 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:29:40.871 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:34.565999+0000", 2024-09-14T06:29:40.871 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.5643000000000002e-05 2024-09-14T06:29:40.871 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:40.871 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:40.871 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:29:40.871 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:34.566025+0000", 2024-09-14T06:29:40.871 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.6040000000000001e-05 2024-09-14T06:29:40.871 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:40.871 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:40.872 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-14T06:29:40.872 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:34.567430+0000", 2024-09-14T06:29:40.872 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.001405653 2024-09-14T06:29:40.872 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:40.872 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:40.872 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:29:40.872 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:34.567483+0000", 2024-09-14T06:29:40.872 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.2299999999999997e-05 2024-09-14T06:29:40.872 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:40.872 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:40.872 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:40.872 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:40.872 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:40.872 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4397.0:38274 3.4 e57/23 3:25bceb98:::rbd_data.112dc58639fa.000000000000003b:head v 57'57, mlcod=56'55)", 2024-09-14T06:29:40.873 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:35.133930+0000", 2024-09-14T06:29:40.873 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 5.6588605479999998, 2024-09-14T06:29:40.873 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0011095949999999999, 2024-09-14T06:29:40.873 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:29:40.873 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:29:40.873 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:29:40.873 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:29:40.873 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:40.873 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:29:40.873 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:35.133930+0000", 2024-09-14T06:29:40.873 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:40.873 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:40.873 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:40.874 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:29:40.874 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:35.133930+0000", 2024-09-14T06:29:40.874 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:40.874 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:40.874 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:40.874 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:29:40.874 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:35.133931+0000", 2024-09-14T06:29:40.874 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.09e-07 2024-09-14T06:29:40.874 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:40.874 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:40.874 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:29:40.874 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:35.133938+0000", 2024-09-14T06:29:40.874 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.9870000000000002e-06 2024-09-14T06:29:40.874 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:40.874 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:40.875 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:29:40.875 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:35.133951+0000", 2024-09-14T06:29:40.875 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.2819999999999999e-05 2024-09-14T06:29:40.875 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:40.875 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:40.875 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:29:40.875 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:35.133959+0000", 2024-09-14T06:29:40.875 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.1769999999999998e-06 2024-09-14T06:29:40.875 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:40.875 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:40.875 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:29:40.875 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:35.134058+0000", 2024-09-14T06:29:40.875 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.9100999999999999e-05 2024-09-14T06:29:40.875 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:40.876 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:40.876 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:29:40.876 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:35.134080+0000", 2024-09-14T06:29:40.876 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.1195e-05 2024-09-14T06:29:40.876 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:40.876 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:40.876 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-14T06:29:40.876 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:35.135000+0000", 2024-09-14T06:29:40.876 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00092012400000000003 2024-09-14T06:29:40.876 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:40.876 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:40.876 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:29:40.876 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:35.135040+0000", 2024-09-14T06:29:40.876 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.0382000000000001e-05 2024-09-14T06:29:40.876 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:40.877 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:40.877 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:40.877 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:40.877 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:40.877 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4397.0:38285 3.0 e57/23 3:071b81bb:::rbd_data.112dc58639fa.0000000000000046:head v 57'31, mlcod=56'28)", 2024-09-14T06:29:40.877 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:35.138648+0000", 2024-09-14T06:29:40.877 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 5.6541431019999999, 2024-09-14T06:29:40.877 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00124873, 2024-09-14T06:29:40.877 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:29:40.877 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:29:40.877 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:29:40.878 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:29:40.878 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:40.878 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:29:40.878 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:35.138648+0000", 2024-09-14T06:29:40.878 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:40.878 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:40.878 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:40.878 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:29:40.878 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:35.138648+0000", 2024-09-14T06:29:40.878 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:40.878 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:40.878 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:40.879 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:29:40.879 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:35.138649+0000", 2024-09-14T06:29:40.879 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.6599999999999996e-07 2024-09-14T06:29:40.879 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:40.879 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:40.879 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:29:40.879 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:35.138654+0000", 2024-09-14T06:29:40.879 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.3940000000000004e-06 2024-09-14T06:29:40.879 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:40.879 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:40.879 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:29:40.879 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:35.138666+0000", 2024-09-14T06:29:40.879 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.1965e-05 2024-09-14T06:29:40.879 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:40.880 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:40.880 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:29:40.880 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:35.138673+0000", 2024-09-14T06:29:40.880 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.9689999999999997e-06 2024-09-14T06:29:40.880 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:40.880 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:40.880 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:29:40.880 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:35.138753+0000", 2024-09-14T06:29:40.880 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.0214000000000001e-05 2024-09-14T06:29:40.880 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:40.880 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:40.880 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:29:40.880 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:35.138781+0000", 2024-09-14T06:29:40.880 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.7702999999999999e-05 2024-09-14T06:29:40.881 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:40.881 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:40.881 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-14T06:29:40.881 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:35.139848+0000", 2024-09-14T06:29:40.881 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.001066883 2024-09-14T06:29:40.881 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:40.881 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:40.881 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:29:40.881 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:35.139897+0000", 2024-09-14T06:29:40.881 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.8936e-05 2024-09-14T06:29:40.881 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:40.881 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:40.881 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:40.881 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:40.881 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:40.882 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4397.0:38301 3.0 e57/23 3:0192c1a2:::rbd_data.112dc58639fa.0000000000000056:head v 57'33, mlcod=57'30)", 2024-09-14T06:29:40.882 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:35.142547+0000", 2024-09-14T06:29:40.882 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 5.6502444860000001, 2024-09-14T06:29:40.882 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.001107823, 2024-09-14T06:29:40.882 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:29:40.882 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:29:40.882 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:29:40.882 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:29:40.882 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:40.882 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:29:40.882 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:35.142547+0000", 2024-09-14T06:29:40.882 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:40.882 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:40.882 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:40.882 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:29:40.883 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:35.142547+0000", 2024-09-14T06:29:40.883 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:40.883 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:40.883 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:40.883 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:29:40.883 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:35.142547+0000", 2024-09-14T06:29:40.883 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.7599999999999997e-07 2024-09-14T06:29:40.883 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:40.884 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:40.884 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:29:40.884 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:35.142559+0000", 2024-09-14T06:29:40.884 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.2051e-05 2024-09-14T06:29:40.884 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:40.884 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:40.884 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:29:40.885 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:35.142573+0000", 2024-09-14T06:29:40.885 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.3696000000000001e-05 2024-09-14T06:29:40.885 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:40.885 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:40.885 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:29:40.885 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:35.142580+0000", 2024-09-14T06:29:40.885 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.8650000000000003e-06 2024-09-14T06:29:40.885 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:40.885 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:40.885 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:29:40.886 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:35.142636+0000", 2024-09-14T06:29:40.886 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.6246000000000003e-05 2024-09-14T06:29:40.886 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:40.886 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:40.886 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:29:40.887 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:35.142649+0000", 2024-09-14T06:29:40.887 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.3485e-05 2024-09-14T06:29:40.887 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:40.887 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:40.887 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-14T06:29:40.887 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:35.143623+0000", 2024-09-14T06:29:40.888 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00097405500000000004 2024-09-14T06:29:40.888 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:40.888 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:40.888 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:29:40.888 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:35.143654+0000", 2024-09-14T06:29:40.888 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.0948999999999998e-05 2024-09-14T06:29:40.888 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:40.889 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:40.889 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:40.889 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:40.889 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:40.889 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4397.0:38339 3.9 e57/23 3:9467a21d:::rbd_data.112dc58639fa.0000000000000003:head v 57'30, mlcod=56'28)", 2024-09-14T06:29:40.889 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:35.349820+0000", 2024-09-14T06:29:40.889 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 5.4429714850000002, 2024-09-14T06:29:40.889 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.001310702, 2024-09-14T06:29:40.889 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:29:40.890 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:29:40.890 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:29:40.890 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:29:40.890 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:40.890 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:29:40.890 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:35.349820+0000", 2024-09-14T06:29:40.890 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:40.890 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:40.890 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:40.890 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:29:40.891 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:35.349820+0000", 2024-09-14T06:29:40.891 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:40.891 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:40.891 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:40.891 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:29:40.891 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:35.349820+0000", 2024-09-14T06:29:40.891 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.2799999999999995e-07 2024-09-14T06:29:40.891 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:40.891 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:40.891 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:29:40.891 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:35.349826+0000", 2024-09-14T06:29:40.891 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.0179999999999998e-06 2024-09-14T06:29:40.891 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:40.891 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:40.891 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:29:40.892 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:35.349838+0000", 2024-09-14T06:29:40.892 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.172e-05 2024-09-14T06:29:40.892 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:40.892 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:40.892 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:29:40.892 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:35.349847+0000", 2024-09-14T06:29:40.892 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.8030000000000003e-06 2024-09-14T06:29:40.892 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:40.892 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:40.892 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:29:40.892 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:35.350052+0000", 2024-09-14T06:29:40.892 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000204973 2024-09-14T06:29:40.892 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:40.892 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:40.893 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:29:40.893 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:35.350083+0000", 2024-09-14T06:29:40.893 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.1578999999999997e-05 2024-09-14T06:29:40.893 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:40.893 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:40.894 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-14T06:29:40.894 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:35.351039+0000", 2024-09-14T06:29:40.894 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00095594799999999998 2024-09-14T06:29:40.894 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:40.894 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:40.894 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:29:40.894 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:35.351130+0000", 2024-09-14T06:29:40.894 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.0833000000000003e-05 2024-09-14T06:29:40.894 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:40.894 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:40.895 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:40.895 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:40.895 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:40.895 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4397.0:42144 3.9 e61/23 3:90747014:::rbd_data.112dca2f6ae1.000000000000002f:head v 61'51, mlcod=61'49)", 2024-09-14T06:29:40.895 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:40.570503+0000", 2024-09-14T06:29:40.895 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 0.22228841299999999, 2024-09-14T06:29:40.895 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.001106276, 2024-09-14T06:29:40.895 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:29:40.895 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:29:40.895 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:29:40.895 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:29:40.895 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:40.895 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:29:40.895 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:40.570503+0000", 2024-09-14T06:29:40.895 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:40.895 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:40.896 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:40.896 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:29:40.896 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:40.570503+0000", 2024-09-14T06:29:40.896 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:40.896 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:40.896 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:40.896 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:29:40.896 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:40.570504+0000", 2024-09-14T06:29:40.896 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.5300000000000002e-07 2024-09-14T06:29:40.896 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:40.896 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:40.896 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:29:40.896 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:40.570521+0000", 2024-09-14T06:29:40.896 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.7716000000000001e-05 2024-09-14T06:29:40.896 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:40.896 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:40.897 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:29:40.897 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:40.570533+0000", 2024-09-14T06:29:40.897 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.1549000000000001e-05 2024-09-14T06:29:40.897 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:40.897 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:40.897 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:29:40.897 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:40.570542+0000", 2024-09-14T06:29:40.897 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.2420000000000006e-06 2024-09-14T06:29:40.897 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:40.897 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:40.897 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:29:40.897 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:40.570599+0000", 2024-09-14T06:29:40.897 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.6588000000000001e-05 2024-09-14T06:29:40.898 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:40.898 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:40.898 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:29:40.898 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:40.570627+0000", 2024-09-14T06:29:40.898 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.8632999999999998e-05 2024-09-14T06:29:40.898 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:40.898 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:40.898 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-14T06:29:40.898 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:40.571564+0000", 2024-09-14T06:29:40.898 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00093674100000000005 2024-09-14T06:29:40.898 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:40.898 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:40.898 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:29:40.898 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:40.571609+0000", 2024-09-14T06:29:40.898 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.4854000000000001e-05 2024-09-14T06:29:40.899 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:40.899 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:40.899 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:40.899 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:40.899 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:40.899 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-14T06:29:40.900 INFO:teuthology.orchestra.run.smithi094.stdout:1182 write 0xc91fe69 thru 0xc92d00c (0xd1a4 bytes) 2024-09-14T06:29:40.900 INFO:teuthology.orchestra.run.smithi094.stdout:1183 read 0xb90f0d6 thru 0xb910967 (0x1892 bytes) 2024-09-14T06:29:40.901 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-14T06:29:40.902 INFO:teuthology.orchestra.run.smithi094.stdout:1185 trunc from 0xc92d00d to 0xcef29a 2024-09-14T06:29:40.902 INFO:teuthology.orchestra.run.smithi094.stdout:1186 write 0xb2784c0 thru 0xb2852c3 (0xce04 bytes) 2024-09-14T06:29:40.902 INFO:teuthology.orchestra.run.smithi094.stdout:1187 read 0x795aef thru 0x79e231 (0x8743 bytes) 2024-09-14T06:29:40.902 INFO:teuthology.orchestra.run.smithi094.stdout:1190 write 0x64ad738 thru 0x64b474f (0x7018 bytes) 2024-09-14T06:29:40.902 INFO:teuthology.orchestra.run.smithi094.stdout:1192 read 0x20b90d8 thru 0x20b95b5 (0x4de bytes) 2024-09-14T06:29:40.903 INFO:teuthology.orchestra.run.smithi094.stdout:1193 write 0xa857931 thru 0xa859662 (0x1d32 bytes) 2024-09-14T06:29:40.903 INFO:teuthology.orchestra.run.smithi094.stdout:1194 writesame 0xaf59d64 thru 0xaf5ba5a data_size 0x1(0x1cf7 bytes) 2024-09-14T06:29:40.903 INFO:teuthology.orchestra.run.smithi094.stdout:1195 write 0x32855d6 thru 0x3286233 (0xc5e bytes) 2024-09-14T06:29:40.903 INFO:teuthology.orchestra.run.smithi094.stdout:1196 write 0xda2026f thru 0xda2edea (0xeb7c bytes) 2024-09-14T06:29:40.903 INFO:teuthology.orchestra.run.smithi094.stdout:1197 read 0x921d5dd thru 0x9227526 (0x9f4a bytes) 2024-09-14T06:29:40.903 INFO:teuthology.orchestra.run.smithi094.stdout:1198 trunc from 0xda2edeb to 0x6c5ce51 2024-09-14T06:29:40.903 INFO:tasks.ceph.osd.2.smithi094.stderr:2024-09-14T06:29:40.899+0000 7faf3269b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33649) UID: 0 2024-09-14T06:29:40.906 INFO:teuthology.orchestra.run.smithi094.stdout:1200 write 0xe81ea84 thru 0xe8255ef (0x6b6c bytes) 2024-09-14T06:29:40.908 INFO:teuthology.orchestra.run.smithi094.stdout:1201 write 0x95e401 thru 0x969e04 (0xba04 bytes) 2024-09-14T06:29:40.911 INFO:teuthology.orchestra.run.smithi094.stdout:1203 read 0x6308e35 thru 0x6311472 (0x863e bytes) 2024-09-14T06:29:40.911 INFO:teuthology.orchestra.run.smithi094.stdout:1204 read 0x7033d29 thru 0x7035376 (0x164e bytes) 2024-09-14T06:29:40.925 INFO:teuthology.orchestra.run.smithi094.stdout:1205 trunc from 0xe8255f0 to 0x2f0e176 2024-09-14T06:29:40.938 INFO:teuthology.orchestra.run.smithi094.stdout:1206 write 0x1c22412 thru 0x1c27a8c (0x567b bytes) 2024-09-14T06:29:40.940 INFO:teuthology.orchestra.run.smithi094.stdout:1207 read 0x937a1e thru 0x93c07f (0x4662 bytes) 2024-09-14T06:29:40.940 INFO:teuthology.orchestra.run.smithi094.stdout:1209 read 0x29a2d46 thru 0x29a517c (0x2437 bytes) 2024-09-14T06:29:40.947 INFO:teuthology.orchestra.run.smithi094.stdout:1211 writesame 0x4213f3f thru 0x421c0c2 data_size 0x36(0x8184 bytes) 2024-09-14T06:29:40.956 INFO:teuthology.orchestra.run.smithi094.stdout:1212 write 0x6ee34ab thru 0x6eea1d5 (0x6d2b bytes) 2024-09-14T06:29:40.959 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-14T06:29:40.962 INFO:teuthology.orchestra.run.smithi094.stdout:1214 trunc from 0x6eea1d6 to 0x47f95d5 2024-09-14T06:29:40.971 INFO:teuthology.orchestra.run.smithi094.stdout:1216 punch from 0x34e302e to 0x34e5c4f, (0x2c21 bytes) 2024-09-14T06:29:40.983 INFO:teuthology.orchestra.run.smithi094.stdout:1217 write 0x97c6c78 thru 0x97d00d6 (0x945f bytes) 2024-09-14T06:29:40.993 INFO:teuthology.orchestra.run.smithi094.stdout:1218 writesame 0xa0a9774 thru 0xa0b7c5c data_size 0x1(0xe4e9 bytes) 2024-09-14T06:29:40.993 INFO:teuthology.orchestra.run.smithi094.stdout:1219 writesame 0x2b4e545 thru 0x2b5c61f data_size 0x1(0xe0db bytes) 2024-09-14T06:29:41.001 INFO:tasks.ceph.osd.3.smithi094.stderr:2024-09-14T06:29:41.000+0000 7feeeafe8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33655) UID: 0 2024-09-14T06:29:41.020 INFO:teuthology.orchestra.run.smithi094.stdout:1220 trunc from 0xa0b7c5d to 0x88ffa0f 2024-09-14T06:29:41.020 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-14T06:29:41.020 INFO:teuthology.orchestra.run.smithi094.stdout: "ops": [], 2024-09-14T06:29:41.020 INFO:teuthology.orchestra.run.smithi094.stdout: "num_ops": 0 2024-09-14T06:29:41.020 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-14T06:29:41.030 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-14T06:29:41.031 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-14T06:29:41.031 INFO:teuthology.orchestra.run.smithi094.stdout:1222 write 0x6cee007 thru 0x6cf1237 (0x3231 bytes) 2024-09-14T06:29:41.034 INFO:teuthology.orchestra.run.smithi094.stdout:1224 trunc from 0x88ffa0f to 0x9c7fadb 2024-09-14T06:29:41.041 INFO:teuthology.orchestra.run.smithi094.stdout:1227 trunc from 0x9c7fadb to 0xa3655c6 2024-09-14T06:29:41.046 INFO:teuthology.orchestra.run.smithi094.stdout:1229 read 0x995a208 thru 0x996225f (0x8058 bytes) 2024-09-14T06:29:41.046 INFO:teuthology.orchestra.run.smithi094.stdout:1231 write 0x5f864e thru 0x5f8d0e (0x6c1 bytes) 2024-09-14T06:29:41.047 INFO:teuthology.orchestra.run.smithi094.stdout:1232 read 0x6d15f7e thru 0x6d1ffa8 (0xa02b bytes) 2024-09-14T06:29:41.047 INFO:teuthology.orchestra.run.smithi094.stdout:1234 writesame 0x8a33568 thru 0x8a34298 data_size 0x1(0xd31 bytes) 2024-09-14T06:29:41.051 INFO:teuthology.orchestra.run.smithi094.stdout:1236 writesame 0x8bb1d17 thru 0x8bbaccd data_size 0x1(0x8fb7 bytes) 2024-09-14T06:29:41.068 INFO:teuthology.orchestra.run.smithi094.stdout:1238 punch from 0x56c86e4 to 0x56d3336, (0xac52 bytes) 2024-09-14T06:29:41.069 INFO:teuthology.orchestra.run.smithi094.stdout:1240 read 0x3585d9 thru 0x366d64 (0xe78c bytes) 2024-09-14T06:29:41.069 INFO:teuthology.orchestra.run.smithi094.stdout:1241 read 0x2cb790f thru 0x2cc1f58 (0xa64a bytes) 2024-09-14T06:29:41.069 INFO:teuthology.orchestra.run.smithi094.stdout:1242 write 0x444b8f6 thru 0x444d5f4 (0x1cff bytes) 2024-09-14T06:29:41.081 INFO:teuthology.orchestra.run.smithi094.stdout:1243 write 0xbf00622 thru 0xbf062e6 (0x5cc5 bytes) 2024-09-14T06:29:41.084 INFO:teuthology.orchestra.run.smithi094.stdout:1244 punch from 0x94d03cc to 0x94dd53e, (0xd172 bytes) 2024-09-14T06:29:41.084 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-14T06:29:41.084 INFO:teuthology.orchestra.run.smithi094.stdout:1247 write 0x5f58c34 thru 0x5f5dc65 (0x5032 bytes) 2024-09-14T06:29:41.097 INFO:teuthology.orchestra.run.smithi094.stdout:1249 trunc from 0xbf062e7 to 0x37e2b33 2024-09-14T06:29:41.102 INFO:tasks.ceph.osd.1.smithi094.stderr:2024-09-14T06:29:41.101+0000 7f328e47a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33641) UID: 0 2024-09-14T06:29:41.109 INFO:teuthology.orchestra.run.smithi094.stdout:1250 punch from 0x3156bda to 0x3163203, (0xc629 bytes) 2024-09-14T06:29:41.117 INFO:teuthology.orchestra.run.smithi094.stdout:1251 write 0x56f27b9 thru 0x56ff040 (0xc888 bytes) 2024-09-14T06:29:41.133 INFO:teuthology.orchestra.run.smithi094.stdout:1254 writesame 0xbcc0edd thru 0xbcc8f10 data_size 0x1e(0x8034 bytes) 2024-09-14T06:29:41.133 INFO:teuthology.orchestra.run.smithi094.stdout:1256 writesame 0xb5ac257 thru 0xb5b22c0 data_size 0x29(0x606a bytes) 2024-09-14T06:29:41.136 INFO:teuthology.orchestra.run.smithi094.stdout:1258 punch from 0x3d00e0b to 0x3d02031, (0x1226 bytes) 2024-09-14T06:29:41.136 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-14T06:29:41.137 INFO:teuthology.orchestra.run.smithi094.stdout:1262 read 0x9cce190 thru 0x9cd0624 (0x2495 bytes) 2024-09-14T06:29:41.137 INFO:teuthology.orchestra.run.smithi094.stdout:1263 read 0xa9fdc0 thru 0xaa4bbb (0x4dfc bytes) 2024-09-14T06:29:41.137 INFO:teuthology.orchestra.run.smithi094.stdout:1264 punch from 0x8081ac1 to 0x80897e2, (0x7d21 bytes) 2024-09-14T06:29:41.137 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-14T06:29:41.137 INFO:teuthology.orchestra.run.smithi094.stdout:1268 trunc from 0xbcc8f11 to 0xb80f3f2 2024-09-14T06:29:41.143 INFO:teuthology.orchestra.run.smithi094.stdout:1270 read 0x20f5b26 thru 0x20f865d (0x2b38 bytes) 2024-09-14T06:29:41.143 INFO:teuthology.orchestra.run.smithi094.stdout:1272 writesame 0x45cc4f1 thru 0x45dac8b data_size 0x1(0xe79b bytes) 2024-09-14T06:29:41.149 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-14T06:29:41.150 INFO:teuthology.orchestra.run.smithi094.stdout: "ops": [], 2024-09-14T06:29:41.150 INFO:teuthology.orchestra.run.smithi094.stdout: "complaint_time": 30, 2024-09-14T06:29:41.150 INFO:teuthology.orchestra.run.smithi094.stdout: "num_blocked_ops": 0 2024-09-14T06:29:41.150 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-14T06:29:41.161 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-14T06:29:41.174 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-14T06:29:41.175 INFO:teuthology.orchestra.run.smithi094.stdout:1273 punch from 0x7ae99b2 to 0x7af83f2, (0xea40 bytes) 2024-09-14T06:29:41.175 INFO:teuthology.orchestra.run.smithi094.stdout:1275 read 0x62872d1 thru 0x6291884 (0xa5b4 bytes) 2024-09-14T06:29:41.183 INFO:teuthology.orchestra.run.smithi094.stdout:1276 writesame 0xe4d40f7 thru 0xe4deef1 data_size 0x1(0xadfb bytes) 2024-09-14T06:29:41.184 INFO:teuthology.orchestra.run.smithi094.stdout:1277 punch from 0x87efe10 to 0x87fb946, (0xbb36 bytes) 2024-09-14T06:29:41.184 INFO:teuthology.orchestra.run.smithi094.stdout:1278 punch from 0xbe3ca2b to 0xbe46d59, (0xa32e bytes) 2024-09-14T06:29:41.185 INFO:teuthology.orchestra.run.smithi094.stdout:1279 trunc from 0xe4deef2 to 0xd39f8e2 2024-09-14T06:29:41.191 INFO:teuthology.orchestra.run.smithi094.stdout:1280 punch from 0x8b86dc4 to 0x8b88658, (0x1894 bytes) 2024-09-14T06:29:41.191 INFO:teuthology.orchestra.run.smithi094.stdout:1281 writesame 0x974e8c6 thru 0x9757c5b data_size 0x9(0x9396 bytes) 2024-09-14T06:29:41.195 INFO:teuthology.orchestra.run.smithi094.stdout:1282 read 0xc5b26f thru 0xc6b0fd (0xfe8f bytes) 2024-09-14T06:29:41.196 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-14T06:29:41.199 INFO:teuthology.orchestra.run.smithi094.stdout:1284 trunc from 0xd39f8e2 to 0xa848dc3 2024-09-14T06:29:41.202 INFO:tasks.ceph.osd.4.smithi138.stderr:2024-09-14T06:29:41.201+0000 7f6c653c8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33615) UID: 0 2024-09-14T06:29:41.222 INFO:teuthology.orchestra.run.smithi094.stdout:1285 trunc from 0xa848dc3 to 0x15e53bb 2024-09-14T06:29:41.238 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-14T06:29:41.238 INFO:teuthology.orchestra.run.smithi094.stdout:1289 read 0x4cc377 thru 0x4cec46 (0x28d0 bytes) 2024-09-14T06:29:41.242 INFO:teuthology.orchestra.run.smithi094.stdout:1291 trunc from 0x15e53bb to 0x4187f6b 2024-09-14T06:29:41.253 INFO:teuthology.orchestra.run.smithi094.stdout:1293 write 0x43db0c1 thru 0x43e15da (0x651a bytes) 2024-09-14T06:29:41.256 INFO:teuthology.orchestra.run.smithi094.stdout:1295 punch from 0x1a9ac1f to 0x1aa9139, (0xe51a bytes) 2024-09-14T06:29:41.258 INFO:teuthology.orchestra.run.smithi094.stdout:1296 trunc from 0x43e15db to 0x5d8430a 2024-09-14T06:29:41.263 INFO:teuthology.orchestra.run.smithi094.stdout:1297 read 0x48f68e6 thru 0x48f8f75 (0x2690 bytes) 2024-09-14T06:29:41.263 INFO:teuthology.orchestra.run.smithi094.stdout:1299 read 0x4e1ee3d thru 0x4e2d50c (0xe6d0 bytes) 2024-09-14T06:29:41.293 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-14T06:29:41.293 INFO:teuthology.orchestra.run.smithi094.stdout: "size": 20, 2024-09-14T06:29:41.293 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 600, 2024-09-14T06:29:41.293 INFO:teuthology.orchestra.run.smithi094.stdout: "ops": [ 2024-09-14T06:29:41.293 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:41.293 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:15608 3.b 3:d24f5f7d:::rbd_data.112d92027590.000000000000000d:f [read 852992~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:29:41.294 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:26.142492+0000", 2024-09-14T06:29:41.294 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 15.148902036999999, 2024-09-14T06:29:41.294 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.048597006999999998, 2024-09-14T06:29:41.294 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:29:41.294 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:29:41.294 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:29:41.294 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:29:41.294 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:29:41.294 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:41.294 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 15608 2024-09-14T06:29:41.294 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:41.294 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:29:41.294 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:41.295 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:29:41.295 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.142492+0000", 2024-09-14T06:29:41.295 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:41.295 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:41.295 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:41.295 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:29:41.295 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.142492+0000", 2024-09-14T06:29:41.295 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:41.295 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:41.295 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:41.295 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:29:41.295 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.142568+0000", 2024-09-14T06:29:41.295 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.6209000000000002e-05 2024-09-14T06:29:41.295 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:41.295 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:41.296 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:29:41.296 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.142571+0000", 2024-09-14T06:29:41.296 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.157e-06 2024-09-14T06:29:41.296 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:41.296 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:41.296 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:29:41.296 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.142579+0000", 2024-09-14T06:29:41.296 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.6879999999999999e-06 2024-09-14T06:29:41.296 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:41.296 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:41.296 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:29:41.296 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.142585+0000", 2024-09-14T06:29:41.296 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.8780000000000003e-06 2024-09-14T06:29:41.297 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:41.297 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:41.297 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:29:41.297 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.190912+0000", 2024-09-14T06:29:41.297 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.048327579000000002 2024-09-14T06:29:41.297 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:41.297 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:41.297 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:29:41.297 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.190990+0000", 2024-09-14T06:29:41.297 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.8119000000000002e-05 2024-09-14T06:29:41.297 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:41.297 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:41.297 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:29:41.298 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.191089+0000", 2024-09-14T06:29:41.298 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.8376999999999997e-05 2024-09-14T06:29:41.298 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:41.298 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:41.298 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:41.298 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:41.298 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:41.298 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:15609 3.b 3:d24f5f7d:::rbd_data.112d92027590.000000000000000d:f [read 863232~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:29:41.298 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:26.142596+0000", 2024-09-14T06:29:41.298 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 15.148798267, 2024-09-14T06:29:41.298 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.048641772999999999, 2024-09-14T06:29:41.298 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:29:41.298 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:29:41.298 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:29:41.298 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:29:41.299 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:29:41.299 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:41.299 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 15609 2024-09-14T06:29:41.299 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:41.299 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:29:41.299 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:41.299 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:29:41.299 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.142596+0000", 2024-09-14T06:29:41.299 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:41.299 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:41.299 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:41.299 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:29:41.299 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.142596+0000", 2024-09-14T06:29:41.300 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:41.300 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:41.300 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:41.300 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:29:41.300 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.142741+0000", 2024-09-14T06:29:41.300 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00014539199999999999 2024-09-14T06:29:41.300 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:41.300 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:41.300 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:29:41.300 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.142745+0000", 2024-09-14T06:29:41.300 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.1470000000000003e-06 2024-09-14T06:29:41.300 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:41.300 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:41.300 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:29:41.301 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.142754+0000", 2024-09-14T06:29:41.301 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.1330000000000005e-06 2024-09-14T06:29:41.301 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:41.301 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:41.301 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:29:41.301 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.142761+0000", 2024-09-14T06:29:41.301 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.63e-06 2024-09-14T06:29:41.301 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:41.301 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:41.301 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:29:41.301 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.191127+0000", 2024-09-14T06:29:41.301 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.048366251999999998 2024-09-14T06:29:41.301 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:41.302 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:41.302 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:29:41.302 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.191178+0000", 2024-09-14T06:29:41.302 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.1359000000000001e-05 2024-09-14T06:29:41.302 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:41.302 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:41.302 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:29:41.302 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.191237+0000", 2024-09-14T06:29:41.302 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.8860000000000002e-05 2024-09-14T06:29:41.302 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:41.302 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:41.302 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:41.302 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:41.302 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:41.303 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:15610 3.b 3:d24f5f7d:::rbd_data.112d92027590.000000000000000d:f [read 873472~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:29:41.303 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:26.142773+0000", 2024-09-14T06:29:41.303 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 15.148620791000001, 2024-09-14T06:29:41.303 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.048677076999999999, 2024-09-14T06:29:41.303 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:29:41.303 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:29:41.303 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:29:41.303 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:29:41.303 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:29:41.303 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:41.303 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 15610 2024-09-14T06:29:41.303 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:41.303 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:29:41.303 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:41.304 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:29:41.304 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.142773+0000", 2024-09-14T06:29:41.304 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:41.304 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:41.304 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:41.304 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:29:41.304 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.142773+0000", 2024-09-14T06:29:41.304 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:41.304 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:41.305 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:41.305 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:29:41.305 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.142855+0000", 2024-09-14T06:29:41.305 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.2180000000000003e-05 2024-09-14T06:29:41.305 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:41.305 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:41.305 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:29:41.305 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.142859+0000", 2024-09-14T06:29:41.305 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.5810000000000001e-06 2024-09-14T06:29:41.305 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:41.305 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:41.305 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:29:41.305 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.142868+0000", 2024-09-14T06:29:41.306 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.8489999999999995e-06 2024-09-14T06:29:41.306 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:41.306 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:41.306 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:29:41.306 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.142874+0000", 2024-09-14T06:29:41.306 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.7750000000000004e-06 2024-09-14T06:29:41.306 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:41.306 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:41.306 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:29:41.306 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.191272+0000", 2024-09-14T06:29:41.306 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.048397430999999998 2024-09-14T06:29:41.306 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:41.306 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:41.306 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:29:41.307 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.191351+0000", 2024-09-14T06:29:41.307 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.9480000000000005e-05 2024-09-14T06:29:41.307 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:41.307 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:41.307 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:29:41.307 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.191450+0000", 2024-09-14T06:29:41.307 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.8781000000000002e-05 2024-09-14T06:29:41.307 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:41.307 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:41.307 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:41.307 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:41.307 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:41.307 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:15611 3.b 3:d24f5f7d:::rbd_data.112d92027590.000000000000000d:f [read 883712~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:29:41.307 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:26.142885+0000", 2024-09-14T06:29:41.307 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 15.148508842, 2024-09-14T06:29:41.308 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.048706685, 2024-09-14T06:29:41.308 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:29:41.308 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:29:41.308 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:29:41.308 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:29:41.308 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:29:41.308 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:41.308 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 15611 2024-09-14T06:29:41.308 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:41.309 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:29:41.309 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:41.309 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:29:41.309 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.142885+0000", 2024-09-14T06:29:41.309 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:41.309 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:41.309 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:41.309 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:29:41.309 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.142885+0000", 2024-09-14T06:29:41.309 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:41.309 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:41.309 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:41.309 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:29:41.309 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.142989+0000", 2024-09-14T06:29:41.310 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00010394 2024-09-14T06:29:41.310 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:41.310 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:41.310 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:29:41.310 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.142992+0000", 2024-09-14T06:29:41.310 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.534e-06 2024-09-14T06:29:41.310 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:41.310 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:41.310 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:29:41.310 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.143001+0000", 2024-09-14T06:29:41.310 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.9050000000000003e-06 2024-09-14T06:29:41.310 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:41.310 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:41.310 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:29:41.310 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.143008+0000", 2024-09-14T06:29:41.310 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.8900000000000001e-06 2024-09-14T06:29:41.311 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:41.311 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:41.311 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:29:41.311 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.191489+0000", 2024-09-14T06:29:41.311 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.048480489000000002 2024-09-14T06:29:41.311 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:41.311 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:41.311 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:29:41.311 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.191532+0000", 2024-09-14T06:29:41.311 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.3244999999999997e-05 2024-09-14T06:29:41.311 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:41.311 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:41.311 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:29:41.311 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.191592+0000", 2024-09-14T06:29:41.311 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.9682000000000002e-05 2024-09-14T06:29:41.312 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:41.312 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:41.312 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:41.312 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:41.312 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:41.312 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:15612 3.b 3:d24f5f7d:::rbd_data.112d92027590.000000000000000d:f [read 893952~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:29:41.312 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:26.143019+0000", 2024-09-14T06:29:41.312 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 15.148375151, 2024-09-14T06:29:41.312 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.048778560999999998, 2024-09-14T06:29:41.312 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:29:41.312 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:29:41.312 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:29:41.312 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:29:41.312 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:29:41.313 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:41.313 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 15612 2024-09-14T06:29:41.313 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:41.313 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:29:41.313 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:41.313 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:29:41.313 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.143019+0000", 2024-09-14T06:29:41.313 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:41.313 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:41.313 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:41.313 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:29:41.313 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.143019+0000", 2024-09-14T06:29:41.313 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:41.313 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:41.313 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:41.314 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:29:41.314 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.143236+0000", 2024-09-14T06:29:41.314 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000217517 2024-09-14T06:29:41.314 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:41.314 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:41.314 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:29:41.314 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.143241+0000", 2024-09-14T06:29:41.314 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.7779999999999998e-06 2024-09-14T06:29:41.314 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:41.314 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:41.314 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:29:41.314 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.143248+0000", 2024-09-14T06:29:41.314 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.7100000000000001e-06 2024-09-14T06:29:41.314 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:41.314 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:41.314 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:29:41.314 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.143254+0000", 2024-09-14T06:29:41.315 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.8950000000000003e-06 2024-09-14T06:29:41.315 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:41.315 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:41.315 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:29:41.315 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.191626+0000", 2024-09-14T06:29:41.315 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.048372747000000001 2024-09-14T06:29:41.315 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:41.315 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:41.315 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:29:41.315 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.191703+0000", 2024-09-14T06:29:41.315 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.6246000000000001e-05 2024-09-14T06:29:41.315 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:41.315 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:41.315 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:29:41.315 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.191797+0000", 2024-09-14T06:29:41.316 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.4667999999999996e-05 2024-09-14T06:29:41.316 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:41.316 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:41.316 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:41.316 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:41.316 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:41.316 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:15613 3.b 3:d24f5f7d:::rbd_data.112d92027590.000000000000000d:f [read 904192~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:29:41.316 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:26.143264+0000", 2024-09-14T06:29:41.316 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 15.148129398, 2024-09-14T06:29:41.316 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.048672880000000002, 2024-09-14T06:29:41.316 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:29:41.316 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:29:41.316 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:29:41.316 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:29:41.316 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:29:41.316 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:41.317 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 15613 2024-09-14T06:29:41.317 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:41.317 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:29:41.317 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:41.317 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:29:41.317 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.143264+0000", 2024-09-14T06:29:41.317 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:41.317 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:41.317 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:41.317 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:29:41.317 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.143264+0000", 2024-09-14T06:29:41.317 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:41.317 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:41.317 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:41.318 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:29:41.318 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.143557+0000", 2024-09-14T06:29:41.318 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000292572 2024-09-14T06:29:41.318 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:41.318 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:41.318 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:29:41.318 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.143560+0000", 2024-09-14T06:29:41.318 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.4690000000000002e-06 2024-09-14T06:29:41.318 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:41.318 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:41.318 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:29:41.318 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.143572+0000", 2024-09-14T06:29:41.318 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.1457000000000001e-05 2024-09-14T06:29:41.318 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:41.319 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:41.319 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:29:41.319 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.143578+0000", 2024-09-14T06:29:41.319 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.5520000000000001e-06 2024-09-14T06:29:41.319 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:41.319 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:41.319 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:29:41.319 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.191838+0000", 2024-09-14T06:29:41.319 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.048259947999999997 2024-09-14T06:29:41.319 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:41.319 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:41.319 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:29:41.319 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.191881+0000", 2024-09-14T06:29:41.319 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.3058e-05 2024-09-14T06:29:41.319 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:41.320 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:41.320 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:29:41.320 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.191937+0000", 2024-09-14T06:29:41.320 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.5823999999999999e-05 2024-09-14T06:29:41.320 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:41.320 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:41.320 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:41.320 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:41.320 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:41.320 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:15614 3.b 3:d24f5f7d:::rbd_data.112d92027590.000000000000000d:f [read 914432~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:29:41.320 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:26.143597+0000", 2024-09-14T06:29:41.320 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 15.147796823, 2024-09-14T06:29:41.320 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.048548172000000001, 2024-09-14T06:29:41.320 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:29:41.320 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:29:41.320 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:29:41.321 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:29:41.321 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:29:41.321 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:41.321 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 15614 2024-09-14T06:29:41.321 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:41.321 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:29:41.321 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:41.321 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:29:41.321 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.143597+0000", 2024-09-14T06:29:41.321 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:41.321 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:41.321 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:41.321 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:29:41.322 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.143597+0000", 2024-09-14T06:29:41.322 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:41.322 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:41.322 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:41.322 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:29:41.322 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.143598+0000", 2024-09-14T06:29:41.322 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.1099999999999996e-07 2024-09-14T06:29:41.322 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:41.322 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:41.322 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:29:41.322 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.143600+0000", 2024-09-14T06:29:41.322 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.4210000000000002e-06 2024-09-14T06:29:41.322 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:41.323 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:41.323 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:29:41.323 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.143608+0000", 2024-09-14T06:29:41.323 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.2379999999999997e-06 2024-09-14T06:29:41.323 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:41.323 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:41.323 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:29:41.323 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.143613+0000", 2024-09-14T06:29:41.323 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.1909999999999999e-06 2024-09-14T06:29:41.323 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:41.323 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:41.323 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:29:41.323 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.191970+0000", 2024-09-14T06:29:41.324 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.048356757 2024-09-14T06:29:41.324 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:41.324 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:41.324 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:29:41.324 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.192049+0000", 2024-09-14T06:29:41.324 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.9252000000000006e-05 2024-09-14T06:29:41.324 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:41.324 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:41.324 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:29:41.324 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.192145+0000", 2024-09-14T06:29:41.324 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.5802000000000002e-05 2024-09-14T06:29:41.324 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:41.324 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:41.325 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:41.325 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:41.325 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:41.325 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:15615 3.b 3:d24f5f7d:::rbd_data.112d92027590.000000000000000d:f [read 924672~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:29:41.325 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:26.143621+0000", 2024-09-14T06:29:41.325 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 15.14777234, 2024-09-14T06:29:41.325 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.04866728, 2024-09-14T06:29:41.325 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:29:41.325 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:29:41.325 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:29:41.325 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:29:41.325 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:29:41.325 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:41.326 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 15615 2024-09-14T06:29:41.326 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:41.326 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:29:41.326 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:41.326 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:29:41.326 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.143621+0000", 2024-09-14T06:29:41.326 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:41.326 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:41.326 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:41.326 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:29:41.326 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.143621+0000", 2024-09-14T06:29:41.326 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:41.326 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:41.326 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:41.327 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:29:41.327 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.145193+0000", 2024-09-14T06:29:41.327 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0015719480000000001 2024-09-14T06:29:41.327 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:41.327 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:41.327 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:29:41.327 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.145198+0000", 2024-09-14T06:29:41.327 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.8770000000000001e-06 2024-09-14T06:29:41.327 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:41.327 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:41.327 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:29:41.327 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.145207+0000", 2024-09-14T06:29:41.327 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.0620000000000007e-06 2024-09-14T06:29:41.327 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:41.327 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:41.328 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:29:41.328 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.145214+0000", 2024-09-14T06:29:41.328 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.156e-06 2024-09-14T06:29:41.328 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:41.328 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:41.328 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:29:41.328 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.192180+0000", 2024-09-14T06:29:41.328 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.046966624999999998 2024-09-14T06:29:41.328 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:41.328 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:41.328 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:29:41.328 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.192231+0000", 2024-09-14T06:29:41.328 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.0754999999999998e-05 2024-09-14T06:29:41.328 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:41.328 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:41.329 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:29:41.329 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.192289+0000", 2024-09-14T06:29:41.329 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.7856999999999998e-05 2024-09-14T06:29:41.329 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:41.329 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:41.329 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:41.329 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:41.329 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:41.329 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:15704 3.b 3:d24f5f7d:::rbd_data.112d92027590.000000000000000d:f [read 137216~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:29:41.329 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:26.148323+0000", 2024-09-14T06:29:41.329 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 15.143070811999999, 2024-09-14T06:29:41.329 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.048538987999999998, 2024-09-14T06:29:41.329 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:29:41.330 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:29:41.330 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:29:41.330 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:29:41.330 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:29:41.330 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:41.330 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 15704 2024-09-14T06:29:41.330 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:41.330 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:29:41.330 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:41.330 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:29:41.330 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.148323+0000", 2024-09-14T06:29:41.330 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:41.330 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:41.330 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:41.330 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:29:41.331 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.148323+0000", 2024-09-14T06:29:41.331 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:41.331 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:41.331 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:41.331 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:29:41.331 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.148448+0000", 2024-09-14T06:29:41.331 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00012475999999999999 2024-09-14T06:29:41.331 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:41.331 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:41.331 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:29:41.331 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.148451+0000", 2024-09-14T06:29:41.331 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.5370000000000002e-06 2024-09-14T06:29:41.331 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:41.331 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:41.332 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:29:41.332 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.148460+0000", 2024-09-14T06:29:41.332 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.2949999999999994e-06 2024-09-14T06:29:41.332 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:41.332 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:41.332 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:29:41.332 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.148465+0000", 2024-09-14T06:29:41.332 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.7570000000000001e-06 2024-09-14T06:29:41.332 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:41.332 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:41.332 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:29:41.332 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.196699+0000", 2024-09-14T06:29:41.332 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.048233542999999997 2024-09-14T06:29:41.332 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:41.332 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:41.333 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:29:41.333 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.196766+0000", 2024-09-14T06:29:41.333 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.7259000000000001e-05 2024-09-14T06:29:41.333 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:41.333 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:41.333 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:29:41.333 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.196862+0000", 2024-09-14T06:29:41.333 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.5836999999999997e-05 2024-09-14T06:29:41.333 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:41.333 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:41.333 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:41.333 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:41.333 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:41.333 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:15705 3.b 3:d24f5f7d:::rbd_data.112d92027590.000000000000000d:f [read 147456~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:29:41.334 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:26.148475+0000", 2024-09-14T06:29:41.334 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 15.142918952, 2024-09-14T06:29:41.334 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.048589192000000003, 2024-09-14T06:29:41.334 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:29:41.334 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:29:41.334 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:29:41.334 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:29:41.334 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:29:41.334 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:41.334 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 15705 2024-09-14T06:29:41.334 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:41.334 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:29:41.334 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:41.334 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:29:41.335 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.148475+0000", 2024-09-14T06:29:41.335 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:41.335 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:41.335 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:41.335 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:29:41.335 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.148475+0000", 2024-09-14T06:29:41.335 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:41.335 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:41.335 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:41.335 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:29:41.335 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.148565+0000", 2024-09-14T06:29:41.335 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.9876000000000005e-05 2024-09-14T06:29:41.335 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:41.335 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:41.336 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:29:41.336 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.148568+0000", 2024-09-14T06:29:41.336 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.2210000000000002e-06 2024-09-14T06:29:41.336 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:41.336 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:41.336 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:29:41.336 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.148580+0000", 2024-09-14T06:29:41.336 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.2442e-05 2024-09-14T06:29:41.336 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:41.336 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:41.336 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:29:41.336 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.148590+0000", 2024-09-14T06:29:41.336 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.0860000000000008e-06 2024-09-14T06:29:41.336 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:41.336 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:41.337 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:29:41.337 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.196893+0000", 2024-09-14T06:29:41.337 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.048303328999999999 2024-09-14T06:29:41.337 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:41.337 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:41.337 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:29:41.337 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.196968+0000", 2024-09-14T06:29:41.337 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.4690000000000005e-05 2024-09-14T06:29:41.337 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:41.337 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:41.337 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:29:41.337 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.197064+0000", 2024-09-14T06:29:41.337 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.6547999999999998e-05 2024-09-14T06:29:41.338 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:41.338 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:41.338 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:41.338 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:41.338 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:41.338 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:15706 3.b 3:d24f5f7d:::rbd_data.112d92027590.000000000000000d:f [read 157696~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:29:41.338 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:26.148603+0000", 2024-09-14T06:29:41.338 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 15.142790759, 2024-09-14T06:29:41.338 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.048652025000000002, 2024-09-14T06:29:41.338 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:29:41.338 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:29:41.338 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:29:41.338 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:29:41.338 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:29:41.339 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:41.339 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 15706 2024-09-14T06:29:41.339 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:41.339 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:29:41.339 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:41.339 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:29:41.339 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.148603+0000", 2024-09-14T06:29:41.339 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:41.339 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:41.339 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:41.339 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:29:41.339 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.148603+0000", 2024-09-14T06:29:41.339 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:41.339 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:41.340 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:41.340 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:29:41.340 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.148791+0000", 2024-09-14T06:29:41.340 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00018775700000000001 2024-09-14T06:29:41.340 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:41.340 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:41.340 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:29:41.340 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.148794+0000", 2024-09-14T06:29:41.340 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.9550000000000001e-06 2024-09-14T06:29:41.340 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:41.340 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:41.340 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:29:41.340 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.148802+0000", 2024-09-14T06:29:41.340 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.7789999999999996e-06 2024-09-14T06:29:41.341 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:41.341 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:41.341 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:29:41.341 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.148814+0000", 2024-09-14T06:29:41.341 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.2886e-05 2024-09-14T06:29:41.341 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:41.341 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:41.341 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:29:41.341 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.197104+0000", 2024-09-14T06:29:41.341 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.048289683 2024-09-14T06:29:41.341 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:41.341 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:41.341 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:29:41.341 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.197171+0000", 2024-09-14T06:29:41.342 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.7137000000000005e-05 2024-09-14T06:29:41.342 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:41.342 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:41.342 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:29:41.342 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.197255+0000", 2024-09-14T06:29:41.342 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.3827999999999998e-05 2024-09-14T06:29:41.342 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:41.342 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:41.342 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:41.342 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:41.342 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:41.342 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:15707 3.b 3:d24f5f7d:::rbd_data.112d92027590.000000000000000d:f [read 167936~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:29:41.342 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:26.148826+0000", 2024-09-14T06:29:41.342 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 15.142568155999999, 2024-09-14T06:29:41.343 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.048578723999999997, 2024-09-14T06:29:41.343 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:29:41.343 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:29:41.343 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:29:41.343 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:29:41.343 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:29:41.343 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:41.343 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 15707 2024-09-14T06:29:41.343 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:41.343 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:29:41.343 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:41.343 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:29:41.343 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.148826+0000", 2024-09-14T06:29:41.344 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:41.344 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:41.344 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:41.344 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:29:41.344 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.148826+0000", 2024-09-14T06:29:41.344 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:41.344 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:41.344 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:41.344 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:29:41.344 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.149053+0000", 2024-09-14T06:29:41.344 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00022741100000000001 2024-09-14T06:29:41.344 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:41.344 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:41.344 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:29:41.345 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.149060+0000", 2024-09-14T06:29:41.345 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.8040000000000004e-06 2024-09-14T06:29:41.345 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:41.345 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:41.345 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:29:41.345 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.149069+0000", 2024-09-14T06:29:41.345 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.8189999999999998e-06 2024-09-14T06:29:41.345 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:41.345 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:41.345 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:29:41.345 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.149075+0000", 2024-09-14T06:29:41.345 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.2079999999999997e-06 2024-09-14T06:29:41.345 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:41.346 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:41.346 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:29:41.346 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.197299+0000", 2024-09-14T06:29:41.346 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.048223749000000003 2024-09-14T06:29:41.346 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:41.346 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:41.346 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:29:41.346 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.197343+0000", 2024-09-14T06:29:41.346 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.4721000000000001e-05 2024-09-14T06:29:41.346 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:41.346 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:41.346 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:29:41.346 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.197404+0000", 2024-09-14T06:29:41.346 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.1011999999999998e-05 2024-09-14T06:29:41.347 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:41.347 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:41.347 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:41.347 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:41.347 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:41.347 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:15708 3.b 3:d24f5f7d:::rbd_data.112d92027590.000000000000000d:f [read 178176~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:29:41.347 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:26.149084+0000", 2024-09-14T06:29:41.347 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 15.142309531, 2024-09-14T06:29:41.347 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.048528769999999999, 2024-09-14T06:29:41.347 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:29:41.347 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:29:41.347 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:29:41.347 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:29:41.347 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:29:41.347 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:41.348 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 15708 2024-09-14T06:29:41.348 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:41.348 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:29:41.348 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:41.348 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:29:41.348 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.149084+0000", 2024-09-14T06:29:41.348 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:41.348 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:41.348 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:41.348 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:29:41.348 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.149084+0000", 2024-09-14T06:29:41.348 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:41.348 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:41.348 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:41.349 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:29:41.349 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.150587+0000", 2024-09-14T06:29:41.349 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.001503187 2024-09-14T06:29:41.349 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:41.349 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:41.349 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:29:41.349 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.150592+0000", 2024-09-14T06:29:41.349 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.07e-06 2024-09-14T06:29:41.349 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:41.349 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:41.349 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:29:41.349 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.150602+0000", 2024-09-14T06:29:41.350 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.0584999999999999e-05 2024-09-14T06:29:41.350 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:41.350 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:41.350 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:29:41.350 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.150609+0000", 2024-09-14T06:29:41.350 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.8560000000000001e-06 2024-09-14T06:29:41.350 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:41.350 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:41.350 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:29:41.350 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.197446+0000", 2024-09-14T06:29:41.350 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.046837328999999997 2024-09-14T06:29:41.350 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:41.350 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:41.350 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:29:41.351 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.197530+0000", 2024-09-14T06:29:41.351 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.3493999999999997e-05 2024-09-14T06:29:41.351 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:41.351 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:41.351 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:29:41.351 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.197613+0000", 2024-09-14T06:29:41.351 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.3249000000000002e-05 2024-09-14T06:29:41.351 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:41.351 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:41.351 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:41.351 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:41.351 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:41.351 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:15736 3.b 3:d24f5f7d:::rbd_data.112d92027590.000000000000000d:f [read 464896~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:29:41.351 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:26.154778+0000", 2024-09-14T06:29:41.352 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 15.136615515999999, 2024-09-14T06:29:41.352 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.048624703999999998, 2024-09-14T06:29:41.352 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:29:41.352 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:29:41.352 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:29:41.352 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:29:41.352 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:29:41.352 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:41.352 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 15736 2024-09-14T06:29:41.352 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:41.352 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:29:41.352 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:41.352 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:29:41.352 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.154778+0000", 2024-09-14T06:29:41.353 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:41.353 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:41.353 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:41.353 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:29:41.353 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.154778+0000", 2024-09-14T06:29:41.353 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:41.353 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:41.353 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:41.353 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:29:41.353 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.154847+0000", 2024-09-14T06:29:41.353 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.8374999999999995e-05 2024-09-14T06:29:41.353 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:41.353 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:41.354 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:29:41.354 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.154851+0000", 2024-09-14T06:29:41.354 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.4830000000000001e-06 2024-09-14T06:29:41.354 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:41.354 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:41.354 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:29:41.354 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.154863+0000", 2024-09-14T06:29:41.354 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.188e-05 2024-09-14T06:29:41.354 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:41.354 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:41.354 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:29:41.354 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.154870+0000", 2024-09-14T06:29:41.354 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.8159999999999996e-06 2024-09-14T06:29:41.354 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:41.355 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:41.355 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:29:41.355 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.203218+0000", 2024-09-14T06:29:41.355 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.048348440999999999 2024-09-14T06:29:41.355 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:41.355 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:41.355 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:29:41.355 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.203302+0000", 2024-09-14T06:29:41.355 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.3764999999999995e-05 2024-09-14T06:29:41.355 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:41.355 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:41.355 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:29:41.355 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.203403+0000", 2024-09-14T06:29:41.355 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000100944 2024-09-14T06:29:41.356 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:41.356 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:41.356 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:41.356 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:41.356 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:41.356 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:15737 3.b 3:d24f5f7d:::rbd_data.112d92027590.000000000000000d:f [read 475136~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:29:41.356 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:26.154883+0000", 2024-09-14T06:29:41.356 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 15.136511303000001, 2024-09-14T06:29:41.356 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.048784932000000003, 2024-09-14T06:29:41.356 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:29:41.356 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:29:41.356 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:29:41.356 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:29:41.356 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:29:41.356 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:41.357 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 15737 2024-09-14T06:29:41.357 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:41.357 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:29:41.357 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:41.357 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:29:41.357 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.154883+0000", 2024-09-14T06:29:41.357 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:41.357 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:41.357 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:41.357 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:29:41.357 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.154883+0000", 2024-09-14T06:29:41.357 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:41.357 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:41.357 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:41.358 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:29:41.358 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.155040+0000", 2024-09-14T06:29:41.358 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000157236 2024-09-14T06:29:41.358 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:41.358 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:41.358 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:29:41.358 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.155044+0000", 2024-09-14T06:29:41.358 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.1330000000000001e-06 2024-09-14T06:29:41.358 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:41.358 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:41.358 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:29:41.358 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.155053+0000", 2024-09-14T06:29:41.358 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.8869999999999998e-06 2024-09-14T06:29:41.358 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:41.359 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:41.359 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:29:41.359 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.155058+0000", 2024-09-14T06:29:41.359 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.3940000000000004e-06 2024-09-14T06:29:41.359 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:41.359 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:41.359 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:29:41.359 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.203478+0000", 2024-09-14T06:29:41.359 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.048420073000000001 2024-09-14T06:29:41.359 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:41.359 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:41.359 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:29:41.359 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.203566+0000", 2024-09-14T06:29:41.359 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.7620999999999993e-05 2024-09-14T06:29:41.359 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:41.360 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:41.360 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:29:41.360 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.203667+0000", 2024-09-14T06:29:41.360 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000101588 2024-09-14T06:29:41.360 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:41.360 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:41.360 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:41.360 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:41.360 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:41.360 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:15738 3.b 3:d24f5f7d:::rbd_data.112d92027590.000000000000000d:f [read 485376~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:29:41.360 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:26.155069+0000", 2024-09-14T06:29:41.360 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 15.136324373000001, 2024-09-14T06:29:41.360 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.048810049000000001, 2024-09-14T06:29:41.360 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:29:41.360 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:29:41.361 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:29:41.361 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:29:41.361 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:29:41.361 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:41.361 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 15738 2024-09-14T06:29:41.361 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:41.361 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:29:41.361 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:41.361 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:29:41.361 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.155069+0000", 2024-09-14T06:29:41.361 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:41.361 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:41.361 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:41.361 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:29:41.362 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.155069+0000", 2024-09-14T06:29:41.362 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:41.362 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:41.362 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:41.362 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:29:41.362 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.155278+0000", 2024-09-14T06:29:41.362 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00020823400000000001 2024-09-14T06:29:41.362 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:41.362 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:41.362 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:29:41.362 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.155282+0000", 2024-09-14T06:29:41.362 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.4730000000000002e-06 2024-09-14T06:29:41.362 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:41.362 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:41.362 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:29:41.363 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.155294+0000", 2024-09-14T06:29:41.363 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.1347e-05 2024-09-14T06:29:41.363 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:41.363 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:41.363 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:29:41.363 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.155302+0000", 2024-09-14T06:29:41.363 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.7339999999999997e-06 2024-09-14T06:29:41.363 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:41.363 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:41.363 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:29:41.363 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.203701+0000", 2024-09-14T06:29:41.363 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.048399144999999998 2024-09-14T06:29:41.363 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:41.364 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:41.364 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:29:41.364 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.203778+0000", 2024-09-14T06:29:41.364 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.6169999999999997e-05 2024-09-14T06:29:41.364 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:41.364 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:41.364 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:29:41.364 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.203880+0000", 2024-09-14T06:29:41.364 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000101946 2024-09-14T06:29:41.364 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:41.364 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:41.364 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:41.364 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:41.364 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:41.365 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:15739 3.b 3:d24f5f7d:::rbd_data.112d92027590.000000000000000d:f [read 495616~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:29:41.365 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:26.155331+0000", 2024-09-14T06:29:41.365 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 15.136062857000001, 2024-09-14T06:29:41.365 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.048708649999999999, 2024-09-14T06:29:41.365 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:29:41.365 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:29:41.365 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:29:41.365 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:29:41.365 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:29:41.365 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:41.365 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 15739 2024-09-14T06:29:41.365 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:41.365 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:29:41.366 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:41.366 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:29:41.366 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.155331+0000", 2024-09-14T06:29:41.366 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:41.366 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:41.366 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:41.366 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:29:41.366 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.155331+0000", 2024-09-14T06:29:41.366 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:41.366 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:41.366 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:41.366 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:29:41.366 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.155526+0000", 2024-09-14T06:29:41.366 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00019500299999999999 2024-09-14T06:29:41.367 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:41.367 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:41.367 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:29:41.367 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.155532+0000", 2024-09-14T06:29:41.367 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.6620000000000002e-06 2024-09-14T06:29:41.367 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:41.367 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:41.367 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:29:41.367 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.155541+0000", 2024-09-14T06:29:41.367 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.2240000000000001e-06 2024-09-14T06:29:41.367 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:41.367 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:41.367 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:29:41.367 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.155547+0000", 2024-09-14T06:29:41.367 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.4559999999999997e-06 2024-09-14T06:29:41.368 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:41.368 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:41.368 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:29:41.368 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.203917+0000", 2024-09-14T06:29:41.368 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.048370017000000001 2024-09-14T06:29:41.368 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:41.368 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:41.368 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:29:41.368 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.203970+0000", 2024-09-14T06:29:41.368 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.2308999999999997e-05 2024-09-14T06:29:41.368 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:41.368 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:41.368 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:29:41.369 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.204040+0000", 2024-09-14T06:29:41.369 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.9979000000000002e-05 2024-09-14T06:29:41.369 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:41.369 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:41.369 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:41.369 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:41.369 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:41.369 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:15740 3.b 3:d24f5f7d:::rbd_data.112d92027590.000000000000000d:f [read 505856~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:29:41.369 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:26.155558+0000", 2024-09-14T06:29:41.369 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 15.135835733, 2024-09-14T06:29:41.369 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.048618724000000002, 2024-09-14T06:29:41.369 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:29:41.369 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:29:41.370 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:29:41.370 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:29:41.370 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:29:41.370 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:41.370 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 15740 2024-09-14T06:29:41.370 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:41.370 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:29:41.370 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:41.370 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:29:41.370 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.155558+0000", 2024-09-14T06:29:41.370 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:41.370 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:41.370 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:41.370 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:29:41.371 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.155558+0000", 2024-09-14T06:29:41.371 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:41.371 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:41.371 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:41.371 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:29:41.371 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.155766+0000", 2024-09-14T06:29:41.371 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00020751999999999999 2024-09-14T06:29:41.371 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:41.371 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:41.371 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:29:41.371 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.155769+0000", 2024-09-14T06:29:41.371 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.6500000000000002e-06 2024-09-14T06:29:41.371 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:41.371 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:41.371 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:29:41.371 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.155779+0000", 2024-09-14T06:29:41.372 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.0179e-05 2024-09-14T06:29:41.372 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:41.372 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:41.372 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:29:41.372 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.155785+0000", 2024-09-14T06:29:41.372 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.9120000000000003e-06 2024-09-14T06:29:41.372 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:41.372 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:41.372 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:29:41.372 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.204072+0000", 2024-09-14T06:29:41.372 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.048286463000000002 2024-09-14T06:29:41.372 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:41.372 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:41.372 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:29:41.373 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.204119+0000", 2024-09-14T06:29:41.373 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.7023e-05 2024-09-14T06:29:41.373 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:41.373 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:41.373 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:29:41.373 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.204177+0000", 2024-09-14T06:29:41.373 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.7976999999999997e-05 2024-09-14T06:29:41.373 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:41.373 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:41.373 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:41.373 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:41.373 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:41.373 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:15742 3.b 3:d24f5f7d:::rbd_data.112d92027590.000000000000000d:f [read 526336~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:29:41.374 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:26.156014+0000", 2024-09-14T06:29:41.374 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 15.135380147999999, 2024-09-14T06:29:41.374 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.048548612999999997, 2024-09-14T06:29:41.374 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:29:41.374 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:29:41.374 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:29:41.374 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:29:41.374 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:29:41.374 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:41.374 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 15742 2024-09-14T06:29:41.374 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:41.374 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:29:41.374 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:41.374 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:29:41.375 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.156014+0000", 2024-09-14T06:29:41.375 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:41.375 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:41.375 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:41.375 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:29:41.375 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.156014+0000", 2024-09-14T06:29:41.375 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:41.375 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:41.375 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:41.375 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:29:41.375 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.156246+0000", 2024-09-14T06:29:41.375 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00023262700000000001 2024-09-14T06:29:41.375 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:41.375 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:41.375 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:29:41.376 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.156251+0000", 2024-09-14T06:29:41.376 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.0209999999999999e-06 2024-09-14T06:29:41.376 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:41.376 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:41.376 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:29:41.376 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.156262+0000", 2024-09-14T06:29:41.376 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.0168e-05 2024-09-14T06:29:41.376 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:41.376 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:41.376 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:29:41.376 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.156269+0000", 2024-09-14T06:29:41.376 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.0999999999999998e-06 2024-09-14T06:29:41.376 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:41.376 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:41.377 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:29:41.377 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.204363+0000", 2024-09-14T06:29:41.377 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.048094455000000001 2024-09-14T06:29:41.377 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:41.377 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:41.377 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:29:41.377 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.204452+0000", 2024-09-14T06:29:41.377 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.8899000000000004e-05 2024-09-14T06:29:41.377 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:41.377 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:41.377 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:29:41.377 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.204562+0000", 2024-09-14T06:29:41.378 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000110343 2024-09-14T06:29:41.378 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:41.378 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:41.378 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:41.378 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:41.378 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:41.378 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:15746 3.b 3:d24f5f7d:::rbd_data.112d92027590.000000000000000d:f [read 567296~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:29:41.378 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:26.156734+0000", 2024-09-14T06:29:41.378 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 15.134659658, 2024-09-14T06:29:41.378 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.048564930999999999, 2024-09-14T06:29:41.378 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:29:41.378 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:29:41.378 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:29:41.378 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:29:41.378 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:29:41.379 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:41.379 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 15746 2024-09-14T06:29:41.379 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:41.379 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:29:41.379 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:41.379 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:29:41.379 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.156734+0000", 2024-09-14T06:29:41.379 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:41.379 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:41.379 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:41.379 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:29:41.379 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.156734+0000", 2024-09-14T06:29:41.379 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:41.379 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:41.379 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:41.380 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:29:41.380 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.156952+0000", 2024-09-14T06:29:41.380 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000217844 2024-09-14T06:29:41.380 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:41.380 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:41.380 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:29:41.380 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.156956+0000", 2024-09-14T06:29:41.380 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.0790000000000002e-06 2024-09-14T06:29:41.380 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:41.380 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:41.380 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:29:41.380 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.156966+0000", 2024-09-14T06:29:41.380 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.4150000000000005e-06 2024-09-14T06:29:41.380 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:41.381 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:41.381 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:29:41.381 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.156972+0000", 2024-09-14T06:29:41.381 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.365e-06 2024-09-14T06:29:41.381 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:41.381 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:41.381 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:29:41.381 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.205128+0000", 2024-09-14T06:29:41.381 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.048155833000000002 2024-09-14T06:29:41.381 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:41.381 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:41.381 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:29:41.381 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.205205+0000", 2024-09-14T06:29:41.381 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.7250999999999996e-05 2024-09-14T06:29:41.382 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:41.382 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:41.382 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:29:41.382 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.205299+0000", 2024-09-14T06:29:41.382 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.4143999999999999e-05 2024-09-14T06:29:41.382 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:41.382 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:41.382 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:41.382 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:41.382 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:41.382 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-14T06:29:41.384 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-14T06:29:41.384 INFO:teuthology.orchestra.run.smithi094.stdout:osd.0: {} 2024-09-14T06:29:41.384 INFO:teuthology.orchestra.run.smithi094.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-14T06:29:41.385 INFO:teuthology.orchestra.run.smithi094.stdout:osd.1: {} 2024-09-14T06:29:41.385 INFO:teuthology.orchestra.run.smithi094.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-14T06:29:41.385 INFO:teuthology.orchestra.run.smithi094.stdout:osd.2: {} 2024-09-14T06:29:41.385 INFO:teuthology.orchestra.run.smithi094.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-14T06:29:41.386 INFO:teuthology.orchestra.run.smithi094.stdout:osd.3: {} 2024-09-14T06:29:41.386 INFO:teuthology.orchestra.run.smithi094.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-14T06:29:41.386 INFO:teuthology.orchestra.run.smithi094.stdout:osd.4: {} 2024-09-14T06:29:41.386 INFO:teuthology.orchestra.run.smithi094.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-14T06:29:41.387 INFO:tasks.ceph.osd.4.smithi138.stderr:2024-09-14T06:29:41.382+0000 7f6c653c8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33615) UID: 0 2024-09-14T06:29:41.389 INFO:teuthology.orchestra.run.smithi094.stdout:osd.5: {} 2024-09-14T06:29:41.389 INFO:teuthology.orchestra.run.smithi094.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-14T06:29:41.405 INFO:teuthology.orchestra.run.smithi094.stdout:osd.6: {} 2024-09-14T06:29:41.405 INFO:teuthology.orchestra.run.smithi094.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-14T06:29:41.422 INFO:teuthology.orchestra.run.smithi094.stdout:osd.7: {} 2024-09-14T06:29:41.422 INFO:teuthology.orchestra.run.smithi094.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-14T06:29:41.475 INFO:teuthology.orchestra.run.smithi094.stdout:1300 write 0x8270f79 thru 0x827e5e8 (0xd670 bytes) 2024-09-14T06:29:41.484 INFO:tasks.ceph.osd.0.smithi094.stderr:2024-09-14T06:29:41.483+0000 7f07c9333640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33636) UID: 0 2024-09-14T06:29:41.490 INFO:teuthology.orchestra.run.smithi094.stdout:1301 write 0xc976c35 thru 0xc983426 (0xc7f2 bytes) 2024-09-14T06:29:41.494 INFO:teuthology.orchestra.run.smithi138.stdout:ERROR: (22) Invalid argument 2024-09-14T06:29:41.494 INFO:teuthology.orchestra.run.smithi138.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:29:41.495 INFO:teuthology.orchestra.run.smithi094.stdout:1302 trunc from 0xc983427 to 0xa0593f3 2024-09-14T06:29:41.505 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-14T06:29:41.505 INFO:teuthology.orchestra.run.smithi094.stdout:1303 trunc from 0xa0593f3 to 0xa48c0af 2024-09-14T06:29:41.510 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-14T06:29:41.510 INFO:teuthology.orchestra.run.smithi094.stdout:1305 write 0x18e797b thru 0x18f1d9c (0xa422 bytes) 2024-09-14T06:29:41.512 INFO:teuthology.orchestra.run.smithi094.stdout:1306 punch from 0x167c273 to 0x16807cb, (0x4558 bytes) 2024-09-14T06:29:41.512 INFO:teuthology.orchestra.run.smithi094.stdout:1307 punch from 0x12ae636 to 0x12af2dc, (0xca6 bytes) 2024-09-14T06:29:41.512 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-14T06:29:41.512 INFO:teuthology.orchestra.run.smithi094.stdout:1310 write 0x1c74a09 thru 0x1c7bc5c (0x7254 bytes) 2024-09-14T06:29:41.516 INFO:teuthology.orchestra.run.smithi094.stdout:1311 trunc from 0xa48c0af to 0x7c823a3 2024-09-14T06:29:41.523 INFO:teuthology.orchestra.run.smithi094.stdout:1312 punch from 0x7417f3d to 0x7426edd, (0xefa0 bytes) 2024-09-14T06:29:41.523 INFO:teuthology.orchestra.run.smithi094.stdout:1314 write 0x537847a thru 0x5378d4f (0x8d6 bytes) 2024-09-14T06:29:41.526 INFO:teuthology.orchestra.run.smithi094.stdout:1315 punch from 0x12b0b14 to 0x12b31f4, (0x26e0 bytes) 2024-09-14T06:29:41.531 INFO:teuthology.orchestra.run.smithi094.stdout:1316 trunc from 0x7c823a3 to 0x3769b36 2024-09-14T06:29:41.542 INFO:teuthology.orchestra.run.smithi094.stdout:1317 punch from 0x1599bbe to 0x15a85d8, (0xea1a bytes) 2024-09-14T06:29:41.542 INFO:teuthology.orchestra.run.smithi094.stdout:1318 punch from 0x1d74970 to 0x1d7bc0e, (0x729e bytes) 2024-09-14T06:29:41.542 INFO:teuthology.orchestra.run.smithi094.stdout:1319 write 0x2907073 thru 0x290f52f (0x84bd bytes) 2024-09-14T06:29:41.558 INFO:teuthology.orchestra.run.smithi094.stdout:1320 write 0x8a8e1d9 thru 0x8a95c36 (0x7a5e bytes) 2024-09-14T06:29:41.562 INFO:teuthology.orchestra.run.smithi094.stdout:1321 punch from 0x5c59dc0 to 0x5c5ad70, (0xfb0 bytes) 2024-09-14T06:29:41.562 INFO:teuthology.orchestra.run.smithi094.stdout:1322 read 0x6e9c334 thru 0x6ea622f (0x9efc bytes) 2024-09-14T06:29:41.562 INFO:teuthology.orchestra.run.smithi094.stdout:1325 write 0x499474e thru 0x4996047 (0x18fa bytes) 2024-09-14T06:29:41.565 INFO:teuthology.orchestra.run.smithi094.stdout:1326 read 0x6eed79a thru 0x6ef0643 (0x2eaa bytes) 2024-09-14T06:29:41.565 INFO:teuthology.orchestra.run.smithi094.stdout:1327 punch from 0x47a411f to 0x47ab007, (0x6ee8 bytes) 2024-09-14T06:29:41.565 INFO:teuthology.orchestra.run.smithi094.stdout:1328 read 0x7fc7706 thru 0x7fcf28f (0x7b8a bytes) 2024-09-14T06:29:41.565 INFO:teuthology.orchestra.run.smithi094.stdout:1330 punch from 0x2f4865b to 0x2f4e7ec, (0x6191 bytes) 2024-09-14T06:29:41.566 INFO:teuthology.orchestra.run.smithi094.stdout:1333 read 0x3c363a9 thru 0x3c3f315 (0x8f6d bytes) 2024-09-14T06:29:41.566 INFO:teuthology.orchestra.run.smithi094.stdout:1334 read 0x7c17faa thru 0x7c180c2 (0x119 bytes) 2024-09-14T06:29:41.568 INFO:teuthology.orchestra.run.smithi094.stdout:1337 trunc from 0x8a95c37 to 0x627d9dd 2024-09-14T06:29:41.578 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-14T06:29:41.578 INFO:teuthology.orchestra.run.smithi094.stdout:1340 write 0x5aed15c thru 0x5afbac3 (0xe968 bytes) 2024-09-14T06:29:41.582 INFO:teuthology.orchestra.run.smithi094.stdout:1342 punch from 0x431f359 to 0x4323bd3, (0x487a bytes) 2024-09-14T06:29:41.582 INFO:teuthology.orchestra.run.smithi094.stdout:1344 read 0x241bc78 thru 0x24240e2 (0x846b bytes) 2024-09-14T06:29:41.585 INFO:tasks.ceph.osd.6.smithi138.stderr:2024-09-14T06:29:41.584+0000 7f8cd8849640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33620) UID: 0 2024-09-14T06:29:41.593 INFO:teuthology.orchestra.run.smithi094.stdout:1345 write 0x9c57bb5 thru 0x9c5d752 (0x5b9e bytes) 2024-09-14T06:29:41.596 INFO:teuthology.orchestra.run.smithi094.stdout:1346 punch from 0x5024429 to 0x502adee, (0x69c5 bytes) 2024-09-14T06:29:41.606 INFO:teuthology.orchestra.run.smithi094.stdout:1347 writesame 0xcebffa2 thru 0xcec18ae data_size 0xb(0x190d bytes) 2024-09-14T06:29:41.606 INFO:teuthology.orchestra.run.smithi094.stdout:1348 read 0x65bdcd9 thru 0x65c9c47 (0xbf6f bytes) 2024-09-14T06:29:41.606 INFO:teuthology.orchestra.run.smithi094.stdout:1351 read 0xba19274 thru 0xba1e77e (0x550b bytes) 2024-09-14T06:29:41.606 INFO:teuthology.orchestra.run.smithi094.stdout:1353 write 0xa5af5c3 thru 0xa5b0a64 (0x14a2 bytes) 2024-09-14T06:29:41.609 INFO:teuthology.orchestra.run.smithi094.stdout:1355 write 0x59f1c40 thru 0x59f889f (0x6c60 bytes) 2024-09-14T06:29:41.613 INFO:teuthology.orchestra.run.smithi094.stdout:1356 write 0xc235717 thru 0xc244ca9 (0xf593 bytes) 2024-09-14T06:29:41.616 INFO:teuthology.orchestra.run.smithi094.stdout:1357 read 0x31259ae thru 0x3133ea0 (0xe4f3 bytes) 2024-09-14T06:29:41.616 INFO:teuthology.orchestra.run.smithi094.stdout:1359 writesame 0x5b43626 thru 0x5b46ce6 data_size 0x1(0x36c1 bytes) 2024-09-14T06:29:41.617 INFO:teuthology.orchestra.run.smithi138.stdout:ERROR: (22) Invalid argument 2024-09-14T06:29:41.617 INFO:teuthology.orchestra.run.smithi138.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:29:41.627 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-14T06:29:41.631 INFO:teuthology.orchestra.run.smithi094.stdout:1360 trunc from 0xcec18af to 0x8c2e9a6 2024-09-14T06:29:41.644 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-14T06:29:41.644 INFO:teuthology.orchestra.run.smithi094.stdout:1362 writesame 0x6f006f6 thru 0x6f037b7 data_size 0x2(0x30c2 bytes) 2024-09-14T06:29:41.648 INFO:teuthology.orchestra.run.smithi094.stdout:1363 read 0x87e1af4 thru 0x87ec26b (0xa778 bytes) 2024-09-14T06:29:41.655 INFO:teuthology.orchestra.run.smithi094.stdout:1364 write 0x9bf6fee thru 0x9c036a6 (0xc6b9 bytes) 2024-09-14T06:29:41.664 INFO:teuthology.orchestra.run.smithi094.stdout:1365 write 0x9ceabd1 thru 0x9ceb402 (0x832 bytes) 2024-09-14T06:29:41.674 INFO:teuthology.orchestra.run.smithi094.stdout:1366 write 0xb0e7259 thru 0xb0e9c30 (0x29d8 bytes) 2024-09-14T06:29:41.677 INFO:teuthology.orchestra.run.smithi094.stdout:1367 write 0x4cdaa61 thru 0x4cdce38 (0x23d8 bytes) 2024-09-14T06:29:41.680 INFO:teuthology.orchestra.run.smithi094.stdout:1368 read 0x958b540 thru 0x959a6ee (0xf1af bytes) 2024-09-14T06:29:41.680 INFO:teuthology.orchestra.run.smithi094.stdout:1369 punch from 0x11cf6d4 to 0x11d3cfa, (0x4626 bytes) 2024-09-14T06:29:41.681 INFO:teuthology.orchestra.run.smithi094.stdout:1370 read 0x1a5c958 thru 0x1a6c43c (0xfae5 bytes) 2024-09-14T06:29:41.681 INFO:teuthology.orchestra.run.smithi094.stdout:1371 writesame 0x4a1937c thru 0x4a1a885 data_size 0x1(0x150a bytes) 2024-09-14T06:29:41.686 INFO:teuthology.orchestra.run.smithi094.stdout:1373 write 0x2855553 thru 0x285e130 (0x8bde bytes) 2024-09-14T06:29:41.688 INFO:teuthology.orchestra.run.smithi094.stdout:1374 read 0x7c36c4a thru 0x7c38b38 (0x1eef bytes) 2024-09-14T06:29:41.688 INFO:teuthology.orchestra.run.smithi094.stdout:1375 read 0x871a27 thru 0x87713f (0x5719 bytes) 2024-09-14T06:29:41.689 INFO:teuthology.orchestra.run.smithi094.stdout:1376 punch from 0x430bb5d to 0x43110fe, (0x55a1 bytes) 2024-09-14T06:29:41.689 INFO:teuthology.orchestra.run.smithi094.stdout:1378 read 0x4dc408d thru 0x4dd3454 (0xf3c8 bytes) 2024-09-14T06:29:41.689 INFO:teuthology.orchestra.run.smithi094.stdout:1379 punch from 0x3e78606 to 0x3e7d8a9, (0x52a3 bytes) 2024-09-14T06:29:41.689 INFO:teuthology.orchestra.run.smithi094.stdout:1380 read 0x2f34d5a thru 0x2f44287 (0xf52e bytes) 2024-09-14T06:29:41.689 INFO:teuthology.orchestra.run.smithi094.stdout:1383 read 0x95709d thru 0x967003 (0xff67 bytes) 2024-09-14T06:29:41.698 INFO:teuthology.orchestra.run.smithi094.stdout:1384 trunc from 0xb0e9c31 to 0x36e9cdf 2024-09-14T06:29:41.711 INFO:tasks.ceph.osd.5.smithi138.stderr:2024-09-14T06:29:41.710+0000 7fe13746f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33618) UID: 0 2024-09-14T06:29:41.718 INFO:teuthology.orchestra.run.smithi094.stdout:1385 read 0xadc08a thru 0xae7820 (0xb797 bytes) 2024-09-14T06:29:41.718 INFO:teuthology.orchestra.run.smithi094.stdout:1386 read 0x728423 thru 0x736da9 (0xe987 bytes) 2024-09-14T06:29:41.719 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-14T06:29:41.739 INFO:teuthology.orchestra.run.smithi094.stdout:1388 write 0xe936ece thru 0xe941a68 (0xab9b bytes) 2024-09-14T06:29:41.740 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-14T06:29:41.743 INFO:teuthology.orchestra.run.smithi094.stdout:1389 write 0x2afdd3e thru 0x2b09e63 (0xc126 bytes) 2024-09-14T06:29:41.755 INFO:teuthology.orchestra.run.smithi094.stdout:1390 trunc from 0xe941a69 to 0x540b779 2024-09-14T06:29:41.766 INFO:teuthology.orchestra.run.smithi094.stdout:1392 read 0x4f816c3 thru 0x4f8fc9d (0xe5db bytes) 2024-09-14T06:29:41.767 INFO:teuthology.orchestra.run.smithi094.stdout:1396 punch from 0xb3feec to 0xb43c51, (0x3d65 bytes) 2024-09-14T06:29:41.767 INFO:teuthology.orchestra.run.smithi094.stdout:1400 read 0x38dea96 thru 0x38e47b6 (0x5d21 bytes) 2024-09-14T06:29:41.767 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-14T06:29:41.767 INFO:teuthology.orchestra.run.smithi094.stdout:1402 punch from 0x21c20c4 to 0x21c77c7, (0x5703 bytes) 2024-09-14T06:29:41.767 INFO:teuthology.orchestra.run.smithi094.stdout:1403 read 0x20dad02 thru 0x20ddda0 (0x309f bytes) 2024-09-14T06:29:41.767 INFO:teuthology.orchestra.run.smithi094.stdout:1408 punch from 0x39886d5 to 0x39934f3, (0xae1e bytes) 2024-09-14T06:29:41.767 INFO:teuthology.orchestra.run.smithi094.stdout:1410 punch from 0x2376831 to 0x23816d8, (0xaea7 bytes) 2024-09-14T06:29:41.778 INFO:teuthology.orchestra.run.smithi138.stdout:ERROR: (22) Invalid argument 2024-09-14T06:29:41.778 INFO:teuthology.orchestra.run.smithi138.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:29:41.781 INFO:teuthology.orchestra.run.smithi094.stdout:1411 write 0xcab4a25 thru 0xcabdb5a (0x9136 bytes) 2024-09-14T06:29:41.784 INFO:teuthology.orchestra.run.smithi094.stdout:1414 writesame 0x5882332 thru 0x588e1cd data_size 0x16(0xbe9c bytes) 2024-09-14T06:29:41.786 INFO:tasks.ceph.osd.2.smithi094.stderr:2024-09-14T06:29:41.785+0000 7faf3269b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33649) UID: 0 2024-09-14T06:29:41.788 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-14T06:29:41.789 INFO:teuthology.orchestra.run.smithi094.stdout:1415 trunc from 0xcabdb5b to 0xb81813a 2024-09-14T06:29:41.795 INFO:teuthology.orchestra.run.smithi094.stdout:1417 read 0x32d437e thru 0x32de262 (0x9ee5 bytes) 2024-09-14T06:29:41.795 INFO:teuthology.orchestra.run.smithi094.stdout:1418 write 0x5c965e2 thru 0x5c9af91 (0x49b0 bytes) 2024-09-14T06:29:41.798 INFO:teuthology.orchestra.run.smithi094.stdout:1420 punch from 0xab74a65 to 0xab760b4, (0x164f bytes) 2024-09-14T06:29:41.798 INFO:teuthology.orchestra.run.smithi094.stdout:1421 writesame 0x60b3524 thru 0x60b3b60 data_size 0x1(0x63d bytes) 2024-09-14T06:29:41.809 INFO:teuthology.orchestra.run.smithi094.stdout:1422 write 0xdb8d33a thru 0xdb8ea34 (0x16fb bytes) 2024-09-14T06:29:41.812 INFO:teuthology.orchestra.run.smithi094.stdout:1423 read 0x8e97d16 thru 0x8ea3d32 (0xc01d bytes) 2024-09-14T06:29:41.812 INFO:teuthology.orchestra.run.smithi094.stdout:1424 read 0x393a2d thru 0x39e744 (0xad18 bytes) 2024-09-14T06:29:41.812 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-14T06:29:41.812 INFO:teuthology.orchestra.run.smithi094.stdout:1426 read 0x940f7a2 thru 0x941d6cf (0xdf2e bytes) 2024-09-14T06:29:41.817 INFO:teuthology.orchestra.run.smithi094.stdout:1428 trunc from 0xdb8ea35 to 0x9a9a3d4 2024-09-14T06:29:41.828 INFO:teuthology.orchestra.run.smithi094.stdout:1430 writesame 0x927846 thru 0x92d1b9 data_size 0x32(0x5974 bytes) 2024-09-14T06:29:41.829 INFO:teuthology.orchestra.run.smithi094.stdout:1432 read 0x581f4a0 thru 0x58204a2 (0x1003 bytes) 2024-09-14T06:29:41.829 INFO:teuthology.orchestra.run.smithi094.stdout:1433 read 0x1409ed1 thru 0x1415fb7 (0xc0e7 bytes) 2024-09-14T06:29:41.830 INFO:teuthology.orchestra.run.smithi094.stdout:1434 punch from 0x14a20f3 to 0x14a4b03, (0x2a10 bytes) 2024-09-14T06:29:41.830 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-14T06:29:41.830 INFO:teuthology.orchestra.run.smithi094.stdout:1438 read 0x156b803 thru 0x1578f08 (0xd706 bytes) 2024-09-14T06:29:41.831 INFO:teuthology.orchestra.run.smithi094.stdout:1439 writesame 0x1416411 thru 0x14194f0 data_size 0x2e(0x30e0 bytes) 2024-09-14T06:29:41.832 INFO:teuthology.orchestra.run.smithi094.stdout:1440 punch from 0x68eba49 to 0x68f52ae, (0x9865 bytes) 2024-09-14T06:29:41.832 INFO:teuthology.orchestra.run.smithi094.stdout:1441 read 0x2c597b7 thru 0x2c5cf8b (0x37d5 bytes) 2024-09-14T06:29:41.832 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-14T06:29:41.844 INFO:teuthology.orchestra.run.smithi094.stdout:1444 write 0xe73da4f thru 0xe746b17 (0x90c9 bytes) 2024-09-14T06:29:41.847 INFO:teuthology.orchestra.run.smithi094.stdout:1445 write 0x3ab159f thru 0x3ac09ad (0xf40f bytes) 2024-09-14T06:29:41.850 INFO:teuthology.orchestra.run.smithi094.stdout:1446 punch from 0x5f80422 to 0x5f87ef1, (0x7acf bytes) 2024-09-14T06:29:41.854 INFO:teuthology.orchestra.run.smithi094.stdout:1447 trunc from 0xe746b18 to 0xb1a748e 2024-09-14T06:29:41.863 INFO:teuthology.orchestra.run.smithi094.stdout:1448 read 0x71a9d51 thru 0x71b5f07 (0xc1b7 bytes) 2024-09-14T06:29:41.873 INFO:teuthology.orchestra.run.smithi094.stdout:1449 write 0xeacad9b thru 0xeadaa31 (0xfc97 bytes) 2024-09-14T06:29:41.877 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-14T06:29:41.877 INFO:teuthology.orchestra.run.smithi094.stdout:1451 write 0xcad52bc thru 0xcad9d82 (0x4ac7 bytes) 2024-09-14T06:29:41.879 INFO:teuthology.orchestra.run.smithi094.stdout:1452 read 0xe636ee6 thru 0xe63bc34 (0x4d4f bytes) 2024-09-14T06:29:41.881 INFO:teuthology.orchestra.run.smithi094.stdout:1453 trunc from 0xeadaa32 to 0xdaf94c1 2024-09-14T06:29:41.887 INFO:tasks.ceph.osd.5.smithi138.stderr:2024-09-14T06:29:41.886+0000 7fe13746f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33618) UID: 0 2024-09-14T06:29:41.891 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-14T06:29:41.891 INFO:teuthology.orchestra.run.smithi094.stdout:1457 read 0x2f2ee18 thru 0x2f2f484 (0x66d bytes) 2024-09-14T06:29:41.891 INFO:teuthology.orchestra.run.smithi094.stdout:1458 clone 11 order 20 su 4096 sc 6 2024-09-14T06:29:41.936 INFO:teuthology.orchestra.run.smithi138.stdout:ERROR: (22) Invalid argument 2024-09-14T06:29:41.936 INFO:teuthology.orchestra.run.smithi138.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:29:41.944 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-14T06:29:42.028 INFO:tasks.ceph.osd.5.smithi138.stderr:2024-09-14T06:29:42.027+0000 7fe13746f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33618) UID: 0 2024-09-14T06:29:42.087 INFO:teuthology.orchestra.run.smithi138.stdout:ERROR: (22) Invalid argument 2024-09-14T06:29:42.087 INFO:teuthology.orchestra.run.smithi138.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:29:42.089 INFO:tasks.ceph.osd.3.smithi094.stderr:2024-09-14T06:29:42.088+0000 7feeeafe8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33655) UID: 0 2024-09-14T06:29:42.095 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-14T06:29:42.190 INFO:tasks.ceph.osd.2.smithi094.stderr:2024-09-14T06:29:42.189+0000 7faf3269b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33649) UID: 0 2024-09-14T06:29:42.239 INFO:teuthology.orchestra.run.smithi138.stdout:ERROR: (22) Invalid argument 2024-09-14T06:29:42.239 INFO:teuthology.orchestra.run.smithi138.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:29:42.248 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-14T06:29:42.291 INFO:tasks.ceph.osd.6.smithi138.stderr:2024-09-14T06:29:42.290+0000 7f8cd8849640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33620) UID: 0 2024-09-14T06:29:42.352 INFO:teuthology.orchestra.run.smithi138.stdout:ERROR: (22) Invalid argument 2024-09-14T06:29:42.352 INFO:teuthology.orchestra.run.smithi138.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:29:42.360 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-14T06:29:42.402 INFO:tasks.ceph.osd.4.smithi138.stderr:2024-09-14T06:29:42.401+0000 7f6c653c8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33615) UID: 0 2024-09-14T06:29:42.493 INFO:tasks.ceph.osd.7.smithi138.stderr:2024-09-14T06:29:42.492+0000 7f3c6a21d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33623) UID: 0 2024-09-14T06:29:42.504 INFO:teuthology.orchestra.run.smithi138.stdout:ERROR: (22) Invalid argument 2024-09-14T06:29:42.504 INFO:teuthology.orchestra.run.smithi138.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:29:42.513 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-14T06:29:42.597 INFO:tasks.ceph.osd.4.smithi138.stderr:2024-09-14T06:29:42.596+0000 7f6c653c8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33615) UID: 0 2024-09-14T06:29:42.657 INFO:teuthology.orchestra.run.smithi138.stdout:ERROR: (22) Invalid argument 2024-09-14T06:29:42.657 INFO:teuthology.orchestra.run.smithi138.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:29:42.666 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-14T06:29:42.694 INFO:tasks.ceph.osd.0.smithi094.stderr:2024-09-14T06:29:42.693+0000 7f07c9333640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33636) UID: 0 2024-09-14T06:29:42.732 INFO:teuthology.orchestra.run.smithi094.stderr:noscrub is unset 2024-09-14T06:29:42.795 INFO:tasks.ceph.osd.4.smithi138.stderr:2024-09-14T06:29:42.794+0000 7f6c653c8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33615) UID: 0 2024-09-14T06:29:42.815 INFO:teuthology.orchestra.run.smithi138.stdout:ERROR: (22) Invalid argument 2024-09-14T06:29:42.815 INFO:teuthology.orchestra.run.smithi138.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:29:42.824 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-14T06:29:42.896 INFO:tasks.ceph.osd.1.smithi094.stderr:2024-09-14T06:29:42.895+0000 7f328e47a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33641) UID: 0 2024-09-14T06:29:42.967 INFO:teuthology.orchestra.run.smithi138.stdout:ERROR: (22) Invalid argument 2024-09-14T06:29:42.967 INFO:teuthology.orchestra.run.smithi138.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:29:42.975 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-14T06:29:43.019 INFO:tasks.ceph.osd.7.smithi138.stderr:2024-09-14T06:29:43.018+0000 7f3c6a21d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33623) UID: 0 2024-09-14T06:29:43.077 INFO:teuthology.orchestra.run.smithi138.stdout:ERROR: (22) Invalid argument 2024-09-14T06:29:43.078 INFO:teuthology.orchestra.run.smithi138.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:29:43.086 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-14T06:29:43.130 INFO:tasks.ceph.osd.0.smithi094.stderr:2024-09-14T06:29:43.129+0000 7f07c9333640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33636) UID: 0 2024-09-14T06:29:43.199 INFO:tasks.ceph.osd.5.smithi138.stderr:2024-09-14T06:29:43.198+0000 7fe13746f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33618) UID: 0 2024-09-14T06:29:43.205 INFO:teuthology.orchestra.run.smithi094.stdout:ERROR: (22) Invalid argument 2024-09-14T06:29:43.205 INFO:teuthology.orchestra.run.smithi094.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:29:43.214 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-14T06:29:43.300 INFO:tasks.ceph.osd.2.smithi094.stderr:2024-09-14T06:29:43.299+0000 7faf3269b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33649) UID: 0 2024-09-14T06:29:43.361 INFO:teuthology.orchestra.run.smithi094.stdout:ERROR: (22) Invalid argument 2024-09-14T06:29:43.361 INFO:teuthology.orchestra.run.smithi094.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:29:43.369 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-14T06:29:43.401 INFO:tasks.ceph.osd.4.smithi138.stderr:2024-09-14T06:29:43.400+0000 7f6c653c8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33615) UID: 0 2024-09-14T06:29:43.433 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-14T06:29:43.502 INFO:tasks.ceph.osd.6.smithi138.stderr:2024-09-14T06:29:43.501+0000 7f8cd8849640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33620) UID: 0 2024-09-14T06:29:43.515 INFO:teuthology.orchestra.run.smithi094.stdout:ERROR: (22) Invalid argument 2024-09-14T06:29:43.515 INFO:teuthology.orchestra.run.smithi094.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:29:43.524 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-14T06:29:43.578 INFO:teuthology.orchestra.run.smithi094.stdout:osd.0: {} 2024-09-14T06:29:43.578 INFO:teuthology.orchestra.run.smithi094.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-14T06:29:43.594 INFO:teuthology.orchestra.run.smithi094.stdout:osd.1: {} 2024-09-14T06:29:43.595 INFO:teuthology.orchestra.run.smithi094.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-14T06:29:43.604 INFO:tasks.ceph.osd.6.smithi138.stderr:2024-09-14T06:29:43.602+0000 7f8cd8849640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33620) UID: 0 2024-09-14T06:29:43.611 INFO:teuthology.orchestra.run.smithi094.stdout:osd.2: {} 2024-09-14T06:29:43.611 INFO:teuthology.orchestra.run.smithi094.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-14T06:29:43.627 INFO:teuthology.orchestra.run.smithi094.stdout:osd.3: {} 2024-09-14T06:29:43.628 INFO:teuthology.orchestra.run.smithi094.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-14T06:29:43.631 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-14T06:29:43.631 INFO:teuthology.orchestra.run.smithi094.stdout: "ops": [], 2024-09-14T06:29:43.631 INFO:teuthology.orchestra.run.smithi094.stdout: "num_ops": 0 2024-09-14T06:29:43.632 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-14T06:29:43.640 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-14T06:29:43.644 INFO:teuthology.orchestra.run.smithi094.stdout:osd.4: {} 2024-09-14T06:29:43.644 INFO:teuthology.orchestra.run.smithi094.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-14T06:29:43.661 INFO:teuthology.orchestra.run.smithi094.stdout:osd.5: {} 2024-09-14T06:29:43.661 INFO:teuthology.orchestra.run.smithi094.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-14T06:29:43.677 INFO:teuthology.orchestra.run.smithi094.stdout:osd.6: {} 2024-09-14T06:29:43.677 INFO:teuthology.orchestra.run.smithi094.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-14T06:29:43.695 INFO:teuthology.orchestra.run.smithi094.stdout:osd.7: {} 2024-09-14T06:29:43.696 INFO:teuthology.orchestra.run.smithi094.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-14T06:29:43.705 INFO:tasks.ceph.osd.5.smithi138.stderr:2024-09-14T06:29:43.703+0000 7fe13746f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33618) UID: 0 2024-09-14T06:29:43.714 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-14T06:29:43.714 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-14T06:29:43.715 INFO:tasks.thrashosds.thrasher:Reweighting by: {'max_change': '0.05', 'overage': '1000', 'type': 'reweight-by-utilization'} 2024-09-14T06:29:43.715 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight-by-utilization 1000 0.05 2024-09-14T06:29:43.750 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-14T06:29:43.750 INFO:teuthology.orchestra.run.smithi094.stdout: "ops": [], 2024-09-14T06:29:43.750 INFO:teuthology.orchestra.run.smithi094.stdout: "complaint_time": 30, 2024-09-14T06:29:43.750 INFO:teuthology.orchestra.run.smithi094.stdout: "num_blocked_ops": 0 2024-09-14T06:29:43.750 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-14T06:29:43.765 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-14T06:29:43.808 INFO:tasks.ceph.osd.2.smithi094.stderr:2024-09-14T06:29:43.807+0000 7faf3269b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33649) UID: 0 2024-09-14T06:29:43.907 INFO:tasks.ceph.osd.6.smithi138.stderr:2024-09-14T06:29:43.905+0000 7f8cd8849640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33620) UID: 0 2024-09-14T06:29:43.974 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-14T06:29:43.975 INFO:teuthology.orchestra.run.smithi094.stdout: "size": 20, 2024-09-14T06:29:43.975 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 600, 2024-09-14T06:29:43.975 INFO:teuthology.orchestra.run.smithi094.stdout: "ops": [ 2024-09-14T06:29:43.975 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:43.975 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:14910 3.0 3:05c39535:::rbd_data.112d92027590.0000000000000009:f [read 713728~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:29:43.975 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:26.083696+0000", 2024-09-14T06:29:43.975 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 17.888244087, 2024-09-14T06:29:43.975 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.040571832000000002, 2024-09-14T06:29:43.975 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:29:43.976 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:29:43.976 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:29:43.976 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:29:43.976 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:29:43.976 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:43.976 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 14910 2024-09-14T06:29:43.976 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:43.976 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:29:43.976 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:43.976 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:29:43.976 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.083696+0000", 2024-09-14T06:29:43.976 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:43.976 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:43.976 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:43.977 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:29:43.977 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.083696+0000", 2024-09-14T06:29:43.977 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:43.977 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:43.977 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:43.977 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:29:43.977 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.083696+0000", 2024-09-14T06:29:43.977 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.0699999999999997e-07 2024-09-14T06:29:43.977 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:43.977 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:43.977 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:29:43.977 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.083698+0000", 2024-09-14T06:29:43.977 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.9429999999999999e-06 2024-09-14T06:29:43.978 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:43.978 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:43.978 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:29:43.978 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.083708+0000", 2024-09-14T06:29:43.978 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.6430000000000007e-06 2024-09-14T06:29:43.978 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:43.978 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:43.978 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:29:43.978 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.083719+0000", 2024-09-14T06:29:43.978 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.1017e-05 2024-09-14T06:29:43.978 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:43.978 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:43.978 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:29:43.978 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.124152+0000", 2024-09-14T06:29:43.979 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.040433104999999997 2024-09-14T06:29:43.979 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:43.979 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:43.979 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:29:43.979 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.124194+0000", 2024-09-14T06:29:43.979 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.1585999999999998e-05 2024-09-14T06:29:43.979 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:43.979 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:43.979 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:29:43.979 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.124268+0000", 2024-09-14T06:29:43.979 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.4030999999999996e-05 2024-09-14T06:29:43.979 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:43.979 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:43.979 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:43.980 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:43.980 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:43.980 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:14911 3.0 3:05c39535:::rbd_data.112d92027590.0000000000000009:f [read 723968~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:29:43.980 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:26.083729+0000", 2024-09-14T06:29:43.980 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 17.88821098, 2024-09-14T06:29:43.980 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.040746899000000003, 2024-09-14T06:29:43.980 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:29:43.980 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:29:43.980 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:29:43.980 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:29:43.980 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:29:43.980 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:43.980 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 14911 2024-09-14T06:29:43.980 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:43.981 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:29:43.981 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:43.981 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:29:43.981 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.083729+0000", 2024-09-14T06:29:43.981 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:43.981 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:43.981 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:43.981 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:29:43.981 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.083729+0000", 2024-09-14T06:29:43.981 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:43.981 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:43.981 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:43.981 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:29:43.982 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.083729+0000", 2024-09-14T06:29:43.982 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.7399999999999998e-07 2024-09-14T06:29:43.982 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:43.982 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:43.982 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:29:43.982 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.083731+0000", 2024-09-14T06:29:43.982 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.9709999999999998e-06 2024-09-14T06:29:43.982 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:43.982 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:43.982 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:29:43.982 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.083737+0000", 2024-09-14T06:29:43.982 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.801e-06 2024-09-14T06:29:43.982 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:43.982 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:43.983 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:29:43.983 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.083743+0000", 2024-09-14T06:29:43.983 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.9529999999999996e-06 2024-09-14T06:29:43.983 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:43.983 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:43.983 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:29:43.983 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.124306+0000", 2024-09-14T06:29:43.983 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.040562333999999999 2024-09-14T06:29:43.983 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:43.983 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:43.983 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:29:43.983 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.124384+0000", 2024-09-14T06:29:43.983 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.8110999999999998e-05 2024-09-14T06:29:43.983 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:43.984 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:43.984 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:29:43.984 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.124476+0000", 2024-09-14T06:29:43.984 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.2255000000000004e-05 2024-09-14T06:29:43.984 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:43.984 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:43.984 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:43.984 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:43.984 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:43.984 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:14912 3.0 3:05c39535:::rbd_data.112d92027590.0000000000000009:f [read 734208~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:29:43.984 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:26.083749+0000", 2024-09-14T06:29:43.984 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 17.888190843, 2024-09-14T06:29:43.984 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.040874964, 2024-09-14T06:29:43.984 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:29:43.985 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:29:43.985 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:29:43.985 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:29:43.985 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:29:43.985 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:43.985 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 14912 2024-09-14T06:29:43.985 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:43.985 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:29:43.985 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:43.985 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:29:43.985 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.083749+0000", 2024-09-14T06:29:43.985 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:43.985 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:43.985 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:43.986 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:29:43.986 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.083749+0000", 2024-09-14T06:29:43.986 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:43.986 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:43.986 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:43.986 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:29:43.986 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.083749+0000", 2024-09-14T06:29:43.986 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.58e-07 2024-09-14T06:29:43.986 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:43.986 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:43.986 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:29:43.986 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.083751+0000", 2024-09-14T06:29:43.986 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.7910000000000001e-06 2024-09-14T06:29:43.986 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:43.986 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:43.987 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:29:43.987 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.083758+0000", 2024-09-14T06:29:43.987 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.3799999999999999e-06 2024-09-14T06:29:43.987 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:43.987 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:43.987 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:29:43.987 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.083761+0000", 2024-09-14T06:29:43.987 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.8480000000000003e-06 2024-09-14T06:29:43.987 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:43.987 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:43.987 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:29:43.987 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.124514+0000", 2024-09-14T06:29:43.987 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.040752743000000001 2024-09-14T06:29:43.987 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:43.987 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:43.988 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:29:43.988 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.124554+0000", 2024-09-14T06:29:43.988 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.0209000000000001e-05 2024-09-14T06:29:43.988 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:43.988 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:43.988 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:29:43.988 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.124624+0000", 2024-09-14T06:29:43.988 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.9635000000000007e-05 2024-09-14T06:29:43.988 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:43.988 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:43.988 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:43.988 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:43.988 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:43.989 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:14913 3.0 3:05c39535:::rbd_data.112d92027590.0000000000000009:f [read 744448~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:29:43.989 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:26.083766+0000", 2024-09-14T06:29:43.989 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 17.888173545000001, 2024-09-14T06:29:43.989 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.041064829999999997, 2024-09-14T06:29:43.989 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:29:43.989 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:29:43.989 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:29:43.989 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:29:43.989 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:29:43.989 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:43.989 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 14913 2024-09-14T06:29:43.989 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:43.989 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:29:43.990 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:43.990 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:29:43.990 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.083766+0000", 2024-09-14T06:29:43.990 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:43.990 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:43.990 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:43.990 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:29:43.990 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.083766+0000", 2024-09-14T06:29:43.990 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:43.990 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:43.990 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:43.990 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:29:43.990 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.083767+0000", 2024-09-14T06:29:43.990 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.1399999999999998e-07 2024-09-14T06:29:43.991 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:43.991 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:43.991 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:29:43.991 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.083768+0000", 2024-09-14T06:29:43.991 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.1629999999999999e-06 2024-09-14T06:29:43.991 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:43.991 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:43.991 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:29:43.991 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.083771+0000", 2024-09-14T06:29:43.991 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.2030000000000001e-06 2024-09-14T06:29:43.991 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:43.991 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:43.991 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:29:43.991 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.083774+0000", 2024-09-14T06:29:43.992 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.627e-06 2024-09-14T06:29:43.992 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:43.992 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:43.992 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:29:43.992 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.124662+0000", 2024-09-14T06:29:43.992 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.040887973000000001 2024-09-14T06:29:43.992 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:43.992 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:43.992 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:29:43.992 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.124741+0000", 2024-09-14T06:29:43.992 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.8888999999999996e-05 2024-09-14T06:29:43.992 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:43.992 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:43.993 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:29:43.993 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.124831+0000", 2024-09-14T06:29:43.993 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.0661000000000005e-05 2024-09-14T06:29:43.993 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:43.993 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:43.993 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:43.993 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:43.993 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:43.993 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:14914 3.0 3:05c39535:::rbd_data.112d92027590.0000000000000009:f [read 754688~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:29:43.993 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:26.083777+0000", 2024-09-14T06:29:43.993 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 17.888163024000001, 2024-09-14T06:29:43.993 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.041197828999999998, 2024-09-14T06:29:43.993 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:29:43.993 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:29:43.994 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:29:43.994 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:29:43.994 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:29:43.994 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:43.994 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 14914 2024-09-14T06:29:43.994 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:43.994 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:29:43.994 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:43.994 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:29:43.994 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.083777+0000", 2024-09-14T06:29:43.994 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:43.994 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:43.994 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:43.995 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:29:43.995 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.083777+0000", 2024-09-14T06:29:43.995 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:43.995 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:43.995 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:43.995 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:29:43.995 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.083777+0000", 2024-09-14T06:29:43.995 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.0200000000000001e-07 2024-09-14T06:29:43.995 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:43.995 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:43.995 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:29:43.995 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.083778+0000", 2024-09-14T06:29:43.995 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.3699999999999999e-07 2024-09-14T06:29:43.995 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:43.995 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:43.996 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:29:43.996 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.083782+0000", 2024-09-14T06:29:43.996 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.563e-06 2024-09-14T06:29:43.996 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:43.996 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:43.996 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:29:43.996 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.083784+0000", 2024-09-14T06:29:43.996 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.508e-06 2024-09-14T06:29:43.996 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:43.996 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:43.996 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:29:43.996 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.124870+0000", 2024-09-14T06:29:43.996 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.041086077999999998 2024-09-14T06:29:43.996 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:43.997 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:43.997 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:29:43.997 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.124912+0000", 2024-09-14T06:29:43.997 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.1742000000000001e-05 2024-09-14T06:29:43.997 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:43.997 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:43.997 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:29:43.997 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.124975+0000", 2024-09-14T06:29:43.997 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.2799000000000006e-05 2024-09-14T06:29:43.997 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:43.997 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:43.997 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:43.997 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:43.998 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:43.998 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:14915 3.0 3:05c39535:::rbd_data.112d92027590.0000000000000009:f [read 764928~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:29:43.998 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:26.083787+0000", 2024-09-14T06:29:43.998 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 17.888152673, 2024-09-14T06:29:43.998 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.041359554999999999, 2024-09-14T06:29:43.998 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:29:43.998 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:29:43.998 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:29:43.998 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:29:43.998 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:29:43.998 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:43.998 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 14915 2024-09-14T06:29:43.998 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:43.998 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:29:43.999 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:43.999 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:29:43.999 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.083787+0000", 2024-09-14T06:29:43.999 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:43.999 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:43.999 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:43.999 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:29:43.999 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.083787+0000", 2024-09-14T06:29:43.999 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:43.999 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:43.999 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:43.999 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:29:43.999 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.083788+0000", 2024-09-14T06:29:43.999 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.2499999999999999e-07 2024-09-14T06:29:44.000 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:44.000 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:44.000 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:29:44.000 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.083788+0000", 2024-09-14T06:29:44.000 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.02e-07 2024-09-14T06:29:44.000 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:44.000 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:44.000 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:29:44.000 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.083791+0000", 2024-09-14T06:29:44.000 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.8849999999999999e-06 2024-09-14T06:29:44.000 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:44.000 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:44.000 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:29:44.001 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.083794+0000", 2024-09-14T06:29:44.001 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.587e-06 2024-09-14T06:29:44.001 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:44.001 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:44.001 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:29:44.001 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.125014+0000", 2024-09-14T06:29:44.001 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.041220075000000002 2024-09-14T06:29:44.001 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:44.001 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:44.001 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:29:44.001 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.125066+0000", 2024-09-14T06:29:44.001 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.1632000000000003e-05 2024-09-14T06:29:44.001 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:44.001 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:44.002 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:29:44.002 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.125147+0000", 2024-09-14T06:29:44.002 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.1248999999999994e-05 2024-09-14T06:29:44.002 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:44.002 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:44.002 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:44.002 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:44.002 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:44.002 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:14916 3.0 3:05c39535:::rbd_data.112d92027590.0000000000000009:f [read 775168~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:29:44.002 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:26.083797+0000", 2024-09-14T06:29:44.002 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 17.888142885000001, 2024-09-14T06:29:44.002 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.041475420999999998, 2024-09-14T06:29:44.002 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:29:44.003 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:29:44.003 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:29:44.003 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:29:44.003 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:29:44.003 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:44.003 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 14916 2024-09-14T06:29:44.003 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:44.003 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:29:44.003 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:44.003 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:29:44.003 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.083797+0000", 2024-09-14T06:29:44.003 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:44.003 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:44.003 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:44.004 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:29:44.004 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.083797+0000", 2024-09-14T06:29:44.004 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:44.004 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:44.004 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:44.004 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:29:44.004 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.083797+0000", 2024-09-14T06:29:44.004 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.2100000000000001e-07 2024-09-14T06:29:44.004 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:44.004 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:44.004 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:29:44.004 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.083812+0000", 2024-09-14T06:29:44.004 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.4630000000000001e-05 2024-09-14T06:29:44.004 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:44.005 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:44.005 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:29:44.005 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.083817+0000", 2024-09-14T06:29:44.005 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.0719999999999999e-06 2024-09-14T06:29:44.005 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:44.005 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:44.005 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:29:44.005 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.083820+0000", 2024-09-14T06:29:44.005 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.8150000000000002e-06 2024-09-14T06:29:44.005 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:44.005 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:44.005 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:29:44.005 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.125180+0000", 2024-09-14T06:29:44.005 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.041360643000000002 2024-09-14T06:29:44.006 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:44.006 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:44.006 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:29:44.006 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.125221+0000", 2024-09-14T06:29:44.006 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.0442999999999999e-05 2024-09-14T06:29:44.006 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:44.006 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:44.006 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:29:44.006 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.125272+0000", 2024-09-14T06:29:44.006 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.1597000000000001e-05 2024-09-14T06:29:44.006 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:44.006 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:44.006 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:44.006 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:44.007 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:44.007 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:14917 3.0 3:05c39535:::rbd_data.112d92027590.0000000000000009:f [read 785408~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:29:44.007 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:26.083824+0000", 2024-09-14T06:29:44.007 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 17.888116045, 2024-09-14T06:29:44.007 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.041594226999999998, 2024-09-14T06:29:44.007 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:29:44.007 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:29:44.007 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:29:44.007 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:29:44.007 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:29:44.007 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:44.007 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 14917 2024-09-14T06:29:44.007 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:44.007 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:29:44.008 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:44.008 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:29:44.008 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.083824+0000", 2024-09-14T06:29:44.008 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:44.008 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:44.008 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:44.008 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:29:44.008 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.083824+0000", 2024-09-14T06:29:44.008 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:44.008 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:44.008 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:44.008 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:29:44.008 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.083824+0000", 2024-09-14T06:29:44.008 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.23e-07 2024-09-14T06:29:44.009 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:44.009 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:44.009 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:29:44.009 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.083825+0000", 2024-09-14T06:29:44.009 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.0690000000000001e-06 2024-09-14T06:29:44.009 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:44.009 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:44.009 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:29:44.009 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.083828+0000", 2024-09-14T06:29:44.009 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.9840000000000001e-06 2024-09-14T06:29:44.009 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:44.009 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:44.009 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:29:44.010 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.083831+0000", 2024-09-14T06:29:44.010 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.4119999999999999e-06 2024-09-14T06:29:44.010 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:44.010 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:44.010 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:29:44.010 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.125303+0000", 2024-09-14T06:29:44.010 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.041472676999999999 2024-09-14T06:29:44.010 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:44.010 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:44.010 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:29:44.010 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.125358+0000", 2024-09-14T06:29:44.010 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.4654000000000002e-05 2024-09-14T06:29:44.010 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:44.010 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:44.010 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:29:44.011 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.125418+0000", 2024-09-14T06:29:44.011 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.0207999999999997e-05 2024-09-14T06:29:44.011 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:44.011 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:44.011 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:44.011 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:44.011 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:44.011 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:14918 3.0 3:05c39535:::rbd_data.112d92027590.0000000000000009:f [read 795648~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:29:44.011 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:26.083834+0000", 2024-09-14T06:29:44.011 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 17.888106336, 2024-09-14T06:29:44.011 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.041776287000000002, 2024-09-14T06:29:44.011 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:29:44.011 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:29:44.011 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:29:44.012 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:29:44.012 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:29:44.012 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:44.012 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 14918 2024-09-14T06:29:44.012 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:44.012 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:29:44.012 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:44.012 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:29:44.012 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.083834+0000", 2024-09-14T06:29:44.012 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:44.012 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:44.012 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:44.012 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:29:44.012 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.083834+0000", 2024-09-14T06:29:44.013 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:44.013 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:44.013 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:44.013 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:29:44.013 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.083834+0000", 2024-09-14T06:29:44.013 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.0100000000000001e-07 2024-09-14T06:29:44.013 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:44.013 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:44.013 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:29:44.013 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.083835+0000", 2024-09-14T06:29:44.013 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.3900000000000003e-07 2024-09-14T06:29:44.013 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:44.013 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:44.013 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:29:44.014 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.083838+0000", 2024-09-14T06:29:44.014 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.9830000000000001e-06 2024-09-14T06:29:44.014 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:44.014 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:44.014 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:29:44.014 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.083840+0000", 2024-09-14T06:29:44.014 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.4679999999999999e-06 2024-09-14T06:29:44.014 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:44.014 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:44.014 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:29:44.014 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.125455+0000", 2024-09-14T06:29:44.014 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.041614578999999999 2024-09-14T06:29:44.014 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:44.014 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:44.015 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:29:44.015 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.125496+0000", 2024-09-14T06:29:44.015 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.1047000000000002e-05 2024-09-14T06:29:44.015 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:44.015 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:44.015 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:29:44.015 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.125610+0000", 2024-09-14T06:29:44.015 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00011407 2024-09-14T06:29:44.015 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:44.015 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:44.015 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:44.015 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:44.015 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:44.016 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:14919 3.0 3:05c39535:::rbd_data.112d92027590.0000000000000009:f [read 805888~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:29:44.016 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:26.083843+0000", 2024-09-14T06:29:44.016 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 17.888096588, 2024-09-14T06:29:44.016 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.041992356000000002, 2024-09-14T06:29:44.016 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:29:44.016 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:29:44.016 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:29:44.016 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:29:44.016 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:29:44.016 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:44.016 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 14919 2024-09-14T06:29:44.016 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:44.016 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:29:44.016 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:44.017 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:29:44.017 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.083843+0000", 2024-09-14T06:29:44.017 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:44.017 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:44.017 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:44.017 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:29:44.017 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.083843+0000", 2024-09-14T06:29:44.017 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:44.017 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:44.017 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:44.017 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:29:44.017 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.083844+0000", 2024-09-14T06:29:44.017 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.3300000000000001e-07 2024-09-14T06:29:44.017 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:44.018 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:44.018 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:29:44.018 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.083845+0000", 2024-09-14T06:29:44.018 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.4729999999999999e-06 2024-09-14T06:29:44.018 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:44.018 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:44.018 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:29:44.018 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.083850+0000", 2024-09-14T06:29:44.018 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.0130000000000001e-06 2024-09-14T06:29:44.018 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:44.018 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:44.018 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:29:44.018 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.083854+0000", 2024-09-14T06:29:44.018 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.4759999999999998e-06 2024-09-14T06:29:44.019 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:44.019 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:44.019 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:29:44.019 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.125636+0000", 2024-09-14T06:29:44.019 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.041782683000000001 2024-09-14T06:29:44.019 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:44.019 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:44.019 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:29:44.019 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.125680+0000", 2024-09-14T06:29:44.019 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.3614e-05 2024-09-14T06:29:44.019 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:44.019 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:44.019 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:29:44.020 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.125836+0000", 2024-09-14T06:29:44.020 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000155864 2024-09-14T06:29:44.020 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:44.020 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:44.020 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:44.020 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:44.020 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:44.020 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:14920 3.0 3:05c39535:::rbd_data.112d92027590.0000000000000009:f [read 816128~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:29:44.020 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:26.083868+0000", 2024-09-14T06:29:44.020 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 17.888071926999999, 2024-09-14T06:29:44.020 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.042166771999999998, 2024-09-14T06:29:44.020 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:29:44.020 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:29:44.021 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:29:44.021 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:29:44.021 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:29:44.021 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:44.021 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 14920 2024-09-14T06:29:44.021 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:44.021 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:29:44.021 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:44.021 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:29:44.021 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.083868+0000", 2024-09-14T06:29:44.021 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:44.021 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:44.021 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:44.021 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:29:44.022 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.083868+0000", 2024-09-14T06:29:44.022 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:44.022 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:44.022 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:44.022 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:29:44.022 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.083869+0000", 2024-09-14T06:29:44.022 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.99e-07 2024-09-14T06:29:44.022 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:44.022 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:44.022 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:29:44.022 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.083870+0000", 2024-09-14T06:29:44.022 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.782e-06 2024-09-14T06:29:44.022 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:44.022 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:44.023 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:29:44.023 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.083876+0000", 2024-09-14T06:29:44.023 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.3809999999999998e-06 2024-09-14T06:29:44.023 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:44.023 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:44.023 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:29:44.023 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.083882+0000", 2024-09-14T06:29:44.023 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.9479999999999996e-06 2024-09-14T06:29:44.023 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:44.023 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:44.023 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:29:44.023 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.125878+0000", 2024-09-14T06:29:44.023 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.041996264999999998 2024-09-14T06:29:44.023 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:44.024 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:44.024 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:29:44.024 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.125929+0000", 2024-09-14T06:29:44.024 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.0498999999999999e-05 2024-09-14T06:29:44.024 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:44.024 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:44.024 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:29:44.024 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.126035+0000", 2024-09-14T06:29:44.024 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000106298 2024-09-14T06:29:44.024 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:44.024 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:44.024 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:44.024 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:44.024 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:44.025 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:14921 3.0 3:05c39535:::rbd_data.112d92027590.0000000000000009:f [read 826368~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:29:44.025 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:26.083889+0000", 2024-09-14T06:29:44.025 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 17.888050917000001, 2024-09-14T06:29:44.025 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.042435309999999997, 2024-09-14T06:29:44.025 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:29:44.025 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:29:44.025 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:29:44.025 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:29:44.025 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:29:44.025 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:44.025 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 14921 2024-09-14T06:29:44.025 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:44.025 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:29:44.026 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:44.026 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:29:44.026 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.083889+0000", 2024-09-14T06:29:44.026 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:44.026 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:44.026 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:44.026 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:29:44.026 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.083889+0000", 2024-09-14T06:29:44.026 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:44.026 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:44.026 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:44.026 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:29:44.026 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.086033+0000", 2024-09-14T06:29:44.026 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0021444179999999999 2024-09-14T06:29:44.027 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:44.027 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:44.027 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:29:44.027 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.086038+0000", 2024-09-14T06:29:44.027 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.0969999999999999e-06 2024-09-14T06:29:44.027 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:44.027 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:44.027 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:29:44.027 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.086046+0000", 2024-09-14T06:29:44.027 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.9440000000000005e-06 2024-09-14T06:29:44.027 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:44.027 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:44.027 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:29:44.027 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.086050+0000", 2024-09-14T06:29:44.028 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.69e-06 2024-09-14T06:29:44.028 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:44.028 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:44.028 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:29:44.028 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.126080+0000", 2024-09-14T06:29:44.028 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.040030237000000003 2024-09-14T06:29:44.028 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:44.028 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:44.028 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:29:44.028 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.126163+0000", 2024-09-14T06:29:44.028 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.2071999999999996e-05 2024-09-14T06:29:44.028 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:44.028 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:44.029 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:29:44.029 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.126324+0000", 2024-09-14T06:29:44.029 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00016185200000000001 2024-09-14T06:29:44.029 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:44.029 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:44.029 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:44.029 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:44.029 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:44.029 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:14922 3.0 3:05c39535:::rbd_data.112d92027590.0000000000000009:f [read 836608~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:29:44.029 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:26.086060+0000", 2024-09-14T06:29:44.029 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 17.885880200999999, 2024-09-14T06:29:44.029 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.040546885999999997, 2024-09-14T06:29:44.029 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:29:44.029 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:29:44.030 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:29:44.030 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:29:44.030 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:29:44.030 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:44.030 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 14922 2024-09-14T06:29:44.030 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:44.030 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:29:44.030 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:44.030 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:29:44.030 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.086060+0000", 2024-09-14T06:29:44.030 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:44.030 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:44.030 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:44.030 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:29:44.031 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.086060+0000", 2024-09-14T06:29:44.031 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:44.031 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:44.031 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:44.031 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:29:44.031 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.086060+0000", 2024-09-14T06:29:44.031 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.5499999999999999e-07 2024-09-14T06:29:44.031 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:44.031 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:44.031 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:29:44.031 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.086061+0000", 2024-09-14T06:29:44.031 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.299e-06 2024-09-14T06:29:44.031 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:44.032 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:44.032 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:29:44.032 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.086065+0000", 2024-09-14T06:29:44.032 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.687e-06 2024-09-14T06:29:44.032 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:44.032 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:44.032 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:29:44.032 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.086068+0000", 2024-09-14T06:29:44.032 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.8219999999999998e-06 2024-09-14T06:29:44.032 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:44.032 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:44.032 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:29:44.032 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.126372+0000", 2024-09-14T06:29:44.032 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.040303701999999997 2024-09-14T06:29:44.033 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:44.033 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:44.033 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:29:44.033 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.126459+0000", 2024-09-14T06:29:44.033 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.7037e-05 2024-09-14T06:29:44.033 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:44.033 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:44.033 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:29:44.033 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.126607+0000", 2024-09-14T06:29:44.033 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00014808399999999999 2024-09-14T06:29:44.033 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:44.033 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:44.033 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:44.034 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:44.034 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:44.034 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:14923 3.0 3:05c39535:::rbd_data.112d92027590.0000000000000009:f [read 846848~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:29:44.034 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:26.086072+0000", 2024-09-14T06:29:44.034 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 17.885868167000002, 2024-09-14T06:29:44.034 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.040715464999999999, 2024-09-14T06:29:44.034 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:29:44.034 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:29:44.034 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:29:44.034 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:29:44.034 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:29:44.034 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:44.034 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 14923 2024-09-14T06:29:44.034 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:44.035 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:29:44.035 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:44.035 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:29:44.035 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.086072+0000", 2024-09-14T06:29:44.035 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:44.035 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:44.035 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:44.035 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:29:44.035 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.086072+0000", 2024-09-14T06:29:44.035 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:44.035 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:44.035 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:44.035 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:29:44.035 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.086072+0000", 2024-09-14T06:29:44.036 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.0699999999999999e-07 2024-09-14T06:29:44.036 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:44.036 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:44.036 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:29:44.036 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.086073+0000", 2024-09-14T06:29:44.036 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.0899999999999999e-06 2024-09-14T06:29:44.036 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:44.036 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:44.036 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:29:44.036 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.086076+0000", 2024-09-14T06:29:44.036 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.157e-06 2024-09-14T06:29:44.036 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:44.036 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:44.036 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:29:44.037 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.086079+0000", 2024-09-14T06:29:44.037 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.5299999999999999e-06 2024-09-14T06:29:44.037 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:44.037 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:44.037 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:29:44.037 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.126633+0000", 2024-09-14T06:29:44.037 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.040553812000000002 2024-09-14T06:29:44.037 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:44.037 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:44.037 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:29:44.037 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.126685+0000", 2024-09-14T06:29:44.037 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.2853999999999999e-05 2024-09-14T06:29:44.037 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:44.037 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:44.038 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:29:44.038 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.126787+0000", 2024-09-14T06:29:44.038 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000101815 2024-09-14T06:29:44.038 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:44.038 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:44.038 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:44.038 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:44.038 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:44.038 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:14924 3.0 3:05c39535:::rbd_data.112d92027590.0000000000000009:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:29:44.038 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:26.086082+0000", 2024-09-14T06:29:44.038 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 17.885857938000001, 2024-09-14T06:29:44.038 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.040904439000000001, 2024-09-14T06:29:44.038 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:29:44.038 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:29:44.039 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:29:44.039 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:29:44.039 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:29:44.039 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:44.039 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 14924 2024-09-14T06:29:44.039 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:44.039 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:29:44.039 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:44.039 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:29:44.039 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.086082+0000", 2024-09-14T06:29:44.039 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:44.039 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:44.039 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:44.039 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:29:44.040 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.086082+0000", 2024-09-14T06:29:44.040 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:44.040 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:44.040 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:44.040 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:29:44.040 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.086082+0000", 2024-09-14T06:29:44.040 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.29e-07 2024-09-14T06:29:44.040 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:44.040 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:44.040 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:29:44.040 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.086083+0000", 2024-09-14T06:29:44.040 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.0410000000000001e-06 2024-09-14T06:29:44.040 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:44.040 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:44.041 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:29:44.041 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.086086+0000", 2024-09-14T06:29:44.041 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.179e-06 2024-09-14T06:29:44.041 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:44.041 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:44.041 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:29:44.041 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.086090+0000", 2024-09-14T06:29:44.041 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.0069999999999999e-06 2024-09-14T06:29:44.041 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:44.041 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:44.041 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:29:44.041 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.126832+0000", 2024-09-14T06:29:44.042 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.040741357999999998 2024-09-14T06:29:44.042 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:44.042 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:44.042 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:29:44.042 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.126906+0000", 2024-09-14T06:29:44.042 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.3935000000000003e-05 2024-09-14T06:29:44.042 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:44.042 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:44.042 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:29:44.042 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.126986+0000", 2024-09-14T06:29:44.042 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.0690000000000002e-05 2024-09-14T06:29:44.042 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:44.042 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:44.042 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:44.042 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:44.043 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:44.043 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:14927 3.0 3:05c39535:::rbd_data.112d92027590.0000000000000009:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:29:44.043 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:26.086130+0000", 2024-09-14T06:29:44.043 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 17.885809678000001, 2024-09-14T06:29:44.043 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.040995357000000003, 2024-09-14T06:29:44.043 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:29:44.043 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:29:44.043 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:29:44.043 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:29:44.043 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:29:44.043 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:44.043 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 14927 2024-09-14T06:29:44.043 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:44.043 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:29:44.044 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:44.044 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:29:44.044 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.086130+0000", 2024-09-14T06:29:44.044 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:44.044 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:44.044 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:44.044 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:29:44.044 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.086130+0000", 2024-09-14T06:29:44.044 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:44.044 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:44.044 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:44.044 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:29:44.044 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.086131+0000", 2024-09-14T06:29:44.045 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.2e-07 2024-09-14T06:29:44.045 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:44.045 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:44.045 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:29:44.045 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.086133+0000", 2024-09-14T06:29:44.045 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.3599999999999999e-06 2024-09-14T06:29:44.045 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:44.045 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:44.045 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:29:44.045 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.086139+0000", 2024-09-14T06:29:44.045 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.9970000000000004e-06 2024-09-14T06:29:44.045 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:44.045 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:44.045 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:29:44.046 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.086144+0000", 2024-09-14T06:29:44.046 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.3029999999999999e-06 2024-09-14T06:29:44.046 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:44.046 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:44.046 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:29:44.046 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.127032+0000", 2024-09-14T06:29:44.046 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.040887236 2024-09-14T06:29:44.046 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:44.046 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:44.046 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:29:44.046 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.127077+0000", 2024-09-14T06:29:44.046 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.5553000000000002e-05 2024-09-14T06:29:44.046 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:44.047 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:44.047 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:29:44.047 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.127126+0000", 2024-09-14T06:29:44.047 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.8488000000000001e-05 2024-09-14T06:29:44.047 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:44.047 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:44.047 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:44.047 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:44.047 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:44.047 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:14932 3.0 3:05c39535:::rbd_data.112d92027590.0000000000000009:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:29:44.047 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:26.086152+0000", 2024-09-14T06:29:44.047 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 17.885787919999999, 2024-09-14T06:29:44.047 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.041108513999999999, 2024-09-14T06:29:44.047 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:29:44.048 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:29:44.048 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:29:44.048 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:29:44.048 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:29:44.048 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:44.048 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 14932 2024-09-14T06:29:44.048 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:44.048 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:29:44.048 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:44.048 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:29:44.048 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.086152+0000", 2024-09-14T06:29:44.048 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:44.048 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:44.048 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:44.049 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:29:44.049 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.086152+0000", 2024-09-14T06:29:44.049 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:44.049 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:44.049 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:44.049 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:29:44.049 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.086152+0000", 2024-09-14T06:29:44.049 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.9000000000000002e-07 2024-09-14T06:29:44.049 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:44.049 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:44.049 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:29:44.049 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.086156+0000", 2024-09-14T06:29:44.049 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.3500000000000001e-06 2024-09-14T06:29:44.049 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:44.050 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:44.050 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:29:44.050 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.086162+0000", 2024-09-14T06:29:44.050 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.0120000000000002e-06 2024-09-14T06:29:44.050 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:44.050 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:44.050 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:29:44.050 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.086166+0000", 2024-09-14T06:29:44.050 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.6269999999999999e-06 2024-09-14T06:29:44.050 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:44.050 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:44.050 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:29:44.050 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.127162+0000", 2024-09-14T06:29:44.051 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.040995386000000002 2024-09-14T06:29:44.051 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:44.051 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:44.051 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:29:44.051 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.127214+0000", 2024-09-14T06:29:44.051 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.1860999999999997e-05 2024-09-14T06:29:44.051 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:44.051 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:44.051 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:29:44.051 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.127261+0000", 2024-09-14T06:29:44.051 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.6888000000000002e-05 2024-09-14T06:29:44.051 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:44.051 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:44.051 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:44.051 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:44.052 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:44.052 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:14937 3.0 3:05c39535:::rbd_data.112d92027590.0000000000000009:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:29:44.052 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:26.086173+0000", 2024-09-14T06:29:44.052 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 17.885767174000001, 2024-09-14T06:29:44.052 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.041239617999999999, 2024-09-14T06:29:44.052 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:29:44.052 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:29:44.052 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:29:44.052 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:29:44.052 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:29:44.052 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:44.052 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 14937 2024-09-14T06:29:44.052 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:44.052 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:29:44.053 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:44.053 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:29:44.053 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.086173+0000", 2024-09-14T06:29:44.053 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:44.053 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:44.053 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:44.053 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:29:44.053 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.086173+0000", 2024-09-14T06:29:44.053 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:44.053 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:44.053 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:44.053 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:29:44.053 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.086173+0000", 2024-09-14T06:29:44.053 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.5400000000000002e-07 2024-09-14T06:29:44.054 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:44.054 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:44.054 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:29:44.054 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.086175+0000", 2024-09-14T06:29:44.054 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.8479999999999999e-06 2024-09-14T06:29:44.054 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:44.054 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:44.054 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:29:44.054 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.086181+0000", 2024-09-14T06:29:44.054 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.8490000000000002e-06 2024-09-14T06:29:44.054 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:44.054 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:44.054 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:29:44.054 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.086185+0000", 2024-09-14T06:29:44.055 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.6190000000000002e-06 2024-09-14T06:29:44.055 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:44.055 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:44.055 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:29:44.055 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.127292+0000", 2024-09-14T06:29:44.055 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.041106119000000003 2024-09-14T06:29:44.055 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:44.055 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:44.055 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:29:44.055 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.127353+0000", 2024-09-14T06:29:44.055 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.1313000000000001e-05 2024-09-14T06:29:44.055 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:44.055 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:44.055 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:29:44.056 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.127412+0000", 2024-09-14T06:29:44.056 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.9515999999999997e-05 2024-09-14T06:29:44.056 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:44.056 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:44.056 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:44.056 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:44.056 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:44.056 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:14942 3.0 3:05c39535:::rbd_data.112d92027590.0000000000000009:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:29:44.056 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:26.086192+0000", 2024-09-14T06:29:44.056 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 17.885748314000001, 2024-09-14T06:29:44.056 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.041417974000000003, 2024-09-14T06:29:44.056 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:29:44.056 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:29:44.056 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:29:44.057 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:29:44.057 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:29:44.057 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:44.057 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 14942 2024-09-14T06:29:44.057 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:44.057 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:29:44.057 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:44.057 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:29:44.057 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.086192+0000", 2024-09-14T06:29:44.057 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:44.057 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:44.057 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:44.057 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:29:44.057 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.086192+0000", 2024-09-14T06:29:44.058 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:44.058 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:44.058 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:44.058 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:29:44.058 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.086192+0000", 2024-09-14T06:29:44.058 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.22e-07 2024-09-14T06:29:44.058 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:44.058 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:44.058 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:29:44.058 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.086199+0000", 2024-09-14T06:29:44.058 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.4789999999999999e-06 2024-09-14T06:29:44.058 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:44.058 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:44.058 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:29:44.059 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.086205+0000", 2024-09-14T06:29:44.059 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.0249999999999999e-06 2024-09-14T06:29:44.059 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:44.059 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:44.059 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:29:44.059 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.086210+0000", 2024-09-14T06:29:44.059 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.7400000000000004e-06 2024-09-14T06:29:44.059 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:44.059 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:44.059 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:29:44.059 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.127455+0000", 2024-09-14T06:29:44.059 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.041244691999999999 2024-09-14T06:29:44.059 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:44.059 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:44.060 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:29:44.060 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.127529+0000", 2024-09-14T06:29:44.060 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.4467000000000004e-05 2024-09-14T06:29:44.060 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:44.060 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:44.060 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:29:44.060 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.127610+0000", 2024-09-14T06:29:44.060 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.0248999999999997e-05 2024-09-14T06:29:44.060 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:44.060 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:44.060 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:44.060 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:44.060 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:44.060 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:14946 3.0 3:05c39535:::rbd_data.112d92027590.0000000000000009:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:29:44.061 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:26.086218+0000", 2024-09-14T06:29:44.061 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 17.885722384000001, 2024-09-14T06:29:44.061 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.041568755999999998, 2024-09-14T06:29:44.061 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:29:44.061 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:29:44.061 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:29:44.061 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:29:44.061 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:29:44.061 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:44.061 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 14946 2024-09-14T06:29:44.061 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:44.061 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:29:44.061 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:44.061 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:29:44.062 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.086218+0000", 2024-09-14T06:29:44.062 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:44.062 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:44.062 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:44.062 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:29:44.062 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.086218+0000", 2024-09-14T06:29:44.062 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:44.062 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:44.062 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:44.062 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:29:44.062 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.086218+0000", 2024-09-14T06:29:44.062 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.7399999999999999e-07 2024-09-14T06:29:44.062 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:44.062 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:44.062 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:29:44.063 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.086220+0000", 2024-09-14T06:29:44.063 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.9649999999999998e-06 2024-09-14T06:29:44.063 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:44.063 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:44.063 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:29:44.063 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.086226+0000", 2024-09-14T06:29:44.063 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.7130000000000002e-06 2024-09-14T06:29:44.063 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:44.063 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:44.063 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:29:44.063 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.086230+0000", 2024-09-14T06:29:44.063 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.5539999999999999e-06 2024-09-14T06:29:44.063 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:44.063 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:44.063 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:29:44.064 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.127640+0000", 2024-09-14T06:29:44.064 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.041409396000000001 2024-09-14T06:29:44.064 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:44.064 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:44.064 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:29:44.064 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.127710+0000", 2024-09-14T06:29:44.064 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.0333000000000006e-05 2024-09-14T06:29:44.064 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:44.064 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:44.064 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:29:44.064 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.127786+0000", 2024-09-14T06:29:44.064 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.6421000000000006e-05 2024-09-14T06:29:44.064 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:44.064 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:44.065 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:44.065 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:44.065 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:44.065 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-14T06:29:44.066 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-14T06:29:44.067 INFO:tasks.ceph.osd.5.smithi138.stderr:2024-09-14T06:29:44.065+0000 7fe13746f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33618) UID: 0 2024-09-14T06:29:44.167 INFO:tasks.ceph.osd.3.smithi094.stderr:2024-09-14T06:29:44.166+0000 7feeeafe8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33655) UID: 0 2024-09-14T06:29:44.226 INFO:teuthology.orchestra.run.smithi094.stdout:moved 0 / 82 (0%) 2024-09-14T06:29:44.226 INFO:teuthology.orchestra.run.smithi094.stdout:avg 10.25 2024-09-14T06:29:44.226 INFO:teuthology.orchestra.run.smithi094.stdout:stddev 3.26917 -> 3.26917 (expected baseline 2.99479) 2024-09-14T06:29:44.227 INFO:teuthology.orchestra.run.smithi094.stdout:min osd.2 with 5 -> 5 pgs (0.487805 -> 0.487805 * mean) 2024-09-14T06:29:44.227 INFO:teuthology.orchestra.run.smithi094.stdout:max osd.7 with 14 -> 14 pgs (1.36585 -> 1.36585 * mean) 2024-09-14T06:29:44.227 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-14T06:29:44.227 INFO:teuthology.orchestra.run.smithi094.stdout:oload 1000 2024-09-14T06:29:44.227 INFO:teuthology.orchestra.run.smithi094.stdout:max_change 0.05 2024-09-14T06:29:44.227 INFO:teuthology.orchestra.run.smithi094.stdout:max_change_osds 4 2024-09-14T06:29:44.227 INFO:teuthology.orchestra.run.smithi094.stdout:average_utilization 0.0004 2024-09-14T06:29:44.227 INFO:teuthology.orchestra.run.smithi094.stdout:overload_utilization 0.0038 2024-09-14T06:29:44.227 INFO:teuthology.orchestra.run.smithi094.stderr:no change 2024-09-14T06:29:44.228 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-14T06:29:44.228 INFO:teuthology.orchestra.run.smithi094.stdout: "ops": [], 2024-09-14T06:29:44.229 INFO:teuthology.orchestra.run.smithi094.stdout: "num_ops": 0 2024-09-14T06:29:44.229 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-14T06:29:44.242 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-14T06:29:44.246 INFO:tasks.thrashosds.thrasher:Reweighting by: {'max_change': '3.0', 'overage': '110', 'type': 'test-reweight-by-utilization'} 2024-09-14T06:29:44.246 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd test-reweight-by-utilization 110 3.0 2024-09-14T06:29:44.268 INFO:tasks.ceph.osd.7.smithi138.stderr:2024-09-14T06:29:44.267+0000 7f3c6a21d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33623) UID: 0 2024-09-14T06:29:44.369 INFO:tasks.ceph.osd.2.smithi094.stderr:2024-09-14T06:29:44.368+0000 7faf3269b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33649) UID: 0 2024-09-14T06:29:44.438 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-14T06:29:44.438 INFO:teuthology.orchestra.run.smithi094.stdout: "ops": [], 2024-09-14T06:29:44.438 INFO:teuthology.orchestra.run.smithi094.stdout: "complaint_time": 30, 2024-09-14T06:29:44.438 INFO:teuthology.orchestra.run.smithi094.stdout: "num_blocked_ops": 0 2024-09-14T06:29:44.438 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-14T06:29:44.449 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-14T06:29:44.490 INFO:tasks.ceph.osd.2.smithi094.stderr:2024-09-14T06:29:44.489+0000 7faf3269b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33649) UID: 0 2024-09-14T06:29:44.572 INFO:tasks.ceph.osd.3.smithi094.stderr:2024-09-14T06:29:44.571+0000 7feeeafe8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33655) UID: 0 2024-09-14T06:29:44.627 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-14T06:29:44.627 INFO:teuthology.orchestra.run.smithi094.stdout: "size": 20, 2024-09-14T06:29:44.628 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 600, 2024-09-14T06:29:44.628 INFO:teuthology.orchestra.run.smithi094.stdout: "ops": [ 2024-09-14T06:29:44.628 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:44.628 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4397.0:937 3.9 e30/23 3:92b37d3e:::rbd_data.112d7dfbd766.0000000000000012:head v 30'7, mlcod=30'5)", 2024-09-14T06:29:44.628 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:07.169523+0000", 2024-09-14T06:29:44.628 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 37.456082303999999, 2024-09-14T06:29:44.628 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0013381339999999999, 2024-09-14T06:29:44.628 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:29:44.628 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:29:44.628 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:29:44.628 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:29:44.628 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:44.629 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:29:44.629 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:07.169523+0000", 2024-09-14T06:29:44.629 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:44.629 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:44.629 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:44.629 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:29:44.629 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:07.169523+0000", 2024-09-14T06:29:44.629 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:44.629 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:44.629 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:44.629 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:29:44.629 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:07.169524+0000", 2024-09-14T06:29:44.629 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.125e-06 2024-09-14T06:29:44.629 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:44.630 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:44.630 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:29:44.630 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:07.170106+0000", 2024-09-14T06:29:44.630 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00058239900000000005 2024-09-14T06:29:44.630 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:44.630 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:44.630 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:29:44.630 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:07.170121+0000", 2024-09-14T06:29:44.630 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.4851999999999999e-05 2024-09-14T06:29:44.630 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:44.630 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:44.630 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:29:44.630 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:07.170130+0000", 2024-09-14T06:29:44.631 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.9199999999999993e-06 2024-09-14T06:29:44.631 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:44.631 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:44.631 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:29:44.631 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:07.170182+0000", 2024-09-14T06:29:44.631 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.1669999999999998e-05 2024-09-14T06:29:44.631 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:44.631 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:44.631 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:29:44.631 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:07.170199+0000", 2024-09-14T06:29:44.631 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.7193999999999998e-05 2024-09-14T06:29:44.631 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:44.631 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:44.631 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-14T06:29:44.632 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:07.170816+0000", 2024-09-14T06:29:44.632 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00061745400000000003 2024-09-14T06:29:44.632 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:44.632 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:44.632 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:29:44.632 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:07.170861+0000", 2024-09-14T06:29:44.632 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.4520000000000001e-05 2024-09-14T06:29:44.632 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:44.632 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:44.632 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:44.632 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:44.632 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:44.632 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4397.0:1848 3.4 e32/23 3:27620a21:::rbd_data.112dcdddac74.000000000000000b:head v 32'40, mlcod=30'38)", 2024-09-14T06:29:44.633 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:08.658649+0000", 2024-09-14T06:29:44.633 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 35.966955931000001, 2024-09-14T06:29:44.633 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.001862867, 2024-09-14T06:29:44.633 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:29:44.633 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:29:44.633 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:29:44.633 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:29:44.633 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:44.633 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:29:44.633 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.658649+0000", 2024-09-14T06:29:44.633 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:44.633 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:44.633 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:44.634 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:29:44.634 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.658649+0000", 2024-09-14T06:29:44.634 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:44.634 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:44.634 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:44.634 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:29:44.634 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.658651+0000", 2024-09-14T06:29:44.634 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.5650000000000001e-06 2024-09-14T06:29:44.634 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:44.634 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:44.634 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:29:44.634 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.658709+0000", 2024-09-14T06:29:44.634 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.8442000000000001e-05 2024-09-14T06:29:44.634 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:44.635 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:44.635 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:29:44.635 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.658730+0000", 2024-09-14T06:29:44.635 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.0509999999999998e-05 2024-09-14T06:29:44.635 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:44.635 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:44.635 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:29:44.635 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.658740+0000", 2024-09-14T06:29:44.635 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.8619999999999998e-06 2024-09-14T06:29:44.635 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:44.635 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:44.635 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:29:44.635 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.658813+0000", 2024-09-14T06:29:44.635 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.3578000000000006e-05 2024-09-14T06:29:44.636 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:44.636 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:44.636 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:29:44.636 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.658836+0000", 2024-09-14T06:29:44.636 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.2677e-05 2024-09-14T06:29:44.636 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:44.636 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:44.636 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-14T06:29:44.636 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.660462+0000", 2024-09-14T06:29:44.636 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00162637 2024-09-14T06:29:44.636 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:44.636 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:44.636 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:29:44.637 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.660512+0000", 2024-09-14T06:29:44.637 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.9863e-05 2024-09-14T06:29:44.637 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:44.637 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:44.637 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:44.637 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:44.637 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:44.637 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4397.0:2199 3.0 e32/23 3:05f3e980:::rbd_data.112dcdddac74.000000000000000f:head v 32'3, mlcod=31'1)", 2024-09-14T06:29:44.637 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:08.680236+0000", 2024-09-14T06:29:44.637 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 35.945369565999997, 2024-09-14T06:29:44.637 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0019068990000000001, 2024-09-14T06:29:44.637 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:29:44.637 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:29:44.637 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:29:44.638 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:29:44.638 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:44.638 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:29:44.638 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.680236+0000", 2024-09-14T06:29:44.638 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:44.638 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:44.638 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:44.638 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:29:44.638 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.680236+0000", 2024-09-14T06:29:44.638 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:44.638 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:44.638 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:44.638 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:29:44.639 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.680237+0000", 2024-09-14T06:29:44.639 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.4980000000000001e-06 2024-09-14T06:29:44.639 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:44.639 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:44.639 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:29:44.639 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.680324+0000", 2024-09-14T06:29:44.639 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.7310000000000003e-05 2024-09-14T06:29:44.639 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:44.639 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:44.639 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:29:44.639 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.680354+0000", 2024-09-14T06:29:44.639 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.97e-05 2024-09-14T06:29:44.639 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:44.639 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:44.640 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:29:44.640 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.680366+0000", 2024-09-14T06:29:44.640 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.2048999999999999e-05 2024-09-14T06:29:44.640 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:44.640 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:44.640 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:29:44.640 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.680439+0000", 2024-09-14T06:29:44.640 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.2910000000000005e-05 2024-09-14T06:29:44.640 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:44.640 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:44.640 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:29:44.640 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.680465+0000", 2024-09-14T06:29:44.640 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.5812999999999998e-05 2024-09-14T06:29:44.640 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:44.641 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:44.641 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-14T06:29:44.641 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.682094+0000", 2024-09-14T06:29:44.641 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.001628876 2024-09-14T06:29:44.641 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:44.641 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:44.641 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:29:44.641 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.682142+0000", 2024-09-14T06:29:44.641 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.8742999999999997e-05 2024-09-14T06:29:44.641 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:44.641 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:44.641 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:44.641 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:44.641 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:44.642 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4397.0:2513 3.0 e32/23 3:06f82ba7:::rbd_data.112dcdddac74.0000000000000028:head v 32'4, mlcod=32'2)", 2024-09-14T06:29:44.642 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:08.703547+0000", 2024-09-14T06:29:44.642 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 35.922057961999997, 2024-09-14T06:29:44.642 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0013041859999999999, 2024-09-14T06:29:44.642 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:29:44.642 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:29:44.642 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:29:44.642 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:29:44.642 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:44.642 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:29:44.642 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.703547+0000", 2024-09-14T06:29:44.642 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:44.642 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:44.642 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:44.643 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:29:44.643 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.703547+0000", 2024-09-14T06:29:44.643 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:44.643 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:44.643 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:44.643 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:29:44.643 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.703549+0000", 2024-09-14T06:29:44.643 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.39e-06 2024-09-14T06:29:44.643 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:44.643 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:44.643 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:29:44.643 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.703666+0000", 2024-09-14T06:29:44.643 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000117314 2024-09-14T06:29:44.643 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:44.644 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:44.644 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:29:44.644 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.703685+0000", 2024-09-14T06:29:44.644 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.8984999999999999e-05 2024-09-14T06:29:44.644 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:44.644 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:44.644 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:29:44.644 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.703697+0000", 2024-09-14T06:29:44.644 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.2405000000000001e-05 2024-09-14T06:29:44.644 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:44.644 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:44.644 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:29:44.644 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.703765+0000", 2024-09-14T06:29:44.645 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.7595000000000006e-05 2024-09-14T06:29:44.645 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:44.645 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:44.645 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:29:44.645 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.703788+0000", 2024-09-14T06:29:44.645 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.3300000000000001e-05 2024-09-14T06:29:44.645 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:44.645 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:44.645 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-14T06:29:44.645 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.704801+0000", 2024-09-14T06:29:44.645 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0010130339999999999 2024-09-14T06:29:44.645 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:44.645 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:44.646 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:29:44.646 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.704851+0000", 2024-09-14T06:29:44.646 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.0163000000000001e-05 2024-09-14T06:29:44.646 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:44.646 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:44.646 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:44.646 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:44.646 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:44.646 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4397.0:3292 3.9 e32/23 3:91dbce57:::rbd_data.112dcdddac74.0000000000000036:head v 32'10, mlcod=32'8)", 2024-09-14T06:29:44.646 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:08.740506+0000", 2024-09-14T06:29:44.646 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 35.885098853999999, 2024-09-14T06:29:44.646 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.001864166, 2024-09-14T06:29:44.646 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:29:44.646 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:29:44.647 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:29:44.647 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:29:44.647 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:44.647 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:29:44.647 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.740506+0000", 2024-09-14T06:29:44.647 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:44.647 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:44.647 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:44.647 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:29:44.647 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.740506+0000", 2024-09-14T06:29:44.647 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:44.647 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:44.647 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:44.647 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:29:44.648 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.740514+0000", 2024-09-14T06:29:44.648 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.2109999999999998e-06 2024-09-14T06:29:44.648 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:44.648 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:44.648 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:29:44.648 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.740745+0000", 2024-09-14T06:29:44.648 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000230194 2024-09-14T06:29:44.648 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:44.648 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:44.648 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:29:44.648 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.740764+0000", 2024-09-14T06:29:44.648 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.9330999999999998e-05 2024-09-14T06:29:44.648 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:44.648 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:44.649 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:29:44.649 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.740774+0000", 2024-09-14T06:29:44.649 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.6260000000000007e-06 2024-09-14T06:29:44.649 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:44.649 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:44.649 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:29:44.649 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.740839+0000", 2024-09-14T06:29:44.649 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.5314000000000005e-05 2024-09-14T06:29:44.649 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:44.649 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:44.649 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:29:44.649 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.740860+0000", 2024-09-14T06:29:44.649 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.0868000000000001e-05 2024-09-14T06:29:44.650 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:44.650 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:44.650 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-14T06:29:44.650 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.742327+0000", 2024-09-14T06:29:44.650 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0014672909999999999 2024-09-14T06:29:44.650 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:44.650 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:44.650 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:29:44.650 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.742370+0000", 2024-09-14T06:29:44.650 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.3331000000000002e-05 2024-09-14T06:29:44.650 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:44.650 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:44.650 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:44.650 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:44.651 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:44.651 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4397.0:12132 3.4 e49/23 3:27ed714d:::rbd_data.112d8f603570.0000000000000003:head v 49'47, mlcod=33'45)", 2024-09-14T06:29:44.651 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:25.671949+0000", 2024-09-14T06:29:44.651 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 18.953656422000002, 2024-09-14T06:29:44.651 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00131298, 2024-09-14T06:29:44.651 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:29:44.651 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:29:44.651 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:29:44.651 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:29:44.651 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:44.651 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:29:44.651 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.671949+0000", 2024-09-14T06:29:44.651 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:44.652 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:44.652 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:44.652 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:29:44.652 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.671949+0000", 2024-09-14T06:29:44.652 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:44.652 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:44.652 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:44.652 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:29:44.652 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.671949+0000", 2024-09-14T06:29:44.652 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.0500000000000004e-07 2024-09-14T06:29:44.652 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:44.652 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:44.652 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:29:44.653 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.672087+0000", 2024-09-14T06:29:44.653 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00013826999999999999 2024-09-14T06:29:44.653 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:44.653 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:44.653 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:29:44.653 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.672098+0000", 2024-09-14T06:29:44.653 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.0753e-05 2024-09-14T06:29:44.653 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:44.653 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:44.653 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:29:44.653 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.672105+0000", 2024-09-14T06:29:44.653 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.613e-06 2024-09-14T06:29:44.653 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:44.654 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:44.654 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:29:44.654 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.672178+0000", 2024-09-14T06:29:44.654 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.3131000000000002e-05 2024-09-14T06:29:44.654 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:44.654 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:44.654 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:29:44.654 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.672202+0000", 2024-09-14T06:29:44.654 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.4301e-05 2024-09-14T06:29:44.654 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:44.654 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:44.654 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-14T06:29:44.654 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.673207+0000", 2024-09-14T06:29:44.654 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0010045779999999999 2024-09-14T06:29:44.655 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:44.655 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:44.655 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:29:44.655 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:25.673262+0000", 2024-09-14T06:29:44.655 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.4829e-05 2024-09-14T06:29:44.655 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:44.655 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:44.655 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:44.656 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:44.656 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:44.656 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4397.0:16705 3.9 e49/23 3:91acaef1:::rbd_data.112d8f603570.0000000000000067:head v 49'20, mlcod=49'18)", 2024-09-14T06:29:44.656 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:26.198172+0000", 2024-09-14T06:29:44.656 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 18.427433571000002, 2024-09-14T06:29:44.656 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00133445, 2024-09-14T06:29:44.656 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:29:44.656 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:29:44.656 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:29:44.656 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:29:44.656 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:44.656 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:29:44.656 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.198172+0000", 2024-09-14T06:29:44.656 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:44.656 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:44.657 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:44.657 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:29:44.657 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.198172+0000", 2024-09-14T06:29:44.657 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:44.657 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:44.657 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:44.657 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:29:44.657 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.198173+0000", 2024-09-14T06:29:44.657 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.4529999999999999e-06 2024-09-14T06:29:44.657 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:44.657 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:44.657 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:29:44.657 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.198318+0000", 2024-09-14T06:29:44.658 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00014472500000000001 2024-09-14T06:29:44.658 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:44.658 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:44.658 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:29:44.658 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.198333+0000", 2024-09-14T06:29:44.658 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.5420000000000001e-05 2024-09-14T06:29:44.658 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:44.658 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:44.658 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:29:44.658 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.198341+0000", 2024-09-14T06:29:44.658 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.2910000000000007e-06 2024-09-14T06:29:44.658 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:44.658 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:44.658 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:29:44.658 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.198403+0000", 2024-09-14T06:29:44.659 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.1985999999999999e-05 2024-09-14T06:29:44.659 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:44.659 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:44.659 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:29:44.659 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.198427+0000", 2024-09-14T06:29:44.659 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.3204e-05 2024-09-14T06:29:44.659 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:44.659 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:44.659 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-14T06:29:44.659 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.199458+0000", 2024-09-14T06:29:44.659 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0010313620000000001 2024-09-14T06:29:44.659 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:44.659 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:44.659 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:29:44.660 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.199506+0000", 2024-09-14T06:29:44.660 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.8009000000000001e-05 2024-09-14T06:29:44.660 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:44.660 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:44.660 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:44.660 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:44.660 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:44.660 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4397.0:26415 3.0 e53/23 3:05fd9be0:::rbd_data.112d38768680.0000000000000006:head v 53'21, mlcod=50'19)", 2024-09-14T06:29:44.660 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:30.721610+0000", 2024-09-14T06:29:44.660 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 13.903994785, 2024-09-14T06:29:44.660 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0021763160000000002, 2024-09-14T06:29:44.660 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:29:44.660 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:29:44.661 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:29:44.661 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:29:44.661 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:44.661 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:29:44.661 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:30.721610+0000", 2024-09-14T06:29:44.661 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:44.661 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:44.661 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:44.661 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:29:44.661 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:30.721610+0000", 2024-09-14T06:29:44.661 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:44.661 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:44.661 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:44.662 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:29:44.662 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:30.721611+0000", 2024-09-14T06:29:44.662 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.41e-07 2024-09-14T06:29:44.662 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:44.662 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:44.662 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:29:44.662 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:30.721905+0000", 2024-09-14T06:29:44.662 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00029394300000000001 2024-09-14T06:29:44.662 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:44.662 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:44.662 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:29:44.662 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:30.721921+0000", 2024-09-14T06:29:44.662 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.6609999999999999e-05 2024-09-14T06:29:44.662 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:44.663 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:44.663 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:29:44.663 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:30.721931+0000", 2024-09-14T06:29:44.663 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.6739999999999992e-06 2024-09-14T06:29:44.663 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:44.663 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:44.663 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:29:44.663 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:30.722001+0000", 2024-09-14T06:29:44.663 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.0549000000000005e-05 2024-09-14T06:29:44.663 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:44.663 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:44.663 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:29:44.663 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:30.722029+0000", 2024-09-14T06:29:44.663 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.7327000000000001e-05 2024-09-14T06:29:44.664 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:44.664 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:44.664 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-14T06:29:44.664 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:30.723727+0000", 2024-09-14T06:29:44.664 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0016977310000000001 2024-09-14T06:29:44.664 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:44.664 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:44.664 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:29:44.664 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:30.723787+0000", 2024-09-14T06:29:44.664 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.0140999999999999e-05 2024-09-14T06:29:44.664 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:44.664 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:44.664 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:44.665 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:44.665 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:44.665 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4397.0:36084 3.0 e56/23 3:05a48ef4:::rbd_data.112dc58639fa.0000000000000058:head v 56'29, mlcod=56'27)", 2024-09-14T06:29:44.665 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:34.508855+0000", 2024-09-14T06:29:44.665 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 10.116750031, 2024-09-14T06:29:44.665 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0020801140000000001, 2024-09-14T06:29:44.665 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:29:44.665 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:29:44.665 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:29:44.666 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:29:44.666 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:44.666 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:29:44.666 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:34.508855+0000", 2024-09-14T06:29:44.666 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:44.666 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:44.666 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:44.666 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:29:44.666 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:34.508855+0000", 2024-09-14T06:29:44.666 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:44.666 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:44.666 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:44.666 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:29:44.667 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:34.508856+0000", 2024-09-14T06:29:44.667 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.1909999999999999e-06 2024-09-14T06:29:44.667 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:44.667 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:44.667 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:29:44.667 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:34.509203+0000", 2024-09-14T06:29:44.667 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00034625900000000002 2024-09-14T06:29:44.667 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:44.667 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:44.667 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:29:44.667 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:34.509218+0000", 2024-09-14T06:29:44.667 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.5091e-05 2024-09-14T06:29:44.667 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:44.667 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:44.668 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:29:44.668 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:34.509227+0000", 2024-09-14T06:29:44.668 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.0850000000000003e-06 2024-09-14T06:29:44.668 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:44.668 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:44.668 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:29:44.668 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:34.509311+0000", 2024-09-14T06:29:44.668 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.4436000000000003e-05 2024-09-14T06:29:44.668 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:44.668 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:44.668 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:29:44.668 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:34.509335+0000", 2024-09-14T06:29:44.668 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.3577000000000001e-05 2024-09-14T06:29:44.668 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:44.669 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:44.669 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-14T06:29:44.669 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:34.510890+0000", 2024-09-14T06:29:44.669 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0015554259999999999 2024-09-14T06:29:44.669 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:44.669 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:44.669 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:29:44.669 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:34.510935+0000", 2024-09-14T06:29:44.670 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.5049000000000002e-05 2024-09-14T06:29:44.670 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:44.670 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:44.671 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:44.671 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:44.672 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:44.672 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4397.0:36966 3.4 e56/23 3:22008318:::rbd_data.112dc58639fa.0000000000000066:head v 56'56, mlcod=50'54)", 2024-09-14T06:29:44.672 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:34.565614+0000", 2024-09-14T06:29:44.672 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 10.059991595, 2024-09-14T06:29:44.672 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0018691929999999999, 2024-09-14T06:29:44.672 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:29:44.672 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:29:44.672 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:29:44.672 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:29:44.672 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:44.672 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:29:44.672 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:34.565614+0000", 2024-09-14T06:29:44.672 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:44.673 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:44.673 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:44.673 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:29:44.673 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:34.565614+0000", 2024-09-14T06:29:44.673 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:44.673 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:44.673 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:44.673 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:29:44.673 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:34.565615+0000", 2024-09-14T06:29:44.673 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.784e-06 2024-09-14T06:29:44.673 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:44.674 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:44.674 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:29:44.674 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:34.565919+0000", 2024-09-14T06:29:44.674 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00030351599999999999 2024-09-14T06:29:44.674 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:44.674 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:44.674 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:29:44.674 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:34.565933+0000", 2024-09-14T06:29:44.674 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.4465e-05 2024-09-14T06:29:44.674 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:44.674 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:44.674 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:29:44.675 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:34.565943+0000", 2024-09-14T06:29:44.675 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.7920000000000004e-06 2024-09-14T06:29:44.675 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:44.675 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:44.675 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:29:44.675 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:34.565999+0000", 2024-09-14T06:29:44.675 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.5643000000000002e-05 2024-09-14T06:29:44.675 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:44.675 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:44.675 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:29:44.675 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:34.566025+0000", 2024-09-14T06:29:44.675 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.6040000000000001e-05 2024-09-14T06:29:44.675 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:44.676 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:44.676 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-14T06:29:44.676 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:34.567430+0000", 2024-09-14T06:29:44.676 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.001405653 2024-09-14T06:29:44.676 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:44.676 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:44.676 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:29:44.676 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:34.567483+0000", 2024-09-14T06:29:44.676 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.2299999999999997e-05 2024-09-14T06:29:44.676 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:44.676 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:44.676 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:44.676 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:44.676 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:44.677 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4397.0:38339 3.9 e57/23 3:9467a21d:::rbd_data.112dc58639fa.0000000000000003:head v 57'30, mlcod=56'28)", 2024-09-14T06:29:44.677 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:35.349820+0000", 2024-09-14T06:29:44.677 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 9.2757856010000008, 2024-09-14T06:29:44.677 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.001310702, 2024-09-14T06:29:44.677 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:29:44.677 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:29:44.677 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:29:44.677 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:29:44.677 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:44.677 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:29:44.677 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:35.349820+0000", 2024-09-14T06:29:44.677 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:44.678 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:44.678 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:44.678 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:29:44.678 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:35.349820+0000", 2024-09-14T06:29:44.678 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:44.678 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:44.678 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:44.678 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:29:44.678 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:35.349820+0000", 2024-09-14T06:29:44.678 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.2799999999999995e-07 2024-09-14T06:29:44.678 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:44.678 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:44.678 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:29:44.679 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:35.349826+0000", 2024-09-14T06:29:44.679 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.0179999999999998e-06 2024-09-14T06:29:44.679 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:44.679 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:44.679 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:29:44.679 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:35.349838+0000", 2024-09-14T06:29:44.679 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.172e-05 2024-09-14T06:29:44.679 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:44.679 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:44.679 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:29:44.679 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:35.349847+0000", 2024-09-14T06:29:44.679 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.8030000000000003e-06 2024-09-14T06:29:44.679 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:44.680 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:44.680 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:29:44.680 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:35.350052+0000", 2024-09-14T06:29:44.680 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000204973 2024-09-14T06:29:44.680 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:44.680 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:44.680 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:29:44.680 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:35.350083+0000", 2024-09-14T06:29:44.680 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.1578999999999997e-05 2024-09-14T06:29:44.680 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:44.680 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:44.680 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-14T06:29:44.680 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:35.351039+0000", 2024-09-14T06:29:44.681 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00095594799999999998 2024-09-14T06:29:44.681 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:44.681 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:44.681 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:29:44.681 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:35.351130+0000", 2024-09-14T06:29:44.681 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.0833000000000003e-05 2024-09-14T06:29:44.681 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:44.681 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:44.681 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:44.681 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:44.681 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:44.681 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4397.0:42285 3.9 e62/23 3:90747014:::rbd_data.112dca2f6ae1.000000000000002f:head v 62'54, mlcod=62'52)", 2024-09-14T06:29:44.681 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:40.755702+0000", 2024-09-14T06:29:44.682 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 3.869903211, 2024-09-14T06:29:44.682 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.001687596, 2024-09-14T06:29:44.682 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:29:44.682 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:29:44.682 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:29:44.682 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:29:44.682 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:44.682 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:29:44.682 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:40.755702+0000", 2024-09-14T06:29:44.682 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:44.682 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:44.682 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:44.682 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:29:44.683 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:40.755702+0000", 2024-09-14T06:29:44.683 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:44.683 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:44.683 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:44.683 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:29:44.683 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:40.755707+0000", 2024-09-14T06:29:44.683 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.4700000000000001e-06 2024-09-14T06:29:44.683 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:44.683 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:44.683 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:29:44.683 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:40.756366+0000", 2024-09-14T06:29:44.683 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00065852900000000004 2024-09-14T06:29:44.683 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:44.683 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:44.684 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:29:44.684 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:40.756383+0000", 2024-09-14T06:29:44.684 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.7275e-05 2024-09-14T06:29:44.684 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:44.684 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:44.684 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:29:44.684 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:40.756393+0000", 2024-09-14T06:29:44.684 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.4749999999999999e-06 2024-09-14T06:29:44.684 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:44.684 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:44.684 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:29:44.684 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:40.756462+0000", 2024-09-14T06:29:44.684 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.9214000000000005e-05 2024-09-14T06:29:44.685 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:44.685 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:44.685 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:29:44.685 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:40.756490+0000", 2024-09-14T06:29:44.685 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.7936000000000002e-05 2024-09-14T06:29:44.685 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:44.685 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:44.685 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-14T06:29:44.685 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:40.757343+0000", 2024-09-14T06:29:44.685 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000853611 2024-09-14T06:29:44.685 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:44.685 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:44.685 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:29:44.685 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:40.757390+0000", 2024-09-14T06:29:44.685 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.6085999999999999e-05 2024-09-14T06:29:44.686 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:44.686 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:44.686 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:44.686 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:44.686 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:44.686 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4397.0:45722 3.9 e64/23 3:91fed098:::rbd_data.112d5a737e72.0000000000000001:head v 64'61, mlcod=62'59)", 2024-09-14T06:29:44.686 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:43.864269+0000", 2024-09-14T06:29:44.686 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 0.76133595600000004, 2024-09-14T06:29:44.686 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.003055224, 2024-09-14T06:29:44.686 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:29:44.686 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:29:44.686 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:29:44.686 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:29:44.687 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:44.687 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:29:44.687 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.864269+0000", 2024-09-14T06:29:44.687 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:44.687 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:44.687 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:44.687 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:29:44.687 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.864269+0000", 2024-09-14T06:29:44.687 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:44.687 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:44.687 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:44.687 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:29:44.687 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.864269+0000", 2024-09-14T06:29:44.687 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.41e-07 2024-09-14T06:29:44.687 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:44.688 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:44.688 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:29:44.688 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.865490+0000", 2024-09-14T06:29:44.688 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.001220552 2024-09-14T06:29:44.688 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:44.688 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:44.688 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:29:44.688 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.865506+0000", 2024-09-14T06:29:44.688 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.6263999999999999e-05 2024-09-14T06:29:44.688 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:44.688 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:44.688 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:29:44.688 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.865516+0000", 2024-09-14T06:29:44.688 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.0091999999999999e-05 2024-09-14T06:29:44.689 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:44.689 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:44.689 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:29:44.689 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.865583+0000", 2024-09-14T06:29:44.689 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.6676999999999999e-05 2024-09-14T06:29:44.689 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:44.689 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:44.689 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:29:44.689 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.865610+0000", 2024-09-14T06:29:44.689 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.6529999999999998e-05 2024-09-14T06:29:44.689 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:44.689 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:44.689 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-14T06:29:44.689 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.867281+0000", 2024-09-14T06:29:44.689 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.001671203 2024-09-14T06:29:44.690 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:44.690 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:44.690 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:29:44.690 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.867324+0000", 2024-09-14T06:29:44.690 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.3665000000000003e-05 2024-09-14T06:29:44.690 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:44.690 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:44.690 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:44.690 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:44.690 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:44.690 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4397.0:49627 3.4 e64/23 3:21e30d4b:::rbd_data.112d5a737e72.0000000000000017:head v 64'69, mlcod=61'67)", 2024-09-14T06:29:44.690 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:44.206446+0000", 2024-09-14T06:29:44.690 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 0.419159318, 2024-09-14T06:29:44.690 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0018142220000000001, 2024-09-14T06:29:44.691 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:29:44.691 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:29:44.691 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:29:44.691 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:29:44.691 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:44.691 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:29:44.691 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.206446+0000", 2024-09-14T06:29:44.691 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:44.691 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:44.691 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:44.691 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:29:44.691 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.206446+0000", 2024-09-14T06:29:44.691 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:44.692 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:44.692 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:44.692 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:29:44.692 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.206446+0000", 2024-09-14T06:29:44.692 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.4900000000000001e-07 2024-09-14T06:29:44.692 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:44.692 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:44.692 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:29:44.692 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.206797+0000", 2024-09-14T06:29:44.692 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00035112799999999998 2024-09-14T06:29:44.692 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:44.692 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:44.692 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:29:44.692 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.206816+0000", 2024-09-14T06:29:44.693 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.8977000000000001e-05 2024-09-14T06:29:44.693 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:44.693 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:44.693 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:29:44.693 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.206825+0000", 2024-09-14T06:29:44.693 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.8279999999999992e-06 2024-09-14T06:29:44.693 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:44.693 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:44.693 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:29:44.693 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.206892+0000", 2024-09-14T06:29:44.693 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.6809000000000004e-05 2024-09-14T06:29:44.693 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:44.693 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:44.694 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:29:44.694 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.206915+0000", 2024-09-14T06:29:44.694 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.315e-05 2024-09-14T06:29:44.694 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:44.694 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:44.694 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-14T06:29:44.694 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.208201+0000", 2024-09-14T06:29:44.694 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0012858819999999999 2024-09-14T06:29:44.694 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:44.694 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:44.694 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:29:44.694 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.208260+0000", 2024-09-14T06:29:44.694 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.9098999999999998e-05 2024-09-14T06:29:44.694 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:44.695 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:44.695 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:44.695 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:44.695 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:44.695 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4397.0:50003 3.9 e64/23 3:94f26c4b:::rbd_data.112d5a737e72.000000000000001f:head v 64'62, mlcod=62'60)", 2024-09-14T06:29:44.695 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:44.220413+0000", 2024-09-14T06:29:44.695 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 0.40519164000000002, 2024-09-14T06:29:44.695 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0019210430000000001, 2024-09-14T06:29:44.695 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:29:44.695 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:29:44.695 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:29:44.695 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:29:44.695 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:44.695 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:29:44.696 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.220413+0000", 2024-09-14T06:29:44.696 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:44.696 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:44.696 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:44.696 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:29:44.696 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.220413+0000", 2024-09-14T06:29:44.696 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:44.696 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:44.696 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:44.696 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:29:44.696 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.220415+0000", 2024-09-14T06:29:44.696 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.575e-06 2024-09-14T06:29:44.696 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:44.696 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:44.697 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:29:44.697 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.220984+0000", 2024-09-14T06:29:44.697 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000568624 2024-09-14T06:29:44.697 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:44.697 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:44.697 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:29:44.697 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.221007+0000", 2024-09-14T06:29:44.697 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.3112000000000002e-05 2024-09-14T06:29:44.697 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:44.697 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:44.697 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:29:44.697 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.221017+0000", 2024-09-14T06:29:44.697 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.0098999999999999e-05 2024-09-14T06:29:44.697 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:44.698 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:44.698 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:29:44.698 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.221110+0000", 2024-09-14T06:29:44.698 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.3268000000000002e-05 2024-09-14T06:29:44.698 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:44.698 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:44.698 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:29:44.698 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.221131+0000", 2024-09-14T06:29:44.698 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.0905999999999999e-05 2024-09-14T06:29:44.698 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:44.698 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:44.698 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-14T06:29:44.698 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.222290+0000", 2024-09-14T06:29:44.698 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.001159408 2024-09-14T06:29:44.698 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:44.699 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:44.699 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:29:44.699 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.222335+0000", 2024-09-14T06:29:44.699 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.4051000000000002e-05 2024-09-14T06:29:44.699 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:44.699 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:44.699 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:44.699 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:44.699 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:44.699 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4397.0:50006 3.4 e64/23 3:20cfe300:::rbd_data.112d5a737e72.0000000000000019:head v 64'70, mlcod=61'68)", 2024-09-14T06:29:44.699 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:44.224218+0000", 2024-09-14T06:29:44.699 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 0.40138675899999998, 2024-09-14T06:29:44.699 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0021104320000000002, 2024-09-14T06:29:44.699 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:29:44.700 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:29:44.700 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:29:44.700 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:29:44.700 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:44.700 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:29:44.700 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.224218+0000", 2024-09-14T06:29:44.700 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:44.700 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:44.700 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:44.700 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:29:44.700 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.224218+0000", 2024-09-14T06:29:44.700 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:44.700 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:44.701 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:44.701 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:29:44.701 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.224219+0000", 2024-09-14T06:29:44.701 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.0049999999999999e-06 2024-09-14T06:29:44.701 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:44.701 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:44.701 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:29:44.701 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.224326+0000", 2024-09-14T06:29:44.701 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000106848 2024-09-14T06:29:44.701 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:44.701 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:44.701 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:29:44.701 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.224345+0000", 2024-09-14T06:29:44.701 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.8587999999999999e-05 2024-09-14T06:29:44.702 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:44.702 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:44.702 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:29:44.702 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.224354+0000", 2024-09-14T06:29:44.702 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.7220000000000005e-06 2024-09-14T06:29:44.702 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:44.702 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:44.702 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:29:44.702 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.224416+0000", 2024-09-14T06:29:44.702 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.2738999999999997e-05 2024-09-14T06:29:44.702 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:44.702 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:44.702 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:29:44.702 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.224441+0000", 2024-09-14T06:29:44.703 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.4408000000000001e-05 2024-09-14T06:29:44.703 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:44.703 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:44.703 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-14T06:29:44.703 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.226295+0000", 2024-09-14T06:29:44.703 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00185428 2024-09-14T06:29:44.703 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:44.703 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:44.703 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:29:44.703 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.226329+0000", 2024-09-14T06:29:44.703 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.3841999999999999e-05 2024-09-14T06:29:44.704 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:44.704 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:44.704 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:44.704 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:44.704 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:44.704 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4397.0:50414 3.0 e64/23 3:03ae4e67:::rbd_data.112d5a737e72.000000000000001a:head v 64'67, mlcod=62'65)", 2024-09-14T06:29:44.704 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:44.240908+0000", 2024-09-14T06:29:44.704 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 0.384697279, 2024-09-14T06:29:44.704 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.001493241, 2024-09-14T06:29:44.704 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:29:44.704 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:29:44.704 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:29:44.704 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:29:44.704 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:44.705 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:29:44.705 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.240908+0000", 2024-09-14T06:29:44.705 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:44.705 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:44.705 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:44.705 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:29:44.705 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.240908+0000", 2024-09-14T06:29:44.705 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:44.705 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:44.705 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:44.705 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:29:44.705 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.240910+0000", 2024-09-14T06:29:44.705 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.7179999999999999e-06 2024-09-14T06:29:44.706 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:44.706 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:44.706 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:29:44.706 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.241049+0000", 2024-09-14T06:29:44.706 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00013932199999999999 2024-09-14T06:29:44.706 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:44.706 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:44.706 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:29:44.706 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.241069+0000", 2024-09-14T06:29:44.706 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.9836000000000001e-05 2024-09-14T06:29:44.706 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:44.706 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:44.706 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:29:44.706 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.241080+0000", 2024-09-14T06:29:44.707 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.0920999999999999e-05 2024-09-14T06:29:44.707 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:44.707 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:44.707 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:29:44.707 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.241150+0000", 2024-09-14T06:29:44.707 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.0568999999999995e-05 2024-09-14T06:29:44.707 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:44.707 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:44.707 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:29:44.707 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.241178+0000", 2024-09-14T06:29:44.707 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.7429000000000001e-05 2024-09-14T06:29:44.707 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:44.707 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:44.707 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-14T06:29:44.707 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.242353+0000", 2024-09-14T06:29:44.708 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0011750770000000001 2024-09-14T06:29:44.708 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:44.708 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:44.708 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:29:44.708 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.242401+0000", 2024-09-14T06:29:44.708 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.8368999999999997e-05 2024-09-14T06:29:44.708 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:44.708 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:44.708 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:44.708 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:44.708 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:44.708 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4397.0:50436 3.0 e64/23 3:003223ce:::rbd_data.112d5a737e72.000000000000001b:head v 64'68, mlcod=62'66)", 2024-09-14T06:29:44.708 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:44.248373+0000", 2024-09-14T06:29:44.708 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 0.37723165400000003, 2024-09-14T06:29:44.709 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.001355251, 2024-09-14T06:29:44.709 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:29:44.709 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:29:44.709 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:29:44.709 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:29:44.709 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:44.709 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:29:44.709 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.248373+0000", 2024-09-14T06:29:44.709 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:44.709 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:44.709 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:44.709 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:29:44.709 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.248373+0000", 2024-09-14T06:29:44.709 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:44.709 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:44.710 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:44.710 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:29:44.710 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.248375+0000", 2024-09-14T06:29:44.710 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.1689999999999999e-06 2024-09-14T06:29:44.710 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:44.710 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:44.710 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:29:44.710 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.248575+0000", 2024-09-14T06:29:44.710 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000200676 2024-09-14T06:29:44.710 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:44.710 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:44.710 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:29:44.710 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.248592+0000", 2024-09-14T06:29:44.710 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.7073000000000001e-05 2024-09-14T06:29:44.711 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:44.711 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:44.711 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:29:44.711 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.248602+0000", 2024-09-14T06:29:44.711 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.8309999999999996e-06 2024-09-14T06:29:44.711 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:44.711 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:44.711 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:29:44.711 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.248665+0000", 2024-09-14T06:29:44.711 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.3128999999999998e-05 2024-09-14T06:29:44.711 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:44.711 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:44.711 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:29:44.712 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.248688+0000", 2024-09-14T06:29:44.712 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.2461999999999999e-05 2024-09-14T06:29:44.712 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:44.712 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:44.712 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-14T06:29:44.712 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.249687+0000", 2024-09-14T06:29:44.712 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000999324 2024-09-14T06:29:44.712 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:44.712 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:44.712 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:29:44.712 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.249729+0000", 2024-09-14T06:29:44.712 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.1587e-05 2024-09-14T06:29:44.712 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:44.712 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:44.713 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:44.713 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:44.713 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:44.713 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4397.0:50560 3.4 e64/23 3:27bad309:::rbd_data.112d5a737e72.000000000000001c:head v 64'71, mlcod=64'69)", 2024-09-14T06:29:44.713 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:44.256368+0000", 2024-09-14T06:29:44.713 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 0.36923714400000002, 2024-09-14T06:29:44.713 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.001488999, 2024-09-14T06:29:44.713 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:29:44.713 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:29:44.713 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:29:44.713 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:29:44.713 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:44.713 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:29:44.713 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.256368+0000", 2024-09-14T06:29:44.714 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:44.714 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:44.714 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:44.714 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:29:44.714 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.256368+0000", 2024-09-14T06:29:44.714 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:44.714 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:44.714 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:44.714 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:29:44.714 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.256369+0000", 2024-09-14T06:29:44.714 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.35e-06 2024-09-14T06:29:44.714 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:44.714 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:44.714 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:29:44.715 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.256634+0000", 2024-09-14T06:29:44.715 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00026467599999999998 2024-09-14T06:29:44.715 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:44.715 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:44.715 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:29:44.715 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.256651+0000", 2024-09-14T06:29:44.715 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.7003e-05 2024-09-14T06:29:44.715 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:44.715 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:44.715 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:29:44.715 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.256661+0000", 2024-09-14T06:29:44.715 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.6339999999999996e-06 2024-09-14T06:29:44.715 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:44.715 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:44.716 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:29:44.716 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.256724+0000", 2024-09-14T06:29:44.716 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.3132999999999993e-05 2024-09-14T06:29:44.716 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:44.716 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:44.716 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:29:44.716 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.256747+0000", 2024-09-14T06:29:44.716 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.3255999999999999e-05 2024-09-14T06:29:44.716 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:44.716 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:44.716 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-14T06:29:44.716 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.257796+0000", 2024-09-14T06:29:44.716 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0010489869999999999 2024-09-14T06:29:44.717 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:44.717 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:44.717 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:29:44.717 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.257857+0000", 2024-09-14T06:29:44.717 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.0959999999999999e-05 2024-09-14T06:29:44.717 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:44.717 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:44.717 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:44.717 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:44.717 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:44.717 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4397.0:51816 3.4 e64/23 3:21a826f9:::rbd_data.112d5a737e72.000000000000002d:head v 64'72, mlcod=64'70)", 2024-09-14T06:29:44.717 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:44.312072+0000", 2024-09-14T06:29:44.717 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 0.31353328600000002, 2024-09-14T06:29:44.717 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0019555240000000002, 2024-09-14T06:29:44.718 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:29:44.718 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:29:44.718 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:29:44.718 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:29:44.718 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:44.718 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:29:44.718 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.312072+0000", 2024-09-14T06:29:44.718 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:44.718 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:44.718 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:44.718 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:29:44.718 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.312072+0000", 2024-09-14T06:29:44.718 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:44.718 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:44.719 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:44.719 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:29:44.719 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.312074+0000", 2024-09-14T06:29:44.719 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.9530000000000002e-06 2024-09-14T06:29:44.719 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:44.719 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:44.719 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:29:44.719 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.312378+0000", 2024-09-14T06:29:44.719 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00030392 2024-09-14T06:29:44.719 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:44.719 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:44.719 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:29:44.719 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.312399+0000", 2024-09-14T06:29:44.719 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.1134000000000001e-05 2024-09-14T06:29:44.720 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:44.720 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:44.720 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:29:44.720 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.312408+0000", 2024-09-14T06:29:44.720 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.2879999999999998e-06 2024-09-14T06:29:44.720 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:44.720 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:44.720 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:29:44.720 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.312494+0000", 2024-09-14T06:29:44.720 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.6280999999999996e-05 2024-09-14T06:29:44.720 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:44.720 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:44.720 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:29:44.721 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.312518+0000", 2024-09-14T06:29:44.721 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.3149000000000001e-05 2024-09-14T06:29:44.721 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:44.721 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:44.721 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-14T06:29:44.721 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.313964+0000", 2024-09-14T06:29:44.721 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.001446754 2024-09-14T06:29:44.721 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:44.721 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:44.721 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:29:44.721 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.314027+0000", 2024-09-14T06:29:44.721 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.3045000000000004e-05 2024-09-14T06:29:44.721 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:44.721 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:44.722 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:44.722 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:44.722 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:44.722 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-14T06:29:44.723 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-14T06:29:44.724 INFO:teuthology.orchestra.run.smithi094.stdout:moved 0 / 82 (0%) 2024-09-14T06:29:44.724 INFO:teuthology.orchestra.run.smithi094.stdout:avg 10.25 2024-09-14T06:29:44.724 INFO:teuthology.orchestra.run.smithi094.stdout:stddev 3.26917 -> 3.26917 (expected baseline 2.99479) 2024-09-14T06:29:44.724 INFO:teuthology.orchestra.run.smithi094.stdout:min osd.2 with 5 -> 5 pgs (0.487805 -> 0.487805 * mean) 2024-09-14T06:29:44.724 INFO:teuthology.orchestra.run.smithi094.stdout:max osd.7 with 14 -> 14 pgs (1.36585 -> 1.36585 * mean) 2024-09-14T06:29:44.724 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-14T06:29:44.724 INFO:teuthology.orchestra.run.smithi094.stdout:oload 110 2024-09-14T06:29:44.724 INFO:teuthology.orchestra.run.smithi094.stdout:max_change 3 2024-09-14T06:29:44.725 INFO:teuthology.orchestra.run.smithi094.stdout:max_change_osds 4 2024-09-14T06:29:44.725 INFO:teuthology.orchestra.run.smithi094.stdout:average_utilization 0.0004 2024-09-14T06:29:44.725 INFO:teuthology.orchestra.run.smithi094.stdout:overload_utilization 0.0004 2024-09-14T06:29:44.725 INFO:teuthology.orchestra.run.smithi094.stderr:no change 2024-09-14T06:29:44.725 INFO:tasks.ceph.osd.4.smithi138.stderr:2024-09-14T06:29:44.722+0000 7f6c653c8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33615) UID: 0 2024-09-14T06:29:44.731 INFO:tasks.thrashosds.thrasher:Reweighting by: {'max_change': '3.0', 'overage': '1000', 'type': 'test-reweight-by-utilization'} 2024-09-14T06:29:44.732 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd test-reweight-by-utilization 1000 3.0 2024-09-14T06:29:44.749 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-14T06:29:44.824 INFO:tasks.ceph.osd.1.smithi094.stderr:2024-09-14T06:29:44.823+0000 7f328e47a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33641) UID: 0 2024-09-14T06:29:44.879 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-14T06:29:44.879 INFO:teuthology.orchestra.run.smithi094.stdout: "ops": [], 2024-09-14T06:29:44.879 INFO:teuthology.orchestra.run.smithi094.stdout: "num_ops": 0 2024-09-14T06:29:44.879 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-14T06:29:44.889 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-14T06:29:44.962 INFO:tasks.ceph.osd.3.smithi094.stderr:2024-09-14T06:29:44.929+0000 7feeeafe8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33655) UID: 0 2024-09-14T06:29:45.026 INFO:tasks.ceph.osd.6.smithi138.stderr:2024-09-14T06:29:45.025+0000 7f8cd8849640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33620) UID: 0 2024-09-14T06:29:45.033 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-14T06:29:45.033 INFO:teuthology.orchestra.run.smithi094.stdout: "ops": [], 2024-09-14T06:29:45.033 INFO:teuthology.orchestra.run.smithi094.stdout: "complaint_time": 30, 2024-09-14T06:29:45.033 INFO:teuthology.orchestra.run.smithi094.stdout: "num_blocked_ops": 0 2024-09-14T06:29:45.033 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-14T06:29:45.042 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-14T06:29:45.072 INFO:teuthology.orchestra.run.smithi094.stdout:moved 0 / 82 (0%) 2024-09-14T06:29:45.072 INFO:teuthology.orchestra.run.smithi094.stdout:avg 10.25 2024-09-14T06:29:45.072 INFO:teuthology.orchestra.run.smithi094.stdout:stddev 3.26917 -> 3.26917 (expected baseline 2.99479) 2024-09-14T06:29:45.072 INFO:teuthology.orchestra.run.smithi094.stdout:min osd.2 with 5 -> 5 pgs (0.487805 -> 0.487805 * mean) 2024-09-14T06:29:45.072 INFO:teuthology.orchestra.run.smithi094.stdout:max osd.7 with 14 -> 14 pgs (1.36585 -> 1.36585 * mean) 2024-09-14T06:29:45.072 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-14T06:29:45.072 INFO:teuthology.orchestra.run.smithi094.stdout:oload 1000 2024-09-14T06:29:45.072 INFO:teuthology.orchestra.run.smithi094.stdout:max_change 3 2024-09-14T06:29:45.072 INFO:teuthology.orchestra.run.smithi094.stdout:max_change_osds 4 2024-09-14T06:29:45.073 INFO:teuthology.orchestra.run.smithi094.stdout:average_utilization 0.0004 2024-09-14T06:29:45.073 INFO:teuthology.orchestra.run.smithi094.stdout:overload_utilization 0.0038 2024-09-14T06:29:45.073 INFO:teuthology.orchestra.run.smithi094.stderr:no change 2024-09-14T06:29:45.085 INFO:tasks.thrashosds.thrasher:Reweighting by: {'max_change': '0.05', 'overage': '110', 'type': 'reweight-by-utilization'} 2024-09-14T06:29:45.085 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight-by-utilization 110 0.05 2024-09-14T06:29:45.127 INFO:tasks.ceph.osd.3.smithi094.stderr:2024-09-14T06:29:45.126+0000 7feeeafe8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33655) UID: 0 2024-09-14T06:29:45.180 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-14T06:29:45.180 INFO:teuthology.orchestra.run.smithi094.stdout: "size": 20, 2024-09-14T06:29:45.180 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 600, 2024-09-14T06:29:45.180 INFO:teuthology.orchestra.run.smithi094.stdout: "ops": [ 2024-09-14T06:29:45.180 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:45.181 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:44425 3.2 3:4603ad7f:::rbd_data.112dca2f6ae1.0000000000000005:17 [read 868352~4096 [fadvise_nocache]] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:29:45.181 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:43.854967+0000", 2024-09-14T06:29:45.181 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 1.3225610839999999, 2024-09-14T06:29:45.181 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.063146025999999994, 2024-09-14T06:29:45.181 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:29:45.181 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:29:45.181 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:29:45.181 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:29:45.181 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:29:45.181 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:45.181 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 44425 2024-09-14T06:29:45.181 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:45.181 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:29:45.182 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:45.182 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:29:45.182 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.854967+0000", 2024-09-14T06:29:45.182 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:45.182 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:45.182 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:45.182 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:29:45.182 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.854967+0000", 2024-09-14T06:29:45.182 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:45.182 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:45.182 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:45.182 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:29:45.182 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.854967+0000", 2024-09-14T06:29:45.183 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.16e-07 2024-09-14T06:29:45.183 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:45.183 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:45.183 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:29:45.183 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.854968+0000", 2024-09-14T06:29:45.183 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.006e-06 2024-09-14T06:29:45.183 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:45.183 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:45.183 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:29:45.183 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.854971+0000", 2024-09-14T06:29:45.183 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.9359999999999999e-06 2024-09-14T06:29:45.184 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:45.184 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:45.184 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:29:45.184 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.854974+0000", 2024-09-14T06:29:45.184 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.1840000000000002e-06 2024-09-14T06:29:45.184 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:45.184 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:45.184 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:29:45.184 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.917945+0000", 2024-09-14T06:29:45.184 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.062970925999999997 2024-09-14T06:29:45.184 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:45.184 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:45.184 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:29:45.184 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.918030+0000", 2024-09-14T06:29:45.185 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.5863000000000001e-05 2024-09-14T06:29:45.185 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:45.185 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:45.185 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:29:45.185 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.918113+0000", 2024-09-14T06:29:45.185 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.2894999999999998e-05 2024-09-14T06:29:45.185 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:45.185 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:45.185 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:45.185 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:45.185 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:45.185 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:44426 3.2 3:4603ad7f:::rbd_data.112dca2f6ae1.0000000000000005:17 [read 892928~4096 [fadvise_nocache]] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:29:45.185 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:43.854977+0000", 2024-09-14T06:29:45.186 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 1.3225516740000001, 2024-09-14T06:29:45.186 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.063369237999999994, 2024-09-14T06:29:45.186 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:29:45.186 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:29:45.186 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:29:45.186 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:29:45.186 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:29:45.186 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:45.186 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 44426 2024-09-14T06:29:45.186 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:45.186 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:29:45.186 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:45.187 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:29:45.187 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.854977+0000", 2024-09-14T06:29:45.187 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:45.187 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:45.187 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:45.187 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:29:45.187 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.854977+0000", 2024-09-14T06:29:45.187 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:45.187 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:45.187 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:45.187 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:29:45.187 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.854977+0000", 2024-09-14T06:29:45.187 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.1199999999999999e-07 2024-09-14T06:29:45.187 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:45.188 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:45.188 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:29:45.188 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.854980+0000", 2024-09-14T06:29:45.188 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.27e-06 2024-09-14T06:29:45.188 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:45.188 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:45.188 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:29:45.188 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.854983+0000", 2024-09-14T06:29:45.188 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.1650000000000002e-06 2024-09-14T06:29:45.188 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:45.188 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:45.188 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:29:45.188 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.854989+0000", 2024-09-14T06:29:45.188 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.6520000000000003e-06 2024-09-14T06:29:45.189 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:45.189 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:45.189 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:29:45.189 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.918143+0000", 2024-09-14T06:29:45.189 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.063154541999999994 2024-09-14T06:29:45.189 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:45.189 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:45.189 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:29:45.189 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.918209+0000", 2024-09-14T06:29:45.189 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.5529000000000002e-05 2024-09-14T06:29:45.189 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:45.189 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:45.189 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:29:45.190 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.918346+0000", 2024-09-14T06:29:45.190 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000136868 2024-09-14T06:29:45.190 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:45.190 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:45.190 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:45.190 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:45.190 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:45.190 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:44427 3.2 3:4603ad7f:::rbd_data.112dca2f6ae1.0000000000000005:17 [read 917504~4096 [fadvise_nocache]] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:29:45.190 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:43.854992+0000", 2024-09-14T06:29:45.190 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 1.322536028, 2024-09-14T06:29:45.190 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.063607792999999996, 2024-09-14T06:29:45.190 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:29:45.190 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:29:45.190 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:29:45.190 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:29:45.191 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:29:45.191 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:45.191 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 44427 2024-09-14T06:29:45.191 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:45.191 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:29:45.191 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:45.191 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:29:45.191 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.854992+0000", 2024-09-14T06:29:45.191 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:45.191 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:45.191 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:45.191 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:29:45.191 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.854992+0000", 2024-09-14T06:29:45.191 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:45.191 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:45.192 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:45.192 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:29:45.192 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.854993+0000", 2024-09-14T06:29:45.192 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.1400000000000001e-07 2024-09-14T06:29:45.192 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:45.192 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:45.192 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:29:45.192 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.854994+0000", 2024-09-14T06:29:45.192 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.091e-06 2024-09-14T06:29:45.192 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:45.192 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:45.192 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:29:45.192 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855001+0000", 2024-09-14T06:29:45.192 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.204e-06 2024-09-14T06:29:45.193 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:45.193 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:45.193 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:29:45.193 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855003+0000", 2024-09-14T06:29:45.193 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.514e-06 2024-09-14T06:29:45.193 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:45.193 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:45.193 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:29:45.193 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.918380+0000", 2024-09-14T06:29:45.193 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.063376738000000002 2024-09-14T06:29:45.193 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:45.193 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:45.193 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:29:45.194 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.918453+0000", 2024-09-14T06:29:45.194 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.3291e-05 2024-09-14T06:29:45.194 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:45.194 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:45.194 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:29:45.194 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.918600+0000", 2024-09-14T06:29:45.194 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00014674100000000001 2024-09-14T06:29:45.194 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:45.194 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:45.194 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:45.194 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:45.194 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:45.194 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:44428 3.2 3:4603ad7f:::rbd_data.112dca2f6ae1.0000000000000005:17 [read 942080~4096 [fadvise_nocache]] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:29:45.194 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:43.855006+0000", 2024-09-14T06:29:45.195 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 1.3225219779999999, 2024-09-14T06:29:45.195 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.063846321999999997, 2024-09-14T06:29:45.195 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:29:45.195 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:29:45.195 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:29:45.195 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:29:45.195 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:29:45.195 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:45.195 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 44428 2024-09-14T06:29:45.195 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:45.195 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:29:45.195 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:45.195 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:29:45.196 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855006+0000", 2024-09-14T06:29:45.196 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:45.196 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:45.196 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:45.196 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:29:45.196 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855006+0000", 2024-09-14T06:29:45.196 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:45.196 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:45.196 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:45.196 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:29:45.196 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855007+0000", 2024-09-14T06:29:45.196 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.1199999999999999e-07 2024-09-14T06:29:45.196 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:45.196 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:45.197 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:29:45.197 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855011+0000", 2024-09-14T06:29:45.197 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.9539999999999998e-06 2024-09-14T06:29:45.197 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:45.197 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:45.197 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:29:45.197 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855014+0000", 2024-09-14T06:29:45.197 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.2930000000000001e-06 2024-09-14T06:29:45.197 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:45.197 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:45.197 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:29:45.197 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855016+0000", 2024-09-14T06:29:45.197 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.3039999999999999e-06 2024-09-14T06:29:45.197 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:45.198 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:45.198 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:29:45.198 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.918634+0000", 2024-09-14T06:29:45.198 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.063617928000000004 2024-09-14T06:29:45.198 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:45.198 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:45.198 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:29:45.198 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.918711+0000", 2024-09-14T06:29:45.198 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.6545000000000007e-05 2024-09-14T06:29:45.198 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:45.198 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:45.198 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:29:45.198 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.918853+0000", 2024-09-14T06:29:45.198 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00014208600000000001 2024-09-14T06:29:45.199 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:45.199 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:45.199 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:45.199 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:45.199 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:45.199 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:44429 3.2 3:4603ad7f:::rbd_data.112dca2f6ae1.0000000000000005:17 [read 966656~4096 [fadvise_nocache]] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:29:45.199 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:43.855019+0000", 2024-09-14T06:29:45.199 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 1.3225093050000001, 2024-09-14T06:29:45.199 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.064059226999999996, 2024-09-14T06:29:45.199 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:29:45.199 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:29:45.199 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:29:45.199 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:29:45.199 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:29:45.200 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:45.200 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 44429 2024-09-14T06:29:45.200 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:45.200 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:29:45.200 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:45.200 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:29:45.200 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855019+0000", 2024-09-14T06:29:45.200 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:45.200 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:45.200 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:45.200 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:29:45.200 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855019+0000", 2024-09-14T06:29:45.200 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:45.200 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:45.201 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:45.201 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:29:45.201 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855019+0000", 2024-09-14T06:29:45.201 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.28e-07 2024-09-14T06:29:45.201 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:45.201 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:45.201 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:29:45.201 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855020+0000", 2024-09-14T06:29:45.201 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.0330000000000001e-06 2024-09-14T06:29:45.201 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:45.201 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:45.201 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:29:45.201 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855023+0000", 2024-09-14T06:29:45.201 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.79e-06 2024-09-14T06:29:45.202 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:45.202 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:45.202 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:29:45.202 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855025+0000", 2024-09-14T06:29:45.202 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.216e-06 2024-09-14T06:29:45.202 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:45.202 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:45.202 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:29:45.202 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.918886+0000", 2024-09-14T06:29:45.202 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.063861037999999995 2024-09-14T06:29:45.202 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:45.202 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:45.202 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:29:45.202 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.918952+0000", 2024-09-14T06:29:45.203 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.5993000000000003e-05 2024-09-14T06:29:45.203 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:45.203 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:45.203 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:29:45.203 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.919078+0000", 2024-09-14T06:29:45.203 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00012592899999999999 2024-09-14T06:29:45.203 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:45.203 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:45.203 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:45.203 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:45.203 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:45.203 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:44430 3.2 3:4603ad7f:::rbd_data.112dca2f6ae1.0000000000000005:17 [read 991232~4096 [fadvise_nocache]] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:29:45.203 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:43.855028+0000", 2024-09-14T06:29:45.203 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 1.322500295, 2024-09-14T06:29:45.204 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.064284346000000006, 2024-09-14T06:29:45.204 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:29:45.204 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:29:45.204 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:29:45.204 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:29:45.204 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:29:45.204 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:45.204 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 44430 2024-09-14T06:29:45.204 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:45.204 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:29:45.204 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:45.204 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:29:45.204 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855028+0000", 2024-09-14T06:29:45.204 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:45.205 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:45.205 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:45.205 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:29:45.205 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855028+0000", 2024-09-14T06:29:45.205 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:45.205 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:45.205 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:45.205 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:29:45.205 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855028+0000", 2024-09-14T06:29:45.205 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.1199999999999999e-07 2024-09-14T06:29:45.205 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:45.206 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:45.206 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:29:45.206 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855029+0000", 2024-09-14T06:29:45.206 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.1100000000000004e-07 2024-09-14T06:29:45.206 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:45.206 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:45.206 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:29:45.206 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855032+0000", 2024-09-14T06:29:45.206 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.7889999999999999e-06 2024-09-14T06:29:45.206 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:45.206 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:45.206 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:29:45.206 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855034+0000", 2024-09-14T06:29:45.206 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.125e-06 2024-09-14T06:29:45.207 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:45.207 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:45.207 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:29:45.207 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.919110+0000", 2024-09-14T06:29:45.207 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.064076413999999998 2024-09-14T06:29:45.207 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:45.207 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:45.207 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:29:45.207 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.919186+0000", 2024-09-14T06:29:45.207 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.5784999999999994e-05 2024-09-14T06:29:45.207 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:45.207 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:45.207 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:29:45.207 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.919312+0000", 2024-09-14T06:29:45.208 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00012611 2024-09-14T06:29:45.208 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:45.208 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:45.208 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:45.208 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:45.208 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:45.208 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:44431 3.2 3:4603ad7f:::rbd_data.112dca2f6ae1.0000000000000005:17 [read 1015808~4096 [fadvise_nocache]] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:29:45.208 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:43.855037+0000", 2024-09-14T06:29:45.208 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 1.3224914270000001, 2024-09-14T06:29:45.208 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.064469755000000004, 2024-09-14T06:29:45.208 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:29:45.208 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:29:45.208 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:29:45.208 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:29:45.209 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:29:45.209 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:45.209 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 44431 2024-09-14T06:29:45.209 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:45.209 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:29:45.209 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:45.209 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:29:45.209 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855037+0000", 2024-09-14T06:29:45.209 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:45.209 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:45.209 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:45.209 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:29:45.209 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855037+0000", 2024-09-14T06:29:45.210 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:45.210 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:45.210 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:45.210 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:29:45.210 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855037+0000", 2024-09-14T06:29:45.210 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.1e-07 2024-09-14T06:29:45.210 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:45.210 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:45.210 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:29:45.210 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855038+0000", 2024-09-14T06:29:45.210 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.9400000000000004e-07 2024-09-14T06:29:45.210 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:45.210 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:45.210 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:29:45.210 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855041+0000", 2024-09-14T06:29:45.211 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.9639999999999999e-06 2024-09-14T06:29:45.211 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:45.211 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:45.211 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:29:45.211 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855043+0000", 2024-09-14T06:29:45.211 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.2259999999999999e-06 2024-09-14T06:29:45.211 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:45.211 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:45.211 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:29:45.211 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.919347+0000", 2024-09-14T06:29:45.211 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.064304091999999993 2024-09-14T06:29:45.211 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:45.211 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:45.211 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:29:45.211 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.919393+0000", 2024-09-14T06:29:45.212 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.5340999999999998e-05 2024-09-14T06:29:45.212 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:45.212 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:45.212 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:29:45.212 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.919507+0000", 2024-09-14T06:29:45.212 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000114028 2024-09-14T06:29:45.212 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:45.212 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:45.212 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:45.212 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:45.212 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:45.212 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:44432 3.2 3:4603ad7f:::rbd_data.112dca2f6ae1.0000000000000005:17 [read 1040384~4096 [fadvise_nocache]] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:29:45.212 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:43.855046+0000", 2024-09-14T06:29:45.212 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 1.3224823589999999, 2024-09-14T06:29:45.212 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.064639253999999993, 2024-09-14T06:29:45.212 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:29:45.213 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:29:45.213 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:29:45.213 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:29:45.213 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:29:45.213 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:45.213 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 44432 2024-09-14T06:29:45.213 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:45.213 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:29:45.213 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:45.213 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:29:45.213 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855046+0000", 2024-09-14T06:29:45.213 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:45.213 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:45.213 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:45.214 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:29:45.214 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855046+0000", 2024-09-14T06:29:45.214 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:45.214 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:45.214 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:45.214 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:29:45.214 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855046+0000", 2024-09-14T06:29:45.214 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.05e-07 2024-09-14T06:29:45.214 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:45.214 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:45.214 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:29:45.214 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855047+0000", 2024-09-14T06:29:45.214 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.879999999999999e-07 2024-09-14T06:29:45.214 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:45.214 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:45.215 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:29:45.215 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855050+0000", 2024-09-14T06:29:45.215 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.8459999999999999e-06 2024-09-14T06:29:45.215 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:45.215 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:45.215 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:29:45.215 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855052+0000", 2024-09-14T06:29:45.215 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.136e-06 2024-09-14T06:29:45.215 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:45.215 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:45.215 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:29:45.215 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.919536+0000", 2024-09-14T06:29:45.215 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.064484132 2024-09-14T06:29:45.215 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:45.216 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:45.216 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:29:45.216 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.919611+0000", 2024-09-14T06:29:45.216 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.4407999999999996e-05 2024-09-14T06:29:45.216 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:45.216 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:45.216 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:29:45.216 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.919685+0000", 2024-09-14T06:29:45.216 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.4538999999999999e-05 2024-09-14T06:29:45.216 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:45.216 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:45.216 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:45.216 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:45.216 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:45.217 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:44518 3.2 3:4190d882:::rbd_data.112dca2f6ae1.0000000000000002:17 [read 12288~4096 [fadvise_nocache]] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:29:45.217 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:43.855055+0000", 2024-09-14T06:29:45.217 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 1.322473475, 2024-09-14T06:29:45.217 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.064894317000000007, 2024-09-14T06:29:45.217 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:29:45.217 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:29:45.217 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:29:45.217 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:29:45.217 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:29:45.217 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:45.217 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 44518 2024-09-14T06:29:45.217 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:45.217 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:29:45.217 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:45.217 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:29:45.218 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855055+0000", 2024-09-14T06:29:45.218 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:45.218 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:45.218 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:45.218 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:29:45.218 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855055+0000", 2024-09-14T06:29:45.218 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:45.218 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:45.218 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:45.218 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:29:45.218 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855055+0000", 2024-09-14T06:29:45.218 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.0699999999999999e-07 2024-09-14T06:29:45.218 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:45.218 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:45.218 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:29:45.218 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855056+0000", 2024-09-14T06:29:45.219 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.9400000000000004e-07 2024-09-14T06:29:45.219 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:45.219 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:45.219 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:29:45.219 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855059+0000", 2024-09-14T06:29:45.219 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.8870000000000001e-06 2024-09-14T06:29:45.219 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:45.219 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:45.219 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:29:45.219 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855061+0000", 2024-09-14T06:29:45.219 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.1919999999999999e-06 2024-09-14T06:29:45.219 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:45.219 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:45.219 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:29:45.220 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.919719+0000", 2024-09-14T06:29:45.220 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.064658013 2024-09-14T06:29:45.220 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:45.220 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:45.220 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:29:45.220 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.919788+0000", 2024-09-14T06:29:45.220 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.9282000000000005e-05 2024-09-14T06:29:45.220 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:45.220 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:45.220 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:29:45.220 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.919949+0000", 2024-09-14T06:29:45.220 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00016084199999999999 2024-09-14T06:29:45.220 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:45.220 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:45.221 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:45.221 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:45.221 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:45.221 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:44519 3.2 3:4190d882:::rbd_data.112dca2f6ae1.0000000000000002:17 [read 36864~4096 [fadvise_nocache]] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:29:45.221 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:43.855064+0000", 2024-09-14T06:29:45.221 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 1.322464498, 2024-09-14T06:29:45.221 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.065081865000000003, 2024-09-14T06:29:45.221 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:29:45.221 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:29:45.221 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:29:45.221 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:29:45.221 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:29:45.221 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:45.221 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 44519 2024-09-14T06:29:45.222 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:45.222 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:29:45.222 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:45.222 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:29:45.223 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855064+0000", 2024-09-14T06:29:45.223 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:45.223 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:45.223 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:45.223 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:29:45.223 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855064+0000", 2024-09-14T06:29:45.223 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:45.224 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:45.224 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:45.224 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:29:45.224 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855064+0000", 2024-09-14T06:29:45.224 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.1e-07 2024-09-14T06:29:45.224 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:45.224 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:45.225 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:29:45.225 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855065+0000", 2024-09-14T06:29:45.225 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.035e-06 2024-09-14T06:29:45.225 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:45.225 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:45.225 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:29:45.225 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855068+0000", 2024-09-14T06:29:45.226 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.8339999999999999e-06 2024-09-14T06:29:45.226 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:45.226 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:45.226 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:29:45.226 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855071+0000", 2024-09-14T06:29:45.226 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.3450000000000002e-06 2024-09-14T06:29:45.227 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:45.227 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:45.227 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:29:45.227 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.919979+0000", 2024-09-14T06:29:45.227 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.064907647999999998 2024-09-14T06:29:45.227 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:45.227 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:45.227 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:29:45.227 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.920026+0000", 2024-09-14T06:29:45.228 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.7595000000000001e-05 2024-09-14T06:29:45.228 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:45.228 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:45.228 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:29:45.228 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.920146+0000", 2024-09-14T06:29:45.228 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00011919800000000001 2024-09-14T06:29:45.228 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:45.228 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:45.229 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:45.229 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:45.229 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:45.229 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:44520 3.2 3:4190d882:::rbd_data.112dca2f6ae1.0000000000000002:17 [read 61440~4096 [fadvise_nocache]] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:29:45.229 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:43.855075+0000", 2024-09-14T06:29:45.229 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 1.322453447, 2024-09-14T06:29:45.230 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.065335078000000005, 2024-09-14T06:29:45.230 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:29:45.230 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:29:45.230 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:29:45.230 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:29:45.230 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:29:45.230 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:45.231 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 44520 2024-09-14T06:29:45.231 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:45.231 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:29:45.231 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:45.231 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:29:45.231 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855075+0000", 2024-09-14T06:29:45.231 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:45.231 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:45.231 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:45.231 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:29:45.232 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855075+0000", 2024-09-14T06:29:45.232 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:45.232 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:45.232 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:45.232 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:29:45.232 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855075+0000", 2024-09-14T06:29:45.232 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.1e-07 2024-09-14T06:29:45.232 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:45.232 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:45.233 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:29:45.233 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855076+0000", 2024-09-14T06:29:45.233 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.0839999999999999e-06 2024-09-14T06:29:45.233 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:45.233 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:45.233 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:29:45.233 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855079+0000", 2024-09-14T06:29:45.233 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.9359999999999999e-06 2024-09-14T06:29:45.234 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:45.234 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:45.234 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:29:45.234 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855084+0000", 2024-09-14T06:29:45.234 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.1730000000000003e-06 2024-09-14T06:29:45.234 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:45.234 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:45.234 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:29:45.234 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.920175+0000", 2024-09-14T06:29:45.235 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.065090234999999996 2024-09-14T06:29:45.235 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:45.235 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:45.235 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:29:45.235 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.920250+0000", 2024-09-14T06:29:45.235 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.5807000000000001e-05 2024-09-14T06:29:45.235 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:45.235 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:45.236 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:29:45.236 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.920410+0000", 2024-09-14T06:29:45.236 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00015963300000000001 2024-09-14T06:29:45.236 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:45.236 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:45.236 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:45.236 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:45.236 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:45.237 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:44521 3.2 3:4190d882:::rbd_data.112dca2f6ae1.0000000000000002:17 [read 86016~4096 [fadvise_nocache]] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:29:45.237 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:43.855087+0000", 2024-09-14T06:29:45.237 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 1.3224409660000001, 2024-09-14T06:29:45.237 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.065563264999999996, 2024-09-14T06:29:45.237 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:29:45.237 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:29:45.237 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:29:45.237 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:29:45.238 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:29:45.238 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:45.238 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 44521 2024-09-14T06:29:45.238 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:45.238 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:29:45.238 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:45.238 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:29:45.238 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855087+0000", 2024-09-14T06:29:45.238 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:45.239 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:45.239 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:45.239 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:29:45.239 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855087+0000", 2024-09-14T06:29:45.239 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:45.239 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:45.239 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:45.239 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:29:45.240 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855088+0000", 2024-09-14T06:29:45.240 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.3699999999999999e-07 2024-09-14T06:29:45.240 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:45.240 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:45.240 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:29:45.240 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855089+0000", 2024-09-14T06:29:45.240 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.9900000000000009e-07 2024-09-14T06:29:45.240 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:45.240 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:45.241 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:29:45.241 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855091+0000", 2024-09-14T06:29:45.241 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.8940000000000002e-06 2024-09-14T06:29:45.241 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:45.241 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:45.241 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:29:45.241 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855094+0000", 2024-09-14T06:29:45.241 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.351e-06 2024-09-14T06:29:45.242 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:45.242 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:45.242 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:29:45.242 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.920437+0000", 2024-09-14T06:29:45.242 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.065343221000000007 2024-09-14T06:29:45.242 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:45.242 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:45.242 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:29:45.242 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.920481+0000", 2024-09-14T06:29:45.243 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.4431000000000002e-05 2024-09-14T06:29:45.243 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:45.243 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:45.243 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:29:45.243 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.920651+0000", 2024-09-14T06:29:45.243 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00016913200000000001 2024-09-14T06:29:45.243 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:45.243 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:45.243 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:45.244 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:45.244 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:45.244 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:44522 3.2 3:4190d882:::rbd_data.112dca2f6ae1.0000000000000002:17 [read 110592~4096 [fadvise_nocache]] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:29:45.244 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:43.855098+0000", 2024-09-14T06:29:45.244 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 1.3224307390000001, 2024-09-14T06:29:45.244 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.065827628999999999, 2024-09-14T06:29:45.244 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:29:45.245 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:29:45.245 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:29:45.245 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:29:45.245 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:29:45.245 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:45.245 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 44522 2024-09-14T06:29:45.245 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:45.245 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:29:45.246 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:45.246 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:29:45.246 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855098+0000", 2024-09-14T06:29:45.246 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:45.246 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:45.246 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:45.246 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:29:45.246 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855098+0000", 2024-09-14T06:29:45.246 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:45.247 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:45.247 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:45.247 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:29:45.247 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855098+0000", 2024-09-14T06:29:45.247 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.1e-07 2024-09-14T06:29:45.247 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:45.247 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:45.247 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:29:45.247 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855099+0000", 2024-09-14T06:29:45.247 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.012e-06 2024-09-14T06:29:45.247 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:45.248 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:45.248 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:29:45.248 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855102+0000", 2024-09-14T06:29:45.248 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.9380000000000001e-06 2024-09-14T06:29:45.248 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:45.248 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:45.248 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:29:45.248 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855108+0000", 2024-09-14T06:29:45.248 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.4339999999999997e-06 2024-09-14T06:29:45.248 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:45.248 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:45.248 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:29:45.248 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.920681+0000", 2024-09-14T06:29:45.249 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.065572943999999994 2024-09-14T06:29:45.249 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:45.249 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:45.249 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:29:45.249 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.920724+0000", 2024-09-14T06:29:45.249 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.2877000000000003e-05 2024-09-14T06:29:45.249 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:45.249 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:45.249 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:29:45.249 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.920925+0000", 2024-09-14T06:29:45.249 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00020121399999999999 2024-09-14T06:29:45.249 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:45.249 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:45.249 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:45.250 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:45.250 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:45.250 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:44523 3.2 3:4190d882:::rbd_data.112dca2f6ae1.0000000000000002:17 [read 135168~4096 [fadvise_nocache]] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:29:45.250 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:43.855111+0000", 2024-09-14T06:29:45.250 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 1.322417017, 2024-09-14T06:29:45.250 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.066028198999999996, 2024-09-14T06:29:45.250 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:29:45.250 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:29:45.250 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:29:45.250 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:29:45.250 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:29:45.250 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:45.250 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 44523 2024-09-14T06:29:45.250 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:45.251 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:29:45.251 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:45.251 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:29:45.251 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855111+0000", 2024-09-14T06:29:45.251 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:45.251 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:45.251 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:45.251 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:29:45.251 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855111+0000", 2024-09-14T06:29:45.251 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:45.251 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:45.251 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:45.251 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:29:45.251 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855112+0000", 2024-09-14T06:29:45.252 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.1199999999999999e-07 2024-09-14T06:29:45.252 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:45.252 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:45.252 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:29:45.252 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855115+0000", 2024-09-14T06:29:45.252 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.3380000000000001e-06 2024-09-14T06:29:45.252 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:45.252 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:45.252 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:29:45.252 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855118+0000", 2024-09-14T06:29:45.252 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.366e-06 2024-09-14T06:29:45.252 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:45.252 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:45.253 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:29:45.253 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855121+0000", 2024-09-14T06:29:45.253 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.8329999999999998e-06 2024-09-14T06:29:45.253 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:45.253 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:45.253 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:29:45.253 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.920956+0000", 2024-09-14T06:29:45.253 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.065835251999999997 2024-09-14T06:29:45.253 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:45.253 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:45.254 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:29:45.254 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.921001+0000", 2024-09-14T06:29:45.254 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.4778999999999999e-05 2024-09-14T06:29:45.254 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:45.254 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:45.254 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:29:45.255 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.921139+0000", 2024-09-14T06:29:45.255 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000138419 2024-09-14T06:29:45.255 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:45.255 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:45.255 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:45.255 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:45.255 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:45.255 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:44524 3.2 3:4190d882:::rbd_data.112dca2f6ae1.0000000000000002:17 [read 159744~4096 [fadvise_nocache]] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:29:45.255 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:43.855124+0000", 2024-09-14T06:29:45.255 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 1.3224040379999999, 2024-09-14T06:29:45.256 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.066249261000000004, 2024-09-14T06:29:45.256 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:29:45.256 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:29:45.256 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:29:45.256 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:29:45.256 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:29:45.256 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:45.256 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 44524 2024-09-14T06:29:45.256 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:45.256 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:29:45.256 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:45.256 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:29:45.256 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855124+0000", 2024-09-14T06:29:45.257 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:45.257 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:45.257 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:45.257 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:29:45.257 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855124+0000", 2024-09-14T06:29:45.257 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:45.257 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:45.257 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:45.257 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:29:45.257 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861086+0000", 2024-09-14T06:29:45.257 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0059613829999999998 2024-09-14T06:29:45.257 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:45.257 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:45.257 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:29:45.258 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861100+0000", 2024-09-14T06:29:45.258 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.4644000000000001e-05 2024-09-14T06:29:45.258 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:45.258 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:45.258 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:29:45.258 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861109+0000", 2024-09-14T06:29:45.258 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.541e-06 2024-09-14T06:29:45.258 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:45.258 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:45.258 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:29:45.259 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861117+0000", 2024-09-14T06:29:45.259 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.1049999999999995e-06 2024-09-14T06:29:45.259 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:45.259 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:45.259 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:29:45.259 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.921169+0000", 2024-09-14T06:29:45.259 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.060052405000000003 2024-09-14T06:29:45.259 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:45.259 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:45.259 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:29:45.259 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.921213+0000", 2024-09-14T06:29:45.259 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.3565e-05 2024-09-14T06:29:45.260 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:45.260 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:45.260 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:29:45.260 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.921374+0000", 2024-09-14T06:29:45.260 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00016061800000000001 2024-09-14T06:29:45.260 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:45.260 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:45.260 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:45.260 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:45.260 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:45.260 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:44536 3.2 3:4190d882:::rbd_data.112dca2f6ae1.0000000000000002:17 [read 454656~4096 [fadvise_nocache]] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:29:45.260 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:43.861384+0000", 2024-09-14T06:29:45.261 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 1.316144481, 2024-09-14T06:29:45.261 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.063058528000000003, 2024-09-14T06:29:45.261 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:29:45.261 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:29:45.261 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:29:45.261 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:29:45.261 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:29:45.261 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:45.261 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 44536 2024-09-14T06:29:45.261 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:45.261 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:29:45.261 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:45.261 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:29:45.262 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861384+0000", 2024-09-14T06:29:45.262 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:45.262 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:45.262 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:45.262 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:29:45.262 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861384+0000", 2024-09-14T06:29:45.262 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:45.262 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:45.262 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:45.262 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:29:45.262 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861384+0000", 2024-09-14T06:29:45.262 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.4000000000000002e-07 2024-09-14T06:29:45.262 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:45.262 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:45.262 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:29:45.263 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861386+0000", 2024-09-14T06:29:45.263 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.891e-06 2024-09-14T06:29:45.263 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:45.263 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:45.263 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:29:45.263 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861392+0000", 2024-09-14T06:29:45.263 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.9939999999999997e-06 2024-09-14T06:29:45.263 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:45.263 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:45.263 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:29:45.264 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861398+0000", 2024-09-14T06:29:45.264 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.507e-06 2024-09-14T06:29:45.264 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:45.264 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:45.264 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:29:45.264 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.924149+0000", 2024-09-14T06:29:45.264 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.062751135999999999 2024-09-14T06:29:45.264 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:45.264 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:45.264 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:29:45.264 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.924226+0000", 2024-09-14T06:29:45.264 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.7517999999999999e-05 2024-09-14T06:29:45.264 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:45.265 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:45.265 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:29:45.265 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.924442+0000", 2024-09-14T06:29:45.265 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000216042 2024-09-14T06:29:45.265 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:45.265 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:45.265 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:45.265 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:45.265 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:45.265 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:44537 3.2 3:4190d882:::rbd_data.112dca2f6ae1.0000000000000002:17 [read 479232~4096 [fadvise_nocache]] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:29:45.265 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:43.861405+0000", 2024-09-14T06:29:45.265 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 1.316123435, 2024-09-14T06:29:45.265 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.063351795000000002, 2024-09-14T06:29:45.265 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:29:45.266 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:29:45.266 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:29:45.266 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:29:45.266 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:29:45.266 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:45.266 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 44537 2024-09-14T06:29:45.266 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:45.266 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:29:45.266 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:45.266 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:29:45.266 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861405+0000", 2024-09-14T06:29:45.266 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:45.266 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:45.266 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:45.267 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:29:45.267 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861405+0000", 2024-09-14T06:29:45.267 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:45.267 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:45.267 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:45.267 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:29:45.267 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861405+0000", 2024-09-14T06:29:45.267 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.9099999999999999e-07 2024-09-14T06:29:45.267 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:45.267 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:45.267 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:29:45.267 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861407+0000", 2024-09-14T06:29:45.267 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.075e-06 2024-09-14T06:29:45.267 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:45.268 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:45.268 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:29:45.268 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861414+0000", 2024-09-14T06:29:45.268 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.4659999999999996e-06 2024-09-14T06:29:45.268 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:45.268 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:45.268 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:29:45.268 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861418+0000", 2024-09-14T06:29:45.268 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.5519999999999998e-06 2024-09-14T06:29:45.268 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:45.268 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:45.268 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:29:45.268 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.924474+0000", 2024-09-14T06:29:45.268 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.063056012999999994 2024-09-14T06:29:45.269 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:45.269 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:45.269 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:29:45.269 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.924556+0000", 2024-09-14T06:29:45.269 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.1181999999999996e-05 2024-09-14T06:29:45.269 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:45.269 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:45.269 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:29:45.270 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.924757+0000", 2024-09-14T06:29:45.270 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00020111599999999999 2024-09-14T06:29:45.270 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:45.270 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:45.270 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:45.270 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:45.270 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:45.270 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:44538 3.2 3:4190d882:::rbd_data.112dca2f6ae1.0000000000000002:17 [read 503808~4096 [fadvise_nocache]] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:29:45.270 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:43.861425+0000", 2024-09-14T06:29:45.270 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 1.316103394, 2024-09-14T06:29:45.270 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.063627125000000007, 2024-09-14T06:29:45.270 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:29:45.270 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:29:45.270 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:29:45.271 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:29:45.271 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:29:45.271 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:45.271 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 44538 2024-09-14T06:29:45.271 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:45.271 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:29:45.271 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:45.271 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:29:45.271 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861425+0000", 2024-09-14T06:29:45.271 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:45.271 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:45.271 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:45.271 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:29:45.271 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861425+0000", 2024-09-14T06:29:45.272 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:45.272 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:45.272 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:45.272 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:29:45.272 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861425+0000", 2024-09-14T06:29:45.272 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.9999999999999998e-07 2024-09-14T06:29:45.272 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:45.272 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:45.272 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:29:45.272 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861427+0000", 2024-09-14T06:29:45.272 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.9829999999999999e-06 2024-09-14T06:29:45.272 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:45.272 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:45.272 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:29:45.273 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861433+0000", 2024-09-14T06:29:45.273 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.2889999999999997e-06 2024-09-14T06:29:45.273 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:45.273 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:45.273 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:29:45.273 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861454+0000", 2024-09-14T06:29:45.273 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.1844999999999998e-05 2024-09-14T06:29:45.273 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:45.273 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:45.273 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:29:45.273 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.924786+0000", 2024-09-14T06:29:45.273 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.063331497 2024-09-14T06:29:45.273 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:45.273 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:45.274 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:29:45.274 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.924864+0000", 2024-09-14T06:29:45.274 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.7744000000000006e-05 2024-09-14T06:29:45.274 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:45.274 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:45.274 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:29:45.274 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.925052+0000", 2024-09-14T06:29:45.274 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00018836699999999999 2024-09-14T06:29:45.274 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:45.274 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:45.274 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:45.274 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:45.274 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:45.274 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:44539 3.2 3:4190d882:::rbd_data.112dca2f6ae1.0000000000000002:17 [read 528384~4096 [fadvise_nocache]] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:29:45.275 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:43.861462+0000", 2024-09-14T06:29:45.275 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 1.316066067, 2024-09-14T06:29:45.275 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.063855705999999998, 2024-09-14T06:29:45.275 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:29:45.275 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:29:45.275 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:29:45.275 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:29:45.275 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:29:45.275 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:45.275 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 44539 2024-09-14T06:29:45.275 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:45.275 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:29:45.275 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:45.275 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:29:45.276 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861462+0000", 2024-09-14T06:29:45.276 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:45.276 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:45.276 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:45.276 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:29:45.276 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861462+0000", 2024-09-14T06:29:45.276 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:45.276 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:45.276 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:45.276 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:29:45.276 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861463+0000", 2024-09-14T06:29:45.276 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.5200000000000002e-07 2024-09-14T06:29:45.276 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:45.276 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:45.277 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:29:45.277 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861465+0000", 2024-09-14T06:29:45.277 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.5189999999999999e-06 2024-09-14T06:29:45.277 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:45.277 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:45.277 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:29:45.277 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861471+0000", 2024-09-14T06:29:45.277 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.6899999999999997e-06 2024-09-14T06:29:45.277 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:45.277 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:45.277 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:29:45.277 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861475+0000", 2024-09-14T06:29:45.277 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.5730000000000001e-06 2024-09-14T06:29:45.277 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:45.278 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:45.278 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:29:45.278 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.925085+0000", 2024-09-14T06:29:45.278 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.063609902999999995 2024-09-14T06:29:45.278 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:45.278 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:45.278 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:29:45.278 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.925145+0000", 2024-09-14T06:29:45.278 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.9383999999999999e-05 2024-09-14T06:29:45.278 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:45.278 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:45.278 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:29:45.278 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.925318+0000", 2024-09-14T06:29:45.278 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00017318499999999999 2024-09-14T06:29:45.279 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:45.279 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:45.279 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:45.279 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:45.279 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:45.279 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:44540 3.2 3:4190d882:::rbd_data.112dca2f6ae1.0000000000000002:17 [read 552960~4096 [fadvise_nocache]] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:29:45.279 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:43.861483+0000", 2024-09-14T06:29:45.279 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 1.3160456360000001, 2024-09-14T06:29:45.279 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.064065681999999999, 2024-09-14T06:29:45.279 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:29:45.279 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:29:45.279 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:29:45.279 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:29:45.280 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:29:45.280 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:45.280 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 44540 2024-09-14T06:29:45.280 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:45.280 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:29:45.280 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:45.280 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:29:45.281 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861483+0000", 2024-09-14T06:29:45.281 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:45.281 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:45.281 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:45.281 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:29:45.281 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861483+0000", 2024-09-14T06:29:45.281 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:45.281 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:45.281 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:45.281 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:29:45.281 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.867079+0000", 2024-09-14T06:29:45.281 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0055966480000000001 2024-09-14T06:29:45.281 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:45.282 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:45.282 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:29:45.282 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.867086+0000", 2024-09-14T06:29:45.282 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.387e-06 2024-09-14T06:29:45.282 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:45.282 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:45.282 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:29:45.282 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.867094+0000", 2024-09-14T06:29:45.282 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.9419999999999995e-06 2024-09-14T06:29:45.282 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:45.282 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:45.282 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:29:45.283 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.867109+0000", 2024-09-14T06:29:45.283 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.5818999999999999e-05 2024-09-14T06:29:45.283 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:45.283 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:45.283 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:29:45.283 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.925354+0000", 2024-09-14T06:29:45.283 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.058244047 2024-09-14T06:29:45.283 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:45.283 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:45.283 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:29:45.283 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.925406+0000", 2024-09-14T06:29:45.283 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.2661999999999998e-05 2024-09-14T06:29:45.283 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:45.284 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:45.284 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:29:45.284 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.925548+0000", 2024-09-14T06:29:45.284 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000142177 2024-09-14T06:29:45.284 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:45.284 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:45.284 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:45.284 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:45.284 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:45.284 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-14T06:29:45.286 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-14T06:29:45.286 INFO:tasks.ceph.osd.4.smithi138.stderr:2024-09-14T06:29:45.285+0000 7f6c653c8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33615) UID: 0 2024-09-14T06:29:45.387 INFO:tasks.ceph.osd.4.smithi138.stderr:2024-09-14T06:29:45.386+0000 7f6c653c8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33615) UID: 0 2024-09-14T06:29:45.390 INFO:teuthology.orchestra.run.smithi138.stdout:{ 2024-09-14T06:29:45.390 INFO:teuthology.orchestra.run.smithi138.stdout: "ops": [], 2024-09-14T06:29:45.390 INFO:teuthology.orchestra.run.smithi138.stdout: "num_ops": 0 2024-09-14T06:29:45.390 INFO:teuthology.orchestra.run.smithi138.stdout:} 2024-09-14T06:29:45.399 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-14T06:29:45.412 INFO:teuthology.orchestra.run.smithi094.stdout:moved 0 / 82 (0%) 2024-09-14T06:29:45.412 INFO:teuthology.orchestra.run.smithi094.stdout:avg 10.25 2024-09-14T06:29:45.412 INFO:teuthology.orchestra.run.smithi094.stdout:stddev 3.26917 -> 3.26917 (expected baseline 2.99479) 2024-09-14T06:29:45.412 INFO:teuthology.orchestra.run.smithi094.stdout:min osd.2 with 5 -> 5 pgs (0.487805 -> 0.487805 * mean) 2024-09-14T06:29:45.412 INFO:teuthology.orchestra.run.smithi094.stdout:max osd.7 with 14 -> 14 pgs (1.36585 -> 1.36585 * mean) 2024-09-14T06:29:45.412 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-14T06:29:45.412 INFO:teuthology.orchestra.run.smithi094.stdout:oload 110 2024-09-14T06:29:45.412 INFO:teuthology.orchestra.run.smithi094.stdout:max_change 0.05 2024-09-14T06:29:45.413 INFO:teuthology.orchestra.run.smithi094.stdout:max_change_osds 4 2024-09-14T06:29:45.413 INFO:teuthology.orchestra.run.smithi094.stdout:average_utilization 0.0004 2024-09-14T06:29:45.413 INFO:teuthology.orchestra.run.smithi094.stdout:overload_utilization 0.0004 2024-09-14T06:29:45.413 INFO:teuthology.orchestra.run.smithi094.stderr:no change 2024-09-14T06:29:45.424 INFO:tasks.thrashosds.thrasher:Reweighting by: {'max_change': '1.0', 'overage': '1000', 'type': 'test-reweight-by-utilization'} 2024-09-14T06:29:45.424 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd test-reweight-by-utilization 1000 1.0 2024-09-14T06:29:45.488 INFO:tasks.ceph.osd.2.smithi094.stderr:2024-09-14T06:29:45.487+0000 7faf3269b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33649) UID: 0 2024-09-14T06:29:45.543 INFO:teuthology.orchestra.run.smithi138.stdout:{ 2024-09-14T06:29:45.543 INFO:teuthology.orchestra.run.smithi138.stdout: "ops": [], 2024-09-14T06:29:45.543 INFO:teuthology.orchestra.run.smithi138.stdout: "complaint_time": 30, 2024-09-14T06:29:45.543 INFO:teuthology.orchestra.run.smithi138.stdout: "num_blocked_ops": 0 2024-09-14T06:29:45.543 INFO:teuthology.orchestra.run.smithi138.stdout:} 2024-09-14T06:29:45.551 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-14T06:29:45.595 INFO:tasks.ceph.osd.4.smithi138.stderr:2024-09-14T06:29:45.594+0000 7f6c653c8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33615) UID: 0 2024-09-14T06:29:45.656 INFO:teuthology.orchestra.run.smithi138.stdout:{ 2024-09-14T06:29:45.656 INFO:teuthology.orchestra.run.smithi138.stdout: "size": 20, 2024-09-14T06:29:45.656 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 600, 2024-09-14T06:29:45.656 INFO:teuthology.orchestra.run.smithi138.stdout: "ops": [ 2024-09-14T06:29:45.656 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:45.657 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:19328 3.3 3:c34e5e51:::rbd_data.112d92027590.0000000000000024:f [read 1926144~1024 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:29:45.657 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:27.059768+0000", 2024-09-14T06:29:45.657 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 18.59462611, 2024-09-14T06:29:45.657 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046369147999999999, 2024-09-14T06:29:45.657 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:29:45.657 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:29:45.657 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:29:45.657 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:29:45.657 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:29:45.657 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:45.657 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 19328 2024-09-14T06:29:45.658 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:45.658 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:29:45.658 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:45.658 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:29:45.658 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.059768+0000", 2024-09-14T06:29:45.658 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:45.658 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:45.658 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:45.658 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:29:45.658 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.059768+0000", 2024-09-14T06:29:45.659 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:45.659 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:45.659 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:45.659 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:29:45.659 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.059874+0000", 2024-09-14T06:29:45.659 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00010551400000000001 2024-09-14T06:29:45.659 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:45.659 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:45.659 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:29:45.659 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.059877+0000", 2024-09-14T06:29:45.659 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.8279999999999999e-06 2024-09-14T06:29:45.659 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:45.659 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:45.659 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:29:45.660 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.059884+0000", 2024-09-14T06:29:45.660 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.6389999999999992e-06 2024-09-14T06:29:45.660 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:45.660 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:45.660 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:29:45.660 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.059890+0000", 2024-09-14T06:29:45.660 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.49e-06 2024-09-14T06:29:45.660 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:45.660 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:45.660 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:29:45.661 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.105804+0000", 2024-09-14T06:29:45.661 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.045914242000000001 2024-09-14T06:29:45.661 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:45.661 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:45.661 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:29:45.661 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.105886+0000", 2024-09-14T06:29:45.661 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.2305999999999994e-05 2024-09-14T06:29:45.661 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:45.661 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:45.661 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:29:45.661 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.106138+0000", 2024-09-14T06:29:45.661 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.000251129 2024-09-14T06:29:45.661 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:45.662 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:29:45.662 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:45.662 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:45.662 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:45.662 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:19329 3.3 3:c34e5e51:::rbd_data.112d92027590.0000000000000024:f [read 1936384~1024 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:29:45.662 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:27.059899+0000", 2024-09-14T06:29:45.662 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 18.594495587000001, 2024-09-14T06:29:45.662 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046572122000000001, 2024-09-14T06:29:45.662 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:29:45.662 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:29:45.662 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:29:45.662 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:29:45.662 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:29:45.662 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:45.662 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 19329 2024-09-14T06:29:45.663 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:45.663 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:29:45.663 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:45.663 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:29:45.663 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.059899+0000", 2024-09-14T06:29:45.663 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:45.663 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:45.663 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:45.663 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:29:45.663 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.059899+0000", 2024-09-14T06:29:45.663 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:45.663 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:45.663 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:45.663 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:29:45.664 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062501+0000", 2024-09-14T06:29:45.664 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.0026016419999999999 2024-09-14T06:29:45.664 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:45.664 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:45.664 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:29:45.664 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062514+0000", 2024-09-14T06:29:45.664 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.2947e-05 2024-09-14T06:29:45.664 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:45.664 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:45.664 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:29:45.664 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062523+0000", 2024-09-14T06:29:45.664 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.5519999999999993e-06 2024-09-14T06:29:45.664 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:45.664 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:45.665 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:29:45.665 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062534+0000", 2024-09-14T06:29:45.665 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.1416e-05 2024-09-14T06:29:45.665 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:45.665 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:45.665 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:29:45.665 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.106179+0000", 2024-09-14T06:29:45.665 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.043644049999999997 2024-09-14T06:29:45.665 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:45.665 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:45.665 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:29:45.666 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.106262+0000", 2024-09-14T06:29:45.666 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.3178999999999998e-05 2024-09-14T06:29:45.666 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:45.666 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:45.666 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:29:45.666 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.106471+0000", 2024-09-14T06:29:45.666 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00020933599999999999 2024-09-14T06:29:45.666 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:45.666 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:29:45.666 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:45.666 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:45.666 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:45.666 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:19351 3.3 3:c34e5e51:::rbd_data.112d92027590.0000000000000024:f [read 1159168~1024 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:29:45.666 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:27.062766+0000", 2024-09-14T06:29:45.667 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 18.591628362000002, 2024-09-14T06:29:45.667 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046099454999999998, 2024-09-14T06:29:45.667 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:29:45.667 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:29:45.667 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:29:45.667 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:29:45.667 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:29:45.667 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:45.667 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 19351 2024-09-14T06:29:45.667 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:45.667 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:29:45.667 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:45.667 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:29:45.668 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062766+0000", 2024-09-14T06:29:45.668 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:45.668 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:45.668 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:45.668 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:29:45.668 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062766+0000", 2024-09-14T06:29:45.668 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:45.668 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:45.668 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:45.669 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:29:45.669 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062766+0000", 2024-09-14T06:29:45.669 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.6e-07 2024-09-14T06:29:45.669 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:45.669 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:45.669 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:29:45.669 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062767+0000", 2024-09-14T06:29:45.669 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.0070000000000001e-06 2024-09-14T06:29:45.669 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:45.669 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:45.669 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:29:45.669 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062785+0000", 2024-09-14T06:29:45.669 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.7884999999999999e-05 2024-09-14T06:29:45.670 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:45.670 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:45.670 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:29:45.670 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062790+0000", 2024-09-14T06:29:45.670 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.0900000000000004e-06 2024-09-14T06:29:45.670 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:45.670 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:45.670 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:29:45.670 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.108735+0000", 2024-09-14T06:29:45.670 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.045944411999999997 2024-09-14T06:29:45.670 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:45.670 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:45.670 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:29:45.670 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.108809+0000", 2024-09-14T06:29:45.671 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.4157000000000002e-05 2024-09-14T06:29:45.671 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:45.671 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:45.671 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:29:45.671 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.108866+0000", 2024-09-14T06:29:45.671 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.6644000000000002e-05 2024-09-14T06:29:45.671 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:45.671 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:29:45.671 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:45.671 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:45.671 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:45.671 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:19352 3.3 3:c34e5e51:::rbd_data.112d92027590.0000000000000024:f [read 1169408~1024 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:29:45.671 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:27.062798+0000", 2024-09-14T06:29:45.671 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 18.591596951, 2024-09-14T06:29:45.672 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046255900000000003, 2024-09-14T06:29:45.672 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:29:45.672 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:29:45.672 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:29:45.672 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:29:45.672 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:29:45.672 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:45.672 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 19352 2024-09-14T06:29:45.672 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:45.672 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:29:45.672 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:45.672 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:29:45.672 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062798+0000", 2024-09-14T06:29:45.672 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:45.672 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:45.673 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:45.673 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:29:45.673 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062798+0000", 2024-09-14T06:29:45.673 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:45.673 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:45.673 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:45.673 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:29:45.673 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062798+0000", 2024-09-14T06:29:45.673 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.37e-07 2024-09-14T06:29:45.673 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:45.674 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:45.674 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:29:45.674 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062800+0000", 2024-09-14T06:29:45.674 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.0269999999999998e-06 2024-09-14T06:29:45.674 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:45.674 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:45.674 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:29:45.674 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062806+0000", 2024-09-14T06:29:45.674 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.6690000000000003e-06 2024-09-14T06:29:45.674 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:45.674 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:45.674 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:29:45.674 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062810+0000", 2024-09-14T06:29:45.674 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.3449999999999999e-06 2024-09-14T06:29:45.675 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:45.675 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:45.675 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:29:45.675 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.108892+0000", 2024-09-14T06:29:45.675 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046081715000000002 2024-09-14T06:29:45.675 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:45.675 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:45.675 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:29:45.675 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.108940+0000", 2024-09-14T06:29:45.675 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.7815999999999998e-05 2024-09-14T06:29:45.675 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:45.675 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:45.675 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:29:45.675 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.109053+0000", 2024-09-14T06:29:45.676 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.000113791 2024-09-14T06:29:45.676 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:45.676 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:29:45.676 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:45.676 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:45.676 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:45.676 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:19353 3.3 3:c34e5e51:::rbd_data.112d92027590.0000000000000024:f [read 1179648~1024 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:29:45.676 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:27.062816+0000", 2024-09-14T06:29:45.676 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 18.591578431999999, 2024-09-14T06:29:45.676 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046481256999999998, 2024-09-14T06:29:45.676 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:29:45.676 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:29:45.676 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:29:45.676 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:29:45.676 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:29:45.677 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:45.677 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 19353 2024-09-14T06:29:45.677 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:45.677 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:29:45.677 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:45.677 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:29:45.677 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062816+0000", 2024-09-14T06:29:45.677 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:45.677 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:45.677 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:45.677 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:29:45.677 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062816+0000", 2024-09-14T06:29:45.677 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:45.677 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:45.678 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:45.678 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:29:45.678 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062816+0000", 2024-09-14T06:29:45.678 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.2899999999999999e-07 2024-09-14T06:29:45.678 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:45.678 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:45.678 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:29:45.678 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062818+0000", 2024-09-14T06:29:45.678 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.699e-06 2024-09-14T06:29:45.678 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:45.679 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:45.679 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:29:45.679 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062825+0000", 2024-09-14T06:29:45.679 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.878e-06 2024-09-14T06:29:45.679 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:45.679 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:45.679 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:29:45.679 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062830+0000", 2024-09-14T06:29:45.679 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.5410000000000002e-06 2024-09-14T06:29:45.679 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:45.679 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:45.679 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:29:45.679 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.109096+0000", 2024-09-14T06:29:45.679 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046266517 2024-09-14T06:29:45.680 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:45.680 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:45.680 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:29:45.680 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.109190+0000", 2024-09-14T06:29:45.680 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.4000999999999997e-05 2024-09-14T06:29:45.680 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:45.680 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:45.680 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:29:45.680 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.109297+0000", 2024-09-14T06:29:45.680 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.000107292 2024-09-14T06:29:45.680 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:45.680 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:29:45.680 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:45.681 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:45.681 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:45.681 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:19354 3.3 3:c34e5e51:::rbd_data.112d92027590.0000000000000024:f [read 1189888~1024 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:29:45.681 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:27.062842+0000", 2024-09-14T06:29:45.681 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 18.591552976999999, 2024-09-14T06:29:45.681 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046594257, 2024-09-14T06:29:45.681 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:29:45.681 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:29:45.681 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:29:45.681 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:29:45.681 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:29:45.682 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:45.682 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 19354 2024-09-14T06:29:45.682 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:45.682 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:29:45.682 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:45.682 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:29:45.682 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062842+0000", 2024-09-14T06:29:45.682 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:45.682 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:45.682 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:45.682 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:29:45.682 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062842+0000", 2024-09-14T06:29:45.682 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:45.682 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:45.683 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:45.683 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:29:45.683 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062842+0000", 2024-09-14T06:29:45.683 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.7399999999999999e-07 2024-09-14T06:29:45.683 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:45.683 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:45.683 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:29:45.683 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062859+0000", 2024-09-14T06:29:45.683 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.7010000000000001e-05 2024-09-14T06:29:45.683 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:45.683 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:45.683 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:29:45.683 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062866+0000", 2024-09-14T06:29:45.683 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.9990000000000002e-06 2024-09-14T06:29:45.684 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:45.684 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:45.684 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:29:45.684 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062873+0000", 2024-09-14T06:29:45.684 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.8569999999999997e-06 2024-09-14T06:29:45.684 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:45.684 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:45.684 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:29:45.684 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.109335+0000", 2024-09-14T06:29:45.684 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046461943999999998 2024-09-14T06:29:45.684 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:45.684 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:45.684 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:29:45.685 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.109377+0000", 2024-09-14T06:29:45.685 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.2558999999999997e-05 2024-09-14T06:29:45.685 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:45.685 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:45.685 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:29:45.685 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.109436+0000", 2024-09-14T06:29:45.685 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.8514000000000003e-05 2024-09-14T06:29:45.685 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:45.685 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:29:45.685 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:45.685 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:45.685 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:45.685 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:19355 3.3 3:c34e5e51:::rbd_data.112d92027590.0000000000000024:f [read 1200128~1024 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:29:45.686 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:27.062884+0000", 2024-09-14T06:29:45.686 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 18.591510304, 2024-09-14T06:29:45.686 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046681650999999998, 2024-09-14T06:29:45.686 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:29:45.686 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:29:45.686 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:29:45.686 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:29:45.686 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:29:45.686 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:45.686 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 19355 2024-09-14T06:29:45.686 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:45.686 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:29:45.686 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:45.687 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:29:45.687 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062884+0000", 2024-09-14T06:29:45.687 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:45.687 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:45.687 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:45.687 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:29:45.687 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062884+0000", 2024-09-14T06:29:45.687 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:45.687 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:45.687 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:45.687 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:29:45.687 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062885+0000", 2024-09-14T06:29:45.687 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.8000000000000001e-07 2024-09-14T06:29:45.687 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:45.688 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:45.688 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:29:45.688 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062886+0000", 2024-09-14T06:29:45.688 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.8920000000000001e-06 2024-09-14T06:29:45.688 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:45.688 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:45.688 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:29:45.688 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062892+0000", 2024-09-14T06:29:45.688 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.84e-06 2024-09-14T06:29:45.688 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:45.688 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:45.688 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:29:45.688 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062899+0000", 2024-09-14T06:29:45.689 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.5339999999999996e-06 2024-09-14T06:29:45.689 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:45.689 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:45.689 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:29:45.689 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.109466+0000", 2024-09-14T06:29:45.689 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046566919999999998 2024-09-14T06:29:45.689 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:45.689 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:45.689 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:29:45.689 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.109510+0000", 2024-09-14T06:29:45.689 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.4153000000000002e-05 2024-09-14T06:29:45.689 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:45.689 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:45.689 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:29:45.690 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.109566+0000", 2024-09-14T06:29:45.690 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.5931999999999999e-05 2024-09-14T06:29:45.690 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:45.690 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:29:45.690 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:45.690 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:45.690 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:45.690 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:19356 3.3 3:c34e5e51:::rbd_data.112d92027590.0000000000000024:f [read 1210368~1024 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:29:45.690 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:27.062908+0000", 2024-09-14T06:29:45.690 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 18.591486867, 2024-09-14T06:29:45.690 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046807498000000003, 2024-09-14T06:29:45.690 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:29:45.690 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:29:45.690 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:29:45.691 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:29:45.691 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:29:45.691 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:45.691 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 19356 2024-09-14T06:29:45.691 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:45.691 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:29:45.691 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:45.691 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:29:45.691 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062908+0000", 2024-09-14T06:29:45.691 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:45.691 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:45.691 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:45.691 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:29:45.691 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062908+0000", 2024-09-14T06:29:45.692 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:45.692 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:45.692 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:45.692 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:29:45.692 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062908+0000", 2024-09-14T06:29:45.692 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.5699999999999998e-07 2024-09-14T06:29:45.692 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:45.692 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:45.692 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:29:45.692 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062910+0000", 2024-09-14T06:29:45.692 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.7710000000000001e-06 2024-09-14T06:29:45.692 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:45.692 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:45.693 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:29:45.693 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062920+0000", 2024-09-14T06:29:45.693 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.0654e-05 2024-09-14T06:29:45.693 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:45.693 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:45.693 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:29:45.693 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062931+0000", 2024-09-14T06:29:45.693 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.075e-05 2024-09-14T06:29:45.693 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:45.693 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:45.694 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:29:45.694 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.109595+0000", 2024-09-14T06:29:45.694 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046663713000000002 2024-09-14T06:29:45.694 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:45.694 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:45.694 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:29:45.694 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.109632+0000", 2024-09-14T06:29:45.694 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.7505000000000001e-05 2024-09-14T06:29:45.694 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:45.695 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:45.695 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:29:45.695 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.109715+0000", 2024-09-14T06:29:45.695 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.2748000000000001e-05 2024-09-14T06:29:45.695 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:45.695 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:29:45.695 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:45.695 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:45.695 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:45.695 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:19357 3.3 3:c34e5e51:::rbd_data.112d92027590.0000000000000024:f [read 1220608~1024 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:29:45.695 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:27.062939+0000", 2024-09-14T06:29:45.695 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 18.591455418999999, 2024-09-14T06:29:45.695 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046920508, 2024-09-14T06:29:45.695 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:29:45.696 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:29:45.696 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:29:45.696 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:29:45.696 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:29:45.696 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:45.696 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 19357 2024-09-14T06:29:45.696 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:45.696 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:29:45.696 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:45.696 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:29:45.696 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062939+0000", 2024-09-14T06:29:45.696 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:45.696 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:45.696 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:45.697 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:29:45.697 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062939+0000", 2024-09-14T06:29:45.697 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:45.697 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:45.697 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:45.697 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:29:45.697 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062940+0000", 2024-09-14T06:29:45.697 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.32e-07 2024-09-14T06:29:45.697 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:45.697 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:45.697 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:29:45.697 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062942+0000", 2024-09-14T06:29:45.697 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.1919999999999999e-06 2024-09-14T06:29:45.697 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:45.698 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:45.698 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:29:45.698 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062948+0000", 2024-09-14T06:29:45.698 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.1149999999999999e-06 2024-09-14T06:29:45.698 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:45.698 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:45.698 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:29:45.698 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062960+0000", 2024-09-14T06:29:45.698 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.1895e-05 2024-09-14T06:29:45.698 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:45.698 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:45.698 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:29:45.698 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.109704+0000", 2024-09-14T06:29:45.698 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046744584999999998 2024-09-14T06:29:45.699 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:45.699 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:45.699 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:29:45.699 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.109777+0000", 2024-09-14T06:29:45.699 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.2282999999999999e-05 2024-09-14T06:29:45.699 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:45.699 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:45.699 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:29:45.699 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.109860+0000", 2024-09-14T06:29:45.699 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.3005999999999998e-05 2024-09-14T06:29:45.699 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:45.699 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:29:45.699 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:45.699 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:45.699 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:45.700 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:19358 3.3 3:c34e5e51:::rbd_data.112d92027590.0000000000000024:f [read 1230848~1024 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:29:45.700 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:27.062967+0000", 2024-09-14T06:29:45.700 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 18.591427789000001, 2024-09-14T06:29:45.700 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047144471, 2024-09-14T06:29:45.700 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:29:45.700 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:29:45.700 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:29:45.700 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:29:45.700 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:29:45.700 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:45.700 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 19358 2024-09-14T06:29:45.700 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:45.700 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:29:45.701 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:45.701 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:29:45.701 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062967+0000", 2024-09-14T06:29:45.701 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:45.701 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:45.701 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:45.701 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:29:45.701 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062967+0000", 2024-09-14T06:29:45.701 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:45.701 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:45.701 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:45.701 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:29:45.701 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062967+0000", 2024-09-14T06:29:45.702 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.89e-07 2024-09-14T06:29:45.702 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:45.702 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:45.702 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:29:45.702 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062969+0000", 2024-09-14T06:29:45.702 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.9929999999999998e-06 2024-09-14T06:29:45.702 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:45.702 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:45.702 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:29:45.702 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062975+0000", 2024-09-14T06:29:45.702 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.9030000000000001e-06 2024-09-14T06:29:45.702 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:45.702 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:45.703 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:29:45.703 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062979+0000", 2024-09-14T06:29:45.703 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.2250000000000002e-06 2024-09-14T06:29:45.703 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:45.703 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:45.703 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:29:45.703 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.109934+0000", 2024-09-14T06:29:45.703 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046954410000000002 2024-09-14T06:29:45.703 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:45.703 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:45.703 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:29:45.703 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.110007+0000", 2024-09-14T06:29:45.703 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.3829000000000001e-05 2024-09-14T06:29:45.703 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:45.704 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:45.704 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:29:45.704 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.110111+0000", 2024-09-14T06:29:45.704 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00010372199999999999 2024-09-14T06:29:45.704 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:45.704 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:29:45.704 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:45.704 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:45.704 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:45.704 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:19359 3.3 3:c34e5e51:::rbd_data.112d92027590.0000000000000024:f [read 1241088~1024 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:29:45.704 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:27.062987+0000", 2024-09-14T06:29:45.704 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 18.591407699000001, 2024-09-14T06:29:45.704 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047297182, 2024-09-14T06:29:45.705 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:29:45.705 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:29:45.705 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:29:45.705 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:29:45.705 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:29:45.705 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:45.705 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 19359 2024-09-14T06:29:45.705 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:45.705 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:29:45.705 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:45.705 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:29:45.705 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062987+0000", 2024-09-14T06:29:45.705 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:45.706 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:45.706 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:45.706 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:29:45.706 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062987+0000", 2024-09-14T06:29:45.706 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:45.706 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:45.706 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:45.706 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:29:45.706 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062987+0000", 2024-09-14T06:29:45.706 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.2500000000000001e-07 2024-09-14T06:29:45.706 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:45.706 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:45.706 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:29:45.706 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062989+0000", 2024-09-14T06:29:45.706 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.6080000000000001e-06 2024-09-14T06:29:45.707 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:45.707 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:45.707 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:29:45.707 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062994+0000", 2024-09-14T06:29:45.707 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.5520000000000004e-06 2024-09-14T06:29:45.707 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:45.707 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:45.707 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:29:45.707 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062999+0000", 2024-09-14T06:29:45.707 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.3930000000000001e-06 2024-09-14T06:29:45.707 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:45.707 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:45.707 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:29:45.708 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.110142+0000", 2024-09-14T06:29:45.708 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047143154 2024-09-14T06:29:45.708 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:45.708 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:45.708 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:29:45.708 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.110219+0000", 2024-09-14T06:29:45.708 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.6737999999999996e-05 2024-09-14T06:29:45.708 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:45.708 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:45.708 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:29:45.708 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.110284+0000", 2024-09-14T06:29:45.708 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.5412000000000003e-05 2024-09-14T06:29:45.708 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:45.708 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:29:45.709 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:45.709 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:45.709 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:45.709 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:19360 3.3 3:c34e5e51:::rbd_data.112d92027590.0000000000000024:f [read 1251328~1024 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:29:45.709 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:27.063017+0000", 2024-09-14T06:29:45.709 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 18.591377238, 2024-09-14T06:29:45.709 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047453823999999999, 2024-09-14T06:29:45.709 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:29:45.709 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:29:45.709 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:29:45.709 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:29:45.709 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:29:45.709 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:45.710 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 19360 2024-09-14T06:29:45.710 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:45.710 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:29:45.710 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:45.710 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:29:45.710 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.063017+0000", 2024-09-14T06:29:45.710 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:45.710 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:45.710 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:45.710 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:29:45.710 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.063017+0000", 2024-09-14T06:29:45.710 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:45.710 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:45.710 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:45.711 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:29:45.711 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.064977+0000", 2024-09-14T06:29:45.711 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.0019597080000000001 2024-09-14T06:29:45.711 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:45.711 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:45.711 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:29:45.711 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.064987+0000", 2024-09-14T06:29:45.711 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.9620000000000005e-06 2024-09-14T06:29:45.711 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:45.711 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:45.711 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:29:45.711 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065000+0000", 2024-09-14T06:29:45.711 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.3173000000000001e-05 2024-09-14T06:29:45.711 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:45.712 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:45.712 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:29:45.712 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065005+0000", 2024-09-14T06:29:45.712 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.9960000000000001e-06 2024-09-14T06:29:45.712 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:45.712 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:45.712 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:29:45.712 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.110314+0000", 2024-09-14T06:29:45.712 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.04530899 2024-09-14T06:29:45.712 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:45.712 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:45.712 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:29:45.712 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.110398+0000", 2024-09-14T06:29:45.712 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.3539000000000001e-05 2024-09-14T06:29:45.713 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:45.713 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:45.713 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:29:45.713 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.110471+0000", 2024-09-14T06:29:45.713 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.3455999999999996e-05 2024-09-14T06:29:45.713 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:45.713 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:29:45.713 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:45.713 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:45.713 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:45.713 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:19365 3.3 3:c34e5e51:::rbd_data.112d92027590.0000000000000024:f [read 1302528~1024 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:29:45.713 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:27.065058+0000", 2024-09-14T06:29:45.714 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 18.589336568, 2024-09-14T06:29:45.714 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046105526000000001, 2024-09-14T06:29:45.714 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:29:45.714 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:29:45.714 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:29:45.714 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:29:45.714 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:29:45.714 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:45.714 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 19365 2024-09-14T06:29:45.714 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:45.714 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:29:45.714 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:45.714 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:29:45.714 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065058+0000", 2024-09-14T06:29:45.715 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:45.715 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:45.715 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:45.715 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:29:45.715 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065058+0000", 2024-09-14T06:29:45.715 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:45.715 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:45.715 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:45.715 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:29:45.715 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065058+0000", 2024-09-14T06:29:45.715 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.1899999999999999e-07 2024-09-14T06:29:45.715 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:45.715 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:45.715 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:29:45.716 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065059+0000", 2024-09-14T06:29:45.716 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.1299999999999998e-07 2024-09-14T06:29:45.716 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:45.716 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:45.716 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:29:45.716 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065063+0000", 2024-09-14T06:29:45.716 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.1099999999999996e-06 2024-09-14T06:29:45.716 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:45.716 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:45.716 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:29:45.716 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065070+0000", 2024-09-14T06:29:45.716 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.973e-06 2024-09-14T06:29:45.716 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:45.717 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:45.717 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:29:45.717 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.111014+0000", 2024-09-14T06:29:45.717 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.045943373000000003 2024-09-14T06:29:45.717 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:45.717 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:45.717 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:29:45.717 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.111073+0000", 2024-09-14T06:29:45.717 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.8987999999999998e-05 2024-09-14T06:29:45.717 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:45.717 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:45.717 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:29:45.717 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.111163+0000", 2024-09-14T06:29:45.717 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.0950000000000002e-05 2024-09-14T06:29:45.718 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:45.718 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:29:45.718 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:45.718 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:45.718 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:45.718 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:19366 3.3 3:c34e5e51:::rbd_data.112d92027590.0000000000000024:f [read 1312768~1024 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:29:45.718 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:27.065074+0000", 2024-09-14T06:29:45.718 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 18.589320718, 2024-09-14T06:29:45.718 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046289858000000003, 2024-09-14T06:29:45.718 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:29:45.718 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:29:45.718 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:29:45.718 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:29:45.718 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:29:45.719 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:45.719 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 19366 2024-09-14T06:29:45.719 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:45.719 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:29:45.719 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:45.719 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:29:45.719 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065074+0000", 2024-09-14T06:29:45.719 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:45.719 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:45.719 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:45.719 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:29:45.719 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065074+0000", 2024-09-14T06:29:45.719 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:45.720 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:45.720 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:45.720 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:29:45.720 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065074+0000", 2024-09-14T06:29:45.720 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.3900000000000001e-07 2024-09-14T06:29:45.720 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:45.720 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:45.720 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:29:45.720 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065078+0000", 2024-09-14T06:29:45.720 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.3229999999999999e-06 2024-09-14T06:29:45.720 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:45.720 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:45.720 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:29:45.720 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065082+0000", 2024-09-14T06:29:45.721 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.8299999999999998e-06 2024-09-14T06:29:45.721 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:45.721 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:45.721 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:29:45.721 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065085+0000", 2024-09-14T06:29:45.721 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.8660000000000002e-06 2024-09-14T06:29:45.721 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:45.721 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:45.721 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:29:45.721 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.111270+0000", 2024-09-14T06:29:45.721 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046185035999999999 2024-09-14T06:29:45.721 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:45.721 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:45.722 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:29:45.722 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.111311+0000", 2024-09-14T06:29:45.722 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.0935e-05 2024-09-14T06:29:45.722 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:45.722 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:45.722 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:29:45.722 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.111364+0000", 2024-09-14T06:29:45.722 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.2629e-05 2024-09-14T06:29:45.722 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:45.722 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:29:45.722 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:45.722 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:45.722 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:45.722 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:19367 3.3 3:c34e5e51:::rbd_data.112d92027590.0000000000000024:f [read 1323008~1024 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:29:45.723 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:27.065115+0000", 2024-09-14T06:29:45.723 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 18.589279157, 2024-09-14T06:29:45.723 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046407003000000002, 2024-09-14T06:29:45.723 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:29:45.723 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:29:45.723 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:29:45.723 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:29:45.723 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:29:45.723 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:45.723 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 19367 2024-09-14T06:29:45.723 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:45.723 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:29:45.723 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:45.724 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:29:45.724 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065115+0000", 2024-09-14T06:29:45.724 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:45.724 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:45.724 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:45.724 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:29:45.724 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065115+0000", 2024-09-14T06:29:45.724 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:45.724 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:45.724 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:45.724 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:29:45.724 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065116+0000", 2024-09-14T06:29:45.724 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.4999999999999998e-07 2024-09-14T06:29:45.724 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:45.725 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:45.725 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:29:45.725 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065117+0000", 2024-09-14T06:29:45.725 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.189e-06 2024-09-14T06:29:45.725 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:45.725 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:45.725 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:29:45.725 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065122+0000", 2024-09-14T06:29:45.725 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.3709999999999999e-06 2024-09-14T06:29:45.725 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:45.725 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:45.725 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:29:45.725 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065125+0000", 2024-09-14T06:29:45.725 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.8940000000000002e-06 2024-09-14T06:29:45.726 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:45.726 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:45.726 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:29:45.726 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.111402+0000", 2024-09-14T06:29:45.726 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046276645999999998 2024-09-14T06:29:45.726 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:45.726 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:45.726 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:29:45.726 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.111445+0000", 2024-09-14T06:29:45.726 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.3288000000000003e-05 2024-09-14T06:29:45.726 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:45.726 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:45.726 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:29:45.726 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.111522+0000", 2024-09-14T06:29:45.727 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.7164999999999997e-05 2024-09-14T06:29:45.727 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:45.727 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:29:45.727 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:45.727 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:45.727 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:45.727 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:19368 3.3 3:c34e5e51:::rbd_data.112d92027590.0000000000000024:f [read 1333248~1024 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:29:45.727 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:27.065130+0000", 2024-09-14T06:29:45.727 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 18.589264867000001, 2024-09-14T06:29:45.727 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046507275000000001, 2024-09-14T06:29:45.727 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:29:45.727 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:29:45.727 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:29:45.728 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:29:45.728 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:29:45.728 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:45.728 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 19368 2024-09-14T06:29:45.728 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:45.728 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:29:45.728 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:45.728 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:29:45.728 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065130+0000", 2024-09-14T06:29:45.728 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:45.728 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:45.728 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:45.728 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:29:45.728 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065130+0000", 2024-09-14T06:29:45.729 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:45.729 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:45.729 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:45.729 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:29:45.729 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065130+0000", 2024-09-14T06:29:45.729 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.0800000000000001e-07 2024-09-14T06:29:45.729 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:45.729 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:45.729 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:29:45.729 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065131+0000", 2024-09-14T06:29:45.729 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.8700000000000004e-07 2024-09-14T06:29:45.729 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:45.729 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:45.730 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:29:45.730 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065134+0000", 2024-09-14T06:29:45.730 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.1039999999999998e-06 2024-09-14T06:29:45.730 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:45.730 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:45.730 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:29:45.730 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065136+0000", 2024-09-14T06:29:45.730 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.3949999999999999e-06 2024-09-14T06:29:45.730 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:45.730 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:45.730 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:29:45.730 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.111541+0000", 2024-09-14T06:29:45.730 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046404206000000003 2024-09-14T06:29:45.731 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:45.731 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:45.731 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:29:45.731 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.111582+0000", 2024-09-14T06:29:45.731 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.1051999999999999e-05 2024-09-14T06:29:45.731 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:45.731 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:45.731 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:29:45.731 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.111637+0000", 2024-09-14T06:29:45.731 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.5322999999999999e-05 2024-09-14T06:29:45.731 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:45.731 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:29:45.731 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:45.731 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:45.732 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:45.732 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:19369 3.3 3:c34e5e51:::rbd_data.112d92027590.0000000000000024:f [read 1343488~1024 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:29:45.732 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:27.065139+0000", 2024-09-14T06:29:45.732 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 18.589255049999998, 2024-09-14T06:29:45.732 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046682169000000003, 2024-09-14T06:29:45.732 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:29:45.732 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:29:45.732 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:29:45.732 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:29:45.732 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:29:45.732 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:45.732 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 19369 2024-09-14T06:29:45.732 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:45.733 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:29:45.733 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:45.733 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:29:45.733 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065139+0000", 2024-09-14T06:29:45.733 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:45.733 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:45.733 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:45.733 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:29:45.733 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065139+0000", 2024-09-14T06:29:45.733 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:45.733 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:45.733 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:45.733 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:29:45.733 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065140+0000", 2024-09-14T06:29:45.734 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.8299999999999998e-07 2024-09-14T06:29:45.734 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:45.734 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:45.734 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:29:45.734 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065141+0000", 2024-09-14T06:29:45.734 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.3900000000000003e-07 2024-09-14T06:29:45.734 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:45.734 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:45.734 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:29:45.734 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065144+0000", 2024-09-14T06:29:45.734 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.7409999999999998e-06 2024-09-14T06:29:45.734 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:45.734 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:45.734 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:29:45.735 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065147+0000", 2024-09-14T06:29:45.735 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.4150000000000002e-06 2024-09-14T06:29:45.735 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:45.735 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:45.735 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:29:45.735 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.111673+0000", 2024-09-14T06:29:45.735 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046526004000000003 2024-09-14T06:29:45.735 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:45.735 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:45.735 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:29:45.735 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.111749+0000", 2024-09-14T06:29:45.735 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.5741999999999994e-05 2024-09-14T06:29:45.735 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:45.736 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:45.736 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:29:45.736 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.111822+0000", 2024-09-14T06:29:45.736 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.3045000000000003e-05 2024-09-14T06:29:45.736 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:45.736 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:29:45.736 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:45.736 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:45.736 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:45.736 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:19370 3.3 3:c34e5e51:::rbd_data.112d92027590.0000000000000024:f [read 1353728~1024 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:29:45.736 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:27.065150+0000", 2024-09-14T06:29:45.736 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 18.589244320999999, 2024-09-14T06:29:45.736 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046897556999999999, 2024-09-14T06:29:45.736 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:29:45.737 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:29:45.737 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:29:45.737 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:29:45.737 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:29:45.737 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:45.737 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 19370 2024-09-14T06:29:45.737 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:45.737 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:29:45.737 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:45.737 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:29:45.737 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065150+0000", 2024-09-14T06:29:45.737 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:45.737 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:45.737 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:45.738 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:29:45.738 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065150+0000", 2024-09-14T06:29:45.738 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:45.738 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:45.738 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:45.738 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:29:45.738 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065150+0000", 2024-09-14T06:29:45.738 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.0200000000000001e-07 2024-09-14T06:29:45.738 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:45.738 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:45.738 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:29:45.738 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065151+0000", 2024-09-14T06:29:45.738 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.1100000000000004e-07 2024-09-14T06:29:45.738 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:45.739 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:45.739 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:29:45.739 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065154+0000", 2024-09-14T06:29:45.739 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.9639999999999999e-06 2024-09-14T06:29:45.739 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:45.739 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:45.739 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:29:45.739 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065157+0000", 2024-09-14T06:29:45.739 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.4260000000000002e-06 2024-09-14T06:29:45.739 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:45.739 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:45.739 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:29:45.739 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.111861+0000", 2024-09-14T06:29:45.739 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046704268 2024-09-14T06:29:45.740 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:45.740 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:45.740 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:29:45.740 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.111939+0000", 2024-09-14T06:29:45.740 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.8231000000000004e-05 2024-09-14T06:29:45.740 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:45.740 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:45.740 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:29:45.740 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.112048+0000", 2024-09-14T06:29:45.740 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00010855500000000001 2024-09-14T06:29:45.740 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:45.740 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:29:45.740 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:45.741 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:45.741 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:45.741 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:19371 3.3 3:c34e5e51:::rbd_data.112d92027590.0000000000000024:f [read 1363968~1024 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:29:45.741 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:27.065161+0000", 2024-09-14T06:29:45.741 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 18.589233670999999, 2024-09-14T06:29:45.741 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047021944000000003, 2024-09-14T06:29:45.741 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:29:45.741 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:29:45.741 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:29:45.741 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:29:45.741 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:29:45.741 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:45.741 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 19371 2024-09-14T06:29:45.741 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:45.742 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:29:45.742 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:45.742 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:29:45.742 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065161+0000", 2024-09-14T06:29:45.742 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:45.742 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:45.742 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:45.742 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:29:45.742 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065161+0000", 2024-09-14T06:29:45.742 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:45.742 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:45.742 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:45.742 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:29:45.742 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065161+0000", 2024-09-14T06:29:45.743 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.0200000000000001e-07 2024-09-14T06:29:45.743 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:45.743 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:45.743 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:29:45.743 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065166+0000", 2024-09-14T06:29:45.743 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.9869999999999999e-06 2024-09-14T06:29:45.744 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:45.744 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:45.744 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:29:45.744 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065169+0000", 2024-09-14T06:29:45.744 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.3799999999999998e-06 2024-09-14T06:29:45.744 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:45.744 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:45.744 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:29:45.744 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065172+0000", 2024-09-14T06:29:45.744 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.4830000000000002e-06 2024-09-14T06:29:45.744 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:45.744 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:45.744 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:29:45.745 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.112071+0000", 2024-09-14T06:29:45.745 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046899529000000002 2024-09-14T06:29:45.745 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:45.745 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:45.745 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:29:45.745 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.112128+0000", 2024-09-14T06:29:45.745 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.6341999999999997e-05 2024-09-14T06:29:45.745 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:45.745 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:45.745 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:29:45.745 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.112183+0000", 2024-09-14T06:29:45.745 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.5021000000000001e-05 2024-09-14T06:29:45.745 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:45.745 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:29:45.746 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:45.746 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:45.746 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:45.746 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:19372 3.3 3:c34e5e51:::rbd_data.112d92027590.0000000000000024:f [read 1374208~1024 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:29:45.746 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:27.065176+0000", 2024-09-14T06:29:45.746 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 18.589218520999999, 2024-09-14T06:29:45.746 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047188889999999997, 2024-09-14T06:29:45.746 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:29:45.746 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:29:45.746 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:29:45.746 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:29:45.746 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:29:45.746 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:45.747 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 19372 2024-09-14T06:29:45.747 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:45.747 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:29:45.747 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:45.747 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:29:45.747 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065176+0000", 2024-09-14T06:29:45.747 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:45.747 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:45.747 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:45.747 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:29:45.747 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065176+0000", 2024-09-14T06:29:45.747 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:45.747 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:45.747 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:45.747 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:29:45.748 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.071117+0000", 2024-09-14T06:29:45.748 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.0059404990000000001 2024-09-14T06:29:45.748 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:45.748 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:45.748 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:29:45.748 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.071120+0000", 2024-09-14T06:29:45.748 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.4340000000000001e-06 2024-09-14T06:29:45.748 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:45.748 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:45.748 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:29:45.748 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.071129+0000", 2024-09-14T06:29:45.748 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.8389999999999996e-06 2024-09-14T06:29:45.748 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:45.748 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:45.748 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:29:45.749 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.071134+0000", 2024-09-14T06:29:45.749 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.412e-06 2024-09-14T06:29:45.749 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:45.749 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:45.749 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:29:45.749 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.112221+0000", 2024-09-14T06:29:45.749 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.041086473999999998 2024-09-14T06:29:45.749 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:45.749 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:45.749 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:29:45.749 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.112299+0000", 2024-09-14T06:29:45.749 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.8746999999999997e-05 2024-09-14T06:29:45.749 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:45.749 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:45.750 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:29:45.750 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.112365+0000", 2024-09-14T06:29:45.750 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.5485000000000001e-05 2024-09-14T06:29:45.750 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:45.750 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:29:45.750 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:45.750 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:45.750 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:29:45.750 INFO:teuthology.orchestra.run.smithi138.stdout:} 2024-09-14T06:29:45.751 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-14T06:29:45.751 INFO:teuthology.orchestra.run.smithi094.stdout:moved 0 / 82 (0%) 2024-09-14T06:29:45.752 INFO:teuthology.orchestra.run.smithi094.stdout:avg 10.25 2024-09-14T06:29:45.752 INFO:teuthology.orchestra.run.smithi094.stdout:stddev 3.26917 -> 3.26917 (expected baseline 2.99479) 2024-09-14T06:29:45.752 INFO:teuthology.orchestra.run.smithi094.stdout:min osd.2 with 5 -> 5 pgs (0.487805 -> 0.487805 * mean) 2024-09-14T06:29:45.752 INFO:teuthology.orchestra.run.smithi094.stdout:max osd.7 with 14 -> 14 pgs (1.36585 -> 1.36585 * mean) 2024-09-14T06:29:45.752 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-14T06:29:45.752 INFO:teuthology.orchestra.run.smithi094.stdout:oload 1000 2024-09-14T06:29:45.752 INFO:teuthology.orchestra.run.smithi094.stdout:max_change 1 2024-09-14T06:29:45.752 INFO:teuthology.orchestra.run.smithi094.stdout:max_change_osds 4 2024-09-14T06:29:45.752 INFO:teuthology.orchestra.run.smithi094.stdout:average_utilization 0.0004 2024-09-14T06:29:45.752 INFO:teuthology.orchestra.run.smithi094.stdout:overload_utilization 0.0038 2024-09-14T06:29:45.753 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-14T06:29:45.754 INFO:tasks.ceph.osd.7.smithi138.stderr:2024-09-14T06:29:45.750+0000 7f3c6a21d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33623) UID: 0 2024-09-14T06:29:45.754 INFO:teuthology.orchestra.run.smithi094.stderr:no change 2024-09-14T06:29:45.755 INFO:teuthology.orchestra.run.smithi094.stderr:nodeep-scrub is unset 2024-09-14T06:29:45.853 INFO:tasks.ceph.osd.1.smithi094.stderr:2024-09-14T06:29:45.852+0000 7f328e47a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33641) UID: 0 2024-09-14T06:29:45.886 INFO:teuthology.orchestra.run.smithi094.stdout:checking clone #9, image image_client.0-clone9 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent10 2024-09-14T06:29:45.898 INFO:teuthology.orchestra.run.smithi138.stdout:{ 2024-09-14T06:29:45.899 INFO:teuthology.orchestra.run.smithi138.stdout: "ops": [], 2024-09-14T06:29:45.899 INFO:teuthology.orchestra.run.smithi138.stdout: "num_ops": 0 2024-09-14T06:29:45.899 INFO:teuthology.orchestra.run.smithi138.stdout:} 2024-09-14T06:29:45.907 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-14T06:29:45.923 INFO:teuthology.orchestra.run.smithi094.stdout:osd.0: {} 2024-09-14T06:29:45.923 INFO:teuthology.orchestra.run.smithi094.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-14T06:29:45.948 INFO:teuthology.orchestra.run.smithi094.stdout:osd.1: {} 2024-09-14T06:29:45.948 INFO:teuthology.orchestra.run.smithi094.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-14T06:29:45.953 INFO:tasks.ceph.osd.2.smithi094.stderr:2024-09-14T06:29:45.952+0000 7faf3269b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33649) UID: 0 2024-09-14T06:29:45.978 INFO:teuthology.orchestra.run.smithi094.stdout:osd.2: {} 2024-09-14T06:29:45.978 INFO:teuthology.orchestra.run.smithi094.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-14T06:29:45.994 INFO:teuthology.orchestra.run.smithi094.stdout:osd.3: {} 2024-09-14T06:29:45.994 INFO:teuthology.orchestra.run.smithi094.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-14T06:29:46.011 INFO:teuthology.orchestra.run.smithi094.stdout:osd.4: {} 2024-09-14T06:29:46.011 INFO:teuthology.orchestra.run.smithi094.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-14T06:29:46.014 INFO:teuthology.orchestra.run.smithi138.stdout:{ 2024-09-14T06:29:46.014 INFO:teuthology.orchestra.run.smithi138.stdout: "ops": [], 2024-09-14T06:29:46.014 INFO:teuthology.orchestra.run.smithi138.stdout: "complaint_time": 30, 2024-09-14T06:29:46.014 INFO:teuthology.orchestra.run.smithi138.stdout: "num_blocked_ops": 0 2024-09-14T06:29:46.014 INFO:teuthology.orchestra.run.smithi138.stdout:} 2024-09-14T06:29:46.023 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-14T06:29:46.038 INFO:teuthology.orchestra.run.smithi094.stdout:osd.5: {} 2024-09-14T06:29:46.038 INFO:teuthology.orchestra.run.smithi094.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-14T06:29:46.054 INFO:teuthology.orchestra.run.smithi094.stdout:osd.6: {} 2024-09-14T06:29:46.055 INFO:teuthology.orchestra.run.smithi094.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-14T06:29:46.055 INFO:tasks.ceph.osd.2.smithi094.stderr:2024-09-14T06:29:46.053+0000 7faf3269b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33649) UID: 0 2024-09-14T06:29:46.071 INFO:teuthology.orchestra.run.smithi094.stdout:osd.7: {} 2024-09-14T06:29:46.071 INFO:teuthology.orchestra.run.smithi094.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-14T06:29:46.125 INFO:teuthology.orchestra.run.smithi138.stdout:ERROR: (22) Invalid argument 2024-09-14T06:29:46.125 INFO:teuthology.orchestra.run.smithi138.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:29:46.134 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-14T06:29:46.149 INFO:teuthology.orchestra.run.smithi094.stdout:1459 writesame 0xdab6cd4 thru 0xdabf9b2 data_size 0x3(0x8cdf bytes) 2024-09-14T06:29:46.166 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-14T06:29:46.166 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-14T06:29:46.167 INFO:teuthology.orchestra.run.smithi094.stdout:1464 punch from 0x97a91be to 0x97b3b93, (0xa9d5 bytes) 2024-09-14T06:29:46.167 INFO:teuthology.orchestra.run.smithi094.stdout:1465 read 0xa618cd1 thru 0xa61a546 (0x1876 bytes) 2024-09-14T06:29:46.173 INFO:teuthology.orchestra.run.smithi094.stdout:1466 writesame 0xeb5039c thru 0xeb51095 data_size 0xb(0xcfa bytes) 2024-09-14T06:29:46.174 INFO:teuthology.orchestra.run.smithi094.stdout:1467 read 0xbcfdc06 thru 0xbcfdea2 (0x29d bytes) 2024-09-14T06:29:46.174 INFO:teuthology.orchestra.run.smithi094.stdout:1468 read 0x123e2e4 thru 0x124b135 (0xce52 bytes) 2024-09-14T06:29:46.174 INFO:teuthology.orchestra.run.smithi094.stdout:1470 punch from 0x76f3a5b to 0x76fcb82, (0x9127 bytes) 2024-09-14T06:29:46.174 INFO:teuthology.orchestra.run.smithi094.stdout:1471 write 0xb571e86 thru 0xb57293a (0xab5 bytes) 2024-09-14T06:29:46.176 INFO:teuthology.orchestra.run.smithi094.stdout:1473 read 0x8fb374e thru 0x8fbfcb1 (0xc564 bytes) 2024-09-14T06:29:46.176 INFO:teuthology.orchestra.run.smithi094.stdout:1475 write 0x35d6125 thru 0x35d76e4 (0x15c0 bytes) 2024-09-14T06:29:46.177 INFO:tasks.ceph.osd.7.smithi138.stderr:2024-09-14T06:29:46.176+0000 7f3c6a21d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33623) UID: 0 2024-09-14T06:29:46.177 INFO:teuthology.orchestra.run.smithi094.stdout:1476 read 0xc2bed49 thru 0xc2c8f9d (0xa255 bytes) 2024-09-14T06:29:46.177 INFO:teuthology.orchestra.run.smithi094.stdout:1477 write 0x6be3a0d thru 0x6bf02e2 (0xc8d6 bytes) 2024-09-14T06:29:46.185 INFO:teuthology.orchestra.run.smithi094.stdout:1478 writesame 0x39d836d thru 0x39dd9c4 data_size 0x24(0x5658 bytes) 2024-09-14T06:29:46.188 INFO:teuthology.orchestra.run.smithi094.stdout:1479 read 0x4680399 thru 0x4686dc1 (0x6a29 bytes) 2024-09-14T06:29:46.188 INFO:teuthology.orchestra.run.smithi094.stdout:1483 read 0x68b06c6 thru 0x68bf2ba (0xebf5 bytes) 2024-09-14T06:29:46.188 INFO:teuthology.orchestra.run.smithi094.stdout:1484 write 0x7238eaa thru 0x723baae (0x2c05 bytes) 2024-09-14T06:29:46.196 INFO:teuthology.orchestra.run.smithi094.stdout:1485 punch from 0xc57e617 to 0xc58a4e1, (0xbeca bytes) 2024-09-14T06:29:46.196 INFO:teuthology.orchestra.run.smithi094.stdout:1486 read 0x9ac0e99 thru 0x9ac7527 (0x668f bytes) 2024-09-14T06:29:46.197 INFO:teuthology.orchestra.run.smithi094.stdout:1487 writesame 0x73fe529 thru 0x73ffb89 data_size 0x1(0x1661 bytes) 2024-09-14T06:29:46.211 INFO:teuthology.orchestra.run.smithi094.stdout:1488 trunc from 0xeb51096 to 0x7c62343 2024-09-14T06:29:46.224 INFO:teuthology.orchestra.run.smithi094.stdout:1489 write 0x4469c99 thru 0x447033f (0x66a7 bytes) 2024-09-14T06:29:46.234 INFO:teuthology.orchestra.run.smithi094.stdout:1490 trunc from 0x7c62343 to 0x63ab095 2024-09-14T06:29:46.246 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-14T06:29:46.246 INFO:teuthology.orchestra.run.smithi094.stdout:1493 write 0x4f1572f thru 0x4f204f2 (0xadc4 bytes) 2024-09-14T06:29:46.253 INFO:teuthology.orchestra.run.smithi094.stdout:1494 write 0x108d0f6 thru 0x108d61f (0x52a bytes) 2024-09-14T06:29:46.254 INFO:teuthology.orchestra.run.smithi094.stdout:1496 read 0x5eddc98 thru 0x5ede08e (0x3f7 bytes) 2024-09-14T06:29:46.256 INFO:teuthology.orchestra.run.smithi138.stdout:ERROR: (22) Invalid argument 2024-09-14T06:29:46.256 INFO:teuthology.orchestra.run.smithi138.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:29:46.257 INFO:tasks.ceph.osd.0.smithi094.stderr:2024-09-14T06:29:46.256+0000 7f07c9333640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33636) UID: 0 2024-09-14T06:29:46.266 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-14T06:29:46.268 INFO:teuthology.orchestra.run.smithi094.stdout:1498 writesame 0xba2e242 thru 0xba3a6ff data_size 0x1(0xc4be bytes) 2024-09-14T06:29:46.275 INFO:teuthology.orchestra.run.smithi094.stdout:1501 write 0xccb1f44 thru 0xccc0486 (0xe543 bytes) 2024-09-14T06:29:46.285 INFO:teuthology.orchestra.run.smithi094.stdout:1502 write 0x55ce5b4 thru 0x55d8013 (0x9a60 bytes) 2024-09-14T06:29:46.287 INFO:teuthology.orchestra.run.smithi094.stdout:1503 write 0x78a74d0 thru 0x78b3035 (0xbb66 bytes) 2024-09-14T06:29:46.297 INFO:teuthology.orchestra.run.smithi094.stdout:1504 writesame 0x8c9c096 thru 0x8ca513e data_size 0x1(0x90a9 bytes) 2024-09-14T06:29:46.311 INFO:teuthology.orchestra.run.smithi094.stdout:1505 read 0xa9b1b1b thru 0xa9bfa9c (0xdf82 bytes) 2024-09-14T06:29:46.311 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-14T06:29:46.311 INFO:teuthology.orchestra.run.smithi094.stdout:1508 read 0x807cfc0 thru 0x808c9c6 (0xfa07 bytes) 2024-09-14T06:29:46.311 INFO:teuthology.orchestra.run.smithi094.stdout:1509 punch from 0x894d716 to 0x895651d, (0x8e07 bytes) 2024-09-14T06:29:46.311 INFO:teuthology.orchestra.run.smithi094.stdout:1510 write 0x48cd322 thru 0x48d3f2a (0x6c09 bytes) 2024-09-14T06:29:46.325 INFO:teuthology.orchestra.run.smithi094.stdout:1511 trunc from 0xccc0487 to 0x28c88fd 2024-09-14T06:29:46.356 INFO:teuthology.orchestra.run.smithi094.stdout:1514 read 0x95ce71 thru 0x9648f1 (0x7a81 bytes) 2024-09-14T06:29:46.356 INFO:teuthology.orchestra.run.smithi094.stdout:1515 read 0x1686650 thru 0x168bad9 (0x548a bytes) 2024-09-14T06:29:46.356 INFO:teuthology.orchestra.run.smithi094.stdout:1516 read 0x191aac4 thru 0x192a070 (0xf5ad bytes) 2024-09-14T06:29:46.357 INFO:teuthology.orchestra.run.smithi094.stdout:1517 read 0x1437bf thru 0x151348 (0xdb8a bytes) 2024-09-14T06:29:46.357 INFO:tasks.ceph.osd.1.smithi094.stderr:2024-09-14T06:29:46.356+0000 7f328e47a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33641) UID: 0 2024-09-14T06:29:46.358 INFO:teuthology.orchestra.run.smithi094.stdout:1518 read 0x27320e3 thru 0x273b5ce (0x94ec bytes) 2024-09-14T06:29:46.367 INFO:teuthology.orchestra.run.smithi094.stdout:1520 write 0x5be5ab2 thru 0x5be927b (0x37ca bytes) 2024-09-14T06:29:46.375 INFO:teuthology.orchestra.run.smithi094.stdout:1521 writesame 0x17d6e10 thru 0x17de50f data_size 0x80(0x7700 bytes) 2024-09-14T06:29:46.378 INFO:teuthology.orchestra.run.smithi094.stdout:1522 read 0x464900e thru 0x4650a7d (0x7a70 bytes) 2024-09-14T06:29:46.378 INFO:teuthology.orchestra.run.smithi094.stdout:1524 punch from 0x1a2cee5 to 0x1a3792a, (0xaa45 bytes) 2024-09-14T06:29:46.378 INFO:teuthology.orchestra.run.smithi094.stdout:1525 writesame 0x23843da thru 0x2386657 data_size 0x5(0x227e bytes) 2024-09-14T06:29:46.390 INFO:teuthology.orchestra.run.smithi094.stdout:1526 trunc from 0x5be927c to 0xe238b03 2024-09-14T06:29:46.395 INFO:teuthology.orchestra.run.smithi094.stdout:1527 write 0xc8eee9c thru 0xc8fdcad (0xee12 bytes) 2024-09-14T06:29:46.406 INFO:teuthology.orchestra.run.smithi094.stdout:1530 write 0x968d65c thru 0x969b164 (0xdb09 bytes) 2024-09-14T06:29:46.413 INFO:teuthology.orchestra.run.smithi094.stdout:1531 write 0xd10f389 thru 0xd111de1 (0x2a59 bytes) 2024-09-14T06:29:46.419 INFO:teuthology.orchestra.run.smithi094.stdout:1533 write 0xd914476 thru 0xd9167b3 (0x233e bytes) 2024-09-14T06:29:46.425 INFO:teuthology.orchestra.run.smithi094.stdout:1534 writesame 0x6f01239 thru 0x6f01baa data_size 0x6(0x972 bytes) 2024-09-14T06:29:46.428 INFO:teuthology.orchestra.run.smithi094.stdout:1535 writesame 0x7cf0c3c thru 0x7cfb56e data_size 0x1(0xa933 bytes) 2024-09-14T06:29:46.430 INFO:teuthology.orchestra.run.smithi138.stdout:ERROR: (22) Invalid argument 2024-09-14T06:29:46.430 INFO:teuthology.orchestra.run.smithi138.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:29:46.440 INFO:teuthology.orchestra.run.smithi094.stdout:1538 read 0xa269970 thru 0xa2755aa (0xbc3b bytes) 2024-09-14T06:29:46.441 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-14T06:29:46.441 INFO:teuthology.orchestra.run.smithi094.stdout:1543 read 0x957c508 thru 0x957ed9a (0x2893 bytes) 2024-09-14T06:29:46.441 INFO:teuthology.orchestra.run.smithi094.stdout:1544 writesame 0x638e8e3 thru 0x639c88e data_size 0x14(0xdfac bytes) 2024-09-14T06:29:46.443 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-14T06:29:46.450 INFO:teuthology.orchestra.run.smithi094.stdout:1545 write 0x25d7af2 thru 0x25e61aa (0xe6b9 bytes) 2024-09-14T06:29:46.452 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-14T06:29:46.452 INFO:teuthology.orchestra.run.smithi094.stdout:1547 clone 12 order 25 su 524288 sc 13 2024-09-14T06:29:46.486 INFO:tasks.ceph.osd.6.smithi138.stderr:2024-09-14T06:29:46.485+0000 7f8cd8849640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33620) UID: 0 2024-09-14T06:29:46.546 INFO:teuthology.orchestra.run.smithi138.stdout:ERROR: (22) Invalid argument 2024-09-14T06:29:46.546 INFO:teuthology.orchestra.run.smithi138.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:29:46.554 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-14T06:29:46.595 INFO:tasks.ceph.osd.4.smithi138.stderr:2024-09-14T06:29:46.594+0000 7f6c653c8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33615) UID: 0 2024-09-14T06:29:46.661 INFO:tasks.ceph.osd.0.smithi094.stderr:2024-09-14T06:29:46.659+0000 7f07c9333640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33636) UID: 0 2024-09-14T06:29:46.696 INFO:teuthology.orchestra.run.smithi138.stdout:ERROR: (22) Invalid argument 2024-09-14T06:29:46.696 INFO:teuthology.orchestra.run.smithi138.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:29:46.705 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-14T06:29:46.761 INFO:tasks.ceph.osd.3.smithi094.stderr:2024-09-14T06:29:46.760+0000 7feeeafe8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33655) UID: 0 2024-09-14T06:29:46.812 INFO:teuthology.orchestra.run.smithi138.stdout:ERROR: (22) Invalid argument 2024-09-14T06:29:46.812 INFO:teuthology.orchestra.run.smithi138.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:29:46.820 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-14T06:29:46.863 INFO:tasks.ceph.osd.1.smithi094.stderr:2024-09-14T06:29:46.862+0000 7f328e47a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33641) UID: 0 2024-09-14T06:29:46.922 INFO:teuthology.orchestra.run.smithi138.stdout:ERROR: (22) Invalid argument 2024-09-14T06:29:46.923 INFO:teuthology.orchestra.run.smithi138.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:29:46.931 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-14T06:29:46.975 INFO:tasks.ceph.osd.2.smithi094.stderr:2024-09-14T06:29:46.974+0000 7faf3269b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33649) UID: 0 2024-09-14T06:29:47.034 INFO:teuthology.orchestra.run.smithi094.stdout:ERROR: (22) Invalid argument 2024-09-14T06:29:47.034 INFO:teuthology.orchestra.run.smithi094.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:29:47.043 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-14T06:29:47.065 INFO:tasks.ceph.osd.7.smithi138.stderr:2024-09-14T06:29:47.065+0000 7f3c6a21d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33623) UID: 0 2024-09-14T06:29:47.166 INFO:tasks.ceph.osd.4.smithi138.stderr:2024-09-14T06:29:47.165+0000 7f6c653c8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33615) UID: 0 2024-09-14T06:29:47.185 INFO:teuthology.orchestra.run.smithi094.stdout:ERROR: (22) Invalid argument 2024-09-14T06:29:47.185 INFO:teuthology.orchestra.run.smithi094.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:29:47.193 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-14T06:29:47.267 INFO:tasks.ceph.osd.6.smithi138.stderr:2024-09-14T06:29:47.267+0000 7f8cd8849640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33620) UID: 0 2024-09-14T06:29:47.296 INFO:teuthology.orchestra.run.smithi094.stdout:ERROR: (22) Invalid argument 2024-09-14T06:29:47.296 INFO:teuthology.orchestra.run.smithi094.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:29:47.305 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-14T06:29:47.368 INFO:tasks.ceph.osd.6.smithi138.stderr:2024-09-14T06:29:47.367+0000 7f8cd8849640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33620) UID: 0 2024-09-14T06:29:47.407 INFO:teuthology.orchestra.run.smithi094.stdout:ERROR: (22) Invalid argument 2024-09-14T06:29:47.408 INFO:teuthology.orchestra.run.smithi094.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:29:47.416 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-14T06:29:47.468 INFO:tasks.ceph.osd.1.smithi094.stderr:2024-09-14T06:29:47.467+0000 7f328e47a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33641) UID: 0 2024-09-14T06:29:47.519 INFO:teuthology.orchestra.run.smithi094.stdout:ERROR: (22) Invalid argument 2024-09-14T06:29:47.519 INFO:teuthology.orchestra.run.smithi094.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:29:47.527 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-14T06:29:47.569 INFO:tasks.ceph.osd.4.smithi138.stderr:2024-09-14T06:29:47.568+0000 7f6c653c8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33615) UID: 0 2024-09-14T06:29:47.670 INFO:teuthology.orchestra.run.smithi094.stdout:ERROR: (22) Invalid argument 2024-09-14T06:29:47.670 INFO:teuthology.orchestra.run.smithi094.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:29:47.670 INFO:tasks.ceph.osd.1.smithi094.stderr:2024-09-14T06:29:47.669+0000 7f328e47a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33641) UID: 0 2024-09-14T06:29:47.678 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-14T06:29:47.756 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-14T06:29:47.807 INFO:tasks.ceph.osd.2.smithi094.stderr:2024-09-14T06:29:47.806+0000 7faf3269b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33649) UID: 0 2024-09-14T06:29:47.872 INFO:tasks.ceph.osd.1.smithi094.stderr:2024-09-14T06:29:47.871+0000 7f328e47a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33641) UID: 0 2024-09-14T06:29:47.876 INFO:teuthology.orchestra.run.smithi094.stdout:ERROR: (22) Invalid argument 2024-09-14T06:29:47.876 INFO:teuthology.orchestra.run.smithi094.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:29:47.891 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-14T06:29:47.975 INFO:tasks.ceph.osd.1.smithi094.stderr:2024-09-14T06:29:47.974+0000 7f328e47a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33641) UID: 0 2024-09-14T06:29:48.074 INFO:teuthology.orchestra.run.smithi094.stdout:ERROR: (22) Invalid argument 2024-09-14T06:29:48.074 INFO:teuthology.orchestra.run.smithi094.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:29:48.075 INFO:tasks.ceph.osd.1.smithi094.stderr:2024-09-14T06:29:48.073+0000 7f328e47a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33641) UID: 0 2024-09-14T06:29:48.083 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-14T06:29:48.083 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-14T06:29:48.175 INFO:tasks.ceph.osd.4.smithi138.stderr:2024-09-14T06:29:48.174+0000 7f6c653c8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33615) UID: 0 2024-09-14T06:29:48.229 INFO:teuthology.orchestra.run.smithi094.stdout:ERROR: (22) Invalid argument 2024-09-14T06:29:48.230 INFO:teuthology.orchestra.run.smithi094.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:29:48.239 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-14T06:29:48.246 INFO:teuthology.orchestra.run.smithi094.stdout:osd.0: {} 2024-09-14T06:29:48.246 INFO:teuthology.orchestra.run.smithi094.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-14T06:29:48.264 INFO:teuthology.orchestra.run.smithi094.stdout:osd.1: {} 2024-09-14T06:29:48.264 INFO:teuthology.orchestra.run.smithi094.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-14T06:29:48.276 INFO:tasks.ceph.osd.7.smithi138.stderr:2024-09-14T06:29:48.275+0000 7f3c6a21d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33623) UID: 0 2024-09-14T06:29:48.280 INFO:teuthology.orchestra.run.smithi094.stdout:osd.2: {} 2024-09-14T06:29:48.280 INFO:teuthology.orchestra.run.smithi094.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-14T06:29:48.296 INFO:teuthology.orchestra.run.smithi094.stdout:osd.3: {} 2024-09-14T06:29:48.296 INFO:teuthology.orchestra.run.smithi094.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-14T06:29:48.313 INFO:teuthology.orchestra.run.smithi094.stdout:osd.4: {} 2024-09-14T06:29:48.313 INFO:teuthology.orchestra.run.smithi094.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-14T06:29:48.331 INFO:teuthology.orchestra.run.smithi094.stdout:osd.5: {} 2024-09-14T06:29:48.331 INFO:teuthology.orchestra.run.smithi094.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-14T06:29:48.344 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-14T06:29:48.344 INFO:teuthology.orchestra.run.smithi094.stdout: "ops": [], 2024-09-14T06:29:48.345 INFO:teuthology.orchestra.run.smithi094.stdout: "num_ops": 0 2024-09-14T06:29:48.345 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-14T06:29:48.347 INFO:teuthology.orchestra.run.smithi094.stdout:osd.6: {} 2024-09-14T06:29:48.347 INFO:teuthology.orchestra.run.smithi094.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-14T06:29:48.354 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-14T06:29:48.364 INFO:teuthology.orchestra.run.smithi094.stdout:osd.7: {} 2024-09-14T06:29:48.365 INFO:teuthology.orchestra.run.smithi094.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-14T06:29:48.377 INFO:tasks.ceph.osd.5.smithi138.stderr:2024-09-14T06:29:48.376+0000 7fe13746f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33618) UID: 0 2024-09-14T06:29:48.467 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-14T06:29:48.468 INFO:teuthology.orchestra.run.smithi094.stdout: "ops": [], 2024-09-14T06:29:48.468 INFO:teuthology.orchestra.run.smithi094.stdout: "complaint_time": 30, 2024-09-14T06:29:48.468 INFO:teuthology.orchestra.run.smithi094.stdout: "num_blocked_ops": 0 2024-09-14T06:29:48.468 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-14T06:29:48.477 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-14T06:29:48.478 INFO:tasks.ceph.osd.4.smithi138.stderr:2024-09-14T06:29:48.477+0000 7f6c653c8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33615) UID: 0 2024-09-14T06:29:48.580 INFO:tasks.ceph.osd.6.smithi138.stderr:2024-09-14T06:29:48.578+0000 7f8cd8849640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33620) UID: 0 2024-09-14T06:29:48.582 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-14T06:29:48.582 INFO:teuthology.orchestra.run.smithi094.stdout: "size": 20, 2024-09-14T06:29:48.582 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 600, 2024-09-14T06:29:48.582 INFO:teuthology.orchestra.run.smithi094.stdout: "ops": [ 2024-09-14T06:29:48.582 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:48.582 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:44425 3.2 3:4603ad7f:::rbd_data.112dca2f6ae1.0000000000000005:17 [read 868352~4096 [fadvise_nocache]] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:29:48.582 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:43.854967+0000", 2024-09-14T06:29:48.582 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 4.7252509219999999, 2024-09-14T06:29:48.582 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.063146025999999994, 2024-09-14T06:29:48.582 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:29:48.583 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:29:48.583 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:29:48.583 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:29:48.583 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:29:48.583 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:48.583 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 44425 2024-09-14T06:29:48.583 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:48.583 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:29:48.583 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:48.583 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:29:48.583 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.854967+0000", 2024-09-14T06:29:48.583 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:48.583 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:48.583 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:48.584 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:29:48.584 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.854967+0000", 2024-09-14T06:29:48.584 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:48.584 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:48.584 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:48.584 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:29:48.584 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.854967+0000", 2024-09-14T06:29:48.584 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.16e-07 2024-09-14T06:29:48.584 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:48.584 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:48.584 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:29:48.584 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.854968+0000", 2024-09-14T06:29:48.584 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.006e-06 2024-09-14T06:29:48.584 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:48.585 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:48.585 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:29:48.585 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.854971+0000", 2024-09-14T06:29:48.585 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.9359999999999999e-06 2024-09-14T06:29:48.585 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:48.585 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:48.585 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:29:48.585 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.854974+0000", 2024-09-14T06:29:48.585 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.1840000000000002e-06 2024-09-14T06:29:48.585 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:48.585 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:48.585 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:29:48.585 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.917945+0000", 2024-09-14T06:29:48.586 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.062970925999999997 2024-09-14T06:29:48.586 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:48.586 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:48.586 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:29:48.586 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.918030+0000", 2024-09-14T06:29:48.586 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.5863000000000001e-05 2024-09-14T06:29:48.586 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:48.586 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:48.586 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:29:48.586 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.918113+0000", 2024-09-14T06:29:48.586 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.2894999999999998e-05 2024-09-14T06:29:48.586 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:48.586 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:48.586 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:48.587 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:48.587 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:48.587 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:44426 3.2 3:4603ad7f:::rbd_data.112dca2f6ae1.0000000000000005:17 [read 892928~4096 [fadvise_nocache]] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:29:48.587 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:43.854977+0000", 2024-09-14T06:29:48.587 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 4.7252415120000002, 2024-09-14T06:29:48.587 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.063369237999999994, 2024-09-14T06:29:48.587 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:29:48.587 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:29:48.587 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:29:48.587 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:29:48.587 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:29:48.587 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:48.587 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 44426 2024-09-14T06:29:48.587 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:48.588 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:29:48.588 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:48.588 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:29:48.588 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.854977+0000", 2024-09-14T06:29:48.588 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:48.588 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:48.588 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:48.588 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:29:48.588 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.854977+0000", 2024-09-14T06:29:48.588 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:48.588 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:48.588 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:48.588 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:29:48.588 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.854977+0000", 2024-09-14T06:29:48.589 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.1199999999999999e-07 2024-09-14T06:29:48.589 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:48.589 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:48.589 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:29:48.589 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.854980+0000", 2024-09-14T06:29:48.589 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.27e-06 2024-09-14T06:29:48.589 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:48.589 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:48.589 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:29:48.589 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.854983+0000", 2024-09-14T06:29:48.590 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.1650000000000002e-06 2024-09-14T06:29:48.590 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:48.590 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:48.590 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:29:48.590 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.854989+0000", 2024-09-14T06:29:48.590 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.6520000000000003e-06 2024-09-14T06:29:48.590 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:48.590 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:48.590 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:29:48.590 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.918143+0000", 2024-09-14T06:29:48.590 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.063154541999999994 2024-09-14T06:29:48.590 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:48.590 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:48.591 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:29:48.591 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.918209+0000", 2024-09-14T06:29:48.591 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.5529000000000002e-05 2024-09-14T06:29:48.591 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:48.591 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:48.591 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:29:48.591 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.918346+0000", 2024-09-14T06:29:48.591 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000136868 2024-09-14T06:29:48.591 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:48.591 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:48.591 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:48.591 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:48.591 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:48.591 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:44427 3.2 3:4603ad7f:::rbd_data.112dca2f6ae1.0000000000000005:17 [read 917504~4096 [fadvise_nocache]] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:29:48.591 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:43.854992+0000", 2024-09-14T06:29:48.592 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 4.7252258659999997, 2024-09-14T06:29:48.592 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.063607792999999996, 2024-09-14T06:29:48.592 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:29:48.592 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:29:48.592 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:29:48.592 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:29:48.592 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:29:48.592 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:48.592 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 44427 2024-09-14T06:29:48.592 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:48.592 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:29:48.592 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:48.592 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:29:48.592 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.854992+0000", 2024-09-14T06:29:48.593 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:48.593 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:48.593 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:48.593 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:29:48.593 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.854992+0000", 2024-09-14T06:29:48.593 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:48.593 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:48.593 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:48.593 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:29:48.593 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.854993+0000", 2024-09-14T06:29:48.593 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.1400000000000001e-07 2024-09-14T06:29:48.593 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:48.593 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:48.593 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:29:48.593 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.854994+0000", 2024-09-14T06:29:48.594 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.091e-06 2024-09-14T06:29:48.594 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:48.594 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:48.594 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:29:48.594 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855001+0000", 2024-09-14T06:29:48.594 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.204e-06 2024-09-14T06:29:48.594 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:48.594 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:48.594 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:29:48.594 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855003+0000", 2024-09-14T06:29:48.594 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.514e-06 2024-09-14T06:29:48.594 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:48.594 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:48.594 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:29:48.595 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.918380+0000", 2024-09-14T06:29:48.595 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.063376738000000002 2024-09-14T06:29:48.595 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:48.595 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:48.595 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:29:48.595 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.918453+0000", 2024-09-14T06:29:48.595 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.3291e-05 2024-09-14T06:29:48.595 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:48.595 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:48.595 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:29:48.595 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.918600+0000", 2024-09-14T06:29:48.595 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00014674100000000001 2024-09-14T06:29:48.595 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:48.595 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:48.595 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:48.596 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:48.596 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:48.596 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:44428 3.2 3:4603ad7f:::rbd_data.112dca2f6ae1.0000000000000005:17 [read 942080~4096 [fadvise_nocache]] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:29:48.596 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:43.855006+0000", 2024-09-14T06:29:48.596 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 4.7252118159999998, 2024-09-14T06:29:48.596 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.063846321999999997, 2024-09-14T06:29:48.596 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:29:48.596 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:29:48.596 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:29:48.596 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:29:48.596 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:29:48.596 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:48.596 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 44428 2024-09-14T06:29:48.597 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:48.597 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:29:48.597 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:48.597 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:29:48.597 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855006+0000", 2024-09-14T06:29:48.597 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:48.597 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:48.597 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:48.597 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:29:48.597 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855006+0000", 2024-09-14T06:29:48.597 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:48.597 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:48.597 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:48.598 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:29:48.598 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855007+0000", 2024-09-14T06:29:48.598 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.1199999999999999e-07 2024-09-14T06:29:48.598 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:48.598 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:48.598 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:29:48.598 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855011+0000", 2024-09-14T06:29:48.598 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.9539999999999998e-06 2024-09-14T06:29:48.598 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:48.598 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:48.598 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:29:48.598 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855014+0000", 2024-09-14T06:29:48.598 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.2930000000000001e-06 2024-09-14T06:29:48.598 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:48.598 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:48.599 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:29:48.599 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855016+0000", 2024-09-14T06:29:48.599 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.3039999999999999e-06 2024-09-14T06:29:48.599 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:48.599 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:48.599 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:29:48.599 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.918634+0000", 2024-09-14T06:29:48.599 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.063617928000000004 2024-09-14T06:29:48.599 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:48.599 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:48.599 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:29:48.599 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.918711+0000", 2024-09-14T06:29:48.599 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.6545000000000007e-05 2024-09-14T06:29:48.599 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:48.600 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:48.600 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:29:48.600 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.918853+0000", 2024-09-14T06:29:48.600 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00014208600000000001 2024-09-14T06:29:48.600 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:48.600 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:48.600 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:48.600 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:48.600 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:48.600 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:44429 3.2 3:4603ad7f:::rbd_data.112dca2f6ae1.0000000000000005:17 [read 966656~4096 [fadvise_nocache]] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:29:48.600 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:43.855019+0000", 2024-09-14T06:29:48.600 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 4.7251991430000002, 2024-09-14T06:29:48.600 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.064059226999999996, 2024-09-14T06:29:48.600 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:29:48.600 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:29:48.601 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:29:48.601 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:29:48.601 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:29:48.601 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:48.601 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 44429 2024-09-14T06:29:48.601 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:48.601 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:29:48.601 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:48.601 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:29:48.601 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855019+0000", 2024-09-14T06:29:48.601 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:48.601 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:48.601 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:48.601 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:29:48.602 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855019+0000", 2024-09-14T06:29:48.602 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:48.602 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:48.602 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:48.602 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:29:48.602 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855019+0000", 2024-09-14T06:29:48.602 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.28e-07 2024-09-14T06:29:48.602 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:48.602 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:48.602 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:29:48.602 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855020+0000", 2024-09-14T06:29:48.602 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.0330000000000001e-06 2024-09-14T06:29:48.602 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:48.602 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:48.602 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:29:48.603 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855023+0000", 2024-09-14T06:29:48.603 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.79e-06 2024-09-14T06:29:48.603 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:48.603 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:48.603 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:29:48.603 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855025+0000", 2024-09-14T06:29:48.603 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.216e-06 2024-09-14T06:29:48.603 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:48.603 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:48.603 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:29:48.603 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.918886+0000", 2024-09-14T06:29:48.603 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.063861037999999995 2024-09-14T06:29:48.603 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:48.603 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:48.603 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:29:48.604 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.918952+0000", 2024-09-14T06:29:48.604 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.5993000000000003e-05 2024-09-14T06:29:48.604 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:48.604 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:48.604 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:29:48.604 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.919078+0000", 2024-09-14T06:29:48.604 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00012592899999999999 2024-09-14T06:29:48.604 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:48.604 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:48.604 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:48.604 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:48.604 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:48.604 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:44430 3.2 3:4603ad7f:::rbd_data.112dca2f6ae1.0000000000000005:17 [read 991232~4096 [fadvise_nocache]] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:29:48.604 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:43.855028+0000", 2024-09-14T06:29:48.605 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 4.7251901329999999, 2024-09-14T06:29:48.605 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.064284346000000006, 2024-09-14T06:29:48.605 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:29:48.605 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:29:48.605 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:29:48.605 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:29:48.605 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:29:48.605 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:48.605 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 44430 2024-09-14T06:29:48.605 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:48.605 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:29:48.605 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:48.605 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:29:48.605 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855028+0000", 2024-09-14T06:29:48.606 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:48.606 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:48.606 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:48.606 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:29:48.606 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855028+0000", 2024-09-14T06:29:48.606 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:48.606 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:48.606 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:48.606 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:29:48.606 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855028+0000", 2024-09-14T06:29:48.606 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.1199999999999999e-07 2024-09-14T06:29:48.606 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:48.606 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:48.606 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:29:48.606 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855029+0000", 2024-09-14T06:29:48.607 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.1100000000000004e-07 2024-09-14T06:29:48.607 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:48.607 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:48.607 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:29:48.607 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855032+0000", 2024-09-14T06:29:48.607 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.7889999999999999e-06 2024-09-14T06:29:48.607 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:48.607 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:48.607 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:29:48.607 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855034+0000", 2024-09-14T06:29:48.607 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.125e-06 2024-09-14T06:29:48.607 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:48.607 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:48.608 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:29:48.608 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.919110+0000", 2024-09-14T06:29:48.608 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.064076413999999998 2024-09-14T06:29:48.608 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:48.608 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:48.608 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:29:48.608 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.919186+0000", 2024-09-14T06:29:48.608 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.5784999999999994e-05 2024-09-14T06:29:48.608 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:48.608 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:48.608 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:29:48.608 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.919312+0000", 2024-09-14T06:29:48.608 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00012611 2024-09-14T06:29:48.608 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:48.608 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:48.609 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:48.609 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:48.609 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:48.609 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:44431 3.2 3:4603ad7f:::rbd_data.112dca2f6ae1.0000000000000005:17 [read 1015808~4096 [fadvise_nocache]] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:29:48.609 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:43.855037+0000", 2024-09-14T06:29:48.609 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 4.7251812649999998, 2024-09-14T06:29:48.609 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.064469755000000004, 2024-09-14T06:29:48.609 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:29:48.609 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:29:48.609 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:29:48.609 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:29:48.609 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:29:48.609 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:48.610 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 44431 2024-09-14T06:29:48.610 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:48.610 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:29:48.610 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:48.610 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:29:48.610 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855037+0000", 2024-09-14T06:29:48.610 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:48.610 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:48.610 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:48.610 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:29:48.610 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855037+0000", 2024-09-14T06:29:48.610 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:48.610 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:48.610 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:48.610 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:29:48.611 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855037+0000", 2024-09-14T06:29:48.611 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.1e-07 2024-09-14T06:29:48.611 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:48.611 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:48.611 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:29:48.611 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855038+0000", 2024-09-14T06:29:48.611 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.9400000000000004e-07 2024-09-14T06:29:48.611 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:48.611 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:48.611 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:29:48.611 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855041+0000", 2024-09-14T06:29:48.611 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.9639999999999999e-06 2024-09-14T06:29:48.611 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:48.611 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:48.612 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:29:48.612 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855043+0000", 2024-09-14T06:29:48.612 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.2259999999999999e-06 2024-09-14T06:29:48.612 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:48.612 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:48.612 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:29:48.612 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.919347+0000", 2024-09-14T06:29:48.612 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.064304091999999993 2024-09-14T06:29:48.612 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:48.612 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:48.612 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:29:48.612 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.919393+0000", 2024-09-14T06:29:48.612 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.5340999999999998e-05 2024-09-14T06:29:48.612 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:48.613 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:48.613 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:29:48.613 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.919507+0000", 2024-09-14T06:29:48.613 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000114028 2024-09-14T06:29:48.613 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:48.613 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:48.613 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:48.613 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:48.613 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:48.613 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:44432 3.2 3:4603ad7f:::rbd_data.112dca2f6ae1.0000000000000005:17 [read 1040384~4096 [fadvise_nocache]] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:29:48.613 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:43.855046+0000", 2024-09-14T06:29:48.613 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 4.725172197, 2024-09-14T06:29:48.613 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.064639253999999993, 2024-09-14T06:29:48.613 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:29:48.614 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:29:48.614 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:29:48.614 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:29:48.614 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:29:48.614 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:48.614 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 44432 2024-09-14T06:29:48.614 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:48.614 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:29:48.614 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:48.614 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:29:48.614 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855046+0000", 2024-09-14T06:29:48.614 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:48.614 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:48.614 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:48.615 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:29:48.615 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855046+0000", 2024-09-14T06:29:48.615 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:48.615 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:48.615 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:48.615 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:29:48.615 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855046+0000", 2024-09-14T06:29:48.615 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.05e-07 2024-09-14T06:29:48.615 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:48.615 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:48.615 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:29:48.615 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855047+0000", 2024-09-14T06:29:48.615 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.879999999999999e-07 2024-09-14T06:29:48.615 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:48.615 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:48.616 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:29:48.616 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855050+0000", 2024-09-14T06:29:48.616 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.8459999999999999e-06 2024-09-14T06:29:48.616 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:48.616 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:48.616 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:29:48.616 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855052+0000", 2024-09-14T06:29:48.616 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.136e-06 2024-09-14T06:29:48.616 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:48.616 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:48.616 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:29:48.616 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.919536+0000", 2024-09-14T06:29:48.616 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.064484132 2024-09-14T06:29:48.617 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:48.617 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:48.617 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:29:48.617 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.919611+0000", 2024-09-14T06:29:48.617 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.4407999999999996e-05 2024-09-14T06:29:48.617 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:48.617 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:48.617 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:29:48.617 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.919685+0000", 2024-09-14T06:29:48.617 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.4538999999999999e-05 2024-09-14T06:29:48.617 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:48.618 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:48.618 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:48.618 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:48.618 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:48.618 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:44518 3.2 3:4190d882:::rbd_data.112dca2f6ae1.0000000000000002:17 [read 12288~4096 [fadvise_nocache]] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:29:48.618 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:43.855055+0000", 2024-09-14T06:29:48.618 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 4.7251633130000004, 2024-09-14T06:29:48.618 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.064894317000000007, 2024-09-14T06:29:48.618 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:29:48.618 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:29:48.618 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:29:48.618 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:29:48.618 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:29:48.618 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:48.619 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 44518 2024-09-14T06:29:48.619 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:48.619 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:29:48.619 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:48.619 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:29:48.619 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855055+0000", 2024-09-14T06:29:48.619 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:48.619 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:48.619 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:48.619 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:29:48.619 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855055+0000", 2024-09-14T06:29:48.619 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:48.619 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:48.619 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:48.619 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:29:48.620 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855055+0000", 2024-09-14T06:29:48.620 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.0699999999999999e-07 2024-09-14T06:29:48.620 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:48.620 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:48.620 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:29:48.620 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855056+0000", 2024-09-14T06:29:48.620 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.9400000000000004e-07 2024-09-14T06:29:48.620 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:48.620 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:48.620 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:29:48.620 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855059+0000", 2024-09-14T06:29:48.620 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.8870000000000001e-06 2024-09-14T06:29:48.620 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:48.620 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:48.621 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:29:48.621 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855061+0000", 2024-09-14T06:29:48.621 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.1919999999999999e-06 2024-09-14T06:29:48.621 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:48.621 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:48.621 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:29:48.621 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.919719+0000", 2024-09-14T06:29:48.621 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.064658013 2024-09-14T06:29:48.621 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:48.621 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:48.621 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:29:48.621 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.919788+0000", 2024-09-14T06:29:48.621 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.9282000000000005e-05 2024-09-14T06:29:48.622 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:48.622 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:48.622 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:29:48.622 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.919949+0000", 2024-09-14T06:29:48.622 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00016084199999999999 2024-09-14T06:29:48.622 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:48.622 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:48.622 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:48.622 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:48.622 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:48.622 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:44519 3.2 3:4190d882:::rbd_data.112dca2f6ae1.0000000000000002:17 [read 36864~4096 [fadvise_nocache]] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:29:48.622 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:43.855064+0000", 2024-09-14T06:29:48.622 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 4.7251543360000001, 2024-09-14T06:29:48.622 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.065081865000000003, 2024-09-14T06:29:48.623 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:29:48.623 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:29:48.623 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:29:48.623 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:29:48.623 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:29:48.623 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:48.623 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 44519 2024-09-14T06:29:48.623 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:48.623 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:29:48.623 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:48.623 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:29:48.623 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855064+0000", 2024-09-14T06:29:48.623 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:48.623 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:48.624 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:48.624 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:29:48.624 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855064+0000", 2024-09-14T06:29:48.624 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:48.624 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:48.624 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:48.624 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:29:48.624 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855064+0000", 2024-09-14T06:29:48.624 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.1e-07 2024-09-14T06:29:48.624 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:48.624 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:48.624 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:29:48.624 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855065+0000", 2024-09-14T06:29:48.624 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.035e-06 2024-09-14T06:29:48.625 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:48.625 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:48.625 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:29:48.625 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855068+0000", 2024-09-14T06:29:48.625 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.8339999999999999e-06 2024-09-14T06:29:48.625 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:48.625 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:48.625 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:29:48.625 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855071+0000", 2024-09-14T06:29:48.625 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.3450000000000002e-06 2024-09-14T06:29:48.625 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:48.625 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:48.625 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:29:48.625 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.919979+0000", 2024-09-14T06:29:48.626 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.064907647999999998 2024-09-14T06:29:48.626 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:48.626 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:48.626 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:29:48.626 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.920026+0000", 2024-09-14T06:29:48.626 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.7595000000000001e-05 2024-09-14T06:29:48.626 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:48.626 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:48.626 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:29:48.626 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.920146+0000", 2024-09-14T06:29:48.626 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00011919800000000001 2024-09-14T06:29:48.626 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:48.626 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:48.626 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:48.627 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:48.627 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:48.627 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:44520 3.2 3:4190d882:::rbd_data.112dca2f6ae1.0000000000000002:17 [read 61440~4096 [fadvise_nocache]] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:29:48.627 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:43.855075+0000", 2024-09-14T06:29:48.627 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 4.7251432849999997, 2024-09-14T06:29:48.627 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.065335078000000005, 2024-09-14T06:29:48.627 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:29:48.627 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:29:48.627 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:29:48.627 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:29:48.627 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:29:48.627 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:48.627 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 44520 2024-09-14T06:29:48.627 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:48.628 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:29:48.628 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:48.628 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:29:48.628 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855075+0000", 2024-09-14T06:29:48.628 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:48.628 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:48.628 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:48.628 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:29:48.628 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855075+0000", 2024-09-14T06:29:48.628 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:48.628 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:48.628 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:48.628 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:29:48.628 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855075+0000", 2024-09-14T06:29:48.628 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.1e-07 2024-09-14T06:29:48.629 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:48.629 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:48.629 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:29:48.629 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855076+0000", 2024-09-14T06:29:48.629 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.0839999999999999e-06 2024-09-14T06:29:48.629 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:48.629 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:48.629 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:29:48.629 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855079+0000", 2024-09-14T06:29:48.629 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.9359999999999999e-06 2024-09-14T06:29:48.629 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:48.629 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:48.629 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:29:48.629 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855084+0000", 2024-09-14T06:29:48.630 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.1730000000000003e-06 2024-09-14T06:29:48.630 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:48.630 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:48.630 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:29:48.630 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.920175+0000", 2024-09-14T06:29:48.630 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.065090234999999996 2024-09-14T06:29:48.630 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:48.630 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:48.630 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:29:48.630 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.920250+0000", 2024-09-14T06:29:48.630 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.5807000000000001e-05 2024-09-14T06:29:48.630 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:48.630 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:48.630 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:29:48.630 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.920410+0000", 2024-09-14T06:29:48.631 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00015963300000000001 2024-09-14T06:29:48.631 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:48.631 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:48.631 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:48.631 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:48.631 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:48.631 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:44521 3.2 3:4190d882:::rbd_data.112dca2f6ae1.0000000000000002:17 [read 86016~4096 [fadvise_nocache]] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:29:48.631 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:43.855087+0000", 2024-09-14T06:29:48.631 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 4.725130804, 2024-09-14T06:29:48.631 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.065563264999999996, 2024-09-14T06:29:48.631 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:29:48.631 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:29:48.631 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:29:48.631 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:29:48.632 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:29:48.632 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:48.632 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 44521 2024-09-14T06:29:48.632 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:48.632 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:29:48.632 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:48.632 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:29:48.632 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855087+0000", 2024-09-14T06:29:48.632 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:48.632 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:48.632 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:48.632 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:29:48.632 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855087+0000", 2024-09-14T06:29:48.632 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:48.632 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:48.633 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:48.633 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:29:48.633 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855088+0000", 2024-09-14T06:29:48.633 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.3699999999999999e-07 2024-09-14T06:29:48.633 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:48.633 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:48.633 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:29:48.633 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855089+0000", 2024-09-14T06:29:48.633 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.9900000000000009e-07 2024-09-14T06:29:48.633 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:48.633 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:48.633 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:29:48.633 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855091+0000", 2024-09-14T06:29:48.634 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.8940000000000002e-06 2024-09-14T06:29:48.634 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:48.634 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:48.634 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:29:48.634 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855094+0000", 2024-09-14T06:29:48.634 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.351e-06 2024-09-14T06:29:48.634 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:48.634 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:48.634 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:29:48.634 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.920437+0000", 2024-09-14T06:29:48.634 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.065343221000000007 2024-09-14T06:29:48.634 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:48.635 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:48.635 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:29:48.635 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.920481+0000", 2024-09-14T06:29:48.635 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.4431000000000002e-05 2024-09-14T06:29:48.635 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:48.635 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:48.635 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:29:48.635 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.920651+0000", 2024-09-14T06:29:48.635 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00016913200000000001 2024-09-14T06:29:48.635 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:48.635 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:48.635 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:48.635 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:48.635 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:48.636 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:44522 3.2 3:4190d882:::rbd_data.112dca2f6ae1.0000000000000002:17 [read 110592~4096 [fadvise_nocache]] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:29:48.636 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:43.855098+0000", 2024-09-14T06:29:48.636 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 4.7251205770000002, 2024-09-14T06:29:48.636 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.065827628999999999, 2024-09-14T06:29:48.636 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:29:48.636 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:29:48.636 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:29:48.636 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:29:48.636 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:29:48.636 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:48.636 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 44522 2024-09-14T06:29:48.636 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:48.636 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:29:48.636 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:48.637 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:29:48.637 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855098+0000", 2024-09-14T06:29:48.637 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:48.637 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:48.637 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:48.637 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:29:48.637 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855098+0000", 2024-09-14T06:29:48.637 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:48.637 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:48.637 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:48.637 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:29:48.637 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855098+0000", 2024-09-14T06:29:48.637 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.1e-07 2024-09-14T06:29:48.637 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:48.638 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:48.638 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:29:48.638 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855099+0000", 2024-09-14T06:29:48.638 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.012e-06 2024-09-14T06:29:48.638 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:48.638 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:48.638 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:29:48.638 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855102+0000", 2024-09-14T06:29:48.638 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.9380000000000001e-06 2024-09-14T06:29:48.638 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:48.638 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:48.638 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:29:48.638 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855108+0000", 2024-09-14T06:29:48.638 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.4339999999999997e-06 2024-09-14T06:29:48.639 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:48.639 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:48.639 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:29:48.639 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.920681+0000", 2024-09-14T06:29:48.639 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.065572943999999994 2024-09-14T06:29:48.639 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:48.639 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:48.639 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:29:48.639 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.920724+0000", 2024-09-14T06:29:48.639 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.2877000000000003e-05 2024-09-14T06:29:48.639 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:48.639 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:48.639 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:29:48.639 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.920925+0000", 2024-09-14T06:29:48.640 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00020121399999999999 2024-09-14T06:29:48.640 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:48.640 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:48.640 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:48.640 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:48.640 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:48.640 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:44523 3.2 3:4190d882:::rbd_data.112dca2f6ae1.0000000000000002:17 [read 135168~4096 [fadvise_nocache]] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:29:48.640 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:43.855111+0000", 2024-09-14T06:29:48.640 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 4.7251068549999999, 2024-09-14T06:29:48.640 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.066028198999999996, 2024-09-14T06:29:48.640 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:29:48.640 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:29:48.640 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:29:48.640 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:29:48.640 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:29:48.641 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:48.641 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 44523 2024-09-14T06:29:48.641 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:48.641 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:29:48.641 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:48.641 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:29:48.641 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855111+0000", 2024-09-14T06:29:48.641 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:48.641 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:48.641 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:48.641 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:29:48.641 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855111+0000", 2024-09-14T06:29:48.641 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:48.642 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:48.642 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:48.642 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:29:48.642 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855112+0000", 2024-09-14T06:29:48.642 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.1199999999999999e-07 2024-09-14T06:29:48.642 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:48.642 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:48.642 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:29:48.642 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855115+0000", 2024-09-14T06:29:48.642 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.3380000000000001e-06 2024-09-14T06:29:48.642 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:48.642 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:48.642 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:29:48.642 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855118+0000", 2024-09-14T06:29:48.643 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.366e-06 2024-09-14T06:29:48.643 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:48.643 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:48.643 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:29:48.643 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855121+0000", 2024-09-14T06:29:48.643 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.8329999999999998e-06 2024-09-14T06:29:48.643 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:48.643 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:48.643 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:29:48.643 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.920956+0000", 2024-09-14T06:29:48.643 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.065835251999999997 2024-09-14T06:29:48.643 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:48.643 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:48.643 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:29:48.643 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.921001+0000", 2024-09-14T06:29:48.644 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.4778999999999999e-05 2024-09-14T06:29:48.644 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:48.644 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:48.644 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:29:48.644 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.921139+0000", 2024-09-14T06:29:48.644 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000138419 2024-09-14T06:29:48.644 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:48.644 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:48.644 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:48.644 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:48.644 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:48.644 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:44524 3.2 3:4190d882:::rbd_data.112dca2f6ae1.0000000000000002:17 [read 159744~4096 [fadvise_nocache]] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:29:48.644 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:43.855124+0000", 2024-09-14T06:29:48.645 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 4.7250938759999999, 2024-09-14T06:29:48.645 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.066249261000000004, 2024-09-14T06:29:48.645 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:29:48.645 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:29:48.645 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:29:48.645 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:29:48.645 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:29:48.645 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:48.645 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 44524 2024-09-14T06:29:48.645 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:48.645 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:29:48.645 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:48.645 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:29:48.646 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855124+0000", 2024-09-14T06:29:48.646 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:48.646 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:48.646 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:48.646 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:29:48.646 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855124+0000", 2024-09-14T06:29:48.646 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:48.646 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:48.646 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:48.646 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:29:48.646 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861086+0000", 2024-09-14T06:29:48.646 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0059613829999999998 2024-09-14T06:29:48.646 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:48.646 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:48.647 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:29:48.647 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861100+0000", 2024-09-14T06:29:48.647 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.4644000000000001e-05 2024-09-14T06:29:48.647 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:48.647 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:48.647 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:29:48.647 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861109+0000", 2024-09-14T06:29:48.647 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.541e-06 2024-09-14T06:29:48.647 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:48.647 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:48.647 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:29:48.647 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861117+0000", 2024-09-14T06:29:48.647 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.1049999999999995e-06 2024-09-14T06:29:48.647 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:48.647 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:48.648 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:29:48.648 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.921169+0000", 2024-09-14T06:29:48.648 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.060052405000000003 2024-09-14T06:29:48.648 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:48.648 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:48.648 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:29:48.648 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.921213+0000", 2024-09-14T06:29:48.648 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.3565e-05 2024-09-14T06:29:48.648 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:48.648 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:48.648 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:29:48.648 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.921374+0000", 2024-09-14T06:29:48.648 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00016061800000000001 2024-09-14T06:29:48.648 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:48.649 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:48.649 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:48.649 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:48.649 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:48.649 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:44536 3.2 3:4190d882:::rbd_data.112dca2f6ae1.0000000000000002:17 [read 454656~4096 [fadvise_nocache]] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:29:48.649 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:43.861384+0000", 2024-09-14T06:29:48.649 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 4.7188343189999999, 2024-09-14T06:29:48.649 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.063058528000000003, 2024-09-14T06:29:48.649 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:29:48.649 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:29:48.649 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:29:48.649 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:29:48.650 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:29:48.650 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:48.650 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 44536 2024-09-14T06:29:48.650 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:48.650 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:29:48.650 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:48.650 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:29:48.650 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861384+0000", 2024-09-14T06:29:48.650 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:48.650 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:48.650 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:48.650 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:29:48.650 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861384+0000", 2024-09-14T06:29:48.650 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:48.650 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:48.651 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:48.651 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:29:48.651 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861384+0000", 2024-09-14T06:29:48.651 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.4000000000000002e-07 2024-09-14T06:29:48.651 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:48.651 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:48.651 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:29:48.651 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861386+0000", 2024-09-14T06:29:48.651 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.891e-06 2024-09-14T06:29:48.651 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:48.651 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:48.651 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:29:48.651 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861392+0000", 2024-09-14T06:29:48.651 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.9939999999999997e-06 2024-09-14T06:29:48.651 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:48.652 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:48.652 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:29:48.652 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861398+0000", 2024-09-14T06:29:48.652 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.507e-06 2024-09-14T06:29:48.652 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:48.652 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:48.652 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:29:48.652 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.924149+0000", 2024-09-14T06:29:48.652 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.062751135999999999 2024-09-14T06:29:48.652 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:48.652 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:48.652 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:29:48.652 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.924226+0000", 2024-09-14T06:29:48.653 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.7517999999999999e-05 2024-09-14T06:29:48.653 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:48.653 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:48.653 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:29:48.653 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.924442+0000", 2024-09-14T06:29:48.653 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000216042 2024-09-14T06:29:48.653 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:48.653 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:48.653 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:48.653 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:48.653 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:48.653 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:44537 3.2 3:4190d882:::rbd_data.112dca2f6ae1.0000000000000002:17 [read 479232~4096 [fadvise_nocache]] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:29:48.653 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:43.861405+0000", 2024-09-14T06:29:48.653 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 4.7188132730000003, 2024-09-14T06:29:48.654 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.063351795000000002, 2024-09-14T06:29:48.654 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:29:48.654 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:29:48.654 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:29:48.654 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:29:48.654 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:29:48.654 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:48.654 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 44537 2024-09-14T06:29:48.654 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:48.654 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:29:48.654 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:48.654 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:29:48.654 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861405+0000", 2024-09-14T06:29:48.654 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:48.654 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:48.654 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:48.655 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:29:48.655 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861405+0000", 2024-09-14T06:29:48.655 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:48.655 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:48.655 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:48.655 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:29:48.655 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861405+0000", 2024-09-14T06:29:48.655 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.9099999999999999e-07 2024-09-14T06:29:48.655 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:48.655 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:48.655 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:29:48.655 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861407+0000", 2024-09-14T06:29:48.655 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.075e-06 2024-09-14T06:29:48.655 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:48.656 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:48.656 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:29:48.656 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861414+0000", 2024-09-14T06:29:48.656 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.4659999999999996e-06 2024-09-14T06:29:48.656 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:48.656 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:48.656 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:29:48.656 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861418+0000", 2024-09-14T06:29:48.656 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.5519999999999998e-06 2024-09-14T06:29:48.656 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:48.656 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:48.656 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:29:48.656 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.924474+0000", 2024-09-14T06:29:48.656 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.063056012999999994 2024-09-14T06:29:48.657 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:48.657 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:48.657 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:29:48.657 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.924556+0000", 2024-09-14T06:29:48.657 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.1181999999999996e-05 2024-09-14T06:29:48.657 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:48.657 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:48.657 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:29:48.657 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.924757+0000", 2024-09-14T06:29:48.657 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00020111599999999999 2024-09-14T06:29:48.657 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:48.657 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:48.657 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:48.657 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:48.658 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:48.658 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:44538 3.2 3:4190d882:::rbd_data.112dca2f6ae1.0000000000000002:17 [read 503808~4096 [fadvise_nocache]] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:29:48.658 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:43.861425+0000", 2024-09-14T06:29:48.658 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 4.7187932320000003, 2024-09-14T06:29:48.658 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.063627125000000007, 2024-09-14T06:29:48.658 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:29:48.658 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:29:48.658 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:29:48.658 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:29:48.658 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:29:48.658 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:48.658 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 44538 2024-09-14T06:29:48.658 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:48.659 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:29:48.659 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:48.659 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:29:48.659 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861425+0000", 2024-09-14T06:29:48.659 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:48.659 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:48.659 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:48.659 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:29:48.659 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861425+0000", 2024-09-14T06:29:48.659 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:48.659 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:48.659 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:48.659 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:29:48.659 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861425+0000", 2024-09-14T06:29:48.659 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.9999999999999998e-07 2024-09-14T06:29:48.660 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:48.660 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:48.660 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:29:48.660 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861427+0000", 2024-09-14T06:29:48.660 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.9829999999999999e-06 2024-09-14T06:29:48.660 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:48.660 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:48.660 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:29:48.660 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861433+0000", 2024-09-14T06:29:48.660 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.2889999999999997e-06 2024-09-14T06:29:48.660 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:48.660 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:48.660 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:29:48.660 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861454+0000", 2024-09-14T06:29:48.661 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.1844999999999998e-05 2024-09-14T06:29:48.661 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:48.661 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:48.661 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:29:48.661 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.924786+0000", 2024-09-14T06:29:48.661 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.063331497 2024-09-14T06:29:48.661 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:48.661 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:48.661 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:29:48.661 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.924864+0000", 2024-09-14T06:29:48.661 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.7744000000000006e-05 2024-09-14T06:29:48.661 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:48.661 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:48.661 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:29:48.661 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.925052+0000", 2024-09-14T06:29:48.662 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00018836699999999999 2024-09-14T06:29:48.662 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:48.662 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:48.662 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:48.662 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:48.662 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:48.662 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:44539 3.2 3:4190d882:::rbd_data.112dca2f6ae1.0000000000000002:17 [read 528384~4096 [fadvise_nocache]] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:29:48.662 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:43.861462+0000", 2024-09-14T06:29:48.662 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 4.7187559050000001, 2024-09-14T06:29:48.662 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.063855705999999998, 2024-09-14T06:29:48.662 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:29:48.662 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:29:48.662 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:29:48.662 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:29:48.662 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:29:48.663 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:48.663 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 44539 2024-09-14T06:29:48.663 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:48.663 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:29:48.663 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:48.663 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:29:48.663 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861462+0000", 2024-09-14T06:29:48.663 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:48.663 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:48.663 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:48.663 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:29:48.663 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861462+0000", 2024-09-14T06:29:48.663 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:48.663 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:48.664 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:48.664 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:29:48.664 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861463+0000", 2024-09-14T06:29:48.664 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.5200000000000002e-07 2024-09-14T06:29:48.664 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:48.664 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:48.664 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:29:48.664 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861465+0000", 2024-09-14T06:29:48.664 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.5189999999999999e-06 2024-09-14T06:29:48.664 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:48.664 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:48.664 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:29:48.664 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861471+0000", 2024-09-14T06:29:48.664 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.6899999999999997e-06 2024-09-14T06:29:48.665 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:48.665 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:48.665 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:29:48.665 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861475+0000", 2024-09-14T06:29:48.665 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.5730000000000001e-06 2024-09-14T06:29:48.665 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:48.665 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:48.665 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:29:48.665 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.925085+0000", 2024-09-14T06:29:48.665 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.063609902999999995 2024-09-14T06:29:48.665 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:48.665 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:48.665 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:29:48.666 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.925145+0000", 2024-09-14T06:29:48.666 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.9383999999999999e-05 2024-09-14T06:29:48.666 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:48.666 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:48.666 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:29:48.666 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.925318+0000", 2024-09-14T06:29:48.666 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00017318499999999999 2024-09-14T06:29:48.666 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:48.666 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:48.666 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:48.666 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:48.666 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:48.666 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:44540 3.2 3:4190d882:::rbd_data.112dca2f6ae1.0000000000000002:17 [read 552960~4096 [fadvise_nocache]] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:29:48.666 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:43.861483+0000", 2024-09-14T06:29:48.666 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 4.7187354739999998, 2024-09-14T06:29:48.667 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.064065681999999999, 2024-09-14T06:29:48.667 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:29:48.667 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:29:48.667 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:29:48.667 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:29:48.667 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:29:48.667 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:48.667 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 44540 2024-09-14T06:29:48.667 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:48.667 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:29:48.667 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:48.667 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:29:48.667 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861483+0000", 2024-09-14T06:29:48.667 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:48.668 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:48.668 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:48.668 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:29:48.668 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861483+0000", 2024-09-14T06:29:48.668 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:48.668 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:48.668 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:48.668 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:29:48.668 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.867079+0000", 2024-09-14T06:29:48.668 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0055966480000000001 2024-09-14T06:29:48.668 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:48.668 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:48.668 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:29:48.668 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.867086+0000", 2024-09-14T06:29:48.669 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.387e-06 2024-09-14T06:29:48.669 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:48.669 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:48.669 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:29:48.669 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.867094+0000", 2024-09-14T06:29:48.669 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.9419999999999995e-06 2024-09-14T06:29:48.669 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:48.669 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:48.669 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:29:48.669 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.867109+0000", 2024-09-14T06:29:48.669 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.5818999999999999e-05 2024-09-14T06:29:48.669 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:48.669 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:48.669 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:29:48.670 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.925354+0000", 2024-09-14T06:29:48.670 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.058244047 2024-09-14T06:29:48.670 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:48.670 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:48.670 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:29:48.670 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.925406+0000", 2024-09-14T06:29:48.670 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.2661999999999998e-05 2024-09-14T06:29:48.670 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:48.670 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:48.670 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:29:48.670 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.925548+0000", 2024-09-14T06:29:48.670 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000142177 2024-09-14T06:29:48.670 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:48.671 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:48.671 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:48.671 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:48.671 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:48.671 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-14T06:29:48.672 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-14T06:29:48.680 INFO:tasks.ceph.osd.0.smithi094.stderr:2024-09-14T06:29:48.679+0000 7f07c9333640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33636) UID: 0 2024-09-14T06:29:48.756 INFO:teuthology.orchestra.run.smithi094.stderr:noscrub is set 2024-09-14T06:29:48.790 INFO:teuthology.orchestra.run.smithi138.stdout:{ 2024-09-14T06:29:48.790 INFO:teuthology.orchestra.run.smithi138.stdout: "ops": [], 2024-09-14T06:29:48.790 INFO:teuthology.orchestra.run.smithi138.stdout: "num_ops": 0 2024-09-14T06:29:48.790 INFO:teuthology.orchestra.run.smithi138.stdout:} 2024-09-14T06:29:48.799 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-14T06:29:48.814 INFO:tasks.ceph.osd.0.smithi094.stderr:2024-09-14T06:29:48.813+0000 7f07c9333640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33636) UID: 0 2024-09-14T06:29:48.881 INFO:tasks.ceph.osd.3.smithi094.stderr:2024-09-14T06:29:48.880+0000 7feeeafe8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33655) UID: 0 2024-09-14T06:29:48.904 INFO:teuthology.orchestra.run.smithi094.stdout:checking clone #10, image image_client.0-clone10 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent11 2024-09-14T06:29:48.905 INFO:teuthology.orchestra.run.smithi138.stdout:{ 2024-09-14T06:29:48.905 INFO:teuthology.orchestra.run.smithi138.stdout: "ops": [], 2024-09-14T06:29:48.905 INFO:teuthology.orchestra.run.smithi138.stdout: "complaint_time": 30, 2024-09-14T06:29:48.905 INFO:teuthology.orchestra.run.smithi138.stdout: "num_blocked_ops": 0 2024-09-14T06:29:48.905 INFO:teuthology.orchestra.run.smithi138.stdout:} 2024-09-14T06:29:48.914 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-14T06:29:48.982 INFO:tasks.ceph.osd.5.smithi138.stderr:2024-09-14T06:29:48.980+0000 7fe13746f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33618) UID: 0 2024-09-14T06:29:49.021 INFO:teuthology.orchestra.run.smithi138.stdout:{ 2024-09-14T06:29:49.022 INFO:teuthology.orchestra.run.smithi138.stdout: "size": 20, 2024-09-14T06:29:49.022 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 600, 2024-09-14T06:29:49.022 INFO:teuthology.orchestra.run.smithi138.stdout: "ops": [ 2024-09-14T06:29:49.022 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:49.022 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:19328 3.3 3:c34e5e51:::rbd_data.112d92027590.0000000000000024:f [read 1926144~1024 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:29:49.023 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:27.059768+0000", 2024-09-14T06:29:49.023 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 21.957604670999999, 2024-09-14T06:29:49.023 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046369147999999999, 2024-09-14T06:29:49.023 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:29:49.023 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:29:49.023 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:29:49.024 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:29:49.024 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:29:49.024 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:49.024 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 19328 2024-09-14T06:29:49.024 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:49.025 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:29:49.025 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:49.025 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:29:49.025 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.059768+0000", 2024-09-14T06:29:49.025 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:49.025 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:49.026 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:49.026 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:29:49.026 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.059768+0000", 2024-09-14T06:29:49.026 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:49.026 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:49.026 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:49.027 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:29:49.027 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.059874+0000", 2024-09-14T06:29:49.027 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00010551400000000001 2024-09-14T06:29:49.027 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:49.027 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:49.027 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:29:49.028 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.059877+0000", 2024-09-14T06:29:49.028 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.8279999999999999e-06 2024-09-14T06:29:49.028 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:49.028 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:49.028 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:29:49.028 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.059884+0000", 2024-09-14T06:29:49.029 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.6389999999999992e-06 2024-09-14T06:29:49.029 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:49.029 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:49.029 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:29:49.029 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.059890+0000", 2024-09-14T06:29:49.029 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.49e-06 2024-09-14T06:29:49.029 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:49.029 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:49.029 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:29:49.029 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.105804+0000", 2024-09-14T06:29:49.030 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.045914242000000001 2024-09-14T06:29:49.030 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:49.030 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:49.030 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:29:49.030 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.105886+0000", 2024-09-14T06:29:49.030 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.2305999999999994e-05 2024-09-14T06:29:49.030 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:49.030 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:49.030 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:29:49.030 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.106138+0000", 2024-09-14T06:29:49.031 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.000251129 2024-09-14T06:29:49.031 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:49.031 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:29:49.031 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:49.031 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:49.031 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:49.031 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:19329 3.3 3:c34e5e51:::rbd_data.112d92027590.0000000000000024:f [read 1936384~1024 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:29:49.031 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:27.059899+0000", 2024-09-14T06:29:49.031 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 21.957474147999999, 2024-09-14T06:29:49.032 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046572122000000001, 2024-09-14T06:29:49.032 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:29:49.032 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:29:49.032 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:29:49.032 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:29:49.032 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:29:49.032 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:49.032 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 19329 2024-09-14T06:29:49.032 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:49.033 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:29:49.033 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:49.033 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:29:49.033 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.059899+0000", 2024-09-14T06:29:49.033 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:49.033 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:49.033 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:49.033 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:29:49.033 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.059899+0000", 2024-09-14T06:29:49.033 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:49.034 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:49.034 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:49.034 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:29:49.034 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062501+0000", 2024-09-14T06:29:49.034 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.0026016419999999999 2024-09-14T06:29:49.034 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:49.034 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:49.034 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:29:49.034 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062514+0000", 2024-09-14T06:29:49.035 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.2947e-05 2024-09-14T06:29:49.035 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:49.035 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:49.035 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:29:49.035 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062523+0000", 2024-09-14T06:29:49.035 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.5519999999999993e-06 2024-09-14T06:29:49.035 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:49.036 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:49.036 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:29:49.036 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062534+0000", 2024-09-14T06:29:49.036 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.1416e-05 2024-09-14T06:29:49.036 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:49.036 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:49.036 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:29:49.036 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.106179+0000", 2024-09-14T06:29:49.036 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.043644049999999997 2024-09-14T06:29:49.037 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:49.037 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:49.037 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:29:49.037 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.106262+0000", 2024-09-14T06:29:49.037 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.3178999999999998e-05 2024-09-14T06:29:49.037 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:49.037 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:49.037 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:29:49.037 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.106471+0000", 2024-09-14T06:29:49.038 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00020933599999999999 2024-09-14T06:29:49.038 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:49.038 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:29:49.038 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:49.038 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:49.038 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:49.038 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:19351 3.3 3:c34e5e51:::rbd_data.112d92027590.0000000000000024:f [read 1159168~1024 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:29:49.038 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:27.062766+0000", 2024-09-14T06:29:49.038 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 21.954606923, 2024-09-14T06:29:49.039 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046099454999999998, 2024-09-14T06:29:49.039 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:29:49.039 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:29:49.039 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:29:49.039 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:29:49.039 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:29:49.039 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:49.039 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 19351 2024-09-14T06:29:49.040 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:49.040 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:29:49.040 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:49.040 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:29:49.040 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062766+0000", 2024-09-14T06:29:49.040 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:49.041 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:49.041 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:49.041 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:29:49.041 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062766+0000", 2024-09-14T06:29:49.041 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:49.042 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:49.042 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:49.042 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:29:49.042 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062766+0000", 2024-09-14T06:29:49.042 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.6e-07 2024-09-14T06:29:49.042 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:49.042 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:49.043 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:29:49.043 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062767+0000", 2024-09-14T06:29:49.043 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.0070000000000001e-06 2024-09-14T06:29:49.043 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:49.043 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:49.043 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:29:49.044 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062785+0000", 2024-09-14T06:29:49.044 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.7884999999999999e-05 2024-09-14T06:29:49.044 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:49.044 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:49.044 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:29:49.044 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062790+0000", 2024-09-14T06:29:49.045 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.0900000000000004e-06 2024-09-14T06:29:49.045 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:49.045 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:49.045 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:29:49.045 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.108735+0000", 2024-09-14T06:29:49.045 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.045944411999999997 2024-09-14T06:29:49.045 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:49.046 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:49.046 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:29:49.046 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.108809+0000", 2024-09-14T06:29:49.046 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.4157000000000002e-05 2024-09-14T06:29:49.046 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:49.046 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:49.046 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:29:49.046 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.108866+0000", 2024-09-14T06:29:49.047 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.6644000000000002e-05 2024-09-14T06:29:49.047 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:49.047 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:29:49.047 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:49.047 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:49.047 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:49.048 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:19352 3.3 3:c34e5e51:::rbd_data.112d92027590.0000000000000024:f [read 1169408~1024 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:29:49.048 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:27.062798+0000", 2024-09-14T06:29:49.048 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 21.954575512000002, 2024-09-14T06:29:49.048 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046255900000000003, 2024-09-14T06:29:49.048 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:29:49.048 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:29:49.049 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:29:49.049 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:29:49.049 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:29:49.049 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:49.049 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 19352 2024-09-14T06:29:49.049 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:49.050 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:29:49.050 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:49.050 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:29:49.050 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062798+0000", 2024-09-14T06:29:49.050 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:49.050 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:49.051 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:49.051 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:29:49.051 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062798+0000", 2024-09-14T06:29:49.051 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:49.051 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:49.051 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:49.052 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:29:49.052 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062798+0000", 2024-09-14T06:29:49.052 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.37e-07 2024-09-14T06:29:49.052 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:49.052 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:49.052 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:29:49.053 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062800+0000", 2024-09-14T06:29:49.053 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.0269999999999998e-06 2024-09-14T06:29:49.053 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:49.053 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:49.053 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:29:49.053 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062806+0000", 2024-09-14T06:29:49.054 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.6690000000000003e-06 2024-09-14T06:29:49.054 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:49.054 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:49.054 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:29:49.054 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062810+0000", 2024-09-14T06:29:49.055 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.3449999999999999e-06 2024-09-14T06:29:49.055 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:49.055 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:49.055 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:29:49.055 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.108892+0000", 2024-09-14T06:29:49.055 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046081715000000002 2024-09-14T06:29:49.056 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:49.056 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:49.056 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:29:49.056 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.108940+0000", 2024-09-14T06:29:49.056 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.7815999999999998e-05 2024-09-14T06:29:49.056 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:49.057 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:49.057 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:29:49.057 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.109053+0000", 2024-09-14T06:29:49.057 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.000113791 2024-09-14T06:29:49.057 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:49.057 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:29:49.058 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:49.058 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:49.058 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:49.058 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:19353 3.3 3:c34e5e51:::rbd_data.112d92027590.0000000000000024:f [read 1179648~1024 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:29:49.058 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:27.062816+0000", 2024-09-14T06:29:49.059 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 21.954556993000001, 2024-09-14T06:29:49.059 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046481256999999998, 2024-09-14T06:29:49.059 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:29:49.059 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:29:49.059 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:29:49.059 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:29:49.060 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:29:49.060 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:49.060 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 19353 2024-09-14T06:29:49.060 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:49.060 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:29:49.060 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:49.061 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:29:49.061 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062816+0000", 2024-09-14T06:29:49.061 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:49.061 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:49.061 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:49.061 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:29:49.062 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062816+0000", 2024-09-14T06:29:49.062 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:49.062 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:49.062 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:49.062 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:29:49.062 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062816+0000", 2024-09-14T06:29:49.062 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.2899999999999999e-07 2024-09-14T06:29:49.063 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:49.063 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:49.063 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:29:49.063 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062818+0000", 2024-09-14T06:29:49.063 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.699e-06 2024-09-14T06:29:49.063 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:49.064 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:49.064 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:29:49.064 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062825+0000", 2024-09-14T06:29:49.064 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.878e-06 2024-09-14T06:29:49.064 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:49.065 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:49.065 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:29:49.065 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062830+0000", 2024-09-14T06:29:49.065 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.5410000000000002e-06 2024-09-14T06:29:49.065 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:49.065 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:49.066 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:29:49.066 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.109096+0000", 2024-09-14T06:29:49.066 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046266517 2024-09-14T06:29:49.066 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:49.066 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:49.066 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:29:49.067 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.109190+0000", 2024-09-14T06:29:49.067 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.4000999999999997e-05 2024-09-14T06:29:49.067 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:49.067 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:49.067 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:29:49.068 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.109297+0000", 2024-09-14T06:29:49.068 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.000107292 2024-09-14T06:29:49.068 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:49.068 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:29:49.068 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:49.068 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:49.069 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:49.069 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:19354 3.3 3:c34e5e51:::rbd_data.112d92027590.0000000000000024:f [read 1189888~1024 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:29:49.069 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:27.062842+0000", 2024-09-14T06:29:49.069 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 21.954531538000001, 2024-09-14T06:29:49.069 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046594257, 2024-09-14T06:29:49.069 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:29:49.070 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:29:49.070 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:29:49.070 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:29:49.070 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:29:49.070 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:49.071 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 19354 2024-09-14T06:29:49.071 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:49.071 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:29:49.071 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:49.071 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:29:49.071 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062842+0000", 2024-09-14T06:29:49.072 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:49.072 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:49.072 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:49.072 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:29:49.072 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062842+0000", 2024-09-14T06:29:49.072 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:49.073 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:49.073 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:49.073 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:29:49.073 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062842+0000", 2024-09-14T06:29:49.073 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.7399999999999999e-07 2024-09-14T06:29:49.073 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:49.074 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:49.074 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:29:49.074 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062859+0000", 2024-09-14T06:29:49.074 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.7010000000000001e-05 2024-09-14T06:29:49.074 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:49.074 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:49.075 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:29:49.075 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062866+0000", 2024-09-14T06:29:49.075 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.9990000000000002e-06 2024-09-14T06:29:49.075 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:49.075 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:49.075 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:29:49.075 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062873+0000", 2024-09-14T06:29:49.075 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.8569999999999997e-06 2024-09-14T06:29:49.075 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:49.076 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:49.076 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:29:49.076 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.109335+0000", 2024-09-14T06:29:49.076 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046461943999999998 2024-09-14T06:29:49.076 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:49.076 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:49.076 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:29:49.076 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.109377+0000", 2024-09-14T06:29:49.076 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.2558999999999997e-05 2024-09-14T06:29:49.077 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:49.077 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:49.077 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:29:49.077 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.109436+0000", 2024-09-14T06:29:49.077 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.8514000000000003e-05 2024-09-14T06:29:49.077 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:49.077 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:29:49.077 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:49.077 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:49.077 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:49.077 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:19355 3.3 3:c34e5e51:::rbd_data.112d92027590.0000000000000024:f [read 1200128~1024 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:29:49.077 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:27.062884+0000", 2024-09-14T06:29:49.077 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 21.954488864999998, 2024-09-14T06:29:49.077 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046681650999999998, 2024-09-14T06:29:49.078 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:29:49.078 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:29:49.078 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:29:49.078 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:29:49.078 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:29:49.078 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:49.078 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 19355 2024-09-14T06:29:49.078 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:49.078 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:29:49.078 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:49.078 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:29:49.078 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062884+0000", 2024-09-14T06:29:49.078 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:49.078 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:49.079 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:49.079 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:29:49.079 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062884+0000", 2024-09-14T06:29:49.079 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:49.079 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:49.079 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:49.079 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:29:49.079 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062885+0000", 2024-09-14T06:29:49.079 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.8000000000000001e-07 2024-09-14T06:29:49.079 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:49.079 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:49.079 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:29:49.079 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062886+0000", 2024-09-14T06:29:49.079 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.8920000000000001e-06 2024-09-14T06:29:49.079 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:49.080 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:49.080 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:29:49.080 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062892+0000", 2024-09-14T06:29:49.080 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.84e-06 2024-09-14T06:29:49.080 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:49.080 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:49.080 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:29:49.080 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062899+0000", 2024-09-14T06:29:49.080 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.5339999999999996e-06 2024-09-14T06:29:49.080 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:49.080 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:49.080 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:29:49.080 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.109466+0000", 2024-09-14T06:29:49.080 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046566919999999998 2024-09-14T06:29:49.081 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:49.081 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:49.081 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:29:49.081 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.109510+0000", 2024-09-14T06:29:49.081 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.4153000000000002e-05 2024-09-14T06:29:49.081 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:49.081 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:49.081 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:29:49.081 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.109566+0000", 2024-09-14T06:29:49.081 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.5931999999999999e-05 2024-09-14T06:29:49.081 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:49.081 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:29:49.081 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:49.081 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:49.081 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:49.082 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:19356 3.3 3:c34e5e51:::rbd_data.112d92027590.0000000000000024:f [read 1210368~1024 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:29:49.082 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:27.062908+0000", 2024-09-14T06:29:49.082 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 21.954465427999999, 2024-09-14T06:29:49.082 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046807498000000003, 2024-09-14T06:29:49.082 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:29:49.082 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:29:49.082 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:29:49.082 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:29:49.082 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:29:49.082 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:49.082 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 19356 2024-09-14T06:29:49.082 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:49.082 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:29:49.082 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:49.083 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:29:49.083 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062908+0000", 2024-09-14T06:29:49.083 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:49.083 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:49.083 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:49.083 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:29:49.083 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062908+0000", 2024-09-14T06:29:49.083 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:49.083 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:49.083 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:49.083 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:29:49.083 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062908+0000", 2024-09-14T06:29:49.083 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.5699999999999998e-07 2024-09-14T06:29:49.083 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:49.083 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:49.084 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:29:49.084 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062910+0000", 2024-09-14T06:29:49.084 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.7710000000000001e-06 2024-09-14T06:29:49.084 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:49.084 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:49.084 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:29:49.084 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062920+0000", 2024-09-14T06:29:49.084 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.0654e-05 2024-09-14T06:29:49.084 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:49.084 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:49.084 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:29:49.084 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062931+0000", 2024-09-14T06:29:49.084 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.075e-05 2024-09-14T06:29:49.084 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:49.085 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:49.085 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:29:49.085 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.109595+0000", 2024-09-14T06:29:49.085 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046663713000000002 2024-09-14T06:29:49.085 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:49.085 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:49.085 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:29:49.085 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.109632+0000", 2024-09-14T06:29:49.085 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.7505000000000001e-05 2024-09-14T06:29:49.085 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:49.085 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:49.085 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:29:49.085 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.109715+0000", 2024-09-14T06:29:49.085 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.2748000000000001e-05 2024-09-14T06:29:49.085 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:49.086 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:29:49.086 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:49.086 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:49.086 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:49.086 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:19357 3.3 3:c34e5e51:::rbd_data.112d92027590.0000000000000024:f [read 1220608~1024 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:29:49.086 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:27.062939+0000", 2024-09-14T06:29:49.086 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 21.954433980000001, 2024-09-14T06:29:49.086 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046920508, 2024-09-14T06:29:49.086 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:29:49.086 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:29:49.086 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:29:49.086 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:29:49.086 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:29:49.086 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:49.086 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 19357 2024-09-14T06:29:49.087 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:49.087 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:29:49.087 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:49.087 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:29:49.087 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062939+0000", 2024-09-14T06:29:49.087 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:49.087 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:49.087 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:49.087 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:29:49.087 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062939+0000", 2024-09-14T06:29:49.087 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:49.087 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:49.087 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:49.087 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:29:49.088 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062940+0000", 2024-09-14T06:29:49.088 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.32e-07 2024-09-14T06:29:49.088 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:49.088 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:49.088 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:29:49.088 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062942+0000", 2024-09-14T06:29:49.088 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.1919999999999999e-06 2024-09-14T06:29:49.088 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:49.088 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:49.088 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:29:49.088 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062948+0000", 2024-09-14T06:29:49.088 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.1149999999999999e-06 2024-09-14T06:29:49.088 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:49.088 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:49.089 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:29:49.089 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062960+0000", 2024-09-14T06:29:49.089 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.1895e-05 2024-09-14T06:29:49.089 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:49.089 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:49.089 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:29:49.089 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.109704+0000", 2024-09-14T06:29:49.089 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046744584999999998 2024-09-14T06:29:49.089 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:49.089 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:49.089 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:29:49.089 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.109777+0000", 2024-09-14T06:29:49.089 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.2282999999999999e-05 2024-09-14T06:29:49.089 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:49.089 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:49.090 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:29:49.090 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.109860+0000", 2024-09-14T06:29:49.090 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.3005999999999998e-05 2024-09-14T06:29:49.090 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:49.090 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:29:49.090 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:49.090 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:49.090 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:49.090 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:19358 3.3 3:c34e5e51:::rbd_data.112d92027590.0000000000000024:f [read 1230848~1024 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:29:49.090 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:27.062967+0000", 2024-09-14T06:29:49.090 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 21.954406349999999, 2024-09-14T06:29:49.090 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047144471, 2024-09-14T06:29:49.090 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:29:49.090 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:29:49.090 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:29:49.091 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:29:49.091 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:29:49.091 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:49.091 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 19358 2024-09-14T06:29:49.091 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:49.091 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:29:49.091 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:49.091 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:29:49.091 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062967+0000", 2024-09-14T06:29:49.091 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:49.091 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:49.091 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:49.091 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:29:49.091 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062967+0000", 2024-09-14T06:29:49.092 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:49.092 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:49.092 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:49.092 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:29:49.092 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062967+0000", 2024-09-14T06:29:49.092 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.89e-07 2024-09-14T06:29:49.092 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:49.092 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:49.092 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:29:49.092 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062969+0000", 2024-09-14T06:29:49.092 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.9929999999999998e-06 2024-09-14T06:29:49.092 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:49.092 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:49.092 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:29:49.093 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062975+0000", 2024-09-14T06:29:49.093 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.9030000000000001e-06 2024-09-14T06:29:49.093 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:49.093 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:49.093 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:29:49.093 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062979+0000", 2024-09-14T06:29:49.093 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.2250000000000002e-06 2024-09-14T06:29:49.093 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:49.093 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:49.093 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:29:49.093 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.109934+0000", 2024-09-14T06:29:49.093 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046954410000000002 2024-09-14T06:29:49.093 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:49.093 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:49.094 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:29:49.094 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.110007+0000", 2024-09-14T06:29:49.094 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.3829000000000001e-05 2024-09-14T06:29:49.094 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:49.094 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:49.094 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:29:49.094 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.110111+0000", 2024-09-14T06:29:49.094 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00010372199999999999 2024-09-14T06:29:49.094 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:49.094 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:29:49.094 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:49.094 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:49.094 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:49.094 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:19359 3.3 3:c34e5e51:::rbd_data.112d92027590.0000000000000024:f [read 1241088~1024 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:29:49.094 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:27.062987+0000", 2024-09-14T06:29:49.095 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 21.95438626, 2024-09-14T06:29:49.095 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047297182, 2024-09-14T06:29:49.095 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:29:49.095 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:29:49.095 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:29:49.095 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:29:49.095 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:29:49.095 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:49.095 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 19359 2024-09-14T06:29:49.095 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:49.095 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:29:49.095 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:49.095 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:29:49.095 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062987+0000", 2024-09-14T06:29:49.095 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:49.096 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:49.096 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:49.096 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:29:49.096 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062987+0000", 2024-09-14T06:29:49.096 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:49.096 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:49.096 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:49.096 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:29:49.096 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062987+0000", 2024-09-14T06:29:49.096 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.2500000000000001e-07 2024-09-14T06:29:49.096 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:49.096 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:49.096 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:29:49.096 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062989+0000", 2024-09-14T06:29:49.097 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.6080000000000001e-06 2024-09-14T06:29:49.097 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:49.097 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:49.097 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:29:49.097 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062994+0000", 2024-09-14T06:29:49.097 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.5520000000000004e-06 2024-09-14T06:29:49.097 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:49.097 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:49.097 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:29:49.097 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062999+0000", 2024-09-14T06:29:49.097 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.3930000000000001e-06 2024-09-14T06:29:49.097 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:49.097 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:49.097 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:29:49.098 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.110142+0000", 2024-09-14T06:29:49.098 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047143154 2024-09-14T06:29:49.098 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:49.098 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:49.098 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:29:49.098 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.110219+0000", 2024-09-14T06:29:49.098 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.6737999999999996e-05 2024-09-14T06:29:49.098 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:49.098 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:49.098 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:29:49.098 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.110284+0000", 2024-09-14T06:29:49.098 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.5412000000000003e-05 2024-09-14T06:29:49.098 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:49.098 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:29:49.098 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:49.099 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:49.099 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:49.099 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:19360 3.3 3:c34e5e51:::rbd_data.112d92027590.0000000000000024:f [read 1251328~1024 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:29:49.099 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:27.063017+0000", 2024-09-14T06:29:49.099 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 21.954355798999998, 2024-09-14T06:29:49.099 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047453823999999999, 2024-09-14T06:29:49.099 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:29:49.099 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:29:49.099 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:29:49.099 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:29:49.099 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:29:49.099 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:49.099 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 19360 2024-09-14T06:29:49.099 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:49.100 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:29:49.100 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:49.100 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:29:49.100 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.063017+0000", 2024-09-14T06:29:49.100 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:49.100 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:49.100 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:49.100 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:29:49.100 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.063017+0000", 2024-09-14T06:29:49.100 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:49.100 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:49.100 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:49.100 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:29:49.100 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.064977+0000", 2024-09-14T06:29:49.100 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.0019597080000000001 2024-09-14T06:29:49.101 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:49.101 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:49.101 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:29:49.101 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.064987+0000", 2024-09-14T06:29:49.101 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.9620000000000005e-06 2024-09-14T06:29:49.101 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:49.101 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:49.101 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:29:49.101 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065000+0000", 2024-09-14T06:29:49.101 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.3173000000000001e-05 2024-09-14T06:29:49.101 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:49.101 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:49.101 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:29:49.101 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065005+0000", 2024-09-14T06:29:49.102 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.9960000000000001e-06 2024-09-14T06:29:49.102 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:49.102 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:49.102 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:29:49.102 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.110314+0000", 2024-09-14T06:29:49.102 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.04530899 2024-09-14T06:29:49.102 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:49.102 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:49.102 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:29:49.102 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.110398+0000", 2024-09-14T06:29:49.102 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.3539000000000001e-05 2024-09-14T06:29:49.102 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:49.102 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:49.102 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:29:49.102 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.110471+0000", 2024-09-14T06:29:49.103 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.3455999999999996e-05 2024-09-14T06:29:49.103 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:49.103 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:29:49.103 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:49.103 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:49.103 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:49.103 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:19365 3.3 3:c34e5e51:::rbd_data.112d92027590.0000000000000024:f [read 1302528~1024 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:29:49.103 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:27.065058+0000", 2024-09-14T06:29:49.103 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 21.952315128999999, 2024-09-14T06:29:49.103 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046105526000000001, 2024-09-14T06:29:49.103 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:29:49.103 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:29:49.103 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:29:49.103 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:29:49.104 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:29:49.104 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:49.104 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 19365 2024-09-14T06:29:49.104 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:49.104 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:29:49.104 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:49.104 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:29:49.104 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065058+0000", 2024-09-14T06:29:49.104 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:49.104 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:49.104 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:49.104 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:29:49.104 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065058+0000", 2024-09-14T06:29:49.105 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:49.105 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:49.105 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:49.105 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:29:49.105 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065058+0000", 2024-09-14T06:29:49.105 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.1899999999999999e-07 2024-09-14T06:29:49.105 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:49.105 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:49.105 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:29:49.105 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065059+0000", 2024-09-14T06:29:49.105 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.1299999999999998e-07 2024-09-14T06:29:49.105 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:49.105 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:49.105 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:29:49.105 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065063+0000", 2024-09-14T06:29:49.106 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.1099999999999996e-06 2024-09-14T06:29:49.106 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:49.106 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:49.106 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:29:49.106 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065070+0000", 2024-09-14T06:29:49.106 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.973e-06 2024-09-14T06:29:49.106 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:49.106 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:49.106 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:29:49.106 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.111014+0000", 2024-09-14T06:29:49.106 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.045943373000000003 2024-09-14T06:29:49.106 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:49.106 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:49.106 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:29:49.107 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.111073+0000", 2024-09-14T06:29:49.107 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.8987999999999998e-05 2024-09-14T06:29:49.107 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:49.107 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:49.107 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:29:49.107 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.111163+0000", 2024-09-14T06:29:49.107 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.0950000000000002e-05 2024-09-14T06:29:49.107 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:49.107 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:29:49.108 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:49.108 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:49.108 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:49.108 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:19366 3.3 3:c34e5e51:::rbd_data.112d92027590.0000000000000024:f [read 1312768~1024 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:29:49.108 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:27.065074+0000", 2024-09-14T06:29:49.108 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 21.952299279000002, 2024-09-14T06:29:49.108 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046289858000000003, 2024-09-14T06:29:49.108 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:29:49.108 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:29:49.108 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:29:49.108 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:29:49.108 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:29:49.108 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:49.108 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 19366 2024-09-14T06:29:49.109 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:49.109 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:29:49.109 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:49.109 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:29:49.109 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065074+0000", 2024-09-14T06:29:49.109 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:49.109 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:49.109 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:49.109 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:29:49.109 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065074+0000", 2024-09-14T06:29:49.109 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:49.109 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:49.109 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:49.109 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:29:49.110 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065074+0000", 2024-09-14T06:29:49.110 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.3900000000000001e-07 2024-09-14T06:29:49.110 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:49.110 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:49.110 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:29:49.110 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065078+0000", 2024-09-14T06:29:49.110 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.3229999999999999e-06 2024-09-14T06:29:49.110 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:49.110 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:49.110 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:29:49.110 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065082+0000", 2024-09-14T06:29:49.110 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.8299999999999998e-06 2024-09-14T06:29:49.110 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:49.110 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:49.110 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:29:49.111 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065085+0000", 2024-09-14T06:29:49.111 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.8660000000000002e-06 2024-09-14T06:29:49.111 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:49.111 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:49.111 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:29:49.111 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.111270+0000", 2024-09-14T06:29:49.111 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046185035999999999 2024-09-14T06:29:49.111 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:49.111 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:49.111 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:29:49.111 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.111311+0000", 2024-09-14T06:29:49.111 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.0935e-05 2024-09-14T06:29:49.111 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:49.111 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:49.112 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:29:49.112 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.111364+0000", 2024-09-14T06:29:49.112 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.2629e-05 2024-09-14T06:29:49.112 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:49.112 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:29:49.112 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:49.112 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:49.112 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:49.112 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:19367 3.3 3:c34e5e51:::rbd_data.112d92027590.0000000000000024:f [read 1323008~1024 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:29:49.112 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:27.065115+0000", 2024-09-14T06:29:49.112 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 21.952257717999998, 2024-09-14T06:29:49.112 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046407003000000002, 2024-09-14T06:29:49.112 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:29:49.112 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:29:49.113 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:29:49.113 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:29:49.113 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:29:49.113 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:49.113 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 19367 2024-09-14T06:29:49.113 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:49.113 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:29:49.113 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:49.113 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:29:49.113 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065115+0000", 2024-09-14T06:29:49.113 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:49.113 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:49.113 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:49.113 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:29:49.114 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065115+0000", 2024-09-14T06:29:49.114 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:49.114 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:49.114 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:49.114 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:29:49.114 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065116+0000", 2024-09-14T06:29:49.114 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.4999999999999998e-07 2024-09-14T06:29:49.114 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:49.114 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:49.114 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:29:49.114 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065117+0000", 2024-09-14T06:29:49.114 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.189e-06 2024-09-14T06:29:49.114 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:49.114 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:49.114 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:29:49.115 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065122+0000", 2024-09-14T06:29:49.115 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.3709999999999999e-06 2024-09-14T06:29:49.115 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:49.115 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:49.115 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:29:49.115 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065125+0000", 2024-09-14T06:29:49.115 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.8940000000000002e-06 2024-09-14T06:29:49.115 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:49.115 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:49.115 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:29:49.115 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.111402+0000", 2024-09-14T06:29:49.115 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046276645999999998 2024-09-14T06:29:49.115 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:49.115 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:49.115 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:29:49.116 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.111445+0000", 2024-09-14T06:29:49.116 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.3288000000000003e-05 2024-09-14T06:29:49.116 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:49.116 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:49.116 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:29:49.116 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.111522+0000", 2024-09-14T06:29:49.116 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.7164999999999997e-05 2024-09-14T06:29:49.116 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:49.116 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:29:49.116 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:49.116 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:49.116 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:49.116 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:19368 3.3 3:c34e5e51:::rbd_data.112d92027590.0000000000000024:f [read 1333248~1024 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:29:49.116 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:27.065130+0000", 2024-09-14T06:29:49.117 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 21.952243427999999, 2024-09-14T06:29:49.117 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046507275000000001, 2024-09-14T06:29:49.117 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:29:49.117 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:29:49.117 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:29:49.117 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:29:49.117 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:29:49.117 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:49.117 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 19368 2024-09-14T06:29:49.117 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:49.117 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:29:49.117 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:49.117 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:29:49.117 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065130+0000", 2024-09-14T06:29:49.117 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:49.118 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:49.118 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:49.118 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:29:49.118 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065130+0000", 2024-09-14T06:29:49.118 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:49.118 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:49.118 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:49.118 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:29:49.118 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065130+0000", 2024-09-14T06:29:49.118 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.0800000000000001e-07 2024-09-14T06:29:49.118 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:49.118 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:49.118 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:29:49.118 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065131+0000", 2024-09-14T06:29:49.119 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.8700000000000004e-07 2024-09-14T06:29:49.119 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:49.119 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:49.119 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:29:49.119 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065134+0000", 2024-09-14T06:29:49.119 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.1039999999999998e-06 2024-09-14T06:29:49.119 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:49.119 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:49.119 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:29:49.119 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065136+0000", 2024-09-14T06:29:49.119 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.3949999999999999e-06 2024-09-14T06:29:49.119 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:49.119 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:49.119 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:29:49.119 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.111541+0000", 2024-09-14T06:29:49.120 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046404206000000003 2024-09-14T06:29:49.120 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:49.120 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:49.120 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:29:49.120 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.111582+0000", 2024-09-14T06:29:49.120 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.1051999999999999e-05 2024-09-14T06:29:49.120 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:49.120 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:49.120 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:29:49.120 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.111637+0000", 2024-09-14T06:29:49.120 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.5322999999999999e-05 2024-09-14T06:29:49.120 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:49.120 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:29:49.120 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:49.121 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:49.121 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:49.121 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:19369 3.3 3:c34e5e51:::rbd_data.112d92027590.0000000000000024:f [read 1343488~1024 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:29:49.121 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:27.065139+0000", 2024-09-14T06:29:49.121 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 21.952233611, 2024-09-14T06:29:49.121 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046682169000000003, 2024-09-14T06:29:49.121 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:29:49.121 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:29:49.121 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:29:49.121 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:29:49.121 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:29:49.121 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:49.121 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 19369 2024-09-14T06:29:49.121 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:49.122 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:29:49.122 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:49.122 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:29:49.122 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065139+0000", 2024-09-14T06:29:49.122 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:49.122 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:49.122 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:49.122 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:29:49.122 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065139+0000", 2024-09-14T06:29:49.122 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:49.122 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:49.122 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:49.122 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:29:49.122 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065140+0000", 2024-09-14T06:29:49.122 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.8299999999999998e-07 2024-09-14T06:29:49.123 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:49.123 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:49.123 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:29:49.123 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065141+0000", 2024-09-14T06:29:49.123 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.3900000000000003e-07 2024-09-14T06:29:49.123 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:49.123 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:49.123 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:29:49.123 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065144+0000", 2024-09-14T06:29:49.123 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.7409999999999998e-06 2024-09-14T06:29:49.123 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:49.123 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:49.123 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:29:49.123 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065147+0000", 2024-09-14T06:29:49.124 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.4150000000000002e-06 2024-09-14T06:29:49.124 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:49.124 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:49.124 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:29:49.124 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.111673+0000", 2024-09-14T06:29:49.124 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046526004000000003 2024-09-14T06:29:49.124 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:49.124 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:49.124 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:29:49.124 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.111749+0000", 2024-09-14T06:29:49.124 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.5741999999999994e-05 2024-09-14T06:29:49.124 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:49.124 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:49.124 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:29:49.125 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.111822+0000", 2024-09-14T06:29:49.125 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.3045000000000003e-05 2024-09-14T06:29:49.125 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:49.125 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:29:49.125 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:49.125 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:49.125 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:49.125 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:19370 3.3 3:c34e5e51:::rbd_data.112d92027590.0000000000000024:f [read 1353728~1024 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:29:49.125 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:27.065150+0000", 2024-09-14T06:29:49.125 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 21.952222882000001, 2024-09-14T06:29:49.125 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046897556999999999, 2024-09-14T06:29:49.125 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:29:49.125 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:29:49.125 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:29:49.126 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:29:49.126 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:29:49.126 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:49.126 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 19370 2024-09-14T06:29:49.126 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:49.126 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:29:49.126 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:49.126 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:29:49.126 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065150+0000", 2024-09-14T06:29:49.126 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:49.126 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:49.126 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:49.126 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:29:49.126 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065150+0000", 2024-09-14T06:29:49.126 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:49.127 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:49.127 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:49.127 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:29:49.127 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065150+0000", 2024-09-14T06:29:49.127 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.0200000000000001e-07 2024-09-14T06:29:49.127 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:49.127 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:49.127 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:29:49.127 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065151+0000", 2024-09-14T06:29:49.127 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.1100000000000004e-07 2024-09-14T06:29:49.127 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:49.127 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:49.127 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:29:49.127 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065154+0000", 2024-09-14T06:29:49.127 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.9639999999999999e-06 2024-09-14T06:29:49.128 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:49.128 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:49.128 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:29:49.128 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065157+0000", 2024-09-14T06:29:49.128 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.4260000000000002e-06 2024-09-14T06:29:49.128 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:49.128 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:49.128 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:29:49.128 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.111861+0000", 2024-09-14T06:29:49.128 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046704268 2024-09-14T06:29:49.128 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:49.128 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:49.128 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:29:49.128 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.111939+0000", 2024-09-14T06:29:49.129 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.8231000000000004e-05 2024-09-14T06:29:49.129 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:49.129 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:49.129 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:29:49.129 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.112048+0000", 2024-09-14T06:29:49.129 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00010855500000000001 2024-09-14T06:29:49.129 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:49.129 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:29:49.129 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:49.129 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:49.129 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:49.129 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:19371 3.3 3:c34e5e51:::rbd_data.112d92027590.0000000000000024:f [read 1363968~1024 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:29:49.129 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:27.065161+0000", 2024-09-14T06:29:49.129 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 21.952212232000001, 2024-09-14T06:29:49.129 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047021944000000003, 2024-09-14T06:29:49.130 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:29:49.130 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:29:49.130 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:29:49.130 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:29:49.130 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:29:49.130 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:49.130 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 19371 2024-09-14T06:29:49.130 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:49.130 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:29:49.130 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:49.130 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:29:49.130 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065161+0000", 2024-09-14T06:29:49.130 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:49.130 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:49.130 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:49.131 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:29:49.131 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065161+0000", 2024-09-14T06:29:49.131 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:49.131 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:49.131 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:49.131 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:29:49.131 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065161+0000", 2024-09-14T06:29:49.131 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.0200000000000001e-07 2024-09-14T06:29:49.131 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:49.131 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:49.131 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:29:49.131 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065166+0000", 2024-09-14T06:29:49.131 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.9869999999999999e-06 2024-09-14T06:29:49.131 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:49.131 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:49.132 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:29:49.132 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065169+0000", 2024-09-14T06:29:49.132 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.3799999999999998e-06 2024-09-14T06:29:49.132 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:49.132 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:49.132 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:29:49.132 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065172+0000", 2024-09-14T06:29:49.132 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.4830000000000002e-06 2024-09-14T06:29:49.132 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:49.132 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:49.132 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:29:49.132 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.112071+0000", 2024-09-14T06:29:49.132 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046899529000000002 2024-09-14T06:29:49.132 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:49.133 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:49.133 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:29:49.133 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.112128+0000", 2024-09-14T06:29:49.133 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.6341999999999997e-05 2024-09-14T06:29:49.133 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:49.133 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:49.133 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:29:49.133 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.112183+0000", 2024-09-14T06:29:49.133 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.5021000000000001e-05 2024-09-14T06:29:49.133 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:49.133 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:29:49.133 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:49.133 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:49.133 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:49.133 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:19372 3.3 3:c34e5e51:::rbd_data.112d92027590.0000000000000024:f [read 1374208~1024 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:29:49.134 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:27.065176+0000", 2024-09-14T06:29:49.134 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 21.952197082000001, 2024-09-14T06:29:49.134 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047188889999999997, 2024-09-14T06:29:49.134 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:29:49.134 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:29:49.134 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:29:49.134 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:29:49.134 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:29:49.134 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:49.134 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 19372 2024-09-14T06:29:49.134 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:49.134 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:29:49.134 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:49.134 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:29:49.134 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065176+0000", 2024-09-14T06:29:49.135 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:49.135 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:49.135 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:49.135 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:29:49.135 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065176+0000", 2024-09-14T06:29:49.135 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:49.135 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:49.135 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:49.135 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:29:49.135 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.071117+0000", 2024-09-14T06:29:49.135 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.0059404990000000001 2024-09-14T06:29:49.135 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:49.135 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:49.135 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:29:49.136 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.071120+0000", 2024-09-14T06:29:49.136 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.4340000000000001e-06 2024-09-14T06:29:49.136 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:49.136 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:49.136 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:29:49.136 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.071129+0000", 2024-09-14T06:29:49.136 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.8389999999999996e-06 2024-09-14T06:29:49.136 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:49.136 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:49.136 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:29:49.136 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.071134+0000", 2024-09-14T06:29:49.136 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.412e-06 2024-09-14T06:29:49.136 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:49.136 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:49.136 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:29:49.137 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.112221+0000", 2024-09-14T06:29:49.137 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.041086473999999998 2024-09-14T06:29:49.137 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:49.137 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:49.137 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:29:49.137 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.112299+0000", 2024-09-14T06:29:49.137 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.8746999999999997e-05 2024-09-14T06:29:49.137 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:49.137 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:49.137 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:29:49.137 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.112365+0000", 2024-09-14T06:29:49.137 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.5485000000000001e-05 2024-09-14T06:29:49.137 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:49.137 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:29:49.137 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:49.138 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:49.138 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:29:49.138 INFO:teuthology.orchestra.run.smithi138.stdout:} 2024-09-14T06:29:49.139 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-14T06:29:49.140 INFO:tasks.ceph.osd.0.smithi094.stderr:2024-09-14T06:29:49.138+0000 7f07c9333640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33636) UID: 0 2024-09-14T06:29:49.152 INFO:teuthology.orchestra.run.smithi094.stdout:1548 trunc from 0xe238b03 to 0x37f4cc0 2024-09-14T06:29:49.173 INFO:teuthology.orchestra.run.smithi094.stdout:1549 writesame 0x68df6e7 thru 0x68ed977 data_size 0x1(0xe291 bytes) 2024-09-14T06:29:49.173 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-14T06:29:49.173 INFO:teuthology.orchestra.run.smithi094.stdout:1552 read 0x666f70f thru 0x667d19d (0xda8f bytes) 2024-09-14T06:29:49.184 INFO:teuthology.orchestra.run.smithi094.stdout:1553 writesame 0xac52907 thru 0xac61c9d data_size 0x1(0xf397 bytes) 2024-09-14T06:29:49.191 INFO:teuthology.orchestra.run.smithi094.stdout:1554 writesame 0xb197f21 thru 0xb19cd28 data_size 0xb(0x4e08 bytes) 2024-09-14T06:29:49.199 INFO:teuthology.orchestra.run.smithi094.stdout:1557 write 0xcdefba6 thru 0xcdf2505 (0x2960 bytes) 2024-09-14T06:29:49.200 INFO:teuthology.orchestra.run.smithi094.stdout:1558 writesame 0x8d83b1c thru 0x8d8e8db data_size 0x40(0xadc0 bytes) 2024-09-14T06:29:49.202 INFO:teuthology.orchestra.run.smithi094.stdout:1559 writesame 0x625ba99 thru 0x625c5ea data_size 0x2a(0xb52 bytes) 2024-09-14T06:29:49.204 INFO:teuthology.orchestra.run.smithi094.stdout:1560 read 0xad1c070 thru 0xad20ff4 (0x4f85 bytes) 2024-09-14T06:29:49.204 INFO:teuthology.orchestra.run.smithi094.stdout:1561 read 0x2dfdf9 thru 0x2e0d70 (0xf78 bytes) 2024-09-14T06:29:49.205 INFO:teuthology.orchestra.run.smithi094.stdout:1562 writesame 0x14859f0 thru 0x148d9bb data_size 0x2(0x7fcc bytes) 2024-09-14T06:29:49.224 INFO:teuthology.orchestra.run.smithi094.stdout:1565 trunc from 0xcdf2506 to 0x3b62c0f 2024-09-14T06:29:49.232 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-14T06:29:49.239 INFO:teuthology.orchestra.run.smithi094.stdout:1567 trunc from 0x3b62c0f to 0x9f2b3c8 2024-09-14T06:29:49.240 INFO:tasks.ceph.osd.1.smithi094.stderr:2024-09-14T06:29:49.239+0000 7f328e47a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33641) UID: 0 2024-09-14T06:29:49.244 INFO:teuthology.orchestra.run.smithi094.stdout:1569 write 0x17c94f4 thru 0x17d5213 (0xbd20 bytes) 2024-09-14T06:29:49.246 INFO:teuthology.orchestra.run.smithi094.stdout:1570 read 0x4af03f7 thru 0x4af605e (0x5c68 bytes) 2024-09-14T06:29:49.247 INFO:teuthology.orchestra.run.smithi094.stdout:1571 trunc from 0x9f2b3c8 to 0x9bd7adb 2024-09-14T06:29:49.261 INFO:teuthology.orchestra.run.smithi094.stdout:1573 write 0xb672c60 thru 0xb67e0f8 (0xb499 bytes) 2024-09-14T06:29:49.271 INFO:teuthology.orchestra.run.smithi094.stdout:1574 write 0xe54dcc1 thru 0xe55b70e (0xda4e bytes) 2024-09-14T06:29:49.273 INFO:teuthology.orchestra.run.smithi094.stdout:1576 punch from 0xd1fc437 to 0xd201249, (0x4e12 bytes) 2024-09-14T06:29:49.273 INFO:teuthology.orchestra.run.smithi094.stdout:1577 writesame 0xbcd42a6 thru 0xbcdaadf data_size 0x3(0x683a bytes) 2024-09-14T06:29:49.278 INFO:teuthology.orchestra.run.smithi094.stdout:1578 write 0xb422798 thru 0xb428995 (0x61fe bytes) 2024-09-14T06:29:49.279 INFO:teuthology.orchestra.run.smithi094.stdout:1579 write 0x99be0c2 thru 0x99cac12 (0xcb51 bytes) 2024-09-14T06:29:49.281 INFO:teuthology.orchestra.run.smithi094.stdout:1580 write 0xb92fa9c thru 0xb936dd4 (0x7339 bytes) 2024-09-14T06:29:49.283 INFO:teuthology.orchestra.run.smithi094.stdout:1581 write 0x1793bc0 thru 0x179494b (0xd8c bytes) 2024-09-14T06:29:49.284 INFO:teuthology.orchestra.run.smithi094.stdout:1582 punch from 0xb41fc82 to 0xb42caaa, (0xce28 bytes) 2024-09-14T06:29:49.285 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-14T06:29:49.285 INFO:teuthology.orchestra.run.smithi094.stdout:1584 trunc from 0xe55b70f to 0xde97ddb 2024-09-14T06:29:49.291 INFO:teuthology.orchestra.run.smithi138.stdout:{ 2024-09-14T06:29:49.291 INFO:teuthology.orchestra.run.smithi138.stdout: "ops": [], 2024-09-14T06:29:49.291 INFO:teuthology.orchestra.run.smithi138.stdout: "num_ops": 0 2024-09-14T06:29:49.291 INFO:teuthology.orchestra.run.smithi138.stdout:} 2024-09-14T06:29:49.295 INFO:teuthology.orchestra.run.smithi094.stdout:1589 punch from 0xc842a28 to 0xc85160f, (0xebe7 bytes) 2024-09-14T06:29:49.295 INFO:teuthology.orchestra.run.smithi094.stdout:1590 write 0x1c97636 thru 0x1c9ead3 (0x749e bytes) 2024-09-14T06:29:49.301 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-14T06:29:49.306 INFO:teuthology.orchestra.run.smithi094.stdout:1591 trunc from 0xde97ddb to 0x54ab1e9 2024-09-14T06:29:49.314 INFO:teuthology.orchestra.run.smithi094.stdout:1592 read 0x38e437c thru 0x38e65d0 (0x2255 bytes) 2024-09-14T06:29:49.315 INFO:teuthology.orchestra.run.smithi094.stdout:1593 read 0x2b9e8ae thru 0x2bae39f (0xfaf2 bytes) 2024-09-14T06:29:49.316 INFO:teuthology.orchestra.run.smithi094.stdout:1594 punch from 0x387bb59 to 0x3886eb6, (0xb35d bytes) 2024-09-14T06:29:49.316 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-14T06:29:49.316 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-14T06:29:49.316 INFO:teuthology.orchestra.run.smithi094.stdout:1600 punch from 0x476faf8 to 0x477084d, (0xd55 bytes) 2024-09-14T06:29:49.316 INFO:teuthology.orchestra.run.smithi094.stdout:1601 punch from 0x1eee057 to 0x1ef4614, (0x65bd bytes) 2024-09-14T06:29:49.316 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-14T06:29:49.316 INFO:teuthology.orchestra.run.smithi094.stdout:1604 read 0x15ad736 thru 0x15b3b96 (0x6461 bytes) 2024-09-14T06:29:49.316 INFO:teuthology.orchestra.run.smithi094.stdout:1605 punch from 0x42cfcde to 0x42d1b1c, (0x1e3e bytes) 2024-09-14T06:29:49.316 INFO:teuthology.orchestra.run.smithi094.stdout:1606 read 0x182faa9 thru 0x183d0a2 (0xd5fa bytes) 2024-09-14T06:29:49.317 INFO:teuthology.orchestra.run.smithi094.stdout:1608 write 0x4e96b3 thru 0x4f2eea (0x9838 bytes) 2024-09-14T06:29:49.318 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-14T06:29:49.324 INFO:teuthology.orchestra.run.smithi094.stdout:1610 writesame 0x6c0dd78 thru 0x6c1b45c data_size 0x1d(0xd6e5 bytes) 2024-09-14T06:29:49.326 INFO:teuthology.orchestra.run.smithi094.stdout:1611 punch from 0x1992e8f to 0x19a24c9, (0xf63a bytes) 2024-09-14T06:29:49.326 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-14T06:29:49.326 INFO:teuthology.orchestra.run.smithi094.stdout:1614 clone 13 order 18 su 16384 sc 15 2024-09-14T06:29:49.341 INFO:tasks.ceph.osd.2.smithi094.stderr:2024-09-14T06:29:49.340+0000 7faf3269b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33649) UID: 0 2024-09-14T06:29:49.405 INFO:teuthology.orchestra.run.smithi138.stdout:{ 2024-09-14T06:29:49.405 INFO:teuthology.orchestra.run.smithi138.stdout: "ops": [], 2024-09-14T06:29:49.405 INFO:teuthology.orchestra.run.smithi138.stdout: "complaint_time": 30, 2024-09-14T06:29:49.405 INFO:teuthology.orchestra.run.smithi138.stdout: "num_blocked_ops": 0 2024-09-14T06:29:49.405 INFO:teuthology.orchestra.run.smithi138.stdout:} 2024-09-14T06:29:49.413 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-14T06:29:49.457 INFO:tasks.ceph.osd.4.smithi138.stderr:2024-09-14T06:29:49.455+0000 7f6c653c8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33615) UID: 0 2024-09-14T06:29:49.519 INFO:teuthology.orchestra.run.smithi138.stdout:{ 2024-09-14T06:29:49.519 INFO:teuthology.orchestra.run.smithi138.stdout: "size": 20, 2024-09-14T06:29:49.519 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 600, 2024-09-14T06:29:49.519 INFO:teuthology.orchestra.run.smithi138.stdout: "ops": [ 2024-09-14T06:29:49.519 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:49.519 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:11453 3.1d 3:bcad7303:::rbd_data.112d92027590.0000000000000001:f [read 94208~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:29:49.519 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:25.654375+0000", 2024-09-14T06:29:49.519 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 23.862560431999999, 2024-09-14T06:29:49.519 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052449318000000002, 2024-09-14T06:29:49.519 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:29:49.519 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:29:49.520 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:29:49.520 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:29:49.520 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:29:49.520 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:49.520 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 11453 2024-09-14T06:29:49.520 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:49.520 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:29:49.521 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:49.521 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:29:49.521 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654375+0000", 2024-09-14T06:29:49.521 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:49.521 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:49.521 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:49.521 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:29:49.521 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654375+0000", 2024-09-14T06:29:49.521 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:49.521 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:49.521 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:49.522 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:29:49.522 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654496+0000", 2024-09-14T06:29:49.522 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00012081 2024-09-14T06:29:49.522 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:49.522 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:49.522 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:29:49.522 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654497+0000", 2024-09-14T06:29:49.522 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.8789999999999999e-06 2024-09-14T06:29:49.522 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:49.522 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:49.522 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:29:49.522 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654501+0000", 2024-09-14T06:29:49.522 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.8999999999999999e-06 2024-09-14T06:29:49.522 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:49.523 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:49.523 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:29:49.523 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654505+0000", 2024-09-14T06:29:49.523 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.2229999999999999e-06 2024-09-14T06:29:49.523 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:49.523 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:49.523 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:29:49.523 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.705917+0000", 2024-09-14T06:29:49.523 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.051412501999999999 2024-09-14T06:29:49.523 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:49.523 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:49.523 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:29:49.523 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.705999+0000", 2024-09-14T06:29:49.523 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.1551999999999994e-05 2024-09-14T06:29:49.524 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:49.524 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:49.524 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:29:49.524 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.706824+0000", 2024-09-14T06:29:49.524 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00082545199999999996 2024-09-14T06:29:49.524 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:49.524 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:29:49.524 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:49.524 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:49.524 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:49.524 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:11454 3.1d 3:bcad7303:::rbd_data.112d92027590.0000000000000001:f [read 104448~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:29:49.524 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:25.654510+0000", 2024-09-14T06:29:49.524 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 23.862425642000002, 2024-09-14T06:29:49.525 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052610713000000003, 2024-09-14T06:29:49.525 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:29:49.525 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:29:49.525 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:29:49.525 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:29:49.525 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:29:49.525 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:49.525 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 11454 2024-09-14T06:29:49.525 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:49.525 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:29:49.525 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:49.525 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:29:49.525 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654510+0000", 2024-09-14T06:29:49.526 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:49.526 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:49.526 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:49.526 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:29:49.526 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654510+0000", 2024-09-14T06:29:49.526 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:49.526 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:49.526 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:49.526 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:29:49.526 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654825+0000", 2024-09-14T06:29:49.526 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00031522400000000002 2024-09-14T06:29:49.526 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:49.526 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:49.526 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:29:49.526 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654827+0000", 2024-09-14T06:29:49.527 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.9709999999999998e-06 2024-09-14T06:29:49.527 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:49.527 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:49.527 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:29:49.527 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654832+0000", 2024-09-14T06:29:49.527 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.4099999999999999e-06 2024-09-14T06:29:49.527 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:49.527 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:49.527 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:29:49.527 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654835+0000", 2024-09-14T06:29:49.527 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.14e-06 2024-09-14T06:29:49.527 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:49.527 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:49.527 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:29:49.528 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.706851+0000", 2024-09-14T06:29:49.528 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052015761000000001 2024-09-14T06:29:49.528 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:49.528 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:49.528 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:29:49.528 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.706934+0000", 2024-09-14T06:29:49.528 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.2484999999999994e-05 2024-09-14T06:29:49.528 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:49.528 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:49.528 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:29:49.528 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.707120+0000", 2024-09-14T06:29:49.529 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00018672200000000001 2024-09-14T06:29:49.529 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:49.529 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:29:49.529 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:49.529 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:49.529 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:49.529 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:11455 3.1d 3:bcad7303:::rbd_data.112d92027590.0000000000000001:f [read 114688~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:29:49.529 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:25.654841+0000", 2024-09-14T06:29:49.529 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 23.862094313, 2024-09-14T06:29:49.529 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052579898, 2024-09-14T06:29:49.529 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:29:49.529 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:29:49.529 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:29:49.529 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:29:49.530 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:29:49.530 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:49.530 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 11455 2024-09-14T06:29:49.530 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:49.530 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:29:49.530 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:49.530 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:29:49.530 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654841+0000", 2024-09-14T06:29:49.530 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:49.530 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:49.530 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:49.530 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:29:49.530 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654841+0000", 2024-09-14T06:29:49.530 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:49.531 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:49.531 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:49.531 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:29:49.531 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654841+0000", 2024-09-14T06:29:49.531 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.7300000000000002e-07 2024-09-14T06:29:49.531 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:49.531 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:49.531 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:29:49.531 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654845+0000", 2024-09-14T06:29:49.531 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.0520000000000003e-06 2024-09-14T06:29:49.531 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:49.532 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:49.532 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:29:49.532 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654849+0000", 2024-09-14T06:29:49.532 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.0459999999999999e-06 2024-09-14T06:29:49.532 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:49.532 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:49.532 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:29:49.532 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654852+0000", 2024-09-14T06:29:49.532 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.9830000000000001e-06 2024-09-14T06:29:49.532 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:49.533 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:49.533 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:29:49.533 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.707159+0000", 2024-09-14T06:29:49.533 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052306826000000001 2024-09-14T06:29:49.533 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:49.533 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:49.534 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:29:49.534 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.707242+0000", 2024-09-14T06:29:49.534 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.2595000000000005e-05 2024-09-14T06:29:49.534 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:49.534 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:49.534 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:29:49.535 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.707421+0000", 2024-09-14T06:29:49.535 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.000179123 2024-09-14T06:29:49.535 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:49.535 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:29:49.535 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:49.535 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:49.535 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:49.535 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:11456 3.1d 3:bcad7303:::rbd_data.112d92027590.0000000000000001:f [read 124928~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:29:49.536 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:25.654856+0000", 2024-09-14T06:29:49.536 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 23.862079467000001, 2024-09-14T06:29:49.536 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052801580000000001, 2024-09-14T06:29:49.536 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:29:49.536 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:29:49.536 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:29:49.536 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:29:49.537 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:29:49.537 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:49.537 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 11456 2024-09-14T06:29:49.537 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:49.537 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:29:49.538 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:49.538 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:29:49.538 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654856+0000", 2024-09-14T06:29:49.538 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:49.538 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:49.538 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:49.539 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:29:49.539 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654856+0000", 2024-09-14T06:29:49.539 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:49.539 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:49.539 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:49.539 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:29:49.540 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654856+0000", 2024-09-14T06:29:49.540 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.29e-07 2024-09-14T06:29:49.540 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:49.540 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:49.540 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:29:49.541 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654857+0000", 2024-09-14T06:29:49.541 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.187e-06 2024-09-14T06:29:49.541 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:49.541 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:49.541 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:29:49.541 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654861+0000", 2024-09-14T06:29:49.541 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.3850000000000002e-06 2024-09-14T06:29:49.541 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:49.542 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:49.542 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:29:49.542 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654863+0000", 2024-09-14T06:29:49.542 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.9500000000000001e-06 2024-09-14T06:29:49.542 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:49.542 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:49.542 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:29:49.542 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.707456+0000", 2024-09-14T06:29:49.542 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052592434 2024-09-14T06:29:49.543 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:49.543 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:49.543 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:29:49.543 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.707516+0000", 2024-09-14T06:29:49.543 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.0167999999999997e-05 2024-09-14T06:29:49.543 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:49.543 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:49.543 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:29:49.543 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.707657+0000", 2024-09-14T06:29:49.543 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00014122700000000001 2024-09-14T06:29:49.543 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:49.543 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:29:49.543 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:49.543 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:49.544 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:49.544 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:11457 3.1d 3:bcad7303:::rbd_data.112d92027590.0000000000000001:f [read 135168~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:29:49.544 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:25.654867+0000", 2024-09-14T06:29:49.544 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 23.862068581999999, 2024-09-14T06:29:49.544 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052921902, 2024-09-14T06:29:49.544 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:29:49.544 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:29:49.544 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:29:49.544 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:29:49.544 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:29:49.544 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:49.544 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 11457 2024-09-14T06:29:49.544 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:49.544 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:29:49.545 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:49.545 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:29:49.545 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654867+0000", 2024-09-14T06:29:49.545 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:49.545 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:49.545 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:49.545 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:29:49.545 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654867+0000", 2024-09-14T06:29:49.545 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:49.545 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:49.545 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:49.546 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:29:49.546 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654972+0000", 2024-09-14T06:29:49.546 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00010584 2024-09-14T06:29:49.546 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:49.546 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:49.546 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:29:49.546 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654975+0000", 2024-09-14T06:29:49.546 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.1169999999999998e-06 2024-09-14T06:29:49.546 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:49.546 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:49.546 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:29:49.546 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654979+0000", 2024-09-14T06:29:49.547 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.1960000000000001e-06 2024-09-14T06:29:49.547 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:49.547 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:49.547 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:29:49.547 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654982+0000", 2024-09-14T06:29:49.547 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.2990000000000001e-06 2024-09-14T06:29:49.547 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:49.547 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:49.547 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:29:49.547 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.707684+0000", 2024-09-14T06:29:49.547 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052701979000000003 2024-09-14T06:29:49.547 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:49.547 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:49.547 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:29:49.548 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.707728+0000", 2024-09-14T06:29:49.548 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.3529999999999998e-05 2024-09-14T06:29:49.548 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:49.548 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:49.548 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:29:49.548 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.707789+0000", 2024-09-14T06:29:49.548 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.0940999999999998e-05 2024-09-14T06:29:49.548 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:49.548 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:29:49.548 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:49.548 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:49.548 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:49.548 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:11458 3.1d 3:bcad7303:::rbd_data.112d92027590.0000000000000001:f [read 145408~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:29:49.548 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:25.654987+0000", 2024-09-14T06:29:49.549 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 23.861948183999999, 2024-09-14T06:29:49.549 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.05291655, 2024-09-14T06:29:49.549 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:29:49.549 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:29:49.549 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:29:49.549 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:29:49.549 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:29:49.549 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:49.549 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 11458 2024-09-14T06:29:49.549 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:49.549 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:29:49.549 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:49.550 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:29:49.550 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654987+0000", 2024-09-14T06:29:49.550 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:49.550 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:49.550 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:49.550 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:29:49.550 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654987+0000", 2024-09-14T06:29:49.550 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:49.550 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:49.550 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:49.550 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:29:49.550 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655077+0000", 2024-09-14T06:29:49.550 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.0253000000000005e-05 2024-09-14T06:29:49.551 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:49.551 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:49.551 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:29:49.551 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655079+0000", 2024-09-14T06:29:49.551 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.9989999999999998e-06 2024-09-14T06:29:49.551 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:49.551 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:49.551 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:29:49.551 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655083+0000", 2024-09-14T06:29:49.551 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.9280000000000003e-06 2024-09-14T06:29:49.551 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:49.551 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:49.551 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:29:49.551 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655086+0000", 2024-09-14T06:29:49.551 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.056e-06 2024-09-14T06:29:49.551 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:49.552 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:49.552 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:29:49.552 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.707805+0000", 2024-09-14T06:29:49.552 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052718627999999997 2024-09-14T06:29:49.552 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:49.552 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:49.552 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:29:49.552 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.707848+0000", 2024-09-14T06:29:49.552 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.2728999999999997e-05 2024-09-14T06:29:49.552 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:49.552 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:49.552 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:29:49.552 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.707904+0000", 2024-09-14T06:29:49.553 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.5957e-05 2024-09-14T06:29:49.553 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:49.553 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:29:49.553 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:49.553 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:49.553 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:49.553 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:11459 3.1d 3:bcad7303:::rbd_data.112d92027590.0000000000000001:f [read 155648~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:29:49.553 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:25.655091+0000", 2024-09-14T06:29:49.553 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 23.861844175999998, 2024-09-14T06:29:49.553 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052944242000000002, 2024-09-14T06:29:49.553 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:29:49.553 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:29:49.553 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:29:49.554 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:29:49.554 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:29:49.554 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:49.554 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 11459 2024-09-14T06:29:49.554 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:49.554 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:29:49.554 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:49.554 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:29:49.554 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655091+0000", 2024-09-14T06:29:49.554 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:49.554 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:49.555 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:49.555 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:29:49.555 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655091+0000", 2024-09-14T06:29:49.555 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:49.555 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:49.555 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:49.555 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:29:49.555 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655237+0000", 2024-09-14T06:29:49.555 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00014570100000000001 2024-09-14T06:29:49.555 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:49.555 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:49.555 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:29:49.555 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655239+0000", 2024-09-14T06:29:49.555 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.8670000000000001e-06 2024-09-14T06:29:49.556 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:49.556 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:49.556 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:29:49.556 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655243+0000", 2024-09-14T06:29:49.556 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.9829999999999998e-06 2024-09-14T06:29:49.556 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:49.556 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:49.556 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:29:49.556 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655246+0000", 2024-09-14T06:29:49.556 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.1540000000000002e-06 2024-09-14T06:29:49.556 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:49.556 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:49.556 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:29:49.556 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.707934+0000", 2024-09-14T06:29:49.557 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052688286000000001 2024-09-14T06:29:49.557 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:49.557 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:49.557 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:29:49.557 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.707977+0000", 2024-09-14T06:29:49.557 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.3291999999999998e-05 2024-09-14T06:29:49.557 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:49.557 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:49.557 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:29:49.557 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.708035+0000", 2024-09-14T06:29:49.557 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.7958999999999999e-05 2024-09-14T06:29:49.557 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:49.557 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:29:49.558 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:49.558 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:49.558 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:49.558 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:11460 3.1d 3:bcad7303:::rbd_data.112d92027590.0000000000000001:f [read 165888~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:29:49.558 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:25.655250+0000", 2024-09-14T06:29:49.558 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 23.861684878999998, 2024-09-14T06:29:49.558 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052910698999999999, 2024-09-14T06:29:49.558 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:29:49.558 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:29:49.558 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:29:49.558 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:29:49.558 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:29:49.559 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:49.559 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 11460 2024-09-14T06:29:49.559 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:49.559 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:29:49.559 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:49.559 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:29:49.559 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655250+0000", 2024-09-14T06:29:49.559 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:49.559 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:49.559 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:49.559 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:29:49.559 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655250+0000", 2024-09-14T06:29:49.559 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:49.559 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:49.560 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:49.560 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:29:49.560 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655356+0000", 2024-09-14T06:29:49.560 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00010535900000000001 2024-09-14T06:29:49.560 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:49.560 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:49.560 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:29:49.560 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655357+0000", 2024-09-14T06:29:49.561 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.8080000000000001e-06 2024-09-14T06:29:49.561 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:49.561 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:49.561 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:29:49.561 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655361+0000", 2024-09-14T06:29:49.561 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.861e-06 2024-09-14T06:29:49.561 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:49.561 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:49.561 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:29:49.561 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655364+0000", 2024-09-14T06:29:49.562 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.1089999999999998e-06 2024-09-14T06:29:49.562 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:49.562 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:49.562 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:29:49.562 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.708065+0000", 2024-09-14T06:29:49.562 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052700595000000003 2024-09-14T06:29:49.563 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:49.563 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:49.563 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:29:49.563 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.708108+0000", 2024-09-14T06:29:49.563 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.2484999999999997e-05 2024-09-14T06:29:49.563 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:49.563 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:49.563 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:29:49.563 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.708161+0000", 2024-09-14T06:29:49.563 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.3482000000000001e-05 2024-09-14T06:29:49.563 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:49.563 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:29:49.564 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:49.564 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:49.564 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:49.564 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:11461 3.1d 3:bcad7303:::rbd_data.112d92027590.0000000000000001:f [read 176128~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:29:49.564 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:25.655370+0000", 2024-09-14T06:29:49.564 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 23.861565596999998, 2024-09-14T06:29:49.564 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052915141999999998, 2024-09-14T06:29:49.564 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:29:49.564 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:29:49.564 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:29:49.564 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:29:49.564 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:29:49.564 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:49.565 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 11461 2024-09-14T06:29:49.565 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:49.565 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:29:49.565 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:49.565 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:29:49.565 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655370+0000", 2024-09-14T06:29:49.565 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:49.565 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:49.565 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:49.565 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:29:49.565 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655370+0000", 2024-09-14T06:29:49.566 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:49.566 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:49.566 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:49.566 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:29:49.566 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655394+0000", 2024-09-14T06:29:49.566 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.4553999999999999e-05 2024-09-14T06:29:49.566 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:49.566 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:49.566 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:29:49.566 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655396+0000", 2024-09-14T06:29:49.566 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.8950000000000001e-06 2024-09-14T06:29:49.566 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:49.566 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:49.567 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:29:49.567 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655401+0000", 2024-09-14T06:29:49.567 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.0089999999999998e-06 2024-09-14T06:29:49.567 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:49.567 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:49.567 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:29:49.567 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655404+0000", 2024-09-14T06:29:49.567 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.3400000000000002e-06 2024-09-14T06:29:49.567 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:49.567 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:49.567 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:29:49.567 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.708186+0000", 2024-09-14T06:29:49.567 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052781726000000001 2024-09-14T06:29:49.567 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:49.568 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:49.568 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:29:49.568 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.708229+0000", 2024-09-14T06:29:49.568 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.2670999999999999e-05 2024-09-14T06:29:49.568 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:49.568 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:49.568 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:29:49.568 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.708285+0000", 2024-09-14T06:29:49.568 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.5946999999999998e-05 2024-09-14T06:29:49.568 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:49.568 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:29:49.568 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:49.568 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:49.569 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:49.569 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:11462 3.1d 3:bcad7303:::rbd_data.112d92027590.0000000000000001:f [read 186368~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:29:49.569 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:25.655410+0000", 2024-09-14T06:29:49.569 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 23.861525119, 2024-09-14T06:29:49.569 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052995258000000003, 2024-09-14T06:29:49.569 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:29:49.569 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:29:49.569 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:29:49.569 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:29:49.569 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:29:49.569 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:49.569 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 11462 2024-09-14T06:29:49.569 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:49.569 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:29:49.570 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:49.570 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:29:49.570 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655410+0000", 2024-09-14T06:29:49.570 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:49.570 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:49.570 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:49.570 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:29:49.570 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655410+0000", 2024-09-14T06:29:49.570 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:49.570 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:49.570 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:49.570 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:29:49.570 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655625+0000", 2024-09-14T06:29:49.571 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00021442600000000001 2024-09-14T06:29:49.571 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:49.571 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:49.571 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:29:49.571 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655626+0000", 2024-09-14T06:29:49.571 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.815e-06 2024-09-14T06:29:49.571 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:49.571 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:49.571 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:29:49.571 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655631+0000", 2024-09-14T06:29:49.571 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.651e-06 2024-09-14T06:29:49.571 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:49.571 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:49.572 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:29:49.572 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655634+0000", 2024-09-14T06:29:49.572 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.2449999999999998e-06 2024-09-14T06:29:49.572 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:49.572 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:49.572 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:29:49.572 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.708312+0000", 2024-09-14T06:29:49.572 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052677412 2024-09-14T06:29:49.572 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:49.573 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:49.573 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:29:49.573 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.708352+0000", 2024-09-14T06:29:49.573 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.0729999999999998e-05 2024-09-14T06:29:49.573 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:49.573 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:49.573 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:29:49.573 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.708405+0000", 2024-09-14T06:29:49.573 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.2979000000000002e-05 2024-09-14T06:29:49.573 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:49.573 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:29:49.573 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:49.573 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:49.574 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:49.574 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:11463 3.1d 3:bcad7303:::rbd_data.112d92027590.0000000000000001:f [read 196608~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:29:49.574 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:25.655639+0000", 2024-09-14T06:29:49.574 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 23.861295973000001, 2024-09-14T06:29:49.574 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052888859000000003, 2024-09-14T06:29:49.574 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:29:49.574 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:29:49.574 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:29:49.574 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:29:49.574 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:29:49.574 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:49.575 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 11463 2024-09-14T06:29:49.575 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:49.575 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:29:49.575 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:49.575 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:29:49.575 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655639+0000", 2024-09-14T06:29:49.575 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:49.575 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:49.575 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:49.575 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:29:49.575 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655639+0000", 2024-09-14T06:29:49.575 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:49.575 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:49.576 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:49.576 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:29:49.576 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655640+0000", 2024-09-14T06:29:49.576 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.08e-07 2024-09-14T06:29:49.576 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:49.576 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:49.576 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:29:49.576 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655641+0000", 2024-09-14T06:29:49.576 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.156e-06 2024-09-14T06:29:49.576 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:49.576 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:49.576 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:29:49.576 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655648+0000", 2024-09-14T06:29:49.576 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.8460000000000002e-06 2024-09-14T06:29:49.577 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:49.577 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:49.577 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:29:49.577 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655651+0000", 2024-09-14T06:29:49.577 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.0079999999999998e-06 2024-09-14T06:29:49.577 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:49.577 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:49.577 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:29:49.577 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.708429+0000", 2024-09-14T06:29:49.577 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052778459 2024-09-14T06:29:49.577 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:49.577 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:49.577 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:29:49.578 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.708473+0000", 2024-09-14T06:29:49.578 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.3782999999999997e-05 2024-09-14T06:29:49.578 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:49.578 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:49.578 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:29:49.578 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.708528+0000", 2024-09-14T06:29:49.578 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.5198999999999998e-05 2024-09-14T06:29:49.578 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:49.578 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:29:49.578 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:49.578 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:49.578 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:49.579 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:11464 3.1d 3:bcad7303:::rbd_data.112d92027590.0000000000000001:f [read 206848~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:29:49.579 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:25.655654+0000", 2024-09-14T06:29:49.579 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 23.861281104, 2024-09-14T06:29:49.579 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052981845999999999, 2024-09-14T06:29:49.579 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:29:49.579 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:29:49.579 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:29:49.579 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:29:49.579 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:29:49.579 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:49.579 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 11464 2024-09-14T06:29:49.579 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:49.579 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:29:49.580 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:49.580 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:29:49.580 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655654+0000", 2024-09-14T06:29:49.580 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:49.580 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:49.580 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:49.580 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:29:49.580 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655654+0000", 2024-09-14T06:29:49.580 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:49.580 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:49.580 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:49.580 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:29:49.580 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655946+0000", 2024-09-14T06:29:49.580 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00029148200000000003 2024-09-14T06:29:49.581 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:49.581 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:49.581 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:29:49.581 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655948+0000", 2024-09-14T06:29:49.581 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.973e-06 2024-09-14T06:29:49.581 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:49.581 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:49.581 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:29:49.581 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655953+0000", 2024-09-14T06:29:49.581 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.1240000000000004e-06 2024-09-14T06:29:49.581 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:49.581 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:49.581 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:29:49.582 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655956+0000", 2024-09-14T06:29:49.582 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.315e-06 2024-09-14T06:29:49.582 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:49.582 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:49.582 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:29:49.582 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.708543+0000", 2024-09-14T06:29:49.582 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052587416999999997 2024-09-14T06:29:49.582 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:49.582 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:49.582 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:29:49.582 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.708584+0000", 2024-09-14T06:29:49.583 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.1004000000000002e-05 2024-09-14T06:29:49.583 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:49.583 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:49.583 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:29:49.583 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.708636+0000", 2024-09-14T06:29:49.583 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.1530999999999998e-05 2024-09-14T06:29:49.583 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:49.583 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:29:49.583 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:49.583 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:49.583 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:49.583 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:11465 3.1d 3:bcad7303:::rbd_data.112d92027590.0000000000000001:f [read 217088~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:29:49.583 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:25.655961+0000", 2024-09-14T06:29:49.583 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 23.860973951999998, 2024-09-14T06:29:49.584 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052798682999999999, 2024-09-14T06:29:49.584 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:29:49.584 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:29:49.584 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:29:49.584 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:29:49.584 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:29:49.584 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:49.585 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 11465 2024-09-14T06:29:49.585 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:49.585 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:29:49.585 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:49.585 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:29:49.585 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655961+0000", 2024-09-14T06:29:49.585 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:49.585 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:49.585 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:49.585 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:29:49.585 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655961+0000", 2024-09-14T06:29:49.585 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:49.585 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:49.586 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:49.586 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:29:49.586 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.656114+0000", 2024-09-14T06:29:49.586 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.000152745 2024-09-14T06:29:49.586 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:49.586 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:49.586 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:29:49.586 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.656116+0000", 2024-09-14T06:29:49.586 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.804e-06 2024-09-14T06:29:49.586 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:49.586 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:49.586 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:29:49.586 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.656120+0000", 2024-09-14T06:29:49.587 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.4700000000000004e-06 2024-09-14T06:29:49.587 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:49.587 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:49.587 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:29:49.587 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.656123+0000", 2024-09-14T06:29:49.587 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.1269999999999999e-06 2024-09-14T06:29:49.587 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:49.587 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:49.587 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:29:49.587 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.708660+0000", 2024-09-14T06:29:49.587 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052536919000000001 2024-09-14T06:29:49.587 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:49.587 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:49.587 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:29:49.588 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.708703+0000", 2024-09-14T06:29:49.588 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.3077000000000001e-05 2024-09-14T06:29:49.588 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:49.588 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:49.588 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:29:49.588 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.708760+0000", 2024-09-14T06:29:49.588 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.6541e-05 2024-09-14T06:29:49.588 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:49.588 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:29:49.588 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:49.588 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:49.588 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:49.589 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:11466 3.1d 3:bcad7303:::rbd_data.112d92027590.0000000000000001:f [read 227328~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:29:49.589 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:25.656129+0000", 2024-09-14T06:29:49.589 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 23.860806564000001, 2024-09-14T06:29:49.589 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052750987999999999, 2024-09-14T06:29:49.589 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:29:49.589 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:29:49.589 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:29:49.589 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:29:49.589 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:29:49.589 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:49.589 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 11466 2024-09-14T06:29:49.589 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:49.589 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:29:49.589 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:49.590 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:29:49.590 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.656129+0000", 2024-09-14T06:29:49.590 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:49.590 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:49.590 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:49.590 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:29:49.590 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.656129+0000", 2024-09-14T06:29:49.590 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:49.590 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:49.590 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:49.590 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:29:49.590 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.656347+0000", 2024-09-14T06:29:49.590 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.000218054 2024-09-14T06:29:49.590 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:49.591 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:49.591 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:29:49.591 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.656349+0000", 2024-09-14T06:29:49.591 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.9199999999999998e-06 2024-09-14T06:29:49.591 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:49.591 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:49.591 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:29:49.591 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.656353+0000", 2024-09-14T06:29:49.592 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.9210000000000002e-06 2024-09-14T06:29:49.592 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:49.592 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:49.592 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:29:49.592 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.656356+0000", 2024-09-14T06:29:49.592 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.7670000000000001e-06 2024-09-14T06:29:49.592 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:49.592 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:49.592 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:29:49.592 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.708787+0000", 2024-09-14T06:29:49.592 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052430840999999999 2024-09-14T06:29:49.593 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:49.593 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:49.593 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:29:49.593 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.708827+0000", 2024-09-14T06:29:49.593 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.9538e-05 2024-09-14T06:29:49.593 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:49.593 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:49.593 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:29:49.593 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.708880+0000", 2024-09-14T06:29:49.593 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.2947e-05 2024-09-14T06:29:49.593 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:49.593 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:29:49.594 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:49.594 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:49.594 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:49.594 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:11467 3.1d 3:bcad7303:::rbd_data.112d92027590.0000000000000001:f [read 237568~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:29:49.594 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:25.656361+0000", 2024-09-14T06:29:49.594 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 23.860573916, 2024-09-14T06:29:49.594 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052643898000000001, 2024-09-14T06:29:49.594 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:29:49.594 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:29:49.594 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:29:49.594 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:29:49.594 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:29:49.595 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:49.595 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 11467 2024-09-14T06:29:49.595 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:49.595 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:29:49.595 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:49.595 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:29:49.595 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.656361+0000", 2024-09-14T06:29:49.595 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:49.595 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:49.595 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:49.595 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:29:49.595 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.656361+0000", 2024-09-14T06:29:49.595 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:49.595 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:49.596 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:49.596 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:29:49.596 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.656855+0000", 2024-09-14T06:29:49.596 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00049372000000000001 2024-09-14T06:29:49.596 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:49.596 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:49.596 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:29:49.596 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.656859+0000", 2024-09-14T06:29:49.596 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.1300000000000003e-06 2024-09-14T06:29:49.596 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:49.596 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:49.596 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:29:49.596 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.656863+0000", 2024-09-14T06:29:49.596 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.2629999999999997e-06 2024-09-14T06:29:49.597 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:49.597 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:49.597 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:29:49.597 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.656867+0000", 2024-09-14T06:29:49.597 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.3220000000000001e-06 2024-09-14T06:29:49.597 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:49.597 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:49.597 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:29:49.597 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.708903+0000", 2024-09-14T06:29:49.597 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052035855999999998 2024-09-14T06:29:49.597 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:49.597 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:49.597 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:29:49.597 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.708949+0000", 2024-09-14T06:29:49.597 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.6276999999999998e-05 2024-09-14T06:29:49.598 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:49.598 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:49.598 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:29:49.598 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.709005+0000", 2024-09-14T06:29:49.598 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.6329999999999998e-05 2024-09-14T06:29:49.598 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:49.598 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:29:49.598 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:49.598 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:49.598 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:49.598 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:11479 3.1d 3:bcad7303:::rbd_data.112d92027590.0000000000000001:f [read 360448~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:29:49.598 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:25.658388+0000", 2024-09-14T06:29:49.598 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 23.858547083000001, 2024-09-14T06:29:49.598 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052408920999999997, 2024-09-14T06:29:49.598 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:29:49.598 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:29:49.599 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:29:49.599 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:29:49.599 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:29:49.599 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:49.599 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 11479 2024-09-14T06:29:49.599 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:49.599 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:29:49.599 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:49.599 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:29:49.599 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.658388+0000", 2024-09-14T06:29:49.599 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:49.599 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:49.599 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:49.599 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:29:49.599 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.658388+0000", 2024-09-14T06:29:49.600 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:49.600 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:49.600 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:49.600 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:29:49.600 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.658534+0000", 2024-09-14T06:29:49.600 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.000145613 2024-09-14T06:29:49.600 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:49.600 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:49.600 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:29:49.600 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.658540+0000", 2024-09-14T06:29:49.600 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.6680000000000004e-06 2024-09-14T06:29:49.600 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:49.600 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:49.600 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:29:49.600 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.658548+0000", 2024-09-14T06:29:49.601 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.7980000000000006e-06 2024-09-14T06:29:49.601 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:49.601 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:49.601 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:29:49.601 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.658554+0000", 2024-09-14T06:29:49.601 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.6869999999999999e-06 2024-09-14T06:29:49.601 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:49.601 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:49.601 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:29:49.601 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.710624+0000", 2024-09-14T06:29:49.601 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052070567999999998 2024-09-14T06:29:49.601 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:49.601 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:49.601 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:29:49.601 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.710666+0000", 2024-09-14T06:29:49.602 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.1467000000000001e-05 2024-09-14T06:29:49.602 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:49.602 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:49.602 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:29:49.602 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.710797+0000", 2024-09-14T06:29:49.602 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00013112 2024-09-14T06:29:49.602 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:49.602 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:29:49.602 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:49.602 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:49.602 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:49.602 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:11480 3.1d 3:bcad7303:::rbd_data.112d92027590.0000000000000001:f [read 370688~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:29:49.602 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:25.658563+0000", 2024-09-14T06:29:49.603 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 23.858372152000001, 2024-09-14T06:29:49.603 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052429164, 2024-09-14T06:29:49.603 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:29:49.603 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:29:49.603 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:29:49.603 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:29:49.603 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:29:49.603 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:49.603 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 11480 2024-09-14T06:29:49.603 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:49.603 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:29:49.603 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:49.603 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:29:49.604 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.658563+0000", 2024-09-14T06:29:49.604 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:49.604 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:49.604 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:49.604 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:29:49.604 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.658563+0000", 2024-09-14T06:29:49.604 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:49.604 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:49.604 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:49.604 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:29:49.604 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.658658+0000", 2024-09-14T06:29:49.604 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.5346999999999994e-05 2024-09-14T06:29:49.604 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:49.604 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:49.605 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:29:49.605 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.658662+0000", 2024-09-14T06:29:49.605 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.14e-06 2024-09-14T06:29:49.605 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:49.605 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:49.605 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:29:49.605 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.658669+0000", 2024-09-14T06:29:49.605 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.7379999999999994e-06 2024-09-14T06:29:49.605 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:49.605 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:49.605 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:29:49.605 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.658675+0000", 2024-09-14T06:29:49.605 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.5430000000000001e-06 2024-09-14T06:29:49.605 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:49.606 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:49.606 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:29:49.606 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.710824+0000", 2024-09-14T06:29:49.606 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052148847999999998 2024-09-14T06:29:49.606 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:49.606 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:49.606 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:29:49.606 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.710864+0000", 2024-09-14T06:29:49.606 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.9925999999999997e-05 2024-09-14T06:29:49.606 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:49.606 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:49.606 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:29:49.606 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.710992+0000", 2024-09-14T06:29:49.606 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.000128622 2024-09-14T06:29:49.606 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:49.607 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:29:49.607 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:49.607 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:49.607 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:49.607 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:11481 3.1d 3:bcad7303:::rbd_data.112d92027590.0000000000000001:f [read 380928~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:29:49.607 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:25.658684+0000", 2024-09-14T06:29:49.607 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 23.858251081999999, 2024-09-14T06:29:49.607 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052508565, 2024-09-14T06:29:49.607 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:29:49.607 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:29:49.607 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:29:49.607 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:29:49.607 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:29:49.607 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:49.608 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 11481 2024-09-14T06:29:49.608 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:49.608 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:29:49.608 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:49.608 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:29:49.608 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.658684+0000", 2024-09-14T06:29:49.608 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:49.608 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:49.608 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:49.608 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:29:49.608 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.658684+0000", 2024-09-14T06:29:49.608 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:49.608 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:49.608 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:49.608 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:29:49.609 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.658715+0000", 2024-09-14T06:29:49.609 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.0703000000000001e-05 2024-09-14T06:29:49.609 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:49.609 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:49.609 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:29:49.609 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.658718+0000", 2024-09-14T06:29:49.609 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.0709999999999999e-06 2024-09-14T06:29:49.609 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:49.609 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:49.609 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:29:49.609 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.658725+0000", 2024-09-14T06:29:49.609 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.2339999999999997e-06 2024-09-14T06:29:49.609 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:49.609 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:49.610 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:29:49.610 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.658731+0000", 2024-09-14T06:29:49.610 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.6450000000000002e-06 2024-09-14T06:29:49.610 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:49.610 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:49.610 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:29:49.610 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.711020+0000", 2024-09-14T06:29:49.610 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052288886 2024-09-14T06:29:49.610 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:49.610 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:49.610 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:29:49.610 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.711061+0000", 2024-09-14T06:29:49.610 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.1084000000000002e-05 2024-09-14T06:29:49.610 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:49.611 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:49.611 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:29:49.611 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.711193+0000", 2024-09-14T06:29:49.611 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00013194199999999999 2024-09-14T06:29:49.611 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:49.611 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:29:49.611 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:49.611 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:49.611 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:49.611 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:11482 3.1d 3:bcad7303:::rbd_data.112d92027590.0000000000000001:f [read 391168~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:29:49.611 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:25.658740+0000", 2024-09-14T06:29:49.611 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 23.858195617, 2024-09-14T06:29:49.611 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052637015000000002, 2024-09-14T06:29:49.612 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:29:49.612 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:29:49.612 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:29:49.612 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:29:49.612 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:29:49.612 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:49.612 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 11482 2024-09-14T06:29:49.612 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:49.612 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:29:49.612 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:49.612 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:29:49.612 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.658740+0000", 2024-09-14T06:29:49.613 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:49.613 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:49.613 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:49.613 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:29:49.613 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.658740+0000", 2024-09-14T06:29:49.613 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:49.613 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:49.613 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:49.613 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:29:49.614 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.658880+0000", 2024-09-14T06:29:49.614 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00014036100000000001 2024-09-14T06:29:49.614 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:49.614 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:49.614 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:29:49.614 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.658883+0000", 2024-09-14T06:29:49.614 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.3969999999999998e-06 2024-09-14T06:29:49.614 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:49.614 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:49.614 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:29:49.614 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.658890+0000", 2024-09-14T06:29:49.614 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.1500000000000002e-06 2024-09-14T06:29:49.614 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:49.615 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:49.615 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:29:49.615 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.658897+0000", 2024-09-14T06:29:49.615 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.1260000000000003e-06 2024-09-14T06:29:49.615 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:49.615 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:49.615 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:29:49.615 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.711207+0000", 2024-09-14T06:29:49.615 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052310777000000003 2024-09-14T06:29:49.615 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:49.616 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:49.616 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:29:49.616 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.711249+0000", 2024-09-14T06:29:49.616 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.1786999999999998e-05 2024-09-14T06:29:49.616 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:49.616 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:49.616 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:29:49.616 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.711377+0000", 2024-09-14T06:29:49.616 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.000127417 2024-09-14T06:29:49.616 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:49.616 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:29:49.616 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:49.616 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:49.616 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:49.617 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:11483 3.1d 3:bcad7303:::rbd_data.112d92027590.0000000000000001:f [read 401408~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:29:49.617 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:25.658906+0000", 2024-09-14T06:29:49.617 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 23.858028832999999, 2024-09-14T06:29:49.617 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052595109000000001, 2024-09-14T06:29:49.617 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:29:49.617 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:29:49.617 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:29:49.617 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:29:49.617 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:29:49.617 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:49.617 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 11483 2024-09-14T06:29:49.617 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:49.617 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:29:49.617 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:49.618 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:29:49.618 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.658906+0000", 2024-09-14T06:29:49.618 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:49.618 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:49.618 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:49.618 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:29:49.618 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.658906+0000", 2024-09-14T06:29:49.618 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:49.618 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:49.618 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:49.618 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:29:49.618 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.664985+0000", 2024-09-14T06:29:49.618 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.0060787250000000001 2024-09-14T06:29:49.618 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:49.618 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:49.619 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:29:49.619 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.664991+0000", 2024-09-14T06:29:49.619 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.7030000000000003e-06 2024-09-14T06:29:49.619 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:49.619 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:49.619 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:29:49.619 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.664999+0000", 2024-09-14T06:29:49.619 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.4430000000000003e-06 2024-09-14T06:29:49.619 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:49.619 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:49.619 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:29:49.619 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.665006+0000", 2024-09-14T06:29:49.619 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.0840000000000003e-06 2024-09-14T06:29:49.619 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:49.620 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:49.620 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:29:49.620 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.711402+0000", 2024-09-14T06:29:49.620 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046395714999999997 2024-09-14T06:29:49.620 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:49.620 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:49.620 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:29:49.620 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.711443+0000", 2024-09-14T06:29:49.620 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.0911999999999997e-05 2024-09-14T06:29:49.620 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:49.620 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:49.620 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:29:49.620 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.711501+0000", 2024-09-14T06:29:49.620 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.8526999999999997e-05 2024-09-14T06:29:49.621 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:49.621 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:29:49.621 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:49.621 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:49.621 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:29:49.621 INFO:teuthology.orchestra.run.smithi138.stdout:} 2024-09-14T06:29:49.623 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-14T06:29:49.623 INFO:tasks.ceph.osd.6.smithi138.stderr:2024-09-14T06:29:49.621+0000 7f8cd8849640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33620) UID: 0 2024-09-14T06:29:49.723 INFO:tasks.ceph.osd.0.smithi094.stderr:2024-09-14T06:29:49.722+0000 7f07c9333640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33636) UID: 0 2024-09-14T06:29:49.769 INFO:teuthology.orchestra.run.smithi138.stdout:{ 2024-09-14T06:29:49.770 INFO:teuthology.orchestra.run.smithi138.stdout: "ops": [], 2024-09-14T06:29:49.770 INFO:teuthology.orchestra.run.smithi138.stdout: "num_ops": 0 2024-09-14T06:29:49.770 INFO:teuthology.orchestra.run.smithi138.stdout:} 2024-09-14T06:29:49.779 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-14T06:29:49.824 INFO:tasks.ceph.osd.3.smithi094.stderr:2024-09-14T06:29:49.823+0000 7feeeafe8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33655) UID: 0 2024-09-14T06:29:49.881 INFO:teuthology.orchestra.run.smithi138.stdout:{ 2024-09-14T06:29:49.882 INFO:teuthology.orchestra.run.smithi138.stdout: "ops": [], 2024-09-14T06:29:49.882 INFO:teuthology.orchestra.run.smithi138.stdout: "complaint_time": 30, 2024-09-14T06:29:49.882 INFO:teuthology.orchestra.run.smithi138.stdout: "num_blocked_ops": 0 2024-09-14T06:29:49.882 INFO:teuthology.orchestra.run.smithi138.stdout:} 2024-09-14T06:29:49.890 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-14T06:29:49.925 INFO:tasks.ceph.osd.0.smithi094.stderr:2024-09-14T06:29:49.924+0000 7f07c9333640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33636) UID: 0 2024-09-14T06:29:49.995 INFO:teuthology.orchestra.run.smithi138.stdout:{ 2024-09-14T06:29:49.995 INFO:teuthology.orchestra.run.smithi138.stdout: "size": 20, 2024-09-14T06:29:49.995 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 600, 2024-09-14T06:29:49.995 INFO:teuthology.orchestra.run.smithi138.stdout: "ops": [ 2024-09-14T06:29:49.995 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:49.995 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:1484 3.a 3:57c28b0f:::rbd_data.112d7dfbd766.000000000000000a:5 [read 208896~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-14T06:29:49.995 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:08.628246+0000", 2024-09-14T06:29:49.995 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 41.364984133999997, 2024-09-14T06:29:49.995 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.018096672000000001, 2024-09-14T06:29:49.995 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:29:49.995 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:29:49.995 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:29:49.996 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:29:49.996 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:29:49.996 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:49.996 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 1484 2024-09-14T06:29:49.996 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:49.996 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:29:49.996 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:49.996 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:29:49.996 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.628246+0000", 2024-09-14T06:29:49.996 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:49.996 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:49.996 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:49.996 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:29:49.996 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.628246+0000", 2024-09-14T06:29:49.997 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:49.997 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:49.997 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:49.997 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:29:49.997 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.628246+0000", 2024-09-14T06:29:49.997 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.79e-07 2024-09-14T06:29:49.997 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:49.997 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:49.997 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:29:49.997 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.628247+0000", 2024-09-14T06:29:49.997 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.5179999999999999e-06 2024-09-14T06:29:49.997 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:49.997 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:49.997 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:29:49.998 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.628252+0000", 2024-09-14T06:29:49.998 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.0139999999999998e-06 2024-09-14T06:29:49.998 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:49.998 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:49.998 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:29:49.998 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.628257+0000", 2024-09-14T06:29:49.998 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.2429999999999999e-06 2024-09-14T06:29:49.998 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:49.998 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:49.998 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:29:49.998 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.646180+0000", 2024-09-14T06:29:49.998 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.017923136999999999 2024-09-14T06:29:49.998 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:49.998 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:49.999 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:29:49.999 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.646251+0000", 2024-09-14T06:29:49.999 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.1285000000000006e-05 2024-09-14T06:29:49.999 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:49.999 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:49.999 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:29:49.999 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.646342+0000", 2024-09-14T06:29:49.999 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.1195999999999999e-05 2024-09-14T06:29:49.999 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:49.999 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:29:49.999 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:49.999 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:49.999 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:49.999 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:1485 3.a 3:57c28b0f:::rbd_data.112d7dfbd766.000000000000000a:5 [read 225280~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-14T06:29:49.999 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:08.628268+0000", 2024-09-14T06:29:50.000 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 41.364961653000002, 2024-09-14T06:29:50.000 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.018257005999999999, 2024-09-14T06:29:50.000 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:29:50.000 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:29:50.000 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:29:50.000 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:29:50.000 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:29:50.000 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:50.000 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 1485 2024-09-14T06:29:50.000 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:50.000 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:29:50.000 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:50.000 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:29:50.000 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.628268+0000", 2024-09-14T06:29:50.001 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:50.001 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:50.001 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:50.001 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:29:50.001 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.628268+0000", 2024-09-14T06:29:50.001 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:50.001 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:50.001 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:50.001 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:29:50.001 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.628269+0000", 2024-09-14T06:29:50.001 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.6800000000000001e-07 2024-09-14T06:29:50.001 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:50.001 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:50.001 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:29:50.002 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.628270+0000", 2024-09-14T06:29:50.002 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.8449999999999999e-06 2024-09-14T06:29:50.002 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:50.002 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:50.002 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:29:50.002 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.628276+0000", 2024-09-14T06:29:50.002 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.998e-06 2024-09-14T06:29:50.002 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:50.002 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:50.002 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:29:50.002 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.628281+0000", 2024-09-14T06:29:50.002 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.3120000000000004e-06 2024-09-14T06:29:50.002 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:50.002 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:50.002 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:29:50.003 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.646371+0000", 2024-09-14T06:29:50.003 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.018090694000000001 2024-09-14T06:29:50.003 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:50.003 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:50.003 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:29:50.003 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.646439+0000", 2024-09-14T06:29:50.003 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.7182999999999997e-05 2024-09-14T06:29:50.003 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:50.003 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:50.003 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:29:50.003 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.646525+0000", 2024-09-14T06:29:50.003 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.6506000000000002e-05 2024-09-14T06:29:50.003 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:50.003 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:29:50.004 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:50.004 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:50.004 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:50.004 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:1486 3.a 3:57c28b0f:::rbd_data.112d7dfbd766.000000000000000a:5 [read 241664~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-14T06:29:50.004 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:08.628288+0000", 2024-09-14T06:29:50.004 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 41.364941766999998, 2024-09-14T06:29:50.004 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.018416895999999999, 2024-09-14T06:29:50.004 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:29:50.004 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:29:50.004 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:29:50.004 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:29:50.004 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:29:50.004 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:50.004 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 1486 2024-09-14T06:29:50.004 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:50.005 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:29:50.005 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:50.005 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:29:50.005 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.628288+0000", 2024-09-14T06:29:50.005 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:50.005 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:50.005 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:50.005 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:29:50.005 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.628288+0000", 2024-09-14T06:29:50.005 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:50.005 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:50.005 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:50.005 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:29:50.005 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.628288+0000", 2024-09-14T06:29:50.006 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.9499999999999998e-07 2024-09-14T06:29:50.006 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:50.006 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:50.006 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:29:50.006 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.628290+0000", 2024-09-14T06:29:50.006 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.6840000000000001e-06 2024-09-14T06:29:50.006 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:50.006 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:50.006 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:29:50.006 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.628295+0000", 2024-09-14T06:29:50.006 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.3059999999999997e-06 2024-09-14T06:29:50.006 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:50.006 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:50.006 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:29:50.006 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.628300+0000", 2024-09-14T06:29:50.007 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.2690000000000001e-06 2024-09-14T06:29:50.007 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:50.007 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:50.007 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:29:50.007 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.646553+0000", 2024-09-14T06:29:50.007 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.018253015000000001 2024-09-14T06:29:50.007 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:50.007 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:50.007 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:29:50.007 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.646626+0000", 2024-09-14T06:29:50.007 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.2939000000000001e-05 2024-09-14T06:29:50.007 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:50.007 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:50.007 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:29:50.007 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.646705+0000", 2024-09-14T06:29:50.008 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.9388000000000006e-05 2024-09-14T06:29:50.008 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:50.008 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:29:50.008 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:50.008 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:50.008 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:50.008 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:1487 3.a 3:57c28b0f:::rbd_data.112d7dfbd766.000000000000000a:5 [read 258048~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-14T06:29:50.008 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:08.628842+0000", 2024-09-14T06:29:50.008 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 41.364387323000003, 2024-09-14T06:29:50.008 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.018037741999999999, 2024-09-14T06:29:50.008 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:29:50.008 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:29:50.008 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:29:50.009 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:29:50.009 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:29:50.009 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:50.009 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 1487 2024-09-14T06:29:50.009 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:50.009 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:29:50.009 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:50.009 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:29:50.009 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.628842+0000", 2024-09-14T06:29:50.009 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:50.009 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:50.009 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:50.009 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:29:50.010 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.628842+0000", 2024-09-14T06:29:50.010 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:50.010 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:50.010 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:50.010 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:29:50.010 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.628843+0000", 2024-09-14T06:29:50.010 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.5600000000000005e-07 2024-09-14T06:29:50.010 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:50.010 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:50.011 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:29:50.011 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.628847+0000", 2024-09-14T06:29:50.011 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.394e-06 2024-09-14T06:29:50.011 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:50.011 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:50.011 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:29:50.011 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.628855+0000", 2024-09-14T06:29:50.011 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.0760000000000003e-06 2024-09-14T06:29:50.011 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:50.011 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:50.011 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:29:50.011 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.628868+0000", 2024-09-14T06:29:50.011 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.2860000000000001e-05 2024-09-14T06:29:50.012 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:50.012 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:50.012 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:29:50.012 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.646733+0000", 2024-09-14T06:29:50.012 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.017865387999999999 2024-09-14T06:29:50.012 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:50.012 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:50.012 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:29:50.012 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.646800+0000", 2024-09-14T06:29:50.012 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.6619999999999996e-05 2024-09-14T06:29:50.012 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:50.012 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:50.012 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:29:50.013 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.646880+0000", 2024-09-14T06:29:50.013 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.0648000000000004e-05 2024-09-14T06:29:50.013 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:50.013 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:29:50.013 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:50.013 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:50.013 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:50.013 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:1488 3.a 3:57c28b0f:::rbd_data.112d7dfbd766.000000000000000a:5 [read 274432~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-14T06:29:50.013 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:08.628878+0000", 2024-09-14T06:29:50.013 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 41.364351704999997, 2024-09-14T06:29:50.013 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.018187624999999999, 2024-09-14T06:29:50.013 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:29:50.013 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:29:50.013 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:29:50.014 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:29:50.014 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:29:50.014 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:50.014 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 1488 2024-09-14T06:29:50.014 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:50.014 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:29:50.014 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:50.014 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:29:50.014 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.628878+0000", 2024-09-14T06:29:50.014 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:50.014 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:50.014 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:50.014 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:29:50.015 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.628878+0000", 2024-09-14T06:29:50.015 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:50.015 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:50.015 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:50.015 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:29:50.015 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.628879+0000", 2024-09-14T06:29:50.015 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.58e-07 2024-09-14T06:29:50.015 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:50.015 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:50.015 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:29:50.015 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.628881+0000", 2024-09-14T06:29:50.015 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.0389999999999999e-06 2024-09-14T06:29:50.015 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:50.015 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:50.016 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:29:50.016 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.628887+0000", 2024-09-14T06:29:50.016 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.9429999999999997e-06 2024-09-14T06:29:50.016 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:50.016 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:50.016 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:29:50.016 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.628891+0000", 2024-09-14T06:29:50.016 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.1309999999999999e-06 2024-09-14T06:29:50.016 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:50.016 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:50.016 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:29:50.016 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.646909+0000", 2024-09-14T06:29:50.016 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.018017920999999999 2024-09-14T06:29:50.016 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:50.017 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:50.017 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:29:50.017 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.646984+0000", 2024-09-14T06:29:50.017 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.4989999999999999e-05 2024-09-14T06:29:50.017 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:50.017 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:50.017 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:29:50.017 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.647066+0000", 2024-09-14T06:29:50.017 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.2143000000000003e-05 2024-09-14T06:29:50.017 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:50.017 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:29:50.017 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:50.017 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:50.017 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:50.018 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:1489 3.a 3:57c28b0f:::rbd_data.112d7dfbd766.000000000000000a:5 [read 290816~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-14T06:29:50.018 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:08.628897+0000", 2024-09-14T06:29:50.018 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 41.364333254000002, 2024-09-14T06:29:50.018 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.018341491000000001, 2024-09-14T06:29:50.018 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:29:50.018 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:29:50.018 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:29:50.018 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:29:50.018 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:29:50.018 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:50.018 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 1489 2024-09-14T06:29:50.018 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:50.018 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:29:50.018 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:50.019 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:29:50.019 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.628897+0000", 2024-09-14T06:29:50.019 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:50.019 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:50.019 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:50.019 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:29:50.019 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.628897+0000", 2024-09-14T06:29:50.019 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:50.019 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:50.019 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:50.019 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:29:50.019 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.628897+0000", 2024-09-14T06:29:50.019 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.6800000000000001e-07 2024-09-14T06:29:50.019 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:50.020 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:50.020 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:29:50.020 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.628899+0000", 2024-09-14T06:29:50.020 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.606e-06 2024-09-14T06:29:50.020 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:50.020 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:50.020 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:29:50.020 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.628904+0000", 2024-09-14T06:29:50.020 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.1839999999999998e-06 2024-09-14T06:29:50.020 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:50.020 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:50.020 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:29:50.020 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.628908+0000", 2024-09-14T06:29:50.020 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.0500000000000002e-06 2024-09-14T06:29:50.021 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:50.021 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:50.021 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:29:50.021 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.647093+0000", 2024-09-14T06:29:50.021 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.018184846000000001 2024-09-14T06:29:50.021 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:50.021 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:50.021 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:29:50.021 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.647158+0000", 2024-09-14T06:29:50.021 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.5068999999999997e-05 2024-09-14T06:29:50.021 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:50.021 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:50.022 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:29:50.022 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.647238+0000", 2024-09-14T06:29:50.022 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.0368e-05 2024-09-14T06:29:50.022 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:50.022 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:29:50.022 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:50.022 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:50.022 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:50.022 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:1490 3.a 3:57c28b0f:::rbd_data.112d7dfbd766.000000000000000a:5 [read 307200~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-14T06:29:50.022 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:08.628921+0000", 2024-09-14T06:29:50.022 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 41.364309257000002, 2024-09-14T06:29:50.022 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.018575338, 2024-09-14T06:29:50.022 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:29:50.022 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:29:50.022 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:29:50.023 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:29:50.023 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:29:50.023 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:50.023 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 1490 2024-09-14T06:29:50.023 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:50.023 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:29:50.023 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:50.023 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:29:50.023 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.628921+0000", 2024-09-14T06:29:50.023 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:50.023 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:50.023 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:50.023 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:29:50.023 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.628921+0000", 2024-09-14T06:29:50.024 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:50.024 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:50.024 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:50.024 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:29:50.024 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.628921+0000", 2024-09-14T06:29:50.024 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.39e-07 2024-09-14T06:29:50.024 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:50.024 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:50.024 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:29:50.024 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.628928+0000", 2024-09-14T06:29:50.024 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.81e-06 2024-09-14T06:29:50.024 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:50.024 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:50.025 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:29:50.025 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.628934+0000", 2024-09-14T06:29:50.025 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.4649999999999999e-06 2024-09-14T06:29:50.025 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:50.025 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:50.025 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:29:50.025 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.628939+0000", 2024-09-14T06:29:50.025 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.3730000000000003e-06 2024-09-14T06:29:50.025 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:50.025 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:50.025 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:29:50.025 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.647265+0000", 2024-09-14T06:29:50.025 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.018326499999999999 2024-09-14T06:29:50.026 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:50.026 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:50.026 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:29:50.026 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.647381+0000", 2024-09-14T06:29:50.026 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.000115549 2024-09-14T06:29:50.026 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:50.026 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:50.026 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:29:50.026 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.647496+0000", 2024-09-14T06:29:50.026 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.000115202 2024-09-14T06:29:50.026 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:50.026 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:29:50.026 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:50.027 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:50.027 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:50.027 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:1491 3.a 3:57c28b0f:::rbd_data.112d7dfbd766.000000000000000a:5 [read 323584~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-14T06:29:50.027 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:08.628964+0000", 2024-09-14T06:29:50.027 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 41.364266194000002, 2024-09-14T06:29:50.027 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.018753204999999998, 2024-09-14T06:29:50.027 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:29:50.027 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:29:50.027 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:29:50.027 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:29:50.027 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:29:50.027 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:50.027 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 1491 2024-09-14T06:29:50.028 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:50.028 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:29:50.028 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:50.028 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:29:50.028 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.628964+0000", 2024-09-14T06:29:50.028 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:50.028 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:50.028 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:50.028 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:29:50.028 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.628964+0000", 2024-09-14T06:29:50.028 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:50.028 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:50.028 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:50.028 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:29:50.029 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.628964+0000", 2024-09-14T06:29:50.029 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.3600000000000004e-07 2024-09-14T06:29:50.029 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:50.029 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:50.029 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:29:50.029 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.628966+0000", 2024-09-14T06:29:50.029 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.119e-06 2024-09-14T06:29:50.029 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:50.029 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:50.029 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:29:50.029 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.628972+0000", 2024-09-14T06:29:50.029 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.0929999999999999e-06 2024-09-14T06:29:50.029 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:50.029 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:50.030 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:29:50.030 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.628977+0000", 2024-09-14T06:29:50.030 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.5360000000000003e-06 2024-09-14T06:29:50.030 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:50.030 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:50.030 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:29:50.030 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.647527+0000", 2024-09-14T06:29:50.030 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.018550433000000002 2024-09-14T06:29:50.030 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:50.030 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:50.030 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:29:50.030 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.647614+0000", 2024-09-14T06:29:50.030 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.6757999999999998e-05 2024-09-14T06:29:50.030 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:50.030 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:50.031 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:29:50.031 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.647717+0000", 2024-09-14T06:29:50.031 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00010273 2024-09-14T06:29:50.031 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:50.031 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:29:50.031 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:50.031 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:50.031 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:50.031 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:1492 3.a 3:57c28b0f:::rbd_data.112d7dfbd766.000000000000000a:5 [read 339968~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-14T06:29:50.031 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:08.628983+0000", 2024-09-14T06:29:50.031 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 41.364246510999997, 2024-09-14T06:29:50.031 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.018937253000000001, 2024-09-14T06:29:50.031 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:29:50.031 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:29:50.032 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:29:50.032 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:29:50.032 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:29:50.032 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:50.032 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 1492 2024-09-14T06:29:50.032 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:50.032 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:29:50.032 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:50.032 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:29:50.032 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.628983+0000", 2024-09-14T06:29:50.032 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:50.032 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:50.032 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:50.032 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:29:50.032 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.628983+0000", 2024-09-14T06:29:50.033 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:50.033 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:50.033 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:50.033 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:29:50.033 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.628984+0000", 2024-09-14T06:29:50.033 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.0100000000000001e-07 2024-09-14T06:29:50.033 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:50.033 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:50.033 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:29:50.033 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.628985+0000", 2024-09-14T06:29:50.033 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.561e-06 2024-09-14T06:29:50.033 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:50.033 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:50.033 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:29:50.034 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.628990+0000", 2024-09-14T06:29:50.034 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.1540000000000001e-06 2024-09-14T06:29:50.034 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:50.034 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:50.034 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:29:50.034 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.628994+0000", 2024-09-14T06:29:50.034 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.8890000000000004e-06 2024-09-14T06:29:50.034 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:50.034 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:50.034 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:29:50.034 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.647745+0000", 2024-09-14T06:29:50.034 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.018751213999999999 2024-09-14T06:29:50.034 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:50.034 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:50.034 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:29:50.035 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.647824+0000", 2024-09-14T06:29:50.035 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.8331999999999995e-05 2024-09-14T06:29:50.035 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:50.035 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:50.035 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:29:50.035 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.647921+0000", 2024-09-14T06:29:50.035 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.6801999999999999e-05 2024-09-14T06:29:50.035 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:50.035 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:29:50.035 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:50.035 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:50.035 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:50.035 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:1493 3.a 3:57c28b0f:::rbd_data.112d7dfbd766.000000000000000a:5 [read 356352~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-14T06:29:50.035 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:08.628999+0000", 2024-09-14T06:29:50.035 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 41.364230595999999, 2024-09-14T06:29:50.036 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.019127128, 2024-09-14T06:29:50.036 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:29:50.036 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:29:50.036 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:29:50.036 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:29:50.036 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:29:50.036 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:50.036 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 1493 2024-09-14T06:29:50.036 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:50.036 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:29:50.036 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:50.036 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:29:50.036 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.628999+0000", 2024-09-14T06:29:50.036 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:50.037 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:50.037 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:50.037 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:29:50.037 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.628999+0000", 2024-09-14T06:29:50.037 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:50.037 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:50.037 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:50.037 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:29:50.037 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.629000+0000", 2024-09-14T06:29:50.037 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.2500000000000001e-07 2024-09-14T06:29:50.037 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:50.037 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:50.037 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:29:50.037 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.629001+0000", 2024-09-14T06:29:50.038 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.511e-06 2024-09-14T06:29:50.038 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:50.038 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:50.038 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:29:50.038 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.629006+0000", 2024-09-14T06:29:50.038 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.1420000000000001e-06 2024-09-14T06:29:50.038 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:50.038 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:50.038 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:29:50.038 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.629010+0000", 2024-09-14T06:29:50.038 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.8120000000000001e-06 2024-09-14T06:29:50.038 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:50.038 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:50.038 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:29:50.038 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.647950+0000", 2024-09-14T06:29:50.040 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.018939568 2024-09-14T06:29:50.041 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:50.041 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:50.041 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:29:50.041 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.648032+0000", 2024-09-14T06:29:50.041 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.2132000000000006e-05 2024-09-14T06:29:50.041 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:50.041 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:50.041 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:29:50.041 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.648126+0000", 2024-09-14T06:29:50.041 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.4637999999999998e-05 2024-09-14T06:29:50.041 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:50.041 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:29:50.042 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:50.042 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:50.042 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:50.042 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:1494 3.a 3:57c28b0f:::rbd_data.112d7dfbd766.000000000000000a:5 [read 372736~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-14T06:29:50.042 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:08.629017+0000", 2024-09-14T06:29:50.042 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 41.364212535, 2024-09-14T06:29:50.042 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.019310414000000001, 2024-09-14T06:29:50.042 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:29:50.042 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:29:50.042 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:29:50.042 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:29:50.042 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:29:50.042 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:50.043 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 1494 2024-09-14T06:29:50.043 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:50.043 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:29:50.043 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:50.043 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:29:50.043 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.629017+0000", 2024-09-14T06:29:50.043 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:50.043 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:50.043 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:50.043 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:29:50.043 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.629017+0000", 2024-09-14T06:29:50.044 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:50.044 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:50.044 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:50.044 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:29:50.044 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.629018+0000", 2024-09-14T06:29:50.044 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.9000000000000002e-07 2024-09-14T06:29:50.044 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:50.044 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:50.044 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:29:50.044 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.629019+0000", 2024-09-14T06:29:50.044 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.5230000000000001e-06 2024-09-14T06:29:50.044 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:50.044 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:50.045 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:29:50.045 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.629024+0000", 2024-09-14T06:29:50.045 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.2299999999999999e-06 2024-09-14T06:29:50.045 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:50.045 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:50.045 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:29:50.045 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.629028+0000", 2024-09-14T06:29:50.045 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.7749999999999999e-06 2024-09-14T06:29:50.045 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:50.045 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:50.045 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:29:50.045 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.648156+0000", 2024-09-14T06:29:50.045 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.019128045999999999 2024-09-14T06:29:50.045 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:50.045 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:50.046 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:29:50.046 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.648238+0000", 2024-09-14T06:29:50.046 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.1877000000000002e-05 2024-09-14T06:29:50.046 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:50.046 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:50.046 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:29:50.046 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.648328+0000", 2024-09-14T06:29:50.046 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.9573000000000005e-05 2024-09-14T06:29:50.046 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:50.046 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:29:50.046 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:50.046 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:50.046 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:50.046 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:1495 3.a 3:57c28b0f:::rbd_data.112d7dfbd766.000000000000000a:5 [read 389120~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-14T06:29:50.046 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:08.629035+0000", 2024-09-14T06:29:50.047 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 41.364194511000001, 2024-09-14T06:29:50.047 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.019486507, 2024-09-14T06:29:50.047 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:29:50.047 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:29:50.047 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:29:50.047 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:29:50.047 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:29:50.047 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:50.047 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 1495 2024-09-14T06:29:50.047 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:50.047 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:29:50.047 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:50.047 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:29:50.047 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.629035+0000", 2024-09-14T06:29:50.048 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:50.048 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:50.048 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:50.048 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:29:50.048 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.629035+0000", 2024-09-14T06:29:50.048 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:50.048 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:50.048 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:50.048 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:29:50.048 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.629036+0000", 2024-09-14T06:29:50.048 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.2000000000000001e-07 2024-09-14T06:29:50.049 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:50.049 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:50.049 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:29:50.049 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.629037+0000", 2024-09-14T06:29:50.049 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.4300000000000001e-06 2024-09-14T06:29:50.049 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:50.049 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:50.049 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:29:50.049 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.629043+0000", 2024-09-14T06:29:50.049 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.575e-06 2024-09-14T06:29:50.049 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:50.049 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:50.049 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:29:50.049 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.629047+0000", 2024-09-14T06:29:50.050 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.9179999999999996e-06 2024-09-14T06:29:50.050 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:50.050 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:50.050 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:29:50.050 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.648356+0000", 2024-09-14T06:29:50.050 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.019309862000000001 2024-09-14T06:29:50.050 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:50.050 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:50.050 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:29:50.050 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.648433+0000", 2024-09-14T06:29:50.050 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.6973999999999999e-05 2024-09-14T06:29:50.050 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:50.050 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:50.050 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:29:50.051 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.648522+0000", 2024-09-14T06:29:50.051 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.8428000000000001e-05 2024-09-14T06:29:50.051 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:50.051 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:29:50.051 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:50.051 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:50.051 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:50.051 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:1496 3.a 3:57c28b0f:::rbd_data.112d7dfbd766.000000000000000a:5 [read 405504~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-14T06:29:50.051 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:08.629086+0000", 2024-09-14T06:29:50.051 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 41.364143466999998, 2024-09-14T06:29:50.051 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.019618813999999998, 2024-09-14T06:29:50.052 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:29:50.052 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:29:50.052 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:29:50.052 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:29:50.052 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:29:50.052 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:50.052 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 1496 2024-09-14T06:29:50.052 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:50.052 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:29:50.052 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:50.052 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:29:50.052 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.629086+0000", 2024-09-14T06:29:50.053 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:50.053 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:50.053 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:50.053 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:29:50.053 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.629086+0000", 2024-09-14T06:29:50.053 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:50.053 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:50.053 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:50.053 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:29:50.053 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.629087+0000", 2024-09-14T06:29:50.053 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.1399999999999996e-07 2024-09-14T06:29:50.054 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:50.054 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:50.054 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:29:50.054 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.629100+0000", 2024-09-14T06:29:50.054 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.2748999999999999e-05 2024-09-14T06:29:50.054 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:50.054 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:50.054 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:29:50.054 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.629111+0000", 2024-09-14T06:29:50.054 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.1531e-05 2024-09-14T06:29:50.054 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:50.054 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:50.054 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:29:50.054 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.629131+0000", 2024-09-14T06:29:50.054 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.9341999999999999e-05 2024-09-14T06:29:50.055 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:50.055 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:50.055 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:29:50.055 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.648550+0000", 2024-09-14T06:29:50.055 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.019418916000000001 2024-09-14T06:29:50.055 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:50.055 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:50.055 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:29:50.055 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.648616+0000", 2024-09-14T06:29:50.055 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.6359000000000006e-05 2024-09-14T06:29:50.055 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:50.055 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:50.055 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:29:50.055 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.648705+0000", 2024-09-14T06:29:50.055 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.9103000000000004e-05 2024-09-14T06:29:50.056 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:50.056 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:29:50.056 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:50.056 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:50.056 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:50.056 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:1497 3.a 3:57c28b0f:::rbd_data.112d7dfbd766.000000000000000a:5 [read 421888~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-14T06:29:50.056 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:08.629145+0000", 2024-09-14T06:29:50.056 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 41.364084689000002, 2024-09-14T06:29:50.056 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.019744725000000001, 2024-09-14T06:29:50.056 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:29:50.056 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:29:50.056 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:29:50.056 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:29:50.056 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:29:50.057 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:50.057 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 1497 2024-09-14T06:29:50.057 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:50.057 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:29:50.057 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:50.057 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:29:50.057 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.629145+0000", 2024-09-14T06:29:50.057 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:50.057 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:50.057 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:50.057 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:29:50.057 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.629145+0000", 2024-09-14T06:29:50.057 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:50.058 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:50.058 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:50.058 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:29:50.058 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.629146+0000", 2024-09-14T06:29:50.058 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.1699999999999999e-07 2024-09-14T06:29:50.058 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:50.058 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:50.058 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:29:50.058 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.629148+0000", 2024-09-14T06:29:50.058 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.024e-06 2024-09-14T06:29:50.058 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:50.058 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:50.058 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:29:50.058 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.629152+0000", 2024-09-14T06:29:50.058 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.5970000000000002e-06 2024-09-14T06:29:50.059 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:50.059 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:50.059 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:29:50.059 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.629157+0000", 2024-09-14T06:29:50.059 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.8119999999999998e-06 2024-09-14T06:29:50.059 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:50.059 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:50.059 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:29:50.059 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.648734+0000", 2024-09-14T06:29:50.059 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.019576935 2024-09-14T06:29:50.059 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:50.059 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:50.059 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:29:50.060 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.648801+0000", 2024-09-14T06:29:50.060 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.7305999999999996e-05 2024-09-14T06:29:50.060 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:50.060 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:50.060 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:29:50.060 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.648890+0000", 2024-09-14T06:29:50.060 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.8634000000000006e-05 2024-09-14T06:29:50.060 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:50.060 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:29:50.060 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:50.060 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:50.060 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:50.060 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:1498 3.a 3:57c28b0f:::rbd_data.112d7dfbd766.000000000000000a:5 [read 438272~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-14T06:29:50.061 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:08.629167+0000", 2024-09-14T06:29:50.061 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 41.364062625000003, 2024-09-14T06:29:50.061 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.019912138999999999, 2024-09-14T06:29:50.061 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:29:50.061 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:29:50.061 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:29:50.061 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:29:50.061 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:29:50.061 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:50.061 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 1498 2024-09-14T06:29:50.061 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:50.061 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:29:50.061 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:50.062 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:29:50.062 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.629167+0000", 2024-09-14T06:29:50.062 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:50.062 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:50.062 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:50.062 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:29:50.062 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.629167+0000", 2024-09-14T06:29:50.062 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:50.062 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:50.062 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:50.062 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:29:50.062 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.629168+0000", 2024-09-14T06:29:50.062 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.1300000000000001e-07 2024-09-14T06:29:50.062 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:50.063 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:50.063 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:29:50.063 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.629170+0000", 2024-09-14T06:29:50.063 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.7209999999999999e-06 2024-09-14T06:29:50.063 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:50.063 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:50.063 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:29:50.063 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.629174+0000", 2024-09-14T06:29:50.063 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.1459999999999998e-06 2024-09-14T06:29:50.063 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:50.063 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:50.063 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:29:50.063 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.629178+0000", 2024-09-14T06:29:50.064 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.8519999999999997e-06 2024-09-14T06:29:50.064 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:50.064 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:50.064 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:29:50.064 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.648924+0000", 2024-09-14T06:29:50.064 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.019746107999999998 2024-09-14T06:29:50.064 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:50.064 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:50.064 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:29:50.064 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.648995+0000", 2024-09-14T06:29:50.064 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.0809999999999995e-05 2024-09-14T06:29:50.064 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:50.064 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:50.064 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:29:50.065 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.649079+0000", 2024-09-14T06:29:50.065 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.4089000000000001e-05 2024-09-14T06:29:50.065 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:50.065 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:29:50.065 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:50.065 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:50.065 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:50.065 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:1499 3.a 3:57c28b0f:::rbd_data.112d7dfbd766.000000000000000a:5 [read 454656~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-14T06:29:50.065 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:08.629187+0000", 2024-09-14T06:29:50.065 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 41.364042660999999, 2024-09-14T06:29:50.065 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.020072543000000002, 2024-09-14T06:29:50.065 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:29:50.065 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:29:50.066 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:29:50.066 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:29:50.066 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:29:50.066 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:50.066 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 1499 2024-09-14T06:29:50.066 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:50.066 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:29:50.066 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:50.066 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:29:50.066 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.629187+0000", 2024-09-14T06:29:50.066 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:50.066 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:50.066 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:50.067 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:29:50.067 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.629187+0000", 2024-09-14T06:29:50.067 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:50.067 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:50.067 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:50.067 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:29:50.067 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.629188+0000", 2024-09-14T06:29:50.067 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.89e-07 2024-09-14T06:29:50.067 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:50.067 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:50.067 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:29:50.067 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.629190+0000", 2024-09-14T06:29:50.068 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.0169999999999999e-06 2024-09-14T06:29:50.068 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:50.068 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:50.068 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:29:50.068 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.629195+0000", 2024-09-14T06:29:50.068 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.4149999999999998e-06 2024-09-14T06:29:50.068 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:50.068 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:50.068 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:29:50.068 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.629198+0000", 2024-09-14T06:29:50.068 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.605e-06 2024-09-14T06:29:50.068 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:50.068 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:50.068 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:29:50.068 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.649107+0000", 2024-09-14T06:29:50.069 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.019909339000000002 2024-09-14T06:29:50.069 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:50.069 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:50.069 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:29:50.069 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.649178+0000", 2024-09-14T06:29:50.069 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.0809999999999995e-05 2024-09-14T06:29:50.069 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:50.069 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:50.069 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:29:50.069 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.649260+0000", 2024-09-14T06:29:50.069 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.1967999999999999e-05 2024-09-14T06:29:50.069 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:50.069 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:29:50.070 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:50.070 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:50.070 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:50.070 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:1500 3.a 3:57c28b0f:::rbd_data.112d7dfbd766.000000000000000a:5 [read 471040~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-14T06:29:50.070 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:08.629204+0000", 2024-09-14T06:29:50.070 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 41.364026168999999, 2024-09-14T06:29:50.070 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.020473080000000001, 2024-09-14T06:29:50.070 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:29:50.070 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:29:50.070 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:29:50.070 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:29:50.070 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:29:50.070 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:50.070 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 1500 2024-09-14T06:29:50.070 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:50.071 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:29:50.071 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:50.071 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:29:50.071 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.629204+0000", 2024-09-14T06:29:50.071 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:50.071 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:50.071 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:50.071 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:29:50.071 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.629204+0000", 2024-09-14T06:29:50.071 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:50.071 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:50.071 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:50.071 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:29:50.071 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.629204+0000", 2024-09-14T06:29:50.071 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.34e-07 2024-09-14T06:29:50.072 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:50.072 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:50.072 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:29:50.072 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.629206+0000", 2024-09-14T06:29:50.072 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.939e-06 2024-09-14T06:29:50.072 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:50.072 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:50.072 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:29:50.072 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.629212+0000", 2024-09-14T06:29:50.072 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.8669999999999999e-06 2024-09-14T06:29:50.072 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:50.072 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:50.072 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:29:50.072 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.629216+0000", 2024-09-14T06:29:50.073 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.3830000000000002e-06 2024-09-14T06:29:50.073 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:50.073 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:50.073 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:29:50.073 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.649288+0000", 2024-09-14T06:29:50.073 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.020072113999999999 2024-09-14T06:29:50.073 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:50.073 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:50.073 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:29:50.073 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.649355+0000", 2024-09-14T06:29:50.073 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.7218000000000006e-05 2024-09-14T06:29:50.073 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:50.073 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:50.073 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:29:50.074 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.649677+0000", 2024-09-14T06:29:50.074 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00032122500000000002 2024-09-14T06:29:50.074 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:50.074 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:29:50.074 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:50.074 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:50.074 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:50.074 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:1501 3.a 3:57c28b0f:::rbd_data.112d7dfbd766.000000000000000a:5 [read 487424~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-14T06:29:50.074 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:08.629223+0000", 2024-09-14T06:29:50.074 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 41.364006719999999, 2024-09-14T06:29:50.074 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.021370209000000001, 2024-09-14T06:29:50.075 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:29:50.075 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:29:50.075 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:29:50.075 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:29:50.075 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:29:50.075 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:50.075 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 1501 2024-09-14T06:29:50.075 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:50.075 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:29:50.075 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:50.075 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:29:50.075 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.629223+0000", 2024-09-14T06:29:50.075 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:50.075 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:50.076 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:50.076 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:29:50.076 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.629223+0000", 2024-09-14T06:29:50.076 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:50.076 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:50.076 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:50.076 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:29:50.076 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.629223+0000", 2024-09-14T06:29:50.076 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.9900000000000001e-07 2024-09-14T06:29:50.076 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:50.076 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:50.077 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:29:50.077 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.629231+0000", 2024-09-14T06:29:50.077 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.9179999999999994e-06 2024-09-14T06:29:50.077 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:50.077 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:50.077 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:29:50.077 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.629238+0000", 2024-09-14T06:29:50.077 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.421e-06 2024-09-14T06:29:50.077 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:50.077 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:50.077 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:29:50.077 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.629242+0000", 2024-09-14T06:29:50.077 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.5419999999999999e-06 2024-09-14T06:29:50.077 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:50.078 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:50.078 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:29:50.078 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.650284+0000", 2024-09-14T06:29:50.078 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.021041131000000001 2024-09-14T06:29:50.078 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:50.078 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:50.078 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:29:50.078 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.650361+0000", 2024-09-14T06:29:50.078 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.7787999999999995e-05 2024-09-14T06:29:50.078 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:50.078 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:50.078 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:29:50.078 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.650593+0000", 2024-09-14T06:29:50.078 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00023201 2024-09-14T06:29:50.079 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:50.079 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:29:50.079 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:50.079 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:50.079 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:50.079 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:1502 3.a 3:57c28b0f:::rbd_data.112d7dfbd766.000000000000000a:5 [read 503808~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-14T06:29:50.079 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:08.629249+0000", 2024-09-14T06:29:50.079 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 41.363980474000002, 2024-09-14T06:29:50.079 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.021713217, 2024-09-14T06:29:50.079 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:29:50.079 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:29:50.079 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:29:50.079 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:29:50.080 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:29:50.080 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:50.080 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 1502 2024-09-14T06:29:50.080 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:50.080 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:29:50.080 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:50.080 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:29:50.080 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.629249+0000", 2024-09-14T06:29:50.080 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:50.080 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:50.081 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:50.081 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:29:50.081 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.629249+0000", 2024-09-14T06:29:50.081 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:50.081 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:50.081 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:50.081 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:29:50.081 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.629250+0000", 2024-09-14T06:29:50.081 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.03e-07 2024-09-14T06:29:50.081 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:50.081 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:50.081 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:29:50.081 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.629252+0000", 2024-09-14T06:29:50.082 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.995e-06 2024-09-14T06:29:50.082 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:50.082 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:50.082 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:29:50.082 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.629257+0000", 2024-09-14T06:29:50.082 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.7010000000000001e-06 2024-09-14T06:29:50.082 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:50.082 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:50.082 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:29:50.082 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.629262+0000", 2024-09-14T06:29:50.082 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.16e-06 2024-09-14T06:29:50.082 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:50.082 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:50.082 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:29:50.083 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.650629+0000", 2024-09-14T06:29:50.083 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.021367684000000001 2024-09-14T06:29:50.083 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:50.083 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:50.083 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:29:50.083 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.650718+0000", 2024-09-14T06:29:50.083 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.8832000000000006e-05 2024-09-14T06:29:50.083 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:50.083 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:50.083 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:29:50.083 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.650963+0000", 2024-09-14T06:29:50.083 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00024444199999999999 2024-09-14T06:29:50.083 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:50.083 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:29:50.083 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:50.083 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:50.084 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:50.084 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:1503 3.a 3:57c28b0f:::rbd_data.112d7dfbd766.000000000000000a:5 [read 520192~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-14T06:29:50.084 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:08.629268+0000", 2024-09-14T06:29:50.084 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 41.363961932999999, 2024-09-14T06:29:50.084 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.022244885999999998, 2024-09-14T06:29:50.084 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:29:50.084 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:29:50.084 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:29:50.084 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:29:50.084 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:29:50.084 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:50.085 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 1503 2024-09-14T06:29:50.085 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:50.085 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:29:50.085 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:50.085 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:29:50.085 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.629268+0000", 2024-09-14T06:29:50.085 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:50.085 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:50.085 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:50.085 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:29:50.085 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.629268+0000", 2024-09-14T06:29:50.085 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:50.085 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:50.085 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:50.086 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:29:50.086 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.629268+0000", 2024-09-14T06:29:50.086 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.6599999999999996e-07 2024-09-14T06:29:50.086 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:50.086 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:50.086 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:29:50.086 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.629270+0000", 2024-09-14T06:29:50.086 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.7129999999999999e-06 2024-09-14T06:29:50.086 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:50.086 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:50.086 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:29:50.086 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.629276+0000", 2024-09-14T06:29:50.086 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.8119999999999996e-06 2024-09-14T06:29:50.086 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:50.087 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:50.087 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:29:50.087 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.629280+0000", 2024-09-14T06:29:50.087 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.4920000000000004e-06 2024-09-14T06:29:50.087 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:50.087 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:50.087 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:29:50.087 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.651039+0000", 2024-09-14T06:29:50.087 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.021758116000000001 2024-09-14T06:29:50.087 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:50.087 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:50.087 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:29:50.087 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.651127+0000", 2024-09-14T06:29:50.087 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.7983000000000001e-05 2024-09-14T06:29:50.087 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:50.088 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:50.088 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:29:50.088 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.651513+0000", 2024-09-14T06:29:50.088 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00038620400000000002 2024-09-14T06:29:50.088 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:50.088 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:29:50.088 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:50.088 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:50.088 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:29:50.088 INFO:teuthology.orchestra.run.smithi138.stdout:} 2024-09-14T06:29:50.090 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-14T06:29:50.090 INFO:tasks.ceph.osd.7.smithi138.stderr:2024-09-14T06:29:50.087+0000 7f3c6a21d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33623) UID: 0 2024-09-14T06:29:50.190 INFO:tasks.ceph.osd.4.smithi138.stderr:2024-09-14T06:29:50.188+0000 7f6c653c8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33615) UID: 0 2024-09-14T06:29:50.235 INFO:teuthology.orchestra.run.smithi138.stdout:{ 2024-09-14T06:29:50.235 INFO:teuthology.orchestra.run.smithi138.stdout: "ops": [], 2024-09-14T06:29:50.235 INFO:teuthology.orchestra.run.smithi138.stdout: "num_ops": 0 2024-09-14T06:29:50.235 INFO:teuthology.orchestra.run.smithi138.stdout:} 2024-09-14T06:29:50.244 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-14T06:29:50.329 INFO:tasks.ceph.osd.7.smithi138.stderr:2024-09-14T06:29:50.327+0000 7f3c6a21d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33623) UID: 0 2024-09-14T06:29:50.376 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-14T06:29:50.386 INFO:teuthology.orchestra.run.smithi138.stdout:{ 2024-09-14T06:29:50.387 INFO:teuthology.orchestra.run.smithi138.stdout: "ops": [], 2024-09-14T06:29:50.387 INFO:teuthology.orchestra.run.smithi138.stdout: "complaint_time": 30, 2024-09-14T06:29:50.387 INFO:teuthology.orchestra.run.smithi138.stdout: "num_blocked_ops": 0 2024-09-14T06:29:50.387 INFO:teuthology.orchestra.run.smithi138.stdout:} 2024-09-14T06:29:50.395 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-14T06:29:50.420 INFO:tasks.ceph.osd.0.smithi094.stderr:2024-09-14T06:29:50.419+0000 7f07c9333640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33636) UID: 0 2024-09-14T06:29:50.492 INFO:tasks.ceph.osd.7.smithi138.stderr:2024-09-14T06:29:50.490+0000 7f3c6a21d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33623) UID: 0 2024-09-14T06:29:50.508 INFO:teuthology.orchestra.run.smithi094.stdout:osd.0: {} 2024-09-14T06:29:50.508 INFO:teuthology.orchestra.run.smithi094.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-14T06:29:50.525 INFO:teuthology.orchestra.run.smithi094.stdout:osd.1: {} 2024-09-14T06:29:50.525 INFO:teuthology.orchestra.run.smithi094.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-14T06:29:50.541 INFO:teuthology.orchestra.run.smithi138.stdout:{ 2024-09-14T06:29:50.541 INFO:teuthology.orchestra.run.smithi138.stdout: "size": 20, 2024-09-14T06:29:50.541 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 600, 2024-09-14T06:29:50.541 INFO:teuthology.orchestra.run.smithi138.stdout: "ops": [ 2024-09-14T06:29:50.541 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:50.541 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:53244 3.f 3:f71f51a7:::rbd_data.112dca2f6ae1.0000000000000015:17 [read 1863680~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:29:50.541 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:44.369399+0000", 2024-09-14T06:29:50.541 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 6.1697671920000001, 2024-09-14T06:29:50.541 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047291189999999997, 2024-09-14T06:29:50.541 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:29:50.542 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:29:50.542 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:29:50.542 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:29:50.542 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:29:50.542 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:50.542 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 53244 2024-09-14T06:29:50.542 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:50.543 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:29:50.543 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:50.543 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:29:50.543 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369399+0000", 2024-09-14T06:29:50.543 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:50.543 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:50.543 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:50.543 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:29:50.543 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369399+0000", 2024-09-14T06:29:50.543 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:50.543 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:50.543 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:50.543 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:29:50.543 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369400+0000", 2024-09-14T06:29:50.544 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.0699999999999999e-07 2024-09-14T06:29:50.544 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:50.544 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:50.544 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:29:50.544 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369401+0000", 2024-09-14T06:29:50.544 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.4300000000000001e-06 2024-09-14T06:29:50.544 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:50.544 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:50.544 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:29:50.545 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369404+0000", 2024-09-14T06:29:50.545 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.061e-06 2024-09-14T06:29:50.545 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:50.545 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:50.545 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:29:50.545 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369407+0000", 2024-09-14T06:29:50.545 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.6139999999999999e-06 2024-09-14T06:29:50.545 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:50.545 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:50.545 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:29:50.545 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.416517+0000", 2024-09-14T06:29:50.545 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047110338000000002 2024-09-14T06:29:50.545 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:50.545 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:50.546 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:29:50.546 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.416616+0000", 2024-09-14T06:29:50.546 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.8651000000000001e-05 2024-09-14T06:29:50.546 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:50.546 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:50.546 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:29:50.546 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.416691+0000", 2024-09-14T06:29:50.546 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.4888999999999994e-05 2024-09-14T06:29:50.546 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:50.546 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:29:50.546 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:50.546 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:50.546 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:50.547 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:53245 3.f 3:f71f51a7:::rbd_data.112dca2f6ae1.0000000000000015:17 [read 1888256~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:29:50.547 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:44.369410+0000", 2024-09-14T06:29:50.547 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 6.1697569039999998, 2024-09-14T06:29:50.547 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047399275999999997, 2024-09-14T06:29:50.547 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:29:50.547 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:29:50.547 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:29:50.547 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:29:50.547 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:29:50.547 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:50.547 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 53245 2024-09-14T06:29:50.547 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:50.547 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:29:50.547 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:50.548 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:29:50.548 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369410+0000", 2024-09-14T06:29:50.548 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:50.548 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:50.548 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:50.548 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:29:50.548 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369410+0000", 2024-09-14T06:29:50.548 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:50.548 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:50.548 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:50.548 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:29:50.548 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369410+0000", 2024-09-14T06:29:50.548 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.1899999999999999e-07 2024-09-14T06:29:50.548 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:50.549 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:50.549 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:29:50.549 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369411+0000", 2024-09-14T06:29:50.549 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.0300000000000001e-06 2024-09-14T06:29:50.549 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:50.549 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:50.549 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:29:50.549 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369414+0000", 2024-09-14T06:29:50.549 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.1099999999999999e-06 2024-09-14T06:29:50.549 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:50.549 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:50.549 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:29:50.549 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369417+0000", 2024-09-14T06:29:50.550 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.6189999999999998e-06 2024-09-14T06:29:50.550 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:50.550 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:50.550 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:29:50.550 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.416718+0000", 2024-09-14T06:29:50.550 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047301322999999999 2024-09-14T06:29:50.550 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:50.550 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:50.550 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:29:50.550 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.416759+0000", 2024-09-14T06:29:50.550 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.1009e-05 2024-09-14T06:29:50.550 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:50.550 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:50.550 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:29:50.551 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.416809+0000", 2024-09-14T06:29:50.551 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.9966000000000002e-05 2024-09-14T06:29:50.551 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:50.551 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:29:50.551 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:50.551 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:50.551 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:50.551 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:53246 3.f 3:f71f51a7:::rbd_data.112dca2f6ae1.0000000000000015:17 [read 1912832~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:29:50.551 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:44.369420+0000", 2024-09-14T06:29:50.552 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 6.1697469600000003, 2024-09-14T06:29:50.552 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047529463000000001, 2024-09-14T06:29:50.552 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:29:50.552 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:29:50.552 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:29:50.552 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:29:50.552 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:29:50.552 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:50.552 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 53246 2024-09-14T06:29:50.552 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:50.552 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:29:50.552 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:50.552 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:29:50.552 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369420+0000", 2024-09-14T06:29:50.553 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:50.553 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:50.553 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:50.553 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:29:50.553 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369420+0000", 2024-09-14T06:29:50.553 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:50.553 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:50.553 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:50.553 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:29:50.553 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369420+0000", 2024-09-14T06:29:50.553 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.1299999999999999e-07 2024-09-14T06:29:50.553 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:50.553 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:50.553 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:29:50.554 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369425+0000", 2024-09-14T06:29:50.554 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.0130000000000001e-06 2024-09-14T06:29:50.554 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:50.554 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:50.554 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:29:50.554 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369428+0000", 2024-09-14T06:29:50.554 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.349e-06 2024-09-14T06:29:50.554 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:50.554 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:50.554 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:29:50.554 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369431+0000", 2024-09-14T06:29:50.554 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.7389999999999999e-06 2024-09-14T06:29:50.554 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:50.554 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:50.555 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:29:50.555 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.416834+0000", 2024-09-14T06:29:50.555 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047402669000000001 2024-09-14T06:29:50.555 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:50.555 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:50.555 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:29:50.555 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.416882+0000", 2024-09-14T06:29:50.555 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.8068999999999997e-05 2024-09-14T06:29:50.555 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:50.555 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:50.555 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:29:50.555 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.416949+0000", 2024-09-14T06:29:50.555 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.7410999999999995e-05 2024-09-14T06:29:50.556 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:50.556 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:29:50.556 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:50.556 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:50.556 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:50.556 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:53247 3.f 3:f71f51a7:::rbd_data.112dca2f6ae1.0000000000000015:17 [read 1937408~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:29:50.556 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:44.369434+0000", 2024-09-14T06:29:50.556 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 6.169732196, 2024-09-14T06:29:50.556 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047652264999999999, 2024-09-14T06:29:50.556 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:29:50.557 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:29:50.557 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:29:50.557 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:29:50.557 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:29:50.557 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:50.557 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 53247 2024-09-14T06:29:50.557 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:50.557 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:29:50.557 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:50.557 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:29:50.557 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369434+0000", 2024-09-14T06:29:50.557 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:50.557 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:50.558 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:50.558 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:29:50.558 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369434+0000", 2024-09-14T06:29:50.558 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:50.558 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:50.558 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:50.558 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:29:50.558 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369435+0000", 2024-09-14T06:29:50.558 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.2100000000000001e-07 2024-09-14T06:29:50.558 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:50.558 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:50.558 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:29:50.558 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369436+0000", 2024-09-14T06:29:50.558 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.122e-06 2024-09-14T06:29:50.559 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:50.559 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:50.559 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:29:50.559 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369439+0000", 2024-09-14T06:29:50.559 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.0400000000000001e-06 2024-09-14T06:29:50.559 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:50.559 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:50.559 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:29:50.559 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369441+0000", 2024-09-14T06:29:50.559 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.728e-06 2024-09-14T06:29:50.559 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:50.559 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:50.559 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:29:50.559 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.416977+0000", 2024-09-14T06:29:50.560 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047535344 2024-09-14T06:29:50.560 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:50.560 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:50.560 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:29:50.560 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.417029+0000", 2024-09-14T06:29:50.560 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.2037999999999999e-05 2024-09-14T06:29:50.560 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:50.560 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:50.560 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:29:50.560 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.417087+0000", 2024-09-14T06:29:50.560 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.7772000000000002e-05 2024-09-14T06:29:50.560 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:50.560 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:29:50.560 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:50.561 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:50.561 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:50.561 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:53248 3.f 3:f71f51a7:::rbd_data.112dca2f6ae1.0000000000000015:17 [read 1961984~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:29:50.561 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:44.369445+0000", 2024-09-14T06:29:50.561 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 6.169721644, 2024-09-14T06:29:50.561 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047798489, 2024-09-14T06:29:50.561 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:29:50.561 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:29:50.561 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:29:50.561 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:29:50.561 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:29:50.562 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:50.562 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 53248 2024-09-14T06:29:50.562 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:50.562 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:29:50.562 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:50.562 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:29:50.562 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369445+0000", 2024-09-14T06:29:50.562 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:50.563 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:50.563 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:50.563 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:29:50.563 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369445+0000", 2024-09-14T06:29:50.563 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:50.563 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:50.563 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:50.563 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:29:50.563 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369445+0000", 2024-09-14T06:29:50.563 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.0699999999999999e-07 2024-09-14T06:29:50.563 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:50.563 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:50.563 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:29:50.564 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369446+0000", 2024-09-14T06:29:50.564 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.1000000000000001e-06 2024-09-14T06:29:50.564 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:50.564 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:50.564 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:29:50.564 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369449+0000", 2024-09-14T06:29:50.564 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.1039999999999998e-06 2024-09-14T06:29:50.564 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:50.564 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:50.564 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:29:50.564 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369452+0000", 2024-09-14T06:29:50.564 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.6199999999999999e-06 2024-09-14T06:29:50.564 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:50.564 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:50.565 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:29:50.565 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.417114+0000", 2024-09-14T06:29:50.565 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047661901999999999 2024-09-14T06:29:50.565 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:50.565 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:50.565 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:29:50.565 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.417181+0000", 2024-09-14T06:29:50.565 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.7385999999999995e-05 2024-09-14T06:29:50.565 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:50.565 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:50.565 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:29:50.565 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.417243+0000", 2024-09-14T06:29:50.565 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.2169999999999996e-05 2024-09-14T06:29:50.566 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:50.566 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:29:50.566 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:50.566 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:50.566 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:50.566 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:53249 3.f 3:f71f51a7:::rbd_data.112dca2f6ae1.0000000000000015:17 [read 1986560~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:29:50.566 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:44.369455+0000", 2024-09-14T06:29:50.566 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 6.1697113330000004, 2024-09-14T06:29:50.566 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047890493999999999, 2024-09-14T06:29:50.566 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:29:50.566 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:29:50.566 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:29:50.566 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:29:50.566 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:29:50.567 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:50.567 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 53249 2024-09-14T06:29:50.567 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:50.567 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:29:50.567 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:50.567 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:29:50.567 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369455+0000", 2024-09-14T06:29:50.567 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:50.567 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:50.567 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:50.567 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:29:50.567 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369455+0000", 2024-09-14T06:29:50.567 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:50.568 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:50.568 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:50.568 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:29:50.568 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369455+0000", 2024-09-14T06:29:50.568 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.0800000000000001e-07 2024-09-14T06:29:50.568 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:50.568 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:50.568 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:29:50.568 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369456+0000", 2024-09-14T06:29:50.568 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.0729999999999999e-06 2024-09-14T06:29:50.568 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:50.569 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:50.569 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:29:50.569 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369460+0000", 2024-09-14T06:29:50.569 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.027e-06 2024-09-14T06:29:50.569 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:50.569 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:50.569 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:29:50.569 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369462+0000", 2024-09-14T06:29:50.569 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.5320000000000001e-06 2024-09-14T06:29:50.569 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:50.569 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:50.569 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:29:50.569 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.417257+0000", 2024-09-14T06:29:50.570 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.04779489 2024-09-14T06:29:50.570 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:50.570 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:50.570 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:29:50.570 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.417296+0000", 2024-09-14T06:29:50.570 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.8658000000000002e-05 2024-09-14T06:29:50.570 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:50.570 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:50.570 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:29:50.570 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.417346+0000", 2024-09-14T06:29:50.570 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.0105999999999998e-05 2024-09-14T06:29:50.570 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:50.570 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:29:50.571 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:50.571 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:50.571 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:50.571 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:53250 3.f 3:f71f51a7:::rbd_data.112dca2f6ae1.0000000000000015:17 [read 2011136~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:29:50.571 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:44.369465+0000", 2024-09-14T06:29:50.571 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 6.1697013629999997, 2024-09-14T06:29:50.571 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.048032230000000002, 2024-09-14T06:29:50.571 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:29:50.571 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:29:50.571 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:29:50.571 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:29:50.571 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:29:50.571 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:50.572 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 53250 2024-09-14T06:29:50.572 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:50.572 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:29:50.572 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:50.572 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:29:50.572 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369465+0000", 2024-09-14T06:29:50.572 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:50.572 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:50.572 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:50.572 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:29:50.572 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369465+0000", 2024-09-14T06:29:50.572 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:50.573 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:50.573 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:50.573 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:29:50.573 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369465+0000", 2024-09-14T06:29:50.573 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.2100000000000001e-07 2024-09-14T06:29:50.573 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:50.573 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:50.573 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:29:50.573 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369466+0000", 2024-09-14T06:29:50.573 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.0640000000000001e-06 2024-09-14T06:29:50.573 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:50.574 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:50.574 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:29:50.574 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369469+0000", 2024-09-14T06:29:50.574 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.0170000000000001e-06 2024-09-14T06:29:50.574 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:50.574 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:50.574 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:29:50.574 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369472+0000", 2024-09-14T06:29:50.574 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.6189999999999998e-06 2024-09-14T06:29:50.574 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:50.574 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:50.574 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:29:50.574 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.417383+0000", 2024-09-14T06:29:50.575 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047911092000000002 2024-09-14T06:29:50.575 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:50.575 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:50.575 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:29:50.575 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.417430+0000", 2024-09-14T06:29:50.575 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.6898999999999999e-05 2024-09-14T06:29:50.575 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:50.575 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:50.575 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:29:50.575 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.417497+0000", 2024-09-14T06:29:50.575 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.7317999999999995e-05 2024-09-14T06:29:50.575 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:50.575 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:29:50.576 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:50.576 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:50.576 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:50.576 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:53251 3.f 3:f71f51a7:::rbd_data.112dca2f6ae1.0000000000000015:17 [read 2035712~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:29:50.576 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:44.369475+0000", 2024-09-14T06:29:50.576 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 6.1696913670000004, 2024-09-14T06:29:50.576 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.048154542000000002, 2024-09-14T06:29:50.576 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:29:50.576 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:29:50.576 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:29:50.576 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:29:50.576 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:29:50.576 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:50.577 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 53251 2024-09-14T06:29:50.577 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:50.577 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:29:50.577 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:50.577 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:29:50.577 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369475+0000", 2024-09-14T06:29:50.577 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:50.577 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:50.577 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:50.578 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:29:50.578 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369475+0000", 2024-09-14T06:29:50.578 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:50.578 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:50.578 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:50.578 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:29:50.578 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369475+0000", 2024-09-14T06:29:50.578 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.0100000000000001e-07 2024-09-14T06:29:50.578 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:50.578 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:50.578 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:29:50.579 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369476+0000", 2024-09-14T06:29:50.579 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.0580000000000001e-06 2024-09-14T06:29:50.579 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:50.579 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:50.579 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:29:50.579 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369479+0000", 2024-09-14T06:29:50.579 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.9950000000000001e-06 2024-09-14T06:29:50.579 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:50.579 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:50.579 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:29:50.579 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369482+0000", 2024-09-14T06:29:50.579 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.6340000000000001e-06 2024-09-14T06:29:50.579 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:50.579 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:50.580 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:29:50.580 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.417530+0000", 2024-09-14T06:29:50.580 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.048047535000000002 2024-09-14T06:29:50.580 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:50.580 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:50.580 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:29:50.580 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.417574+0000", 2024-09-14T06:29:50.580 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.4255999999999998e-05 2024-09-14T06:29:50.580 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:50.580 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:50.580 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:29:50.580 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.417630+0000", 2024-09-14T06:29:50.580 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.5862999999999997e-05 2024-09-14T06:29:50.580 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:50.581 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:29:50.581 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:50.581 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:50.581 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:50.581 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:53252 3.f 3:f71f51a7:::rbd_data.112dca2f6ae1.0000000000000015:17 [read 2060288~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:29:50.581 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:44.369485+0000", 2024-09-14T06:29:50.581 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 6.1696815190000001, 2024-09-14T06:29:50.581 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.048254116, 2024-09-14T06:29:50.581 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:29:50.581 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:29:50.581 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:29:50.581 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:29:50.581 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:29:50.581 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:50.582 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 53252 2024-09-14T06:29:50.582 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:50.582 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:29:50.582 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:50.582 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:29:50.582 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369485+0000", 2024-09-14T06:29:50.582 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:50.582 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:50.582 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:50.582 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:29:50.582 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369485+0000", 2024-09-14T06:29:50.583 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:50.583 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:50.583 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:50.583 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:29:50.583 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369485+0000", 2024-09-14T06:29:50.583 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.1e-07 2024-09-14T06:29:50.583 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:50.583 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:50.583 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:29:50.583 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369487+0000", 2024-09-14T06:29:50.583 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.42e-06 2024-09-14T06:29:50.583 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:50.583 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:50.583 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:29:50.583 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369490+0000", 2024-09-14T06:29:50.584 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.9639999999999999e-06 2024-09-14T06:29:50.584 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:50.584 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:50.584 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:29:50.584 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369492+0000", 2024-09-14T06:29:50.584 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.5679999999999998e-06 2024-09-14T06:29:50.584 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:50.584 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:50.584 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:29:50.584 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.417645+0000", 2024-09-14T06:29:50.584 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.048152753 2024-09-14T06:29:50.584 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:50.584 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:50.584 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:29:50.585 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.417687+0000", 2024-09-14T06:29:50.585 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.1900999999999997e-05 2024-09-14T06:29:50.585 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:50.585 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:50.585 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:29:50.585 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.417739+0000", 2024-09-14T06:29:50.585 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.2299999999999997e-05 2024-09-14T06:29:50.585 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:50.585 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:29:50.585 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:50.585 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:50.586 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:50.586 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:53253 3.f 3:f71f51a7:::rbd_data.112dca2f6ae1.0000000000000015:17 [read 2084864~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:29:50.586 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:44.369495+0000", 2024-09-14T06:29:50.586 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 6.1696713949999999, 2024-09-14T06:29:50.586 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.048366785000000002, 2024-09-14T06:29:50.586 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:29:50.586 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:29:50.586 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:29:50.587 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:29:50.587 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:29:50.587 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:50.587 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 53253 2024-09-14T06:29:50.587 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:50.587 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:29:50.587 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:50.587 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:29:50.587 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369495+0000", 2024-09-14T06:29:50.587 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:50.587 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:50.587 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:50.587 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:29:50.588 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369495+0000", 2024-09-14T06:29:50.588 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:50.588 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:50.588 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:50.588 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:29:50.588 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370388+0000", 2024-09-14T06:29:50.588 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00089287999999999998 2024-09-14T06:29:50.588 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:50.588 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:50.588 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:29:50.588 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370392+0000", 2024-09-14T06:29:50.588 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.2130000000000002e-06 2024-09-14T06:29:50.588 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:50.589 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:50.589 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:29:50.589 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370401+0000", 2024-09-14T06:29:50.589 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.7649999999999999e-06 2024-09-14T06:29:50.589 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:50.589 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:50.589 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:29:50.589 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370406+0000", 2024-09-14T06:29:50.589 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.0159999999999999e-06 2024-09-14T06:29:50.589 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:50.589 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:50.589 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:29:50.589 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.417764+0000", 2024-09-14T06:29:50.590 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047358348000000001 2024-09-14T06:29:50.590 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:50.590 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:50.590 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:29:50.590 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.417806+0000", 2024-09-14T06:29:50.590 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.1149999999999997e-05 2024-09-14T06:29:50.590 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:50.590 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:50.590 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:29:50.590 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.417862+0000", 2024-09-14T06:29:50.590 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.6412999999999997e-05 2024-09-14T06:29:50.590 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:50.590 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:29:50.590 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:50.591 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:50.591 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:50.591 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:53340 3.f 3:f71f51a7:::rbd_data.112dca2f6ae1.0000000000000015:17 [read 1056768~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:29:50.591 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:44.370421+0000", 2024-09-14T06:29:50.591 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 6.1687455150000003, 2024-09-14T06:29:50.591 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047566313999999998, 2024-09-14T06:29:50.591 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:29:50.591 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:29:50.591 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:29:50.591 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:29:50.591 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:29:50.591 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:50.591 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 53340 2024-09-14T06:29:50.592 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:50.592 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:29:50.592 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:50.592 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:29:50.592 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370421+0000", 2024-09-14T06:29:50.592 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:50.592 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:50.592 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:50.592 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:29:50.592 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370421+0000", 2024-09-14T06:29:50.592 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:50.592 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:50.592 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:50.592 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:29:50.593 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370422+0000", 2024-09-14T06:29:50.593 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.3799999999999996e-07 2024-09-14T06:29:50.593 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:50.593 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:50.593 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:29:50.593 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370424+0000", 2024-09-14T06:29:50.593 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.6919999999999998e-06 2024-09-14T06:29:50.593 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:50.593 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:50.593 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:29:50.593 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370432+0000", 2024-09-14T06:29:50.593 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.2439999999999996e-06 2024-09-14T06:29:50.593 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:50.593 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:50.594 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:29:50.594 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370437+0000", 2024-09-14T06:29:50.594 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.5389999999999998e-06 2024-09-14T06:29:50.594 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:50.594 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:50.594 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:29:50.594 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.417888+0000", 2024-09-14T06:29:50.594 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047450432000000001 2024-09-14T06:29:50.594 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:50.594 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:50.594 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:29:50.594 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.417935+0000", 2024-09-14T06:29:50.594 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.7454999999999999e-05 2024-09-14T06:29:50.594 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:50.595 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:50.595 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:29:50.595 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.417987+0000", 2024-09-14T06:29:50.595 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.2213999999999998e-05 2024-09-14T06:29:50.595 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:50.595 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:29:50.595 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:50.595 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:50.595 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:50.595 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:53341 3.f 3:f71f51a7:::rbd_data.112dca2f6ae1.0000000000000015:17 [read 1081344~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:29:50.595 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:44.370446+0000", 2024-09-14T06:29:50.595 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 6.168720242, 2024-09-14T06:29:50.595 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047665723, 2024-09-14T06:29:50.595 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:29:50.596 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:29:50.596 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:29:50.596 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:29:50.596 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:29:50.596 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:50.596 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 53341 2024-09-14T06:29:50.596 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:50.596 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:29:50.596 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:50.596 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:29:50.596 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370446+0000", 2024-09-14T06:29:50.596 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:50.596 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:50.596 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:50.597 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:29:50.597 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370446+0000", 2024-09-14T06:29:50.597 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:50.597 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:50.597 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:50.597 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:29:50.597 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370447+0000", 2024-09-14T06:29:50.597 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.6100000000000002e-07 2024-09-14T06:29:50.597 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:50.597 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:50.597 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:29:50.597 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370448+0000", 2024-09-14T06:29:50.597 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.7370000000000001e-06 2024-09-14T06:29:50.597 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:50.597 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:50.598 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:29:50.598 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370462+0000", 2024-09-14T06:29:50.598 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.3227000000000001e-05 2024-09-14T06:29:50.598 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:50.598 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:50.598 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:29:50.598 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370467+0000", 2024-09-14T06:29:50.598 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.6540000000000004e-06 2024-09-14T06:29:50.598 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:50.598 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:50.598 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:29:50.598 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.418013+0000", 2024-09-14T06:29:50.598 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047545325999999999 2024-09-14T06:29:50.598 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:50.599 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:50.599 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:29:50.599 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.418054+0000", 2024-09-14T06:29:50.599 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.1696999999999997e-05 2024-09-14T06:29:50.599 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:50.599 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:50.599 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:29:50.599 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.418112+0000", 2024-09-14T06:29:50.599 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.7720999999999998e-05 2024-09-14T06:29:50.599 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:50.599 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:29:50.599 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:50.599 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:50.600 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:50.600 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:53342 3.f 3:f71f51a7:::rbd_data.112dca2f6ae1.0000000000000015:17 [read 1105920~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:29:50.600 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:44.370475+0000", 2024-09-14T06:29:50.600 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 6.1686911809999998, 2024-09-14T06:29:50.600 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047748848000000003, 2024-09-14T06:29:50.600 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:29:50.600 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:29:50.600 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:29:50.600 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:29:50.600 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:29:50.600 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:50.600 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 53342 2024-09-14T06:29:50.600 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:50.600 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:29:50.600 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:50.601 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:29:50.601 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370475+0000", 2024-09-14T06:29:50.601 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:50.601 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:50.601 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:50.601 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:29:50.601 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370475+0000", 2024-09-14T06:29:50.601 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:50.601 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:50.601 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:50.601 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:29:50.601 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370476+0000", 2024-09-14T06:29:50.601 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.4700000000000002e-07 2024-09-14T06:29:50.602 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:50.602 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:50.602 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:29:50.602 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370478+0000", 2024-09-14T06:29:50.602 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.5890000000000001e-06 2024-09-14T06:29:50.602 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:50.602 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:50.602 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:29:50.602 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370486+0000", 2024-09-14T06:29:50.602 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.3470000000000001e-06 2024-09-14T06:29:50.602 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:50.602 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:50.602 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:29:50.602 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370491+0000", 2024-09-14T06:29:50.602 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.5720000000000002e-06 2024-09-14T06:29:50.603 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:50.603 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:50.603 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:29:50.603 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.418135+0000", 2024-09-14T06:29:50.603 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047644094999999997 2024-09-14T06:29:50.603 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:50.603 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:50.603 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:29:50.603 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.418174+0000", 2024-09-14T06:29:50.603 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.8204000000000002e-05 2024-09-14T06:29:50.603 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:50.603 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:50.603 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:29:50.603 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.418224+0000", 2024-09-14T06:29:50.603 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.0593999999999997e-05 2024-09-14T06:29:50.604 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:50.604 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:29:50.604 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:50.604 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:50.604 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:50.604 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:53343 3.f 3:f71f51a7:::rbd_data.112dca2f6ae1.0000000000000015:17 [read 1130496~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:29:50.604 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:44.370498+0000", 2024-09-14T06:29:50.604 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 6.1686683049999997, 2024-09-14T06:29:50.604 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047841574999999997, 2024-09-14T06:29:50.604 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:29:50.604 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:29:50.604 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:29:50.604 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:29:50.604 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:29:50.605 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:50.605 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 53343 2024-09-14T06:29:50.605 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:50.605 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:29:50.605 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:50.605 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:29:50.605 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370498+0000", 2024-09-14T06:29:50.605 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:50.605 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:50.605 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:50.605 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:29:50.605 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370498+0000", 2024-09-14T06:29:50.605 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:50.605 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:50.606 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:50.606 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:29:50.606 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370499+0000", 2024-09-14T06:29:50.606 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.9999999999999998e-07 2024-09-14T06:29:50.606 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:50.606 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:50.606 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:29:50.606 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370501+0000", 2024-09-14T06:29:50.606 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.0310000000000001e-06 2024-09-14T06:29:50.606 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:50.606 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:50.606 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:29:50.606 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370507+0000", 2024-09-14T06:29:50.607 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.4169999999999997e-06 2024-09-14T06:29:50.607 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:50.607 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:50.607 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:29:50.607 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370516+0000", 2024-09-14T06:29:50.607 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.7190000000000007e-06 2024-09-14T06:29:50.607 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:50.607 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:50.607 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:29:50.607 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.418247+0000", 2024-09-14T06:29:50.607 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047731190999999999 2024-09-14T06:29:50.607 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:50.607 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:50.607 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:29:50.607 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.418286+0000", 2024-09-14T06:29:50.608 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.8988e-05 2024-09-14T06:29:50.608 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:50.608 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:50.608 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:29:50.608 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.418340+0000", 2024-09-14T06:29:50.608 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.3829000000000002e-05 2024-09-14T06:29:50.608 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:50.608 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:29:50.608 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:50.608 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:50.608 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:50.608 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:53344 3.f 3:f71f51a7:::rbd_data.112dca2f6ae1.0000000000000015:17 [read 1155072~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:29:50.608 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:44.370524+0000", 2024-09-14T06:29:50.608 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 6.168642535, 2024-09-14T06:29:50.609 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047972262000000002, 2024-09-14T06:29:50.609 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:29:50.609 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:29:50.609 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:29:50.609 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:29:50.609 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:29:50.609 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:50.609 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 53344 2024-09-14T06:29:50.609 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:50.609 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:29:50.609 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:50.609 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:29:50.609 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370524+0000", 2024-09-14T06:29:50.609 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:50.609 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:50.610 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:50.610 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:29:50.610 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370524+0000", 2024-09-14T06:29:50.610 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:50.610 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:50.610 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:50.610 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:29:50.610 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370618+0000", 2024-09-14T06:29:50.610 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.4425999999999993e-05 2024-09-14T06:29:50.610 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:50.610 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:50.610 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:29:50.610 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370622+0000", 2024-09-14T06:29:50.610 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.3519999999999998e-06 2024-09-14T06:29:50.610 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:50.610 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:50.611 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:29:50.611 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370631+0000", 2024-09-14T06:29:50.611 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.9299999999999992e-06 2024-09-14T06:29:50.611 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:50.611 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:50.611 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:29:50.611 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370637+0000", 2024-09-14T06:29:50.611 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.3269999999999997e-06 2024-09-14T06:29:50.611 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:50.611 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:50.611 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:29:50.611 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.418372+0000", 2024-09-14T06:29:50.611 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047734764999999998 2024-09-14T06:29:50.611 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:50.611 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:50.612 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:29:50.612 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.418431+0000", 2024-09-14T06:29:50.612 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.8696999999999997e-05 2024-09-14T06:29:50.612 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:50.612 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:50.612 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:29:50.612 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.418496+0000", 2024-09-14T06:29:50.612 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.5765000000000005e-05 2024-09-14T06:29:50.612 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:50.612 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:29:50.612 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:50.612 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:50.612 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:50.612 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:53345 3.f 3:f71f51a7:::rbd_data.112dca2f6ae1.0000000000000015:17 [read 1179648~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:29:50.612 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:44.370647+0000", 2024-09-14T06:29:50.613 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 6.1685190680000002, 2024-09-14T06:29:50.613 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047953775999999997, 2024-09-14T06:29:50.613 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:29:50.613 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:29:50.613 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:29:50.613 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:29:50.613 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:29:50.613 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:50.613 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 53345 2024-09-14T06:29:50.613 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:50.613 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:29:50.613 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:50.613 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:29:50.614 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370647+0000", 2024-09-14T06:29:50.614 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:50.614 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:50.614 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:50.614 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:29:50.614 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370647+0000", 2024-09-14T06:29:50.614 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:50.614 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:50.614 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:50.614 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:29:50.614 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.372346+0000", 2024-09-14T06:29:50.614 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.001698284 2024-09-14T06:29:50.614 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:50.614 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:50.614 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:29:50.615 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.372351+0000", 2024-09-14T06:29:50.615 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.8890000000000001e-06 2024-09-14T06:29:50.615 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:50.615 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:50.615 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:29:50.615 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.372362+0000", 2024-09-14T06:29:50.615 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.0941000000000001e-05 2024-09-14T06:29:50.615 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:50.615 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:50.615 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:29:50.615 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.372369+0000", 2024-09-14T06:29:50.615 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.362e-06 2024-09-14T06:29:50.615 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:50.615 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:50.616 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:29:50.616 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.418510+0000", 2024-09-14T06:29:50.616 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046140914999999998 2024-09-14T06:29:50.616 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:50.616 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:50.616 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:29:50.616 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.418549+0000", 2024-09-14T06:29:50.616 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.8717999999999998e-05 2024-09-14T06:29:50.616 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:50.616 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:50.616 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:29:50.616 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.418601+0000", 2024-09-14T06:29:50.616 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.2667000000000002e-05 2024-09-14T06:29:50.616 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:50.616 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:29:50.617 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:50.617 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:50.617 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:50.617 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:53351 3.f 3:f71f51a7:::rbd_data.112dca2f6ae1.0000000000000015:17 [read 1327104~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:29:50.617 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:44.372499+0000", 2024-09-14T06:29:50.617 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 6.1666679469999996, 2024-09-14T06:29:50.617 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047140596, 2024-09-14T06:29:50.617 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:29:50.617 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:29:50.617 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:29:50.617 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:29:50.617 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:29:50.617 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:50.618 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 53351 2024-09-14T06:29:50.618 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:50.618 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:29:50.618 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:50.618 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:29:50.618 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.372499+0000", 2024-09-14T06:29:50.618 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:50.618 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:50.618 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:50.618 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:29:50.618 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.372499+0000", 2024-09-14T06:29:50.618 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:50.618 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:50.618 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:50.618 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:29:50.619 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.372499+0000", 2024-09-14T06:29:50.619 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.3000000000000001e-07 2024-09-14T06:29:50.619 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:50.619 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:50.619 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:29:50.619 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.372501+0000", 2024-09-14T06:29:50.619 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.0820000000000001e-06 2024-09-14T06:29:50.619 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:50.619 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:50.619 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:29:50.619 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.372513+0000", 2024-09-14T06:29:50.619 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.1610999999999999e-05 2024-09-14T06:29:50.619 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:50.620 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:50.620 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:29:50.620 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.372518+0000", 2024-09-14T06:29:50.620 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.327e-06 2024-09-14T06:29:50.620 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:50.620 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:50.620 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:29:50.620 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.419530+0000", 2024-09-14T06:29:50.620 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047011798 2024-09-14T06:29:50.620 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:50.620 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:50.620 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:29:50.620 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.419578+0000", 2024-09-14T06:29:50.620 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.8446999999999999e-05 2024-09-14T06:29:50.620 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:50.621 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:50.621 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:29:50.621 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.419639+0000", 2024-09-14T06:29:50.621 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.0900999999999998e-05 2024-09-14T06:29:50.621 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:50.621 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:29:50.621 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:50.621 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:50.621 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:50.621 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:53352 3.f 3:f71f51a7:::rbd_data.112dca2f6ae1.0000000000000015:17 [read 1351680~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:29:50.621 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:44.372525+0000", 2024-09-14T06:29:50.621 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 6.1666420329999996, 2024-09-14T06:29:50.621 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047259847000000001, 2024-09-14T06:29:50.621 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:29:50.622 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:29:50.622 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:29:50.622 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:29:50.622 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:29:50.622 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:50.622 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 53352 2024-09-14T06:29:50.622 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:50.622 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:29:50.622 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:50.622 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:29:50.622 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.372525+0000", 2024-09-14T06:29:50.622 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:50.622 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:50.622 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:50.623 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:29:50.623 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.372525+0000", 2024-09-14T06:29:50.623 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:50.623 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:50.623 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:50.623 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:29:50.623 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.372525+0000", 2024-09-14T06:29:50.623 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.0600000000000001e-07 2024-09-14T06:29:50.623 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:50.623 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:50.623 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:29:50.623 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.372527+0000", 2024-09-14T06:29:50.623 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.0499999999999999e-06 2024-09-14T06:29:50.623 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:50.623 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:50.624 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:29:50.624 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.372533+0000", 2024-09-14T06:29:50.624 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.4320000000000004e-06 2024-09-14T06:29:50.624 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:50.624 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:50.624 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:29:50.624 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.372538+0000", 2024-09-14T06:29:50.624 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.9130000000000002e-06 2024-09-14T06:29:50.624 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:50.624 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:50.624 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:29:50.624 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.419667+0000", 2024-09-14T06:29:50.624 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047128976000000003 2024-09-14T06:29:50.624 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:50.625 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:50.625 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:29:50.625 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.419715+0000", 2024-09-14T06:29:50.625 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.7868999999999999e-05 2024-09-14T06:29:50.625 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:50.625 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:50.625 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:29:50.625 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.419784+0000", 2024-09-14T06:29:50.625 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.9201000000000004e-05 2024-09-14T06:29:50.625 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:50.625 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:29:50.625 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:50.625 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:50.625 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:50.625 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:53353 3.f 3:f71f51a7:::rbd_data.112dca2f6ae1.0000000000000015:17 [read 1376256~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:29:50.626 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:44.372545+0000", 2024-09-14T06:29:50.626 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 6.1666219929999997, 2024-09-14T06:29:50.626 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047364232999999999, 2024-09-14T06:29:50.626 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:29:50.626 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:29:50.626 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:29:50.626 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:29:50.626 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:29:50.626 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:50.626 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 53353 2024-09-14T06:29:50.626 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:50.626 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:29:50.626 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:50.627 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:29:50.627 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.372545+0000", 2024-09-14T06:29:50.627 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:50.627 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:50.627 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:50.627 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:29:50.627 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.372545+0000", 2024-09-14T06:29:50.627 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:50.627 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:50.627 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:50.627 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:29:50.627 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.372545+0000", 2024-09-14T06:29:50.627 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.27e-07 2024-09-14T06:29:50.627 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:50.628 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:50.628 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:29:50.628 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.372547+0000", 2024-09-14T06:29:50.628 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.0600000000000002e-06 2024-09-14T06:29:50.628 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:50.628 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:50.628 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:29:50.628 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.372553+0000", 2024-09-14T06:29:50.628 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.7980000000000002e-06 2024-09-14T06:29:50.628 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:50.628 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:50.629 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:29:50.629 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.372557+0000", 2024-09-14T06:29:50.629 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.7389999999999999e-06 2024-09-14T06:29:50.629 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:50.629 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:50.629 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:29:50.629 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.419811+0000", 2024-09-14T06:29:50.629 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047253310999999999 2024-09-14T06:29:50.629 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:50.629 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:50.629 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:29:50.629 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.419854+0000", 2024-09-14T06:29:50.629 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.3448000000000001e-05 2024-09-14T06:29:50.630 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:50.630 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:50.630 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:29:50.630 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.419909+0000", 2024-09-14T06:29:50.630 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.4549999999999998e-05 2024-09-14T06:29:50.630 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:50.630 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:29:50.630 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:50.630 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:50.630 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:50.630 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:53354 3.f 3:f71f51a7:::rbd_data.112dca2f6ae1.0000000000000015:17 [read 1400832~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:29:50.630 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:44.372563+0000", 2024-09-14T06:29:50.630 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 6.1666035560000001, 2024-09-14T06:29:50.630 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047482233999999998, 2024-09-14T06:29:50.631 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:29:50.631 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:29:50.631 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:29:50.631 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:29:50.631 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:29:50.631 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:50.631 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 53354 2024-09-14T06:29:50.631 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:50.631 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:29:50.631 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:50.631 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:29:50.631 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.372563+0000", 2024-09-14T06:29:50.631 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:50.631 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:50.631 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:50.632 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:29:50.632 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.372563+0000", 2024-09-14T06:29:50.632 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:50.632 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:50.632 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:50.632 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:29:50.632 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.375009+0000", 2024-09-14T06:29:50.632 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.0024457680000000001 2024-09-14T06:29:50.632 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:50.632 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:50.632 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:29:50.632 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.375014+0000", 2024-09-14T06:29:50.632 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.0599999999999998e-06 2024-09-14T06:29:50.633 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:50.633 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:50.633 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:29:50.633 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.375026+0000", 2024-09-14T06:29:50.633 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.1783999999999999e-05 2024-09-14T06:29:50.633 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:50.633 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:50.633 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:29:50.633 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.375035+0000", 2024-09-14T06:29:50.633 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.7799999999999995e-06 2024-09-14T06:29:50.633 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:50.633 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:50.633 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:29:50.633 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.419942+0000", 2024-09-14T06:29:50.634 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.044906982999999998 2024-09-14T06:29:50.634 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:50.634 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:50.634 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:29:50.634 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.419989+0000", 2024-09-14T06:29:50.634 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.7104999999999997e-05 2024-09-14T06:29:50.634 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:50.634 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:50.634 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:29:50.634 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.420045+0000", 2024-09-14T06:29:50.634 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.5754000000000002e-05 2024-09-14T06:29:50.634 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:50.634 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:29:50.634 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:50.634 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:50.635 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:29:50.635 INFO:teuthology.orchestra.run.smithi138.stdout:} 2024-09-14T06:29:50.635 INFO:teuthology.orchestra.run.smithi094.stdout:osd.2: {} 2024-09-14T06:29:50.636 INFO:teuthology.orchestra.run.smithi094.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-14T06:29:50.636 INFO:teuthology.orchestra.run.smithi094.stdout:osd.3: {} 2024-09-14T06:29:50.637 INFO:teuthology.orchestra.run.smithi094.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-14T06:29:50.637 INFO:tasks.ceph.osd.5.smithi138.stderr:2024-09-14T06:29:50.634+0000 7fe13746f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33618) UID: 0 2024-09-14T06:29:50.637 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-14T06:29:50.638 INFO:teuthology.orchestra.run.smithi094.stdout:osd.4: {} 2024-09-14T06:29:50.638 INFO:teuthology.orchestra.run.smithi094.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-14T06:29:50.638 INFO:teuthology.orchestra.run.smithi094.stdout:osd.5: {} 2024-09-14T06:29:50.639 INFO:teuthology.orchestra.run.smithi094.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-14T06:29:50.639 INFO:teuthology.orchestra.run.smithi094.stdout:osd.6: {} 2024-09-14T06:29:50.639 INFO:teuthology.orchestra.run.smithi094.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-14T06:29:50.639 INFO:teuthology.orchestra.run.smithi094.stdout:osd.7: {} 2024-09-14T06:29:50.640 INFO:teuthology.orchestra.run.smithi094.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-14T06:29:50.737 INFO:tasks.ceph.osd.4.smithi138.stderr:2024-09-14T06:29:50.735+0000 7f6c653c8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33615) UID: 0 2024-09-14T06:29:50.755 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-14T06:29:50.755 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-14T06:29:50.756 INFO:tasks.thrashosds.thrasher:thrash_pg_upmap_items 2024-09-14T06:29:50.756 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump -f json-pretty 2024-09-14T06:29:50.773 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-14T06:29:50.791 INFO:teuthology.orchestra.run.smithi094.stdout:ERROR: (22) Invalid argument 2024-09-14T06:29:50.791 INFO:teuthology.orchestra.run.smithi094.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:29:50.805 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-14T06:29:50.847 INFO:tasks.ceph.osd.0.smithi094.stderr:2024-09-14T06:29:50.846+0000 7f07c9333640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33636) UID: 0 2024-09-14T06:29:50.940 INFO:tasks.ceph.osd.1.smithi094.stderr:2024-09-14T06:29:50.938+0000 7f328e47a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33641) UID: 0 2024-09-14T06:29:51.033 INFO:teuthology.orchestra.run.smithi094.stdout:ERROR: (22) Invalid argument 2024-09-14T06:29:51.033 INFO:teuthology.orchestra.run.smithi094.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:29:51.039 INFO:tasks.ceph.osd.3.smithi094.stderr:2024-09-14T06:29:51.038+0000 7feeeafe8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33655) UID: 0 2024-09-14T06:29:51.048 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-14T06:29:51.140 INFO:tasks.ceph.osd.6.smithi138.stderr:2024-09-14T06:29:51.138+0000 7f8cd8849640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33620) UID: 0 2024-09-14T06:29:51.231 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-14T06:29:51.231 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-14T06:29:51.231 INFO:teuthology.orchestra.run.smithi094.stdout: "epoch": 70, 2024-09-14T06:29:51.231 INFO:teuthology.orchestra.run.smithi094.stdout: "fsid": "200ad3da-bd63-4e04-af74-efb5ca0af75e", 2024-09-14T06:29:51.231 INFO:teuthology.orchestra.run.smithi094.stdout: "created": "2024-09-14T06:28:30.782046+0000", 2024-09-14T06:29:51.231 INFO:teuthology.orchestra.run.smithi094.stdout: "modified": "2024-09-14T06:29:50.575617+0000", 2024-09-14T06:29:51.231 INFO:teuthology.orchestra.run.smithi094.stdout: "last_up_change": "2024-09-14T06:28:42.827239+0000", 2024-09-14T06:29:51.231 INFO:teuthology.orchestra.run.smithi094.stdout: "last_in_change": "2024-09-14T06:28:33.724176+0000", 2024-09-14T06:29:51.231 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": "noscrub,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2024-09-14T06:29:51.232 INFO:teuthology.orchestra.run.smithi094.stdout: "flags_num": 5801984, 2024-09-14T06:29:51.232 INFO:teuthology.orchestra.run.smithi094.stdout: "flags_set": [ 2024-09-14T06:29:51.232 INFO:teuthology.orchestra.run.smithi094.stdout: "noscrub", 2024-09-14T06:29:51.232 INFO:teuthology.orchestra.run.smithi094.stdout: "pglog_hardlimit", 2024-09-14T06:29:51.232 INFO:teuthology.orchestra.run.smithi094.stdout: "purged_snapdirs", 2024-09-14T06:29:51.232 INFO:teuthology.orchestra.run.smithi094.stdout: "recovery_deletes", 2024-09-14T06:29:51.232 INFO:teuthology.orchestra.run.smithi094.stdout: "sortbitwise" 2024-09-14T06:29:51.232 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-09-14T06:29:51.232 INFO:teuthology.orchestra.run.smithi094.stdout: "crush_version": 5, 2024-09-14T06:29:51.232 INFO:teuthology.orchestra.run.smithi094.stdout: "full_ratio": 0.94999998807907104, 2024-09-14T06:29:51.232 INFO:teuthology.orchestra.run.smithi094.stdout: "backfillfull_ratio": 0.89999997615814209, 2024-09-14T06:29:51.232 INFO:teuthology.orchestra.run.smithi094.stdout: "nearfull_ratio": 0.85000002384185791, 2024-09-14T06:29:51.232 INFO:teuthology.orchestra.run.smithi094.stdout: "cluster_snapshot": "", 2024-09-14T06:29:51.233 INFO:teuthology.orchestra.run.smithi094.stdout: "pool_max": 3, 2024-09-14T06:29:51.233 INFO:teuthology.orchestra.run.smithi094.stdout: "max_osd": 8, 2024-09-14T06:29:51.233 INFO:teuthology.orchestra.run.smithi094.stdout: "require_min_compat_client": "luminous", 2024-09-14T06:29:51.233 INFO:teuthology.orchestra.run.smithi094.stdout: "min_compat_client": "jewel", 2024-09-14T06:29:51.233 INFO:teuthology.orchestra.run.smithi094.stdout: "require_osd_release": "squid", 2024-09-14T06:29:51.233 INFO:teuthology.orchestra.run.smithi094.stdout: "allow_crimson": false, 2024-09-14T06:29:51.233 INFO:teuthology.orchestra.run.smithi094.stdout: "pools": [ 2024-09-14T06:29:51.233 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:51.233 INFO:teuthology.orchestra.run.smithi094.stdout: "pool": 1, 2024-09-14T06:29:51.233 INFO:teuthology.orchestra.run.smithi094.stdout: "pool_name": ".mgr", 2024-09-14T06:29:51.233 INFO:teuthology.orchestra.run.smithi094.stdout: "create_time": "2024-09-14T06:28:43.295990+0000", 2024-09-14T06:29:51.233 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-14T06:29:51.233 INFO:teuthology.orchestra.run.smithi094.stdout: "flags_names": "hashpspool", 2024-09-14T06:29:51.234 INFO:teuthology.orchestra.run.smithi094.stdout: "type": 1, 2024-09-14T06:29:51.234 INFO:teuthology.orchestra.run.smithi094.stdout: "size": 2, 2024-09-14T06:29:51.234 INFO:teuthology.orchestra.run.smithi094.stdout: "min_size": 1, 2024-09-14T06:29:51.234 INFO:teuthology.orchestra.run.smithi094.stdout: "crush_rule": 0, 2024-09-14T06:29:51.234 INFO:teuthology.orchestra.run.smithi094.stdout: "peering_crush_bucket_count": 0, 2024-09-14T06:29:51.234 INFO:teuthology.orchestra.run.smithi094.stdout: "peering_crush_bucket_target": 0, 2024-09-14T06:29:51.234 INFO:teuthology.orchestra.run.smithi094.stdout: "peering_crush_bucket_barrier": 0, 2024-09-14T06:29:51.234 INFO:teuthology.orchestra.run.smithi094.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-14T06:29:51.234 INFO:teuthology.orchestra.run.smithi094.stdout: "is_stretch_pool": false, 2024-09-14T06:29:51.234 INFO:teuthology.orchestra.run.smithi094.stdout: "object_hash": 2, 2024-09-14T06:29:51.234 INFO:teuthology.orchestra.run.smithi094.stdout: "pg_autoscale_mode": "off", 2024-09-14T06:29:51.234 INFO:teuthology.orchestra.run.smithi094.stdout: "pg_num": 1, 2024-09-14T06:29:51.234 INFO:teuthology.orchestra.run.smithi094.stdout: "pg_placement_num": 1, 2024-09-14T06:29:51.234 INFO:teuthology.orchestra.run.smithi094.stdout: "pg_placement_num_target": 1, 2024-09-14T06:29:51.235 INFO:teuthology.orchestra.run.smithi094.stdout: "pg_num_target": 1, 2024-09-14T06:29:51.235 INFO:teuthology.orchestra.run.smithi094.stdout: "pg_num_pending": 1, 2024-09-14T06:29:51.235 INFO:teuthology.orchestra.run.smithi094.stdout: "last_pg_merge_meta": { 2024-09-14T06:29:51.235 INFO:teuthology.orchestra.run.smithi094.stdout: "source_pgid": "0.0", 2024-09-14T06:29:51.235 INFO:teuthology.orchestra.run.smithi094.stdout: "ready_epoch": 0, 2024-09-14T06:29:51.235 INFO:teuthology.orchestra.run.smithi094.stdout: "last_epoch_started": 0, 2024-09-14T06:29:51.235 INFO:teuthology.orchestra.run.smithi094.stdout: "last_epoch_clean": 0, 2024-09-14T06:29:51.235 INFO:teuthology.orchestra.run.smithi094.stdout: "source_version": "0'0", 2024-09-14T06:29:51.235 INFO:teuthology.orchestra.run.smithi094.stdout: "target_version": "0'0" 2024-09-14T06:29:51.235 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:51.235 INFO:teuthology.orchestra.run.smithi094.stdout: "last_change": "18", 2024-09-14T06:29:51.235 INFO:teuthology.orchestra.run.smithi094.stdout: "last_force_op_resend": "0", 2024-09-14T06:29:51.235 INFO:teuthology.orchestra.run.smithi094.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-14T06:29:51.235 INFO:teuthology.orchestra.run.smithi094.stdout: "last_force_op_resend_preluminous": "0", 2024-09-14T06:29:51.236 INFO:teuthology.orchestra.run.smithi094.stdout: "auid": 0, 2024-09-14T06:29:51.236 INFO:teuthology.orchestra.run.smithi094.stdout: "snap_mode": "selfmanaged", 2024-09-14T06:29:51.236 INFO:teuthology.orchestra.run.smithi094.stdout: "snap_seq": 0, 2024-09-14T06:29:51.236 INFO:teuthology.orchestra.run.smithi094.stdout: "snap_epoch": 0, 2024-09-14T06:29:51.236 INFO:teuthology.orchestra.run.smithi094.stdout: "pool_snaps": [], 2024-09-14T06:29:51.236 INFO:teuthology.orchestra.run.smithi094.stdout: "removed_snaps": "[]", 2024-09-14T06:29:51.236 INFO:teuthology.orchestra.run.smithi094.stdout: "quota_max_bytes": 0, 2024-09-14T06:29:51.236 INFO:teuthology.orchestra.run.smithi094.stdout: "quota_max_objects": 0, 2024-09-14T06:29:51.236 INFO:teuthology.orchestra.run.smithi094.stdout: "tiers": [], 2024-09-14T06:29:51.236 INFO:teuthology.orchestra.run.smithi094.stdout: "tier_of": -1, 2024-09-14T06:29:51.236 INFO:teuthology.orchestra.run.smithi094.stdout: "read_tier": -1, 2024-09-14T06:29:51.236 INFO:teuthology.orchestra.run.smithi094.stdout: "write_tier": -1, 2024-09-14T06:29:51.236 INFO:teuthology.orchestra.run.smithi094.stdout: "cache_mode": "none", 2024-09-14T06:29:51.236 INFO:teuthology.orchestra.run.smithi094.stdout: "target_max_bytes": 0, 2024-09-14T06:29:51.236 INFO:teuthology.orchestra.run.smithi094.stdout: "target_max_objects": 0, 2024-09-14T06:29:51.237 INFO:teuthology.orchestra.run.smithi094.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-14T06:29:51.237 INFO:teuthology.orchestra.run.smithi094.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-14T06:29:51.237 INFO:teuthology.orchestra.run.smithi094.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-14T06:29:51.237 INFO:teuthology.orchestra.run.smithi094.stdout: "cache_min_flush_age": 0, 2024-09-14T06:29:51.237 INFO:teuthology.orchestra.run.smithi094.stdout: "cache_min_evict_age": 0, 2024-09-14T06:29:51.237 INFO:teuthology.orchestra.run.smithi094.stdout: "erasure_code_profile": "", 2024-09-14T06:29:51.237 INFO:teuthology.orchestra.run.smithi094.stdout: "hit_set_params": { 2024-09-14T06:29:51.237 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "none" 2024-09-14T06:29:51.237 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:51.237 INFO:teuthology.orchestra.run.smithi094.stdout: "hit_set_period": 0, 2024-09-14T06:29:51.237 INFO:teuthology.orchestra.run.smithi094.stdout: "hit_set_count": 0, 2024-09-14T06:29:51.237 INFO:teuthology.orchestra.run.smithi094.stdout: "use_gmt_hitset": true, 2024-09-14T06:29:51.237 INFO:teuthology.orchestra.run.smithi094.stdout: "min_read_recency_for_promote": 0, 2024-09-14T06:29:51.237 INFO:teuthology.orchestra.run.smithi094.stdout: "min_write_recency_for_promote": 0, 2024-09-14T06:29:51.238 INFO:teuthology.orchestra.run.smithi094.stdout: "hit_set_grade_decay_rate": 0, 2024-09-14T06:29:51.238 INFO:teuthology.orchestra.run.smithi094.stdout: "hit_set_search_last_n": 0, 2024-09-14T06:29:51.238 INFO:teuthology.orchestra.run.smithi094.stdout: "grade_table": [], 2024-09-14T06:29:51.238 INFO:teuthology.orchestra.run.smithi094.stdout: "stripe_width": 0, 2024-09-14T06:29:51.238 INFO:teuthology.orchestra.run.smithi094.stdout: "expected_num_objects": 0, 2024-09-14T06:29:51.238 INFO:teuthology.orchestra.run.smithi094.stdout: "fast_read": false, 2024-09-14T06:29:51.238 INFO:teuthology.orchestra.run.smithi094.stdout: "options": { 2024-09-14T06:29:51.238 INFO:teuthology.orchestra.run.smithi094.stdout: "pg_num_max": 32, 2024-09-14T06:29:51.238 INFO:teuthology.orchestra.run.smithi094.stdout: "pg_num_min": 1 2024-09-14T06:29:51.238 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:51.238 INFO:teuthology.orchestra.run.smithi094.stdout: "application_metadata": { 2024-09-14T06:29:51.238 INFO:teuthology.orchestra.run.smithi094.stdout: "mgr": {} 2024-09-14T06:29:51.238 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:51.238 INFO:teuthology.orchestra.run.smithi094.stdout: "read_balance": { 2024-09-14T06:29:51.238 INFO:teuthology.orchestra.run.smithi094.stdout: "score_type": "Fair distribution", 2024-09-14T06:29:51.239 INFO:teuthology.orchestra.run.smithi094.stdout: "score_acting": 8, 2024-09-14T06:29:51.239 INFO:teuthology.orchestra.run.smithi094.stdout: "score_stable": 8, 2024-09-14T06:29:51.239 INFO:teuthology.orchestra.run.smithi094.stdout: "optimal_score": 0.25, 2024-09-14T06:29:51.239 INFO:teuthology.orchestra.run.smithi094.stdout: "raw_score_acting": 2, 2024-09-14T06:29:51.239 INFO:teuthology.orchestra.run.smithi094.stdout: "raw_score_stable": 2, 2024-09-14T06:29:51.239 INFO:teuthology.orchestra.run.smithi094.stdout: "primary_affinity_weighted": 1, 2024-09-14T06:29:51.239 INFO:teuthology.orchestra.run.smithi094.stdout: "average_primary_affinity": 1, 2024-09-14T06:29:51.239 INFO:teuthology.orchestra.run.smithi094.stdout: "average_primary_affinity_weighted": 1 2024-09-14T06:29:51.239 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:51.239 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:51.239 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:51.239 INFO:teuthology.orchestra.run.smithi094.stdout: "pool": 2, 2024-09-14T06:29:51.239 INFO:teuthology.orchestra.run.smithi094.stdout: "pool_name": "rbd", 2024-09-14T06:29:51.239 INFO:teuthology.orchestra.run.smithi094.stdout: "create_time": "2024-09-14T06:28:49.451714+0000", 2024-09-14T06:29:51.240 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 8193, 2024-09-14T06:29:51.240 INFO:teuthology.orchestra.run.smithi094.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-09-14T06:29:51.240 INFO:teuthology.orchestra.run.smithi094.stdout: "type": 1, 2024-09-14T06:29:51.240 INFO:teuthology.orchestra.run.smithi094.stdout: "size": 2, 2024-09-14T06:29:51.240 INFO:teuthology.orchestra.run.smithi094.stdout: "min_size": 1, 2024-09-14T06:29:51.240 INFO:teuthology.orchestra.run.smithi094.stdout: "crush_rule": 0, 2024-09-14T06:29:51.240 INFO:teuthology.orchestra.run.smithi094.stdout: "peering_crush_bucket_count": 0, 2024-09-14T06:29:51.240 INFO:teuthology.orchestra.run.smithi094.stdout: "peering_crush_bucket_target": 0, 2024-09-14T06:29:51.240 INFO:teuthology.orchestra.run.smithi094.stdout: "peering_crush_bucket_barrier": 0, 2024-09-14T06:29:51.240 INFO:teuthology.orchestra.run.smithi094.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-14T06:29:51.240 INFO:teuthology.orchestra.run.smithi094.stdout: "is_stretch_pool": false, 2024-09-14T06:29:51.240 INFO:teuthology.orchestra.run.smithi094.stdout: "object_hash": 2, 2024-09-14T06:29:51.240 INFO:teuthology.orchestra.run.smithi094.stdout: "pg_autoscale_mode": "off", 2024-09-14T06:29:51.240 INFO:teuthology.orchestra.run.smithi094.stdout: "pg_num": 8, 2024-09-14T06:29:51.241 INFO:teuthology.orchestra.run.smithi094.stdout: "pg_placement_num": 8, 2024-09-14T06:29:51.241 INFO:teuthology.orchestra.run.smithi094.stdout: "pg_placement_num_target": 8, 2024-09-14T06:29:51.241 INFO:teuthology.orchestra.run.smithi094.stdout: "pg_num_target": 8, 2024-09-14T06:29:51.241 INFO:teuthology.orchestra.run.smithi094.stdout: "pg_num_pending": 8, 2024-09-14T06:29:51.241 INFO:teuthology.orchestra.run.smithi094.stdout: "last_pg_merge_meta": { 2024-09-14T06:29:51.241 INFO:teuthology.orchestra.run.smithi094.stdout: "source_pgid": "0.0", 2024-09-14T06:29:51.241 INFO:teuthology.orchestra.run.smithi094.stdout: "ready_epoch": 0, 2024-09-14T06:29:51.241 INFO:teuthology.orchestra.run.smithi094.stdout: "last_epoch_started": 0, 2024-09-14T06:29:51.241 INFO:teuthology.orchestra.run.smithi094.stdout: "last_epoch_clean": 0, 2024-09-14T06:29:51.241 INFO:teuthology.orchestra.run.smithi094.stdout: "source_version": "0'0", 2024-09-14T06:29:51.241 INFO:teuthology.orchestra.run.smithi094.stdout: "target_version": "0'0" 2024-09-14T06:29:51.241 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:51.241 INFO:teuthology.orchestra.run.smithi094.stdout: "last_change": "22", 2024-09-14T06:29:51.241 INFO:teuthology.orchestra.run.smithi094.stdout: "last_force_op_resend": "0", 2024-09-14T06:29:51.241 INFO:teuthology.orchestra.run.smithi094.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-14T06:29:51.242 INFO:teuthology.orchestra.run.smithi094.stdout: "last_force_op_resend_preluminous": "0", 2024-09-14T06:29:51.242 INFO:teuthology.orchestra.run.smithi094.stdout: "auid": 0, 2024-09-14T06:29:51.242 INFO:teuthology.orchestra.run.smithi094.stdout: "snap_mode": "selfmanaged", 2024-09-14T06:29:51.242 INFO:teuthology.orchestra.run.smithi094.stdout: "snap_seq": 2, 2024-09-14T06:29:51.242 INFO:teuthology.orchestra.run.smithi094.stdout: "snap_epoch": 22, 2024-09-14T06:29:51.242 INFO:teuthology.orchestra.run.smithi094.stdout: "pool_snaps": [], 2024-09-14T06:29:51.242 INFO:teuthology.orchestra.run.smithi094.stdout: "removed_snaps": "[]", 2024-09-14T06:29:51.242 INFO:teuthology.orchestra.run.smithi094.stdout: "quota_max_bytes": 0, 2024-09-14T06:29:51.242 INFO:teuthology.orchestra.run.smithi094.stdout: "quota_max_objects": 0, 2024-09-14T06:29:51.242 INFO:teuthology.orchestra.run.smithi094.stdout: "tiers": [], 2024-09-14T06:29:51.242 INFO:teuthology.orchestra.run.smithi094.stdout: "tier_of": -1, 2024-09-14T06:29:51.242 INFO:teuthology.orchestra.run.smithi094.stdout: "read_tier": -1, 2024-09-14T06:29:51.242 INFO:teuthology.orchestra.run.smithi094.stdout: "write_tier": -1, 2024-09-14T06:29:51.242 INFO:teuthology.orchestra.run.smithi094.stdout: "cache_mode": "none", 2024-09-14T06:29:51.243 INFO:teuthology.orchestra.run.smithi094.stdout: "target_max_bytes": 0, 2024-09-14T06:29:51.243 INFO:teuthology.orchestra.run.smithi094.stdout: "target_max_objects": 0, 2024-09-14T06:29:51.243 INFO:teuthology.orchestra.run.smithi094.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-14T06:29:51.243 INFO:teuthology.orchestra.run.smithi094.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-14T06:29:51.243 INFO:teuthology.orchestra.run.smithi094.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-14T06:29:51.243 INFO:teuthology.orchestra.run.smithi094.stdout: "cache_min_flush_age": 0, 2024-09-14T06:29:51.243 INFO:teuthology.orchestra.run.smithi094.stdout: "cache_min_evict_age": 0, 2024-09-14T06:29:51.243 INFO:teuthology.orchestra.run.smithi094.stdout: "erasure_code_profile": "", 2024-09-14T06:29:51.243 INFO:teuthology.orchestra.run.smithi094.stdout: "hit_set_params": { 2024-09-14T06:29:51.243 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "none" 2024-09-14T06:29:51.243 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:51.243 INFO:teuthology.orchestra.run.smithi094.stdout: "hit_set_period": 0, 2024-09-14T06:29:51.243 INFO:teuthology.orchestra.run.smithi094.stdout: "hit_set_count": 0, 2024-09-14T06:29:51.243 INFO:teuthology.orchestra.run.smithi094.stdout: "use_gmt_hitset": true, 2024-09-14T06:29:51.244 INFO:teuthology.orchestra.run.smithi094.stdout: "min_read_recency_for_promote": 0, 2024-09-14T06:29:51.244 INFO:teuthology.orchestra.run.smithi094.stdout: "min_write_recency_for_promote": 0, 2024-09-14T06:29:51.244 INFO:teuthology.orchestra.run.smithi094.stdout: "hit_set_grade_decay_rate": 0, 2024-09-14T06:29:51.244 INFO:teuthology.orchestra.run.smithi094.stdout: "hit_set_search_last_n": 0, 2024-09-14T06:29:51.244 INFO:teuthology.orchestra.run.smithi094.stdout: "grade_table": [], 2024-09-14T06:29:51.244 INFO:teuthology.orchestra.run.smithi094.stdout: "stripe_width": 0, 2024-09-14T06:29:51.244 INFO:teuthology.orchestra.run.smithi094.stdout: "expected_num_objects": 0, 2024-09-14T06:29:51.244 INFO:teuthology.orchestra.run.smithi094.stdout: "fast_read": false, 2024-09-14T06:29:51.244 INFO:teuthology.orchestra.run.smithi094.stdout: "options": {}, 2024-09-14T06:29:51.244 INFO:teuthology.orchestra.run.smithi094.stdout: "application_metadata": { 2024-09-14T06:29:51.244 INFO:teuthology.orchestra.run.smithi094.stdout: "rbd": {} 2024-09-14T06:29:51.244 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:51.244 INFO:teuthology.orchestra.run.smithi094.stdout: "read_balance": { 2024-09-14T06:29:51.244 INFO:teuthology.orchestra.run.smithi094.stdout: "score_type": "Fair distribution", 2024-09-14T06:29:51.245 INFO:teuthology.orchestra.run.smithi094.stdout: "score_acting": 1.9900000095367432, 2024-09-14T06:29:51.245 INFO:teuthology.orchestra.run.smithi094.stdout: "score_stable": 1.9900000095367432, 2024-09-14T06:29:51.245 INFO:teuthology.orchestra.run.smithi094.stdout: "optimal_score": 0.87999999523162842, 2024-09-14T06:29:51.245 INFO:teuthology.orchestra.run.smithi094.stdout: "raw_score_acting": 1.75, 2024-09-14T06:29:51.245 INFO:teuthology.orchestra.run.smithi094.stdout: "raw_score_stable": 1.75, 2024-09-14T06:29:51.245 INFO:teuthology.orchestra.run.smithi094.stdout: "primary_affinity_weighted": 1, 2024-09-14T06:29:51.245 INFO:teuthology.orchestra.run.smithi094.stdout: "average_primary_affinity": 1, 2024-09-14T06:29:51.245 INFO:teuthology.orchestra.run.smithi094.stdout: "average_primary_affinity_weighted": 1 2024-09-14T06:29:51.245 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:51.245 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:51.246 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:51.246 INFO:teuthology.orchestra.run.smithi094.stdout: "pool": 3, 2024-09-14T06:29:51.246 INFO:teuthology.orchestra.run.smithi094.stdout: "pool_name": "pool_client.0", 2024-09-14T06:29:51.246 INFO:teuthology.orchestra.run.smithi094.stdout: "create_time": "2024-09-14T06:28:59.688535+0000", 2024-09-14T06:29:51.246 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 8193, 2024-09-14T06:29:51.246 INFO:teuthology.orchestra.run.smithi094.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-09-14T06:29:51.246 INFO:teuthology.orchestra.run.smithi094.stdout: "type": 1, 2024-09-14T06:29:51.246 INFO:teuthology.orchestra.run.smithi094.stdout: "size": 2, 2024-09-14T06:29:51.246 INFO:teuthology.orchestra.run.smithi094.stdout: "min_size": 1, 2024-09-14T06:29:51.246 INFO:teuthology.orchestra.run.smithi094.stdout: "crush_rule": 0, 2024-09-14T06:29:51.246 INFO:teuthology.orchestra.run.smithi094.stdout: "peering_crush_bucket_count": 0, 2024-09-14T06:29:51.246 INFO:teuthology.orchestra.run.smithi094.stdout: "peering_crush_bucket_target": 0, 2024-09-14T06:29:51.246 INFO:teuthology.orchestra.run.smithi094.stdout: "peering_crush_bucket_barrier": 0, 2024-09-14T06:29:51.246 INFO:teuthology.orchestra.run.smithi094.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-14T06:29:51.247 INFO:teuthology.orchestra.run.smithi094.stdout: "is_stretch_pool": false, 2024-09-14T06:29:51.247 INFO:teuthology.orchestra.run.smithi094.stdout: "object_hash": 2, 2024-09-14T06:29:51.247 INFO:teuthology.orchestra.run.smithi094.stdout: "pg_autoscale_mode": "off", 2024-09-14T06:29:51.247 INFO:teuthology.orchestra.run.smithi094.stdout: "pg_num": 32, 2024-09-14T06:29:51.247 INFO:teuthology.orchestra.run.smithi094.stdout: "pg_placement_num": 32, 2024-09-14T06:29:51.247 INFO:teuthology.orchestra.run.smithi094.stdout: "pg_placement_num_target": 32, 2024-09-14T06:29:51.247 INFO:teuthology.orchestra.run.smithi094.stdout: "pg_num_target": 32, 2024-09-14T06:29:51.247 INFO:teuthology.orchestra.run.smithi094.stdout: "pg_num_pending": 32, 2024-09-14T06:29:51.247 INFO:teuthology.orchestra.run.smithi094.stdout: "last_pg_merge_meta": { 2024-09-14T06:29:51.247 INFO:teuthology.orchestra.run.smithi094.stdout: "source_pgid": "0.0", 2024-09-14T06:29:51.247 INFO:teuthology.orchestra.run.smithi094.stdout: "ready_epoch": 0, 2024-09-14T06:29:51.247 INFO:teuthology.orchestra.run.smithi094.stdout: "last_epoch_started": 0, 2024-09-14T06:29:51.247 INFO:teuthology.orchestra.run.smithi094.stdout: "last_epoch_clean": 0, 2024-09-14T06:29:51.247 INFO:teuthology.orchestra.run.smithi094.stdout: "source_version": "0'0", 2024-09-14T06:29:51.248 INFO:teuthology.orchestra.run.smithi094.stdout: "target_version": "0'0" 2024-09-14T06:29:51.248 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:51.248 INFO:teuthology.orchestra.run.smithi094.stdout: "last_change": "70", 2024-09-14T06:29:51.248 INFO:teuthology.orchestra.run.smithi094.stdout: "last_force_op_resend": "0", 2024-09-14T06:29:51.248 INFO:teuthology.orchestra.run.smithi094.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-14T06:29:51.248 INFO:teuthology.orchestra.run.smithi094.stdout: "last_force_op_resend_preluminous": "0", 2024-09-14T06:29:51.248 INFO:teuthology.orchestra.run.smithi094.stdout: "auid": 0, 2024-09-14T06:29:51.248 INFO:teuthology.orchestra.run.smithi094.stdout: "snap_mode": "selfmanaged", 2024-09-14T06:29:51.248 INFO:teuthology.orchestra.run.smithi094.stdout: "snap_seq": 28, 2024-09-14T06:29:51.248 INFO:teuthology.orchestra.run.smithi094.stdout: "snap_epoch": 70, 2024-09-14T06:29:51.248 INFO:teuthology.orchestra.run.smithi094.stdout: "pool_snaps": [], 2024-09-14T06:29:51.248 INFO:teuthology.orchestra.run.smithi094.stdout: "removed_snaps": "[]", 2024-09-14T06:29:51.248 INFO:teuthology.orchestra.run.smithi094.stdout: "quota_max_bytes": 0, 2024-09-14T06:29:51.249 INFO:teuthology.orchestra.run.smithi094.stdout: "quota_max_objects": 0, 2024-09-14T06:29:51.249 INFO:teuthology.orchestra.run.smithi094.stdout: "tiers": [], 2024-09-14T06:29:51.249 INFO:teuthology.orchestra.run.smithi094.stdout: "tier_of": -1, 2024-09-14T06:29:51.249 INFO:teuthology.orchestra.run.smithi094.stdout: "read_tier": -1, 2024-09-14T06:29:51.249 INFO:teuthology.orchestra.run.smithi094.stdout: "write_tier": -1, 2024-09-14T06:29:51.249 INFO:teuthology.orchestra.run.smithi094.stdout: "cache_mode": "none", 2024-09-14T06:29:51.249 INFO:teuthology.orchestra.run.smithi094.stdout: "target_max_bytes": 0, 2024-09-14T06:29:51.249 INFO:teuthology.orchestra.run.smithi094.stdout: "target_max_objects": 0, 2024-09-14T06:29:51.249 INFO:teuthology.orchestra.run.smithi094.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-14T06:29:51.249 INFO:teuthology.orchestra.run.smithi094.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-14T06:29:51.249 INFO:teuthology.orchestra.run.smithi094.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-14T06:29:51.249 INFO:teuthology.orchestra.run.smithi094.stdout: "cache_min_flush_age": 0, 2024-09-14T06:29:51.249 INFO:teuthology.orchestra.run.smithi094.stdout: "cache_min_evict_age": 0, 2024-09-14T06:29:51.249 INFO:teuthology.orchestra.run.smithi094.stdout: "erasure_code_profile": "", 2024-09-14T06:29:51.249 INFO:teuthology.orchestra.run.smithi094.stdout: "hit_set_params": { 2024-09-14T06:29:51.250 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "none" 2024-09-14T06:29:51.250 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:51.250 INFO:teuthology.orchestra.run.smithi094.stdout: "hit_set_period": 0, 2024-09-14T06:29:51.250 INFO:teuthology.orchestra.run.smithi094.stdout: "hit_set_count": 0, 2024-09-14T06:29:51.250 INFO:teuthology.orchestra.run.smithi094.stdout: "use_gmt_hitset": true, 2024-09-14T06:29:51.250 INFO:teuthology.orchestra.run.smithi094.stdout: "min_read_recency_for_promote": 0, 2024-09-14T06:29:51.250 INFO:teuthology.orchestra.run.smithi094.stdout: "min_write_recency_for_promote": 0, 2024-09-14T06:29:51.250 INFO:teuthology.orchestra.run.smithi094.stdout: "hit_set_grade_decay_rate": 0, 2024-09-14T06:29:51.250 INFO:teuthology.orchestra.run.smithi094.stdout: "hit_set_search_last_n": 0, 2024-09-14T06:29:51.250 INFO:teuthology.orchestra.run.smithi094.stdout: "grade_table": [], 2024-09-14T06:29:51.250 INFO:teuthology.orchestra.run.smithi094.stdout: "stripe_width": 0, 2024-09-14T06:29:51.250 INFO:teuthology.orchestra.run.smithi094.stdout: "expected_num_objects": 0, 2024-09-14T06:29:51.250 INFO:teuthology.orchestra.run.smithi094.stdout: "fast_read": false, 2024-09-14T06:29:51.250 INFO:teuthology.orchestra.run.smithi094.stdout: "options": {}, 2024-09-14T06:29:51.251 INFO:teuthology.orchestra.run.smithi094.stdout: "application_metadata": { 2024-09-14T06:29:51.251 INFO:teuthology.orchestra.run.smithi094.stdout: "rbd": {} 2024-09-14T06:29:51.251 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:51.251 INFO:teuthology.orchestra.run.smithi094.stdout: "read_balance": { 2024-09-14T06:29:51.251 INFO:teuthology.orchestra.run.smithi094.stdout: "score_type": "Fair distribution", 2024-09-14T06:29:51.251 INFO:teuthology.orchestra.run.smithi094.stdout: "score_acting": 1.5, 2024-09-14T06:29:51.251 INFO:teuthology.orchestra.run.smithi094.stdout: "score_stable": 1.5, 2024-09-14T06:29:51.251 INFO:teuthology.orchestra.run.smithi094.stdout: "optimal_score": 1, 2024-09-14T06:29:51.251 INFO:teuthology.orchestra.run.smithi094.stdout: "raw_score_acting": 1.5, 2024-09-14T06:29:51.251 INFO:teuthology.orchestra.run.smithi094.stdout: "raw_score_stable": 1.5, 2024-09-14T06:29:51.251 INFO:teuthology.orchestra.run.smithi094.stdout: "primary_affinity_weighted": 1, 2024-09-14T06:29:51.251 INFO:teuthology.orchestra.run.smithi094.stdout: "average_primary_affinity": 1, 2024-09-14T06:29:51.251 INFO:teuthology.orchestra.run.smithi094.stdout: "average_primary_affinity_weighted": 1 2024-09-14T06:29:51.251 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:51.252 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:51.252 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-09-14T06:29:51.252 INFO:teuthology.orchestra.run.smithi094.stdout: "osds": [ 2024-09-14T06:29:51.252 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:51.252 INFO:teuthology.orchestra.run.smithi094.stdout: "osd": 0, 2024-09-14T06:29:51.252 INFO:teuthology.orchestra.run.smithi094.stdout: "uuid": "8b22a249-b842-406a-a02c-c0f31f57b635", 2024-09-14T06:29:51.252 INFO:teuthology.orchestra.run.smithi094.stdout: "up": 1, 2024-09-14T06:29:51.252 INFO:teuthology.orchestra.run.smithi094.stdout: "in": 1, 2024-09-14T06:29:51.252 INFO:teuthology.orchestra.run.smithi094.stdout: "weight": 1, 2024-09-14T06:29:51.252 INFO:teuthology.orchestra.run.smithi094.stdout: "primary_affinity": 1, 2024-09-14T06:29:51.252 INFO:teuthology.orchestra.run.smithi094.stdout: "last_clean_begin": 0, 2024-09-14T06:29:51.252 INFO:teuthology.orchestra.run.smithi094.stdout: "last_clean_end": 0, 2024-09-14T06:29:51.252 INFO:teuthology.orchestra.run.smithi094.stdout: "up_from": 14, 2024-09-14T06:29:51.252 INFO:teuthology.orchestra.run.smithi094.stdout: "up_thru": 23, 2024-09-14T06:29:51.253 INFO:teuthology.orchestra.run.smithi094.stdout: "down_at": 0, 2024-09-14T06:29:51.253 INFO:teuthology.orchestra.run.smithi094.stdout: "lost_at": 0, 2024-09-14T06:29:51.253 INFO:teuthology.orchestra.run.smithi094.stdout: "public_addrs": { 2024-09-14T06:29:51.253 INFO:teuthology.orchestra.run.smithi094.stdout: "addrvec": [ 2024-09-14T06:29:51.253 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:51.253 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v2", 2024-09-14T06:29:51.253 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.94:6824", 2024-09-14T06:29:51.253 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 1809564923 2024-09-14T06:29:51.253 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:51.253 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:51.253 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v1", 2024-09-14T06:29:51.253 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.94:6825", 2024-09-14T06:29:51.253 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 1809564923 2024-09-14T06:29:51.253 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:51.254 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:51.254 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:51.254 INFO:teuthology.orchestra.run.smithi094.stdout: "cluster_addrs": { 2024-09-14T06:29:51.254 INFO:teuthology.orchestra.run.smithi094.stdout: "addrvec": [ 2024-09-14T06:29:51.254 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:51.254 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v2", 2024-09-14T06:29:51.254 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.94:6826", 2024-09-14T06:29:51.254 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 1809564923 2024-09-14T06:29:51.254 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:51.254 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:51.254 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v1", 2024-09-14T06:29:51.254 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.94:6827", 2024-09-14T06:29:51.254 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 1809564923 2024-09-14T06:29:51.254 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:51.255 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:51.255 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:51.255 INFO:teuthology.orchestra.run.smithi094.stdout: "heartbeat_back_addrs": { 2024-09-14T06:29:51.255 INFO:teuthology.orchestra.run.smithi094.stdout: "addrvec": [ 2024-09-14T06:29:51.255 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:51.255 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v2", 2024-09-14T06:29:51.255 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.94:6830", 2024-09-14T06:29:51.255 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 1809564923 2024-09-14T06:29:51.255 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:51.255 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:51.255 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v1", 2024-09-14T06:29:51.255 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.94:6831", 2024-09-14T06:29:51.255 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 1809564923 2024-09-14T06:29:51.255 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:51.256 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:51.256 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:51.256 INFO:teuthology.orchestra.run.smithi094.stdout: "heartbeat_front_addrs": { 2024-09-14T06:29:51.256 INFO:teuthology.orchestra.run.smithi094.stdout: "addrvec": [ 2024-09-14T06:29:51.256 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:51.256 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v2", 2024-09-14T06:29:51.256 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.94:6828", 2024-09-14T06:29:51.256 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 1809564923 2024-09-14T06:29:51.256 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:51.256 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:51.256 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v1", 2024-09-14T06:29:51.256 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.94:6829", 2024-09-14T06:29:51.256 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 1809564923 2024-09-14T06:29:51.256 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:51.257 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:51.257 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:51.257 INFO:teuthology.orchestra.run.smithi094.stdout: "public_addr": "172.21.15.94:6825/1809564923", 2024-09-14T06:29:51.257 INFO:teuthology.orchestra.run.smithi094.stdout: "cluster_addr": "172.21.15.94:6827/1809564923", 2024-09-14T06:29:51.257 INFO:teuthology.orchestra.run.smithi094.stdout: "heartbeat_back_addr": "172.21.15.94:6831/1809564923", 2024-09-14T06:29:51.257 INFO:teuthology.orchestra.run.smithi094.stdout: "heartbeat_front_addr": "172.21.15.94:6829/1809564923", 2024-09-14T06:29:51.257 INFO:teuthology.orchestra.run.smithi094.stdout: "state": [ 2024-09-14T06:29:51.257 INFO:teuthology.orchestra.run.smithi094.stdout: "exists", 2024-09-14T06:29:51.257 INFO:teuthology.orchestra.run.smithi094.stdout: "up" 2024-09-14T06:29:51.257 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:51.257 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:51.257 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:51.257 INFO:teuthology.orchestra.run.smithi094.stdout: "osd": 1, 2024-09-14T06:29:51.258 INFO:teuthology.orchestra.run.smithi094.stdout: "uuid": "02251a03-84a0-414d-a980-fac4fd043cb6", 2024-09-14T06:29:51.258 INFO:teuthology.orchestra.run.smithi094.stdout: "up": 1, 2024-09-14T06:29:51.258 INFO:teuthology.orchestra.run.smithi094.stdout: "in": 1, 2024-09-14T06:29:51.258 INFO:teuthology.orchestra.run.smithi094.stdout: "weight": 1, 2024-09-14T06:29:51.258 INFO:teuthology.orchestra.run.smithi094.stdout: "primary_affinity": 1, 2024-09-14T06:29:51.258 INFO:teuthology.orchestra.run.smithi094.stdout: "last_clean_begin": 0, 2024-09-14T06:29:51.258 INFO:teuthology.orchestra.run.smithi094.stdout: "last_clean_end": 0, 2024-09-14T06:29:51.258 INFO:teuthology.orchestra.run.smithi094.stdout: "up_from": 15, 2024-09-14T06:29:51.258 INFO:teuthology.orchestra.run.smithi094.stdout: "up_thru": 23, 2024-09-14T06:29:51.258 INFO:teuthology.orchestra.run.smithi094.stdout: "down_at": 0, 2024-09-14T06:29:51.258 INFO:teuthology.orchestra.run.smithi094.stdout: "lost_at": 0, 2024-09-14T06:29:51.258 INFO:teuthology.orchestra.run.smithi094.stdout: "public_addrs": { 2024-09-14T06:29:51.258 INFO:teuthology.orchestra.run.smithi094.stdout: "addrvec": [ 2024-09-14T06:29:51.258 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:51.259 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v2", 2024-09-14T06:29:51.259 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.94:6816", 2024-09-14T06:29:51.259 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 736402652 2024-09-14T06:29:51.259 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:51.259 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:51.259 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v1", 2024-09-14T06:29:51.259 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.94:6817", 2024-09-14T06:29:51.259 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 736402652 2024-09-14T06:29:51.259 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:51.259 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:51.259 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:51.259 INFO:teuthology.orchestra.run.smithi094.stdout: "cluster_addrs": { 2024-09-14T06:29:51.259 INFO:teuthology.orchestra.run.smithi094.stdout: "addrvec": [ 2024-09-14T06:29:51.259 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:51.260 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v2", 2024-09-14T06:29:51.260 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.94:6818", 2024-09-14T06:29:51.260 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 736402652 2024-09-14T06:29:51.260 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:51.260 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:51.260 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v1", 2024-09-14T06:29:51.260 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.94:6819", 2024-09-14T06:29:51.260 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 736402652 2024-09-14T06:29:51.260 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:51.260 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:51.260 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:51.260 INFO:teuthology.orchestra.run.smithi094.stdout: "heartbeat_back_addrs": { 2024-09-14T06:29:51.260 INFO:teuthology.orchestra.run.smithi094.stdout: "addrvec": [ 2024-09-14T06:29:51.260 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:51.261 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v2", 2024-09-14T06:29:51.261 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.94:6822", 2024-09-14T06:29:51.261 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 736402652 2024-09-14T06:29:51.261 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:51.261 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:51.261 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v1", 2024-09-14T06:29:51.261 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.94:6823", 2024-09-14T06:29:51.261 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 736402652 2024-09-14T06:29:51.261 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:51.261 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:51.261 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:51.261 INFO:teuthology.orchestra.run.smithi094.stdout: "heartbeat_front_addrs": { 2024-09-14T06:29:51.261 INFO:teuthology.orchestra.run.smithi094.stdout: "addrvec": [ 2024-09-14T06:29:51.262 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:51.262 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v2", 2024-09-14T06:29:51.262 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.94:6820", 2024-09-14T06:29:51.262 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 736402652 2024-09-14T06:29:51.262 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:51.262 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:51.262 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v1", 2024-09-14T06:29:51.262 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.94:6821", 2024-09-14T06:29:51.262 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 736402652 2024-09-14T06:29:51.262 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:51.262 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:51.262 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:51.262 INFO:teuthology.orchestra.run.smithi094.stdout: "public_addr": "172.21.15.94:6817/736402652", 2024-09-14T06:29:51.262 INFO:teuthology.orchestra.run.smithi094.stdout: "cluster_addr": "172.21.15.94:6819/736402652", 2024-09-14T06:29:51.263 INFO:teuthology.orchestra.run.smithi094.stdout: "heartbeat_back_addr": "172.21.15.94:6823/736402652", 2024-09-14T06:29:51.263 INFO:teuthology.orchestra.run.smithi094.stdout: "heartbeat_front_addr": "172.21.15.94:6821/736402652", 2024-09-14T06:29:51.263 INFO:teuthology.orchestra.run.smithi094.stdout: "state": [ 2024-09-14T06:29:51.263 INFO:teuthology.orchestra.run.smithi094.stdout: "exists", 2024-09-14T06:29:51.263 INFO:teuthology.orchestra.run.smithi094.stdout: "up" 2024-09-14T06:29:51.263 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:51.263 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:51.263 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:51.263 INFO:teuthology.orchestra.run.smithi094.stdout: "osd": 2, 2024-09-14T06:29:51.263 INFO:teuthology.orchestra.run.smithi094.stdout: "uuid": "d45bf470-bb29-4739-8294-cd122b2b4102", 2024-09-14T06:29:51.263 INFO:teuthology.orchestra.run.smithi094.stdout: "up": 1, 2024-09-14T06:29:51.263 INFO:teuthology.orchestra.run.smithi094.stdout: "in": 1, 2024-09-14T06:29:51.263 INFO:teuthology.orchestra.run.smithi094.stdout: "weight": 1, 2024-09-14T06:29:51.263 INFO:teuthology.orchestra.run.smithi094.stdout: "primary_affinity": 1, 2024-09-14T06:29:51.264 INFO:teuthology.orchestra.run.smithi094.stdout: "last_clean_begin": 0, 2024-09-14T06:29:51.264 INFO:teuthology.orchestra.run.smithi094.stdout: "last_clean_end": 0, 2024-09-14T06:29:51.264 INFO:teuthology.orchestra.run.smithi094.stdout: "up_from": 14, 2024-09-14T06:29:51.264 INFO:teuthology.orchestra.run.smithi094.stdout: "up_thru": 19, 2024-09-14T06:29:51.264 INFO:teuthology.orchestra.run.smithi094.stdout: "down_at": 0, 2024-09-14T06:29:51.264 INFO:teuthology.orchestra.run.smithi094.stdout: "lost_at": 0, 2024-09-14T06:29:51.264 INFO:teuthology.orchestra.run.smithi094.stdout: "public_addrs": { 2024-09-14T06:29:51.264 INFO:teuthology.orchestra.run.smithi094.stdout: "addrvec": [ 2024-09-14T06:29:51.264 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:51.264 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v2", 2024-09-14T06:29:51.264 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.94:6808", 2024-09-14T06:29:51.264 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 1063204782 2024-09-14T06:29:51.264 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:51.264 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:51.264 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v1", 2024-09-14T06:29:51.265 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.94:6809", 2024-09-14T06:29:51.265 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 1063204782 2024-09-14T06:29:51.265 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:51.265 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:51.265 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:51.265 INFO:teuthology.orchestra.run.smithi094.stdout: "cluster_addrs": { 2024-09-14T06:29:51.265 INFO:teuthology.orchestra.run.smithi094.stdout: "addrvec": [ 2024-09-14T06:29:51.265 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:51.265 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v2", 2024-09-14T06:29:51.265 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.94:6810", 2024-09-14T06:29:51.265 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 1063204782 2024-09-14T06:29:51.265 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:51.265 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:51.265 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v1", 2024-09-14T06:29:51.266 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.94:6811", 2024-09-14T06:29:51.266 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 1063204782 2024-09-14T06:29:51.266 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:51.266 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:51.266 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:51.266 INFO:teuthology.orchestra.run.smithi094.stdout: "heartbeat_back_addrs": { 2024-09-14T06:29:51.266 INFO:teuthology.orchestra.run.smithi094.stdout: "addrvec": [ 2024-09-14T06:29:51.266 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:51.266 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v2", 2024-09-14T06:29:51.266 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.94:6814", 2024-09-14T06:29:51.266 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 1063204782 2024-09-14T06:29:51.266 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:51.266 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:51.267 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v1", 2024-09-14T06:29:51.267 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.94:6815", 2024-09-14T06:29:51.267 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 1063204782 2024-09-14T06:29:51.267 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:51.267 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:51.267 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:51.267 INFO:teuthology.orchestra.run.smithi094.stdout: "heartbeat_front_addrs": { 2024-09-14T06:29:51.267 INFO:teuthology.orchestra.run.smithi094.stdout: "addrvec": [ 2024-09-14T06:29:51.267 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:51.267 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v2", 2024-09-14T06:29:51.268 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.94:6812", 2024-09-14T06:29:51.268 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 1063204782 2024-09-14T06:29:51.268 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:51.268 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:51.268 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v1", 2024-09-14T06:29:51.268 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.94:6813", 2024-09-14T06:29:51.268 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 1063204782 2024-09-14T06:29:51.268 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:51.268 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:51.268 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:51.268 INFO:teuthology.orchestra.run.smithi094.stdout: "public_addr": "172.21.15.94:6809/1063204782", 2024-09-14T06:29:51.268 INFO:teuthology.orchestra.run.smithi094.stdout: "cluster_addr": "172.21.15.94:6811/1063204782", 2024-09-14T06:29:51.269 INFO:teuthology.orchestra.run.smithi094.stdout: "heartbeat_back_addr": "172.21.15.94:6815/1063204782", 2024-09-14T06:29:51.269 INFO:teuthology.orchestra.run.smithi094.stdout: "heartbeat_front_addr": "172.21.15.94:6813/1063204782", 2024-09-14T06:29:51.269 INFO:teuthology.orchestra.run.smithi094.stdout: "state": [ 2024-09-14T06:29:51.269 INFO:teuthology.orchestra.run.smithi094.stdout: "exists", 2024-09-14T06:29:51.269 INFO:teuthology.orchestra.run.smithi094.stdout: "up" 2024-09-14T06:29:51.269 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:51.269 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:51.269 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:51.269 INFO:teuthology.orchestra.run.smithi094.stdout: "osd": 3, 2024-09-14T06:29:51.269 INFO:teuthology.orchestra.run.smithi094.stdout: "uuid": "456ec2d9-c438-4243-b931-335704f661eb", 2024-09-14T06:29:51.269 INFO:teuthology.orchestra.run.smithi094.stdout: "up": 1, 2024-09-14T06:29:51.269 INFO:teuthology.orchestra.run.smithi094.stdout: "in": 1, 2024-09-14T06:29:51.269 INFO:teuthology.orchestra.run.smithi094.stdout: "weight": 1, 2024-09-14T06:29:51.269 INFO:teuthology.orchestra.run.smithi094.stdout: "primary_affinity": 1, 2024-09-14T06:29:51.270 INFO:teuthology.orchestra.run.smithi094.stdout: "last_clean_begin": 0, 2024-09-14T06:29:51.270 INFO:teuthology.orchestra.run.smithi094.stdout: "last_clean_end": 0, 2024-09-14T06:29:51.270 INFO:teuthology.orchestra.run.smithi094.stdout: "up_from": 13, 2024-09-14T06:29:51.270 INFO:teuthology.orchestra.run.smithi094.stdout: "up_thru": 23, 2024-09-14T06:29:51.270 INFO:teuthology.orchestra.run.smithi094.stdout: "down_at": 0, 2024-09-14T06:29:51.270 INFO:teuthology.orchestra.run.smithi094.stdout: "lost_at": 0, 2024-09-14T06:29:51.270 INFO:teuthology.orchestra.run.smithi094.stdout: "public_addrs": { 2024-09-14T06:29:51.270 INFO:teuthology.orchestra.run.smithi094.stdout: "addrvec": [ 2024-09-14T06:29:51.270 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:51.270 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v2", 2024-09-14T06:29:51.270 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.94:6800", 2024-09-14T06:29:51.270 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 3382046754 2024-09-14T06:29:51.270 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:51.271 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:51.271 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v1", 2024-09-14T06:29:51.271 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.94:6801", 2024-09-14T06:29:51.271 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 3382046754 2024-09-14T06:29:51.271 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:51.271 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:51.271 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:51.271 INFO:teuthology.orchestra.run.smithi094.stdout: "cluster_addrs": { 2024-09-14T06:29:51.271 INFO:teuthology.orchestra.run.smithi094.stdout: "addrvec": [ 2024-09-14T06:29:51.271 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:51.271 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v2", 2024-09-14T06:29:51.271 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.94:6802", 2024-09-14T06:29:51.271 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 3382046754 2024-09-14T06:29:51.271 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:51.272 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:51.272 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v1", 2024-09-14T06:29:51.272 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.94:6803", 2024-09-14T06:29:51.272 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 3382046754 2024-09-14T06:29:51.272 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:51.272 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:51.272 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:51.272 INFO:teuthology.orchestra.run.smithi094.stdout: "heartbeat_back_addrs": { 2024-09-14T06:29:51.272 INFO:teuthology.orchestra.run.smithi094.stdout: "addrvec": [ 2024-09-14T06:29:51.272 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:51.272 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v2", 2024-09-14T06:29:51.272 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.94:6806", 2024-09-14T06:29:51.272 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 3382046754 2024-09-14T06:29:51.273 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:51.273 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:51.273 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v1", 2024-09-14T06:29:51.273 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.94:6807", 2024-09-14T06:29:51.273 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 3382046754 2024-09-14T06:29:51.273 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:51.273 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:51.273 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:51.273 INFO:teuthology.orchestra.run.smithi094.stdout: "heartbeat_front_addrs": { 2024-09-14T06:29:51.273 INFO:teuthology.orchestra.run.smithi094.stdout: "addrvec": [ 2024-09-14T06:29:51.273 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:51.273 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v2", 2024-09-14T06:29:51.273 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.94:6804", 2024-09-14T06:29:51.274 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 3382046754 2024-09-14T06:29:51.274 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:51.274 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:51.274 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v1", 2024-09-14T06:29:51.274 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.94:6805", 2024-09-14T06:29:51.274 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 3382046754 2024-09-14T06:29:51.274 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:51.274 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:51.274 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:51.274 INFO:teuthology.orchestra.run.smithi094.stdout: "public_addr": "172.21.15.94:6801/3382046754", 2024-09-14T06:29:51.274 INFO:teuthology.orchestra.run.smithi094.stdout: "cluster_addr": "172.21.15.94:6803/3382046754", 2024-09-14T06:29:51.274 INFO:teuthology.orchestra.run.smithi094.stdout: "heartbeat_back_addr": "172.21.15.94:6807/3382046754", 2024-09-14T06:29:51.274 INFO:teuthology.orchestra.run.smithi094.stdout: "heartbeat_front_addr": "172.21.15.94:6805/3382046754", 2024-09-14T06:29:51.274 INFO:teuthology.orchestra.run.smithi094.stdout: "state": [ 2024-09-14T06:29:51.275 INFO:teuthology.orchestra.run.smithi094.stdout: "exists", 2024-09-14T06:29:51.275 INFO:teuthology.orchestra.run.smithi094.stdout: "up" 2024-09-14T06:29:51.275 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:51.275 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:51.275 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:51.275 INFO:teuthology.orchestra.run.smithi094.stdout: "osd": 4, 2024-09-14T06:29:51.275 INFO:teuthology.orchestra.run.smithi094.stdout: "uuid": "2dab948e-c6e4-4437-bf17-017d1ac6821c", 2024-09-14T06:29:51.275 INFO:teuthology.orchestra.run.smithi094.stdout: "up": 1, 2024-09-14T06:29:51.275 INFO:teuthology.orchestra.run.smithi094.stdout: "in": 1, 2024-09-14T06:29:51.275 INFO:teuthology.orchestra.run.smithi094.stdout: "weight": 1, 2024-09-14T06:29:51.275 INFO:teuthology.orchestra.run.smithi094.stdout: "primary_affinity": 1, 2024-09-14T06:29:51.275 INFO:teuthology.orchestra.run.smithi094.stdout: "last_clean_begin": 0, 2024-09-14T06:29:51.275 INFO:teuthology.orchestra.run.smithi094.stdout: "last_clean_end": 0, 2024-09-14T06:29:51.275 INFO:teuthology.orchestra.run.smithi094.stdout: "up_from": 15, 2024-09-14T06:29:51.276 INFO:teuthology.orchestra.run.smithi094.stdout: "up_thru": 23, 2024-09-14T06:29:51.276 INFO:teuthology.orchestra.run.smithi094.stdout: "down_at": 0, 2024-09-14T06:29:51.276 INFO:teuthology.orchestra.run.smithi094.stdout: "lost_at": 0, 2024-09-14T06:29:51.276 INFO:teuthology.orchestra.run.smithi094.stdout: "public_addrs": { 2024-09-14T06:29:51.276 INFO:teuthology.orchestra.run.smithi094.stdout: "addrvec": [ 2024-09-14T06:29:51.276 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:51.276 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v2", 2024-09-14T06:29:51.276 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.138:6810", 2024-09-14T06:29:51.276 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 501726729 2024-09-14T06:29:51.276 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:51.276 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:51.276 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v1", 2024-09-14T06:29:51.276 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.138:6811", 2024-09-14T06:29:51.276 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 501726729 2024-09-14T06:29:51.277 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:51.277 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:51.277 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:51.277 INFO:teuthology.orchestra.run.smithi094.stdout: "cluster_addrs": { 2024-09-14T06:29:51.277 INFO:teuthology.orchestra.run.smithi094.stdout: "addrvec": [ 2024-09-14T06:29:51.277 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:51.277 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v2", 2024-09-14T06:29:51.277 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.138:6812", 2024-09-14T06:29:51.277 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 501726729 2024-09-14T06:29:51.277 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:51.277 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:51.277 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v1", 2024-09-14T06:29:51.277 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.138:6813", 2024-09-14T06:29:51.278 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 501726729 2024-09-14T06:29:51.278 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:51.278 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:51.278 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:51.278 INFO:teuthology.orchestra.run.smithi094.stdout: "heartbeat_back_addrs": { 2024-09-14T06:29:51.278 INFO:teuthology.orchestra.run.smithi094.stdout: "addrvec": [ 2024-09-14T06:29:51.278 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:51.278 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v2", 2024-09-14T06:29:51.278 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.138:6816", 2024-09-14T06:29:51.278 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 501726729 2024-09-14T06:29:51.278 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:51.278 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:51.278 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v1", 2024-09-14T06:29:51.278 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.138:6817", 2024-09-14T06:29:51.279 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 501726729 2024-09-14T06:29:51.279 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:51.279 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:51.279 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:51.279 INFO:teuthology.orchestra.run.smithi094.stdout: "heartbeat_front_addrs": { 2024-09-14T06:29:51.279 INFO:teuthology.orchestra.run.smithi094.stdout: "addrvec": [ 2024-09-14T06:29:51.279 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:51.279 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v2", 2024-09-14T06:29:51.279 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.138:6814", 2024-09-14T06:29:51.279 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 501726729 2024-09-14T06:29:51.279 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:51.279 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:51.279 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v1", 2024-09-14T06:29:51.280 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.138:6815", 2024-09-14T06:29:51.280 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 501726729 2024-09-14T06:29:51.280 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:51.280 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:51.280 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:51.280 INFO:teuthology.orchestra.run.smithi094.stdout: "public_addr": "172.21.15.138:6811/501726729", 2024-09-14T06:29:51.280 INFO:teuthology.orchestra.run.smithi094.stdout: "cluster_addr": "172.21.15.138:6813/501726729", 2024-09-14T06:29:51.280 INFO:teuthology.orchestra.run.smithi094.stdout: "heartbeat_back_addr": "172.21.15.138:6817/501726729", 2024-09-14T06:29:51.280 INFO:teuthology.orchestra.run.smithi094.stdout: "heartbeat_front_addr": "172.21.15.138:6815/501726729", 2024-09-14T06:29:51.280 INFO:teuthology.orchestra.run.smithi094.stdout: "state": [ 2024-09-14T06:29:51.280 INFO:teuthology.orchestra.run.smithi094.stdout: "exists", 2024-09-14T06:29:51.280 INFO:teuthology.orchestra.run.smithi094.stdout: "up" 2024-09-14T06:29:51.280 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:51.281 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:51.281 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:51.281 INFO:teuthology.orchestra.run.smithi094.stdout: "osd": 5, 2024-09-14T06:29:51.281 INFO:teuthology.orchestra.run.smithi094.stdout: "uuid": "8caf3aa3-ad87-4d33-b82e-4c15195176fd", 2024-09-14T06:29:51.281 INFO:teuthology.orchestra.run.smithi094.stdout: "up": 1, 2024-09-14T06:29:51.281 INFO:teuthology.orchestra.run.smithi094.stdout: "in": 1, 2024-09-14T06:29:51.281 INFO:teuthology.orchestra.run.smithi094.stdout: "weight": 1, 2024-09-14T06:29:51.281 INFO:teuthology.orchestra.run.smithi094.stdout: "primary_affinity": 1, 2024-09-14T06:29:51.281 INFO:teuthology.orchestra.run.smithi094.stdout: "last_clean_begin": 0, 2024-09-14T06:29:51.281 INFO:teuthology.orchestra.run.smithi094.stdout: "last_clean_end": 0, 2024-09-14T06:29:51.281 INFO:teuthology.orchestra.run.smithi094.stdout: "up_from": 15, 2024-09-14T06:29:51.281 INFO:teuthology.orchestra.run.smithi094.stdout: "up_thru": 23, 2024-09-14T06:29:51.281 INFO:teuthology.orchestra.run.smithi094.stdout: "down_at": 0, 2024-09-14T06:29:51.281 INFO:teuthology.orchestra.run.smithi094.stdout: "lost_at": 0, 2024-09-14T06:29:51.282 INFO:teuthology.orchestra.run.smithi094.stdout: "public_addrs": { 2024-09-14T06:29:51.282 INFO:teuthology.orchestra.run.smithi094.stdout: "addrvec": [ 2024-09-14T06:29:51.282 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:51.282 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v2", 2024-09-14T06:29:51.282 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.138:6826", 2024-09-14T06:29:51.282 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 1599292735 2024-09-14T06:29:51.282 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:51.282 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:51.282 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v1", 2024-09-14T06:29:51.282 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.138:6827", 2024-09-14T06:29:51.282 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 1599292735 2024-09-14T06:29:51.282 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:51.282 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:51.282 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:51.283 INFO:teuthology.orchestra.run.smithi094.stdout: "cluster_addrs": { 2024-09-14T06:29:51.283 INFO:teuthology.orchestra.run.smithi094.stdout: "addrvec": [ 2024-09-14T06:29:51.283 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:51.283 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v2", 2024-09-14T06:29:51.283 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.138:6828", 2024-09-14T06:29:51.283 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 1599292735 2024-09-14T06:29:51.283 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:51.283 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:51.283 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v1", 2024-09-14T06:29:51.283 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.138:6829", 2024-09-14T06:29:51.283 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 1599292735 2024-09-14T06:29:51.283 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:51.283 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:51.283 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:51.284 INFO:teuthology.orchestra.run.smithi094.stdout: "heartbeat_back_addrs": { 2024-09-14T06:29:51.284 INFO:teuthology.orchestra.run.smithi094.stdout: "addrvec": [ 2024-09-14T06:29:51.284 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:51.284 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v2", 2024-09-14T06:29:51.284 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.138:6832", 2024-09-14T06:29:51.284 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 1599292735 2024-09-14T06:29:51.284 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:51.284 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:51.284 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v1", 2024-09-14T06:29:51.284 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.138:6833", 2024-09-14T06:29:51.284 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 1599292735 2024-09-14T06:29:51.284 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:51.284 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:51.285 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:51.285 INFO:teuthology.orchestra.run.smithi094.stdout: "heartbeat_front_addrs": { 2024-09-14T06:29:51.285 INFO:teuthology.orchestra.run.smithi094.stdout: "addrvec": [ 2024-09-14T06:29:51.285 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:51.285 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v2", 2024-09-14T06:29:51.285 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.138:6830", 2024-09-14T06:29:51.285 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 1599292735 2024-09-14T06:29:51.285 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:51.285 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:51.285 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v1", 2024-09-14T06:29:51.285 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.138:6831", 2024-09-14T06:29:51.285 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 1599292735 2024-09-14T06:29:51.285 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:51.285 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:51.286 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:51.286 INFO:teuthology.orchestra.run.smithi094.stdout: "public_addr": "172.21.15.138:6827/1599292735", 2024-09-14T06:29:51.286 INFO:teuthology.orchestra.run.smithi094.stdout: "cluster_addr": "172.21.15.138:6829/1599292735", 2024-09-14T06:29:51.286 INFO:teuthology.orchestra.run.smithi094.stdout: "heartbeat_back_addr": "172.21.15.138:6833/1599292735", 2024-09-14T06:29:51.286 INFO:teuthology.orchestra.run.smithi094.stdout: "heartbeat_front_addr": "172.21.15.138:6831/1599292735", 2024-09-14T06:29:51.286 INFO:teuthology.orchestra.run.smithi094.stdout: "state": [ 2024-09-14T06:29:51.286 INFO:teuthology.orchestra.run.smithi094.stdout: "exists", 2024-09-14T06:29:51.286 INFO:teuthology.orchestra.run.smithi094.stdout: "up" 2024-09-14T06:29:51.286 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:51.286 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:51.286 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:51.286 INFO:teuthology.orchestra.run.smithi094.stdout: "osd": 6, 2024-09-14T06:29:51.286 INFO:teuthology.orchestra.run.smithi094.stdout: "uuid": "e55d11fd-433d-4462-93a7-d5e84560221b", 2024-09-14T06:29:51.286 INFO:teuthology.orchestra.run.smithi094.stdout: "up": 1, 2024-09-14T06:29:51.287 INFO:teuthology.orchestra.run.smithi094.stdout: "in": 1, 2024-09-14T06:29:51.287 INFO:teuthology.orchestra.run.smithi094.stdout: "weight": 1, 2024-09-14T06:29:51.287 INFO:teuthology.orchestra.run.smithi094.stdout: "primary_affinity": 1, 2024-09-14T06:29:51.287 INFO:teuthology.orchestra.run.smithi094.stdout: "last_clean_begin": 0, 2024-09-14T06:29:51.287 INFO:teuthology.orchestra.run.smithi094.stdout: "last_clean_end": 0, 2024-09-14T06:29:51.287 INFO:teuthology.orchestra.run.smithi094.stdout: "up_from": 15, 2024-09-14T06:29:51.287 INFO:teuthology.orchestra.run.smithi094.stdout: "up_thru": 23, 2024-09-14T06:29:51.287 INFO:teuthology.orchestra.run.smithi094.stdout: "down_at": 0, 2024-09-14T06:29:51.287 INFO:teuthology.orchestra.run.smithi094.stdout: "lost_at": 0, 2024-09-14T06:29:51.287 INFO:teuthology.orchestra.run.smithi094.stdout: "public_addrs": { 2024-09-14T06:29:51.287 INFO:teuthology.orchestra.run.smithi094.stdout: "addrvec": [ 2024-09-14T06:29:51.287 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:51.287 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v2", 2024-09-14T06:29:51.287 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.138:6818", 2024-09-14T06:29:51.288 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 163531479 2024-09-14T06:29:51.288 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:51.288 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:51.288 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v1", 2024-09-14T06:29:51.288 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.138:6819", 2024-09-14T06:29:51.288 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 163531479 2024-09-14T06:29:51.288 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:51.288 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:51.288 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:51.288 INFO:teuthology.orchestra.run.smithi094.stdout: "cluster_addrs": { 2024-09-14T06:29:51.288 INFO:teuthology.orchestra.run.smithi094.stdout: "addrvec": [ 2024-09-14T06:29:51.288 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:51.288 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v2", 2024-09-14T06:29:51.288 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.138:6820", 2024-09-14T06:29:51.289 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 163531479 2024-09-14T06:29:51.289 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:51.289 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:51.289 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v1", 2024-09-14T06:29:51.289 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.138:6821", 2024-09-14T06:29:51.289 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 163531479 2024-09-14T06:29:51.289 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:51.289 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:51.289 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:51.289 INFO:teuthology.orchestra.run.smithi094.stdout: "heartbeat_back_addrs": { 2024-09-14T06:29:51.289 INFO:teuthology.orchestra.run.smithi094.stdout: "addrvec": [ 2024-09-14T06:29:51.289 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:51.289 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v2", 2024-09-14T06:29:51.289 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.138:6824", 2024-09-14T06:29:51.290 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 163531479 2024-09-14T06:29:51.290 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:51.290 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:51.290 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v1", 2024-09-14T06:29:51.290 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.138:6825", 2024-09-14T06:29:51.290 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 163531479 2024-09-14T06:29:51.290 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:51.290 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:51.290 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:51.290 INFO:teuthology.orchestra.run.smithi094.stdout: "heartbeat_front_addrs": { 2024-09-14T06:29:51.290 INFO:teuthology.orchestra.run.smithi094.stdout: "addrvec": [ 2024-09-14T06:29:51.290 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:51.290 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v2", 2024-09-14T06:29:51.290 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.138:6822", 2024-09-14T06:29:51.291 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 163531479 2024-09-14T06:29:51.291 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:51.291 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:51.291 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v1", 2024-09-14T06:29:51.291 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.138:6823", 2024-09-14T06:29:51.291 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 163531479 2024-09-14T06:29:51.291 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:51.291 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:51.291 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:51.291 INFO:teuthology.orchestra.run.smithi094.stdout: "public_addr": "172.21.15.138:6819/163531479", 2024-09-14T06:29:51.291 INFO:teuthology.orchestra.run.smithi094.stdout: "cluster_addr": "172.21.15.138:6821/163531479", 2024-09-14T06:29:51.291 INFO:teuthology.orchestra.run.smithi094.stdout: "heartbeat_back_addr": "172.21.15.138:6825/163531479", 2024-09-14T06:29:51.291 INFO:teuthology.orchestra.run.smithi094.stdout: "heartbeat_front_addr": "172.21.15.138:6823/163531479", 2024-09-14T06:29:51.292 INFO:teuthology.orchestra.run.smithi094.stdout: "state": [ 2024-09-14T06:29:51.292 INFO:teuthology.orchestra.run.smithi094.stdout: "exists", 2024-09-14T06:29:51.292 INFO:teuthology.orchestra.run.smithi094.stdout: "up" 2024-09-14T06:29:51.292 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:51.292 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:51.292 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:51.292 INFO:teuthology.orchestra.run.smithi094.stdout: "osd": 7, 2024-09-14T06:29:51.292 INFO:teuthology.orchestra.run.smithi094.stdout: "uuid": "ba14ba69-2c6c-4d09-81e2-2593dc3a7ea9", 2024-09-14T06:29:51.292 INFO:teuthology.orchestra.run.smithi094.stdout: "up": 1, 2024-09-14T06:29:51.292 INFO:teuthology.orchestra.run.smithi094.stdout: "in": 1, 2024-09-14T06:29:51.292 INFO:teuthology.orchestra.run.smithi094.stdout: "weight": 1, 2024-09-14T06:29:51.292 INFO:teuthology.orchestra.run.smithi094.stdout: "primary_affinity": 1, 2024-09-14T06:29:51.292 INFO:teuthology.orchestra.run.smithi094.stdout: "last_clean_begin": 0, 2024-09-14T06:29:51.292 INFO:teuthology.orchestra.run.smithi094.stdout: "last_clean_end": 0, 2024-09-14T06:29:51.293 INFO:teuthology.orchestra.run.smithi094.stdout: "up_from": 15, 2024-09-14T06:29:51.293 INFO:teuthology.orchestra.run.smithi094.stdout: "up_thru": 23, 2024-09-14T06:29:51.293 INFO:teuthology.orchestra.run.smithi094.stdout: "down_at": 0, 2024-09-14T06:29:51.293 INFO:teuthology.orchestra.run.smithi094.stdout: "lost_at": 0, 2024-09-14T06:29:51.293 INFO:teuthology.orchestra.run.smithi094.stdout: "public_addrs": { 2024-09-14T06:29:51.293 INFO:teuthology.orchestra.run.smithi094.stdout: "addrvec": [ 2024-09-14T06:29:51.293 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:51.293 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v2", 2024-09-14T06:29:51.293 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.138:6800", 2024-09-14T06:29:51.293 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 218507235 2024-09-14T06:29:51.293 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:51.293 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:51.293 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v1", 2024-09-14T06:29:51.293 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.138:6801", 2024-09-14T06:29:51.294 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 218507235 2024-09-14T06:29:51.294 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:51.294 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:51.294 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:51.294 INFO:teuthology.orchestra.run.smithi094.stdout: "cluster_addrs": { 2024-09-14T06:29:51.294 INFO:teuthology.orchestra.run.smithi094.stdout: "addrvec": [ 2024-09-14T06:29:51.294 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:51.294 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v2", 2024-09-14T06:29:51.294 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.138:6802", 2024-09-14T06:29:51.294 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 218507235 2024-09-14T06:29:51.294 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:51.294 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:51.294 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v1", 2024-09-14T06:29:51.294 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.138:6803", 2024-09-14T06:29:51.295 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 218507235 2024-09-14T06:29:51.295 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:51.295 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:51.295 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:51.295 INFO:teuthology.orchestra.run.smithi094.stdout: "heartbeat_back_addrs": { 2024-09-14T06:29:51.295 INFO:teuthology.orchestra.run.smithi094.stdout: "addrvec": [ 2024-09-14T06:29:51.295 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:51.295 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v2", 2024-09-14T06:29:51.295 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.138:6806", 2024-09-14T06:29:51.295 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 218507235 2024-09-14T06:29:51.295 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:51.295 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:51.295 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v1", 2024-09-14T06:29:51.295 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.138:6807", 2024-09-14T06:29:51.296 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 218507235 2024-09-14T06:29:51.296 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:51.296 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:51.296 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:51.296 INFO:teuthology.orchestra.run.smithi094.stdout: "heartbeat_front_addrs": { 2024-09-14T06:29:51.296 INFO:teuthology.orchestra.run.smithi094.stdout: "addrvec": [ 2024-09-14T06:29:51.296 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:51.296 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v2", 2024-09-14T06:29:51.296 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.138:6804", 2024-09-14T06:29:51.296 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 218507235 2024-09-14T06:29:51.296 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:51.296 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:51.296 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v1", 2024-09-14T06:29:51.296 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.138:6805", 2024-09-14T06:29:51.297 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 218507235 2024-09-14T06:29:51.297 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:51.297 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:51.297 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:51.297 INFO:teuthology.orchestra.run.smithi094.stdout: "public_addr": "172.21.15.138:6801/218507235", 2024-09-14T06:29:51.297 INFO:teuthology.orchestra.run.smithi094.stdout: "cluster_addr": "172.21.15.138:6803/218507235", 2024-09-14T06:29:51.297 INFO:teuthology.orchestra.run.smithi094.stdout: "heartbeat_back_addr": "172.21.15.138:6807/218507235", 2024-09-14T06:29:51.297 INFO:teuthology.orchestra.run.smithi094.stdout: "heartbeat_front_addr": "172.21.15.138:6805/218507235", 2024-09-14T06:29:51.297 INFO:teuthology.orchestra.run.smithi094.stdout: "state": [ 2024-09-14T06:29:51.297 INFO:teuthology.orchestra.run.smithi094.stdout: "exists", 2024-09-14T06:29:51.297 INFO:teuthology.orchestra.run.smithi094.stdout: "up" 2024-09-14T06:29:51.297 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:51.297 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:51.297 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-09-14T06:29:51.298 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_xinfo": [ 2024-09-14T06:29:51.298 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:51.298 INFO:teuthology.orchestra.run.smithi094.stdout: "osd": 0, 2024-09-14T06:29:51.298 INFO:teuthology.orchestra.run.smithi094.stdout: "down_stamp": "0.000000", 2024-09-14T06:29:51.298 INFO:teuthology.orchestra.run.smithi094.stdout: "laggy_probability": 0, 2024-09-14T06:29:51.298 INFO:teuthology.orchestra.run.smithi094.stdout: "laggy_interval": 0, 2024-09-14T06:29:51.298 INFO:teuthology.orchestra.run.smithi094.stdout: "features": 4540719139924082687, 2024-09-14T06:29:51.298 INFO:teuthology.orchestra.run.smithi094.stdout: "old_weight": 0, 2024-09-14T06:29:51.298 INFO:teuthology.orchestra.run.smithi094.stdout: "last_purged_snaps_scrub": "2024-09-14T06:28:39.906639+0000", 2024-09-14T06:29:51.298 INFO:teuthology.orchestra.run.smithi094.stdout: "dead_epoch": 0 2024-09-14T06:29:51.298 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:51.298 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:51.298 INFO:teuthology.orchestra.run.smithi094.stdout: "osd": 1, 2024-09-14T06:29:51.298 INFO:teuthology.orchestra.run.smithi094.stdout: "down_stamp": "0.000000", 2024-09-14T06:29:51.299 INFO:teuthology.orchestra.run.smithi094.stdout: "laggy_probability": 0, 2024-09-14T06:29:51.299 INFO:teuthology.orchestra.run.smithi094.stdout: "laggy_interval": 0, 2024-09-14T06:29:51.299 INFO:teuthology.orchestra.run.smithi094.stdout: "features": 4540719139924082687, 2024-09-14T06:29:51.299 INFO:teuthology.orchestra.run.smithi094.stdout: "old_weight": 0, 2024-09-14T06:29:51.299 INFO:teuthology.orchestra.run.smithi094.stdout: "last_purged_snaps_scrub": "2024-09-14T06:28:40.541279+0000", 2024-09-14T06:29:51.299 INFO:teuthology.orchestra.run.smithi094.stdout: "dead_epoch": 0 2024-09-14T06:29:51.299 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:51.299 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:51.299 INFO:teuthology.orchestra.run.smithi094.stdout: "osd": 2, 2024-09-14T06:29:51.299 INFO:teuthology.orchestra.run.smithi094.stdout: "down_stamp": "0.000000", 2024-09-14T06:29:51.299 INFO:teuthology.orchestra.run.smithi094.stdout: "laggy_probability": 0, 2024-09-14T06:29:51.299 INFO:teuthology.orchestra.run.smithi094.stdout: "laggy_interval": 0, 2024-09-14T06:29:51.299 INFO:teuthology.orchestra.run.smithi094.stdout: "features": 4540719139924082687, 2024-09-14T06:29:51.299 INFO:teuthology.orchestra.run.smithi094.stdout: "old_weight": 0, 2024-09-14T06:29:51.300 INFO:teuthology.orchestra.run.smithi094.stdout: "last_purged_snaps_scrub": "2024-09-14T06:28:40.025350+0000", 2024-09-14T06:29:51.300 INFO:teuthology.orchestra.run.smithi094.stdout: "dead_epoch": 0 2024-09-14T06:29:51.300 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:51.300 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:51.300 INFO:teuthology.orchestra.run.smithi094.stdout: "osd": 3, 2024-09-14T06:29:51.300 INFO:teuthology.orchestra.run.smithi094.stdout: "down_stamp": "0.000000", 2024-09-14T06:29:51.300 INFO:teuthology.orchestra.run.smithi094.stdout: "laggy_probability": 0, 2024-09-14T06:29:51.300 INFO:teuthology.orchestra.run.smithi094.stdout: "laggy_interval": 0, 2024-09-14T06:29:51.300 INFO:teuthology.orchestra.run.smithi094.stdout: "features": 4540719139924082687, 2024-09-14T06:29:51.300 INFO:teuthology.orchestra.run.smithi094.stdout: "old_weight": 0, 2024-09-14T06:29:51.300 INFO:teuthology.orchestra.run.smithi094.stdout: "last_purged_snaps_scrub": "2024-09-14T06:28:39.217944+0000", 2024-09-14T06:29:51.300 INFO:teuthology.orchestra.run.smithi094.stdout: "dead_epoch": 0 2024-09-14T06:29:51.300 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:51.300 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:51.301 INFO:teuthology.orchestra.run.smithi094.stdout: "osd": 4, 2024-09-14T06:29:51.301 INFO:teuthology.orchestra.run.smithi094.stdout: "down_stamp": "0.000000", 2024-09-14T06:29:51.301 INFO:teuthology.orchestra.run.smithi094.stdout: "laggy_probability": 0, 2024-09-14T06:29:51.301 INFO:teuthology.orchestra.run.smithi094.stdout: "laggy_interval": 0, 2024-09-14T06:29:51.301 INFO:teuthology.orchestra.run.smithi094.stdout: "features": 4540719139924082687, 2024-09-14T06:29:51.301 INFO:teuthology.orchestra.run.smithi094.stdout: "old_weight": 0, 2024-09-14T06:29:51.301 INFO:teuthology.orchestra.run.smithi094.stdout: "last_purged_snaps_scrub": "2024-09-14T06:28:40.185898+0000", 2024-09-14T06:29:51.301 INFO:teuthology.orchestra.run.smithi094.stdout: "dead_epoch": 0 2024-09-14T06:29:51.301 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:51.301 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:51.301 INFO:teuthology.orchestra.run.smithi094.stdout: "osd": 5, 2024-09-14T06:29:51.301 INFO:teuthology.orchestra.run.smithi094.stdout: "down_stamp": "0.000000", 2024-09-14T06:29:51.301 INFO:teuthology.orchestra.run.smithi094.stdout: "laggy_probability": 0, 2024-09-14T06:29:51.301 INFO:teuthology.orchestra.run.smithi094.stdout: "laggy_interval": 0, 2024-09-14T06:29:51.302 INFO:teuthology.orchestra.run.smithi094.stdout: "features": 4540719139924082687, 2024-09-14T06:29:51.302 INFO:teuthology.orchestra.run.smithi094.stdout: "old_weight": 0, 2024-09-14T06:29:51.302 INFO:teuthology.orchestra.run.smithi094.stdout: "last_purged_snaps_scrub": "2024-09-14T06:28:40.783504+0000", 2024-09-14T06:29:51.302 INFO:teuthology.orchestra.run.smithi094.stdout: "dead_epoch": 0 2024-09-14T06:29:51.302 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:51.302 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:51.302 INFO:teuthology.orchestra.run.smithi094.stdout: "osd": 6, 2024-09-14T06:29:51.302 INFO:teuthology.orchestra.run.smithi094.stdout: "down_stamp": "0.000000", 2024-09-14T06:29:51.302 INFO:teuthology.orchestra.run.smithi094.stdout: "laggy_probability": 0, 2024-09-14T06:29:51.302 INFO:teuthology.orchestra.run.smithi094.stdout: "laggy_interval": 0, 2024-09-14T06:29:51.302 INFO:teuthology.orchestra.run.smithi094.stdout: "features": 4540719139924082687, 2024-09-14T06:29:51.302 INFO:teuthology.orchestra.run.smithi094.stdout: "old_weight": 0, 2024-09-14T06:29:51.302 INFO:teuthology.orchestra.run.smithi094.stdout: "last_purged_snaps_scrub": "2024-09-14T06:28:40.527305+0000", 2024-09-14T06:29:51.302 INFO:teuthology.orchestra.run.smithi094.stdout: "dead_epoch": 0 2024-09-14T06:29:51.303 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:51.303 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:51.303 INFO:teuthology.orchestra.run.smithi094.stdout: "osd": 7, 2024-09-14T06:29:51.303 INFO:teuthology.orchestra.run.smithi094.stdout: "down_stamp": "0.000000", 2024-09-14T06:29:51.303 INFO:teuthology.orchestra.run.smithi094.stdout: "laggy_probability": 0, 2024-09-14T06:29:51.303 INFO:teuthology.orchestra.run.smithi094.stdout: "laggy_interval": 0, 2024-09-14T06:29:51.303 INFO:teuthology.orchestra.run.smithi094.stdout: "features": 4540719139924082687, 2024-09-14T06:29:51.303 INFO:teuthology.orchestra.run.smithi094.stdout: "old_weight": 0, 2024-09-14T06:29:51.303 INFO:teuthology.orchestra.run.smithi094.stdout: "last_purged_snaps_scrub": "2024-09-14T06:28:40.349676+0000", 2024-09-14T06:29:51.303 INFO:teuthology.orchestra.run.smithi094.stdout: "dead_epoch": 0 2024-09-14T06:29:51.303 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:51.303 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-09-14T06:29:51.303 INFO:teuthology.orchestra.run.smithi094.stdout: "pg_upmap": [], 2024-09-14T06:29:51.304 INFO:teuthology.orchestra.run.smithi094.stdout: "pg_upmap_items": [], 2024-09-14T06:29:51.304 INFO:teuthology.orchestra.run.smithi094.stdout: "pg_upmap_primaries": [], 2024-09-14T06:29:51.304 INFO:teuthology.orchestra.run.smithi094.stdout: "pg_temp": [], 2024-09-14T06:29:51.304 INFO:teuthology.orchestra.run.smithi094.stdout: "primary_temp": [], 2024-09-14T06:29:51.304 INFO:teuthology.orchestra.run.smithi094.stdout: "blocklist": {}, 2024-09-14T06:29:51.304 INFO:teuthology.orchestra.run.smithi094.stdout: "range_blocklist": {}, 2024-09-14T06:29:51.304 INFO:teuthology.orchestra.run.smithi094.stdout: "erasure_code_profiles": { 2024-09-14T06:29:51.304 INFO:teuthology.orchestra.run.smithi094.stdout: "default": { 2024-09-14T06:29:51.304 INFO:teuthology.orchestra.run.smithi094.stdout: "crush-failure-domain": "osd", 2024-09-14T06:29:51.304 INFO:teuthology.orchestra.run.smithi094.stdout: "k": "2", 2024-09-14T06:29:51.304 INFO:teuthology.orchestra.run.smithi094.stdout: "m": "1", 2024-09-14T06:29:51.304 INFO:teuthology.orchestra.run.smithi094.stdout: "plugin": "jerasure", 2024-09-14T06:29:51.304 INFO:teuthology.orchestra.run.smithi094.stdout: "technique": "reed_sol_van" 2024-09-14T06:29:51.304 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:51.305 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:51.305 INFO:teuthology.orchestra.run.smithi094.stdout: "removed_snaps_queue": [ 2024-09-14T06:29:51.305 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:51.305 INFO:teuthology.orchestra.run.smithi094.stdout: "pool": 3, 2024-09-14T06:29:51.305 INFO:teuthology.orchestra.run.smithi094.stdout: "snaps": [ 2024-09-14T06:29:51.305 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:51.305 INFO:teuthology.orchestra.run.smithi094.stdout: "begin": 24, 2024-09-14T06:29:51.305 INFO:teuthology.orchestra.run.smithi094.stdout: "length": 1 2024-09-14T06:29:51.305 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:51.305 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:51.305 INFO:teuthology.orchestra.run.smithi094.stdout: "begin": 26, 2024-09-14T06:29:51.305 INFO:teuthology.orchestra.run.smithi094.stdout: "length": 1 2024-09-14T06:29:51.305 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:51.306 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:51.306 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:51.306 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-09-14T06:29:51.306 INFO:teuthology.orchestra.run.smithi094.stdout: "new_removed_snaps": [], 2024-09-14T06:29:51.306 INFO:teuthology.orchestra.run.smithi094.stdout: "new_purged_snaps": [ 2024-09-14T06:29:51.306 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:51.306 INFO:teuthology.orchestra.run.smithi094.stdout: "pool": 3, 2024-09-14T06:29:51.306 INFO:teuthology.orchestra.run.smithi094.stdout: "snaps": [ 2024-09-14T06:29:51.306 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:51.306 INFO:teuthology.orchestra.run.smithi094.stdout: "begin": 22, 2024-09-14T06:29:51.306 INFO:teuthology.orchestra.run.smithi094.stdout: "length": 1 2024-09-14T06:29:51.306 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:51.306 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:51.306 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:51.307 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-09-14T06:29:51.307 INFO:teuthology.orchestra.run.smithi094.stdout: "crush_node_flags": {}, 2024-09-14T06:29:51.307 INFO:teuthology.orchestra.run.smithi094.stdout: "device_class_flags": {}, 2024-09-14T06:29:51.307 INFO:teuthology.orchestra.run.smithi094.stdout: "stretch_mode": { 2024-09-14T06:29:51.307 INFO:teuthology.orchestra.run.smithi094.stdout: "stretch_mode_enabled": false, 2024-09-14T06:29:51.307 INFO:teuthology.orchestra.run.smithi094.stdout: "stretch_bucket_count": 0, 2024-09-14T06:29:51.307 INFO:teuthology.orchestra.run.smithi094.stdout: "degraded_stretch_mode": 0, 2024-09-14T06:29:51.307 INFO:teuthology.orchestra.run.smithi094.stdout: "recovering_stretch_mode": 0, 2024-09-14T06:29:51.307 INFO:teuthology.orchestra.run.smithi094.stdout: "stretch_mode_bucket": 0 2024-09-14T06:29:51.307 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:51.307 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-14T06:29:51.308 INFO:teuthology.orchestra.run.smithi094.stdout:ERROR: (22) Invalid argument 2024-09-14T06:29:51.308 INFO:teuthology.orchestra.run.smithi094.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:29:51.310 INFO:tasks.thrashosds.thrasher:j is {'epoch': 70, 'fsid': '200ad3da-bd63-4e04-af74-efb5ca0af75e', 'created': '2024-09-14T06:28:30.782046+0000', 'modified': '2024-09-14T06:29:50.575617+0000', 'last_up_change': '2024-09-14T06:28:42.827239+0000', 'last_in_change': '2024-09-14T06:28:33.724176+0000', 'flags': 'noscrub,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit', 'flags_num': 5801984, 'flags_set': ['noscrub', 'pglog_hardlimit', 'purged_snapdirs', 'recovery_deletes', 'sortbitwise'], 'crush_version': 5, 'full_ratio': 0.949999988079071, 'backfillfull_ratio': 0.8999999761581421, 'nearfull_ratio': 0.8500000238418579, 'cluster_snapshot': '', 'pool_max': 3, 'max_osd': 8, 'require_min_compat_client': 'luminous', 'min_compat_client': 'jewel', 'require_osd_release': 'squid', 'allow_crimson': False, 'pools': [{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-14T06:28:43.295990+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2024-09-14T06:28:49.451714+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 2, 'snap_epoch': 22, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.9900000095367432, 'score_stable': 1.9900000095367432, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'pool_client.0', 'create_time': '2024-09-14T06:28:59.688535+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '70', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 28, 'snap_epoch': 70, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}], 'osds': [{'osd': 0, 'uuid': '8b22a249-b842-406a-a02c-c0f31f57b635', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 23, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.94:6824', 'nonce': 1809564923}, {'type': 'v1', 'addr': '172.21.15.94:6825', 'nonce': 1809564923}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.94:6826', 'nonce': 1809564923}, {'type': 'v1', 'addr': '172.21.15.94:6827', 'nonce': 1809564923}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.94:6830', 'nonce': 1809564923}, {'type': 'v1', 'addr': '172.21.15.94:6831', 'nonce': 1809564923}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.94:6828', 'nonce': 1809564923}, {'type': 'v1', 'addr': '172.21.15.94:6829', 'nonce': 1809564923}]}, 'public_addr': '172.21.15.94:6825/1809564923', 'cluster_addr': '172.21.15.94:6827/1809564923', 'heartbeat_back_addr': '172.21.15.94:6831/1809564923', 'heartbeat_front_addr': '172.21.15.94:6829/1809564923', 'state': ['exists', 'up']}, {'osd': 1, 'uuid': '02251a03-84a0-414d-a980-fac4fd043cb6', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 23, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.94:6816', 'nonce': 736402652}, {'type': 'v1', 'addr': '172.21.15.94:6817', 'nonce': 736402652}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.94:6818', 'nonce': 736402652}, {'type': 'v1', 'addr': '172.21.15.94:6819', 'nonce': 736402652}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.94:6822', 'nonce': 736402652}, {'type': 'v1', 'addr': '172.21.15.94:6823', 'nonce': 736402652}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.94:6820', 'nonce': 736402652}, {'type': 'v1', 'addr': '172.21.15.94:6821', 'nonce': 736402652}]}, 'public_addr': '172.21.15.94:6817/736402652', 'cluster_addr': '172.21.15.94:6819/736402652', 'heartbeat_back_addr': '172.21.15.94:6823/736402652', 'heartbeat_front_addr': '172.21.15.94:6821/736402652', 'state': ['exists', 'up']}, {'osd': 2, 'uuid': 'd45bf470-bb29-4739-8294-cd122b2b4102', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 19, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.94:6808', 'nonce': 1063204782}, {'type': 'v1', 'addr': '172.21.15.94:6809', 'nonce': 1063204782}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.94:6810', 'nonce': 1063204782}, {'type': 'v1', 'addr': '172.21.15.94:6811', 'nonce': 1063204782}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.94:6814', 'nonce': 1063204782}, {'type': 'v1', 'addr': '172.21.15.94:6815', 'nonce': 1063204782}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.94:6812', 'nonce': 1063204782}, {'type': 'v1', 'addr': '172.21.15.94:6813', 'nonce': 1063204782}]}, 'public_addr': '172.21.15.94:6809/1063204782', 'cluster_addr': '172.21.15.94:6811/1063204782', 'heartbeat_back_addr': '172.21.15.94:6815/1063204782', 'heartbeat_front_addr': '172.21.15.94:6813/1063204782', 'state': ['exists', 'up']}, {'osd': 3, 'uuid': '456ec2d9-c438-4243-b931-335704f661eb', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 23, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.94:6800', 'nonce': 3382046754}, {'type': 'v1', 'addr': '172.21.15.94:6801', 'nonce': 3382046754}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.94:6802', 'nonce': 3382046754}, {'type': 'v1', 'addr': '172.21.15.94:6803', 'nonce': 3382046754}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.94:6806', 'nonce': 3382046754}, {'type': 'v1', 'addr': '172.21.15.94:6807', 'nonce': 3382046754}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.94:6804', 'nonce': 3382046754}, {'type': 'v1', 'addr': '172.21.15.94:6805', 'nonce': 3382046754}]}, 'public_addr': '172.21.15.94:6801/3382046754', 'cluster_addr': '172.21.15.94:6803/3382046754', 'heartbeat_back_addr': '172.21.15.94:6807/3382046754', 'heartbeat_front_addr': '172.21.15.94:6805/3382046754', 'state': ['exists', 'up']}, {'osd': 4, 'uuid': '2dab948e-c6e4-4437-bf17-017d1ac6821c', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 23, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.138:6810', 'nonce': 501726729}, {'type': 'v1', 'addr': '172.21.15.138:6811', 'nonce': 501726729}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.138:6812', 'nonce': 501726729}, {'type': 'v1', 'addr': '172.21.15.138:6813', 'nonce': 501726729}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.138:6816', 'nonce': 501726729}, {'type': 'v1', 'addr': '172.21.15.138:6817', 'nonce': 501726729}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.138:6814', 'nonce': 501726729}, {'type': 'v1', 'addr': '172.21.15.138:6815', 'nonce': 501726729}]}, 'public_addr': '172.21.15.138:6811/501726729', 'cluster_addr': '172.21.15.138:6813/501726729', 'heartbeat_back_addr': '172.21.15.138:6817/501726729', 'heartbeat_front_addr': '172.21.15.138:6815/501726729', 'state': ['exists', 'up']}, {'osd': 5, 'uuid': '8caf3aa3-ad87-4d33-b82e-4c15195176fd', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 23, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.138:6826', 'nonce': 1599292735}, {'type': 'v1', 'addr': '172.21.15.138:6827', 'nonce': 1599292735}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.138:6828', 'nonce': 1599292735}, {'type': 'v1', 'addr': '172.21.15.138:6829', 'nonce': 1599292735}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.138:6832', 'nonce': 1599292735}, {'type': 'v1', 'addr': '172.21.15.138:6833', 'nonce': 1599292735}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.138:6830', 'nonce': 1599292735}, {'type': 'v1', 'addr': '172.21.15.138:6831', 'nonce': 1599292735}]}, 'public_addr': '172.21.15.138:6827/1599292735', 'cluster_addr': '172.21.15.138:6829/1599292735', 'heartbeat_back_addr': '172.21.15.138:6833/1599292735', 'heartbeat_front_addr': '172.21.15.138:6831/1599292735', 'state': ['exists', 'up']}, {'osd': 6, 'uuid': 'e55d11fd-433d-4462-93a7-d5e84560221b', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 23, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.138:6818', 'nonce': 163531479}, {'type': 'v1', 'addr': '172.21.15.138:6819', 'nonce': 163531479}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.138:6820', 'nonce': 163531479}, {'type': 'v1', 'addr': '172.21.15.138:6821', 'nonce': 163531479}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.138:6824', 'nonce': 163531479}, {'type': 'v1', 'addr': '172.21.15.138:6825', 'nonce': 163531479}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.138:6822', 'nonce': 163531479}, {'type': 'v1', 'addr': '172.21.15.138:6823', 'nonce': 163531479}]}, 'public_addr': '172.21.15.138:6819/163531479', 'cluster_addr': '172.21.15.138:6821/163531479', 'heartbeat_back_addr': '172.21.15.138:6825/163531479', 'heartbeat_front_addr': '172.21.15.138:6823/163531479', 'state': ['exists', 'up']}, {'osd': 7, 'uuid': 'ba14ba69-2c6c-4d09-81e2-2593dc3a7ea9', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 23, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.138:6800', 'nonce': 218507235}, {'type': 'v1', 'addr': '172.21.15.138:6801', 'nonce': 218507235}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.138:6802', 'nonce': 218507235}, {'type': 'v1', 'addr': '172.21.15.138:6803', 'nonce': 218507235}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.138:6806', 'nonce': 218507235}, {'type': 'v1', 'addr': '172.21.15.138:6807', 'nonce': 218507235}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.138:6804', 'nonce': 218507235}, {'type': 'v1', 'addr': '172.21.15.138:6805', 'nonce': 218507235}]}, 'public_addr': '172.21.15.138:6801/218507235', 'cluster_addr': '172.21.15.138:6803/218507235', 'heartbeat_back_addr': '172.21.15.138:6807/218507235', 'heartbeat_front_addr': '172.21.15.138:6805/218507235', 'state': ['exists', 'up']}], 'osd_xinfo': [{'osd': 0, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-14T06:28:39.906639+0000', 'dead_epoch': 0}, {'osd': 1, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-14T06:28:40.541279+0000', 'dead_epoch': 0}, {'osd': 2, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-14T06:28:40.025350+0000', 'dead_epoch': 0}, {'osd': 3, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-14T06:28:39.217944+0000', 'dead_epoch': 0}, {'osd': 4, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-14T06:28:40.185898+0000', 'dead_epoch': 0}, {'osd': 5, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-14T06:28:40.783504+0000', 'dead_epoch': 0}, {'osd': 6, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-14T06:28:40.527305+0000', 'dead_epoch': 0}, {'osd': 7, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-14T06:28:40.349676+0000', 'dead_epoch': 0}], 'pg_upmap': [], 'pg_upmap_items': [], 'pg_upmap_primaries': [], 'pg_temp': [], 'primary_temp': [], 'blocklist': {}, 'range_blocklist': {}, 'erasure_code_profiles': {'default': {'crush-failure-domain': 'osd', 'k': '2', 'm': '1', 'plugin': 'jerasure', 'technique': 'reed_sol_van'}}, 'removed_snaps_queue': [{'pool': 3, 'snaps': [{'begin': 24, 'length': 1}, {'begin': 26, 'length': 1}]}], 'new_removed_snaps': [], 'new_purged_snaps': [{'pool': 3, 'snaps': [{'begin': 22, 'length': 1}]}], 'crush_node_flags': {}, 'device_class_flags': {}, 'stretch_mode': {'stretch_mode_enabled': False, 'stretch_bucket_count': 0, 'degraded_stretch_mode': 0, 'recovering_stretch_mode': 0, 'stretch_mode_bucket': 0}} 2024-09-14T06:29:51.310 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-14T06:29:51.311 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-14T06:29:51.311 INFO:tasks.ceph.osd.1.smithi094.stderr:2024-09-14T06:29:51.307+0000 7f328e47a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33641) UID: 0 2024-09-14T06:29:51.410 INFO:tasks.ceph.osd.7.smithi138.stderr:2024-09-14T06:29:51.408+0000 7f3c6a21d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33623) UID: 0 2024-09-14T06:29:51.457 INFO:teuthology.orchestra.run.smithi094.stdout:ERROR: (22) Invalid argument 2024-09-14T06:29:51.458 INFO:teuthology.orchestra.run.smithi094.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:29:51.467 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-14T06:29:51.510 INFO:tasks.ceph.osd.4.smithi138.stderr:2024-09-14T06:29:51.509+0000 7f6c653c8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33615) UID: 0 2024-09-14T06:29:51.570 INFO:teuthology.orchestra.run.smithi094.stdout:ERROR: (22) Invalid argument 2024-09-14T06:29:51.570 INFO:teuthology.orchestra.run.smithi094.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:29:51.579 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-14T06:29:51.623 INFO:tasks.ceph.osd.1.smithi094.stderr:2024-09-14T06:29:51.622+0000 7f328e47a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33641) UID: 0 2024-09-14T06:29:51.641 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-14T06:29:51.642 INFO:teuthology.orchestra.run.smithi094.stderr:dumped all 2024-09-14T06:29:51.654 INFO:teuthology.orchestra.run.smithi094.stdout:{"pg_ready":true,"pg_map":{"version":99,"stamp":"2024-09-14T06:29:51.291980+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":619909020,"num_objects":513,"num_object_clones":0,"num_object_copies":1026,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":513,"num_whiteouts":0,"num_read":61784,"num_read_kb":354253,"num_write":6694,"num_write_kb":299926,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":4675,"ondisk_log_size":4675,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":308972,"kb_used_data":34484,"kb_used_omap":12,"kb_used_meta":274291,"kb_avail":754665748,"statfs":{"total":773094113280,"available":772777725952,"internally_reserved":0,"allocated":35311616,"data_stored":214048454,"data_compressed":13464364,"data_compressed_allocated":18149376,"data_compressed_original":202591878,"omap_allocated":12751,"internal_metadata":280874545},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":136687206,"num_objects":82,"num_object_clones":0,"num_object_copies":164,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":82,"num_whiteouts":0,"num_read":14391,"num_read_kb":89245,"num_write":924,"num_write_kb":66036,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.000696"},"pg_stats":[{"pgid":"3.1f","version":"65'59","reported_seq":2110,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-14T06:29:47.819686+0000","last_change":"2024-09-14T06:29:45.738645+0000","last_active":"2024-09-14T06:29:47.819686+0000","last_peered":"2024-09-14T06:29:47.819686+0000","last_clean":"2024-09-14T06:29:47.819686+0000","last_became_active":"2024-09-14T06:29:00.579883+0000","last_became_peered":"2024-09-14T06:29:00.579883+0000","last_unstale":"2024-09-14T06:29:47.819686+0000","last_undegraded":"2024-09-14T06:29:47.819686+0000","last_fullsized":"2024-09-14T06:29:47.819686+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":59,"log_dups_size":0,"ondisk_log_size":59,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T07:37:18.561215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000438054,"stat_sum":{"num_bytes":20082011,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":0,"num_read":1874,"num_read_kb":17378,"num_write":87,"num_write_kb":7967,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"16","length":"1"},{"start":"18","length":"1"}]},{"pgid":"3.1e","version":"70'52","reported_seq":2028,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-14T06:29:50.814895+0000","last_change":"2024-09-14T06:29:48.758689+0000","last_active":"2024-09-14T06:29:50.814895+0000","last_peered":"2024-09-14T06:29:50.814895+0000","last_clean":"2024-09-14T06:29:50.814895+0000","last_became_active":"2024-09-14T06:29:00.580507+0000","last_became_peered":"2024-09-14T06:29:00.580507+0000","last_unstale":"2024-09-14T06:29:50.814895+0000","last_undegraded":"2024-09-14T06:29:50.814895+0000","last_fullsized":"2024-09-14T06:29:50.814895+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":52,"log_dups_size":0,"ondisk_log_size":52,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T13:19:42.210607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097948400000000008,"stat_sum":{"num_bytes":13343014,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":0,"num_read":1784,"num_read_kb":7055,"num_write":90,"num_write_kb":7530,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"18","length":"1"},{"start":"1a","length":"1"}]},{"pgid":"3.1d","version":"67'374","reported_seq":3033,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-14T06:29:48.814811+0000","last_change":"2024-09-14T06:29:48.815071+0000","last_active":"2024-09-14T06:29:48.814811+0000","last_peered":"2024-09-14T06:29:48.814811+0000","last_clean":"2024-09-14T06:29:48.814811+0000","last_became_active":"2024-09-14T06:29:00.579345+0000","last_became_peered":"2024-09-14T06:29:00.579345+0000","last_unstale":"2024-09-14T06:29:48.814811+0000","last_undegraded":"2024-09-14T06:29:48.814811+0000","last_fullsized":"2024-09-14T06:29:48.814811+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":374,"log_dups_size":0,"ondisk_log_size":374,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T15:44:56.550530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064099800000000005,"stat_sum":{"num_bytes":30400134,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":0,"num_read":3927,"num_read_kb":9183,"num_write":610,"num_write_kb":12739,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"16","length":"1"},{"start":"18","length":"1"},{"start":"1a","length":"1"}]},{"pgid":"3.1c","version":"67'151","reported_seq":1309,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-14T06:29:48.814507+0000","last_change":"2024-09-14T06:29:48.814866+0000","last_active":"2024-09-14T06:29:48.814507+0000","last_peered":"2024-09-14T06:29:48.814507+0000","last_clean":"2024-09-14T06:29:48.814507+0000","last_became_active":"2024-09-14T06:29:00.579807+0000","last_became_peered":"2024-09-14T06:29:00.579807+0000","last_unstale":"2024-09-14T06:29:48.814507+0000","last_undegraded":"2024-09-14T06:29:48.814507+0000","last_fullsized":"2024-09-14T06:29:48.814507+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":151,"log_dups_size":0,"ondisk_log_size":151,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T08:15:06.295646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066884500000000005,"stat_sum":{"num_bytes":21897840,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":0,"num_read":1512,"num_read_kb":6535,"num_write":203,"num_write_kb":11622,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"16","length":"1"},{"start":"18","length":"1"},{"start":"1a","length":"1"}]},{"pgid":"3.1b","version":"65'61","reported_seq":1776,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-14T06:29:47.817320+0000","last_change":"2024-09-14T06:29:45.739024+0000","last_active":"2024-09-14T06:29:47.817320+0000","last_peered":"2024-09-14T06:29:47.817320+0000","last_clean":"2024-09-14T06:29:47.817320+0000","last_became_active":"2024-09-14T06:29:00.586183+0000","last_became_peered":"2024-09-14T06:29:00.586183+0000","last_unstale":"2024-09-14T06:29:47.817320+0000","last_undegraded":"2024-09-14T06:29:47.817320+0000","last_fullsized":"2024-09-14T06:29:47.817320+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":61,"log_dups_size":0,"ondisk_log_size":61,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T14:52:40.608898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057055199999999999,"stat_sum":{"num_bytes":8472568,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":0,"num_read":1536,"num_read_kb":4520,"num_write":73,"num_write_kb":6964,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"16","length":"1"},{"start":"18","length":"1"}]},{"pgid":"3.1a","version":"65'637","reported_seq":1720,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-14T06:29:48.814606+0000","last_change":"2024-09-14T06:29:48.815230+0000","last_active":"2024-09-14T06:29:48.814606+0000","last_peered":"2024-09-14T06:29:48.814606+0000","last_clean":"2024-09-14T06:29:48.814606+0000","last_became_active":"2024-09-14T06:29:00.576206+0000","last_became_peered":"2024-09-14T06:29:00.576206+0000","last_unstale":"2024-09-14T06:29:48.814606+0000","last_undegraded":"2024-09-14T06:29:48.814606+0000","last_fullsized":"2024-09-14T06:29:48.814606+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":637,"log_dups_size":0,"ondisk_log_size":637,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T10:37:35.795425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001350553,"stat_sum":{"num_bytes":13591082,"num_objects":20,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":0,"num_read":3826,"num_read_kb":6371,"num_write":1016,"num_write_kb":8817,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"16","length":"1"},{"start":"18","length":"1"},{"start":"1a","length":"1"}]},{"pgid":"3.19","version":"67'167","reported_seq":1668,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-14T06:29:48.764737+0000","last_change":"2024-09-14T06:29:48.764959+0000","last_active":"2024-09-14T06:29:48.764737+0000","last_peered":"2024-09-14T06:29:48.764737+0000","last_clean":"2024-09-14T06:29:48.764737+0000","last_became_active":"2024-09-14T06:29:00.575639+0000","last_became_peered":"2024-09-14T06:29:00.575639+0000","last_unstale":"2024-09-14T06:29:48.764737+0000","last_undegraded":"2024-09-14T06:29:48.764737+0000","last_fullsized":"2024-09-14T06:29:48.764737+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":167,"log_dups_size":0,"ondisk_log_size":167,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T09:05:36.516106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00046624900000000001,"stat_sum":{"num_bytes":31274213,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":0,"num_read":1873,"num_read_kb":22962,"num_write":264,"num_write_kb":14399,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"16","length":"1"},{"start":"18","length":"1"},{"start":"1a","length":"1"}]},{"pgid":"3.18","version":"70'93","reported_seq":1557,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-14T06:29:50.816162+0000","last_change":"2024-09-14T06:29:48.758492+0000","last_active":"2024-09-14T06:29:50.816162+0000","last_peered":"2024-09-14T06:29:50.816162+0000","last_clean":"2024-09-14T06:29:50.816162+0000","last_became_active":"2024-09-14T06:29:00.578017+0000","last_became_peered":"2024-09-14T06:29:00.578017+0000","last_unstale":"2024-09-14T06:29:50.816162+0000","last_undegraded":"2024-09-14T06:29:50.816162+0000","last_fullsized":"2024-09-14T06:29:50.816162+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":93,"log_dups_size":0,"ondisk_log_size":93,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T08:50:19.102413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001000936,"stat_sum":{"num_bytes":14560997,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":1336,"num_read_kb":10301,"num_write":97,"num_write_kb":7723,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"18","length":"1"},{"start":"1a","length":"1"}]},{"pgid":"3.17","version":"66'274","reported_seq":5623,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-14T06:29:47.819439+0000","last_change":"2024-09-14T06:29:45.739326+0000","last_active":"2024-09-14T06:29:47.819439+0000","last_peered":"2024-09-14T06:29:47.819439+0000","last_clean":"2024-09-14T06:29:47.819439+0000","last_became_active":"2024-09-14T06:29:00.579714+0000","last_became_peered":"2024-09-14T06:29:00.579714+0000","last_unstale":"2024-09-14T06:29:47.819439+0000","last_undegraded":"2024-09-14T06:29:47.819439+0000","last_fullsized":"2024-09-14T06:29:47.819439+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":274,"log_dups_size":0,"ondisk_log_size":274,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T10:10:27.757121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053868199999999996,"stat_sum":{"num_bytes":40592411,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":6125,"num_read_kb":28760,"num_write":320,"num_write_kb":11700,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"16","length":"1"},{"start":"18","length":"1"}]},{"pgid":"3.16","version":"65'77","reported_seq":473,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-14T06:29:48.814565+0000","last_change":"2024-09-14T06:29:48.814762+0000","last_active":"2024-09-14T06:29:48.814565+0000","last_peered":"2024-09-14T06:29:48.814565+0000","last_clean":"2024-09-14T06:29:48.814565+0000","last_became_active":"2024-09-14T06:29:00.580070+0000","last_became_peered":"2024-09-14T06:29:00.580070+0000","last_unstale":"2024-09-14T06:29:48.814565+0000","last_undegraded":"2024-09-14T06:29:48.814565+0000","last_fullsized":"2024-09-14T06:29:48.814565+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":77,"log_dups_size":0,"ondisk_log_size":77,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T12:44:31.627224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00043228,"stat_sum":{"num_bytes":9944089,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":0,"num_read":270,"num_read_kb":4619,"num_write":104,"num_write_kb":6435,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"16","length":"1"},{"start":"18","length":"1"},{"start":"1a","length":"1"}]},{"pgid":"3.15","version":"65'86","reported_seq":1377,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-14T06:29:48.763080+0000","last_change":"2024-09-14T06:29:48.763245+0000","last_active":"2024-09-14T06:29:48.763080+0000","last_peered":"2024-09-14T06:29:48.763080+0000","last_clean":"2024-09-14T06:29:48.763080+0000","last_became_active":"2024-09-14T06:29:00.578451+0000","last_became_peered":"2024-09-14T06:29:00.578451+0000","last_unstale":"2024-09-14T06:29:48.763080+0000","last_undegraded":"2024-09-14T06:29:48.763080+0000","last_fullsized":"2024-09-14T06:29:48.763080+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":86,"log_dups_size":0,"ondisk_log_size":86,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T09:37:07.797600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057621199999999999,"stat_sum":{"num_bytes":10933142,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":1283,"num_read_kb":11376,"num_write":111,"num_write_kb":9631,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"16","length":"1"},{"start":"18","length":"1"},{"start":"1a","length":"1"}]},{"pgid":"3.14","version":"65'237","reported_seq":1768,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-14T06:29:48.814149+0000","last_change":"2024-09-14T06:29:48.814387+0000","last_active":"2024-09-14T06:29:48.814149+0000","last_peered":"2024-09-14T06:29:48.814149+0000","last_clean":"2024-09-14T06:29:48.814149+0000","last_became_active":"2024-09-14T06:29:00.582595+0000","last_became_peered":"2024-09-14T06:29:00.582595+0000","last_unstale":"2024-09-14T06:29:48.814149+0000","last_undegraded":"2024-09-14T06:29:48.814149+0000","last_fullsized":"2024-09-14T06:29:48.814149+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":237,"log_dups_size":0,"ondisk_log_size":237,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T14:11:36.136748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00060941599999999999,"stat_sum":{"num_bytes":8751218,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":3048,"num_read_kb":4217,"num_write":252,"num_write_kb":4146,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"16","length":"1"},{"start":"18","length":"1"},{"start":"1a","length":"1"}]},{"pgid":"2.7","version":"0'0","reported_seq":110,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-14T06:29:48.758171+0000","last_change":"2024-09-14T06:28:53.625193+0000","last_active":"2024-09-14T06:29:48.758171+0000","last_peered":"2024-09-14T06:29:48.758171+0000","last_clean":"2024-09-14T06:29:48.758171+0000","last_became_active":"2024-09-14T06:28:51.223535+0000","last_became_peered":"2024-09-14T06:28:51.223535+0000","last_unstale":"2024-09-14T06:29:48.758171+0000","last_undegraded":"2024-09-14T06:29:48.758171+0000","last_fullsized":"2024-09-14T06:29:48.758171+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:28:50.205350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:28:50.205350+0000","last_clean_scrub_stamp":"2024-09-14T06:28:50.205350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T18:14:22.822245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00052660300000000005,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"66'95","reported_seq":1023,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-14T06:29:47.796054+0000","last_change":"2024-09-14T06:29:45.739018+0000","last_active":"2024-09-14T06:29:47.796054+0000","last_peered":"2024-09-14T06:29:47.796054+0000","last_clean":"2024-09-14T06:29:47.796054+0000","last_became_active":"2024-09-14T06:29:00.578283+0000","last_became_peered":"2024-09-14T06:29:00.578283+0000","last_unstale":"2024-09-14T06:29:47.796054+0000","last_undegraded":"2024-09-14T06:29:47.796054+0000","last_fullsized":"2024-09-14T06:29:47.796054+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":95,"log_dups_size":0,"ondisk_log_size":95,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T18:10:47.015523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065807400000000003,"stat_sum":{"num_bytes":8601058,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":0,"num_read":905,"num_read_kb":2934,"num_write":99,"num_write_kb":8262,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"16","length":"1"},{"start":"18","length":"1"}]},{"pgid":"2.6","version":"0'0","reported_seq":112,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-14T06:29:48.764783+0000","last_change":"2024-09-14T06:28:53.226122+0000","last_active":"2024-09-14T06:29:48.764783+0000","last_peered":"2024-09-14T06:29:48.764783+0000","last_clean":"2024-09-14T06:29:48.764783+0000","last_became_active":"2024-09-14T06:28:51.217205+0000","last_became_peered":"2024-09-14T06:28:51.217205+0000","last_unstale":"2024-09-14T06:29:48.764783+0000","last_undegraded":"2024-09-14T06:29:48.764783+0000","last_fullsized":"2024-09-14T06:29:48.764783+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:28:50.205350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:28:50.205350+0000","last_clean_scrub_stamp":"2024-09-14T06:28:50.205350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T15:29:19.563302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065018200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"70'251","reported_seq":1083,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-14T06:29:50.578488+0000","last_change":"2024-09-14T06:29:48.758333+0000","last_active":"2024-09-14T06:29:50.578488+0000","last_peered":"2024-09-14T06:29:50.578488+0000","last_clean":"2024-09-14T06:29:50.578488+0000","last_became_active":"2024-09-14T06:29:00.579908+0000","last_became_peered":"2024-09-14T06:29:00.579908+0000","last_unstale":"2024-09-14T06:29:50.578488+0000","last_undegraded":"2024-09-14T06:29:50.578488+0000","last_fullsized":"2024-09-14T06:29:50.578488+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":251,"log_dups_size":0,"ondisk_log_size":251,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T07:39:53.368764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00078681499999999995,"stat_sum":{"num_bytes":18419560,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":0,"num_read":1445,"num_read_kb":3423,"num_write":378,"num_write_kb":6842,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"18","length":"1"},{"start":"1a","length":"1"}]},{"pgid":"2.5","version":"0'0","reported_seq":112,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-14T06:29:48.762891+0000","last_change":"2024-09-14T06:28:53.640035+0000","last_active":"2024-09-14T06:29:48.762891+0000","last_peered":"2024-09-14T06:29:48.762891+0000","last_clean":"2024-09-14T06:29:48.762891+0000","last_became_active":"2024-09-14T06:28:51.308058+0000","last_became_peered":"2024-09-14T06:28:51.308058+0000","last_unstale":"2024-09-14T06:29:48.762891+0000","last_undegraded":"2024-09-14T06:29:48.762891+0000","last_fullsized":"2024-09-14T06:29:48.762891+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:28:50.205350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:28:50.205350+0000","last_clean_scrub_stamp":"2024-09-14T06:28:50.205350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T13:35:48.544057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00037824000000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"65'78","reported_seq":398,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-14T06:29:48.764843+0000","last_change":"2024-09-14T06:29:48.765042+0000","last_active":"2024-09-14T06:29:48.764843+0000","last_peered":"2024-09-14T06:29:48.764843+0000","last_clean":"2024-09-14T06:29:48.764843+0000","last_became_active":"2024-09-14T06:29:00.574895+0000","last_became_peered":"2024-09-14T06:29:00.574895+0000","last_unstale":"2024-09-14T06:29:48.764843+0000","last_undegraded":"2024-09-14T06:29:48.764843+0000","last_fullsized":"2024-09-14T06:29:48.764843+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":78,"log_dups_size":0,"ondisk_log_size":78,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T12:35:26.422818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039232500000000001,"stat_sum":{"num_bytes":4610095,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":295,"num_read_kb":3090,"num_write":110,"num_write_kb":5619,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"16","length":"1"},{"start":"18","length":"1"},{"start":"1a","length":"1"}]},{"pgid":"2.4","version":"0'0","reported_seq":112,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-14T06:29:48.764649+0000","last_change":"2024-09-14T06:28:53.226072+0000","last_active":"2024-09-14T06:29:48.764649+0000","last_peered":"2024-09-14T06:29:48.764649+0000","last_clean":"2024-09-14T06:29:48.764649+0000","last_became_active":"2024-09-14T06:28:51.308072+0000","last_became_peered":"2024-09-14T06:28:51.308072+0000","last_unstale":"2024-09-14T06:29:48.764649+0000","last_undegraded":"2024-09-14T06:29:48.764649+0000","last_fullsized":"2024-09-14T06:29:48.764649+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:28:50.205350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:28:50.205350+0000","last_clean_scrub_stamp":"2024-09-14T06:28:50.205350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T13:15:09.052328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066197399999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"65'88","reported_seq":622,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-14T06:29:48.814940+0000","last_change":"2024-09-14T06:29:48.815184+0000","last_active":"2024-09-14T06:29:48.814940+0000","last_peered":"2024-09-14T06:29:48.814940+0000","last_clean":"2024-09-14T06:29:48.814940+0000","last_became_active":"2024-09-14T06:29:00.581874+0000","last_became_peered":"2024-09-14T06:29:00.581874+0000","last_unstale":"2024-09-14T06:29:48.814940+0000","last_undegraded":"2024-09-14T06:29:48.814940+0000","last_fullsized":"2024-09-14T06:29:48.814940+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":88,"log_dups_size":0,"ondisk_log_size":88,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T14:29:18.624065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062003000000000004,"stat_sum":{"num_bytes":13140819,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":0,"num_read":358,"num_read_kb":1861,"num_write":113,"num_write_kb":9656,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"16","length":"1"},{"start":"18","length":"1"},{"start":"1a","length":"1"}]},{"pgid":"3.3","version":"67'78","reported_seq":3887,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-14T06:29:48.814205+0000","last_change":"2024-09-14T06:29:48.814389+0000","last_active":"2024-09-14T06:29:48.814205+0000","last_peered":"2024-09-14T06:29:48.814205+0000","last_clean":"2024-09-14T06:29:48.814205+0000","last_became_active":"2024-09-14T06:29:00.584678+0000","last_became_peered":"2024-09-14T06:29:00.584678+0000","last_unstale":"2024-09-14T06:29:48.814205+0000","last_undegraded":"2024-09-14T06:29:48.814205+0000","last_fullsized":"2024-09-14T06:29:48.814205+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":78,"log_dups_size":0,"ondisk_log_size":78,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T08:11:31.020371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050989099999999995,"stat_sum":{"num_bytes":36881480,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":0,"num_read":3863,"num_read_kb":20705,"num_write":100,"num_write_kb":8843,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"16","length":"1"},{"start":"18","length":"1"},{"start":"1a","length":"1"}]},{"pgid":"2.2","version":"22'2","reported_seq":114,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-14T06:29:48.814056+0000","last_change":"2024-09-14T06:28:53.231428+0000","last_active":"2024-09-14T06:29:48.814056+0000","last_peered":"2024-09-14T06:29:48.814056+0000","last_clean":"2024-09-14T06:29:48.814056+0000","last_became_active":"2024-09-14T06:28:51.222937+0000","last_became_peered":"2024-09-14T06:28:51.222937+0000","last_unstale":"2024-09-14T06:29:48.814056+0000","last_undegraded":"2024-09-14T06:29:48.814056+0000","last_fullsized":"2024-09-14T06:29:48.814056+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:28:50.205350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:28:50.205350+0000","last_clean_scrub_stamp":"2024-09-14T06:28:50.205350+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T14:27:37.233976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010072970000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.0","version":"65'81","reported_seq":3216,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-14T06:29:48.765306+0000","last_change":"2024-09-14T06:29:48.765514+0000","last_active":"2024-09-14T06:29:48.765306+0000","last_peered":"2024-09-14T06:29:48.765306+0000","last_clean":"2024-09-14T06:29:48.765306+0000","last_became_active":"2024-09-14T06:29:00.574873+0000","last_became_peered":"2024-09-14T06:29:00.574873+0000","last_unstale":"2024-09-14T06:29:48.765306+0000","last_undegraded":"2024-09-14T06:29:48.765306+0000","last_fullsized":"2024-09-14T06:29:48.765306+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T14:05:01.082785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000488872,"stat_sum":{"num_bytes":28015453,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":2959,"num_read_kb":24989,"num_write":115,"num_write_kb":12669,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"16","length":"1"},{"start":"18","length":"1"},{"start":"1a","length":"1"}]},{"pgid":"2.1","version":"0'0","reported_seq":109,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-14T06:29:47.753133+0000","last_change":"2024-09-14T06:28:53.226264+0000","last_active":"2024-09-14T06:29:47.753133+0000","last_peered":"2024-09-14T06:29:47.753133+0000","last_clean":"2024-09-14T06:29:47.753133+0000","last_became_active":"2024-09-14T06:28:51.223252+0000","last_became_peered":"2024-09-14T06:28:51.223252+0000","last_unstale":"2024-09-14T06:29:47.753133+0000","last_undegraded":"2024-09-14T06:29:47.753133+0000","last_fullsized":"2024-09-14T06:29:47.753133+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:28:50.205350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:28:50.205350+0000","last_clean_scrub_stamp":"2024-09-14T06:28:50.205350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T11:56:04.373414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053403100000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1","version":"65'40","reported_seq":1141,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-14T06:29:47.813601+0000","last_change":"2024-09-14T06:29:45.738577+0000","last_active":"2024-09-14T06:29:47.813601+0000","last_peered":"2024-09-14T06:29:47.813601+0000","last_clean":"2024-09-14T06:29:47.813601+0000","last_became_active":"2024-09-14T06:29:00.585673+0000","last_became_peered":"2024-09-14T06:29:00.585673+0000","last_unstale":"2024-09-14T06:29:47.813601+0000","last_undegraded":"2024-09-14T06:29:47.813601+0000","last_fullsized":"2024-09-14T06:29:47.813601+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":40,"log_dups_size":0,"ondisk_log_size":40,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T17:02:30.698392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00044192899999999998,"stat_sum":{"num_bytes":13175713,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":942,"num_read_kb":14884,"num_write":56,"num_write_kb":3226,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"16","length":"1"},{"start":"18","length":"1"}]},{"pgid":"2.0","version":"0'0","reported_seq":112,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-14T06:29:48.762457+0000","last_change":"2024-09-14T06:28:53.640096+0000","last_active":"2024-09-14T06:29:48.762457+0000","last_peered":"2024-09-14T06:29:48.762457+0000","last_clean":"2024-09-14T06:29:48.762457+0000","last_became_active":"2024-09-14T06:28:51.223069+0000","last_became_peered":"2024-09-14T06:28:51.223069+0000","last_unstale":"2024-09-14T06:29:48.762457+0000","last_undegraded":"2024-09-14T06:29:48.762457+0000","last_fullsized":"2024-09-14T06:29:48.762457+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:28:50.205350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:28:50.205350+0000","last_clean_scrub_stamp":"2024-09-14T06:28:50.205350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T15:13:00.471509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00043714600000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"70'105","reported_seq":3499,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-14T06:29:50.695680+0000","last_change":"2024-09-14T06:29:48.758392+0000","last_active":"2024-09-14T06:29:50.695680+0000","last_peered":"2024-09-14T06:29:50.695680+0000","last_clean":"2024-09-14T06:29:50.695680+0000","last_became_active":"2024-09-14T06:29:00.578574+0000","last_became_peered":"2024-09-14T06:29:00.578574+0000","last_unstale":"2024-09-14T06:29:50.695680+0000","last_undegraded":"2024-09-14T06:29:50.695680+0000","last_fullsized":"2024-09-14T06:29:50.695680+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":105,"log_dups_size":0,"ondisk_log_size":105,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T14:12:40.683913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011063430000000001,"stat_sum":{"num_bytes":35905085,"num_objects":30,"num_object_clones":0,"num_object_copies":60,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":30,"num_whiteouts":0,"num_read":3206,"num_read_kb":27654,"num_write":139,"num_write_kb":17468,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"18","length":"1"},{"start":"1a","length":"1"}]},{"pgid":"2.3","version":"20'1","reported_seq":113,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-14T06:29:48.814130+0000","last_change":"2024-09-14T06:28:53.230933+0000","last_active":"2024-09-14T06:29:48.814130+0000","last_peered":"2024-09-14T06:29:48.814130+0000","last_clean":"2024-09-14T06:29:48.814130+0000","last_became_active":"2024-09-14T06:28:51.222874+0000","last_became_peered":"2024-09-14T06:28:51.222874+0000","last_unstale":"2024-09-14T06:29:48.814130+0000","last_undegraded":"2024-09-14T06:29:48.814130+0000","last_fullsized":"2024-09-14T06:29:48.814130+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:28:50.205350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:28:50.205350+0000","last_clean_scrub_stamp":"2024-09-14T06:28:50.205350+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T12:04:25.190171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050808899999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'72","reported_seq":203,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-14T06:29:48.762487+0000","last_change":"2024-09-14T06:28:44.839885+0000","last_active":"2024-09-14T06:29:48.762487+0000","last_peered":"2024-09-14T06:29:48.762487+0000","last_clean":"2024-09-14T06:29:48.762487+0000","last_became_active":"2024-09-14T06:28:44.839573+0000","last_became_peered":"2024-09-14T06:28:44.839573+0000","last_unstale":"2024-09-14T06:29:48.762487+0000","last_undegraded":"2024-09-14T06:29:48.762487+0000","last_fullsized":"2024-09-14T06:29:48.762487+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:28:43.828494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:28:43.828494+0000","last_clean_scrub_stamp":"2024-09-14T06:28:43.828494+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T14:41:25.787293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.8","version":"70'116","reported_seq":2184,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-14T06:29:50.815928+0000","last_change":"2024-09-14T06:29:48.758728+0000","last_active":"2024-09-14T06:29:50.815928+0000","last_peered":"2024-09-14T06:29:50.815928+0000","last_clean":"2024-09-14T06:29:50.815928+0000","last_became_active":"2024-09-14T06:29:00.574458+0000","last_became_peered":"2024-09-14T06:29:00.574458+0000","last_unstale":"2024-09-14T06:29:50.815928+0000","last_undegraded":"2024-09-14T06:29:50.815928+0000","last_fullsized":"2024-09-14T06:29:50.815928+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":116,"log_dups_size":0,"ondisk_log_size":116,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T08:36:53.800846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00095950400000000002,"stat_sum":{"num_bytes":26902879,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":0,"num_read":2387,"num_read_kb":22372,"num_write":150,"num_write_kb":13301,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"18","length":"1"},{"start":"1a","length":"1"}]},{"pgid":"3.9","version":"65'79","reported_seq":654,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-14T06:29:48.814767+0000","last_change":"2024-09-14T06:29:48.815072+0000","last_active":"2024-09-14T06:29:48.814767+0000","last_peered":"2024-09-14T06:29:48.814767+0000","last_clean":"2024-09-14T06:29:48.814767+0000","last_became_active":"2024-09-14T06:29:00.573179+0000","last_became_peered":"2024-09-14T06:29:00.573179+0000","last_unstale":"2024-09-14T06:29:48.814767+0000","last_undegraded":"2024-09-14T06:29:48.814767+0000","last_fullsized":"2024-09-14T06:29:48.814767+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T11:06:07.348263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001015838,"stat_sum":{"num_bytes":6971336,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":0,"num_read":387,"num_read_kb":7155,"num_write":96,"num_write_kb":9039,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"16","length":"1"},{"start":"18","length":"1"},{"start":"1a","length":"1"}]},{"pgid":"3.a","version":"65'54","reported_seq":1391,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-14T06:29:48.758517+0000","last_change":"2024-09-14T06:29:48.758729+0000","last_active":"2024-09-14T06:29:48.758517+0000","last_peered":"2024-09-14T06:29:48.758517+0000","last_clean":"2024-09-14T06:29:48.758517+0000","last_became_active":"2024-09-14T06:29:00.582426+0000","last_became_peered":"2024-09-14T06:29:00.582426+0000","last_unstale":"2024-09-14T06:29:48.758517+0000","last_undegraded":"2024-09-14T06:29:48.758517+0000","last_fullsized":"2024-09-14T06:29:48.758517+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":54,"log_dups_size":0,"ondisk_log_size":54,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T09:43:59.226168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00048726100000000001,"stat_sum":{"num_bytes":26873464,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":1153,"num_read_kb":16873,"num_write":79,"num_write_kb":9692,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"16","length":"1"},{"start":"18","length":"1"},{"start":"1a","length":"1"}]},{"pgid":"3.b","version":"70'87","reported_seq":1775,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-14T06:29:50.760748+0000","last_change":"2024-09-14T06:29:48.757733+0000","last_active":"2024-09-14T06:29:50.760748+0000","last_peered":"2024-09-14T06:29:50.760748+0000","last_clean":"2024-09-14T06:29:50.760748+0000","last_became_active":"2024-09-14T06:29:00.578483+0000","last_became_peered":"2024-09-14T06:29:00.578483+0000","last_unstale":"2024-09-14T06:29:50.760748+0000","last_undegraded":"2024-09-14T06:29:50.760748+0000","last_fullsized":"2024-09-14T06:29:50.760748+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":87,"log_dups_size":0,"ondisk_log_size":87,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T18:22:57.996974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047487700000000001,"stat_sum":{"num_bytes":18502993,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":1526,"num_read_kb":3719,"num_write":113,"num_write_kb":8804,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"18","length":"1"},{"start":"1a","length":"1"}]},{"pgid":"3.c","version":"67'74","reported_seq":1827,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-14T06:29:48.815118+0000","last_change":"2024-09-14T06:29:48.815436+0000","last_active":"2024-09-14T06:29:48.815118+0000","last_peered":"2024-09-14T06:29:48.815118+0000","last_clean":"2024-09-14T06:29:48.815118+0000","last_became_active":"2024-09-14T06:29:00.583406+0000","last_became_peered":"2024-09-14T06:29:00.583406+0000","last_unstale":"2024-09-14T06:29:48.815118+0000","last_undegraded":"2024-09-14T06:29:48.815118+0000","last_fullsized":"2024-09-14T06:29:48.815118+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":74,"log_dups_size":0,"ondisk_log_size":74,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T10:41:27.666507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00060933899999999997,"stat_sum":{"num_bytes":32318058,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":1565,"num_read_kb":6843,"num_write":98,"num_write_kb":10355,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"16","length":"1"},{"start":"18","length":"1"},{"start":"1a","length":"1"}]},{"pgid":"3.d","version":"67'105","reported_seq":1864,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-14T06:29:48.763522+0000","last_change":"2024-09-14T06:29:48.763720+0000","last_active":"2024-09-14T06:29:48.763522+0000","last_peered":"2024-09-14T06:29:48.763522+0000","last_clean":"2024-09-14T06:29:48.763522+0000","last_became_active":"2024-09-14T06:29:00.580362+0000","last_became_peered":"2024-09-14T06:29:00.580362+0000","last_unstale":"2024-09-14T06:29:48.763522+0000","last_undegraded":"2024-09-14T06:29:48.763522+0000","last_fullsized":"2024-09-14T06:29:48.763522+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":105,"log_dups_size":0,"ondisk_log_size":105,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T15:39:28.345128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00094614199999999997,"stat_sum":{"num_bytes":19552502,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":0,"num_read":1804,"num_read_kb":8962,"num_write":140,"num_write_kb":7604,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"16","length":"1"},{"start":"18","length":"1"},{"start":"1a","length":"1"}]},{"pgid":"3.e","version":"65'119","reported_seq":612,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-14T06:29:48.763580+0000","last_change":"2024-09-14T06:29:48.763781+0000","last_active":"2024-09-14T06:29:48.763580+0000","last_peered":"2024-09-14T06:29:48.763580+0000","last_clean":"2024-09-14T06:29:48.763580+0000","last_became_active":"2024-09-14T06:29:00.586374+0000","last_became_peered":"2024-09-14T06:29:00.586374+0000","last_unstale":"2024-09-14T06:29:48.763580+0000","last_undegraded":"2024-09-14T06:29:48.763580+0000","last_fullsized":"2024-09-14T06:29:48.763580+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":119,"log_dups_size":0,"ondisk_log_size":119,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T10:28:00.957143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00095293299999999999,"stat_sum":{"num_bytes":5726430,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":0,"num_read":564,"num_read_kb":3187,"num_write":195,"num_write_kb":8078,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"16","length":"1"},{"start":"18","length":"1"},{"start":"1a","length":"1"}]},{"pgid":"3.f","version":"65'120","reported_seq":2642,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-14T06:29:48.763309+0000","last_change":"2024-09-14T06:29:48.763483+0000","last_active":"2024-09-14T06:29:48.763309+0000","last_peered":"2024-09-14T06:29:48.763309+0000","last_clean":"2024-09-14T06:29:48.763309+0000","last_became_active":"2024-09-14T06:29:00.586491+0000","last_became_peered":"2024-09-14T06:29:00.586491+0000","last_unstale":"2024-09-14T06:29:48.763309+0000","last_undegraded":"2024-09-14T06:29:48.763309+0000","last_fullsized":"2024-09-14T06:29:48.763309+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":120,"log_dups_size":0,"ondisk_log_size":120,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T06:33:20.317175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070229799999999996,"stat_sum":{"num_bytes":14654878,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":0,"num_read":2579,"num_read_kb":10497,"num_write":173,"num_write_kb":8169,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"16","length":"1"},{"start":"18","length":"1"},{"start":"1a","length":"1"}]},{"pgid":"3.10","version":"65'92","reported_seq":547,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-14T06:29:48.758461+0000","last_change":"2024-09-14T06:29:48.758841+0000","last_active":"2024-09-14T06:29:48.758461+0000","last_peered":"2024-09-14T06:29:48.758461+0000","last_clean":"2024-09-14T06:29:48.758461+0000","last_became_active":"2024-09-14T06:29:00.579539+0000","last_became_peered":"2024-09-14T06:29:00.579539+0000","last_unstale":"2024-09-14T06:29:48.758461+0000","last_undegraded":"2024-09-14T06:29:48.758461+0000","last_fullsized":"2024-09-14T06:29:48.758461+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T10:05:52.497480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000600609,"stat_sum":{"num_bytes":9332925,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":0,"num_read":295,"num_read_kb":822,"num_write":120,"num_write_kb":8439,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"16","length":"1"},{"start":"18","length":"1"},{"start":"1a","length":"1"}]},{"pgid":"3.11","version":"67'176","reported_seq":2721,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-14T06:29:48.763049+0000","last_change":"2024-09-14T06:29:48.763193+0000","last_active":"2024-09-14T06:29:48.763049+0000","last_peered":"2024-09-14T06:29:48.763049+0000","last_clean":"2024-09-14T06:29:48.763049+0000","last_became_active":"2024-09-14T06:29:00.586480+0000","last_became_peered":"2024-09-14T06:29:00.586480+0000","last_unstale":"2024-09-14T06:29:48.763049+0000","last_undegraded":"2024-09-14T06:29:48.763049+0000","last_fullsized":"2024-09-14T06:29:48.763049+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":176,"log_dups_size":0,"ondisk_log_size":176,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T07:15:36.108934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059973100000000005,"stat_sum":{"num_bytes":31664246,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":0,"num_read":2816,"num_read_kb":23174,"num_write":274,"num_write_kb":13955,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"16","length":"1"},{"start":"18","length":"1"},{"start":"1a","length":"1"}]},{"pgid":"3.12","version":"67'226","reported_seq":2191,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-14T06:29:47.818783+0000","last_change":"2024-09-14T06:29:45.739327+0000","last_active":"2024-09-14T06:29:47.818783+0000","last_peered":"2024-09-14T06:29:47.818783+0000","last_clean":"2024-09-14T06:29:47.818783+0000","last_became_active":"2024-09-14T06:29:00.579826+0000","last_became_peered":"2024-09-14T06:29:00.579826+0000","last_unstale":"2024-09-14T06:29:47.818783+0000","last_undegraded":"2024-09-14T06:29:47.818783+0000","last_fullsized":"2024-09-14T06:29:47.818783+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":226,"log_dups_size":0,"ondisk_log_size":226,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T13:44:30.318106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066930899999999998,"stat_sum":{"num_bytes":10186161,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":0,"num_read":2645,"num_read_kb":12988,"num_write":352,"num_write_kb":5822,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"16","length":"1"},{"start":"18","length":"1"}]},{"pgid":"3.13","version":"67'268","reported_seq":1177,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-14T06:29:48.763247+0000","last_change":"2024-09-14T06:29:48.763481+0000","last_active":"2024-09-14T06:29:48.763247+0000","last_peered":"2024-09-14T06:29:48.763247+0000","last_clean":"2024-09-14T06:29:48.763247+0000","last_became_active":"2024-09-14T06:29:00.586520+0000","last_became_peered":"2024-09-14T06:29:00.586520+0000","last_unstale":"2024-09-14T06:29:48.763247+0000","last_undegraded":"2024-09-14T06:29:48.763247+0000","last_fullsized":"2024-09-14T06:29:48.763247+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":268,"log_dups_size":0,"ondisk_log_size":268,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T07:18:14.758506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00074591999999999996,"stat_sum":{"num_bytes":34040779,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":0,"num_read":1630,"num_read_kb":4789,"num_write":464,"num_write_kb":12780,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"16","length":"1"},{"start":"18","length":"1"},{"start":"1a","length":"1"}]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":619318633,"num_objects":509,"num_object_clones":0,"num_object_copies":1018,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":509,"num_whiteouts":0,"num_read":61718,"num_read_kb":354198,"num_write":6591,"num_write_kb":298296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":29585408,"data_stored":217652533,"data_compressed":13889515,"data_compressed_allocated":18677760,"data_compressed_original":209047174,"omap_allocated":0,"internal_metadata":0},"log_size":4600,"ondisk_log_size":4600,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":131072,"data_stored":1180736,"data_compressed":65166,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509456,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":39900,"kb_used_data":4548,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94331940,"statfs":{"total":96636764160,"available":96595906560,"internally_reserved":0,"allocated":4657152,"data_stored":28802846,"data_compressed":1614880,"data_compressed_allocated":2187264,"data_compressed_original":27096311,"omap_allocated":1590,"internal_metadata":36174282},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509455,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28704,"kb_used_data":1672,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343136,"statfs":{"total":96636764160,"available":96607371264,"internally_reserved":0,"allocated":1712128,"data_stored":7474438,"data_compressed":387963,"data_compressed_allocated":573440,"data_compressed_original":6833481,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509457,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":41052,"kb_used_data":5704,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94330788,"statfs":{"total":96636764160,"available":96594726912,"internally_reserved":0,"allocated":5840896,"data_stored":35999817,"data_compressed":2457259,"data_compressed_allocated":3268608,"data_compressed_original":34229445,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509456,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":46368,"kb_used_data":6856,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94325472,"statfs":{"total":96636764160,"available":96589283328,"internally_reserved":0,"allocated":7020544,"data_stored":50009073,"data_compressed":3228955,"data_compressed_allocated":4247552,"data_compressed_original":48126435,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509457,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":38740,"kb_used_data":3392,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94333100,"statfs":{"total":96636764160,"available":96597094400,"internally_reserved":0,"allocated":3473408,"data_stored":20241633,"data_compressed":1309112,"data_compressed_allocated":1789952,"data_compressed_original":19207351,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":14,"seq":60129542160,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33132,"kb_used_data":1940,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338708,"statfs":{"total":96636764160,"available":96602836992,"internally_reserved":0,"allocated":1986560,"data_stored":9679597,"data_compressed":592233,"data_compressed_allocated":835584,"data_compressed_original":9056969,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542159,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":39632,"kb_used_data":4280,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94332208,"statfs":{"total":96636764160,"available":96596180992,"internally_reserved":0,"allocated":4382720,"data_stored":21998194,"data_compressed":1311952,"data_compressed_allocated":1777664,"data_compressed_original":20162686,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574865,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":41444,"kb_used_data":6092,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94330396,"statfs":{"total":96636764160,"available":96594325504,"internally_reserved":0,"allocated":6238208,"data_stored":39842856,"data_compressed":2562010,"data_compressed_allocated":3469312,"data_compressed_original":37879200,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32583,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32583,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":3555328,"data_stored":21062605,"data_compressed":1279369,"data_compressed_allocated":1732608,"data_compressed_original":19589246,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":2695168,"data_stored":19890238,"data_compressed":1309112,"data_compressed_allocated":1789952,"data_compressed_original":19207351,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":1224704,"data_stored":9334376,"data_compressed":592233,"data_compressed_allocated":835584,"data_compressed_original":9056969,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":6057984,"data_stored":47081258,"data_compressed":3052327,"data_compressed_allocated":4087808,"data_compressed_original":45481376,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":6246400,"data_stored":49657697,"data_compressed":3228955,"data_compressed_allocated":4247552,"data_compressed_original":48126435,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":5062656,"data_stored":35648422,"data_compressed":2457259,"data_compressed_allocated":3268608,"data_compressed_original":34229445,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":925696,"data_stored":7116835,"data_compressed":387963,"data_compressed_allocated":573440,"data_compressed_original":6833481,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":3817472,"data_stored":27861102,"data_compressed":1582297,"data_compressed_allocated":2142208,"data_compressed_original":26522871,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-14T06:29:51.657 INFO:tasks.thrashosds.thrasher:Setting 3.7 to [3, 0, 1, 5] 2024-09-14T06:29:51.657 INFO:tasks.thrashosds.thrasher:cmd ['osd', 'pg-upmap-items', '3.7', '3', '0', '1', '5'] 2024-09-14T06:29:51.657 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pg-upmap-items 3.7 3 0 1 5 2024-09-14T06:29:51.684 INFO:teuthology.orchestra.run.smithi094.stdout:ERROR: (22) Invalid argument 2024-09-14T06:29:51.684 INFO:teuthology.orchestra.run.smithi094.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:29:51.693 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-14T06:29:51.712 INFO:tasks.ceph.osd.6.smithi138.stderr:2024-09-14T06:29:51.711+0000 7f8cd8849640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33620) UID: 0 2024-09-14T06:29:51.765 INFO:teuthology.orchestra.run.smithi094.stderr:nodeep-scrub is set 2024-09-14T06:29:51.824 INFO:tasks.ceph.osd.0.smithi094.stderr:2024-09-14T06:29:51.823+0000 7f07c9333640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33636) UID: 0 2024-09-14T06:29:51.847 INFO:teuthology.orchestra.run.smithi094.stdout:ERROR: (22) Invalid argument 2024-09-14T06:29:51.847 INFO:teuthology.orchestra.run.smithi094.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:29:51.857 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-14T06:29:51.871 INFO:teuthology.orchestra.run.smithi094.stdout:checking clone #11, image image_client.0-clone11 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent12 2024-09-14T06:29:51.915 INFO:tasks.ceph.osd.0.smithi094.stderr:2024-09-14T06:29:51.914+0000 7f07c9333640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33636) UID: 0 2024-09-14T06:29:51.996 INFO:teuthology.orchestra.run.smithi094.stdout:ERROR: (22) Invalid argument 2024-09-14T06:29:51.996 INFO:teuthology.orchestra.run.smithi094.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:29:52.005 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-14T06:29:52.047 INFO:tasks.ceph.osd.3.smithi094.stderr:2024-09-14T06:29:52.046+0000 7feeeafe8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33655) UID: 0 2024-09-14T06:29:52.116 INFO:tasks.ceph.osd.2.smithi094.stderr:2024-09-14T06:29:52.115+0000 7faf3269b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33649) UID: 0 2024-09-14T06:29:52.151 INFO:teuthology.orchestra.run.smithi094.stdout:1615 punch from 0x287247f to 0x287d233, (0xadb4 bytes) 2024-09-14T06:29:52.153 INFO:teuthology.orchestra.run.smithi094.stdout:ERROR: (22) Invalid argument 2024-09-14T06:29:52.154 INFO:teuthology.orchestra.run.smithi094.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:29:52.155 INFO:teuthology.orchestra.run.smithi094.stdout:1617 punch from 0x8e0f80 to 0x8eb559, (0xa5d9 bytes) 2024-09-14T06:29:52.160 INFO:teuthology.orchestra.run.smithi094.stdout:1618 trunc from 0x6c1b45d to 0x319757e 2024-09-14T06:29:52.166 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-14T06:29:52.215 INFO:teuthology.orchestra.run.smithi094.stdout:1619 trunc from 0x319757e to 0xba43786 2024-09-14T06:29:52.216 INFO:tasks.ceph.osd.2.smithi094.stderr:2024-09-14T06:29:52.215+0000 7faf3269b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33649) UID: 0 2024-09-14T06:29:52.221 INFO:teuthology.orchestra.run.smithi094.stdout:1621 read 0x92a86e9 thru 0x92a93c3 (0xcdb bytes) 2024-09-14T06:29:52.221 INFO:teuthology.orchestra.run.smithi094.stdout:1622 write 0x65f28be thru 0x65f46f4 (0x1e37 bytes) 2024-09-14T06:29:52.225 INFO:teuthology.orchestra.run.smithi094.stdout:1623 writesame 0x3162a90 thru 0x3168e5a data_size 0x1(0x63cb bytes) 2024-09-14T06:29:52.234 INFO:teuthology.orchestra.run.smithi094.stdout:1625 write 0x421420a thru 0x42147bc (0x5b3 bytes) 2024-09-14T06:29:52.237 INFO:teuthology.orchestra.run.smithi094.stdout:1626 punch from 0x495bbea to 0x496a304, (0xe71a bytes) 2024-09-14T06:29:52.237 INFO:teuthology.orchestra.run.smithi094.stdout:1627 write 0x4243801 thru 0x4245ae2 (0x22e2 bytes) 2024-09-14T06:29:52.249 INFO:teuthology.orchestra.run.smithi094.stdout:1628 write 0xe2510d5 thru 0xe258ca7 (0x7bd3 bytes) 2024-09-14T06:29:52.253 INFO:teuthology.orchestra.run.smithi094.stdout:1629 write 0x5bacfd3 thru 0x5bb5ff0 (0x901e bytes) 2024-09-14T06:29:52.257 INFO:teuthology.orchestra.run.smithi094.stdout:1630 writesame 0x3023444 thru 0x302aaf7 data_size 0x4(0x76b4 bytes) 2024-09-14T06:29:52.262 INFO:teuthology.orchestra.run.smithi094.stdout:1633 writesame 0xb07bd thru 0xb4bcd data_size 0x29(0x4411 bytes) 2024-09-14T06:29:52.264 INFO:teuthology.orchestra.run.smithi094.stdout:1636 writesame 0x2605df3 thru 0x2614c2a data_size 0x84(0xee38 bytes) 2024-09-14T06:29:52.270 INFO:teuthology.orchestra.run.smithi094.stdout:1639 punch from 0xd740da7 to 0xd7449bc, (0x3c15 bytes) 2024-09-14T06:29:52.270 INFO:teuthology.orchestra.run.smithi094.stdout:1640 read 0x6ad9 thru 0x6f62 (0x48a bytes) 2024-09-14T06:29:52.276 INFO:teuthology.orchestra.run.smithi094.stdout:1641 write 0xedae5cc thru 0xedb8294 (0x9cc9 bytes) 2024-09-14T06:29:52.284 INFO:teuthology.orchestra.run.smithi094.stdout:1643 clone 14 order 22 su 131072 sc 4 2024-09-14T06:29:52.317 INFO:tasks.ceph.osd.0.smithi094.stderr:2024-09-14T06:29:52.316+0000 7f07c9333640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33636) UID: 0 2024-09-14T06:29:52.320 INFO:teuthology.orchestra.run.smithi094.stdout:ERROR: (22) Invalid argument 2024-09-14T06:29:52.320 INFO:teuthology.orchestra.run.smithi094.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:29:52.329 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-14T06:29:52.418 INFO:tasks.ceph.osd.0.smithi094.stderr:2024-09-14T06:29:52.417+0000 7f07c9333640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33636) UID: 0 2024-09-14T06:29:52.473 INFO:teuthology.orchestra.run.smithi094.stdout:ERROR: (22) Invalid argument 2024-09-14T06:29:52.473 INFO:teuthology.orchestra.run.smithi094.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:29:52.481 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-14T06:29:52.522 INFO:tasks.ceph.osd.2.smithi094.stderr:2024-09-14T06:29:52.521+0000 7faf3269b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33649) UID: 0 2024-09-14T06:29:52.621 INFO:tasks.ceph.osd.1.smithi094.stderr:2024-09-14T06:29:52.620+0000 7f328e47a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33641) UID: 0 2024-09-14T06:29:52.623 INFO:teuthology.orchestra.run.smithi094.stdout:ERROR: (22) Invalid argument 2024-09-14T06:29:52.623 INFO:teuthology.orchestra.run.smithi094.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:29:52.632 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-14T06:29:52.657 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-14T06:29:52.722 INFO:tasks.ceph.osd.3.smithi094.stderr:2024-09-14T06:29:52.721+0000 7feeeafe8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33655) UID: 0 2024-09-14T06:29:52.736 INFO:teuthology.orchestra.run.smithi138.stdout:ERROR: (22) Invalid argument 2024-09-14T06:29:52.737 INFO:teuthology.orchestra.run.smithi138.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:29:52.745 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-14T06:29:52.770 INFO:teuthology.orchestra.run.smithi094.stderr:set 3.7 pg_upmap_items mapping to [3->0,1->5] 2024-09-14T06:29:52.797 INFO:teuthology.orchestra.run.smithi094.stdout:osd.0: {} 2024-09-14T06:29:52.797 INFO:teuthology.orchestra.run.smithi094.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-14T06:29:52.816 INFO:teuthology.orchestra.run.smithi094.stdout:osd.1: {} 2024-09-14T06:29:52.816 INFO:teuthology.orchestra.run.smithi094.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-14T06:29:52.822 INFO:tasks.ceph.osd.3.smithi094.stderr:2024-09-14T06:29:52.821+0000 7feeeafe8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33655) UID: 0 2024-09-14T06:29:52.833 INFO:teuthology.orchestra.run.smithi094.stdout:osd.2: {} 2024-09-14T06:29:52.833 INFO:teuthology.orchestra.run.smithi094.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-14T06:29:52.849 INFO:teuthology.orchestra.run.smithi094.stdout:osd.3: {} 2024-09-14T06:29:52.849 INFO:teuthology.orchestra.run.smithi094.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-14T06:29:52.853 INFO:teuthology.orchestra.run.smithi138.stdout:ERROR: (22) Invalid argument 2024-09-14T06:29:52.853 INFO:teuthology.orchestra.run.smithi138.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:29:52.862 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-14T06:29:52.865 INFO:teuthology.orchestra.run.smithi094.stdout:osd.4: {} 2024-09-14T06:29:52.865 INFO:teuthology.orchestra.run.smithi094.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-14T06:29:52.882 INFO:teuthology.orchestra.run.smithi094.stdout:osd.5: {} 2024-09-14T06:29:52.883 INFO:teuthology.orchestra.run.smithi094.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-14T06:29:52.900 INFO:teuthology.orchestra.run.smithi094.stdout:osd.6: {} 2024-09-14T06:29:52.900 INFO:teuthology.orchestra.run.smithi094.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-14T06:29:52.916 INFO:teuthology.orchestra.run.smithi094.stdout:osd.7: {} 2024-09-14T06:29:52.916 INFO:teuthology.orchestra.run.smithi094.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-14T06:29:52.923 INFO:tasks.ceph.osd.2.smithi094.stderr:2024-09-14T06:29:52.922+0000 7faf3269b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33649) UID: 0 2024-09-14T06:29:52.967 INFO:teuthology.orchestra.run.smithi138.stdout:{ 2024-09-14T06:29:52.968 INFO:teuthology.orchestra.run.smithi138.stdout: "size": 20, 2024-09-14T06:29:52.968 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 600, 2024-09-14T06:29:52.968 INFO:teuthology.orchestra.run.smithi138.stdout: "ops": [ 2024-09-14T06:29:52.968 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:52.968 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:19328 3.3 3:c34e5e51:::rbd_data.112d92027590.0000000000000024:f [read 1926144~1024 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:29:52.968 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:27.059768+0000", 2024-09-14T06:29:52.968 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 25.906236784000001, 2024-09-14T06:29:52.968 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046369147999999999, 2024-09-14T06:29:52.968 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:29:52.968 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:29:52.968 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:29:52.969 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:29:52.969 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:29:52.969 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:52.969 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 19328 2024-09-14T06:29:52.969 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:52.969 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:29:52.969 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:52.969 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:29:52.969 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.059768+0000", 2024-09-14T06:29:52.969 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:52.969 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:52.969 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:52.969 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:29:52.969 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.059768+0000", 2024-09-14T06:29:52.969 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:52.970 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:52.970 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:52.970 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:29:52.970 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.059874+0000", 2024-09-14T06:29:52.970 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00010551400000000001 2024-09-14T06:29:52.970 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:52.970 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:52.970 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:29:52.970 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.059877+0000", 2024-09-14T06:29:52.970 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.8279999999999999e-06 2024-09-14T06:29:52.970 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:52.970 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:52.970 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:29:52.970 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.059884+0000", 2024-09-14T06:29:52.971 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.6389999999999992e-06 2024-09-14T06:29:52.971 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:52.971 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:52.971 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:29:52.971 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.059890+0000", 2024-09-14T06:29:52.971 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.49e-06 2024-09-14T06:29:52.971 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:52.971 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:52.971 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:29:52.971 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.105804+0000", 2024-09-14T06:29:52.971 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.045914242000000001 2024-09-14T06:29:52.971 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:52.971 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:52.971 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:29:52.972 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.105886+0000", 2024-09-14T06:29:52.972 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.2305999999999994e-05 2024-09-14T06:29:52.972 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:52.972 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:52.972 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:29:52.972 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.106138+0000", 2024-09-14T06:29:52.972 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.000251129 2024-09-14T06:29:52.972 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:52.972 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:29:52.972 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:52.972 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:52.972 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:52.972 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:19329 3.3 3:c34e5e51:::rbd_data.112d92027590.0000000000000024:f [read 1936384~1024 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:29:52.973 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:27.059899+0000", 2024-09-14T06:29:52.973 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 25.906106261000001, 2024-09-14T06:29:52.973 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046572122000000001, 2024-09-14T06:29:52.973 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:29:52.973 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:29:52.973 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:29:52.973 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:29:52.973 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:29:52.973 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:52.973 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 19329 2024-09-14T06:29:52.973 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:52.973 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:29:52.973 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:52.973 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:29:52.974 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.059899+0000", 2024-09-14T06:29:52.974 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:52.974 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:52.974 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:52.974 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:29:52.974 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.059899+0000", 2024-09-14T06:29:52.974 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:52.974 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:52.974 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:52.974 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:29:52.974 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062501+0000", 2024-09-14T06:29:52.974 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.0026016419999999999 2024-09-14T06:29:52.974 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:52.974 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:52.974 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:29:52.975 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062514+0000", 2024-09-14T06:29:52.975 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.2947e-05 2024-09-14T06:29:52.975 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:52.975 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:52.975 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:29:52.975 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062523+0000", 2024-09-14T06:29:52.975 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.5519999999999993e-06 2024-09-14T06:29:52.975 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:52.975 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:52.975 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:29:52.975 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062534+0000", 2024-09-14T06:29:52.975 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.1416e-05 2024-09-14T06:29:52.975 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:52.975 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:52.975 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:29:52.975 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.106179+0000", 2024-09-14T06:29:52.976 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.043644049999999997 2024-09-14T06:29:52.976 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:52.976 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:52.976 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:29:52.976 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.106262+0000", 2024-09-14T06:29:52.976 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.3178999999999998e-05 2024-09-14T06:29:52.976 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:52.976 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:52.976 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:29:52.976 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.106471+0000", 2024-09-14T06:29:52.976 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00020933599999999999 2024-09-14T06:29:52.976 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:52.976 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:29:52.976 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:52.976 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:52.977 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:52.977 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:19351 3.3 3:c34e5e51:::rbd_data.112d92027590.0000000000000024:f [read 1159168~1024 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:29:52.977 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:27.062766+0000", 2024-09-14T06:29:52.977 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 25.903239035999999, 2024-09-14T06:29:52.977 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046099454999999998, 2024-09-14T06:29:52.977 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:29:52.977 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:29:52.977 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:29:52.977 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:29:52.977 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:29:52.977 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:52.977 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 19351 2024-09-14T06:29:52.977 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:52.977 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:29:52.977 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:52.978 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:29:52.978 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062766+0000", 2024-09-14T06:29:52.978 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:52.978 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:52.978 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:52.978 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:29:52.978 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062766+0000", 2024-09-14T06:29:52.978 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:52.978 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:52.978 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:52.978 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:29:52.978 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062766+0000", 2024-09-14T06:29:52.978 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.6e-07 2024-09-14T06:29:52.978 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:52.978 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:52.979 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:29:52.979 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062767+0000", 2024-09-14T06:29:52.979 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.0070000000000001e-06 2024-09-14T06:29:52.979 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:52.979 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:52.979 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:29:52.979 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062785+0000", 2024-09-14T06:29:52.979 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.7884999999999999e-05 2024-09-14T06:29:52.979 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:52.979 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:52.979 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:29:52.979 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062790+0000", 2024-09-14T06:29:52.979 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.0900000000000004e-06 2024-09-14T06:29:52.979 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:52.979 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:52.980 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:29:52.980 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.108735+0000", 2024-09-14T06:29:52.980 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.045944411999999997 2024-09-14T06:29:52.980 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:52.980 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:52.980 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:29:52.980 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.108809+0000", 2024-09-14T06:29:52.980 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.4157000000000002e-05 2024-09-14T06:29:52.980 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:52.980 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:52.980 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:29:52.980 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.108866+0000", 2024-09-14T06:29:52.980 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.6644000000000002e-05 2024-09-14T06:29:52.980 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:52.981 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:29:52.981 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:52.981 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:52.981 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:52.981 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:19352 3.3 3:c34e5e51:::rbd_data.112d92027590.0000000000000024:f [read 1169408~1024 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:29:52.981 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:27.062798+0000", 2024-09-14T06:29:52.981 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 25.903207625, 2024-09-14T06:29:52.981 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046255900000000003, 2024-09-14T06:29:52.981 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:29:52.981 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:29:52.981 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:29:52.981 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:29:52.981 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:29:52.982 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:52.982 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 19352 2024-09-14T06:29:52.982 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:52.982 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:29:52.982 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:52.982 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:29:52.982 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062798+0000", 2024-09-14T06:29:52.982 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:52.982 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:52.982 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:52.982 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:29:52.982 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062798+0000", 2024-09-14T06:29:52.982 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:52.982 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:52.982 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:52.982 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:29:52.983 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062798+0000", 2024-09-14T06:29:52.983 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.37e-07 2024-09-14T06:29:52.983 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:52.983 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:52.983 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:29:52.983 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062800+0000", 2024-09-14T06:29:52.983 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.0269999999999998e-06 2024-09-14T06:29:52.983 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:52.983 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:52.983 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:29:52.983 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062806+0000", 2024-09-14T06:29:52.983 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.6690000000000003e-06 2024-09-14T06:29:52.983 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:52.983 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:52.983 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:29:52.983 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062810+0000", 2024-09-14T06:29:52.984 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.3449999999999999e-06 2024-09-14T06:29:52.984 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:52.984 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:52.984 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:29:52.984 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.108892+0000", 2024-09-14T06:29:52.984 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046081715000000002 2024-09-14T06:29:52.984 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:52.984 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:52.984 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:29:52.984 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.108940+0000", 2024-09-14T06:29:52.984 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.7815999999999998e-05 2024-09-14T06:29:52.984 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:52.984 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:52.984 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:29:52.985 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.109053+0000", 2024-09-14T06:29:52.985 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.000113791 2024-09-14T06:29:52.985 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:52.985 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:29:52.985 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:52.985 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:52.985 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:52.985 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:19353 3.3 3:c34e5e51:::rbd_data.112d92027590.0000000000000024:f [read 1179648~1024 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:29:52.985 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:27.062816+0000", 2024-09-14T06:29:52.985 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 25.903189105999999, 2024-09-14T06:29:52.985 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046481256999999998, 2024-09-14T06:29:52.985 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:29:52.985 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:29:52.986 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:29:52.986 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:29:52.986 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:29:52.986 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:52.986 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 19353 2024-09-14T06:29:52.986 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:52.986 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:29:52.986 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:52.986 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:29:52.986 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062816+0000", 2024-09-14T06:29:52.986 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:52.986 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:52.986 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:52.986 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:29:52.987 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062816+0000", 2024-09-14T06:29:52.987 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:52.987 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:52.987 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:52.987 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:29:52.987 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062816+0000", 2024-09-14T06:29:52.987 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.2899999999999999e-07 2024-09-14T06:29:52.987 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:52.987 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:52.987 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:29:52.987 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062818+0000", 2024-09-14T06:29:52.987 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.699e-06 2024-09-14T06:29:52.987 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:52.987 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:52.987 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:29:52.988 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062825+0000", 2024-09-14T06:29:52.988 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.878e-06 2024-09-14T06:29:52.988 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:52.988 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:52.988 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:29:52.988 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062830+0000", 2024-09-14T06:29:52.988 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.5410000000000002e-06 2024-09-14T06:29:52.988 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:52.988 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:52.988 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:29:52.988 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.109096+0000", 2024-09-14T06:29:52.988 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046266517 2024-09-14T06:29:52.988 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:52.988 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:52.988 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:29:52.988 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.109190+0000", 2024-09-14T06:29:52.989 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.4000999999999997e-05 2024-09-14T06:29:52.989 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:52.989 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:52.989 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:29:52.989 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.109297+0000", 2024-09-14T06:29:52.989 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.000107292 2024-09-14T06:29:52.989 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:52.989 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:29:52.989 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:52.989 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:52.989 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:52.989 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:19354 3.3 3:c34e5e51:::rbd_data.112d92027590.0000000000000024:f [read 1189888~1024 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:29:52.989 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:27.062842+0000", 2024-09-14T06:29:52.989 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 25.903163651, 2024-09-14T06:29:52.989 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046594257, 2024-09-14T06:29:52.990 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:29:52.990 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:29:52.990 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:29:52.990 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:29:52.990 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:29:52.990 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:52.990 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 19354 2024-09-14T06:29:52.990 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:52.990 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:29:52.990 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:52.990 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:29:52.990 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062842+0000", 2024-09-14T06:29:52.990 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:52.990 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:52.991 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:52.991 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:29:52.991 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062842+0000", 2024-09-14T06:29:52.991 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:52.991 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:52.991 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:52.991 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:29:52.991 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062842+0000", 2024-09-14T06:29:52.991 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.7399999999999999e-07 2024-09-14T06:29:52.991 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:52.991 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:52.991 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:29:52.991 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062859+0000", 2024-09-14T06:29:52.991 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.7010000000000001e-05 2024-09-14T06:29:52.991 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:52.992 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:52.992 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:29:52.992 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062866+0000", 2024-09-14T06:29:52.992 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.9990000000000002e-06 2024-09-14T06:29:52.992 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:52.992 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:52.992 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:29:52.992 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062873+0000", 2024-09-14T06:29:52.992 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.8569999999999997e-06 2024-09-14T06:29:52.992 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:52.992 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:52.992 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:29:52.992 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.109335+0000", 2024-09-14T06:29:52.992 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046461943999999998 2024-09-14T06:29:52.992 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:52.993 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:52.993 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:29:52.993 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.109377+0000", 2024-09-14T06:29:52.993 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.2558999999999997e-05 2024-09-14T06:29:52.993 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:52.993 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:52.993 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:29:52.993 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.109436+0000", 2024-09-14T06:29:52.993 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.8514000000000003e-05 2024-09-14T06:29:52.993 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:52.993 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:29:52.993 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:52.993 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:52.993 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:52.993 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:19355 3.3 3:c34e5e51:::rbd_data.112d92027590.0000000000000024:f [read 1200128~1024 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:29:52.994 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:27.062884+0000", 2024-09-14T06:29:52.994 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 25.903120978, 2024-09-14T06:29:52.994 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046681650999999998, 2024-09-14T06:29:52.994 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:29:52.994 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:29:52.994 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:29:52.994 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:29:52.994 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:29:52.994 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:52.994 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 19355 2024-09-14T06:29:52.994 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:52.994 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:29:52.994 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:52.994 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:29:52.994 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062884+0000", 2024-09-14T06:29:52.994 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:52.995 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:52.995 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:52.995 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:29:52.995 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062884+0000", 2024-09-14T06:29:52.995 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:52.995 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:52.995 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:52.995 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:29:52.995 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062885+0000", 2024-09-14T06:29:52.995 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.8000000000000001e-07 2024-09-14T06:29:52.995 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:52.995 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:52.995 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:29:52.995 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062886+0000", 2024-09-14T06:29:52.995 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.8920000000000001e-06 2024-09-14T06:29:52.996 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:52.996 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:52.996 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:29:52.996 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062892+0000", 2024-09-14T06:29:52.996 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.84e-06 2024-09-14T06:29:52.996 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:52.996 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:52.996 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:29:52.996 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062899+0000", 2024-09-14T06:29:52.996 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.5339999999999996e-06 2024-09-14T06:29:52.996 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:52.996 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:52.996 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:29:52.996 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.109466+0000", 2024-09-14T06:29:52.996 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046566919999999998 2024-09-14T06:29:52.996 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:52.997 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:52.997 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:29:52.997 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.109510+0000", 2024-09-14T06:29:52.997 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.4153000000000002e-05 2024-09-14T06:29:52.997 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:52.997 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:52.997 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:29:52.997 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.109566+0000", 2024-09-14T06:29:52.997 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.5931999999999999e-05 2024-09-14T06:29:52.997 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:52.997 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:29:52.997 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:52.997 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:52.997 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:52.997 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:19356 3.3 3:c34e5e51:::rbd_data.112d92027590.0000000000000024:f [read 1210368~1024 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:29:52.998 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:27.062908+0000", 2024-09-14T06:29:52.998 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 25.903097541000001, 2024-09-14T06:29:52.998 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046807498000000003, 2024-09-14T06:29:52.998 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:29:52.998 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:29:52.998 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:29:52.998 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:29:52.998 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:29:52.998 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:52.998 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 19356 2024-09-14T06:29:52.998 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:52.998 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:29:52.998 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:52.998 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:29:52.998 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062908+0000", 2024-09-14T06:29:52.999 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:52.999 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:52.999 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:52.999 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:29:52.999 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062908+0000", 2024-09-14T06:29:52.999 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:52.999 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:52.999 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:52.999 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:29:52.999 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062908+0000", 2024-09-14T06:29:52.999 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.5699999999999998e-07 2024-09-14T06:29:52.999 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:52.999 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:52.999 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:29:52.999 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062910+0000", 2024-09-14T06:29:52.999 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.7710000000000001e-06 2024-09-14T06:29:53.000 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:53.000 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:53.000 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:29:53.000 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062920+0000", 2024-09-14T06:29:53.000 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.0654e-05 2024-09-14T06:29:53.000 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:53.000 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:53.000 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:29:53.000 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062931+0000", 2024-09-14T06:29:53.000 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.075e-05 2024-09-14T06:29:53.000 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:53.000 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:53.000 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:29:53.000 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.109595+0000", 2024-09-14T06:29:53.001 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046663713000000002 2024-09-14T06:29:53.001 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:53.001 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:53.001 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:29:53.001 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.109632+0000", 2024-09-14T06:29:53.001 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.7505000000000001e-05 2024-09-14T06:29:53.001 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:53.001 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:53.001 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:29:53.001 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.109715+0000", 2024-09-14T06:29:53.001 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.2748000000000001e-05 2024-09-14T06:29:53.001 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:53.001 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:29:53.001 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:53.002 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:53.002 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:53.002 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:19357 3.3 3:c34e5e51:::rbd_data.112d92027590.0000000000000024:f [read 1220608~1024 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:29:53.002 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:27.062939+0000", 2024-09-14T06:29:53.002 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 25.903066093, 2024-09-14T06:29:53.002 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046920508, 2024-09-14T06:29:53.002 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:29:53.002 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:29:53.002 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:29:53.002 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:29:53.002 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:29:53.002 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:53.002 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 19357 2024-09-14T06:29:53.002 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:53.002 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:29:53.003 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:53.003 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:29:53.003 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062939+0000", 2024-09-14T06:29:53.003 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:53.003 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:53.003 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:53.003 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:29:53.003 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062939+0000", 2024-09-14T06:29:53.003 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:53.003 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:53.003 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:53.003 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:29:53.003 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062940+0000", 2024-09-14T06:29:53.003 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.32e-07 2024-09-14T06:29:53.003 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:53.003 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:53.004 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:29:53.004 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062942+0000", 2024-09-14T06:29:53.004 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.1919999999999999e-06 2024-09-14T06:29:53.004 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:53.004 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:53.004 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:29:53.004 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062948+0000", 2024-09-14T06:29:53.004 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.1149999999999999e-06 2024-09-14T06:29:53.004 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:53.004 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:53.004 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:29:53.004 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062960+0000", 2024-09-14T06:29:53.004 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.1895e-05 2024-09-14T06:29:53.004 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:53.005 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:53.005 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:29:53.005 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.109704+0000", 2024-09-14T06:29:53.005 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046744584999999998 2024-09-14T06:29:53.005 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:53.005 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:53.005 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:29:53.005 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.109777+0000", 2024-09-14T06:29:53.005 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.2282999999999999e-05 2024-09-14T06:29:53.005 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:53.005 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:53.005 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:29:53.005 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.109860+0000", 2024-09-14T06:29:53.005 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.3005999999999998e-05 2024-09-14T06:29:53.005 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:53.005 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:29:53.006 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:53.006 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:53.006 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:53.006 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:19358 3.3 3:c34e5e51:::rbd_data.112d92027590.0000000000000024:f [read 1230848~1024 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:29:53.006 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:27.062967+0000", 2024-09-14T06:29:53.006 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 25.903038463000001, 2024-09-14T06:29:53.006 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047144471, 2024-09-14T06:29:53.006 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:29:53.006 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:29:53.006 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:29:53.006 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:29:53.006 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:29:53.006 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:53.006 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 19358 2024-09-14T06:29:53.006 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:53.007 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:29:53.007 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:53.007 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:29:53.007 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062967+0000", 2024-09-14T06:29:53.007 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:53.007 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:53.007 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:53.007 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:29:53.007 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062967+0000", 2024-09-14T06:29:53.007 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:53.007 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:53.007 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:53.007 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:29:53.007 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062967+0000", 2024-09-14T06:29:53.007 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.89e-07 2024-09-14T06:29:53.008 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:53.008 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:53.008 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:29:53.008 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062969+0000", 2024-09-14T06:29:53.008 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.9929999999999998e-06 2024-09-14T06:29:53.008 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:53.008 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:53.008 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:29:53.008 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062975+0000", 2024-09-14T06:29:53.008 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.9030000000000001e-06 2024-09-14T06:29:53.008 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:53.008 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:53.008 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:29:53.008 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062979+0000", 2024-09-14T06:29:53.008 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.2250000000000002e-06 2024-09-14T06:29:53.009 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:53.009 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:53.009 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:29:53.009 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.109934+0000", 2024-09-14T06:29:53.009 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046954410000000002 2024-09-14T06:29:53.009 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:53.009 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:53.009 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:29:53.009 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.110007+0000", 2024-09-14T06:29:53.009 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.3829000000000001e-05 2024-09-14T06:29:53.009 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:53.009 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:53.009 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:29:53.009 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.110111+0000", 2024-09-14T06:29:53.010 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00010372199999999999 2024-09-14T06:29:53.010 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:53.010 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:29:53.010 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:53.010 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:53.010 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:53.010 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:19359 3.3 3:c34e5e51:::rbd_data.112d92027590.0000000000000024:f [read 1241088~1024 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:29:53.010 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:27.062987+0000", 2024-09-14T06:29:53.010 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 25.903018372999998, 2024-09-14T06:29:53.010 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047297182, 2024-09-14T06:29:53.010 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:29:53.010 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:29:53.010 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:29:53.010 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:29:53.011 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:29:53.011 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:53.011 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 19359 2024-09-14T06:29:53.011 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:53.011 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:29:53.011 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:53.011 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:29:53.011 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062987+0000", 2024-09-14T06:29:53.011 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:53.011 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:53.011 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:53.011 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:29:53.011 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062987+0000", 2024-09-14T06:29:53.011 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:53.012 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:53.012 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:53.012 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:29:53.012 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062987+0000", 2024-09-14T06:29:53.012 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.2500000000000001e-07 2024-09-14T06:29:53.012 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:53.012 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:53.012 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:29:53.012 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062989+0000", 2024-09-14T06:29:53.012 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.6080000000000001e-06 2024-09-14T06:29:53.012 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:53.012 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:53.012 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:29:53.012 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062994+0000", 2024-09-14T06:29:53.013 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.5520000000000004e-06 2024-09-14T06:29:53.013 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:53.013 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:53.013 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:29:53.013 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062999+0000", 2024-09-14T06:29:53.013 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.3930000000000001e-06 2024-09-14T06:29:53.013 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:53.013 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:53.013 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:29:53.013 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.110142+0000", 2024-09-14T06:29:53.013 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047143154 2024-09-14T06:29:53.013 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:53.013 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:53.013 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:29:53.013 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.110219+0000", 2024-09-14T06:29:53.014 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.6737999999999996e-05 2024-09-14T06:29:53.014 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:53.014 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:53.014 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:29:53.014 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.110284+0000", 2024-09-14T06:29:53.014 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.5412000000000003e-05 2024-09-14T06:29:53.014 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:53.014 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:29:53.014 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:53.014 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:53.014 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:53.014 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:19360 3.3 3:c34e5e51:::rbd_data.112d92027590.0000000000000024:f [read 1251328~1024 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:29:53.014 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:27.063017+0000", 2024-09-14T06:29:53.014 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 25.902987912, 2024-09-14T06:29:53.014 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047453823999999999, 2024-09-14T06:29:53.015 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:29:53.015 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:29:53.015 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:29:53.015 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:29:53.015 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:29:53.015 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:53.015 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 19360 2024-09-14T06:29:53.015 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:53.015 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:29:53.015 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:53.015 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:29:53.015 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.063017+0000", 2024-09-14T06:29:53.015 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:53.015 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:53.015 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:53.016 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:29:53.016 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.063017+0000", 2024-09-14T06:29:53.016 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:53.016 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:53.016 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:53.016 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:29:53.016 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.064977+0000", 2024-09-14T06:29:53.016 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.0019597080000000001 2024-09-14T06:29:53.016 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:53.016 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:53.016 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:29:53.016 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.064987+0000", 2024-09-14T06:29:53.016 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.9620000000000005e-06 2024-09-14T06:29:53.016 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:53.016 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:53.017 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:29:53.017 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065000+0000", 2024-09-14T06:29:53.017 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.3173000000000001e-05 2024-09-14T06:29:53.017 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:53.017 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:53.017 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:29:53.017 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065005+0000", 2024-09-14T06:29:53.017 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.9960000000000001e-06 2024-09-14T06:29:53.017 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:53.017 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:53.017 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:29:53.017 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.110314+0000", 2024-09-14T06:29:53.017 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.04530899 2024-09-14T06:29:53.017 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:53.017 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:53.018 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:29:53.018 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.110398+0000", 2024-09-14T06:29:53.018 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.3539000000000001e-05 2024-09-14T06:29:53.018 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:53.018 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:53.018 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:29:53.018 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.110471+0000", 2024-09-14T06:29:53.018 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.3455999999999996e-05 2024-09-14T06:29:53.018 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:53.018 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:29:53.018 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:53.018 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:53.018 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:53.018 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:19365 3.3 3:c34e5e51:::rbd_data.112d92027590.0000000000000024:f [read 1302528~1024 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:29:53.018 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:27.065058+0000", 2024-09-14T06:29:53.019 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 25.900947242000001, 2024-09-14T06:29:53.019 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046105526000000001, 2024-09-14T06:29:53.019 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:29:53.019 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:29:53.019 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:29:53.019 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:29:53.019 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:29:53.019 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:53.019 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 19365 2024-09-14T06:29:53.019 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:53.019 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:29:53.019 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:53.019 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:29:53.019 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065058+0000", 2024-09-14T06:29:53.019 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:53.020 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:53.020 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:53.020 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:29:53.020 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065058+0000", 2024-09-14T06:29:53.020 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:53.020 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:53.020 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:53.020 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:29:53.020 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065058+0000", 2024-09-14T06:29:53.020 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.1899999999999999e-07 2024-09-14T06:29:53.020 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:53.020 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:53.020 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:29:53.020 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065059+0000", 2024-09-14T06:29:53.021 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.1299999999999998e-07 2024-09-14T06:29:53.021 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:53.021 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:53.021 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:29:53.021 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065063+0000", 2024-09-14T06:29:53.021 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.1099999999999996e-06 2024-09-14T06:29:53.021 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:53.021 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:53.021 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:29:53.021 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065070+0000", 2024-09-14T06:29:53.021 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.973e-06 2024-09-14T06:29:53.021 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:53.021 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:53.022 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:29:53.022 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.111014+0000", 2024-09-14T06:29:53.022 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.045943373000000003 2024-09-14T06:29:53.022 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:53.022 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:53.022 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:29:53.022 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.111073+0000", 2024-09-14T06:29:53.022 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.8987999999999998e-05 2024-09-14T06:29:53.022 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:53.022 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:53.022 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:29:53.022 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.111163+0000", 2024-09-14T06:29:53.022 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.0950000000000002e-05 2024-09-14T06:29:53.022 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:53.022 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:29:53.023 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:53.023 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:53.023 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:53.023 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:19366 3.3 3:c34e5e51:::rbd_data.112d92027590.0000000000000024:f [read 1312768~1024 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:29:53.023 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:27.065074+0000", 2024-09-14T06:29:53.023 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 25.900931392, 2024-09-14T06:29:53.023 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046289858000000003, 2024-09-14T06:29:53.023 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:29:53.023 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:29:53.023 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:29:53.023 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:29:53.023 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:29:53.023 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:53.023 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 19366 2024-09-14T06:29:53.023 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:53.023 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:29:53.024 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:53.024 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:29:53.024 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065074+0000", 2024-09-14T06:29:53.024 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:53.024 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:53.024 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:53.024 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:29:53.024 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065074+0000", 2024-09-14T06:29:53.024 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:53.024 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:53.024 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:53.024 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:29:53.024 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065074+0000", 2024-09-14T06:29:53.024 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.3900000000000001e-07 2024-09-14T06:29:53.024 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:53.025 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:53.025 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:29:53.025 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065078+0000", 2024-09-14T06:29:53.025 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.3229999999999999e-06 2024-09-14T06:29:53.025 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:53.025 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:53.025 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:29:53.025 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065082+0000", 2024-09-14T06:29:53.025 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.8299999999999998e-06 2024-09-14T06:29:53.025 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:53.025 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:53.025 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:29:53.025 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065085+0000", 2024-09-14T06:29:53.025 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.8660000000000002e-06 2024-09-14T06:29:53.025 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:53.025 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:53.026 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:29:53.026 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.111270+0000", 2024-09-14T06:29:53.026 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046185035999999999 2024-09-14T06:29:53.026 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:53.026 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:53.026 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:29:53.026 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.111311+0000", 2024-09-14T06:29:53.026 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.0935e-05 2024-09-14T06:29:53.026 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:53.026 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:53.026 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:29:53.026 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.111364+0000", 2024-09-14T06:29:53.026 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.2629e-05 2024-09-14T06:29:53.026 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:53.027 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:29:53.027 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:53.027 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:53.027 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:53.027 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:19367 3.3 3:c34e5e51:::rbd_data.112d92027590.0000000000000024:f [read 1323008~1024 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:29:53.027 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:27.065115+0000", 2024-09-14T06:29:53.027 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 25.900889831000001, 2024-09-14T06:29:53.027 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046407003000000002, 2024-09-14T06:29:53.027 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:29:53.027 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:29:53.027 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:29:53.027 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:29:53.027 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:29:53.027 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:53.028 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 19367 2024-09-14T06:29:53.028 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:53.028 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:29:53.028 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:53.028 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:29:53.028 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065115+0000", 2024-09-14T06:29:53.028 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:53.028 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:53.028 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:53.028 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:29:53.028 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065115+0000", 2024-09-14T06:29:53.028 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:53.028 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:53.028 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:53.029 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:29:53.029 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065116+0000", 2024-09-14T06:29:53.029 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.4999999999999998e-07 2024-09-14T06:29:53.029 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:53.029 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:53.029 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:29:53.029 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065117+0000", 2024-09-14T06:29:53.029 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.189e-06 2024-09-14T06:29:53.029 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:53.029 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:53.029 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:29:53.029 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065122+0000", 2024-09-14T06:29:53.029 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.3709999999999999e-06 2024-09-14T06:29:53.029 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:53.029 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:53.030 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:29:53.030 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065125+0000", 2024-09-14T06:29:53.030 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.8940000000000002e-06 2024-09-14T06:29:53.030 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:53.030 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:53.030 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:29:53.030 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.111402+0000", 2024-09-14T06:29:53.030 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046276645999999998 2024-09-14T06:29:53.030 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:53.030 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:53.030 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:29:53.030 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.111445+0000", 2024-09-14T06:29:53.030 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.3288000000000003e-05 2024-09-14T06:29:53.030 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:53.030 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:53.031 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:29:53.031 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.111522+0000", 2024-09-14T06:29:53.031 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.7164999999999997e-05 2024-09-14T06:29:53.031 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:53.031 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:29:53.031 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:53.031 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:53.031 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:53.031 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:19368 3.3 3:c34e5e51:::rbd_data.112d92027590.0000000000000024:f [read 1333248~1024 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:29:53.031 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:27.065130+0000", 2024-09-14T06:29:53.031 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 25.900875541000001, 2024-09-14T06:29:53.031 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046507275000000001, 2024-09-14T06:29:53.031 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:29:53.031 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:29:53.032 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:29:53.032 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:29:53.032 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:29:53.032 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:53.032 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 19368 2024-09-14T06:29:53.032 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:53.032 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:29:53.032 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:53.032 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:29:53.032 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065130+0000", 2024-09-14T06:29:53.032 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:53.032 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:53.032 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:53.033 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:29:53.033 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065130+0000", 2024-09-14T06:29:53.033 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:53.033 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:53.033 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:53.033 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:29:53.033 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065130+0000", 2024-09-14T06:29:53.033 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.0800000000000001e-07 2024-09-14T06:29:53.033 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:53.033 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:53.033 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:29:53.033 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065131+0000", 2024-09-14T06:29:53.034 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.8700000000000004e-07 2024-09-14T06:29:53.034 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:53.034 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:53.034 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:29:53.034 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065134+0000", 2024-09-14T06:29:53.034 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.1039999999999998e-06 2024-09-14T06:29:53.034 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:53.034 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:53.034 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:29:53.034 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065136+0000", 2024-09-14T06:29:53.034 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.3949999999999999e-06 2024-09-14T06:29:53.034 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:53.034 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:53.035 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:29:53.035 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.111541+0000", 2024-09-14T06:29:53.035 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046404206000000003 2024-09-14T06:29:53.035 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:53.035 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:53.035 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:29:53.035 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.111582+0000", 2024-09-14T06:29:53.035 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.1051999999999999e-05 2024-09-14T06:29:53.035 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:53.035 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:53.035 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:29:53.035 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.111637+0000", 2024-09-14T06:29:53.035 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.5322999999999999e-05 2024-09-14T06:29:53.035 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:53.036 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:29:53.036 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:53.036 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:53.036 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:53.036 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:19369 3.3 3:c34e5e51:::rbd_data.112d92027590.0000000000000024:f [read 1343488~1024 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:29:53.036 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:27.065139+0000", 2024-09-14T06:29:53.036 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 25.900865723999999, 2024-09-14T06:29:53.036 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046682169000000003, 2024-09-14T06:29:53.036 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:29:53.036 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:29:53.036 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:29:53.036 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:29:53.036 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:29:53.036 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:53.037 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 19369 2024-09-14T06:29:53.037 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:53.037 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:29:53.037 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:53.037 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:29:53.037 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065139+0000", 2024-09-14T06:29:53.037 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:53.037 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:53.037 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:53.037 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:29:53.037 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065139+0000", 2024-09-14T06:29:53.037 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:53.037 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:53.038 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:53.038 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:29:53.038 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065140+0000", 2024-09-14T06:29:53.038 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.8299999999999998e-07 2024-09-14T06:29:53.038 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:53.038 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:53.038 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:29:53.038 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065141+0000", 2024-09-14T06:29:53.038 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.3900000000000003e-07 2024-09-14T06:29:53.038 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:53.039 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:53.039 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:29:53.039 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065144+0000", 2024-09-14T06:29:53.039 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.7409999999999998e-06 2024-09-14T06:29:53.039 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:53.039 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:53.039 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:29:53.039 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065147+0000", 2024-09-14T06:29:53.039 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.4150000000000002e-06 2024-09-14T06:29:53.039 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:53.039 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:53.039 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:29:53.039 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.111673+0000", 2024-09-14T06:29:53.039 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046526004000000003 2024-09-14T06:29:53.040 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:53.040 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:53.040 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:29:53.040 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.111749+0000", 2024-09-14T06:29:53.040 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.5741999999999994e-05 2024-09-14T06:29:53.040 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:53.040 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:53.040 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:29:53.040 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.111822+0000", 2024-09-14T06:29:53.040 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.3045000000000003e-05 2024-09-14T06:29:53.040 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:53.040 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:29:53.040 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:53.041 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:53.041 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:53.041 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:19370 3.3 3:c34e5e51:::rbd_data.112d92027590.0000000000000024:f [read 1353728~1024 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:29:53.041 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:27.065150+0000", 2024-09-14T06:29:53.041 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 25.900854995, 2024-09-14T06:29:53.041 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046897556999999999, 2024-09-14T06:29:53.041 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:29:53.041 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:29:53.041 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:29:53.041 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:29:53.041 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:29:53.041 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:53.041 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 19370 2024-09-14T06:29:53.041 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:53.041 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:29:53.041 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:53.042 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:29:53.042 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065150+0000", 2024-09-14T06:29:53.042 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:53.042 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:53.042 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:53.042 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:29:53.042 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065150+0000", 2024-09-14T06:29:53.042 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:53.042 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:53.042 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:53.042 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:29:53.042 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065150+0000", 2024-09-14T06:29:53.042 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.0200000000000001e-07 2024-09-14T06:29:53.042 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:53.042 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:53.042 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:29:53.043 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065151+0000", 2024-09-14T06:29:53.043 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.1100000000000004e-07 2024-09-14T06:29:53.043 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:53.043 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:53.043 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:29:53.043 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065154+0000", 2024-09-14T06:29:53.043 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.9639999999999999e-06 2024-09-14T06:29:53.043 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:53.043 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:53.043 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:29:53.043 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065157+0000", 2024-09-14T06:29:53.043 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.4260000000000002e-06 2024-09-14T06:29:53.043 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:53.043 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:53.043 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:29:53.044 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.111861+0000", 2024-09-14T06:29:53.044 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046704268 2024-09-14T06:29:53.044 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:53.044 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:53.044 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:29:53.044 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.111939+0000", 2024-09-14T06:29:53.044 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.8231000000000004e-05 2024-09-14T06:29:53.044 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:53.044 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:53.044 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:29:53.044 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.112048+0000", 2024-09-14T06:29:53.044 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00010855500000000001 2024-09-14T06:29:53.044 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:53.044 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:29:53.044 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:53.044 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:53.045 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:53.045 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:19371 3.3 3:c34e5e51:::rbd_data.112d92027590.0000000000000024:f [read 1363968~1024 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:29:53.045 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:27.065161+0000", 2024-09-14T06:29:53.045 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 25.900844344999999, 2024-09-14T06:29:53.045 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047021944000000003, 2024-09-14T06:29:53.045 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:29:53.045 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:29:53.045 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:29:53.045 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:29:53.045 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:29:53.045 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:53.045 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 19371 2024-09-14T06:29:53.045 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:53.045 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:29:53.045 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:53.046 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:29:53.046 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065161+0000", 2024-09-14T06:29:53.046 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:53.046 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:53.046 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:53.046 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:29:53.046 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065161+0000", 2024-09-14T06:29:53.046 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:53.046 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:53.046 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:53.046 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:29:53.046 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065161+0000", 2024-09-14T06:29:53.046 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.0200000000000001e-07 2024-09-14T06:29:53.046 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:53.046 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:53.046 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:29:53.047 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065166+0000", 2024-09-14T06:29:53.047 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.9869999999999999e-06 2024-09-14T06:29:53.047 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:53.047 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:53.047 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:29:53.047 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065169+0000", 2024-09-14T06:29:53.047 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.3799999999999998e-06 2024-09-14T06:29:53.047 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:53.047 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:53.047 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:29:53.047 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065172+0000", 2024-09-14T06:29:53.047 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.4830000000000002e-06 2024-09-14T06:29:53.047 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:53.047 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:53.047 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:29:53.048 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.112071+0000", 2024-09-14T06:29:53.048 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046899529000000002 2024-09-14T06:29:53.048 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:53.048 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:53.048 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:29:53.048 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.112128+0000", 2024-09-14T06:29:53.048 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.6341999999999997e-05 2024-09-14T06:29:53.048 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:53.048 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:53.048 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:29:53.048 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.112183+0000", 2024-09-14T06:29:53.048 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.5021000000000001e-05 2024-09-14T06:29:53.048 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:53.048 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:29:53.048 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:53.049 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:53.049 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:53.049 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:19372 3.3 3:c34e5e51:::rbd_data.112d92027590.0000000000000024:f [read 1374208~1024 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:29:53.049 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:27.065176+0000", 2024-09-14T06:29:53.049 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 25.900829195, 2024-09-14T06:29:53.049 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047188889999999997, 2024-09-14T06:29:53.049 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:29:53.049 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:29:53.049 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:29:53.049 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:29:53.049 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:29:53.049 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:53.049 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 19372 2024-09-14T06:29:53.049 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:53.050 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:29:53.050 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:53.050 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:29:53.050 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065176+0000", 2024-09-14T06:29:53.050 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:53.050 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:53.050 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:53.050 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:29:53.050 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065176+0000", 2024-09-14T06:29:53.050 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:53.051 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:53.051 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:53.051 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:29:53.051 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.071117+0000", 2024-09-14T06:29:53.051 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.0059404990000000001 2024-09-14T06:29:53.051 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:53.051 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:53.051 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:29:53.051 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.071120+0000", 2024-09-14T06:29:53.051 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.4340000000000001e-06 2024-09-14T06:29:53.051 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:53.051 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:53.051 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:29:53.052 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.071129+0000", 2024-09-14T06:29:53.052 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.8389999999999996e-06 2024-09-14T06:29:53.052 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:53.052 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:53.052 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:29:53.052 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.071134+0000", 2024-09-14T06:29:53.052 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.412e-06 2024-09-14T06:29:53.052 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:53.052 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:53.052 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:29:53.052 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.112221+0000", 2024-09-14T06:29:53.052 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.041086473999999998 2024-09-14T06:29:53.052 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:53.052 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:53.052 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:29:53.052 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.112299+0000", 2024-09-14T06:29:53.053 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.8746999999999997e-05 2024-09-14T06:29:53.053 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:53.053 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:53.053 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:29:53.053 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.112365+0000", 2024-09-14T06:29:53.053 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.5485000000000001e-05 2024-09-14T06:29:53.053 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:53.053 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:29:53.053 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:53.053 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:53.053 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:29:53.053 INFO:teuthology.orchestra.run.smithi138.stdout:} 2024-09-14T06:29:53.055 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-14T06:29:53.055 INFO:tasks.ceph.osd.7.smithi138.stderr:2024-09-14T06:29:53.053+0000 7f3c6a21d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33623) UID: 0 2024-09-14T06:29:53.156 INFO:tasks.ceph.osd.2.smithi094.stderr:2024-09-14T06:29:53.155+0000 7faf3269b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33649) UID: 0 2024-09-14T06:29:53.199 INFO:teuthology.orchestra.run.smithi138.stdout:{ 2024-09-14T06:29:53.199 INFO:teuthology.orchestra.run.smithi138.stdout: "ops": [], 2024-09-14T06:29:53.199 INFO:teuthology.orchestra.run.smithi138.stdout: "num_ops": 0 2024-09-14T06:29:53.199 INFO:teuthology.orchestra.run.smithi138.stdout:} 2024-09-14T06:29:53.208 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-14T06:29:53.256 INFO:tasks.ceph.osd.6.smithi138.stderr:2024-09-14T06:29:53.255+0000 7f8cd8849640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33620) UID: 0 2024-09-14T06:29:53.311 INFO:teuthology.orchestra.run.smithi138.stdout:{ 2024-09-14T06:29:53.311 INFO:teuthology.orchestra.run.smithi138.stdout: "ops": [], 2024-09-14T06:29:53.311 INFO:teuthology.orchestra.run.smithi138.stdout: "complaint_time": 30, 2024-09-14T06:29:53.311 INFO:teuthology.orchestra.run.smithi138.stdout: "num_blocked_ops": 0 2024-09-14T06:29:53.311 INFO:teuthology.orchestra.run.smithi138.stdout:} 2024-09-14T06:29:53.320 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-14T06:29:53.361 INFO:tasks.ceph.osd.5.smithi138.stderr:2024-09-14T06:29:53.360+0000 7fe13746f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33618) UID: 0 2024-09-14T06:29:53.459 INFO:tasks.ceph.osd.5.smithi138.stderr:2024-09-14T06:29:53.457+0000 7fe13746f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33618) UID: 0 2024-09-14T06:29:53.465 INFO:teuthology.orchestra.run.smithi138.stdout:{ 2024-09-14T06:29:53.465 INFO:teuthology.orchestra.run.smithi138.stdout: "size": 20, 2024-09-14T06:29:53.465 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 600, 2024-09-14T06:29:53.465 INFO:teuthology.orchestra.run.smithi138.stdout: "ops": [ 2024-09-14T06:29:53.465 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:53.465 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:11453 3.1d 3:bcad7303:::rbd_data.112d92027590.0000000000000001:f [read 94208~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:29:53.465 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:25.654375+0000", 2024-09-14T06:29:53.465 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 27.808926883000002, 2024-09-14T06:29:53.465 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052449318000000002, 2024-09-14T06:29:53.465 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:29:53.465 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:29:53.465 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:29:53.466 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:29:53.466 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:29:53.466 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:53.466 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 11453 2024-09-14T06:29:53.466 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:53.466 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:29:53.466 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:53.466 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:29:53.466 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654375+0000", 2024-09-14T06:29:53.466 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:53.466 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:53.466 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:53.466 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:29:53.466 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654375+0000", 2024-09-14T06:29:53.467 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:53.467 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:53.467 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:53.467 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:29:53.467 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654496+0000", 2024-09-14T06:29:53.467 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00012081 2024-09-14T06:29:53.467 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:53.467 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:53.467 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:29:53.467 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654497+0000", 2024-09-14T06:29:53.467 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.8789999999999999e-06 2024-09-14T06:29:53.467 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:53.467 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:53.467 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:29:53.468 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654501+0000", 2024-09-14T06:29:53.468 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.8999999999999999e-06 2024-09-14T06:29:53.468 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:53.468 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:53.468 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:29:53.468 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654505+0000", 2024-09-14T06:29:53.468 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.2229999999999999e-06 2024-09-14T06:29:53.468 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:53.468 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:53.468 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:29:53.468 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.705917+0000", 2024-09-14T06:29:53.468 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.051412501999999999 2024-09-14T06:29:53.468 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:53.468 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:53.469 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:29:53.469 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.705999+0000", 2024-09-14T06:29:53.469 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.1551999999999994e-05 2024-09-14T06:29:53.469 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:53.469 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:53.469 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:29:53.469 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.706824+0000", 2024-09-14T06:29:53.469 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00082545199999999996 2024-09-14T06:29:53.469 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:53.469 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:29:53.469 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:53.469 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:53.469 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:53.469 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:11454 3.1d 3:bcad7303:::rbd_data.112d92027590.0000000000000001:f [read 104448~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:29:53.470 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:25.654510+0000", 2024-09-14T06:29:53.470 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 27.808792093000001, 2024-09-14T06:29:53.470 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052610713000000003, 2024-09-14T06:29:53.470 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:29:53.470 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:29:53.470 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:29:53.470 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:29:53.470 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:29:53.470 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:53.470 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 11454 2024-09-14T06:29:53.470 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:53.470 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:29:53.470 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:53.470 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:29:53.471 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654510+0000", 2024-09-14T06:29:53.471 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:53.471 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:53.471 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:53.471 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:29:53.471 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654510+0000", 2024-09-14T06:29:53.471 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:53.471 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:53.471 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:53.471 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:29:53.471 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654825+0000", 2024-09-14T06:29:53.472 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00031522400000000002 2024-09-14T06:29:53.472 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:53.472 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:53.472 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:29:53.472 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654827+0000", 2024-09-14T06:29:53.472 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.9709999999999998e-06 2024-09-14T06:29:53.472 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:53.472 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:53.472 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:29:53.472 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654832+0000", 2024-09-14T06:29:53.472 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.4099999999999999e-06 2024-09-14T06:29:53.472 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:53.472 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:53.473 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:29:53.473 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654835+0000", 2024-09-14T06:29:53.473 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.14e-06 2024-09-14T06:29:53.473 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:53.473 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:53.473 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:29:53.473 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.706851+0000", 2024-09-14T06:29:53.473 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052015761000000001 2024-09-14T06:29:53.473 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:53.473 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:53.473 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:29:53.473 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.706934+0000", 2024-09-14T06:29:53.473 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.2484999999999994e-05 2024-09-14T06:29:53.474 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:53.474 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:53.474 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:29:53.474 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.707120+0000", 2024-09-14T06:29:53.474 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00018672200000000001 2024-09-14T06:29:53.474 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:53.474 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:29:53.474 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:53.474 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:53.474 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:53.474 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:11455 3.1d 3:bcad7303:::rbd_data.112d92027590.0000000000000001:f [read 114688~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:29:53.474 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:25.654841+0000", 2024-09-14T06:29:53.475 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 27.808460763999999, 2024-09-14T06:29:53.475 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052579898, 2024-09-14T06:29:53.475 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:29:53.475 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:29:53.476 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:29:53.476 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:29:53.476 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:29:53.476 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:53.476 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 11455 2024-09-14T06:29:53.476 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:53.476 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:29:53.476 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:53.476 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:29:53.476 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654841+0000", 2024-09-14T06:29:53.476 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:53.476 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:53.476 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:53.476 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:29:53.477 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654841+0000", 2024-09-14T06:29:53.477 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:53.477 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:53.477 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:53.477 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:29:53.477 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654841+0000", 2024-09-14T06:29:53.477 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.7300000000000002e-07 2024-09-14T06:29:53.477 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:53.477 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:53.477 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:29:53.477 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654845+0000", 2024-09-14T06:29:53.477 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.0520000000000003e-06 2024-09-14T06:29:53.477 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:53.477 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:53.478 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:29:53.478 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654849+0000", 2024-09-14T06:29:53.478 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.0459999999999999e-06 2024-09-14T06:29:53.478 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:53.478 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:53.478 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:29:53.478 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654852+0000", 2024-09-14T06:29:53.478 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.9830000000000001e-06 2024-09-14T06:29:53.478 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:53.478 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:53.478 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:29:53.478 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.707159+0000", 2024-09-14T06:29:53.478 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052306826000000001 2024-09-14T06:29:53.478 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:53.479 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:53.479 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:29:53.479 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.707242+0000", 2024-09-14T06:29:53.479 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.2595000000000005e-05 2024-09-14T06:29:53.479 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:53.479 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:53.479 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:29:53.479 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.707421+0000", 2024-09-14T06:29:53.479 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.000179123 2024-09-14T06:29:53.479 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:53.479 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:29:53.479 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:53.479 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:53.479 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:53.480 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:11456 3.1d 3:bcad7303:::rbd_data.112d92027590.0000000000000001:f [read 124928~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:29:53.480 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:25.654856+0000", 2024-09-14T06:29:53.480 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 27.808445918, 2024-09-14T06:29:53.480 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052801580000000001, 2024-09-14T06:29:53.480 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:29:53.480 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:29:53.480 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:29:53.480 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:29:53.480 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:29:53.480 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:53.480 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 11456 2024-09-14T06:29:53.480 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:53.480 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:29:53.480 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:53.481 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:29:53.481 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654856+0000", 2024-09-14T06:29:53.481 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:53.481 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:53.481 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:53.481 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:29:53.481 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654856+0000", 2024-09-14T06:29:53.481 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:53.481 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:53.481 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:53.481 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:29:53.481 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654856+0000", 2024-09-14T06:29:53.481 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.29e-07 2024-09-14T06:29:53.481 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:53.482 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:53.482 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:29:53.482 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654857+0000", 2024-09-14T06:29:53.482 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.187e-06 2024-09-14T06:29:53.482 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:53.482 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:53.482 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:29:53.482 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654861+0000", 2024-09-14T06:29:53.482 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.3850000000000002e-06 2024-09-14T06:29:53.482 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:53.482 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:53.482 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:29:53.482 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654863+0000", 2024-09-14T06:29:53.483 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.9500000000000001e-06 2024-09-14T06:29:53.483 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:53.483 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:53.483 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:29:53.483 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.707456+0000", 2024-09-14T06:29:53.483 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052592434 2024-09-14T06:29:53.483 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:53.483 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:53.483 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:29:53.483 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.707516+0000", 2024-09-14T06:29:53.483 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.0167999999999997e-05 2024-09-14T06:29:53.483 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:53.483 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:53.483 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:29:53.484 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.707657+0000", 2024-09-14T06:29:53.484 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00014122700000000001 2024-09-14T06:29:53.484 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:53.484 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:29:53.484 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:53.484 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:53.484 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:53.484 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:11457 3.1d 3:bcad7303:::rbd_data.112d92027590.0000000000000001:f [read 135168~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:29:53.484 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:25.654867+0000", 2024-09-14T06:29:53.484 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 27.808435032999999, 2024-09-14T06:29:53.484 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052921902, 2024-09-14T06:29:53.484 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:29:53.484 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:29:53.484 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:29:53.485 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:29:53.485 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:29:53.485 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:53.485 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 11457 2024-09-14T06:29:53.485 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:53.485 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:29:53.485 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:53.485 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:29:53.485 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654867+0000", 2024-09-14T06:29:53.485 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:53.485 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:53.485 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:53.485 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:29:53.485 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654867+0000", 2024-09-14T06:29:53.486 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:53.486 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:53.486 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:53.486 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:29:53.486 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654972+0000", 2024-09-14T06:29:53.486 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00010584 2024-09-14T06:29:53.486 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:53.486 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:53.486 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:29:53.486 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654975+0000", 2024-09-14T06:29:53.486 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.1169999999999998e-06 2024-09-14T06:29:53.487 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:53.487 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:53.487 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:29:53.487 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654979+0000", 2024-09-14T06:29:53.487 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.1960000000000001e-06 2024-09-14T06:29:53.487 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:53.487 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:53.487 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:29:53.487 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654982+0000", 2024-09-14T06:29:53.487 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.2990000000000001e-06 2024-09-14T06:29:53.488 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:53.488 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:53.488 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:29:53.488 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.707684+0000", 2024-09-14T06:29:53.488 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052701979000000003 2024-09-14T06:29:53.488 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:53.488 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:53.488 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:29:53.488 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.707728+0000", 2024-09-14T06:29:53.489 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.3529999999999998e-05 2024-09-14T06:29:53.489 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:53.489 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:53.489 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:29:53.489 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.707789+0000", 2024-09-14T06:29:53.489 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.0940999999999998e-05 2024-09-14T06:29:53.489 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:53.489 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:29:53.489 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:53.489 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:53.489 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:53.489 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:11458 3.1d 3:bcad7303:::rbd_data.112d92027590.0000000000000001:f [read 145408~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:29:53.489 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:25.654987+0000", 2024-09-14T06:29:53.489 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 27.808314634999999, 2024-09-14T06:29:53.490 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.05291655, 2024-09-14T06:29:53.490 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:29:53.490 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:29:53.490 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:29:53.490 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:29:53.490 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:29:53.490 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:53.490 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 11458 2024-09-14T06:29:53.490 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:53.490 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:29:53.490 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:53.490 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:29:53.490 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654987+0000", 2024-09-14T06:29:53.490 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:53.491 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:53.491 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:53.491 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:29:53.491 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654987+0000", 2024-09-14T06:29:53.491 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:53.491 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:53.491 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:53.491 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:29:53.491 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655077+0000", 2024-09-14T06:29:53.491 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.0253000000000005e-05 2024-09-14T06:29:53.491 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:53.491 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:53.491 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:29:53.491 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655079+0000", 2024-09-14T06:29:53.492 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.9989999999999998e-06 2024-09-14T06:29:53.492 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:53.492 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:53.492 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:29:53.492 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655083+0000", 2024-09-14T06:29:53.492 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.9280000000000003e-06 2024-09-14T06:29:53.492 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:53.492 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:53.492 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:29:53.492 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655086+0000", 2024-09-14T06:29:53.492 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.056e-06 2024-09-14T06:29:53.492 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:53.492 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:53.492 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:29:53.493 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.707805+0000", 2024-09-14T06:29:53.493 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052718627999999997 2024-09-14T06:29:53.493 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:53.493 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:53.493 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:29:53.493 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.707848+0000", 2024-09-14T06:29:53.493 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.2728999999999997e-05 2024-09-14T06:29:53.493 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:53.493 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:53.493 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:29:53.493 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.707904+0000", 2024-09-14T06:29:53.493 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.5957e-05 2024-09-14T06:29:53.493 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:53.493 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:29:53.494 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:53.494 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:53.494 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:53.494 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:11459 3.1d 3:bcad7303:::rbd_data.112d92027590.0000000000000001:f [read 155648~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:29:53.494 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:25.655091+0000", 2024-09-14T06:29:53.494 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 27.808210627000001, 2024-09-14T06:29:53.494 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052944242000000002, 2024-09-14T06:29:53.494 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:29:53.494 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:29:53.494 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:29:53.494 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:29:53.494 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:29:53.494 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:53.495 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 11459 2024-09-14T06:29:53.495 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:53.495 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:29:53.495 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:53.495 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:29:53.495 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655091+0000", 2024-09-14T06:29:53.495 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:53.495 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:53.495 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:53.495 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:29:53.495 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655091+0000", 2024-09-14T06:29:53.495 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:53.495 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:53.495 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:53.496 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:29:53.496 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655237+0000", 2024-09-14T06:29:53.496 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00014570100000000001 2024-09-14T06:29:53.496 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:53.496 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:53.496 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:29:53.496 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655239+0000", 2024-09-14T06:29:53.496 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.8670000000000001e-06 2024-09-14T06:29:53.496 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:53.496 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:53.496 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:29:53.496 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655243+0000", 2024-09-14T06:29:53.496 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.9829999999999998e-06 2024-09-14T06:29:53.496 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:53.497 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:53.497 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:29:53.497 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655246+0000", 2024-09-14T06:29:53.497 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.1540000000000002e-06 2024-09-14T06:29:53.497 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:53.497 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:53.497 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:29:53.497 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.707934+0000", 2024-09-14T06:29:53.497 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052688286000000001 2024-09-14T06:29:53.497 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:53.497 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:53.497 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:29:53.497 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.707977+0000", 2024-09-14T06:29:53.497 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.3291999999999998e-05 2024-09-14T06:29:53.498 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:53.498 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:53.498 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:29:53.498 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.708035+0000", 2024-09-14T06:29:53.498 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.7958999999999999e-05 2024-09-14T06:29:53.498 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:53.498 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:29:53.498 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:53.498 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:53.498 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:53.498 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:11460 3.1d 3:bcad7303:::rbd_data.112d92027590.0000000000000001:f [read 165888~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:29:53.498 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:25.655250+0000", 2024-09-14T06:29:53.498 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 27.808051330000001, 2024-09-14T06:29:53.498 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052910698999999999, 2024-09-14T06:29:53.498 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:29:53.499 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:29:53.499 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:29:53.499 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:29:53.499 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:29:53.499 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:53.499 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 11460 2024-09-14T06:29:53.499 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:53.499 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:29:53.499 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:53.499 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:29:53.499 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655250+0000", 2024-09-14T06:29:53.499 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:53.499 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:53.499 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:53.500 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:29:53.500 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655250+0000", 2024-09-14T06:29:53.500 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:53.500 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:53.500 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:53.500 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:29:53.500 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655356+0000", 2024-09-14T06:29:53.500 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00010535900000000001 2024-09-14T06:29:53.500 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:53.500 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:53.500 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:29:53.500 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655357+0000", 2024-09-14T06:29:53.500 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.8080000000000001e-06 2024-09-14T06:29:53.500 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:53.501 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:53.501 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:29:53.501 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655361+0000", 2024-09-14T06:29:53.501 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.861e-06 2024-09-14T06:29:53.501 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:53.501 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:53.501 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:29:53.501 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655364+0000", 2024-09-14T06:29:53.501 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.1089999999999998e-06 2024-09-14T06:29:53.501 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:53.501 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:53.501 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:29:53.501 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.708065+0000", 2024-09-14T06:29:53.501 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052700595000000003 2024-09-14T06:29:53.501 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:53.502 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:53.502 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:29:53.502 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.708108+0000", 2024-09-14T06:29:53.502 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.2484999999999997e-05 2024-09-14T06:29:53.502 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:53.502 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:53.502 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:29:53.502 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.708161+0000", 2024-09-14T06:29:53.502 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.3482000000000001e-05 2024-09-14T06:29:53.502 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:53.502 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:29:53.502 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:53.502 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:53.502 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:53.503 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:11461 3.1d 3:bcad7303:::rbd_data.112d92027590.0000000000000001:f [read 176128~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:29:53.503 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:25.655370+0000", 2024-09-14T06:29:53.503 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 27.807932048000001, 2024-09-14T06:29:53.503 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052915141999999998, 2024-09-14T06:29:53.503 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:29:53.503 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:29:53.503 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:29:53.503 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:29:53.503 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:29:53.503 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:53.503 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 11461 2024-09-14T06:29:53.503 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:53.503 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:29:53.503 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:53.503 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:29:53.504 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655370+0000", 2024-09-14T06:29:53.504 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:53.504 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:53.504 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:53.504 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:29:53.504 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655370+0000", 2024-09-14T06:29:53.504 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:53.504 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:53.504 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:53.504 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:29:53.504 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655394+0000", 2024-09-14T06:29:53.504 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.4553999999999999e-05 2024-09-14T06:29:53.504 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:53.504 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:53.505 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:29:53.505 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655396+0000", 2024-09-14T06:29:53.505 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.8950000000000001e-06 2024-09-14T06:29:53.505 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:53.505 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:53.505 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:29:53.505 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655401+0000", 2024-09-14T06:29:53.505 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.0089999999999998e-06 2024-09-14T06:29:53.505 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:53.505 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:53.505 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:29:53.505 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655404+0000", 2024-09-14T06:29:53.505 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.3400000000000002e-06 2024-09-14T06:29:53.505 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:53.506 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:53.506 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:29:53.506 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.708186+0000", 2024-09-14T06:29:53.506 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052781726000000001 2024-09-14T06:29:53.506 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:53.506 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:53.506 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:29:53.506 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.708229+0000", 2024-09-14T06:29:53.506 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.2670999999999999e-05 2024-09-14T06:29:53.506 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:53.506 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:53.506 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:29:53.506 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.708285+0000", 2024-09-14T06:29:53.506 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.5946999999999998e-05 2024-09-14T06:29:53.506 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:53.507 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:29:53.507 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:53.507 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:53.507 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:53.507 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:11462 3.1d 3:bcad7303:::rbd_data.112d92027590.0000000000000001:f [read 186368~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:29:53.507 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:25.655410+0000", 2024-09-14T06:29:53.507 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 27.807891569999999, 2024-09-14T06:29:53.507 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052995258000000003, 2024-09-14T06:29:53.507 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:29:53.507 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:29:53.507 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:29:53.507 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:29:53.507 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:29:53.507 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:53.508 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 11462 2024-09-14T06:29:53.508 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:53.508 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:29:53.508 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:53.508 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:29:53.508 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655410+0000", 2024-09-14T06:29:53.508 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:53.508 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:53.508 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:53.508 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:29:53.508 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655410+0000", 2024-09-14T06:29:53.508 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:53.508 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:53.509 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:53.509 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:29:53.509 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655625+0000", 2024-09-14T06:29:53.509 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00021442600000000001 2024-09-14T06:29:53.509 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:53.509 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:53.509 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:29:53.509 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655626+0000", 2024-09-14T06:29:53.509 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.815e-06 2024-09-14T06:29:53.509 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:53.509 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:53.509 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:29:53.509 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655631+0000", 2024-09-14T06:29:53.510 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.651e-06 2024-09-14T06:29:53.510 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:53.510 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:53.510 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:29:53.510 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655634+0000", 2024-09-14T06:29:53.510 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.2449999999999998e-06 2024-09-14T06:29:53.510 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:53.510 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:53.510 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:29:53.510 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.708312+0000", 2024-09-14T06:29:53.510 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052677412 2024-09-14T06:29:53.510 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:53.510 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:53.510 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:29:53.511 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.708352+0000", 2024-09-14T06:29:53.511 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.0729999999999998e-05 2024-09-14T06:29:53.511 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:53.511 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:53.511 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:29:53.511 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.708405+0000", 2024-09-14T06:29:53.511 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.2979000000000002e-05 2024-09-14T06:29:53.511 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:53.511 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:29:53.511 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:53.511 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:53.511 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:53.511 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:11463 3.1d 3:bcad7303:::rbd_data.112d92027590.0000000000000001:f [read 196608~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:29:53.511 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:25.655639+0000", 2024-09-14T06:29:53.512 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 27.807662424, 2024-09-14T06:29:53.512 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052888859000000003, 2024-09-14T06:29:53.512 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:29:53.512 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:29:53.512 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:29:53.512 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:29:53.512 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:29:53.512 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:53.512 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 11463 2024-09-14T06:29:53.512 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:53.512 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:29:53.512 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:53.512 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:29:53.513 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655639+0000", 2024-09-14T06:29:53.513 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:53.513 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:53.513 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:53.513 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:29:53.513 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655639+0000", 2024-09-14T06:29:53.513 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:53.513 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:53.513 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:53.513 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:29:53.513 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655640+0000", 2024-09-14T06:29:53.513 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.08e-07 2024-09-14T06:29:53.513 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:53.513 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:53.513 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:29:53.514 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655641+0000", 2024-09-14T06:29:53.514 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.156e-06 2024-09-14T06:29:53.514 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:53.514 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:53.514 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:29:53.514 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655648+0000", 2024-09-14T06:29:53.514 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.8460000000000002e-06 2024-09-14T06:29:53.514 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:53.514 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:53.514 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:29:53.514 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655651+0000", 2024-09-14T06:29:53.514 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.0079999999999998e-06 2024-09-14T06:29:53.514 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:53.514 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:53.514 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:29:53.515 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.708429+0000", 2024-09-14T06:29:53.515 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052778459 2024-09-14T06:29:53.515 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:53.515 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:53.515 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:29:53.515 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.708473+0000", 2024-09-14T06:29:53.515 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.3782999999999997e-05 2024-09-14T06:29:53.515 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:53.515 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:53.515 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:29:53.515 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.708528+0000", 2024-09-14T06:29:53.515 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.5198999999999998e-05 2024-09-14T06:29:53.515 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:53.515 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:29:53.516 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:53.516 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:53.516 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:53.516 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:11464 3.1d 3:bcad7303:::rbd_data.112d92027590.0000000000000001:f [read 206848~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:29:53.516 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:25.655654+0000", 2024-09-14T06:29:53.516 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 27.807647554999999, 2024-09-14T06:29:53.516 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052981845999999999, 2024-09-14T06:29:53.516 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:29:53.516 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:29:53.516 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:29:53.516 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:29:53.516 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:29:53.516 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:53.516 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 11464 2024-09-14T06:29:53.517 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:53.517 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:29:53.517 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:53.517 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:29:53.517 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655654+0000", 2024-09-14T06:29:53.517 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:53.517 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:53.517 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:53.517 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:29:53.517 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655654+0000", 2024-09-14T06:29:53.517 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:53.517 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:53.517 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:53.517 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:29:53.517 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655946+0000", 2024-09-14T06:29:53.518 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00029148200000000003 2024-09-14T06:29:53.518 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:53.518 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:53.518 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:29:53.518 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655948+0000", 2024-09-14T06:29:53.518 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.973e-06 2024-09-14T06:29:53.518 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:53.518 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:53.518 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:29:53.518 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655953+0000", 2024-09-14T06:29:53.518 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.1240000000000004e-06 2024-09-14T06:29:53.518 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:53.518 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:53.519 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:29:53.519 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655956+0000", 2024-09-14T06:29:53.519 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.315e-06 2024-09-14T06:29:53.519 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:53.519 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:53.519 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:29:53.519 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.708543+0000", 2024-09-14T06:29:53.519 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052587416999999997 2024-09-14T06:29:53.519 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:53.519 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:53.519 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:29:53.519 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.708584+0000", 2024-09-14T06:29:53.519 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.1004000000000002e-05 2024-09-14T06:29:53.519 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:53.520 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:53.520 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:29:53.520 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.708636+0000", 2024-09-14T06:29:53.520 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.1530999999999998e-05 2024-09-14T06:29:53.520 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:53.520 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:29:53.520 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:53.520 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:53.520 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:53.520 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:11465 3.1d 3:bcad7303:::rbd_data.112d92027590.0000000000000001:f [read 217088~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:29:53.520 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:25.655961+0000", 2024-09-14T06:29:53.520 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 27.807340403000001, 2024-09-14T06:29:53.520 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052798682999999999, 2024-09-14T06:29:53.520 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:29:53.521 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:29:53.521 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:29:53.521 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:29:53.521 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:29:53.521 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:53.521 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 11465 2024-09-14T06:29:53.521 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:53.521 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:29:53.521 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:53.521 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:29:53.521 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655961+0000", 2024-09-14T06:29:53.521 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:53.521 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:53.521 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:53.522 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:29:53.522 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655961+0000", 2024-09-14T06:29:53.522 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:53.522 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:53.522 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:53.522 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:29:53.522 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.656114+0000", 2024-09-14T06:29:53.522 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.000152745 2024-09-14T06:29:53.522 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:53.522 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:53.522 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:29:53.522 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.656116+0000", 2024-09-14T06:29:53.522 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.804e-06 2024-09-14T06:29:53.522 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:53.523 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:53.523 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:29:53.523 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.656120+0000", 2024-09-14T06:29:53.523 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.4700000000000004e-06 2024-09-14T06:29:53.523 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:53.523 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:53.523 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:29:53.523 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.656123+0000", 2024-09-14T06:29:53.523 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.1269999999999999e-06 2024-09-14T06:29:53.523 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:53.524 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:53.524 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:29:53.524 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.708660+0000", 2024-09-14T06:29:53.524 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052536919000000001 2024-09-14T06:29:53.524 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:53.524 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:53.524 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:29:53.524 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.708703+0000", 2024-09-14T06:29:53.524 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.3077000000000001e-05 2024-09-14T06:29:53.524 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:53.524 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:53.524 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:29:53.524 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.708760+0000", 2024-09-14T06:29:53.525 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.6541e-05 2024-09-14T06:29:53.525 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:53.525 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:29:53.525 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:53.525 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:53.525 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:53.525 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:11466 3.1d 3:bcad7303:::rbd_data.112d92027590.0000000000000001:f [read 227328~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:29:53.525 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:25.656129+0000", 2024-09-14T06:29:53.525 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 27.807173015, 2024-09-14T06:29:53.525 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052750987999999999, 2024-09-14T06:29:53.525 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:29:53.525 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:29:53.525 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:29:53.525 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:29:53.526 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:29:53.526 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:53.526 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 11466 2024-09-14T06:29:53.526 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:53.526 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:29:53.526 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:53.526 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:29:53.526 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.656129+0000", 2024-09-14T06:29:53.526 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:53.526 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:53.526 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:53.526 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:29:53.526 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.656129+0000", 2024-09-14T06:29:53.526 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:53.526 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:53.527 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:53.527 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:29:53.527 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.656347+0000", 2024-09-14T06:29:53.527 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.000218054 2024-09-14T06:29:53.528 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:53.528 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:53.528 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:29:53.528 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.656349+0000", 2024-09-14T06:29:53.528 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.9199999999999998e-06 2024-09-14T06:29:53.528 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:53.528 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:53.528 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:29:53.529 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.656353+0000", 2024-09-14T06:29:53.529 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.9210000000000002e-06 2024-09-14T06:29:53.529 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:53.529 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:53.529 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:29:53.529 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.656356+0000", 2024-09-14T06:29:53.529 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.7670000000000001e-06 2024-09-14T06:29:53.529 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:53.529 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:53.529 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:29:53.529 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.708787+0000", 2024-09-14T06:29:53.529 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052430840999999999 2024-09-14T06:29:53.529 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:53.529 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:53.530 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:29:53.530 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.708827+0000", 2024-09-14T06:29:53.530 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.9538e-05 2024-09-14T06:29:53.530 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:53.530 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:53.530 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:29:53.530 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.708880+0000", 2024-09-14T06:29:53.530 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.2947e-05 2024-09-14T06:29:53.530 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:53.530 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:29:53.530 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:53.530 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:53.530 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:53.530 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:11467 3.1d 3:bcad7303:::rbd_data.112d92027590.0000000000000001:f [read 237568~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:29:53.531 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:25.656361+0000", 2024-09-14T06:29:53.531 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 27.806940366999999, 2024-09-14T06:29:53.531 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052643898000000001, 2024-09-14T06:29:53.531 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:29:53.531 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:29:53.531 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:29:53.531 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:29:53.531 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:29:53.531 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:53.531 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 11467 2024-09-14T06:29:53.531 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:53.531 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:29:53.531 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:53.531 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:29:53.531 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.656361+0000", 2024-09-14T06:29:53.532 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:53.532 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:53.532 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:53.532 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:29:53.532 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.656361+0000", 2024-09-14T06:29:53.532 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:53.532 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:53.532 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:53.532 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:29:53.532 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.656855+0000", 2024-09-14T06:29:53.532 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00049372000000000001 2024-09-14T06:29:53.532 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:53.532 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:53.533 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:29:53.533 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.656859+0000", 2024-09-14T06:29:53.533 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.1300000000000003e-06 2024-09-14T06:29:53.533 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:53.533 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:53.533 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:29:53.533 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.656863+0000", 2024-09-14T06:29:53.533 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.2629999999999997e-06 2024-09-14T06:29:53.533 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:53.533 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:53.533 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:29:53.533 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.656867+0000", 2024-09-14T06:29:53.533 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.3220000000000001e-06 2024-09-14T06:29:53.533 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:53.533 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:53.534 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:29:53.534 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.708903+0000", 2024-09-14T06:29:53.534 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052035855999999998 2024-09-14T06:29:53.534 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:53.534 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:53.534 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:29:53.534 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.708949+0000", 2024-09-14T06:29:53.534 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.6276999999999998e-05 2024-09-14T06:29:53.534 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:53.534 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:53.534 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:29:53.534 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.709005+0000", 2024-09-14T06:29:53.534 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.6329999999999998e-05 2024-09-14T06:29:53.534 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:53.535 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:29:53.535 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:53.535 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:53.535 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:53.535 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:11479 3.1d 3:bcad7303:::rbd_data.112d92027590.0000000000000001:f [read 360448~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:29:53.535 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:25.658388+0000", 2024-09-14T06:29:53.535 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 27.804913534000001, 2024-09-14T06:29:53.535 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052408920999999997, 2024-09-14T06:29:53.535 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:29:53.535 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:29:53.535 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:29:53.535 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:29:53.535 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:29:53.536 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:53.536 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 11479 2024-09-14T06:29:53.536 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:53.536 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:29:53.536 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:53.536 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:29:53.536 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.658388+0000", 2024-09-14T06:29:53.536 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:53.536 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:53.536 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:53.536 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:29:53.536 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.658388+0000", 2024-09-14T06:29:53.536 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:53.537 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:53.537 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:53.537 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:29:53.537 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.658534+0000", 2024-09-14T06:29:53.537 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.000145613 2024-09-14T06:29:53.537 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:53.537 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:53.537 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:29:53.537 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.658540+0000", 2024-09-14T06:29:53.537 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.6680000000000004e-06 2024-09-14T06:29:53.537 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:53.537 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:53.537 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:29:53.537 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.658548+0000", 2024-09-14T06:29:53.538 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.7980000000000006e-06 2024-09-14T06:29:53.538 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:53.538 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:53.538 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:29:53.538 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.658554+0000", 2024-09-14T06:29:53.538 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.6869999999999999e-06 2024-09-14T06:29:53.538 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:53.538 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:53.538 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:29:53.538 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.710624+0000", 2024-09-14T06:29:53.538 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052070567999999998 2024-09-14T06:29:53.538 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:53.538 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:53.538 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:29:53.539 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.710666+0000", 2024-09-14T06:29:53.539 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.1467000000000001e-05 2024-09-14T06:29:53.539 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:53.539 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:53.539 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:29:53.539 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.710797+0000", 2024-09-14T06:29:53.539 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00013112 2024-09-14T06:29:53.539 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:53.539 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:29:53.539 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:53.539 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:53.539 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:53.539 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:11480 3.1d 3:bcad7303:::rbd_data.112d92027590.0000000000000001:f [read 370688~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:29:53.539 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:25.658563+0000", 2024-09-14T06:29:53.540 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 27.804738603000001, 2024-09-14T06:29:53.540 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052429164, 2024-09-14T06:29:53.540 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:29:53.540 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:29:53.540 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:29:53.540 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:29:53.540 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:29:53.540 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:53.540 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 11480 2024-09-14T06:29:53.540 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:53.540 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:29:53.540 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:53.540 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:29:53.540 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.658563+0000", 2024-09-14T06:29:53.540 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:53.541 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:53.541 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:53.541 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:29:53.541 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.658563+0000", 2024-09-14T06:29:53.541 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:53.541 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:53.541 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:53.541 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:29:53.541 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.658658+0000", 2024-09-14T06:29:53.541 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.5346999999999994e-05 2024-09-14T06:29:53.541 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:53.541 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:53.541 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:29:53.541 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.658662+0000", 2024-09-14T06:29:53.541 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.14e-06 2024-09-14T06:29:53.542 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:53.542 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:53.542 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:29:53.542 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.658669+0000", 2024-09-14T06:29:53.542 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.7379999999999994e-06 2024-09-14T06:29:53.542 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:53.542 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:53.542 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:29:53.542 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.658675+0000", 2024-09-14T06:29:53.542 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.5430000000000001e-06 2024-09-14T06:29:53.542 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:53.542 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:53.542 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:29:53.542 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.710824+0000", 2024-09-14T06:29:53.542 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052148847999999998 2024-09-14T06:29:53.542 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:53.543 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:53.543 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:29:53.543 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.710864+0000", 2024-09-14T06:29:53.543 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.9925999999999997e-05 2024-09-14T06:29:53.543 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:53.543 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:53.543 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:29:53.543 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.710992+0000", 2024-09-14T06:29:53.543 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.000128622 2024-09-14T06:29:53.543 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:53.543 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:29:53.543 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:53.543 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:53.543 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:53.544 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:11481 3.1d 3:bcad7303:::rbd_data.112d92027590.0000000000000001:f [read 380928~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:29:53.544 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:25.658684+0000", 2024-09-14T06:29:53.544 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 27.804617532999998, 2024-09-14T06:29:53.544 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052508565, 2024-09-14T06:29:53.544 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:29:53.544 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:29:53.544 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:29:53.544 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:29:53.544 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:29:53.544 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:53.544 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 11481 2024-09-14T06:29:53.544 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:53.544 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:29:53.545 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:53.545 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:29:53.545 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.658684+0000", 2024-09-14T06:29:53.545 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:53.545 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:53.545 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:53.545 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:29:53.545 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.658684+0000", 2024-09-14T06:29:53.545 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:53.545 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:53.545 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:53.545 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:29:53.545 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.658715+0000", 2024-09-14T06:29:53.545 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.0703000000000001e-05 2024-09-14T06:29:53.545 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:53.546 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:53.546 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:29:53.546 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.658718+0000", 2024-09-14T06:29:53.546 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.0709999999999999e-06 2024-09-14T06:29:53.546 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:53.546 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:53.546 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:29:53.546 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.658725+0000", 2024-09-14T06:29:53.546 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.2339999999999997e-06 2024-09-14T06:29:53.546 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:53.546 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:53.546 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:29:53.547 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.658731+0000", 2024-09-14T06:29:53.547 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.6450000000000002e-06 2024-09-14T06:29:53.547 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:53.547 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:53.547 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:29:53.547 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.711020+0000", 2024-09-14T06:29:53.547 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052288886 2024-09-14T06:29:53.547 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:53.547 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:53.548 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:29:53.548 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.711061+0000", 2024-09-14T06:29:53.548 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.1084000000000002e-05 2024-09-14T06:29:53.548 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:53.548 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:53.548 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:29:53.548 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.711193+0000", 2024-09-14T06:29:53.548 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00013194199999999999 2024-09-14T06:29:53.548 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:53.548 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:29:53.548 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:53.548 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:53.548 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:53.549 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:11482 3.1d 3:bcad7303:::rbd_data.112d92027590.0000000000000001:f [read 391168~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:29:53.549 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:25.658740+0000", 2024-09-14T06:29:53.549 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 27.804562067999999, 2024-09-14T06:29:53.549 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052637015000000002, 2024-09-14T06:29:53.549 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:29:53.549 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:29:53.549 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:29:53.549 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:29:53.549 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:29:53.549 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:53.549 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 11482 2024-09-14T06:29:53.549 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:53.549 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:29:53.549 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:53.549 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:29:53.550 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.658740+0000", 2024-09-14T06:29:53.550 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:53.550 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:53.550 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:53.550 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:29:53.550 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.658740+0000", 2024-09-14T06:29:53.550 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:53.550 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:53.550 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:53.550 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:29:53.551 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.658880+0000", 2024-09-14T06:29:53.551 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00014036100000000001 2024-09-14T06:29:53.551 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:53.551 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:53.551 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:29:53.551 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.658883+0000", 2024-09-14T06:29:53.551 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.3969999999999998e-06 2024-09-14T06:29:53.551 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:53.551 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:53.551 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:29:53.551 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.658890+0000", 2024-09-14T06:29:53.552 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.1500000000000002e-06 2024-09-14T06:29:53.552 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:53.552 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:53.552 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:29:53.552 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.658897+0000", 2024-09-14T06:29:53.552 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.1260000000000003e-06 2024-09-14T06:29:53.552 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:53.552 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:53.552 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:29:53.552 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.711207+0000", 2024-09-14T06:29:53.552 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052310777000000003 2024-09-14T06:29:53.552 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:53.552 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:53.553 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:29:53.553 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.711249+0000", 2024-09-14T06:29:53.553 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.1786999999999998e-05 2024-09-14T06:29:53.553 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:53.553 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:53.553 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:29:53.553 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.711377+0000", 2024-09-14T06:29:53.553 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.000127417 2024-09-14T06:29:53.553 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:53.553 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:29:53.553 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:53.554 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:53.554 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:53.554 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:11483 3.1d 3:bcad7303:::rbd_data.112d92027590.0000000000000001:f [read 401408~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:29:53.554 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:25.658906+0000", 2024-09-14T06:29:53.554 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 27.804395284000002, 2024-09-14T06:29:53.554 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052595109000000001, 2024-09-14T06:29:53.554 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:29:53.554 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:29:53.554 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:29:53.554 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:29:53.554 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:29:53.554 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:53.554 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 11483 2024-09-14T06:29:53.554 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:53.554 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:29:53.555 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:53.555 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:29:53.555 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.658906+0000", 2024-09-14T06:29:53.555 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:53.555 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:53.555 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:53.555 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:29:53.555 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.658906+0000", 2024-09-14T06:29:53.555 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:53.555 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:53.555 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:53.555 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:29:53.555 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.664985+0000", 2024-09-14T06:29:53.555 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.0060787250000000001 2024-09-14T06:29:53.556 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:53.556 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:53.556 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:29:53.556 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.664991+0000", 2024-09-14T06:29:53.556 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.7030000000000003e-06 2024-09-14T06:29:53.556 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:53.556 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:53.556 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:29:53.556 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.664999+0000", 2024-09-14T06:29:53.557 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.4430000000000003e-06 2024-09-14T06:29:53.557 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:53.557 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:53.557 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:29:53.557 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.665006+0000", 2024-09-14T06:29:53.557 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.0840000000000003e-06 2024-09-14T06:29:53.557 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:53.557 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:53.557 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:29:53.557 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.711402+0000", 2024-09-14T06:29:53.557 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046395714999999997 2024-09-14T06:29:53.557 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:53.557 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:53.557 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:29:53.558 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.711443+0000", 2024-09-14T06:29:53.558 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.0911999999999997e-05 2024-09-14T06:29:53.558 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:53.558 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:53.558 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:29:53.558 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.711501+0000", 2024-09-14T06:29:53.558 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.8526999999999997e-05 2024-09-14T06:29:53.558 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:53.558 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:29:53.558 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:53.558 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:53.558 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:29:53.558 INFO:teuthology.orchestra.run.smithi138.stdout:} 2024-09-14T06:29:53.560 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-14T06:29:53.560 INFO:tasks.ceph.osd.4.smithi138.stderr:2024-09-14T06:29:53.558+0000 7f6c653c8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33615) UID: 0 2024-09-14T06:29:53.660 INFO:tasks.ceph.osd.5.smithi138.stderr:2024-09-14T06:29:53.659+0000 7fe13746f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33618) UID: 0 2024-09-14T06:29:53.704 INFO:teuthology.orchestra.run.smithi138.stdout:{ 2024-09-14T06:29:53.704 INFO:teuthology.orchestra.run.smithi138.stdout: "ops": [], 2024-09-14T06:29:53.704 INFO:teuthology.orchestra.run.smithi138.stdout: "num_ops": 0 2024-09-14T06:29:53.704 INFO:teuthology.orchestra.run.smithi138.stdout:} 2024-09-14T06:29:53.712 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-14T06:29:53.784 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-14T06:29:53.797 INFO:tasks.ceph.osd.5.smithi138.stderr:2024-09-14T06:29:53.796+0000 7fe13746f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33618) UID: 0 2024-09-14T06:29:53.861 INFO:tasks.ceph.osd.3.smithi094.stderr:2024-09-14T06:29:53.860+0000 7feeeafe8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33655) UID: 0 2024-09-14T06:29:53.904 INFO:teuthology.orchestra.run.smithi138.stdout:{ 2024-09-14T06:29:53.904 INFO:teuthology.orchestra.run.smithi138.stdout: "ops": [], 2024-09-14T06:29:53.904 INFO:teuthology.orchestra.run.smithi138.stdout: "complaint_time": 30, 2024-09-14T06:29:53.905 INFO:teuthology.orchestra.run.smithi138.stdout: "num_blocked_ops": 0 2024-09-14T06:29:53.905 INFO:teuthology.orchestra.run.smithi138.stdout:} 2024-09-14T06:29:53.919 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-14T06:29:53.964 INFO:tasks.ceph.osd.1.smithi094.stderr:2024-09-14T06:29:53.962+0000 7f328e47a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33641) UID: 0 2024-09-14T06:29:54.062 INFO:tasks.ceph.osd.3.smithi094.stderr:2024-09-14T06:29:54.061+0000 7feeeafe8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33655) UID: 0 2024-09-14T06:29:54.065 INFO:teuthology.orchestra.run.smithi138.stdout:{ 2024-09-14T06:29:54.065 INFO:teuthology.orchestra.run.smithi138.stdout: "size": 20, 2024-09-14T06:29:54.065 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 600, 2024-09-14T06:29:54.065 INFO:teuthology.orchestra.run.smithi138.stdout: "ops": [ 2024-09-14T06:29:54.065 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:54.065 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:1500 3.a 3:57c28b0f:::rbd_data.112d7dfbd766.000000000000000a:5 [read 471040~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-14T06:29:54.065 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:08.629204+0000", 2024-09-14T06:29:54.065 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 45.434187856000001, 2024-09-14T06:29:54.065 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.020473080000000001, 2024-09-14T06:29:54.066 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:29:54.066 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:29:54.066 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:29:54.066 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:29:54.066 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:29:54.066 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:54.066 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 1500 2024-09-14T06:29:54.066 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:54.066 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:29:54.066 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:54.066 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:29:54.066 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.629204+0000", 2024-09-14T06:29:54.066 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:54.066 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:54.067 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:54.067 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:29:54.067 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.629204+0000", 2024-09-14T06:29:54.067 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:54.067 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:54.067 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:54.067 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:29:54.067 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.629204+0000", 2024-09-14T06:29:54.067 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.34e-07 2024-09-14T06:29:54.067 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:54.067 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:54.067 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:29:54.067 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.629206+0000", 2024-09-14T06:29:54.067 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.939e-06 2024-09-14T06:29:54.068 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:54.068 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:54.068 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:29:54.068 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.629212+0000", 2024-09-14T06:29:54.068 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.8669999999999999e-06 2024-09-14T06:29:54.068 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:54.068 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:54.068 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:29:54.068 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.629216+0000", 2024-09-14T06:29:54.068 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.3830000000000002e-06 2024-09-14T06:29:54.068 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:54.068 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:54.068 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:29:54.068 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.649288+0000", 2024-09-14T06:29:54.069 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.020072113999999999 2024-09-14T06:29:54.069 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:54.069 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:54.069 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:29:54.069 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.649355+0000", 2024-09-14T06:29:54.069 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.7218000000000006e-05 2024-09-14T06:29:54.069 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:54.069 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:54.069 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:29:54.069 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.649677+0000", 2024-09-14T06:29:54.069 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00032122500000000002 2024-09-14T06:29:54.069 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:54.069 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:29:54.069 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:54.070 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:54.070 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:54.070 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:1501 3.a 3:57c28b0f:::rbd_data.112d7dfbd766.000000000000000a:5 [read 487424~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-14T06:29:54.070 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:08.629223+0000", 2024-09-14T06:29:54.070 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 45.434168407000001, 2024-09-14T06:29:54.070 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.021370209000000001, 2024-09-14T06:29:54.070 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:29:54.070 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:29:54.070 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:29:54.070 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:29:54.070 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:29:54.070 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:54.070 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 1501 2024-09-14T06:29:54.071 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:54.071 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:29:54.071 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:54.071 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:29:54.071 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.629223+0000", 2024-09-14T06:29:54.071 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:54.071 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:54.071 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:54.071 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:29:54.071 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.629223+0000", 2024-09-14T06:29:54.071 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:54.071 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:54.071 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:54.071 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:29:54.071 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.629223+0000", 2024-09-14T06:29:54.072 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.9900000000000001e-07 2024-09-14T06:29:54.072 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:54.072 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:54.072 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:29:54.072 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.629231+0000", 2024-09-14T06:29:54.072 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.9179999999999994e-06 2024-09-14T06:29:54.072 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:54.072 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:54.072 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:29:54.072 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.629238+0000", 2024-09-14T06:29:54.072 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.421e-06 2024-09-14T06:29:54.072 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:54.072 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:54.073 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:29:54.073 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.629242+0000", 2024-09-14T06:29:54.073 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.5419999999999999e-06 2024-09-14T06:29:54.073 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:54.073 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:54.073 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:29:54.073 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.650284+0000", 2024-09-14T06:29:54.073 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.021041131000000001 2024-09-14T06:29:54.073 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:54.073 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:54.073 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:29:54.073 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.650361+0000", 2024-09-14T06:29:54.073 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.7787999999999995e-05 2024-09-14T06:29:54.073 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:54.073 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:54.074 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:29:54.074 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.650593+0000", 2024-09-14T06:29:54.074 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00023201 2024-09-14T06:29:54.074 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:54.074 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:29:54.074 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:54.074 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:54.074 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:54.074 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:1502 3.a 3:57c28b0f:::rbd_data.112d7dfbd766.000000000000000a:5 [read 503808~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-14T06:29:54.074 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:08.629249+0000", 2024-09-14T06:29:54.074 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 45.434142160999997, 2024-09-14T06:29:54.074 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.021713217, 2024-09-14T06:29:54.074 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:29:54.074 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:29:54.074 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:29:54.075 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:29:54.075 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:29:54.075 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:54.075 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 1502 2024-09-14T06:29:54.075 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:54.075 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:29:54.075 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:54.075 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:29:54.075 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.629249+0000", 2024-09-14T06:29:54.075 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:54.075 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:54.075 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:54.075 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:29:54.075 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.629249+0000", 2024-09-14T06:29:54.076 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:54.076 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:54.076 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:54.076 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:29:54.076 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.629250+0000", 2024-09-14T06:29:54.076 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.03e-07 2024-09-14T06:29:54.076 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:54.076 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:54.076 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:29:54.076 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.629252+0000", 2024-09-14T06:29:54.076 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.995e-06 2024-09-14T06:29:54.076 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:54.076 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:54.076 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:29:54.077 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.629257+0000", 2024-09-14T06:29:54.077 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.7010000000000001e-06 2024-09-14T06:29:54.077 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:54.077 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:54.077 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:29:54.077 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.629262+0000", 2024-09-14T06:29:54.077 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.16e-06 2024-09-14T06:29:54.077 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:54.077 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:54.077 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:29:54.077 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.650629+0000", 2024-09-14T06:29:54.077 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.021367684000000001 2024-09-14T06:29:54.077 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:54.078 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:54.078 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:29:54.078 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.650718+0000", 2024-09-14T06:29:54.078 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.8832000000000006e-05 2024-09-14T06:29:54.078 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:54.078 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:54.078 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:29:54.078 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.650963+0000", 2024-09-14T06:29:54.078 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00024444199999999999 2024-09-14T06:29:54.078 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:54.078 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:29:54.078 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:54.078 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:54.078 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:54.078 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:1503 3.a 3:57c28b0f:::rbd_data.112d7dfbd766.000000000000000a:5 [read 520192~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-14T06:29:54.079 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:08.629268+0000", 2024-09-14T06:29:54.079 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 45.434123620000001, 2024-09-14T06:29:54.079 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.022244885999999998, 2024-09-14T06:29:54.079 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:29:54.079 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:29:54.079 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:29:54.079 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:29:54.079 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:29:54.079 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:54.079 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 1503 2024-09-14T06:29:54.079 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:54.079 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:29:54.079 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:54.079 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:29:54.080 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.629268+0000", 2024-09-14T06:29:54.080 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:54.080 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:54.080 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:54.080 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:29:54.080 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.629268+0000", 2024-09-14T06:29:54.080 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:54.080 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:54.080 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:54.080 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:29:54.080 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.629268+0000", 2024-09-14T06:29:54.080 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.6599999999999996e-07 2024-09-14T06:29:54.080 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:54.080 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:54.081 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:29:54.081 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.629270+0000", 2024-09-14T06:29:54.081 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.7129999999999999e-06 2024-09-14T06:29:54.081 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:54.081 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:54.081 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:29:54.081 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.629276+0000", 2024-09-14T06:29:54.081 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.8119999999999996e-06 2024-09-14T06:29:54.081 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:54.081 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:54.081 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:29:54.081 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.629280+0000", 2024-09-14T06:29:54.081 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.4920000000000004e-06 2024-09-14T06:29:54.081 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:54.082 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:54.082 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:29:54.082 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.651039+0000", 2024-09-14T06:29:54.082 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.021758116000000001 2024-09-14T06:29:54.082 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:54.082 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:54.082 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:29:54.082 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.651127+0000", 2024-09-14T06:29:54.082 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.7983000000000001e-05 2024-09-14T06:29:54.082 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:54.082 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:54.082 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:29:54.082 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.651513+0000", 2024-09-14T06:29:54.082 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00038620400000000002 2024-09-14T06:29:54.083 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:54.083 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:29:54.083 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:54.083 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:54.083 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:54.083 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:73587 3.a 3:5604efef:::rbd_data.112d4c988ca3.000000000000001d:1d [read 98304~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-14T06:29:54.083 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:53.828877+0000", 2024-09-14T06:29:54.083 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 0.23451478100000001, 2024-09-14T06:29:54.083 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.020953949999999999, 2024-09-14T06:29:54.083 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:29:54.083 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:29:54.083 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:29:54.083 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:29:54.083 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:29:54.084 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:54.084 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 73587 2024-09-14T06:29:54.084 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:54.084 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:29:54.084 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:54.084 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:29:54.084 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:53.828877+0000", 2024-09-14T06:29:54.084 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:54.084 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:54.084 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:54.084 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:29:54.084 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:53.828877+0000", 2024-09-14T06:29:54.084 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:54.084 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:54.085 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:54.085 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:29:54.085 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:53.828877+0000", 2024-09-14T06:29:54.085 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.3799999999999998e-07 2024-09-14T06:29:54.085 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:54.085 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:54.085 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:29:54.085 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:53.828879+0000", 2024-09-14T06:29:54.085 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.7269999999999999e-06 2024-09-14T06:29:54.085 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:54.085 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:54.085 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:29:54.085 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:53.828884+0000", 2024-09-14T06:29:54.085 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.383e-06 2024-09-14T06:29:54.086 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:54.086 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:54.086 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:29:54.086 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:53.828888+0000", 2024-09-14T06:29:54.086 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.25e-06 2024-09-14T06:29:54.086 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:54.086 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:54.086 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:29:54.086 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:53.849117+0000", 2024-09-14T06:29:54.086 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.020228349999999999 2024-09-14T06:29:54.086 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:54.086 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:54.086 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:29:54.086 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:53.849237+0000", 2024-09-14T06:29:54.087 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00011999500000000001 2024-09-14T06:29:54.087 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:54.087 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:54.087 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:29:54.087 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:53.849831+0000", 2024-09-14T06:29:54.087 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00059390699999999996 2024-09-14T06:29:54.087 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:54.087 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:29:54.087 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:54.087 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:54.087 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:54.087 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:73588 3.a 3:5604efef:::rbd_data.112d4c988ca3.000000000000001d:1d [read 245760~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-14T06:29:54.087 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:53.828894+0000", 2024-09-14T06:29:54.087 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 0.234497873, 2024-09-14T06:29:54.088 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.021322085000000001, 2024-09-14T06:29:54.088 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:29:54.088 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:29:54.088 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:29:54.088 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:29:54.088 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:29:54.088 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:54.088 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 73588 2024-09-14T06:29:54.088 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:54.088 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:29:54.088 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:54.088 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:29:54.088 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:53.828894+0000", 2024-09-14T06:29:54.089 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:54.089 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:54.089 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:54.089 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:29:54.089 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:53.828894+0000", 2024-09-14T06:29:54.089 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:54.089 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:54.089 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:54.089 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:29:54.089 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:53.828894+0000", 2024-09-14T06:29:54.089 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.22e-07 2024-09-14T06:29:54.089 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:54.089 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:54.089 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:29:54.090 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:53.828896+0000", 2024-09-14T06:29:54.090 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.668e-06 2024-09-14T06:29:54.090 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:54.090 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:54.090 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:29:54.090 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:53.828901+0000", 2024-09-14T06:29:54.090 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.3480000000000003e-06 2024-09-14T06:29:54.090 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:54.090 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:54.090 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:29:54.090 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:53.828905+0000", 2024-09-14T06:29:54.090 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.2350000000000001e-06 2024-09-14T06:29:54.090 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:54.090 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:54.091 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:29:54.091 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:53.849887+0000", 2024-09-14T06:29:54.091 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.020981883 2024-09-14T06:29:54.091 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:54.091 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:54.091 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:29:54.091 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:53.849988+0000", 2024-09-14T06:29:54.091 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00010122499999999999 2024-09-14T06:29:54.092 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:54.092 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:54.092 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:29:54.092 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:53.850216+0000", 2024-09-14T06:29:54.092 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00022740400000000001 2024-09-14T06:29:54.092 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:54.092 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:29:54.092 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:54.092 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:54.092 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:54.092 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:73605 3.a 3:549bef42:::rbd_data.112d4c988ca3.0000000000000022:1d [read 98304~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-14T06:29:54.092 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:53.828910+0000", 2024-09-14T06:29:54.092 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 0.23448113600000001, 2024-09-14T06:29:54.092 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.021797369, 2024-09-14T06:29:54.093 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:29:54.093 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:29:54.093 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:29:54.093 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:29:54.093 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:29:54.093 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:54.093 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 73605 2024-09-14T06:29:54.093 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:54.093 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:29:54.093 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:54.093 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:29:54.093 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:53.828910+0000", 2024-09-14T06:29:54.093 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:54.094 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:54.094 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:54.094 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:29:54.094 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:53.828910+0000", 2024-09-14T06:29:54.094 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:54.094 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:54.094 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:54.094 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:29:54.094 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:53.828915+0000", 2024-09-14T06:29:54.094 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.5220000000000001e-06 2024-09-14T06:29:54.094 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:54.094 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:54.094 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:29:54.094 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:53.828917+0000", 2024-09-14T06:29:54.094 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.812e-06 2024-09-14T06:29:54.095 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:54.095 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:54.095 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:29:54.095 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:53.828923+0000", 2024-09-14T06:29:54.095 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.8499999999999999e-06 2024-09-14T06:29:54.095 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:54.095 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:54.095 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:29:54.095 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:53.828927+0000", 2024-09-14T06:29:54.095 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.6450000000000004e-06 2024-09-14T06:29:54.095 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:54.095 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:54.095 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:29:54.096 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:53.850248+0000", 2024-09-14T06:29:54.096 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.021320543000000001 2024-09-14T06:29:54.096 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:54.096 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:54.096 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:29:54.096 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:53.850500+0000", 2024-09-14T06:29:54.096 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00025234999999999998 2024-09-14T06:29:54.096 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:54.096 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:54.096 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:29:54.096 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:53.850708+0000", 2024-09-14T06:29:54.096 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00020764700000000001 2024-09-14T06:29:54.096 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:54.096 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:29:54.097 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:54.097 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:54.097 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:54.097 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:73606 3.a 3:549bef42:::rbd_data.112d4c988ca3.0000000000000022:1d [read 131072~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-14T06:29:54.097 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:53.828933+0000", 2024-09-14T06:29:54.097 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 0.234458526, 2024-09-14T06:29:54.097 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.022113200999999999, 2024-09-14T06:29:54.097 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:29:54.097 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:29:54.097 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:29:54.097 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:29:54.097 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:29:54.097 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:54.098 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 73606 2024-09-14T06:29:54.098 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:54.098 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:29:54.098 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:54.098 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:29:54.098 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:53.828933+0000", 2024-09-14T06:29:54.098 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:54.098 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:54.098 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:54.098 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:29:54.098 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:53.828933+0000", 2024-09-14T06:29:54.098 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:54.098 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:54.098 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:54.099 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:29:54.099 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:53.828933+0000", 2024-09-14T06:29:54.099 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.53e-07 2024-09-14T06:29:54.099 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:54.099 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:54.099 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:29:54.099 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:53.828935+0000", 2024-09-14T06:29:54.099 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.559e-06 2024-09-14T06:29:54.099 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:54.099 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:54.099 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:29:54.099 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:53.828940+0000", 2024-09-14T06:29:54.099 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.5600000000000001e-06 2024-09-14T06:29:54.099 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:54.099 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:54.100 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:29:54.100 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:53.828945+0000", 2024-09-14T06:29:54.100 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.4469999999999999e-06 2024-09-14T06:29:54.100 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:54.100 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:54.100 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:29:54.100 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:53.850738+0000", 2024-09-14T06:29:54.100 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.021793265999999999 2024-09-14T06:29:54.100 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:54.100 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:54.100 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:29:54.100 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:53.850813+0000", 2024-09-14T06:29:54.100 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.4810999999999999e-05 2024-09-14T06:29:54.100 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:54.101 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:54.101 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:29:54.101 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:53.851046+0000", 2024-09-14T06:29:54.101 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.000233205 2024-09-14T06:29:54.101 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:54.101 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:29:54.101 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:54.101 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:54.101 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:54.101 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:73607 3.a 3:549bef42:::rbd_data.112d4c988ca3.0000000000000022:1d [read 163840~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-14T06:29:54.101 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:53.828950+0000", 2024-09-14T06:29:54.101 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 0.23444137200000001, 2024-09-14T06:29:54.101 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.022421109000000002, 2024-09-14T06:29:54.102 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:29:54.102 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:29:54.102 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:29:54.102 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:29:54.102 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:29:54.102 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:54.102 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 73607 2024-09-14T06:29:54.102 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:54.102 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:29:54.102 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:54.102 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:29:54.102 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:53.828950+0000", 2024-09-14T06:29:54.102 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:54.102 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:54.103 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:54.103 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:29:54.103 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:53.828950+0000", 2024-09-14T06:29:54.103 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:54.103 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:54.103 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:54.103 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:29:54.103 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:53.828950+0000", 2024-09-14T06:29:54.103 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.3999999999999997e-07 2024-09-14T06:29:54.103 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:54.103 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:54.103 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:29:54.103 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:53.828956+0000", 2024-09-14T06:29:54.103 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.5639999999999996e-06 2024-09-14T06:29:54.103 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:54.103 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:54.103 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:29:54.104 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:53.828962+0000", 2024-09-14T06:29:54.104 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.9159999999999998e-06 2024-09-14T06:29:54.104 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:54.104 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:54.104 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:29:54.104 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:53.828966+0000", 2024-09-14T06:29:54.104 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.5050000000000001e-06 2024-09-14T06:29:54.104 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:54.104 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:54.104 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:29:54.104 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:53.851077+0000", 2024-09-14T06:29:54.104 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.022110603999999999 2024-09-14T06:29:54.104 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:54.104 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:54.104 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:29:54.104 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:53.851156+0000", 2024-09-14T06:29:54.105 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.8781000000000003e-05 2024-09-14T06:29:54.105 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:54.105 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:54.105 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:29:54.105 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:53.851371+0000", 2024-09-14T06:29:54.105 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00021539899999999999 2024-09-14T06:29:54.105 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:54.105 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:29:54.105 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:54.105 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:54.105 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:54.105 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:73608 3.a 3:549bef42:::rbd_data.112d4c988ca3.0000000000000022:1d [read 196608~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-14T06:29:54.105 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:53.828972+0000", 2024-09-14T06:29:54.105 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 0.23441920299999999, 2024-09-14T06:29:54.105 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.022703864000000001, 2024-09-14T06:29:54.106 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:29:54.106 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:29:54.106 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:29:54.106 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:29:54.106 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:29:54.106 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:54.106 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 73608 2024-09-14T06:29:54.106 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:54.106 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:29:54.106 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:54.106 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:29:54.106 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:53.828972+0000", 2024-09-14T06:29:54.106 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:54.106 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:54.106 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:54.107 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:29:54.107 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:53.828972+0000", 2024-09-14T06:29:54.107 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:54.107 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:54.107 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:54.107 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:29:54.107 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:53.828973+0000", 2024-09-14T06:29:54.107 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.5600000000000001e-07 2024-09-14T06:29:54.107 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:54.107 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:54.107 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:29:54.107 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:53.828974+0000", 2024-09-14T06:29:54.107 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.6950000000000001e-06 2024-09-14T06:29:54.107 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:54.107 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:54.108 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:29:54.108 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:53.828979+0000", 2024-09-14T06:29:54.108 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.0880000000000002e-06 2024-09-14T06:29:54.108 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:54.108 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:54.108 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:29:54.108 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:53.829825+0000", 2024-09-14T06:29:54.108 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00084584999999999999 2024-09-14T06:29:54.108 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:54.108 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:54.108 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:29:54.108 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:53.851403+0000", 2024-09-14T06:29:54.108 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.021577875999999999 2024-09-14T06:29:54.108 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:54.108 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:54.108 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:29:54.109 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:53.851482+0000", 2024-09-14T06:29:54.109 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.8668e-05 2024-09-14T06:29:54.109 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:54.109 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:54.109 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:29:54.109 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:53.851676+0000", 2024-09-14T06:29:54.109 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00019433100000000001 2024-09-14T06:29:54.109 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:54.109 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:29:54.109 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:54.109 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:54.109 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:54.109 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:73873 3.a 3:52a26c8a:::rbd_data.112d4c988ca3.00000000000000a4:1d [read 131072~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-14T06:29:54.109 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:53.831416+0000", 2024-09-14T06:29:54.109 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 0.23197567399999999, 2024-09-14T06:29:54.110 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.020616973, 2024-09-14T06:29:54.110 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:29:54.110 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:29:54.110 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:29:54.110 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:29:54.110 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:29:54.110 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:54.110 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 73873 2024-09-14T06:29:54.110 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:54.110 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:29:54.110 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:54.110 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:29:54.110 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:53.831416+0000", 2024-09-14T06:29:54.110 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:54.110 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:54.110 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:54.111 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:29:54.111 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:53.831416+0000", 2024-09-14T06:29:54.111 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:54.111 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:54.111 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:54.111 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:29:54.111 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:53.831416+0000", 2024-09-14T06:29:54.111 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.9099999999999999e-07 2024-09-14T06:29:54.111 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:54.111 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:54.111 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:29:54.111 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:53.831670+0000", 2024-09-14T06:29:54.111 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00025330600000000002 2024-09-14T06:29:54.111 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:54.111 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:54.111 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:29:54.112 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:53.831679+0000", 2024-09-14T06:29:54.112 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.0210000000000005e-06 2024-09-14T06:29:54.112 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:54.112 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:54.112 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:29:54.112 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:53.831685+0000", 2024-09-14T06:29:54.112 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.3729999999999998e-06 2024-09-14T06:29:54.112 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:54.112 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:54.112 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:29:54.112 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:53.851704+0000", 2024-09-14T06:29:54.112 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.020018978 2024-09-14T06:29:54.112 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:54.112 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:54.112 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:29:54.113 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:53.851773+0000", 2024-09-14T06:29:54.113 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.881e-05 2024-09-14T06:29:54.113 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:54.113 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:54.113 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:29:54.113 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:53.852033+0000", 2024-09-14T06:29:54.113 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00026009399999999999 2024-09-14T06:29:54.113 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:54.113 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:29:54.113 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:54.113 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:54.113 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:54.113 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:73874 3.a 3:52a26c8a:::rbd_data.112d4c988ca3.00000000000000a4:1d [read 163840~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-14T06:29:54.114 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:53.831694+0000", 2024-09-14T06:29:54.114 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 0.23169724, 2024-09-14T06:29:54.114 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.020649862000000001, 2024-09-14T06:29:54.114 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:29:54.114 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:29:54.114 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:29:54.114 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:29:54.114 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:29:54.114 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:54.114 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 73874 2024-09-14T06:29:54.114 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:54.114 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:29:54.114 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:54.114 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:29:54.114 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:53.831694+0000", 2024-09-14T06:29:54.115 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:54.115 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:54.115 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:54.115 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:29:54.115 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:53.831694+0000", 2024-09-14T06:29:54.115 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:54.115 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:54.115 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:54.115 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:29:54.115 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:53.831695+0000", 2024-09-14T06:29:54.115 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.0699999999999997e-07 2024-09-14T06:29:54.115 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:54.115 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:54.115 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:29:54.115 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:53.831697+0000", 2024-09-14T06:29:54.116 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.9929999999999998e-06 2024-09-14T06:29:54.116 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:54.116 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:54.116 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:29:54.116 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:53.831703+0000", 2024-09-14T06:29:54.116 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.6610000000000003e-06 2024-09-14T06:29:54.116 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:54.116 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:54.116 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:29:54.116 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:53.831708+0000", 2024-09-14T06:29:54.116 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.8570000000000003e-06 2024-09-14T06:29:54.116 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:54.116 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:54.116 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:29:54.117 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:53.852061+0000", 2024-09-14T06:29:54.117 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.020352577 2024-09-14T06:29:54.117 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:54.117 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:54.117 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:29:54.117 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:53.852135+0000", 2024-09-14T06:29:54.117 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.3656000000000001e-05 2024-09-14T06:29:54.117 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:54.117 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:54.117 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:29:54.117 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:53.852344+0000", 2024-09-14T06:29:54.117 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00020961100000000001 2024-09-14T06:29:54.117 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:54.117 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:29:54.118 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:54.118 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:54.118 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:54.118 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:73875 3.a 3:52a26c8a:::rbd_data.112d4c988ca3.00000000000000a4:1d [read 196608~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-14T06:29:54.118 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:53.831797+0000", 2024-09-14T06:29:54.118 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 0.231594518, 2024-09-14T06:29:54.118 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.020875273, 2024-09-14T06:29:54.118 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:29:54.118 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:29:54.118 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:29:54.118 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:29:54.118 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:29:54.118 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:54.118 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 73875 2024-09-14T06:29:54.119 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:54.119 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:29:54.119 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:54.119 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:29:54.119 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:53.831797+0000", 2024-09-14T06:29:54.119 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:54.119 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:54.119 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:54.119 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:29:54.119 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:53.831797+0000", 2024-09-14T06:29:54.119 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:54.119 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:54.119 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:54.119 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:29:54.119 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:53.831798+0000", 2024-09-14T06:29:54.119 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.0399999999999996e-07 2024-09-14T06:29:54.120 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:54.120 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:54.120 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:29:54.120 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:53.831800+0000", 2024-09-14T06:29:54.120 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.79e-06 2024-09-14T06:29:54.120 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:54.120 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:54.120 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:29:54.120 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:53.831809+0000", 2024-09-14T06:29:54.120 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.8119999999999997e-06 2024-09-14T06:29:54.120 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:54.120 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:54.120 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:29:54.120 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:53.831822+0000", 2024-09-14T06:29:54.121 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.2781e-05 2024-09-14T06:29:54.121 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:54.121 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:54.121 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:29:54.121 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:53.852379+0000", 2024-09-14T06:29:54.121 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.020557392000000001 2024-09-14T06:29:54.121 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:54.121 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:54.121 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:29:54.121 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:53.852461+0000", 2024-09-14T06:29:54.121 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.2055e-05 2024-09-14T06:29:54.121 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:54.121 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:54.121 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:29:54.122 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:53.852672+0000", 2024-09-14T06:29:54.122 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00021083899999999999 2024-09-14T06:29:54.122 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:54.122 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:29:54.122 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:54.122 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:54.122 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:54.122 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:73876 3.a 3:52a26c8a:::rbd_data.112d4c988ca3.00000000000000a4:1d [read 229376~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-14T06:29:54.122 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:53.831833+0000", 2024-09-14T06:29:54.122 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 0.23155819, 2024-09-14T06:29:54.122 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.02119861, 2024-09-14T06:29:54.122 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:29:54.122 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:29:54.122 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:29:54.123 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:29:54.123 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:29:54.123 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:54.123 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 73876 2024-09-14T06:29:54.123 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:54.123 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:29:54.123 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:54.123 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:29:54.123 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:53.831833+0000", 2024-09-14T06:29:54.123 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:54.123 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:54.123 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:54.123 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:29:54.123 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:53.831833+0000", 2024-09-14T06:29:54.123 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:54.124 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:54.124 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:54.124 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:29:54.124 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:53.831834+0000", 2024-09-14T06:29:54.124 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.0100000000000005e-07 2024-09-14T06:29:54.124 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:54.124 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:54.124 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:29:54.124 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:53.831836+0000", 2024-09-14T06:29:54.124 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.227e-06 2024-09-14T06:29:54.124 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:54.124 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:54.124 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:29:54.124 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:53.831844+0000", 2024-09-14T06:29:54.125 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.6090000000000003e-06 2024-09-14T06:29:54.125 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:54.125 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:54.125 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:29:54.125 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:53.831849+0000", 2024-09-14T06:29:54.125 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.9219999999999996e-06 2024-09-14T06:29:54.125 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:54.125 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:54.125 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:29:54.125 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:53.852704+0000", 2024-09-14T06:29:54.125 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.020854961000000002 2024-09-14T06:29:54.125 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:54.125 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:54.125 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:29:54.126 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:53.852778+0000", 2024-09-14T06:29:54.126 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.4148999999999998e-05 2024-09-14T06:29:54.126 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:54.126 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:54.126 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:29:54.126 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:53.853032+0000", 2024-09-14T06:29:54.126 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00025424099999999999 2024-09-14T06:29:54.126 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:54.126 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:29:54.126 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:54.126 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:54.126 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:54.126 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:73936 3.a 3:529fd2a8:::rbd_data.112d4c988ca3.00000000000000b5:1d [read 0~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-14T06:29:54.126 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:53.832060+0000", 2024-09-14T06:29:54.127 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 0.23133182299999999, 2024-09-14T06:29:54.127 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.021161967, 2024-09-14T06:29:54.127 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:29:54.127 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:29:54.127 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:29:54.127 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:29:54.127 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:29:54.127 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:54.127 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 73936 2024-09-14T06:29:54.127 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:54.127 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:29:54.127 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:54.127 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:29:54.127 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:53.832060+0000", 2024-09-14T06:29:54.127 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:54.128 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:54.128 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:54.128 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:29:54.128 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:53.832060+0000", 2024-09-14T06:29:54.128 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:54.128 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:54.128 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:54.128 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:29:54.128 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:53.832060+0000", 2024-09-14T06:29:54.128 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.63e-07 2024-09-14T06:29:54.128 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:54.128 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:54.128 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:29:54.128 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:53.832062+0000", 2024-09-14T06:29:54.129 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.125e-06 2024-09-14T06:29:54.129 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:54.129 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:54.129 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:29:54.129 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:53.832069+0000", 2024-09-14T06:29:54.129 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.3990000000000001e-06 2024-09-14T06:29:54.129 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:54.129 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:54.129 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:29:54.129 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:53.832074+0000", 2024-09-14T06:29:54.129 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.0479999999999998e-06 2024-09-14T06:29:54.129 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:54.129 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:54.130 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:29:54.130 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:53.853060+0000", 2024-09-14T06:29:54.130 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.020986707 2024-09-14T06:29:54.130 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:54.130 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:54.130 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:29:54.130 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:53.853139+0000", 2024-09-14T06:29:54.130 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.8636999999999999e-05 2024-09-14T06:29:54.130 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:54.130 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:54.130 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:29:54.130 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:53.853222+0000", 2024-09-14T06:29:54.130 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.2588000000000003e-05 2024-09-14T06:29:54.130 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:54.130 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:29:54.131 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:54.131 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:54.131 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:54.131 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:73937 3.a 3:529fd2a8:::rbd_data.112d4c988ca3.00000000000000b5:1d [read 180224~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-14T06:29:54.131 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:53.832084+0000", 2024-09-14T06:29:54.131 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 0.23130743600000001, 2024-09-14T06:29:54.131 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.021667265000000002, 2024-09-14T06:29:54.131 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:29:54.131 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:29:54.131 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:29:54.131 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:29:54.131 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:29:54.131 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:54.131 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 73937 2024-09-14T06:29:54.132 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:54.132 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:29:54.132 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:54.132 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:29:54.132 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:53.832084+0000", 2024-09-14T06:29:54.132 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:54.132 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:54.132 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:54.132 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:29:54.132 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:53.832084+0000", 2024-09-14T06:29:54.132 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:54.132 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:54.132 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:54.132 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:29:54.132 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:53.832085+0000", 2024-09-14T06:29:54.133 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.3700000000000001e-07 2024-09-14T06:29:54.133 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:54.133 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:54.133 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:29:54.133 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:53.832087+0000", 2024-09-14T06:29:54.133 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.3779999999999999e-06 2024-09-14T06:29:54.133 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:54.133 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:54.133 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:29:54.133 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:53.832096+0000", 2024-09-14T06:29:54.133 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.8270000000000004e-06 2024-09-14T06:29:54.133 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:54.133 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:54.133 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:29:54.134 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:53.832110+0000", 2024-09-14T06:29:54.134 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.4229999999999999e-05 2024-09-14T06:29:54.134 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:54.134 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:54.134 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:29:54.134 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:53.853249+0000", 2024-09-14T06:29:54.134 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.021139128 2024-09-14T06:29:54.134 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:54.134 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:54.134 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:29:54.134 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:53.853314+0000", 2024-09-14T06:29:54.134 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.4988999999999997e-05 2024-09-14T06:29:54.134 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:54.134 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:54.135 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:29:54.135 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:53.853751+0000", 2024-09-14T06:29:54.135 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00043727600000000001 2024-09-14T06:29:54.135 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:54.135 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:29:54.135 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:54.135 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:54.135 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:54.135 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:73953 3.a 3:57f2eddd:::rbd_data.112d4c988ca3.00000000000000bb:1d [read 0~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-14T06:29:54.135 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:53.833064+0000", 2024-09-14T06:29:54.135 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 0.23032775599999999, 2024-09-14T06:29:54.135 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.020900548000000001, 2024-09-14T06:29:54.135 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:29:54.135 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:29:54.135 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:29:54.136 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:29:54.136 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:29:54.136 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:54.136 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 73953 2024-09-14T06:29:54.136 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:54.136 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:29:54.136 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:54.136 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:29:54.136 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:53.833064+0000", 2024-09-14T06:29:54.136 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:54.136 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:54.136 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:54.136 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:29:54.136 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:53.833064+0000", 2024-09-14T06:29:54.137 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:54.137 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:54.137 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:54.137 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:29:54.137 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:53.833065+0000", 2024-09-14T06:29:54.137 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.1659999999999999e-06 2024-09-14T06:29:54.137 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:54.137 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:54.137 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:29:54.137 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:53.833070+0000", 2024-09-14T06:29:54.137 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.2519999999999999e-06 2024-09-14T06:29:54.137 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:54.137 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:54.137 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:29:54.138 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:53.833082+0000", 2024-09-14T06:29:54.138 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.2333e-05 2024-09-14T06:29:54.138 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:54.138 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:54.138 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:29:54.138 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:53.833090+0000", 2024-09-14T06:29:54.138 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.8150000000000006e-06 2024-09-14T06:29:54.138 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:54.138 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:54.138 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:29:54.138 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:53.853781+0000", 2024-09-14T06:29:54.138 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.020690664000000001 2024-09-14T06:29:54.138 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:54.138 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:54.139 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:29:54.139 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:53.853859+0000", 2024-09-14T06:29:54.139 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.7614999999999995e-05 2024-09-14T06:29:54.139 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:54.139 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:54.139 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:29:54.139 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:53.853964+0000", 2024-09-14T06:29:54.139 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.000105703 2024-09-14T06:29:54.139 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:54.139 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:29:54.139 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:54.139 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:54.139 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:54.140 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:73954 3.a 3:57f2eddd:::rbd_data.112d4c988ca3.00000000000000bb:1d [read 32768~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-14T06:29:54.140 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:53.833104+0000", 2024-09-14T06:29:54.140 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 0.230287468, 2024-09-14T06:29:54.140 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.021252760999999998, 2024-09-14T06:29:54.140 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:29:54.140 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:29:54.140 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:29:54.140 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:29:54.140 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:29:54.140 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:54.140 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 73954 2024-09-14T06:29:54.140 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:54.140 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:29:54.140 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:54.140 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:29:54.141 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:53.833104+0000", 2024-09-14T06:29:54.141 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:54.141 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:54.141 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:54.141 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:29:54.141 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:53.833104+0000", 2024-09-14T06:29:54.141 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:54.141 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:54.141 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:54.141 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:29:54.141 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:53.833105+0000", 2024-09-14T06:29:54.141 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.1799999999999995e-07 2024-09-14T06:29:54.141 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:54.141 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:54.142 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:29:54.142 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:53.833108+0000", 2024-09-14T06:29:54.142 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.1030000000000002e-06 2024-09-14T06:29:54.142 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:54.142 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:54.142 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:29:54.142 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:53.833115+0000", 2024-09-14T06:29:54.142 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.2930000000000003e-06 2024-09-14T06:29:54.142 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:54.142 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:54.142 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:29:54.142 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:53.833120+0000", 2024-09-14T06:29:54.142 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.3199999999999999e-06 2024-09-14T06:29:54.142 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:54.143 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:54.143 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:29:54.143 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:53.853995+0000", 2024-09-14T06:29:54.143 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.020875210000000002 2024-09-14T06:29:54.143 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:54.143 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:54.143 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:29:54.143 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:53.854085+0000", 2024-09-14T06:29:54.143 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.9339999999999995e-05 2024-09-14T06:29:54.143 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:54.143 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:54.143 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:29:54.143 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:53.854357+0000", 2024-09-14T06:29:54.143 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00027197700000000002 2024-09-14T06:29:54.143 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:54.144 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:29:54.144 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:54.144 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:54.144 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:54.144 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:73955 3.a 3:57f2eddd:::rbd_data.112d4c988ca3.00000000000000bb:1d [read 65536~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-14T06:29:54.144 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:53.833128+0000", 2024-09-14T06:29:54.144 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 0.23026305, 2024-09-14T06:29:54.144 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.021578482999999999, 2024-09-14T06:29:54.144 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:29:54.144 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:29:54.144 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:29:54.144 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:29:54.144 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:29:54.144 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:54.144 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 73955 2024-09-14T06:29:54.145 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:54.145 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:29:54.145 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:54.145 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:29:54.145 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:53.833128+0000", 2024-09-14T06:29:54.145 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:54.145 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:54.145 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:54.145 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:29:54.145 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:53.833128+0000", 2024-09-14T06:29:54.145 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:54.145 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:54.145 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:54.145 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:29:54.145 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:53.833129+0000", 2024-09-14T06:29:54.145 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.2199999999999999e-07 2024-09-14T06:29:54.146 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:54.146 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:54.146 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:29:54.146 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:53.833131+0000", 2024-09-14T06:29:54.146 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.5009999999999999e-06 2024-09-14T06:29:54.146 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:54.146 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:54.146 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:29:54.146 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:53.833138+0000", 2024-09-14T06:29:54.146 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.3999999999999997e-06 2024-09-14T06:29:54.146 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:54.146 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:54.146 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:29:54.146 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:53.833142+0000", 2024-09-14T06:29:54.146 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.6160000000000004e-06 2024-09-14T06:29:54.146 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:54.147 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:54.147 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:29:54.147 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:53.854389+0000", 2024-09-14T06:29:54.147 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.021246273999999999 2024-09-14T06:29:54.147 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:54.147 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:54.147 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:29:54.147 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:53.854462+0000", 2024-09-14T06:29:54.147 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.2953000000000004e-05 2024-09-14T06:29:54.147 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:54.147 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:54.147 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:29:54.147 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:53.854707+0000", 2024-09-14T06:29:54.147 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.000245317 2024-09-14T06:29:54.147 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:54.147 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:29:54.148 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:54.148 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:54.148 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:54.148 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:73956 3.a 3:57f2eddd:::rbd_data.112d4c988ca3.00000000000000bb:1d [read 98304~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-14T06:29:54.148 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:53.833149+0000", 2024-09-14T06:29:54.148 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 0.230242432, 2024-09-14T06:29:54.148 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.021918799999999999, 2024-09-14T06:29:54.148 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:29:54.148 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:29:54.148 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:29:54.148 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:29:54.148 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:29:54.148 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:54.148 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 73956 2024-09-14T06:29:54.148 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:54.149 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:29:54.149 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:54.149 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:29:54.149 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:53.833149+0000", 2024-09-14T06:29:54.149 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:54.149 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:54.149 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:54.149 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:29:54.149 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:53.833149+0000", 2024-09-14T06:29:54.149 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:54.149 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:54.149 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:54.149 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:29:54.149 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:53.833149+0000", 2024-09-14T06:29:54.150 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.0400000000000002e-07 2024-09-14T06:29:54.150 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:54.150 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:54.150 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:29:54.150 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:53.833152+0000", 2024-09-14T06:29:54.150 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.323e-06 2024-09-14T06:29:54.150 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:54.150 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:54.150 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:29:54.150 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:53.833158+0000", 2024-09-14T06:29:54.150 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.8599999999999998e-06 2024-09-14T06:29:54.150 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:54.150 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:54.150 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:29:54.150 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:53.833162+0000", 2024-09-14T06:29:54.151 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.707e-06 2024-09-14T06:29:54.151 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:54.151 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:54.151 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:29:54.151 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:53.854737+0000", 2024-09-14T06:29:54.151 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.021575014 2024-09-14T06:29:54.151 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:54.151 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:54.151 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:29:54.151 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:53.854814+0000", 2024-09-14T06:29:54.151 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.6972999999999996e-05 2024-09-14T06:29:54.151 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:54.151 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:54.151 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:29:54.151 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:53.855068+0000", 2024-09-14T06:29:54.152 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00025351900000000001 2024-09-14T06:29:54.152 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:54.152 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:29:54.152 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:54.152 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:54.152 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:29:54.152 INFO:teuthology.orchestra.run.smithi138.stdout:} 2024-09-14T06:29:54.153 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-14T06:29:54.197 INFO:tasks.ceph.osd.5.smithi138.stderr:2024-09-14T06:29:54.196+0000 7fe13746f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33618) UID: 0 2024-09-14T06:29:54.259 INFO:teuthology.orchestra.run.smithi138.stdout:{ 2024-09-14T06:29:54.259 INFO:teuthology.orchestra.run.smithi138.stdout: "ops": [], 2024-09-14T06:29:54.259 INFO:teuthology.orchestra.run.smithi138.stdout: "num_ops": 0 2024-09-14T06:29:54.260 INFO:teuthology.orchestra.run.smithi138.stdout:} 2024-09-14T06:29:54.264 INFO:tasks.ceph.osd.7.smithi138.stderr:2024-09-14T06:29:54.263+0000 7f3c6a21d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33623) UID: 0 2024-09-14T06:29:54.268 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-14T06:29:54.365 INFO:tasks.ceph.osd.4.smithi138.stderr:2024-09-14T06:29:54.364+0000 7f6c653c8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33615) UID: 0 2024-09-14T06:29:54.412 INFO:teuthology.orchestra.run.smithi138.stdout:{ 2024-09-14T06:29:54.412 INFO:teuthology.orchestra.run.smithi138.stdout: "ops": [], 2024-09-14T06:29:54.412 INFO:teuthology.orchestra.run.smithi138.stdout: "complaint_time": 30, 2024-09-14T06:29:54.412 INFO:teuthology.orchestra.run.smithi138.stdout: "num_blocked_ops": 0 2024-09-14T06:29:54.412 INFO:teuthology.orchestra.run.smithi138.stdout:} 2024-09-14T06:29:54.421 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-14T06:29:54.505 INFO:tasks.ceph.osd.4.smithi138.stderr:2024-09-14T06:29:54.504+0000 7f6c653c8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33615) UID: 0 2024-09-14T06:29:54.566 INFO:teuthology.orchestra.run.smithi138.stdout:{ 2024-09-14T06:29:54.566 INFO:teuthology.orchestra.run.smithi138.stdout: "size": 20, 2024-09-14T06:29:54.567 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 600, 2024-09-14T06:29:54.567 INFO:teuthology.orchestra.run.smithi138.stdout: "ops": [ 2024-09-14T06:29:54.567 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:54.567 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:53244 3.f 3:f71f51a7:::rbd_data.112dca2f6ae1.0000000000000015:17 [read 1863680~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:29:54.567 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:44.369399+0000", 2024-09-14T06:29:54.567 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 10.194697975, 2024-09-14T06:29:54.567 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047291189999999997, 2024-09-14T06:29:54.567 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:29:54.567 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:29:54.567 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:29:54.567 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:29:54.567 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:29:54.567 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:54.568 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 53244 2024-09-14T06:29:54.568 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:54.568 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:29:54.568 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:54.568 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:29:54.568 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369399+0000", 2024-09-14T06:29:54.568 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:54.568 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:54.568 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:54.568 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:29:54.568 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369399+0000", 2024-09-14T06:29:54.568 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:54.568 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:54.569 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:54.569 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:29:54.569 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369400+0000", 2024-09-14T06:29:54.569 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.0699999999999999e-07 2024-09-14T06:29:54.569 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:54.569 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:54.569 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:29:54.569 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369401+0000", 2024-09-14T06:29:54.569 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.4300000000000001e-06 2024-09-14T06:29:54.569 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:54.569 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:54.569 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:29:54.569 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369404+0000", 2024-09-14T06:29:54.570 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.061e-06 2024-09-14T06:29:54.570 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:54.570 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:54.570 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:29:54.570 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369407+0000", 2024-09-14T06:29:54.570 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.6139999999999999e-06 2024-09-14T06:29:54.570 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:54.570 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:54.570 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:29:54.570 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.416517+0000", 2024-09-14T06:29:54.570 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047110338000000002 2024-09-14T06:29:54.570 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:54.570 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:54.570 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:29:54.571 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.416616+0000", 2024-09-14T06:29:54.571 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.8651000000000001e-05 2024-09-14T06:29:54.571 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:54.571 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:54.571 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:29:54.571 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.416691+0000", 2024-09-14T06:29:54.571 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.4888999999999994e-05 2024-09-14T06:29:54.571 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:54.571 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:29:54.571 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:54.571 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:54.571 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:54.571 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:53245 3.f 3:f71f51a7:::rbd_data.112dca2f6ae1.0000000000000015:17 [read 1888256~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:29:54.571 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:44.369410+0000", 2024-09-14T06:29:54.572 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 10.194687687, 2024-09-14T06:29:54.572 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047399275999999997, 2024-09-14T06:29:54.572 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:29:54.572 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:29:54.572 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:29:54.572 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:29:54.572 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:29:54.572 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:54.572 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 53245 2024-09-14T06:29:54.572 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:54.572 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:29:54.572 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:54.572 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:29:54.572 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369410+0000", 2024-09-14T06:29:54.573 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:54.573 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:54.573 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:54.573 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:29:54.573 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369410+0000", 2024-09-14T06:29:54.573 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:54.573 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:54.573 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:54.573 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:29:54.573 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369410+0000", 2024-09-14T06:29:54.573 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.1899999999999999e-07 2024-09-14T06:29:54.574 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:54.574 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:54.574 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:29:54.574 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369411+0000", 2024-09-14T06:29:54.574 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.0300000000000001e-06 2024-09-14T06:29:54.574 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:54.574 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:54.574 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:29:54.574 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369414+0000", 2024-09-14T06:29:54.574 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.1099999999999999e-06 2024-09-14T06:29:54.574 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:54.574 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:54.574 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:29:54.574 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369417+0000", 2024-09-14T06:29:54.575 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.6189999999999998e-06 2024-09-14T06:29:54.575 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:54.575 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:54.575 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:29:54.575 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.416718+0000", 2024-09-14T06:29:54.575 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047301322999999999 2024-09-14T06:29:54.575 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:54.575 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:54.575 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:29:54.575 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.416759+0000", 2024-09-14T06:29:54.575 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.1009e-05 2024-09-14T06:29:54.575 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:54.575 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:54.576 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:29:54.576 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.416809+0000", 2024-09-14T06:29:54.576 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.9966000000000002e-05 2024-09-14T06:29:54.576 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:54.576 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:29:54.576 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:54.576 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:54.576 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:54.576 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:53246 3.f 3:f71f51a7:::rbd_data.112dca2f6ae1.0000000000000015:17 [read 1912832~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:29:54.577 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:44.369420+0000", 2024-09-14T06:29:54.577 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 10.194677743, 2024-09-14T06:29:54.577 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047529463000000001, 2024-09-14T06:29:54.577 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:29:54.577 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:29:54.577 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:29:54.577 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:29:54.578 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:29:54.578 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:54.578 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 53246 2024-09-14T06:29:54.578 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:54.578 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:29:54.578 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:54.578 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:29:54.578 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369420+0000", 2024-09-14T06:29:54.578 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:54.578 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:54.578 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:54.578 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:29:54.578 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369420+0000", 2024-09-14T06:29:54.578 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:54.579 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:54.579 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:54.579 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:29:54.579 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369420+0000", 2024-09-14T06:29:54.579 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.1299999999999999e-07 2024-09-14T06:29:54.579 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:54.579 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:54.579 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:29:54.579 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369425+0000", 2024-09-14T06:29:54.579 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.0130000000000001e-06 2024-09-14T06:29:54.579 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:54.579 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:54.579 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:29:54.579 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369428+0000", 2024-09-14T06:29:54.580 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.349e-06 2024-09-14T06:29:54.580 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:54.580 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:54.580 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:29:54.580 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369431+0000", 2024-09-14T06:29:54.580 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.7389999999999999e-06 2024-09-14T06:29:54.580 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:54.580 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:54.580 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:29:54.580 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.416834+0000", 2024-09-14T06:29:54.580 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047402669000000001 2024-09-14T06:29:54.580 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:54.580 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:54.580 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:29:54.581 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.416882+0000", 2024-09-14T06:29:54.581 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.8068999999999997e-05 2024-09-14T06:29:54.581 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:54.581 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:54.581 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:29:54.581 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.416949+0000", 2024-09-14T06:29:54.581 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.7410999999999995e-05 2024-09-14T06:29:54.581 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:54.581 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:29:54.581 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:54.582 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:54.582 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:54.582 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:53247 3.f 3:f71f51a7:::rbd_data.112dca2f6ae1.0000000000000015:17 [read 1937408~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:29:54.582 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:44.369434+0000", 2024-09-14T06:29:54.582 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 10.194662979, 2024-09-14T06:29:54.582 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047652264999999999, 2024-09-14T06:29:54.582 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:29:54.582 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:29:54.582 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:29:54.582 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:29:54.582 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:29:54.582 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:54.582 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 53247 2024-09-14T06:29:54.582 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:54.583 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:29:54.583 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:54.583 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:29:54.583 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369434+0000", 2024-09-14T06:29:54.583 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:54.583 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:54.583 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:54.583 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:29:54.583 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369434+0000", 2024-09-14T06:29:54.583 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:54.583 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:54.583 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:54.583 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:29:54.583 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369435+0000", 2024-09-14T06:29:54.584 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.2100000000000001e-07 2024-09-14T06:29:54.584 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:54.584 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:54.584 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:29:54.584 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369436+0000", 2024-09-14T06:29:54.584 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.122e-06 2024-09-14T06:29:54.584 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:54.584 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:54.584 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:29:54.584 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369439+0000", 2024-09-14T06:29:54.584 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.0400000000000001e-06 2024-09-14T06:29:54.584 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:54.584 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:54.585 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:29:54.585 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369441+0000", 2024-09-14T06:29:54.585 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.728e-06 2024-09-14T06:29:54.585 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:54.585 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:54.585 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:29:54.585 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.416977+0000", 2024-09-14T06:29:54.585 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047535344 2024-09-14T06:29:54.585 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:54.585 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:54.585 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:29:54.585 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.417029+0000", 2024-09-14T06:29:54.585 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.2037999999999999e-05 2024-09-14T06:29:54.585 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:54.586 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:54.586 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:29:54.586 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.417087+0000", 2024-09-14T06:29:54.586 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.7772000000000002e-05 2024-09-14T06:29:54.586 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:54.586 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:29:54.586 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:54.586 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:54.586 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:54.586 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:53248 3.f 3:f71f51a7:::rbd_data.112dca2f6ae1.0000000000000015:17 [read 1961984~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:29:54.586 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:44.369445+0000", 2024-09-14T06:29:54.586 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 10.194652426999999, 2024-09-14T06:29:54.586 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047798489, 2024-09-14T06:29:54.586 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:29:54.587 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:29:54.587 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:29:54.587 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:29:54.587 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:29:54.587 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:54.587 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 53248 2024-09-14T06:29:54.587 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:54.587 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:29:54.587 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:54.587 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:29:54.588 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369445+0000", 2024-09-14T06:29:54.588 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:54.588 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:54.588 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:54.588 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:29:54.589 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369445+0000", 2024-09-14T06:29:54.589 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:54.589 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:54.589 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:54.589 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:29:54.589 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369445+0000", 2024-09-14T06:29:54.589 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.0699999999999999e-07 2024-09-14T06:29:54.589 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:54.590 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:54.590 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:29:54.590 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369446+0000", 2024-09-14T06:29:54.590 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.1000000000000001e-06 2024-09-14T06:29:54.590 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:54.590 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:54.590 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:29:54.590 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369449+0000", 2024-09-14T06:29:54.590 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.1039999999999998e-06 2024-09-14T06:29:54.590 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:54.591 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:54.591 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:29:54.591 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369452+0000", 2024-09-14T06:29:54.591 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.6199999999999999e-06 2024-09-14T06:29:54.591 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:54.591 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:54.591 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:29:54.591 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.417114+0000", 2024-09-14T06:29:54.591 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047661901999999999 2024-09-14T06:29:54.591 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:54.591 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:54.591 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:29:54.591 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.417181+0000", 2024-09-14T06:29:54.591 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.7385999999999995e-05 2024-09-14T06:29:54.592 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:54.592 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:54.592 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:29:54.592 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.417243+0000", 2024-09-14T06:29:54.592 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.2169999999999996e-05 2024-09-14T06:29:54.592 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:54.592 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:29:54.592 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:54.592 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:54.592 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:54.592 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:53249 3.f 3:f71f51a7:::rbd_data.112dca2f6ae1.0000000000000015:17 [read 1986560~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:29:54.592 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:44.369455+0000", 2024-09-14T06:29:54.592 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 10.194642116000001, 2024-09-14T06:29:54.593 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047890493999999999, 2024-09-14T06:29:54.593 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:29:54.593 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:29:54.593 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:29:54.593 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:29:54.593 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:29:54.593 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:54.593 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 53249 2024-09-14T06:29:54.593 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:54.593 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:29:54.593 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:54.593 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:29:54.593 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369455+0000", 2024-09-14T06:29:54.594 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:54.594 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:54.594 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:54.594 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:29:54.594 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369455+0000", 2024-09-14T06:29:54.594 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:54.594 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:54.594 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:54.594 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:29:54.594 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369455+0000", 2024-09-14T06:29:54.594 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.0800000000000001e-07 2024-09-14T06:29:54.594 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:54.594 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:54.594 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:29:54.594 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369456+0000", 2024-09-14T06:29:54.595 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.0729999999999999e-06 2024-09-14T06:29:54.595 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:54.595 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:54.595 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:29:54.595 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369460+0000", 2024-09-14T06:29:54.595 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.027e-06 2024-09-14T06:29:54.595 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:54.595 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:54.595 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:29:54.595 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369462+0000", 2024-09-14T06:29:54.595 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.5320000000000001e-06 2024-09-14T06:29:54.595 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:54.596 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:54.596 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:29:54.596 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.417257+0000", 2024-09-14T06:29:54.596 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.04779489 2024-09-14T06:29:54.596 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:54.596 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:54.596 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:29:54.596 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.417296+0000", 2024-09-14T06:29:54.596 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.8658000000000002e-05 2024-09-14T06:29:54.596 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:54.596 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:54.597 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:29:54.597 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.417346+0000", 2024-09-14T06:29:54.597 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.0105999999999998e-05 2024-09-14T06:29:54.597 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:54.597 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:29:54.597 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:54.597 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:54.597 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:54.597 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:53250 3.f 3:f71f51a7:::rbd_data.112dca2f6ae1.0000000000000015:17 [read 2011136~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:29:54.597 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:44.369465+0000", 2024-09-14T06:29:54.597 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 10.194632146, 2024-09-14T06:29:54.597 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.048032230000000002, 2024-09-14T06:29:54.597 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:29:54.597 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:29:54.598 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:29:54.598 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:29:54.598 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:29:54.598 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:54.598 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 53250 2024-09-14T06:29:54.598 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:54.598 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:29:54.598 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:54.598 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:29:54.598 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369465+0000", 2024-09-14T06:29:54.598 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:54.598 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:54.598 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:54.598 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:29:54.599 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369465+0000", 2024-09-14T06:29:54.599 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:54.599 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:54.599 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:54.599 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:29:54.599 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369465+0000", 2024-09-14T06:29:54.599 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.2100000000000001e-07 2024-09-14T06:29:54.599 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:54.599 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:54.599 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:29:54.599 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369466+0000", 2024-09-14T06:29:54.599 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.0640000000000001e-06 2024-09-14T06:29:54.599 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:54.600 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:54.600 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:29:54.600 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369469+0000", 2024-09-14T06:29:54.600 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.0170000000000001e-06 2024-09-14T06:29:54.600 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:54.600 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:54.601 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:29:54.601 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369472+0000", 2024-09-14T06:29:54.601 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.6189999999999998e-06 2024-09-14T06:29:54.601 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:54.601 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:54.602 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:29:54.602 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.417383+0000", 2024-09-14T06:29:54.602 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047911092000000002 2024-09-14T06:29:54.602 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:54.602 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:54.602 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:29:54.602 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.417430+0000", 2024-09-14T06:29:54.603 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.6898999999999999e-05 2024-09-14T06:29:54.603 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:54.603 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:54.603 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:29:54.603 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.417497+0000", 2024-09-14T06:29:54.603 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.7317999999999995e-05 2024-09-14T06:29:54.603 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:54.604 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:29:54.604 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:54.604 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:54.604 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:54.604 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:53251 3.f 3:f71f51a7:::rbd_data.112dca2f6ae1.0000000000000015:17 [read 2035712~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:29:54.604 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:44.369475+0000", 2024-09-14T06:29:54.604 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 10.194622150000001, 2024-09-14T06:29:54.605 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.048154542000000002, 2024-09-14T06:29:54.605 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:29:54.605 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:29:54.605 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:29:54.605 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:29:54.605 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:29:54.606 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:54.606 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 53251 2024-09-14T06:29:54.606 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:54.606 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:29:54.606 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:54.606 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:29:54.606 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369475+0000", 2024-09-14T06:29:54.606 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:54.607 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:54.607 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:54.607 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:29:54.607 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369475+0000", 2024-09-14T06:29:54.607 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:54.607 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:54.607 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:54.607 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:29:54.608 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369475+0000", 2024-09-14T06:29:54.608 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.0100000000000001e-07 2024-09-14T06:29:54.608 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:54.608 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:54.608 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:29:54.608 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369476+0000", 2024-09-14T06:29:54.608 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.0580000000000001e-06 2024-09-14T06:29:54.609 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:54.609 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:54.609 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:29:54.609 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369479+0000", 2024-09-14T06:29:54.609 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.9950000000000001e-06 2024-09-14T06:29:54.609 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:54.610 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:54.610 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:29:54.610 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369482+0000", 2024-09-14T06:29:54.610 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.6340000000000001e-06 2024-09-14T06:29:54.610 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:54.611 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:54.611 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:29:54.611 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.417530+0000", 2024-09-14T06:29:54.611 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.048047535000000002 2024-09-14T06:29:54.611 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:54.611 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:54.611 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:29:54.612 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.417574+0000", 2024-09-14T06:29:54.612 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.4255999999999998e-05 2024-09-14T06:29:54.612 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:54.612 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:54.612 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:29:54.612 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.417630+0000", 2024-09-14T06:29:54.612 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.5862999999999997e-05 2024-09-14T06:29:54.613 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:54.613 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:29:54.613 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:54.613 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:54.613 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:54.613 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:53252 3.f 3:f71f51a7:::rbd_data.112dca2f6ae1.0000000000000015:17 [read 2060288~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:29:54.613 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:44.369485+0000", 2024-09-14T06:29:54.613 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 10.194612301999999, 2024-09-14T06:29:54.614 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.048254116, 2024-09-14T06:29:54.614 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:29:54.614 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:29:54.614 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:29:54.614 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:29:54.614 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:29:54.614 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:54.615 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 53252 2024-09-14T06:29:54.615 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:54.615 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:29:54.615 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:54.615 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:29:54.615 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369485+0000", 2024-09-14T06:29:54.616 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:54.616 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:54.616 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:54.616 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:29:54.616 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369485+0000", 2024-09-14T06:29:54.616 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:54.617 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:54.617 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:54.617 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:29:54.617 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369485+0000", 2024-09-14T06:29:54.617 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.1e-07 2024-09-14T06:29:54.618 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:54.618 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:54.618 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:29:54.618 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369487+0000", 2024-09-14T06:29:54.619 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.42e-06 2024-09-14T06:29:54.619 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:54.619 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:54.619 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:29:54.619 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369490+0000", 2024-09-14T06:29:54.619 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.9639999999999999e-06 2024-09-14T06:29:54.619 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:54.620 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:54.620 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:29:54.620 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369492+0000", 2024-09-14T06:29:54.620 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.5679999999999998e-06 2024-09-14T06:29:54.620 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:54.620 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:54.620 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:29:54.620 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.417645+0000", 2024-09-14T06:29:54.620 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.048152753 2024-09-14T06:29:54.620 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:54.620 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:54.620 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:29:54.620 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.417687+0000", 2024-09-14T06:29:54.621 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.1900999999999997e-05 2024-09-14T06:29:54.621 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:54.621 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:54.621 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:29:54.621 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.417739+0000", 2024-09-14T06:29:54.621 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.2299999999999997e-05 2024-09-14T06:29:54.621 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:54.621 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:29:54.621 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:54.621 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:54.621 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:54.621 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:53253 3.f 3:f71f51a7:::rbd_data.112dca2f6ae1.0000000000000015:17 [read 2084864~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:29:54.621 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:44.369495+0000", 2024-09-14T06:29:54.622 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 10.194602178, 2024-09-14T06:29:54.622 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.048366785000000002, 2024-09-14T06:29:54.622 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:29:54.622 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:29:54.622 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:29:54.622 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:29:54.622 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:29:54.622 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:54.622 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 53253 2024-09-14T06:29:54.622 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:54.622 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:29:54.622 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:54.622 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:29:54.623 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369495+0000", 2024-09-14T06:29:54.623 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:54.623 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:54.623 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:54.623 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:29:54.623 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369495+0000", 2024-09-14T06:29:54.623 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:54.623 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:54.623 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:54.623 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:29:54.623 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370388+0000", 2024-09-14T06:29:54.623 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00089287999999999998 2024-09-14T06:29:54.623 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:54.623 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:54.624 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:29:54.624 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370392+0000", 2024-09-14T06:29:54.624 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.2130000000000002e-06 2024-09-14T06:29:54.624 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:54.624 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:54.624 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:29:54.624 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370401+0000", 2024-09-14T06:29:54.624 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.7649999999999999e-06 2024-09-14T06:29:54.624 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:54.624 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:54.624 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:29:54.624 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370406+0000", 2024-09-14T06:29:54.624 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.0159999999999999e-06 2024-09-14T06:29:54.625 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:54.625 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:54.625 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:29:54.625 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.417764+0000", 2024-09-14T06:29:54.625 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047358348000000001 2024-09-14T06:29:54.625 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:54.625 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:54.625 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:29:54.625 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.417806+0000", 2024-09-14T06:29:54.625 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.1149999999999997e-05 2024-09-14T06:29:54.625 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:54.626 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:54.626 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:29:54.626 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.417862+0000", 2024-09-14T06:29:54.626 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.6412999999999997e-05 2024-09-14T06:29:54.626 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:54.626 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:29:54.626 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:54.626 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:54.626 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:54.626 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:53340 3.f 3:f71f51a7:::rbd_data.112dca2f6ae1.0000000000000015:17 [read 1056768~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:29:54.626 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:44.370421+0000", 2024-09-14T06:29:54.626 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 10.193676298, 2024-09-14T06:29:54.626 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047566313999999998, 2024-09-14T06:29:54.626 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:29:54.627 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:29:54.627 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:29:54.627 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:29:54.627 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:29:54.627 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:54.627 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 53340 2024-09-14T06:29:54.627 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:54.627 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:29:54.627 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:54.627 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:29:54.627 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370421+0000", 2024-09-14T06:29:54.627 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:54.628 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:54.628 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:54.628 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:29:54.628 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370421+0000", 2024-09-14T06:29:54.628 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:54.628 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:54.628 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:54.628 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:29:54.629 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370422+0000", 2024-09-14T06:29:54.629 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.3799999999999996e-07 2024-09-14T06:29:54.629 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:54.629 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:54.629 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:29:54.629 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370424+0000", 2024-09-14T06:29:54.629 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.6919999999999998e-06 2024-09-14T06:29:54.629 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:54.629 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:54.629 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:29:54.629 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370432+0000", 2024-09-14T06:29:54.629 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.2439999999999996e-06 2024-09-14T06:29:54.629 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:54.630 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:54.630 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:29:54.630 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370437+0000", 2024-09-14T06:29:54.630 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.5389999999999998e-06 2024-09-14T06:29:54.630 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:54.630 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:54.630 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:29:54.630 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.417888+0000", 2024-09-14T06:29:54.630 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047450432000000001 2024-09-14T06:29:54.630 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:54.630 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:54.630 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:29:54.630 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.417935+0000", 2024-09-14T06:29:54.631 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.7454999999999999e-05 2024-09-14T06:29:54.631 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:54.631 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:54.631 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:29:54.631 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.417987+0000", 2024-09-14T06:29:54.631 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.2213999999999998e-05 2024-09-14T06:29:54.631 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:54.631 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:29:54.631 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:54.631 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:54.631 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:54.631 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:53341 3.f 3:f71f51a7:::rbd_data.112dca2f6ae1.0000000000000015:17 [read 1081344~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:29:54.631 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:44.370446+0000", 2024-09-14T06:29:54.631 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 10.193651024999999, 2024-09-14T06:29:54.632 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047665723, 2024-09-14T06:29:54.632 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:29:54.632 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:29:54.632 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:29:54.632 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:29:54.632 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:29:54.632 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:54.632 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 53341 2024-09-14T06:29:54.632 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:54.632 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:29:54.633 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:54.633 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:29:54.633 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370446+0000", 2024-09-14T06:29:54.633 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:54.633 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:54.633 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:54.633 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:29:54.633 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370446+0000", 2024-09-14T06:29:54.633 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:54.633 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:54.633 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:54.633 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:29:54.633 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370447+0000", 2024-09-14T06:29:54.633 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.6100000000000002e-07 2024-09-14T06:29:54.634 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:54.634 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:54.634 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:29:54.634 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370448+0000", 2024-09-14T06:29:54.634 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.7370000000000001e-06 2024-09-14T06:29:54.634 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:54.634 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:54.634 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:29:54.634 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370462+0000", 2024-09-14T06:29:54.634 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.3227000000000001e-05 2024-09-14T06:29:54.634 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:54.634 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:54.635 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:29:54.635 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370467+0000", 2024-09-14T06:29:54.635 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.6540000000000004e-06 2024-09-14T06:29:54.635 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:54.635 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:54.635 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:29:54.635 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.418013+0000", 2024-09-14T06:29:54.635 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047545325999999999 2024-09-14T06:29:54.635 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:54.635 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:54.635 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:29:54.635 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.418054+0000", 2024-09-14T06:29:54.635 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.1696999999999997e-05 2024-09-14T06:29:54.635 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:54.636 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:54.636 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:29:54.636 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.418112+0000", 2024-09-14T06:29:54.636 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.7720999999999998e-05 2024-09-14T06:29:54.636 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:54.636 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:29:54.636 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:54.636 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:54.636 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:54.636 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:53342 3.f 3:f71f51a7:::rbd_data.112dca2f6ae1.0000000000000015:17 [read 1105920~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:29:54.636 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:44.370475+0000", 2024-09-14T06:29:54.636 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 10.193621964, 2024-09-14T06:29:54.636 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047748848000000003, 2024-09-14T06:29:54.636 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:29:54.637 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:29:54.637 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:29:54.637 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:29:54.637 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:29:54.637 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:54.637 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 53342 2024-09-14T06:29:54.637 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:54.637 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:29:54.637 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:54.637 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:29:54.637 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370475+0000", 2024-09-14T06:29:54.637 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:54.637 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:54.637 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:54.638 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:29:54.638 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370475+0000", 2024-09-14T06:29:54.638 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:54.638 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:54.638 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:54.638 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:29:54.638 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370476+0000", 2024-09-14T06:29:54.638 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.4700000000000002e-07 2024-09-14T06:29:54.638 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:54.638 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:54.638 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:29:54.639 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370478+0000", 2024-09-14T06:29:54.639 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.5890000000000001e-06 2024-09-14T06:29:54.639 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:54.639 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:54.639 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:29:54.639 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370486+0000", 2024-09-14T06:29:54.639 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.3470000000000001e-06 2024-09-14T06:29:54.639 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:54.639 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:54.639 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:29:54.639 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370491+0000", 2024-09-14T06:29:54.639 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.5720000000000002e-06 2024-09-14T06:29:54.639 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:54.640 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:54.640 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:29:54.640 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.418135+0000", 2024-09-14T06:29:54.640 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047644094999999997 2024-09-14T06:29:54.640 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:54.640 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:54.640 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:29:54.640 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.418174+0000", 2024-09-14T06:29:54.640 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.8204000000000002e-05 2024-09-14T06:29:54.640 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:54.640 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:54.640 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:29:54.640 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.418224+0000", 2024-09-14T06:29:54.641 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.0593999999999997e-05 2024-09-14T06:29:54.641 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:54.641 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:29:54.641 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:54.641 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:54.641 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:54.641 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:53343 3.f 3:f71f51a7:::rbd_data.112dca2f6ae1.0000000000000015:17 [read 1130496~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:29:54.641 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:44.370498+0000", 2024-09-14T06:29:54.641 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 10.193599087999999, 2024-09-14T06:29:54.641 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047841574999999997, 2024-09-14T06:29:54.641 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:29:54.641 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:29:54.641 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:29:54.641 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:29:54.642 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:29:54.642 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:54.642 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 53343 2024-09-14T06:29:54.642 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:54.642 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:29:54.642 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:54.642 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:29:54.642 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370498+0000", 2024-09-14T06:29:54.642 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:54.642 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:54.642 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:54.642 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:29:54.642 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370498+0000", 2024-09-14T06:29:54.643 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:54.643 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:54.643 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:54.643 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:29:54.643 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370499+0000", 2024-09-14T06:29:54.643 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.9999999999999998e-07 2024-09-14T06:29:54.643 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:54.643 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:54.644 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:29:54.644 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370501+0000", 2024-09-14T06:29:54.644 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.0310000000000001e-06 2024-09-14T06:29:54.644 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:54.644 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:54.644 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:29:54.644 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370507+0000", 2024-09-14T06:29:54.644 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.4169999999999997e-06 2024-09-14T06:29:54.644 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:54.644 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:54.644 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:29:54.644 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370516+0000", 2024-09-14T06:29:54.644 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.7190000000000007e-06 2024-09-14T06:29:54.645 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:54.645 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:54.645 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:29:54.645 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.418247+0000", 2024-09-14T06:29:54.645 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047731190999999999 2024-09-14T06:29:54.645 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:54.645 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:54.645 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:29:54.645 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.418286+0000", 2024-09-14T06:29:54.645 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.8988e-05 2024-09-14T06:29:54.645 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:54.645 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:54.645 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:29:54.646 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.418340+0000", 2024-09-14T06:29:54.646 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.3829000000000002e-05 2024-09-14T06:29:54.646 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:54.646 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:29:54.646 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:54.646 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:54.646 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:54.646 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:53344 3.f 3:f71f51a7:::rbd_data.112dca2f6ae1.0000000000000015:17 [read 1155072~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:29:54.646 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:44.370524+0000", 2024-09-14T06:29:54.646 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 10.193573318, 2024-09-14T06:29:54.646 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047972262000000002, 2024-09-14T06:29:54.646 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:29:54.646 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:29:54.646 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:29:54.647 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:29:54.647 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:29:54.647 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:54.647 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 53344 2024-09-14T06:29:54.647 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:54.647 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:29:54.647 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:54.647 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:29:54.647 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370524+0000", 2024-09-14T06:29:54.647 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:54.647 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:54.647 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:54.647 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:29:54.647 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370524+0000", 2024-09-14T06:29:54.648 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:54.648 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:54.648 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:54.648 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:29:54.648 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370618+0000", 2024-09-14T06:29:54.648 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.4425999999999993e-05 2024-09-14T06:29:54.648 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:54.648 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:54.648 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:29:54.648 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370622+0000", 2024-09-14T06:29:54.648 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.3519999999999998e-06 2024-09-14T06:29:54.648 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:54.648 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:54.649 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:29:54.649 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370631+0000", 2024-09-14T06:29:54.649 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.9299999999999992e-06 2024-09-14T06:29:54.649 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:54.649 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:54.649 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:29:54.649 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370637+0000", 2024-09-14T06:29:54.649 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.3269999999999997e-06 2024-09-14T06:29:54.649 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:54.649 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:54.649 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:29:54.649 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.418372+0000", 2024-09-14T06:29:54.649 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047734764999999998 2024-09-14T06:29:54.650 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:54.650 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:54.650 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:29:54.650 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.418431+0000", 2024-09-14T06:29:54.650 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.8696999999999997e-05 2024-09-14T06:29:54.650 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:54.650 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:54.650 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:29:54.650 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.418496+0000", 2024-09-14T06:29:54.650 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.5765000000000005e-05 2024-09-14T06:29:54.650 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:54.650 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:29:54.650 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:54.650 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:54.651 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:54.651 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:53345 3.f 3:f71f51a7:::rbd_data.112dca2f6ae1.0000000000000015:17 [read 1179648~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:29:54.651 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:44.370647+0000", 2024-09-14T06:29:54.651 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 10.193449851, 2024-09-14T06:29:54.651 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047953775999999997, 2024-09-14T06:29:54.651 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:29:54.651 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:29:54.651 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:29:54.651 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:29:54.651 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:29:54.651 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:54.651 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 53345 2024-09-14T06:29:54.651 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:54.651 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:29:54.652 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:54.652 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:29:54.652 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370647+0000", 2024-09-14T06:29:54.652 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:54.652 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:54.652 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:54.652 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:29:54.652 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370647+0000", 2024-09-14T06:29:54.652 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:54.653 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:54.653 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:54.653 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:29:54.653 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.372346+0000", 2024-09-14T06:29:54.653 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.001698284 2024-09-14T06:29:54.653 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:54.653 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:54.653 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:29:54.653 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.372351+0000", 2024-09-14T06:29:54.653 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.8890000000000001e-06 2024-09-14T06:29:54.654 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:54.654 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:54.654 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:29:54.654 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.372362+0000", 2024-09-14T06:29:54.654 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.0941000000000001e-05 2024-09-14T06:29:54.654 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:54.654 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:54.654 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:29:54.654 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.372369+0000", 2024-09-14T06:29:54.654 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.362e-06 2024-09-14T06:29:54.654 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:54.654 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:54.655 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:29:54.655 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.418510+0000", 2024-09-14T06:29:54.655 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046140914999999998 2024-09-14T06:29:54.655 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:54.655 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:54.655 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:29:54.655 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.418549+0000", 2024-09-14T06:29:54.655 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.8717999999999998e-05 2024-09-14T06:29:54.655 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:54.655 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:54.655 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:29:54.655 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.418601+0000", 2024-09-14T06:29:54.656 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.2667000000000002e-05 2024-09-14T06:29:54.656 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:54.656 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:29:54.656 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:54.656 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:54.656 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:54.656 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:53351 3.f 3:f71f51a7:::rbd_data.112dca2f6ae1.0000000000000015:17 [read 1327104~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:29:54.656 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:44.372499+0000", 2024-09-14T06:29:54.656 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 10.191598730000001, 2024-09-14T06:29:54.656 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047140596, 2024-09-14T06:29:54.656 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:29:54.656 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:29:54.656 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:29:54.657 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:29:54.657 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:29:54.657 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:54.657 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 53351 2024-09-14T06:29:54.657 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:54.657 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:29:54.657 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:54.657 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:29:54.657 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.372499+0000", 2024-09-14T06:29:54.657 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:54.657 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:54.657 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:54.657 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:29:54.658 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.372499+0000", 2024-09-14T06:29:54.658 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:54.658 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:54.658 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:54.658 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:29:54.658 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.372499+0000", 2024-09-14T06:29:54.658 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.3000000000000001e-07 2024-09-14T06:29:54.658 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:54.658 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:54.658 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:29:54.658 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.372501+0000", 2024-09-14T06:29:54.658 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.0820000000000001e-06 2024-09-14T06:29:54.658 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:54.658 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:54.659 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:29:54.659 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.372513+0000", 2024-09-14T06:29:54.659 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.1610999999999999e-05 2024-09-14T06:29:54.659 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:54.659 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:54.659 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:29:54.659 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.372518+0000", 2024-09-14T06:29:54.659 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.327e-06 2024-09-14T06:29:54.659 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:54.659 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:54.659 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:29:54.659 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.419530+0000", 2024-09-14T06:29:54.659 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047011798 2024-09-14T06:29:54.659 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:54.660 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:54.660 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:29:54.660 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.419578+0000", 2024-09-14T06:29:54.660 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.8446999999999999e-05 2024-09-14T06:29:54.660 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:54.660 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:54.660 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:29:54.660 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.419639+0000", 2024-09-14T06:29:54.660 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.0900999999999998e-05 2024-09-14T06:29:54.660 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:54.660 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:29:54.660 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:54.660 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:54.661 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:54.661 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:53352 3.f 3:f71f51a7:::rbd_data.112dca2f6ae1.0000000000000015:17 [read 1351680~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:29:54.661 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:44.372525+0000", 2024-09-14T06:29:54.661 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 10.191572816000001, 2024-09-14T06:29:54.661 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047259847000000001, 2024-09-14T06:29:54.661 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:29:54.661 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:29:54.661 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:29:54.661 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:29:54.661 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:29:54.661 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:54.661 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 53352 2024-09-14T06:29:54.661 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:54.662 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:29:54.662 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:54.662 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:29:54.662 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.372525+0000", 2024-09-14T06:29:54.662 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:54.662 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:54.662 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:54.662 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:29:54.662 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.372525+0000", 2024-09-14T06:29:54.662 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:54.662 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:54.662 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:54.662 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:29:54.663 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.372525+0000", 2024-09-14T06:29:54.663 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.0600000000000001e-07 2024-09-14T06:29:54.663 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:54.663 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:54.663 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:29:54.663 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.372527+0000", 2024-09-14T06:29:54.663 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.0499999999999999e-06 2024-09-14T06:29:54.663 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:54.663 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:54.663 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:29:54.663 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.372533+0000", 2024-09-14T06:29:54.663 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.4320000000000004e-06 2024-09-14T06:29:54.663 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:54.664 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:54.664 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:29:54.664 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.372538+0000", 2024-09-14T06:29:54.664 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.9130000000000002e-06 2024-09-14T06:29:54.664 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:54.664 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:54.664 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:29:54.664 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.419667+0000", 2024-09-14T06:29:54.664 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047128976000000003 2024-09-14T06:29:54.664 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:54.664 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:54.664 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:29:54.664 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.419715+0000", 2024-09-14T06:29:54.664 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.7868999999999999e-05 2024-09-14T06:29:54.665 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:54.665 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:54.665 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:29:54.665 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.419784+0000", 2024-09-14T06:29:54.665 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.9201000000000004e-05 2024-09-14T06:29:54.665 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:54.665 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:29:54.665 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:54.665 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:54.665 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:54.666 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:53353 3.f 3:f71f51a7:::rbd_data.112dca2f6ae1.0000000000000015:17 [read 1376256~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:29:54.666 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:44.372545+0000", 2024-09-14T06:29:54.666 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 10.191552776, 2024-09-14T06:29:54.666 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047364232999999999, 2024-09-14T06:29:54.666 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:29:54.666 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:29:54.666 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:29:54.666 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:29:54.666 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:29:54.666 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:54.666 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 53353 2024-09-14T06:29:54.666 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:54.666 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:29:54.666 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:54.667 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:29:54.667 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.372545+0000", 2024-09-14T06:29:54.667 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:54.667 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:54.667 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:54.667 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:29:54.667 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.372545+0000", 2024-09-14T06:29:54.667 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:54.667 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:54.667 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:54.667 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:29:54.667 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.372545+0000", 2024-09-14T06:29:54.667 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.27e-07 2024-09-14T06:29:54.667 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:54.667 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:54.668 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:29:54.668 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.372547+0000", 2024-09-14T06:29:54.668 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.0600000000000002e-06 2024-09-14T06:29:54.668 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:54.668 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:54.668 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:29:54.668 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.372553+0000", 2024-09-14T06:29:54.668 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.7980000000000002e-06 2024-09-14T06:29:54.668 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:54.668 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:54.668 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:29:54.668 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.372557+0000", 2024-09-14T06:29:54.668 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.7389999999999999e-06 2024-09-14T06:29:54.669 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:54.669 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:54.669 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:29:54.669 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.419811+0000", 2024-09-14T06:29:54.669 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047253310999999999 2024-09-14T06:29:54.669 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:54.669 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:54.669 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:29:54.669 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.419854+0000", 2024-09-14T06:29:54.669 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.3448000000000001e-05 2024-09-14T06:29:54.669 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:54.669 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:54.669 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:29:54.670 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.419909+0000", 2024-09-14T06:29:54.670 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.4549999999999998e-05 2024-09-14T06:29:54.670 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:54.670 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:29:54.670 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:54.670 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:54.670 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:54.670 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:53354 3.f 3:f71f51a7:::rbd_data.112dca2f6ae1.0000000000000015:17 [read 1400832~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:29:54.670 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:44.372563+0000", 2024-09-14T06:29:54.670 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 10.191534339, 2024-09-14T06:29:54.670 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047482233999999998, 2024-09-14T06:29:54.670 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:29:54.670 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:29:54.671 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:29:54.671 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:29:54.671 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:29:54.671 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:54.671 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 53354 2024-09-14T06:29:54.671 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:54.671 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:29:54.671 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:54.671 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:29:54.671 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.372563+0000", 2024-09-14T06:29:54.671 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:54.671 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:54.672 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:54.672 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:29:54.672 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.372563+0000", 2024-09-14T06:29:54.672 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:54.672 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:54.672 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:54.672 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:29:54.672 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.375009+0000", 2024-09-14T06:29:54.672 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.0024457680000000001 2024-09-14T06:29:54.672 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:54.672 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:54.672 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:29:54.672 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.375014+0000", 2024-09-14T06:29:54.673 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.0599999999999998e-06 2024-09-14T06:29:54.673 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:54.673 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:54.673 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:29:54.673 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.375026+0000", 2024-09-14T06:29:54.673 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.1783999999999999e-05 2024-09-14T06:29:54.673 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:54.673 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:54.673 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:29:54.673 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.375035+0000", 2024-09-14T06:29:54.673 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.7799999999999995e-06 2024-09-14T06:29:54.673 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:54.673 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:54.674 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:29:54.674 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.419942+0000", 2024-09-14T06:29:54.674 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.044906982999999998 2024-09-14T06:29:54.674 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:54.674 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:54.674 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:29:54.674 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.419989+0000", 2024-09-14T06:29:54.674 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.7104999999999997e-05 2024-09-14T06:29:54.674 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:54.674 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:54.674 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:29:54.674 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.420045+0000", 2024-09-14T06:29:54.674 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.5754000000000002e-05 2024-09-14T06:29:54.674 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:54.674 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:29:54.675 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:54.675 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:54.675 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:29:54.675 INFO:teuthology.orchestra.run.smithi138.stdout:} 2024-09-14T06:29:54.676 INFO:tasks.ceph.osd.4.smithi138.stderr:2024-09-14T06:29:54.675+0000 7f6c653c8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33615) UID: 0 2024-09-14T06:29:54.677 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-14T06:29:54.775 INFO:teuthology.orchestra.run.smithi094.stderr:noscrub is unset 2024-09-14T06:29:54.776 INFO:tasks.ceph.osd.3.smithi094.stderr:2024-09-14T06:29:54.775+0000 7feeeafe8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33655) UID: 0 2024-09-14T06:29:54.785 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-14T06:29:54.785 INFO:teuthology.orchestra.run.smithi094.stdout: "ops": [], 2024-09-14T06:29:54.785 INFO:teuthology.orchestra.run.smithi094.stdout: "num_ops": 0 2024-09-14T06:29:54.785 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-14T06:29:54.794 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-14T06:29:54.878 INFO:tasks.ceph.osd.3.smithi094.stderr:2024-09-14T06:29:54.877+0000 7feeeafe8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33655) UID: 0 2024-09-14T06:29:54.927 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-14T06:29:54.937 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-14T06:29:54.937 INFO:teuthology.orchestra.run.smithi094.stdout: "ops": [], 2024-09-14T06:29:54.937 INFO:teuthology.orchestra.run.smithi094.stdout: "complaint_time": 30, 2024-09-14T06:29:54.938 INFO:teuthology.orchestra.run.smithi094.stdout: "num_blocked_ops": 0 2024-09-14T06:29:54.938 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-14T06:29:54.946 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-14T06:29:54.978 INFO:tasks.ceph.osd.5.smithi138.stderr:2024-09-14T06:29:54.977+0000 7fe13746f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33618) UID: 0 2024-09-14T06:29:55.065 INFO:teuthology.orchestra.run.smithi094.stdout:osd.0: {} 2024-09-14T06:29:55.065 INFO:teuthology.orchestra.run.smithi094.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-14T06:29:55.079 INFO:tasks.ceph.osd.0.smithi094.stderr:2024-09-14T06:29:55.078+0000 7f07c9333640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33636) UID: 0 2024-09-14T06:29:55.084 INFO:teuthology.orchestra.run.smithi094.stdout:osd.1: {} 2024-09-14T06:29:55.085 INFO:teuthology.orchestra.run.smithi094.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-14T06:29:55.100 INFO:teuthology.orchestra.run.smithi094.stdout:osd.2: {} 2024-09-14T06:29:55.100 INFO:teuthology.orchestra.run.smithi094.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-14T06:29:55.106 INFO:teuthology.orchestra.run.smithi094.stdout:ERROR: (22) Invalid argument 2024-09-14T06:29:55.106 INFO:teuthology.orchestra.run.smithi094.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:29:55.115 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-14T06:29:55.117 INFO:teuthology.orchestra.run.smithi094.stdout:osd.3: {} 2024-09-14T06:29:55.117 INFO:teuthology.orchestra.run.smithi094.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-14T06:29:55.137 INFO:teuthology.orchestra.run.smithi094.stdout:osd.4: {} 2024-09-14T06:29:55.137 INFO:teuthology.orchestra.run.smithi094.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-14T06:29:55.153 INFO:teuthology.orchestra.run.smithi094.stdout:osd.5: {} 2024-09-14T06:29:55.154 INFO:teuthology.orchestra.run.smithi094.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-14T06:29:55.172 INFO:teuthology.orchestra.run.smithi094.stdout:osd.6: {} 2024-09-14T06:29:55.173 INFO:teuthology.orchestra.run.smithi094.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-14T06:29:55.180 INFO:tasks.ceph.osd.7.smithi138.stderr:2024-09-14T06:29:55.179+0000 7f3c6a21d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33623) UID: 0 2024-09-14T06:29:55.189 INFO:teuthology.orchestra.run.smithi094.stdout:osd.7: {} 2024-09-14T06:29:55.189 INFO:teuthology.orchestra.run.smithi094.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-14T06:29:55.222 INFO:teuthology.orchestra.run.smithi094.stdout:ERROR: (22) Invalid argument 2024-09-14T06:29:55.222 INFO:teuthology.orchestra.run.smithi094.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:29:55.231 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-14T06:29:55.281 INFO:tasks.ceph.osd.5.smithi138.stderr:2024-09-14T06:29:55.279+0000 7fe13746f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33618) UID: 0 2024-09-14T06:29:55.381 INFO:tasks.ceph.osd.6.smithi138.stderr:2024-09-14T06:29:55.380+0000 7f8cd8849640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33620) UID: 0 2024-09-14T06:29:55.422 INFO:teuthology.orchestra.run.smithi094.stdout:ERROR: (22) Invalid argument 2024-09-14T06:29:55.422 INFO:teuthology.orchestra.run.smithi094.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:29:55.431 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-14T06:29:55.482 INFO:tasks.ceph.osd.0.smithi094.stderr:2024-09-14T06:29:55.481+0000 7f07c9333640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33636) UID: 0 2024-09-14T06:29:55.533 INFO:teuthology.orchestra.run.smithi094.stdout:ERROR: (22) Invalid argument 2024-09-14T06:29:55.533 INFO:teuthology.orchestra.run.smithi094.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:29:55.541 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-14T06:29:55.584 INFO:tasks.ceph.osd.5.smithi138.stderr:2024-09-14T06:29:55.582+0000 7fe13746f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33618) UID: 0 2024-09-14T06:29:55.685 INFO:tasks.ceph.osd.7.smithi138.stderr:2024-09-14T06:29:55.683+0000 7f3c6a21d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33623) UID: 0 2024-09-14T06:29:55.692 INFO:teuthology.orchestra.run.smithi094.stdout:ERROR: (22) Invalid argument 2024-09-14T06:29:55.692 INFO:teuthology.orchestra.run.smithi094.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:29:55.702 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-14T06:29:55.731 INFO:teuthology.orchestra.run.smithi094.stdout:checking clone #12, image image_client.0-clone12 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent13 2024-09-14T06:29:55.786 INFO:tasks.ceph.osd.3.smithi094.stderr:2024-09-14T06:29:55.785+0000 7feeeafe8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33655) UID: 0 2024-09-14T06:29:55.821 INFO:teuthology.orchestra.run.smithi094.stdout:ERROR: (22) Invalid argument 2024-09-14T06:29:55.821 INFO:teuthology.orchestra.run.smithi094.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:29:55.830 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-14T06:29:55.883 INFO:teuthology.orchestra.run.smithi094.stdout:1644 trunc from 0xedb8295 to 0x72223d4 2024-09-14T06:29:55.887 INFO:tasks.ceph.osd.0.smithi094.stderr:2024-09-14T06:29:55.886+0000 7f07c9333640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33636) UID: 0 2024-09-14T06:29:55.897 INFO:teuthology.orchestra.run.smithi094.stdout:1645 trunc from 0x72223d4 to 0x6733bf9 2024-09-14T06:29:55.903 INFO:teuthology.orchestra.run.smithi094.stdout:1646 read 0x2958a65 thru 0x2960929 (0x7ec5 bytes) 2024-09-14T06:29:55.904 INFO:teuthology.orchestra.run.smithi094.stdout:1647 read 0x119c57d thru 0x11a5f78 (0x99fc bytes) 2024-09-14T06:29:55.913 INFO:teuthology.orchestra.run.smithi094.stdout:1648 trunc from 0x6733bf9 to 0xd1a9107 2024-09-14T06:29:55.918 INFO:teuthology.orchestra.run.smithi094.stdout:1653 writesame 0xaa77e9c thru 0xaa8062a data_size 0x1(0x878f bytes) 2024-09-14T06:29:55.937 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-14T06:29:55.937 INFO:teuthology.orchestra.run.smithi094.stdout:1658 clone 15 order 20 su 131072 sc 11 2024-09-14T06:29:55.984 INFO:teuthology.orchestra.run.smithi094.stdout:ERROR: (22) Invalid argument 2024-09-14T06:29:55.985 INFO:teuthology.orchestra.run.smithi094.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:29:55.988 INFO:tasks.ceph.osd.4.smithi138.stderr:2024-09-14T06:29:55.987+0000 7f6c653c8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33615) UID: 0 2024-09-14T06:29:55.993 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-14T06:29:56.089 INFO:tasks.ceph.osd.4.smithi138.stderr:2024-09-14T06:29:56.088+0000 7f6c653c8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33615) UID: 0 2024-09-14T06:29:56.136 INFO:teuthology.orchestra.run.smithi094.stdout:ERROR: (22) Invalid argument 2024-09-14T06:29:56.136 INFO:teuthology.orchestra.run.smithi094.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:29:56.145 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-14T06:29:56.190 INFO:tasks.ceph.osd.0.smithi094.stderr:2024-09-14T06:29:56.189+0000 7f07c9333640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33636) UID: 0 2024-09-14T06:29:56.247 INFO:teuthology.orchestra.run.smithi094.stdout:ERROR: (22) Invalid argument 2024-09-14T06:29:56.247 INFO:teuthology.orchestra.run.smithi094.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:29:56.255 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-14T06:29:56.296 INFO:tasks.ceph.osd.1.smithi094.stderr:2024-09-14T06:29:56.295+0000 7f328e47a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33641) UID: 0 2024-09-14T06:29:56.394 INFO:tasks.ceph.osd.5.smithi138.stderr:2024-09-14T06:29:56.392+0000 7fe13746f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33618) UID: 0 2024-09-14T06:29:56.397 INFO:teuthology.orchestra.run.smithi094.stdout:ERROR: (22) Invalid argument 2024-09-14T06:29:56.397 INFO:teuthology.orchestra.run.smithi094.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:29:56.405 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-14T06:29:56.494 INFO:tasks.ceph.osd.6.smithi138.stderr:2024-09-14T06:29:56.493+0000 7f8cd8849640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33620) UID: 0 2024-09-14T06:29:56.510 INFO:teuthology.orchestra.run.smithi138.stdout:ERROR: (22) Invalid argument 2024-09-14T06:29:56.511 INFO:teuthology.orchestra.run.smithi138.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:29:56.519 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-14T06:29:56.604 INFO:tasks.ceph.osd.5.smithi138.stderr:2024-09-14T06:29:56.603+0000 7fe13746f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33618) UID: 0 2024-09-14T06:29:56.667 INFO:teuthology.orchestra.run.smithi138.stdout:ERROR: (22) Invalid argument 2024-09-14T06:29:56.667 INFO:teuthology.orchestra.run.smithi138.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:29:56.675 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-14T06:29:56.718 INFO:tasks.ceph.osd.4.smithi138.stderr:2024-09-14T06:29:56.716+0000 7f6c653c8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33615) UID: 0 2024-09-14T06:29:56.792 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-14T06:29:56.820 INFO:teuthology.orchestra.run.smithi138.stdout:ERROR: (22) Invalid argument 2024-09-14T06:29:56.820 INFO:teuthology.orchestra.run.smithi138.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:29:56.828 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-14T06:29:56.836 INFO:tasks.ceph.osd.2.smithi094.stderr:2024-09-14T06:29:56.835+0000 7faf3269b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33649) UID: 0 2024-09-14T06:29:56.897 INFO:tasks.ceph.osd.7.smithi138.stderr:2024-09-14T06:29:56.896+0000 7f3c6a21d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33623) UID: 0 2024-09-14T06:29:56.930 INFO:teuthology.orchestra.run.smithi138.stdout:ERROR: (22) Invalid argument 2024-09-14T06:29:56.931 INFO:teuthology.orchestra.run.smithi138.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:29:56.939 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-14T06:29:57.024 INFO:tasks.ceph.osd.5.smithi138.stderr:2024-09-14T06:29:57.023+0000 7fe13746f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33618) UID: 0 2024-09-14T06:29:57.082 INFO:teuthology.orchestra.run.smithi138.stdout:ERROR: (22) Invalid argument 2024-09-14T06:29:57.082 INFO:teuthology.orchestra.run.smithi138.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:29:57.091 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-14T06:29:57.100 INFO:tasks.ceph.osd.1.smithi094.stderr:2024-09-14T06:29:57.099+0000 7f328e47a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33641) UID: 0 2024-09-14T06:29:57.200 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-14T06:29:57.201 INFO:tasks.ceph.osd.1.smithi094.stderr:2024-09-14T06:29:57.199+0000 7f328e47a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33641) UID: 0 2024-09-14T06:29:57.235 INFO:teuthology.orchestra.run.smithi138.stdout:ERROR: (22) Invalid argument 2024-09-14T06:29:57.235 INFO:teuthology.orchestra.run.smithi138.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:29:57.243 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-14T06:29:57.301 INFO:tasks.ceph.osd.4.smithi138.stderr:2024-09-14T06:29:57.300+0000 7f6c653c8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33615) UID: 0 2024-09-14T06:29:57.319 INFO:teuthology.orchestra.run.smithi094.stdout:osd.0: {} 2024-09-14T06:29:57.319 INFO:teuthology.orchestra.run.smithi094.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-14T06:29:57.336 INFO:teuthology.orchestra.run.smithi094.stdout:osd.1: {} 2024-09-14T06:29:57.336 INFO:teuthology.orchestra.run.smithi094.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-14T06:29:57.348 INFO:teuthology.orchestra.run.smithi138.stdout:ERROR: (22) Invalid argument 2024-09-14T06:29:57.348 INFO:teuthology.orchestra.run.smithi138.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:29:57.352 INFO:teuthology.orchestra.run.smithi094.stdout:osd.2: {} 2024-09-14T06:29:57.352 INFO:teuthology.orchestra.run.smithi094.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-14T06:29:57.356 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-14T06:29:57.368 INFO:teuthology.orchestra.run.smithi094.stdout:osd.3: {} 2024-09-14T06:29:57.368 INFO:teuthology.orchestra.run.smithi094.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-14T06:29:57.384 INFO:teuthology.orchestra.run.smithi094.stdout:osd.4: {} 2024-09-14T06:29:57.384 INFO:teuthology.orchestra.run.smithi094.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-14T06:29:57.401 INFO:teuthology.orchestra.run.smithi094.stdout:osd.5: {} 2024-09-14T06:29:57.401 INFO:teuthology.orchestra.run.smithi094.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-14T06:29:57.417 INFO:teuthology.orchestra.run.smithi094.stdout:osd.6: {} 2024-09-14T06:29:57.417 INFO:teuthology.orchestra.run.smithi094.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-14T06:29:57.433 INFO:teuthology.orchestra.run.smithi094.stdout:osd.7: {} 2024-09-14T06:29:57.434 INFO:teuthology.orchestra.run.smithi094.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-14T06:29:57.440 INFO:tasks.ceph.osd.4.smithi138.stderr:2024-09-14T06:29:57.439+0000 7f6c653c8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33615) UID: 0 2024-09-14T06:29:57.498 INFO:teuthology.orchestra.run.smithi138.stdout:{ 2024-09-14T06:29:57.498 INFO:teuthology.orchestra.run.smithi138.stdout: "ops": [], 2024-09-14T06:29:57.498 INFO:teuthology.orchestra.run.smithi138.stdout: "complaint_time": 30, 2024-09-14T06:29:57.499 INFO:teuthology.orchestra.run.smithi138.stdout: "num_blocked_ops": 0 2024-09-14T06:29:57.499 INFO:teuthology.orchestra.run.smithi138.stdout:} 2024-09-14T06:29:57.503 INFO:tasks.ceph.osd.2.smithi094.stderr:2024-09-14T06:29:57.502+0000 7faf3269b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33649) UID: 0 2024-09-14T06:29:57.507 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-14T06:29:57.580 INFO:teuthology.orchestra.run.smithi094.stderr:nodeep-scrub is unset 2024-09-14T06:29:57.640 INFO:tasks.ceph.osd.3.smithi094.stderr:2024-09-14T06:29:57.639+0000 7feeeafe8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33655) UID: 0 2024-09-14T06:29:57.703 INFO:teuthology.orchestra.run.smithi138.stdout:{ 2024-09-14T06:29:57.704 INFO:teuthology.orchestra.run.smithi138.stdout: "size": 20, 2024-09-14T06:29:57.704 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 600, 2024-09-14T06:29:57.704 INFO:teuthology.orchestra.run.smithi138.stdout: "ops": [ 2024-09-14T06:29:57.704 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:57.704 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:1500 3.a 3:57c28b0f:::rbd_data.112d7dfbd766.000000000000000a:5 [read 471040~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-14T06:29:57.704 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:08.629204+0000", 2024-09-14T06:29:57.704 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 49.072074264999998, 2024-09-14T06:29:57.704 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.020473080000000001, 2024-09-14T06:29:57.704 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:29:57.704 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:29:57.705 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:29:57.705 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:29:57.705 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:29:57.705 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:57.705 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 1500 2024-09-14T06:29:57.705 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:57.705 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:29:57.705 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:57.705 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:29:57.705 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.629204+0000", 2024-09-14T06:29:57.705 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:57.705 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:57.705 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:57.706 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:29:57.706 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.629204+0000", 2024-09-14T06:29:57.706 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:57.706 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:57.706 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:57.706 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:29:57.706 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.629204+0000", 2024-09-14T06:29:57.706 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.34e-07 2024-09-14T06:29:57.706 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:57.706 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:57.707 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:29:57.707 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.629206+0000", 2024-09-14T06:29:57.707 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.939e-06 2024-09-14T06:29:57.707 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:57.707 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:57.707 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:29:57.707 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.629212+0000", 2024-09-14T06:29:57.707 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.8669999999999999e-06 2024-09-14T06:29:57.707 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:57.707 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:57.707 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:29:57.707 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.629216+0000", 2024-09-14T06:29:57.707 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.3830000000000002e-06 2024-09-14T06:29:57.708 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:57.708 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:57.708 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:29:57.708 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.649288+0000", 2024-09-14T06:29:57.708 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.020072113999999999 2024-09-14T06:29:57.708 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:57.708 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:57.709 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:29:57.709 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.649355+0000", 2024-09-14T06:29:57.709 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.7218000000000006e-05 2024-09-14T06:29:57.709 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:57.709 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:57.709 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:29:57.709 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.649677+0000", 2024-09-14T06:29:57.709 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00032122500000000002 2024-09-14T06:29:57.709 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:57.709 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:29:57.709 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:57.710 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:57.710 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:57.710 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:1501 3.a 3:57c28b0f:::rbd_data.112d7dfbd766.000000000000000a:5 [read 487424~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-14T06:29:57.710 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:08.629223+0000", 2024-09-14T06:29:57.710 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 49.072054815999998, 2024-09-14T06:29:57.710 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.021370209000000001, 2024-09-14T06:29:57.710 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:29:57.710 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:29:57.710 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:29:57.710 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:29:57.711 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:29:57.711 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:57.711 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 1501 2024-09-14T06:29:57.711 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:57.711 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:29:57.711 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:57.711 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:29:57.711 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.629223+0000", 2024-09-14T06:29:57.711 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:57.711 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:57.711 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:57.711 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:29:57.711 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.629223+0000", 2024-09-14T06:29:57.711 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:57.712 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:57.712 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:57.712 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:29:57.712 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.629223+0000", 2024-09-14T06:29:57.712 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.9900000000000001e-07 2024-09-14T06:29:57.712 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:57.712 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:57.712 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:29:57.712 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.629231+0000", 2024-09-14T06:29:57.712 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.9179999999999994e-06 2024-09-14T06:29:57.712 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:57.712 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:57.712 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:29:57.713 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.629238+0000", 2024-09-14T06:29:57.713 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.421e-06 2024-09-14T06:29:57.713 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:57.713 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:57.713 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:29:57.713 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.629242+0000", 2024-09-14T06:29:57.713 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.5419999999999999e-06 2024-09-14T06:29:57.713 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:57.713 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:57.713 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:29:57.713 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.650284+0000", 2024-09-14T06:29:57.713 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.021041131000000001 2024-09-14T06:29:57.713 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:57.714 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:57.714 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:29:57.714 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.650361+0000", 2024-09-14T06:29:57.714 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.7787999999999995e-05 2024-09-14T06:29:57.714 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:57.714 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:57.714 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:29:57.714 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.650593+0000", 2024-09-14T06:29:57.714 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00023201 2024-09-14T06:29:57.714 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:57.714 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:29:57.714 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:57.714 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:57.715 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:57.715 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:1502 3.a 3:57c28b0f:::rbd_data.112d7dfbd766.000000000000000a:5 [read 503808~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-14T06:29:57.715 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:08.629249+0000", 2024-09-14T06:29:57.715 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 49.072028570000001, 2024-09-14T06:29:57.715 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.021713217, 2024-09-14T06:29:57.715 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:29:57.715 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:29:57.715 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:29:57.715 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:29:57.715 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:29:57.715 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:57.715 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 1502 2024-09-14T06:29:57.715 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:57.715 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:29:57.715 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:57.716 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:29:57.716 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.629249+0000", 2024-09-14T06:29:57.716 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:57.716 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:57.716 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:57.716 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:29:57.716 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.629249+0000", 2024-09-14T06:29:57.716 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:57.716 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:57.716 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:57.716 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:29:57.716 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.629250+0000", 2024-09-14T06:29:57.716 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.03e-07 2024-09-14T06:29:57.717 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:57.717 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:57.717 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:29:57.717 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.629252+0000", 2024-09-14T06:29:57.717 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.995e-06 2024-09-14T06:29:57.717 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:57.717 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:57.717 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:29:57.717 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.629257+0000", 2024-09-14T06:29:57.717 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.7010000000000001e-06 2024-09-14T06:29:57.717 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:57.717 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:57.717 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:29:57.718 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.629262+0000", 2024-09-14T06:29:57.718 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.16e-06 2024-09-14T06:29:57.718 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:57.718 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:57.718 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:29:57.718 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.650629+0000", 2024-09-14T06:29:57.718 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.021367684000000001 2024-09-14T06:29:57.718 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:57.718 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:57.718 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:29:57.718 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.650718+0000", 2024-09-14T06:29:57.718 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.8832000000000006e-05 2024-09-14T06:29:57.718 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:57.718 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:57.719 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:29:57.719 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.650963+0000", 2024-09-14T06:29:57.719 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00024444199999999999 2024-09-14T06:29:57.719 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:57.719 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:29:57.719 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:57.719 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:57.719 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:57.719 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:1503 3.a 3:57c28b0f:::rbd_data.112d7dfbd766.000000000000000a:5 [read 520192~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-14T06:29:57.719 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:08.629268+0000", 2024-09-14T06:29:57.719 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 49.072010028999998, 2024-09-14T06:29:57.719 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.022244885999999998, 2024-09-14T06:29:57.719 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:29:57.720 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:29:57.720 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:29:57.720 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:29:57.720 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:29:57.720 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:57.720 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 1503 2024-09-14T06:29:57.720 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:57.720 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:29:57.720 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:57.720 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:29:57.721 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.629268+0000", 2024-09-14T06:29:57.721 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:57.721 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:57.721 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:57.721 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:29:57.721 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.629268+0000", 2024-09-14T06:29:57.721 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:57.721 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:57.721 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:57.721 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:29:57.722 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.629268+0000", 2024-09-14T06:29:57.722 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.6599999999999996e-07 2024-09-14T06:29:57.722 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:57.722 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:57.722 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:29:57.722 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.629270+0000", 2024-09-14T06:29:57.722 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.7129999999999999e-06 2024-09-14T06:29:57.722 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:57.722 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:57.722 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:29:57.722 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.629276+0000", 2024-09-14T06:29:57.722 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.8119999999999996e-06 2024-09-14T06:29:57.722 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:57.723 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:57.723 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:29:57.723 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.629280+0000", 2024-09-14T06:29:57.723 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.4920000000000004e-06 2024-09-14T06:29:57.723 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:57.723 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:57.723 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:29:57.723 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.651039+0000", 2024-09-14T06:29:57.723 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.021758116000000001 2024-09-14T06:29:57.723 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:57.723 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:57.723 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:29:57.723 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.651127+0000", 2024-09-14T06:29:57.724 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.7983000000000001e-05 2024-09-14T06:29:57.724 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:57.724 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:57.724 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:29:57.724 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:08.651513+0000", 2024-09-14T06:29:57.724 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00038620400000000002 2024-09-14T06:29:57.724 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:57.724 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:29:57.724 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:57.724 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:57.724 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:57.724 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:73587 3.a 3:5604efef:::rbd_data.112d4c988ca3.000000000000001d:1d [read 98304~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-14T06:29:57.724 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:53.828877+0000", 2024-09-14T06:29:57.725 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 3.8724011900000002, 2024-09-14T06:29:57.725 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.020953949999999999, 2024-09-14T06:29:57.725 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:29:57.725 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:29:57.725 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:29:57.725 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:29:57.725 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:29:57.725 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:57.725 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 73587 2024-09-14T06:29:57.725 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:57.726 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:29:57.726 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:57.726 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:29:57.726 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:53.828877+0000", 2024-09-14T06:29:57.726 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:57.726 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:57.726 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:57.726 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:29:57.726 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:53.828877+0000", 2024-09-14T06:29:57.726 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:57.726 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:57.726 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:57.727 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:29:57.727 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:53.828877+0000", 2024-09-14T06:29:57.727 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.3799999999999998e-07 2024-09-14T06:29:57.727 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:57.727 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:57.727 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:29:57.727 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:53.828879+0000", 2024-09-14T06:29:57.727 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.7269999999999999e-06 2024-09-14T06:29:57.727 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:57.727 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:57.727 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:29:57.727 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:53.828884+0000", 2024-09-14T06:29:57.727 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.383e-06 2024-09-14T06:29:57.728 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:57.728 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:57.728 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:29:57.728 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:53.828888+0000", 2024-09-14T06:29:57.728 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.25e-06 2024-09-14T06:29:57.728 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:57.728 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:57.728 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:29:57.728 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:53.849117+0000", 2024-09-14T06:29:57.728 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.020228349999999999 2024-09-14T06:29:57.728 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:57.728 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:57.728 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:29:57.729 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:53.849237+0000", 2024-09-14T06:29:57.729 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00011999500000000001 2024-09-14T06:29:57.729 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:57.729 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:57.729 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:29:57.729 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:53.849831+0000", 2024-09-14T06:29:57.729 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00059390699999999996 2024-09-14T06:29:57.729 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:57.729 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:29:57.729 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:57.729 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:57.729 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:57.730 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:73588 3.a 3:5604efef:::rbd_data.112d4c988ca3.000000000000001d:1d [read 245760~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-14T06:29:57.730 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:53.828894+0000", 2024-09-14T06:29:57.730 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 3.8723842820000001, 2024-09-14T06:29:57.730 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.021322085000000001, 2024-09-14T06:29:57.730 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:29:57.730 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:29:57.730 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:29:57.730 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:29:57.730 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:29:57.730 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:57.730 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 73588 2024-09-14T06:29:57.730 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:57.731 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:29:57.731 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:57.731 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:29:57.731 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:53.828894+0000", 2024-09-14T06:29:57.731 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:57.731 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:57.731 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:57.731 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:29:57.731 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:53.828894+0000", 2024-09-14T06:29:57.731 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:57.731 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:57.731 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:57.732 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:29:57.732 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:53.828894+0000", 2024-09-14T06:29:57.732 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.22e-07 2024-09-14T06:29:57.732 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:57.732 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:57.732 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:29:57.732 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:53.828896+0000", 2024-09-14T06:29:57.732 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.668e-06 2024-09-14T06:29:57.732 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:57.732 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:57.732 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:29:57.732 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:53.828901+0000", 2024-09-14T06:29:57.733 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.3480000000000003e-06 2024-09-14T06:29:57.733 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:57.733 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:57.733 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:29:57.733 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:53.828905+0000", 2024-09-14T06:29:57.733 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.2350000000000001e-06 2024-09-14T06:29:57.733 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:57.733 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:57.733 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:29:57.733 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:53.849887+0000", 2024-09-14T06:29:57.733 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.020981883 2024-09-14T06:29:57.733 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:57.733 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:57.734 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:29:57.734 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:53.849988+0000", 2024-09-14T06:29:57.734 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00010122499999999999 2024-09-14T06:29:57.734 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:57.734 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:57.734 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:29:57.734 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:53.850216+0000", 2024-09-14T06:29:57.734 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00022740400000000001 2024-09-14T06:29:57.734 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:57.734 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:29:57.734 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:57.734 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:57.734 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:57.734 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:73605 3.a 3:549bef42:::rbd_data.112d4c988ca3.0000000000000022:1d [read 98304~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-14T06:29:57.735 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:53.828910+0000", 2024-09-14T06:29:57.735 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 3.8723675449999999, 2024-09-14T06:29:57.735 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.021797369, 2024-09-14T06:29:57.735 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:29:57.735 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:29:57.735 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:29:57.735 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:29:57.735 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:29:57.735 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:57.735 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 73605 2024-09-14T06:29:57.735 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:57.735 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:29:57.735 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:57.736 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:29:57.736 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:53.828910+0000", 2024-09-14T06:29:57.736 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:57.736 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:57.736 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:57.736 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:29:57.736 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:53.828910+0000", 2024-09-14T06:29:57.736 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:57.736 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:57.736 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:57.736 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:29:57.736 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:53.828915+0000", 2024-09-14T06:29:57.736 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.5220000000000001e-06 2024-09-14T06:29:57.737 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:57.737 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:57.737 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:29:57.737 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:53.828917+0000", 2024-09-14T06:29:57.737 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.812e-06 2024-09-14T06:29:57.737 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:57.737 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:57.737 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:29:57.737 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:53.828923+0000", 2024-09-14T06:29:57.737 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.8499999999999999e-06 2024-09-14T06:29:57.737 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:57.737 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:57.737 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:29:57.738 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:53.828927+0000", 2024-09-14T06:29:57.738 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.6450000000000004e-06 2024-09-14T06:29:57.738 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:57.738 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:57.738 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:29:57.738 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:53.850248+0000", 2024-09-14T06:29:57.738 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.021320543000000001 2024-09-14T06:29:57.738 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:57.738 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:57.738 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:29:57.738 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:53.850500+0000", 2024-09-14T06:29:57.738 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00025234999999999998 2024-09-14T06:29:57.738 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:57.738 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:57.739 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:29:57.739 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:53.850708+0000", 2024-09-14T06:29:57.739 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00020764700000000001 2024-09-14T06:29:57.739 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:57.739 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:29:57.739 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:57.739 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:57.739 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:57.739 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:73606 3.a 3:549bef42:::rbd_data.112d4c988ca3.0000000000000022:1d [read 131072~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-14T06:29:57.739 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:53.828933+0000", 2024-09-14T06:29:57.739 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 3.8723449350000001, 2024-09-14T06:29:57.739 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.022113200999999999, 2024-09-14T06:29:57.739 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:29:57.739 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:29:57.740 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:29:57.740 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:29:57.740 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:29:57.740 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:57.740 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 73606 2024-09-14T06:29:57.740 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:57.740 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:29:57.740 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:57.740 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:29:57.740 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:53.828933+0000", 2024-09-14T06:29:57.740 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:57.740 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:57.740 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:57.741 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:29:57.741 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:53.828933+0000", 2024-09-14T06:29:57.741 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:57.741 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:57.741 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:57.741 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:29:57.741 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:53.828933+0000", 2024-09-14T06:29:57.741 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.53e-07 2024-09-14T06:29:57.741 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:57.741 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:57.741 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:29:57.741 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:53.828935+0000", 2024-09-14T06:29:57.742 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.559e-06 2024-09-14T06:29:57.742 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:57.742 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:57.742 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:29:57.742 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:53.828940+0000", 2024-09-14T06:29:57.742 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.5600000000000001e-06 2024-09-14T06:29:57.742 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:57.742 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:57.742 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:29:57.742 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:53.828945+0000", 2024-09-14T06:29:57.742 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.4469999999999999e-06 2024-09-14T06:29:57.742 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:57.743 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:57.743 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:29:57.743 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:53.850738+0000", 2024-09-14T06:29:57.743 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.021793265999999999 2024-09-14T06:29:57.743 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:57.743 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:57.743 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:29:57.743 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:53.850813+0000", 2024-09-14T06:29:57.743 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.4810999999999999e-05 2024-09-14T06:29:57.743 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:57.743 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:57.743 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:29:57.743 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:53.851046+0000", 2024-09-14T06:29:57.744 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.000233205 2024-09-14T06:29:57.744 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:57.744 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:29:57.744 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:57.744 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:57.744 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:57.744 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:73607 3.a 3:549bef42:::rbd_data.112d4c988ca3.0000000000000022:1d [read 163840~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-14T06:29:57.744 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:53.828950+0000", 2024-09-14T06:29:57.744 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 3.8723277810000001, 2024-09-14T06:29:57.744 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.022421109000000002, 2024-09-14T06:29:57.744 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:29:57.744 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:29:57.744 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:29:57.745 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:29:57.745 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:29:57.745 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:57.745 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 73607 2024-09-14T06:29:57.745 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:57.745 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:29:57.745 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:57.745 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:29:57.745 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:53.828950+0000", 2024-09-14T06:29:57.745 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:57.746 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:57.746 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:57.746 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:29:57.746 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:53.828950+0000", 2024-09-14T06:29:57.746 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:57.746 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:57.746 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:57.746 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:29:57.746 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:53.828950+0000", 2024-09-14T06:29:57.747 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.3999999999999997e-07 2024-09-14T06:29:57.747 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:57.747 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:57.747 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:29:57.747 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:53.828956+0000", 2024-09-14T06:29:57.747 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.5639999999999996e-06 2024-09-14T06:29:57.747 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:57.747 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:57.747 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:29:57.747 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:53.828962+0000", 2024-09-14T06:29:57.747 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.9159999999999998e-06 2024-09-14T06:29:57.747 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:57.747 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:57.747 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:29:57.748 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:53.828966+0000", 2024-09-14T06:29:57.748 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.5050000000000001e-06 2024-09-14T06:29:57.748 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:57.748 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:57.748 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:29:57.748 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:53.851077+0000", 2024-09-14T06:29:57.748 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.022110603999999999 2024-09-14T06:29:57.748 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:57.748 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:57.748 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:29:57.748 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:53.851156+0000", 2024-09-14T06:29:57.748 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.8781000000000003e-05 2024-09-14T06:29:57.748 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:57.749 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:57.749 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:29:57.749 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:53.851371+0000", 2024-09-14T06:29:57.749 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00021539899999999999 2024-09-14T06:29:57.749 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:57.749 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:29:57.749 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:57.749 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:57.749 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:57.749 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:73608 3.a 3:549bef42:::rbd_data.112d4c988ca3.0000000000000022:1d [read 196608~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-14T06:29:57.749 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:53.828972+0000", 2024-09-14T06:29:57.749 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 3.8723056119999999, 2024-09-14T06:29:57.749 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.022703864000000001, 2024-09-14T06:29:57.750 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:29:57.750 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:29:57.750 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:29:57.750 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:29:57.750 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:29:57.750 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:57.750 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 73608 2024-09-14T06:29:57.750 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:57.750 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:29:57.750 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:57.750 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:29:57.750 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:53.828972+0000", 2024-09-14T06:29:57.750 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:57.750 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:57.751 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:57.751 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:29:57.751 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:53.828972+0000", 2024-09-14T06:29:57.751 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:57.751 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:57.751 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:57.751 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:29:57.751 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:53.828973+0000", 2024-09-14T06:29:57.751 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.5600000000000001e-07 2024-09-14T06:29:57.751 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:57.752 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:57.752 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:29:57.752 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:53.828974+0000", 2024-09-14T06:29:57.752 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.6950000000000001e-06 2024-09-14T06:29:57.752 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:57.752 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:57.752 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:29:57.752 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:53.828979+0000", 2024-09-14T06:29:57.752 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.0880000000000002e-06 2024-09-14T06:29:57.752 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:57.752 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:57.752 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:29:57.753 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:53.829825+0000", 2024-09-14T06:29:57.753 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00084584999999999999 2024-09-14T06:29:57.753 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:57.753 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:57.753 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:29:57.753 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:53.851403+0000", 2024-09-14T06:29:57.753 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.021577875999999999 2024-09-14T06:29:57.753 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:57.753 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:57.753 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:29:57.753 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:53.851482+0000", 2024-09-14T06:29:57.753 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.8668e-05 2024-09-14T06:29:57.753 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:57.754 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:57.754 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:29:57.754 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:53.851676+0000", 2024-09-14T06:29:57.754 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00019433100000000001 2024-09-14T06:29:57.754 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:57.754 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:29:57.754 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:57.754 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:57.754 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:57.754 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:73873 3.a 3:52a26c8a:::rbd_data.112d4c988ca3.00000000000000a4:1d [read 131072~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-14T06:29:57.754 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:53.831416+0000", 2024-09-14T06:29:57.754 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 3.8698620830000001, 2024-09-14T06:29:57.754 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.020616973, 2024-09-14T06:29:57.754 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:29:57.755 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:29:57.755 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:29:57.755 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:29:57.755 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:29:57.755 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:57.755 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 73873 2024-09-14T06:29:57.755 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:57.755 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:29:57.755 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:57.755 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:29:57.755 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:53.831416+0000", 2024-09-14T06:29:57.755 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:57.755 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:57.756 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:57.756 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:29:57.756 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:53.831416+0000", 2024-09-14T06:29:57.756 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:57.756 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:57.756 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:57.756 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:29:57.756 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:53.831416+0000", 2024-09-14T06:29:57.756 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.9099999999999999e-07 2024-09-14T06:29:57.756 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:57.756 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:57.756 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:29:57.756 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:53.831670+0000", 2024-09-14T06:29:57.757 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00025330600000000002 2024-09-14T06:29:57.757 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:57.757 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:57.757 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:29:57.757 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:53.831679+0000", 2024-09-14T06:29:57.757 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.0210000000000005e-06 2024-09-14T06:29:57.757 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:57.757 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:57.757 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:29:57.757 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:53.831685+0000", 2024-09-14T06:29:57.757 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.3729999999999998e-06 2024-09-14T06:29:57.757 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:57.758 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:57.758 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:29:57.758 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:53.851704+0000", 2024-09-14T06:29:57.758 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.020018978 2024-09-14T06:29:57.758 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:57.758 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:57.758 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:29:57.758 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:53.851773+0000", 2024-09-14T06:29:57.758 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.881e-05 2024-09-14T06:29:57.758 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:57.758 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:57.758 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:29:57.758 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:53.852033+0000", 2024-09-14T06:29:57.759 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00026009399999999999 2024-09-14T06:29:57.759 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:57.759 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:29:57.759 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:57.759 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:57.759 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:57.759 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:73874 3.a 3:52a26c8a:::rbd_data.112d4c988ca3.00000000000000a4:1d [read 163840~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-14T06:29:57.759 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:53.831694+0000", 2024-09-14T06:29:57.759 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 3.869583649, 2024-09-14T06:29:57.759 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.020649862000000001, 2024-09-14T06:29:57.759 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:29:57.759 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:29:57.759 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:29:57.759 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:29:57.760 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:29:57.760 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:57.760 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 73874 2024-09-14T06:29:57.760 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:57.760 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:29:57.760 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:57.760 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:29:57.760 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:53.831694+0000", 2024-09-14T06:29:57.760 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:57.760 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:57.760 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:57.760 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:29:57.760 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:53.831694+0000", 2024-09-14T06:29:57.761 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:57.761 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:57.761 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:57.761 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:29:57.761 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:53.831695+0000", 2024-09-14T06:29:57.761 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.0699999999999997e-07 2024-09-14T06:29:57.761 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:57.761 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:57.761 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:29:57.761 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:53.831697+0000", 2024-09-14T06:29:57.761 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.9929999999999998e-06 2024-09-14T06:29:57.761 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:57.762 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:57.762 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:29:57.762 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:53.831703+0000", 2024-09-14T06:29:57.762 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.6610000000000003e-06 2024-09-14T06:29:57.762 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:57.762 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:57.762 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:29:57.762 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:53.831708+0000", 2024-09-14T06:29:57.762 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.8570000000000003e-06 2024-09-14T06:29:57.762 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:57.762 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:57.762 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:29:57.762 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:53.852061+0000", 2024-09-14T06:29:57.762 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.020352577 2024-09-14T06:29:57.762 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:57.762 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:57.763 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:29:57.763 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:53.852135+0000", 2024-09-14T06:29:57.763 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.3656000000000001e-05 2024-09-14T06:29:57.763 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:57.763 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:57.763 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:29:57.763 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:53.852344+0000", 2024-09-14T06:29:57.763 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00020961100000000001 2024-09-14T06:29:57.763 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:57.763 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:29:57.763 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:57.763 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:57.763 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:57.763 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:73875 3.a 3:52a26c8a:::rbd_data.112d4c988ca3.00000000000000a4:1d [read 196608~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-14T06:29:57.764 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:53.831797+0000", 2024-09-14T06:29:57.764 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 3.8694809270000001, 2024-09-14T06:29:57.764 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.020875273, 2024-09-14T06:29:57.764 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:29:57.764 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:29:57.764 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:29:57.764 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:29:57.764 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:29:57.764 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:57.764 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 73875 2024-09-14T06:29:57.764 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:57.764 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:29:57.765 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:57.765 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:29:57.765 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:53.831797+0000", 2024-09-14T06:29:57.765 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:57.765 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:57.765 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:57.765 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:29:57.765 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:53.831797+0000", 2024-09-14T06:29:57.765 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:57.765 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:57.765 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:57.765 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:29:57.765 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:53.831798+0000", 2024-09-14T06:29:57.765 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.0399999999999996e-07 2024-09-14T06:29:57.766 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:57.766 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:57.766 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:29:57.766 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:53.831800+0000", 2024-09-14T06:29:57.766 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.79e-06 2024-09-14T06:29:57.766 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:57.766 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:57.766 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:29:57.766 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:53.831809+0000", 2024-09-14T06:29:57.766 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.8119999999999997e-06 2024-09-14T06:29:57.766 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:57.766 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:57.766 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:29:57.767 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:53.831822+0000", 2024-09-14T06:29:57.767 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.2781e-05 2024-09-14T06:29:57.767 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:57.767 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:57.767 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:29:57.767 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:53.852379+0000", 2024-09-14T06:29:57.767 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.020557392000000001 2024-09-14T06:29:57.767 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:57.767 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:57.767 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:29:57.768 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:53.852461+0000", 2024-09-14T06:29:57.768 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.2055e-05 2024-09-14T06:29:57.768 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:57.768 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:57.768 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:29:57.768 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:53.852672+0000", 2024-09-14T06:29:57.768 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00021083899999999999 2024-09-14T06:29:57.768 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:57.768 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:29:57.768 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:57.768 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:57.768 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:57.769 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:73876 3.a 3:52a26c8a:::rbd_data.112d4c988ca3.00000000000000a4:1d [read 229376~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-14T06:29:57.769 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:53.831833+0000", 2024-09-14T06:29:57.769 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 3.8694445989999999, 2024-09-14T06:29:57.769 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.02119861, 2024-09-14T06:29:57.769 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:29:57.769 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:29:57.769 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:29:57.769 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:29:57.769 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:29:57.769 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:57.769 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 73876 2024-09-14T06:29:57.769 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:57.769 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:29:57.770 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:57.770 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:29:57.770 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:53.831833+0000", 2024-09-14T06:29:57.770 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:57.770 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:57.770 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:57.770 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:29:57.770 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:53.831833+0000", 2024-09-14T06:29:57.770 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:57.770 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:57.770 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:57.770 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:29:57.771 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:53.831834+0000", 2024-09-14T06:29:57.771 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.0100000000000005e-07 2024-09-14T06:29:57.771 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:57.771 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:57.771 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:29:57.771 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:53.831836+0000", 2024-09-14T06:29:57.771 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.227e-06 2024-09-14T06:29:57.771 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:57.771 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:57.772 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:29:57.772 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:53.831844+0000", 2024-09-14T06:29:57.772 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.6090000000000003e-06 2024-09-14T06:29:57.772 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:57.772 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:57.772 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:29:57.772 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:53.831849+0000", 2024-09-14T06:29:57.772 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.9219999999999996e-06 2024-09-14T06:29:57.772 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:57.772 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:57.772 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:29:57.772 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:53.852704+0000", 2024-09-14T06:29:57.772 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.020854961000000002 2024-09-14T06:29:57.773 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:57.773 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:57.773 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:29:57.773 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:53.852778+0000", 2024-09-14T06:29:57.773 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.4148999999999998e-05 2024-09-14T06:29:57.773 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:57.773 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:57.773 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:29:57.773 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:53.853032+0000", 2024-09-14T06:29:57.773 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00025424099999999999 2024-09-14T06:29:57.773 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:57.773 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:29:57.773 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:57.774 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:57.774 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:57.774 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:73936 3.a 3:529fd2a8:::rbd_data.112d4c988ca3.00000000000000b5:1d [read 0~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-14T06:29:57.774 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:53.832060+0000", 2024-09-14T06:29:57.774 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 3.8692182320000001, 2024-09-14T06:29:57.774 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.021161967, 2024-09-14T06:29:57.774 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:29:57.774 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:29:57.774 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:29:57.774 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:29:57.774 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:29:57.774 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:57.774 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 73936 2024-09-14T06:29:57.775 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:57.775 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:29:57.775 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:57.775 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:29:57.775 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:53.832060+0000", 2024-09-14T06:29:57.775 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:57.775 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:57.775 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:57.775 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:29:57.775 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:53.832060+0000", 2024-09-14T06:29:57.775 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:57.775 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:57.775 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:57.776 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:29:57.776 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:53.832060+0000", 2024-09-14T06:29:57.776 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.63e-07 2024-09-14T06:29:57.776 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:57.776 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:57.776 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:29:57.776 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:53.832062+0000", 2024-09-14T06:29:57.776 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.125e-06 2024-09-14T06:29:57.776 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:57.776 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:57.776 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:29:57.776 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:53.832069+0000", 2024-09-14T06:29:57.776 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.3990000000000001e-06 2024-09-14T06:29:57.776 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:57.777 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:57.777 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:29:57.777 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:53.832074+0000", 2024-09-14T06:29:57.777 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.0479999999999998e-06 2024-09-14T06:29:57.777 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:57.777 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:57.777 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:29:57.777 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:53.853060+0000", 2024-09-14T06:29:57.777 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.020986707 2024-09-14T06:29:57.777 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:57.777 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:57.777 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:29:57.777 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:53.853139+0000", 2024-09-14T06:29:57.777 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.8636999999999999e-05 2024-09-14T06:29:57.778 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:57.778 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:57.778 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:29:57.778 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:53.853222+0000", 2024-09-14T06:29:57.778 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.2588000000000003e-05 2024-09-14T06:29:57.778 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:57.778 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:29:57.778 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:57.778 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:57.778 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:57.778 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:73937 3.a 3:529fd2a8:::rbd_data.112d4c988ca3.00000000000000b5:1d [read 180224~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-14T06:29:57.778 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:53.832084+0000", 2024-09-14T06:29:57.778 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 3.8691938449999999, 2024-09-14T06:29:57.779 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.021667265000000002, 2024-09-14T06:29:57.779 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:29:57.779 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:29:57.779 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:29:57.779 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:29:57.779 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:29:57.779 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:57.779 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 73937 2024-09-14T06:29:57.779 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:57.779 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:29:57.779 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:57.779 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:29:57.779 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:53.832084+0000", 2024-09-14T06:29:57.779 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:57.780 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:57.780 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:57.780 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:29:57.780 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:53.832084+0000", 2024-09-14T06:29:57.780 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:57.780 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:57.780 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:57.780 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:29:57.780 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:53.832085+0000", 2024-09-14T06:29:57.780 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.3700000000000001e-07 2024-09-14T06:29:57.780 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:57.780 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:57.780 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:29:57.781 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:53.832087+0000", 2024-09-14T06:29:57.781 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.3779999999999999e-06 2024-09-14T06:29:57.781 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:57.781 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:57.781 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:29:57.781 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:53.832096+0000", 2024-09-14T06:29:57.781 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.8270000000000004e-06 2024-09-14T06:29:57.781 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:57.781 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:57.781 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:29:57.781 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:53.832110+0000", 2024-09-14T06:29:57.781 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.4229999999999999e-05 2024-09-14T06:29:57.781 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:57.782 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:57.782 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:29:57.782 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:53.853249+0000", 2024-09-14T06:29:57.782 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.021139128 2024-09-14T06:29:57.782 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:57.782 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:57.782 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:29:57.782 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:53.853314+0000", 2024-09-14T06:29:57.782 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.4988999999999997e-05 2024-09-14T06:29:57.782 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:57.782 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:57.782 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:29:57.782 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:53.853751+0000", 2024-09-14T06:29:57.782 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00043727600000000001 2024-09-14T06:29:57.783 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:57.783 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:29:57.783 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:57.783 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:57.783 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:57.783 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:73953 3.a 3:57f2eddd:::rbd_data.112d4c988ca3.00000000000000bb:1d [read 0~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-14T06:29:57.783 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:53.833064+0000", 2024-09-14T06:29:57.783 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 3.8682141649999999, 2024-09-14T06:29:57.783 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.020900548000000001, 2024-09-14T06:29:57.783 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:29:57.783 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:29:57.784 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:29:57.784 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:29:57.784 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:29:57.784 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:57.784 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 73953 2024-09-14T06:29:57.784 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:57.784 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:29:57.784 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:57.784 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:29:57.784 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:53.833064+0000", 2024-09-14T06:29:57.785 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:57.785 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:57.785 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:57.785 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:29:57.785 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:53.833064+0000", 2024-09-14T06:29:57.785 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:57.785 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:57.785 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:57.785 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:29:57.785 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:53.833065+0000", 2024-09-14T06:29:57.785 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.1659999999999999e-06 2024-09-14T06:29:57.785 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:57.785 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:57.786 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:29:57.786 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:53.833070+0000", 2024-09-14T06:29:57.786 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.2519999999999999e-06 2024-09-14T06:29:57.786 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:57.786 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:57.786 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:29:57.786 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:53.833082+0000", 2024-09-14T06:29:57.786 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.2333e-05 2024-09-14T06:29:57.786 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:57.786 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:57.786 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:29:57.786 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:53.833090+0000", 2024-09-14T06:29:57.787 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.8150000000000006e-06 2024-09-14T06:29:57.787 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:57.787 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:57.787 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:29:57.787 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:53.853781+0000", 2024-09-14T06:29:57.787 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.020690664000000001 2024-09-14T06:29:57.787 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:57.787 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:57.787 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:29:57.787 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:53.853859+0000", 2024-09-14T06:29:57.787 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.7614999999999995e-05 2024-09-14T06:29:57.787 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:57.787 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:57.787 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:29:57.788 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:53.853964+0000", 2024-09-14T06:29:57.788 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.000105703 2024-09-14T06:29:57.788 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:57.788 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:29:57.788 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:57.788 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:57.788 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:57.788 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:73954 3.a 3:57f2eddd:::rbd_data.112d4c988ca3.00000000000000bb:1d [read 32768~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-14T06:29:57.788 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:53.833104+0000", 2024-09-14T06:29:57.788 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 3.8681738770000003, 2024-09-14T06:29:57.788 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.021252760999999998, 2024-09-14T06:29:57.788 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:29:57.789 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:29:57.789 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:29:57.789 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:29:57.789 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:29:57.789 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:57.789 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 73954 2024-09-14T06:29:57.789 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:57.789 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:29:57.789 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:57.789 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:29:57.789 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:53.833104+0000", 2024-09-14T06:29:57.789 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:57.790 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:57.790 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:57.790 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:29:57.790 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:53.833104+0000", 2024-09-14T06:29:57.790 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:57.790 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:57.790 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:57.790 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:29:57.790 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:53.833105+0000", 2024-09-14T06:29:57.790 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.1799999999999995e-07 2024-09-14T06:29:57.790 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:57.790 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:57.790 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:29:57.791 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:53.833108+0000", 2024-09-14T06:29:57.791 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.1030000000000002e-06 2024-09-14T06:29:57.791 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:57.791 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:57.791 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:29:57.791 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:53.833115+0000", 2024-09-14T06:29:57.791 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.2930000000000003e-06 2024-09-14T06:29:57.791 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:57.791 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:57.791 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:29:57.791 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:53.833120+0000", 2024-09-14T06:29:57.791 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.3199999999999999e-06 2024-09-14T06:29:57.791 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:57.791 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:57.792 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:29:57.792 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:53.853995+0000", 2024-09-14T06:29:57.792 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.020875210000000002 2024-09-14T06:29:57.792 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:57.792 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:57.792 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:29:57.792 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:53.854085+0000", 2024-09-14T06:29:57.792 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.9339999999999995e-05 2024-09-14T06:29:57.792 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:57.792 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:57.793 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:29:57.793 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:53.854357+0000", 2024-09-14T06:29:57.793 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00027197700000000002 2024-09-14T06:29:57.793 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:57.793 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:29:57.793 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:57.793 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:57.793 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:57.793 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:73955 3.a 3:57f2eddd:::rbd_data.112d4c988ca3.00000000000000bb:1d [read 65536~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-14T06:29:57.793 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:53.833128+0000", 2024-09-14T06:29:57.793 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 3.8681494590000001, 2024-09-14T06:29:57.793 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.021578482999999999, 2024-09-14T06:29:57.794 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:29:57.794 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:29:57.794 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:29:57.794 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:29:57.794 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:29:57.794 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:57.794 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 73955 2024-09-14T06:29:57.794 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:57.794 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:29:57.794 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:57.794 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:29:57.794 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:53.833128+0000", 2024-09-14T06:29:57.794 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:57.794 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:57.794 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:57.795 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:29:57.795 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:53.833128+0000", 2024-09-14T06:29:57.795 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:57.795 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:57.795 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:57.795 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:29:57.795 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:53.833129+0000", 2024-09-14T06:29:57.795 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.2199999999999999e-07 2024-09-14T06:29:57.795 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:57.795 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:57.795 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:29:57.795 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:53.833131+0000", 2024-09-14T06:29:57.795 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.5009999999999999e-06 2024-09-14T06:29:57.796 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:57.796 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:57.796 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:29:57.796 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:53.833138+0000", 2024-09-14T06:29:57.796 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.3999999999999997e-06 2024-09-14T06:29:57.796 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:57.796 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:57.796 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:29:57.796 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:53.833142+0000", 2024-09-14T06:29:57.796 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.6160000000000004e-06 2024-09-14T06:29:57.796 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:57.796 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:57.796 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:29:57.797 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:53.854389+0000", 2024-09-14T06:29:57.797 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.021246273999999999 2024-09-14T06:29:57.797 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:57.797 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:57.797 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:29:57.797 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:53.854462+0000", 2024-09-14T06:29:57.797 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.2953000000000004e-05 2024-09-14T06:29:57.797 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:57.797 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:57.797 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:29:57.797 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:53.854707+0000", 2024-09-14T06:29:57.797 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.000245317 2024-09-14T06:29:57.797 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:57.797 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:29:57.798 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:57.798 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:57.798 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:57.798 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:73956 3.a 3:57f2eddd:::rbd_data.112d4c988ca3.00000000000000bb:1d [read 98304~16384 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-09-14T06:29:57.798 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:53.833149+0000", 2024-09-14T06:29:57.798 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 3.8681288409999999, 2024-09-14T06:29:57.798 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.021918799999999999, 2024-09-14T06:29:57.798 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:29:57.798 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:29:57.798 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:29:57.798 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:29:57.799 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:29:57.799 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:57.799 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 73956 2024-09-14T06:29:57.799 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:57.799 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:29:57.799 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:57.799 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:29:57.799 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:53.833149+0000", 2024-09-14T06:29:57.799 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:57.799 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:57.799 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:57.799 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:29:57.799 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:53.833149+0000", 2024-09-14T06:29:57.800 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:57.800 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:57.800 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:57.800 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:29:57.800 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:53.833149+0000", 2024-09-14T06:29:57.800 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.0400000000000002e-07 2024-09-14T06:29:57.800 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:57.800 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:57.800 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:29:57.800 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:53.833152+0000", 2024-09-14T06:29:57.800 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.323e-06 2024-09-14T06:29:57.800 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:57.800 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:57.800 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:29:57.801 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:53.833158+0000", 2024-09-14T06:29:57.801 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.8599999999999998e-06 2024-09-14T06:29:57.801 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:57.801 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:57.801 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:29:57.801 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:53.833162+0000", 2024-09-14T06:29:57.801 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.707e-06 2024-09-14T06:29:57.801 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:57.801 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:57.801 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:29:57.801 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:53.854737+0000", 2024-09-14T06:29:57.801 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.021575014 2024-09-14T06:29:57.801 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:57.801 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:57.802 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:29:57.802 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:53.854814+0000", 2024-09-14T06:29:57.802 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.6972999999999996e-05 2024-09-14T06:29:57.802 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:57.802 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:57.802 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:29:57.802 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:53.855068+0000", 2024-09-14T06:29:57.802 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00025351900000000001 2024-09-14T06:29:57.802 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:57.802 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:29:57.802 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:57.802 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:57.802 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:29:57.803 INFO:teuthology.orchestra.run.smithi138.stdout:} 2024-09-14T06:29:57.803 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-14T06:29:57.803 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 0 1 2024-09-14T06:29:57.805 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-14T06:29:57.805 INFO:tasks.ceph.osd.4.smithi138.stderr:2024-09-14T06:29:57.803+0000 7f6c653c8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33615) UID: 0 2024-09-14T06:29:57.905 INFO:tasks.ceph.osd.7.smithi138.stderr:2024-09-14T06:29:57.904+0000 7f3c6a21d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33623) UID: 0 2024-09-14T06:29:57.953 INFO:teuthology.orchestra.run.smithi138.stdout:{ 2024-09-14T06:29:57.953 INFO:teuthology.orchestra.run.smithi138.stdout: "ops": [], 2024-09-14T06:29:57.954 INFO:teuthology.orchestra.run.smithi138.stdout: "num_ops": 0 2024-09-14T06:29:57.954 INFO:teuthology.orchestra.run.smithi138.stdout:} 2024-09-14T06:29:57.962 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-14T06:29:58.005 INFO:tasks.ceph.osd.5.smithi138.stderr:2024-09-14T06:29:58.004+0000 7fe13746f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33618) UID: 0 2024-09-14T06:29:58.106 INFO:teuthology.orchestra.run.smithi138.stdout:{ 2024-09-14T06:29:58.106 INFO:teuthology.orchestra.run.smithi138.stdout: "ops": [], 2024-09-14T06:29:58.106 INFO:teuthology.orchestra.run.smithi138.stdout: "complaint_time": 30, 2024-09-14T06:29:58.106 INFO:teuthology.orchestra.run.smithi138.stdout: "num_blocked_ops": 0 2024-09-14T06:29:58.106 INFO:teuthology.orchestra.run.smithi138.stdout:} 2024-09-14T06:29:58.107 INFO:tasks.ceph.osd.3.smithi094.stderr:2024-09-14T06:29:58.105+0000 7feeeafe8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33655) UID: 0 2024-09-14T06:29:58.114 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-14T06:29:58.207 INFO:tasks.ceph.osd.3.smithi094.stderr:2024-09-14T06:29:58.206+0000 7feeeafe8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33655) UID: 0 2024-09-14T06:29:58.217 INFO:teuthology.orchestra.run.smithi138.stdout:{ 2024-09-14T06:29:58.217 INFO:teuthology.orchestra.run.smithi138.stdout: "size": 20, 2024-09-14T06:29:58.217 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 600, 2024-09-14T06:29:58.217 INFO:teuthology.orchestra.run.smithi138.stdout: "ops": [ 2024-09-14T06:29:58.218 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:58.218 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:53244 3.f 3:f71f51a7:::rbd_data.112dca2f6ae1.0000000000000015:17 [read 1863680~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:29:58.218 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:44.369399+0000", 2024-09-14T06:29:58.218 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 13.846461618999999, 2024-09-14T06:29:58.218 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047291189999999997, 2024-09-14T06:29:58.218 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:29:58.218 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:29:58.218 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:29:58.218 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:29:58.218 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:29:58.218 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:58.218 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 53244 2024-09-14T06:29:58.218 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:58.219 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:29:58.219 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:58.219 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:29:58.219 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369399+0000", 2024-09-14T06:29:58.219 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:58.219 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:58.219 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:58.219 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:29:58.219 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369399+0000", 2024-09-14T06:29:58.219 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:58.219 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:58.219 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:58.219 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:29:58.220 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369400+0000", 2024-09-14T06:29:58.220 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.0699999999999999e-07 2024-09-14T06:29:58.220 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:58.220 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:58.220 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:29:58.220 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369401+0000", 2024-09-14T06:29:58.220 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.4300000000000001e-06 2024-09-14T06:29:58.220 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:58.220 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:58.220 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:29:58.220 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369404+0000", 2024-09-14T06:29:58.220 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.061e-06 2024-09-14T06:29:58.220 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:58.220 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:58.221 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:29:58.221 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369407+0000", 2024-09-14T06:29:58.221 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.6139999999999999e-06 2024-09-14T06:29:58.221 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:58.221 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:58.221 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:29:58.221 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.416517+0000", 2024-09-14T06:29:58.221 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047110338000000002 2024-09-14T06:29:58.221 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:58.221 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:58.222 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:29:58.222 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.416616+0000", 2024-09-14T06:29:58.222 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.8651000000000001e-05 2024-09-14T06:29:58.222 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:58.222 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:58.222 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:29:58.222 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.416691+0000", 2024-09-14T06:29:58.222 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.4888999999999994e-05 2024-09-14T06:29:58.222 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:58.222 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:29:58.222 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:58.222 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:58.222 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:58.222 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:53245 3.f 3:f71f51a7:::rbd_data.112dca2f6ae1.0000000000000015:17 [read 1888256~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:29:58.222 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:44.369410+0000", 2024-09-14T06:29:58.223 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 13.846451331000001, 2024-09-14T06:29:58.223 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047399275999999997, 2024-09-14T06:29:58.223 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:29:58.223 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:29:58.223 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:29:58.223 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:29:58.223 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:29:58.223 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:58.223 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 53245 2024-09-14T06:29:58.223 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:58.223 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:29:58.223 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:58.223 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:29:58.223 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369410+0000", 2024-09-14T06:29:58.223 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:58.224 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:58.224 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:58.224 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:29:58.224 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369410+0000", 2024-09-14T06:29:58.224 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:58.224 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:58.224 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:58.224 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:29:58.224 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369410+0000", 2024-09-14T06:29:58.224 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.1899999999999999e-07 2024-09-14T06:29:58.224 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:58.224 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:58.225 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:29:58.225 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369411+0000", 2024-09-14T06:29:58.225 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.0300000000000001e-06 2024-09-14T06:29:58.225 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:58.225 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:58.225 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:29:58.225 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369414+0000", 2024-09-14T06:29:58.225 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.1099999999999999e-06 2024-09-14T06:29:58.225 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:58.225 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:58.225 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:29:58.225 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369417+0000", 2024-09-14T06:29:58.226 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.6189999999999998e-06 2024-09-14T06:29:58.226 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:58.226 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:58.226 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:29:58.226 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.416718+0000", 2024-09-14T06:29:58.226 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047301322999999999 2024-09-14T06:29:58.226 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:58.226 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:58.226 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:29:58.226 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.416759+0000", 2024-09-14T06:29:58.226 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.1009e-05 2024-09-14T06:29:58.226 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:58.226 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:58.227 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:29:58.227 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.416809+0000", 2024-09-14T06:29:58.227 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.9966000000000002e-05 2024-09-14T06:29:58.227 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:58.227 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:29:58.227 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:58.227 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:58.227 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:58.227 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:53246 3.f 3:f71f51a7:::rbd_data.112dca2f6ae1.0000000000000015:17 [read 1912832~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:29:58.227 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:44.369420+0000", 2024-09-14T06:29:58.227 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 13.846441387, 2024-09-14T06:29:58.227 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047529463000000001, 2024-09-14T06:29:58.227 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:29:58.227 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:29:58.228 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:29:58.228 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:29:58.228 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:29:58.228 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:58.228 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 53246 2024-09-14T06:29:58.228 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:58.228 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:29:58.228 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:58.228 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:29:58.228 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369420+0000", 2024-09-14T06:29:58.228 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:58.228 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:58.228 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:58.229 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:29:58.229 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369420+0000", 2024-09-14T06:29:58.229 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:58.229 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:58.229 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:58.229 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:29:58.229 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369420+0000", 2024-09-14T06:29:58.229 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.1299999999999999e-07 2024-09-14T06:29:58.229 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:58.229 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:58.229 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:29:58.229 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369425+0000", 2024-09-14T06:29:58.230 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.0130000000000001e-06 2024-09-14T06:29:58.230 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:58.230 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:58.230 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:29:58.230 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369428+0000", 2024-09-14T06:29:58.230 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.349e-06 2024-09-14T06:29:58.230 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:58.230 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:58.230 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:29:58.230 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369431+0000", 2024-09-14T06:29:58.230 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.7389999999999999e-06 2024-09-14T06:29:58.230 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:58.230 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:58.230 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:29:58.231 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.416834+0000", 2024-09-14T06:29:58.231 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047402669000000001 2024-09-14T06:29:58.231 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:58.231 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:58.231 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:29:58.231 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.416882+0000", 2024-09-14T06:29:58.231 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.8068999999999997e-05 2024-09-14T06:29:58.231 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:58.231 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:58.231 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:29:58.231 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.416949+0000", 2024-09-14T06:29:58.232 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.7410999999999995e-05 2024-09-14T06:29:58.232 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:58.232 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:29:58.232 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:58.232 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:58.232 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:58.232 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:53247 3.f 3:f71f51a7:::rbd_data.112dca2f6ae1.0000000000000015:17 [read 1937408~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:29:58.232 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:44.369434+0000", 2024-09-14T06:29:58.232 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 13.846426622999999, 2024-09-14T06:29:58.232 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047652264999999999, 2024-09-14T06:29:58.232 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:29:58.232 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:29:58.233 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:29:58.233 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:29:58.233 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:29:58.233 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:58.233 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 53247 2024-09-14T06:29:58.233 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:58.233 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:29:58.233 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:58.233 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:29:58.233 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369434+0000", 2024-09-14T06:29:58.233 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:58.233 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:58.233 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:58.234 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:29:58.234 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369434+0000", 2024-09-14T06:29:58.234 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:58.234 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:58.234 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:58.234 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:29:58.234 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369435+0000", 2024-09-14T06:29:58.234 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.2100000000000001e-07 2024-09-14T06:29:58.234 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:58.234 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:58.234 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:29:58.234 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369436+0000", 2024-09-14T06:29:58.234 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.122e-06 2024-09-14T06:29:58.235 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:58.235 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:58.235 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:29:58.235 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369439+0000", 2024-09-14T06:29:58.235 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.0400000000000001e-06 2024-09-14T06:29:58.235 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:58.235 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:58.235 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:29:58.235 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369441+0000", 2024-09-14T06:29:58.235 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.728e-06 2024-09-14T06:29:58.235 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:58.235 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:58.235 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:29:58.236 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.416977+0000", 2024-09-14T06:29:58.236 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047535344 2024-09-14T06:29:58.236 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:58.236 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:58.236 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:29:58.236 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.417029+0000", 2024-09-14T06:29:58.236 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.2037999999999999e-05 2024-09-14T06:29:58.236 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:58.236 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:58.236 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:29:58.236 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.417087+0000", 2024-09-14T06:29:58.236 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.7772000000000002e-05 2024-09-14T06:29:58.236 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:58.237 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:29:58.237 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:58.237 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:58.237 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:58.237 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:53248 3.f 3:f71f51a7:::rbd_data.112dca2f6ae1.0000000000000015:17 [read 1961984~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:29:58.237 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:44.369445+0000", 2024-09-14T06:29:58.237 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 13.846416071, 2024-09-14T06:29:58.237 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047798489, 2024-09-14T06:29:58.237 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:29:58.237 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:29:58.237 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:29:58.237 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:29:58.237 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:29:58.238 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:58.238 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 53248 2024-09-14T06:29:58.238 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:58.238 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:29:58.238 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:58.238 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:29:58.238 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369445+0000", 2024-09-14T06:29:58.238 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:58.238 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:58.238 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:58.238 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:29:58.238 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369445+0000", 2024-09-14T06:29:58.238 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:58.239 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:58.239 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:58.239 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:29:58.239 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369445+0000", 2024-09-14T06:29:58.239 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.0699999999999999e-07 2024-09-14T06:29:58.239 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:58.239 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:58.239 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:29:58.239 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369446+0000", 2024-09-14T06:29:58.239 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.1000000000000001e-06 2024-09-14T06:29:58.239 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:58.239 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:58.239 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:29:58.240 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369449+0000", 2024-09-14T06:29:58.240 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.1039999999999998e-06 2024-09-14T06:29:58.240 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:58.240 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:58.240 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:29:58.240 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369452+0000", 2024-09-14T06:29:58.240 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.6199999999999999e-06 2024-09-14T06:29:58.240 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:58.240 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:58.240 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:29:58.240 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.417114+0000", 2024-09-14T06:29:58.240 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047661901999999999 2024-09-14T06:29:58.240 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:58.241 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:58.241 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:29:58.241 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.417181+0000", 2024-09-14T06:29:58.241 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.7385999999999995e-05 2024-09-14T06:29:58.241 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:58.241 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:58.241 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:29:58.241 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.417243+0000", 2024-09-14T06:29:58.241 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.2169999999999996e-05 2024-09-14T06:29:58.241 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:58.241 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:29:58.241 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:58.242 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:58.242 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:58.242 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:53249 3.f 3:f71f51a7:::rbd_data.112dca2f6ae1.0000000000000015:17 [read 1986560~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:29:58.242 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:44.369455+0000", 2024-09-14T06:29:58.242 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 13.84640576, 2024-09-14T06:29:58.242 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047890493999999999, 2024-09-14T06:29:58.242 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:29:58.242 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:29:58.242 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:29:58.242 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:29:58.243 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:29:58.243 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:58.243 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 53249 2024-09-14T06:29:58.243 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:58.243 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:29:58.243 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:58.243 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:29:58.243 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369455+0000", 2024-09-14T06:29:58.243 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:58.243 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:58.243 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:58.243 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:29:58.243 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369455+0000", 2024-09-14T06:29:58.243 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:58.244 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:58.244 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:58.244 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:29:58.244 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369455+0000", 2024-09-14T06:29:58.244 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.0800000000000001e-07 2024-09-14T06:29:58.244 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:58.244 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:58.244 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:29:58.244 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369456+0000", 2024-09-14T06:29:58.244 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.0729999999999999e-06 2024-09-14T06:29:58.244 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:58.244 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:58.244 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:29:58.245 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369460+0000", 2024-09-14T06:29:58.245 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.027e-06 2024-09-14T06:29:58.245 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:58.245 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:58.245 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:29:58.245 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369462+0000", 2024-09-14T06:29:58.245 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.5320000000000001e-06 2024-09-14T06:29:58.245 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:58.245 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:58.245 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:29:58.245 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.417257+0000", 2024-09-14T06:29:58.245 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.04779489 2024-09-14T06:29:58.245 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:58.246 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:58.246 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:29:58.246 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.417296+0000", 2024-09-14T06:29:58.246 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.8658000000000002e-05 2024-09-14T06:29:58.246 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:58.246 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:58.246 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:29:58.246 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.417346+0000", 2024-09-14T06:29:58.246 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.0105999999999998e-05 2024-09-14T06:29:58.246 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:58.246 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:29:58.246 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:58.246 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:58.246 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:58.247 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:53250 3.f 3:f71f51a7:::rbd_data.112dca2f6ae1.0000000000000015:17 [read 2011136~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:29:58.247 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:44.369465+0000", 2024-09-14T06:29:58.247 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 13.846395790000001, 2024-09-14T06:29:58.247 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.048032230000000002, 2024-09-14T06:29:58.247 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:29:58.247 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:29:58.247 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:29:58.247 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:29:58.247 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:29:58.247 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:58.247 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 53250 2024-09-14T06:29:58.247 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:58.247 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:29:58.247 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:58.248 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:29:58.248 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369465+0000", 2024-09-14T06:29:58.248 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:58.248 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:58.248 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:58.248 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:29:58.248 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369465+0000", 2024-09-14T06:29:58.248 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:58.248 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:58.248 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:58.248 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:29:58.249 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369465+0000", 2024-09-14T06:29:58.249 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.2100000000000001e-07 2024-09-14T06:29:58.249 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:58.249 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:58.249 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:29:58.249 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369466+0000", 2024-09-14T06:29:58.249 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.0640000000000001e-06 2024-09-14T06:29:58.249 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:58.249 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:58.250 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:29:58.250 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369469+0000", 2024-09-14T06:29:58.250 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.0170000000000001e-06 2024-09-14T06:29:58.250 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:58.250 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:58.250 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:29:58.250 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369472+0000", 2024-09-14T06:29:58.250 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.6189999999999998e-06 2024-09-14T06:29:58.250 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:58.250 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:58.250 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:29:58.250 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.417383+0000", 2024-09-14T06:29:58.250 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047911092000000002 2024-09-14T06:29:58.250 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:58.250 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:58.251 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:29:58.251 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.417430+0000", 2024-09-14T06:29:58.251 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.6898999999999999e-05 2024-09-14T06:29:58.251 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:58.251 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:58.251 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:29:58.251 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.417497+0000", 2024-09-14T06:29:58.251 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.7317999999999995e-05 2024-09-14T06:29:58.251 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:58.251 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:29:58.251 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:58.251 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:58.251 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:58.251 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:53251 3.f 3:f71f51a7:::rbd_data.112dca2f6ae1.0000000000000015:17 [read 2035712~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:29:58.251 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:44.369475+0000", 2024-09-14T06:29:58.252 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 13.846385794, 2024-09-14T06:29:58.252 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.048154542000000002, 2024-09-14T06:29:58.252 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:29:58.252 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:29:58.252 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:29:58.252 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:29:58.252 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:29:58.252 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:58.252 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 53251 2024-09-14T06:29:58.253 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:58.253 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:29:58.253 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:58.253 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:29:58.253 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369475+0000", 2024-09-14T06:29:58.253 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:58.253 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:58.253 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:58.253 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:29:58.253 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369475+0000", 2024-09-14T06:29:58.253 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:58.253 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:58.254 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:58.254 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:29:58.254 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369475+0000", 2024-09-14T06:29:58.254 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.0100000000000001e-07 2024-09-14T06:29:58.254 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:58.254 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:58.254 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:29:58.254 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369476+0000", 2024-09-14T06:29:58.254 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.0580000000000001e-06 2024-09-14T06:29:58.254 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:58.254 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:58.254 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:29:58.254 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369479+0000", 2024-09-14T06:29:58.254 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.9950000000000001e-06 2024-09-14T06:29:58.254 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:58.255 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:58.255 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:29:58.255 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369482+0000", 2024-09-14T06:29:58.255 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.6340000000000001e-06 2024-09-14T06:29:58.255 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:58.255 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:58.255 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:29:58.255 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.417530+0000", 2024-09-14T06:29:58.255 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.048047535000000002 2024-09-14T06:29:58.255 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:58.255 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:58.255 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:29:58.255 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.417574+0000", 2024-09-14T06:29:58.255 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.4255999999999998e-05 2024-09-14T06:29:58.255 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:58.255 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:58.256 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:29:58.256 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.417630+0000", 2024-09-14T06:29:58.256 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.5862999999999997e-05 2024-09-14T06:29:58.256 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:58.256 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:29:58.256 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:58.256 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:58.256 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:58.256 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:53252 3.f 3:f71f51a7:::rbd_data.112dca2f6ae1.0000000000000015:17 [read 2060288~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:29:58.256 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:44.369485+0000", 2024-09-14T06:29:58.256 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 13.846375946, 2024-09-14T06:29:58.256 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.048254116, 2024-09-14T06:29:58.256 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:29:58.256 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:29:58.257 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:29:58.257 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:29:58.257 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:29:58.257 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:58.257 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 53252 2024-09-14T06:29:58.257 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:58.257 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:29:58.257 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:58.257 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:29:58.257 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369485+0000", 2024-09-14T06:29:58.257 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:58.257 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:58.257 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:58.258 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:29:58.258 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369485+0000", 2024-09-14T06:29:58.258 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:58.258 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:58.258 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:58.258 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:29:58.258 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369485+0000", 2024-09-14T06:29:58.258 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.1e-07 2024-09-14T06:29:58.258 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:58.258 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:58.258 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:29:58.258 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369487+0000", 2024-09-14T06:29:58.258 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.42e-06 2024-09-14T06:29:58.258 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:58.259 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:58.259 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:29:58.259 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369490+0000", 2024-09-14T06:29:58.259 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.9639999999999999e-06 2024-09-14T06:29:58.259 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:58.259 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:58.259 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:29:58.259 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369492+0000", 2024-09-14T06:29:58.259 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.5679999999999998e-06 2024-09-14T06:29:58.259 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:58.259 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:58.259 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:29:58.259 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.417645+0000", 2024-09-14T06:29:58.259 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.048152753 2024-09-14T06:29:58.259 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:58.260 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:58.260 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:29:58.260 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.417687+0000", 2024-09-14T06:29:58.260 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.1900999999999997e-05 2024-09-14T06:29:58.260 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:58.260 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:58.260 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:29:58.260 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.417739+0000", 2024-09-14T06:29:58.260 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.2299999999999997e-05 2024-09-14T06:29:58.260 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:58.260 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:29:58.260 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:58.260 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:58.261 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:58.261 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:53253 3.f 3:f71f51a7:::rbd_data.112dca2f6ae1.0000000000000015:17 [read 2084864~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:29:58.261 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:44.369495+0000", 2024-09-14T06:29:58.261 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 13.846365821999999, 2024-09-14T06:29:58.261 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.048366785000000002, 2024-09-14T06:29:58.261 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:29:58.261 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:29:58.261 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:29:58.261 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:29:58.261 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:29:58.261 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:58.261 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 53253 2024-09-14T06:29:58.261 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:58.261 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:29:58.262 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:58.262 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:29:58.262 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369495+0000", 2024-09-14T06:29:58.262 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:58.262 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:58.262 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:58.262 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:29:58.262 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369495+0000", 2024-09-14T06:29:58.262 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:58.262 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:58.262 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:58.263 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:29:58.263 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370388+0000", 2024-09-14T06:29:58.263 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00089287999999999998 2024-09-14T06:29:58.263 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:58.263 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:58.263 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:29:58.263 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370392+0000", 2024-09-14T06:29:58.263 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.2130000000000002e-06 2024-09-14T06:29:58.263 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:58.263 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:58.263 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:29:58.263 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370401+0000", 2024-09-14T06:29:58.263 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.7649999999999999e-06 2024-09-14T06:29:58.264 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:58.264 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:58.264 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:29:58.264 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370406+0000", 2024-09-14T06:29:58.264 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.0159999999999999e-06 2024-09-14T06:29:58.264 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:58.264 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:58.264 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:29:58.264 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.417764+0000", 2024-09-14T06:29:58.264 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047358348000000001 2024-09-14T06:29:58.264 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:58.264 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:58.264 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:29:58.265 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.417806+0000", 2024-09-14T06:29:58.265 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.1149999999999997e-05 2024-09-14T06:29:58.265 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:58.265 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:58.265 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:29:58.265 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.417862+0000", 2024-09-14T06:29:58.265 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.6412999999999997e-05 2024-09-14T06:29:58.265 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:58.265 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:29:58.265 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:58.265 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:58.265 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:58.265 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:53340 3.f 3:f71f51a7:::rbd_data.112dca2f6ae1.0000000000000015:17 [read 1056768~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:29:58.266 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:44.370421+0000", 2024-09-14T06:29:58.266 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 13.845439942, 2024-09-14T06:29:58.266 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047566313999999998, 2024-09-14T06:29:58.266 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:29:58.266 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:29:58.266 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:29:58.266 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:29:58.266 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:29:58.266 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:58.266 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 53340 2024-09-14T06:29:58.266 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:58.266 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:29:58.267 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:58.267 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:29:58.267 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370421+0000", 2024-09-14T06:29:58.267 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:58.267 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:58.267 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:58.267 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:29:58.267 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370421+0000", 2024-09-14T06:29:58.267 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:58.267 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:58.267 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:58.267 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:29:58.267 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370422+0000", 2024-09-14T06:29:58.267 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.3799999999999996e-07 2024-09-14T06:29:58.268 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:58.268 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:58.268 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:29:58.268 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370424+0000", 2024-09-14T06:29:58.268 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.6919999999999998e-06 2024-09-14T06:29:58.268 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:58.268 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:58.268 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:29:58.268 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370432+0000", 2024-09-14T06:29:58.268 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.2439999999999996e-06 2024-09-14T06:29:58.269 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:58.269 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:58.269 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:29:58.269 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370437+0000", 2024-09-14T06:29:58.269 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.5389999999999998e-06 2024-09-14T06:29:58.269 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:58.269 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:58.269 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:29:58.269 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.417888+0000", 2024-09-14T06:29:58.269 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047450432000000001 2024-09-14T06:29:58.269 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:58.269 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:58.269 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:29:58.270 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.417935+0000", 2024-09-14T06:29:58.270 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.7454999999999999e-05 2024-09-14T06:29:58.270 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:58.270 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:58.270 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:29:58.270 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.417987+0000", 2024-09-14T06:29:58.270 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.2213999999999998e-05 2024-09-14T06:29:58.270 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:58.270 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:29:58.270 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:58.270 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:58.271 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:58.271 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:53341 3.f 3:f71f51a7:::rbd_data.112dca2f6ae1.0000000000000015:17 [read 1081344~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:29:58.271 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:44.370446+0000", 2024-09-14T06:29:58.271 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 13.845414669, 2024-09-14T06:29:58.271 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047665723, 2024-09-14T06:29:58.271 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:29:58.271 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:29:58.271 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:29:58.271 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:29:58.271 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:29:58.271 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:58.272 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 53341 2024-09-14T06:29:58.272 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:58.272 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:29:58.272 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:58.272 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:29:58.272 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370446+0000", 2024-09-14T06:29:58.272 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:58.272 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:58.272 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:58.272 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:29:58.273 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370446+0000", 2024-09-14T06:29:58.273 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:58.273 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:58.273 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:58.273 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:29:58.274 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370447+0000", 2024-09-14T06:29:58.274 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.6100000000000002e-07 2024-09-14T06:29:58.274 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:58.274 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:58.274 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:29:58.274 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370448+0000", 2024-09-14T06:29:58.274 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.7370000000000001e-06 2024-09-14T06:29:58.274 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:58.274 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:58.274 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:29:58.274 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370462+0000", 2024-09-14T06:29:58.274 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.3227000000000001e-05 2024-09-14T06:29:58.274 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:58.275 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:58.275 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:29:58.275 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370467+0000", 2024-09-14T06:29:58.275 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.6540000000000004e-06 2024-09-14T06:29:58.275 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:58.275 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:58.275 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:29:58.275 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.418013+0000", 2024-09-14T06:29:58.275 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047545325999999999 2024-09-14T06:29:58.275 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:58.275 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:58.275 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:29:58.275 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.418054+0000", 2024-09-14T06:29:58.276 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.1696999999999997e-05 2024-09-14T06:29:58.276 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:58.276 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:58.276 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:29:58.276 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.418112+0000", 2024-09-14T06:29:58.276 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.7720999999999998e-05 2024-09-14T06:29:58.276 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:58.276 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:29:58.276 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:58.276 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:58.276 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:58.276 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:53342 3.f 3:f71f51a7:::rbd_data.112dca2f6ae1.0000000000000015:17 [read 1105920~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:29:58.276 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:44.370475+0000", 2024-09-14T06:29:58.277 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 13.845385608000001, 2024-09-14T06:29:58.277 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047748848000000003, 2024-09-14T06:29:58.277 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:29:58.277 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:29:58.277 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:29:58.277 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:29:58.277 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:29:58.277 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:58.277 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 53342 2024-09-14T06:29:58.277 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:58.277 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:29:58.277 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:58.277 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:29:58.278 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370475+0000", 2024-09-14T06:29:58.278 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:58.278 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:58.278 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:58.278 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:29:58.278 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370475+0000", 2024-09-14T06:29:58.278 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:58.278 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:58.278 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:58.278 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:29:58.278 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370476+0000", 2024-09-14T06:29:58.278 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.4700000000000002e-07 2024-09-14T06:29:58.278 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:58.279 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:58.279 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:29:58.279 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370478+0000", 2024-09-14T06:29:58.279 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.5890000000000001e-06 2024-09-14T06:29:58.279 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:58.279 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:58.279 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:29:58.279 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370486+0000", 2024-09-14T06:29:58.279 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.3470000000000001e-06 2024-09-14T06:29:58.279 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:58.279 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:58.279 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:29:58.279 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370491+0000", 2024-09-14T06:29:58.280 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.5720000000000002e-06 2024-09-14T06:29:58.280 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:58.280 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:58.280 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:29:58.280 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.418135+0000", 2024-09-14T06:29:58.280 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047644094999999997 2024-09-14T06:29:58.280 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:58.280 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:58.280 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:29:58.280 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.418174+0000", 2024-09-14T06:29:58.280 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.8204000000000002e-05 2024-09-14T06:29:58.280 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:58.281 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:58.281 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:29:58.281 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.418224+0000", 2024-09-14T06:29:58.281 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.0593999999999997e-05 2024-09-14T06:29:58.281 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:58.281 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:29:58.281 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:58.281 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:58.281 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:58.281 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:53343 3.f 3:f71f51a7:::rbd_data.112dca2f6ae1.0000000000000015:17 [read 1130496~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:29:58.281 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:44.370498+0000", 2024-09-14T06:29:58.281 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 13.845362732, 2024-09-14T06:29:58.281 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047841574999999997, 2024-09-14T06:29:58.282 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:29:58.282 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:29:58.282 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:29:58.282 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:29:58.282 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:29:58.282 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:58.282 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 53343 2024-09-14T06:29:58.282 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:58.282 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:29:58.282 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:58.282 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:29:58.282 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370498+0000", 2024-09-14T06:29:58.282 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:58.282 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:58.283 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:58.283 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:29:58.283 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370498+0000", 2024-09-14T06:29:58.283 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:58.283 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:58.283 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:58.283 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:29:58.283 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370499+0000", 2024-09-14T06:29:58.283 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.9999999999999998e-07 2024-09-14T06:29:58.283 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:58.283 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:58.283 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:29:58.283 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370501+0000", 2024-09-14T06:29:58.284 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.0310000000000001e-06 2024-09-14T06:29:58.284 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:58.284 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:58.284 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:29:58.284 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370507+0000", 2024-09-14T06:29:58.284 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.4169999999999997e-06 2024-09-14T06:29:58.284 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:58.284 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:58.284 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:29:58.284 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370516+0000", 2024-09-14T06:29:58.285 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.7190000000000007e-06 2024-09-14T06:29:58.285 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:58.285 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:58.285 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:29:58.285 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.418247+0000", 2024-09-14T06:29:58.285 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047731190999999999 2024-09-14T06:29:58.285 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:58.285 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:58.285 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:29:58.285 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.418286+0000", 2024-09-14T06:29:58.285 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.8988e-05 2024-09-14T06:29:58.285 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:58.285 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:58.286 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:29:58.286 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.418340+0000", 2024-09-14T06:29:58.286 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.3829000000000002e-05 2024-09-14T06:29:58.286 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:58.286 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:29:58.286 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:58.286 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:58.286 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:58.286 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:53344 3.f 3:f71f51a7:::rbd_data.112dca2f6ae1.0000000000000015:17 [read 1155072~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:29:58.286 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:44.370524+0000", 2024-09-14T06:29:58.286 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 13.845336961999999, 2024-09-14T06:29:58.286 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047972262000000002, 2024-09-14T06:29:58.286 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:29:58.286 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:29:58.287 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:29:58.287 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:29:58.287 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:29:58.287 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:58.287 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 53344 2024-09-14T06:29:58.287 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:58.287 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:29:58.287 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:58.287 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:29:58.287 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370524+0000", 2024-09-14T06:29:58.287 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:58.287 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:58.287 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:58.287 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:29:58.288 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370524+0000", 2024-09-14T06:29:58.288 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:58.288 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:58.288 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:58.288 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:29:58.288 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370618+0000", 2024-09-14T06:29:58.288 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.4425999999999993e-05 2024-09-14T06:29:58.288 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:58.288 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:58.288 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:29:58.288 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370622+0000", 2024-09-14T06:29:58.288 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.3519999999999998e-06 2024-09-14T06:29:58.289 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:58.289 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:58.289 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:29:58.289 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370631+0000", 2024-09-14T06:29:58.289 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.9299999999999992e-06 2024-09-14T06:29:58.289 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:58.289 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:58.289 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:29:58.289 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370637+0000", 2024-09-14T06:29:58.289 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.3269999999999997e-06 2024-09-14T06:29:58.289 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:58.290 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:58.290 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:29:58.290 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.418372+0000", 2024-09-14T06:29:58.290 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047734764999999998 2024-09-14T06:29:58.290 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:58.290 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:58.290 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:29:58.290 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.418431+0000", 2024-09-14T06:29:58.290 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.8696999999999997e-05 2024-09-14T06:29:58.290 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:58.290 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:58.290 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:29:58.290 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.418496+0000", 2024-09-14T06:29:58.290 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.5765000000000005e-05 2024-09-14T06:29:58.291 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:58.291 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:29:58.291 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:58.291 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:58.291 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:58.291 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:53345 3.f 3:f71f51a7:::rbd_data.112dca2f6ae1.0000000000000015:17 [read 1179648~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:29:58.291 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:44.370647+0000", 2024-09-14T06:29:58.291 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 13.845213494999999, 2024-09-14T06:29:58.291 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047953775999999997, 2024-09-14T06:29:58.291 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:29:58.291 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:29:58.291 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:29:58.292 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:29:58.292 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:29:58.292 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:58.292 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 53345 2024-09-14T06:29:58.292 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:58.292 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:29:58.292 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:58.292 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:29:58.292 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370647+0000", 2024-09-14T06:29:58.292 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:58.292 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:58.292 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:58.292 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:29:58.293 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370647+0000", 2024-09-14T06:29:58.293 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:58.293 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:58.293 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:58.293 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:29:58.293 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.372346+0000", 2024-09-14T06:29:58.293 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.001698284 2024-09-14T06:29:58.293 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:58.293 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:58.293 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:29:58.293 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.372351+0000", 2024-09-14T06:29:58.293 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.8890000000000001e-06 2024-09-14T06:29:58.294 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:58.294 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:58.294 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:29:58.294 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.372362+0000", 2024-09-14T06:29:58.294 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.0941000000000001e-05 2024-09-14T06:29:58.294 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:58.294 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:58.294 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:29:58.294 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.372369+0000", 2024-09-14T06:29:58.294 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.362e-06 2024-09-14T06:29:58.294 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:58.294 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:58.294 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:29:58.295 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.418510+0000", 2024-09-14T06:29:58.295 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046140914999999998 2024-09-14T06:29:58.295 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:58.295 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:58.295 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:29:58.295 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.418549+0000", 2024-09-14T06:29:58.295 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.8717999999999998e-05 2024-09-14T06:29:58.295 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:58.295 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:58.295 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:29:58.295 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.418601+0000", 2024-09-14T06:29:58.295 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.2667000000000002e-05 2024-09-14T06:29:58.295 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:58.295 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:29:58.295 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:58.296 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:58.296 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:58.296 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:53351 3.f 3:f71f51a7:::rbd_data.112dca2f6ae1.0000000000000015:17 [read 1327104~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:29:58.296 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:44.372499+0000", 2024-09-14T06:29:58.296 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 13.843362374, 2024-09-14T06:29:58.296 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047140596, 2024-09-14T06:29:58.296 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:29:58.296 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:29:58.296 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:29:58.296 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:29:58.296 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:29:58.296 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:58.297 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 53351 2024-09-14T06:29:58.297 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:58.297 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:29:58.297 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:58.297 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:29:58.297 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.372499+0000", 2024-09-14T06:29:58.297 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:58.297 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:58.297 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:58.297 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:29:58.297 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.372499+0000", 2024-09-14T06:29:58.297 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:58.298 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:58.298 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:58.298 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:29:58.298 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.372499+0000", 2024-09-14T06:29:58.298 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.3000000000000001e-07 2024-09-14T06:29:58.298 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:58.298 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:58.298 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:29:58.298 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.372501+0000", 2024-09-14T06:29:58.298 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.0820000000000001e-06 2024-09-14T06:29:58.298 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:58.298 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:58.299 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:29:58.299 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.372513+0000", 2024-09-14T06:29:58.299 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.1610999999999999e-05 2024-09-14T06:29:58.299 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:58.299 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:58.299 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:29:58.299 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.372518+0000", 2024-09-14T06:29:58.299 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.327e-06 2024-09-14T06:29:58.299 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:58.299 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:58.299 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:29:58.299 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.419530+0000", 2024-09-14T06:29:58.299 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047011798 2024-09-14T06:29:58.300 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:58.300 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:58.300 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:29:58.300 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.419578+0000", 2024-09-14T06:29:58.300 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.8446999999999999e-05 2024-09-14T06:29:58.300 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:58.300 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:58.300 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:29:58.300 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.419639+0000", 2024-09-14T06:29:58.300 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.0900999999999998e-05 2024-09-14T06:29:58.300 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:58.300 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:29:58.301 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:58.301 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:58.301 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:58.301 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:53352 3.f 3:f71f51a7:::rbd_data.112dca2f6ae1.0000000000000015:17 [read 1351680~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:29:58.301 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:44.372525+0000", 2024-09-14T06:29:58.301 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 13.84333646, 2024-09-14T06:29:58.301 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047259847000000001, 2024-09-14T06:29:58.301 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:29:58.301 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:29:58.301 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:29:58.301 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:29:58.301 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:29:58.301 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:58.301 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 53352 2024-09-14T06:29:58.302 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:58.302 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:29:58.302 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:58.302 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:29:58.302 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.372525+0000", 2024-09-14T06:29:58.302 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:58.302 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:58.302 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:58.302 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:29:58.302 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.372525+0000", 2024-09-14T06:29:58.302 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:58.302 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:58.302 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:58.302 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:29:58.303 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.372525+0000", 2024-09-14T06:29:58.303 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.0600000000000001e-07 2024-09-14T06:29:58.303 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:58.303 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:58.303 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:29:58.303 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.372527+0000", 2024-09-14T06:29:58.303 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.0499999999999999e-06 2024-09-14T06:29:58.303 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:58.303 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:58.303 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:29:58.303 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.372533+0000", 2024-09-14T06:29:58.303 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.4320000000000004e-06 2024-09-14T06:29:58.303 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:58.303 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:58.304 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:29:58.304 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.372538+0000", 2024-09-14T06:29:58.304 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.9130000000000002e-06 2024-09-14T06:29:58.304 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:58.304 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:58.304 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:29:58.304 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.419667+0000", 2024-09-14T06:29:58.304 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047128976000000003 2024-09-14T06:29:58.304 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:58.304 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:58.304 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:29:58.304 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.419715+0000", 2024-09-14T06:29:58.304 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.7868999999999999e-05 2024-09-14T06:29:58.305 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:58.305 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:58.305 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:29:58.305 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.419784+0000", 2024-09-14T06:29:58.305 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.9201000000000004e-05 2024-09-14T06:29:58.305 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:58.305 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:29:58.305 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:58.305 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:58.305 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:58.305 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:53353 3.f 3:f71f51a7:::rbd_data.112dca2f6ae1.0000000000000015:17 [read 1376256~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:29:58.305 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:44.372545+0000", 2024-09-14T06:29:58.305 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 13.843316420000001, 2024-09-14T06:29:58.306 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047364232999999999, 2024-09-14T06:29:58.306 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:29:58.306 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:29:58.306 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:29:58.306 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:29:58.306 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:29:58.306 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:58.306 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 53353 2024-09-14T06:29:58.306 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:58.306 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:29:58.306 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:58.306 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:29:58.307 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.372545+0000", 2024-09-14T06:29:58.307 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:58.307 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:58.307 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:58.307 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:29:58.307 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.372545+0000", 2024-09-14T06:29:58.307 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:58.307 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:58.307 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:58.307 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:29:58.307 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.372545+0000", 2024-09-14T06:29:58.307 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.27e-07 2024-09-14T06:29:58.307 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:58.307 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:58.308 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:29:58.308 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.372547+0000", 2024-09-14T06:29:58.308 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.0600000000000002e-06 2024-09-14T06:29:58.308 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:58.308 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:58.308 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:29:58.308 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.372553+0000", 2024-09-14T06:29:58.308 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.7980000000000002e-06 2024-09-14T06:29:58.308 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:58.308 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:58.308 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:29:58.308 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.372557+0000", 2024-09-14T06:29:58.308 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.7389999999999999e-06 2024-09-14T06:29:58.309 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:58.309 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:58.309 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:29:58.309 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.419811+0000", 2024-09-14T06:29:58.309 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047253310999999999 2024-09-14T06:29:58.309 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:58.309 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:58.309 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:29:58.309 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.419854+0000", 2024-09-14T06:29:58.309 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.3448000000000001e-05 2024-09-14T06:29:58.309 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:58.309 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:58.309 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:29:58.310 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.419909+0000", 2024-09-14T06:29:58.310 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.4549999999999998e-05 2024-09-14T06:29:58.310 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:58.310 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:29:58.310 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:58.310 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:58.310 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:58.310 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:53354 3.f 3:f71f51a7:::rbd_data.112dca2f6ae1.0000000000000015:17 [read 1400832~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:29:58.310 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:44.372563+0000", 2024-09-14T06:29:58.310 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 13.843297982999999, 2024-09-14T06:29:58.311 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047482233999999998, 2024-09-14T06:29:58.311 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:29:58.311 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:29:58.311 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:29:58.311 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:29:58.311 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:29:58.311 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:58.311 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 53354 2024-09-14T06:29:58.311 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:58.311 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:29:58.312 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:58.312 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:29:58.312 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.372563+0000", 2024-09-14T06:29:58.312 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:58.312 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:58.312 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:58.312 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:29:58.312 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.372563+0000", 2024-09-14T06:29:58.312 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:29:58.312 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:58.312 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:58.313 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:29:58.313 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.375009+0000", 2024-09-14T06:29:58.313 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.0024457680000000001 2024-09-14T06:29:58.313 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:58.313 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:58.313 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:29:58.313 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.375014+0000", 2024-09-14T06:29:58.313 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.0599999999999998e-06 2024-09-14T06:29:58.313 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:58.313 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:58.314 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:29:58.314 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.375026+0000", 2024-09-14T06:29:58.314 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.1783999999999999e-05 2024-09-14T06:29:58.314 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:58.314 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:58.314 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:29:58.314 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.375035+0000", 2024-09-14T06:29:58.314 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.7799999999999995e-06 2024-09-14T06:29:58.314 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:58.314 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:58.314 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:29:58.314 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.419942+0000", 2024-09-14T06:29:58.314 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.044906982999999998 2024-09-14T06:29:58.315 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:58.315 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:58.315 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:29:58.315 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.419989+0000", 2024-09-14T06:29:58.315 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.7104999999999997e-05 2024-09-14T06:29:58.315 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:29:58.315 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:29:58.315 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:29:58.315 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.420045+0000", 2024-09-14T06:29:58.315 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.5754000000000002e-05 2024-09-14T06:29:58.315 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:58.315 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:29:58.315 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:58.315 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:29:58.316 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:29:58.316 INFO:teuthology.orchestra.run.smithi138.stdout:} 2024-09-14T06:29:58.318 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-14T06:29:58.318 INFO:tasks.ceph.osd.0.smithi094.stderr:2024-09-14T06:29:58.316+0000 7f07c9333640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33636) UID: 0 2024-09-14T06:29:58.418 INFO:tasks.ceph.osd.5.smithi138.stderr:2024-09-14T06:29:58.417+0000 7fe13746f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33618) UID: 0 2024-09-14T06:29:58.422 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-14T06:29:58.422 INFO:teuthology.orchestra.run.smithi094.stdout: "ops": [], 2024-09-14T06:29:58.422 INFO:teuthology.orchestra.run.smithi094.stdout: "num_ops": 0 2024-09-14T06:29:58.422 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-14T06:29:58.430 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-14T06:29:58.519 INFO:tasks.ceph.osd.7.smithi138.stderr:2024-09-14T06:29:58.518+0000 7f3c6a21d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33623) UID: 0 2024-09-14T06:29:58.532 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-14T06:29:58.532 INFO:teuthology.orchestra.run.smithi094.stdout: "ops": [], 2024-09-14T06:29:58.532 INFO:teuthology.orchestra.run.smithi094.stdout: "complaint_time": 30, 2024-09-14T06:29:58.532 INFO:teuthology.orchestra.run.smithi094.stdout: "num_blocked_ops": 0 2024-09-14T06:29:58.532 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-14T06:29:58.540 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-14T06:29:58.619 INFO:tasks.ceph.osd.3.smithi094.stderr:2024-09-14T06:29:58.618+0000 7feeeafe8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33655) UID: 0 2024-09-14T06:29:58.645 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-14T06:29:58.645 INFO:teuthology.orchestra.run.smithi094.stdout: "size": 20, 2024-09-14T06:29:58.645 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 600, 2024-09-14T06:29:58.645 INFO:teuthology.orchestra.run.smithi094.stdout: "ops": [ 2024-09-14T06:29:58.645 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:58.645 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:44355 3.17 3:ea948ecb:::rbd_data.112dca2f6ae1.0000000000000004:17 [read 196608~4096 [fadvise_nocache]] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:29:58.645 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:43.850672+0000", 2024-09-14T06:29:58.645 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 14.792322327000001, 2024-09-14T06:29:58.645 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.068480821999999997, 2024-09-14T06:29:58.645 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:29:58.645 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:29:58.645 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:29:58.646 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:29:58.646 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:29:58.646 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:58.646 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 44355 2024-09-14T06:29:58.646 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:58.646 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:29:58.646 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:58.646 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:29:58.646 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.850672+0000", 2024-09-14T06:29:58.646 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:58.646 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:58.646 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:58.646 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:29:58.646 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.850672+0000", 2024-09-14T06:29:58.646 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:58.647 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:58.647 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:58.647 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:29:58.647 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.850673+0000", 2024-09-14T06:29:58.647 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.0499999999999999e-07 2024-09-14T06:29:58.647 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:58.647 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:58.647 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:29:58.647 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.850675+0000", 2024-09-14T06:29:58.647 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.9400000000000001e-06 2024-09-14T06:29:58.647 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:58.647 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:58.647 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:29:58.648 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.850680+0000", 2024-09-14T06:29:58.648 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.4229999999999996e-06 2024-09-14T06:29:58.648 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:58.648 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:58.648 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:29:58.648 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.850685+0000", 2024-09-14T06:29:58.648 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.4920000000000004e-06 2024-09-14T06:29:58.648 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:58.648 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:58.648 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:29:58.648 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.919002+0000", 2024-09-14T06:29:58.648 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.068317131000000003 2024-09-14T06:29:58.648 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:58.649 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:58.649 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:29:58.649 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.919041+0000", 2024-09-14T06:29:58.649 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.9653999999999997e-05 2024-09-14T06:29:58.649 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:58.649 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:58.649 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:29:58.649 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.919153+0000", 2024-09-14T06:29:58.649 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000111777 2024-09-14T06:29:58.649 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:58.649 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:58.649 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:58.649 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:58.650 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:58.650 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:44356 3.17 3:ea948ecb:::rbd_data.112dca2f6ae1.0000000000000004:17 [read 221184~4096 [fadvise_nocache]] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:29:58.650 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:43.850690+0000", 2024-09-14T06:29:58.650 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 14.792304836, 2024-09-14T06:29:58.650 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.068790293000000002, 2024-09-14T06:29:58.650 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:29:58.650 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:29:58.650 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:29:58.650 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:29:58.650 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:29:58.650 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:58.650 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 44356 2024-09-14T06:29:58.650 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:58.650 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:29:58.651 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:58.651 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:29:58.651 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.850690+0000", 2024-09-14T06:29:58.651 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:58.651 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:58.651 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:58.651 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:29:58.651 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.850690+0000", 2024-09-14T06:29:58.651 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:58.651 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:58.651 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:58.651 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:29:58.651 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.850690+0000", 2024-09-14T06:29:58.651 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.5900000000000003e-07 2024-09-14T06:29:58.652 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:58.652 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:58.652 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:29:58.652 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.850692+0000", 2024-09-14T06:29:58.652 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.697e-06 2024-09-14T06:29:58.652 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:58.652 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:58.652 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:29:58.652 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.850782+0000", 2024-09-14T06:29:58.652 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.9777000000000005e-05 2024-09-14T06:29:58.652 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:58.652 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:58.652 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:29:58.652 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.850789+0000", 2024-09-14T06:29:58.653 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.063e-06 2024-09-14T06:29:58.653 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:58.653 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:58.653 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:29:58.654 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.919199+0000", 2024-09-14T06:29:58.656 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.068410152000000002 2024-09-14T06:29:58.656 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:58.656 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:58.656 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:29:58.656 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.919271+0000", 2024-09-14T06:29:58.656 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.1771000000000001e-05 2024-09-14T06:29:58.656 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:58.656 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:58.656 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:29:58.657 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.919480+0000", 2024-09-14T06:29:58.657 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00020947399999999999 2024-09-14T06:29:58.657 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:58.657 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:58.657 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:58.657 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:58.657 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:58.657 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:44357 3.17 3:ea948ecb:::rbd_data.112dca2f6ae1.0000000000000004:17 [read 245760~4096 [fadvise_nocache]] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:29:58.657 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:43.850800+0000", 2024-09-14T06:29:58.657 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 14.792194593, 2024-09-14T06:29:58.657 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.068852802000000005, 2024-09-14T06:29:58.657 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:29:58.657 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:29:58.658 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:29:58.658 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:29:58.658 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:29:58.658 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:58.658 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 44357 2024-09-14T06:29:58.658 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:58.658 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:29:58.658 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:58.658 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:29:58.658 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.850800+0000", 2024-09-14T06:29:58.658 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:58.658 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:58.658 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:58.658 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:29:58.659 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.850800+0000", 2024-09-14T06:29:58.659 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:58.659 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:58.659 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:58.659 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:29:58.659 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.850801+0000", 2024-09-14T06:29:58.659 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.5600000000000005e-07 2024-09-14T06:29:58.659 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:58.659 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:58.659 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:29:58.659 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.850838+0000", 2024-09-14T06:29:58.659 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.7481999999999998e-05 2024-09-14T06:29:58.659 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:58.659 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:58.659 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:29:58.660 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.850847+0000", 2024-09-14T06:29:58.660 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.4910000000000005e-06 2024-09-14T06:29:58.660 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:58.660 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:58.660 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:29:58.660 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.850856+0000", 2024-09-14T06:29:58.660 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.6309999999999998e-06 2024-09-14T06:29:58.660 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:58.660 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:58.660 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:29:58.660 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.919498+0000", 2024-09-14T06:29:58.660 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.068641743000000005 2024-09-14T06:29:58.660 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:58.660 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:58.661 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:29:58.661 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.919538+0000", 2024-09-14T06:29:58.661 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.9706999999999998e-05 2024-09-14T06:29:58.661 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:58.661 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:58.661 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:29:58.661 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.919653+0000", 2024-09-14T06:29:58.661 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000115092 2024-09-14T06:29:58.661 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:58.661 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:58.661 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:58.661 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:58.661 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:58.662 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:44358 3.17 3:ea948ecb:::rbd_data.112dca2f6ae1.0000000000000004:17 [read 270336~4096 [fadvise_nocache]] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:29:58.662 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:43.850865+0000", 2024-09-14T06:29:58.662 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 14.792129764, 2024-09-14T06:29:58.662 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.068983668999999997, 2024-09-14T06:29:58.662 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:29:58.662 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:29:58.662 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:29:58.662 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:29:58.662 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:29:58.662 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:58.662 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 44358 2024-09-14T06:29:58.662 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:58.662 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:29:58.663 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:58.663 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:29:58.663 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.850865+0000", 2024-09-14T06:29:58.663 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:58.663 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:58.663 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:58.663 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:29:58.663 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.850865+0000", 2024-09-14T06:29:58.663 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:58.663 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:58.663 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:58.663 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:29:58.663 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.850865+0000", 2024-09-14T06:29:58.663 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.3000000000000001e-07 2024-09-14T06:29:58.664 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:58.664 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:58.664 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:29:58.664 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.850868+0000", 2024-09-14T06:29:58.664 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.3099999999999999e-06 2024-09-14T06:29:58.664 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:58.664 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:58.664 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:29:58.664 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.850874+0000", 2024-09-14T06:29:58.664 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.3999999999999997e-06 2024-09-14T06:29:58.664 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:58.664 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:58.664 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:29:58.664 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.850885+0000", 2024-09-14T06:29:58.665 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.0553999999999999e-05 2024-09-14T06:29:58.665 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:58.665 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:58.665 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:29:58.665 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.919670+0000", 2024-09-14T06:29:58.665 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.068785464000000004 2024-09-14T06:29:58.665 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:58.665 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:58.665 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:29:58.665 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.919713+0000", 2024-09-14T06:29:58.665 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.3028999999999997e-05 2024-09-14T06:29:58.665 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:58.665 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:58.666 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:29:58.666 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.919849+0000", 2024-09-14T06:29:58.666 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000135382 2024-09-14T06:29:58.666 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:58.666 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:58.666 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:58.666 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:58.666 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:58.666 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:44359 3.17 3:ea948ecb:::rbd_data.112dca2f6ae1.0000000000000004:17 [read 294912~4096 [fadvise_nocache]] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:29:58.666 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:43.850892+0000", 2024-09-14T06:29:58.666 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 14.792102742000001, 2024-09-14T06:29:58.666 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.069282708999999998, 2024-09-14T06:29:58.666 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:29:58.667 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:29:58.667 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:29:58.667 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:29:58.667 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:29:58.667 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:58.667 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 44359 2024-09-14T06:29:58.667 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:58.667 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:29:58.667 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:58.667 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:29:58.667 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.850892+0000", 2024-09-14T06:29:58.667 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:58.667 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:58.668 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:58.668 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:29:58.668 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.850892+0000", 2024-09-14T06:29:58.668 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:58.668 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:58.668 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:58.668 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:29:58.668 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.850892+0000", 2024-09-14T06:29:58.668 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.3599999999999999e-07 2024-09-14T06:29:58.668 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:58.668 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:58.668 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:29:58.668 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.850895+0000", 2024-09-14T06:29:58.668 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.2050000000000001e-06 2024-09-14T06:29:58.669 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:58.669 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:58.669 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:29:58.669 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.850902+0000", 2024-09-14T06:29:58.669 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.4460000000000004e-06 2024-09-14T06:29:58.669 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:58.669 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:58.669 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:29:58.669 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.850907+0000", 2024-09-14T06:29:58.669 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.899e-06 2024-09-14T06:29:58.669 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:58.669 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:58.669 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:29:58.669 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.919896+0000", 2024-09-14T06:29:58.670 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.068988840999999995 2024-09-14T06:29:58.670 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:58.670 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:58.670 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:29:58.670 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.919968+0000", 2024-09-14T06:29:58.670 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.2454999999999997e-05 2024-09-14T06:29:58.670 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:58.670 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:58.670 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:29:58.670 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.920175+0000", 2024-09-14T06:29:58.670 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00020642699999999999 2024-09-14T06:29:58.670 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:58.670 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:58.670 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:58.671 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:58.671 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:58.671 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:44360 3.17 3:ea948ecb:::rbd_data.112dca2f6ae1.0000000000000004:17 [read 319488~4096 [fadvise_nocache]] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:29:58.671 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:43.850915+0000", 2024-09-14T06:29:58.671 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 14.792079435, 2024-09-14T06:29:58.671 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.069420047999999998, 2024-09-14T06:29:58.671 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:29:58.671 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:29:58.671 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:29:58.671 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:29:58.671 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:29:58.671 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:58.671 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 44360 2024-09-14T06:29:58.672 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:58.672 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:29:58.672 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:58.672 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:29:58.672 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.850915+0000", 2024-09-14T06:29:58.672 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:58.672 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:58.673 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:58.673 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:29:58.673 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.850915+0000", 2024-09-14T06:29:58.673 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:58.673 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:58.673 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:58.674 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:29:58.674 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.850916+0000", 2024-09-14T06:29:58.674 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.3500000000000002e-07 2024-09-14T06:29:58.674 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:58.674 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:58.674 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:29:58.674 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.850918+0000", 2024-09-14T06:29:58.675 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.0990000000000001e-06 2024-09-14T06:29:58.675 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:58.675 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:58.675 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:29:58.675 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.850925+0000", 2024-09-14T06:29:58.675 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.08e-06 2024-09-14T06:29:58.675 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:58.676 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:58.676 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:29:58.676 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.850930+0000", 2024-09-14T06:29:58.676 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.8069999999999999e-06 2024-09-14T06:29:58.676 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:58.677 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:58.677 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:29:58.677 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.920191+0000", 2024-09-14T06:29:58.677 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.069261113999999999 2024-09-14T06:29:58.677 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:58.677 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:58.678 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:29:58.678 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.920230+0000", 2024-09-14T06:29:58.678 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.8940999999999999e-05 2024-09-14T06:29:58.678 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:58.678 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:58.678 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:29:58.679 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.920335+0000", 2024-09-14T06:29:58.679 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000105572 2024-09-14T06:29:58.679 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:58.679 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:58.679 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:58.679 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:58.680 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:58.680 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:44361 3.17 3:ea948ecb:::rbd_data.112dca2f6ae1.0000000000000004:17 [read 344064~4096 [fadvise_nocache]] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:29:58.680 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:43.850936+0000", 2024-09-14T06:29:58.680 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 14.792058431999999, 2024-09-14T06:29:58.680 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.069568603000000007, 2024-09-14T06:29:58.680 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:29:58.681 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:29:58.681 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:29:58.681 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:29:58.681 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:29:58.681 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:58.682 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 44361 2024-09-14T06:29:58.682 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:58.682 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:29:58.682 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:58.682 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:29:58.682 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.850936+0000", 2024-09-14T06:29:58.683 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:58.683 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:58.683 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:58.683 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:29:58.684 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.850936+0000", 2024-09-14T06:29:58.684 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:58.684 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:58.684 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:58.684 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:29:58.685 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.851117+0000", 2024-09-14T06:29:58.685 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00018043700000000001 2024-09-14T06:29:58.685 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:58.686 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:58.686 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:29:58.686 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.851123+0000", 2024-09-14T06:29:58.686 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.8179999999999998e-06 2024-09-14T06:29:58.686 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:58.686 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:58.687 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:29:58.687 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.851131+0000", 2024-09-14T06:29:58.687 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.7689999999999996e-06 2024-09-14T06:29:58.687 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:58.687 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:58.688 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:29:58.688 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.851139+0000", 2024-09-14T06:29:58.688 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.3390000000000004e-06 2024-09-14T06:29:58.688 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:58.688 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:58.688 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:29:58.689 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.920353+0000", 2024-09-14T06:29:58.689 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.069214576999999999 2024-09-14T06:29:58.689 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:58.689 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:58.689 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:29:58.690 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.920391+0000", 2024-09-14T06:29:58.690 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.7883999999999998e-05 2024-09-14T06:29:58.690 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:58.690 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:58.690 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:29:58.691 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.920505+0000", 2024-09-14T06:29:58.692 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000113779 2024-09-14T06:29:58.692 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:58.692 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:58.692 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:58.692 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:58.693 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:58.693 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:44362 3.17 3:ea948ecb:::rbd_data.112dca2f6ae1.0000000000000004:17 [read 368640~4096 [fadvise_nocache]] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:29:58.693 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:43.851149+0000", 2024-09-14T06:29:58.693 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 14.791845652999999, 2024-09-14T06:29:58.693 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.069537131000000002, 2024-09-14T06:29:58.693 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:29:58.694 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:29:58.694 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:29:58.694 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:29:58.694 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:29:58.694 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:58.694 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 44362 2024-09-14T06:29:58.695 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:58.695 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:29:58.695 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:58.695 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:29:58.695 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.851149+0000", 2024-09-14T06:29:58.695 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:58.696 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:58.696 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:58.696 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:29:58.696 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.851149+0000", 2024-09-14T06:29:58.696 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:58.697 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:58.697 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:58.697 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:29:58.697 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.857003+0000", 2024-09-14T06:29:58.697 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0058535360000000003 2024-09-14T06:29:58.697 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:58.698 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:58.698 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:29:58.698 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.857009+0000", 2024-09-14T06:29:58.698 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.579e-06 2024-09-14T06:29:58.698 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:58.699 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:58.699 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:29:58.699 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.857019+0000", 2024-09-14T06:29:58.699 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.8330000000000006e-06 2024-09-14T06:29:58.699 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:58.699 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:58.700 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:29:58.700 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.857025+0000", 2024-09-14T06:29:58.700 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.8930000000000002e-06 2024-09-14T06:29:58.700 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:58.700 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:58.700 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:29:58.701 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.920522+0000", 2024-09-14T06:29:58.701 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.063496737999999997 2024-09-14T06:29:58.701 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:58.701 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:58.701 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:29:58.701 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.920567+0000", 2024-09-14T06:29:58.702 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.4956999999999997e-05 2024-09-14T06:29:58.702 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:58.702 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:58.702 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:29:58.702 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.920686+0000", 2024-09-14T06:29:58.702 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000119595 2024-09-14T06:29:58.703 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:58.703 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:58.703 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:58.703 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:58.703 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:58.703 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:44562 3.17 3:ea6a820a:::rbd_data.112dca2f6ae1.0000000000000003:17 [read 45056~4096 [fadvise_nocache]] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:29:58.704 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:43.857630+0000", 2024-09-14T06:29:58.704 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 14.785364895000001, 2024-09-14T06:29:58.704 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.068561975999999997, 2024-09-14T06:29:58.704 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:29:58.704 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:29:58.704 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:29:58.705 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:29:58.705 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:29:58.705 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:58.705 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 44562 2024-09-14T06:29:58.705 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:58.705 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:29:58.706 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:58.706 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:29:58.706 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.857630+0000", 2024-09-14T06:29:58.706 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:58.706 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:58.707 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:58.707 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:29:58.707 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.857630+0000", 2024-09-14T06:29:58.707 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:58.707 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:58.707 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:58.708 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:29:58.708 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.862817+0000", 2024-09-14T06:29:58.708 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0051874699999999996 2024-09-14T06:29:58.708 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:58.708 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:58.709 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:29:58.709 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.862823+0000", 2024-09-14T06:29:58.709 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.1299999999999998e-06 2024-09-14T06:29:58.709 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:58.709 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:58.709 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:29:58.710 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.862836+0000", 2024-09-14T06:29:58.710 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.3071000000000001e-05 2024-09-14T06:29:58.710 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:58.710 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:58.710 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:29:58.711 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.862845+0000", 2024-09-14T06:29:58.711 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.4630000000000001e-06 2024-09-14T06:29:58.711 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:58.711 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:58.711 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:29:58.711 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.925934+0000", 2024-09-14T06:29:58.712 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.063088985 2024-09-14T06:29:58.712 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:58.712 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:58.713 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:29:58.713 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.926013+0000", 2024-09-14T06:29:58.713 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.9507999999999998e-05 2024-09-14T06:29:58.713 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:58.713 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:58.713 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:29:58.714 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.926192+0000", 2024-09-14T06:29:58.714 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00017834900000000001 2024-09-14T06:29:58.714 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:58.714 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:58.714 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:58.715 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:58.715 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:58.715 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:44600 3.17 3:ea6a820a:::rbd_data.112dca2f6ae1.0000000000000003:17 [read 978944~4096 [fadvise_nocache]] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:29:58.715 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:43.869417+0000", 2024-09-14T06:29:58.715 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 14.773577206000001, 2024-09-14T06:29:58.715 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.068698802000000003, 2024-09-14T06:29:58.716 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:29:58.716 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:29:58.716 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:29:58.716 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:29:58.716 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:29:58.716 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:58.717 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 44600 2024-09-14T06:29:58.717 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:58.717 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:29:58.717 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:58.717 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:29:58.718 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.869417+0000", 2024-09-14T06:29:58.718 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:58.718 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:58.718 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:58.718 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:29:58.718 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.869417+0000", 2024-09-14T06:29:58.719 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:58.719 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:58.719 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:58.719 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:29:58.719 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.874912+0000", 2024-09-14T06:29:58.720 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0054948970000000003 2024-09-14T06:29:58.720 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:58.720 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:58.720 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:29:58.720 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.874917+0000", 2024-09-14T06:29:58.720 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.1259999999999997e-06 2024-09-14T06:29:58.721 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:58.721 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:58.721 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:29:58.721 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.874926+0000", 2024-09-14T06:29:58.721 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.7490000000000004e-06 2024-09-14T06:29:58.721 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:58.722 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:58.722 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:29:58.722 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.874939+0000", 2024-09-14T06:29:58.722 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.2831e-05 2024-09-14T06:29:58.722 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:58.722 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:58.723 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:29:58.723 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.937797+0000", 2024-09-14T06:29:58.723 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.062858019000000001 2024-09-14T06:29:58.723 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:58.723 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:58.723 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:29:58.724 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.937879+0000", 2024-09-14T06:29:58.724 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.1932000000000001e-05 2024-09-14T06:29:58.724 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:58.724 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:58.724 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:29:58.724 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.938116+0000", 2024-09-14T06:29:58.724 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00023724799999999999 2024-09-14T06:29:58.725 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:58.725 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:58.725 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:58.725 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:58.725 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:58.725 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:44617 3.17 3:ea948ecb:::rbd_data.112dca2f6ae1.0000000000000004:17 [read 348160~4096 [fadvise_nocache]] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:29:58.725 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:43.875196+0000", 2024-09-14T06:29:58.725 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 14.767798209, 2024-09-14T06:29:58.726 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.068462323000000005, 2024-09-14T06:29:58.726 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:29:58.726 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:29:58.726 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:29:58.726 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:29:58.726 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:29:58.726 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:58.726 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 44617 2024-09-14T06:29:58.727 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:58.727 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:29:58.727 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:58.727 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:29:58.727 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.875196+0000", 2024-09-14T06:29:58.727 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:58.727 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:58.728 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:58.728 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:29:58.728 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.875196+0000", 2024-09-14T06:29:58.728 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:58.728 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:58.728 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:58.728 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:29:58.729 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.875197+0000", 2024-09-14T06:29:58.729 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.7100000000000002e-07 2024-09-14T06:29:58.729 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:58.729 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:58.729 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:29:58.729 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.875199+0000", 2024-09-14T06:29:58.730 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.4609999999999998e-06 2024-09-14T06:29:58.730 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:58.730 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:58.730 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:29:58.730 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.875214+0000", 2024-09-14T06:29:58.730 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.4613000000000001e-05 2024-09-14T06:29:58.730 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:58.730 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:58.731 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:29:58.731 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.875221+0000", 2024-09-14T06:29:58.731 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.4309999999999997e-06 2024-09-14T06:29:58.731 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:58.731 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:58.731 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:29:58.731 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.943357+0000", 2024-09-14T06:29:58.731 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.068135412000000006 2024-09-14T06:29:58.732 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:58.732 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:58.732 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:29:58.732 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.943428+0000", 2024-09-14T06:29:58.732 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.0735000000000006e-05 2024-09-14T06:29:58.732 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:58.732 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:58.732 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:29:58.732 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.943659+0000", 2024-09-14T06:29:58.732 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00023120000000000001 2024-09-14T06:29:58.732 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:58.732 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:58.733 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:58.733 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:58.733 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:58.733 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:44618 3.17 3:ea948ecb:::rbd_data.112dca2f6ae1.0000000000000004:17 [read 372736~4096 [fadvise_nocache]] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:29:58.733 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:43.875236+0000", 2024-09-14T06:29:58.733 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 14.767758269, 2024-09-14T06:29:58.734 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.068766076999999995, 2024-09-14T06:29:58.734 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:29:58.734 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:29:58.734 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:29:58.734 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:29:58.735 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:29:58.735 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:58.735 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 44618 2024-09-14T06:29:58.735 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:58.735 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:29:58.735 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:58.736 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:29:58.736 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.875236+0000", 2024-09-14T06:29:58.736 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:58.736 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:58.736 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:58.736 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:29:58.737 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.875236+0000", 2024-09-14T06:29:58.737 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:58.737 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:58.737 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:58.737 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:29:58.737 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.875237+0000", 2024-09-14T06:29:58.738 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.8999999999999996e-07 2024-09-14T06:29:58.738 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:58.738 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:58.738 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:29:58.738 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.875239+0000", 2024-09-14T06:29:58.738 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.419e-06 2024-09-14T06:29:58.739 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:58.739 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:58.739 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:29:58.739 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.875245+0000", 2024-09-14T06:29:58.739 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.9909999999999999e-06 2024-09-14T06:29:58.739 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:58.739 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:58.739 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:29:58.739 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.875256+0000", 2024-09-14T06:29:58.740 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.0399e-05 2024-09-14T06:29:58.740 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:58.740 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:58.740 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:29:58.740 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.943690+0000", 2024-09-14T06:29:58.740 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.068434637000000006 2024-09-14T06:29:58.740 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:58.740 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:58.740 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:29:58.740 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.943765+0000", 2024-09-14T06:29:58.741 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.4418000000000005e-05 2024-09-14T06:29:58.741 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:58.741 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:58.741 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:29:58.741 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.944002+0000", 2024-09-14T06:29:58.741 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00023762300000000001 2024-09-14T06:29:58.742 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:58.742 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:58.742 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:58.742 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:58.742 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:58.742 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:44619 3.17 3:ea948ecb:::rbd_data.112dca2f6ae1.0000000000000004:17 [read 397312~4096 [fadvise_nocache]] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:29:58.743 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:43.875263+0000", 2024-09-14T06:29:58.743 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 14.767732002000001, 2024-09-14T06:29:58.743 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.069074144000000004, 2024-09-14T06:29:58.743 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:29:58.743 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:29:58.743 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:29:58.743 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:29:58.743 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:29:58.744 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:58.744 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 44619 2024-09-14T06:29:58.744 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:58.744 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:29:58.744 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:58.744 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:29:58.744 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.875263+0000", 2024-09-14T06:29:58.744 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:58.744 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:58.744 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:58.744 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:29:58.744 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.875263+0000", 2024-09-14T06:29:58.744 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:58.744 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:58.745 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:58.745 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:29:58.745 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.875263+0000", 2024-09-14T06:29:58.745 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.2199999999999999e-07 2024-09-14T06:29:58.745 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:58.745 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:58.745 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:29:58.745 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.875265+0000", 2024-09-14T06:29:58.745 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.1270000000000001e-06 2024-09-14T06:29:58.745 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:58.745 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:58.745 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:29:58.745 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.875271+0000", 2024-09-14T06:29:58.745 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.3329999999999996e-06 2024-09-14T06:29:58.746 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:58.746 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:58.746 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:29:58.746 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.875284+0000", 2024-09-14T06:29:58.746 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.3036e-05 2024-09-14T06:29:58.746 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:58.746 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:58.746 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:29:58.746 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.944032+0000", 2024-09-14T06:29:58.746 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.068748751999999996 2024-09-14T06:29:58.746 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:58.746 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:58.746 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:29:58.746 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.944106+0000", 2024-09-14T06:29:58.747 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.3441000000000004e-05 2024-09-14T06:29:58.747 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:58.747 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:58.747 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:29:58.747 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.944337+0000", 2024-09-14T06:29:58.747 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00023103300000000001 2024-09-14T06:29:58.747 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:58.747 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:58.747 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:58.747 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:58.747 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:58.747 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:44620 3.17 3:ea948ecb:::rbd_data.112dca2f6ae1.0000000000000004:17 [read 421888~4096 [fadvise_nocache]] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:29:58.747 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:43.875291+0000", 2024-09-14T06:29:58.747 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 14.767704117999999, 2024-09-14T06:29:58.748 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.069988984000000004, 2024-09-14T06:29:58.748 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:29:58.748 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:29:58.748 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:29:58.748 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:29:58.748 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:29:58.748 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:58.748 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 44620 2024-09-14T06:29:58.748 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:58.748 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:29:58.748 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:58.748 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:29:58.748 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.875291+0000", 2024-09-14T06:29:58.748 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:58.749 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:58.749 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:58.749 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:29:58.749 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.875291+0000", 2024-09-14T06:29:58.749 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:58.749 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:58.749 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:58.749 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:29:58.749 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.880818+0000", 2024-09-14T06:29:58.749 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0055275350000000001 2024-09-14T06:29:58.749 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:58.749 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:58.749 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:29:58.749 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.880824+0000", 2024-09-14T06:29:58.750 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.4240000000000001e-06 2024-09-14T06:29:58.750 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:58.750 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:58.750 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:29:58.750 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.880834+0000", 2024-09-14T06:29:58.750 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.0096e-05 2024-09-14T06:29:58.750 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:58.750 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:58.750 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:29:58.750 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.880842+0000", 2024-09-14T06:29:58.750 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.6349999999999995e-06 2024-09-14T06:29:58.750 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:58.750 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:58.750 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:29:58.751 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.944380+0000", 2024-09-14T06:29:58.751 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.063537934000000004 2024-09-14T06:29:58.751 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:58.751 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:58.751 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:29:58.751 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.944459+0000", 2024-09-14T06:29:58.751 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.8867000000000002e-05 2024-09-14T06:29:58.751 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:58.751 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:58.751 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:29:58.752 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.945280+0000", 2024-09-14T06:29:58.752 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00082049300000000005 2024-09-14T06:29:58.752 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:58.752 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:58.752 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:58.752 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:58.752 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:58.752 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:44644 3.17 3:ea948ecb:::rbd_data.112dca2f6ae1.0000000000000004:17 [read 1011712~4096 [fadvise_nocache]] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:29:58.752 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:43.881869+0000", 2024-09-14T06:29:58.752 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 14.761126164, 2024-09-14T06:29:58.752 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.068512922000000004, 2024-09-14T06:29:58.752 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:29:58.752 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:29:58.753 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:29:58.753 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:29:58.753 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:29:58.753 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:58.753 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 44644 2024-09-14T06:29:58.753 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:58.753 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:29:58.753 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:58.753 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:29:58.753 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.881869+0000", 2024-09-14T06:29:58.753 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:58.753 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:58.753 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:58.753 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:29:58.753 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.881869+0000", 2024-09-14T06:29:58.754 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:58.754 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:58.754 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:58.754 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:29:58.754 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.881869+0000", 2024-09-14T06:29:58.754 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.9299999999999998e-07 2024-09-14T06:29:58.754 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:58.754 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:58.754 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:29:58.754 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.881871+0000", 2024-09-14T06:29:58.754 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.362e-06 2024-09-14T06:29:58.754 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:58.754 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:58.754 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:29:58.755 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.881878+0000", 2024-09-14T06:29:58.755 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.6239999999999996e-06 2024-09-14T06:29:58.755 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:58.755 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:58.755 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:29:58.755 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.881885+0000", 2024-09-14T06:29:58.755 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.2649999999999999e-06 2024-09-14T06:29:58.755 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:58.755 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:58.755 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:29:58.755 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.950183+0000", 2024-09-14T06:29:58.755 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.068298225000000004 2024-09-14T06:29:58.755 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:58.756 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:58.756 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:29:58.756 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.950270+0000", 2024-09-14T06:29:58.756 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.6331999999999999e-05 2024-09-14T06:29:58.756 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:58.756 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:58.756 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:29:58.756 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.950381+0000", 2024-09-14T06:29:58.756 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00011162099999999999 2024-09-14T06:29:58.756 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:58.756 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:58.756 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:58.756 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:58.756 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:58.756 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:44645 3.17 3:ea948ecb:::rbd_data.112dca2f6ae1.0000000000000004:17 [read 1036288~4096 [fadvise_nocache]] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:29:58.757 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:43.881893+0000", 2024-09-14T06:29:58.757 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 14.761101838, 2024-09-14T06:29:58.757 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.068700357000000004, 2024-09-14T06:29:58.757 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:29:58.757 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:29:58.757 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:29:58.757 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:29:58.757 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:29:58.757 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:58.757 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 44645 2024-09-14T06:29:58.757 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:58.757 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:29:58.757 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:58.757 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:29:58.758 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.881893+0000", 2024-09-14T06:29:58.758 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:58.758 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:58.758 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:58.758 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:29:58.758 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.881893+0000", 2024-09-14T06:29:58.758 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:58.758 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:58.758 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:58.758 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:29:58.758 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.881893+0000", 2024-09-14T06:29:58.758 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.8299999999999998e-07 2024-09-14T06:29:58.758 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:58.758 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:58.759 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:29:58.759 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.881895+0000", 2024-09-14T06:29:58.759 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.88e-06 2024-09-14T06:29:58.759 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:58.759 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:58.759 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:29:58.759 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.881902+0000", 2024-09-14T06:29:58.759 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.6710000000000002e-06 2024-09-14T06:29:58.759 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:58.759 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:58.759 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:29:58.759 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.881915+0000", 2024-09-14T06:29:58.759 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.312e-05 2024-09-14T06:29:58.759 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:58.760 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:58.760 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:29:58.760 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.950420+0000", 2024-09-14T06:29:58.760 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.068505359000000002 2024-09-14T06:29:58.760 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:58.760 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:58.760 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:29:58.760 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.950497+0000", 2024-09-14T06:29:58.760 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.6454999999999999e-05 2024-09-14T06:29:58.760 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:58.760 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:58.760 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:29:58.760 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.950593+0000", 2024-09-14T06:29:58.760 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.6489000000000004e-05 2024-09-14T06:29:58.760 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:58.761 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:58.761 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:58.761 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:58.761 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:58.761 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:44777 3.17 3:ee6e7e6f:::rbd_data.112dca2f6ae1.0000000000000007:17 [read 8192~4096 [fadvise_nocache]] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:29:58.761 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:43.881935+0000", 2024-09-14T06:29:58.761 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 14.761059640999999, 2024-09-14T06:29:58.761 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.068908843999999997, 2024-09-14T06:29:58.761 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:29:58.761 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:29:58.761 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:29:58.761 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:29:58.761 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:29:58.761 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:58.762 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 44777 2024-09-14T06:29:58.762 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:58.762 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:29:58.762 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:58.762 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:29:58.762 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.881935+0000", 2024-09-14T06:29:58.762 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:58.762 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:58.762 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:58.762 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:29:58.762 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.881935+0000", 2024-09-14T06:29:58.762 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:58.762 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:58.762 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:58.762 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:29:58.763 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.881936+0000", 2024-09-14T06:29:58.763 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.7100000000000002e-07 2024-09-14T06:29:58.763 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:58.763 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:58.763 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:29:58.763 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.881938+0000", 2024-09-14T06:29:58.763 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.6010000000000002e-06 2024-09-14T06:29:58.763 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:58.763 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:58.763 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:29:58.763 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.881946+0000", 2024-09-14T06:29:58.763 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.9179999999999994e-06 2024-09-14T06:29:58.763 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:58.764 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:58.764 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:29:58.764 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.881951+0000", 2024-09-14T06:29:58.764 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.3199999999999999e-06 2024-09-14T06:29:58.764 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:58.764 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:58.764 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:29:58.764 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.950638+0000", 2024-09-14T06:29:58.764 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.068686149000000002 2024-09-14T06:29:58.764 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:58.764 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:58.764 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:29:58.764 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.950725+0000", 2024-09-14T06:29:58.765 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.7535999999999997e-05 2024-09-14T06:29:58.765 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:58.765 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:58.765 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:29:58.765 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.950844+0000", 2024-09-14T06:29:58.765 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000118749 2024-09-14T06:29:58.765 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:58.765 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:58.765 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:58.765 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:58.765 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:58.765 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:44778 3.17 3:ee6e7e6f:::rbd_data.112dca2f6ae1.0000000000000007:17 [read 32768~4096 [fadvise_nocache]] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:29:58.765 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:43.881960+0000", 2024-09-14T06:29:58.765 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 14.761034994999999, 2024-09-14T06:29:58.765 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.069098804999999999, 2024-09-14T06:29:58.766 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:29:58.766 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:29:58.766 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:29:58.766 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:29:58.766 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:29:58.766 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:58.766 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 44778 2024-09-14T06:29:58.766 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:58.766 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:29:58.766 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:58.766 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:29:58.766 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.881960+0000", 2024-09-14T06:29:58.766 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:58.766 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:58.767 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:58.767 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:29:58.767 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.881960+0000", 2024-09-14T06:29:58.767 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:58.767 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:58.767 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:58.767 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:29:58.767 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.881960+0000", 2024-09-14T06:29:58.767 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.6199999999999998e-07 2024-09-14T06:29:58.767 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:58.767 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:58.767 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:29:58.767 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.881963+0000", 2024-09-14T06:29:58.767 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.2740000000000002e-06 2024-09-14T06:29:58.768 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:58.768 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:58.768 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:29:58.768 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.881971+0000", 2024-09-14T06:29:58.768 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.1750000000000005e-06 2024-09-14T06:29:58.768 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:58.768 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:58.768 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:29:58.768 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.881977+0000", 2024-09-14T06:29:58.768 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.5030000000000002e-06 2024-09-14T06:29:58.768 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:58.768 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:58.768 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:29:58.768 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.950884+0000", 2024-09-14T06:29:58.768 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.068907066000000003 2024-09-14T06:29:58.769 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:58.769 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:58.769 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:29:58.769 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.950963+0000", 2024-09-14T06:29:58.769 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.8789999999999996e-05 2024-09-14T06:29:58.769 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:58.769 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:58.769 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:29:58.769 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.951058+0000", 2024-09-14T06:29:58.769 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.5434999999999997e-05 2024-09-14T06:29:58.769 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:58.769 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:58.769 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:58.769 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:58.770 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:58.770 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:44779 3.17 3:ee6e7e6f:::rbd_data.112dca2f6ae1.0000000000000007:17 [read 57344~4096 [fadvise_nocache]] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:29:58.770 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:43.881984+0000", 2024-09-14T06:29:58.770 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 14.761010401, 2024-09-14T06:29:58.770 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.069297828000000006, 2024-09-14T06:29:58.770 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:29:58.770 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:29:58.770 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:29:58.770 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:29:58.770 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:29:58.770 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:58.770 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 44779 2024-09-14T06:29:58.770 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:58.770 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:29:58.771 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:58.771 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:29:58.771 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.881984+0000", 2024-09-14T06:29:58.771 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:58.771 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:58.771 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:58.771 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:29:58.771 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.881984+0000", 2024-09-14T06:29:58.771 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:58.771 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:58.771 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:58.771 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:29:58.771 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.881985+0000", 2024-09-14T06:29:58.771 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.7099999999999997e-07 2024-09-14T06:29:58.771 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:58.772 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:58.772 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:29:58.772 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.881987+0000", 2024-09-14T06:29:58.772 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.0449999999999999e-06 2024-09-14T06:29:58.772 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:58.772 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:58.772 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:29:58.772 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.881994+0000", 2024-09-14T06:29:58.772 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.3849999999999996e-06 2024-09-14T06:29:58.772 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:58.772 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:58.772 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:29:58.772 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.881999+0000", 2024-09-14T06:29:58.772 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.0050000000000004e-06 2024-09-14T06:29:58.773 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:58.773 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:58.773 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:29:58.773 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.951101+0000", 2024-09-14T06:29:58.773 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.069101878000000005 2024-09-14T06:29:58.773 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:58.773 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:58.773 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:29:58.773 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.951178+0000", 2024-09-14T06:29:58.773 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.6823999999999995e-05 2024-09-14T06:29:58.773 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:58.773 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:58.773 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:29:58.774 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.951282+0000", 2024-09-14T06:29:58.774 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00010432 2024-09-14T06:29:58.774 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:58.774 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:58.774 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:58.774 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:58.774 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:58.774 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:44780 3.17 3:ee6e7e6f:::rbd_data.112dca2f6ae1.0000000000000007:17 [read 81920~4096 [fadvise_nocache]] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:29:58.774 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:43.882006+0000", 2024-09-14T06:29:58.774 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 14.760989091000001, 2024-09-14T06:29:58.774 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.069494627000000003, 2024-09-14T06:29:58.774 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:29:58.774 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:29:58.774 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:29:58.775 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:29:58.775 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:29:58.775 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:58.775 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 44780 2024-09-14T06:29:58.775 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:58.775 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:29:58.775 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:58.775 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:29:58.775 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.882006+0000", 2024-09-14T06:29:58.775 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:58.775 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:58.775 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:58.775 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:29:58.775 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.882006+0000", 2024-09-14T06:29:58.776 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:58.776 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:58.776 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:58.776 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:29:58.776 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.887819+0000", 2024-09-14T06:29:58.776 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0058133489999999998 2024-09-14T06:29:58.776 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:58.776 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:58.776 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:29:58.776 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.887826+0000", 2024-09-14T06:29:58.776 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.3889999999999999e-06 2024-09-14T06:29:58.776 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:58.776 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:58.776 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:29:58.776 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.887847+0000", 2024-09-14T06:29:58.777 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.0984000000000001e-05 2024-09-14T06:29:58.777 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:58.777 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:58.777 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:29:58.777 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.887857+0000", 2024-09-14T06:29:58.777 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.645e-06 2024-09-14T06:29:58.777 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:58.777 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:58.777 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:29:58.777 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.951325+0000", 2024-09-14T06:29:58.777 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.063468399999999994 2024-09-14T06:29:58.777 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:58.777 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:58.777 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:29:58.778 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.951401+0000", 2024-09-14T06:29:58.778 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.5499000000000003e-05 2024-09-14T06:29:58.778 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:58.778 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:58.778 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:29:58.778 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.951500+0000", 2024-09-14T06:29:58.778 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.9361e-05 2024-09-14T06:29:58.778 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:58.778 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:58.778 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:58.778 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:58.778 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:58.778 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-14T06:29:58.780 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-14T06:29:58.780 INFO:tasks.ceph.osd.5.smithi138.stderr:2024-09-14T06:29:58.778+0000 7fe13746f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33618) UID: 0 2024-09-14T06:29:58.791 INFO:teuthology.orchestra.run.smithi094.stderr:reweighted osd.0 to 1 (10000) 2024-09-14T06:29:58.809 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 1 1 2024-09-14T06:29:58.881 INFO:tasks.ceph.osd.7.smithi138.stderr:2024-09-14T06:29:58.880+0000 7f3c6a21d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33623) UID: 0 2024-09-14T06:29:58.904 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-14T06:29:58.904 INFO:teuthology.orchestra.run.smithi094.stdout: "ops": [], 2024-09-14T06:29:58.904 INFO:teuthology.orchestra.run.smithi094.stdout: "num_ops": 0 2024-09-14T06:29:58.904 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-14T06:29:58.914 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-14T06:29:58.946 INFO:teuthology.orchestra.run.smithi094.stdout:checking clone #13, image image_client.0-clone13 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent14 2024-09-14T06:29:58.982 INFO:tasks.ceph.osd.0.smithi094.stderr:2024-09-14T06:29:58.981+0000 7f07c9333640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33636) UID: 0 2024-09-14T06:29:59.053 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-14T06:29:59.053 INFO:teuthology.orchestra.run.smithi094.stdout: "ops": [], 2024-09-14T06:29:59.053 INFO:teuthology.orchestra.run.smithi094.stdout: "complaint_time": 30, 2024-09-14T06:29:59.054 INFO:teuthology.orchestra.run.smithi094.stdout: "num_blocked_ops": 0 2024-09-14T06:29:59.054 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-14T06:29:59.064 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-14T06:29:59.105 INFO:tasks.ceph.osd.1.smithi094.stderr:2024-09-14T06:29:59.104+0000 7f328e47a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33641) UID: 0 2024-09-14T06:29:59.184 INFO:tasks.ceph.osd.6.smithi138.stderr:2024-09-14T06:29:59.183+0000 7f8cd8849640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33620) UID: 0 2024-09-14T06:29:59.214 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-14T06:29:59.214 INFO:teuthology.orchestra.run.smithi094.stdout: "size": 20, 2024-09-14T06:29:59.214 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 600, 2024-09-14T06:29:59.214 INFO:teuthology.orchestra.run.smithi094.stdout: "ops": [ 2024-09-14T06:29:59.214 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:59.214 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:14918 3.0 3:05c39535:::rbd_data.112d92027590.0000000000000009:f [read 795648~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:29:59.214 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:26.083834+0000", 2024-09-14T06:29:59.215 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 33.128191831000002, 2024-09-14T06:29:59.215 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.041776287000000002, 2024-09-14T06:29:59.215 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:29:59.215 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:29:59.215 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:29:59.215 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:29:59.215 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:29:59.215 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:59.215 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 14918 2024-09-14T06:29:59.215 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:59.215 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:29:59.215 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:59.215 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:29:59.216 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.083834+0000", 2024-09-14T06:29:59.216 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:59.216 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:59.216 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:59.216 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:29:59.216 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.083834+0000", 2024-09-14T06:29:59.216 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:59.216 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:59.216 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:59.216 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:29:59.216 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.083834+0000", 2024-09-14T06:29:59.216 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.0100000000000001e-07 2024-09-14T06:29:59.216 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:59.216 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:59.217 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:29:59.217 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.083835+0000", 2024-09-14T06:29:59.217 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.3900000000000003e-07 2024-09-14T06:29:59.217 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:59.217 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:59.217 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:29:59.217 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.083838+0000", 2024-09-14T06:29:59.217 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.9830000000000001e-06 2024-09-14T06:29:59.217 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:59.217 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:59.217 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:29:59.217 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.083840+0000", 2024-09-14T06:29:59.217 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.4679999999999999e-06 2024-09-14T06:29:59.218 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:59.218 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:59.218 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:29:59.218 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.125455+0000", 2024-09-14T06:29:59.218 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.041614578999999999 2024-09-14T06:29:59.218 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:59.218 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:59.218 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:29:59.218 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.125496+0000", 2024-09-14T06:29:59.218 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.1047000000000002e-05 2024-09-14T06:29:59.218 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:59.218 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:59.219 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:29:59.219 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.125610+0000", 2024-09-14T06:29:59.219 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00011407 2024-09-14T06:29:59.219 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:59.219 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:59.219 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:59.219 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:59.219 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:59.219 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:14919 3.0 3:05c39535:::rbd_data.112d92027590.0000000000000009:f [read 805888~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:29:59.219 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:26.083843+0000", 2024-09-14T06:29:59.219 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 33.128182082999999, 2024-09-14T06:29:59.219 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.041992356000000002, 2024-09-14T06:29:59.219 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:29:59.219 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:29:59.220 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:29:59.220 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:29:59.220 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:29:59.220 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:59.220 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 14919 2024-09-14T06:29:59.220 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:59.220 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:29:59.220 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:59.220 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:29:59.220 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.083843+0000", 2024-09-14T06:29:59.220 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:59.220 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:59.221 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:59.221 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:29:59.221 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.083843+0000", 2024-09-14T06:29:59.221 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:59.221 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:59.221 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:59.221 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:29:59.221 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.083844+0000", 2024-09-14T06:29:59.221 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.3300000000000001e-07 2024-09-14T06:29:59.221 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:59.221 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:59.221 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:29:59.221 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.083845+0000", 2024-09-14T06:29:59.221 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.4729999999999999e-06 2024-09-14T06:29:59.221 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:59.222 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:59.222 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:29:59.222 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.083850+0000", 2024-09-14T06:29:59.222 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.0130000000000001e-06 2024-09-14T06:29:59.222 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:59.222 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:59.222 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:29:59.222 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.083854+0000", 2024-09-14T06:29:59.222 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.4759999999999998e-06 2024-09-14T06:29:59.222 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:59.222 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:59.222 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:29:59.222 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.125636+0000", 2024-09-14T06:29:59.222 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.041782683000000001 2024-09-14T06:29:59.223 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:59.223 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:59.223 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:29:59.223 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.125680+0000", 2024-09-14T06:29:59.223 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.3614e-05 2024-09-14T06:29:59.223 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:59.223 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:59.223 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:29:59.223 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.125836+0000", 2024-09-14T06:29:59.223 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000155864 2024-09-14T06:29:59.223 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:59.223 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:59.223 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:59.223 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:59.223 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:59.224 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:14920 3.0 3:05c39535:::rbd_data.112d92027590.0000000000000009:f [read 816128~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:29:59.224 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:26.083868+0000", 2024-09-14T06:29:59.224 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 33.128157422000001, 2024-09-14T06:29:59.224 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.042166771999999998, 2024-09-14T06:29:59.224 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:29:59.224 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:29:59.224 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:29:59.224 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:29:59.224 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:29:59.224 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:59.224 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 14920 2024-09-14T06:29:59.224 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:59.224 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:29:59.224 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:59.224 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:29:59.225 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.083868+0000", 2024-09-14T06:29:59.225 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:59.225 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:59.225 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:59.225 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:29:59.225 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.083868+0000", 2024-09-14T06:29:59.225 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:59.225 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:59.225 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:59.225 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:29:59.225 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.083869+0000", 2024-09-14T06:29:59.225 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.99e-07 2024-09-14T06:29:59.225 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:59.225 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:59.226 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:29:59.226 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.083870+0000", 2024-09-14T06:29:59.226 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.782e-06 2024-09-14T06:29:59.226 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:59.226 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:59.226 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:29:59.226 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.083876+0000", 2024-09-14T06:29:59.226 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.3809999999999998e-06 2024-09-14T06:29:59.226 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:59.226 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:59.226 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:29:59.226 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.083882+0000", 2024-09-14T06:29:59.226 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.9479999999999996e-06 2024-09-14T06:29:59.227 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:59.227 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:59.227 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:29:59.227 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.125878+0000", 2024-09-14T06:29:59.227 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.041996264999999998 2024-09-14T06:29:59.227 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:59.227 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:59.227 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:29:59.227 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.125929+0000", 2024-09-14T06:29:59.227 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.0498999999999999e-05 2024-09-14T06:29:59.227 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:59.227 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:59.227 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:29:59.227 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.126035+0000", 2024-09-14T06:29:59.228 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000106298 2024-09-14T06:29:59.228 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:59.228 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:59.228 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:59.228 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:59.228 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:59.228 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:14921 3.0 3:05c39535:::rbd_data.112d92027590.0000000000000009:f [read 826368~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:29:59.228 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:26.083889+0000", 2024-09-14T06:29:59.228 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 33.128136412000003, 2024-09-14T06:29:59.228 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.042435309999999997, 2024-09-14T06:29:59.228 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:29:59.228 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:29:59.228 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:29:59.229 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:29:59.229 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:29:59.229 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:59.229 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 14921 2024-09-14T06:29:59.229 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:59.229 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:29:59.229 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:59.229 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:29:59.229 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.083889+0000", 2024-09-14T06:29:59.229 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:59.229 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:59.229 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:59.229 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:29:59.230 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.083889+0000", 2024-09-14T06:29:59.230 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:59.230 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:59.230 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:59.230 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:29:59.230 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.086033+0000", 2024-09-14T06:29:59.230 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0021444179999999999 2024-09-14T06:29:59.230 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:59.230 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:59.230 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:29:59.230 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.086038+0000", 2024-09-14T06:29:59.230 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.0969999999999999e-06 2024-09-14T06:29:59.231 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:59.231 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:59.231 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:29:59.231 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.086046+0000", 2024-09-14T06:29:59.231 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.9440000000000005e-06 2024-09-14T06:29:59.231 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:59.231 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:59.231 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:29:59.231 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.086050+0000", 2024-09-14T06:29:59.231 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.69e-06 2024-09-14T06:29:59.231 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:59.231 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:59.231 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:29:59.231 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.126080+0000", 2024-09-14T06:29:59.232 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.040030237000000003 2024-09-14T06:29:59.232 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:59.232 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:59.232 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:29:59.232 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.126163+0000", 2024-09-14T06:29:59.232 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.2071999999999996e-05 2024-09-14T06:29:59.232 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:59.232 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:59.232 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:29:59.232 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.126324+0000", 2024-09-14T06:29:59.232 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00016185200000000001 2024-09-14T06:29:59.232 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:59.232 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:59.233 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:59.233 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:59.233 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:59.233 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:55627 3.0 3:06490baf:::rbd_data.112dca2f6ae1.0000000000000030:17 [read 2002944~4096 [fadvise_nocache]] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:29:59.233 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:44.599966+0000", 2024-09-14T06:29:59.233 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 14.612059464, 2024-09-14T06:29:59.233 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.041868546999999999, 2024-09-14T06:29:59.233 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:29:59.233 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:29:59.233 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:29:59.233 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:29:59.233 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:29:59.233 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:59.233 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 55627 2024-09-14T06:29:59.234 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:59.234 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:29:59.234 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:59.234 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:29:59.234 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.599966+0000", 2024-09-14T06:29:59.234 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:59.234 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:59.234 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:59.234 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:29:59.234 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.599966+0000", 2024-09-14T06:29:59.234 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:59.235 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:59.235 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:59.235 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:29:59.235 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.599966+0000", 2024-09-14T06:29:59.235 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.0900000000000001e-07 2024-09-14T06:29:59.235 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:59.235 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:59.235 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:29:59.235 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.599967+0000", 2024-09-14T06:29:59.235 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.64e-07 2024-09-14T06:29:59.235 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:59.236 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:59.236 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:29:59.236 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.599970+0000", 2024-09-14T06:29:59.236 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.9670000000000001e-06 2024-09-14T06:29:59.236 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:59.236 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:59.236 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:29:59.236 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.599973+0000", 2024-09-14T06:29:59.236 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.632e-06 2024-09-14T06:29:59.236 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:59.236 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:59.236 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:29:59.237 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.641665+0000", 2024-09-14T06:29:59.237 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.041692547000000003 2024-09-14T06:29:59.237 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:59.237 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:59.237 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:29:59.237 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.641740+0000", 2024-09-14T06:29:59.237 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.4559000000000002e-05 2024-09-14T06:29:59.237 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:59.237 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:59.237 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:29:59.237 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.641835+0000", 2024-09-14T06:29:59.237 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.4668999999999998e-05 2024-09-14T06:29:59.237 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:59.238 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:59.238 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:59.238 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:59.238 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:59.238 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:55628 3.0 3:06490baf:::rbd_data.112dca2f6ae1.0000000000000030:17 [read 2027520~4096 [fadvise_nocache]] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:29:59.238 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:44.599976+0000", 2024-09-14T06:29:59.238 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 14.612049677, 2024-09-14T06:29:59.238 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.041991621, 2024-09-14T06:29:59.238 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:29:59.238 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:29:59.238 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:29:59.238 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:29:59.238 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:29:59.239 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:59.239 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 55628 2024-09-14T06:29:59.239 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:59.239 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:29:59.239 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:59.239 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:29:59.239 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.599976+0000", 2024-09-14T06:29:59.239 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:59.239 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:59.239 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:59.239 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:29:59.239 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.599976+0000", 2024-09-14T06:29:59.239 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:59.240 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:59.240 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:59.240 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:29:59.240 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.599976+0000", 2024-09-14T06:29:59.240 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.1299999999999999e-07 2024-09-14T06:29:59.240 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:59.240 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:59.240 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:29:59.240 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.599977+0000", 2024-09-14T06:29:59.240 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.7999999999999993e-07 2024-09-14T06:29:59.240 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:59.240 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:59.241 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:29:59.241 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.599980+0000", 2024-09-14T06:29:59.241 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.01e-06 2024-09-14T06:29:59.241 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:59.241 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:59.241 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:29:59.241 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.599983+0000", 2024-09-14T06:29:59.241 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.5409999999999999e-06 2024-09-14T06:29:59.241 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:59.241 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:59.241 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:29:59.241 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.641869+0000", 2024-09-14T06:29:59.241 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.041886235000000001 2024-09-14T06:29:59.241 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:59.242 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:59.242 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:29:59.242 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.641913+0000", 2024-09-14T06:29:59.242 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.4357000000000002e-05 2024-09-14T06:29:59.242 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:59.242 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:59.242 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:29:59.242 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.641967+0000", 2024-09-14T06:29:59.242 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.4284999999999999e-05 2024-09-14T06:29:59.242 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:59.242 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:59.242 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:59.242 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:59.242 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:59.243 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:55629 3.0 3:06490baf:::rbd_data.112dca2f6ae1.0000000000000030:17 [read 2052096~4096 [fadvise_nocache]] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:29:59.243 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:44.599985+0000", 2024-09-14T06:29:59.243 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 14.612039971, 2024-09-14T06:29:59.243 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.042183375000000002, 2024-09-14T06:29:59.243 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:29:59.243 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:29:59.243 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:29:59.243 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:29:59.244 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:29:59.244 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:59.244 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 55629 2024-09-14T06:29:59.244 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:59.244 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:29:59.244 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:59.244 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:29:59.244 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.599985+0000", 2024-09-14T06:29:59.244 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:59.244 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:59.244 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:59.244 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:29:59.244 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.599985+0000", 2024-09-14T06:29:59.245 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:59.245 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:59.245 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:59.245 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:29:59.245 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.599986+0000", 2024-09-14T06:29:59.245 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.05e-07 2024-09-14T06:29:59.245 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:59.245 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:59.245 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:29:59.245 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.599987+0000", 2024-09-14T06:29:59.245 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.8400000000000002e-07 2024-09-14T06:29:59.245 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:59.246 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:59.246 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:29:59.246 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.599990+0000", 2024-09-14T06:29:59.246 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.5640000000000001e-06 2024-09-14T06:29:59.246 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:59.246 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:59.246 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:29:59.246 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.599993+0000", 2024-09-14T06:29:59.246 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.615e-06 2024-09-14T06:29:59.246 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:59.246 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:59.246 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:29:59.246 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.642003+0000", 2024-09-14T06:29:59.247 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.042010345999999997 2024-09-14T06:29:59.247 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:59.247 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:59.247 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:29:59.247 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.642080+0000", 2024-09-14T06:29:59.247 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.6538999999999993e-05 2024-09-14T06:29:59.247 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:59.247 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:59.247 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:29:59.247 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.642169+0000", 2024-09-14T06:29:59.247 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.9122000000000005e-05 2024-09-14T06:29:59.247 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:59.247 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:59.247 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:59.248 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:59.248 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:59.248 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:55630 3.0 3:06490baf:::rbd_data.112dca2f6ae1.0000000000000030:17 [read 2076672~4096 [fadvise_nocache]] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:29:59.248 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:44.599996+0000", 2024-09-14T06:29:59.248 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 14.612029493, 2024-09-14T06:29:59.248 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.042299615999999998, 2024-09-14T06:29:59.248 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:29:59.248 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:29:59.248 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:29:59.248 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:29:59.248 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:29:59.248 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:59.248 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 55630 2024-09-14T06:29:59.249 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:59.249 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:29:59.249 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:59.249 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:29:59.249 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.599996+0000", 2024-09-14T06:29:59.249 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:59.249 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:59.249 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:59.249 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:29:59.249 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.599996+0000", 2024-09-14T06:29:59.249 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:59.249 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:59.249 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:59.250 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:29:59.250 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.599996+0000", 2024-09-14T06:29:59.250 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.1500000000000001e-07 2024-09-14T06:29:59.250 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:59.250 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:59.250 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:29:59.250 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.599997+0000", 2024-09-14T06:29:59.250 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.33e-07 2024-09-14T06:29:59.250 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:59.250 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:59.250 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:29:59.250 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600000+0000", 2024-09-14T06:29:59.250 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.0240000000000002e-06 2024-09-14T06:29:59.250 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:59.251 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:59.251 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:29:59.251 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600003+0000", 2024-09-14T06:29:59.251 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.3640000000000002e-06 2024-09-14T06:29:59.251 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:59.251 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:59.251 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:29:59.251 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.642200+0000", 2024-09-14T06:29:59.251 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.042197571000000003 2024-09-14T06:29:59.251 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:59.251 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:59.252 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:29:59.252 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.642243+0000", 2024-09-14T06:29:59.252 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.2765000000000001e-05 2024-09-14T06:29:59.252 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:59.252 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:59.252 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:29:59.252 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.642296+0000", 2024-09-14T06:29:59.252 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.2744000000000002e-05 2024-09-14T06:29:59.252 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:59.252 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:59.252 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:59.252 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:59.252 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:59.253 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:55631 3.0 3:06490baf:::rbd_data.112dca2f6ae1.0000000000000030:17 [read 1048576~4096 [fadvise_nocache]] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:29:59.253 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:44.600006+0000", 2024-09-14T06:29:59.253 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 14.612019905, 2024-09-14T06:29:59.253 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.042487905999999999, 2024-09-14T06:29:59.253 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:29:59.253 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:29:59.253 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:29:59.253 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:29:59.253 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:29:59.253 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:59.253 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 55631 2024-09-14T06:29:59.253 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:59.253 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:29:59.254 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:59.254 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:29:59.254 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600006+0000", 2024-09-14T06:29:59.254 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:59.254 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:59.254 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:59.254 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:29:59.254 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600006+0000", 2024-09-14T06:29:59.254 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:59.254 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:59.254 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:59.254 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:29:59.254 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600006+0000", 2024-09-14T06:29:59.254 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.0800000000000001e-07 2024-09-14T06:29:59.255 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:59.255 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:59.255 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:29:59.255 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600007+0000", 2024-09-14T06:29:59.255 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.7199999999999997e-07 2024-09-14T06:29:59.255 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:59.255 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:59.255 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:29:59.255 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600010+0000", 2024-09-14T06:29:59.255 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.0280000000000001e-06 2024-09-14T06:29:59.255 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:59.255 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:59.255 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:29:59.255 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600012+0000", 2024-09-14T06:29:59.256 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.475e-06 2024-09-14T06:29:59.256 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:59.256 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:59.256 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:29:59.256 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.642329+0000", 2024-09-14T06:29:59.256 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.042317157000000001 2024-09-14T06:29:59.256 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:59.256 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:59.256 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:29:59.256 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.642403+0000", 2024-09-14T06:29:59.256 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.3941000000000002e-05 2024-09-14T06:29:59.256 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:59.256 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:59.257 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:29:59.257 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.642493+0000", 2024-09-14T06:29:59.257 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.0124999999999995e-05 2024-09-14T06:29:59.257 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:59.257 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:59.257 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:59.257 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:59.257 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:59.257 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:55632 3.0 3:06490baf:::rbd_data.112dca2f6ae1.0000000000000030:17 [read 1073152~4096 [fadvise_nocache]] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:29:59.257 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:44.600015+0000", 2024-09-14T06:29:59.257 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 14.612010152, 2024-09-14T06:29:59.257 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.042604922000000003, 2024-09-14T06:29:59.258 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:29:59.258 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:29:59.258 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:29:59.258 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:29:59.258 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:29:59.258 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:59.258 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 55632 2024-09-14T06:29:59.258 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:59.258 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:29:59.258 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:59.258 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:29:59.258 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600015+0000", 2024-09-14T06:29:59.258 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:59.258 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:59.259 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:59.259 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:29:59.259 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600015+0000", 2024-09-14T06:29:59.259 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:59.259 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:59.259 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:59.259 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:29:59.259 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600016+0000", 2024-09-14T06:29:59.259 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.1e-07 2024-09-14T06:29:59.259 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:59.259 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:59.259 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:29:59.259 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600017+0000", 2024-09-14T06:29:59.259 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.9900000000000009e-07 2024-09-14T06:29:59.260 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:59.260 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:59.260 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:29:59.260 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600020+0000", 2024-09-14T06:29:59.260 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.4170000000000001e-06 2024-09-14T06:29:59.260 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:59.260 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:59.260 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:29:59.260 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600023+0000", 2024-09-14T06:29:59.260 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.588e-06 2024-09-14T06:29:59.260 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:59.260 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:59.260 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:29:59.261 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.642526+0000", 2024-09-14T06:29:59.261 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.042503236999999999 2024-09-14T06:29:59.261 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:59.261 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:59.261 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:29:59.261 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.642568+0000", 2024-09-14T06:29:59.261 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.1925000000000003e-05 2024-09-14T06:29:59.261 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:59.261 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:59.261 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:29:59.261 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.642620+0000", 2024-09-14T06:29:59.261 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.2546000000000001e-05 2024-09-14T06:29:59.261 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:59.262 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:59.262 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:59.262 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:59.262 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:59.262 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:55633 3.0 3:06490baf:::rbd_data.112dca2f6ae1.0000000000000030:17 [read 1097728~4096 [fadvise_nocache]] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:29:59.262 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:44.600026+0000", 2024-09-14T06:29:59.262 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 14.611999882999999, 2024-09-14T06:29:59.262 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.042797474000000002, 2024-09-14T06:29:59.262 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:29:59.262 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:29:59.262 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:29:59.262 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:29:59.262 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:29:59.263 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:59.263 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 55633 2024-09-14T06:29:59.263 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:59.263 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:29:59.263 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:59.263 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:29:59.263 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600026+0000", 2024-09-14T06:29:59.263 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:59.263 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:59.263 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:59.263 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:29:59.263 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600026+0000", 2024-09-14T06:29:59.263 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:59.263 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:59.264 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:59.264 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:29:59.264 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600026+0000", 2024-09-14T06:29:59.264 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.3099999999999999e-07 2024-09-14T06:29:59.264 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:59.264 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:59.264 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:29:59.264 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600029+0000", 2024-09-14T06:29:59.264 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.304e-06 2024-09-14T06:29:59.264 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:59.264 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:59.264 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:29:59.265 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600034+0000", 2024-09-14T06:29:59.265 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.1429999999999997e-06 2024-09-14T06:29:59.265 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:59.265 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:59.265 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:29:59.265 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600038+0000", 2024-09-14T06:29:59.265 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.692e-06 2024-09-14T06:29:59.265 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:59.265 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:59.265 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:29:59.266 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.642654+0000", 2024-09-14T06:29:59.266 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.042615564000000002 2024-09-14T06:29:59.266 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:59.266 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:59.266 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:29:59.266 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.642728+0000", 2024-09-14T06:29:59.266 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.4018999999999997e-05 2024-09-14T06:29:59.266 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:59.266 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:59.266 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:29:59.266 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.642823+0000", 2024-09-14T06:29:59.266 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.5520999999999996e-05 2024-09-14T06:29:59.266 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:59.266 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:59.267 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:59.267 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:59.267 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:59.267 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:55634 3.0 3:06490baf:::rbd_data.112dca2f6ae1.0000000000000030:17 [read 1122304~4096 [fadvise_nocache]] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:29:59.267 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:44.600042+0000", 2024-09-14T06:29:59.267 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 14.611983406, 2024-09-14T06:29:59.267 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.042903729000000002, 2024-09-14T06:29:59.267 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:29:59.267 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:29:59.267 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:29:59.267 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:29:59.267 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:29:59.267 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:59.267 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 55634 2024-09-14T06:29:59.268 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:59.268 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:29:59.268 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:59.268 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:29:59.268 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600042+0000", 2024-09-14T06:29:59.268 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:59.268 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:59.268 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:59.268 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:29:59.268 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600042+0000", 2024-09-14T06:29:59.268 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:59.268 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:59.268 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:59.268 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:29:59.269 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600043+0000", 2024-09-14T06:29:59.269 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.7000000000000005e-07 2024-09-14T06:29:59.269 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:59.269 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:59.269 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:29:59.269 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600044+0000", 2024-09-14T06:29:59.270 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.742e-06 2024-09-14T06:29:59.270 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:59.270 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:59.270 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:29:59.270 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600050+0000", 2024-09-14T06:29:59.270 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.0800000000000002e-06 2024-09-14T06:29:59.271 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:59.271 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:59.271 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:29:59.271 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600055+0000", 2024-09-14T06:29:59.271 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.3889999999999998e-06 2024-09-14T06:29:59.271 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:59.271 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:59.271 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:29:59.272 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.642845+0000", 2024-09-14T06:29:59.272 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.042790040000000001 2024-09-14T06:29:59.272 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:59.272 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:59.272 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:29:59.273 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.642888+0000", 2024-09-14T06:29:59.273 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.2790000000000002e-05 2024-09-14T06:29:59.273 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:59.276 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:59.276 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:29:59.276 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.642946+0000", 2024-09-14T06:29:59.276 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.8118000000000002e-05 2024-09-14T06:29:59.276 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:59.277 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:59.277 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:59.277 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:59.277 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:59.277 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:55635 3.0 3:06490baf:::rbd_data.112dca2f6ae1.0000000000000030:17 [read 1146880~4096 [fadvise_nocache]] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:29:59.277 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:44.600061+0000", 2024-09-14T06:29:59.277 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 14.611964835, 2024-09-14T06:29:59.277 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.043092695, 2024-09-14T06:29:59.277 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:29:59.278 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:29:59.278 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:29:59.278 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:29:59.278 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:29:59.278 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:59.278 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 55635 2024-09-14T06:29:59.278 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:59.278 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:29:59.278 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:59.278 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:29:59.278 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600061+0000", 2024-09-14T06:29:59.278 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:59.278 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:59.278 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:59.279 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:29:59.279 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600061+0000", 2024-09-14T06:29:59.279 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:59.279 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:59.279 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:59.279 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:29:59.279 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600061+0000", 2024-09-14T06:29:59.279 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.3099999999999999e-07 2024-09-14T06:29:59.279 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:59.279 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:59.279 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:29:59.279 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600063+0000", 2024-09-14T06:29:59.279 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.6220000000000001e-06 2024-09-14T06:29:59.279 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:59.280 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:59.280 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:29:59.280 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600068+0000", 2024-09-14T06:29:59.280 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.4790000000000004e-06 2024-09-14T06:29:59.280 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:59.280 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:59.280 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:29:59.280 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600072+0000", 2024-09-14T06:29:59.280 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.3100000000000002e-06 2024-09-14T06:29:59.280 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:59.280 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:59.280 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:29:59.281 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.642987+0000", 2024-09-14T06:29:59.281 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.042914489 2024-09-14T06:29:59.281 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:59.281 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:59.281 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:29:59.281 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.643062+0000", 2024-09-14T06:29:59.281 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.5011000000000004e-05 2024-09-14T06:29:59.281 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:59.281 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:59.281 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:29:59.281 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.643153+0000", 2024-09-14T06:29:59.281 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.1452999999999993e-05 2024-09-14T06:29:59.281 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:59.281 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:59.282 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:59.282 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:59.282 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:59.282 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:55636 3.0 3:06490baf:::rbd_data.112dca2f6ae1.0000000000000030:17 [read 1171456~4096 [fadvise_nocache]] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:29:59.282 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:44.600078+0000", 2024-09-14T06:29:59.282 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 14.611947423, 2024-09-14T06:29:59.282 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.043207966, 2024-09-14T06:29:59.282 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:29:59.282 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:29:59.282 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:29:59.282 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:29:59.282 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:29:59.282 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:59.283 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 55636 2024-09-14T06:29:59.283 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:59.283 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:29:59.283 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:59.283 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:29:59.283 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600078+0000", 2024-09-14T06:29:59.283 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:59.283 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:59.283 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:59.283 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:29:59.283 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600078+0000", 2024-09-14T06:29:59.283 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:59.283 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:59.284 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:59.284 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:29:59.284 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600078+0000", 2024-09-14T06:29:59.284 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.34e-07 2024-09-14T06:29:59.284 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:59.284 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:59.284 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:29:59.284 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600080+0000", 2024-09-14T06:29:59.284 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.641e-06 2024-09-14T06:29:59.284 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:59.284 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:59.284 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:29:59.284 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600086+0000", 2024-09-14T06:29:59.285 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.3860000000000004e-06 2024-09-14T06:29:59.285 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:59.285 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:59.285 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:29:59.285 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600091+0000", 2024-09-14T06:29:59.285 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.5970000000000002e-06 2024-09-14T06:29:59.285 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:59.285 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:59.285 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:29:59.285 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.643184+0000", 2024-09-14T06:29:59.285 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.043092893 2024-09-14T06:29:59.285 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:59.285 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:59.285 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:29:59.286 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.643228+0000", 2024-09-14T06:29:59.286 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.4536999999999997e-05 2024-09-14T06:29:59.286 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:59.286 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:59.286 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:29:59.286 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.643286+0000", 2024-09-14T06:29:59.286 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.7578000000000003e-05 2024-09-14T06:29:59.286 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:59.286 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:59.286 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:59.286 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:59.286 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:59.286 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:55637 3.0 3:06490baf:::rbd_data.112dca2f6ae1.0000000000000030:17 [read 1196032~4096 [fadvise_nocache]] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:29:59.287 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:44.600097+0000", 2024-09-14T06:29:59.287 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 14.611928859000001, 2024-09-14T06:29:59.287 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.043399124999999997, 2024-09-14T06:29:59.287 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:29:59.287 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:29:59.287 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:29:59.287 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:29:59.287 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:29:59.287 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:59.287 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 55637 2024-09-14T06:29:59.287 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:59.287 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:29:59.287 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:59.288 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:29:59.288 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600097+0000", 2024-09-14T06:29:59.288 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:59.288 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:59.288 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:59.288 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:29:59.288 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600097+0000", 2024-09-14T06:29:59.288 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:59.288 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:59.288 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:59.288 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:29:59.288 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.604372+0000", 2024-09-14T06:29:59.289 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0042751489999999998 2024-09-14T06:29:59.289 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:59.289 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:59.289 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:29:59.289 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.604377+0000", 2024-09-14T06:29:59.289 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.7899999999999999e-06 2024-09-14T06:29:59.289 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:59.289 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:59.289 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:29:59.289 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.604389+0000", 2024-09-14T06:29:59.289 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.2197e-05 2024-09-14T06:29:59.289 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:59.289 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:59.290 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:29:59.290 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.604397+0000", 2024-09-14T06:29:59.290 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.8730000000000007e-06 2024-09-14T06:29:59.290 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:59.290 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:59.290 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:29:59.290 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.643324+0000", 2024-09-14T06:29:59.290 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.038926897000000002 2024-09-14T06:29:59.290 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:59.290 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:59.290 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:29:59.290 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.643402+0000", 2024-09-14T06:29:59.290 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.8063000000000001e-05 2024-09-14T06:29:59.290 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:59.291 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:59.291 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:29:59.291 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.643496+0000", 2024-09-14T06:29:59.291 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.4155999999999998e-05 2024-09-14T06:29:59.291 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:59.291 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:59.291 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:59.291 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:59.291 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:59.291 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:55657 3.0 3:06490baf:::rbd_data.112dca2f6ae1.0000000000000030:17 [read 1687552~4096 [fadvise_nocache]] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:29:59.291 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:44.604841+0000", 2024-09-14T06:29:59.291 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 14.60718413, 2024-09-14T06:29:59.291 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.041890087999999999, 2024-09-14T06:29:59.292 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:29:59.292 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:29:59.292 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:29:59.292 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:29:59.292 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:29:59.292 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:59.292 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 55657 2024-09-14T06:29:59.292 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:59.292 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:29:59.292 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:59.292 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:29:59.292 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.604841+0000", 2024-09-14T06:29:59.292 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:59.292 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:59.293 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:59.293 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:29:59.293 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.604841+0000", 2024-09-14T06:29:59.293 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:59.293 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:59.293 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:59.293 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:29:59.293 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.604842+0000", 2024-09-14T06:29:59.293 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.0900000000000001e-07 2024-09-14T06:29:59.293 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:59.293 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:59.293 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:29:59.293 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.604846+0000", 2024-09-14T06:29:59.294 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.5759999999999999e-06 2024-09-14T06:29:59.294 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:59.294 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:59.294 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:29:59.294 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.604850+0000", 2024-09-14T06:29:59.294 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.405e-06 2024-09-14T06:29:59.294 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:59.294 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:59.294 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:29:59.294 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.604852+0000", 2024-09-14T06:29:59.294 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.8590000000000001e-06 2024-09-14T06:29:59.294 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:59.294 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:59.294 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:29:59.295 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.646633+0000", 2024-09-14T06:29:59.295 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.041780772000000001 2024-09-14T06:29:59.295 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:59.295 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:59.295 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:29:59.295 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.646677+0000", 2024-09-14T06:29:59.295 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.4115e-05 2024-09-14T06:29:59.295 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:59.295 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:59.296 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:29:59.296 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.646731+0000", 2024-09-14T06:29:59.296 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.4151999999999999e-05 2024-09-14T06:29:59.296 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:59.296 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:59.296 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:59.296 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:59.296 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:59.296 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:55658 3.0 3:06490baf:::rbd_data.112dca2f6ae1.0000000000000030:17 [read 1712128~4096 [fadvise_nocache]] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:29:59.297 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:44.604856+0000", 2024-09-14T06:29:59.297 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 14.60716953, 2024-09-14T06:29:59.297 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.041986227000000001, 2024-09-14T06:29:59.297 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:29:59.297 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:29:59.297 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:29:59.298 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:29:59.298 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:29:59.298 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:59.298 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 55658 2024-09-14T06:29:59.298 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:59.298 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:29:59.298 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:59.298 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:29:59.298 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.604856+0000", 2024-09-14T06:29:59.298 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:59.298 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:59.298 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:59.298 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:29:59.299 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.604856+0000", 2024-09-14T06:29:59.299 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:59.299 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:59.299 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:59.299 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:29:59.299 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.604856+0000", 2024-09-14T06:29:59.299 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.23e-07 2024-09-14T06:29:59.299 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:59.299 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:59.299 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:29:59.299 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.604857+0000", 2024-09-14T06:29:59.299 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.9999999999999995e-07 2024-09-14T06:29:59.300 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:59.300 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:59.300 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:29:59.300 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.604861+0000", 2024-09-14T06:29:59.300 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.9890000000000003e-06 2024-09-14T06:29:59.300 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:59.300 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:59.300 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:29:59.300 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.604864+0000", 2024-09-14T06:29:59.300 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.7429999999999998e-06 2024-09-14T06:29:59.300 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:59.300 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:59.300 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:29:59.300 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.646744+0000", 2024-09-14T06:29:59.301 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.041880325000000003 2024-09-14T06:29:59.301 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:59.301 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:59.301 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:29:59.301 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.646784+0000", 2024-09-14T06:29:59.301 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.9604000000000002e-05 2024-09-14T06:29:59.301 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:59.301 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:59.301 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:29:59.301 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.646842+0000", 2024-09-14T06:29:59.301 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.8343e-05 2024-09-14T06:29:59.301 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:59.301 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:59.302 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:59.302 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:59.302 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:59.302 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:55659 3.0 3:06490baf:::rbd_data.112dca2f6ae1.0000000000000030:17 [read 1736704~4096 [fadvise_nocache]] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:29:59.302 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:44.604867+0000", 2024-09-14T06:29:59.302 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 14.607158376999999, 2024-09-14T06:29:59.302 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.042097082000000001, 2024-09-14T06:29:59.302 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:29:59.302 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:29:59.302 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:29:59.302 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:29:59.302 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:29:59.302 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:59.303 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 55659 2024-09-14T06:29:59.303 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:59.303 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:29:59.303 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:59.303 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:29:59.303 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.604867+0000", 2024-09-14T06:29:59.303 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:59.303 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:59.303 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:59.303 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:29:59.303 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.604867+0000", 2024-09-14T06:29:59.303 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:59.303 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:59.303 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:59.304 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:29:59.304 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.604867+0000", 2024-09-14T06:29:59.304 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.2000000000000001e-07 2024-09-14T06:29:59.304 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:59.304 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:59.304 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:29:59.304 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.604868+0000", 2024-09-14T06:29:59.304 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.04e-06 2024-09-14T06:29:59.304 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:59.304 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:59.304 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:29:59.304 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.604871+0000", 2024-09-14T06:29:59.304 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.9689999999999999e-06 2024-09-14T06:29:59.305 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:59.305 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:59.305 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:29:59.305 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.604874+0000", 2024-09-14T06:29:59.305 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.5069999999999999e-06 2024-09-14T06:29:59.305 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:59.305 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:59.305 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:29:59.305 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.646874+0000", 2024-09-14T06:29:59.305 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.042000513000000003 2024-09-14T06:29:59.305 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:59.305 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:59.305 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:29:59.305 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.646914+0000", 2024-09-14T06:29:59.306 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.9443999999999997e-05 2024-09-14T06:29:59.306 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:59.306 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:59.306 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:29:59.306 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.646964+0000", 2024-09-14T06:29:59.306 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.0389000000000002e-05 2024-09-14T06:29:59.306 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:59.306 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:59.306 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:59.306 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:59.306 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:59.306 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:55660 3.0 3:06490baf:::rbd_data.112dca2f6ae1.0000000000000030:17 [read 1761280~4096 [fadvise_nocache]] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:29:59.306 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:44.604877+0000", 2024-09-14T06:29:59.307 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 14.607148628000001, 2024-09-14T06:29:59.307 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.042196349000000001, 2024-09-14T06:29:59.307 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:29:59.307 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:29:59.307 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:29:59.307 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:29:59.307 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:29:59.307 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:59.307 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 55660 2024-09-14T06:29:59.307 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:59.307 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:29:59.308 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:59.308 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:29:59.308 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.604877+0000", 2024-09-14T06:29:59.308 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:59.308 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:59.308 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:59.308 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:29:59.308 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.604877+0000", 2024-09-14T06:29:59.308 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:59.308 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:59.308 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:59.309 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:29:59.309 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.605810+0000", 2024-09-14T06:29:59.309 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00093336899999999997 2024-09-14T06:29:59.309 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:59.309 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:59.309 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:29:59.310 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.605815+0000", 2024-09-14T06:29:59.310 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.6140000000000002e-06 2024-09-14T06:29:59.310 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:59.310 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:59.310 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:29:59.311 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.605825+0000", 2024-09-14T06:29:59.311 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.8889999999999997e-06 2024-09-14T06:29:59.311 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:59.311 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:59.311 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:29:59.312 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.605831+0000", 2024-09-14T06:29:59.312 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.9800000000000003e-06 2024-09-14T06:29:59.312 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:59.312 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:59.312 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:29:59.312 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.646989+0000", 2024-09-14T06:29:59.312 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.041157878000000002 2024-09-14T06:29:59.312 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:59.312 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:59.313 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:29:59.313 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.647026+0000", 2024-09-14T06:29:59.313 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.7305000000000003e-05 2024-09-14T06:29:59.313 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:59.313 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:59.313 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:29:59.313 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.647073+0000", 2024-09-14T06:29:59.313 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.7314000000000001e-05 2024-09-14T06:29:59.313 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:59.313 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:59.313 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:59.313 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:59.314 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:59.314 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:55666 3.0 3:06490baf:::rbd_data.112dca2f6ae1.0000000000000030:17 [read 1908736~4096 [fadvise_nocache]] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:29:59.314 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:44.605898+0000", 2024-09-14T06:29:59.314 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 14.606127881999999, 2024-09-14T06:29:59.314 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.041862887000000001, 2024-09-14T06:29:59.314 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:29:59.314 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:29:59.314 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:29:59.314 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:29:59.314 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:29:59.314 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:29:59.315 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 55666 2024-09-14T06:29:59.315 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:59.315 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:29:59.315 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:59.315 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:29:59.315 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.605898+0000", 2024-09-14T06:29:59.315 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:59.315 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:59.315 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:59.315 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:29:59.315 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.605898+0000", 2024-09-14T06:29:59.315 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:29:59.315 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:59.315 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:59.316 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:29:59.316 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.611874+0000", 2024-09-14T06:29:59.316 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0059764049999999997 2024-09-14T06:29:59.316 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:59.316 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:59.316 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:29:59.316 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.611879+0000", 2024-09-14T06:29:59.316 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.8749999999999999e-06 2024-09-14T06:29:59.316 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:59.316 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:59.316 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:29:59.316 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.611887+0000", 2024-09-14T06:29:59.316 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.4109999999999996e-06 2024-09-14T06:29:59.317 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:59.317 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:59.317 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:29:59.317 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.611893+0000", 2024-09-14T06:29:59.317 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.2959999999999998e-06 2024-09-14T06:29:59.317 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:59.317 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:59.317 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:29:59.317 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.647674+0000", 2024-09-14T06:29:59.317 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.035781281999999998 2024-09-14T06:29:59.317 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:59.318 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:59.318 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:29:59.318 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.647711+0000", 2024-09-14T06:29:59.318 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.7434000000000001e-05 2024-09-14T06:29:59.318 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:29:59.318 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:29:59.318 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:29:59.318 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.647760+0000", 2024-09-14T06:29:59.318 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.9184000000000002e-05 2024-09-14T06:29:59.318 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:59.318 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:59.319 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:59.319 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:29:59.319 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:29:59.319 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-14T06:29:59.320 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-14T06:29:59.321 INFO:teuthology.orchestra.run.smithi094.stdout:1662 read 0x944c67a thru 0x94558bb (0x9242 bytes) 2024-09-14T06:29:59.321 INFO:teuthology.orchestra.run.smithi094.stdout:1663 clone 16 order 25 su 2097152 sc 10 2024-09-14T06:29:59.321 INFO:tasks.ceph.osd.1.smithi094.stderr:2024-09-14T06:29:59.319+0000 7f328e47a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33641) UID: 0 2024-09-14T06:29:59.421 INFO:tasks.ceph.osd.0.smithi094.stderr:2024-09-14T06:29:59.420+0000 7f07c9333640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33636) UID: 0 2024-09-14T06:29:59.445 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-14T06:29:59.466 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-14T06:29:59.466 INFO:teuthology.orchestra.run.smithi094.stdout: "ops": [], 2024-09-14T06:29:59.466 INFO:teuthology.orchestra.run.smithi094.stdout: "num_ops": 0 2024-09-14T06:29:59.466 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-14T06:29:59.475 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-14T06:29:59.559 INFO:tasks.ceph.osd.0.smithi094.stderr:2024-09-14T06:29:59.558+0000 7f07c9333640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33636) UID: 0 2024-09-14T06:29:59.568 INFO:teuthology.orchestra.run.smithi094.stdout:osd.0: {} 2024-09-14T06:29:59.568 INFO:teuthology.orchestra.run.smithi094.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-14T06:29:59.585 INFO:teuthology.orchestra.run.smithi094.stdout:osd.1: {} 2024-09-14T06:29:59.585 INFO:teuthology.orchestra.run.smithi094.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-14T06:29:59.600 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-14T06:29:59.603 INFO:teuthology.orchestra.run.smithi094.stdout:osd.2: {} 2024-09-14T06:29:59.603 INFO:teuthology.orchestra.run.smithi094.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-14T06:29:59.623 INFO:teuthology.orchestra.run.smithi094.stdout:ERROR: (22) Invalid argument 2024-09-14T06:29:59.623 INFO:teuthology.orchestra.run.smithi094.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:29:59.623 INFO:tasks.ceph.osd.3.smithi094.stderr:2024-09-14T06:29:59.622+0000 7feeeafe8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33655) UID: 0 2024-09-14T06:29:59.632 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-14T06:29:59.722 INFO:tasks.ceph.osd.1.smithi094.stderr:2024-09-14T06:29:59.722+0000 7f328e47a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33641) UID: 0 2024-09-14T06:29:59.797 INFO:teuthology.orchestra.run.smithi094.stdout:ERROR: (22) Invalid argument 2024-09-14T06:29:59.797 INFO:teuthology.orchestra.run.smithi094.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:29:59.799 INFO:teuthology.orchestra.run.smithi094.stderr:reweighted osd.1 to 1 (10000) 2024-09-14T06:29:59.810 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-14T06:29:59.818 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 2 1 2024-09-14T06:29:59.824 INFO:teuthology.orchestra.run.smithi094.stdout:osd.3: {} 2024-09-14T06:29:59.824 INFO:teuthology.orchestra.run.smithi094.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-14T06:29:59.826 INFO:tasks.ceph.osd.3.smithi094.stderr:2024-09-14T06:29:59.824+0000 7feeeafe8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33655) UID: 0 2024-09-14T06:29:59.848 INFO:teuthology.orchestra.run.smithi094.stdout:osd.4: {} 2024-09-14T06:29:59.848 INFO:teuthology.orchestra.run.smithi094.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-14T06:29:59.867 INFO:teuthology.orchestra.run.smithi094.stdout:osd.5: {} 2024-09-14T06:29:59.867 INFO:teuthology.orchestra.run.smithi094.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-14T06:29:59.884 INFO:teuthology.orchestra.run.smithi094.stdout:osd.6: {} 2024-09-14T06:29:59.884 INFO:teuthology.orchestra.run.smithi094.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-14T06:29:59.901 INFO:teuthology.orchestra.run.smithi094.stdout:osd.7: {} 2024-09-14T06:29:59.901 INFO:teuthology.orchestra.run.smithi094.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-14T06:29:59.924 INFO:tasks.ceph.osd.5.smithi138.stderr:2024-09-14T06:29:59.922+0000 7fe13746f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33618) UID: 0 2024-09-14T06:29:59.940 INFO:teuthology.orchestra.run.smithi094.stdout:ERROR: (22) Invalid argument 2024-09-14T06:29:59.940 INFO:teuthology.orchestra.run.smithi094.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:29:59.949 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-14T06:30:00.024 INFO:tasks.ceph.osd.4.smithi138.stderr:2024-09-14T06:30:00.023+0000 7f6c653c8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33615) UID: 0 2024-09-14T06:30:00.097 INFO:teuthology.orchestra.run.smithi094.stdout:ERROR: (22) Invalid argument 2024-09-14T06:30:00.097 INFO:teuthology.orchestra.run.smithi094.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:30:00.106 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-14T06:30:00.125 INFO:tasks.ceph.osd.4.smithi138.stderr:2024-09-14T06:30:00.124+0000 7f6c653c8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33615) UID: 0 2024-09-14T06:30:00.209 INFO:teuthology.orchestra.run.smithi094.stdout:ERROR: (22) Invalid argument 2024-09-14T06:30:00.209 INFO:teuthology.orchestra.run.smithi094.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:30:00.218 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-14T06:30:00.225 INFO:tasks.ceph.osd.3.smithi094.stderr:2024-09-14T06:30:00.225+0000 7feeeafe8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33655) UID: 0 2024-09-14T06:30:00.326 INFO:tasks.ceph.osd.0.smithi094.stderr:2024-09-14T06:30:00.325+0000 7f07c9333640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33636) UID: 0 2024-09-14T06:30:00.333 INFO:teuthology.orchestra.run.smithi138.stdout:ERROR: (22) Invalid argument 2024-09-14T06:30:00.333 INFO:teuthology.orchestra.run.smithi138.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:30:00.342 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-14T06:30:00.427 INFO:tasks.ceph.osd.5.smithi138.stderr:2024-09-14T06:30:00.426+0000 7fe13746f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33618) UID: 0 2024-09-14T06:30:00.445 INFO:teuthology.orchestra.run.smithi138.stdout:ERROR: (22) Invalid argument 2024-09-14T06:30:00.446 INFO:teuthology.orchestra.run.smithi138.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:30:00.454 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-14T06:30:00.538 INFO:tasks.ceph.osd.5.smithi138.stderr:2024-09-14T06:30:00.537+0000 7fe13746f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33618) UID: 0 2024-09-14T06:30:00.579 INFO:teuthology.orchestra.run.smithi094.stderr:reweighted osd.2 to 1 (10000) 2024-09-14T06:30:00.582 INFO:teuthology.orchestra.run.smithi094.stderr:noscrub is set 2024-09-14T06:30:00.608 INFO:teuthology.orchestra.run.smithi138.stdout:ERROR: (22) Invalid argument 2024-09-14T06:30:00.608 INFO:teuthology.orchestra.run.smithi138.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:30:00.611 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 3 1 2024-09-14T06:30:00.620 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-14T06:30:00.661 INFO:tasks.ceph.osd.4.smithi138.stderr:2024-09-14T06:30:00.660+0000 7f6c653c8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33615) UID: 0 2024-09-14T06:30:00.730 INFO:tasks.ceph.osd.0.smithi094.stderr:2024-09-14T06:30:00.729+0000 7f07c9333640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33636) UID: 0 2024-09-14T06:30:00.784 INFO:teuthology.orchestra.run.smithi138.stdout:ERROR: (22) Invalid argument 2024-09-14T06:30:00.784 INFO:teuthology.orchestra.run.smithi138.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:30:00.793 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-14T06:30:00.837 INFO:tasks.ceph.osd.4.smithi138.stderr:2024-09-14T06:30:00.836+0000 7f6c653c8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33615) UID: 0 2024-09-14T06:30:00.896 INFO:teuthology.orchestra.run.smithi138.stdout:ERROR: (22) Invalid argument 2024-09-14T06:30:00.897 INFO:teuthology.orchestra.run.smithi138.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:30:00.905 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-14T06:30:00.946 INFO:tasks.ceph.osd.4.smithi138.stderr:2024-09-14T06:30:00.945+0000 7f6c653c8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33615) UID: 0 2024-09-14T06:30:01.033 INFO:tasks.ceph.osd.2.smithi094.stderr:2024-09-14T06:30:01.032+0000 7faf3269b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33649) UID: 0 2024-09-14T06:30:01.047 INFO:teuthology.orchestra.run.smithi138.stdout:ERROR: (22) Invalid argument 2024-09-14T06:30:01.047 INFO:teuthology.orchestra.run.smithi138.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:30:01.056 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-14T06:30:01.134 INFO:tasks.ceph.osd.6.smithi138.stderr:2024-09-14T06:30:01.133+0000 7f8cd8849640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33620) UID: 0 2024-09-14T06:30:01.159 INFO:teuthology.orchestra.run.smithi138.stdout:ERROR: (22) Invalid argument 2024-09-14T06:30:01.159 INFO:teuthology.orchestra.run.smithi138.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:30:01.168 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-14T06:30:01.235 INFO:tasks.ceph.osd.2.smithi094.stderr:2024-09-14T06:30:01.234+0000 7faf3269b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33649) UID: 0 2024-09-14T06:30:01.312 INFO:teuthology.orchestra.run.smithi138.stdout:ERROR: (22) Invalid argument 2024-09-14T06:30:01.312 INFO:teuthology.orchestra.run.smithi138.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:30:01.320 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-14T06:30:01.336 INFO:tasks.ceph.osd.3.smithi094.stderr:2024-09-14T06:30:01.335+0000 7feeeafe8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33655) UID: 0 2024-09-14T06:30:01.424 INFO:teuthology.orchestra.run.smithi138.stdout:ERROR: (22) Invalid argument 2024-09-14T06:30:01.424 INFO:teuthology.orchestra.run.smithi138.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:30:01.434 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-14T06:30:01.436 INFO:tasks.ceph.osd.0.smithi094.stderr:2024-09-14T06:30:01.435+0000 7f07c9333640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33636) UID: 0 2024-09-14T06:30:01.538 INFO:teuthology.orchestra.run.smithi138.stdout:ERROR: (22) Invalid argument 2024-09-14T06:30:01.538 INFO:teuthology.orchestra.run.smithi138.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:30:01.538 INFO:tasks.ceph.osd.1.smithi094.stderr:2024-09-14T06:30:01.537+0000 7f328e47a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33641) UID: 0 2024-09-14T06:30:01.546 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-14T06:30:01.584 INFO:teuthology.orchestra.run.smithi094.stderr:reweighted osd.3 to 1 (10000) 2024-09-14T06:30:01.599 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 4 1 2024-09-14T06:30:01.640 INFO:tasks.ceph.osd.2.smithi094.stderr:2024-09-14T06:30:01.639+0000 7faf3269b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33649) UID: 0 2024-09-14T06:30:01.650 INFO:teuthology.orchestra.run.smithi138.stdout:ERROR: (22) Invalid argument 2024-09-14T06:30:01.650 INFO:teuthology.orchestra.run.smithi138.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:30:01.659 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-14T06:30:01.719 INFO:teuthology.orchestra.run.smithi094.stdout:checking clone #14, image image_client.0-clone14 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent15 2024-09-14T06:30:01.739 INFO:tasks.ceph.osd.5.smithi138.stderr:2024-09-14T06:30:01.738+0000 7fe13746f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33618) UID: 0 2024-09-14T06:30:01.762 INFO:teuthology.orchestra.run.smithi138.stdout:ERROR: (22) Invalid argument 2024-09-14T06:30:01.762 INFO:teuthology.orchestra.run.smithi138.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:30:01.775 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-14T06:30:01.840 INFO:tasks.ceph.osd.5.smithi138.stderr:2024-09-14T06:30:01.839+0000 7fe13746f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33618) UID: 0 2024-09-14T06:30:01.891 INFO:teuthology.orchestra.run.smithi094.stdout:ERROR: (22) Invalid argument 2024-09-14T06:30:01.891 INFO:teuthology.orchestra.run.smithi094.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:30:01.901 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-14T06:30:01.914 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-14T06:30:01.941 INFO:tasks.ceph.osd.6.smithi138.stderr:2024-09-14T06:30:01.940+0000 7f8cd8849640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33620) UID: 0 2024-09-14T06:30:02.006 INFO:teuthology.orchestra.run.smithi094.stdout:1665 trunc from 0xd1a9107 to 0x26e0080 2024-09-14T06:30:02.012 INFO:teuthology.orchestra.run.smithi094.stdout:ERROR: (22) Invalid argument 2024-09-14T06:30:02.012 INFO:teuthology.orchestra.run.smithi094.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:30:02.020 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-14T06:30:02.020 INFO:teuthology.orchestra.run.smithi094.stdout:1667 read 0x215b883 thru 0x215b906 (0x84 bytes) 2024-09-14T06:30:02.022 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-14T06:30:02.026 INFO:teuthology.orchestra.run.smithi094.stdout:1668 trunc from 0x26e0080 to 0x6e0a751 2024-09-14T06:30:02.039 INFO:teuthology.orchestra.run.smithi094.stdout:1670 trunc from 0x6e0a751 to 0xd2cbc76 2024-09-14T06:30:02.041 INFO:tasks.ceph.osd.1.smithi094.stderr:2024-09-14T06:30:02.040+0000 7f328e47a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33641) UID: 0 2024-09-14T06:30:02.044 INFO:teuthology.orchestra.run.smithi094.stdout:1673 writesame 0x6f5f36f thru 0x6f656b4 data_size 0x2(0x6346 bytes) 2024-09-14T06:30:02.051 INFO:teuthology.orchestra.run.smithi094.stdout:1676 read 0xcb53b5d thru 0xcb5ae9c (0x7340 bytes) 2024-09-14T06:30:02.051 INFO:teuthology.orchestra.run.smithi094.stdout:1677 write 0xce0ee49 thru 0xce14281 (0x5439 bytes) 2024-09-14T06:30:02.053 INFO:teuthology.orchestra.run.smithi094.stdout:1679 writesame 0xad2fe43 thru 0xad35e66 data_size 0x1c(0x6024 bytes) 2024-09-14T06:30:02.055 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-14T06:30:02.055 INFO:teuthology.orchestra.run.smithi094.stdout:1683 read 0xaa410d8 thru 0xaa49126 (0x804f bytes) 2024-09-14T06:30:02.055 INFO:teuthology.orchestra.run.smithi094.stdout:1684 read 0xcb99791 thru 0xcba1e6f (0x86df bytes) 2024-09-14T06:30:02.055 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-14T06:30:02.062 INFO:teuthology.orchestra.run.smithi094.stdout:1686 trunc from 0xd2cbc76 to 0x74d8b73 2024-09-14T06:30:02.077 INFO:teuthology.orchestra.run.smithi094.stdout:1687 trunc from 0x74d8b73 to 0x1343688 2024-09-14T06:30:02.090 INFO:teuthology.orchestra.run.smithi094.stdout:osd.0: {} 2024-09-14T06:30:02.090 INFO:teuthology.orchestra.run.smithi094.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-14T06:30:02.100 INFO:teuthology.orchestra.run.smithi094.stdout:1688 write 0x86628a2 thru 0x866c5e6 (0x9d45 bytes) 2024-09-14T06:30:02.101 INFO:teuthology.orchestra.run.smithi094.stdout:1689 read 0x7732fda thru 0x773c787 (0x97ae bytes) 2024-09-14T06:30:02.102 INFO:teuthology.orchestra.run.smithi094.stdout:1690 write 0x346119c thru 0x346fc65 (0xeaca bytes) 2024-09-14T06:30:02.113 INFO:teuthology.orchestra.run.smithi094.stdout:1691 write 0xb77a523 thru 0xb77a8f6 (0x3d4 bytes) 2024-09-14T06:30:02.114 INFO:teuthology.orchestra.run.smithi094.stdout:osd.1: {} 2024-09-14T06:30:02.114 INFO:teuthology.orchestra.run.smithi094.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-14T06:30:02.120 INFO:teuthology.orchestra.run.smithi094.stdout:1694 writesame 0xc39016e thru 0xc39c3dc data_size 0x37(0xc26f bytes) 2024-09-14T06:30:02.121 INFO:teuthology.orchestra.run.smithi094.stdout:1695 write 0x531b403 thru 0x531ff90 (0x4b8e bytes) 2024-09-14T06:30:02.127 INFO:teuthology.orchestra.run.smithi094.stdout:1696 trunc from 0xc39c3dd to 0x8fd7d0a 2024-09-14T06:30:02.135 INFO:teuthology.orchestra.run.smithi094.stdout:osd.2: {} 2024-09-14T06:30:02.136 INFO:teuthology.orchestra.run.smithi094.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-14T06:30:02.144 INFO:tasks.ceph.osd.7.smithi138.stderr:2024-09-14T06:30:02.141+0000 7f3c6a21d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33623) UID: 0 2024-09-14T06:30:02.145 INFO:teuthology.orchestra.run.smithi094.stdout:1697 trunc from 0x8fd7d0a to 0x1f5960 2024-09-14T06:30:02.155 INFO:teuthology.orchestra.run.smithi094.stdout:osd.3: {} 2024-09-14T06:30:02.155 INFO:teuthology.orchestra.run.smithi094.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-14T06:30:02.162 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-14T06:30:02.162 INFO:teuthology.orchestra.run.smithi094.stdout: "size": 20, 2024-09-14T06:30:02.162 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 600, 2024-09-14T06:30:02.162 INFO:teuthology.orchestra.run.smithi094.stdout: "ops": [ 2024-09-14T06:30:02.162 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:02.162 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:44355 3.17 3:ea948ecb:::rbd_data.112dca2f6ae1.0000000000000004:17 [read 196608~4096 [fadvise_nocache]] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:30:02.162 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:43.850672+0000", 2024-09-14T06:30:02.162 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 18.309157328000001, 2024-09-14T06:30:02.162 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.068480821999999997, 2024-09-14T06:30:02.163 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:30:02.163 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:30:02.163 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:30:02.163 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:30:02.163 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:30:02.163 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:02.163 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 44355 2024-09-14T06:30:02.163 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:02.163 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:30:02.163 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:02.163 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:30:02.163 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.850672+0000", 2024-09-14T06:30:02.163 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:02.163 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:02.164 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:02.164 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:30:02.164 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.850672+0000", 2024-09-14T06:30:02.164 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:02.164 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:02.164 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:02.164 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:30:02.164 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.850673+0000", 2024-09-14T06:30:02.164 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.0499999999999999e-07 2024-09-14T06:30:02.164 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:02.164 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:02.165 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:30:02.165 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.850675+0000", 2024-09-14T06:30:02.165 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.9400000000000001e-06 2024-09-14T06:30:02.165 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:02.165 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:02.165 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:30:02.165 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.850680+0000", 2024-09-14T06:30:02.165 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.4229999999999996e-06 2024-09-14T06:30:02.165 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:02.165 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:02.165 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:30:02.165 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.850685+0000", 2024-09-14T06:30:02.166 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.4920000000000004e-06 2024-09-14T06:30:02.166 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:02.166 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:02.166 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:30:02.166 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.919002+0000", 2024-09-14T06:30:02.166 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.068317131000000003 2024-09-14T06:30:02.166 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:02.166 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:02.166 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:30:02.166 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.919041+0000", 2024-09-14T06:30:02.166 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.9653999999999997e-05 2024-09-14T06:30:02.166 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:02.167 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:02.167 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:30:02.167 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.919153+0000", 2024-09-14T06:30:02.167 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000111777 2024-09-14T06:30:02.167 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:02.167 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:02.167 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:02.167 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:02.167 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:02.167 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:44356 3.17 3:ea948ecb:::rbd_data.112dca2f6ae1.0000000000000004:17 [read 221184~4096 [fadvise_nocache]] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:30:02.167 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:43.850690+0000", 2024-09-14T06:30:02.167 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 18.309139837, 2024-09-14T06:30:02.167 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.068790293000000002, 2024-09-14T06:30:02.167 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:30:02.167 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:30:02.168 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:30:02.168 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:30:02.168 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:30:02.168 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:02.168 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 44356 2024-09-14T06:30:02.168 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:02.168 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:30:02.168 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:02.168 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:30:02.168 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.850690+0000", 2024-09-14T06:30:02.168 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:02.168 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:02.169 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:02.169 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:30:02.169 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.850690+0000", 2024-09-14T06:30:02.169 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:02.169 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:02.169 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:02.169 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:30:02.169 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.850690+0000", 2024-09-14T06:30:02.169 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.5900000000000003e-07 2024-09-14T06:30:02.169 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:02.170 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:02.170 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:30:02.170 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.850692+0000", 2024-09-14T06:30:02.170 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.697e-06 2024-09-14T06:30:02.170 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:02.170 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:02.170 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:30:02.170 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.850782+0000", 2024-09-14T06:30:02.171 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.9777000000000005e-05 2024-09-14T06:30:02.171 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:02.171 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:02.171 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:30:02.171 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.850789+0000", 2024-09-14T06:30:02.171 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.063e-06 2024-09-14T06:30:02.171 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:02.171 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:02.171 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:30:02.171 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.919199+0000", 2024-09-14T06:30:02.171 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.068410152000000002 2024-09-14T06:30:02.172 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:02.172 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:02.172 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:30:02.172 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.919271+0000", 2024-09-14T06:30:02.172 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.1771000000000001e-05 2024-09-14T06:30:02.172 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:02.172 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:02.172 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:30:02.172 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.919480+0000", 2024-09-14T06:30:02.172 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00020947399999999999 2024-09-14T06:30:02.172 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:02.172 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:02.172 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:02.173 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:02.173 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:02.173 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:44357 3.17 3:ea948ecb:::rbd_data.112dca2f6ae1.0000000000000004:17 [read 245760~4096 [fadvise_nocache]] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:30:02.173 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:43.850800+0000", 2024-09-14T06:30:02.173 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 18.309029593999998, 2024-09-14T06:30:02.173 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.068852802000000005, 2024-09-14T06:30:02.173 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:30:02.173 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:30:02.173 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:30:02.173 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:30:02.173 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:30:02.173 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:02.173 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 44357 2024-09-14T06:30:02.174 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:02.174 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:30:02.174 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:02.174 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:30:02.174 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.850800+0000", 2024-09-14T06:30:02.174 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:02.174 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:02.174 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:02.174 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:30:02.174 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.850800+0000", 2024-09-14T06:30:02.174 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:02.174 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:02.174 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:02.174 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:30:02.175 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.850801+0000", 2024-09-14T06:30:02.175 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.5600000000000005e-07 2024-09-14T06:30:02.175 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:02.175 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:02.175 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:30:02.175 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.850838+0000", 2024-09-14T06:30:02.175 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.7481999999999998e-05 2024-09-14T06:30:02.175 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:02.175 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:02.175 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:30:02.175 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.850847+0000", 2024-09-14T06:30:02.175 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.4910000000000005e-06 2024-09-14T06:30:02.176 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:02.176 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:02.176 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:30:02.176 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.850856+0000", 2024-09-14T06:30:02.176 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.6309999999999998e-06 2024-09-14T06:30:02.176 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:02.176 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:02.176 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:30:02.176 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.919498+0000", 2024-09-14T06:30:02.176 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.068641743000000005 2024-09-14T06:30:02.176 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:02.176 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:02.176 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:30:02.176 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.919538+0000", 2024-09-14T06:30:02.177 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.9706999999999998e-05 2024-09-14T06:30:02.177 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:02.177 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:02.177 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:30:02.177 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.919653+0000", 2024-09-14T06:30:02.177 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000115092 2024-09-14T06:30:02.177 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:02.177 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:02.177 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:02.177 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:02.177 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:02.177 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:44358 3.17 3:ea948ecb:::rbd_data.112dca2f6ae1.0000000000000004:17 [read 270336~4096 [fadvise_nocache]] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:30:02.178 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:43.850865+0000", 2024-09-14T06:30:02.178 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 18.308964764999999, 2024-09-14T06:30:02.178 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.068983668999999997, 2024-09-14T06:30:02.178 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:30:02.178 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:30:02.178 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:30:02.178 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:30:02.178 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:30:02.178 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:02.178 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 44358 2024-09-14T06:30:02.178 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:02.179 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:30:02.179 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:02.179 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:30:02.179 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.850865+0000", 2024-09-14T06:30:02.179 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:02.179 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:02.179 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:02.179 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:30:02.179 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.850865+0000", 2024-09-14T06:30:02.179 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:02.180 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:02.180 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:02.180 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:30:02.180 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.850865+0000", 2024-09-14T06:30:02.180 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.3000000000000001e-07 2024-09-14T06:30:02.180 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:02.180 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:02.181 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:30:02.181 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.850868+0000", 2024-09-14T06:30:02.181 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.3099999999999999e-06 2024-09-14T06:30:02.181 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:02.181 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:02.181 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:30:02.181 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.850874+0000", 2024-09-14T06:30:02.181 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.3999999999999997e-06 2024-09-14T06:30:02.181 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:02.181 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:02.182 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:30:02.182 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.850885+0000", 2024-09-14T06:30:02.182 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.0553999999999999e-05 2024-09-14T06:30:02.182 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:02.182 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:02.182 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:30:02.182 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.919670+0000", 2024-09-14T06:30:02.182 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.068785464000000004 2024-09-14T06:30:02.182 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:02.182 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:02.182 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:30:02.183 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.919713+0000", 2024-09-14T06:30:02.183 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.3028999999999997e-05 2024-09-14T06:30:02.183 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:02.183 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:02.183 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:30:02.183 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.919849+0000", 2024-09-14T06:30:02.183 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000135382 2024-09-14T06:30:02.183 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:02.183 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:02.183 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:02.183 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:02.184 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:02.184 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:44359 3.17 3:ea948ecb:::rbd_data.112dca2f6ae1.0000000000000004:17 [read 294912~4096 [fadvise_nocache]] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:30:02.184 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:43.850892+0000", 2024-09-14T06:30:02.184 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 18.308937743000001, 2024-09-14T06:30:02.184 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.069282708999999998, 2024-09-14T06:30:02.184 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:30:02.184 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:30:02.184 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:30:02.184 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:30:02.184 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:30:02.184 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:02.184 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 44359 2024-09-14T06:30:02.184 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:02.185 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:30:02.185 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:02.185 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:30:02.185 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.850892+0000", 2024-09-14T06:30:02.185 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:02.185 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:02.185 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:02.185 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:30:02.185 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.850892+0000", 2024-09-14T06:30:02.185 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:02.185 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:02.185 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:02.186 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:30:02.186 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.850892+0000", 2024-09-14T06:30:02.186 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.3599999999999999e-07 2024-09-14T06:30:02.186 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:02.186 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:02.186 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:30:02.186 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.850895+0000", 2024-09-14T06:30:02.186 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.2050000000000001e-06 2024-09-14T06:30:02.186 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:02.186 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:02.186 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:30:02.186 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.850902+0000", 2024-09-14T06:30:02.186 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.4460000000000004e-06 2024-09-14T06:30:02.187 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:02.187 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:02.187 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:30:02.187 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.850907+0000", 2024-09-14T06:30:02.187 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.899e-06 2024-09-14T06:30:02.187 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:02.187 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:02.187 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:30:02.187 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.919896+0000", 2024-09-14T06:30:02.187 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.068988840999999995 2024-09-14T06:30:02.187 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:02.187 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:02.187 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:30:02.188 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.919968+0000", 2024-09-14T06:30:02.188 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.2454999999999997e-05 2024-09-14T06:30:02.188 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:02.188 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:02.188 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:30:02.188 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.920175+0000", 2024-09-14T06:30:02.188 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00020642699999999999 2024-09-14T06:30:02.188 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:02.189 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:02.189 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:02.189 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:02.189 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:02.189 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:44360 3.17 3:ea948ecb:::rbd_data.112dca2f6ae1.0000000000000004:17 [read 319488~4096 [fadvise_nocache]] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:30:02.189 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:43.850915+0000", 2024-09-14T06:30:02.189 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 18.308914435999998, 2024-09-14T06:30:02.189 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.069420047999999998, 2024-09-14T06:30:02.189 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:30:02.189 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:30:02.189 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:30:02.189 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:30:02.189 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:30:02.190 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:02.190 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 44360 2024-09-14T06:30:02.190 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:02.190 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:30:02.190 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:02.190 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:30:02.190 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.850915+0000", 2024-09-14T06:30:02.190 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:02.190 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:02.190 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:02.191 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:30:02.191 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.850915+0000", 2024-09-14T06:30:02.191 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:02.191 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:02.191 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:02.191 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:30:02.191 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.850916+0000", 2024-09-14T06:30:02.191 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.3500000000000002e-07 2024-09-14T06:30:02.191 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:02.191 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:02.191 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:30:02.192 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.850918+0000", 2024-09-14T06:30:02.192 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.0990000000000001e-06 2024-09-14T06:30:02.192 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:02.192 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:02.192 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:30:02.192 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.850925+0000", 2024-09-14T06:30:02.192 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.08e-06 2024-09-14T06:30:02.192 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:02.192 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:02.192 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:30:02.192 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.850930+0000", 2024-09-14T06:30:02.192 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.8069999999999999e-06 2024-09-14T06:30:02.192 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:02.193 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:02.193 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:30:02.193 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.920191+0000", 2024-09-14T06:30:02.193 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.069261113999999999 2024-09-14T06:30:02.193 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:02.193 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:02.193 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:30:02.193 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.920230+0000", 2024-09-14T06:30:02.193 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.8940999999999999e-05 2024-09-14T06:30:02.193 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:02.193 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:02.194 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:30:02.194 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.920335+0000", 2024-09-14T06:30:02.194 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000105572 2024-09-14T06:30:02.194 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:02.194 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:02.194 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:02.194 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:02.194 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:02.194 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:44361 3.17 3:ea948ecb:::rbd_data.112dca2f6ae1.0000000000000004:17 [read 344064~4096 [fadvise_nocache]] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:30:02.194 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:43.850936+0000", 2024-09-14T06:30:02.194 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 18.308893433000001, 2024-09-14T06:30:02.194 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.069568603000000007, 2024-09-14T06:30:02.194 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:30:02.195 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:30:02.195 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:30:02.195 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:30:02.195 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:30:02.195 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:02.195 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 44361 2024-09-14T06:30:02.195 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:02.195 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:30:02.195 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:02.195 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:30:02.195 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.850936+0000", 2024-09-14T06:30:02.196 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:02.196 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:02.196 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:02.196 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:30:02.196 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.850936+0000", 2024-09-14T06:30:02.196 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:02.196 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:02.196 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:02.196 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:30:02.196 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.851117+0000", 2024-09-14T06:30:02.196 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00018043700000000001 2024-09-14T06:30:02.196 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:02.196 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:02.196 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:30:02.197 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.851123+0000", 2024-09-14T06:30:02.197 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.8179999999999998e-06 2024-09-14T06:30:02.197 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:02.197 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:02.197 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:30:02.197 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.851131+0000", 2024-09-14T06:30:02.197 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.7689999999999996e-06 2024-09-14T06:30:02.197 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:02.197 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:02.197 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:30:02.197 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.851139+0000", 2024-09-14T06:30:02.197 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.3390000000000004e-06 2024-09-14T06:30:02.198 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:02.198 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:02.198 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:30:02.198 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.920353+0000", 2024-09-14T06:30:02.198 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.069214576999999999 2024-09-14T06:30:02.198 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:02.198 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:02.198 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:30:02.198 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.920391+0000", 2024-09-14T06:30:02.198 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.7883999999999998e-05 2024-09-14T06:30:02.198 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:02.198 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:02.198 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:30:02.199 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.920505+0000", 2024-09-14T06:30:02.199 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000113779 2024-09-14T06:30:02.199 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:02.199 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:02.199 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:02.199 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:02.199 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:02.199 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:44362 3.17 3:ea948ecb:::rbd_data.112dca2f6ae1.0000000000000004:17 [read 368640~4096 [fadvise_nocache]] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:30:02.199 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:43.851149+0000", 2024-09-14T06:30:02.199 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 18.308680654, 2024-09-14T06:30:02.199 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.069537131000000002, 2024-09-14T06:30:02.199 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:30:02.199 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:30:02.199 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:30:02.199 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:30:02.200 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:30:02.200 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:02.200 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 44362 2024-09-14T06:30:02.200 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:02.200 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:30:02.200 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:02.200 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:30:02.200 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.851149+0000", 2024-09-14T06:30:02.200 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:02.200 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:02.201 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:02.201 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:30:02.201 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.851149+0000", 2024-09-14T06:30:02.201 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:02.201 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:02.201 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:02.201 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:30:02.201 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.857003+0000", 2024-09-14T06:30:02.201 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0058535360000000003 2024-09-14T06:30:02.201 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:02.202 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:02.202 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:30:02.202 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.857009+0000", 2024-09-14T06:30:02.202 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.579e-06 2024-09-14T06:30:02.202 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:02.202 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:02.202 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:30:02.202 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.857019+0000", 2024-09-14T06:30:02.202 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.8330000000000006e-06 2024-09-14T06:30:02.202 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:02.202 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:02.202 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:30:02.202 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.857025+0000", 2024-09-14T06:30:02.202 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.8930000000000002e-06 2024-09-14T06:30:02.203 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:02.203 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:02.203 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:30:02.203 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.920522+0000", 2024-09-14T06:30:02.203 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.063496737999999997 2024-09-14T06:30:02.203 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:02.203 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:02.203 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:30:02.203 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.920567+0000", 2024-09-14T06:30:02.203 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.4956999999999997e-05 2024-09-14T06:30:02.203 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:02.203 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:02.204 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:30:02.204 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.920686+0000", 2024-09-14T06:30:02.204 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000119595 2024-09-14T06:30:02.204 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:02.204 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:02.204 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:02.204 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:02.204 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:02.204 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:44562 3.17 3:ea6a820a:::rbd_data.112dca2f6ae1.0000000000000003:17 [read 45056~4096 [fadvise_nocache]] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:30:02.204 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:43.857630+0000", 2024-09-14T06:30:02.204 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 18.302199896000001, 2024-09-14T06:30:02.204 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.068561975999999997, 2024-09-14T06:30:02.204 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:30:02.205 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:30:02.205 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:30:02.205 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:30:02.205 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:30:02.205 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:02.205 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 44562 2024-09-14T06:30:02.205 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:02.205 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:30:02.205 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:02.205 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:30:02.205 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.857630+0000", 2024-09-14T06:30:02.206 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:02.206 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:02.206 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:02.206 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:30:02.206 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.857630+0000", 2024-09-14T06:30:02.206 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:02.206 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:02.206 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:02.206 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:30:02.206 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.862817+0000", 2024-09-14T06:30:02.206 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0051874699999999996 2024-09-14T06:30:02.206 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:02.207 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:02.207 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:30:02.207 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.862823+0000", 2024-09-14T06:30:02.207 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.1299999999999998e-06 2024-09-14T06:30:02.207 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:02.207 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:02.207 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:30:02.207 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.862836+0000", 2024-09-14T06:30:02.207 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.3071000000000001e-05 2024-09-14T06:30:02.207 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:02.207 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:02.207 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:30:02.207 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.862845+0000", 2024-09-14T06:30:02.208 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.4630000000000001e-06 2024-09-14T06:30:02.208 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:02.208 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:02.208 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:30:02.208 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.925934+0000", 2024-09-14T06:30:02.208 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.063088985 2024-09-14T06:30:02.208 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:02.208 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:02.208 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:30:02.208 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.926013+0000", 2024-09-14T06:30:02.208 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.9507999999999998e-05 2024-09-14T06:30:02.208 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:02.209 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:02.209 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:30:02.209 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.926192+0000", 2024-09-14T06:30:02.209 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00017834900000000001 2024-09-14T06:30:02.209 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:02.209 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:02.209 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:02.209 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:02.209 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:02.209 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:44600 3.17 3:ea6a820a:::rbd_data.112dca2f6ae1.0000000000000003:17 [read 978944~4096 [fadvise_nocache]] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:30:02.210 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:43.869417+0000", 2024-09-14T06:30:02.210 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 18.290412206999999, 2024-09-14T06:30:02.210 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.068698802000000003, 2024-09-14T06:30:02.210 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:30:02.210 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:30:02.210 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:30:02.210 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:30:02.210 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:30:02.210 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:02.210 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 44600 2024-09-14T06:30:02.210 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:02.210 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:30:02.210 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:02.211 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:30:02.211 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.869417+0000", 2024-09-14T06:30:02.211 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:02.211 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:02.211 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:02.211 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:30:02.211 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.869417+0000", 2024-09-14T06:30:02.211 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:02.211 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:02.212 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:02.212 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:30:02.212 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.874912+0000", 2024-09-14T06:30:02.212 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0054948970000000003 2024-09-14T06:30:02.212 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:02.212 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:02.212 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:30:02.212 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.874917+0000", 2024-09-14T06:30:02.212 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.1259999999999997e-06 2024-09-14T06:30:02.212 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:02.212 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:02.212 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:30:02.212 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.874926+0000", 2024-09-14T06:30:02.212 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.7490000000000004e-06 2024-09-14T06:30:02.213 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:02.213 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:02.213 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:30:02.213 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.874939+0000", 2024-09-14T06:30:02.213 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.2831e-05 2024-09-14T06:30:02.213 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:02.213 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:02.213 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:30:02.213 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.937797+0000", 2024-09-14T06:30:02.213 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.062858019000000001 2024-09-14T06:30:02.213 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:02.213 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:02.213 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:30:02.213 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.937879+0000", 2024-09-14T06:30:02.214 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.1932000000000001e-05 2024-09-14T06:30:02.214 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:02.214 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:02.214 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:30:02.214 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.938116+0000", 2024-09-14T06:30:02.214 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00023724799999999999 2024-09-14T06:30:02.214 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:02.214 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:02.214 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:02.214 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:02.214 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:02.214 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:44617 3.17 3:ea948ecb:::rbd_data.112dca2f6ae1.0000000000000004:17 [read 348160~4096 [fadvise_nocache]] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:30:02.214 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:43.875196+0000", 2024-09-14T06:30:02.214 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 18.284633209999999, 2024-09-14T06:30:02.214 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.068462323000000005, 2024-09-14T06:30:02.215 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:30:02.215 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:30:02.215 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:30:02.215 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:30:02.215 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:30:02.215 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:02.215 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 44617 2024-09-14T06:30:02.215 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:02.215 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:30:02.215 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:02.215 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:30:02.216 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.875196+0000", 2024-09-14T06:30:02.216 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:02.216 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:02.216 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:02.216 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:30:02.216 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.875196+0000", 2024-09-14T06:30:02.216 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:02.216 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:02.216 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:02.216 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:30:02.216 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.875197+0000", 2024-09-14T06:30:02.216 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.7100000000000002e-07 2024-09-14T06:30:02.216 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:02.216 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:02.216 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:30:02.217 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.875199+0000", 2024-09-14T06:30:02.217 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.4609999999999998e-06 2024-09-14T06:30:02.217 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:02.217 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:02.217 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:30:02.217 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.875214+0000", 2024-09-14T06:30:02.217 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.4613000000000001e-05 2024-09-14T06:30:02.217 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:02.217 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:02.217 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:30:02.217 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.875221+0000", 2024-09-14T06:30:02.217 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.4309999999999997e-06 2024-09-14T06:30:02.217 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:02.217 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:02.218 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:30:02.218 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.943357+0000", 2024-09-14T06:30:02.218 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.068135412000000006 2024-09-14T06:30:02.218 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:02.218 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:02.218 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:30:02.218 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.943428+0000", 2024-09-14T06:30:02.218 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.0735000000000006e-05 2024-09-14T06:30:02.218 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:02.218 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:02.218 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:30:02.218 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.943659+0000", 2024-09-14T06:30:02.218 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00023120000000000001 2024-09-14T06:30:02.218 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:02.219 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:02.219 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:02.219 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:02.219 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:02.219 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:44618 3.17 3:ea948ecb:::rbd_data.112dca2f6ae1.0000000000000004:17 [read 372736~4096 [fadvise_nocache]] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:30:02.219 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:43.875236+0000", 2024-09-14T06:30:02.219 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 18.284593269999998, 2024-09-14T06:30:02.219 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.068766076999999995, 2024-09-14T06:30:02.219 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:30:02.219 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:30:02.219 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:30:02.219 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:30:02.220 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:30:02.220 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:02.220 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 44618 2024-09-14T06:30:02.220 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:02.220 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:30:02.220 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:02.220 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:30:02.220 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.875236+0000", 2024-09-14T06:30:02.220 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:02.220 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:02.220 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:02.220 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:30:02.220 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.875236+0000", 2024-09-14T06:30:02.221 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:02.221 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:02.221 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:02.221 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:30:02.221 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.875237+0000", 2024-09-14T06:30:02.221 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.8999999999999996e-07 2024-09-14T06:30:02.221 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:02.221 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:02.221 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:30:02.221 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.875239+0000", 2024-09-14T06:30:02.221 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.419e-06 2024-09-14T06:30:02.221 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:02.221 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:02.222 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:30:02.222 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.875245+0000", 2024-09-14T06:30:02.222 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.9909999999999999e-06 2024-09-14T06:30:02.222 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:02.222 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:02.222 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:30:02.222 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.875256+0000", 2024-09-14T06:30:02.222 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.0399e-05 2024-09-14T06:30:02.222 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:02.222 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:02.222 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:30:02.222 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.943690+0000", 2024-09-14T06:30:02.222 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.068434637000000006 2024-09-14T06:30:02.223 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:02.223 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:02.223 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:30:02.223 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.943765+0000", 2024-09-14T06:30:02.223 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.4418000000000005e-05 2024-09-14T06:30:02.223 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:02.223 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:02.223 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:30:02.223 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.944002+0000", 2024-09-14T06:30:02.223 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00023762300000000001 2024-09-14T06:30:02.223 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:02.223 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:02.223 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:02.224 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:02.224 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:02.224 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:44619 3.17 3:ea948ecb:::rbd_data.112dca2f6ae1.0000000000000004:17 [read 397312~4096 [fadvise_nocache]] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:30:02.224 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:43.875263+0000", 2024-09-14T06:30:02.224 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 18.284567002999999, 2024-09-14T06:30:02.224 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.069074144000000004, 2024-09-14T06:30:02.224 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:30:02.224 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:30:02.224 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:30:02.224 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:30:02.224 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:30:02.225 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:02.225 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 44619 2024-09-14T06:30:02.225 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:02.225 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:30:02.225 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:02.225 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:30:02.225 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.875263+0000", 2024-09-14T06:30:02.225 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:02.225 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:02.225 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:02.225 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:30:02.225 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.875263+0000", 2024-09-14T06:30:02.226 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:02.226 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:02.226 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:02.226 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:30:02.226 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.875263+0000", 2024-09-14T06:30:02.226 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.2199999999999999e-07 2024-09-14T06:30:02.226 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:02.226 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:02.226 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:30:02.226 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.875265+0000", 2024-09-14T06:30:02.226 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.1270000000000001e-06 2024-09-14T06:30:02.226 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:02.227 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:02.227 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:30:02.227 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.875271+0000", 2024-09-14T06:30:02.227 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.3329999999999996e-06 2024-09-14T06:30:02.227 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:02.227 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:02.227 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:30:02.227 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.875284+0000", 2024-09-14T06:30:02.227 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.3036e-05 2024-09-14T06:30:02.227 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:02.227 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:02.228 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:30:02.228 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.944032+0000", 2024-09-14T06:30:02.228 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.068748751999999996 2024-09-14T06:30:02.228 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:02.228 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:02.228 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:30:02.228 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.944106+0000", 2024-09-14T06:30:02.228 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.3441000000000004e-05 2024-09-14T06:30:02.228 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:02.228 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:02.228 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:30:02.229 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.944337+0000", 2024-09-14T06:30:02.229 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00023103300000000001 2024-09-14T06:30:02.229 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:02.229 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:02.229 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:02.229 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:02.229 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:02.229 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:44620 3.17 3:ea948ecb:::rbd_data.112dca2f6ae1.0000000000000004:17 [read 421888~4096 [fadvise_nocache]] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:30:02.229 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:43.875291+0000", 2024-09-14T06:30:02.229 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 18.284539119000002, 2024-09-14T06:30:02.229 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.069988984000000004, 2024-09-14T06:30:02.230 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:30:02.230 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:30:02.230 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:30:02.230 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:30:02.230 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:30:02.230 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:02.230 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 44620 2024-09-14T06:30:02.230 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:02.230 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:30:02.231 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:02.231 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:30:02.231 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.875291+0000", 2024-09-14T06:30:02.231 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:02.231 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:02.232 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:02.232 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:30:02.232 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.875291+0000", 2024-09-14T06:30:02.232 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:02.232 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:02.232 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:02.233 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:30:02.233 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.880818+0000", 2024-09-14T06:30:02.233 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0055275350000000001 2024-09-14T06:30:02.233 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:02.233 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:02.233 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:30:02.234 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.880824+0000", 2024-09-14T06:30:02.234 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.4240000000000001e-06 2024-09-14T06:30:02.234 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:02.234 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:02.234 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:30:02.234 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.880834+0000", 2024-09-14T06:30:02.235 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.0096e-05 2024-09-14T06:30:02.235 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:02.235 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:02.235 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:30:02.235 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.880842+0000", 2024-09-14T06:30:02.235 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.6349999999999995e-06 2024-09-14T06:30:02.235 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:02.236 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:02.236 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:30:02.236 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.944380+0000", 2024-09-14T06:30:02.236 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.063537934000000004 2024-09-14T06:30:02.236 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:02.237 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:02.237 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:30:02.237 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.944459+0000", 2024-09-14T06:30:02.237 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.8867000000000002e-05 2024-09-14T06:30:02.237 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:02.237 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:02.238 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:30:02.238 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.945280+0000", 2024-09-14T06:30:02.238 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00082049300000000005 2024-09-14T06:30:02.238 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:02.238 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:02.238 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:02.239 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:02.239 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:02.239 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:44644 3.17 3:ea948ecb:::rbd_data.112dca2f6ae1.0000000000000004:17 [read 1011712~4096 [fadvise_nocache]] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:30:02.239 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:43.881869+0000", 2024-09-14T06:30:02.239 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 18.277961165000001, 2024-09-14T06:30:02.240 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.068512922000000004, 2024-09-14T06:30:02.240 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:30:02.240 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:30:02.240 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:30:02.240 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:30:02.240 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:30:02.241 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:02.241 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 44644 2024-09-14T06:30:02.241 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:02.241 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:30:02.241 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:02.241 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:30:02.242 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.881869+0000", 2024-09-14T06:30:02.242 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:02.242 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:02.242 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:02.242 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:30:02.242 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.881869+0000", 2024-09-14T06:30:02.243 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:02.243 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:02.243 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:02.243 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:30:02.243 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.881869+0000", 2024-09-14T06:30:02.243 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.9299999999999998e-07 2024-09-14T06:30:02.244 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:02.244 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:02.244 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:30:02.244 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.881871+0000", 2024-09-14T06:30:02.244 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.362e-06 2024-09-14T06:30:02.244 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:02.245 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:02.245 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:30:02.245 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.881878+0000", 2024-09-14T06:30:02.245 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.6239999999999996e-06 2024-09-14T06:30:02.245 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:02.245 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:02.245 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:30:02.246 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.881885+0000", 2024-09-14T06:30:02.246 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.2649999999999999e-06 2024-09-14T06:30:02.246 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:02.246 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:02.246 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:30:02.246 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.950183+0000", 2024-09-14T06:30:02.247 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.068298225000000004 2024-09-14T06:30:02.247 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:02.247 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:02.247 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:30:02.247 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.950270+0000", 2024-09-14T06:30:02.248 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.6331999999999999e-05 2024-09-14T06:30:02.248 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:02.248 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:02.248 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:30:02.248 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.950381+0000", 2024-09-14T06:30:02.248 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00011162099999999999 2024-09-14T06:30:02.249 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:02.249 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:02.249 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:02.249 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:02.249 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:02.249 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:44645 3.17 3:ea948ecb:::rbd_data.112dca2f6ae1.0000000000000004:17 [read 1036288~4096 [fadvise_nocache]] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:30:02.249 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:43.881893+0000", 2024-09-14T06:30:02.249 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 18.277936838999999, 2024-09-14T06:30:02.249 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.068700357000000004, 2024-09-14T06:30:02.249 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:30:02.249 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:30:02.249 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:30:02.250 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:30:02.250 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:30:02.250 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:02.250 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 44645 2024-09-14T06:30:02.250 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:02.250 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:30:02.250 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:02.250 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:30:02.250 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.881893+0000", 2024-09-14T06:30:02.250 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:02.250 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:02.250 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:02.250 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:30:02.250 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.881893+0000", 2024-09-14T06:30:02.251 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:02.251 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:02.251 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:02.251 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:30:02.251 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.881893+0000", 2024-09-14T06:30:02.251 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.8299999999999998e-07 2024-09-14T06:30:02.251 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:02.251 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:02.251 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:30:02.251 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.881895+0000", 2024-09-14T06:30:02.251 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.88e-06 2024-09-14T06:30:02.251 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:02.251 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:02.252 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:30:02.252 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.881902+0000", 2024-09-14T06:30:02.252 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.6710000000000002e-06 2024-09-14T06:30:02.252 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:02.252 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:02.252 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:30:02.252 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.881915+0000", 2024-09-14T06:30:02.252 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.312e-05 2024-09-14T06:30:02.253 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:02.253 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:02.253 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:30:02.253 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.950420+0000", 2024-09-14T06:30:02.253 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.068505359000000002 2024-09-14T06:30:02.253 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:02.253 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:02.253 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:30:02.253 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.950497+0000", 2024-09-14T06:30:02.253 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.6454999999999999e-05 2024-09-14T06:30:02.253 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:02.254 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:02.254 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:30:02.254 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.950593+0000", 2024-09-14T06:30:02.254 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.6489000000000004e-05 2024-09-14T06:30:02.254 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:02.254 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:02.254 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:02.254 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:02.254 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:02.254 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:44777 3.17 3:ee6e7e6f:::rbd_data.112dca2f6ae1.0000000000000007:17 [read 8192~4096 [fadvise_nocache]] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:30:02.254 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:43.881935+0000", 2024-09-14T06:30:02.254 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 18.277894642, 2024-09-14T06:30:02.255 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.068908843999999997, 2024-09-14T06:30:02.255 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:30:02.255 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:30:02.255 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:30:02.255 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:30:02.255 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:30:02.255 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:02.255 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 44777 2024-09-14T06:30:02.255 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:02.255 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:30:02.255 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:02.255 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:30:02.255 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.881935+0000", 2024-09-14T06:30:02.256 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:02.256 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:02.256 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:02.256 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:30:02.256 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.881935+0000", 2024-09-14T06:30:02.256 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:02.256 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:02.256 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:02.256 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:30:02.256 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.881936+0000", 2024-09-14T06:30:02.256 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.7100000000000002e-07 2024-09-14T06:30:02.256 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:02.256 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:02.257 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:30:02.257 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.881938+0000", 2024-09-14T06:30:02.257 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.6010000000000002e-06 2024-09-14T06:30:02.257 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:02.257 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:02.257 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:30:02.257 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.881946+0000", 2024-09-14T06:30:02.257 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.9179999999999994e-06 2024-09-14T06:30:02.257 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:02.257 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:02.257 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:30:02.258 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.881951+0000", 2024-09-14T06:30:02.258 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.3199999999999999e-06 2024-09-14T06:30:02.258 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:02.258 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:02.258 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:30:02.258 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.950638+0000", 2024-09-14T06:30:02.258 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.068686149000000002 2024-09-14T06:30:02.258 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:02.258 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:02.258 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:30:02.258 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.950725+0000", 2024-09-14T06:30:02.258 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.7535999999999997e-05 2024-09-14T06:30:02.258 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:02.258 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:02.259 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:30:02.259 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.950844+0000", 2024-09-14T06:30:02.259 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000118749 2024-09-14T06:30:02.259 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:02.259 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:02.259 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:02.259 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:02.259 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:02.259 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:44778 3.17 3:ee6e7e6f:::rbd_data.112dca2f6ae1.0000000000000007:17 [read 32768~4096 [fadvise_nocache]] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:30:02.259 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:43.881960+0000", 2024-09-14T06:30:02.259 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 18.277869996, 2024-09-14T06:30:02.259 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.069098804999999999, 2024-09-14T06:30:02.259 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:30:02.259 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:30:02.260 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:30:02.260 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:30:02.260 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:30:02.260 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:02.260 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 44778 2024-09-14T06:30:02.260 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:02.260 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:30:02.260 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:02.260 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:30:02.260 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.881960+0000", 2024-09-14T06:30:02.260 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:02.260 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:02.260 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:02.261 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:30:02.261 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.881960+0000", 2024-09-14T06:30:02.261 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:02.261 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:02.261 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:02.261 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:30:02.261 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.881960+0000", 2024-09-14T06:30:02.261 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.6199999999999998e-07 2024-09-14T06:30:02.261 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:02.261 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:02.261 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:30:02.261 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.881963+0000", 2024-09-14T06:30:02.261 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.2740000000000002e-06 2024-09-14T06:30:02.261 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:02.262 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:02.262 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:30:02.262 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.881971+0000", 2024-09-14T06:30:02.262 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.1750000000000005e-06 2024-09-14T06:30:02.262 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:02.262 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:02.262 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:30:02.262 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.881977+0000", 2024-09-14T06:30:02.262 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.5030000000000002e-06 2024-09-14T06:30:02.262 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:02.262 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:02.262 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:30:02.263 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.950884+0000", 2024-09-14T06:30:02.263 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.068907066000000003 2024-09-14T06:30:02.263 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:02.263 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:02.263 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:30:02.263 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.950963+0000", 2024-09-14T06:30:02.263 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.8789999999999996e-05 2024-09-14T06:30:02.263 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:02.263 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:02.263 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:30:02.263 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.951058+0000", 2024-09-14T06:30:02.263 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.5434999999999997e-05 2024-09-14T06:30:02.263 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:02.263 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:02.264 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:02.264 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:02.264 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:02.264 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:44779 3.17 3:ee6e7e6f:::rbd_data.112dca2f6ae1.0000000000000007:17 [read 57344~4096 [fadvise_nocache]] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:30:02.264 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:43.881984+0000", 2024-09-14T06:30:02.264 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 18.277845402000001, 2024-09-14T06:30:02.264 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.069297828000000006, 2024-09-14T06:30:02.264 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:30:02.264 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:30:02.264 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:30:02.264 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:30:02.264 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:30:02.264 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:02.265 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 44779 2024-09-14T06:30:02.265 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:02.265 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:30:02.265 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:02.265 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:30:02.265 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.881984+0000", 2024-09-14T06:30:02.265 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:02.265 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:02.265 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:02.265 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:30:02.265 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.881984+0000", 2024-09-14T06:30:02.265 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:02.265 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:02.265 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:02.266 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:30:02.266 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.881985+0000", 2024-09-14T06:30:02.266 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.7099999999999997e-07 2024-09-14T06:30:02.266 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:02.266 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:02.266 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:30:02.266 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.881987+0000", 2024-09-14T06:30:02.266 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.0449999999999999e-06 2024-09-14T06:30:02.267 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:02.267 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:02.267 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:30:02.267 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.881994+0000", 2024-09-14T06:30:02.267 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.3849999999999996e-06 2024-09-14T06:30:02.267 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:02.267 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:02.267 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:30:02.267 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.881999+0000", 2024-09-14T06:30:02.267 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.0050000000000004e-06 2024-09-14T06:30:02.267 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:02.267 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:02.267 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:30:02.267 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.951101+0000", 2024-09-14T06:30:02.267 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.069101878000000005 2024-09-14T06:30:02.268 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:02.268 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:02.268 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:30:02.268 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.951178+0000", 2024-09-14T06:30:02.268 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.6823999999999995e-05 2024-09-14T06:30:02.268 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:02.268 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:02.268 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:30:02.268 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.951282+0000", 2024-09-14T06:30:02.268 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00010432 2024-09-14T06:30:02.268 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:02.268 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:02.268 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:02.269 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:02.269 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:02.269 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:44780 3.17 3:ee6e7e6f:::rbd_data.112dca2f6ae1.0000000000000007:17 [read 81920~4096 [fadvise_nocache]] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:30:02.269 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:43.882006+0000", 2024-09-14T06:30:02.269 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 18.277824091999999, 2024-09-14T06:30:02.269 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.069494627000000003, 2024-09-14T06:30:02.269 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:30:02.269 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:30:02.269 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:30:02.269 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:30:02.270 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:30:02.270 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:02.270 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 44780 2024-09-14T06:30:02.270 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:02.270 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:30:02.270 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:02.270 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:30:02.270 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.882006+0000", 2024-09-14T06:30:02.270 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:02.270 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:02.270 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:02.270 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:30:02.270 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.882006+0000", 2024-09-14T06:30:02.271 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:02.271 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:02.271 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:02.271 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:30:02.271 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.887819+0000", 2024-09-14T06:30:02.271 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0058133489999999998 2024-09-14T06:30:02.271 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:02.271 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:02.271 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:30:02.271 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.887826+0000", 2024-09-14T06:30:02.271 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.3889999999999999e-06 2024-09-14T06:30:02.271 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:02.271 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:02.272 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:30:02.272 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.887847+0000", 2024-09-14T06:30:02.272 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.0984000000000001e-05 2024-09-14T06:30:02.272 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:02.272 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:02.272 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:30:02.272 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.887857+0000", 2024-09-14T06:30:02.272 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.645e-06 2024-09-14T06:30:02.272 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:02.272 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:02.272 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:30:02.272 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.951325+0000", 2024-09-14T06:30:02.272 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.063468399999999994 2024-09-14T06:30:02.273 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:02.273 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:02.273 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:30:02.273 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.951401+0000", 2024-09-14T06:30:02.273 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.5499000000000003e-05 2024-09-14T06:30:02.273 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:02.273 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:02.273 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:30:02.273 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.951500+0000", 2024-09-14T06:30:02.273 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.9361e-05 2024-09-14T06:30:02.273 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:02.273 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:02.274 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:02.274 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:02.274 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:02.274 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-14T06:30:02.275 INFO:teuthology.orchestra.run.smithi094.stdout:1698 write 0x28811e thru 0x2883fb (0x2de bytes) 2024-09-14T06:30:02.276 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-14T06:30:02.276 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-14T06:30:02.276 INFO:teuthology.orchestra.run.smithi094.stdout:osd.4: {} 2024-09-14T06:30:02.277 INFO:teuthology.orchestra.run.smithi094.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-14T06:30:02.277 INFO:teuthology.orchestra.run.smithi094.stdout:1701 write 0x5c09db8 thru 0x5c126d1 (0x891a bytes) 2024-09-14T06:30:02.278 INFO:teuthology.orchestra.run.smithi094.stdout:1706 punch from 0x52c8220 to 0x52ccc19, (0x49f9 bytes) 2024-09-14T06:30:02.278 INFO:teuthology.orchestra.run.smithi094.stdout:1707 writesame 0x59239f7 thru 0x5925e9e data_size 0x8(0x24a8 bytes) 2024-09-14T06:30:02.278 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-14T06:30:02.278 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-14T06:30:02.278 INFO:teuthology.orchestra.run.smithi094.stdout:1710 read 0x36f19f1 thru 0x3700192 (0xe7a2 bytes) 2024-09-14T06:30:02.278 INFO:teuthology.orchestra.run.smithi094.stdout:1712 write 0x411e5d thru 0x418163 (0x6307 bytes) 2024-09-14T06:30:02.278 INFO:teuthology.orchestra.run.smithi094.stdout:osd.5: {} 2024-09-14T06:30:02.278 INFO:teuthology.orchestra.run.smithi094.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-14T06:30:02.279 INFO:teuthology.orchestra.run.smithi094.stdout:1713 punch from 0x2232fbb to 0x22401da, (0xd21f bytes) 2024-09-14T06:30:02.279 INFO:teuthology.orchestra.run.smithi094.stdout:1714 trunc from 0x5c126d2 to 0x6b0f23f 2024-09-14T06:30:02.279 INFO:teuthology.orchestra.run.smithi094.stdout:1715 write 0x36a18ad thru 0x36a9ef4 (0x8648 bytes) 2024-09-14T06:30:02.280 INFO:teuthology.orchestra.run.smithi094.stdout:1716 read 0x3bb086d thru 0x3bb617f (0x5913 bytes) 2024-09-14T06:30:02.280 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-14T06:30:02.280 INFO:teuthology.orchestra.run.smithi094.stdout:1718 write 0x7356f15 thru 0x735dc09 (0x6cf5 bytes) 2024-09-14T06:30:02.280 INFO:teuthology.orchestra.run.smithi094.stdout:1719 write 0x3f0bb99 thru 0x3f1a5fa (0xea62 bytes) 2024-09-14T06:30:02.280 INFO:teuthology.orchestra.run.smithi094.stdout:osd.6: {} 2024-09-14T06:30:02.281 INFO:teuthology.orchestra.run.smithi094.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-14T06:30:02.281 INFO:teuthology.orchestra.run.smithi094.stdout:1720 writesame 0xa1bc185 thru 0xa1c9c2c data_size 0x4(0xdaa8 bytes) 2024-09-14T06:30:02.281 INFO:teuthology.orchestra.run.smithi094.stdout:1723 trunc from 0xa1c9c2d to 0x888db5d 2024-09-14T06:30:02.281 INFO:teuthology.orchestra.run.smithi094.stdout:osd.7: {} 2024-09-14T06:30:02.282 INFO:teuthology.orchestra.run.smithi094.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-14T06:30:02.282 INFO:teuthology.orchestra.run.smithi094.stdout:1724 trunc from 0x888db5d to 0x15b24ef 2024-09-14T06:30:02.283 INFO:teuthology.orchestra.run.smithi094.stdout:1725 write 0xc5797ad thru 0xc57b914 (0x2168 bytes) 2024-09-14T06:30:02.283 INFO:teuthology.orchestra.run.smithi094.stdout:1726 trunc from 0xc57b915 to 0x4013dbe 2024-09-14T06:30:02.284 INFO:tasks.ceph.osd.4.smithi138.stderr:2024-09-14T06:30:02.274+0000 7f6c653c8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33615) UID: 0 2024-09-14T06:30:02.287 INFO:teuthology.orchestra.run.smithi094.stdout:1727 read 0x2f13c2d thru 0x2f1d08f (0x9463 bytes) 2024-09-14T06:30:02.302 INFO:teuthology.orchestra.run.smithi094.stdout:1728 write 0xc1ae191 thru 0xc1b1a10 (0x3880 bytes) 2024-09-14T06:30:02.310 INFO:teuthology.orchestra.run.smithi094.stdout:1729 write 0xd8cc8b5 thru 0xd8d64a7 (0x9bf3 bytes) 2024-09-14T06:30:02.312 INFO:teuthology.orchestra.run.smithi094.stdout:1731 read 0x1678bbc thru 0x16887f7 (0xfc3c bytes) 2024-09-14T06:30:02.312 INFO:teuthology.orchestra.run.smithi094.stdout:1732 read 0x9cb5ed9 thru 0x9cbabbe (0x4ce6 bytes) 2024-09-14T06:30:02.312 INFO:teuthology.orchestra.run.smithi094.stdout:1734 read 0x60f3143 thru 0x60f95d1 (0x648f bytes) 2024-09-14T06:30:02.313 INFO:teuthology.orchestra.run.smithi094.stdout:1735 write 0x9a6ace5 thru 0x9a6e7e0 (0x3afc bytes) 2024-09-14T06:30:02.314 INFO:teuthology.orchestra.run.smithi094.stdout:1737 write 0x551b140 thru 0x55245c3 (0x9484 bytes) 2024-09-14T06:30:02.316 INFO:teuthology.orchestra.run.smithi094.stdout:1738 write 0x23d1aa2 thru 0x23e08ea (0xee49 bytes) 2024-09-14T06:30:02.318 INFO:teuthology.orchestra.run.smithi094.stdout:1739 punch from 0x63afa7d to 0x63b071b, (0xc9e bytes) 2024-09-14T06:30:02.318 INFO:teuthology.orchestra.run.smithi094.stdout:1740 punch from 0x5661d34 to 0x5665090, (0x335c bytes) 2024-09-14T06:30:02.318 INFO:teuthology.orchestra.run.smithi094.stdout:1741 punch from 0x379cc1c to 0x37a6625, (0x9a09 bytes) 2024-09-14T06:30:02.318 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-14T06:30:02.318 INFO:teuthology.orchestra.run.smithi094.stdout:1744 read 0x78c4989 thru 0x78cdf43 (0x95bb bytes) 2024-09-14T06:30:02.318 INFO:teuthology.orchestra.run.smithi094.stdout:1745 write 0xc9b035b thru 0xc9b6a35 (0x66db bytes) 2024-09-14T06:30:02.319 INFO:teuthology.orchestra.run.smithi094.stdout:1747 write 0x178f944 thru 0x179edf7 (0xf4b4 bytes) 2024-09-14T06:30:02.321 INFO:teuthology.orchestra.run.smithi094.stdout:1748 read 0x10d5818 thru 0x10d5d33 (0x51c bytes) 2024-09-14T06:30:02.321 INFO:teuthology.orchestra.run.smithi094.stdout:1749 write 0x5266bd9 thru 0x5275042 (0xe46a bytes) 2024-09-14T06:30:02.322 INFO:teuthology.orchestra.run.smithi094.stdout:1750 write 0xcee0694 thru 0xcee6031 (0x599e bytes) 2024-09-14T06:30:02.325 INFO:teuthology.orchestra.run.smithi094.stdout:1752 punch from 0x4ffec6a to 0x50001d9, (0x156f bytes) 2024-09-14T06:30:02.325 INFO:teuthology.orchestra.run.smithi094.stdout:1754 writesame 0x108a4b5 thru 0x1099eec data_size 0x66(0xfa38 bytes) 2024-09-14T06:30:02.328 INFO:teuthology.orchestra.run.smithi094.stdout:1755 trunc from 0xd8d64a8 to 0xe9f0f2a 2024-09-14T06:30:02.333 INFO:teuthology.orchestra.run.smithi094.stdout:1756 write 0xbb64c20 thru 0xbb71f84 (0xd365 bytes) 2024-09-14T06:30:02.334 INFO:teuthology.orchestra.run.smithi094.stdout:1757 read 0xc0ccdc2 thru 0xc0d65b4 (0x97f3 bytes) 2024-09-14T06:30:02.335 INFO:teuthology.orchestra.run.smithi094.stdout:1758 writesame 0x22b2892 thru 0x22ba51e data_size 0x7(0x7c8d bytes) 2024-09-14T06:30:02.338 INFO:teuthology.orchestra.run.smithi094.stdout:1761 write 0x2c174b2 thru 0x2c19ad1 (0x2620 bytes) 2024-09-14T06:30:02.339 INFO:teuthology.orchestra.run.smithi094.stdout:1762 read 0x9f80177 thru 0x9f8ff47 (0xfdd1 bytes) 2024-09-14T06:30:02.340 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-14T06:30:02.340 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-14T06:30:02.340 INFO:teuthology.orchestra.run.smithi094.stdout:1765 write 0x53cd77 thru 0x544d3d (0x7fc7 bytes) 2024-09-14T06:30:02.341 INFO:teuthology.orchestra.run.smithi094.stdout:1767 read 0x3169cde thru 0x31781b2 (0xe4d5 bytes) 2024-09-14T06:30:02.341 INFO:teuthology.orchestra.run.smithi094.stdout:1768 write 0xfcfd3c thru 0xfd8c45 (0x8f0a bytes) 2024-09-14T06:30:02.343 INFO:teuthology.orchestra.run.smithi094.stdout:1769 writesame 0xae2469 thru 0xae4099 data_size 0x7(0x1c31 bytes) 2024-09-14T06:30:02.363 INFO:teuthology.orchestra.run.smithi094.stdout:1770 trunc from 0xe9f0f2a to 0x6e927c 2024-09-14T06:30:02.375 INFO:tasks.ceph.osd.1.smithi094.stderr:2024-09-14T06:30:02.374+0000 7f328e47a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33641) UID: 0 2024-09-14T06:30:02.387 INFO:teuthology.orchestra.run.smithi094.stdout:1771 write 0x41497cd thru 0x41576b9 (0xdeed bytes) 2024-09-14T06:30:02.402 INFO:teuthology.orchestra.run.smithi094.stdout:1772 write 0xae71a4a thru 0xae80f21 (0xf4d8 bytes) 2024-09-14T06:30:02.407 INFO:teuthology.orchestra.run.smithi094.stdout:1773 trunc from 0xae80f22 to 0xc885f8b 2024-09-14T06:30:02.407 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-14T06:30:02.407 INFO:teuthology.orchestra.run.smithi094.stdout: "ops": [], 2024-09-14T06:30:02.407 INFO:teuthology.orchestra.run.smithi094.stdout: "num_ops": 0 2024-09-14T06:30:02.407 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-14T06:30:02.418 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-14T06:30:02.424 INFO:teuthology.orchestra.run.smithi094.stdout:1774 trunc from 0xc885f8b to 0x2a0dbd8 2024-09-14T06:30:02.430 INFO:teuthology.orchestra.run.smithi094.stdout:1775 read 0xa1e9bb thru 0xa289d8 (0xa01e bytes) 2024-09-14T06:30:02.430 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-14T06:30:02.430 INFO:teuthology.orchestra.run.smithi094.stdout:1778 punch from 0x7b2521 to 0x7b69bd, (0x449c bytes) 2024-09-14T06:30:02.431 INFO:teuthology.orchestra.run.smithi094.stdout:1780 punch from 0x251ca64 to 0x2526285, (0x9821 bytes) 2024-09-14T06:30:02.443 INFO:teuthology.orchestra.run.smithi094.stdout:1781 write 0x944f70c thru 0x9457246 (0x7b3b bytes) 2024-09-14T06:30:02.446 INFO:teuthology.orchestra.run.smithi094.stdout:1782 write 0x2b5537d thru 0x2b55937 (0x5bb bytes) 2024-09-14T06:30:02.447 INFO:teuthology.orchestra.run.smithi094.stdout:1783 write 0x2b38209 thru 0x2b4203e (0x9e36 bytes) 2024-09-14T06:30:02.449 INFO:teuthology.orchestra.run.smithi094.stdout:1784 write 0x5353d8b thru 0x5353eee (0x164 bytes) 2024-09-14T06:30:02.453 INFO:teuthology.orchestra.run.smithi094.stdout:1785 trunc from 0x9457247 to 0xc1ea84c 2024-09-14T06:30:02.470 INFO:teuthology.orchestra.run.smithi094.stdout:1786 trunc from 0xc1ea84c to 0x1a49d89 2024-09-14T06:30:02.476 INFO:tasks.ceph.osd.3.smithi094.stderr:2024-09-14T06:30:02.475+0000 7feeeafe8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33655) UID: 0 2024-09-14T06:30:02.491 INFO:teuthology.orchestra.run.smithi094.stdout:1789 writesame 0xa851814 thru 0xa853e7f data_size 0x2(0x266c bytes) 2024-09-14T06:30:02.492 INFO:teuthology.orchestra.run.smithi094.stdout:1790 read 0x1e5ea3 thru 0x1f0ac8 (0xac26 bytes) 2024-09-14T06:30:02.493 INFO:teuthology.orchestra.run.smithi094.stdout:1792 punch from 0x91a0dae to 0x91a28dc, (0x1b2e bytes) 2024-09-14T06:30:02.494 INFO:teuthology.orchestra.run.smithi094.stdout:1793 read 0x3285a76 thru 0x328c164 (0x66ef bytes) 2024-09-14T06:30:02.501 INFO:teuthology.orchestra.run.smithi094.stdout:1794 write 0xcdcc743 thru 0xcdd5b8f (0x944d bytes) 2024-09-14T06:30:02.503 INFO:teuthology.orchestra.run.smithi094.stdout:1795 punch from 0x6dfd7e7 to 0x6e05792, (0x7fab bytes) 2024-09-14T06:30:02.503 INFO:teuthology.orchestra.run.smithi094.stdout:1796 writesame 0x639baf9 thru 0x63a8e35 data_size 0x1(0xd33d bytes) 2024-09-14T06:30:02.535 INFO:teuthology.orchestra.run.smithi094.stdout:1799 write 0xe9b4ea7 thru 0xe9befa6 (0xa100 bytes) 2024-09-14T06:30:02.538 INFO:teuthology.orchestra.run.smithi094.stdout:1801 write 0x786c1ba thru 0x786f050 (0x2e97 bytes) 2024-09-14T06:30:02.540 INFO:teuthology.orchestra.run.smithi094.stdout:1802 write 0xc59243e thru 0xc59fd02 (0xd8c5 bytes) 2024-09-14T06:30:02.542 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-14T06:30:02.542 INFO:teuthology.orchestra.run.smithi094.stdout: "ops": [], 2024-09-14T06:30:02.542 INFO:teuthology.orchestra.run.smithi094.stdout: "complaint_time": 30, 2024-09-14T06:30:02.542 INFO:teuthology.orchestra.run.smithi094.stdout: "num_blocked_ops": 0 2024-09-14T06:30:02.542 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-14T06:30:02.543 INFO:teuthology.orchestra.run.smithi094.stdout:1803 read 0x6d6cb9b thru 0x6d70a88 (0x3eee bytes) 2024-09-14T06:30:02.555 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-14T06:30:02.557 INFO:teuthology.orchestra.run.smithi094.stdout:1804 trunc from 0xe9befa7 to 0x3855231 2024-09-14T06:30:02.564 INFO:teuthology.orchestra.run.smithi094.stdout:1807 read 0x1e38932 thru 0x1e39c29 (0x12f8 bytes) 2024-09-14T06:30:02.564 INFO:teuthology.orchestra.run.smithi094.stdout:1808 trunc from 0x3855231 to 0x3f979b4 2024-09-14T06:30:02.570 INFO:teuthology.orchestra.run.smithi094.stdout:1809 punch from 0x30b2b4c to 0x30b4cd4, (0x2188 bytes) 2024-09-14T06:30:02.570 INFO:teuthology.orchestra.run.smithi094.stdout:1811 read 0x314614a thru 0x314a45d (0x4314 bytes) 2024-09-14T06:30:02.570 INFO:teuthology.orchestra.run.smithi094.stdout:1812 write 0x2153a55 thru 0x215ccc6 (0x9272 bytes) 2024-09-14T06:30:02.578 INFO:tasks.ceph.osd.1.smithi094.stderr:2024-09-14T06:30:02.577+0000 7f328e47a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33641) UID: 0 2024-09-14T06:30:02.584 INFO:teuthology.orchestra.run.smithi094.stdout:1813 trunc from 0x3f979b4 to 0xd672146 2024-09-14T06:30:02.599 INFO:teuthology.orchestra.run.smithi094.stdout:1814 trunc from 0xd672146 to 0x5ad9061 2024-09-14T06:30:02.605 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-14T06:30:02.609 INFO:teuthology.orchestra.run.smithi094.stdout:1815 read 0x24fde14 thru 0x2504536 (0x6723 bytes) 2024-09-14T06:30:02.617 INFO:teuthology.orchestra.run.smithi094.stdout:1816 write 0x6c6c0c5 thru 0x6c73ea3 (0x7ddf bytes) 2024-09-14T06:30:02.619 INFO:teuthology.orchestra.run.smithi094.stdout:1817 punch from 0x558e352 to 0x55932be, (0x4f6c bytes) 2024-09-14T06:30:02.619 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-14T06:30:02.619 INFO:teuthology.orchestra.run.smithi094.stdout:1819 write 0x328db5d thru 0x3297f25 (0xa3c9 bytes) 2024-09-14T06:30:02.634 INFO:teuthology.orchestra.run.smithi094.stdout:1821 write 0xba1154a thru 0xba153a6 (0x3e5d bytes) 2024-09-14T06:30:02.635 INFO:teuthology.orchestra.run.smithi094.stdout:1822 write 0x9bb6b12 thru 0x9bc4ff5 (0xe4e4 bytes) 2024-09-14T06:30:02.645 INFO:teuthology.orchestra.run.smithi094.stdout:1823 trunc from 0xba153a7 to 0x4e7b5c9 2024-09-14T06:30:02.655 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-14T06:30:02.655 INFO:teuthology.orchestra.run.smithi094.stdout:1825 read 0x4332e01 thru 0x433fc1e (0xce1e bytes) 2024-09-14T06:30:02.663 INFO:teuthology.orchestra.run.smithi094.stdout:1826 write 0x71691e1 thru 0x7174852 (0xb672 bytes) 2024-09-14T06:30:02.665 INFO:teuthology.orchestra.run.smithi094.stdout:1827 punch from 0xe81305 to 0xe8e771, (0xd46c bytes) 2024-09-14T06:30:02.676 INFO:teuthology.orchestra.run.smithi094.stdout:1828 write 0xc5117a1 thru 0xc520b27 (0xf387 bytes) 2024-09-14T06:30:02.678 INFO:teuthology.orchestra.run.smithi094.stdout:1829 write 0x68508de thru 0x6851ddc (0x14ff bytes) 2024-09-14T06:30:02.679 INFO:tasks.ceph.osd.4.smithi138.stderr:2024-09-14T06:30:02.677+0000 7f6c653c8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33615) UID: 0 2024-09-14T06:30:02.679 INFO:teuthology.orchestra.run.smithi094.stdout:1831 read 0x48fee6 thru 0x4921b1 (0x22cc bytes) 2024-09-14T06:30:02.680 INFO:teuthology.orchestra.run.smithi094.stdout:1832 writesame 0x52426d4 thru 0x524ebb5 data_size 0x4f(0xc4e2 bytes) 2024-09-14T06:30:02.681 INFO:teuthology.orchestra.run.smithi094.stdout:1833 trunc from 0xc520b28 to 0xca0c817 2024-09-14T06:30:02.691 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-14T06:30:02.691 INFO:teuthology.orchestra.run.smithi094.stdout: "size": 20, 2024-09-14T06:30:02.692 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 600, 2024-09-14T06:30:02.692 INFO:teuthology.orchestra.run.smithi094.stdout: "ops": [ 2024-09-14T06:30:02.692 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:02.692 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:14918 3.0 3:05c39535:::rbd_data.112d92027590.0000000000000009:f [read 795648~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:30:02.692 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:26.083834+0000", 2024-09-14T06:30:02.692 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 36.605498955999998, 2024-09-14T06:30:02.692 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.041776287000000002, 2024-09-14T06:30:02.692 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:30:02.692 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:30:02.692 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:30:02.692 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:30:02.693 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:30:02.693 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:02.693 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 14918 2024-09-14T06:30:02.693 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:02.693 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:30:02.693 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:02.693 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:30:02.693 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.083834+0000", 2024-09-14T06:30:02.693 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:02.693 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:02.693 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:02.693 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:30:02.693 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.083834+0000", 2024-09-14T06:30:02.693 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:02.694 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:02.694 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:02.694 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:30:02.694 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.083834+0000", 2024-09-14T06:30:02.694 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.0100000000000001e-07 2024-09-14T06:30:02.694 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:02.694 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:02.694 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:30:02.694 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.083835+0000", 2024-09-14T06:30:02.694 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.3900000000000003e-07 2024-09-14T06:30:02.694 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:02.694 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:02.694 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:30:02.694 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.083838+0000", 2024-09-14T06:30:02.695 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.9830000000000001e-06 2024-09-14T06:30:02.695 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:02.695 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:02.695 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:30:02.695 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.083840+0000", 2024-09-14T06:30:02.695 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.4679999999999999e-06 2024-09-14T06:30:02.695 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:02.695 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:02.695 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:30:02.695 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.125455+0000", 2024-09-14T06:30:02.695 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.041614578999999999 2024-09-14T06:30:02.695 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:02.696 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:02.696 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:30:02.696 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.125496+0000", 2024-09-14T06:30:02.696 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.1047000000000002e-05 2024-09-14T06:30:02.696 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:02.696 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:02.696 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:30:02.696 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.125610+0000", 2024-09-14T06:30:02.696 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00011407 2024-09-14T06:30:02.696 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:02.696 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:02.697 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:02.697 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:02.697 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:02.697 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:14919 3.0 3:05c39535:::rbd_data.112d92027590.0000000000000009:f [read 805888~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:30:02.697 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:26.083843+0000", 2024-09-14T06:30:02.697 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 36.605489208000002, 2024-09-14T06:30:02.697 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.041992356000000002, 2024-09-14T06:30:02.697 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:30:02.697 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:30:02.697 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:30:02.697 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:30:02.697 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:30:02.698 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:02.698 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 14919 2024-09-14T06:30:02.698 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:02.698 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:30:02.698 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:02.698 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:30:02.698 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.083843+0000", 2024-09-14T06:30:02.698 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:02.698 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:02.698 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:02.698 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:30:02.698 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.083843+0000", 2024-09-14T06:30:02.698 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:02.698 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:02.698 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:02.699 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:30:02.699 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.083844+0000", 2024-09-14T06:30:02.699 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.3300000000000001e-07 2024-09-14T06:30:02.699 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:02.699 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:02.699 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:30:02.699 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.083845+0000", 2024-09-14T06:30:02.699 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.4729999999999999e-06 2024-09-14T06:30:02.699 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:02.699 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:02.699 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:30:02.699 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.083850+0000", 2024-09-14T06:30:02.700 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.0130000000000001e-06 2024-09-14T06:30:02.700 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:02.700 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:02.700 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:30:02.700 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.083854+0000", 2024-09-14T06:30:02.700 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.4759999999999998e-06 2024-09-14T06:30:02.700 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:02.700 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:02.700 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:30:02.701 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.125636+0000", 2024-09-14T06:30:02.701 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.041782683000000001 2024-09-14T06:30:02.701 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:02.701 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:02.701 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:30:02.701 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.125680+0000", 2024-09-14T06:30:02.701 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.3614e-05 2024-09-14T06:30:02.701 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:02.701 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:02.701 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:30:02.701 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.125836+0000", 2024-09-14T06:30:02.701 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000155864 2024-09-14T06:30:02.702 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:02.702 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:02.702 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:02.702 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:02.702 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:02.702 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:14920 3.0 3:05c39535:::rbd_data.112d92027590.0000000000000009:f [read 816128~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:30:02.702 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:26.083868+0000", 2024-09-14T06:30:02.702 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 36.605464546999997, 2024-09-14T06:30:02.702 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.042166771999999998, 2024-09-14T06:30:02.702 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:30:02.702 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:30:02.702 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:30:02.702 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:30:02.703 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:30:02.703 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:02.703 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 14920 2024-09-14T06:30:02.703 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:02.703 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:30:02.703 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:02.703 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:30:02.704 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.083868+0000", 2024-09-14T06:30:02.704 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:02.704 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:02.704 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:02.704 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:30:02.704 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.083868+0000", 2024-09-14T06:30:02.705 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:02.705 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:02.705 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:02.705 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:30:02.705 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.083869+0000", 2024-09-14T06:30:02.705 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.99e-07 2024-09-14T06:30:02.706 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:02.706 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:02.706 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:30:02.706 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.083870+0000", 2024-09-14T06:30:02.706 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.782e-06 2024-09-14T06:30:02.706 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:02.706 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:02.707 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:30:02.707 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.083876+0000", 2024-09-14T06:30:02.707 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.3809999999999998e-06 2024-09-14T06:30:02.707 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:02.707 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:02.707 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:30:02.707 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.083882+0000", 2024-09-14T06:30:02.707 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.9479999999999996e-06 2024-09-14T06:30:02.708 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:02.708 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:02.708 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:30:02.708 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.125878+0000", 2024-09-14T06:30:02.708 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.041996264999999998 2024-09-14T06:30:02.709 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:02.709 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:02.709 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:30:02.709 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.125929+0000", 2024-09-14T06:30:02.709 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.0498999999999999e-05 2024-09-14T06:30:02.709 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:02.710 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:02.710 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:30:02.710 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.126035+0000", 2024-09-14T06:30:02.710 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000106298 2024-09-14T06:30:02.710 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:02.710 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:02.711 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:02.711 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:02.711 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:02.711 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:14921 3.0 3:05c39535:::rbd_data.112d92027590.0000000000000009:f [read 826368~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:30:02.711 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:26.083889+0000", 2024-09-14T06:30:02.711 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 36.605443536999999, 2024-09-14T06:30:02.711 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.042435309999999997, 2024-09-14T06:30:02.711 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:30:02.711 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:30:02.712 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:30:02.712 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:30:02.712 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:30:02.712 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:02.712 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 14921 2024-09-14T06:30:02.712 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:02.712 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:30:02.712 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:02.712 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:30:02.712 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.083889+0000", 2024-09-14T06:30:02.712 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:02.712 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:02.712 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:02.713 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:30:02.713 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.083889+0000", 2024-09-14T06:30:02.713 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:02.713 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:02.713 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:02.713 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:30:02.713 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.086033+0000", 2024-09-14T06:30:02.713 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0021444179999999999 2024-09-14T06:30:02.713 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:02.713 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:02.713 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:30:02.713 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.086038+0000", 2024-09-14T06:30:02.713 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.0969999999999999e-06 2024-09-14T06:30:02.713 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:02.714 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:02.714 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:30:02.714 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.086046+0000", 2024-09-14T06:30:02.714 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.9440000000000005e-06 2024-09-14T06:30:02.714 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:02.714 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:02.714 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:30:02.714 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.086050+0000", 2024-09-14T06:30:02.714 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.69e-06 2024-09-14T06:30:02.714 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:02.714 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:02.714 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:30:02.714 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.126080+0000", 2024-09-14T06:30:02.715 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.040030237000000003 2024-09-14T06:30:02.715 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:02.715 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:02.715 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:30:02.715 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.126163+0000", 2024-09-14T06:30:02.715 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.2071999999999996e-05 2024-09-14T06:30:02.715 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:02.715 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:02.715 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:30:02.715 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.126324+0000", 2024-09-14T06:30:02.715 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00016185200000000001 2024-09-14T06:30:02.715 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:02.715 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:02.716 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:02.716 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:02.716 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:02.716 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:55627 3.0 3:06490baf:::rbd_data.112dca2f6ae1.0000000000000030:17 [read 2002944~4096 [fadvise_nocache]] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:30:02.716 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:44.599966+0000", 2024-09-14T06:30:02.716 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 18.089366589000001, 2024-09-14T06:30:02.716 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.041868546999999999, 2024-09-14T06:30:02.716 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:30:02.716 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:30:02.716 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:30:02.716 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:30:02.716 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:30:02.716 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:02.717 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 55627 2024-09-14T06:30:02.717 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:02.717 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:30:02.717 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:02.717 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:30:02.717 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.599966+0000", 2024-09-14T06:30:02.717 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:02.717 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:02.717 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:02.717 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:30:02.717 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.599966+0000", 2024-09-14T06:30:02.717 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:02.717 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:02.718 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:02.718 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:30:02.718 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.599966+0000", 2024-09-14T06:30:02.718 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.0900000000000001e-07 2024-09-14T06:30:02.718 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:02.718 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:02.718 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:30:02.718 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.599967+0000", 2024-09-14T06:30:02.718 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.64e-07 2024-09-14T06:30:02.718 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:02.719 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:02.719 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:30:02.719 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.599970+0000", 2024-09-14T06:30:02.719 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.9670000000000001e-06 2024-09-14T06:30:02.719 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:02.719 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:02.719 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:30:02.719 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.599973+0000", 2024-09-14T06:30:02.719 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.632e-06 2024-09-14T06:30:02.719 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:02.719 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:02.720 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:30:02.720 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.641665+0000", 2024-09-14T06:30:02.720 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.041692547000000003 2024-09-14T06:30:02.720 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:02.720 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:02.720 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:30:02.720 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.641740+0000", 2024-09-14T06:30:02.720 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.4559000000000002e-05 2024-09-14T06:30:02.720 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:02.720 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:02.720 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:30:02.720 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.641835+0000", 2024-09-14T06:30:02.721 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.4668999999999998e-05 2024-09-14T06:30:02.721 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:02.721 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:02.721 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:02.721 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:02.721 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:02.721 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:55628 3.0 3:06490baf:::rbd_data.112dca2f6ae1.0000000000000030:17 [read 2027520~4096 [fadvise_nocache]] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:30:02.721 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:44.599976+0000", 2024-09-14T06:30:02.721 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 18.089356802000001, 2024-09-14T06:30:02.722 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.041991621, 2024-09-14T06:30:02.722 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:30:02.722 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:30:02.722 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:30:02.722 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:30:02.722 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:30:02.722 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:02.722 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 55628 2024-09-14T06:30:02.722 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:02.722 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:30:02.722 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:02.722 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:30:02.722 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.599976+0000", 2024-09-14T06:30:02.722 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:02.723 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:02.723 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:02.723 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:30:02.723 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.599976+0000", 2024-09-14T06:30:02.723 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:02.723 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:02.723 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:02.723 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:30:02.723 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.599976+0000", 2024-09-14T06:30:02.723 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.1299999999999999e-07 2024-09-14T06:30:02.723 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:02.723 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:02.723 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:30:02.723 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.599977+0000", 2024-09-14T06:30:02.724 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.7999999999999993e-07 2024-09-14T06:30:02.724 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:02.724 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:02.724 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:30:02.724 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.599980+0000", 2024-09-14T06:30:02.724 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.01e-06 2024-09-14T06:30:02.724 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:02.724 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:02.724 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:30:02.724 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.599983+0000", 2024-09-14T06:30:02.724 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.5409999999999999e-06 2024-09-14T06:30:02.724 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:02.724 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:02.725 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:30:02.725 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.641869+0000", 2024-09-14T06:30:02.725 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.041886235000000001 2024-09-14T06:30:02.725 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:02.725 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:02.725 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:30:02.725 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.641913+0000", 2024-09-14T06:30:02.725 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.4357000000000002e-05 2024-09-14T06:30:02.725 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:02.725 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:02.725 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:30:02.725 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.641967+0000", 2024-09-14T06:30:02.725 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.4284999999999999e-05 2024-09-14T06:30:02.725 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:02.726 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:02.726 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:02.726 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:02.726 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:02.726 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:55629 3.0 3:06490baf:::rbd_data.112dca2f6ae1.0000000000000030:17 [read 2052096~4096 [fadvise_nocache]] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:30:02.726 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:44.599985+0000", 2024-09-14T06:30:02.726 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 18.089347096000001, 2024-09-14T06:30:02.726 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.042183375000000002, 2024-09-14T06:30:02.726 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:30:02.726 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:30:02.726 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:30:02.727 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:30:02.727 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:30:02.727 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:02.727 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 55629 2024-09-14T06:30:02.727 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:02.727 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:30:02.727 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:02.727 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:30:02.727 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.599985+0000", 2024-09-14T06:30:02.727 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:02.727 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:02.727 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:02.727 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:30:02.728 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.599985+0000", 2024-09-14T06:30:02.728 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:02.728 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:02.728 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:02.728 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:30:02.728 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.599986+0000", 2024-09-14T06:30:02.728 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.05e-07 2024-09-14T06:30:02.728 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:02.728 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:02.728 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:30:02.728 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.599987+0000", 2024-09-14T06:30:02.728 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.8400000000000002e-07 2024-09-14T06:30:02.729 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:02.729 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:02.729 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:30:02.729 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.599990+0000", 2024-09-14T06:30:02.729 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.5640000000000001e-06 2024-09-14T06:30:02.729 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:02.729 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:02.729 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:30:02.729 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.599993+0000", 2024-09-14T06:30:02.729 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.615e-06 2024-09-14T06:30:02.729 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:02.729 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:02.729 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:30:02.730 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.642003+0000", 2024-09-14T06:30:02.730 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.042010345999999997 2024-09-14T06:30:02.730 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:02.730 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:02.730 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:30:02.730 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.642080+0000", 2024-09-14T06:30:02.730 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.6538999999999993e-05 2024-09-14T06:30:02.730 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:02.730 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:02.730 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:30:02.730 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.642169+0000", 2024-09-14T06:30:02.731 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.9122000000000005e-05 2024-09-14T06:30:02.731 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:02.731 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:02.731 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:02.731 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:02.731 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:02.731 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:55630 3.0 3:06490baf:::rbd_data.112dca2f6ae1.0000000000000030:17 [read 2076672~4096 [fadvise_nocache]] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:30:02.731 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:44.599996+0000", 2024-09-14T06:30:02.731 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 18.089336618000001, 2024-09-14T06:30:02.731 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.042299615999999998, 2024-09-14T06:30:02.731 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:30:02.731 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:30:02.732 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:30:02.732 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:30:02.732 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:30:02.732 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:02.732 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 55630 2024-09-14T06:30:02.732 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:02.732 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:30:02.732 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:02.732 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:30:02.732 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.599996+0000", 2024-09-14T06:30:02.732 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:02.732 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:02.732 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:02.733 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:30:02.733 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.599996+0000", 2024-09-14T06:30:02.733 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:02.733 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:02.733 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:02.733 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:30:02.733 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.599996+0000", 2024-09-14T06:30:02.733 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.1500000000000001e-07 2024-09-14T06:30:02.733 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:02.733 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:02.733 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:30:02.733 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.599997+0000", 2024-09-14T06:30:02.733 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.33e-07 2024-09-14T06:30:02.734 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:02.734 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:02.734 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:30:02.734 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600000+0000", 2024-09-14T06:30:02.734 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.0240000000000002e-06 2024-09-14T06:30:02.734 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:02.734 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:02.734 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:30:02.734 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600003+0000", 2024-09-14T06:30:02.734 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.3640000000000002e-06 2024-09-14T06:30:02.734 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:02.735 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:02.735 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:30:02.735 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.642200+0000", 2024-09-14T06:30:02.735 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.042197571000000003 2024-09-14T06:30:02.735 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:02.735 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:02.735 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:30:02.735 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.642243+0000", 2024-09-14T06:30:02.735 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.2765000000000001e-05 2024-09-14T06:30:02.735 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:02.735 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:02.735 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:30:02.735 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.642296+0000", 2024-09-14T06:30:02.735 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.2744000000000002e-05 2024-09-14T06:30:02.736 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:02.736 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:02.736 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:02.736 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:02.736 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:02.736 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:55631 3.0 3:06490baf:::rbd_data.112dca2f6ae1.0000000000000030:17 [read 1048576~4096 [fadvise_nocache]] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:30:02.736 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:44.600006+0000", 2024-09-14T06:30:02.736 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 18.08932703, 2024-09-14T06:30:02.736 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.042487905999999999, 2024-09-14T06:30:02.736 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:30:02.736 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:30:02.736 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:30:02.736 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:30:02.736 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:30:02.737 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:02.737 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 55631 2024-09-14T06:30:02.737 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:02.737 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:30:02.737 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:02.737 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:30:02.737 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600006+0000", 2024-09-14T06:30:02.737 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:02.737 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:02.737 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:02.737 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:30:02.737 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600006+0000", 2024-09-14T06:30:02.737 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:02.737 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:02.738 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:02.738 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:30:02.738 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600006+0000", 2024-09-14T06:30:02.738 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.0800000000000001e-07 2024-09-14T06:30:02.738 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:02.738 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:02.738 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:30:02.738 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600007+0000", 2024-09-14T06:30:02.738 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.7199999999999997e-07 2024-09-14T06:30:02.738 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:02.738 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:02.738 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:30:02.738 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600010+0000", 2024-09-14T06:30:02.738 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.0280000000000001e-06 2024-09-14T06:30:02.738 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:02.739 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:02.739 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:30:02.739 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600012+0000", 2024-09-14T06:30:02.739 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.475e-06 2024-09-14T06:30:02.739 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:02.739 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:02.739 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:30:02.739 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.642329+0000", 2024-09-14T06:30:02.739 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.042317157000000001 2024-09-14T06:30:02.739 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:02.739 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:02.739 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:30:02.739 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.642403+0000", 2024-09-14T06:30:02.739 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.3941000000000002e-05 2024-09-14T06:30:02.739 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:02.740 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:02.740 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:30:02.740 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.642493+0000", 2024-09-14T06:30:02.740 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.0124999999999995e-05 2024-09-14T06:30:02.740 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:02.740 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:02.740 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:02.740 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:02.740 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:02.740 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:55632 3.0 3:06490baf:::rbd_data.112dca2f6ae1.0000000000000030:17 [read 1073152~4096 [fadvise_nocache]] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:30:02.740 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:44.600015+0000", 2024-09-14T06:30:02.740 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 18.089317276999999, 2024-09-14T06:30:02.740 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.042604922000000003, 2024-09-14T06:30:02.741 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:30:02.741 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:30:02.741 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:30:02.741 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:30:02.741 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:30:02.741 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:02.741 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 55632 2024-09-14T06:30:02.741 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:02.741 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:30:02.741 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:02.742 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:30:02.742 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600015+0000", 2024-09-14T06:30:02.742 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:02.742 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:02.742 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:02.742 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:30:02.742 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600015+0000", 2024-09-14T06:30:02.742 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:02.742 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:02.742 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:02.742 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:30:02.742 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600016+0000", 2024-09-14T06:30:02.742 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.1e-07 2024-09-14T06:30:02.742 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:02.742 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:02.743 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:30:02.743 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600017+0000", 2024-09-14T06:30:02.743 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.9900000000000009e-07 2024-09-14T06:30:02.743 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:02.743 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:02.743 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:30:02.743 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600020+0000", 2024-09-14T06:30:02.743 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.4170000000000001e-06 2024-09-14T06:30:02.743 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:02.743 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:02.743 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:30:02.743 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600023+0000", 2024-09-14T06:30:02.743 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.588e-06 2024-09-14T06:30:02.743 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:02.744 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:02.744 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:30:02.744 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.642526+0000", 2024-09-14T06:30:02.744 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.042503236999999999 2024-09-14T06:30:02.744 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:02.744 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:02.744 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:30:02.744 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.642568+0000", 2024-09-14T06:30:02.744 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.1925000000000003e-05 2024-09-14T06:30:02.744 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:02.744 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:02.744 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:30:02.744 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.642620+0000", 2024-09-14T06:30:02.744 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.2546000000000001e-05 2024-09-14T06:30:02.745 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:02.745 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:02.745 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:02.745 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:02.745 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:02.745 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:55633 3.0 3:06490baf:::rbd_data.112dca2f6ae1.0000000000000030:17 [read 1097728~4096 [fadvise_nocache]] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:30:02.745 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:44.600026+0000", 2024-09-14T06:30:02.745 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 18.089307007999999, 2024-09-14T06:30:02.745 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.042797474000000002, 2024-09-14T06:30:02.745 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:30:02.745 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:30:02.745 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:30:02.745 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:30:02.745 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:30:02.746 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:02.746 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 55633 2024-09-14T06:30:02.746 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:02.746 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:30:02.746 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:02.746 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:30:02.746 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600026+0000", 2024-09-14T06:30:02.746 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:02.746 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:02.746 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:02.746 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:30:02.746 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600026+0000", 2024-09-14T06:30:02.746 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:02.747 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:02.747 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:02.747 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:30:02.747 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600026+0000", 2024-09-14T06:30:02.747 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.3099999999999999e-07 2024-09-14T06:30:02.747 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:02.747 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:02.747 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:30:02.747 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600029+0000", 2024-09-14T06:30:02.747 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.304e-06 2024-09-14T06:30:02.747 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:02.747 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:02.747 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:30:02.748 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600034+0000", 2024-09-14T06:30:02.748 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.1429999999999997e-06 2024-09-14T06:30:02.748 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:02.748 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:02.748 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:30:02.748 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600038+0000", 2024-09-14T06:30:02.748 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.692e-06 2024-09-14T06:30:02.748 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:02.748 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:02.748 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:30:02.748 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.642654+0000", 2024-09-14T06:30:02.748 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.042615564000000002 2024-09-14T06:30:02.749 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:02.749 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:02.749 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:30:02.749 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.642728+0000", 2024-09-14T06:30:02.749 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.4018999999999997e-05 2024-09-14T06:30:02.749 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:02.749 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:02.749 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:30:02.749 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.642823+0000", 2024-09-14T06:30:02.749 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.5520999999999996e-05 2024-09-14T06:30:02.749 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:02.749 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:02.749 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:02.749 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:02.750 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:02.750 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:55634 3.0 3:06490baf:::rbd_data.112dca2f6ae1.0000000000000030:17 [read 1122304~4096 [fadvise_nocache]] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:30:02.750 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:44.600042+0000", 2024-09-14T06:30:02.750 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 18.089290531, 2024-09-14T06:30:02.750 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.042903729000000002, 2024-09-14T06:30:02.750 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:30:02.750 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:30:02.750 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:30:02.750 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:30:02.750 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:30:02.750 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:02.750 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 55634 2024-09-14T06:30:02.751 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:02.751 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:30:02.751 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:02.751 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:30:02.751 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600042+0000", 2024-09-14T06:30:02.751 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:02.751 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:02.751 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:02.751 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:30:02.751 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600042+0000", 2024-09-14T06:30:02.751 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:02.751 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:02.751 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:02.751 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:30:02.752 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600043+0000", 2024-09-14T06:30:02.752 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.7000000000000005e-07 2024-09-14T06:30:02.752 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:02.752 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:02.752 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:30:02.752 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600044+0000", 2024-09-14T06:30:02.752 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.742e-06 2024-09-14T06:30:02.752 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:02.752 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:02.752 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:30:02.752 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600050+0000", 2024-09-14T06:30:02.752 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.0800000000000002e-06 2024-09-14T06:30:02.752 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:02.753 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:02.753 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:30:02.753 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600055+0000", 2024-09-14T06:30:02.753 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.3889999999999998e-06 2024-09-14T06:30:02.753 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:02.753 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:02.753 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:30:02.753 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.642845+0000", 2024-09-14T06:30:02.753 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.042790040000000001 2024-09-14T06:30:02.754 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:02.754 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:02.754 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:30:02.754 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.642888+0000", 2024-09-14T06:30:02.754 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.2790000000000002e-05 2024-09-14T06:30:02.754 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:02.754 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:02.754 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:30:02.754 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.642946+0000", 2024-09-14T06:30:02.754 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.8118000000000002e-05 2024-09-14T06:30:02.754 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:02.755 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:02.755 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:02.755 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:02.755 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:02.755 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:55635 3.0 3:06490baf:::rbd_data.112dca2f6ae1.0000000000000030:17 [read 1146880~4096 [fadvise_nocache]] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:30:02.755 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:44.600061+0000", 2024-09-14T06:30:02.755 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 18.089271960000001, 2024-09-14T06:30:02.755 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.043092695, 2024-09-14T06:30:02.755 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:30:02.755 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:30:02.755 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:30:02.755 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:30:02.756 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:30:02.756 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:02.756 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 55635 2024-09-14T06:30:02.756 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:02.756 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:30:02.756 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:02.756 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:30:02.756 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600061+0000", 2024-09-14T06:30:02.756 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:02.756 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:02.757 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:02.757 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:30:02.757 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600061+0000", 2024-09-14T06:30:02.757 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:02.757 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:02.757 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:02.760 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:30:02.760 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600061+0000", 2024-09-14T06:30:02.760 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.3099999999999999e-07 2024-09-14T06:30:02.760 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:02.760 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:02.761 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:30:02.761 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600063+0000", 2024-09-14T06:30:02.761 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.6220000000000001e-06 2024-09-14T06:30:02.761 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:02.761 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:02.761 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:30:02.761 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600068+0000", 2024-09-14T06:30:02.761 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.4790000000000004e-06 2024-09-14T06:30:02.761 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:02.761 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:02.761 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:30:02.761 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600072+0000", 2024-09-14T06:30:02.762 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.3100000000000002e-06 2024-09-14T06:30:02.762 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:02.762 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:02.762 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:30:02.762 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.642987+0000", 2024-09-14T06:30:02.762 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.042914489 2024-09-14T06:30:02.762 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:02.762 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:02.762 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:30:02.762 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.643062+0000", 2024-09-14T06:30:02.762 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.5011000000000004e-05 2024-09-14T06:30:02.762 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:02.762 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:02.763 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:30:02.763 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.643153+0000", 2024-09-14T06:30:02.763 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.1452999999999993e-05 2024-09-14T06:30:02.763 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:02.763 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:02.763 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:02.763 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:02.763 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:02.763 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:55636 3.0 3:06490baf:::rbd_data.112dca2f6ae1.0000000000000030:17 [read 1171456~4096 [fadvise_nocache]] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:30:02.763 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:44.600078+0000", 2024-09-14T06:30:02.763 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 18.089254548, 2024-09-14T06:30:02.763 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.043207966, 2024-09-14T06:30:02.763 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:30:02.764 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:30:02.764 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:30:02.764 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:30:02.764 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:30:02.764 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:02.764 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 55636 2024-09-14T06:30:02.764 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:02.764 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:30:02.764 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:02.764 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:30:02.764 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600078+0000", 2024-09-14T06:30:02.764 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:02.764 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:02.765 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:02.765 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:30:02.765 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600078+0000", 2024-09-14T06:30:02.765 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:02.765 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:02.765 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:02.765 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:30:02.765 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600078+0000", 2024-09-14T06:30:02.765 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.34e-07 2024-09-14T06:30:02.765 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:02.765 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:02.765 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:30:02.765 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600080+0000", 2024-09-14T06:30:02.766 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.641e-06 2024-09-14T06:30:02.766 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:02.766 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:02.766 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:30:02.766 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600086+0000", 2024-09-14T06:30:02.766 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.3860000000000004e-06 2024-09-14T06:30:02.766 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:02.766 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:02.766 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:30:02.766 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600091+0000", 2024-09-14T06:30:02.766 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.5970000000000002e-06 2024-09-14T06:30:02.766 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:02.767 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:02.767 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:30:02.767 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.643184+0000", 2024-09-14T06:30:02.767 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.043092893 2024-09-14T06:30:02.767 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:02.767 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:02.767 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:30:02.767 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.643228+0000", 2024-09-14T06:30:02.767 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.4536999999999997e-05 2024-09-14T06:30:02.767 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:02.767 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:02.767 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:30:02.768 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.643286+0000", 2024-09-14T06:30:02.768 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.7578000000000003e-05 2024-09-14T06:30:02.768 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:02.768 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:02.768 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:02.768 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:02.769 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:02.769 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:55637 3.0 3:06490baf:::rbd_data.112dca2f6ae1.0000000000000030:17 [read 1196032~4096 [fadvise_nocache]] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:30:02.769 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:44.600097+0000", 2024-09-14T06:30:02.769 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 18.089235983999998, 2024-09-14T06:30:02.769 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.043399124999999997, 2024-09-14T06:30:02.769 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:30:02.770 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:30:02.770 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:30:02.770 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:30:02.770 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:30:02.770 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:02.770 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 55637 2024-09-14T06:30:02.771 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:02.771 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:30:02.771 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:02.771 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:30:02.771 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600097+0000", 2024-09-14T06:30:02.771 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:02.771 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:02.772 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:02.772 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:30:02.772 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600097+0000", 2024-09-14T06:30:02.772 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:02.772 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:02.772 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:02.773 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:30:02.773 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.604372+0000", 2024-09-14T06:30:02.773 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0042751489999999998 2024-09-14T06:30:02.773 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:02.773 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:02.773 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:30:02.774 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.604377+0000", 2024-09-14T06:30:02.774 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.7899999999999999e-06 2024-09-14T06:30:02.774 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:02.774 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:02.774 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:30:02.774 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.604389+0000", 2024-09-14T06:30:02.775 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.2197e-05 2024-09-14T06:30:02.775 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:02.775 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:02.775 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:30:02.775 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.604397+0000", 2024-09-14T06:30:02.775 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.8730000000000007e-06 2024-09-14T06:30:02.776 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:02.776 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:02.776 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:30:02.776 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.643324+0000", 2024-09-14T06:30:02.776 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.038926897000000002 2024-09-14T06:30:02.776 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:02.777 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:02.777 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:30:02.777 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.643402+0000", 2024-09-14T06:30:02.777 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.8063000000000001e-05 2024-09-14T06:30:02.777 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:02.777 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:02.777 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:30:02.777 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.643496+0000", 2024-09-14T06:30:02.777 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.4155999999999998e-05 2024-09-14T06:30:02.777 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:02.777 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:02.777 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:02.778 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:02.778 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:02.778 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:55657 3.0 3:06490baf:::rbd_data.112dca2f6ae1.0000000000000030:17 [read 1687552~4096 [fadvise_nocache]] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:30:02.778 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:44.604841+0000", 2024-09-14T06:30:02.778 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 18.084491255, 2024-09-14T06:30:02.778 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.041890087999999999, 2024-09-14T06:30:02.778 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:30:02.778 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:30:02.778 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:30:02.778 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:30:02.778 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:30:02.778 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:02.779 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 55657 2024-09-14T06:30:02.779 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:02.779 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:30:02.779 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:02.779 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:30:02.779 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.604841+0000", 2024-09-14T06:30:02.779 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:02.779 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:02.779 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:02.779 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:30:02.779 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.604841+0000", 2024-09-14T06:30:02.780 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:02.780 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:02.780 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:02.780 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:30:02.780 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.604842+0000", 2024-09-14T06:30:02.780 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.0900000000000001e-07 2024-09-14T06:30:02.780 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:02.780 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:02.780 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:30:02.780 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.604846+0000", 2024-09-14T06:30:02.780 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.5759999999999999e-06 2024-09-14T06:30:02.780 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:02.780 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:02.781 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:30:02.781 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.604850+0000", 2024-09-14T06:30:02.781 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.405e-06 2024-09-14T06:30:02.781 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:02.781 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:02.781 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:30:02.781 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.604852+0000", 2024-09-14T06:30:02.781 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.8590000000000001e-06 2024-09-14T06:30:02.781 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:02.781 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:02.781 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:30:02.781 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.646633+0000", 2024-09-14T06:30:02.781 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.041780772000000001 2024-09-14T06:30:02.782 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:02.782 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:02.782 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:30:02.782 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.646677+0000", 2024-09-14T06:30:02.782 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.4115e-05 2024-09-14T06:30:02.782 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:02.783 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:02.783 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:30:02.783 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.646731+0000", 2024-09-14T06:30:02.783 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.4151999999999999e-05 2024-09-14T06:30:02.783 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:02.784 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:02.784 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:02.784 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:02.784 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:02.784 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:55658 3.0 3:06490baf:::rbd_data.112dca2f6ae1.0000000000000030:17 [read 1712128~4096 [fadvise_nocache]] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:30:02.784 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:44.604856+0000", 2024-09-14T06:30:02.785 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 18.084476655, 2024-09-14T06:30:02.785 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.041986227000000001, 2024-09-14T06:30:02.785 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:30:02.785 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:30:02.785 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:30:02.785 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:30:02.785 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:30:02.785 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:02.785 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 55658 2024-09-14T06:30:02.785 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:02.785 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:30:02.785 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:02.786 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:30:02.786 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.604856+0000", 2024-09-14T06:30:02.786 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:02.786 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:02.786 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:02.786 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:30:02.786 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.604856+0000", 2024-09-14T06:30:02.786 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:02.786 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:02.786 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:02.786 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:30:02.786 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.604856+0000", 2024-09-14T06:30:02.786 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.23e-07 2024-09-14T06:30:02.787 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:02.787 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:02.787 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:30:02.787 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.604857+0000", 2024-09-14T06:30:02.787 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.9999999999999995e-07 2024-09-14T06:30:02.787 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:02.787 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:02.787 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:30:02.787 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.604861+0000", 2024-09-14T06:30:02.787 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.9890000000000003e-06 2024-09-14T06:30:02.787 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:02.787 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:02.787 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:30:02.787 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.604864+0000", 2024-09-14T06:30:02.787 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.7429999999999998e-06 2024-09-14T06:30:02.788 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:02.788 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:02.788 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:30:02.788 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.646744+0000", 2024-09-14T06:30:02.788 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.041880325000000003 2024-09-14T06:30:02.788 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:02.788 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:02.788 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:30:02.788 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.646784+0000", 2024-09-14T06:30:02.788 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.9604000000000002e-05 2024-09-14T06:30:02.788 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:02.788 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:02.788 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:30:02.788 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.646842+0000", 2024-09-14T06:30:02.789 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.8343e-05 2024-09-14T06:30:02.789 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:02.789 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:02.789 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:02.789 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:02.789 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:02.789 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:55659 3.0 3:06490baf:::rbd_data.112dca2f6ae1.0000000000000030:17 [read 1736704~4096 [fadvise_nocache]] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:30:02.789 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:44.604867+0000", 2024-09-14T06:30:02.789 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 18.084465502, 2024-09-14T06:30:02.790 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.042097082000000001, 2024-09-14T06:30:02.790 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:30:02.790 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:30:02.790 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:30:02.790 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:30:02.790 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:30:02.790 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:02.790 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 55659 2024-09-14T06:30:02.790 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:02.790 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:30:02.790 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:02.790 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:30:02.791 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.604867+0000", 2024-09-14T06:30:02.791 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:02.791 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:02.791 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:02.791 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:30:02.791 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.604867+0000", 2024-09-14T06:30:02.791 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:02.791 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:02.791 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:02.791 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:30:02.791 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.604867+0000", 2024-09-14T06:30:02.791 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.2000000000000001e-07 2024-09-14T06:30:02.791 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:02.791 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:02.792 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:30:02.792 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.604868+0000", 2024-09-14T06:30:02.792 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.04e-06 2024-09-14T06:30:02.792 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:02.792 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:02.792 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:30:02.792 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.604871+0000", 2024-09-14T06:30:02.792 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.9689999999999999e-06 2024-09-14T06:30:02.792 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:02.792 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:02.793 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:30:02.793 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.604874+0000", 2024-09-14T06:30:02.793 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.5069999999999999e-06 2024-09-14T06:30:02.793 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:02.793 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:02.793 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:30:02.793 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.646874+0000", 2024-09-14T06:30:02.793 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.042000513000000003 2024-09-14T06:30:02.793 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:02.793 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:02.793 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:30:02.793 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.646914+0000", 2024-09-14T06:30:02.794 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.9443999999999997e-05 2024-09-14T06:30:02.794 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:02.794 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:02.794 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:30:02.794 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.646964+0000", 2024-09-14T06:30:02.794 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.0389000000000002e-05 2024-09-14T06:30:02.794 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:02.794 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:02.794 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:02.794 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:02.794 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:02.794 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:55660 3.0 3:06490baf:::rbd_data.112dca2f6ae1.0000000000000030:17 [read 1761280~4096 [fadvise_nocache]] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:30:02.794 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:44.604877+0000", 2024-09-14T06:30:02.795 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 18.084455753, 2024-09-14T06:30:02.795 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.042196349000000001, 2024-09-14T06:30:02.795 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:30:02.795 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:30:02.795 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:30:02.795 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:30:02.796 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:30:02.796 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:02.796 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 55660 2024-09-14T06:30:02.796 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:02.796 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:30:02.797 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:02.797 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:30:02.797 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.604877+0000", 2024-09-14T06:30:02.797 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:02.797 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:02.798 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:02.798 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:30:02.798 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.604877+0000", 2024-09-14T06:30:02.798 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:02.798 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:02.799 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:02.799 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:30:02.799 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.605810+0000", 2024-09-14T06:30:02.799 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00093336899999999997 2024-09-14T06:30:02.799 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:02.799 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:02.800 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:30:02.800 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.605815+0000", 2024-09-14T06:30:02.800 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.6140000000000002e-06 2024-09-14T06:30:02.800 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:02.800 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:02.800 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:30:02.801 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.605825+0000", 2024-09-14T06:30:02.801 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.8889999999999997e-06 2024-09-14T06:30:02.801 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:02.801 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:02.801 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:30:02.801 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.605831+0000", 2024-09-14T06:30:02.802 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.9800000000000003e-06 2024-09-14T06:30:02.802 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:02.802 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:02.802 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:30:02.802 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.646989+0000", 2024-09-14T06:30:02.803 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.041157878000000002 2024-09-14T06:30:02.803 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:02.803 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:02.803 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:30:02.803 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.647026+0000", 2024-09-14T06:30:02.803 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.7305000000000003e-05 2024-09-14T06:30:02.804 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:02.804 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:02.804 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:30:02.804 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.647073+0000", 2024-09-14T06:30:02.804 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.7314000000000001e-05 2024-09-14T06:30:02.804 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:02.804 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:02.805 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:02.805 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:02.805 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:02.805 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:55666 3.0 3:06490baf:::rbd_data.112dca2f6ae1.0000000000000030:17 [read 1908736~4096 [fadvise_nocache]] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:30:02.805 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:44.605898+0000", 2024-09-14T06:30:02.805 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 18.083435006999999, 2024-09-14T06:30:02.805 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.041862887000000001, 2024-09-14T06:30:02.806 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:30:02.806 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:30:02.806 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:30:02.806 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:30:02.806 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:30:02.806 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:02.806 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 55666 2024-09-14T06:30:02.807 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:02.807 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:30:02.807 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:02.807 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:30:02.807 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.605898+0000", 2024-09-14T06:30:02.808 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:02.808 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:02.808 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:02.808 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:30:02.808 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.605898+0000", 2024-09-14T06:30:02.809 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:02.809 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:02.809 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:02.809 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:30:02.809 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.611874+0000", 2024-09-14T06:30:02.810 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0059764049999999997 2024-09-14T06:30:02.810 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:02.810 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:02.810 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:30:02.810 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.611879+0000", 2024-09-14T06:30:02.810 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.8749999999999999e-06 2024-09-14T06:30:02.811 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:02.811 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:02.811 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:30:02.811 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.611887+0000", 2024-09-14T06:30:02.811 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.4109999999999996e-06 2024-09-14T06:30:02.812 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:02.812 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:02.812 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:30:02.812 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.611893+0000", 2024-09-14T06:30:02.812 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.2959999999999998e-06 2024-09-14T06:30:02.812 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:02.812 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:02.813 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:30:02.813 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.647674+0000", 2024-09-14T06:30:02.813 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.035781281999999998 2024-09-14T06:30:02.813 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:02.813 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:02.813 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:30:02.813 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.647711+0000", 2024-09-14T06:30:02.814 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.7434000000000001e-05 2024-09-14T06:30:02.814 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:02.814 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:02.814 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:30:02.814 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.647760+0000", 2024-09-14T06:30:02.814 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.9184000000000002e-05 2024-09-14T06:30:02.814 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:02.814 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:02.815 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:02.815 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:02.815 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:02.815 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-14T06:30:02.816 INFO:teuthology.orchestra.run.smithi094.stdout:1834 trunc from 0xca0c817 to 0x75d0e10 2024-09-14T06:30:02.817 INFO:tasks.ceph.osd.5.smithi138.stderr:2024-09-14T06:30:02.815+0000 7fe13746f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33618) UID: 0 2024-09-14T06:30:02.818 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-14T06:30:02.820 INFO:teuthology.orchestra.run.smithi094.stdout:1835 write 0x8eef325 thru 0x8ef76ba (0x8396 bytes) 2024-09-14T06:30:02.821 INFO:teuthology.orchestra.run.smithi094.stdout:1836 punch from 0x6bee1fa to 0x6bef60a, (0x1410 bytes) 2024-09-14T06:30:02.821 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-14T06:30:02.821 INFO:teuthology.orchestra.run.smithi094.stdout:1840 read 0x8b64ab thru 0x8b86b9 (0x220f bytes) 2024-09-14T06:30:02.821 INFO:teuthology.orchestra.run.smithi094.stdout:1842 write 0x450d9cd thru 0x451affb (0xd62f bytes) 2024-09-14T06:30:02.821 INFO:teuthology.orchestra.run.smithi094.stdout:1843 punch from 0xbb72be to 0xbc3045, (0xbd87 bytes) 2024-09-14T06:30:02.821 INFO:teuthology.orchestra.run.smithi094.stdout:1845 write 0xab0d685 thru 0xab109bc (0x3338 bytes) 2024-09-14T06:30:02.821 INFO:teuthology.orchestra.run.smithi094.stdout:1846 trunc from 0xab109bd to 0xa8a2746 2024-09-14T06:30:02.821 INFO:teuthology.orchestra.run.smithi094.stdout:1847 punch from 0xa047061 to 0xa049fd7, (0x2f76 bytes) 2024-09-14T06:30:02.821 INFO:teuthology.orchestra.run.smithi094.stdout:1848 trunc from 0xa8a2746 to 0x87ee1b1 2024-09-14T06:30:02.821 INFO:teuthology.orchestra.run.smithi094.stdout:1851 writesame 0xea21b9b thru 0xea23e35 data_size 0x1(0x229b bytes) 2024-09-14T06:30:02.821 INFO:teuthology.orchestra.run.smithi094.stdout:1852 read 0xc7c315f thru 0xc7cbb70 (0x8a12 bytes) 2024-09-14T06:30:02.821 INFO:teuthology.orchestra.run.smithi094.stdout:1856 read 0x36dd57f thru 0x36ebcef (0xe771 bytes) 2024-09-14T06:30:02.822 INFO:teuthology.orchestra.run.smithi094.stdout:1857 writesame 0xcefa5fb thru 0xcf00c9d data_size 0x5(0x66a3 bytes) 2024-09-14T06:30:02.822 INFO:teuthology.orchestra.run.smithi094.stdout:1859 read 0x48b782d thru 0x48c1c78 (0xa44c bytes) 2024-09-14T06:30:02.822 INFO:teuthology.orchestra.run.smithi094.stdout:1861 writesame 0x12d0627 thru 0x12d896f data_size 0x11(0x8349 bytes) 2024-09-14T06:30:02.822 INFO:teuthology.orchestra.run.smithi094.stdout:1863 punch from 0xba8c315 to 0xba92260, (0x5f4b bytes) 2024-09-14T06:30:02.822 INFO:teuthology.orchestra.run.smithi094.stdout:1864 read 0x7f6f22d thru 0x7f7b64e (0xc422 bytes) 2024-09-14T06:30:02.822 INFO:teuthology.orchestra.run.smithi094.stdout:1865 write 0x90404a8 thru 0x904e658 (0xe1b1 bytes) 2024-09-14T06:30:02.822 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-14T06:30:02.822 INFO:teuthology.orchestra.run.smithi094.stdout:1867 write 0xbfb9e6 thru 0xbfc744 (0xd5f bytes) 2024-09-14T06:30:02.822 INFO:teuthology.orchestra.run.smithi094.stdout:1870 trunc from 0xea23e36 to 0x7f13ff8 2024-09-14T06:30:02.822 INFO:teuthology.orchestra.run.smithi094.stdout:1871 writesame 0x88e27aa thru 0x88edec5 data_size 0x2(0xb71c bytes) 2024-09-14T06:30:02.822 INFO:teuthology.orchestra.run.smithi094.stdout:1872 writesame 0x48e84f4 thru 0x48ecae8 data_size 0x1(0x45f5 bytes) 2024-09-14T06:30:02.822 INFO:teuthology.orchestra.run.smithi094.stdout:1873 read 0x545f3b0 thru 0x5469488 (0xa0d9 bytes) 2024-09-14T06:30:02.822 INFO:teuthology.orchestra.run.smithi094.stdout:1874 punch from 0x5dad029 to 0x5dbc371, (0xf348 bytes) 2024-09-14T06:30:02.822 INFO:teuthology.orchestra.run.smithi094.stdout:1875 write 0x9e364a8 thru 0x9e40249 (0x9da2 bytes) 2024-09-14T06:30:02.823 INFO:teuthology.orchestra.run.smithi094.stderr:reweighted osd.4 to 1 (10000) 2024-09-14T06:30:02.823 INFO:teuthology.orchestra.run.smithi094.stdout:1876 writesame 0xa1d246 thru 0xa1f6de data_size 0x9(0x2499 bytes) 2024-09-14T06:30:02.823 INFO:teuthology.orchestra.run.smithi094.stdout:1877 write 0x478bd8d thru 0x4794046 (0x82ba bytes) 2024-09-14T06:30:02.823 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-14T06:30:02.824 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-14T06:30:02.824 INFO:teuthology.orchestra.run.smithi094.stdout:1880 write 0x10318a thru 0x10417d (0xff4 bytes) 2024-09-14T06:30:02.824 INFO:teuthology.orchestra.run.smithi094.stdout:1881 write 0x80be6bf thru 0x80c93ad (0xacef bytes) 2024-09-14T06:30:02.825 INFO:teuthology.orchestra.run.smithi094.stdout:1882 writesame 0xb93f7b7 thru 0xb940a8b data_size 0x1(0x12d5 bytes) 2024-09-14T06:30:02.830 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 5 1 2024-09-14T06:30:02.833 INFO:teuthology.orchestra.run.smithi094.stdout:1884 write 0xceb370a thru 0xcebbc43 (0x853a bytes) 2024-09-14T06:30:02.835 INFO:teuthology.orchestra.run.smithi094.stdout:1886 write 0x9dccfe2 thru 0x9ddb14e (0xe16d bytes) 2024-09-14T06:30:02.837 INFO:teuthology.orchestra.run.smithi094.stdout:1887 writesame 0x93c3c67 thru 0x93d2813 data_size 0x33(0xebad bytes) 2024-09-14T06:30:02.840 INFO:teuthology.orchestra.run.smithi094.stdout:1889 trunc from 0xcebbc44 to 0xe313570 2024-09-14T06:30:02.846 INFO:teuthology.orchestra.run.smithi094.stdout:1890 read 0x224aad5 thru 0x2251bc1 (0x70ed bytes) 2024-09-14T06:30:02.847 INFO:teuthology.orchestra.run.smithi094.stdout:1891 writesame 0x91b1a51 thru 0x91b3e11 data_size 0x1b(0x23c1 bytes) 2024-09-14T06:30:02.848 INFO:teuthology.orchestra.run.smithi094.stdout:1892 punch from 0x9686642 to 0x9690467, (0x9e25 bytes) 2024-09-14T06:30:02.848 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-14T06:30:02.849 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-14T06:30:02.849 INFO:teuthology.orchestra.run.smithi094.stdout:1895 write 0x5fdc34c thru 0x5fe3bb3 (0x7868 bytes) 2024-09-14T06:30:02.851 INFO:teuthology.orchestra.run.smithi094.stdout:1897 read 0x583b0b7 thru 0x5848eca (0xde14 bytes) 2024-09-14T06:30:02.851 INFO:teuthology.orchestra.run.smithi094.stdout:1899 writesame 0x361b45b thru 0x362968b data_size 0x25(0xe231 bytes) 2024-09-14T06:30:02.853 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-14T06:30:02.853 INFO:teuthology.orchestra.run.smithi094.stdout:1905 punch from 0xb861252 to 0xb86611d, (0x4ecb bytes) 2024-09-14T06:30:02.854 INFO:teuthology.orchestra.run.smithi094.stdout:1906 read 0xaa04e7b thru 0xaa0a1b2 (0x5338 bytes) 2024-09-14T06:30:02.854 INFO:teuthology.orchestra.run.smithi094.stdout:1907 trunc from 0xe313570 to 0xdde8caa 2024-09-14T06:30:02.862 INFO:teuthology.orchestra.run.smithi094.stdout:1908 write 0xa7fdee1 thru 0xa80ada1 (0xcec1 bytes) 2024-09-14T06:30:02.868 INFO:teuthology.orchestra.run.smithi094.stdout:1909 trunc from 0xdde8caa to 0xa66b52b 2024-09-14T06:30:02.878 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-14T06:30:02.878 INFO:teuthology.orchestra.run.smithi094.stdout:1913 write 0x904cd91 thru 0x9051bcf (0x4e3f bytes) 2024-09-14T06:30:02.880 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-14T06:30:02.880 INFO:teuthology.orchestra.run.smithi094.stdout:1915 write 0x49b8aeb thru 0x49bbf9b (0x34b1 bytes) 2024-09-14T06:30:02.882 INFO:teuthology.orchestra.run.smithi094.stdout:1916 write 0x6afac55 thru 0x6b01a87 (0x6e33 bytes) 2024-09-14T06:30:02.889 INFO:teuthology.orchestra.run.smithi094.stdout:1917 trunc from 0xa66b52b to 0x56a08d2 2024-09-14T06:30:02.897 INFO:teuthology.orchestra.run.smithi094.stdout:1920 punch from 0x1a31eac to 0x1a34868, (0x29bc bytes) 2024-09-14T06:30:02.905 INFO:teuthology.orchestra.run.smithi094.stdout:1921 write 0x7934cc7 thru 0x79376bb (0x29f5 bytes) 2024-09-14T06:30:02.907 INFO:teuthology.orchestra.run.smithi094.stdout:1922 read 0x2fc1875 thru 0x2fc986d (0x7ff9 bytes) 2024-09-14T06:30:02.907 INFO:teuthology.orchestra.run.smithi094.stdout:1923 read 0x30b15bf thru 0x30b47a8 (0x31ea bytes) 2024-09-14T06:30:02.917 INFO:tasks.ceph.osd.1.smithi094.stderr:2024-09-14T06:30:02.916+0000 7f328e47a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33641) UID: 0 2024-09-14T06:30:02.920 INFO:teuthology.orchestra.run.smithi094.stdout:1926 writesame 0xd544cf8 thru 0xd54aa34 data_size 0x1(0x5d3d bytes) 2024-09-14T06:30:02.927 INFO:teuthology.orchestra.run.smithi094.stdout:1927 trunc from 0xd54aa35 to 0x9db36b1 2024-09-14T06:30:02.936 INFO:teuthology.orchestra.run.smithi094.stdout:1928 read 0x4c9fb2c thru 0x4ca35c2 (0x3a97 bytes) 2024-09-14T06:30:02.937 INFO:teuthology.orchestra.run.smithi094.stdout:1929 punch from 0x3c57cd to 0x3c7578, (0x1dab bytes) 2024-09-14T06:30:02.937 INFO:teuthology.orchestra.run.smithi094.stdout:1931 writesame 0x2259ce5 thru 0x225e853 data_size 0x29(0x4b6f bytes) 2024-09-14T06:30:02.949 INFO:teuthology.orchestra.run.smithi094.stdout:1933 write 0xdae3ecb thru 0xdae8538 (0x466e bytes) 2024-09-14T06:30:02.950 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-14T06:30:02.950 INFO:teuthology.orchestra.run.smithi094.stdout: "ops": [], 2024-09-14T06:30:02.950 INFO:teuthology.orchestra.run.smithi094.stdout: "num_ops": 0 2024-09-14T06:30:02.950 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-14T06:30:02.951 INFO:teuthology.orchestra.run.smithi094.stdout:1934 punch from 0x38dabf9 to 0x38dc8a3, (0x1caa bytes) 2024-09-14T06:30:02.952 INFO:teuthology.orchestra.run.smithi094.stdout:1935 writesame 0x17e0092 thru 0x17e4480 data_size 0x33(0x43ef bytes) 2024-09-14T06:30:02.952 INFO:teuthology.orchestra.run.smithi094.stdout:1936 read 0xc0ff6 thru 0xc9626 (0x8631 bytes) 2024-09-14T06:30:02.953 INFO:teuthology.orchestra.run.smithi094.stdout:1937 punch from 0xd26fc0 to 0xd27c08, (0xc48 bytes) 2024-09-14T06:30:02.953 INFO:teuthology.orchestra.run.smithi094.stdout:1938 write 0x3b2413e thru 0x3b248f8 (0x7bb bytes) 2024-09-14T06:30:02.955 INFO:teuthology.orchestra.run.smithi094.stdout:1939 read 0x40a6114 thru 0x40aa995 (0x4882 bytes) 2024-09-14T06:30:02.955 INFO:teuthology.orchestra.run.smithi094.stdout:1940 read 0x7bb1830 thru 0x7bc136e (0xfb3f bytes) 2024-09-14T06:30:02.956 INFO:teuthology.orchestra.run.smithi094.stdout:1941 read 0x3207c75 thru 0x3211e82 (0xa20e bytes) 2024-09-14T06:30:02.957 INFO:teuthology.orchestra.run.smithi094.stdout:1942 read 0xad525b5 thru 0xad5475f (0x21ab bytes) 2024-09-14T06:30:02.957 INFO:teuthology.orchestra.run.smithi094.stdout:1943 writesame 0x8653f59 thru 0x865ff5b data_size 0x57(0xc003 bytes) 2024-09-14T06:30:02.958 INFO:teuthology.orchestra.run.smithi094.stdout:1944 write 0x8450ee8 thru 0x845869e (0x77b7 bytes) 2024-09-14T06:30:02.964 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-14T06:30:02.965 INFO:teuthology.orchestra.run.smithi094.stdout:1945 trunc from 0xdae8539 to 0xa0832bd 2024-09-14T06:30:02.973 INFO:teuthology.orchestra.run.smithi094.stdout:1947 read 0x8c4c14f thru 0x8c5065e (0x4510 bytes) 2024-09-14T06:30:02.974 INFO:teuthology.orchestra.run.smithi094.stdout:1948 read 0x8106d9b thru 0x81152b4 (0xe51a bytes) 2024-09-14T06:30:02.975 INFO:teuthology.orchestra.run.smithi094.stdout:1951 punch from 0x8db94ec to 0x8dbf9b4, (0x64c8 bytes) 2024-09-14T06:30:02.975 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-14T06:30:02.975 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-14T06:30:02.975 INFO:teuthology.orchestra.run.smithi094.stdout:1954 punch from 0x7e9899 to 0x7f3258, (0x99bf bytes) 2024-09-14T06:30:02.975 INFO:teuthology.orchestra.run.smithi094.stdout:1955 punch from 0x7c832dc to 0x7c8ed32, (0xba56 bytes) 2024-09-14T06:30:02.975 INFO:teuthology.orchestra.run.smithi094.stdout:1957 write 0x3aee40d thru 0x3af25d3 (0x41c7 bytes) 2024-09-14T06:30:02.976 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-14T06:30:02.976 INFO:teuthology.orchestra.run.smithi094.stdout:1959 writesame 0x289ddee thru 0x28a5dda data_size 0x1(0x7fed bytes) 2024-09-14T06:30:02.992 INFO:teuthology.orchestra.run.smithi094.stdout:1962 read 0x257588e thru 0x257630f (0xa82 bytes) 2024-09-14T06:30:02.992 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-14T06:30:02.992 INFO:teuthology.orchestra.run.smithi094.stdout:1965 write 0x5ee3163 thru 0x5ee5b39 (0x29d7 bytes) 2024-09-14T06:30:02.994 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-14T06:30:02.994 INFO:teuthology.orchestra.run.smithi094.stdout:1967 read 0x947cded thru 0x948ca6d (0xfc81 bytes) 2024-09-14T06:30:03.000 INFO:teuthology.orchestra.run.smithi094.stdout:1969 trunc from 0xa0832bd to 0xea9ebff 2024-09-14T06:30:03.005 INFO:teuthology.orchestra.run.smithi094.stdout:1970 write 0x4190cce thru 0x419cf7d (0xc2b0 bytes) 2024-09-14T06:30:03.007 INFO:teuthology.orchestra.run.smithi094.stdout:1972 write 0x156b8b4 thru 0x156fdbf (0x450c bytes) 2024-09-14T06:30:03.008 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-14T06:30:03.008 INFO:teuthology.orchestra.run.smithi094.stdout:1975 write 0xba38519 thru 0xba428d8 (0xa3c0 bytes) 2024-09-14T06:30:03.010 INFO:teuthology.orchestra.run.smithi094.stdout:1976 punch from 0xf68933 to 0xf719ee, (0x90bb bytes) 2024-09-14T06:30:03.010 INFO:teuthology.orchestra.run.smithi094.stdout:1977 write 0xbc26f59 thru 0xbc2d196 (0x623e bytes) 2024-09-14T06:30:03.011 INFO:teuthology.orchestra.run.smithi094.stdout:1978 read 0xe618007 thru 0xe61a97a (0x2974 bytes) 2024-09-14T06:30:03.012 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-14T06:30:03.012 INFO:teuthology.orchestra.run.smithi094.stdout:1982 write 0xb028140 thru 0xb02d7f7 (0x56b8 bytes) 2024-09-14T06:30:03.013 INFO:teuthology.orchestra.run.smithi094.stdout:1983 punch from 0x9db1c35 to 0x9dbdecd, (0xc298 bytes) 2024-09-14T06:30:03.013 INFO:teuthology.orchestra.run.smithi094.stdout:1986 punch from 0x2baa2aa to 0x2bb80d4, (0xde2a bytes) 2024-09-14T06:30:03.018 INFO:tasks.ceph.osd.1.smithi094.stderr:2024-09-14T06:30:03.017+0000 7f328e47a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33641) UID: 0 2024-09-14T06:30:03.030 INFO:teuthology.orchestra.run.smithi094.stdout:1988 trunc from 0xea9ebff to 0x6c8d00 2024-09-14T06:30:03.059 INFO:teuthology.orchestra.run.smithi094.stdout:1989 write 0x62a5ff0 thru 0x62b35ea (0xd5fb bytes) 2024-09-14T06:30:03.063 INFO:teuthology.orchestra.run.smithi094.stdout:1991 punch from 0x365d2fc to 0x365ef2b, (0x1c2f bytes) 2024-09-14T06:30:03.063 INFO:teuthology.orchestra.run.smithi094.stdout:1992 read 0x3144a29 thru 0x3146c0d (0x21e5 bytes) 2024-09-14T06:30:03.063 INFO:teuthology.orchestra.run.smithi094.stdout:1993 read 0x4d098ec thru 0x4d145e3 (0xacf8 bytes) 2024-09-14T06:30:03.064 INFO:teuthology.orchestra.run.smithi094.stdout:1994 read 0x4f7c302 thru 0x4f85160 (0x8e5f bytes) 2024-09-14T06:30:03.064 INFO:teuthology.orchestra.run.smithi094.stdout:1997 read 0x5bbaf51 thru 0x5bbf9ca (0x4a7a bytes) 2024-09-14T06:30:03.074 INFO:teuthology.orchestra.run.smithi094.stdout:1998 writesame 0xad1cb0c thru 0xad24c6f data_size 0x62(0x8164 bytes) 2024-09-14T06:30:03.074 INFO:teuthology.orchestra.run.smithi094.stdout:1999 writesame 0x48a3f32 thru 0x48aedea data_size 0x1(0xaeb9 bytes) 2024-09-14T06:30:03.085 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-14T06:30:03.085 INFO:teuthology.orchestra.run.smithi094.stdout: "ops": [], 2024-09-14T06:30:03.086 INFO:teuthology.orchestra.run.smithi094.stdout: "complaint_time": 30, 2024-09-14T06:30:03.086 INFO:teuthology.orchestra.run.smithi094.stdout: "num_blocked_ops": 0 2024-09-14T06:30:03.086 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-14T06:30:03.094 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-14T06:30:03.096 INFO:teuthology.orchestra.run.smithi094.stdout:2000 write 0x9fa09bb thru 0x9faa585 (0x9bcb bytes) 2024-09-14T06:30:03.098 INFO:teuthology.orchestra.run.smithi094.stdout:2001 writesame 0x31f1dcb thru 0x31fdcbb data_size 0x1(0xbef1 bytes) 2024-09-14T06:30:03.119 INFO:tasks.ceph.osd.7.smithi138.stderr:2024-09-14T06:30:03.118+0000 7f3c6a21d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33623) UID: 0 2024-09-14T06:30:03.124 INFO:teuthology.orchestra.run.smithi094.stdout:2002 trunc from 0xad24c70 to 0x9774ddc 2024-09-14T06:30:03.133 INFO:teuthology.orchestra.run.smithi094.stdout:2004 write 0x672d9ab thru 0x673c254 (0xe8aa bytes) 2024-09-14T06:30:03.139 INFO:teuthology.orchestra.run.smithi094.stdout:2005 trunc from 0x9774ddc to 0x677fbb5 2024-09-14T06:30:03.155 INFO:teuthology.orchestra.run.smithi094.stdout:2006 write 0xa440e89 thru 0xa450089 (0xf201 bytes) 2024-09-14T06:30:03.157 INFO:teuthology.orchestra.run.smithi094.stdout:2007 write 0x916e081 thru 0x91789df (0xa95f bytes) 2024-09-14T06:30:03.158 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-14T06:30:03.158 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-14T06:30:03.158 INFO:teuthology.orchestra.run.smithi094.stdout:2012 write 0x1f0fc8b thru 0x1f1ced5 (0xd24b bytes) 2024-09-14T06:30:03.166 INFO:teuthology.orchestra.run.smithi094.stdout:2013 trunc from 0xa45008a to 0xe651857 2024-09-14T06:30:03.171 INFO:teuthology.orchestra.run.smithi094.stdout:2014 write 0x76d2845 thru 0x76d91af (0x696b bytes) 2024-09-14T06:30:03.174 INFO:teuthology.orchestra.run.smithi094.stdout:2015 write 0x63eea73 thru 0x63f9d65 (0xb2f3 bytes) 2024-09-14T06:30:03.175 INFO:teuthology.orchestra.run.smithi094.stdout:2016 read 0xe19cd8f thru 0xe1a25a3 (0x5815 bytes) 2024-09-14T06:30:03.176 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-14T06:30:03.176 INFO:teuthology.orchestra.run.smithi094.stdout:2018 punch from 0x97b5316 to 0x97bb510, (0x61fa bytes) 2024-09-14T06:30:03.176 INFO:teuthology.orchestra.run.smithi094.stdout:2019 punch from 0xa2cf392 to 0xa2da231, (0xae9f bytes) 2024-09-14T06:30:03.176 INFO:teuthology.orchestra.run.smithi094.stdout:2020 punch from 0xc431c84 to 0xc4354fb, (0x3877 bytes) 2024-09-14T06:30:03.176 INFO:teuthology.orchestra.run.smithi094.stdout:2021 writesame 0x8c1e6ea thru 0x8c1ffa6 data_size 0x1(0x18bd bytes) 2024-09-14T06:30:03.179 INFO:teuthology.orchestra.run.smithi094.stdout:2022 read 0xcd80a1d thru 0xcd87666 (0x6c4a bytes) 2024-09-14T06:30:03.179 INFO:teuthology.orchestra.run.smithi094.stdout:2024 write 0x84fe3ca thru 0x8500bb5 (0x27ec bytes) 2024-09-14T06:30:03.187 INFO:teuthology.orchestra.run.smithi094.stdout:2026 trunc from 0xe651857 to 0x8c69b3c 2024-09-14T06:30:03.195 INFO:teuthology.orchestra.run.smithi094.stdout:2027 write 0x121a146 thru 0x122713f (0xcffa bytes) 2024-09-14T06:30:03.199 INFO:teuthology.orchestra.run.smithi094.stdout:2028 trunc from 0x8c69b3c to 0xa771651 2024-09-14T06:30:03.204 INFO:teuthology.orchestra.run.smithi094.stdout:2029 write 0x7727347 thru 0x772842f (0x10e9 bytes) 2024-09-14T06:30:03.209 INFO:teuthology.orchestra.run.smithi094.stdout:2030 trunc from 0xa771651 to 0x7159d79 2024-09-14T06:30:03.216 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-14T06:30:03.217 INFO:teuthology.orchestra.run.smithi094.stdout: "size": 20, 2024-09-14T06:30:03.217 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 600, 2024-09-14T06:30:03.217 INFO:teuthology.orchestra.run.smithi094.stdout: "ops": [ 2024-09-14T06:30:03.217 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:03.217 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4397.0:1848 3.4 e32/23 3:27620a21:::rbd_data.112dcdddac74.000000000000000b:head v 32'40, mlcod=30'38)", 2024-09-14T06:30:03.217 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:08.658649+0000", 2024-09-14T06:30:03.218 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 54.555640361999998, 2024-09-14T06:30:03.218 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.001862867, 2024-09-14T06:30:03.218 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:30:03.218 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:30:03.218 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:30:03.218 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:30:03.218 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:03.219 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:30:03.219 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.658649+0000", 2024-09-14T06:30:03.219 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:03.219 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:03.219 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:03.219 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:30:03.219 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.658649+0000", 2024-09-14T06:30:03.219 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:03.219 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:03.220 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:03.220 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:30:03.220 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.658651+0000", 2024-09-14T06:30:03.220 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.5650000000000001e-06 2024-09-14T06:30:03.220 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:03.220 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:03.220 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:30:03.220 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.658709+0000", 2024-09-14T06:30:03.221 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.8442000000000001e-05 2024-09-14T06:30:03.221 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:03.221 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:03.221 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:30:03.221 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.658730+0000", 2024-09-14T06:30:03.221 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.0509999999999998e-05 2024-09-14T06:30:03.221 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:03.221 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:03.221 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:30:03.221 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.658740+0000", 2024-09-14T06:30:03.221 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.8619999999999998e-06 2024-09-14T06:30:03.221 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:03.222 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:03.222 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:30:03.222 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.658813+0000", 2024-09-14T06:30:03.222 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.3578000000000006e-05 2024-09-14T06:30:03.222 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:03.222 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:03.222 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:30:03.222 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.658836+0000", 2024-09-14T06:30:03.222 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.2677e-05 2024-09-14T06:30:03.223 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:03.223 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:03.223 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-14T06:30:03.223 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.660462+0000", 2024-09-14T06:30:03.223 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00162637 2024-09-14T06:30:03.223 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:03.223 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:03.224 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:30:03.224 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.660512+0000", 2024-09-14T06:30:03.224 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.9863e-05 2024-09-14T06:30:03.224 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:03.224 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:03.224 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:03.224 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:03.224 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:03.224 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4397.0:2199 3.0 e32/23 3:05f3e980:::rbd_data.112dcdddac74.000000000000000f:head v 32'3, mlcod=31'1)", 2024-09-14T06:30:03.224 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:08.680236+0000", 2024-09-14T06:30:03.224 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 54.534053997000001, 2024-09-14T06:30:03.224 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0019068990000000001, 2024-09-14T06:30:03.224 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:30:03.225 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:30:03.225 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:30:03.225 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:30:03.225 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:03.225 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:30:03.225 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.680236+0000", 2024-09-14T06:30:03.225 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:03.225 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:03.225 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:03.225 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:30:03.225 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.680236+0000", 2024-09-14T06:30:03.225 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:03.225 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:03.225 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:03.226 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:30:03.226 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.680237+0000", 2024-09-14T06:30:03.226 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.4980000000000001e-06 2024-09-14T06:30:03.226 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:03.226 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:03.226 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:30:03.226 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.680324+0000", 2024-09-14T06:30:03.226 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.7310000000000003e-05 2024-09-14T06:30:03.226 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:03.226 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:03.226 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:30:03.226 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.680354+0000", 2024-09-14T06:30:03.226 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.97e-05 2024-09-14T06:30:03.227 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:03.227 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:03.227 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:30:03.227 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.680366+0000", 2024-09-14T06:30:03.227 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.2048999999999999e-05 2024-09-14T06:30:03.227 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:03.227 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:03.227 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:30:03.227 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.680439+0000", 2024-09-14T06:30:03.227 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.2910000000000005e-05 2024-09-14T06:30:03.227 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:03.227 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:03.227 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:30:03.227 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.680465+0000", 2024-09-14T06:30:03.228 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.5812999999999998e-05 2024-09-14T06:30:03.228 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:03.228 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:03.228 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-14T06:30:03.228 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.682094+0000", 2024-09-14T06:30:03.228 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.001628876 2024-09-14T06:30:03.228 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:03.228 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:03.229 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:30:03.229 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.682142+0000", 2024-09-14T06:30:03.229 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.8742999999999997e-05 2024-09-14T06:30:03.229 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:03.229 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:03.229 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:03.229 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:03.229 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:03.229 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4397.0:3292 3.9 e32/23 3:91dbce57:::rbd_data.112dcdddac74.0000000000000036:head v 32'10, mlcod=32'8)", 2024-09-14T06:30:03.229 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:08.740506+0000", 2024-09-14T06:30:03.229 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 54.473783285000003, 2024-09-14T06:30:03.229 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.001864166, 2024-09-14T06:30:03.230 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:30:03.230 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:30:03.230 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:30:03.230 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:30:03.230 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:03.230 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:30:03.230 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.740506+0000", 2024-09-14T06:30:03.230 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:03.230 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:03.230 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:03.230 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:30:03.230 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.740506+0000", 2024-09-14T06:30:03.230 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:03.231 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:03.231 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:03.231 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:30:03.231 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.740514+0000", 2024-09-14T06:30:03.231 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.2109999999999998e-06 2024-09-14T06:30:03.231 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:03.231 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:03.231 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:30:03.231 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.740745+0000", 2024-09-14T06:30:03.231 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000230194 2024-09-14T06:30:03.231 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:03.231 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:03.231 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:30:03.231 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.740764+0000", 2024-09-14T06:30:03.232 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.9330999999999998e-05 2024-09-14T06:30:03.232 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:03.232 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:03.232 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:30:03.232 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.740774+0000", 2024-09-14T06:30:03.232 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.6260000000000007e-06 2024-09-14T06:30:03.232 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:03.232 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:03.232 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:30:03.232 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.740839+0000", 2024-09-14T06:30:03.232 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.5314000000000005e-05 2024-09-14T06:30:03.232 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:03.232 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:03.233 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:30:03.233 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.740860+0000", 2024-09-14T06:30:03.233 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.0868000000000001e-05 2024-09-14T06:30:03.233 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:03.233 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:03.233 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-14T06:30:03.233 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.742327+0000", 2024-09-14T06:30:03.233 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0014672909999999999 2024-09-14T06:30:03.233 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:03.234 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:03.234 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:30:03.234 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.742370+0000", 2024-09-14T06:30:03.234 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.3331000000000002e-05 2024-09-14T06:30:03.234 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:03.234 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:03.234 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:03.234 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:03.234 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:03.234 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4397.0:26415 3.0 e53/23 3:05fd9be0:::rbd_data.112d38768680.0000000000000006:head v 53'21, mlcod=50'19)", 2024-09-14T06:30:03.234 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:30.721610+0000", 2024-09-14T06:30:03.235 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 32.492679215999999, 2024-09-14T06:30:03.235 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0021763160000000002, 2024-09-14T06:30:03.235 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:30:03.235 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:30:03.235 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:30:03.235 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:30:03.235 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:03.235 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:30:03.235 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:30.721610+0000", 2024-09-14T06:30:03.235 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:03.235 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:03.235 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:03.236 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:30:03.236 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:30.721610+0000", 2024-09-14T06:30:03.236 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:03.236 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:03.236 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:03.236 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:30:03.236 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:30.721611+0000", 2024-09-14T06:30:03.236 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.41e-07 2024-09-14T06:30:03.236 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:03.236 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:03.236 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:30:03.236 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:30.721905+0000", 2024-09-14T06:30:03.236 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00029394300000000001 2024-09-14T06:30:03.237 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:03.237 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:03.237 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:30:03.237 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:30.721921+0000", 2024-09-14T06:30:03.237 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.6609999999999999e-05 2024-09-14T06:30:03.237 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:03.237 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:03.237 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:30:03.237 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:30.721931+0000", 2024-09-14T06:30:03.237 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.6739999999999992e-06 2024-09-14T06:30:03.237 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:03.237 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:03.237 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:30:03.238 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:30.722001+0000", 2024-09-14T06:30:03.238 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.0549000000000005e-05 2024-09-14T06:30:03.238 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:03.238 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:03.238 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:30:03.238 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:30.722029+0000", 2024-09-14T06:30:03.238 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.7327000000000001e-05 2024-09-14T06:30:03.238 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:03.238 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:03.238 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-14T06:30:03.238 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:30.723727+0000", 2024-09-14T06:30:03.238 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0016977310000000001 2024-09-14T06:30:03.238 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:03.238 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:03.239 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:30:03.239 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:30.723787+0000", 2024-09-14T06:30:03.239 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.0140999999999999e-05 2024-09-14T06:30:03.239 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:03.239 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:03.239 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:03.239 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:03.239 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:03.239 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4397.0:36084 3.0 e56/23 3:05a48ef4:::rbd_data.112dc58639fa.0000000000000058:head v 56'29, mlcod=56'27)", 2024-09-14T06:30:03.239 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:34.508855+0000", 2024-09-14T06:30:03.239 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 28.705434461999999, 2024-09-14T06:30:03.239 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0020801140000000001, 2024-09-14T06:30:03.239 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:30:03.239 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:30:03.239 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:30:03.240 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:30:03.240 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:03.240 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:30:03.240 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:34.508855+0000", 2024-09-14T06:30:03.240 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:03.240 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:03.240 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:03.240 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:30:03.240 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:34.508855+0000", 2024-09-14T06:30:03.240 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:03.240 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:03.240 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:03.240 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:30:03.241 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:34.508856+0000", 2024-09-14T06:30:03.241 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.1909999999999999e-06 2024-09-14T06:30:03.241 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:03.241 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:03.241 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:30:03.241 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:34.509203+0000", 2024-09-14T06:30:03.241 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00034625900000000002 2024-09-14T06:30:03.241 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:03.241 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:03.241 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:30:03.241 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:34.509218+0000", 2024-09-14T06:30:03.242 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.5091e-05 2024-09-14T06:30:03.242 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:03.242 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:03.242 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:30:03.242 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:34.509227+0000", 2024-09-14T06:30:03.242 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.0850000000000003e-06 2024-09-14T06:30:03.242 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:03.242 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:03.242 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:30:03.242 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:34.509311+0000", 2024-09-14T06:30:03.243 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.4436000000000003e-05 2024-09-14T06:30:03.243 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:03.243 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:03.243 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:30:03.243 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:34.509335+0000", 2024-09-14T06:30:03.243 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.3577000000000001e-05 2024-09-14T06:30:03.243 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:03.243 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:03.243 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-14T06:30:03.243 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:34.510890+0000", 2024-09-14T06:30:03.243 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0015554259999999999 2024-09-14T06:30:03.243 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:03.243 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:03.243 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:30:03.244 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:34.510935+0000", 2024-09-14T06:30:03.244 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.5049000000000002e-05 2024-09-14T06:30:03.244 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:03.244 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:03.244 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:03.245 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:03.245 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:03.245 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4397.0:36966 3.4 e56/23 3:22008318:::rbd_data.112dc58639fa.0000000000000066:head v 56'56, mlcod=50'54)", 2024-09-14T06:30:03.245 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:34.565614+0000", 2024-09-14T06:30:03.245 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 28.648676026, 2024-09-14T06:30:03.245 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0018691929999999999, 2024-09-14T06:30:03.245 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:30:03.245 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:30:03.245 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:30:03.245 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:30:03.246 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:03.246 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:30:03.246 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:34.565614+0000", 2024-09-14T06:30:03.246 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:03.246 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:03.246 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:03.246 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:30:03.246 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:34.565614+0000", 2024-09-14T06:30:03.246 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:03.246 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:03.246 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:03.246 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:30:03.246 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:34.565615+0000", 2024-09-14T06:30:03.246 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.784e-06 2024-09-14T06:30:03.247 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:03.247 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:03.247 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:30:03.247 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:34.565919+0000", 2024-09-14T06:30:03.247 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00030351599999999999 2024-09-14T06:30:03.247 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:03.247 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:03.247 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:30:03.247 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:34.565933+0000", 2024-09-14T06:30:03.247 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.4465e-05 2024-09-14T06:30:03.247 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:03.247 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:03.248 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:30:03.248 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:34.565943+0000", 2024-09-14T06:30:03.248 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.7920000000000004e-06 2024-09-14T06:30:03.248 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:03.248 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:03.248 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:30:03.248 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:34.565999+0000", 2024-09-14T06:30:03.248 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.5643000000000002e-05 2024-09-14T06:30:03.248 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:03.249 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:03.249 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:30:03.249 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:34.566025+0000", 2024-09-14T06:30:03.249 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.6040000000000001e-05 2024-09-14T06:30:03.249 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:03.249 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:03.249 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-14T06:30:03.249 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:34.567430+0000", 2024-09-14T06:30:03.249 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.001405653 2024-09-14T06:30:03.249 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:03.249 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:03.249 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:30:03.249 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:34.567483+0000", 2024-09-14T06:30:03.249 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.2299999999999997e-05 2024-09-14T06:30:03.250 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:03.250 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:03.250 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:03.250 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:03.250 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:03.250 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4397.0:42285 3.9 e62/23 3:90747014:::rbd_data.112dca2f6ae1.000000000000002f:head v 62'54, mlcod=62'52)", 2024-09-14T06:30:03.250 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:40.755702+0000", 2024-09-14T06:30:03.250 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 22.458587642000001, 2024-09-14T06:30:03.250 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.001687596, 2024-09-14T06:30:03.250 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:30:03.250 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:30:03.250 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:30:03.251 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:30:03.251 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:03.251 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:30:03.251 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:40.755702+0000", 2024-09-14T06:30:03.251 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:03.251 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:03.252 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:03.252 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:30:03.252 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:40.755702+0000", 2024-09-14T06:30:03.252 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:03.252 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:03.252 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:03.252 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:30:03.252 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:40.755707+0000", 2024-09-14T06:30:03.252 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.4700000000000001e-06 2024-09-14T06:30:03.252 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:03.252 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:03.252 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:30:03.253 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:40.756366+0000", 2024-09-14T06:30:03.253 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00065852900000000004 2024-09-14T06:30:03.253 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:03.253 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:03.253 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:30:03.253 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:40.756383+0000", 2024-09-14T06:30:03.253 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.7275e-05 2024-09-14T06:30:03.253 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:03.253 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:03.253 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:30:03.253 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:40.756393+0000", 2024-09-14T06:30:03.253 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.4749999999999999e-06 2024-09-14T06:30:03.253 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:03.254 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:03.254 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:30:03.254 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:40.756462+0000", 2024-09-14T06:30:03.254 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.9214000000000005e-05 2024-09-14T06:30:03.254 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:03.254 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:03.254 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:30:03.254 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:40.756490+0000", 2024-09-14T06:30:03.254 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.7936000000000002e-05 2024-09-14T06:30:03.254 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:03.254 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:03.255 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-14T06:30:03.255 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:40.757343+0000", 2024-09-14T06:30:03.255 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000853611 2024-09-14T06:30:03.255 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:03.255 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:03.255 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:30:03.255 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:40.757390+0000", 2024-09-14T06:30:03.255 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.6085999999999999e-05 2024-09-14T06:30:03.255 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:03.255 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:03.255 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:03.256 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:03.256 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:03.256 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4397.0:45722 3.9 e64/23 3:91fed098:::rbd_data.112d5a737e72.0000000000000001:head v 64'61, mlcod=62'59)", 2024-09-14T06:30:03.256 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:43.864269+0000", 2024-09-14T06:30:03.256 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 19.350020387000001, 2024-09-14T06:30:03.256 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.003055224, 2024-09-14T06:30:03.256 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:30:03.256 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:30:03.256 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:30:03.256 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:30:03.256 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:03.256 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:30:03.256 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.864269+0000", 2024-09-14T06:30:03.257 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:03.257 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:03.257 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:03.257 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:30:03.257 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.864269+0000", 2024-09-14T06:30:03.257 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:03.257 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:03.257 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:03.257 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:30:03.257 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.864269+0000", 2024-09-14T06:30:03.257 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.41e-07 2024-09-14T06:30:03.257 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:03.257 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:03.258 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:30:03.258 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.865490+0000", 2024-09-14T06:30:03.258 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.001220552 2024-09-14T06:30:03.258 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:03.258 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:03.258 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:30:03.258 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.865506+0000", 2024-09-14T06:30:03.258 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.6263999999999999e-05 2024-09-14T06:30:03.258 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:03.258 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:03.259 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:30:03.259 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.865516+0000", 2024-09-14T06:30:03.259 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.0091999999999999e-05 2024-09-14T06:30:03.259 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:03.259 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:03.259 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:30:03.259 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.865583+0000", 2024-09-14T06:30:03.259 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.6676999999999999e-05 2024-09-14T06:30:03.259 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:03.259 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:03.259 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:30:03.259 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.865610+0000", 2024-09-14T06:30:03.259 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.6529999999999998e-05 2024-09-14T06:30:03.260 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:03.260 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:03.260 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-14T06:30:03.260 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.867281+0000", 2024-09-14T06:30:03.260 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.001671203 2024-09-14T06:30:03.260 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:03.260 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:03.260 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:30:03.260 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.867324+0000", 2024-09-14T06:30:03.260 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.3665000000000003e-05 2024-09-14T06:30:03.260 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:03.260 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:03.260 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:03.261 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:03.261 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:03.261 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4397.0:49627 3.4 e64/23 3:21e30d4b:::rbd_data.112d5a737e72.0000000000000017:head v 64'69, mlcod=61'67)", 2024-09-14T06:30:03.261 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:44.206446+0000", 2024-09-14T06:30:03.261 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 19.007843748999999, 2024-09-14T06:30:03.261 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0018142220000000001, 2024-09-14T06:30:03.262 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:30:03.262 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:30:03.262 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:30:03.262 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:30:03.262 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:03.262 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:30:03.263 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.206446+0000", 2024-09-14T06:30:03.263 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:03.263 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:03.263 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:03.263 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:30:03.263 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.206446+0000", 2024-09-14T06:30:03.263 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:03.263 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:03.263 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:03.263 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:30:03.263 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.206446+0000", 2024-09-14T06:30:03.263 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.4900000000000001e-07 2024-09-14T06:30:03.263 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:03.264 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:03.264 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:30:03.264 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.206797+0000", 2024-09-14T06:30:03.264 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00035112799999999998 2024-09-14T06:30:03.264 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:03.264 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:03.264 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:30:03.264 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.206816+0000", 2024-09-14T06:30:03.264 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.8977000000000001e-05 2024-09-14T06:30:03.264 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:03.264 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:03.264 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:30:03.265 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.206825+0000", 2024-09-14T06:30:03.265 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.8279999999999992e-06 2024-09-14T06:30:03.265 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:03.265 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:03.265 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:30:03.266 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.206892+0000", 2024-09-14T06:30:03.266 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.6809000000000004e-05 2024-09-14T06:30:03.266 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:03.266 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:03.266 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:30:03.266 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.206915+0000", 2024-09-14T06:30:03.266 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.315e-05 2024-09-14T06:30:03.266 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:03.266 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:03.266 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-14T06:30:03.266 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.208201+0000", 2024-09-14T06:30:03.266 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0012858819999999999 2024-09-14T06:30:03.267 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:03.267 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:03.267 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:30:03.267 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.208260+0000", 2024-09-14T06:30:03.267 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.9098999999999998e-05 2024-09-14T06:30:03.267 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:03.267 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:03.267 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:03.267 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:03.267 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:03.267 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4397.0:50003 3.9 e64/23 3:94f26c4b:::rbd_data.112d5a737e72.000000000000001f:head v 64'62, mlcod=62'60)", 2024-09-14T06:30:03.267 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:44.220413+0000", 2024-09-14T06:30:03.268 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 18.993876070999999, 2024-09-14T06:30:03.268 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0019210430000000001, 2024-09-14T06:30:03.268 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:30:03.268 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:30:03.268 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:30:03.268 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:30:03.268 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:03.268 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:30:03.268 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.220413+0000", 2024-09-14T06:30:03.268 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:03.268 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:03.269 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:03.269 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:30:03.269 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.220413+0000", 2024-09-14T06:30:03.269 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:03.269 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:03.269 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:03.269 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:30:03.269 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.220415+0000", 2024-09-14T06:30:03.269 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.575e-06 2024-09-14T06:30:03.269 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:03.269 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:03.269 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:30:03.269 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.220984+0000", 2024-09-14T06:30:03.270 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000568624 2024-09-14T06:30:03.270 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:03.270 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:03.270 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:30:03.270 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.221007+0000", 2024-09-14T06:30:03.270 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.3112000000000002e-05 2024-09-14T06:30:03.270 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:03.270 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:03.270 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:30:03.270 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.221017+0000", 2024-09-14T06:30:03.270 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.0098999999999999e-05 2024-09-14T06:30:03.270 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:03.271 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:03.271 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:30:03.271 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.221110+0000", 2024-09-14T06:30:03.271 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.3268000000000002e-05 2024-09-14T06:30:03.271 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:03.271 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:03.271 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:30:03.272 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.221131+0000", 2024-09-14T06:30:03.272 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.0905999999999999e-05 2024-09-14T06:30:03.272 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:03.272 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:03.272 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-14T06:30:03.272 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.222290+0000", 2024-09-14T06:30:03.272 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.001159408 2024-09-14T06:30:03.272 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:03.272 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:03.272 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:30:03.272 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.222335+0000", 2024-09-14T06:30:03.272 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.4051000000000002e-05 2024-09-14T06:30:03.272 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:03.273 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:03.273 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:03.273 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:03.273 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:03.273 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4397.0:50006 3.4 e64/23 3:20cfe300:::rbd_data.112d5a737e72.0000000000000019:head v 64'70, mlcod=61'68)", 2024-09-14T06:30:03.273 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:44.224218+0000", 2024-09-14T06:30:03.273 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 18.990071189999998, 2024-09-14T06:30:03.273 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0021104320000000002, 2024-09-14T06:30:03.273 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:30:03.273 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:30:03.273 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:30:03.273 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:30:03.273 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:03.273 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:30:03.274 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.224218+0000", 2024-09-14T06:30:03.274 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:03.274 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:03.274 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:03.274 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:30:03.274 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.224218+0000", 2024-09-14T06:30:03.274 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:03.274 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:03.274 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:03.274 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:30:03.275 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.224219+0000", 2024-09-14T06:30:03.275 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.0049999999999999e-06 2024-09-14T06:30:03.275 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:03.275 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:03.275 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:30:03.275 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.224326+0000", 2024-09-14T06:30:03.275 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000106848 2024-09-14T06:30:03.275 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:03.275 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:03.275 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:30:03.275 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.224345+0000", 2024-09-14T06:30:03.275 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.8587999999999999e-05 2024-09-14T06:30:03.275 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:03.276 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:03.276 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:30:03.276 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.224354+0000", 2024-09-14T06:30:03.276 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.7220000000000005e-06 2024-09-14T06:30:03.276 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:03.276 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:03.276 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:30:03.276 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.224416+0000", 2024-09-14T06:30:03.276 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.2738999999999997e-05 2024-09-14T06:30:03.277 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:03.277 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:03.277 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:30:03.277 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.224441+0000", 2024-09-14T06:30:03.277 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.4408000000000001e-05 2024-09-14T06:30:03.277 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:03.277 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:03.277 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-14T06:30:03.277 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.226295+0000", 2024-09-14T06:30:03.277 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00185428 2024-09-14T06:30:03.277 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:03.277 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:03.278 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:30:03.278 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.226329+0000", 2024-09-14T06:30:03.278 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.3841999999999999e-05 2024-09-14T06:30:03.278 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:03.278 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:03.278 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:03.278 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:03.278 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:03.278 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4397.0:50414 3.0 e64/23 3:03ae4e67:::rbd_data.112d5a737e72.000000000000001a:head v 64'67, mlcod=62'65)", 2024-09-14T06:30:03.278 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:44.240908+0000", 2024-09-14T06:30:03.278 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 18.973381709999998, 2024-09-14T06:30:03.278 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.001493241, 2024-09-14T06:30:03.278 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:30:03.279 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:30:03.279 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:30:03.279 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:30:03.279 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:03.279 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:30:03.279 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.240908+0000", 2024-09-14T06:30:03.279 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:03.279 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:03.279 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:03.279 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:30:03.279 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.240908+0000", 2024-09-14T06:30:03.279 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:03.280 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:03.280 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:03.280 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:30:03.280 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.240910+0000", 2024-09-14T06:30:03.280 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.7179999999999999e-06 2024-09-14T06:30:03.280 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:03.280 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:03.280 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:30:03.280 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.241049+0000", 2024-09-14T06:30:03.280 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00013932199999999999 2024-09-14T06:30:03.280 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:03.280 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:03.280 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:30:03.280 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.241069+0000", 2024-09-14T06:30:03.281 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.9836000000000001e-05 2024-09-14T06:30:03.281 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:03.281 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:03.281 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:30:03.281 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.241080+0000", 2024-09-14T06:30:03.281 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.0920999999999999e-05 2024-09-14T06:30:03.281 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:03.281 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:03.281 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:30:03.281 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.241150+0000", 2024-09-14T06:30:03.281 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.0568999999999995e-05 2024-09-14T06:30:03.281 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:03.281 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:03.281 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:30:03.282 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.241178+0000", 2024-09-14T06:30:03.282 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.7429000000000001e-05 2024-09-14T06:30:03.282 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:03.282 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:03.282 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-14T06:30:03.282 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.242353+0000", 2024-09-14T06:30:03.282 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0011750770000000001 2024-09-14T06:30:03.282 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:03.282 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:03.282 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:30:03.282 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.242401+0000", 2024-09-14T06:30:03.282 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.8368999999999997e-05 2024-09-14T06:30:03.282 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:03.282 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:03.283 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:03.283 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:03.283 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:03.283 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4397.0:51816 3.4 e64/23 3:21a826f9:::rbd_data.112d5a737e72.000000000000002d:head v 64'72, mlcod=64'70)", 2024-09-14T06:30:03.283 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:44.312072+0000", 2024-09-14T06:30:03.283 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 18.902217716999999, 2024-09-14T06:30:03.283 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0019555240000000002, 2024-09-14T06:30:03.283 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:30:03.283 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:30:03.283 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:30:03.283 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:30:03.283 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:03.283 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:30:03.283 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.312072+0000", 2024-09-14T06:30:03.284 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:03.284 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:03.284 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:03.284 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:30:03.284 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.312072+0000", 2024-09-14T06:30:03.285 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:03.285 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:03.285 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:03.285 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:30:03.285 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.312074+0000", 2024-09-14T06:30:03.285 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.9530000000000002e-06 2024-09-14T06:30:03.285 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:03.285 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:03.286 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:30:03.286 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.312378+0000", 2024-09-14T06:30:03.286 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00030392 2024-09-14T06:30:03.286 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:03.286 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:03.286 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:30:03.286 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.312399+0000", 2024-09-14T06:30:03.286 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.1134000000000001e-05 2024-09-14T06:30:03.287 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:03.287 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:03.287 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:30:03.287 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.312408+0000", 2024-09-14T06:30:03.287 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.2879999999999998e-06 2024-09-14T06:30:03.287 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:03.287 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:03.287 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:30:03.288 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.312494+0000", 2024-09-14T06:30:03.288 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.6280999999999996e-05 2024-09-14T06:30:03.288 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:03.288 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:03.288 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:30:03.288 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.312518+0000", 2024-09-14T06:30:03.288 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.3149000000000001e-05 2024-09-14T06:30:03.288 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:03.289 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:03.289 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-14T06:30:03.289 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.313964+0000", 2024-09-14T06:30:03.289 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.001446754 2024-09-14T06:30:03.289 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:03.289 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:03.289 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:30:03.290 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.314027+0000", 2024-09-14T06:30:03.290 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.3045000000000004e-05 2024-09-14T06:30:03.290 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:03.290 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:03.290 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:03.290 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:03.290 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:03.290 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4397.0:74241 3.0 e73/23 3:06e93fb9:::rbd_data.112d80ca42af.0000000000000000:head v 73'102, mlcod=71'100)", 2024-09-14T06:30:03.291 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:53.876730+0000", 2024-09-14T06:30:03.291 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 9.3375592869999995, 2024-09-14T06:30:03.291 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0079470329999999992, 2024-09-14T06:30:03.291 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:30:03.291 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:30:03.291 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:30:03.291 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:30:03.291 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:03.291 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:30:03.292 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:53.876730+0000", 2024-09-14T06:30:03.292 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:03.292 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:03.292 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:03.292 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:30:03.292 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:53.876730+0000", 2024-09-14T06:30:03.292 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:03.293 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:03.293 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:03.293 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:30:03.293 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:53.876731+0000", 2024-09-14T06:30:03.293 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.0500000000000004e-07 2024-09-14T06:30:03.293 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:03.293 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:03.293 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:30:03.294 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:53.879616+0000", 2024-09-14T06:30:03.294 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0028856680000000001 2024-09-14T06:30:03.294 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:03.294 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:03.294 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:30:03.294 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:53.879643+0000", 2024-09-14T06:30:03.294 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.6120999999999999e-05 2024-09-14T06:30:03.294 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:03.294 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:03.295 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:30:03.295 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:53.879654+0000", 2024-09-14T06:30:03.295 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.1586999999999999e-05 2024-09-14T06:30:03.295 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:03.295 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:03.295 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:30:03.295 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:53.879756+0000", 2024-09-14T06:30:03.295 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00010223899999999999 2024-09-14T06:30:03.296 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:03.296 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:03.296 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:30:03.296 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:53.879783+0000", 2024-09-14T06:30:03.296 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.6913000000000001e-05 2024-09-14T06:30:03.296 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:03.296 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:03.296 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-14T06:30:03.297 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:53.884594+0000", 2024-09-14T06:30:03.297 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0048110729999999999 2024-09-14T06:30:03.297 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:03.297 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:03.297 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:30:03.297 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:53.884677+0000", 2024-09-14T06:30:03.297 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.2927000000000001e-05 2024-09-14T06:30:03.297 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:03.297 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:03.298 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:03.298 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:03.298 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:03.298 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4397.0:74243 3.9 e73/23 3:94977f81:::rbd_data.112d80ca42af.0000000000000002:head v 73'94, mlcod=71'92)", 2024-09-14T06:30:03.298 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:53.885156+0000", 2024-09-14T06:30:03.298 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 9.3291333690000009, 2024-09-14T06:30:03.298 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0096009860000000006, 2024-09-14T06:30:03.298 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:30:03.299 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:30:03.299 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:30:03.299 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:30:03.299 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:03.299 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:30:03.299 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:53.885156+0000", 2024-09-14T06:30:03.299 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:03.299 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:03.299 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:03.299 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:30:03.300 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:53.885156+0000", 2024-09-14T06:30:03.300 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:03.300 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:03.300 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:03.300 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:30:03.300 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:53.885156+0000", 2024-09-14T06:30:03.300 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.23e-07 2024-09-14T06:30:03.300 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:03.300 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:03.300 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:30:03.300 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:53.889694+0000", 2024-09-14T06:30:03.301 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0045378709999999997 2024-09-14T06:30:03.301 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:03.301 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:03.301 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:30:03.301 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:53.889720+0000", 2024-09-14T06:30:03.301 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.5259e-05 2024-09-14T06:30:03.301 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:03.301 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:03.301 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:30:03.301 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:53.889731+0000", 2024-09-14T06:30:03.301 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.1739000000000001e-05 2024-09-14T06:30:03.301 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:03.301 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:03.302 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:30:03.302 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:53.889822+0000", 2024-09-14T06:30:03.302 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.1200999999999996e-05 2024-09-14T06:30:03.302 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:03.302 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:03.302 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:30:03.302 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:53.889850+0000", 2024-09-14T06:30:03.302 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.7084999999999999e-05 2024-09-14T06:30:03.302 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:03.302 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:03.302 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-14T06:30:03.302 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:53.894703+0000", 2024-09-14T06:30:03.302 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0048532690000000003 2024-09-14T06:30:03.302 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:03.303 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:03.303 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:30:03.303 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:53.894757+0000", 2024-09-14T06:30:03.303 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.4339000000000003e-05 2024-09-14T06:30:03.303 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:03.303 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:03.303 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:03.303 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:03.303 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:03.303 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4397.0:74249 3.0 e73/23 3:009b0225:::rbd_data.112d80ca42af.0000000000000009:head v 73'103, mlcod=71'101)", 2024-09-14T06:30:03.303 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:53.893219+0000", 2024-09-14T06:30:03.303 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 9.3210708909999997, 2024-09-14T06:30:03.303 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0065381980000000003, 2024-09-14T06:30:03.303 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:30:03.303 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:30:03.303 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:30:03.304 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:30:03.304 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:03.304 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:30:03.304 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:53.893219+0000", 2024-09-14T06:30:03.304 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:03.304 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:03.304 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:03.304 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:30:03.304 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:53.893219+0000", 2024-09-14T06:30:03.304 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:03.304 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:03.304 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:03.304 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:30:03.304 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:53.893221+0000", 2024-09-14T06:30:03.304 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.4329999999999998e-06 2024-09-14T06:30:03.305 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:03.305 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:03.305 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:30:03.305 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:53.895968+0000", 2024-09-14T06:30:03.305 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.002746746 2024-09-14T06:30:03.305 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:03.305 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:03.305 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:30:03.305 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:53.895988+0000", 2024-09-14T06:30:03.305 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.0227000000000001e-05 2024-09-14T06:30:03.305 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:03.305 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:03.305 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:30:03.305 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:53.896000+0000", 2024-09-14T06:30:03.305 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.2359e-05 2024-09-14T06:30:03.306 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:03.306 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:03.306 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:30:03.306 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:53.896084+0000", 2024-09-14T06:30:03.306 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.3659999999999995e-05 2024-09-14T06:30:03.306 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:03.306 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:03.306 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:30:03.306 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:53.896109+0000", 2024-09-14T06:30:03.306 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.5259999999999999e-05 2024-09-14T06:30:03.306 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:03.306 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:03.307 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-14T06:30:03.307 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:53.899704+0000", 2024-09-14T06:30:03.307 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0035942209999999999 2024-09-14T06:30:03.307 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:03.307 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:03.307 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:30:03.307 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:53.899757+0000", 2024-09-14T06:30:03.307 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.3291999999999997e-05 2024-09-14T06:30:03.307 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:03.307 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:03.307 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:03.307 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:03.307 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:03.307 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4397.0:78263 3.0 e77/23 3:00899b22:::rbd_data.112ddfbec471.0000000000000016:head v 77'104, mlcod=73'102)", 2024-09-14T06:30:03.308 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:57.634578+0000", 2024-09-14T06:30:03.308 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 5.5797118389999998, 2024-09-14T06:30:03.308 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.002372454, 2024-09-14T06:30:03.308 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:30:03.308 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:30:03.308 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:30:03.308 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:30:03.308 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:03.308 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:30:03.308 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:57.634578+0000", 2024-09-14T06:30:03.308 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:03.308 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:03.308 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:03.309 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:30:03.309 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:57.634578+0000", 2024-09-14T06:30:03.309 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:03.309 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:03.309 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:03.309 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:30:03.309 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:57.634578+0000", 2024-09-14T06:30:03.309 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.5699999999999998e-07 2024-09-14T06:30:03.309 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:03.309 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:03.310 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:30:03.310 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:57.635347+0000", 2024-09-14T06:30:03.310 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00076848300000000003 2024-09-14T06:30:03.310 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:03.310 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:03.310 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:30:03.310 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:57.635361+0000", 2024-09-14T06:30:03.310 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.4238e-05 2024-09-14T06:30:03.310 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:03.310 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:03.310 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:30:03.310 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:57.635371+0000", 2024-09-14T06:30:03.310 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.0087e-05 2024-09-14T06:30:03.311 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:03.311 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:03.311 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:30:03.311 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:57.635439+0000", 2024-09-14T06:30:03.311 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.7953000000000005e-05 2024-09-14T06:30:03.311 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:03.311 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:03.311 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:30:03.311 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:57.635464+0000", 2024-09-14T06:30:03.312 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.5062000000000002e-05 2024-09-14T06:30:03.312 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:03.312 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:03.312 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-14T06:30:03.312 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:57.636914+0000", 2024-09-14T06:30:03.312 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.001449625 2024-09-14T06:30:03.312 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:03.312 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:03.312 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:30:03.312 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:57.636950+0000", 2024-09-14T06:30:03.312 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.6548999999999999e-05 2024-09-14T06:30:03.312 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:03.312 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:03.312 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:03.313 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:03.313 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:03.313 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4397.0:78604 3.9 e77/23 3:94c62a0e:::rbd_data.112ddfbec471.000000000000002c:head v 77'96, mlcod=73'94)", 2024-09-14T06:30:03.313 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:57.677568+0000", 2024-09-14T06:30:03.313 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 5.536721451, 2024-09-14T06:30:03.313 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.002564846, 2024-09-14T06:30:03.313 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:30:03.313 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:30:03.313 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:30:03.313 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:30:03.313 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:03.313 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:30:03.313 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:57.677568+0000", 2024-09-14T06:30:03.313 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:03.314 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:03.314 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:03.314 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:30:03.314 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:57.677568+0000", 2024-09-14T06:30:03.314 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:03.314 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:03.314 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:03.314 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:30:03.314 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:57.677568+0000", 2024-09-14T06:30:03.314 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.8900000000000001e-07 2024-09-14T06:30:03.314 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:03.314 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:03.314 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:30:03.315 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:57.678488+0000", 2024-09-14T06:30:03.315 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00091963899999999998 2024-09-14T06:30:03.315 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:03.315 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:03.315 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:30:03.315 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:57.678500+0000", 2024-09-14T06:30:03.315 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.1902000000000001e-05 2024-09-14T06:30:03.315 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:03.315 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:03.315 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:30:03.315 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:57.678507+0000", 2024-09-14T06:30:03.315 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.7739999999999999e-06 2024-09-14T06:30:03.316 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:03.316 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:03.316 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:30:03.316 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:57.678586+0000", 2024-09-14T06:30:03.316 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.9103000000000005e-05 2024-09-14T06:30:03.316 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:03.316 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:03.316 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:30:03.316 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:57.678614+0000", 2024-09-14T06:30:03.316 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.7912e-05 2024-09-14T06:30:03.316 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:03.316 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:03.316 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-14T06:30:03.316 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:57.680075+0000", 2024-09-14T06:30:03.317 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0014613370000000001 2024-09-14T06:30:03.317 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:03.317 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:03.317 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:30:03.317 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:57.680133+0000", 2024-09-14T06:30:03.317 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.7890000000000003e-05 2024-09-14T06:30:03.317 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:03.317 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:03.317 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:03.317 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:03.317 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:03.317 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4397.0:78640 3.9 e77/23 3:97fd73eb:::rbd_data.112ddfbec471.000000000000002f:head v 77'97, mlcod=73'95)", 2024-09-14T06:30:03.317 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:57.681153+0000", 2024-09-14T06:30:03.318 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 5.533136152, 2024-09-14T06:30:03.318 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0016647109999999999, 2024-09-14T06:30:03.318 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:30:03.318 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:30:03.318 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:30:03.318 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:30:03.318 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:03.318 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:30:03.318 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:57.681153+0000", 2024-09-14T06:30:03.318 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:03.318 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:03.318 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:03.318 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:30:03.319 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:57.681153+0000", 2024-09-14T06:30:03.319 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:03.319 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:03.319 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:03.319 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:30:03.319 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:57.681155+0000", 2024-09-14T06:30:03.319 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.127e-06 2024-09-14T06:30:03.319 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:03.319 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:03.319 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:30:03.319 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:57.681704+0000", 2024-09-14T06:30:03.319 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000549103 2024-09-14T06:30:03.319 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:03.319 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:03.319 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:30:03.320 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:57.681733+0000", 2024-09-14T06:30:03.320 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.9641e-05 2024-09-14T06:30:03.320 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:03.320 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:03.320 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:30:03.320 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:57.681742+0000", 2024-09-14T06:30:03.320 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.9660000000000002e-06 2024-09-14T06:30:03.320 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:03.321 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:03.321 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:30:03.321 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:57.681802+0000", 2024-09-14T06:30:03.321 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.9935000000000001e-05 2024-09-14T06:30:03.321 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:03.321 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:03.321 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:30:03.321 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:57.681825+0000", 2024-09-14T06:30:03.321 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.3017999999999999e-05 2024-09-14T06:30:03.322 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:03.322 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:03.322 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-14T06:30:03.322 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:57.682777+0000", 2024-09-14T06:30:03.322 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00095157399999999995 2024-09-14T06:30:03.322 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:03.322 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:03.322 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:30:03.322 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:57.682818+0000", 2024-09-14T06:30:03.322 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.1347000000000002e-05 2024-09-14T06:30:03.322 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:03.323 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:03.323 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:03.323 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:03.323 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:03.323 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4397.0:78641 3.9 e77/23 3:914b72e4:::rbd_data.112ddfbec471.0000000000000030:head v 77'98, mlcod=73'95)", 2024-09-14T06:30:03.323 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:57.682289+0000", 2024-09-14T06:30:03.323 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 5.5320007699999998, 2024-09-14T06:30:03.323 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0020171400000000002, 2024-09-14T06:30:03.323 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:30:03.323 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:30:03.324 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:30:03.324 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:30:03.324 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:03.324 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:30:03.324 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:57.682289+0000", 2024-09-14T06:30:03.324 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:03.324 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:03.324 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:03.324 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:30:03.324 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:57.682289+0000", 2024-09-14T06:30:03.324 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:03.324 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:03.324 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:03.325 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:30:03.325 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:57.682290+0000", 2024-09-14T06:30:03.325 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.0100000000000003e-07 2024-09-14T06:30:03.325 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:03.325 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:03.325 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:30:03.325 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:57.683230+0000", 2024-09-14T06:30:03.325 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00094047599999999999 2024-09-14T06:30:03.325 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:03.325 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:03.325 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:30:03.325 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:57.683243+0000", 2024-09-14T06:30:03.325 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.2926e-05 2024-09-14T06:30:03.326 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:03.326 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:03.326 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:30:03.326 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:57.683250+0000", 2024-09-14T06:30:03.326 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.6109999999999999e-06 2024-09-14T06:30:03.326 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:03.326 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:03.326 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:30:03.326 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:57.683313+0000", 2024-09-14T06:30:03.326 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.2943999999999999e-05 2024-09-14T06:30:03.326 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:03.326 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:03.326 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:30:03.326 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:57.683337+0000", 2024-09-14T06:30:03.327 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.4377999999999999e-05 2024-09-14T06:30:03.327 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:03.327 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:03.327 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-14T06:30:03.327 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:57.684264+0000", 2024-09-14T06:30:03.327 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00092733099999999999 2024-09-14T06:30:03.327 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:03.327 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:03.327 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:30:03.327 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:57.684306+0000", 2024-09-14T06:30:03.327 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.1573000000000003e-05 2024-09-14T06:30:03.327 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:03.327 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:03.327 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:03.327 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:03.328 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:03.328 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-14T06:30:03.328 INFO:teuthology.orchestra.run.smithi094.stdout:2031 write 0x28744b3 thru 0x2879346 (0x4e94 bytes) 2024-09-14T06:30:03.329 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-14T06:30:03.333 INFO:teuthology.orchestra.run.smithi094.stdout:2033 writesame 0xb6edc41 thru 0xb6f4e55 data_size 0x3b(0x7215 bytes) 2024-09-14T06:30:03.333 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-14T06:30:03.334 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-14T06:30:03.334 INFO:teuthology.orchestra.run.smithi094.stdout:2036 punch from 0x740d27 to 0x746ad5, (0x5dae bytes) 2024-09-14T06:30:03.334 INFO:teuthology.orchestra.run.smithi094.stdout:2037 write 0x3e301fc thru 0x3e351e7 (0x4fec bytes) 2024-09-14T06:30:03.334 INFO:teuthology.orchestra.run.smithi094.stdout:2038 read 0xad5c437 thru 0xad6a91a (0xe4e4 bytes) 2024-09-14T06:30:03.334 INFO:teuthology.orchestra.run.smithi094.stdout:2039 writesame 0x68aa4e0 thru 0x68b6bf1 data_size 0x2(0xc712 bytes) 2024-09-14T06:30:03.334 INFO:teuthology.orchestra.run.smithi094.stdout:2040 trunc from 0xb6f4e56 to 0x7a5ff7d 2024-09-14T06:30:03.334 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-14T06:30:03.334 INFO:teuthology.orchestra.run.smithi094.stdout:2042 write 0x9ba79e5 thru 0x9baf2f5 (0x7911 bytes) 2024-09-14T06:30:03.334 INFO:teuthology.orchestra.run.smithi094.stdout:2044 read 0xd283b8 thru 0xd28b37 (0x780 bytes) 2024-09-14T06:30:03.334 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-14T06:30:03.334 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-14T06:30:03.334 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-14T06:30:03.334 INFO:teuthology.orchestra.run.smithi094.stdout:2048 write 0x53d1457 thru 0x53da302 (0x8eac bytes) 2024-09-14T06:30:03.334 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-14T06:30:03.334 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-14T06:30:03.335 INFO:teuthology.orchestra.run.smithi094.stdout:2055 punch from 0x2531f1a to 0x253f6e8, (0xd7ce bytes) 2024-09-14T06:30:03.335 INFO:teuthology.orchestra.run.smithi094.stdout:2056 punch from 0x1db187c to 0x1db956a, (0x7cee bytes) 2024-09-14T06:30:03.335 INFO:teuthology.orchestra.run.smithi094.stdout:2057 trunc from 0x9baf2f6 to 0x1ef01c0 2024-09-14T06:30:03.335 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-14T06:30:03.335 INFO:teuthology.orchestra.run.smithi094.stdout:2060 write 0xd77a218 thru 0xd7838fd (0x96e6 bytes) 2024-09-14T06:30:03.335 INFO:teuthology.orchestra.run.smithi094.stdout:2061 write 0xde18b52 thru 0xde1a4f6 (0x19a5 bytes) 2024-09-14T06:30:03.335 INFO:teuthology.orchestra.run.smithi094.stdout:2064 write 0x2ab42e4 thru 0x2ac2eba (0xebd7 bytes) 2024-09-14T06:30:03.335 INFO:teuthology.orchestra.run.smithi094.stdout:2067 read 0x785a88 thru 0x78db27 (0x80a0 bytes) 2024-09-14T06:30:03.335 INFO:teuthology.orchestra.run.smithi094.stdout:2068 read 0x1f6247f thru 0x1f62c81 (0x803 bytes) 2024-09-14T06:30:03.335 INFO:teuthology.orchestra.run.smithi094.stdout:2069 punch from 0xd6f92d to 0xd769ba, (0x708d bytes) 2024-09-14T06:30:03.336 INFO:teuthology.orchestra.run.smithi094.stdout:2070 punch from 0x30981a7 to 0x309d915, (0x576e bytes) 2024-09-14T06:30:03.336 INFO:teuthology.orchestra.run.smithi094.stdout:2072 writesame 0x845aa5d thru 0x8465cfa data_size 0x3(0xb29e bytes) 2024-09-14T06:30:03.336 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-14T06:30:03.336 INFO:teuthology.orchestra.run.smithi094.stdout:2074 write 0x2458f64 thru 0x245f4fc (0x6599 bytes) 2024-09-14T06:30:03.336 INFO:teuthology.orchestra.run.smithi094.stdout:2075 punch from 0x7cf85a3 to 0x7cfbaf1, (0x354e bytes) 2024-09-14T06:30:03.336 INFO:teuthology.orchestra.run.smithi094.stdout:2076 read 0xa1634ee thru 0xa164b71 (0x1684 bytes) 2024-09-14T06:30:03.336 INFO:teuthology.orchestra.run.smithi094.stdout:2077 writesame 0x480b7fc thru 0x48138db data_size 0x10(0x80e0 bytes) 2024-09-14T06:30:03.336 INFO:teuthology.orchestra.run.smithi094.stdout:2078 read 0xf58c79 thru 0xf66358 (0xd6e0 bytes) 2024-09-14T06:30:03.336 INFO:teuthology.orchestra.run.smithi094.stdout:2079 read 0x4f3bb61 thru 0x4f4430a (0x87aa bytes) 2024-09-14T06:30:03.336 INFO:teuthology.orchestra.run.smithi094.stdout:2080 read 0xcdc1280 thru 0xcdc86ba (0x743b bytes) 2024-09-14T06:30:03.337 INFO:tasks.ceph.osd.3.smithi094.stderr:2024-09-14T06:30:03.331+0000 7feeeafe8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33655) UID: 0 2024-09-14T06:30:03.337 INFO:teuthology.orchestra.run.smithi094.stdout:2082 write 0xc1659db thru 0xc169fba (0x45e0 bytes) 2024-09-14T06:30:03.338 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-14T06:30:03.341 INFO:teuthology.orchestra.run.smithi094.stdout:2086 trunc from 0xde1a4f7 to 0x75f0fa2 2024-09-14T06:30:03.360 INFO:teuthology.orchestra.run.smithi094.stdout:2087 write 0xc23a158 thru 0xc23fddc (0x5c85 bytes) 2024-09-14T06:30:03.361 INFO:teuthology.orchestra.run.smithi094.stdout:2090 writesame 0x30f49ae thru 0x3102799 data_size 0xe(0xddec bytes) 2024-09-14T06:30:03.364 INFO:teuthology.orchestra.run.smithi094.stdout:2091 write 0x9677a6 thru 0x9748e6 (0xd141 bytes) 2024-09-14T06:30:03.365 INFO:teuthology.orchestra.run.smithi094.stdout:2092 write 0x91d25f2 thru 0x91d57f7 (0x3206 bytes) 2024-09-14T06:30:03.367 INFO:teuthology.orchestra.run.smithi094.stdout:2093 write 0x431574 thru 0x43a301 (0x8d8e bytes) 2024-09-14T06:30:03.375 INFO:teuthology.orchestra.run.smithi094.stdout:2094 trunc from 0xc23fddd to 0x6cfe94b 2024-09-14T06:30:03.394 INFO:teuthology.orchestra.run.smithi094.stdout:2095 write 0xd25b591 thru 0xd26b46e (0xfede bytes) 2024-09-14T06:30:03.407 INFO:teuthology.orchestra.run.smithi094.stdout:2096 trunc from 0xd26b46f to 0x449ac78 2024-09-14T06:30:03.413 INFO:teuthology.orchestra.run.smithi094.stdout:2099 read 0x419ab24 thru 0x419bcc9 (0x11a6 bytes) 2024-09-14T06:30:03.417 INFO:teuthology.orchestra.run.smithi094.stdout:2100 trunc from 0x449ac78 to 0x145ffc1 2024-09-14T06:30:03.432 INFO:tasks.ceph.osd.2.smithi094.stderr:2024-09-14T06:30:03.431+0000 7faf3269b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33649) UID: 0 2024-09-14T06:30:03.434 INFO:teuthology.orchestra.run.smithi094.stdout:2102 writesame 0x4c454ef thru 0x4c5074a data_size 0x1e(0xb25c bytes) 2024-09-14T06:30:03.436 INFO:teuthology.orchestra.run.smithi094.stdout:2103 punch from 0x1a0e45d to 0x1a1c28d, (0xde30 bytes) 2024-09-14T06:30:03.436 INFO:teuthology.orchestra.run.smithi094.stdout:2104 punch from 0x4381cd6 to 0x438cf0e, (0xb238 bytes) 2024-09-14T06:30:03.436 INFO:teuthology.orchestra.run.smithi094.stdout:2105 write 0x4c39a12 thru 0x4c3fec2 (0x64b1 bytes) 2024-09-14T06:30:03.437 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-14T06:30:03.437 INFO:teuthology.orchestra.run.smithi094.stdout:2107 read 0x8253a1 thru 0x82bb9c (0x67fc bytes) 2024-09-14T06:30:03.444 INFO:teuthology.orchestra.run.smithi094.stdout:2108 trunc from 0x4c5074b to 0x9ce421b 2024-09-14T06:30:03.449 INFO:teuthology.orchestra.run.smithi094.stdout:2109 read 0x595d783 thru 0x5966840 (0x90be bytes) 2024-09-14T06:30:03.454 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-14T06:30:03.454 INFO:teuthology.orchestra.run.smithi094.stdout: "ops": [], 2024-09-14T06:30:03.454 INFO:teuthology.orchestra.run.smithi094.stdout: "num_ops": 0 2024-09-14T06:30:03.454 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-14T06:30:03.459 INFO:teuthology.orchestra.run.smithi094.stdout:2110 trunc from 0x9ce421b to 0x1c31af4 2024-09-14T06:30:03.466 INFO:teuthology.orchestra.run.smithi094.stdout:2111 read 0xcf54d thru 0xd9781 (0xa235 bytes) 2024-09-14T06:30:03.467 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-14T06:30:03.477 INFO:teuthology.orchestra.run.smithi094.stdout:2112 write 0x6b21183 thru 0x6b2d764 (0xc5e2 bytes) 2024-09-14T06:30:03.479 INFO:teuthology.orchestra.run.smithi094.stdout:2113 read 0x3bedfee thru 0x3bf4dff (0x6e12 bytes) 2024-09-14T06:30:03.480 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-14T06:30:03.485 INFO:teuthology.orchestra.run.smithi094.stdout:2115 writesame 0x761dfa5 thru 0x762be79 data_size 0x5(0xded5 bytes) 2024-09-14T06:30:03.494 INFO:teuthology.orchestra.run.smithi094.stdout:2116 write 0x9cb6fa4 thru 0x9cb72d0 (0x32d bytes) 2024-09-14T06:30:03.496 INFO:teuthology.orchestra.run.smithi094.stdout:2117 read 0x26bc639 thru 0x26c1b75 (0x553d bytes) 2024-09-14T06:30:03.496 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-14T06:30:03.496 INFO:teuthology.orchestra.run.smithi094.stdout:2119 read 0x22d78ce thru 0x22de08c (0x67bf bytes) 2024-09-14T06:30:03.497 INFO:teuthology.orchestra.run.smithi094.stdout:2120 write 0x8416455 thru 0x841b47d (0x5029 bytes) 2024-09-14T06:30:03.498 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-14T06:30:03.498 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-14T06:30:03.499 INFO:teuthology.orchestra.run.smithi094.stdout:2123 punch from 0xa7054d to 0xa70af5, (0x5a8 bytes) 2024-09-14T06:30:03.499 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-14T06:30:03.508 INFO:teuthology.orchestra.run.smithi094.stdout:2127 trunc from 0x9cb72d1 to 0x198981c 2024-09-14T06:30:03.515 INFO:teuthology.orchestra.run.smithi094.stdout:2128 read 0x61e95 thru 0x6a92c (0x8a98 bytes) 2024-09-14T06:30:03.516 INFO:teuthology.orchestra.run.smithi094.stdout:2129 read 0xe3dc20 thru 0xe418d3 (0x3cb4 bytes) 2024-09-14T06:30:03.516 INFO:teuthology.orchestra.run.smithi094.stdout:2130 read 0xabb25b thru 0xac1869 (0x660f bytes) 2024-09-14T06:30:03.530 INFO:teuthology.orchestra.run.smithi094.stdout:2131 write 0x8989f01 thru 0x898b5d8 (0x16d8 bytes) 2024-09-14T06:30:03.534 INFO:tasks.ceph.osd.6.smithi138.stderr:2024-09-14T06:30:03.532+0000 7f8cd8849640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33620) UID: 0 2024-09-14T06:30:03.538 INFO:teuthology.orchestra.run.smithi094.stdout:2132 trunc from 0x898b5d9 to 0xde39d80 2024-09-14T06:30:03.543 INFO:teuthology.orchestra.run.smithi094.stdout:2133 writesame 0xcb4ecb1 thru 0xcb55341 data_size 0x1f(0x6691 bytes) 2024-09-14T06:30:03.545 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-14T06:30:03.545 INFO:teuthology.orchestra.run.smithi094.stdout:2136 punch from 0x15996e3 to 0x159a609, (0xf26 bytes) 2024-09-14T06:30:03.545 INFO:teuthology.orchestra.run.smithi094.stdout:2137 writesame 0x30a4c90 thru 0x30ada17 data_size 0x1c(0x8d88 bytes) 2024-09-14T06:30:03.546 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-14T06:30:03.546 INFO:teuthology.orchestra.run.smithi094.stdout:2140 read 0x9a2f55b thru 0x9a35008 (0x5aae bytes) 2024-09-14T06:30:03.547 INFO:teuthology.orchestra.run.smithi094.stdout:2141 read 0xd9ab146 thru 0xd9b8da7 (0xdc62 bytes) 2024-09-14T06:30:03.547 INFO:teuthology.orchestra.run.smithi094.stdout:2143 write 0xc0c0694 thru 0xc0cc05d (0xb9ca bytes) 2024-09-14T06:30:03.549 INFO:teuthology.orchestra.run.smithi094.stdout:2147 writesame 0x72375ab thru 0x7245db1 data_size 0x1(0xe807 bytes) 2024-09-14T06:30:03.576 INFO:teuthology.orchestra.run.smithi094.stdout:2148 write 0xd28afb9 thru 0xd28bbd8 (0xc20 bytes) 2024-09-14T06:30:03.578 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-14T06:30:03.578 INFO:teuthology.orchestra.run.smithi094.stdout:2150 writesame 0xb12559f thru 0xb127bf3 data_size 0x1(0x2655 bytes) 2024-09-14T06:30:03.583 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-14T06:30:03.583 INFO:teuthology.orchestra.run.smithi094.stdout:2152 write 0xf0fe6d thru 0xf133ec (0x3580 bytes) 2024-09-14T06:30:03.590 INFO:teuthology.orchestra.run.smithi094.stdout:2153 write 0xe1c211c thru 0xe1c78c7 (0x57ac bytes) 2024-09-14T06:30:03.591 INFO:teuthology.orchestra.run.smithi094.stdout:2155 read 0x3929bec thru 0x3929da8 (0x1bd bytes) 2024-09-14T06:30:03.591 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-14T06:30:03.592 INFO:teuthology.orchestra.run.smithi094.stdout: "ops": [], 2024-09-14T06:30:03.592 INFO:teuthology.orchestra.run.smithi094.stdout: "complaint_time": 30, 2024-09-14T06:30:03.592 INFO:teuthology.orchestra.run.smithi094.stdout: "num_blocked_ops": 0 2024-09-14T06:30:03.592 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-14T06:30:03.592 INFO:teuthology.orchestra.run.smithi094.stdout:2156 write 0xb8ee4cd thru 0xb8f8370 (0x9ea4 bytes) 2024-09-14T06:30:03.593 INFO:teuthology.orchestra.run.smithi094.stdout:2157 read 0xaed4c91 thru 0xaed56fc (0xa6c bytes) 2024-09-14T06:30:03.593 INFO:teuthology.orchestra.run.smithi094.stdout:2158 write 0x7d913 thru 0x7ebf4 (0x12e2 bytes) 2024-09-14T06:30:03.595 INFO:teuthology.orchestra.run.smithi094.stdout:2159 read 0x4ee569f thru 0x4ee9192 (0x3af4 bytes) 2024-09-14T06:30:03.595 INFO:teuthology.orchestra.run.smithi094.stdout:2160 write 0x7f0a99 thru 0x7fa247 (0x97af bytes) 2024-09-14T06:30:03.601 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-14T06:30:03.607 INFO:teuthology.orchestra.run.smithi094.stdout:2161 trunc from 0xe1c78c8 to 0x55e0a90 2024-09-14T06:30:03.614 INFO:teuthology.orchestra.run.smithi094.stdout:2163 write 0x23f7fcf thru 0x240334f (0xb381 bytes) 2024-09-14T06:30:03.616 INFO:teuthology.orchestra.run.smithi094.stdout:2164 read 0x2702c70 thru 0x2710358 (0xd6e9 bytes) 2024-09-14T06:30:03.616 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-14T06:30:03.617 INFO:teuthology.orchestra.run.smithi094.stdout:2166 read 0xe8f8c0 thru 0xe96a03 (0x7144 bytes) 2024-09-14T06:30:03.617 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-14T06:30:03.617 INFO:teuthology.orchestra.run.smithi094.stdout:2171 read 0x16c925b thru 0x16cdfff (0x4da5 bytes) 2024-09-14T06:30:03.617 INFO:teuthology.orchestra.run.smithi094.stdout:2172 punch from 0x479db22 to 0x47a7634, (0x9b12 bytes) 2024-09-14T06:30:03.618 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-14T06:30:03.618 INFO:teuthology.orchestra.run.smithi094.stdout:2174 writesame 0x4a8afb2 thru 0x4a8b14d data_size 0x2(0x19c bytes) 2024-09-14T06:30:03.618 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-14T06:30:03.619 INFO:teuthology.orchestra.run.smithi094.stdout:2176 writesame 0x53321ed thru 0x533afd8 data_size 0x3e(0x8dec bytes) 2024-09-14T06:30:03.620 INFO:teuthology.orchestra.run.smithi094.stdout:2177 read 0x55485a3 thru 0x555305f (0xaabd bytes) 2024-09-14T06:30:03.622 INFO:teuthology.orchestra.run.smithi094.stdout:2178 trunc from 0x55e0a90 to 0x3e081d1 2024-09-14T06:30:03.630 INFO:teuthology.orchestra.run.smithi094.stdout:2179 read 0x341d23c thru 0x342bb03 (0xe8c8 bytes) 2024-09-14T06:30:03.631 INFO:teuthology.orchestra.run.smithi094.stdout:2180 writesame 0x254f306 thru 0x255b361 data_size 0x1a(0xc05c bytes) 2024-09-14T06:30:03.633 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-14T06:30:03.634 INFO:tasks.ceph.osd.1.smithi094.stderr:2024-09-14T06:30:03.633+0000 7f328e47a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33641) UID: 0 2024-09-14T06:30:03.640 INFO:teuthology.orchestra.run.smithi094.stdout:2183 trunc from 0x3e081d1 to 0x9e45c8f 2024-09-14T06:30:03.656 INFO:teuthology.orchestra.run.smithi094.stdout:2184 writesame 0xe952b69 thru 0xe9619c8 data_size 0x10(0xee60 bytes) 2024-09-14T06:30:03.675 INFO:teuthology.orchestra.run.smithi094.stdout:2185 trunc from 0xe9619c9 to 0x948d8d 2024-09-14T06:30:03.691 INFO:teuthology.orchestra.run.smithi094.stdout:2187 read 0x34bf08 thru 0x352ce1 (0x6dda bytes) 2024-09-14T06:30:03.691 INFO:teuthology.orchestra.run.smithi094.stdout:2188 read 0x1a6490 thru 0x1b3807 (0xd378 bytes) 2024-09-14T06:30:03.692 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-14T06:30:03.714 INFO:teuthology.orchestra.run.smithi094.stdout:2191 write 0xe9abcce thru 0xe9af85a (0x3b8d bytes) 2024-09-14T06:30:03.717 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-14T06:30:03.717 INFO:teuthology.orchestra.run.smithi094.stdout:2194 writesame 0x6b9b2da thru 0x6ba5876 data_size 0x1(0xa59d bytes) 2024-09-14T06:30:03.729 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-14T06:30:03.729 INFO:teuthology.orchestra.run.smithi094.stdout: "size": 20, 2024-09-14T06:30:03.729 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 600, 2024-09-14T06:30:03.729 INFO:teuthology.orchestra.run.smithi094.stdout: "ops": [ 2024-09-14T06:30:03.729 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:03.730 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:44425 3.2 3:4603ad7f:::rbd_data.112dca2f6ae1.0000000000000005:17 [read 868352~4096 [fadvise_nocache]] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:30:03.730 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:43.854967+0000", 2024-09-14T06:30:03.730 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 19.871910188000001, 2024-09-14T06:30:03.730 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.063146025999999994, 2024-09-14T06:30:03.730 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:30:03.730 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:30:03.730 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:30:03.730 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:30:03.730 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:30:03.730 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:03.730 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 44425 2024-09-14T06:30:03.730 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:03.731 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:30:03.731 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:03.731 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:30:03.731 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.854967+0000", 2024-09-14T06:30:03.731 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:03.731 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:03.731 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:03.731 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:30:03.731 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.854967+0000", 2024-09-14T06:30:03.731 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:03.731 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:03.731 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:03.731 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:30:03.731 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.854967+0000", 2024-09-14T06:30:03.731 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.16e-07 2024-09-14T06:30:03.732 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:03.732 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:03.732 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:30:03.732 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.854968+0000", 2024-09-14T06:30:03.732 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.006e-06 2024-09-14T06:30:03.732 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:03.732 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:03.732 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:30:03.732 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.854971+0000", 2024-09-14T06:30:03.732 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.9359999999999999e-06 2024-09-14T06:30:03.732 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:03.732 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:03.732 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:30:03.732 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.854974+0000", 2024-09-14T06:30:03.733 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.1840000000000002e-06 2024-09-14T06:30:03.733 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:03.733 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:03.733 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:30:03.733 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.917945+0000", 2024-09-14T06:30:03.733 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.062970925999999997 2024-09-14T06:30:03.733 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:03.733 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:03.733 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:30:03.733 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.918030+0000", 2024-09-14T06:30:03.733 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.5863000000000001e-05 2024-09-14T06:30:03.733 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:03.733 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:03.734 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:30:03.734 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.918113+0000", 2024-09-14T06:30:03.734 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.2894999999999998e-05 2024-09-14T06:30:03.734 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:03.734 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:03.734 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:03.734 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:03.734 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:03.734 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:44426 3.2 3:4603ad7f:::rbd_data.112dca2f6ae1.0000000000000005:17 [read 892928~4096 [fadvise_nocache]] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:30:03.734 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:43.854977+0000", 2024-09-14T06:30:03.734 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 19.871900778000001, 2024-09-14T06:30:03.734 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.063369237999999994, 2024-09-14T06:30:03.735 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:30:03.735 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:30:03.735 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:30:03.735 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:30:03.735 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:30:03.735 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:03.735 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 44426 2024-09-14T06:30:03.735 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:03.735 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:30:03.735 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:03.735 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:30:03.735 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.854977+0000", 2024-09-14T06:30:03.735 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:03.735 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:03.736 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:03.736 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:30:03.736 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.854977+0000", 2024-09-14T06:30:03.736 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:03.736 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:03.736 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:03.736 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:30:03.736 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.854977+0000", 2024-09-14T06:30:03.736 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.1199999999999999e-07 2024-09-14T06:30:03.736 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:03.736 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:03.736 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:30:03.736 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.854980+0000", 2024-09-14T06:30:03.736 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.27e-06 2024-09-14T06:30:03.737 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:03.737 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:03.737 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:30:03.737 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.854983+0000", 2024-09-14T06:30:03.737 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.1650000000000002e-06 2024-09-14T06:30:03.737 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:03.737 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:03.737 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:30:03.737 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.854989+0000", 2024-09-14T06:30:03.737 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.6520000000000003e-06 2024-09-14T06:30:03.737 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:03.737 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:03.737 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:30:03.738 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.918143+0000", 2024-09-14T06:30:03.738 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.063154541999999994 2024-09-14T06:30:03.738 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:03.738 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:03.738 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:30:03.738 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.918209+0000", 2024-09-14T06:30:03.738 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.5529000000000002e-05 2024-09-14T06:30:03.738 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:03.738 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:03.738 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:30:03.738 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.918346+0000", 2024-09-14T06:30:03.738 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000136868 2024-09-14T06:30:03.739 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:03.739 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:03.739 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:03.739 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:03.739 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:03.739 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:44427 3.2 3:4603ad7f:::rbd_data.112dca2f6ae1.0000000000000005:17 [read 917504~4096 [fadvise_nocache]] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:30:03.739 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:43.854992+0000", 2024-09-14T06:30:03.739 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 19.871885131999999, 2024-09-14T06:30:03.739 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.063607792999999996, 2024-09-14T06:30:03.739 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:30:03.739 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:30:03.739 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:30:03.739 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:30:03.739 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:30:03.740 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:03.740 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 44427 2024-09-14T06:30:03.740 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:03.740 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:30:03.740 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:03.740 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:30:03.740 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.854992+0000", 2024-09-14T06:30:03.740 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:03.740 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:03.740 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:03.740 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:30:03.740 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.854992+0000", 2024-09-14T06:30:03.740 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:03.740 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:03.741 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:03.741 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:30:03.741 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.854993+0000", 2024-09-14T06:30:03.741 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.1400000000000001e-07 2024-09-14T06:30:03.741 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:03.741 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:03.741 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:30:03.741 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.854994+0000", 2024-09-14T06:30:03.741 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.091e-06 2024-09-14T06:30:03.741 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:03.741 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:03.742 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:30:03.742 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855001+0000", 2024-09-14T06:30:03.742 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.204e-06 2024-09-14T06:30:03.742 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:03.742 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:03.742 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:30:03.742 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855003+0000", 2024-09-14T06:30:03.742 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.514e-06 2024-09-14T06:30:03.742 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:03.742 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:03.742 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:30:03.742 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.918380+0000", 2024-09-14T06:30:03.742 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.063376738000000002 2024-09-14T06:30:03.742 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:03.742 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:03.743 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:30:03.743 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.918453+0000", 2024-09-14T06:30:03.743 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.3291e-05 2024-09-14T06:30:03.743 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:03.743 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:03.743 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:30:03.743 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.918600+0000", 2024-09-14T06:30:03.743 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00014674100000000001 2024-09-14T06:30:03.743 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:03.743 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:03.743 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:03.743 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:03.743 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:03.743 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:44428 3.2 3:4603ad7f:::rbd_data.112dca2f6ae1.0000000000000005:17 [read 942080~4096 [fadvise_nocache]] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:30:03.743 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:43.855006+0000", 2024-09-14T06:30:03.744 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 19.871871081999998, 2024-09-14T06:30:03.744 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.063846321999999997, 2024-09-14T06:30:03.744 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:30:03.744 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:30:03.744 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:30:03.744 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:30:03.744 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:30:03.744 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:03.744 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 44428 2024-09-14T06:30:03.744 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:03.744 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:30:03.744 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:03.745 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:30:03.745 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855006+0000", 2024-09-14T06:30:03.745 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:03.745 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:03.745 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:03.745 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:30:03.745 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855006+0000", 2024-09-14T06:30:03.745 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:03.745 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:03.745 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:03.745 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:30:03.745 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855007+0000", 2024-09-14T06:30:03.745 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.1199999999999999e-07 2024-09-14T06:30:03.745 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:03.746 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:03.746 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:30:03.746 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855011+0000", 2024-09-14T06:30:03.746 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.9539999999999998e-06 2024-09-14T06:30:03.746 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:03.746 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:03.746 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:30:03.746 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855014+0000", 2024-09-14T06:30:03.746 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.2930000000000001e-06 2024-09-14T06:30:03.746 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:03.746 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:03.746 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:30:03.746 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855016+0000", 2024-09-14T06:30:03.746 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.3039999999999999e-06 2024-09-14T06:30:03.746 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:03.747 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:03.747 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:30:03.747 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.918634+0000", 2024-09-14T06:30:03.747 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.063617928000000004 2024-09-14T06:30:03.747 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:03.747 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:03.747 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:30:03.747 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.918711+0000", 2024-09-14T06:30:03.747 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.6545000000000007e-05 2024-09-14T06:30:03.747 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:03.747 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:03.747 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:30:03.747 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.918853+0000", 2024-09-14T06:30:03.747 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00014208600000000001 2024-09-14T06:30:03.747 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:03.747 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:03.748 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:03.748 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:03.748 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:03.748 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:44429 3.2 3:4603ad7f:::rbd_data.112dca2f6ae1.0000000000000005:17 [read 966656~4096 [fadvise_nocache]] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:30:03.748 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:43.855019+0000", 2024-09-14T06:30:03.748 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 19.871858409000001, 2024-09-14T06:30:03.748 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.064059226999999996, 2024-09-14T06:30:03.748 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:30:03.748 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:30:03.748 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:30:03.748 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:30:03.748 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:30:03.748 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:03.749 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 44429 2024-09-14T06:30:03.749 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:03.749 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:30:03.749 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:03.749 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:30:03.749 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855019+0000", 2024-09-14T06:30:03.749 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:03.749 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:03.749 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:03.749 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:30:03.749 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855019+0000", 2024-09-14T06:30:03.749 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:03.749 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:03.749 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:03.750 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:30:03.750 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855019+0000", 2024-09-14T06:30:03.750 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.28e-07 2024-09-14T06:30:03.750 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:03.750 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:03.750 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:30:03.750 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855020+0000", 2024-09-14T06:30:03.750 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.0330000000000001e-06 2024-09-14T06:30:03.750 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:03.750 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:03.750 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:30:03.750 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855023+0000", 2024-09-14T06:30:03.750 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.79e-06 2024-09-14T06:30:03.750 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:03.751 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:03.751 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:30:03.751 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855025+0000", 2024-09-14T06:30:03.751 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.216e-06 2024-09-14T06:30:03.751 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:03.751 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:03.751 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:30:03.751 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.918886+0000", 2024-09-14T06:30:03.751 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.063861037999999995 2024-09-14T06:30:03.751 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:03.751 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:03.751 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:30:03.751 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.918952+0000", 2024-09-14T06:30:03.751 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.5993000000000003e-05 2024-09-14T06:30:03.752 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:03.752 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:03.752 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:30:03.752 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.919078+0000", 2024-09-14T06:30:03.752 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00012592899999999999 2024-09-14T06:30:03.752 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:03.752 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:03.752 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:03.752 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:03.752 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:03.752 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:44430 3.2 3:4603ad7f:::rbd_data.112dca2f6ae1.0000000000000005:17 [read 991232~4096 [fadvise_nocache]] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:30:03.752 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:43.855028+0000", 2024-09-14T06:30:03.752 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 19.871849398999998, 2024-09-14T06:30:03.752 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.064284346000000006, 2024-09-14T06:30:03.752 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:30:03.753 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:30:03.753 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:30:03.753 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:30:03.753 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:30:03.753 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:03.753 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 44430 2024-09-14T06:30:03.753 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:03.753 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:30:03.753 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:03.753 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:30:03.753 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855028+0000", 2024-09-14T06:30:03.753 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:03.753 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:03.753 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:03.753 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:30:03.754 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855028+0000", 2024-09-14T06:30:03.754 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:03.754 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:03.754 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:03.754 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:30:03.754 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855028+0000", 2024-09-14T06:30:03.754 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.1199999999999999e-07 2024-09-14T06:30:03.754 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:03.754 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:03.754 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:30:03.754 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855029+0000", 2024-09-14T06:30:03.754 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.1100000000000004e-07 2024-09-14T06:30:03.754 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:03.755 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:03.755 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:30:03.755 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855032+0000", 2024-09-14T06:30:03.755 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.7889999999999999e-06 2024-09-14T06:30:03.755 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:03.755 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:03.755 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:30:03.755 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855034+0000", 2024-09-14T06:30:03.755 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.125e-06 2024-09-14T06:30:03.755 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:03.755 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:03.755 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:30:03.755 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.919110+0000", 2024-09-14T06:30:03.755 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.064076413999999998 2024-09-14T06:30:03.755 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:03.756 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:03.756 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:30:03.756 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.919186+0000", 2024-09-14T06:30:03.756 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.5784999999999994e-05 2024-09-14T06:30:03.756 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:03.756 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:03.756 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:30:03.756 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.919312+0000", 2024-09-14T06:30:03.756 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00012611 2024-09-14T06:30:03.756 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:03.756 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:03.756 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:03.756 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:03.757 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:03.757 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:44431 3.2 3:4603ad7f:::rbd_data.112dca2f6ae1.0000000000000005:17 [read 1015808~4096 [fadvise_nocache]] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:30:03.757 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:43.855037+0000", 2024-09-14T06:30:03.757 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 19.871840531, 2024-09-14T06:30:03.757 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.064469755000000004, 2024-09-14T06:30:03.757 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:30:03.757 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:30:03.757 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:30:03.757 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:30:03.757 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:30:03.757 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:03.757 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 44431 2024-09-14T06:30:03.757 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:03.758 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:30:03.758 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:03.758 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:30:03.758 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855037+0000", 2024-09-14T06:30:03.758 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:03.758 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:03.758 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:03.758 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:30:03.758 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855037+0000", 2024-09-14T06:30:03.758 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:03.758 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:03.758 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:03.758 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:30:03.758 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855037+0000", 2024-09-14T06:30:03.758 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.1e-07 2024-09-14T06:30:03.759 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:03.759 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:03.759 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:30:03.759 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855038+0000", 2024-09-14T06:30:03.759 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.9400000000000004e-07 2024-09-14T06:30:03.759 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:03.759 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:03.759 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:30:03.759 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855041+0000", 2024-09-14T06:30:03.759 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.9639999999999999e-06 2024-09-14T06:30:03.759 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:03.759 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:03.759 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:30:03.759 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855043+0000", 2024-09-14T06:30:03.760 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.2259999999999999e-06 2024-09-14T06:30:03.760 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:03.760 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:03.760 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:30:03.760 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.919347+0000", 2024-09-14T06:30:03.760 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.064304091999999993 2024-09-14T06:30:03.760 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:03.760 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:03.760 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:30:03.760 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.919393+0000", 2024-09-14T06:30:03.760 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.5340999999999998e-05 2024-09-14T06:30:03.760 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:03.760 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:03.761 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:30:03.761 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.919507+0000", 2024-09-14T06:30:03.761 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000114028 2024-09-14T06:30:03.761 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:03.761 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:03.761 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:03.761 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:03.761 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:03.761 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:44432 3.2 3:4603ad7f:::rbd_data.112dca2f6ae1.0000000000000005:17 [read 1040384~4096 [fadvise_nocache]] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:30:03.761 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:43.855046+0000", 2024-09-14T06:30:03.761 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 19.871831462999999, 2024-09-14T06:30:03.761 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.064639253999999993, 2024-09-14T06:30:03.761 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:30:03.762 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:30:03.762 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:30:03.762 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:30:03.762 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:30:03.762 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:03.762 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 44432 2024-09-14T06:30:03.762 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:03.762 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:30:03.762 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:03.762 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:30:03.762 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855046+0000", 2024-09-14T06:30:03.762 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:03.762 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:03.762 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:03.762 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:30:03.763 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855046+0000", 2024-09-14T06:30:03.763 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:03.763 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:03.763 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:03.763 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:30:03.763 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855046+0000", 2024-09-14T06:30:03.763 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.05e-07 2024-09-14T06:30:03.763 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:03.763 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:03.763 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:30:03.763 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855047+0000", 2024-09-14T06:30:03.763 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.879999999999999e-07 2024-09-14T06:30:03.763 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:03.764 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:03.764 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:30:03.764 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855050+0000", 2024-09-14T06:30:03.764 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.8459999999999999e-06 2024-09-14T06:30:03.764 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:03.764 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:03.764 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:30:03.764 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855052+0000", 2024-09-14T06:30:03.764 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.136e-06 2024-09-14T06:30:03.765 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:03.765 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:03.765 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:30:03.765 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.919536+0000", 2024-09-14T06:30:03.765 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.064484132 2024-09-14T06:30:03.765 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:03.765 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:03.765 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:30:03.765 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.919611+0000", 2024-09-14T06:30:03.765 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.4407999999999996e-05 2024-09-14T06:30:03.765 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:03.765 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:03.765 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:30:03.766 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.919685+0000", 2024-09-14T06:30:03.766 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.4538999999999999e-05 2024-09-14T06:30:03.766 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:03.766 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:03.766 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:03.766 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:03.766 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:03.766 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:44518 3.2 3:4190d882:::rbd_data.112dca2f6ae1.0000000000000002:17 [read 12288~4096 [fadvise_nocache]] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:30:03.766 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:43.855055+0000", 2024-09-14T06:30:03.766 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 19.871822579, 2024-09-14T06:30:03.766 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.064894317000000007, 2024-09-14T06:30:03.766 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:30:03.766 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:30:03.767 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:30:03.767 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:30:03.767 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:30:03.767 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:03.767 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 44518 2024-09-14T06:30:03.767 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:03.767 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:30:03.767 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:03.767 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:30:03.767 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855055+0000", 2024-09-14T06:30:03.767 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:03.767 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:03.767 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:03.767 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:30:03.768 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855055+0000", 2024-09-14T06:30:03.768 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:03.768 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:03.768 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:03.768 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:30:03.768 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855055+0000", 2024-09-14T06:30:03.768 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.0699999999999999e-07 2024-09-14T06:30:03.768 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:03.768 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:03.768 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:30:03.768 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855056+0000", 2024-09-14T06:30:03.768 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.9400000000000004e-07 2024-09-14T06:30:03.768 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:03.769 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:03.769 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:30:03.769 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855059+0000", 2024-09-14T06:30:03.769 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.8870000000000001e-06 2024-09-14T06:30:03.769 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:03.769 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:03.769 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:30:03.769 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855061+0000", 2024-09-14T06:30:03.769 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.1919999999999999e-06 2024-09-14T06:30:03.769 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:03.769 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:03.769 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:30:03.769 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.919719+0000", 2024-09-14T06:30:03.769 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.064658013 2024-09-14T06:30:03.770 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:03.770 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:03.770 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:30:03.770 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.919788+0000", 2024-09-14T06:30:03.770 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.9282000000000005e-05 2024-09-14T06:30:03.770 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:03.770 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:03.770 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:30:03.770 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.919949+0000", 2024-09-14T06:30:03.770 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00016084199999999999 2024-09-14T06:30:03.770 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:03.770 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:03.770 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:03.770 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:03.771 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:03.771 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:44519 3.2 3:4190d882:::rbd_data.112dca2f6ae1.0000000000000002:17 [read 36864~4096 [fadvise_nocache]] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:30:03.771 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:43.855064+0000", 2024-09-14T06:30:03.771 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 19.871813602, 2024-09-14T06:30:03.771 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.065081865000000003, 2024-09-14T06:30:03.771 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:30:03.771 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:30:03.771 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:30:03.771 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:30:03.771 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:30:03.771 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:03.771 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 44519 2024-09-14T06:30:03.771 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:03.772 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:30:03.772 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:03.772 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:30:03.772 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855064+0000", 2024-09-14T06:30:03.772 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:03.772 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:03.772 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:03.772 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:30:03.772 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855064+0000", 2024-09-14T06:30:03.772 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:03.772 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:03.772 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:03.772 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:30:03.773 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855064+0000", 2024-09-14T06:30:03.773 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.1e-07 2024-09-14T06:30:03.773 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:03.773 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:03.773 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:30:03.773 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855065+0000", 2024-09-14T06:30:03.773 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.035e-06 2024-09-14T06:30:03.773 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:03.773 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:03.773 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:30:03.773 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855068+0000", 2024-09-14T06:30:03.773 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.8339999999999999e-06 2024-09-14T06:30:03.774 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:03.774 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:03.774 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:30:03.774 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855071+0000", 2024-09-14T06:30:03.774 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.3450000000000002e-06 2024-09-14T06:30:03.774 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:03.774 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:03.774 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:30:03.774 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.919979+0000", 2024-09-14T06:30:03.774 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.064907647999999998 2024-09-14T06:30:03.775 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:03.775 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:03.775 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:30:03.775 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.920026+0000", 2024-09-14T06:30:03.775 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.7595000000000001e-05 2024-09-14T06:30:03.775 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:03.775 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:03.775 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:30:03.775 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.920146+0000", 2024-09-14T06:30:03.775 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00011919800000000001 2024-09-14T06:30:03.776 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:03.776 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:03.776 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:03.776 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:03.776 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:03.776 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:44520 3.2 3:4190d882:::rbd_data.112dca2f6ae1.0000000000000002:17 [read 61440~4096 [fadvise_nocache]] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:30:03.776 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:43.855075+0000", 2024-09-14T06:30:03.776 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 19.871802551000002, 2024-09-14T06:30:03.776 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.065335078000000005, 2024-09-14T06:30:03.776 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:30:03.776 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:30:03.776 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:30:03.777 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:30:03.777 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:30:03.777 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:03.777 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 44520 2024-09-14T06:30:03.777 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:03.777 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:30:03.777 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:03.777 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:30:03.777 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855075+0000", 2024-09-14T06:30:03.777 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:03.777 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:03.777 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:03.777 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:30:03.777 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855075+0000", 2024-09-14T06:30:03.778 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:03.778 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:03.778 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:03.778 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:30:03.778 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855075+0000", 2024-09-14T06:30:03.778 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.1e-07 2024-09-14T06:30:03.778 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:03.778 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:03.778 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:30:03.778 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855076+0000", 2024-09-14T06:30:03.778 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.0839999999999999e-06 2024-09-14T06:30:03.778 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:03.778 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:03.779 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:30:03.779 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855079+0000", 2024-09-14T06:30:03.779 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.9359999999999999e-06 2024-09-14T06:30:03.779 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:03.779 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:03.779 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:30:03.779 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855084+0000", 2024-09-14T06:30:03.779 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.1730000000000003e-06 2024-09-14T06:30:03.779 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:03.779 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:03.779 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:30:03.779 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.920175+0000", 2024-09-14T06:30:03.779 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.065090234999999996 2024-09-14T06:30:03.779 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:03.780 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:03.780 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:30:03.780 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.920250+0000", 2024-09-14T06:30:03.780 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.5807000000000001e-05 2024-09-14T06:30:03.780 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:03.780 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:03.780 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:30:03.780 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.920410+0000", 2024-09-14T06:30:03.780 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00015963300000000001 2024-09-14T06:30:03.780 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:03.780 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:03.780 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:03.780 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:03.780 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:03.781 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:44521 3.2 3:4190d882:::rbd_data.112dca2f6ae1.0000000000000002:17 [read 86016~4096 [fadvise_nocache]] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:30:03.781 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:43.855087+0000", 2024-09-14T06:30:03.781 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 19.871790069999999, 2024-09-14T06:30:03.781 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.065563264999999996, 2024-09-14T06:30:03.781 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:30:03.781 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:30:03.781 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:30:03.781 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:30:03.781 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:30:03.781 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:03.782 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 44521 2024-09-14T06:30:03.782 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:03.782 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:30:03.782 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:03.782 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:30:03.782 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855087+0000", 2024-09-14T06:30:03.782 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:03.782 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:03.782 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:03.782 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:30:03.782 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855087+0000", 2024-09-14T06:30:03.782 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:03.782 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:03.782 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:03.782 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:30:03.783 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855088+0000", 2024-09-14T06:30:03.783 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.3699999999999999e-07 2024-09-14T06:30:03.783 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:03.783 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:03.783 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:30:03.783 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855089+0000", 2024-09-14T06:30:03.783 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.9900000000000009e-07 2024-09-14T06:30:03.783 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:03.783 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:03.783 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:30:03.783 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855091+0000", 2024-09-14T06:30:03.783 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.8940000000000002e-06 2024-09-14T06:30:03.783 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:03.783 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:03.784 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:30:03.784 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855094+0000", 2024-09-14T06:30:03.784 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.351e-06 2024-09-14T06:30:03.784 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:03.784 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:03.784 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:30:03.784 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.920437+0000", 2024-09-14T06:30:03.784 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.065343221000000007 2024-09-14T06:30:03.784 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:03.784 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:03.784 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:30:03.784 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.920481+0000", 2024-09-14T06:30:03.784 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.4431000000000002e-05 2024-09-14T06:30:03.784 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:03.785 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:03.785 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:30:03.785 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.920651+0000", 2024-09-14T06:30:03.785 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00016913200000000001 2024-09-14T06:30:03.785 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:03.785 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:03.785 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:03.785 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:03.785 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:03.785 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:44522 3.2 3:4190d882:::rbd_data.112dca2f6ae1.0000000000000002:17 [read 110592~4096 [fadvise_nocache]] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:30:03.785 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:43.855098+0000", 2024-09-14T06:30:03.786 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 19.871779842999999, 2024-09-14T06:30:03.786 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.065827628999999999, 2024-09-14T06:30:03.786 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:30:03.786 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:30:03.786 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:30:03.786 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:30:03.786 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:30:03.786 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:03.786 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 44522 2024-09-14T06:30:03.786 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:03.787 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:30:03.787 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:03.787 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:30:03.787 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855098+0000", 2024-09-14T06:30:03.787 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:03.787 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:03.787 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:03.787 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:30:03.787 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855098+0000", 2024-09-14T06:30:03.787 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:03.787 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:03.787 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:03.787 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:30:03.787 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855098+0000", 2024-09-14T06:30:03.788 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.1e-07 2024-09-14T06:30:03.788 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:03.788 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:03.788 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:30:03.788 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855099+0000", 2024-09-14T06:30:03.788 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.012e-06 2024-09-14T06:30:03.788 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:03.788 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:03.788 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:30:03.788 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855102+0000", 2024-09-14T06:30:03.788 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.9380000000000001e-06 2024-09-14T06:30:03.788 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:03.788 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:03.788 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:30:03.789 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855108+0000", 2024-09-14T06:30:03.789 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.4339999999999997e-06 2024-09-14T06:30:03.789 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:03.789 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:03.789 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:30:03.789 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.920681+0000", 2024-09-14T06:30:03.789 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.065572943999999994 2024-09-14T06:30:03.790 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:03.790 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:03.790 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:30:03.790 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.920724+0000", 2024-09-14T06:30:03.790 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.2877000000000003e-05 2024-09-14T06:30:03.790 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:03.791 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:03.791 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:30:03.791 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.920925+0000", 2024-09-14T06:30:03.791 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00020121399999999999 2024-09-14T06:30:03.791 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:03.791 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:03.791 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:03.791 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:03.791 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:03.791 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:44523 3.2 3:4190d882:::rbd_data.112dca2f6ae1.0000000000000002:17 [read 135168~4096 [fadvise_nocache]] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:30:03.791 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:43.855111+0000", 2024-09-14T06:30:03.791 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 19.871766121, 2024-09-14T06:30:03.791 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.066028198999999996, 2024-09-14T06:30:03.792 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:30:03.792 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:30:03.792 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:30:03.792 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:30:03.792 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:30:03.792 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:03.792 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 44523 2024-09-14T06:30:03.792 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:03.792 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:30:03.792 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:03.792 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:30:03.792 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855111+0000", 2024-09-14T06:30:03.792 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:03.792 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:03.793 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:03.793 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:30:03.793 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855111+0000", 2024-09-14T06:30:03.793 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:03.793 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:03.793 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:03.794 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:30:03.794 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855112+0000", 2024-09-14T06:30:03.794 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.1199999999999999e-07 2024-09-14T06:30:03.794 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:03.794 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:03.794 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:30:03.794 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855115+0000", 2024-09-14T06:30:03.794 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.3380000000000001e-06 2024-09-14T06:30:03.794 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:03.794 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:03.794 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:30:03.794 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855118+0000", 2024-09-14T06:30:03.794 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.366e-06 2024-09-14T06:30:03.794 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:03.795 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:03.795 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:30:03.795 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855121+0000", 2024-09-14T06:30:03.795 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.8329999999999998e-06 2024-09-14T06:30:03.795 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:03.795 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:03.795 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:30:03.795 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.920956+0000", 2024-09-14T06:30:03.795 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.065835251999999997 2024-09-14T06:30:03.795 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:03.795 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:03.796 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:30:03.796 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.921001+0000", 2024-09-14T06:30:03.796 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.4778999999999999e-05 2024-09-14T06:30:03.796 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:03.796 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:03.796 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:30:03.796 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.921139+0000", 2024-09-14T06:30:03.796 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000138419 2024-09-14T06:30:03.796 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:03.796 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:03.796 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:03.796 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:03.796 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:03.796 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:44524 3.2 3:4190d882:::rbd_data.112dca2f6ae1.0000000000000002:17 [read 159744~4096 [fadvise_nocache]] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:30:03.796 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:43.855124+0000", 2024-09-14T06:30:03.797 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 19.871753141999999, 2024-09-14T06:30:03.797 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.066249261000000004, 2024-09-14T06:30:03.797 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:30:03.797 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:30:03.797 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:30:03.797 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:30:03.797 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:30:03.797 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:03.797 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 44524 2024-09-14T06:30:03.797 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:03.797 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:30:03.797 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:03.797 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:30:03.798 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855124+0000", 2024-09-14T06:30:03.798 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:03.798 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:03.798 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:03.798 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:30:03.798 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855124+0000", 2024-09-14T06:30:03.798 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:03.798 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:03.798 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:03.798 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:30:03.798 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861086+0000", 2024-09-14T06:30:03.798 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0059613829999999998 2024-09-14T06:30:03.798 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:03.798 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:03.798 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:30:03.799 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861100+0000", 2024-09-14T06:30:03.799 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.4644000000000001e-05 2024-09-14T06:30:03.799 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:03.799 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:03.799 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:30:03.799 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861109+0000", 2024-09-14T06:30:03.799 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.541e-06 2024-09-14T06:30:03.799 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:03.799 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:03.799 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:30:03.800 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861117+0000", 2024-09-14T06:30:03.800 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.1049999999999995e-06 2024-09-14T06:30:03.800 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:03.800 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:03.800 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:30:03.800 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.921169+0000", 2024-09-14T06:30:03.800 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.060052405000000003 2024-09-14T06:30:03.800 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:03.800 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:03.800 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:30:03.800 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.921213+0000", 2024-09-14T06:30:03.800 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.3565e-05 2024-09-14T06:30:03.800 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:03.801 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:03.801 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:30:03.801 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.921374+0000", 2024-09-14T06:30:03.801 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00016061800000000001 2024-09-14T06:30:03.801 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:03.801 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:03.801 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:03.801 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:03.801 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:03.801 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:44536 3.2 3:4190d882:::rbd_data.112dca2f6ae1.0000000000000002:17 [read 454656~4096 [fadvise_nocache]] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:30:03.801 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:43.861384+0000", 2024-09-14T06:30:03.801 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 19.865493584999999, 2024-09-14T06:30:03.801 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.063058528000000003, 2024-09-14T06:30:03.802 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:30:03.802 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:30:03.802 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:30:03.802 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:30:03.802 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:30:03.802 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:03.802 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 44536 2024-09-14T06:30:03.802 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:03.802 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:30:03.802 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:03.802 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:30:03.802 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861384+0000", 2024-09-14T06:30:03.802 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:03.802 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:03.803 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:03.803 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:30:03.803 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861384+0000", 2024-09-14T06:30:03.803 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:03.803 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:03.803 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:03.803 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:30:03.803 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861384+0000", 2024-09-14T06:30:03.803 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.4000000000000002e-07 2024-09-14T06:30:03.803 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:03.803 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:03.803 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:30:03.803 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861386+0000", 2024-09-14T06:30:03.803 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.891e-06 2024-09-14T06:30:03.804 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:03.804 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:03.804 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:30:03.804 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861392+0000", 2024-09-14T06:30:03.804 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.9939999999999997e-06 2024-09-14T06:30:03.804 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:03.804 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:03.804 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:30:03.804 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861398+0000", 2024-09-14T06:30:03.804 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.507e-06 2024-09-14T06:30:03.804 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:03.804 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:03.804 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:30:03.805 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.924149+0000", 2024-09-14T06:30:03.805 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.062751135999999999 2024-09-14T06:30:03.805 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:03.805 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:03.805 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:30:03.805 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.924226+0000", 2024-09-14T06:30:03.805 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.7517999999999999e-05 2024-09-14T06:30:03.805 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:03.805 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:03.805 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:30:03.805 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.924442+0000", 2024-09-14T06:30:03.805 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000216042 2024-09-14T06:30:03.805 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:03.806 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:03.806 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:03.806 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:03.806 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:03.806 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:44537 3.2 3:4190d882:::rbd_data.112dca2f6ae1.0000000000000002:17 [read 479232~4096 [fadvise_nocache]] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:30:03.806 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:43.861405+0000", 2024-09-14T06:30:03.806 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 19.865472538999999, 2024-09-14T06:30:03.806 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.063351795000000002, 2024-09-14T06:30:03.806 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:30:03.806 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:30:03.806 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:30:03.807 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:30:03.807 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:30:03.807 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:03.807 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 44537 2024-09-14T06:30:03.807 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:03.807 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:30:03.807 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:03.807 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:30:03.807 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861405+0000", 2024-09-14T06:30:03.807 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:03.807 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:03.807 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:03.807 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:30:03.807 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861405+0000", 2024-09-14T06:30:03.808 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:03.808 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:03.808 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:03.808 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:30:03.808 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861405+0000", 2024-09-14T06:30:03.808 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.9099999999999999e-07 2024-09-14T06:30:03.808 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:03.808 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:03.808 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:30:03.808 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861407+0000", 2024-09-14T06:30:03.808 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.075e-06 2024-09-14T06:30:03.808 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:03.808 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:03.809 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:30:03.809 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861414+0000", 2024-09-14T06:30:03.809 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.4659999999999996e-06 2024-09-14T06:30:03.809 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:03.809 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:03.809 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:30:03.809 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861418+0000", 2024-09-14T06:30:03.809 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.5519999999999998e-06 2024-09-14T06:30:03.809 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:03.809 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:03.809 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:30:03.809 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.924474+0000", 2024-09-14T06:30:03.809 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.063056012999999994 2024-09-14T06:30:03.810 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:03.810 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:03.810 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:30:03.810 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.924556+0000", 2024-09-14T06:30:03.810 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.1181999999999996e-05 2024-09-14T06:30:03.810 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:03.810 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:03.810 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:30:03.810 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.924757+0000", 2024-09-14T06:30:03.810 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00020111599999999999 2024-09-14T06:30:03.810 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:03.810 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:03.810 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:03.811 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:03.811 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:03.811 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:44538 3.2 3:4190d882:::rbd_data.112dca2f6ae1.0000000000000002:17 [read 503808~4096 [fadvise_nocache]] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:30:03.811 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:43.861425+0000", 2024-09-14T06:30:03.811 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 19.865452498, 2024-09-14T06:30:03.811 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.063627125000000007, 2024-09-14T06:30:03.811 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:30:03.811 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:30:03.811 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:30:03.811 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:30:03.811 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:30:03.811 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:03.811 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 44538 2024-09-14T06:30:03.811 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:03.812 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:30:03.812 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:03.812 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:30:03.812 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861425+0000", 2024-09-14T06:30:03.812 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:03.812 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:03.812 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:03.812 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:30:03.812 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861425+0000", 2024-09-14T06:30:03.812 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:03.812 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:03.812 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:03.813 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:30:03.813 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861425+0000", 2024-09-14T06:30:03.813 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.9999999999999998e-07 2024-09-14T06:30:03.813 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:03.813 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:03.813 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:30:03.813 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861427+0000", 2024-09-14T06:30:03.813 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.9829999999999999e-06 2024-09-14T06:30:03.813 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:03.813 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:03.813 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:30:03.813 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861433+0000", 2024-09-14T06:30:03.813 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.2889999999999997e-06 2024-09-14T06:30:03.813 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:03.814 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:03.814 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:30:03.814 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861454+0000", 2024-09-14T06:30:03.814 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.1844999999999998e-05 2024-09-14T06:30:03.814 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:03.814 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:03.814 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:30:03.814 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.924786+0000", 2024-09-14T06:30:03.814 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.063331497 2024-09-14T06:30:03.814 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:03.814 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:03.814 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:30:03.814 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.924864+0000", 2024-09-14T06:30:03.814 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.7744000000000006e-05 2024-09-14T06:30:03.814 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:03.815 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:03.815 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:30:03.815 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.925052+0000", 2024-09-14T06:30:03.815 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00018836699999999999 2024-09-14T06:30:03.815 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:03.815 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:03.815 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:03.815 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:03.815 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:03.815 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:44539 3.2 3:4190d882:::rbd_data.112dca2f6ae1.0000000000000002:17 [read 528384~4096 [fadvise_nocache]] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:30:03.815 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:43.861462+0000", 2024-09-14T06:30:03.815 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 19.865415170999999, 2024-09-14T06:30:03.815 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.063855705999999998, 2024-09-14T06:30:03.815 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:30:03.815 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:30:03.816 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:30:03.816 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:30:03.816 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:30:03.816 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:03.816 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 44539 2024-09-14T06:30:03.816 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:03.816 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:30:03.816 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:03.816 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:30:03.816 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861462+0000", 2024-09-14T06:30:03.816 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:03.816 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:03.817 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:03.817 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:30:03.817 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861462+0000", 2024-09-14T06:30:03.817 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:03.817 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:03.817 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:03.817 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:30:03.817 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861463+0000", 2024-09-14T06:30:03.817 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.5200000000000002e-07 2024-09-14T06:30:03.817 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:03.817 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:03.817 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:30:03.818 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861465+0000", 2024-09-14T06:30:03.818 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.5189999999999999e-06 2024-09-14T06:30:03.818 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:03.818 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:03.818 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:30:03.818 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861471+0000", 2024-09-14T06:30:03.818 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.6899999999999997e-06 2024-09-14T06:30:03.818 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:03.818 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:03.818 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:30:03.818 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861475+0000", 2024-09-14T06:30:03.818 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.5730000000000001e-06 2024-09-14T06:30:03.819 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:03.819 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:03.819 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:30:03.819 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.925085+0000", 2024-09-14T06:30:03.819 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.063609902999999995 2024-09-14T06:30:03.819 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:03.819 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:03.819 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:30:03.820 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.925145+0000", 2024-09-14T06:30:03.820 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.9383999999999999e-05 2024-09-14T06:30:03.820 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:03.820 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:03.820 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:30:03.820 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.925318+0000", 2024-09-14T06:30:03.820 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00017318499999999999 2024-09-14T06:30:03.820 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:03.820 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:03.820 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:03.820 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:03.820 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:03.820 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:44540 3.2 3:4190d882:::rbd_data.112dca2f6ae1.0000000000000002:17 [read 552960~4096 [fadvise_nocache]] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:30:03.821 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:43.861483+0000", 2024-09-14T06:30:03.821 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 19.865394739999999, 2024-09-14T06:30:03.821 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.064065681999999999, 2024-09-14T06:30:03.821 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:30:03.821 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:30:03.821 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:30:03.821 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:30:03.821 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:30:03.821 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:03.821 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 44540 2024-09-14T06:30:03.821 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:03.821 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:30:03.822 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:03.822 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:30:03.822 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861483+0000", 2024-09-14T06:30:03.822 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:03.822 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:03.822 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:03.822 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:30:03.822 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861483+0000", 2024-09-14T06:30:03.822 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:03.822 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:03.822 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:03.822 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:30:03.822 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.867079+0000", 2024-09-14T06:30:03.822 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0055966480000000001 2024-09-14T06:30:03.823 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:03.823 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:03.823 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:30:03.823 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.867086+0000", 2024-09-14T06:30:03.823 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.387e-06 2024-09-14T06:30:03.823 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:03.823 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:03.823 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:30:03.823 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.867094+0000", 2024-09-14T06:30:03.823 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.9419999999999995e-06 2024-09-14T06:30:03.823 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:03.823 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:03.823 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:30:03.823 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.867109+0000", 2024-09-14T06:30:03.824 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.5818999999999999e-05 2024-09-14T06:30:03.824 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:03.824 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:03.824 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:30:03.824 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.925354+0000", 2024-09-14T06:30:03.824 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.058244047 2024-09-14T06:30:03.824 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:03.824 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:03.824 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:30:03.824 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.925406+0000", 2024-09-14T06:30:03.824 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.2661999999999998e-05 2024-09-14T06:30:03.824 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:03.825 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:03.825 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:30:03.825 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.925548+0000", 2024-09-14T06:30:03.825 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000142177 2024-09-14T06:30:03.825 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:03.825 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:03.825 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:03.825 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:03.825 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:03.825 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-14T06:30:03.827 INFO:teuthology.orchestra.run.smithi094.stdout:2195 write 0xc567ce7 thru 0xc56e4a3 (0x67bd bytes) 2024-09-14T06:30:03.827 INFO:tasks.ceph.osd.4.smithi138.stderr:2024-09-14T06:30:03.825+0000 7f6c653c8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33615) UID: 0 2024-09-14T06:30:03.828 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-14T06:30:03.831 INFO:teuthology.orchestra.run.smithi094.stdout:2196 read 0x2800c07 thru 0x2807c22 (0x701c bytes) 2024-09-14T06:30:03.831 INFO:teuthology.orchestra.run.smithi094.stdout:2198 writesame 0xc409d4 thru 0xc45d92 data_size 0x1(0x53bf bytes) 2024-09-14T06:30:03.831 INFO:teuthology.orchestra.run.smithi094.stdout:2199 write 0x97719e9 thru 0x977b0b5 (0x96cd bytes) 2024-09-14T06:30:03.832 INFO:teuthology.orchestra.run.smithi094.stdout:2200 write 0x6d3a116 thru 0x6d3b387 (0x1272 bytes) 2024-09-14T06:30:03.832 INFO:teuthology.orchestra.run.smithi094.stdout:2201 write 0x3a55485 thru 0x3a597e5 (0x4361 bytes) 2024-09-14T06:30:03.832 INFO:teuthology.orchestra.run.smithi094.stdout:2202 write 0xc601422 thru 0xc6054da (0x40b9 bytes) 2024-09-14T06:30:03.832 INFO:teuthology.orchestra.run.smithi094.stdout:2203 read 0xa3d4d61 thru 0xa3d88e2 (0x3b82 bytes) 2024-09-14T06:30:03.832 INFO:teuthology.orchestra.run.smithi094.stdout:2204 read 0x830f631 thru 0x8318ff6 (0x99c6 bytes) 2024-09-14T06:30:03.833 INFO:teuthology.orchestra.run.smithi094.stdout:2205 read 0x1f9a3e5 thru 0x1f9e6ea (0x4306 bytes) 2024-09-14T06:30:03.833 INFO:teuthology.orchestra.run.smithi094.stdout:2206 write 0xaad7453 thru 0xaad84aa (0x1058 bytes) 2024-09-14T06:30:03.833 INFO:teuthology.orchestra.run.smithi094.stdout:2209 read 0x6f75c05 thru 0x6f84190 (0xe58c bytes) 2024-09-14T06:30:03.833 INFO:teuthology.orchestra.run.smithi094.stdout:2210 write 0x2211b11 thru 0x2213bc7 (0x20b7 bytes) 2024-09-14T06:30:03.833 INFO:teuthology.orchestra.run.smithi094.stdout:2211 read 0x4cf8792 thru 0x4cffde8 (0x7657 bytes) 2024-09-14T06:30:03.834 INFO:teuthology.orchestra.run.smithi094.stdout:2212 write 0xdeb7c16 thru 0xdec10f8 (0x94e3 bytes) 2024-09-14T06:30:03.834 INFO:teuthology.orchestra.run.smithi094.stdout:2213 read 0xb3521d5 thru 0xb35ac2d (0x8a59 bytes) 2024-09-14T06:30:03.834 INFO:teuthology.orchestra.run.smithi094.stdout:2214 trunc from 0xe9af85b to 0x52d55f1 2024-09-14T06:30:03.834 INFO:teuthology.orchestra.run.smithi094.stdout:2215 trunc from 0x52d55f1 to 0x2573bef 2024-09-14T06:30:03.834 INFO:teuthology.orchestra.run.smithi094.stdout:2216 write 0x5d7cdf3 thru 0x5d885e6 (0xb7f4 bytes) 2024-09-14T06:30:03.834 INFO:teuthology.orchestra.run.smithi094.stdout:2217 write 0x5013e55 thru 0x5018d3f (0x4eeb bytes) 2024-09-14T06:30:03.834 INFO:teuthology.orchestra.run.smithi094.stdout:2218 read 0x677d01 thru 0x67ecbc (0x6fbc bytes) 2024-09-14T06:30:03.834 INFO:teuthology.orchestra.run.smithi094.stderr:nodeep-scrub is set 2024-09-14T06:30:03.835 INFO:teuthology.orchestra.run.smithi094.stderr:reweighted osd.5 to 1 (10000) 2024-09-14T06:30:03.835 INFO:teuthology.orchestra.run.smithi094.stdout:2221 write 0x67d1fa1 thru 0x67e150a (0xf56a bytes) 2024-09-14T06:30:03.836 INFO:teuthology.orchestra.run.smithi094.stdout:2222 writesame 0x24f192a thru 0x24fe858 data_size 0x1(0xcf2f bytes) 2024-09-14T06:30:03.837 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 6 1 2024-09-14T06:30:03.854 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-14T06:30:03.854 INFO:teuthology.orchestra.run.smithi094.stdout:2225 read 0x43de439 thru 0x43e04c6 (0x208e bytes) 2024-09-14T06:30:03.864 INFO:teuthology.orchestra.run.smithi094.stdout:2227 writesame 0x89eed64 thru 0x89f3183 data_size 0x50(0x4420 bytes) 2024-09-14T06:30:03.873 INFO:teuthology.orchestra.run.smithi094.stdout:2228 write 0xa9db334 thru 0xa9decef (0x39bc bytes) 2024-09-14T06:30:03.875 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-14T06:30:03.875 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-14T06:30:03.875 INFO:teuthology.orchestra.run.smithi094.stdout:2232 punch from 0x225b546 to 0x2267369, (0xbe23 bytes) 2024-09-14T06:30:03.875 INFO:teuthology.orchestra.run.smithi094.stdout:2233 punch from 0x1af8b40 to 0x1b00b07, (0x7fc7 bytes) 2024-09-14T06:30:03.875 INFO:teuthology.orchestra.run.smithi094.stdout:2234 punch from 0x81ccf65 to 0x81cf5a3, (0x263e bytes) 2024-09-14T06:30:03.881 INFO:teuthology.orchestra.run.smithi094.stdout:2235 write 0xb36927a thru 0xb3725c0 (0x9347 bytes) 2024-09-14T06:30:03.882 INFO:teuthology.orchestra.run.smithi094.stdout:2236 read 0x4137793 thru 0x4140a04 (0x9272 bytes) 2024-09-14T06:30:03.883 INFO:teuthology.orchestra.run.smithi094.stdout:2237 writesame 0xb35fee7 thru 0xb36eb8a data_size 0x41(0xeca4 bytes) 2024-09-14T06:30:03.884 INFO:teuthology.orchestra.run.smithi094.stdout:2238 write 0x9b166c0 thru 0x9b169a0 (0x2e1 bytes) 2024-09-14T06:30:03.885 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-14T06:30:03.890 INFO:teuthology.orchestra.run.smithi094.stdout:2241 trunc from 0xb3725c1 to 0xec0b3c5 2024-09-14T06:30:03.905 INFO:teuthology.orchestra.run.smithi094.stdout:2242 trunc from 0xec0b3c5 to 0x6158f2f 2024-09-14T06:30:03.914 INFO:teuthology.orchestra.run.smithi094.stdout:2243 trunc from 0x6158f2f to 0x5aa665d 2024-09-14T06:30:03.921 INFO:teuthology.orchestra.run.smithi094.stdout:2245 write 0x1320023 thru 0x132d036 (0xd014 bytes) 2024-09-14T06:30:03.923 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-14T06:30:03.923 INFO:teuthology.orchestra.run.smithi094.stdout:2248 write 0x4b2ec95 thru 0x4b35d81 (0x70ed bytes) 2024-09-14T06:30:03.928 INFO:tasks.ceph.osd.6.smithi138.stderr:2024-09-14T06:30:03.927+0000 7f8cd8849640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33620) UID: 0 2024-09-14T06:30:03.930 INFO:teuthology.orchestra.run.smithi094.stdout:2249 write 0x64508f9 thru 0x6457d7a (0x7482 bytes) 2024-09-14T06:30:03.931 INFO:teuthology.orchestra.run.smithi094.stdout:2250 read 0x21e1744 thru 0x21e7b80 (0x643d bytes) 2024-09-14T06:30:03.937 INFO:teuthology.orchestra.run.smithi094.stdout:2251 trunc from 0x6457d7b to 0x20da746 2024-09-14T06:30:03.946 INFO:teuthology.orchestra.run.smithi138.stdout:{ 2024-09-14T06:30:03.946 INFO:teuthology.orchestra.run.smithi138.stdout: "ops": [], 2024-09-14T06:30:03.946 INFO:teuthology.orchestra.run.smithi138.stdout: "num_ops": 0 2024-09-14T06:30:03.946 INFO:teuthology.orchestra.run.smithi138.stdout:} 2024-09-14T06:30:03.955 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-14T06:30:03.965 INFO:teuthology.orchestra.run.smithi094.stdout:2252 writesame 0xcd86020 thru 0xcd8fcac data_size 0x49(0x9c8d bytes) 2024-09-14T06:30:03.966 INFO:teuthology.orchestra.run.smithi094.stdout:2253 read 0x47d433e thru 0x47d7d86 (0x3a49 bytes) 2024-09-14T06:30:03.967 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-14T06:30:03.972 INFO:teuthology.orchestra.run.smithi094.stdout:2255 writesame 0xd2cab20 thru 0xd2ce01b data_size 0x2(0x34fc bytes) 2024-09-14T06:30:03.973 INFO:teuthology.orchestra.run.smithi094.stdout:2256 read 0x96e67b9 thru 0x96ebafc (0x5344 bytes) 2024-09-14T06:30:03.974 INFO:teuthology.orchestra.run.smithi094.stdout:2257 writesame 0xc70e10c thru 0xc7124d6 data_size 0xf(0x43cb bytes) 2024-09-14T06:30:03.975 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-14T06:30:03.975 INFO:teuthology.orchestra.run.smithi094.stdout:2260 punch from 0x4b86484 to 0x4b879ec, (0x1568 bytes) 2024-09-14T06:30:03.976 INFO:teuthology.orchestra.run.smithi094.stdout:2261 read 0x7be2255 thru 0x7be779b (0x5547 bytes) 2024-09-14T06:30:03.976 INFO:teuthology.orchestra.run.smithi094.stdout:2262 read 0x989e15b thru 0x989e559 (0x3ff bytes) 2024-09-14T06:30:03.976 INFO:teuthology.orchestra.run.smithi094.stdout:2265 write 0xab358b0 thru 0xab40963 (0xb0b4 bytes) 2024-09-14T06:30:03.977 INFO:teuthology.orchestra.run.smithi094.stdout:2266 punch from 0xcc39ee to 0xcc77e5, (0x3df7 bytes) 2024-09-14T06:30:03.978 INFO:teuthology.orchestra.run.smithi094.stdout:2269 read 0x5d727a2 thru 0x5d74ad4 (0x2333 bytes) 2024-09-14T06:30:03.978 INFO:teuthology.orchestra.run.smithi094.stdout:2270 write 0x4355973 thru 0x435d04f (0x76dd bytes) 2024-09-14T06:30:03.979 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-14T06:30:03.979 INFO:teuthology.orchestra.run.smithi094.stdout:2272 writesame 0xd170822 thru 0xd17188f data_size 0x3(0x106e bytes) 2024-09-14T06:30:03.981 INFO:teuthology.orchestra.run.smithi094.stdout:2274 write 0xbb06098 thru 0xbb0837a (0x22e3 bytes) 2024-09-14T06:30:03.982 INFO:teuthology.orchestra.run.smithi094.stdout:2277 write 0xb410a7c thru 0xb418f5c (0x84e1 bytes) 2024-09-14T06:30:03.984 INFO:teuthology.orchestra.run.smithi094.stdout:2278 trunc from 0xd2ce01c to 0xc7475c7 2024-09-14T06:30:03.994 INFO:teuthology.orchestra.run.smithi094.stdout:2279 punch from 0x5564e47 to 0x5574d32, (0xfeeb bytes) 2024-09-14T06:30:03.994 INFO:teuthology.orchestra.run.smithi094.stdout:2280 write 0x642ce90 thru 0x643960c (0xc77d bytes) 2024-09-14T06:30:03.995 INFO:teuthology.orchestra.run.smithi094.stdout:2281 punch from 0x36cb720 to 0x36d8f11, (0xd7f1 bytes) 2024-09-14T06:30:03.996 INFO:teuthology.orchestra.run.smithi094.stdout:2284 writesame 0x908b3d3 thru 0x9094635 data_size 0x5(0x9263 bytes) 2024-09-14T06:30:03.999 INFO:teuthology.orchestra.run.smithi094.stdout:2285 punch from 0x5781aad to 0x57849b1, (0x2f04 bytes) 2024-09-14T06:30:03.999 INFO:teuthology.orchestra.run.smithi094.stdout:2286 writesame 0x32450b8 thru 0x324b588 data_size 0x7(0x64d1 bytes) 2024-09-14T06:30:04.002 INFO:teuthology.orchestra.run.smithi094.stdout:2287 write 0x6ebf60 thru 0x6eeab2 (0x2b53 bytes) 2024-09-14T06:30:04.003 INFO:teuthology.orchestra.run.smithi094.stdout:2290 write 0x8e4e338 thru 0x8e5db1f (0xf7e8 bytes) 2024-09-14T06:30:04.005 INFO:teuthology.orchestra.run.smithi094.stdout:2291 read 0x57e99c3 thru 0x57f85e4 (0xec22 bytes) 2024-09-14T06:30:04.013 INFO:teuthology.orchestra.run.smithi094.stdout:2292 write 0xecda496 thru 0xece50a1 (0xac0c bytes) 2024-09-14T06:30:04.018 INFO:teuthology.orchestra.run.smithi094.stdout:2293 trunc from 0xece50a2 to 0xc607564 2024-09-14T06:30:04.026 INFO:teuthology.orchestra.run.smithi094.stdout:2295 writesame 0x90c7960 thru 0x90d6256 data_size 0x1(0xe8f7 bytes) 2024-09-14T06:30:04.029 INFO:tasks.ceph.osd.3.smithi094.stderr:2024-09-14T06:30:04.028+0000 7feeeafe8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33655) UID: 0 2024-09-14T06:30:04.052 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-14T06:30:04.060 INFO:teuthology.orchestra.run.smithi094.stdout:2297 trunc from 0xc607564 to 0x5dfe13b 2024-09-14T06:30:04.067 INFO:teuthology.orchestra.run.smithi094.stdout:2298 writesame 0x59f7c07 thru 0x59f87aa data_size 0xa(0xba4 bytes) 2024-09-14T06:30:04.069 INFO:teuthology.orchestra.run.smithi094.stdout:2299 read 0x180077d thru 0x180b200 (0xaa84 bytes) 2024-09-14T06:30:04.069 INFO:teuthology.orchestra.run.smithi094.stdout:2301 read 0x1d06af thru 0x1dc113 (0xba65 bytes) 2024-09-14T06:30:04.076 INFO:teuthology.orchestra.run.smithi094.stdout:2302 writesame 0x67bff64 thru 0x67cd9f1 data_size 0x4b(0xda8e bytes) 2024-09-14T06:30:04.090 INFO:teuthology.orchestra.run.smithi094.stdout:2303 write 0xd335291 thru 0xd340b42 (0xb8b2 bytes) 2024-09-14T06:30:04.092 INFO:teuthology.orchestra.run.smithi094.stdout:2304 write 0x77e39c7 thru 0x77f1998 (0xdfd2 bytes) 2024-09-14T06:30:04.093 INFO:teuthology.orchestra.run.smithi094.stdout:2305 write 0x1c6f414 thru 0x1c78ebf (0x9aac bytes) 2024-09-14T06:30:04.101 INFO:teuthology.orchestra.run.smithi094.stdout:2307 write 0xe0ceeb1 thru 0xe0da94a (0xba9a bytes) 2024-09-14T06:30:04.102 INFO:teuthology.orchestra.run.smithi094.stdout:2308 read 0xa0d5bd6 thru 0xa0e4d0a (0xf135 bytes) 2024-09-14T06:30:04.104 INFO:teuthology.orchestra.run.smithi138.stdout:{ 2024-09-14T06:30:04.104 INFO:teuthology.orchestra.run.smithi138.stdout: "ops": [], 2024-09-14T06:30:04.104 INFO:teuthology.orchestra.run.smithi138.stdout: "complaint_time": 30, 2024-09-14T06:30:04.104 INFO:teuthology.orchestra.run.smithi138.stdout: "num_blocked_ops": 0 2024-09-14T06:30:04.104 INFO:teuthology.orchestra.run.smithi138.stdout:} 2024-09-14T06:30:04.113 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-14T06:30:04.115 INFO:teuthology.orchestra.run.smithi094.stdout:2310 trunc from 0xe0da94b to 0x3d4eb28 2024-09-14T06:30:04.122 INFO:teuthology.orchestra.run.smithi094.stdout:2312 read 0x3eb20c thru 0x3f18f5 (0x66ea bytes) 2024-09-14T06:30:04.123 INFO:teuthology.orchestra.run.smithi094.stdout:2313 read 0x255791b thru 0x255a456 (0x2b3c bytes) 2024-09-14T06:30:04.123 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-14T06:30:04.135 INFO:teuthology.orchestra.run.smithi094.stdout:2319 writesame 0x9830c0e thru 0x983d485 data_size 0x14(0xc878 bytes) 2024-09-14T06:30:04.137 INFO:teuthology.orchestra.run.smithi094.stdout:2320 writesame 0xf228d4 thru 0xf31ba4 data_size 0x1(0xf2d1 bytes) 2024-09-14T06:30:04.158 INFO:tasks.ceph.osd.6.smithi138.stderr:2024-09-14T06:30:04.157+0000 7f8cd8849640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33620) UID: 0 2024-09-14T06:30:04.159 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-14T06:30:04.160 INFO:teuthology.orchestra.run.smithi094.stdout:2323 read 0x7549413 thru 0x7550f63 (0x7b51 bytes) 2024-09-14T06:30:04.160 INFO:teuthology.orchestra.run.smithi094.stdout:2324 read 0x53f1cee thru 0x53fe799 (0xcaac bytes) 2024-09-14T06:30:04.167 INFO:teuthology.orchestra.run.smithi094.stdout:2325 writesame 0x9fc1519 thru 0x9fcb2f0 data_size 0x4(0x9dd8 bytes) 2024-09-14T06:30:04.175 INFO:teuthology.orchestra.run.smithi094.stdout:2326 write 0xba8a2a1 thru 0xba95bb0 (0xb910 bytes) 2024-09-14T06:30:04.177 INFO:teuthology.orchestra.run.smithi094.stdout:2327 writesame 0x1fc427b thru 0x1fcbab0 data_size 0x2e(0x7836 bytes) 2024-09-14T06:30:04.179 INFO:teuthology.orchestra.run.smithi094.stdout:2328 write 0xa76cf73 thru 0xa77bfb0 (0xf03e bytes) 2024-09-14T06:30:04.180 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-14T06:30:04.180 INFO:teuthology.orchestra.run.smithi094.stdout:2330 read 0x12c3e9c thru 0x12ce594 (0xa6f9 bytes) 2024-09-14T06:30:04.195 INFO:teuthology.orchestra.run.smithi094.stdout:2332 trunc from 0xba95bb1 to 0x163cae0 2024-09-14T06:30:04.203 INFO:teuthology.orchestra.run.smithi094.stdout:2333 read 0x760e79 thru 0x766536 (0x56be bytes) 2024-09-14T06:30:04.203 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-14T06:30:04.215 INFO:teuthology.orchestra.run.smithi094.stdout:2335 trunc from 0x163cae0 to 0xb18b85f 2024-09-14T06:30:04.220 INFO:teuthology.orchestra.run.smithi094.stdout:2336 trunc from 0xb18b85f to 0xade78bd 2024-09-14T06:30:04.222 INFO:teuthology.orchestra.run.smithi138.stdout:{ 2024-09-14T06:30:04.223 INFO:teuthology.orchestra.run.smithi138.stdout: "size": 20, 2024-09-14T06:30:04.223 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 600, 2024-09-14T06:30:04.223 INFO:teuthology.orchestra.run.smithi138.stdout: "ops": [ 2024-09-14T06:30:04.223 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:04.223 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:19328 3.3 3:c34e5e51:::rbd_data.112d92027590.0000000000000024:f [read 1926144~1024 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:30:04.223 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:27.059768+0000", 2024-09-14T06:30:04.223 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 37.161220139000001, 2024-09-14T06:30:04.223 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046369147999999999, 2024-09-14T06:30:04.223 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:30:04.223 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:30:04.223 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:30:04.223 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:30:04.223 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:30:04.224 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:04.224 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 19328 2024-09-14T06:30:04.224 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:04.224 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:30:04.224 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:04.224 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:30:04.224 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.059768+0000", 2024-09-14T06:30:04.224 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:04.224 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:04.224 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:04.224 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:30:04.224 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.059768+0000", 2024-09-14T06:30:04.224 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:04.224 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:04.225 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:04.225 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:30:04.225 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.059874+0000", 2024-09-14T06:30:04.225 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00010551400000000001 2024-09-14T06:30:04.225 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:04.225 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:04.225 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:30:04.225 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.059877+0000", 2024-09-14T06:30:04.225 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.8279999999999999e-06 2024-09-14T06:30:04.225 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:04.225 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:04.225 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:30:04.225 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.059884+0000", 2024-09-14T06:30:04.225 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.6389999999999992e-06 2024-09-14T06:30:04.226 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:04.226 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:04.226 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:30:04.226 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.059890+0000", 2024-09-14T06:30:04.226 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.49e-06 2024-09-14T06:30:04.226 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:04.226 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:04.226 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:30:04.226 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.105804+0000", 2024-09-14T06:30:04.227 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.045914242000000001 2024-09-14T06:30:04.227 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:04.227 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:04.227 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:30:04.227 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.105886+0000", 2024-09-14T06:30:04.227 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.2305999999999994e-05 2024-09-14T06:30:04.227 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:04.228 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:04.228 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:30:04.228 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.106138+0000", 2024-09-14T06:30:04.228 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.000251129 2024-09-14T06:30:04.228 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:04.228 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:30:04.228 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:04.228 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:04.228 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:04.228 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:19329 3.3 3:c34e5e51:::rbd_data.112d92027590.0000000000000024:f [read 1936384~1024 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:30:04.228 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:27.059899+0000", 2024-09-14T06:30:04.228 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 37.161089615999998, 2024-09-14T06:30:04.228 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046572122000000001, 2024-09-14T06:30:04.229 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:30:04.229 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:30:04.229 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:30:04.229 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:30:04.229 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:30:04.229 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:04.229 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 19329 2024-09-14T06:30:04.229 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:04.229 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:30:04.229 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:04.229 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:30:04.229 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.059899+0000", 2024-09-14T06:30:04.229 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:04.230 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:04.230 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:04.230 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:30:04.230 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.059899+0000", 2024-09-14T06:30:04.230 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:04.230 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:04.230 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:04.230 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:30:04.230 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062501+0000", 2024-09-14T06:30:04.230 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.0026016419999999999 2024-09-14T06:30:04.230 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:04.230 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:04.231 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:30:04.231 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062514+0000", 2024-09-14T06:30:04.231 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.2947e-05 2024-09-14T06:30:04.231 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:04.231 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:04.231 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:30:04.231 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062523+0000", 2024-09-14T06:30:04.231 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.5519999999999993e-06 2024-09-14T06:30:04.231 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:04.231 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:04.232 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:30:04.232 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062534+0000", 2024-09-14T06:30:04.232 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.1416e-05 2024-09-14T06:30:04.232 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:04.232 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:04.232 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:30:04.232 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.106179+0000", 2024-09-14T06:30:04.233 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.043644049999999997 2024-09-14T06:30:04.233 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:04.233 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:04.233 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:30:04.233 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.106262+0000", 2024-09-14T06:30:04.233 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.3178999999999998e-05 2024-09-14T06:30:04.233 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:04.233 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:04.233 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:30:04.233 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.106471+0000", 2024-09-14T06:30:04.233 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00020933599999999999 2024-09-14T06:30:04.233 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:04.233 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:30:04.234 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:04.234 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:04.234 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:04.234 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:19351 3.3 3:c34e5e51:::rbd_data.112d92027590.0000000000000024:f [read 1159168~1024 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:30:04.234 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:27.062766+0000", 2024-09-14T06:30:04.234 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 37.158222391000002, 2024-09-14T06:30:04.234 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046099454999999998, 2024-09-14T06:30:04.234 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:30:04.234 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:30:04.234 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:30:04.234 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:30:04.234 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:30:04.234 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:04.234 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 19351 2024-09-14T06:30:04.235 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:04.235 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:30:04.235 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:04.235 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:30:04.235 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062766+0000", 2024-09-14T06:30:04.235 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:04.235 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:04.235 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:04.235 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:30:04.235 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062766+0000", 2024-09-14T06:30:04.235 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:04.235 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:04.235 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:04.235 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:30:04.236 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062766+0000", 2024-09-14T06:30:04.236 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.6e-07 2024-09-14T06:30:04.236 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:04.236 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:04.236 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:30:04.236 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062767+0000", 2024-09-14T06:30:04.236 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.0070000000000001e-06 2024-09-14T06:30:04.236 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:04.236 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:04.236 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:30:04.236 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062785+0000", 2024-09-14T06:30:04.236 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.7884999999999999e-05 2024-09-14T06:30:04.236 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:04.237 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:04.237 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:30:04.237 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062790+0000", 2024-09-14T06:30:04.237 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.0900000000000004e-06 2024-09-14T06:30:04.237 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:04.237 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:04.237 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:30:04.237 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.108735+0000", 2024-09-14T06:30:04.237 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.045944411999999997 2024-09-14T06:30:04.237 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:04.237 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:04.237 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:30:04.237 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.108809+0000", 2024-09-14T06:30:04.237 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.4157000000000002e-05 2024-09-14T06:30:04.238 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:04.238 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:04.238 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:30:04.238 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.108866+0000", 2024-09-14T06:30:04.238 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.6644000000000002e-05 2024-09-14T06:30:04.238 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:04.238 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:30:04.238 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:04.238 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:04.238 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:04.238 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:19352 3.3 3:c34e5e51:::rbd_data.112d92027590.0000000000000024:f [read 1169408~1024 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:30:04.238 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:27.062798+0000", 2024-09-14T06:30:04.238 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 37.158190980000001, 2024-09-14T06:30:04.239 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046255900000000003, 2024-09-14T06:30:04.239 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:30:04.239 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:30:04.239 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:30:04.239 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:30:04.239 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:30:04.239 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:04.239 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 19352 2024-09-14T06:30:04.239 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:04.239 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:30:04.239 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:04.240 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:30:04.240 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062798+0000", 2024-09-14T06:30:04.240 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:04.240 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:04.240 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:04.240 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:30:04.240 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062798+0000", 2024-09-14T06:30:04.240 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:04.240 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:04.240 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:04.240 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:30:04.241 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062798+0000", 2024-09-14T06:30:04.241 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.37e-07 2024-09-14T06:30:04.241 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:04.241 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:04.241 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:30:04.241 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062800+0000", 2024-09-14T06:30:04.241 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.0269999999999998e-06 2024-09-14T06:30:04.241 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:04.241 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:04.241 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:30:04.241 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062806+0000", 2024-09-14T06:30:04.241 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.6690000000000003e-06 2024-09-14T06:30:04.241 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:04.242 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:04.242 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:30:04.242 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062810+0000", 2024-09-14T06:30:04.242 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.3449999999999999e-06 2024-09-14T06:30:04.242 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:04.242 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:04.242 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:30:04.242 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.108892+0000", 2024-09-14T06:30:04.242 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046081715000000002 2024-09-14T06:30:04.242 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:04.242 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:04.242 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:30:04.242 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.108940+0000", 2024-09-14T06:30:04.242 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.7815999999999998e-05 2024-09-14T06:30:04.243 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:04.243 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:04.243 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:30:04.243 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.109053+0000", 2024-09-14T06:30:04.243 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.000113791 2024-09-14T06:30:04.243 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:04.243 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:30:04.243 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:04.243 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:04.243 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:04.243 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:19353 3.3 3:c34e5e51:::rbd_data.112d92027590.0000000000000024:f [read 1179648~1024 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:30:04.243 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:27.062816+0000", 2024-09-14T06:30:04.243 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 37.158172460999999, 2024-09-14T06:30:04.243 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046481256999999998, 2024-09-14T06:30:04.244 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:30:04.244 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:30:04.244 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:30:04.244 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:30:04.244 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:30:04.244 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:04.244 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 19353 2024-09-14T06:30:04.244 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:04.244 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:30:04.244 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:04.244 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:30:04.244 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062816+0000", 2024-09-14T06:30:04.244 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:04.244 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:04.245 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:04.245 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:30:04.245 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062816+0000", 2024-09-14T06:30:04.245 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:04.245 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:04.245 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:04.245 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:30:04.245 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062816+0000", 2024-09-14T06:30:04.245 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.2899999999999999e-07 2024-09-14T06:30:04.245 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:04.245 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:04.245 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:30:04.245 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062818+0000", 2024-09-14T06:30:04.245 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.699e-06 2024-09-14T06:30:04.246 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:04.246 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:04.246 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:30:04.246 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062825+0000", 2024-09-14T06:30:04.246 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.878e-06 2024-09-14T06:30:04.246 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:04.246 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:04.246 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:30:04.246 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062830+0000", 2024-09-14T06:30:04.246 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.5410000000000002e-06 2024-09-14T06:30:04.246 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:04.246 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:04.246 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:30:04.246 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.109096+0000", 2024-09-14T06:30:04.246 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046266517 2024-09-14T06:30:04.247 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:04.247 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:04.247 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:30:04.247 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.109190+0000", 2024-09-14T06:30:04.247 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.4000999999999997e-05 2024-09-14T06:30:04.247 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:04.247 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:04.247 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:30:04.247 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.109297+0000", 2024-09-14T06:30:04.247 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.000107292 2024-09-14T06:30:04.248 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:04.248 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:30:04.248 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:04.248 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:04.248 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:04.248 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:19354 3.3 3:c34e5e51:::rbd_data.112d92027590.0000000000000024:f [read 1189888~1024 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:30:04.248 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:27.062842+0000", 2024-09-14T06:30:04.248 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 37.158147006, 2024-09-14T06:30:04.248 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046594257, 2024-09-14T06:30:04.248 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:30:04.248 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:30:04.248 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:30:04.248 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:30:04.249 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:30:04.249 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:04.249 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 19354 2024-09-14T06:30:04.249 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:04.249 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:30:04.249 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:04.249 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:30:04.249 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062842+0000", 2024-09-14T06:30:04.249 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:04.249 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:04.249 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:04.249 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:30:04.249 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062842+0000", 2024-09-14T06:30:04.249 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:04.250 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:04.250 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:04.250 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:30:04.250 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062842+0000", 2024-09-14T06:30:04.250 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.7399999999999999e-07 2024-09-14T06:30:04.250 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:04.250 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:04.250 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:30:04.250 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062859+0000", 2024-09-14T06:30:04.250 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.7010000000000001e-05 2024-09-14T06:30:04.250 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:04.250 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:04.250 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:30:04.250 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062866+0000", 2024-09-14T06:30:04.251 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.9990000000000002e-06 2024-09-14T06:30:04.251 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:04.251 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:04.251 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:30:04.251 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062873+0000", 2024-09-14T06:30:04.251 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.8569999999999997e-06 2024-09-14T06:30:04.251 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:04.251 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:04.251 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:30:04.251 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.109335+0000", 2024-09-14T06:30:04.251 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046461943999999998 2024-09-14T06:30:04.251 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:04.251 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:04.251 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:30:04.252 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.109377+0000", 2024-09-14T06:30:04.252 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.2558999999999997e-05 2024-09-14T06:30:04.252 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:04.252 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:04.252 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:30:04.252 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.109436+0000", 2024-09-14T06:30:04.252 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.8514000000000003e-05 2024-09-14T06:30:04.252 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:04.252 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:30:04.252 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:04.252 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:04.253 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:04.253 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:19355 3.3 3:c34e5e51:::rbd_data.112d92027590.0000000000000024:f [read 1200128~1024 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:30:04.253 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:27.062884+0000", 2024-09-14T06:30:04.253 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 37.158104332999997, 2024-09-14T06:30:04.253 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046681650999999998, 2024-09-14T06:30:04.253 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:30:04.253 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:30:04.253 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:30:04.253 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:30:04.253 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:30:04.253 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:04.253 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 19355 2024-09-14T06:30:04.253 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:04.254 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:30:04.254 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:04.254 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:30:04.254 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062884+0000", 2024-09-14T06:30:04.254 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:04.254 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:04.254 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:04.254 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:30:04.254 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062884+0000", 2024-09-14T06:30:04.254 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:04.254 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:04.254 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:04.254 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:30:04.254 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062885+0000", 2024-09-14T06:30:04.255 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.8000000000000001e-07 2024-09-14T06:30:04.255 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:04.255 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:04.255 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:30:04.255 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062886+0000", 2024-09-14T06:30:04.255 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.8920000000000001e-06 2024-09-14T06:30:04.255 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:04.255 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:04.255 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:30:04.255 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062892+0000", 2024-09-14T06:30:04.255 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.84e-06 2024-09-14T06:30:04.255 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:04.255 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:04.255 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:30:04.256 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062899+0000", 2024-09-14T06:30:04.256 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.5339999999999996e-06 2024-09-14T06:30:04.256 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:04.256 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:04.256 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:30:04.256 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.109466+0000", 2024-09-14T06:30:04.256 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046566919999999998 2024-09-14T06:30:04.256 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:04.256 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:04.256 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:30:04.256 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.109510+0000", 2024-09-14T06:30:04.256 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.4153000000000002e-05 2024-09-14T06:30:04.256 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:04.256 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:04.257 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:30:04.257 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.109566+0000", 2024-09-14T06:30:04.257 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.5931999999999999e-05 2024-09-14T06:30:04.257 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:04.257 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:30:04.257 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:04.257 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:04.257 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:04.257 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:19356 3.3 3:c34e5e51:::rbd_data.112d92027590.0000000000000024:f [read 1210368~1024 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:30:04.257 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:27.062908+0000", 2024-09-14T06:30:04.257 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 37.158080896000001, 2024-09-14T06:30:04.257 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046807498000000003, 2024-09-14T06:30:04.257 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:30:04.257 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:30:04.258 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:30:04.258 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:30:04.258 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:30:04.258 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:04.258 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 19356 2024-09-14T06:30:04.258 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:04.258 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:30:04.258 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:04.258 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:30:04.258 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062908+0000", 2024-09-14T06:30:04.258 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:04.258 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:04.258 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:04.258 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:30:04.259 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062908+0000", 2024-09-14T06:30:04.259 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:04.259 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:04.259 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:04.259 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:30:04.259 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062908+0000", 2024-09-14T06:30:04.259 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.5699999999999998e-07 2024-09-14T06:30:04.259 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:04.259 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:04.259 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:30:04.259 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062910+0000", 2024-09-14T06:30:04.259 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.7710000000000001e-06 2024-09-14T06:30:04.259 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:04.260 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:04.260 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:30:04.260 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062920+0000", 2024-09-14T06:30:04.260 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.0654e-05 2024-09-14T06:30:04.260 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:04.260 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:04.260 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:30:04.260 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062931+0000", 2024-09-14T06:30:04.260 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.075e-05 2024-09-14T06:30:04.260 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:04.260 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:04.260 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:30:04.260 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.109595+0000", 2024-09-14T06:30:04.260 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046663713000000002 2024-09-14T06:30:04.261 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:04.261 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:04.261 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:30:04.261 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.109632+0000", 2024-09-14T06:30:04.261 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.7505000000000001e-05 2024-09-14T06:30:04.261 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:04.261 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:04.261 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:30:04.261 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.109715+0000", 2024-09-14T06:30:04.261 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.2748000000000001e-05 2024-09-14T06:30:04.262 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:04.262 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:30:04.262 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:04.262 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:04.262 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:04.262 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:19357 3.3 3:c34e5e51:::rbd_data.112d92027590.0000000000000024:f [read 1220608~1024 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:30:04.262 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:27.062939+0000", 2024-09-14T06:30:04.262 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 37.158049448, 2024-09-14T06:30:04.262 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046920508, 2024-09-14T06:30:04.262 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:30:04.262 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:30:04.262 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:30:04.262 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:30:04.262 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:30:04.262 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:04.263 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 19357 2024-09-14T06:30:04.263 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:04.263 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:30:04.263 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:04.263 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:30:04.263 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062939+0000", 2024-09-14T06:30:04.263 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:04.263 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:04.263 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:04.263 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:30:04.263 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062939+0000", 2024-09-14T06:30:04.263 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:04.264 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:04.264 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:04.264 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:30:04.264 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062940+0000", 2024-09-14T06:30:04.264 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.32e-07 2024-09-14T06:30:04.264 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:04.264 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:04.264 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:30:04.264 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062942+0000", 2024-09-14T06:30:04.264 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.1919999999999999e-06 2024-09-14T06:30:04.265 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:04.265 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:04.265 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:30:04.265 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062948+0000", 2024-09-14T06:30:04.265 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.1149999999999999e-06 2024-09-14T06:30:04.265 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:04.265 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:04.265 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:30:04.265 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062960+0000", 2024-09-14T06:30:04.265 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.1895e-05 2024-09-14T06:30:04.265 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:04.265 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:04.265 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:30:04.266 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.109704+0000", 2024-09-14T06:30:04.266 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046744584999999998 2024-09-14T06:30:04.266 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:04.266 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:04.266 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:30:04.266 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.109777+0000", 2024-09-14T06:30:04.266 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.2282999999999999e-05 2024-09-14T06:30:04.266 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:04.266 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:04.267 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:30:04.267 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.109860+0000", 2024-09-14T06:30:04.267 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.3005999999999998e-05 2024-09-14T06:30:04.267 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:04.267 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:30:04.267 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:04.267 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:04.267 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:04.267 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:19358 3.3 3:c34e5e51:::rbd_data.112d92027590.0000000000000024:f [read 1230848~1024 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:30:04.267 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:27.062967+0000", 2024-09-14T06:30:04.267 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 37.158021818000002, 2024-09-14T06:30:04.267 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047144471, 2024-09-14T06:30:04.267 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:30:04.268 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:30:04.268 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:30:04.268 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:30:04.268 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:30:04.268 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:04.268 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 19358 2024-09-14T06:30:04.268 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:04.268 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:30:04.268 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:04.268 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:30:04.268 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062967+0000", 2024-09-14T06:30:04.268 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:04.269 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:04.269 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:04.269 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:30:04.269 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062967+0000", 2024-09-14T06:30:04.269 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:04.269 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:04.269 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:04.269 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:30:04.269 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062967+0000", 2024-09-14T06:30:04.269 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.89e-07 2024-09-14T06:30:04.269 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:04.270 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:04.270 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:30:04.270 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062969+0000", 2024-09-14T06:30:04.270 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.9929999999999998e-06 2024-09-14T06:30:04.270 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:04.270 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:04.270 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:30:04.270 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062975+0000", 2024-09-14T06:30:04.270 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.9030000000000001e-06 2024-09-14T06:30:04.270 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:04.270 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:04.270 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:30:04.270 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062979+0000", 2024-09-14T06:30:04.270 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.2250000000000002e-06 2024-09-14T06:30:04.271 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:04.271 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:04.271 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:30:04.271 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.109934+0000", 2024-09-14T06:30:04.271 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046954410000000002 2024-09-14T06:30:04.271 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:04.271 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:04.271 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:30:04.271 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.110007+0000", 2024-09-14T06:30:04.271 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.3829000000000001e-05 2024-09-14T06:30:04.271 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:04.271 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:04.271 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:30:04.271 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.110111+0000", 2024-09-14T06:30:04.272 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00010372199999999999 2024-09-14T06:30:04.272 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:04.272 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:30:04.272 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:04.272 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:04.272 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:04.272 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:19359 3.3 3:c34e5e51:::rbd_data.112d92027590.0000000000000024:f [read 1241088~1024 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:30:04.272 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:27.062987+0000", 2024-09-14T06:30:04.272 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 37.158001728000002, 2024-09-14T06:30:04.272 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047297182, 2024-09-14T06:30:04.273 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:30:04.273 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:30:04.273 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:30:04.273 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:30:04.273 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:30:04.273 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:04.273 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 19359 2024-09-14T06:30:04.273 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:04.273 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:30:04.273 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:04.273 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:30:04.273 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062987+0000", 2024-09-14T06:30:04.273 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:04.274 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:04.274 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:04.274 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:30:04.274 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062987+0000", 2024-09-14T06:30:04.274 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:04.274 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:04.274 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:04.274 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:30:04.274 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062987+0000", 2024-09-14T06:30:04.274 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.2500000000000001e-07 2024-09-14T06:30:04.274 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:04.274 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:04.274 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:30:04.274 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062989+0000", 2024-09-14T06:30:04.275 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.6080000000000001e-06 2024-09-14T06:30:04.275 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:04.275 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:04.275 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:30:04.275 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062994+0000", 2024-09-14T06:30:04.275 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.5520000000000004e-06 2024-09-14T06:30:04.275 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:04.275 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:04.275 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:30:04.275 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062999+0000", 2024-09-14T06:30:04.275 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.3930000000000001e-06 2024-09-14T06:30:04.275 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:04.275 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:04.275 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:30:04.276 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.110142+0000", 2024-09-14T06:30:04.276 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047143154 2024-09-14T06:30:04.276 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:04.276 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:04.276 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:30:04.277 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.110219+0000", 2024-09-14T06:30:04.277 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.6737999999999996e-05 2024-09-14T06:30:04.277 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:04.277 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:04.277 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:30:04.277 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.110284+0000", 2024-09-14T06:30:04.277 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.5412000000000003e-05 2024-09-14T06:30:04.277 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:04.277 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:30:04.277 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:04.277 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:04.278 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:04.278 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:19360 3.3 3:c34e5e51:::rbd_data.112d92027590.0000000000000024:f [read 1251328~1024 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:30:04.278 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:27.063017+0000", 2024-09-14T06:30:04.278 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 37.157971267000001, 2024-09-14T06:30:04.278 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047453823999999999, 2024-09-14T06:30:04.278 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:30:04.278 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:30:04.278 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:30:04.278 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:30:04.278 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:30:04.278 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:04.278 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 19360 2024-09-14T06:30:04.278 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:04.278 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:30:04.279 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:04.279 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:30:04.279 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.063017+0000", 2024-09-14T06:30:04.279 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:04.279 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:04.279 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:04.279 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:30:04.279 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.063017+0000", 2024-09-14T06:30:04.279 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:04.279 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:04.279 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:04.279 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:30:04.280 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.064977+0000", 2024-09-14T06:30:04.280 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.0019597080000000001 2024-09-14T06:30:04.280 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:04.280 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:04.280 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:30:04.280 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.064987+0000", 2024-09-14T06:30:04.280 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.9620000000000005e-06 2024-09-14T06:30:04.280 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:04.280 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:04.280 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:30:04.280 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065000+0000", 2024-09-14T06:30:04.280 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.3173000000000001e-05 2024-09-14T06:30:04.281 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:04.281 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:04.281 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:30:04.281 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065005+0000", 2024-09-14T06:30:04.281 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.9960000000000001e-06 2024-09-14T06:30:04.281 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:04.281 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:04.281 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:30:04.281 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.110314+0000", 2024-09-14T06:30:04.281 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.04530899 2024-09-14T06:30:04.281 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:04.281 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:04.281 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:30:04.281 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.110398+0000", 2024-09-14T06:30:04.282 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.3539000000000001e-05 2024-09-14T06:30:04.282 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:04.282 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:04.282 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:30:04.282 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.110471+0000", 2024-09-14T06:30:04.282 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.3455999999999996e-05 2024-09-14T06:30:04.282 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:04.282 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:30:04.282 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:04.282 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:04.282 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:04.282 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:19365 3.3 3:c34e5e51:::rbd_data.112d92027590.0000000000000024:f [read 1302528~1024 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:30:04.282 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:27.065058+0000", 2024-09-14T06:30:04.282 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 37.155930597000001, 2024-09-14T06:30:04.282 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046105526000000001, 2024-09-14T06:30:04.283 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:30:04.283 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:30:04.283 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:30:04.283 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:30:04.283 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:30:04.283 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:04.283 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 19365 2024-09-14T06:30:04.283 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:04.283 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:30:04.283 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:04.284 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:30:04.284 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065058+0000", 2024-09-14T06:30:04.284 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:04.284 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:04.284 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:04.284 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:30:04.284 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065058+0000", 2024-09-14T06:30:04.284 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:04.284 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:04.284 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:04.284 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:30:04.284 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065058+0000", 2024-09-14T06:30:04.285 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.1899999999999999e-07 2024-09-14T06:30:04.285 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:04.285 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:04.285 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:30:04.285 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065059+0000", 2024-09-14T06:30:04.285 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.1299999999999998e-07 2024-09-14T06:30:04.285 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:04.285 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:04.285 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:30:04.285 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065063+0000", 2024-09-14T06:30:04.285 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.1099999999999996e-06 2024-09-14T06:30:04.285 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:04.285 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:04.286 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:30:04.286 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065070+0000", 2024-09-14T06:30:04.286 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.973e-06 2024-09-14T06:30:04.286 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:04.286 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:04.286 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:30:04.286 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.111014+0000", 2024-09-14T06:30:04.286 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.045943373000000003 2024-09-14T06:30:04.286 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:04.286 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:04.286 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:30:04.286 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.111073+0000", 2024-09-14T06:30:04.286 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.8987999999999998e-05 2024-09-14T06:30:04.286 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:04.287 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:04.287 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:30:04.287 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.111163+0000", 2024-09-14T06:30:04.287 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.0950000000000002e-05 2024-09-14T06:30:04.287 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:04.287 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:30:04.287 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:04.287 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:04.287 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:04.287 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:19366 3.3 3:c34e5e51:::rbd_data.112d92027590.0000000000000024:f [read 1312768~1024 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:30:04.287 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:27.065074+0000", 2024-09-14T06:30:04.287 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 37.155914746999997, 2024-09-14T06:30:04.287 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046289858000000003, 2024-09-14T06:30:04.288 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:30:04.288 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:30:04.288 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:30:04.288 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:30:04.288 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:30:04.288 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:04.288 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 19366 2024-09-14T06:30:04.288 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:04.288 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:30:04.289 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:04.289 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:30:04.289 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065074+0000", 2024-09-14T06:30:04.289 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:04.289 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:04.289 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:04.289 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:30:04.289 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065074+0000", 2024-09-14T06:30:04.289 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:04.289 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:04.289 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:04.289 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:30:04.290 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065074+0000", 2024-09-14T06:30:04.290 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.3900000000000001e-07 2024-09-14T06:30:04.290 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:04.290 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:04.290 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:30:04.290 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065078+0000", 2024-09-14T06:30:04.290 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.3229999999999999e-06 2024-09-14T06:30:04.290 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:04.290 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:04.290 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:30:04.290 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065082+0000", 2024-09-14T06:30:04.290 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.8299999999999998e-06 2024-09-14T06:30:04.290 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:04.290 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:04.291 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:30:04.291 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065085+0000", 2024-09-14T06:30:04.291 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.8660000000000002e-06 2024-09-14T06:30:04.291 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:04.291 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:04.291 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:30:04.291 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.111270+0000", 2024-09-14T06:30:04.291 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046185035999999999 2024-09-14T06:30:04.291 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:04.291 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:04.292 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:30:04.292 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.111311+0000", 2024-09-14T06:30:04.292 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.0935e-05 2024-09-14T06:30:04.292 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:04.292 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:04.292 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:30:04.292 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.111364+0000", 2024-09-14T06:30:04.292 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.2629e-05 2024-09-14T06:30:04.292 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:04.292 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:30:04.292 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:04.292 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:04.293 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:04.293 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:19367 3.3 3:c34e5e51:::rbd_data.112d92027590.0000000000000024:f [read 1323008~1024 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:30:04.293 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:27.065115+0000", 2024-09-14T06:30:04.293 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 37.155873186000001, 2024-09-14T06:30:04.293 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046407003000000002, 2024-09-14T06:30:04.293 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:30:04.293 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:30:04.293 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:30:04.293 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:30:04.293 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:30:04.293 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:04.293 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 19367 2024-09-14T06:30:04.293 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:04.294 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:30:04.294 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:04.294 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:30:04.294 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065115+0000", 2024-09-14T06:30:04.294 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:04.294 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:04.294 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:04.294 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:30:04.294 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065115+0000", 2024-09-14T06:30:04.294 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:04.294 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:04.294 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:04.295 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:30:04.295 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065116+0000", 2024-09-14T06:30:04.295 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.4999999999999998e-07 2024-09-14T06:30:04.295 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:04.295 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:04.295 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:30:04.295 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065117+0000", 2024-09-14T06:30:04.295 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.189e-06 2024-09-14T06:30:04.295 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:04.295 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:04.295 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:30:04.295 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065122+0000", 2024-09-14T06:30:04.295 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.3709999999999999e-06 2024-09-14T06:30:04.295 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:04.296 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:04.296 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:30:04.296 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065125+0000", 2024-09-14T06:30:04.296 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.8940000000000002e-06 2024-09-14T06:30:04.296 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:04.296 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:04.296 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:30:04.296 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.111402+0000", 2024-09-14T06:30:04.296 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046276645999999998 2024-09-14T06:30:04.296 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:04.296 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:04.296 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:30:04.296 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.111445+0000", 2024-09-14T06:30:04.297 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.3288000000000003e-05 2024-09-14T06:30:04.297 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:04.297 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:04.297 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:30:04.297 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.111522+0000", 2024-09-14T06:30:04.297 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.7164999999999997e-05 2024-09-14T06:30:04.297 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:04.297 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:30:04.297 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:04.297 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:04.297 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:04.297 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:19368 3.3 3:c34e5e51:::rbd_data.112d92027590.0000000000000024:f [read 1333248~1024 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:30:04.297 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:27.065130+0000", 2024-09-14T06:30:04.297 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 37.155858895999998, 2024-09-14T06:30:04.298 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046507275000000001, 2024-09-14T06:30:04.298 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:30:04.298 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:30:04.298 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:30:04.298 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:30:04.298 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:30:04.298 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:04.298 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 19368 2024-09-14T06:30:04.298 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:04.298 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:30:04.298 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:04.298 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:30:04.298 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065130+0000", 2024-09-14T06:30:04.298 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:04.299 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:04.299 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:04.299 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:30:04.299 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065130+0000", 2024-09-14T06:30:04.299 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:04.299 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:04.299 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:04.299 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:30:04.299 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065130+0000", 2024-09-14T06:30:04.299 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.0800000000000001e-07 2024-09-14T06:30:04.299 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:04.299 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:04.299 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:30:04.300 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065131+0000", 2024-09-14T06:30:04.300 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.8700000000000004e-07 2024-09-14T06:30:04.300 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:04.300 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:04.300 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:30:04.300 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065134+0000", 2024-09-14T06:30:04.300 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.1039999999999998e-06 2024-09-14T06:30:04.300 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:04.300 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:04.300 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:30:04.300 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065136+0000", 2024-09-14T06:30:04.300 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.3949999999999999e-06 2024-09-14T06:30:04.300 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:04.301 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:04.301 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:30:04.301 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.111541+0000", 2024-09-14T06:30:04.301 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046404206000000003 2024-09-14T06:30:04.301 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:04.301 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:04.301 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:30:04.301 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.111582+0000", 2024-09-14T06:30:04.301 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.1051999999999999e-05 2024-09-14T06:30:04.301 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:04.302 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:04.302 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:30:04.302 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.111637+0000", 2024-09-14T06:30:04.302 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.5322999999999999e-05 2024-09-14T06:30:04.302 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:04.302 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:30:04.302 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:04.302 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:04.302 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:04.302 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:19369 3.3 3:c34e5e51:::rbd_data.112d92027590.0000000000000024:f [read 1343488~1024 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:30:04.302 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:27.065139+0000", 2024-09-14T06:30:04.302 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 37.155849078999999, 2024-09-14T06:30:04.302 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046682169000000003, 2024-09-14T06:30:04.302 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:30:04.303 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:30:04.303 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:30:04.303 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:30:04.303 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:30:04.303 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:04.303 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 19369 2024-09-14T06:30:04.303 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:04.303 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:30:04.303 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:04.303 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:30:04.303 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065139+0000", 2024-09-14T06:30:04.303 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:04.303 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:04.303 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:04.304 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:30:04.304 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065139+0000", 2024-09-14T06:30:04.304 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:04.304 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:04.304 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:04.304 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:30:04.304 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065140+0000", 2024-09-14T06:30:04.304 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.8299999999999998e-07 2024-09-14T06:30:04.304 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:04.304 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:04.304 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:30:04.304 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065141+0000", 2024-09-14T06:30:04.304 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.3900000000000003e-07 2024-09-14T06:30:04.305 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:04.305 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:04.305 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:30:04.305 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065144+0000", 2024-09-14T06:30:04.305 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.7409999999999998e-06 2024-09-14T06:30:04.305 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:04.305 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:04.305 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:30:04.305 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065147+0000", 2024-09-14T06:30:04.305 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.4150000000000002e-06 2024-09-14T06:30:04.305 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:04.305 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:04.306 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:30:04.306 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.111673+0000", 2024-09-14T06:30:04.306 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046526004000000003 2024-09-14T06:30:04.306 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:04.306 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:04.306 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:30:04.306 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.111749+0000", 2024-09-14T06:30:04.306 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.5741999999999994e-05 2024-09-14T06:30:04.306 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:04.306 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:04.306 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:30:04.306 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.111822+0000", 2024-09-14T06:30:04.306 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.3045000000000003e-05 2024-09-14T06:30:04.307 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:04.307 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:30:04.307 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:04.307 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:04.307 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:04.307 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:19370 3.3 3:c34e5e51:::rbd_data.112d92027590.0000000000000024:f [read 1353728~1024 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:30:04.307 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:27.065150+0000", 2024-09-14T06:30:04.307 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 37.155838350000003, 2024-09-14T06:30:04.307 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046897556999999999, 2024-09-14T06:30:04.307 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:30:04.307 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:30:04.307 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:30:04.307 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:30:04.307 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:30:04.308 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:04.308 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 19370 2024-09-14T06:30:04.308 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:04.308 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:30:04.308 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:04.308 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:30:04.308 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065150+0000", 2024-09-14T06:30:04.308 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:04.308 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:04.308 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:04.308 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:30:04.308 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065150+0000", 2024-09-14T06:30:04.308 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:04.308 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:04.309 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:04.309 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:30:04.309 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065150+0000", 2024-09-14T06:30:04.309 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.0200000000000001e-07 2024-09-14T06:30:04.309 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:04.309 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:04.309 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:30:04.309 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065151+0000", 2024-09-14T06:30:04.309 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.1100000000000004e-07 2024-09-14T06:30:04.309 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:04.309 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:04.309 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:30:04.309 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065154+0000", 2024-09-14T06:30:04.309 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.9639999999999999e-06 2024-09-14T06:30:04.310 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:04.310 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:04.310 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:30:04.310 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065157+0000", 2024-09-14T06:30:04.310 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.4260000000000002e-06 2024-09-14T06:30:04.310 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:04.310 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:04.310 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:30:04.310 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.111861+0000", 2024-09-14T06:30:04.310 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046704268 2024-09-14T06:30:04.310 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:04.310 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:04.310 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:30:04.310 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.111939+0000", 2024-09-14T06:30:04.311 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.8231000000000004e-05 2024-09-14T06:30:04.311 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:04.311 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:04.311 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:30:04.311 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.112048+0000", 2024-09-14T06:30:04.311 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00010855500000000001 2024-09-14T06:30:04.311 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:04.311 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:30:04.311 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:04.311 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:04.311 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:04.311 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:19371 3.3 3:c34e5e51:::rbd_data.112d92027590.0000000000000024:f [read 1363968~1024 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:30:04.311 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:27.065161+0000", 2024-09-14T06:30:04.311 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 37.155827700000003, 2024-09-14T06:30:04.311 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047021944000000003, 2024-09-14T06:30:04.312 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:30:04.312 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:30:04.312 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:30:04.312 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:30:04.312 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:30:04.312 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:04.312 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 19371 2024-09-14T06:30:04.312 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:04.312 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:30:04.312 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:04.312 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:30:04.312 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065161+0000", 2024-09-14T06:30:04.312 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:04.312 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:04.313 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:04.313 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:30:04.313 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065161+0000", 2024-09-14T06:30:04.313 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:04.313 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:04.313 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:04.313 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:30:04.313 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065161+0000", 2024-09-14T06:30:04.313 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.0200000000000001e-07 2024-09-14T06:30:04.313 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:04.314 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:04.314 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:30:04.314 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065166+0000", 2024-09-14T06:30:04.314 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.9869999999999999e-06 2024-09-14T06:30:04.314 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:04.314 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:04.314 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:30:04.314 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065169+0000", 2024-09-14T06:30:04.314 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.3799999999999998e-06 2024-09-14T06:30:04.314 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:04.314 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:04.315 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:30:04.315 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065172+0000", 2024-09-14T06:30:04.315 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.4830000000000002e-06 2024-09-14T06:30:04.315 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:04.315 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:04.315 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:30:04.315 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.112071+0000", 2024-09-14T06:30:04.315 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046899529000000002 2024-09-14T06:30:04.315 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:04.315 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:04.315 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:30:04.315 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.112128+0000", 2024-09-14T06:30:04.315 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.6341999999999997e-05 2024-09-14T06:30:04.315 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:04.316 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:04.316 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:30:04.316 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.112183+0000", 2024-09-14T06:30:04.316 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.5021000000000001e-05 2024-09-14T06:30:04.316 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:04.316 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:30:04.316 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:04.316 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:04.316 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:04.316 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:19372 3.3 3:c34e5e51:::rbd_data.112d92027590.0000000000000024:f [read 1374208~1024 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:30:04.316 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:27.065176+0000", 2024-09-14T06:30:04.316 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 37.15581255, 2024-09-14T06:30:04.316 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047188889999999997, 2024-09-14T06:30:04.316 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:30:04.317 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:30:04.317 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:30:04.317 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:30:04.317 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:30:04.317 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:04.317 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 19372 2024-09-14T06:30:04.317 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:04.317 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:30:04.317 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:04.317 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:30:04.317 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065176+0000", 2024-09-14T06:30:04.317 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:04.318 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:04.318 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:04.318 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:30:04.318 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065176+0000", 2024-09-14T06:30:04.318 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:04.318 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:04.318 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:04.318 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:30:04.318 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.071117+0000", 2024-09-14T06:30:04.318 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.0059404990000000001 2024-09-14T06:30:04.318 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:04.319 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:04.319 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:30:04.319 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.071120+0000", 2024-09-14T06:30:04.319 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.4340000000000001e-06 2024-09-14T06:30:04.319 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:04.319 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:04.319 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:30:04.319 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.071129+0000", 2024-09-14T06:30:04.319 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.8389999999999996e-06 2024-09-14T06:30:04.319 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:04.319 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:04.319 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:30:04.319 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.071134+0000", 2024-09-14T06:30:04.319 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.412e-06 2024-09-14T06:30:04.320 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:04.320 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:04.320 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:30:04.320 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.112221+0000", 2024-09-14T06:30:04.320 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.041086473999999998 2024-09-14T06:30:04.320 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:04.320 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:04.320 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:30:04.320 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.112299+0000", 2024-09-14T06:30:04.320 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.8746999999999997e-05 2024-09-14T06:30:04.320 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:04.320 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:04.320 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:30:04.320 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.112365+0000", 2024-09-14T06:30:04.321 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.5485000000000001e-05 2024-09-14T06:30:04.321 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:04.321 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:30:04.321 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:04.321 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:04.321 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:30:04.321 INFO:teuthology.orchestra.run.smithi138.stdout:} 2024-09-14T06:30:04.322 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-14T06:30:04.324 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-14T06:30:04.324 INFO:tasks.ceph.osd.7.smithi138.stderr:2024-09-14T06:30:04.321+0000 7f3c6a21d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33623) UID: 0 2024-09-14T06:30:04.326 INFO:teuthology.orchestra.run.smithi094.stdout:2338 trunc from 0xade78bd to 0x2c7d559 2024-09-14T06:30:04.326 INFO:teuthology.orchestra.run.smithi094.stdout:2339 write 0x482af8b thru 0x483177b (0x67f1 bytes) 2024-09-14T06:30:04.326 INFO:teuthology.orchestra.run.smithi094.stdout:2342 read 0x3a7aa26 thru 0x3a820dc (0x76b7 bytes) 2024-09-14T06:30:04.327 INFO:teuthology.orchestra.run.smithi094.stdout:2343 trunc from 0x483177c to 0xcf960c3 2024-09-14T06:30:04.327 INFO:teuthology.orchestra.run.smithi094.stdout:2345 write 0xc73d0a1 thru 0xc748680 (0xb5e0 bytes) 2024-09-14T06:30:04.327 INFO:teuthology.orchestra.run.smithi094.stdout:2347 read 0xf4d3ef thru 0xf5662d (0x923f bytes) 2024-09-14T06:30:04.327 INFO:teuthology.orchestra.run.smithi094.stdout:2348 write 0x75e8509 thru 0x75f1138 (0x8c30 bytes) 2024-09-14T06:30:04.327 INFO:teuthology.orchestra.run.smithi094.stdout:2349 punch from 0x5523465 to 0x552807c, (0x4c17 bytes) 2024-09-14T06:30:04.327 INFO:teuthology.orchestra.run.smithi094.stdout:2350 punch from 0xcaf7a7 to 0xcbf409, (0xfc62 bytes) 2024-09-14T06:30:04.327 INFO:teuthology.orchestra.run.smithi094.stdout:2351 write 0xd84863 thru 0xd89342 (0x4ae0 bytes) 2024-09-14T06:30:04.328 INFO:teuthology.orchestra.run.smithi094.stdout:2353 read 0x8155426 thru 0x815f87b (0xa456 bytes) 2024-09-14T06:30:04.328 INFO:teuthology.orchestra.run.smithi094.stdout:2354 read 0x9e3bd9 thru 0x9ec826 (0x8c4e bytes) 2024-09-14T06:30:04.328 INFO:teuthology.orchestra.run.smithi094.stdout:2355 write 0x189052c thru 0x1890dfa (0x8cf bytes) 2024-09-14T06:30:04.328 INFO:teuthology.orchestra.run.smithi094.stdout:2356 read 0x59b5b6e thru 0x59c3fae (0xe441 bytes) 2024-09-14T06:30:04.328 INFO:teuthology.orchestra.run.smithi094.stdout:2358 write 0xb648470 thru 0xb652bc9 (0xa75a bytes) 2024-09-14T06:30:04.328 INFO:teuthology.orchestra.run.smithi094.stdout:2359 writesame 0x6eae282 thru 0x6eb2dc1 data_size 0x8(0x4b40 bytes) 2024-09-14T06:30:04.328 INFO:teuthology.orchestra.run.smithi094.stdout:2361 punch from 0x30ba2ac to 0x30c67b4, (0xc508 bytes) 2024-09-14T06:30:04.328 INFO:teuthology.orchestra.run.smithi094.stdout:2362 read 0x7df5043 thru 0x7df6b40 (0x1afe bytes) 2024-09-14T06:30:04.328 INFO:teuthology.orchestra.run.smithi094.stdout:2363 read 0xb471e77 thru 0xb47dd42 (0xbecc bytes) 2024-09-14T06:30:04.328 INFO:teuthology.orchestra.run.smithi094.stdout:2364 read 0xa66118c thru 0xa661b63 (0x9d8 bytes) 2024-09-14T06:30:04.328 INFO:teuthology.orchestra.run.smithi094.stdout:2365 trunc from 0xcf960c3 to 0xca48a9b 2024-09-14T06:30:04.328 INFO:teuthology.orchestra.run.smithi094.stdout:2367 read 0x7819370 thru 0x781e995 (0x5626 bytes) 2024-09-14T06:30:04.328 INFO:teuthology.orchestra.run.smithi094.stdout:2368 punch from 0x36f2205 to 0x36f9f9c, (0x7d97 bytes) 2024-09-14T06:30:04.329 INFO:teuthology.orchestra.run.smithi094.stdout:2369 write 0xc76d8ed thru 0xc77c04c (0xe760 bytes) 2024-09-14T06:30:04.329 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-14T06:30:04.329 INFO:teuthology.orchestra.run.smithi094.stdout:2371 punch from 0xd520b7 to 0xd52a36, (0x97f bytes) 2024-09-14T06:30:04.329 INFO:teuthology.orchestra.run.smithi094.stdout:2372 write 0xdda94f0 thru 0xddabe69 (0x297a bytes) 2024-09-14T06:30:04.329 INFO:teuthology.orchestra.run.smithi094.stdout:2373 write 0x6869646 thru 0x68777c0 (0xe17b bytes) 2024-09-14T06:30:04.329 INFO:teuthology.orchestra.run.smithi094.stdout:2374 read 0x3942305 thru 0x394c9fb (0xa6f7 bytes) 2024-09-14T06:30:04.329 INFO:teuthology.orchestra.run.smithi094.stdout:2375 trunc from 0xddabe6a to 0x49cfa75 2024-09-14T06:30:04.329 INFO:teuthology.orchestra.run.smithi094.stdout:2376 write 0x645265d thru 0x6458870 (0x6214 bytes) 2024-09-14T06:30:04.334 INFO:teuthology.orchestra.run.smithi094.stdout:2377 writesame 0x82bf402 thru 0x82c6bf7 data_size 0x53(0x77f6 bytes) 2024-09-14T06:30:04.344 INFO:teuthology.orchestra.run.smithi094.stdout:2379 writesame 0xb911af9 thru 0xb9175bc data_size 0x4a(0x5ac4 bytes) 2024-09-14T06:30:04.345 INFO:teuthology.orchestra.run.smithi094.stdout:2380 read 0x547ae03 thru 0x547f336 (0x4534 bytes) 2024-09-14T06:30:04.346 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-14T06:30:04.346 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-14T06:30:04.346 INFO:teuthology.orchestra.run.smithi094.stdout:2384 read 0xa7566df thru 0xa763ad7 (0xd3f9 bytes) 2024-09-14T06:30:04.346 INFO:teuthology.orchestra.run.smithi094.stdout:2385 writesame 0x34c4c94 thru 0x34d3f33 data_size 0x30(0xf2a0 bytes) 2024-09-14T06:30:04.348 INFO:teuthology.orchestra.run.smithi094.stdout:2388 write 0x624c968 thru 0x62515ed (0x4c86 bytes) 2024-09-14T06:30:04.349 INFO:teuthology.orchestra.run.smithi094.stdout:2389 writesame 0x612f4de thru 0x6130d66 data_size 0x1(0x1889 bytes) 2024-09-14T06:30:04.352 INFO:teuthology.orchestra.run.smithi094.stdout:2390 punch from 0x93c640b to 0x93d35b3, (0xd1a8 bytes) 2024-09-14T06:30:04.353 INFO:teuthology.orchestra.run.smithi094.stdout:2391 write 0x5cd8a9a thru 0x5cded92 (0x62f9 bytes) 2024-09-14T06:30:04.356 INFO:teuthology.orchestra.run.smithi094.stdout:2392 trunc from 0xb9175bd to 0xd12a6f4 2024-09-14T06:30:04.361 INFO:teuthology.orchestra.run.smithi094.stdout:2393 read 0x8d9f4f0 thru 0x8da9a56 (0xa567 bytes) 2024-09-14T06:30:04.362 INFO:teuthology.orchestra.run.smithi094.stdout:2394 trunc from 0xd12a6f4 to 0xdcaf97e 2024-09-14T06:30:04.367 INFO:teuthology.orchestra.run.smithi094.stdout:2395 writesame 0xbc20657 thru 0xbc2bde5 data_size 0x31(0xb78f bytes) 2024-09-14T06:30:04.369 INFO:teuthology.orchestra.run.smithi094.stdout:2396 punch from 0xae40ce7 to 0xae45ae1, (0x4dfa bytes) 2024-09-14T06:30:04.369 INFO:teuthology.orchestra.run.smithi094.stdout:2397 punch from 0x47dbcaa to 0x47e899e, (0xccf4 bytes) 2024-09-14T06:30:04.369 INFO:teuthology.orchestra.run.smithi094.stdout:2398 write 0x8e758cc thru 0x8e78bf8 (0x332d bytes) 2024-09-14T06:30:04.370 INFO:teuthology.orchestra.run.smithi094.stdout:2400 read 0x5afbda7 thru 0x5b0025c (0x44b6 bytes) 2024-09-14T06:30:04.371 INFO:teuthology.orchestra.run.smithi094.stdout:2404 read 0x9e92d42 thru 0x9e9a808 (0x7ac7 bytes) 2024-09-14T06:30:04.371 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-14T06:30:04.371 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-14T06:30:04.371 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-14T06:30:04.371 INFO:teuthology.orchestra.run.smithi094.stdout:2408 read 0x1dfec7e thru 0x1e016e9 (0x2a6c bytes) 2024-09-14T06:30:04.372 INFO:teuthology.orchestra.run.smithi094.stdout:2409 writesame 0x3aa78ae thru 0x3aadcd2 data_size 0x1(0x6425 bytes) 2024-09-14T06:30:04.384 INFO:teuthology.orchestra.run.smithi094.stdout:2413 read 0xb0a1dec thru 0xb0a2c97 (0xeac bytes) 2024-09-14T06:30:04.384 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-14T06:30:04.385 INFO:teuthology.orchestra.run.smithi094.stdout:2416 read 0x7c67f32 thru 0x7c720e7 (0xa1b6 bytes) 2024-09-14T06:30:04.385 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-14T06:30:04.387 INFO:teuthology.orchestra.run.smithi094.stdout:2418 trunc from 0xdcaf97e to 0xc53074f 2024-09-14T06:30:04.394 INFO:teuthology.orchestra.run.smithi094.stdout:2419 writesame 0x6a41620 thru 0x6a4f378 data_size 0x7(0xdd59 bytes) 2024-09-14T06:30:04.398 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-14T06:30:04.398 INFO:teuthology.orchestra.run.smithi094.stdout:2423 writesame 0x36af98f thru 0x36b0090 data_size 0x1a(0x702 bytes) 2024-09-14T06:30:04.400 INFO:teuthology.orchestra.run.smithi094.stdout:2424 write 0xad072da thru 0xad0eef6 (0x7c1d bytes) 2024-09-14T06:30:04.408 INFO:teuthology.orchestra.run.smithi094.stdout:2425 writesame 0xda6eaf3 thru 0xda6f345 data_size 0x1(0x853 bytes) 2024-09-14T06:30:04.409 INFO:teuthology.orchestra.run.smithi094.stdout:2426 write 0x5f5e09c thru 0x5f6745b (0x93c0 bytes) 2024-09-14T06:30:04.411 INFO:teuthology.orchestra.run.smithi094.stdout:2427 punch from 0x1b7698f to 0x1b7d6e5, (0x6d56 bytes) 2024-09-14T06:30:04.411 INFO:teuthology.orchestra.run.smithi094.stdout:2429 write 0xa34ddcd thru 0xa34f78f (0x19c3 bytes) 2024-09-14T06:30:04.412 INFO:teuthology.orchestra.run.smithi094.stdout:2430 read 0x46e3235 thru 0x46e728c (0x4058 bytes) 2024-09-14T06:30:04.412 INFO:teuthology.orchestra.run.smithi094.stdout:2431 punch from 0x9117c85 to 0x91269f6, (0xed71 bytes) 2024-09-14T06:30:04.413 INFO:teuthology.orchestra.run.smithi094.stdout:2435 writesame 0x6487f2 thru 0x658190 data_size 0x15(0xf99f bytes) 2024-09-14T06:30:04.415 INFO:teuthology.orchestra.run.smithi094.stdout:2436 punch from 0x6630cbf to 0x663c869, (0xbbaa bytes) 2024-09-14T06:30:04.415 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-14T06:30:04.415 INFO:teuthology.orchestra.run.smithi094.stdout:2438 writesame 0x46fbe76 thru 0x46ffc31 data_size 0x12(0x3dbc bytes) 2024-09-14T06:30:04.416 INFO:teuthology.orchestra.run.smithi094.stdout:2439 writesame 0xb1a2558 thru 0xb1a711d data_size 0x6a(0x4bc6 bytes) 2024-09-14T06:30:04.417 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-14T06:30:04.417 INFO:teuthology.orchestra.run.smithi094.stdout:2442 writesame 0x3ef428b thru 0x3f0190c data_size 0x1(0xd682 bytes) 2024-09-14T06:30:04.423 INFO:tasks.ceph.osd.6.smithi138.stderr:2024-09-14T06:30:04.422+0000 7f8cd8849640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33620) UID: 0 2024-09-14T06:30:04.438 INFO:teuthology.orchestra.run.smithi094.stdout:2444 write 0x284f90e thru 0x285be62 (0xc555 bytes) 2024-09-14T06:30:04.446 INFO:teuthology.orchestra.run.smithi094.stdout:2445 write 0xe8f8e6f thru 0xe90478b (0xb91d bytes) 2024-09-14T06:30:04.447 INFO:teuthology.orchestra.run.smithi094.stdout:2448 write 0x409cba thru 0x418951 (0xec98 bytes) 2024-09-14T06:30:04.449 INFO:teuthology.orchestra.run.smithi094.stdout:2449 read 0xde0b20f thru 0xde14958 (0x974a bytes) 2024-09-14T06:30:04.449 INFO:teuthology.orchestra.run.smithi094.stdout:2451 punch from 0x842fb23 to 0x843b925, (0xbe02 bytes) 2024-09-14T06:30:04.455 INFO:teuthology.orchestra.run.smithi094.stdout:2452 write 0xeb2a535 thru 0xeb36fa3 (0xca6f bytes) 2024-09-14T06:30:04.456 INFO:teuthology.orchestra.run.smithi094.stdout:osd.0: {} 2024-09-14T06:30:04.456 INFO:teuthology.orchestra.run.smithi094.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-14T06:30:04.457 INFO:teuthology.orchestra.run.smithi094.stdout:2455 write 0x2d06baa thru 0x2d09ef2 (0x3349 bytes) 2024-09-14T06:30:04.458 INFO:teuthology.orchestra.run.smithi094.stdout:2456 punch from 0x2a0eaa5 to 0x2a102e4, (0x183f bytes) 2024-09-14T06:30:04.458 INFO:teuthology.orchestra.run.smithi094.stdout:2457 read 0x2f55733 thru 0x2f62acc (0xd39a bytes) 2024-09-14T06:30:04.458 INFO:teuthology.orchestra.run.smithi094.stdout:2460 writesame 0x323c11d thru 0x3243423 data_size 0x1(0x7307 bytes) 2024-09-14T06:30:04.469 INFO:teuthology.orchestra.run.smithi094.stdout:2461 write 0x335e3b2 thru 0x33652b2 (0x6f01 bytes) 2024-09-14T06:30:04.472 INFO:teuthology.orchestra.run.smithi094.stdout:2462 trunc from 0xeb36fa4 to 0xdeca2cb 2024-09-14T06:30:04.474 INFO:teuthology.orchestra.run.smithi138.stdout:{ 2024-09-14T06:30:04.474 INFO:teuthology.orchestra.run.smithi138.stdout: "ops": [], 2024-09-14T06:30:04.474 INFO:teuthology.orchestra.run.smithi138.stdout: "num_ops": 0 2024-09-14T06:30:04.474 INFO:teuthology.orchestra.run.smithi138.stdout:} 2024-09-14T06:30:04.475 INFO:teuthology.orchestra.run.smithi094.stdout:osd.1: {} 2024-09-14T06:30:04.475 INFO:teuthology.orchestra.run.smithi094.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-14T06:30:04.479 INFO:teuthology.orchestra.run.smithi094.stdout:2463 read 0x4ce2297 thru 0x4ce6a79 (0x47e3 bytes) 2024-09-14T06:30:04.481 INFO:teuthology.orchestra.run.smithi094.stdout:2464 trunc from 0xdeca2cb to 0xc14d65c 2024-09-14T06:30:04.485 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-14T06:30:04.488 INFO:teuthology.orchestra.run.smithi094.stdout:2465 write 0xba4c4ed thru 0xba4e1ec (0x1d00 bytes) 2024-09-14T06:30:04.490 INFO:teuthology.orchestra.run.smithi094.stdout:2466 punch from 0x4832d48 to 0x4840fcb, (0xe283 bytes) 2024-09-14T06:30:04.493 INFO:teuthology.orchestra.run.smithi094.stdout:2468 trunc from 0xc14d65c to 0x94ccd98 2024-09-14T06:30:04.495 INFO:teuthology.orchestra.run.smithi094.stdout:osd.2: {} 2024-09-14T06:30:04.495 INFO:teuthology.orchestra.run.smithi094.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-14T06:30:04.500 INFO:teuthology.orchestra.run.smithi094.stdout:2469 writesame 0x7c158b6 thru 0x7c1e741 data_size 0x6(0x8e8c bytes) 2024-09-14T06:30:04.503 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-14T06:30:04.503 INFO:teuthology.orchestra.run.smithi094.stdout:2471 punch from 0x3668590 to 0x366f638, (0x70a8 bytes) 2024-09-14T06:30:04.504 INFO:teuthology.orchestra.run.smithi094.stdout:2472 read 0x8437f1 thru 0x84b489 (0x7c99 bytes) 2024-09-14T06:30:04.513 INFO:teuthology.orchestra.run.smithi094.stdout:osd.3: {} 2024-09-14T06:30:04.513 INFO:teuthology.orchestra.run.smithi094.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-14T06:30:04.513 INFO:teuthology.orchestra.run.smithi094.stdout:2473 trunc from 0x94ccd98 to 0x1649b3a 2024-09-14T06:30:04.524 INFO:tasks.ceph.osd.0.smithi094.stderr:2024-09-14T06:30:04.523+0000 7f07c9333640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33636) UID: 0 2024-09-14T06:30:04.532 INFO:teuthology.orchestra.run.smithi094.stdout:osd.4: {} 2024-09-14T06:30:04.532 INFO:teuthology.orchestra.run.smithi094.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-14T06:30:04.533 INFO:teuthology.orchestra.run.smithi094.stdout:2474 write 0x7d20c94 thru 0x7d2ae08 (0xa175 bytes) 2024-09-14T06:30:04.540 INFO:teuthology.orchestra.run.smithi094.stdout:2476 trunc from 0x7d2ae09 to 0x3408a69 2024-09-14T06:30:04.548 INFO:teuthology.orchestra.run.smithi094.stdout:2477 read 0x13e42e2 thru 0x13ef8d3 (0xb5f2 bytes) 2024-09-14T06:30:04.551 INFO:teuthology.orchestra.run.smithi094.stdout:osd.5: {} 2024-09-14T06:30:04.551 INFO:teuthology.orchestra.run.smithi094.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-14T06:30:04.561 INFO:teuthology.orchestra.run.smithi094.stdout:2479 write 0x91d0520 thru 0x91d1336 (0xe17 bytes) 2024-09-14T06:30:04.562 INFO:teuthology.orchestra.run.smithi094.stdout:2480 punch from 0x265cb10 to 0x265fd86, (0x3276 bytes) 2024-09-14T06:30:04.563 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-14T06:30:04.563 INFO:teuthology.orchestra.run.smithi094.stdout:2482 write 0x5508207 thru 0x5511ee3 (0x9cdd bytes) 2024-09-14T06:30:04.564 INFO:teuthology.orchestra.run.smithi094.stdout:2483 write 0x3f0acc9 thru 0x3f125f7 (0x792f bytes) 2024-09-14T06:30:04.565 INFO:teuthology.orchestra.run.smithi094.stdout:2484 read 0x8186227 thru 0x818b814 (0x55ee bytes) 2024-09-14T06:30:04.566 INFO:teuthology.orchestra.run.smithi094.stdout:2485 trunc from 0x91d1337 to 0x9e4443a 2024-09-14T06:30:04.569 INFO:teuthology.orchestra.run.smithi094.stdout:osd.6: {} 2024-09-14T06:30:04.569 INFO:teuthology.orchestra.run.smithi094.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-14T06:30:04.572 INFO:teuthology.orchestra.run.smithi094.stdout:2487 writesame 0x47beeab thru 0x47bfbf0 data_size 0x1(0xd46 bytes) 2024-09-14T06:30:04.574 INFO:teuthology.orchestra.run.smithi094.stdout:2488 read 0x7d100af thru 0x7d110e7 (0x1039 bytes) 2024-09-14T06:30:04.574 INFO:teuthology.orchestra.run.smithi094.stdout:2490 write 0x2184e64 thru 0x21921ad (0xd34a bytes) 2024-09-14T06:30:04.576 INFO:teuthology.orchestra.run.smithi094.stdout:2491 write 0x8427c6f thru 0x843799a (0xfd2c bytes) 2024-09-14T06:30:04.584 INFO:teuthology.orchestra.run.smithi094.stdout:2493 writesame 0xaa66d6e thru 0xaa6765a data_size 0x1(0x8ed bytes) 2024-09-14T06:30:04.585 INFO:teuthology.orchestra.run.smithi094.stdout:2494 write 0x3e190ed thru 0x3e1b849 (0x275d bytes) 2024-09-14T06:30:04.587 INFO:teuthology.orchestra.run.smithi094.stdout:2495 write 0xa0058de thru 0xa00877c (0x2e9f bytes) 2024-09-14T06:30:04.588 INFO:teuthology.orchestra.run.smithi094.stdout:2497 write 0x1aefb6f thru 0x1af4440 (0x48d2 bytes) 2024-09-14T06:30:04.589 INFO:teuthology.orchestra.run.smithi094.stdout:osd.7: {} 2024-09-14T06:30:04.589 INFO:teuthology.orchestra.run.smithi094.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-14T06:30:04.590 INFO:teuthology.orchestra.run.smithi094.stdout:2498 write 0x59f000d thru 0x59f25b6 (0x25aa bytes) 2024-09-14T06:30:04.591 INFO:teuthology.orchestra.run.smithi094.stdout:2499 punch from 0x28550ae to 0x285d3ec, (0x833e bytes) 2024-09-14T06:30:04.591 INFO:teuthology.orchestra.run.smithi094.stdout:2500 read 0x911ebb6 thru 0x912d336 (0xe781 bytes) 2024-09-14T06:30:04.591 INFO:teuthology.orchestra.run.smithi094.stdout:2501 read 0x78eab87 thru 0x78f6eae (0xc328 bytes) 2024-09-14T06:30:04.602 INFO:teuthology.orchestra.run.smithi094.stdout:2502 trunc from 0xaa6765b to 0x3e04701 2024-09-14T06:30:04.625 INFO:tasks.ceph.osd.7.smithi138.stderr:2024-09-14T06:30:04.624+0000 7f3c6a21d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33623) UID: 0 2024-09-14T06:30:04.627 INFO:teuthology.orchestra.run.smithi094.stdout:2505 write 0xe2b3f8d thru 0xe2b69c6 (0x2a3a bytes) 2024-09-14T06:30:04.628 INFO:teuthology.orchestra.run.smithi094.stdout:2506 write 0x620d8a0 thru 0x6218edd (0xb63e bytes) 2024-09-14T06:30:04.629 INFO:teuthology.orchestra.run.smithi094.stdout:2507 punch from 0x5c42355 to 0x5c46b24, (0x47cf bytes) 2024-09-14T06:30:04.630 INFO:teuthology.orchestra.run.smithi094.stdout:2508 writesame 0x619338 thru 0x61dda8 data_size 0x1(0x4a71 bytes) 2024-09-14T06:30:04.635 INFO:teuthology.orchestra.run.smithi138.stdout:ERROR: (22) Invalid argument 2024-09-14T06:30:04.636 INFO:teuthology.orchestra.run.smithi138.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:30:04.638 INFO:teuthology.orchestra.run.smithi094.stdout:2510 write 0xc6a1f74 thru 0xc6ad2d9 (0xb366 bytes) 2024-09-14T06:30:04.645 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-14T06:30:04.647 INFO:teuthology.orchestra.run.smithi094.stdout:2511 trunc from 0xe2b69c7 to 0x74eb1af 2024-09-14T06:30:04.654 INFO:teuthology.orchestra.run.smithi094.stdout:2512 clone 17 order 18 su 4096 sc 5 2024-09-14T06:30:04.729 INFO:tasks.ceph.osd.6.smithi138.stderr:2024-09-14T06:30:04.728+0000 7f8cd8849640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33620) UID: 0 2024-09-14T06:30:04.789 INFO:teuthology.orchestra.run.smithi138.stdout:ERROR: (22) Invalid argument 2024-09-14T06:30:04.789 INFO:teuthology.orchestra.run.smithi138.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:30:04.798 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-14T06:30:04.820 INFO:teuthology.orchestra.run.smithi094.stderr:reweighted osd.6 to 1 (10000) 2024-09-14T06:30:04.834 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 7 1 2024-09-14T06:30:04.840 INFO:tasks.ceph.osd.5.smithi138.stderr:2024-09-14T06:30:04.839+0000 7fe13746f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33618) UID: 0 2024-09-14T06:30:04.928 INFO:tasks.ceph.osd.7.smithi138.stderr:2024-09-14T06:30:04.927+0000 7f3c6a21d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33623) UID: 0 2024-09-14T06:30:04.944 INFO:teuthology.orchestra.run.smithi138.stdout:ERROR: (22) Invalid argument 2024-09-14T06:30:04.944 INFO:teuthology.orchestra.run.smithi138.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:30:04.952 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-14T06:30:05.037 INFO:tasks.ceph.osd.6.smithi138.stderr:2024-09-14T06:30:05.036+0000 7f8cd8849640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33620) UID: 0 2024-09-14T06:30:05.096 INFO:teuthology.orchestra.run.smithi138.stdout:ERROR: (22) Invalid argument 2024-09-14T06:30:05.096 INFO:teuthology.orchestra.run.smithi138.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:30:05.104 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-14T06:30:05.130 INFO:tasks.ceph.osd.3.smithi094.stderr:2024-09-14T06:30:05.129+0000 7feeeafe8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33655) UID: 0 2024-09-14T06:30:05.231 INFO:tasks.ceph.osd.4.smithi138.stderr:2024-09-14T06:30:05.230+0000 7f6c653c8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33615) UID: 0 2024-09-14T06:30:05.248 INFO:teuthology.orchestra.run.smithi138.stdout:ERROR: (22) Invalid argument 2024-09-14T06:30:05.249 INFO:teuthology.orchestra.run.smithi138.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:30:05.258 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-14T06:30:05.331 INFO:tasks.ceph.osd.0.smithi094.stderr:2024-09-14T06:30:05.330+0000 7f07c9333640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33636) UID: 0 2024-09-14T06:30:05.401 INFO:teuthology.orchestra.run.smithi138.stdout:ERROR: (22) Invalid argument 2024-09-14T06:30:05.402 INFO:teuthology.orchestra.run.smithi138.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:30:05.410 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-14T06:30:05.454 INFO:tasks.ceph.osd.7.smithi138.stderr:2024-09-14T06:30:05.453+0000 7f3c6a21d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33623) UID: 0 2024-09-14T06:30:05.513 INFO:teuthology.orchestra.run.smithi138.stdout:ERROR: (22) Invalid argument 2024-09-14T06:30:05.513 INFO:teuthology.orchestra.run.smithi138.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:30:05.521 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-14T06:30:05.534 INFO:tasks.ceph.osd.2.smithi094.stderr:2024-09-14T06:30:05.533+0000 7faf3269b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33649) UID: 0 2024-09-14T06:30:05.635 INFO:tasks.ceph.osd.6.smithi138.stderr:2024-09-14T06:30:05.634+0000 7f8cd8849640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33620) UID: 0 2024-09-14T06:30:05.665 INFO:teuthology.orchestra.run.smithi138.stdout:ERROR: (22) Invalid argument 2024-09-14T06:30:05.665 INFO:teuthology.orchestra.run.smithi138.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:30:05.674 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-14T06:30:05.736 INFO:tasks.ceph.osd.1.smithi094.stderr:2024-09-14T06:30:05.735+0000 7f328e47a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33641) UID: 0 2024-09-14T06:30:05.780 INFO:teuthology.orchestra.run.smithi094.stdout:ERROR: (22) Invalid argument 2024-09-14T06:30:05.780 INFO:teuthology.orchestra.run.smithi094.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:30:05.789 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-14T06:30:05.825 INFO:teuthology.orchestra.run.smithi094.stderr:reweighted osd.7 to 1 (10000) 2024-09-14T06:30:05.837 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-14T06:30:05.841 INFO:tasks.ceph.ceph_manager.ceph:waiting for recovery to complete 2024-09-14T06:30:05.841 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-14T06:30:05.841 INFO:tasks.ceph.osd.1.smithi094.stderr:2024-09-14T06:30:05.839+0000 7f328e47a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33641) UID: 0 2024-09-14T06:30:05.937 INFO:tasks.ceph.osd.1.smithi094.stderr:2024-09-14T06:30:05.936+0000 7f328e47a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33641) UID: 0 2024-09-14T06:30:05.948 INFO:teuthology.orchestra.run.smithi094.stdout:ERROR: (22) Invalid argument 2024-09-14T06:30:05.948 INFO:teuthology.orchestra.run.smithi094.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:30:05.961 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-14T06:30:06.040 INFO:tasks.ceph.osd.7.smithi138.stderr:2024-09-14T06:30:06.039+0000 7f3c6a21d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33623) UID: 0 2024-09-14T06:30:06.141 INFO:tasks.ceph.osd.1.smithi094.stderr:2024-09-14T06:30:06.140+0000 7f328e47a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33641) UID: 0 2024-09-14T06:30:06.163 INFO:teuthology.orchestra.run.smithi094.stdout:ERROR: (22) Invalid argument 2024-09-14T06:30:06.163 INFO:teuthology.orchestra.run.smithi094.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:30:06.174 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-14T06:30:06.242 INFO:tasks.ceph.osd.4.smithi138.stderr:2024-09-14T06:30:06.240+0000 7f6c653c8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33615) UID: 0 2024-09-14T06:30:06.311 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-14T06:30:06.312 INFO:teuthology.orchestra.run.smithi094.stderr:dumped all 2024-09-14T06:30:06.330 INFO:teuthology.orchestra.run.smithi094.stdout:{"pg_ready":true,"pg_map":{"version":121,"stamp":"2024-09-14T06:30:05.825126+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":866286807,"num_objects":860,"num_object_clones":0,"num_object_copies":1720,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":860,"num_whiteouts":0,"num_read":80712,"num_read_kb":739961,"num_write":9202,"num_write_kb":526422,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":39,"num_bytes_recovered":1400964,"num_keys_recovered":11,"num_objects_omap":18,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":6540,"ondisk_log_size":6540,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":374916,"kb_used_data":61588,"kb_used_omap":12,"kb_used_meta":313139,"kb_avail":754599804,"statfs":{"total":773094113280,"available":772710199296,"internally_reserved":0,"allocated":63066112,"data_stored":536665407,"data_compressed":32938624,"data_compressed_allocated":43614208,"data_compressed_original":523887248,"omap_allocated":12751,"internal_metadata":320654897},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":71425945,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":1461,"num_read_kb":134888,"num_write":321,"num_write_kb":49951,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.244324"},"pg_stats":[{"pgid":"3.1f","version":"81'229","reported_seq":2685,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-14T06:30:02.808588+0000","last_change":"2024-09-14T06:30:01.583714+0000","last_active":"2024-09-14T06:30:02.808588+0000","last_peered":"2024-09-14T06:30:02.808588+0000","last_clean":"2024-09-14T06:30:02.808588+0000","last_became_active":"2024-09-14T06:29:00.579883+0000","last_became_peered":"2024-09-14T06:29:00.579883+0000","last_unstale":"2024-09-14T06:30:02.808588+0000","last_undegraded":"2024-09-14T06:30:02.808588+0000","last_fullsized":"2024-09-14T06:30:02.808588+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":229,"log_dups_size":0,"ondisk_log_size":229,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T17:52:49.735970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001043078,"stat_sum":{"num_bytes":22726718,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":2963,"num_read_kb":23663,"num_write":389,"num_write_kb":11216,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"20","length":"1"},{"start":"22","length":"1"}]},{"pgid":"3.1e","version":"80'66","reported_seq":2738,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-14T06:30:00.655359+0000","last_change":"2024-09-14T06:29:58.930339+0000","last_active":"2024-09-14T06:30:00.655359+0000","last_peered":"2024-09-14T06:30:00.655359+0000","last_clean":"2024-09-14T06:30:00.655359+0000","last_became_active":"2024-09-14T06:29:00.580507+0000","last_became_peered":"2024-09-14T06:29:00.580507+0000","last_unstale":"2024-09-14T06:30:00.655359+0000","last_undegraded":"2024-09-14T06:30:00.655359+0000","last_fullsized":"2024-09-14T06:30:00.655359+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":66,"log_dups_size":0,"ondisk_log_size":66,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T09:05:14.289994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000808777,"stat_sum":{"num_bytes":24669861,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":0,"num_read":2444,"num_read_kb":19651,"num_write":111,"num_write_kb":16097,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"20","length":"1"}]},{"pgid":"3.1d","version":"77'401","reported_seq":4607,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-14T06:30:03.831117+0000","last_change":"2024-09-14T06:30:01.588016+0000","last_active":"2024-09-14T06:30:03.831117+0000","last_peered":"2024-09-14T06:30:03.831117+0000","last_clean":"2024-09-14T06:30:03.831117+0000","last_became_active":"2024-09-14T06:29:00.579345+0000","last_became_peered":"2024-09-14T06:29:00.579345+0000","last_unstale":"2024-09-14T06:30:03.831117+0000","last_undegraded":"2024-09-14T06:30:03.831117+0000","last_fullsized":"2024-09-14T06:30:03.831117+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":401,"log_dups_size":0,"ondisk_log_size":401,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T16:13:35.506526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00046322099999999999,"stat_sum":{"num_bytes":40054615,"num_objects":25,"num_object_clones":0,"num_object_copies":50,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":0,"num_read":5418,"num_read_kb":30785,"num_write":641,"num_write_kb":15193,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"20","length":"1"},{"start":"22","length":"1"}]},{"pgid":"3.1c","version":"79'190","reported_seq":2183,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-14T06:30:03.830966+0000","last_change":"2024-09-14T06:30:01.588259+0000","last_active":"2024-09-14T06:30:03.830966+0000","last_peered":"2024-09-14T06:30:03.830966+0000","last_clean":"2024-09-14T06:30:03.830966+0000","last_became_active":"2024-09-14T06:29:00.579807+0000","last_became_peered":"2024-09-14T06:29:00.579807+0000","last_unstale":"2024-09-14T06:30:03.830966+0000","last_undegraded":"2024-09-14T06:30:03.830966+0000","last_fullsized":"2024-09-14T06:30:03.830966+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":190,"log_dups_size":0,"ondisk_log_size":190,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T11:59:38.940641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061292200000000001,"stat_sum":{"num_bytes":14789931,"num_objects":31,"num_object_clones":0,"num_object_copies":62,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":31,"num_whiteouts":0,"num_read":2299,"num_read_kb":20022,"num_write":250,"num_write_kb":15894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"20","length":"1"},{"start":"22","length":"1"}]},{"pgid":"3.1b","version":"77'87","reported_seq":2288,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-14T06:30:02.808717+0000","last_change":"2024-09-14T06:30:01.583647+0000","last_active":"2024-09-14T06:30:02.808717+0000","last_peered":"2024-09-14T06:30:02.808717+0000","last_clean":"2024-09-14T06:30:02.808717+0000","last_became_active":"2024-09-14T06:29:00.586183+0000","last_became_peered":"2024-09-14T06:29:00.586183+0000","last_unstale":"2024-09-14T06:30:02.808717+0000","last_undegraded":"2024-09-14T06:30:02.808717+0000","last_fullsized":"2024-09-14T06:30:02.808717+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":87,"log_dups_size":0,"ondisk_log_size":87,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T11:00:24.039624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010736809999999999,"stat_sum":{"num_bytes":16863005,"num_objects":34,"num_object_clones":0,"num_object_copies":68,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":34,"num_whiteouts":0,"num_read":1961,"num_read_kb":18184,"num_write":105,"num_write_kb":14147,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"20","length":"1"},{"start":"22","length":"1"}]},{"pgid":"3.1a","version":"82'721","reported_seq":2396,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-14T06:30:03.825148+0000","last_change":"2024-09-14T06:30:01.585441+0000","last_active":"2024-09-14T06:30:03.825148+0000","last_peered":"2024-09-14T06:30:03.825148+0000","last_clean":"2024-09-14T06:30:03.825148+0000","last_became_active":"2024-09-14T06:29:00.576206+0000","last_became_peered":"2024-09-14T06:29:00.576206+0000","last_unstale":"2024-09-14T06:30:03.825148+0000","last_undegraded":"2024-09-14T06:30:03.825148+0000","last_fullsized":"2024-09-14T06:30:03.825148+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":721,"log_dups_size":0,"ondisk_log_size":721,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T14:17:30.310989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066857199999999998,"stat_sum":{"num_bytes":32208779,"num_objects":37,"num_object_clones":0,"num_object_copies":74,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":37,"num_whiteouts":0,"num_read":4361,"num_read_kb":21811,"num_write":1093,"num_write_kb":21479,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"20","length":"1"},{"start":"22","length":"1"}]},{"pgid":"3.19","version":"79'193","reported_seq":2493,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-14T06:30:03.817895+0000","last_change":"2024-09-14T06:30:01.706276+0000","last_active":"2024-09-14T06:30:03.817895+0000","last_peered":"2024-09-14T06:30:03.817895+0000","last_clean":"2024-09-14T06:30:03.817895+0000","last_became_active":"2024-09-14T06:29:00.575639+0000","last_became_peered":"2024-09-14T06:29:00.575639+0000","last_unstale":"2024-09-14T06:30:03.817895+0000","last_undegraded":"2024-09-14T06:30:03.817895+0000","last_fullsized":"2024-09-14T06:30:03.817895+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":193,"log_dups_size":0,"ondisk_log_size":193,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T18:09:49.882842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000497135,"stat_sum":{"num_bytes":31419494,"num_objects":28,"num_object_clones":0,"num_object_copies":56,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":28,"num_whiteouts":0,"num_read":2619,"num_read_kb":35762,"num_write":292,"num_write_kb":17729,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"20","length":"1"},{"start":"22","length":"1"}]},{"pgid":"3.18","version":"77'105","reported_seq":2014,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-14T06:30:00.654686+0000","last_change":"2024-09-14T06:29:58.930424+0000","last_active":"2024-09-14T06:30:00.654686+0000","last_peered":"2024-09-14T06:30:00.654686+0000","last_clean":"2024-09-14T06:30:00.654686+0000","last_became_active":"2024-09-14T06:29:00.578017+0000","last_became_peered":"2024-09-14T06:29:00.578017+0000","last_unstale":"2024-09-14T06:30:00.654686+0000","last_undegraded":"2024-09-14T06:30:00.654686+0000","last_fullsized":"2024-09-14T06:30:00.654686+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":105,"log_dups_size":0,"ondisk_log_size":105,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T09:22:45.229396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088733499999999995,"stat_sum":{"num_bytes":20393701,"num_objects":27,"num_object_clones":0,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":27,"num_whiteouts":0,"num_read":1740,"num_read_kb":21992,"num_write":116,"num_write_kb":13739,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"20","length":"1"}]},{"pgid":"3.17","version":"77'303","reported_seq":6030,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-14T06:30:02.808682+0000","last_change":"2024-09-14T06:30:01.583877+0000","last_active":"2024-09-14T06:30:02.808682+0000","last_peered":"2024-09-14T06:30:02.808682+0000","last_clean":"2024-09-14T06:30:02.808682+0000","last_became_active":"2024-09-14T06:29:00.579714+0000","last_became_peered":"2024-09-14T06:29:00.579714+0000","last_unstale":"2024-09-14T06:30:02.808682+0000","last_undegraded":"2024-09-14T06:30:02.808682+0000","last_fullsized":"2024-09-14T06:30:02.808682+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":303,"log_dups_size":0,"ondisk_log_size":303,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T09:15:35.532551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061936499999999998,"stat_sum":{"num_bytes":46911255,"num_objects":39,"num_object_clones":0,"num_object_copies":78,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":39,"num_whiteouts":0,"num_read":6465,"num_read_kb":38839,"num_write":355,"num_write_kb":18194,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"20","length":"1"},{"start":"22","length":"1"}]},{"pgid":"3.16","version":"77'97","reported_seq":647,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-14T06:30:03.831035+0000","last_change":"2024-09-14T06:30:01.587991+0000","last_active":"2024-09-14T06:30:03.831035+0000","last_peered":"2024-09-14T06:30:03.831035+0000","last_clean":"2024-09-14T06:30:03.831035+0000","last_became_active":"2024-09-14T06:29:00.580070+0000","last_became_peered":"2024-09-14T06:29:00.580070+0000","last_unstale":"2024-09-14T06:30:03.831035+0000","last_undegraded":"2024-09-14T06:30:03.831035+0000","last_fullsized":"2024-09-14T06:30:03.831035+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T16:15:49.162288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00044688399999999999,"stat_sum":{"num_bytes":12401689,"num_objects":20,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":0,"num_read":366,"num_read_kb":6863,"num_write":138,"num_write_kb":8227,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"20","length":"1"},{"start":"22","length":"1"}]},{"pgid":"3.15","version":"82'175","reported_seq":1981,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-14T06:30:03.827812+0000","last_change":"2024-09-14T06:30:01.782915+0000","last_active":"2024-09-14T06:30:03.827812+0000","last_peered":"2024-09-14T06:30:03.827812+0000","last_clean":"2024-09-14T06:30:03.827812+0000","last_became_active":"2024-09-14T06:29:00.578451+0000","last_became_peered":"2024-09-14T06:29:00.578451+0000","last_unstale":"2024-09-14T06:30:03.827812+0000","last_undegraded":"2024-09-14T06:30:03.827812+0000","last_fullsized":"2024-09-14T06:30:03.827812+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":175,"log_dups_size":0,"ondisk_log_size":175,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T09:10:11.045999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.011822360000000001,"stat_sum":{"num_bytes":32116574,"num_objects":31,"num_object_clones":0,"num_object_copies":62,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":31,"num_whiteouts":0,"num_read":1740,"num_read_kb":39431,"num_write":194,"num_write_kb":29018,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"20","length":"1"},{"start":"22","length":"1"}]},{"pgid":"3.14","version":"77'602","reported_seq":2392,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-14T06:30:03.825318+0000","last_change":"2024-09-14T06:30:01.585471+0000","last_active":"2024-09-14T06:30:03.825318+0000","last_peered":"2024-09-14T06:30:03.825318+0000","last_clean":"2024-09-14T06:30:03.825318+0000","last_became_active":"2024-09-14T06:29:00.582595+0000","last_became_peered":"2024-09-14T06:29:00.582595+0000","last_unstale":"2024-09-14T06:30:03.825318+0000","last_undegraded":"2024-09-14T06:30:03.825318+0000","last_fullsized":"2024-09-14T06:30:03.825318+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":602,"log_dups_size":0,"ondisk_log_size":602,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T17:35:26.891395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059378199999999995,"stat_sum":{"num_bytes":15411299,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":5001,"num_read_kb":10129,"num_write":968,"num_write_kb":8386,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"20","length":"1"},{"start":"22","length":"1"}]},{"pgid":"2.7","version":"0'0","reported_seq":140,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-14T06:30:03.822182+0000","last_change":"2024-09-14T06:28:53.625193+0000","last_active":"2024-09-14T06:30:03.822182+0000","last_peered":"2024-09-14T06:30:03.822182+0000","last_clean":"2024-09-14T06:30:03.822182+0000","last_became_active":"2024-09-14T06:28:51.223535+0000","last_became_peered":"2024-09-14T06:28:51.223535+0000","last_unstale":"2024-09-14T06:30:03.822182+0000","last_undegraded":"2024-09-14T06:30:03.822182+0000","last_fullsized":"2024-09-14T06:30:03.822182+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:28:50.205350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:28:50.205350+0000","last_clean_scrub_stamp":"2024-09-14T06:28:50.205350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T18:14:22.822245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00052660300000000005,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"77'154","reported_seq":1389,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-14T06:30:02.808617+0000","last_change":"2024-09-14T06:30:01.583078+0000","last_active":"2024-09-14T06:30:02.808617+0000","last_peered":"2024-09-14T06:30:02.808617+0000","last_clean":"2024-09-14T06:30:02.808617+0000","last_became_active":"2024-09-14T06:29:00.578283+0000","last_became_peered":"2024-09-14T06:29:00.578283+0000","last_unstale":"2024-09-14T06:30:02.808617+0000","last_undegraded":"2024-09-14T06:30:02.808617+0000","last_fullsized":"2024-09-14T06:30:02.808617+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":154,"log_dups_size":0,"ondisk_log_size":154,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T15:38:49.773742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00099433300000000002,"stat_sum":{"num_bytes":13549074,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":1254,"num_read_kb":6973,"num_write":166,"num_write_kb":10842,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"20","length":"1"},{"start":"22","length":"1"}]},{"pgid":"2.6","version":"0'0","reported_seq":142,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-14T06:30:03.818124+0000","last_change":"2024-09-14T06:28:53.226122+0000","last_active":"2024-09-14T06:30:03.818124+0000","last_peered":"2024-09-14T06:30:03.818124+0000","last_clean":"2024-09-14T06:30:03.818124+0000","last_became_active":"2024-09-14T06:28:51.217205+0000","last_became_peered":"2024-09-14T06:28:51.217205+0000","last_unstale":"2024-09-14T06:30:03.818124+0000","last_undegraded":"2024-09-14T06:30:03.818124+0000","last_fullsized":"2024-09-14T06:30:03.818124+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:28:50.205350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:28:50.205350+0000","last_clean_scrub_stamp":"2024-09-14T06:28:50.205350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T15:29:19.563302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065018200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"77'266","reported_seq":1304,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-14T06:30:02.808486+0000","last_change":"2024-09-14T06:30:01.583402+0000","last_active":"2024-09-14T06:30:02.808486+0000","last_peered":"2024-09-14T06:30:02.808486+0000","last_clean":"2024-09-14T06:30:02.808486+0000","last_became_active":"2024-09-14T06:29:56.584268+0000","last_became_peered":"2024-09-14T06:29:56.584268+0000","last_unstale":"2024-09-14T06:30:02.808486+0000","last_undegraded":"2024-09-14T06:30:02.808486+0000","last_fullsized":"2024-09-14T06:30:02.808486+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":266,"log_dups_size":0,"ondisk_log_size":266,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T16:14:09.318332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0009848630000000001,"stat_sum":{"num_bytes":22196057,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":1570,"num_read_kb":8713,"num_write":400,"num_write_kb":11202,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":39,"num_bytes_recovered":1400964,"num_keys_recovered":11,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"20","length":"1"},{"start":"22","length":"1"}]},{"pgid":"2.5","version":"0'0","reported_seq":142,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-14T06:30:03.827987+0000","last_change":"2024-09-14T06:28:53.640035+0000","last_active":"2024-09-14T06:30:03.827987+0000","last_peered":"2024-09-14T06:30:03.827987+0000","last_clean":"2024-09-14T06:30:03.827987+0000","last_became_active":"2024-09-14T06:28:51.308058+0000","last_became_peered":"2024-09-14T06:28:51.308058+0000","last_unstale":"2024-09-14T06:30:03.827987+0000","last_undegraded":"2024-09-14T06:30:03.827987+0000","last_fullsized":"2024-09-14T06:30:03.827987+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:28:50.205350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:28:50.205350+0000","last_clean_scrub_stamp":"2024-09-14T06:28:50.205350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T13:35:48.544057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00037824000000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"74'90","reported_seq":670,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-14T06:30:03.817904+0000","last_change":"2024-09-14T06:30:01.706202+0000","last_active":"2024-09-14T06:30:03.817904+0000","last_peered":"2024-09-14T06:30:03.817904+0000","last_clean":"2024-09-14T06:30:03.817904+0000","last_became_active":"2024-09-14T06:29:00.574895+0000","last_became_peered":"2024-09-14T06:29:00.574895+0000","last_unstale":"2024-09-14T06:30:03.817904+0000","last_undegraded":"2024-09-14T06:30:03.817904+0000","last_fullsized":"2024-09-14T06:30:03.817904+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":90,"log_dups_size":0,"ondisk_log_size":90,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T15:07:22.923485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039147900000000002,"stat_sum":{"num_bytes":9345071,"num_objects":20,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":0,"num_read":497,"num_read_kb":8381,"num_write":127,"num_write_kb":7683,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"20","length":"1"},{"start":"22","length":"1"}]},{"pgid":"2.4","version":"0'0","reported_seq":142,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-14T06:30:03.817981+0000","last_change":"2024-09-14T06:28:53.226072+0000","last_active":"2024-09-14T06:30:03.817981+0000","last_peered":"2024-09-14T06:30:03.817981+0000","last_clean":"2024-09-14T06:30:03.817981+0000","last_became_active":"2024-09-14T06:28:51.308072+0000","last_became_peered":"2024-09-14T06:28:51.308072+0000","last_unstale":"2024-09-14T06:30:03.817981+0000","last_undegraded":"2024-09-14T06:30:03.817981+0000","last_fullsized":"2024-09-14T06:30:03.817981+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:28:50.205350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:28:50.205350+0000","last_clean_scrub_stamp":"2024-09-14T06:28:50.205350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T13:15:09.052328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066197399999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"83'150","reported_seq":943,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-14T06:30:03.831148+0000","last_change":"2024-09-14T06:30:01.588466+0000","last_active":"2024-09-14T06:30:03.831148+0000","last_peered":"2024-09-14T06:30:03.831148+0000","last_clean":"2024-09-14T06:30:03.831148+0000","last_became_active":"2024-09-14T06:29:00.581874+0000","last_became_peered":"2024-09-14T06:29:00.581874+0000","last_unstale":"2024-09-14T06:30:03.831148+0000","last_undegraded":"2024-09-14T06:30:03.831148+0000","last_fullsized":"2024-09-14T06:30:03.831148+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":150,"log_dups_size":0,"ondisk_log_size":150,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T18:02:48.518500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069224200000000003,"stat_sum":{"num_bytes":25483813,"num_objects":25,"num_object_clones":0,"num_object_copies":50,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":0,"num_read":559,"num_read_kb":7790,"num_write":171,"num_write_kb":19042,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"20","length":"1"},{"start":"22","length":"1"}]},{"pgid":"3.3","version":"82'144","reported_seq":4731,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-14T06:30:03.825210+0000","last_change":"2024-09-14T06:30:01.585689+0000","last_active":"2024-09-14T06:30:03.825210+0000","last_peered":"2024-09-14T06:30:03.825210+0000","last_clean":"2024-09-14T06:30:03.825210+0000","last_became_active":"2024-09-14T06:29:00.584678+0000","last_became_peered":"2024-09-14T06:29:00.584678+0000","last_unstale":"2024-09-14T06:30:03.825210+0000","last_undegraded":"2024-09-14T06:30:03.825210+0000","last_fullsized":"2024-09-14T06:30:03.825210+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":144,"log_dups_size":0,"ondisk_log_size":144,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T12:11:29.858753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00073073599999999995,"stat_sum":{"num_bytes":38201242,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":4583,"num_read_kb":46886,"num_write":148,"num_write_kb":24654,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"20","length":"1"},{"start":"22","length":"1"}]},{"pgid":"2.2","version":"22'2","reported_seq":144,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-14T06:30:03.830962+0000","last_change":"2024-09-14T06:28:53.231428+0000","last_active":"2024-09-14T06:30:03.830962+0000","last_peered":"2024-09-14T06:30:03.830962+0000","last_clean":"2024-09-14T06:30:03.830962+0000","last_became_active":"2024-09-14T06:28:51.222937+0000","last_became_peered":"2024-09-14T06:28:51.222937+0000","last_unstale":"2024-09-14T06:30:03.830962+0000","last_undegraded":"2024-09-14T06:30:03.830962+0000","last_fullsized":"2024-09-14T06:30:03.830962+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:28:50.205350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:28:50.205350+0000","last_clean_scrub_stamp":"2024-09-14T06:28:50.205350+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T14:27:37.233976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010072970000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.0","version":"77'106","reported_seq":3585,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-14T06:30:03.818046+0000","last_change":"2024-09-14T06:30:01.706436+0000","last_active":"2024-09-14T06:30:03.818046+0000","last_peered":"2024-09-14T06:30:03.818046+0000","last_clean":"2024-09-14T06:30:03.818046+0000","last_became_active":"2024-09-14T06:29:00.574873+0000","last_became_peered":"2024-09-14T06:29:00.574873+0000","last_unstale":"2024-09-14T06:30:03.818046+0000","last_undegraded":"2024-09-14T06:30:03.818046+0000","last_fullsized":"2024-09-14T06:30:03.818046+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":106,"log_dups_size":0,"ondisk_log_size":106,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T06:47:32.862300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000594219,"stat_sum":{"num_bytes":40518226,"num_objects":36,"num_object_clones":0,"num_object_copies":72,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":36,"num_whiteouts":0,"num_read":3245,"num_read_kb":46318,"num_write":150,"num_write_kb":24391,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"20","length":"1"},{"start":"22","length":"1"}]},{"pgid":"2.1","version":"0'0","reported_seq":137,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-14T06:30:01.750022+0000","last_change":"2024-09-14T06:28:53.226264+0000","last_active":"2024-09-14T06:30:01.750022+0000","last_peered":"2024-09-14T06:30:01.750022+0000","last_clean":"2024-09-14T06:30:01.750022+0000","last_became_active":"2024-09-14T06:28:51.223252+0000","last_became_peered":"2024-09-14T06:28:51.223252+0000","last_unstale":"2024-09-14T06:30:01.750022+0000","last_undegraded":"2024-09-14T06:30:01.750022+0000","last_fullsized":"2024-09-14T06:30:01.750022+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:28:50.205350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:28:50.205350+0000","last_clean_scrub_stamp":"2024-09-14T06:28:50.205350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T11:56:04.373414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053403100000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1","version":"81'147","reported_seq":2336,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-14T06:30:02.808284+0000","last_change":"2024-09-14T06:30:01.583154+0000","last_active":"2024-09-14T06:30:02.808284+0000","last_peered":"2024-09-14T06:30:02.808284+0000","last_clean":"2024-09-14T06:30:02.808284+0000","last_became_active":"2024-09-14T06:29:00.585673+0000","last_became_peered":"2024-09-14T06:29:00.585673+0000","last_unstale":"2024-09-14T06:30:02.808284+0000","last_undegraded":"2024-09-14T06:30:02.808284+0000","last_fullsized":"2024-09-14T06:30:02.808284+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":147,"log_dups_size":0,"ondisk_log_size":147,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T10:23:40.580683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010502129999999999,"stat_sum":{"num_bytes":38469268,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":2289,"num_read_kb":29456,"num_write":174,"num_write_kb":15394,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"20","length":"1"},{"start":"22","length":"1"}]},{"pgid":"2.0","version":"0'0","reported_seq":142,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-14T06:30:03.827666+0000","last_change":"2024-09-14T06:28:53.640096+0000","last_active":"2024-09-14T06:30:03.827666+0000","last_peered":"2024-09-14T06:30:03.827666+0000","last_clean":"2024-09-14T06:30:03.827666+0000","last_became_active":"2024-09-14T06:28:51.223069+0000","last_became_peered":"2024-09-14T06:28:51.223069+0000","last_unstale":"2024-09-14T06:30:03.827666+0000","last_undegraded":"2024-09-14T06:30:03.827666+0000","last_fullsized":"2024-09-14T06:30:03.827666+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:28:50.205350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:28:50.205350+0000","last_clean_scrub_stamp":"2024-09-14T06:28:50.205350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T15:13:00.471509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00043714600000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"80'136","reported_seq":3805,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-14T06:30:00.752852+0000","last_change":"2024-09-14T06:29:58.930688+0000","last_active":"2024-09-14T06:30:00.752852+0000","last_peered":"2024-09-14T06:30:00.752852+0000","last_clean":"2024-09-14T06:30:00.752852+0000","last_became_active":"2024-09-14T06:29:00.578574+0000","last_became_peered":"2024-09-14T06:29:00.578574+0000","last_unstale":"2024-09-14T06:30:00.752852+0000","last_undegraded":"2024-09-14T06:30:00.752852+0000","last_fullsized":"2024-09-14T06:30:00.752852+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":136,"log_dups_size":0,"ondisk_log_size":136,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T07:07:09.250046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001001734,"stat_sum":{"num_bytes":42049226,"num_objects":44,"num_object_clones":0,"num_object_copies":88,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":44,"num_whiteouts":0,"num_read":3458,"num_read_kb":37811,"num_write":182,"num_write_kb":22061,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"20","length":"1"}]},{"pgid":"2.3","version":"20'1","reported_seq":143,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-14T06:30:03.830953+0000","last_change":"2024-09-14T06:28:53.230933+0000","last_active":"2024-09-14T06:30:03.830953+0000","last_peered":"2024-09-14T06:30:03.830953+0000","last_clean":"2024-09-14T06:30:03.830953+0000","last_became_active":"2024-09-14T06:28:51.222874+0000","last_became_peered":"2024-09-14T06:28:51.222874+0000","last_unstale":"2024-09-14T06:30:03.830953+0000","last_undegraded":"2024-09-14T06:30:03.830953+0000","last_fullsized":"2024-09-14T06:30:03.830953+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:28:50.205350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:28:50.205350+0000","last_clean_scrub_stamp":"2024-09-14T06:28:50.205350+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T12:04:25.190171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050808899999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'72","reported_seq":233,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-14T06:30:03.827726+0000","last_change":"2024-09-14T06:28:44.839885+0000","last_active":"2024-09-14T06:30:03.827726+0000","last_peered":"2024-09-14T06:30:03.827726+0000","last_clean":"2024-09-14T06:30:03.827726+0000","last_became_active":"2024-09-14T06:28:44.839573+0000","last_became_peered":"2024-09-14T06:28:44.839573+0000","last_unstale":"2024-09-14T06:30:03.827726+0000","last_undegraded":"2024-09-14T06:30:03.827726+0000","last_fullsized":"2024-09-14T06:30:03.827726+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:28:43.828494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:28:43.828494+0000","last_clean_scrub_stamp":"2024-09-14T06:28:43.828494+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T14:41:25.787293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.8","version":"80'141","reported_seq":3029,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-14T06:30:00.754801+0000","last_change":"2024-09-14T06:29:58.930511+0000","last_active":"2024-09-14T06:30:00.754801+0000","last_peered":"2024-09-14T06:30:00.754801+0000","last_clean":"2024-09-14T06:30:00.754801+0000","last_became_active":"2024-09-14T06:29:00.574458+0000","last_became_peered":"2024-09-14T06:29:00.574458+0000","last_unstale":"2024-09-14T06:30:00.754801+0000","last_undegraded":"2024-09-14T06:30:00.754801+0000","last_fullsized":"2024-09-14T06:30:00.754801+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":141,"log_dups_size":0,"ondisk_log_size":141,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T09:44:54.218044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00091564700000000001,"stat_sum":{"num_bytes":40534367,"num_objects":27,"num_object_clones":0,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":27,"num_whiteouts":0,"num_read":3218,"num_read_kb":45608,"num_write":189,"num_write_kb":26810,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"20","length":"1"}]},{"pgid":"3.9","version":"77'99","reported_seq":1013,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-14T06:30:03.825090+0000","last_change":"2024-09-14T06:30:01.585736+0000","last_active":"2024-09-14T06:30:03.825090+0000","last_peered":"2024-09-14T06:30:03.825090+0000","last_clean":"2024-09-14T06:30:03.825090+0000","last_became_active":"2024-09-14T06:29:00.573179+0000","last_became_peered":"2024-09-14T06:29:00.573179+0000","last_unstale":"2024-09-14T06:30:03.825090+0000","last_undegraded":"2024-09-14T06:30:03.825090+0000","last_fullsized":"2024-09-14T06:30:03.825090+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":99,"log_dups_size":0,"ondisk_log_size":99,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T17:46:12.749590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00072056099999999999,"stat_sum":{"num_bytes":16127302,"num_objects":31,"num_object_clones":0,"num_object_copies":62,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":31,"num_whiteouts":0,"num_read":668,"num_read_kb":22502,"num_write":128,"num_write_kb":17044,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"20","length":"1"},{"start":"22","length":"1"}]},{"pgid":"3.a","version":"78'110","reported_seq":1916,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-14T06:30:03.822146+0000","last_change":"2024-09-14T06:30:01.777988+0000","last_active":"2024-09-14T06:30:03.822146+0000","last_peered":"2024-09-14T06:30:03.822146+0000","last_clean":"2024-09-14T06:30:03.822146+0000","last_became_active":"2024-09-14T06:29:00.582426+0000","last_became_peered":"2024-09-14T06:29:00.582426+0000","last_unstale":"2024-09-14T06:30:03.822146+0000","last_undegraded":"2024-09-14T06:30:03.822146+0000","last_fullsized":"2024-09-14T06:30:03.822146+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":110,"log_dups_size":0,"ondisk_log_size":110,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T07:15:24.069571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00650681,"stat_sum":{"num_bytes":35180168,"num_objects":28,"num_object_clones":0,"num_object_copies":56,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":28,"num_whiteouts":0,"num_read":1770,"num_read_kb":31856,"num_write":140,"num_write_kb":18235,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"20","length":"1"},{"start":"22","length":"1"}]},{"pgid":"3.b","version":"80'114","reported_seq":2082,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-14T06:30:00.754384+0000","last_change":"2024-09-14T06:29:58.929913+0000","last_active":"2024-09-14T06:30:00.754384+0000","last_peered":"2024-09-14T06:30:00.754384+0000","last_clean":"2024-09-14T06:30:00.754384+0000","last_became_active":"2024-09-14T06:29:00.578483+0000","last_became_peered":"2024-09-14T06:29:00.578483+0000","last_unstale":"2024-09-14T06:30:00.754384+0000","last_undegraded":"2024-09-14T06:30:00.754384+0000","last_fullsized":"2024-09-14T06:30:00.754384+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":114,"log_dups_size":0,"ondisk_log_size":114,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T10:09:58.409222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049572899999999998,"stat_sum":{"num_bytes":19633489,"num_objects":28,"num_object_clones":0,"num_object_copies":56,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":28,"num_whiteouts":0,"num_read":1841,"num_read_kb":8727,"num_write":147,"num_write_kb":10060,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"20","length":"1"}]},{"pgid":"3.c","version":"77'105","reported_seq":2703,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-14T06:30:03.830998+0000","last_change":"2024-09-14T06:30:01.588321+0000","last_active":"2024-09-14T06:30:03.830998+0000","last_peered":"2024-09-14T06:30:03.830998+0000","last_clean":"2024-09-14T06:30:03.830998+0000","last_became_active":"2024-09-14T06:29:00.583406+0000","last_became_peered":"2024-09-14T06:29:00.583406+0000","last_unstale":"2024-09-14T06:30:03.830998+0000","last_undegraded":"2024-09-14T06:30:03.830998+0000","last_fullsized":"2024-09-14T06:30:03.830998+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":105,"log_dups_size":0,"ondisk_log_size":105,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T07:55:56.360463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000633751,"stat_sum":{"num_bytes":23576801,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":2352,"num_read_kb":20629,"num_write":139,"num_write_kb":14435,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"20","length":"1"},{"start":"22","length":"1"}]},{"pgid":"3.d","version":"77'131","reported_seq":2170,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-14T06:30:03.827867+0000","last_change":"2024-09-14T06:30:01.782915+0000","last_active":"2024-09-14T06:30:03.827867+0000","last_peered":"2024-09-14T06:30:03.827867+0000","last_clean":"2024-09-14T06:30:03.827867+0000","last_became_active":"2024-09-14T06:29:00.580362+0000","last_became_peered":"2024-09-14T06:29:00.580362+0000","last_unstale":"2024-09-14T06:30:03.827867+0000","last_undegraded":"2024-09-14T06:30:03.827867+0000","last_fullsized":"2024-09-14T06:30:03.827867+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":131,"log_dups_size":0,"ondisk_log_size":131,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T09:08:21.070574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.011537659,"stat_sum":{"num_bytes":24008950,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":2026,"num_read_kb":19581,"num_write":178,"num_write_kb":13060,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"20","length":"1"},{"start":"22","length":"1"}]},{"pgid":"3.e","version":"77'175","reported_seq":938,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-14T06:30:03.827811+0000","last_change":"2024-09-14T06:30:01.772841+0000","last_active":"2024-09-14T06:30:03.827811+0000","last_peered":"2024-09-14T06:30:03.827811+0000","last_clean":"2024-09-14T06:30:03.827811+0000","last_became_active":"2024-09-14T06:29:00.586374+0000","last_became_peered":"2024-09-14T06:29:00.586374+0000","last_unstale":"2024-09-14T06:30:03.827811+0000","last_undegraded":"2024-09-14T06:30:03.827811+0000","last_fullsized":"2024-09-14T06:30:03.827811+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":175,"log_dups_size":0,"ondisk_log_size":175,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T07:09:30.313393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002035986,"stat_sum":{"num_bytes":12789513,"num_objects":28,"num_object_clones":0,"num_object_copies":56,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":28,"num_whiteouts":0,"num_read":933,"num_read_kb":8881,"num_write":292,"num_write_kb":11167,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"20","length":"1"},{"start":"22","length":"1"}]},{"pgid":"3.f","version":"71'133","reported_seq":2814,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-14T06:30:03.827867+0000","last_change":"2024-09-14T06:30:01.772286+0000","last_active":"2024-09-14T06:30:03.827867+0000","last_peered":"2024-09-14T06:30:03.827867+0000","last_clean":"2024-09-14T06:30:03.827867+0000","last_became_active":"2024-09-14T06:29:00.586491+0000","last_became_peered":"2024-09-14T06:29:00.586491+0000","last_unstale":"2024-09-14T06:30:03.827867+0000","last_undegraded":"2024-09-14T06:30:03.827867+0000","last_fullsized":"2024-09-14T06:30:03.827867+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":133,"log_dups_size":0,"ondisk_log_size":133,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T10:41:46.283525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021298290000000002,"stat_sum":{"num_bytes":15703454,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":0,"num_read":2680,"num_read_kb":14801,"num_write":194,"num_write_kb":9645,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"20","length":"1"},{"start":"22","length":"1"}]},{"pgid":"3.10","version":"82'162","reported_seq":835,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-14T06:30:03.822146+0000","last_change":"2024-09-14T06:30:01.771917+0000","last_active":"2024-09-14T06:30:03.822146+0000","last_peered":"2024-09-14T06:30:03.822146+0000","last_clean":"2024-09-14T06:30:03.822146+0000","last_became_active":"2024-09-14T06:29:00.579539+0000","last_became_peered":"2024-09-14T06:29:00.579539+0000","last_unstale":"2024-09-14T06:30:03.822146+0000","last_undegraded":"2024-09-14T06:30:03.822146+0000","last_fullsized":"2024-09-14T06:30:03.822146+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":162,"log_dups_size":0,"ondisk_log_size":162,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T12:28:41.506185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00084384200000000003,"stat_sum":{"num_bytes":15460541,"num_objects":20,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":0,"num_read":455,"num_read_kb":4173,"num_write":204,"num_write_kb":15611,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"20","length":"1"},{"start":"22","length":"1"}]},{"pgid":"3.11","version":"82'265","reported_seq":3744,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-14T06:30:03.827737+0000","last_change":"2024-09-14T06:30:01.770588+0000","last_active":"2024-09-14T06:30:03.827737+0000","last_peered":"2024-09-14T06:30:03.827737+0000","last_clean":"2024-09-14T06:30:03.827737+0000","last_became_active":"2024-09-14T06:29:00.586480+0000","last_became_peered":"2024-09-14T06:29:00.586480+0000","last_unstale":"2024-09-14T06:30:03.827737+0000","last_undegraded":"2024-09-14T06:30:03.827737+0000","last_fullsized":"2024-09-14T06:30:03.827737+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":265,"log_dups_size":0,"ondisk_log_size":265,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T13:26:16.448079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00086384899999999998,"stat_sum":{"num_bytes":48470041,"num_objects":34,"num_object_clones":0,"num_object_copies":68,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":34,"num_whiteouts":0,"num_read":3710,"num_read_kb":41711,"num_write":338,"num_write_kb":25233,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"20","length":"1"},{"start":"22","length":"1"}]},{"pgid":"3.12","version":"81'308","reported_seq":2679,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-14T06:30:02.808768+0000","last_change":"2024-09-14T06:30:01.583385+0000","last_active":"2024-09-14T06:30:02.808768+0000","last_peered":"2024-09-14T06:30:02.808768+0000","last_clean":"2024-09-14T06:30:02.808768+0000","last_became_active":"2024-09-14T06:29:00.579826+0000","last_became_peered":"2024-09-14T06:29:00.579826+0000","last_unstale":"2024-09-14T06:30:02.808768+0000","last_undegraded":"2024-09-14T06:30:02.808768+0000","last_fullsized":"2024-09-14T06:30:02.808768+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":308,"log_dups_size":0,"ondisk_log_size":308,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T10:32:15.909854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012732069999999999,"stat_sum":{"num_bytes":29535665,"num_objects":25,"num_object_clones":0,"num_object_copies":50,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":0,"num_read":3264,"num_read_kb":18201,"num_write":448,"num_write_kb":16842,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"20","length":"1"},{"start":"22","length":"1"}]},{"pgid":"3.13","version":"83'360","reported_seq":2593,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-14T06:30:03.827957+0000","last_change":"2024-09-14T06:30:01.772456+0000","last_active":"2024-09-14T06:30:03.827957+0000","last_peered":"2024-09-14T06:30:03.827957+0000","last_clean":"2024-09-14T06:30:03.827957+0000","last_became_active":"2024-09-14T06:29:00.586520+0000","last_became_peered":"2024-09-14T06:29:00.586520+0000","last_unstale":"2024-09-14T06:30:03.827957+0000","last_undegraded":"2024-09-14T06:30:03.827957+0000","last_fullsized":"2024-09-14T06:30:03.827957+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":360,"log_dups_size":0,"ondisk_log_size":360,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T17:37:24.551610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026580280000000002,"stat_sum":{"num_bytes":44897231,"num_objects":28,"num_object_clones":0,"num_object_copies":56,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":28,"num_whiteouts":0,"num_read":2897,"num_read_kb":23776,"num_write":532,"num_write_kb":22062,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"20","length":"1"},{"start":"22","length":"1"}]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":865696420,"num_objects":856,"num_object_clones":0,"num_object_copies":1712,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":856,"num_whiteouts":0,"num_read":80646,"num_read_kb":739906,"num_write":9099,"num_write_kb":524792,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":39,"num_bytes_recovered":1400964,"num_keys_recovered":11,"num_objects_omap":18,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":56561664,"data_stored":546554082,"data_compressed":33720813,"data_compressed_allocated":44613632,"data_compressed_original":537246924,"omap_allocated":0,"internal_metadata":0},"log_size":6465,"ondisk_log_size":6465,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":131072,"data_stored":1180736,"data_compressed":65166,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509459,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":47824,"kb_used_data":8312,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94324016,"statfs":{"total":96636764160,"available":96587792384,"internally_reserved":0,"allocated":8511488,"data_stored":70958167,"data_compressed":4271172,"data_compressed_allocated":5627904,"data_compressed_original":68968491,"omap_allocated":1590,"internal_metadata":40434122},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509458,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":34164,"kb_used_data":2976,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94337676,"statfs":{"total":96636764160,"available":96601780224,"internally_reserved":0,"allocated":3047424,"data_stored":20175802,"data_compressed":1133495,"data_compressed_allocated":1564672,"data_compressed_original":19298897,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509460,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":43648,"kb_used_data":8304,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94328192,"statfs":{"total":96636764160,"available":96592068608,"internally_reserved":0,"allocated":8503296,"data_stored":67118852,"data_compressed":4256238,"data_compressed_allocated":5693440,"data_compressed_original":65237623,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509459,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":69796,"kb_used_data":12240,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94302044,"statfs":{"total":96636764160,"available":96565293056,"internally_reserved":0,"allocated":12533760,"data_stored":115396563,"data_compressed":7242765,"data_compressed_allocated":9498624,"data_compressed_original":113404550,"omap_allocated":1595,"internal_metadata":58915269},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509460,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":46220,"kb_used_data":6712,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94325620,"statfs":{"total":96636764160,"available":96589434880,"internally_reserved":0,"allocated":6873088,"data_stored":60377977,"data_compressed":3718315,"data_compressed_allocated":4911104,"data_compressed_original":59188469,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":14,"seq":60129542163,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":35008,"kb_used_data":3812,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94336832,"statfs":{"total":96636764160,"available":96600915968,"internally_reserved":0,"allocated":3903488,"data_stored":30643001,"data_compressed":1912399,"data_compressed_allocated":2539520,"data_compressed_original":29909705,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542162,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":48592,"kb_used_data":9084,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94323248,"statfs":{"total":96636764160,"available":96587005952,"internally_reserved":0,"allocated":9302016,"data_stored":78299366,"data_compressed":4714808,"data_compressed_allocated":6209536,"data_compressed_original":76133817,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574867,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":49664,"kb_used_data":10148,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94322176,"statfs":{"total":96636764160,"available":96585908224,"internally_reserved":0,"allocated":10391552,"data_stored":93695679,"data_compressed":5689432,"data_compressed_allocated":7569408,"data_compressed_original":91745696,"omap_allocated":1593,"internal_metadata":40434119},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32583,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32583,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8429568,"data_stored":77438064,"data_compressed":4685347,"data_compressed_allocated":6172672,"data_compressed_original":75625913,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":5935104,"data_stored":60002176,"data_compressed":3721229,"data_compressed_allocated":4919296,"data_compressed_original":59249909,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":2957312,"data_stored":30204542,"data_compressed":1912399,"data_compressed_allocated":2539520,"data_compressed_original":29909705,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":10534912,"data_stored":107521186,"data_compressed":6524172,"data_compressed_allocated":8630272,"data_compressed_original":105995680,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":11620352,"data_stored":115003250,"data_compressed":7244126,"data_compressed_allocated":9502720,"data_compressed_original":113428388,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":7540736,"data_stored":66721711,"data_compressed":4258559,"data_compressed_allocated":5697536,"data_compressed_original":65286775,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":2031616,"data_stored":19677920,"data_compressed":1133495,"data_compressed_allocated":1564672,"data_compressed_original":19298897,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":7512064,"data_stored":69985233,"data_compressed":4241486,"data_compressed_allocated":5586944,"data_compressed_original":68451657,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-14T06:30:06.334 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-14T06:30:06.361 INFO:teuthology.orchestra.run.smithi094.stdout:ERROR: (22) Invalid argument 2024-09-14T06:30:06.361 INFO:teuthology.orchestra.run.smithi094.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:30:06.361 INFO:tasks.ceph.osd.2.smithi094.stderr:2024-09-14T06:30:06.360+0000 7faf3269b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33649) UID: 0 2024-09-14T06:30:06.372 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-14T06:30:06.456 INFO:tasks.ceph.osd.2.smithi094.stderr:2024-09-14T06:30:06.455+0000 7faf3269b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33649) UID: 0 2024-09-14T06:30:06.544 INFO:tasks.ceph.osd.7.smithi138.stderr:2024-09-14T06:30:06.543+0000 7f3c6a21d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33623) UID: 0 2024-09-14T06:30:06.565 INFO:teuthology.orchestra.run.smithi094.stdout:ERROR: (22) Invalid argument 2024-09-14T06:30:06.565 INFO:teuthology.orchestra.run.smithi094.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:30:06.579 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-14T06:30:06.613 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-14T06:30:06.664 INFO:tasks.ceph.osd.0.smithi094.stderr:2024-09-14T06:30:06.663+0000 7f07c9333640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33636) UID: 0 2024-09-14T06:30:06.745 INFO:tasks.ceph.osd.4.smithi138.stderr:2024-09-14T06:30:06.744+0000 7f6c653c8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33615) UID: 0 2024-09-14T06:30:06.787 INFO:teuthology.orchestra.run.smithi094.stdout:ERROR: (22) Invalid argument 2024-09-14T06:30:06.787 INFO:teuthology.orchestra.run.smithi094.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:30:06.788 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-14T06:30:06.790 INFO:teuthology.orchestra.run.smithi094.stderr:dumped all 2024-09-14T06:30:06.793 INFO:teuthology.orchestra.run.smithi094.stdout:osd.0: {} 2024-09-14T06:30:06.794 INFO:teuthology.orchestra.run.smithi094.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-14T06:30:06.798 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-14T06:30:06.805 INFO:teuthology.orchestra.run.smithi094.stdout:{"pg_ready":true,"pg_map":{"version":121,"stamp":"2024-09-14T06:30:05.825126+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":866286807,"num_objects":860,"num_object_clones":0,"num_object_copies":1720,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":860,"num_whiteouts":0,"num_read":80712,"num_read_kb":739961,"num_write":9202,"num_write_kb":526422,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":39,"num_bytes_recovered":1400964,"num_keys_recovered":11,"num_objects_omap":18,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":6540,"ondisk_log_size":6540,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":374916,"kb_used_data":61588,"kb_used_omap":12,"kb_used_meta":313139,"kb_avail":754599804,"statfs":{"total":773094113280,"available":772710199296,"internally_reserved":0,"allocated":63066112,"data_stored":536665407,"data_compressed":32938624,"data_compressed_allocated":43614208,"data_compressed_original":523887248,"omap_allocated":12751,"internal_metadata":320654897},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":71425945,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":1461,"num_read_kb":134888,"num_write":321,"num_write_kb":49951,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.244324"},"pg_stats":[{"pgid":"3.1f","version":"81'229","reported_seq":2685,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-14T06:30:02.808588+0000","last_change":"2024-09-14T06:30:01.583714+0000","last_active":"2024-09-14T06:30:02.808588+0000","last_peered":"2024-09-14T06:30:02.808588+0000","last_clean":"2024-09-14T06:30:02.808588+0000","last_became_active":"2024-09-14T06:29:00.579883+0000","last_became_peered":"2024-09-14T06:29:00.579883+0000","last_unstale":"2024-09-14T06:30:02.808588+0000","last_undegraded":"2024-09-14T06:30:02.808588+0000","last_fullsized":"2024-09-14T06:30:02.808588+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":229,"log_dups_size":0,"ondisk_log_size":229,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T17:52:49.735970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001043078,"stat_sum":{"num_bytes":22726718,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":2963,"num_read_kb":23663,"num_write":389,"num_write_kb":11216,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"20","length":"1"},{"start":"22","length":"1"}]},{"pgid":"3.1e","version":"80'66","reported_seq":2738,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-14T06:30:00.655359+0000","last_change":"2024-09-14T06:29:58.930339+0000","last_active":"2024-09-14T06:30:00.655359+0000","last_peered":"2024-09-14T06:30:00.655359+0000","last_clean":"2024-09-14T06:30:00.655359+0000","last_became_active":"2024-09-14T06:29:00.580507+0000","last_became_peered":"2024-09-14T06:29:00.580507+0000","last_unstale":"2024-09-14T06:30:00.655359+0000","last_undegraded":"2024-09-14T06:30:00.655359+0000","last_fullsized":"2024-09-14T06:30:00.655359+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":66,"log_dups_size":0,"ondisk_log_size":66,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T09:05:14.289994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000808777,"stat_sum":{"num_bytes":24669861,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":0,"num_read":2444,"num_read_kb":19651,"num_write":111,"num_write_kb":16097,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"20","length":"1"}]},{"pgid":"3.1d","version":"77'401","reported_seq":4607,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-14T06:30:03.831117+0000","last_change":"2024-09-14T06:30:01.588016+0000","last_active":"2024-09-14T06:30:03.831117+0000","last_peered":"2024-09-14T06:30:03.831117+0000","last_clean":"2024-09-14T06:30:03.831117+0000","last_became_active":"2024-09-14T06:29:00.579345+0000","last_became_peered":"2024-09-14T06:29:00.579345+0000","last_unstale":"2024-09-14T06:30:03.831117+0000","last_undegraded":"2024-09-14T06:30:03.831117+0000","last_fullsized":"2024-09-14T06:30:03.831117+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":401,"log_dups_size":0,"ondisk_log_size":401,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T16:13:35.506526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00046322099999999999,"stat_sum":{"num_bytes":40054615,"num_objects":25,"num_object_clones":0,"num_object_copies":50,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":0,"num_read":5418,"num_read_kb":30785,"num_write":641,"num_write_kb":15193,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"20","length":"1"},{"start":"22","length":"1"}]},{"pgid":"3.1c","version":"79'190","reported_seq":2183,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-14T06:30:03.830966+0000","last_change":"2024-09-14T06:30:01.588259+0000","last_active":"2024-09-14T06:30:03.830966+0000","last_peered":"2024-09-14T06:30:03.830966+0000","last_clean":"2024-09-14T06:30:03.830966+0000","last_became_active":"2024-09-14T06:29:00.579807+0000","last_became_peered":"2024-09-14T06:29:00.579807+0000","last_unstale":"2024-09-14T06:30:03.830966+0000","last_undegraded":"2024-09-14T06:30:03.830966+0000","last_fullsized":"2024-09-14T06:30:03.830966+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":190,"log_dups_size":0,"ondisk_log_size":190,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T11:59:38.940641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061292200000000001,"stat_sum":{"num_bytes":14789931,"num_objects":31,"num_object_clones":0,"num_object_copies":62,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":31,"num_whiteouts":0,"num_read":2299,"num_read_kb":20022,"num_write":250,"num_write_kb":15894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"20","length":"1"},{"start":"22","length":"1"}]},{"pgid":"3.1b","version":"77'87","reported_seq":2288,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-14T06:30:02.808717+0000","last_change":"2024-09-14T06:30:01.583647+0000","last_active":"2024-09-14T06:30:02.808717+0000","last_peered":"2024-09-14T06:30:02.808717+0000","last_clean":"2024-09-14T06:30:02.808717+0000","last_became_active":"2024-09-14T06:29:00.586183+0000","last_became_peered":"2024-09-14T06:29:00.586183+0000","last_unstale":"2024-09-14T06:30:02.808717+0000","last_undegraded":"2024-09-14T06:30:02.808717+0000","last_fullsized":"2024-09-14T06:30:02.808717+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":87,"log_dups_size":0,"ondisk_log_size":87,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T11:00:24.039624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010736809999999999,"stat_sum":{"num_bytes":16863005,"num_objects":34,"num_object_clones":0,"num_object_copies":68,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":34,"num_whiteouts":0,"num_read":1961,"num_read_kb":18184,"num_write":105,"num_write_kb":14147,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"20","length":"1"},{"start":"22","length":"1"}]},{"pgid":"3.1a","version":"82'721","reported_seq":2396,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-14T06:30:03.825148+0000","last_change":"2024-09-14T06:30:01.585441+0000","last_active":"2024-09-14T06:30:03.825148+0000","last_peered":"2024-09-14T06:30:03.825148+0000","last_clean":"2024-09-14T06:30:03.825148+0000","last_became_active":"2024-09-14T06:29:00.576206+0000","last_became_peered":"2024-09-14T06:29:00.576206+0000","last_unstale":"2024-09-14T06:30:03.825148+0000","last_undegraded":"2024-09-14T06:30:03.825148+0000","last_fullsized":"2024-09-14T06:30:03.825148+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":721,"log_dups_size":0,"ondisk_log_size":721,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T14:17:30.310989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066857199999999998,"stat_sum":{"num_bytes":32208779,"num_objects":37,"num_object_clones":0,"num_object_copies":74,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":37,"num_whiteouts":0,"num_read":4361,"num_read_kb":21811,"num_write":1093,"num_write_kb":21479,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"20","length":"1"},{"start":"22","length":"1"}]},{"pgid":"3.19","version":"79'193","reported_seq":2493,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-14T06:30:03.817895+0000","last_change":"2024-09-14T06:30:01.706276+0000","last_active":"2024-09-14T06:30:03.817895+0000","last_peered":"2024-09-14T06:30:03.817895+0000","last_clean":"2024-09-14T06:30:03.817895+0000","last_became_active":"2024-09-14T06:29:00.575639+0000","last_became_peered":"2024-09-14T06:29:00.575639+0000","last_unstale":"2024-09-14T06:30:03.817895+0000","last_undegraded":"2024-09-14T06:30:03.817895+0000","last_fullsized":"2024-09-14T06:30:03.817895+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":193,"log_dups_size":0,"ondisk_log_size":193,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T18:09:49.882842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000497135,"stat_sum":{"num_bytes":31419494,"num_objects":28,"num_object_clones":0,"num_object_copies":56,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":28,"num_whiteouts":0,"num_read":2619,"num_read_kb":35762,"num_write":292,"num_write_kb":17729,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"20","length":"1"},{"start":"22","length":"1"}]},{"pgid":"3.18","version":"77'105","reported_seq":2014,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-14T06:30:00.654686+0000","last_change":"2024-09-14T06:29:58.930424+0000","last_active":"2024-09-14T06:30:00.654686+0000","last_peered":"2024-09-14T06:30:00.654686+0000","last_clean":"2024-09-14T06:30:00.654686+0000","last_became_active":"2024-09-14T06:29:00.578017+0000","last_became_peered":"2024-09-14T06:29:00.578017+0000","last_unstale":"2024-09-14T06:30:00.654686+0000","last_undegraded":"2024-09-14T06:30:00.654686+0000","last_fullsized":"2024-09-14T06:30:00.654686+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":105,"log_dups_size":0,"ondisk_log_size":105,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T09:22:45.229396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088733499999999995,"stat_sum":{"num_bytes":20393701,"num_objects":27,"num_object_clones":0,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":27,"num_whiteouts":0,"num_read":1740,"num_read_kb":21992,"num_write":116,"num_write_kb":13739,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"20","length":"1"}]},{"pgid":"3.17","version":"77'303","reported_seq":6030,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-14T06:30:02.808682+0000","last_change":"2024-09-14T06:30:01.583877+0000","last_active":"2024-09-14T06:30:02.808682+0000","last_peered":"2024-09-14T06:30:02.808682+0000","last_clean":"2024-09-14T06:30:02.808682+0000","last_became_active":"2024-09-14T06:29:00.579714+0000","last_became_peered":"2024-09-14T06:29:00.579714+0000","last_unstale":"2024-09-14T06:30:02.808682+0000","last_undegraded":"2024-09-14T06:30:02.808682+0000","last_fullsized":"2024-09-14T06:30:02.808682+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":303,"log_dups_size":0,"ondisk_log_size":303,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T09:15:35.532551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061936499999999998,"stat_sum":{"num_bytes":46911255,"num_objects":39,"num_object_clones":0,"num_object_copies":78,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":39,"num_whiteouts":0,"num_read":6465,"num_read_kb":38839,"num_write":355,"num_write_kb":18194,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"20","length":"1"},{"start":"22","length":"1"}]},{"pgid":"3.16","version":"77'97","reported_seq":647,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-14T06:30:03.831035+0000","last_change":"2024-09-14T06:30:01.587991+0000","last_active":"2024-09-14T06:30:03.831035+0000","last_peered":"2024-09-14T06:30:03.831035+0000","last_clean":"2024-09-14T06:30:03.831035+0000","last_became_active":"2024-09-14T06:29:00.580070+0000","last_became_peered":"2024-09-14T06:29:00.580070+0000","last_unstale":"2024-09-14T06:30:03.831035+0000","last_undegraded":"2024-09-14T06:30:03.831035+0000","last_fullsized":"2024-09-14T06:30:03.831035+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T16:15:49.162288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00044688399999999999,"stat_sum":{"num_bytes":12401689,"num_objects":20,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":0,"num_read":366,"num_read_kb":6863,"num_write":138,"num_write_kb":8227,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"20","length":"1"},{"start":"22","length":"1"}]},{"pgid":"3.15","version":"82'175","reported_seq":1981,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-14T06:30:03.827812+0000","last_change":"2024-09-14T06:30:01.782915+0000","last_active":"2024-09-14T06:30:03.827812+0000","last_peered":"2024-09-14T06:30:03.827812+0000","last_clean":"2024-09-14T06:30:03.827812+0000","last_became_active":"2024-09-14T06:29:00.578451+0000","last_became_peered":"2024-09-14T06:29:00.578451+0000","last_unstale":"2024-09-14T06:30:03.827812+0000","last_undegraded":"2024-09-14T06:30:03.827812+0000","last_fullsized":"2024-09-14T06:30:03.827812+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":175,"log_dups_size":0,"ondisk_log_size":175,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T09:10:11.045999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.011822360000000001,"stat_sum":{"num_bytes":32116574,"num_objects":31,"num_object_clones":0,"num_object_copies":62,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":31,"num_whiteouts":0,"num_read":1740,"num_read_kb":39431,"num_write":194,"num_write_kb":29018,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"20","length":"1"},{"start":"22","length":"1"}]},{"pgid":"3.14","version":"77'602","reported_seq":2392,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-14T06:30:03.825318+0000","last_change":"2024-09-14T06:30:01.585471+0000","last_active":"2024-09-14T06:30:03.825318+0000","last_peered":"2024-09-14T06:30:03.825318+0000","last_clean":"2024-09-14T06:30:03.825318+0000","last_became_active":"2024-09-14T06:29:00.582595+0000","last_became_peered":"2024-09-14T06:29:00.582595+0000","last_unstale":"2024-09-14T06:30:03.825318+0000","last_undegraded":"2024-09-14T06:30:03.825318+0000","last_fullsized":"2024-09-14T06:30:03.825318+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":602,"log_dups_size":0,"ondisk_log_size":602,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T17:35:26.891395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059378199999999995,"stat_sum":{"num_bytes":15411299,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":5001,"num_read_kb":10129,"num_write":968,"num_write_kb":8386,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"20","length":"1"},{"start":"22","length":"1"}]},{"pgid":"2.7","version":"0'0","reported_seq":140,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-14T06:30:03.822182+0000","last_change":"2024-09-14T06:28:53.625193+0000","last_active":"2024-09-14T06:30:03.822182+0000","last_peered":"2024-09-14T06:30:03.822182+0000","last_clean":"2024-09-14T06:30:03.822182+0000","last_became_active":"2024-09-14T06:28:51.223535+0000","last_became_peered":"2024-09-14T06:28:51.223535+0000","last_unstale":"2024-09-14T06:30:03.822182+0000","last_undegraded":"2024-09-14T06:30:03.822182+0000","last_fullsized":"2024-09-14T06:30:03.822182+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:28:50.205350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:28:50.205350+0000","last_clean_scrub_stamp":"2024-09-14T06:28:50.205350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T18:14:22.822245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00052660300000000005,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"77'154","reported_seq":1389,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-14T06:30:02.808617+0000","last_change":"2024-09-14T06:30:01.583078+0000","last_active":"2024-09-14T06:30:02.808617+0000","last_peered":"2024-09-14T06:30:02.808617+0000","last_clean":"2024-09-14T06:30:02.808617+0000","last_became_active":"2024-09-14T06:29:00.578283+0000","last_became_peered":"2024-09-14T06:29:00.578283+0000","last_unstale":"2024-09-14T06:30:02.808617+0000","last_undegraded":"2024-09-14T06:30:02.808617+0000","last_fullsized":"2024-09-14T06:30:02.808617+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":154,"log_dups_size":0,"ondisk_log_size":154,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T15:38:49.773742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00099433300000000002,"stat_sum":{"num_bytes":13549074,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":1254,"num_read_kb":6973,"num_write":166,"num_write_kb":10842,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"20","length":"1"},{"start":"22","length":"1"}]},{"pgid":"2.6","version":"0'0","reported_seq":142,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-14T06:30:03.818124+0000","last_change":"2024-09-14T06:28:53.226122+0000","last_active":"2024-09-14T06:30:03.818124+0000","last_peered":"2024-09-14T06:30:03.818124+0000","last_clean":"2024-09-14T06:30:03.818124+0000","last_became_active":"2024-09-14T06:28:51.217205+0000","last_became_peered":"2024-09-14T06:28:51.217205+0000","last_unstale":"2024-09-14T06:30:03.818124+0000","last_undegraded":"2024-09-14T06:30:03.818124+0000","last_fullsized":"2024-09-14T06:30:03.818124+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:28:50.205350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:28:50.205350+0000","last_clean_scrub_stamp":"2024-09-14T06:28:50.205350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T15:29:19.563302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065018200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"77'266","reported_seq":1304,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-14T06:30:02.808486+0000","last_change":"2024-09-14T06:30:01.583402+0000","last_active":"2024-09-14T06:30:02.808486+0000","last_peered":"2024-09-14T06:30:02.808486+0000","last_clean":"2024-09-14T06:30:02.808486+0000","last_became_active":"2024-09-14T06:29:56.584268+0000","last_became_peered":"2024-09-14T06:29:56.584268+0000","last_unstale":"2024-09-14T06:30:02.808486+0000","last_undegraded":"2024-09-14T06:30:02.808486+0000","last_fullsized":"2024-09-14T06:30:02.808486+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":266,"log_dups_size":0,"ondisk_log_size":266,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T16:14:09.318332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0009848630000000001,"stat_sum":{"num_bytes":22196057,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":1570,"num_read_kb":8713,"num_write":400,"num_write_kb":11202,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":39,"num_bytes_recovered":1400964,"num_keys_recovered":11,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"20","length":"1"},{"start":"22","length":"1"}]},{"pgid":"2.5","version":"0'0","reported_seq":142,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-14T06:30:03.827987+0000","last_change":"2024-09-14T06:28:53.640035+0000","last_active":"2024-09-14T06:30:03.827987+0000","last_peered":"2024-09-14T06:30:03.827987+0000","last_clean":"2024-09-14T06:30:03.827987+0000","last_became_active":"2024-09-14T06:28:51.308058+0000","last_became_peered":"2024-09-14T06:28:51.308058+0000","last_unstale":"2024-09-14T06:30:03.827987+0000","last_undegraded":"2024-09-14T06:30:03.827987+0000","last_fullsized":"2024-09-14T06:30:03.827987+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:28:50.205350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:28:50.205350+0000","last_clean_scrub_stamp":"2024-09-14T06:28:50.205350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T13:35:48.544057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00037824000000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"74'90","reported_seq":670,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-14T06:30:03.817904+0000","last_change":"2024-09-14T06:30:01.706202+0000","last_active":"2024-09-14T06:30:03.817904+0000","last_peered":"2024-09-14T06:30:03.817904+0000","last_clean":"2024-09-14T06:30:03.817904+0000","last_became_active":"2024-09-14T06:29:00.574895+0000","last_became_peered":"2024-09-14T06:29:00.574895+0000","last_unstale":"2024-09-14T06:30:03.817904+0000","last_undegraded":"2024-09-14T06:30:03.817904+0000","last_fullsized":"2024-09-14T06:30:03.817904+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":90,"log_dups_size":0,"ondisk_log_size":90,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T15:07:22.923485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039147900000000002,"stat_sum":{"num_bytes":9345071,"num_objects":20,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":0,"num_read":497,"num_read_kb":8381,"num_write":127,"num_write_kb":7683,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"20","length":"1"},{"start":"22","length":"1"}]},{"pgid":"2.4","version":"0'0","reported_seq":142,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-14T06:30:03.817981+0000","last_change":"2024-09-14T06:28:53.226072+0000","last_active":"2024-09-14T06:30:03.817981+0000","last_peered":"2024-09-14T06:30:03.817981+0000","last_clean":"2024-09-14T06:30:03.817981+0000","last_became_active":"2024-09-14T06:28:51.308072+0000","last_became_peered":"2024-09-14T06:28:51.308072+0000","last_unstale":"2024-09-14T06:30:03.817981+0000","last_undegraded":"2024-09-14T06:30:03.817981+0000","last_fullsized":"2024-09-14T06:30:03.817981+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:28:50.205350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:28:50.205350+0000","last_clean_scrub_stamp":"2024-09-14T06:28:50.205350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T13:15:09.052328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066197399999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"83'150","reported_seq":943,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-14T06:30:03.831148+0000","last_change":"2024-09-14T06:30:01.588466+0000","last_active":"2024-09-14T06:30:03.831148+0000","last_peered":"2024-09-14T06:30:03.831148+0000","last_clean":"2024-09-14T06:30:03.831148+0000","last_became_active":"2024-09-14T06:29:00.581874+0000","last_became_peered":"2024-09-14T06:29:00.581874+0000","last_unstale":"2024-09-14T06:30:03.831148+0000","last_undegraded":"2024-09-14T06:30:03.831148+0000","last_fullsized":"2024-09-14T06:30:03.831148+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":150,"log_dups_size":0,"ondisk_log_size":150,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T18:02:48.518500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069224200000000003,"stat_sum":{"num_bytes":25483813,"num_objects":25,"num_object_clones":0,"num_object_copies":50,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":0,"num_read":559,"num_read_kb":7790,"num_write":171,"num_write_kb":19042,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"20","length":"1"},{"start":"22","length":"1"}]},{"pgid":"3.3","version":"82'144","reported_seq":4731,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-14T06:30:03.825210+0000","last_change":"2024-09-14T06:30:01.585689+0000","last_active":"2024-09-14T06:30:03.825210+0000","last_peered":"2024-09-14T06:30:03.825210+0000","last_clean":"2024-09-14T06:30:03.825210+0000","last_became_active":"2024-09-14T06:29:00.584678+0000","last_became_peered":"2024-09-14T06:29:00.584678+0000","last_unstale":"2024-09-14T06:30:03.825210+0000","last_undegraded":"2024-09-14T06:30:03.825210+0000","last_fullsized":"2024-09-14T06:30:03.825210+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":144,"log_dups_size":0,"ondisk_log_size":144,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T12:11:29.858753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00073073599999999995,"stat_sum":{"num_bytes":38201242,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":4583,"num_read_kb":46886,"num_write":148,"num_write_kb":24654,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"20","length":"1"},{"start":"22","length":"1"}]},{"pgid":"2.2","version":"22'2","reported_seq":144,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-14T06:30:03.830962+0000","last_change":"2024-09-14T06:28:53.231428+0000","last_active":"2024-09-14T06:30:03.830962+0000","last_peered":"2024-09-14T06:30:03.830962+0000","last_clean":"2024-09-14T06:30:03.830962+0000","last_became_active":"2024-09-14T06:28:51.222937+0000","last_became_peered":"2024-09-14T06:28:51.222937+0000","last_unstale":"2024-09-14T06:30:03.830962+0000","last_undegraded":"2024-09-14T06:30:03.830962+0000","last_fullsized":"2024-09-14T06:30:03.830962+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:28:50.205350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:28:50.205350+0000","last_clean_scrub_stamp":"2024-09-14T06:28:50.205350+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T14:27:37.233976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010072970000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.0","version":"77'106","reported_seq":3585,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-14T06:30:03.818046+0000","last_change":"2024-09-14T06:30:01.706436+0000","last_active":"2024-09-14T06:30:03.818046+0000","last_peered":"2024-09-14T06:30:03.818046+0000","last_clean":"2024-09-14T06:30:03.818046+0000","last_became_active":"2024-09-14T06:29:00.574873+0000","last_became_peered":"2024-09-14T06:29:00.574873+0000","last_unstale":"2024-09-14T06:30:03.818046+0000","last_undegraded":"2024-09-14T06:30:03.818046+0000","last_fullsized":"2024-09-14T06:30:03.818046+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":106,"log_dups_size":0,"ondisk_log_size":106,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T06:47:32.862300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000594219,"stat_sum":{"num_bytes":40518226,"num_objects":36,"num_object_clones":0,"num_object_copies":72,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":36,"num_whiteouts":0,"num_read":3245,"num_read_kb":46318,"num_write":150,"num_write_kb":24391,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"20","length":"1"},{"start":"22","length":"1"}]},{"pgid":"2.1","version":"0'0","reported_seq":137,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-14T06:30:01.750022+0000","last_change":"2024-09-14T06:28:53.226264+0000","last_active":"2024-09-14T06:30:01.750022+0000","last_peered":"2024-09-14T06:30:01.750022+0000","last_clean":"2024-09-14T06:30:01.750022+0000","last_became_active":"2024-09-14T06:28:51.223252+0000","last_became_peered":"2024-09-14T06:28:51.223252+0000","last_unstale":"2024-09-14T06:30:01.750022+0000","last_undegraded":"2024-09-14T06:30:01.750022+0000","last_fullsized":"2024-09-14T06:30:01.750022+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:28:50.205350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:28:50.205350+0000","last_clean_scrub_stamp":"2024-09-14T06:28:50.205350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T11:56:04.373414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053403100000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1","version":"81'147","reported_seq":2336,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-14T06:30:02.808284+0000","last_change":"2024-09-14T06:30:01.583154+0000","last_active":"2024-09-14T06:30:02.808284+0000","last_peered":"2024-09-14T06:30:02.808284+0000","last_clean":"2024-09-14T06:30:02.808284+0000","last_became_active":"2024-09-14T06:29:00.585673+0000","last_became_peered":"2024-09-14T06:29:00.585673+0000","last_unstale":"2024-09-14T06:30:02.808284+0000","last_undegraded":"2024-09-14T06:30:02.808284+0000","last_fullsized":"2024-09-14T06:30:02.808284+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":147,"log_dups_size":0,"ondisk_log_size":147,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T10:23:40.580683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010502129999999999,"stat_sum":{"num_bytes":38469268,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":2289,"num_read_kb":29456,"num_write":174,"num_write_kb":15394,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"20","length":"1"},{"start":"22","length":"1"}]},{"pgid":"2.0","version":"0'0","reported_seq":142,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-14T06:30:03.827666+0000","last_change":"2024-09-14T06:28:53.640096+0000","last_active":"2024-09-14T06:30:03.827666+0000","last_peered":"2024-09-14T06:30:03.827666+0000","last_clean":"2024-09-14T06:30:03.827666+0000","last_became_active":"2024-09-14T06:28:51.223069+0000","last_became_peered":"2024-09-14T06:28:51.223069+0000","last_unstale":"2024-09-14T06:30:03.827666+0000","last_undegraded":"2024-09-14T06:30:03.827666+0000","last_fullsized":"2024-09-14T06:30:03.827666+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:28:50.205350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:28:50.205350+0000","last_clean_scrub_stamp":"2024-09-14T06:28:50.205350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T15:13:00.471509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00043714600000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"80'136","reported_seq":3805,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-14T06:30:00.752852+0000","last_change":"2024-09-14T06:29:58.930688+0000","last_active":"2024-09-14T06:30:00.752852+0000","last_peered":"2024-09-14T06:30:00.752852+0000","last_clean":"2024-09-14T06:30:00.752852+0000","last_became_active":"2024-09-14T06:29:00.578574+0000","last_became_peered":"2024-09-14T06:29:00.578574+0000","last_unstale":"2024-09-14T06:30:00.752852+0000","last_undegraded":"2024-09-14T06:30:00.752852+0000","last_fullsized":"2024-09-14T06:30:00.752852+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":136,"log_dups_size":0,"ondisk_log_size":136,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T07:07:09.250046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001001734,"stat_sum":{"num_bytes":42049226,"num_objects":44,"num_object_clones":0,"num_object_copies":88,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":44,"num_whiteouts":0,"num_read":3458,"num_read_kb":37811,"num_write":182,"num_write_kb":22061,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"20","length":"1"}]},{"pgid":"2.3","version":"20'1","reported_seq":143,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-14T06:30:03.830953+0000","last_change":"2024-09-14T06:28:53.230933+0000","last_active":"2024-09-14T06:30:03.830953+0000","last_peered":"2024-09-14T06:30:03.830953+0000","last_clean":"2024-09-14T06:30:03.830953+0000","last_became_active":"2024-09-14T06:28:51.222874+0000","last_became_peered":"2024-09-14T06:28:51.222874+0000","last_unstale":"2024-09-14T06:30:03.830953+0000","last_undegraded":"2024-09-14T06:30:03.830953+0000","last_fullsized":"2024-09-14T06:30:03.830953+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:28:50.205350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:28:50.205350+0000","last_clean_scrub_stamp":"2024-09-14T06:28:50.205350+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T12:04:25.190171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050808899999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'72","reported_seq":233,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-14T06:30:03.827726+0000","last_change":"2024-09-14T06:28:44.839885+0000","last_active":"2024-09-14T06:30:03.827726+0000","last_peered":"2024-09-14T06:30:03.827726+0000","last_clean":"2024-09-14T06:30:03.827726+0000","last_became_active":"2024-09-14T06:28:44.839573+0000","last_became_peered":"2024-09-14T06:28:44.839573+0000","last_unstale":"2024-09-14T06:30:03.827726+0000","last_undegraded":"2024-09-14T06:30:03.827726+0000","last_fullsized":"2024-09-14T06:30:03.827726+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:28:43.828494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:28:43.828494+0000","last_clean_scrub_stamp":"2024-09-14T06:28:43.828494+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T14:41:25.787293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.8","version":"80'141","reported_seq":3029,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-14T06:30:00.754801+0000","last_change":"2024-09-14T06:29:58.930511+0000","last_active":"2024-09-14T06:30:00.754801+0000","last_peered":"2024-09-14T06:30:00.754801+0000","last_clean":"2024-09-14T06:30:00.754801+0000","last_became_active":"2024-09-14T06:29:00.574458+0000","last_became_peered":"2024-09-14T06:29:00.574458+0000","last_unstale":"2024-09-14T06:30:00.754801+0000","last_undegraded":"2024-09-14T06:30:00.754801+0000","last_fullsized":"2024-09-14T06:30:00.754801+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":141,"log_dups_size":0,"ondisk_log_size":141,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T09:44:54.218044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00091564700000000001,"stat_sum":{"num_bytes":40534367,"num_objects":27,"num_object_clones":0,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":27,"num_whiteouts":0,"num_read":3218,"num_read_kb":45608,"num_write":189,"num_write_kb":26810,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"20","length":"1"}]},{"pgid":"3.9","version":"77'99","reported_seq":1013,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-14T06:30:03.825090+0000","last_change":"2024-09-14T06:30:01.585736+0000","last_active":"2024-09-14T06:30:03.825090+0000","last_peered":"2024-09-14T06:30:03.825090+0000","last_clean":"2024-09-14T06:30:03.825090+0000","last_became_active":"2024-09-14T06:29:00.573179+0000","last_became_peered":"2024-09-14T06:29:00.573179+0000","last_unstale":"2024-09-14T06:30:03.825090+0000","last_undegraded":"2024-09-14T06:30:03.825090+0000","last_fullsized":"2024-09-14T06:30:03.825090+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":99,"log_dups_size":0,"ondisk_log_size":99,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T17:46:12.749590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00072056099999999999,"stat_sum":{"num_bytes":16127302,"num_objects":31,"num_object_clones":0,"num_object_copies":62,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":31,"num_whiteouts":0,"num_read":668,"num_read_kb":22502,"num_write":128,"num_write_kb":17044,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"20","length":"1"},{"start":"22","length":"1"}]},{"pgid":"3.a","version":"78'110","reported_seq":1916,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-14T06:30:03.822146+0000","last_change":"2024-09-14T06:30:01.777988+0000","last_active":"2024-09-14T06:30:03.822146+0000","last_peered":"2024-09-14T06:30:03.822146+0000","last_clean":"2024-09-14T06:30:03.822146+0000","last_became_active":"2024-09-14T06:29:00.582426+0000","last_became_peered":"2024-09-14T06:29:00.582426+0000","last_unstale":"2024-09-14T06:30:03.822146+0000","last_undegraded":"2024-09-14T06:30:03.822146+0000","last_fullsized":"2024-09-14T06:30:03.822146+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":110,"log_dups_size":0,"ondisk_log_size":110,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T07:15:24.069571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00650681,"stat_sum":{"num_bytes":35180168,"num_objects":28,"num_object_clones":0,"num_object_copies":56,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":28,"num_whiteouts":0,"num_read":1770,"num_read_kb":31856,"num_write":140,"num_write_kb":18235,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"20","length":"1"},{"start":"22","length":"1"}]},{"pgid":"3.b","version":"80'114","reported_seq":2082,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-14T06:30:00.754384+0000","last_change":"2024-09-14T06:29:58.929913+0000","last_active":"2024-09-14T06:30:00.754384+0000","last_peered":"2024-09-14T06:30:00.754384+0000","last_clean":"2024-09-14T06:30:00.754384+0000","last_became_active":"2024-09-14T06:29:00.578483+0000","last_became_peered":"2024-09-14T06:29:00.578483+0000","last_unstale":"2024-09-14T06:30:00.754384+0000","last_undegraded":"2024-09-14T06:30:00.754384+0000","last_fullsized":"2024-09-14T06:30:00.754384+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":114,"log_dups_size":0,"ondisk_log_size":114,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T10:09:58.409222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049572899999999998,"stat_sum":{"num_bytes":19633489,"num_objects":28,"num_object_clones":0,"num_object_copies":56,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":28,"num_whiteouts":0,"num_read":1841,"num_read_kb":8727,"num_write":147,"num_write_kb":10060,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"20","length":"1"}]},{"pgid":"3.c","version":"77'105","reported_seq":2703,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-14T06:30:03.830998+0000","last_change":"2024-09-14T06:30:01.588321+0000","last_active":"2024-09-14T06:30:03.830998+0000","last_peered":"2024-09-14T06:30:03.830998+0000","last_clean":"2024-09-14T06:30:03.830998+0000","last_became_active":"2024-09-14T06:29:00.583406+0000","last_became_peered":"2024-09-14T06:29:00.583406+0000","last_unstale":"2024-09-14T06:30:03.830998+0000","last_undegraded":"2024-09-14T06:30:03.830998+0000","last_fullsized":"2024-09-14T06:30:03.830998+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":105,"log_dups_size":0,"ondisk_log_size":105,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T07:55:56.360463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000633751,"stat_sum":{"num_bytes":23576801,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":2352,"num_read_kb":20629,"num_write":139,"num_write_kb":14435,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"20","length":"1"},{"start":"22","length":"1"}]},{"pgid":"3.d","version":"77'131","reported_seq":2170,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-14T06:30:03.827867+0000","last_change":"2024-09-14T06:30:01.782915+0000","last_active":"2024-09-14T06:30:03.827867+0000","last_peered":"2024-09-14T06:30:03.827867+0000","last_clean":"2024-09-14T06:30:03.827867+0000","last_became_active":"2024-09-14T06:29:00.580362+0000","last_became_peered":"2024-09-14T06:29:00.580362+0000","last_unstale":"2024-09-14T06:30:03.827867+0000","last_undegraded":"2024-09-14T06:30:03.827867+0000","last_fullsized":"2024-09-14T06:30:03.827867+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":131,"log_dups_size":0,"ondisk_log_size":131,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T09:08:21.070574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.011537659,"stat_sum":{"num_bytes":24008950,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":2026,"num_read_kb":19581,"num_write":178,"num_write_kb":13060,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"20","length":"1"},{"start":"22","length":"1"}]},{"pgid":"3.e","version":"77'175","reported_seq":938,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-14T06:30:03.827811+0000","last_change":"2024-09-14T06:30:01.772841+0000","last_active":"2024-09-14T06:30:03.827811+0000","last_peered":"2024-09-14T06:30:03.827811+0000","last_clean":"2024-09-14T06:30:03.827811+0000","last_became_active":"2024-09-14T06:29:00.586374+0000","last_became_peered":"2024-09-14T06:29:00.586374+0000","last_unstale":"2024-09-14T06:30:03.827811+0000","last_undegraded":"2024-09-14T06:30:03.827811+0000","last_fullsized":"2024-09-14T06:30:03.827811+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":175,"log_dups_size":0,"ondisk_log_size":175,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T07:09:30.313393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002035986,"stat_sum":{"num_bytes":12789513,"num_objects":28,"num_object_clones":0,"num_object_copies":56,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":28,"num_whiteouts":0,"num_read":933,"num_read_kb":8881,"num_write":292,"num_write_kb":11167,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"20","length":"1"},{"start":"22","length":"1"}]},{"pgid":"3.f","version":"71'133","reported_seq":2814,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-14T06:30:03.827867+0000","last_change":"2024-09-14T06:30:01.772286+0000","last_active":"2024-09-14T06:30:03.827867+0000","last_peered":"2024-09-14T06:30:03.827867+0000","last_clean":"2024-09-14T06:30:03.827867+0000","last_became_active":"2024-09-14T06:29:00.586491+0000","last_became_peered":"2024-09-14T06:29:00.586491+0000","last_unstale":"2024-09-14T06:30:03.827867+0000","last_undegraded":"2024-09-14T06:30:03.827867+0000","last_fullsized":"2024-09-14T06:30:03.827867+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":133,"log_dups_size":0,"ondisk_log_size":133,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T10:41:46.283525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021298290000000002,"stat_sum":{"num_bytes":15703454,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":0,"num_read":2680,"num_read_kb":14801,"num_write":194,"num_write_kb":9645,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"20","length":"1"},{"start":"22","length":"1"}]},{"pgid":"3.10","version":"82'162","reported_seq":835,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-14T06:30:03.822146+0000","last_change":"2024-09-14T06:30:01.771917+0000","last_active":"2024-09-14T06:30:03.822146+0000","last_peered":"2024-09-14T06:30:03.822146+0000","last_clean":"2024-09-14T06:30:03.822146+0000","last_became_active":"2024-09-14T06:29:00.579539+0000","last_became_peered":"2024-09-14T06:29:00.579539+0000","last_unstale":"2024-09-14T06:30:03.822146+0000","last_undegraded":"2024-09-14T06:30:03.822146+0000","last_fullsized":"2024-09-14T06:30:03.822146+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":162,"log_dups_size":0,"ondisk_log_size":162,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T12:28:41.506185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00084384200000000003,"stat_sum":{"num_bytes":15460541,"num_objects":20,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":0,"num_read":455,"num_read_kb":4173,"num_write":204,"num_write_kb":15611,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"20","length":"1"},{"start":"22","length":"1"}]},{"pgid":"3.11","version":"82'265","reported_seq":3744,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-14T06:30:03.827737+0000","last_change":"2024-09-14T06:30:01.770588+0000","last_active":"2024-09-14T06:30:03.827737+0000","last_peered":"2024-09-14T06:30:03.827737+0000","last_clean":"2024-09-14T06:30:03.827737+0000","last_became_active":"2024-09-14T06:29:00.586480+0000","last_became_peered":"2024-09-14T06:29:00.586480+0000","last_unstale":"2024-09-14T06:30:03.827737+0000","last_undegraded":"2024-09-14T06:30:03.827737+0000","last_fullsized":"2024-09-14T06:30:03.827737+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":265,"log_dups_size":0,"ondisk_log_size":265,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T13:26:16.448079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00086384899999999998,"stat_sum":{"num_bytes":48470041,"num_objects":34,"num_object_clones":0,"num_object_copies":68,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":34,"num_whiteouts":0,"num_read":3710,"num_read_kb":41711,"num_write":338,"num_write_kb":25233,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"20","length":"1"},{"start":"22","length":"1"}]},{"pgid":"3.12","version":"81'308","reported_seq":2679,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-14T06:30:02.808768+0000","last_change":"2024-09-14T06:30:01.583385+0000","last_active":"2024-09-14T06:30:02.808768+0000","last_peered":"2024-09-14T06:30:02.808768+0000","last_clean":"2024-09-14T06:30:02.808768+0000","last_became_active":"2024-09-14T06:29:00.579826+0000","last_became_peered":"2024-09-14T06:29:00.579826+0000","last_unstale":"2024-09-14T06:30:02.808768+0000","last_undegraded":"2024-09-14T06:30:02.808768+0000","last_fullsized":"2024-09-14T06:30:02.808768+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":308,"log_dups_size":0,"ondisk_log_size":308,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T10:32:15.909854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012732069999999999,"stat_sum":{"num_bytes":29535665,"num_objects":25,"num_object_clones":0,"num_object_copies":50,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":0,"num_read":3264,"num_read_kb":18201,"num_write":448,"num_write_kb":16842,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"20","length":"1"},{"start":"22","length":"1"}]},{"pgid":"3.13","version":"83'360","reported_seq":2593,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-14T06:30:03.827957+0000","last_change":"2024-09-14T06:30:01.772456+0000","last_active":"2024-09-14T06:30:03.827957+0000","last_peered":"2024-09-14T06:30:03.827957+0000","last_clean":"2024-09-14T06:30:03.827957+0000","last_became_active":"2024-09-14T06:29:00.586520+0000","last_became_peered":"2024-09-14T06:29:00.586520+0000","last_unstale":"2024-09-14T06:30:03.827957+0000","last_undegraded":"2024-09-14T06:30:03.827957+0000","last_fullsized":"2024-09-14T06:30:03.827957+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":360,"log_dups_size":0,"ondisk_log_size":360,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T17:37:24.551610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026580280000000002,"stat_sum":{"num_bytes":44897231,"num_objects":28,"num_object_clones":0,"num_object_copies":56,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":28,"num_whiteouts":0,"num_read":2897,"num_read_kb":23776,"num_write":532,"num_write_kb":22062,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"20","length":"1"},{"start":"22","length":"1"}]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":865696420,"num_objects":856,"num_object_clones":0,"num_object_copies":1712,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":856,"num_whiteouts":0,"num_read":80646,"num_read_kb":739906,"num_write":9099,"num_write_kb":524792,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":39,"num_bytes_recovered":1400964,"num_keys_recovered":11,"num_objects_omap":18,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":56561664,"data_stored":546554082,"data_compressed":33720813,"data_compressed_allocated":44613632,"data_compressed_original":537246924,"omap_allocated":0,"internal_metadata":0},"log_size":6465,"ondisk_log_size":6465,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":131072,"data_stored":1180736,"data_compressed":65166,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509459,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":47824,"kb_used_data":8312,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94324016,"statfs":{"total":96636764160,"available":96587792384,"internally_reserved":0,"allocated":8511488,"data_stored":70958167,"data_compressed":4271172,"data_compressed_allocated":5627904,"data_compressed_original":68968491,"omap_allocated":1590,"internal_metadata":40434122},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509458,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":34164,"kb_used_data":2976,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94337676,"statfs":{"total":96636764160,"available":96601780224,"internally_reserved":0,"allocated":3047424,"data_stored":20175802,"data_compressed":1133495,"data_compressed_allocated":1564672,"data_compressed_original":19298897,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509460,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":43648,"kb_used_data":8304,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94328192,"statfs":{"total":96636764160,"available":96592068608,"internally_reserved":0,"allocated":8503296,"data_stored":67118852,"data_compressed":4256238,"data_compressed_allocated":5693440,"data_compressed_original":65237623,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509459,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":69796,"kb_used_data":12240,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94302044,"statfs":{"total":96636764160,"available":96565293056,"internally_reserved":0,"allocated":12533760,"data_stored":115396563,"data_compressed":7242765,"data_compressed_allocated":9498624,"data_compressed_original":113404550,"omap_allocated":1595,"internal_metadata":58915269},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509460,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":46220,"kb_used_data":6712,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94325620,"statfs":{"total":96636764160,"available":96589434880,"internally_reserved":0,"allocated":6873088,"data_stored":60377977,"data_compressed":3718315,"data_compressed_allocated":4911104,"data_compressed_original":59188469,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":14,"seq":60129542163,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":35008,"kb_used_data":3812,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94336832,"statfs":{"total":96636764160,"available":96600915968,"internally_reserved":0,"allocated":3903488,"data_stored":30643001,"data_compressed":1912399,"data_compressed_allocated":2539520,"data_compressed_original":29909705,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542162,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":48592,"kb_used_data":9084,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94323248,"statfs":{"total":96636764160,"available":96587005952,"internally_reserved":0,"allocated":9302016,"data_stored":78299366,"data_compressed":4714808,"data_compressed_allocated":6209536,"data_compressed_original":76133817,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574867,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":49664,"kb_used_data":10148,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94322176,"statfs":{"total":96636764160,"available":96585908224,"internally_reserved":0,"allocated":10391552,"data_stored":93695679,"data_compressed":5689432,"data_compressed_allocated":7569408,"data_compressed_original":91745696,"omap_allocated":1593,"internal_metadata":40434119},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32583,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32583,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8429568,"data_stored":77438064,"data_compressed":4685347,"data_compressed_allocated":6172672,"data_compressed_original":75625913,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":5935104,"data_stored":60002176,"data_compressed":3721229,"data_compressed_allocated":4919296,"data_compressed_original":59249909,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":2957312,"data_stored":30204542,"data_compressed":1912399,"data_compressed_allocated":2539520,"data_compressed_original":29909705,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":10534912,"data_stored":107521186,"data_compressed":6524172,"data_compressed_allocated":8630272,"data_compressed_original":105995680,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":11620352,"data_stored":115003250,"data_compressed":7244126,"data_compressed_allocated":9502720,"data_compressed_original":113428388,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":7540736,"data_stored":66721711,"data_compressed":4258559,"data_compressed_allocated":5697536,"data_compressed_original":65286775,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":2031616,"data_stored":19677920,"data_compressed":1133495,"data_compressed_allocated":1564672,"data_compressed_original":19298897,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":7512064,"data_stored":69985233,"data_compressed":4241486,"data_compressed_allocated":5586944,"data_compressed_original":68451657,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-14T06:30:06.807 INFO:tasks.ceph.ceph_manager.ceph:recovered! 2024-09-14T06:30:06.807 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-14T06:30:06.807 INFO:tasks.thrashosds.thrasher:inject_pause on osd.1 2024-09-14T06:30:06.807 INFO:tasks.thrashosds.thrasher:Testing filestore_inject_stall pause injection for duration 3 2024-09-14T06:30:06.808 INFO:tasks.thrashosds.thrasher:Checking after 0, should_be_down=False 2024-09-14T06:30:06.808 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok config set filestore_inject_stall 3 2024-09-14T06:30:06.814 INFO:teuthology.orchestra.run.smithi094.stdout:osd.1: {} 2024-09-14T06:30:06.815 INFO:teuthology.orchestra.run.smithi094.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-14T06:30:06.841 INFO:teuthology.orchestra.run.smithi094.stderr:noscrub is unset 2024-09-14T06:30:06.845 INFO:teuthology.orchestra.run.smithi094.stdout:osd.2: {} 2024-09-14T06:30:06.845 INFO:teuthology.orchestra.run.smithi094.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-14T06:30:06.846 INFO:tasks.ceph.osd.7.smithi138.stderr:2024-09-14T06:30:06.845+0000 7f3c6a21d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33623) UID: 0 2024-09-14T06:30:06.874 INFO:teuthology.orchestra.run.smithi094.stdout:osd.3: {} 2024-09-14T06:30:06.874 INFO:teuthology.orchestra.run.smithi094.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-14T06:30:06.900 INFO:teuthology.orchestra.run.smithi094.stdout:osd.4: {} 2024-09-14T06:30:06.901 INFO:teuthology.orchestra.run.smithi094.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-14T06:30:06.931 INFO:teuthology.orchestra.run.smithi094.stdout:osd.5: {} 2024-09-14T06:30:06.931 INFO:teuthology.orchestra.run.smithi094.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-14T06:30:06.947 INFO:tasks.ceph.osd.6.smithi138.stderr:2024-09-14T06:30:06.946+0000 7f8cd8849640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33620) UID: 0 2024-09-14T06:30:06.960 INFO:teuthology.orchestra.run.smithi094.stdout:osd.6: {} 2024-09-14T06:30:06.960 INFO:teuthology.orchestra.run.smithi094.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-14T06:30:06.981 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-14T06:30:06.981 INFO:teuthology.orchestra.run.smithi094.stdout: "success": "filestore_inject_stall = '' (not observed, change may require restart) " 2024-09-14T06:30:06.981 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-14T06:30:06.982 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-14T06:30:06.982 INFO:teuthology.orchestra.run.smithi094.stdout: "ops": [], 2024-09-14T06:30:06.982 INFO:teuthology.orchestra.run.smithi094.stdout: "num_ops": 0 2024-09-14T06:30:06.982 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-14T06:30:06.991 INFO:teuthology.orchestra.run.smithi094.stdout:osd.7: {} 2024-09-14T06:30:06.991 INFO:teuthology.orchestra.run.smithi094.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-14T06:30:06.998 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-14T06:30:07.052 INFO:tasks.ceph.osd.3.smithi094.stderr:2024-09-14T06:30:07.051+0000 7feeeafe8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33655) UID: 0 2024-09-14T06:30:07.148 INFO:tasks.ceph.osd.2.smithi094.stderr:2024-09-14T06:30:07.147+0000 7faf3269b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33649) UID: 0 2024-09-14T06:30:07.174 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-14T06:30:07.174 INFO:teuthology.orchestra.run.smithi094.stdout: "ops": [], 2024-09-14T06:30:07.174 INFO:teuthology.orchestra.run.smithi094.stdout: "complaint_time": 30, 2024-09-14T06:30:07.175 INFO:teuthology.orchestra.run.smithi094.stdout: "num_blocked_ops": 0 2024-09-14T06:30:07.175 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-14T06:30:07.186 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-14T06:30:07.249 INFO:tasks.ceph.osd.6.smithi138.stderr:2024-09-14T06:30:07.248+0000 7f8cd8849640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33620) UID: 0 2024-09-14T06:30:07.350 INFO:tasks.ceph.osd.2.smithi094.stderr:2024-09-14T06:30:07.349+0000 7faf3269b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33649) UID: 0 2024-09-14T06:30:07.380 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-14T06:30:07.380 INFO:teuthology.orchestra.run.smithi094.stdout: "size": 20, 2024-09-14T06:30:07.380 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 600, 2024-09-14T06:30:07.380 INFO:teuthology.orchestra.run.smithi094.stdout: "ops": [ 2024-09-14T06:30:07.380 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:07.380 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4397.0:1848 3.4 e32/23 3:27620a21:::rbd_data.112dcdddac74.000000000000000b:head v 32'40, mlcod=30'38)", 2024-09-14T06:30:07.380 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:08.658649+0000", 2024-09-14T06:30:07.380 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 58.719136591000002, 2024-09-14T06:30:07.380 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.001862867, 2024-09-14T06:30:07.380 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:30:07.380 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:30:07.380 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:30:07.380 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:30:07.381 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:07.381 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:30:07.381 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.658649+0000", 2024-09-14T06:30:07.381 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:07.381 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:07.381 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:07.381 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:30:07.381 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.658649+0000", 2024-09-14T06:30:07.381 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:07.381 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:07.381 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:07.381 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:30:07.381 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.658651+0000", 2024-09-14T06:30:07.382 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.5650000000000001e-06 2024-09-14T06:30:07.382 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:07.382 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:07.382 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:30:07.382 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.658709+0000", 2024-09-14T06:30:07.382 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.8442000000000001e-05 2024-09-14T06:30:07.382 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:07.382 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:07.382 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:30:07.382 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.658730+0000", 2024-09-14T06:30:07.382 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.0509999999999998e-05 2024-09-14T06:30:07.382 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:07.382 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:07.382 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:30:07.383 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.658740+0000", 2024-09-14T06:30:07.383 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.8619999999999998e-06 2024-09-14T06:30:07.383 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:07.383 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:07.383 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:30:07.383 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.658813+0000", 2024-09-14T06:30:07.383 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.3578000000000006e-05 2024-09-14T06:30:07.383 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:07.383 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:07.383 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:30:07.383 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.658836+0000", 2024-09-14T06:30:07.383 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.2677e-05 2024-09-14T06:30:07.383 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:07.384 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:07.384 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-14T06:30:07.384 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.660462+0000", 2024-09-14T06:30:07.384 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00162637 2024-09-14T06:30:07.384 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:07.384 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:07.384 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:30:07.384 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.660512+0000", 2024-09-14T06:30:07.384 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.9863e-05 2024-09-14T06:30:07.384 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:07.384 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:07.384 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:07.385 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:07.385 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:07.385 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4397.0:2199 3.0 e32/23 3:05f3e980:::rbd_data.112dcdddac74.000000000000000f:head v 32'3, mlcod=31'1)", 2024-09-14T06:30:07.385 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:08.680236+0000", 2024-09-14T06:30:07.385 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 58.697550225999997, 2024-09-14T06:30:07.385 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0019068990000000001, 2024-09-14T06:30:07.385 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:30:07.385 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:30:07.385 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:30:07.385 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:30:07.385 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:07.385 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:30:07.385 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.680236+0000", 2024-09-14T06:30:07.386 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:07.386 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:07.386 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:07.386 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:30:07.386 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.680236+0000", 2024-09-14T06:30:07.386 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:07.386 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:07.386 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:07.386 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:30:07.386 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.680237+0000", 2024-09-14T06:30:07.386 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.4980000000000001e-06 2024-09-14T06:30:07.386 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:07.386 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:07.386 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:30:07.387 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.680324+0000", 2024-09-14T06:30:07.387 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.7310000000000003e-05 2024-09-14T06:30:07.387 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:07.387 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:07.387 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:30:07.387 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.680354+0000", 2024-09-14T06:30:07.387 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.97e-05 2024-09-14T06:30:07.387 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:07.387 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:07.387 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:30:07.387 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.680366+0000", 2024-09-14T06:30:07.387 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.2048999999999999e-05 2024-09-14T06:30:07.387 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:07.387 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:07.388 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:30:07.388 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.680439+0000", 2024-09-14T06:30:07.388 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.2910000000000005e-05 2024-09-14T06:30:07.388 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:07.388 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:07.388 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:30:07.388 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.680465+0000", 2024-09-14T06:30:07.388 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.5812999999999998e-05 2024-09-14T06:30:07.388 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:07.388 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:07.388 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-14T06:30:07.388 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.682094+0000", 2024-09-14T06:30:07.388 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.001628876 2024-09-14T06:30:07.388 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:07.389 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:07.389 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:30:07.389 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.682142+0000", 2024-09-14T06:30:07.389 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.8742999999999997e-05 2024-09-14T06:30:07.389 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:07.389 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:07.389 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:07.389 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:07.389 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:07.389 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4397.0:3292 3.9 e32/23 3:91dbce57:::rbd_data.112dcdddac74.0000000000000036:head v 32'10, mlcod=32'8)", 2024-09-14T06:30:07.389 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:08.740506+0000", 2024-09-14T06:30:07.389 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 58.637279513999999, 2024-09-14T06:30:07.389 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.001864166, 2024-09-14T06:30:07.389 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:30:07.390 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:30:07.390 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:30:07.390 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:30:07.390 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:07.390 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:30:07.390 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.740506+0000", 2024-09-14T06:30:07.390 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:07.390 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:07.390 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:07.390 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:30:07.390 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.740506+0000", 2024-09-14T06:30:07.390 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:07.390 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:07.391 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:07.391 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:30:07.391 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.740514+0000", 2024-09-14T06:30:07.391 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.2109999999999998e-06 2024-09-14T06:30:07.391 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:07.391 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:07.391 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:30:07.391 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.740745+0000", 2024-09-14T06:30:07.391 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000230194 2024-09-14T06:30:07.391 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:07.391 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:07.391 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:30:07.391 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.740764+0000", 2024-09-14T06:30:07.392 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.9330999999999998e-05 2024-09-14T06:30:07.392 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:07.392 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:07.392 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:30:07.392 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.740774+0000", 2024-09-14T06:30:07.392 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.6260000000000007e-06 2024-09-14T06:30:07.392 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:07.392 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:07.392 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:30:07.392 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.740839+0000", 2024-09-14T06:30:07.392 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.5314000000000005e-05 2024-09-14T06:30:07.392 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:07.392 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:07.393 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:30:07.393 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.740860+0000", 2024-09-14T06:30:07.393 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.0868000000000001e-05 2024-09-14T06:30:07.393 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:07.393 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:07.393 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-14T06:30:07.393 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.742327+0000", 2024-09-14T06:30:07.393 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0014672909999999999 2024-09-14T06:30:07.393 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:07.393 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:07.393 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:30:07.393 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.742370+0000", 2024-09-14T06:30:07.393 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.3331000000000002e-05 2024-09-14T06:30:07.393 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:07.394 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:07.394 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:07.394 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:07.394 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:07.394 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4397.0:26415 3.0 e53/23 3:05fd9be0:::rbd_data.112d38768680.0000000000000006:head v 53'21, mlcod=50'19)", 2024-09-14T06:30:07.394 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:30.721610+0000", 2024-09-14T06:30:07.394 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 36.656175445000002, 2024-09-14T06:30:07.394 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0021763160000000002, 2024-09-14T06:30:07.394 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:30:07.394 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:30:07.394 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:30:07.394 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:30:07.394 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:07.395 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:30:07.395 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:30.721610+0000", 2024-09-14T06:30:07.395 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:07.395 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:07.395 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:07.395 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:30:07.395 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:30.721610+0000", 2024-09-14T06:30:07.395 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:07.395 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:07.395 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:07.395 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:30:07.395 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:30.721611+0000", 2024-09-14T06:30:07.395 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.41e-07 2024-09-14T06:30:07.395 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:07.395 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:07.396 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:30:07.396 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:30.721905+0000", 2024-09-14T06:30:07.396 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00029394300000000001 2024-09-14T06:30:07.396 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:07.396 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:07.396 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:30:07.396 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:30.721921+0000", 2024-09-14T06:30:07.396 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.6609999999999999e-05 2024-09-14T06:30:07.396 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:07.396 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:07.396 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:30:07.396 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:30.721931+0000", 2024-09-14T06:30:07.396 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.6739999999999992e-06 2024-09-14T06:30:07.396 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:07.396 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:07.397 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:30:07.397 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:30.722001+0000", 2024-09-14T06:30:07.397 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.0549000000000005e-05 2024-09-14T06:30:07.397 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:07.397 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:07.397 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:30:07.397 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:30.722029+0000", 2024-09-14T06:30:07.397 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.7327000000000001e-05 2024-09-14T06:30:07.397 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:07.397 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:07.397 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-14T06:30:07.397 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:30.723727+0000", 2024-09-14T06:30:07.397 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0016977310000000001 2024-09-14T06:30:07.397 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:07.398 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:07.398 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:30:07.398 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:30.723787+0000", 2024-09-14T06:30:07.398 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.0140999999999999e-05 2024-09-14T06:30:07.398 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:07.398 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:07.398 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:07.398 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:07.398 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:07.398 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4397.0:36084 3.0 e56/23 3:05a48ef4:::rbd_data.112dc58639fa.0000000000000058:head v 56'29, mlcod=56'27)", 2024-09-14T06:30:07.398 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:34.508855+0000", 2024-09-14T06:30:07.398 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 32.868930691000003, 2024-09-14T06:30:07.398 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0020801140000000001, 2024-09-14T06:30:07.398 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:30:07.399 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:30:07.399 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:30:07.399 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:30:07.399 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:07.399 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:30:07.399 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:34.508855+0000", 2024-09-14T06:30:07.399 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:07.399 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:07.399 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:07.399 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:30:07.399 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:34.508855+0000", 2024-09-14T06:30:07.399 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:07.399 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:07.399 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:07.400 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:30:07.400 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:34.508856+0000", 2024-09-14T06:30:07.400 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.1909999999999999e-06 2024-09-14T06:30:07.400 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:07.400 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:07.400 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:30:07.400 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:34.509203+0000", 2024-09-14T06:30:07.400 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00034625900000000002 2024-09-14T06:30:07.400 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:07.400 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:07.400 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:30:07.400 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:34.509218+0000", 2024-09-14T06:30:07.400 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.5091e-05 2024-09-14T06:30:07.401 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:07.401 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:07.401 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:30:07.401 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:34.509227+0000", 2024-09-14T06:30:07.401 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.0850000000000003e-06 2024-09-14T06:30:07.401 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:07.401 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:07.401 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:30:07.401 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:34.509311+0000", 2024-09-14T06:30:07.401 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.4436000000000003e-05 2024-09-14T06:30:07.401 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:07.401 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:07.401 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:30:07.401 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:34.509335+0000", 2024-09-14T06:30:07.402 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.3577000000000001e-05 2024-09-14T06:30:07.402 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:07.402 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:07.402 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-14T06:30:07.402 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:34.510890+0000", 2024-09-14T06:30:07.402 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0015554259999999999 2024-09-14T06:30:07.402 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:07.402 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:07.402 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:30:07.402 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:34.510935+0000", 2024-09-14T06:30:07.402 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.5049000000000002e-05 2024-09-14T06:30:07.402 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:07.402 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:07.402 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:07.403 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:07.403 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:07.403 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4397.0:36966 3.4 e56/23 3:22008318:::rbd_data.112dc58639fa.0000000000000066:head v 56'56, mlcod=50'54)", 2024-09-14T06:30:07.403 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:34.565614+0000", 2024-09-14T06:30:07.403 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 32.812172255, 2024-09-14T06:30:07.403 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0018691929999999999, 2024-09-14T06:30:07.403 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:30:07.403 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:30:07.403 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:30:07.403 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:30:07.403 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:07.403 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:30:07.403 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:34.565614+0000", 2024-09-14T06:30:07.403 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:07.404 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:07.404 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:07.404 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:30:07.404 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:34.565614+0000", 2024-09-14T06:30:07.404 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:07.404 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:07.404 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:07.404 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:30:07.404 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:34.565615+0000", 2024-09-14T06:30:07.404 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.784e-06 2024-09-14T06:30:07.404 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:07.404 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:07.404 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:30:07.405 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:34.565919+0000", 2024-09-14T06:30:07.405 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00030351599999999999 2024-09-14T06:30:07.405 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:07.405 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:07.405 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:30:07.405 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:34.565933+0000", 2024-09-14T06:30:07.405 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.4465e-05 2024-09-14T06:30:07.405 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:07.405 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:07.405 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:30:07.405 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:34.565943+0000", 2024-09-14T06:30:07.405 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.7920000000000004e-06 2024-09-14T06:30:07.405 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:07.405 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:07.406 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:30:07.406 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:34.565999+0000", 2024-09-14T06:30:07.406 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.5643000000000002e-05 2024-09-14T06:30:07.406 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:07.406 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:07.406 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:30:07.406 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:34.566025+0000", 2024-09-14T06:30:07.406 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.6040000000000001e-05 2024-09-14T06:30:07.406 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:07.406 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:07.406 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-14T06:30:07.406 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:34.567430+0000", 2024-09-14T06:30:07.406 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.001405653 2024-09-14T06:30:07.406 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:07.407 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:07.407 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:30:07.407 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:34.567483+0000", 2024-09-14T06:30:07.407 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.2299999999999997e-05 2024-09-14T06:30:07.407 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:07.407 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:07.407 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:07.407 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:07.407 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:07.407 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4397.0:42285 3.9 e62/23 3:90747014:::rbd_data.112dca2f6ae1.000000000000002f:head v 62'54, mlcod=62'52)", 2024-09-14T06:30:07.407 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:40.755702+0000", 2024-09-14T06:30:07.407 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 26.622083871000001, 2024-09-14T06:30:07.407 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.001687596, 2024-09-14T06:30:07.407 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:30:07.408 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:30:07.408 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:30:07.408 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:30:07.408 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:07.408 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:30:07.408 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:40.755702+0000", 2024-09-14T06:30:07.408 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:07.408 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:07.408 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:07.408 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:30:07.408 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:40.755702+0000", 2024-09-14T06:30:07.408 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:07.408 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:07.409 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:07.409 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:30:07.409 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:40.755707+0000", 2024-09-14T06:30:07.409 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.4700000000000001e-06 2024-09-14T06:30:07.409 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:07.409 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:07.409 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:30:07.409 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:40.756366+0000", 2024-09-14T06:30:07.409 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00065852900000000004 2024-09-14T06:30:07.409 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:07.409 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:07.409 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:30:07.409 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:40.756383+0000", 2024-09-14T06:30:07.409 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.7275e-05 2024-09-14T06:30:07.410 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:07.410 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:07.410 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:30:07.410 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:40.756393+0000", 2024-09-14T06:30:07.410 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.4749999999999999e-06 2024-09-14T06:30:07.410 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:07.410 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:07.410 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:30:07.410 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:40.756462+0000", 2024-09-14T06:30:07.410 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.9214000000000005e-05 2024-09-14T06:30:07.410 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:07.410 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:07.410 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:30:07.410 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:40.756490+0000", 2024-09-14T06:30:07.411 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.7936000000000002e-05 2024-09-14T06:30:07.411 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:07.411 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:07.411 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-14T06:30:07.411 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:40.757343+0000", 2024-09-14T06:30:07.411 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000853611 2024-09-14T06:30:07.411 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:07.411 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:07.411 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:30:07.411 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:40.757390+0000", 2024-09-14T06:30:07.411 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.6085999999999999e-05 2024-09-14T06:30:07.411 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:07.411 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:07.411 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:07.412 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:07.412 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:07.412 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4397.0:45722 3.9 e64/23 3:91fed098:::rbd_data.112d5a737e72.0000000000000001:head v 64'61, mlcod=62'59)", 2024-09-14T06:30:07.412 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:43.864269+0000", 2024-09-14T06:30:07.412 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 23.513516616, 2024-09-14T06:30:07.412 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.003055224, 2024-09-14T06:30:07.412 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:30:07.412 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:30:07.412 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:30:07.412 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:30:07.412 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:07.412 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:30:07.412 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.864269+0000", 2024-09-14T06:30:07.413 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:07.413 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:07.413 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:07.413 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:30:07.413 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.864269+0000", 2024-09-14T06:30:07.413 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:07.413 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:07.413 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:07.413 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:30:07.413 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.864269+0000", 2024-09-14T06:30:07.413 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.41e-07 2024-09-14T06:30:07.413 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:07.413 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:07.413 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:30:07.414 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.865490+0000", 2024-09-14T06:30:07.414 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.001220552 2024-09-14T06:30:07.414 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:07.414 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:07.414 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:30:07.414 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.865506+0000", 2024-09-14T06:30:07.414 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.6263999999999999e-05 2024-09-14T06:30:07.414 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:07.414 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:07.414 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:30:07.414 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.865516+0000", 2024-09-14T06:30:07.414 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.0091999999999999e-05 2024-09-14T06:30:07.414 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:07.414 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:07.415 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:30:07.415 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.865583+0000", 2024-09-14T06:30:07.415 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.6676999999999999e-05 2024-09-14T06:30:07.415 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:07.415 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:07.415 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:30:07.415 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.865610+0000", 2024-09-14T06:30:07.415 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.6529999999999998e-05 2024-09-14T06:30:07.415 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:07.415 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:07.415 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-14T06:30:07.415 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.867281+0000", 2024-09-14T06:30:07.415 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.001671203 2024-09-14T06:30:07.415 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:07.416 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:07.416 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:30:07.416 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.867324+0000", 2024-09-14T06:30:07.416 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.3665000000000003e-05 2024-09-14T06:30:07.416 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:07.416 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:07.416 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:07.416 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:07.416 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:07.416 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4397.0:49627 3.4 e64/23 3:21e30d4b:::rbd_data.112d5a737e72.0000000000000017:head v 64'69, mlcod=61'67)", 2024-09-14T06:30:07.416 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:44.206446+0000", 2024-09-14T06:30:07.416 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 23.171339977999999, 2024-09-14T06:30:07.416 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0018142220000000001, 2024-09-14T06:30:07.416 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:30:07.417 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:30:07.417 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:30:07.417 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:30:07.417 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:07.417 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:30:07.417 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.206446+0000", 2024-09-14T06:30:07.417 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:07.417 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:07.417 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:07.417 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:30:07.417 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.206446+0000", 2024-09-14T06:30:07.417 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:07.417 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:07.417 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:07.418 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:30:07.418 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.206446+0000", 2024-09-14T06:30:07.418 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.4900000000000001e-07 2024-09-14T06:30:07.418 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:07.418 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:07.418 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:30:07.418 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.206797+0000", 2024-09-14T06:30:07.418 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00035112799999999998 2024-09-14T06:30:07.418 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:07.418 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:07.418 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:30:07.418 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.206816+0000", 2024-09-14T06:30:07.418 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.8977000000000001e-05 2024-09-14T06:30:07.418 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:07.419 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:07.419 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:30:07.419 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.206825+0000", 2024-09-14T06:30:07.419 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.8279999999999992e-06 2024-09-14T06:30:07.419 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:07.419 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:07.419 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:30:07.419 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.206892+0000", 2024-09-14T06:30:07.419 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.6809000000000004e-05 2024-09-14T06:30:07.419 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:07.419 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:07.419 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:30:07.419 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.206915+0000", 2024-09-14T06:30:07.419 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.315e-05 2024-09-14T06:30:07.419 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:07.420 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:07.420 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-14T06:30:07.420 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.208201+0000", 2024-09-14T06:30:07.420 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0012858819999999999 2024-09-14T06:30:07.420 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:07.420 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:07.420 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:30:07.420 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.208260+0000", 2024-09-14T06:30:07.420 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.9098999999999998e-05 2024-09-14T06:30:07.420 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:07.420 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:07.420 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:07.420 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:07.420 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:07.421 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4397.0:50003 3.9 e64/23 3:94f26c4b:::rbd_data.112d5a737e72.000000000000001f:head v 64'62, mlcod=62'60)", 2024-09-14T06:30:07.421 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:44.220413+0000", 2024-09-14T06:30:07.421 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 23.157372299999999, 2024-09-14T06:30:07.421 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0019210430000000001, 2024-09-14T06:30:07.421 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:30:07.421 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:30:07.421 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:30:07.421 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:30:07.421 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:07.421 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:30:07.421 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.220413+0000", 2024-09-14T06:30:07.421 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:07.421 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:07.421 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:07.422 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:30:07.422 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.220413+0000", 2024-09-14T06:30:07.422 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:07.422 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:07.422 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:07.422 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:30:07.422 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.220415+0000", 2024-09-14T06:30:07.422 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.575e-06 2024-09-14T06:30:07.422 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:07.422 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:07.422 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:30:07.422 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.220984+0000", 2024-09-14T06:30:07.422 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000568624 2024-09-14T06:30:07.423 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:07.423 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:07.423 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:30:07.423 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.221007+0000", 2024-09-14T06:30:07.423 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.3112000000000002e-05 2024-09-14T06:30:07.423 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:07.423 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:07.423 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:30:07.423 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.221017+0000", 2024-09-14T06:30:07.423 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.0098999999999999e-05 2024-09-14T06:30:07.423 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:07.423 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:07.423 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:30:07.423 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.221110+0000", 2024-09-14T06:30:07.423 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.3268000000000002e-05 2024-09-14T06:30:07.424 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:07.424 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:07.424 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:30:07.424 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.221131+0000", 2024-09-14T06:30:07.424 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.0905999999999999e-05 2024-09-14T06:30:07.424 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:07.424 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:07.424 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-14T06:30:07.424 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.222290+0000", 2024-09-14T06:30:07.424 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.001159408 2024-09-14T06:30:07.424 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:07.424 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:07.424 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:30:07.425 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.222335+0000", 2024-09-14T06:30:07.425 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.4051000000000002e-05 2024-09-14T06:30:07.425 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:07.425 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:07.425 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:07.425 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:07.425 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:07.425 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4397.0:50006 3.4 e64/23 3:20cfe300:::rbd_data.112d5a737e72.0000000000000019:head v 64'70, mlcod=61'68)", 2024-09-14T06:30:07.425 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:44.224218+0000", 2024-09-14T06:30:07.425 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 23.153567419000002, 2024-09-14T06:30:07.425 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0021104320000000002, 2024-09-14T06:30:07.425 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:30:07.425 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:30:07.425 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:30:07.426 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:30:07.426 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:07.426 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:30:07.426 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.224218+0000", 2024-09-14T06:30:07.426 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:07.426 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:07.426 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:07.426 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:30:07.426 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.224218+0000", 2024-09-14T06:30:07.426 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:07.426 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:07.426 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:07.426 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:30:07.426 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.224219+0000", 2024-09-14T06:30:07.427 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.0049999999999999e-06 2024-09-14T06:30:07.427 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:07.427 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:07.427 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:30:07.427 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.224326+0000", 2024-09-14T06:30:07.427 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000106848 2024-09-14T06:30:07.427 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:07.427 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:07.427 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:30:07.427 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.224345+0000", 2024-09-14T06:30:07.427 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.8587999999999999e-05 2024-09-14T06:30:07.427 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:07.427 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:07.427 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:30:07.428 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.224354+0000", 2024-09-14T06:30:07.428 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.7220000000000005e-06 2024-09-14T06:30:07.428 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:07.428 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:07.428 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:30:07.428 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.224416+0000", 2024-09-14T06:30:07.428 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.2738999999999997e-05 2024-09-14T06:30:07.428 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:07.428 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:07.428 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:30:07.428 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.224441+0000", 2024-09-14T06:30:07.428 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.4408000000000001e-05 2024-09-14T06:30:07.428 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:07.428 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:07.429 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-14T06:30:07.429 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.226295+0000", 2024-09-14T06:30:07.429 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00185428 2024-09-14T06:30:07.429 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:07.429 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:07.429 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:30:07.429 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.226329+0000", 2024-09-14T06:30:07.429 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.3841999999999999e-05 2024-09-14T06:30:07.429 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:07.429 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:07.429 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:07.429 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:07.429 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:07.430 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4397.0:50414 3.0 e64/23 3:03ae4e67:::rbd_data.112d5a737e72.000000000000001a:head v 64'67, mlcod=62'65)", 2024-09-14T06:30:07.430 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:44.240908+0000", 2024-09-14T06:30:07.430 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 23.136877939000001, 2024-09-14T06:30:07.430 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.001493241, 2024-09-14T06:30:07.430 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:30:07.430 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:30:07.430 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:30:07.430 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:30:07.430 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:07.430 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:30:07.430 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.240908+0000", 2024-09-14T06:30:07.430 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:07.430 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:07.430 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:07.431 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:30:07.431 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.240908+0000", 2024-09-14T06:30:07.431 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:07.431 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:07.431 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:07.431 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:30:07.431 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.240910+0000", 2024-09-14T06:30:07.431 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.7179999999999999e-06 2024-09-14T06:30:07.431 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:07.431 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:07.431 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:30:07.431 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.241049+0000", 2024-09-14T06:30:07.431 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00013932199999999999 2024-09-14T06:30:07.431 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:07.431 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:07.432 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:30:07.432 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.241069+0000", 2024-09-14T06:30:07.432 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.9836000000000001e-05 2024-09-14T06:30:07.432 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:07.432 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:07.432 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:30:07.432 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.241080+0000", 2024-09-14T06:30:07.432 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.0920999999999999e-05 2024-09-14T06:30:07.432 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:07.432 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:07.432 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:30:07.432 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.241150+0000", 2024-09-14T06:30:07.432 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.0568999999999995e-05 2024-09-14T06:30:07.432 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:07.433 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:07.433 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:30:07.433 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.241178+0000", 2024-09-14T06:30:07.433 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.7429000000000001e-05 2024-09-14T06:30:07.433 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:07.433 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:07.433 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-14T06:30:07.433 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.242353+0000", 2024-09-14T06:30:07.433 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0011750770000000001 2024-09-14T06:30:07.433 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:07.433 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:07.433 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:30:07.433 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.242401+0000", 2024-09-14T06:30:07.433 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.8368999999999997e-05 2024-09-14T06:30:07.434 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:07.434 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:07.434 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:07.434 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:07.434 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:07.434 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4397.0:51816 3.4 e64/23 3:21a826f9:::rbd_data.112d5a737e72.000000000000002d:head v 64'72, mlcod=64'70)", 2024-09-14T06:30:07.434 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:44.312072+0000", 2024-09-14T06:30:07.434 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 23.065713945999999, 2024-09-14T06:30:07.434 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0019555240000000002, 2024-09-14T06:30:07.434 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:30:07.434 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:30:07.434 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:30:07.434 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:30:07.434 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:07.435 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:30:07.435 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.312072+0000", 2024-09-14T06:30:07.435 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:07.435 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:07.435 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:07.435 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:30:07.435 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.312072+0000", 2024-09-14T06:30:07.435 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:07.435 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:07.435 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:07.435 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:30:07.435 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.312074+0000", 2024-09-14T06:30:07.435 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.9530000000000002e-06 2024-09-14T06:30:07.436 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:07.436 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:07.436 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:30:07.436 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.312378+0000", 2024-09-14T06:30:07.436 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00030392 2024-09-14T06:30:07.436 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:07.436 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:07.436 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:30:07.436 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.312399+0000", 2024-09-14T06:30:07.436 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.1134000000000001e-05 2024-09-14T06:30:07.436 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:07.436 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:07.436 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:30:07.437 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.312408+0000", 2024-09-14T06:30:07.437 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.2879999999999998e-06 2024-09-14T06:30:07.437 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:07.437 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:07.437 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:30:07.437 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.312494+0000", 2024-09-14T06:30:07.437 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.6280999999999996e-05 2024-09-14T06:30:07.437 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:07.437 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:07.437 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:30:07.437 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.312518+0000", 2024-09-14T06:30:07.437 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.3149000000000001e-05 2024-09-14T06:30:07.437 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:07.438 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:07.438 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-14T06:30:07.438 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.313964+0000", 2024-09-14T06:30:07.438 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.001446754 2024-09-14T06:30:07.438 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:07.438 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:07.438 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:30:07.438 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.314027+0000", 2024-09-14T06:30:07.438 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.3045000000000004e-05 2024-09-14T06:30:07.438 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:07.438 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:07.438 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:07.438 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:07.439 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:07.439 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4397.0:74241 3.0 e73/23 3:06e93fb9:::rbd_data.112d80ca42af.0000000000000000:head v 73'102, mlcod=71'100)", 2024-09-14T06:30:07.439 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:53.876730+0000", 2024-09-14T06:30:07.439 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 13.501055515999999, 2024-09-14T06:30:07.439 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0079470329999999992, 2024-09-14T06:30:07.439 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:30:07.439 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:30:07.439 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:30:07.439 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:30:07.439 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:07.439 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:30:07.439 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:53.876730+0000", 2024-09-14T06:30:07.439 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:07.439 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:07.440 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:07.440 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:30:07.440 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:53.876730+0000", 2024-09-14T06:30:07.440 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:07.440 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:07.440 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:07.440 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:30:07.440 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:53.876731+0000", 2024-09-14T06:30:07.440 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.0500000000000004e-07 2024-09-14T06:30:07.440 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:07.440 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:07.440 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:30:07.440 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:53.879616+0000", 2024-09-14T06:30:07.441 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0028856680000000001 2024-09-14T06:30:07.441 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:07.441 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:07.441 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:30:07.441 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:53.879643+0000", 2024-09-14T06:30:07.441 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.6120999999999999e-05 2024-09-14T06:30:07.441 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:07.441 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:07.441 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:30:07.441 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:53.879654+0000", 2024-09-14T06:30:07.441 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.1586999999999999e-05 2024-09-14T06:30:07.441 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:07.441 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:07.441 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:30:07.442 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:53.879756+0000", 2024-09-14T06:30:07.442 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00010223899999999999 2024-09-14T06:30:07.442 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:07.442 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:07.442 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:30:07.442 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:53.879783+0000", 2024-09-14T06:30:07.442 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.6913000000000001e-05 2024-09-14T06:30:07.442 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:07.442 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:07.442 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-14T06:30:07.442 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:53.884594+0000", 2024-09-14T06:30:07.442 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0048110729999999999 2024-09-14T06:30:07.442 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:07.442 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:07.443 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:30:07.443 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:53.884677+0000", 2024-09-14T06:30:07.443 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.2927000000000001e-05 2024-09-14T06:30:07.443 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:07.443 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:07.443 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:07.443 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:07.443 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:07.443 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4397.0:74243 3.9 e73/23 3:94977f81:::rbd_data.112d80ca42af.0000000000000002:head v 73'94, mlcod=71'92)", 2024-09-14T06:30:07.443 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:53.885156+0000", 2024-09-14T06:30:07.443 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 13.492629598000001, 2024-09-14T06:30:07.443 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0096009860000000006, 2024-09-14T06:30:07.443 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:30:07.443 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:30:07.444 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:30:07.444 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:30:07.444 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:07.444 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:30:07.444 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:53.885156+0000", 2024-09-14T06:30:07.444 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:07.444 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:07.444 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:07.444 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:30:07.444 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:53.885156+0000", 2024-09-14T06:30:07.444 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:07.444 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:07.444 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:07.444 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:30:07.445 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:53.885156+0000", 2024-09-14T06:30:07.445 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.23e-07 2024-09-14T06:30:07.445 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:07.445 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:07.445 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:30:07.445 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:53.889694+0000", 2024-09-14T06:30:07.445 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0045378709999999997 2024-09-14T06:30:07.445 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:07.445 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:07.445 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:30:07.445 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:53.889720+0000", 2024-09-14T06:30:07.445 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.5259e-05 2024-09-14T06:30:07.445 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:07.446 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:07.446 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:30:07.446 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:53.889731+0000", 2024-09-14T06:30:07.446 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.1739000000000001e-05 2024-09-14T06:30:07.446 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:07.446 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:07.446 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:30:07.446 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:53.889822+0000", 2024-09-14T06:30:07.446 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.1200999999999996e-05 2024-09-14T06:30:07.446 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:07.446 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:07.446 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:30:07.446 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:53.889850+0000", 2024-09-14T06:30:07.446 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.7084999999999999e-05 2024-09-14T06:30:07.447 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:07.447 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:07.447 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-14T06:30:07.447 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:53.894703+0000", 2024-09-14T06:30:07.447 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0048532690000000003 2024-09-14T06:30:07.447 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:07.447 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:07.447 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:30:07.447 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:53.894757+0000", 2024-09-14T06:30:07.447 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.4339000000000003e-05 2024-09-14T06:30:07.447 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:07.447 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:07.447 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:07.447 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:07.448 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:07.448 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4397.0:74249 3.0 e73/23 3:009b0225:::rbd_data.112d80ca42af.0000000000000009:head v 73'103, mlcod=71'101)", 2024-09-14T06:30:07.448 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:53.893219+0000", 2024-09-14T06:30:07.448 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 13.484567119999999, 2024-09-14T06:30:07.448 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0065381980000000003, 2024-09-14T06:30:07.448 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:30:07.448 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:30:07.448 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:30:07.448 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:30:07.448 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:07.448 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:30:07.448 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:53.893219+0000", 2024-09-14T06:30:07.448 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:07.448 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:07.449 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:07.449 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:30:07.449 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:53.893219+0000", 2024-09-14T06:30:07.449 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:07.449 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:07.449 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:07.449 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:30:07.449 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:53.893221+0000", 2024-09-14T06:30:07.449 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.4329999999999998e-06 2024-09-14T06:30:07.449 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:07.449 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:07.449 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:30:07.449 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:53.895968+0000", 2024-09-14T06:30:07.449 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.002746746 2024-09-14T06:30:07.450 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:07.450 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:07.450 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:30:07.450 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:53.895988+0000", 2024-09-14T06:30:07.450 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.0227000000000001e-05 2024-09-14T06:30:07.450 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:07.450 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:07.450 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:30:07.450 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:53.896000+0000", 2024-09-14T06:30:07.450 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.2359e-05 2024-09-14T06:30:07.450 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:07.450 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:07.450 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:30:07.450 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:53.896084+0000", 2024-09-14T06:30:07.451 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.3659999999999995e-05 2024-09-14T06:30:07.451 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:07.451 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:07.451 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:30:07.451 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:53.896109+0000", 2024-09-14T06:30:07.451 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.5259999999999999e-05 2024-09-14T06:30:07.451 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:07.451 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:07.451 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-14T06:30:07.451 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:53.899704+0000", 2024-09-14T06:30:07.451 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0035942209999999999 2024-09-14T06:30:07.451 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:07.451 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:07.451 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:30:07.452 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:53.899757+0000", 2024-09-14T06:30:07.452 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.3291999999999997e-05 2024-09-14T06:30:07.452 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:07.452 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:07.452 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:07.452 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:07.452 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:07.452 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4397.0:78263 3.0 e77/23 3:00899b22:::rbd_data.112ddfbec471.0000000000000016:head v 77'104, mlcod=73'102)", 2024-09-14T06:30:07.452 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:57.634578+0000", 2024-09-14T06:30:07.452 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 9.7432080679999995, 2024-09-14T06:30:07.452 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.002372454, 2024-09-14T06:30:07.452 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:30:07.452 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:30:07.452 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:30:07.453 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:30:07.453 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:07.453 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:30:07.453 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:57.634578+0000", 2024-09-14T06:30:07.453 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:07.453 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:07.453 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:07.453 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:30:07.453 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:57.634578+0000", 2024-09-14T06:30:07.453 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:07.453 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:07.453 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:07.453 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:30:07.453 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:57.634578+0000", 2024-09-14T06:30:07.454 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.5699999999999998e-07 2024-09-14T06:30:07.454 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:07.454 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:07.454 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:30:07.454 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:57.635347+0000", 2024-09-14T06:30:07.454 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00076848300000000003 2024-09-14T06:30:07.454 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:07.454 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:07.454 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:30:07.454 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:57.635361+0000", 2024-09-14T06:30:07.454 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.4238e-05 2024-09-14T06:30:07.454 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:07.454 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:07.455 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:30:07.455 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:57.635371+0000", 2024-09-14T06:30:07.455 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.0087e-05 2024-09-14T06:30:07.455 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:07.455 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:07.455 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:30:07.455 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:57.635439+0000", 2024-09-14T06:30:07.455 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.7953000000000005e-05 2024-09-14T06:30:07.455 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:07.455 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:07.455 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:30:07.455 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:57.635464+0000", 2024-09-14T06:30:07.455 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.5062000000000002e-05 2024-09-14T06:30:07.455 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:07.456 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:07.456 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-14T06:30:07.456 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:57.636914+0000", 2024-09-14T06:30:07.456 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.001449625 2024-09-14T06:30:07.456 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:07.456 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:07.456 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:30:07.456 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:57.636950+0000", 2024-09-14T06:30:07.456 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.6548999999999999e-05 2024-09-14T06:30:07.456 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:07.456 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:07.456 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:07.456 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:07.456 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:07.457 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4397.0:78604 3.9 e77/23 3:94c62a0e:::rbd_data.112ddfbec471.000000000000002c:head v 77'96, mlcod=73'94)", 2024-09-14T06:30:07.457 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:57.677568+0000", 2024-09-14T06:30:07.457 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 9.7002176799999997, 2024-09-14T06:30:07.457 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.002564846, 2024-09-14T06:30:07.457 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:30:07.457 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:30:07.457 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:30:07.457 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:30:07.457 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:07.457 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:30:07.457 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:57.677568+0000", 2024-09-14T06:30:07.457 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:07.457 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:07.458 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:07.458 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:30:07.458 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:57.677568+0000", 2024-09-14T06:30:07.458 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:07.458 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:07.458 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:07.458 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:30:07.458 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:57.677568+0000", 2024-09-14T06:30:07.458 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.8900000000000001e-07 2024-09-14T06:30:07.458 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:07.458 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:07.458 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:30:07.458 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:57.678488+0000", 2024-09-14T06:30:07.458 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00091963899999999998 2024-09-14T06:30:07.459 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:07.459 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:07.459 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:30:07.459 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:57.678500+0000", 2024-09-14T06:30:07.459 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.1902000000000001e-05 2024-09-14T06:30:07.459 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:07.459 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:07.459 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:30:07.459 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:57.678507+0000", 2024-09-14T06:30:07.459 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.7739999999999999e-06 2024-09-14T06:30:07.459 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:07.459 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:07.459 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:30:07.459 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:57.678586+0000", 2024-09-14T06:30:07.460 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.9103000000000005e-05 2024-09-14T06:30:07.460 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:07.460 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:07.460 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:30:07.460 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:57.678614+0000", 2024-09-14T06:30:07.460 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.7912e-05 2024-09-14T06:30:07.460 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:07.460 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:07.460 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-14T06:30:07.460 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:57.680075+0000", 2024-09-14T06:30:07.460 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0014613370000000001 2024-09-14T06:30:07.460 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:07.460 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:07.460 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:30:07.461 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:57.680133+0000", 2024-09-14T06:30:07.461 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.7890000000000003e-05 2024-09-14T06:30:07.461 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:07.461 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:07.461 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:07.461 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:07.461 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:07.461 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4397.0:78640 3.9 e77/23 3:97fd73eb:::rbd_data.112ddfbec471.000000000000002f:head v 77'97, mlcod=73'95)", 2024-09-14T06:30:07.461 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:57.681153+0000", 2024-09-14T06:30:07.461 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 9.6966323810000006, 2024-09-14T06:30:07.461 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0016647109999999999, 2024-09-14T06:30:07.461 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:30:07.461 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:30:07.461 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:30:07.462 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:30:07.462 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:07.462 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:30:07.462 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:57.681153+0000", 2024-09-14T06:30:07.462 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:07.462 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:07.462 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:07.462 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:30:07.462 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:57.681153+0000", 2024-09-14T06:30:07.462 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:07.462 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:07.462 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:07.462 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:30:07.462 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:57.681155+0000", 2024-09-14T06:30:07.462 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.127e-06 2024-09-14T06:30:07.462 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:07.463 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:07.463 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:30:07.463 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:57.681704+0000", 2024-09-14T06:30:07.463 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000549103 2024-09-14T06:30:07.463 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:07.463 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:07.463 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:30:07.463 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:57.681733+0000", 2024-09-14T06:30:07.463 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.9641e-05 2024-09-14T06:30:07.463 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:07.463 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:07.463 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:30:07.463 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:57.681742+0000", 2024-09-14T06:30:07.463 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.9660000000000002e-06 2024-09-14T06:30:07.464 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:07.464 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:07.464 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:30:07.464 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:57.681802+0000", 2024-09-14T06:30:07.464 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.9935000000000001e-05 2024-09-14T06:30:07.464 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:07.464 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:07.464 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:30:07.464 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:57.681825+0000", 2024-09-14T06:30:07.464 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.3017999999999999e-05 2024-09-14T06:30:07.464 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:07.464 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:07.464 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-14T06:30:07.465 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:57.682777+0000", 2024-09-14T06:30:07.465 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00095157399999999995 2024-09-14T06:30:07.465 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:07.465 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:07.465 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:30:07.465 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:57.682818+0000", 2024-09-14T06:30:07.465 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.1347000000000002e-05 2024-09-14T06:30:07.465 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:07.465 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:07.465 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:07.465 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:07.465 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:07.465 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4397.0:78641 3.9 e77/23 3:914b72e4:::rbd_data.112ddfbec471.0000000000000030:head v 77'98, mlcod=73'95)", 2024-09-14T06:30:07.465 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:57.682289+0000", 2024-09-14T06:30:07.466 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 9.6954969989999995, 2024-09-14T06:30:07.466 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0020171400000000002, 2024-09-14T06:30:07.466 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:30:07.466 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:30:07.466 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:30:07.466 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:30:07.466 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:07.466 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:30:07.466 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:57.682289+0000", 2024-09-14T06:30:07.466 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:07.466 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:07.466 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:07.466 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:30:07.466 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:57.682289+0000", 2024-09-14T06:30:07.466 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:07.467 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:07.467 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:07.467 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:30:07.467 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:57.682290+0000", 2024-09-14T06:30:07.467 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.0100000000000003e-07 2024-09-14T06:30:07.467 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:07.467 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:07.467 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:30:07.467 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:57.683230+0000", 2024-09-14T06:30:07.467 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00094047599999999999 2024-09-14T06:30:07.467 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:07.467 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:07.467 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:30:07.467 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:57.683243+0000", 2024-09-14T06:30:07.467 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.2926e-05 2024-09-14T06:30:07.468 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:07.468 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:07.468 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:30:07.468 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:57.683250+0000", 2024-09-14T06:30:07.468 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.6109999999999999e-06 2024-09-14T06:30:07.468 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:07.468 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:07.468 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:30:07.468 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:57.683313+0000", 2024-09-14T06:30:07.468 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.2943999999999999e-05 2024-09-14T06:30:07.468 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:07.468 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:07.468 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:30:07.468 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:57.683337+0000", 2024-09-14T06:30:07.468 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.4377999999999999e-05 2024-09-14T06:30:07.469 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:07.469 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:07.469 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-14T06:30:07.469 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:57.684264+0000", 2024-09-14T06:30:07.469 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00092733099999999999 2024-09-14T06:30:07.469 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:07.469 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:07.469 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:30:07.469 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:57.684306+0000", 2024-09-14T06:30:07.469 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.1573000000000003e-05 2024-09-14T06:30:07.469 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:07.469 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:07.469 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:07.470 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:07.470 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:07.470 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-14T06:30:07.471 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-14T06:30:07.471 INFO:tasks.ceph.osd.5.smithi138.stderr:2024-09-14T06:30:07.470+0000 7fe13746f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33618) UID: 0 2024-09-14T06:30:07.572 INFO:tasks.ceph.osd.4.smithi138.stderr:2024-09-14T06:30:07.571+0000 7f6c653c8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33615) UID: 0 2024-09-14T06:30:07.614 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-14T06:30:07.614 INFO:teuthology.orchestra.run.smithi094.stdout: "ops": [], 2024-09-14T06:30:07.614 INFO:teuthology.orchestra.run.smithi094.stdout: "num_ops": 0 2024-09-14T06:30:07.614 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-14T06:30:07.622 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-14T06:30:07.673 INFO:tasks.ceph.osd.3.smithi094.stderr:2024-09-14T06:30:07.672+0000 7feeeafe8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33655) UID: 0 2024-09-14T06:30:07.725 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-14T06:30:07.725 INFO:teuthology.orchestra.run.smithi094.stdout: "ops": [], 2024-09-14T06:30:07.725 INFO:teuthology.orchestra.run.smithi094.stdout: "complaint_time": 30, 2024-09-14T06:30:07.725 INFO:teuthology.orchestra.run.smithi094.stdout: "num_blocked_ops": 0 2024-09-14T06:30:07.725 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-14T06:30:07.733 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-14T06:30:07.774 INFO:tasks.ceph.osd.1.smithi094.stderr:2024-09-14T06:30:07.773+0000 7f328e47a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33641) UID: 0 2024-09-14T06:30:07.875 INFO:tasks.ceph.osd.7.smithi138.stderr:2024-09-14T06:30:07.874+0000 7f3c6a21d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33623) UID: 0 2024-09-14T06:30:07.883 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-14T06:30:07.883 INFO:teuthology.orchestra.run.smithi094.stdout: "size": 20, 2024-09-14T06:30:07.883 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 600, 2024-09-14T06:30:07.883 INFO:teuthology.orchestra.run.smithi094.stdout: "ops": [ 2024-09-14T06:30:07.883 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:07.883 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:44425 3.2 3:4603ad7f:::rbd_data.112dca2f6ae1.0000000000000005:17 [read 868352~4096 [fadvise_nocache]] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:30:07.883 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:43.854967+0000", 2024-09-14T06:30:07.883 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 24.026003346, 2024-09-14T06:30:07.883 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.063146025999999994, 2024-09-14T06:30:07.883 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:30:07.883 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:30:07.884 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:30:07.884 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:30:07.884 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:30:07.884 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:07.884 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 44425 2024-09-14T06:30:07.884 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:07.884 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:30:07.884 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:07.884 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:30:07.884 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.854967+0000", 2024-09-14T06:30:07.884 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:07.884 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:07.884 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:07.885 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:30:07.885 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.854967+0000", 2024-09-14T06:30:07.885 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:07.885 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:07.885 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:07.885 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:30:07.885 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.854967+0000", 2024-09-14T06:30:07.885 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.16e-07 2024-09-14T06:30:07.885 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:07.885 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:07.885 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:30:07.885 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.854968+0000", 2024-09-14T06:30:07.885 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.006e-06 2024-09-14T06:30:07.886 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:07.886 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:07.886 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:30:07.886 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.854971+0000", 2024-09-14T06:30:07.886 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.9359999999999999e-06 2024-09-14T06:30:07.886 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:07.886 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:07.886 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:30:07.886 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.854974+0000", 2024-09-14T06:30:07.886 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.1840000000000002e-06 2024-09-14T06:30:07.886 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:07.886 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:07.887 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:30:07.887 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.917945+0000", 2024-09-14T06:30:07.887 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.062970925999999997 2024-09-14T06:30:07.887 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:07.887 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:07.887 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:30:07.887 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.918030+0000", 2024-09-14T06:30:07.887 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.5863000000000001e-05 2024-09-14T06:30:07.887 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:07.887 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:07.887 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:30:07.887 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.918113+0000", 2024-09-14T06:30:07.887 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.2894999999999998e-05 2024-09-14T06:30:07.887 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:07.888 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:07.888 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:07.888 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:07.888 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:07.888 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:44426 3.2 3:4603ad7f:::rbd_data.112dca2f6ae1.0000000000000005:17 [read 892928~4096 [fadvise_nocache]] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:30:07.888 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:43.854977+0000", 2024-09-14T06:30:07.888 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 24.025993935999999, 2024-09-14T06:30:07.888 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.063369237999999994, 2024-09-14T06:30:07.888 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:30:07.888 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:30:07.888 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:30:07.888 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:30:07.888 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:30:07.889 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:07.889 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 44426 2024-09-14T06:30:07.889 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:07.889 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:30:07.889 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:07.889 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:30:07.889 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.854977+0000", 2024-09-14T06:30:07.889 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:07.889 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:07.889 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:07.889 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:30:07.889 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.854977+0000", 2024-09-14T06:30:07.889 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:07.890 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:07.890 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:07.890 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:30:07.890 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.854977+0000", 2024-09-14T06:30:07.890 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.1199999999999999e-07 2024-09-14T06:30:07.890 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:07.890 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:07.890 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:30:07.890 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.854980+0000", 2024-09-14T06:30:07.890 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.27e-06 2024-09-14T06:30:07.890 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:07.890 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:07.890 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:30:07.890 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.854983+0000", 2024-09-14T06:30:07.891 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.1650000000000002e-06 2024-09-14T06:30:07.891 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:07.891 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:07.891 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:30:07.891 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.854989+0000", 2024-09-14T06:30:07.891 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.6520000000000003e-06 2024-09-14T06:30:07.891 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:07.891 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:07.891 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:30:07.891 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.918143+0000", 2024-09-14T06:30:07.891 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.063154541999999994 2024-09-14T06:30:07.891 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:07.892 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:07.892 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:30:07.892 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.918209+0000", 2024-09-14T06:30:07.892 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.5529000000000002e-05 2024-09-14T06:30:07.892 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:07.892 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:07.892 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:30:07.892 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.918346+0000", 2024-09-14T06:30:07.892 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000136868 2024-09-14T06:30:07.892 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:07.892 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:07.892 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:07.892 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:07.892 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:07.892 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:44427 3.2 3:4603ad7f:::rbd_data.112dca2f6ae1.0000000000000005:17 [read 917504~4096 [fadvise_nocache]] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:30:07.893 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:43.854992+0000", 2024-09-14T06:30:07.893 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 24.025978290000001, 2024-09-14T06:30:07.893 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.063607792999999996, 2024-09-14T06:30:07.893 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:30:07.893 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:30:07.893 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:30:07.893 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:30:07.893 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:30:07.893 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:07.893 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 44427 2024-09-14T06:30:07.893 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:07.893 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:30:07.893 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:07.893 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:30:07.894 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.854992+0000", 2024-09-14T06:30:07.894 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:07.894 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:07.894 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:07.894 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:30:07.894 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.854992+0000", 2024-09-14T06:30:07.894 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:07.894 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:07.894 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:07.894 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:30:07.894 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.854993+0000", 2024-09-14T06:30:07.894 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.1400000000000001e-07 2024-09-14T06:30:07.894 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:07.894 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:07.895 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:30:07.895 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.854994+0000", 2024-09-14T06:30:07.895 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.091e-06 2024-09-14T06:30:07.895 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:07.895 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:07.895 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:30:07.895 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855001+0000", 2024-09-14T06:30:07.895 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.204e-06 2024-09-14T06:30:07.895 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:07.895 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:07.895 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:30:07.895 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855003+0000", 2024-09-14T06:30:07.895 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.514e-06 2024-09-14T06:30:07.895 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:07.895 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:07.896 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:30:07.896 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.918380+0000", 2024-09-14T06:30:07.896 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.063376738000000002 2024-09-14T06:30:07.896 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:07.896 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:07.896 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:30:07.896 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.918453+0000", 2024-09-14T06:30:07.896 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.3291e-05 2024-09-14T06:30:07.896 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:07.896 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:07.896 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:30:07.896 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.918600+0000", 2024-09-14T06:30:07.896 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00014674100000000001 2024-09-14T06:30:07.897 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:07.897 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:07.897 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:07.897 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:07.897 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:07.897 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:44428 3.2 3:4603ad7f:::rbd_data.112dca2f6ae1.0000000000000005:17 [read 942080~4096 [fadvise_nocache]] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:30:07.897 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:43.855006+0000", 2024-09-14T06:30:07.897 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 24.02596424, 2024-09-14T06:30:07.897 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.063846321999999997, 2024-09-14T06:30:07.897 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:30:07.897 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:30:07.897 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:30:07.897 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:30:07.898 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:30:07.898 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:07.898 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 44428 2024-09-14T06:30:07.898 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:07.898 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:30:07.898 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:07.898 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:30:07.898 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855006+0000", 2024-09-14T06:30:07.898 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:07.898 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:07.898 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:07.898 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:30:07.898 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855006+0000", 2024-09-14T06:30:07.898 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:07.898 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:07.899 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:07.899 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:30:07.899 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855007+0000", 2024-09-14T06:30:07.899 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.1199999999999999e-07 2024-09-14T06:30:07.899 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:07.899 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:07.899 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:30:07.899 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855011+0000", 2024-09-14T06:30:07.899 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.9539999999999998e-06 2024-09-14T06:30:07.899 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:07.899 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:07.899 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:30:07.899 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855014+0000", 2024-09-14T06:30:07.899 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.2930000000000001e-06 2024-09-14T06:30:07.900 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:07.900 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:07.900 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:30:07.900 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855016+0000", 2024-09-14T06:30:07.900 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.3039999999999999e-06 2024-09-14T06:30:07.900 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:07.900 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:07.900 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:30:07.900 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.918634+0000", 2024-09-14T06:30:07.900 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.063617928000000004 2024-09-14T06:30:07.900 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:07.900 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:07.900 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:30:07.900 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.918711+0000", 2024-09-14T06:30:07.900 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.6545000000000007e-05 2024-09-14T06:30:07.901 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:07.901 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:07.901 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:30:07.901 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.918853+0000", 2024-09-14T06:30:07.901 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00014208600000000001 2024-09-14T06:30:07.901 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:07.901 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:07.901 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:07.901 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:07.901 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:07.901 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:44429 3.2 3:4603ad7f:::rbd_data.112dca2f6ae1.0000000000000005:17 [read 966656~4096 [fadvise_nocache]] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:30:07.901 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:43.855019+0000", 2024-09-14T06:30:07.901 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 24.025951567, 2024-09-14T06:30:07.901 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.064059226999999996, 2024-09-14T06:30:07.902 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:30:07.902 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:30:07.902 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:30:07.902 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:30:07.902 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:30:07.902 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:07.902 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 44429 2024-09-14T06:30:07.902 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:07.902 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:30:07.902 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:07.902 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:30:07.902 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855019+0000", 2024-09-14T06:30:07.902 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:07.903 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:07.903 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:07.903 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:30:07.903 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855019+0000", 2024-09-14T06:30:07.903 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:07.903 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:07.903 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:07.903 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:30:07.903 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855019+0000", 2024-09-14T06:30:07.903 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.28e-07 2024-09-14T06:30:07.903 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:07.903 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:07.903 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:30:07.904 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855020+0000", 2024-09-14T06:30:07.904 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.0330000000000001e-06 2024-09-14T06:30:07.904 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:07.904 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:07.904 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:30:07.904 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855023+0000", 2024-09-14T06:30:07.904 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.79e-06 2024-09-14T06:30:07.904 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:07.904 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:07.904 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:30:07.904 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855025+0000", 2024-09-14T06:30:07.904 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.216e-06 2024-09-14T06:30:07.904 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:07.904 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:07.905 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:30:07.905 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.918886+0000", 2024-09-14T06:30:07.905 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.063861037999999995 2024-09-14T06:30:07.905 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:07.905 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:07.905 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:30:07.905 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.918952+0000", 2024-09-14T06:30:07.905 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.5993000000000003e-05 2024-09-14T06:30:07.905 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:07.905 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:07.905 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:30:07.905 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.919078+0000", 2024-09-14T06:30:07.905 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00012592899999999999 2024-09-14T06:30:07.906 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:07.906 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:07.906 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:07.906 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:07.906 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:07.906 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:44430 3.2 3:4603ad7f:::rbd_data.112dca2f6ae1.0000000000000005:17 [read 991232~4096 [fadvise_nocache]] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:30:07.906 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:43.855028+0000", 2024-09-14T06:30:07.906 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 24.025942557, 2024-09-14T06:30:07.906 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.064284346000000006, 2024-09-14T06:30:07.906 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:30:07.906 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:30:07.906 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:30:07.906 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:30:07.907 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:30:07.907 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:07.907 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 44430 2024-09-14T06:30:07.907 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:07.907 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:30:07.907 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:07.907 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:30:07.907 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855028+0000", 2024-09-14T06:30:07.907 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:07.907 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:07.907 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:07.907 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:30:07.907 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855028+0000", 2024-09-14T06:30:07.907 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:07.908 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:07.908 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:07.908 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:30:07.908 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855028+0000", 2024-09-14T06:30:07.908 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.1199999999999999e-07 2024-09-14T06:30:07.908 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:07.908 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:07.908 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:30:07.908 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855029+0000", 2024-09-14T06:30:07.908 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.1100000000000004e-07 2024-09-14T06:30:07.908 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:07.908 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:07.909 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:30:07.909 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855032+0000", 2024-09-14T06:30:07.909 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.7889999999999999e-06 2024-09-14T06:30:07.909 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:07.909 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:07.909 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:30:07.909 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855034+0000", 2024-09-14T06:30:07.909 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.125e-06 2024-09-14T06:30:07.909 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:07.909 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:07.909 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:30:07.909 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.919110+0000", 2024-09-14T06:30:07.909 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.064076413999999998 2024-09-14T06:30:07.909 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:07.910 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:07.910 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:30:07.910 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.919186+0000", 2024-09-14T06:30:07.910 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.5784999999999994e-05 2024-09-14T06:30:07.910 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:07.910 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:07.910 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:30:07.910 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.919312+0000", 2024-09-14T06:30:07.910 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00012611 2024-09-14T06:30:07.910 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:07.910 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:07.910 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:07.910 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:07.910 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:07.911 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:44431 3.2 3:4603ad7f:::rbd_data.112dca2f6ae1.0000000000000005:17 [read 1015808~4096 [fadvise_nocache]] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:30:07.911 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:43.855037+0000", 2024-09-14T06:30:07.911 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 24.025933688999999, 2024-09-14T06:30:07.911 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.064469755000000004, 2024-09-14T06:30:07.911 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:30:07.911 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:30:07.911 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:30:07.911 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:30:07.911 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:30:07.911 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:07.911 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 44431 2024-09-14T06:30:07.911 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:07.911 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:30:07.911 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:07.912 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:30:07.912 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855037+0000", 2024-09-14T06:30:07.912 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:07.912 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:07.912 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:07.912 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:30:07.912 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855037+0000", 2024-09-14T06:30:07.912 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:07.912 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:07.912 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:07.912 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:30:07.912 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855037+0000", 2024-09-14T06:30:07.912 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.1e-07 2024-09-14T06:30:07.912 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:07.913 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:07.913 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:30:07.913 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855038+0000", 2024-09-14T06:30:07.913 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.9400000000000004e-07 2024-09-14T06:30:07.913 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:07.913 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:07.913 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:30:07.913 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855041+0000", 2024-09-14T06:30:07.913 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.9639999999999999e-06 2024-09-14T06:30:07.913 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:07.913 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:07.913 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:30:07.913 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855043+0000", 2024-09-14T06:30:07.913 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.2259999999999999e-06 2024-09-14T06:30:07.914 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:07.914 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:07.914 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:30:07.914 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.919347+0000", 2024-09-14T06:30:07.914 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.064304091999999993 2024-09-14T06:30:07.914 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:07.914 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:07.914 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:30:07.914 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.919393+0000", 2024-09-14T06:30:07.914 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.5340999999999998e-05 2024-09-14T06:30:07.914 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:07.914 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:07.914 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:30:07.914 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.919507+0000", 2024-09-14T06:30:07.915 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000114028 2024-09-14T06:30:07.915 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:07.915 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:07.915 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:07.915 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:07.915 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:07.915 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:44432 3.2 3:4603ad7f:::rbd_data.112dca2f6ae1.0000000000000005:17 [read 1040384~4096 [fadvise_nocache]] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:30:07.915 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:43.855046+0000", 2024-09-14T06:30:07.915 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 24.025924621000001, 2024-09-14T06:30:07.915 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.064639253999999993, 2024-09-14T06:30:07.915 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:30:07.915 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:30:07.915 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:30:07.915 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:30:07.916 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:30:07.916 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:07.916 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 44432 2024-09-14T06:30:07.916 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:07.916 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:30:07.916 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:07.916 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:30:07.916 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855046+0000", 2024-09-14T06:30:07.916 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:07.916 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:07.916 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:07.916 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:30:07.916 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855046+0000", 2024-09-14T06:30:07.916 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:07.917 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:07.917 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:07.917 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:30:07.917 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855046+0000", 2024-09-14T06:30:07.917 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.05e-07 2024-09-14T06:30:07.917 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:07.917 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:07.917 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:30:07.917 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855047+0000", 2024-09-14T06:30:07.917 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.879999999999999e-07 2024-09-14T06:30:07.917 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:07.917 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:07.917 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:30:07.917 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855050+0000", 2024-09-14T06:30:07.918 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.8459999999999999e-06 2024-09-14T06:30:07.918 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:07.918 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:07.918 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:30:07.918 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855052+0000", 2024-09-14T06:30:07.918 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.136e-06 2024-09-14T06:30:07.918 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:07.918 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:07.918 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:30:07.918 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.919536+0000", 2024-09-14T06:30:07.918 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.064484132 2024-09-14T06:30:07.918 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:07.918 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:07.918 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:30:07.919 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.919611+0000", 2024-09-14T06:30:07.919 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.4407999999999996e-05 2024-09-14T06:30:07.919 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:07.919 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:07.919 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:30:07.919 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.919685+0000", 2024-09-14T06:30:07.919 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.4538999999999999e-05 2024-09-14T06:30:07.919 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:07.919 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:07.919 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:07.919 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:07.919 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:07.919 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:44518 3.2 3:4190d882:::rbd_data.112dca2f6ae1.0000000000000002:17 [read 12288~4096 [fadvise_nocache]] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:30:07.920 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:43.855055+0000", 2024-09-14T06:30:07.920 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 24.025915736999998, 2024-09-14T06:30:07.920 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.064894317000000007, 2024-09-14T06:30:07.920 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:30:07.920 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:30:07.920 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:30:07.920 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:30:07.920 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:30:07.920 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:07.920 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 44518 2024-09-14T06:30:07.920 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:07.920 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:30:07.921 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:07.921 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:30:07.921 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855055+0000", 2024-09-14T06:30:07.921 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:07.921 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:07.921 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:07.921 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:30:07.921 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855055+0000", 2024-09-14T06:30:07.921 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:07.921 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:07.921 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:07.921 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:30:07.921 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855055+0000", 2024-09-14T06:30:07.921 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.0699999999999999e-07 2024-09-14T06:30:07.921 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:07.922 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:07.922 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:30:07.922 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855056+0000", 2024-09-14T06:30:07.922 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.9400000000000004e-07 2024-09-14T06:30:07.922 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:07.922 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:07.922 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:30:07.922 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855059+0000", 2024-09-14T06:30:07.922 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.8870000000000001e-06 2024-09-14T06:30:07.922 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:07.922 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:07.922 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:30:07.922 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855061+0000", 2024-09-14T06:30:07.922 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.1919999999999999e-06 2024-09-14T06:30:07.923 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:07.923 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:07.923 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:30:07.923 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.919719+0000", 2024-09-14T06:30:07.923 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.064658013 2024-09-14T06:30:07.923 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:07.923 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:07.923 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:30:07.923 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.919788+0000", 2024-09-14T06:30:07.923 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.9282000000000005e-05 2024-09-14T06:30:07.923 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:07.923 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:07.923 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:30:07.924 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.919949+0000", 2024-09-14T06:30:07.924 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00016084199999999999 2024-09-14T06:30:07.924 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:07.924 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:07.924 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:07.924 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:07.924 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:07.924 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:44519 3.2 3:4190d882:::rbd_data.112dca2f6ae1.0000000000000002:17 [read 36864~4096 [fadvise_nocache]] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:30:07.924 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:43.855064+0000", 2024-09-14T06:30:07.924 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 24.025906760000002, 2024-09-14T06:30:07.924 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.065081865000000003, 2024-09-14T06:30:07.924 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:30:07.924 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:30:07.925 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:30:07.925 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:30:07.925 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:30:07.925 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:07.925 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 44519 2024-09-14T06:30:07.925 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:07.925 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:30:07.925 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:07.925 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:30:07.925 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855064+0000", 2024-09-14T06:30:07.925 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:07.925 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:07.925 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:07.926 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:30:07.926 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855064+0000", 2024-09-14T06:30:07.926 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:07.926 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:07.926 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:07.926 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:30:07.926 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855064+0000", 2024-09-14T06:30:07.926 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.1e-07 2024-09-14T06:30:07.926 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:07.926 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:07.926 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:30:07.926 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855065+0000", 2024-09-14T06:30:07.926 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.035e-06 2024-09-14T06:30:07.926 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:07.927 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:07.927 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:30:07.927 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855068+0000", 2024-09-14T06:30:07.927 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.8339999999999999e-06 2024-09-14T06:30:07.927 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:07.927 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:07.927 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:30:07.927 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855071+0000", 2024-09-14T06:30:07.927 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.3450000000000002e-06 2024-09-14T06:30:07.927 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:07.927 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:07.927 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:30:07.927 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.919979+0000", 2024-09-14T06:30:07.927 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.064907647999999998 2024-09-14T06:30:07.928 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:07.928 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:07.928 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:30:07.928 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.920026+0000", 2024-09-14T06:30:07.928 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.7595000000000001e-05 2024-09-14T06:30:07.928 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:07.928 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:07.928 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:30:07.928 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.920146+0000", 2024-09-14T06:30:07.928 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00011919800000000001 2024-09-14T06:30:07.928 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:07.928 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:07.928 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:07.929 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:07.929 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:07.929 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:44520 3.2 3:4190d882:::rbd_data.112dca2f6ae1.0000000000000002:17 [read 61440~4096 [fadvise_nocache]] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:30:07.929 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:43.855075+0000", 2024-09-14T06:30:07.929 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 24.025895709, 2024-09-14T06:30:07.929 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.065335078000000005, 2024-09-14T06:30:07.929 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:30:07.929 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:30:07.929 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:30:07.929 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:30:07.929 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:30:07.929 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:07.929 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 44520 2024-09-14T06:30:07.929 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:07.930 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:30:07.930 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:07.930 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:30:07.930 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855075+0000", 2024-09-14T06:30:07.930 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:07.930 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:07.930 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:07.930 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:30:07.930 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855075+0000", 2024-09-14T06:30:07.930 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:07.930 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:07.930 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:07.930 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:30:07.930 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855075+0000", 2024-09-14T06:30:07.931 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.1e-07 2024-09-14T06:30:07.931 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:07.931 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:07.931 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:30:07.931 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855076+0000", 2024-09-14T06:30:07.931 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.0839999999999999e-06 2024-09-14T06:30:07.931 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:07.931 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:07.931 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:30:07.931 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855079+0000", 2024-09-14T06:30:07.931 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.9359999999999999e-06 2024-09-14T06:30:07.931 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:07.931 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:07.931 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:30:07.931 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855084+0000", 2024-09-14T06:30:07.932 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.1730000000000003e-06 2024-09-14T06:30:07.932 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:07.932 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:07.932 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:30:07.932 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.920175+0000", 2024-09-14T06:30:07.932 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.065090234999999996 2024-09-14T06:30:07.932 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:07.932 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:07.932 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:30:07.932 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.920250+0000", 2024-09-14T06:30:07.932 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.5807000000000001e-05 2024-09-14T06:30:07.932 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:07.932 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:07.933 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:30:07.933 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.920410+0000", 2024-09-14T06:30:07.933 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00015963300000000001 2024-09-14T06:30:07.933 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:07.933 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:07.933 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:07.933 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:07.933 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:07.933 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:44521 3.2 3:4190d882:::rbd_data.112dca2f6ae1.0000000000000002:17 [read 86016~4096 [fadvise_nocache]] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:30:07.933 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:43.855087+0000", 2024-09-14T06:30:07.933 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 24.025883228000001, 2024-09-14T06:30:07.933 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.065563264999999996, 2024-09-14T06:30:07.933 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:30:07.933 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:30:07.934 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:30:07.934 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:30:07.934 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:30:07.934 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:07.934 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 44521 2024-09-14T06:30:07.934 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:07.934 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:30:07.934 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:07.934 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:30:07.934 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855087+0000", 2024-09-14T06:30:07.934 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:07.934 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:07.934 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:07.934 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:30:07.935 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855087+0000", 2024-09-14T06:30:07.935 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:07.935 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:07.935 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:07.935 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:30:07.935 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855088+0000", 2024-09-14T06:30:07.935 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.3699999999999999e-07 2024-09-14T06:30:07.935 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:07.935 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:07.935 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:30:07.935 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855089+0000", 2024-09-14T06:30:07.935 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.9900000000000009e-07 2024-09-14T06:30:07.935 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:07.935 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:07.936 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:30:07.936 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855091+0000", 2024-09-14T06:30:07.936 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.8940000000000002e-06 2024-09-14T06:30:07.936 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:07.936 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:07.936 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:30:07.936 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855094+0000", 2024-09-14T06:30:07.936 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.351e-06 2024-09-14T06:30:07.936 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:07.936 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:07.936 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:30:07.936 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.920437+0000", 2024-09-14T06:30:07.936 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.065343221000000007 2024-09-14T06:30:07.936 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:07.937 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:07.937 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:30:07.937 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.920481+0000", 2024-09-14T06:30:07.937 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.4431000000000002e-05 2024-09-14T06:30:07.937 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:07.937 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:07.937 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:30:07.937 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.920651+0000", 2024-09-14T06:30:07.937 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00016913200000000001 2024-09-14T06:30:07.937 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:07.937 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:07.937 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:07.937 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:07.937 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:07.938 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:44522 3.2 3:4190d882:::rbd_data.112dca2f6ae1.0000000000000002:17 [read 110592~4096 [fadvise_nocache]] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:30:07.938 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:43.855098+0000", 2024-09-14T06:30:07.938 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 24.025873001000001, 2024-09-14T06:30:07.938 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.065827628999999999, 2024-09-14T06:30:07.938 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:30:07.938 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:30:07.938 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:30:07.938 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:30:07.938 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:30:07.938 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:07.938 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 44522 2024-09-14T06:30:07.938 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:07.938 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:30:07.938 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:07.939 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:30:07.939 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855098+0000", 2024-09-14T06:30:07.939 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:07.939 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:07.939 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:07.939 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:30:07.939 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855098+0000", 2024-09-14T06:30:07.939 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:07.939 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:07.939 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:07.939 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:30:07.939 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855098+0000", 2024-09-14T06:30:07.939 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.1e-07 2024-09-14T06:30:07.940 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:07.940 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:07.940 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:30:07.940 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855099+0000", 2024-09-14T06:30:07.940 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.012e-06 2024-09-14T06:30:07.940 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:07.940 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:07.940 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:30:07.940 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855102+0000", 2024-09-14T06:30:07.940 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.9380000000000001e-06 2024-09-14T06:30:07.940 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:07.940 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:07.940 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:30:07.941 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855108+0000", 2024-09-14T06:30:07.941 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.4339999999999997e-06 2024-09-14T06:30:07.941 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:07.941 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:07.941 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:30:07.941 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.920681+0000", 2024-09-14T06:30:07.941 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.065572943999999994 2024-09-14T06:30:07.941 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:07.941 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:07.941 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:30:07.941 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.920724+0000", 2024-09-14T06:30:07.941 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.2877000000000003e-05 2024-09-14T06:30:07.941 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:07.942 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:07.942 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:30:07.942 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.920925+0000", 2024-09-14T06:30:07.942 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00020121399999999999 2024-09-14T06:30:07.942 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:07.942 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:07.942 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:07.942 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:07.942 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:07.942 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:44523 3.2 3:4190d882:::rbd_data.112dca2f6ae1.0000000000000002:17 [read 135168~4096 [fadvise_nocache]] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:30:07.942 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:43.855111+0000", 2024-09-14T06:30:07.942 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 24.025859278999999, 2024-09-14T06:30:07.942 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.066028198999999996, 2024-09-14T06:30:07.942 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:30:07.943 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:30:07.943 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:30:07.943 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:30:07.943 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:30:07.943 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:07.943 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 44523 2024-09-14T06:30:07.943 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:07.943 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:30:07.943 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:07.943 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:30:07.943 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855111+0000", 2024-09-14T06:30:07.943 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:07.943 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:07.943 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:07.944 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:30:07.944 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855111+0000", 2024-09-14T06:30:07.944 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:07.944 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:07.944 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:07.944 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:30:07.944 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855112+0000", 2024-09-14T06:30:07.944 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.1199999999999999e-07 2024-09-14T06:30:07.944 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:07.944 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:07.944 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:30:07.944 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855115+0000", 2024-09-14T06:30:07.944 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.3380000000000001e-06 2024-09-14T06:30:07.945 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:07.945 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:07.945 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:30:07.945 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855118+0000", 2024-09-14T06:30:07.945 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.366e-06 2024-09-14T06:30:07.945 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:07.945 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:07.945 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:30:07.945 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855121+0000", 2024-09-14T06:30:07.945 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.8329999999999998e-06 2024-09-14T06:30:07.945 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:07.945 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:07.945 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:30:07.946 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.920956+0000", 2024-09-14T06:30:07.946 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.065835251999999997 2024-09-14T06:30:07.946 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:07.946 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:07.946 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:30:07.946 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.921001+0000", 2024-09-14T06:30:07.946 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.4778999999999999e-05 2024-09-14T06:30:07.946 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:07.946 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:07.946 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:30:07.946 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.921139+0000", 2024-09-14T06:30:07.946 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000138419 2024-09-14T06:30:07.946 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:07.946 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:07.946 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:07.947 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:07.947 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:07.947 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:44524 3.2 3:4190d882:::rbd_data.112dca2f6ae1.0000000000000002:17 [read 159744~4096 [fadvise_nocache]] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:30:07.947 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:43.855124+0000", 2024-09-14T06:30:07.947 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 24.025846300000001, 2024-09-14T06:30:07.947 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.066249261000000004, 2024-09-14T06:30:07.947 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:30:07.947 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:30:07.947 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:30:07.947 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:30:07.947 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:30:07.947 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:07.947 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 44524 2024-09-14T06:30:07.947 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:07.948 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:30:07.948 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:07.948 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:30:07.948 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855124+0000", 2024-09-14T06:30:07.948 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:07.948 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:07.948 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:07.948 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:30:07.948 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855124+0000", 2024-09-14T06:30:07.948 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:07.948 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:07.948 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:07.948 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:30:07.948 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861086+0000", 2024-09-14T06:30:07.949 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0059613829999999998 2024-09-14T06:30:07.949 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:07.949 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:07.949 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:30:07.949 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861100+0000", 2024-09-14T06:30:07.949 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.4644000000000001e-05 2024-09-14T06:30:07.949 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:07.949 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:07.949 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:30:07.949 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861109+0000", 2024-09-14T06:30:07.949 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.541e-06 2024-09-14T06:30:07.949 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:07.949 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:07.949 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:30:07.950 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861117+0000", 2024-09-14T06:30:07.950 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.1049999999999995e-06 2024-09-14T06:30:07.950 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:07.950 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:07.950 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:30:07.950 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.921169+0000", 2024-09-14T06:30:07.950 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.060052405000000003 2024-09-14T06:30:07.950 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:07.950 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:07.950 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:30:07.950 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.921213+0000", 2024-09-14T06:30:07.950 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.3565e-05 2024-09-14T06:30:07.950 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:07.950 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:07.951 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:30:07.951 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.921374+0000", 2024-09-14T06:30:07.951 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00016061800000000001 2024-09-14T06:30:07.951 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:07.951 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:07.951 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:07.951 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:07.951 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:07.951 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:44536 3.2 3:4190d882:::rbd_data.112dca2f6ae1.0000000000000002:17 [read 454656~4096 [fadvise_nocache]] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:30:07.951 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:43.861384+0000", 2024-09-14T06:30:07.951 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 24.019586743000001, 2024-09-14T06:30:07.951 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.063058528000000003, 2024-09-14T06:30:07.951 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:30:07.951 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:30:07.951 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:30:07.952 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:30:07.952 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:30:07.952 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:07.952 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 44536 2024-09-14T06:30:07.952 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:07.952 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:30:07.952 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:07.952 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:30:07.952 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861384+0000", 2024-09-14T06:30:07.952 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:07.952 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:07.952 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:07.952 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:30:07.952 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861384+0000", 2024-09-14T06:30:07.953 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:07.953 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:07.953 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:07.953 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:30:07.953 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861384+0000", 2024-09-14T06:30:07.953 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.4000000000000002e-07 2024-09-14T06:30:07.953 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:07.953 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:07.953 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:30:07.953 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861386+0000", 2024-09-14T06:30:07.953 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.891e-06 2024-09-14T06:30:07.953 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:07.953 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:07.953 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:30:07.954 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861392+0000", 2024-09-14T06:30:07.954 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.9939999999999997e-06 2024-09-14T06:30:07.954 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:07.954 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:07.954 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:30:07.954 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861398+0000", 2024-09-14T06:30:07.954 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.507e-06 2024-09-14T06:30:07.954 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:07.954 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:07.954 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:30:07.954 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.924149+0000", 2024-09-14T06:30:07.954 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.062751135999999999 2024-09-14T06:30:07.954 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:07.954 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:07.955 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:30:07.955 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.924226+0000", 2024-09-14T06:30:07.955 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.7517999999999999e-05 2024-09-14T06:30:07.955 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:07.955 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:07.955 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:30:07.955 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.924442+0000", 2024-09-14T06:30:07.955 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000216042 2024-09-14T06:30:07.955 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:07.955 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:07.955 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:07.955 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:07.955 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:07.955 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:44537 3.2 3:4190d882:::rbd_data.112dca2f6ae1.0000000000000002:17 [read 479232~4096 [fadvise_nocache]] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:30:07.956 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:43.861405+0000", 2024-09-14T06:30:07.956 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 24.019565697000001, 2024-09-14T06:30:07.956 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.063351795000000002, 2024-09-14T06:30:07.956 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:30:07.956 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:30:07.956 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:30:07.956 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:30:07.956 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:30:07.956 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:07.956 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 44537 2024-09-14T06:30:07.956 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:07.956 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:30:07.956 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:07.956 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:30:07.956 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861405+0000", 2024-09-14T06:30:07.957 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:07.957 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:07.957 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:07.957 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:30:07.957 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861405+0000", 2024-09-14T06:30:07.957 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:07.957 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:07.957 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:07.957 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:30:07.957 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861405+0000", 2024-09-14T06:30:07.957 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.9099999999999999e-07 2024-09-14T06:30:07.957 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:07.957 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:07.958 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:30:07.958 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861407+0000", 2024-09-14T06:30:07.958 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.075e-06 2024-09-14T06:30:07.958 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:07.958 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:07.958 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:30:07.958 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861414+0000", 2024-09-14T06:30:07.958 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.4659999999999996e-06 2024-09-14T06:30:07.958 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:07.958 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:07.958 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:30:07.958 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861418+0000", 2024-09-14T06:30:07.958 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.5519999999999998e-06 2024-09-14T06:30:07.958 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:07.959 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:07.959 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:30:07.959 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.924474+0000", 2024-09-14T06:30:07.959 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.063056012999999994 2024-09-14T06:30:07.959 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:07.959 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:07.959 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:30:07.959 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.924556+0000", 2024-09-14T06:30:07.959 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.1181999999999996e-05 2024-09-14T06:30:07.959 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:07.959 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:07.959 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:30:07.959 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.924757+0000", 2024-09-14T06:30:07.959 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00020111599999999999 2024-09-14T06:30:07.959 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:07.960 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:07.960 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:07.960 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:07.960 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:07.960 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:44538 3.2 3:4190d882:::rbd_data.112dca2f6ae1.0000000000000002:17 [read 503808~4096 [fadvise_nocache]] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:30:07.960 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:43.861425+0000", 2024-09-14T06:30:07.960 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 24.019545655999998, 2024-09-14T06:30:07.960 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.063627125000000007, 2024-09-14T06:30:07.960 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:30:07.960 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:30:07.960 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:30:07.960 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:30:07.960 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:30:07.960 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:07.960 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 44538 2024-09-14T06:30:07.961 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:07.961 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:30:07.961 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:07.961 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:30:07.961 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861425+0000", 2024-09-14T06:30:07.961 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:07.961 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:07.961 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:07.961 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:30:07.961 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861425+0000", 2024-09-14T06:30:07.961 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:07.961 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:07.961 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:07.962 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:30:07.962 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861425+0000", 2024-09-14T06:30:07.962 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.9999999999999998e-07 2024-09-14T06:30:07.962 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:07.962 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:07.962 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:30:07.962 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861427+0000", 2024-09-14T06:30:07.962 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.9829999999999999e-06 2024-09-14T06:30:07.962 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:07.962 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:07.962 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:30:07.962 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861433+0000", 2024-09-14T06:30:07.962 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.2889999999999997e-06 2024-09-14T06:30:07.962 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:07.963 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:07.963 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:30:07.963 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861454+0000", 2024-09-14T06:30:07.963 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.1844999999999998e-05 2024-09-14T06:30:07.963 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:07.963 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:07.963 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:30:07.963 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.924786+0000", 2024-09-14T06:30:07.963 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.063331497 2024-09-14T06:30:07.963 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:07.963 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:07.963 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:30:07.963 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.924864+0000", 2024-09-14T06:30:07.963 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.7744000000000006e-05 2024-09-14T06:30:07.964 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:07.964 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:07.964 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:30:07.964 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.925052+0000", 2024-09-14T06:30:07.964 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00018836699999999999 2024-09-14T06:30:07.964 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:07.964 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:07.964 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:07.964 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:07.964 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:07.964 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:44539 3.2 3:4190d882:::rbd_data.112dca2f6ae1.0000000000000002:17 [read 528384~4096 [fadvise_nocache]] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:30:07.964 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:43.861462+0000", 2024-09-14T06:30:07.964 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 24.019508329000001, 2024-09-14T06:30:07.965 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.063855705999999998, 2024-09-14T06:30:07.965 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:30:07.965 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:30:07.965 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:30:07.965 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:30:07.965 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:30:07.965 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:07.965 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 44539 2024-09-14T06:30:07.965 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:07.965 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:30:07.965 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:07.965 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:30:07.965 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861462+0000", 2024-09-14T06:30:07.965 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:07.966 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:07.966 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:07.966 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:30:07.966 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861462+0000", 2024-09-14T06:30:07.966 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:07.966 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:07.966 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:07.966 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:30:07.966 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861463+0000", 2024-09-14T06:30:07.966 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.5200000000000002e-07 2024-09-14T06:30:07.966 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:07.966 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:07.966 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:30:07.967 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861465+0000", 2024-09-14T06:30:07.967 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.5189999999999999e-06 2024-09-14T06:30:07.967 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:07.967 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:07.967 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:30:07.967 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861471+0000", 2024-09-14T06:30:07.967 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.6899999999999997e-06 2024-09-14T06:30:07.967 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:07.967 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:07.967 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:30:07.967 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861475+0000", 2024-09-14T06:30:07.967 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.5730000000000001e-06 2024-09-14T06:30:07.967 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:07.968 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:07.968 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:30:07.968 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.925085+0000", 2024-09-14T06:30:07.968 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.063609902999999995 2024-09-14T06:30:07.968 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:07.968 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:07.968 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:30:07.968 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.925145+0000", 2024-09-14T06:30:07.968 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.9383999999999999e-05 2024-09-14T06:30:07.968 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:07.968 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:07.968 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:30:07.968 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.925318+0000", 2024-09-14T06:30:07.968 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00017318499999999999 2024-09-14T06:30:07.969 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:07.969 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:07.969 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:07.969 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:07.969 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:07.969 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:44540 3.2 3:4190d882:::rbd_data.112dca2f6ae1.0000000000000002:17 [read 552960~4096 [fadvise_nocache]] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:30:07.969 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:43.861483+0000", 2024-09-14T06:30:07.969 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 24.019487898000001, 2024-09-14T06:30:07.969 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.064065681999999999, 2024-09-14T06:30:07.969 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:30:07.969 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:30:07.969 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:30:07.969 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:30:07.969 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:30:07.970 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:07.970 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 44540 2024-09-14T06:30:07.970 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:07.970 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:30:07.970 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:07.970 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:30:07.970 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861483+0000", 2024-09-14T06:30:07.970 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:07.970 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:07.970 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:07.970 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:30:07.970 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861483+0000", 2024-09-14T06:30:07.970 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:07.970 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:07.971 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:07.971 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:30:07.971 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.867079+0000", 2024-09-14T06:30:07.971 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0055966480000000001 2024-09-14T06:30:07.971 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:07.971 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:07.971 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:30:07.971 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.867086+0000", 2024-09-14T06:30:07.971 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.387e-06 2024-09-14T06:30:07.971 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:07.971 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:07.971 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:30:07.971 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.867094+0000", 2024-09-14T06:30:07.971 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.9419999999999995e-06 2024-09-14T06:30:07.972 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:07.972 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:07.972 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:30:07.972 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.867109+0000", 2024-09-14T06:30:07.972 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.5818999999999999e-05 2024-09-14T06:30:07.972 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:07.972 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:07.972 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:30:07.972 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.925354+0000", 2024-09-14T06:30:07.972 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.058244047 2024-09-14T06:30:07.972 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:07.972 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:07.972 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:30:07.973 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.925406+0000", 2024-09-14T06:30:07.973 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.2661999999999998e-05 2024-09-14T06:30:07.973 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:07.973 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:07.973 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:30:07.973 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.925548+0000", 2024-09-14T06:30:07.973 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000142177 2024-09-14T06:30:07.973 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:07.973 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:07.973 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:07.973 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:07.973 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:07.973 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-14T06:30:07.975 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-14T06:30:07.975 INFO:teuthology.orchestra.run.smithi094.stdout:checking clone #15, image image_client.0-clone15 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent16 2024-09-14T06:30:07.976 INFO:tasks.ceph.osd.4.smithi138.stderr:2024-09-14T06:30:07.975+0000 7f6c653c8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33615) UID: 0 2024-09-14T06:30:08.077 INFO:tasks.ceph.osd.1.smithi094.stderr:2024-09-14T06:30:08.076+0000 7f328e47a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33641) UID: 0 2024-09-14T06:30:08.085 INFO:teuthology.orchestra.run.smithi138.stdout:{ 2024-09-14T06:30:08.085 INFO:teuthology.orchestra.run.smithi138.stdout: "ops": [], 2024-09-14T06:30:08.085 INFO:teuthology.orchestra.run.smithi138.stdout: "num_ops": 0 2024-09-14T06:30:08.085 INFO:teuthology.orchestra.run.smithi138.stdout:} 2024-09-14T06:30:08.094 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-14T06:30:08.178 INFO:tasks.ceph.osd.3.smithi094.stderr:2024-09-14T06:30:08.177+0000 7feeeafe8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33655) UID: 0 2024-09-14T06:30:08.191 INFO:teuthology.orchestra.run.smithi094.stdout:2513 read 0x53c6d7b thru 0x53d1747 (0xa9cd bytes) 2024-09-14T06:30:08.191 INFO:teuthology.orchestra.run.smithi094.stdout:2515 punch from 0x8c7c1c to 0x8c9342, (0x1726 bytes) 2024-09-14T06:30:08.195 INFO:teuthology.orchestra.run.smithi094.stdout:2517 write 0x4d35a22 thru 0x4d408ba (0xae99 bytes) 2024-09-14T06:30:08.197 INFO:teuthology.orchestra.run.smithi138.stdout:{ 2024-09-14T06:30:08.197 INFO:teuthology.orchestra.run.smithi138.stdout: "ops": [], 2024-09-14T06:30:08.197 INFO:teuthology.orchestra.run.smithi138.stdout: "complaint_time": 30, 2024-09-14T06:30:08.197 INFO:teuthology.orchestra.run.smithi138.stdout: "num_blocked_ops": 0 2024-09-14T06:30:08.197 INFO:teuthology.orchestra.run.smithi138.stdout:} 2024-09-14T06:30:08.201 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-14T06:30:08.201 INFO:teuthology.orchestra.run.smithi094.stdout:2522 write 0x2a5d143 thru 0x2a68161 (0xb01f bytes) 2024-09-14T06:30:08.207 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-14T06:30:08.214 INFO:teuthology.orchestra.run.smithi094.stdout:2523 writesame 0x963f961 thru 0x964ba82 data_size 0x3b(0xc122 bytes) 2024-09-14T06:30:08.214 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-14T06:30:08.214 INFO:teuthology.orchestra.run.smithi094.stdout:2526 punch from 0x7c2972f to 0x7c3888a, (0xf15b bytes) 2024-09-14T06:30:08.221 INFO:teuthology.orchestra.run.smithi094.stdout:2528 writesame 0xa3e5a72 thru 0xa3ebf3c data_size 0x3d(0x64cb bytes) 2024-09-14T06:30:08.221 INFO:teuthology.orchestra.run.smithi094.stdout:2529 read 0x71f8f91 thru 0x71f9695 (0x705 bytes) 2024-09-14T06:30:08.221 INFO:teuthology.orchestra.run.smithi094.stdout:2531 write 0x4c309d thru 0x4cba23 (0x8987 bytes) 2024-09-14T06:30:08.223 INFO:teuthology.orchestra.run.smithi094.stdout:2533 punch from 0x46b508a to 0x46c4b50, (0xfac6 bytes) 2024-09-14T06:30:08.223 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-14T06:30:08.224 INFO:teuthology.orchestra.run.smithi094.stdout:2539 trunc from 0xa3ebf3d to 0x95754a5 2024-09-14T06:30:08.231 INFO:teuthology.orchestra.run.smithi094.stdout:2540 read 0x67febed thru 0x68049bb (0x5dcf bytes) 2024-09-14T06:30:08.231 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-14T06:30:08.231 INFO:teuthology.orchestra.run.smithi094.stdout:2542 punch from 0x4590aa7 to 0x4597dd9, (0x7332 bytes) 2024-09-14T06:30:08.231 INFO:teuthology.orchestra.run.smithi094.stdout:2543 write 0x59d8a9c thru 0x59e65a2 (0xdb07 bytes) 2024-09-14T06:30:08.237 INFO:teuthology.orchestra.run.smithi094.stdout:2544 read 0x21cb142 thru 0x21d6677 (0xb536 bytes) 2024-09-14T06:30:08.246 INFO:teuthology.orchestra.run.smithi094.stdout:2546 write 0xb29021d thru 0xb299c57 (0x9a3b bytes) 2024-09-14T06:30:08.261 INFO:teuthology.orchestra.run.smithi094.stdout:2547 writesame 0xd53ff53 thru 0xd5488f7 data_size 0x1(0x89a5 bytes) 2024-09-14T06:30:08.269 INFO:teuthology.orchestra.run.smithi094.stdout:2548 write 0xec504b9 thru 0xec5d716 (0xd25e bytes) 2024-09-14T06:30:08.276 INFO:teuthology.orchestra.run.smithi094.stdout:2549 read 0x574f34d thru 0x5750a17 (0x16cb bytes) 2024-09-14T06:30:08.277 INFO:teuthology.orchestra.run.smithi094.stdout:2550 write 0x3011c83 thru 0x301da75 (0xbdf3 bytes) 2024-09-14T06:30:08.279 INFO:teuthology.orchestra.run.smithi094.stdout:2551 read 0x6056194 thru 0x605c260 (0x60cd bytes) 2024-09-14T06:30:08.279 INFO:tasks.ceph.osd.6.smithi138.stderr:2024-09-14T06:30:08.278+0000 7f8cd8849640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33620) UID: 0 2024-09-14T06:30:08.283 INFO:teuthology.orchestra.run.smithi094.stdout:2552 trunc from 0xec5d717 to 0xba2e293 2024-09-14T06:30:08.293 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-14T06:30:08.293 INFO:teuthology.orchestra.run.smithi094.stdout:2557 write 0x6bb64b3 thru 0x6bb6d58 (0x8a6 bytes) 2024-09-14T06:30:08.294 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-14T06:30:08.294 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-14T06:30:08.294 INFO:teuthology.orchestra.run.smithi094.stdout:2560 read 0x8af2e57 thru 0x8af9a34 (0x6bde bytes) 2024-09-14T06:30:08.294 INFO:teuthology.orchestra.run.smithi094.stdout:2561 write 0x633f03a thru 0x63425c9 (0x3590 bytes) 2024-09-14T06:30:08.296 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-14T06:30:08.296 INFO:teuthology.orchestra.run.smithi094.stdout:2563 writesame 0x62259d2 thru 0x622db0b data_size 0x22(0x813a bytes) 2024-09-14T06:30:08.298 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-14T06:30:08.298 INFO:teuthology.orchestra.run.smithi094.stdout:2567 read 0x32f3450 thru 0x32fbef4 (0x8aa5 bytes) 2024-09-14T06:30:08.299 INFO:teuthology.orchestra.run.smithi094.stdout:2568 write 0x8099537 thru 0x809f6e9 (0x61b3 bytes) 2024-09-14T06:30:08.306 INFO:teuthology.orchestra.run.smithi094.stdout:2569 read 0x4902707 thru 0x490bf57 (0x9851 bytes) 2024-09-14T06:30:08.311 INFO:teuthology.orchestra.run.smithi094.stdout:2570 write 0xbe66b1d thru 0xbe6ffe1 (0x94c5 bytes) 2024-09-14T06:30:08.318 INFO:teuthology.orchestra.run.smithi094.stdout:2571 write 0x1591fee thru 0x159a7fc (0x880f bytes) 2024-09-14T06:30:08.321 INFO:teuthology.orchestra.run.smithi094.stdout:2572 read 0x7821bf6 thru 0x7831aab (0xfeb6 bytes) 2024-09-14T06:30:08.321 INFO:teuthology.orchestra.run.smithi094.stdout:2573 read 0x128c409 thru 0x129485e (0x8456 bytes) 2024-09-14T06:30:08.321 INFO:teuthology.orchestra.run.smithi094.stdout:2574 punch from 0x81f7ece to 0x820770a, (0xf83c bytes) 2024-09-14T06:30:08.321 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-14T06:30:08.321 INFO:teuthology.orchestra.run.smithi094.stdout:2576 write 0xa5a6601 thru 0xa5ad52b (0x6f2b bytes) 2024-09-14T06:30:08.323 INFO:teuthology.orchestra.run.smithi138.stdout:{ 2024-09-14T06:30:08.323 INFO:teuthology.orchestra.run.smithi138.stdout: "size": 20, 2024-09-14T06:30:08.323 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 600, 2024-09-14T06:30:08.323 INFO:teuthology.orchestra.run.smithi138.stdout: "ops": [ 2024-09-14T06:30:08.323 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:08.323 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:19328 3.3 3:c34e5e51:::rbd_data.112d92027590.0000000000000024:f [read 1926144~1024 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:30:08.323 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:27.059768+0000", 2024-09-14T06:30:08.323 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 41.261064564000002, 2024-09-14T06:30:08.323 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046369147999999999, 2024-09-14T06:30:08.323 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:30:08.323 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:30:08.323 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:30:08.324 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:30:08.324 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:30:08.324 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:08.324 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 19328 2024-09-14T06:30:08.324 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:08.324 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:30:08.324 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:08.324 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:30:08.324 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.059768+0000", 2024-09-14T06:30:08.324 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:08.324 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:08.324 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:08.324 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:30:08.324 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.059768+0000", 2024-09-14T06:30:08.325 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:08.325 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:08.325 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:08.325 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:30:08.325 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.059874+0000", 2024-09-14T06:30:08.325 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00010551400000000001 2024-09-14T06:30:08.325 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:08.325 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:08.325 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:30:08.325 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.059877+0000", 2024-09-14T06:30:08.325 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.8279999999999999e-06 2024-09-14T06:30:08.325 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:08.325 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:08.326 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:30:08.326 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.059884+0000", 2024-09-14T06:30:08.326 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.6389999999999992e-06 2024-09-14T06:30:08.326 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:08.326 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:08.326 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:30:08.326 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.059890+0000", 2024-09-14T06:30:08.326 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.49e-06 2024-09-14T06:30:08.326 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:08.326 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:08.326 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:30:08.326 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.105804+0000", 2024-09-14T06:30:08.326 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.045914242000000001 2024-09-14T06:30:08.326 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:08.327 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:08.327 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:30:08.327 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.105886+0000", 2024-09-14T06:30:08.327 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.2305999999999994e-05 2024-09-14T06:30:08.327 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:08.327 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:08.327 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:30:08.327 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.106138+0000", 2024-09-14T06:30:08.327 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.000251129 2024-09-14T06:30:08.327 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:08.327 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:30:08.327 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:08.327 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:08.328 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:08.328 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:19329 3.3 3:c34e5e51:::rbd_data.112d92027590.0000000000000024:f [read 1936384~1024 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:30:08.328 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:27.059899+0000", 2024-09-14T06:30:08.328 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 41.260934040999999, 2024-09-14T06:30:08.328 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046572122000000001, 2024-09-14T06:30:08.328 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:30:08.328 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:30:08.328 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:30:08.328 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:30:08.328 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:30:08.328 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:08.328 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 19329 2024-09-14T06:30:08.328 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:08.329 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:30:08.329 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:08.329 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:30:08.329 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.059899+0000", 2024-09-14T06:30:08.329 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:08.329 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:08.329 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:08.329 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:30:08.329 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.059899+0000", 2024-09-14T06:30:08.329 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:08.329 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:08.329 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:08.330 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:30:08.330 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062501+0000", 2024-09-14T06:30:08.330 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.0026016419999999999 2024-09-14T06:30:08.330 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:08.330 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:08.330 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:30:08.330 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062514+0000", 2024-09-14T06:30:08.330 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.2947e-05 2024-09-14T06:30:08.330 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:08.330 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:08.330 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:30:08.330 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062523+0000", 2024-09-14T06:30:08.330 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.5519999999999993e-06 2024-09-14T06:30:08.330 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:08.331 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:08.331 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:30:08.331 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062534+0000", 2024-09-14T06:30:08.331 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.1416e-05 2024-09-14T06:30:08.331 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:08.331 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:08.331 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:30:08.331 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.106179+0000", 2024-09-14T06:30:08.331 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.043644049999999997 2024-09-14T06:30:08.331 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:08.331 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:08.331 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:30:08.331 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.106262+0000", 2024-09-14T06:30:08.332 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.3178999999999998e-05 2024-09-14T06:30:08.332 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:08.332 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:08.332 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:30:08.332 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.106471+0000", 2024-09-14T06:30:08.332 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00020933599999999999 2024-09-14T06:30:08.332 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:08.332 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:30:08.332 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:08.332 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:08.332 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:08.332 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:19351 3.3 3:c34e5e51:::rbd_data.112d92027590.0000000000000024:f [read 1159168~1024 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:30:08.332 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:27.062766+0000", 2024-09-14T06:30:08.333 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 41.258066816000003, 2024-09-14T06:30:08.333 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046099454999999998, 2024-09-14T06:30:08.333 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:30:08.333 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:30:08.333 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:30:08.333 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:30:08.333 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:30:08.333 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:08.333 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 19351 2024-09-14T06:30:08.333 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:08.333 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:30:08.333 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:08.333 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:30:08.334 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062766+0000", 2024-09-14T06:30:08.334 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:08.334 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:08.334 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:08.334 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:30:08.334 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062766+0000", 2024-09-14T06:30:08.334 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:08.334 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:08.334 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:08.334 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:30:08.334 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062766+0000", 2024-09-14T06:30:08.334 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.6e-07 2024-09-14T06:30:08.334 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:08.334 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:08.335 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:30:08.335 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062767+0000", 2024-09-14T06:30:08.335 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.0070000000000001e-06 2024-09-14T06:30:08.335 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:08.335 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:08.335 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:30:08.335 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062785+0000", 2024-09-14T06:30:08.335 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.7884999999999999e-05 2024-09-14T06:30:08.335 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:08.335 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:08.335 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:30:08.335 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062790+0000", 2024-09-14T06:30:08.335 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.0900000000000004e-06 2024-09-14T06:30:08.335 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:08.336 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:08.336 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:30:08.336 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.108735+0000", 2024-09-14T06:30:08.336 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.045944411999999997 2024-09-14T06:30:08.336 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:08.336 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:08.336 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:30:08.336 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.108809+0000", 2024-09-14T06:30:08.336 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.4157000000000002e-05 2024-09-14T06:30:08.336 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:08.336 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:08.336 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:30:08.336 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.108866+0000", 2024-09-14T06:30:08.337 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.6644000000000002e-05 2024-09-14T06:30:08.337 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:08.337 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:30:08.337 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:08.337 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:08.337 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:08.337 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:19352 3.3 3:c34e5e51:::rbd_data.112d92027590.0000000000000024:f [read 1169408~1024 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:30:08.337 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:27.062798+0000", 2024-09-14T06:30:08.337 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 41.258035405000001, 2024-09-14T06:30:08.337 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046255900000000003, 2024-09-14T06:30:08.337 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:30:08.337 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:30:08.337 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:30:08.338 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:30:08.338 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:30:08.338 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:08.338 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 19352 2024-09-14T06:30:08.338 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:08.338 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:30:08.338 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:08.338 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:30:08.338 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062798+0000", 2024-09-14T06:30:08.338 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:08.338 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:08.338 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:08.338 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:30:08.338 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062798+0000", 2024-09-14T06:30:08.339 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:08.339 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:08.339 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:08.339 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:30:08.339 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062798+0000", 2024-09-14T06:30:08.339 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.37e-07 2024-09-14T06:30:08.339 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:08.339 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:08.339 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:30:08.339 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062800+0000", 2024-09-14T06:30:08.339 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.0269999999999998e-06 2024-09-14T06:30:08.339 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:08.339 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:08.339 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:30:08.340 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062806+0000", 2024-09-14T06:30:08.340 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.6690000000000003e-06 2024-09-14T06:30:08.340 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:08.340 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:08.340 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:30:08.340 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062810+0000", 2024-09-14T06:30:08.340 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.3449999999999999e-06 2024-09-14T06:30:08.340 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:08.340 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:08.340 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:30:08.340 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.108892+0000", 2024-09-14T06:30:08.340 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046081715000000002 2024-09-14T06:30:08.341 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:08.341 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:08.341 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:30:08.341 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.108940+0000", 2024-09-14T06:30:08.341 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.7815999999999998e-05 2024-09-14T06:30:08.341 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:08.341 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:08.341 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:30:08.341 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.109053+0000", 2024-09-14T06:30:08.341 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.000113791 2024-09-14T06:30:08.341 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:08.341 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:30:08.341 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:08.341 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:08.342 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:08.342 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:19353 3.3 3:c34e5e51:::rbd_data.112d92027590.0000000000000024:f [read 1179648~1024 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:30:08.342 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:27.062816+0000", 2024-09-14T06:30:08.342 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 41.258016886, 2024-09-14T06:30:08.342 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046481256999999998, 2024-09-14T06:30:08.342 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:30:08.342 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:30:08.342 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:30:08.342 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:30:08.342 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:30:08.342 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:08.342 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 19353 2024-09-14T06:30:08.342 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:08.343 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:30:08.343 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:08.343 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:30:08.343 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062816+0000", 2024-09-14T06:30:08.343 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:08.343 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:08.343 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:08.343 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:30:08.343 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062816+0000", 2024-09-14T06:30:08.343 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:08.343 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:08.343 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:08.343 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:30:08.344 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062816+0000", 2024-09-14T06:30:08.344 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.2899999999999999e-07 2024-09-14T06:30:08.344 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:08.344 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:08.344 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:30:08.344 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062818+0000", 2024-09-14T06:30:08.344 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.699e-06 2024-09-14T06:30:08.344 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:08.344 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:08.344 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:30:08.344 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062825+0000", 2024-09-14T06:30:08.344 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.878e-06 2024-09-14T06:30:08.344 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:08.344 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:08.345 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:30:08.345 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062830+0000", 2024-09-14T06:30:08.345 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.5410000000000002e-06 2024-09-14T06:30:08.345 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:08.345 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:08.345 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:30:08.345 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.109096+0000", 2024-09-14T06:30:08.345 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046266517 2024-09-14T06:30:08.345 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:08.345 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:08.345 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:30:08.345 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.109190+0000", 2024-09-14T06:30:08.345 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.4000999999999997e-05 2024-09-14T06:30:08.345 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:08.346 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:08.346 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:30:08.346 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.109297+0000", 2024-09-14T06:30:08.346 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.000107292 2024-09-14T06:30:08.346 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:08.346 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:30:08.346 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:08.346 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:08.346 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:08.346 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:19354 3.3 3:c34e5e51:::rbd_data.112d92027590.0000000000000024:f [read 1189888~1024 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:30:08.346 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:27.062842+0000", 2024-09-14T06:30:08.346 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 41.257991431000001, 2024-09-14T06:30:08.346 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046594257, 2024-09-14T06:30:08.347 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:30:08.347 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:30:08.347 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:30:08.347 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:30:08.347 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:30:08.347 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:08.347 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 19354 2024-09-14T06:30:08.347 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:08.347 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:30:08.347 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:08.347 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:30:08.347 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062842+0000", 2024-09-14T06:30:08.347 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:08.348 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:08.348 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:08.348 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:30:08.348 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062842+0000", 2024-09-14T06:30:08.348 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:08.348 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:08.348 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:08.348 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:30:08.348 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062842+0000", 2024-09-14T06:30:08.348 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.7399999999999999e-07 2024-09-14T06:30:08.348 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:08.348 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:08.348 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:30:08.348 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062859+0000", 2024-09-14T06:30:08.349 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.7010000000000001e-05 2024-09-14T06:30:08.349 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:08.349 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:08.349 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:30:08.349 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062866+0000", 2024-09-14T06:30:08.349 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.9990000000000002e-06 2024-09-14T06:30:08.349 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:08.349 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:08.349 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:30:08.349 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062873+0000", 2024-09-14T06:30:08.349 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.8569999999999997e-06 2024-09-14T06:30:08.349 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:08.349 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:08.350 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:30:08.350 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.109335+0000", 2024-09-14T06:30:08.350 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046461943999999998 2024-09-14T06:30:08.350 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:08.350 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:08.350 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:30:08.350 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.109377+0000", 2024-09-14T06:30:08.350 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.2558999999999997e-05 2024-09-14T06:30:08.350 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:08.350 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:08.350 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:30:08.350 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.109436+0000", 2024-09-14T06:30:08.350 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.8514000000000003e-05 2024-09-14T06:30:08.351 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:08.351 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:30:08.351 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:08.351 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:08.351 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:08.351 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:19355 3.3 3:c34e5e51:::rbd_data.112d92027590.0000000000000024:f [read 1200128~1024 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:30:08.351 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:27.062884+0000", 2024-09-14T06:30:08.351 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 41.257948757999998, 2024-09-14T06:30:08.351 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046681650999999998, 2024-09-14T06:30:08.351 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:30:08.351 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:30:08.351 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:30:08.351 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:30:08.351 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:30:08.352 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:08.352 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 19355 2024-09-14T06:30:08.352 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:08.352 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:30:08.352 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:08.352 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:30:08.352 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062884+0000", 2024-09-14T06:30:08.352 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:08.352 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:08.352 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:08.352 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:30:08.352 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062884+0000", 2024-09-14T06:30:08.353 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:08.353 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:08.353 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:08.353 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:30:08.353 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062885+0000", 2024-09-14T06:30:08.353 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.8000000000000001e-07 2024-09-14T06:30:08.353 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:08.353 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:08.353 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:30:08.353 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062886+0000", 2024-09-14T06:30:08.353 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.8920000000000001e-06 2024-09-14T06:30:08.353 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:08.353 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:08.354 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:30:08.354 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062892+0000", 2024-09-14T06:30:08.354 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.84e-06 2024-09-14T06:30:08.354 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:08.354 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:08.354 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:30:08.354 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062899+0000", 2024-09-14T06:30:08.354 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.5339999999999996e-06 2024-09-14T06:30:08.354 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:08.354 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:08.354 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:30:08.354 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.109466+0000", 2024-09-14T06:30:08.354 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046566919999999998 2024-09-14T06:30:08.354 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:08.355 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:08.355 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:30:08.355 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.109510+0000", 2024-09-14T06:30:08.355 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.4153000000000002e-05 2024-09-14T06:30:08.355 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:08.355 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:08.355 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:30:08.355 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.109566+0000", 2024-09-14T06:30:08.355 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.5931999999999999e-05 2024-09-14T06:30:08.355 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:08.355 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:30:08.355 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:08.355 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:08.356 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:08.356 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:19356 3.3 3:c34e5e51:::rbd_data.112d92027590.0000000000000024:f [read 1210368~1024 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:30:08.356 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:27.062908+0000", 2024-09-14T06:30:08.356 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 41.257925321000002, 2024-09-14T06:30:08.356 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046807498000000003, 2024-09-14T06:30:08.356 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:30:08.356 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:30:08.356 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:30:08.356 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:30:08.356 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:30:08.356 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:08.356 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 19356 2024-09-14T06:30:08.356 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:08.357 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:30:08.357 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:08.357 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:30:08.357 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062908+0000", 2024-09-14T06:30:08.357 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:08.357 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:08.357 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:08.357 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:30:08.357 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062908+0000", 2024-09-14T06:30:08.357 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:08.357 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:08.357 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:08.357 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:30:08.358 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062908+0000", 2024-09-14T06:30:08.358 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.5699999999999998e-07 2024-09-14T06:30:08.358 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:08.358 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:08.358 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:30:08.358 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062910+0000", 2024-09-14T06:30:08.358 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.7710000000000001e-06 2024-09-14T06:30:08.358 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:08.358 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:08.358 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:30:08.358 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062920+0000", 2024-09-14T06:30:08.358 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.0654e-05 2024-09-14T06:30:08.358 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:08.359 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:08.359 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:30:08.359 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062931+0000", 2024-09-14T06:30:08.359 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.075e-05 2024-09-14T06:30:08.359 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:08.359 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:08.359 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:30:08.359 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.109595+0000", 2024-09-14T06:30:08.359 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046663713000000002 2024-09-14T06:30:08.359 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:08.359 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:08.359 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:30:08.359 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.109632+0000", 2024-09-14T06:30:08.359 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.7505000000000001e-05 2024-09-14T06:30:08.360 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:08.360 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:08.360 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:30:08.360 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.109715+0000", 2024-09-14T06:30:08.360 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.2748000000000001e-05 2024-09-14T06:30:08.360 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:08.360 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:30:08.360 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:08.360 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:08.360 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:08.360 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:19357 3.3 3:c34e5e51:::rbd_data.112d92027590.0000000000000024:f [read 1220608~1024 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:30:08.360 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:27.062939+0000", 2024-09-14T06:30:08.360 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 41.257893873, 2024-09-14T06:30:08.361 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046920508, 2024-09-14T06:30:08.361 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:30:08.361 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:30:08.361 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:30:08.361 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:30:08.361 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:30:08.361 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:08.361 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 19357 2024-09-14T06:30:08.361 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:08.361 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:30:08.361 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:08.361 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:30:08.361 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062939+0000", 2024-09-14T06:30:08.362 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:08.362 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:08.362 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:08.362 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:30:08.362 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062939+0000", 2024-09-14T06:30:08.362 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:08.362 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:08.362 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:08.362 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:30:08.362 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062940+0000", 2024-09-14T06:30:08.362 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.32e-07 2024-09-14T06:30:08.362 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:08.362 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:08.362 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:30:08.363 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062942+0000", 2024-09-14T06:30:08.363 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.1919999999999999e-06 2024-09-14T06:30:08.363 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:08.363 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:08.363 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:30:08.363 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062948+0000", 2024-09-14T06:30:08.363 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.1149999999999999e-06 2024-09-14T06:30:08.363 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:08.363 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:08.363 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:30:08.363 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062960+0000", 2024-09-14T06:30:08.363 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.1895e-05 2024-09-14T06:30:08.363 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:08.363 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:08.364 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:30:08.364 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.109704+0000", 2024-09-14T06:30:08.364 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046744584999999998 2024-09-14T06:30:08.364 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:08.364 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:08.364 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:30:08.364 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.109777+0000", 2024-09-14T06:30:08.364 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.2282999999999999e-05 2024-09-14T06:30:08.364 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:08.364 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:08.364 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:30:08.364 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.109860+0000", 2024-09-14T06:30:08.364 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.3005999999999998e-05 2024-09-14T06:30:08.365 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:08.365 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:30:08.365 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:08.365 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:08.365 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:08.365 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:19358 3.3 3:c34e5e51:::rbd_data.112d92027590.0000000000000024:f [read 1230848~1024 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:30:08.365 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:27.062967+0000", 2024-09-14T06:30:08.365 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 41.257866243000002, 2024-09-14T06:30:08.365 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047144471, 2024-09-14T06:30:08.365 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:30:08.365 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:30:08.365 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:30:08.365 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:30:08.365 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:30:08.366 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:08.366 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 19358 2024-09-14T06:30:08.366 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:08.366 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:30:08.366 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:08.366 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:30:08.366 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062967+0000", 2024-09-14T06:30:08.366 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:08.366 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:08.366 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:08.366 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:30:08.366 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062967+0000", 2024-09-14T06:30:08.366 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:08.366 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:08.367 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:08.367 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:30:08.367 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062967+0000", 2024-09-14T06:30:08.367 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.89e-07 2024-09-14T06:30:08.367 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:08.367 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:08.367 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:30:08.367 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062969+0000", 2024-09-14T06:30:08.367 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.9929999999999998e-06 2024-09-14T06:30:08.367 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:08.367 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:08.367 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:30:08.367 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062975+0000", 2024-09-14T06:30:08.368 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.9030000000000001e-06 2024-09-14T06:30:08.368 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:08.368 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:08.368 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:30:08.368 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062979+0000", 2024-09-14T06:30:08.368 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.2250000000000002e-06 2024-09-14T06:30:08.368 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:08.368 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:08.368 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:30:08.368 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.109934+0000", 2024-09-14T06:30:08.368 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046954410000000002 2024-09-14T06:30:08.368 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:08.368 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:08.369 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:30:08.369 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.110007+0000", 2024-09-14T06:30:08.369 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.3829000000000001e-05 2024-09-14T06:30:08.369 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:08.369 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:08.369 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:30:08.369 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.110111+0000", 2024-09-14T06:30:08.369 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00010372199999999999 2024-09-14T06:30:08.369 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:08.369 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:30:08.369 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:08.369 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:08.369 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:08.370 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:19359 3.3 3:c34e5e51:::rbd_data.112d92027590.0000000000000024:f [read 1241088~1024 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:30:08.370 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:27.062987+0000", 2024-09-14T06:30:08.370 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 41.257846153000003, 2024-09-14T06:30:08.370 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047297182, 2024-09-14T06:30:08.370 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:30:08.370 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:30:08.370 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:30:08.370 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:30:08.370 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:30:08.370 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:08.370 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 19359 2024-09-14T06:30:08.370 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:08.370 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:30:08.370 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:08.371 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:30:08.371 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062987+0000", 2024-09-14T06:30:08.371 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:08.371 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:08.371 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:08.371 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:30:08.371 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062987+0000", 2024-09-14T06:30:08.371 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:08.371 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:08.371 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:08.371 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:30:08.371 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062987+0000", 2024-09-14T06:30:08.371 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.2500000000000001e-07 2024-09-14T06:30:08.372 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:08.372 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:08.372 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:30:08.372 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062989+0000", 2024-09-14T06:30:08.372 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.6080000000000001e-06 2024-09-14T06:30:08.372 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:08.372 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:08.372 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:30:08.372 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062994+0000", 2024-09-14T06:30:08.372 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.5520000000000004e-06 2024-09-14T06:30:08.372 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:08.372 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:08.372 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:30:08.372 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062999+0000", 2024-09-14T06:30:08.373 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.3930000000000001e-06 2024-09-14T06:30:08.373 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:08.373 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:08.373 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:30:08.373 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.110142+0000", 2024-09-14T06:30:08.373 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047143154 2024-09-14T06:30:08.373 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:08.373 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:08.373 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:30:08.373 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.110219+0000", 2024-09-14T06:30:08.373 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.6737999999999996e-05 2024-09-14T06:30:08.373 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:08.373 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:08.374 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:30:08.374 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.110284+0000", 2024-09-14T06:30:08.374 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.5412000000000003e-05 2024-09-14T06:30:08.374 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:08.374 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:30:08.374 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:08.374 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:08.374 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:08.374 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:19360 3.3 3:c34e5e51:::rbd_data.112d92027590.0000000000000024:f [read 1251328~1024 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:30:08.374 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:27.063017+0000", 2024-09-14T06:30:08.374 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 41.257815692000001, 2024-09-14T06:30:08.374 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047453823999999999, 2024-09-14T06:30:08.374 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:30:08.375 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:30:08.375 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:30:08.375 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:30:08.375 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:30:08.375 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:08.375 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 19360 2024-09-14T06:30:08.375 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:08.375 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:30:08.375 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:08.375 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:30:08.375 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.063017+0000", 2024-09-14T06:30:08.375 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:08.375 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:08.376 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:08.376 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:30:08.376 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.063017+0000", 2024-09-14T06:30:08.376 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:08.376 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:08.376 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:08.376 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:30:08.376 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.064977+0000", 2024-09-14T06:30:08.376 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.0019597080000000001 2024-09-14T06:30:08.376 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:08.376 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:08.376 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:30:08.376 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.064987+0000", 2024-09-14T06:30:08.377 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.9620000000000005e-06 2024-09-14T06:30:08.377 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:08.377 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:08.377 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:30:08.377 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065000+0000", 2024-09-14T06:30:08.377 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.3173000000000001e-05 2024-09-14T06:30:08.377 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:08.377 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:08.377 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:30:08.377 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065005+0000", 2024-09-14T06:30:08.377 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.9960000000000001e-06 2024-09-14T06:30:08.377 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:08.377 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:08.378 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:30:08.378 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.110314+0000", 2024-09-14T06:30:08.378 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.04530899 2024-09-14T06:30:08.378 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:08.378 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:08.378 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:30:08.378 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.110398+0000", 2024-09-14T06:30:08.378 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.3539000000000001e-05 2024-09-14T06:30:08.378 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:08.378 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:08.378 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:30:08.378 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.110471+0000", 2024-09-14T06:30:08.378 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.3455999999999996e-05 2024-09-14T06:30:08.379 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:08.379 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:30:08.379 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:08.379 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:08.379 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:08.379 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:19365 3.3 3:c34e5e51:::rbd_data.112d92027590.0000000000000024:f [read 1302528~1024 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:30:08.379 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:27.065058+0000", 2024-09-14T06:30:08.379 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 41.255775022000002, 2024-09-14T06:30:08.379 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046105526000000001, 2024-09-14T06:30:08.379 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:30:08.379 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:30:08.379 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:30:08.379 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:30:08.380 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:30:08.380 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:08.380 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 19365 2024-09-14T06:30:08.380 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:08.380 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:30:08.380 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:08.380 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:30:08.380 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065058+0000", 2024-09-14T06:30:08.380 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:08.380 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:08.380 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:08.380 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:30:08.381 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065058+0000", 2024-09-14T06:30:08.381 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:08.381 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:08.381 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:08.381 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:30:08.381 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065058+0000", 2024-09-14T06:30:08.381 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.1899999999999999e-07 2024-09-14T06:30:08.381 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:08.381 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:08.381 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:30:08.381 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065059+0000", 2024-09-14T06:30:08.381 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.1299999999999998e-07 2024-09-14T06:30:08.381 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:08.381 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:08.382 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:30:08.382 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065063+0000", 2024-09-14T06:30:08.382 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.1099999999999996e-06 2024-09-14T06:30:08.382 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:08.382 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:08.382 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:30:08.382 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065070+0000", 2024-09-14T06:30:08.382 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.973e-06 2024-09-14T06:30:08.382 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:08.382 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:08.382 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:30:08.382 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.111014+0000", 2024-09-14T06:30:08.382 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.045943373000000003 2024-09-14T06:30:08.383 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:08.383 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:08.383 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:30:08.383 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.111073+0000", 2024-09-14T06:30:08.383 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.8987999999999998e-05 2024-09-14T06:30:08.383 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:08.383 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:08.383 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:30:08.383 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.111163+0000", 2024-09-14T06:30:08.383 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.0950000000000002e-05 2024-09-14T06:30:08.383 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:08.383 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:30:08.383 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:08.383 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:08.384 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:08.384 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:19366 3.3 3:c34e5e51:::rbd_data.112d92027590.0000000000000024:f [read 1312768~1024 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:30:08.384 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:27.065074+0000", 2024-09-14T06:30:08.384 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 41.255759171999998, 2024-09-14T06:30:08.384 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046289858000000003, 2024-09-14T06:30:08.384 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:30:08.384 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:30:08.384 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:30:08.384 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:30:08.384 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:30:08.384 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:08.384 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 19366 2024-09-14T06:30:08.384 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:08.385 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:30:08.385 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:08.385 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:30:08.385 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065074+0000", 2024-09-14T06:30:08.385 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:08.385 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:08.385 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:08.385 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:30:08.385 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065074+0000", 2024-09-14T06:30:08.385 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:08.385 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:08.385 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:08.385 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:30:08.386 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065074+0000", 2024-09-14T06:30:08.386 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.3900000000000001e-07 2024-09-14T06:30:08.386 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:08.386 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:08.386 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:30:08.386 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065078+0000", 2024-09-14T06:30:08.386 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.3229999999999999e-06 2024-09-14T06:30:08.386 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:08.386 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:08.386 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:30:08.386 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065082+0000", 2024-09-14T06:30:08.386 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.8299999999999998e-06 2024-09-14T06:30:08.386 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:08.387 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:08.387 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:30:08.387 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065085+0000", 2024-09-14T06:30:08.387 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.8660000000000002e-06 2024-09-14T06:30:08.387 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:08.387 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:08.387 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:30:08.387 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.111270+0000", 2024-09-14T06:30:08.387 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046185035999999999 2024-09-14T06:30:08.387 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:08.387 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:08.387 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:30:08.387 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.111311+0000", 2024-09-14T06:30:08.388 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.0935e-05 2024-09-14T06:30:08.388 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:08.388 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:08.388 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:30:08.388 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.111364+0000", 2024-09-14T06:30:08.388 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.2629e-05 2024-09-14T06:30:08.388 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:08.388 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:30:08.388 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:08.388 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:08.388 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:08.388 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:19367 3.3 3:c34e5e51:::rbd_data.112d92027590.0000000000000024:f [read 1323008~1024 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:30:08.388 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:27.065115+0000", 2024-09-14T06:30:08.389 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 41.255717611000001, 2024-09-14T06:30:08.389 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046407003000000002, 2024-09-14T06:30:08.389 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:30:08.389 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:30:08.389 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:30:08.389 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:30:08.389 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:30:08.389 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:08.389 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 19367 2024-09-14T06:30:08.389 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:08.389 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:30:08.389 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:08.389 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:30:08.390 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065115+0000", 2024-09-14T06:30:08.390 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:08.390 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:08.390 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:08.390 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:30:08.390 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065115+0000", 2024-09-14T06:30:08.390 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:08.390 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:08.390 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:08.390 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:30:08.390 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065116+0000", 2024-09-14T06:30:08.390 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.4999999999999998e-07 2024-09-14T06:30:08.390 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:08.391 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:08.391 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:30:08.391 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065117+0000", 2024-09-14T06:30:08.391 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.189e-06 2024-09-14T06:30:08.391 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:08.391 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:08.391 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:30:08.391 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065122+0000", 2024-09-14T06:30:08.391 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.3709999999999999e-06 2024-09-14T06:30:08.391 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:08.391 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:08.391 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:30:08.391 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065125+0000", 2024-09-14T06:30:08.391 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.8940000000000002e-06 2024-09-14T06:30:08.392 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:08.392 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:08.392 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:30:08.392 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.111402+0000", 2024-09-14T06:30:08.392 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046276645999999998 2024-09-14T06:30:08.392 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:08.392 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:08.392 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:30:08.392 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.111445+0000", 2024-09-14T06:30:08.392 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.3288000000000003e-05 2024-09-14T06:30:08.392 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:08.392 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:08.392 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:30:08.393 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.111522+0000", 2024-09-14T06:30:08.393 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.7164999999999997e-05 2024-09-14T06:30:08.393 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:08.393 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:30:08.393 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:08.393 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:08.393 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:08.393 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:19368 3.3 3:c34e5e51:::rbd_data.112d92027590.0000000000000024:f [read 1333248~1024 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:30:08.393 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:27.065130+0000", 2024-09-14T06:30:08.393 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 41.255703320999999, 2024-09-14T06:30:08.393 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046507275000000001, 2024-09-14T06:30:08.393 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:30:08.393 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:30:08.394 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:30:08.394 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:30:08.394 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:30:08.394 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:08.394 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 19368 2024-09-14T06:30:08.394 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:08.394 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:30:08.394 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:08.394 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:30:08.394 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065130+0000", 2024-09-14T06:30:08.394 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:08.394 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:08.394 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:08.395 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:30:08.395 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065130+0000", 2024-09-14T06:30:08.395 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:08.395 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:08.395 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:08.395 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:30:08.395 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065130+0000", 2024-09-14T06:30:08.395 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.0800000000000001e-07 2024-09-14T06:30:08.395 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:08.395 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:08.395 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:30:08.395 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065131+0000", 2024-09-14T06:30:08.395 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.8700000000000004e-07 2024-09-14T06:30:08.396 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:08.396 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:08.396 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:30:08.396 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065134+0000", 2024-09-14T06:30:08.396 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.1039999999999998e-06 2024-09-14T06:30:08.396 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:08.396 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:08.396 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:30:08.396 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065136+0000", 2024-09-14T06:30:08.396 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.3949999999999999e-06 2024-09-14T06:30:08.396 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:08.396 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:08.396 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:30:08.396 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.111541+0000", 2024-09-14T06:30:08.397 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046404206000000003 2024-09-14T06:30:08.397 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:08.397 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:08.397 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:30:08.397 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.111582+0000", 2024-09-14T06:30:08.397 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.1051999999999999e-05 2024-09-14T06:30:08.397 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:08.397 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:08.397 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:30:08.397 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.111637+0000", 2024-09-14T06:30:08.397 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.5322999999999999e-05 2024-09-14T06:30:08.397 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:08.397 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:30:08.398 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:08.398 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:08.398 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:08.398 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:19369 3.3 3:c34e5e51:::rbd_data.112d92027590.0000000000000024:f [read 1343488~1024 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:30:08.398 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:27.065139+0000", 2024-09-14T06:30:08.398 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 41.255693504, 2024-09-14T06:30:08.398 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046682169000000003, 2024-09-14T06:30:08.398 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:30:08.398 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:30:08.398 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:30:08.398 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:30:08.398 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:30:08.398 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:08.399 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 19369 2024-09-14T06:30:08.399 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:08.399 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:30:08.399 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:08.399 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:30:08.399 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065139+0000", 2024-09-14T06:30:08.399 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:08.399 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:08.399 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:08.399 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:30:08.399 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065139+0000", 2024-09-14T06:30:08.399 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:08.399 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:08.399 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:08.399 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:30:08.400 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065140+0000", 2024-09-14T06:30:08.400 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.8299999999999998e-07 2024-09-14T06:30:08.400 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:08.400 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:08.400 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:30:08.400 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065141+0000", 2024-09-14T06:30:08.400 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.3900000000000003e-07 2024-09-14T06:30:08.400 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:08.400 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:08.400 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:30:08.400 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065144+0000", 2024-09-14T06:30:08.400 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.7409999999999998e-06 2024-09-14T06:30:08.400 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:08.401 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:08.401 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:30:08.401 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065147+0000", 2024-09-14T06:30:08.401 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.4150000000000002e-06 2024-09-14T06:30:08.401 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:08.401 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:08.401 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:30:08.401 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.111673+0000", 2024-09-14T06:30:08.401 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046526004000000003 2024-09-14T06:30:08.401 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:08.401 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:08.401 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:30:08.401 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.111749+0000", 2024-09-14T06:30:08.402 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.5741999999999994e-05 2024-09-14T06:30:08.402 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:08.402 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:08.402 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:30:08.402 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.111822+0000", 2024-09-14T06:30:08.402 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.3045000000000003e-05 2024-09-14T06:30:08.402 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:08.402 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:30:08.402 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:08.402 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:08.402 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:08.402 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:19370 3.3 3:c34e5e51:::rbd_data.112d92027590.0000000000000024:f [read 1353728~1024 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:30:08.402 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:27.065150+0000", 2024-09-14T06:30:08.402 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 41.255682774999997, 2024-09-14T06:30:08.403 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046897556999999999, 2024-09-14T06:30:08.403 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:30:08.403 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:30:08.403 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:30:08.403 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:30:08.403 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:30:08.403 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:08.403 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 19370 2024-09-14T06:30:08.403 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:08.403 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:30:08.403 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:08.403 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:30:08.403 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065150+0000", 2024-09-14T06:30:08.403 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:08.404 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:08.404 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:08.404 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:30:08.404 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065150+0000", 2024-09-14T06:30:08.404 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:08.404 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:08.404 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:08.404 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:30:08.404 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065150+0000", 2024-09-14T06:30:08.404 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.0200000000000001e-07 2024-09-14T06:30:08.404 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:08.404 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:08.404 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:30:08.404 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065151+0000", 2024-09-14T06:30:08.405 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.1100000000000004e-07 2024-09-14T06:30:08.405 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:08.405 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:08.405 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:30:08.405 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065154+0000", 2024-09-14T06:30:08.405 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.9639999999999999e-06 2024-09-14T06:30:08.405 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:08.405 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:08.405 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:30:08.405 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065157+0000", 2024-09-14T06:30:08.405 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.4260000000000002e-06 2024-09-14T06:30:08.405 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:08.405 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:08.406 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:30:08.406 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.111861+0000", 2024-09-14T06:30:08.406 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046704268 2024-09-14T06:30:08.406 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:08.406 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:08.406 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:30:08.406 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.111939+0000", 2024-09-14T06:30:08.406 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.8231000000000004e-05 2024-09-14T06:30:08.406 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:08.406 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:08.406 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:30:08.406 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.112048+0000", 2024-09-14T06:30:08.406 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00010855500000000001 2024-09-14T06:30:08.407 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:08.407 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:30:08.407 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:08.407 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:08.407 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:08.407 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:19371 3.3 3:c34e5e51:::rbd_data.112d92027590.0000000000000024:f [read 1363968~1024 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:30:08.407 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:27.065161+0000", 2024-09-14T06:30:08.407 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 41.255672124999997, 2024-09-14T06:30:08.407 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047021944000000003, 2024-09-14T06:30:08.407 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:30:08.407 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:30:08.407 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:30:08.407 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:30:08.407 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:30:08.408 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:08.408 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 19371 2024-09-14T06:30:08.408 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:08.408 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:30:08.408 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:08.408 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:30:08.408 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065161+0000", 2024-09-14T06:30:08.408 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:08.408 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:08.408 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:08.408 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:30:08.408 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065161+0000", 2024-09-14T06:30:08.408 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:08.409 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:08.409 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:08.409 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:30:08.409 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065161+0000", 2024-09-14T06:30:08.409 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.0200000000000001e-07 2024-09-14T06:30:08.409 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:08.409 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:08.409 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:30:08.409 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065166+0000", 2024-09-14T06:30:08.409 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.9869999999999999e-06 2024-09-14T06:30:08.409 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:08.409 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:08.409 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:30:08.410 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065169+0000", 2024-09-14T06:30:08.410 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.3799999999999998e-06 2024-09-14T06:30:08.410 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:08.410 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:08.410 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:30:08.410 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065172+0000", 2024-09-14T06:30:08.410 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.4830000000000002e-06 2024-09-14T06:30:08.410 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:08.410 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:08.410 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:30:08.410 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.112071+0000", 2024-09-14T06:30:08.410 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046899529000000002 2024-09-14T06:30:08.410 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:08.411 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:08.411 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:30:08.411 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.112128+0000", 2024-09-14T06:30:08.411 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.6341999999999997e-05 2024-09-14T06:30:08.411 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:08.411 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:08.411 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:30:08.411 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.112183+0000", 2024-09-14T06:30:08.411 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.5021000000000001e-05 2024-09-14T06:30:08.411 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:08.411 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:30:08.411 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:08.411 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:08.412 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:08.412 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:19372 3.3 3:c34e5e51:::rbd_data.112d92027590.0000000000000024:f [read 1374208~1024 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:30:08.412 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:27.065176+0000", 2024-09-14T06:30:08.412 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 41.255656975000001, 2024-09-14T06:30:08.412 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047188889999999997, 2024-09-14T06:30:08.412 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:30:08.412 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:30:08.412 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:30:08.412 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:30:08.412 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:30:08.412 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:08.412 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 19372 2024-09-14T06:30:08.412 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:08.412 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:30:08.413 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:08.413 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:30:08.413 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065176+0000", 2024-09-14T06:30:08.413 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:08.413 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:08.413 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:08.413 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:30:08.413 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065176+0000", 2024-09-14T06:30:08.413 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:08.413 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:08.413 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:08.413 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:30:08.413 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.071117+0000", 2024-09-14T06:30:08.413 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.0059404990000000001 2024-09-14T06:30:08.414 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:08.414 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:08.414 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:30:08.414 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.071120+0000", 2024-09-14T06:30:08.414 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.4340000000000001e-06 2024-09-14T06:30:08.414 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:08.414 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:08.414 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:30:08.414 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.071129+0000", 2024-09-14T06:30:08.414 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.8389999999999996e-06 2024-09-14T06:30:08.414 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:08.414 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:08.414 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:30:08.414 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.071134+0000", 2024-09-14T06:30:08.415 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.412e-06 2024-09-14T06:30:08.415 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:08.415 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:08.415 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:30:08.415 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.112221+0000", 2024-09-14T06:30:08.415 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.041086473999999998 2024-09-14T06:30:08.415 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:08.415 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:08.415 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:30:08.415 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.112299+0000", 2024-09-14T06:30:08.415 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.8746999999999997e-05 2024-09-14T06:30:08.415 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:08.415 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:08.415 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:30:08.416 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.112365+0000", 2024-09-14T06:30:08.416 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.5485000000000001e-05 2024-09-14T06:30:08.416 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:08.416 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:30:08.416 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:08.416 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:08.416 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:30:08.416 INFO:teuthology.orchestra.run.smithi138.stdout:} 2024-09-14T06:30:08.418 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-14T06:30:08.420 INFO:teuthology.orchestra.run.smithi094.stdout:2578 punch from 0x36e659 to 0x370901, (0x22a8 bytes) 2024-09-14T06:30:08.420 INFO:teuthology.orchestra.run.smithi094.stdout:2579 write 0xbf04766 thru 0xbf0a564 (0x5dff bytes) 2024-09-14T06:30:08.420 INFO:teuthology.orchestra.run.smithi094.stdout:2580 read 0xa746217 thru 0xa7497cc (0x35b6 bytes) 2024-09-14T06:30:08.420 INFO:teuthology.orchestra.run.smithi094.stdout:2581 trunc from 0xbf0a565 to 0xd01306a 2024-09-14T06:30:08.420 INFO:teuthology.orchestra.run.smithi094.stdout:2582 read 0x5780f0f thru 0x578bc20 (0xad12 bytes) 2024-09-14T06:30:08.420 INFO:teuthology.orchestra.run.smithi094.stdout:2584 read 0x229ba8c thru 0x22a7473 (0xb9e8 bytes) 2024-09-14T06:30:08.420 INFO:teuthology.orchestra.run.smithi094.stdout:2586 read 0x62d1235 thru 0x62d614b (0x4f17 bytes) 2024-09-14T06:30:08.420 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-14T06:30:08.420 INFO:teuthology.orchestra.run.smithi094.stdout:2588 write 0xa1d3eca thru 0xa1e0bf6 (0xcd2d bytes) 2024-09-14T06:30:08.420 INFO:teuthology.orchestra.run.smithi094.stdout:2589 writesame 0x1b55c64 thru 0x1b650f9 data_size 0x1(0xf496 bytes) 2024-09-14T06:30:08.420 INFO:teuthology.orchestra.run.smithi094.stdout:2590 write 0xc1d9981 thru 0xc1e641a (0xca9a bytes) 2024-09-14T06:30:08.420 INFO:teuthology.orchestra.run.smithi094.stdout:2591 writesame 0xab3952d thru 0xab47cf4 data_size 0x4(0xe7c8 bytes) 2024-09-14T06:30:08.421 INFO:teuthology.orchestra.run.smithi094.stdout:2592 punch from 0x3652084 to 0x365c50b, (0xa487 bytes) 2024-09-14T06:30:08.421 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-14T06:30:08.421 INFO:teuthology.orchestra.run.smithi094.stdout:2594 punch from 0x1710cdc to 0x1710ffa, (0x31e bytes) 2024-09-14T06:30:08.421 INFO:teuthology.orchestra.run.smithi094.stdout:2598 write 0x78fbf32 thru 0x7903b32 (0x7c01 bytes) 2024-09-14T06:30:08.421 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-14T06:30:08.421 INFO:teuthology.orchestra.run.smithi094.stdout:2600 writesame 0x8499795 thru 0x84a9629 data_size 0x1(0xfe95 bytes) 2024-09-14T06:30:08.421 INFO:teuthology.orchestra.run.smithi094.stdout:2601 write 0xa8727c thru 0xa8d04b (0x5dd0 bytes) 2024-09-14T06:30:08.421 INFO:teuthology.orchestra.run.smithi094.stdout:2603 read 0x876323d thru 0x8772855 (0xf619 bytes) 2024-09-14T06:30:08.421 INFO:teuthology.orchestra.run.smithi094.stdout:2604 write 0x9bb63dc thru 0x9bb69c1 (0x5e6 bytes) 2024-09-14T06:30:08.421 INFO:tasks.ceph.osd.1.smithi094.stderr:2024-09-14T06:30:08.416+0000 7f328e47a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33641) UID: 0 2024-09-14T06:30:08.422 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-14T06:30:08.422 INFO:teuthology.orchestra.run.smithi094.stdout:2606 read 0x1dd46c0 thru 0x1dd4f43 (0x884 bytes) 2024-09-14T06:30:08.422 INFO:teuthology.orchestra.run.smithi094.stdout:2607 read 0xc290c4f thru 0xc293423 (0x27d5 bytes) 2024-09-14T06:30:08.422 INFO:teuthology.orchestra.run.smithi094.stdout:2608 writesame 0xab4596f thru 0xab546b4 data_size 0x16(0xed46 bytes) 2024-09-14T06:30:08.423 INFO:teuthology.orchestra.run.smithi094.stdout:2610 write 0xc712e70 thru 0xc71a007 (0x7198 bytes) 2024-09-14T06:30:08.429 INFO:teuthology.orchestra.run.smithi094.stdout:2611 punch from 0x4e6dd78 to 0x4e7193b, (0x3bc3 bytes) 2024-09-14T06:30:08.430 INFO:teuthology.orchestra.run.smithi094.stdout:2612 write 0x1177d55 thru 0x117ebee (0x6e9a bytes) 2024-09-14T06:30:08.431 INFO:teuthology.orchestra.run.smithi094.stdout:2613 write 0x8156fa5 thru 0x81591bb (0x2217 bytes) 2024-09-14T06:30:08.437 INFO:teuthology.orchestra.run.smithi094.stdout:2615 writesame 0x9b5b7d2 thru 0x9b65860 data_size 0x3(0xa08f bytes) 2024-09-14T06:30:08.445 INFO:teuthology.orchestra.run.smithi094.stdout:2616 write 0xbf6233a thru 0xbf6c628 (0xa2ef bytes) 2024-09-14T06:30:08.452 INFO:teuthology.orchestra.run.smithi094.stdout:2617 write 0x6e1eb82 thru 0x6e2d6c4 (0xeb43 bytes) 2024-09-14T06:30:08.522 INFO:teuthology.orchestra.run.smithi094.stdout:2619 read 0xbe09096 thru 0xbe09b42 (0xaad bytes) 2024-09-14T06:30:08.522 INFO:teuthology.orchestra.run.smithi094.stdout:2620 trunc from 0xd01306a to 0xad79de8 2024-09-14T06:30:08.523 INFO:teuthology.orchestra.run.smithi094.stdout:2621 read 0xa220555 thru 0xa226ad1 (0x657d bytes) 2024-09-14T06:30:08.523 INFO:teuthology.orchestra.run.smithi094.stdout:2622 write 0x3aec0e7 thru 0x3aeca68 (0x982 bytes) 2024-09-14T06:30:08.523 INFO:teuthology.orchestra.run.smithi094.stdout:2623 trunc from 0xad79de8 to 0x9cf7b95 2024-09-14T06:30:08.523 INFO:teuthology.orchestra.run.smithi094.stdout:2624 punch from 0x537087c to 0x537126e, (0x9f2 bytes) 2024-09-14T06:30:08.523 INFO:teuthology.orchestra.run.smithi094.stdout:2625 writesame 0xe45d6d7 thru 0xe462e5a data_size 0x6(0x5784 bytes) 2024-09-14T06:30:08.523 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-14T06:30:08.523 INFO:teuthology.orchestra.run.smithi094.stdout:2627 write 0xdffed73 thru 0xe00c48d (0xd71b bytes) 2024-09-14T06:30:08.523 INFO:teuthology.orchestra.run.smithi094.stdout:2628 read 0xd186b7e thru 0xd18bdce (0x5251 bytes) 2024-09-14T06:30:08.523 INFO:teuthology.orchestra.run.smithi094.stdout:2629 clone 18 order 18 su 131072 sc 2 2024-09-14T06:30:08.523 INFO:tasks.ceph.osd.1.smithi094.stderr:2024-09-14T06:30:08.520+0000 7f328e47a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33641) UID: 0 2024-09-14T06:30:08.622 INFO:tasks.ceph.osd.1.smithi094.stderr:2024-09-14T06:30:08.621+0000 7f328e47a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33641) UID: 0 2024-09-14T06:30:08.627 INFO:teuthology.orchestra.run.smithi138.stdout:{ 2024-09-14T06:30:08.627 INFO:teuthology.orchestra.run.smithi138.stdout: "ops": [], 2024-09-14T06:30:08.627 INFO:teuthology.orchestra.run.smithi138.stdout: "num_ops": 0 2024-09-14T06:30:08.627 INFO:teuthology.orchestra.run.smithi138.stdout:} 2024-09-14T06:30:08.636 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-14T06:30:08.723 INFO:tasks.ceph.osd.2.smithi094.stderr:2024-09-14T06:30:08.722+0000 7faf3269b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33649) UID: 0 2024-09-14T06:30:08.740 INFO:teuthology.orchestra.run.smithi138.stdout:{ 2024-09-14T06:30:08.740 INFO:teuthology.orchestra.run.smithi138.stdout: "ops": [], 2024-09-14T06:30:08.740 INFO:teuthology.orchestra.run.smithi138.stdout: "complaint_time": 30, 2024-09-14T06:30:08.740 INFO:teuthology.orchestra.run.smithi138.stdout: "num_blocked_ops": 0 2024-09-14T06:30:08.740 INFO:teuthology.orchestra.run.smithi138.stdout:} 2024-09-14T06:30:08.749 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-14T06:30:08.824 INFO:tasks.ceph.osd.4.smithi138.stderr:2024-09-14T06:30:08.823+0000 7f6c653c8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33615) UID: 0 2024-09-14T06:30:08.864 INFO:teuthology.orchestra.run.smithi138.stdout:{ 2024-09-14T06:30:08.864 INFO:teuthology.orchestra.run.smithi138.stdout: "size": 20, 2024-09-14T06:30:08.864 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 600, 2024-09-14T06:30:08.864 INFO:teuthology.orchestra.run.smithi138.stdout: "ops": [ 2024-09-14T06:30:08.865 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:08.865 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:11453 3.1d 3:bcad7303:::rbd_data.112d92027590.0000000000000001:f [read 94208~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:30:08.865 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:25.654375+0000", 2024-09-14T06:30:08.865 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 43.205771708999997, 2024-09-14T06:30:08.865 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052449318000000002, 2024-09-14T06:30:08.865 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:30:08.865 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:30:08.865 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:30:08.865 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:30:08.865 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:30:08.865 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:08.865 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 11453 2024-09-14T06:30:08.865 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:08.865 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:30:08.866 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:08.866 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:30:08.866 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654375+0000", 2024-09-14T06:30:08.866 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:08.866 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:08.866 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:08.866 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:30:08.866 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654375+0000", 2024-09-14T06:30:08.866 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:08.866 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:08.866 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:08.866 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:30:08.866 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654496+0000", 2024-09-14T06:30:08.867 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00012081 2024-09-14T06:30:08.867 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:08.867 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:08.867 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:30:08.867 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654497+0000", 2024-09-14T06:30:08.867 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.8789999999999999e-06 2024-09-14T06:30:08.867 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:08.867 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:08.867 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:30:08.867 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654501+0000", 2024-09-14T06:30:08.867 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.8999999999999999e-06 2024-09-14T06:30:08.868 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:08.868 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:08.868 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:30:08.868 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654505+0000", 2024-09-14T06:30:08.868 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.2229999999999999e-06 2024-09-14T06:30:08.868 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:08.868 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:08.868 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:30:08.868 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.705917+0000", 2024-09-14T06:30:08.868 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.051412501999999999 2024-09-14T06:30:08.869 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:08.869 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:08.869 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:30:08.869 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.705999+0000", 2024-09-14T06:30:08.869 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.1551999999999994e-05 2024-09-14T06:30:08.869 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:08.869 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:08.869 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:30:08.869 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.706824+0000", 2024-09-14T06:30:08.869 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00082545199999999996 2024-09-14T06:30:08.869 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:08.869 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:30:08.869 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:08.870 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:08.870 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:08.870 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:11454 3.1d 3:bcad7303:::rbd_data.112d92027590.0000000000000001:f [read 104448~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:30:08.870 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:25.654510+0000", 2024-09-14T06:30:08.870 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 43.205636919, 2024-09-14T06:30:08.870 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052610713000000003, 2024-09-14T06:30:08.870 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:30:08.870 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:30:08.870 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:30:08.870 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:30:08.870 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:30:08.870 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:08.870 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 11454 2024-09-14T06:30:08.871 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:08.871 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:30:08.871 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:08.871 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:30:08.871 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654510+0000", 2024-09-14T06:30:08.871 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:08.871 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:08.871 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:08.871 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:30:08.871 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654510+0000", 2024-09-14T06:30:08.871 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:08.871 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:08.871 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:08.871 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:30:08.871 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654825+0000", 2024-09-14T06:30:08.872 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00031522400000000002 2024-09-14T06:30:08.872 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:08.872 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:08.872 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:30:08.872 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654827+0000", 2024-09-14T06:30:08.872 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.9709999999999998e-06 2024-09-14T06:30:08.872 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:08.872 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:08.872 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:30:08.872 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654832+0000", 2024-09-14T06:30:08.872 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.4099999999999999e-06 2024-09-14T06:30:08.872 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:08.872 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:08.873 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:30:08.873 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654835+0000", 2024-09-14T06:30:08.873 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.14e-06 2024-09-14T06:30:08.873 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:08.873 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:08.873 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:30:08.873 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.706851+0000", 2024-09-14T06:30:08.873 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052015761000000001 2024-09-14T06:30:08.873 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:08.873 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:08.873 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:30:08.873 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.706934+0000", 2024-09-14T06:30:08.873 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.2484999999999994e-05 2024-09-14T06:30:08.873 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:08.873 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:08.874 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:30:08.874 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.707120+0000", 2024-09-14T06:30:08.874 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00018672200000000001 2024-09-14T06:30:08.874 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:08.874 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:30:08.874 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:08.874 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:08.874 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:08.874 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:11455 3.1d 3:bcad7303:::rbd_data.112d92027590.0000000000000001:f [read 114688~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:30:08.874 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:25.654841+0000", 2024-09-14T06:30:08.874 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 43.205305590000002, 2024-09-14T06:30:08.874 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052579898, 2024-09-14T06:30:08.874 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:30:08.874 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:30:08.875 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:30:08.875 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:30:08.875 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:30:08.875 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:08.875 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 11455 2024-09-14T06:30:08.875 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:08.875 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:30:08.875 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:08.875 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:30:08.875 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654841+0000", 2024-09-14T06:30:08.875 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:08.875 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:08.875 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:08.875 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:30:08.876 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654841+0000", 2024-09-14T06:30:08.876 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:08.876 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:08.876 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:08.876 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:30:08.876 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654841+0000", 2024-09-14T06:30:08.876 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.7300000000000002e-07 2024-09-14T06:30:08.876 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:08.876 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:08.876 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:30:08.876 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654845+0000", 2024-09-14T06:30:08.876 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.0520000000000003e-06 2024-09-14T06:30:08.876 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:08.876 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:08.877 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:30:08.877 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654849+0000", 2024-09-14T06:30:08.877 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.0459999999999999e-06 2024-09-14T06:30:08.877 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:08.877 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:08.877 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:30:08.877 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654852+0000", 2024-09-14T06:30:08.877 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.9830000000000001e-06 2024-09-14T06:30:08.877 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:08.877 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:08.877 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:30:08.877 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.707159+0000", 2024-09-14T06:30:08.877 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052306826000000001 2024-09-14T06:30:08.877 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:08.877 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:08.878 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:30:08.878 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.707242+0000", 2024-09-14T06:30:08.878 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.2595000000000005e-05 2024-09-14T06:30:08.878 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:08.878 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:08.878 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:30:08.878 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.707421+0000", 2024-09-14T06:30:08.878 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.000179123 2024-09-14T06:30:08.878 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:08.878 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:30:08.878 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:08.878 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:08.878 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:08.878 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:11456 3.1d 3:bcad7303:::rbd_data.112d92027590.0000000000000001:f [read 124928~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:30:08.879 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:25.654856+0000", 2024-09-14T06:30:08.879 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 43.205290744000003, 2024-09-14T06:30:08.879 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052801580000000001, 2024-09-14T06:30:08.879 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:30:08.879 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:30:08.879 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:30:08.879 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:30:08.879 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:30:08.879 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:08.879 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 11456 2024-09-14T06:30:08.879 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:08.879 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:30:08.879 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:08.880 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:30:08.880 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654856+0000", 2024-09-14T06:30:08.880 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:08.880 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:08.880 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:08.880 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:30:08.880 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654856+0000", 2024-09-14T06:30:08.880 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:08.880 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:08.880 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:08.880 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:30:08.880 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654856+0000", 2024-09-14T06:30:08.880 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.29e-07 2024-09-14T06:30:08.881 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:08.881 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:08.881 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:30:08.881 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654857+0000", 2024-09-14T06:30:08.881 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.187e-06 2024-09-14T06:30:08.881 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:08.881 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:08.881 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:30:08.881 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654861+0000", 2024-09-14T06:30:08.881 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.3850000000000002e-06 2024-09-14T06:30:08.881 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:08.881 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:08.881 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:30:08.881 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654863+0000", 2024-09-14T06:30:08.881 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.9500000000000001e-06 2024-09-14T06:30:08.882 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:08.882 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:08.882 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:30:08.882 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.707456+0000", 2024-09-14T06:30:08.882 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052592434 2024-09-14T06:30:08.882 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:08.882 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:08.882 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:30:08.882 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.707516+0000", 2024-09-14T06:30:08.882 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.0167999999999997e-05 2024-09-14T06:30:08.882 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:08.883 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:08.883 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:30:08.883 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.707657+0000", 2024-09-14T06:30:08.883 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00014122700000000001 2024-09-14T06:30:08.883 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:08.883 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:30:08.883 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:08.883 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:08.883 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:08.883 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:11457 3.1d 3:bcad7303:::rbd_data.112d92027590.0000000000000001:f [read 135168~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:30:08.883 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:25.654867+0000", 2024-09-14T06:30:08.883 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 43.205279859000001, 2024-09-14T06:30:08.884 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052921902, 2024-09-14T06:30:08.884 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:30:08.884 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:30:08.884 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:30:08.884 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:30:08.884 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:30:08.884 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:08.884 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 11457 2024-09-14T06:30:08.884 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:08.885 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:30:08.885 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:08.885 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:30:08.885 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654867+0000", 2024-09-14T06:30:08.885 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:08.885 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:08.885 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:08.885 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:30:08.885 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654867+0000", 2024-09-14T06:30:08.885 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:08.885 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:08.885 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:08.885 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:30:08.885 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654972+0000", 2024-09-14T06:30:08.886 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00010584 2024-09-14T06:30:08.886 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:08.886 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:08.886 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:30:08.886 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654975+0000", 2024-09-14T06:30:08.886 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.1169999999999998e-06 2024-09-14T06:30:08.886 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:08.886 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:08.886 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:30:08.886 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654979+0000", 2024-09-14T06:30:08.886 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.1960000000000001e-06 2024-09-14T06:30:08.886 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:08.886 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:08.886 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:30:08.887 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654982+0000", 2024-09-14T06:30:08.887 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.2990000000000001e-06 2024-09-14T06:30:08.887 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:08.887 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:08.887 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:30:08.887 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.707684+0000", 2024-09-14T06:30:08.887 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052701979000000003 2024-09-14T06:30:08.887 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:08.887 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:08.887 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:30:08.887 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.707728+0000", 2024-09-14T06:30:08.887 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.3529999999999998e-05 2024-09-14T06:30:08.887 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:08.888 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:08.888 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:30:08.888 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.707789+0000", 2024-09-14T06:30:08.888 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.0940999999999998e-05 2024-09-14T06:30:08.888 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:08.888 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:30:08.888 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:08.888 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:08.888 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:08.888 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:11458 3.1d 3:bcad7303:::rbd_data.112d92027590.0000000000000001:f [read 145408~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:30:08.889 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:25.654987+0000", 2024-09-14T06:30:08.889 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 43.205159461000001, 2024-09-14T06:30:08.889 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.05291655, 2024-09-14T06:30:08.889 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:30:08.889 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:30:08.889 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:30:08.889 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:30:08.889 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:30:08.889 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:08.890 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 11458 2024-09-14T06:30:08.890 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:08.890 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:30:08.890 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:08.890 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:30:08.890 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654987+0000", 2024-09-14T06:30:08.890 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:08.890 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:08.890 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:08.890 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:30:08.890 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654987+0000", 2024-09-14T06:30:08.890 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:08.891 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:08.891 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:08.891 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:30:08.891 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655077+0000", 2024-09-14T06:30:08.891 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.0253000000000005e-05 2024-09-14T06:30:08.891 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:08.891 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:08.891 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:30:08.891 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655079+0000", 2024-09-14T06:30:08.891 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.9989999999999998e-06 2024-09-14T06:30:08.891 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:08.892 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:08.892 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:30:08.892 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655083+0000", 2024-09-14T06:30:08.892 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.9280000000000003e-06 2024-09-14T06:30:08.892 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:08.892 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:08.892 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:30:08.892 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655086+0000", 2024-09-14T06:30:08.892 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.056e-06 2024-09-14T06:30:08.892 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:08.892 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:08.892 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:30:08.892 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.707805+0000", 2024-09-14T06:30:08.892 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052718627999999997 2024-09-14T06:30:08.893 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:08.893 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:08.893 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:30:08.893 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.707848+0000", 2024-09-14T06:30:08.893 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.2728999999999997e-05 2024-09-14T06:30:08.893 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:08.893 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:08.893 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:30:08.893 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.707904+0000", 2024-09-14T06:30:08.893 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.5957e-05 2024-09-14T06:30:08.893 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:08.893 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:30:08.893 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:08.894 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:08.894 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:08.894 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:11459 3.1d 3:bcad7303:::rbd_data.112d92027590.0000000000000001:f [read 155648~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:30:08.894 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:25.655091+0000", 2024-09-14T06:30:08.894 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 43.205055453, 2024-09-14T06:30:08.894 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052944242000000002, 2024-09-14T06:30:08.894 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:30:08.894 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:30:08.894 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:30:08.894 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:30:08.894 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:30:08.894 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:08.894 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 11459 2024-09-14T06:30:08.895 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:08.895 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:30:08.895 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:08.895 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:30:08.895 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655091+0000", 2024-09-14T06:30:08.895 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:08.895 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:08.895 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:08.895 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:30:08.895 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655091+0000", 2024-09-14T06:30:08.896 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:08.896 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:08.896 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:08.896 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:30:08.896 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655237+0000", 2024-09-14T06:30:08.896 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00014570100000000001 2024-09-14T06:30:08.896 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:08.896 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:08.896 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:30:08.896 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655239+0000", 2024-09-14T06:30:08.896 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.8670000000000001e-06 2024-09-14T06:30:08.896 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:08.897 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:08.897 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:30:08.897 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655243+0000", 2024-09-14T06:30:08.897 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.9829999999999998e-06 2024-09-14T06:30:08.897 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:08.897 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:08.897 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:30:08.897 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655246+0000", 2024-09-14T06:30:08.897 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.1540000000000002e-06 2024-09-14T06:30:08.897 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:08.897 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:08.897 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:30:08.897 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.707934+0000", 2024-09-14T06:30:08.897 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052688286000000001 2024-09-14T06:30:08.898 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:08.898 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:08.898 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:30:08.898 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.707977+0000", 2024-09-14T06:30:08.898 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.3291999999999998e-05 2024-09-14T06:30:08.898 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:08.898 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:08.898 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:30:08.898 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.708035+0000", 2024-09-14T06:30:08.898 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.7958999999999999e-05 2024-09-14T06:30:08.898 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:08.898 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:30:08.898 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:08.898 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:08.898 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:08.899 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:11460 3.1d 3:bcad7303:::rbd_data.112d92027590.0000000000000001:f [read 165888~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:30:08.899 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:25.655250+0000", 2024-09-14T06:30:08.899 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 43.204896155999997, 2024-09-14T06:30:08.899 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052910698999999999, 2024-09-14T06:30:08.899 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:30:08.899 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:30:08.899 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:30:08.899 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:30:08.899 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:30:08.899 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:08.899 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 11460 2024-09-14T06:30:08.899 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:08.899 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:30:08.899 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:08.899 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:30:08.900 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655250+0000", 2024-09-14T06:30:08.900 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:08.900 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:08.900 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:08.900 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:30:08.900 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655250+0000", 2024-09-14T06:30:08.900 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:08.900 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:08.900 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:08.900 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:30:08.900 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655356+0000", 2024-09-14T06:30:08.900 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00010535900000000001 2024-09-14T06:30:08.900 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:08.901 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:08.901 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:30:08.901 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655357+0000", 2024-09-14T06:30:08.901 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.8080000000000001e-06 2024-09-14T06:30:08.901 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:08.901 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:08.901 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:30:08.901 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655361+0000", 2024-09-14T06:30:08.901 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.861e-06 2024-09-14T06:30:08.901 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:08.901 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:08.901 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:30:08.901 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655364+0000", 2024-09-14T06:30:08.901 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.1089999999999998e-06 2024-09-14T06:30:08.902 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:08.902 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:08.902 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:30:08.902 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.708065+0000", 2024-09-14T06:30:08.902 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052700595000000003 2024-09-14T06:30:08.902 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:08.902 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:08.902 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:30:08.902 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.708108+0000", 2024-09-14T06:30:08.902 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.2484999999999997e-05 2024-09-14T06:30:08.902 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:08.902 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:08.902 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:30:08.902 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.708161+0000", 2024-09-14T06:30:08.902 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.3482000000000001e-05 2024-09-14T06:30:08.903 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:08.903 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:30:08.903 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:08.903 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:08.903 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:08.903 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:11461 3.1d 3:bcad7303:::rbd_data.112d92027590.0000000000000001:f [read 176128~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:30:08.903 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:25.655370+0000", 2024-09-14T06:30:08.903 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 43.204776873999997, 2024-09-14T06:30:08.903 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052915141999999998, 2024-09-14T06:30:08.903 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:30:08.903 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:30:08.903 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:30:08.903 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:30:08.903 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:30:08.904 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:08.904 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 11461 2024-09-14T06:30:08.904 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:08.904 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:30:08.904 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:08.904 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:30:08.904 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655370+0000", 2024-09-14T06:30:08.904 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:08.904 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:08.904 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:08.904 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:30:08.904 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655370+0000", 2024-09-14T06:30:08.904 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:08.904 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:08.905 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:08.905 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:30:08.905 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655394+0000", 2024-09-14T06:30:08.905 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.4553999999999999e-05 2024-09-14T06:30:08.905 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:08.905 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:08.905 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:30:08.905 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655396+0000", 2024-09-14T06:30:08.905 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.8950000000000001e-06 2024-09-14T06:30:08.905 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:08.905 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:08.905 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:30:08.905 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655401+0000", 2024-09-14T06:30:08.905 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.0089999999999998e-06 2024-09-14T06:30:08.906 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:08.906 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:08.906 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:30:08.906 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655404+0000", 2024-09-14T06:30:08.906 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.3400000000000002e-06 2024-09-14T06:30:08.906 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:08.906 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:08.906 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:30:08.906 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.708186+0000", 2024-09-14T06:30:08.906 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052781726000000001 2024-09-14T06:30:08.906 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:08.907 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:08.907 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:30:08.907 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.708229+0000", 2024-09-14T06:30:08.907 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.2670999999999999e-05 2024-09-14T06:30:08.907 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:08.907 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:08.907 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:30:08.907 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.708285+0000", 2024-09-14T06:30:08.907 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.5946999999999998e-05 2024-09-14T06:30:08.907 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:08.907 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:30:08.907 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:08.907 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:08.907 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:08.908 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:11462 3.1d 3:bcad7303:::rbd_data.112d92027590.0000000000000001:f [read 186368~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:30:08.908 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:25.655410+0000", 2024-09-14T06:30:08.908 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 43.204736396000001, 2024-09-14T06:30:08.908 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052995258000000003, 2024-09-14T06:30:08.908 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:30:08.908 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:30:08.908 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:30:08.908 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:30:08.908 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:30:08.908 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:08.908 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 11462 2024-09-14T06:30:08.908 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:08.908 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:30:08.908 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:08.909 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:30:08.909 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655410+0000", 2024-09-14T06:30:08.909 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:08.909 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:08.909 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:08.909 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:30:08.909 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655410+0000", 2024-09-14T06:30:08.909 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:08.909 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:08.909 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:08.909 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:30:08.909 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655625+0000", 2024-09-14T06:30:08.909 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00021442600000000001 2024-09-14T06:30:08.910 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:08.910 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:08.910 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:30:08.910 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655626+0000", 2024-09-14T06:30:08.910 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.815e-06 2024-09-14T06:30:08.910 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:08.910 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:08.910 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:30:08.910 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655631+0000", 2024-09-14T06:30:08.910 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.651e-06 2024-09-14T06:30:08.910 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:08.910 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:08.910 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:30:08.910 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655634+0000", 2024-09-14T06:30:08.911 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.2449999999999998e-06 2024-09-14T06:30:08.911 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:08.911 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:08.911 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:30:08.911 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.708312+0000", 2024-09-14T06:30:08.911 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052677412 2024-09-14T06:30:08.911 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:08.911 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:08.911 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:30:08.911 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.708352+0000", 2024-09-14T06:30:08.911 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.0729999999999998e-05 2024-09-14T06:30:08.911 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:08.911 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:08.911 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:30:08.912 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.708405+0000", 2024-09-14T06:30:08.912 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.2979000000000002e-05 2024-09-14T06:30:08.912 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:08.912 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:30:08.912 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:08.912 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:08.912 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:08.912 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:11463 3.1d 3:bcad7303:::rbd_data.112d92027590.0000000000000001:f [read 196608~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:30:08.912 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:25.655639+0000", 2024-09-14T06:30:08.912 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 43.204507249999999, 2024-09-14T06:30:08.912 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052888859000000003, 2024-09-14T06:30:08.913 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:30:08.913 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:30:08.913 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:30:08.913 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:30:08.913 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:30:08.913 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:08.913 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 11463 2024-09-14T06:30:08.913 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:08.914 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:30:08.914 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:08.914 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:30:08.914 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655639+0000", 2024-09-14T06:30:08.914 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:08.914 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:08.914 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:08.914 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:30:08.914 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655639+0000", 2024-09-14T06:30:08.914 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:08.914 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:08.914 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:08.914 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:30:08.914 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655640+0000", 2024-09-14T06:30:08.914 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.08e-07 2024-09-14T06:30:08.915 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:08.915 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:08.915 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:30:08.915 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655641+0000", 2024-09-14T06:30:08.915 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.156e-06 2024-09-14T06:30:08.915 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:08.915 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:08.915 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:30:08.915 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655648+0000", 2024-09-14T06:30:08.915 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.8460000000000002e-06 2024-09-14T06:30:08.915 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:08.915 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:08.915 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:30:08.915 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655651+0000", 2024-09-14T06:30:08.915 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.0079999999999998e-06 2024-09-14T06:30:08.916 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:08.916 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:08.916 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:30:08.916 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.708429+0000", 2024-09-14T06:30:08.916 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052778459 2024-09-14T06:30:08.916 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:08.916 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:08.916 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:30:08.916 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.708473+0000", 2024-09-14T06:30:08.916 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.3782999999999997e-05 2024-09-14T06:30:08.916 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:08.916 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:08.916 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:30:08.916 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.708528+0000", 2024-09-14T06:30:08.917 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.5198999999999998e-05 2024-09-14T06:30:08.917 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:08.917 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:30:08.917 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:08.917 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:08.917 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:08.917 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:11464 3.1d 3:bcad7303:::rbd_data.112d92027590.0000000000000001:f [read 206848~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:30:08.917 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:25.655654+0000", 2024-09-14T06:30:08.917 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 43.204492381000001, 2024-09-14T06:30:08.917 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052981845999999999, 2024-09-14T06:30:08.917 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:30:08.917 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:30:08.917 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:30:08.917 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:30:08.918 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:30:08.918 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:08.918 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 11464 2024-09-14T06:30:08.918 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:08.918 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:30:08.918 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:08.918 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:30:08.918 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655654+0000", 2024-09-14T06:30:08.918 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:08.918 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:08.918 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:08.918 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:30:08.918 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655654+0000", 2024-09-14T06:30:08.918 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:08.918 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:08.919 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:08.919 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:30:08.919 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655946+0000", 2024-09-14T06:30:08.919 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00029148200000000003 2024-09-14T06:30:08.919 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:08.919 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:08.919 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:30:08.919 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655948+0000", 2024-09-14T06:30:08.919 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.973e-06 2024-09-14T06:30:08.919 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:08.919 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:08.919 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:30:08.919 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655953+0000", 2024-09-14T06:30:08.919 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.1240000000000004e-06 2024-09-14T06:30:08.919 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:08.920 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:08.920 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:30:08.920 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655956+0000", 2024-09-14T06:30:08.920 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.315e-06 2024-09-14T06:30:08.920 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:08.920 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:08.920 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:30:08.920 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.708543+0000", 2024-09-14T06:30:08.920 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052587416999999997 2024-09-14T06:30:08.920 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:08.920 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:08.920 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:30:08.920 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.708584+0000", 2024-09-14T06:30:08.920 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.1004000000000002e-05 2024-09-14T06:30:08.921 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:08.921 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:08.921 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:30:08.921 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.708636+0000", 2024-09-14T06:30:08.921 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.1530999999999998e-05 2024-09-14T06:30:08.921 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:08.921 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:30:08.921 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:08.921 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:08.921 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:08.921 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:11465 3.1d 3:bcad7303:::rbd_data.112d92027590.0000000000000001:f [read 217088~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:30:08.921 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:25.655961+0000", 2024-09-14T06:30:08.921 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 43.204185228999997, 2024-09-14T06:30:08.921 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052798682999999999, 2024-09-14T06:30:08.922 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:30:08.922 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:30:08.922 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:30:08.922 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:30:08.922 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:30:08.922 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:08.922 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 11465 2024-09-14T06:30:08.922 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:08.922 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:30:08.922 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:08.922 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:30:08.922 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655961+0000", 2024-09-14T06:30:08.922 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:08.922 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:08.922 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:08.923 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:30:08.923 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655961+0000", 2024-09-14T06:30:08.923 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:08.923 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:08.923 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:08.923 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:30:08.923 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.656114+0000", 2024-09-14T06:30:08.923 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.000152745 2024-09-14T06:30:08.923 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:08.923 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:08.923 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:30:08.923 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.656116+0000", 2024-09-14T06:30:08.923 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.804e-06 2024-09-14T06:30:08.923 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:08.923 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:08.924 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:30:08.924 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.656120+0000", 2024-09-14T06:30:08.924 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.4700000000000004e-06 2024-09-14T06:30:08.924 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:08.924 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:08.924 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:30:08.924 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.656123+0000", 2024-09-14T06:30:08.924 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.1269999999999999e-06 2024-09-14T06:30:08.924 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:08.924 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:08.924 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:30:08.924 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.708660+0000", 2024-09-14T06:30:08.924 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052536919000000001 2024-09-14T06:30:08.924 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:08.924 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:08.925 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:30:08.925 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.708703+0000", 2024-09-14T06:30:08.925 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.3077000000000001e-05 2024-09-14T06:30:08.925 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:08.925 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:08.925 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:30:08.925 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.708760+0000", 2024-09-14T06:30:08.925 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.6541e-05 2024-09-14T06:30:08.925 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:08.925 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:30:08.925 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:08.925 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:08.925 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:08.925 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:11466 3.1d 3:bcad7303:::rbd_data.112d92027590.0000000000000001:f [read 227328~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:30:08.925 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:25.656129+0000", 2024-09-14T06:30:08.926 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 43.204017841000002, 2024-09-14T06:30:08.926 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052750987999999999, 2024-09-14T06:30:08.926 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:30:08.926 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:30:08.926 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:30:08.926 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:30:08.926 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:30:08.926 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:08.926 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 11466 2024-09-14T06:30:08.926 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:08.926 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:30:08.926 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:08.926 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:30:08.926 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.656129+0000", 2024-09-14T06:30:08.926 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:08.927 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:08.927 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:08.927 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:30:08.927 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.656129+0000", 2024-09-14T06:30:08.927 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:08.927 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:08.927 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:08.927 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:30:08.927 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.656347+0000", 2024-09-14T06:30:08.927 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.000218054 2024-09-14T06:30:08.927 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:08.927 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:08.927 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:30:08.927 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.656349+0000", 2024-09-14T06:30:08.928 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.9199999999999998e-06 2024-09-14T06:30:08.928 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:08.928 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:08.928 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:30:08.928 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.656353+0000", 2024-09-14T06:30:08.928 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.9210000000000002e-06 2024-09-14T06:30:08.928 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:08.928 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:08.928 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:30:08.928 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.656356+0000", 2024-09-14T06:30:08.928 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.7670000000000001e-06 2024-09-14T06:30:08.928 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:08.928 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:08.928 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:30:08.928 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.708787+0000", 2024-09-14T06:30:08.929 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052430840999999999 2024-09-14T06:30:08.929 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:08.929 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:08.929 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:30:08.929 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.708827+0000", 2024-09-14T06:30:08.929 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.9538e-05 2024-09-14T06:30:08.929 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:08.929 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:08.929 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:30:08.929 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.708880+0000", 2024-09-14T06:30:08.929 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.2947e-05 2024-09-14T06:30:08.929 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:08.929 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:30:08.929 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:08.929 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:08.930 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:08.930 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:11467 3.1d 3:bcad7303:::rbd_data.112d92027590.0000000000000001:f [read 237568~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:30:08.930 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:25.656361+0000", 2024-09-14T06:30:08.930 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 43.203785193000002, 2024-09-14T06:30:08.930 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052643898000000001, 2024-09-14T06:30:08.930 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:30:08.930 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:30:08.930 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:30:08.930 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:30:08.930 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:30:08.930 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:08.930 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 11467 2024-09-14T06:30:08.930 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:08.930 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:30:08.931 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:08.931 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:30:08.931 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.656361+0000", 2024-09-14T06:30:08.931 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:08.931 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:08.931 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:08.931 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:30:08.931 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.656361+0000", 2024-09-14T06:30:08.931 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:08.931 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:08.931 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:08.931 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:30:08.931 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.656855+0000", 2024-09-14T06:30:08.931 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00049372000000000001 2024-09-14T06:30:08.932 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:08.932 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:08.932 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:30:08.932 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.656859+0000", 2024-09-14T06:30:08.932 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.1300000000000003e-06 2024-09-14T06:30:08.932 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:08.932 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:08.932 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:30:08.932 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.656863+0000", 2024-09-14T06:30:08.932 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.2629999999999997e-06 2024-09-14T06:30:08.932 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:08.932 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:08.932 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:30:08.932 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.656867+0000", 2024-09-14T06:30:08.932 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.3220000000000001e-06 2024-09-14T06:30:08.933 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:08.933 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:08.933 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:30:08.933 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.708903+0000", 2024-09-14T06:30:08.933 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052035855999999998 2024-09-14T06:30:08.933 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:08.933 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:08.933 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:30:08.933 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.708949+0000", 2024-09-14T06:30:08.933 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.6276999999999998e-05 2024-09-14T06:30:08.933 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:08.933 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:08.933 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:30:08.933 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.709005+0000", 2024-09-14T06:30:08.934 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.6329999999999998e-05 2024-09-14T06:30:08.934 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:08.934 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:30:08.934 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:08.934 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:08.934 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:08.934 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:11479 3.1d 3:bcad7303:::rbd_data.112d92027590.0000000000000001:f [read 360448~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:30:08.934 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:25.658388+0000", 2024-09-14T06:30:08.934 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 43.201758359999999, 2024-09-14T06:30:08.934 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052408920999999997, 2024-09-14T06:30:08.934 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:30:08.934 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:30:08.934 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:30:08.934 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:30:08.934 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:30:08.935 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:08.935 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 11479 2024-09-14T06:30:08.935 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:08.935 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:30:08.935 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:08.935 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:30:08.935 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.658388+0000", 2024-09-14T06:30:08.935 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:08.935 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:08.935 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:08.935 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:30:08.935 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.658388+0000", 2024-09-14T06:30:08.935 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:08.936 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:08.936 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:08.936 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:30:08.936 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.658534+0000", 2024-09-14T06:30:08.936 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.000145613 2024-09-14T06:30:08.936 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:08.936 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:08.936 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:30:08.936 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.658540+0000", 2024-09-14T06:30:08.936 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.6680000000000004e-06 2024-09-14T06:30:08.936 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:08.936 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:08.936 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:30:08.936 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.658548+0000", 2024-09-14T06:30:08.937 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.7980000000000006e-06 2024-09-14T06:30:08.937 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:08.937 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:08.937 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:30:08.937 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.658554+0000", 2024-09-14T06:30:08.937 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.6869999999999999e-06 2024-09-14T06:30:08.937 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:08.937 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:08.937 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:30:08.937 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.710624+0000", 2024-09-14T06:30:08.937 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052070567999999998 2024-09-14T06:30:08.937 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:08.937 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:08.937 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:30:08.938 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.710666+0000", 2024-09-14T06:30:08.938 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.1467000000000001e-05 2024-09-14T06:30:08.938 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:08.938 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:08.938 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:30:08.938 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.710797+0000", 2024-09-14T06:30:08.938 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00013112 2024-09-14T06:30:08.938 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:08.938 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:30:08.938 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:08.938 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:08.938 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:08.938 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:11480 3.1d 3:bcad7303:::rbd_data.112d92027590.0000000000000001:f [read 370688~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:30:08.938 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:25.658563+0000", 2024-09-14T06:30:08.938 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 43.201583429000003, 2024-09-14T06:30:08.939 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052429164, 2024-09-14T06:30:08.939 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:30:08.939 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:30:08.939 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:30:08.939 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:30:08.939 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:30:08.939 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:08.939 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 11480 2024-09-14T06:30:08.939 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:08.939 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:30:08.939 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:08.939 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:30:08.939 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.658563+0000", 2024-09-14T06:30:08.939 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:08.940 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:08.940 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:08.940 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:30:08.940 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.658563+0000", 2024-09-14T06:30:08.940 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:08.940 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:08.940 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:08.940 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:30:08.940 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.658658+0000", 2024-09-14T06:30:08.940 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.5346999999999994e-05 2024-09-14T06:30:08.940 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:08.940 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:08.940 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:30:08.940 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.658662+0000", 2024-09-14T06:30:08.941 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.14e-06 2024-09-14T06:30:08.941 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:08.941 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:08.941 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:30:08.941 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.658669+0000", 2024-09-14T06:30:08.941 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.7379999999999994e-06 2024-09-14T06:30:08.941 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:08.941 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:08.941 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:30:08.941 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.658675+0000", 2024-09-14T06:30:08.941 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.5430000000000001e-06 2024-09-14T06:30:08.941 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:08.941 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:08.941 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:30:08.942 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.710824+0000", 2024-09-14T06:30:08.942 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052148847999999998 2024-09-14T06:30:08.942 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:08.942 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:08.942 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:30:08.942 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.710864+0000", 2024-09-14T06:30:08.942 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.9925999999999997e-05 2024-09-14T06:30:08.942 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:08.942 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:08.942 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:30:08.942 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.710992+0000", 2024-09-14T06:30:08.942 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.000128622 2024-09-14T06:30:08.942 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:08.942 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:30:08.942 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:08.943 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:08.943 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:08.943 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:11481 3.1d 3:bcad7303:::rbd_data.112d92027590.0000000000000001:f [read 380928~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:30:08.943 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:25.658684+0000", 2024-09-14T06:30:08.943 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 43.201462358999997, 2024-09-14T06:30:08.943 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052508565, 2024-09-14T06:30:08.943 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:30:08.943 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:30:08.943 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:30:08.943 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:30:08.943 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:30:08.943 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:08.943 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 11481 2024-09-14T06:30:08.943 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:08.943 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:30:08.944 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:08.944 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:30:08.944 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.658684+0000", 2024-09-14T06:30:08.944 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:08.944 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:08.944 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:08.944 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:30:08.944 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.658684+0000", 2024-09-14T06:30:08.944 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:08.944 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:08.944 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:08.944 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:30:08.944 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.658715+0000", 2024-09-14T06:30:08.944 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.0703000000000001e-05 2024-09-14T06:30:08.945 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:08.945 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:08.945 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:30:08.945 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.658718+0000", 2024-09-14T06:30:08.945 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.0709999999999999e-06 2024-09-14T06:30:08.945 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:08.945 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:08.945 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:30:08.945 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.658725+0000", 2024-09-14T06:30:08.945 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.2339999999999997e-06 2024-09-14T06:30:08.945 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:08.945 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:08.945 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:30:08.945 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.658731+0000", 2024-09-14T06:30:08.945 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.6450000000000002e-06 2024-09-14T06:30:08.946 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:08.946 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:08.946 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:30:08.946 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.711020+0000", 2024-09-14T06:30:08.946 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052288886 2024-09-14T06:30:08.946 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:08.946 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:08.946 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:30:08.946 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.711061+0000", 2024-09-14T06:30:08.946 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.1084000000000002e-05 2024-09-14T06:30:08.946 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:08.946 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:08.946 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:30:08.946 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.711193+0000", 2024-09-14T06:30:08.946 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00013194199999999999 2024-09-14T06:30:08.946 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:08.947 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:30:08.947 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:08.947 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:08.947 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:08.947 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:11482 3.1d 3:bcad7303:::rbd_data.112d92027590.0000000000000001:f [read 391168~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:30:08.947 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:25.658740+0000", 2024-09-14T06:30:08.947 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 43.201406894000002, 2024-09-14T06:30:08.947 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052637015000000002, 2024-09-14T06:30:08.947 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:30:08.947 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:30:08.947 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:30:08.947 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:30:08.947 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:30:08.947 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:08.947 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 11482 2024-09-14T06:30:08.948 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:08.948 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:30:08.948 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:08.948 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:30:08.948 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.658740+0000", 2024-09-14T06:30:08.948 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:08.948 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:08.948 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:08.948 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:30:08.948 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.658740+0000", 2024-09-14T06:30:08.948 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:08.948 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:08.948 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:08.948 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:30:08.948 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.658880+0000", 2024-09-14T06:30:08.948 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00014036100000000001 2024-09-14T06:30:08.949 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:08.949 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:08.949 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:30:08.949 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.658883+0000", 2024-09-14T06:30:08.949 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.3969999999999998e-06 2024-09-14T06:30:08.949 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:08.949 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:08.949 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:30:08.949 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.658890+0000", 2024-09-14T06:30:08.949 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.1500000000000002e-06 2024-09-14T06:30:08.949 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:08.949 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:08.949 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:30:08.949 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.658897+0000", 2024-09-14T06:30:08.950 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.1260000000000003e-06 2024-09-14T06:30:08.950 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:08.950 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:08.950 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:30:08.950 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.711207+0000", 2024-09-14T06:30:08.950 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052310777000000003 2024-09-14T06:30:08.950 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:08.950 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:08.950 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:30:08.950 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.711249+0000", 2024-09-14T06:30:08.950 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.1786999999999998e-05 2024-09-14T06:30:08.950 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:08.950 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:08.950 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:30:08.951 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.711377+0000", 2024-09-14T06:30:08.951 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.000127417 2024-09-14T06:30:08.951 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:08.951 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:30:08.951 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:08.951 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:08.951 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:08.951 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:11483 3.1d 3:bcad7303:::rbd_data.112d92027590.0000000000000001:f [read 401408~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:30:08.951 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:25.658906+0000", 2024-09-14T06:30:08.951 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 43.201240110000001, 2024-09-14T06:30:08.951 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052595109000000001, 2024-09-14T06:30:08.951 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:30:08.951 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:30:08.951 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:30:08.951 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:30:08.952 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:30:08.952 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:08.952 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 11483 2024-09-14T06:30:08.952 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:08.952 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:30:08.952 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:08.952 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:30:08.952 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.658906+0000", 2024-09-14T06:30:08.952 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:08.952 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:08.952 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:08.952 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:30:08.952 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.658906+0000", 2024-09-14T06:30:08.952 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:08.952 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:08.952 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:08.953 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:30:08.953 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.664985+0000", 2024-09-14T06:30:08.953 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.0060787250000000001 2024-09-14T06:30:08.953 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:08.953 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:08.953 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:30:08.953 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.664991+0000", 2024-09-14T06:30:08.953 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.7030000000000003e-06 2024-09-14T06:30:08.953 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:08.953 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:08.953 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:30:08.953 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.664999+0000", 2024-09-14T06:30:08.953 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.4430000000000003e-06 2024-09-14T06:30:08.953 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:08.953 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:08.954 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:30:08.954 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.665006+0000", 2024-09-14T06:30:08.954 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.0840000000000003e-06 2024-09-14T06:30:08.954 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:08.954 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:08.954 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:30:08.954 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.711402+0000", 2024-09-14T06:30:08.954 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046395714999999997 2024-09-14T06:30:08.954 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:08.954 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:08.954 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:30:08.954 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.711443+0000", 2024-09-14T06:30:08.954 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.0911999999999997e-05 2024-09-14T06:30:08.954 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:08.954 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:08.954 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:30:08.955 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.711501+0000", 2024-09-14T06:30:08.955 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.8526999999999997e-05 2024-09-14T06:30:08.955 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:08.955 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:30:08.955 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:08.955 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:08.955 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:30:08.955 INFO:teuthology.orchestra.run.smithi138.stdout:} 2024-09-14T06:30:08.955 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-14T06:30:08.957 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-14T06:30:08.958 INFO:tasks.ceph.osd.4.smithi138.stderr:2024-09-14T06:30:08.955+0000 7f6c653c8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33615) UID: 0 2024-09-14T06:30:09.011 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-14T06:30:09.057 INFO:tasks.ceph.osd.2.smithi094.stderr:2024-09-14T06:30:09.056+0000 7faf3269b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33649) UID: 0 2024-09-14T06:30:09.102 INFO:teuthology.orchestra.run.smithi138.stdout:{ 2024-09-14T06:30:09.102 INFO:teuthology.orchestra.run.smithi138.stdout: "ops": [], 2024-09-14T06:30:09.102 INFO:teuthology.orchestra.run.smithi138.stdout: "num_ops": 0 2024-09-14T06:30:09.103 INFO:teuthology.orchestra.run.smithi138.stdout:} 2024-09-14T06:30:09.111 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-14T06:30:09.133 INFO:teuthology.orchestra.run.smithi094.stdout:osd.0: {} 2024-09-14T06:30:09.133 INFO:teuthology.orchestra.run.smithi094.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-14T06:30:09.150 INFO:teuthology.orchestra.run.smithi094.stdout:osd.1: {} 2024-09-14T06:30:09.150 INFO:teuthology.orchestra.run.smithi094.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-14T06:30:09.158 INFO:tasks.ceph.osd.0.smithi094.stderr:2024-09-14T06:30:09.157+0000 7f07c9333640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33636) UID: 0 2024-09-14T06:30:09.166 INFO:teuthology.orchestra.run.smithi094.stdout:osd.2: {} 2024-09-14T06:30:09.167 INFO:teuthology.orchestra.run.smithi094.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-14T06:30:09.192 INFO:teuthology.orchestra.run.smithi094.stdout:osd.3: {} 2024-09-14T06:30:09.193 INFO:teuthology.orchestra.run.smithi094.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-14T06:30:09.210 INFO:teuthology.orchestra.run.smithi094.stdout:osd.4: {} 2024-09-14T06:30:09.210 INFO:teuthology.orchestra.run.smithi094.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-14T06:30:09.216 INFO:teuthology.orchestra.run.smithi138.stdout:{ 2024-09-14T06:30:09.217 INFO:teuthology.orchestra.run.smithi138.stdout: "ops": [], 2024-09-14T06:30:09.217 INFO:teuthology.orchestra.run.smithi138.stdout: "complaint_time": 30, 2024-09-14T06:30:09.217 INFO:teuthology.orchestra.run.smithi138.stdout: "num_blocked_ops": 0 2024-09-14T06:30:09.217 INFO:teuthology.orchestra.run.smithi138.stdout:} 2024-09-14T06:30:09.225 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-14T06:30:09.226 INFO:teuthology.orchestra.run.smithi094.stdout:osd.5: {} 2024-09-14T06:30:09.227 INFO:teuthology.orchestra.run.smithi094.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-14T06:30:09.244 INFO:teuthology.orchestra.run.smithi094.stdout:osd.6: {} 2024-09-14T06:30:09.244 INFO:teuthology.orchestra.run.smithi094.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-14T06:30:09.260 INFO:teuthology.orchestra.run.smithi094.stdout:osd.7: {} 2024-09-14T06:30:09.260 INFO:teuthology.orchestra.run.smithi094.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-14T06:30:09.269 INFO:tasks.ceph.osd.4.smithi138.stderr:2024-09-14T06:30:09.268+0000 7f6c653c8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33615) UID: 0 2024-09-14T06:30:09.330 INFO:teuthology.orchestra.run.smithi138.stdout:ERROR: (22) Invalid argument 2024-09-14T06:30:09.330 INFO:teuthology.orchestra.run.smithi138.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:30:09.339 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-14T06:30:09.360 INFO:tasks.ceph.osd.2.smithi094.stderr:2024-09-14T06:30:09.359+0000 7faf3269b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33649) UID: 0 2024-09-14T06:30:09.461 INFO:tasks.ceph.osd.6.smithi138.stderr:2024-09-14T06:30:09.460+0000 7f8cd8849640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33620) UID: 0 2024-09-14T06:30:09.484 INFO:teuthology.orchestra.run.smithi138.stdout:ERROR: (22) Invalid argument 2024-09-14T06:30:09.485 INFO:teuthology.orchestra.run.smithi138.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:30:09.494 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-14T06:30:09.561 INFO:tasks.ceph.osd.1.smithi094.stderr:2024-09-14T06:30:09.561+0000 7f328e47a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33641) UID: 0 2024-09-14T06:30:09.639 INFO:teuthology.orchestra.run.smithi138.stdout:ERROR: (22) Invalid argument 2024-09-14T06:30:09.639 INFO:teuthology.orchestra.run.smithi138.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:30:09.648 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-14T06:30:09.663 INFO:tasks.ceph.osd.0.smithi094.stderr:2024-09-14T06:30:09.662+0000 7f07c9333640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33636) UID: 0 2024-09-14T06:30:09.750 INFO:teuthology.orchestra.run.smithi138.stdout:ERROR: (22) Invalid argument 2024-09-14T06:30:09.750 INFO:teuthology.orchestra.run.smithi138.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:30:09.758 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-14T06:30:09.802 INFO:tasks.ceph.osd.2.smithi094.stderr:2024-09-14T06:30:09.801+0000 7faf3269b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33649) UID: 0 2024-09-14T06:30:09.848 INFO:teuthology.orchestra.run.smithi094.stderr:nodeep-scrub is unset 2024-09-14T06:30:09.864 INFO:tasks.ceph.osd.6.smithi138.stderr:2024-09-14T06:30:09.862+0000 7f8cd8849640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33620) UID: 0 2024-09-14T06:30:09.874 INFO:teuthology.orchestra.run.smithi094.stdout:ERROR: (22) Invalid argument 2024-09-14T06:30:09.875 INFO:teuthology.orchestra.run.smithi094.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:30:09.891 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-14T06:30:09.965 INFO:tasks.ceph.osd.6.smithi138.stderr:2024-09-14T06:30:09.964+0000 7f8cd8849640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33620) UID: 0 2024-09-14T06:30:10.066 INFO:tasks.ceph.osd.3.smithi094.stderr:2024-09-14T06:30:10.065+0000 7feeeafe8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33655) UID: 0 2024-09-14T06:30:10.105 INFO:teuthology.orchestra.run.smithi094.stdout:ERROR: (22) Invalid argument 2024-09-14T06:30:10.106 INFO:teuthology.orchestra.run.smithi094.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:30:10.119 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-14T06:30:10.203 INFO:tasks.ceph.osd.3.smithi094.stderr:2024-09-14T06:30:10.202+0000 7feeeafe8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33655) UID: 0 2024-09-14T06:30:10.267 INFO:tasks.ceph.osd.7.smithi138.stderr:2024-09-14T06:30:10.266+0000 7f3c6a21d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33623) UID: 0 2024-09-14T06:30:10.328 INFO:teuthology.orchestra.run.smithi094.stdout:ERROR: (22) Invalid argument 2024-09-14T06:30:10.329 INFO:teuthology.orchestra.run.smithi094.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:30:10.344 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-14T06:30:10.367 INFO:tasks.ceph.osd.7.smithi138.stderr:2024-09-14T06:30:10.366+0000 7f3c6a21d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33623) UID: 0 2024-09-14T06:30:10.469 INFO:tasks.ceph.osd.3.smithi094.stderr:2024-09-14T06:30:10.468+0000 7feeeafe8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33655) UID: 0 2024-09-14T06:30:10.491 INFO:teuthology.orchestra.run.smithi094.stdout:ERROR: (22) Invalid argument 2024-09-14T06:30:10.492 INFO:teuthology.orchestra.run.smithi094.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:30:10.500 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-14T06:30:10.570 INFO:tasks.ceph.osd.7.smithi138.stderr:2024-09-14T06:30:10.569+0000 7f3c6a21d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33623) UID: 0 2024-09-14T06:30:10.649 INFO:teuthology.orchestra.run.smithi094.stdout:ERROR: (22) Invalid argument 2024-09-14T06:30:10.649 INFO:teuthology.orchestra.run.smithi094.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:30:10.659 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-14T06:30:10.671 INFO:tasks.ceph.osd.4.smithi138.stderr:2024-09-14T06:30:10.670+0000 7f6c653c8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33615) UID: 0 2024-09-14T06:30:10.746 INFO:teuthology.orchestra.run.smithi094.stdout:checking clone #16, image image_client.0-clone16 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent17 2024-09-14T06:30:10.771 INFO:teuthology.orchestra.run.smithi094.stdout:ERROR: (22) Invalid argument 2024-09-14T06:30:10.772 INFO:teuthology.orchestra.run.smithi094.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:30:10.772 INFO:tasks.ceph.osd.1.smithi094.stderr:2024-09-14T06:30:10.771+0000 7f328e47a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33641) UID: 0 2024-09-14T06:30:10.782 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-14T06:30:10.873 INFO:tasks.ceph.osd.7.smithi138.stderr:2024-09-14T06:30:10.872+0000 7f3c6a21d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33623) UID: 0 2024-09-14T06:30:10.884 INFO:teuthology.orchestra.run.smithi094.stdout:2630 punch from 0x4ada2cd to 0x4ae4d50, (0xaa83 bytes) 2024-09-14T06:30:10.886 INFO:teuthology.orchestra.run.smithi094.stdout:2631 write 0x3ac59a0 thru 0x3ac7aad (0x210e bytes) 2024-09-14T06:30:10.888 INFO:teuthology.orchestra.run.smithi094.stdout:2632 read 0xba26dac thru 0xba2f357 (0x85ac bytes) 2024-09-14T06:30:10.904 INFO:teuthology.orchestra.run.smithi094.stdout:2633 trunc from 0xe462e5b to 0x125de9b 2024-09-14T06:30:10.945 INFO:teuthology.orchestra.run.smithi094.stdout:ERROR: (22) Invalid argument 2024-09-14T06:30:10.945 INFO:teuthology.orchestra.run.smithi094.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:30:10.955 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-14T06:30:11.000 INFO:tasks.ceph.osd.3.smithi094.stderr:2024-09-14T06:30:10.999+0000 7feeeafe8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33655) UID: 0 2024-09-14T06:30:11.003 INFO:teuthology.orchestra.run.smithi094.stdout:2634 trunc from 0x125de9b to 0x7ba7bbe 2024-09-14T06:30:11.009 INFO:teuthology.orchestra.run.smithi094.stdout:2637 punch from 0x412f0b0 to 0x413b910, (0xc860 bytes) 2024-09-14T06:30:11.015 INFO:teuthology.orchestra.run.smithi094.stdout:2638 write 0x7e1eddb thru 0x7e29fd1 (0xb1f7 bytes) 2024-09-14T06:30:11.019 INFO:teuthology.orchestra.run.smithi094.stdout:2640 read 0x4bde1ec thru 0x4be4b2d (0x6942 bytes) 2024-09-14T06:30:11.019 INFO:teuthology.orchestra.run.smithi094.stdout:2641 read 0x4057748 thru 0x4061429 (0x9ce2 bytes) 2024-09-14T06:30:11.019 INFO:teuthology.orchestra.run.smithi094.stdout:2644 punch from 0x5cd8def to 0x5ce0493, (0x76a4 bytes) 2024-09-14T06:30:11.019 INFO:teuthology.orchestra.run.smithi094.stdout:2646 writesame 0x2b7723f thru 0x2b81619 data_size 0x1(0xa3db bytes) 2024-09-14T06:30:11.038 INFO:teuthology.orchestra.run.smithi094.stdout:2647 punch from 0x5a3cbe1 to 0x5a3f4a8, (0x28c7 bytes) 2024-09-14T06:30:11.038 INFO:teuthology.orchestra.run.smithi094.stdout:2648 write 0xcf475f thru 0xcfe5ae (0x9e50 bytes) 2024-09-14T06:30:11.040 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-14T06:30:11.040 INFO:teuthology.orchestra.run.smithi094.stdout:2650 read 0x73f0698 thru 0x73f2e1a (0x2783 bytes) 2024-09-14T06:30:11.054 INFO:teuthology.orchestra.run.smithi094.stdout:2652 writesame 0xdfc35b3 thru 0xdfc8e27 data_size 0x7(0x5875 bytes) 2024-09-14T06:30:11.069 INFO:teuthology.orchestra.run.smithi094.stdout:2653 trunc from 0xdfc8e28 to 0x13394ed 2024-09-14T06:30:11.075 INFO:teuthology.orchestra.run.smithi094.stdout:ERROR: (22) Invalid argument 2024-09-14T06:30:11.075 INFO:teuthology.orchestra.run.smithi094.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:30:11.075 INFO:tasks.ceph.osd.4.smithi138.stderr:2024-09-14T06:30:11.074+0000 7f6c653c8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33615) UID: 0 2024-09-14T06:30:11.084 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-14T06:30:11.095 INFO:teuthology.orchestra.run.smithi094.stdout:2654 write 0x4391c37 thru 0x439e2b7 (0xc681 bytes) 2024-09-14T06:30:11.117 INFO:teuthology.orchestra.run.smithi094.stdout:2655 writesame 0xe558d9e thru 0xe55e51f data_size 0x17(0x5782 bytes) 2024-09-14T06:30:11.117 INFO:teuthology.orchestra.run.smithi094.stdout:2656 write 0xc54508d thru 0xc550795 (0xb709 bytes) 2024-09-14T06:30:11.120 INFO:teuthology.orchestra.run.smithi094.stdout:2657 read 0x12b2cbf thru 0x12ba825 (0x7b67 bytes) 2024-09-14T06:30:11.120 INFO:teuthology.orchestra.run.smithi094.stdout:2658 punch from 0x4be13c7 to 0x4be9432, (0x806b bytes) 2024-09-14T06:30:11.120 INFO:teuthology.orchestra.run.smithi094.stdout:2659 read 0x57f0dab thru 0x57fb61a (0xa870 bytes) 2024-09-14T06:30:11.120 INFO:teuthology.orchestra.run.smithi094.stdout:2661 writesame 0xc19acfc thru 0xc1a443f data_size 0x2a(0x9744 bytes) 2024-09-14T06:30:11.126 INFO:teuthology.orchestra.run.smithi094.stdout:2663 read 0x5112c38 thru 0x51190ca (0x6493 bytes) 2024-09-14T06:30:11.127 INFO:teuthology.orchestra.run.smithi094.stdout:2664 punch from 0xc4524c0 to 0xc460779, (0xe2b9 bytes) 2024-09-14T06:30:11.127 INFO:teuthology.orchestra.run.smithi094.stdout:2665 write 0xcf6fc5 thru 0xd04703 (0xd73f bytes) 2024-09-14T06:30:11.129 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-14T06:30:11.141 INFO:teuthology.orchestra.run.smithi094.stdout:2667 trunc from 0xe55e520 to 0x4d9fed3 2024-09-14T06:30:11.154 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-14T06:30:11.155 INFO:teuthology.orchestra.run.smithi094.stdout:2670 read 0x1cac127 thru 0x1cb7133 (0xb00d bytes) 2024-09-14T06:30:11.170 INFO:teuthology.orchestra.run.smithi094.stdout:2671 write 0xc927a73 thru 0xc92fb4a (0x80d8 bytes) 2024-09-14T06:30:11.176 INFO:teuthology.orchestra.run.smithi094.stdout:2672 write 0xbd9c4f9 thru 0xbda15e1 (0x50e9 bytes) 2024-09-14T06:30:11.177 INFO:tasks.ceph.osd.5.smithi138.stderr:2024-09-14T06:30:11.176+0000 7fe13746f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33618) UID: 0 2024-09-14T06:30:11.177 INFO:teuthology.orchestra.run.smithi094.stdout:2673 writesame 0x192290d thru 0x1924711 data_size 0x1d(0x1e05 bytes) 2024-09-14T06:30:11.180 INFO:teuthology.orchestra.run.smithi094.stdout:2675 write 0xa9356c3 thru 0xa9455dd (0xff1b bytes) 2024-09-14T06:30:11.192 INFO:teuthology.orchestra.run.smithi094.stdout:2677 trunc from 0xc92fb4b to 0x58c08a7 2024-09-14T06:30:11.202 INFO:teuthology.orchestra.run.smithi094.stdout:2678 punch from 0x13e364f to 0x13e7b65, (0x4516 bytes) 2024-09-14T06:30:11.212 INFO:teuthology.orchestra.run.smithi094.stdout:ERROR: (22) Invalid argument 2024-09-14T06:30:11.212 INFO:teuthology.orchestra.run.smithi094.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:30:11.214 INFO:teuthology.orchestra.run.smithi094.stdout:2679 write 0xb633f1c thru 0xb6396c0 (0x57a5 bytes) 2024-09-14T06:30:11.217 INFO:teuthology.orchestra.run.smithi094.stdout:2681 read 0x260c08e thru 0x26189e6 (0xc959 bytes) 2024-09-14T06:30:11.217 INFO:teuthology.orchestra.run.smithi094.stdout:2683 write 0x33e3e96 thru 0x33edf18 (0xa083 bytes) 2024-09-14T06:30:11.220 INFO:teuthology.orchestra.run.smithi094.stdout:2684 write 0xa12d124 thru 0xa1359b6 (0x8893 bytes) 2024-09-14T06:30:11.222 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-14T06:30:11.224 INFO:teuthology.orchestra.run.smithi094.stdout:2685 read 0xb3dcf59 thru 0xb3dd486 (0x52e bytes) 2024-09-14T06:30:11.229 INFO:teuthology.orchestra.run.smithi094.stdout:2690 trunc from 0xb6396c1 to 0x6f42cee 2024-09-14T06:30:11.239 INFO:teuthology.orchestra.run.smithi094.stdout:2691 write 0x64754e3 thru 0x64837a3 (0xe2c1 bytes) 2024-09-14T06:30:11.247 INFO:teuthology.orchestra.run.smithi094.stdout:2692 write 0x737eafc thru 0x737f699 (0xb9e bytes) 2024-09-14T06:30:11.250 INFO:teuthology.orchestra.run.smithi094.stdout:2693 writesame 0x358ed5d thru 0x359e4ec data_size 0x88(0xf790 bytes) 2024-09-14T06:30:11.260 INFO:teuthology.orchestra.run.smithi094.stdout:2694 trunc from 0x737f69a to 0x1e4de3a 2024-09-14T06:30:11.270 INFO:teuthology.orchestra.run.smithi094.stdout:2695 read 0x178b77d thru 0x1791a7b (0x62ff bytes) 2024-09-14T06:30:11.271 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-14T06:30:11.278 INFO:tasks.ceph.osd.5.smithi138.stderr:2024-09-14T06:30:11.276+0000 7fe13746f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33618) UID: 0 2024-09-14T06:30:11.289 INFO:teuthology.orchestra.run.smithi094.stdout:2697 write 0xdc3f44f thru 0xdc46124 (0x6cd6 bytes) 2024-09-14T06:30:11.294 INFO:teuthology.orchestra.run.smithi094.stdout:2699 read 0x9729405 thru 0x972e5cf (0x51cb bytes) 2024-09-14T06:30:11.294 INFO:teuthology.orchestra.run.smithi094.stdout:2704 read 0x7533ae1 thru 0x7542d8e (0xf2ae bytes) 2024-09-14T06:30:11.294 INFO:teuthology.orchestra.run.smithi094.stdout:2705 read 0xa8bd3e7 thru 0xa8c39e5 (0x65ff bytes) 2024-09-14T06:30:11.294 INFO:teuthology.orchestra.run.smithi094.stdout:2706 writesame 0x2fd5dfb thru 0x2fd646a data_size 0x8(0x670 bytes) 2024-09-14T06:30:11.296 INFO:teuthology.orchestra.run.smithi094.stdout:2710 writesame 0x257f826 thru 0x258b57c data_size 0x3(0xbd57 bytes) 2024-09-14T06:30:11.309 INFO:teuthology.orchestra.run.smithi094.stdout:2712 read 0xb52fb62 thru 0xb53cda0 (0xd23f bytes) 2024-09-14T06:30:11.309 INFO:teuthology.orchestra.run.smithi094.stdout:2713 read 0x46c54cf thru 0x46c886c (0x339e bytes) 2024-09-14T06:30:11.316 INFO:teuthology.orchestra.run.smithi094.stdout:2714 trunc from 0xdc46125 to 0x7eadd67 2024-09-14T06:30:11.326 INFO:teuthology.orchestra.run.smithi094.stdout:2715 punch from 0x1d5393e to 0x1d5b46b, (0x7b2d bytes) 2024-09-14T06:30:11.326 INFO:teuthology.orchestra.run.smithi094.stdout:2716 read 0x41808bb thru 0x418e36a (0xdab0 bytes) 2024-09-14T06:30:11.327 INFO:teuthology.orchestra.run.smithi094.stdout:2718 write 0x78e4ae9 thru 0x78ea01c (0x5534 bytes) 2024-09-14T06:30:11.335 INFO:teuthology.orchestra.run.smithi094.stdout:2719 write 0x8e42db0 thru 0x8e461f3 (0x3444 bytes) 2024-09-14T06:30:11.338 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-14T06:30:11.338 INFO:teuthology.orchestra.run.smithi094.stdout:2721 read 0x7c99b1c thru 0x7ca8665 (0xeb4a bytes) 2024-09-14T06:30:11.338 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-14T06:30:11.344 INFO:teuthology.orchestra.run.smithi094.stdout:ERROR: (22) Invalid argument 2024-09-14T06:30:11.345 INFO:teuthology.orchestra.run.smithi094.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:30:11.347 INFO:teuthology.orchestra.run.smithi094.stdout:2726 trunc from 0x8e461f4 to 0x15bbaf6 2024-09-14T06:30:11.356 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-14T06:30:11.357 INFO:teuthology.orchestra.run.smithi094.stdout:2727 read 0x5838d8 thru 0x591fc9 (0xe6f2 bytes) 2024-09-14T06:30:11.358 INFO:teuthology.orchestra.run.smithi094.stdout:2728 read 0x3f1c67 thru 0x3fc212 (0xa5ac bytes) 2024-09-14T06:30:11.373 INFO:teuthology.orchestra.run.smithi094.stdout:2729 trunc from 0x15bbaf6 to 0xe2b72ac 2024-09-14T06:30:11.380 INFO:teuthology.orchestra.run.smithi094.stdout:2730 writesame 0x5ce62ff thru 0x5ceb5b8 data_size 0x1(0x52ba bytes) 2024-09-14T06:30:11.381 INFO:tasks.ceph.osd.2.smithi094.stderr:2024-09-14T06:30:11.379+0000 7faf3269b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33649) UID: 0 2024-09-14T06:30:11.391 INFO:teuthology.orchestra.run.smithi094.stdout:2731 writesame 0x9b10d9e thru 0x9b11d08 data_size 0x1(0xf6b bytes) 2024-09-14T06:30:11.395 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-14T06:30:11.395 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-14T06:30:11.395 INFO:teuthology.orchestra.run.smithi094.stdout:2735 punch from 0x9e70308 to 0x9e7f7d4, (0xf4cc bytes) 2024-09-14T06:30:11.395 INFO:teuthology.orchestra.run.smithi094.stdout:2736 write 0xfc86c5 thru 0xfd45ef (0xbf2b bytes) 2024-09-14T06:30:11.396 INFO:teuthology.orchestra.run.smithi094.stdout:2737 writesame 0x4b26f10 thru 0x4b2b06c data_size 0x1(0x415d bytes) 2024-09-14T06:30:11.411 INFO:teuthology.orchestra.run.smithi094.stdout:osd.0: {} 2024-09-14T06:30:11.411 INFO:teuthology.orchestra.run.smithi094.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-14T06:30:11.423 INFO:teuthology.orchestra.run.smithi094.stdout:2738 trunc from 0xe2b72ac to 0x32af27 2024-09-14T06:30:11.433 INFO:teuthology.orchestra.run.smithi094.stdout:osd.1: {} 2024-09-14T06:30:11.434 INFO:teuthology.orchestra.run.smithi094.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-14T06:30:11.454 INFO:teuthology.orchestra.run.smithi094.stdout:osd.2: {} 2024-09-14T06:30:11.454 INFO:teuthology.orchestra.run.smithi094.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-14T06:30:11.463 INFO:teuthology.orchestra.run.smithi094.stdout:2740 trunc from 0x32af27 to 0x6cacdb0 2024-09-14T06:30:11.470 INFO:teuthology.orchestra.run.smithi094.stdout:2743 write 0x39f55ca thru 0x3a048ff (0xf336 bytes) 2024-09-14T06:30:11.472 INFO:teuthology.orchestra.run.smithi094.stdout:osd.3: {} 2024-09-14T06:30:11.472 INFO:teuthology.orchestra.run.smithi094.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-14T06:30:11.477 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-14T06:30:11.477 INFO:teuthology.orchestra.run.smithi094.stdout:2745 read 0x40be441 thru 0x40cc95e (0xe51e bytes) 2024-09-14T06:30:11.478 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-14T06:30:11.478 INFO:teuthology.orchestra.run.smithi094.stdout:2747 read 0x3ba619 thru 0x3c1ca9 (0x7691 bytes) 2024-09-14T06:30:11.478 INFO:teuthology.orchestra.run.smithi094.stdout:2748 writesame 0x85783e thru 0x866aaf data_size 0x1(0xf272 bytes) 2024-09-14T06:30:11.481 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-14T06:30:11.481 INFO:teuthology.orchestra.run.smithi094.stdout: "ops": [], 2024-09-14T06:30:11.481 INFO:teuthology.orchestra.run.smithi094.stdout: "complaint_time": 30, 2024-09-14T06:30:11.482 INFO:teuthology.orchestra.run.smithi094.stdout: "num_blocked_ops": 0 2024-09-14T06:30:11.482 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-14T06:30:11.482 INFO:tasks.ceph.osd.0.smithi094.stderr:2024-09-14T06:30:11.481+0000 7f07c9333640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33636) UID: 0 2024-09-14T06:30:11.490 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-14T06:30:11.491 INFO:teuthology.orchestra.run.smithi094.stdout:osd.4: {} 2024-09-14T06:30:11.491 INFO:teuthology.orchestra.run.smithi094.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-14T06:30:11.501 INFO:teuthology.orchestra.run.smithi094.stdout:2749 trunc from 0x6cacdb0 to 0x3719e95 2024-09-14T06:30:11.510 INFO:teuthology.orchestra.run.smithi094.stdout:osd.5: {} 2024-09-14T06:30:11.510 INFO:teuthology.orchestra.run.smithi094.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-14T06:30:11.517 INFO:teuthology.orchestra.run.smithi094.stdout:2750 writesame 0x701a95 thru 0x710b8b data_size 0x1(0xf0f7 bytes) 2024-09-14T06:30:11.528 INFO:teuthology.orchestra.run.smithi094.stdout:osd.6: {} 2024-09-14T06:30:11.554 INFO:teuthology.orchestra.run.smithi094.stdout:osd.7: {} 2024-09-14T06:30:11.554 INFO:teuthology.orchestra.run.smithi094.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-14T06:30:11.563 INFO:teuthology.orchestra.run.smithi094.stdout:2752 write 0xd906dc9 thru 0xd908ef4 (0x212c bytes) 2024-09-14T06:30:11.567 INFO:teuthology.orchestra.run.smithi094.stdout:2754 write 0x5212b4d thru 0x5213e91 (0x1345 bytes) 2024-09-14T06:30:11.570 INFO:teuthology.orchestra.run.smithi094.stdout:2755 writesame 0x57d27a9 thru 0x57d7e40 data_size 0x44(0x5698 bytes) 2024-09-14T06:30:11.572 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-14T06:30:11.573 INFO:teuthology.orchestra.run.smithi094.stdout:2757 read 0x3d99bc4 thru 0x3d9eb22 (0x4f5f bytes) 2024-09-14T06:30:11.573 INFO:teuthology.orchestra.run.smithi094.stdout:2758 punch from 0x4e5047 to 0x4f239a, (0xd353 bytes) 2024-09-14T06:30:11.575 INFO:teuthology.orchestra.run.smithi094.stdout:2760 trunc from 0xd908ef5 to 0xbb229da 2024-09-14T06:30:11.585 INFO:teuthology.orchestra.run.smithi094.stdout:2761 punch from 0x273383b to 0x273decd, (0xa692 bytes) 2024-09-14T06:30:11.585 INFO:teuthology.orchestra.run.smithi094.stdout:2762 writesame 0x9ffc589 thru 0xa0009c6 data_size 0x5(0x443e bytes) 2024-09-14T06:30:11.585 INFO:tasks.ceph.osd.1.smithi094.stderr:2024-09-14T06:30:11.584+0000 7f328e47a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33641) UID: 0 2024-09-14T06:30:11.589 INFO:teuthology.orchestra.run.smithi094.stdout:2763 write 0x14aa3a4 thru 0x14b6aa6 (0xc703 bytes) 2024-09-14T06:30:11.592 INFO:teuthology.orchestra.run.smithi094.stdout:2764 punch from 0xca338 to 0xd0e45, (0x6b0d bytes) 2024-09-14T06:30:11.592 INFO:teuthology.orchestra.run.smithi094.stdout:2765 punch from 0x142ad4 to 0x14d839, (0xad65 bytes) 2024-09-14T06:30:11.593 INFO:teuthology.orchestra.run.smithi094.stdout:2766 punch from 0x9ece4d6 to 0x9ed2059, (0x3b83 bytes) 2024-09-14T06:30:11.593 INFO:teuthology.orchestra.run.smithi094.stdout:2768 read 0x140e3d2 thru 0x141d8e2 (0xf511 bytes) 2024-09-14T06:30:11.593 INFO:teuthology.orchestra.run.smithi094.stdout:2769 write 0x1f20438 thru 0x1f224c0 (0x2089 bytes) 2024-09-14T06:30:11.596 INFO:teuthology.orchestra.run.smithi094.stdout:2770 writesame 0x17ea45f thru 0x17f7ca2 data_size 0x2(0xd844 bytes) 2024-09-14T06:30:11.608 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-14T06:30:11.608 INFO:teuthology.orchestra.run.smithi094.stdout: "size": 20, 2024-09-14T06:30:11.608 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 600, 2024-09-14T06:30:11.608 INFO:teuthology.orchestra.run.smithi094.stdout: "ops": [ 2024-09-14T06:30:11.609 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:11.609 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:44425 3.2 3:4603ad7f:::rbd_data.112dca2f6ae1.0000000000000005:17 [read 868352~4096 [fadvise_nocache]] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:30:11.609 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:43.854967+0000", 2024-09-14T06:30:11.609 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 27.751311456, 2024-09-14T06:30:11.609 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.063146025999999994, 2024-09-14T06:30:11.609 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:30:11.609 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:30:11.609 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:30:11.609 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:30:11.609 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:30:11.609 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:11.609 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 44425 2024-09-14T06:30:11.609 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:11.610 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:30:11.610 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:11.610 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:30:11.610 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.854967+0000", 2024-09-14T06:30:11.610 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:11.610 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:11.610 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:11.610 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:30:11.610 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.854967+0000", 2024-09-14T06:30:11.610 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:11.610 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:11.610 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:11.610 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:30:11.610 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.854967+0000", 2024-09-14T06:30:11.611 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.16e-07 2024-09-14T06:30:11.611 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:11.611 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:11.611 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:30:11.611 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.854968+0000", 2024-09-14T06:30:11.611 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.006e-06 2024-09-14T06:30:11.611 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:11.611 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:11.611 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:30:11.611 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.854971+0000", 2024-09-14T06:30:11.611 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.9359999999999999e-06 2024-09-14T06:30:11.611 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:11.611 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:11.611 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:30:11.612 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.854974+0000", 2024-09-14T06:30:11.612 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.1840000000000002e-06 2024-09-14T06:30:11.612 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:11.612 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:11.612 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:30:11.612 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.917945+0000", 2024-09-14T06:30:11.612 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.062970925999999997 2024-09-14T06:30:11.612 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:11.612 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:11.612 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:30:11.612 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.918030+0000", 2024-09-14T06:30:11.612 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.5863000000000001e-05 2024-09-14T06:30:11.612 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:11.612 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:11.613 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:30:11.613 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.918113+0000", 2024-09-14T06:30:11.613 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.2894999999999998e-05 2024-09-14T06:30:11.613 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:11.613 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:11.613 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:11.613 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:11.613 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:11.613 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:44426 3.2 3:4603ad7f:::rbd_data.112dca2f6ae1.0000000000000005:17 [read 892928~4096 [fadvise_nocache]] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:30:11.613 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:43.854977+0000", 2024-09-14T06:30:11.613 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 27.751302045999999, 2024-09-14T06:30:11.613 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.063369237999999994, 2024-09-14T06:30:11.613 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:30:11.613 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:30:11.614 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:30:11.614 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:30:11.614 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:30:11.614 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:11.614 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 44426 2024-09-14T06:30:11.614 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:11.614 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:30:11.614 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:11.614 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:30:11.614 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.854977+0000", 2024-09-14T06:30:11.614 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:11.614 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:11.614 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:11.614 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:30:11.615 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.854977+0000", 2024-09-14T06:30:11.615 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:11.615 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:11.615 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:11.615 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:30:11.615 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.854977+0000", 2024-09-14T06:30:11.615 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.1199999999999999e-07 2024-09-14T06:30:11.615 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:11.615 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:11.615 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:30:11.615 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.854980+0000", 2024-09-14T06:30:11.615 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.27e-06 2024-09-14T06:30:11.615 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:11.615 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:11.616 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:30:11.616 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.854983+0000", 2024-09-14T06:30:11.616 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.1650000000000002e-06 2024-09-14T06:30:11.616 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:11.616 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:11.616 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:30:11.616 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.854989+0000", 2024-09-14T06:30:11.616 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.6520000000000003e-06 2024-09-14T06:30:11.616 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:11.616 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:11.616 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:30:11.616 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.918143+0000", 2024-09-14T06:30:11.616 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.063154541999999994 2024-09-14T06:30:11.616 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:11.617 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:11.617 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:30:11.617 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.918209+0000", 2024-09-14T06:30:11.617 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.5529000000000002e-05 2024-09-14T06:30:11.617 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:11.617 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:11.617 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:30:11.617 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.918346+0000", 2024-09-14T06:30:11.617 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000136868 2024-09-14T06:30:11.617 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:11.617 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:11.617 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:11.617 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:11.617 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:11.617 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:44427 3.2 3:4603ad7f:::rbd_data.112dca2f6ae1.0000000000000005:17 [read 917504~4096 [fadvise_nocache]] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:30:11.618 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:43.854992+0000", 2024-09-14T06:30:11.618 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 27.751286400000001, 2024-09-14T06:30:11.618 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.063607792999999996, 2024-09-14T06:30:11.618 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:30:11.618 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:30:11.618 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:30:11.618 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:30:11.618 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:30:11.618 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:11.618 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 44427 2024-09-14T06:30:11.618 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:11.618 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:30:11.618 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:11.618 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:30:11.619 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.854992+0000", 2024-09-14T06:30:11.619 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:11.619 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:11.619 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:11.619 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:30:11.619 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.854992+0000", 2024-09-14T06:30:11.619 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:11.619 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:11.619 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:11.619 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:30:11.619 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.854993+0000", 2024-09-14T06:30:11.619 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.1400000000000001e-07 2024-09-14T06:30:11.619 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:11.619 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:11.620 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:30:11.620 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.854994+0000", 2024-09-14T06:30:11.620 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.091e-06 2024-09-14T06:30:11.620 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:11.620 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:11.620 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:30:11.620 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855001+0000", 2024-09-14T06:30:11.620 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.204e-06 2024-09-14T06:30:11.620 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:11.620 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:11.621 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:30:11.621 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855003+0000", 2024-09-14T06:30:11.621 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.514e-06 2024-09-14T06:30:11.621 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:11.621 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:11.621 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:30:11.621 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.918380+0000", 2024-09-14T06:30:11.621 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.063376738000000002 2024-09-14T06:30:11.621 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:11.621 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:11.621 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:30:11.621 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.918453+0000", 2024-09-14T06:30:11.621 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.3291e-05 2024-09-14T06:30:11.621 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:11.622 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:11.622 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:30:11.622 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.918600+0000", 2024-09-14T06:30:11.622 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00014674100000000001 2024-09-14T06:30:11.622 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:11.622 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:11.622 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:11.622 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:11.622 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:11.622 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:44428 3.2 3:4603ad7f:::rbd_data.112dca2f6ae1.0000000000000005:17 [read 942080~4096 [fadvise_nocache]] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:30:11.622 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:43.855006+0000", 2024-09-14T06:30:11.622 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 27.751272350000001, 2024-09-14T06:30:11.622 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.063846321999999997, 2024-09-14T06:30:11.623 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:30:11.623 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:30:11.623 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:30:11.623 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:30:11.623 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:30:11.623 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:11.623 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 44428 2024-09-14T06:30:11.623 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:11.623 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:30:11.623 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:11.623 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:30:11.623 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855006+0000", 2024-09-14T06:30:11.623 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:11.623 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:11.624 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:11.624 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:30:11.624 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855006+0000", 2024-09-14T06:30:11.624 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:11.624 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:11.624 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:11.624 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:30:11.624 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855007+0000", 2024-09-14T06:30:11.624 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.1199999999999999e-07 2024-09-14T06:30:11.624 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:11.624 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:11.624 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:30:11.624 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855011+0000", 2024-09-14T06:30:11.625 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.9539999999999998e-06 2024-09-14T06:30:11.625 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:11.625 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:11.625 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:30:11.625 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855014+0000", 2024-09-14T06:30:11.625 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.2930000000000001e-06 2024-09-14T06:30:11.625 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:11.625 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:11.625 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:30:11.625 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855016+0000", 2024-09-14T06:30:11.625 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.3039999999999999e-06 2024-09-14T06:30:11.625 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:11.625 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:11.626 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:30:11.626 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.918634+0000", 2024-09-14T06:30:11.626 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.063617928000000004 2024-09-14T06:30:11.626 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:11.626 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:11.626 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:30:11.626 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.918711+0000", 2024-09-14T06:30:11.626 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.6545000000000007e-05 2024-09-14T06:30:11.626 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:11.626 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:11.626 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:30:11.626 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.918853+0000", 2024-09-14T06:30:11.626 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00014208600000000001 2024-09-14T06:30:11.626 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:11.626 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:11.627 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:11.627 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:11.627 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:11.627 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:44429 3.2 3:4603ad7f:::rbd_data.112dca2f6ae1.0000000000000005:17 [read 966656~4096 [fadvise_nocache]] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:30:11.627 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:43.855019+0000", 2024-09-14T06:30:11.627 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 27.751259677, 2024-09-14T06:30:11.627 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.064059226999999996, 2024-09-14T06:30:11.627 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:30:11.627 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:30:11.627 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:30:11.627 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:30:11.627 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:30:11.627 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:11.628 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 44429 2024-09-14T06:30:11.628 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:11.628 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:30:11.628 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:11.628 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:30:11.628 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855019+0000", 2024-09-14T06:30:11.628 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:11.628 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:11.628 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:11.628 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:30:11.628 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855019+0000", 2024-09-14T06:30:11.628 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:11.628 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:11.629 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:11.629 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:30:11.629 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855019+0000", 2024-09-14T06:30:11.629 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.28e-07 2024-09-14T06:30:11.629 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:11.629 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:11.629 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:30:11.629 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855020+0000", 2024-09-14T06:30:11.629 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.0330000000000001e-06 2024-09-14T06:30:11.629 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:11.629 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:11.629 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:30:11.629 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855023+0000", 2024-09-14T06:30:11.629 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.79e-06 2024-09-14T06:30:11.630 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:11.630 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:11.630 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:30:11.630 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855025+0000", 2024-09-14T06:30:11.630 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.216e-06 2024-09-14T06:30:11.630 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:11.630 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:11.630 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:30:11.630 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.918886+0000", 2024-09-14T06:30:11.630 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.063861037999999995 2024-09-14T06:30:11.630 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:11.630 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:11.630 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:30:11.630 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.918952+0000", 2024-09-14T06:30:11.631 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.5993000000000003e-05 2024-09-14T06:30:11.631 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:11.631 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:11.631 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:30:11.631 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.919078+0000", 2024-09-14T06:30:11.631 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00012592899999999999 2024-09-14T06:30:11.631 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:11.631 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:11.631 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:11.631 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:11.631 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:11.631 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:44430 3.2 3:4603ad7f:::rbd_data.112dca2f6ae1.0000000000000005:17 [read 991232~4096 [fadvise_nocache]] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:30:11.631 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:43.855028+0000", 2024-09-14T06:30:11.631 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 27.751250667000001, 2024-09-14T06:30:11.631 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.064284346000000006, 2024-09-14T06:30:11.632 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:30:11.632 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:30:11.632 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:30:11.632 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:30:11.632 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:30:11.632 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:11.632 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 44430 2024-09-14T06:30:11.632 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:11.632 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:30:11.632 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:11.632 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:30:11.632 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855028+0000", 2024-09-14T06:30:11.632 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:11.632 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:11.633 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:11.633 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:30:11.633 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855028+0000", 2024-09-14T06:30:11.633 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:11.633 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:11.633 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:11.633 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:30:11.633 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855028+0000", 2024-09-14T06:30:11.633 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.1199999999999999e-07 2024-09-14T06:30:11.633 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:11.633 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:11.633 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:30:11.633 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855029+0000", 2024-09-14T06:30:11.633 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.1100000000000004e-07 2024-09-14T06:30:11.634 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:11.634 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:11.634 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:30:11.634 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855032+0000", 2024-09-14T06:30:11.634 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.7889999999999999e-06 2024-09-14T06:30:11.634 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:11.634 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:11.634 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:30:11.634 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855034+0000", 2024-09-14T06:30:11.634 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.125e-06 2024-09-14T06:30:11.634 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:11.634 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:11.634 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:30:11.634 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.919110+0000", 2024-09-14T06:30:11.635 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.064076413999999998 2024-09-14T06:30:11.635 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:11.635 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:11.635 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:30:11.635 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.919186+0000", 2024-09-14T06:30:11.635 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.5784999999999994e-05 2024-09-14T06:30:11.635 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:11.635 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:11.635 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:30:11.635 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.919312+0000", 2024-09-14T06:30:11.635 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00012611 2024-09-14T06:30:11.635 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:11.635 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:11.635 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:11.635 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:11.636 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:11.636 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:44431 3.2 3:4603ad7f:::rbd_data.112dca2f6ae1.0000000000000005:17 [read 1015808~4096 [fadvise_nocache]] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:30:11.636 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:43.855037+0000", 2024-09-14T06:30:11.636 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 27.751241798999999, 2024-09-14T06:30:11.636 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.064469755000000004, 2024-09-14T06:30:11.636 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:30:11.636 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:30:11.636 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:30:11.636 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:30:11.636 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:30:11.636 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:11.636 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 44431 2024-09-14T06:30:11.636 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:11.637 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:30:11.637 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:11.637 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:30:11.637 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855037+0000", 2024-09-14T06:30:11.637 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:11.637 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:11.637 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:11.637 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:30:11.637 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855037+0000", 2024-09-14T06:30:11.637 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:11.637 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:11.637 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:11.637 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:30:11.637 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855037+0000", 2024-09-14T06:30:11.638 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.1e-07 2024-09-14T06:30:11.638 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:11.638 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:11.638 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:30:11.638 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855038+0000", 2024-09-14T06:30:11.638 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.9400000000000004e-07 2024-09-14T06:30:11.638 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:11.638 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:11.638 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:30:11.638 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855041+0000", 2024-09-14T06:30:11.638 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.9639999999999999e-06 2024-09-14T06:30:11.638 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:11.638 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:11.638 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:30:11.639 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855043+0000", 2024-09-14T06:30:11.639 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.2259999999999999e-06 2024-09-14T06:30:11.639 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:11.639 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:11.639 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:30:11.639 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.919347+0000", 2024-09-14T06:30:11.639 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.064304091999999993 2024-09-14T06:30:11.639 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:11.639 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:11.639 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:30:11.639 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.919393+0000", 2024-09-14T06:30:11.639 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.5340999999999998e-05 2024-09-14T06:30:11.639 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:11.639 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:11.640 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:30:11.640 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.919507+0000", 2024-09-14T06:30:11.640 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000114028 2024-09-14T06:30:11.640 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:11.640 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:11.640 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:11.640 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:11.640 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:11.640 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:44432 3.2 3:4603ad7f:::rbd_data.112dca2f6ae1.0000000000000005:17 [read 1040384~4096 [fadvise_nocache]] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:30:11.640 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:43.855046+0000", 2024-09-14T06:30:11.640 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 27.751232731000002, 2024-09-14T06:30:11.640 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.064639253999999993, 2024-09-14T06:30:11.640 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:30:11.641 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:30:11.641 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:30:11.641 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:30:11.641 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:30:11.641 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:11.641 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 44432 2024-09-14T06:30:11.641 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:11.641 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:30:11.641 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:11.641 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:30:11.641 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855046+0000", 2024-09-14T06:30:11.641 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:11.641 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:11.641 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:11.642 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:30:11.642 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855046+0000", 2024-09-14T06:30:11.642 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:11.642 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:11.642 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:11.642 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:30:11.642 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855046+0000", 2024-09-14T06:30:11.642 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.05e-07 2024-09-14T06:30:11.642 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:11.642 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:11.642 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:30:11.642 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855047+0000", 2024-09-14T06:30:11.642 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.879999999999999e-07 2024-09-14T06:30:11.642 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:11.643 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:11.643 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:30:11.643 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855050+0000", 2024-09-14T06:30:11.643 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.8459999999999999e-06 2024-09-14T06:30:11.643 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:11.643 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:11.643 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:30:11.643 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855052+0000", 2024-09-14T06:30:11.643 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.136e-06 2024-09-14T06:30:11.643 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:11.643 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:11.643 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:30:11.643 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.919536+0000", 2024-09-14T06:30:11.643 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.064484132 2024-09-14T06:30:11.644 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:11.644 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:11.644 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:30:11.644 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.919611+0000", 2024-09-14T06:30:11.644 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.4407999999999996e-05 2024-09-14T06:30:11.644 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:11.644 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:11.644 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:30:11.644 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.919685+0000", 2024-09-14T06:30:11.644 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.4538999999999999e-05 2024-09-14T06:30:11.644 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:11.644 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:11.644 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:11.644 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:11.645 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:11.645 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:44518 3.2 3:4190d882:::rbd_data.112dca2f6ae1.0000000000000002:17 [read 12288~4096 [fadvise_nocache]] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:30:11.645 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:43.855055+0000", 2024-09-14T06:30:11.645 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 27.751223846999999, 2024-09-14T06:30:11.645 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.064894317000000007, 2024-09-14T06:30:11.645 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:30:11.645 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:30:11.645 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:30:11.645 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:30:11.645 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:30:11.645 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:11.645 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 44518 2024-09-14T06:30:11.645 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:11.645 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:30:11.646 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:11.646 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:30:11.646 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855055+0000", 2024-09-14T06:30:11.646 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:11.646 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:11.646 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:11.646 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:30:11.646 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855055+0000", 2024-09-14T06:30:11.646 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:11.646 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:11.646 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:11.646 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:30:11.646 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855055+0000", 2024-09-14T06:30:11.646 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.0699999999999999e-07 2024-09-14T06:30:11.647 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:11.647 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:11.647 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:30:11.647 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855056+0000", 2024-09-14T06:30:11.647 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.9400000000000004e-07 2024-09-14T06:30:11.647 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:11.647 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:11.647 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:30:11.647 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855059+0000", 2024-09-14T06:30:11.647 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.8870000000000001e-06 2024-09-14T06:30:11.647 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:11.647 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:11.647 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:30:11.648 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855061+0000", 2024-09-14T06:30:11.648 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.1919999999999999e-06 2024-09-14T06:30:11.648 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:11.648 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:11.648 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:30:11.648 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.919719+0000", 2024-09-14T06:30:11.648 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.064658013 2024-09-14T06:30:11.648 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:11.648 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:11.648 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:30:11.648 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.919788+0000", 2024-09-14T06:30:11.648 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.9282000000000005e-05 2024-09-14T06:30:11.648 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:11.649 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:11.649 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:30:11.649 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.919949+0000", 2024-09-14T06:30:11.649 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00016084199999999999 2024-09-14T06:30:11.649 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:11.649 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:11.649 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:11.649 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:11.649 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:11.649 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:44519 3.2 3:4190d882:::rbd_data.112dca2f6ae1.0000000000000002:17 [read 36864~4096 [fadvise_nocache]] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:30:11.649 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:43.855064+0000", 2024-09-14T06:30:11.649 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 27.751214869999998, 2024-09-14T06:30:11.649 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.065081865000000003, 2024-09-14T06:30:11.649 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:30:11.650 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:30:11.650 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:30:11.650 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:30:11.650 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:30:11.650 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:11.650 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 44519 2024-09-14T06:30:11.650 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:11.650 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:30:11.650 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:11.650 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:30:11.650 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855064+0000", 2024-09-14T06:30:11.650 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:11.650 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:11.650 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:11.651 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:30:11.651 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855064+0000", 2024-09-14T06:30:11.651 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:11.651 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:11.651 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:11.651 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:30:11.651 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855064+0000", 2024-09-14T06:30:11.651 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.1e-07 2024-09-14T06:30:11.651 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:11.651 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:11.651 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:30:11.651 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855065+0000", 2024-09-14T06:30:11.651 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.035e-06 2024-09-14T06:30:11.652 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:11.652 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:11.652 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:30:11.652 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855068+0000", 2024-09-14T06:30:11.652 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.8339999999999999e-06 2024-09-14T06:30:11.652 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:11.652 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:11.652 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:30:11.652 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855071+0000", 2024-09-14T06:30:11.652 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.3450000000000002e-06 2024-09-14T06:30:11.652 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:11.652 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:11.652 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:30:11.653 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.919979+0000", 2024-09-14T06:30:11.653 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.064907647999999998 2024-09-14T06:30:11.653 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:11.653 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:11.653 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:30:11.653 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.920026+0000", 2024-09-14T06:30:11.653 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.7595000000000001e-05 2024-09-14T06:30:11.653 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:11.653 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:11.653 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:30:11.653 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.920146+0000", 2024-09-14T06:30:11.653 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00011919800000000001 2024-09-14T06:30:11.653 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:11.653 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:11.654 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:11.654 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:11.654 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:11.654 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:44520 3.2 3:4190d882:::rbd_data.112dca2f6ae1.0000000000000002:17 [read 61440~4096 [fadvise_nocache]] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:30:11.654 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:43.855075+0000", 2024-09-14T06:30:11.654 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 27.751203819000001, 2024-09-14T06:30:11.654 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.065335078000000005, 2024-09-14T06:30:11.654 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:30:11.654 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:30:11.654 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:30:11.654 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:30:11.654 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:30:11.654 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:11.655 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 44520 2024-09-14T06:30:11.655 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:11.655 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:30:11.655 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:11.655 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:30:11.655 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855075+0000", 2024-09-14T06:30:11.655 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:11.655 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:11.655 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:11.655 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:30:11.655 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855075+0000", 2024-09-14T06:30:11.655 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:11.655 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:11.655 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:11.655 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:30:11.656 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855075+0000", 2024-09-14T06:30:11.656 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.1e-07 2024-09-14T06:30:11.656 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:11.656 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:11.656 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:30:11.656 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855076+0000", 2024-09-14T06:30:11.656 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.0839999999999999e-06 2024-09-14T06:30:11.656 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:11.656 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:11.656 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:30:11.656 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855079+0000", 2024-09-14T06:30:11.656 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.9359999999999999e-06 2024-09-14T06:30:11.656 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:11.657 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:11.657 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:30:11.657 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855084+0000", 2024-09-14T06:30:11.657 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.1730000000000003e-06 2024-09-14T06:30:11.657 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:11.657 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:11.657 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:30:11.657 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.920175+0000", 2024-09-14T06:30:11.657 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.065090234999999996 2024-09-14T06:30:11.657 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:11.657 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:11.657 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:30:11.657 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.920250+0000", 2024-09-14T06:30:11.658 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.5807000000000001e-05 2024-09-14T06:30:11.658 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:11.658 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:11.658 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:30:11.658 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.920410+0000", 2024-09-14T06:30:11.658 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00015963300000000001 2024-09-14T06:30:11.658 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:11.658 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:11.658 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:11.658 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:11.658 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:11.658 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:44521 3.2 3:4190d882:::rbd_data.112dca2f6ae1.0000000000000002:17 [read 86016~4096 [fadvise_nocache]] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:30:11.658 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:43.855087+0000", 2024-09-14T06:30:11.658 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 27.751191338000002, 2024-09-14T06:30:11.659 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.065563264999999996, 2024-09-14T06:30:11.659 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:30:11.659 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:30:11.659 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:30:11.659 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:30:11.659 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:30:11.659 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:11.659 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 44521 2024-09-14T06:30:11.659 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:11.659 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:30:11.659 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:11.659 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:30:11.659 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855087+0000", 2024-09-14T06:30:11.660 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:11.660 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:11.660 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:11.660 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:30:11.660 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855087+0000", 2024-09-14T06:30:11.660 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:11.660 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:11.660 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:11.660 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:30:11.660 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855088+0000", 2024-09-14T06:30:11.660 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.3699999999999999e-07 2024-09-14T06:30:11.660 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:11.660 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:11.661 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:30:11.661 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855089+0000", 2024-09-14T06:30:11.661 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.9900000000000009e-07 2024-09-14T06:30:11.661 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:11.661 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:11.661 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:30:11.661 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855091+0000", 2024-09-14T06:30:11.661 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.8940000000000002e-06 2024-09-14T06:30:11.661 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:11.661 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:11.661 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:30:11.661 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855094+0000", 2024-09-14T06:30:11.661 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.351e-06 2024-09-14T06:30:11.661 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:11.662 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:11.662 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:30:11.662 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.920437+0000", 2024-09-14T06:30:11.662 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.065343221000000007 2024-09-14T06:30:11.662 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:11.662 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:11.662 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:30:11.662 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.920481+0000", 2024-09-14T06:30:11.662 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.4431000000000002e-05 2024-09-14T06:30:11.662 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:11.662 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:11.662 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:30:11.662 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.920651+0000", 2024-09-14T06:30:11.663 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00016913200000000001 2024-09-14T06:30:11.663 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:11.663 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:11.663 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:11.663 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:11.663 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:11.663 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:44522 3.2 3:4190d882:::rbd_data.112dca2f6ae1.0000000000000002:17 [read 110592~4096 [fadvise_nocache]] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:30:11.663 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:43.855098+0000", 2024-09-14T06:30:11.663 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 27.751181111000001, 2024-09-14T06:30:11.663 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.065827628999999999, 2024-09-14T06:30:11.663 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:30:11.663 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:30:11.663 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:30:11.663 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:30:11.664 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:30:11.664 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:11.664 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 44522 2024-09-14T06:30:11.664 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:11.664 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:30:11.664 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:11.664 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:30:11.664 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855098+0000", 2024-09-14T06:30:11.664 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:11.664 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:11.664 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:11.664 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:30:11.664 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855098+0000", 2024-09-14T06:30:11.665 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:11.665 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:11.665 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:11.665 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:30:11.665 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855098+0000", 2024-09-14T06:30:11.665 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.1e-07 2024-09-14T06:30:11.665 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:11.665 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:11.665 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:30:11.665 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855099+0000", 2024-09-14T06:30:11.665 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.012e-06 2024-09-14T06:30:11.665 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:11.665 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:11.665 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:30:11.666 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855102+0000", 2024-09-14T06:30:11.666 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.9380000000000001e-06 2024-09-14T06:30:11.666 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:11.666 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:11.666 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:30:11.666 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855108+0000", 2024-09-14T06:30:11.666 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.4339999999999997e-06 2024-09-14T06:30:11.666 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:11.666 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:11.666 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:30:11.666 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.920681+0000", 2024-09-14T06:30:11.666 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.065572943999999994 2024-09-14T06:30:11.666 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:11.666 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:11.667 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:30:11.667 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.920724+0000", 2024-09-14T06:30:11.667 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.2877000000000003e-05 2024-09-14T06:30:11.667 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:11.667 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:11.667 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:30:11.667 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.920925+0000", 2024-09-14T06:30:11.667 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00020121399999999999 2024-09-14T06:30:11.667 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:11.667 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:11.667 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:11.667 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:11.667 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:11.668 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:44523 3.2 3:4190d882:::rbd_data.112dca2f6ae1.0000000000000002:17 [read 135168~4096 [fadvise_nocache]] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:30:11.668 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:43.855111+0000", 2024-09-14T06:30:11.668 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 27.751167388999999, 2024-09-14T06:30:11.668 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.066028198999999996, 2024-09-14T06:30:11.668 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:30:11.668 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:30:11.668 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:30:11.668 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:30:11.668 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:30:11.668 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:11.668 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 44523 2024-09-14T06:30:11.668 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:11.668 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:30:11.669 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:11.669 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:30:11.669 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855111+0000", 2024-09-14T06:30:11.669 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:11.669 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:11.669 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:11.669 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:30:11.669 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855111+0000", 2024-09-14T06:30:11.669 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:11.669 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:11.669 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:11.669 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:30:11.669 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855112+0000", 2024-09-14T06:30:11.670 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.1199999999999999e-07 2024-09-14T06:30:11.670 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:11.670 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:11.670 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:30:11.670 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855115+0000", 2024-09-14T06:30:11.670 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.3380000000000001e-06 2024-09-14T06:30:11.670 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:11.670 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:11.670 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:30:11.670 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855118+0000", 2024-09-14T06:30:11.670 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.366e-06 2024-09-14T06:30:11.670 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:11.670 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:11.670 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:30:11.671 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855121+0000", 2024-09-14T06:30:11.671 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.8329999999999998e-06 2024-09-14T06:30:11.671 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:11.671 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:11.671 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:30:11.671 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.920956+0000", 2024-09-14T06:30:11.671 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.065835251999999997 2024-09-14T06:30:11.671 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:11.671 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:11.671 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:30:11.671 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.921001+0000", 2024-09-14T06:30:11.671 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.4778999999999999e-05 2024-09-14T06:30:11.671 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:11.671 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:11.672 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:30:11.672 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.921139+0000", 2024-09-14T06:30:11.672 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000138419 2024-09-14T06:30:11.672 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:11.672 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:11.672 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:11.672 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:11.672 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:11.672 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:44524 3.2 3:4190d882:::rbd_data.112dca2f6ae1.0000000000000002:17 [read 159744~4096 [fadvise_nocache]] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:30:11.672 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:43.855124+0000", 2024-09-14T06:30:11.672 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 27.751154410000002, 2024-09-14T06:30:11.672 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.066249261000000004, 2024-09-14T06:30:11.672 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:30:11.673 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:30:11.673 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:30:11.673 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:30:11.673 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:30:11.673 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:11.673 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 44524 2024-09-14T06:30:11.673 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:11.673 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:30:11.673 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:11.673 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:30:11.673 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855124+0000", 2024-09-14T06:30:11.673 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:11.673 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:11.673 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:11.674 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:30:11.674 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855124+0000", 2024-09-14T06:30:11.674 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:11.674 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:11.674 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:11.674 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:30:11.674 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861086+0000", 2024-09-14T06:30:11.674 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0059613829999999998 2024-09-14T06:30:11.674 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:11.674 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:11.674 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:30:11.674 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861100+0000", 2024-09-14T06:30:11.674 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.4644000000000001e-05 2024-09-14T06:30:11.674 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:11.674 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:11.675 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:30:11.675 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861109+0000", 2024-09-14T06:30:11.675 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.541e-06 2024-09-14T06:30:11.675 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:11.675 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:11.675 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:30:11.675 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861117+0000", 2024-09-14T06:30:11.675 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.1049999999999995e-06 2024-09-14T06:30:11.675 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:11.675 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:11.675 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:30:11.675 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.921169+0000", 2024-09-14T06:30:11.675 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.060052405000000003 2024-09-14T06:30:11.675 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:11.676 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:11.676 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:30:11.676 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.921213+0000", 2024-09-14T06:30:11.676 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.3565e-05 2024-09-14T06:30:11.676 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:11.676 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:11.676 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:30:11.676 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.921374+0000", 2024-09-14T06:30:11.676 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00016061800000000001 2024-09-14T06:30:11.676 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:11.676 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:11.676 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:11.676 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:11.676 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:11.677 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:44536 3.2 3:4190d882:::rbd_data.112dca2f6ae1.0000000000000002:17 [read 454656~4096 [fadvise_nocache]] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:30:11.677 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:43.861384+0000", 2024-09-14T06:30:11.677 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 27.744894853000002, 2024-09-14T06:30:11.677 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.063058528000000003, 2024-09-14T06:30:11.677 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:30:11.677 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:30:11.677 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:30:11.677 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:30:11.677 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:30:11.677 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:11.677 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 44536 2024-09-14T06:30:11.677 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:11.677 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:30:11.677 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:11.678 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:30:11.678 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861384+0000", 2024-09-14T06:30:11.678 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:11.678 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:11.678 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:11.678 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:30:11.678 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861384+0000", 2024-09-14T06:30:11.678 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:11.678 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:11.678 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:11.678 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:30:11.678 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861384+0000", 2024-09-14T06:30:11.678 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.4000000000000002e-07 2024-09-14T06:30:11.679 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:11.679 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:11.679 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:30:11.679 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861386+0000", 2024-09-14T06:30:11.679 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.891e-06 2024-09-14T06:30:11.679 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:11.679 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:11.679 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:30:11.679 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861392+0000", 2024-09-14T06:30:11.679 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.9939999999999997e-06 2024-09-14T06:30:11.679 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:11.679 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:11.679 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:30:11.680 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861398+0000", 2024-09-14T06:30:11.680 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.507e-06 2024-09-14T06:30:11.680 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:11.680 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:11.680 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:30:11.680 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.924149+0000", 2024-09-14T06:30:11.680 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.062751135999999999 2024-09-14T06:30:11.680 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:11.680 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:11.680 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:30:11.680 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.924226+0000", 2024-09-14T06:30:11.680 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.7517999999999999e-05 2024-09-14T06:30:11.680 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:11.680 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:11.681 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:30:11.681 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.924442+0000", 2024-09-14T06:30:11.681 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000216042 2024-09-14T06:30:11.681 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:11.681 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:11.681 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:11.681 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:11.681 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:11.681 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:44537 3.2 3:4190d882:::rbd_data.112dca2f6ae1.0000000000000002:17 [read 479232~4096 [fadvise_nocache]] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:30:11.681 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:43.861405+0000", 2024-09-14T06:30:11.681 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 27.744873807000001, 2024-09-14T06:30:11.681 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.063351795000000002, 2024-09-14T06:30:11.681 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:30:11.682 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:30:11.682 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:30:11.682 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:30:11.682 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:30:11.682 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:11.682 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 44537 2024-09-14T06:30:11.682 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:11.682 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:30:11.682 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:11.682 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:30:11.682 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861405+0000", 2024-09-14T06:30:11.682 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:11.682 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:11.683 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:11.683 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:30:11.683 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861405+0000", 2024-09-14T06:30:11.683 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:11.683 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:11.683 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:11.683 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:30:11.683 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861405+0000", 2024-09-14T06:30:11.683 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.9099999999999999e-07 2024-09-14T06:30:11.683 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:11.683 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:11.683 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:30:11.683 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861407+0000", 2024-09-14T06:30:11.683 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.075e-06 2024-09-14T06:30:11.684 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:11.684 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:11.684 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:30:11.684 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861414+0000", 2024-09-14T06:30:11.684 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.4659999999999996e-06 2024-09-14T06:30:11.684 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:11.684 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:11.684 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:30:11.684 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861418+0000", 2024-09-14T06:30:11.684 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.5519999999999998e-06 2024-09-14T06:30:11.684 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:11.684 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:11.684 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:30:11.684 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.924474+0000", 2024-09-14T06:30:11.685 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.063056012999999994 2024-09-14T06:30:11.685 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:11.685 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:11.685 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:30:11.685 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.924556+0000", 2024-09-14T06:30:11.685 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.1181999999999996e-05 2024-09-14T06:30:11.685 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:11.685 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:11.685 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:30:11.685 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.924757+0000", 2024-09-14T06:30:11.685 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00020111599999999999 2024-09-14T06:30:11.685 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:11.685 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:11.685 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:11.686 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:11.686 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:11.686 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:44538 3.2 3:4190d882:::rbd_data.112dca2f6ae1.0000000000000002:17 [read 503808~4096 [fadvise_nocache]] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:30:11.686 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:43.861425+0000", 2024-09-14T06:30:11.686 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 27.744853765999999, 2024-09-14T06:30:11.686 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.063627125000000007, 2024-09-14T06:30:11.686 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:30:11.686 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:30:11.686 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:30:11.686 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:30:11.686 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:30:11.686 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:11.686 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 44538 2024-09-14T06:30:11.687 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:11.687 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:30:11.687 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:11.687 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:30:11.687 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861425+0000", 2024-09-14T06:30:11.687 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:11.687 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:11.687 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:11.687 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:30:11.687 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861425+0000", 2024-09-14T06:30:11.687 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:11.687 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:11.687 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:11.687 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:30:11.688 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861425+0000", 2024-09-14T06:30:11.688 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.9999999999999998e-07 2024-09-14T06:30:11.688 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:11.688 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:11.688 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:30:11.688 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861427+0000", 2024-09-14T06:30:11.688 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.9829999999999999e-06 2024-09-14T06:30:11.688 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:11.688 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:11.688 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:30:11.688 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861433+0000", 2024-09-14T06:30:11.688 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.2889999999999997e-06 2024-09-14T06:30:11.688 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:11.689 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:11.689 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:30:11.689 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861454+0000", 2024-09-14T06:30:11.689 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.1844999999999998e-05 2024-09-14T06:30:11.689 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:11.689 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:11.689 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:30:11.689 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.924786+0000", 2024-09-14T06:30:11.689 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.063331497 2024-09-14T06:30:11.689 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:11.689 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:11.689 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:30:11.689 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.924864+0000", 2024-09-14T06:30:11.689 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.7744000000000006e-05 2024-09-14T06:30:11.690 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:11.690 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:11.690 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:30:11.690 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.925052+0000", 2024-09-14T06:30:11.690 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00018836699999999999 2024-09-14T06:30:11.690 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:11.690 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:11.690 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:11.690 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:11.690 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:11.690 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:44539 3.2 3:4190d882:::rbd_data.112dca2f6ae1.0000000000000002:17 [read 528384~4096 [fadvise_nocache]] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:30:11.690 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:43.861462+0000", 2024-09-14T06:30:11.690 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 27.744816439000001, 2024-09-14T06:30:11.690 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.063855705999999998, 2024-09-14T06:30:11.691 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:30:11.691 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:30:11.691 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:30:11.691 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:30:11.691 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:30:11.691 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:11.691 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 44539 2024-09-14T06:30:11.691 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:11.691 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:30:11.691 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:11.691 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:30:11.691 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861462+0000", 2024-09-14T06:30:11.691 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:11.692 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:11.692 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:11.692 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:30:11.692 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861462+0000", 2024-09-14T06:30:11.692 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:11.692 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:11.692 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:11.692 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:30:11.692 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861463+0000", 2024-09-14T06:30:11.692 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.5200000000000002e-07 2024-09-14T06:30:11.692 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:11.692 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:11.692 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:30:11.692 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861465+0000", 2024-09-14T06:30:11.693 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.5189999999999999e-06 2024-09-14T06:30:11.693 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:11.693 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:11.693 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:30:11.693 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861471+0000", 2024-09-14T06:30:11.693 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.6899999999999997e-06 2024-09-14T06:30:11.693 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:11.693 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:11.693 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:30:11.693 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861475+0000", 2024-09-14T06:30:11.693 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.5730000000000001e-06 2024-09-14T06:30:11.693 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:11.693 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:11.693 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:30:11.694 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.925085+0000", 2024-09-14T06:30:11.694 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.063609902999999995 2024-09-14T06:30:11.694 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:11.694 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:11.694 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:30:11.694 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.925145+0000", 2024-09-14T06:30:11.694 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.9383999999999999e-05 2024-09-14T06:30:11.694 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:11.694 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:11.694 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:30:11.694 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.925318+0000", 2024-09-14T06:30:11.694 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00017318499999999999 2024-09-14T06:30:11.694 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:11.694 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:11.695 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:11.695 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:11.695 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:11.695 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:44540 3.2 3:4190d882:::rbd_data.112dca2f6ae1.0000000000000002:17 [read 552960~4096 [fadvise_nocache]] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:30:11.695 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:43.861483+0000", 2024-09-14T06:30:11.695 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 27.744796008000002, 2024-09-14T06:30:11.695 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.064065681999999999, 2024-09-14T06:30:11.695 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:30:11.695 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:30:11.695 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:30:11.695 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:30:11.695 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:30:11.695 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:11.695 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 44540 2024-09-14T06:30:11.695 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:11.696 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:30:11.696 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:11.696 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:30:11.696 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861483+0000", 2024-09-14T06:30:11.696 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:11.696 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:11.696 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:11.696 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:30:11.696 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861483+0000", 2024-09-14T06:30:11.696 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:11.696 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:11.696 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:11.696 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:30:11.696 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.867079+0000", 2024-09-14T06:30:11.697 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0055966480000000001 2024-09-14T06:30:11.697 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:11.697 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:11.697 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:30:11.697 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.867086+0000", 2024-09-14T06:30:11.697 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.387e-06 2024-09-14T06:30:11.697 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:11.697 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:11.697 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:30:11.697 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.867094+0000", 2024-09-14T06:30:11.697 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.9419999999999995e-06 2024-09-14T06:30:11.697 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:11.697 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:11.697 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:30:11.697 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.867109+0000", 2024-09-14T06:30:11.698 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.5818999999999999e-05 2024-09-14T06:30:11.698 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:11.698 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:11.698 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:30:11.698 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.925354+0000", 2024-09-14T06:30:11.698 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.058244047 2024-09-14T06:30:11.698 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:11.698 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:11.698 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:30:11.698 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.925406+0000", 2024-09-14T06:30:11.698 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.2661999999999998e-05 2024-09-14T06:30:11.698 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:11.698 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:11.699 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:30:11.699 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.925548+0000", 2024-09-14T06:30:11.699 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000142177 2024-09-14T06:30:11.699 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:11.699 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:11.699 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:11.699 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:11.699 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:11.699 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-14T06:30:11.700 INFO:teuthology.orchestra.run.smithi094.stdout:2771 punch from 0xbaa7b26 to 0xbaad62b, (0x5b05 bytes) 2024-09-14T06:30:11.700 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-14T06:30:11.701 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-14T06:30:11.703 INFO:teuthology.orchestra.run.smithi094.stdout:2775 write 0xdbbdefe thru 0xdbc433e (0x6441 bytes) 2024-09-14T06:30:11.703 INFO:teuthology.orchestra.run.smithi094.stdout:2777 read 0x7b94439 thru 0x7b9beac (0x7a74 bytes) 2024-09-14T06:30:11.703 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-14T06:30:11.703 INFO:teuthology.orchestra.run.smithi094.stdout:2779 read 0x780e7a0 thru 0x781a9af (0xc210 bytes) 2024-09-14T06:30:11.703 INFO:teuthology.orchestra.run.smithi094.stdout:2780 read 0x724c5f thru 0x72fb71 (0xaf13 bytes) 2024-09-14T06:30:11.703 INFO:teuthology.orchestra.run.smithi094.stdout:2783 write 0x293ad3c thru 0x293ed44 (0x4009 bytes) 2024-09-14T06:30:11.703 INFO:teuthology.orchestra.run.smithi094.stdout:2784 read 0x8891cec thru 0x8892e3f (0x1154 bytes) 2024-09-14T06:30:11.704 INFO:teuthology.orchestra.run.smithi094.stdout:2785 trunc from 0xdbc433f to 0x247e2f0 2024-09-14T06:30:11.704 INFO:teuthology.orchestra.run.smithi094.stdout:2786 write 0x59a85a9 thru 0x59aa3b2 (0x1e0a bytes) 2024-09-14T06:30:11.704 INFO:teuthology.orchestra.run.smithi094.stdout:2787 write 0x51796b thru 0x5210d7 (0x976d bytes) 2024-09-14T06:30:11.704 INFO:teuthology.orchestra.run.smithi094.stdout:2788 write 0xb65abd7 thru 0xb662094 (0x74be bytes) 2024-09-14T06:30:11.704 INFO:teuthology.orchestra.run.smithi094.stdout:2790 write 0xd5d44aa thru 0xd5d7769 (0x32c0 bytes) 2024-09-14T06:30:11.704 INFO:teuthology.orchestra.run.smithi094.stdout:2791 punch from 0x3b79e80 to 0x3b88b85, (0xed05 bytes) 2024-09-14T06:30:11.704 INFO:tasks.ceph.osd.1.smithi094.stderr:2024-09-14T06:30:11.699+0000 7f328e47a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33641) UID: 0 2024-09-14T06:30:11.708 INFO:teuthology.orchestra.run.smithi094.stdout:2792 trunc from 0xd5d776a to 0x63fc9a0 2024-09-14T06:30:11.720 INFO:teuthology.orchestra.run.smithi094.stdout:2793 punch from 0x1e44abb to 0x1e47818, (0x2d5d bytes) 2024-09-14T06:30:11.720 INFO:teuthology.orchestra.run.smithi094.stdout:2794 read 0x4e74365 thru 0x4e79a4c (0x56e8 bytes) 2024-09-14T06:30:11.720 INFO:teuthology.orchestra.run.smithi094.stdout:2796 write 0x5175fcf thru 0x5180fab (0xafdd bytes) 2024-09-14T06:30:11.724 INFO:teuthology.orchestra.run.smithi094.stdout:2797 writesame 0x5b9bddb thru 0x5ba6219 data_size 0x1(0xa43f bytes) 2024-09-14T06:30:11.739 INFO:teuthology.orchestra.run.smithi094.stdout:2799 write 0x60b3aab thru 0x60bd22c (0x9782 bytes) 2024-09-14T06:30:11.749 INFO:teuthology.orchestra.run.smithi094.stdout:2800 trunc from 0x63fc9a0 to 0x11f5a97 2024-09-14T06:30:11.777 INFO:teuthology.orchestra.run.smithi094.stdout:2801 write 0x8f45fe5 thru 0x8f47424 (0x1440 bytes) 2024-09-14T06:30:11.781 INFO:teuthology.orchestra.run.smithi094.stdout:2803 write 0x319c3cb thru 0x31ab02f (0xec65 bytes) 2024-09-14T06:30:11.784 INFO:teuthology.orchestra.run.smithi094.stdout:2804 write 0x38d1147 thru 0x38d42ef (0x31a9 bytes) 2024-09-14T06:30:11.787 INFO:teuthology.orchestra.run.smithi094.stdout:2805 read 0x4f9fe3b thru 0x4fafc9a (0xfe60 bytes) 2024-09-14T06:30:11.798 INFO:teuthology.orchestra.run.smithi094.stdout:2806 write 0xb2966d5 thru 0xb29e8c5 (0x81f1 bytes) 2024-09-14T06:30:11.801 INFO:tasks.ceph.osd.2.smithi094.stderr:2024-09-14T06:30:11.800+0000 7faf3269b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33649) UID: 0 2024-09-14T06:30:11.810 INFO:teuthology.orchestra.run.smithi094.stdout:2807 write 0xd6a0721 thru 0xd6a2f23 (0x2803 bytes) 2024-09-14T06:30:11.813 INFO:teuthology.orchestra.run.smithi138.stdout:{ 2024-09-14T06:30:11.813 INFO:teuthology.orchestra.run.smithi138.stdout: "ops": [ 2024-09-14T06:30:11.813 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:11.813 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:110661 3.14 3:2b439104:::rbd_data.112d92ae6cee.000000000000035b:head [set-alloc-hint object_size 262144 write_size 262144,write 1825~10243 in=10243b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e90)", 2024-09-14T06:30:11.814 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:30:11.811564+0000", 2024-09-14T06:30:11.814 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 0.001176453, 2024-09-14T06:30:11.814 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.001195607, 2024-09-14T06:30:11.814 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:30:11.814 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:30:11.814 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "waiting for sub ops", 2024-09-14T06:30:11.814 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:30:11.814 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:30:11.814 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:11.814 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 110661 2024-09-14T06:30:11.814 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:11.814 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:30:11.814 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:11.814 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:30:11.814 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:11.811564+0000", 2024-09-14T06:30:11.815 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:11.815 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:11.815 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:11.815 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:30:11.815 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:11.811564+0000", 2024-09-14T06:30:11.815 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:11.815 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:11.815 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:11.815 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:30:11.815 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:11.811565+0000", 2024-09-14T06:30:11.815 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.5899999999999996e-07 2024-09-14T06:30:11.815 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:11.815 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:11.816 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:30:11.816 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:11.811579+0000", 2024-09-14T06:30:11.816 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.4208e-05 2024-09-14T06:30:11.816 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:11.816 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:11.816 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:30:11.816 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:11.811587+0000", 2024-09-14T06:30:11.816 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.3820000000000004e-06 2024-09-14T06:30:11.816 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:11.816 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:11.816 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:30:11.816 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:11.811592+0000", 2024-09-14T06:30:11.816 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.9409999999999998e-06 2024-09-14T06:30:11.817 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:11.817 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:11.817 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:30:11.817 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:11.811641+0000", 2024-09-14T06:30:11.817 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.9236000000000001e-05 2024-09-14T06:30:11.817 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:11.817 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:11.817 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:30:11.817 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:11.811775+0000", 2024-09-14T06:30:11.817 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00013341199999999999 2024-09-14T06:30:11.817 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:11.817 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:11.817 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "waiting for subops from 7", 2024-09-14T06:30:11.818 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:11.811926+0000", 2024-09-14T06:30:11.818 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00015130699999999999 2024-09-14T06:30:11.818 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:11.818 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:30:11.818 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:11.818 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:11.818 INFO:teuthology.orchestra.run.smithi138.stdout: ], 2024-09-14T06:30:11.818 INFO:teuthology.orchestra.run.smithi138.stdout: "num_ops": 1 2024-09-14T06:30:11.818 INFO:teuthology.orchestra.run.smithi138.stdout:} 2024-09-14T06:30:11.818 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-14T06:30:11.818 INFO:teuthology.orchestra.run.smithi094.stdout:2810 write 0x4a90d31 thru 0x4a995b7 (0x8887 bytes) 2024-09-14T06:30:11.819 INFO:teuthology.orchestra.run.smithi094.stdout:2811 writesame 0x525017b thru 0x5258428 data_size 0x2b(0x82ae bytes) 2024-09-14T06:30:11.820 INFO:teuthology.orchestra.run.smithi094.stdout:2812 punch from 0x326bff5 to 0x32739fa, (0x7a05 bytes) 2024-09-14T06:30:11.820 INFO:teuthology.orchestra.run.smithi094.stdout:2813 read 0x4e2d779 thru 0x4e2ef34 (0x17bc bytes) 2024-09-14T06:30:11.821 INFO:teuthology.orchestra.run.smithi094.stdout:2814 trunc from 0xd6a2f24 to 0xe718425 2024-09-14T06:30:11.822 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-14T06:30:11.827 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-14T06:30:11.827 INFO:teuthology.orchestra.run.smithi094.stdout:2816 read 0xd4daae8 thru 0xd4e7438 (0xc951 bytes) 2024-09-14T06:30:11.827 INFO:teuthology.orchestra.run.smithi094.stdout:2817 punch from 0xb6ba5f8 to 0xb6baed4, (0x8dc bytes) 2024-09-14T06:30:11.827 INFO:teuthology.orchestra.run.smithi094.stdout:2818 write 0xa314093 thru 0xa320c7b (0xcbe9 bytes) 2024-09-14T06:30:11.830 INFO:teuthology.orchestra.run.smithi094.stdout:2819 read 0x13a4c22 thru 0x13b18cf (0xccae bytes) 2024-09-14T06:30:11.831 INFO:teuthology.orchestra.run.smithi094.stdout:2820 writesame 0x19b4e11 thru 0x19b753b data_size 0x1(0x272b bytes) 2024-09-14T06:30:11.838 INFO:teuthology.orchestra.run.smithi094.stdout:2821 trunc from 0xe718425 to 0xd32b317 2024-09-14T06:30:11.854 INFO:teuthology.orchestra.run.smithi094.stdout:2823 read 0x373107e thru 0x373ec04 (0xdb87 bytes) 2024-09-14T06:30:11.854 INFO:teuthology.orchestra.run.smithi094.stdout:2824 writesame 0x71b56d6 thru 0x71c4ff7 data_size 0x8f(0xf922 bytes) 2024-09-14T06:30:11.860 INFO:teuthology.orchestra.run.smithi094.stdout:2825 punch from 0x61ff81e to 0x6202908, (0x30ea bytes) 2024-09-14T06:30:11.860 INFO:teuthology.orchestra.run.smithi094.stdout:2826 punch from 0x4e28987 to 0x4e2f99e, (0x7017 bytes) 2024-09-14T06:30:11.865 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-14T06:30:11.868 INFO:teuthology.orchestra.run.smithi094.stdout:2827 write 0xd3d54ff thru 0xd3d5731 (0x233 bytes) 2024-09-14T06:30:11.871 INFO:teuthology.orchestra.run.smithi094.stdout:2828 punch from 0x3be664b to 0x3bf0a62, (0xa417 bytes) 2024-09-14T06:30:11.871 INFO:teuthology.orchestra.run.smithi094.stdout:2829 read 0xc997def thru 0xc99f4af (0x76c1 bytes) 2024-09-14T06:30:11.871 INFO:teuthology.orchestra.run.smithi094.stdout:2830 write 0xb20f352 thru 0xb215527 (0x61d6 bytes) 2024-09-14T06:30:11.875 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-14T06:30:11.875 INFO:teuthology.orchestra.run.smithi094.stdout:2832 read 0x3c15d1a thru 0x3c245c1 (0xe8a8 bytes) 2024-09-14T06:30:11.875 INFO:teuthology.orchestra.run.smithi094.stdout:2835 write 0xc651099 thru 0xc6574de (0x6446 bytes) 2024-09-14T06:30:11.879 INFO:teuthology.orchestra.run.smithi094.stdout:2836 writesame 0x34d3d52 thru 0x34df86d data_size 0x32(0xbb1c bytes) 2024-09-14T06:30:11.883 INFO:teuthology.orchestra.run.smithi094.stdout:2839 punch from 0x2ac7000 to 0x2ad09a5, (0x99a5 bytes) 2024-09-14T06:30:11.883 INFO:teuthology.orchestra.run.smithi094.stdout:2840 read 0x4f0da2d thru 0x4f0f706 (0x1cda bytes) 2024-09-14T06:30:11.883 INFO:teuthology.orchestra.run.smithi094.stdout:2841 read 0x47b5e1d thru 0x47c55b0 (0xf794 bytes) 2024-09-14T06:30:11.886 INFO:teuthology.orchestra.run.smithi094.stdout:2842 trunc from 0xd3d5732 to 0xad5bac0 2024-09-14T06:30:11.901 INFO:teuthology.orchestra.run.smithi094.stdout:2843 read 0x7961668 thru 0x7969fc1 (0x895a bytes) 2024-09-14T06:30:11.901 INFO:teuthology.orchestra.run.smithi094.stdout:2846 write 0x884edb1 thru 0x88531d2 (0x4422 bytes) 2024-09-14T06:30:11.902 INFO:tasks.ceph.osd.4.smithi138.stderr:2024-09-14T06:30:11.901+0000 7f6c653c8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33615) UID: 0 2024-09-14T06:30:11.905 INFO:teuthology.orchestra.run.smithi094.stdout:2847 write 0x89c685b thru 0x89d4c41 (0xe3e7 bytes) 2024-09-14T06:30:11.916 INFO:teuthology.orchestra.run.smithi094.stdout:2848 trunc from 0xad5bac0 to 0x405f775 2024-09-14T06:30:11.931 INFO:teuthology.orchestra.run.smithi138.stdout:{ 2024-09-14T06:30:11.931 INFO:teuthology.orchestra.run.smithi138.stdout: "ops": [], 2024-09-14T06:30:11.931 INFO:teuthology.orchestra.run.smithi138.stdout: "complaint_time": 30, 2024-09-14T06:30:11.931 INFO:teuthology.orchestra.run.smithi138.stdout: "num_blocked_ops": 0 2024-09-14T06:30:11.931 INFO:teuthology.orchestra.run.smithi138.stdout:} 2024-09-14T06:30:11.940 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-14T06:30:11.946 INFO:teuthology.orchestra.run.smithi094.stdout:2849 write 0x953ccdc thru 0x9545bda (0x8eff bytes) 2024-09-14T06:30:11.955 INFO:teuthology.orchestra.run.smithi094.stdout:2850 write 0x67fbf26 thru 0x680204e (0x6129 bytes) 2024-09-14T06:30:11.962 INFO:teuthology.orchestra.run.smithi094.stdout:2851 trunc from 0x9545bdb to 0x70827a5 2024-09-14T06:30:11.971 INFO:teuthology.orchestra.run.smithi094.stdout:2853 read 0x282e836 thru 0x282ef0f (0x6da bytes) 2024-09-14T06:30:11.971 INFO:teuthology.orchestra.run.smithi094.stdout:2854 clone 19 order 24 su 8388608 sc 7 2024-09-14T06:30:11.998 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-14T06:30:11.998 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-14T06:30:11.998 INFO:tasks.thrashosds.thrasher:thrash_pg_upmap 2024-09-14T06:30:11.998 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump -f json-pretty 2024-09-14T06:30:12.042 INFO:tasks.ceph.osd.2.smithi094.stderr:2024-09-14T06:30:12.041+0000 7faf3269b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33649) UID: 0 2024-09-14T06:30:12.084 INFO:teuthology.orchestra.run.smithi138.stdout:{ 2024-09-14T06:30:12.084 INFO:teuthology.orchestra.run.smithi138.stdout: "size": 20, 2024-09-14T06:30:12.084 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 600, 2024-09-14T06:30:12.084 INFO:teuthology.orchestra.run.smithi138.stdout: "ops": [ 2024-09-14T06:30:12.084 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:12.085 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:19328 3.3 3:c34e5e51:::rbd_data.112d92027590.0000000000000024:f [read 1926144~1024 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:30:12.085 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:27.059768+0000", 2024-09-14T06:30:12.085 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 45.022751221, 2024-09-14T06:30:12.085 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046369147999999999, 2024-09-14T06:30:12.085 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:30:12.085 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:30:12.085 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:30:12.085 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:30:12.085 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:30:12.085 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:12.085 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 19328 2024-09-14T06:30:12.085 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:12.085 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:30:12.085 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:12.086 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:30:12.086 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.059768+0000", 2024-09-14T06:30:12.086 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:12.086 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:12.086 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:12.086 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:30:12.086 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.059768+0000", 2024-09-14T06:30:12.086 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:12.086 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:12.086 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:12.086 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:30:12.086 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.059874+0000", 2024-09-14T06:30:12.086 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00010551400000000001 2024-09-14T06:30:12.086 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:12.086 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:12.086 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:30:12.087 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.059877+0000", 2024-09-14T06:30:12.087 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.8279999999999999e-06 2024-09-14T06:30:12.087 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:12.087 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:12.087 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:30:12.087 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.059884+0000", 2024-09-14T06:30:12.087 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.6389999999999992e-06 2024-09-14T06:30:12.087 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:12.087 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:12.087 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:30:12.087 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.059890+0000", 2024-09-14T06:30:12.087 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.49e-06 2024-09-14T06:30:12.087 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:12.087 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:12.088 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:30:12.088 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.105804+0000", 2024-09-14T06:30:12.088 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.045914242000000001 2024-09-14T06:30:12.088 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:12.088 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:12.088 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:30:12.088 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.105886+0000", 2024-09-14T06:30:12.088 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.2305999999999994e-05 2024-09-14T06:30:12.088 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:12.088 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:12.088 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:30:12.088 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.106138+0000", 2024-09-14T06:30:12.088 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.000251129 2024-09-14T06:30:12.088 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:12.089 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:30:12.089 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:12.089 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:12.089 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:12.089 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:19329 3.3 3:c34e5e51:::rbd_data.112d92027590.0000000000000024:f [read 1936384~1024 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:30:12.089 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:27.059899+0000", 2024-09-14T06:30:12.089 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 45.022620697999997, 2024-09-14T06:30:12.089 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046572122000000001, 2024-09-14T06:30:12.089 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:30:12.089 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:30:12.089 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:30:12.089 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:30:12.089 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:30:12.089 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:12.090 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 19329 2024-09-14T06:30:12.090 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:12.090 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:30:12.090 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:12.090 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:30:12.090 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.059899+0000", 2024-09-14T06:30:12.090 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:12.090 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:12.090 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:12.090 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:30:12.090 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.059899+0000", 2024-09-14T06:30:12.090 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:12.090 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:12.090 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:12.091 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:30:12.091 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062501+0000", 2024-09-14T06:30:12.091 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.0026016419999999999 2024-09-14T06:30:12.091 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:12.091 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:12.091 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:30:12.091 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062514+0000", 2024-09-14T06:30:12.091 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.2947e-05 2024-09-14T06:30:12.091 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:12.091 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:12.091 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:30:12.091 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062523+0000", 2024-09-14T06:30:12.091 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.5519999999999993e-06 2024-09-14T06:30:12.091 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:12.091 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:12.092 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:30:12.092 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062534+0000", 2024-09-14T06:30:12.092 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.1416e-05 2024-09-14T06:30:12.092 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:12.092 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:12.092 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:30:12.092 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.106179+0000", 2024-09-14T06:30:12.092 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.043644049999999997 2024-09-14T06:30:12.092 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:12.092 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:12.092 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:30:12.092 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.106262+0000", 2024-09-14T06:30:12.092 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.3178999999999998e-05 2024-09-14T06:30:12.093 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:12.093 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:12.093 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:30:12.093 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.106471+0000", 2024-09-14T06:30:12.093 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00020933599999999999 2024-09-14T06:30:12.093 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:12.093 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:30:12.093 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:12.093 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:12.093 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:12.093 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:19351 3.3 3:c34e5e51:::rbd_data.112d92027590.0000000000000024:f [read 1159168~1024 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:30:12.093 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:27.062766+0000", 2024-09-14T06:30:12.093 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 45.019753473000002, 2024-09-14T06:30:12.093 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046099454999999998, 2024-09-14T06:30:12.093 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:30:12.094 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:30:12.094 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:30:12.094 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:30:12.094 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:30:12.094 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:12.094 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 19351 2024-09-14T06:30:12.094 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:12.094 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:30:12.094 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:12.094 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:30:12.094 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062766+0000", 2024-09-14T06:30:12.094 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:12.094 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:12.094 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:12.095 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:30:12.095 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062766+0000", 2024-09-14T06:30:12.095 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:12.095 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:12.095 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:12.095 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:30:12.095 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062766+0000", 2024-09-14T06:30:12.095 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.6e-07 2024-09-14T06:30:12.095 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:12.095 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:12.095 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:30:12.095 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062767+0000", 2024-09-14T06:30:12.095 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.0070000000000001e-06 2024-09-14T06:30:12.095 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:12.095 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:12.096 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:30:12.096 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062785+0000", 2024-09-14T06:30:12.096 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.7884999999999999e-05 2024-09-14T06:30:12.096 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:12.096 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:12.096 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:30:12.096 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062790+0000", 2024-09-14T06:30:12.096 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.0900000000000004e-06 2024-09-14T06:30:12.096 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:12.096 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:12.096 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:30:12.096 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.108735+0000", 2024-09-14T06:30:12.096 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.045944411999999997 2024-09-14T06:30:12.096 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:12.097 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:12.097 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:30:12.097 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.108809+0000", 2024-09-14T06:30:12.097 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.4157000000000002e-05 2024-09-14T06:30:12.097 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:12.097 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:12.097 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:30:12.097 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.108866+0000", 2024-09-14T06:30:12.097 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.6644000000000002e-05 2024-09-14T06:30:12.097 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:12.097 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:30:12.097 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:12.097 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:12.098 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:12.098 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:19352 3.3 3:c34e5e51:::rbd_data.112d92027590.0000000000000024:f [read 1169408~1024 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:30:12.098 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:27.062798+0000", 2024-09-14T06:30:12.098 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 45.019722062, 2024-09-14T06:30:12.098 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046255900000000003, 2024-09-14T06:30:12.098 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:30:12.098 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:30:12.098 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:30:12.098 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:30:12.098 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:30:12.098 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:12.098 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 19352 2024-09-14T06:30:12.098 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:12.098 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:30:12.099 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:12.099 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:30:12.099 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062798+0000", 2024-09-14T06:30:12.099 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:12.099 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:12.099 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:12.099 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:30:12.099 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062798+0000", 2024-09-14T06:30:12.099 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:12.099 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:12.099 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:12.099 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:30:12.099 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062798+0000", 2024-09-14T06:30:12.099 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.37e-07 2024-09-14T06:30:12.100 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:12.100 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:12.100 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:30:12.100 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062800+0000", 2024-09-14T06:30:12.100 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.0269999999999998e-06 2024-09-14T06:30:12.100 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:12.100 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:12.100 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:30:12.100 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062806+0000", 2024-09-14T06:30:12.100 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.6690000000000003e-06 2024-09-14T06:30:12.100 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:12.100 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:12.100 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:30:12.100 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062810+0000", 2024-09-14T06:30:12.101 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.3449999999999999e-06 2024-09-14T06:30:12.101 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:12.101 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:12.101 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:30:12.101 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.108892+0000", 2024-09-14T06:30:12.101 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046081715000000002 2024-09-14T06:30:12.101 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:12.101 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:12.101 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:30:12.101 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.108940+0000", 2024-09-14T06:30:12.101 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.7815999999999998e-05 2024-09-14T06:30:12.101 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:12.101 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:12.102 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:30:12.102 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.109053+0000", 2024-09-14T06:30:12.102 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.000113791 2024-09-14T06:30:12.102 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:12.102 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:30:12.102 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:12.102 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:12.102 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:12.102 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:19353 3.3 3:c34e5e51:::rbd_data.112d92027590.0000000000000024:f [read 1179648~1024 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:30:12.102 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:27.062816+0000", 2024-09-14T06:30:12.102 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 45.019703542999999, 2024-09-14T06:30:12.102 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046481256999999998, 2024-09-14T06:30:12.102 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:30:12.102 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:30:12.103 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:30:12.103 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:30:12.103 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:30:12.103 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:12.103 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 19353 2024-09-14T06:30:12.103 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:12.103 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:30:12.103 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:12.103 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:30:12.103 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062816+0000", 2024-09-14T06:30:12.103 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:12.103 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:12.103 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:12.103 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:30:12.104 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062816+0000", 2024-09-14T06:30:12.104 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:12.104 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:12.104 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:12.104 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:30:12.104 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062816+0000", 2024-09-14T06:30:12.104 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.2899999999999999e-07 2024-09-14T06:30:12.104 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:12.104 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:12.104 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:30:12.104 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062818+0000", 2024-09-14T06:30:12.104 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.699e-06 2024-09-14T06:30:12.104 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:12.105 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:12.105 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:30:12.105 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062825+0000", 2024-09-14T06:30:12.105 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.878e-06 2024-09-14T06:30:12.105 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:12.105 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:12.105 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:30:12.105 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062830+0000", 2024-09-14T06:30:12.105 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.5410000000000002e-06 2024-09-14T06:30:12.105 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:12.105 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:12.105 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:30:12.105 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.109096+0000", 2024-09-14T06:30:12.105 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046266517 2024-09-14T06:30:12.105 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:12.106 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:12.106 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:30:12.106 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.109190+0000", 2024-09-14T06:30:12.106 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.4000999999999997e-05 2024-09-14T06:30:12.106 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:12.106 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:12.106 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:30:12.106 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.109297+0000", 2024-09-14T06:30:12.106 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.000107292 2024-09-14T06:30:12.106 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:12.106 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:30:12.106 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:12.106 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:12.106 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:12.107 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:19354 3.3 3:c34e5e51:::rbd_data.112d92027590.0000000000000024:f [read 1189888~1024 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:30:12.107 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:27.062842+0000", 2024-09-14T06:30:12.107 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 45.019678087999999, 2024-09-14T06:30:12.107 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046594257, 2024-09-14T06:30:12.107 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:30:12.107 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:30:12.107 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:30:12.107 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:30:12.107 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:30:12.107 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:12.107 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 19354 2024-09-14T06:30:12.107 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:12.107 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:30:12.107 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:12.107 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:30:12.108 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062842+0000", 2024-09-14T06:30:12.108 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:12.108 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:12.108 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:12.108 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:30:12.108 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062842+0000", 2024-09-14T06:30:12.108 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:12.108 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:12.108 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:12.108 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:30:12.108 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062842+0000", 2024-09-14T06:30:12.108 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.7399999999999999e-07 2024-09-14T06:30:12.108 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:12.108 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:12.109 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:30:12.109 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062859+0000", 2024-09-14T06:30:12.109 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.7010000000000001e-05 2024-09-14T06:30:12.109 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:12.109 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:12.109 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:30:12.109 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062866+0000", 2024-09-14T06:30:12.109 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.9990000000000002e-06 2024-09-14T06:30:12.109 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:12.109 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:12.109 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:30:12.109 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062873+0000", 2024-09-14T06:30:12.109 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.8569999999999997e-06 2024-09-14T06:30:12.109 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:12.110 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:12.110 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:30:12.110 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.109335+0000", 2024-09-14T06:30:12.110 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046461943999999998 2024-09-14T06:30:12.110 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:12.110 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:12.110 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:30:12.110 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.109377+0000", 2024-09-14T06:30:12.110 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.2558999999999997e-05 2024-09-14T06:30:12.110 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:12.110 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:12.110 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:30:12.110 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.109436+0000", 2024-09-14T06:30:12.111 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.8514000000000003e-05 2024-09-14T06:30:12.111 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:12.111 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:30:12.111 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:12.111 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:12.111 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:12.111 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:19355 3.3 3:c34e5e51:::rbd_data.112d92027590.0000000000000024:f [read 1200128~1024 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:30:12.111 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:27.062884+0000", 2024-09-14T06:30:12.111 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 45.019635415000003, 2024-09-14T06:30:12.111 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046681650999999998, 2024-09-14T06:30:12.111 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:30:12.111 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:30:12.111 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:30:12.111 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:30:12.112 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:30:12.112 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:12.112 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 19355 2024-09-14T06:30:12.112 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:12.112 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:30:12.112 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:12.112 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:30:12.112 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062884+0000", 2024-09-14T06:30:12.112 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:12.112 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:12.112 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:12.112 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:30:12.112 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062884+0000", 2024-09-14T06:30:12.113 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:12.113 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:12.113 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:12.113 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:30:12.113 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062885+0000", 2024-09-14T06:30:12.113 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.8000000000000001e-07 2024-09-14T06:30:12.113 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:12.113 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:12.113 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:30:12.113 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062886+0000", 2024-09-14T06:30:12.113 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.8920000000000001e-06 2024-09-14T06:30:12.113 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:12.113 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:12.114 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:30:12.114 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062892+0000", 2024-09-14T06:30:12.114 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.84e-06 2024-09-14T06:30:12.114 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:12.114 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:12.114 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:30:12.114 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062899+0000", 2024-09-14T06:30:12.114 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.5339999999999996e-06 2024-09-14T06:30:12.114 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:12.114 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:12.114 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:30:12.114 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.109466+0000", 2024-09-14T06:30:12.114 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046566919999999998 2024-09-14T06:30:12.114 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:12.115 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:12.115 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:30:12.115 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.109510+0000", 2024-09-14T06:30:12.115 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.4153000000000002e-05 2024-09-14T06:30:12.115 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:12.115 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:12.115 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:30:12.115 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.109566+0000", 2024-09-14T06:30:12.115 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.5931999999999999e-05 2024-09-14T06:30:12.115 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:12.115 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:30:12.115 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:12.115 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:12.115 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:12.116 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:19356 3.3 3:c34e5e51:::rbd_data.112d92027590.0000000000000024:f [read 1210368~1024 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:30:12.116 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:27.062908+0000", 2024-09-14T06:30:12.116 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 45.019611978, 2024-09-14T06:30:12.116 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046807498000000003, 2024-09-14T06:30:12.116 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:30:12.116 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:30:12.116 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:30:12.116 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:30:12.116 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:30:12.116 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:12.116 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 19356 2024-09-14T06:30:12.116 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:12.116 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:30:12.116 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:12.117 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:30:12.117 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062908+0000", 2024-09-14T06:30:12.117 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:12.117 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:12.117 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:12.117 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:30:12.117 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062908+0000", 2024-09-14T06:30:12.117 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:12.117 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:12.117 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:12.117 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:30:12.117 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062908+0000", 2024-09-14T06:30:12.117 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.5699999999999998e-07 2024-09-14T06:30:12.117 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:12.118 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:12.118 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:30:12.118 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062910+0000", 2024-09-14T06:30:12.118 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.7710000000000001e-06 2024-09-14T06:30:12.118 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:12.118 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:12.118 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:30:12.118 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062920+0000", 2024-09-14T06:30:12.118 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.0654e-05 2024-09-14T06:30:12.118 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:12.118 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:12.118 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:30:12.118 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062931+0000", 2024-09-14T06:30:12.118 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.075e-05 2024-09-14T06:30:12.119 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:12.119 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:12.119 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:30:12.119 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.109595+0000", 2024-09-14T06:30:12.119 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046663713000000002 2024-09-14T06:30:12.119 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:12.119 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:12.119 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:30:12.119 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.109632+0000", 2024-09-14T06:30:12.119 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.7505000000000001e-05 2024-09-14T06:30:12.119 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:12.119 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:12.119 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:30:12.120 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.109715+0000", 2024-09-14T06:30:12.120 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.2748000000000001e-05 2024-09-14T06:30:12.120 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:12.120 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:30:12.120 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:12.120 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:12.120 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:12.120 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:19357 3.3 3:c34e5e51:::rbd_data.112d92027590.0000000000000024:f [read 1220608~1024 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:30:12.120 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:27.062939+0000", 2024-09-14T06:30:12.120 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 45.019580529999999, 2024-09-14T06:30:12.120 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046920508, 2024-09-14T06:30:12.120 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:30:12.120 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:30:12.121 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:30:12.121 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:30:12.121 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:30:12.121 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:12.121 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 19357 2024-09-14T06:30:12.121 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:12.121 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:30:12.121 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:12.121 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:30:12.121 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062939+0000", 2024-09-14T06:30:12.121 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:12.121 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:12.121 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:12.121 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:30:12.122 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062939+0000", 2024-09-14T06:30:12.122 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:12.122 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:12.122 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:12.122 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:30:12.122 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062940+0000", 2024-09-14T06:30:12.122 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.32e-07 2024-09-14T06:30:12.122 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:12.122 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:12.122 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:30:12.122 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062942+0000", 2024-09-14T06:30:12.122 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.1919999999999999e-06 2024-09-14T06:30:12.122 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:12.122 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:12.123 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:30:12.123 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062948+0000", 2024-09-14T06:30:12.123 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.1149999999999999e-06 2024-09-14T06:30:12.123 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:12.123 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:12.123 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:30:12.123 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062960+0000", 2024-09-14T06:30:12.123 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.1895e-05 2024-09-14T06:30:12.123 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:12.123 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:12.123 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:30:12.123 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.109704+0000", 2024-09-14T06:30:12.123 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046744584999999998 2024-09-14T06:30:12.123 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:12.123 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:12.124 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:30:12.124 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.109777+0000", 2024-09-14T06:30:12.124 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.2282999999999999e-05 2024-09-14T06:30:12.124 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:12.124 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:12.124 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:30:12.124 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.109860+0000", 2024-09-14T06:30:12.124 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.3005999999999998e-05 2024-09-14T06:30:12.124 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:12.124 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:30:12.124 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:12.124 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:12.124 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:12.125 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:19358 3.3 3:c34e5e51:::rbd_data.112d92027590.0000000000000024:f [read 1230848~1024 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:30:12.125 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:27.062967+0000", 2024-09-14T06:30:12.125 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 45.019552900000001, 2024-09-14T06:30:12.125 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047144471, 2024-09-14T06:30:12.125 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:30:12.125 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:30:12.125 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:30:12.125 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:30:12.125 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:30:12.125 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:12.125 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 19358 2024-09-14T06:30:12.125 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:12.125 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:30:12.125 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:12.126 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:30:12.126 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062967+0000", 2024-09-14T06:30:12.126 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:12.126 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:12.126 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:12.126 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:30:12.126 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062967+0000", 2024-09-14T06:30:12.126 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:12.126 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:12.126 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:12.126 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:30:12.126 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062967+0000", 2024-09-14T06:30:12.126 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.89e-07 2024-09-14T06:30:12.127 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:12.127 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:12.127 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:30:12.127 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062969+0000", 2024-09-14T06:30:12.127 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.9929999999999998e-06 2024-09-14T06:30:12.127 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:12.127 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:12.127 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:30:12.127 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062975+0000", 2024-09-14T06:30:12.127 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.9030000000000001e-06 2024-09-14T06:30:12.127 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:12.127 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:12.127 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:30:12.128 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062979+0000", 2024-09-14T06:30:12.128 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.2250000000000002e-06 2024-09-14T06:30:12.128 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:12.128 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:12.128 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:30:12.128 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.109934+0000", 2024-09-14T06:30:12.128 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046954410000000002 2024-09-14T06:30:12.128 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:12.128 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:12.128 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:30:12.128 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.110007+0000", 2024-09-14T06:30:12.128 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.3829000000000001e-05 2024-09-14T06:30:12.128 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:12.128 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:12.129 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:30:12.129 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.110111+0000", 2024-09-14T06:30:12.129 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00010372199999999999 2024-09-14T06:30:12.129 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:12.129 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:30:12.129 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:12.129 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:12.129 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:12.129 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:19359 3.3 3:c34e5e51:::rbd_data.112d92027590.0000000000000024:f [read 1241088~1024 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:30:12.129 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:27.062987+0000", 2024-09-14T06:30:12.129 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 45.019532810000001, 2024-09-14T06:30:12.129 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047297182, 2024-09-14T06:30:12.129 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:30:12.129 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:30:12.130 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:30:12.130 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:30:12.130 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:30:12.130 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:12.130 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 19359 2024-09-14T06:30:12.130 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:12.130 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:30:12.130 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:12.130 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:30:12.130 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062987+0000", 2024-09-14T06:30:12.130 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:12.130 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:12.130 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:12.130 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:30:12.131 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062987+0000", 2024-09-14T06:30:12.131 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:12.131 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:12.131 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:12.131 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:30:12.131 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062987+0000", 2024-09-14T06:30:12.131 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.2500000000000001e-07 2024-09-14T06:30:12.131 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:12.131 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:12.131 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:30:12.131 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062989+0000", 2024-09-14T06:30:12.131 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.6080000000000001e-06 2024-09-14T06:30:12.131 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:12.131 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:12.132 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:30:12.132 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062994+0000", 2024-09-14T06:30:12.132 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.5520000000000004e-06 2024-09-14T06:30:12.132 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:12.132 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:12.132 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:30:12.132 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062999+0000", 2024-09-14T06:30:12.132 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.3930000000000001e-06 2024-09-14T06:30:12.132 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:12.132 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:12.132 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:30:12.132 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.110142+0000", 2024-09-14T06:30:12.132 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047143154 2024-09-14T06:30:12.133 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:12.133 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:12.133 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:30:12.133 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.110219+0000", 2024-09-14T06:30:12.133 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.6737999999999996e-05 2024-09-14T06:30:12.133 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:12.133 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:12.133 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:30:12.133 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.110284+0000", 2024-09-14T06:30:12.133 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.5412000000000003e-05 2024-09-14T06:30:12.133 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:12.133 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:30:12.133 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:12.133 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:12.134 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:12.134 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:19360 3.3 3:c34e5e51:::rbd_data.112d92027590.0000000000000024:f [read 1251328~1024 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:30:12.134 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:27.063017+0000", 2024-09-14T06:30:12.134 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 45.019502349, 2024-09-14T06:30:12.134 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047453823999999999, 2024-09-14T06:30:12.134 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:30:12.134 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:30:12.134 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:30:12.134 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:30:12.134 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:30:12.134 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:12.134 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 19360 2024-09-14T06:30:12.134 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:12.134 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:30:12.135 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:12.135 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:30:12.135 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.063017+0000", 2024-09-14T06:30:12.135 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:12.135 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:12.135 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:12.135 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:30:12.135 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.063017+0000", 2024-09-14T06:30:12.135 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:12.135 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:12.135 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:12.135 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:30:12.135 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.064977+0000", 2024-09-14T06:30:12.135 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.0019597080000000001 2024-09-14T06:30:12.136 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:12.136 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:12.136 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:30:12.136 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.064987+0000", 2024-09-14T06:30:12.136 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.9620000000000005e-06 2024-09-14T06:30:12.136 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:12.136 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:12.136 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:30:12.136 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065000+0000", 2024-09-14T06:30:12.136 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.3173000000000001e-05 2024-09-14T06:30:12.136 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:12.136 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:12.136 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:30:12.137 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065005+0000", 2024-09-14T06:30:12.137 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.9960000000000001e-06 2024-09-14T06:30:12.137 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:12.137 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:12.137 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:30:12.137 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.110314+0000", 2024-09-14T06:30:12.137 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.04530899 2024-09-14T06:30:12.137 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:12.137 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:12.137 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:30:12.137 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.110398+0000", 2024-09-14T06:30:12.137 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.3539000000000001e-05 2024-09-14T06:30:12.137 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:12.137 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:12.137 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:30:12.138 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.110471+0000", 2024-09-14T06:30:12.138 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.3455999999999996e-05 2024-09-14T06:30:12.138 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:12.138 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:30:12.138 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:12.138 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:12.138 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:12.138 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:19365 3.3 3:c34e5e51:::rbd_data.112d92027590.0000000000000024:f [read 1302528~1024 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:30:12.138 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:27.065058+0000", 2024-09-14T06:30:12.138 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 45.017461679, 2024-09-14T06:30:12.138 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046105526000000001, 2024-09-14T06:30:12.138 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:30:12.138 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:30:12.138 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:30:12.138 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:30:12.139 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:30:12.139 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:12.139 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 19365 2024-09-14T06:30:12.139 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:12.139 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:30:12.139 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:12.139 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:30:12.139 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065058+0000", 2024-09-14T06:30:12.139 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:12.139 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:12.139 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:12.139 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:30:12.139 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065058+0000", 2024-09-14T06:30:12.139 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:12.139 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:12.140 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:12.140 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:30:12.140 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065058+0000", 2024-09-14T06:30:12.140 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.1899999999999999e-07 2024-09-14T06:30:12.140 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:12.140 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:12.140 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:30:12.140 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065059+0000", 2024-09-14T06:30:12.140 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.1299999999999998e-07 2024-09-14T06:30:12.140 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:12.140 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:12.140 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:30:12.140 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065063+0000", 2024-09-14T06:30:12.141 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.1099999999999996e-06 2024-09-14T06:30:12.141 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:12.141 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:12.141 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:30:12.141 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065070+0000", 2024-09-14T06:30:12.141 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.973e-06 2024-09-14T06:30:12.141 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:12.141 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:12.141 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:30:12.141 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.111014+0000", 2024-09-14T06:30:12.141 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.045943373000000003 2024-09-14T06:30:12.141 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:12.141 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:12.141 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:30:12.142 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.111073+0000", 2024-09-14T06:30:12.142 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.8987999999999998e-05 2024-09-14T06:30:12.142 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:12.142 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:12.142 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:30:12.142 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.111163+0000", 2024-09-14T06:30:12.142 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.0950000000000002e-05 2024-09-14T06:30:12.142 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:12.142 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:30:12.142 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:12.142 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:12.142 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:12.142 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:19366 3.3 3:c34e5e51:::rbd_data.112d92027590.0000000000000024:f [read 1312768~1024 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:30:12.143 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:27.065074+0000", 2024-09-14T06:30:12.143 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 45.017445829000003, 2024-09-14T06:30:12.143 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046289858000000003, 2024-09-14T06:30:12.143 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:30:12.143 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:30:12.143 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:30:12.143 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:30:12.143 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:30:12.143 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:12.143 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 19366 2024-09-14T06:30:12.143 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:12.143 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:30:12.143 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:12.143 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:30:12.143 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065074+0000", 2024-09-14T06:30:12.144 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:12.144 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:12.144 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:12.144 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:30:12.144 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065074+0000", 2024-09-14T06:30:12.144 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:12.144 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:12.144 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:12.144 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:30:12.144 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065074+0000", 2024-09-14T06:30:12.144 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.3900000000000001e-07 2024-09-14T06:30:12.144 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:12.144 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:12.145 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:30:12.145 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065078+0000", 2024-09-14T06:30:12.145 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.3229999999999999e-06 2024-09-14T06:30:12.145 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:12.145 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:12.145 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:30:12.145 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065082+0000", 2024-09-14T06:30:12.145 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.8299999999999998e-06 2024-09-14T06:30:12.145 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:12.145 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:12.145 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:30:12.145 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065085+0000", 2024-09-14T06:30:12.145 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.8660000000000002e-06 2024-09-14T06:30:12.145 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:12.146 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:12.146 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:30:12.146 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.111270+0000", 2024-09-14T06:30:12.146 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046185035999999999 2024-09-14T06:30:12.146 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:12.146 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:12.146 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:30:12.146 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.111311+0000", 2024-09-14T06:30:12.146 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.0935e-05 2024-09-14T06:30:12.146 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:12.146 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:12.146 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:30:12.146 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.111364+0000", 2024-09-14T06:30:12.147 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.2629e-05 2024-09-14T06:30:12.147 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:12.147 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:30:12.147 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:12.147 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:12.147 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:12.147 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:19367 3.3 3:c34e5e51:::rbd_data.112d92027590.0000000000000024:f [read 1323008~1024 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:30:12.147 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:27.065115+0000", 2024-09-14T06:30:12.147 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 45.017404268, 2024-09-14T06:30:12.147 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046407003000000002, 2024-09-14T06:30:12.147 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:30:12.147 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:30:12.147 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:30:12.147 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:30:12.148 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:30:12.148 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:12.148 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 19367 2024-09-14T06:30:12.148 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:12.148 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:30:12.148 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:12.148 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:30:12.148 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065115+0000", 2024-09-14T06:30:12.148 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:12.148 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:12.148 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:12.148 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:30:12.148 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065115+0000", 2024-09-14T06:30:12.149 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:12.149 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:12.149 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:12.149 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:30:12.149 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065116+0000", 2024-09-14T06:30:12.149 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.4999999999999998e-07 2024-09-14T06:30:12.149 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:12.149 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:12.149 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:30:12.149 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065117+0000", 2024-09-14T06:30:12.149 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.189e-06 2024-09-14T06:30:12.149 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:12.149 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:12.149 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:30:12.150 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065122+0000", 2024-09-14T06:30:12.150 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.3709999999999999e-06 2024-09-14T06:30:12.150 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:12.150 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:12.150 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:30:12.150 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065125+0000", 2024-09-14T06:30:12.150 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.8940000000000002e-06 2024-09-14T06:30:12.150 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:12.150 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:12.150 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:30:12.150 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.111402+0000", 2024-09-14T06:30:12.150 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046276645999999998 2024-09-14T06:30:12.150 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:12.150 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:12.151 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:30:12.151 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.111445+0000", 2024-09-14T06:30:12.151 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.3288000000000003e-05 2024-09-14T06:30:12.151 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:12.151 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:12.151 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:30:12.151 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.111522+0000", 2024-09-14T06:30:12.151 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.7164999999999997e-05 2024-09-14T06:30:12.151 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:12.151 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:30:12.151 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:12.151 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:12.151 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:12.151 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:19368 3.3 3:c34e5e51:::rbd_data.112d92027590.0000000000000024:f [read 1333248~1024 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:30:12.152 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:27.065130+0000", 2024-09-14T06:30:12.152 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 45.017389977999997, 2024-09-14T06:30:12.152 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046507275000000001, 2024-09-14T06:30:12.152 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:30:12.152 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:30:12.152 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:30:12.152 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:30:12.152 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:30:12.152 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:12.152 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 19368 2024-09-14T06:30:12.152 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:12.152 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:30:12.152 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:12.153 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:30:12.153 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065130+0000", 2024-09-14T06:30:12.153 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:12.153 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:12.153 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:12.153 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:30:12.153 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065130+0000", 2024-09-14T06:30:12.153 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:12.153 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:12.153 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:12.153 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:30:12.153 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065130+0000", 2024-09-14T06:30:12.153 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.0800000000000001e-07 2024-09-14T06:30:12.154 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:12.154 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:12.154 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:30:12.154 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065131+0000", 2024-09-14T06:30:12.154 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.8700000000000004e-07 2024-09-14T06:30:12.154 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:12.154 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:12.154 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:30:12.154 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065134+0000", 2024-09-14T06:30:12.154 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.1039999999999998e-06 2024-09-14T06:30:12.154 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:12.154 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:12.154 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:30:12.154 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065136+0000", 2024-09-14T06:30:12.155 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.3949999999999999e-06 2024-09-14T06:30:12.155 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:12.155 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:12.155 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:30:12.155 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.111541+0000", 2024-09-14T06:30:12.155 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046404206000000003 2024-09-14T06:30:12.155 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:12.155 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:12.155 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:30:12.155 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.111582+0000", 2024-09-14T06:30:12.155 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.1051999999999999e-05 2024-09-14T06:30:12.155 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:12.155 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:12.155 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:30:12.156 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.111637+0000", 2024-09-14T06:30:12.156 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.5322999999999999e-05 2024-09-14T06:30:12.156 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:12.156 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:30:12.156 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:12.156 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:12.156 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:12.156 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:19369 3.3 3:c34e5e51:::rbd_data.112d92027590.0000000000000024:f [read 1343488~1024 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:30:12.156 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:27.065139+0000", 2024-09-14T06:30:12.156 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 45.017380160999998, 2024-09-14T06:30:12.156 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046682169000000003, 2024-09-14T06:30:12.156 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:30:12.157 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:30:12.157 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:30:12.157 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:30:12.157 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:30:12.157 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:12.157 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 19369 2024-09-14T06:30:12.157 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:12.157 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:30:12.157 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:12.157 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:30:12.157 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065139+0000", 2024-09-14T06:30:12.157 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:12.157 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:12.158 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:12.158 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:30:12.158 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065139+0000", 2024-09-14T06:30:12.158 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:12.158 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:12.158 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:12.158 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:30:12.158 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065140+0000", 2024-09-14T06:30:12.158 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.8299999999999998e-07 2024-09-14T06:30:12.158 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:12.158 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:12.158 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:30:12.158 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065141+0000", 2024-09-14T06:30:12.158 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.3900000000000003e-07 2024-09-14T06:30:12.158 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:12.159 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:12.159 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:30:12.159 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065144+0000", 2024-09-14T06:30:12.159 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.7409999999999998e-06 2024-09-14T06:30:12.159 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:12.159 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:12.159 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:30:12.159 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065147+0000", 2024-09-14T06:30:12.159 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.4150000000000002e-06 2024-09-14T06:30:12.159 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:12.159 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:12.159 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:30:12.159 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.111673+0000", 2024-09-14T06:30:12.159 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046526004000000003 2024-09-14T06:30:12.160 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:12.160 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:12.160 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:30:12.160 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.111749+0000", 2024-09-14T06:30:12.160 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.5741999999999994e-05 2024-09-14T06:30:12.160 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:12.160 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:12.160 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:30:12.160 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.111822+0000", 2024-09-14T06:30:12.160 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.3045000000000003e-05 2024-09-14T06:30:12.160 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:12.160 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:30:12.160 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:12.160 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:12.160 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:12.161 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:19370 3.3 3:c34e5e51:::rbd_data.112d92027590.0000000000000024:f [read 1353728~1024 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:30:12.161 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:27.065150+0000", 2024-09-14T06:30:12.161 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 45.017369432000002, 2024-09-14T06:30:12.161 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046897556999999999, 2024-09-14T06:30:12.161 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:30:12.161 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:30:12.161 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:30:12.161 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:30:12.161 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:30:12.161 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:12.161 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 19370 2024-09-14T06:30:12.161 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:12.161 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:30:12.161 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:12.162 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:30:12.162 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065150+0000", 2024-09-14T06:30:12.162 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:12.162 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:12.162 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:12.162 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:30:12.162 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065150+0000", 2024-09-14T06:30:12.162 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:12.162 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:12.162 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:12.162 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:30:12.162 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065150+0000", 2024-09-14T06:30:12.162 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.0200000000000001e-07 2024-09-14T06:30:12.163 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:12.163 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:12.163 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:30:12.163 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065151+0000", 2024-09-14T06:30:12.163 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.1100000000000004e-07 2024-09-14T06:30:12.163 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:12.163 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:12.163 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:30:12.163 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065154+0000", 2024-09-14T06:30:12.163 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.9639999999999999e-06 2024-09-14T06:30:12.163 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:12.163 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:12.163 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:30:12.163 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065157+0000", 2024-09-14T06:30:12.163 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.4260000000000002e-06 2024-09-14T06:30:12.164 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:12.164 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:12.164 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:30:12.164 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.111861+0000", 2024-09-14T06:30:12.164 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046704268 2024-09-14T06:30:12.164 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:12.164 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:12.164 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:30:12.164 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.111939+0000", 2024-09-14T06:30:12.164 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.8231000000000004e-05 2024-09-14T06:30:12.164 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:12.164 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:12.164 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:30:12.164 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.112048+0000", 2024-09-14T06:30:12.164 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00010855500000000001 2024-09-14T06:30:12.165 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:12.165 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:30:12.165 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:12.165 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:12.165 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:12.165 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:19371 3.3 3:c34e5e51:::rbd_data.112d92027590.0000000000000024:f [read 1363968~1024 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:30:12.165 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:27.065161+0000", 2024-09-14T06:30:12.165 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 45.017358782000002, 2024-09-14T06:30:12.165 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047021944000000003, 2024-09-14T06:30:12.165 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:30:12.165 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:30:12.165 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:30:12.165 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:30:12.166 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:30:12.166 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:12.166 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 19371 2024-09-14T06:30:12.166 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:12.166 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:30:12.166 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:12.166 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:30:12.166 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065161+0000", 2024-09-14T06:30:12.166 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:12.166 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:12.166 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:12.166 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:30:12.166 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065161+0000", 2024-09-14T06:30:12.166 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:12.167 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:12.167 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:12.167 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:30:12.167 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065161+0000", 2024-09-14T06:30:12.167 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.0200000000000001e-07 2024-09-14T06:30:12.167 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:12.167 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:12.167 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:30:12.167 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065166+0000", 2024-09-14T06:30:12.167 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.9869999999999999e-06 2024-09-14T06:30:12.167 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:12.167 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:12.167 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:30:12.167 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065169+0000", 2024-09-14T06:30:12.167 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.3799999999999998e-06 2024-09-14T06:30:12.168 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:12.168 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:12.168 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:30:12.168 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065172+0000", 2024-09-14T06:30:12.168 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.4830000000000002e-06 2024-09-14T06:30:12.168 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:12.168 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:12.168 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:30:12.168 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.112071+0000", 2024-09-14T06:30:12.168 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046899529000000002 2024-09-14T06:30:12.168 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:12.168 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:12.168 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:30:12.168 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.112128+0000", 2024-09-14T06:30:12.169 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.6341999999999997e-05 2024-09-14T06:30:12.169 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:12.169 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:12.169 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:30:12.169 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.112183+0000", 2024-09-14T06:30:12.169 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.5021000000000001e-05 2024-09-14T06:30:12.169 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:12.169 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:30:12.169 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:12.169 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:12.169 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:12.169 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:19372 3.3 3:c34e5e51:::rbd_data.112d92027590.0000000000000024:f [read 1374208~1024 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:30:12.169 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:27.065176+0000", 2024-09-14T06:30:12.170 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 45.017343631999999, 2024-09-14T06:30:12.170 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047188889999999997, 2024-09-14T06:30:12.170 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:30:12.170 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:30:12.170 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:30:12.170 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:30:12.170 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:30:12.170 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:12.170 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 19372 2024-09-14T06:30:12.170 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:12.170 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:30:12.170 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:12.170 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:30:12.170 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065176+0000", 2024-09-14T06:30:12.170 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:12.171 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:12.171 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:12.171 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:30:12.171 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065176+0000", 2024-09-14T06:30:12.171 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:12.171 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:12.171 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:12.171 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:30:12.171 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.071117+0000", 2024-09-14T06:30:12.171 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.0059404990000000001 2024-09-14T06:30:12.171 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:12.171 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:12.171 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:30:12.172 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.071120+0000", 2024-09-14T06:30:12.172 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.4340000000000001e-06 2024-09-14T06:30:12.172 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:12.172 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:12.172 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:30:12.172 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.071129+0000", 2024-09-14T06:30:12.172 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.8389999999999996e-06 2024-09-14T06:30:12.172 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:12.172 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:12.172 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:30:12.172 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.071134+0000", 2024-09-14T06:30:12.172 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.412e-06 2024-09-14T06:30:12.172 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:12.173 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:12.173 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:30:12.173 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.112221+0000", 2024-09-14T06:30:12.173 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.041086473999999998 2024-09-14T06:30:12.173 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:12.173 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:12.173 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:30:12.173 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.112299+0000", 2024-09-14T06:30:12.173 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.8746999999999997e-05 2024-09-14T06:30:12.173 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:12.173 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:12.173 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:30:12.173 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.112365+0000", 2024-09-14T06:30:12.173 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.5485000000000001e-05 2024-09-14T06:30:12.173 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:12.174 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:30:12.174 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:12.174 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:12.174 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:30:12.174 INFO:teuthology.orchestra.run.smithi138.stdout:} 2024-09-14T06:30:12.175 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-14T06:30:12.176 INFO:tasks.ceph.osd.3.smithi094.stderr:2024-09-14T06:30:12.174+0000 7feeeafe8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33655) UID: 0 2024-09-14T06:30:12.276 INFO:tasks.ceph.osd.7.smithi138.stderr:2024-09-14T06:30:12.275+0000 7f3c6a21d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33623) UID: 0 2024-09-14T06:30:12.278 INFO:teuthology.orchestra.run.smithi138.stdout:{ 2024-09-14T06:30:12.278 INFO:teuthology.orchestra.run.smithi138.stdout: "ops": [], 2024-09-14T06:30:12.278 INFO:teuthology.orchestra.run.smithi138.stdout: "num_ops": 0 2024-09-14T06:30:12.278 INFO:teuthology.orchestra.run.smithi138.stdout:} 2024-09-14T06:30:12.287 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-14T06:30:12.293 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-14T06:30:12.293 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-14T06:30:12.293 INFO:teuthology.orchestra.run.smithi094.stdout: "epoch": 90, 2024-09-14T06:30:12.293 INFO:teuthology.orchestra.run.smithi094.stdout: "fsid": "200ad3da-bd63-4e04-af74-efb5ca0af75e", 2024-09-14T06:30:12.293 INFO:teuthology.orchestra.run.smithi094.stdout: "created": "2024-09-14T06:28:30.782046+0000", 2024-09-14T06:30:12.293 INFO:teuthology.orchestra.run.smithi094.stdout: "modified": "2024-09-14T06:30:10.579472+0000", 2024-09-14T06:30:12.293 INFO:teuthology.orchestra.run.smithi094.stdout: "last_up_change": "2024-09-14T06:28:42.827239+0000", 2024-09-14T06:30:12.293 INFO:teuthology.orchestra.run.smithi094.stdout: "last_in_change": "2024-09-14T06:28:33.724176+0000", 2024-09-14T06:30:12.294 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2024-09-14T06:30:12.294 INFO:teuthology.orchestra.run.smithi094.stdout: "flags_num": 5799936, 2024-09-14T06:30:12.294 INFO:teuthology.orchestra.run.smithi094.stdout: "flags_set": [ 2024-09-14T06:30:12.294 INFO:teuthology.orchestra.run.smithi094.stdout: "pglog_hardlimit", 2024-09-14T06:30:12.294 INFO:teuthology.orchestra.run.smithi094.stdout: "purged_snapdirs", 2024-09-14T06:30:12.294 INFO:teuthology.orchestra.run.smithi094.stdout: "recovery_deletes", 2024-09-14T06:30:12.294 INFO:teuthology.orchestra.run.smithi094.stdout: "sortbitwise" 2024-09-14T06:30:12.294 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-09-14T06:30:12.294 INFO:teuthology.orchestra.run.smithi094.stdout: "crush_version": 5, 2024-09-14T06:30:12.294 INFO:teuthology.orchestra.run.smithi094.stdout: "full_ratio": 0.94999998807907104, 2024-09-14T06:30:12.294 INFO:teuthology.orchestra.run.smithi094.stdout: "backfillfull_ratio": 0.89999997615814209, 2024-09-14T06:30:12.294 INFO:teuthology.orchestra.run.smithi094.stdout: "nearfull_ratio": 0.85000002384185791, 2024-09-14T06:30:12.295 INFO:teuthology.orchestra.run.smithi094.stdout: "cluster_snapshot": "", 2024-09-14T06:30:12.295 INFO:teuthology.orchestra.run.smithi094.stdout: "pool_max": 3, 2024-09-14T06:30:12.295 INFO:teuthology.orchestra.run.smithi094.stdout: "max_osd": 8, 2024-09-14T06:30:12.295 INFO:teuthology.orchestra.run.smithi094.stdout: "require_min_compat_client": "luminous", 2024-09-14T06:30:12.295 INFO:teuthology.orchestra.run.smithi094.stdout: "min_compat_client": "luminous", 2024-09-14T06:30:12.295 INFO:teuthology.orchestra.run.smithi094.stdout: "require_osd_release": "squid", 2024-09-14T06:30:12.295 INFO:teuthology.orchestra.run.smithi094.stdout: "allow_crimson": false, 2024-09-14T06:30:12.295 INFO:teuthology.orchestra.run.smithi094.stdout: "pools": [ 2024-09-14T06:30:12.295 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:12.295 INFO:teuthology.orchestra.run.smithi094.stdout: "pool": 1, 2024-09-14T06:30:12.296 INFO:teuthology.orchestra.run.smithi094.stdout: "pool_name": ".mgr", 2024-09-14T06:30:12.296 INFO:teuthology.orchestra.run.smithi094.stdout: "create_time": "2024-09-14T06:28:43.295990+0000", 2024-09-14T06:30:12.296 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-14T06:30:12.296 INFO:teuthology.orchestra.run.smithi094.stdout: "flags_names": "hashpspool", 2024-09-14T06:30:12.296 INFO:teuthology.orchestra.run.smithi094.stdout: "type": 1, 2024-09-14T06:30:12.296 INFO:teuthology.orchestra.run.smithi094.stdout: "size": 2, 2024-09-14T06:30:12.296 INFO:teuthology.orchestra.run.smithi094.stdout: "min_size": 1, 2024-09-14T06:30:12.296 INFO:teuthology.orchestra.run.smithi094.stdout: "crush_rule": 0, 2024-09-14T06:30:12.296 INFO:teuthology.orchestra.run.smithi094.stdout: "peering_crush_bucket_count": 0, 2024-09-14T06:30:12.296 INFO:teuthology.orchestra.run.smithi094.stdout: "peering_crush_bucket_target": 0, 2024-09-14T06:30:12.296 INFO:teuthology.orchestra.run.smithi094.stdout: "peering_crush_bucket_barrier": 0, 2024-09-14T06:30:12.296 INFO:teuthology.orchestra.run.smithi094.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-14T06:30:12.297 INFO:teuthology.orchestra.run.smithi094.stdout: "is_stretch_pool": false, 2024-09-14T06:30:12.297 INFO:teuthology.orchestra.run.smithi094.stdout: "object_hash": 2, 2024-09-14T06:30:12.297 INFO:teuthology.orchestra.run.smithi094.stdout: "pg_autoscale_mode": "off", 2024-09-14T06:30:12.297 INFO:teuthology.orchestra.run.smithi094.stdout: "pg_num": 1, 2024-09-14T06:30:12.297 INFO:teuthology.orchestra.run.smithi094.stdout: "pg_placement_num": 1, 2024-09-14T06:30:12.297 INFO:teuthology.orchestra.run.smithi094.stdout: "pg_placement_num_target": 1, 2024-09-14T06:30:12.297 INFO:teuthology.orchestra.run.smithi094.stdout: "pg_num_target": 1, 2024-09-14T06:30:12.297 INFO:teuthology.orchestra.run.smithi094.stdout: "pg_num_pending": 1, 2024-09-14T06:30:12.297 INFO:teuthology.orchestra.run.smithi094.stdout: "last_pg_merge_meta": { 2024-09-14T06:30:12.297 INFO:teuthology.orchestra.run.smithi094.stdout: "source_pgid": "0.0", 2024-09-14T06:30:12.297 INFO:teuthology.orchestra.run.smithi094.stdout: "ready_epoch": 0, 2024-09-14T06:30:12.297 INFO:teuthology.orchestra.run.smithi094.stdout: "last_epoch_started": 0, 2024-09-14T06:30:12.297 INFO:teuthology.orchestra.run.smithi094.stdout: "last_epoch_clean": 0, 2024-09-14T06:30:12.297 INFO:teuthology.orchestra.run.smithi094.stdout: "source_version": "0'0", 2024-09-14T06:30:12.298 INFO:teuthology.orchestra.run.smithi094.stdout: "target_version": "0'0" 2024-09-14T06:30:12.298 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:12.298 INFO:teuthology.orchestra.run.smithi094.stdout: "last_change": "18", 2024-09-14T06:30:12.298 INFO:teuthology.orchestra.run.smithi094.stdout: "last_force_op_resend": "0", 2024-09-14T06:30:12.298 INFO:teuthology.orchestra.run.smithi094.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-14T06:30:12.298 INFO:teuthology.orchestra.run.smithi094.stdout: "last_force_op_resend_preluminous": "0", 2024-09-14T06:30:12.298 INFO:teuthology.orchestra.run.smithi094.stdout: "auid": 0, 2024-09-14T06:30:12.298 INFO:teuthology.orchestra.run.smithi094.stdout: "snap_mode": "selfmanaged", 2024-09-14T06:30:12.298 INFO:teuthology.orchestra.run.smithi094.stdout: "snap_seq": 0, 2024-09-14T06:30:12.298 INFO:teuthology.orchestra.run.smithi094.stdout: "snap_epoch": 0, 2024-09-14T06:30:12.298 INFO:teuthology.orchestra.run.smithi094.stdout: "pool_snaps": [], 2024-09-14T06:30:12.298 INFO:teuthology.orchestra.run.smithi094.stdout: "removed_snaps": "[]", 2024-09-14T06:30:12.298 INFO:teuthology.orchestra.run.smithi094.stdout: "quota_max_bytes": 0, 2024-09-14T06:30:12.299 INFO:teuthology.orchestra.run.smithi094.stdout: "quota_max_objects": 0, 2024-09-14T06:30:12.299 INFO:teuthology.orchestra.run.smithi094.stdout: "tiers": [], 2024-09-14T06:30:12.299 INFO:teuthology.orchestra.run.smithi094.stdout: "tier_of": -1, 2024-09-14T06:30:12.299 INFO:teuthology.orchestra.run.smithi094.stdout: "read_tier": -1, 2024-09-14T06:30:12.299 INFO:teuthology.orchestra.run.smithi094.stdout: "write_tier": -1, 2024-09-14T06:30:12.299 INFO:teuthology.orchestra.run.smithi094.stdout: "cache_mode": "none", 2024-09-14T06:30:12.299 INFO:teuthology.orchestra.run.smithi094.stdout: "target_max_bytes": 0, 2024-09-14T06:30:12.299 INFO:teuthology.orchestra.run.smithi094.stdout: "target_max_objects": 0, 2024-09-14T06:30:12.299 INFO:teuthology.orchestra.run.smithi094.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-14T06:30:12.299 INFO:teuthology.orchestra.run.smithi094.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-14T06:30:12.299 INFO:teuthology.orchestra.run.smithi094.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-14T06:30:12.299 INFO:teuthology.orchestra.run.smithi094.stdout: "cache_min_flush_age": 0, 2024-09-14T06:30:12.299 INFO:teuthology.orchestra.run.smithi094.stdout: "cache_min_evict_age": 0, 2024-09-14T06:30:12.299 INFO:teuthology.orchestra.run.smithi094.stdout: "erasure_code_profile": "", 2024-09-14T06:30:12.300 INFO:teuthology.orchestra.run.smithi094.stdout: "hit_set_params": { 2024-09-14T06:30:12.300 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "none" 2024-09-14T06:30:12.300 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:12.300 INFO:teuthology.orchestra.run.smithi094.stdout: "hit_set_period": 0, 2024-09-14T06:30:12.300 INFO:teuthology.orchestra.run.smithi094.stdout: "hit_set_count": 0, 2024-09-14T06:30:12.300 INFO:teuthology.orchestra.run.smithi094.stdout: "use_gmt_hitset": true, 2024-09-14T06:30:12.300 INFO:teuthology.orchestra.run.smithi094.stdout: "min_read_recency_for_promote": 0, 2024-09-14T06:30:12.300 INFO:teuthology.orchestra.run.smithi094.stdout: "min_write_recency_for_promote": 0, 2024-09-14T06:30:12.300 INFO:teuthology.orchestra.run.smithi094.stdout: "hit_set_grade_decay_rate": 0, 2024-09-14T06:30:12.300 INFO:teuthology.orchestra.run.smithi094.stdout: "hit_set_search_last_n": 0, 2024-09-14T06:30:12.301 INFO:teuthology.orchestra.run.smithi094.stdout: "grade_table": [], 2024-09-14T06:30:12.301 INFO:teuthology.orchestra.run.smithi094.stdout: "stripe_width": 0, 2024-09-14T06:30:12.301 INFO:teuthology.orchestra.run.smithi094.stdout: "expected_num_objects": 0, 2024-09-14T06:30:12.301 INFO:teuthology.orchestra.run.smithi094.stdout: "fast_read": false, 2024-09-14T06:30:12.301 INFO:teuthology.orchestra.run.smithi094.stdout: "options": { 2024-09-14T06:30:12.301 INFO:teuthology.orchestra.run.smithi094.stdout: "pg_num_max": 32, 2024-09-14T06:30:12.301 INFO:teuthology.orchestra.run.smithi094.stdout: "pg_num_min": 1 2024-09-14T06:30:12.301 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:12.301 INFO:teuthology.orchestra.run.smithi094.stdout: "application_metadata": { 2024-09-14T06:30:12.301 INFO:teuthology.orchestra.run.smithi094.stdout: "mgr": {} 2024-09-14T06:30:12.301 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:12.301 INFO:teuthology.orchestra.run.smithi094.stdout: "read_balance": { 2024-09-14T06:30:12.302 INFO:teuthology.orchestra.run.smithi094.stdout: "score_type": "Fair distribution", 2024-09-14T06:30:12.302 INFO:teuthology.orchestra.run.smithi094.stdout: "score_acting": 8, 2024-09-14T06:30:12.302 INFO:teuthology.orchestra.run.smithi094.stdout: "score_stable": 8, 2024-09-14T06:30:12.302 INFO:teuthology.orchestra.run.smithi094.stdout: "optimal_score": 0.25, 2024-09-14T06:30:12.302 INFO:teuthology.orchestra.run.smithi094.stdout: "raw_score_acting": 2, 2024-09-14T06:30:12.302 INFO:teuthology.orchestra.run.smithi094.stdout: "raw_score_stable": 2, 2024-09-14T06:30:12.302 INFO:teuthology.orchestra.run.smithi094.stdout: "primary_affinity_weighted": 1, 2024-09-14T06:30:12.302 INFO:teuthology.orchestra.run.smithi094.stdout: "average_primary_affinity": 1, 2024-09-14T06:30:12.302 INFO:teuthology.orchestra.run.smithi094.stdout: "average_primary_affinity_weighted": 1 2024-09-14T06:30:12.302 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:12.302 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:12.302 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:12.302 INFO:teuthology.orchestra.run.smithi094.stdout: "pool": 2, 2024-09-14T06:30:12.302 INFO:teuthology.orchestra.run.smithi094.stdout: "pool_name": "rbd", 2024-09-14T06:30:12.303 INFO:teuthology.orchestra.run.smithi094.stdout: "create_time": "2024-09-14T06:28:49.451714+0000", 2024-09-14T06:30:12.303 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 8193, 2024-09-14T06:30:12.303 INFO:teuthology.orchestra.run.smithi094.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-09-14T06:30:12.303 INFO:teuthology.orchestra.run.smithi094.stdout: "type": 1, 2024-09-14T06:30:12.303 INFO:teuthology.orchestra.run.smithi094.stdout: "size": 2, 2024-09-14T06:30:12.303 INFO:teuthology.orchestra.run.smithi094.stdout: "min_size": 1, 2024-09-14T06:30:12.303 INFO:teuthology.orchestra.run.smithi094.stdout: "crush_rule": 0, 2024-09-14T06:30:12.303 INFO:teuthology.orchestra.run.smithi094.stdout: "peering_crush_bucket_count": 0, 2024-09-14T06:30:12.303 INFO:teuthology.orchestra.run.smithi094.stdout: "peering_crush_bucket_target": 0, 2024-09-14T06:30:12.303 INFO:teuthology.orchestra.run.smithi094.stdout: "peering_crush_bucket_barrier": 0, 2024-09-14T06:30:12.303 INFO:teuthology.orchestra.run.smithi094.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-14T06:30:12.303 INFO:teuthology.orchestra.run.smithi094.stdout: "is_stretch_pool": false, 2024-09-14T06:30:12.303 INFO:teuthology.orchestra.run.smithi094.stdout: "object_hash": 2, 2024-09-14T06:30:12.303 INFO:teuthology.orchestra.run.smithi094.stdout: "pg_autoscale_mode": "off", 2024-09-14T06:30:12.304 INFO:teuthology.orchestra.run.smithi094.stdout: "pg_num": 8, 2024-09-14T06:30:12.304 INFO:teuthology.orchestra.run.smithi094.stdout: "pg_placement_num": 8, 2024-09-14T06:30:12.304 INFO:teuthology.orchestra.run.smithi094.stdout: "pg_placement_num_target": 8, 2024-09-14T06:30:12.304 INFO:teuthology.orchestra.run.smithi094.stdout: "pg_num_target": 8, 2024-09-14T06:30:12.304 INFO:teuthology.orchestra.run.smithi094.stdout: "pg_num_pending": 8, 2024-09-14T06:30:12.304 INFO:teuthology.orchestra.run.smithi094.stdout: "last_pg_merge_meta": { 2024-09-14T06:30:12.304 INFO:teuthology.orchestra.run.smithi094.stdout: "source_pgid": "0.0", 2024-09-14T06:30:12.304 INFO:teuthology.orchestra.run.smithi094.stdout: "ready_epoch": 0, 2024-09-14T06:30:12.304 INFO:teuthology.orchestra.run.smithi094.stdout: "last_epoch_started": 0, 2024-09-14T06:30:12.304 INFO:teuthology.orchestra.run.smithi094.stdout: "last_epoch_clean": 0, 2024-09-14T06:30:12.304 INFO:teuthology.orchestra.run.smithi094.stdout: "source_version": "0'0", 2024-09-14T06:30:12.304 INFO:teuthology.orchestra.run.smithi094.stdout: "target_version": "0'0" 2024-09-14T06:30:12.304 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:12.305 INFO:teuthology.orchestra.run.smithi094.stdout: "last_change": "22", 2024-09-14T06:30:12.305 INFO:teuthology.orchestra.run.smithi094.stdout: "last_force_op_resend": "0", 2024-09-14T06:30:12.305 INFO:teuthology.orchestra.run.smithi094.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-14T06:30:12.305 INFO:teuthology.orchestra.run.smithi094.stdout: "last_force_op_resend_preluminous": "0", 2024-09-14T06:30:12.305 INFO:teuthology.orchestra.run.smithi094.stdout: "auid": 0, 2024-09-14T06:30:12.305 INFO:teuthology.orchestra.run.smithi094.stdout: "snap_mode": "selfmanaged", 2024-09-14T06:30:12.305 INFO:teuthology.orchestra.run.smithi094.stdout: "snap_seq": 2, 2024-09-14T06:30:12.305 INFO:teuthology.orchestra.run.smithi094.stdout: "snap_epoch": 22, 2024-09-14T06:30:12.305 INFO:teuthology.orchestra.run.smithi094.stdout: "pool_snaps": [], 2024-09-14T06:30:12.305 INFO:teuthology.orchestra.run.smithi094.stdout: "removed_snaps": "[]", 2024-09-14T06:30:12.305 INFO:teuthology.orchestra.run.smithi094.stdout: "quota_max_bytes": 0, 2024-09-14T06:30:12.305 INFO:teuthology.orchestra.run.smithi094.stdout: "quota_max_objects": 0, 2024-09-14T06:30:12.305 INFO:teuthology.orchestra.run.smithi094.stdout: "tiers": [], 2024-09-14T06:30:12.306 INFO:teuthology.orchestra.run.smithi094.stdout: "tier_of": -1, 2024-09-14T06:30:12.306 INFO:teuthology.orchestra.run.smithi094.stdout: "read_tier": -1, 2024-09-14T06:30:12.306 INFO:teuthology.orchestra.run.smithi094.stdout: "write_tier": -1, 2024-09-14T06:30:12.306 INFO:teuthology.orchestra.run.smithi094.stdout: "cache_mode": "none", 2024-09-14T06:30:12.306 INFO:teuthology.orchestra.run.smithi094.stdout: "target_max_bytes": 0, 2024-09-14T06:30:12.306 INFO:teuthology.orchestra.run.smithi094.stdout: "target_max_objects": 0, 2024-09-14T06:30:12.306 INFO:teuthology.orchestra.run.smithi094.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-14T06:30:12.306 INFO:teuthology.orchestra.run.smithi094.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-14T06:30:12.306 INFO:teuthology.orchestra.run.smithi094.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-14T06:30:12.306 INFO:teuthology.orchestra.run.smithi094.stdout: "cache_min_flush_age": 0, 2024-09-14T06:30:12.306 INFO:teuthology.orchestra.run.smithi094.stdout: "cache_min_evict_age": 0, 2024-09-14T06:30:12.306 INFO:teuthology.orchestra.run.smithi094.stdout: "erasure_code_profile": "", 2024-09-14T06:30:12.306 INFO:teuthology.orchestra.run.smithi094.stdout: "hit_set_params": { 2024-09-14T06:30:12.306 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "none" 2024-09-14T06:30:12.307 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:12.307 INFO:teuthology.orchestra.run.smithi094.stdout: "hit_set_period": 0, 2024-09-14T06:30:12.307 INFO:teuthology.orchestra.run.smithi094.stdout: "hit_set_count": 0, 2024-09-14T06:30:12.307 INFO:teuthology.orchestra.run.smithi094.stdout: "use_gmt_hitset": true, 2024-09-14T06:30:12.307 INFO:teuthology.orchestra.run.smithi094.stdout: "min_read_recency_for_promote": 0, 2024-09-14T06:30:12.307 INFO:teuthology.orchestra.run.smithi094.stdout: "min_write_recency_for_promote": 0, 2024-09-14T06:30:12.307 INFO:teuthology.orchestra.run.smithi094.stdout: "hit_set_grade_decay_rate": 0, 2024-09-14T06:30:12.307 INFO:teuthology.orchestra.run.smithi094.stdout: "hit_set_search_last_n": 0, 2024-09-14T06:30:12.307 INFO:teuthology.orchestra.run.smithi094.stdout: "grade_table": [], 2024-09-14T06:30:12.307 INFO:teuthology.orchestra.run.smithi094.stdout: "stripe_width": 0, 2024-09-14T06:30:12.307 INFO:teuthology.orchestra.run.smithi094.stdout: "expected_num_objects": 0, 2024-09-14T06:30:12.307 INFO:teuthology.orchestra.run.smithi094.stdout: "fast_read": false, 2024-09-14T06:30:12.307 INFO:teuthology.orchestra.run.smithi094.stdout: "options": {}, 2024-09-14T06:30:12.307 INFO:teuthology.orchestra.run.smithi094.stdout: "application_metadata": { 2024-09-14T06:30:12.308 INFO:teuthology.orchestra.run.smithi094.stdout: "rbd": {} 2024-09-14T06:30:12.308 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:12.308 INFO:teuthology.orchestra.run.smithi094.stdout: "read_balance": { 2024-09-14T06:30:12.308 INFO:teuthology.orchestra.run.smithi094.stdout: "score_type": "Fair distribution", 2024-09-14T06:30:12.308 INFO:teuthology.orchestra.run.smithi094.stdout: "score_acting": 1.9900000095367432, 2024-09-14T06:30:12.308 INFO:teuthology.orchestra.run.smithi094.stdout: "score_stable": 1.9900000095367432, 2024-09-14T06:30:12.308 INFO:teuthology.orchestra.run.smithi094.stdout: "optimal_score": 0.87999999523162842, 2024-09-14T06:30:12.308 INFO:teuthology.orchestra.run.smithi094.stdout: "raw_score_acting": 1.75, 2024-09-14T06:30:12.308 INFO:teuthology.orchestra.run.smithi094.stdout: "raw_score_stable": 1.75, 2024-09-14T06:30:12.308 INFO:teuthology.orchestra.run.smithi094.stdout: "primary_affinity_weighted": 1, 2024-09-14T06:30:12.308 INFO:teuthology.orchestra.run.smithi094.stdout: "average_primary_affinity": 1, 2024-09-14T06:30:12.308 INFO:teuthology.orchestra.run.smithi094.stdout: "average_primary_affinity_weighted": 1 2024-09-14T06:30:12.308 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:12.309 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:12.309 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:12.309 INFO:teuthology.orchestra.run.smithi094.stdout: "pool": 3, 2024-09-14T06:30:12.309 INFO:teuthology.orchestra.run.smithi094.stdout: "pool_name": "pool_client.0", 2024-09-14T06:30:12.309 INFO:teuthology.orchestra.run.smithi094.stdout: "create_time": "2024-09-14T06:28:59.688535+0000", 2024-09-14T06:30:12.309 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 8193, 2024-09-14T06:30:12.309 INFO:teuthology.orchestra.run.smithi094.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-09-14T06:30:12.309 INFO:teuthology.orchestra.run.smithi094.stdout: "type": 1, 2024-09-14T06:30:12.309 INFO:teuthology.orchestra.run.smithi094.stdout: "size": 2, 2024-09-14T06:30:12.309 INFO:teuthology.orchestra.run.smithi094.stdout: "min_size": 1, 2024-09-14T06:30:12.309 INFO:teuthology.orchestra.run.smithi094.stdout: "crush_rule": 0, 2024-09-14T06:30:12.309 INFO:teuthology.orchestra.run.smithi094.stdout: "peering_crush_bucket_count": 0, 2024-09-14T06:30:12.309 INFO:teuthology.orchestra.run.smithi094.stdout: "peering_crush_bucket_target": 0, 2024-09-14T06:30:12.310 INFO:teuthology.orchestra.run.smithi094.stdout: "peering_crush_bucket_barrier": 0, 2024-09-14T06:30:12.310 INFO:teuthology.orchestra.run.smithi094.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-14T06:30:12.310 INFO:teuthology.orchestra.run.smithi094.stdout: "is_stretch_pool": false, 2024-09-14T06:30:12.310 INFO:teuthology.orchestra.run.smithi094.stdout: "object_hash": 2, 2024-09-14T06:30:12.310 INFO:teuthology.orchestra.run.smithi094.stdout: "pg_autoscale_mode": "off", 2024-09-14T06:30:12.310 INFO:teuthology.orchestra.run.smithi094.stdout: "pg_num": 32, 2024-09-14T06:30:12.310 INFO:teuthology.orchestra.run.smithi094.stdout: "pg_placement_num": 32, 2024-09-14T06:30:12.310 INFO:teuthology.orchestra.run.smithi094.stdout: "pg_placement_num_target": 32, 2024-09-14T06:30:12.310 INFO:teuthology.orchestra.run.smithi094.stdout: "pg_num_target": 32, 2024-09-14T06:30:12.310 INFO:teuthology.orchestra.run.smithi094.stdout: "pg_num_pending": 32, 2024-09-14T06:30:12.310 INFO:teuthology.orchestra.run.smithi094.stdout: "last_pg_merge_meta": { 2024-09-14T06:30:12.310 INFO:teuthology.orchestra.run.smithi094.stdout: "source_pgid": "0.0", 2024-09-14T06:30:12.310 INFO:teuthology.orchestra.run.smithi094.stdout: "ready_epoch": 0, 2024-09-14T06:30:12.311 INFO:teuthology.orchestra.run.smithi094.stdout: "last_epoch_started": 0, 2024-09-14T06:30:12.311 INFO:teuthology.orchestra.run.smithi094.stdout: "last_epoch_clean": 0, 2024-09-14T06:30:12.311 INFO:teuthology.orchestra.run.smithi094.stdout: "source_version": "0'0", 2024-09-14T06:30:12.311 INFO:teuthology.orchestra.run.smithi094.stdout: "target_version": "0'0" 2024-09-14T06:30:12.311 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:12.311 INFO:teuthology.orchestra.run.smithi094.stdout: "last_change": "90", 2024-09-14T06:30:12.311 INFO:teuthology.orchestra.run.smithi094.stdout: "last_force_op_resend": "0", 2024-09-14T06:30:12.311 INFO:teuthology.orchestra.run.smithi094.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-14T06:30:12.311 INFO:teuthology.orchestra.run.smithi094.stdout: "last_force_op_resend_preluminous": "0", 2024-09-14T06:30:12.311 INFO:teuthology.orchestra.run.smithi094.stdout: "auid": 0, 2024-09-14T06:30:12.311 INFO:teuthology.orchestra.run.smithi094.stdout: "snap_mode": "selfmanaged", 2024-09-14T06:30:12.311 INFO:teuthology.orchestra.run.smithi094.stdout: "snap_seq": 38, 2024-09-14T06:30:12.311 INFO:teuthology.orchestra.run.smithi094.stdout: "snap_epoch": 90, 2024-09-14T06:30:12.312 INFO:teuthology.orchestra.run.smithi094.stdout: "pool_snaps": [], 2024-09-14T06:30:12.312 INFO:teuthology.orchestra.run.smithi094.stdout: "removed_snaps": "[]", 2024-09-14T06:30:12.312 INFO:teuthology.orchestra.run.smithi094.stdout: "quota_max_bytes": 0, 2024-09-14T06:30:12.312 INFO:teuthology.orchestra.run.smithi094.stdout: "quota_max_objects": 0, 2024-09-14T06:30:12.312 INFO:teuthology.orchestra.run.smithi094.stdout: "tiers": [], 2024-09-14T06:30:12.312 INFO:teuthology.orchestra.run.smithi094.stdout: "tier_of": -1, 2024-09-14T06:30:12.312 INFO:teuthology.orchestra.run.smithi094.stdout: "read_tier": -1, 2024-09-14T06:30:12.312 INFO:teuthology.orchestra.run.smithi094.stdout: "write_tier": -1, 2024-09-14T06:30:12.312 INFO:teuthology.orchestra.run.smithi094.stdout: "cache_mode": "none", 2024-09-14T06:30:12.312 INFO:teuthology.orchestra.run.smithi094.stdout: "target_max_bytes": 0, 2024-09-14T06:30:12.312 INFO:teuthology.orchestra.run.smithi094.stdout: "target_max_objects": 0, 2024-09-14T06:30:12.312 INFO:teuthology.orchestra.run.smithi094.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-14T06:30:12.312 INFO:teuthology.orchestra.run.smithi094.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-14T06:30:12.313 INFO:teuthology.orchestra.run.smithi094.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-14T06:30:12.313 INFO:teuthology.orchestra.run.smithi094.stdout: "cache_min_flush_age": 0, 2024-09-14T06:30:12.313 INFO:teuthology.orchestra.run.smithi094.stdout: "cache_min_evict_age": 0, 2024-09-14T06:30:12.313 INFO:teuthology.orchestra.run.smithi094.stdout: "erasure_code_profile": "", 2024-09-14T06:30:12.313 INFO:teuthology.orchestra.run.smithi094.stdout: "hit_set_params": { 2024-09-14T06:30:12.313 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "none" 2024-09-14T06:30:12.313 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:12.313 INFO:teuthology.orchestra.run.smithi094.stdout: "hit_set_period": 0, 2024-09-14T06:30:12.313 INFO:teuthology.orchestra.run.smithi094.stdout: "hit_set_count": 0, 2024-09-14T06:30:12.313 INFO:teuthology.orchestra.run.smithi094.stdout: "use_gmt_hitset": true, 2024-09-14T06:30:12.313 INFO:teuthology.orchestra.run.smithi094.stdout: "min_read_recency_for_promote": 0, 2024-09-14T06:30:12.313 INFO:teuthology.orchestra.run.smithi094.stdout: "min_write_recency_for_promote": 0, 2024-09-14T06:30:12.313 INFO:teuthology.orchestra.run.smithi094.stdout: "hit_set_grade_decay_rate": 0, 2024-09-14T06:30:12.314 INFO:teuthology.orchestra.run.smithi094.stdout: "hit_set_search_last_n": 0, 2024-09-14T06:30:12.314 INFO:teuthology.orchestra.run.smithi094.stdout: "grade_table": [], 2024-09-14T06:30:12.314 INFO:teuthology.orchestra.run.smithi094.stdout: "stripe_width": 0, 2024-09-14T06:30:12.314 INFO:teuthology.orchestra.run.smithi094.stdout: "expected_num_objects": 0, 2024-09-14T06:30:12.314 INFO:teuthology.orchestra.run.smithi094.stdout: "fast_read": false, 2024-09-14T06:30:12.314 INFO:teuthology.orchestra.run.smithi094.stdout: "options": {}, 2024-09-14T06:30:12.314 INFO:teuthology.orchestra.run.smithi094.stdout: "application_metadata": { 2024-09-14T06:30:12.314 INFO:teuthology.orchestra.run.smithi094.stdout: "rbd": {} 2024-09-14T06:30:12.314 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:12.314 INFO:teuthology.orchestra.run.smithi094.stdout: "read_balance": { 2024-09-14T06:30:12.314 INFO:teuthology.orchestra.run.smithi094.stdout: "score_type": "Fair distribution", 2024-09-14T06:30:12.314 INFO:teuthology.orchestra.run.smithi094.stdout: "score_acting": 1.75, 2024-09-14T06:30:12.314 INFO:teuthology.orchestra.run.smithi094.stdout: "score_stable": 1.75, 2024-09-14T06:30:12.315 INFO:teuthology.orchestra.run.smithi094.stdout: "optimal_score": 1, 2024-09-14T06:30:12.315 INFO:teuthology.orchestra.run.smithi094.stdout: "raw_score_acting": 1.75, 2024-09-14T06:30:12.315 INFO:teuthology.orchestra.run.smithi094.stdout: "raw_score_stable": 1.75, 2024-09-14T06:30:12.315 INFO:teuthology.orchestra.run.smithi094.stdout: "primary_affinity_weighted": 1, 2024-09-14T06:30:12.315 INFO:teuthology.orchestra.run.smithi094.stdout: "average_primary_affinity": 1, 2024-09-14T06:30:12.315 INFO:teuthology.orchestra.run.smithi094.stdout: "average_primary_affinity_weighted": 1 2024-09-14T06:30:12.315 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:12.315 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:12.315 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-09-14T06:30:12.315 INFO:teuthology.orchestra.run.smithi094.stdout: "osds": [ 2024-09-14T06:30:12.315 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:12.315 INFO:teuthology.orchestra.run.smithi094.stdout: "osd": 0, 2024-09-14T06:30:12.315 INFO:teuthology.orchestra.run.smithi094.stdout: "uuid": "8b22a249-b842-406a-a02c-c0f31f57b635", 2024-09-14T06:30:12.315 INFO:teuthology.orchestra.run.smithi094.stdout: "up": 1, 2024-09-14T06:30:12.316 INFO:teuthology.orchestra.run.smithi094.stdout: "in": 1, 2024-09-14T06:30:12.316 INFO:teuthology.orchestra.run.smithi094.stdout: "weight": 1, 2024-09-14T06:30:12.316 INFO:teuthology.orchestra.run.smithi094.stdout: "primary_affinity": 1, 2024-09-14T06:30:12.316 INFO:teuthology.orchestra.run.smithi094.stdout: "last_clean_begin": 0, 2024-09-14T06:30:12.316 INFO:teuthology.orchestra.run.smithi094.stdout: "last_clean_end": 0, 2024-09-14T06:30:12.316 INFO:teuthology.orchestra.run.smithi094.stdout: "up_from": 14, 2024-09-14T06:30:12.316 INFO:teuthology.orchestra.run.smithi094.stdout: "up_thru": 75, 2024-09-14T06:30:12.316 INFO:teuthology.orchestra.run.smithi094.stdout: "down_at": 0, 2024-09-14T06:30:12.316 INFO:teuthology.orchestra.run.smithi094.stdout: "lost_at": 0, 2024-09-14T06:30:12.316 INFO:teuthology.orchestra.run.smithi094.stdout: "public_addrs": { 2024-09-14T06:30:12.316 INFO:teuthology.orchestra.run.smithi094.stdout: "addrvec": [ 2024-09-14T06:30:12.316 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:12.316 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v2", 2024-09-14T06:30:12.317 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.94:6824", 2024-09-14T06:30:12.317 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 1809564923 2024-09-14T06:30:12.317 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:12.317 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:12.317 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v1", 2024-09-14T06:30:12.317 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.94:6825", 2024-09-14T06:30:12.317 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 1809564923 2024-09-14T06:30:12.317 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:12.317 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:12.317 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:12.317 INFO:teuthology.orchestra.run.smithi094.stdout: "cluster_addrs": { 2024-09-14T06:30:12.317 INFO:teuthology.orchestra.run.smithi094.stdout: "addrvec": [ 2024-09-14T06:30:12.317 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:12.318 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v2", 2024-09-14T06:30:12.318 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.94:6826", 2024-09-14T06:30:12.318 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 1809564923 2024-09-14T06:30:12.318 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:12.318 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:12.318 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v1", 2024-09-14T06:30:12.318 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.94:6827", 2024-09-14T06:30:12.318 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 1809564923 2024-09-14T06:30:12.318 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:12.318 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:12.318 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:12.318 INFO:teuthology.orchestra.run.smithi094.stdout: "heartbeat_back_addrs": { 2024-09-14T06:30:12.318 INFO:teuthology.orchestra.run.smithi094.stdout: "addrvec": [ 2024-09-14T06:30:12.318 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:12.319 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v2", 2024-09-14T06:30:12.319 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.94:6830", 2024-09-14T06:30:12.319 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 1809564923 2024-09-14T06:30:12.319 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:12.319 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:12.319 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v1", 2024-09-14T06:30:12.319 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.94:6831", 2024-09-14T06:30:12.319 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 1809564923 2024-09-14T06:30:12.319 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:12.319 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:12.319 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:12.319 INFO:teuthology.orchestra.run.smithi094.stdout: "heartbeat_front_addrs": { 2024-09-14T06:30:12.319 INFO:teuthology.orchestra.run.smithi094.stdout: "addrvec": [ 2024-09-14T06:30:12.320 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:12.320 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v2", 2024-09-14T06:30:12.320 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.94:6828", 2024-09-14T06:30:12.320 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 1809564923 2024-09-14T06:30:12.320 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:12.320 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:12.320 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v1", 2024-09-14T06:30:12.320 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.94:6829", 2024-09-14T06:30:12.320 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 1809564923 2024-09-14T06:30:12.320 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:12.320 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:12.320 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:12.320 INFO:teuthology.orchestra.run.smithi094.stdout: "public_addr": "172.21.15.94:6825/1809564923", 2024-09-14T06:30:12.320 INFO:teuthology.orchestra.run.smithi094.stdout: "cluster_addr": "172.21.15.94:6827/1809564923", 2024-09-14T06:30:12.321 INFO:teuthology.orchestra.run.smithi094.stdout: "heartbeat_back_addr": "172.21.15.94:6831/1809564923", 2024-09-14T06:30:12.321 INFO:teuthology.orchestra.run.smithi094.stdout: "heartbeat_front_addr": "172.21.15.94:6829/1809564923", 2024-09-14T06:30:12.321 INFO:teuthology.orchestra.run.smithi094.stdout: "state": [ 2024-09-14T06:30:12.321 INFO:teuthology.orchestra.run.smithi094.stdout: "exists", 2024-09-14T06:30:12.321 INFO:teuthology.orchestra.run.smithi094.stdout: "up" 2024-09-14T06:30:12.321 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:12.321 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:12.321 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:12.321 INFO:teuthology.orchestra.run.smithi094.stdout: "osd": 1, 2024-09-14T06:30:12.321 INFO:teuthology.orchestra.run.smithi094.stdout: "uuid": "02251a03-84a0-414d-a980-fac4fd043cb6", 2024-09-14T06:30:12.321 INFO:teuthology.orchestra.run.smithi094.stdout: "up": 1, 2024-09-14T06:30:12.321 INFO:teuthology.orchestra.run.smithi094.stdout: "in": 1, 2024-09-14T06:30:12.321 INFO:teuthology.orchestra.run.smithi094.stdout: "weight": 1, 2024-09-14T06:30:12.321 INFO:teuthology.orchestra.run.smithi094.stdout: "primary_affinity": 1, 2024-09-14T06:30:12.322 INFO:teuthology.orchestra.run.smithi094.stdout: "last_clean_begin": 0, 2024-09-14T06:30:12.322 INFO:teuthology.orchestra.run.smithi094.stdout: "last_clean_end": 0, 2024-09-14T06:30:12.322 INFO:teuthology.orchestra.run.smithi094.stdout: "up_from": 15, 2024-09-14T06:30:12.322 INFO:teuthology.orchestra.run.smithi094.stdout: "up_thru": 23, 2024-09-14T06:30:12.322 INFO:teuthology.orchestra.run.smithi094.stdout: "down_at": 0, 2024-09-14T06:30:12.322 INFO:teuthology.orchestra.run.smithi094.stdout: "lost_at": 0, 2024-09-14T06:30:12.322 INFO:teuthology.orchestra.run.smithi094.stdout: "public_addrs": { 2024-09-14T06:30:12.322 INFO:teuthology.orchestra.run.smithi094.stdout: "addrvec": [ 2024-09-14T06:30:12.322 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:12.322 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v2", 2024-09-14T06:30:12.322 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.94:6816", 2024-09-14T06:30:12.322 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 736402652 2024-09-14T06:30:12.322 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:12.323 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:12.323 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v1", 2024-09-14T06:30:12.323 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.94:6817", 2024-09-14T06:30:12.323 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 736402652 2024-09-14T06:30:12.323 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:12.323 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:12.323 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:12.323 INFO:teuthology.orchestra.run.smithi094.stdout: "cluster_addrs": { 2024-09-14T06:30:12.323 INFO:teuthology.orchestra.run.smithi094.stdout: "addrvec": [ 2024-09-14T06:30:12.323 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:12.323 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v2", 2024-09-14T06:30:12.323 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.94:6818", 2024-09-14T06:30:12.323 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 736402652 2024-09-14T06:30:12.323 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:12.324 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:12.324 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v1", 2024-09-14T06:30:12.324 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.94:6819", 2024-09-14T06:30:12.324 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 736402652 2024-09-14T06:30:12.324 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:12.324 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:12.324 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:12.324 INFO:teuthology.orchestra.run.smithi094.stdout: "heartbeat_back_addrs": { 2024-09-14T06:30:12.324 INFO:teuthology.orchestra.run.smithi094.stdout: "addrvec": [ 2024-09-14T06:30:12.324 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:12.324 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v2", 2024-09-14T06:30:12.324 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.94:6822", 2024-09-14T06:30:12.324 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 736402652 2024-09-14T06:30:12.325 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:12.325 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:12.325 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v1", 2024-09-14T06:30:12.325 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.94:6823", 2024-09-14T06:30:12.325 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 736402652 2024-09-14T06:30:12.325 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:12.325 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:12.325 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:12.325 INFO:teuthology.orchestra.run.smithi094.stdout: "heartbeat_front_addrs": { 2024-09-14T06:30:12.325 INFO:teuthology.orchestra.run.smithi094.stdout: "addrvec": [ 2024-09-14T06:30:12.325 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:12.325 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v2", 2024-09-14T06:30:12.325 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.94:6820", 2024-09-14T06:30:12.326 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 736402652 2024-09-14T06:30:12.326 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:12.326 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:12.326 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v1", 2024-09-14T06:30:12.326 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.94:6821", 2024-09-14T06:30:12.326 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 736402652 2024-09-14T06:30:12.326 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:12.326 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:12.326 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:12.326 INFO:teuthology.orchestra.run.smithi094.stdout: "public_addr": "172.21.15.94:6817/736402652", 2024-09-14T06:30:12.326 INFO:teuthology.orchestra.run.smithi094.stdout: "cluster_addr": "172.21.15.94:6819/736402652", 2024-09-14T06:30:12.326 INFO:teuthology.orchestra.run.smithi094.stdout: "heartbeat_back_addr": "172.21.15.94:6823/736402652", 2024-09-14T06:30:12.326 INFO:teuthology.orchestra.run.smithi094.stdout: "heartbeat_front_addr": "172.21.15.94:6821/736402652", 2024-09-14T06:30:12.326 INFO:teuthology.orchestra.run.smithi094.stdout: "state": [ 2024-09-14T06:30:12.326 INFO:teuthology.orchestra.run.smithi094.stdout: "exists", 2024-09-14T06:30:12.327 INFO:teuthology.orchestra.run.smithi094.stdout: "up" 2024-09-14T06:30:12.327 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:12.327 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:12.327 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:12.327 INFO:teuthology.orchestra.run.smithi094.stdout: "osd": 2, 2024-09-14T06:30:12.327 INFO:teuthology.orchestra.run.smithi094.stdout: "uuid": "d45bf470-bb29-4739-8294-cd122b2b4102", 2024-09-14T06:30:12.327 INFO:teuthology.orchestra.run.smithi094.stdout: "up": 1, 2024-09-14T06:30:12.327 INFO:teuthology.orchestra.run.smithi094.stdout: "in": 1, 2024-09-14T06:30:12.327 INFO:teuthology.orchestra.run.smithi094.stdout: "weight": 1, 2024-09-14T06:30:12.327 INFO:teuthology.orchestra.run.smithi094.stdout: "primary_affinity": 1, 2024-09-14T06:30:12.327 INFO:teuthology.orchestra.run.smithi094.stdout: "last_clean_begin": 0, 2024-09-14T06:30:12.327 INFO:teuthology.orchestra.run.smithi094.stdout: "last_clean_end": 0, 2024-09-14T06:30:12.327 INFO:teuthology.orchestra.run.smithi094.stdout: "up_from": 14, 2024-09-14T06:30:12.328 INFO:teuthology.orchestra.run.smithi094.stdout: "up_thru": 19, 2024-09-14T06:30:12.328 INFO:teuthology.orchestra.run.smithi094.stdout: "down_at": 0, 2024-09-14T06:30:12.328 INFO:teuthology.orchestra.run.smithi094.stdout: "lost_at": 0, 2024-09-14T06:30:12.328 INFO:teuthology.orchestra.run.smithi094.stdout: "public_addrs": { 2024-09-14T06:30:12.328 INFO:teuthology.orchestra.run.smithi094.stdout: "addrvec": [ 2024-09-14T06:30:12.328 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:12.328 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v2", 2024-09-14T06:30:12.328 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.94:6808", 2024-09-14T06:30:12.328 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 1063204782 2024-09-14T06:30:12.328 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:12.328 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:12.328 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v1", 2024-09-14T06:30:12.328 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.94:6809", 2024-09-14T06:30:12.328 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 1063204782 2024-09-14T06:30:12.329 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:12.329 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:12.329 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:12.329 INFO:teuthology.orchestra.run.smithi094.stdout: "cluster_addrs": { 2024-09-14T06:30:12.329 INFO:teuthology.orchestra.run.smithi094.stdout: "addrvec": [ 2024-09-14T06:30:12.329 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:12.329 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v2", 2024-09-14T06:30:12.329 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.94:6810", 2024-09-14T06:30:12.329 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 1063204782 2024-09-14T06:30:12.329 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:12.329 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:12.329 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v1", 2024-09-14T06:30:12.329 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.94:6811", 2024-09-14T06:30:12.330 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 1063204782 2024-09-14T06:30:12.330 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:12.330 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:12.330 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:12.330 INFO:teuthology.orchestra.run.smithi094.stdout: "heartbeat_back_addrs": { 2024-09-14T06:30:12.330 INFO:teuthology.orchestra.run.smithi094.stdout: "addrvec": [ 2024-09-14T06:30:12.330 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:12.330 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v2", 2024-09-14T06:30:12.330 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.94:6814", 2024-09-14T06:30:12.330 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 1063204782 2024-09-14T06:30:12.330 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:12.330 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:12.330 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v1", 2024-09-14T06:30:12.330 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.94:6815", 2024-09-14T06:30:12.331 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 1063204782 2024-09-14T06:30:12.331 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:12.331 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:12.331 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:12.331 INFO:teuthology.orchestra.run.smithi094.stdout: "heartbeat_front_addrs": { 2024-09-14T06:30:12.331 INFO:teuthology.orchestra.run.smithi094.stdout: "addrvec": [ 2024-09-14T06:30:12.331 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:12.331 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v2", 2024-09-14T06:30:12.331 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.94:6812", 2024-09-14T06:30:12.331 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 1063204782 2024-09-14T06:30:12.331 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:12.331 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:12.331 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v1", 2024-09-14T06:30:12.331 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.94:6813", 2024-09-14T06:30:12.332 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 1063204782 2024-09-14T06:30:12.332 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:12.332 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:12.332 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:12.332 INFO:teuthology.orchestra.run.smithi094.stdout: "public_addr": "172.21.15.94:6809/1063204782", 2024-09-14T06:30:12.332 INFO:teuthology.orchestra.run.smithi094.stdout: "cluster_addr": "172.21.15.94:6811/1063204782", 2024-09-14T06:30:12.332 INFO:teuthology.orchestra.run.smithi094.stdout: "heartbeat_back_addr": "172.21.15.94:6815/1063204782", 2024-09-14T06:30:12.332 INFO:teuthology.orchestra.run.smithi094.stdout: "heartbeat_front_addr": "172.21.15.94:6813/1063204782", 2024-09-14T06:30:12.332 INFO:teuthology.orchestra.run.smithi094.stdout: "state": [ 2024-09-14T06:30:12.332 INFO:teuthology.orchestra.run.smithi094.stdout: "exists", 2024-09-14T06:30:12.332 INFO:teuthology.orchestra.run.smithi094.stdout: "up" 2024-09-14T06:30:12.332 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:12.332 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:12.333 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:12.333 INFO:teuthology.orchestra.run.smithi094.stdout: "osd": 3, 2024-09-14T06:30:12.333 INFO:teuthology.orchestra.run.smithi094.stdout: "uuid": "456ec2d9-c438-4243-b931-335704f661eb", 2024-09-14T06:30:12.333 INFO:teuthology.orchestra.run.smithi094.stdout: "up": 1, 2024-09-14T06:30:12.333 INFO:teuthology.orchestra.run.smithi094.stdout: "in": 1, 2024-09-14T06:30:12.333 INFO:teuthology.orchestra.run.smithi094.stdout: "weight": 1, 2024-09-14T06:30:12.333 INFO:teuthology.orchestra.run.smithi094.stdout: "primary_affinity": 1, 2024-09-14T06:30:12.333 INFO:teuthology.orchestra.run.smithi094.stdout: "last_clean_begin": 0, 2024-09-14T06:30:12.333 INFO:teuthology.orchestra.run.smithi094.stdout: "last_clean_end": 0, 2024-09-14T06:30:12.333 INFO:teuthology.orchestra.run.smithi094.stdout: "up_from": 13, 2024-09-14T06:30:12.333 INFO:teuthology.orchestra.run.smithi094.stdout: "up_thru": 74, 2024-09-14T06:30:12.333 INFO:teuthology.orchestra.run.smithi094.stdout: "down_at": 0, 2024-09-14T06:30:12.333 INFO:teuthology.orchestra.run.smithi094.stdout: "lost_at": 0, 2024-09-14T06:30:12.333 INFO:teuthology.orchestra.run.smithi094.stdout: "public_addrs": { 2024-09-14T06:30:12.334 INFO:teuthology.orchestra.run.smithi094.stdout: "addrvec": [ 2024-09-14T06:30:12.334 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:12.334 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v2", 2024-09-14T06:30:12.334 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.94:6800", 2024-09-14T06:30:12.334 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 3382046754 2024-09-14T06:30:12.334 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:12.334 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:12.334 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v1", 2024-09-14T06:30:12.334 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.94:6801", 2024-09-14T06:30:12.334 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 3382046754 2024-09-14T06:30:12.334 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:12.334 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:12.334 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:12.334 INFO:teuthology.orchestra.run.smithi094.stdout: "cluster_addrs": { 2024-09-14T06:30:12.334 INFO:teuthology.orchestra.run.smithi094.stdout: "addrvec": [ 2024-09-14T06:30:12.335 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:12.335 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v2", 2024-09-14T06:30:12.335 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.94:6802", 2024-09-14T06:30:12.335 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 3382046754 2024-09-14T06:30:12.335 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:12.335 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:12.335 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v1", 2024-09-14T06:30:12.335 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.94:6803", 2024-09-14T06:30:12.335 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 3382046754 2024-09-14T06:30:12.335 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:12.335 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:12.335 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:12.335 INFO:teuthology.orchestra.run.smithi094.stdout: "heartbeat_back_addrs": { 2024-09-14T06:30:12.335 INFO:teuthology.orchestra.run.smithi094.stdout: "addrvec": [ 2024-09-14T06:30:12.336 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:12.336 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v2", 2024-09-14T06:30:12.336 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.94:6806", 2024-09-14T06:30:12.336 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 3382046754 2024-09-14T06:30:12.336 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:12.336 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:12.336 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v1", 2024-09-14T06:30:12.336 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.94:6807", 2024-09-14T06:30:12.336 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 3382046754 2024-09-14T06:30:12.336 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:12.336 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:12.336 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:12.336 INFO:teuthology.orchestra.run.smithi094.stdout: "heartbeat_front_addrs": { 2024-09-14T06:30:12.336 INFO:teuthology.orchestra.run.smithi094.stdout: "addrvec": [ 2024-09-14T06:30:12.337 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:12.337 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v2", 2024-09-14T06:30:12.337 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.94:6804", 2024-09-14T06:30:12.337 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 3382046754 2024-09-14T06:30:12.337 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:12.337 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:12.337 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v1", 2024-09-14T06:30:12.337 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.94:6805", 2024-09-14T06:30:12.337 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 3382046754 2024-09-14T06:30:12.337 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:12.337 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:12.337 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:12.337 INFO:teuthology.orchestra.run.smithi094.stdout: "public_addr": "172.21.15.94:6801/3382046754", 2024-09-14T06:30:12.337 INFO:teuthology.orchestra.run.smithi094.stdout: "cluster_addr": "172.21.15.94:6803/3382046754", 2024-09-14T06:30:12.338 INFO:teuthology.orchestra.run.smithi094.stdout: "heartbeat_back_addr": "172.21.15.94:6807/3382046754", 2024-09-14T06:30:12.338 INFO:teuthology.orchestra.run.smithi094.stdout: "heartbeat_front_addr": "172.21.15.94:6805/3382046754", 2024-09-14T06:30:12.338 INFO:teuthology.orchestra.run.smithi094.stdout: "state": [ 2024-09-14T06:30:12.338 INFO:teuthology.orchestra.run.smithi094.stdout: "exists", 2024-09-14T06:30:12.338 INFO:teuthology.orchestra.run.smithi094.stdout: "up" 2024-09-14T06:30:12.338 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:12.338 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:12.338 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:12.338 INFO:teuthology.orchestra.run.smithi094.stdout: "osd": 4, 2024-09-14T06:30:12.338 INFO:teuthology.orchestra.run.smithi094.stdout: "uuid": "2dab948e-c6e4-4437-bf17-017d1ac6821c", 2024-09-14T06:30:12.338 INFO:teuthology.orchestra.run.smithi094.stdout: "up": 1, 2024-09-14T06:30:12.338 INFO:teuthology.orchestra.run.smithi094.stdout: "in": 1, 2024-09-14T06:30:12.338 INFO:teuthology.orchestra.run.smithi094.stdout: "weight": 1, 2024-09-14T06:30:12.338 INFO:teuthology.orchestra.run.smithi094.stdout: "primary_affinity": 1, 2024-09-14T06:30:12.338 INFO:teuthology.orchestra.run.smithi094.stdout: "last_clean_begin": 0, 2024-09-14T06:30:12.339 INFO:teuthology.orchestra.run.smithi094.stdout: "last_clean_end": 0, 2024-09-14T06:30:12.339 INFO:teuthology.orchestra.run.smithi094.stdout: "up_from": 15, 2024-09-14T06:30:12.339 INFO:teuthology.orchestra.run.smithi094.stdout: "up_thru": 23, 2024-09-14T06:30:12.339 INFO:teuthology.orchestra.run.smithi094.stdout: "down_at": 0, 2024-09-14T06:30:12.339 INFO:teuthology.orchestra.run.smithi094.stdout: "lost_at": 0, 2024-09-14T06:30:12.339 INFO:teuthology.orchestra.run.smithi094.stdout: "public_addrs": { 2024-09-14T06:30:12.339 INFO:teuthology.orchestra.run.smithi094.stdout: "addrvec": [ 2024-09-14T06:30:12.339 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:12.339 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v2", 2024-09-14T06:30:12.339 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.138:6810", 2024-09-14T06:30:12.339 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 501726729 2024-09-14T06:30:12.339 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:12.339 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:12.340 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v1", 2024-09-14T06:30:12.340 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.138:6811", 2024-09-14T06:30:12.340 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 501726729 2024-09-14T06:30:12.340 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:12.340 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:12.340 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:12.340 INFO:teuthology.orchestra.run.smithi094.stdout: "cluster_addrs": { 2024-09-14T06:30:12.340 INFO:teuthology.orchestra.run.smithi094.stdout: "addrvec": [ 2024-09-14T06:30:12.340 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:12.340 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v2", 2024-09-14T06:30:12.340 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.138:6812", 2024-09-14T06:30:12.340 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 501726729 2024-09-14T06:30:12.340 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:12.340 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:12.341 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v1", 2024-09-14T06:30:12.341 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.138:6813", 2024-09-14T06:30:12.341 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 501726729 2024-09-14T06:30:12.341 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:12.341 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:12.341 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:12.341 INFO:teuthology.orchestra.run.smithi094.stdout: "heartbeat_back_addrs": { 2024-09-14T06:30:12.341 INFO:teuthology.orchestra.run.smithi094.stdout: "addrvec": [ 2024-09-14T06:30:12.341 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:12.341 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v2", 2024-09-14T06:30:12.341 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.138:6816", 2024-09-14T06:30:12.341 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 501726729 2024-09-14T06:30:12.341 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:12.342 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:12.342 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v1", 2024-09-14T06:30:12.342 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.138:6817", 2024-09-14T06:30:12.342 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 501726729 2024-09-14T06:30:12.342 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:12.342 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:12.342 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:12.342 INFO:teuthology.orchestra.run.smithi094.stdout: "heartbeat_front_addrs": { 2024-09-14T06:30:12.342 INFO:teuthology.orchestra.run.smithi094.stdout: "addrvec": [ 2024-09-14T06:30:12.342 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:12.342 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v2", 2024-09-14T06:30:12.342 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.138:6814", 2024-09-14T06:30:12.342 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 501726729 2024-09-14T06:30:12.343 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:12.343 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:12.343 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v1", 2024-09-14T06:30:12.343 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.138:6815", 2024-09-14T06:30:12.343 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 501726729 2024-09-14T06:30:12.343 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:12.343 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:12.343 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:12.343 INFO:teuthology.orchestra.run.smithi094.stdout: "public_addr": "172.21.15.138:6811/501726729", 2024-09-14T06:30:12.343 INFO:teuthology.orchestra.run.smithi094.stdout: "cluster_addr": "172.21.15.138:6813/501726729", 2024-09-14T06:30:12.343 INFO:teuthology.orchestra.run.smithi094.stdout: "heartbeat_back_addr": "172.21.15.138:6817/501726729", 2024-09-14T06:30:12.343 INFO:teuthology.orchestra.run.smithi094.stdout: "heartbeat_front_addr": "172.21.15.138:6815/501726729", 2024-09-14T06:30:12.344 INFO:teuthology.orchestra.run.smithi094.stdout: "state": [ 2024-09-14T06:30:12.344 INFO:teuthology.orchestra.run.smithi094.stdout: "exists", 2024-09-14T06:30:12.344 INFO:teuthology.orchestra.run.smithi094.stdout: "up" 2024-09-14T06:30:12.344 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:12.344 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:12.344 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:12.344 INFO:teuthology.orchestra.run.smithi094.stdout: "osd": 5, 2024-09-14T06:30:12.344 INFO:teuthology.orchestra.run.smithi094.stdout: "uuid": "8caf3aa3-ad87-4d33-b82e-4c15195176fd", 2024-09-14T06:30:12.344 INFO:teuthology.orchestra.run.smithi094.stdout: "up": 1, 2024-09-14T06:30:12.344 INFO:teuthology.orchestra.run.smithi094.stdout: "in": 1, 2024-09-14T06:30:12.344 INFO:teuthology.orchestra.run.smithi094.stdout: "weight": 1, 2024-09-14T06:30:12.344 INFO:teuthology.orchestra.run.smithi094.stdout: "primary_affinity": 1, 2024-09-14T06:30:12.344 INFO:teuthology.orchestra.run.smithi094.stdout: "last_clean_begin": 0, 2024-09-14T06:30:12.345 INFO:teuthology.orchestra.run.smithi094.stdout: "last_clean_end": 0, 2024-09-14T06:30:12.345 INFO:teuthology.orchestra.run.smithi094.stdout: "up_from": 15, 2024-09-14T06:30:12.345 INFO:teuthology.orchestra.run.smithi094.stdout: "up_thru": 23, 2024-09-14T06:30:12.345 INFO:teuthology.orchestra.run.smithi094.stdout: "down_at": 0, 2024-09-14T06:30:12.345 INFO:teuthology.orchestra.run.smithi094.stdout: "lost_at": 0, 2024-09-14T06:30:12.345 INFO:teuthology.orchestra.run.smithi094.stdout: "public_addrs": { 2024-09-14T06:30:12.345 INFO:teuthology.orchestra.run.smithi094.stdout: "addrvec": [ 2024-09-14T06:30:12.345 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:12.345 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v2", 2024-09-14T06:30:12.345 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.138:6826", 2024-09-14T06:30:12.345 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 1599292735 2024-09-14T06:30:12.345 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:12.345 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:12.346 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v1", 2024-09-14T06:30:12.346 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.138:6827", 2024-09-14T06:30:12.346 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 1599292735 2024-09-14T06:30:12.346 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:12.346 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:12.346 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:12.346 INFO:teuthology.orchestra.run.smithi094.stdout: "cluster_addrs": { 2024-09-14T06:30:12.346 INFO:teuthology.orchestra.run.smithi094.stdout: "addrvec": [ 2024-09-14T06:30:12.346 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:12.346 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v2", 2024-09-14T06:30:12.346 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.138:6828", 2024-09-14T06:30:12.346 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 1599292735 2024-09-14T06:30:12.346 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:12.346 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:12.347 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v1", 2024-09-14T06:30:12.347 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.138:6829", 2024-09-14T06:30:12.347 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 1599292735 2024-09-14T06:30:12.347 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:12.347 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:12.347 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:12.347 INFO:teuthology.orchestra.run.smithi094.stdout: "heartbeat_back_addrs": { 2024-09-14T06:30:12.347 INFO:teuthology.orchestra.run.smithi094.stdout: "addrvec": [ 2024-09-14T06:30:12.347 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:12.347 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v2", 2024-09-14T06:30:12.347 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.138:6832", 2024-09-14T06:30:12.347 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 1599292735 2024-09-14T06:30:12.347 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:12.347 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:12.347 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v1", 2024-09-14T06:30:12.348 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.138:6833", 2024-09-14T06:30:12.348 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 1599292735 2024-09-14T06:30:12.348 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:12.348 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:12.348 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:12.348 INFO:teuthology.orchestra.run.smithi094.stdout: "heartbeat_front_addrs": { 2024-09-14T06:30:12.348 INFO:teuthology.orchestra.run.smithi094.stdout: "addrvec": [ 2024-09-14T06:30:12.348 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:12.348 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v2", 2024-09-14T06:30:12.348 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.138:6830", 2024-09-14T06:30:12.348 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 1599292735 2024-09-14T06:30:12.348 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:12.348 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:12.348 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v1", 2024-09-14T06:30:12.349 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.138:6831", 2024-09-14T06:30:12.349 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 1599292735 2024-09-14T06:30:12.349 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:12.349 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:12.349 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:12.349 INFO:teuthology.orchestra.run.smithi094.stdout: "public_addr": "172.21.15.138:6827/1599292735", 2024-09-14T06:30:12.349 INFO:teuthology.orchestra.run.smithi094.stdout: "cluster_addr": "172.21.15.138:6829/1599292735", 2024-09-14T06:30:12.349 INFO:teuthology.orchestra.run.smithi094.stdout: "heartbeat_back_addr": "172.21.15.138:6833/1599292735", 2024-09-14T06:30:12.349 INFO:teuthology.orchestra.run.smithi094.stdout: "heartbeat_front_addr": "172.21.15.138:6831/1599292735", 2024-09-14T06:30:12.349 INFO:teuthology.orchestra.run.smithi094.stdout: "state": [ 2024-09-14T06:30:12.349 INFO:teuthology.orchestra.run.smithi094.stdout: "exists", 2024-09-14T06:30:12.349 INFO:teuthology.orchestra.run.smithi094.stdout: "up" 2024-09-14T06:30:12.349 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:12.350 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:12.350 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:12.350 INFO:teuthology.orchestra.run.smithi094.stdout: "osd": 6, 2024-09-14T06:30:12.350 INFO:teuthology.orchestra.run.smithi094.stdout: "uuid": "e55d11fd-433d-4462-93a7-d5e84560221b", 2024-09-14T06:30:12.350 INFO:teuthology.orchestra.run.smithi094.stdout: "up": 1, 2024-09-14T06:30:12.350 INFO:teuthology.orchestra.run.smithi094.stdout: "in": 1, 2024-09-14T06:30:12.350 INFO:teuthology.orchestra.run.smithi094.stdout: "weight": 1, 2024-09-14T06:30:12.350 INFO:teuthology.orchestra.run.smithi094.stdout: "primary_affinity": 1, 2024-09-14T06:30:12.350 INFO:teuthology.orchestra.run.smithi094.stdout: "last_clean_begin": 0, 2024-09-14T06:30:12.350 INFO:teuthology.orchestra.run.smithi094.stdout: "last_clean_end": 0, 2024-09-14T06:30:12.350 INFO:teuthology.orchestra.run.smithi094.stdout: "up_from": 15, 2024-09-14T06:30:12.350 INFO:teuthology.orchestra.run.smithi094.stdout: "up_thru": 23, 2024-09-14T06:30:12.350 INFO:teuthology.orchestra.run.smithi094.stdout: "down_at": 0, 2024-09-14T06:30:12.350 INFO:teuthology.orchestra.run.smithi094.stdout: "lost_at": 0, 2024-09-14T06:30:12.350 INFO:teuthology.orchestra.run.smithi094.stdout: "public_addrs": { 2024-09-14T06:30:12.351 INFO:teuthology.orchestra.run.smithi094.stdout: "addrvec": [ 2024-09-14T06:30:12.351 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:12.351 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v2", 2024-09-14T06:30:12.351 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.138:6818", 2024-09-14T06:30:12.351 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 163531479 2024-09-14T06:30:12.351 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:12.351 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:12.351 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v1", 2024-09-14T06:30:12.351 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.138:6819", 2024-09-14T06:30:12.351 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 163531479 2024-09-14T06:30:12.351 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:12.351 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:12.351 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:12.352 INFO:teuthology.orchestra.run.smithi094.stdout: "cluster_addrs": { 2024-09-14T06:30:12.352 INFO:teuthology.orchestra.run.smithi094.stdout: "addrvec": [ 2024-09-14T06:30:12.352 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:12.352 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v2", 2024-09-14T06:30:12.352 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.138:6820", 2024-09-14T06:30:12.352 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 163531479 2024-09-14T06:30:12.352 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:12.352 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:12.352 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v1", 2024-09-14T06:30:12.352 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.138:6821", 2024-09-14T06:30:12.352 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 163531479 2024-09-14T06:30:12.352 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:12.352 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:12.352 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:12.353 INFO:teuthology.orchestra.run.smithi094.stdout: "heartbeat_back_addrs": { 2024-09-14T06:30:12.353 INFO:teuthology.orchestra.run.smithi094.stdout: "addrvec": [ 2024-09-14T06:30:12.353 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:12.353 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v2", 2024-09-14T06:30:12.353 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.138:6824", 2024-09-14T06:30:12.353 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 163531479 2024-09-14T06:30:12.353 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:12.353 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:12.353 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v1", 2024-09-14T06:30:12.353 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.138:6825", 2024-09-14T06:30:12.353 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 163531479 2024-09-14T06:30:12.353 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:12.353 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:12.354 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:12.354 INFO:teuthology.orchestra.run.smithi094.stdout: "heartbeat_front_addrs": { 2024-09-14T06:30:12.354 INFO:teuthology.orchestra.run.smithi094.stdout: "addrvec": [ 2024-09-14T06:30:12.354 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:12.354 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v2", 2024-09-14T06:30:12.354 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.138:6822", 2024-09-14T06:30:12.354 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 163531479 2024-09-14T06:30:12.354 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:12.354 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:12.354 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v1", 2024-09-14T06:30:12.354 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.138:6823", 2024-09-14T06:30:12.354 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 163531479 2024-09-14T06:30:12.354 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:12.354 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:12.354 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:12.355 INFO:teuthology.orchestra.run.smithi094.stdout: "public_addr": "172.21.15.138:6819/163531479", 2024-09-14T06:30:12.355 INFO:teuthology.orchestra.run.smithi094.stdout: "cluster_addr": "172.21.15.138:6821/163531479", 2024-09-14T06:30:12.355 INFO:teuthology.orchestra.run.smithi094.stdout: "heartbeat_back_addr": "172.21.15.138:6825/163531479", 2024-09-14T06:30:12.355 INFO:teuthology.orchestra.run.smithi094.stdout: "heartbeat_front_addr": "172.21.15.138:6823/163531479", 2024-09-14T06:30:12.355 INFO:teuthology.orchestra.run.smithi094.stdout: "state": [ 2024-09-14T06:30:12.355 INFO:teuthology.orchestra.run.smithi094.stdout: "exists", 2024-09-14T06:30:12.355 INFO:teuthology.orchestra.run.smithi094.stdout: "up" 2024-09-14T06:30:12.355 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:12.355 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:12.355 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:12.355 INFO:teuthology.orchestra.run.smithi094.stdout: "osd": 7, 2024-09-14T06:30:12.355 INFO:teuthology.orchestra.run.smithi094.stdout: "uuid": "ba14ba69-2c6c-4d09-81e2-2593dc3a7ea9", 2024-09-14T06:30:12.355 INFO:teuthology.orchestra.run.smithi094.stdout: "up": 1, 2024-09-14T06:30:12.355 INFO:teuthology.orchestra.run.smithi094.stdout: "in": 1, 2024-09-14T06:30:12.355 INFO:teuthology.orchestra.run.smithi094.stdout: "weight": 1, 2024-09-14T06:30:12.356 INFO:teuthology.orchestra.run.smithi094.stdout: "primary_affinity": 1, 2024-09-14T06:30:12.356 INFO:teuthology.orchestra.run.smithi094.stdout: "last_clean_begin": 0, 2024-09-14T06:30:12.356 INFO:teuthology.orchestra.run.smithi094.stdout: "last_clean_end": 0, 2024-09-14T06:30:12.356 INFO:teuthology.orchestra.run.smithi094.stdout: "up_from": 15, 2024-09-14T06:30:12.356 INFO:teuthology.orchestra.run.smithi094.stdout: "up_thru": 23, 2024-09-14T06:30:12.356 INFO:teuthology.orchestra.run.smithi094.stdout: "down_at": 0, 2024-09-14T06:30:12.356 INFO:teuthology.orchestra.run.smithi094.stdout: "lost_at": 0, 2024-09-14T06:30:12.356 INFO:teuthology.orchestra.run.smithi094.stdout: "public_addrs": { 2024-09-14T06:30:12.356 INFO:teuthology.orchestra.run.smithi094.stdout: "addrvec": [ 2024-09-14T06:30:12.356 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:12.356 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v2", 2024-09-14T06:30:12.356 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.138:6800", 2024-09-14T06:30:12.356 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 218507235 2024-09-14T06:30:12.357 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:12.357 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:12.357 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v1", 2024-09-14T06:30:12.357 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.138:6801", 2024-09-14T06:30:12.357 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 218507235 2024-09-14T06:30:12.357 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:12.357 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:12.357 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:12.357 INFO:teuthology.orchestra.run.smithi094.stdout: "cluster_addrs": { 2024-09-14T06:30:12.357 INFO:teuthology.orchestra.run.smithi094.stdout: "addrvec": [ 2024-09-14T06:30:12.357 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:12.357 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v2", 2024-09-14T06:30:12.357 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.138:6802", 2024-09-14T06:30:12.358 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 218507235 2024-09-14T06:30:12.358 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:12.358 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:12.358 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v1", 2024-09-14T06:30:12.358 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.138:6803", 2024-09-14T06:30:12.358 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 218507235 2024-09-14T06:30:12.358 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:12.358 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:12.358 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:12.358 INFO:teuthology.orchestra.run.smithi094.stdout: "heartbeat_back_addrs": { 2024-09-14T06:30:12.358 INFO:teuthology.orchestra.run.smithi094.stdout: "addrvec": [ 2024-09-14T06:30:12.358 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:12.358 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v2", 2024-09-14T06:30:12.358 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.138:6806", 2024-09-14T06:30:12.358 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 218507235 2024-09-14T06:30:12.358 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:12.359 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:12.359 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v1", 2024-09-14T06:30:12.359 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.138:6807", 2024-09-14T06:30:12.359 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 218507235 2024-09-14T06:30:12.359 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:12.359 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:12.359 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:12.359 INFO:teuthology.orchestra.run.smithi094.stdout: "heartbeat_front_addrs": { 2024-09-14T06:30:12.359 INFO:teuthology.orchestra.run.smithi094.stdout: "addrvec": [ 2024-09-14T06:30:12.359 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:12.359 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v2", 2024-09-14T06:30:12.359 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.138:6804", 2024-09-14T06:30:12.359 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 218507235 2024-09-14T06:30:12.359 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:12.360 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:12.360 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v1", 2024-09-14T06:30:12.360 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.138:6805", 2024-09-14T06:30:12.360 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 218507235 2024-09-14T06:30:12.360 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:12.360 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:12.360 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:12.360 INFO:teuthology.orchestra.run.smithi094.stdout: "public_addr": "172.21.15.138:6801/218507235", 2024-09-14T06:30:12.360 INFO:teuthology.orchestra.run.smithi094.stdout: "cluster_addr": "172.21.15.138:6803/218507235", 2024-09-14T06:30:12.360 INFO:teuthology.orchestra.run.smithi094.stdout: "heartbeat_back_addr": "172.21.15.138:6807/218507235", 2024-09-14T06:30:12.360 INFO:teuthology.orchestra.run.smithi094.stdout: "heartbeat_front_addr": "172.21.15.138:6805/218507235", 2024-09-14T06:30:12.360 INFO:teuthology.orchestra.run.smithi094.stdout: "state": [ 2024-09-14T06:30:12.360 INFO:teuthology.orchestra.run.smithi094.stdout: "exists", 2024-09-14T06:30:12.360 INFO:teuthology.orchestra.run.smithi094.stdout: "up" 2024-09-14T06:30:12.360 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:12.361 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:12.361 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-09-14T06:30:12.361 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_xinfo": [ 2024-09-14T06:30:12.361 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:12.361 INFO:teuthology.orchestra.run.smithi094.stdout: "osd": 0, 2024-09-14T06:30:12.361 INFO:teuthology.orchestra.run.smithi094.stdout: "down_stamp": "0.000000", 2024-09-14T06:30:12.361 INFO:teuthology.orchestra.run.smithi094.stdout: "laggy_probability": 0, 2024-09-14T06:30:12.361 INFO:teuthology.orchestra.run.smithi094.stdout: "laggy_interval": 0, 2024-09-14T06:30:12.361 INFO:teuthology.orchestra.run.smithi094.stdout: "features": 4540719139924082687, 2024-09-14T06:30:12.361 INFO:teuthology.orchestra.run.smithi094.stdout: "old_weight": 0, 2024-09-14T06:30:12.361 INFO:teuthology.orchestra.run.smithi094.stdout: "last_purged_snaps_scrub": "2024-09-14T06:28:39.906639+0000", 2024-09-14T06:30:12.361 INFO:teuthology.orchestra.run.smithi094.stdout: "dead_epoch": 0 2024-09-14T06:30:12.361 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:12.361 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:12.362 INFO:teuthology.orchestra.run.smithi094.stdout: "osd": 1, 2024-09-14T06:30:12.362 INFO:teuthology.orchestra.run.smithi094.stdout: "down_stamp": "0.000000", 2024-09-14T06:30:12.362 INFO:teuthology.orchestra.run.smithi094.stdout: "laggy_probability": 0, 2024-09-14T06:30:12.362 INFO:teuthology.orchestra.run.smithi094.stdout: "laggy_interval": 0, 2024-09-14T06:30:12.362 INFO:teuthology.orchestra.run.smithi094.stdout: "features": 4540719139924082687, 2024-09-14T06:30:12.362 INFO:teuthology.orchestra.run.smithi094.stdout: "old_weight": 0, 2024-09-14T06:30:12.362 INFO:teuthology.orchestra.run.smithi094.stdout: "last_purged_snaps_scrub": "2024-09-14T06:28:40.541279+0000", 2024-09-14T06:30:12.362 INFO:teuthology.orchestra.run.smithi094.stdout: "dead_epoch": 0 2024-09-14T06:30:12.362 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:12.362 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:12.362 INFO:teuthology.orchestra.run.smithi094.stdout: "osd": 2, 2024-09-14T06:30:12.362 INFO:teuthology.orchestra.run.smithi094.stdout: "down_stamp": "0.000000", 2024-09-14T06:30:12.362 INFO:teuthology.orchestra.run.smithi094.stdout: "laggy_probability": 0, 2024-09-14T06:30:12.362 INFO:teuthology.orchestra.run.smithi094.stdout: "laggy_interval": 0, 2024-09-14T06:30:12.363 INFO:teuthology.orchestra.run.smithi094.stdout: "features": 4540719139924082687, 2024-09-14T06:30:12.363 INFO:teuthology.orchestra.run.smithi094.stdout: "old_weight": 0, 2024-09-14T06:30:12.363 INFO:teuthology.orchestra.run.smithi094.stdout: "last_purged_snaps_scrub": "2024-09-14T06:28:40.025350+0000", 2024-09-14T06:30:12.363 INFO:teuthology.orchestra.run.smithi094.stdout: "dead_epoch": 0 2024-09-14T06:30:12.363 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:12.363 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:12.363 INFO:teuthology.orchestra.run.smithi094.stdout: "osd": 3, 2024-09-14T06:30:12.363 INFO:teuthology.orchestra.run.smithi094.stdout: "down_stamp": "0.000000", 2024-09-14T06:30:12.363 INFO:teuthology.orchestra.run.smithi094.stdout: "laggy_probability": 0, 2024-09-14T06:30:12.363 INFO:teuthology.orchestra.run.smithi094.stdout: "laggy_interval": 0, 2024-09-14T06:30:12.363 INFO:teuthology.orchestra.run.smithi094.stdout: "features": 4540719139924082687, 2024-09-14T06:30:12.363 INFO:teuthology.orchestra.run.smithi094.stdout: "old_weight": 0, 2024-09-14T06:30:12.363 INFO:teuthology.orchestra.run.smithi094.stdout: "last_purged_snaps_scrub": "2024-09-14T06:28:39.217944+0000", 2024-09-14T06:30:12.363 INFO:teuthology.orchestra.run.smithi094.stdout: "dead_epoch": 0 2024-09-14T06:30:12.364 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:12.364 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:12.364 INFO:teuthology.orchestra.run.smithi094.stdout: "osd": 4, 2024-09-14T06:30:12.364 INFO:teuthology.orchestra.run.smithi094.stdout: "down_stamp": "0.000000", 2024-09-14T06:30:12.364 INFO:teuthology.orchestra.run.smithi094.stdout: "laggy_probability": 0, 2024-09-14T06:30:12.364 INFO:teuthology.orchestra.run.smithi094.stdout: "laggy_interval": 0, 2024-09-14T06:30:12.364 INFO:teuthology.orchestra.run.smithi094.stdout: "features": 4540719139924082687, 2024-09-14T06:30:12.364 INFO:teuthology.orchestra.run.smithi094.stdout: "old_weight": 0, 2024-09-14T06:30:12.364 INFO:teuthology.orchestra.run.smithi094.stdout: "last_purged_snaps_scrub": "2024-09-14T06:28:40.185898+0000", 2024-09-14T06:30:12.364 INFO:teuthology.orchestra.run.smithi094.stdout: "dead_epoch": 0 2024-09-14T06:30:12.364 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:12.364 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:12.364 INFO:teuthology.orchestra.run.smithi094.stdout: "osd": 5, 2024-09-14T06:30:12.364 INFO:teuthology.orchestra.run.smithi094.stdout: "down_stamp": "0.000000", 2024-09-14T06:30:12.364 INFO:teuthology.orchestra.run.smithi094.stdout: "laggy_probability": 0, 2024-09-14T06:30:12.365 INFO:teuthology.orchestra.run.smithi094.stdout: "laggy_interval": 0, 2024-09-14T06:30:12.365 INFO:teuthology.orchestra.run.smithi094.stdout: "features": 4540719139924082687, 2024-09-14T06:30:12.365 INFO:teuthology.orchestra.run.smithi094.stdout: "old_weight": 0, 2024-09-14T06:30:12.365 INFO:teuthology.orchestra.run.smithi094.stdout: "last_purged_snaps_scrub": "2024-09-14T06:28:40.783504+0000", 2024-09-14T06:30:12.365 INFO:teuthology.orchestra.run.smithi094.stdout: "dead_epoch": 0 2024-09-14T06:30:12.365 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:12.365 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:12.365 INFO:teuthology.orchestra.run.smithi094.stdout: "osd": 6, 2024-09-14T06:30:12.365 INFO:teuthology.orchestra.run.smithi094.stdout: "down_stamp": "0.000000", 2024-09-14T06:30:12.365 INFO:teuthology.orchestra.run.smithi094.stdout: "laggy_probability": 0, 2024-09-14T06:30:12.365 INFO:teuthology.orchestra.run.smithi094.stdout: "laggy_interval": 0, 2024-09-14T06:30:12.365 INFO:teuthology.orchestra.run.smithi094.stdout: "features": 4540719139924082687, 2024-09-14T06:30:12.365 INFO:teuthology.orchestra.run.smithi094.stdout: "old_weight": 0, 2024-09-14T06:30:12.365 INFO:teuthology.orchestra.run.smithi094.stdout: "last_purged_snaps_scrub": "2024-09-14T06:28:40.527305+0000", 2024-09-14T06:30:12.365 INFO:teuthology.orchestra.run.smithi094.stdout: "dead_epoch": 0 2024-09-14T06:30:12.366 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:12.366 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:12.366 INFO:teuthology.orchestra.run.smithi094.stdout: "osd": 7, 2024-09-14T06:30:12.366 INFO:teuthology.orchestra.run.smithi094.stdout: "down_stamp": "0.000000", 2024-09-14T06:30:12.366 INFO:teuthology.orchestra.run.smithi094.stdout: "laggy_probability": 0, 2024-09-14T06:30:12.366 INFO:teuthology.orchestra.run.smithi094.stdout: "laggy_interval": 0, 2024-09-14T06:30:12.366 INFO:teuthology.orchestra.run.smithi094.stdout: "features": 4540719139924082687, 2024-09-14T06:30:12.366 INFO:teuthology.orchestra.run.smithi094.stdout: "old_weight": 0, 2024-09-14T06:30:12.366 INFO:teuthology.orchestra.run.smithi094.stdout: "last_purged_snaps_scrub": "2024-09-14T06:28:40.349676+0000", 2024-09-14T06:30:12.366 INFO:teuthology.orchestra.run.smithi094.stdout: "dead_epoch": 0 2024-09-14T06:30:12.366 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:12.366 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-09-14T06:30:12.366 INFO:teuthology.orchestra.run.smithi094.stdout: "pg_upmap": [], 2024-09-14T06:30:12.366 INFO:teuthology.orchestra.run.smithi094.stdout: "pg_upmap_items": [ 2024-09-14T06:30:12.367 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:12.367 INFO:teuthology.orchestra.run.smithi094.stdout: "pgid": "3.7", 2024-09-14T06:30:12.367 INFO:teuthology.orchestra.run.smithi094.stdout: "mappings": [ 2024-09-14T06:30:12.367 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:12.367 INFO:teuthology.orchestra.run.smithi094.stdout: "from": 3, 2024-09-14T06:30:12.367 INFO:teuthology.orchestra.run.smithi094.stdout: "to": 0 2024-09-14T06:30:12.367 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:12.367 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:12.367 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:12.367 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-09-14T06:30:12.367 INFO:teuthology.orchestra.run.smithi094.stdout: "pg_upmap_primaries": [], 2024-09-14T06:30:12.367 INFO:teuthology.orchestra.run.smithi094.stdout: "pg_temp": [], 2024-09-14T06:30:12.367 INFO:teuthology.orchestra.run.smithi094.stdout: "primary_temp": [], 2024-09-14T06:30:12.367 INFO:teuthology.orchestra.run.smithi094.stdout: "blocklist": {}, 2024-09-14T06:30:12.367 INFO:teuthology.orchestra.run.smithi094.stdout: "range_blocklist": {}, 2024-09-14T06:30:12.368 INFO:teuthology.orchestra.run.smithi094.stdout: "erasure_code_profiles": { 2024-09-14T06:30:12.368 INFO:teuthology.orchestra.run.smithi094.stdout: "default": { 2024-09-14T06:30:12.368 INFO:teuthology.orchestra.run.smithi094.stdout: "crush-failure-domain": "osd", 2024-09-14T06:30:12.368 INFO:teuthology.orchestra.run.smithi094.stdout: "k": "2", 2024-09-14T06:30:12.368 INFO:teuthology.orchestra.run.smithi094.stdout: "m": "1", 2024-09-14T06:30:12.368 INFO:teuthology.orchestra.run.smithi094.stdout: "plugin": "jerasure", 2024-09-14T06:30:12.368 INFO:teuthology.orchestra.run.smithi094.stdout: "technique": "reed_sol_van" 2024-09-14T06:30:12.368 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:12.368 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:12.368 INFO:teuthology.orchestra.run.smithi094.stdout: "removed_snaps_queue": [ 2024-09-14T06:30:12.368 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:12.368 INFO:teuthology.orchestra.run.smithi094.stdout: "pool": 3, 2024-09-14T06:30:12.368 INFO:teuthology.orchestra.run.smithi094.stdout: "snaps": [ 2024-09-14T06:30:12.368 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:12.368 INFO:teuthology.orchestra.run.smithi094.stdout: "begin": 36, 2024-09-14T06:30:12.369 INFO:teuthology.orchestra.run.smithi094.stdout: "length": 1 2024-09-14T06:30:12.369 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:12.369 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:12.369 INFO:teuthology.orchestra.run.smithi094.stdout: "begin": 38, 2024-09-14T06:30:12.369 INFO:teuthology.orchestra.run.smithi094.stdout: "length": 1 2024-09-14T06:30:12.369 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:12.369 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:12.369 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:12.369 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-09-14T06:30:12.369 INFO:teuthology.orchestra.run.smithi094.stdout: "new_removed_snaps": [ 2024-09-14T06:30:12.369 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:12.369 INFO:teuthology.orchestra.run.smithi094.stdout: "pool": 3, 2024-09-14T06:30:12.369 INFO:teuthology.orchestra.run.smithi094.stdout: "snaps": [ 2024-09-14T06:30:12.370 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:12.370 INFO:teuthology.orchestra.run.smithi094.stdout: "begin": 38, 2024-09-14T06:30:12.370 INFO:teuthology.orchestra.run.smithi094.stdout: "length": 1 2024-09-14T06:30:12.370 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:12.370 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:12.370 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:12.370 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-09-14T06:30:12.370 INFO:teuthology.orchestra.run.smithi094.stdout: "new_purged_snaps": [ 2024-09-14T06:30:12.370 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:12.370 INFO:teuthology.orchestra.run.smithi094.stdout: "pool": 3, 2024-09-14T06:30:12.370 INFO:teuthology.orchestra.run.smithi094.stdout: "snaps": [ 2024-09-14T06:30:12.370 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:12.370 INFO:teuthology.orchestra.run.smithi094.stdout: "begin": 32, 2024-09-14T06:30:12.370 INFO:teuthology.orchestra.run.smithi094.stdout: "length": 1 2024-09-14T06:30:12.370 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:12.371 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:12.371 INFO:teuthology.orchestra.run.smithi094.stdout: "begin": 34, 2024-09-14T06:30:12.371 INFO:teuthology.orchestra.run.smithi094.stdout: "length": 1 2024-09-14T06:30:12.371 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:12.371 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:12.371 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:12.371 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-09-14T06:30:12.371 INFO:teuthology.orchestra.run.smithi094.stdout: "crush_node_flags": {}, 2024-09-14T06:30:12.371 INFO:teuthology.orchestra.run.smithi094.stdout: "device_class_flags": {}, 2024-09-14T06:30:12.371 INFO:teuthology.orchestra.run.smithi094.stdout: "stretch_mode": { 2024-09-14T06:30:12.371 INFO:teuthology.orchestra.run.smithi094.stdout: "stretch_mode_enabled": false, 2024-09-14T06:30:12.371 INFO:teuthology.orchestra.run.smithi094.stdout: "stretch_bucket_count": 0, 2024-09-14T06:30:12.371 INFO:teuthology.orchestra.run.smithi094.stdout: "degraded_stretch_mode": 0, 2024-09-14T06:30:12.371 INFO:teuthology.orchestra.run.smithi094.stdout: "recovering_stretch_mode": 0, 2024-09-14T06:30:12.371 INFO:teuthology.orchestra.run.smithi094.stdout: "stretch_mode_bucket": 0 2024-09-14T06:30:12.371 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:12.372 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-14T06:30:12.374 INFO:tasks.thrashosds.thrasher:j is {'epoch': 90, 'fsid': '200ad3da-bd63-4e04-af74-efb5ca0af75e', 'created': '2024-09-14T06:28:30.782046+0000', 'modified': '2024-09-14T06:30:10.579472+0000', 'last_up_change': '2024-09-14T06:28:42.827239+0000', 'last_in_change': '2024-09-14T06:28:33.724176+0000', 'flags': 'sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit', 'flags_num': 5799936, 'flags_set': ['pglog_hardlimit', 'purged_snapdirs', 'recovery_deletes', 'sortbitwise'], 'crush_version': 5, 'full_ratio': 0.949999988079071, 'backfillfull_ratio': 0.8999999761581421, 'nearfull_ratio': 0.8500000238418579, 'cluster_snapshot': '', 'pool_max': 3, 'max_osd': 8, 'require_min_compat_client': 'luminous', 'min_compat_client': 'luminous', 'require_osd_release': 'squid', 'allow_crimson': False, 'pools': [{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-14T06:28:43.295990+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2024-09-14T06:28:49.451714+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 2, 'snap_epoch': 22, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.9900000095367432, 'score_stable': 1.9900000095367432, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'pool_client.0', 'create_time': '2024-09-14T06:28:59.688535+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '90', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 38, 'snap_epoch': 90, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}], 'osds': [{'osd': 0, 'uuid': '8b22a249-b842-406a-a02c-c0f31f57b635', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 75, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.94:6824', 'nonce': 1809564923}, {'type': 'v1', 'addr': '172.21.15.94:6825', 'nonce': 1809564923}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.94:6826', 'nonce': 1809564923}, {'type': 'v1', 'addr': '172.21.15.94:6827', 'nonce': 1809564923}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.94:6830', 'nonce': 1809564923}, {'type': 'v1', 'addr': '172.21.15.94:6831', 'nonce': 1809564923}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.94:6828', 'nonce': 1809564923}, {'type': 'v1', 'addr': '172.21.15.94:6829', 'nonce': 1809564923}]}, 'public_addr': '172.21.15.94:6825/1809564923', 'cluster_addr': '172.21.15.94:6827/1809564923', 'heartbeat_back_addr': '172.21.15.94:6831/1809564923', 'heartbeat_front_addr': '172.21.15.94:6829/1809564923', 'state': ['exists', 'up']}, {'osd': 1, 'uuid': '02251a03-84a0-414d-a980-fac4fd043cb6', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 23, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.94:6816', 'nonce': 736402652}, {'type': 'v1', 'addr': '172.21.15.94:6817', 'nonce': 736402652}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.94:6818', 'nonce': 736402652}, {'type': 'v1', 'addr': '172.21.15.94:6819', 'nonce': 736402652}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.94:6822', 'nonce': 736402652}, {'type': 'v1', 'addr': '172.21.15.94:6823', 'nonce': 736402652}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.94:6820', 'nonce': 736402652}, {'type': 'v1', 'addr': '172.21.15.94:6821', 'nonce': 736402652}]}, 'public_addr': '172.21.15.94:6817/736402652', 'cluster_addr': '172.21.15.94:6819/736402652', 'heartbeat_back_addr': '172.21.15.94:6823/736402652', 'heartbeat_front_addr': '172.21.15.94:6821/736402652', 'state': ['exists', 'up']}, {'osd': 2, 'uuid': 'd45bf470-bb29-4739-8294-cd122b2b4102', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 19, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.94:6808', 'nonce': 1063204782}, {'type': 'v1', 'addr': '172.21.15.94:6809', 'nonce': 1063204782}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.94:6810', 'nonce': 1063204782}, {'type': 'v1', 'addr': '172.21.15.94:6811', 'nonce': 1063204782}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.94:6814', 'nonce': 1063204782}, {'type': 'v1', 'addr': '172.21.15.94:6815', 'nonce': 1063204782}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.94:6812', 'nonce': 1063204782}, {'type': 'v1', 'addr': '172.21.15.94:6813', 'nonce': 1063204782}]}, 'public_addr': '172.21.15.94:6809/1063204782', 'cluster_addr': '172.21.15.94:6811/1063204782', 'heartbeat_back_addr': '172.21.15.94:6815/1063204782', 'heartbeat_front_addr': '172.21.15.94:6813/1063204782', 'state': ['exists', 'up']}, {'osd': 3, 'uuid': '456ec2d9-c438-4243-b931-335704f661eb', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 74, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.94:6800', 'nonce': 3382046754}, {'type': 'v1', 'addr': '172.21.15.94:6801', 'nonce': 3382046754}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.94:6802', 'nonce': 3382046754}, {'type': 'v1', 'addr': '172.21.15.94:6803', 'nonce': 3382046754}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.94:6806', 'nonce': 3382046754}, {'type': 'v1', 'addr': '172.21.15.94:6807', 'nonce': 3382046754}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.94:6804', 'nonce': 3382046754}, {'type': 'v1', 'addr': '172.21.15.94:6805', 'nonce': 3382046754}]}, 'public_addr': '172.21.15.94:6801/3382046754', 'cluster_addr': '172.21.15.94:6803/3382046754', 'heartbeat_back_addr': '172.21.15.94:6807/3382046754', 'heartbeat_front_addr': '172.21.15.94:6805/3382046754', 'state': ['exists', 'up']}, {'osd': 4, 'uuid': '2dab948e-c6e4-4437-bf17-017d1ac6821c', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 23, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.138:6810', 'nonce': 501726729}, {'type': 'v1', 'addr': '172.21.15.138:6811', 'nonce': 501726729}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.138:6812', 'nonce': 501726729}, {'type': 'v1', 'addr': '172.21.15.138:6813', 'nonce': 501726729}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.138:6816', 'nonce': 501726729}, {'type': 'v1', 'addr': '172.21.15.138:6817', 'nonce': 501726729}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.138:6814', 'nonce': 501726729}, {'type': 'v1', 'addr': '172.21.15.138:6815', 'nonce': 501726729}]}, 'public_addr': '172.21.15.138:6811/501726729', 'cluster_addr': '172.21.15.138:6813/501726729', 'heartbeat_back_addr': '172.21.15.138:6817/501726729', 'heartbeat_front_addr': '172.21.15.138:6815/501726729', 'state': ['exists', 'up']}, {'osd': 5, 'uuid': '8caf3aa3-ad87-4d33-b82e-4c15195176fd', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 23, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.138:6826', 'nonce': 1599292735}, {'type': 'v1', 'addr': '172.21.15.138:6827', 'nonce': 1599292735}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.138:6828', 'nonce': 1599292735}, {'type': 'v1', 'addr': '172.21.15.138:6829', 'nonce': 1599292735}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.138:6832', 'nonce': 1599292735}, {'type': 'v1', 'addr': '172.21.15.138:6833', 'nonce': 1599292735}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.138:6830', 'nonce': 1599292735}, {'type': 'v1', 'addr': '172.21.15.138:6831', 'nonce': 1599292735}]}, 'public_addr': '172.21.15.138:6827/1599292735', 'cluster_addr': '172.21.15.138:6829/1599292735', 'heartbeat_back_addr': '172.21.15.138:6833/1599292735', 'heartbeat_front_addr': '172.21.15.138:6831/1599292735', 'state': ['exists', 'up']}, {'osd': 6, 'uuid': 'e55d11fd-433d-4462-93a7-d5e84560221b', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 23, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.138:6818', 'nonce': 163531479}, {'type': 'v1', 'addr': '172.21.15.138:6819', 'nonce': 163531479}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.138:6820', 'nonce': 163531479}, {'type': 'v1', 'addr': '172.21.15.138:6821', 'nonce': 163531479}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.138:6824', 'nonce': 163531479}, {'type': 'v1', 'addr': '172.21.15.138:6825', 'nonce': 163531479}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.138:6822', 'nonce': 163531479}, {'type': 'v1', 'addr': '172.21.15.138:6823', 'nonce': 163531479}]}, 'public_addr': '172.21.15.138:6819/163531479', 'cluster_addr': '172.21.15.138:6821/163531479', 'heartbeat_back_addr': '172.21.15.138:6825/163531479', 'heartbeat_front_addr': '172.21.15.138:6823/163531479', 'state': ['exists', 'up']}, {'osd': 7, 'uuid': 'ba14ba69-2c6c-4d09-81e2-2593dc3a7ea9', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 23, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.138:6800', 'nonce': 218507235}, {'type': 'v1', 'addr': '172.21.15.138:6801', 'nonce': 218507235}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.138:6802', 'nonce': 218507235}, {'type': 'v1', 'addr': '172.21.15.138:6803', 'nonce': 218507235}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.138:6806', 'nonce': 218507235}, {'type': 'v1', 'addr': '172.21.15.138:6807', 'nonce': 218507235}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.138:6804', 'nonce': 218507235}, {'type': 'v1', 'addr': '172.21.15.138:6805', 'nonce': 218507235}]}, 'public_addr': '172.21.15.138:6801/218507235', 'cluster_addr': '172.21.15.138:6803/218507235', 'heartbeat_back_addr': '172.21.15.138:6807/218507235', 'heartbeat_front_addr': '172.21.15.138:6805/218507235', 'state': ['exists', 'up']}], 'osd_xinfo': [{'osd': 0, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-14T06:28:39.906639+0000', 'dead_epoch': 0}, {'osd': 1, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-14T06:28:40.541279+0000', 'dead_epoch': 0}, {'osd': 2, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-14T06:28:40.025350+0000', 'dead_epoch': 0}, {'osd': 3, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-14T06:28:39.217944+0000', 'dead_epoch': 0}, {'osd': 4, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-14T06:28:40.185898+0000', 'dead_epoch': 0}, {'osd': 5, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-14T06:28:40.783504+0000', 'dead_epoch': 0}, {'osd': 6, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-14T06:28:40.527305+0000', 'dead_epoch': 0}, {'osd': 7, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-14T06:28:40.349676+0000', 'dead_epoch': 0}], 'pg_upmap': [], 'pg_upmap_items': [{'pgid': '3.7', 'mappings': [{'from': 3, 'to': 0}]}], 'pg_upmap_primaries': [], 'pg_temp': [], 'primary_temp': [], 'blocklist': {}, 'range_blocklist': {}, 'erasure_code_profiles': {'default': {'crush-failure-domain': 'osd', 'k': '2', 'm': '1', 'plugin': 'jerasure', 'technique': 'reed_sol_van'}}, 'removed_snaps_queue': [{'pool': 3, 'snaps': [{'begin': 36, 'length': 1}, {'begin': 38, 'length': 1}]}], 'new_removed_snaps': [{'pool': 3, 'snaps': [{'begin': 38, 'length': 1}]}], 'new_purged_snaps': [{'pool': 3, 'snaps': [{'begin': 32, 'length': 1}, {'begin': 34, 'length': 1}]}], 'crush_node_flags': {}, 'device_class_flags': {}, 'stretch_mode': {'stretch_mode_enabled': False, 'stretch_bucket_count': 0, 'degraded_stretch_mode': 0, 'recovering_stretch_mode': 0, 'stretch_mode_bucket': 0}} 2024-09-14T06:30:12.374 INFO:tasks.thrashosds.thrasher:No pg_upmap entries; doing nothing 2024-09-14T06:30:12.416 INFO:tasks.ceph.osd.7.smithi138.stderr:2024-09-14T06:30:12.415+0000 7f3c6a21d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33623) UID: 0 2024-09-14T06:30:12.475 INFO:teuthology.orchestra.run.smithi138.stdout:{ 2024-09-14T06:30:12.475 INFO:teuthology.orchestra.run.smithi138.stdout: "ops": [], 2024-09-14T06:30:12.475 INFO:teuthology.orchestra.run.smithi138.stdout: "complaint_time": 30, 2024-09-14T06:30:12.475 INFO:teuthology.orchestra.run.smithi138.stdout: "num_blocked_ops": 0 2024-09-14T06:30:12.475 INFO:teuthology.orchestra.run.smithi138.stdout:} 2024-09-14T06:30:12.477 INFO:tasks.ceph.osd.4.smithi138.stderr:2024-09-14T06:30:12.476+0000 7f6c653c8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33615) UID: 0 2024-09-14T06:30:12.483 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-14T06:30:12.578 INFO:tasks.ceph.osd.2.smithi094.stderr:2024-09-14T06:30:12.578+0000 7faf3269b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33649) UID: 0 2024-09-14T06:30:12.631 INFO:teuthology.orchestra.run.smithi138.stdout:{ 2024-09-14T06:30:12.631 INFO:teuthology.orchestra.run.smithi138.stdout: "size": 20, 2024-09-14T06:30:12.631 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 600, 2024-09-14T06:30:12.631 INFO:teuthology.orchestra.run.smithi138.stdout: "ops": [ 2024-09-14T06:30:12.631 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:12.631 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:11453 3.1d 3:bcad7303:::rbd_data.112d92027590.0000000000000001:f [read 94208~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:30:12.631 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:25.654375+0000", 2024-09-14T06:30:12.631 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 46.974696444000003, 2024-09-14T06:30:12.631 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052449318000000002, 2024-09-14T06:30:12.631 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:30:12.631 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:30:12.632 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:30:12.632 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:30:12.632 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:30:12.632 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:12.632 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 11453 2024-09-14T06:30:12.632 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:12.632 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:30:12.632 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:12.632 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:30:12.632 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654375+0000", 2024-09-14T06:30:12.632 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:12.632 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:12.632 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:12.633 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:30:12.633 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654375+0000", 2024-09-14T06:30:12.633 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:12.633 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:12.633 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:12.633 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:30:12.633 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654496+0000", 2024-09-14T06:30:12.633 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00012081 2024-09-14T06:30:12.633 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:12.633 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:12.633 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:30:12.633 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654497+0000", 2024-09-14T06:30:12.633 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.8789999999999999e-06 2024-09-14T06:30:12.634 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:12.634 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:12.634 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:30:12.634 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654501+0000", 2024-09-14T06:30:12.634 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.8999999999999999e-06 2024-09-14T06:30:12.634 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:12.634 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:12.634 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:30:12.634 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654505+0000", 2024-09-14T06:30:12.634 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.2229999999999999e-06 2024-09-14T06:30:12.634 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:12.634 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:12.634 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:30:12.635 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.705917+0000", 2024-09-14T06:30:12.635 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.051412501999999999 2024-09-14T06:30:12.635 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:12.635 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:12.635 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:30:12.635 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.705999+0000", 2024-09-14T06:30:12.635 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.1551999999999994e-05 2024-09-14T06:30:12.635 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:12.635 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:12.636 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:30:12.636 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.706824+0000", 2024-09-14T06:30:12.636 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00082545199999999996 2024-09-14T06:30:12.636 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:12.636 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:30:12.636 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:12.636 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:12.636 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:12.636 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:11454 3.1d 3:bcad7303:::rbd_data.112d92027590.0000000000000001:f [read 104448~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:30:12.636 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:25.654510+0000", 2024-09-14T06:30:12.636 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 46.974561653999999, 2024-09-14T06:30:12.636 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052610713000000003, 2024-09-14T06:30:12.636 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:30:12.637 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:30:12.637 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:30:12.637 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:30:12.637 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:30:12.637 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:12.637 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 11454 2024-09-14T06:30:12.637 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:12.637 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:30:12.637 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:12.637 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:30:12.637 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654510+0000", 2024-09-14T06:30:12.637 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:12.637 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:12.637 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:12.638 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:30:12.638 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654510+0000", 2024-09-14T06:30:12.638 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:12.638 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:12.638 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:12.638 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:30:12.638 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654825+0000", 2024-09-14T06:30:12.638 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00031522400000000002 2024-09-14T06:30:12.638 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:12.638 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:12.638 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:30:12.638 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654827+0000", 2024-09-14T06:30:12.638 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.9709999999999998e-06 2024-09-14T06:30:12.638 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:12.639 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:12.639 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:30:12.639 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654832+0000", 2024-09-14T06:30:12.639 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.4099999999999999e-06 2024-09-14T06:30:12.639 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:12.639 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:12.639 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:30:12.639 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654835+0000", 2024-09-14T06:30:12.639 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.14e-06 2024-09-14T06:30:12.639 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:12.639 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:12.639 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:30:12.639 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.706851+0000", 2024-09-14T06:30:12.639 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052015761000000001 2024-09-14T06:30:12.639 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:12.640 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:12.640 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:30:12.640 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.706934+0000", 2024-09-14T06:30:12.640 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.2484999999999994e-05 2024-09-14T06:30:12.640 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:12.640 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:12.640 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:30:12.640 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.707120+0000", 2024-09-14T06:30:12.640 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00018672200000000001 2024-09-14T06:30:12.640 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:12.640 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:30:12.640 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:12.640 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:12.641 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:12.641 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:11455 3.1d 3:bcad7303:::rbd_data.112d92027590.0000000000000001:f [read 114688~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:30:12.641 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:25.654841+0000", 2024-09-14T06:30:12.641 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 46.974230325000001, 2024-09-14T06:30:12.641 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052579898, 2024-09-14T06:30:12.641 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:30:12.641 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:30:12.641 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:30:12.641 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:30:12.641 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:30:12.641 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:12.641 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 11455 2024-09-14T06:30:12.641 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:12.641 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:30:12.642 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:12.642 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:30:12.642 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654841+0000", 2024-09-14T06:30:12.642 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:12.642 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:12.642 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:12.642 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:30:12.642 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654841+0000", 2024-09-14T06:30:12.642 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:12.642 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:12.642 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:12.642 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:30:12.642 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654841+0000", 2024-09-14T06:30:12.642 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.7300000000000002e-07 2024-09-14T06:30:12.643 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:12.643 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:12.643 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:30:12.643 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654845+0000", 2024-09-14T06:30:12.643 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.0520000000000003e-06 2024-09-14T06:30:12.643 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:12.643 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:12.643 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:30:12.643 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654849+0000", 2024-09-14T06:30:12.643 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.0459999999999999e-06 2024-09-14T06:30:12.643 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:12.643 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:12.643 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:30:12.643 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654852+0000", 2024-09-14T06:30:12.644 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.9830000000000001e-06 2024-09-14T06:30:12.644 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:12.644 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:12.644 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:30:12.644 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.707159+0000", 2024-09-14T06:30:12.644 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052306826000000001 2024-09-14T06:30:12.644 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:12.644 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:12.644 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:30:12.644 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.707242+0000", 2024-09-14T06:30:12.644 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.2595000000000005e-05 2024-09-14T06:30:12.644 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:12.645 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:12.645 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:30:12.645 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.707421+0000", 2024-09-14T06:30:12.645 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.000179123 2024-09-14T06:30:12.645 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:12.645 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:30:12.645 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:12.645 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:12.645 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:12.645 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:11456 3.1d 3:bcad7303:::rbd_data.112d92027590.0000000000000001:f [read 124928~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:30:12.646 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:25.654856+0000", 2024-09-14T06:30:12.646 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 46.974215479000001, 2024-09-14T06:30:12.646 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052801580000000001, 2024-09-14T06:30:12.646 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:30:12.646 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:30:12.646 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:30:12.646 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:30:12.646 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:30:12.646 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:12.646 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 11456 2024-09-14T06:30:12.646 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:12.646 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:30:12.646 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:12.646 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:30:12.647 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654856+0000", 2024-09-14T06:30:12.647 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:12.647 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:12.647 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:12.647 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:30:12.647 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654856+0000", 2024-09-14T06:30:12.647 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:12.647 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:12.647 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:12.647 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:30:12.647 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654856+0000", 2024-09-14T06:30:12.647 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.29e-07 2024-09-14T06:30:12.647 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:12.648 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:12.648 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:30:12.648 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654857+0000", 2024-09-14T06:30:12.648 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.187e-06 2024-09-14T06:30:12.648 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:12.648 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:12.648 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:30:12.648 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654861+0000", 2024-09-14T06:30:12.648 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.3850000000000002e-06 2024-09-14T06:30:12.648 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:12.648 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:12.648 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:30:12.648 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654863+0000", 2024-09-14T06:30:12.649 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.9500000000000001e-06 2024-09-14T06:30:12.649 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:12.649 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:12.649 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:30:12.649 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.707456+0000", 2024-09-14T06:30:12.649 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052592434 2024-09-14T06:30:12.649 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:12.649 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:12.649 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:30:12.649 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.707516+0000", 2024-09-14T06:30:12.649 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.0167999999999997e-05 2024-09-14T06:30:12.649 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:12.649 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:12.650 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:30:12.650 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.707657+0000", 2024-09-14T06:30:12.650 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00014122700000000001 2024-09-14T06:30:12.650 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:12.650 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:30:12.650 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:12.650 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:12.650 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:12.650 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:11457 3.1d 3:bcad7303:::rbd_data.112d92027590.0000000000000001:f [read 135168~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:30:12.650 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:25.654867+0000", 2024-09-14T06:30:12.650 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 46.974204594, 2024-09-14T06:30:12.650 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052921902, 2024-09-14T06:30:12.650 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:30:12.650 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:30:12.651 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:30:12.651 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:30:12.651 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:30:12.651 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:12.651 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 11457 2024-09-14T06:30:12.651 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:12.651 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:30:12.651 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:12.651 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:30:12.651 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654867+0000", 2024-09-14T06:30:12.651 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:12.651 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:12.651 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:12.652 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:30:12.652 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654867+0000", 2024-09-14T06:30:12.652 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:12.652 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:12.652 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:12.652 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:30:12.652 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654972+0000", 2024-09-14T06:30:12.652 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00010584 2024-09-14T06:30:12.652 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:12.652 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:12.652 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:30:12.652 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654975+0000", 2024-09-14T06:30:12.652 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.1169999999999998e-06 2024-09-14T06:30:12.653 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:12.653 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:12.653 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:30:12.653 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654979+0000", 2024-09-14T06:30:12.653 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.1960000000000001e-06 2024-09-14T06:30:12.653 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:12.653 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:12.653 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:30:12.653 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654982+0000", 2024-09-14T06:30:12.653 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.2990000000000001e-06 2024-09-14T06:30:12.653 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:12.653 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:12.653 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:30:12.653 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.707684+0000", 2024-09-14T06:30:12.653 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052701979000000003 2024-09-14T06:30:12.654 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:12.654 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:12.654 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:30:12.654 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.707728+0000", 2024-09-14T06:30:12.654 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.3529999999999998e-05 2024-09-14T06:30:12.654 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:12.654 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:12.654 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:30:12.654 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.707789+0000", 2024-09-14T06:30:12.654 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.0940999999999998e-05 2024-09-14T06:30:12.654 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:12.654 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:30:12.654 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:12.654 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:12.655 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:12.655 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:11458 3.1d 3:bcad7303:::rbd_data.112d92027590.0000000000000001:f [read 145408~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:30:12.655 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:25.654987+0000", 2024-09-14T06:30:12.655 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 46.974084196, 2024-09-14T06:30:12.655 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.05291655, 2024-09-14T06:30:12.655 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:30:12.655 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:30:12.655 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:30:12.655 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:30:12.655 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:30:12.655 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:12.655 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 11458 2024-09-14T06:30:12.655 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:12.655 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:30:12.655 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:12.656 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:30:12.656 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654987+0000", 2024-09-14T06:30:12.656 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:12.656 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:12.656 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:12.656 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:30:12.656 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654987+0000", 2024-09-14T06:30:12.656 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:12.656 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:12.656 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:12.656 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:30:12.656 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655077+0000", 2024-09-14T06:30:12.656 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.0253000000000005e-05 2024-09-14T06:30:12.656 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:12.657 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:12.657 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:30:12.657 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655079+0000", 2024-09-14T06:30:12.657 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.9989999999999998e-06 2024-09-14T06:30:12.657 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:12.657 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:12.657 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:30:12.657 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655083+0000", 2024-09-14T06:30:12.657 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.9280000000000003e-06 2024-09-14T06:30:12.657 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:12.657 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:12.657 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:30:12.657 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655086+0000", 2024-09-14T06:30:12.658 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.056e-06 2024-09-14T06:30:12.658 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:12.658 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:12.658 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:30:12.658 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.707805+0000", 2024-09-14T06:30:12.658 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052718627999999997 2024-09-14T06:30:12.658 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:12.658 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:12.658 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:30:12.658 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.707848+0000", 2024-09-14T06:30:12.659 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.2728999999999997e-05 2024-09-14T06:30:12.659 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:12.659 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:12.659 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:30:12.659 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.707904+0000", 2024-09-14T06:30:12.659 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.5957e-05 2024-09-14T06:30:12.659 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:12.659 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:30:12.659 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:12.659 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:12.659 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:12.659 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:11459 3.1d 3:bcad7303:::rbd_data.112d92027590.0000000000000001:f [read 155648~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:30:12.659 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:25.655091+0000", 2024-09-14T06:30:12.659 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 46.973980187999999, 2024-09-14T06:30:12.660 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052944242000000002, 2024-09-14T06:30:12.660 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:30:12.660 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:30:12.660 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:30:12.660 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:30:12.660 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:30:12.660 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:12.660 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 11459 2024-09-14T06:30:12.660 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:12.660 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:30:12.660 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:12.660 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:30:12.660 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655091+0000", 2024-09-14T06:30:12.661 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:12.661 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:12.661 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:12.661 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:30:12.661 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655091+0000", 2024-09-14T06:30:12.661 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:12.661 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:12.661 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:12.661 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:30:12.661 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655237+0000", 2024-09-14T06:30:12.661 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00014570100000000001 2024-09-14T06:30:12.661 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:12.661 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:12.661 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:30:12.662 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655239+0000", 2024-09-14T06:30:12.662 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.8670000000000001e-06 2024-09-14T06:30:12.662 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:12.662 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:12.662 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:30:12.662 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655243+0000", 2024-09-14T06:30:12.662 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.9829999999999998e-06 2024-09-14T06:30:12.662 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:12.662 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:12.662 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:30:12.662 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655246+0000", 2024-09-14T06:30:12.662 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.1540000000000002e-06 2024-09-14T06:30:12.662 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:12.663 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:12.663 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:30:12.663 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.707934+0000", 2024-09-14T06:30:12.663 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052688286000000001 2024-09-14T06:30:12.663 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:12.663 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:12.663 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:30:12.663 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.707977+0000", 2024-09-14T06:30:12.663 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.3291999999999998e-05 2024-09-14T06:30:12.663 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:12.663 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:12.663 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:30:12.663 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.708035+0000", 2024-09-14T06:30:12.664 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.7958999999999999e-05 2024-09-14T06:30:12.664 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:12.664 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:30:12.664 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:12.664 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:12.664 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:12.664 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:11460 3.1d 3:bcad7303:::rbd_data.112d92027590.0000000000000001:f [read 165888~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:30:12.664 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:25.655250+0000", 2024-09-14T06:30:12.664 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 46.973820891000003, 2024-09-14T06:30:12.664 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052910698999999999, 2024-09-14T06:30:12.664 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:30:12.664 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:30:12.664 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:30:12.665 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:30:12.665 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:30:12.665 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:12.665 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 11460 2024-09-14T06:30:12.665 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:12.665 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:30:12.665 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:12.665 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:30:12.665 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655250+0000", 2024-09-14T06:30:12.665 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:12.665 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:12.665 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:12.665 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:30:12.666 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655250+0000", 2024-09-14T06:30:12.666 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:12.666 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:12.666 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:12.666 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:30:12.666 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655356+0000", 2024-09-14T06:30:12.666 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00010535900000000001 2024-09-14T06:30:12.666 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:12.666 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:12.666 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:30:12.666 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655357+0000", 2024-09-14T06:30:12.666 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.8080000000000001e-06 2024-09-14T06:30:12.666 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:12.666 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:12.667 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:30:12.667 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655361+0000", 2024-09-14T06:30:12.667 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.861e-06 2024-09-14T06:30:12.667 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:12.667 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:12.667 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:30:12.667 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655364+0000", 2024-09-14T06:30:12.667 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.1089999999999998e-06 2024-09-14T06:30:12.667 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:12.667 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:12.667 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:30:12.667 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.708065+0000", 2024-09-14T06:30:12.667 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052700595000000003 2024-09-14T06:30:12.667 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:12.668 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:12.668 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:30:12.668 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.708108+0000", 2024-09-14T06:30:12.668 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.2484999999999997e-05 2024-09-14T06:30:12.668 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:12.668 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:12.668 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:30:12.668 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.708161+0000", 2024-09-14T06:30:12.668 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.3482000000000001e-05 2024-09-14T06:30:12.668 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:12.668 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:30:12.668 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:12.669 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:12.669 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:12.669 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:11461 3.1d 3:bcad7303:::rbd_data.112d92027590.0000000000000001:f [read 176128~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:30:12.669 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:25.655370+0000", 2024-09-14T06:30:12.669 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 46.973701609000003, 2024-09-14T06:30:12.669 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052915141999999998, 2024-09-14T06:30:12.669 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:30:12.669 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:30:12.669 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:30:12.669 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:30:12.669 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:30:12.669 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:12.669 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 11461 2024-09-14T06:30:12.669 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:12.669 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:30:12.670 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:12.670 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:30:12.670 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655370+0000", 2024-09-14T06:30:12.670 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:12.670 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:12.670 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:12.670 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:30:12.670 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655370+0000", 2024-09-14T06:30:12.670 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:12.670 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:12.670 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:12.670 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:30:12.670 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655394+0000", 2024-09-14T06:30:12.671 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.4553999999999999e-05 2024-09-14T06:30:12.671 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:12.671 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:12.671 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:30:12.671 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655396+0000", 2024-09-14T06:30:12.671 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.8950000000000001e-06 2024-09-14T06:30:12.671 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:12.671 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:12.671 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:30:12.671 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655401+0000", 2024-09-14T06:30:12.671 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.0089999999999998e-06 2024-09-14T06:30:12.671 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:12.671 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:12.671 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:30:12.672 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655404+0000", 2024-09-14T06:30:12.672 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.3400000000000002e-06 2024-09-14T06:30:12.672 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:12.672 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:12.672 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:30:12.672 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.708186+0000", 2024-09-14T06:30:12.672 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052781726000000001 2024-09-14T06:30:12.672 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:12.672 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:12.672 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:30:12.672 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.708229+0000", 2024-09-14T06:30:12.672 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.2670999999999999e-05 2024-09-14T06:30:12.672 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:12.673 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:12.673 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:30:12.673 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.708285+0000", 2024-09-14T06:30:12.673 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.5946999999999998e-05 2024-09-14T06:30:12.673 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:12.673 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:30:12.673 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:12.673 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:12.673 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:12.673 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:11462 3.1d 3:bcad7303:::rbd_data.112d92027590.0000000000000001:f [read 186368~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:30:12.673 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:25.655410+0000", 2024-09-14T06:30:12.673 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 46.973661131, 2024-09-14T06:30:12.673 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052995258000000003, 2024-09-14T06:30:12.673 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:30:12.674 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:30:12.674 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:30:12.674 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:30:12.674 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:30:12.674 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:12.674 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 11462 2024-09-14T06:30:12.674 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:12.674 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:30:12.674 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:12.674 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:30:12.674 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655410+0000", 2024-09-14T06:30:12.674 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:12.674 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:12.675 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:12.675 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:30:12.675 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655410+0000", 2024-09-14T06:30:12.675 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:12.675 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:12.675 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:12.675 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:30:12.675 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655625+0000", 2024-09-14T06:30:12.675 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00021442600000000001 2024-09-14T06:30:12.675 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:12.675 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:12.675 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:30:12.675 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655626+0000", 2024-09-14T06:30:12.675 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.815e-06 2024-09-14T06:30:12.676 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:12.676 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:12.676 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:30:12.676 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655631+0000", 2024-09-14T06:30:12.676 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.651e-06 2024-09-14T06:30:12.676 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:12.676 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:12.676 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:30:12.676 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655634+0000", 2024-09-14T06:30:12.676 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.2449999999999998e-06 2024-09-14T06:30:12.676 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:12.676 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:12.676 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:30:12.677 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.708312+0000", 2024-09-14T06:30:12.677 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052677412 2024-09-14T06:30:12.677 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:12.677 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:12.677 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:30:12.677 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.708352+0000", 2024-09-14T06:30:12.677 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.0729999999999998e-05 2024-09-14T06:30:12.677 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:12.677 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:12.677 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:30:12.677 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.708405+0000", 2024-09-14T06:30:12.677 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.2979000000000002e-05 2024-09-14T06:30:12.677 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:12.677 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:30:12.678 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:12.678 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:12.678 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:12.678 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:11463 3.1d 3:bcad7303:::rbd_data.112d92027590.0000000000000001:f [read 196608~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:30:12.678 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:25.655639+0000", 2024-09-14T06:30:12.678 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 46.973431984999998, 2024-09-14T06:30:12.678 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052888859000000003, 2024-09-14T06:30:12.678 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:30:12.678 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:30:12.678 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:30:12.678 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:30:12.678 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:30:12.678 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:12.678 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 11463 2024-09-14T06:30:12.679 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:12.679 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:30:12.679 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:12.679 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:30:12.679 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655639+0000", 2024-09-14T06:30:12.679 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:12.679 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:12.679 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:12.679 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:30:12.679 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655639+0000", 2024-09-14T06:30:12.679 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:12.679 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:12.679 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:12.679 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:30:12.680 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655640+0000", 2024-09-14T06:30:12.680 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.08e-07 2024-09-14T06:30:12.680 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:12.680 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:12.680 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:30:12.680 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655641+0000", 2024-09-14T06:30:12.680 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.156e-06 2024-09-14T06:30:12.680 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:12.680 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:12.680 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:30:12.680 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655648+0000", 2024-09-14T06:30:12.680 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.8460000000000002e-06 2024-09-14T06:30:12.680 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:12.680 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:12.681 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:30:12.681 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655651+0000", 2024-09-14T06:30:12.681 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.0079999999999998e-06 2024-09-14T06:30:12.681 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:12.681 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:12.681 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:30:12.681 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.708429+0000", 2024-09-14T06:30:12.681 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052778459 2024-09-14T06:30:12.681 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:12.681 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:12.681 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:30:12.681 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.708473+0000", 2024-09-14T06:30:12.681 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.3782999999999997e-05 2024-09-14T06:30:12.681 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:12.682 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:12.682 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:30:12.682 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.708528+0000", 2024-09-14T06:30:12.682 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.5198999999999998e-05 2024-09-14T06:30:12.682 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:12.682 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:30:12.682 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:12.682 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:12.682 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:12.682 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:11464 3.1d 3:bcad7303:::rbd_data.112d92027590.0000000000000001:f [read 206848~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:30:12.682 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:25.655654+0000", 2024-09-14T06:30:12.682 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 46.973417116, 2024-09-14T06:30:12.682 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052981845999999999, 2024-09-14T06:30:12.682 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:30:12.683 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:30:12.683 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:30:12.683 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:30:12.683 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:30:12.683 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:12.683 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 11464 2024-09-14T06:30:12.683 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:12.683 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:30:12.683 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:12.683 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:30:12.683 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655654+0000", 2024-09-14T06:30:12.683 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:12.683 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:12.683 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:12.684 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:30:12.684 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655654+0000", 2024-09-14T06:30:12.684 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:12.684 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:12.684 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:12.684 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:30:12.684 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655946+0000", 2024-09-14T06:30:12.684 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00029148200000000003 2024-09-14T06:30:12.684 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:12.684 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:12.684 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:30:12.684 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655948+0000", 2024-09-14T06:30:12.684 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.973e-06 2024-09-14T06:30:12.685 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:12.685 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:12.685 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:30:12.685 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655953+0000", 2024-09-14T06:30:12.685 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.1240000000000004e-06 2024-09-14T06:30:12.685 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:12.685 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:12.685 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:30:12.685 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655956+0000", 2024-09-14T06:30:12.685 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.315e-06 2024-09-14T06:30:12.685 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:12.685 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:12.685 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:30:12.685 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.708543+0000", 2024-09-14T06:30:12.686 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052587416999999997 2024-09-14T06:30:12.686 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:12.686 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:12.686 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:30:12.686 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.708584+0000", 2024-09-14T06:30:12.686 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.1004000000000002e-05 2024-09-14T06:30:12.686 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:12.686 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:12.686 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:30:12.686 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.708636+0000", 2024-09-14T06:30:12.686 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.1530999999999998e-05 2024-09-14T06:30:12.686 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:12.686 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:30:12.686 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:12.687 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:12.687 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:12.687 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:11465 3.1d 3:bcad7303:::rbd_data.112d92027590.0000000000000001:f [read 217088~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:30:12.687 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:25.655961+0000", 2024-09-14T06:30:12.687 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 46.973109964000002, 2024-09-14T06:30:12.687 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052798682999999999, 2024-09-14T06:30:12.687 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:30:12.687 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:30:12.687 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:30:12.687 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:30:12.687 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:30:12.687 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:12.687 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 11465 2024-09-14T06:30:12.688 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:12.688 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:30:12.688 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:12.688 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:30:12.688 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655961+0000", 2024-09-14T06:30:12.688 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:12.688 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:12.688 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:12.688 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:30:12.688 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655961+0000", 2024-09-14T06:30:12.688 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:12.688 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:12.688 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:12.688 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:30:12.689 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.656114+0000", 2024-09-14T06:30:12.689 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.000152745 2024-09-14T06:30:12.689 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:12.689 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:12.689 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:30:12.689 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.656116+0000", 2024-09-14T06:30:12.689 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.804e-06 2024-09-14T06:30:12.689 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:12.689 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:12.689 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:30:12.689 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.656120+0000", 2024-09-14T06:30:12.689 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.4700000000000004e-06 2024-09-14T06:30:12.689 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:12.690 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:12.690 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:30:12.690 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.656123+0000", 2024-09-14T06:30:12.690 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.1269999999999999e-06 2024-09-14T06:30:12.690 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:12.690 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:12.690 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:30:12.690 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.708660+0000", 2024-09-14T06:30:12.690 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052536919000000001 2024-09-14T06:30:12.690 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:12.690 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:12.690 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:30:12.690 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.708703+0000", 2024-09-14T06:30:12.690 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.3077000000000001e-05 2024-09-14T06:30:12.691 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:12.691 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:12.691 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:30:12.691 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.708760+0000", 2024-09-14T06:30:12.691 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.6541e-05 2024-09-14T06:30:12.691 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:12.691 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:30:12.691 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:12.691 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:12.691 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:12.691 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:11466 3.1d 3:bcad7303:::rbd_data.112d92027590.0000000000000001:f [read 227328~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:30:12.691 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:25.656129+0000", 2024-09-14T06:30:12.691 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 46.972942576000001, 2024-09-14T06:30:12.691 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052750987999999999, 2024-09-14T06:30:12.692 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:30:12.692 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:30:12.692 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:30:12.692 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:30:12.692 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:30:12.692 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:12.692 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 11466 2024-09-14T06:30:12.692 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:12.692 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:30:12.692 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:12.692 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:30:12.692 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.656129+0000", 2024-09-14T06:30:12.692 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:12.693 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:12.693 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:12.693 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:30:12.693 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.656129+0000", 2024-09-14T06:30:12.693 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:12.693 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:12.693 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:12.693 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:30:12.693 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.656347+0000", 2024-09-14T06:30:12.693 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.000218054 2024-09-14T06:30:12.693 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:12.693 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:12.693 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:30:12.694 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.656349+0000", 2024-09-14T06:30:12.694 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.9199999999999998e-06 2024-09-14T06:30:12.694 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:12.694 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:12.694 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:30:12.694 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.656353+0000", 2024-09-14T06:30:12.694 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.9210000000000002e-06 2024-09-14T06:30:12.694 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:12.694 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:12.694 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:30:12.694 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.656356+0000", 2024-09-14T06:30:12.694 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.7670000000000001e-06 2024-09-14T06:30:12.694 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:12.695 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:12.695 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:30:12.695 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.708787+0000", 2024-09-14T06:30:12.695 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052430840999999999 2024-09-14T06:30:12.695 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:12.695 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:12.695 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:30:12.695 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.708827+0000", 2024-09-14T06:30:12.695 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.9538e-05 2024-09-14T06:30:12.695 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:12.695 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:12.695 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:30:12.695 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.708880+0000", 2024-09-14T06:30:12.696 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.2947e-05 2024-09-14T06:30:12.696 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:12.696 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:30:12.696 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:12.696 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:12.696 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:12.696 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:11467 3.1d 3:bcad7303:::rbd_data.112d92027590.0000000000000001:f [read 237568~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:30:12.696 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:25.656361+0000", 2024-09-14T06:30:12.696 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 46.972709928, 2024-09-14T06:30:12.696 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052643898000000001, 2024-09-14T06:30:12.696 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:30:12.696 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:30:12.696 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:30:12.697 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:30:12.697 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:30:12.697 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:12.697 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 11467 2024-09-14T06:30:12.697 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:12.697 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:30:12.697 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:12.697 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:30:12.697 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.656361+0000", 2024-09-14T06:30:12.697 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:12.697 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:12.697 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:12.697 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:30:12.697 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.656361+0000", 2024-09-14T06:30:12.698 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:12.698 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:12.698 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:12.698 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:30:12.698 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.656855+0000", 2024-09-14T06:30:12.698 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00049372000000000001 2024-09-14T06:30:12.698 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:12.698 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:12.698 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:30:12.698 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.656859+0000", 2024-09-14T06:30:12.698 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.1300000000000003e-06 2024-09-14T06:30:12.698 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:12.698 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:12.699 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:30:12.699 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.656863+0000", 2024-09-14T06:30:12.699 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.2629999999999997e-06 2024-09-14T06:30:12.699 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:12.699 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:12.699 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:30:12.699 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.656867+0000", 2024-09-14T06:30:12.699 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.3220000000000001e-06 2024-09-14T06:30:12.699 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:12.699 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:12.699 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:30:12.699 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.708903+0000", 2024-09-14T06:30:12.699 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052035855999999998 2024-09-14T06:30:12.700 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:12.700 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:12.700 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:30:12.700 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.708949+0000", 2024-09-14T06:30:12.700 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.6276999999999998e-05 2024-09-14T06:30:12.700 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:12.700 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:12.700 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:30:12.700 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.709005+0000", 2024-09-14T06:30:12.700 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.6329999999999998e-05 2024-09-14T06:30:12.700 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:12.700 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:30:12.700 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:12.701 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:12.701 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:12.701 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:11479 3.1d 3:bcad7303:::rbd_data.112d92027590.0000000000000001:f [read 360448~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:30:12.701 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:25.658388+0000", 2024-09-14T06:30:12.701 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 46.970683094999998, 2024-09-14T06:30:12.701 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052408920999999997, 2024-09-14T06:30:12.701 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:30:12.701 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:30:12.701 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:30:12.701 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:30:12.701 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:30:12.701 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:12.701 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 11479 2024-09-14T06:30:12.701 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:12.702 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:30:12.702 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:12.702 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:30:12.702 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.658388+0000", 2024-09-14T06:30:12.702 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:12.702 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:12.702 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:12.702 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:30:12.702 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.658388+0000", 2024-09-14T06:30:12.702 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:12.702 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:12.702 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:12.702 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:30:12.703 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.658534+0000", 2024-09-14T06:30:12.703 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.000145613 2024-09-14T06:30:12.703 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:12.703 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:12.703 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:30:12.703 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.658540+0000", 2024-09-14T06:30:12.703 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.6680000000000004e-06 2024-09-14T06:30:12.703 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:12.703 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:12.703 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:30:12.703 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.658548+0000", 2024-09-14T06:30:12.703 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.7980000000000006e-06 2024-09-14T06:30:12.703 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:12.703 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:12.704 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:30:12.704 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.658554+0000", 2024-09-14T06:30:12.704 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.6869999999999999e-06 2024-09-14T06:30:12.704 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:12.704 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:12.704 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:30:12.704 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.710624+0000", 2024-09-14T06:30:12.704 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052070567999999998 2024-09-14T06:30:12.704 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:12.704 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:12.704 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:30:12.704 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.710666+0000", 2024-09-14T06:30:12.704 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.1467000000000001e-05 2024-09-14T06:30:12.705 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:12.705 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:12.705 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:30:12.705 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.710797+0000", 2024-09-14T06:30:12.705 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00013112 2024-09-14T06:30:12.705 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:12.705 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:30:12.705 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:12.705 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:12.705 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:12.705 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:11480 3.1d 3:bcad7303:::rbd_data.112d92027590.0000000000000001:f [read 370688~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:30:12.705 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:25.658563+0000", 2024-09-14T06:30:12.705 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 46.970508164000002, 2024-09-14T06:30:12.706 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052429164, 2024-09-14T06:30:12.706 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:30:12.706 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:30:12.706 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:30:12.706 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:30:12.706 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:30:12.706 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:12.706 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 11480 2024-09-14T06:30:12.706 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:12.706 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:30:12.706 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:12.706 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:30:12.706 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.658563+0000", 2024-09-14T06:30:12.706 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:12.707 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:12.707 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:12.707 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:30:12.707 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.658563+0000", 2024-09-14T06:30:12.707 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:12.707 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:12.707 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:12.707 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:30:12.707 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.658658+0000", 2024-09-14T06:30:12.707 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.5346999999999994e-05 2024-09-14T06:30:12.707 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:12.707 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:12.707 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:30:12.707 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.658662+0000", 2024-09-14T06:30:12.708 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.14e-06 2024-09-14T06:30:12.708 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:12.708 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:12.708 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:30:12.708 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.658669+0000", 2024-09-14T06:30:12.708 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.7379999999999994e-06 2024-09-14T06:30:12.708 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:12.708 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:12.708 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:30:12.708 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.658675+0000", 2024-09-14T06:30:12.708 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.5430000000000001e-06 2024-09-14T06:30:12.708 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:12.708 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:12.708 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:30:12.709 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.710824+0000", 2024-09-14T06:30:12.709 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052148847999999998 2024-09-14T06:30:12.709 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:12.709 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:12.709 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:30:12.709 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.710864+0000", 2024-09-14T06:30:12.709 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.9925999999999997e-05 2024-09-14T06:30:12.709 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:12.709 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:12.709 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:30:12.709 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.710992+0000", 2024-09-14T06:30:12.709 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.000128622 2024-09-14T06:30:12.709 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:12.710 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:30:12.710 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:12.710 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:12.710 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:12.710 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:11481 3.1d 3:bcad7303:::rbd_data.112d92027590.0000000000000001:f [read 380928~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:30:12.710 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:25.658684+0000", 2024-09-14T06:30:12.710 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 46.970387094000003, 2024-09-14T06:30:12.710 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052508565, 2024-09-14T06:30:12.710 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:30:12.710 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:30:12.710 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:30:12.710 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:30:12.710 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:30:12.710 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:12.711 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 11481 2024-09-14T06:30:12.711 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:12.711 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:30:12.711 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:12.711 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:30:12.711 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.658684+0000", 2024-09-14T06:30:12.711 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:12.711 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:12.711 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:12.711 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:30:12.711 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.658684+0000", 2024-09-14T06:30:12.711 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:12.711 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:12.711 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:12.712 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:30:12.712 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.658715+0000", 2024-09-14T06:30:12.712 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.0703000000000001e-05 2024-09-14T06:30:12.712 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:12.712 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:12.712 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:30:12.712 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.658718+0000", 2024-09-14T06:30:12.712 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.0709999999999999e-06 2024-09-14T06:30:12.712 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:12.712 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:12.712 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:30:12.712 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.658725+0000", 2024-09-14T06:30:12.712 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.2339999999999997e-06 2024-09-14T06:30:12.712 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:12.713 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:12.713 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:30:12.713 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.658731+0000", 2024-09-14T06:30:12.713 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.6450000000000002e-06 2024-09-14T06:30:12.713 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:12.713 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:12.713 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:30:12.713 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.711020+0000", 2024-09-14T06:30:12.713 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052288886 2024-09-14T06:30:12.713 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:12.713 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:12.713 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:30:12.713 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.711061+0000", 2024-09-14T06:30:12.713 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.1084000000000002e-05 2024-09-14T06:30:12.714 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:12.714 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:12.714 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:30:12.714 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.711193+0000", 2024-09-14T06:30:12.714 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00013194199999999999 2024-09-14T06:30:12.714 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:12.714 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:30:12.714 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:12.714 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:12.714 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:12.714 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:11482 3.1d 3:bcad7303:::rbd_data.112d92027590.0000000000000001:f [read 391168~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:30:12.714 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:25.658740+0000", 2024-09-14T06:30:12.714 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 46.970331629, 2024-09-14T06:30:12.715 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052637015000000002, 2024-09-14T06:30:12.715 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:30:12.715 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:30:12.715 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:30:12.715 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:30:12.715 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:30:12.715 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:12.715 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 11482 2024-09-14T06:30:12.715 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:12.715 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:30:12.715 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:12.715 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:30:12.715 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.658740+0000", 2024-09-14T06:30:12.715 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:12.715 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:12.716 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:12.716 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:30:12.716 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.658740+0000", 2024-09-14T06:30:12.716 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:12.716 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:12.716 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:12.716 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:30:12.716 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.658880+0000", 2024-09-14T06:30:12.716 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00014036100000000001 2024-09-14T06:30:12.716 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:12.716 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:12.716 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:30:12.716 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.658883+0000", 2024-09-14T06:30:12.717 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.3969999999999998e-06 2024-09-14T06:30:12.717 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:12.717 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:12.717 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:30:12.717 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.658890+0000", 2024-09-14T06:30:12.717 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.1500000000000002e-06 2024-09-14T06:30:12.717 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:12.717 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:12.717 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:30:12.717 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.658897+0000", 2024-09-14T06:30:12.717 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.1260000000000003e-06 2024-09-14T06:30:12.717 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:12.717 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:12.717 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:30:12.718 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.711207+0000", 2024-09-14T06:30:12.718 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052310777000000003 2024-09-14T06:30:12.718 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:12.718 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:12.718 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:30:12.718 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.711249+0000", 2024-09-14T06:30:12.718 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.1786999999999998e-05 2024-09-14T06:30:12.718 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:12.718 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:12.718 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:30:12.718 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.711377+0000", 2024-09-14T06:30:12.718 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.000127417 2024-09-14T06:30:12.718 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:12.719 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:30:12.719 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:12.719 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:12.719 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:12.719 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:11483 3.1d 3:bcad7303:::rbd_data.112d92027590.0000000000000001:f [read 401408~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:30:12.719 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:25.658906+0000", 2024-09-14T06:30:12.719 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 46.970164844999999, 2024-09-14T06:30:12.719 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052595109000000001, 2024-09-14T06:30:12.719 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:30:12.719 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:30:12.719 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:30:12.719 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:30:12.719 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:30:12.719 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:12.720 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 11483 2024-09-14T06:30:12.720 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:12.720 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:30:12.720 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:12.720 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:30:12.720 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.658906+0000", 2024-09-14T06:30:12.720 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:12.720 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:12.720 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:12.720 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:30:12.720 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.658906+0000", 2024-09-14T06:30:12.720 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:12.720 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:12.720 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:12.721 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:30:12.721 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.664985+0000", 2024-09-14T06:30:12.721 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.0060787250000000001 2024-09-14T06:30:12.721 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:12.721 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:12.721 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:30:12.721 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.664991+0000", 2024-09-14T06:30:12.721 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.7030000000000003e-06 2024-09-14T06:30:12.721 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:12.721 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:12.721 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:30:12.721 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.664999+0000", 2024-09-14T06:30:12.721 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.4430000000000003e-06 2024-09-14T06:30:12.722 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:12.722 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:12.722 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:30:12.722 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.665006+0000", 2024-09-14T06:30:12.722 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.0840000000000003e-06 2024-09-14T06:30:12.722 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:12.722 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:12.722 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:30:12.722 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.711402+0000", 2024-09-14T06:30:12.722 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046395714999999997 2024-09-14T06:30:12.722 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:12.722 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:12.722 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:30:12.722 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.711443+0000", 2024-09-14T06:30:12.723 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.0911999999999997e-05 2024-09-14T06:30:12.723 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:12.723 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:12.723 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:30:12.723 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.711501+0000", 2024-09-14T06:30:12.723 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.8526999999999997e-05 2024-09-14T06:30:12.723 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:12.723 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:30:12.723 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:12.723 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:12.723 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:30:12.723 INFO:teuthology.orchestra.run.smithi138.stdout:} 2024-09-14T06:30:12.725 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-14T06:30:12.725 INFO:tasks.ceph.osd.5.smithi138.stderr:2024-09-14T06:30:12.724+0000 7fe13746f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33618) UID: 0 2024-09-14T06:30:12.826 INFO:tasks.ceph.osd.2.smithi094.stderr:2024-09-14T06:30:12.825+0000 7faf3269b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33649) UID: 0 2024-09-14T06:30:12.857 INFO:teuthology.orchestra.run.smithi094.stderr:noscrub is set 2024-09-14T06:30:12.871 INFO:teuthology.orchestra.run.smithi138.stdout:{ 2024-09-14T06:30:12.871 INFO:teuthology.orchestra.run.smithi138.stdout: "ops": [], 2024-09-14T06:30:12.871 INFO:teuthology.orchestra.run.smithi138.stdout: "num_ops": 0 2024-09-14T06:30:12.871 INFO:teuthology.orchestra.run.smithi138.stdout:} 2024-09-14T06:30:12.880 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-14T06:30:12.927 INFO:tasks.ceph.osd.3.smithi094.stderr:2024-09-14T06:30:12.926+0000 7feeeafe8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33655) UID: 0 2024-09-14T06:30:12.982 INFO:teuthology.orchestra.run.smithi138.stdout:{ 2024-09-14T06:30:12.982 INFO:teuthology.orchestra.run.smithi138.stdout: "ops": [], 2024-09-14T06:30:12.982 INFO:teuthology.orchestra.run.smithi138.stdout: "complaint_time": 30, 2024-09-14T06:30:12.982 INFO:teuthology.orchestra.run.smithi138.stdout: "num_blocked_ops": 0 2024-09-14T06:30:12.982 INFO:teuthology.orchestra.run.smithi138.stdout:} 2024-09-14T06:30:12.991 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-14T06:30:13.028 INFO:tasks.ceph.osd.3.smithi094.stderr:2024-09-14T06:30:13.027+0000 7feeeafe8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33655) UID: 0 2024-09-14T06:30:13.095 INFO:teuthology.orchestra.run.smithi138.stdout:{ 2024-09-14T06:30:13.095 INFO:teuthology.orchestra.run.smithi138.stdout: "size": 20, 2024-09-14T06:30:13.095 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 600, 2024-09-14T06:30:13.095 INFO:teuthology.orchestra.run.smithi138.stdout: "ops": [ 2024-09-14T06:30:13.095 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:13.095 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:98505 3.10 3.1bf56f90 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T06:30:13.095 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:30:07.045453+0000", 2024-09-14T06:30:13.095 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 6.0479606959999996, 2024-09-14T06:30:13.096 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032144440000000003, 2024-09-14T06:30:13.096 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:30:13.096 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:30:13.096 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "reached pg", 2024-09-14T06:30:13.096 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:30:13.096 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:30:13.096 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:13.096 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 98505 2024-09-14T06:30:13.096 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:13.096 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:30:13.096 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:13.096 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:30:13.096 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045453+0000", 2024-09-14T06:30:13.097 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:13.097 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:13.097 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:13.097 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:30:13.097 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045453+0000", 2024-09-14T06:30:13.097 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:13.097 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:13.097 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:13.097 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:30:13.097 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045453+0000", 2024-09-14T06:30:13.097 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.36e-07 2024-09-14T06:30:13.097 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:13.097 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:13.098 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:30:13.098 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045454+0000", 2024-09-14T06:30:13.098 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.1299999999999998e-07 2024-09-14T06:30:13.098 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:13.098 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:13.098 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:30:13.098 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045457+0000", 2024-09-14T06:30:13.098 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.8559999999999998e-06 2024-09-14T06:30:13.098 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:13.098 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:13.098 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:30:13.098 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045460+0000", 2024-09-14T06:30:13.098 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.8750000000000002e-06 2024-09-14T06:30:13.098 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:13.099 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:13.099 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:30:13.099 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.077592+0000", 2024-09-14T06:30:13.099 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032131727999999998 2024-09-14T06:30:13.099 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:13.099 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:13.099 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:30:13.099 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.077597+0000", 2024-09-14T06:30:13.099 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.8319999999999996e-06 2024-09-14T06:30:13.099 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:13.099 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:30:13.099 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:13.099 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:13.099 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:13.099 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:98506 3.10 3.1bf56f90 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T06:30:13.100 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:30:07.045481+0000", 2024-09-14T06:30:13.100 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 6.0479324410000004, 2024-09-14T06:30:13.100 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032142759999999999, 2024-09-14T06:30:13.100 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:30:13.100 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:30:13.100 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "reached pg", 2024-09-14T06:30:13.100 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:30:13.100 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:30:13.100 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:13.100 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 98506 2024-09-14T06:30:13.100 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:13.100 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:30:13.100 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:13.101 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:30:13.101 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045481+0000", 2024-09-14T06:30:13.101 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:13.101 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:13.101 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:13.101 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:30:13.101 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045481+0000", 2024-09-14T06:30:13.101 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:13.101 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:13.101 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:13.101 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:30:13.101 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045481+0000", 2024-09-14T06:30:13.101 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.8500000000000001e-07 2024-09-14T06:30:13.101 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:13.102 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:13.102 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:30:13.102 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045483+0000", 2024-09-14T06:30:13.102 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.8250000000000001e-06 2024-09-14T06:30:13.102 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:13.102 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:13.102 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:30:13.102 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045489+0000", 2024-09-14T06:30:13.102 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.5740000000000003e-06 2024-09-14T06:30:13.102 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:13.102 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:13.102 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:30:13.102 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045495+0000", 2024-09-14T06:30:13.102 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.7690000000000002e-06 2024-09-14T06:30:13.103 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:13.103 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:13.103 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:30:13.103 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.077619+0000", 2024-09-14T06:30:13.103 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032124621999999999 2024-09-14T06:30:13.103 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:13.103 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:13.103 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:30:13.103 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.077624+0000", 2024-09-14T06:30:13.103 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.3850000000000004e-06 2024-09-14T06:30:13.103 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:13.103 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:30:13.103 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:13.103 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:13.104 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:13.104 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:98507 3.10 3.1bf56f90 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T06:30:13.104 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:30:07.045502+0000", 2024-09-14T06:30:13.104 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 6.0479110199999999, 2024-09-14T06:30:13.104 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032147219999999997, 2024-09-14T06:30:13.104 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:30:13.104 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:30:13.104 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "reached pg", 2024-09-14T06:30:13.104 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:30:13.104 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:30:13.104 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:13.104 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 98507 2024-09-14T06:30:13.104 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:13.105 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:30:13.105 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:13.105 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:30:13.105 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045502+0000", 2024-09-14T06:30:13.105 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:13.105 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:13.105 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:13.105 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:30:13.105 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045502+0000", 2024-09-14T06:30:13.105 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:13.105 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:13.105 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:13.105 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:30:13.105 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045503+0000", 2024-09-14T06:30:13.106 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.4700000000000002e-07 2024-09-14T06:30:13.106 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:13.106 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:13.106 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:30:13.106 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045509+0000", 2024-09-14T06:30:13.106 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.7329999999999997e-06 2024-09-14T06:30:13.106 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:13.106 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:13.106 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:30:13.106 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045515+0000", 2024-09-14T06:30:13.106 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.8030000000000002e-06 2024-09-14T06:30:13.106 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:13.106 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:13.106 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:30:13.107 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045522+0000", 2024-09-14T06:30:13.107 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.0369999999999997e-06 2024-09-14T06:30:13.107 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:13.107 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:13.107 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:30:13.107 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.077641+0000", 2024-09-14T06:30:13.107 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032119091000000002 2024-09-14T06:30:13.107 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:13.107 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:13.107 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:30:13.107 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.077650+0000", 2024-09-14T06:30:13.107 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.2090000000000005e-06 2024-09-14T06:30:13.107 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:13.108 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:30:13.108 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:13.108 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:13.108 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:13.108 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:98508 3.10 3.1bf56f90 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T06:30:13.108 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:30:07.045530+0000", 2024-09-14T06:30:13.108 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 6.0478834030000002, 2024-09-14T06:30:13.108 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032198773999999999, 2024-09-14T06:30:13.108 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:30:13.108 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:30:13.108 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "reached pg", 2024-09-14T06:30:13.108 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:30:13.108 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:30:13.109 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:13.109 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 98508 2024-09-14T06:30:13.109 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:13.109 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:30:13.109 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:13.109 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:30:13.109 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045530+0000", 2024-09-14T06:30:13.109 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:13.109 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:13.109 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:13.109 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:30:13.109 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045530+0000", 2024-09-14T06:30:13.109 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:13.109 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:13.110 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:13.110 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:30:13.110 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045530+0000", 2024-09-14T06:30:13.110 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.3099999999999998e-07 2024-09-14T06:30:13.110 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:13.110 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:13.110 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:30:13.110 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045539+0000", 2024-09-14T06:30:13.110 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.4780000000000008e-06 2024-09-14T06:30:13.110 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:13.110 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:13.111 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:30:13.111 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045546+0000", 2024-09-14T06:30:13.111 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.3259999999999998e-06 2024-09-14T06:30:13.111 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:13.111 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:13.111 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:30:13.111 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045551+0000", 2024-09-14T06:30:13.111 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.9960000000000001e-06 2024-09-14T06:30:13.111 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:13.111 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:13.111 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:30:13.112 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.077716+0000", 2024-09-14T06:30:13.112 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032165081999999998 2024-09-14T06:30:13.112 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:13.112 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:13.112 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:30:13.112 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.077729+0000", 2024-09-14T06:30:13.112 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.2461e-05 2024-09-14T06:30:13.112 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:13.112 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:30:13.112 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:13.112 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:13.112 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:13.112 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:98509 3.10 3.1bf56f90 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T06:30:13.112 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:30:07.045558+0000", 2024-09-14T06:30:13.113 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 6.047855202, 2024-09-14T06:30:13.113 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032242198, 2024-09-14T06:30:13.113 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:30:13.113 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:30:13.113 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "reached pg", 2024-09-14T06:30:13.113 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:30:13.113 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:30:13.113 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:13.113 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 98509 2024-09-14T06:30:13.113 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:13.113 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:30:13.113 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:13.113 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:30:13.114 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045558+0000", 2024-09-14T06:30:13.114 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:13.114 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:13.114 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:13.114 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:30:13.114 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045558+0000", 2024-09-14T06:30:13.114 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:13.114 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:13.114 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:13.114 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:30:13.114 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045559+0000", 2024-09-14T06:30:13.114 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.27e-07 2024-09-14T06:30:13.114 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:13.115 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:13.115 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:30:13.115 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045560+0000", 2024-09-14T06:30:13.115 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.6959999999999999e-06 2024-09-14T06:30:13.115 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:13.115 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:13.115 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:30:13.115 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045567+0000", 2024-09-14T06:30:13.115 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.6900000000000003e-06 2024-09-14T06:30:13.115 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:13.115 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:13.115 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:30:13.115 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045571+0000", 2024-09-14T06:30:13.115 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.5750000000000002e-06 2024-09-14T06:30:13.116 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:13.116 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:13.116 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:30:13.116 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.077789+0000", 2024-09-14T06:30:13.116 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032217186000000002 2024-09-14T06:30:13.116 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:13.116 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:13.116 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:30:13.116 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.077800+0000", 2024-09-14T06:30:13.116 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.1623999999999999e-05 2024-09-14T06:30:13.116 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:13.116 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:30:13.116 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:13.117 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:13.117 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:13.117 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:98510 3.10 3.1bf56f90 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T06:30:13.117 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:30:07.045580+0000", 2024-09-14T06:30:13.117 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 6.0478333879999999, 2024-09-14T06:30:13.117 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032299845000000001, 2024-09-14T06:30:13.117 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:30:13.117 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:30:13.117 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "reached pg", 2024-09-14T06:30:13.117 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:30:13.117 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:30:13.117 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:13.117 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 98510 2024-09-14T06:30:13.118 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:13.118 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:30:13.118 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:13.118 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:30:13.118 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045580+0000", 2024-09-14T06:30:13.118 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:13.118 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:13.118 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:13.118 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:30:13.118 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045580+0000", 2024-09-14T06:30:13.118 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:13.118 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:13.118 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:13.118 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:30:13.119 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045580+0000", 2024-09-14T06:30:13.119 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.0400000000000002e-07 2024-09-14T06:30:13.119 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:13.119 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:13.119 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:30:13.119 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045582+0000", 2024-09-14T06:30:13.119 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.7600000000000001e-06 2024-09-14T06:30:13.119 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:13.119 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:13.119 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:30:13.119 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045588+0000", 2024-09-14T06:30:13.119 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.5650000000000001e-06 2024-09-14T06:30:13.119 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:13.119 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:13.120 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:30:13.120 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045597+0000", 2024-09-14T06:30:13.120 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.465e-06 2024-09-14T06:30:13.120 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:13.120 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:13.120 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:30:13.120 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.077871+0000", 2024-09-14T06:30:13.120 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032273562999999998 2024-09-14T06:30:13.120 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:13.120 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:13.120 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:30:13.120 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.077880+0000", 2024-09-14T06:30:13.120 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.0880000000000001e-06 2024-09-14T06:30:13.120 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:13.121 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:30:13.121 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:13.121 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:13.121 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:13.121 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:98511 3.10 3.1bf56f90 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T06:30:13.121 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:30:07.045610+0000", 2024-09-14T06:30:13.121 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 6.0478035410000004, 2024-09-14T06:30:13.121 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032338894, 2024-09-14T06:30:13.121 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:30:13.121 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:30:13.121 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "reached pg", 2024-09-14T06:30:13.121 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:30:13.121 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:30:13.122 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:13.122 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 98511 2024-09-14T06:30:13.122 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:13.122 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:30:13.122 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:13.122 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:30:13.122 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045610+0000", 2024-09-14T06:30:13.122 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:13.122 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:13.122 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:13.122 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:30:13.122 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045610+0000", 2024-09-14T06:30:13.122 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:13.122 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:13.123 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:13.123 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:30:13.123 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045610+0000", 2024-09-14T06:30:13.123 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.1600000000000002e-07 2024-09-14T06:30:13.123 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:13.123 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:13.123 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:30:13.123 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045612+0000", 2024-09-14T06:30:13.123 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.1150000000000001e-06 2024-09-14T06:30:13.123 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:13.123 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:13.123 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:30:13.123 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045618+0000", 2024-09-14T06:30:13.124 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.0499999999999997e-06 2024-09-14T06:30:13.124 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:13.124 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:13.124 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:30:13.124 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045623+0000", 2024-09-14T06:30:13.124 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.6770000000000003e-06 2024-09-14T06:30:13.124 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:13.124 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:13.124 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:30:13.124 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.077928+0000", 2024-09-14T06:30:13.124 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032305111999999997 2024-09-14T06:30:13.124 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:13.124 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:13.124 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:30:13.125 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.077949+0000", 2024-09-14T06:30:13.125 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.0523999999999999e-05 2024-09-14T06:30:13.125 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:13.125 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:30:13.125 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:13.125 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:13.125 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:13.125 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:98512 3.10 3.1bf56f90 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T06:30:13.125 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:30:07.045630+0000", 2024-09-14T06:30:13.125 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 6.0477835359999998, 2024-09-14T06:30:13.125 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032346632, 2024-09-14T06:30:13.125 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:30:13.125 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:30:13.126 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "reached pg", 2024-09-14T06:30:13.126 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:30:13.126 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:30:13.126 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:13.126 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 98512 2024-09-14T06:30:13.126 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:13.126 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:30:13.126 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:13.126 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:30:13.126 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045630+0000", 2024-09-14T06:30:13.126 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:13.126 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:13.126 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:13.126 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:30:13.127 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045630+0000", 2024-09-14T06:30:13.127 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:13.127 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:13.127 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:13.127 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:30:13.127 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045630+0000", 2024-09-14T06:30:13.127 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.8200000000000001e-07 2024-09-14T06:30:13.127 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:13.127 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:13.127 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:30:13.127 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045632+0000", 2024-09-14T06:30:13.127 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.7430000000000001e-06 2024-09-14T06:30:13.127 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:13.127 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:13.128 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:30:13.128 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045637+0000", 2024-09-14T06:30:13.128 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.4550000000000003e-06 2024-09-14T06:30:13.128 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:13.128 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:13.128 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:30:13.128 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045644+0000", 2024-09-14T06:30:13.128 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.1330000000000001e-06 2024-09-14T06:30:13.128 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:13.128 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:13.128 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:30:13.128 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.077968+0000", 2024-09-14T06:30:13.128 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032323401000000002 2024-09-14T06:30:13.129 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:13.129 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:13.129 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:30:13.129 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.077976+0000", 2024-09-14T06:30:13.129 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.5180000000000004e-06 2024-09-14T06:30:13.129 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:13.129 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:30:13.129 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:13.129 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:13.129 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:13.129 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:98513 3.10 3.1bf56f90 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T06:30:13.129 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:30:07.045652+0000", 2024-09-14T06:30:13.129 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 6.0477609719999998, 2024-09-14T06:30:13.129 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032353436999999999, 2024-09-14T06:30:13.130 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:30:13.130 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:30:13.130 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "reached pg", 2024-09-14T06:30:13.130 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:30:13.130 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:30:13.130 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:13.130 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 98513 2024-09-14T06:30:13.130 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:13.130 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:30:13.130 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:13.130 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:30:13.130 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045652+0000", 2024-09-14T06:30:13.130 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:13.130 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:13.131 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:13.131 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:30:13.131 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045652+0000", 2024-09-14T06:30:13.131 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:13.131 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:13.131 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:13.131 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:30:13.131 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045653+0000", 2024-09-14T06:30:13.131 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.3700000000000001e-07 2024-09-14T06:30:13.131 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:13.131 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:13.131 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:30:13.131 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045655+0000", 2024-09-14T06:30:13.131 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.854e-06 2024-09-14T06:30:13.131 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:13.131 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:13.132 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:30:13.132 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045661+0000", 2024-09-14T06:30:13.132 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.5509999999999996e-06 2024-09-14T06:30:13.132 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:13.132 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:13.132 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:30:13.132 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045668+0000", 2024-09-14T06:30:13.132 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.1980000000000004e-06 2024-09-14T06:30:13.132 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:13.132 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:13.132 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:30:13.132 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.077997+0000", 2024-09-14T06:30:13.132 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032328575999999998 2024-09-14T06:30:13.132 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:13.132 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:13.132 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:30:13.133 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.078006+0000", 2024-09-14T06:30:13.133 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.8210000000000008e-06 2024-09-14T06:30:13.133 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:13.133 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:30:13.133 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:13.133 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:13.133 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:13.133 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:98514 3.10 3.1bf56f90 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T06:30:13.133 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:30:07.045699+0000", 2024-09-14T06:30:13.133 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 6.0477146370000003, 2024-09-14T06:30:13.133 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032330223999999998, 2024-09-14T06:30:13.133 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:30:13.133 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:30:13.134 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "reached pg", 2024-09-14T06:30:13.134 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:30:13.134 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:30:13.134 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:13.134 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 98514 2024-09-14T06:30:13.134 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:13.134 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:30:13.134 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:13.134 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:30:13.134 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045699+0000", 2024-09-14T06:30:13.134 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:13.134 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:13.134 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:13.134 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:30:13.135 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045699+0000", 2024-09-14T06:30:13.135 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:13.135 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:13.135 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:13.135 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:30:13.135 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045699+0000", 2024-09-14T06:30:13.135 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.3200000000000001e-07 2024-09-14T06:30:13.135 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:13.135 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:13.135 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:30:13.135 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045701+0000", 2024-09-14T06:30:13.135 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.762e-06 2024-09-14T06:30:13.135 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:13.135 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:13.136 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:30:13.136 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045705+0000", 2024-09-14T06:30:13.136 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.2359999999999998e-06 2024-09-14T06:30:13.136 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:13.136 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:13.136 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:30:13.136 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045708+0000", 2024-09-14T06:30:13.136 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.796e-06 2024-09-14T06:30:13.136 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:13.136 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:13.136 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:30:13.136 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.078023+0000", 2024-09-14T06:30:13.136 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032315213000000002 2024-09-14T06:30:13.136 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:13.137 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:13.137 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:30:13.137 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.078029+0000", 2024-09-14T06:30:13.137 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.8850000000000004e-06 2024-09-14T06:30:13.137 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:13.137 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:30:13.137 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:13.137 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:13.137 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:13.137 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:98515 3.10 3.1bf56f90 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T06:30:13.137 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:30:07.045712+0000", 2024-09-14T06:30:13.137 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 6.0477013260000003, 2024-09-14T06:30:13.137 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032346871999999999, 2024-09-14T06:30:13.137 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:30:13.137 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:30:13.138 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "reached pg", 2024-09-14T06:30:13.138 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:30:13.138 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:30:13.138 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:13.138 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 98515 2024-09-14T06:30:13.138 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:13.138 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:30:13.138 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:13.138 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:30:13.138 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045712+0000", 2024-09-14T06:30:13.138 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:13.138 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:13.138 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:13.139 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:30:13.139 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045712+0000", 2024-09-14T06:30:13.139 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:13.139 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:13.139 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:13.139 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:30:13.139 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045712+0000", 2024-09-14T06:30:13.139 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.2100000000000001e-07 2024-09-14T06:30:13.139 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:13.139 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:13.139 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:30:13.139 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045713+0000", 2024-09-14T06:30:13.139 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.068e-06 2024-09-14T06:30:13.139 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:13.140 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:13.140 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:30:13.140 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045716+0000", 2024-09-14T06:30:13.140 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.971e-06 2024-09-14T06:30:13.140 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:13.140 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:13.140 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:30:13.140 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045719+0000", 2024-09-14T06:30:13.140 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.3879999999999998e-06 2024-09-14T06:30:13.140 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:13.140 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:13.140 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:30:13.140 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.078049+0000", 2024-09-14T06:30:13.140 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032330553999999997 2024-09-14T06:30:13.141 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:13.141 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:13.141 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:30:13.141 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.078059+0000", 2024-09-14T06:30:13.141 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.6700000000000006e-06 2024-09-14T06:30:13.141 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:13.141 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:30:13.141 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:13.141 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:13.141 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:13.141 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:98516 3.10 3.1bf56f90 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T06:30:13.141 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:30:07.045722+0000", 2024-09-14T06:30:13.141 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 6.0476915729999998, 2024-09-14T06:30:13.142 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032374334999999997, 2024-09-14T06:30:13.142 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:30:13.142 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:30:13.142 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "reached pg", 2024-09-14T06:30:13.142 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:30:13.142 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:30:13.142 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:13.142 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 98516 2024-09-14T06:30:13.142 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:13.142 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:30:13.142 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:13.142 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:30:13.142 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045722+0000", 2024-09-14T06:30:13.142 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:13.143 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:13.143 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:13.143 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:30:13.143 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045722+0000", 2024-09-14T06:30:13.143 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:13.143 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:13.143 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:13.143 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:30:13.143 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045722+0000", 2024-09-14T06:30:13.143 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.8999999999999998e-07 2024-09-14T06:30:13.143 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:13.143 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:13.143 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:30:13.143 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045723+0000", 2024-09-14T06:30:13.144 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.0049999999999999e-06 2024-09-14T06:30:13.144 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:13.144 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:13.144 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:30:13.144 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045726+0000", 2024-09-14T06:30:13.144 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.9019999999999999e-06 2024-09-14T06:30:13.144 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:13.144 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:13.144 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:30:13.144 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045728+0000", 2024-09-14T06:30:13.144 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.3290000000000001e-06 2024-09-14T06:30:13.144 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:13.144 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:13.144 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:30:13.145 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.078090+0000", 2024-09-14T06:30:13.145 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032362123 2024-09-14T06:30:13.145 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:13.145 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:13.145 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:30:13.145 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.078096+0000", 2024-09-14T06:30:13.145 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.6860000000000003e-06 2024-09-14T06:30:13.145 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:13.145 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:30:13.145 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:13.145 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:13.145 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:13.145 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:98517 3.10 3.1bf56f90 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T06:30:13.146 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:30:07.045731+0000", 2024-09-14T06:30:13.146 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 6.0476818420000003, 2024-09-14T06:30:13.146 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.03241227, 2024-09-14T06:30:13.146 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:30:13.146 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:30:13.146 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "reached pg", 2024-09-14T06:30:13.146 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:30:13.146 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:30:13.146 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:13.146 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 98517 2024-09-14T06:30:13.146 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:13.146 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:30:13.146 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:13.146 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:30:13.147 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045731+0000", 2024-09-14T06:30:13.147 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:13.147 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:13.147 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:13.147 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:30:13.147 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045731+0000", 2024-09-14T06:30:13.147 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:13.147 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:13.147 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:13.147 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:30:13.147 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045732+0000", 2024-09-14T06:30:13.147 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.0699999999999999e-07 2024-09-14T06:30:13.147 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:13.147 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:13.148 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:30:13.148 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045733+0000", 2024-09-14T06:30:13.148 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.9199999999999999e-07 2024-09-14T06:30:13.148 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:13.148 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:13.148 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:30:13.148 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045735+0000", 2024-09-14T06:30:13.148 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.8150000000000002e-06 2024-09-14T06:30:13.148 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:13.148 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:13.148 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:30:13.148 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045738+0000", 2024-09-14T06:30:13.148 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.3640000000000002e-06 2024-09-14T06:30:13.149 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:13.149 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:13.149 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:30:13.149 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.078126+0000", 2024-09-14T06:30:13.149 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032388206000000003 2024-09-14T06:30:13.149 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:13.149 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:13.149 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:30:13.149 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.078144+0000", 2024-09-14T06:30:13.149 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.7686e-05 2024-09-14T06:30:13.149 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:13.149 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:30:13.149 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:13.150 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:13.150 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:13.150 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:98518 3.10 3.1bf56f90 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T06:30:13.150 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:30:07.045741+0000", 2024-09-14T06:30:13.150 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 6.0476724470000001, 2024-09-14T06:30:13.150 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032451793999999999, 2024-09-14T06:30:13.150 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:30:13.150 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:30:13.150 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "reached pg", 2024-09-14T06:30:13.150 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:30:13.150 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:30:13.150 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:13.150 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 98518 2024-09-14T06:30:13.151 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:13.151 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:30:13.151 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:13.151 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:30:13.151 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045741+0000", 2024-09-14T06:30:13.151 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:13.151 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:13.151 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:13.151 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:30:13.151 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045741+0000", 2024-09-14T06:30:13.151 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:13.151 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:13.151 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:13.151 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:30:13.152 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045741+0000", 2024-09-14T06:30:13.152 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.05e-07 2024-09-14T06:30:13.152 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:13.152 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:13.152 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:30:13.152 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045746+0000", 2024-09-14T06:30:13.152 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.2259999999999996e-06 2024-09-14T06:30:13.152 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:13.152 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:13.152 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:30:13.152 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045750+0000", 2024-09-14T06:30:13.152 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.2490000000000001e-06 2024-09-14T06:30:13.152 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:13.152 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:13.153 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:30:13.153 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045752+0000", 2024-09-14T06:30:13.153 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.3980000000000002e-06 2024-09-14T06:30:13.153 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:13.153 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:13.153 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:30:13.153 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.078184+0000", 2024-09-14T06:30:13.153 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032431593000000002 2024-09-14T06:30:13.153 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:13.153 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:13.153 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:30:13.153 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.078193+0000", 2024-09-14T06:30:13.153 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.1230000000000006e-06 2024-09-14T06:30:13.153 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:13.154 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:30:13.154 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:13.154 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:13.154 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:13.154 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:98519 3.10 3.1bf56f90 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T06:30:13.154 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:30:07.045755+0000", 2024-09-14T06:30:13.154 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 6.0476580220000002, 2024-09-14T06:30:13.154 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032489431999999999, 2024-09-14T06:30:13.154 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:30:13.154 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:30:13.154 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "reached pg", 2024-09-14T06:30:13.154 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:30:13.154 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:30:13.154 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:13.155 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 98519 2024-09-14T06:30:13.155 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:13.155 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:30:13.155 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:13.155 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:30:13.155 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045755+0000", 2024-09-14T06:30:13.155 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:13.155 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:13.155 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:13.155 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:30:13.155 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045755+0000", 2024-09-14T06:30:13.155 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:13.155 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:13.155 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:13.155 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:30:13.156 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045756+0000", 2024-09-14T06:30:13.156 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.23e-07 2024-09-14T06:30:13.156 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:13.156 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:13.156 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:30:13.156 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045757+0000", 2024-09-14T06:30:13.156 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.1060000000000001e-06 2024-09-14T06:30:13.156 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:13.156 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:13.156 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:30:13.156 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045760+0000", 2024-09-14T06:30:13.156 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.988e-06 2024-09-14T06:30:13.156 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:13.156 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:13.157 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:30:13.157 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045765+0000", 2024-09-14T06:30:13.157 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.2689999999999999e-06 2024-09-14T06:30:13.157 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:13.157 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:13.157 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:30:13.157 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.078234+0000", 2024-09-14T06:30:13.157 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032468915000000001 2024-09-14T06:30:13.157 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:13.157 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:13.157 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:30:13.157 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.078245+0000", 2024-09-14T06:30:13.157 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.0930999999999999e-05 2024-09-14T06:30:13.158 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:13.158 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:30:13.158 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:13.158 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:13.158 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:13.158 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:98520 3.10 3.1bf56f90 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T06:30:13.158 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:30:07.045768+0000", 2024-09-14T06:30:13.158 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 6.0476451439999996, 2024-09-14T06:30:13.158 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032531379999999999, 2024-09-14T06:30:13.158 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:30:13.158 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:30:13.158 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "reached pg", 2024-09-14T06:30:13.158 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:30:13.158 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:30:13.159 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:13.159 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 98520 2024-09-14T06:30:13.159 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:13.159 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:30:13.159 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:13.159 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:30:13.159 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045768+0000", 2024-09-14T06:30:13.159 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:13.159 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:13.159 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:13.159 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:30:13.159 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045768+0000", 2024-09-14T06:30:13.159 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:13.159 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:13.160 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:13.160 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:30:13.160 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045768+0000", 2024-09-14T06:30:13.160 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.22e-07 2024-09-14T06:30:13.160 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:13.160 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:13.160 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:30:13.160 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045773+0000", 2024-09-14T06:30:13.160 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.3470000000000001e-06 2024-09-14T06:30:13.160 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:13.160 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:13.160 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:30:13.160 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045776+0000", 2024-09-14T06:30:13.160 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.1319999999999998e-06 2024-09-14T06:30:13.161 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:13.161 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:13.161 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:30:13.161 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045779+0000", 2024-09-14T06:30:13.161 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.9749999999999999e-06 2024-09-14T06:30:13.161 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:13.161 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:13.161 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:30:13.161 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.078289+0000", 2024-09-14T06:30:13.161 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032509693999999999 2024-09-14T06:30:13.161 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:13.161 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:13.161 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:30:13.162 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.078300+0000", 2024-09-14T06:30:13.162 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.101e-05 2024-09-14T06:30:13.162 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:13.162 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:30:13.162 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:13.162 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:13.162 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:13.162 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:98521 3.10 3.1bf56f90 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T06:30:13.162 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:30:07.045782+0000", 2024-09-14T06:30:13.162 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 6.0476309639999997, 2024-09-14T06:30:13.162 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032566109000000003, 2024-09-14T06:30:13.162 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:30:13.162 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:30:13.163 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "reached pg", 2024-09-14T06:30:13.163 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:30:13.163 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:30:13.163 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:13.163 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 98521 2024-09-14T06:30:13.163 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:13.163 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:30:13.163 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:13.163 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:30:13.163 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045782+0000", 2024-09-14T06:30:13.163 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:13.163 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:13.163 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:13.163 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:30:13.164 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045782+0000", 2024-09-14T06:30:13.164 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:13.164 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:13.164 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:13.164 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:30:13.164 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045783+0000", 2024-09-14T06:30:13.164 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.2499999999999999e-07 2024-09-14T06:30:13.164 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:13.164 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:13.164 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:30:13.164 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045784+0000", 2024-09-14T06:30:13.164 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.0109999999999999e-06 2024-09-14T06:30:13.164 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:13.164 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:13.164 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:30:13.164 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045786+0000", 2024-09-14T06:30:13.165 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.8289999999999999e-06 2024-09-14T06:30:13.165 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:13.165 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:13.165 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:30:13.165 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045793+0000", 2024-09-14T06:30:13.165 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.579e-06 2024-09-14T06:30:13.165 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:13.165 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:13.165 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:30:13.165 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.078338+0000", 2024-09-14T06:30:13.165 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032545389000000001 2024-09-14T06:30:13.165 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:13.165 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:13.166 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:30:13.166 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.078348+0000", 2024-09-14T06:30:13.166 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.0076e-05 2024-09-14T06:30:13.166 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:13.166 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:30:13.166 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:13.166 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:13.166 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:13.166 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:98522 3.10 3.1bf56f90 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T06:30:13.166 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:30:07.045796+0000", 2024-09-14T06:30:13.166 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 6.047617056, 2024-09-14T06:30:13.166 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032577846000000001, 2024-09-14T06:30:13.166 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:30:13.166 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:30:13.167 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "reached pg", 2024-09-14T06:30:13.167 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:30:13.167 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:30:13.167 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:13.167 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 98522 2024-09-14T06:30:13.167 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:13.167 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:30:13.167 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:13.167 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:30:13.167 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045796+0000", 2024-09-14T06:30:13.167 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:13.167 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:13.167 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:13.167 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:30:13.168 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045796+0000", 2024-09-14T06:30:13.168 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:13.168 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:13.168 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:13.168 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:30:13.168 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045796+0000", 2024-09-14T06:30:13.168 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.2100000000000001e-07 2024-09-14T06:30:13.168 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:13.168 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:13.168 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:30:13.168 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045797+0000", 2024-09-14T06:30:13.168 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.6899999999999996e-07 2024-09-14T06:30:13.168 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:13.169 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:13.169 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:30:13.169 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045800+0000", 2024-09-14T06:30:13.169 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.9979999999999999e-06 2024-09-14T06:30:13.169 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:13.169 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:13.169 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:30:13.169 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045803+0000", 2024-09-14T06:30:13.169 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.2850000000000001e-06 2024-09-14T06:30:13.169 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:13.169 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:13.169 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:30:13.169 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.078366+0000", 2024-09-14T06:30:13.170 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032562932000000003 2024-09-14T06:30:13.170 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:13.170 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:13.170 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:30:13.170 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.078374+0000", 2024-09-14T06:30:13.170 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.4409999999999993e-06 2024-09-14T06:30:13.170 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:13.170 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:30:13.170 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:13.170 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:13.170 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:13.170 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:98523 3.10 3.1bf56f90 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T06:30:13.170 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:30:07.045806+0000", 2024-09-14T06:30:13.170 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 6.047607513, 2024-09-14T06:30:13.170 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032611167000000003, 2024-09-14T06:30:13.171 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:30:13.171 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:30:13.171 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "reached pg", 2024-09-14T06:30:13.171 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:30:13.171 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:30:13.171 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:13.171 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 98523 2024-09-14T06:30:13.171 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:13.171 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:30:13.171 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:13.171 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:30:13.171 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045806+0000", 2024-09-14T06:30:13.171 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:13.171 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:13.171 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:13.171 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:30:13.172 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045806+0000", 2024-09-14T06:30:13.172 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:13.172 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:13.172 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:13.172 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:30:13.172 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045806+0000", 2024-09-14T06:30:13.172 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.2000000000000001e-07 2024-09-14T06:30:13.172 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:13.172 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:13.172 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:30:13.172 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045807+0000", 2024-09-14T06:30:13.172 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.4e-07 2024-09-14T06:30:13.172 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:13.172 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:13.172 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:30:13.173 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045810+0000", 2024-09-14T06:30:13.173 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.9229999999999998e-06 2024-09-14T06:30:13.173 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:13.173 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:13.173 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:30:13.173 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045812+0000", 2024-09-14T06:30:13.173 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.2220000000000001e-06 2024-09-14T06:30:13.173 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:13.173 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:13.173 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:30:13.173 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.078407+0000", 2024-09-14T06:30:13.173 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032594513999999998 2024-09-14T06:30:13.173 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:13.173 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:13.173 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:30:13.174 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.078417+0000", 2024-09-14T06:30:13.174 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.0348e-05 2024-09-14T06:30:13.174 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:13.174 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:30:13.174 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:13.174 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:13.174 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:13.174 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:98524 3.10 3.1bf56f90 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T06:30:13.174 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:30:07.045815+0000", 2024-09-14T06:30:13.174 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 6.0475981450000003, 2024-09-14T06:30:13.174 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032643208999999999, 2024-09-14T06:30:13.174 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:30:13.174 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:30:13.174 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "reached pg", 2024-09-14T06:30:13.174 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:30:13.174 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:30:13.175 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:13.175 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 98524 2024-09-14T06:30:13.175 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:13.175 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:30:13.175 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:13.175 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:30:13.175 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045815+0000", 2024-09-14T06:30:13.175 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:13.175 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:13.175 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:13.175 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:30:13.175 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045815+0000", 2024-09-14T06:30:13.175 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:13.175 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:13.176 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:13.176 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:30:13.176 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045815+0000", 2024-09-14T06:30:13.176 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.11e-07 2024-09-14T06:30:13.176 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:13.176 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:13.176 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:30:13.176 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045816+0000", 2024-09-14T06:30:13.176 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.9099999999999991e-07 2024-09-14T06:30:13.176 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:13.176 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:13.176 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:30:13.176 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045819+0000", 2024-09-14T06:30:13.176 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.9399999999999998e-06 2024-09-14T06:30:13.177 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:13.177 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:13.177 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:30:13.177 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045822+0000", 2024-09-14T06:30:13.177 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.3470000000000001e-06 2024-09-14T06:30:13.177 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:13.177 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:13.177 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:30:13.177 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.078449+0000", 2024-09-14T06:30:13.177 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032627529000000002 2024-09-14T06:30:13.177 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:13.177 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:13.177 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:30:13.178 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.078458+0000", 2024-09-14T06:30:13.178 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.1910000000000006e-06 2024-09-14T06:30:13.178 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:13.178 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:30:13.178 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:13.178 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:13.178 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:30:13.178 INFO:teuthology.orchestra.run.smithi138.stdout:} 2024-09-14T06:30:13.179 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-14T06:30:13.180 INFO:tasks.ceph.osd.0.smithi094.stderr:2024-09-14T06:30:13.178+0000 7f07c9333640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33636) UID: 0 2024-09-14T06:30:13.281 INFO:tasks.ceph.osd.2.smithi094.stderr:2024-09-14T06:30:13.280+0000 7faf3269b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33649) UID: 0 2024-09-14T06:30:13.283 INFO:teuthology.orchestra.run.smithi138.stdout:{ 2024-09-14T06:30:13.283 INFO:teuthology.orchestra.run.smithi138.stdout: "ops": [], 2024-09-14T06:30:13.283 INFO:teuthology.orchestra.run.smithi138.stdout: "num_ops": 0 2024-09-14T06:30:13.283 INFO:teuthology.orchestra.run.smithi138.stdout:} 2024-09-14T06:30:13.291 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-14T06:30:13.381 INFO:tasks.ceph.osd.4.smithi138.stderr:2024-09-14T06:30:13.380+0000 7f6c653c8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33615) UID: 0 2024-09-14T06:30:13.393 INFO:teuthology.orchestra.run.smithi138.stdout:{ 2024-09-14T06:30:13.393 INFO:teuthology.orchestra.run.smithi138.stdout: "ops": [], 2024-09-14T06:30:13.393 INFO:teuthology.orchestra.run.smithi138.stdout: "complaint_time": 30, 2024-09-14T06:30:13.393 INFO:teuthology.orchestra.run.smithi138.stdout: "num_blocked_ops": 0 2024-09-14T06:30:13.393 INFO:teuthology.orchestra.run.smithi138.stdout:} 2024-09-14T06:30:13.402 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-14T06:30:13.486 INFO:tasks.ceph.osd.4.smithi138.stderr:2024-09-14T06:30:13.485+0000 7f6c653c8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33615) UID: 0 2024-09-14T06:30:13.547 INFO:teuthology.orchestra.run.smithi138.stdout:{ 2024-09-14T06:30:13.547 INFO:teuthology.orchestra.run.smithi138.stdout: "size": 20, 2024-09-14T06:30:13.547 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 600, 2024-09-14T06:30:13.547 INFO:teuthology.orchestra.run.smithi138.stdout: "ops": [ 2024-09-14T06:30:13.547 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:13.547 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:53244 3.f 3:f71f51a7:::rbd_data.112dca2f6ae1.0000000000000015:17 [read 1863680~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:30:13.547 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:44.369399+0000", 2024-09-14T06:30:13.547 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 29.175627105, 2024-09-14T06:30:13.547 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047291189999999997, 2024-09-14T06:30:13.547 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:30:13.547 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:30:13.548 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:30:13.548 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:30:13.548 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:30:13.548 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:13.548 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 53244 2024-09-14T06:30:13.548 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:13.548 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:30:13.548 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:13.548 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:30:13.548 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369399+0000", 2024-09-14T06:30:13.548 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:13.548 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:13.548 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:13.548 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:30:13.549 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369399+0000", 2024-09-14T06:30:13.549 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:13.549 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:13.549 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:13.549 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:30:13.549 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369400+0000", 2024-09-14T06:30:13.549 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.0699999999999999e-07 2024-09-14T06:30:13.549 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:13.549 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:13.549 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:30:13.549 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369401+0000", 2024-09-14T06:30:13.549 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.4300000000000001e-06 2024-09-14T06:30:13.549 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:13.550 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:13.550 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:30:13.550 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369404+0000", 2024-09-14T06:30:13.550 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.061e-06 2024-09-14T06:30:13.550 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:13.550 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:13.550 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:30:13.550 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369407+0000", 2024-09-14T06:30:13.550 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.6139999999999999e-06 2024-09-14T06:30:13.550 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:13.550 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:13.550 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:30:13.550 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.416517+0000", 2024-09-14T06:30:13.550 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047110338000000002 2024-09-14T06:30:13.551 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:13.551 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:13.551 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:30:13.551 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.416616+0000", 2024-09-14T06:30:13.551 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.8651000000000001e-05 2024-09-14T06:30:13.551 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:13.551 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:13.551 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:30:13.551 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.416691+0000", 2024-09-14T06:30:13.551 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.4888999999999994e-05 2024-09-14T06:30:13.551 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:13.551 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:30:13.551 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:13.551 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:13.552 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:13.552 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:53245 3.f 3:f71f51a7:::rbd_data.112dca2f6ae1.0000000000000015:17 [read 1888256~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:30:13.552 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:44.369410+0000", 2024-09-14T06:30:13.552 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 29.175616817000002, 2024-09-14T06:30:13.552 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047399275999999997, 2024-09-14T06:30:13.552 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:30:13.552 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:30:13.552 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:30:13.552 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:30:13.552 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:30:13.552 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:13.552 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 53245 2024-09-14T06:30:13.552 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:13.553 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:30:13.553 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:13.553 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:30:13.553 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369410+0000", 2024-09-14T06:30:13.553 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:13.553 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:13.553 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:13.553 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:30:13.553 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369410+0000", 2024-09-14T06:30:13.553 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:13.553 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:13.553 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:13.553 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:30:13.553 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369410+0000", 2024-09-14T06:30:13.554 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.1899999999999999e-07 2024-09-14T06:30:13.554 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:13.554 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:13.554 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:30:13.554 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369411+0000", 2024-09-14T06:30:13.554 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.0300000000000001e-06 2024-09-14T06:30:13.554 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:13.554 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:13.554 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:30:13.554 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369414+0000", 2024-09-14T06:30:13.554 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.1099999999999999e-06 2024-09-14T06:30:13.554 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:13.554 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:13.555 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:30:13.555 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369417+0000", 2024-09-14T06:30:13.555 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.6189999999999998e-06 2024-09-14T06:30:13.555 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:13.555 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:13.555 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:30:13.555 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.416718+0000", 2024-09-14T06:30:13.555 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047301322999999999 2024-09-14T06:30:13.555 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:13.555 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:13.555 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:30:13.555 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.416759+0000", 2024-09-14T06:30:13.555 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.1009e-05 2024-09-14T06:30:13.555 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:13.555 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:13.556 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:30:13.556 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.416809+0000", 2024-09-14T06:30:13.556 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.9966000000000002e-05 2024-09-14T06:30:13.556 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:13.556 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:30:13.556 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:13.556 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:13.556 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:13.556 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:53246 3.f 3:f71f51a7:::rbd_data.112dca2f6ae1.0000000000000015:17 [read 1912832~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:30:13.556 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:44.369420+0000", 2024-09-14T06:30:13.556 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 29.175606873, 2024-09-14T06:30:13.556 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047529463000000001, 2024-09-14T06:30:13.556 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:30:13.556 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:30:13.557 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:30:13.557 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:30:13.557 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:30:13.557 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:13.557 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 53246 2024-09-14T06:30:13.557 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:13.557 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:30:13.557 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:13.557 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:30:13.557 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369420+0000", 2024-09-14T06:30:13.557 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:13.557 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:13.557 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:13.557 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:30:13.558 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369420+0000", 2024-09-14T06:30:13.558 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:13.558 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:13.558 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:13.558 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:30:13.558 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369420+0000", 2024-09-14T06:30:13.558 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.1299999999999999e-07 2024-09-14T06:30:13.558 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:13.558 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:13.558 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:30:13.558 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369425+0000", 2024-09-14T06:30:13.558 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.0130000000000001e-06 2024-09-14T06:30:13.558 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:13.558 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:13.559 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:30:13.559 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369428+0000", 2024-09-14T06:30:13.559 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.349e-06 2024-09-14T06:30:13.559 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:13.559 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:13.559 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:30:13.559 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369431+0000", 2024-09-14T06:30:13.559 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.7389999999999999e-06 2024-09-14T06:30:13.559 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:13.559 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:13.559 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:30:13.559 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.416834+0000", 2024-09-14T06:30:13.559 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047402669000000001 2024-09-14T06:30:13.559 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:13.560 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:13.560 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:30:13.560 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.416882+0000", 2024-09-14T06:30:13.560 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.8068999999999997e-05 2024-09-14T06:30:13.560 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:13.560 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:13.560 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:30:13.560 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.416949+0000", 2024-09-14T06:30:13.560 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.7410999999999995e-05 2024-09-14T06:30:13.560 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:13.560 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:30:13.560 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:13.560 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:13.560 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:13.561 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:53247 3.f 3:f71f51a7:::rbd_data.112dca2f6ae1.0000000000000015:17 [read 1937408~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:30:13.561 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:44.369434+0000", 2024-09-14T06:30:13.561 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 29.175592109, 2024-09-14T06:30:13.561 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047652264999999999, 2024-09-14T06:30:13.561 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:30:13.561 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:30:13.561 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:30:13.561 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:30:13.561 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:30:13.561 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:13.561 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 53247 2024-09-14T06:30:13.561 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:13.562 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:30:13.562 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:13.562 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:30:13.562 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369434+0000", 2024-09-14T06:30:13.562 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:13.562 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:13.562 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:13.562 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:30:13.562 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369434+0000", 2024-09-14T06:30:13.562 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:13.562 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:13.562 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:13.562 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:30:13.562 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369435+0000", 2024-09-14T06:30:13.563 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.2100000000000001e-07 2024-09-14T06:30:13.563 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:13.563 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:13.563 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:30:13.563 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369436+0000", 2024-09-14T06:30:13.563 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.122e-06 2024-09-14T06:30:13.563 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:13.563 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:13.563 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:30:13.563 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369439+0000", 2024-09-14T06:30:13.563 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.0400000000000001e-06 2024-09-14T06:30:13.563 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:13.563 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:13.564 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:30:13.564 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369441+0000", 2024-09-14T06:30:13.564 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.728e-06 2024-09-14T06:30:13.564 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:13.564 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:13.564 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:30:13.564 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.416977+0000", 2024-09-14T06:30:13.564 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047535344 2024-09-14T06:30:13.564 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:13.564 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:13.564 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:30:13.564 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.417029+0000", 2024-09-14T06:30:13.564 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.2037999999999999e-05 2024-09-14T06:30:13.564 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:13.565 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:13.565 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:30:13.565 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.417087+0000", 2024-09-14T06:30:13.565 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.7772000000000002e-05 2024-09-14T06:30:13.565 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:13.565 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:30:13.565 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:13.565 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:13.565 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:13.565 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:53248 3.f 3:f71f51a7:::rbd_data.112dca2f6ae1.0000000000000015:17 [read 1961984~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:30:13.565 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:44.369445+0000", 2024-09-14T06:30:13.565 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 29.175581557000001, 2024-09-14T06:30:13.565 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047798489, 2024-09-14T06:30:13.565 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:30:13.566 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:30:13.566 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:30:13.566 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:30:13.566 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:30:13.566 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:13.566 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 53248 2024-09-14T06:30:13.566 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:13.566 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:30:13.566 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:13.566 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:30:13.566 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369445+0000", 2024-09-14T06:30:13.567 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:13.567 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:13.567 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:13.567 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:30:13.567 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369445+0000", 2024-09-14T06:30:13.567 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:13.567 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:13.567 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:13.567 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:30:13.567 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369445+0000", 2024-09-14T06:30:13.567 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.0699999999999999e-07 2024-09-14T06:30:13.568 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:13.568 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:13.568 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:30:13.568 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369446+0000", 2024-09-14T06:30:13.568 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.1000000000000001e-06 2024-09-14T06:30:13.568 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:13.568 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:13.568 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:30:13.568 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369449+0000", 2024-09-14T06:30:13.568 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.1039999999999998e-06 2024-09-14T06:30:13.568 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:13.568 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:13.568 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:30:13.569 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369452+0000", 2024-09-14T06:30:13.569 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.6199999999999999e-06 2024-09-14T06:30:13.569 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:13.569 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:13.569 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:30:13.569 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.417114+0000", 2024-09-14T06:30:13.569 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047661901999999999 2024-09-14T06:30:13.569 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:13.569 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:13.569 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:30:13.569 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.417181+0000", 2024-09-14T06:30:13.569 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.7385999999999995e-05 2024-09-14T06:30:13.570 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:13.570 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:13.570 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:30:13.570 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.417243+0000", 2024-09-14T06:30:13.570 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.2169999999999996e-05 2024-09-14T06:30:13.570 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:13.570 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:30:13.570 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:13.570 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:13.570 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:13.570 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:53249 3.f 3:f71f51a7:::rbd_data.112dca2f6ae1.0000000000000015:17 [read 1986560~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:30:13.570 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:44.369455+0000", 2024-09-14T06:30:13.570 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 29.175571246000001, 2024-09-14T06:30:13.570 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047890493999999999, 2024-09-14T06:30:13.571 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:30:13.571 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:30:13.571 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:30:13.571 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:30:13.571 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:30:13.571 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:13.571 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 53249 2024-09-14T06:30:13.571 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:13.571 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:30:13.571 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:13.571 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:30:13.571 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369455+0000", 2024-09-14T06:30:13.571 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:13.571 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:13.572 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:13.572 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:30:13.572 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369455+0000", 2024-09-14T06:30:13.572 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:13.572 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:13.572 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:13.572 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:30:13.572 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369455+0000", 2024-09-14T06:30:13.572 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.0800000000000001e-07 2024-09-14T06:30:13.572 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:13.572 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:13.572 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:30:13.572 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369456+0000", 2024-09-14T06:30:13.572 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.0729999999999999e-06 2024-09-14T06:30:13.573 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:13.573 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:13.573 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:30:13.573 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369460+0000", 2024-09-14T06:30:13.573 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.027e-06 2024-09-14T06:30:13.573 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:13.573 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:13.573 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:30:13.573 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369462+0000", 2024-09-14T06:30:13.573 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.5320000000000001e-06 2024-09-14T06:30:13.573 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:13.573 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:13.573 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:30:13.573 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.417257+0000", 2024-09-14T06:30:13.574 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.04779489 2024-09-14T06:30:13.574 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:13.574 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:13.574 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:30:13.574 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.417296+0000", 2024-09-14T06:30:13.574 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.8658000000000002e-05 2024-09-14T06:30:13.574 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:13.574 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:13.574 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:30:13.574 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.417346+0000", 2024-09-14T06:30:13.574 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.0105999999999998e-05 2024-09-14T06:30:13.574 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:13.574 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:30:13.574 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:13.575 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:13.575 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:13.575 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:53250 3.f 3:f71f51a7:::rbd_data.112dca2f6ae1.0000000000000015:17 [read 2011136~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:30:13.575 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:44.369465+0000", 2024-09-14T06:30:13.575 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 29.175561276, 2024-09-14T06:30:13.575 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.048032230000000002, 2024-09-14T06:30:13.575 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:30:13.575 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:30:13.575 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:30:13.575 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:30:13.575 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:30:13.575 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:13.575 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 53250 2024-09-14T06:30:13.575 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:13.576 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:30:13.576 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:13.576 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:30:13.576 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369465+0000", 2024-09-14T06:30:13.576 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:13.576 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:13.576 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:13.576 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:30:13.576 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369465+0000", 2024-09-14T06:30:13.576 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:13.576 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:13.576 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:13.576 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:30:13.577 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369465+0000", 2024-09-14T06:30:13.577 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.2100000000000001e-07 2024-09-14T06:30:13.577 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:13.577 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:13.577 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:30:13.577 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369466+0000", 2024-09-14T06:30:13.577 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.0640000000000001e-06 2024-09-14T06:30:13.577 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:13.577 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:13.577 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:30:13.578 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369469+0000", 2024-09-14T06:30:13.578 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.0170000000000001e-06 2024-09-14T06:30:13.578 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:13.578 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:13.578 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:30:13.578 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369472+0000", 2024-09-14T06:30:13.578 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.6189999999999998e-06 2024-09-14T06:30:13.578 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:13.578 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:13.578 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:30:13.578 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.417383+0000", 2024-09-14T06:30:13.578 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047911092000000002 2024-09-14T06:30:13.578 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:13.578 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:13.579 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:30:13.579 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.417430+0000", 2024-09-14T06:30:13.579 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.6898999999999999e-05 2024-09-14T06:30:13.579 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:13.579 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:13.579 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:30:13.579 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.417497+0000", 2024-09-14T06:30:13.579 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.7317999999999995e-05 2024-09-14T06:30:13.579 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:13.579 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:30:13.579 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:13.579 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:13.579 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:13.580 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:53251 3.f 3:f71f51a7:::rbd_data.112dca2f6ae1.0000000000000015:17 [read 2035712~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:30:13.580 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:44.369475+0000", 2024-09-14T06:30:13.580 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 29.175551280000001, 2024-09-14T06:30:13.580 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.048154542000000002, 2024-09-14T06:30:13.580 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:30:13.580 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:30:13.580 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:30:13.580 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:30:13.580 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:30:13.580 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:13.580 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 53251 2024-09-14T06:30:13.580 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:13.580 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:30:13.581 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:13.581 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:30:13.581 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369475+0000", 2024-09-14T06:30:13.581 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:13.581 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:13.581 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:13.581 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:30:13.581 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369475+0000", 2024-09-14T06:30:13.581 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:13.581 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:13.581 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:13.581 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:30:13.581 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369475+0000", 2024-09-14T06:30:13.582 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.0100000000000001e-07 2024-09-14T06:30:13.582 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:13.582 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:13.582 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:30:13.582 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369476+0000", 2024-09-14T06:30:13.582 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.0580000000000001e-06 2024-09-14T06:30:13.582 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:13.582 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:13.582 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:30:13.582 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369479+0000", 2024-09-14T06:30:13.582 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.9950000000000001e-06 2024-09-14T06:30:13.582 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:13.582 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:13.582 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:30:13.582 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369482+0000", 2024-09-14T06:30:13.583 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.6340000000000001e-06 2024-09-14T06:30:13.583 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:13.583 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:13.583 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:30:13.583 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.417530+0000", 2024-09-14T06:30:13.583 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.048047535000000002 2024-09-14T06:30:13.583 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:13.583 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:13.583 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:30:13.583 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.417574+0000", 2024-09-14T06:30:13.583 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.4255999999999998e-05 2024-09-14T06:30:13.583 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:13.583 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:13.583 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:30:13.584 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.417630+0000", 2024-09-14T06:30:13.584 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.5862999999999997e-05 2024-09-14T06:30:13.584 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:13.584 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:30:13.584 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:13.584 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:13.584 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:13.584 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:53252 3.f 3:f71f51a7:::rbd_data.112dca2f6ae1.0000000000000015:17 [read 2060288~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:30:13.584 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:44.369485+0000", 2024-09-14T06:30:13.584 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 29.175541431999999, 2024-09-14T06:30:13.584 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.048254116, 2024-09-14T06:30:13.584 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:30:13.584 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:30:13.584 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:30:13.585 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:30:13.585 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:30:13.585 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:13.585 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 53252 2024-09-14T06:30:13.585 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:13.585 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:30:13.585 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:13.585 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:30:13.585 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369485+0000", 2024-09-14T06:30:13.585 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:13.585 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:13.585 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:13.585 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:30:13.585 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369485+0000", 2024-09-14T06:30:13.586 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:13.586 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:13.586 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:13.586 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:30:13.586 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369485+0000", 2024-09-14T06:30:13.586 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.1e-07 2024-09-14T06:30:13.586 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:13.586 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:13.586 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:30:13.586 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369487+0000", 2024-09-14T06:30:13.586 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.42e-06 2024-09-14T06:30:13.586 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:13.586 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:13.586 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:30:13.587 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369490+0000", 2024-09-14T06:30:13.587 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.9639999999999999e-06 2024-09-14T06:30:13.587 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:13.587 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:13.587 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:30:13.587 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369492+0000", 2024-09-14T06:30:13.587 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.5679999999999998e-06 2024-09-14T06:30:13.587 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:13.587 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:13.587 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:30:13.587 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.417645+0000", 2024-09-14T06:30:13.587 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.048152753 2024-09-14T06:30:13.587 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:13.587 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:13.588 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:30:13.588 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.417687+0000", 2024-09-14T06:30:13.588 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.1900999999999997e-05 2024-09-14T06:30:13.588 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:13.588 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:13.588 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:30:13.588 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.417739+0000", 2024-09-14T06:30:13.588 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.2299999999999997e-05 2024-09-14T06:30:13.588 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:13.588 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:30:13.588 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:13.588 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:13.588 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:13.588 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:53253 3.f 3:f71f51a7:::rbd_data.112dca2f6ae1.0000000000000015:17 [read 2084864~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:30:13.589 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:44.369495+0000", 2024-09-14T06:30:13.589 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 29.175531308, 2024-09-14T06:30:13.589 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.048366785000000002, 2024-09-14T06:30:13.589 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:30:13.589 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:30:13.589 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:30:13.589 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:30:13.589 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:30:13.589 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:13.589 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 53253 2024-09-14T06:30:13.589 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:13.589 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:30:13.589 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:13.589 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:30:13.590 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369495+0000", 2024-09-14T06:30:13.590 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:13.590 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:13.590 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:13.590 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:30:13.590 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369495+0000", 2024-09-14T06:30:13.590 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:13.590 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:13.590 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:13.590 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:30:13.590 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370388+0000", 2024-09-14T06:30:13.590 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00089287999999999998 2024-09-14T06:30:13.590 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:13.590 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:13.591 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:30:13.591 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370392+0000", 2024-09-14T06:30:13.591 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.2130000000000002e-06 2024-09-14T06:30:13.591 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:13.591 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:13.591 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:30:13.591 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370401+0000", 2024-09-14T06:30:13.591 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.7649999999999999e-06 2024-09-14T06:30:13.591 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:13.591 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:13.591 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:30:13.591 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370406+0000", 2024-09-14T06:30:13.591 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.0159999999999999e-06 2024-09-14T06:30:13.591 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:13.592 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:13.592 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:30:13.592 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.417764+0000", 2024-09-14T06:30:13.592 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047358348000000001 2024-09-14T06:30:13.592 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:13.592 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:13.592 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:30:13.592 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.417806+0000", 2024-09-14T06:30:13.592 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.1149999999999997e-05 2024-09-14T06:30:13.592 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:13.592 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:13.592 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:30:13.592 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.417862+0000", 2024-09-14T06:30:13.593 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.6412999999999997e-05 2024-09-14T06:30:13.593 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:13.593 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:30:13.593 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:13.593 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:13.593 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:13.593 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:53340 3.f 3:f71f51a7:::rbd_data.112dca2f6ae1.0000000000000015:17 [read 1056768~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:30:13.593 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:44.370421+0000", 2024-09-14T06:30:13.593 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 29.174605428, 2024-09-14T06:30:13.593 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047566313999999998, 2024-09-14T06:30:13.593 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:30:13.593 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:30:13.593 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:30:13.594 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:30:13.594 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:30:13.594 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:13.594 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 53340 2024-09-14T06:30:13.594 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:13.594 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:30:13.594 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:13.594 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:30:13.594 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370421+0000", 2024-09-14T06:30:13.594 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:13.594 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:13.594 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:13.594 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:30:13.594 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370421+0000", 2024-09-14T06:30:13.595 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:13.595 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:13.595 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:13.595 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:30:13.595 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370422+0000", 2024-09-14T06:30:13.595 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.3799999999999996e-07 2024-09-14T06:30:13.595 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:13.595 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:13.595 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:30:13.595 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370424+0000", 2024-09-14T06:30:13.595 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.6919999999999998e-06 2024-09-14T06:30:13.595 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:13.595 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:13.595 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:30:13.596 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370432+0000", 2024-09-14T06:30:13.596 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.2439999999999996e-06 2024-09-14T06:30:13.596 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:13.596 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:13.596 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:30:13.596 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370437+0000", 2024-09-14T06:30:13.596 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.5389999999999998e-06 2024-09-14T06:30:13.596 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:13.596 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:13.596 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:30:13.596 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.417888+0000", 2024-09-14T06:30:13.596 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047450432000000001 2024-09-14T06:30:13.596 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:13.597 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:13.597 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:30:13.597 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.417935+0000", 2024-09-14T06:30:13.597 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.7454999999999999e-05 2024-09-14T06:30:13.597 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:13.597 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:13.597 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:30:13.597 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.417987+0000", 2024-09-14T06:30:13.597 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.2213999999999998e-05 2024-09-14T06:30:13.597 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:13.597 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:30:13.597 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:13.597 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:13.598 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:13.598 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:53341 3.f 3:f71f51a7:::rbd_data.112dca2f6ae1.0000000000000015:17 [read 1081344~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:30:13.598 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:44.370446+0000", 2024-09-14T06:30:13.598 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 29.174580155000001, 2024-09-14T06:30:13.598 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047665723, 2024-09-14T06:30:13.598 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:30:13.598 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:30:13.598 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:30:13.598 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:30:13.598 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:30:13.598 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:13.598 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 53341 2024-09-14T06:30:13.598 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:13.598 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:30:13.598 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:13.599 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:30:13.599 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370446+0000", 2024-09-14T06:30:13.599 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:13.599 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:13.599 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:13.599 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:30:13.599 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370446+0000", 2024-09-14T06:30:13.599 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:13.599 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:13.599 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:13.599 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:30:13.599 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370447+0000", 2024-09-14T06:30:13.599 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.6100000000000002e-07 2024-09-14T06:30:13.600 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:13.600 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:13.600 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:30:13.600 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370448+0000", 2024-09-14T06:30:13.600 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.7370000000000001e-06 2024-09-14T06:30:13.600 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:13.600 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:13.600 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:30:13.600 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370462+0000", 2024-09-14T06:30:13.600 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.3227000000000001e-05 2024-09-14T06:30:13.600 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:13.600 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:13.600 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:30:13.600 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370467+0000", 2024-09-14T06:30:13.601 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.6540000000000004e-06 2024-09-14T06:30:13.601 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:13.601 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:13.601 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:30:13.601 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.418013+0000", 2024-09-14T06:30:13.601 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047545325999999999 2024-09-14T06:30:13.601 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:13.601 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:13.601 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:30:13.601 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.418054+0000", 2024-09-14T06:30:13.601 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.1696999999999997e-05 2024-09-14T06:30:13.601 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:13.601 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:13.601 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:30:13.602 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.418112+0000", 2024-09-14T06:30:13.602 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.7720999999999998e-05 2024-09-14T06:30:13.602 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:13.602 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:30:13.602 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:13.602 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:13.602 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:13.602 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:53342 3.f 3:f71f51a7:::rbd_data.112dca2f6ae1.0000000000000015:17 [read 1105920~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:30:13.602 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:44.370475+0000", 2024-09-14T06:30:13.602 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 29.174551094000002, 2024-09-14T06:30:13.602 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047748848000000003, 2024-09-14T06:30:13.602 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:30:13.602 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:30:13.603 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:30:13.603 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:30:13.603 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:30:13.603 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:13.603 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 53342 2024-09-14T06:30:13.603 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:13.603 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:30:13.603 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:13.603 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:30:13.603 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370475+0000", 2024-09-14T06:30:13.603 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:13.603 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:13.603 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:13.603 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:30:13.603 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370475+0000", 2024-09-14T06:30:13.604 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:13.604 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:13.604 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:13.604 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:30:13.604 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370476+0000", 2024-09-14T06:30:13.604 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.4700000000000002e-07 2024-09-14T06:30:13.604 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:13.604 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:13.604 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:30:13.604 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370478+0000", 2024-09-14T06:30:13.604 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.5890000000000001e-06 2024-09-14T06:30:13.604 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:13.604 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:13.605 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:30:13.605 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370486+0000", 2024-09-14T06:30:13.605 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.3470000000000001e-06 2024-09-14T06:30:13.605 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:13.605 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:13.605 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:30:13.605 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370491+0000", 2024-09-14T06:30:13.605 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.5720000000000002e-06 2024-09-14T06:30:13.605 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:13.605 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:13.605 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:30:13.605 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.418135+0000", 2024-09-14T06:30:13.605 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047644094999999997 2024-09-14T06:30:13.606 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:13.606 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:13.606 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:30:13.606 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.418174+0000", 2024-09-14T06:30:13.606 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.8204000000000002e-05 2024-09-14T06:30:13.606 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:13.606 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:13.606 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:30:13.606 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.418224+0000", 2024-09-14T06:30:13.606 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.0593999999999997e-05 2024-09-14T06:30:13.606 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:13.606 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:30:13.606 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:13.606 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:13.607 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:13.607 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:53343 3.f 3:f71f51a7:::rbd_data.112dca2f6ae1.0000000000000015:17 [read 1130496~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:30:13.607 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:44.370498+0000", 2024-09-14T06:30:13.607 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 29.174528217999999, 2024-09-14T06:30:13.607 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047841574999999997, 2024-09-14T06:30:13.607 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:30:13.607 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:30:13.607 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:30:13.607 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:30:13.607 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:30:13.607 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:13.607 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 53343 2024-09-14T06:30:13.607 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:13.608 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:30:13.608 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:13.608 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:30:13.608 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370498+0000", 2024-09-14T06:30:13.608 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:13.608 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:13.608 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:13.608 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:30:13.608 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370498+0000", 2024-09-14T06:30:13.608 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:13.608 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:13.608 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:13.608 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:30:13.609 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370499+0000", 2024-09-14T06:30:13.609 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.9999999999999998e-07 2024-09-14T06:30:13.609 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:13.609 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:13.609 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:30:13.609 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370501+0000", 2024-09-14T06:30:13.609 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.0310000000000001e-06 2024-09-14T06:30:13.609 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:13.609 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:13.609 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:30:13.609 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370507+0000", 2024-09-14T06:30:13.609 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.4169999999999997e-06 2024-09-14T06:30:13.609 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:13.610 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:13.610 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:30:13.610 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370516+0000", 2024-09-14T06:30:13.610 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.7190000000000007e-06 2024-09-14T06:30:13.610 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:13.610 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:13.610 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:30:13.610 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.418247+0000", 2024-09-14T06:30:13.610 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047731190999999999 2024-09-14T06:30:13.610 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:13.610 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:13.610 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:30:13.610 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.418286+0000", 2024-09-14T06:30:13.611 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.8988e-05 2024-09-14T06:30:13.611 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:13.611 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:13.611 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:30:13.611 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.418340+0000", 2024-09-14T06:30:13.611 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.3829000000000002e-05 2024-09-14T06:30:13.611 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:13.611 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:30:13.611 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:13.611 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:13.611 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:13.611 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:53344 3.f 3:f71f51a7:::rbd_data.112dca2f6ae1.0000000000000015:17 [read 1155072~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:30:13.611 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:44.370524+0000", 2024-09-14T06:30:13.611 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 29.174502447999998, 2024-09-14T06:30:13.612 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047972262000000002, 2024-09-14T06:30:13.612 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:30:13.612 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:30:13.612 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:30:13.612 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:30:13.612 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:30:13.612 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:13.612 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 53344 2024-09-14T06:30:13.612 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:13.612 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:30:13.612 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:13.612 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:30:13.612 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370524+0000", 2024-09-14T06:30:13.612 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:13.613 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:13.613 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:13.613 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:30:13.613 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370524+0000", 2024-09-14T06:30:13.613 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:13.613 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:13.613 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:13.613 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:30:13.613 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370618+0000", 2024-09-14T06:30:13.613 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.4425999999999993e-05 2024-09-14T06:30:13.613 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:13.613 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:13.613 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:30:13.614 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370622+0000", 2024-09-14T06:30:13.614 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.3519999999999998e-06 2024-09-14T06:30:13.614 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:13.614 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:13.614 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:30:13.614 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370631+0000", 2024-09-14T06:30:13.614 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.9299999999999992e-06 2024-09-14T06:30:13.614 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:13.614 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:13.614 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:30:13.614 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370637+0000", 2024-09-14T06:30:13.614 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.3269999999999997e-06 2024-09-14T06:30:13.614 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:13.615 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:13.615 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:30:13.615 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.418372+0000", 2024-09-14T06:30:13.615 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047734764999999998 2024-09-14T06:30:13.615 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:13.615 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:13.615 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:30:13.615 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.418431+0000", 2024-09-14T06:30:13.615 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.8696999999999997e-05 2024-09-14T06:30:13.615 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:13.615 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:13.615 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:30:13.615 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.418496+0000", 2024-09-14T06:30:13.615 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.5765000000000005e-05 2024-09-14T06:30:13.616 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:13.616 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:30:13.616 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:13.616 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:13.616 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:13.616 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:53345 3.f 3:f71f51a7:::rbd_data.112dca2f6ae1.0000000000000015:17 [read 1179648~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:30:13.616 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:44.370647+0000", 2024-09-14T06:30:13.616 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 29.174378981, 2024-09-14T06:30:13.616 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047953775999999997, 2024-09-14T06:30:13.616 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:30:13.616 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:30:13.616 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:30:13.616 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:30:13.616 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:30:13.617 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:13.617 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 53345 2024-09-14T06:30:13.617 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:13.617 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:30:13.617 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:13.617 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:30:13.617 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370647+0000", 2024-09-14T06:30:13.617 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:13.617 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:13.617 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:13.617 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:30:13.617 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370647+0000", 2024-09-14T06:30:13.617 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:13.617 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:13.618 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:13.618 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:30:13.618 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.372346+0000", 2024-09-14T06:30:13.618 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.001698284 2024-09-14T06:30:13.618 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:13.618 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:13.618 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:30:13.618 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.372351+0000", 2024-09-14T06:30:13.618 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.8890000000000001e-06 2024-09-14T06:30:13.618 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:13.618 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:13.618 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:30:13.618 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.372362+0000", 2024-09-14T06:30:13.618 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.0941000000000001e-05 2024-09-14T06:30:13.619 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:13.619 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:13.619 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:30:13.619 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.372369+0000", 2024-09-14T06:30:13.619 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.362e-06 2024-09-14T06:30:13.619 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:13.619 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:13.619 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:30:13.619 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.418510+0000", 2024-09-14T06:30:13.619 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046140914999999998 2024-09-14T06:30:13.619 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:13.619 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:13.619 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:30:13.619 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.418549+0000", 2024-09-14T06:30:13.620 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.8717999999999998e-05 2024-09-14T06:30:13.620 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:13.620 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:13.620 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:30:13.620 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.418601+0000", 2024-09-14T06:30:13.620 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.2667000000000002e-05 2024-09-14T06:30:13.620 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:13.620 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:30:13.620 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:13.620 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:13.620 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:13.620 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:53351 3.f 3:f71f51a7:::rbd_data.112dca2f6ae1.0000000000000015:17 [read 1327104~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:30:13.620 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:44.372499+0000", 2024-09-14T06:30:13.621 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 29.172527859999999, 2024-09-14T06:30:13.621 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047140596, 2024-09-14T06:30:13.621 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:30:13.621 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:30:13.621 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:30:13.621 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:30:13.621 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:30:13.621 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:13.621 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 53351 2024-09-14T06:30:13.621 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:13.621 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:30:13.621 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:13.621 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:30:13.622 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.372499+0000", 2024-09-14T06:30:13.622 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:13.622 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:13.622 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:13.622 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:30:13.622 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.372499+0000", 2024-09-14T06:30:13.622 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:13.622 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:13.622 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:13.622 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:30:13.622 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.372499+0000", 2024-09-14T06:30:13.622 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.3000000000000001e-07 2024-09-14T06:30:13.622 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:13.622 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:13.623 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:30:13.623 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.372501+0000", 2024-09-14T06:30:13.623 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.0820000000000001e-06 2024-09-14T06:30:13.623 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:13.623 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:13.623 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:30:13.623 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.372513+0000", 2024-09-14T06:30:13.623 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.1610999999999999e-05 2024-09-14T06:30:13.623 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:13.623 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:13.623 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:30:13.623 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.372518+0000", 2024-09-14T06:30:13.623 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.327e-06 2024-09-14T06:30:13.624 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:13.624 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:13.624 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:30:13.624 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.419530+0000", 2024-09-14T06:30:13.624 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047011798 2024-09-14T06:30:13.624 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:13.624 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:13.624 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:30:13.624 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.419578+0000", 2024-09-14T06:30:13.624 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.8446999999999999e-05 2024-09-14T06:30:13.624 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:13.624 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:13.624 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:30:13.625 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.419639+0000", 2024-09-14T06:30:13.625 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.0900999999999998e-05 2024-09-14T06:30:13.625 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:13.625 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:30:13.625 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:13.625 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:13.625 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:13.625 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:53352 3.f 3:f71f51a7:::rbd_data.112dca2f6ae1.0000000000000015:17 [read 1351680~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:30:13.625 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:44.372525+0000", 2024-09-14T06:30:13.625 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 29.172501946000001, 2024-09-14T06:30:13.625 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047259847000000001, 2024-09-14T06:30:13.625 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:30:13.625 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:30:13.626 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:30:13.626 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:30:13.626 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:30:13.626 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:13.626 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 53352 2024-09-14T06:30:13.626 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:13.626 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:30:13.626 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:13.626 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:30:13.626 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.372525+0000", 2024-09-14T06:30:13.626 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:13.626 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:13.626 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:13.626 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:30:13.627 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.372525+0000", 2024-09-14T06:30:13.627 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:13.627 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:13.627 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:13.627 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:30:13.627 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.372525+0000", 2024-09-14T06:30:13.627 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.0600000000000001e-07 2024-09-14T06:30:13.627 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:13.627 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:13.627 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:30:13.627 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.372527+0000", 2024-09-14T06:30:13.627 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.0499999999999999e-06 2024-09-14T06:30:13.627 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:13.627 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:13.628 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:30:13.628 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.372533+0000", 2024-09-14T06:30:13.628 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.4320000000000004e-06 2024-09-14T06:30:13.628 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:13.628 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:13.628 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:30:13.628 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.372538+0000", 2024-09-14T06:30:13.628 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.9130000000000002e-06 2024-09-14T06:30:13.628 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:13.628 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:13.628 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:30:13.628 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.419667+0000", 2024-09-14T06:30:13.628 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047128976000000003 2024-09-14T06:30:13.629 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:13.629 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:13.629 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:30:13.629 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.419715+0000", 2024-09-14T06:30:13.629 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.7868999999999999e-05 2024-09-14T06:30:13.629 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:13.629 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:13.629 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:30:13.629 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.419784+0000", 2024-09-14T06:30:13.629 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.9201000000000004e-05 2024-09-14T06:30:13.629 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:13.629 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:30:13.629 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:13.629 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:13.630 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:13.630 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:53353 3.f 3:f71f51a7:::rbd_data.112dca2f6ae1.0000000000000015:17 [read 1376256~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:30:13.630 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:44.372545+0000", 2024-09-14T06:30:13.630 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 29.172481906000002, 2024-09-14T06:30:13.630 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047364232999999999, 2024-09-14T06:30:13.630 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:30:13.630 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:30:13.630 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:30:13.630 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:30:13.630 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:30:13.630 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:13.630 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 53353 2024-09-14T06:30:13.630 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:13.630 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:30:13.631 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:13.631 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:30:13.631 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.372545+0000", 2024-09-14T06:30:13.631 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:13.631 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:13.631 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:13.631 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:30:13.631 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.372545+0000", 2024-09-14T06:30:13.631 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:13.631 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:13.631 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:13.631 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:30:13.631 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.372545+0000", 2024-09-14T06:30:13.631 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.27e-07 2024-09-14T06:30:13.632 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:13.632 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:13.632 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:30:13.632 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.372547+0000", 2024-09-14T06:30:13.632 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.0600000000000002e-06 2024-09-14T06:30:13.632 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:13.632 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:13.632 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:30:13.632 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.372553+0000", 2024-09-14T06:30:13.632 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.7980000000000002e-06 2024-09-14T06:30:13.632 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:13.632 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:13.632 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:30:13.632 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.372557+0000", 2024-09-14T06:30:13.633 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.7389999999999999e-06 2024-09-14T06:30:13.633 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:13.633 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:13.633 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:30:13.633 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.419811+0000", 2024-09-14T06:30:13.633 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047253310999999999 2024-09-14T06:30:13.633 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:13.633 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:13.633 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:30:13.633 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.419854+0000", 2024-09-14T06:30:13.633 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.3448000000000001e-05 2024-09-14T06:30:13.633 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:13.633 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:13.633 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:30:13.634 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.419909+0000", 2024-09-14T06:30:13.634 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.4549999999999998e-05 2024-09-14T06:30:13.634 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:13.634 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:30:13.634 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:13.634 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:13.634 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:13.634 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:53354 3.f 3:f71f51a7:::rbd_data.112dca2f6ae1.0000000000000015:17 [read 1400832~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:30:13.634 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:44.372563+0000", 2024-09-14T06:30:13.634 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 29.172463469, 2024-09-14T06:30:13.634 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047482233999999998, 2024-09-14T06:30:13.634 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:30:13.634 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:30:13.635 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:30:13.635 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:30:13.635 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:30:13.635 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:13.635 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 53354 2024-09-14T06:30:13.635 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:13.635 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:30:13.635 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:13.635 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:30:13.635 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.372563+0000", 2024-09-14T06:30:13.635 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:13.635 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:13.635 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:13.635 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:30:13.635 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.372563+0000", 2024-09-14T06:30:13.636 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:13.636 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:13.636 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:13.636 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:30:13.636 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.375009+0000", 2024-09-14T06:30:13.636 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.0024457680000000001 2024-09-14T06:30:13.636 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:13.636 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:13.636 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:30:13.636 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.375014+0000", 2024-09-14T06:30:13.636 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.0599999999999998e-06 2024-09-14T06:30:13.636 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:13.636 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:13.637 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:30:13.637 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.375026+0000", 2024-09-14T06:30:13.637 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.1783999999999999e-05 2024-09-14T06:30:13.637 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:13.637 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:13.637 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:30:13.637 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.375035+0000", 2024-09-14T06:30:13.637 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.7799999999999995e-06 2024-09-14T06:30:13.637 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:13.637 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:13.637 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:30:13.637 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.419942+0000", 2024-09-14T06:30:13.637 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.044906982999999998 2024-09-14T06:30:13.637 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:13.638 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:13.638 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:30:13.638 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.419989+0000", 2024-09-14T06:30:13.638 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.7104999999999997e-05 2024-09-14T06:30:13.638 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:13.638 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:13.638 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:30:13.638 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.420045+0000", 2024-09-14T06:30:13.638 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.5754000000000002e-05 2024-09-14T06:30:13.638 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:13.638 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:30:13.638 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:13.638 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:13.638 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:30:13.639 INFO:teuthology.orchestra.run.smithi138.stdout:} 2024-09-14T06:30:13.639 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-14T06:30:13.641 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-14T06:30:13.641 INFO:tasks.ceph.osd.0.smithi094.stderr:2024-09-14T06:30:13.639+0000 7f07c9333640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33636) UID: 0 2024-09-14T06:30:13.741 INFO:tasks.ceph.osd.4.smithi138.stderr:2024-09-14T06:30:13.740+0000 7f6c653c8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33615) UID: 0 2024-09-14T06:30:13.761 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-14T06:30:13.761 INFO:teuthology.orchestra.run.smithi094.stdout: "ops": [], 2024-09-14T06:30:13.762 INFO:teuthology.orchestra.run.smithi094.stdout: "num_ops": 0 2024-09-14T06:30:13.762 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-14T06:30:13.771 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-14T06:30:13.778 INFO:teuthology.orchestra.run.smithi094.stdout:osd.0: {} 2024-09-14T06:30:13.778 INFO:teuthology.orchestra.run.smithi094.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-14T06:30:13.796 INFO:teuthology.orchestra.run.smithi094.stdout:osd.1: {} 2024-09-14T06:30:13.796 INFO:teuthology.orchestra.run.smithi094.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-14T06:30:13.813 INFO:teuthology.orchestra.run.smithi094.stdout:osd.2: {} 2024-09-14T06:30:13.813 INFO:teuthology.orchestra.run.smithi094.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-14T06:30:13.829 INFO:teuthology.orchestra.run.smithi094.stdout:osd.3: {} 2024-09-14T06:30:13.829 INFO:teuthology.orchestra.run.smithi094.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-14T06:30:13.841 INFO:tasks.ceph.osd.2.smithi094.stderr:2024-09-14T06:30:13.840+0000 7faf3269b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33649) UID: 0 2024-09-14T06:30:13.885 INFO:teuthology.orchestra.run.smithi094.stdout:ERROR: (22) Invalid argument 2024-09-14T06:30:13.886 INFO:teuthology.orchestra.run.smithi094.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:30:13.897 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-14T06:30:13.942 INFO:tasks.ceph.osd.5.smithi138.stderr:2024-09-14T06:30:13.941+0000 7fe13746f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33618) UID: 0 2024-09-14T06:30:14.040 INFO:teuthology.orchestra.run.smithi094.stdout:ERROR: (22) Invalid argument 2024-09-14T06:30:14.040 INFO:teuthology.orchestra.run.smithi094.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:30:14.042 INFO:tasks.ceph.osd.3.smithi094.stderr:2024-09-14T06:30:14.041+0000 7feeeafe8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33655) UID: 0 2024-09-14T06:30:14.047 INFO:teuthology.orchestra.run.smithi094.stdout:osd.4: {} 2024-09-14T06:30:14.047 INFO:teuthology.orchestra.run.smithi094.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-14T06:30:14.053 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-14T06:30:14.064 INFO:teuthology.orchestra.run.smithi094.stdout:osd.5: {} 2024-09-14T06:30:14.064 INFO:teuthology.orchestra.run.smithi094.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-14T06:30:14.081 INFO:teuthology.orchestra.run.smithi094.stdout:osd.6: {} 2024-09-14T06:30:14.081 INFO:teuthology.orchestra.run.smithi094.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-14T06:30:14.097 INFO:teuthology.orchestra.run.smithi094.stdout:osd.7: {} 2024-09-14T06:30:14.097 INFO:teuthology.orchestra.run.smithi094.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-14T06:30:14.150 INFO:tasks.ceph.osd.3.smithi094.stderr:2024-09-14T06:30:14.149+0000 7feeeafe8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33655) UID: 0 2024-09-14T06:30:14.169 INFO:teuthology.orchestra.run.smithi094.stdout:ERROR: (22) Invalid argument 2024-09-14T06:30:14.169 INFO:teuthology.orchestra.run.smithi094.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:30:14.178 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-14T06:30:14.263 INFO:tasks.ceph.osd.3.smithi094.stderr:2024-09-14T06:30:14.262+0000 7feeeafe8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33655) UID: 0 2024-09-14T06:30:14.321 INFO:teuthology.orchestra.run.smithi094.stdout:ERROR: (22) Invalid argument 2024-09-14T06:30:14.322 INFO:teuthology.orchestra.run.smithi094.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:30:14.330 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-14T06:30:14.346 INFO:tasks.ceph.osd.6.smithi138.stderr:2024-09-14T06:30:14.344+0000 7f8cd8849640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33620) UID: 0 2024-09-14T06:30:14.446 INFO:tasks.ceph.osd.0.smithi094.stderr:2024-09-14T06:30:14.445+0000 7f07c9333640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33636) UID: 0 2024-09-14T06:30:14.472 INFO:teuthology.orchestra.run.smithi094.stdout:ERROR: (22) Invalid argument 2024-09-14T06:30:14.473 INFO:teuthology.orchestra.run.smithi094.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:30:14.481 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-14T06:30:14.547 INFO:tasks.ceph.osd.4.smithi138.stderr:2024-09-14T06:30:14.546+0000 7f6c653c8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33615) UID: 0 2024-09-14T06:30:14.623 INFO:teuthology.orchestra.run.smithi094.stdout:ERROR: (22) Invalid argument 2024-09-14T06:30:14.623 INFO:teuthology.orchestra.run.smithi094.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:30:14.633 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-14T06:30:14.676 INFO:tasks.ceph.osd.3.smithi094.stderr:2024-09-14T06:30:14.675+0000 7feeeafe8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33655) UID: 0 2024-09-14T06:30:14.735 INFO:teuthology.orchestra.run.smithi094.stdout:ERROR: (22) Invalid argument 2024-09-14T06:30:14.735 INFO:teuthology.orchestra.run.smithi094.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:30:14.744 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-14T06:30:14.750 INFO:tasks.ceph.osd.7.smithi138.stderr:2024-09-14T06:30:14.749+0000 7f3c6a21d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33623) UID: 0 2024-09-14T06:30:14.851 INFO:tasks.ceph.osd.7.smithi138.stderr:2024-09-14T06:30:14.850+0000 7f3c6a21d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33623) UID: 0 2024-09-14T06:30:14.874 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-14T06:30:14.892 INFO:teuthology.orchestra.run.smithi094.stdout:ERROR: (22) Invalid argument 2024-09-14T06:30:14.892 INFO:teuthology.orchestra.run.smithi094.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:30:14.902 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-14T06:30:14.968 INFO:teuthology.orchestra.run.smithi094.stdout:checking clone #17, image image_client.0-clone17 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent18 2024-09-14T06:30:14.969 INFO:tasks.ceph.osd.3.smithi094.stderr:2024-09-14T06:30:14.967+0000 7feeeafe8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33655) UID: 0 2024-09-14T06:30:15.053 INFO:tasks.ceph.osd.1.smithi094.stderr:2024-09-14T06:30:15.052+0000 7f328e47a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33641) UID: 0 2024-09-14T06:30:15.081 INFO:teuthology.orchestra.run.smithi094.stdout:ERROR: (22) Invalid argument 2024-09-14T06:30:15.081 INFO:teuthology.orchestra.run.smithi094.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:30:15.092 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-14T06:30:15.177 INFO:tasks.ceph.osd.1.smithi094.stderr:2024-09-14T06:30:15.176+0000 7f328e47a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33641) UID: 0 2024-09-14T06:30:15.241 INFO:teuthology.orchestra.run.smithi094.stdout:ERROR: (22) Invalid argument 2024-09-14T06:30:15.241 INFO:teuthology.orchestra.run.smithi094.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:30:15.251 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-14T06:30:15.292 INFO:tasks.ceph.osd.1.smithi094.stderr:2024-09-14T06:30:15.291+0000 7f328e47a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33641) UID: 0 2024-09-14T06:30:15.296 INFO:teuthology.orchestra.run.smithi094.stdout:2856 write 0x855c318 thru 0x855d281 (0xf6a bytes) 2024-09-14T06:30:15.300 INFO:teuthology.orchestra.run.smithi094.stdout:2857 write 0x4b282c5 thru 0x4b2f690 (0x73cc bytes) 2024-09-14T06:30:15.303 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-14T06:30:15.303 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-14T06:30:15.303 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-14T06:30:15.303 INFO:teuthology.orchestra.run.smithi094.stdout:2865 writesame 0x820a95d thru 0x8215a29 data_size 0x9(0xb0cd bytes) 2024-09-14T06:30:15.307 INFO:teuthology.orchestra.run.smithi094.stdout:2867 read 0x3a68fc thru 0x3b1bf0 (0xb2f5 bytes) 2024-09-14T06:30:15.307 INFO:teuthology.orchestra.run.smithi094.stdout:2868 read 0x30fc505 thru 0x3101e2b (0x5927 bytes) 2024-09-14T06:30:15.307 INFO:teuthology.orchestra.run.smithi094.stdout:2869 read 0x164fd44 thru 0x1655840 (0x5afd bytes) 2024-09-14T06:30:15.308 INFO:teuthology.orchestra.run.smithi094.stdout:2870 write 0x3ee00bc thru 0x3eebdcc (0xbd11 bytes) 2024-09-14T06:30:15.310 INFO:teuthology.orchestra.run.smithi094.stdout:2872 write 0x84097a5 thru 0x84123f2 (0x8c4e bytes) 2024-09-14T06:30:15.312 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-14T06:30:15.312 INFO:teuthology.orchestra.run.smithi094.stdout:2875 read 0x38451f4 thru 0x384d485 (0x8292 bytes) 2024-09-14T06:30:15.312 INFO:teuthology.orchestra.run.smithi094.stdout:2876 write 0x486755e thru 0x486f9cb (0x846e bytes) 2024-09-14T06:30:15.315 INFO:teuthology.orchestra.run.smithi094.stdout:2878 writesame 0x3af9b35 thru 0x3af9f3f data_size 0x9(0x40b bytes) 2024-09-14T06:30:15.316 INFO:teuthology.orchestra.run.smithi094.stdout:2879 read 0x7adecdc thru 0x7ae9b7b (0xaea0 bytes) 2024-09-14T06:30:15.323 INFO:teuthology.orchestra.run.smithi094.stdout:2881 writesame 0x89c14be thru 0x89c9d39 data_size 0xa(0x887c bytes) 2024-09-14T06:30:15.323 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-14T06:30:15.323 INFO:teuthology.orchestra.run.smithi094.stdout:2883 writesame 0x57e787d thru 0x57ee9ba data_size 0x41(0x713e bytes) 2024-09-14T06:30:15.324 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-14T06:30:15.325 INFO:teuthology.orchestra.run.smithi094.stdout:2885 read 0x7b94235 thru 0x7b97ae8 (0x38b4 bytes) 2024-09-14T06:30:15.325 INFO:teuthology.orchestra.run.smithi094.stdout:2889 writesame 0x54c4b2f thru 0x54cb94b data_size 0x1(0x6e1d bytes) 2024-09-14T06:30:15.347 INFO:teuthology.orchestra.run.smithi094.stdout:2890 trunc from 0x89c9d3a to 0x23c1cde 2024-09-14T06:30:15.357 INFO:tasks.ceph.osd.7.smithi138.stderr:2024-09-14T06:30:15.355+0000 7f3c6a21d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33623) UID: 0 2024-09-14T06:30:15.390 INFO:teuthology.orchestra.run.smithi094.stdout:2891 write 0xe951aab thru 0xe9580aa (0x6600 bytes) 2024-09-14T06:30:15.393 INFO:teuthology.orchestra.run.smithi094.stdout:2893 writesame 0x5969e52 thru 0x596de4f data_size 0x1(0x3ffe bytes) 2024-09-14T06:30:15.402 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-14T06:30:15.403 INFO:teuthology.orchestra.run.smithi094.stdout:2895 trunc from 0xe9580ab to 0xd3a99ec 2024-09-14T06:30:15.404 INFO:teuthology.orchestra.run.smithi094.stdout:ERROR: (22) Invalid argument 2024-09-14T06:30:15.405 INFO:teuthology.orchestra.run.smithi094.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:30:15.414 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-14T06:30:15.415 INFO:teuthology.orchestra.run.smithi094.stdout:2896 write 0x6eb9fec thru 0x6ec544a (0xb45f bytes) 2024-09-14T06:30:15.418 INFO:teuthology.orchestra.run.smithi094.stdout:2897 writesame 0x864c1cb thru 0x8651f58 data_size 0x19(0x5d8e bytes) 2024-09-14T06:30:15.433 INFO:teuthology.orchestra.run.smithi094.stdout:2898 trunc from 0xd3a99ec to 0x43dec34 2024-09-14T06:30:15.456 INFO:teuthology.orchestra.run.smithi094.stdout:2899 writesame 0x69954a3 thru 0x69963eb data_size 0xd(0xf49 bytes) 2024-09-14T06:30:15.457 INFO:teuthology.orchestra.run.smithi094.stdout:2901 read 0x322225b thru 0x32280d0 (0x5e76 bytes) 2024-09-14T06:30:15.458 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-14T06:30:15.458 INFO:tasks.ceph.osd.7.smithi138.stderr:2024-09-14T06:30:15.457+0000 7f3c6a21d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33623) UID: 0 2024-09-14T06:30:15.459 INFO:teuthology.orchestra.run.smithi094.stdout:2904 trunc from 0x69963ec to 0x79071de 2024-09-14T06:30:15.464 INFO:teuthology.orchestra.run.smithi094.stdout:2905 write 0x33cd69d thru 0x33d9464 (0xbdc8 bytes) 2024-09-14T06:30:15.466 INFO:teuthology.orchestra.run.smithi094.stdout:2906 writesame 0x6d17f6c thru 0x6d22aa5 data_size 0x3e(0xab3a bytes) 2024-09-14T06:30:15.468 INFO:teuthology.orchestra.run.smithi094.stdout:2908 read 0x6a722f8 thru 0x6a7c2d2 (0x9fdb bytes) 2024-09-14T06:30:15.469 INFO:teuthology.orchestra.run.smithi094.stdout:2909 punch from 0x5407d1f to 0x540e34d, (0x662e bytes) 2024-09-14T06:30:15.469 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-14T06:30:15.469 INFO:teuthology.orchestra.run.smithi094.stdout:2913 read 0x2478f2a thru 0x247a42e (0x1505 bytes) 2024-09-14T06:30:15.469 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-14T06:30:15.469 INFO:teuthology.orchestra.run.smithi094.stdout:2917 write 0x5df90ef thru 0x5df9940 (0x852 bytes) 2024-09-14T06:30:15.476 INFO:teuthology.orchestra.run.smithi094.stdout:2918 write 0x83eca27 thru 0x83f55ff (0x8bd9 bytes) 2024-09-14T06:30:15.479 INFO:teuthology.orchestra.run.smithi094.stdout:2919 write 0xd9a957 thru 0xda20be (0x7768 bytes) 2024-09-14T06:30:15.487 INFO:teuthology.orchestra.run.smithi094.stdout:2921 trunc from 0x83f5600 to 0x2bcfea9 2024-09-14T06:30:15.502 INFO:teuthology.orchestra.run.smithi094.stdout:2923 read 0x212781b thru 0x21329e7 (0xb1cd bytes) 2024-09-14T06:30:15.520 INFO:teuthology.orchestra.run.smithi094.stdout:2925 write 0xca1cf2f thru 0xca29bb8 (0xcc8a bytes) 2024-09-14T06:30:15.523 INFO:teuthology.orchestra.run.smithi094.stdout:2926 punch from 0x68ecfe1 to 0x68f374f, (0x676e bytes) 2024-09-14T06:30:15.523 INFO:teuthology.orchestra.run.smithi094.stdout:2927 read 0x9a0b37e thru 0x9a0c226 (0xea9 bytes) 2024-09-14T06:30:15.523 INFO:teuthology.orchestra.run.smithi094.stdout:2928 punch from 0x6a54616 to 0x6a621a4, (0xdb8e bytes) 2024-09-14T06:30:15.523 INFO:teuthology.orchestra.run.smithi094.stdout:2929 read 0x812e3c8 thru 0x813e2fe (0xff37 bytes) 2024-09-14T06:30:15.524 INFO:teuthology.orchestra.run.smithi094.stdout:2931 read 0x3ece59d thru 0x3ed5366 (0x6dca bytes) 2024-09-14T06:30:15.524 INFO:teuthology.orchestra.run.smithi094.stdout:2932 punch from 0x3610646 to 0x3614bc6, (0x4580 bytes) 2024-09-14T06:30:15.524 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-14T06:30:15.529 INFO:teuthology.orchestra.run.smithi094.stdout:2935 write 0xd5c6050 thru 0xd5ca7e7 (0x4798 bytes) 2024-09-14T06:30:15.530 INFO:teuthology.orchestra.run.smithi138.stdout:ERROR: (22) Invalid argument 2024-09-14T06:30:15.530 INFO:teuthology.orchestra.run.smithi138.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:30:15.539 INFO:teuthology.orchestra.run.smithi094.stdout:2936 write 0xea47ba2 thru 0xea497c7 (0x1c26 bytes) 2024-09-14T06:30:15.540 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-14T06:30:15.542 INFO:teuthology.orchestra.run.smithi094.stdout:2938 write 0xdb5bdde thru 0xdb6481f (0x8a42 bytes) 2024-09-14T06:30:15.545 INFO:teuthology.orchestra.run.smithi094.stdout:2939 write 0x83d4086 thru 0x83d53cd (0x1348 bytes) 2024-09-14T06:30:15.548 INFO:teuthology.orchestra.run.smithi094.stdout:2940 writesame 0xbe5ab86 thru 0xbe6296b data_size 0x37(0x7de6 bytes) 2024-09-14T06:30:15.549 INFO:teuthology.orchestra.run.smithi094.stdout:2942 writesame 0x81c17db thru 0x81c5953 data_size 0x25(0x4179 bytes) 2024-09-14T06:30:15.551 INFO:teuthology.orchestra.run.smithi094.stdout:2944 write 0x68d7c16 thru 0x68e0383 (0x876e bytes) 2024-09-14T06:30:15.553 INFO:teuthology.orchestra.run.smithi094.stdout:2945 read 0xcc9c4b7 thru 0xcca9091 (0xcbdb bytes) 2024-09-14T06:30:15.554 INFO:teuthology.orchestra.run.smithi094.stdout:2946 writesame 0x95184c1 thru 0x951a49c data_size 0x2(0x1fdc bytes) 2024-09-14T06:30:15.557 INFO:teuthology.orchestra.run.smithi094.stdout:2947 punch from 0x745acae to 0x745f3d7, (0x4729 bytes) 2024-09-14T06:30:15.557 INFO:teuthology.orchestra.run.smithi094.stdout:2948 read 0xe7275a6 thru 0xe72f22c (0x7c87 bytes) 2024-09-14T06:30:15.557 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-14T06:30:15.557 INFO:teuthology.orchestra.run.smithi094.stdout:2950 write 0x6b70b17 thru 0x6b7adf5 (0xa2df bytes) 2024-09-14T06:30:15.558 INFO:teuthology.orchestra.run.smithi094.stdout:2951 write 0x543b67d thru 0x543f5b8 (0x3f3c bytes) 2024-09-14T06:30:15.558 INFO:tasks.ceph.osd.0.smithi094.stderr:2024-09-14T06:30:15.557+0000 7f07c9333640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33636) UID: 0 2024-09-14T06:30:15.559 INFO:teuthology.orchestra.run.smithi094.stdout:2952 read 0x55b6143 thru 0x55c4a59 (0xe917 bytes) 2024-09-14T06:30:15.560 INFO:teuthology.orchestra.run.smithi094.stdout:2955 read 0xe2ca400 thru 0xe2d3947 (0x9548 bytes) 2024-09-14T06:30:15.561 INFO:teuthology.orchestra.run.smithi094.stdout:2958 punch from 0x55d161e to 0x55da2c2, (0x8ca4 bytes) 2024-09-14T06:30:15.561 INFO:teuthology.orchestra.run.smithi094.stdout:2959 write 0x2d1cb6f thru 0x2d28829 (0xbcbb bytes) 2024-09-14T06:30:15.563 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-14T06:30:15.568 INFO:teuthology.orchestra.run.smithi094.stdout:2961 trunc from 0xea497c8 to 0xac71ef9 2024-09-14T06:30:15.581 INFO:teuthology.orchestra.run.smithi094.stdout:2962 writesame 0x8a1bb26 thru 0x8a1e7fc data_size 0x1(0x2cd7 bytes) 2024-09-14T06:30:15.585 INFO:teuthology.orchestra.run.smithi094.stderr:nodeep-scrub is set 2024-09-14T06:30:15.593 INFO:teuthology.orchestra.run.smithi094.stdout:2964 punch from 0x628daec to 0x6296f30, (0x9444 bytes) 2024-09-14T06:30:15.601 INFO:teuthology.orchestra.run.smithi094.stdout:2966 write 0xdca8694 thru 0xdca88c3 (0x230 bytes) 2024-09-14T06:30:15.610 INFO:teuthology.orchestra.run.smithi094.stdout:2968 read 0xca563c3 thru 0xca65013 (0xec51 bytes) 2024-09-14T06:30:15.610 INFO:teuthology.orchestra.run.smithi094.stdout:2969 punch from 0xc29b516 to 0xc29eed3, (0x39bd bytes) 2024-09-14T06:30:15.616 INFO:teuthology.orchestra.run.smithi094.stdout:2970 write 0xea9e968 thru 0xeaa3ff7 (0x5690 bytes) 2024-09-14T06:30:15.620 INFO:teuthology.orchestra.run.smithi094.stdout:2972 read 0xb2a7c7b thru 0xb2b1afb (0x9e81 bytes) 2024-09-14T06:30:15.620 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-14T06:30:15.620 INFO:teuthology.orchestra.run.smithi094.stdout:2974 write 0xdfada22 thru 0xdfb9ef9 (0xc4d8 bytes) 2024-09-14T06:30:15.622 INFO:teuthology.orchestra.run.smithi094.stdout:2975 punch from 0x8a4416c to 0x8a4ccd4, (0x8b68 bytes) 2024-09-14T06:30:15.622 INFO:teuthology.orchestra.run.smithi094.stdout:2976 writesame 0x2e5e1f7 thru 0x2e66400 data_size 0x5(0x820a bytes) 2024-09-14T06:30:15.626 INFO:teuthology.orchestra.run.smithi094.stdout:2977 write 0x67e28f4 thru 0x67e406e (0x177b bytes) 2024-09-14T06:30:15.628 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-14T06:30:15.628 INFO:teuthology.orchestra.run.smithi094.stdout:2980 read 0x78997d4 thru 0x789ee5b (0x5688 bytes) 2024-09-14T06:30:15.628 INFO:teuthology.orchestra.run.smithi094.stdout:2981 punch from 0x5f68a6b to 0x5f6ae12, (0x23a7 bytes) 2024-09-14T06:30:15.628 INFO:teuthology.orchestra.run.smithi094.stdout:2983 read 0x32daa8f thru 0x32e9b5a (0xf0cc bytes) 2024-09-14T06:30:15.628 INFO:teuthology.orchestra.run.smithi094.stdout:2984 write 0x5c25bfe thru 0x5c2fd6d (0xa170 bytes) 2024-09-14T06:30:15.630 INFO:teuthology.orchestra.run.smithi094.stdout:2985 read 0x6dda983 thru 0x6de415d (0x97db bytes) 2024-09-14T06:30:15.630 INFO:teuthology.orchestra.run.smithi094.stdout:2986 read 0x7a7e10f thru 0x7a89303 (0xb1f5 bytes) 2024-09-14T06:30:15.635 INFO:teuthology.orchestra.run.smithi094.stdout:2987 trunc from 0xeaa3ff8 to 0xac3a62a 2024-09-14T06:30:15.649 INFO:teuthology.orchestra.run.smithi094.stdout:2989 write 0xfe5332 thru 0xff17a6 (0xc475 bytes) 2024-09-14T06:30:15.650 INFO:teuthology.orchestra.run.smithi094.stdout:2990 punch from 0x6b3b184 to 0x6b3d47c, (0x22f8 bytes) 2024-09-14T06:30:15.651 INFO:teuthology.orchestra.run.smithi094.stdout:2991 writesame 0x67e8342 thru 0x67ec45a data_size 0x37(0x4119 bytes) 2024-09-14T06:30:15.652 INFO:teuthology.orchestra.run.smithi094.stdout:2992 read 0x51031d8 thru 0x510ebf9 (0xba22 bytes) 2024-09-14T06:30:15.653 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-14T06:30:15.653 INFO:teuthology.orchestra.run.smithi094.stdout:2995 write 0x8d56817 thru 0x8d5ddf6 (0x75e0 bytes) 2024-09-14T06:30:15.654 INFO:teuthology.orchestra.run.smithi094.stdout:2996 write 0x2904ef thru 0x2915b7 (0x10c9 bytes) 2024-09-14T06:30:15.656 INFO:teuthology.orchestra.run.smithi094.stdout:2997 clone 20 order 19 su 16384 sc 15 2024-09-14T06:30:15.659 INFO:tasks.ceph.osd.6.smithi138.stderr:2024-09-14T06:30:15.658+0000 7f8cd8849640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33620) UID: 0 2024-09-14T06:30:15.691 INFO:teuthology.orchestra.run.smithi138.stdout:ERROR: (22) Invalid argument 2024-09-14T06:30:15.691 INFO:teuthology.orchestra.run.smithi138.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:30:15.700 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-14T06:30:15.784 INFO:tasks.ceph.osd.6.smithi138.stderr:2024-09-14T06:30:15.783+0000 7f8cd8849640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33620) UID: 0 2024-09-14T06:30:15.844 INFO:teuthology.orchestra.run.smithi138.stdout:ERROR: (22) Invalid argument 2024-09-14T06:30:15.844 INFO:teuthology.orchestra.run.smithi138.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:30:15.853 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-14T06:30:15.861 INFO:tasks.ceph.osd.2.smithi094.stderr:2024-09-14T06:30:15.859+0000 7faf3269b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33649) UID: 0 2024-09-14T06:30:15.961 INFO:tasks.ceph.osd.5.smithi138.stderr:2024-09-14T06:30:15.960+0000 7fe13746f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33618) UID: 0 2024-09-14T06:30:15.997 INFO:teuthology.orchestra.run.smithi138.stdout:ERROR: (22) Invalid argument 2024-09-14T06:30:15.998 INFO:teuthology.orchestra.run.smithi138.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:30:16.006 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-14T06:30:16.090 INFO:tasks.ceph.osd.4.smithi138.stderr:2024-09-14T06:30:16.089+0000 7f6c653c8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33615) UID: 0 2024-09-14T06:30:16.109 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-14T06:30:16.148 INFO:teuthology.orchestra.run.smithi138.stdout:ERROR: (22) Invalid argument 2024-09-14T06:30:16.148 INFO:teuthology.orchestra.run.smithi138.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:30:16.156 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-14T06:30:16.198 INFO:tasks.ceph.osd.4.smithi138.stderr:2024-09-14T06:30:16.197+0000 7f6c653c8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33615) UID: 0 2024-09-14T06:30:16.229 INFO:teuthology.orchestra.run.smithi094.stdout:osd.0: {} 2024-09-14T06:30:16.229 INFO:teuthology.orchestra.run.smithi094.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-14T06:30:16.246 INFO:teuthology.orchestra.run.smithi094.stdout:osd.1: {} 2024-09-14T06:30:16.246 INFO:teuthology.orchestra.run.smithi094.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-14T06:30:16.262 INFO:teuthology.orchestra.run.smithi094.stdout:osd.2: {} 2024-09-14T06:30:16.263 INFO:teuthology.orchestra.run.smithi094.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-14T06:30:16.263 INFO:tasks.ceph.osd.6.smithi138.stderr:2024-09-14T06:30:16.261+0000 7f8cd8849640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33620) UID: 0 2024-09-14T06:30:16.278 INFO:teuthology.orchestra.run.smithi094.stdout:osd.3: {} 2024-09-14T06:30:16.278 INFO:teuthology.orchestra.run.smithi094.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-14T06:30:16.294 INFO:teuthology.orchestra.run.smithi094.stdout:osd.4: {} 2024-09-14T06:30:16.295 INFO:teuthology.orchestra.run.smithi094.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-14T06:30:16.304 INFO:teuthology.orchestra.run.smithi138.stdout:ERROR: (22) Invalid argument 2024-09-14T06:30:16.304 INFO:teuthology.orchestra.run.smithi138.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:30:16.311 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-14T06:30:16.311 INFO:teuthology.orchestra.run.smithi094.stdout:osd.5: {} 2024-09-14T06:30:16.312 INFO:teuthology.orchestra.run.smithi094.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-14T06:30:16.328 INFO:teuthology.orchestra.run.smithi094.stdout:osd.6: {} 2024-09-14T06:30:16.328 INFO:teuthology.orchestra.run.smithi094.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-14T06:30:16.344 INFO:teuthology.orchestra.run.smithi094.stdout:osd.7: {} 2024-09-14T06:30:16.345 INFO:teuthology.orchestra.run.smithi094.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-14T06:30:16.363 INFO:tasks.ceph.osd.0.smithi094.stderr:2024-09-14T06:30:16.362+0000 7f07c9333640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33636) UID: 0 2024-09-14T06:30:16.454 INFO:teuthology.orchestra.run.smithi138.stdout:{ 2024-09-14T06:30:16.454 INFO:teuthology.orchestra.run.smithi138.stdout: "ops": [], 2024-09-14T06:30:16.454 INFO:teuthology.orchestra.run.smithi138.stdout: "num_ops": 0 2024-09-14T06:30:16.454 INFO:teuthology.orchestra.run.smithi138.stdout:} 2024-09-14T06:30:16.462 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-14T06:30:16.463 INFO:tasks.ceph.osd.4.smithi138.stderr:2024-09-14T06:30:16.462+0000 7f6c653c8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33615) UID: 0 2024-09-14T06:30:16.565 INFO:teuthology.orchestra.run.smithi138.stdout:{ 2024-09-14T06:30:16.565 INFO:teuthology.orchestra.run.smithi138.stdout: "ops": [], 2024-09-14T06:30:16.565 INFO:teuthology.orchestra.run.smithi138.stdout: "complaint_time": 30, 2024-09-14T06:30:16.565 INFO:teuthology.orchestra.run.smithi138.stdout: "num_blocked_ops": 0 2024-09-14T06:30:16.565 INFO:teuthology.orchestra.run.smithi138.stdout:} 2024-09-14T06:30:16.566 INFO:tasks.ceph.osd.4.smithi138.stderr:2024-09-14T06:30:16.564+0000 7f6c653c8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33615) UID: 0 2024-09-14T06:30:16.574 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-14T06:30:16.666 INFO:tasks.ceph.osd.1.smithi094.stderr:2024-09-14T06:30:16.665+0000 7f328e47a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33641) UID: 0 2024-09-14T06:30:16.719 INFO:teuthology.orchestra.run.smithi138.stdout:{ 2024-09-14T06:30:16.719 INFO:teuthology.orchestra.run.smithi138.stdout: "size": 20, 2024-09-14T06:30:16.719 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 600, 2024-09-14T06:30:16.719 INFO:teuthology.orchestra.run.smithi138.stdout: "ops": [ 2024-09-14T06:30:16.719 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:16.719 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:98505 3.10 3.1bf56f90 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T06:30:16.719 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:30:07.045453+0000", 2024-09-14T06:30:16.719 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 9.6717027469999994, 2024-09-14T06:30:16.719 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032144440000000003, 2024-09-14T06:30:16.720 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:30:16.720 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:30:16.720 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "reached pg", 2024-09-14T06:30:16.720 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:30:16.720 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:30:16.720 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:16.720 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 98505 2024-09-14T06:30:16.720 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:16.720 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:30:16.720 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:16.720 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:30:16.720 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045453+0000", 2024-09-14T06:30:16.720 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:16.721 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:16.721 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:16.721 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:30:16.721 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045453+0000", 2024-09-14T06:30:16.721 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:16.721 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:16.721 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:16.721 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:30:16.721 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045453+0000", 2024-09-14T06:30:16.721 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.36e-07 2024-09-14T06:30:16.722 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:16.722 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:16.722 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:30:16.722 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045454+0000", 2024-09-14T06:30:16.722 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.1299999999999998e-07 2024-09-14T06:30:16.722 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:16.722 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:16.722 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:30:16.722 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045457+0000", 2024-09-14T06:30:16.722 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.8559999999999998e-06 2024-09-14T06:30:16.722 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:16.723 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:16.723 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:30:16.723 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045460+0000", 2024-09-14T06:30:16.723 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.8750000000000002e-06 2024-09-14T06:30:16.723 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:16.723 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:16.723 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:30:16.723 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.077592+0000", 2024-09-14T06:30:16.723 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032131727999999998 2024-09-14T06:30:16.723 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:16.723 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:16.723 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:30:16.723 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.077597+0000", 2024-09-14T06:30:16.724 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.8319999999999996e-06 2024-09-14T06:30:16.724 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:16.724 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:30:16.724 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:16.724 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:16.724 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:16.724 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:98506 3.10 3.1bf56f90 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T06:30:16.724 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:30:07.045481+0000", 2024-09-14T06:30:16.724 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 9.6716744919999993, 2024-09-14T06:30:16.725 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032142759999999999, 2024-09-14T06:30:16.725 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:30:16.725 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:30:16.725 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "reached pg", 2024-09-14T06:30:16.725 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:30:16.725 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:30:16.725 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:16.725 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 98506 2024-09-14T06:30:16.725 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:16.725 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:30:16.725 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:16.725 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:30:16.725 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045481+0000", 2024-09-14T06:30:16.726 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:16.726 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:16.726 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:16.726 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:30:16.726 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045481+0000", 2024-09-14T06:30:16.726 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:16.726 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:16.726 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:16.726 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:30:16.726 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045481+0000", 2024-09-14T06:30:16.726 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.8500000000000001e-07 2024-09-14T06:30:16.726 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:16.726 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:16.726 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:30:16.727 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045483+0000", 2024-09-14T06:30:16.727 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.8250000000000001e-06 2024-09-14T06:30:16.727 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:16.727 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:16.727 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:30:16.727 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045489+0000", 2024-09-14T06:30:16.727 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.5740000000000003e-06 2024-09-14T06:30:16.727 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:16.727 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:16.727 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:30:16.727 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045495+0000", 2024-09-14T06:30:16.727 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.7690000000000002e-06 2024-09-14T06:30:16.727 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:16.727 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:16.728 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:30:16.728 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.077619+0000", 2024-09-14T06:30:16.728 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032124621999999999 2024-09-14T06:30:16.728 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:16.728 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:16.728 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:30:16.728 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.077624+0000", 2024-09-14T06:30:16.728 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.3850000000000004e-06 2024-09-14T06:30:16.728 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:16.728 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:30:16.729 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:16.729 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:16.729 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:16.729 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:98507 3.10 3.1bf56f90 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T06:30:16.729 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:30:07.045502+0000", 2024-09-14T06:30:16.729 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 9.6716530709999997, 2024-09-14T06:30:16.729 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032147219999999997, 2024-09-14T06:30:16.729 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:30:16.729 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:30:16.729 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "reached pg", 2024-09-14T06:30:16.729 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:30:16.730 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:30:16.730 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:16.730 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 98507 2024-09-14T06:30:16.730 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:16.730 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:30:16.730 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:16.730 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:30:16.730 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045502+0000", 2024-09-14T06:30:16.730 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:16.730 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:16.730 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:16.730 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:30:16.730 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045502+0000", 2024-09-14T06:30:16.731 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:16.731 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:16.731 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:16.731 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:30:16.731 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045503+0000", 2024-09-14T06:30:16.731 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.4700000000000002e-07 2024-09-14T06:30:16.731 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:16.731 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:16.731 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:30:16.731 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045509+0000", 2024-09-14T06:30:16.731 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.7329999999999997e-06 2024-09-14T06:30:16.731 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:16.731 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:16.731 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:30:16.732 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045515+0000", 2024-09-14T06:30:16.732 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.8030000000000002e-06 2024-09-14T06:30:16.732 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:16.732 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:16.732 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:30:16.732 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045522+0000", 2024-09-14T06:30:16.732 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.0369999999999997e-06 2024-09-14T06:30:16.732 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:16.732 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:16.732 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:30:16.732 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.077641+0000", 2024-09-14T06:30:16.733 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032119091000000002 2024-09-14T06:30:16.733 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:16.733 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:16.733 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:30:16.733 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.077650+0000", 2024-09-14T06:30:16.733 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.2090000000000005e-06 2024-09-14T06:30:16.733 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:16.733 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:30:16.733 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:16.733 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:16.733 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:16.734 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:98508 3.10 3.1bf56f90 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T06:30:16.734 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:30:07.045530+0000", 2024-09-14T06:30:16.734 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 9.6716254540000008, 2024-09-14T06:30:16.734 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032198773999999999, 2024-09-14T06:30:16.734 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:30:16.734 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:30:16.734 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "reached pg", 2024-09-14T06:30:16.734 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:30:16.734 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:30:16.734 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:16.734 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 98508 2024-09-14T06:30:16.734 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:16.734 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:30:16.734 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:16.735 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:30:16.735 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045530+0000", 2024-09-14T06:30:16.735 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:16.735 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:16.735 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:16.735 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:30:16.735 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045530+0000", 2024-09-14T06:30:16.735 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:16.735 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:16.735 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:16.735 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:30:16.735 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045530+0000", 2024-09-14T06:30:16.736 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.3099999999999998e-07 2024-09-14T06:30:16.736 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:16.736 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:16.736 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:30:16.736 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045539+0000", 2024-09-14T06:30:16.736 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.4780000000000008e-06 2024-09-14T06:30:16.736 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:16.736 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:16.736 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:30:16.736 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045546+0000", 2024-09-14T06:30:16.736 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.3259999999999998e-06 2024-09-14T06:30:16.736 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:16.737 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:16.737 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:30:16.737 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045551+0000", 2024-09-14T06:30:16.737 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.9960000000000001e-06 2024-09-14T06:30:16.737 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:16.737 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:16.737 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:30:16.737 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.077716+0000", 2024-09-14T06:30:16.737 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032165081999999998 2024-09-14T06:30:16.737 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:16.737 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:16.737 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:30:16.737 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.077729+0000", 2024-09-14T06:30:16.738 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.2461e-05 2024-09-14T06:30:16.738 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:16.738 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:30:16.738 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:16.738 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:16.738 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:16.738 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:98509 3.10 3.1bf56f90 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T06:30:16.738 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:30:07.045558+0000", 2024-09-14T06:30:16.738 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 9.6715972529999998, 2024-09-14T06:30:16.739 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032242198, 2024-09-14T06:30:16.739 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:30:16.739 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:30:16.739 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "reached pg", 2024-09-14T06:30:16.739 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:30:16.739 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:30:16.739 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:16.739 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 98509 2024-09-14T06:30:16.739 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:16.739 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:30:16.739 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:16.739 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:30:16.739 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045558+0000", 2024-09-14T06:30:16.740 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:16.740 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:16.740 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:16.740 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:30:16.740 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045558+0000", 2024-09-14T06:30:16.740 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:16.740 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:16.740 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:16.740 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:30:16.740 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045559+0000", 2024-09-14T06:30:16.740 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.27e-07 2024-09-14T06:30:16.740 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:16.740 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:16.741 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:30:16.741 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045560+0000", 2024-09-14T06:30:16.741 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.6959999999999999e-06 2024-09-14T06:30:16.741 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:16.741 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:16.741 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:30:16.741 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045567+0000", 2024-09-14T06:30:16.741 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.6900000000000003e-06 2024-09-14T06:30:16.741 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:16.741 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:16.741 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:30:16.741 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045571+0000", 2024-09-14T06:30:16.742 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.5750000000000002e-06 2024-09-14T06:30:16.742 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:16.742 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:16.742 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:30:16.742 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.077789+0000", 2024-09-14T06:30:16.742 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032217186000000002 2024-09-14T06:30:16.742 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:16.743 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:16.743 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:30:16.743 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.077800+0000", 2024-09-14T06:30:16.743 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.1623999999999999e-05 2024-09-14T06:30:16.743 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:16.743 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:30:16.743 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:16.743 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:16.744 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:16.744 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:98510 3.10 3.1bf56f90 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T06:30:16.744 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:30:07.045580+0000", 2024-09-14T06:30:16.744 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 9.6715754389999997, 2024-09-14T06:30:16.744 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032299845000000001, 2024-09-14T06:30:16.744 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:30:16.744 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:30:16.744 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "reached pg", 2024-09-14T06:30:16.744 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:30:16.744 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:30:16.744 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:16.744 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 98510 2024-09-14T06:30:16.745 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:16.745 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:30:16.745 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:16.745 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:30:16.745 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045580+0000", 2024-09-14T06:30:16.745 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:16.745 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:16.745 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:16.745 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:30:16.746 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045580+0000", 2024-09-14T06:30:16.746 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:16.746 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:16.746 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:16.746 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:30:16.746 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045580+0000", 2024-09-14T06:30:16.746 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.0400000000000002e-07 2024-09-14T06:30:16.746 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:16.746 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:16.746 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:30:16.746 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045582+0000", 2024-09-14T06:30:16.747 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.7600000000000001e-06 2024-09-14T06:30:16.747 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:16.747 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:16.747 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:30:16.747 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045588+0000", 2024-09-14T06:30:16.747 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.5650000000000001e-06 2024-09-14T06:30:16.747 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:16.747 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:16.747 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:30:16.747 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045597+0000", 2024-09-14T06:30:16.748 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.465e-06 2024-09-14T06:30:16.748 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:16.748 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:16.748 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:30:16.748 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.077871+0000", 2024-09-14T06:30:16.749 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032273562999999998 2024-09-14T06:30:16.749 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:16.749 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:16.749 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:30:16.749 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.077880+0000", 2024-09-14T06:30:16.749 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.0880000000000001e-06 2024-09-14T06:30:16.749 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:16.750 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:30:16.750 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:16.750 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:16.750 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:16.750 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:98511 3.10 3.1bf56f90 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T06:30:16.750 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:30:07.045610+0000", 2024-09-14T06:30:16.750 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 9.6715455919999993, 2024-09-14T06:30:16.750 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032338894, 2024-09-14T06:30:16.751 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:30:16.751 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:30:16.751 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "reached pg", 2024-09-14T06:30:16.751 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:30:16.751 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:30:16.751 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:16.751 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 98511 2024-09-14T06:30:16.751 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:16.751 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:30:16.751 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:16.751 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:30:16.752 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045610+0000", 2024-09-14T06:30:16.752 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:16.752 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:16.752 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:16.752 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:30:16.752 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045610+0000", 2024-09-14T06:30:16.752 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:16.752 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:16.752 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:16.752 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:30:16.752 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045610+0000", 2024-09-14T06:30:16.752 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.1600000000000002e-07 2024-09-14T06:30:16.752 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:16.752 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:16.753 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:30:16.753 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045612+0000", 2024-09-14T06:30:16.753 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.1150000000000001e-06 2024-09-14T06:30:16.753 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:16.753 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:16.753 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:30:16.754 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045618+0000", 2024-09-14T06:30:16.754 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.0499999999999997e-06 2024-09-14T06:30:16.754 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:16.754 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:16.754 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:30:16.754 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045623+0000", 2024-09-14T06:30:16.754 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.6770000000000003e-06 2024-09-14T06:30:16.754 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:16.754 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:16.754 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:30:16.755 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.077928+0000", 2024-09-14T06:30:16.755 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032305111999999997 2024-09-14T06:30:16.755 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:16.755 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:16.755 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:30:16.755 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.077949+0000", 2024-09-14T06:30:16.755 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.0523999999999999e-05 2024-09-14T06:30:16.755 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:16.755 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:30:16.755 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:16.755 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:16.755 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:16.755 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:98512 3.10 3.1bf56f90 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T06:30:16.756 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:30:07.045630+0000", 2024-09-14T06:30:16.756 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 9.6715255869999996, 2024-09-14T06:30:16.756 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032346632, 2024-09-14T06:30:16.756 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:30:16.756 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:30:16.756 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "reached pg", 2024-09-14T06:30:16.756 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:30:16.756 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:30:16.756 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:16.756 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 98512 2024-09-14T06:30:16.756 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:16.756 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:30:16.756 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:16.757 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:30:16.757 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045630+0000", 2024-09-14T06:30:16.757 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:16.757 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:16.757 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:16.757 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:30:16.757 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045630+0000", 2024-09-14T06:30:16.757 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:16.757 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:16.757 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:16.757 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:30:16.757 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045630+0000", 2024-09-14T06:30:16.757 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.8200000000000001e-07 2024-09-14T06:30:16.757 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:16.758 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:16.758 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:30:16.758 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045632+0000", 2024-09-14T06:30:16.758 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.7430000000000001e-06 2024-09-14T06:30:16.758 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:16.758 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:16.758 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:30:16.758 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045637+0000", 2024-09-14T06:30:16.758 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.4550000000000003e-06 2024-09-14T06:30:16.759 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:16.759 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:16.759 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:30:16.759 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045644+0000", 2024-09-14T06:30:16.759 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.1330000000000001e-06 2024-09-14T06:30:16.759 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:16.759 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:16.759 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:30:16.759 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.077968+0000", 2024-09-14T06:30:16.759 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032323401000000002 2024-09-14T06:30:16.759 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:16.760 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:16.760 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:30:16.760 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.077976+0000", 2024-09-14T06:30:16.760 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.5180000000000004e-06 2024-09-14T06:30:16.760 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:16.760 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:30:16.760 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:16.760 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:16.760 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:16.760 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:98513 3.10 3.1bf56f90 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T06:30:16.760 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:30:07.045652+0000", 2024-09-14T06:30:16.760 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 9.6715030229999996, 2024-09-14T06:30:16.760 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032353436999999999, 2024-09-14T06:30:16.761 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:30:16.761 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:30:16.761 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "reached pg", 2024-09-14T06:30:16.761 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:30:16.761 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:30:16.761 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:16.761 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 98513 2024-09-14T06:30:16.761 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:16.761 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:30:16.761 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:16.761 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:30:16.761 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045652+0000", 2024-09-14T06:30:16.761 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:16.762 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:16.762 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:16.762 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:30:16.762 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045652+0000", 2024-09-14T06:30:16.762 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:16.762 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:16.762 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:16.762 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:30:16.762 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045653+0000", 2024-09-14T06:30:16.762 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.3700000000000001e-07 2024-09-14T06:30:16.762 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:16.762 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:16.763 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:30:16.763 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045655+0000", 2024-09-14T06:30:16.763 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.854e-06 2024-09-14T06:30:16.763 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:16.763 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:16.763 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:30:16.763 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045661+0000", 2024-09-14T06:30:16.763 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.5509999999999996e-06 2024-09-14T06:30:16.763 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:16.763 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:16.763 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:30:16.763 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045668+0000", 2024-09-14T06:30:16.763 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.1980000000000004e-06 2024-09-14T06:30:16.764 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:16.764 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:16.764 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:30:16.764 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.077997+0000", 2024-09-14T06:30:16.764 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032328575999999998 2024-09-14T06:30:16.764 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:16.764 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:16.764 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:30:16.764 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.078006+0000", 2024-09-14T06:30:16.764 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.8210000000000008e-06 2024-09-14T06:30:16.764 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:16.764 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:30:16.765 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:16.765 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:16.765 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:16.765 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:98514 3.10 3.1bf56f90 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T06:30:16.765 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:30:07.045699+0000", 2024-09-14T06:30:16.765 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 9.6714566879999992, 2024-09-14T06:30:16.765 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032330223999999998, 2024-09-14T06:30:16.765 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:30:16.765 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:30:16.765 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "reached pg", 2024-09-14T06:30:16.765 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:30:16.765 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:30:16.765 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:16.765 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 98514 2024-09-14T06:30:16.766 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:16.766 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:30:16.766 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:16.766 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:30:16.766 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045699+0000", 2024-09-14T06:30:16.766 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:16.766 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:16.766 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:16.766 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:30:16.766 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045699+0000", 2024-09-14T06:30:16.766 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:16.766 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:16.766 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:16.767 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:30:16.767 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045699+0000", 2024-09-14T06:30:16.767 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.3200000000000001e-07 2024-09-14T06:30:16.767 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:16.767 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:16.767 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:30:16.767 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045701+0000", 2024-09-14T06:30:16.767 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.762e-06 2024-09-14T06:30:16.767 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:16.767 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:16.767 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:30:16.767 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045705+0000", 2024-09-14T06:30:16.767 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.2359999999999998e-06 2024-09-14T06:30:16.767 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:16.768 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:16.768 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:30:16.768 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045708+0000", 2024-09-14T06:30:16.768 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.796e-06 2024-09-14T06:30:16.768 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:16.768 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:16.768 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:30:16.768 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.078023+0000", 2024-09-14T06:30:16.768 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032315213000000002 2024-09-14T06:30:16.768 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:16.768 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:16.768 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:30:16.768 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.078029+0000", 2024-09-14T06:30:16.769 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.8850000000000004e-06 2024-09-14T06:30:16.769 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:16.769 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:30:16.769 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:16.769 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:16.769 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:16.769 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:98515 3.10 3.1bf56f90 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T06:30:16.769 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:30:07.045712+0000", 2024-09-14T06:30:16.769 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 9.6714433769999992, 2024-09-14T06:30:16.769 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032346871999999999, 2024-09-14T06:30:16.769 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:30:16.769 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:30:16.769 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "reached pg", 2024-09-14T06:30:16.770 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:30:16.770 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:30:16.770 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:16.770 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 98515 2024-09-14T06:30:16.770 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:16.770 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:30:16.770 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:16.770 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:30:16.770 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045712+0000", 2024-09-14T06:30:16.770 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:16.770 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:16.770 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:16.771 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:30:16.771 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045712+0000", 2024-09-14T06:30:16.771 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:16.771 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:16.771 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:16.771 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:30:16.771 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045712+0000", 2024-09-14T06:30:16.771 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.2100000000000001e-07 2024-09-14T06:30:16.771 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:16.771 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:16.771 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:30:16.771 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045713+0000", 2024-09-14T06:30:16.772 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.068e-06 2024-09-14T06:30:16.772 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:16.772 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:16.772 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:30:16.772 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045716+0000", 2024-09-14T06:30:16.772 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.971e-06 2024-09-14T06:30:16.773 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:16.773 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:16.773 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:30:16.773 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045719+0000", 2024-09-14T06:30:16.773 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.3879999999999998e-06 2024-09-14T06:30:16.773 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:16.773 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:16.773 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:30:16.773 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.078049+0000", 2024-09-14T06:30:16.773 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032330553999999997 2024-09-14T06:30:16.773 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:16.774 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:16.774 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:30:16.774 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.078059+0000", 2024-09-14T06:30:16.774 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.6700000000000006e-06 2024-09-14T06:30:16.774 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:16.774 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:30:16.774 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:16.774 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:16.775 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:16.775 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:98516 3.10 3.1bf56f90 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T06:30:16.775 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:30:07.045722+0000", 2024-09-14T06:30:16.775 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 9.6714336240000005, 2024-09-14T06:30:16.775 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032374334999999997, 2024-09-14T06:30:16.776 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:30:16.776 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:30:16.776 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "reached pg", 2024-09-14T06:30:16.776 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:30:16.776 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:30:16.776 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:16.776 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 98516 2024-09-14T06:30:16.777 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:16.777 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:30:16.777 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:16.777 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:30:16.777 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045722+0000", 2024-09-14T06:30:16.777 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:16.777 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:16.777 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:16.777 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:30:16.778 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045722+0000", 2024-09-14T06:30:16.778 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:16.778 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:16.778 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:16.778 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:30:16.778 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045722+0000", 2024-09-14T06:30:16.778 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.8999999999999998e-07 2024-09-14T06:30:16.778 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:16.779 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:16.779 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:30:16.779 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045723+0000", 2024-09-14T06:30:16.779 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.0049999999999999e-06 2024-09-14T06:30:16.779 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:16.779 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:16.779 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:30:16.779 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045726+0000", 2024-09-14T06:30:16.779 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.9019999999999999e-06 2024-09-14T06:30:16.779 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:16.780 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:16.780 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:30:16.780 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045728+0000", 2024-09-14T06:30:16.780 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.3290000000000001e-06 2024-09-14T06:30:16.780 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:16.780 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:16.780 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:30:16.780 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.078090+0000", 2024-09-14T06:30:16.780 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032362123 2024-09-14T06:30:16.780 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:16.780 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:16.780 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:30:16.780 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.078096+0000", 2024-09-14T06:30:16.781 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.6860000000000003e-06 2024-09-14T06:30:16.781 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:16.781 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:30:16.781 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:16.781 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:16.781 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:16.781 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:98517 3.10 3.1bf56f90 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T06:30:16.781 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:30:07.045731+0000", 2024-09-14T06:30:16.781 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 9.671423893, 2024-09-14T06:30:16.781 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.03241227, 2024-09-14T06:30:16.781 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:30:16.781 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:30:16.782 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "reached pg", 2024-09-14T06:30:16.782 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:30:16.782 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:30:16.782 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:16.782 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 98517 2024-09-14T06:30:16.782 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:16.782 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:30:16.782 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:16.782 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:30:16.782 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045731+0000", 2024-09-14T06:30:16.782 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:16.782 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:16.782 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:16.782 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:30:16.783 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045731+0000", 2024-09-14T06:30:16.783 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:16.783 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:16.783 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:16.783 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:30:16.783 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045732+0000", 2024-09-14T06:30:16.783 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.0699999999999999e-07 2024-09-14T06:30:16.783 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:16.783 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:16.783 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:30:16.783 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045733+0000", 2024-09-14T06:30:16.783 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.9199999999999999e-07 2024-09-14T06:30:16.783 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:16.783 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:16.784 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:30:16.784 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045735+0000", 2024-09-14T06:30:16.784 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.8150000000000002e-06 2024-09-14T06:30:16.784 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:16.784 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:16.784 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:30:16.784 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045738+0000", 2024-09-14T06:30:16.784 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.3640000000000002e-06 2024-09-14T06:30:16.784 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:16.784 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:16.784 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:30:16.784 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.078126+0000", 2024-09-14T06:30:16.784 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032388206000000003 2024-09-14T06:30:16.785 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:16.785 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:16.785 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:30:16.785 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.078144+0000", 2024-09-14T06:30:16.785 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.7686e-05 2024-09-14T06:30:16.785 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:16.785 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:30:16.785 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:16.785 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:16.785 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:16.785 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:98518 3.10 3.1bf56f90 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T06:30:16.785 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:30:07.045741+0000", 2024-09-14T06:30:16.785 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 9.6714144980000007, 2024-09-14T06:30:16.786 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032451793999999999, 2024-09-14T06:30:16.786 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:30:16.786 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:30:16.786 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "reached pg", 2024-09-14T06:30:16.786 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:30:16.786 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:30:16.786 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:16.786 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 98518 2024-09-14T06:30:16.786 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:16.786 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:30:16.786 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:16.786 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:30:16.786 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045741+0000", 2024-09-14T06:30:16.786 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:16.787 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:16.787 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:16.787 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:30:16.787 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045741+0000", 2024-09-14T06:30:16.787 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:16.787 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:16.787 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:16.787 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:30:16.787 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045741+0000", 2024-09-14T06:30:16.787 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.05e-07 2024-09-14T06:30:16.787 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:16.787 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:16.787 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:30:16.787 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045746+0000", 2024-09-14T06:30:16.788 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.2259999999999996e-06 2024-09-14T06:30:16.788 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:16.788 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:16.788 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:30:16.788 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045750+0000", 2024-09-14T06:30:16.788 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.2490000000000001e-06 2024-09-14T06:30:16.788 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:16.788 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:16.788 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:30:16.788 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045752+0000", 2024-09-14T06:30:16.788 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.3980000000000002e-06 2024-09-14T06:30:16.788 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:16.788 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:16.789 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:30:16.789 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.078184+0000", 2024-09-14T06:30:16.789 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032431593000000002 2024-09-14T06:30:16.789 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:16.789 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:16.789 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:30:16.789 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.078193+0000", 2024-09-14T06:30:16.789 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.1230000000000006e-06 2024-09-14T06:30:16.789 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:16.789 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:30:16.790 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:16.790 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:16.790 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:16.790 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:98519 3.10 3.1bf56f90 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T06:30:16.790 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:30:07.045755+0000", 2024-09-14T06:30:16.790 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 9.6714000730000009, 2024-09-14T06:30:16.790 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032489431999999999, 2024-09-14T06:30:16.790 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:30:16.790 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:30:16.790 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "reached pg", 2024-09-14T06:30:16.790 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:30:16.790 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:30:16.790 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:16.790 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 98519 2024-09-14T06:30:16.791 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:16.791 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:30:16.791 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:16.791 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:30:16.791 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045755+0000", 2024-09-14T06:30:16.791 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:16.791 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:16.791 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:16.791 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:30:16.792 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045755+0000", 2024-09-14T06:30:16.792 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:16.792 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:16.792 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:16.792 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:30:16.792 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045756+0000", 2024-09-14T06:30:16.792 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.23e-07 2024-09-14T06:30:16.792 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:16.792 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:16.793 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:30:16.793 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045757+0000", 2024-09-14T06:30:16.793 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.1060000000000001e-06 2024-09-14T06:30:16.793 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:16.793 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:16.793 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:30:16.793 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045760+0000", 2024-09-14T06:30:16.793 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.988e-06 2024-09-14T06:30:16.793 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:16.793 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:16.793 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:30:16.793 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045765+0000", 2024-09-14T06:30:16.793 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.2689999999999999e-06 2024-09-14T06:30:16.794 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:16.794 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:16.794 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:30:16.794 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.078234+0000", 2024-09-14T06:30:16.794 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032468915000000001 2024-09-14T06:30:16.794 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:16.794 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:16.794 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:30:16.794 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.078245+0000", 2024-09-14T06:30:16.794 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.0930999999999999e-05 2024-09-14T06:30:16.794 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:16.794 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:30:16.794 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:16.794 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:16.795 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:16.795 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:98520 3.10 3.1bf56f90 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T06:30:16.795 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:30:07.045768+0000", 2024-09-14T06:30:16.795 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 9.6713871949999994, 2024-09-14T06:30:16.795 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032531379999999999, 2024-09-14T06:30:16.795 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:30:16.795 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:30:16.795 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "reached pg", 2024-09-14T06:30:16.795 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:30:16.795 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:30:16.795 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:16.795 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 98520 2024-09-14T06:30:16.795 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:16.796 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:30:16.796 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:16.796 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:30:16.796 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045768+0000", 2024-09-14T06:30:16.796 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:16.796 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:16.796 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:16.796 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:30:16.796 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045768+0000", 2024-09-14T06:30:16.796 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:16.796 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:16.796 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:16.796 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:30:16.797 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045768+0000", 2024-09-14T06:30:16.797 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.22e-07 2024-09-14T06:30:16.797 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:16.797 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:16.797 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:30:16.797 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045773+0000", 2024-09-14T06:30:16.797 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.3470000000000001e-06 2024-09-14T06:30:16.797 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:16.797 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:16.797 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:30:16.797 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045776+0000", 2024-09-14T06:30:16.798 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.1319999999999998e-06 2024-09-14T06:30:16.798 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:16.798 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:16.798 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:30:16.798 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045779+0000", 2024-09-14T06:30:16.798 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.9749999999999999e-06 2024-09-14T06:30:16.798 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:16.798 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:16.798 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:30:16.798 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.078289+0000", 2024-09-14T06:30:16.798 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032509693999999999 2024-09-14T06:30:16.799 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:16.799 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:16.799 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:30:16.799 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.078300+0000", 2024-09-14T06:30:16.799 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.101e-05 2024-09-14T06:30:16.799 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:16.799 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:30:16.799 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:16.799 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:16.799 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:16.799 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:98521 3.10 3.1bf56f90 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T06:30:16.799 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:30:07.045782+0000", 2024-09-14T06:30:16.799 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 9.6713730150000004, 2024-09-14T06:30:16.800 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032566109000000003, 2024-09-14T06:30:16.800 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:30:16.800 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:30:16.800 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "reached pg", 2024-09-14T06:30:16.800 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:30:16.800 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:30:16.800 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:16.800 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 98521 2024-09-14T06:30:16.800 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:16.800 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:30:16.800 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:16.800 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:30:16.801 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045782+0000", 2024-09-14T06:30:16.801 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:16.801 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:16.801 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:16.801 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:30:16.801 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045782+0000", 2024-09-14T06:30:16.801 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:16.801 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:16.801 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:16.801 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:30:16.801 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045783+0000", 2024-09-14T06:30:16.801 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.2499999999999999e-07 2024-09-14T06:30:16.801 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:16.801 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:16.802 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:30:16.802 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045784+0000", 2024-09-14T06:30:16.802 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.0109999999999999e-06 2024-09-14T06:30:16.802 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:16.802 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:16.802 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:30:16.802 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045786+0000", 2024-09-14T06:30:16.802 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.8289999999999999e-06 2024-09-14T06:30:16.802 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:16.802 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:16.803 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:30:16.803 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045793+0000", 2024-09-14T06:30:16.803 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.579e-06 2024-09-14T06:30:16.803 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:16.803 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:16.803 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:30:16.803 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.078338+0000", 2024-09-14T06:30:16.803 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032545389000000001 2024-09-14T06:30:16.803 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:16.803 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:16.803 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:30:16.803 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.078348+0000", 2024-09-14T06:30:16.803 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.0076e-05 2024-09-14T06:30:16.804 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:16.804 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:30:16.804 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:16.804 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:16.804 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:16.804 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:98522 3.10 3.1bf56f90 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T06:30:16.804 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:30:07.045796+0000", 2024-09-14T06:30:16.804 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 9.6713591070000007, 2024-09-14T06:30:16.804 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032577846000000001, 2024-09-14T06:30:16.804 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:30:16.804 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:30:16.804 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "reached pg", 2024-09-14T06:30:16.805 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:30:16.805 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:30:16.805 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:16.805 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 98522 2024-09-14T06:30:16.805 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:16.805 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:30:16.805 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:16.805 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:30:16.805 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045796+0000", 2024-09-14T06:30:16.805 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:16.805 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:16.805 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:16.805 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:30:16.805 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045796+0000", 2024-09-14T06:30:16.806 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:16.806 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:16.806 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:16.806 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:30:16.806 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045796+0000", 2024-09-14T06:30:16.806 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.2100000000000001e-07 2024-09-14T06:30:16.806 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:16.806 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:16.806 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:30:16.806 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045797+0000", 2024-09-14T06:30:16.806 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.6899999999999996e-07 2024-09-14T06:30:16.806 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:16.806 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:16.807 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:30:16.807 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045800+0000", 2024-09-14T06:30:16.807 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.9979999999999999e-06 2024-09-14T06:30:16.807 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:16.807 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:16.807 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:30:16.807 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045803+0000", 2024-09-14T06:30:16.807 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.2850000000000001e-06 2024-09-14T06:30:16.807 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:16.807 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:16.807 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:30:16.807 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.078366+0000", 2024-09-14T06:30:16.807 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032562932000000003 2024-09-14T06:30:16.807 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:16.808 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:16.808 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:30:16.808 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.078374+0000", 2024-09-14T06:30:16.808 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.4409999999999993e-06 2024-09-14T06:30:16.808 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:16.808 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:30:16.808 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:16.808 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:16.808 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:16.808 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:98523 3.10 3.1bf56f90 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T06:30:16.809 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:30:07.045806+0000", 2024-09-14T06:30:16.809 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 9.6713495639999998, 2024-09-14T06:30:16.809 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032611167000000003, 2024-09-14T06:30:16.809 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:30:16.809 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:30:16.809 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "reached pg", 2024-09-14T06:30:16.809 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:30:16.809 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:30:16.809 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:16.809 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 98523 2024-09-14T06:30:16.809 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:16.809 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:30:16.810 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:16.810 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:30:16.810 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045806+0000", 2024-09-14T06:30:16.810 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:16.810 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:16.810 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:16.810 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:30:16.810 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045806+0000", 2024-09-14T06:30:16.810 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:16.810 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:16.810 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:16.810 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:30:16.810 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045806+0000", 2024-09-14T06:30:16.810 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.2000000000000001e-07 2024-09-14T06:30:16.811 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:16.811 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:16.811 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:30:16.811 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045807+0000", 2024-09-14T06:30:16.811 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.4e-07 2024-09-14T06:30:16.811 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:16.811 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:16.811 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:30:16.811 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045810+0000", 2024-09-14T06:30:16.811 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.9229999999999998e-06 2024-09-14T06:30:16.811 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:16.811 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:16.811 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:30:16.811 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045812+0000", 2024-09-14T06:30:16.812 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.2220000000000001e-06 2024-09-14T06:30:16.812 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:16.812 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:16.812 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:30:16.812 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.078407+0000", 2024-09-14T06:30:16.812 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032594513999999998 2024-09-14T06:30:16.812 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:16.812 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:16.812 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:30:16.812 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.078417+0000", 2024-09-14T06:30:16.812 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.0348e-05 2024-09-14T06:30:16.812 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:16.812 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:30:16.813 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:16.813 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:16.813 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:16.813 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:98524 3.10 3.1bf56f90 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T06:30:16.813 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:30:07.045815+0000", 2024-09-14T06:30:16.813 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 9.6713401959999992, 2024-09-14T06:30:16.813 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032643208999999999, 2024-09-14T06:30:16.813 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:30:16.813 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:30:16.813 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "reached pg", 2024-09-14T06:30:16.814 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:30:16.814 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:30:16.814 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:16.814 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 98524 2024-09-14T06:30:16.814 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:16.814 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:30:16.814 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:16.814 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:30:16.814 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045815+0000", 2024-09-14T06:30:16.814 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:16.814 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:16.815 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:16.815 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:30:16.815 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045815+0000", 2024-09-14T06:30:16.815 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:16.815 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:16.815 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:16.815 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:30:16.815 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045815+0000", 2024-09-14T06:30:16.815 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.11e-07 2024-09-14T06:30:16.815 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:16.815 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:16.816 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:30:16.816 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045816+0000", 2024-09-14T06:30:16.816 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.9099999999999991e-07 2024-09-14T06:30:16.816 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:16.816 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:16.816 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:30:16.816 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045819+0000", 2024-09-14T06:30:16.816 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.9399999999999998e-06 2024-09-14T06:30:16.816 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:16.816 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:16.816 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:30:16.816 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045822+0000", 2024-09-14T06:30:16.816 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.3470000000000001e-06 2024-09-14T06:30:16.817 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:16.817 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:16.817 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:30:16.817 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.078449+0000", 2024-09-14T06:30:16.817 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032627529000000002 2024-09-14T06:30:16.817 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:16.817 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:16.817 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:30:16.817 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.078458+0000", 2024-09-14T06:30:16.817 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.1910000000000006e-06 2024-09-14T06:30:16.817 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:16.817 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:30:16.817 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:16.818 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:16.818 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:30:16.818 INFO:teuthology.orchestra.run.smithi138.stdout:} 2024-09-14T06:30:16.819 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-14T06:30:16.820 INFO:tasks.ceph.osd.5.smithi138.stderr:2024-09-14T06:30:16.818+0000 7fe13746f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33618) UID: 0 2024-09-14T06:30:16.923 INFO:tasks.ceph.osd.2.smithi094.stderr:2024-09-14T06:30:16.921+0000 7faf3269b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33649) UID: 0 2024-09-14T06:30:16.963 INFO:teuthology.orchestra.run.smithi138.stdout:{ 2024-09-14T06:30:16.963 INFO:teuthology.orchestra.run.smithi138.stdout: "ops": [], 2024-09-14T06:30:16.963 INFO:teuthology.orchestra.run.smithi138.stdout: "num_ops": 0 2024-09-14T06:30:16.963 INFO:teuthology.orchestra.run.smithi138.stdout:} 2024-09-14T06:30:16.972 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-14T06:30:17.023 INFO:tasks.ceph.osd.2.smithi094.stderr:2024-09-14T06:30:17.022+0000 7faf3269b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33649) UID: 0 2024-09-14T06:30:17.074 INFO:teuthology.orchestra.run.smithi138.stdout:{ 2024-09-14T06:30:17.074 INFO:teuthology.orchestra.run.smithi138.stdout: "ops": [], 2024-09-14T06:30:17.074 INFO:teuthology.orchestra.run.smithi138.stdout: "complaint_time": 30, 2024-09-14T06:30:17.074 INFO:teuthology.orchestra.run.smithi138.stdout: "num_blocked_ops": 0 2024-09-14T06:30:17.075 INFO:teuthology.orchestra.run.smithi138.stdout:} 2024-09-14T06:30:17.082 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-14T06:30:17.125 INFO:tasks.ceph.osd.2.smithi094.stderr:2024-09-14T06:30:17.124+0000 7faf3269b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33649) UID: 0 2024-09-14T06:30:17.188 INFO:teuthology.orchestra.run.smithi138.stdout:{ 2024-09-14T06:30:17.188 INFO:teuthology.orchestra.run.smithi138.stdout: "size": 20, 2024-09-14T06:30:17.188 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 600, 2024-09-14T06:30:17.188 INFO:teuthology.orchestra.run.smithi138.stdout: "ops": [ 2024-09-14T06:30:17.189 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:17.189 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:53244 3.f 3:f71f51a7:::rbd_data.112dca2f6ae1.0000000000000015:17 [read 1863680~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:30:17.189 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:44.369399+0000", 2024-09-14T06:30:17.189 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 32.816603608000001, 2024-09-14T06:30:17.189 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047291189999999997, 2024-09-14T06:30:17.189 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:30:17.189 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:30:17.189 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:30:17.189 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:30:17.190 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:30:17.190 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:17.190 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 53244 2024-09-14T06:30:17.190 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:17.190 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:30:17.190 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:17.190 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:30:17.190 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369399+0000", 2024-09-14T06:30:17.190 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:17.190 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:17.191 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:17.191 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:30:17.191 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369399+0000", 2024-09-14T06:30:17.191 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:17.191 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:17.191 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:17.191 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:30:17.191 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369400+0000", 2024-09-14T06:30:17.191 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.0699999999999999e-07 2024-09-14T06:30:17.191 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:17.191 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:17.191 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:30:17.191 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369401+0000", 2024-09-14T06:30:17.192 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.4300000000000001e-06 2024-09-14T06:30:17.192 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:17.192 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:17.192 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:30:17.192 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369404+0000", 2024-09-14T06:30:17.192 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.061e-06 2024-09-14T06:30:17.192 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:17.192 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:17.192 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:30:17.192 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369407+0000", 2024-09-14T06:30:17.192 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.6139999999999999e-06 2024-09-14T06:30:17.192 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:17.193 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:17.193 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:30:17.193 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.416517+0000", 2024-09-14T06:30:17.193 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047110338000000002 2024-09-14T06:30:17.193 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:17.193 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:17.193 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:30:17.193 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.416616+0000", 2024-09-14T06:30:17.193 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.8651000000000001e-05 2024-09-14T06:30:17.193 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:17.193 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:17.193 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:30:17.193 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.416691+0000", 2024-09-14T06:30:17.193 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.4888999999999994e-05 2024-09-14T06:30:17.194 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:17.194 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:30:17.194 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:17.194 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:17.194 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:17.194 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:53245 3.f 3:f71f51a7:::rbd_data.112dca2f6ae1.0000000000000015:17 [read 1888256~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:30:17.194 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:44.369410+0000", 2024-09-14T06:30:17.194 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 32.816593320000003, 2024-09-14T06:30:17.194 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047399275999999997, 2024-09-14T06:30:17.194 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:30:17.194 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:30:17.194 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:30:17.194 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:30:17.194 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:30:17.195 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:17.195 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 53245 2024-09-14T06:30:17.195 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:17.195 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:30:17.195 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:17.195 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:30:17.195 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369410+0000", 2024-09-14T06:30:17.195 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:17.195 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:17.195 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:17.195 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:30:17.195 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369410+0000", 2024-09-14T06:30:17.195 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:17.196 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:17.196 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:17.196 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:30:17.196 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369410+0000", 2024-09-14T06:30:17.196 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.1899999999999999e-07 2024-09-14T06:30:17.196 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:17.196 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:17.196 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:30:17.196 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369411+0000", 2024-09-14T06:30:17.196 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.0300000000000001e-06 2024-09-14T06:30:17.196 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:17.196 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:17.196 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:30:17.196 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369414+0000", 2024-09-14T06:30:17.197 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.1099999999999999e-06 2024-09-14T06:30:17.197 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:17.197 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:17.197 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:30:17.197 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369417+0000", 2024-09-14T06:30:17.197 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.6189999999999998e-06 2024-09-14T06:30:17.197 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:17.197 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:17.197 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:30:17.197 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.416718+0000", 2024-09-14T06:30:17.197 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047301322999999999 2024-09-14T06:30:17.197 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:17.197 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:17.197 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:30:17.197 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.416759+0000", 2024-09-14T06:30:17.198 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.1009e-05 2024-09-14T06:30:17.198 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:17.198 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:17.198 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:30:17.198 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.416809+0000", 2024-09-14T06:30:17.198 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.9966000000000002e-05 2024-09-14T06:30:17.198 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:17.198 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:30:17.198 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:17.198 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:17.198 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:17.198 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:53246 3.f 3:f71f51a7:::rbd_data.112dca2f6ae1.0000000000000015:17 [read 1912832~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:30:17.198 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:44.369420+0000", 2024-09-14T06:30:17.198 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 32.816583375999997, 2024-09-14T06:30:17.199 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047529463000000001, 2024-09-14T06:30:17.199 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:30:17.199 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:30:17.199 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:30:17.199 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:30:17.199 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:30:17.200 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:17.200 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 53246 2024-09-14T06:30:17.200 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:17.200 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:30:17.200 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:17.201 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:30:17.201 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369420+0000", 2024-09-14T06:30:17.201 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:17.201 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:17.201 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:17.202 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:30:17.202 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369420+0000", 2024-09-14T06:30:17.202 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:17.202 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:17.202 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:17.202 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:30:17.203 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369420+0000", 2024-09-14T06:30:17.203 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.1299999999999999e-07 2024-09-14T06:30:17.203 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:17.203 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:17.203 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:30:17.203 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369425+0000", 2024-09-14T06:30:17.203 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.0130000000000001e-06 2024-09-14T06:30:17.204 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:17.204 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:17.204 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:30:17.204 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369428+0000", 2024-09-14T06:30:17.204 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.349e-06 2024-09-14T06:30:17.204 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:17.204 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:17.204 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:30:17.204 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369431+0000", 2024-09-14T06:30:17.204 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.7389999999999999e-06 2024-09-14T06:30:17.204 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:17.204 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:17.204 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:30:17.204 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.416834+0000", 2024-09-14T06:30:17.205 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047402669000000001 2024-09-14T06:30:17.205 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:17.205 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:17.205 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:30:17.205 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.416882+0000", 2024-09-14T06:30:17.205 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.8068999999999997e-05 2024-09-14T06:30:17.205 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:17.205 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:17.205 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:30:17.205 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.416949+0000", 2024-09-14T06:30:17.205 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.7410999999999995e-05 2024-09-14T06:30:17.205 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:17.205 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:30:17.205 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:17.206 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:17.206 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:17.206 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:53247 3.f 3:f71f51a7:::rbd_data.112dca2f6ae1.0000000000000015:17 [read 1937408~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:30:17.206 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:44.369434+0000", 2024-09-14T06:30:17.206 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 32.816568611999998, 2024-09-14T06:30:17.206 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047652264999999999, 2024-09-14T06:30:17.206 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:30:17.206 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:30:17.206 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:30:17.206 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:30:17.206 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:30:17.206 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:17.206 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 53247 2024-09-14T06:30:17.206 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:17.207 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:30:17.207 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:17.207 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:30:17.207 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369434+0000", 2024-09-14T06:30:17.207 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:17.207 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:17.207 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:17.207 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:30:17.207 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369434+0000", 2024-09-14T06:30:17.207 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:17.207 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:17.207 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:17.207 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:30:17.207 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369435+0000", 2024-09-14T06:30:17.208 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.2100000000000001e-07 2024-09-14T06:30:17.208 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:17.208 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:17.208 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:30:17.208 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369436+0000", 2024-09-14T06:30:17.208 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.122e-06 2024-09-14T06:30:17.208 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:17.208 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:17.208 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:30:17.208 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369439+0000", 2024-09-14T06:30:17.208 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.0400000000000001e-06 2024-09-14T06:30:17.208 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:17.208 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:17.208 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:30:17.209 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369441+0000", 2024-09-14T06:30:17.209 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.728e-06 2024-09-14T06:30:17.209 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:17.209 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:17.209 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:30:17.209 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.416977+0000", 2024-09-14T06:30:17.209 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047535344 2024-09-14T06:30:17.209 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:17.209 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:17.209 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:30:17.209 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.417029+0000", 2024-09-14T06:30:17.209 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.2037999999999999e-05 2024-09-14T06:30:17.209 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:17.209 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:17.210 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:30:17.210 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.417087+0000", 2024-09-14T06:30:17.210 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.7772000000000002e-05 2024-09-14T06:30:17.210 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:17.210 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:30:17.210 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:17.210 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:17.210 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:17.210 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:53248 3.f 3:f71f51a7:::rbd_data.112dca2f6ae1.0000000000000015:17 [read 1961984~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:30:17.210 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:44.369445+0000", 2024-09-14T06:30:17.210 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 32.816558059999998, 2024-09-14T06:30:17.210 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047798489, 2024-09-14T06:30:17.211 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:30:17.211 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:30:17.211 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:30:17.211 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:30:17.211 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:30:17.211 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:17.211 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 53248 2024-09-14T06:30:17.211 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:17.211 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:30:17.211 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:17.211 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:30:17.212 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369445+0000", 2024-09-14T06:30:17.212 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:17.212 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:17.212 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:17.212 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:30:17.212 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369445+0000", 2024-09-14T06:30:17.212 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:17.212 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:17.212 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:17.212 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:30:17.212 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369445+0000", 2024-09-14T06:30:17.212 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.0699999999999999e-07 2024-09-14T06:30:17.213 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:17.213 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:17.213 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:30:17.213 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369446+0000", 2024-09-14T06:30:17.213 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.1000000000000001e-06 2024-09-14T06:30:17.213 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:17.213 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:17.213 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:30:17.213 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369449+0000", 2024-09-14T06:30:17.213 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.1039999999999998e-06 2024-09-14T06:30:17.213 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:17.213 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:17.213 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:30:17.213 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369452+0000", 2024-09-14T06:30:17.214 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.6199999999999999e-06 2024-09-14T06:30:17.214 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:17.214 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:17.214 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:30:17.214 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.417114+0000", 2024-09-14T06:30:17.214 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047661901999999999 2024-09-14T06:30:17.214 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:17.214 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:17.214 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:30:17.214 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.417181+0000", 2024-09-14T06:30:17.214 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.7385999999999995e-05 2024-09-14T06:30:17.214 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:17.214 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:17.214 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:30:17.215 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.417243+0000", 2024-09-14T06:30:17.215 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.2169999999999996e-05 2024-09-14T06:30:17.215 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:17.215 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:30:17.215 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:17.215 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:17.215 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:17.215 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:53249 3.f 3:f71f51a7:::rbd_data.112dca2f6ae1.0000000000000015:17 [read 1986560~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:30:17.215 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:44.369455+0000", 2024-09-14T06:30:17.215 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 32.816547749000001, 2024-09-14T06:30:17.215 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047890493999999999, 2024-09-14T06:30:17.215 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:30:17.215 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:30:17.215 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:30:17.216 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:30:17.216 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:30:17.216 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:17.216 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 53249 2024-09-14T06:30:17.216 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:17.216 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:30:17.216 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:17.216 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:30:17.216 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369455+0000", 2024-09-14T06:30:17.216 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:17.216 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:17.216 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:17.216 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:30:17.217 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369455+0000", 2024-09-14T06:30:17.217 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:17.217 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:17.217 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:17.217 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:30:17.217 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369455+0000", 2024-09-14T06:30:17.217 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.0800000000000001e-07 2024-09-14T06:30:17.217 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:17.217 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:17.217 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:30:17.217 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369456+0000", 2024-09-14T06:30:17.217 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.0729999999999999e-06 2024-09-14T06:30:17.217 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:17.217 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:17.218 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:30:17.218 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369460+0000", 2024-09-14T06:30:17.218 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.027e-06 2024-09-14T06:30:17.218 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:17.218 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:17.218 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:30:17.218 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369462+0000", 2024-09-14T06:30:17.218 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.5320000000000001e-06 2024-09-14T06:30:17.218 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:17.218 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:17.218 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:30:17.218 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.417257+0000", 2024-09-14T06:30:17.218 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.04779489 2024-09-14T06:30:17.219 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:17.219 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:17.219 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:30:17.219 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.417296+0000", 2024-09-14T06:30:17.219 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.8658000000000002e-05 2024-09-14T06:30:17.219 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:17.219 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:17.219 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:30:17.219 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.417346+0000", 2024-09-14T06:30:17.219 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.0105999999999998e-05 2024-09-14T06:30:17.219 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:17.219 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:30:17.219 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:17.219 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:17.220 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:17.220 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:53250 3.f 3:f71f51a7:::rbd_data.112dca2f6ae1.0000000000000015:17 [read 2011136~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:30:17.220 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:44.369465+0000", 2024-09-14T06:30:17.220 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 32.816537779000001, 2024-09-14T06:30:17.220 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.048032230000000002, 2024-09-14T06:30:17.220 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:30:17.220 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:30:17.220 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:30:17.220 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:30:17.220 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:30:17.220 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:17.220 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 53250 2024-09-14T06:30:17.220 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:17.221 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:30:17.221 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:17.221 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:30:17.221 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369465+0000", 2024-09-14T06:30:17.221 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:17.221 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:17.221 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:17.221 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:30:17.221 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369465+0000", 2024-09-14T06:30:17.221 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:17.221 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:17.221 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:17.221 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:30:17.222 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369465+0000", 2024-09-14T06:30:17.222 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.2100000000000001e-07 2024-09-14T06:30:17.222 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:17.222 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:17.222 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:30:17.222 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369466+0000", 2024-09-14T06:30:17.222 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.0640000000000001e-06 2024-09-14T06:30:17.222 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:17.222 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:17.222 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:30:17.222 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369469+0000", 2024-09-14T06:30:17.223 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.0170000000000001e-06 2024-09-14T06:30:17.223 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:17.223 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:17.223 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:30:17.223 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369472+0000", 2024-09-14T06:30:17.223 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.6189999999999998e-06 2024-09-14T06:30:17.223 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:17.223 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:17.223 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:30:17.223 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.417383+0000", 2024-09-14T06:30:17.223 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047911092000000002 2024-09-14T06:30:17.223 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:17.223 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:17.223 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:30:17.224 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.417430+0000", 2024-09-14T06:30:17.224 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.6898999999999999e-05 2024-09-14T06:30:17.224 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:17.224 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:17.224 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:30:17.224 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.417497+0000", 2024-09-14T06:30:17.224 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.7317999999999995e-05 2024-09-14T06:30:17.224 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:17.224 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:30:17.224 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:17.224 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:17.225 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:17.225 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:53251 3.f 3:f71f51a7:::rbd_data.112dca2f6ae1.0000000000000015:17 [read 2035712~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:30:17.225 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:44.369475+0000", 2024-09-14T06:30:17.225 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 32.816527782999998, 2024-09-14T06:30:17.225 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.048154542000000002, 2024-09-14T06:30:17.225 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:30:17.225 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:30:17.225 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:30:17.226 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:30:17.226 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:30:17.226 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:17.226 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 53251 2024-09-14T06:30:17.226 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:17.226 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:30:17.226 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:17.226 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:30:17.226 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369475+0000", 2024-09-14T06:30:17.226 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:17.226 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:17.226 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:17.226 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:30:17.227 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369475+0000", 2024-09-14T06:30:17.227 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:17.227 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:17.227 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:17.227 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:30:17.227 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369475+0000", 2024-09-14T06:30:17.227 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.0100000000000001e-07 2024-09-14T06:30:17.227 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:17.227 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:17.227 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:30:17.227 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369476+0000", 2024-09-14T06:30:17.227 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.0580000000000001e-06 2024-09-14T06:30:17.227 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:17.227 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:17.227 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:30:17.228 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369479+0000", 2024-09-14T06:30:17.228 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.9950000000000001e-06 2024-09-14T06:30:17.228 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:17.228 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:17.228 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:30:17.228 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369482+0000", 2024-09-14T06:30:17.228 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.6340000000000001e-06 2024-09-14T06:30:17.228 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:17.228 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:17.228 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:30:17.228 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.417530+0000", 2024-09-14T06:30:17.228 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.048047535000000002 2024-09-14T06:30:17.228 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:17.229 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:17.229 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:30:17.229 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.417574+0000", 2024-09-14T06:30:17.229 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.4255999999999998e-05 2024-09-14T06:30:17.229 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:17.229 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:17.229 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:30:17.229 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.417630+0000", 2024-09-14T06:30:17.229 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.5862999999999997e-05 2024-09-14T06:30:17.229 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:17.229 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:30:17.230 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:17.230 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:17.230 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:17.230 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:53252 3.f 3:f71f51a7:::rbd_data.112dca2f6ae1.0000000000000015:17 [read 2060288~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:30:17.230 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:44.369485+0000", 2024-09-14T06:30:17.230 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 32.816517935, 2024-09-14T06:30:17.230 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.048254116, 2024-09-14T06:30:17.230 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:30:17.230 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:30:17.230 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:30:17.230 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:30:17.230 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:30:17.230 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:17.230 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 53252 2024-09-14T06:30:17.231 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:17.231 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:30:17.231 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:17.231 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:30:17.231 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369485+0000", 2024-09-14T06:30:17.231 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:17.231 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:17.231 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:17.231 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:30:17.231 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369485+0000", 2024-09-14T06:30:17.231 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:17.231 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:17.231 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:17.232 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:30:17.232 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369485+0000", 2024-09-14T06:30:17.232 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.1e-07 2024-09-14T06:30:17.232 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:17.232 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:17.232 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:30:17.232 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369487+0000", 2024-09-14T06:30:17.232 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.42e-06 2024-09-14T06:30:17.233 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:17.233 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:17.233 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:30:17.233 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369490+0000", 2024-09-14T06:30:17.233 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.9639999999999999e-06 2024-09-14T06:30:17.233 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:17.233 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:17.234 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:30:17.234 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369492+0000", 2024-09-14T06:30:17.234 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.5679999999999998e-06 2024-09-14T06:30:17.234 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:17.234 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:17.234 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:30:17.234 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.417645+0000", 2024-09-14T06:30:17.234 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.048152753 2024-09-14T06:30:17.234 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:17.234 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:17.234 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:30:17.234 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.417687+0000", 2024-09-14T06:30:17.234 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.1900999999999997e-05 2024-09-14T06:30:17.235 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:17.235 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:17.235 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:30:17.235 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.417739+0000", 2024-09-14T06:30:17.235 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.2299999999999997e-05 2024-09-14T06:30:17.235 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:17.235 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:30:17.235 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:17.235 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:17.235 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:17.235 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:53253 3.f 3:f71f51a7:::rbd_data.112dca2f6ae1.0000000000000015:17 [read 2084864~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:30:17.235 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:44.369495+0000", 2024-09-14T06:30:17.236 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 32.816507811000001, 2024-09-14T06:30:17.236 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.048366785000000002, 2024-09-14T06:30:17.236 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:30:17.236 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:30:17.236 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:30:17.236 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:30:17.236 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:30:17.236 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:17.236 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 53253 2024-09-14T06:30:17.236 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:17.236 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:30:17.236 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:17.236 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:30:17.237 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369495+0000", 2024-09-14T06:30:17.237 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:17.237 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:17.237 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:17.237 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:30:17.237 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369495+0000", 2024-09-14T06:30:17.237 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:17.237 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:17.237 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:17.237 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:30:17.237 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370388+0000", 2024-09-14T06:30:17.237 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00089287999999999998 2024-09-14T06:30:17.237 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:17.238 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:17.238 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:30:17.238 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370392+0000", 2024-09-14T06:30:17.238 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.2130000000000002e-06 2024-09-14T06:30:17.238 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:17.238 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:17.238 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:30:17.238 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370401+0000", 2024-09-14T06:30:17.238 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.7649999999999999e-06 2024-09-14T06:30:17.238 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:17.238 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:17.238 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:30:17.238 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370406+0000", 2024-09-14T06:30:17.238 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.0159999999999999e-06 2024-09-14T06:30:17.239 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:17.239 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:17.239 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:30:17.239 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.417764+0000", 2024-09-14T06:30:17.239 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047358348000000001 2024-09-14T06:30:17.239 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:17.239 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:17.239 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:30:17.239 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.417806+0000", 2024-09-14T06:30:17.239 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.1149999999999997e-05 2024-09-14T06:30:17.239 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:17.239 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:17.240 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:30:17.240 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.417862+0000", 2024-09-14T06:30:17.240 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.6412999999999997e-05 2024-09-14T06:30:17.240 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:17.240 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:30:17.240 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:17.240 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:17.240 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:17.240 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:53340 3.f 3:f71f51a7:::rbd_data.112dca2f6ae1.0000000000000015:17 [read 1056768~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:30:17.240 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:44.370421+0000", 2024-09-14T06:30:17.240 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 32.815581930999997, 2024-09-14T06:30:17.240 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047566313999999998, 2024-09-14T06:30:17.240 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:30:17.241 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:30:17.241 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:30:17.241 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:30:17.241 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:30:17.241 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:17.241 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 53340 2024-09-14T06:30:17.241 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:17.241 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:30:17.241 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:17.241 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:30:17.241 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370421+0000", 2024-09-14T06:30:17.241 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:17.241 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:17.241 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:17.242 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:30:17.242 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370421+0000", 2024-09-14T06:30:17.242 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:17.242 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:17.242 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:17.242 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:30:17.242 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370422+0000", 2024-09-14T06:30:17.242 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.3799999999999996e-07 2024-09-14T06:30:17.242 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:17.242 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:17.242 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:30:17.243 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370424+0000", 2024-09-14T06:30:17.243 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.6919999999999998e-06 2024-09-14T06:30:17.243 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:17.243 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:17.243 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:30:17.243 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370432+0000", 2024-09-14T06:30:17.243 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.2439999999999996e-06 2024-09-14T06:30:17.243 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:17.243 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:17.243 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:30:17.243 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370437+0000", 2024-09-14T06:30:17.243 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.5389999999999998e-06 2024-09-14T06:30:17.243 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:17.244 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:17.244 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:30:17.244 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.417888+0000", 2024-09-14T06:30:17.244 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047450432000000001 2024-09-14T06:30:17.244 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:17.244 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:17.244 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:30:17.244 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.417935+0000", 2024-09-14T06:30:17.244 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.7454999999999999e-05 2024-09-14T06:30:17.245 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:17.245 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:17.245 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:30:17.245 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.417987+0000", 2024-09-14T06:30:17.245 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.2213999999999998e-05 2024-09-14T06:30:17.245 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:17.245 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:30:17.245 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:17.245 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:17.245 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:17.245 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:53341 3.f 3:f71f51a7:::rbd_data.112dca2f6ae1.0000000000000015:17 [read 1081344~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:30:17.245 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:44.370446+0000", 2024-09-14T06:30:17.245 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 32.815556657999998, 2024-09-14T06:30:17.245 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047665723, 2024-09-14T06:30:17.246 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:30:17.246 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:30:17.246 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:30:17.246 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:30:17.246 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:30:17.246 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:17.246 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 53341 2024-09-14T06:30:17.246 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:17.246 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:30:17.246 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:17.246 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:30:17.246 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370446+0000", 2024-09-14T06:30:17.246 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:17.246 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:17.247 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:17.247 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:30:17.247 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370446+0000", 2024-09-14T06:30:17.247 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:17.247 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:17.247 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:17.247 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:30:17.247 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370447+0000", 2024-09-14T06:30:17.247 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.6100000000000002e-07 2024-09-14T06:30:17.247 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:17.247 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:17.247 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:30:17.248 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370448+0000", 2024-09-14T06:30:17.248 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.7370000000000001e-06 2024-09-14T06:30:17.248 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:17.248 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:17.248 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:30:17.248 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370462+0000", 2024-09-14T06:30:17.248 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.3227000000000001e-05 2024-09-14T06:30:17.248 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:17.248 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:17.248 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:30:17.248 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370467+0000", 2024-09-14T06:30:17.249 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.6540000000000004e-06 2024-09-14T06:30:17.249 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:17.249 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:17.249 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:30:17.249 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.418013+0000", 2024-09-14T06:30:17.249 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047545325999999999 2024-09-14T06:30:17.249 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:17.249 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:17.249 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:30:17.249 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.418054+0000", 2024-09-14T06:30:17.249 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.1696999999999997e-05 2024-09-14T06:30:17.249 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:17.249 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:17.250 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:30:17.250 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.418112+0000", 2024-09-14T06:30:17.250 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.7720999999999998e-05 2024-09-14T06:30:17.250 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:17.250 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:30:17.250 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:17.250 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:17.250 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:17.250 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:53342 3.f 3:f71f51a7:::rbd_data.112dca2f6ae1.0000000000000015:17 [read 1105920~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:30:17.250 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:44.370475+0000", 2024-09-14T06:30:17.250 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 32.815527596999999, 2024-09-14T06:30:17.250 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047748848000000003, 2024-09-14T06:30:17.250 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:30:17.250 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:30:17.251 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:30:17.251 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:30:17.251 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:30:17.251 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:17.251 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 53342 2024-09-14T06:30:17.251 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:17.251 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:30:17.251 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:17.251 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:30:17.251 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370475+0000", 2024-09-14T06:30:17.251 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:17.251 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:17.251 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:17.251 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:30:17.252 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370475+0000", 2024-09-14T06:30:17.252 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:17.252 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:17.252 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:17.252 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:30:17.252 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370476+0000", 2024-09-14T06:30:17.252 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.4700000000000002e-07 2024-09-14T06:30:17.252 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:17.252 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:17.252 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:30:17.252 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370478+0000", 2024-09-14T06:30:17.253 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.5890000000000001e-06 2024-09-14T06:30:17.253 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:17.253 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:17.253 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:30:17.253 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370486+0000", 2024-09-14T06:30:17.253 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.3470000000000001e-06 2024-09-14T06:30:17.253 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:17.253 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:17.253 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:30:17.253 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370491+0000", 2024-09-14T06:30:17.253 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.5720000000000002e-06 2024-09-14T06:30:17.254 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:17.254 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:17.254 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:30:17.254 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.418135+0000", 2024-09-14T06:30:17.254 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047644094999999997 2024-09-14T06:30:17.254 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:17.255 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:17.255 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:30:17.255 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.418174+0000", 2024-09-14T06:30:17.255 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.8204000000000002e-05 2024-09-14T06:30:17.255 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:17.255 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:17.255 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:30:17.255 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.418224+0000", 2024-09-14T06:30:17.255 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.0593999999999997e-05 2024-09-14T06:30:17.256 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:17.256 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:30:17.256 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:17.256 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:17.256 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:17.256 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:53343 3.f 3:f71f51a7:::rbd_data.112dca2f6ae1.0000000000000015:17 [read 1130496~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:30:17.256 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:44.370498+0000", 2024-09-14T06:30:17.256 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 32.815504721000003, 2024-09-14T06:30:17.256 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047841574999999997, 2024-09-14T06:30:17.256 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:30:17.256 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:30:17.256 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:30:17.256 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:30:17.256 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:30:17.257 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:17.257 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 53343 2024-09-14T06:30:17.257 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:17.257 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:30:17.257 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:17.257 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:30:17.257 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370498+0000", 2024-09-14T06:30:17.257 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:17.257 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:17.257 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:17.257 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:30:17.257 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370498+0000", 2024-09-14T06:30:17.257 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:17.258 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:17.258 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:17.258 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:30:17.258 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370499+0000", 2024-09-14T06:30:17.258 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.9999999999999998e-07 2024-09-14T06:30:17.258 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:17.258 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:17.258 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:30:17.258 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370501+0000", 2024-09-14T06:30:17.258 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.0310000000000001e-06 2024-09-14T06:30:17.258 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:17.258 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:17.258 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:30:17.259 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370507+0000", 2024-09-14T06:30:17.259 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.4169999999999997e-06 2024-09-14T06:30:17.259 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:17.259 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:17.259 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:30:17.259 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370516+0000", 2024-09-14T06:30:17.259 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.7190000000000007e-06 2024-09-14T06:30:17.259 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:17.259 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:17.259 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:30:17.259 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.418247+0000", 2024-09-14T06:30:17.259 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047731190999999999 2024-09-14T06:30:17.259 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:17.259 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:17.260 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:30:17.260 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.418286+0000", 2024-09-14T06:30:17.260 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.8988e-05 2024-09-14T06:30:17.260 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:17.260 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:17.260 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:30:17.260 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.418340+0000", 2024-09-14T06:30:17.260 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.3829000000000002e-05 2024-09-14T06:30:17.260 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:17.260 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:30:17.260 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:17.260 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:17.261 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:17.261 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:53344 3.f 3:f71f51a7:::rbd_data.112dca2f6ae1.0000000000000015:17 [read 1155072~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:30:17.261 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:44.370524+0000", 2024-09-14T06:30:17.261 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 32.815478951000003, 2024-09-14T06:30:17.261 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047972262000000002, 2024-09-14T06:30:17.261 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:30:17.261 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:30:17.261 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:30:17.261 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:30:17.261 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:30:17.262 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:17.262 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 53344 2024-09-14T06:30:17.262 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:17.262 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:30:17.262 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:17.262 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:30:17.262 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370524+0000", 2024-09-14T06:30:17.262 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:17.262 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:17.262 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:17.262 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:30:17.262 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370524+0000", 2024-09-14T06:30:17.262 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:17.262 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:17.263 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:17.263 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:30:17.263 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370618+0000", 2024-09-14T06:30:17.263 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.4425999999999993e-05 2024-09-14T06:30:17.263 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:17.263 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:17.263 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:30:17.263 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370622+0000", 2024-09-14T06:30:17.263 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.3519999999999998e-06 2024-09-14T06:30:17.263 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:17.263 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:17.263 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:30:17.263 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370631+0000", 2024-09-14T06:30:17.263 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.9299999999999992e-06 2024-09-14T06:30:17.264 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:17.264 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:17.264 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:30:17.264 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370637+0000", 2024-09-14T06:30:17.264 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.3269999999999997e-06 2024-09-14T06:30:17.264 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:17.264 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:17.264 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:30:17.264 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.418372+0000", 2024-09-14T06:30:17.264 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047734764999999998 2024-09-14T06:30:17.264 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:17.264 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:17.264 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:30:17.264 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.418431+0000", 2024-09-14T06:30:17.265 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.8696999999999997e-05 2024-09-14T06:30:17.265 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:17.265 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:17.265 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:30:17.265 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.418496+0000", 2024-09-14T06:30:17.265 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.5765000000000005e-05 2024-09-14T06:30:17.265 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:17.265 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:30:17.265 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:17.265 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:17.265 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:17.265 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:53345 3.f 3:f71f51a7:::rbd_data.112dca2f6ae1.0000000000000015:17 [read 1179648~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:30:17.265 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:44.370647+0000", 2024-09-14T06:30:17.266 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 32.815355484000001, 2024-09-14T06:30:17.266 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047953775999999997, 2024-09-14T06:30:17.266 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:30:17.266 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:30:17.266 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:30:17.266 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:30:17.266 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:30:17.266 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:17.266 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 53345 2024-09-14T06:30:17.266 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:17.266 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:30:17.266 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:17.266 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:30:17.266 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370647+0000", 2024-09-14T06:30:17.266 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:17.267 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:17.267 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:17.267 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:30:17.267 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370647+0000", 2024-09-14T06:30:17.267 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:17.267 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:17.267 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:17.267 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:30:17.267 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.372346+0000", 2024-09-14T06:30:17.267 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.001698284 2024-09-14T06:30:17.267 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:17.267 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:17.267 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:30:17.267 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.372351+0000", 2024-09-14T06:30:17.268 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.8890000000000001e-06 2024-09-14T06:30:17.268 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:17.268 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:17.268 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:30:17.268 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.372362+0000", 2024-09-14T06:30:17.268 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.0941000000000001e-05 2024-09-14T06:30:17.268 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:17.268 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:17.268 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:30:17.268 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.372369+0000", 2024-09-14T06:30:17.268 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.362e-06 2024-09-14T06:30:17.268 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:17.268 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:17.269 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:30:17.269 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.418510+0000", 2024-09-14T06:30:17.269 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046140914999999998 2024-09-14T06:30:17.269 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:17.269 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:17.269 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:30:17.269 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.418549+0000", 2024-09-14T06:30:17.269 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.8717999999999998e-05 2024-09-14T06:30:17.269 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:17.269 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:17.269 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:30:17.269 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.418601+0000", 2024-09-14T06:30:17.269 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.2667000000000002e-05 2024-09-14T06:30:17.270 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:17.270 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:30:17.270 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:17.270 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:17.270 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:17.270 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:53351 3.f 3:f71f51a7:::rbd_data.112dca2f6ae1.0000000000000015:17 [read 1327104~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:30:17.270 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:44.372499+0000", 2024-09-14T06:30:17.270 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 32.813504363, 2024-09-14T06:30:17.270 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047140596, 2024-09-14T06:30:17.270 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:30:17.270 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:30:17.271 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:30:17.271 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:30:17.271 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:30:17.271 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:17.271 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 53351 2024-09-14T06:30:17.271 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:17.271 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:30:17.271 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:17.271 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:30:17.271 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.372499+0000", 2024-09-14T06:30:17.271 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:17.271 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:17.271 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:17.271 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:30:17.272 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.372499+0000", 2024-09-14T06:30:17.272 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:17.272 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:17.272 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:17.272 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:30:17.272 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.372499+0000", 2024-09-14T06:30:17.272 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.3000000000000001e-07 2024-09-14T06:30:17.272 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:17.272 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:17.272 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:30:17.272 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.372501+0000", 2024-09-14T06:30:17.272 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.0820000000000001e-06 2024-09-14T06:30:17.272 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:17.273 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:17.273 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:30:17.273 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.372513+0000", 2024-09-14T06:30:17.273 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.1610999999999999e-05 2024-09-14T06:30:17.273 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:17.273 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:17.273 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:30:17.273 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.372518+0000", 2024-09-14T06:30:17.273 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.327e-06 2024-09-14T06:30:17.273 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:17.273 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:17.274 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:30:17.274 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.419530+0000", 2024-09-14T06:30:17.274 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047011798 2024-09-14T06:30:17.274 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:17.274 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:17.274 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:30:17.274 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.419578+0000", 2024-09-14T06:30:17.274 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.8446999999999999e-05 2024-09-14T06:30:17.275 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:17.275 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:17.275 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:30:17.275 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.419639+0000", 2024-09-14T06:30:17.275 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.0900999999999998e-05 2024-09-14T06:30:17.275 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:17.275 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:30:17.275 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:17.275 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:17.275 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:17.275 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:53352 3.f 3:f71f51a7:::rbd_data.112dca2f6ae1.0000000000000015:17 [read 1351680~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:30:17.275 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:44.372525+0000", 2024-09-14T06:30:17.275 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 32.813478449000002, 2024-09-14T06:30:17.275 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047259847000000001, 2024-09-14T06:30:17.276 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:30:17.276 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:30:17.276 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:30:17.276 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:30:17.276 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:30:17.276 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:17.276 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 53352 2024-09-14T06:30:17.276 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:17.276 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:30:17.276 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:17.276 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:30:17.276 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.372525+0000", 2024-09-14T06:30:17.277 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:17.277 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:17.277 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:17.277 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:30:17.277 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.372525+0000", 2024-09-14T06:30:17.277 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:17.277 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:17.277 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:17.277 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:30:17.277 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.372525+0000", 2024-09-14T06:30:17.277 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.0600000000000001e-07 2024-09-14T06:30:17.277 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:17.277 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:17.278 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:30:17.278 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.372527+0000", 2024-09-14T06:30:17.278 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.0499999999999999e-06 2024-09-14T06:30:17.278 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:17.278 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:17.278 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:30:17.278 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.372533+0000", 2024-09-14T06:30:17.278 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.4320000000000004e-06 2024-09-14T06:30:17.278 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:17.278 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:17.278 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:30:17.278 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.372538+0000", 2024-09-14T06:30:17.278 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.9130000000000002e-06 2024-09-14T06:30:17.278 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:17.279 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:17.279 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:30:17.279 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.419667+0000", 2024-09-14T06:30:17.279 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047128976000000003 2024-09-14T06:30:17.279 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:17.279 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:17.279 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:30:17.279 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.419715+0000", 2024-09-14T06:30:17.279 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.7868999999999999e-05 2024-09-14T06:30:17.279 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:17.279 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:17.279 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:30:17.279 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.419784+0000", 2024-09-14T06:30:17.280 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.9201000000000004e-05 2024-09-14T06:30:17.280 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:17.280 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:30:17.280 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:17.280 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:17.280 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:17.280 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:53353 3.f 3:f71f51a7:::rbd_data.112dca2f6ae1.0000000000000015:17 [read 1376256~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:30:17.280 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:44.372545+0000", 2024-09-14T06:30:17.280 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 32.813458408999999, 2024-09-14T06:30:17.280 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047364232999999999, 2024-09-14T06:30:17.281 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:30:17.281 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:30:17.281 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:30:17.281 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:30:17.281 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:30:17.281 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:17.281 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 53353 2024-09-14T06:30:17.281 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:17.281 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:30:17.281 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:17.281 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:30:17.281 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.372545+0000", 2024-09-14T06:30:17.281 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:17.282 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:17.282 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:17.282 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:30:17.282 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.372545+0000", 2024-09-14T06:30:17.282 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:17.282 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:17.282 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:17.282 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:30:17.282 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.372545+0000", 2024-09-14T06:30:17.282 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.27e-07 2024-09-14T06:30:17.282 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:17.282 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:17.282 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:30:17.282 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.372547+0000", 2024-09-14T06:30:17.282 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.0600000000000002e-06 2024-09-14T06:30:17.283 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:17.283 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:17.283 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:30:17.283 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.372553+0000", 2024-09-14T06:30:17.283 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.7980000000000002e-06 2024-09-14T06:30:17.283 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:17.283 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:17.283 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:30:17.283 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.372557+0000", 2024-09-14T06:30:17.283 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.7389999999999999e-06 2024-09-14T06:30:17.283 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:17.283 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:17.283 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:30:17.284 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.419811+0000", 2024-09-14T06:30:17.284 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047253310999999999 2024-09-14T06:30:17.284 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:17.284 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:17.284 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:30:17.284 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.419854+0000", 2024-09-14T06:30:17.284 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.3448000000000001e-05 2024-09-14T06:30:17.284 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:17.284 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:17.284 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:30:17.284 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.419909+0000", 2024-09-14T06:30:17.284 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.4549999999999998e-05 2024-09-14T06:30:17.284 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:17.284 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:30:17.285 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:17.285 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:17.285 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:17.285 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:53354 3.f 3:f71f51a7:::rbd_data.112dca2f6ae1.0000000000000015:17 [read 1400832~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:30:17.285 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:44.372563+0000", 2024-09-14T06:30:17.285 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 32.813439971999998, 2024-09-14T06:30:17.285 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047482233999999998, 2024-09-14T06:30:17.285 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:30:17.285 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:30:17.285 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:30:17.285 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:30:17.285 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:30:17.285 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:17.285 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 53354 2024-09-14T06:30:17.286 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:17.286 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:30:17.286 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:17.286 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:30:17.286 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.372563+0000", 2024-09-14T06:30:17.286 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:17.286 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:17.286 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:17.286 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:30:17.286 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.372563+0000", 2024-09-14T06:30:17.286 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:17.286 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:17.286 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:17.286 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:30:17.287 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.375009+0000", 2024-09-14T06:30:17.287 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.0024457680000000001 2024-09-14T06:30:17.287 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:17.287 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:17.287 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:30:17.287 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.375014+0000", 2024-09-14T06:30:17.287 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.0599999999999998e-06 2024-09-14T06:30:17.287 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:17.287 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:17.287 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:30:17.287 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.375026+0000", 2024-09-14T06:30:17.287 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.1783999999999999e-05 2024-09-14T06:30:17.287 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:17.287 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:17.288 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:30:17.288 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.375035+0000", 2024-09-14T06:30:17.288 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.7799999999999995e-06 2024-09-14T06:30:17.288 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:17.288 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:17.288 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:30:17.288 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.419942+0000", 2024-09-14T06:30:17.288 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.044906982999999998 2024-09-14T06:30:17.288 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:17.288 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:17.288 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:30:17.288 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.419989+0000", 2024-09-14T06:30:17.288 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.7104999999999997e-05 2024-09-14T06:30:17.288 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:17.289 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:17.289 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:30:17.289 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.420045+0000", 2024-09-14T06:30:17.289 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.5754000000000002e-05 2024-09-14T06:30:17.289 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:17.289 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:30:17.289 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:17.289 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:17.289 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:30:17.289 INFO:teuthology.orchestra.run.smithi138.stdout:} 2024-09-14T06:30:17.291 INFO:tasks.ceph.osd.4.smithi138.stderr:2024-09-14T06:30:17.289+0000 7f6c653c8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33615) UID: 0 2024-09-14T06:30:17.291 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-14T06:30:17.374 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-14T06:30:17.375 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 0 1 2024-09-14T06:30:17.408 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-14T06:30:17.409 INFO:teuthology.orchestra.run.smithi094.stdout: "ops": [], 2024-09-14T06:30:17.409 INFO:teuthology.orchestra.run.smithi094.stdout: "num_ops": 0 2024-09-14T06:30:17.409 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-14T06:30:17.409 INFO:tasks.ceph.osd.0.smithi094.stderr:2024-09-14T06:30:17.407+0000 7f07c9333640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33636) UID: 0 2024-09-14T06:30:17.418 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-14T06:30:17.502 INFO:tasks.ceph.osd.2.smithi094.stderr:2024-09-14T06:30:17.501+0000 7faf3269b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33649) UID: 0 2024-09-14T06:30:17.563 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-14T06:30:17.564 INFO:teuthology.orchestra.run.smithi094.stdout: "ops": [], 2024-09-14T06:30:17.564 INFO:teuthology.orchestra.run.smithi094.stdout: "complaint_time": 30, 2024-09-14T06:30:17.564 INFO:teuthology.orchestra.run.smithi094.stdout: "num_blocked_ops": 0 2024-09-14T06:30:17.564 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-14T06:30:17.572 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-14T06:30:17.601 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-14T06:30:17.614 INFO:tasks.ceph.osd.3.smithi094.stderr:2024-09-14T06:30:17.613+0000 7feeeafe8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33655) UID: 0 2024-09-14T06:30:17.693 INFO:tasks.ceph.osd.6.smithi138.stderr:2024-09-14T06:30:17.692+0000 7f8cd8849640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33620) UID: 0 2024-09-14T06:30:17.746 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-14T06:30:17.747 INFO:teuthology.orchestra.run.smithi094.stdout: "size": 20, 2024-09-14T06:30:17.747 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 600, 2024-09-14T06:30:17.747 INFO:teuthology.orchestra.run.smithi094.stdout: "ops": [ 2024-09-14T06:30:17.747 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:17.747 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:44355 3.17 3:ea948ecb:::rbd_data.112dca2f6ae1.0000000000000004:17 [read 196608~4096 [fadvise_nocache]] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:30:17.747 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:43.850672+0000", 2024-09-14T06:30:17.747 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 33.891813429000003, 2024-09-14T06:30:17.747 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.068480821999999997, 2024-09-14T06:30:17.747 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:30:17.747 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:30:17.747 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:30:17.747 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:30:17.748 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:30:17.748 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:17.748 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 44355 2024-09-14T06:30:17.748 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:17.748 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:30:17.748 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:17.748 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:30:17.748 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.850672+0000", 2024-09-14T06:30:17.748 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:17.748 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:17.748 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:17.748 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:30:17.748 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.850672+0000", 2024-09-14T06:30:17.748 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:17.749 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:17.749 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:17.749 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:30:17.749 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.850673+0000", 2024-09-14T06:30:17.749 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.0499999999999999e-07 2024-09-14T06:30:17.749 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:17.749 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:17.749 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:30:17.749 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.850675+0000", 2024-09-14T06:30:17.749 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.9400000000000001e-06 2024-09-14T06:30:17.749 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:17.750 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:17.750 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:30:17.750 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.850680+0000", 2024-09-14T06:30:17.750 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.4229999999999996e-06 2024-09-14T06:30:17.750 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:17.750 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:17.751 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:30:17.751 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.850685+0000", 2024-09-14T06:30:17.751 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.4920000000000004e-06 2024-09-14T06:30:17.751 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:17.751 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:17.751 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:30:17.751 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.919002+0000", 2024-09-14T06:30:17.752 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.068317131000000003 2024-09-14T06:30:17.752 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:17.752 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:17.752 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:30:17.752 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.919041+0000", 2024-09-14T06:30:17.752 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.9653999999999997e-05 2024-09-14T06:30:17.752 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:17.752 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:17.752 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:30:17.752 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.919153+0000", 2024-09-14T06:30:17.752 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000111777 2024-09-14T06:30:17.753 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:17.753 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:17.753 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:17.753 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:17.753 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:17.753 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:44356 3.17 3:ea948ecb:::rbd_data.112dca2f6ae1.0000000000000004:17 [read 221184~4096 [fadvise_nocache]] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:30:17.753 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:43.850690+0000", 2024-09-14T06:30:17.753 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 33.891795938000001, 2024-09-14T06:30:17.753 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.068790293000000002, 2024-09-14T06:30:17.753 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:30:17.753 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:30:17.753 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:30:17.753 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:30:17.753 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:30:17.754 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:17.754 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 44356 2024-09-14T06:30:17.754 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:17.754 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:30:17.754 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:17.754 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:30:17.754 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.850690+0000", 2024-09-14T06:30:17.754 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:17.754 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:17.754 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:17.754 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:30:17.755 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.850690+0000", 2024-09-14T06:30:17.755 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:17.755 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:17.755 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:17.755 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:30:17.755 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.850690+0000", 2024-09-14T06:30:17.755 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.5900000000000003e-07 2024-09-14T06:30:17.755 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:17.755 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:17.755 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:30:17.755 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.850692+0000", 2024-09-14T06:30:17.756 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.697e-06 2024-09-14T06:30:17.756 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:17.756 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:17.756 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:30:17.756 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.850782+0000", 2024-09-14T06:30:17.756 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.9777000000000005e-05 2024-09-14T06:30:17.756 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:17.756 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:17.756 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:30:17.756 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.850789+0000", 2024-09-14T06:30:17.756 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.063e-06 2024-09-14T06:30:17.756 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:17.757 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:17.757 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:30:17.757 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.919199+0000", 2024-09-14T06:30:17.757 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.068410152000000002 2024-09-14T06:30:17.757 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:17.757 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:17.757 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:30:17.757 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.919271+0000", 2024-09-14T06:30:17.757 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.1771000000000001e-05 2024-09-14T06:30:17.757 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:17.757 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:17.757 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:30:17.757 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.919480+0000", 2024-09-14T06:30:17.758 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00020947399999999999 2024-09-14T06:30:17.758 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:17.758 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:17.758 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:17.758 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:17.758 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:17.758 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:44357 3.17 3:ea948ecb:::rbd_data.112dca2f6ae1.0000000000000004:17 [read 245760~4096 [fadvise_nocache]] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:30:17.759 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:43.850800+0000", 2024-09-14T06:30:17.759 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 33.891685695, 2024-09-14T06:30:17.759 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.068852802000000005, 2024-09-14T06:30:17.759 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:30:17.759 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:30:17.759 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:30:17.760 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:30:17.760 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:30:17.760 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:17.760 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 44357 2024-09-14T06:30:17.760 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:17.760 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:30:17.761 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:17.761 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:30:17.761 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.850800+0000", 2024-09-14T06:30:17.761 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:17.761 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:17.761 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:17.762 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:30:17.762 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.850800+0000", 2024-09-14T06:30:17.762 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:17.762 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:17.762 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:17.762 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:30:17.763 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.850801+0000", 2024-09-14T06:30:17.763 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.5600000000000005e-07 2024-09-14T06:30:17.763 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:17.763 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:17.763 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:30:17.763 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.850838+0000", 2024-09-14T06:30:17.763 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.7481999999999998e-05 2024-09-14T06:30:17.763 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:17.763 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:17.763 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:30:17.764 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.850847+0000", 2024-09-14T06:30:17.764 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.4910000000000005e-06 2024-09-14T06:30:17.764 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:17.764 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:17.764 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:30:17.764 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.850856+0000", 2024-09-14T06:30:17.765 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.6309999999999998e-06 2024-09-14T06:30:17.765 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:17.765 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:17.765 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:30:17.765 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.919498+0000", 2024-09-14T06:30:17.765 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.068641743000000005 2024-09-14T06:30:17.765 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:17.765 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:17.765 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:30:17.765 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.919538+0000", 2024-09-14T06:30:17.765 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.9706999999999998e-05 2024-09-14T06:30:17.766 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:17.766 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:17.766 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:30:17.766 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.919653+0000", 2024-09-14T06:30:17.766 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000115092 2024-09-14T06:30:17.766 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:17.766 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:17.766 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:17.766 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:17.766 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:17.766 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:44358 3.17 3:ea948ecb:::rbd_data.112dca2f6ae1.0000000000000004:17 [read 270336~4096 [fadvise_nocache]] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:30:17.767 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:43.850865+0000", 2024-09-14T06:30:17.767 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 33.891620865999997, 2024-09-14T06:30:17.767 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.068983668999999997, 2024-09-14T06:30:17.767 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:30:17.767 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:30:17.767 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:30:17.767 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:30:17.767 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:30:17.767 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:17.767 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 44358 2024-09-14T06:30:17.767 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:17.768 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:30:17.768 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:17.768 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:30:17.768 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.850865+0000", 2024-09-14T06:30:17.768 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:17.768 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:17.768 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:17.768 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:30:17.768 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.850865+0000", 2024-09-14T06:30:17.768 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:17.768 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:17.768 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:17.768 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:30:17.768 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.850865+0000", 2024-09-14T06:30:17.769 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.3000000000000001e-07 2024-09-14T06:30:17.769 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:17.769 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:17.769 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:30:17.769 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.850868+0000", 2024-09-14T06:30:17.769 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.3099999999999999e-06 2024-09-14T06:30:17.769 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:17.769 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:17.769 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:30:17.769 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.850874+0000", 2024-09-14T06:30:17.769 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.3999999999999997e-06 2024-09-14T06:30:17.769 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:17.769 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:17.769 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:30:17.770 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.850885+0000", 2024-09-14T06:30:17.770 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.0553999999999999e-05 2024-09-14T06:30:17.770 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:17.770 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:17.770 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:30:17.770 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.919670+0000", 2024-09-14T06:30:17.770 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.068785464000000004 2024-09-14T06:30:17.770 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:17.770 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:17.770 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:30:17.770 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.919713+0000", 2024-09-14T06:30:17.770 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.3028999999999997e-05 2024-09-14T06:30:17.770 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:17.771 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:17.771 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:30:17.771 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.919849+0000", 2024-09-14T06:30:17.771 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000135382 2024-09-14T06:30:17.771 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:17.771 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:17.771 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:17.771 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:17.771 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:17.771 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:44359 3.17 3:ea948ecb:::rbd_data.112dca2f6ae1.0000000000000004:17 [read 294912~4096 [fadvise_nocache]] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:30:17.771 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:43.850892+0000", 2024-09-14T06:30:17.771 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 33.891593843999999, 2024-09-14T06:30:17.771 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.069282708999999998, 2024-09-14T06:30:17.772 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:30:17.772 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:30:17.772 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:30:17.772 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:30:17.772 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:30:17.772 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:17.772 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 44359 2024-09-14T06:30:17.772 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:17.772 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:30:17.772 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:17.772 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:30:17.772 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.850892+0000", 2024-09-14T06:30:17.772 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:17.772 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:17.773 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:17.773 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:30:17.773 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.850892+0000", 2024-09-14T06:30:17.773 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:17.773 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:17.773 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:17.773 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:30:17.773 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.850892+0000", 2024-09-14T06:30:17.773 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.3599999999999999e-07 2024-09-14T06:30:17.773 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:17.773 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:17.773 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:30:17.774 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.850895+0000", 2024-09-14T06:30:17.774 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.2050000000000001e-06 2024-09-14T06:30:17.774 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:17.774 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:17.774 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:30:17.774 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.850902+0000", 2024-09-14T06:30:17.774 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.4460000000000004e-06 2024-09-14T06:30:17.774 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:17.774 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:17.774 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:30:17.774 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.850907+0000", 2024-09-14T06:30:17.774 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.899e-06 2024-09-14T06:30:17.774 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:17.774 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:17.775 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:30:17.775 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.919896+0000", 2024-09-14T06:30:17.775 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.068988840999999995 2024-09-14T06:30:17.775 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:17.775 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:17.775 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:30:17.775 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.919968+0000", 2024-09-14T06:30:17.775 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.2454999999999997e-05 2024-09-14T06:30:17.775 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:17.775 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:17.775 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:30:17.775 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.920175+0000", 2024-09-14T06:30:17.775 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00020642699999999999 2024-09-14T06:30:17.776 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:17.776 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:17.776 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:17.776 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:17.776 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:17.776 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:44360 3.17 3:ea948ecb:::rbd_data.112dca2f6ae1.0000000000000004:17 [read 319488~4096 [fadvise_nocache]] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:30:17.776 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:43.850915+0000", 2024-09-14T06:30:17.776 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 33.891570537, 2024-09-14T06:30:17.776 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.069420047999999998, 2024-09-14T06:30:17.776 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:30:17.776 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:30:17.777 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:30:17.777 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:30:17.777 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:30:17.777 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:17.777 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 44360 2024-09-14T06:30:17.777 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:17.777 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:30:17.777 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:17.777 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:30:17.777 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.850915+0000", 2024-09-14T06:30:17.777 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:17.777 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:17.777 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:17.778 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:30:17.778 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.850915+0000", 2024-09-14T06:30:17.778 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:17.778 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:17.778 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:17.778 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:30:17.778 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.850916+0000", 2024-09-14T06:30:17.778 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.3500000000000002e-07 2024-09-14T06:30:17.778 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:17.778 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:17.778 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:30:17.778 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.850918+0000", 2024-09-14T06:30:17.778 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.0990000000000001e-06 2024-09-14T06:30:17.779 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:17.779 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:17.779 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:30:17.779 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.850925+0000", 2024-09-14T06:30:17.779 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.08e-06 2024-09-14T06:30:17.779 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:17.779 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:17.779 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:30:17.779 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.850930+0000", 2024-09-14T06:30:17.779 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.8069999999999999e-06 2024-09-14T06:30:17.779 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:17.779 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:17.780 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:30:17.780 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.920191+0000", 2024-09-14T06:30:17.780 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.069261113999999999 2024-09-14T06:30:17.780 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:17.780 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:17.780 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:30:17.780 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.920230+0000", 2024-09-14T06:30:17.780 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.8940999999999999e-05 2024-09-14T06:30:17.780 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:17.780 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:17.780 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:30:17.780 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.920335+0000", 2024-09-14T06:30:17.780 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000105572 2024-09-14T06:30:17.781 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:17.781 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:17.781 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:17.781 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:17.781 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:17.781 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:44361 3.17 3:ea948ecb:::rbd_data.112dca2f6ae1.0000000000000004:17 [read 344064~4096 [fadvise_nocache]] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:30:17.781 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:43.850936+0000", 2024-09-14T06:30:17.781 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 33.891549533999999, 2024-09-14T06:30:17.781 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.069568603000000007, 2024-09-14T06:30:17.781 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:30:17.781 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:30:17.781 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:30:17.782 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:30:17.782 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:30:17.782 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:17.782 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 44361 2024-09-14T06:30:17.782 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:17.782 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:30:17.782 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:17.782 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:30:17.782 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.850936+0000", 2024-09-14T06:30:17.782 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:17.782 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:17.782 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:17.782 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:30:17.782 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.850936+0000", 2024-09-14T06:30:17.783 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:17.783 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:17.783 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:17.783 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:30:17.783 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.851117+0000", 2024-09-14T06:30:17.783 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00018043700000000001 2024-09-14T06:30:17.783 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:17.783 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:17.783 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:30:17.784 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.851123+0000", 2024-09-14T06:30:17.784 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.8179999999999998e-06 2024-09-14T06:30:17.784 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:17.784 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:17.784 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:30:17.784 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.851131+0000", 2024-09-14T06:30:17.784 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.7689999999999996e-06 2024-09-14T06:30:17.784 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:17.784 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:17.784 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:30:17.784 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.851139+0000", 2024-09-14T06:30:17.784 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.3390000000000004e-06 2024-09-14T06:30:17.784 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:17.785 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:17.785 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:30:17.785 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.920353+0000", 2024-09-14T06:30:17.785 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.069214576999999999 2024-09-14T06:30:17.785 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:17.785 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:17.785 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:30:17.785 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.920391+0000", 2024-09-14T06:30:17.785 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.7883999999999998e-05 2024-09-14T06:30:17.785 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:17.785 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:17.785 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:30:17.785 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.920505+0000", 2024-09-14T06:30:17.786 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000113779 2024-09-14T06:30:17.786 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:17.786 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:17.786 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:17.786 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:17.786 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:17.786 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:44362 3.17 3:ea948ecb:::rbd_data.112dca2f6ae1.0000000000000004:17 [read 368640~4096 [fadvise_nocache]] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:30:17.786 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:43.851149+0000", 2024-09-14T06:30:17.786 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 33.891336754999998, 2024-09-14T06:30:17.786 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.069537131000000002, 2024-09-14T06:30:17.786 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:30:17.786 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:30:17.787 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:30:17.787 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:30:17.787 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:30:17.787 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:17.787 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 44362 2024-09-14T06:30:17.787 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:17.787 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:30:17.787 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:17.787 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:30:17.787 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.851149+0000", 2024-09-14T06:30:17.787 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:17.787 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:17.787 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:17.787 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:30:17.788 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.851149+0000", 2024-09-14T06:30:17.788 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:17.788 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:17.788 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:17.788 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:30:17.788 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.857003+0000", 2024-09-14T06:30:17.788 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0058535360000000003 2024-09-14T06:30:17.788 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:17.788 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:17.788 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:30:17.788 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.857009+0000", 2024-09-14T06:30:17.788 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.579e-06 2024-09-14T06:30:17.788 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:17.788 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:17.788 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:30:17.789 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.857019+0000", 2024-09-14T06:30:17.789 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.8330000000000006e-06 2024-09-14T06:30:17.789 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:17.789 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:17.789 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:30:17.789 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.857025+0000", 2024-09-14T06:30:17.789 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.8930000000000002e-06 2024-09-14T06:30:17.789 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:17.789 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:17.789 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:30:17.789 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.920522+0000", 2024-09-14T06:30:17.790 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.063496737999999997 2024-09-14T06:30:17.790 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:17.790 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:17.790 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:30:17.790 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.920567+0000", 2024-09-14T06:30:17.790 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.4956999999999997e-05 2024-09-14T06:30:17.790 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:17.790 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:17.790 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:30:17.790 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.920686+0000", 2024-09-14T06:30:17.790 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000119595 2024-09-14T06:30:17.790 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:17.790 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:17.790 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:17.791 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:17.791 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:17.791 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:44562 3.17 3:ea6a820a:::rbd_data.112dca2f6ae1.0000000000000003:17 [read 45056~4096 [fadvise_nocache]] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:30:17.791 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:43.857630+0000", 2024-09-14T06:30:17.791 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 33.884855997000003, 2024-09-14T06:30:17.791 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.068561975999999997, 2024-09-14T06:30:17.791 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:30:17.791 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:30:17.791 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:30:17.791 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:30:17.791 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:30:17.791 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:17.791 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 44562 2024-09-14T06:30:17.791 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:17.791 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:30:17.792 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:17.792 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:30:17.792 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.857630+0000", 2024-09-14T06:30:17.792 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:17.792 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:17.792 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:17.792 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:30:17.792 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.857630+0000", 2024-09-14T06:30:17.792 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:17.792 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:17.792 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:17.792 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:30:17.792 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.862817+0000", 2024-09-14T06:30:17.793 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0051874699999999996 2024-09-14T06:30:17.793 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:17.793 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:17.793 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:30:17.793 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.862823+0000", 2024-09-14T06:30:17.793 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.1299999999999998e-06 2024-09-14T06:30:17.793 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:17.793 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:17.793 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:30:17.793 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.862836+0000", 2024-09-14T06:30:17.793 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.3071000000000001e-05 2024-09-14T06:30:17.793 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:17.793 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:17.794 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:30:17.794 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.862845+0000", 2024-09-14T06:30:17.794 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.4630000000000001e-06 2024-09-14T06:30:17.794 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:17.794 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:17.794 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:30:17.794 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.925934+0000", 2024-09-14T06:30:17.794 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.063088985 2024-09-14T06:30:17.794 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:17.794 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:17.794 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:30:17.795 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.926013+0000", 2024-09-14T06:30:17.795 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.9507999999999998e-05 2024-09-14T06:30:17.795 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:17.795 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:17.795 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:30:17.795 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.926192+0000", 2024-09-14T06:30:17.795 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00017834900000000001 2024-09-14T06:30:17.795 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:17.795 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:17.795 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:17.795 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:17.795 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:17.796 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:44600 3.17 3:ea6a820a:::rbd_data.112dca2f6ae1.0000000000000003:17 [read 978944~4096 [fadvise_nocache]] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:30:17.796 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:43.869417+0000", 2024-09-14T06:30:17.796 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 33.873068308000001, 2024-09-14T06:30:17.796 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.068698802000000003, 2024-09-14T06:30:17.796 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:30:17.796 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:30:17.796 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:30:17.796 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:30:17.796 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:30:17.796 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:17.796 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 44600 2024-09-14T06:30:17.796 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:17.797 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:30:17.797 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:17.797 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:30:17.797 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.869417+0000", 2024-09-14T06:30:17.797 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:17.797 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:17.797 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:17.797 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:30:17.797 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.869417+0000", 2024-09-14T06:30:17.797 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:17.797 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:17.798 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:17.798 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:30:17.798 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.874912+0000", 2024-09-14T06:30:17.798 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0054948970000000003 2024-09-14T06:30:17.798 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:17.798 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:17.798 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:30:17.798 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.874917+0000", 2024-09-14T06:30:17.798 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.1259999999999997e-06 2024-09-14T06:30:17.799 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:17.799 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:17.799 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:30:17.799 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.874926+0000", 2024-09-14T06:30:17.799 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.7490000000000004e-06 2024-09-14T06:30:17.799 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:17.799 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:17.799 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:30:17.799 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.874939+0000", 2024-09-14T06:30:17.799 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.2831e-05 2024-09-14T06:30:17.799 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:17.799 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:17.799 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:30:17.799 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.937797+0000", 2024-09-14T06:30:17.800 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.062858019000000001 2024-09-14T06:30:17.800 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:17.800 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:17.800 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:30:17.800 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.937879+0000", 2024-09-14T06:30:17.800 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.1932000000000001e-05 2024-09-14T06:30:17.800 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:17.800 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:17.800 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:30:17.800 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.938116+0000", 2024-09-14T06:30:17.800 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00023724799999999999 2024-09-14T06:30:17.800 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:17.800 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:17.801 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:17.801 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:17.801 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:17.801 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:44617 3.17 3:ea948ecb:::rbd_data.112dca2f6ae1.0000000000000004:17 [read 348160~4096 [fadvise_nocache]] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:30:17.801 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:43.875196+0000", 2024-09-14T06:30:17.801 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 33.867289311, 2024-09-14T06:30:17.801 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.068462323000000005, 2024-09-14T06:30:17.801 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:30:17.801 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:30:17.801 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:30:17.801 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:30:17.801 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:30:17.801 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:17.802 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 44617 2024-09-14T06:30:17.802 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:17.802 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:30:17.802 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:17.802 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:30:17.802 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.875196+0000", 2024-09-14T06:30:17.802 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:17.802 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:17.802 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:17.802 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:30:17.802 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.875196+0000", 2024-09-14T06:30:17.802 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:17.802 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:17.802 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:17.802 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:30:17.803 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.875197+0000", 2024-09-14T06:30:17.803 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.7100000000000002e-07 2024-09-14T06:30:17.803 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:17.803 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:17.803 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:30:17.803 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.875199+0000", 2024-09-14T06:30:17.803 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.4609999999999998e-06 2024-09-14T06:30:17.803 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:17.803 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:17.803 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:30:17.803 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.875214+0000", 2024-09-14T06:30:17.803 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.4613000000000001e-05 2024-09-14T06:30:17.803 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:17.804 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:17.804 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:30:17.804 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.875221+0000", 2024-09-14T06:30:17.804 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.4309999999999997e-06 2024-09-14T06:30:17.804 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:17.804 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:17.804 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:30:17.804 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.943357+0000", 2024-09-14T06:30:17.804 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.068135412000000006 2024-09-14T06:30:17.804 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:17.804 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:17.804 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:30:17.805 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.943428+0000", 2024-09-14T06:30:17.805 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.0735000000000006e-05 2024-09-14T06:30:17.805 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:17.805 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:17.805 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:30:17.805 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.943659+0000", 2024-09-14T06:30:17.805 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00023120000000000001 2024-09-14T06:30:17.805 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:17.805 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:17.805 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:17.805 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:17.805 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:17.805 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:44618 3.17 3:ea948ecb:::rbd_data.112dca2f6ae1.0000000000000004:17 [read 372736~4096 [fadvise_nocache]] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:30:17.806 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:43.875236+0000", 2024-09-14T06:30:17.806 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 33.867249371, 2024-09-14T06:30:17.806 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.068766076999999995, 2024-09-14T06:30:17.806 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:30:17.806 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:30:17.806 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:30:17.806 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:30:17.806 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:30:17.806 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:17.806 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 44618 2024-09-14T06:30:17.806 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:17.806 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:30:17.806 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:17.806 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:30:17.807 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.875236+0000", 2024-09-14T06:30:17.807 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:17.807 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:17.807 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:17.807 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:30:17.807 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.875236+0000", 2024-09-14T06:30:17.807 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:17.807 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:17.807 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:17.807 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:30:17.807 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.875237+0000", 2024-09-14T06:30:17.807 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.8999999999999996e-07 2024-09-14T06:30:17.808 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:17.808 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:17.808 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:30:17.808 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.875239+0000", 2024-09-14T06:30:17.808 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.419e-06 2024-09-14T06:30:17.808 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:17.808 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:17.808 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:30:17.808 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.875245+0000", 2024-09-14T06:30:17.808 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.9909999999999999e-06 2024-09-14T06:30:17.808 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:17.808 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:17.808 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:30:17.809 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.875256+0000", 2024-09-14T06:30:17.809 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.0399e-05 2024-09-14T06:30:17.809 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:17.809 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:17.809 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:30:17.809 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.943690+0000", 2024-09-14T06:30:17.809 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.068434637000000006 2024-09-14T06:30:17.809 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:17.809 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:17.809 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:30:17.809 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.943765+0000", 2024-09-14T06:30:17.809 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.4418000000000005e-05 2024-09-14T06:30:17.809 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:17.809 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:17.810 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:30:17.810 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.944002+0000", 2024-09-14T06:30:17.810 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00023762300000000001 2024-09-14T06:30:17.810 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:17.810 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:17.810 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:17.810 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:17.810 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:17.810 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:44619 3.17 3:ea948ecb:::rbd_data.112dca2f6ae1.0000000000000004:17 [read 397312~4096 [fadvise_nocache]] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:30:17.810 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:43.875263+0000", 2024-09-14T06:30:17.810 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 33.867223103999997, 2024-09-14T06:30:17.811 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.069074144000000004, 2024-09-14T06:30:17.811 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:30:17.811 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:30:17.811 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:30:17.811 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:30:17.811 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:30:17.811 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:17.811 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 44619 2024-09-14T06:30:17.811 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:17.811 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:30:17.811 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:17.811 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:30:17.811 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.875263+0000", 2024-09-14T06:30:17.812 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:17.812 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:17.812 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:17.812 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:30:17.812 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.875263+0000", 2024-09-14T06:30:17.812 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:17.812 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:17.812 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:17.812 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:30:17.812 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.875263+0000", 2024-09-14T06:30:17.812 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.2199999999999999e-07 2024-09-14T06:30:17.812 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:17.812 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:17.813 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:30:17.813 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.875265+0000", 2024-09-14T06:30:17.813 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.1270000000000001e-06 2024-09-14T06:30:17.813 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:17.813 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:17.813 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:30:17.813 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.875271+0000", 2024-09-14T06:30:17.813 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.3329999999999996e-06 2024-09-14T06:30:17.813 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:17.813 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:17.813 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:30:17.813 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.875284+0000", 2024-09-14T06:30:17.813 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.3036e-05 2024-09-14T06:30:17.813 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:17.814 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:17.814 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:30:17.814 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.944032+0000", 2024-09-14T06:30:17.814 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.068748751999999996 2024-09-14T06:30:17.814 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:17.814 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:17.814 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:30:17.814 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.944106+0000", 2024-09-14T06:30:17.814 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.3441000000000004e-05 2024-09-14T06:30:17.814 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:17.815 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:17.815 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:30:17.815 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.944337+0000", 2024-09-14T06:30:17.815 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00023103300000000001 2024-09-14T06:30:17.815 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:17.815 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:17.815 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:17.815 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:17.815 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:17.815 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:44620 3.17 3:ea948ecb:::rbd_data.112dca2f6ae1.0000000000000004:17 [read 421888~4096 [fadvise_nocache]] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:30:17.815 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:43.875291+0000", 2024-09-14T06:30:17.815 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 33.867195219999999, 2024-09-14T06:30:17.815 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.069988984000000004, 2024-09-14T06:30:17.816 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:30:17.816 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:30:17.816 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:30:17.816 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:30:17.816 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:30:17.816 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:17.816 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 44620 2024-09-14T06:30:17.816 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:17.816 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:30:17.816 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:17.816 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:30:17.816 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.875291+0000", 2024-09-14T06:30:17.816 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:17.817 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:17.817 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:17.817 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:30:17.817 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.875291+0000", 2024-09-14T06:30:17.817 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:17.817 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:17.817 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:17.817 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:30:17.818 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.880818+0000", 2024-09-14T06:30:17.818 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0055275350000000001 2024-09-14T06:30:17.818 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:17.818 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:17.818 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:30:17.818 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.880824+0000", 2024-09-14T06:30:17.818 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.4240000000000001e-06 2024-09-14T06:30:17.818 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:17.818 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:17.818 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:30:17.819 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.880834+0000", 2024-09-14T06:30:17.819 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.0096e-05 2024-09-14T06:30:17.819 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:17.819 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:17.819 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:30:17.819 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.880842+0000", 2024-09-14T06:30:17.819 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.6349999999999995e-06 2024-09-14T06:30:17.819 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:17.819 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:17.819 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:30:17.820 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.944380+0000", 2024-09-14T06:30:17.820 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.063537934000000004 2024-09-14T06:30:17.820 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:17.820 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:17.820 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:30:17.820 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.944459+0000", 2024-09-14T06:30:17.820 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.8867000000000002e-05 2024-09-14T06:30:17.820 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:17.820 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:17.820 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:30:17.821 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.945280+0000", 2024-09-14T06:30:17.821 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00082049300000000005 2024-09-14T06:30:17.821 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:17.821 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:17.821 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:17.821 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:17.821 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:17.821 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:44644 3.17 3:ea948ecb:::rbd_data.112dca2f6ae1.0000000000000004:17 [read 1011712~4096 [fadvise_nocache]] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:30:17.821 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:43.881869+0000", 2024-09-14T06:30:17.821 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 33.860617265999998, 2024-09-14T06:30:17.821 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.068512922000000004, 2024-09-14T06:30:17.821 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:30:17.821 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:30:17.822 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:30:17.822 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:30:17.822 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:30:17.822 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:17.822 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 44644 2024-09-14T06:30:17.822 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:17.822 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:30:17.822 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:17.822 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:30:17.822 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.881869+0000", 2024-09-14T06:30:17.822 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:17.822 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:17.822 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:17.822 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:30:17.822 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.881869+0000", 2024-09-14T06:30:17.823 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:17.823 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:17.823 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:17.823 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:30:17.823 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.881869+0000", 2024-09-14T06:30:17.823 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.9299999999999998e-07 2024-09-14T06:30:17.823 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:17.823 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:17.823 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:30:17.823 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.881871+0000", 2024-09-14T06:30:17.823 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.362e-06 2024-09-14T06:30:17.823 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:17.823 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:17.823 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:30:17.824 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.881878+0000", 2024-09-14T06:30:17.824 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.6239999999999996e-06 2024-09-14T06:30:17.824 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:17.824 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:17.824 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:30:17.824 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.881885+0000", 2024-09-14T06:30:17.824 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.2649999999999999e-06 2024-09-14T06:30:17.824 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:17.824 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:17.824 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:30:17.824 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.950183+0000", 2024-09-14T06:30:17.824 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.068298225000000004 2024-09-14T06:30:17.825 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:17.825 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:17.825 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:30:17.825 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.950270+0000", 2024-09-14T06:30:17.825 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.6331999999999999e-05 2024-09-14T06:30:17.825 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:17.825 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:17.825 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:30:17.825 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.950381+0000", 2024-09-14T06:30:17.825 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00011162099999999999 2024-09-14T06:30:17.825 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:17.825 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:17.825 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:17.825 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:17.826 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:17.826 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:44645 3.17 3:ea948ecb:::rbd_data.112dca2f6ae1.0000000000000004:17 [read 1036288~4096 [fadvise_nocache]] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:30:17.826 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:43.881893+0000", 2024-09-14T06:30:17.826 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 33.860592939999997, 2024-09-14T06:30:17.826 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.068700357000000004, 2024-09-14T06:30:17.826 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:30:17.826 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:30:17.826 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:30:17.826 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:30:17.826 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:30:17.826 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:17.827 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 44645 2024-09-14T06:30:17.827 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:17.827 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:30:17.827 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:17.827 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:30:17.827 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.881893+0000", 2024-09-14T06:30:17.827 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:17.827 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:17.827 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:17.827 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:30:17.828 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.881893+0000", 2024-09-14T06:30:17.828 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:17.828 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:17.828 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:17.828 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:30:17.828 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.881893+0000", 2024-09-14T06:30:17.828 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.8299999999999998e-07 2024-09-14T06:30:17.828 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:17.828 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:17.828 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:30:17.828 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.881895+0000", 2024-09-14T06:30:17.828 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.88e-06 2024-09-14T06:30:17.829 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:17.829 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:17.829 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:30:17.829 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.881902+0000", 2024-09-14T06:30:17.829 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.6710000000000002e-06 2024-09-14T06:30:17.829 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:17.829 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:17.829 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:30:17.829 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.881915+0000", 2024-09-14T06:30:17.829 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.312e-05 2024-09-14T06:30:17.830 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:17.830 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:17.830 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:30:17.830 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.950420+0000", 2024-09-14T06:30:17.830 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.068505359000000002 2024-09-14T06:30:17.830 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:17.830 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:17.830 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:30:17.830 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.950497+0000", 2024-09-14T06:30:17.830 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.6454999999999999e-05 2024-09-14T06:30:17.830 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:17.830 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:17.830 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:30:17.830 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.950593+0000", 2024-09-14T06:30:17.831 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.6489000000000004e-05 2024-09-14T06:30:17.831 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:17.831 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:17.831 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:17.831 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:17.831 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:17.831 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:44777 3.17 3:ee6e7e6f:::rbd_data.112dca2f6ae1.0000000000000007:17 [read 8192~4096 [fadvise_nocache]] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:30:17.831 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:43.881935+0000", 2024-09-14T06:30:17.831 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 33.860550742999997, 2024-09-14T06:30:17.831 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.068908843999999997, 2024-09-14T06:30:17.831 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:30:17.831 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:30:17.832 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:30:17.832 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:30:17.832 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:30:17.832 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:17.832 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 44777 2024-09-14T06:30:17.832 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:17.832 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:30:17.832 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:17.832 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:30:17.833 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.881935+0000", 2024-09-14T06:30:17.833 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:17.833 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:17.833 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:17.833 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:30:17.833 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.881935+0000", 2024-09-14T06:30:17.833 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:17.833 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:17.833 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:17.833 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:30:17.833 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.881936+0000", 2024-09-14T06:30:17.833 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.7100000000000002e-07 2024-09-14T06:30:17.833 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:17.833 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:17.834 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:30:17.834 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.881938+0000", 2024-09-14T06:30:17.834 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.6010000000000002e-06 2024-09-14T06:30:17.834 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:17.834 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:17.834 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:30:17.834 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.881946+0000", 2024-09-14T06:30:17.834 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.9179999999999994e-06 2024-09-14T06:30:17.834 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:17.834 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:17.834 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:30:17.834 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.881951+0000", 2024-09-14T06:30:17.834 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.3199999999999999e-06 2024-09-14T06:30:17.835 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:17.835 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:17.835 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:30:17.835 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.950638+0000", 2024-09-14T06:30:17.835 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.068686149000000002 2024-09-14T06:30:17.835 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:17.835 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:17.835 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:30:17.835 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.950725+0000", 2024-09-14T06:30:17.835 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.7535999999999997e-05 2024-09-14T06:30:17.835 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:17.835 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:17.835 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:30:17.835 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.950844+0000", 2024-09-14T06:30:17.835 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000118749 2024-09-14T06:30:17.836 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:17.836 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:17.836 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:17.836 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:17.836 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:17.836 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:44778 3.17 3:ee6e7e6f:::rbd_data.112dca2f6ae1.0000000000000007:17 [read 32768~4096 [fadvise_nocache]] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:30:17.836 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:43.881960+0000", 2024-09-14T06:30:17.836 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 33.860526096999997, 2024-09-14T06:30:17.836 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.069098804999999999, 2024-09-14T06:30:17.836 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:30:17.837 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:30:17.837 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:30:17.837 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:30:17.837 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:30:17.837 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:17.837 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 44778 2024-09-14T06:30:17.837 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:17.837 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:30:17.837 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:17.837 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:30:17.837 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.881960+0000", 2024-09-14T06:30:17.837 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:17.838 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:17.838 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:17.838 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:30:17.838 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.881960+0000", 2024-09-14T06:30:17.838 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:17.838 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:17.838 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:17.838 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:30:17.838 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.881960+0000", 2024-09-14T06:30:17.838 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.6199999999999998e-07 2024-09-14T06:30:17.838 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:17.839 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:17.839 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:30:17.839 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.881963+0000", 2024-09-14T06:30:17.839 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.2740000000000002e-06 2024-09-14T06:30:17.839 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:17.839 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:17.839 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:30:17.839 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.881971+0000", 2024-09-14T06:30:17.839 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.1750000000000005e-06 2024-09-14T06:30:17.839 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:17.839 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:17.839 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:30:17.839 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.881977+0000", 2024-09-14T06:30:17.839 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.5030000000000002e-06 2024-09-14T06:30:17.840 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:17.840 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:17.840 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:30:17.840 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.950884+0000", 2024-09-14T06:30:17.840 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.068907066000000003 2024-09-14T06:30:17.840 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:17.840 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:17.840 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:30:17.840 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.950963+0000", 2024-09-14T06:30:17.840 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.8789999999999996e-05 2024-09-14T06:30:17.840 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:17.840 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:17.840 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:30:17.841 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.951058+0000", 2024-09-14T06:30:17.841 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.5434999999999997e-05 2024-09-14T06:30:17.841 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:17.841 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:17.841 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:17.841 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:17.841 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:17.841 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:44779 3.17 3:ee6e7e6f:::rbd_data.112dca2f6ae1.0000000000000007:17 [read 57344~4096 [fadvise_nocache]] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:30:17.841 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:43.881984+0000", 2024-09-14T06:30:17.841 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 33.860501503000002, 2024-09-14T06:30:17.841 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.069297828000000006, 2024-09-14T06:30:17.841 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:30:17.841 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:30:17.842 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:30:17.842 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:30:17.842 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:30:17.842 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:17.842 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 44779 2024-09-14T06:30:17.842 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:17.842 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:30:17.842 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:17.842 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:30:17.842 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.881984+0000", 2024-09-14T06:30:17.843 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:17.843 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:17.843 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:17.843 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:30:17.843 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.881984+0000", 2024-09-14T06:30:17.843 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:17.843 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:17.843 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:17.843 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:30:17.843 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.881985+0000", 2024-09-14T06:30:17.843 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.7099999999999997e-07 2024-09-14T06:30:17.843 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:17.843 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:17.843 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:30:17.844 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.881987+0000", 2024-09-14T06:30:17.844 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.0449999999999999e-06 2024-09-14T06:30:17.844 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:17.844 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:17.844 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:30:17.844 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.881994+0000", 2024-09-14T06:30:17.844 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.3849999999999996e-06 2024-09-14T06:30:17.844 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:17.844 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:17.844 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:30:17.844 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.881999+0000", 2024-09-14T06:30:17.845 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.0050000000000004e-06 2024-09-14T06:30:17.845 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:17.845 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:17.845 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:30:17.845 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.951101+0000", 2024-09-14T06:30:17.845 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.069101878000000005 2024-09-14T06:30:17.845 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:17.845 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:17.846 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:30:17.846 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.951178+0000", 2024-09-14T06:30:17.846 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.6823999999999995e-05 2024-09-14T06:30:17.846 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:17.846 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:17.846 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:30:17.846 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.951282+0000", 2024-09-14T06:30:17.846 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00010432 2024-09-14T06:30:17.846 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:17.846 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:17.846 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:17.847 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:17.847 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:17.847 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:44780 3.17 3:ee6e7e6f:::rbd_data.112dca2f6ae1.0000000000000007:17 [read 81920~4096 [fadvise_nocache]] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:30:17.847 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:43.882006+0000", 2024-09-14T06:30:17.847 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 33.860480193000001, 2024-09-14T06:30:17.847 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.069494627000000003, 2024-09-14T06:30:17.847 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:30:17.847 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:30:17.847 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:30:17.847 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:30:17.847 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:30:17.847 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:17.847 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 44780 2024-09-14T06:30:17.848 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:17.848 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:30:17.848 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:17.848 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:30:17.848 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.882006+0000", 2024-09-14T06:30:17.848 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:17.848 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:17.848 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:17.848 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:30:17.848 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.882006+0000", 2024-09-14T06:30:17.848 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:17.849 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:17.849 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:17.849 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:30:17.849 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.887819+0000", 2024-09-14T06:30:17.849 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0058133489999999998 2024-09-14T06:30:17.849 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:17.849 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:17.849 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:30:17.849 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.887826+0000", 2024-09-14T06:30:17.849 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.3889999999999999e-06 2024-09-14T06:30:17.849 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:17.849 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:17.849 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:30:17.849 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.887847+0000", 2024-09-14T06:30:17.850 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.0984000000000001e-05 2024-09-14T06:30:17.850 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:17.850 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:17.850 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:30:17.850 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.887857+0000", 2024-09-14T06:30:17.850 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.645e-06 2024-09-14T06:30:17.850 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:17.850 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:17.850 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:30:17.850 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.951325+0000", 2024-09-14T06:30:17.850 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.063468399999999994 2024-09-14T06:30:17.850 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:17.850 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:17.850 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:30:17.851 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.951401+0000", 2024-09-14T06:30:17.851 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.5499000000000003e-05 2024-09-14T06:30:17.851 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:17.851 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:17.851 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:30:17.851 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.951500+0000", 2024-09-14T06:30:17.851 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.9361e-05 2024-09-14T06:30:17.851 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:17.851 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:17.851 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:17.851 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:17.851 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:17.851 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-14T06:30:17.853 INFO:tasks.ceph.osd.5.smithi138.stderr:2024-09-14T06:30:17.851+0000 7fe13746f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33618) UID: 0 2024-09-14T06:30:17.853 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-14T06:30:17.954 INFO:tasks.ceph.osd.4.smithi138.stderr:2024-09-14T06:30:17.953+0000 7f6c653c8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33615) UID: 0 2024-09-14T06:30:17.984 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-14T06:30:17.984 INFO:teuthology.orchestra.run.smithi094.stdout: "ops": [], 2024-09-14T06:30:17.984 INFO:teuthology.orchestra.run.smithi094.stdout: "num_ops": 0 2024-09-14T06:30:17.984 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-14T06:30:17.993 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-14T06:30:18.055 INFO:tasks.ceph.osd.7.smithi138.stderr:2024-09-14T06:30:18.054+0000 7f3c6a21d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33623) UID: 0 2024-09-14T06:30:18.125 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-14T06:30:18.125 INFO:teuthology.orchestra.run.smithi094.stdout: "ops": [], 2024-09-14T06:30:18.125 INFO:teuthology.orchestra.run.smithi094.stdout: "complaint_time": 30, 2024-09-14T06:30:18.125 INFO:teuthology.orchestra.run.smithi094.stdout: "num_blocked_ops": 0 2024-09-14T06:30:18.125 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-14T06:30:18.135 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-14T06:30:18.179 INFO:tasks.ceph.osd.0.smithi094.stderr:2024-09-14T06:30:18.178+0000 7f07c9333640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33636) UID: 0 2024-09-14T06:30:18.257 INFO:tasks.ceph.osd.7.smithi138.stderr:2024-09-14T06:30:18.256+0000 7f3c6a21d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33623) UID: 0 2024-09-14T06:30:18.278 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-14T06:30:18.278 INFO:teuthology.orchestra.run.smithi094.stdout: "size": 20, 2024-09-14T06:30:18.278 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 600, 2024-09-14T06:30:18.278 INFO:teuthology.orchestra.run.smithi094.stdout: "ops": [ 2024-09-14T06:30:18.278 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:18.278 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:14918 3.0 3:05c39535:::rbd_data.112d92027590.0000000000000009:f [read 795648~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:30:18.278 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:26.083834+0000", 2024-09-14T06:30:18.279 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 52.192034337000003, 2024-09-14T06:30:18.279 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.041776287000000002, 2024-09-14T06:30:18.279 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:30:18.279 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:30:18.279 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:30:18.279 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:30:18.279 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:30:18.279 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:18.279 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 14918 2024-09-14T06:30:18.279 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:18.279 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:30:18.279 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:18.279 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:30:18.280 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.083834+0000", 2024-09-14T06:30:18.280 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:18.280 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:18.280 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:18.280 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:30:18.280 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.083834+0000", 2024-09-14T06:30:18.280 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:18.280 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:18.280 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:18.280 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:30:18.280 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.083834+0000", 2024-09-14T06:30:18.280 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.0100000000000001e-07 2024-09-14T06:30:18.280 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:18.281 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:18.281 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:30:18.281 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.083835+0000", 2024-09-14T06:30:18.281 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.3900000000000003e-07 2024-09-14T06:30:18.281 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:18.281 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:18.281 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:30:18.281 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.083838+0000", 2024-09-14T06:30:18.281 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.9830000000000001e-06 2024-09-14T06:30:18.281 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:18.281 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:18.281 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:30:18.281 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.083840+0000", 2024-09-14T06:30:18.282 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.4679999999999999e-06 2024-09-14T06:30:18.282 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:18.282 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:18.282 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:30:18.282 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.125455+0000", 2024-09-14T06:30:18.282 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.041614578999999999 2024-09-14T06:30:18.282 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:18.282 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:18.282 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:30:18.282 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.125496+0000", 2024-09-14T06:30:18.282 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.1047000000000002e-05 2024-09-14T06:30:18.282 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:18.282 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:18.283 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:30:18.283 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.125610+0000", 2024-09-14T06:30:18.283 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00011407 2024-09-14T06:30:18.283 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:18.283 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:18.283 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:18.283 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:18.283 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:18.283 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:14919 3.0 3:05c39535:::rbd_data.112d92027590.0000000000000009:f [read 805888~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:30:18.283 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:26.083843+0000", 2024-09-14T06:30:18.283 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 52.192024588999999, 2024-09-14T06:30:18.283 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.041992356000000002, 2024-09-14T06:30:18.283 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:30:18.284 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:30:18.284 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:30:18.284 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:30:18.284 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:30:18.284 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:18.284 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 14919 2024-09-14T06:30:18.284 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:18.284 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:30:18.284 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:18.284 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:30:18.284 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.083843+0000", 2024-09-14T06:30:18.284 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:18.284 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:18.285 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:18.285 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:30:18.285 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.083843+0000", 2024-09-14T06:30:18.285 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:18.285 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:18.285 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:18.285 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:30:18.285 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.083844+0000", 2024-09-14T06:30:18.285 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.3300000000000001e-07 2024-09-14T06:30:18.285 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:18.285 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:18.285 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:30:18.285 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.083845+0000", 2024-09-14T06:30:18.286 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.4729999999999999e-06 2024-09-14T06:30:18.286 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:18.286 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:18.286 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:30:18.286 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.083850+0000", 2024-09-14T06:30:18.286 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.0130000000000001e-06 2024-09-14T06:30:18.286 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:18.286 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:18.286 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:30:18.286 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.083854+0000", 2024-09-14T06:30:18.286 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.4759999999999998e-06 2024-09-14T06:30:18.287 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:18.287 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:18.287 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:30:18.287 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.125636+0000", 2024-09-14T06:30:18.287 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.041782683000000001 2024-09-14T06:30:18.287 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:18.287 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:18.288 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:30:18.288 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.125680+0000", 2024-09-14T06:30:18.288 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.3614e-05 2024-09-14T06:30:18.288 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:18.289 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:18.289 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:30:18.289 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.125836+0000", 2024-09-14T06:30:18.289 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000155864 2024-09-14T06:30:18.289 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:18.289 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:18.289 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:18.289 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:18.289 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:18.290 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:14920 3.0 3:05c39535:::rbd_data.112d92027590.0000000000000009:f [read 816128~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:30:18.290 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:26.083868+0000", 2024-09-14T06:30:18.290 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 52.191999928000001, 2024-09-14T06:30:18.290 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.042166771999999998, 2024-09-14T06:30:18.291 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:30:18.291 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:30:18.291 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:30:18.291 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:30:18.291 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:30:18.291 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:18.291 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 14920 2024-09-14T06:30:18.291 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:18.292 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:30:18.292 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:18.292 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:30:18.292 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.083868+0000", 2024-09-14T06:30:18.292 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:18.293 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:18.293 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:18.293 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:30:18.293 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.083868+0000", 2024-09-14T06:30:18.293 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:18.293 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:18.293 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:18.293 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:30:18.293 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.083869+0000", 2024-09-14T06:30:18.293 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.99e-07 2024-09-14T06:30:18.293 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:18.294 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:18.294 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:30:18.294 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.083870+0000", 2024-09-14T06:30:18.294 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.782e-06 2024-09-14T06:30:18.294 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:18.294 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:18.294 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:30:18.294 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.083876+0000", 2024-09-14T06:30:18.294 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.3809999999999998e-06 2024-09-14T06:30:18.294 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:18.294 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:18.294 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:30:18.294 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.083882+0000", 2024-09-14T06:30:18.294 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.9479999999999996e-06 2024-09-14T06:30:18.295 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:18.295 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:18.295 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:30:18.295 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.125878+0000", 2024-09-14T06:30:18.295 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.041996264999999998 2024-09-14T06:30:18.295 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:18.295 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:18.295 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:30:18.295 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.125929+0000", 2024-09-14T06:30:18.295 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.0498999999999999e-05 2024-09-14T06:30:18.295 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:18.295 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:18.295 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:30:18.296 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.126035+0000", 2024-09-14T06:30:18.296 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000106298 2024-09-14T06:30:18.296 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:18.296 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:18.296 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:18.296 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:18.296 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:18.296 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:14921 3.0 3:05c39535:::rbd_data.112d92027590.0000000000000009:f [read 826368~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:30:18.296 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:26.083889+0000", 2024-09-14T06:30:18.296 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 52.191978917999997, 2024-09-14T06:30:18.296 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.042435309999999997, 2024-09-14T06:30:18.296 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:30:18.297 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:30:18.297 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:30:18.297 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:30:18.297 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:30:18.297 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:18.297 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 14921 2024-09-14T06:30:18.297 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:18.297 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:30:18.297 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:18.297 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:30:18.297 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.083889+0000", 2024-09-14T06:30:18.297 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:18.297 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:18.297 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:18.298 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:30:18.298 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.083889+0000", 2024-09-14T06:30:18.298 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:18.298 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:18.298 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:18.298 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:30:18.298 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.086033+0000", 2024-09-14T06:30:18.298 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0021444179999999999 2024-09-14T06:30:18.298 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:18.298 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:18.298 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:30:18.298 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.086038+0000", 2024-09-14T06:30:18.298 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.0969999999999999e-06 2024-09-14T06:30:18.298 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:18.299 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:18.299 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:30:18.299 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.086046+0000", 2024-09-14T06:30:18.299 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.9440000000000005e-06 2024-09-14T06:30:18.299 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:18.299 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:18.299 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:30:18.299 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.086050+0000", 2024-09-14T06:30:18.299 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.69e-06 2024-09-14T06:30:18.299 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:18.299 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:18.299 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:30:18.299 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.126080+0000", 2024-09-14T06:30:18.300 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.040030237000000003 2024-09-14T06:30:18.300 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:18.300 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:18.300 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:30:18.300 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.126163+0000", 2024-09-14T06:30:18.300 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.2071999999999996e-05 2024-09-14T06:30:18.300 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:18.300 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:18.300 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:30:18.301 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.126324+0000", 2024-09-14T06:30:18.301 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00016185200000000001 2024-09-14T06:30:18.301 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:18.301 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:18.301 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:18.301 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:18.301 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:18.301 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:55627 3.0 3:06490baf:::rbd_data.112dca2f6ae1.0000000000000030:17 [read 2002944~4096 [fadvise_nocache]] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:30:18.301 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:44.599966+0000", 2024-09-14T06:30:18.301 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 33.675901969999998, 2024-09-14T06:30:18.301 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.041868546999999999, 2024-09-14T06:30:18.301 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:30:18.301 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:30:18.302 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:30:18.302 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:30:18.302 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:30:18.302 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:18.302 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 55627 2024-09-14T06:30:18.302 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:18.302 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:30:18.302 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:18.302 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:30:18.302 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.599966+0000", 2024-09-14T06:30:18.303 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:18.303 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:18.303 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:18.303 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:30:18.303 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.599966+0000", 2024-09-14T06:30:18.303 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:18.303 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:18.303 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:18.303 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:30:18.303 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.599966+0000", 2024-09-14T06:30:18.303 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.0900000000000001e-07 2024-09-14T06:30:18.303 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:18.303 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:18.304 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:30:18.304 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.599967+0000", 2024-09-14T06:30:18.304 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.64e-07 2024-09-14T06:30:18.304 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:18.304 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:18.304 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:30:18.304 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.599970+0000", 2024-09-14T06:30:18.304 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.9670000000000001e-06 2024-09-14T06:30:18.304 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:18.304 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:18.304 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:30:18.304 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.599973+0000", 2024-09-14T06:30:18.304 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.632e-06 2024-09-14T06:30:18.305 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:18.305 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:18.305 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:30:18.305 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.641665+0000", 2024-09-14T06:30:18.305 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.041692547000000003 2024-09-14T06:30:18.305 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:18.305 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:18.305 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:30:18.305 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.641740+0000", 2024-09-14T06:30:18.306 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.4559000000000002e-05 2024-09-14T06:30:18.306 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:18.306 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:18.306 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:30:18.306 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.641835+0000", 2024-09-14T06:30:18.306 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.4668999999999998e-05 2024-09-14T06:30:18.306 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:18.306 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:18.306 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:18.306 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:18.306 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:18.306 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:55628 3.0 3:06490baf:::rbd_data.112dca2f6ae1.0000000000000030:17 [read 2027520~4096 [fadvise_nocache]] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:30:18.306 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:44.599976+0000", 2024-09-14T06:30:18.307 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 33.675892183000002, 2024-09-14T06:30:18.307 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.041991621, 2024-09-14T06:30:18.307 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:30:18.307 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:30:18.307 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:30:18.307 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:30:18.307 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:30:18.307 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:18.307 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 55628 2024-09-14T06:30:18.307 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:18.307 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:30:18.307 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:18.307 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:30:18.307 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.599976+0000", 2024-09-14T06:30:18.308 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:18.308 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:18.308 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:18.308 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:30:18.308 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.599976+0000", 2024-09-14T06:30:18.308 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:18.308 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:18.308 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:18.308 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:30:18.308 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.599976+0000", 2024-09-14T06:30:18.309 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.1299999999999999e-07 2024-09-14T06:30:18.309 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:18.309 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:18.309 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:30:18.309 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.599977+0000", 2024-09-14T06:30:18.309 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.7999999999999993e-07 2024-09-14T06:30:18.309 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:18.309 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:18.309 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:30:18.309 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.599980+0000", 2024-09-14T06:30:18.309 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.01e-06 2024-09-14T06:30:18.309 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:18.309 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:18.309 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:30:18.310 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.599983+0000", 2024-09-14T06:30:18.310 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.5409999999999999e-06 2024-09-14T06:30:18.310 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:18.310 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:18.310 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:30:18.310 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.641869+0000", 2024-09-14T06:30:18.310 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.041886235000000001 2024-09-14T06:30:18.310 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:18.310 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:18.310 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:30:18.310 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.641913+0000", 2024-09-14T06:30:18.310 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.4357000000000002e-05 2024-09-14T06:30:18.310 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:18.310 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:18.311 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:30:18.311 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.641967+0000", 2024-09-14T06:30:18.311 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.4284999999999999e-05 2024-09-14T06:30:18.311 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:18.311 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:18.311 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:18.311 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:18.311 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:18.311 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:55629 3.0 3:06490baf:::rbd_data.112dca2f6ae1.0000000000000030:17 [read 2052096~4096 [fadvise_nocache]] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:30:18.311 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:44.599985+0000", 2024-09-14T06:30:18.311 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 33.675882477000002, 2024-09-14T06:30:18.311 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.042183375000000002, 2024-09-14T06:30:18.312 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:30:18.312 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:30:18.312 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:30:18.312 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:30:18.312 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:30:18.312 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:18.312 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 55629 2024-09-14T06:30:18.312 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:18.312 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:30:18.312 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:18.312 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:30:18.312 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.599985+0000", 2024-09-14T06:30:18.313 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:18.313 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:18.313 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:18.313 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:30:18.313 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.599985+0000", 2024-09-14T06:30:18.313 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:18.313 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:18.313 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:18.313 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:30:18.313 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.599986+0000", 2024-09-14T06:30:18.313 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.05e-07 2024-09-14T06:30:18.313 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:18.313 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:18.314 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:30:18.314 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.599987+0000", 2024-09-14T06:30:18.314 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.8400000000000002e-07 2024-09-14T06:30:18.314 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:18.314 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:18.314 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:30:18.314 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.599990+0000", 2024-09-14T06:30:18.314 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.5640000000000001e-06 2024-09-14T06:30:18.314 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:18.315 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:18.315 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:30:18.315 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.599993+0000", 2024-09-14T06:30:18.315 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.615e-06 2024-09-14T06:30:18.315 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:18.315 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:18.315 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:30:18.315 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.642003+0000", 2024-09-14T06:30:18.315 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.042010345999999997 2024-09-14T06:30:18.315 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:18.315 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:18.315 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:30:18.315 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.642080+0000", 2024-09-14T06:30:18.316 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.6538999999999993e-05 2024-09-14T06:30:18.316 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:18.316 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:18.316 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:30:18.316 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.642169+0000", 2024-09-14T06:30:18.316 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.9122000000000005e-05 2024-09-14T06:30:18.316 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:18.316 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:18.316 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:18.317 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:18.317 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:18.317 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:55630 3.0 3:06490baf:::rbd_data.112dca2f6ae1.0000000000000030:17 [read 2076672~4096 [fadvise_nocache]] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:30:18.317 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:44.599996+0000", 2024-09-14T06:30:18.317 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 33.675871999000002, 2024-09-14T06:30:18.317 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.042299615999999998, 2024-09-14T06:30:18.317 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:30:18.317 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:30:18.317 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:30:18.317 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:30:18.317 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:30:18.317 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:18.317 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 55630 2024-09-14T06:30:18.318 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:18.318 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:30:18.318 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:18.318 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:30:18.318 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.599996+0000", 2024-09-14T06:30:18.318 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:18.318 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:18.318 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:18.318 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:30:18.318 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.599996+0000", 2024-09-14T06:30:18.318 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:18.318 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:18.318 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:18.319 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:30:18.319 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.599996+0000", 2024-09-14T06:30:18.319 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.1500000000000001e-07 2024-09-14T06:30:18.319 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:18.319 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:18.319 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:30:18.319 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.599997+0000", 2024-09-14T06:30:18.319 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.33e-07 2024-09-14T06:30:18.319 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:18.319 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:18.320 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:30:18.320 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600000+0000", 2024-09-14T06:30:18.320 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.0240000000000002e-06 2024-09-14T06:30:18.320 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:18.320 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:18.320 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:30:18.320 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600003+0000", 2024-09-14T06:30:18.320 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.3640000000000002e-06 2024-09-14T06:30:18.321 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:18.321 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:18.321 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:30:18.321 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.642200+0000", 2024-09-14T06:30:18.321 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.042197571000000003 2024-09-14T06:30:18.321 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:18.321 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:18.321 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:30:18.321 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.642243+0000", 2024-09-14T06:30:18.321 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.2765000000000001e-05 2024-09-14T06:30:18.321 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:18.321 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:18.321 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:30:18.321 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.642296+0000", 2024-09-14T06:30:18.322 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.2744000000000002e-05 2024-09-14T06:30:18.322 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:18.322 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:18.322 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:18.322 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:18.322 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:18.322 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:55631 3.0 3:06490baf:::rbd_data.112dca2f6ae1.0000000000000030:17 [read 1048576~4096 [fadvise_nocache]] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:30:18.322 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:44.600006+0000", 2024-09-14T06:30:18.322 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 33.675862410999997, 2024-09-14T06:30:18.322 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.042487905999999999, 2024-09-14T06:30:18.322 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:30:18.322 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:30:18.322 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:30:18.323 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:30:18.323 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:30:18.323 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:18.323 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 55631 2024-09-14T06:30:18.323 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:18.323 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:30:18.323 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:18.323 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:30:18.323 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600006+0000", 2024-09-14T06:30:18.323 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:18.324 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:18.324 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:18.324 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:30:18.324 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600006+0000", 2024-09-14T06:30:18.324 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:18.324 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:18.324 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:18.324 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:30:18.324 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600006+0000", 2024-09-14T06:30:18.324 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.0800000000000001e-07 2024-09-14T06:30:18.324 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:18.324 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:18.324 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:30:18.324 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600007+0000", 2024-09-14T06:30:18.325 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.7199999999999997e-07 2024-09-14T06:30:18.325 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:18.325 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:18.325 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:30:18.325 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600010+0000", 2024-09-14T06:30:18.325 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.0280000000000001e-06 2024-09-14T06:30:18.325 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:18.325 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:18.325 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:30:18.325 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600012+0000", 2024-09-14T06:30:18.325 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.475e-06 2024-09-14T06:30:18.325 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:18.325 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:18.325 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:30:18.326 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.642329+0000", 2024-09-14T06:30:18.326 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.042317157000000001 2024-09-14T06:30:18.326 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:18.326 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:18.326 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:30:18.326 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.642403+0000", 2024-09-14T06:30:18.326 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.3941000000000002e-05 2024-09-14T06:30:18.326 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:18.326 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:18.326 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:30:18.326 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.642493+0000", 2024-09-14T06:30:18.326 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.0124999999999995e-05 2024-09-14T06:30:18.326 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:18.327 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:18.327 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:18.327 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:18.327 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:18.327 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:55632 3.0 3:06490baf:::rbd_data.112dca2f6ae1.0000000000000030:17 [read 1073152~4096 [fadvise_nocache]] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:30:18.327 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:44.600015+0000", 2024-09-14T06:30:18.327 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 33.675852657999997, 2024-09-14T06:30:18.327 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.042604922000000003, 2024-09-14T06:30:18.327 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:30:18.327 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:30:18.327 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:30:18.327 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:30:18.327 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:30:18.327 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:18.328 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 55632 2024-09-14T06:30:18.328 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:18.328 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:30:18.328 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:18.328 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:30:18.328 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600015+0000", 2024-09-14T06:30:18.328 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:18.328 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:18.328 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:18.329 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:30:18.329 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600015+0000", 2024-09-14T06:30:18.329 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:18.329 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:18.329 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:18.329 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:30:18.329 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600016+0000", 2024-09-14T06:30:18.329 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.1e-07 2024-09-14T06:30:18.329 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:18.329 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:18.329 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:30:18.329 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600017+0000", 2024-09-14T06:30:18.329 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.9900000000000009e-07 2024-09-14T06:30:18.329 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:18.330 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:18.330 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:30:18.330 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600020+0000", 2024-09-14T06:30:18.330 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.4170000000000001e-06 2024-09-14T06:30:18.330 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:18.330 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:18.330 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:30:18.330 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600023+0000", 2024-09-14T06:30:18.330 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.588e-06 2024-09-14T06:30:18.330 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:18.330 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:18.330 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:30:18.330 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.642526+0000", 2024-09-14T06:30:18.331 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.042503236999999999 2024-09-14T06:30:18.331 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:18.331 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:18.331 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:30:18.331 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.642568+0000", 2024-09-14T06:30:18.331 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.1925000000000003e-05 2024-09-14T06:30:18.331 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:18.331 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:18.331 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:30:18.331 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.642620+0000", 2024-09-14T06:30:18.331 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.2546000000000001e-05 2024-09-14T06:30:18.331 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:18.331 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:18.331 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:18.332 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:18.332 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:18.332 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:55633 3.0 3:06490baf:::rbd_data.112dca2f6ae1.0000000000000030:17 [read 1097728~4096 [fadvise_nocache]] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:30:18.332 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:44.600026+0000", 2024-09-14T06:30:18.332 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 33.675842389000003, 2024-09-14T06:30:18.332 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.042797474000000002, 2024-09-14T06:30:18.332 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:30:18.332 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:30:18.332 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:30:18.332 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:30:18.333 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:30:18.333 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:18.333 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 55633 2024-09-14T06:30:18.333 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:18.333 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:30:18.333 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:18.333 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:30:18.333 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600026+0000", 2024-09-14T06:30:18.334 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:18.334 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:18.334 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:18.334 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:30:18.334 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600026+0000", 2024-09-14T06:30:18.334 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:18.334 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:18.335 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:18.335 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:30:18.335 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600026+0000", 2024-09-14T06:30:18.335 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.3099999999999999e-07 2024-09-14T06:30:18.335 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:18.335 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:18.336 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:30:18.336 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600029+0000", 2024-09-14T06:30:18.336 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.304e-06 2024-09-14T06:30:18.336 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:18.336 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:18.337 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:30:18.337 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600034+0000", 2024-09-14T06:30:18.337 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.1429999999999997e-06 2024-09-14T06:30:18.337 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:18.337 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:18.337 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:30:18.338 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600038+0000", 2024-09-14T06:30:18.338 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.692e-06 2024-09-14T06:30:18.338 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:18.338 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:18.338 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:30:18.338 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.642654+0000", 2024-09-14T06:30:18.339 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.042615564000000002 2024-09-14T06:30:18.339 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:18.339 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:18.339 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:30:18.339 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.642728+0000", 2024-09-14T06:30:18.339 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.4018999999999997e-05 2024-09-14T06:30:18.340 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:18.340 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:18.340 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:30:18.340 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.642823+0000", 2024-09-14T06:30:18.340 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.5520999999999996e-05 2024-09-14T06:30:18.340 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:18.341 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:18.341 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:18.341 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:18.341 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:18.341 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:55634 3.0 3:06490baf:::rbd_data.112dca2f6ae1.0000000000000030:17 [read 1122304~4096 [fadvise_nocache]] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:30:18.342 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:44.600042+0000", 2024-09-14T06:30:18.342 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 33.675825912000001, 2024-09-14T06:30:18.342 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.042903729000000002, 2024-09-14T06:30:18.342 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:30:18.342 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:30:18.342 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:30:18.343 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:30:18.343 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:30:18.343 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:18.343 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 55634 2024-09-14T06:30:18.343 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:18.343 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:30:18.344 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:18.344 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:30:18.344 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600042+0000", 2024-09-14T06:30:18.344 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:18.344 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:18.345 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:18.345 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:30:18.345 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600042+0000", 2024-09-14T06:30:18.345 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:18.345 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:18.346 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:18.346 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:30:18.346 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600043+0000", 2024-09-14T06:30:18.346 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.7000000000000005e-07 2024-09-14T06:30:18.346 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:18.346 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:18.347 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:30:18.347 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600044+0000", 2024-09-14T06:30:18.347 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.742e-06 2024-09-14T06:30:18.347 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:18.347 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:18.347 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:30:18.348 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600050+0000", 2024-09-14T06:30:18.348 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.0800000000000002e-06 2024-09-14T06:30:18.348 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:18.348 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:18.348 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:30:18.349 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600055+0000", 2024-09-14T06:30:18.349 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.3889999999999998e-06 2024-09-14T06:30:18.349 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:18.349 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:18.350 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:30:18.350 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.642845+0000", 2024-09-14T06:30:18.350 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.042790040000000001 2024-09-14T06:30:18.350 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:18.350 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:18.350 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:30:18.351 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.642888+0000", 2024-09-14T06:30:18.351 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.2790000000000002e-05 2024-09-14T06:30:18.351 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:18.351 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:18.351 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:30:18.352 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.642946+0000", 2024-09-14T06:30:18.352 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.8118000000000002e-05 2024-09-14T06:30:18.352 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:18.352 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:18.352 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:18.352 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:18.353 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:18.353 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:55635 3.0 3:06490baf:::rbd_data.112dca2f6ae1.0000000000000030:17 [read 1146880~4096 [fadvise_nocache]] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:30:18.353 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:44.600061+0000", 2024-09-14T06:30:18.353 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 33.675807341000002, 2024-09-14T06:30:18.353 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.043092695, 2024-09-14T06:30:18.353 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:30:18.354 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:30:18.354 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:30:18.354 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:30:18.354 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:30:18.354 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:18.355 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 55635 2024-09-14T06:30:18.355 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:18.355 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:30:18.355 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:18.355 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:30:18.355 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600061+0000", 2024-09-14T06:30:18.356 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:18.356 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:18.356 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:18.356 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:30:18.356 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600061+0000", 2024-09-14T06:30:18.356 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:18.357 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:18.357 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:18.357 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:30:18.357 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600061+0000", 2024-09-14T06:30:18.357 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.3099999999999999e-07 2024-09-14T06:30:18.358 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:18.358 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:18.358 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:30:18.358 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600063+0000", 2024-09-14T06:30:18.358 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.6220000000000001e-06 2024-09-14T06:30:18.358 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:18.359 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:18.359 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:30:18.359 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600068+0000", 2024-09-14T06:30:18.359 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.4790000000000004e-06 2024-09-14T06:30:18.359 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:18.359 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:18.360 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:30:18.360 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600072+0000", 2024-09-14T06:30:18.360 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.3100000000000002e-06 2024-09-14T06:30:18.360 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:18.360 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:18.360 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:30:18.360 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.642987+0000", 2024-09-14T06:30:18.361 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.042914489 2024-09-14T06:30:18.361 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:18.361 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:18.361 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:30:18.361 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.643062+0000", 2024-09-14T06:30:18.361 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.5011000000000004e-05 2024-09-14T06:30:18.361 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:18.362 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:18.362 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:30:18.362 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.643153+0000", 2024-09-14T06:30:18.362 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.1452999999999993e-05 2024-09-14T06:30:18.362 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:18.362 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:18.362 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:18.362 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:18.362 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:18.362 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:55636 3.0 3:06490baf:::rbd_data.112dca2f6ae1.0000000000000030:17 [read 1171456~4096 [fadvise_nocache]] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:30:18.363 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:44.600078+0000", 2024-09-14T06:30:18.363 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 33.675789928999997, 2024-09-14T06:30:18.363 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.043207966, 2024-09-14T06:30:18.363 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:30:18.363 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:30:18.363 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:30:18.363 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:30:18.363 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:30:18.363 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:18.363 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 55636 2024-09-14T06:30:18.364 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:18.364 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:30:18.364 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:18.364 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:30:18.364 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600078+0000", 2024-09-14T06:30:18.364 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:18.364 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:18.364 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:18.364 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:30:18.364 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600078+0000", 2024-09-14T06:30:18.364 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:18.364 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:18.364 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:18.365 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:30:18.365 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600078+0000", 2024-09-14T06:30:18.365 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.34e-07 2024-09-14T06:30:18.365 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:18.365 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:18.365 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:30:18.365 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600080+0000", 2024-09-14T06:30:18.365 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.641e-06 2024-09-14T06:30:18.365 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:18.365 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:18.365 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:30:18.365 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600086+0000", 2024-09-14T06:30:18.365 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.3860000000000004e-06 2024-09-14T06:30:18.366 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:18.366 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:18.366 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:30:18.366 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600091+0000", 2024-09-14T06:30:18.366 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.5970000000000002e-06 2024-09-14T06:30:18.366 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:18.366 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:18.366 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:30:18.366 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.643184+0000", 2024-09-14T06:30:18.366 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.043092893 2024-09-14T06:30:18.366 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:18.366 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:18.366 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:30:18.366 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.643228+0000", 2024-09-14T06:30:18.367 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.4536999999999997e-05 2024-09-14T06:30:18.367 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:18.367 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:18.367 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:30:18.367 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.643286+0000", 2024-09-14T06:30:18.367 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.7578000000000003e-05 2024-09-14T06:30:18.367 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:18.367 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:18.367 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:18.367 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:18.367 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:18.367 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:55637 3.0 3:06490baf:::rbd_data.112dca2f6ae1.0000000000000030:17 [read 1196032~4096 [fadvise_nocache]] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:30:18.367 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:44.600097+0000", 2024-09-14T06:30:18.367 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 33.675771365000003, 2024-09-14T06:30:18.368 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.043399124999999997, 2024-09-14T06:30:18.368 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:30:18.368 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:30:18.368 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:30:18.368 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:30:18.368 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:30:18.368 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:18.368 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 55637 2024-09-14T06:30:18.368 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:18.368 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:30:18.368 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:18.368 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:30:18.368 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600097+0000", 2024-09-14T06:30:18.369 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:18.369 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:18.369 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:18.369 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:30:18.369 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600097+0000", 2024-09-14T06:30:18.369 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:18.369 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:18.369 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:18.369 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:30:18.369 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.604372+0000", 2024-09-14T06:30:18.369 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0042751489999999998 2024-09-14T06:30:18.369 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:18.369 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:18.370 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:30:18.370 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.604377+0000", 2024-09-14T06:30:18.370 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.7899999999999999e-06 2024-09-14T06:30:18.370 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:18.370 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:18.370 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:30:18.370 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.604389+0000", 2024-09-14T06:30:18.370 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.2197e-05 2024-09-14T06:30:18.370 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:18.370 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:18.370 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:30:18.370 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.604397+0000", 2024-09-14T06:30:18.370 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.8730000000000007e-06 2024-09-14T06:30:18.370 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:18.371 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:18.371 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:30:18.371 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.643324+0000", 2024-09-14T06:30:18.371 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.038926897000000002 2024-09-14T06:30:18.371 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:18.371 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:18.371 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:30:18.371 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.643402+0000", 2024-09-14T06:30:18.371 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.8063000000000001e-05 2024-09-14T06:30:18.371 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:18.371 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:18.371 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:30:18.371 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.643496+0000", 2024-09-14T06:30:18.371 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.4155999999999998e-05 2024-09-14T06:30:18.372 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:18.372 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:18.372 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:18.372 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:18.372 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:18.372 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:55657 3.0 3:06490baf:::rbd_data.112dca2f6ae1.0000000000000030:17 [read 1687552~4096 [fadvise_nocache]] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:30:18.372 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:44.604841+0000", 2024-09-14T06:30:18.372 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 33.671026636000001, 2024-09-14T06:30:18.372 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.041890087999999999, 2024-09-14T06:30:18.372 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:30:18.372 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:30:18.372 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:30:18.372 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:30:18.373 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:30:18.373 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:18.373 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 55657 2024-09-14T06:30:18.373 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:18.373 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:30:18.373 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:18.373 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:30:18.373 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.604841+0000", 2024-09-14T06:30:18.373 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:18.373 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:18.373 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:18.373 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:30:18.373 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.604841+0000", 2024-09-14T06:30:18.373 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:18.374 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:18.374 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:18.374 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:30:18.374 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.604842+0000", 2024-09-14T06:30:18.374 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.0900000000000001e-07 2024-09-14T06:30:18.374 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:18.374 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:18.374 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:30:18.374 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.604846+0000", 2024-09-14T06:30:18.374 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.5759999999999999e-06 2024-09-14T06:30:18.374 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:18.374 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:18.374 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:30:18.374 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.604850+0000", 2024-09-14T06:30:18.375 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.405e-06 2024-09-14T06:30:18.375 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:18.375 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:18.375 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:30:18.375 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.604852+0000", 2024-09-14T06:30:18.375 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.8590000000000001e-06 2024-09-14T06:30:18.375 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:18.375 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:18.375 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:30:18.375 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.646633+0000", 2024-09-14T06:30:18.375 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.041780772000000001 2024-09-14T06:30:18.375 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:18.375 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:18.376 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:30:18.376 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.646677+0000", 2024-09-14T06:30:18.376 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.4115e-05 2024-09-14T06:30:18.376 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:18.376 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:18.376 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:30:18.376 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.646731+0000", 2024-09-14T06:30:18.376 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.4151999999999999e-05 2024-09-14T06:30:18.376 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:18.376 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:18.376 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:18.376 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:18.377 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:18.377 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:55658 3.0 3:06490baf:::rbd_data.112dca2f6ae1.0000000000000030:17 [read 1712128~4096 [fadvise_nocache]] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:30:18.377 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:44.604856+0000", 2024-09-14T06:30:18.377 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 33.671012036, 2024-09-14T06:30:18.377 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.041986227000000001, 2024-09-14T06:30:18.377 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:30:18.377 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:30:18.377 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:30:18.377 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:30:18.378 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:30:18.378 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:18.378 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 55658 2024-09-14T06:30:18.378 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:18.378 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:30:18.378 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:18.378 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:30:18.378 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.604856+0000", 2024-09-14T06:30:18.379 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:18.379 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:18.379 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:18.379 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:30:18.379 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.604856+0000", 2024-09-14T06:30:18.379 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:18.379 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:18.379 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:18.379 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:30:18.379 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.604856+0000", 2024-09-14T06:30:18.379 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.23e-07 2024-09-14T06:30:18.380 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:18.380 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:18.380 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:30:18.380 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.604857+0000", 2024-09-14T06:30:18.380 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.9999999999999995e-07 2024-09-14T06:30:18.380 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:18.380 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:18.380 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:30:18.380 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.604861+0000", 2024-09-14T06:30:18.380 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.9890000000000003e-06 2024-09-14T06:30:18.380 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:18.380 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:18.381 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:30:18.381 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.604864+0000", 2024-09-14T06:30:18.381 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.7429999999999998e-06 2024-09-14T06:30:18.381 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:18.381 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:18.381 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:30:18.381 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.646744+0000", 2024-09-14T06:30:18.381 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.041880325000000003 2024-09-14T06:30:18.381 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:18.381 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:18.381 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:30:18.381 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.646784+0000", 2024-09-14T06:30:18.382 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.9604000000000002e-05 2024-09-14T06:30:18.382 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:18.382 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:18.382 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:30:18.382 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.646842+0000", 2024-09-14T06:30:18.382 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.8343e-05 2024-09-14T06:30:18.382 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:18.382 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:18.382 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:18.382 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:18.382 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:18.382 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:55659 3.0 3:06490baf:::rbd_data.112dca2f6ae1.0000000000000030:17 [read 1736704~4096 [fadvise_nocache]] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:30:18.382 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:44.604867+0000", 2024-09-14T06:30:18.383 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 33.671000882999998, 2024-09-14T06:30:18.383 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.042097082000000001, 2024-09-14T06:30:18.383 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:30:18.383 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:30:18.383 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:30:18.383 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:30:18.383 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:30:18.383 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:18.384 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 55659 2024-09-14T06:30:18.384 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:18.384 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:30:18.384 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:18.384 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:30:18.384 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.604867+0000", 2024-09-14T06:30:18.384 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:18.384 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:18.384 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:18.384 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:30:18.385 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.604867+0000", 2024-09-14T06:30:18.385 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:18.385 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:18.385 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:18.385 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:30:18.385 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.604867+0000", 2024-09-14T06:30:18.386 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.2000000000000001e-07 2024-09-14T06:30:18.386 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:18.386 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:18.386 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:30:18.386 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.604868+0000", 2024-09-14T06:30:18.386 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.04e-06 2024-09-14T06:30:18.386 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:18.386 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:18.386 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:30:18.386 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.604871+0000", 2024-09-14T06:30:18.387 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.9689999999999999e-06 2024-09-14T06:30:18.387 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:18.387 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:18.387 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:30:18.387 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.604874+0000", 2024-09-14T06:30:18.387 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.5069999999999999e-06 2024-09-14T06:30:18.387 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:18.387 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:18.387 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:30:18.387 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.646874+0000", 2024-09-14T06:30:18.387 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.042000513000000003 2024-09-14T06:30:18.387 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:18.387 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:18.387 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:30:18.388 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.646914+0000", 2024-09-14T06:30:18.388 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.9443999999999997e-05 2024-09-14T06:30:18.388 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:18.388 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:18.388 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:30:18.388 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.646964+0000", 2024-09-14T06:30:18.388 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.0389000000000002e-05 2024-09-14T06:30:18.388 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:18.389 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:18.389 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:18.389 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:18.389 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:18.389 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:55660 3.0 3:06490baf:::rbd_data.112dca2f6ae1.0000000000000030:17 [read 1761280~4096 [fadvise_nocache]] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:30:18.390 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:44.604877+0000", 2024-09-14T06:30:18.390 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 33.670991133999998, 2024-09-14T06:30:18.390 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.042196349000000001, 2024-09-14T06:30:18.390 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:30:18.390 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:30:18.391 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:30:18.391 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:30:18.391 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:30:18.391 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:18.391 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 55660 2024-09-14T06:30:18.391 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:18.392 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:30:18.392 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:18.392 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:30:18.392 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.604877+0000", 2024-09-14T06:30:18.392 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:18.392 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:18.392 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:18.392 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:30:18.392 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.604877+0000", 2024-09-14T06:30:18.393 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:18.393 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:18.393 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:18.393 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:30:18.393 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.605810+0000", 2024-09-14T06:30:18.393 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00093336899999999997 2024-09-14T06:30:18.393 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:18.393 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:18.393 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:30:18.393 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.605815+0000", 2024-09-14T06:30:18.393 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.6140000000000002e-06 2024-09-14T06:30:18.393 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:18.394 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:18.394 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:30:18.394 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.605825+0000", 2024-09-14T06:30:18.394 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.8889999999999997e-06 2024-09-14T06:30:18.394 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:18.394 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:18.394 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:30:18.394 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.605831+0000", 2024-09-14T06:30:18.394 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.9800000000000003e-06 2024-09-14T06:30:18.394 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:18.394 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:18.394 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:30:18.394 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.646989+0000", 2024-09-14T06:30:18.394 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.041157878000000002 2024-09-14T06:30:18.395 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:18.395 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:18.395 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:30:18.395 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.647026+0000", 2024-09-14T06:30:18.395 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.7305000000000003e-05 2024-09-14T06:30:18.395 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:18.395 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:18.395 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:30:18.395 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.647073+0000", 2024-09-14T06:30:18.395 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.7314000000000001e-05 2024-09-14T06:30:18.395 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:18.395 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:18.395 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:18.395 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:18.396 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:18.396 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:55666 3.0 3:06490baf:::rbd_data.112dca2f6ae1.0000000000000030:17 [read 1908736~4096 [fadvise_nocache]] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:30:18.396 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:44.605898+0000", 2024-09-14T06:30:18.396 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 33.669970388000003, 2024-09-14T06:30:18.396 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.041862887000000001, 2024-09-14T06:30:18.396 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:30:18.396 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:30:18.396 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:30:18.396 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:30:18.396 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:30:18.396 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:18.396 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 55666 2024-09-14T06:30:18.397 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:18.397 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:30:18.397 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:18.397 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:30:18.397 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.605898+0000", 2024-09-14T06:30:18.397 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:18.397 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:18.397 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:18.397 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:30:18.397 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.605898+0000", 2024-09-14T06:30:18.397 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:18.397 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:18.397 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:18.398 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:30:18.398 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.611874+0000", 2024-09-14T06:30:18.398 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0059764049999999997 2024-09-14T06:30:18.398 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:18.398 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:18.398 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:30:18.398 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.611879+0000", 2024-09-14T06:30:18.398 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.8749999999999999e-06 2024-09-14T06:30:18.398 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:18.398 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:18.398 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:30:18.398 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.611887+0000", 2024-09-14T06:30:18.398 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.4109999999999996e-06 2024-09-14T06:30:18.398 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:18.398 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:18.399 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:30:18.399 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.611893+0000", 2024-09-14T06:30:18.399 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.2959999999999998e-06 2024-09-14T06:30:18.399 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:18.399 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:18.399 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:30:18.399 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.647674+0000", 2024-09-14T06:30:18.399 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.035781281999999998 2024-09-14T06:30:18.399 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:18.399 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:18.399 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:30:18.399 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.647711+0000", 2024-09-14T06:30:18.399 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.7434000000000001e-05 2024-09-14T06:30:18.399 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:18.400 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:18.400 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:30:18.400 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.647760+0000", 2024-09-14T06:30:18.400 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.9184000000000002e-05 2024-09-14T06:30:18.400 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:18.400 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:18.400 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:18.400 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:18.400 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:18.400 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-14T06:30:18.401 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-14T06:30:18.402 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-14T06:30:18.403 INFO:tasks.ceph.osd.7.smithi138.stderr:2024-09-14T06:30:18.401+0000 7f3c6a21d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33623) UID: 0 2024-09-14T06:30:18.503 INFO:tasks.ceph.osd.2.smithi094.stderr:2024-09-14T06:30:18.502+0000 7faf3269b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33649) UID: 0 2024-09-14T06:30:18.586 INFO:teuthology.orchestra.run.smithi094.stderr:reweighted osd.0 to 1 (10000) 2024-09-14T06:30:18.586 INFO:teuthology.orchestra.run.smithi094.stderr:noscrub is unset 2024-09-14T06:30:18.590 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-14T06:30:18.590 INFO:teuthology.orchestra.run.smithi094.stdout: "ops": [], 2024-09-14T06:30:18.590 INFO:teuthology.orchestra.run.smithi094.stdout: "num_ops": 0 2024-09-14T06:30:18.590 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-14T06:30:18.604 INFO:tasks.ceph.osd.6.smithi138.stderr:2024-09-14T06:30:18.603+0000 7f8cd8849640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33620) UID: 0 2024-09-14T06:30:18.606 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-14T06:30:18.614 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 1 1 2024-09-14T06:30:18.628 INFO:teuthology.orchestra.run.smithi094.stdout:osd.0: {} 2024-09-14T06:30:18.629 INFO:teuthology.orchestra.run.smithi094.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-14T06:30:18.655 INFO:teuthology.orchestra.run.smithi094.stdout:osd.1: {} 2024-09-14T06:30:18.655 INFO:teuthology.orchestra.run.smithi094.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-14T06:30:18.677 INFO:teuthology.orchestra.run.smithi094.stdout:osd.2: {} 2024-09-14T06:30:18.678 INFO:teuthology.orchestra.run.smithi094.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-14T06:30:18.697 INFO:teuthology.orchestra.run.smithi094.stdout:osd.3: {} 2024-09-14T06:30:18.697 INFO:teuthology.orchestra.run.smithi094.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-14T06:30:18.705 INFO:tasks.ceph.osd.4.smithi138.stderr:2024-09-14T06:30:18.704+0000 7f6c653c8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33615) UID: 0 2024-09-14T06:30:18.715 INFO:teuthology.orchestra.run.smithi094.stdout:osd.4: {} 2024-09-14T06:30:18.715 INFO:teuthology.orchestra.run.smithi094.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-14T06:30:18.733 INFO:teuthology.orchestra.run.smithi094.stdout:osd.5: {} 2024-09-14T06:30:18.733 INFO:teuthology.orchestra.run.smithi094.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-14T06:30:18.747 INFO:teuthology.orchestra.run.smithi094.stdout:ERROR: (22) Invalid argument 2024-09-14T06:30:18.748 INFO:teuthology.orchestra.run.smithi094.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:30:18.749 INFO:teuthology.orchestra.run.smithi094.stdout:osd.6: {} 2024-09-14T06:30:18.750 INFO:teuthology.orchestra.run.smithi094.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-14T06:30:18.758 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-14T06:30:18.767 INFO:teuthology.orchestra.run.smithi094.stdout:osd.7: {} 2024-09-14T06:30:18.768 INFO:teuthology.orchestra.run.smithi094.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-14T06:30:18.806 INFO:tasks.ceph.osd.7.smithi138.stderr:2024-09-14T06:30:18.805+0000 7f3c6a21d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33623) UID: 0 2024-09-14T06:30:18.871 INFO:teuthology.orchestra.run.smithi094.stdout:ERROR: (22) Invalid argument 2024-09-14T06:30:18.872 INFO:teuthology.orchestra.run.smithi094.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:30:18.880 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-14T06:30:18.907 INFO:tasks.ceph.osd.5.smithi138.stderr:2024-09-14T06:30:18.905+0000 7fe13746f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33618) UID: 0 2024-09-14T06:30:18.984 INFO:teuthology.orchestra.run.smithi094.stdout:ERROR: (22) Invalid argument 2024-09-14T06:30:18.984 INFO:teuthology.orchestra.run.smithi094.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:30:18.992 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-14T06:30:19.008 INFO:tasks.ceph.osd.6.smithi138.stderr:2024-09-14T06:30:19.007+0000 7f8cd8849640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33620) UID: 0 2024-09-14T06:30:19.095 INFO:teuthology.orchestra.run.smithi094.stdout:ERROR: (22) Invalid argument 2024-09-14T06:30:19.096 INFO:teuthology.orchestra.run.smithi094.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:30:19.104 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-14T06:30:19.148 INFO:tasks.ceph.osd.1.smithi094.stderr:2024-09-14T06:30:19.147+0000 7f328e47a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33641) UID: 0 2024-09-14T06:30:19.206 INFO:teuthology.orchestra.run.smithi094.stdout:ERROR: (22) Invalid argument 2024-09-14T06:30:19.207 INFO:teuthology.orchestra.run.smithi094.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:30:19.210 INFO:tasks.ceph.osd.6.smithi138.stderr:2024-09-14T06:30:19.209+0000 7f8cd8849640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33620) UID: 0 2024-09-14T06:30:19.215 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-14T06:30:19.310 INFO:tasks.ceph.osd.0.smithi094.stderr:2024-09-14T06:30:19.309+0000 7f07c9333640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33636) UID: 0 2024-09-14T06:30:19.320 INFO:teuthology.orchestra.run.smithi138.stdout:ERROR: (22) Invalid argument 2024-09-14T06:30:19.320 INFO:teuthology.orchestra.run.smithi138.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:30:19.328 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-14T06:30:19.412 INFO:tasks.ceph.osd.5.smithi138.stderr:2024-09-14T06:30:19.410+0000 7fe13746f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33618) UID: 0 2024-09-14T06:30:19.431 INFO:teuthology.orchestra.run.smithi138.stdout:ERROR: (22) Invalid argument 2024-09-14T06:30:19.431 INFO:teuthology.orchestra.run.smithi138.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:30:19.440 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-14T06:30:19.512 INFO:tasks.ceph.osd.2.smithi094.stderr:2024-09-14T06:30:19.511+0000 7faf3269b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33649) UID: 0 2024-09-14T06:30:19.584 INFO:teuthology.orchestra.run.smithi138.stdout:ERROR: (22) Invalid argument 2024-09-14T06:30:19.585 INFO:teuthology.orchestra.run.smithi138.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:30:19.593 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-14T06:30:19.595 INFO:teuthology.orchestra.run.smithi094.stderr:reweighted osd.1 to 1 (10000) 2024-09-14T06:30:19.612 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 2 1 2024-09-14T06:30:19.654 INFO:tasks.ceph.osd.1.smithi094.stderr:2024-09-14T06:30:19.652+0000 7f328e47a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33641) UID: 0 2024-09-14T06:30:19.701 INFO:teuthology.orchestra.run.smithi138.stdout:ERROR: (22) Invalid argument 2024-09-14T06:30:19.702 INFO:teuthology.orchestra.run.smithi138.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:30:19.710 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-14T06:30:19.713 INFO:tasks.ceph.osd.2.smithi094.stderr:2024-09-14T06:30:19.712+0000 7faf3269b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33649) UID: 0 2024-09-14T06:30:19.725 INFO:teuthology.orchestra.run.smithi094.stdout:checking clone #18, image image_client.0-clone18 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent19 2024-09-14T06:30:19.814 INFO:tasks.ceph.osd.4.smithi138.stderr:2024-09-14T06:30:19.813+0000 7f6c653c8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33615) UID: 0 2024-09-14T06:30:19.820 INFO:teuthology.orchestra.run.smithi138.stdout:ERROR: (22) Invalid argument 2024-09-14T06:30:19.820 INFO:teuthology.orchestra.run.smithi138.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:30:19.829 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-14T06:30:19.859 INFO:teuthology.orchestra.run.smithi094.stdout:2999 read 0x202fea8 thru 0x203d8f5 (0xda4e bytes) 2024-09-14T06:30:19.860 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-14T06:30:19.865 INFO:teuthology.orchestra.run.smithi094.stdout:3001 trunc from 0xac3a62a to 0x68b6372 2024-09-14T06:30:19.909 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-14T06:30:19.909 INFO:teuthology.orchestra.run.smithi094.stdout:3004 punch from 0x8c7e37 to 0x8d50af, (0xd278 bytes) 2024-09-14T06:30:19.909 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-14T06:30:19.915 INFO:tasks.ceph.osd.1.smithi094.stderr:2024-09-14T06:30:19.914+0000 7f328e47a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33641) UID: 0 2024-09-14T06:30:19.916 INFO:teuthology.orchestra.run.smithi094.stdout:3006 write 0x84f71fc thru 0x8503f62 (0xcd67 bytes) 2024-09-14T06:30:19.922 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-14T06:30:19.927 INFO:teuthology.orchestra.run.smithi094.stdout:3009 writesame 0x89eaa86 thru 0x89ecb7d data_size 0x14(0x20f8 bytes) 2024-09-14T06:30:19.927 INFO:teuthology.orchestra.run.smithi094.stdout:3010 read 0x4a5184a thru 0x4a6157c (0xfd33 bytes) 2024-09-14T06:30:19.927 INFO:teuthology.orchestra.run.smithi094.stdout:3011 read 0x37e983 thru 0x37f007 (0x685 bytes) 2024-09-14T06:30:19.927 INFO:teuthology.orchestra.run.smithi094.stdout:3012 write 0x76ddc54 thru 0x76e51a0 (0x754d bytes) 2024-09-14T06:30:19.933 INFO:teuthology.orchestra.run.smithi094.stdout:3013 write 0x307451d thru 0x307c331 (0x7e15 bytes) 2024-09-14T06:30:19.935 INFO:teuthology.orchestra.run.smithi094.stdout:3015 read 0xcab70 thru 0xd1083 (0x6514 bytes) 2024-09-14T06:30:19.936 INFO:teuthology.orchestra.run.smithi094.stdout:3016 punch from 0x51f2183 to 0x51ff8a7, (0xd724 bytes) 2024-09-14T06:30:19.936 INFO:teuthology.orchestra.run.smithi094.stdout:3018 read 0x344d2a0 thru 0x345bf8d (0xecee bytes) 2024-09-14T06:30:19.945 INFO:teuthology.orchestra.run.smithi094.stdout:3019 trunc from 0x89ecb7e to 0xeb86ea0 2024-09-14T06:30:19.950 INFO:teuthology.orchestra.run.smithi094.stdout:3020 read 0x5bc2bbc thru 0x5bd16ce (0xeb13 bytes) 2024-09-14T06:30:19.951 INFO:teuthology.orchestra.run.smithi094.stdout:3022 read 0xc7c274a thru 0xc7ccf65 (0xa81c bytes) 2024-09-14T06:30:19.951 INFO:teuthology.orchestra.run.smithi094.stdout:3023 read 0x6b334ff thru 0x6b38ad4 (0x55d6 bytes) 2024-09-14T06:30:19.951 INFO:teuthology.orchestra.run.smithi094.stdout:3025 read 0xe394dcc thru 0xe3a3a71 (0xeca6 bytes) 2024-09-14T06:30:19.952 INFO:teuthology.orchestra.run.smithi094.stdout:3026 punch from 0x9f532e1 to 0x9f5fde3, (0xcb02 bytes) 2024-09-14T06:30:19.952 INFO:teuthology.orchestra.run.smithi094.stdout:3028 read 0x1814361 thru 0x1820a9a (0xc73a bytes) 2024-09-14T06:30:19.957 INFO:teuthology.orchestra.run.smithi094.stdout:3029 trunc from 0xeb86ea0 to 0x9e4947e 2024-09-14T06:30:19.963 INFO:teuthology.orchestra.run.smithi094.stdout:3030 write 0x59da5ea thru 0x59e9c89 (0xf6a0 bytes) 2024-09-14T06:30:19.965 INFO:teuthology.orchestra.run.smithi094.stdout:3031 punch from 0x669cbbb to 0x66a3492, (0x68d7 bytes) 2024-09-14T06:30:19.975 INFO:teuthology.orchestra.run.smithi094.stdout:3032 trunc from 0x9e4947e to 0x228b7be 2024-09-14T06:30:19.997 INFO:teuthology.orchestra.run.smithi138.stdout:ERROR: (22) Invalid argument 2024-09-14T06:30:19.997 INFO:teuthology.orchestra.run.smithi138.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:30:20.010 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-14T06:30:20.016 INFO:tasks.ceph.osd.3.smithi094.stderr:2024-09-14T06:30:20.015+0000 7feeeafe8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33655) UID: 0 2024-09-14T06:30:20.040 INFO:teuthology.orchestra.run.smithi094.stdout:3035 write 0xd721f82 thru 0xd730a55 (0xead4 bytes) 2024-09-14T06:30:20.047 INFO:teuthology.orchestra.run.smithi094.stdout:3036 writesame 0x1777c4 thru 0x17bcbb data_size 0x4(0x44f8 bytes) 2024-09-14T06:30:20.050 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-14T06:30:20.050 INFO:teuthology.orchestra.run.smithi094.stdout:3039 write 0x7e02bc9 thru 0x7e08cb7 (0x60ef bytes) 2024-09-14T06:30:20.054 INFO:teuthology.orchestra.run.smithi094.stdout:3041 read 0x9bcccea thru 0x9bd88b8 (0xbbcf bytes) 2024-09-14T06:30:20.054 INFO:teuthology.orchestra.run.smithi094.stdout:3042 punch from 0x7950cf4 to 0x79603a9, (0xf6b5 bytes) 2024-09-14T06:30:20.054 INFO:teuthology.orchestra.run.smithi094.stdout:3043 writesame 0x767927a thru 0x768809e data_size 0x59(0xee25 bytes) 2024-09-14T06:30:20.060 INFO:teuthology.orchestra.run.smithi094.stdout:3044 writesame 0x51eceb0 thru 0x51f6e3b data_size 0x2(0x9f8c bytes) 2024-09-14T06:30:20.071 INFO:teuthology.orchestra.run.smithi094.stdout:3045 write 0x35838ab thru 0x358bb91 (0x82e7 bytes) 2024-09-14T06:30:20.075 INFO:teuthology.orchestra.run.smithi094.stdout:3048 punch from 0x8870975 to 0x887273b, (0x1dc6 bytes) 2024-09-14T06:30:20.075 INFO:teuthology.orchestra.run.smithi094.stdout:3049 writesame 0x9dbd537 thru 0x9dcbb27 data_size 0x3d(0xe5f1 bytes) 2024-09-14T06:30:20.080 INFO:teuthology.orchestra.run.smithi094.stdout:3050 read 0x5579410 thru 0x558529d (0xbe8e bytes) 2024-09-14T06:30:20.080 INFO:teuthology.orchestra.run.smithi094.stdout:3051 write 0x6bb9c35 thru 0x6bc913a (0xf506 bytes) 2024-09-14T06:30:20.087 INFO:teuthology.orchestra.run.smithi094.stdout:3052 write 0x6a7a3ff thru 0x6a83454 (0x9056 bytes) 2024-09-14T06:30:20.092 INFO:teuthology.orchestra.run.smithi094.stdout:3054 writesame 0x5b24916 thru 0x5b29fd5 data_size 0x8(0x56c0 bytes) 2024-09-14T06:30:20.096 INFO:teuthology.orchestra.run.smithi094.stdout:3055 read 0xa342c6f thru 0xa34be62 (0x91f4 bytes) 2024-09-14T06:30:20.096 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-14T06:30:20.096 INFO:teuthology.orchestra.run.smithi094.stdout:3059 writesame 0x17ff10e thru 0x180a176 data_size 0x1(0xb069 bytes) 2024-09-14T06:30:20.106 INFO:teuthology.orchestra.run.smithi094.stdout:3060 read 0x41f5b61 thru 0x41fbee7 (0x6387 bytes) 2024-09-14T06:30:20.106 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-14T06:30:20.114 INFO:teuthology.orchestra.run.smithi094.stdout:3062 writesame 0xe7f47b9 thru 0xe8002bf data_size 0x1d(0xbb07 bytes) 2024-09-14T06:30:20.114 INFO:teuthology.orchestra.run.smithi094.stdout:3063 write 0x8414a54 thru 0x8420ea7 (0xc454 bytes) 2024-09-14T06:30:20.116 INFO:tasks.ceph.osd.3.smithi094.stderr:2024-09-14T06:30:20.115+0000 7feeeafe8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33655) UID: 0 2024-09-14T06:30:20.121 INFO:teuthology.orchestra.run.smithi094.stdout:3065 write 0xa79de7d thru 0xa7a38d6 (0x5a5a bytes) 2024-09-14T06:30:20.130 INFO:teuthology.orchestra.run.smithi138.stdout:ERROR: (22) Invalid argument 2024-09-14T06:30:20.130 INFO:teuthology.orchestra.run.smithi138.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:30:20.133 INFO:teuthology.orchestra.run.smithi094.stdout:3066 trunc from 0xe8002c0 to 0x716e741 2024-09-14T06:30:20.139 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-14T06:30:20.157 INFO:teuthology.orchestra.run.smithi094.stdout:3067 write 0x88b6014 thru 0x88bdbac (0x7b99 bytes) 2024-09-14T06:30:20.167 INFO:teuthology.orchestra.run.smithi094.stdout:3068 write 0x45e9f40 thru 0x45f440c (0xa4cd bytes) 2024-09-14T06:30:20.173 INFO:teuthology.orchestra.run.smithi094.stdout:3069 read 0x3435819 thru 0x3437500 (0x1ce8 bytes) 2024-09-14T06:30:20.173 INFO:teuthology.orchestra.run.smithi094.stdout:3070 write 0x6ad6e9b thru 0x6adb449 (0x45af bytes) 2024-09-14T06:30:20.174 INFO:teuthology.orchestra.run.smithi094.stdout:3074 write 0x7fafc15 thru 0x7fbce62 (0xd24e bytes) 2024-09-14T06:30:20.181 INFO:teuthology.orchestra.run.smithi094.stdout:3075 write 0x72abc12 thru 0x72ba449 (0xe838 bytes) 2024-09-14T06:30:20.187 INFO:teuthology.orchestra.run.smithi094.stdout:3076 read 0x9f1926 thru 0xa012b3 (0xf98e bytes) 2024-09-14T06:30:20.188 INFO:teuthology.orchestra.run.smithi094.stdout:3079 punch from 0xc91fb9 to 0xc95787, (0x37ce bytes) 2024-09-14T06:30:20.188 INFO:teuthology.orchestra.run.smithi094.stdout:3083 read 0x5dceaf0 thru 0x5dd6137 (0x7648 bytes) 2024-09-14T06:30:20.188 INFO:teuthology.orchestra.run.smithi094.stdout:3084 read 0x1cfc77b thru 0x1d0a1c0 (0xda46 bytes) 2024-09-14T06:30:20.201 INFO:teuthology.orchestra.run.smithi094.stdout:3085 write 0xccb7e9d thru 0xccc336e (0xb4d2 bytes) 2024-09-14T06:30:20.218 INFO:tasks.ceph.osd.5.smithi138.stderr:2024-09-14T06:30:20.217+0000 7fe13746f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33618) UID: 0 2024-09-14T06:30:20.227 INFO:teuthology.orchestra.run.smithi094.stdout:3088 trunc from 0xccc336f to 0x742edc 2024-09-14T06:30:20.262 INFO:teuthology.orchestra.run.smithi138.stdout:ERROR: (22) Invalid argument 2024-09-14T06:30:20.262 INFO:teuthology.orchestra.run.smithi138.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:30:20.273 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-14T06:30:20.278 INFO:teuthology.orchestra.run.smithi094.stdout:3090 read 0x3e6a66 thru 0x3ef4ae (0x8a49 bytes) 2024-09-14T06:30:20.295 INFO:teuthology.orchestra.run.smithi094.stdout:3091 trunc from 0x742edc to 0xec34b93 2024-09-14T06:30:20.301 INFO:teuthology.orchestra.run.smithi094.stdout:3093 read 0x32cee9d thru 0x32d01ec (0x1350 bytes) 2024-09-14T06:30:20.302 INFO:teuthology.orchestra.run.smithi094.stdout:3094 read 0x400c8f8 thru 0x400d750 (0xe59 bytes) 2024-09-14T06:30:20.302 INFO:teuthology.orchestra.run.smithi094.stdout:3095 punch from 0x6ef60f1 to 0x6ef8bce, (0x2add bytes) 2024-09-14T06:30:20.302 INFO:teuthology.orchestra.run.smithi094.stdout:3096 clone 21 order 25 su 16777216 sc 5 2024-09-14T06:30:20.357 INFO:tasks.ceph.osd.7.smithi138.stderr:2024-09-14T06:30:20.356+0000 7f3c6a21d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33623) UID: 0 2024-09-14T06:30:20.416 INFO:teuthology.orchestra.run.smithi138.stdout:ERROR: (22) Invalid argument 2024-09-14T06:30:20.416 INFO:teuthology.orchestra.run.smithi138.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:30:20.419 INFO:tasks.ceph.osd.4.smithi138.stderr:2024-09-14T06:30:20.418+0000 7f6c653c8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33615) UID: 0 2024-09-14T06:30:20.425 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-14T06:30:20.520 INFO:tasks.ceph.osd.5.smithi138.stderr:2024-09-14T06:30:20.519+0000 7fe13746f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33618) UID: 0 2024-09-14T06:30:20.569 INFO:teuthology.orchestra.run.smithi138.stdout:ERROR: (22) Invalid argument 2024-09-14T06:30:20.569 INFO:teuthology.orchestra.run.smithi138.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:30:20.577 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-14T06:30:20.586 INFO:teuthology.orchestra.run.smithi094.stderr:reweighted osd.2 to 1 (10000) 2024-09-14T06:30:20.601 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 3 1 2024-09-14T06:30:20.614 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-14T06:30:20.661 INFO:tasks.ceph.osd.5.smithi138.stderr:2024-09-14T06:30:20.660+0000 7fe13746f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33618) UID: 0 2024-09-14T06:30:20.720 INFO:teuthology.orchestra.run.smithi138.stdout:ERROR: (22) Invalid argument 2024-09-14T06:30:20.720 INFO:teuthology.orchestra.run.smithi138.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:30:20.721 INFO:tasks.ceph.osd.2.smithi094.stderr:2024-09-14T06:30:20.720+0000 7faf3269b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33649) UID: 0 2024-09-14T06:30:20.728 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-14T06:30:20.779 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-14T06:30:20.823 INFO:tasks.ceph.osd.2.smithi094.stderr:2024-09-14T06:30:20.822+0000 7faf3269b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33649) UID: 0 2024-09-14T06:30:20.872 INFO:teuthology.orchestra.run.smithi138.stdout:ERROR: (22) Invalid argument 2024-09-14T06:30:20.872 INFO:teuthology.orchestra.run.smithi138.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:30:20.880 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-14T06:30:20.921 INFO:teuthology.orchestra.run.smithi094.stdout:osd.0: {} 2024-09-14T06:30:20.922 INFO:teuthology.orchestra.run.smithi094.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-14T06:30:20.923 INFO:tasks.ceph.osd.3.smithi094.stderr:2024-09-14T06:30:20.922+0000 7feeeafe8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33655) UID: 0 2024-09-14T06:30:20.960 INFO:teuthology.orchestra.run.smithi094.stdout:osd.1: {} 2024-09-14T06:30:20.960 INFO:teuthology.orchestra.run.smithi094.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-14T06:30:20.988 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-14T06:30:20.988 INFO:teuthology.orchestra.run.smithi094.stdout: "ops": [], 2024-09-14T06:30:20.988 INFO:teuthology.orchestra.run.smithi094.stdout: "num_ops": 0 2024-09-14T06:30:20.988 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-14T06:30:20.990 INFO:teuthology.orchestra.run.smithi094.stdout:osd.2: {} 2024-09-14T06:30:20.990 INFO:teuthology.orchestra.run.smithi094.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-14T06:30:20.997 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-14T06:30:21.006 INFO:teuthology.orchestra.run.smithi094.stdout:osd.3: {} 2024-09-14T06:30:21.007 INFO:teuthology.orchestra.run.smithi094.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-14T06:30:21.023 INFO:teuthology.orchestra.run.smithi094.stdout:osd.4: {} 2024-09-14T06:30:21.024 INFO:teuthology.orchestra.run.smithi094.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-14T06:30:21.024 INFO:tasks.ceph.osd.3.smithi094.stderr:2024-09-14T06:30:21.023+0000 7feeeafe8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33655) UID: 0 2024-09-14T06:30:21.041 INFO:teuthology.orchestra.run.smithi094.stdout:osd.5: {} 2024-09-14T06:30:21.041 INFO:teuthology.orchestra.run.smithi094.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-14T06:30:21.058 INFO:teuthology.orchestra.run.smithi094.stdout:osd.6: {} 2024-09-14T06:30:21.058 INFO:teuthology.orchestra.run.smithi094.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-14T06:30:21.074 INFO:teuthology.orchestra.run.smithi094.stdout:osd.7: {} 2024-09-14T06:30:21.074 INFO:teuthology.orchestra.run.smithi094.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-14T06:30:21.111 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-14T06:30:21.111 INFO:teuthology.orchestra.run.smithi094.stdout: "ops": [], 2024-09-14T06:30:21.112 INFO:teuthology.orchestra.run.smithi094.stdout: "complaint_time": 30, 2024-09-14T06:30:21.112 INFO:teuthology.orchestra.run.smithi094.stdout: "num_blocked_ops": 0 2024-09-14T06:30:21.112 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-14T06:30:21.120 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-14T06:30:21.125 INFO:tasks.ceph.osd.7.smithi138.stderr:2024-09-14T06:30:21.123+0000 7f3c6a21d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33623) UID: 0 2024-09-14T06:30:21.225 INFO:tasks.ceph.osd.5.smithi138.stderr:2024-09-14T06:30:21.224+0000 7fe13746f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33618) UID: 0 2024-09-14T06:30:21.264 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-14T06:30:21.264 INFO:teuthology.orchestra.run.smithi094.stdout: "size": 20, 2024-09-14T06:30:21.265 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 600, 2024-09-14T06:30:21.265 INFO:teuthology.orchestra.run.smithi094.stdout: "ops": [ 2024-09-14T06:30:21.265 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:21.265 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:115193 3.1 3:856be886:::rbd_data.112d4d24c558.0000000000000004:29 [read 1523712~16384 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e96)", 2024-09-14T06:30:21.265 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:30:17.939854+0000", 2024-09-14T06:30:21.265 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 3.3229416089999999, 2024-09-14T06:30:21.265 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.073897516999999996, 2024-09-14T06:30:21.265 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:30:21.266 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:30:21.266 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:30:21.266 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:30:21.266 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:30:21.266 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:21.266 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 115193 2024-09-14T06:30:21.266 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:21.266 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:30:21.266 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:21.266 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:30:21.266 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.939854+0000", 2024-09-14T06:30:21.266 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:21.267 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:21.267 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:21.267 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:30:21.267 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.939854+0000", 2024-09-14T06:30:21.267 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:21.267 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:21.267 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:21.267 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:30:21.267 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.939854+0000", 2024-09-14T06:30:21.267 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.9500000000000002e-07 2024-09-14T06:30:21.267 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:21.267 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:21.268 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:30:21.268 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.939856+0000", 2024-09-14T06:30:21.268 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.0959999999999999e-06 2024-09-14T06:30:21.268 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:21.268 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:21.268 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:30:21.268 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.939863+0000", 2024-09-14T06:30:21.268 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.1129999999999997e-06 2024-09-14T06:30:21.268 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:21.268 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:21.268 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:30:21.268 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.939867+0000", 2024-09-14T06:30:21.268 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.8350000000000003e-06 2024-09-14T06:30:21.269 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:21.269 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:21.269 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:30:21.269 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.013626+0000", 2024-09-14T06:30:21.269 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.073758799999999999 2024-09-14T06:30:21.269 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:21.269 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:21.269 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:30:21.269 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.013690+0000", 2024-09-14T06:30:21.269 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.3918999999999996e-05 2024-09-14T06:30:21.269 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:21.269 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:21.269 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:30:21.269 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.013751+0000", 2024-09-14T06:30:21.270 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.1159000000000002e-05 2024-09-14T06:30:21.270 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:21.270 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:21.270 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:21.270 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:21.270 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:21.270 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:115194 3.1 3:856be886:::rbd_data.112d4d24c558.0000000000000004:29 [read 1769472~16384 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e96)", 2024-09-14T06:30:21.270 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:30:17.939874+0000", 2024-09-14T06:30:21.270 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 3.3229212979999998, 2024-09-14T06:30:21.270 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.074016988000000006, 2024-09-14T06:30:21.270 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:30:21.270 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:30:21.270 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:30:21.270 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:30:21.271 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:30:21.271 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:21.271 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 115194 2024-09-14T06:30:21.271 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:21.271 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:30:21.271 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:21.271 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:30:21.271 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.939874+0000", 2024-09-14T06:30:21.271 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:21.271 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:21.271 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:21.271 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:30:21.271 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.939874+0000", 2024-09-14T06:30:21.272 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:21.272 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:21.272 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:21.272 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:30:21.272 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.939875+0000", 2024-09-14T06:30:21.272 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.2600000000000002e-07 2024-09-14T06:30:21.272 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:21.272 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:21.272 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:30:21.272 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.939877+0000", 2024-09-14T06:30:21.272 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.9420000000000002e-06 2024-09-14T06:30:21.272 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:21.272 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:21.272 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:30:21.272 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.939882+0000", 2024-09-14T06:30:21.273 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.8749999999999997e-06 2024-09-14T06:30:21.273 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:21.273 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:21.273 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:30:21.273 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.939887+0000", 2024-09-14T06:30:21.273 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.741e-06 2024-09-14T06:30:21.273 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:21.273 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:21.273 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:30:21.273 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.013767+0000", 2024-09-14T06:30:21.273 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.073879720999999995 2024-09-14T06:30:21.273 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:21.273 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:21.274 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:30:21.274 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.013832+0000", 2024-09-14T06:30:21.274 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.5312000000000001e-05 2024-09-14T06:30:21.274 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:21.274 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:21.274 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:30:21.274 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.013891+0000", 2024-09-14T06:30:21.274 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.8870999999999999e-05 2024-09-14T06:30:21.274 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:21.274 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:21.274 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:21.274 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:21.274 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:21.275 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:115195 3.1 3:856be886:::rbd_data.112d4d24c558.0000000000000004:29 [read 2015232~16384 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e96)", 2024-09-14T06:30:21.275 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:30:17.939903+0000", 2024-09-14T06:30:21.275 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 3.322892113, 2024-09-14T06:30:21.275 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.074128199000000006, 2024-09-14T06:30:21.275 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:30:21.275 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:30:21.275 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:30:21.275 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:30:21.275 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:30:21.275 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:21.275 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 115195 2024-09-14T06:30:21.275 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:21.275 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:30:21.275 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:21.275 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:30:21.276 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.939903+0000", 2024-09-14T06:30:21.276 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:21.276 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:21.276 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:21.276 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:30:21.276 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.939903+0000", 2024-09-14T06:30:21.276 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:21.276 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:21.276 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:21.276 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:30:21.276 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.939904+0000", 2024-09-14T06:30:21.276 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.5799999999999999e-07 2024-09-14T06:30:21.276 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:21.276 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:21.276 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:30:21.277 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.939906+0000", 2024-09-14T06:30:21.277 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.2350000000000002e-06 2024-09-14T06:30:21.277 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:21.277 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:21.277 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:30:21.277 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.939913+0000", 2024-09-14T06:30:21.277 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.5649999999999998e-06 2024-09-14T06:30:21.277 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:21.277 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:21.277 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:30:21.277 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.939918+0000", 2024-09-14T06:30:21.277 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.784e-06 2024-09-14T06:30:21.278 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:21.278 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:21.278 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:30:21.278 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.013921+0000", 2024-09-14T06:30:21.278 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.074002911000000005 2024-09-14T06:30:21.278 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:21.278 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:21.278 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:30:21.278 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.013971+0000", 2024-09-14T06:30:21.278 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.9551e-05 2024-09-14T06:30:21.278 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:21.278 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:21.278 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:30:21.278 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.014031+0000", 2024-09-14T06:30:21.278 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.0594999999999998e-05 2024-09-14T06:30:21.279 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:21.279 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:21.279 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:21.279 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:21.279 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:21.279 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:115196 3.1 3:856be886:::rbd_data.112d4d24c558.0000000000000004:29 [read 2260992~16384 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e96)", 2024-09-14T06:30:21.279 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:30:17.939925+0000", 2024-09-14T06:30:21.279 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 3.3228699210000001, 2024-09-14T06:30:21.279 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.074320137999999994, 2024-09-14T06:30:21.279 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:30:21.279 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:30:21.279 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:30:21.279 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:30:21.280 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:30:21.280 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:21.280 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 115196 2024-09-14T06:30:21.280 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:21.280 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:30:21.280 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:21.280 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:30:21.280 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.939925+0000", 2024-09-14T06:30:21.280 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:21.280 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:21.280 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:21.280 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:30:21.280 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.939925+0000", 2024-09-14T06:30:21.281 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:21.281 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:21.281 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:21.281 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:30:21.281 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.939926+0000", 2024-09-14T06:30:21.281 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.9999999999999998e-07 2024-09-14T06:30:21.281 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:21.281 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:21.281 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:30:21.281 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.939928+0000", 2024-09-14T06:30:21.281 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.9549999999999999e-06 2024-09-14T06:30:21.281 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:21.282 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:21.282 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:30:21.282 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.939934+0000", 2024-09-14T06:30:21.282 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.127e-06 2024-09-14T06:30:21.282 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:21.282 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:21.282 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:30:21.282 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.939939+0000", 2024-09-14T06:30:21.282 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.775e-06 2024-09-14T06:30:21.282 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:21.282 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:21.282 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:30:21.282 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.014075+0000", 2024-09-14T06:30:21.282 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.074136598999999997 2024-09-14T06:30:21.283 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:21.283 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:21.283 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:30:21.283 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.014162+0000", 2024-09-14T06:30:21.283 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.6477999999999994e-05 2024-09-14T06:30:21.283 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:21.283 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:21.283 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:30:21.283 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.014246+0000", 2024-09-14T06:30:21.283 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.3703999999999997e-05 2024-09-14T06:30:21.283 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:21.283 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:21.284 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:21.284 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:21.284 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:21.284 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:115197 3.1 3:856be886:::rbd_data.112d4d24c558.0000000000000004:29 [read 2506752~16384 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e96)", 2024-09-14T06:30:21.284 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:30:17.939969+0000", 2024-09-14T06:30:21.284 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 3.3228264360000002, 2024-09-14T06:30:21.284 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.074528544000000002, 2024-09-14T06:30:21.284 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:30:21.284 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:30:21.284 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:30:21.284 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:30:21.284 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:30:21.284 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:21.284 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 115197 2024-09-14T06:30:21.285 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:21.285 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:30:21.285 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:21.285 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:30:21.285 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.939969+0000", 2024-09-14T06:30:21.285 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:21.285 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:21.285 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:21.285 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:30:21.285 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.939969+0000", 2024-09-14T06:30:21.285 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:21.285 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:21.285 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:21.286 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:30:21.286 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.939970+0000", 2024-09-14T06:30:21.286 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.3600000000000003e-07 2024-09-14T06:30:21.286 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:21.286 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:21.286 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:30:21.286 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.939972+0000", 2024-09-14T06:30:21.286 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.3180000000000001e-06 2024-09-14T06:30:21.286 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:21.286 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:21.286 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:30:21.286 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.939979+0000", 2024-09-14T06:30:21.286 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.8800000000000002e-06 2024-09-14T06:30:21.286 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:21.286 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:21.287 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:30:21.287 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.939984+0000", 2024-09-14T06:30:21.287 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.3510000000000001e-06 2024-09-14T06:30:21.287 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:21.287 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:21.287 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:30:21.287 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.014302+0000", 2024-09-14T06:30:21.287 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.074317416999999997 2024-09-14T06:30:21.287 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:21.287 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:21.287 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:30:21.287 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.014392+0000", 2024-09-14T06:30:21.287 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.0181999999999998e-05 2024-09-14T06:30:21.287 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:21.287 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:21.288 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:30:21.288 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.014497+0000", 2024-09-14T06:30:21.288 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00010576 2024-09-14T06:30:21.288 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:21.288 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:21.288 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:21.288 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:21.288 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:21.288 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:115198 3.1 3:856be886:::rbd_data.112d4d24c558.0000000000000004:29 [read 2752512~16384 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e96)", 2024-09-14T06:30:21.288 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:30:17.940004+0000", 2024-09-14T06:30:21.288 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 3.3227911560000001, 2024-09-14T06:30:21.288 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.074736367999999997, 2024-09-14T06:30:21.288 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:30:21.288 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:30:21.288 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:30:21.289 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:30:21.289 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:30:21.289 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:21.289 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 115198 2024-09-14T06:30:21.289 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:21.289 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:30:21.289 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:21.289 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:30:21.289 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940004+0000", 2024-09-14T06:30:21.289 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:21.289 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:21.289 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:21.289 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:30:21.290 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940004+0000", 2024-09-14T06:30:21.290 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:21.290 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:21.290 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:21.290 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:30:21.290 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940005+0000", 2024-09-14T06:30:21.290 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.2200000000000004e-07 2024-09-14T06:30:21.290 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:21.290 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:21.290 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:30:21.290 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940007+0000", 2024-09-14T06:30:21.290 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.322e-06 2024-09-14T06:30:21.290 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:21.290 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:21.291 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:30:21.291 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940014+0000", 2024-09-14T06:30:21.291 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.6220000000000003e-06 2024-09-14T06:30:21.291 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:21.291 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:21.291 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:30:21.291 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940019+0000", 2024-09-14T06:30:21.291 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.6380000000000001e-06 2024-09-14T06:30:21.291 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:21.291 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:21.291 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:30:21.291 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.014562+0000", 2024-09-14T06:30:21.291 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.074542320999999995 2024-09-14T06:30:21.291 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:21.292 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:21.292 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:30:21.292 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.014646+0000", 2024-09-14T06:30:21.292 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.4079999999999995e-05 2024-09-14T06:30:21.292 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:21.292 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:21.292 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:30:21.292 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.014741+0000", 2024-09-14T06:30:21.292 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.4763000000000001e-05 2024-09-14T06:30:21.292 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:21.292 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:21.292 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:21.292 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:21.292 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:21.292 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:115199 3.1 3:856be886:::rbd_data.112d4d24c558.0000000000000004:29 [read 2998272~16384 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e96)", 2024-09-14T06:30:21.293 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:30:17.940040+0000", 2024-09-14T06:30:21.293 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 3.3227554010000002, 2024-09-14T06:30:21.293 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.074957134999999994, 2024-09-14T06:30:21.293 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:30:21.293 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:30:21.293 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:30:21.293 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:30:21.293 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:30:21.293 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:21.293 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 115199 2024-09-14T06:30:21.293 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:21.293 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:30:21.293 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:21.293 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:30:21.294 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940040+0000", 2024-09-14T06:30:21.294 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:21.294 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:21.294 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:21.294 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:30:21.294 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940040+0000", 2024-09-14T06:30:21.294 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:21.294 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:21.294 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:21.294 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:30:21.294 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940041+0000", 2024-09-14T06:30:21.294 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.4000000000000001e-07 2024-09-14T06:30:21.294 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:21.294 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:21.295 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:30:21.295 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940046+0000", 2024-09-14T06:30:21.295 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.7250000000000002e-06 2024-09-14T06:30:21.295 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:21.295 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:21.295 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:30:21.295 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940053+0000", 2024-09-14T06:30:21.295 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.6109999999999999e-06 2024-09-14T06:30:21.295 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:21.295 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:21.295 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:30:21.295 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940058+0000", 2024-09-14T06:30:21.295 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.1590000000000001e-06 2024-09-14T06:30:21.295 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:21.296 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:21.296 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:30:21.296 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.014768+0000", 2024-09-14T06:30:21.296 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.074709624000000002 2024-09-14T06:30:21.296 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:21.296 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:21.296 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:30:21.296 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.014907+0000", 2024-09-14T06:30:21.296 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000139665 2024-09-14T06:30:21.296 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:21.296 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:21.296 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:30:21.296 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.014997+0000", 2024-09-14T06:30:21.297 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.9710999999999995e-05 2024-09-14T06:30:21.297 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:21.297 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:21.297 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:21.297 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:21.297 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:21.297 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:115200 3.1 3:856be886:::rbd_data.112d4d24c558.0000000000000004:29 [read 3244032~16384 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e96)", 2024-09-14T06:30:21.297 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:30:17.940080+0000", 2024-09-14T06:30:21.297 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 3.3227151959999999, 2024-09-14T06:30:21.297 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.075137291999999994, 2024-09-14T06:30:21.297 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:30:21.297 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:30:21.297 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:30:21.298 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:30:21.298 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:30:21.298 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:21.298 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 115200 2024-09-14T06:30:21.298 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:21.298 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:30:21.298 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:21.298 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:30:21.298 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940080+0000", 2024-09-14T06:30:21.298 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:21.298 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:21.298 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:21.299 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:30:21.299 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940080+0000", 2024-09-14T06:30:21.299 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:21.299 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:21.299 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:21.299 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:30:21.299 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940081+0000", 2024-09-14T06:30:21.299 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.0800000000000004e-07 2024-09-14T06:30:21.299 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:21.299 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:21.299 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:30:21.299 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940083+0000", 2024-09-14T06:30:21.300 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.3920000000000001e-06 2024-09-14T06:30:21.300 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:21.300 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:21.300 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:30:21.300 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940091+0000", 2024-09-14T06:30:21.300 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.1389999999999995e-06 2024-09-14T06:30:21.300 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:21.300 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:21.300 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:30:21.300 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940097+0000", 2024-09-14T06:30:21.300 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.5550000000000002e-06 2024-09-14T06:30:21.300 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:21.301 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:21.301 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:30:21.301 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.015025+0000", 2024-09-14T06:30:21.301 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.074928579999999995 2024-09-14T06:30:21.301 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:21.301 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:21.301 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:30:21.301 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.015096+0000", 2024-09-14T06:30:21.301 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.0705999999999997e-05 2024-09-14T06:30:21.301 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:21.301 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:21.301 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:30:21.302 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.015217+0000", 2024-09-14T06:30:21.302 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00012131200000000001 2024-09-14T06:30:21.302 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:21.302 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:21.302 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:21.302 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:21.302 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:21.302 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:115201 3.1 3:856be886:::rbd_data.112d4d24c558.0000000000000004:29 [read 3489792~16384 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e96)", 2024-09-14T06:30:21.302 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:30:17.940113+0000", 2024-09-14T06:30:21.302 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 3.3226824229999998, 2024-09-14T06:30:21.302 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.075246899000000006, 2024-09-14T06:30:21.302 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:30:21.302 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:30:21.302 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:30:21.303 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:30:21.303 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:30:21.303 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:21.303 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 115201 2024-09-14T06:30:21.303 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:21.303 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:30:21.303 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:21.303 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:30:21.303 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940113+0000", 2024-09-14T06:30:21.303 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:21.303 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:21.303 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:21.303 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:30:21.303 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940113+0000", 2024-09-14T06:30:21.304 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:21.304 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:21.304 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:21.304 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:30:21.304 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940114+0000", 2024-09-14T06:30:21.304 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.1500000000000004e-07 2024-09-14T06:30:21.304 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:21.304 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:21.304 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:30:21.304 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940116+0000", 2024-09-14T06:30:21.304 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.2809999999999998e-06 2024-09-14T06:30:21.304 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:21.304 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:21.304 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:30:21.305 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940123+0000", 2024-09-14T06:30:21.305 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.9229999999999996e-06 2024-09-14T06:30:21.305 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:21.305 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:21.305 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:30:21.305 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940128+0000", 2024-09-14T06:30:21.305 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.7790000000000003e-06 2024-09-14T06:30:21.305 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:21.305 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:21.305 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:30:21.306 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.015253+0000", 2024-09-14T06:30:21.306 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.075125792999999996 2024-09-14T06:30:21.306 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:21.306 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:21.306 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:30:21.306 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.015297+0000", 2024-09-14T06:30:21.306 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.3847000000000002e-05 2024-09-14T06:30:21.306 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:21.306 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:21.306 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:30:21.306 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.015360+0000", 2024-09-14T06:30:21.306 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.2661000000000002e-05 2024-09-14T06:30:21.307 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:21.307 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:21.307 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:21.307 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:21.307 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:21.307 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:115202 3.1 3:856be886:::rbd_data.112d4d24c558.0000000000000004:29 [read 3735552~16384 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e96)", 2024-09-14T06:30:21.307 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:30:17.940135+0000", 2024-09-14T06:30:21.307 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 3.3226606439999999, 2024-09-14T06:30:21.307 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.075451105000000004, 2024-09-14T06:30:21.307 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:30:21.307 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:30:21.307 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:30:21.307 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:30:21.307 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:30:21.308 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:21.308 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 115202 2024-09-14T06:30:21.308 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:21.308 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:30:21.308 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:21.308 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:30:21.308 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940135+0000", 2024-09-14T06:30:21.308 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:21.308 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:21.308 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:21.308 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:30:21.308 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940135+0000", 2024-09-14T06:30:21.308 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:21.308 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:21.309 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:21.309 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:30:21.309 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940135+0000", 2024-09-14T06:30:21.309 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.0999999999999999e-07 2024-09-14T06:30:21.309 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:21.309 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:21.309 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:30:21.309 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940137+0000", 2024-09-14T06:30:21.309 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.0020000000000001e-06 2024-09-14T06:30:21.309 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:21.309 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:21.309 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:30:21.309 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940143+0000", 2024-09-14T06:30:21.309 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.9850000000000003e-06 2024-09-14T06:30:21.310 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:21.310 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:21.310 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:30:21.310 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940148+0000", 2024-09-14T06:30:21.310 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.9880000000000004e-06 2024-09-14T06:30:21.310 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:21.310 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:21.310 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:30:21.310 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.015400+0000", 2024-09-14T06:30:21.310 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.075252186999999998 2024-09-14T06:30:21.310 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:21.310 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:21.310 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:30:21.311 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.015481+0000", 2024-09-14T06:30:21.311 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.0698000000000006e-05 2024-09-14T06:30:21.311 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:21.311 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:21.311 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:30:21.311 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.015586+0000", 2024-09-14T06:30:21.311 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00010483499999999999 2024-09-14T06:30:21.311 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:21.311 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:21.311 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:21.311 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:21.311 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:21.311 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:115203 3.1 3:856be886:::rbd_data.112d4d24c558.0000000000000004:29 [read 3981312~16384 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e96)", 2024-09-14T06:30:21.311 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:30:17.940156+0000", 2024-09-14T06:30:21.312 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 3.3226394190000001, 2024-09-14T06:30:21.312 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.075620860999999998, 2024-09-14T06:30:21.312 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:30:21.312 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:30:21.312 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:30:21.312 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:30:21.312 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:30:21.312 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:21.312 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 115203 2024-09-14T06:30:21.312 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:21.312 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:30:21.312 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:21.312 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:30:21.312 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940156+0000", 2024-09-14T06:30:21.313 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:21.313 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:21.313 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:21.313 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:30:21.313 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940156+0000", 2024-09-14T06:30:21.313 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:21.313 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:21.313 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:21.313 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:30:21.313 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940156+0000", 2024-09-14T06:30:21.313 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.1500000000000005e-07 2024-09-14T06:30:21.313 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:21.313 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:21.314 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:30:21.314 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940158+0000", 2024-09-14T06:30:21.314 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.967e-06 2024-09-14T06:30:21.314 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:21.314 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:21.314 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:30:21.314 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940165+0000", 2024-09-14T06:30:21.314 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.6939999999999998e-06 2024-09-14T06:30:21.314 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:21.314 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:21.314 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:30:21.315 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940170+0000", 2024-09-14T06:30:21.315 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.7539999999999997e-06 2024-09-14T06:30:21.315 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:21.315 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:21.315 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:30:21.315 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.015633+0000", 2024-09-14T06:30:21.315 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.075463334000000007 2024-09-14T06:30:21.315 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:21.315 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:21.315 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:30:21.315 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.015693+0000", 2024-09-14T06:30:21.315 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.9367000000000003e-05 2024-09-14T06:30:21.315 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:21.315 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:21.315 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:30:21.316 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.015777+0000", 2024-09-14T06:30:21.316 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.4229999999999998e-05 2024-09-14T06:30:21.316 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:21.316 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:21.316 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:21.316 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:21.316 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:21.316 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:115204 3.1 3:856be886:::rbd_data.112d4d24c558.0000000000000004:29 [read 4227072~16384 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e96)", 2024-09-14T06:30:21.316 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:30:17.940176+0000", 2024-09-14T06:30:21.316 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 3.3226188859999999, 2024-09-14T06:30:21.316 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.075785505000000003, 2024-09-14T06:30:21.316 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:30:21.316 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:30:21.316 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:30:21.316 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:30:21.317 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:30:21.317 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:21.317 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 115204 2024-09-14T06:30:21.317 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:21.317 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:30:21.317 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:21.317 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:30:21.317 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940176+0000", 2024-09-14T06:30:21.317 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:21.317 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:21.317 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:21.317 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:30:21.317 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940176+0000", 2024-09-14T06:30:21.317 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:21.317 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:21.317 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:21.318 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:30:21.318 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940177+0000", 2024-09-14T06:30:21.318 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.7899999999999999e-07 2024-09-14T06:30:21.318 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:21.318 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:21.318 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:30:21.318 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940179+0000", 2024-09-14T06:30:21.318 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.103e-06 2024-09-14T06:30:21.318 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:21.318 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:21.318 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:30:21.318 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940185+0000", 2024-09-14T06:30:21.318 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.7239999999999997e-06 2024-09-14T06:30:21.318 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:21.318 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:21.319 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:30:21.319 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940190+0000", 2024-09-14T06:30:21.319 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.7269999999999998e-06 2024-09-14T06:30:21.319 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:21.319 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:21.319 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:30:21.319 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.015814+0000", 2024-09-14T06:30:21.319 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.075624114000000006 2024-09-14T06:30:21.319 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:21.319 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:21.319 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:30:21.319 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.015862+0000", 2024-09-14T06:30:21.319 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.8770999999999998e-05 2024-09-14T06:30:21.319 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:21.319 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:21.319 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:30:21.320 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.015962+0000", 2024-09-14T06:30:21.320 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.9586999999999994e-05 2024-09-14T06:30:21.320 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:21.320 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:21.320 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:21.320 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:21.320 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:21.320 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:115205 3.1 3:856be886:::rbd_data.112d4d24c558.0000000000000004:29 [read 4472832~16384 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e96)", 2024-09-14T06:30:21.320 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:30:17.940197+0000", 2024-09-14T06:30:21.320 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 3.3225981770000002, 2024-09-14T06:30:21.320 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.076068920999999998, 2024-09-14T06:30:21.320 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:30:21.320 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:30:21.321 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:30:21.321 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:30:21.321 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:30:21.321 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:21.321 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 115205 2024-09-14T06:30:21.321 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:21.321 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:30:21.321 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:21.321 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:30:21.321 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940197+0000", 2024-09-14T06:30:21.321 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:21.321 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:21.322 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:21.322 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:30:21.322 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940197+0000", 2024-09-14T06:30:21.322 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:21.322 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:21.322 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:21.322 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:30:21.322 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940198+0000", 2024-09-14T06:30:21.322 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.5999999999999999e-07 2024-09-14T06:30:21.322 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:21.322 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:21.322 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:30:21.322 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940200+0000", 2024-09-14T06:30:21.323 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.0899999999999999e-06 2024-09-14T06:30:21.323 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:21.323 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:21.323 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:30:21.323 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940206+0000", 2024-09-14T06:30:21.323 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.8710000000000002e-06 2024-09-14T06:30:21.323 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:21.323 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:21.323 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:30:21.323 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940211+0000", 2024-09-14T06:30:21.323 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.9230000000000001e-06 2024-09-14T06:30:21.323 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:21.323 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:21.324 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:30:21.324 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.016059+0000", 2024-09-14T06:30:21.324 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.075848051999999999 2024-09-14T06:30:21.324 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:21.324 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:21.324 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:30:21.324 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.016151+0000", 2024-09-14T06:30:21.324 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.1982000000000001e-05 2024-09-14T06:30:21.324 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:21.324 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:21.324 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:30:21.324 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.016266+0000", 2024-09-14T06:30:21.324 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000115543 2024-09-14T06:30:21.324 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:21.325 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:21.325 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:21.325 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:21.325 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:21.325 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:115206 3.1 3:856be886:::rbd_data.112d4d24c558.0000000000000004:29 [read 4718592~16384 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e96)", 2024-09-14T06:30:21.325 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:30:17.940223+0000", 2024-09-14T06:30:21.325 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 3.3225720870000002, 2024-09-14T06:30:21.325 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.076265115999999994, 2024-09-14T06:30:21.325 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:30:21.325 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:30:21.325 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:30:21.325 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:30:21.325 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:30:21.326 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:21.326 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 115206 2024-09-14T06:30:21.326 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:21.326 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:30:21.326 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:21.326 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:30:21.326 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940223+0000", 2024-09-14T06:30:21.326 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:21.326 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:21.326 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:21.326 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:30:21.326 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940223+0000", 2024-09-14T06:30:21.326 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:21.326 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:21.327 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:21.327 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:30:21.327 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940224+0000", 2024-09-14T06:30:21.327 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.0399999999999996e-07 2024-09-14T06:30:21.327 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:21.327 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:21.327 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:30:21.327 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940226+0000", 2024-09-14T06:30:21.327 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.1780000000000002e-06 2024-09-14T06:30:21.327 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:21.327 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:21.327 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:30:21.327 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940232+0000", 2024-09-14T06:30:21.327 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.5010000000000001e-06 2024-09-14T06:30:21.327 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:21.328 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:21.328 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:30:21.328 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940237+0000", 2024-09-14T06:30:21.328 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.0250000000000002e-06 2024-09-14T06:30:21.328 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:21.328 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:21.328 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:30:21.328 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.016305+0000", 2024-09-14T06:30:21.328 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.076067262999999996 2024-09-14T06:30:21.328 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:21.328 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:21.328 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:30:21.328 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.016383+0000", 2024-09-14T06:30:21.328 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.8578000000000006e-05 2024-09-14T06:30:21.329 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:21.329 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:21.329 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:30:21.329 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.016488+0000", 2024-09-14T06:30:21.329 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000105067 2024-09-14T06:30:21.329 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:21.329 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:21.329 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:21.329 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:21.329 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:21.329 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:115207 3.1 3:856be886:::rbd_data.112d4d24c558.0000000000000004:29 [read 4964352~16384 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e96)", 2024-09-14T06:30:21.329 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:30:17.940244+0000", 2024-09-14T06:30:21.329 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 3.3225509710000001, 2024-09-14T06:30:21.329 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.076371154999999996, 2024-09-14T06:30:21.330 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:30:21.330 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:30:21.330 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:30:21.330 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:30:21.330 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:30:21.330 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:21.330 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 115207 2024-09-14T06:30:21.330 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:21.330 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:30:21.330 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:21.330 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:30:21.330 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940244+0000", 2024-09-14T06:30:21.330 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:21.330 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:21.331 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:21.331 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:30:21.331 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940244+0000", 2024-09-14T06:30:21.331 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:21.331 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:21.331 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:21.331 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:30:21.331 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940245+0000", 2024-09-14T06:30:21.331 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.1200000000000003e-07 2024-09-14T06:30:21.331 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:21.331 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:21.331 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:30:21.331 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940247+0000", 2024-09-14T06:30:21.331 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.0030000000000001e-06 2024-09-14T06:30:21.332 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:21.332 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:21.332 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:30:21.332 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940252+0000", 2024-09-14T06:30:21.332 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.5520000000000004e-06 2024-09-14T06:30:21.332 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:21.332 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:21.332 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:30:21.332 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940257+0000", 2024-09-14T06:30:21.332 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.6759999999999998e-06 2024-09-14T06:30:21.332 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:21.332 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:21.332 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:30:21.332 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.016521+0000", 2024-09-14T06:30:21.332 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.076263499999999998 2024-09-14T06:30:21.333 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:21.333 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:21.333 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:30:21.333 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.016562+0000", 2024-09-14T06:30:21.333 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.0967000000000003e-05 2024-09-14T06:30:21.333 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:21.333 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:21.333 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:30:21.333 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.016616+0000", 2024-09-14T06:30:21.333 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.3944999999999999e-05 2024-09-14T06:30:21.333 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:21.333 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:21.333 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:21.333 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:21.333 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:21.334 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:115208 3.1 3:856be886:::rbd_data.112d4d24c558.0000000000000004:29 [read 5210112~16384 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e96)", 2024-09-14T06:30:21.334 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:30:17.940264+0000", 2024-09-14T06:30:21.334 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 3.3225311140000002, 2024-09-14T06:30:21.334 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.076558288000000002, 2024-09-14T06:30:21.334 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:30:21.334 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:30:21.334 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:30:21.334 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:30:21.334 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:30:21.334 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:21.334 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 115208 2024-09-14T06:30:21.334 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:21.334 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:30:21.334 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:21.334 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:30:21.334 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940264+0000", 2024-09-14T06:30:21.335 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:21.335 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:21.335 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:21.335 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:30:21.335 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940264+0000", 2024-09-14T06:30:21.335 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:21.335 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:21.335 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:21.335 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:30:21.335 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940265+0000", 2024-09-14T06:30:21.335 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.0500000000000004e-07 2024-09-14T06:30:21.335 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:21.335 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:21.335 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:30:21.335 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940267+0000", 2024-09-14T06:30:21.336 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.9810000000000002e-06 2024-09-14T06:30:21.336 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:21.336 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:21.336 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:30:21.336 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940272+0000", 2024-09-14T06:30:21.336 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.6999999999999996e-06 2024-09-14T06:30:21.336 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:21.336 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:21.336 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:30:21.336 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940277+0000", 2024-09-14T06:30:21.336 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.6480000000000002e-06 2024-09-14T06:30:21.336 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:21.336 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:21.336 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:30:21.336 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.016650+0000", 2024-09-14T06:30:21.337 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.076373089000000005 2024-09-14T06:30:21.337 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:21.337 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:21.337 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:30:21.337 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.016724+0000", 2024-09-14T06:30:21.337 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.4105999999999998e-05 2024-09-14T06:30:21.337 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:21.337 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:21.337 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:30:21.337 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.016823+0000", 2024-09-14T06:30:21.337 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.8258999999999996e-05 2024-09-14T06:30:21.337 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:21.337 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:21.337 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:21.337 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:21.338 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:21.338 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:115209 3.1 3:856be886:::rbd_data.112d4d24c558.0000000000000004:29 [read 5455872~16384 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e96)", 2024-09-14T06:30:21.338 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:30:17.940283+0000", 2024-09-14T06:30:21.338 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 3.3225120879999999, 2024-09-14T06:30:21.338 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.076724029999999999, 2024-09-14T06:30:21.338 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:30:21.338 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:30:21.338 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:30:21.338 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:30:21.338 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:30:21.338 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:21.338 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 115209 2024-09-14T06:30:21.338 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:21.338 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:30:21.338 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:21.339 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:30:21.339 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940283+0000", 2024-09-14T06:30:21.339 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:21.339 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:21.339 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:21.339 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:30:21.339 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940283+0000", 2024-09-14T06:30:21.339 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:21.339 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:21.339 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:21.339 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:30:21.339 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940284+0000", 2024-09-14T06:30:21.339 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.6700000000000003e-07 2024-09-14T06:30:21.339 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:21.339 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:21.340 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:30:21.340 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940286+0000", 2024-09-14T06:30:21.340 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.9520000000000001e-06 2024-09-14T06:30:21.340 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:21.340 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:21.340 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:30:21.340 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940292+0000", 2024-09-14T06:30:21.340 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.7339999999999996e-06 2024-09-14T06:30:21.340 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:21.340 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:21.340 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:30:21.340 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940296+0000", 2024-09-14T06:30:21.340 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.7700000000000001e-06 2024-09-14T06:30:21.341 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:21.341 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:21.341 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:30:21.341 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.016859+0000", 2024-09-14T06:30:21.341 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.07656317 2024-09-14T06:30:21.341 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:21.341 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:21.341 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:30:21.341 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.016929+0000", 2024-09-14T06:30:21.341 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.9679999999999997e-05 2024-09-14T06:30:21.341 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:21.341 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:21.341 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:30:21.341 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.017007+0000", 2024-09-14T06:30:21.342 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.8157000000000004e-05 2024-09-14T06:30:21.342 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:21.342 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:21.342 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:21.342 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:21.342 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:21.342 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:115210 3.1 3:856be886:::rbd_data.112d4d24c558.0000000000000004:29 [read 5701632~16384 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e96)", 2024-09-14T06:30:21.342 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:30:17.940303+0000", 2024-09-14T06:30:21.342 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 3.3224925710000002, 2024-09-14T06:30:21.342 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.076821358000000006, 2024-09-14T06:30:21.342 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:30:21.342 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:30:21.342 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:30:21.342 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:30:21.342 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:30:21.342 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:21.343 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 115210 2024-09-14T06:30:21.343 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:21.343 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:30:21.343 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:21.343 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:30:21.343 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940303+0000", 2024-09-14T06:30:21.343 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:21.343 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:21.343 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:21.343 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:30:21.343 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940303+0000", 2024-09-14T06:30:21.343 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:21.343 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:21.343 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:21.343 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:30:21.344 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940303+0000", 2024-09-14T06:30:21.344 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.3000000000000001e-07 2024-09-14T06:30:21.344 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:21.344 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:21.344 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:30:21.344 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940306+0000", 2024-09-14T06:30:21.344 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.255e-06 2024-09-14T06:30:21.344 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:21.344 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:21.344 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:30:21.344 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940312+0000", 2024-09-14T06:30:21.344 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.9710000000000001e-06 2024-09-14T06:30:21.344 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:21.344 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:21.344 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:30:21.345 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940316+0000", 2024-09-14T06:30:21.345 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.5340000000000001e-06 2024-09-14T06:30:21.345 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:21.345 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:21.345 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:30:21.345 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.017033+0000", 2024-09-14T06:30:21.345 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.076717169000000002 2024-09-14T06:30:21.345 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:21.345 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:21.345 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:30:21.345 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.017073+0000", 2024-09-14T06:30:21.345 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.9582000000000002e-05 2024-09-14T06:30:21.345 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:21.345 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:21.345 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:30:21.346 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.017124+0000", 2024-09-14T06:30:21.346 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.1316999999999997e-05 2024-09-14T06:30:21.346 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:21.346 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:21.346 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:21.346 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:21.346 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:21.346 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:115219 3.1 3:856be886:::rbd_data.112d4d24c558.0000000000000004:29 [read 65536~16384 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e96)", 2024-09-14T06:30:21.346 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:30:17.940323+0000", 2024-09-14T06:30:21.346 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 3.32247271, 2024-09-14T06:30:21.346 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.076915205, 2024-09-14T06:30:21.346 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:30:21.346 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:30:21.346 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:30:21.346 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:30:21.346 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:30:21.347 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:21.347 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 115219 2024-09-14T06:30:21.347 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:21.347 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:30:21.347 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:21.347 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:30:21.347 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940323+0000", 2024-09-14T06:30:21.347 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:21.347 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:21.347 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:21.347 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:30:21.347 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940323+0000", 2024-09-14T06:30:21.347 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:21.347 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:21.347 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:21.347 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:30:21.348 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940323+0000", 2024-09-14T06:30:21.348 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.0800000000000005e-07 2024-09-14T06:30:21.348 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:21.348 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:21.348 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:30:21.348 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940328+0000", 2024-09-14T06:30:21.348 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.887e-06 2024-09-14T06:30:21.348 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:21.348 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:21.348 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:30:21.348 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940334+0000", 2024-09-14T06:30:21.348 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.3269999999999997e-06 2024-09-14T06:30:21.348 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:21.348 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:21.348 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:30:21.349 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940339+0000", 2024-09-14T06:30:21.349 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.7960000000000003e-06 2024-09-14T06:30:21.349 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:21.349 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:21.349 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:30:21.349 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.017148+0000", 2024-09-14T06:30:21.349 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.076809125000000006 2024-09-14T06:30:21.349 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:21.349 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:21.349 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:30:21.349 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.017186+0000", 2024-09-14T06:30:21.349 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.7283000000000002e-05 2024-09-14T06:30:21.349 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:21.350 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:21.350 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:30:21.350 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.017238+0000", 2024-09-14T06:30:21.350 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.2278999999999999e-05 2024-09-14T06:30:21.350 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:21.350 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:21.350 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:21.350 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:21.350 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:21.350 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:115220 3.1 3:856be886:::rbd_data.112d4d24c558.0000000000000004:29 [read 311296~16384 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e96)", 2024-09-14T06:30:21.350 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:30:17.940346+0000", 2024-09-14T06:30:21.350 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 3.3224495809999999, 2024-09-14T06:30:21.350 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.077033293000000003, 2024-09-14T06:30:21.351 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:30:21.351 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:30:21.351 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:30:21.351 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:30:21.351 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:30:21.351 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:21.351 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 115220 2024-09-14T06:30:21.351 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:21.351 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:30:21.351 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:21.351 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:30:21.351 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940346+0000", 2024-09-14T06:30:21.351 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:21.352 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:21.352 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:21.352 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:30:21.352 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940346+0000", 2024-09-14T06:30:21.352 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:21.352 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:21.352 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:21.352 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:30:21.352 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940346+0000", 2024-09-14T06:30:21.352 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.4300000000000003e-07 2024-09-14T06:30:21.352 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:21.352 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:21.352 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:30:21.352 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940348+0000", 2024-09-14T06:30:21.352 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.0030000000000001e-06 2024-09-14T06:30:21.353 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:21.353 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:21.353 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:30:21.353 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.004655+0000", 2024-09-14T06:30:21.353 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.064306323999999998 2024-09-14T06:30:21.353 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:21.353 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:21.353 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:30:21.353 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.004671+0000", 2024-09-14T06:30:21.353 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.6589999999999999e-05 2024-09-14T06:30:21.353 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:21.353 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:21.353 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:30:21.353 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.017262+0000", 2024-09-14T06:30:21.354 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.012590570000000001 2024-09-14T06:30:21.354 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:21.354 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:21.354 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:30:21.354 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.017321+0000", 2024-09-14T06:30:21.354 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.9210999999999999e-05 2024-09-14T06:30:21.354 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:21.354 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:21.354 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:30:21.354 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.017379+0000", 2024-09-14T06:30:21.354 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.8051999999999999e-05 2024-09-14T06:30:21.354 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:21.354 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:21.354 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:21.354 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:21.355 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:21.355 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-14T06:30:21.356 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-14T06:30:21.356 INFO:tasks.ceph.osd.4.smithi138.stderr:2024-09-14T06:30:21.354+0000 7f6c653c8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33615) UID: 0 2024-09-14T06:30:21.457 INFO:tasks.ceph.osd.2.smithi094.stderr:2024-09-14T06:30:21.456+0000 7faf3269b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33649) UID: 0 2024-09-14T06:30:21.458 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-14T06:30:21.459 INFO:teuthology.orchestra.run.smithi094.stdout: "ops": [], 2024-09-14T06:30:21.459 INFO:teuthology.orchestra.run.smithi094.stdout: "num_ops": 0 2024-09-14T06:30:21.459 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-14T06:30:21.467 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-14T06:30:21.557 INFO:tasks.ceph.osd.4.smithi138.stderr:2024-09-14T06:30:21.556+0000 7f6c653c8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33615) UID: 0 2024-09-14T06:30:21.600 INFO:teuthology.orchestra.run.smithi094.stderr:nodeep-scrub is unset 2024-09-14T06:30:21.600 INFO:teuthology.orchestra.run.smithi094.stderr:reweighted osd.3 to 1 (10000) 2024-09-14T06:30:21.613 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-14T06:30:21.613 INFO:teuthology.orchestra.run.smithi094.stdout: "ops": [], 2024-09-14T06:30:21.613 INFO:teuthology.orchestra.run.smithi094.stdout: "complaint_time": 30, 2024-09-14T06:30:21.613 INFO:teuthology.orchestra.run.smithi094.stdout: "num_blocked_ops": 0 2024-09-14T06:30:21.614 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-14T06:30:21.618 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 4 1 2024-09-14T06:30:21.629 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-14T06:30:21.670 INFO:tasks.ceph.osd.3.smithi094.stderr:2024-09-14T06:30:21.669+0000 7feeeafe8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33655) UID: 0 2024-09-14T06:30:21.760 INFO:tasks.ceph.osd.7.smithi138.stderr:2024-09-14T06:30:21.759+0000 7f3c6a21d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33623) UID: 0 2024-09-14T06:30:21.778 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-14T06:30:21.778 INFO:teuthology.orchestra.run.smithi094.stdout: "size": 20, 2024-09-14T06:30:21.778 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 600, 2024-09-14T06:30:21.778 INFO:teuthology.orchestra.run.smithi094.stdout: "ops": [ 2024-09-14T06:30:21.778 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:21.778 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:14918 3.0 3:05c39535:::rbd_data.112d92027590.0000000000000009:f [read 795648~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:30:21.778 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:26.083834+0000", 2024-09-14T06:30:21.778 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 55.692500928000001, 2024-09-14T06:30:21.779 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.041776287000000002, 2024-09-14T06:30:21.779 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:30:21.779 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:30:21.779 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:30:21.779 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:30:21.779 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:30:21.779 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:21.779 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 14918 2024-09-14T06:30:21.779 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:21.779 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:30:21.779 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:21.779 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:30:21.779 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.083834+0000", 2024-09-14T06:30:21.779 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:21.780 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:21.780 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:21.780 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:30:21.780 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.083834+0000", 2024-09-14T06:30:21.780 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:21.780 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:21.780 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:21.780 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:30:21.780 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.083834+0000", 2024-09-14T06:30:21.780 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.0100000000000001e-07 2024-09-14T06:30:21.781 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:21.781 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:21.781 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:30:21.781 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.083835+0000", 2024-09-14T06:30:21.781 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.3900000000000003e-07 2024-09-14T06:30:21.781 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:21.781 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:21.781 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:30:21.781 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.083838+0000", 2024-09-14T06:30:21.781 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.9830000000000001e-06 2024-09-14T06:30:21.781 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:21.781 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:21.781 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:30:21.782 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.083840+0000", 2024-09-14T06:30:21.782 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.4679999999999999e-06 2024-09-14T06:30:21.782 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:21.782 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:21.782 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:30:21.782 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.125455+0000", 2024-09-14T06:30:21.782 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.041614578999999999 2024-09-14T06:30:21.782 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:21.782 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:21.782 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:30:21.782 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.125496+0000", 2024-09-14T06:30:21.782 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.1047000000000002e-05 2024-09-14T06:30:21.782 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:21.782 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:21.782 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:30:21.783 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.125610+0000", 2024-09-14T06:30:21.783 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00011407 2024-09-14T06:30:21.783 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:21.783 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:21.783 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:21.783 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:21.783 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:21.783 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:14919 3.0 3:05c39535:::rbd_data.112d92027590.0000000000000009:f [read 805888~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:30:21.783 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:26.083843+0000", 2024-09-14T06:30:21.783 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 55.692491179999998, 2024-09-14T06:30:21.783 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.041992356000000002, 2024-09-14T06:30:21.783 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:30:21.783 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:30:21.784 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:30:21.784 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:30:21.784 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:30:21.784 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:21.784 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 14919 2024-09-14T06:30:21.784 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:21.784 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:30:21.784 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:21.784 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:30:21.784 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.083843+0000", 2024-09-14T06:30:21.784 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:21.784 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:21.784 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:21.784 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:30:21.785 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.083843+0000", 2024-09-14T06:30:21.785 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:21.785 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:21.785 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:21.785 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:30:21.785 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.083844+0000", 2024-09-14T06:30:21.785 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.3300000000000001e-07 2024-09-14T06:30:21.785 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:21.785 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:21.785 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:30:21.785 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.083845+0000", 2024-09-14T06:30:21.785 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.4729999999999999e-06 2024-09-14T06:30:21.785 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:21.785 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:21.786 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:30:21.786 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.083850+0000", 2024-09-14T06:30:21.786 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.0130000000000001e-06 2024-09-14T06:30:21.786 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:21.786 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:21.786 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:30:21.786 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.083854+0000", 2024-09-14T06:30:21.786 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.4759999999999998e-06 2024-09-14T06:30:21.786 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:21.786 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:21.786 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:30:21.786 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.125636+0000", 2024-09-14T06:30:21.786 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.041782683000000001 2024-09-14T06:30:21.787 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:21.787 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:21.787 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:30:21.787 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.125680+0000", 2024-09-14T06:30:21.787 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.3614e-05 2024-09-14T06:30:21.787 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:21.787 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:21.787 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:30:21.787 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.125836+0000", 2024-09-14T06:30:21.787 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000155864 2024-09-14T06:30:21.787 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:21.787 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:21.787 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:21.787 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:21.788 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:21.788 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:14920 3.0 3:05c39535:::rbd_data.112d92027590.0000000000000009:f [read 816128~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:30:21.788 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:26.083868+0000", 2024-09-14T06:30:21.788 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 55.692466519, 2024-09-14T06:30:21.788 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.042166771999999998, 2024-09-14T06:30:21.788 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:30:21.788 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:30:21.788 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:30:21.788 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:30:21.788 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:30:21.788 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:21.788 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 14920 2024-09-14T06:30:21.788 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:21.789 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:30:21.789 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:21.789 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:30:21.789 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.083868+0000", 2024-09-14T06:30:21.789 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:21.789 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:21.789 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:21.789 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:30:21.789 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.083868+0000", 2024-09-14T06:30:21.789 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:21.789 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:21.790 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:21.790 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:30:21.790 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.083869+0000", 2024-09-14T06:30:21.790 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.99e-07 2024-09-14T06:30:21.790 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:21.790 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:21.790 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:30:21.790 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.083870+0000", 2024-09-14T06:30:21.790 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.782e-06 2024-09-14T06:30:21.790 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:21.791 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:21.791 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:30:21.791 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.083876+0000", 2024-09-14T06:30:21.791 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.3809999999999998e-06 2024-09-14T06:30:21.791 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:21.791 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:21.791 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:30:21.792 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.083882+0000", 2024-09-14T06:30:21.792 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.9479999999999996e-06 2024-09-14T06:30:21.792 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:21.792 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:21.792 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:30:21.792 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.125878+0000", 2024-09-14T06:30:21.792 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.041996264999999998 2024-09-14T06:30:21.792 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:21.792 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:21.792 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:30:21.792 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.125929+0000", 2024-09-14T06:30:21.792 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.0498999999999999e-05 2024-09-14T06:30:21.793 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:21.793 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:21.793 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:30:21.793 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.126035+0000", 2024-09-14T06:30:21.793 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000106298 2024-09-14T06:30:21.793 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:21.793 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:21.793 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:21.793 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:21.793 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:21.793 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:14921 3.0 3:05c39535:::rbd_data.112d92027590.0000000000000009:f [read 826368~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:30:21.793 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:26.083889+0000", 2024-09-14T06:30:21.794 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 55.692445509000002, 2024-09-14T06:30:21.794 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.042435309999999997, 2024-09-14T06:30:21.794 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:30:21.794 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:30:21.794 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:30:21.794 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:30:21.794 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:30:21.794 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:21.794 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 14921 2024-09-14T06:30:21.794 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:21.794 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:30:21.794 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:21.794 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:30:21.795 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.083889+0000", 2024-09-14T06:30:21.795 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:21.795 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:21.795 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:21.795 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:30:21.795 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.083889+0000", 2024-09-14T06:30:21.795 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:21.795 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:21.795 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:21.795 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:30:21.795 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.086033+0000", 2024-09-14T06:30:21.795 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0021444179999999999 2024-09-14T06:30:21.795 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:21.796 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:21.796 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:30:21.796 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.086038+0000", 2024-09-14T06:30:21.796 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.0969999999999999e-06 2024-09-14T06:30:21.796 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:21.796 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:21.796 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:30:21.796 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.086046+0000", 2024-09-14T06:30:21.796 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.9440000000000005e-06 2024-09-14T06:30:21.796 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:21.796 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:21.797 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:30:21.797 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.086050+0000", 2024-09-14T06:30:21.797 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.69e-06 2024-09-14T06:30:21.797 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:21.797 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:21.797 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:30:21.797 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.126080+0000", 2024-09-14T06:30:21.797 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.040030237000000003 2024-09-14T06:30:21.797 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:21.797 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:21.797 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:30:21.798 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.126163+0000", 2024-09-14T06:30:21.798 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.2071999999999996e-05 2024-09-14T06:30:21.798 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:21.798 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:21.798 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:30:21.798 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.126324+0000", 2024-09-14T06:30:21.798 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00016185200000000001 2024-09-14T06:30:21.798 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:21.798 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:21.798 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:21.798 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:21.798 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:21.798 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:55627 3.0 3:06490baf:::rbd_data.112dca2f6ae1.0000000000000030:17 [read 2002944~4096 [fadvise_nocache]] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:30:21.799 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:44.599966+0000", 2024-09-14T06:30:21.799 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 37.176368560999997, 2024-09-14T06:30:21.799 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.041868546999999999, 2024-09-14T06:30:21.799 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:30:21.799 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:30:21.799 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:30:21.799 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:30:21.799 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:30:21.799 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:21.799 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 55627 2024-09-14T06:30:21.799 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:21.799 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:30:21.799 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:21.799 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:30:21.799 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.599966+0000", 2024-09-14T06:30:21.800 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:21.800 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:21.800 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:21.800 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:30:21.800 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.599966+0000", 2024-09-14T06:30:21.800 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:21.800 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:21.800 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:21.800 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:30:21.800 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.599966+0000", 2024-09-14T06:30:21.800 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.0900000000000001e-07 2024-09-14T06:30:21.800 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:21.800 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:21.801 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:30:21.801 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.599967+0000", 2024-09-14T06:30:21.801 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.64e-07 2024-09-14T06:30:21.801 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:21.801 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:21.801 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:30:21.801 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.599970+0000", 2024-09-14T06:30:21.801 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.9670000000000001e-06 2024-09-14T06:30:21.801 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:21.801 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:21.801 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:30:21.801 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.599973+0000", 2024-09-14T06:30:21.801 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.632e-06 2024-09-14T06:30:21.802 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:21.802 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:21.803 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:30:21.803 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.641665+0000", 2024-09-14T06:30:21.803 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.041692547000000003 2024-09-14T06:30:21.803 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:21.803 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:21.803 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:30:21.803 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.641740+0000", 2024-09-14T06:30:21.803 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.4559000000000002e-05 2024-09-14T06:30:21.803 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:21.803 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:21.803 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:30:21.803 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.641835+0000", 2024-09-14T06:30:21.803 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.4668999999999998e-05 2024-09-14T06:30:21.804 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:21.804 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:21.804 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:21.804 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:21.804 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:21.804 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:55628 3.0 3:06490baf:::rbd_data.112dca2f6ae1.0000000000000030:17 [read 2027520~4096 [fadvise_nocache]] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:30:21.804 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:44.599976+0000", 2024-09-14T06:30:21.804 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 37.176358774000001, 2024-09-14T06:30:21.804 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.041991621, 2024-09-14T06:30:21.804 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:30:21.804 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:30:21.804 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:30:21.804 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:30:21.804 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:30:21.805 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:21.805 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 55628 2024-09-14T06:30:21.805 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:21.805 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:30:21.805 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:21.805 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:30:21.805 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.599976+0000", 2024-09-14T06:30:21.805 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:21.805 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:21.805 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:21.805 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:30:21.805 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.599976+0000", 2024-09-14T06:30:21.805 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:21.806 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:21.806 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:21.806 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:30:21.806 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.599976+0000", 2024-09-14T06:30:21.806 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.1299999999999999e-07 2024-09-14T06:30:21.806 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:21.806 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:21.806 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:30:21.806 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.599977+0000", 2024-09-14T06:30:21.806 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.7999999999999993e-07 2024-09-14T06:30:21.806 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:21.806 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:21.806 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:30:21.806 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.599980+0000", 2024-09-14T06:30:21.806 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.01e-06 2024-09-14T06:30:21.807 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:21.807 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:21.807 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:30:21.807 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.599983+0000", 2024-09-14T06:30:21.807 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.5409999999999999e-06 2024-09-14T06:30:21.807 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:21.807 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:21.807 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:30:21.807 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.641869+0000", 2024-09-14T06:30:21.807 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.041886235000000001 2024-09-14T06:30:21.807 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:21.807 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:21.807 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:30:21.807 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.641913+0000", 2024-09-14T06:30:21.808 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.4357000000000002e-05 2024-09-14T06:30:21.808 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:21.808 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:21.808 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:30:21.808 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.641967+0000", 2024-09-14T06:30:21.808 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.4284999999999999e-05 2024-09-14T06:30:21.808 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:21.808 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:21.808 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:21.808 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:21.808 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:21.808 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:55629 3.0 3:06490baf:::rbd_data.112dca2f6ae1.0000000000000030:17 [read 2052096~4096 [fadvise_nocache]] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:30:21.808 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:44.599985+0000", 2024-09-14T06:30:21.808 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 37.176349068, 2024-09-14T06:30:21.809 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.042183375000000002, 2024-09-14T06:30:21.809 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:30:21.809 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:30:21.809 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:30:21.809 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:30:21.809 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:30:21.809 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:21.809 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 55629 2024-09-14T06:30:21.809 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:21.809 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:30:21.809 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:21.809 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:30:21.809 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.599985+0000", 2024-09-14T06:30:21.810 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:21.810 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:21.810 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:21.810 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:30:21.810 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.599985+0000", 2024-09-14T06:30:21.810 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:21.810 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:21.810 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:21.810 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:30:21.810 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.599986+0000", 2024-09-14T06:30:21.810 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.05e-07 2024-09-14T06:30:21.810 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:21.810 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:21.810 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:30:21.811 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.599987+0000", 2024-09-14T06:30:21.811 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.8400000000000002e-07 2024-09-14T06:30:21.811 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:21.811 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:21.811 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:30:21.811 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.599990+0000", 2024-09-14T06:30:21.811 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.5640000000000001e-06 2024-09-14T06:30:21.811 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:21.811 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:21.811 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:30:21.811 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.599993+0000", 2024-09-14T06:30:21.811 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.615e-06 2024-09-14T06:30:21.811 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:21.811 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:21.812 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:30:21.812 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.642003+0000", 2024-09-14T06:30:21.812 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.042010345999999997 2024-09-14T06:30:21.812 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:21.812 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:21.812 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:30:21.812 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.642080+0000", 2024-09-14T06:30:21.812 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.6538999999999993e-05 2024-09-14T06:30:21.812 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:21.812 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:21.812 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:30:21.812 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.642169+0000", 2024-09-14T06:30:21.812 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.9122000000000005e-05 2024-09-14T06:30:21.812 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:21.813 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:21.813 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:21.813 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:21.813 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:21.813 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:55630 3.0 3:06490baf:::rbd_data.112dca2f6ae1.0000000000000030:17 [read 2076672~4096 [fadvise_nocache]] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:30:21.813 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:44.599996+0000", 2024-09-14T06:30:21.813 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 37.17633859, 2024-09-14T06:30:21.813 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.042299615999999998, 2024-09-14T06:30:21.813 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:30:21.813 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:30:21.813 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:30:21.813 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:30:21.813 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:30:21.814 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:21.814 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 55630 2024-09-14T06:30:21.814 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:21.814 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:30:21.814 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:21.814 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:30:21.814 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.599996+0000", 2024-09-14T06:30:21.814 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:21.814 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:21.814 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:21.814 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:30:21.814 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.599996+0000", 2024-09-14T06:30:21.814 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:21.815 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:21.815 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:21.815 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:30:21.815 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.599996+0000", 2024-09-14T06:30:21.815 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.1500000000000001e-07 2024-09-14T06:30:21.815 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:21.815 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:21.815 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:30:21.815 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.599997+0000", 2024-09-14T06:30:21.815 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.33e-07 2024-09-14T06:30:21.815 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:21.815 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:21.815 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:30:21.816 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600000+0000", 2024-09-14T06:30:21.816 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.0240000000000002e-06 2024-09-14T06:30:21.816 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:21.816 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:21.816 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:30:21.816 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600003+0000", 2024-09-14T06:30:21.816 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.3640000000000002e-06 2024-09-14T06:30:21.816 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:21.816 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:21.816 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:30:21.816 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.642200+0000", 2024-09-14T06:30:21.816 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.042197571000000003 2024-09-14T06:30:21.816 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:21.816 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:21.817 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:30:21.817 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.642243+0000", 2024-09-14T06:30:21.817 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.2765000000000001e-05 2024-09-14T06:30:21.817 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:21.817 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:21.817 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:30:21.817 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.642296+0000", 2024-09-14T06:30:21.817 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.2744000000000002e-05 2024-09-14T06:30:21.817 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:21.817 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:21.817 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:21.817 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:21.817 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:21.817 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:55631 3.0 3:06490baf:::rbd_data.112dca2f6ae1.0000000000000030:17 [read 1048576~4096 [fadvise_nocache]] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:30:21.818 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:44.600006+0000", 2024-09-14T06:30:21.818 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 37.176329002000003, 2024-09-14T06:30:21.818 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.042487905999999999, 2024-09-14T06:30:21.818 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:30:21.818 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:30:21.818 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:30:21.818 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:30:21.818 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:30:21.818 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:21.818 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 55631 2024-09-14T06:30:21.818 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:21.818 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:30:21.818 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:21.819 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:30:21.819 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600006+0000", 2024-09-14T06:30:21.819 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:21.819 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:21.819 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:21.819 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:30:21.819 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600006+0000", 2024-09-14T06:30:21.819 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:21.819 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:21.819 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:21.819 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:30:21.819 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600006+0000", 2024-09-14T06:30:21.819 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.0800000000000001e-07 2024-09-14T06:30:21.819 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:21.819 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:21.820 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:30:21.820 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600007+0000", 2024-09-14T06:30:21.820 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.7199999999999997e-07 2024-09-14T06:30:21.820 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:21.820 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:21.820 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:30:21.820 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600010+0000", 2024-09-14T06:30:21.820 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.0280000000000001e-06 2024-09-14T06:30:21.820 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:21.820 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:21.820 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:30:21.820 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600012+0000", 2024-09-14T06:30:21.821 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.475e-06 2024-09-14T06:30:21.821 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:21.821 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:21.821 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:30:21.821 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.642329+0000", 2024-09-14T06:30:21.821 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.042317157000000001 2024-09-14T06:30:21.821 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:21.821 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:21.821 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:30:21.821 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.642403+0000", 2024-09-14T06:30:21.821 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.3941000000000002e-05 2024-09-14T06:30:21.821 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:21.821 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:21.822 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:30:21.822 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.642493+0000", 2024-09-14T06:30:21.822 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.0124999999999995e-05 2024-09-14T06:30:21.822 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:21.822 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:21.822 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:21.822 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:21.822 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:21.822 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:55632 3.0 3:06490baf:::rbd_data.112dca2f6ae1.0000000000000030:17 [read 1073152~4096 [fadvise_nocache]] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:30:21.822 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:44.600015+0000", 2024-09-14T06:30:21.822 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 37.176319249000002, 2024-09-14T06:30:21.822 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.042604922000000003, 2024-09-14T06:30:21.822 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:30:21.823 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:30:21.823 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:30:21.823 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:30:21.823 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:30:21.823 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:21.823 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 55632 2024-09-14T06:30:21.823 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:21.823 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:30:21.823 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:21.823 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:30:21.823 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600015+0000", 2024-09-14T06:30:21.823 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:21.823 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:21.824 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:21.824 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:30:21.824 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600015+0000", 2024-09-14T06:30:21.824 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:21.824 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:21.824 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:21.824 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:30:21.824 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600016+0000", 2024-09-14T06:30:21.824 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.1e-07 2024-09-14T06:30:21.824 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:21.824 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:21.824 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:30:21.824 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600017+0000", 2024-09-14T06:30:21.825 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.9900000000000009e-07 2024-09-14T06:30:21.825 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:21.825 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:21.825 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:30:21.825 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600020+0000", 2024-09-14T06:30:21.825 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.4170000000000001e-06 2024-09-14T06:30:21.825 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:21.825 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:21.825 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:30:21.825 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600023+0000", 2024-09-14T06:30:21.825 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.588e-06 2024-09-14T06:30:21.825 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:21.825 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:21.826 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:30:21.826 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.642526+0000", 2024-09-14T06:30:21.826 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.042503236999999999 2024-09-14T06:30:21.826 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:21.826 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:21.826 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:30:21.826 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.642568+0000", 2024-09-14T06:30:21.826 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.1925000000000003e-05 2024-09-14T06:30:21.826 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:21.826 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:21.826 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:30:21.826 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.642620+0000", 2024-09-14T06:30:21.826 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.2546000000000001e-05 2024-09-14T06:30:21.827 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:21.827 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:21.827 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:21.827 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:21.827 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:21.827 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:55633 3.0 3:06490baf:::rbd_data.112dca2f6ae1.0000000000000030:17 [read 1097728~4096 [fadvise_nocache]] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:30:21.827 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:44.600026+0000", 2024-09-14T06:30:21.827 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 37.176308980000002, 2024-09-14T06:30:21.827 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.042797474000000002, 2024-09-14T06:30:21.827 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:30:21.827 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:30:21.827 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:30:21.827 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:30:21.828 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:30:21.828 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:21.828 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 55633 2024-09-14T06:30:21.828 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:21.828 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:30:21.828 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:21.828 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:30:21.828 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600026+0000", 2024-09-14T06:30:21.828 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:21.828 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:21.828 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:21.828 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:30:21.828 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600026+0000", 2024-09-14T06:30:21.829 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:21.829 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:21.829 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:21.829 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:30:21.829 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600026+0000", 2024-09-14T06:30:21.829 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.3099999999999999e-07 2024-09-14T06:30:21.829 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:21.829 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:21.829 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:30:21.829 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600029+0000", 2024-09-14T06:30:21.829 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.304e-06 2024-09-14T06:30:21.829 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:21.829 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:21.830 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:30:21.830 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600034+0000", 2024-09-14T06:30:21.830 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.1429999999999997e-06 2024-09-14T06:30:21.830 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:21.830 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:21.833 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:30:21.833 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600038+0000", 2024-09-14T06:30:21.833 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.692e-06 2024-09-14T06:30:21.833 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:21.833 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:21.833 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:30:21.833 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.642654+0000", 2024-09-14T06:30:21.834 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.042615564000000002 2024-09-14T06:30:21.834 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:21.834 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:21.834 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:30:21.834 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.642728+0000", 2024-09-14T06:30:21.834 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.4018999999999997e-05 2024-09-14T06:30:21.834 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:21.834 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:21.834 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:30:21.834 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.642823+0000", 2024-09-14T06:30:21.834 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.5520999999999996e-05 2024-09-14T06:30:21.834 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:21.834 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:21.835 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:21.835 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:21.835 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:21.835 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:55634 3.0 3:06490baf:::rbd_data.112dca2f6ae1.0000000000000030:17 [read 1122304~4096 [fadvise_nocache]] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:30:21.835 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:44.600042+0000", 2024-09-14T06:30:21.835 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 37.176292502999999, 2024-09-14T06:30:21.835 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.042903729000000002, 2024-09-14T06:30:21.835 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:30:21.835 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:30:21.835 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:30:21.835 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:30:21.835 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:30:21.835 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:21.835 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 55634 2024-09-14T06:30:21.836 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:21.836 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:30:21.836 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:21.836 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:30:21.836 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600042+0000", 2024-09-14T06:30:21.836 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:21.836 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:21.836 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:21.836 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:30:21.836 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600042+0000", 2024-09-14T06:30:21.836 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:21.836 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:21.837 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:21.837 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:30:21.837 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600043+0000", 2024-09-14T06:30:21.837 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.7000000000000005e-07 2024-09-14T06:30:21.837 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:21.837 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:21.837 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:30:21.837 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600044+0000", 2024-09-14T06:30:21.837 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.742e-06 2024-09-14T06:30:21.837 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:21.837 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:21.838 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:30:21.838 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600050+0000", 2024-09-14T06:30:21.838 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.0800000000000002e-06 2024-09-14T06:30:21.838 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:21.838 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:21.838 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:30:21.838 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600055+0000", 2024-09-14T06:30:21.838 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.3889999999999998e-06 2024-09-14T06:30:21.838 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:21.838 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:21.838 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:30:21.838 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.642845+0000", 2024-09-14T06:30:21.838 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.042790040000000001 2024-09-14T06:30:21.839 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:21.839 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:21.839 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:30:21.839 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.642888+0000", 2024-09-14T06:30:21.839 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.2790000000000002e-05 2024-09-14T06:30:21.839 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:21.839 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:21.839 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:30:21.839 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.642946+0000", 2024-09-14T06:30:21.839 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.8118000000000002e-05 2024-09-14T06:30:21.839 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:21.839 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:21.839 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:21.839 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:21.839 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:21.840 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:55635 3.0 3:06490baf:::rbd_data.112dca2f6ae1.0000000000000030:17 [read 1146880~4096 [fadvise_nocache]] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:30:21.840 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:44.600061+0000", 2024-09-14T06:30:21.840 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 37.176273932000001, 2024-09-14T06:30:21.840 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.043092695, 2024-09-14T06:30:21.840 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:30:21.840 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:30:21.840 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:30:21.840 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:30:21.840 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:30:21.840 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:21.840 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 55635 2024-09-14T06:30:21.840 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:21.840 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:30:21.841 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:21.841 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:30:21.841 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600061+0000", 2024-09-14T06:30:21.841 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:21.841 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:21.841 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:21.841 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:30:21.841 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600061+0000", 2024-09-14T06:30:21.841 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:21.841 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:21.841 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:21.841 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:30:21.841 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600061+0000", 2024-09-14T06:30:21.842 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.3099999999999999e-07 2024-09-14T06:30:21.842 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:21.842 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:21.842 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:30:21.842 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600063+0000", 2024-09-14T06:30:21.842 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.6220000000000001e-06 2024-09-14T06:30:21.842 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:21.842 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:21.842 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:30:21.842 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600068+0000", 2024-09-14T06:30:21.842 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.4790000000000004e-06 2024-09-14T06:30:21.842 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:21.842 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:21.843 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:30:21.843 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600072+0000", 2024-09-14T06:30:21.843 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.3100000000000002e-06 2024-09-14T06:30:21.843 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:21.843 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:21.843 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:30:21.843 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.642987+0000", 2024-09-14T06:30:21.843 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.042914489 2024-09-14T06:30:21.843 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:21.843 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:21.843 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:30:21.844 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.643062+0000", 2024-09-14T06:30:21.844 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.5011000000000004e-05 2024-09-14T06:30:21.844 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:21.844 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:21.844 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:30:21.844 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.643153+0000", 2024-09-14T06:30:21.844 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.1452999999999993e-05 2024-09-14T06:30:21.844 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:21.844 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:21.844 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:21.844 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:21.845 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:21.845 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:55636 3.0 3:06490baf:::rbd_data.112dca2f6ae1.0000000000000030:17 [read 1171456~4096 [fadvise_nocache]] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:30:21.845 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:44.600078+0000", 2024-09-14T06:30:21.845 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 37.176256520000003, 2024-09-14T06:30:21.845 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.043207966, 2024-09-14T06:30:21.845 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:30:21.845 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:30:21.845 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:30:21.845 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:30:21.845 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:30:21.846 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:21.846 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 55636 2024-09-14T06:30:21.846 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:21.846 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:30:21.846 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:21.846 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:30:21.846 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600078+0000", 2024-09-14T06:30:21.846 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:21.846 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:21.846 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:21.846 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:30:21.846 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600078+0000", 2024-09-14T06:30:21.847 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:21.847 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:21.847 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:21.847 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:30:21.847 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600078+0000", 2024-09-14T06:30:21.847 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.34e-07 2024-09-14T06:30:21.847 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:21.847 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:21.847 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:30:21.847 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600080+0000", 2024-09-14T06:30:21.847 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.641e-06 2024-09-14T06:30:21.847 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:21.847 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:21.848 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:30:21.848 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600086+0000", 2024-09-14T06:30:21.848 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.3860000000000004e-06 2024-09-14T06:30:21.848 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:21.848 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:21.848 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:30:21.848 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600091+0000", 2024-09-14T06:30:21.848 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.5970000000000002e-06 2024-09-14T06:30:21.848 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:21.848 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:21.848 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:30:21.848 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.643184+0000", 2024-09-14T06:30:21.848 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.043092893 2024-09-14T06:30:21.849 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:21.849 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:21.849 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:30:21.849 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.643228+0000", 2024-09-14T06:30:21.849 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.4536999999999997e-05 2024-09-14T06:30:21.849 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:21.849 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:21.849 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:30:21.849 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.643286+0000", 2024-09-14T06:30:21.849 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.7578000000000003e-05 2024-09-14T06:30:21.849 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:21.849 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:21.849 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:21.850 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:21.850 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:21.850 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:55637 3.0 3:06490baf:::rbd_data.112dca2f6ae1.0000000000000030:17 [read 1196032~4096 [fadvise_nocache]] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:30:21.850 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:44.600097+0000", 2024-09-14T06:30:21.850 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 37.176237956000001, 2024-09-14T06:30:21.850 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.043399124999999997, 2024-09-14T06:30:21.850 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:30:21.850 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:30:21.850 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:30:21.850 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:30:21.850 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:30:21.850 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:21.850 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 55637 2024-09-14T06:30:21.850 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:21.851 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:30:21.851 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:21.851 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:30:21.851 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600097+0000", 2024-09-14T06:30:21.851 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:21.851 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:21.851 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:21.851 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:30:21.851 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600097+0000", 2024-09-14T06:30:21.851 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:21.851 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:21.851 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:21.851 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:30:21.851 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.604372+0000", 2024-09-14T06:30:21.852 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0042751489999999998 2024-09-14T06:30:21.852 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:21.852 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:21.852 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:30:21.852 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.604377+0000", 2024-09-14T06:30:21.852 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.7899999999999999e-06 2024-09-14T06:30:21.852 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:21.852 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:21.852 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:30:21.852 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.604389+0000", 2024-09-14T06:30:21.852 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.2197e-05 2024-09-14T06:30:21.852 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:21.852 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:21.852 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:30:21.852 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.604397+0000", 2024-09-14T06:30:21.853 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.8730000000000007e-06 2024-09-14T06:30:21.853 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:21.853 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:21.853 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:30:21.853 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.643324+0000", 2024-09-14T06:30:21.853 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.038926897000000002 2024-09-14T06:30:21.853 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:21.853 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:21.853 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:30:21.853 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.643402+0000", 2024-09-14T06:30:21.853 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.8063000000000001e-05 2024-09-14T06:30:21.853 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:21.853 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:21.853 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:30:21.854 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.643496+0000", 2024-09-14T06:30:21.854 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.4155999999999998e-05 2024-09-14T06:30:21.854 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:21.854 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:21.854 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:21.854 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:21.854 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:21.854 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:55657 3.0 3:06490baf:::rbd_data.112dca2f6ae1.0000000000000030:17 [read 1687552~4096 [fadvise_nocache]] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:30:21.854 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:44.604841+0000", 2024-09-14T06:30:21.854 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 37.171493226999999, 2024-09-14T06:30:21.854 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.041890087999999999, 2024-09-14T06:30:21.854 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:30:21.854 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:30:21.854 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:30:21.855 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:30:21.855 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:30:21.855 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:21.855 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 55657 2024-09-14T06:30:21.855 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:21.855 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:30:21.855 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:21.855 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:30:21.855 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.604841+0000", 2024-09-14T06:30:21.855 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:21.855 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:21.855 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:21.855 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:30:21.856 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.604841+0000", 2024-09-14T06:30:21.856 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:21.856 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:21.856 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:21.856 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:30:21.856 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.604842+0000", 2024-09-14T06:30:21.856 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.0900000000000001e-07 2024-09-14T06:30:21.856 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:21.856 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:21.856 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:30:21.856 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.604846+0000", 2024-09-14T06:30:21.856 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.5759999999999999e-06 2024-09-14T06:30:21.856 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:21.856 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:21.857 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:30:21.857 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.604850+0000", 2024-09-14T06:30:21.857 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.405e-06 2024-09-14T06:30:21.857 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:21.857 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:21.857 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:30:21.857 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.604852+0000", 2024-09-14T06:30:21.857 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.8590000000000001e-06 2024-09-14T06:30:21.857 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:21.857 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:21.857 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:30:21.857 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.646633+0000", 2024-09-14T06:30:21.857 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.041780772000000001 2024-09-14T06:30:21.858 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:21.858 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:21.858 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:30:21.858 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.646677+0000", 2024-09-14T06:30:21.858 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.4115e-05 2024-09-14T06:30:21.858 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:21.858 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:21.858 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:30:21.858 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.646731+0000", 2024-09-14T06:30:21.858 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.4151999999999999e-05 2024-09-14T06:30:21.858 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:21.858 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:21.858 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:21.859 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:21.859 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:21.859 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:55658 3.0 3:06490baf:::rbd_data.112dca2f6ae1.0000000000000030:17 [read 1712128~4096 [fadvise_nocache]] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:30:21.859 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:44.604856+0000", 2024-09-14T06:30:21.859 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 37.171478626999999, 2024-09-14T06:30:21.859 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.041986227000000001, 2024-09-14T06:30:21.859 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:30:21.859 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:30:21.859 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:30:21.859 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:30:21.859 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:30:21.859 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:21.859 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 55658 2024-09-14T06:30:21.859 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:21.860 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:30:21.860 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:21.860 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:30:21.860 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.604856+0000", 2024-09-14T06:30:21.860 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:21.860 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:21.860 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:21.860 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:30:21.860 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.604856+0000", 2024-09-14T06:30:21.860 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:21.860 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:21.860 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:21.860 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:30:21.861 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.604856+0000", 2024-09-14T06:30:21.861 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.23e-07 2024-09-14T06:30:21.861 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:21.861 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:21.861 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:30:21.861 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.604857+0000", 2024-09-14T06:30:21.861 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.9999999999999995e-07 2024-09-14T06:30:21.861 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:21.861 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:21.861 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:30:21.861 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.604861+0000", 2024-09-14T06:30:21.861 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.9890000000000003e-06 2024-09-14T06:30:21.861 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:21.862 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:21.862 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:30:21.862 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.604864+0000", 2024-09-14T06:30:21.862 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.7429999999999998e-06 2024-09-14T06:30:21.862 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:21.862 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:21.862 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:30:21.862 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.646744+0000", 2024-09-14T06:30:21.862 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.041880325000000003 2024-09-14T06:30:21.862 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:21.862 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:21.863 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:30:21.863 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.646784+0000", 2024-09-14T06:30:21.863 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.9604000000000002e-05 2024-09-14T06:30:21.863 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:21.863 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:21.863 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:30:21.863 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.646842+0000", 2024-09-14T06:30:21.863 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.8343e-05 2024-09-14T06:30:21.863 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:21.863 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:21.863 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:21.863 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:21.864 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:21.864 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:55659 3.0 3:06490baf:::rbd_data.112dca2f6ae1.0000000000000030:17 [read 1736704~4096 [fadvise_nocache]] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:30:21.864 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:44.604867+0000", 2024-09-14T06:30:21.864 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 37.171467474000004, 2024-09-14T06:30:21.864 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.042097082000000001, 2024-09-14T06:30:21.864 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:30:21.864 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:30:21.864 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:30:21.864 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:30:21.864 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:30:21.864 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:21.864 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 55659 2024-09-14T06:30:21.864 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:21.864 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:30:21.865 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:21.865 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:30:21.865 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.604867+0000", 2024-09-14T06:30:21.865 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:21.865 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:21.865 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:21.865 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:30:21.865 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.604867+0000", 2024-09-14T06:30:21.865 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:21.865 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:21.865 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:21.865 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:30:21.865 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.604867+0000", 2024-09-14T06:30:21.865 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.2000000000000001e-07 2024-09-14T06:30:21.866 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:21.866 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:21.866 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:30:21.866 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.604868+0000", 2024-09-14T06:30:21.866 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.04e-06 2024-09-14T06:30:21.866 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:21.866 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:21.866 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:30:21.866 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.604871+0000", 2024-09-14T06:30:21.866 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.9689999999999999e-06 2024-09-14T06:30:21.866 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:21.866 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:21.866 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:30:21.866 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.604874+0000", 2024-09-14T06:30:21.866 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.5069999999999999e-06 2024-09-14T06:30:21.866 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:21.867 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:21.867 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:30:21.867 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.646874+0000", 2024-09-14T06:30:21.867 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.042000513000000003 2024-09-14T06:30:21.867 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:21.867 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:21.867 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:30:21.867 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.646914+0000", 2024-09-14T06:30:21.867 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.9443999999999997e-05 2024-09-14T06:30:21.867 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:21.867 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:21.867 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:30:21.867 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.646964+0000", 2024-09-14T06:30:21.868 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.0389000000000002e-05 2024-09-14T06:30:21.868 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:21.868 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:21.868 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:21.868 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:21.868 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:21.868 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:55660 3.0 3:06490baf:::rbd_data.112dca2f6ae1.0000000000000030:17 [read 1761280~4096 [fadvise_nocache]] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:30:21.868 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:44.604877+0000", 2024-09-14T06:30:21.868 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 37.171457725000003, 2024-09-14T06:30:21.868 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.042196349000000001, 2024-09-14T06:30:21.868 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:30:21.868 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:30:21.868 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:30:21.868 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:30:21.869 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:30:21.869 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:21.869 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 55660 2024-09-14T06:30:21.869 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:21.869 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:30:21.869 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:21.869 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:30:21.869 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.604877+0000", 2024-09-14T06:30:21.869 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:21.869 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:21.869 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:21.869 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:30:21.869 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.604877+0000", 2024-09-14T06:30:21.869 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:21.869 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:21.869 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:21.870 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:30:21.870 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.605810+0000", 2024-09-14T06:30:21.870 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00093336899999999997 2024-09-14T06:30:21.870 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:21.870 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:21.870 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:30:21.870 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.605815+0000", 2024-09-14T06:30:21.870 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.6140000000000002e-06 2024-09-14T06:30:21.870 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:21.870 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:21.870 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:30:21.870 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.605825+0000", 2024-09-14T06:30:21.870 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.8889999999999997e-06 2024-09-14T06:30:21.871 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:21.871 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:21.871 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:30:21.871 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.605831+0000", 2024-09-14T06:30:21.871 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.9800000000000003e-06 2024-09-14T06:30:21.871 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:21.871 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:21.871 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:30:21.871 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.646989+0000", 2024-09-14T06:30:21.871 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.041157878000000002 2024-09-14T06:30:21.871 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:21.871 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:21.871 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:30:21.871 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.647026+0000", 2024-09-14T06:30:21.871 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.7305000000000003e-05 2024-09-14T06:30:21.872 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:21.872 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:21.872 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:30:21.872 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.647073+0000", 2024-09-14T06:30:21.872 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.7314000000000001e-05 2024-09-14T06:30:21.872 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:21.872 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:21.872 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:21.872 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:21.872 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:21.872 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:55666 3.0 3:06490baf:::rbd_data.112dca2f6ae1.0000000000000030:17 [read 1908736~4096 [fadvise_nocache]] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:30:21.872 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:44.605898+0000", 2024-09-14T06:30:21.872 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 37.170436979000002, 2024-09-14T06:30:21.872 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.041862887000000001, 2024-09-14T06:30:21.872 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:30:21.873 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:30:21.873 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:30:21.873 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:30:21.873 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:30:21.873 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:21.873 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 55666 2024-09-14T06:30:21.873 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:21.873 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:30:21.873 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:21.873 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:30:21.873 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.605898+0000", 2024-09-14T06:30:21.873 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:21.873 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:21.873 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:21.873 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:30:21.873 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.605898+0000", 2024-09-14T06:30:21.874 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:21.874 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:21.874 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:21.874 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:30:21.874 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.611874+0000", 2024-09-14T06:30:21.874 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0059764049999999997 2024-09-14T06:30:21.874 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:21.874 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:21.874 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:30:21.874 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.611879+0000", 2024-09-14T06:30:21.874 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.8749999999999999e-06 2024-09-14T06:30:21.874 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:21.874 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:21.874 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:30:21.874 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.611887+0000", 2024-09-14T06:30:21.875 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.4109999999999996e-06 2024-09-14T06:30:21.875 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:21.875 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:21.875 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:30:21.875 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.611893+0000", 2024-09-14T06:30:21.875 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.2959999999999998e-06 2024-09-14T06:30:21.875 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:21.875 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:21.875 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:30:21.875 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.647674+0000", 2024-09-14T06:30:21.875 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.035781281999999998 2024-09-14T06:30:21.875 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:21.875 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:21.875 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:30:21.875 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.647711+0000", 2024-09-14T06:30:21.876 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.7434000000000001e-05 2024-09-14T06:30:21.876 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:21.876 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:21.876 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:30:21.876 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.647760+0000", 2024-09-14T06:30:21.876 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.9184000000000002e-05 2024-09-14T06:30:21.876 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:21.876 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:21.876 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:21.876 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:21.876 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:21.876 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-14T06:30:21.878 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-14T06:30:21.878 INFO:tasks.ceph.osd.7.smithi138.stderr:2024-09-14T06:30:21.877+0000 7f3c6a21d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33623) UID: 0 2024-09-14T06:30:21.978 INFO:tasks.ceph.osd.3.smithi094.stderr:2024-09-14T06:30:21.977+0000 7feeeafe8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33655) UID: 0 2024-09-14T06:30:21.980 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-14T06:30:21.981 INFO:teuthology.orchestra.run.smithi094.stdout: "ops": [], 2024-09-14T06:30:21.981 INFO:teuthology.orchestra.run.smithi094.stdout: "num_ops": 0 2024-09-14T06:30:21.981 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-14T06:30:21.989 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-14T06:30:22.078 INFO:tasks.ceph.osd.3.smithi094.stderr:2024-09-14T06:30:22.077+0000 7feeeafe8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33655) UID: 0 2024-09-14T06:30:22.131 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-14T06:30:22.131 INFO:teuthology.orchestra.run.smithi094.stdout: "ops": [], 2024-09-14T06:30:22.131 INFO:teuthology.orchestra.run.smithi094.stdout: "complaint_time": 30, 2024-09-14T06:30:22.131 INFO:teuthology.orchestra.run.smithi094.stdout: "num_blocked_ops": 0 2024-09-14T06:30:22.131 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-14T06:30:22.139 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-14T06:30:22.179 INFO:tasks.ceph.osd.5.smithi138.stderr:2024-09-14T06:30:22.178+0000 7fe13746f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33618) UID: 0 2024-09-14T06:30:22.279 INFO:tasks.ceph.osd.0.smithi094.stderr:2024-09-14T06:30:22.278+0000 7f07c9333640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33636) UID: 0 2024-09-14T06:30:22.283 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-14T06:30:22.283 INFO:teuthology.orchestra.run.smithi094.stdout: "size": 20, 2024-09-14T06:30:22.283 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 600, 2024-09-14T06:30:22.283 INFO:teuthology.orchestra.run.smithi094.stdout: "ops": [ 2024-09-14T06:30:22.283 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:22.283 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4397.0:1848 3.4 e32/23 3:27620a21:::rbd_data.112dcdddac74.000000000000000b:head v 32'40, mlcod=30'38)", 2024-09-14T06:30:22.284 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:08.658649+0000", 2024-09-14T06:30:22.284 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 73.622899404999998, 2024-09-14T06:30:22.284 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.001862867, 2024-09-14T06:30:22.284 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:30:22.284 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:30:22.284 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:30:22.284 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:30:22.284 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:22.284 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:30:22.284 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.658649+0000", 2024-09-14T06:30:22.284 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:22.284 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:22.284 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:22.285 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:30:22.285 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.658649+0000", 2024-09-14T06:30:22.285 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:22.285 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:22.285 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:22.285 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:30:22.285 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.658651+0000", 2024-09-14T06:30:22.285 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.5650000000000001e-06 2024-09-14T06:30:22.285 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:22.285 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:22.285 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:30:22.285 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.658709+0000", 2024-09-14T06:30:22.285 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.8442000000000001e-05 2024-09-14T06:30:22.285 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:22.285 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:22.286 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:30:22.286 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.658730+0000", 2024-09-14T06:30:22.286 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.0509999999999998e-05 2024-09-14T06:30:22.286 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:22.286 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:22.286 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:30:22.286 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.658740+0000", 2024-09-14T06:30:22.286 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.8619999999999998e-06 2024-09-14T06:30:22.286 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:22.286 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:22.286 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:30:22.286 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.658813+0000", 2024-09-14T06:30:22.286 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.3578000000000006e-05 2024-09-14T06:30:22.287 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:22.287 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:22.287 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:30:22.287 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.658836+0000", 2024-09-14T06:30:22.287 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.2677e-05 2024-09-14T06:30:22.287 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:22.287 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:22.287 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-14T06:30:22.287 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.660462+0000", 2024-09-14T06:30:22.287 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00162637 2024-09-14T06:30:22.287 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:22.287 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:22.287 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:30:22.287 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.660512+0000", 2024-09-14T06:30:22.288 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.9863e-05 2024-09-14T06:30:22.288 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:22.288 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:22.288 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:22.288 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:22.288 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:22.288 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4397.0:2199 3.0 e32/23 3:05f3e980:::rbd_data.112dcdddac74.000000000000000f:head v 32'3, mlcod=31'1)", 2024-09-14T06:30:22.288 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:08.680236+0000", 2024-09-14T06:30:22.288 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 73.601313039999994, 2024-09-14T06:30:22.288 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0019068990000000001, 2024-09-14T06:30:22.288 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:30:22.288 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:30:22.288 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:30:22.289 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:30:22.289 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:22.289 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:30:22.289 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.680236+0000", 2024-09-14T06:30:22.289 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:22.289 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:22.289 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:22.289 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:30:22.289 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.680236+0000", 2024-09-14T06:30:22.289 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:22.289 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:22.289 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:22.289 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:30:22.289 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.680237+0000", 2024-09-14T06:30:22.289 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.4980000000000001e-06 2024-09-14T06:30:22.290 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:22.290 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:22.290 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:30:22.290 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.680324+0000", 2024-09-14T06:30:22.290 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.7310000000000003e-05 2024-09-14T06:30:22.290 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:22.290 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:22.290 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:30:22.290 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.680354+0000", 2024-09-14T06:30:22.290 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.97e-05 2024-09-14T06:30:22.290 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:22.290 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:22.290 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:30:22.290 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.680366+0000", 2024-09-14T06:30:22.291 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.2048999999999999e-05 2024-09-14T06:30:22.291 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:22.291 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:22.291 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:30:22.291 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.680439+0000", 2024-09-14T06:30:22.291 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.2910000000000005e-05 2024-09-14T06:30:22.291 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:22.291 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:22.291 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:30:22.291 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.680465+0000", 2024-09-14T06:30:22.291 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.5812999999999998e-05 2024-09-14T06:30:22.291 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:22.291 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:22.291 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-14T06:30:22.291 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.682094+0000", 2024-09-14T06:30:22.292 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.001628876 2024-09-14T06:30:22.292 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:22.292 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:22.292 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:30:22.292 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.682142+0000", 2024-09-14T06:30:22.292 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.8742999999999997e-05 2024-09-14T06:30:22.292 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:22.292 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:22.292 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:22.292 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:22.292 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:22.292 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4397.0:3292 3.9 e32/23 3:91dbce57:::rbd_data.112dcdddac74.0000000000000036:head v 32'10, mlcod=32'8)", 2024-09-14T06:30:22.292 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:08.740506+0000", 2024-09-14T06:30:22.292 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 73.541042328000003, 2024-09-14T06:30:22.293 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.001864166, 2024-09-14T06:30:22.293 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:30:22.293 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:30:22.293 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:30:22.293 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:30:22.293 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:22.293 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:30:22.293 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.740506+0000", 2024-09-14T06:30:22.293 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:22.293 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:22.293 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:22.293 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:30:22.293 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.740506+0000", 2024-09-14T06:30:22.293 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:22.293 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:22.294 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:22.294 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:30:22.294 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.740514+0000", 2024-09-14T06:30:22.294 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.2109999999999998e-06 2024-09-14T06:30:22.294 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:22.294 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:22.294 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:30:22.294 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.740745+0000", 2024-09-14T06:30:22.294 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000230194 2024-09-14T06:30:22.294 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:22.294 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:22.294 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:30:22.294 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.740764+0000", 2024-09-14T06:30:22.295 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.9330999999999998e-05 2024-09-14T06:30:22.295 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:22.295 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:22.295 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:30:22.295 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.740774+0000", 2024-09-14T06:30:22.295 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.6260000000000007e-06 2024-09-14T06:30:22.295 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:22.295 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:22.295 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:30:22.295 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.740839+0000", 2024-09-14T06:30:22.295 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.5314000000000005e-05 2024-09-14T06:30:22.295 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:22.295 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:22.295 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:30:22.296 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.740860+0000", 2024-09-14T06:30:22.296 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.0868000000000001e-05 2024-09-14T06:30:22.296 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:22.296 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:22.296 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-14T06:30:22.296 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.742327+0000", 2024-09-14T06:30:22.296 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0014672909999999999 2024-09-14T06:30:22.296 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:22.296 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:22.296 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:30:22.296 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.742370+0000", 2024-09-14T06:30:22.296 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.3331000000000002e-05 2024-09-14T06:30:22.296 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:22.296 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:22.297 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:22.297 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:22.297 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:22.297 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4397.0:26415 3.0 e53/23 3:05fd9be0:::rbd_data.112d38768680.0000000000000006:head v 53'21, mlcod=50'19)", 2024-09-14T06:30:22.297 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:30.721610+0000", 2024-09-14T06:30:22.297 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 51.559938258999999, 2024-09-14T06:30:22.297 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0021763160000000002, 2024-09-14T06:30:22.297 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:30:22.297 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:30:22.297 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:30:22.297 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:30:22.297 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:22.297 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:30:22.297 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:30.721610+0000", 2024-09-14T06:30:22.298 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:22.298 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:22.298 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:22.298 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:30:22.298 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:30.721610+0000", 2024-09-14T06:30:22.298 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:22.298 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:22.298 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:22.298 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:30:22.298 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:30.721611+0000", 2024-09-14T06:30:22.298 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.41e-07 2024-09-14T06:30:22.298 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:22.298 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:22.298 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:30:22.299 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:30.721905+0000", 2024-09-14T06:30:22.299 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00029394300000000001 2024-09-14T06:30:22.299 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:22.299 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:22.299 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:30:22.299 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:30.721921+0000", 2024-09-14T06:30:22.299 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.6609999999999999e-05 2024-09-14T06:30:22.299 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:22.299 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:22.299 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:30:22.299 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:30.721931+0000", 2024-09-14T06:30:22.299 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.6739999999999992e-06 2024-09-14T06:30:22.299 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:22.299 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:22.299 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:30:22.300 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:30.722001+0000", 2024-09-14T06:30:22.300 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.0549000000000005e-05 2024-09-14T06:30:22.300 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:22.300 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:22.300 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:30:22.300 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:30.722029+0000", 2024-09-14T06:30:22.300 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.7327000000000001e-05 2024-09-14T06:30:22.300 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:22.300 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:22.300 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-14T06:30:22.300 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:30.723727+0000", 2024-09-14T06:30:22.300 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0016977310000000001 2024-09-14T06:30:22.301 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:22.301 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:22.301 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:30:22.301 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:30.723787+0000", 2024-09-14T06:30:22.301 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.0140999999999999e-05 2024-09-14T06:30:22.301 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:22.301 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:22.301 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:22.301 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:22.301 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:22.301 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4397.0:36084 3.0 e56/23 3:05a48ef4:::rbd_data.112dc58639fa.0000000000000058:head v 56'29, mlcod=56'27)", 2024-09-14T06:30:22.301 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:34.508855+0000", 2024-09-14T06:30:22.301 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 47.772693504999999, 2024-09-14T06:30:22.301 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0020801140000000001, 2024-09-14T06:30:22.302 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:30:22.302 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:30:22.302 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:30:22.302 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:30:22.302 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:22.302 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:30:22.302 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:34.508855+0000", 2024-09-14T06:30:22.302 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:22.302 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:22.302 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:22.302 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:30:22.302 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:34.508855+0000", 2024-09-14T06:30:22.302 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:22.302 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:22.302 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:22.303 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:30:22.303 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:34.508856+0000", 2024-09-14T06:30:22.303 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.1909999999999999e-06 2024-09-14T06:30:22.303 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:22.303 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:22.303 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:30:22.303 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:34.509203+0000", 2024-09-14T06:30:22.303 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00034625900000000002 2024-09-14T06:30:22.303 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:22.303 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:22.303 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:30:22.303 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:34.509218+0000", 2024-09-14T06:30:22.303 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.5091e-05 2024-09-14T06:30:22.303 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:22.303 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:22.303 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:30:22.304 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:34.509227+0000", 2024-09-14T06:30:22.304 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.0850000000000003e-06 2024-09-14T06:30:22.304 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:22.304 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:22.304 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:30:22.304 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:34.509311+0000", 2024-09-14T06:30:22.304 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.4436000000000003e-05 2024-09-14T06:30:22.304 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:22.304 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:22.304 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:30:22.304 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:34.509335+0000", 2024-09-14T06:30:22.304 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.3577000000000001e-05 2024-09-14T06:30:22.304 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:22.305 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:22.305 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-14T06:30:22.305 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:34.510890+0000", 2024-09-14T06:30:22.305 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0015554259999999999 2024-09-14T06:30:22.305 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:22.305 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:22.305 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:30:22.305 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:34.510935+0000", 2024-09-14T06:30:22.305 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.5049000000000002e-05 2024-09-14T06:30:22.305 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:22.305 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:22.305 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:22.305 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:22.305 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:22.305 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4397.0:36966 3.4 e56/23 3:22008318:::rbd_data.112dc58639fa.0000000000000066:head v 56'56, mlcod=50'54)", 2024-09-14T06:30:22.306 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:34.565614+0000", 2024-09-14T06:30:22.306 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 47.715935068999997, 2024-09-14T06:30:22.306 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0018691929999999999, 2024-09-14T06:30:22.306 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:30:22.306 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:30:22.306 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:30:22.306 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:30:22.306 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:22.306 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:30:22.306 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:34.565614+0000", 2024-09-14T06:30:22.306 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:22.306 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:22.306 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:22.306 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:30:22.306 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:34.565614+0000", 2024-09-14T06:30:22.306 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:22.307 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:22.307 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:22.307 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:30:22.307 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:34.565615+0000", 2024-09-14T06:30:22.307 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.784e-06 2024-09-14T06:30:22.307 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:22.307 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:22.307 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:30:22.307 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:34.565919+0000", 2024-09-14T06:30:22.307 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00030351599999999999 2024-09-14T06:30:22.307 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:22.307 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:22.307 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:30:22.308 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:34.565933+0000", 2024-09-14T06:30:22.308 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.4465e-05 2024-09-14T06:30:22.308 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:22.308 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:22.308 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:30:22.308 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:34.565943+0000", 2024-09-14T06:30:22.308 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.7920000000000004e-06 2024-09-14T06:30:22.308 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:22.308 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:22.308 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:30:22.308 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:34.565999+0000", 2024-09-14T06:30:22.308 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.5643000000000002e-05 2024-09-14T06:30:22.308 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:22.308 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:22.308 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:30:22.309 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:34.566025+0000", 2024-09-14T06:30:22.309 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.6040000000000001e-05 2024-09-14T06:30:22.309 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:22.309 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:22.309 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-14T06:30:22.309 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:34.567430+0000", 2024-09-14T06:30:22.309 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.001405653 2024-09-14T06:30:22.309 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:22.309 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:22.309 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:30:22.309 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:34.567483+0000", 2024-09-14T06:30:22.309 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.2299999999999997e-05 2024-09-14T06:30:22.309 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:22.309 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:22.309 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:22.310 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:22.310 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:22.310 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4397.0:42285 3.9 e62/23 3:90747014:::rbd_data.112dca2f6ae1.000000000000002f:head v 62'54, mlcod=62'52)", 2024-09-14T06:30:22.310 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:40.755702+0000", 2024-09-14T06:30:22.310 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 41.525846684999998, 2024-09-14T06:30:22.310 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.001687596, 2024-09-14T06:30:22.310 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:30:22.310 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:30:22.310 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:30:22.310 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:30:22.310 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:22.310 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:30:22.310 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:40.755702+0000", 2024-09-14T06:30:22.310 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:22.310 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:22.311 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:22.311 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:30:22.311 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:40.755702+0000", 2024-09-14T06:30:22.311 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:22.311 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:22.311 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:22.311 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:30:22.311 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:40.755707+0000", 2024-09-14T06:30:22.311 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.4700000000000001e-06 2024-09-14T06:30:22.311 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:22.311 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:22.311 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:30:22.311 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:40.756366+0000", 2024-09-14T06:30:22.311 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00065852900000000004 2024-09-14T06:30:22.312 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:22.312 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:22.312 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:30:22.312 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:40.756383+0000", 2024-09-14T06:30:22.312 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.7275e-05 2024-09-14T06:30:22.312 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:22.312 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:22.312 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:30:22.312 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:40.756393+0000", 2024-09-14T06:30:22.312 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.4749999999999999e-06 2024-09-14T06:30:22.312 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:22.312 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:22.312 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:30:22.312 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:40.756462+0000", 2024-09-14T06:30:22.312 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.9214000000000005e-05 2024-09-14T06:30:22.313 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:22.313 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:22.313 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:30:22.313 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:40.756490+0000", 2024-09-14T06:30:22.313 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.7936000000000002e-05 2024-09-14T06:30:22.313 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:22.313 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:22.313 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-14T06:30:22.313 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:40.757343+0000", 2024-09-14T06:30:22.313 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000853611 2024-09-14T06:30:22.313 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:22.313 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:22.313 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:30:22.313 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:40.757390+0000", 2024-09-14T06:30:22.313 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.6085999999999999e-05 2024-09-14T06:30:22.313 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:22.314 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:22.314 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:22.314 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:22.314 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:22.314 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4397.0:45722 3.9 e64/23 3:91fed098:::rbd_data.112d5a737e72.0000000000000001:head v 64'61, mlcod=62'59)", 2024-09-14T06:30:22.314 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:43.864269+0000", 2024-09-14T06:30:22.314 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 38.417279430000001, 2024-09-14T06:30:22.314 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.003055224, 2024-09-14T06:30:22.314 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:30:22.314 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:30:22.314 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:30:22.314 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:30:22.314 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:22.314 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:30:22.314 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.864269+0000", 2024-09-14T06:30:22.315 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:22.315 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:22.315 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:22.315 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:30:22.315 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.864269+0000", 2024-09-14T06:30:22.315 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:22.315 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:22.315 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:22.315 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:30:22.315 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.864269+0000", 2024-09-14T06:30:22.315 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.41e-07 2024-09-14T06:30:22.315 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:22.315 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:22.315 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:30:22.315 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.865490+0000", 2024-09-14T06:30:22.316 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.001220552 2024-09-14T06:30:22.316 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:22.316 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:22.316 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:30:22.316 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.865506+0000", 2024-09-14T06:30:22.316 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.6263999999999999e-05 2024-09-14T06:30:22.316 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:22.316 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:22.316 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:30:22.316 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.865516+0000", 2024-09-14T06:30:22.316 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.0091999999999999e-05 2024-09-14T06:30:22.316 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:22.316 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:22.316 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:30:22.316 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.865583+0000", 2024-09-14T06:30:22.317 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.6676999999999999e-05 2024-09-14T06:30:22.317 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:22.317 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:22.317 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:30:22.317 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.865610+0000", 2024-09-14T06:30:22.317 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.6529999999999998e-05 2024-09-14T06:30:22.317 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:22.317 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:22.317 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-14T06:30:22.317 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.867281+0000", 2024-09-14T06:30:22.317 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.001671203 2024-09-14T06:30:22.317 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:22.317 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:22.318 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:30:22.318 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.867324+0000", 2024-09-14T06:30:22.318 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.3665000000000003e-05 2024-09-14T06:30:22.318 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:22.318 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:22.318 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:22.318 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:22.318 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:22.318 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4397.0:49627 3.4 e64/23 3:21e30d4b:::rbd_data.112d5a737e72.0000000000000017:head v 64'69, mlcod=61'67)", 2024-09-14T06:30:22.318 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:44.206446+0000", 2024-09-14T06:30:22.318 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 38.075102792000003, 2024-09-14T06:30:22.318 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0018142220000000001, 2024-09-14T06:30:22.318 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:30:22.319 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:30:22.319 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:30:22.319 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:30:22.319 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:22.319 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:30:22.319 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.206446+0000", 2024-09-14T06:30:22.319 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:22.319 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:22.319 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:22.319 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:30:22.319 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.206446+0000", 2024-09-14T06:30:22.319 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:22.319 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:22.319 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:22.319 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:30:22.319 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.206446+0000", 2024-09-14T06:30:22.320 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.4900000000000001e-07 2024-09-14T06:30:22.320 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:22.320 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:22.320 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:30:22.320 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.206797+0000", 2024-09-14T06:30:22.320 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00035112799999999998 2024-09-14T06:30:22.320 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:22.320 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:22.320 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:30:22.320 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.206816+0000", 2024-09-14T06:30:22.320 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.8977000000000001e-05 2024-09-14T06:30:22.320 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:22.320 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:22.321 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:30:22.321 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.206825+0000", 2024-09-14T06:30:22.321 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.8279999999999992e-06 2024-09-14T06:30:22.321 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:22.321 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:22.321 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:30:22.321 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.206892+0000", 2024-09-14T06:30:22.321 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.6809000000000004e-05 2024-09-14T06:30:22.321 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:22.321 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:22.321 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:30:22.321 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.206915+0000", 2024-09-14T06:30:22.321 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.315e-05 2024-09-14T06:30:22.321 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:22.322 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:22.322 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-14T06:30:22.322 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.208201+0000", 2024-09-14T06:30:22.322 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0012858819999999999 2024-09-14T06:30:22.322 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:22.322 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:22.322 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:30:22.322 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.208260+0000", 2024-09-14T06:30:22.322 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.9098999999999998e-05 2024-09-14T06:30:22.322 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:22.322 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:22.322 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:22.322 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:22.322 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:22.323 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4397.0:50003 3.9 e64/23 3:94f26c4b:::rbd_data.112d5a737e72.000000000000001f:head v 64'62, mlcod=62'60)", 2024-09-14T06:30:22.323 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:44.220413+0000", 2024-09-14T06:30:22.323 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 38.061135114000002, 2024-09-14T06:30:22.323 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0019210430000000001, 2024-09-14T06:30:22.323 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:30:22.323 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:30:22.323 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:30:22.323 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:30:22.323 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:22.323 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:30:22.323 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.220413+0000", 2024-09-14T06:30:22.323 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:22.323 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:22.324 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:22.324 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:30:22.324 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.220413+0000", 2024-09-14T06:30:22.324 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:22.324 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:22.324 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:22.324 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:30:22.324 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.220415+0000", 2024-09-14T06:30:22.324 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.575e-06 2024-09-14T06:30:22.324 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:22.324 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:22.324 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:30:22.324 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.220984+0000", 2024-09-14T06:30:22.324 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000568624 2024-09-14T06:30:22.325 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:22.325 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:22.325 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:30:22.325 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.221007+0000", 2024-09-14T06:30:22.325 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.3112000000000002e-05 2024-09-14T06:30:22.325 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:22.325 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:22.325 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:30:22.325 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.221017+0000", 2024-09-14T06:30:22.325 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.0098999999999999e-05 2024-09-14T06:30:22.325 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:22.325 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:22.325 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:30:22.326 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.221110+0000", 2024-09-14T06:30:22.326 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.3268000000000002e-05 2024-09-14T06:30:22.326 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:22.326 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:22.326 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:30:22.326 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.221131+0000", 2024-09-14T06:30:22.326 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.0905999999999999e-05 2024-09-14T06:30:22.326 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:22.326 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:22.326 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-14T06:30:22.326 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.222290+0000", 2024-09-14T06:30:22.326 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.001159408 2024-09-14T06:30:22.326 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:22.327 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:22.327 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:30:22.327 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.222335+0000", 2024-09-14T06:30:22.327 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.4051000000000002e-05 2024-09-14T06:30:22.327 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:22.327 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:22.327 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:22.327 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:22.327 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:22.327 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4397.0:50006 3.4 e64/23 3:20cfe300:::rbd_data.112d5a737e72.0000000000000019:head v 64'70, mlcod=61'68)", 2024-09-14T06:30:22.327 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:44.224218+0000", 2024-09-14T06:30:22.327 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 38.057330233000002, 2024-09-14T06:30:22.327 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0021104320000000002, 2024-09-14T06:30:22.327 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:30:22.327 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:30:22.328 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:30:22.328 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:30:22.328 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:22.328 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:30:22.328 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.224218+0000", 2024-09-14T06:30:22.328 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:22.328 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:22.328 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:22.328 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:30:22.328 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.224218+0000", 2024-09-14T06:30:22.328 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:22.328 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:22.328 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:22.328 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:30:22.329 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.224219+0000", 2024-09-14T06:30:22.329 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.0049999999999999e-06 2024-09-14T06:30:22.329 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:22.329 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:22.329 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:30:22.329 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.224326+0000", 2024-09-14T06:30:22.329 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000106848 2024-09-14T06:30:22.329 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:22.329 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:22.329 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:30:22.329 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.224345+0000", 2024-09-14T06:30:22.329 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.8587999999999999e-05 2024-09-14T06:30:22.329 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:22.329 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:22.330 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:30:22.330 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.224354+0000", 2024-09-14T06:30:22.330 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.7220000000000005e-06 2024-09-14T06:30:22.330 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:22.330 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:22.330 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:30:22.330 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.224416+0000", 2024-09-14T06:30:22.330 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.2738999999999997e-05 2024-09-14T06:30:22.330 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:22.330 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:22.330 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:30:22.330 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.224441+0000", 2024-09-14T06:30:22.330 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.4408000000000001e-05 2024-09-14T06:30:22.330 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:22.331 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:22.331 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-14T06:30:22.331 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.226295+0000", 2024-09-14T06:30:22.331 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00185428 2024-09-14T06:30:22.331 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:22.331 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:22.331 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:30:22.331 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.226329+0000", 2024-09-14T06:30:22.331 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.3841999999999999e-05 2024-09-14T06:30:22.331 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:22.331 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:22.331 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:22.331 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:22.331 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:22.332 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4397.0:51816 3.4 e64/23 3:21a826f9:::rbd_data.112d5a737e72.000000000000002d:head v 64'72, mlcod=64'70)", 2024-09-14T06:30:22.332 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:44.312072+0000", 2024-09-14T06:30:22.332 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 37.969476759999999, 2024-09-14T06:30:22.332 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0019555240000000002, 2024-09-14T06:30:22.332 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:30:22.332 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:30:22.332 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:30:22.332 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:30:22.332 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:22.332 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:30:22.332 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.312072+0000", 2024-09-14T06:30:22.332 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:22.332 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:22.332 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:22.333 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:30:22.333 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.312072+0000", 2024-09-14T06:30:22.333 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:22.333 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:22.333 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:22.333 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:30:22.333 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.312074+0000", 2024-09-14T06:30:22.333 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.9530000000000002e-06 2024-09-14T06:30:22.333 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:22.333 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:22.333 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:30:22.333 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.312378+0000", 2024-09-14T06:30:22.333 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00030392 2024-09-14T06:30:22.333 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:22.334 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:22.334 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:30:22.334 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.312399+0000", 2024-09-14T06:30:22.334 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.1134000000000001e-05 2024-09-14T06:30:22.334 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:22.334 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:22.334 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:30:22.334 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.312408+0000", 2024-09-14T06:30:22.334 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.2879999999999998e-06 2024-09-14T06:30:22.334 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:22.334 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:22.334 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:30:22.334 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.312494+0000", 2024-09-14T06:30:22.335 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.6280999999999996e-05 2024-09-14T06:30:22.335 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:22.335 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:22.335 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:30:22.335 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.312518+0000", 2024-09-14T06:30:22.335 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.3149000000000001e-05 2024-09-14T06:30:22.335 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:22.335 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:22.335 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-14T06:30:22.335 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.313964+0000", 2024-09-14T06:30:22.335 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.001446754 2024-09-14T06:30:22.335 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:22.335 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:22.335 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:30:22.336 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.314027+0000", 2024-09-14T06:30:22.336 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.3045000000000004e-05 2024-09-14T06:30:22.336 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:22.336 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:22.336 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:22.336 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:22.336 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:22.336 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4397.0:74241 3.0 e73/23 3:06e93fb9:::rbd_data.112d80ca42af.0000000000000000:head v 73'102, mlcod=71'100)", 2024-09-14T06:30:22.336 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:53.876730+0000", 2024-09-14T06:30:22.336 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 28.404818330000001, 2024-09-14T06:30:22.336 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0079470329999999992, 2024-09-14T06:30:22.336 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:30:22.336 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:30:22.337 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:30:22.337 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:30:22.337 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:22.337 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:30:22.337 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:53.876730+0000", 2024-09-14T06:30:22.337 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:22.337 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:22.337 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:22.337 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:30:22.337 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:53.876730+0000", 2024-09-14T06:30:22.337 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:22.337 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:22.337 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:22.337 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:30:22.337 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:53.876731+0000", 2024-09-14T06:30:22.338 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.0500000000000004e-07 2024-09-14T06:30:22.338 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:22.338 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:22.338 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:30:22.338 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:53.879616+0000", 2024-09-14T06:30:22.338 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0028856680000000001 2024-09-14T06:30:22.338 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:22.338 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:22.338 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:30:22.338 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:53.879643+0000", 2024-09-14T06:30:22.338 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.6120999999999999e-05 2024-09-14T06:30:22.338 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:22.338 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:22.338 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:30:22.338 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:53.879654+0000", 2024-09-14T06:30:22.339 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.1586999999999999e-05 2024-09-14T06:30:22.339 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:22.339 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:22.339 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:30:22.339 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:53.879756+0000", 2024-09-14T06:30:22.339 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00010223899999999999 2024-09-14T06:30:22.339 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:22.339 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:22.339 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:30:22.339 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:53.879783+0000", 2024-09-14T06:30:22.339 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.6913000000000001e-05 2024-09-14T06:30:22.339 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:22.339 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:22.339 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-14T06:30:22.339 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:53.884594+0000", 2024-09-14T06:30:22.340 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0048110729999999999 2024-09-14T06:30:22.340 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:22.340 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:22.340 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:30:22.340 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:53.884677+0000", 2024-09-14T06:30:22.340 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.2927000000000001e-05 2024-09-14T06:30:22.340 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:22.340 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:22.340 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:22.340 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:22.340 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:22.340 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4397.0:74243 3.9 e73/23 3:94977f81:::rbd_data.112d80ca42af.0000000000000002:head v 73'94, mlcod=71'92)", 2024-09-14T06:30:22.341 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:53.885156+0000", 2024-09-14T06:30:22.341 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 28.396392412000001, 2024-09-14T06:30:22.341 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0096009860000000006, 2024-09-14T06:30:22.341 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:30:22.341 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:30:22.341 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:30:22.341 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:30:22.341 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:22.341 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:30:22.341 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:53.885156+0000", 2024-09-14T06:30:22.341 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:22.341 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:22.341 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:22.341 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:30:22.342 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:53.885156+0000", 2024-09-14T06:30:22.342 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:22.342 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:22.342 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:22.342 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:30:22.342 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:53.885156+0000", 2024-09-14T06:30:22.342 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.23e-07 2024-09-14T06:30:22.342 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:22.342 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:22.342 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:30:22.342 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:53.889694+0000", 2024-09-14T06:30:22.342 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0045378709999999997 2024-09-14T06:30:22.342 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:22.342 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:22.342 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:30:22.343 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:53.889720+0000", 2024-09-14T06:30:22.343 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.5259e-05 2024-09-14T06:30:22.343 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:22.343 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:22.343 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:30:22.343 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:53.889731+0000", 2024-09-14T06:30:22.343 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.1739000000000001e-05 2024-09-14T06:30:22.343 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:22.343 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:22.343 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:30:22.343 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:53.889822+0000", 2024-09-14T06:30:22.343 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.1200999999999996e-05 2024-09-14T06:30:22.343 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:22.343 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:22.343 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:30:22.344 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:53.889850+0000", 2024-09-14T06:30:22.344 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.7084999999999999e-05 2024-09-14T06:30:22.344 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:22.344 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:22.344 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-14T06:30:22.344 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:53.894703+0000", 2024-09-14T06:30:22.344 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0048532690000000003 2024-09-14T06:30:22.344 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:22.344 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:22.344 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:30:22.344 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:53.894757+0000", 2024-09-14T06:30:22.344 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.4339000000000003e-05 2024-09-14T06:30:22.344 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:22.344 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:22.345 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:22.345 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:22.345 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:22.345 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4397.0:74249 3.0 e73/23 3:009b0225:::rbd_data.112d80ca42af.0000000000000009:head v 73'103, mlcod=71'101)", 2024-09-14T06:30:22.345 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:53.893219+0000", 2024-09-14T06:30:22.345 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 28.388329934000001, 2024-09-14T06:30:22.345 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0065381980000000003, 2024-09-14T06:30:22.345 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:30:22.345 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:30:22.345 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:30:22.345 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:30:22.345 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:22.345 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:30:22.345 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:53.893219+0000", 2024-09-14T06:30:22.345 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:22.346 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:22.346 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:22.346 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:30:22.346 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:53.893219+0000", 2024-09-14T06:30:22.346 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:22.346 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:22.346 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:22.346 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:30:22.346 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:53.893221+0000", 2024-09-14T06:30:22.346 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.4329999999999998e-06 2024-09-14T06:30:22.346 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:22.346 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:22.346 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:30:22.347 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:53.895968+0000", 2024-09-14T06:30:22.347 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.002746746 2024-09-14T06:30:22.347 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:22.347 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:22.347 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:30:22.347 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:53.895988+0000", 2024-09-14T06:30:22.347 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.0227000000000001e-05 2024-09-14T06:30:22.347 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:22.347 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:22.347 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:30:22.347 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:53.896000+0000", 2024-09-14T06:30:22.347 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.2359e-05 2024-09-14T06:30:22.347 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:22.347 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:22.347 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:30:22.347 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:53.896084+0000", 2024-09-14T06:30:22.348 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.3659999999999995e-05 2024-09-14T06:30:22.348 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:22.348 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:22.348 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:30:22.348 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:53.896109+0000", 2024-09-14T06:30:22.348 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.5259999999999999e-05 2024-09-14T06:30:22.348 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:22.348 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:22.348 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-14T06:30:22.348 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:53.899704+0000", 2024-09-14T06:30:22.348 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0035942209999999999 2024-09-14T06:30:22.348 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:22.348 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:22.348 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:30:22.348 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:53.899757+0000", 2024-09-14T06:30:22.349 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.3291999999999997e-05 2024-09-14T06:30:22.349 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:22.349 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:22.349 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:22.349 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:22.349 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:22.349 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4397.0:78263 3.0 e77/23 3:00899b22:::rbd_data.112ddfbec471.0000000000000016:head v 77'104, mlcod=73'102)", 2024-09-14T06:30:22.349 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:57.634578+0000", 2024-09-14T06:30:22.349 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 24.646970882000002, 2024-09-14T06:30:22.349 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.002372454, 2024-09-14T06:30:22.349 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:30:22.349 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:30:22.349 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:30:22.349 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:30:22.349 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:22.350 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:30:22.350 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:57.634578+0000", 2024-09-14T06:30:22.350 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:22.350 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:22.350 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:22.350 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:30:22.350 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:57.634578+0000", 2024-09-14T06:30:22.350 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:22.350 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:22.350 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:22.350 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:30:22.350 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:57.634578+0000", 2024-09-14T06:30:22.350 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.5699999999999998e-07 2024-09-14T06:30:22.350 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:22.350 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:22.351 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:30:22.351 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:57.635347+0000", 2024-09-14T06:30:22.351 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00076848300000000003 2024-09-14T06:30:22.351 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:22.351 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:22.351 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:30:22.351 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:57.635361+0000", 2024-09-14T06:30:22.351 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.4238e-05 2024-09-14T06:30:22.351 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:22.351 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:22.351 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:30:22.351 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:57.635371+0000", 2024-09-14T06:30:22.351 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.0087e-05 2024-09-14T06:30:22.351 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:22.351 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:22.351 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:30:22.352 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:57.635439+0000", 2024-09-14T06:30:22.352 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.7953000000000005e-05 2024-09-14T06:30:22.352 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:22.352 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:22.352 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:30:22.352 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:57.635464+0000", 2024-09-14T06:30:22.352 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.5062000000000002e-05 2024-09-14T06:30:22.352 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:22.352 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:22.352 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-14T06:30:22.352 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:57.636914+0000", 2024-09-14T06:30:22.352 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.001449625 2024-09-14T06:30:22.353 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:22.353 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:22.353 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:30:22.353 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:57.636950+0000", 2024-09-14T06:30:22.353 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.6548999999999999e-05 2024-09-14T06:30:22.353 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:22.353 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:22.353 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:22.353 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:22.353 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:22.353 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4397.0:78604 3.9 e77/23 3:94c62a0e:::rbd_data.112ddfbec471.000000000000002c:head v 77'96, mlcod=73'94)", 2024-09-14T06:30:22.353 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:57.677568+0000", 2024-09-14T06:30:22.354 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 24.603980493999998, 2024-09-14T06:30:22.354 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.002564846, 2024-09-14T06:30:22.354 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:30:22.354 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:30:22.354 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:30:22.354 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:30:22.354 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:22.354 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:30:22.354 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:57.677568+0000", 2024-09-14T06:30:22.354 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:22.354 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:22.354 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:22.354 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:30:22.354 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:57.677568+0000", 2024-09-14T06:30:22.355 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:22.355 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:22.355 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:22.355 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:30:22.355 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:57.677568+0000", 2024-09-14T06:30:22.355 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.8900000000000001e-07 2024-09-14T06:30:22.355 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:22.355 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:22.355 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:30:22.355 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:57.678488+0000", 2024-09-14T06:30:22.355 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00091963899999999998 2024-09-14T06:30:22.355 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:22.355 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:22.356 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:30:22.356 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:57.678500+0000", 2024-09-14T06:30:22.356 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.1902000000000001e-05 2024-09-14T06:30:22.356 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:22.356 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:22.356 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:30:22.356 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:57.678507+0000", 2024-09-14T06:30:22.356 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.7739999999999999e-06 2024-09-14T06:30:22.356 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:22.356 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:22.356 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:30:22.356 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:57.678586+0000", 2024-09-14T06:30:22.357 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.9103000000000005e-05 2024-09-14T06:30:22.357 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:22.357 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:22.357 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:30:22.357 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:57.678614+0000", 2024-09-14T06:30:22.357 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.7912e-05 2024-09-14T06:30:22.357 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:22.357 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:22.357 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-14T06:30:22.357 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:57.680075+0000", 2024-09-14T06:30:22.357 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0014613370000000001 2024-09-14T06:30:22.357 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:22.357 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:22.357 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:30:22.358 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:57.680133+0000", 2024-09-14T06:30:22.358 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.7890000000000003e-05 2024-09-14T06:30:22.358 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:22.358 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:22.358 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:22.358 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:22.358 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:22.358 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4397.0:78640 3.9 e77/23 3:97fd73eb:::rbd_data.112ddfbec471.000000000000002f:head v 77'97, mlcod=73'95)", 2024-09-14T06:30:22.358 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:57.681153+0000", 2024-09-14T06:30:22.358 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 24.600395195000001, 2024-09-14T06:30:22.358 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0016647109999999999, 2024-09-14T06:30:22.358 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:30:22.358 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:30:22.359 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:30:22.359 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:30:22.359 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:22.359 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:30:22.359 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:57.681153+0000", 2024-09-14T06:30:22.359 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:22.359 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:22.359 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:22.359 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:30:22.359 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:57.681153+0000", 2024-09-14T06:30:22.359 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:22.359 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:22.359 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:22.359 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:30:22.360 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:57.681155+0000", 2024-09-14T06:30:22.360 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.127e-06 2024-09-14T06:30:22.360 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:22.360 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:22.360 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:30:22.360 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:57.681704+0000", 2024-09-14T06:30:22.360 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000549103 2024-09-14T06:30:22.360 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:22.360 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:22.360 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:30:22.360 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:57.681733+0000", 2024-09-14T06:30:22.360 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.9641e-05 2024-09-14T06:30:22.360 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:22.360 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:22.361 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:30:22.361 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:57.681742+0000", 2024-09-14T06:30:22.361 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.9660000000000002e-06 2024-09-14T06:30:22.361 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:22.361 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:22.361 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:30:22.361 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:57.681802+0000", 2024-09-14T06:30:22.361 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.9935000000000001e-05 2024-09-14T06:30:22.361 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:22.361 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:22.361 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:30:22.362 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:57.681825+0000", 2024-09-14T06:30:22.362 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.3017999999999999e-05 2024-09-14T06:30:22.362 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:22.362 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:22.362 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-14T06:30:22.362 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:57.682777+0000", 2024-09-14T06:30:22.362 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00095157399999999995 2024-09-14T06:30:22.362 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:22.362 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:22.362 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:30:22.362 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:57.682818+0000", 2024-09-14T06:30:22.362 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.1347000000000002e-05 2024-09-14T06:30:22.362 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:22.363 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:22.363 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:22.363 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:22.363 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:22.363 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4397.0:78641 3.9 e77/23 3:914b72e4:::rbd_data.112ddfbec471.0000000000000030:head v 77'98, mlcod=73'95)", 2024-09-14T06:30:22.363 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:57.682289+0000", 2024-09-14T06:30:22.363 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 24.599259813, 2024-09-14T06:30:22.363 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0020171400000000002, 2024-09-14T06:30:22.363 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:30:22.363 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:30:22.363 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:30:22.363 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:30:22.363 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:22.364 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:30:22.364 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:57.682289+0000", 2024-09-14T06:30:22.364 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:22.364 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:22.364 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:22.364 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:30:22.364 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:57.682289+0000", 2024-09-14T06:30:22.364 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:22.364 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:22.364 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:22.364 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:30:22.364 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:57.682290+0000", 2024-09-14T06:30:22.364 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.0100000000000003e-07 2024-09-14T06:30:22.365 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:22.365 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:22.365 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:30:22.365 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:57.683230+0000", 2024-09-14T06:30:22.365 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00094047599999999999 2024-09-14T06:30:22.365 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:22.365 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:22.365 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:30:22.365 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:57.683243+0000", 2024-09-14T06:30:22.365 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.2926e-05 2024-09-14T06:30:22.365 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:22.365 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:22.365 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:30:22.366 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:57.683250+0000", 2024-09-14T06:30:22.366 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.6109999999999999e-06 2024-09-14T06:30:22.366 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:22.366 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:22.366 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:30:22.366 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:57.683313+0000", 2024-09-14T06:30:22.366 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.2943999999999999e-05 2024-09-14T06:30:22.366 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:22.366 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:22.366 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:30:22.366 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:57.683337+0000", 2024-09-14T06:30:22.366 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.4377999999999999e-05 2024-09-14T06:30:22.366 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:22.366 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:22.366 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-14T06:30:22.367 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:57.684264+0000", 2024-09-14T06:30:22.367 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00092733099999999999 2024-09-14T06:30:22.367 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:22.367 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:22.367 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:30:22.367 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:57.684306+0000", 2024-09-14T06:30:22.367 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.1573000000000003e-05 2024-09-14T06:30:22.367 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:22.367 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:22.367 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:22.367 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:22.368 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:22.368 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4397.0:113444 3.9 e96/23 3:9619bd5a:::rbd_data.112d4f226fa.0000000000000013:head v 96'128, mlcod=90'126)", 2024-09-14T06:30:22.368 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:30:17.730149+0000", 2024-09-14T06:30:22.368 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 4.5513997020000003, 2024-09-14T06:30:22.368 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0019322790000000001, 2024-09-14T06:30:22.368 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:30:22.368 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:30:22.368 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:30:22.368 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:30:22.368 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:22.368 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:30:22.368 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.730149+0000", 2024-09-14T06:30:22.368 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:22.368 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:22.369 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:22.369 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:30:22.369 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.730149+0000", 2024-09-14T06:30:22.369 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:22.369 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:22.369 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:22.369 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:30:22.369 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.730149+0000", 2024-09-14T06:30:22.369 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.0499999999999999e-07 2024-09-14T06:30:22.369 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:22.369 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:22.369 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:30:22.369 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.730852+0000", 2024-09-14T06:30:22.370 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00070294699999999999 2024-09-14T06:30:22.370 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:22.370 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:22.370 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:30:22.370 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.730867+0000", 2024-09-14T06:30:22.370 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.5294999999999998e-05 2024-09-14T06:30:22.370 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:22.370 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:22.370 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:30:22.370 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.730877+0000", 2024-09-14T06:30:22.370 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.3519999999999996e-06 2024-09-14T06:30:22.370 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:22.370 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:22.370 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:30:22.371 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.730947+0000", 2024-09-14T06:30:22.371 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.0162999999999999e-05 2024-09-14T06:30:22.371 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:22.371 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:22.371 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:30:22.371 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.730972+0000", 2024-09-14T06:30:22.371 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.4589000000000001e-05 2024-09-14T06:30:22.371 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:22.371 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:22.371 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-14T06:30:22.371 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.732034+0000", 2024-09-14T06:30:22.371 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.001062508 2024-09-14T06:30:22.371 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:22.371 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:22.371 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:30:22.372 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.732081+0000", 2024-09-14T06:30:22.372 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.7120000000000003e-05 2024-09-14T06:30:22.372 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:22.372 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:22.372 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:22.372 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:22.372 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:22.372 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-14T06:30:22.373 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-14T06:30:22.380 INFO:tasks.ceph.osd.4.smithi138.stderr:2024-09-14T06:30:22.380+0000 7f6c653c8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33615) UID: 0 2024-09-14T06:30:22.481 INFO:tasks.ceph.osd.3.smithi094.stderr:2024-09-14T06:30:22.481+0000 7feeeafe8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33655) UID: 0 2024-09-14T06:30:22.515 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-14T06:30:22.515 INFO:teuthology.orchestra.run.smithi094.stdout: "ops": [], 2024-09-14T06:30:22.515 INFO:teuthology.orchestra.run.smithi094.stdout: "num_ops": 0 2024-09-14T06:30:22.515 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-14T06:30:22.523 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-14T06:30:22.583 INFO:tasks.ceph.osd.5.smithi138.stderr:2024-09-14T06:30:22.582+0000 7fe13746f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33618) UID: 0 2024-09-14T06:30:22.605 INFO:teuthology.orchestra.run.smithi094.stderr:reweighted osd.4 to 1 (10000) 2024-09-14T06:30:22.622 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 5 1 2024-09-14T06:30:22.676 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-14T06:30:22.677 INFO:teuthology.orchestra.run.smithi094.stdout: "ops": [], 2024-09-14T06:30:22.677 INFO:teuthology.orchestra.run.smithi094.stdout: "complaint_time": 30, 2024-09-14T06:30:22.677 INFO:teuthology.orchestra.run.smithi094.stdout: "num_blocked_ops": 0 2024-09-14T06:30:22.677 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-14T06:30:22.684 INFO:tasks.ceph.osd.1.smithi094.stderr:2024-09-14T06:30:22.683+0000 7f328e47a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33641) UID: 0 2024-09-14T06:30:22.687 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-14T06:30:22.776 INFO:teuthology.orchestra.run.smithi094.stdout:checking clone #19, image image_client.0-clone19 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent20 2024-09-14T06:30:22.784 INFO:tasks.ceph.osd.5.smithi138.stderr:2024-09-14T06:30:22.783+0000 7fe13746f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33618) UID: 0 2024-09-14T06:30:22.868 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-14T06:30:22.868 INFO:teuthology.orchestra.run.smithi094.stdout: "size": 20, 2024-09-14T06:30:22.868 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 600, 2024-09-14T06:30:22.868 INFO:teuthology.orchestra.run.smithi094.stdout: "ops": [ 2024-09-14T06:30:22.868 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:22.868 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:44425 3.2 3:4603ad7f:::rbd_data.112dca2f6ae1.0000000000000005:17 [read 868352~4096 [fadvise_nocache]] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:30:22.868 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:43.854967+0000", 2024-09-14T06:30:22.869 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 39.011158227000003, 2024-09-14T06:30:22.869 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.063146025999999994, 2024-09-14T06:30:22.869 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:30:22.869 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:30:22.869 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:30:22.869 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:30:22.869 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:30:22.869 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:22.869 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 44425 2024-09-14T06:30:22.869 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:22.869 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:30:22.870 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:22.870 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:30:22.870 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.854967+0000", 2024-09-14T06:30:22.870 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:22.870 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:22.870 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:22.870 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:30:22.870 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.854967+0000", 2024-09-14T06:30:22.870 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:22.870 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:22.870 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:22.870 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:30:22.870 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.854967+0000", 2024-09-14T06:30:22.871 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.16e-07 2024-09-14T06:30:22.871 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:22.871 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:22.871 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:30:22.871 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.854968+0000", 2024-09-14T06:30:22.871 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.006e-06 2024-09-14T06:30:22.871 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:22.871 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:22.871 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:30:22.871 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.854971+0000", 2024-09-14T06:30:22.871 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.9359999999999999e-06 2024-09-14T06:30:22.871 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:22.871 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:22.871 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:30:22.872 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.854974+0000", 2024-09-14T06:30:22.872 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.1840000000000002e-06 2024-09-14T06:30:22.872 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:22.872 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:22.872 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:30:22.872 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.917945+0000", 2024-09-14T06:30:22.872 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.062970925999999997 2024-09-14T06:30:22.872 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:22.872 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:22.872 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:30:22.872 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.918030+0000", 2024-09-14T06:30:22.872 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.5863000000000001e-05 2024-09-14T06:30:22.872 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:22.872 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:22.873 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:30:22.873 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.918113+0000", 2024-09-14T06:30:22.873 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.2894999999999998e-05 2024-09-14T06:30:22.873 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:22.873 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:22.873 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:22.873 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:22.873 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:22.873 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:44426 3.2 3:4603ad7f:::rbd_data.112dca2f6ae1.0000000000000005:17 [read 892928~4096 [fadvise_nocache]] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:30:22.873 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:43.854977+0000", 2024-09-14T06:30:22.873 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 39.011148816999999, 2024-09-14T06:30:22.873 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.063369237999999994, 2024-09-14T06:30:22.873 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:30:22.873 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:30:22.874 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:30:22.874 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:30:22.874 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:30:22.874 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:22.874 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 44426 2024-09-14T06:30:22.874 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:22.874 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:30:22.874 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:22.874 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:30:22.874 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.854977+0000", 2024-09-14T06:30:22.874 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:22.874 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:22.874 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:22.875 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:30:22.875 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.854977+0000", 2024-09-14T06:30:22.875 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:22.875 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:22.875 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:22.875 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:30:22.875 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.854977+0000", 2024-09-14T06:30:22.875 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.1199999999999999e-07 2024-09-14T06:30:22.875 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:22.875 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:22.875 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:30:22.875 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.854980+0000", 2024-09-14T06:30:22.875 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.27e-06 2024-09-14T06:30:22.875 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:22.875 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:22.875 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:30:22.876 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.854983+0000", 2024-09-14T06:30:22.876 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.1650000000000002e-06 2024-09-14T06:30:22.876 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:22.876 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:22.876 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:30:22.876 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.854989+0000", 2024-09-14T06:30:22.876 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.6520000000000003e-06 2024-09-14T06:30:22.876 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:22.876 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:22.876 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:30:22.876 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.918143+0000", 2024-09-14T06:30:22.876 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.063154541999999994 2024-09-14T06:30:22.876 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:22.876 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:22.877 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:30:22.877 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.918209+0000", 2024-09-14T06:30:22.877 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.5529000000000002e-05 2024-09-14T06:30:22.877 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:22.877 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:22.877 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:30:22.877 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.918346+0000", 2024-09-14T06:30:22.877 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000136868 2024-09-14T06:30:22.877 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:22.877 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:22.877 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:22.877 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:22.877 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:22.877 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:44427 3.2 3:4603ad7f:::rbd_data.112dca2f6ae1.0000000000000005:17 [read 917504~4096 [fadvise_nocache]] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:30:22.878 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:43.854992+0000", 2024-09-14T06:30:22.878 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 39.011133170999997, 2024-09-14T06:30:22.878 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.063607792999999996, 2024-09-14T06:30:22.878 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:30:22.878 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:30:22.878 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:30:22.878 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:30:22.878 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:30:22.878 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:22.878 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 44427 2024-09-14T06:30:22.878 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:22.878 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:30:22.878 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:22.878 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:30:22.878 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.854992+0000", 2024-09-14T06:30:22.879 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:22.879 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:22.879 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:22.879 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:30:22.879 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.854992+0000", 2024-09-14T06:30:22.879 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:22.879 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:22.879 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:22.879 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:30:22.879 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.854993+0000", 2024-09-14T06:30:22.879 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.1400000000000001e-07 2024-09-14T06:30:22.879 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:22.879 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:22.879 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:30:22.879 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.854994+0000", 2024-09-14T06:30:22.880 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.091e-06 2024-09-14T06:30:22.880 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:22.880 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:22.880 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:30:22.880 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855001+0000", 2024-09-14T06:30:22.880 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.204e-06 2024-09-14T06:30:22.880 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:22.880 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:22.880 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:30:22.880 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855003+0000", 2024-09-14T06:30:22.880 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.514e-06 2024-09-14T06:30:22.880 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:22.880 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:22.880 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:30:22.881 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.918380+0000", 2024-09-14T06:30:22.881 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.063376738000000002 2024-09-14T06:30:22.881 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:22.881 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:22.881 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:30:22.881 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.918453+0000", 2024-09-14T06:30:22.881 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.3291e-05 2024-09-14T06:30:22.881 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:22.881 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:22.881 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:30:22.881 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.918600+0000", 2024-09-14T06:30:22.881 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00014674100000000001 2024-09-14T06:30:22.881 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:22.882 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:22.882 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:22.882 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:22.882 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:22.882 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:44428 3.2 3:4603ad7f:::rbd_data.112dca2f6ae1.0000000000000005:17 [read 942080~4096 [fadvise_nocache]] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:30:22.882 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:43.855006+0000", 2024-09-14T06:30:22.882 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 39.011119121, 2024-09-14T06:30:22.882 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.063846321999999997, 2024-09-14T06:30:22.882 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:30:22.882 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:30:22.882 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:30:22.882 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:30:22.882 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:30:22.882 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:22.882 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 44428 2024-09-14T06:30:22.883 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:22.883 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:30:22.883 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:22.883 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:30:22.883 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855006+0000", 2024-09-14T06:30:22.883 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:22.883 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:22.883 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:22.883 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:30:22.883 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855006+0000", 2024-09-14T06:30:22.883 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:22.883 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:22.883 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:22.883 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:30:22.883 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855007+0000", 2024-09-14T06:30:22.883 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.1199999999999999e-07 2024-09-14T06:30:22.884 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:22.884 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:22.884 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:30:22.884 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855011+0000", 2024-09-14T06:30:22.884 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.9539999999999998e-06 2024-09-14T06:30:22.884 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:22.884 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:22.884 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:30:22.884 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855014+0000", 2024-09-14T06:30:22.884 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.2930000000000001e-06 2024-09-14T06:30:22.884 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:22.884 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:22.884 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:30:22.884 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855016+0000", 2024-09-14T06:30:22.884 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.3039999999999999e-06 2024-09-14T06:30:22.885 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:22.885 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:22.885 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:30:22.885 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.918634+0000", 2024-09-14T06:30:22.885 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.063617928000000004 2024-09-14T06:30:22.885 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:22.885 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:22.885 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:30:22.885 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.918711+0000", 2024-09-14T06:30:22.885 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.6545000000000007e-05 2024-09-14T06:30:22.885 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:22.885 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:22.885 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:30:22.885 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.918853+0000", 2024-09-14T06:30:22.885 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00014208600000000001 2024-09-14T06:30:22.886 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:22.886 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:22.886 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:22.886 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:22.886 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:22.886 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:44429 3.2 3:4603ad7f:::rbd_data.112dca2f6ae1.0000000000000005:17 [read 966656~4096 [fadvise_nocache]] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:30:22.886 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:43.855019+0000", 2024-09-14T06:30:22.886 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 39.011106448, 2024-09-14T06:30:22.886 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.064059226999999996, 2024-09-14T06:30:22.886 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:30:22.886 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:30:22.886 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:30:22.886 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:30:22.886 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:30:22.886 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:22.887 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 44429 2024-09-14T06:30:22.887 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:22.887 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:30:22.887 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:22.887 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:30:22.887 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855019+0000", 2024-09-14T06:30:22.887 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:22.887 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:22.887 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:22.887 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:30:22.887 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855019+0000", 2024-09-14T06:30:22.887 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:22.887 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:22.887 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:22.887 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:30:22.888 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855019+0000", 2024-09-14T06:30:22.888 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.28e-07 2024-09-14T06:30:22.888 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:22.888 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:22.888 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:30:22.888 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855020+0000", 2024-09-14T06:30:22.888 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.0330000000000001e-06 2024-09-14T06:30:22.888 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:22.888 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:22.888 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:30:22.888 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855023+0000", 2024-09-14T06:30:22.888 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.79e-06 2024-09-14T06:30:22.888 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:22.888 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:22.889 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:30:22.889 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855025+0000", 2024-09-14T06:30:22.889 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.216e-06 2024-09-14T06:30:22.889 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:22.889 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:22.889 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:30:22.889 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.918886+0000", 2024-09-14T06:30:22.889 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.063861037999999995 2024-09-14T06:30:22.889 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:22.889 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:22.889 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:30:22.889 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.918952+0000", 2024-09-14T06:30:22.889 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.5993000000000003e-05 2024-09-14T06:30:22.889 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:22.889 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:22.890 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:30:22.890 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.919078+0000", 2024-09-14T06:30:22.890 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00012592899999999999 2024-09-14T06:30:22.890 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:22.890 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:22.890 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:22.890 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:22.890 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:22.890 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:44430 3.2 3:4603ad7f:::rbd_data.112dca2f6ae1.0000000000000005:17 [read 991232~4096 [fadvise_nocache]] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:30:22.890 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:43.855028+0000", 2024-09-14T06:30:22.890 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 39.011097438, 2024-09-14T06:30:22.890 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.064284346000000006, 2024-09-14T06:30:22.890 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:30:22.890 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:30:22.890 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:30:22.891 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:30:22.891 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:30:22.891 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:22.891 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 44430 2024-09-14T06:30:22.891 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:22.891 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:30:22.891 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:22.891 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:30:22.891 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855028+0000", 2024-09-14T06:30:22.891 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:22.891 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:22.891 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:22.891 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:30:22.891 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855028+0000", 2024-09-14T06:30:22.891 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:22.892 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:22.892 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:22.892 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:30:22.892 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855028+0000", 2024-09-14T06:30:22.892 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.1199999999999999e-07 2024-09-14T06:30:22.892 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:22.892 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:22.892 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:30:22.892 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855029+0000", 2024-09-14T06:30:22.892 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.1100000000000004e-07 2024-09-14T06:30:22.892 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:22.892 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:22.892 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:30:22.892 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855032+0000", 2024-09-14T06:30:22.892 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.7889999999999999e-06 2024-09-14T06:30:22.892 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:22.893 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:22.893 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:30:22.893 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855034+0000", 2024-09-14T06:30:22.893 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.125e-06 2024-09-14T06:30:22.893 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:22.893 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:22.893 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:30:22.893 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.919110+0000", 2024-09-14T06:30:22.893 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.064076413999999998 2024-09-14T06:30:22.893 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:22.893 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:22.893 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:30:22.893 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.919186+0000", 2024-09-14T06:30:22.893 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.5784999999999994e-05 2024-09-14T06:30:22.893 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:22.894 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:22.894 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:30:22.894 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.919312+0000", 2024-09-14T06:30:22.894 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00012611 2024-09-14T06:30:22.894 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:22.894 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:22.894 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:22.894 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:22.894 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:22.894 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:44431 3.2 3:4603ad7f:::rbd_data.112dca2f6ae1.0000000000000005:17 [read 1015808~4096 [fadvise_nocache]] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:30:22.894 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:43.855037+0000", 2024-09-14T06:30:22.894 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 39.011088569999998, 2024-09-14T06:30:22.894 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.064469755000000004, 2024-09-14T06:30:22.894 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:30:22.894 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:30:22.894 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:30:22.895 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:30:22.895 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:30:22.895 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:22.895 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 44431 2024-09-14T06:30:22.895 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:22.895 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:30:22.895 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:22.895 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:30:22.895 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855037+0000", 2024-09-14T06:30:22.895 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:22.895 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:22.895 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:22.895 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:30:22.895 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855037+0000", 2024-09-14T06:30:22.895 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:22.896 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:22.896 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:22.896 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:30:22.896 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855037+0000", 2024-09-14T06:30:22.896 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.1e-07 2024-09-14T06:30:22.896 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:22.896 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:22.896 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:30:22.896 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855038+0000", 2024-09-14T06:30:22.896 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.9400000000000004e-07 2024-09-14T06:30:22.896 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:22.896 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:22.896 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:30:22.896 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855041+0000", 2024-09-14T06:30:22.896 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.9639999999999999e-06 2024-09-14T06:30:22.897 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:22.897 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:22.897 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:30:22.897 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855043+0000", 2024-09-14T06:30:22.897 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.2259999999999999e-06 2024-09-14T06:30:22.897 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:22.897 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:22.897 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:30:22.897 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.919347+0000", 2024-09-14T06:30:22.897 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.064304091999999993 2024-09-14T06:30:22.897 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:22.897 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:22.897 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:30:22.897 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.919393+0000", 2024-09-14T06:30:22.897 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.5340999999999998e-05 2024-09-14T06:30:22.898 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:22.898 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:22.898 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:30:22.898 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.919507+0000", 2024-09-14T06:30:22.898 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000114028 2024-09-14T06:30:22.898 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:22.898 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:22.898 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:22.898 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:22.898 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:22.898 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:44432 3.2 3:4603ad7f:::rbd_data.112dca2f6ae1.0000000000000005:17 [read 1040384~4096 [fadvise_nocache]] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:30:22.898 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:43.855046+0000", 2024-09-14T06:30:22.898 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 39.011079502000001, 2024-09-14T06:30:22.898 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.064639253999999993, 2024-09-14T06:30:22.899 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:30:22.899 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:30:22.899 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:30:22.899 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:30:22.899 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:30:22.899 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:22.899 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 44432 2024-09-14T06:30:22.899 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:22.899 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:30:22.899 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:22.899 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:30:22.899 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855046+0000", 2024-09-14T06:30:22.899 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:22.899 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:22.899 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:22.899 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:30:22.900 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855046+0000", 2024-09-14T06:30:22.900 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:22.900 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:22.900 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:22.900 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:30:22.900 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855046+0000", 2024-09-14T06:30:22.900 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.05e-07 2024-09-14T06:30:22.900 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:22.900 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:22.900 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:30:22.900 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855047+0000", 2024-09-14T06:30:22.900 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.879999999999999e-07 2024-09-14T06:30:22.900 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:22.900 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:22.901 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:30:22.901 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855050+0000", 2024-09-14T06:30:22.901 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.8459999999999999e-06 2024-09-14T06:30:22.901 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:22.901 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:22.901 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:30:22.901 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855052+0000", 2024-09-14T06:30:22.901 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.136e-06 2024-09-14T06:30:22.901 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:22.901 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:22.901 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:30:22.901 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.919536+0000", 2024-09-14T06:30:22.901 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.064484132 2024-09-14T06:30:22.901 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:22.901 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:22.902 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:30:22.902 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.919611+0000", 2024-09-14T06:30:22.902 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.4407999999999996e-05 2024-09-14T06:30:22.902 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:22.902 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:22.902 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:30:22.902 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.919685+0000", 2024-09-14T06:30:22.902 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.4538999999999999e-05 2024-09-14T06:30:22.902 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:22.902 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:22.902 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:22.902 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:22.902 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:22.902 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:44518 3.2 3:4190d882:::rbd_data.112dca2f6ae1.0000000000000002:17 [read 12288~4096 [fadvise_nocache]] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:30:22.903 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:43.855055+0000", 2024-09-14T06:30:22.903 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 39.011070617999998, 2024-09-14T06:30:22.903 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.064894317000000007, 2024-09-14T06:30:22.903 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:30:22.903 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:30:22.903 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:30:22.903 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:30:22.903 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:30:22.903 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:22.903 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 44518 2024-09-14T06:30:22.903 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:22.903 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:30:22.903 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:22.903 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:30:22.903 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855055+0000", 2024-09-14T06:30:22.904 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:22.904 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:22.904 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:22.904 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:30:22.904 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855055+0000", 2024-09-14T06:30:22.904 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:22.904 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:22.904 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:22.904 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:30:22.904 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855055+0000", 2024-09-14T06:30:22.904 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.0699999999999999e-07 2024-09-14T06:30:22.904 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:22.904 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:22.904 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:30:22.905 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855056+0000", 2024-09-14T06:30:22.905 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.9400000000000004e-07 2024-09-14T06:30:22.905 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:22.905 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:22.905 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:30:22.905 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855059+0000", 2024-09-14T06:30:22.905 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.8870000000000001e-06 2024-09-14T06:30:22.905 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:22.905 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:22.905 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:30:22.905 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855061+0000", 2024-09-14T06:30:22.905 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.1919999999999999e-06 2024-09-14T06:30:22.905 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:22.905 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:22.905 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:30:22.906 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.919719+0000", 2024-09-14T06:30:22.906 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.064658013 2024-09-14T06:30:22.906 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:22.906 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:22.906 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:30:22.906 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.919788+0000", 2024-09-14T06:30:22.906 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.9282000000000005e-05 2024-09-14T06:30:22.906 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:22.906 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:22.906 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:30:22.906 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.919949+0000", 2024-09-14T06:30:22.906 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00016084199999999999 2024-09-14T06:30:22.907 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:22.907 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:22.907 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:22.907 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:22.907 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:22.907 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:44519 3.2 3:4190d882:::rbd_data.112dca2f6ae1.0000000000000002:17 [read 36864~4096 [fadvise_nocache]] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:30:22.907 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:43.855064+0000", 2024-09-14T06:30:22.907 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 39.011061640999998, 2024-09-14T06:30:22.907 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.065081865000000003, 2024-09-14T06:30:22.907 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:30:22.907 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:30:22.907 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:30:22.907 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:30:22.908 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:30:22.908 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:22.908 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 44519 2024-09-14T06:30:22.908 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:22.908 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:30:22.908 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:22.908 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:30:22.908 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855064+0000", 2024-09-14T06:30:22.908 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:22.908 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:22.908 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:22.908 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:30:22.908 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855064+0000", 2024-09-14T06:30:22.909 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:22.909 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:22.909 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:22.909 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:30:22.909 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855064+0000", 2024-09-14T06:30:22.909 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.1e-07 2024-09-14T06:30:22.909 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:22.909 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:22.909 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:30:22.909 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855065+0000", 2024-09-14T06:30:22.909 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.035e-06 2024-09-14T06:30:22.909 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:22.909 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:22.910 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:30:22.910 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855068+0000", 2024-09-14T06:30:22.910 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.8339999999999999e-06 2024-09-14T06:30:22.910 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:22.910 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:22.910 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:30:22.910 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855071+0000", 2024-09-14T06:30:22.910 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.3450000000000002e-06 2024-09-14T06:30:22.910 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:22.910 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:22.910 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:30:22.910 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.919979+0000", 2024-09-14T06:30:22.910 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.064907647999999998 2024-09-14T06:30:22.911 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:22.911 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:22.911 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:30:22.911 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.920026+0000", 2024-09-14T06:30:22.911 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.7595000000000001e-05 2024-09-14T06:30:22.911 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:22.911 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:22.911 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:30:22.911 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.920146+0000", 2024-09-14T06:30:22.911 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00011919800000000001 2024-09-14T06:30:22.911 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:22.911 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:22.911 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:22.912 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:22.912 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:22.912 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:44520 3.2 3:4190d882:::rbd_data.112dca2f6ae1.0000000000000002:17 [read 61440~4096 [fadvise_nocache]] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:30:22.912 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:43.855075+0000", 2024-09-14T06:30:22.912 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 39.011050590000004, 2024-09-14T06:30:22.912 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.065335078000000005, 2024-09-14T06:30:22.912 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:30:22.912 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:30:22.912 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:30:22.912 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:30:22.912 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:30:22.912 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:22.912 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 44520 2024-09-14T06:30:22.912 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:22.913 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:30:22.913 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:22.913 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:30:22.913 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855075+0000", 2024-09-14T06:30:22.913 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:22.913 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:22.913 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:22.913 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:30:22.913 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855075+0000", 2024-09-14T06:30:22.913 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:22.913 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:22.913 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:22.913 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:30:22.913 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855075+0000", 2024-09-14T06:30:22.913 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.1e-07 2024-09-14T06:30:22.914 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:22.914 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:22.914 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:30:22.914 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855076+0000", 2024-09-14T06:30:22.914 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.0839999999999999e-06 2024-09-14T06:30:22.914 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:22.914 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:22.914 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:30:22.914 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855079+0000", 2024-09-14T06:30:22.914 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.9359999999999999e-06 2024-09-14T06:30:22.914 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:22.914 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:22.914 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:30:22.914 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855084+0000", 2024-09-14T06:30:22.914 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.1730000000000003e-06 2024-09-14T06:30:22.915 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:22.915 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:22.915 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:30:22.915 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.920175+0000", 2024-09-14T06:30:22.915 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.065090234999999996 2024-09-14T06:30:22.915 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:22.915 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:22.915 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:30:22.915 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.920250+0000", 2024-09-14T06:30:22.915 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.5807000000000001e-05 2024-09-14T06:30:22.915 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:22.915 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:22.915 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:30:22.915 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.920410+0000", 2024-09-14T06:30:22.915 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00015963300000000001 2024-09-14T06:30:22.915 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:22.916 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:22.916 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:22.916 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:22.916 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:22.916 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:44521 3.2 3:4190d882:::rbd_data.112dca2f6ae1.0000000000000002:17 [read 86016~4096 [fadvise_nocache]] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:30:22.916 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:43.855087+0000", 2024-09-14T06:30:22.916 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 39.011038108999998, 2024-09-14T06:30:22.916 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.065563264999999996, 2024-09-14T06:30:22.916 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:30:22.916 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:30:22.916 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:30:22.916 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:30:22.916 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:30:22.916 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:22.916 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 44521 2024-09-14T06:30:22.917 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:22.917 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:30:22.917 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:22.917 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:30:22.917 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855087+0000", 2024-09-14T06:30:22.917 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:22.917 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:22.917 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:22.917 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:30:22.917 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855087+0000", 2024-09-14T06:30:22.917 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:22.917 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:22.917 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:22.917 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:30:22.917 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855088+0000", 2024-09-14T06:30:22.918 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.3699999999999999e-07 2024-09-14T06:30:22.918 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:22.918 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:22.918 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:30:22.918 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855089+0000", 2024-09-14T06:30:22.918 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.9900000000000009e-07 2024-09-14T06:30:22.918 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:22.918 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:22.918 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:30:22.918 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855091+0000", 2024-09-14T06:30:22.918 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.8940000000000002e-06 2024-09-14T06:30:22.918 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:22.918 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:22.918 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:30:22.919 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855094+0000", 2024-09-14T06:30:22.919 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.351e-06 2024-09-14T06:30:22.919 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:22.919 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:22.919 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:30:22.919 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.920437+0000", 2024-09-14T06:30:22.919 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.065343221000000007 2024-09-14T06:30:22.919 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:22.919 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:22.919 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:30:22.919 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.920481+0000", 2024-09-14T06:30:22.920 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.4431000000000002e-05 2024-09-14T06:30:22.920 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:22.920 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:22.920 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:30:22.920 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.920651+0000", 2024-09-14T06:30:22.920 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00016913200000000001 2024-09-14T06:30:22.920 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:22.920 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:22.920 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:22.920 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:22.920 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:22.920 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:44522 3.2 3:4190d882:::rbd_data.112dca2f6ae1.0000000000000002:17 [read 110592~4096 [fadvise_nocache]] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:30:22.920 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:43.855098+0000", 2024-09-14T06:30:22.920 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 39.011027882, 2024-09-14T06:30:22.921 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.065827628999999999, 2024-09-14T06:30:22.921 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:30:22.921 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:30:22.921 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:30:22.921 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:30:22.921 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:30:22.921 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:22.921 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 44522 2024-09-14T06:30:22.921 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:22.921 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:30:22.921 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:22.921 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:30:22.921 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855098+0000", 2024-09-14T06:30:22.921 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:22.921 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:22.922 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:22.922 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:30:22.922 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855098+0000", 2024-09-14T06:30:22.922 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:22.922 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:22.922 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:22.922 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:30:22.922 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855098+0000", 2024-09-14T06:30:22.922 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.1e-07 2024-09-14T06:30:22.922 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:22.922 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:22.922 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:30:22.922 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855099+0000", 2024-09-14T06:30:22.922 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.012e-06 2024-09-14T06:30:22.922 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:22.923 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:22.923 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:30:22.923 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855102+0000", 2024-09-14T06:30:22.923 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.9380000000000001e-06 2024-09-14T06:30:22.923 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:22.923 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:22.923 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:30:22.923 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855108+0000", 2024-09-14T06:30:22.923 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.4339999999999997e-06 2024-09-14T06:30:22.923 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:22.923 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:22.923 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:30:22.923 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.920681+0000", 2024-09-14T06:30:22.923 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.065572943999999994 2024-09-14T06:30:22.923 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:22.924 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:22.924 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:30:22.924 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.920724+0000", 2024-09-14T06:30:22.924 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.2877000000000003e-05 2024-09-14T06:30:22.924 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:22.924 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:22.924 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:30:22.924 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.920925+0000", 2024-09-14T06:30:22.924 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00020121399999999999 2024-09-14T06:30:22.924 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:22.924 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:22.924 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:22.924 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:22.924 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:22.925 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:44523 3.2 3:4190d882:::rbd_data.112dca2f6ae1.0000000000000002:17 [read 135168~4096 [fadvise_nocache]] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:30:22.925 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:43.855111+0000", 2024-09-14T06:30:22.925 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 39.011014160000002, 2024-09-14T06:30:22.925 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.066028198999999996, 2024-09-14T06:30:22.925 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:30:22.925 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:30:22.925 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:30:22.925 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:30:22.925 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:30:22.925 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:22.925 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 44523 2024-09-14T06:30:22.925 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:22.925 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:30:22.925 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:22.925 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:30:22.926 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855111+0000", 2024-09-14T06:30:22.926 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:22.926 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:22.926 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:22.926 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:30:22.926 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855111+0000", 2024-09-14T06:30:22.926 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:22.926 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:22.926 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:22.926 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:30:22.926 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855112+0000", 2024-09-14T06:30:22.926 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.1199999999999999e-07 2024-09-14T06:30:22.926 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:22.926 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:22.926 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:30:22.927 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855115+0000", 2024-09-14T06:30:22.927 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.3380000000000001e-06 2024-09-14T06:30:22.927 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:22.927 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:22.927 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:30:22.927 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855118+0000", 2024-09-14T06:30:22.927 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.366e-06 2024-09-14T06:30:22.927 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:22.927 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:22.927 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:30:22.927 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855121+0000", 2024-09-14T06:30:22.927 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.8329999999999998e-06 2024-09-14T06:30:22.927 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:22.927 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:22.927 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:30:22.927 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.920956+0000", 2024-09-14T06:30:22.928 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.065835251999999997 2024-09-14T06:30:22.928 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:22.928 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:22.928 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:30:22.928 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.921001+0000", 2024-09-14T06:30:22.928 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.4778999999999999e-05 2024-09-14T06:30:22.928 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:22.928 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:22.928 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:30:22.928 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.921139+0000", 2024-09-14T06:30:22.928 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000138419 2024-09-14T06:30:22.928 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:22.928 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:22.928 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:22.928 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:22.929 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:22.929 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:44524 3.2 3:4190d882:::rbd_data.112dca2f6ae1.0000000000000002:17 [read 159744~4096 [fadvise_nocache]] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:30:22.929 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:43.855124+0000", 2024-09-14T06:30:22.929 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 39.011001180999997, 2024-09-14T06:30:22.929 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.066249261000000004, 2024-09-14T06:30:22.929 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:30:22.929 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:30:22.929 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:30:22.929 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:30:22.929 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:30:22.929 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:22.929 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 44524 2024-09-14T06:30:22.929 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:22.929 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:30:22.929 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:22.930 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:30:22.930 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855124+0000", 2024-09-14T06:30:22.930 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:22.930 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:22.930 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:22.930 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:30:22.930 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855124+0000", 2024-09-14T06:30:22.930 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:22.930 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:22.930 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:22.930 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:30:22.930 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861086+0000", 2024-09-14T06:30:22.930 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0059613829999999998 2024-09-14T06:30:22.930 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:22.930 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:22.931 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:30:22.931 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861100+0000", 2024-09-14T06:30:22.931 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.4644000000000001e-05 2024-09-14T06:30:22.931 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:22.931 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:22.931 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:30:22.931 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861109+0000", 2024-09-14T06:30:22.931 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.541e-06 2024-09-14T06:30:22.931 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:22.931 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:22.931 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:30:22.931 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861117+0000", 2024-09-14T06:30:22.931 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.1049999999999995e-06 2024-09-14T06:30:22.931 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:22.931 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:22.932 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:30:22.932 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.921169+0000", 2024-09-14T06:30:22.932 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.060052405000000003 2024-09-14T06:30:22.932 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:22.932 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:22.932 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:30:22.932 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.921213+0000", 2024-09-14T06:30:22.932 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.3565e-05 2024-09-14T06:30:22.932 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:22.932 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:22.932 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:30:22.932 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.921374+0000", 2024-09-14T06:30:22.932 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00016061800000000001 2024-09-14T06:30:22.932 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:22.933 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:22.933 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:22.933 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:22.933 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:22.933 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:44536 3.2 3:4190d882:::rbd_data.112dca2f6ae1.0000000000000002:17 [read 454656~4096 [fadvise_nocache]] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:30:22.933 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:43.861384+0000", 2024-09-14T06:30:22.933 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 39.004741623999998, 2024-09-14T06:30:22.933 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.063058528000000003, 2024-09-14T06:30:22.933 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:30:22.933 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:30:22.933 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:30:22.933 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:30:22.933 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:30:22.933 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:22.933 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 44536 2024-09-14T06:30:22.934 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:22.934 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:30:22.934 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:22.934 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:30:22.934 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861384+0000", 2024-09-14T06:30:22.934 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:22.934 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:22.934 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:22.934 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:30:22.934 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861384+0000", 2024-09-14T06:30:22.934 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:22.934 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:22.934 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:22.934 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:30:22.935 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861384+0000", 2024-09-14T06:30:22.935 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.4000000000000002e-07 2024-09-14T06:30:22.935 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:22.935 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:22.935 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:30:22.935 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861386+0000", 2024-09-14T06:30:22.935 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.891e-06 2024-09-14T06:30:22.935 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:22.935 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:22.935 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:30:22.935 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861392+0000", 2024-09-14T06:30:22.935 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.9939999999999997e-06 2024-09-14T06:30:22.935 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:22.935 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:22.935 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:30:22.936 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861398+0000", 2024-09-14T06:30:22.936 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.507e-06 2024-09-14T06:30:22.936 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:22.936 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:22.936 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:30:22.936 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.924149+0000", 2024-09-14T06:30:22.936 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.062751135999999999 2024-09-14T06:30:22.936 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:22.936 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:22.936 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:30:22.936 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.924226+0000", 2024-09-14T06:30:22.936 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.7517999999999999e-05 2024-09-14T06:30:22.936 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:22.936 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:22.936 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:30:22.937 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.924442+0000", 2024-09-14T06:30:22.937 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000216042 2024-09-14T06:30:22.937 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:22.937 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:22.937 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:22.937 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:22.937 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:22.937 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:44537 3.2 3:4190d882:::rbd_data.112dca2f6ae1.0000000000000002:17 [read 479232~4096 [fadvise_nocache]] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:30:22.937 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:43.861405+0000", 2024-09-14T06:30:22.937 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 39.004720577999997, 2024-09-14T06:30:22.937 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.063351795000000002, 2024-09-14T06:30:22.937 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:30:22.937 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:30:22.938 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:30:22.938 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:30:22.938 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:30:22.938 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:22.938 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 44537 2024-09-14T06:30:22.938 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:22.938 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:30:22.938 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:22.938 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:30:22.938 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861405+0000", 2024-09-14T06:30:22.938 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:22.938 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:22.938 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:22.938 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:30:22.938 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861405+0000", 2024-09-14T06:30:22.939 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:22.939 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:22.939 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:22.939 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:30:22.939 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861405+0000", 2024-09-14T06:30:22.939 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.9099999999999999e-07 2024-09-14T06:30:22.939 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:22.939 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:22.939 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:30:22.939 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861407+0000", 2024-09-14T06:30:22.939 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.075e-06 2024-09-14T06:30:22.939 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:22.939 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:22.939 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:30:22.940 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861414+0000", 2024-09-14T06:30:22.940 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.4659999999999996e-06 2024-09-14T06:30:22.940 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:22.940 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:22.940 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:30:22.940 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861418+0000", 2024-09-14T06:30:22.940 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.5519999999999998e-06 2024-09-14T06:30:22.940 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:22.940 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:22.940 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:30:22.940 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.924474+0000", 2024-09-14T06:30:22.940 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.063056012999999994 2024-09-14T06:30:22.940 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:22.941 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:22.941 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:30:22.941 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.924556+0000", 2024-09-14T06:30:22.941 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.1181999999999996e-05 2024-09-14T06:30:22.941 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:22.941 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:22.941 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:30:22.941 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.924757+0000", 2024-09-14T06:30:22.941 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00020111599999999999 2024-09-14T06:30:22.941 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:22.941 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:22.941 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:22.941 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:22.942 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:22.942 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:44538 3.2 3:4190d882:::rbd_data.112dca2f6ae1.0000000000000002:17 [read 503808~4096 [fadvise_nocache]] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:30:22.942 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:43.861425+0000", 2024-09-14T06:30:22.942 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 39.004700536999998, 2024-09-14T06:30:22.942 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.063627125000000007, 2024-09-14T06:30:22.942 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:30:22.942 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:30:22.942 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:30:22.942 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:30:22.942 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:30:22.942 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:22.942 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 44538 2024-09-14T06:30:22.942 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:22.942 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:30:22.943 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:22.943 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:30:22.943 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861425+0000", 2024-09-14T06:30:22.943 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:22.943 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:22.943 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:22.943 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:30:22.943 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861425+0000", 2024-09-14T06:30:22.943 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:22.943 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:22.943 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:22.943 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:30:22.943 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861425+0000", 2024-09-14T06:30:22.943 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.9999999999999998e-07 2024-09-14T06:30:22.944 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:22.944 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:22.944 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:30:22.944 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861427+0000", 2024-09-14T06:30:22.944 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.9829999999999999e-06 2024-09-14T06:30:22.944 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:22.944 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:22.944 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:30:22.944 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861433+0000", 2024-09-14T06:30:22.944 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.2889999999999997e-06 2024-09-14T06:30:22.944 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:22.944 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:22.944 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:30:22.945 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861454+0000", 2024-09-14T06:30:22.945 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.1844999999999998e-05 2024-09-14T06:30:22.945 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:22.945 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:22.945 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:30:22.945 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.924786+0000", 2024-09-14T06:30:22.945 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.063331497 2024-09-14T06:30:22.945 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:22.945 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:22.945 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:30:22.945 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.924864+0000", 2024-09-14T06:30:22.945 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.7744000000000006e-05 2024-09-14T06:30:22.945 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:22.946 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:22.946 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:30:22.946 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.925052+0000", 2024-09-14T06:30:22.946 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00018836699999999999 2024-09-14T06:30:22.946 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:22.946 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:22.946 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:22.946 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:22.946 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:22.946 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:44539 3.2 3:4190d882:::rbd_data.112dca2f6ae1.0000000000000002:17 [read 528384~4096 [fadvise_nocache]] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:30:22.946 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:43.861462+0000", 2024-09-14T06:30:22.946 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 39.004663209999997, 2024-09-14T06:30:22.946 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.063855705999999998, 2024-09-14T06:30:22.946 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:30:22.947 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:30:22.947 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:30:22.947 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:30:22.947 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:30:22.947 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:22.947 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 44539 2024-09-14T06:30:22.947 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:22.947 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:30:22.947 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:22.947 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:30:22.947 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861462+0000", 2024-09-14T06:30:22.947 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:22.947 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:22.947 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:22.948 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:30:22.948 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861462+0000", 2024-09-14T06:30:22.948 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:22.948 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:22.948 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:22.948 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:30:22.948 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861463+0000", 2024-09-14T06:30:22.948 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.5200000000000002e-07 2024-09-14T06:30:22.948 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:22.948 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:22.948 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:30:22.948 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861465+0000", 2024-09-14T06:30:22.949 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.5189999999999999e-06 2024-09-14T06:30:22.949 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:22.949 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:22.949 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:30:22.949 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861471+0000", 2024-09-14T06:30:22.949 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.6899999999999997e-06 2024-09-14T06:30:22.949 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:22.949 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:22.949 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:30:22.949 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861475+0000", 2024-09-14T06:30:22.949 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.5730000000000001e-06 2024-09-14T06:30:22.949 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:22.949 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:22.950 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:30:22.950 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.925085+0000", 2024-09-14T06:30:22.950 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.063609902999999995 2024-09-14T06:30:22.950 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:22.950 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:22.950 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:30:22.950 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.925145+0000", 2024-09-14T06:30:22.950 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.9383999999999999e-05 2024-09-14T06:30:22.950 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:22.950 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:22.950 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:30:22.950 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.925318+0000", 2024-09-14T06:30:22.951 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00017318499999999999 2024-09-14T06:30:22.951 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:22.951 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:22.951 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:22.951 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:22.951 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:22.951 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:44540 3.2 3:4190d882:::rbd_data.112dca2f6ae1.0000000000000002:17 [read 552960~4096 [fadvise_nocache]] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:30:22.951 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:43.861483+0000", 2024-09-14T06:30:22.951 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 39.004642779000001, 2024-09-14T06:30:22.951 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.064065681999999999, 2024-09-14T06:30:22.951 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:30:22.951 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:30:22.951 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:30:22.952 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:30:22.952 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:30:22.952 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:22.952 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 44540 2024-09-14T06:30:22.952 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:22.952 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:30:22.952 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:22.952 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:30:22.952 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861483+0000", 2024-09-14T06:30:22.952 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:22.952 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:22.952 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:22.953 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:30:22.953 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861483+0000", 2024-09-14T06:30:22.953 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:22.953 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:22.953 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:22.953 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:30:22.953 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.867079+0000", 2024-09-14T06:30:22.953 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0055966480000000001 2024-09-14T06:30:22.953 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:22.953 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:22.953 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:30:22.953 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.867086+0000", 2024-09-14T06:30:22.953 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.387e-06 2024-09-14T06:30:22.953 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:22.954 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:22.954 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:30:22.954 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.867094+0000", 2024-09-14T06:30:22.954 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.9419999999999995e-06 2024-09-14T06:30:22.954 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:22.954 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:22.954 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:30:22.954 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.867109+0000", 2024-09-14T06:30:22.954 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.5818999999999999e-05 2024-09-14T06:30:22.954 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:22.954 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:22.954 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:30:22.954 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.925354+0000", 2024-09-14T06:30:22.954 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.058244047 2024-09-14T06:30:22.954 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:22.955 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:22.955 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:30:22.955 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.925406+0000", 2024-09-14T06:30:22.955 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.2661999999999998e-05 2024-09-14T06:30:22.955 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:22.955 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:22.955 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:30:22.955 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.925548+0000", 2024-09-14T06:30:22.955 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000142177 2024-09-14T06:30:22.955 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:22.955 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:22.955 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:22.955 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:22.955 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:22.955 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-14T06:30:22.957 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-14T06:30:22.958 INFO:tasks.ceph.osd.7.smithi138.stderr:2024-09-14T06:30:22.955+0000 7f3c6a21d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33623) UID: 0 2024-09-14T06:30:22.980 INFO:teuthology.orchestra.run.smithi094.stdout:3097 write 0x529e77a thru 0x52a7068 (0x88ef bytes) 2024-09-14T06:30:22.986 INFO:teuthology.orchestra.run.smithi094.stdout:3099 write 0x5e9c213 thru 0x5e9f841 (0x362f bytes) 2024-09-14T06:30:22.988 INFO:teuthology.orchestra.run.smithi094.stdout:3101 write 0x7d05de9 thru 0x7d09ac2 (0x3cda bytes) 2024-09-14T06:30:22.991 INFO:teuthology.orchestra.run.smithi094.stdout:3102 read 0xb4ba50f thru 0xb4bc4dd (0x1fcf bytes) 2024-09-14T06:30:22.991 INFO:teuthology.orchestra.run.smithi094.stdout:3103 write 0x1178989 thru 0x117e338 (0x59b0 bytes) 2024-09-14T06:30:22.995 INFO:teuthology.orchestra.run.smithi094.stdout:3105 writesame 0xbef27bc thru 0xbf00c3c data_size 0x5d(0xe481 bytes) 2024-09-14T06:30:22.998 INFO:teuthology.orchestra.run.smithi094.stdout:3106 write 0x7c3cf47 thru 0x7c4c97e (0xfa38 bytes) 2024-09-14T06:30:22.999 INFO:teuthology.orchestra.run.smithi094.stdout:3107 read 0x225a8ad thru 0x2260778 (0x5ecc bytes) 2024-09-14T06:30:22.999 INFO:teuthology.orchestra.run.smithi094.stdout:3108 write 0xb16a689 thru 0xb16fcc8 (0x5640 bytes) 2024-09-14T06:30:23.001 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-14T06:30:23.001 INFO:teuthology.orchestra.run.smithi094.stdout:3110 writesame 0x2f25389 thru 0x2f34d5c data_size 0x3b(0xf9d4 bytes) 2024-09-14T06:30:23.003 INFO:teuthology.orchestra.run.smithi094.stdout:3111 read 0x226d2bd thru 0x227356f (0x62b3 bytes) 2024-09-14T06:30:23.003 INFO:teuthology.orchestra.run.smithi094.stdout:3112 write 0x1ca3381 thru 0x1ca7214 (0x3e94 bytes) 2024-09-14T06:30:23.005 INFO:teuthology.orchestra.run.smithi094.stdout:3113 read 0x82d9fad thru 0x82e5187 (0xb1db bytes) 2024-09-14T06:30:23.006 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-14T06:30:23.006 INFO:teuthology.orchestra.run.smithi094.stdout:3115 write 0x2b6b376 thru 0x2b797fe (0xe489 bytes) 2024-09-14T06:30:23.007 INFO:teuthology.orchestra.run.smithi094.stdout:3116 writesame 0xaf7425a thru 0xaf7d4da data_size 0xd(0x9281 bytes) 2024-09-14T06:30:23.011 INFO:teuthology.orchestra.run.smithi094.stdout:3117 punch from 0x9488388 to 0x9492362, (0x9fda bytes) 2024-09-14T06:30:23.012 INFO:teuthology.orchestra.run.smithi094.stdout:3122 writesame 0x69492cb thru 0x6952d96 data_size 0x2(0x9acc bytes) 2024-09-14T06:30:23.021 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-14T06:30:23.022 INFO:teuthology.orchestra.run.smithi094.stdout:3124 punch from 0x26cab8d to 0x26cef1d, (0x4390 bytes) 2024-09-14T06:30:23.022 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-14T06:30:23.022 INFO:teuthology.orchestra.run.smithi094.stdout:3127 write 0x6ed411 thru 0x6efe3d (0x2a2d bytes) 2024-09-14T06:30:23.023 INFO:teuthology.orchestra.run.smithi094.stdout:3129 punch from 0x8650a78 to 0x8658259, (0x77e1 bytes) 2024-09-14T06:30:23.023 INFO:teuthology.orchestra.run.smithi094.stdout:3131 write 0x86f1a3f thru 0x86f3a8c (0x204e bytes) 2024-09-14T06:30:23.036 INFO:teuthology.orchestra.run.smithi094.stdout:3132 trunc from 0xec34b93 to 0x7347f52 2024-09-14T06:30:23.057 INFO:tasks.ceph.osd.6.smithi138.stderr:2024-09-14T06:30:23.056+0000 7f8cd8849640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33620) UID: 0 2024-09-14T06:30:23.059 INFO:teuthology.orchestra.run.smithi094.stdout:3134 write 0xafdec60 thru 0xafe3c8b (0x502c bytes) 2024-09-14T06:30:23.062 INFO:teuthology.orchestra.run.smithi094.stdout:3136 writesame 0x26907eb thru 0x269bb7d data_size 0x1(0xb393 bytes) 2024-09-14T06:30:23.078 INFO:teuthology.orchestra.run.smithi138.stdout:{ 2024-09-14T06:30:23.078 INFO:teuthology.orchestra.run.smithi138.stdout: "ops": [], 2024-09-14T06:30:23.078 INFO:teuthology.orchestra.run.smithi138.stdout: "num_ops": 0 2024-09-14T06:30:23.078 INFO:teuthology.orchestra.run.smithi138.stdout:} 2024-09-14T06:30:23.083 INFO:teuthology.orchestra.run.smithi094.stdout:3137 write 0x3775622 thru 0x3777c8c (0x266b bytes) 2024-09-14T06:30:23.086 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-14T06:30:23.088 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-14T06:30:23.093 INFO:teuthology.orchestra.run.smithi094.stdout:3138 writesame 0xd684a36 thru 0xd693c63 data_size 0x3(0xf22e bytes) 2024-09-14T06:30:23.093 INFO:teuthology.orchestra.run.smithi094.stdout:3139 read 0x957b7cc thru 0x957f967 (0x419c bytes) 2024-09-14T06:30:23.093 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-14T06:30:23.093 INFO:teuthology.orchestra.run.smithi094.stdout:3141 read 0x88e89a3 thru 0x88f08a6 (0x7f04 bytes) 2024-09-14T06:30:23.094 INFO:teuthology.orchestra.run.smithi094.stdout:3142 writesame 0x6637d91 thru 0x663879a data_size 0x5(0xa0a bytes) 2024-09-14T06:30:23.096 INFO:teuthology.orchestra.run.smithi094.stdout:3143 trunc from 0xd693c64 to 0xe1601b5 2024-09-14T06:30:23.102 INFO:teuthology.orchestra.run.smithi094.stdout:3144 writesame 0x3077cee thru 0x3084f72 data_size 0x1(0xd285 bytes) 2024-09-14T06:30:23.133 INFO:teuthology.orchestra.run.smithi094.stdout:3146 writesame 0xe835a7b thru 0xe83d087 data_size 0x1(0x760d bytes) 2024-09-14T06:30:23.133 INFO:teuthology.orchestra.run.smithi094.stdout:3147 punch from 0xa9ffc5d to 0xaa0a721, (0xaac4 bytes) 2024-09-14T06:30:23.133 INFO:teuthology.orchestra.run.smithi094.stdout:3148 write 0x790c4d7 thru 0x7912b81 (0x66ab bytes) 2024-09-14T06:30:23.142 INFO:teuthology.orchestra.run.smithi094.stdout:3149 trunc from 0xe83d088 to 0x872cb77 2024-09-14T06:30:23.156 INFO:teuthology.orchestra.run.smithi094.stdout:3150 trunc from 0x872cb77 to 0xae9ab47 2024-09-14T06:30:23.158 INFO:tasks.ceph.osd.1.smithi094.stderr:2024-09-14T06:30:23.157+0000 7f328e47a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33641) UID: 0 2024-09-14T06:30:23.169 INFO:teuthology.orchestra.run.smithi094.stdout:3153 trunc from 0xae9ab47 to 0x51e0575 2024-09-14T06:30:23.176 INFO:teuthology.orchestra.run.smithi094.stdout:3155 read 0x315ad6d thru 0x315c54a (0x17de bytes) 2024-09-14T06:30:23.182 INFO:teuthology.orchestra.run.smithi094.stdout:3157 trunc from 0x51e0575 to 0x4335d1 2024-09-14T06:30:23.195 INFO:teuthology.orchestra.run.smithi094.stdout:3158 punch from 0x5362e to 0x61597, (0xdf69 bytes) 2024-09-14T06:30:23.204 INFO:teuthology.orchestra.run.smithi094.stdout:3159 write 0x334cfac thru 0x334ffe2 (0x3037 bytes) 2024-09-14T06:30:23.220 INFO:teuthology.orchestra.run.smithi094.stdout:3160 trunc from 0x334ffe3 to 0xdb7843e 2024-09-14T06:30:23.221 INFO:teuthology.orchestra.run.smithi094.stdout:osd.0: {} 2024-09-14T06:30:23.221 INFO:teuthology.orchestra.run.smithi094.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-14T06:30:23.225 INFO:teuthology.orchestra.run.smithi094.stdout:3161 write 0x745aea3 thru 0x7460e9a (0x5ff8 bytes) 2024-09-14T06:30:23.237 INFO:teuthology.orchestra.run.smithi138.stdout:{ 2024-09-14T06:30:23.237 INFO:teuthology.orchestra.run.smithi138.stdout: "ops": [], 2024-09-14T06:30:23.237 INFO:teuthology.orchestra.run.smithi138.stdout: "complaint_time": 30, 2024-09-14T06:30:23.237 INFO:teuthology.orchestra.run.smithi138.stdout: "num_blocked_ops": 0 2024-09-14T06:30:23.237 INFO:teuthology.orchestra.run.smithi138.stdout:} 2024-09-14T06:30:23.239 INFO:teuthology.orchestra.run.smithi094.stdout:3162 trunc from 0xdb7843e to 0x45fb057 2024-09-14T06:30:23.239 INFO:teuthology.orchestra.run.smithi094.stdout:osd.1: {} 2024-09-14T06:30:23.240 INFO:teuthology.orchestra.run.smithi094.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-14T06:30:23.246 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-14T06:30:23.253 INFO:teuthology.orchestra.run.smithi094.stdout:3164 trunc from 0x45fb057 to 0xa6898b2 2024-09-14T06:30:23.256 INFO:teuthology.orchestra.run.smithi094.stdout:osd.2: {} 2024-09-14T06:30:23.257 INFO:teuthology.orchestra.run.smithi094.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-14T06:30:23.259 INFO:teuthology.orchestra.run.smithi094.stdout:3166 write 0x7833624 thru 0x783565e (0x203b bytes) 2024-09-14T06:30:23.269 INFO:teuthology.orchestra.run.smithi094.stdout:3170 trunc from 0xa6898b2 to 0x2d85231 2024-09-14T06:30:23.275 INFO:teuthology.orchestra.run.smithi094.stdout:osd.3: {} 2024-09-14T06:30:23.275 INFO:teuthology.orchestra.run.smithi094.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-14T06:30:23.290 INFO:tasks.ceph.osd.4.smithi138.stderr:2024-09-14T06:30:23.289+0000 7f6c653c8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33615) UID: 0 2024-09-14T06:30:23.290 INFO:teuthology.orchestra.run.smithi094.stdout:3172 write 0x73d6200 thru 0x73d8772 (0x2573 bytes) 2024-09-14T06:30:23.292 INFO:teuthology.orchestra.run.smithi094.stdout:3173 read 0x537560e thru 0x5380926 (0xb319 bytes) 2024-09-14T06:30:23.292 INFO:teuthology.orchestra.run.smithi094.stdout:3174 write 0x252565d thru 0x2530033 (0xa9d7 bytes) 2024-09-14T06:30:23.293 INFO:teuthology.orchestra.run.smithi094.stdout:osd.4: {} 2024-09-14T06:30:23.293 INFO:teuthology.orchestra.run.smithi094.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-14T06:30:23.294 INFO:teuthology.orchestra.run.smithi094.stdout:3176 read 0x37ef6c8 thru 0x37fda90 (0xe3c9 bytes) 2024-09-14T06:30:23.294 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-14T06:30:23.294 INFO:teuthology.orchestra.run.smithi094.stdout:3178 punch from 0x4107c1e to 0x4113b0a, (0xbeec bytes) 2024-09-14T06:30:23.294 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-14T06:30:23.302 INFO:teuthology.orchestra.run.smithi094.stdout:3180 writesame 0x8bee2e1 thru 0x8bfb682 data_size 0x2(0xd3a2 bytes) 2024-09-14T06:30:23.302 INFO:teuthology.orchestra.run.smithi094.stdout:3181 punch from 0x7a8b879 to 0x7a91f98, (0x671f bytes) 2024-09-14T06:30:23.302 INFO:teuthology.orchestra.run.smithi094.stdout:3182 writesame 0x3620716 thru 0x3621189 data_size 0x6(0xa74 bytes) 2024-09-14T06:30:23.308 INFO:teuthology.orchestra.run.smithi094.stdout:3183 trunc from 0x8bfb683 to 0xc2c13d6 2024-09-14T06:30:23.312 INFO:teuthology.orchestra.run.smithi094.stdout:osd.5: {} 2024-09-14T06:30:23.313 INFO:teuthology.orchestra.run.smithi094.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-14T06:30:23.314 INFO:teuthology.orchestra.run.smithi094.stdout:3184 write 0x9d3d8b1 thru 0x9d40544 (0x2c94 bytes) 2024-09-14T06:30:23.318 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-14T06:30:23.318 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-14T06:30:23.318 INFO:teuthology.orchestra.run.smithi094.stdout:3187 write 0x45216ab thru 0x4528ae4 (0x743a bytes) 2024-09-14T06:30:23.319 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-14T06:30:23.319 INFO:teuthology.orchestra.run.smithi094.stdout:3191 read 0xaf2fdc7 thru 0xaf30da2 (0xfdc bytes) 2024-09-14T06:30:23.319 INFO:teuthology.orchestra.run.smithi094.stdout:3193 writesame 0x76bec05 thru 0x76c4197 data_size 0x1(0x5593 bytes) 2024-09-14T06:30:23.330 INFO:teuthology.orchestra.run.smithi094.stdout:3195 write 0xab2fbbb thru 0xab36b62 (0x6fa8 bytes) 2024-09-14T06:30:23.330 INFO:teuthology.orchestra.run.smithi094.stdout:osd.6: {} 2024-09-14T06:30:23.331 INFO:teuthology.orchestra.run.smithi094.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-14T06:30:23.347 INFO:teuthology.orchestra.run.smithi094.stdout:3196 trunc from 0xc2c13d6 to 0x556abc 2024-09-14T06:30:23.348 INFO:teuthology.orchestra.run.smithi094.stdout:osd.7: {} 2024-09-14T06:30:23.348 INFO:teuthology.orchestra.run.smithi094.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-14T06:30:23.358 INFO:teuthology.orchestra.run.smithi138.stdout:ERROR: (22) Invalid argument 2024-09-14T06:30:23.358 INFO:teuthology.orchestra.run.smithi138.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:30:23.360 INFO:tasks.ceph.osd.3.smithi094.stderr:2024-09-14T06:30:23.359+0000 7feeeafe8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33655) UID: 0 2024-09-14T06:30:23.367 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-14T06:30:23.370 INFO:teuthology.orchestra.run.smithi094.stdout:3197 read 0x232539 thru 0x242053 (0xfb1b bytes) 2024-09-14T06:30:23.371 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-14T06:30:23.371 INFO:teuthology.orchestra.run.smithi094.stdout:3199 punch from 0x46da65 to 0x47da3e, (0xffd9 bytes) 2024-09-14T06:30:23.385 INFO:teuthology.orchestra.run.smithi094.stdout:3200 write 0x539c4fb thru 0x53a79c8 (0xb4ce bytes) 2024-09-14T06:30:23.387 INFO:teuthology.orchestra.run.smithi094.stdout:3201 read 0x10f5ea7 thru 0x110433a (0xe494 bytes) 2024-09-14T06:30:23.400 INFO:teuthology.orchestra.run.smithi094.stdout:3202 write 0xb190b51 thru 0xb193583 (0x2a33 bytes) 2024-09-14T06:30:23.403 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-14T06:30:23.403 INFO:teuthology.orchestra.run.smithi094.stdout:3206 write 0x134f909 thru 0x1352457 (0x2b4f bytes) 2024-09-14T06:30:23.417 INFO:teuthology.orchestra.run.smithi094.stdout:3208 trunc from 0xb193584 to 0x14596f5 2024-09-14T06:30:23.428 INFO:teuthology.orchestra.run.smithi094.stdout:3209 punch from 0x4e9f3 to 0x55bb2, (0x71bf bytes) 2024-09-14T06:30:23.450 INFO:teuthology.orchestra.run.smithi094.stdout:3210 write 0xd0fe5f6 thru 0xd1034bc (0x4ec7 bytes) 2024-09-14T06:30:23.453 INFO:teuthology.orchestra.run.smithi094.stdout:3211 write 0x6dab11f thru 0x6dba870 (0xf752 bytes) 2024-09-14T06:30:23.455 INFO:teuthology.orchestra.run.smithi094.stdout:3212 read 0x414c61f thru 0x414d2d5 (0xcb7 bytes) 2024-09-14T06:30:23.455 INFO:teuthology.orchestra.run.smithi094.stdout:3213 read 0xc1f568c thru 0xc1ff2f7 (0x9c6c bytes) 2024-09-14T06:30:23.455 INFO:teuthology.orchestra.run.smithi094.stdout:3214 read 0x20dddd thru 0x212094 (0x42b8 bytes) 2024-09-14T06:30:23.456 INFO:teuthology.orchestra.run.smithi094.stdout:3215 read 0x891c66d thru 0x8925afb (0x948f bytes) 2024-09-14T06:30:23.458 INFO:teuthology.orchestra.run.smithi094.stdout:3216 trunc from 0xd1034bd to 0xa98f138 2024-09-14T06:30:23.460 INFO:tasks.ceph.osd.2.smithi094.stderr:2024-09-14T06:30:23.460+0000 7faf3269b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33649) UID: 0 2024-09-14T06:30:23.469 INFO:teuthology.orchestra.run.smithi094.stdout:3217 punch from 0x151b5e3 to 0x15200eb, (0x4b08 bytes) 2024-09-14T06:30:23.469 INFO:teuthology.orchestra.run.smithi094.stdout:3218 write 0x43fc75a thru 0x440b641 (0xeee8 bytes) 2024-09-14T06:30:23.474 INFO:teuthology.orchestra.run.smithi094.stdout:3220 write 0x9e27b13 thru 0x9e34430 (0xc91e bytes) 2024-09-14T06:30:23.474 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-14T06:30:23.474 INFO:teuthology.orchestra.run.smithi094.stdout:3223 writesame 0x72c5bf5 thru 0x72ca058 data_size 0x6(0x4464 bytes) 2024-09-14T06:30:23.481 INFO:teuthology.orchestra.run.smithi094.stdout:3224 trunc from 0xa98f138 to 0xe5ed7dd 2024-09-14T06:30:23.487 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-14T06:30:23.487 INFO:teuthology.orchestra.run.smithi094.stdout:3227 read 0x8f04e01 thru 0x8f0a94c (0x5b4c bytes) 2024-09-14T06:30:23.487 INFO:teuthology.orchestra.run.smithi094.stdout:3228 read 0x3c549c8 thru 0x3c63e74 (0xf4ad bytes) 2024-09-14T06:30:23.487 INFO:teuthology.orchestra.run.smithi094.stdout:3229 writesame 0x80edee thru 0x81d008 data_size 0x1(0xe21b bytes) 2024-09-14T06:30:23.513 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-14T06:30:23.514 INFO:teuthology.orchestra.run.smithi094.stdout:3231 read 0x31209a0 thru 0x312f6d7 (0xed38 bytes) 2024-09-14T06:30:23.514 INFO:teuthology.orchestra.run.smithi094.stdout:3232 read 0x8de270 thru 0x8e6a42 (0x87d3 bytes) 2024-09-14T06:30:23.514 INFO:teuthology.orchestra.run.smithi094.stdout:3233 punch from 0x37c848a to 0x37d107d, (0x8bf3 bytes) 2024-09-14T06:30:23.514 INFO:teuthology.orchestra.run.smithi138.stdout:ERROR: (22) Invalid argument 2024-09-14T06:30:23.514 INFO:teuthology.orchestra.run.smithi138.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:30:23.522 INFO:teuthology.orchestra.run.smithi094.stdout:3234 trunc from 0xe5ed7dd to 0x7a17a03 2024-09-14T06:30:23.523 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-14T06:30:23.530 INFO:teuthology.orchestra.run.smithi094.stdout:3236 writesame 0x706ca5e thru 0x7074bbe data_size 0x1(0x8161 bytes) 2024-09-14T06:30:23.545 INFO:teuthology.orchestra.run.smithi094.stdout:3237 read 0x1e4ad10 thru 0x1e4d6da (0x29cb bytes) 2024-09-14T06:30:23.545 INFO:teuthology.orchestra.run.smithi094.stdout:3238 write 0xb0863 thru 0xb8f9b (0x8739 bytes) 2024-09-14T06:30:23.557 INFO:teuthology.orchestra.run.smithi094.stdout:3239 write 0xb0104e1 thru 0xb0176f3 (0x7213 bytes) 2024-09-14T06:30:23.561 INFO:tasks.ceph.osd.2.smithi094.stderr:2024-09-14T06:30:23.560+0000 7faf3269b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33649) UID: 0 2024-09-14T06:30:23.570 INFO:teuthology.orchestra.run.smithi094.stdout:3240 trunc from 0xb0176f4 to 0x263b16a 2024-09-14T06:30:23.597 INFO:teuthology.orchestra.run.smithi094.stdout:3241 trunc from 0x263b16a to 0xc6c13dd 2024-09-14T06:30:23.611 INFO:teuthology.orchestra.run.smithi094.stdout:3243 write 0xd102855 thru 0xd108aa6 (0x6252 bytes) 2024-09-14T06:30:23.613 INFO:teuthology.orchestra.run.smithi094.stderr:reweighted osd.5 to 1 (10000) 2024-09-14T06:30:23.618 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-14T06:30:23.628 INFO:teuthology.orchestra.run.smithi094.stdout:3244 writesame 0xead97dd thru 0xeadcc01 data_size 0x1(0x3425 bytes) 2024-09-14T06:30:23.628 INFO:teuthology.orchestra.run.smithi094.stdout:3245 clone 22 order 19 su 8192 sc 8 2024-09-14T06:30:23.629 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 6 1 2024-09-14T06:30:23.631 INFO:teuthology.orchestra.run.smithi138.stdout:ERROR: (22) Invalid argument 2024-09-14T06:30:23.631 INFO:teuthology.orchestra.run.smithi138.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:30:23.640 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-14T06:30:23.671 INFO:tasks.ceph.osd.0.smithi094.stderr:2024-09-14T06:30:23.670+0000 7f07c9333640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33636) UID: 0 2024-09-14T06:30:23.743 INFO:teuthology.orchestra.run.smithi138.stdout:ERROR: (22) Invalid argument 2024-09-14T06:30:23.743 INFO:teuthology.orchestra.run.smithi138.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:30:23.752 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-14T06:30:23.763 INFO:tasks.ceph.osd.3.smithi094.stderr:2024-09-14T06:30:23.762+0000 7feeeafe8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33655) UID: 0 2024-09-14T06:30:23.856 INFO:teuthology.orchestra.run.smithi138.stdout:ERROR: (22) Invalid argument 2024-09-14T06:30:23.856 INFO:teuthology.orchestra.run.smithi138.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:30:23.864 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-14T06:30:23.865 INFO:tasks.ceph.osd.1.smithi094.stderr:2024-09-14T06:30:23.863+0000 7f328e47a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33641) UID: 0 2024-09-14T06:30:23.965 INFO:tasks.ceph.osd.1.smithi094.stderr:2024-09-14T06:30:23.964+0000 7f328e47a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33641) UID: 0 2024-09-14T06:30:23.968 INFO:teuthology.orchestra.run.smithi138.stdout:ERROR: (22) Invalid argument 2024-09-14T06:30:23.968 INFO:teuthology.orchestra.run.smithi138.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:30:23.977 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-14T06:30:24.066 INFO:tasks.ceph.osd.3.smithi094.stderr:2024-09-14T06:30:24.066+0000 7feeeafe8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33655) UID: 0 2024-09-14T06:30:24.079 INFO:teuthology.orchestra.run.smithi138.stdout:ERROR: (22) Invalid argument 2024-09-14T06:30:24.080 INFO:teuthology.orchestra.run.smithi138.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:30:24.090 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-14T06:30:24.167 INFO:tasks.ceph.osd.4.smithi138.stderr:2024-09-14T06:30:24.166+0000 7f6c653c8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33615) UID: 0 2024-09-14T06:30:24.192 INFO:teuthology.orchestra.run.smithi138.stdout:ERROR: (22) Invalid argument 2024-09-14T06:30:24.192 INFO:teuthology.orchestra.run.smithi138.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:30:24.201 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-14T06:30:24.285 INFO:tasks.ceph.osd.7.smithi138.stderr:2024-09-14T06:30:24.284+0000 7f3c6a21d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33623) UID: 0 2024-09-14T06:30:24.344 INFO:teuthology.orchestra.run.smithi138.stdout:ERROR: (22) Invalid argument 2024-09-14T06:30:24.344 INFO:teuthology.orchestra.run.smithi138.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:30:24.353 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-14T06:30:24.369 INFO:tasks.ceph.osd.0.smithi094.stderr:2024-09-14T06:30:24.368+0000 7f07c9333640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33636) UID: 0 2024-09-14T06:30:24.470 INFO:tasks.ceph.osd.0.smithi094.stderr:2024-09-14T06:30:24.469+0000 7f07c9333640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33636) UID: 0 2024-09-14T06:30:24.496 INFO:teuthology.orchestra.run.smithi138.stdout:ERROR: (22) Invalid argument 2024-09-14T06:30:24.496 INFO:teuthology.orchestra.run.smithi138.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:30:24.505 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-14T06:30:24.571 INFO:tasks.ceph.osd.0.smithi094.stderr:2024-09-14T06:30:24.570+0000 7f07c9333640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33636) UID: 0 2024-09-14T06:30:24.616 INFO:teuthology.orchestra.run.smithi094.stderr:noscrub is set 2024-09-14T06:30:24.616 INFO:teuthology.orchestra.run.smithi094.stderr:reweighted osd.6 to 1 (10000) 2024-09-14T06:30:24.617 INFO:teuthology.orchestra.run.smithi094.stdout:ERROR: (22) Invalid argument 2024-09-14T06:30:24.617 INFO:teuthology.orchestra.run.smithi094.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:30:24.629 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-14T06:30:24.637 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 7 1 2024-09-14T06:30:24.679 INFO:tasks.ceph.osd.0.smithi094.stderr:2024-09-14T06:30:24.678+0000 7f07c9333640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33636) UID: 0 2024-09-14T06:30:24.774 INFO:tasks.ceph.osd.5.smithi138.stderr:2024-09-14T06:30:24.773+0000 7fe13746f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33618) UID: 0 2024-09-14T06:30:24.782 INFO:teuthology.orchestra.run.smithi094.stdout:ERROR: (22) Invalid argument 2024-09-14T06:30:24.782 INFO:teuthology.orchestra.run.smithi094.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:30:24.793 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-14T06:30:24.875 INFO:tasks.ceph.osd.0.smithi094.stderr:2024-09-14T06:30:24.874+0000 7f07c9333640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33636) UID: 0 2024-09-14T06:30:24.898 INFO:teuthology.orchestra.run.smithi094.stdout:ERROR: (22) Invalid argument 2024-09-14T06:30:24.898 INFO:teuthology.orchestra.run.smithi094.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:30:24.907 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-14T06:30:24.991 INFO:tasks.ceph.osd.2.smithi094.stderr:2024-09-14T06:30:24.990+0000 7faf3269b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33649) UID: 0 2024-09-14T06:30:25.049 INFO:teuthology.orchestra.run.smithi094.stdout:ERROR: (22) Invalid argument 2024-09-14T06:30:25.049 INFO:teuthology.orchestra.run.smithi094.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:30:25.058 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-14T06:30:25.076 INFO:tasks.ceph.osd.5.smithi138.stderr:2024-09-14T06:30:25.075+0000 7fe13746f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33618) UID: 0 2024-09-14T06:30:25.177 INFO:tasks.ceph.osd.5.smithi138.stderr:2024-09-14T06:30:25.176+0000 7fe13746f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33618) UID: 0 2024-09-14T06:30:25.201 INFO:teuthology.orchestra.run.smithi094.stdout:ERROR: (22) Invalid argument 2024-09-14T06:30:25.201 INFO:teuthology.orchestra.run.smithi094.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:30:25.210 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-14T06:30:25.277 INFO:tasks.ceph.osd.4.smithi138.stderr:2024-09-14T06:30:25.277+0000 7f6c653c8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33615) UID: 0 2024-09-14T06:30:25.312 INFO:teuthology.orchestra.run.smithi094.stdout:ERROR: (22) Invalid argument 2024-09-14T06:30:25.312 INFO:teuthology.orchestra.run.smithi094.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:30:25.320 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-14T06:30:25.360 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-14T06:30:25.409 INFO:tasks.ceph.osd.1.smithi094.stderr:2024-09-14T06:30:25.408+0000 7f328e47a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33641) UID: 0 2024-09-14T06:30:25.426 INFO:teuthology.orchestra.run.smithi094.stdout:ERROR: (22) Invalid argument 2024-09-14T06:30:25.426 INFO:teuthology.orchestra.run.smithi094.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:30:25.435 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-14T06:30:25.479 INFO:tasks.ceph.osd.7.smithi138.stderr:2024-09-14T06:30:25.478+0000 7f3c6a21d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33623) UID: 0 2024-09-14T06:30:25.487 INFO:teuthology.orchestra.run.smithi094.stdout:osd.0: {} 2024-09-14T06:30:25.487 INFO:teuthology.orchestra.run.smithi094.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-14T06:30:25.504 INFO:teuthology.orchestra.run.smithi094.stdout:osd.1: {} 2024-09-14T06:30:25.504 INFO:teuthology.orchestra.run.smithi094.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-14T06:30:25.520 INFO:teuthology.orchestra.run.smithi094.stdout:osd.2: {} 2024-09-14T06:30:25.521 INFO:teuthology.orchestra.run.smithi094.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-14T06:30:25.537 INFO:teuthology.orchestra.run.smithi094.stdout:osd.3: {} 2024-09-14T06:30:25.537 INFO:teuthology.orchestra.run.smithi094.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-14T06:30:25.554 INFO:teuthology.orchestra.run.smithi094.stdout:osd.4: {} 2024-09-14T06:30:25.554 INFO:teuthology.orchestra.run.smithi094.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-14T06:30:25.571 INFO:teuthology.orchestra.run.smithi094.stdout:osd.5: {} 2024-09-14T06:30:25.571 INFO:teuthology.orchestra.run.smithi094.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-14T06:30:25.580 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-14T06:30:25.580 INFO:teuthology.orchestra.run.smithi094.stdout: "ops": [], 2024-09-14T06:30:25.580 INFO:teuthology.orchestra.run.smithi094.stdout: "complaint_time": 30, 2024-09-14T06:30:25.581 INFO:teuthology.orchestra.run.smithi094.stdout: "num_blocked_ops": 0 2024-09-14T06:30:25.581 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-14T06:30:25.581 INFO:tasks.ceph.osd.6.smithi138.stderr:2024-09-14T06:30:25.580+0000 7f8cd8849640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33620) UID: 0 2024-09-14T06:30:25.589 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-14T06:30:25.589 INFO:teuthology.orchestra.run.smithi094.stdout:osd.6: {} 2024-09-14T06:30:25.590 INFO:teuthology.orchestra.run.smithi094.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-14T06:30:25.605 INFO:teuthology.orchestra.run.smithi094.stdout:osd.7: {} 2024-09-14T06:30:25.606 INFO:teuthology.orchestra.run.smithi094.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-14T06:30:25.623 INFO:teuthology.orchestra.run.smithi094.stderr:reweighted osd.7 to 1 (10000) 2024-09-14T06:30:25.638 INFO:tasks.ceph.ceph_manager.ceph:waiting for recovery to complete 2024-09-14T06:30:25.638 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-14T06:30:25.681 INFO:tasks.ceph.osd.4.smithi138.stderr:2024-09-14T06:30:25.680+0000 7f6c653c8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33615) UID: 0 2024-09-14T06:30:25.744 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-14T06:30:25.744 INFO:teuthology.orchestra.run.smithi094.stdout: "size": 20, 2024-09-14T06:30:25.744 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 600, 2024-09-14T06:30:25.744 INFO:teuthology.orchestra.run.smithi094.stdout: "ops": [ 2024-09-14T06:30:25.744 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:25.744 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4397.0:1848 3.4 e32/23 3:27620a21:::rbd_data.112dcdddac74.000000000000000b:head v 32'40, mlcod=30'38)", 2024-09-14T06:30:25.744 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:08.658649+0000", 2024-09-14T06:30:25.744 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 77.082805905000001, 2024-09-14T06:30:25.744 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.001862867, 2024-09-14T06:30:25.745 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:30:25.745 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:30:25.745 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:30:25.745 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:30:25.745 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:25.745 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:30:25.745 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.658649+0000", 2024-09-14T06:30:25.745 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:25.745 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:25.745 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:25.745 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:30:25.745 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.658649+0000", 2024-09-14T06:30:25.745 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:25.746 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:25.746 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:25.746 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:30:25.746 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.658651+0000", 2024-09-14T06:30:25.746 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.5650000000000001e-06 2024-09-14T06:30:25.746 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:25.746 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:25.746 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:30:25.746 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.658709+0000", 2024-09-14T06:30:25.746 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.8442000000000001e-05 2024-09-14T06:30:25.746 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:25.746 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:25.746 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:30:25.747 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.658730+0000", 2024-09-14T06:30:25.747 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.0509999999999998e-05 2024-09-14T06:30:25.747 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:25.747 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:25.747 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:30:25.747 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.658740+0000", 2024-09-14T06:30:25.747 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.8619999999999998e-06 2024-09-14T06:30:25.748 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:25.748 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:25.748 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:30:25.748 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.658813+0000", 2024-09-14T06:30:25.748 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.3578000000000006e-05 2024-09-14T06:30:25.748 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:25.748 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:25.748 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:30:25.748 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.658836+0000", 2024-09-14T06:30:25.748 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.2677e-05 2024-09-14T06:30:25.748 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:25.749 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:25.749 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-14T06:30:25.749 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.660462+0000", 2024-09-14T06:30:25.749 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00162637 2024-09-14T06:30:25.749 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:25.749 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:25.749 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:30:25.749 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.660512+0000", 2024-09-14T06:30:25.749 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.9863e-05 2024-09-14T06:30:25.749 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:25.749 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:25.749 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:25.750 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:25.750 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:25.750 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4397.0:2199 3.0 e32/23 3:05f3e980:::rbd_data.112dcdddac74.000000000000000f:head v 32'3, mlcod=31'1)", 2024-09-14T06:30:25.750 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:08.680236+0000", 2024-09-14T06:30:25.750 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 77.061219539999996, 2024-09-14T06:30:25.750 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0019068990000000001, 2024-09-14T06:30:25.750 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:30:25.750 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:30:25.750 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:30:25.750 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:30:25.750 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:25.750 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:30:25.750 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.680236+0000", 2024-09-14T06:30:25.751 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:25.751 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:25.751 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:25.751 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:30:25.751 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.680236+0000", 2024-09-14T06:30:25.751 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:25.751 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:25.751 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:25.751 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:30:25.751 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.680237+0000", 2024-09-14T06:30:25.751 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.4980000000000001e-06 2024-09-14T06:30:25.751 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:25.752 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:25.752 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:30:25.752 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.680324+0000", 2024-09-14T06:30:25.752 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.7310000000000003e-05 2024-09-14T06:30:25.752 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:25.752 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:25.752 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:30:25.752 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.680354+0000", 2024-09-14T06:30:25.752 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.97e-05 2024-09-14T06:30:25.752 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:25.752 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:25.752 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:30:25.752 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.680366+0000", 2024-09-14T06:30:25.752 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.2048999999999999e-05 2024-09-14T06:30:25.753 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:25.753 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:25.753 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:30:25.753 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.680439+0000", 2024-09-14T06:30:25.753 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.2910000000000005e-05 2024-09-14T06:30:25.753 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:25.753 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:25.753 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:30:25.753 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.680465+0000", 2024-09-14T06:30:25.753 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.5812999999999998e-05 2024-09-14T06:30:25.753 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:25.753 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:25.753 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-14T06:30:25.754 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.682094+0000", 2024-09-14T06:30:25.754 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.001628876 2024-09-14T06:30:25.754 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:25.754 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:25.754 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:30:25.754 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.682142+0000", 2024-09-14T06:30:25.754 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.8742999999999997e-05 2024-09-14T06:30:25.754 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:25.754 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:25.754 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:25.754 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:25.754 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:25.754 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4397.0:3292 3.9 e32/23 3:91dbce57:::rbd_data.112dcdddac74.0000000000000036:head v 32'10, mlcod=32'8)", 2024-09-14T06:30:25.755 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:08.740506+0000", 2024-09-14T06:30:25.755 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 77.000948828000006, 2024-09-14T06:30:25.755 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.001864166, 2024-09-14T06:30:25.755 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:30:25.755 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:30:25.755 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:30:25.755 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:30:25.755 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:25.755 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:30:25.755 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.740506+0000", 2024-09-14T06:30:25.755 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:25.755 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:25.755 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:25.756 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:30:25.756 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.740506+0000", 2024-09-14T06:30:25.756 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:25.756 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:25.756 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:25.756 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:30:25.756 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.740514+0000", 2024-09-14T06:30:25.756 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.2109999999999998e-06 2024-09-14T06:30:25.756 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:25.756 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:25.756 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:30:25.756 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.740745+0000", 2024-09-14T06:30:25.756 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000230194 2024-09-14T06:30:25.756 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:25.757 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:25.757 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:30:25.757 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.740764+0000", 2024-09-14T06:30:25.757 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.9330999999999998e-05 2024-09-14T06:30:25.757 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:25.757 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:25.757 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:30:25.757 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.740774+0000", 2024-09-14T06:30:25.757 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.6260000000000007e-06 2024-09-14T06:30:25.757 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:25.757 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:25.758 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:30:25.758 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.740839+0000", 2024-09-14T06:30:25.758 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.5314000000000005e-05 2024-09-14T06:30:25.758 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:25.758 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:25.758 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:30:25.758 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.740860+0000", 2024-09-14T06:30:25.758 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.0868000000000001e-05 2024-09-14T06:30:25.758 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:25.758 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:25.758 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-14T06:30:25.758 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.742327+0000", 2024-09-14T06:30:25.758 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0014672909999999999 2024-09-14T06:30:25.758 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:25.759 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:25.759 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:30:25.759 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.742370+0000", 2024-09-14T06:30:25.759 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.3331000000000002e-05 2024-09-14T06:30:25.759 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:25.759 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:25.759 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:25.759 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:25.759 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:25.759 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4397.0:26415 3.0 e53/23 3:05fd9be0:::rbd_data.112d38768680.0000000000000006:head v 53'21, mlcod=50'19)", 2024-09-14T06:30:25.759 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:30.721610+0000", 2024-09-14T06:30:25.759 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 55.019844759000001, 2024-09-14T06:30:25.759 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0021763160000000002, 2024-09-14T06:30:25.759 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:30:25.760 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:30:25.760 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:30:25.760 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:30:25.760 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:25.760 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:30:25.760 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:30.721610+0000", 2024-09-14T06:30:25.760 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:25.760 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:25.760 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:25.760 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:30:25.760 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:30.721610+0000", 2024-09-14T06:30:25.760 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:25.760 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:25.761 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:25.761 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:30:25.761 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:30.721611+0000", 2024-09-14T06:30:25.761 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.41e-07 2024-09-14T06:30:25.761 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:25.761 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:25.761 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:30:25.761 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:30.721905+0000", 2024-09-14T06:30:25.761 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00029394300000000001 2024-09-14T06:30:25.761 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:25.761 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:25.761 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:30:25.761 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:30.721921+0000", 2024-09-14T06:30:25.762 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.6609999999999999e-05 2024-09-14T06:30:25.762 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:25.762 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:25.762 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:30:25.762 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:30.721931+0000", 2024-09-14T06:30:25.762 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.6739999999999992e-06 2024-09-14T06:30:25.762 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:25.762 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:25.762 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:30:25.762 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:30.722001+0000", 2024-09-14T06:30:25.762 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.0549000000000005e-05 2024-09-14T06:30:25.762 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:25.762 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:25.762 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:30:25.763 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:30.722029+0000", 2024-09-14T06:30:25.763 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.7327000000000001e-05 2024-09-14T06:30:25.763 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:25.763 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:25.763 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-14T06:30:25.763 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:30.723727+0000", 2024-09-14T06:30:25.763 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0016977310000000001 2024-09-14T06:30:25.763 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:25.763 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:25.763 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:30:25.763 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:30.723787+0000", 2024-09-14T06:30:25.763 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.0140999999999999e-05 2024-09-14T06:30:25.763 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:25.763 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:25.764 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:25.764 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:25.764 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:25.764 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4397.0:36084 3.0 e56/23 3:05a48ef4:::rbd_data.112dc58639fa.0000000000000058:head v 56'29, mlcod=56'27)", 2024-09-14T06:30:25.764 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:34.508855+0000", 2024-09-14T06:30:25.764 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 51.232600005000002, 2024-09-14T06:30:25.764 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0020801140000000001, 2024-09-14T06:30:25.764 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:30:25.764 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:30:25.764 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:30:25.764 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:30:25.764 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:25.764 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:30:25.765 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:34.508855+0000", 2024-09-14T06:30:25.765 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:25.765 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:25.765 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:25.765 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:30:25.765 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:34.508855+0000", 2024-09-14T06:30:25.765 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:25.765 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:25.765 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:25.765 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:30:25.765 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:34.508856+0000", 2024-09-14T06:30:25.765 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.1909999999999999e-06 2024-09-14T06:30:25.765 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:25.766 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:25.766 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:30:25.766 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:34.509203+0000", 2024-09-14T06:30:25.766 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00034625900000000002 2024-09-14T06:30:25.766 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:25.766 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:25.766 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:30:25.766 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:34.509218+0000", 2024-09-14T06:30:25.766 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.5091e-05 2024-09-14T06:30:25.766 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:25.766 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:25.766 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:30:25.766 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:34.509227+0000", 2024-09-14T06:30:25.766 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.0850000000000003e-06 2024-09-14T06:30:25.767 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:25.767 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:25.767 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:30:25.767 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:34.509311+0000", 2024-09-14T06:30:25.767 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.4436000000000003e-05 2024-09-14T06:30:25.767 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:25.767 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:25.767 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:30:25.767 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:34.509335+0000", 2024-09-14T06:30:25.767 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.3577000000000001e-05 2024-09-14T06:30:25.767 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:25.767 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:25.767 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-14T06:30:25.767 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:34.510890+0000", 2024-09-14T06:30:25.767 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0015554259999999999 2024-09-14T06:30:25.768 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:25.768 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:25.768 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:30:25.768 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:34.510935+0000", 2024-09-14T06:30:25.768 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.5049000000000002e-05 2024-09-14T06:30:25.768 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:25.768 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:25.768 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:25.768 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:25.768 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:25.768 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4397.0:36966 3.4 e56/23 3:22008318:::rbd_data.112dc58639fa.0000000000000066:head v 56'56, mlcod=50'54)", 2024-09-14T06:30:25.768 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:34.565614+0000", 2024-09-14T06:30:25.768 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 51.175841568999999, 2024-09-14T06:30:25.768 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0018691929999999999, 2024-09-14T06:30:25.768 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:30:25.769 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:30:25.769 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:30:25.769 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:30:25.769 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:25.769 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:30:25.769 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:34.565614+0000", 2024-09-14T06:30:25.769 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:25.769 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:25.769 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:25.769 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:30:25.769 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:34.565614+0000", 2024-09-14T06:30:25.769 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:25.769 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:25.769 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:25.769 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:30:25.770 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:34.565615+0000", 2024-09-14T06:30:25.770 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.784e-06 2024-09-14T06:30:25.770 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:25.770 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:25.770 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:30:25.770 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:34.565919+0000", 2024-09-14T06:30:25.770 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00030351599999999999 2024-09-14T06:30:25.770 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:25.770 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:25.770 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:30:25.770 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:34.565933+0000", 2024-09-14T06:30:25.770 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.4465e-05 2024-09-14T06:30:25.770 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:25.771 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:25.771 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:30:25.771 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:34.565943+0000", 2024-09-14T06:30:25.771 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.7920000000000004e-06 2024-09-14T06:30:25.771 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:25.771 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:25.771 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:30:25.771 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:34.565999+0000", 2024-09-14T06:30:25.771 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.5643000000000002e-05 2024-09-14T06:30:25.771 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:25.771 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:25.771 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:30:25.771 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:34.566025+0000", 2024-09-14T06:30:25.771 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.6040000000000001e-05 2024-09-14T06:30:25.772 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:25.772 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:25.772 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-14T06:30:25.772 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:34.567430+0000", 2024-09-14T06:30:25.772 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.001405653 2024-09-14T06:30:25.772 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:25.772 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:25.772 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:30:25.772 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:34.567483+0000", 2024-09-14T06:30:25.772 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.2299999999999997e-05 2024-09-14T06:30:25.772 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:25.772 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:25.772 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:25.772 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:25.772 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:25.773 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4397.0:42285 3.9 e62/23 3:90747014:::rbd_data.112dca2f6ae1.000000000000002f:head v 62'54, mlcod=62'52)", 2024-09-14T06:30:25.773 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:40.755702+0000", 2024-09-14T06:30:25.773 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 44.985753185, 2024-09-14T06:30:25.773 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.001687596, 2024-09-14T06:30:25.773 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:30:25.773 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:30:25.773 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:30:25.773 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:30:25.773 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:25.773 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:30:25.773 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:40.755702+0000", 2024-09-14T06:30:25.773 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:25.773 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:25.773 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:25.774 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:30:25.774 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:40.755702+0000", 2024-09-14T06:30:25.774 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:25.774 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:25.774 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:25.774 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:30:25.774 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:40.755707+0000", 2024-09-14T06:30:25.774 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.4700000000000001e-06 2024-09-14T06:30:25.774 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:25.774 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:25.774 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:30:25.774 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:40.756366+0000", 2024-09-14T06:30:25.774 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00065852900000000004 2024-09-14T06:30:25.774 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:25.774 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:25.775 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:30:25.775 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:40.756383+0000", 2024-09-14T06:30:25.775 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.7275e-05 2024-09-14T06:30:25.775 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:25.775 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:25.775 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:30:25.775 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:40.756393+0000", 2024-09-14T06:30:25.775 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.4749999999999999e-06 2024-09-14T06:30:25.775 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:25.775 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:25.775 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:30:25.775 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:40.756462+0000", 2024-09-14T06:30:25.775 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.9214000000000005e-05 2024-09-14T06:30:25.775 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:25.775 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:25.776 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:30:25.776 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:40.756490+0000", 2024-09-14T06:30:25.776 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.7936000000000002e-05 2024-09-14T06:30:25.776 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:25.776 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:25.776 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-14T06:30:25.776 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:40.757343+0000", 2024-09-14T06:30:25.776 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000853611 2024-09-14T06:30:25.776 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:25.776 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:25.776 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:30:25.776 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:40.757390+0000", 2024-09-14T06:30:25.776 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.6085999999999999e-05 2024-09-14T06:30:25.776 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:25.776 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:25.777 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:25.777 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:25.777 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:25.777 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4397.0:45722 3.9 e64/23 3:91fed098:::rbd_data.112d5a737e72.0000000000000001:head v 64'61, mlcod=62'59)", 2024-09-14T06:30:25.777 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:43.864269+0000", 2024-09-14T06:30:25.777 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 41.877185930000003, 2024-09-14T06:30:25.777 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.003055224, 2024-09-14T06:30:25.777 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:30:25.777 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:30:25.777 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:30:25.777 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:30:25.777 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:25.777 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:30:25.777 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.864269+0000", 2024-09-14T06:30:25.777 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:25.777 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:25.778 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:25.778 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:30:25.778 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.864269+0000", 2024-09-14T06:30:25.778 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:25.778 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:25.778 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:25.778 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:30:25.778 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.864269+0000", 2024-09-14T06:30:25.778 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.41e-07 2024-09-14T06:30:25.778 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:25.778 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:25.778 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:30:25.778 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.865490+0000", 2024-09-14T06:30:25.778 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.001220552 2024-09-14T06:30:25.778 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:25.779 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:25.779 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:30:25.779 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.865506+0000", 2024-09-14T06:30:25.779 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.6263999999999999e-05 2024-09-14T06:30:25.779 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:25.779 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:25.779 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:30:25.779 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.865516+0000", 2024-09-14T06:30:25.779 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.0091999999999999e-05 2024-09-14T06:30:25.779 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:25.779 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:25.779 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:30:25.779 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.865583+0000", 2024-09-14T06:30:25.779 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.6676999999999999e-05 2024-09-14T06:30:25.779 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:25.779 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:25.780 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:30:25.780 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.865610+0000", 2024-09-14T06:30:25.780 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.6529999999999998e-05 2024-09-14T06:30:25.780 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:25.780 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:25.780 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-14T06:30:25.780 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.867281+0000", 2024-09-14T06:30:25.780 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.001671203 2024-09-14T06:30:25.780 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:25.780 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:25.780 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:30:25.780 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.867324+0000", 2024-09-14T06:30:25.780 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.3665000000000003e-05 2024-09-14T06:30:25.780 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:25.781 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:25.781 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:25.781 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:25.781 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:25.781 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4397.0:49627 3.4 e64/23 3:21e30d4b:::rbd_data.112d5a737e72.0000000000000017:head v 64'69, mlcod=61'67)", 2024-09-14T06:30:25.781 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:44.206446+0000", 2024-09-14T06:30:25.781 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 41.535009291999998, 2024-09-14T06:30:25.781 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0018142220000000001, 2024-09-14T06:30:25.781 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:30:25.781 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:30:25.781 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:30:25.781 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:30:25.781 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:25.781 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:30:25.782 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.206446+0000", 2024-09-14T06:30:25.782 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:25.782 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:25.782 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:25.782 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:30:25.782 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.206446+0000", 2024-09-14T06:30:25.782 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:25.782 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:25.782 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:25.783 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:30:25.783 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.206446+0000", 2024-09-14T06:30:25.783 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.4900000000000001e-07 2024-09-14T06:30:25.783 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:25.783 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:25.783 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:30:25.783 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.206797+0000", 2024-09-14T06:30:25.783 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00035112799999999998 2024-09-14T06:30:25.783 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:25.783 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:25.783 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:30:25.783 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.206816+0000", 2024-09-14T06:30:25.783 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.8977000000000001e-05 2024-09-14T06:30:25.783 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:25.783 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:25.784 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:30:25.784 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.206825+0000", 2024-09-14T06:30:25.784 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.8279999999999992e-06 2024-09-14T06:30:25.784 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:25.784 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:25.784 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:30:25.784 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.206892+0000", 2024-09-14T06:30:25.784 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.6809000000000004e-05 2024-09-14T06:30:25.784 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:25.784 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:25.784 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:30:25.784 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.206915+0000", 2024-09-14T06:30:25.784 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.315e-05 2024-09-14T06:30:25.784 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:25.785 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:25.785 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-14T06:30:25.785 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.208201+0000", 2024-09-14T06:30:25.785 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0012858819999999999 2024-09-14T06:30:25.785 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:25.785 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:25.785 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:30:25.785 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.208260+0000", 2024-09-14T06:30:25.785 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.9098999999999998e-05 2024-09-14T06:30:25.785 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:25.785 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:25.785 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:25.785 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:25.785 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:25.786 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4397.0:50003 3.9 e64/23 3:94f26c4b:::rbd_data.112d5a737e72.000000000000001f:head v 64'62, mlcod=62'60)", 2024-09-14T06:30:25.786 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:44.220413+0000", 2024-09-14T06:30:25.786 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 41.521041613999998, 2024-09-14T06:30:25.786 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0019210430000000001, 2024-09-14T06:30:25.786 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:30:25.786 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:30:25.786 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:30:25.786 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:30:25.786 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:25.786 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:30:25.786 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.220413+0000", 2024-09-14T06:30:25.786 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:25.787 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:25.787 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:25.787 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:30:25.787 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.220413+0000", 2024-09-14T06:30:25.787 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:25.787 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:25.787 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:25.787 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:30:25.787 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.220415+0000", 2024-09-14T06:30:25.787 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.575e-06 2024-09-14T06:30:25.787 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:25.787 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:25.787 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:30:25.788 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.220984+0000", 2024-09-14T06:30:25.788 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000568624 2024-09-14T06:30:25.788 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:25.788 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:25.788 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:30:25.788 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.221007+0000", 2024-09-14T06:30:25.788 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.3112000000000002e-05 2024-09-14T06:30:25.788 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:25.788 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:25.788 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:30:25.788 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.221017+0000", 2024-09-14T06:30:25.788 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.0098999999999999e-05 2024-09-14T06:30:25.788 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:25.788 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:25.788 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:30:25.789 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.221110+0000", 2024-09-14T06:30:25.789 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.3268000000000002e-05 2024-09-14T06:30:25.789 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:25.789 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:25.789 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:30:25.789 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.221131+0000", 2024-09-14T06:30:25.789 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.0905999999999999e-05 2024-09-14T06:30:25.789 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:25.789 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:25.789 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-14T06:30:25.789 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.222290+0000", 2024-09-14T06:30:25.789 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.001159408 2024-09-14T06:30:25.789 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:25.789 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:25.789 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:30:25.790 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.222335+0000", 2024-09-14T06:30:25.790 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.4051000000000002e-05 2024-09-14T06:30:25.790 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:25.790 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:25.790 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:25.790 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:25.790 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:25.790 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4397.0:50006 3.4 e64/23 3:20cfe300:::rbd_data.112d5a737e72.0000000000000019:head v 64'70, mlcod=61'68)", 2024-09-14T06:30:25.790 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:44.224218+0000", 2024-09-14T06:30:25.790 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 41.517236732999997, 2024-09-14T06:30:25.790 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0021104320000000002, 2024-09-14T06:30:25.790 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:30:25.790 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:30:25.790 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:30:25.790 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:30:25.791 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:25.791 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:30:25.791 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.224218+0000", 2024-09-14T06:30:25.791 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:25.791 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:25.791 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:25.791 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:30:25.791 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.224218+0000", 2024-09-14T06:30:25.791 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:25.791 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:25.791 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:25.791 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:30:25.791 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.224219+0000", 2024-09-14T06:30:25.791 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.0049999999999999e-06 2024-09-14T06:30:25.792 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:25.792 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:25.792 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:30:25.792 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.224326+0000", 2024-09-14T06:30:25.792 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000106848 2024-09-14T06:30:25.792 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:25.792 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:25.792 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:30:25.792 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.224345+0000", 2024-09-14T06:30:25.792 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.8587999999999999e-05 2024-09-14T06:30:25.792 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:25.792 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:25.792 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:30:25.792 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.224354+0000", 2024-09-14T06:30:25.792 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.7220000000000005e-06 2024-09-14T06:30:25.793 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:25.793 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:25.793 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:30:25.793 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.224416+0000", 2024-09-14T06:30:25.793 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.2738999999999997e-05 2024-09-14T06:30:25.793 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:25.793 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:25.793 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:30:25.793 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.224441+0000", 2024-09-14T06:30:25.793 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.4408000000000001e-05 2024-09-14T06:30:25.793 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:25.793 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:25.793 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-14T06:30:25.793 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.226295+0000", 2024-09-14T06:30:25.793 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00185428 2024-09-14T06:30:25.794 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:25.794 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:25.794 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:30:25.794 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.226329+0000", 2024-09-14T06:30:25.794 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.3841999999999999e-05 2024-09-14T06:30:25.794 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:25.794 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:25.794 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:25.794 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:25.794 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:25.794 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4397.0:51816 3.4 e64/23 3:21a826f9:::rbd_data.112d5a737e72.000000000000002d:head v 64'72, mlcod=64'70)", 2024-09-14T06:30:25.794 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:44.312072+0000", 2024-09-14T06:30:25.794 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 41.429383260000002, 2024-09-14T06:30:25.794 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0019555240000000002, 2024-09-14T06:30:25.795 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:30:25.795 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:30:25.795 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:30:25.795 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:30:25.795 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:25.795 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:30:25.795 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.312072+0000", 2024-09-14T06:30:25.795 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:25.795 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:25.795 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:25.795 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:30:25.795 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.312072+0000", 2024-09-14T06:30:25.795 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:25.795 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:25.796 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:25.796 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:30:25.796 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.312074+0000", 2024-09-14T06:30:25.796 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.9530000000000002e-06 2024-09-14T06:30:25.796 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:25.796 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:25.796 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:30:25.796 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.312378+0000", 2024-09-14T06:30:25.796 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00030392 2024-09-14T06:30:25.796 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:25.796 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:25.796 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:30:25.796 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.312399+0000", 2024-09-14T06:30:25.796 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.1134000000000001e-05 2024-09-14T06:30:25.797 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:25.797 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:25.797 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:30:25.797 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.312408+0000", 2024-09-14T06:30:25.797 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.2879999999999998e-06 2024-09-14T06:30:25.797 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:25.797 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:25.797 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:30:25.797 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.312494+0000", 2024-09-14T06:30:25.797 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.6280999999999996e-05 2024-09-14T06:30:25.797 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:25.797 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:25.797 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:30:25.797 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.312518+0000", 2024-09-14T06:30:25.798 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.3149000000000001e-05 2024-09-14T06:30:25.798 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:25.798 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:25.798 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-14T06:30:25.798 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.313964+0000", 2024-09-14T06:30:25.798 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.001446754 2024-09-14T06:30:25.798 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:25.798 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:25.798 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:30:25.798 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.314027+0000", 2024-09-14T06:30:25.798 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.3045000000000004e-05 2024-09-14T06:30:25.798 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:25.798 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:25.798 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:25.798 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:25.799 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:25.799 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4397.0:74241 3.0 e73/23 3:06e93fb9:::rbd_data.112d80ca42af.0000000000000000:head v 73'102, mlcod=71'100)", 2024-09-14T06:30:25.799 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:53.876730+0000", 2024-09-14T06:30:25.799 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 31.86472483, 2024-09-14T06:30:25.799 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0079470329999999992, 2024-09-14T06:30:25.799 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:30:25.799 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:30:25.799 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:30:25.799 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:30:25.799 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:25.799 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:30:25.799 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:53.876730+0000", 2024-09-14T06:30:25.799 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:25.799 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:25.800 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:25.800 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:30:25.800 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:53.876730+0000", 2024-09-14T06:30:25.800 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:25.800 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:25.800 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:25.800 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:30:25.800 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:53.876731+0000", 2024-09-14T06:30:25.800 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.0500000000000004e-07 2024-09-14T06:30:25.800 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:25.800 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:25.800 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:30:25.800 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:53.879616+0000", 2024-09-14T06:30:25.800 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0028856680000000001 2024-09-14T06:30:25.801 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:25.801 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:25.801 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:30:25.801 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:53.879643+0000", 2024-09-14T06:30:25.801 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.6120999999999999e-05 2024-09-14T06:30:25.801 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:25.801 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:25.801 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:30:25.801 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:53.879654+0000", 2024-09-14T06:30:25.801 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.1586999999999999e-05 2024-09-14T06:30:25.801 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:25.801 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:25.801 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:30:25.801 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:53.879756+0000", 2024-09-14T06:30:25.802 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00010223899999999999 2024-09-14T06:30:25.802 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:25.802 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:25.802 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:30:25.802 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:53.879783+0000", 2024-09-14T06:30:25.802 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.6913000000000001e-05 2024-09-14T06:30:25.802 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:25.802 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:25.802 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-14T06:30:25.802 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:53.884594+0000", 2024-09-14T06:30:25.802 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0048110729999999999 2024-09-14T06:30:25.802 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:25.802 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:25.802 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:30:25.802 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:53.884677+0000", 2024-09-14T06:30:25.802 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.2927000000000001e-05 2024-09-14T06:30:25.803 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:25.803 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:25.803 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:25.803 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:25.803 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:25.803 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4397.0:74243 3.9 e73/23 3:94977f81:::rbd_data.112d80ca42af.0000000000000002:head v 73'94, mlcod=71'92)", 2024-09-14T06:30:25.803 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:53.885156+0000", 2024-09-14T06:30:25.803 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 31.856298912, 2024-09-14T06:30:25.803 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0096009860000000006, 2024-09-14T06:30:25.803 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:30:25.803 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:30:25.803 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:30:25.803 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:30:25.803 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:25.803 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:30:25.804 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:53.885156+0000", 2024-09-14T06:30:25.804 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:25.804 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:25.804 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:25.804 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:30:25.804 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:53.885156+0000", 2024-09-14T06:30:25.804 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:25.804 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:25.804 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:25.804 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:30:25.804 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:53.885156+0000", 2024-09-14T06:30:25.804 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.23e-07 2024-09-14T06:30:25.804 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:25.805 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:25.805 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:30:25.805 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:53.889694+0000", 2024-09-14T06:30:25.805 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0045378709999999997 2024-09-14T06:30:25.805 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:25.805 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:25.805 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:30:25.805 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:53.889720+0000", 2024-09-14T06:30:25.805 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.5259e-05 2024-09-14T06:30:25.805 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:25.805 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:25.805 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:30:25.805 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:53.889731+0000", 2024-09-14T06:30:25.805 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.1739000000000001e-05 2024-09-14T06:30:25.805 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:25.806 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:25.806 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:30:25.806 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:53.889822+0000", 2024-09-14T06:30:25.806 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.1200999999999996e-05 2024-09-14T06:30:25.806 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:25.806 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:25.806 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:30:25.806 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:53.889850+0000", 2024-09-14T06:30:25.806 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.7084999999999999e-05 2024-09-14T06:30:25.806 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:25.806 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:25.806 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-14T06:30:25.806 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:53.894703+0000", 2024-09-14T06:30:25.806 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0048532690000000003 2024-09-14T06:30:25.807 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:25.807 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:25.807 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:30:25.807 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:53.894757+0000", 2024-09-14T06:30:25.807 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.4339000000000003e-05 2024-09-14T06:30:25.807 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:25.807 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:25.807 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:25.807 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:25.807 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:25.807 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4397.0:74249 3.0 e73/23 3:009b0225:::rbd_data.112d80ca42af.0000000000000009:head v 73'103, mlcod=71'101)", 2024-09-14T06:30:25.807 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:53.893219+0000", 2024-09-14T06:30:25.807 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 31.848236434, 2024-09-14T06:30:25.808 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0065381980000000003, 2024-09-14T06:30:25.808 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:30:25.808 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:30:25.808 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:30:25.808 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:30:25.808 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:25.808 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:30:25.808 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:53.893219+0000", 2024-09-14T06:30:25.808 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:25.808 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:25.808 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:25.808 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:30:25.808 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:53.893219+0000", 2024-09-14T06:30:25.808 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:25.809 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:25.809 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:25.809 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:30:25.809 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:53.893221+0000", 2024-09-14T06:30:25.809 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.4329999999999998e-06 2024-09-14T06:30:25.809 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:25.809 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:25.809 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:30:25.809 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:53.895968+0000", 2024-09-14T06:30:25.809 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.002746746 2024-09-14T06:30:25.809 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:25.809 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:25.809 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:30:25.809 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:53.895988+0000", 2024-09-14T06:30:25.810 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.0227000000000001e-05 2024-09-14T06:30:25.810 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:25.810 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:25.810 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:30:25.810 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:53.896000+0000", 2024-09-14T06:30:25.810 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.2359e-05 2024-09-14T06:30:25.810 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:25.810 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:25.810 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:30:25.810 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:53.896084+0000", 2024-09-14T06:30:25.810 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.3659999999999995e-05 2024-09-14T06:30:25.810 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:25.810 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:25.810 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:30:25.811 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:53.896109+0000", 2024-09-14T06:30:25.811 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.5259999999999999e-05 2024-09-14T06:30:25.811 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:25.811 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:25.811 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-14T06:30:25.811 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:53.899704+0000", 2024-09-14T06:30:25.811 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0035942209999999999 2024-09-14T06:30:25.811 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:25.811 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:25.811 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:30:25.811 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:53.899757+0000", 2024-09-14T06:30:25.811 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.3291999999999997e-05 2024-09-14T06:30:25.811 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:25.812 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:25.812 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:25.812 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:25.812 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:25.812 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4397.0:78263 3.0 e77/23 3:00899b22:::rbd_data.112ddfbec471.0000000000000016:head v 77'104, mlcod=73'102)", 2024-09-14T06:30:25.812 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:57.634578+0000", 2024-09-14T06:30:25.812 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 28.106877382, 2024-09-14T06:30:25.812 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.002372454, 2024-09-14T06:30:25.812 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:30:25.812 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:30:25.812 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:30:25.812 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:30:25.812 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:25.812 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:30:25.812 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:57.634578+0000", 2024-09-14T06:30:25.813 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:25.813 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:25.813 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:25.813 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:30:25.813 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:57.634578+0000", 2024-09-14T06:30:25.813 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:25.813 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:25.813 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:25.813 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:30:25.813 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:57.634578+0000", 2024-09-14T06:30:25.813 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.5699999999999998e-07 2024-09-14T06:30:25.813 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:25.813 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:25.813 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:30:25.814 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:57.635347+0000", 2024-09-14T06:30:25.814 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00076848300000000003 2024-09-14T06:30:25.814 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:25.814 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:25.814 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:30:25.814 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:57.635361+0000", 2024-09-14T06:30:25.814 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.4238e-05 2024-09-14T06:30:25.814 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:25.814 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:25.814 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:30:25.814 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:57.635371+0000", 2024-09-14T06:30:25.814 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.0087e-05 2024-09-14T06:30:25.814 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:25.814 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:25.815 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:30:25.815 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:57.635439+0000", 2024-09-14T06:30:25.815 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.7953000000000005e-05 2024-09-14T06:30:25.815 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:25.815 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:25.815 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:30:25.815 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:57.635464+0000", 2024-09-14T06:30:25.815 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.5062000000000002e-05 2024-09-14T06:30:25.815 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:25.815 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:25.815 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-14T06:30:25.815 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:57.636914+0000", 2024-09-14T06:30:25.815 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.001449625 2024-09-14T06:30:25.815 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:25.816 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:25.816 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:30:25.816 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:57.636950+0000", 2024-09-14T06:30:25.816 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.6548999999999999e-05 2024-09-14T06:30:25.816 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:25.816 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:25.816 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:25.816 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:25.816 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:25.816 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4397.0:78604 3.9 e77/23 3:94c62a0e:::rbd_data.112ddfbec471.000000000000002c:head v 77'96, mlcod=73'94)", 2024-09-14T06:30:25.816 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:57.677568+0000", 2024-09-14T06:30:25.816 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 28.063886994000001, 2024-09-14T06:30:25.816 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.002564846, 2024-09-14T06:30:25.816 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:30:25.817 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:30:25.817 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:30:25.817 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:30:25.817 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:25.817 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:30:25.817 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:57.677568+0000", 2024-09-14T06:30:25.817 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:25.817 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:25.817 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:25.817 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:30:25.817 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:57.677568+0000", 2024-09-14T06:30:25.817 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:25.817 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:25.817 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:25.818 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:30:25.818 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:57.677568+0000", 2024-09-14T06:30:25.818 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.8900000000000001e-07 2024-09-14T06:30:25.818 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:25.818 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:25.818 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:30:25.818 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:57.678488+0000", 2024-09-14T06:30:25.818 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00091963899999999998 2024-09-14T06:30:25.818 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:25.818 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:25.818 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:30:25.818 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:57.678500+0000", 2024-09-14T06:30:25.818 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.1902000000000001e-05 2024-09-14T06:30:25.818 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:25.819 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:25.819 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:30:25.819 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:57.678507+0000", 2024-09-14T06:30:25.819 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.7739999999999999e-06 2024-09-14T06:30:25.819 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:25.819 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:25.819 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:30:25.819 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:57.678586+0000", 2024-09-14T06:30:25.819 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.9103000000000005e-05 2024-09-14T06:30:25.819 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:25.819 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:25.819 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:30:25.819 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:57.678614+0000", 2024-09-14T06:30:25.819 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.7912e-05 2024-09-14T06:30:25.820 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:25.820 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:25.820 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-14T06:30:25.820 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:57.680075+0000", 2024-09-14T06:30:25.820 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0014613370000000001 2024-09-14T06:30:25.820 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:25.820 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:25.820 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:30:25.820 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:57.680133+0000", 2024-09-14T06:30:25.820 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.7890000000000003e-05 2024-09-14T06:30:25.820 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:25.820 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:25.820 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:25.821 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:25.821 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:25.821 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4397.0:78640 3.9 e77/23 3:97fd73eb:::rbd_data.112ddfbec471.000000000000002f:head v 77'97, mlcod=73'95)", 2024-09-14T06:30:25.821 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:57.681153+0000", 2024-09-14T06:30:25.821 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 28.060301695, 2024-09-14T06:30:25.821 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0016647109999999999, 2024-09-14T06:30:25.821 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:30:25.821 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:30:25.821 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:30:25.821 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:30:25.821 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:25.821 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:30:25.821 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:57.681153+0000", 2024-09-14T06:30:25.822 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:25.822 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:25.822 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:25.822 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:30:25.822 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:57.681153+0000", 2024-09-14T06:30:25.822 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:25.822 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:25.822 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:25.822 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:30:25.822 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:57.681155+0000", 2024-09-14T06:30:25.822 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.127e-06 2024-09-14T06:30:25.822 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:25.822 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:25.822 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:30:25.823 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:57.681704+0000", 2024-09-14T06:30:25.823 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000549103 2024-09-14T06:30:25.823 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:25.823 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:25.823 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:30:25.823 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:57.681733+0000", 2024-09-14T06:30:25.823 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.9641e-05 2024-09-14T06:30:25.823 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:25.823 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:25.823 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:30:25.823 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:57.681742+0000", 2024-09-14T06:30:25.823 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.9660000000000002e-06 2024-09-14T06:30:25.823 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:25.823 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:25.824 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:30:25.824 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:57.681802+0000", 2024-09-14T06:30:25.824 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.9935000000000001e-05 2024-09-14T06:30:25.824 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:25.824 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:25.824 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:30:25.824 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:57.681825+0000", 2024-09-14T06:30:25.824 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.3017999999999999e-05 2024-09-14T06:30:25.824 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:25.824 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:25.824 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-14T06:30:25.824 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:57.682777+0000", 2024-09-14T06:30:25.824 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00095157399999999995 2024-09-14T06:30:25.824 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:25.825 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:25.825 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:30:25.825 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:57.682818+0000", 2024-09-14T06:30:25.825 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.1347000000000002e-05 2024-09-14T06:30:25.825 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:25.825 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:25.825 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:25.825 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:25.825 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:25.825 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4397.0:78641 3.9 e77/23 3:914b72e4:::rbd_data.112ddfbec471.0000000000000030:head v 77'98, mlcod=73'95)", 2024-09-14T06:30:25.825 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:57.682289+0000", 2024-09-14T06:30:25.825 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 28.059166312999999, 2024-09-14T06:30:25.825 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0020171400000000002, 2024-09-14T06:30:25.825 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:30:25.826 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:30:25.826 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:30:25.826 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:30:25.826 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:25.826 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:30:25.826 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:57.682289+0000", 2024-09-14T06:30:25.826 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:25.826 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:25.826 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:25.826 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:30:25.826 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:57.682289+0000", 2024-09-14T06:30:25.826 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:25.826 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:25.826 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:25.826 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:30:25.827 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:57.682290+0000", 2024-09-14T06:30:25.827 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.0100000000000003e-07 2024-09-14T06:30:25.827 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:25.827 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:25.827 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:30:25.827 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:57.683230+0000", 2024-09-14T06:30:25.827 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00094047599999999999 2024-09-14T06:30:25.827 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:25.827 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:25.827 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:30:25.827 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:57.683243+0000", 2024-09-14T06:30:25.827 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.2926e-05 2024-09-14T06:30:25.827 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:25.827 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:25.828 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:30:25.828 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:57.683250+0000", 2024-09-14T06:30:25.828 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.6109999999999999e-06 2024-09-14T06:30:25.828 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:25.828 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:25.828 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:30:25.828 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:57.683313+0000", 2024-09-14T06:30:25.828 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.2943999999999999e-05 2024-09-14T06:30:25.828 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:25.828 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:25.828 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:30:25.828 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:57.683337+0000", 2024-09-14T06:30:25.828 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.4377999999999999e-05 2024-09-14T06:30:25.828 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:25.829 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:25.829 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-14T06:30:25.829 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:57.684264+0000", 2024-09-14T06:30:25.829 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00092733099999999999 2024-09-14T06:30:25.829 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:25.829 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:25.829 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:30:25.829 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:57.684306+0000", 2024-09-14T06:30:25.829 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.1573000000000003e-05 2024-09-14T06:30:25.829 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:25.829 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:25.829 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:25.829 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:25.830 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:25.830 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4397.0:113444 3.9 e96/23 3:9619bd5a:::rbd_data.112d4f226fa.0000000000000013:head v 96'128, mlcod=90'126)", 2024-09-14T06:30:25.830 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:30:17.730149+0000", 2024-09-14T06:30:25.830 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 8.0113062020000001, 2024-09-14T06:30:25.830 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0019322790000000001, 2024-09-14T06:30:25.830 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:30:25.830 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:30:25.830 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:30:25.830 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:30:25.830 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:25.830 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:30:25.830 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.730149+0000", 2024-09-14T06:30:25.830 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:25.830 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:25.831 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:25.831 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:30:25.831 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.730149+0000", 2024-09-14T06:30:25.831 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:25.831 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:25.831 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:25.831 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:30:25.831 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.730149+0000", 2024-09-14T06:30:25.831 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.0499999999999999e-07 2024-09-14T06:30:25.831 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:25.831 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:25.831 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:30:25.831 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.730852+0000", 2024-09-14T06:30:25.831 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00070294699999999999 2024-09-14T06:30:25.832 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:25.832 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:25.832 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:30:25.832 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.730867+0000", 2024-09-14T06:30:25.832 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.5294999999999998e-05 2024-09-14T06:30:25.832 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:25.832 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:25.832 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:30:25.832 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.730877+0000", 2024-09-14T06:30:25.832 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.3519999999999996e-06 2024-09-14T06:30:25.832 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:25.832 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:25.832 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:30:25.832 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.730947+0000", 2024-09-14T06:30:25.832 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.0162999999999999e-05 2024-09-14T06:30:25.833 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:25.833 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:25.833 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:30:25.833 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.730972+0000", 2024-09-14T06:30:25.833 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.4589000000000001e-05 2024-09-14T06:30:25.833 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:25.833 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:25.833 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-14T06:30:25.833 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.732034+0000", 2024-09-14T06:30:25.833 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.001062508 2024-09-14T06:30:25.833 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:25.833 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:25.833 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:30:25.833 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.732081+0000", 2024-09-14T06:30:25.833 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.7120000000000003e-05 2024-09-14T06:30:25.834 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:25.834 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:25.834 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:25.834 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:25.834 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:25.834 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-14T06:30:25.835 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-14T06:30:25.836 INFO:tasks.ceph.osd.4.smithi138.stderr:2024-09-14T06:30:25.834+0000 7f6c653c8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33615) UID: 0 2024-09-14T06:30:25.936 INFO:tasks.ceph.osd.3.smithi094.stderr:2024-09-14T06:30:25.935+0000 7feeeafe8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33655) UID: 0 2024-09-14T06:30:25.988 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-14T06:30:25.988 INFO:teuthology.orchestra.run.smithi094.stdout: "ops": [], 2024-09-14T06:30:25.988 INFO:teuthology.orchestra.run.smithi094.stdout: "num_ops": 0 2024-09-14T06:30:25.988 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-14T06:30:25.999 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-14T06:30:26.040 INFO:tasks.ceph.osd.0.smithi094.stderr:2024-09-14T06:30:26.039+0000 7f07c9333640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33636) UID: 0 2024-09-14T06:30:26.084 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-14T06:30:26.085 INFO:teuthology.orchestra.run.smithi094.stderr:dumped all 2024-09-14T06:30:26.100 INFO:teuthology.orchestra.run.smithi094.stdout:{"pg_ready":true,"pg_map":{"version":150,"stamp":"2024-09-14T06:30:25.626366+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1076304674,"num_objects":1273,"num_object_clones":0,"num_object_copies":2546,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1273,"num_whiteouts":0,"num_read":126140,"num_read_kb":1168852,"num_write":14922,"num_write_kb":810860,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":39,"num_bytes_recovered":1400964,"num_keys_recovered":11,"num_objects_omap":23,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":11074,"ondisk_log_size":11074,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":417556,"kb_used_data":80608,"kb_used_omap":12,"kb_used_meta":336755,"kb_avail":754557164,"statfs":{"total":773094113280,"available":772666535936,"internally_reserved":0,"allocated":82542592,"data_stored":718853824,"data_compressed":43901138,"data_compressed_allocated":57856000,"data_compressed_original":702433820,"omap_allocated":12751,"internal_metadata":344837681},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-76259709,"num_objects":-173,"num_object_clones":0,"num_object_copies":-346,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-173,"num_whiteouts":0,"num_read":3698,"num_read_kb":44877,"num_write":708,"num_write_kb":27347,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.025376"},"pg_stats":[{"pgid":"3.1f","version":"101'334","reported_seq":3778,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-14T06:30:22.776159+0000","last_change":"2024-09-14T06:30:22.609542+0000","last_active":"2024-09-14T06:30:22.776159+0000","last_peered":"2024-09-14T06:30:22.776159+0000","last_clean":"2024-09-14T06:30:22.776159+0000","last_became_active":"2024-09-14T06:29:00.579883+0000","last_became_peered":"2024-09-14T06:29:00.579883+0000","last_unstale":"2024-09-14T06:30:22.776159+0000","last_undegraded":"2024-09-14T06:30:22.776159+0000","last_fullsized":"2024-09-14T06:30:22.776159+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":334,"log_dups_size":0,"ondisk_log_size":334,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T18:04:30.063494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070539599999999995,"stat_sum":{"num_bytes":34617406,"num_objects":37,"num_object_clones":0,"num_object_copies":74,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":37,"num_whiteouts":0,"num_read":4250,"num_read_kb":37880,"num_write":488,"num_write_kb":19418,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"2a","length":"1"},{"start":"2c","length":"1"}]},{"pgid":"3.1e","version":"99'201","reported_seq":4809,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-09-14T06:30:20.627444+0000","last_change":"2024-09-14T06:30:19.605970+0000","last_active":"2024-09-14T06:30:20.627444+0000","last_peered":"2024-09-14T06:30:20.627444+0000","last_clean":"2024-09-14T06:30:20.627444+0000","last_became_active":"2024-09-14T06:29:00.580507+0000","last_became_peered":"2024-09-14T06:29:00.580507+0000","last_unstale":"2024-09-14T06:30:20.627444+0000","last_undegraded":"2024-09-14T06:30:20.627444+0000","last_fullsized":"2024-09-14T06:30:20.627444+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":201,"log_dups_size":0,"ondisk_log_size":201,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T14:51:47.269336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00099787300000000003,"stat_sum":{"num_bytes":28440908,"num_objects":33,"num_object_clones":0,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":33,"num_whiteouts":0,"num_read":4312,"num_read_kb":34438,"num_write":214,"num_write_kb":24792,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2a","length":"1"}]},{"pgid":"3.1d","version":"98'475","reported_seq":4963,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-14T06:30:23.623900+0000","last_change":"2024-09-14T06:30:22.612583+0000","last_active":"2024-09-14T06:30:23.623900+0000","last_peered":"2024-09-14T06:30:23.623900+0000","last_clean":"2024-09-14T06:30:23.623900+0000","last_became_active":"2024-09-14T06:29:00.579345+0000","last_became_peered":"2024-09-14T06:29:00.579345+0000","last_unstale":"2024-09-14T06:30:23.623900+0000","last_undegraded":"2024-09-14T06:30:23.623900+0000","last_fullsized":"2024-09-14T06:30:23.623900+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":475,"log_dups_size":0,"ondisk_log_size":475,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T15:03:36.602483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014172709999999999,"stat_sum":{"num_bytes":42985469,"num_objects":38,"num_object_clones":0,"num_object_copies":76,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":38,"num_whiteouts":0,"num_read":5627,"num_read_kb":37020,"num_write":717,"num_write_kb":24288,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2a","length":"1"},{"start":"2c","length":"1"}]},{"pgid":"3.1c","version":"99'275","reported_seq":2557,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-14T06:30:23.623753+0000","last_change":"2024-09-14T06:30:22.611576+0000","last_active":"2024-09-14T06:30:23.623753+0000","last_peered":"2024-09-14T06:30:23.623753+0000","last_clean":"2024-09-14T06:30:23.623753+0000","last_became_active":"2024-09-14T06:29:00.579807+0000","last_became_peered":"2024-09-14T06:29:00.579807+0000","last_unstale":"2024-09-14T06:30:23.623753+0000","last_undegraded":"2024-09-14T06:30:23.623753+0000","last_fullsized":"2024-09-14T06:30:23.623753+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":275,"log_dups_size":0,"ondisk_log_size":275,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T17:01:10.314780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00044109299999999999,"stat_sum":{"num_bytes":17988923,"num_objects":45,"num_object_clones":0,"num_object_copies":90,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":45,"num_whiteouts":0,"num_read":2531,"num_read_kb":28395,"num_write":335,"num_write_kb":27771,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2a","length":"1"},{"start":"2c","length":"1"}]},{"pgid":"3.1b","version":"100'146","reported_seq":2624,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-14T06:30:22.609118+0000","last_change":"2024-09-14T06:30:22.609284+0000","last_active":"2024-09-14T06:30:22.609118+0000","last_peered":"2024-09-14T06:30:22.609118+0000","last_clean":"2024-09-14T06:30:22.609118+0000","last_became_active":"2024-09-14T06:29:00.586183+0000","last_became_peered":"2024-09-14T06:29:00.586183+0000","last_unstale":"2024-09-14T06:30:22.609118+0000","last_undegraded":"2024-09-14T06:30:22.609118+0000","last_fullsized":"2024-09-14T06:30:22.609118+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":146,"log_dups_size":0,"ondisk_log_size":146,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T16:39:32.847224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069743099999999998,"stat_sum":{"num_bytes":28753401,"num_objects":51,"num_object_clones":0,"num_object_copies":102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":51,"num_whiteouts":0,"num_read":2165,"num_read_kb":28869,"num_write":165,"num_write_kb":31023,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"2a","length":"1"},{"start":"2c","length":"1"}]},{"pgid":"3.1a","version":"101'861","reported_seq":5350,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-14T06:30:23.612816+0000","last_change":"2024-09-14T06:30:22.607748+0000","last_active":"2024-09-14T06:30:23.612816+0000","last_peered":"2024-09-14T06:30:23.612816+0000","last_clean":"2024-09-14T06:30:23.612816+0000","last_became_active":"2024-09-14T06:29:00.576206+0000","last_became_peered":"2024-09-14T06:29:00.576206+0000","last_unstale":"2024-09-14T06:30:23.612816+0000","last_undegraded":"2024-09-14T06:30:23.612816+0000","last_fullsized":"2024-09-14T06:30:23.612816+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":861,"log_dups_size":0,"ondisk_log_size":861,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T11:53:22.482997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00055519200000000003,"stat_sum":{"num_bytes":39465380,"num_objects":49,"num_object_clones":0,"num_object_copies":98,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":49,"num_whiteouts":0,"num_read":7381,"num_read_kb":42158,"num_write":1224,"num_write_kb":31953,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"2a","length":"1"},{"start":"2c","length":"1"}]},{"pgid":"3.19","version":"98'251","reported_seq":2833,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-14T06:30:23.612350+0000","last_change":"2024-09-14T06:30:22.607773+0000","last_active":"2024-09-14T06:30:23.612350+0000","last_peered":"2024-09-14T06:30:23.612350+0000","last_clean":"2024-09-14T06:30:23.612350+0000","last_became_active":"2024-09-14T06:29:00.575639+0000","last_became_peered":"2024-09-14T06:29:00.575639+0000","last_unstale":"2024-09-14T06:30:23.612350+0000","last_undegraded":"2024-09-14T06:30:23.612350+0000","last_fullsized":"2024-09-14T06:30:23.612350+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":251,"log_dups_size":0,"ondisk_log_size":251,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T18:21:05.685332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00042186099999999999,"stat_sum":{"num_bytes":34299046,"num_objects":42,"num_object_clones":0,"num_object_copies":84,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":42,"num_whiteouts":0,"num_read":2835,"num_read_kb":42010,"num_write":350,"num_write_kb":26598,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2a","length":"1"},{"start":"2c","length":"1"}]},{"pgid":"3.18","version":"98'162","reported_seq":2269,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-09-14T06:30:20.594128+0000","last_change":"2024-09-14T06:30:19.605348+0000","last_active":"2024-09-14T06:30:20.594128+0000","last_peered":"2024-09-14T06:30:20.594128+0000","last_clean":"2024-09-14T06:30:20.594128+0000","last_became_active":"2024-09-14T06:29:00.578017+0000","last_became_peered":"2024-09-14T06:29:00.578017+0000","last_unstale":"2024-09-14T06:30:20.594128+0000","last_undegraded":"2024-09-14T06:30:20.594128+0000","last_fullsized":"2024-09-14T06:30:20.594128+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":162,"log_dups_size":0,"ondisk_log_size":162,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T10:21:31.247709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00052285199999999997,"stat_sum":{"num_bytes":22128117,"num_objects":35,"num_object_clones":0,"num_object_copies":70,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":35,"num_whiteouts":0,"num_read":1871,"num_read_kb":30431,"num_write":173,"num_write_kb":20338,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2a","length":"1"}]},{"pgid":"3.17","version":"98'431","reported_seq":6415,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-14T06:30:22.609174+0000","last_change":"2024-09-14T06:30:22.609358+0000","last_active":"2024-09-14T06:30:22.609174+0000","last_peered":"2024-09-14T06:30:22.609174+0000","last_clean":"2024-09-14T06:30:22.609174+0000","last_became_active":"2024-09-14T06:29:00.579714+0000","last_became_peered":"2024-09-14T06:29:00.579714+0000","last_unstale":"2024-09-14T06:30:22.609174+0000","last_undegraded":"2024-09-14T06:30:22.609174+0000","last_fullsized":"2024-09-14T06:30:22.609174+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":431,"log_dups_size":0,"ondisk_log_size":431,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T06:50:19.988471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00073233300000000005,"stat_sum":{"num_bytes":50200381,"num_objects":54,"num_object_clones":0,"num_object_copies":108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":54,"num_whiteouts":0,"num_read":6922,"num_read_kb":48124,"num_write":516,"num_write_kb":27292,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"2a","length":"1"},{"start":"2c","length":"1"}]},{"pgid":"3.16","version":"102'198","reported_seq":1240,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-14T06:30:23.628690+0000","last_change":"2024-09-14T06:30:22.612495+0000","last_active":"2024-09-14T06:30:23.628690+0000","last_peered":"2024-09-14T06:30:23.628690+0000","last_clean":"2024-09-14T06:30:23.628690+0000","last_became_active":"2024-09-14T06:29:00.580070+0000","last_became_peered":"2024-09-14T06:29:00.580070+0000","last_unstale":"2024-09-14T06:30:23.628690+0000","last_undegraded":"2024-09-14T06:30:23.628690+0000","last_fullsized":"2024-09-14T06:30:23.628690+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":198,"log_dups_size":0,"ondisk_log_size":198,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T14:14:36.261460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013373479999999999,"stat_sum":{"num_bytes":14704807,"num_objects":33,"num_object_clones":0,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":33,"num_whiteouts":0,"num_read":1219,"num_read_kb":11360,"num_write":246,"num_write_kb":14337,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2a","length":"1"},{"start":"2c","length":"1"}]},{"pgid":"3.15","version":"101'266","reported_seq":2864,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-14T06:30:23.625554+0000","last_change":"2024-09-14T06:30:22.614903+0000","last_active":"2024-09-14T06:30:23.625554+0000","last_peered":"2024-09-14T06:30:23.625554+0000","last_clean":"2024-09-14T06:30:23.625554+0000","last_became_active":"2024-09-14T06:29:00.578451+0000","last_became_peered":"2024-09-14T06:29:00.578451+0000","last_unstale":"2024-09-14T06:30:23.625554+0000","last_undegraded":"2024-09-14T06:30:23.625554+0000","last_fullsized":"2024-09-14T06:30:23.625554+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":266,"log_dups_size":0,"ondisk_log_size":266,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T17:53:54.817492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071216900000000002,"stat_sum":{"num_bytes":34640790,"num_objects":40,"num_object_clones":0,"num_object_copies":80,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":40,"num_whiteouts":0,"num_read":2459,"num_read_kb":49809,"num_write":281,"num_write_kb":36728,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"2a","length":"1"},{"start":"2c","length":"1"}]},{"pgid":"3.14","version":"98'669","reported_seq":3782,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-14T06:30:23.612653+0000","last_change":"2024-09-14T06:30:22.608051+0000","last_active":"2024-09-14T06:30:23.612653+0000","last_peered":"2024-09-14T06:30:23.612653+0000","last_clean":"2024-09-14T06:30:23.612653+0000","last_became_active":"2024-09-14T06:29:00.582595+0000","last_became_peered":"2024-09-14T06:29:00.582595+0000","last_unstale":"2024-09-14T06:30:23.612653+0000","last_undegraded":"2024-09-14T06:30:23.612653+0000","last_fullsized":"2024-09-14T06:30:23.612653+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":669,"log_dups_size":0,"ondisk_log_size":669,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T12:33:31.116704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070081199999999998,"stat_sum":{"num_bytes":34531427,"num_objects":32,"num_object_clones":0,"num_object_copies":64,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":32,"num_whiteouts":0,"num_read":6278,"num_read_kb":32936,"num_write":1041,"num_write_kb":15101,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"2a","length":"1"},{"start":"2c","length":"1"}]},{"pgid":"2.7","version":"0'0","reported_seq":178,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-14T06:30:23.616881+0000","last_change":"2024-09-14T06:28:53.625193+0000","last_active":"2024-09-14T06:30:23.616881+0000","last_peered":"2024-09-14T06:30:23.616881+0000","last_clean":"2024-09-14T06:30:23.616881+0000","last_became_active":"2024-09-14T06:28:51.223535+0000","last_became_peered":"2024-09-14T06:28:51.223535+0000","last_unstale":"2024-09-14T06:30:23.616881+0000","last_undegraded":"2024-09-14T06:30:23.616881+0000","last_fullsized":"2024-09-14T06:30:23.616881+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:28:50.205350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:28:50.205350+0000","last_clean_scrub_stamp":"2024-09-14T06:28:50.205350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T18:14:22.822245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00052660300000000005,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"98'212","reported_seq":2211,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-14T06:30:22.608745+0000","last_change":"2024-09-14T06:30:22.608745+0000","last_active":"2024-09-14T06:30:22.608745+0000","last_peered":"2024-09-14T06:30:22.608745+0000","last_clean":"2024-09-14T06:30:22.608745+0000","last_became_active":"2024-09-14T06:29:00.578283+0000","last_became_peered":"2024-09-14T06:29:00.578283+0000","last_unstale":"2024-09-14T06:30:22.608745+0000","last_undegraded":"2024-09-14T06:30:22.608745+0000","last_fullsized":"2024-09-14T06:30:22.608745+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":212,"log_dups_size":0,"ondisk_log_size":212,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T14:11:15.343781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059847500000000001,"stat_sum":{"num_bytes":24088082,"num_objects":32,"num_object_clones":0,"num_object_copies":64,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":32,"num_whiteouts":0,"num_read":1945,"num_read_kb":19588,"num_write":227,"num_write_kb":16712,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"2a","length":"1"},{"start":"2c","length":"1"}]},{"pgid":"2.6","version":"0'0","reported_seq":180,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-14T06:30:23.612577+0000","last_change":"2024-09-14T06:28:53.226122+0000","last_active":"2024-09-14T06:30:23.612577+0000","last_peered":"2024-09-14T06:30:23.612577+0000","last_clean":"2024-09-14T06:30:23.612577+0000","last_became_active":"2024-09-14T06:28:51.217205+0000","last_became_peered":"2024-09-14T06:28:51.217205+0000","last_unstale":"2024-09-14T06:30:23.612577+0000","last_undegraded":"2024-09-14T06:30:23.612577+0000","last_fullsized":"2024-09-14T06:30:23.612577+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:28:50.205350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:28:50.205350+0000","last_clean_scrub_stamp":"2024-09-14T06:28:50.205350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T15:29:19.563302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065018200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"100'329","reported_seq":1601,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-14T06:30:22.608812+0000","last_change":"2024-09-14T06:30:22.609046+0000","last_active":"2024-09-14T06:30:22.608812+0000","last_peered":"2024-09-14T06:30:22.608812+0000","last_clean":"2024-09-14T06:30:22.608812+0000","last_became_active":"2024-09-14T06:29:56.584268+0000","last_became_peered":"2024-09-14T06:29:56.584268+0000","last_unstale":"2024-09-14T06:30:22.608812+0000","last_undegraded":"2024-09-14T06:30:22.608812+0000","last_fullsized":"2024-09-14T06:30:22.608812+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":329,"log_dups_size":0,"ondisk_log_size":329,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T11:34:58.334730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000683674,"stat_sum":{"num_bytes":24555353,"num_objects":30,"num_object_clones":0,"num_object_copies":60,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":30,"num_whiteouts":0,"num_read":1738,"num_read_kb":16219,"num_write":466,"num_write_kb":18867,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":39,"num_bytes_recovered":1400964,"num_keys_recovered":11,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"2a","length":"1"},{"start":"2c","length":"1"}]},{"pgid":"2.5","version":"0'0","reported_seq":180,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-14T06:30:23.625776+0000","last_change":"2024-09-14T06:28:53.640035+0000","last_active":"2024-09-14T06:30:23.625776+0000","last_peered":"2024-09-14T06:30:23.625776+0000","last_clean":"2024-09-14T06:30:23.625776+0000","last_became_active":"2024-09-14T06:28:51.308058+0000","last_became_peered":"2024-09-14T06:28:51.308058+0000","last_unstale":"2024-09-14T06:30:23.625776+0000","last_undegraded":"2024-09-14T06:30:23.625776+0000","last_fullsized":"2024-09-14T06:30:23.625776+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:28:50.205350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:28:50.205350+0000","last_clean_scrub_stamp":"2024-09-14T06:28:50.205350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T13:35:48.544057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00037824000000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"101'190","reported_seq":1076,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-14T06:30:23.612476+0000","last_change":"2024-09-14T06:30:22.607515+0000","last_active":"2024-09-14T06:30:23.612476+0000","last_peered":"2024-09-14T06:30:23.612476+0000","last_clean":"2024-09-14T06:30:23.612476+0000","last_became_active":"2024-09-14T06:29:00.574895+0000","last_became_peered":"2024-09-14T06:29:00.574895+0000","last_unstale":"2024-09-14T06:30:23.612476+0000","last_undegraded":"2024-09-14T06:30:23.612476+0000","last_fullsized":"2024-09-14T06:30:23.612476+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":190,"log_dups_size":0,"ondisk_log_size":190,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T08:52:01.755709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00044466900000000002,"stat_sum":{"num_bytes":13236271,"num_objects":35,"num_object_clones":0,"num_object_copies":70,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":35,"num_whiteouts":0,"num_read":730,"num_read_kb":14922,"num_write":238,"num_write_kb":19950,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2a","length":"1"},{"start":"2c","length":"1"}]},{"pgid":"2.4","version":"0'0","reported_seq":180,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-14T06:30:23.612538+0000","last_change":"2024-09-14T06:28:53.226072+0000","last_active":"2024-09-14T06:30:23.612538+0000","last_peered":"2024-09-14T06:30:23.612538+0000","last_clean":"2024-09-14T06:30:23.612538+0000","last_became_active":"2024-09-14T06:28:51.308072+0000","last_became_peered":"2024-09-14T06:28:51.308072+0000","last_unstale":"2024-09-14T06:30:23.612538+0000","last_undegraded":"2024-09-14T06:30:23.612538+0000","last_fullsized":"2024-09-14T06:30:23.612538+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:28:50.205350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:28:50.205350+0000","last_clean_scrub_stamp":"2024-09-14T06:28:50.205350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T13:15:09.052328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066197399999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"98'541","reported_seq":2231,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-14T06:30:23.623910+0000","last_change":"2024-09-14T06:30:22.612828+0000","last_active":"2024-09-14T06:30:23.623910+0000","last_peered":"2024-09-14T06:30:23.623910+0000","last_clean":"2024-09-14T06:30:23.623910+0000","last_became_active":"2024-09-14T06:29:00.581874+0000","last_became_peered":"2024-09-14T06:29:00.581874+0000","last_unstale":"2024-09-14T06:30:23.623910+0000","last_undegraded":"2024-09-14T06:30:23.623910+0000","last_fullsized":"2024-09-14T06:30:23.623910+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":541,"log_dups_size":0,"ondisk_log_size":541,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T14:14:30.760746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015671820000000001,"stat_sum":{"num_bytes":27242971,"num_objects":41,"num_object_clones":0,"num_object_copies":82,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":41,"num_whiteouts":0,"num_read":2810,"num_read_kb":19328,"num_write":837,"num_write_kb":30177,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2a","length":"1"},{"start":"2c","length":"1"}]},{"pgid":"3.3","version":"99'222","reported_seq":7695,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-14T06:30:23.612660+0000","last_change":"2024-09-14T06:30:22.608139+0000","last_active":"2024-09-14T06:30:23.612660+0000","last_peered":"2024-09-14T06:30:23.612660+0000","last_clean":"2024-09-14T06:30:23.612660+0000","last_became_active":"2024-09-14T06:29:00.584678+0000","last_became_peered":"2024-09-14T06:29:00.584678+0000","last_unstale":"2024-09-14T06:30:23.612660+0000","last_undegraded":"2024-09-14T06:30:23.612660+0000","last_fullsized":"2024-09-14T06:30:23.612660+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":222,"log_dups_size":0,"ondisk_log_size":222,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T07:15:15.023452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00072175699999999995,"stat_sum":{"num_bytes":49275751,"num_objects":32,"num_object_clones":0,"num_object_copies":64,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":32,"num_whiteouts":0,"num_read":7398,"num_read_kb":63784,"num_write":218,"num_write_kb":32775,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"2a","length":"1"},{"start":"2c","length":"1"}]},{"pgid":"2.2","version":"22'2","reported_seq":182,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-14T06:30:23.623758+0000","last_change":"2024-09-14T06:28:53.231428+0000","last_active":"2024-09-14T06:30:23.623758+0000","last_peered":"2024-09-14T06:30:23.623758+0000","last_clean":"2024-09-14T06:30:23.623758+0000","last_became_active":"2024-09-14T06:28:51.222937+0000","last_became_peered":"2024-09-14T06:28:51.222937+0000","last_unstale":"2024-09-14T06:30:23.623758+0000","last_undegraded":"2024-09-14T06:30:23.623758+0000","last_fullsized":"2024-09-14T06:30:23.623758+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:28:50.205350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:28:50.205350+0000","last_clean_scrub_stamp":"2024-09-14T06:28:50.205350+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T14:27:37.233976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010072970000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.0","version":"101'175","reported_seq":3918,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-14T06:30:23.612343+0000","last_change":"2024-09-14T06:30:22.607287+0000","last_active":"2024-09-14T06:30:23.612343+0000","last_peered":"2024-09-14T06:30:23.612343+0000","last_clean":"2024-09-14T06:30:23.612343+0000","last_became_active":"2024-09-14T06:29:00.574873+0000","last_became_peered":"2024-09-14T06:29:00.574873+0000","last_unstale":"2024-09-14T06:30:23.612343+0000","last_undegraded":"2024-09-14T06:30:23.612343+0000","last_fullsized":"2024-09-14T06:30:23.612343+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":175,"log_dups_size":0,"ondisk_log_size":175,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T08:41:53.465666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054049399999999998,"stat_sum":{"num_bytes":43263336,"num_objects":50,"num_object_clones":0,"num_object_copies":100,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":50,"num_whiteouts":0,"num_read":3438,"num_read_kb":52268,"num_write":221,"num_write_kb":31522,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2a","length":"1"},{"start":"2c","length":"1"}]},{"pgid":"2.1","version":"0'0","reported_seq":177,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-14T06:30:22.701684+0000","last_change":"2024-09-14T06:28:53.226264+0000","last_active":"2024-09-14T06:30:22.701684+0000","last_peered":"2024-09-14T06:30:22.701684+0000","last_clean":"2024-09-14T06:30:22.701684+0000","last_became_active":"2024-09-14T06:28:51.223252+0000","last_became_peered":"2024-09-14T06:28:51.223252+0000","last_unstale":"2024-09-14T06:30:22.701684+0000","last_undegraded":"2024-09-14T06:30:22.701684+0000","last_fullsized":"2024-09-14T06:30:22.701684+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:28:50.205350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:28:50.205350+0000","last_clean_scrub_stamp":"2024-09-14T06:28:50.205350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T11:56:04.373414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053403100000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1","version":"98'438","reported_seq":7025,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-14T06:30:22.608782+0000","last_change":"2024-09-14T06:30:22.608782+0000","last_active":"2024-09-14T06:30:22.608782+0000","last_peered":"2024-09-14T06:30:22.608782+0000","last_clean":"2024-09-14T06:30:22.608782+0000","last_became_active":"2024-09-14T06:29:00.585673+0000","last_became_peered":"2024-09-14T06:29:00.585673+0000","last_unstale":"2024-09-14T06:30:22.608782+0000","last_undegraded":"2024-09-14T06:30:22.608782+0000","last_fullsized":"2024-09-14T06:30:22.608782+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":438,"log_dups_size":0,"ondisk_log_size":438,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T10:35:05.573347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066394999999999996,"stat_sum":{"num_bytes":53320861,"num_objects":37,"num_object_clones":0,"num_object_copies":74,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":37,"num_whiteouts":0,"num_read":7457,"num_read_kb":62632,"num_write":553,"num_write_kb":24186,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"2a","length":"1"},{"start":"2c","length":"1"}]},{"pgid":"2.0","version":"0'0","reported_seq":180,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-14T06:30:23.625513+0000","last_change":"2024-09-14T06:28:53.640096+0000","last_active":"2024-09-14T06:30:23.625513+0000","last_peered":"2024-09-14T06:30:23.625513+0000","last_clean":"2024-09-14T06:30:23.625513+0000","last_became_active":"2024-09-14T06:28:51.223069+0000","last_became_peered":"2024-09-14T06:28:51.223069+0000","last_unstale":"2024-09-14T06:30:23.625513+0000","last_undegraded":"2024-09-14T06:30:23.625513+0000","last_fullsized":"2024-09-14T06:30:23.625513+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:28:50.205350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:28:50.205350+0000","last_clean_scrub_stamp":"2024-09-14T06:28:50.205350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T15:13:00.471509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00043714600000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"98'443","reported_seq":4312,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-09-14T06:30:20.594325+0000","last_change":"2024-09-14T06:30:19.605702+0000","last_active":"2024-09-14T06:30:20.594325+0000","last_peered":"2024-09-14T06:30:20.594325+0000","last_clean":"2024-09-14T06:30:20.594325+0000","last_became_active":"2024-09-14T06:29:00.578574+0000","last_became_peered":"2024-09-14T06:29:00.578574+0000","last_unstale":"2024-09-14T06:30:20.594325+0000","last_undegraded":"2024-09-14T06:30:20.594325+0000","last_fullsized":"2024-09-14T06:30:20.594325+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":443,"log_dups_size":0,"ondisk_log_size":443,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T16:33:33.451809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000795983,"stat_sum":{"num_bytes":43675338,"num_objects":51,"num_object_clones":0,"num_object_copies":102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":51,"num_whiteouts":0,"num_read":4515,"num_read_kb":44617,"num_write":566,"num_write_kb":32487,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2a","length":"1"}]},{"pgid":"2.3","version":"20'1","reported_seq":181,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-14T06:30:23.623743+0000","last_change":"2024-09-14T06:28:53.230933+0000","last_active":"2024-09-14T06:30:23.623743+0000","last_peered":"2024-09-14T06:30:23.623743+0000","last_clean":"2024-09-14T06:30:23.623743+0000","last_became_active":"2024-09-14T06:28:51.222874+0000","last_became_peered":"2024-09-14T06:28:51.222874+0000","last_unstale":"2024-09-14T06:30:23.623743+0000","last_undegraded":"2024-09-14T06:30:23.623743+0000","last_fullsized":"2024-09-14T06:30:23.623743+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:28:50.205350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:28:50.205350+0000","last_clean_scrub_stamp":"2024-09-14T06:28:50.205350+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T12:04:25.190171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050808899999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'72","reported_seq":271,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-14T06:30:23.625525+0000","last_change":"2024-09-14T06:28:44.839885+0000","last_active":"2024-09-14T06:30:23.625525+0000","last_peered":"2024-09-14T06:30:23.625525+0000","last_clean":"2024-09-14T06:30:23.625525+0000","last_became_active":"2024-09-14T06:28:44.839573+0000","last_became_peered":"2024-09-14T06:28:44.839573+0000","last_unstale":"2024-09-14T06:30:23.625525+0000","last_undegraded":"2024-09-14T06:30:23.625525+0000","last_fullsized":"2024-09-14T06:30:23.625525+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:28:43.828494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:28:43.828494+0000","last_clean_scrub_stamp":"2024-09-14T06:28:43.828494+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T14:41:25.787293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.8","version":"98'223","reported_seq":3470,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-09-14T06:30:20.594320+0000","last_change":"2024-09-14T06:30:19.605423+0000","last_active":"2024-09-14T06:30:20.594320+0000","last_peered":"2024-09-14T06:30:20.594320+0000","last_clean":"2024-09-14T06:30:20.594320+0000","last_became_active":"2024-09-14T06:29:00.574458+0000","last_became_peered":"2024-09-14T06:29:00.574458+0000","last_unstale":"2024-09-14T06:30:20.594320+0000","last_undegraded":"2024-09-14T06:30:20.594320+0000","last_fullsized":"2024-09-14T06:30:20.594320+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":223,"log_dups_size":0,"ondisk_log_size":223,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T07:20:14.169687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00055249400000000005,"stat_sum":{"num_bytes":45016747,"num_objects":46,"num_object_clones":0,"num_object_copies":92,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":46,"num_whiteouts":0,"num_read":3511,"num_read_kb":66239,"num_write":275,"num_write_kb":38280,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2a","length":"1"}]},{"pgid":"3.9","version":"99'143","reported_seq":1318,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-14T06:30:23.612617+0000","last_change":"2024-09-14T06:30:22.607854+0000","last_active":"2024-09-14T06:30:23.612617+0000","last_peered":"2024-09-14T06:30:23.612617+0000","last_clean":"2024-09-14T06:30:23.612617+0000","last_became_active":"2024-09-14T06:29:00.573179+0000","last_became_peered":"2024-09-14T06:29:00.573179+0000","last_unstale":"2024-09-14T06:30:23.612617+0000","last_undegraded":"2024-09-14T06:30:23.612617+0000","last_fullsized":"2024-09-14T06:30:23.612617+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":143,"log_dups_size":0,"ondisk_log_size":143,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T06:49:09.156105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00052945200000000003,"stat_sum":{"num_bytes":18711894,"num_objects":43,"num_object_clones":0,"num_object_copies":86,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":43,"num_whiteouts":0,"num_read":861,"num_read_kb":29231,"num_write":173,"num_write_kb":22284,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"2a","length":"1"},{"start":"2c","length":"1"}]},{"pgid":"3.a","version":"98'178","reported_seq":2197,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-14T06:30:23.616877+0000","last_change":"2024-09-14T06:30:22.621471+0000","last_active":"2024-09-14T06:30:23.616877+0000","last_peered":"2024-09-14T06:30:23.616877+0000","last_clean":"2024-09-14T06:30:23.616877+0000","last_became_active":"2024-09-14T06:29:00.582426+0000","last_became_peered":"2024-09-14T06:29:00.582426+0000","last_unstale":"2024-09-14T06:30:23.616877+0000","last_undegraded":"2024-09-14T06:30:23.616877+0000","last_fullsized":"2024-09-14T06:30:23.616877+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":178,"log_dups_size":0,"ondisk_log_size":178,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T06:44:46.350248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049978999999999996,"stat_sum":{"num_bytes":37141935,"num_objects":37,"num_object_clones":0,"num_object_copies":74,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":37,"num_whiteouts":0,"num_read":1910,"num_read_kb":36958,"num_write":216,"num_write_kb":27112,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"2a","length":"1"},{"start":"2c","length":"1"}]},{"pgid":"3.b","version":"98'1081","reported_seq":4280,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-09-14T06:30:20.594226+0000","last_change":"2024-09-14T06:30:19.605593+0000","last_active":"2024-09-14T06:30:20.594226+0000","last_peered":"2024-09-14T06:30:20.594226+0000","last_clean":"2024-09-14T06:30:20.594226+0000","last_became_active":"2024-09-14T06:29:00.578483+0000","last_became_peered":"2024-09-14T06:29:00.578483+0000","last_unstale":"2024-09-14T06:30:20.594226+0000","last_undegraded":"2024-09-14T06:30:20.594226+0000","last_fullsized":"2024-09-14T06:30:20.594226+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":1081,"log_dups_size":0,"ondisk_log_size":1081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T06:43:20.596961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00068472499999999996,"stat_sum":{"num_bytes":21757058,"num_objects":41,"num_object_clones":0,"num_object_copies":82,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":41,"num_whiteouts":0,"num_read":8569,"num_read_kb":18146,"num_write":1817,"num_write_kb":17643,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2a","length":"1"}]},{"pgid":"3.c","version":"99'268","reported_seq":3130,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-14T06:30:23.623785+0000","last_change":"2024-09-14T06:30:22.612819+0000","last_active":"2024-09-14T06:30:23.623785+0000","last_peered":"2024-09-14T06:30:23.623785+0000","last_clean":"2024-09-14T06:30:23.623785+0000","last_became_active":"2024-09-14T06:29:00.583406+0000","last_became_peered":"2024-09-14T06:29:00.583406+0000","last_unstale":"2024-09-14T06:30:23.623785+0000","last_undegraded":"2024-09-14T06:30:23.623785+0000","last_fullsized":"2024-09-14T06:30:23.623785+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":268,"log_dups_size":0,"ondisk_log_size":268,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T08:19:59.181425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0016396290000000001,"stat_sum":{"num_bytes":26190419,"num_objects":36,"num_object_clones":0,"num_object_copies":72,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":36,"num_whiteouts":0,"num_read":3044,"num_read_kb":26426,"num_write":377,"num_write_kb":22175,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2a","length":"1"},{"start":"2c","length":"1"}]},{"pgid":"3.d","version":"98'200","reported_seq":3650,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-14T06:30:23.625667+0000","last_change":"2024-09-14T06:30:22.615128+0000","last_active":"2024-09-14T06:30:23.625667+0000","last_peered":"2024-09-14T06:30:23.625667+0000","last_clean":"2024-09-14T06:30:23.625667+0000","last_became_active":"2024-09-14T06:29:00.580362+0000","last_became_peered":"2024-09-14T06:29:00.580362+0000","last_unstale":"2024-09-14T06:30:23.625667+0000","last_undegraded":"2024-09-14T06:30:23.625667+0000","last_fullsized":"2024-09-14T06:30:23.625667+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":200,"log_dups_size":0,"ondisk_log_size":200,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T14:26:33.039528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00072795100000000001,"stat_sum":{"num_bytes":44608849,"num_objects":40,"num_object_clones":0,"num_object_copies":80,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":40,"num_whiteouts":0,"num_read":3364,"num_read_kb":45708,"num_write":248,"num_write_kb":21217,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"2a","length":"1"},{"start":"2c","length":"1"}]},{"pgid":"3.e","version":"98'273","reported_seq":1869,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-14T06:30:23.625718+0000","last_change":"2024-09-14T06:30:22.614970+0000","last_active":"2024-09-14T06:30:23.625718+0000","last_peered":"2024-09-14T06:30:23.625718+0000","last_clean":"2024-09-14T06:30:23.625718+0000","last_became_active":"2024-09-14T06:29:00.586374+0000","last_became_peered":"2024-09-14T06:29:00.586374+0000","last_unstale":"2024-09-14T06:30:23.625718+0000","last_undegraded":"2024-09-14T06:30:23.625718+0000","last_fullsized":"2024-09-14T06:30:23.625718+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":273,"log_dups_size":0,"ondisk_log_size":273,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T13:13:45.418557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069576999999999996,"stat_sum":{"num_bytes":25133180,"num_objects":45,"num_object_clones":0,"num_object_copies":90,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":45,"num_whiteouts":0,"num_read":1697,"num_read_kb":27585,"num_write":399,"num_write_kb":25425,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"2a","length":"1"},{"start":"2c","length":"1"}]},{"pgid":"3.f","version":"98'246","reported_seq":3429,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-14T06:30:23.625716+0000","last_change":"2024-09-14T06:30:22.615204+0000","last_active":"2024-09-14T06:30:23.625716+0000","last_peered":"2024-09-14T06:30:23.625716+0000","last_clean":"2024-09-14T06:30:23.625716+0000","last_became_active":"2024-09-14T06:29:00.586491+0000","last_became_peered":"2024-09-14T06:29:00.586491+0000","last_unstale":"2024-09-14T06:30:23.625716+0000","last_undegraded":"2024-09-14T06:30:23.625716+0000","last_fullsized":"2024-09-14T06:30:23.625716+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":246,"log_dups_size":0,"ondisk_log_size":246,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T13:20:04.734122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076521300000000005,"stat_sum":{"num_bytes":17046942,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":3732,"num_read_kb":18272,"num_write":303,"num_write_kb":14026,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"2a","length":"1"},{"start":"2c","length":"1"}]},{"pgid":"3.10","version":"98'284","reported_seq":2980,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-14T06:30:23.616866+0000","last_change":"2024-09-14T06:30:22.621397+0000","last_active":"2024-09-14T06:30:23.616866+0000","last_peered":"2024-09-14T06:30:23.616866+0000","last_clean":"2024-09-14T06:30:23.616866+0000","last_became_active":"2024-09-14T06:29:00.579539+0000","last_became_peered":"2024-09-14T06:29:00.579539+0000","last_unstale":"2024-09-14T06:30:23.616866+0000","last_undegraded":"2024-09-14T06:30:23.616866+0000","last_fullsized":"2024-09-14T06:30:23.616866+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":284,"log_dups_size":0,"ondisk_log_size":284,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T07:50:34.368807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00044170899999999998,"stat_sum":{"num_bytes":19497509,"num_objects":30,"num_object_clones":0,"num_object_copies":60,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":30,"num_whiteouts":0,"num_read":2707,"num_read_kb":14268,"num_write":307,"num_write_kb":22669,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"2a","length":"1"},{"start":"2c","length":"1"}]},{"pgid":"3.11","version":"98'375","reported_seq":7677,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-14T06:30:23.625513+0000","last_change":"2024-09-14T06:30:22.614639+0000","last_active":"2024-09-14T06:30:23.625513+0000","last_peered":"2024-09-14T06:30:23.625513+0000","last_clean":"2024-09-14T06:30:23.625513+0000","last_became_active":"2024-09-14T06:29:00.586480+0000","last_became_peered":"2024-09-14T06:29:00.586480+0000","last_unstale":"2024-09-14T06:30:23.625513+0000","last_undegraded":"2024-09-14T06:30:23.625513+0000","last_fullsized":"2024-09-14T06:30:23.625513+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":375,"log_dups_size":0,"ondisk_log_size":375,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T17:18:48.953355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00055691300000000003,"stat_sum":{"num_bytes":72839101,"num_objects":49,"num_object_clones":0,"num_object_copies":98,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":49,"num_whiteouts":0,"num_read":7460,"num_read_kb":81273,"num_write":440,"num_write_kb":34087,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"2a","length":"1"},{"start":"2c","length":"1"}]},{"pgid":"3.12","version":"98'456","reported_seq":5486,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-14T06:30:22.609092+0000","last_change":"2024-09-14T06:30:22.609092+0000","last_active":"2024-09-14T06:30:22.609092+0000","last_peered":"2024-09-14T06:30:22.609092+0000","last_clean":"2024-09-14T06:30:22.609092+0000","last_became_active":"2024-09-14T06:29:00.579826+0000","last_became_peered":"2024-09-14T06:29:00.579826+0000","last_unstale":"2024-09-14T06:30:22.609092+0000","last_undegraded":"2024-09-14T06:30:22.609092+0000","last_fullsized":"2024-09-14T06:30:22.609092+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":456,"log_dups_size":0,"ondisk_log_size":456,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T09:44:15.157725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070490600000000004,"stat_sum":{"num_bytes":33207124,"num_objects":43,"num_object_clones":0,"num_object_copies":86,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":43,"num_whiteouts":0,"num_read":5850,"num_read_kb":42269,"num_write":571,"num_write_kb":26908,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"2a","length":"1"},{"start":"2c","length":"1"}]},{"pgid":"3.13","version":"98'453","reported_seq":5350,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-14T06:30:23.625563+0000","last_change":"2024-09-14T06:30:22.614719+0000","last_active":"2024-09-14T06:30:23.625563+0000","last_peered":"2024-09-14T06:30:23.625563+0000","last_clean":"2024-09-14T06:30:23.625563+0000","last_became_active":"2024-09-14T06:29:00.586520+0000","last_became_peered":"2024-09-14T06:29:00.586520+0000","last_unstale":"2024-09-14T06:30:23.625563+0000","last_undegraded":"2024-09-14T06:30:23.625563+0000","last_fullsized":"2024-09-14T06:30:23.625563+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":453,"log_dups_size":0,"ondisk_log_size":453,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T10:44:48.405672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053809500000000004,"stat_sum":{"num_bytes":53149511,"num_objects":38,"num_object_clones":0,"num_object_copies":76,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":38,"num_whiteouts":0,"num_read":5488,"num_read_kb":45634,"num_write":617,"num_write_kb":31089,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"2a","length":"1"},{"start":"2c","length":"1"}]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":1075714287,"num_objects":1269,"num_object_clones":0,"num_object_copies":2538,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1269,"num_whiteouts":0,"num_read":126074,"num_read_kb":1168797,"num_write":14819,"num_write_kb":809230,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":39,"num_bytes_recovered":1400964,"num_keys_recovered":11,"num_objects_omap":23,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":72732672,"data_stored":710056519,"data_compressed":43652395,"data_compressed_allocated":57528320,"data_compressed_original":698095796,"omap_allocated":0,"internal_metadata":0},"log_size":10999,"ondisk_log_size":10999,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":131072,"data_stored":1180736,"data_compressed":65166,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509463,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55100,"kb_used_data":11432,"kb_used_omap":1,"kb_used_meta":43646,"kb_avail":94316740,"statfs":{"total":96636764160,"available":96580341760,"internally_reserved":0,"allocated":11706368,"data_stored":99810101,"data_compressed":6064561,"data_compressed_allocated":7933952,"data_compressed_original":97174851,"omap_allocated":1590,"internal_metadata":44693962},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509462,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":34868,"kb_used_data":3684,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94336972,"statfs":{"total":96636764160,"available":96601059328,"internally_reserved":0,"allocated":3772416,"data_stored":26601695,"data_compressed":1494839,"data_compressed_allocated":2039808,"data_compressed_original":25607719,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509464,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":50064,"kb_used_data":10552,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94321776,"statfs":{"total":96636764160,"available":96585498624,"internally_reserved":0,"allocated":10805248,"data_stored":89545146,"data_compressed":5484896,"data_compressed_allocated":7340032,"data_compressed_original":87216976,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509463,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":66060,"kb_used_data":16820,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94305780,"statfs":{"total":96636764160,"available":96569118720,"internally_reserved":0,"allocated":17223680,"data_stored":159476697,"data_compressed":10154445,"data_compressed_allocated":13172736,"data_compressed_original":156746019,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509464,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":47928,"kb_used_data":8424,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94323912,"statfs":{"total":96636764160,"available":96587685888,"internally_reserved":0,"allocated":8626176,"data_stored":76286321,"data_compressed":4608422,"data_compressed_allocated":6103040,"data_compressed_original":74710041,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":14,"seq":60129542167,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40072,"kb_used_data":4720,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94331768,"statfs":{"total":96636764160,"available":96595730432,"internally_reserved":0,"allocated":4833280,"data_stored":38471287,"data_compressed":2357654,"data_compressed_allocated":3133440,"data_compressed_original":37542635,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542166,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":61932,"kb_used_data":12696,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94309908,"statfs":{"total":96636764160,"available":96573345792,"internally_reserved":0,"allocated":13000704,"data_stored":113700341,"data_compressed":6847207,"data_compressed_allocated":8962048,"data_compressed_original":110815485,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574871,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":61532,"kb_used_data":12280,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94310308,"statfs":{"total":96636764160,"available":96573755392,"internally_reserved":0,"allocated":12574720,"data_stored":114962236,"data_compressed":6889114,"data_compressed_allocated":9170944,"data_compressed_original":112620094,"omap_allocated":1593,"internal_metadata":50395591},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32583,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32583,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":11743232,"data_stored":112548892,"data_compressed":6814624,"data_compressed_allocated":8916992,"data_compressed_original":110242045,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":7430144,"data_stored":75725221,"data_compressed":4608422,"data_compressed_allocated":6103040,"data_compressed_original":74710041,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":3653632,"data_stored":37916454,"data_compressed":2357654,"data_compressed_allocated":3133440,"data_compressed_original":37542635,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":11423744,"data_stored":114419880,"data_compressed":6889114,"data_compressed_allocated":9170944,"data_compressed_original":112620094,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":15855616,"data_stored":155775103,"data_compressed":9971268,"data_compressed_allocated":12939264,"data_compressed_original":153563067,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":9609216,"data_stored":88984046,"data_compressed":5484896,"data_compressed_allocated":7340032,"data_compressed_original":87216976,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":2580480,"data_stored":26040614,"data_compressed":1494839,"data_compressed_allocated":2039808,"data_compressed_original":25607719,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":10436608,"data_stored":98646309,"data_compressed":6031578,"data_compressed_allocated":7884800,"data_compressed_original":96593219,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-14T06:30:26.102 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-14T06:30:26.138 INFO:tasks.ceph.osd.5.smithi138.stderr:2024-09-14T06:30:26.137+0000 7fe13746f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33618) UID: 0 2024-09-14T06:30:26.159 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-14T06:30:26.160 INFO:teuthology.orchestra.run.smithi094.stdout: "ops": [], 2024-09-14T06:30:26.160 INFO:teuthology.orchestra.run.smithi094.stdout: "complaint_time": 30, 2024-09-14T06:30:26.160 INFO:teuthology.orchestra.run.smithi094.stdout: "num_blocked_ops": 0 2024-09-14T06:30:26.160 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-14T06:30:26.169 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-14T06:30:26.239 INFO:tasks.ceph.osd.7.smithi138.stderr:2024-09-14T06:30:26.238+0000 7f3c6a21d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33623) UID: 0 2024-09-14T06:30:26.330 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-14T06:30:26.331 INFO:teuthology.orchestra.run.smithi094.stdout: "size": 20, 2024-09-14T06:30:26.331 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 600, 2024-09-14T06:30:26.331 INFO:teuthology.orchestra.run.smithi094.stdout: "ops": [ 2024-09-14T06:30:26.331 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:26.332 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:44425 3.2 3:4603ad7f:::rbd_data.112dca2f6ae1.0000000000000005:17 [read 868352~4096 [fadvise_nocache]] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:30:26.332 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:43.854967+0000", 2024-09-14T06:30:26.332 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 42.473032203999999, 2024-09-14T06:30:26.332 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.063146025999999994, 2024-09-14T06:30:26.332 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:30:26.332 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:30:26.333 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:30:26.333 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:30:26.333 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:30:26.333 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:26.333 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 44425 2024-09-14T06:30:26.333 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:26.334 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:30:26.334 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:26.334 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:30:26.334 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.854967+0000", 2024-09-14T06:30:26.334 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:26.334 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:26.334 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:26.335 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:30:26.335 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.854967+0000", 2024-09-14T06:30:26.335 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:26.335 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:26.335 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:26.335 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:30:26.336 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.854967+0000", 2024-09-14T06:30:26.336 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.16e-07 2024-09-14T06:30:26.336 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:26.336 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:26.336 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:30:26.336 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.854968+0000", 2024-09-14T06:30:26.337 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.006e-06 2024-09-14T06:30:26.337 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:26.337 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:26.337 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:30:26.337 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.854971+0000", 2024-09-14T06:30:26.337 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.9359999999999999e-06 2024-09-14T06:30:26.338 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:26.338 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:26.338 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:30:26.338 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.854974+0000", 2024-09-14T06:30:26.338 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.1840000000000002e-06 2024-09-14T06:30:26.338 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:26.339 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:26.339 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:30:26.339 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.917945+0000", 2024-09-14T06:30:26.339 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.062970925999999997 2024-09-14T06:30:26.339 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:26.339 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:26.340 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:30:26.340 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.918030+0000", 2024-09-14T06:30:26.340 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.5863000000000001e-05 2024-09-14T06:30:26.340 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:26.340 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:26.340 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:30:26.341 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.918113+0000", 2024-09-14T06:30:26.341 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.2894999999999998e-05 2024-09-14T06:30:26.341 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:26.341 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:26.341 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:26.341 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:26.342 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:26.342 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:44426 3.2 3:4603ad7f:::rbd_data.112dca2f6ae1.0000000000000005:17 [read 892928~4096 [fadvise_nocache]] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:30:26.342 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:43.854977+0000", 2024-09-14T06:30:26.342 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 42.473022794000002, 2024-09-14T06:30:26.342 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.063369237999999994, 2024-09-14T06:30:26.342 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:30:26.343 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:30:26.343 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:30:26.343 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:30:26.343 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:30:26.343 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:26.343 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 44426 2024-09-14T06:30:26.344 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:26.344 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:30:26.344 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:26.344 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:30:26.344 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.854977+0000", 2024-09-14T06:30:26.344 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:26.345 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:26.345 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:26.345 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:30:26.345 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.854977+0000", 2024-09-14T06:30:26.345 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:26.348 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:26.348 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:26.348 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:30:26.348 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.854977+0000", 2024-09-14T06:30:26.348 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.1199999999999999e-07 2024-09-14T06:30:26.348 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:26.349 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:26.349 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:30:26.349 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.854980+0000", 2024-09-14T06:30:26.349 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.27e-06 2024-09-14T06:30:26.349 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:26.349 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:26.349 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:30:26.349 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.854983+0000", 2024-09-14T06:30:26.349 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.1650000000000002e-06 2024-09-14T06:30:26.349 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:26.350 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:26.350 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:30:26.350 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.854989+0000", 2024-09-14T06:30:26.350 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.6520000000000003e-06 2024-09-14T06:30:26.350 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:26.350 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:26.350 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:30:26.350 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.918143+0000", 2024-09-14T06:30:26.350 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.063154541999999994 2024-09-14T06:30:26.350 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:26.350 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:26.350 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:30:26.351 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.918209+0000", 2024-09-14T06:30:26.351 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.5529000000000002e-05 2024-09-14T06:30:26.351 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:26.351 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:26.351 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:30:26.351 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.918346+0000", 2024-09-14T06:30:26.351 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000136868 2024-09-14T06:30:26.351 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:26.351 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:26.351 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:26.351 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:26.351 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:26.351 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:44427 3.2 3:4603ad7f:::rbd_data.112dca2f6ae1.0000000000000005:17 [read 917504~4096 [fadvise_nocache]] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:30:26.351 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:43.854992+0000", 2024-09-14T06:30:26.351 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 42.473007148000001, 2024-09-14T06:30:26.352 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.063607792999999996, 2024-09-14T06:30:26.352 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:30:26.352 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:30:26.352 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:30:26.352 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:30:26.352 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:30:26.352 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:26.352 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 44427 2024-09-14T06:30:26.352 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:26.352 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:30:26.352 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:26.352 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:30:26.352 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.854992+0000", 2024-09-14T06:30:26.353 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:26.353 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:26.353 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:26.353 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:30:26.353 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.854992+0000", 2024-09-14T06:30:26.353 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:26.353 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:26.353 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:26.353 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:30:26.353 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.854993+0000", 2024-09-14T06:30:26.353 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.1400000000000001e-07 2024-09-14T06:30:26.353 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:26.353 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:26.353 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:30:26.354 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.854994+0000", 2024-09-14T06:30:26.354 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.091e-06 2024-09-14T06:30:26.354 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:26.354 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:26.354 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:30:26.354 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855001+0000", 2024-09-14T06:30:26.354 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.204e-06 2024-09-14T06:30:26.354 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:26.354 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:26.354 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:30:26.354 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855003+0000", 2024-09-14T06:30:26.354 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.514e-06 2024-09-14T06:30:26.354 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:26.354 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:26.354 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:30:26.355 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.918380+0000", 2024-09-14T06:30:26.355 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.063376738000000002 2024-09-14T06:30:26.355 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:26.355 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:26.355 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:30:26.355 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.918453+0000", 2024-09-14T06:30:26.355 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.3291e-05 2024-09-14T06:30:26.355 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:26.355 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:26.355 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:30:26.355 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.918600+0000", 2024-09-14T06:30:26.355 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00014674100000000001 2024-09-14T06:30:26.355 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:26.355 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:26.356 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:26.356 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:26.356 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:26.356 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:44428 3.2 3:4603ad7f:::rbd_data.112dca2f6ae1.0000000000000005:17 [read 942080~4096 [fadvise_nocache]] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:30:26.356 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:43.855006+0000", 2024-09-14T06:30:26.356 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 42.472993098000003, 2024-09-14T06:30:26.356 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.063846321999999997, 2024-09-14T06:30:26.356 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:30:26.356 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:30:26.356 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:30:26.356 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:30:26.356 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:30:26.356 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:26.356 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 44428 2024-09-14T06:30:26.357 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:26.357 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:30:26.357 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:26.357 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:30:26.357 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855006+0000", 2024-09-14T06:30:26.357 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:26.357 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:26.357 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:26.357 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:30:26.357 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855006+0000", 2024-09-14T06:30:26.357 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:26.357 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:26.357 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:26.357 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:30:26.358 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855007+0000", 2024-09-14T06:30:26.358 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.1199999999999999e-07 2024-09-14T06:30:26.358 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:26.358 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:26.358 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:30:26.358 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855011+0000", 2024-09-14T06:30:26.358 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.9539999999999998e-06 2024-09-14T06:30:26.358 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:26.358 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:26.358 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:30:26.358 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855014+0000", 2024-09-14T06:30:26.358 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.2930000000000001e-06 2024-09-14T06:30:26.358 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:26.358 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:26.359 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:30:26.359 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855016+0000", 2024-09-14T06:30:26.359 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.3039999999999999e-06 2024-09-14T06:30:26.359 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:26.359 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:26.359 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:30:26.359 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.918634+0000", 2024-09-14T06:30:26.359 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.063617928000000004 2024-09-14T06:30:26.359 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:26.359 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:26.359 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:30:26.359 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.918711+0000", 2024-09-14T06:30:26.359 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.6545000000000007e-05 2024-09-14T06:30:26.359 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:26.360 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:26.360 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:30:26.360 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.918853+0000", 2024-09-14T06:30:26.360 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00014208600000000001 2024-09-14T06:30:26.360 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:26.360 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:26.360 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:26.360 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:26.360 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:26.360 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:44429 3.2 3:4603ad7f:::rbd_data.112dca2f6ae1.0000000000000005:17 [read 966656~4096 [fadvise_nocache]] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:30:26.360 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:43.855019+0000", 2024-09-14T06:30:26.360 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 42.472980425000003, 2024-09-14T06:30:26.360 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.064059226999999996, 2024-09-14T06:30:26.360 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:30:26.361 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:30:26.361 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:30:26.361 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:30:26.361 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:30:26.361 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:26.361 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 44429 2024-09-14T06:30:26.361 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:26.361 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:30:26.361 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:26.361 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:30:26.362 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855019+0000", 2024-09-14T06:30:26.362 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:26.362 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:26.362 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:26.362 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:30:26.362 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855019+0000", 2024-09-14T06:30:26.362 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:26.362 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:26.362 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:26.362 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:30:26.362 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855019+0000", 2024-09-14T06:30:26.362 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.28e-07 2024-09-14T06:30:26.362 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:26.362 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:26.363 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:30:26.363 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855020+0000", 2024-09-14T06:30:26.363 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.0330000000000001e-06 2024-09-14T06:30:26.363 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:26.363 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:26.363 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:30:26.363 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855023+0000", 2024-09-14T06:30:26.363 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.79e-06 2024-09-14T06:30:26.363 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:26.363 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:26.363 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:30:26.363 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855025+0000", 2024-09-14T06:30:26.363 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.216e-06 2024-09-14T06:30:26.363 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:26.364 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:26.364 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:30:26.364 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.918886+0000", 2024-09-14T06:30:26.364 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.063861037999999995 2024-09-14T06:30:26.364 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:26.364 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:26.364 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:30:26.364 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.918952+0000", 2024-09-14T06:30:26.364 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.5993000000000003e-05 2024-09-14T06:30:26.364 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:26.364 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:26.364 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:30:26.364 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.919078+0000", 2024-09-14T06:30:26.364 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00012592899999999999 2024-09-14T06:30:26.364 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:26.365 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:26.365 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:26.365 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:26.365 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:26.365 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:44430 3.2 3:4603ad7f:::rbd_data.112dca2f6ae1.0000000000000005:17 [read 991232~4096 [fadvise_nocache]] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:30:26.365 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:43.855028+0000", 2024-09-14T06:30:26.365 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 42.472971415000004, 2024-09-14T06:30:26.365 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.064284346000000006, 2024-09-14T06:30:26.365 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:30:26.365 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:30:26.365 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:30:26.365 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:30:26.365 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:30:26.365 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:26.366 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 44430 2024-09-14T06:30:26.366 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:26.366 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:30:26.366 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:26.366 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:30:26.366 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855028+0000", 2024-09-14T06:30:26.366 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:26.366 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:26.366 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:26.366 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:30:26.366 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855028+0000", 2024-09-14T06:30:26.366 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:26.366 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:26.366 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:26.366 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:30:26.367 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855028+0000", 2024-09-14T06:30:26.367 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.1199999999999999e-07 2024-09-14T06:30:26.367 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:26.367 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:26.367 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:30:26.367 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855029+0000", 2024-09-14T06:30:26.367 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.1100000000000004e-07 2024-09-14T06:30:26.367 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:26.367 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:26.367 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:30:26.367 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855032+0000", 2024-09-14T06:30:26.367 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.7889999999999999e-06 2024-09-14T06:30:26.367 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:26.367 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:26.367 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:30:26.368 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855034+0000", 2024-09-14T06:30:26.368 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.125e-06 2024-09-14T06:30:26.368 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:26.368 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:26.368 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:30:26.368 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.919110+0000", 2024-09-14T06:30:26.368 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.064076413999999998 2024-09-14T06:30:26.368 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:26.368 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:26.368 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:30:26.368 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.919186+0000", 2024-09-14T06:30:26.368 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.5784999999999994e-05 2024-09-14T06:30:26.368 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:26.368 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:26.369 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:30:26.369 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.919312+0000", 2024-09-14T06:30:26.369 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00012611 2024-09-14T06:30:26.369 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:26.369 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:26.369 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:26.369 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:26.369 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:26.369 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:44431 3.2 3:4603ad7f:::rbd_data.112dca2f6ae1.0000000000000005:17 [read 1015808~4096 [fadvise_nocache]] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:30:26.369 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:43.855037+0000", 2024-09-14T06:30:26.369 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 42.472962547000002, 2024-09-14T06:30:26.369 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.064469755000000004, 2024-09-14T06:30:26.369 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:30:26.369 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:30:26.370 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:30:26.370 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:30:26.370 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:30:26.370 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:26.370 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 44431 2024-09-14T06:30:26.370 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:26.370 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:30:26.370 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:26.370 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:30:26.370 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855037+0000", 2024-09-14T06:30:26.370 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:26.370 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:26.370 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:26.370 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:30:26.370 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855037+0000", 2024-09-14T06:30:26.371 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:26.371 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:26.371 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:26.371 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:30:26.371 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855037+0000", 2024-09-14T06:30:26.371 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.1e-07 2024-09-14T06:30:26.371 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:26.371 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:26.371 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:30:26.371 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855038+0000", 2024-09-14T06:30:26.371 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.9400000000000004e-07 2024-09-14T06:30:26.371 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:26.371 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:26.371 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:30:26.372 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855041+0000", 2024-09-14T06:30:26.372 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.9639999999999999e-06 2024-09-14T06:30:26.372 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:26.372 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:26.372 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:30:26.372 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855043+0000", 2024-09-14T06:30:26.372 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.2259999999999999e-06 2024-09-14T06:30:26.372 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:26.372 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:26.372 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:30:26.372 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.919347+0000", 2024-09-14T06:30:26.372 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.064304091999999993 2024-09-14T06:30:26.372 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:26.372 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:26.372 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:30:26.373 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.919393+0000", 2024-09-14T06:30:26.373 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.5340999999999998e-05 2024-09-14T06:30:26.373 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:26.373 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:26.373 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:30:26.373 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.919507+0000", 2024-09-14T06:30:26.373 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000114028 2024-09-14T06:30:26.373 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:26.373 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:26.373 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:26.373 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:26.373 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:26.373 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:44432 3.2 3:4603ad7f:::rbd_data.112dca2f6ae1.0000000000000005:17 [read 1040384~4096 [fadvise_nocache]] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:30:26.374 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:43.855046+0000", 2024-09-14T06:30:26.374 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 42.472953478999997, 2024-09-14T06:30:26.374 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.064639253999999993, 2024-09-14T06:30:26.374 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:30:26.374 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:30:26.374 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:30:26.374 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:30:26.374 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:30:26.374 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:26.374 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 44432 2024-09-14T06:30:26.374 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:26.374 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:30:26.374 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:26.374 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:30:26.375 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855046+0000", 2024-09-14T06:30:26.375 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:26.375 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:26.375 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:26.375 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:30:26.375 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855046+0000", 2024-09-14T06:30:26.375 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:26.375 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:26.375 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:26.375 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:30:26.375 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855046+0000", 2024-09-14T06:30:26.375 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.05e-07 2024-09-14T06:30:26.375 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:26.375 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:26.376 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:30:26.376 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855047+0000", 2024-09-14T06:30:26.376 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.879999999999999e-07 2024-09-14T06:30:26.376 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:26.376 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:26.376 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:30:26.376 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855050+0000", 2024-09-14T06:30:26.376 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.8459999999999999e-06 2024-09-14T06:30:26.376 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:26.376 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:26.376 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:30:26.376 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855052+0000", 2024-09-14T06:30:26.376 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.136e-06 2024-09-14T06:30:26.377 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:26.377 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:26.377 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:30:26.377 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.919536+0000", 2024-09-14T06:30:26.377 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.064484132 2024-09-14T06:30:26.377 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:26.377 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:26.377 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:30:26.377 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.919611+0000", 2024-09-14T06:30:26.377 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.4407999999999996e-05 2024-09-14T06:30:26.377 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:26.377 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:26.377 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:30:26.378 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.919685+0000", 2024-09-14T06:30:26.378 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.4538999999999999e-05 2024-09-14T06:30:26.378 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:26.378 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:26.378 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:26.378 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:26.378 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:26.378 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:44518 3.2 3:4190d882:::rbd_data.112dca2f6ae1.0000000000000002:17 [read 12288~4096 [fadvise_nocache]] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:30:26.378 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:43.855055+0000", 2024-09-14T06:30:26.378 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 42.472944595000001, 2024-09-14T06:30:26.378 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.064894317000000007, 2024-09-14T06:30:26.378 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:30:26.378 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:30:26.379 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:30:26.379 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:30:26.379 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:30:26.379 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:26.379 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 44518 2024-09-14T06:30:26.379 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:26.379 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:30:26.379 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:26.379 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:30:26.379 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855055+0000", 2024-09-14T06:30:26.379 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:26.379 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:26.379 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:26.379 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:30:26.379 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855055+0000", 2024-09-14T06:30:26.380 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:26.380 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:26.380 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:26.380 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:30:26.380 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855055+0000", 2024-09-14T06:30:26.380 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.0699999999999999e-07 2024-09-14T06:30:26.380 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:26.380 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:26.380 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:30:26.380 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855056+0000", 2024-09-14T06:30:26.380 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.9400000000000004e-07 2024-09-14T06:30:26.380 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:26.380 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:26.380 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:30:26.381 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855059+0000", 2024-09-14T06:30:26.381 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.8870000000000001e-06 2024-09-14T06:30:26.381 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:26.381 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:26.381 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:30:26.381 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855061+0000", 2024-09-14T06:30:26.381 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.1919999999999999e-06 2024-09-14T06:30:26.381 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:26.381 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:26.381 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:30:26.381 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.919719+0000", 2024-09-14T06:30:26.381 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.064658013 2024-09-14T06:30:26.381 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:26.381 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:26.382 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:30:26.382 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.919788+0000", 2024-09-14T06:30:26.382 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.9282000000000005e-05 2024-09-14T06:30:26.382 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:26.382 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:26.382 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:30:26.382 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.919949+0000", 2024-09-14T06:30:26.382 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00016084199999999999 2024-09-14T06:30:26.382 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:26.382 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:26.382 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:26.382 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:26.382 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:26.382 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:44519 3.2 3:4190d882:::rbd_data.112dca2f6ae1.0000000000000002:17 [read 36864~4096 [fadvise_nocache]] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:30:26.382 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:43.855064+0000", 2024-09-14T06:30:26.382 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 42.472935618000001, 2024-09-14T06:30:26.383 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.065081865000000003, 2024-09-14T06:30:26.383 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:30:26.383 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:30:26.383 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:30:26.383 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:30:26.383 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:30:26.383 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:26.383 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 44519 2024-09-14T06:30:26.383 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:26.383 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:30:26.383 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:26.383 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:30:26.383 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855064+0000", 2024-09-14T06:30:26.383 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:26.383 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:26.384 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:26.384 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:30:26.384 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855064+0000", 2024-09-14T06:30:26.384 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:26.384 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:26.384 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:26.384 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:30:26.384 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855064+0000", 2024-09-14T06:30:26.384 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.1e-07 2024-09-14T06:30:26.384 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:26.385 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:26.385 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:30:26.385 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855065+0000", 2024-09-14T06:30:26.385 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.035e-06 2024-09-14T06:30:26.385 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:26.385 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:26.385 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:30:26.385 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855068+0000", 2024-09-14T06:30:26.385 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.8339999999999999e-06 2024-09-14T06:30:26.385 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:26.385 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:26.385 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:30:26.385 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855071+0000", 2024-09-14T06:30:26.385 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.3450000000000002e-06 2024-09-14T06:30:26.386 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:26.386 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:26.386 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:30:26.386 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.919979+0000", 2024-09-14T06:30:26.386 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.064907647999999998 2024-09-14T06:30:26.386 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:26.386 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:26.386 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:30:26.386 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.920026+0000", 2024-09-14T06:30:26.386 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.7595000000000001e-05 2024-09-14T06:30:26.386 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:26.386 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:26.386 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:30:26.386 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.920146+0000", 2024-09-14T06:30:26.387 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00011919800000000001 2024-09-14T06:30:26.387 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:26.387 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:26.387 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:26.387 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:26.387 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:26.387 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:44520 3.2 3:4190d882:::rbd_data.112dca2f6ae1.0000000000000002:17 [read 61440~4096 [fadvise_nocache]] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:30:26.387 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:43.855075+0000", 2024-09-14T06:30:26.387 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 42.472924567, 2024-09-14T06:30:26.387 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.065335078000000005, 2024-09-14T06:30:26.387 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:30:26.387 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:30:26.387 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:30:26.387 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:30:26.387 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:30:26.388 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:26.388 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 44520 2024-09-14T06:30:26.388 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:26.388 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:30:26.388 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:26.388 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:30:26.388 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855075+0000", 2024-09-14T06:30:26.388 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:26.388 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:26.388 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:26.388 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:30:26.388 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855075+0000", 2024-09-14T06:30:26.388 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:26.388 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:26.388 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:26.389 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:30:26.389 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855075+0000", 2024-09-14T06:30:26.389 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.1e-07 2024-09-14T06:30:26.389 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:26.389 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:26.389 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:30:26.389 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855076+0000", 2024-09-14T06:30:26.389 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.0839999999999999e-06 2024-09-14T06:30:26.389 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:26.389 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:26.389 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:30:26.389 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855079+0000", 2024-09-14T06:30:26.389 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.9359999999999999e-06 2024-09-14T06:30:26.390 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:26.390 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:26.390 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:30:26.390 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855084+0000", 2024-09-14T06:30:26.390 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.1730000000000003e-06 2024-09-14T06:30:26.390 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:26.390 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:26.390 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:30:26.390 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.920175+0000", 2024-09-14T06:30:26.390 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.065090234999999996 2024-09-14T06:30:26.390 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:26.390 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:26.390 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:30:26.390 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.920250+0000", 2024-09-14T06:30:26.391 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.5807000000000001e-05 2024-09-14T06:30:26.391 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:26.391 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:26.391 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:30:26.391 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.920410+0000", 2024-09-14T06:30:26.391 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00015963300000000001 2024-09-14T06:30:26.391 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:26.391 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:26.391 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:26.391 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:26.391 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:26.391 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:44521 3.2 3:4190d882:::rbd_data.112dca2f6ae1.0000000000000002:17 [read 86016~4096 [fadvise_nocache]] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:30:26.391 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:43.855087+0000", 2024-09-14T06:30:26.391 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 42.472912086000001, 2024-09-14T06:30:26.392 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.065563264999999996, 2024-09-14T06:30:26.392 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:30:26.392 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:30:26.392 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:30:26.392 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:30:26.392 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:30:26.392 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:26.392 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 44521 2024-09-14T06:30:26.392 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:26.392 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:30:26.392 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:26.392 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:30:26.392 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855087+0000", 2024-09-14T06:30:26.392 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:26.393 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:26.393 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:26.393 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:30:26.393 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855087+0000", 2024-09-14T06:30:26.393 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:26.393 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:26.393 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:26.393 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:30:26.393 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855088+0000", 2024-09-14T06:30:26.393 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.3699999999999999e-07 2024-09-14T06:30:26.393 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:26.393 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:26.393 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:30:26.393 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855089+0000", 2024-09-14T06:30:26.394 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.9900000000000009e-07 2024-09-14T06:30:26.394 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:26.394 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:26.394 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:30:26.394 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855091+0000", 2024-09-14T06:30:26.394 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.8940000000000002e-06 2024-09-14T06:30:26.394 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:26.394 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:26.394 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:30:26.394 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855094+0000", 2024-09-14T06:30:26.394 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.351e-06 2024-09-14T06:30:26.394 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:26.394 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:26.394 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:30:26.394 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.920437+0000", 2024-09-14T06:30:26.395 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.065343221000000007 2024-09-14T06:30:26.395 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:26.395 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:26.395 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:30:26.395 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.920481+0000", 2024-09-14T06:30:26.395 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.4431000000000002e-05 2024-09-14T06:30:26.395 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:26.395 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:26.395 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:30:26.395 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.920651+0000", 2024-09-14T06:30:26.395 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00016913200000000001 2024-09-14T06:30:26.395 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:26.395 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:26.396 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:26.396 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:26.396 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:26.396 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:44522 3.2 3:4190d882:::rbd_data.112dca2f6ae1.0000000000000002:17 [read 110592~4096 [fadvise_nocache]] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:30:26.396 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:43.855098+0000", 2024-09-14T06:30:26.396 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 42.472901858999997, 2024-09-14T06:30:26.396 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.065827628999999999, 2024-09-14T06:30:26.396 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:30:26.396 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:30:26.396 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:30:26.396 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:30:26.396 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:30:26.396 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:26.396 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 44522 2024-09-14T06:30:26.396 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:26.397 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:30:26.397 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:26.397 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:30:26.397 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855098+0000", 2024-09-14T06:30:26.397 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:26.397 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:26.397 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:26.397 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:30:26.397 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855098+0000", 2024-09-14T06:30:26.397 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:26.397 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:26.397 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:26.397 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:30:26.398 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855098+0000", 2024-09-14T06:30:26.398 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.1e-07 2024-09-14T06:30:26.398 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:26.398 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:26.398 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:30:26.398 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855099+0000", 2024-09-14T06:30:26.398 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.012e-06 2024-09-14T06:30:26.398 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:26.398 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:26.398 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:30:26.398 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855102+0000", 2024-09-14T06:30:26.398 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.9380000000000001e-06 2024-09-14T06:30:26.398 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:26.398 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:26.398 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:30:26.398 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855108+0000", 2024-09-14T06:30:26.399 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.4339999999999997e-06 2024-09-14T06:30:26.399 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:26.399 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:26.399 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:30:26.399 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.920681+0000", 2024-09-14T06:30:26.399 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.065572943999999994 2024-09-14T06:30:26.399 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:26.399 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:26.399 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:30:26.399 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.920724+0000", 2024-09-14T06:30:26.399 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.2877000000000003e-05 2024-09-14T06:30:26.399 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:26.399 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:26.399 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:30:26.400 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.920925+0000", 2024-09-14T06:30:26.400 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00020121399999999999 2024-09-14T06:30:26.400 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:26.400 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:26.400 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:26.400 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:26.400 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:26.400 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:44523 3.2 3:4190d882:::rbd_data.112dca2f6ae1.0000000000000002:17 [read 135168~4096 [fadvise_nocache]] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:30:26.400 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:43.855111+0000", 2024-09-14T06:30:26.400 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 42.472888136999998, 2024-09-14T06:30:26.400 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.066028198999999996, 2024-09-14T06:30:26.400 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:30:26.400 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:30:26.401 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:30:26.401 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:30:26.401 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:30:26.401 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:26.401 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 44523 2024-09-14T06:30:26.401 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:26.401 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:30:26.401 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:26.401 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:30:26.401 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855111+0000", 2024-09-14T06:30:26.401 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:26.401 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:26.401 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:26.401 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:30:26.402 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855111+0000", 2024-09-14T06:30:26.402 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:26.402 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:26.402 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:26.402 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:30:26.402 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855112+0000", 2024-09-14T06:30:26.402 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.1199999999999999e-07 2024-09-14T06:30:26.402 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:26.402 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:26.402 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:30:26.402 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855115+0000", 2024-09-14T06:30:26.402 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.3380000000000001e-06 2024-09-14T06:30:26.403 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:26.403 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:26.403 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:30:26.403 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855118+0000", 2024-09-14T06:30:26.403 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.366e-06 2024-09-14T06:30:26.403 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:26.403 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:26.403 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:30:26.403 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855121+0000", 2024-09-14T06:30:26.403 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.8329999999999998e-06 2024-09-14T06:30:26.403 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:26.403 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:26.403 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:30:26.403 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.920956+0000", 2024-09-14T06:30:26.404 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.065835251999999997 2024-09-14T06:30:26.404 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:26.404 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:26.404 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:30:26.404 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.921001+0000", 2024-09-14T06:30:26.404 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.4778999999999999e-05 2024-09-14T06:30:26.404 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:26.404 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:26.404 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:30:26.404 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.921139+0000", 2024-09-14T06:30:26.404 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000138419 2024-09-14T06:30:26.404 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:26.404 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:26.404 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:26.405 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:26.405 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:26.405 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:44524 3.2 3:4190d882:::rbd_data.112dca2f6ae1.0000000000000002:17 [read 159744~4096 [fadvise_nocache]] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:30:26.405 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:43.855124+0000", 2024-09-14T06:30:26.405 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 42.472875158000001, 2024-09-14T06:30:26.405 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.066249261000000004, 2024-09-14T06:30:26.405 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:30:26.405 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:30:26.405 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:30:26.405 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:30:26.405 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:30:26.405 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:26.405 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 44524 2024-09-14T06:30:26.405 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:26.406 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:30:26.406 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:26.406 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:30:26.406 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855124+0000", 2024-09-14T06:30:26.406 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:26.406 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:26.406 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:26.406 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:30:26.406 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855124+0000", 2024-09-14T06:30:26.406 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:26.406 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:26.406 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:26.406 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:30:26.406 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861086+0000", 2024-09-14T06:30:26.407 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0059613829999999998 2024-09-14T06:30:26.407 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:26.407 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:26.407 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:30:26.407 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861100+0000", 2024-09-14T06:30:26.407 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.4644000000000001e-05 2024-09-14T06:30:26.407 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:26.407 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:26.407 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:30:26.407 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861109+0000", 2024-09-14T06:30:26.407 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.541e-06 2024-09-14T06:30:26.407 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:26.407 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:26.407 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:30:26.408 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861117+0000", 2024-09-14T06:30:26.408 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.1049999999999995e-06 2024-09-14T06:30:26.408 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:26.408 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:26.408 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:30:26.408 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.921169+0000", 2024-09-14T06:30:26.408 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.060052405000000003 2024-09-14T06:30:26.408 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:26.408 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:26.408 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:30:26.408 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.921213+0000", 2024-09-14T06:30:26.408 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.3565e-05 2024-09-14T06:30:26.408 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:26.408 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:26.409 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:30:26.409 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.921374+0000", 2024-09-14T06:30:26.409 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00016061800000000001 2024-09-14T06:30:26.409 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:26.409 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:26.409 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:26.409 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:26.409 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:26.409 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:44536 3.2 3:4190d882:::rbd_data.112dca2f6ae1.0000000000000002:17 [read 454656~4096 [fadvise_nocache]] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:30:26.409 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:43.861384+0000", 2024-09-14T06:30:26.409 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 42.466615601000001, 2024-09-14T06:30:26.409 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.063058528000000003, 2024-09-14T06:30:26.409 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:30:26.409 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:30:26.410 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:30:26.410 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:30:26.410 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:30:26.410 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:26.410 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 44536 2024-09-14T06:30:26.410 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:26.410 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:30:26.410 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:26.410 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:30:26.410 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861384+0000", 2024-09-14T06:30:26.410 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:26.410 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:26.410 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:26.410 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:30:26.410 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861384+0000", 2024-09-14T06:30:26.411 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:26.411 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:26.411 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:26.411 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:30:26.411 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861384+0000", 2024-09-14T06:30:26.411 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.4000000000000002e-07 2024-09-14T06:30:26.411 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:26.411 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:26.411 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:30:26.411 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861386+0000", 2024-09-14T06:30:26.411 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.891e-06 2024-09-14T06:30:26.411 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:26.411 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:26.412 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:30:26.412 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861392+0000", 2024-09-14T06:30:26.412 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.9939999999999997e-06 2024-09-14T06:30:26.412 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:26.412 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:26.412 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:30:26.412 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861398+0000", 2024-09-14T06:30:26.412 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.507e-06 2024-09-14T06:30:26.412 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:26.412 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:26.412 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:30:26.412 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.924149+0000", 2024-09-14T06:30:26.412 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.062751135999999999 2024-09-14T06:30:26.413 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:26.413 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:26.413 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:30:26.413 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.924226+0000", 2024-09-14T06:30:26.413 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.7517999999999999e-05 2024-09-14T06:30:26.413 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:26.413 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:26.413 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:30:26.413 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.924442+0000", 2024-09-14T06:30:26.413 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000216042 2024-09-14T06:30:26.413 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:26.413 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:26.414 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:26.414 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:26.414 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:26.414 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:44537 3.2 3:4190d882:::rbd_data.112dca2f6ae1.0000000000000002:17 [read 479232~4096 [fadvise_nocache]] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:30:26.414 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:43.861405+0000", 2024-09-14T06:30:26.414 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 42.466594555, 2024-09-14T06:30:26.414 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.063351795000000002, 2024-09-14T06:30:26.414 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:30:26.414 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:30:26.414 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:30:26.414 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:30:26.414 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:30:26.414 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:26.414 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 44537 2024-09-14T06:30:26.415 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:26.415 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:30:26.415 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:26.415 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:30:26.415 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861405+0000", 2024-09-14T06:30:26.415 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:26.415 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:26.415 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:26.415 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:30:26.415 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861405+0000", 2024-09-14T06:30:26.415 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:26.415 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:26.415 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:26.415 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:30:26.416 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861405+0000", 2024-09-14T06:30:26.416 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.9099999999999999e-07 2024-09-14T06:30:26.416 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:26.416 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:26.416 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:30:26.416 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861407+0000", 2024-09-14T06:30:26.416 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.075e-06 2024-09-14T06:30:26.416 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:26.416 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:26.416 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:30:26.416 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861414+0000", 2024-09-14T06:30:26.416 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.4659999999999996e-06 2024-09-14T06:30:26.416 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:26.416 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:26.417 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:30:26.417 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861418+0000", 2024-09-14T06:30:26.417 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.5519999999999998e-06 2024-09-14T06:30:26.417 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:26.417 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:26.417 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:30:26.417 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.924474+0000", 2024-09-14T06:30:26.417 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.063056012999999994 2024-09-14T06:30:26.417 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:26.417 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:26.417 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:30:26.417 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.924556+0000", 2024-09-14T06:30:26.417 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.1181999999999996e-05 2024-09-14T06:30:26.417 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:26.418 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:26.418 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:30:26.418 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.924757+0000", 2024-09-14T06:30:26.418 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00020111599999999999 2024-09-14T06:30:26.418 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:26.418 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:26.418 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:26.418 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:26.418 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:26.418 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:44538 3.2 3:4190d882:::rbd_data.112dca2f6ae1.0000000000000002:17 [read 503808~4096 [fadvise_nocache]] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:30:26.418 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:43.861425+0000", 2024-09-14T06:30:26.418 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 42.466574514000001, 2024-09-14T06:30:26.418 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.063627125000000007, 2024-09-14T06:30:26.418 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:30:26.418 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:30:26.419 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:30:26.419 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:30:26.419 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:30:26.419 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:26.419 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 44538 2024-09-14T06:30:26.419 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:26.419 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:30:26.419 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:26.419 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:30:26.419 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861425+0000", 2024-09-14T06:30:26.419 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:26.419 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:26.419 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:26.420 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:30:26.420 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861425+0000", 2024-09-14T06:30:26.420 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:26.420 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:26.420 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:26.420 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:30:26.420 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861425+0000", 2024-09-14T06:30:26.420 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.9999999999999998e-07 2024-09-14T06:30:26.420 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:26.420 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:26.420 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:30:26.420 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861427+0000", 2024-09-14T06:30:26.420 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.9829999999999999e-06 2024-09-14T06:30:26.421 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:26.421 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:26.421 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:30:26.421 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861433+0000", 2024-09-14T06:30:26.421 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.2889999999999997e-06 2024-09-14T06:30:26.421 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:26.421 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:26.421 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:30:26.421 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861454+0000", 2024-09-14T06:30:26.421 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.1844999999999998e-05 2024-09-14T06:30:26.421 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:26.421 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:26.421 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:30:26.421 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.924786+0000", 2024-09-14T06:30:26.422 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.063331497 2024-09-14T06:30:26.422 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:26.422 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:26.422 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:30:26.422 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.924864+0000", 2024-09-14T06:30:26.422 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.7744000000000006e-05 2024-09-14T06:30:26.422 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:26.422 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:26.422 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:30:26.422 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.925052+0000", 2024-09-14T06:30:26.422 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00018836699999999999 2024-09-14T06:30:26.422 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:26.422 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:26.422 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:26.423 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:26.423 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:26.423 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:44539 3.2 3:4190d882:::rbd_data.112dca2f6ae1.0000000000000002:17 [read 528384~4096 [fadvise_nocache]] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:30:26.423 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:43.861462+0000", 2024-09-14T06:30:26.423 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 42.466537187, 2024-09-14T06:30:26.423 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.063855705999999998, 2024-09-14T06:30:26.423 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:30:26.423 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:30:26.423 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:30:26.423 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:30:26.423 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:30:26.423 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:26.423 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 44539 2024-09-14T06:30:26.423 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:26.424 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:30:26.424 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:26.424 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:30:26.424 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861462+0000", 2024-09-14T06:30:26.424 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:26.424 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:26.424 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:26.424 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:30:26.424 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861462+0000", 2024-09-14T06:30:26.424 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:26.424 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:26.424 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:26.424 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:30:26.424 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861463+0000", 2024-09-14T06:30:26.425 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.5200000000000002e-07 2024-09-14T06:30:26.425 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:26.425 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:26.425 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:30:26.425 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861465+0000", 2024-09-14T06:30:26.425 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.5189999999999999e-06 2024-09-14T06:30:26.425 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:26.425 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:26.425 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:30:26.425 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861471+0000", 2024-09-14T06:30:26.425 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.6899999999999997e-06 2024-09-14T06:30:26.425 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:26.425 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:26.426 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:30:26.426 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861475+0000", 2024-09-14T06:30:26.426 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.5730000000000001e-06 2024-09-14T06:30:26.426 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:26.426 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:26.426 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:30:26.426 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.925085+0000", 2024-09-14T06:30:26.426 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.063609902999999995 2024-09-14T06:30:26.426 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:26.426 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:26.426 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:30:26.426 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.925145+0000", 2024-09-14T06:30:26.426 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.9383999999999999e-05 2024-09-14T06:30:26.426 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:26.427 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:26.427 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:30:26.427 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.925318+0000", 2024-09-14T06:30:26.427 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00017318499999999999 2024-09-14T06:30:26.427 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:26.427 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:26.427 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:26.427 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:26.427 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:26.427 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:44540 3.2 3:4190d882:::rbd_data.112dca2f6ae1.0000000000000002:17 [read 552960~4096 [fadvise_nocache]] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:30:26.427 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:43.861483+0000", 2024-09-14T06:30:26.427 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 42.466516755999997, 2024-09-14T06:30:26.427 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.064065681999999999, 2024-09-14T06:30:26.427 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:30:26.427 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:30:26.428 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:30:26.428 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:30:26.428 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:30:26.428 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:26.428 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 44540 2024-09-14T06:30:26.428 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:26.428 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:30:26.428 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:26.428 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:30:26.428 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861483+0000", 2024-09-14T06:30:26.428 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:26.428 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:26.428 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:26.428 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:30:26.429 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861483+0000", 2024-09-14T06:30:26.429 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:26.429 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:26.429 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:26.429 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:30:26.429 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.867079+0000", 2024-09-14T06:30:26.429 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0055966480000000001 2024-09-14T06:30:26.429 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:26.429 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:26.429 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:30:26.429 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.867086+0000", 2024-09-14T06:30:26.429 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.387e-06 2024-09-14T06:30:26.429 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:26.429 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:26.430 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:30:26.430 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.867094+0000", 2024-09-14T06:30:26.430 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.9419999999999995e-06 2024-09-14T06:30:26.430 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:26.430 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:26.430 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:30:26.430 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.867109+0000", 2024-09-14T06:30:26.430 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.5818999999999999e-05 2024-09-14T06:30:26.430 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:26.430 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:26.430 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:30:26.430 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.925354+0000", 2024-09-14T06:30:26.430 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.058244047 2024-09-14T06:30:26.430 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:26.431 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:26.431 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:30:26.431 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.925406+0000", 2024-09-14T06:30:26.431 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.2661999999999998e-05 2024-09-14T06:30:26.431 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:26.431 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:26.431 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:30:26.431 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.925548+0000", 2024-09-14T06:30:26.431 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000142177 2024-09-14T06:30:26.431 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:26.431 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:26.431 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:26.431 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:26.431 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:26.432 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-14T06:30:26.433 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-14T06:30:26.434 INFO:tasks.ceph.osd.4.smithi138.stderr:2024-09-14T06:30:26.432+0000 7f6c653c8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33615) UID: 0 2024-09-14T06:30:26.486 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-14T06:30:26.486 INFO:teuthology.orchestra.run.smithi094.stderr:dumped all 2024-09-14T06:30:26.501 INFO:teuthology.orchestra.run.smithi094.stdout:{"pg_ready":true,"pg_map":{"version":150,"stamp":"2024-09-14T06:30:25.626366+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1076304674,"num_objects":1273,"num_object_clones":0,"num_object_copies":2546,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1273,"num_whiteouts":0,"num_read":126140,"num_read_kb":1168852,"num_write":14922,"num_write_kb":810860,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":39,"num_bytes_recovered":1400964,"num_keys_recovered":11,"num_objects_omap":23,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":11074,"ondisk_log_size":11074,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":417556,"kb_used_data":80608,"kb_used_omap":12,"kb_used_meta":336755,"kb_avail":754557164,"statfs":{"total":773094113280,"available":772666535936,"internally_reserved":0,"allocated":82542592,"data_stored":718853824,"data_compressed":43901138,"data_compressed_allocated":57856000,"data_compressed_original":702433820,"omap_allocated":12751,"internal_metadata":344837681},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-76259709,"num_objects":-173,"num_object_clones":0,"num_object_copies":-346,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-173,"num_whiteouts":0,"num_read":3698,"num_read_kb":44877,"num_write":708,"num_write_kb":27347,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.025376"},"pg_stats":[{"pgid":"3.1f","version":"101'334","reported_seq":3778,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-14T06:30:22.776159+0000","last_change":"2024-09-14T06:30:22.609542+0000","last_active":"2024-09-14T06:30:22.776159+0000","last_peered":"2024-09-14T06:30:22.776159+0000","last_clean":"2024-09-14T06:30:22.776159+0000","last_became_active":"2024-09-14T06:29:00.579883+0000","last_became_peered":"2024-09-14T06:29:00.579883+0000","last_unstale":"2024-09-14T06:30:22.776159+0000","last_undegraded":"2024-09-14T06:30:22.776159+0000","last_fullsized":"2024-09-14T06:30:22.776159+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":334,"log_dups_size":0,"ondisk_log_size":334,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T18:04:30.063494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070539599999999995,"stat_sum":{"num_bytes":34617406,"num_objects":37,"num_object_clones":0,"num_object_copies":74,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":37,"num_whiteouts":0,"num_read":4250,"num_read_kb":37880,"num_write":488,"num_write_kb":19418,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"2a","length":"1"},{"start":"2c","length":"1"}]},{"pgid":"3.1e","version":"99'201","reported_seq":4809,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-09-14T06:30:20.627444+0000","last_change":"2024-09-14T06:30:19.605970+0000","last_active":"2024-09-14T06:30:20.627444+0000","last_peered":"2024-09-14T06:30:20.627444+0000","last_clean":"2024-09-14T06:30:20.627444+0000","last_became_active":"2024-09-14T06:29:00.580507+0000","last_became_peered":"2024-09-14T06:29:00.580507+0000","last_unstale":"2024-09-14T06:30:20.627444+0000","last_undegraded":"2024-09-14T06:30:20.627444+0000","last_fullsized":"2024-09-14T06:30:20.627444+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":201,"log_dups_size":0,"ondisk_log_size":201,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T14:51:47.269336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00099787300000000003,"stat_sum":{"num_bytes":28440908,"num_objects":33,"num_object_clones":0,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":33,"num_whiteouts":0,"num_read":4312,"num_read_kb":34438,"num_write":214,"num_write_kb":24792,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2a","length":"1"}]},{"pgid":"3.1d","version":"98'475","reported_seq":4963,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-14T06:30:23.623900+0000","last_change":"2024-09-14T06:30:22.612583+0000","last_active":"2024-09-14T06:30:23.623900+0000","last_peered":"2024-09-14T06:30:23.623900+0000","last_clean":"2024-09-14T06:30:23.623900+0000","last_became_active":"2024-09-14T06:29:00.579345+0000","last_became_peered":"2024-09-14T06:29:00.579345+0000","last_unstale":"2024-09-14T06:30:23.623900+0000","last_undegraded":"2024-09-14T06:30:23.623900+0000","last_fullsized":"2024-09-14T06:30:23.623900+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":475,"log_dups_size":0,"ondisk_log_size":475,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T15:03:36.602483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014172709999999999,"stat_sum":{"num_bytes":42985469,"num_objects":38,"num_object_clones":0,"num_object_copies":76,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":38,"num_whiteouts":0,"num_read":5627,"num_read_kb":37020,"num_write":717,"num_write_kb":24288,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2a","length":"1"},{"start":"2c","length":"1"}]},{"pgid":"3.1c","version":"99'275","reported_seq":2557,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-14T06:30:23.623753+0000","last_change":"2024-09-14T06:30:22.611576+0000","last_active":"2024-09-14T06:30:23.623753+0000","last_peered":"2024-09-14T06:30:23.623753+0000","last_clean":"2024-09-14T06:30:23.623753+0000","last_became_active":"2024-09-14T06:29:00.579807+0000","last_became_peered":"2024-09-14T06:29:00.579807+0000","last_unstale":"2024-09-14T06:30:23.623753+0000","last_undegraded":"2024-09-14T06:30:23.623753+0000","last_fullsized":"2024-09-14T06:30:23.623753+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":275,"log_dups_size":0,"ondisk_log_size":275,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T17:01:10.314780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00044109299999999999,"stat_sum":{"num_bytes":17988923,"num_objects":45,"num_object_clones":0,"num_object_copies":90,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":45,"num_whiteouts":0,"num_read":2531,"num_read_kb":28395,"num_write":335,"num_write_kb":27771,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2a","length":"1"},{"start":"2c","length":"1"}]},{"pgid":"3.1b","version":"100'146","reported_seq":2624,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-14T06:30:22.609118+0000","last_change":"2024-09-14T06:30:22.609284+0000","last_active":"2024-09-14T06:30:22.609118+0000","last_peered":"2024-09-14T06:30:22.609118+0000","last_clean":"2024-09-14T06:30:22.609118+0000","last_became_active":"2024-09-14T06:29:00.586183+0000","last_became_peered":"2024-09-14T06:29:00.586183+0000","last_unstale":"2024-09-14T06:30:22.609118+0000","last_undegraded":"2024-09-14T06:30:22.609118+0000","last_fullsized":"2024-09-14T06:30:22.609118+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":146,"log_dups_size":0,"ondisk_log_size":146,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T16:39:32.847224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069743099999999998,"stat_sum":{"num_bytes":28753401,"num_objects":51,"num_object_clones":0,"num_object_copies":102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":51,"num_whiteouts":0,"num_read":2165,"num_read_kb":28869,"num_write":165,"num_write_kb":31023,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"2a","length":"1"},{"start":"2c","length":"1"}]},{"pgid":"3.1a","version":"101'861","reported_seq":5350,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-14T06:30:23.612816+0000","last_change":"2024-09-14T06:30:22.607748+0000","last_active":"2024-09-14T06:30:23.612816+0000","last_peered":"2024-09-14T06:30:23.612816+0000","last_clean":"2024-09-14T06:30:23.612816+0000","last_became_active":"2024-09-14T06:29:00.576206+0000","last_became_peered":"2024-09-14T06:29:00.576206+0000","last_unstale":"2024-09-14T06:30:23.612816+0000","last_undegraded":"2024-09-14T06:30:23.612816+0000","last_fullsized":"2024-09-14T06:30:23.612816+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":861,"log_dups_size":0,"ondisk_log_size":861,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T11:53:22.482997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00055519200000000003,"stat_sum":{"num_bytes":39465380,"num_objects":49,"num_object_clones":0,"num_object_copies":98,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":49,"num_whiteouts":0,"num_read":7381,"num_read_kb":42158,"num_write":1224,"num_write_kb":31953,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"2a","length":"1"},{"start":"2c","length":"1"}]},{"pgid":"3.19","version":"98'251","reported_seq":2833,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-14T06:30:23.612350+0000","last_change":"2024-09-14T06:30:22.607773+0000","last_active":"2024-09-14T06:30:23.612350+0000","last_peered":"2024-09-14T06:30:23.612350+0000","last_clean":"2024-09-14T06:30:23.612350+0000","last_became_active":"2024-09-14T06:29:00.575639+0000","last_became_peered":"2024-09-14T06:29:00.575639+0000","last_unstale":"2024-09-14T06:30:23.612350+0000","last_undegraded":"2024-09-14T06:30:23.612350+0000","last_fullsized":"2024-09-14T06:30:23.612350+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":251,"log_dups_size":0,"ondisk_log_size":251,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T18:21:05.685332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00042186099999999999,"stat_sum":{"num_bytes":34299046,"num_objects":42,"num_object_clones":0,"num_object_copies":84,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":42,"num_whiteouts":0,"num_read":2835,"num_read_kb":42010,"num_write":350,"num_write_kb":26598,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2a","length":"1"},{"start":"2c","length":"1"}]},{"pgid":"3.18","version":"98'162","reported_seq":2269,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-09-14T06:30:20.594128+0000","last_change":"2024-09-14T06:30:19.605348+0000","last_active":"2024-09-14T06:30:20.594128+0000","last_peered":"2024-09-14T06:30:20.594128+0000","last_clean":"2024-09-14T06:30:20.594128+0000","last_became_active":"2024-09-14T06:29:00.578017+0000","last_became_peered":"2024-09-14T06:29:00.578017+0000","last_unstale":"2024-09-14T06:30:20.594128+0000","last_undegraded":"2024-09-14T06:30:20.594128+0000","last_fullsized":"2024-09-14T06:30:20.594128+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":162,"log_dups_size":0,"ondisk_log_size":162,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T10:21:31.247709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00052285199999999997,"stat_sum":{"num_bytes":22128117,"num_objects":35,"num_object_clones":0,"num_object_copies":70,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":35,"num_whiteouts":0,"num_read":1871,"num_read_kb":30431,"num_write":173,"num_write_kb":20338,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2a","length":"1"}]},{"pgid":"3.17","version":"98'431","reported_seq":6415,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-14T06:30:22.609174+0000","last_change":"2024-09-14T06:30:22.609358+0000","last_active":"2024-09-14T06:30:22.609174+0000","last_peered":"2024-09-14T06:30:22.609174+0000","last_clean":"2024-09-14T06:30:22.609174+0000","last_became_active":"2024-09-14T06:29:00.579714+0000","last_became_peered":"2024-09-14T06:29:00.579714+0000","last_unstale":"2024-09-14T06:30:22.609174+0000","last_undegraded":"2024-09-14T06:30:22.609174+0000","last_fullsized":"2024-09-14T06:30:22.609174+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":431,"log_dups_size":0,"ondisk_log_size":431,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T06:50:19.988471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00073233300000000005,"stat_sum":{"num_bytes":50200381,"num_objects":54,"num_object_clones":0,"num_object_copies":108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":54,"num_whiteouts":0,"num_read":6922,"num_read_kb":48124,"num_write":516,"num_write_kb":27292,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"2a","length":"1"},{"start":"2c","length":"1"}]},{"pgid":"3.16","version":"102'198","reported_seq":1240,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-14T06:30:23.628690+0000","last_change":"2024-09-14T06:30:22.612495+0000","last_active":"2024-09-14T06:30:23.628690+0000","last_peered":"2024-09-14T06:30:23.628690+0000","last_clean":"2024-09-14T06:30:23.628690+0000","last_became_active":"2024-09-14T06:29:00.580070+0000","last_became_peered":"2024-09-14T06:29:00.580070+0000","last_unstale":"2024-09-14T06:30:23.628690+0000","last_undegraded":"2024-09-14T06:30:23.628690+0000","last_fullsized":"2024-09-14T06:30:23.628690+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":198,"log_dups_size":0,"ondisk_log_size":198,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T14:14:36.261460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013373479999999999,"stat_sum":{"num_bytes":14704807,"num_objects":33,"num_object_clones":0,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":33,"num_whiteouts":0,"num_read":1219,"num_read_kb":11360,"num_write":246,"num_write_kb":14337,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2a","length":"1"},{"start":"2c","length":"1"}]},{"pgid":"3.15","version":"101'266","reported_seq":2864,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-14T06:30:23.625554+0000","last_change":"2024-09-14T06:30:22.614903+0000","last_active":"2024-09-14T06:30:23.625554+0000","last_peered":"2024-09-14T06:30:23.625554+0000","last_clean":"2024-09-14T06:30:23.625554+0000","last_became_active":"2024-09-14T06:29:00.578451+0000","last_became_peered":"2024-09-14T06:29:00.578451+0000","last_unstale":"2024-09-14T06:30:23.625554+0000","last_undegraded":"2024-09-14T06:30:23.625554+0000","last_fullsized":"2024-09-14T06:30:23.625554+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":266,"log_dups_size":0,"ondisk_log_size":266,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T17:53:54.817492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071216900000000002,"stat_sum":{"num_bytes":34640790,"num_objects":40,"num_object_clones":0,"num_object_copies":80,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":40,"num_whiteouts":0,"num_read":2459,"num_read_kb":49809,"num_write":281,"num_write_kb":36728,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"2a","length":"1"},{"start":"2c","length":"1"}]},{"pgid":"3.14","version":"98'669","reported_seq":3782,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-14T06:30:23.612653+0000","last_change":"2024-09-14T06:30:22.608051+0000","last_active":"2024-09-14T06:30:23.612653+0000","last_peered":"2024-09-14T06:30:23.612653+0000","last_clean":"2024-09-14T06:30:23.612653+0000","last_became_active":"2024-09-14T06:29:00.582595+0000","last_became_peered":"2024-09-14T06:29:00.582595+0000","last_unstale":"2024-09-14T06:30:23.612653+0000","last_undegraded":"2024-09-14T06:30:23.612653+0000","last_fullsized":"2024-09-14T06:30:23.612653+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":669,"log_dups_size":0,"ondisk_log_size":669,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T12:33:31.116704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070081199999999998,"stat_sum":{"num_bytes":34531427,"num_objects":32,"num_object_clones":0,"num_object_copies":64,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":32,"num_whiteouts":0,"num_read":6278,"num_read_kb":32936,"num_write":1041,"num_write_kb":15101,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"2a","length":"1"},{"start":"2c","length":"1"}]},{"pgid":"2.7","version":"0'0","reported_seq":178,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-14T06:30:23.616881+0000","last_change":"2024-09-14T06:28:53.625193+0000","last_active":"2024-09-14T06:30:23.616881+0000","last_peered":"2024-09-14T06:30:23.616881+0000","last_clean":"2024-09-14T06:30:23.616881+0000","last_became_active":"2024-09-14T06:28:51.223535+0000","last_became_peered":"2024-09-14T06:28:51.223535+0000","last_unstale":"2024-09-14T06:30:23.616881+0000","last_undegraded":"2024-09-14T06:30:23.616881+0000","last_fullsized":"2024-09-14T06:30:23.616881+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:28:50.205350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:28:50.205350+0000","last_clean_scrub_stamp":"2024-09-14T06:28:50.205350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T18:14:22.822245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00052660300000000005,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"98'212","reported_seq":2211,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-14T06:30:22.608745+0000","last_change":"2024-09-14T06:30:22.608745+0000","last_active":"2024-09-14T06:30:22.608745+0000","last_peered":"2024-09-14T06:30:22.608745+0000","last_clean":"2024-09-14T06:30:22.608745+0000","last_became_active":"2024-09-14T06:29:00.578283+0000","last_became_peered":"2024-09-14T06:29:00.578283+0000","last_unstale":"2024-09-14T06:30:22.608745+0000","last_undegraded":"2024-09-14T06:30:22.608745+0000","last_fullsized":"2024-09-14T06:30:22.608745+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":212,"log_dups_size":0,"ondisk_log_size":212,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T14:11:15.343781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059847500000000001,"stat_sum":{"num_bytes":24088082,"num_objects":32,"num_object_clones":0,"num_object_copies":64,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":32,"num_whiteouts":0,"num_read":1945,"num_read_kb":19588,"num_write":227,"num_write_kb":16712,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"2a","length":"1"},{"start":"2c","length":"1"}]},{"pgid":"2.6","version":"0'0","reported_seq":180,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-14T06:30:23.612577+0000","last_change":"2024-09-14T06:28:53.226122+0000","last_active":"2024-09-14T06:30:23.612577+0000","last_peered":"2024-09-14T06:30:23.612577+0000","last_clean":"2024-09-14T06:30:23.612577+0000","last_became_active":"2024-09-14T06:28:51.217205+0000","last_became_peered":"2024-09-14T06:28:51.217205+0000","last_unstale":"2024-09-14T06:30:23.612577+0000","last_undegraded":"2024-09-14T06:30:23.612577+0000","last_fullsized":"2024-09-14T06:30:23.612577+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:28:50.205350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:28:50.205350+0000","last_clean_scrub_stamp":"2024-09-14T06:28:50.205350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T15:29:19.563302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065018200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"100'329","reported_seq":1601,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-14T06:30:22.608812+0000","last_change":"2024-09-14T06:30:22.609046+0000","last_active":"2024-09-14T06:30:22.608812+0000","last_peered":"2024-09-14T06:30:22.608812+0000","last_clean":"2024-09-14T06:30:22.608812+0000","last_became_active":"2024-09-14T06:29:56.584268+0000","last_became_peered":"2024-09-14T06:29:56.584268+0000","last_unstale":"2024-09-14T06:30:22.608812+0000","last_undegraded":"2024-09-14T06:30:22.608812+0000","last_fullsized":"2024-09-14T06:30:22.608812+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":329,"log_dups_size":0,"ondisk_log_size":329,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T11:34:58.334730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000683674,"stat_sum":{"num_bytes":24555353,"num_objects":30,"num_object_clones":0,"num_object_copies":60,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":30,"num_whiteouts":0,"num_read":1738,"num_read_kb":16219,"num_write":466,"num_write_kb":18867,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":39,"num_bytes_recovered":1400964,"num_keys_recovered":11,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"2a","length":"1"},{"start":"2c","length":"1"}]},{"pgid":"2.5","version":"0'0","reported_seq":180,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-14T06:30:23.625776+0000","last_change":"2024-09-14T06:28:53.640035+0000","last_active":"2024-09-14T06:30:23.625776+0000","last_peered":"2024-09-14T06:30:23.625776+0000","last_clean":"2024-09-14T06:30:23.625776+0000","last_became_active":"2024-09-14T06:28:51.308058+0000","last_became_peered":"2024-09-14T06:28:51.308058+0000","last_unstale":"2024-09-14T06:30:23.625776+0000","last_undegraded":"2024-09-14T06:30:23.625776+0000","last_fullsized":"2024-09-14T06:30:23.625776+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:28:50.205350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:28:50.205350+0000","last_clean_scrub_stamp":"2024-09-14T06:28:50.205350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T13:35:48.544057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00037824000000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"101'190","reported_seq":1076,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-14T06:30:23.612476+0000","last_change":"2024-09-14T06:30:22.607515+0000","last_active":"2024-09-14T06:30:23.612476+0000","last_peered":"2024-09-14T06:30:23.612476+0000","last_clean":"2024-09-14T06:30:23.612476+0000","last_became_active":"2024-09-14T06:29:00.574895+0000","last_became_peered":"2024-09-14T06:29:00.574895+0000","last_unstale":"2024-09-14T06:30:23.612476+0000","last_undegraded":"2024-09-14T06:30:23.612476+0000","last_fullsized":"2024-09-14T06:30:23.612476+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":190,"log_dups_size":0,"ondisk_log_size":190,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T08:52:01.755709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00044466900000000002,"stat_sum":{"num_bytes":13236271,"num_objects":35,"num_object_clones":0,"num_object_copies":70,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":35,"num_whiteouts":0,"num_read":730,"num_read_kb":14922,"num_write":238,"num_write_kb":19950,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2a","length":"1"},{"start":"2c","length":"1"}]},{"pgid":"2.4","version":"0'0","reported_seq":180,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-14T06:30:23.612538+0000","last_change":"2024-09-14T06:28:53.226072+0000","last_active":"2024-09-14T06:30:23.612538+0000","last_peered":"2024-09-14T06:30:23.612538+0000","last_clean":"2024-09-14T06:30:23.612538+0000","last_became_active":"2024-09-14T06:28:51.308072+0000","last_became_peered":"2024-09-14T06:28:51.308072+0000","last_unstale":"2024-09-14T06:30:23.612538+0000","last_undegraded":"2024-09-14T06:30:23.612538+0000","last_fullsized":"2024-09-14T06:30:23.612538+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:28:50.205350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:28:50.205350+0000","last_clean_scrub_stamp":"2024-09-14T06:28:50.205350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T13:15:09.052328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066197399999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"98'541","reported_seq":2231,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-14T06:30:23.623910+0000","last_change":"2024-09-14T06:30:22.612828+0000","last_active":"2024-09-14T06:30:23.623910+0000","last_peered":"2024-09-14T06:30:23.623910+0000","last_clean":"2024-09-14T06:30:23.623910+0000","last_became_active":"2024-09-14T06:29:00.581874+0000","last_became_peered":"2024-09-14T06:29:00.581874+0000","last_unstale":"2024-09-14T06:30:23.623910+0000","last_undegraded":"2024-09-14T06:30:23.623910+0000","last_fullsized":"2024-09-14T06:30:23.623910+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":541,"log_dups_size":0,"ondisk_log_size":541,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T14:14:30.760746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015671820000000001,"stat_sum":{"num_bytes":27242971,"num_objects":41,"num_object_clones":0,"num_object_copies":82,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":41,"num_whiteouts":0,"num_read":2810,"num_read_kb":19328,"num_write":837,"num_write_kb":30177,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2a","length":"1"},{"start":"2c","length":"1"}]},{"pgid":"3.3","version":"99'222","reported_seq":7695,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-14T06:30:23.612660+0000","last_change":"2024-09-14T06:30:22.608139+0000","last_active":"2024-09-14T06:30:23.612660+0000","last_peered":"2024-09-14T06:30:23.612660+0000","last_clean":"2024-09-14T06:30:23.612660+0000","last_became_active":"2024-09-14T06:29:00.584678+0000","last_became_peered":"2024-09-14T06:29:00.584678+0000","last_unstale":"2024-09-14T06:30:23.612660+0000","last_undegraded":"2024-09-14T06:30:23.612660+0000","last_fullsized":"2024-09-14T06:30:23.612660+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":222,"log_dups_size":0,"ondisk_log_size":222,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T07:15:15.023452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00072175699999999995,"stat_sum":{"num_bytes":49275751,"num_objects":32,"num_object_clones":0,"num_object_copies":64,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":32,"num_whiteouts":0,"num_read":7398,"num_read_kb":63784,"num_write":218,"num_write_kb":32775,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"2a","length":"1"},{"start":"2c","length":"1"}]},{"pgid":"2.2","version":"22'2","reported_seq":182,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-14T06:30:23.623758+0000","last_change":"2024-09-14T06:28:53.231428+0000","last_active":"2024-09-14T06:30:23.623758+0000","last_peered":"2024-09-14T06:30:23.623758+0000","last_clean":"2024-09-14T06:30:23.623758+0000","last_became_active":"2024-09-14T06:28:51.222937+0000","last_became_peered":"2024-09-14T06:28:51.222937+0000","last_unstale":"2024-09-14T06:30:23.623758+0000","last_undegraded":"2024-09-14T06:30:23.623758+0000","last_fullsized":"2024-09-14T06:30:23.623758+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:28:50.205350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:28:50.205350+0000","last_clean_scrub_stamp":"2024-09-14T06:28:50.205350+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T14:27:37.233976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010072970000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.0","version":"101'175","reported_seq":3918,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-14T06:30:23.612343+0000","last_change":"2024-09-14T06:30:22.607287+0000","last_active":"2024-09-14T06:30:23.612343+0000","last_peered":"2024-09-14T06:30:23.612343+0000","last_clean":"2024-09-14T06:30:23.612343+0000","last_became_active":"2024-09-14T06:29:00.574873+0000","last_became_peered":"2024-09-14T06:29:00.574873+0000","last_unstale":"2024-09-14T06:30:23.612343+0000","last_undegraded":"2024-09-14T06:30:23.612343+0000","last_fullsized":"2024-09-14T06:30:23.612343+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":175,"log_dups_size":0,"ondisk_log_size":175,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T08:41:53.465666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054049399999999998,"stat_sum":{"num_bytes":43263336,"num_objects":50,"num_object_clones":0,"num_object_copies":100,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":50,"num_whiteouts":0,"num_read":3438,"num_read_kb":52268,"num_write":221,"num_write_kb":31522,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2a","length":"1"},{"start":"2c","length":"1"}]},{"pgid":"2.1","version":"0'0","reported_seq":177,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-14T06:30:22.701684+0000","last_change":"2024-09-14T06:28:53.226264+0000","last_active":"2024-09-14T06:30:22.701684+0000","last_peered":"2024-09-14T06:30:22.701684+0000","last_clean":"2024-09-14T06:30:22.701684+0000","last_became_active":"2024-09-14T06:28:51.223252+0000","last_became_peered":"2024-09-14T06:28:51.223252+0000","last_unstale":"2024-09-14T06:30:22.701684+0000","last_undegraded":"2024-09-14T06:30:22.701684+0000","last_fullsized":"2024-09-14T06:30:22.701684+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:28:50.205350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:28:50.205350+0000","last_clean_scrub_stamp":"2024-09-14T06:28:50.205350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T11:56:04.373414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053403100000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1","version":"98'438","reported_seq":7025,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-14T06:30:22.608782+0000","last_change":"2024-09-14T06:30:22.608782+0000","last_active":"2024-09-14T06:30:22.608782+0000","last_peered":"2024-09-14T06:30:22.608782+0000","last_clean":"2024-09-14T06:30:22.608782+0000","last_became_active":"2024-09-14T06:29:00.585673+0000","last_became_peered":"2024-09-14T06:29:00.585673+0000","last_unstale":"2024-09-14T06:30:22.608782+0000","last_undegraded":"2024-09-14T06:30:22.608782+0000","last_fullsized":"2024-09-14T06:30:22.608782+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":438,"log_dups_size":0,"ondisk_log_size":438,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T10:35:05.573347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066394999999999996,"stat_sum":{"num_bytes":53320861,"num_objects":37,"num_object_clones":0,"num_object_copies":74,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":37,"num_whiteouts":0,"num_read":7457,"num_read_kb":62632,"num_write":553,"num_write_kb":24186,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"2a","length":"1"},{"start":"2c","length":"1"}]},{"pgid":"2.0","version":"0'0","reported_seq":180,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-14T06:30:23.625513+0000","last_change":"2024-09-14T06:28:53.640096+0000","last_active":"2024-09-14T06:30:23.625513+0000","last_peered":"2024-09-14T06:30:23.625513+0000","last_clean":"2024-09-14T06:30:23.625513+0000","last_became_active":"2024-09-14T06:28:51.223069+0000","last_became_peered":"2024-09-14T06:28:51.223069+0000","last_unstale":"2024-09-14T06:30:23.625513+0000","last_undegraded":"2024-09-14T06:30:23.625513+0000","last_fullsized":"2024-09-14T06:30:23.625513+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:28:50.205350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:28:50.205350+0000","last_clean_scrub_stamp":"2024-09-14T06:28:50.205350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T15:13:00.471509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00043714600000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"98'443","reported_seq":4312,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-09-14T06:30:20.594325+0000","last_change":"2024-09-14T06:30:19.605702+0000","last_active":"2024-09-14T06:30:20.594325+0000","last_peered":"2024-09-14T06:30:20.594325+0000","last_clean":"2024-09-14T06:30:20.594325+0000","last_became_active":"2024-09-14T06:29:00.578574+0000","last_became_peered":"2024-09-14T06:29:00.578574+0000","last_unstale":"2024-09-14T06:30:20.594325+0000","last_undegraded":"2024-09-14T06:30:20.594325+0000","last_fullsized":"2024-09-14T06:30:20.594325+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":443,"log_dups_size":0,"ondisk_log_size":443,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T16:33:33.451809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000795983,"stat_sum":{"num_bytes":43675338,"num_objects":51,"num_object_clones":0,"num_object_copies":102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":51,"num_whiteouts":0,"num_read":4515,"num_read_kb":44617,"num_write":566,"num_write_kb":32487,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2a","length":"1"}]},{"pgid":"2.3","version":"20'1","reported_seq":181,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-14T06:30:23.623743+0000","last_change":"2024-09-14T06:28:53.230933+0000","last_active":"2024-09-14T06:30:23.623743+0000","last_peered":"2024-09-14T06:30:23.623743+0000","last_clean":"2024-09-14T06:30:23.623743+0000","last_became_active":"2024-09-14T06:28:51.222874+0000","last_became_peered":"2024-09-14T06:28:51.222874+0000","last_unstale":"2024-09-14T06:30:23.623743+0000","last_undegraded":"2024-09-14T06:30:23.623743+0000","last_fullsized":"2024-09-14T06:30:23.623743+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:28:50.205350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:28:50.205350+0000","last_clean_scrub_stamp":"2024-09-14T06:28:50.205350+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T12:04:25.190171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050808899999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'72","reported_seq":271,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-14T06:30:23.625525+0000","last_change":"2024-09-14T06:28:44.839885+0000","last_active":"2024-09-14T06:30:23.625525+0000","last_peered":"2024-09-14T06:30:23.625525+0000","last_clean":"2024-09-14T06:30:23.625525+0000","last_became_active":"2024-09-14T06:28:44.839573+0000","last_became_peered":"2024-09-14T06:28:44.839573+0000","last_unstale":"2024-09-14T06:30:23.625525+0000","last_undegraded":"2024-09-14T06:30:23.625525+0000","last_fullsized":"2024-09-14T06:30:23.625525+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:28:43.828494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:28:43.828494+0000","last_clean_scrub_stamp":"2024-09-14T06:28:43.828494+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T14:41:25.787293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.8","version":"98'223","reported_seq":3470,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-09-14T06:30:20.594320+0000","last_change":"2024-09-14T06:30:19.605423+0000","last_active":"2024-09-14T06:30:20.594320+0000","last_peered":"2024-09-14T06:30:20.594320+0000","last_clean":"2024-09-14T06:30:20.594320+0000","last_became_active":"2024-09-14T06:29:00.574458+0000","last_became_peered":"2024-09-14T06:29:00.574458+0000","last_unstale":"2024-09-14T06:30:20.594320+0000","last_undegraded":"2024-09-14T06:30:20.594320+0000","last_fullsized":"2024-09-14T06:30:20.594320+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":223,"log_dups_size":0,"ondisk_log_size":223,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T07:20:14.169687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00055249400000000005,"stat_sum":{"num_bytes":45016747,"num_objects":46,"num_object_clones":0,"num_object_copies":92,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":46,"num_whiteouts":0,"num_read":3511,"num_read_kb":66239,"num_write":275,"num_write_kb":38280,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2a","length":"1"}]},{"pgid":"3.9","version":"99'143","reported_seq":1318,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-14T06:30:23.612617+0000","last_change":"2024-09-14T06:30:22.607854+0000","last_active":"2024-09-14T06:30:23.612617+0000","last_peered":"2024-09-14T06:30:23.612617+0000","last_clean":"2024-09-14T06:30:23.612617+0000","last_became_active":"2024-09-14T06:29:00.573179+0000","last_became_peered":"2024-09-14T06:29:00.573179+0000","last_unstale":"2024-09-14T06:30:23.612617+0000","last_undegraded":"2024-09-14T06:30:23.612617+0000","last_fullsized":"2024-09-14T06:30:23.612617+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":143,"log_dups_size":0,"ondisk_log_size":143,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T06:49:09.156105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00052945200000000003,"stat_sum":{"num_bytes":18711894,"num_objects":43,"num_object_clones":0,"num_object_copies":86,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":43,"num_whiteouts":0,"num_read":861,"num_read_kb":29231,"num_write":173,"num_write_kb":22284,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"2a","length":"1"},{"start":"2c","length":"1"}]},{"pgid":"3.a","version":"98'178","reported_seq":2197,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-14T06:30:23.616877+0000","last_change":"2024-09-14T06:30:22.621471+0000","last_active":"2024-09-14T06:30:23.616877+0000","last_peered":"2024-09-14T06:30:23.616877+0000","last_clean":"2024-09-14T06:30:23.616877+0000","last_became_active":"2024-09-14T06:29:00.582426+0000","last_became_peered":"2024-09-14T06:29:00.582426+0000","last_unstale":"2024-09-14T06:30:23.616877+0000","last_undegraded":"2024-09-14T06:30:23.616877+0000","last_fullsized":"2024-09-14T06:30:23.616877+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":178,"log_dups_size":0,"ondisk_log_size":178,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T06:44:46.350248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049978999999999996,"stat_sum":{"num_bytes":37141935,"num_objects":37,"num_object_clones":0,"num_object_copies":74,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":37,"num_whiteouts":0,"num_read":1910,"num_read_kb":36958,"num_write":216,"num_write_kb":27112,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"2a","length":"1"},{"start":"2c","length":"1"}]},{"pgid":"3.b","version":"98'1081","reported_seq":4280,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-09-14T06:30:20.594226+0000","last_change":"2024-09-14T06:30:19.605593+0000","last_active":"2024-09-14T06:30:20.594226+0000","last_peered":"2024-09-14T06:30:20.594226+0000","last_clean":"2024-09-14T06:30:20.594226+0000","last_became_active":"2024-09-14T06:29:00.578483+0000","last_became_peered":"2024-09-14T06:29:00.578483+0000","last_unstale":"2024-09-14T06:30:20.594226+0000","last_undegraded":"2024-09-14T06:30:20.594226+0000","last_fullsized":"2024-09-14T06:30:20.594226+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":1081,"log_dups_size":0,"ondisk_log_size":1081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T06:43:20.596961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00068472499999999996,"stat_sum":{"num_bytes":21757058,"num_objects":41,"num_object_clones":0,"num_object_copies":82,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":41,"num_whiteouts":0,"num_read":8569,"num_read_kb":18146,"num_write":1817,"num_write_kb":17643,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2a","length":"1"}]},{"pgid":"3.c","version":"99'268","reported_seq":3130,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-14T06:30:23.623785+0000","last_change":"2024-09-14T06:30:22.612819+0000","last_active":"2024-09-14T06:30:23.623785+0000","last_peered":"2024-09-14T06:30:23.623785+0000","last_clean":"2024-09-14T06:30:23.623785+0000","last_became_active":"2024-09-14T06:29:00.583406+0000","last_became_peered":"2024-09-14T06:29:00.583406+0000","last_unstale":"2024-09-14T06:30:23.623785+0000","last_undegraded":"2024-09-14T06:30:23.623785+0000","last_fullsized":"2024-09-14T06:30:23.623785+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":268,"log_dups_size":0,"ondisk_log_size":268,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T08:19:59.181425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0016396290000000001,"stat_sum":{"num_bytes":26190419,"num_objects":36,"num_object_clones":0,"num_object_copies":72,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":36,"num_whiteouts":0,"num_read":3044,"num_read_kb":26426,"num_write":377,"num_write_kb":22175,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2a","length":"1"},{"start":"2c","length":"1"}]},{"pgid":"3.d","version":"98'200","reported_seq":3650,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-14T06:30:23.625667+0000","last_change":"2024-09-14T06:30:22.615128+0000","last_active":"2024-09-14T06:30:23.625667+0000","last_peered":"2024-09-14T06:30:23.625667+0000","last_clean":"2024-09-14T06:30:23.625667+0000","last_became_active":"2024-09-14T06:29:00.580362+0000","last_became_peered":"2024-09-14T06:29:00.580362+0000","last_unstale":"2024-09-14T06:30:23.625667+0000","last_undegraded":"2024-09-14T06:30:23.625667+0000","last_fullsized":"2024-09-14T06:30:23.625667+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":200,"log_dups_size":0,"ondisk_log_size":200,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T14:26:33.039528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00072795100000000001,"stat_sum":{"num_bytes":44608849,"num_objects":40,"num_object_clones":0,"num_object_copies":80,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":40,"num_whiteouts":0,"num_read":3364,"num_read_kb":45708,"num_write":248,"num_write_kb":21217,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"2a","length":"1"},{"start":"2c","length":"1"}]},{"pgid":"3.e","version":"98'273","reported_seq":1869,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-14T06:30:23.625718+0000","last_change":"2024-09-14T06:30:22.614970+0000","last_active":"2024-09-14T06:30:23.625718+0000","last_peered":"2024-09-14T06:30:23.625718+0000","last_clean":"2024-09-14T06:30:23.625718+0000","last_became_active":"2024-09-14T06:29:00.586374+0000","last_became_peered":"2024-09-14T06:29:00.586374+0000","last_unstale":"2024-09-14T06:30:23.625718+0000","last_undegraded":"2024-09-14T06:30:23.625718+0000","last_fullsized":"2024-09-14T06:30:23.625718+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":273,"log_dups_size":0,"ondisk_log_size":273,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T13:13:45.418557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069576999999999996,"stat_sum":{"num_bytes":25133180,"num_objects":45,"num_object_clones":0,"num_object_copies":90,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":45,"num_whiteouts":0,"num_read":1697,"num_read_kb":27585,"num_write":399,"num_write_kb":25425,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"2a","length":"1"},{"start":"2c","length":"1"}]},{"pgid":"3.f","version":"98'246","reported_seq":3429,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-14T06:30:23.625716+0000","last_change":"2024-09-14T06:30:22.615204+0000","last_active":"2024-09-14T06:30:23.625716+0000","last_peered":"2024-09-14T06:30:23.625716+0000","last_clean":"2024-09-14T06:30:23.625716+0000","last_became_active":"2024-09-14T06:29:00.586491+0000","last_became_peered":"2024-09-14T06:29:00.586491+0000","last_unstale":"2024-09-14T06:30:23.625716+0000","last_undegraded":"2024-09-14T06:30:23.625716+0000","last_fullsized":"2024-09-14T06:30:23.625716+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":246,"log_dups_size":0,"ondisk_log_size":246,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T13:20:04.734122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076521300000000005,"stat_sum":{"num_bytes":17046942,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":3732,"num_read_kb":18272,"num_write":303,"num_write_kb":14026,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"2a","length":"1"},{"start":"2c","length":"1"}]},{"pgid":"3.10","version":"98'284","reported_seq":2980,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-14T06:30:23.616866+0000","last_change":"2024-09-14T06:30:22.621397+0000","last_active":"2024-09-14T06:30:23.616866+0000","last_peered":"2024-09-14T06:30:23.616866+0000","last_clean":"2024-09-14T06:30:23.616866+0000","last_became_active":"2024-09-14T06:29:00.579539+0000","last_became_peered":"2024-09-14T06:29:00.579539+0000","last_unstale":"2024-09-14T06:30:23.616866+0000","last_undegraded":"2024-09-14T06:30:23.616866+0000","last_fullsized":"2024-09-14T06:30:23.616866+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":284,"log_dups_size":0,"ondisk_log_size":284,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T07:50:34.368807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00044170899999999998,"stat_sum":{"num_bytes":19497509,"num_objects":30,"num_object_clones":0,"num_object_copies":60,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":30,"num_whiteouts":0,"num_read":2707,"num_read_kb":14268,"num_write":307,"num_write_kb":22669,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"2a","length":"1"},{"start":"2c","length":"1"}]},{"pgid":"3.11","version":"98'375","reported_seq":7677,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-14T06:30:23.625513+0000","last_change":"2024-09-14T06:30:22.614639+0000","last_active":"2024-09-14T06:30:23.625513+0000","last_peered":"2024-09-14T06:30:23.625513+0000","last_clean":"2024-09-14T06:30:23.625513+0000","last_became_active":"2024-09-14T06:29:00.586480+0000","last_became_peered":"2024-09-14T06:29:00.586480+0000","last_unstale":"2024-09-14T06:30:23.625513+0000","last_undegraded":"2024-09-14T06:30:23.625513+0000","last_fullsized":"2024-09-14T06:30:23.625513+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":375,"log_dups_size":0,"ondisk_log_size":375,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T17:18:48.953355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00055691300000000003,"stat_sum":{"num_bytes":72839101,"num_objects":49,"num_object_clones":0,"num_object_copies":98,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":49,"num_whiteouts":0,"num_read":7460,"num_read_kb":81273,"num_write":440,"num_write_kb":34087,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"2a","length":"1"},{"start":"2c","length":"1"}]},{"pgid":"3.12","version":"98'456","reported_seq":5486,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-14T06:30:22.609092+0000","last_change":"2024-09-14T06:30:22.609092+0000","last_active":"2024-09-14T06:30:22.609092+0000","last_peered":"2024-09-14T06:30:22.609092+0000","last_clean":"2024-09-14T06:30:22.609092+0000","last_became_active":"2024-09-14T06:29:00.579826+0000","last_became_peered":"2024-09-14T06:29:00.579826+0000","last_unstale":"2024-09-14T06:30:22.609092+0000","last_undegraded":"2024-09-14T06:30:22.609092+0000","last_fullsized":"2024-09-14T06:30:22.609092+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":456,"log_dups_size":0,"ondisk_log_size":456,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T09:44:15.157725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070490600000000004,"stat_sum":{"num_bytes":33207124,"num_objects":43,"num_object_clones":0,"num_object_copies":86,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":43,"num_whiteouts":0,"num_read":5850,"num_read_kb":42269,"num_write":571,"num_write_kb":26908,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"2a","length":"1"},{"start":"2c","length":"1"}]},{"pgid":"3.13","version":"98'453","reported_seq":5350,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-14T06:30:23.625563+0000","last_change":"2024-09-14T06:30:22.614719+0000","last_active":"2024-09-14T06:30:23.625563+0000","last_peered":"2024-09-14T06:30:23.625563+0000","last_clean":"2024-09-14T06:30:23.625563+0000","last_became_active":"2024-09-14T06:29:00.586520+0000","last_became_peered":"2024-09-14T06:29:00.586520+0000","last_unstale":"2024-09-14T06:30:23.625563+0000","last_undegraded":"2024-09-14T06:30:23.625563+0000","last_fullsized":"2024-09-14T06:30:23.625563+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":453,"log_dups_size":0,"ondisk_log_size":453,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T10:44:48.405672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053809500000000004,"stat_sum":{"num_bytes":53149511,"num_objects":38,"num_object_clones":0,"num_object_copies":76,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":38,"num_whiteouts":0,"num_read":5488,"num_read_kb":45634,"num_write":617,"num_write_kb":31089,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"2a","length":"1"},{"start":"2c","length":"1"}]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":1075714287,"num_objects":1269,"num_object_clones":0,"num_object_copies":2538,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1269,"num_whiteouts":0,"num_read":126074,"num_read_kb":1168797,"num_write":14819,"num_write_kb":809230,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":39,"num_bytes_recovered":1400964,"num_keys_recovered":11,"num_objects_omap":23,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":72732672,"data_stored":710056519,"data_compressed":43652395,"data_compressed_allocated":57528320,"data_compressed_original":698095796,"omap_allocated":0,"internal_metadata":0},"log_size":10999,"ondisk_log_size":10999,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":131072,"data_stored":1180736,"data_compressed":65166,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509463,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55100,"kb_used_data":11432,"kb_used_omap":1,"kb_used_meta":43646,"kb_avail":94316740,"statfs":{"total":96636764160,"available":96580341760,"internally_reserved":0,"allocated":11706368,"data_stored":99810101,"data_compressed":6064561,"data_compressed_allocated":7933952,"data_compressed_original":97174851,"omap_allocated":1590,"internal_metadata":44693962},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509462,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":34868,"kb_used_data":3684,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94336972,"statfs":{"total":96636764160,"available":96601059328,"internally_reserved":0,"allocated":3772416,"data_stored":26601695,"data_compressed":1494839,"data_compressed_allocated":2039808,"data_compressed_original":25607719,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509464,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":50064,"kb_used_data":10552,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94321776,"statfs":{"total":96636764160,"available":96585498624,"internally_reserved":0,"allocated":10805248,"data_stored":89545146,"data_compressed":5484896,"data_compressed_allocated":7340032,"data_compressed_original":87216976,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509463,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":66060,"kb_used_data":16820,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94305780,"statfs":{"total":96636764160,"available":96569118720,"internally_reserved":0,"allocated":17223680,"data_stored":159476697,"data_compressed":10154445,"data_compressed_allocated":13172736,"data_compressed_original":156746019,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509464,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":47928,"kb_used_data":8424,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94323912,"statfs":{"total":96636764160,"available":96587685888,"internally_reserved":0,"allocated":8626176,"data_stored":76286321,"data_compressed":4608422,"data_compressed_allocated":6103040,"data_compressed_original":74710041,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":14,"seq":60129542167,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40072,"kb_used_data":4720,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94331768,"statfs":{"total":96636764160,"available":96595730432,"internally_reserved":0,"allocated":4833280,"data_stored":38471287,"data_compressed":2357654,"data_compressed_allocated":3133440,"data_compressed_original":37542635,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542166,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":61932,"kb_used_data":12696,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94309908,"statfs":{"total":96636764160,"available":96573345792,"internally_reserved":0,"allocated":13000704,"data_stored":113700341,"data_compressed":6847207,"data_compressed_allocated":8962048,"data_compressed_original":110815485,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574871,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":61532,"kb_used_data":12280,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94310308,"statfs":{"total":96636764160,"available":96573755392,"internally_reserved":0,"allocated":12574720,"data_stored":114962236,"data_compressed":6889114,"data_compressed_allocated":9170944,"data_compressed_original":112620094,"omap_allocated":1593,"internal_metadata":50395591},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32583,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32583,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":11743232,"data_stored":112548892,"data_compressed":6814624,"data_compressed_allocated":8916992,"data_compressed_original":110242045,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":7430144,"data_stored":75725221,"data_compressed":4608422,"data_compressed_allocated":6103040,"data_compressed_original":74710041,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":3653632,"data_stored":37916454,"data_compressed":2357654,"data_compressed_allocated":3133440,"data_compressed_original":37542635,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":11423744,"data_stored":114419880,"data_compressed":6889114,"data_compressed_allocated":9170944,"data_compressed_original":112620094,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":15855616,"data_stored":155775103,"data_compressed":9971268,"data_compressed_allocated":12939264,"data_compressed_original":153563067,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":9609216,"data_stored":88984046,"data_compressed":5484896,"data_compressed_allocated":7340032,"data_compressed_original":87216976,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":2580480,"data_stored":26040614,"data_compressed":1494839,"data_compressed_allocated":2039808,"data_compressed_original":25607719,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":10436608,"data_stored":98646309,"data_compressed":6031578,"data_compressed_allocated":7884800,"data_compressed_original":96593219,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-14T06:30:26.504 INFO:tasks.ceph.ceph_manager.ceph:recovered! 2024-09-14T06:30:26.504 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-14T06:30:26.504 INFO:tasks.thrashosds.thrasher:thrash_pg_upmap_items 2024-09-14T06:30:26.504 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump -f json-pretty 2024-09-14T06:30:26.533 INFO:tasks.ceph.osd.4.smithi138.stderr:2024-09-14T06:30:26.532+0000 7f6c653c8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33615) UID: 0 2024-09-14T06:30:26.542 INFO:teuthology.orchestra.run.smithi138.stdout:{ 2024-09-14T06:30:26.542 INFO:teuthology.orchestra.run.smithi138.stdout: "ops": [], 2024-09-14T06:30:26.542 INFO:teuthology.orchestra.run.smithi138.stdout: "num_ops": 0 2024-09-14T06:30:26.542 INFO:teuthology.orchestra.run.smithi138.stdout:} 2024-09-14T06:30:26.553 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-14T06:30:26.638 INFO:tasks.ceph.osd.6.smithi138.stderr:2024-09-14T06:30:26.637+0000 7f8cd8849640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33620) UID: 0 2024-09-14T06:30:26.638 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-14T06:30:26.704 INFO:teuthology.orchestra.run.smithi138.stdout:{ 2024-09-14T06:30:26.704 INFO:teuthology.orchestra.run.smithi138.stdout: "ops": [], 2024-09-14T06:30:26.704 INFO:teuthology.orchestra.run.smithi138.stdout: "complaint_time": 30, 2024-09-14T06:30:26.704 INFO:teuthology.orchestra.run.smithi138.stdout: "num_blocked_ops": 0 2024-09-14T06:30:26.705 INFO:teuthology.orchestra.run.smithi138.stdout:} 2024-09-14T06:30:26.713 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-14T06:30:26.735 INFO:tasks.ceph.osd.0.smithi094.stderr:2024-09-14T06:30:26.734+0000 7f07c9333640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33636) UID: 0 2024-09-14T06:30:26.836 INFO:tasks.ceph.osd.1.smithi094.stderr:2024-09-14T06:30:26.835+0000 7f328e47a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33641) UID: 0 2024-09-14T06:30:26.861 INFO:teuthology.orchestra.run.smithi138.stdout:{ 2024-09-14T06:30:26.861 INFO:teuthology.orchestra.run.smithi138.stdout: "size": 20, 2024-09-14T06:30:26.861 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 600, 2024-09-14T06:30:26.861 INFO:teuthology.orchestra.run.smithi138.stdout: "ops": [ 2024-09-14T06:30:26.861 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:26.861 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:19328 3.3 3:c34e5e51:::rbd_data.112d92027590.0000000000000024:f [read 1926144~1024 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:30:26.861 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:27.059768+0000", 2024-09-14T06:30:26.862 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 59.799387418000002, 2024-09-14T06:30:26.862 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046369147999999999, 2024-09-14T06:30:26.862 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:30:26.862 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:30:26.862 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:30:26.862 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:30:26.862 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:30:26.862 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:26.862 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 19328 2024-09-14T06:30:26.862 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:26.862 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:30:26.862 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:26.862 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:30:26.862 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.059768+0000", 2024-09-14T06:30:26.863 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:26.863 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:26.863 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:26.863 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:30:26.863 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.059768+0000", 2024-09-14T06:30:26.863 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:26.863 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:26.863 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:26.863 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:30:26.863 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.059874+0000", 2024-09-14T06:30:26.863 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00010551400000000001 2024-09-14T06:30:26.863 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:26.863 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:26.863 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:30:26.864 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.059877+0000", 2024-09-14T06:30:26.864 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.8279999999999999e-06 2024-09-14T06:30:26.864 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:26.864 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:26.864 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:30:26.864 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.059884+0000", 2024-09-14T06:30:26.864 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.6389999999999992e-06 2024-09-14T06:30:26.864 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:26.864 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:26.864 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:30:26.864 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.059890+0000", 2024-09-14T06:30:26.865 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.49e-06 2024-09-14T06:30:26.865 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:26.865 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:26.865 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:30:26.865 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.105804+0000", 2024-09-14T06:30:26.865 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.045914242000000001 2024-09-14T06:30:26.866 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:26.866 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:26.866 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:30:26.866 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.105886+0000", 2024-09-14T06:30:26.866 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.2305999999999994e-05 2024-09-14T06:30:26.866 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:26.867 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:26.867 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:30:26.867 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.106138+0000", 2024-09-14T06:30:26.867 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.000251129 2024-09-14T06:30:26.867 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:26.867 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:30:26.868 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:26.868 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:26.868 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:26.868 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:19329 3.3 3:c34e5e51:::rbd_data.112d92027590.0000000000000024:f [read 1936384~1024 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:30:26.868 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:27.059899+0000", 2024-09-14T06:30:26.868 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 59.799256894999999, 2024-09-14T06:30:26.868 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046572122000000001, 2024-09-14T06:30:26.868 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:30:26.868 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:30:26.868 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:30:26.868 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:30:26.869 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:30:26.869 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:26.869 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 19329 2024-09-14T06:30:26.869 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:26.869 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:30:26.869 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:26.869 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:30:26.870 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.059899+0000", 2024-09-14T06:30:26.870 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:26.870 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:26.870 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:26.870 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:30:26.870 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.059899+0000", 2024-09-14T06:30:26.871 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:26.871 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:26.871 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:26.871 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:30:26.871 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062501+0000", 2024-09-14T06:30:26.871 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.0026016419999999999 2024-09-14T06:30:26.872 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:26.872 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:26.872 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:30:26.872 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062514+0000", 2024-09-14T06:30:26.872 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.2947e-05 2024-09-14T06:30:26.872 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:26.872 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:26.873 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:30:26.873 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062523+0000", 2024-09-14T06:30:26.873 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.5519999999999993e-06 2024-09-14T06:30:26.873 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:26.873 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:26.873 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:30:26.874 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062534+0000", 2024-09-14T06:30:26.874 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.1416e-05 2024-09-14T06:30:26.874 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:26.874 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:26.874 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:30:26.874 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.106179+0000", 2024-09-14T06:30:26.874 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.043644049999999997 2024-09-14T06:30:26.875 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:26.875 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:26.875 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:30:26.875 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.106262+0000", 2024-09-14T06:30:26.875 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.3178999999999998e-05 2024-09-14T06:30:26.875 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:26.876 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:26.876 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:30:26.876 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.106471+0000", 2024-09-14T06:30:26.876 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00020933599999999999 2024-09-14T06:30:26.876 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:26.876 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:30:26.876 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:26.876 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:26.877 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:26.877 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:19351 3.3 3:c34e5e51:::rbd_data.112d92027590.0000000000000024:f [read 1159168~1024 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:30:26.877 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:27.062766+0000", 2024-09-14T06:30:26.877 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 59.796389670000003, 2024-09-14T06:30:26.877 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046099454999999998, 2024-09-14T06:30:26.878 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:30:26.878 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:30:26.878 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:30:26.878 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:30:26.878 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:30:26.878 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:26.878 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 19351 2024-09-14T06:30:26.879 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:26.879 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:30:26.879 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:26.879 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:30:26.879 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062766+0000", 2024-09-14T06:30:26.880 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:26.880 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:26.880 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:26.880 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:30:26.880 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062766+0000", 2024-09-14T06:30:26.880 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:26.881 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:26.881 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:26.881 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:30:26.881 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062766+0000", 2024-09-14T06:30:26.881 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.6e-07 2024-09-14T06:30:26.881 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:26.881 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:26.882 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:30:26.882 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062767+0000", 2024-09-14T06:30:26.882 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.0070000000000001e-06 2024-09-14T06:30:26.882 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:26.882 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:26.882 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:30:26.882 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062785+0000", 2024-09-14T06:30:26.883 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.7884999999999999e-05 2024-09-14T06:30:26.883 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:26.883 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:26.883 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:30:26.883 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062790+0000", 2024-09-14T06:30:26.883 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.0900000000000004e-06 2024-09-14T06:30:26.883 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:26.884 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:26.884 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:30:26.884 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.108735+0000", 2024-09-14T06:30:26.884 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.045944411999999997 2024-09-14T06:30:26.884 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:26.884 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:26.884 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:30:26.885 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.108809+0000", 2024-09-14T06:30:26.885 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.4157000000000002e-05 2024-09-14T06:30:26.885 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:26.885 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:26.885 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:30:26.886 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.108866+0000", 2024-09-14T06:30:26.886 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.6644000000000002e-05 2024-09-14T06:30:26.886 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:26.886 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:30:26.886 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:26.886 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:26.887 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:26.887 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:19352 3.3 3:c34e5e51:::rbd_data.112d92027590.0000000000000024:f [read 1169408~1024 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:30:26.887 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:27.062798+0000", 2024-09-14T06:30:26.887 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 59.796358259000002, 2024-09-14T06:30:26.887 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046255900000000003, 2024-09-14T06:30:26.887 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:30:26.888 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:30:26.888 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:30:26.888 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:30:26.888 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:30:26.888 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:26.888 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 19352 2024-09-14T06:30:26.889 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:26.889 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:30:26.889 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:26.889 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:30:26.889 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062798+0000", 2024-09-14T06:30:26.889 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:26.889 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:26.890 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:26.890 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:30:26.890 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062798+0000", 2024-09-14T06:30:26.890 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:26.890 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:26.890 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:26.891 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:30:26.891 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062798+0000", 2024-09-14T06:30:26.891 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.37e-07 2024-09-14T06:30:26.891 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:26.891 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:26.891 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:30:26.892 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062800+0000", 2024-09-14T06:30:26.892 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.0269999999999998e-06 2024-09-14T06:30:26.892 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:26.892 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:26.892 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:30:26.893 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062806+0000", 2024-09-14T06:30:26.893 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.6690000000000003e-06 2024-09-14T06:30:26.893 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:26.893 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:26.893 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:30:26.893 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062810+0000", 2024-09-14T06:30:26.893 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.3449999999999999e-06 2024-09-14T06:30:26.894 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:26.894 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:26.894 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:30:26.894 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.108892+0000", 2024-09-14T06:30:26.894 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046081715000000002 2024-09-14T06:30:26.894 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:26.895 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:26.895 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:30:26.895 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.108940+0000", 2024-09-14T06:30:26.895 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.7815999999999998e-05 2024-09-14T06:30:26.895 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:26.895 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:26.896 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:30:26.896 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.109053+0000", 2024-09-14T06:30:26.896 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.000113791 2024-09-14T06:30:26.896 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:26.896 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:30:26.896 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:26.897 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:26.897 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:26.897 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:19353 3.3 3:c34e5e51:::rbd_data.112d92027590.0000000000000024:f [read 1179648~1024 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:30:26.897 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:27.062816+0000", 2024-09-14T06:30:26.897 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 59.796339740000001, 2024-09-14T06:30:26.897 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046481256999999998, 2024-09-14T06:30:26.897 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:30:26.898 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:30:26.898 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:30:26.898 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:30:26.898 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:30:26.898 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:26.898 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 19353 2024-09-14T06:30:26.899 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:26.899 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:30:26.899 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:26.899 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:30:26.899 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062816+0000", 2024-09-14T06:30:26.899 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:26.900 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:26.900 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:26.900 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:30:26.900 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062816+0000", 2024-09-14T06:30:26.900 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:26.900 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:26.900 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:26.901 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:30:26.901 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062816+0000", 2024-09-14T06:30:26.901 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.2899999999999999e-07 2024-09-14T06:30:26.901 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:26.901 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:26.901 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:30:26.901 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062818+0000", 2024-09-14T06:30:26.901 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.699e-06 2024-09-14T06:30:26.902 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:26.902 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:26.902 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:30:26.902 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062825+0000", 2024-09-14T06:30:26.902 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.878e-06 2024-09-14T06:30:26.902 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:26.902 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:26.903 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:30:26.903 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062830+0000", 2024-09-14T06:30:26.903 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.5410000000000002e-06 2024-09-14T06:30:26.903 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:26.903 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:26.903 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:30:26.904 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.109096+0000", 2024-09-14T06:30:26.904 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046266517 2024-09-14T06:30:26.904 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:26.904 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:26.904 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:30:26.904 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.109190+0000", 2024-09-14T06:30:26.905 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.4000999999999997e-05 2024-09-14T06:30:26.905 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:26.905 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:26.905 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:30:26.905 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.109297+0000", 2024-09-14T06:30:26.906 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.000107292 2024-09-14T06:30:26.906 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:26.906 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:30:26.906 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:26.906 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:26.906 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:26.906 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:19354 3.3 3:c34e5e51:::rbd_data.112d92027590.0000000000000024:f [read 1189888~1024 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:30:26.906 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:27.062842+0000", 2024-09-14T06:30:26.906 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 59.796314285000001, 2024-09-14T06:30:26.907 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046594257, 2024-09-14T06:30:26.907 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:30:26.907 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:30:26.907 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:30:26.907 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:30:26.907 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:30:26.907 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:26.907 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 19354 2024-09-14T06:30:26.907 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:26.907 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:30:26.907 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:26.908 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:30:26.908 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062842+0000", 2024-09-14T06:30:26.908 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:26.908 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:26.908 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:26.908 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:30:26.908 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062842+0000", 2024-09-14T06:30:26.908 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:26.909 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:26.909 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:26.909 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:30:26.909 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062842+0000", 2024-09-14T06:30:26.909 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.7399999999999999e-07 2024-09-14T06:30:26.909 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:26.909 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:26.910 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:30:26.910 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062859+0000", 2024-09-14T06:30:26.910 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.7010000000000001e-05 2024-09-14T06:30:26.910 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:26.910 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:26.910 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:30:26.911 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062866+0000", 2024-09-14T06:30:26.911 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.9990000000000002e-06 2024-09-14T06:30:26.911 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:26.911 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:26.911 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:30:26.911 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062873+0000", 2024-09-14T06:30:26.911 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.8569999999999997e-06 2024-09-14T06:30:26.912 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:26.912 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:26.912 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:30:26.912 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.109335+0000", 2024-09-14T06:30:26.912 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046461943999999998 2024-09-14T06:30:26.912 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:26.912 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:26.913 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:30:26.913 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.109377+0000", 2024-09-14T06:30:26.913 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.2558999999999997e-05 2024-09-14T06:30:26.913 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:26.913 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:26.913 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:30:26.913 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.109436+0000", 2024-09-14T06:30:26.914 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.8514000000000003e-05 2024-09-14T06:30:26.914 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:26.914 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:30:26.914 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:26.914 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:26.914 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:26.914 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:19355 3.3 3:c34e5e51:::rbd_data.112d92027590.0000000000000024:f [read 1200128~1024 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:30:26.915 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:27.062884+0000", 2024-09-14T06:30:26.915 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 59.796271611999998, 2024-09-14T06:30:26.915 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046681650999999998, 2024-09-14T06:30:26.915 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:30:26.915 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:30:26.915 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:30:26.916 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:30:26.916 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:30:26.916 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:26.916 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 19355 2024-09-14T06:30:26.916 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:26.916 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:30:26.917 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:26.917 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:30:26.917 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062884+0000", 2024-09-14T06:30:26.917 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:26.917 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:26.918 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:26.918 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:30:26.918 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062884+0000", 2024-09-14T06:30:26.918 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:26.918 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:26.918 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:26.919 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:30:26.919 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062885+0000", 2024-09-14T06:30:26.919 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.8000000000000001e-07 2024-09-14T06:30:26.919 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:26.919 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:26.919 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:30:26.920 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062886+0000", 2024-09-14T06:30:26.920 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.8920000000000001e-06 2024-09-14T06:30:26.920 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:26.920 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:26.920 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:30:26.920 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062892+0000", 2024-09-14T06:30:26.920 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.84e-06 2024-09-14T06:30:26.920 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:26.920 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:26.920 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:30:26.921 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062899+0000", 2024-09-14T06:30:26.921 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.5339999999999996e-06 2024-09-14T06:30:26.921 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:26.921 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:26.921 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:30:26.921 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.109466+0000", 2024-09-14T06:30:26.921 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046566919999999998 2024-09-14T06:30:26.921 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:26.921 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:26.921 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:30:26.921 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.109510+0000", 2024-09-14T06:30:26.921 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.4153000000000002e-05 2024-09-14T06:30:26.921 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:26.922 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:26.922 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:30:26.922 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.109566+0000", 2024-09-14T06:30:26.922 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.5931999999999999e-05 2024-09-14T06:30:26.922 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:26.922 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:30:26.922 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:26.922 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:26.922 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:26.922 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:19356 3.3 3:c34e5e51:::rbd_data.112d92027590.0000000000000024:f [read 1210368~1024 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:30:26.922 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:27.062908+0000", 2024-09-14T06:30:26.922 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 59.796248175000002, 2024-09-14T06:30:26.922 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046807498000000003, 2024-09-14T06:30:26.922 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:30:26.922 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:30:26.923 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:30:26.923 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:30:26.923 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:30:26.923 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:26.923 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 19356 2024-09-14T06:30:26.923 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:26.923 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:30:26.923 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:26.923 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:30:26.923 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062908+0000", 2024-09-14T06:30:26.923 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:26.923 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:26.923 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:26.923 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:30:26.924 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062908+0000", 2024-09-14T06:30:26.924 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:26.924 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:26.924 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:26.924 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:30:26.924 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062908+0000", 2024-09-14T06:30:26.924 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.5699999999999998e-07 2024-09-14T06:30:26.924 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:26.924 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:26.924 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:30:26.924 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062910+0000", 2024-09-14T06:30:26.925 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.7710000000000001e-06 2024-09-14T06:30:26.925 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:26.925 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:26.925 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:30:26.925 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062920+0000", 2024-09-14T06:30:26.925 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.0654e-05 2024-09-14T06:30:26.925 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:26.925 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:26.925 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:30:26.925 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062931+0000", 2024-09-14T06:30:26.925 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.075e-05 2024-09-14T06:30:26.925 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:26.925 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:26.925 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:30:26.926 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.109595+0000", 2024-09-14T06:30:26.926 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046663713000000002 2024-09-14T06:30:26.926 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:26.926 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:26.926 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:30:26.926 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.109632+0000", 2024-09-14T06:30:26.926 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.7505000000000001e-05 2024-09-14T06:30:26.926 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:26.926 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:26.926 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:30:26.926 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.109715+0000", 2024-09-14T06:30:26.926 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.2748000000000001e-05 2024-09-14T06:30:26.926 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:26.926 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:30:26.927 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:26.927 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:26.927 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:26.927 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:19357 3.3 3:c34e5e51:::rbd_data.112d92027590.0000000000000024:f [read 1220608~1024 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:30:26.927 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:27.062939+0000", 2024-09-14T06:30:26.927 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 59.796216727000001, 2024-09-14T06:30:26.927 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046920508, 2024-09-14T06:30:26.927 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:30:26.927 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:30:26.927 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:30:26.927 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:30:26.927 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:30:26.927 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:26.928 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 19357 2024-09-14T06:30:26.928 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:26.928 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:30:26.928 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:26.928 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:30:26.928 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062939+0000", 2024-09-14T06:30:26.928 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:26.928 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:26.928 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:26.928 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:30:26.928 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062939+0000", 2024-09-14T06:30:26.928 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:26.928 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:26.928 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:26.928 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:30:26.929 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062940+0000", 2024-09-14T06:30:26.929 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.32e-07 2024-09-14T06:30:26.929 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:26.929 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:26.929 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:30:26.929 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062942+0000", 2024-09-14T06:30:26.929 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.1919999999999999e-06 2024-09-14T06:30:26.929 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:26.929 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:26.929 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:30:26.929 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062948+0000", 2024-09-14T06:30:26.929 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.1149999999999999e-06 2024-09-14T06:30:26.929 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:26.929 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:26.930 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:30:26.930 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062960+0000", 2024-09-14T06:30:26.930 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.1895e-05 2024-09-14T06:30:26.930 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:26.930 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:26.930 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:30:26.930 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.109704+0000", 2024-09-14T06:30:26.930 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046744584999999998 2024-09-14T06:30:26.930 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:26.930 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:26.930 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:30:26.931 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.109777+0000", 2024-09-14T06:30:26.931 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.2282999999999999e-05 2024-09-14T06:30:26.931 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:26.931 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:26.931 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:30:26.931 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.109860+0000", 2024-09-14T06:30:26.931 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.3005999999999998e-05 2024-09-14T06:30:26.932 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:26.932 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:30:26.932 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:26.932 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:26.932 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:26.932 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:19358 3.3 3:c34e5e51:::rbd_data.112d92027590.0000000000000024:f [read 1230848~1024 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:30:26.933 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:27.062967+0000", 2024-09-14T06:30:26.933 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 59.796189097000003, 2024-09-14T06:30:26.933 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047144471, 2024-09-14T06:30:26.933 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:30:26.933 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:30:26.934 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:30:26.934 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:30:26.934 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:30:26.934 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:26.934 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 19358 2024-09-14T06:30:26.935 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:26.935 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:30:26.935 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:26.935 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:30:26.935 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062967+0000", 2024-09-14T06:30:26.935 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:26.935 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:26.936 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:26.936 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:30:26.936 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062967+0000", 2024-09-14T06:30:26.936 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:26.936 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:26.936 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:26.937 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:30:26.937 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062967+0000", 2024-09-14T06:30:26.937 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.89e-07 2024-09-14T06:30:26.937 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:26.937 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:26.937 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:30:26.937 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062969+0000", 2024-09-14T06:30:26.937 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.9929999999999998e-06 2024-09-14T06:30:26.937 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:26.938 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:26.938 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:30:26.938 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062975+0000", 2024-09-14T06:30:26.938 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.9030000000000001e-06 2024-09-14T06:30:26.938 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:26.938 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:26.938 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:30:26.938 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062979+0000", 2024-09-14T06:30:26.938 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.2250000000000002e-06 2024-09-14T06:30:26.938 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:26.938 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:26.938 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:30:26.938 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.109934+0000", 2024-09-14T06:30:26.938 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046954410000000002 2024-09-14T06:30:26.939 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:26.939 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:26.939 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:30:26.939 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.110007+0000", 2024-09-14T06:30:26.939 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.3829000000000001e-05 2024-09-14T06:30:26.939 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:26.939 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:26.939 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:30:26.939 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.110111+0000", 2024-09-14T06:30:26.939 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00010372199999999999 2024-09-14T06:30:26.939 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:26.939 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:30:26.939 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:26.939 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:26.940 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:26.940 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:19359 3.3 3:c34e5e51:::rbd_data.112d92027590.0000000000000024:f [read 1241088~1024 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:30:26.940 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:27.062987+0000", 2024-09-14T06:30:26.940 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 59.796169007000003, 2024-09-14T06:30:26.940 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047297182, 2024-09-14T06:30:26.940 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:30:26.940 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:30:26.940 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:30:26.940 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:30:26.940 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:30:26.940 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:26.940 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 19359 2024-09-14T06:30:26.940 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:26.940 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:30:26.941 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:26.941 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:30:26.941 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062987+0000", 2024-09-14T06:30:26.941 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:26.941 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:26.941 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:26.941 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:30:26.941 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062987+0000", 2024-09-14T06:30:26.941 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:26.941 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:26.941 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:26.941 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:30:26.941 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062987+0000", 2024-09-14T06:30:26.941 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.2500000000000001e-07 2024-09-14T06:30:26.942 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:26.942 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:26.942 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:30:26.942 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062989+0000", 2024-09-14T06:30:26.942 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.6080000000000001e-06 2024-09-14T06:30:26.942 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:26.942 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:26.942 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:30:26.942 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062994+0000", 2024-09-14T06:30:26.942 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.5520000000000004e-06 2024-09-14T06:30:26.942 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:26.942 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:26.942 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:30:26.942 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062999+0000", 2024-09-14T06:30:26.943 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.3930000000000001e-06 2024-09-14T06:30:26.943 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:26.943 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:26.943 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:30:26.943 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.110142+0000", 2024-09-14T06:30:26.943 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047143154 2024-09-14T06:30:26.943 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:26.943 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:26.943 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:30:26.943 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.110219+0000", 2024-09-14T06:30:26.943 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.6737999999999996e-05 2024-09-14T06:30:26.943 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:26.943 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:26.944 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:30:26.944 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.110284+0000", 2024-09-14T06:30:26.944 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.5412000000000003e-05 2024-09-14T06:30:26.944 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:26.944 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:30:26.944 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:26.944 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:26.944 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:26.944 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:19360 3.3 3:c34e5e51:::rbd_data.112d92027590.0000000000000024:f [read 1251328~1024 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:30:26.944 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:27.063017+0000", 2024-09-14T06:30:26.944 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 59.796138546000002, 2024-09-14T06:30:26.944 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047453823999999999, 2024-09-14T06:30:26.944 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:30:26.944 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:30:26.945 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:30:26.945 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:30:26.945 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:30:26.945 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:26.945 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 19360 2024-09-14T06:30:26.945 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:26.945 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:30:26.945 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:26.945 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:30:26.945 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.063017+0000", 2024-09-14T06:30:26.945 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:26.945 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:26.945 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:26.946 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:30:26.946 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.063017+0000", 2024-09-14T06:30:26.946 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:26.946 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:26.946 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:26.946 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:30:26.946 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.064977+0000", 2024-09-14T06:30:26.946 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.0019597080000000001 2024-09-14T06:30:26.946 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:26.946 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:26.946 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:30:26.946 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.064987+0000", 2024-09-14T06:30:26.946 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.9620000000000005e-06 2024-09-14T06:30:26.946 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:26.947 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:26.947 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:30:26.947 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065000+0000", 2024-09-14T06:30:26.947 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.3173000000000001e-05 2024-09-14T06:30:26.947 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:26.947 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:26.947 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:30:26.947 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065005+0000", 2024-09-14T06:30:26.947 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.9960000000000001e-06 2024-09-14T06:30:26.947 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:26.947 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:26.947 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:30:26.947 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.110314+0000", 2024-09-14T06:30:26.947 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.04530899 2024-09-14T06:30:26.947 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:26.948 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:26.948 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:30:26.948 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.110398+0000", 2024-09-14T06:30:26.948 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.3539000000000001e-05 2024-09-14T06:30:26.948 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:26.948 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:26.948 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:30:26.948 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.110471+0000", 2024-09-14T06:30:26.948 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.3455999999999996e-05 2024-09-14T06:30:26.948 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:26.948 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:30:26.948 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:26.948 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:26.949 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:26.949 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:19365 3.3 3:c34e5e51:::rbd_data.112d92027590.0000000000000024:f [read 1302528~1024 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:30:26.949 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:27.065058+0000", 2024-09-14T06:30:26.949 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 59.794097876000002, 2024-09-14T06:30:26.949 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046105526000000001, 2024-09-14T06:30:26.949 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:30:26.949 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:30:26.949 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:30:26.949 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:30:26.949 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:30:26.949 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:26.949 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 19365 2024-09-14T06:30:26.949 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:26.949 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:30:26.950 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:26.950 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:30:26.950 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065058+0000", 2024-09-14T06:30:26.950 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:26.950 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:26.950 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:26.950 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:30:26.950 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065058+0000", 2024-09-14T06:30:26.950 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:26.950 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:26.950 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:26.950 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:30:26.950 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065058+0000", 2024-09-14T06:30:26.950 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.1899999999999999e-07 2024-09-14T06:30:26.951 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:26.951 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:26.951 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:30:26.951 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065059+0000", 2024-09-14T06:30:26.951 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.1299999999999998e-07 2024-09-14T06:30:26.951 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:26.951 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:26.951 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:30:26.951 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065063+0000", 2024-09-14T06:30:26.951 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.1099999999999996e-06 2024-09-14T06:30:26.951 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:26.951 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:26.951 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:30:26.951 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065070+0000", 2024-09-14T06:30:26.952 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.973e-06 2024-09-14T06:30:26.952 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:26.952 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:26.952 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:30:26.952 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.111014+0000", 2024-09-14T06:30:26.952 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.045943373000000003 2024-09-14T06:30:26.952 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:26.952 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:26.952 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:30:26.952 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.111073+0000", 2024-09-14T06:30:26.952 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.8987999999999998e-05 2024-09-14T06:30:26.952 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:26.952 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:26.952 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:30:26.953 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.111163+0000", 2024-09-14T06:30:26.953 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.0950000000000002e-05 2024-09-14T06:30:26.953 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:26.953 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:30:26.953 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:26.953 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:26.953 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:26.953 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:19366 3.3 3:c34e5e51:::rbd_data.112d92027590.0000000000000024:f [read 1312768~1024 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:30:26.953 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:27.065074+0000", 2024-09-14T06:30:26.953 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 59.794082025999998, 2024-09-14T06:30:26.953 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046289858000000003, 2024-09-14T06:30:26.953 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:30:26.953 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:30:26.953 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:30:26.954 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:30:26.954 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:30:26.954 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:26.954 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 19366 2024-09-14T06:30:26.954 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:26.954 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:30:26.954 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:26.954 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:30:26.954 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065074+0000", 2024-09-14T06:30:26.954 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:26.954 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:26.954 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:26.954 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:30:26.954 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065074+0000", 2024-09-14T06:30:26.955 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:26.955 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:26.955 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:26.955 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:30:26.955 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065074+0000", 2024-09-14T06:30:26.955 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.3900000000000001e-07 2024-09-14T06:30:26.955 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:26.955 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:26.955 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:30:26.955 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065078+0000", 2024-09-14T06:30:26.955 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.3229999999999999e-06 2024-09-14T06:30:26.955 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:26.955 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:26.955 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:30:26.955 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065082+0000", 2024-09-14T06:30:26.956 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.8299999999999998e-06 2024-09-14T06:30:26.956 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:26.956 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:26.956 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:30:26.956 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065085+0000", 2024-09-14T06:30:26.956 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.8660000000000002e-06 2024-09-14T06:30:26.956 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:26.956 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:26.956 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:30:26.956 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.111270+0000", 2024-09-14T06:30:26.956 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046185035999999999 2024-09-14T06:30:26.956 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:26.956 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:26.956 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:30:26.957 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.111311+0000", 2024-09-14T06:30:26.957 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.0935e-05 2024-09-14T06:30:26.957 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:26.957 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:26.957 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:30:26.957 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.111364+0000", 2024-09-14T06:30:26.957 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.2629e-05 2024-09-14T06:30:26.960 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:26.960 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:30:26.960 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:26.960 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:26.960 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:26.960 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:19367 3.3 3:c34e5e51:::rbd_data.112d92027590.0000000000000024:f [read 1323008~1024 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:30:26.960 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:27.065115+0000", 2024-09-14T06:30:26.960 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 59.794040465000002, 2024-09-14T06:30:26.960 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046407003000000002, 2024-09-14T06:30:26.960 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:30:26.960 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:30:26.961 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:30:26.961 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:30:26.961 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:30:26.961 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:26.961 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 19367 2024-09-14T06:30:26.961 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:26.961 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:30:26.961 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:26.961 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:30:26.961 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065115+0000", 2024-09-14T06:30:26.961 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:26.961 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:26.961 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:26.961 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:30:26.962 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065115+0000", 2024-09-14T06:30:26.962 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:26.962 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:26.962 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:26.962 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:30:26.962 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065116+0000", 2024-09-14T06:30:26.962 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.4999999999999998e-07 2024-09-14T06:30:26.962 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:26.962 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:26.962 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:30:26.962 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065117+0000", 2024-09-14T06:30:26.962 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.189e-06 2024-09-14T06:30:26.962 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:26.962 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:26.962 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:30:26.963 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065122+0000", 2024-09-14T06:30:26.963 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.3709999999999999e-06 2024-09-14T06:30:26.963 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:26.963 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:26.963 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:30:26.963 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065125+0000", 2024-09-14T06:30:26.963 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.8940000000000002e-06 2024-09-14T06:30:26.963 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:26.963 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:26.963 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:30:26.963 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.111402+0000", 2024-09-14T06:30:26.963 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046276645999999998 2024-09-14T06:30:26.963 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:26.963 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:26.964 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:30:26.964 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.111445+0000", 2024-09-14T06:30:26.964 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.3288000000000003e-05 2024-09-14T06:30:26.964 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:26.964 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:26.964 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:30:26.964 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.111522+0000", 2024-09-14T06:30:26.964 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.7164999999999997e-05 2024-09-14T06:30:26.964 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:26.964 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:30:26.964 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:26.964 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:26.964 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:26.964 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:19368 3.3 3:c34e5e51:::rbd_data.112d92027590.0000000000000024:f [read 1333248~1024 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:30:26.965 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:27.065130+0000", 2024-09-14T06:30:26.965 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 59.794026174999999, 2024-09-14T06:30:26.965 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046507275000000001, 2024-09-14T06:30:26.965 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:30:26.965 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:30:26.965 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:30:26.965 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:30:26.965 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:30:26.965 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:26.965 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 19368 2024-09-14T06:30:26.965 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:26.965 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:30:26.965 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:26.965 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:30:26.965 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065130+0000", 2024-09-14T06:30:26.966 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:26.966 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:26.966 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:26.966 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:30:26.966 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065130+0000", 2024-09-14T06:30:26.966 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:26.966 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:26.966 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:26.966 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:30:26.966 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065130+0000", 2024-09-14T06:30:26.966 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.0800000000000001e-07 2024-09-14T06:30:26.966 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:26.966 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:26.967 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:30:26.967 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065131+0000", 2024-09-14T06:30:26.967 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.8700000000000004e-07 2024-09-14T06:30:26.967 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:26.967 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:26.967 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:30:26.967 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065134+0000", 2024-09-14T06:30:26.967 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.1039999999999998e-06 2024-09-14T06:30:26.967 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:26.967 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:26.967 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:30:26.967 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065136+0000", 2024-09-14T06:30:26.967 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.3949999999999999e-06 2024-09-14T06:30:26.967 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:26.967 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:26.968 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:30:26.968 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.111541+0000", 2024-09-14T06:30:26.968 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046404206000000003 2024-09-14T06:30:26.968 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:26.968 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:26.968 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:30:26.968 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.111582+0000", 2024-09-14T06:30:26.968 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.1051999999999999e-05 2024-09-14T06:30:26.968 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:26.968 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:26.968 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:30:26.968 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.111637+0000", 2024-09-14T06:30:26.968 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.5322999999999999e-05 2024-09-14T06:30:26.968 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:26.969 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:30:26.969 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:26.969 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:26.969 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:26.969 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:19369 3.3 3:c34e5e51:::rbd_data.112d92027590.0000000000000024:f [read 1343488~1024 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:30:26.969 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:27.065139+0000", 2024-09-14T06:30:26.969 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 59.794016358, 2024-09-14T06:30:26.969 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046682169000000003, 2024-09-14T06:30:26.969 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:30:26.969 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:30:26.969 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:30:26.969 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:30:26.969 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:30:26.970 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:26.970 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 19369 2024-09-14T06:30:26.970 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:26.970 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:30:26.970 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:26.970 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:30:26.970 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065139+0000", 2024-09-14T06:30:26.970 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:26.970 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:26.970 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:26.970 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:30:26.970 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065139+0000", 2024-09-14T06:30:26.970 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:26.971 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:26.971 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:26.971 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:30:26.971 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065140+0000", 2024-09-14T06:30:26.971 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.8299999999999998e-07 2024-09-14T06:30:26.971 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:26.971 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:26.971 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:30:26.971 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065141+0000", 2024-09-14T06:30:26.971 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.3900000000000003e-07 2024-09-14T06:30:26.971 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:26.971 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:26.971 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:30:26.971 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065144+0000", 2024-09-14T06:30:26.971 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.7409999999999998e-06 2024-09-14T06:30:26.972 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:26.972 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:26.972 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:30:26.972 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065147+0000", 2024-09-14T06:30:26.972 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.4150000000000002e-06 2024-09-14T06:30:26.972 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:26.972 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:26.972 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:30:26.972 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.111673+0000", 2024-09-14T06:30:26.972 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046526004000000003 2024-09-14T06:30:26.972 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:26.972 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:26.972 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:30:26.973 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.111749+0000", 2024-09-14T06:30:26.973 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.5741999999999994e-05 2024-09-14T06:30:26.973 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:26.973 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:26.973 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:30:26.973 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.111822+0000", 2024-09-14T06:30:26.973 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.3045000000000003e-05 2024-09-14T06:30:26.973 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:26.973 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:30:26.973 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:26.973 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:26.973 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:26.973 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:19370 3.3 3:c34e5e51:::rbd_data.112d92027590.0000000000000024:f [read 1353728~1024 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:30:26.973 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:27.065150+0000", 2024-09-14T06:30:26.974 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 59.794005628999997, 2024-09-14T06:30:26.974 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046897556999999999, 2024-09-14T06:30:26.974 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:30:26.974 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:30:26.974 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:30:26.974 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:30:26.974 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:30:26.974 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:26.974 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 19370 2024-09-14T06:30:26.974 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:26.974 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:30:26.974 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:26.974 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:30:26.974 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065150+0000", 2024-09-14T06:30:26.975 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:26.975 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:26.975 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:26.975 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:30:26.975 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065150+0000", 2024-09-14T06:30:26.975 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:26.975 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:26.975 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:26.975 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:30:26.975 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065150+0000", 2024-09-14T06:30:26.975 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.0200000000000001e-07 2024-09-14T06:30:26.975 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:26.975 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:26.975 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:30:26.976 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065151+0000", 2024-09-14T06:30:26.976 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.1100000000000004e-07 2024-09-14T06:30:26.976 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:26.976 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:26.976 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:30:26.976 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065154+0000", 2024-09-14T06:30:26.976 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.9639999999999999e-06 2024-09-14T06:30:26.976 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:26.976 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:26.976 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:30:26.976 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065157+0000", 2024-09-14T06:30:26.976 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.4260000000000002e-06 2024-09-14T06:30:26.976 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:26.976 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:26.977 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:30:26.977 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.111861+0000", 2024-09-14T06:30:26.977 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046704268 2024-09-14T06:30:26.977 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:26.977 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:26.977 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:30:26.977 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.111939+0000", 2024-09-14T06:30:26.977 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.8231000000000004e-05 2024-09-14T06:30:26.977 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:26.977 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:26.977 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:30:26.977 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.112048+0000", 2024-09-14T06:30:26.977 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00010855500000000001 2024-09-14T06:30:26.977 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:26.978 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:30:26.978 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:26.978 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:26.978 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:26.978 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:19371 3.3 3:c34e5e51:::rbd_data.112d92027590.0000000000000024:f [read 1363968~1024 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:30:26.978 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:27.065161+0000", 2024-09-14T06:30:26.978 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 59.793994978999997, 2024-09-14T06:30:26.978 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047021944000000003, 2024-09-14T06:30:26.978 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:30:26.978 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:30:26.978 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:30:26.978 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:30:26.978 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:30:26.978 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:26.979 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 19371 2024-09-14T06:30:26.979 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:26.979 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:30:26.979 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:26.979 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:30:26.979 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065161+0000", 2024-09-14T06:30:26.979 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:26.979 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:26.979 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:26.979 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:30:26.979 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065161+0000", 2024-09-14T06:30:26.979 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:26.979 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:26.979 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:26.979 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:30:26.979 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065161+0000", 2024-09-14T06:30:26.980 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.0200000000000001e-07 2024-09-14T06:30:26.980 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:26.980 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:26.980 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:30:26.980 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065166+0000", 2024-09-14T06:30:26.980 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.9869999999999999e-06 2024-09-14T06:30:26.980 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:26.980 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:26.980 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:30:26.980 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065169+0000", 2024-09-14T06:30:26.980 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.3799999999999998e-06 2024-09-14T06:30:26.980 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:26.980 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:26.980 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:30:26.980 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065172+0000", 2024-09-14T06:30:26.981 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.4830000000000002e-06 2024-09-14T06:30:26.981 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:26.981 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:26.981 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:30:26.981 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.112071+0000", 2024-09-14T06:30:26.981 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046899529000000002 2024-09-14T06:30:26.981 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:26.981 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:26.981 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:30:26.981 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.112128+0000", 2024-09-14T06:30:26.981 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.6341999999999997e-05 2024-09-14T06:30:26.981 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:26.981 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:26.981 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:30:26.982 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.112183+0000", 2024-09-14T06:30:26.982 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.5021000000000001e-05 2024-09-14T06:30:26.982 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:26.982 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:30:26.982 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:26.982 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:26.982 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:26.982 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:19372 3.3 3:c34e5e51:::rbd_data.112d92027590.0000000000000024:f [read 1374208~1024 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:30:26.982 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:27.065176+0000", 2024-09-14T06:30:26.982 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 59.793979829000001, 2024-09-14T06:30:26.982 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047188889999999997, 2024-09-14T06:30:26.982 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:30:26.982 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:30:26.982 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:30:26.982 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:30:26.983 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:30:26.983 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:26.983 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 19372 2024-09-14T06:30:26.983 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:26.983 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:30:26.983 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:26.983 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:30:26.983 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065176+0000", 2024-09-14T06:30:26.983 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:26.983 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:26.983 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:26.983 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:30:26.983 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065176+0000", 2024-09-14T06:30:26.983 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:26.984 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:26.984 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:26.984 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:30:26.984 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.071117+0000", 2024-09-14T06:30:26.984 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.0059404990000000001 2024-09-14T06:30:26.984 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:26.984 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:26.984 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:30:26.984 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.071120+0000", 2024-09-14T06:30:26.984 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.4340000000000001e-06 2024-09-14T06:30:26.984 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:26.984 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:26.984 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:30:26.984 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.071129+0000", 2024-09-14T06:30:26.985 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.8389999999999996e-06 2024-09-14T06:30:26.985 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:26.985 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:26.985 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:30:26.985 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.071134+0000", 2024-09-14T06:30:26.985 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.412e-06 2024-09-14T06:30:26.985 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:26.985 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:26.985 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:30:26.985 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.112221+0000", 2024-09-14T06:30:26.985 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.041086473999999998 2024-09-14T06:30:26.985 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:26.985 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:26.985 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:30:26.985 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.112299+0000", 2024-09-14T06:30:26.986 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.8746999999999997e-05 2024-09-14T06:30:26.986 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:26.986 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:26.986 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:30:26.986 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.112365+0000", 2024-09-14T06:30:26.986 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.5485000000000001e-05 2024-09-14T06:30:26.986 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:26.986 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:30:26.986 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:26.986 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:26.986 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:30:26.986 INFO:teuthology.orchestra.run.smithi138.stdout:} 2024-09-14T06:30:26.988 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-14T06:30:26.989 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-14T06:30:26.989 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-14T06:30:26.989 INFO:teuthology.orchestra.run.smithi094.stdout: "epoch": 104, 2024-09-14T06:30:26.989 INFO:teuthology.orchestra.run.smithi094.stdout: "fsid": "200ad3da-bd63-4e04-af74-efb5ca0af75e", 2024-09-14T06:30:26.989 INFO:teuthology.orchestra.run.smithi094.stdout: "created": "2024-09-14T06:28:30.782046+0000", 2024-09-14T06:30:26.989 INFO:teuthology.orchestra.run.smithi094.stdout: "modified": "2024-09-14T06:30:25.616275+0000", 2024-09-14T06:30:26.990 INFO:teuthology.orchestra.run.smithi094.stdout: "last_up_change": "2024-09-14T06:28:42.827239+0000", 2024-09-14T06:30:26.990 INFO:teuthology.orchestra.run.smithi094.stdout: "last_in_change": "2024-09-14T06:28:33.724176+0000", 2024-09-14T06:30:26.990 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": "noscrub,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2024-09-14T06:30:26.990 INFO:teuthology.orchestra.run.smithi094.stdout: "flags_num": 5801984, 2024-09-14T06:30:26.990 INFO:teuthology.orchestra.run.smithi094.stdout: "flags_set": [ 2024-09-14T06:30:26.990 INFO:teuthology.orchestra.run.smithi094.stdout: "noscrub", 2024-09-14T06:30:26.990 INFO:teuthology.orchestra.run.smithi094.stdout: "pglog_hardlimit", 2024-09-14T06:30:26.990 INFO:teuthology.orchestra.run.smithi094.stdout: "purged_snapdirs", 2024-09-14T06:30:26.990 INFO:teuthology.orchestra.run.smithi094.stdout: "recovery_deletes", 2024-09-14T06:30:26.990 INFO:teuthology.orchestra.run.smithi094.stdout: "sortbitwise" 2024-09-14T06:30:26.990 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-09-14T06:30:26.990 INFO:teuthology.orchestra.run.smithi094.stdout: "crush_version": 5, 2024-09-14T06:30:26.990 INFO:teuthology.orchestra.run.smithi094.stdout: "full_ratio": 0.94999998807907104, 2024-09-14T06:30:26.991 INFO:teuthology.orchestra.run.smithi094.stdout: "backfillfull_ratio": 0.89999997615814209, 2024-09-14T06:30:26.991 INFO:teuthology.orchestra.run.smithi094.stdout: "nearfull_ratio": 0.85000002384185791, 2024-09-14T06:30:26.991 INFO:teuthology.orchestra.run.smithi094.stdout: "cluster_snapshot": "", 2024-09-14T06:30:26.991 INFO:teuthology.orchestra.run.smithi094.stdout: "pool_max": 3, 2024-09-14T06:30:26.991 INFO:teuthology.orchestra.run.smithi094.stdout: "max_osd": 8, 2024-09-14T06:30:26.991 INFO:teuthology.orchestra.run.smithi094.stdout: "require_min_compat_client": "luminous", 2024-09-14T06:30:26.991 INFO:teuthology.orchestra.run.smithi094.stdout: "min_compat_client": "luminous", 2024-09-14T06:30:26.991 INFO:teuthology.orchestra.run.smithi094.stdout: "require_osd_release": "squid", 2024-09-14T06:30:26.991 INFO:teuthology.orchestra.run.smithi094.stdout: "allow_crimson": false, 2024-09-14T06:30:26.991 INFO:teuthology.orchestra.run.smithi094.stdout: "pools": [ 2024-09-14T06:30:26.991 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:26.991 INFO:teuthology.orchestra.run.smithi094.stdout: "pool": 1, 2024-09-14T06:30:26.991 INFO:teuthology.orchestra.run.smithi094.stdout: "pool_name": ".mgr", 2024-09-14T06:30:26.991 INFO:teuthology.orchestra.run.smithi094.stdout: "create_time": "2024-09-14T06:28:43.295990+0000", 2024-09-14T06:30:26.992 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-14T06:30:26.992 INFO:teuthology.orchestra.run.smithi094.stdout: "flags_names": "hashpspool", 2024-09-14T06:30:26.992 INFO:teuthology.orchestra.run.smithi094.stdout: "type": 1, 2024-09-14T06:30:26.992 INFO:teuthology.orchestra.run.smithi094.stdout: "size": 2, 2024-09-14T06:30:26.992 INFO:teuthology.orchestra.run.smithi094.stdout: "min_size": 1, 2024-09-14T06:30:26.992 INFO:teuthology.orchestra.run.smithi094.stdout: "crush_rule": 0, 2024-09-14T06:30:26.992 INFO:teuthology.orchestra.run.smithi094.stdout: "peering_crush_bucket_count": 0, 2024-09-14T06:30:26.992 INFO:teuthology.orchestra.run.smithi094.stdout: "peering_crush_bucket_target": 0, 2024-09-14T06:30:26.992 INFO:teuthology.orchestra.run.smithi094.stdout: "peering_crush_bucket_barrier": 0, 2024-09-14T06:30:26.992 INFO:teuthology.orchestra.run.smithi094.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-14T06:30:26.992 INFO:teuthology.orchestra.run.smithi094.stdout: "is_stretch_pool": false, 2024-09-14T06:30:26.992 INFO:teuthology.orchestra.run.smithi094.stdout: "object_hash": 2, 2024-09-14T06:30:26.992 INFO:teuthology.orchestra.run.smithi094.stdout: "pg_autoscale_mode": "off", 2024-09-14T06:30:26.992 INFO:teuthology.orchestra.run.smithi094.stdout: "pg_num": 1, 2024-09-14T06:30:26.993 INFO:teuthology.orchestra.run.smithi094.stdout: "pg_placement_num": 1, 2024-09-14T06:30:26.993 INFO:teuthology.orchestra.run.smithi094.stdout: "pg_placement_num_target": 1, 2024-09-14T06:30:26.993 INFO:teuthology.orchestra.run.smithi094.stdout: "pg_num_target": 1, 2024-09-14T06:30:26.993 INFO:teuthology.orchestra.run.smithi094.stdout: "pg_num_pending": 1, 2024-09-14T06:30:26.993 INFO:teuthology.orchestra.run.smithi094.stdout: "last_pg_merge_meta": { 2024-09-14T06:30:26.993 INFO:teuthology.orchestra.run.smithi094.stdout: "source_pgid": "0.0", 2024-09-14T06:30:26.993 INFO:teuthology.orchestra.run.smithi094.stdout: "ready_epoch": 0, 2024-09-14T06:30:26.993 INFO:teuthology.orchestra.run.smithi094.stdout: "last_epoch_started": 0, 2024-09-14T06:30:26.993 INFO:teuthology.orchestra.run.smithi094.stdout: "last_epoch_clean": 0, 2024-09-14T06:30:26.993 INFO:teuthology.orchestra.run.smithi094.stdout: "source_version": "0'0", 2024-09-14T06:30:26.993 INFO:teuthology.orchestra.run.smithi094.stdout: "target_version": "0'0" 2024-09-14T06:30:26.993 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:26.993 INFO:teuthology.orchestra.run.smithi094.stdout: "last_change": "18", 2024-09-14T06:30:26.993 INFO:teuthology.orchestra.run.smithi094.stdout: "last_force_op_resend": "0", 2024-09-14T06:30:26.994 INFO:teuthology.orchestra.run.smithi094.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-14T06:30:26.994 INFO:teuthology.orchestra.run.smithi094.stdout: "last_force_op_resend_preluminous": "0", 2024-09-14T06:30:26.994 INFO:teuthology.orchestra.run.smithi094.stdout: "auid": 0, 2024-09-14T06:30:26.994 INFO:teuthology.orchestra.run.smithi094.stdout: "snap_mode": "selfmanaged", 2024-09-14T06:30:26.994 INFO:teuthology.orchestra.run.smithi094.stdout: "snap_seq": 0, 2024-09-14T06:30:26.994 INFO:teuthology.orchestra.run.smithi094.stdout: "snap_epoch": 0, 2024-09-14T06:30:26.994 INFO:teuthology.orchestra.run.smithi094.stdout: "pool_snaps": [], 2024-09-14T06:30:26.994 INFO:teuthology.orchestra.run.smithi094.stdout: "removed_snaps": "[]", 2024-09-14T06:30:26.994 INFO:teuthology.orchestra.run.smithi094.stdout: "quota_max_bytes": 0, 2024-09-14T06:30:26.994 INFO:teuthology.orchestra.run.smithi094.stdout: "quota_max_objects": 0, 2024-09-14T06:30:26.994 INFO:teuthology.orchestra.run.smithi094.stdout: "tiers": [], 2024-09-14T06:30:26.994 INFO:teuthology.orchestra.run.smithi094.stdout: "tier_of": -1, 2024-09-14T06:30:26.994 INFO:teuthology.orchestra.run.smithi094.stdout: "read_tier": -1, 2024-09-14T06:30:26.994 INFO:teuthology.orchestra.run.smithi094.stdout: "write_tier": -1, 2024-09-14T06:30:26.995 INFO:teuthology.orchestra.run.smithi094.stdout: "cache_mode": "none", 2024-09-14T06:30:26.995 INFO:teuthology.orchestra.run.smithi094.stdout: "target_max_bytes": 0, 2024-09-14T06:30:26.995 INFO:teuthology.orchestra.run.smithi094.stdout: "target_max_objects": 0, 2024-09-14T06:30:26.995 INFO:teuthology.orchestra.run.smithi094.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-14T06:30:26.995 INFO:teuthology.orchestra.run.smithi094.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-14T06:30:26.995 INFO:teuthology.orchestra.run.smithi094.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-14T06:30:26.995 INFO:teuthology.orchestra.run.smithi094.stdout: "cache_min_flush_age": 0, 2024-09-14T06:30:26.995 INFO:teuthology.orchestra.run.smithi094.stdout: "cache_min_evict_age": 0, 2024-09-14T06:30:26.995 INFO:teuthology.orchestra.run.smithi094.stdout: "erasure_code_profile": "", 2024-09-14T06:30:26.995 INFO:teuthology.orchestra.run.smithi094.stdout: "hit_set_params": { 2024-09-14T06:30:26.995 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "none" 2024-09-14T06:30:26.995 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:26.995 INFO:teuthology.orchestra.run.smithi094.stdout: "hit_set_period": 0, 2024-09-14T06:30:26.995 INFO:teuthology.orchestra.run.smithi094.stdout: "hit_set_count": 0, 2024-09-14T06:30:26.996 INFO:teuthology.orchestra.run.smithi094.stdout: "use_gmt_hitset": true, 2024-09-14T06:30:26.996 INFO:teuthology.orchestra.run.smithi094.stdout: "min_read_recency_for_promote": 0, 2024-09-14T06:30:26.996 INFO:teuthology.orchestra.run.smithi094.stdout: "min_write_recency_for_promote": 0, 2024-09-14T06:30:26.996 INFO:teuthology.orchestra.run.smithi094.stdout: "hit_set_grade_decay_rate": 0, 2024-09-14T06:30:26.996 INFO:teuthology.orchestra.run.smithi094.stdout: "hit_set_search_last_n": 0, 2024-09-14T06:30:26.996 INFO:teuthology.orchestra.run.smithi094.stdout: "grade_table": [], 2024-09-14T06:30:26.996 INFO:teuthology.orchestra.run.smithi094.stdout: "stripe_width": 0, 2024-09-14T06:30:26.996 INFO:teuthology.orchestra.run.smithi094.stdout: "expected_num_objects": 0, 2024-09-14T06:30:26.996 INFO:teuthology.orchestra.run.smithi094.stdout: "fast_read": false, 2024-09-14T06:30:26.996 INFO:teuthology.orchestra.run.smithi094.stdout: "options": { 2024-09-14T06:30:26.996 INFO:teuthology.orchestra.run.smithi094.stdout: "pg_num_max": 32, 2024-09-14T06:30:26.996 INFO:teuthology.orchestra.run.smithi094.stdout: "pg_num_min": 1 2024-09-14T06:30:26.996 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:26.996 INFO:teuthology.orchestra.run.smithi094.stdout: "application_metadata": { 2024-09-14T06:30:26.997 INFO:teuthology.orchestra.run.smithi094.stdout: "mgr": {} 2024-09-14T06:30:26.997 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:26.997 INFO:teuthology.orchestra.run.smithi094.stdout: "read_balance": { 2024-09-14T06:30:26.997 INFO:teuthology.orchestra.run.smithi094.stdout: "score_type": "Fair distribution", 2024-09-14T06:30:26.997 INFO:teuthology.orchestra.run.smithi094.stdout: "score_acting": 8, 2024-09-14T06:30:26.997 INFO:teuthology.orchestra.run.smithi094.stdout: "score_stable": 8, 2024-09-14T06:30:26.997 INFO:teuthology.orchestra.run.smithi094.stdout: "optimal_score": 0.25, 2024-09-14T06:30:26.997 INFO:teuthology.orchestra.run.smithi094.stdout: "raw_score_acting": 2, 2024-09-14T06:30:26.997 INFO:teuthology.orchestra.run.smithi094.stdout: "raw_score_stable": 2, 2024-09-14T06:30:26.997 INFO:teuthology.orchestra.run.smithi094.stdout: "primary_affinity_weighted": 1, 2024-09-14T06:30:26.997 INFO:teuthology.orchestra.run.smithi094.stdout: "average_primary_affinity": 1, 2024-09-14T06:30:26.997 INFO:teuthology.orchestra.run.smithi094.stdout: "average_primary_affinity_weighted": 1 2024-09-14T06:30:26.997 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:26.997 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:26.998 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:26.998 INFO:teuthology.orchestra.run.smithi094.stdout: "pool": 2, 2024-09-14T06:30:26.998 INFO:teuthology.orchestra.run.smithi094.stdout: "pool_name": "rbd", 2024-09-14T06:30:26.998 INFO:teuthology.orchestra.run.smithi094.stdout: "create_time": "2024-09-14T06:28:49.451714+0000", 2024-09-14T06:30:26.998 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 8193, 2024-09-14T06:30:26.998 INFO:teuthology.orchestra.run.smithi094.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-09-14T06:30:26.998 INFO:teuthology.orchestra.run.smithi094.stdout: "type": 1, 2024-09-14T06:30:26.998 INFO:teuthology.orchestra.run.smithi094.stdout: "size": 2, 2024-09-14T06:30:26.998 INFO:teuthology.orchestra.run.smithi094.stdout: "min_size": 1, 2024-09-14T06:30:26.998 INFO:teuthology.orchestra.run.smithi094.stdout: "crush_rule": 0, 2024-09-14T06:30:26.998 INFO:teuthology.orchestra.run.smithi094.stdout: "peering_crush_bucket_count": 0, 2024-09-14T06:30:26.998 INFO:teuthology.orchestra.run.smithi094.stdout: "peering_crush_bucket_target": 0, 2024-09-14T06:30:26.998 INFO:teuthology.orchestra.run.smithi094.stdout: "peering_crush_bucket_barrier": 0, 2024-09-14T06:30:26.998 INFO:teuthology.orchestra.run.smithi094.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-14T06:30:26.999 INFO:teuthology.orchestra.run.smithi094.stdout: "is_stretch_pool": false, 2024-09-14T06:30:26.999 INFO:teuthology.orchestra.run.smithi094.stdout: "object_hash": 2, 2024-09-14T06:30:26.999 INFO:teuthology.orchestra.run.smithi094.stdout: "pg_autoscale_mode": "off", 2024-09-14T06:30:26.999 INFO:teuthology.orchestra.run.smithi094.stdout: "pg_num": 8, 2024-09-14T06:30:26.999 INFO:teuthology.orchestra.run.smithi094.stdout: "pg_placement_num": 8, 2024-09-14T06:30:26.999 INFO:teuthology.orchestra.run.smithi094.stdout: "pg_placement_num_target": 8, 2024-09-14T06:30:26.999 INFO:teuthology.orchestra.run.smithi094.stdout: "pg_num_target": 8, 2024-09-14T06:30:26.999 INFO:teuthology.orchestra.run.smithi094.stdout: "pg_num_pending": 8, 2024-09-14T06:30:26.999 INFO:teuthology.orchestra.run.smithi094.stdout: "last_pg_merge_meta": { 2024-09-14T06:30:26.999 INFO:teuthology.orchestra.run.smithi094.stdout: "source_pgid": "0.0", 2024-09-14T06:30:26.999 INFO:teuthology.orchestra.run.smithi094.stdout: "ready_epoch": 0, 2024-09-14T06:30:26.999 INFO:teuthology.orchestra.run.smithi094.stdout: "last_epoch_started": 0, 2024-09-14T06:30:26.999 INFO:teuthology.orchestra.run.smithi094.stdout: "last_epoch_clean": 0, 2024-09-14T06:30:27.000 INFO:teuthology.orchestra.run.smithi094.stdout: "source_version": "0'0", 2024-09-14T06:30:27.000 INFO:teuthology.orchestra.run.smithi094.stdout: "target_version": "0'0" 2024-09-14T06:30:27.000 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:27.000 INFO:teuthology.orchestra.run.smithi094.stdout: "last_change": "22", 2024-09-14T06:30:27.000 INFO:teuthology.orchestra.run.smithi094.stdout: "last_force_op_resend": "0", 2024-09-14T06:30:27.000 INFO:teuthology.orchestra.run.smithi094.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-14T06:30:27.000 INFO:teuthology.orchestra.run.smithi094.stdout: "last_force_op_resend_preluminous": "0", 2024-09-14T06:30:27.000 INFO:teuthology.orchestra.run.smithi094.stdout: "auid": 0, 2024-09-14T06:30:27.000 INFO:teuthology.orchestra.run.smithi094.stdout: "snap_mode": "selfmanaged", 2024-09-14T06:30:27.000 INFO:teuthology.orchestra.run.smithi094.stdout: "snap_seq": 2, 2024-09-14T06:30:27.000 INFO:teuthology.orchestra.run.smithi094.stdout: "snap_epoch": 22, 2024-09-14T06:30:27.000 INFO:teuthology.orchestra.run.smithi094.stdout: "pool_snaps": [], 2024-09-14T06:30:27.000 INFO:teuthology.orchestra.run.smithi094.stdout: "removed_snaps": "[]", 2024-09-14T06:30:27.000 INFO:teuthology.orchestra.run.smithi094.stdout: "quota_max_bytes": 0, 2024-09-14T06:30:27.001 INFO:teuthology.orchestra.run.smithi094.stdout: "quota_max_objects": 0, 2024-09-14T06:30:27.001 INFO:teuthology.orchestra.run.smithi094.stdout: "tiers": [], 2024-09-14T06:30:27.001 INFO:teuthology.orchestra.run.smithi094.stdout: "tier_of": -1, 2024-09-14T06:30:27.001 INFO:teuthology.orchestra.run.smithi094.stdout: "read_tier": -1, 2024-09-14T06:30:27.001 INFO:teuthology.orchestra.run.smithi094.stdout: "write_tier": -1, 2024-09-14T06:30:27.001 INFO:teuthology.orchestra.run.smithi094.stdout: "cache_mode": "none", 2024-09-14T06:30:27.001 INFO:teuthology.orchestra.run.smithi094.stdout: "target_max_bytes": 0, 2024-09-14T06:30:27.001 INFO:teuthology.orchestra.run.smithi094.stdout: "target_max_objects": 0, 2024-09-14T06:30:27.001 INFO:teuthology.orchestra.run.smithi094.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-14T06:30:27.001 INFO:teuthology.orchestra.run.smithi094.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-14T06:30:27.001 INFO:teuthology.orchestra.run.smithi094.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-14T06:30:27.001 INFO:teuthology.orchestra.run.smithi094.stdout: "cache_min_flush_age": 0, 2024-09-14T06:30:27.001 INFO:teuthology.orchestra.run.smithi094.stdout: "cache_min_evict_age": 0, 2024-09-14T06:30:27.002 INFO:teuthology.orchestra.run.smithi094.stdout: "erasure_code_profile": "", 2024-09-14T06:30:27.002 INFO:teuthology.orchestra.run.smithi094.stdout: "hit_set_params": { 2024-09-14T06:30:27.002 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "none" 2024-09-14T06:30:27.002 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:27.002 INFO:teuthology.orchestra.run.smithi094.stdout: "hit_set_period": 0, 2024-09-14T06:30:27.002 INFO:teuthology.orchestra.run.smithi094.stdout: "hit_set_count": 0, 2024-09-14T06:30:27.002 INFO:teuthology.orchestra.run.smithi094.stdout: "use_gmt_hitset": true, 2024-09-14T06:30:27.002 INFO:teuthology.orchestra.run.smithi094.stdout: "min_read_recency_for_promote": 0, 2024-09-14T06:30:27.002 INFO:teuthology.orchestra.run.smithi094.stdout: "min_write_recency_for_promote": 0, 2024-09-14T06:30:27.002 INFO:teuthology.orchestra.run.smithi094.stdout: "hit_set_grade_decay_rate": 0, 2024-09-14T06:30:27.002 INFO:teuthology.orchestra.run.smithi094.stdout: "hit_set_search_last_n": 0, 2024-09-14T06:30:27.002 INFO:teuthology.orchestra.run.smithi094.stdout: "grade_table": [], 2024-09-14T06:30:27.002 INFO:teuthology.orchestra.run.smithi094.stdout: "stripe_width": 0, 2024-09-14T06:30:27.002 INFO:teuthology.orchestra.run.smithi094.stdout: "expected_num_objects": 0, 2024-09-14T06:30:27.002 INFO:teuthology.orchestra.run.smithi094.stdout: "fast_read": false, 2024-09-14T06:30:27.003 INFO:teuthology.orchestra.run.smithi094.stdout: "options": {}, 2024-09-14T06:30:27.003 INFO:teuthology.orchestra.run.smithi094.stdout: "application_metadata": { 2024-09-14T06:30:27.003 INFO:teuthology.orchestra.run.smithi094.stdout: "rbd": {} 2024-09-14T06:30:27.003 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:27.003 INFO:teuthology.orchestra.run.smithi094.stdout: "read_balance": { 2024-09-14T06:30:27.003 INFO:teuthology.orchestra.run.smithi094.stdout: "score_type": "Fair distribution", 2024-09-14T06:30:27.003 INFO:teuthology.orchestra.run.smithi094.stdout: "score_acting": 1.9900000095367432, 2024-09-14T06:30:27.003 INFO:teuthology.orchestra.run.smithi094.stdout: "score_stable": 1.9900000095367432, 2024-09-14T06:30:27.003 INFO:teuthology.orchestra.run.smithi094.stdout: "optimal_score": 0.87999999523162842, 2024-09-14T06:30:27.003 INFO:teuthology.orchestra.run.smithi094.stdout: "raw_score_acting": 1.75, 2024-09-14T06:30:27.003 INFO:teuthology.orchestra.run.smithi094.stdout: "raw_score_stable": 1.75, 2024-09-14T06:30:27.003 INFO:teuthology.orchestra.run.smithi094.stdout: "primary_affinity_weighted": 1, 2024-09-14T06:30:27.003 INFO:teuthology.orchestra.run.smithi094.stdout: "average_primary_affinity": 1, 2024-09-14T06:30:27.004 INFO:teuthology.orchestra.run.smithi094.stdout: "average_primary_affinity_weighted": 1 2024-09-14T06:30:27.004 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:27.004 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:27.004 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:27.004 INFO:teuthology.orchestra.run.smithi094.stdout: "pool": 3, 2024-09-14T06:30:27.004 INFO:teuthology.orchestra.run.smithi094.stdout: "pool_name": "pool_client.0", 2024-09-14T06:30:27.004 INFO:teuthology.orchestra.run.smithi094.stdout: "create_time": "2024-09-14T06:28:59.688535+0000", 2024-09-14T06:30:27.004 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 8193, 2024-09-14T06:30:27.004 INFO:teuthology.orchestra.run.smithi094.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-09-14T06:30:27.004 INFO:teuthology.orchestra.run.smithi094.stdout: "type": 1, 2024-09-14T06:30:27.004 INFO:teuthology.orchestra.run.smithi094.stdout: "size": 2, 2024-09-14T06:30:27.004 INFO:teuthology.orchestra.run.smithi094.stdout: "min_size": 1, 2024-09-14T06:30:27.004 INFO:teuthology.orchestra.run.smithi094.stdout: "crush_rule": 0, 2024-09-14T06:30:27.004 INFO:teuthology.orchestra.run.smithi094.stdout: "peering_crush_bucket_count": 0, 2024-09-14T06:30:27.005 INFO:teuthology.orchestra.run.smithi094.stdout: "peering_crush_bucket_target": 0, 2024-09-14T06:30:27.005 INFO:teuthology.orchestra.run.smithi094.stdout: "peering_crush_bucket_barrier": 0, 2024-09-14T06:30:27.005 INFO:teuthology.orchestra.run.smithi094.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-14T06:30:27.005 INFO:teuthology.orchestra.run.smithi094.stdout: "is_stretch_pool": false, 2024-09-14T06:30:27.005 INFO:teuthology.orchestra.run.smithi094.stdout: "object_hash": 2, 2024-09-14T06:30:27.005 INFO:teuthology.orchestra.run.smithi094.stdout: "pg_autoscale_mode": "off", 2024-09-14T06:30:27.005 INFO:teuthology.orchestra.run.smithi094.stdout: "pg_num": 32, 2024-09-14T06:30:27.005 INFO:teuthology.orchestra.run.smithi094.stdout: "pg_placement_num": 32, 2024-09-14T06:30:27.005 INFO:teuthology.orchestra.run.smithi094.stdout: "pg_placement_num_target": 32, 2024-09-14T06:30:27.005 INFO:teuthology.orchestra.run.smithi094.stdout: "pg_num_target": 32, 2024-09-14T06:30:27.005 INFO:teuthology.orchestra.run.smithi094.stdout: "pg_num_pending": 32, 2024-09-14T06:30:27.005 INFO:teuthology.orchestra.run.smithi094.stdout: "last_pg_merge_meta": { 2024-09-14T06:30:27.005 INFO:teuthology.orchestra.run.smithi094.stdout: "source_pgid": "0.0", 2024-09-14T06:30:27.005 INFO:teuthology.orchestra.run.smithi094.stdout: "ready_epoch": 0, 2024-09-14T06:30:27.006 INFO:teuthology.orchestra.run.smithi094.stdout: "last_epoch_started": 0, 2024-09-14T06:30:27.006 INFO:teuthology.orchestra.run.smithi094.stdout: "last_epoch_clean": 0, 2024-09-14T06:30:27.006 INFO:teuthology.orchestra.run.smithi094.stdout: "source_version": "0'0", 2024-09-14T06:30:27.006 INFO:teuthology.orchestra.run.smithi094.stdout: "target_version": "0'0" 2024-09-14T06:30:27.006 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:27.006 INFO:teuthology.orchestra.run.smithi094.stdout: "last_change": "104", 2024-09-14T06:30:27.006 INFO:teuthology.orchestra.run.smithi094.stdout: "last_force_op_resend": "0", 2024-09-14T06:30:27.006 INFO:teuthology.orchestra.run.smithi094.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-14T06:30:27.006 INFO:teuthology.orchestra.run.smithi094.stdout: "last_force_op_resend_preluminous": "0", 2024-09-14T06:30:27.006 INFO:teuthology.orchestra.run.smithi094.stdout: "auid": 0, 2024-09-14T06:30:27.006 INFO:teuthology.orchestra.run.smithi094.stdout: "snap_mode": "selfmanaged", 2024-09-14T06:30:27.006 INFO:teuthology.orchestra.run.smithi094.stdout: "snap_seq": 46, 2024-09-14T06:30:27.006 INFO:teuthology.orchestra.run.smithi094.stdout: "snap_epoch": 104, 2024-09-14T06:30:27.007 INFO:teuthology.orchestra.run.smithi094.stdout: "pool_snaps": [], 2024-09-14T06:30:27.007 INFO:teuthology.orchestra.run.smithi094.stdout: "removed_snaps": "[]", 2024-09-14T06:30:27.007 INFO:teuthology.orchestra.run.smithi094.stdout: "quota_max_bytes": 0, 2024-09-14T06:30:27.007 INFO:teuthology.orchestra.run.smithi094.stdout: "quota_max_objects": 0, 2024-09-14T06:30:27.007 INFO:teuthology.orchestra.run.smithi094.stdout: "tiers": [], 2024-09-14T06:30:27.007 INFO:teuthology.orchestra.run.smithi094.stdout: "tier_of": -1, 2024-09-14T06:30:27.007 INFO:teuthology.orchestra.run.smithi094.stdout: "read_tier": -1, 2024-09-14T06:30:27.007 INFO:teuthology.orchestra.run.smithi094.stdout: "write_tier": -1, 2024-09-14T06:30:27.007 INFO:teuthology.orchestra.run.smithi094.stdout: "cache_mode": "none", 2024-09-14T06:30:27.007 INFO:teuthology.orchestra.run.smithi094.stdout: "target_max_bytes": 0, 2024-09-14T06:30:27.007 INFO:teuthology.orchestra.run.smithi094.stdout: "target_max_objects": 0, 2024-09-14T06:30:27.007 INFO:teuthology.orchestra.run.smithi094.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-14T06:30:27.007 INFO:teuthology.orchestra.run.smithi094.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-14T06:30:27.008 INFO:teuthology.orchestra.run.smithi094.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-14T06:30:27.008 INFO:teuthology.orchestra.run.smithi094.stdout: "cache_min_flush_age": 0, 2024-09-14T06:30:27.008 INFO:teuthology.orchestra.run.smithi094.stdout: "cache_min_evict_age": 0, 2024-09-14T06:30:27.008 INFO:teuthology.orchestra.run.smithi094.stdout: "erasure_code_profile": "", 2024-09-14T06:30:27.008 INFO:teuthology.orchestra.run.smithi094.stdout: "hit_set_params": { 2024-09-14T06:30:27.008 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "none" 2024-09-14T06:30:27.008 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:27.008 INFO:teuthology.orchestra.run.smithi094.stdout: "hit_set_period": 0, 2024-09-14T06:30:27.008 INFO:teuthology.orchestra.run.smithi094.stdout: "hit_set_count": 0, 2024-09-14T06:30:27.008 INFO:teuthology.orchestra.run.smithi094.stdout: "use_gmt_hitset": true, 2024-09-14T06:30:27.008 INFO:teuthology.orchestra.run.smithi094.stdout: "min_read_recency_for_promote": 0, 2024-09-14T06:30:27.008 INFO:teuthology.orchestra.run.smithi094.stdout: "min_write_recency_for_promote": 0, 2024-09-14T06:30:27.008 INFO:teuthology.orchestra.run.smithi094.stdout: "hit_set_grade_decay_rate": 0, 2024-09-14T06:30:27.009 INFO:teuthology.orchestra.run.smithi094.stdout: "hit_set_search_last_n": 0, 2024-09-14T06:30:27.009 INFO:teuthology.orchestra.run.smithi094.stdout: "grade_table": [], 2024-09-14T06:30:27.009 INFO:teuthology.orchestra.run.smithi094.stdout: "stripe_width": 0, 2024-09-14T06:30:27.009 INFO:teuthology.orchestra.run.smithi094.stdout: "expected_num_objects": 0, 2024-09-14T06:30:27.009 INFO:teuthology.orchestra.run.smithi094.stdout: "fast_read": false, 2024-09-14T06:30:27.009 INFO:teuthology.orchestra.run.smithi094.stdout: "options": {}, 2024-09-14T06:30:27.009 INFO:teuthology.orchestra.run.smithi094.stdout: "application_metadata": { 2024-09-14T06:30:27.009 INFO:teuthology.orchestra.run.smithi094.stdout: "rbd": {} 2024-09-14T06:30:27.009 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:27.009 INFO:teuthology.orchestra.run.smithi094.stdout: "read_balance": { 2024-09-14T06:30:27.009 INFO:teuthology.orchestra.run.smithi094.stdout: "score_type": "Fair distribution", 2024-09-14T06:30:27.009 INFO:teuthology.orchestra.run.smithi094.stdout: "score_acting": 1.75, 2024-09-14T06:30:27.009 INFO:teuthology.orchestra.run.smithi094.stdout: "score_stable": 1.75, 2024-09-14T06:30:27.009 INFO:teuthology.orchestra.run.smithi094.stdout: "optimal_score": 1, 2024-09-14T06:30:27.010 INFO:teuthology.orchestra.run.smithi094.stdout: "raw_score_acting": 1.75, 2024-09-14T06:30:27.010 INFO:teuthology.orchestra.run.smithi094.stdout: "raw_score_stable": 1.75, 2024-09-14T06:30:27.010 INFO:teuthology.orchestra.run.smithi094.stdout: "primary_affinity_weighted": 1, 2024-09-14T06:30:27.010 INFO:teuthology.orchestra.run.smithi094.stdout: "average_primary_affinity": 1, 2024-09-14T06:30:27.010 INFO:teuthology.orchestra.run.smithi094.stdout: "average_primary_affinity_weighted": 1 2024-09-14T06:30:27.010 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:27.010 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:27.010 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-09-14T06:30:27.010 INFO:teuthology.orchestra.run.smithi094.stdout: "osds": [ 2024-09-14T06:30:27.010 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:27.010 INFO:teuthology.orchestra.run.smithi094.stdout: "osd": 0, 2024-09-14T06:30:27.010 INFO:teuthology.orchestra.run.smithi094.stdout: "uuid": "8b22a249-b842-406a-a02c-c0f31f57b635", 2024-09-14T06:30:27.010 INFO:teuthology.orchestra.run.smithi094.stdout: "up": 1, 2024-09-14T06:30:27.010 INFO:teuthology.orchestra.run.smithi094.stdout: "in": 1, 2024-09-14T06:30:27.011 INFO:teuthology.orchestra.run.smithi094.stdout: "weight": 1, 2024-09-14T06:30:27.011 INFO:teuthology.orchestra.run.smithi094.stdout: "primary_affinity": 1, 2024-09-14T06:30:27.011 INFO:teuthology.orchestra.run.smithi094.stdout: "last_clean_begin": 0, 2024-09-14T06:30:27.011 INFO:teuthology.orchestra.run.smithi094.stdout: "last_clean_end": 0, 2024-09-14T06:30:27.011 INFO:teuthology.orchestra.run.smithi094.stdout: "up_from": 14, 2024-09-14T06:30:27.011 INFO:teuthology.orchestra.run.smithi094.stdout: "up_thru": 75, 2024-09-14T06:30:27.011 INFO:teuthology.orchestra.run.smithi094.stdout: "down_at": 0, 2024-09-14T06:30:27.011 INFO:teuthology.orchestra.run.smithi094.stdout: "lost_at": 0, 2024-09-14T06:30:27.011 INFO:teuthology.orchestra.run.smithi094.stdout: "public_addrs": { 2024-09-14T06:30:27.011 INFO:teuthology.orchestra.run.smithi094.stdout: "addrvec": [ 2024-09-14T06:30:27.011 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:27.011 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v2", 2024-09-14T06:30:27.011 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.94:6824", 2024-09-14T06:30:27.011 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 1809564923 2024-09-14T06:30:27.012 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:27.012 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:27.012 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v1", 2024-09-14T06:30:27.012 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.94:6825", 2024-09-14T06:30:27.012 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 1809564923 2024-09-14T06:30:27.012 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:27.012 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:27.012 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:27.012 INFO:teuthology.orchestra.run.smithi094.stdout: "cluster_addrs": { 2024-09-14T06:30:27.012 INFO:teuthology.orchestra.run.smithi094.stdout: "addrvec": [ 2024-09-14T06:30:27.012 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:27.012 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v2", 2024-09-14T06:30:27.012 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.94:6826", 2024-09-14T06:30:27.012 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 1809564923 2024-09-14T06:30:27.013 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:27.013 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:27.013 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v1", 2024-09-14T06:30:27.013 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.94:6827", 2024-09-14T06:30:27.013 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 1809564923 2024-09-14T06:30:27.013 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:27.013 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:27.013 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:27.013 INFO:teuthology.orchestra.run.smithi094.stdout: "heartbeat_back_addrs": { 2024-09-14T06:30:27.013 INFO:teuthology.orchestra.run.smithi094.stdout: "addrvec": [ 2024-09-14T06:30:27.013 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:27.013 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v2", 2024-09-14T06:30:27.013 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.94:6830", 2024-09-14T06:30:27.013 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 1809564923 2024-09-14T06:30:27.014 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:27.014 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:27.014 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v1", 2024-09-14T06:30:27.014 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.94:6831", 2024-09-14T06:30:27.014 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 1809564923 2024-09-14T06:30:27.014 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:27.014 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:27.014 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:27.014 INFO:teuthology.orchestra.run.smithi094.stdout: "heartbeat_front_addrs": { 2024-09-14T06:30:27.014 INFO:teuthology.orchestra.run.smithi094.stdout: "addrvec": [ 2024-09-14T06:30:27.014 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:27.014 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v2", 2024-09-14T06:30:27.014 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.94:6828", 2024-09-14T06:30:27.014 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 1809564923 2024-09-14T06:30:27.015 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:27.015 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:27.015 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v1", 2024-09-14T06:30:27.015 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.94:6829", 2024-09-14T06:30:27.015 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 1809564923 2024-09-14T06:30:27.015 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:27.015 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:27.015 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:27.015 INFO:teuthology.orchestra.run.smithi094.stdout: "public_addr": "172.21.15.94:6825/1809564923", 2024-09-14T06:30:27.015 INFO:teuthology.orchestra.run.smithi094.stdout: "cluster_addr": "172.21.15.94:6827/1809564923", 2024-09-14T06:30:27.015 INFO:teuthology.orchestra.run.smithi094.stdout: "heartbeat_back_addr": "172.21.15.94:6831/1809564923", 2024-09-14T06:30:27.015 INFO:teuthology.orchestra.run.smithi094.stdout: "heartbeat_front_addr": "172.21.15.94:6829/1809564923", 2024-09-14T06:30:27.015 INFO:teuthology.orchestra.run.smithi094.stdout: "state": [ 2024-09-14T06:30:27.016 INFO:teuthology.orchestra.run.smithi094.stdout: "exists", 2024-09-14T06:30:27.016 INFO:teuthology.orchestra.run.smithi094.stdout: "up" 2024-09-14T06:30:27.016 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:27.016 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:27.016 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:27.016 INFO:teuthology.orchestra.run.smithi094.stdout: "osd": 1, 2024-09-14T06:30:27.016 INFO:teuthology.orchestra.run.smithi094.stdout: "uuid": "02251a03-84a0-414d-a980-fac4fd043cb6", 2024-09-14T06:30:27.016 INFO:teuthology.orchestra.run.smithi094.stdout: "up": 1, 2024-09-14T06:30:27.016 INFO:teuthology.orchestra.run.smithi094.stdout: "in": 1, 2024-09-14T06:30:27.016 INFO:teuthology.orchestra.run.smithi094.stdout: "weight": 1, 2024-09-14T06:30:27.016 INFO:teuthology.orchestra.run.smithi094.stdout: "primary_affinity": 1, 2024-09-14T06:30:27.016 INFO:teuthology.orchestra.run.smithi094.stdout: "last_clean_begin": 0, 2024-09-14T06:30:27.016 INFO:teuthology.orchestra.run.smithi094.stdout: "last_clean_end": 0, 2024-09-14T06:30:27.016 INFO:teuthology.orchestra.run.smithi094.stdout: "up_from": 15, 2024-09-14T06:30:27.017 INFO:teuthology.orchestra.run.smithi094.stdout: "up_thru": 23, 2024-09-14T06:30:27.017 INFO:teuthology.orchestra.run.smithi094.stdout: "down_at": 0, 2024-09-14T06:30:27.017 INFO:teuthology.orchestra.run.smithi094.stdout: "lost_at": 0, 2024-09-14T06:30:27.017 INFO:teuthology.orchestra.run.smithi094.stdout: "public_addrs": { 2024-09-14T06:30:27.017 INFO:teuthology.orchestra.run.smithi094.stdout: "addrvec": [ 2024-09-14T06:30:27.017 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:27.017 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v2", 2024-09-14T06:30:27.017 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.94:6816", 2024-09-14T06:30:27.017 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 736402652 2024-09-14T06:30:27.017 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:27.017 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:27.017 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v1", 2024-09-14T06:30:27.017 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.94:6817", 2024-09-14T06:30:27.017 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 736402652 2024-09-14T06:30:27.018 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:27.018 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:27.018 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:27.018 INFO:teuthology.orchestra.run.smithi094.stdout: "cluster_addrs": { 2024-09-14T06:30:27.018 INFO:teuthology.orchestra.run.smithi094.stdout: "addrvec": [ 2024-09-14T06:30:27.018 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:27.018 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v2", 2024-09-14T06:30:27.018 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.94:6818", 2024-09-14T06:30:27.018 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 736402652 2024-09-14T06:30:27.018 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:27.018 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:27.018 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v1", 2024-09-14T06:30:27.018 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.94:6819", 2024-09-14T06:30:27.019 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 736402652 2024-09-14T06:30:27.019 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:27.019 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:27.019 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:27.019 INFO:teuthology.orchestra.run.smithi094.stdout: "heartbeat_back_addrs": { 2024-09-14T06:30:27.019 INFO:teuthology.orchestra.run.smithi094.stdout: "addrvec": [ 2024-09-14T06:30:27.019 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:27.019 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v2", 2024-09-14T06:30:27.019 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.94:6822", 2024-09-14T06:30:27.019 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 736402652 2024-09-14T06:30:27.019 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:27.019 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:27.019 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v1", 2024-09-14T06:30:27.019 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.94:6823", 2024-09-14T06:30:27.019 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 736402652 2024-09-14T06:30:27.020 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:27.020 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:27.020 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:27.020 INFO:teuthology.orchestra.run.smithi094.stdout: "heartbeat_front_addrs": { 2024-09-14T06:30:27.020 INFO:teuthology.orchestra.run.smithi094.stdout: "addrvec": [ 2024-09-14T06:30:27.020 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:27.020 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v2", 2024-09-14T06:30:27.020 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.94:6820", 2024-09-14T06:30:27.020 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 736402652 2024-09-14T06:30:27.020 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:27.020 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:27.020 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v1", 2024-09-14T06:30:27.020 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.94:6821", 2024-09-14T06:30:27.020 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 736402652 2024-09-14T06:30:27.021 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:27.021 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:27.021 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:27.021 INFO:teuthology.orchestra.run.smithi094.stdout: "public_addr": "172.21.15.94:6817/736402652", 2024-09-14T06:30:27.021 INFO:teuthology.orchestra.run.smithi094.stdout: "cluster_addr": "172.21.15.94:6819/736402652", 2024-09-14T06:30:27.021 INFO:teuthology.orchestra.run.smithi094.stdout: "heartbeat_back_addr": "172.21.15.94:6823/736402652", 2024-09-14T06:30:27.021 INFO:teuthology.orchestra.run.smithi094.stdout: "heartbeat_front_addr": "172.21.15.94:6821/736402652", 2024-09-14T06:30:27.021 INFO:teuthology.orchestra.run.smithi094.stdout: "state": [ 2024-09-14T06:30:27.021 INFO:teuthology.orchestra.run.smithi094.stdout: "exists", 2024-09-14T06:30:27.021 INFO:teuthology.orchestra.run.smithi094.stdout: "up" 2024-09-14T06:30:27.021 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:27.021 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:27.021 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:27.021 INFO:teuthology.orchestra.run.smithi094.stdout: "osd": 2, 2024-09-14T06:30:27.022 INFO:teuthology.orchestra.run.smithi094.stdout: "uuid": "d45bf470-bb29-4739-8294-cd122b2b4102", 2024-09-14T06:30:27.022 INFO:teuthology.orchestra.run.smithi094.stdout: "up": 1, 2024-09-14T06:30:27.022 INFO:teuthology.orchestra.run.smithi094.stdout: "in": 1, 2024-09-14T06:30:27.022 INFO:teuthology.orchestra.run.smithi094.stdout: "weight": 1, 2024-09-14T06:30:27.022 INFO:teuthology.orchestra.run.smithi094.stdout: "primary_affinity": 1, 2024-09-14T06:30:27.022 INFO:teuthology.orchestra.run.smithi094.stdout: "last_clean_begin": 0, 2024-09-14T06:30:27.022 INFO:teuthology.orchestra.run.smithi094.stdout: "last_clean_end": 0, 2024-09-14T06:30:27.022 INFO:teuthology.orchestra.run.smithi094.stdout: "up_from": 14, 2024-09-14T06:30:27.022 INFO:teuthology.orchestra.run.smithi094.stdout: "up_thru": 19, 2024-09-14T06:30:27.022 INFO:teuthology.orchestra.run.smithi094.stdout: "down_at": 0, 2024-09-14T06:30:27.022 INFO:teuthology.orchestra.run.smithi094.stdout: "lost_at": 0, 2024-09-14T06:30:27.022 INFO:teuthology.orchestra.run.smithi094.stdout: "public_addrs": { 2024-09-14T06:30:27.022 INFO:teuthology.orchestra.run.smithi094.stdout: "addrvec": [ 2024-09-14T06:30:27.022 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:27.022 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v2", 2024-09-14T06:30:27.023 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.94:6808", 2024-09-14T06:30:27.023 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 1063204782 2024-09-14T06:30:27.023 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:27.023 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:27.023 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v1", 2024-09-14T06:30:27.023 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.94:6809", 2024-09-14T06:30:27.023 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 1063204782 2024-09-14T06:30:27.023 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:27.023 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:27.023 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:27.023 INFO:teuthology.orchestra.run.smithi094.stdout: "cluster_addrs": { 2024-09-14T06:30:27.023 INFO:teuthology.orchestra.run.smithi094.stdout: "addrvec": [ 2024-09-14T06:30:27.023 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:27.023 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v2", 2024-09-14T06:30:27.024 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.94:6810", 2024-09-14T06:30:27.024 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 1063204782 2024-09-14T06:30:27.024 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:27.024 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:27.024 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v1", 2024-09-14T06:30:27.024 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.94:6811", 2024-09-14T06:30:27.024 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 1063204782 2024-09-14T06:30:27.024 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:27.024 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:27.024 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:27.024 INFO:teuthology.orchestra.run.smithi094.stdout: "heartbeat_back_addrs": { 2024-09-14T06:30:27.024 INFO:teuthology.orchestra.run.smithi094.stdout: "addrvec": [ 2024-09-14T06:30:27.024 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:27.024 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v2", 2024-09-14T06:30:27.025 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.94:6814", 2024-09-14T06:30:27.025 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 1063204782 2024-09-14T06:30:27.025 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:27.025 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:27.025 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v1", 2024-09-14T06:30:27.025 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.94:6815", 2024-09-14T06:30:27.025 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 1063204782 2024-09-14T06:30:27.025 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:27.025 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:27.025 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:27.025 INFO:teuthology.orchestra.run.smithi094.stdout: "heartbeat_front_addrs": { 2024-09-14T06:30:27.025 INFO:teuthology.orchestra.run.smithi094.stdout: "addrvec": [ 2024-09-14T06:30:27.025 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:27.025 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v2", 2024-09-14T06:30:27.026 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.94:6812", 2024-09-14T06:30:27.026 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 1063204782 2024-09-14T06:30:27.026 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:27.026 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:27.026 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v1", 2024-09-14T06:30:27.026 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.94:6813", 2024-09-14T06:30:27.026 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 1063204782 2024-09-14T06:30:27.026 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:27.026 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:27.026 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:27.026 INFO:teuthology.orchestra.run.smithi094.stdout: "public_addr": "172.21.15.94:6809/1063204782", 2024-09-14T06:30:27.026 INFO:teuthology.orchestra.run.smithi094.stdout: "cluster_addr": "172.21.15.94:6811/1063204782", 2024-09-14T06:30:27.026 INFO:teuthology.orchestra.run.smithi094.stdout: "heartbeat_back_addr": "172.21.15.94:6815/1063204782", 2024-09-14T06:30:27.026 INFO:teuthology.orchestra.run.smithi094.stdout: "heartbeat_front_addr": "172.21.15.94:6813/1063204782", 2024-09-14T06:30:27.027 INFO:teuthology.orchestra.run.smithi094.stdout: "state": [ 2024-09-14T06:30:27.027 INFO:teuthology.orchestra.run.smithi094.stdout: "exists", 2024-09-14T06:30:27.027 INFO:teuthology.orchestra.run.smithi094.stdout: "up" 2024-09-14T06:30:27.027 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:27.027 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:27.027 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:27.027 INFO:teuthology.orchestra.run.smithi094.stdout: "osd": 3, 2024-09-14T06:30:27.027 INFO:teuthology.orchestra.run.smithi094.stdout: "uuid": "456ec2d9-c438-4243-b931-335704f661eb", 2024-09-14T06:30:27.027 INFO:teuthology.orchestra.run.smithi094.stdout: "up": 1, 2024-09-14T06:30:27.027 INFO:teuthology.orchestra.run.smithi094.stdout: "in": 1, 2024-09-14T06:30:27.027 INFO:teuthology.orchestra.run.smithi094.stdout: "weight": 1, 2024-09-14T06:30:27.027 INFO:teuthology.orchestra.run.smithi094.stdout: "primary_affinity": 1, 2024-09-14T06:30:27.027 INFO:teuthology.orchestra.run.smithi094.stdout: "last_clean_begin": 0, 2024-09-14T06:30:27.027 INFO:teuthology.orchestra.run.smithi094.stdout: "last_clean_end": 0, 2024-09-14T06:30:27.028 INFO:teuthology.orchestra.run.smithi094.stdout: "up_from": 13, 2024-09-14T06:30:27.028 INFO:teuthology.orchestra.run.smithi094.stdout: "up_thru": 74, 2024-09-14T06:30:27.028 INFO:teuthology.orchestra.run.smithi094.stdout: "down_at": 0, 2024-09-14T06:30:27.028 INFO:teuthology.orchestra.run.smithi094.stdout: "lost_at": 0, 2024-09-14T06:30:27.028 INFO:teuthology.orchestra.run.smithi094.stdout: "public_addrs": { 2024-09-14T06:30:27.028 INFO:teuthology.orchestra.run.smithi094.stdout: "addrvec": [ 2024-09-14T06:30:27.028 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:27.028 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v2", 2024-09-14T06:30:27.028 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.94:6800", 2024-09-14T06:30:27.028 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 3382046754 2024-09-14T06:30:27.028 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:27.028 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:27.028 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v1", 2024-09-14T06:30:27.028 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.94:6801", 2024-09-14T06:30:27.029 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 3382046754 2024-09-14T06:30:27.029 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:27.029 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:27.029 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:27.029 INFO:teuthology.orchestra.run.smithi094.stdout: "cluster_addrs": { 2024-09-14T06:30:27.029 INFO:teuthology.orchestra.run.smithi094.stdout: "addrvec": [ 2024-09-14T06:30:27.029 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:27.029 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v2", 2024-09-14T06:30:27.029 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.94:6802", 2024-09-14T06:30:27.029 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 3382046754 2024-09-14T06:30:27.029 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:27.029 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:27.029 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v1", 2024-09-14T06:30:27.030 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.94:6803", 2024-09-14T06:30:27.030 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 3382046754 2024-09-14T06:30:27.030 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:27.030 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:27.030 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:27.030 INFO:teuthology.orchestra.run.smithi094.stdout: "heartbeat_back_addrs": { 2024-09-14T06:30:27.030 INFO:teuthology.orchestra.run.smithi094.stdout: "addrvec": [ 2024-09-14T06:30:27.030 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:27.030 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v2", 2024-09-14T06:30:27.030 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.94:6806", 2024-09-14T06:30:27.030 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 3382046754 2024-09-14T06:30:27.030 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:27.030 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:27.030 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v1", 2024-09-14T06:30:27.030 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.94:6807", 2024-09-14T06:30:27.031 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 3382046754 2024-09-14T06:30:27.031 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:27.031 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:27.031 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:27.031 INFO:teuthology.orchestra.run.smithi094.stdout: "heartbeat_front_addrs": { 2024-09-14T06:30:27.031 INFO:teuthology.orchestra.run.smithi094.stdout: "addrvec": [ 2024-09-14T06:30:27.031 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:27.031 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v2", 2024-09-14T06:30:27.031 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.94:6804", 2024-09-14T06:30:27.031 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 3382046754 2024-09-14T06:30:27.031 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:27.031 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:27.031 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v1", 2024-09-14T06:30:27.031 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.94:6805", 2024-09-14T06:30:27.032 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 3382046754 2024-09-14T06:30:27.032 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:27.032 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:27.032 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:27.032 INFO:teuthology.orchestra.run.smithi094.stdout: "public_addr": "172.21.15.94:6801/3382046754", 2024-09-14T06:30:27.032 INFO:teuthology.orchestra.run.smithi094.stdout: "cluster_addr": "172.21.15.94:6803/3382046754", 2024-09-14T06:30:27.032 INFO:teuthology.orchestra.run.smithi094.stdout: "heartbeat_back_addr": "172.21.15.94:6807/3382046754", 2024-09-14T06:30:27.032 INFO:teuthology.orchestra.run.smithi094.stdout: "heartbeat_front_addr": "172.21.15.94:6805/3382046754", 2024-09-14T06:30:27.032 INFO:teuthology.orchestra.run.smithi094.stdout: "state": [ 2024-09-14T06:30:27.032 INFO:teuthology.orchestra.run.smithi094.stdout: "exists", 2024-09-14T06:30:27.032 INFO:teuthology.orchestra.run.smithi094.stdout: "up" 2024-09-14T06:30:27.032 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:27.032 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:27.032 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:27.033 INFO:teuthology.orchestra.run.smithi094.stdout: "osd": 4, 2024-09-14T06:30:27.033 INFO:teuthology.orchestra.run.smithi094.stdout: "uuid": "2dab948e-c6e4-4437-bf17-017d1ac6821c", 2024-09-14T06:30:27.033 INFO:teuthology.orchestra.run.smithi094.stdout: "up": 1, 2024-09-14T06:30:27.033 INFO:teuthology.orchestra.run.smithi094.stdout: "in": 1, 2024-09-14T06:30:27.033 INFO:teuthology.orchestra.run.smithi094.stdout: "weight": 1, 2024-09-14T06:30:27.033 INFO:teuthology.orchestra.run.smithi094.stdout: "primary_affinity": 1, 2024-09-14T06:30:27.033 INFO:teuthology.orchestra.run.smithi094.stdout: "last_clean_begin": 0, 2024-09-14T06:30:27.033 INFO:teuthology.orchestra.run.smithi094.stdout: "last_clean_end": 0, 2024-09-14T06:30:27.033 INFO:teuthology.orchestra.run.smithi094.stdout: "up_from": 15, 2024-09-14T06:30:27.033 INFO:teuthology.orchestra.run.smithi094.stdout: "up_thru": 23, 2024-09-14T06:30:27.033 INFO:teuthology.orchestra.run.smithi094.stdout: "down_at": 0, 2024-09-14T06:30:27.033 INFO:teuthology.orchestra.run.smithi094.stdout: "lost_at": 0, 2024-09-14T06:30:27.033 INFO:teuthology.orchestra.run.smithi094.stdout: "public_addrs": { 2024-09-14T06:30:27.033 INFO:teuthology.orchestra.run.smithi094.stdout: "addrvec": [ 2024-09-14T06:30:27.034 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:27.034 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v2", 2024-09-14T06:30:27.034 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.138:6810", 2024-09-14T06:30:27.034 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 501726729 2024-09-14T06:30:27.034 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:27.034 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:27.034 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v1", 2024-09-14T06:30:27.034 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.138:6811", 2024-09-14T06:30:27.034 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 501726729 2024-09-14T06:30:27.034 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:27.034 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:27.034 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:27.034 INFO:teuthology.orchestra.run.smithi094.stdout: "cluster_addrs": { 2024-09-14T06:30:27.035 INFO:teuthology.orchestra.run.smithi094.stdout: "addrvec": [ 2024-09-14T06:30:27.035 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:27.035 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v2", 2024-09-14T06:30:27.035 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.138:6812", 2024-09-14T06:30:27.035 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 501726729 2024-09-14T06:30:27.035 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:27.035 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:27.035 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v1", 2024-09-14T06:30:27.035 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.138:6813", 2024-09-14T06:30:27.035 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 501726729 2024-09-14T06:30:27.035 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:27.035 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:27.035 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:27.035 INFO:teuthology.orchestra.run.smithi094.stdout: "heartbeat_back_addrs": { 2024-09-14T06:30:27.036 INFO:teuthology.orchestra.run.smithi094.stdout: "addrvec": [ 2024-09-14T06:30:27.036 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:27.036 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v2", 2024-09-14T06:30:27.036 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.138:6816", 2024-09-14T06:30:27.036 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 501726729 2024-09-14T06:30:27.036 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:27.036 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:27.036 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v1", 2024-09-14T06:30:27.036 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.138:6817", 2024-09-14T06:30:27.036 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 501726729 2024-09-14T06:30:27.036 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:27.036 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:27.036 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:27.036 INFO:teuthology.orchestra.run.smithi094.stdout: "heartbeat_front_addrs": { 2024-09-14T06:30:27.037 INFO:teuthology.orchestra.run.smithi094.stdout: "addrvec": [ 2024-09-14T06:30:27.037 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:27.037 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v2", 2024-09-14T06:30:27.037 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.138:6814", 2024-09-14T06:30:27.037 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 501726729 2024-09-14T06:30:27.037 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:27.037 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:27.037 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v1", 2024-09-14T06:30:27.037 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.138:6815", 2024-09-14T06:30:27.037 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 501726729 2024-09-14T06:30:27.037 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:27.037 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:27.037 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:27.037 INFO:teuthology.orchestra.run.smithi094.stdout: "public_addr": "172.21.15.138:6811/501726729", 2024-09-14T06:30:27.038 INFO:teuthology.orchestra.run.smithi094.stdout: "cluster_addr": "172.21.15.138:6813/501726729", 2024-09-14T06:30:27.038 INFO:teuthology.orchestra.run.smithi094.stdout: "heartbeat_back_addr": "172.21.15.138:6817/501726729", 2024-09-14T06:30:27.038 INFO:teuthology.orchestra.run.smithi094.stdout: "heartbeat_front_addr": "172.21.15.138:6815/501726729", 2024-09-14T06:30:27.038 INFO:teuthology.orchestra.run.smithi094.stdout: "state": [ 2024-09-14T06:30:27.038 INFO:teuthology.orchestra.run.smithi094.stdout: "exists", 2024-09-14T06:30:27.038 INFO:teuthology.orchestra.run.smithi094.stdout: "up" 2024-09-14T06:30:27.038 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:27.038 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:27.038 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:27.038 INFO:teuthology.orchestra.run.smithi094.stdout: "osd": 5, 2024-09-14T06:30:27.038 INFO:teuthology.orchestra.run.smithi094.stdout: "uuid": "8caf3aa3-ad87-4d33-b82e-4c15195176fd", 2024-09-14T06:30:27.038 INFO:teuthology.orchestra.run.smithi094.stdout: "up": 1, 2024-09-14T06:30:27.038 INFO:teuthology.orchestra.run.smithi094.stdout: "in": 1, 2024-09-14T06:30:27.038 INFO:teuthology.orchestra.run.smithi094.stdout: "weight": 1, 2024-09-14T06:30:27.039 INFO:teuthology.orchestra.run.smithi094.stdout: "primary_affinity": 1, 2024-09-14T06:30:27.039 INFO:teuthology.orchestra.run.smithi094.stdout: "last_clean_begin": 0, 2024-09-14T06:30:27.039 INFO:teuthology.orchestra.run.smithi094.stdout: "last_clean_end": 0, 2024-09-14T06:30:27.039 INFO:teuthology.orchestra.run.smithi094.stdout: "up_from": 15, 2024-09-14T06:30:27.039 INFO:teuthology.orchestra.run.smithi094.stdout: "up_thru": 23, 2024-09-14T06:30:27.039 INFO:teuthology.orchestra.run.smithi094.stdout: "down_at": 0, 2024-09-14T06:30:27.039 INFO:teuthology.orchestra.run.smithi094.stdout: "lost_at": 0, 2024-09-14T06:30:27.039 INFO:teuthology.orchestra.run.smithi094.stdout: "public_addrs": { 2024-09-14T06:30:27.039 INFO:teuthology.orchestra.run.smithi094.stdout: "addrvec": [ 2024-09-14T06:30:27.039 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:27.039 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v2", 2024-09-14T06:30:27.039 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.138:6826", 2024-09-14T06:30:27.039 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 1599292735 2024-09-14T06:30:27.039 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:27.040 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:27.040 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v1", 2024-09-14T06:30:27.040 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.138:6827", 2024-09-14T06:30:27.040 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 1599292735 2024-09-14T06:30:27.040 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:27.040 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:27.040 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:27.040 INFO:teuthology.orchestra.run.smithi094.stdout: "cluster_addrs": { 2024-09-14T06:30:27.040 INFO:teuthology.orchestra.run.smithi094.stdout: "addrvec": [ 2024-09-14T06:30:27.040 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:27.040 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v2", 2024-09-14T06:30:27.040 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.138:6828", 2024-09-14T06:30:27.040 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 1599292735 2024-09-14T06:30:27.040 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:27.041 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:27.041 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v1", 2024-09-14T06:30:27.041 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.138:6829", 2024-09-14T06:30:27.041 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 1599292735 2024-09-14T06:30:27.041 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:27.041 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:27.041 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:27.041 INFO:teuthology.orchestra.run.smithi094.stdout: "heartbeat_back_addrs": { 2024-09-14T06:30:27.041 INFO:teuthology.orchestra.run.smithi094.stdout: "addrvec": [ 2024-09-14T06:30:27.041 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:27.041 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v2", 2024-09-14T06:30:27.041 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.138:6832", 2024-09-14T06:30:27.041 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 1599292735 2024-09-14T06:30:27.042 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:27.042 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:27.042 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v1", 2024-09-14T06:30:27.042 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.138:6833", 2024-09-14T06:30:27.042 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 1599292735 2024-09-14T06:30:27.042 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:27.042 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:27.042 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:27.042 INFO:teuthology.orchestra.run.smithi094.stdout: "heartbeat_front_addrs": { 2024-09-14T06:30:27.042 INFO:teuthology.orchestra.run.smithi094.stdout: "addrvec": [ 2024-09-14T06:30:27.042 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:27.042 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v2", 2024-09-14T06:30:27.042 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.138:6830", 2024-09-14T06:30:27.042 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 1599292735 2024-09-14T06:30:27.043 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:27.043 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:27.043 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v1", 2024-09-14T06:30:27.043 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.138:6831", 2024-09-14T06:30:27.043 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 1599292735 2024-09-14T06:30:27.043 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:27.043 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:27.043 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:27.043 INFO:teuthology.orchestra.run.smithi094.stdout: "public_addr": "172.21.15.138:6827/1599292735", 2024-09-14T06:30:27.043 INFO:teuthology.orchestra.run.smithi094.stdout: "cluster_addr": "172.21.15.138:6829/1599292735", 2024-09-14T06:30:27.043 INFO:teuthology.orchestra.run.smithi094.stdout: "heartbeat_back_addr": "172.21.15.138:6833/1599292735", 2024-09-14T06:30:27.043 INFO:teuthology.orchestra.run.smithi094.stdout: "heartbeat_front_addr": "172.21.15.138:6831/1599292735", 2024-09-14T06:30:27.043 INFO:teuthology.orchestra.run.smithi094.stdout: "state": [ 2024-09-14T06:30:27.043 INFO:teuthology.orchestra.run.smithi094.stdout: "exists", 2024-09-14T06:30:27.043 INFO:teuthology.orchestra.run.smithi094.stdout: "up" 2024-09-14T06:30:27.044 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:27.044 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:27.044 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:27.044 INFO:teuthology.orchestra.run.smithi094.stdout: "osd": 6, 2024-09-14T06:30:27.044 INFO:teuthology.orchestra.run.smithi094.stdout: "uuid": "e55d11fd-433d-4462-93a7-d5e84560221b", 2024-09-14T06:30:27.044 INFO:teuthology.orchestra.run.smithi094.stdout: "up": 1, 2024-09-14T06:30:27.044 INFO:teuthology.orchestra.run.smithi094.stdout: "in": 1, 2024-09-14T06:30:27.044 INFO:teuthology.orchestra.run.smithi094.stdout: "weight": 1, 2024-09-14T06:30:27.044 INFO:teuthology.orchestra.run.smithi094.stdout: "primary_affinity": 1, 2024-09-14T06:30:27.044 INFO:teuthology.orchestra.run.smithi094.stdout: "last_clean_begin": 0, 2024-09-14T06:30:27.044 INFO:teuthology.orchestra.run.smithi094.stdout: "last_clean_end": 0, 2024-09-14T06:30:27.044 INFO:teuthology.orchestra.run.smithi094.stdout: "up_from": 15, 2024-09-14T06:30:27.044 INFO:teuthology.orchestra.run.smithi094.stdout: "up_thru": 23, 2024-09-14T06:30:27.044 INFO:teuthology.orchestra.run.smithi094.stdout: "down_at": 0, 2024-09-14T06:30:27.045 INFO:teuthology.orchestra.run.smithi094.stdout: "lost_at": 0, 2024-09-14T06:30:27.045 INFO:teuthology.orchestra.run.smithi094.stdout: "public_addrs": { 2024-09-14T06:30:27.045 INFO:teuthology.orchestra.run.smithi094.stdout: "addrvec": [ 2024-09-14T06:30:27.045 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:27.045 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v2", 2024-09-14T06:30:27.045 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.138:6818", 2024-09-14T06:30:27.045 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 163531479 2024-09-14T06:30:27.045 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:27.045 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:27.045 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v1", 2024-09-14T06:30:27.045 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.138:6819", 2024-09-14T06:30:27.045 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 163531479 2024-09-14T06:30:27.045 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:27.045 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:27.046 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:27.046 INFO:teuthology.orchestra.run.smithi094.stdout: "cluster_addrs": { 2024-09-14T06:30:27.046 INFO:teuthology.orchestra.run.smithi094.stdout: "addrvec": [ 2024-09-14T06:30:27.046 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:27.046 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v2", 2024-09-14T06:30:27.046 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.138:6820", 2024-09-14T06:30:27.046 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 163531479 2024-09-14T06:30:27.046 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:27.046 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:27.046 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v1", 2024-09-14T06:30:27.046 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.138:6821", 2024-09-14T06:30:27.046 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 163531479 2024-09-14T06:30:27.046 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:27.046 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:27.047 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:27.047 INFO:teuthology.orchestra.run.smithi094.stdout: "heartbeat_back_addrs": { 2024-09-14T06:30:27.047 INFO:teuthology.orchestra.run.smithi094.stdout: "addrvec": [ 2024-09-14T06:30:27.047 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:27.047 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v2", 2024-09-14T06:30:27.047 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.138:6824", 2024-09-14T06:30:27.047 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 163531479 2024-09-14T06:30:27.047 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:27.047 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:27.047 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v1", 2024-09-14T06:30:27.047 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.138:6825", 2024-09-14T06:30:27.047 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 163531479 2024-09-14T06:30:27.047 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:27.047 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:27.047 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:27.048 INFO:teuthology.orchestra.run.smithi094.stdout: "heartbeat_front_addrs": { 2024-09-14T06:30:27.048 INFO:teuthology.orchestra.run.smithi094.stdout: "addrvec": [ 2024-09-14T06:30:27.048 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:27.048 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v2", 2024-09-14T06:30:27.048 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.138:6822", 2024-09-14T06:30:27.048 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 163531479 2024-09-14T06:30:27.048 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:27.048 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:27.048 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v1", 2024-09-14T06:30:27.048 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.138:6823", 2024-09-14T06:30:27.048 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 163531479 2024-09-14T06:30:27.048 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:27.048 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:27.049 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:27.049 INFO:teuthology.orchestra.run.smithi094.stdout: "public_addr": "172.21.15.138:6819/163531479", 2024-09-14T06:30:27.049 INFO:teuthology.orchestra.run.smithi094.stdout: "cluster_addr": "172.21.15.138:6821/163531479", 2024-09-14T06:30:27.049 INFO:teuthology.orchestra.run.smithi094.stdout: "heartbeat_back_addr": "172.21.15.138:6825/163531479", 2024-09-14T06:30:27.049 INFO:teuthology.orchestra.run.smithi094.stdout: "heartbeat_front_addr": "172.21.15.138:6823/163531479", 2024-09-14T06:30:27.049 INFO:teuthology.orchestra.run.smithi094.stdout: "state": [ 2024-09-14T06:30:27.049 INFO:teuthology.orchestra.run.smithi094.stdout: "exists", 2024-09-14T06:30:27.049 INFO:teuthology.orchestra.run.smithi094.stdout: "up" 2024-09-14T06:30:27.049 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:27.049 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:27.049 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:27.049 INFO:teuthology.orchestra.run.smithi094.stdout: "osd": 7, 2024-09-14T06:30:27.049 INFO:teuthology.orchestra.run.smithi094.stdout: "uuid": "ba14ba69-2c6c-4d09-81e2-2593dc3a7ea9", 2024-09-14T06:30:27.049 INFO:teuthology.orchestra.run.smithi094.stdout: "up": 1, 2024-09-14T06:30:27.050 INFO:teuthology.orchestra.run.smithi094.stdout: "in": 1, 2024-09-14T06:30:27.050 INFO:teuthology.orchestra.run.smithi094.stdout: "weight": 1, 2024-09-14T06:30:27.050 INFO:teuthology.orchestra.run.smithi094.stdout: "primary_affinity": 1, 2024-09-14T06:30:27.050 INFO:teuthology.orchestra.run.smithi094.stdout: "last_clean_begin": 0, 2024-09-14T06:30:27.050 INFO:teuthology.orchestra.run.smithi094.stdout: "last_clean_end": 0, 2024-09-14T06:30:27.050 INFO:teuthology.orchestra.run.smithi094.stdout: "up_from": 15, 2024-09-14T06:30:27.050 INFO:teuthology.orchestra.run.smithi094.stdout: "up_thru": 23, 2024-09-14T06:30:27.050 INFO:teuthology.orchestra.run.smithi094.stdout: "down_at": 0, 2024-09-14T06:30:27.050 INFO:teuthology.orchestra.run.smithi094.stdout: "lost_at": 0, 2024-09-14T06:30:27.050 INFO:teuthology.orchestra.run.smithi094.stdout: "public_addrs": { 2024-09-14T06:30:27.050 INFO:teuthology.orchestra.run.smithi094.stdout: "addrvec": [ 2024-09-14T06:30:27.050 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:27.050 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v2", 2024-09-14T06:30:27.050 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.138:6800", 2024-09-14T06:30:27.051 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 218507235 2024-09-14T06:30:27.051 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:27.051 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:27.051 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v1", 2024-09-14T06:30:27.051 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.138:6801", 2024-09-14T06:30:27.051 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 218507235 2024-09-14T06:30:27.051 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:27.051 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:27.051 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:27.051 INFO:teuthology.orchestra.run.smithi094.stdout: "cluster_addrs": { 2024-09-14T06:30:27.051 INFO:teuthology.orchestra.run.smithi094.stdout: "addrvec": [ 2024-09-14T06:30:27.051 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:27.051 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v2", 2024-09-14T06:30:27.051 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.138:6802", 2024-09-14T06:30:27.051 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 218507235 2024-09-14T06:30:27.052 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:27.052 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:27.052 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v1", 2024-09-14T06:30:27.052 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.138:6803", 2024-09-14T06:30:27.052 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 218507235 2024-09-14T06:30:27.052 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:27.052 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:27.052 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:27.052 INFO:teuthology.orchestra.run.smithi094.stdout: "heartbeat_back_addrs": { 2024-09-14T06:30:27.052 INFO:teuthology.orchestra.run.smithi094.stdout: "addrvec": [ 2024-09-14T06:30:27.052 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:27.052 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v2", 2024-09-14T06:30:27.052 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.138:6806", 2024-09-14T06:30:27.052 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 218507235 2024-09-14T06:30:27.053 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:27.053 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:27.053 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v1", 2024-09-14T06:30:27.053 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.138:6807", 2024-09-14T06:30:27.053 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 218507235 2024-09-14T06:30:27.053 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:27.053 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:27.053 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:27.053 INFO:teuthology.orchestra.run.smithi094.stdout: "heartbeat_front_addrs": { 2024-09-14T06:30:27.053 INFO:teuthology.orchestra.run.smithi094.stdout: "addrvec": [ 2024-09-14T06:30:27.053 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:27.053 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v2", 2024-09-14T06:30:27.053 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.138:6804", 2024-09-14T06:30:27.053 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 218507235 2024-09-14T06:30:27.053 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:27.054 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:27.054 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v1", 2024-09-14T06:30:27.054 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.138:6805", 2024-09-14T06:30:27.054 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 218507235 2024-09-14T06:30:27.054 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:27.054 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:27.054 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:27.054 INFO:teuthology.orchestra.run.smithi094.stdout: "public_addr": "172.21.15.138:6801/218507235", 2024-09-14T06:30:27.054 INFO:teuthology.orchestra.run.smithi094.stdout: "cluster_addr": "172.21.15.138:6803/218507235", 2024-09-14T06:30:27.054 INFO:teuthology.orchestra.run.smithi094.stdout: "heartbeat_back_addr": "172.21.15.138:6807/218507235", 2024-09-14T06:30:27.054 INFO:teuthology.orchestra.run.smithi094.stdout: "heartbeat_front_addr": "172.21.15.138:6805/218507235", 2024-09-14T06:30:27.054 INFO:teuthology.orchestra.run.smithi094.stdout: "state": [ 2024-09-14T06:30:27.054 INFO:teuthology.orchestra.run.smithi094.stdout: "exists", 2024-09-14T06:30:27.054 INFO:teuthology.orchestra.run.smithi094.stdout: "up" 2024-09-14T06:30:27.055 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:27.055 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:27.055 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-09-14T06:30:27.055 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_xinfo": [ 2024-09-14T06:30:27.055 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:27.055 INFO:teuthology.orchestra.run.smithi094.stdout: "osd": 0, 2024-09-14T06:30:27.055 INFO:teuthology.orchestra.run.smithi094.stdout: "down_stamp": "0.000000", 2024-09-14T06:30:27.055 INFO:teuthology.orchestra.run.smithi094.stdout: "laggy_probability": 0, 2024-09-14T06:30:27.055 INFO:teuthology.orchestra.run.smithi094.stdout: "laggy_interval": 0, 2024-09-14T06:30:27.055 INFO:teuthology.orchestra.run.smithi094.stdout: "features": 4540719139924082687, 2024-09-14T06:30:27.055 INFO:teuthology.orchestra.run.smithi094.stdout: "old_weight": 0, 2024-09-14T06:30:27.055 INFO:teuthology.orchestra.run.smithi094.stdout: "last_purged_snaps_scrub": "2024-09-14T06:28:39.906639+0000", 2024-09-14T06:30:27.055 INFO:teuthology.orchestra.run.smithi094.stdout: "dead_epoch": 0 2024-09-14T06:30:27.055 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:27.055 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:27.056 INFO:teuthology.orchestra.run.smithi094.stdout: "osd": 1, 2024-09-14T06:30:27.056 INFO:teuthology.orchestra.run.smithi094.stdout: "down_stamp": "0.000000", 2024-09-14T06:30:27.056 INFO:teuthology.orchestra.run.smithi094.stdout: "laggy_probability": 0, 2024-09-14T06:30:27.056 INFO:teuthology.orchestra.run.smithi094.stdout: "laggy_interval": 0, 2024-09-14T06:30:27.056 INFO:teuthology.orchestra.run.smithi094.stdout: "features": 4540719139924082687, 2024-09-14T06:30:27.056 INFO:teuthology.orchestra.run.smithi094.stdout: "old_weight": 0, 2024-09-14T06:30:27.056 INFO:teuthology.orchestra.run.smithi094.stdout: "last_purged_snaps_scrub": "2024-09-14T06:28:40.541279+0000", 2024-09-14T06:30:27.056 INFO:teuthology.orchestra.run.smithi094.stdout: "dead_epoch": 0 2024-09-14T06:30:27.056 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:27.056 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:27.056 INFO:teuthology.orchestra.run.smithi094.stdout: "osd": 2, 2024-09-14T06:30:27.056 INFO:teuthology.orchestra.run.smithi094.stdout: "down_stamp": "0.000000", 2024-09-14T06:30:27.056 INFO:teuthology.orchestra.run.smithi094.stdout: "laggy_probability": 0, 2024-09-14T06:30:27.056 INFO:teuthology.orchestra.run.smithi094.stdout: "laggy_interval": 0, 2024-09-14T06:30:27.057 INFO:teuthology.orchestra.run.smithi094.stdout: "features": 4540719139924082687, 2024-09-14T06:30:27.057 INFO:teuthology.orchestra.run.smithi094.stdout: "old_weight": 0, 2024-09-14T06:30:27.057 INFO:teuthology.orchestra.run.smithi094.stdout: "last_purged_snaps_scrub": "2024-09-14T06:28:40.025350+0000", 2024-09-14T06:30:27.057 INFO:teuthology.orchestra.run.smithi094.stdout: "dead_epoch": 0 2024-09-14T06:30:27.057 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:27.057 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:27.057 INFO:teuthology.orchestra.run.smithi094.stdout: "osd": 3, 2024-09-14T06:30:27.057 INFO:teuthology.orchestra.run.smithi094.stdout: "down_stamp": "0.000000", 2024-09-14T06:30:27.057 INFO:teuthology.orchestra.run.smithi094.stdout: "laggy_probability": 0, 2024-09-14T06:30:27.057 INFO:teuthology.orchestra.run.smithi094.stdout: "laggy_interval": 0, 2024-09-14T06:30:27.057 INFO:teuthology.orchestra.run.smithi094.stdout: "features": 4540719139924082687, 2024-09-14T06:30:27.057 INFO:teuthology.orchestra.run.smithi094.stdout: "old_weight": 0, 2024-09-14T06:30:27.057 INFO:teuthology.orchestra.run.smithi094.stdout: "last_purged_snaps_scrub": "2024-09-14T06:28:39.217944+0000", 2024-09-14T06:30:27.058 INFO:teuthology.orchestra.run.smithi094.stdout: "dead_epoch": 0 2024-09-14T06:30:27.058 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:27.058 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:27.058 INFO:teuthology.orchestra.run.smithi094.stdout: "osd": 4, 2024-09-14T06:30:27.058 INFO:teuthology.orchestra.run.smithi094.stdout: "down_stamp": "0.000000", 2024-09-14T06:30:27.058 INFO:teuthology.orchestra.run.smithi094.stdout: "laggy_probability": 0, 2024-09-14T06:30:27.058 INFO:teuthology.orchestra.run.smithi094.stdout: "laggy_interval": 0, 2024-09-14T06:30:27.058 INFO:teuthology.orchestra.run.smithi094.stdout: "features": 4540719139924082687, 2024-09-14T06:30:27.058 INFO:teuthology.orchestra.run.smithi094.stdout: "old_weight": 0, 2024-09-14T06:30:27.058 INFO:teuthology.orchestra.run.smithi094.stdout: "last_purged_snaps_scrub": "2024-09-14T06:28:40.185898+0000", 2024-09-14T06:30:27.058 INFO:teuthology.orchestra.run.smithi094.stdout: "dead_epoch": 0 2024-09-14T06:30:27.058 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:27.058 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:27.058 INFO:teuthology.orchestra.run.smithi094.stdout: "osd": 5, 2024-09-14T06:30:27.058 INFO:teuthology.orchestra.run.smithi094.stdout: "down_stamp": "0.000000", 2024-09-14T06:30:27.059 INFO:teuthology.orchestra.run.smithi094.stdout: "laggy_probability": 0, 2024-09-14T06:30:27.059 INFO:teuthology.orchestra.run.smithi094.stdout: "laggy_interval": 0, 2024-09-14T06:30:27.059 INFO:teuthology.orchestra.run.smithi094.stdout: "features": 4540719139924082687, 2024-09-14T06:30:27.059 INFO:teuthology.orchestra.run.smithi094.stdout: "old_weight": 0, 2024-09-14T06:30:27.059 INFO:teuthology.orchestra.run.smithi094.stdout: "last_purged_snaps_scrub": "2024-09-14T06:28:40.783504+0000", 2024-09-14T06:30:27.059 INFO:teuthology.orchestra.run.smithi094.stdout: "dead_epoch": 0 2024-09-14T06:30:27.059 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:27.059 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:27.059 INFO:teuthology.orchestra.run.smithi094.stdout: "osd": 6, 2024-09-14T06:30:27.059 INFO:teuthology.orchestra.run.smithi094.stdout: "down_stamp": "0.000000", 2024-09-14T06:30:27.059 INFO:teuthology.orchestra.run.smithi094.stdout: "laggy_probability": 0, 2024-09-14T06:30:27.059 INFO:teuthology.orchestra.run.smithi094.stdout: "laggy_interval": 0, 2024-09-14T06:30:27.059 INFO:teuthology.orchestra.run.smithi094.stdout: "features": 4540719139924082687, 2024-09-14T06:30:27.059 INFO:teuthology.orchestra.run.smithi094.stdout: "old_weight": 0, 2024-09-14T06:30:27.059 INFO:teuthology.orchestra.run.smithi094.stdout: "last_purged_snaps_scrub": "2024-09-14T06:28:40.527305+0000", 2024-09-14T06:30:27.060 INFO:teuthology.orchestra.run.smithi094.stdout: "dead_epoch": 0 2024-09-14T06:30:27.060 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:27.060 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:27.060 INFO:teuthology.orchestra.run.smithi094.stdout: "osd": 7, 2024-09-14T06:30:27.060 INFO:teuthology.orchestra.run.smithi094.stdout: "down_stamp": "0.000000", 2024-09-14T06:30:27.060 INFO:teuthology.orchestra.run.smithi094.stdout: "laggy_probability": 0, 2024-09-14T06:30:27.060 INFO:teuthology.orchestra.run.smithi094.stdout: "laggy_interval": 0, 2024-09-14T06:30:27.060 INFO:teuthology.orchestra.run.smithi094.stdout: "features": 4540719139924082687, 2024-09-14T06:30:27.060 INFO:teuthology.orchestra.run.smithi094.stdout: "old_weight": 0, 2024-09-14T06:30:27.060 INFO:teuthology.orchestra.run.smithi094.stdout: "last_purged_snaps_scrub": "2024-09-14T06:28:40.349676+0000", 2024-09-14T06:30:27.060 INFO:teuthology.orchestra.run.smithi094.stdout: "dead_epoch": 0 2024-09-14T06:30:27.060 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:27.060 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-09-14T06:30:27.060 INFO:teuthology.orchestra.run.smithi094.stdout: "pg_upmap": [], 2024-09-14T06:30:27.061 INFO:teuthology.orchestra.run.smithi094.stdout: "pg_upmap_items": [ 2024-09-14T06:30:27.061 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:27.061 INFO:teuthology.orchestra.run.smithi094.stdout: "pgid": "3.7", 2024-09-14T06:30:27.061 INFO:teuthology.orchestra.run.smithi094.stdout: "mappings": [ 2024-09-14T06:30:27.061 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:27.061 INFO:teuthology.orchestra.run.smithi094.stdout: "from": 3, 2024-09-14T06:30:27.061 INFO:teuthology.orchestra.run.smithi094.stdout: "to": 0 2024-09-14T06:30:27.061 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:27.061 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:27.061 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:27.061 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-09-14T06:30:27.061 INFO:teuthology.orchestra.run.smithi094.stdout: "pg_upmap_primaries": [], 2024-09-14T06:30:27.061 INFO:teuthology.orchestra.run.smithi094.stdout: "pg_temp": [], 2024-09-14T06:30:27.061 INFO:teuthology.orchestra.run.smithi094.stdout: "primary_temp": [], 2024-09-14T06:30:27.062 INFO:teuthology.orchestra.run.smithi094.stdout: "blocklist": {}, 2024-09-14T06:30:27.062 INFO:teuthology.orchestra.run.smithi094.stdout: "range_blocklist": {}, 2024-09-14T06:30:27.062 INFO:teuthology.orchestra.run.smithi094.stdout: "erasure_code_profiles": { 2024-09-14T06:30:27.062 INFO:teuthology.orchestra.run.smithi094.stdout: "default": { 2024-09-14T06:30:27.062 INFO:teuthology.orchestra.run.smithi094.stdout: "crush-failure-domain": "osd", 2024-09-14T06:30:27.062 INFO:teuthology.orchestra.run.smithi094.stdout: "k": "2", 2024-09-14T06:30:27.062 INFO:teuthology.orchestra.run.smithi094.stdout: "m": "1", 2024-09-14T06:30:27.062 INFO:teuthology.orchestra.run.smithi094.stdout: "plugin": "jerasure", 2024-09-14T06:30:27.062 INFO:teuthology.orchestra.run.smithi094.stdout: "technique": "reed_sol_van" 2024-09-14T06:30:27.062 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:27.062 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:27.062 INFO:teuthology.orchestra.run.smithi094.stdout: "removed_snaps_queue": [ 2024-09-14T06:30:27.062 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:27.062 INFO:teuthology.orchestra.run.smithi094.stdout: "pool": 3, 2024-09-14T06:30:27.063 INFO:teuthology.orchestra.run.smithi094.stdout: "snaps": [ 2024-09-14T06:30:27.063 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:27.063 INFO:teuthology.orchestra.run.smithi094.stdout: "begin": 42, 2024-09-14T06:30:27.063 INFO:teuthology.orchestra.run.smithi094.stdout: "length": 1 2024-09-14T06:30:27.063 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:27.063 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:27.063 INFO:teuthology.orchestra.run.smithi094.stdout: "begin": 44, 2024-09-14T06:30:27.063 INFO:teuthology.orchestra.run.smithi094.stdout: "length": 1 2024-09-14T06:30:27.063 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:27.063 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:27.063 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:27.063 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-09-14T06:30:27.063 INFO:teuthology.orchestra.run.smithi094.stdout: "new_removed_snaps": [], 2024-09-14T06:30:27.063 INFO:teuthology.orchestra.run.smithi094.stdout: "new_purged_snaps": [], 2024-09-14T06:30:27.064 INFO:teuthology.orchestra.run.smithi094.stdout: "crush_node_flags": {}, 2024-09-14T06:30:27.064 INFO:teuthology.orchestra.run.smithi094.stdout: "device_class_flags": {}, 2024-09-14T06:30:27.064 INFO:teuthology.orchestra.run.smithi094.stdout: "stretch_mode": { 2024-09-14T06:30:27.064 INFO:teuthology.orchestra.run.smithi094.stdout: "stretch_mode_enabled": false, 2024-09-14T06:30:27.064 INFO:teuthology.orchestra.run.smithi094.stdout: "stretch_bucket_count": 0, 2024-09-14T06:30:27.064 INFO:teuthology.orchestra.run.smithi094.stdout: "degraded_stretch_mode": 0, 2024-09-14T06:30:27.064 INFO:teuthology.orchestra.run.smithi094.stdout: "recovering_stretch_mode": 0, 2024-09-14T06:30:27.064 INFO:teuthology.orchestra.run.smithi094.stdout: "stretch_mode_bucket": 0 2024-09-14T06:30:27.064 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:27.064 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-14T06:30:27.065 INFO:tasks.thrashosds.thrasher:j is {'epoch': 104, 'fsid': '200ad3da-bd63-4e04-af74-efb5ca0af75e', 'created': '2024-09-14T06:28:30.782046+0000', 'modified': '2024-09-14T06:30:25.616275+0000', 'last_up_change': '2024-09-14T06:28:42.827239+0000', 'last_in_change': '2024-09-14T06:28:33.724176+0000', 'flags': 'noscrub,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit', 'flags_num': 5801984, 'flags_set': ['noscrub', 'pglog_hardlimit', 'purged_snapdirs', 'recovery_deletes', 'sortbitwise'], 'crush_version': 5, 'full_ratio': 0.949999988079071, 'backfillfull_ratio': 0.8999999761581421, 'nearfull_ratio': 0.8500000238418579, 'cluster_snapshot': '', 'pool_max': 3, 'max_osd': 8, 'require_min_compat_client': 'luminous', 'min_compat_client': 'luminous', 'require_osd_release': 'squid', 'allow_crimson': False, 'pools': [{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-14T06:28:43.295990+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2024-09-14T06:28:49.451714+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 2, 'snap_epoch': 22, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.9900000095367432, 'score_stable': 1.9900000095367432, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'pool_client.0', 'create_time': '2024-09-14T06:28:59.688535+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '104', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 46, 'snap_epoch': 104, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}], 'osds': [{'osd': 0, 'uuid': '8b22a249-b842-406a-a02c-c0f31f57b635', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 75, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.94:6824', 'nonce': 1809564923}, {'type': 'v1', 'addr': '172.21.15.94:6825', 'nonce': 1809564923}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.94:6826', 'nonce': 1809564923}, {'type': 'v1', 'addr': '172.21.15.94:6827', 'nonce': 1809564923}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.94:6830', 'nonce': 1809564923}, {'type': 'v1', 'addr': '172.21.15.94:6831', 'nonce': 1809564923}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.94:6828', 'nonce': 1809564923}, {'type': 'v1', 'addr': '172.21.15.94:6829', 'nonce': 1809564923}]}, 'public_addr': '172.21.15.94:6825/1809564923', 'cluster_addr': '172.21.15.94:6827/1809564923', 'heartbeat_back_addr': '172.21.15.94:6831/1809564923', 'heartbeat_front_addr': '172.21.15.94:6829/1809564923', 'state': ['exists', 'up']}, {'osd': 1, 'uuid': '02251a03-84a0-414d-a980-fac4fd043cb6', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 23, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.94:6816', 'nonce': 736402652}, {'type': 'v1', 'addr': '172.21.15.94:6817', 'nonce': 736402652}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.94:6818', 'nonce': 736402652}, {'type': 'v1', 'addr': '172.21.15.94:6819', 'nonce': 736402652}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.94:6822', 'nonce': 736402652}, {'type': 'v1', 'addr': '172.21.15.94:6823', 'nonce': 736402652}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.94:6820', 'nonce': 736402652}, {'type': 'v1', 'addr': '172.21.15.94:6821', 'nonce': 736402652}]}, 'public_addr': '172.21.15.94:6817/736402652', 'cluster_addr': '172.21.15.94:6819/736402652', 'heartbeat_back_addr': '172.21.15.94:6823/736402652', 'heartbeat_front_addr': '172.21.15.94:6821/736402652', 'state': ['exists', 'up']}, {'osd': 2, 'uuid': 'd45bf470-bb29-4739-8294-cd122b2b4102', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 19, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.94:6808', 'nonce': 1063204782}, {'type': 'v1', 'addr': '172.21.15.94:6809', 'nonce': 1063204782}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.94:6810', 'nonce': 1063204782}, {'type': 'v1', 'addr': '172.21.15.94:6811', 'nonce': 1063204782}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.94:6814', 'nonce': 1063204782}, {'type': 'v1', 'addr': '172.21.15.94:6815', 'nonce': 1063204782}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.94:6812', 'nonce': 1063204782}, {'type': 'v1', 'addr': '172.21.15.94:6813', 'nonce': 1063204782}]}, 'public_addr': '172.21.15.94:6809/1063204782', 'cluster_addr': '172.21.15.94:6811/1063204782', 'heartbeat_back_addr': '172.21.15.94:6815/1063204782', 'heartbeat_front_addr': '172.21.15.94:6813/1063204782', 'state': ['exists', 'up']}, {'osd': 3, 'uuid': '456ec2d9-c438-4243-b931-335704f661eb', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 74, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.94:6800', 'nonce': 3382046754}, {'type': 'v1', 'addr': '172.21.15.94:6801', 'nonce': 3382046754}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.94:6802', 'nonce': 3382046754}, {'type': 'v1', 'addr': '172.21.15.94:6803', 'nonce': 3382046754}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.94:6806', 'nonce': 3382046754}, {'type': 'v1', 'addr': '172.21.15.94:6807', 'nonce': 3382046754}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.94:6804', 'nonce': 3382046754}, {'type': 'v1', 'addr': '172.21.15.94:6805', 'nonce': 3382046754}]}, 'public_addr': '172.21.15.94:6801/3382046754', 'cluster_addr': '172.21.15.94:6803/3382046754', 'heartbeat_back_addr': '172.21.15.94:6807/3382046754', 'heartbeat_front_addr': '172.21.15.94:6805/3382046754', 'state': ['exists', 'up']}, {'osd': 4, 'uuid': '2dab948e-c6e4-4437-bf17-017d1ac6821c', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 23, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.138:6810', 'nonce': 501726729}, {'type': 'v1', 'addr': '172.21.15.138:6811', 'nonce': 501726729}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.138:6812', 'nonce': 501726729}, {'type': 'v1', 'addr': '172.21.15.138:6813', 'nonce': 501726729}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.138:6816', 'nonce': 501726729}, {'type': 'v1', 'addr': '172.21.15.138:6817', 'nonce': 501726729}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.138:6814', 'nonce': 501726729}, {'type': 'v1', 'addr': '172.21.15.138:6815', 'nonce': 501726729}]}, 'public_addr': '172.21.15.138:6811/501726729', 'cluster_addr': '172.21.15.138:6813/501726729', 'heartbeat_back_addr': '172.21.15.138:6817/501726729', 'heartbeat_front_addr': '172.21.15.138:6815/501726729', 'state': ['exists', 'up']}, {'osd': 5, 'uuid': '8caf3aa3-ad87-4d33-b82e-4c15195176fd', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 23, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.138:6826', 'nonce': 1599292735}, {'type': 'v1', 'addr': '172.21.15.138:6827', 'nonce': 1599292735}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.138:6828', 'nonce': 1599292735}, {'type': 'v1', 'addr': '172.21.15.138:6829', 'nonce': 1599292735}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.138:6832', 'nonce': 1599292735}, {'type': 'v1', 'addr': '172.21.15.138:6833', 'nonce': 1599292735}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.138:6830', 'nonce': 1599292735}, {'type': 'v1', 'addr': '172.21.15.138:6831', 'nonce': 1599292735}]}, 'public_addr': '172.21.15.138:6827/1599292735', 'cluster_addr': '172.21.15.138:6829/1599292735', 'heartbeat_back_addr': '172.21.15.138:6833/1599292735', 'heartbeat_front_addr': '172.21.15.138:6831/1599292735', 'state': ['exists', 'up']}, {'osd': 6, 'uuid': 'e55d11fd-433d-4462-93a7-d5e84560221b', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 23, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.138:6818', 'nonce': 163531479}, {'type': 'v1', 'addr': '172.21.15.138:6819', 'nonce': 163531479}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.138:6820', 'nonce': 163531479}, {'type': 'v1', 'addr': '172.21.15.138:6821', 'nonce': 163531479}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.138:6824', 'nonce': 163531479}, {'type': 'v1', 'addr': '172.21.15.138:6825', 'nonce': 163531479}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.138:6822', 'nonce': 163531479}, {'type': 'v1', 'addr': '172.21.15.138:6823', 'nonce': 163531479}]}, 'public_addr': '172.21.15.138:6819/163531479', 'cluster_addr': '172.21.15.138:6821/163531479', 'heartbeat_back_addr': '172.21.15.138:6825/163531479', 'heartbeat_front_addr': '172.21.15.138:6823/163531479', 'state': ['exists', 'up']}, {'osd': 7, 'uuid': 'ba14ba69-2c6c-4d09-81e2-2593dc3a7ea9', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 23, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.138:6800', 'nonce': 218507235}, {'type': 'v1', 'addr': '172.21.15.138:6801', 'nonce': 218507235}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.138:6802', 'nonce': 218507235}, {'type': 'v1', 'addr': '172.21.15.138:6803', 'nonce': 218507235}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.138:6806', 'nonce': 218507235}, {'type': 'v1', 'addr': '172.21.15.138:6807', 'nonce': 218507235}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.138:6804', 'nonce': 218507235}, {'type': 'v1', 'addr': '172.21.15.138:6805', 'nonce': 218507235}]}, 'public_addr': '172.21.15.138:6801/218507235', 'cluster_addr': '172.21.15.138:6803/218507235', 'heartbeat_back_addr': '172.21.15.138:6807/218507235', 'heartbeat_front_addr': '172.21.15.138:6805/218507235', 'state': ['exists', 'up']}], 'osd_xinfo': [{'osd': 0, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-14T06:28:39.906639+0000', 'dead_epoch': 0}, {'osd': 1, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-14T06:28:40.541279+0000', 'dead_epoch': 0}, {'osd': 2, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-14T06:28:40.025350+0000', 'dead_epoch': 0}, {'osd': 3, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-14T06:28:39.217944+0000', 'dead_epoch': 0}, {'osd': 4, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-14T06:28:40.185898+0000', 'dead_epoch': 0}, {'osd': 5, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-14T06:28:40.783504+0000', 'dead_epoch': 0}, {'osd': 6, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-14T06:28:40.527305+0000', 'dead_epoch': 0}, {'osd': 7, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-14T06:28:40.349676+0000', 'dead_epoch': 0}], 'pg_upmap': [], 'pg_upmap_items': [{'pgid': '3.7', 'mappings': [{'from': 3, 'to': 0}]}], 'pg_upmap_primaries': [], 'pg_temp': [], 'primary_temp': [], 'blocklist': {}, 'range_blocklist': {}, 'erasure_code_profiles': {'default': {'crush-failure-domain': 'osd', 'k': '2', 'm': '1', 'plugin': 'jerasure', 'technique': 'reed_sol_van'}}, 'removed_snaps_queue': [{'pool': 3, 'snaps': [{'begin': 42, 'length': 1}, {'begin': 44, 'length': 1}]}], 'new_removed_snaps': [], 'new_purged_snaps': [], 'crush_node_flags': {}, 'device_class_flags': {}, 'stretch_mode': {'stretch_mode_enabled': False, 'stretch_bucket_count': 0, 'degraded_stretch_mode': 0, 'recovering_stretch_mode': 0, 'stretch_mode_bucket': 0}} 2024-09-14T06:30:27.065 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-14T06:30:27.066 INFO:tasks.ceph.osd.1.smithi094.stderr:2024-09-14T06:30:26.987+0000 7f328e47a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33641) UID: 0 2024-09-14T06:30:27.110 INFO:tasks.ceph.osd.4.smithi138.stderr:2024-09-14T06:30:27.109+0000 7f6c653c8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33615) UID: 0 2024-09-14T06:30:27.173 INFO:teuthology.orchestra.run.smithi138.stdout:{ 2024-09-14T06:30:27.173 INFO:teuthology.orchestra.run.smithi138.stdout: "ops": [], 2024-09-14T06:30:27.173 INFO:teuthology.orchestra.run.smithi138.stdout: "num_ops": 0 2024-09-14T06:30:27.173 INFO:teuthology.orchestra.run.smithi138.stdout:} 2024-09-14T06:30:27.182 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-14T06:30:27.223 INFO:tasks.ceph.osd.4.smithi138.stderr:2024-09-14T06:30:27.222+0000 7f6c653c8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33615) UID: 0 2024-09-14T06:30:27.289 INFO:tasks.ceph.osd.2.smithi094.stderr:2024-09-14T06:30:27.288+0000 7faf3269b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33649) UID: 0 2024-09-14T06:30:27.325 INFO:teuthology.orchestra.run.smithi138.stdout:{ 2024-09-14T06:30:27.325 INFO:teuthology.orchestra.run.smithi138.stdout: "ops": [], 2024-09-14T06:30:27.325 INFO:teuthology.orchestra.run.smithi138.stdout: "complaint_time": 30, 2024-09-14T06:30:27.325 INFO:teuthology.orchestra.run.smithi138.stdout: "num_blocked_ops": 0 2024-09-14T06:30:27.325 INFO:teuthology.orchestra.run.smithi138.stdout:} 2024-09-14T06:30:27.334 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-14T06:30:27.350 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-14T06:30:27.350 INFO:teuthology.orchestra.run.smithi094.stderr:dumped all 2024-09-14T06:30:27.362 INFO:teuthology.orchestra.run.smithi094.stdout:{"pg_ready":true,"pg_map":{"version":151,"stamp":"2024-09-14T06:30:27.298775+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1079485897,"num_objects":1279,"num_object_clones":0,"num_object_copies":2558,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1279,"num_whiteouts":0,"num_read":126570,"num_read_kb":1169687,"num_write":15065,"num_write_kb":813083,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":39,"num_bytes_recovered":1400964,"num_keys_recovered":11,"num_objects_omap":24,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":11172,"ondisk_log_size":11172,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":417620,"kb_used_data":80684,"kb_used_omap":12,"kb_used_meta":336755,"kb_avail":754557100,"statfs":{"total":773094113280,"available":772666470400,"internally_reserved":0,"allocated":82620416,"data_stored":718909765,"data_compressed":43902115,"data_compressed_allocated":57860096,"data_compressed_original":702454300,"omap_allocated":12751,"internal_metadata":344837681},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-73078486,"num_objects":-167,"num_object_clones":0,"num_object_copies":-334,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-167,"num_whiteouts":0,"num_read":4128,"num_read_kb":45712,"num_write":851,"num_write_kb":29570,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.692944"},"pg_stats":[{"pgid":"3.1f","version":"101'334","reported_seq":3778,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-14T06:30:22.776159+0000","last_change":"2024-09-14T06:30:22.609542+0000","last_active":"2024-09-14T06:30:22.776159+0000","last_peered":"2024-09-14T06:30:22.776159+0000","last_clean":"2024-09-14T06:30:22.776159+0000","last_became_active":"2024-09-14T06:29:00.579883+0000","last_became_peered":"2024-09-14T06:29:00.579883+0000","last_unstale":"2024-09-14T06:30:22.776159+0000","last_undegraded":"2024-09-14T06:30:22.776159+0000","last_fullsized":"2024-09-14T06:30:22.776159+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":334,"log_dups_size":0,"ondisk_log_size":334,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T18:04:30.063494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070539599999999995,"stat_sum":{"num_bytes":34617406,"num_objects":37,"num_object_clones":0,"num_object_copies":74,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":37,"num_whiteouts":0,"num_read":4250,"num_read_kb":37880,"num_write":488,"num_write_kb":19418,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"2a","length":"1"},{"start":"2c","length":"1"}]},{"pgid":"3.1e","version":"103'286","reported_seq":4916,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-14T06:30:25.625891+0000","last_change":"2024-09-14T06:30:22.620042+0000","last_active":"2024-09-14T06:30:25.625891+0000","last_peered":"2024-09-14T06:30:25.625891+0000","last_clean":"2024-09-14T06:30:25.625891+0000","last_became_active":"2024-09-14T06:29:00.580507+0000","last_became_peered":"2024-09-14T06:29:00.580507+0000","last_unstale":"2024-09-14T06:30:25.625891+0000","last_undegraded":"2024-09-14T06:30:25.625891+0000","last_fullsized":"2024-09-14T06:30:25.625891+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":286,"log_dups_size":0,"ondisk_log_size":286,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T08:24:17.527821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011926829999999999,"stat_sum":{"num_bytes":28440908,"num_objects":33,"num_object_clones":0,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":33,"num_whiteouts":0,"num_read":4697,"num_read_kb":34742,"num_write":343,"num_write_kb":24916,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2a","length":"1"},{"start":"2c","length":"1"}]},{"pgid":"3.1d","version":"98'475","reported_seq":4963,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-14T06:30:23.623900+0000","last_change":"2024-09-14T06:30:22.612583+0000","last_active":"2024-09-14T06:30:23.623900+0000","last_peered":"2024-09-14T06:30:23.623900+0000","last_clean":"2024-09-14T06:30:23.623900+0000","last_became_active":"2024-09-14T06:29:00.579345+0000","last_became_peered":"2024-09-14T06:29:00.579345+0000","last_unstale":"2024-09-14T06:30:23.623900+0000","last_undegraded":"2024-09-14T06:30:23.623900+0000","last_fullsized":"2024-09-14T06:30:23.623900+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":475,"log_dups_size":0,"ondisk_log_size":475,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T15:03:36.602483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014172709999999999,"stat_sum":{"num_bytes":42985469,"num_objects":38,"num_object_clones":0,"num_object_copies":76,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":38,"num_whiteouts":0,"num_read":5627,"num_read_kb":37020,"num_write":717,"num_write_kb":24288,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2a","length":"1"},{"start":"2c","length":"1"}]},{"pgid":"3.1c","version":"99'275","reported_seq":2557,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-14T06:30:23.623753+0000","last_change":"2024-09-14T06:30:22.611576+0000","last_active":"2024-09-14T06:30:23.623753+0000","last_peered":"2024-09-14T06:30:23.623753+0000","last_clean":"2024-09-14T06:30:23.623753+0000","last_became_active":"2024-09-14T06:29:00.579807+0000","last_became_peered":"2024-09-14T06:29:00.579807+0000","last_unstale":"2024-09-14T06:30:23.623753+0000","last_undegraded":"2024-09-14T06:30:23.623753+0000","last_fullsized":"2024-09-14T06:30:23.623753+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":275,"log_dups_size":0,"ondisk_log_size":275,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T17:01:10.314780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00044109299999999999,"stat_sum":{"num_bytes":17988923,"num_objects":45,"num_object_clones":0,"num_object_copies":90,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":45,"num_whiteouts":0,"num_read":2531,"num_read_kb":28395,"num_write":335,"num_write_kb":27771,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2a","length":"1"},{"start":"2c","length":"1"}]},{"pgid":"3.1b","version":"100'146","reported_seq":2624,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-14T06:30:22.609118+0000","last_change":"2024-09-14T06:30:22.609284+0000","last_active":"2024-09-14T06:30:22.609118+0000","last_peered":"2024-09-14T06:30:22.609118+0000","last_clean":"2024-09-14T06:30:22.609118+0000","last_became_active":"2024-09-14T06:29:00.586183+0000","last_became_peered":"2024-09-14T06:29:00.586183+0000","last_unstale":"2024-09-14T06:30:22.609118+0000","last_undegraded":"2024-09-14T06:30:22.609118+0000","last_fullsized":"2024-09-14T06:30:22.609118+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":146,"log_dups_size":0,"ondisk_log_size":146,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T16:39:32.847224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069743099999999998,"stat_sum":{"num_bytes":28753401,"num_objects":51,"num_object_clones":0,"num_object_copies":102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":51,"num_whiteouts":0,"num_read":2165,"num_read_kb":28869,"num_write":165,"num_write_kb":31023,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"2a","length":"1"},{"start":"2c","length":"1"}]},{"pgid":"3.1a","version":"101'861","reported_seq":5350,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-14T06:30:23.612816+0000","last_change":"2024-09-14T06:30:22.607748+0000","last_active":"2024-09-14T06:30:23.612816+0000","last_peered":"2024-09-14T06:30:23.612816+0000","last_clean":"2024-09-14T06:30:23.612816+0000","last_became_active":"2024-09-14T06:29:00.576206+0000","last_became_peered":"2024-09-14T06:29:00.576206+0000","last_unstale":"2024-09-14T06:30:23.612816+0000","last_undegraded":"2024-09-14T06:30:23.612816+0000","last_fullsized":"2024-09-14T06:30:23.612816+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":861,"log_dups_size":0,"ondisk_log_size":861,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T11:53:22.482997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00055519200000000003,"stat_sum":{"num_bytes":39465380,"num_objects":49,"num_object_clones":0,"num_object_copies":98,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":49,"num_whiteouts":0,"num_read":7381,"num_read_kb":42158,"num_write":1224,"num_write_kb":31953,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"2a","length":"1"},{"start":"2c","length":"1"}]},{"pgid":"3.19","version":"98'251","reported_seq":2833,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-14T06:30:23.612350+0000","last_change":"2024-09-14T06:30:22.607773+0000","last_active":"2024-09-14T06:30:23.612350+0000","last_peered":"2024-09-14T06:30:23.612350+0000","last_clean":"2024-09-14T06:30:23.612350+0000","last_became_active":"2024-09-14T06:29:00.575639+0000","last_became_peered":"2024-09-14T06:29:00.575639+0000","last_unstale":"2024-09-14T06:30:23.612350+0000","last_undegraded":"2024-09-14T06:30:23.612350+0000","last_fullsized":"2024-09-14T06:30:23.612350+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":251,"log_dups_size":0,"ondisk_log_size":251,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T18:21:05.685332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00042186099999999999,"stat_sum":{"num_bytes":34299046,"num_objects":42,"num_object_clones":0,"num_object_copies":84,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":42,"num_whiteouts":0,"num_read":2835,"num_read_kb":42010,"num_write":350,"num_write_kb":26598,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2a","length":"1"},{"start":"2c","length":"1"}]},{"pgid":"3.18","version":"104'165","reported_seq":2290,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-14T06:30:25.714092+0000","last_change":"2024-09-14T06:30:22.619334+0000","last_active":"2024-09-14T06:30:25.714092+0000","last_peered":"2024-09-14T06:30:25.714092+0000","last_clean":"2024-09-14T06:30:25.714092+0000","last_became_active":"2024-09-14T06:29:00.578017+0000","last_became_peered":"2024-09-14T06:29:00.578017+0000","last_unstale":"2024-09-14T06:30:25.714092+0000","last_undegraded":"2024-09-14T06:30:25.714092+0000","last_fullsized":"2024-09-14T06:30:25.714092+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":165,"log_dups_size":0,"ondisk_log_size":165,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T13:15:05.331803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00052650400000000001,"stat_sum":{"num_bytes":23700981,"num_objects":38,"num_object_clones":0,"num_object_copies":76,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":38,"num_whiteouts":0,"num_read":1871,"num_read_kb":30431,"num_write":176,"num_write_kb":21874,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2a","length":"1"},{"start":"2c","length":"1"}]},{"pgid":"3.17","version":"98'431","reported_seq":6415,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-14T06:30:22.609174+0000","last_change":"2024-09-14T06:30:22.609358+0000","last_active":"2024-09-14T06:30:22.609174+0000","last_peered":"2024-09-14T06:30:22.609174+0000","last_clean":"2024-09-14T06:30:22.609174+0000","last_became_active":"2024-09-14T06:29:00.579714+0000","last_became_peered":"2024-09-14T06:29:00.579714+0000","last_unstale":"2024-09-14T06:30:22.609174+0000","last_undegraded":"2024-09-14T06:30:22.609174+0000","last_fullsized":"2024-09-14T06:30:22.609174+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":431,"log_dups_size":0,"ondisk_log_size":431,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T06:50:19.988471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00073233300000000005,"stat_sum":{"num_bytes":50200381,"num_objects":54,"num_object_clones":0,"num_object_copies":108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":54,"num_whiteouts":0,"num_read":6922,"num_read_kb":48124,"num_write":516,"num_write_kb":27292,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"2a","length":"1"},{"start":"2c","length":"1"}]},{"pgid":"3.16","version":"102'198","reported_seq":1240,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-14T06:30:23.628690+0000","last_change":"2024-09-14T06:30:22.612495+0000","last_active":"2024-09-14T06:30:23.628690+0000","last_peered":"2024-09-14T06:30:23.628690+0000","last_clean":"2024-09-14T06:30:23.628690+0000","last_became_active":"2024-09-14T06:29:00.580070+0000","last_became_peered":"2024-09-14T06:29:00.580070+0000","last_unstale":"2024-09-14T06:30:23.628690+0000","last_undegraded":"2024-09-14T06:30:23.628690+0000","last_fullsized":"2024-09-14T06:30:23.628690+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":198,"log_dups_size":0,"ondisk_log_size":198,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T14:14:36.261460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013373479999999999,"stat_sum":{"num_bytes":14704807,"num_objects":33,"num_object_clones":0,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":33,"num_whiteouts":0,"num_read":1219,"num_read_kb":11360,"num_write":246,"num_write_kb":14337,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2a","length":"1"},{"start":"2c","length":"1"}]},{"pgid":"3.15","version":"101'266","reported_seq":2864,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-14T06:30:23.625554+0000","last_change":"2024-09-14T06:30:22.614903+0000","last_active":"2024-09-14T06:30:23.625554+0000","last_peered":"2024-09-14T06:30:23.625554+0000","last_clean":"2024-09-14T06:30:23.625554+0000","last_became_active":"2024-09-14T06:29:00.578451+0000","last_became_peered":"2024-09-14T06:29:00.578451+0000","last_unstale":"2024-09-14T06:30:23.625554+0000","last_undegraded":"2024-09-14T06:30:23.625554+0000","last_fullsized":"2024-09-14T06:30:23.625554+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":266,"log_dups_size":0,"ondisk_log_size":266,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T17:53:54.817492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071216900000000002,"stat_sum":{"num_bytes":34640790,"num_objects":40,"num_object_clones":0,"num_object_copies":80,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":40,"num_whiteouts":0,"num_read":2459,"num_read_kb":49809,"num_write":281,"num_write_kb":36728,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"2a","length":"1"},{"start":"2c","length":"1"}]},{"pgid":"3.14","version":"98'669","reported_seq":3782,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-14T06:30:23.612653+0000","last_change":"2024-09-14T06:30:22.608051+0000","last_active":"2024-09-14T06:30:23.612653+0000","last_peered":"2024-09-14T06:30:23.612653+0000","last_clean":"2024-09-14T06:30:23.612653+0000","last_became_active":"2024-09-14T06:29:00.582595+0000","last_became_peered":"2024-09-14T06:29:00.582595+0000","last_unstale":"2024-09-14T06:30:23.612653+0000","last_undegraded":"2024-09-14T06:30:23.612653+0000","last_fullsized":"2024-09-14T06:30:23.612653+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":669,"log_dups_size":0,"ondisk_log_size":669,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T12:33:31.116704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070081199999999998,"stat_sum":{"num_bytes":34531427,"num_objects":32,"num_object_clones":0,"num_object_copies":64,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":32,"num_whiteouts":0,"num_read":6278,"num_read_kb":32936,"num_write":1041,"num_write_kb":15101,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"2a","length":"1"},{"start":"2c","length":"1"}]},{"pgid":"2.7","version":"0'0","reported_seq":178,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-14T06:30:23.616881+0000","last_change":"2024-09-14T06:28:53.625193+0000","last_active":"2024-09-14T06:30:23.616881+0000","last_peered":"2024-09-14T06:30:23.616881+0000","last_clean":"2024-09-14T06:30:23.616881+0000","last_became_active":"2024-09-14T06:28:51.223535+0000","last_became_peered":"2024-09-14T06:28:51.223535+0000","last_unstale":"2024-09-14T06:30:23.616881+0000","last_undegraded":"2024-09-14T06:30:23.616881+0000","last_fullsized":"2024-09-14T06:30:23.616881+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:28:50.205350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:28:50.205350+0000","last_clean_scrub_stamp":"2024-09-14T06:28:50.205350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T18:14:22.822245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00052660300000000005,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"98'212","reported_seq":2211,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-14T06:30:22.608745+0000","last_change":"2024-09-14T06:30:22.608745+0000","last_active":"2024-09-14T06:30:22.608745+0000","last_peered":"2024-09-14T06:30:22.608745+0000","last_clean":"2024-09-14T06:30:22.608745+0000","last_became_active":"2024-09-14T06:29:00.578283+0000","last_became_peered":"2024-09-14T06:29:00.578283+0000","last_unstale":"2024-09-14T06:30:22.608745+0000","last_undegraded":"2024-09-14T06:30:22.608745+0000","last_fullsized":"2024-09-14T06:30:22.608745+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":212,"log_dups_size":0,"ondisk_log_size":212,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T14:11:15.343781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059847500000000001,"stat_sum":{"num_bytes":24088082,"num_objects":32,"num_object_clones":0,"num_object_copies":64,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":32,"num_whiteouts":0,"num_read":1945,"num_read_kb":19588,"num_write":227,"num_write_kb":16712,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"2a","length":"1"},{"start":"2c","length":"1"}]},{"pgid":"2.6","version":"0'0","reported_seq":180,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-14T06:30:23.612577+0000","last_change":"2024-09-14T06:28:53.226122+0000","last_active":"2024-09-14T06:30:23.612577+0000","last_peered":"2024-09-14T06:30:23.612577+0000","last_clean":"2024-09-14T06:30:23.612577+0000","last_became_active":"2024-09-14T06:28:51.217205+0000","last_became_peered":"2024-09-14T06:28:51.217205+0000","last_unstale":"2024-09-14T06:30:23.612577+0000","last_undegraded":"2024-09-14T06:30:23.612577+0000","last_fullsized":"2024-09-14T06:30:23.612577+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:28:50.205350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:28:50.205350+0000","last_clean_scrub_stamp":"2024-09-14T06:28:50.205350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T15:29:19.563302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065018200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"100'329","reported_seq":1601,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-14T06:30:22.608812+0000","last_change":"2024-09-14T06:30:22.609046+0000","last_active":"2024-09-14T06:30:22.608812+0000","last_peered":"2024-09-14T06:30:22.608812+0000","last_clean":"2024-09-14T06:30:22.608812+0000","last_became_active":"2024-09-14T06:29:56.584268+0000","last_became_peered":"2024-09-14T06:29:56.584268+0000","last_unstale":"2024-09-14T06:30:22.608812+0000","last_undegraded":"2024-09-14T06:30:22.608812+0000","last_fullsized":"2024-09-14T06:30:22.608812+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":329,"log_dups_size":0,"ondisk_log_size":329,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T11:34:58.334730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000683674,"stat_sum":{"num_bytes":24555353,"num_objects":30,"num_object_clones":0,"num_object_copies":60,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":30,"num_whiteouts":0,"num_read":1738,"num_read_kb":16219,"num_write":466,"num_write_kb":18867,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":39,"num_bytes_recovered":1400964,"num_keys_recovered":11,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"2a","length":"1"},{"start":"2c","length":"1"}]},{"pgid":"2.5","version":"0'0","reported_seq":180,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-14T06:30:23.625776+0000","last_change":"2024-09-14T06:28:53.640035+0000","last_active":"2024-09-14T06:30:23.625776+0000","last_peered":"2024-09-14T06:30:23.625776+0000","last_clean":"2024-09-14T06:30:23.625776+0000","last_became_active":"2024-09-14T06:28:51.308058+0000","last_became_peered":"2024-09-14T06:28:51.308058+0000","last_unstale":"2024-09-14T06:30:23.625776+0000","last_undegraded":"2024-09-14T06:30:23.625776+0000","last_fullsized":"2024-09-14T06:30:23.625776+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:28:50.205350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:28:50.205350+0000","last_clean_scrub_stamp":"2024-09-14T06:28:50.205350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T13:35:48.544057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00037824000000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"101'190","reported_seq":1076,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-14T06:30:23.612476+0000","last_change":"2024-09-14T06:30:22.607515+0000","last_active":"2024-09-14T06:30:23.612476+0000","last_peered":"2024-09-14T06:30:23.612476+0000","last_clean":"2024-09-14T06:30:23.612476+0000","last_became_active":"2024-09-14T06:29:00.574895+0000","last_became_peered":"2024-09-14T06:29:00.574895+0000","last_unstale":"2024-09-14T06:30:23.612476+0000","last_undegraded":"2024-09-14T06:30:23.612476+0000","last_fullsized":"2024-09-14T06:30:23.612476+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":190,"log_dups_size":0,"ondisk_log_size":190,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T08:52:01.755709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00044466900000000002,"stat_sum":{"num_bytes":13236271,"num_objects":35,"num_object_clones":0,"num_object_copies":70,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":35,"num_whiteouts":0,"num_read":730,"num_read_kb":14922,"num_write":238,"num_write_kb":19950,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2a","length":"1"},{"start":"2c","length":"1"}]},{"pgid":"2.4","version":"0'0","reported_seq":180,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-14T06:30:23.612538+0000","last_change":"2024-09-14T06:28:53.226072+0000","last_active":"2024-09-14T06:30:23.612538+0000","last_peered":"2024-09-14T06:30:23.612538+0000","last_clean":"2024-09-14T06:30:23.612538+0000","last_became_active":"2024-09-14T06:28:51.308072+0000","last_became_peered":"2024-09-14T06:28:51.308072+0000","last_unstale":"2024-09-14T06:30:23.612538+0000","last_undegraded":"2024-09-14T06:30:23.612538+0000","last_fullsized":"2024-09-14T06:30:23.612538+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:28:50.205350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:28:50.205350+0000","last_clean_scrub_stamp":"2024-09-14T06:28:50.205350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T13:15:09.052328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066197399999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"98'541","reported_seq":2231,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-14T06:30:23.623910+0000","last_change":"2024-09-14T06:30:22.612828+0000","last_active":"2024-09-14T06:30:23.623910+0000","last_peered":"2024-09-14T06:30:23.623910+0000","last_clean":"2024-09-14T06:30:23.623910+0000","last_became_active":"2024-09-14T06:29:00.581874+0000","last_became_peered":"2024-09-14T06:29:00.581874+0000","last_unstale":"2024-09-14T06:30:23.623910+0000","last_undegraded":"2024-09-14T06:30:23.623910+0000","last_fullsized":"2024-09-14T06:30:23.623910+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":541,"log_dups_size":0,"ondisk_log_size":541,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T14:14:30.760746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015671820000000001,"stat_sum":{"num_bytes":27242971,"num_objects":41,"num_object_clones":0,"num_object_copies":82,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":41,"num_whiteouts":0,"num_read":2810,"num_read_kb":19328,"num_write":837,"num_write_kb":30177,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2a","length":"1"},{"start":"2c","length":"1"}]},{"pgid":"3.3","version":"99'222","reported_seq":7695,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-14T06:30:23.612660+0000","last_change":"2024-09-14T06:30:22.608139+0000","last_active":"2024-09-14T06:30:23.612660+0000","last_peered":"2024-09-14T06:30:23.612660+0000","last_clean":"2024-09-14T06:30:23.612660+0000","last_became_active":"2024-09-14T06:29:00.584678+0000","last_became_peered":"2024-09-14T06:29:00.584678+0000","last_unstale":"2024-09-14T06:30:23.612660+0000","last_undegraded":"2024-09-14T06:30:23.612660+0000","last_fullsized":"2024-09-14T06:30:23.612660+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":222,"log_dups_size":0,"ondisk_log_size":222,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T07:15:15.023452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00072175699999999995,"stat_sum":{"num_bytes":49275751,"num_objects":32,"num_object_clones":0,"num_object_copies":64,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":32,"num_whiteouts":0,"num_read":7398,"num_read_kb":63784,"num_write":218,"num_write_kb":32775,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"2a","length":"1"},{"start":"2c","length":"1"}]},{"pgid":"2.2","version":"22'2","reported_seq":182,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-14T06:30:23.623758+0000","last_change":"2024-09-14T06:28:53.231428+0000","last_active":"2024-09-14T06:30:23.623758+0000","last_peered":"2024-09-14T06:30:23.623758+0000","last_clean":"2024-09-14T06:30:23.623758+0000","last_became_active":"2024-09-14T06:28:51.222937+0000","last_became_peered":"2024-09-14T06:28:51.222937+0000","last_unstale":"2024-09-14T06:30:23.623758+0000","last_undegraded":"2024-09-14T06:30:23.623758+0000","last_fullsized":"2024-09-14T06:30:23.623758+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:28:50.205350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:28:50.205350+0000","last_clean_scrub_stamp":"2024-09-14T06:28:50.205350+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T14:27:37.233976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010072970000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.0","version":"101'175","reported_seq":3918,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-14T06:30:23.612343+0000","last_change":"2024-09-14T06:30:22.607287+0000","last_active":"2024-09-14T06:30:23.612343+0000","last_peered":"2024-09-14T06:30:23.612343+0000","last_clean":"2024-09-14T06:30:23.612343+0000","last_became_active":"2024-09-14T06:29:00.574873+0000","last_became_peered":"2024-09-14T06:29:00.574873+0000","last_unstale":"2024-09-14T06:30:23.612343+0000","last_undegraded":"2024-09-14T06:30:23.612343+0000","last_fullsized":"2024-09-14T06:30:23.612343+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":175,"log_dups_size":0,"ondisk_log_size":175,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T08:41:53.465666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054049399999999998,"stat_sum":{"num_bytes":43263336,"num_objects":50,"num_object_clones":0,"num_object_copies":100,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":50,"num_whiteouts":0,"num_read":3438,"num_read_kb":52268,"num_write":221,"num_write_kb":31522,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2a","length":"1"},{"start":"2c","length":"1"}]},{"pgid":"2.1","version":"0'0","reported_seq":177,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-14T06:30:22.701684+0000","last_change":"2024-09-14T06:28:53.226264+0000","last_active":"2024-09-14T06:30:22.701684+0000","last_peered":"2024-09-14T06:30:22.701684+0000","last_clean":"2024-09-14T06:30:22.701684+0000","last_became_active":"2024-09-14T06:28:51.223252+0000","last_became_peered":"2024-09-14T06:28:51.223252+0000","last_unstale":"2024-09-14T06:30:22.701684+0000","last_undegraded":"2024-09-14T06:30:22.701684+0000","last_fullsized":"2024-09-14T06:30:22.701684+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:28:50.205350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:28:50.205350+0000","last_clean_scrub_stamp":"2024-09-14T06:28:50.205350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T11:56:04.373414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053403100000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1","version":"98'438","reported_seq":7025,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-14T06:30:22.608782+0000","last_change":"2024-09-14T06:30:22.608782+0000","last_active":"2024-09-14T06:30:22.608782+0000","last_peered":"2024-09-14T06:30:22.608782+0000","last_clean":"2024-09-14T06:30:22.608782+0000","last_became_active":"2024-09-14T06:29:00.585673+0000","last_became_peered":"2024-09-14T06:29:00.585673+0000","last_unstale":"2024-09-14T06:30:22.608782+0000","last_undegraded":"2024-09-14T06:30:22.608782+0000","last_fullsized":"2024-09-14T06:30:22.608782+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":438,"log_dups_size":0,"ondisk_log_size":438,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T10:35:05.573347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066394999999999996,"stat_sum":{"num_bytes":53320861,"num_objects":37,"num_object_clones":0,"num_object_copies":74,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":37,"num_whiteouts":0,"num_read":7457,"num_read_kb":62632,"num_write":553,"num_write_kb":24186,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"2a","length":"1"},{"start":"2c","length":"1"}]},{"pgid":"2.0","version":"0'0","reported_seq":180,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-14T06:30:23.625513+0000","last_change":"2024-09-14T06:28:53.640096+0000","last_active":"2024-09-14T06:30:23.625513+0000","last_peered":"2024-09-14T06:30:23.625513+0000","last_clean":"2024-09-14T06:30:23.625513+0000","last_became_active":"2024-09-14T06:28:51.223069+0000","last_became_peered":"2024-09-14T06:28:51.223069+0000","last_unstale":"2024-09-14T06:30:23.625513+0000","last_undegraded":"2024-09-14T06:30:23.625513+0000","last_fullsized":"2024-09-14T06:30:23.625513+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:28:50.205350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:28:50.205350+0000","last_clean_scrub_stamp":"2024-09-14T06:28:50.205350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T15:13:00.471509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00043714600000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"104'444","reported_seq":4331,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-14T06:30:25.625434+0000","last_change":"2024-09-14T06:30:22.620371+0000","last_active":"2024-09-14T06:30:25.625434+0000","last_peered":"2024-09-14T06:30:25.625434+0000","last_clean":"2024-09-14T06:30:25.625434+0000","last_became_active":"2024-09-14T06:29:00.578574+0000","last_became_peered":"2024-09-14T06:29:00.578574+0000","last_unstale":"2024-09-14T06:30:25.625434+0000","last_undegraded":"2024-09-14T06:30:25.625434+0000","last_fullsized":"2024-09-14T06:30:25.625434+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":444,"log_dups_size":0,"ondisk_log_size":444,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T11:08:47.042669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011884160000000001,"stat_sum":{"num_bytes":44199626,"num_objects":52,"num_object_clones":0,"num_object_copies":104,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":52,"num_whiteouts":0,"num_read":4515,"num_read_kb":44617,"num_write":567,"num_write_kb":32999,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2a","length":"1"},{"start":"2c","length":"1"}]},{"pgid":"2.3","version":"20'1","reported_seq":181,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-14T06:30:23.623743+0000","last_change":"2024-09-14T06:28:53.230933+0000","last_active":"2024-09-14T06:30:23.623743+0000","last_peered":"2024-09-14T06:30:23.623743+0000","last_clean":"2024-09-14T06:30:23.623743+0000","last_became_active":"2024-09-14T06:28:51.222874+0000","last_became_peered":"2024-09-14T06:28:51.222874+0000","last_unstale":"2024-09-14T06:30:23.623743+0000","last_undegraded":"2024-09-14T06:30:23.623743+0000","last_fullsized":"2024-09-14T06:30:23.623743+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:28:50.205350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:28:50.205350+0000","last_clean_scrub_stamp":"2024-09-14T06:28:50.205350+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T12:04:25.190171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050808899999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'72","reported_seq":271,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-14T06:30:23.625525+0000","last_change":"2024-09-14T06:28:44.839885+0000","last_active":"2024-09-14T06:30:23.625525+0000","last_peered":"2024-09-14T06:30:23.625525+0000","last_clean":"2024-09-14T06:30:23.625525+0000","last_became_active":"2024-09-14T06:28:44.839573+0000","last_became_peered":"2024-09-14T06:28:44.839573+0000","last_unstale":"2024-09-14T06:30:23.625525+0000","last_undegraded":"2024-09-14T06:30:23.625525+0000","last_fullsized":"2024-09-14T06:30:23.625525+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:28:43.828494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:28:43.828494+0000","last_clean_scrub_stamp":"2024-09-14T06:28:43.828494+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T14:41:25.787293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.8","version":"98'223","reported_seq":3490,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-14T06:30:25.625240+0000","last_change":"2024-09-14T06:30:22.620278+0000","last_active":"2024-09-14T06:30:25.625240+0000","last_peered":"2024-09-14T06:30:25.625240+0000","last_clean":"2024-09-14T06:30:25.625240+0000","last_became_active":"2024-09-14T06:29:00.574458+0000","last_became_peered":"2024-09-14T06:29:00.574458+0000","last_unstale":"2024-09-14T06:30:25.625240+0000","last_undegraded":"2024-09-14T06:30:25.625240+0000","last_fullsized":"2024-09-14T06:30:25.625240+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":223,"log_dups_size":0,"ondisk_log_size":223,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T15:58:38.545307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001174804,"stat_sum":{"num_bytes":45016747,"num_objects":46,"num_object_clones":0,"num_object_copies":92,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":46,"num_whiteouts":0,"num_read":3513,"num_read_kb":66735,"num_write":275,"num_write_kb":38280,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2a","length":"1"},{"start":"2c","length":"1"}]},{"pgid":"3.9","version":"99'143","reported_seq":1318,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-14T06:30:23.612617+0000","last_change":"2024-09-14T06:30:22.607854+0000","last_active":"2024-09-14T06:30:23.612617+0000","last_peered":"2024-09-14T06:30:23.612617+0000","last_clean":"2024-09-14T06:30:23.612617+0000","last_became_active":"2024-09-14T06:29:00.573179+0000","last_became_peered":"2024-09-14T06:29:00.573179+0000","last_unstale":"2024-09-14T06:30:23.612617+0000","last_undegraded":"2024-09-14T06:30:23.612617+0000","last_fullsized":"2024-09-14T06:30:23.612617+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":143,"log_dups_size":0,"ondisk_log_size":143,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T06:49:09.156105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00052945200000000003,"stat_sum":{"num_bytes":18711894,"num_objects":43,"num_object_clones":0,"num_object_copies":86,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":43,"num_whiteouts":0,"num_read":861,"num_read_kb":29231,"num_write":173,"num_write_kb":22284,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"2a","length":"1"},{"start":"2c","length":"1"}]},{"pgid":"3.a","version":"98'178","reported_seq":2197,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-14T06:30:23.616877+0000","last_change":"2024-09-14T06:30:22.621471+0000","last_active":"2024-09-14T06:30:23.616877+0000","last_peered":"2024-09-14T06:30:23.616877+0000","last_clean":"2024-09-14T06:30:23.616877+0000","last_became_active":"2024-09-14T06:29:00.582426+0000","last_became_peered":"2024-09-14T06:29:00.582426+0000","last_unstale":"2024-09-14T06:30:23.616877+0000","last_undegraded":"2024-09-14T06:30:23.616877+0000","last_fullsized":"2024-09-14T06:30:23.616877+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":178,"log_dups_size":0,"ondisk_log_size":178,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T06:44:46.350248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049978999999999996,"stat_sum":{"num_bytes":37141935,"num_objects":37,"num_object_clones":0,"num_object_copies":74,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":37,"num_whiteouts":0,"num_read":1910,"num_read_kb":36958,"num_write":216,"num_write_kb":27112,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"2a","length":"1"},{"start":"2c","length":"1"}]},{"pgid":"3.b","version":"104'1090","reported_seq":4324,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-14T06:30:25.634956+0000","last_change":"2024-09-14T06:30:22.620099+0000","last_active":"2024-09-14T06:30:25.634956+0000","last_peered":"2024-09-14T06:30:25.634956+0000","last_clean":"2024-09-14T06:30:25.634956+0000","last_became_active":"2024-09-14T06:29:00.578483+0000","last_became_peered":"2024-09-14T06:29:00.578483+0000","last_unstale":"2024-09-14T06:30:25.634956+0000","last_undegraded":"2024-09-14T06:30:25.634956+0000","last_fullsized":"2024-09-14T06:30:25.634956+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":1090,"log_dups_size":0,"ondisk_log_size":1090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T08:11:08.700596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012358149999999999,"stat_sum":{"num_bytes":22841129,"num_objects":43,"num_object_clones":0,"num_object_copies":86,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":43,"num_whiteouts":0,"num_read":8612,"num_read_kb":18181,"num_write":1827,"num_write_kb":17694,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2a","length":"1"},{"start":"2c","length":"1"}]},{"pgid":"3.c","version":"99'268","reported_seq":3130,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-14T06:30:23.623785+0000","last_change":"2024-09-14T06:30:22.612819+0000","last_active":"2024-09-14T06:30:23.623785+0000","last_peered":"2024-09-14T06:30:23.623785+0000","last_clean":"2024-09-14T06:30:23.623785+0000","last_became_active":"2024-09-14T06:29:00.583406+0000","last_became_peered":"2024-09-14T06:29:00.583406+0000","last_unstale":"2024-09-14T06:30:23.623785+0000","last_undegraded":"2024-09-14T06:30:23.623785+0000","last_fullsized":"2024-09-14T06:30:23.623785+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":268,"log_dups_size":0,"ondisk_log_size":268,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T08:19:59.181425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0016396290000000001,"stat_sum":{"num_bytes":26190419,"num_objects":36,"num_object_clones":0,"num_object_copies":72,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":36,"num_whiteouts":0,"num_read":3044,"num_read_kb":26426,"num_write":377,"num_write_kb":22175,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2a","length":"1"},{"start":"2c","length":"1"}]},{"pgid":"3.d","version":"98'200","reported_seq":3650,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-14T06:30:23.625667+0000","last_change":"2024-09-14T06:30:22.615128+0000","last_active":"2024-09-14T06:30:23.625667+0000","last_peered":"2024-09-14T06:30:23.625667+0000","last_clean":"2024-09-14T06:30:23.625667+0000","last_became_active":"2024-09-14T06:29:00.580362+0000","last_became_peered":"2024-09-14T06:29:00.580362+0000","last_unstale":"2024-09-14T06:30:23.625667+0000","last_undegraded":"2024-09-14T06:30:23.625667+0000","last_fullsized":"2024-09-14T06:30:23.625667+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":200,"log_dups_size":0,"ondisk_log_size":200,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T14:26:33.039528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00072795100000000001,"stat_sum":{"num_bytes":44608849,"num_objects":40,"num_object_clones":0,"num_object_copies":80,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":40,"num_whiteouts":0,"num_read":3364,"num_read_kb":45708,"num_write":248,"num_write_kb":21217,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"2a","length":"1"},{"start":"2c","length":"1"}]},{"pgid":"3.e","version":"98'273","reported_seq":1869,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-14T06:30:23.625718+0000","last_change":"2024-09-14T06:30:22.614970+0000","last_active":"2024-09-14T06:30:23.625718+0000","last_peered":"2024-09-14T06:30:23.625718+0000","last_clean":"2024-09-14T06:30:23.625718+0000","last_became_active":"2024-09-14T06:29:00.586374+0000","last_became_peered":"2024-09-14T06:29:00.586374+0000","last_unstale":"2024-09-14T06:30:23.625718+0000","last_undegraded":"2024-09-14T06:30:23.625718+0000","last_fullsized":"2024-09-14T06:30:23.625718+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":273,"log_dups_size":0,"ondisk_log_size":273,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T13:13:45.418557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069576999999999996,"stat_sum":{"num_bytes":25133180,"num_objects":45,"num_object_clones":0,"num_object_copies":90,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":45,"num_whiteouts":0,"num_read":1697,"num_read_kb":27585,"num_write":399,"num_write_kb":25425,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"2a","length":"1"},{"start":"2c","length":"1"}]},{"pgid":"3.f","version":"98'246","reported_seq":3429,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-14T06:30:23.625716+0000","last_change":"2024-09-14T06:30:22.615204+0000","last_active":"2024-09-14T06:30:23.625716+0000","last_peered":"2024-09-14T06:30:23.625716+0000","last_clean":"2024-09-14T06:30:23.625716+0000","last_became_active":"2024-09-14T06:29:00.586491+0000","last_became_peered":"2024-09-14T06:29:00.586491+0000","last_unstale":"2024-09-14T06:30:23.625716+0000","last_undegraded":"2024-09-14T06:30:23.625716+0000","last_fullsized":"2024-09-14T06:30:23.625716+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":246,"log_dups_size":0,"ondisk_log_size":246,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T13:20:04.734122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076521300000000005,"stat_sum":{"num_bytes":17046942,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":3732,"num_read_kb":18272,"num_write":303,"num_write_kb":14026,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"2a","length":"1"},{"start":"2c","length":"1"}]},{"pgid":"3.10","version":"98'284","reported_seq":2980,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-14T06:30:23.616866+0000","last_change":"2024-09-14T06:30:22.621397+0000","last_active":"2024-09-14T06:30:23.616866+0000","last_peered":"2024-09-14T06:30:23.616866+0000","last_clean":"2024-09-14T06:30:23.616866+0000","last_became_active":"2024-09-14T06:29:00.579539+0000","last_became_peered":"2024-09-14T06:29:00.579539+0000","last_unstale":"2024-09-14T06:30:23.616866+0000","last_undegraded":"2024-09-14T06:30:23.616866+0000","last_fullsized":"2024-09-14T06:30:23.616866+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":284,"log_dups_size":0,"ondisk_log_size":284,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T07:50:34.368807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00044170899999999998,"stat_sum":{"num_bytes":19497509,"num_objects":30,"num_object_clones":0,"num_object_copies":60,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":30,"num_whiteouts":0,"num_read":2707,"num_read_kb":14268,"num_write":307,"num_write_kb":22669,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"2a","length":"1"},{"start":"2c","length":"1"}]},{"pgid":"3.11","version":"98'375","reported_seq":7677,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-14T06:30:23.625513+0000","last_change":"2024-09-14T06:30:22.614639+0000","last_active":"2024-09-14T06:30:23.625513+0000","last_peered":"2024-09-14T06:30:23.625513+0000","last_clean":"2024-09-14T06:30:23.625513+0000","last_became_active":"2024-09-14T06:29:00.586480+0000","last_became_peered":"2024-09-14T06:29:00.586480+0000","last_unstale":"2024-09-14T06:30:23.625513+0000","last_undegraded":"2024-09-14T06:30:23.625513+0000","last_fullsized":"2024-09-14T06:30:23.625513+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":375,"log_dups_size":0,"ondisk_log_size":375,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T17:18:48.953355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00055691300000000003,"stat_sum":{"num_bytes":72839101,"num_objects":49,"num_object_clones":0,"num_object_copies":98,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":49,"num_whiteouts":0,"num_read":7460,"num_read_kb":81273,"num_write":440,"num_write_kb":34087,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"2a","length":"1"},{"start":"2c","length":"1"}]},{"pgid":"3.12","version":"98'456","reported_seq":5486,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-14T06:30:22.609092+0000","last_change":"2024-09-14T06:30:22.609092+0000","last_active":"2024-09-14T06:30:22.609092+0000","last_peered":"2024-09-14T06:30:22.609092+0000","last_clean":"2024-09-14T06:30:22.609092+0000","last_became_active":"2024-09-14T06:29:00.579826+0000","last_became_peered":"2024-09-14T06:29:00.579826+0000","last_unstale":"2024-09-14T06:30:22.609092+0000","last_undegraded":"2024-09-14T06:30:22.609092+0000","last_fullsized":"2024-09-14T06:30:22.609092+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":456,"log_dups_size":0,"ondisk_log_size":456,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T09:44:15.157725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070490600000000004,"stat_sum":{"num_bytes":33207124,"num_objects":43,"num_object_clones":0,"num_object_copies":86,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":43,"num_whiteouts":0,"num_read":5850,"num_read_kb":42269,"num_write":571,"num_write_kb":26908,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"2a","length":"1"},{"start":"2c","length":"1"}]},{"pgid":"3.13","version":"98'453","reported_seq":5350,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-14T06:30:23.625563+0000","last_change":"2024-09-14T06:30:22.614719+0000","last_active":"2024-09-14T06:30:23.625563+0000","last_peered":"2024-09-14T06:30:23.625563+0000","last_clean":"2024-09-14T06:30:23.625563+0000","last_became_active":"2024-09-14T06:29:00.586520+0000","last_became_peered":"2024-09-14T06:29:00.586520+0000","last_unstale":"2024-09-14T06:30:23.625563+0000","last_undegraded":"2024-09-14T06:30:23.625563+0000","last_fullsized":"2024-09-14T06:30:23.625563+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":453,"log_dups_size":0,"ondisk_log_size":453,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T10:44:48.405672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053809500000000004,"stat_sum":{"num_bytes":53149511,"num_objects":38,"num_object_clones":0,"num_object_copies":76,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":38,"num_whiteouts":0,"num_read":5488,"num_read_kb":45634,"num_write":617,"num_write_kb":31089,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"2a","length":"1"},{"start":"2c","length":"1"}]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":1078895510,"num_objects":1275,"num_object_clones":0,"num_object_copies":2550,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1275,"num_whiteouts":0,"num_read":126504,"num_read_kb":1169632,"num_write":14962,"num_write_kb":811453,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":39,"num_bytes_recovered":1400964,"num_keys_recovered":11,"num_objects_omap":24,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":73011200,"data_stored":712703129,"data_compressed":43868295,"data_compressed_allocated":57798656,"data_compressed_original":700737716,"omap_allocated":0,"internal_metadata":0},"log_size":11097,"ondisk_log_size":11097,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":131072,"data_stored":1180736,"data_compressed":65166,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509463,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55100,"kb_used_data":11432,"kb_used_omap":1,"kb_used_meta":43646,"kb_avail":94316740,"statfs":{"total":96636764160,"available":96580341760,"internally_reserved":0,"allocated":11706368,"data_stored":99810101,"data_compressed":6064561,"data_compressed_allocated":7933952,"data_compressed_original":97174851,"omap_allocated":1590,"internal_metadata":44693962},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509462,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":34868,"kb_used_data":3684,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94336972,"statfs":{"total":96636764160,"available":96601059328,"internally_reserved":0,"allocated":3772416,"data_stored":26601695,"data_compressed":1494839,"data_compressed_allocated":2039808,"data_compressed_original":25607719,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509464,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":50064,"kb_used_data":10552,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94321776,"statfs":{"total":96636764160,"available":96585498624,"internally_reserved":0,"allocated":10805248,"data_stored":89545146,"data_compressed":5484896,"data_compressed_allocated":7340032,"data_compressed_original":87216976,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509463,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":66060,"kb_used_data":16820,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94305780,"statfs":{"total":96636764160,"available":96569118720,"internally_reserved":0,"allocated":17223680,"data_stored":159476697,"data_compressed":10154445,"data_compressed_allocated":13172736,"data_compressed_original":156746019,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509464,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":47928,"kb_used_data":8424,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94323912,"statfs":{"total":96636764160,"available":96587685888,"internally_reserved":0,"allocated":8626176,"data_stored":76286321,"data_compressed":4608422,"data_compressed_allocated":6103040,"data_compressed_original":74710041,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":14,"seq":60129542167,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40072,"kb_used_data":4720,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94331768,"statfs":{"total":96636764160,"available":96595730432,"internally_reserved":0,"allocated":4833280,"data_stored":38471287,"data_compressed":2357654,"data_compressed_allocated":3133440,"data_compressed_original":37542635,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542166,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":61932,"kb_used_data":12696,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94309908,"statfs":{"total":96636764160,"available":96573345792,"internally_reserved":0,"allocated":13000704,"data_stored":113700341,"data_compressed":6847207,"data_compressed_allocated":8962048,"data_compressed_original":110815485,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574872,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":61596,"kb_used_data":12356,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94310244,"statfs":{"total":96636764160,"available":96573689856,"internally_reserved":0,"allocated":12652544,"data_stored":115018177,"data_compressed":6890091,"data_compressed_allocated":9175040,"data_compressed_original":112640574,"omap_allocated":1593,"internal_metadata":50395591},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32583,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32583,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":11743232,"data_stored":112548892,"data_compressed":6814624,"data_compressed_allocated":8916992,"data_compressed_original":110242045,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":7430144,"data_stored":75725221,"data_compressed":4608422,"data_compressed_allocated":6103040,"data_compressed_original":74710041,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":3653632,"data_stored":37916454,"data_compressed":2357654,"data_compressed_allocated":3133440,"data_compressed_original":37542635,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":11702272,"data_stored":117066490,"data_compressed":7105014,"data_compressed_allocated":9441280,"data_compressed_original":115262014,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":15855616,"data_stored":155775103,"data_compressed":9971268,"data_compressed_allocated":12939264,"data_compressed_original":153563067,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":9609216,"data_stored":88984046,"data_compressed":5484896,"data_compressed_allocated":7340032,"data_compressed_original":87216976,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":2580480,"data_stored":26040614,"data_compressed":1494839,"data_compressed_allocated":2039808,"data_compressed_original":25607719,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":10436608,"data_stored":98646309,"data_compressed":6031578,"data_compressed_allocated":7884800,"data_compressed_original":96593219,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-14T06:30:27.365 INFO:tasks.thrashosds.thrasher:Setting 3.a to [2, 3, 0, 6] 2024-09-14T06:30:27.365 INFO:tasks.thrashosds.thrasher:cmd ['osd', 'pg-upmap-items', '3.a', '2', '3', '0', '6'] 2024-09-14T06:30:27.365 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pg-upmap-items 3.a 2 3 0 6 2024-09-14T06:30:27.404 INFO:tasks.ceph.osd.3.smithi094.stderr:2024-09-14T06:30:27.403+0000 7feeeafe8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33655) UID: 0 2024-09-14T06:30:27.439 INFO:teuthology.orchestra.run.smithi138.stdout:{ 2024-09-14T06:30:27.439 INFO:teuthology.orchestra.run.smithi138.stdout: "size": 20, 2024-09-14T06:30:27.439 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 600, 2024-09-14T06:30:27.439 INFO:teuthology.orchestra.run.smithi138.stdout: "ops": [ 2024-09-14T06:30:27.439 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:27.439 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:11453 3.1d 3:bcad7303:::rbd_data.112d92027590.0000000000000001:f [read 94208~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:30:27.439 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:25.654375+0000", 2024-09-14T06:30:27.439 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 61.782938831999999, 2024-09-14T06:30:27.439 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052449318000000002, 2024-09-14T06:30:27.439 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:30:27.440 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:30:27.440 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:30:27.440 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:30:27.440 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:30:27.440 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:27.440 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 11453 2024-09-14T06:30:27.440 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:27.440 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:30:27.440 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:27.440 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:30:27.440 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654375+0000", 2024-09-14T06:30:27.441 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:27.441 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:27.441 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:27.441 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:30:27.441 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654375+0000", 2024-09-14T06:30:27.441 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:27.441 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:27.441 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:27.441 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:30:27.441 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654496+0000", 2024-09-14T06:30:27.441 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00012081 2024-09-14T06:30:27.441 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:27.442 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:27.442 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:30:27.442 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654497+0000", 2024-09-14T06:30:27.442 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.8789999999999999e-06 2024-09-14T06:30:27.442 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:27.442 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:27.442 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:30:27.442 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654501+0000", 2024-09-14T06:30:27.442 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.8999999999999999e-06 2024-09-14T06:30:27.442 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:27.442 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:27.442 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:30:27.442 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654505+0000", 2024-09-14T06:30:27.442 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.2229999999999999e-06 2024-09-14T06:30:27.443 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:27.443 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:27.443 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:30:27.443 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.705917+0000", 2024-09-14T06:30:27.443 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.051412501999999999 2024-09-14T06:30:27.443 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:27.443 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:27.443 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:30:27.443 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.705999+0000", 2024-09-14T06:30:27.443 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.1551999999999994e-05 2024-09-14T06:30:27.443 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:27.443 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:27.443 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:30:27.443 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.706824+0000", 2024-09-14T06:30:27.444 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00082545199999999996 2024-09-14T06:30:27.444 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:27.444 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:30:27.444 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:27.444 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:27.444 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:27.444 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:11454 3.1d 3:bcad7303:::rbd_data.112d92027590.0000000000000001:f [read 104448~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:30:27.444 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:25.654510+0000", 2024-09-14T06:30:27.444 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 61.782804042000002, 2024-09-14T06:30:27.444 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052610713000000003, 2024-09-14T06:30:27.444 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:30:27.444 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:30:27.445 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:30:27.445 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:30:27.445 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:30:27.445 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:27.445 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 11454 2024-09-14T06:30:27.445 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:27.445 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:30:27.445 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:27.445 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:30:27.445 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654510+0000", 2024-09-14T06:30:27.445 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:27.445 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:27.445 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:27.445 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:30:27.445 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654510+0000", 2024-09-14T06:30:27.446 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:27.446 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:27.446 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:27.446 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:30:27.446 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654825+0000", 2024-09-14T06:30:27.446 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00031522400000000002 2024-09-14T06:30:27.446 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:27.446 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:27.446 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:30:27.446 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654827+0000", 2024-09-14T06:30:27.446 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.9709999999999998e-06 2024-09-14T06:30:27.446 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:27.446 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:27.446 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:30:27.447 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654832+0000", 2024-09-14T06:30:27.447 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.4099999999999999e-06 2024-09-14T06:30:27.447 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:27.447 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:27.447 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:30:27.447 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654835+0000", 2024-09-14T06:30:27.447 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.14e-06 2024-09-14T06:30:27.447 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:27.447 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:27.447 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:30:27.447 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.706851+0000", 2024-09-14T06:30:27.447 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052015761000000001 2024-09-14T06:30:27.447 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:27.447 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:27.448 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:30:27.448 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.706934+0000", 2024-09-14T06:30:27.448 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.2484999999999994e-05 2024-09-14T06:30:27.448 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:27.448 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:27.448 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:30:27.448 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.707120+0000", 2024-09-14T06:30:27.448 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00018672200000000001 2024-09-14T06:30:27.448 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:27.448 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:30:27.448 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:27.448 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:27.448 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:27.449 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:11455 3.1d 3:bcad7303:::rbd_data.112d92027590.0000000000000001:f [read 114688~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:30:27.449 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:25.654841+0000", 2024-09-14T06:30:27.449 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 61.782472712999997, 2024-09-14T06:30:27.449 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052579898, 2024-09-14T06:30:27.449 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:30:27.449 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:30:27.449 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:30:27.449 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:30:27.449 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:30:27.449 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:27.449 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 11455 2024-09-14T06:30:27.449 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:27.449 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:30:27.450 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:27.450 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:30:27.450 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654841+0000", 2024-09-14T06:30:27.450 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:27.450 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:27.450 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:27.450 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:30:27.450 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654841+0000", 2024-09-14T06:30:27.450 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:27.450 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:27.450 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:27.450 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:30:27.450 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654841+0000", 2024-09-14T06:30:27.450 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.7300000000000002e-07 2024-09-14T06:30:27.450 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:27.451 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:27.451 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:30:27.451 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654845+0000", 2024-09-14T06:30:27.451 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.0520000000000003e-06 2024-09-14T06:30:27.451 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:27.451 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:27.451 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:30:27.451 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654849+0000", 2024-09-14T06:30:27.451 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.0459999999999999e-06 2024-09-14T06:30:27.451 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:27.451 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:27.451 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:30:27.451 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654852+0000", 2024-09-14T06:30:27.451 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.9830000000000001e-06 2024-09-14T06:30:27.452 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:27.452 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:27.452 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:30:27.452 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.707159+0000", 2024-09-14T06:30:27.452 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052306826000000001 2024-09-14T06:30:27.452 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:27.452 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:27.452 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:30:27.452 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.707242+0000", 2024-09-14T06:30:27.452 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.2595000000000005e-05 2024-09-14T06:30:27.452 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:27.452 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:27.452 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:30:27.452 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.707421+0000", 2024-09-14T06:30:27.453 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.000179123 2024-09-14T06:30:27.453 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:27.453 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:30:27.453 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:27.453 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:27.453 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:27.453 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:11456 3.1d 3:bcad7303:::rbd_data.112d92027590.0000000000000001:f [read 124928~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:30:27.453 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:25.654856+0000", 2024-09-14T06:30:27.453 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 61.782457866999998, 2024-09-14T06:30:27.453 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052801580000000001, 2024-09-14T06:30:27.453 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:30:27.453 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:30:27.453 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:30:27.454 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:30:27.454 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:30:27.454 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:27.454 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 11456 2024-09-14T06:30:27.454 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:27.454 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:30:27.454 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:27.454 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:30:27.454 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654856+0000", 2024-09-14T06:30:27.454 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:27.454 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:27.454 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:27.454 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:30:27.454 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654856+0000", 2024-09-14T06:30:27.455 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:27.455 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:27.455 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:27.455 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:30:27.455 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654856+0000", 2024-09-14T06:30:27.455 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.29e-07 2024-09-14T06:30:27.455 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:27.455 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:27.455 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:30:27.455 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654857+0000", 2024-09-14T06:30:27.456 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.187e-06 2024-09-14T06:30:27.456 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:27.456 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:27.456 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:30:27.456 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654861+0000", 2024-09-14T06:30:27.456 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.3850000000000002e-06 2024-09-14T06:30:27.456 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:27.456 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:27.456 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:30:27.456 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654863+0000", 2024-09-14T06:30:27.456 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.9500000000000001e-06 2024-09-14T06:30:27.456 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:27.457 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:27.457 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:30:27.457 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.707456+0000", 2024-09-14T06:30:27.457 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052592434 2024-09-14T06:30:27.457 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:27.457 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:27.457 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:30:27.457 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.707516+0000", 2024-09-14T06:30:27.457 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.0167999999999997e-05 2024-09-14T06:30:27.457 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:27.457 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:27.457 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:30:27.457 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.707657+0000", 2024-09-14T06:30:27.457 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00014122700000000001 2024-09-14T06:30:27.458 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:27.458 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:30:27.458 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:27.458 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:27.458 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:27.458 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:11457 3.1d 3:bcad7303:::rbd_data.112d92027590.0000000000000001:f [read 135168~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:30:27.458 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:25.654867+0000", 2024-09-14T06:30:27.458 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 61.782446982000003, 2024-09-14T06:30:27.458 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052921902, 2024-09-14T06:30:27.458 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:30:27.458 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:30:27.458 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:30:27.458 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:30:27.459 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:30:27.459 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:27.459 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 11457 2024-09-14T06:30:27.459 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:27.459 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:30:27.459 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:27.459 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:30:27.459 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654867+0000", 2024-09-14T06:30:27.459 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:27.459 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:27.459 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:27.459 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:30:27.459 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654867+0000", 2024-09-14T06:30:27.459 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:27.460 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:27.460 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:27.460 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:30:27.460 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654972+0000", 2024-09-14T06:30:27.460 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00010584 2024-09-14T06:30:27.460 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:27.460 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:27.460 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:30:27.460 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654975+0000", 2024-09-14T06:30:27.460 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.1169999999999998e-06 2024-09-14T06:30:27.460 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:27.460 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:27.460 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:30:27.460 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654979+0000", 2024-09-14T06:30:27.461 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.1960000000000001e-06 2024-09-14T06:30:27.461 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:27.461 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:27.461 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:30:27.461 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654982+0000", 2024-09-14T06:30:27.461 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.2990000000000001e-06 2024-09-14T06:30:27.461 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:27.461 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:27.461 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:30:27.461 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.707684+0000", 2024-09-14T06:30:27.461 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052701979000000003 2024-09-14T06:30:27.461 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:27.461 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:27.462 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:30:27.462 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.707728+0000", 2024-09-14T06:30:27.462 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.3529999999999998e-05 2024-09-14T06:30:27.462 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:27.462 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:27.462 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:30:27.462 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.707789+0000", 2024-09-14T06:30:27.462 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.0940999999999998e-05 2024-09-14T06:30:27.462 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:27.462 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:30:27.462 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:27.462 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:27.462 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:27.462 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:11458 3.1d 3:bcad7303:::rbd_data.112d92027590.0000000000000001:f [read 145408~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:30:27.463 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:25.654987+0000", 2024-09-14T06:30:27.463 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 61.782326584000003, 2024-09-14T06:30:27.463 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.05291655, 2024-09-14T06:30:27.463 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:30:27.463 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:30:27.463 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:30:27.463 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:30:27.463 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:30:27.463 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:27.463 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 11458 2024-09-14T06:30:27.463 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:27.463 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:30:27.463 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:27.463 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:30:27.464 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654987+0000", 2024-09-14T06:30:27.464 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:27.464 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:27.464 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:27.464 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:30:27.464 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654987+0000", 2024-09-14T06:30:27.464 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:27.464 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:27.464 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:27.464 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:30:27.464 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655077+0000", 2024-09-14T06:30:27.464 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.0253000000000005e-05 2024-09-14T06:30:27.464 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:27.465 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:27.465 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:30:27.465 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655079+0000", 2024-09-14T06:30:27.465 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.9989999999999998e-06 2024-09-14T06:30:27.465 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:27.465 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:27.465 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:30:27.465 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655083+0000", 2024-09-14T06:30:27.465 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.9280000000000003e-06 2024-09-14T06:30:27.465 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:27.465 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:27.465 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:30:27.465 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655086+0000", 2024-09-14T06:30:27.466 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.056e-06 2024-09-14T06:30:27.466 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:27.466 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:27.466 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:30:27.466 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.707805+0000", 2024-09-14T06:30:27.466 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052718627999999997 2024-09-14T06:30:27.466 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:27.466 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:27.466 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:30:27.466 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.707848+0000", 2024-09-14T06:30:27.466 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.2728999999999997e-05 2024-09-14T06:30:27.466 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:27.466 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:27.467 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:30:27.467 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.707904+0000", 2024-09-14T06:30:27.467 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.5957e-05 2024-09-14T06:30:27.467 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:27.467 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:30:27.467 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:27.467 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:27.467 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:27.467 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:11459 3.1d 3:bcad7303:::rbd_data.112d92027590.0000000000000001:f [read 155648~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:30:27.467 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:25.655091+0000", 2024-09-14T06:30:27.467 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 61.782222576000002, 2024-09-14T06:30:27.467 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052944242000000002, 2024-09-14T06:30:27.467 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:30:27.467 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:30:27.468 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:30:27.468 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:30:27.468 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:30:27.468 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:27.468 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 11459 2024-09-14T06:30:27.468 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:27.468 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:30:27.468 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:27.468 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:30:27.468 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655091+0000", 2024-09-14T06:30:27.468 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:27.468 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:27.468 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:27.469 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:30:27.469 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655091+0000", 2024-09-14T06:30:27.469 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:27.469 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:27.469 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:27.469 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:30:27.469 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655237+0000", 2024-09-14T06:30:27.469 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00014570100000000001 2024-09-14T06:30:27.469 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:27.469 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:27.469 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:30:27.469 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655239+0000", 2024-09-14T06:30:27.469 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.8670000000000001e-06 2024-09-14T06:30:27.470 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:27.470 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:27.470 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:30:27.470 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655243+0000", 2024-09-14T06:30:27.470 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.9829999999999998e-06 2024-09-14T06:30:27.470 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:27.470 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:27.470 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:30:27.470 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655246+0000", 2024-09-14T06:30:27.470 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.1540000000000002e-06 2024-09-14T06:30:27.470 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:27.470 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:27.470 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:30:27.470 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.707934+0000", 2024-09-14T06:30:27.471 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052688286000000001 2024-09-14T06:30:27.471 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:27.471 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:27.471 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:30:27.471 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.707977+0000", 2024-09-14T06:30:27.471 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.3291999999999998e-05 2024-09-14T06:30:27.471 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:27.471 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:27.471 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:30:27.471 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.708035+0000", 2024-09-14T06:30:27.471 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.7958999999999999e-05 2024-09-14T06:30:27.471 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:27.472 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:30:27.472 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:27.472 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:27.472 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:27.472 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:11460 3.1d 3:bcad7303:::rbd_data.112d92027590.0000000000000001:f [read 165888~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:30:27.472 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:25.655250+0000", 2024-09-14T06:30:27.472 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 61.782063278999999, 2024-09-14T06:30:27.472 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052910698999999999, 2024-09-14T06:30:27.472 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:30:27.472 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:30:27.472 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:30:27.472 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:30:27.472 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:30:27.473 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:27.473 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 11460 2024-09-14T06:30:27.473 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:27.473 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:30:27.473 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:27.473 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:30:27.473 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655250+0000", 2024-09-14T06:30:27.473 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:27.473 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:27.473 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:27.473 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:30:27.473 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655250+0000", 2024-09-14T06:30:27.474 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:27.474 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:27.474 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:27.474 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:30:27.474 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655356+0000", 2024-09-14T06:30:27.474 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00010535900000000001 2024-09-14T06:30:27.474 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:27.474 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:27.474 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:30:27.474 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655357+0000", 2024-09-14T06:30:27.474 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.8080000000000001e-06 2024-09-14T06:30:27.474 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:27.474 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:27.474 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:30:27.475 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655361+0000", 2024-09-14T06:30:27.475 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.861e-06 2024-09-14T06:30:27.475 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:27.475 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:27.475 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:30:27.475 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655364+0000", 2024-09-14T06:30:27.475 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.1089999999999998e-06 2024-09-14T06:30:27.475 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:27.475 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:27.475 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:30:27.475 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.708065+0000", 2024-09-14T06:30:27.475 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052700595000000003 2024-09-14T06:30:27.475 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:27.475 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:27.476 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:30:27.476 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.708108+0000", 2024-09-14T06:30:27.476 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.2484999999999997e-05 2024-09-14T06:30:27.476 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:27.476 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:27.476 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:30:27.476 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.708161+0000", 2024-09-14T06:30:27.476 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.3482000000000001e-05 2024-09-14T06:30:27.476 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:27.476 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:30:27.476 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:27.476 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:27.476 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:27.476 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:11461 3.1d 3:bcad7303:::rbd_data.112d92027590.0000000000000001:f [read 176128~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:30:27.477 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:25.655370+0000", 2024-09-14T06:30:27.477 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 61.781943996999999, 2024-09-14T06:30:27.477 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052915141999999998, 2024-09-14T06:30:27.477 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:30:27.477 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:30:27.477 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:30:27.477 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:30:27.477 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:30:27.477 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:27.477 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 11461 2024-09-14T06:30:27.477 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:27.477 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:30:27.477 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:27.477 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:30:27.478 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655370+0000", 2024-09-14T06:30:27.478 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:27.478 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:27.478 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:27.478 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:30:27.478 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655370+0000", 2024-09-14T06:30:27.478 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:27.478 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:27.478 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:27.478 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:30:27.478 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655394+0000", 2024-09-14T06:30:27.478 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.4553999999999999e-05 2024-09-14T06:30:27.478 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:27.478 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:27.479 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:30:27.479 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655396+0000", 2024-09-14T06:30:27.479 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.8950000000000001e-06 2024-09-14T06:30:27.479 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:27.479 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:27.479 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:30:27.479 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655401+0000", 2024-09-14T06:30:27.479 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.0089999999999998e-06 2024-09-14T06:30:27.479 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:27.479 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:27.479 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:30:27.479 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655404+0000", 2024-09-14T06:30:27.479 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.3400000000000002e-06 2024-09-14T06:30:27.479 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:27.480 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:27.480 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:30:27.480 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.708186+0000", 2024-09-14T06:30:27.480 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052781726000000001 2024-09-14T06:30:27.480 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:27.480 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:27.480 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:30:27.480 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.708229+0000", 2024-09-14T06:30:27.480 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.2670999999999999e-05 2024-09-14T06:30:27.480 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:27.480 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:27.480 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:30:27.480 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.708285+0000", 2024-09-14T06:30:27.480 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.5946999999999998e-05 2024-09-14T06:30:27.481 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:27.481 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:30:27.481 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:27.481 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:27.481 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:27.481 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:11462 3.1d 3:bcad7303:::rbd_data.112d92027590.0000000000000001:f [read 186368~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:30:27.481 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:25.655410+0000", 2024-09-14T06:30:27.481 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 61.781903518999997, 2024-09-14T06:30:27.481 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052995258000000003, 2024-09-14T06:30:27.481 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:30:27.481 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:30:27.481 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:30:27.481 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:30:27.481 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:30:27.482 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:27.482 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 11462 2024-09-14T06:30:27.482 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:27.482 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:30:27.482 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:27.482 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:30:27.482 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655410+0000", 2024-09-14T06:30:27.482 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:27.482 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:27.482 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:27.482 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:30:27.482 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655410+0000", 2024-09-14T06:30:27.482 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:27.483 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:27.483 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:27.483 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:30:27.483 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655625+0000", 2024-09-14T06:30:27.483 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00021442600000000001 2024-09-14T06:30:27.483 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:27.483 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:27.483 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:30:27.483 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655626+0000", 2024-09-14T06:30:27.483 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.815e-06 2024-09-14T06:30:27.483 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:27.483 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:27.483 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:30:27.483 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655631+0000", 2024-09-14T06:30:27.484 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.651e-06 2024-09-14T06:30:27.484 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:27.484 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:27.484 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:30:27.484 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655634+0000", 2024-09-14T06:30:27.484 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.2449999999999998e-06 2024-09-14T06:30:27.484 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:27.484 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:27.484 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:30:27.484 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.708312+0000", 2024-09-14T06:30:27.484 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052677412 2024-09-14T06:30:27.484 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:27.484 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:27.484 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:30:27.485 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.708352+0000", 2024-09-14T06:30:27.485 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.0729999999999998e-05 2024-09-14T06:30:27.485 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:27.485 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:27.485 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:30:27.485 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.708405+0000", 2024-09-14T06:30:27.485 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.2979000000000002e-05 2024-09-14T06:30:27.485 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:27.485 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:30:27.485 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:27.485 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:27.485 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:27.485 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:11463 3.1d 3:bcad7303:::rbd_data.112d92027590.0000000000000001:f [read 196608~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:30:27.485 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:25.655639+0000", 2024-09-14T06:30:27.486 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 61.781674373000001, 2024-09-14T06:30:27.486 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052888859000000003, 2024-09-14T06:30:27.486 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:30:27.486 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:30:27.486 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:30:27.486 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:30:27.486 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:30:27.486 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:27.486 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 11463 2024-09-14T06:30:27.486 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:27.486 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:30:27.486 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:27.486 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:30:27.486 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655639+0000", 2024-09-14T06:30:27.487 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:27.487 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:27.487 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:27.487 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:30:27.487 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655639+0000", 2024-09-14T06:30:27.487 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:27.487 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:27.487 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:27.487 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:30:27.487 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655640+0000", 2024-09-14T06:30:27.487 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.08e-07 2024-09-14T06:30:27.487 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:27.487 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:27.487 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:30:27.487 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655641+0000", 2024-09-14T06:30:27.488 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.156e-06 2024-09-14T06:30:27.488 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:27.488 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:27.488 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:30:27.488 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655648+0000", 2024-09-14T06:30:27.488 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.8460000000000002e-06 2024-09-14T06:30:27.488 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:27.488 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:27.488 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:30:27.488 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655651+0000", 2024-09-14T06:30:27.488 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.0079999999999998e-06 2024-09-14T06:30:27.488 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:27.488 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:27.489 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:30:27.489 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.708429+0000", 2024-09-14T06:30:27.489 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052778459 2024-09-14T06:30:27.489 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:27.489 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:27.489 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:30:27.489 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.708473+0000", 2024-09-14T06:30:27.489 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.3782999999999997e-05 2024-09-14T06:30:27.489 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:27.489 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:27.489 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:30:27.489 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.708528+0000", 2024-09-14T06:30:27.489 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.5198999999999998e-05 2024-09-14T06:30:27.489 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:27.490 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:30:27.490 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:27.490 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:27.490 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:27.490 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:11464 3.1d 3:bcad7303:::rbd_data.112d92027590.0000000000000001:f [read 206848~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:30:27.490 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:25.655654+0000", 2024-09-14T06:30:27.490 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 61.781659503999997, 2024-09-14T06:30:27.490 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052981845999999999, 2024-09-14T06:30:27.490 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:30:27.490 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:30:27.490 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:30:27.490 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:30:27.490 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:30:27.490 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:27.491 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 11464 2024-09-14T06:30:27.491 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:27.491 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:30:27.491 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:27.491 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:30:27.491 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655654+0000", 2024-09-14T06:30:27.491 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:27.491 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:27.491 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:27.491 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:30:27.491 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655654+0000", 2024-09-14T06:30:27.491 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:27.491 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:27.491 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:27.492 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:30:27.492 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655946+0000", 2024-09-14T06:30:27.492 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00029148200000000003 2024-09-14T06:30:27.492 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:27.492 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:27.492 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:30:27.492 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655948+0000", 2024-09-14T06:30:27.492 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.973e-06 2024-09-14T06:30:27.492 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:27.492 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:27.492 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:30:27.492 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655953+0000", 2024-09-14T06:30:27.492 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.1240000000000004e-06 2024-09-14T06:30:27.493 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:27.493 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:27.493 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:30:27.493 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655956+0000", 2024-09-14T06:30:27.493 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.315e-06 2024-09-14T06:30:27.493 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:27.493 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:27.493 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:30:27.493 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.708543+0000", 2024-09-14T06:30:27.493 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052587416999999997 2024-09-14T06:30:27.493 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:27.493 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:27.493 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:30:27.493 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.708584+0000", 2024-09-14T06:30:27.494 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.1004000000000002e-05 2024-09-14T06:30:27.494 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:27.494 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:27.494 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:30:27.494 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.708636+0000", 2024-09-14T06:30:27.494 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.1530999999999998e-05 2024-09-14T06:30:27.494 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:27.494 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:30:27.494 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:27.494 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:27.494 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:27.494 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:11465 3.1d 3:bcad7303:::rbd_data.112d92027590.0000000000000001:f [read 217088~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:30:27.494 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:25.655961+0000", 2024-09-14T06:30:27.495 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 61.781352351999999, 2024-09-14T06:30:27.495 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052798682999999999, 2024-09-14T06:30:27.495 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:30:27.495 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:30:27.495 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:30:27.495 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:30:27.495 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:30:27.495 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:27.495 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 11465 2024-09-14T06:30:27.495 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:27.495 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:30:27.495 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:27.496 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:30:27.496 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655961+0000", 2024-09-14T06:30:27.496 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:27.496 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:27.496 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:27.496 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:30:27.496 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655961+0000", 2024-09-14T06:30:27.496 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:27.496 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:27.496 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:27.496 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:30:27.496 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.656114+0000", 2024-09-14T06:30:27.497 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.000152745 2024-09-14T06:30:27.497 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:27.497 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:27.497 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:30:27.497 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.656116+0000", 2024-09-14T06:30:27.497 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.804e-06 2024-09-14T06:30:27.497 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:27.497 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:27.497 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:30:27.497 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.656120+0000", 2024-09-14T06:30:27.497 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.4700000000000004e-06 2024-09-14T06:30:27.497 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:27.497 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:27.497 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:30:27.498 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.656123+0000", 2024-09-14T06:30:27.498 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.1269999999999999e-06 2024-09-14T06:30:27.498 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:27.498 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:27.498 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:30:27.498 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.708660+0000", 2024-09-14T06:30:27.498 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052536919000000001 2024-09-14T06:30:27.498 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:27.498 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:27.498 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:30:27.498 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.708703+0000", 2024-09-14T06:30:27.498 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.3077000000000001e-05 2024-09-14T06:30:27.498 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:27.499 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:27.499 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:30:27.499 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.708760+0000", 2024-09-14T06:30:27.499 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.6541e-05 2024-09-14T06:30:27.499 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:27.499 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:30:27.499 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:27.499 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:27.499 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:27.499 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:11466 3.1d 3:bcad7303:::rbd_data.112d92027590.0000000000000001:f [read 227328~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:30:27.499 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:25.656129+0000", 2024-09-14T06:30:27.499 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 61.781184963999998, 2024-09-14T06:30:27.499 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052750987999999999, 2024-09-14T06:30:27.499 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:30:27.500 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:30:27.500 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:30:27.500 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:30:27.500 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:30:27.500 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:27.500 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 11466 2024-09-14T06:30:27.500 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:27.500 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:30:27.500 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:27.500 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:30:27.500 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.656129+0000", 2024-09-14T06:30:27.500 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:27.500 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:27.500 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:27.501 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:30:27.501 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.656129+0000", 2024-09-14T06:30:27.501 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:27.501 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:27.501 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:27.501 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:30:27.501 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.656347+0000", 2024-09-14T06:30:27.501 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.000218054 2024-09-14T06:30:27.501 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:27.501 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:27.501 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:30:27.501 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.656349+0000", 2024-09-14T06:30:27.501 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.9199999999999998e-06 2024-09-14T06:30:27.502 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:27.502 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:27.502 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:30:27.502 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.656353+0000", 2024-09-14T06:30:27.502 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.9210000000000002e-06 2024-09-14T06:30:27.502 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:27.502 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:27.502 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:30:27.502 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.656356+0000", 2024-09-14T06:30:27.502 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.7670000000000001e-06 2024-09-14T06:30:27.502 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:27.502 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:27.502 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:30:27.502 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.708787+0000", 2024-09-14T06:30:27.503 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052430840999999999 2024-09-14T06:30:27.503 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:27.503 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:27.503 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:30:27.503 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.708827+0000", 2024-09-14T06:30:27.503 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.9538e-05 2024-09-14T06:30:27.503 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:27.503 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:27.503 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:30:27.503 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.708880+0000", 2024-09-14T06:30:27.503 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.2947e-05 2024-09-14T06:30:27.503 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:27.504 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:30:27.504 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:27.504 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:27.504 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:27.504 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:11467 3.1d 3:bcad7303:::rbd_data.112d92027590.0000000000000001:f [read 237568~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:30:27.504 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:25.656361+0000", 2024-09-14T06:30:27.504 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 61.780952315999997, 2024-09-14T06:30:27.504 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052643898000000001, 2024-09-14T06:30:27.504 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:30:27.504 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:30:27.504 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:30:27.504 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:30:27.504 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:30:27.505 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:27.505 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 11467 2024-09-14T06:30:27.505 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:27.505 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:30:27.505 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:27.505 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:30:27.505 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.656361+0000", 2024-09-14T06:30:27.505 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:27.505 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:27.505 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:27.505 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:30:27.505 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.656361+0000", 2024-09-14T06:30:27.505 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:27.506 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:27.506 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:27.506 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:30:27.506 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.656855+0000", 2024-09-14T06:30:27.506 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00049372000000000001 2024-09-14T06:30:27.506 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:27.506 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:27.506 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:30:27.506 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.656859+0000", 2024-09-14T06:30:27.506 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.1300000000000003e-06 2024-09-14T06:30:27.506 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:27.506 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:27.506 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:30:27.507 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.656863+0000", 2024-09-14T06:30:27.507 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.2629999999999997e-06 2024-09-14T06:30:27.507 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:27.507 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:27.507 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:30:27.507 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.656867+0000", 2024-09-14T06:30:27.507 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.3220000000000001e-06 2024-09-14T06:30:27.507 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:27.507 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:27.507 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:30:27.507 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.708903+0000", 2024-09-14T06:30:27.507 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052035855999999998 2024-09-14T06:30:27.507 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:27.508 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:27.508 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:30:27.508 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.708949+0000", 2024-09-14T06:30:27.508 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.6276999999999998e-05 2024-09-14T06:30:27.508 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:27.508 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:27.508 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:30:27.508 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.709005+0000", 2024-09-14T06:30:27.508 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.6329999999999998e-05 2024-09-14T06:30:27.508 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:27.508 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:30:27.508 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:27.508 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:27.508 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:27.509 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:11479 3.1d 3:bcad7303:::rbd_data.112d92027590.0000000000000001:f [read 360448~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:30:27.509 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:25.658388+0000", 2024-09-14T06:30:27.509 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 61.778925483000002, 2024-09-14T06:30:27.509 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052408920999999997, 2024-09-14T06:30:27.509 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:30:27.509 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:30:27.509 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:30:27.509 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:30:27.509 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:30:27.509 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:27.509 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 11479 2024-09-14T06:30:27.509 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:27.509 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:30:27.510 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:27.510 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:30:27.510 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.658388+0000", 2024-09-14T06:30:27.510 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:27.510 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:27.510 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:27.510 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:30:27.510 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.658388+0000", 2024-09-14T06:30:27.510 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:27.510 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:27.510 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:27.510 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:30:27.510 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.658534+0000", 2024-09-14T06:30:27.510 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.000145613 2024-09-14T06:30:27.510 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:27.511 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:27.511 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:30:27.511 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.658540+0000", 2024-09-14T06:30:27.511 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.6680000000000004e-06 2024-09-14T06:30:27.511 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:27.511 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:27.511 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:30:27.511 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.658548+0000", 2024-09-14T06:30:27.511 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.7980000000000006e-06 2024-09-14T06:30:27.511 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:27.511 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:27.511 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:30:27.511 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.658554+0000", 2024-09-14T06:30:27.511 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.6869999999999999e-06 2024-09-14T06:30:27.512 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:27.512 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:27.512 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:30:27.512 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.710624+0000", 2024-09-14T06:30:27.512 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052070567999999998 2024-09-14T06:30:27.512 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:27.512 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:27.512 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:30:27.512 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.710666+0000", 2024-09-14T06:30:27.512 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.1467000000000001e-05 2024-09-14T06:30:27.512 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:27.512 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:27.512 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:30:27.513 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.710797+0000", 2024-09-14T06:30:27.513 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00013112 2024-09-14T06:30:27.513 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:27.513 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:30:27.513 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:27.513 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:27.513 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:27.513 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:11480 3.1d 3:bcad7303:::rbd_data.112d92027590.0000000000000001:f [read 370688~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:30:27.513 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:25.658563+0000", 2024-09-14T06:30:27.513 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 61.778750551999998, 2024-09-14T06:30:27.513 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052429164, 2024-09-14T06:30:27.513 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:30:27.513 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:30:27.513 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:30:27.514 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:30:27.514 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:30:27.514 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:27.514 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 11480 2024-09-14T06:30:27.514 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:27.514 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:30:27.514 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:27.514 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:30:27.514 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.658563+0000", 2024-09-14T06:30:27.514 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:27.514 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:27.514 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:27.514 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:30:27.514 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.658563+0000", 2024-09-14T06:30:27.514 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:27.515 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:27.515 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:27.515 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:30:27.515 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.658658+0000", 2024-09-14T06:30:27.515 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.5346999999999994e-05 2024-09-14T06:30:27.515 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:27.515 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:27.515 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:30:27.515 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.658662+0000", 2024-09-14T06:30:27.515 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.14e-06 2024-09-14T06:30:27.515 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:27.515 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:27.515 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:30:27.515 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.658669+0000", 2024-09-14T06:30:27.515 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.7379999999999994e-06 2024-09-14T06:30:27.516 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:27.516 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:27.516 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:30:27.516 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.658675+0000", 2024-09-14T06:30:27.516 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.5430000000000001e-06 2024-09-14T06:30:27.516 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:27.516 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:27.516 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:30:27.516 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.710824+0000", 2024-09-14T06:30:27.516 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052148847999999998 2024-09-14T06:30:27.516 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:27.516 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:27.516 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:30:27.517 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.710864+0000", 2024-09-14T06:30:27.517 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.9925999999999997e-05 2024-09-14T06:30:27.517 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:27.517 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:27.517 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:30:27.517 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.710992+0000", 2024-09-14T06:30:27.517 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.000128622 2024-09-14T06:30:27.517 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:27.517 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:30:27.517 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:27.517 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:27.517 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:27.517 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:11481 3.1d 3:bcad7303:::rbd_data.112d92027590.0000000000000001:f [read 380928~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:30:27.517 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:25.658684+0000", 2024-09-14T06:30:27.517 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 61.778629481999999, 2024-09-14T06:30:27.518 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052508565, 2024-09-14T06:30:27.518 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:30:27.518 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:30:27.518 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:30:27.518 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:30:27.518 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:30:27.518 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:27.518 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 11481 2024-09-14T06:30:27.518 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:27.518 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:30:27.518 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:27.518 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:30:27.518 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.658684+0000", 2024-09-14T06:30:27.519 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:27.519 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:27.519 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:27.519 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:30:27.519 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.658684+0000", 2024-09-14T06:30:27.519 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:27.519 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:27.519 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:27.519 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:30:27.519 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.658715+0000", 2024-09-14T06:30:27.519 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.0703000000000001e-05 2024-09-14T06:30:27.519 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:27.519 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:27.519 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:30:27.519 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.658718+0000", 2024-09-14T06:30:27.520 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.0709999999999999e-06 2024-09-14T06:30:27.520 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:27.520 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:27.520 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:30:27.520 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.658725+0000", 2024-09-14T06:30:27.520 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.2339999999999997e-06 2024-09-14T06:30:27.520 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:27.520 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:27.520 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:30:27.520 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.658731+0000", 2024-09-14T06:30:27.520 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.6450000000000002e-06 2024-09-14T06:30:27.520 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:27.520 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:27.521 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:30:27.521 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.711020+0000", 2024-09-14T06:30:27.521 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052288886 2024-09-14T06:30:27.521 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:27.521 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:27.521 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:30:27.521 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.711061+0000", 2024-09-14T06:30:27.521 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.1084000000000002e-05 2024-09-14T06:30:27.521 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:27.521 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:27.521 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:30:27.521 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.711193+0000", 2024-09-14T06:30:27.521 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00013194199999999999 2024-09-14T06:30:27.522 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:27.522 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:30:27.522 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:27.522 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:27.522 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:27.522 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:11482 3.1d 3:bcad7303:::rbd_data.112d92027590.0000000000000001:f [read 391168~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:30:27.522 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:25.658740+0000", 2024-09-14T06:30:27.522 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 61.778574016999997, 2024-09-14T06:30:27.522 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052637015000000002, 2024-09-14T06:30:27.522 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:30:27.522 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:30:27.522 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:30:27.522 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:30:27.522 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:30:27.523 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:27.523 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 11482 2024-09-14T06:30:27.523 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:27.523 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:30:27.523 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:27.523 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:30:27.523 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.658740+0000", 2024-09-14T06:30:27.523 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:27.523 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:27.523 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:27.523 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:30:27.523 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.658740+0000", 2024-09-14T06:30:27.523 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:27.523 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:27.524 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:27.524 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:30:27.524 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.658880+0000", 2024-09-14T06:30:27.524 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00014036100000000001 2024-09-14T06:30:27.524 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:27.524 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:27.524 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:30:27.524 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.658883+0000", 2024-09-14T06:30:27.524 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.3969999999999998e-06 2024-09-14T06:30:27.524 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:27.524 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:27.524 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:30:27.524 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.658890+0000", 2024-09-14T06:30:27.524 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.1500000000000002e-06 2024-09-14T06:30:27.525 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:27.525 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:27.525 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:30:27.525 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.658897+0000", 2024-09-14T06:30:27.525 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.1260000000000003e-06 2024-09-14T06:30:27.525 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:27.525 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:27.525 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:30:27.525 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.711207+0000", 2024-09-14T06:30:27.525 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052310777000000003 2024-09-14T06:30:27.525 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:27.525 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:27.525 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:30:27.525 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.711249+0000", 2024-09-14T06:30:27.526 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.1786999999999998e-05 2024-09-14T06:30:27.526 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:27.526 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:27.526 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:30:27.526 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.711377+0000", 2024-09-14T06:30:27.526 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.000127417 2024-09-14T06:30:27.526 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:27.526 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:30:27.526 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:27.526 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:27.526 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:27.526 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:11483 3.1d 3:bcad7303:::rbd_data.112d92027590.0000000000000001:f [read 401408~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:30:27.526 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:25.658906+0000", 2024-09-14T06:30:27.526 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 61.778407233000003, 2024-09-14T06:30:27.527 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052595109000000001, 2024-09-14T06:30:27.527 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:30:27.527 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:30:27.527 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:30:27.527 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:30:27.527 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:30:27.527 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:27.527 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 11483 2024-09-14T06:30:27.527 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:27.527 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:30:27.527 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:27.528 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:30:27.528 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.658906+0000", 2024-09-14T06:30:27.528 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:27.528 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:27.528 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:27.528 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:30:27.528 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.658906+0000", 2024-09-14T06:30:27.528 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:27.528 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:27.528 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:27.529 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:30:27.529 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.664985+0000", 2024-09-14T06:30:27.529 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.0060787250000000001 2024-09-14T06:30:27.529 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:27.529 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:27.529 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:30:27.529 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.664991+0000", 2024-09-14T06:30:27.529 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.7030000000000003e-06 2024-09-14T06:30:27.529 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:27.529 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:27.529 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:30:27.529 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.664999+0000", 2024-09-14T06:30:27.529 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.4430000000000003e-06 2024-09-14T06:30:27.529 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:27.530 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:27.530 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:30:27.530 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.665006+0000", 2024-09-14T06:30:27.530 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.0840000000000003e-06 2024-09-14T06:30:27.530 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:27.530 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:27.530 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:30:27.530 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.711402+0000", 2024-09-14T06:30:27.530 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046395714999999997 2024-09-14T06:30:27.530 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:27.530 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:27.530 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:30:27.530 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.711443+0000", 2024-09-14T06:30:27.530 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.0911999999999997e-05 2024-09-14T06:30:27.531 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:27.531 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:27.531 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:30:27.531 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.711501+0000", 2024-09-14T06:30:27.531 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.8526999999999997e-05 2024-09-14T06:30:27.531 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:27.531 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:30:27.531 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:27.531 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:27.531 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:30:27.531 INFO:teuthology.orchestra.run.smithi138.stdout:} 2024-09-14T06:30:27.533 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-14T06:30:27.533 INFO:tasks.ceph.osd.3.smithi094.stderr:2024-09-14T06:30:27.531+0000 7feeeafe8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33655) UID: 0 2024-09-14T06:30:27.618 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-14T06:30:27.630 INFO:teuthology.orchestra.run.smithi094.stderr:nodeep-scrub is set 2024-09-14T06:30:27.634 INFO:tasks.ceph.osd.3.smithi094.stderr:2024-09-14T06:30:27.633+0000 7feeeafe8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33655) UID: 0 2024-09-14T06:30:27.639 INFO:teuthology.orchestra.run.smithi138.stdout:{ 2024-09-14T06:30:27.639 INFO:teuthology.orchestra.run.smithi138.stdout: "ops": [], 2024-09-14T06:30:27.640 INFO:teuthology.orchestra.run.smithi138.stdout: "num_ops": 0 2024-09-14T06:30:27.640 INFO:teuthology.orchestra.run.smithi138.stdout:} 2024-09-14T06:30:27.649 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-14T06:30:27.735 INFO:tasks.ceph.osd.6.smithi138.stderr:2024-09-14T06:30:27.734+0000 7f8cd8849640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33620) UID: 0 2024-09-14T06:30:27.751 INFO:teuthology.orchestra.run.smithi138.stdout:{ 2024-09-14T06:30:27.752 INFO:teuthology.orchestra.run.smithi138.stdout: "ops": [], 2024-09-14T06:30:27.752 INFO:teuthology.orchestra.run.smithi138.stdout: "complaint_time": 30, 2024-09-14T06:30:27.752 INFO:teuthology.orchestra.run.smithi138.stdout: "num_blocked_ops": 0 2024-09-14T06:30:27.752 INFO:teuthology.orchestra.run.smithi138.stdout:} 2024-09-14T06:30:27.752 INFO:teuthology.orchestra.run.smithi094.stdout:osd.0: {} 2024-09-14T06:30:27.752 INFO:teuthology.orchestra.run.smithi094.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-14T06:30:27.760 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-14T06:30:27.770 INFO:teuthology.orchestra.run.smithi094.stdout:osd.1: {} 2024-09-14T06:30:27.770 INFO:teuthology.orchestra.run.smithi094.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-14T06:30:27.787 INFO:teuthology.orchestra.run.smithi094.stdout:osd.2: {} 2024-09-14T06:30:27.788 INFO:teuthology.orchestra.run.smithi094.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-14T06:30:27.789 INFO:teuthology.orchestra.run.smithi094.stdout:checking clone #20, image image_client.0-clone20 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent21 2024-09-14T06:30:27.805 INFO:teuthology.orchestra.run.smithi094.stdout:osd.3: {} 2024-09-14T06:30:27.805 INFO:teuthology.orchestra.run.smithi094.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-14T06:30:27.823 INFO:teuthology.orchestra.run.smithi094.stdout:osd.4: {} 2024-09-14T06:30:27.823 INFO:teuthology.orchestra.run.smithi094.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-14T06:30:27.840 INFO:teuthology.orchestra.run.smithi094.stdout:osd.5: {} 2024-09-14T06:30:27.841 INFO:teuthology.orchestra.run.smithi094.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-14T06:30:27.845 INFO:tasks.ceph.osd.5.smithi138.stderr:2024-09-14T06:30:27.844+0000 7fe13746f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33618) UID: 0 2024-09-14T06:30:27.870 INFO:teuthology.orchestra.run.smithi094.stdout:osd.6: {} 2024-09-14T06:30:27.871 INFO:teuthology.orchestra.run.smithi094.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-14T06:30:27.888 INFO:teuthology.orchestra.run.smithi094.stdout:osd.7: {} 2024-09-14T06:30:27.889 INFO:teuthology.orchestra.run.smithi094.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-14T06:30:27.911 INFO:teuthology.orchestra.run.smithi138.stdout:ERROR: (22) Invalid argument 2024-09-14T06:30:27.912 INFO:teuthology.orchestra.run.smithi138.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:30:27.920 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-14T06:30:27.937 INFO:tasks.ceph.osd.3.smithi094.stderr:2024-09-14T06:30:27.936+0000 7feeeafe8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33655) UID: 0 2024-09-14T06:30:28.038 INFO:tasks.ceph.osd.3.smithi094.stderr:2024-09-14T06:30:28.037+0000 7feeeafe8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33655) UID: 0 2024-09-14T06:30:28.063 INFO:teuthology.orchestra.run.smithi138.stdout:ERROR: (22) Invalid argument 2024-09-14T06:30:28.063 INFO:teuthology.orchestra.run.smithi138.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:30:28.072 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-14T06:30:28.075 INFO:teuthology.orchestra.run.smithi094.stdout:3246 trunc from 0xeadcc02 to 0x13bddac 2024-09-14T06:30:28.115 INFO:teuthology.orchestra.run.smithi094.stdout:3249 writesame 0x4cab20e thru 0x4cadd77 data_size 0x1(0x2b6a bytes) 2024-09-14T06:30:28.124 INFO:teuthology.orchestra.run.smithi094.stdout:3251 writesame 0x7f447b7 thru 0x7f4a000 data_size 0x6(0x584a bytes) 2024-09-14T06:30:28.125 INFO:teuthology.orchestra.run.smithi094.stdout:3252 punch from 0x23bdf4 to 0x244bcd, (0x8dd9 bytes) 2024-09-14T06:30:28.136 INFO:teuthology.orchestra.run.smithi094.stdout:3253 writesame 0xc1719af thru 0xc1724f2 data_size 0x4(0xb44 bytes) 2024-09-14T06:30:28.136 INFO:teuthology.orchestra.run.smithi094.stdout:3254 punch from 0x7b2ce14 to 0x7b2cfcf, (0x1bb bytes) 2024-09-14T06:30:28.138 INFO:tasks.ceph.osd.2.smithi094.stderr:2024-09-14T06:30:28.137+0000 7faf3269b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33649) UID: 0 2024-09-14T06:30:28.143 INFO:teuthology.orchestra.run.smithi094.stdout:3255 writesame 0xe174b0d thru 0xe179877 data_size 0x1(0x4d6b bytes) 2024-09-14T06:30:28.151 INFO:teuthology.orchestra.run.smithi094.stdout:3257 trunc from 0xe179878 to 0x7b7ee16 2024-09-14T06:30:28.157 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-14T06:30:28.157 INFO:teuthology.orchestra.run.smithi094.stdout:3259 write 0x2d4a434 thru 0x2d586b9 (0xe286 bytes) 2024-09-14T06:30:28.178 INFO:teuthology.orchestra.run.smithi094.stdout:3260 writesame 0xb0ff8a2 thru 0xb108cd5 data_size 0x23(0x9434 bytes) 2024-09-14T06:30:28.178 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-14T06:30:28.178 INFO:teuthology.orchestra.run.smithi094.stdout:3262 punch from 0x3c13a15 to 0x3c1af97, (0x7582 bytes) 2024-09-14T06:30:28.178 INFO:teuthology.orchestra.run.smithi094.stdout:3263 punch from 0x70fb518 to 0x71093fe, (0xdee6 bytes) 2024-09-14T06:30:28.178 INFO:teuthology.orchestra.run.smithi094.stdout:3264 punch from 0x80212e0 to 0x8021ca2, (0x9c2 bytes) 2024-09-14T06:30:28.178 INFO:teuthology.orchestra.run.smithi094.stdout:3265 write 0x32b7d7f thru 0x32bd849 (0x5acb bytes) 2024-09-14T06:30:28.183 INFO:teuthology.orchestra.run.smithi138.stdout:ERROR: (22) Invalid argument 2024-09-14T06:30:28.184 INFO:teuthology.orchestra.run.smithi138.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:30:28.191 INFO:teuthology.orchestra.run.smithi094.stdout:3267 write 0xdc1f9eb thru 0xdc22138 (0x274e bytes) 2024-09-14T06:30:28.193 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-14T06:30:28.197 INFO:teuthology.orchestra.run.smithi094.stdout:3268 write 0xba795f3 thru 0xba88c82 (0xf690 bytes) 2024-09-14T06:30:28.205 INFO:teuthology.orchestra.run.smithi094.stdout:3269 write 0x973c2bb thru 0x973c544 (0x28a bytes) 2024-09-14T06:30:28.207 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-14T06:30:28.208 INFO:teuthology.orchestra.run.smithi094.stdout:3272 read 0xa3cf2a thru 0xa4ce31 (0xff08 bytes) 2024-09-14T06:30:28.209 INFO:teuthology.orchestra.run.smithi094.stdout:3273 punch from 0x5f1f062 to 0x5f2cee0, (0xde7e bytes) 2024-09-14T06:30:28.209 INFO:teuthology.orchestra.run.smithi094.stdout:3274 read 0x9e3ac70 thru 0x9e42dd3 (0x8164 bytes) 2024-09-14T06:30:28.209 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-14T06:30:28.209 INFO:teuthology.orchestra.run.smithi094.stdout:3276 write 0x47f9e1d thru 0x47fc352 (0x2536 bytes) 2024-09-14T06:30:28.213 INFO:teuthology.orchestra.run.smithi094.stdout:3277 write 0xd0bf2a6 thru 0xd0cb264 (0xbfbf bytes) 2024-09-14T06:30:28.221 INFO:teuthology.orchestra.run.smithi094.stdout:3278 writesame 0x1a15184 thru 0x1a21c66 data_size 0x57(0xcae3 bytes) 2024-09-14T06:30:28.229 INFO:teuthology.orchestra.run.smithi094.stdout:3279 read 0x4ff6daf thru 0x5006193 (0xf3e5 bytes) 2024-09-14T06:30:28.230 INFO:teuthology.orchestra.run.smithi094.stdout:3280 read 0x3851487 thru 0x3860ab4 (0xf62e bytes) 2024-09-14T06:30:28.230 INFO:teuthology.orchestra.run.smithi094.stdout:3281 read 0x8e2201b thru 0x8e264ef (0x44d5 bytes) 2024-09-14T06:30:28.230 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-14T06:30:28.235 INFO:teuthology.orchestra.run.smithi094.stdout:3283 trunc from 0xdc22139 to 0x93993e0 2024-09-14T06:30:28.239 INFO:tasks.ceph.osd.6.smithi138.stderr:2024-09-14T06:30:28.238+0000 7f8cd8849640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33620) UID: 0 2024-09-14T06:30:28.249 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-14T06:30:28.249 INFO:teuthology.orchestra.run.smithi094.stdout:3286 write 0x65109af thru 0x65187c3 (0x7e15 bytes) 2024-09-14T06:30:28.255 INFO:teuthology.orchestra.run.smithi094.stdout:3288 read 0x11d90bc thru 0x11dad05 (0x1c4a bytes) 2024-09-14T06:30:28.268 INFO:teuthology.orchestra.run.smithi094.stdout:3293 writesame 0xea2e897 thru 0xea3758d data_size 0x17(0x8cf7 bytes) 2024-09-14T06:30:28.268 INFO:teuthology.orchestra.run.smithi094.stdout:3294 write 0xa31c57f thru 0xa3241f0 (0x7c72 bytes) 2024-09-14T06:30:28.278 INFO:teuthology.orchestra.run.smithi094.stdout:3295 read 0xa87a4b8 thru 0xa87f4db (0x5024 bytes) 2024-09-14T06:30:28.279 INFO:teuthology.orchestra.run.smithi094.stdout:3297 writesame 0x8e7e17e thru 0x8e8c392 data_size 0x1(0xe215 bytes) 2024-09-14T06:30:28.298 INFO:teuthology.orchestra.run.smithi094.stdout:3298 write 0x23151f9 thru 0x2319caa (0x4ab2 bytes) 2024-09-14T06:30:28.306 INFO:teuthology.orchestra.run.smithi094.stdout:3300 read 0xdc89e71 thru 0xdc8f7da (0x596a bytes) 2024-09-14T06:30:28.306 INFO:teuthology.orchestra.run.smithi094.stdout:3301 punch from 0x2216d68 to 0x2217af8, (0xd90 bytes) 2024-09-14T06:30:28.306 INFO:teuthology.orchestra.run.smithi094.stdout:3302 read 0x93216e9 thru 0x9321f4e (0x866 bytes) 2024-09-14T06:30:28.307 INFO:teuthology.orchestra.run.smithi094.stdout:3303 write 0xbd733c0 thru 0xbd73993 (0x5d4 bytes) 2024-09-14T06:30:28.309 INFO:teuthology.orchestra.run.smithi138.stdout:ERROR: (22) Invalid argument 2024-09-14T06:30:28.309 INFO:teuthology.orchestra.run.smithi138.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:30:28.309 INFO:teuthology.orchestra.run.smithi094.stdout:3304 write 0xb7f9759 thru 0xb8060d4 (0xc97c bytes) 2024-09-14T06:30:28.318 INFO:teuthology.orchestra.run.smithi094.stdout:3305 punch from 0xbaa2f8f to 0xbaab03a, (0x80ab bytes) 2024-09-14T06:30:28.318 INFO:teuthology.orchestra.run.smithi094.stdout:3306 writesame 0xb2f5550 thru 0xb2f7e7c data_size 0x1(0x292d bytes) 2024-09-14T06:30:28.320 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-14T06:30:28.328 INFO:teuthology.orchestra.run.smithi094.stdout:3308 trunc from 0xea3758e to 0x9f31956 2024-09-14T06:30:28.342 INFO:teuthology.orchestra.run.smithi094.stdout:3309 read 0x72086a2 thru 0x720aa1d (0x237c bytes) 2024-09-14T06:30:28.344 INFO:teuthology.orchestra.run.smithi094.stdout:3310 trunc from 0x9f31956 to 0x88226e9 2024-09-14T06:30:28.355 INFO:teuthology.orchestra.run.smithi094.stdout:3312 read 0x31a891e thru 0x31b2d52 (0xa435 bytes) 2024-09-14T06:30:28.361 INFO:tasks.ceph.osd.5.smithi138.stderr:2024-09-14T06:30:28.360+0000 7fe13746f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33618) UID: 0 2024-09-14T06:30:28.363 INFO:teuthology.orchestra.run.smithi094.stdout:3313 trunc from 0x88226e9 to 0xea85cb6 2024-09-14T06:30:28.368 INFO:teuthology.orchestra.run.smithi094.stdout:3314 write 0xba3acff thru 0xba41e81 (0x7183 bytes) 2024-09-14T06:30:28.373 INFO:teuthology.orchestra.run.smithi094.stdout:3316 write 0x9972751 thru 0x9974f80 (0x2830 bytes) 2024-09-14T06:30:28.376 INFO:teuthology.orchestra.run.smithi094.stdout:3317 clone 23 order 23 su 1048576 sc 15 2024-09-14T06:30:28.434 INFO:teuthology.orchestra.run.smithi094.stdout:ERROR: (22) Invalid argument 2024-09-14T06:30:28.434 INFO:teuthology.orchestra.run.smithi094.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:30:28.441 INFO:tasks.ceph.osd.7.smithi138.stderr:2024-09-14T06:30:28.439+0000 7f3c6a21d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33623) UID: 0 2024-09-14T06:30:28.442 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-14T06:30:28.541 INFO:tasks.ceph.osd.0.smithi094.stderr:2024-09-14T06:30:28.540+0000 7f07c9333640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33636) UID: 0 2024-09-14T06:30:28.586 INFO:teuthology.orchestra.run.smithi094.stdout:ERROR: (22) Invalid argument 2024-09-14T06:30:28.587 INFO:teuthology.orchestra.run.smithi094.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:30:28.595 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-14T06:30:28.632 INFO:teuthology.orchestra.run.smithi094.stderr:set 3.a pg_upmap_items mapping to [2->3,0->6] 2024-09-14T06:30:28.642 INFO:tasks.ceph.osd.5.smithi138.stderr:2024-09-14T06:30:28.641+0000 7fe13746f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33618) UID: 0 2024-09-14T06:30:28.740 INFO:teuthology.orchestra.run.smithi094.stdout:ERROR: (22) Invalid argument 2024-09-14T06:30:28.740 INFO:teuthology.orchestra.run.smithi094.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:30:28.742 INFO:tasks.ceph.osd.3.smithi094.stderr:2024-09-14T06:30:28.741+0000 7feeeafe8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33655) UID: 0 2024-09-14T06:30:28.748 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-14T06:30:28.843 INFO:tasks.ceph.osd.7.smithi138.stderr:2024-09-14T06:30:28.842+0000 7f3c6a21d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33623) UID: 0 2024-09-14T06:30:28.891 INFO:teuthology.orchestra.run.smithi094.stdout:ERROR: (22) Invalid argument 2024-09-14T06:30:28.891 INFO:teuthology.orchestra.run.smithi094.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:30:28.900 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-14T06:30:28.943 INFO:tasks.ceph.osd.6.smithi138.stderr:2024-09-14T06:30:28.943+0000 7f8cd8849640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33620) UID: 0 2024-09-14T06:30:29.027 INFO:teuthology.orchestra.run.smithi094.stdout:ERROR: (22) Invalid argument 2024-09-14T06:30:29.027 INFO:teuthology.orchestra.run.smithi094.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:30:29.027 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-14T06:30:29.072 INFO:tasks.ceph.osd.0.smithi094.stderr:2024-09-14T06:30:29.070+0000 7f07c9333640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33636) UID: 0 2024-09-14T06:30:29.130 INFO:teuthology.orchestra.run.smithi094.stdout:ERROR: (22) Invalid argument 2024-09-14T06:30:29.130 INFO:teuthology.orchestra.run.smithi094.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:30:29.138 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-14T06:30:29.146 INFO:tasks.ceph.osd.4.smithi138.stderr:2024-09-14T06:30:29.145+0000 7f6c653c8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33615) UID: 0 2024-09-14T06:30:29.246 INFO:tasks.ceph.osd.5.smithi138.stderr:2024-09-14T06:30:29.245+0000 7fe13746f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33618) UID: 0 2024-09-14T06:30:29.281 INFO:teuthology.orchestra.run.smithi094.stdout:ERROR: (22) Invalid argument 2024-09-14T06:30:29.281 INFO:teuthology.orchestra.run.smithi094.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:30:29.289 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-14T06:30:29.347 INFO:tasks.ceph.osd.6.smithi138.stderr:2024-09-14T06:30:29.346+0000 7f8cd8849640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33620) UID: 0 2024-09-14T06:30:29.391 INFO:teuthology.orchestra.run.smithi094.stdout:ERROR: (22) Invalid argument 2024-09-14T06:30:29.391 INFO:teuthology.orchestra.run.smithi094.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:30:29.399 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-14T06:30:29.448 INFO:tasks.ceph.osd.1.smithi094.stderr:2024-09-14T06:30:29.446+0000 7f328e47a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33641) UID: 0 2024-09-14T06:30:29.502 INFO:teuthology.orchestra.run.smithi094.stdout:ERROR: (22) Invalid argument 2024-09-14T06:30:29.502 INFO:teuthology.orchestra.run.smithi094.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:30:29.511 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-14T06:30:29.548 INFO:tasks.ceph.osd.4.smithi138.stderr:2024-09-14T06:30:29.548+0000 7f6c653c8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33615) UID: 0 2024-09-14T06:30:29.650 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-14T06:30:29.651 INFO:tasks.ceph.osd.2.smithi094.stderr:2024-09-14T06:30:29.648+0000 7faf3269b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33649) UID: 0 2024-09-14T06:30:29.668 INFO:teuthology.orchestra.run.smithi094.stdout:ERROR: (22) Invalid argument 2024-09-14T06:30:29.668 INFO:teuthology.orchestra.run.smithi094.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:30:29.687 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-14T06:30:29.751 INFO:tasks.ceph.osd.6.smithi138.stderr:2024-09-14T06:30:29.750+0000 7f8cd8849640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33620) UID: 0 2024-09-14T06:30:29.839 INFO:teuthology.orchestra.run.smithi094.stdout:ERROR: (22) Invalid argument 2024-09-14T06:30:29.839 INFO:teuthology.orchestra.run.smithi094.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:30:29.848 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-14T06:30:29.852 INFO:tasks.ceph.osd.7.smithi138.stderr:2024-09-14T06:30:29.851+0000 7f3c6a21d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33623) UID: 0 2024-09-14T06:30:29.901 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-14T06:30:29.952 INFO:tasks.ceph.osd.4.smithi138.stderr:2024-09-14T06:30:29.952+0000 7f6c653c8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33615) UID: 0 2024-09-14T06:30:29.953 INFO:teuthology.orchestra.run.smithi094.stdout:ERROR: (22) Invalid argument 2024-09-14T06:30:29.953 INFO:teuthology.orchestra.run.smithi094.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:30:29.962 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-14T06:30:30.023 INFO:teuthology.orchestra.run.smithi094.stdout:osd.0: {} 2024-09-14T06:30:30.024 INFO:teuthology.orchestra.run.smithi094.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-14T06:30:30.040 INFO:teuthology.orchestra.run.smithi094.stdout:osd.1: {} 2024-09-14T06:30:30.040 INFO:teuthology.orchestra.run.smithi094.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-14T06:30:30.053 INFO:tasks.ceph.osd.1.smithi094.stderr:2024-09-14T06:30:30.051+0000 7f328e47a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33641) UID: 0 2024-09-14T06:30:30.056 INFO:teuthology.orchestra.run.smithi094.stdout:osd.2: {} 2024-09-14T06:30:30.056 INFO:teuthology.orchestra.run.smithi094.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-14T06:30:30.068 INFO:teuthology.orchestra.run.smithi138.stdout:ERROR: (22) Invalid argument 2024-09-14T06:30:30.068 INFO:teuthology.orchestra.run.smithi138.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:30:30.071 INFO:teuthology.orchestra.run.smithi094.stdout:osd.3: {} 2024-09-14T06:30:30.072 INFO:teuthology.orchestra.run.smithi094.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-14T06:30:30.077 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-14T06:30:30.088 INFO:teuthology.orchestra.run.smithi094.stdout:osd.4: {} 2024-09-14T06:30:30.088 INFO:teuthology.orchestra.run.smithi094.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-14T06:30:30.104 INFO:teuthology.orchestra.run.smithi094.stdout:osd.5: {} 2024-09-14T06:30:30.105 INFO:teuthology.orchestra.run.smithi094.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-14T06:30:30.121 INFO:teuthology.orchestra.run.smithi094.stdout:osd.6: {} 2024-09-14T06:30:30.121 INFO:teuthology.orchestra.run.smithi094.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-14T06:30:30.137 INFO:teuthology.orchestra.run.smithi094.stdout:osd.7: {} 2024-09-14T06:30:30.137 INFO:teuthology.orchestra.run.smithi094.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-14T06:30:30.180 INFO:teuthology.orchestra.run.smithi138.stdout:{ 2024-09-14T06:30:30.181 INFO:teuthology.orchestra.run.smithi138.stdout: "ops": [], 2024-09-14T06:30:30.181 INFO:teuthology.orchestra.run.smithi138.stdout: "complaint_time": 30, 2024-09-14T06:30:30.181 INFO:teuthology.orchestra.run.smithi138.stdout: "num_blocked_ops": 0 2024-09-14T06:30:30.181 INFO:teuthology.orchestra.run.smithi138.stdout:} 2024-09-14T06:30:30.189 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-14T06:30:30.190 INFO:tasks.ceph.osd.0.smithi094.stderr:2024-09-14T06:30:30.187+0000 7f07c9333640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33636) UID: 0 2024-09-14T06:30:30.254 INFO:tasks.ceph.osd.1.smithi094.stderr:2024-09-14T06:30:30.253+0000 7f328e47a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33641) UID: 0 2024-09-14T06:30:30.294 INFO:teuthology.orchestra.run.smithi138.stdout:{ 2024-09-14T06:30:30.294 INFO:teuthology.orchestra.run.smithi138.stdout: "size": 20, 2024-09-14T06:30:30.294 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 600, 2024-09-14T06:30:30.294 INFO:teuthology.orchestra.run.smithi138.stdout: "ops": [ 2024-09-14T06:30:30.294 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:30.294 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:19328 3.3 3:c34e5e51:::rbd_data.112d92027590.0000000000000024:f [read 1926144~1024 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:30:30.294 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:27.059768+0000", 2024-09-14T06:30:30.294 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 63.232392933, 2024-09-14T06:30:30.294 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046369147999999999, 2024-09-14T06:30:30.294 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:30:30.295 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:30:30.295 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:30:30.295 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:30:30.295 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:30:30.295 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:30.295 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 19328 2024-09-14T06:30:30.295 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:30.295 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:30:30.295 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:30.295 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:30:30.295 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.059768+0000", 2024-09-14T06:30:30.295 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:30.296 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:30.296 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:30.296 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:30:30.296 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.059768+0000", 2024-09-14T06:30:30.296 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:30.296 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:30.296 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:30.296 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:30:30.296 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.059874+0000", 2024-09-14T06:30:30.296 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00010551400000000001 2024-09-14T06:30:30.296 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:30.296 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:30.296 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:30:30.296 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.059877+0000", 2024-09-14T06:30:30.297 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.8279999999999999e-06 2024-09-14T06:30:30.297 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:30.297 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:30.297 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:30:30.297 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.059884+0000", 2024-09-14T06:30:30.297 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.6389999999999992e-06 2024-09-14T06:30:30.297 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:30.297 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:30.297 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:30:30.297 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.059890+0000", 2024-09-14T06:30:30.297 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.49e-06 2024-09-14T06:30:30.297 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:30.297 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:30.298 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:30:30.298 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.105804+0000", 2024-09-14T06:30:30.298 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.045914242000000001 2024-09-14T06:30:30.298 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:30.298 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:30.298 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:30:30.298 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.105886+0000", 2024-09-14T06:30:30.298 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.2305999999999994e-05 2024-09-14T06:30:30.298 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:30.299 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:30.299 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:30:30.299 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.106138+0000", 2024-09-14T06:30:30.299 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.000251129 2024-09-14T06:30:30.299 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:30.299 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:30:30.299 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:30.299 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:30.299 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:30.299 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:19329 3.3 3:c34e5e51:::rbd_data.112d92027590.0000000000000024:f [read 1936384~1024 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:30:30.299 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:27.059899+0000", 2024-09-14T06:30:30.299 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 63.232262409999997, 2024-09-14T06:30:30.299 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046572122000000001, 2024-09-14T06:30:30.299 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:30:30.300 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:30:30.300 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:30:30.300 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:30:30.300 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:30:30.300 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:30.300 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 19329 2024-09-14T06:30:30.300 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:30.300 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:30:30.300 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:30.300 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:30:30.300 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.059899+0000", 2024-09-14T06:30:30.300 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:30.300 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:30.301 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:30.301 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:30:30.301 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.059899+0000", 2024-09-14T06:30:30.301 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:30.301 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:30.301 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:30.301 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:30:30.301 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062501+0000", 2024-09-14T06:30:30.301 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.0026016419999999999 2024-09-14T06:30:30.301 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:30.301 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:30.301 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:30:30.301 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062514+0000", 2024-09-14T06:30:30.301 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.2947e-05 2024-09-14T06:30:30.302 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:30.302 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:30.302 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:30:30.302 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062523+0000", 2024-09-14T06:30:30.302 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.5519999999999993e-06 2024-09-14T06:30:30.302 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:30.302 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:30.302 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:30:30.302 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062534+0000", 2024-09-14T06:30:30.302 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.1416e-05 2024-09-14T06:30:30.302 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:30.302 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:30.302 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:30:30.302 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.106179+0000", 2024-09-14T06:30:30.303 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.043644049999999997 2024-09-14T06:30:30.303 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:30.303 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:30.303 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:30:30.303 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.106262+0000", 2024-09-14T06:30:30.303 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.3178999999999998e-05 2024-09-14T06:30:30.303 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:30.303 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:30.303 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:30:30.303 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.106471+0000", 2024-09-14T06:30:30.303 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00020933599999999999 2024-09-14T06:30:30.303 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:30.303 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:30:30.303 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:30.304 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:30.304 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:30.304 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:19351 3.3 3:c34e5e51:::rbd_data.112d92027590.0000000000000024:f [read 1159168~1024 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:30:30.304 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:27.062766+0000", 2024-09-14T06:30:30.304 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 63.229395185000001, 2024-09-14T06:30:30.304 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046099454999999998, 2024-09-14T06:30:30.304 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:30:30.304 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:30:30.304 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:30:30.304 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:30:30.304 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:30:30.304 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:30.304 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 19351 2024-09-14T06:30:30.304 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:30.304 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:30:30.305 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:30.305 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:30:30.305 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062766+0000", 2024-09-14T06:30:30.305 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:30.305 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:30.305 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:30.305 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:30:30.305 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062766+0000", 2024-09-14T06:30:30.305 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:30.305 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:30.305 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:30.305 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:30:30.305 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062766+0000", 2024-09-14T06:30:30.305 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.6e-07 2024-09-14T06:30:30.306 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:30.306 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:30.306 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:30:30.306 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062767+0000", 2024-09-14T06:30:30.306 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.0070000000000001e-06 2024-09-14T06:30:30.306 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:30.306 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:30.306 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:30:30.306 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062785+0000", 2024-09-14T06:30:30.306 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.7884999999999999e-05 2024-09-14T06:30:30.306 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:30.306 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:30.306 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:30:30.306 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062790+0000", 2024-09-14T06:30:30.306 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.0900000000000004e-06 2024-09-14T06:30:30.307 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:30.307 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:30.307 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:30:30.307 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.108735+0000", 2024-09-14T06:30:30.307 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.045944411999999997 2024-09-14T06:30:30.307 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:30.307 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:30.307 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:30:30.307 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.108809+0000", 2024-09-14T06:30:30.307 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.4157000000000002e-05 2024-09-14T06:30:30.307 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:30.307 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:30.307 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:30:30.307 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.108866+0000", 2024-09-14T06:30:30.307 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.6644000000000002e-05 2024-09-14T06:30:30.307 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:30.308 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:30:30.308 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:30.308 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:30.308 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:30.308 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:19352 3.3 3:c34e5e51:::rbd_data.112d92027590.0000000000000024:f [read 1169408~1024 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:30:30.308 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:27.062798+0000", 2024-09-14T06:30:30.308 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 63.229363773999999, 2024-09-14T06:30:30.308 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046255900000000003, 2024-09-14T06:30:30.308 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:30:30.308 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:30:30.308 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:30:30.308 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:30:30.308 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:30:30.308 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:30.309 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 19352 2024-09-14T06:30:30.309 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:30.309 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:30:30.309 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:30.309 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:30:30.309 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062798+0000", 2024-09-14T06:30:30.309 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:30.309 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:30.309 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:30.309 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:30:30.309 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062798+0000", 2024-09-14T06:30:30.309 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:30.309 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:30.309 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:30.310 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:30:30.310 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062798+0000", 2024-09-14T06:30:30.310 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.37e-07 2024-09-14T06:30:30.310 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:30.310 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:30.310 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:30:30.310 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062800+0000", 2024-09-14T06:30:30.310 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.0269999999999998e-06 2024-09-14T06:30:30.310 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:30.310 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:30.310 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:30:30.310 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062806+0000", 2024-09-14T06:30:30.310 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.6690000000000003e-06 2024-09-14T06:30:30.310 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:30.310 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:30.311 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:30:30.311 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062810+0000", 2024-09-14T06:30:30.311 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.3449999999999999e-06 2024-09-14T06:30:30.311 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:30.311 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:30.311 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:30:30.311 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.108892+0000", 2024-09-14T06:30:30.311 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046081715000000002 2024-09-14T06:30:30.311 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:30.311 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:30.311 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:30:30.311 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.108940+0000", 2024-09-14T06:30:30.311 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.7815999999999998e-05 2024-09-14T06:30:30.311 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:30.311 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:30.312 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:30:30.312 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.109053+0000", 2024-09-14T06:30:30.312 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.000113791 2024-09-14T06:30:30.312 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:30.312 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:30:30.312 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:30.312 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:30.312 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:30.312 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:19353 3.3 3:c34e5e51:::rbd_data.112d92027590.0000000000000024:f [read 1179648~1024 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:30:30.312 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:27.062816+0000", 2024-09-14T06:30:30.312 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 63.229345254999998, 2024-09-14T06:30:30.312 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046481256999999998, 2024-09-14T06:30:30.312 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:30:30.312 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:30:30.312 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:30:30.313 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:30:30.313 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:30:30.313 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:30.313 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 19353 2024-09-14T06:30:30.313 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:30.313 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:30:30.313 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:30.313 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:30:30.313 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062816+0000", 2024-09-14T06:30:30.313 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:30.313 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:30.313 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:30.313 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:30:30.313 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062816+0000", 2024-09-14T06:30:30.314 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:30.314 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:30.314 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:30.314 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:30:30.314 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062816+0000", 2024-09-14T06:30:30.314 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.2899999999999999e-07 2024-09-14T06:30:30.314 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:30.314 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:30.314 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:30:30.314 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062818+0000", 2024-09-14T06:30:30.314 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.699e-06 2024-09-14T06:30:30.314 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:30.314 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:30.314 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:30:30.314 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062825+0000", 2024-09-14T06:30:30.315 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.878e-06 2024-09-14T06:30:30.315 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:30.315 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:30.315 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:30:30.315 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062830+0000", 2024-09-14T06:30:30.315 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.5410000000000002e-06 2024-09-14T06:30:30.315 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:30.315 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:30.315 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:30:30.315 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.109096+0000", 2024-09-14T06:30:30.315 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046266517 2024-09-14T06:30:30.315 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:30.315 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:30.315 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:30:30.316 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.109190+0000", 2024-09-14T06:30:30.316 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.4000999999999997e-05 2024-09-14T06:30:30.316 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:30.316 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:30.316 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:30:30.316 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.109297+0000", 2024-09-14T06:30:30.316 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.000107292 2024-09-14T06:30:30.316 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:30.316 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:30:30.316 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:30.316 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:30.316 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:30.316 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:19354 3.3 3:c34e5e51:::rbd_data.112d92027590.0000000000000024:f [read 1189888~1024 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:30:30.316 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:27.062842+0000", 2024-09-14T06:30:30.317 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 63.229319799999999, 2024-09-14T06:30:30.317 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046594257, 2024-09-14T06:30:30.317 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:30:30.317 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:30:30.317 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:30:30.317 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:30:30.317 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:30:30.317 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:30.317 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 19354 2024-09-14T06:30:30.317 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:30.317 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:30:30.317 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:30.317 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:30:30.317 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062842+0000", 2024-09-14T06:30:30.318 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:30.318 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:30.318 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:30.318 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:30:30.318 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062842+0000", 2024-09-14T06:30:30.318 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:30.318 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:30.318 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:30.318 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:30:30.318 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062842+0000", 2024-09-14T06:30:30.318 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.7399999999999999e-07 2024-09-14T06:30:30.318 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:30.318 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:30.318 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:30:30.318 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062859+0000", 2024-09-14T06:30:30.319 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.7010000000000001e-05 2024-09-14T06:30:30.319 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:30.319 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:30.319 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:30:30.319 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062866+0000", 2024-09-14T06:30:30.319 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.9990000000000002e-06 2024-09-14T06:30:30.319 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:30.319 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:30.319 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:30:30.319 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062873+0000", 2024-09-14T06:30:30.319 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.8569999999999997e-06 2024-09-14T06:30:30.319 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:30.319 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:30.319 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:30:30.319 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.109335+0000", 2024-09-14T06:30:30.320 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046461943999999998 2024-09-14T06:30:30.320 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:30.320 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:30.320 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:30:30.320 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.109377+0000", 2024-09-14T06:30:30.320 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.2558999999999997e-05 2024-09-14T06:30:30.320 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:30.320 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:30.320 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:30:30.320 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.109436+0000", 2024-09-14T06:30:30.320 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.8514000000000003e-05 2024-09-14T06:30:30.320 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:30.320 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:30:30.320 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:30.321 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:30.321 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:30.321 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:19355 3.3 3:c34e5e51:::rbd_data.112d92027590.0000000000000024:f [read 1200128~1024 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:30:30.321 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:27.062884+0000", 2024-09-14T06:30:30.321 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 63.229277127000003, 2024-09-14T06:30:30.321 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046681650999999998, 2024-09-14T06:30:30.321 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:30:30.321 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:30:30.321 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:30:30.321 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:30:30.321 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:30:30.321 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:30.321 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 19355 2024-09-14T06:30:30.321 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:30.322 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:30:30.322 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:30.322 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:30:30.322 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062884+0000", 2024-09-14T06:30:30.322 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:30.322 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:30.322 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:30.322 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:30:30.322 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062884+0000", 2024-09-14T06:30:30.322 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:30.322 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:30.322 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:30.322 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:30:30.322 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062885+0000", 2024-09-14T06:30:30.323 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.8000000000000001e-07 2024-09-14T06:30:30.323 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:30.323 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:30.323 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:30:30.323 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062886+0000", 2024-09-14T06:30:30.323 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.8920000000000001e-06 2024-09-14T06:30:30.323 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:30.323 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:30.323 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:30:30.323 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062892+0000", 2024-09-14T06:30:30.323 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.84e-06 2024-09-14T06:30:30.323 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:30.323 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:30.324 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:30:30.324 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062899+0000", 2024-09-14T06:30:30.324 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.5339999999999996e-06 2024-09-14T06:30:30.324 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:30.324 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:30.324 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:30:30.324 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.109466+0000", 2024-09-14T06:30:30.324 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046566919999999998 2024-09-14T06:30:30.324 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:30.324 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:30.324 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:30:30.324 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.109510+0000", 2024-09-14T06:30:30.324 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.4153000000000002e-05 2024-09-14T06:30:30.324 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:30.325 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:30.325 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:30:30.325 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.109566+0000", 2024-09-14T06:30:30.325 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.5931999999999999e-05 2024-09-14T06:30:30.325 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:30.325 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:30:30.325 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:30.325 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:30.325 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:30.325 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:19356 3.3 3:c34e5e51:::rbd_data.112d92027590.0000000000000024:f [read 1210368~1024 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:30:30.325 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:27.062908+0000", 2024-09-14T06:30:30.325 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 63.22925369, 2024-09-14T06:30:30.325 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046807498000000003, 2024-09-14T06:30:30.325 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:30:30.326 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:30:30.326 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:30:30.326 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:30:30.326 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:30:30.326 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:30.326 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 19356 2024-09-14T06:30:30.326 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:30.326 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:30:30.326 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:30.326 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:30:30.326 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062908+0000", 2024-09-14T06:30:30.326 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:30.326 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:30.327 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:30.327 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:30:30.327 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062908+0000", 2024-09-14T06:30:30.327 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:30.327 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:30.327 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:30.327 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:30:30.327 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062908+0000", 2024-09-14T06:30:30.327 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.5699999999999998e-07 2024-09-14T06:30:30.327 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:30.327 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:30.327 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:30:30.327 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062910+0000", 2024-09-14T06:30:30.327 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.7710000000000001e-06 2024-09-14T06:30:30.328 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:30.328 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:30.328 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:30:30.328 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062920+0000", 2024-09-14T06:30:30.328 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.0654e-05 2024-09-14T06:30:30.328 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:30.328 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:30.328 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:30:30.328 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062931+0000", 2024-09-14T06:30:30.328 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.075e-05 2024-09-14T06:30:30.328 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:30.328 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:30.328 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:30:30.328 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.109595+0000", 2024-09-14T06:30:30.329 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046663713000000002 2024-09-14T06:30:30.329 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:30.329 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:30.329 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:30:30.329 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.109632+0000", 2024-09-14T06:30:30.329 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.7505000000000001e-05 2024-09-14T06:30:30.329 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:30.329 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:30.329 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:30:30.329 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.109715+0000", 2024-09-14T06:30:30.329 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.2748000000000001e-05 2024-09-14T06:30:30.329 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:30.329 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:30:30.330 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:30.330 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:30.330 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:30.330 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:19357 3.3 3:c34e5e51:::rbd_data.112d92027590.0000000000000024:f [read 1220608~1024 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:30:30.330 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:27.062939+0000", 2024-09-14T06:30:30.330 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 63.229222241999999, 2024-09-14T06:30:30.330 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046920508, 2024-09-14T06:30:30.330 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:30:30.330 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:30:30.330 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:30:30.330 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:30:30.330 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:30:30.330 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:30.330 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 19357 2024-09-14T06:30:30.331 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:30.331 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:30:30.331 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:30.331 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:30:30.331 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062939+0000", 2024-09-14T06:30:30.331 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:30.331 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:30.331 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:30.331 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:30:30.331 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062939+0000", 2024-09-14T06:30:30.331 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:30.331 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:30.332 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:30.332 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:30:30.332 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062940+0000", 2024-09-14T06:30:30.332 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.32e-07 2024-09-14T06:30:30.332 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:30.332 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:30.332 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:30:30.332 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062942+0000", 2024-09-14T06:30:30.332 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.1919999999999999e-06 2024-09-14T06:30:30.332 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:30.332 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:30.332 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:30:30.332 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062948+0000", 2024-09-14T06:30:30.332 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.1149999999999999e-06 2024-09-14T06:30:30.333 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:30.333 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:30.333 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:30:30.333 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062960+0000", 2024-09-14T06:30:30.333 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.1895e-05 2024-09-14T06:30:30.333 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:30.333 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:30.333 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:30:30.333 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.109704+0000", 2024-09-14T06:30:30.333 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046744584999999998 2024-09-14T06:30:30.333 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:30.333 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:30.333 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:30:30.333 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.109777+0000", 2024-09-14T06:30:30.334 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.2282999999999999e-05 2024-09-14T06:30:30.334 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:30.334 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:30.334 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:30:30.334 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.109860+0000", 2024-09-14T06:30:30.334 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.3005999999999998e-05 2024-09-14T06:30:30.334 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:30.334 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:30:30.334 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:30.334 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:30.334 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:30.334 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:19358 3.3 3:c34e5e51:::rbd_data.112d92027590.0000000000000024:f [read 1230848~1024 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:30:30.334 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:27.062967+0000", 2024-09-14T06:30:30.334 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 63.229194612000001, 2024-09-14T06:30:30.335 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047144471, 2024-09-14T06:30:30.335 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:30:30.335 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:30:30.335 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:30:30.335 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:30:30.335 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:30:30.335 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:30.335 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 19358 2024-09-14T06:30:30.335 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:30.335 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:30:30.335 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:30.335 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:30:30.335 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062967+0000", 2024-09-14T06:30:30.335 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:30.335 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:30.336 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:30.336 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:30:30.336 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062967+0000", 2024-09-14T06:30:30.336 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:30.336 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:30.336 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:30.336 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:30:30.336 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062967+0000", 2024-09-14T06:30:30.336 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.89e-07 2024-09-14T06:30:30.336 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:30.336 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:30.336 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:30:30.336 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062969+0000", 2024-09-14T06:30:30.336 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.9929999999999998e-06 2024-09-14T06:30:30.337 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:30.337 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:30.337 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:30:30.337 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062975+0000", 2024-09-14T06:30:30.337 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.9030000000000001e-06 2024-09-14T06:30:30.337 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:30.337 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:30.337 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:30:30.337 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062979+0000", 2024-09-14T06:30:30.337 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.2250000000000002e-06 2024-09-14T06:30:30.338 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:30.338 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:30.338 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:30:30.338 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.109934+0000", 2024-09-14T06:30:30.338 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046954410000000002 2024-09-14T06:30:30.338 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:30.338 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:30.338 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:30:30.338 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.110007+0000", 2024-09-14T06:30:30.338 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.3829000000000001e-05 2024-09-14T06:30:30.338 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:30.338 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:30.338 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:30:30.338 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.110111+0000", 2024-09-14T06:30:30.339 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00010372199999999999 2024-09-14T06:30:30.339 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:30.339 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:30:30.339 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:30.339 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:30.339 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:30.339 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:19359 3.3 3:c34e5e51:::rbd_data.112d92027590.0000000000000024:f [read 1241088~1024 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:30:30.339 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:27.062987+0000", 2024-09-14T06:30:30.339 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 63.229174522000001, 2024-09-14T06:30:30.339 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047297182, 2024-09-14T06:30:30.339 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:30:30.339 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:30:30.339 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:30:30.340 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:30:30.340 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:30:30.340 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:30.340 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 19359 2024-09-14T06:30:30.340 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:30.340 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:30:30.340 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:30.340 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:30:30.340 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062987+0000", 2024-09-14T06:30:30.340 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:30.340 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:30.340 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:30.340 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:30:30.341 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062987+0000", 2024-09-14T06:30:30.341 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:30.341 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:30.341 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:30.341 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:30:30.341 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062987+0000", 2024-09-14T06:30:30.341 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.2500000000000001e-07 2024-09-14T06:30:30.341 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:30.341 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:30.341 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:30:30.341 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062989+0000", 2024-09-14T06:30:30.341 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.6080000000000001e-06 2024-09-14T06:30:30.341 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:30.341 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:30.342 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:30:30.342 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062994+0000", 2024-09-14T06:30:30.342 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.5520000000000004e-06 2024-09-14T06:30:30.342 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:30.342 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:30.342 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:30:30.342 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062999+0000", 2024-09-14T06:30:30.342 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.3930000000000001e-06 2024-09-14T06:30:30.342 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:30.342 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:30.342 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:30:30.342 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.110142+0000", 2024-09-14T06:30:30.342 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047143154 2024-09-14T06:30:30.342 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:30.343 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:30.343 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:30:30.343 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.110219+0000", 2024-09-14T06:30:30.343 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.6737999999999996e-05 2024-09-14T06:30:30.343 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:30.343 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:30.343 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:30:30.343 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.110284+0000", 2024-09-14T06:30:30.343 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.5412000000000003e-05 2024-09-14T06:30:30.343 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:30.343 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:30:30.343 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:30.344 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:30.344 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:30.344 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:19360 3.3 3:c34e5e51:::rbd_data.112d92027590.0000000000000024:f [read 1251328~1024 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:30:30.344 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:27.063017+0000", 2024-09-14T06:30:30.344 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 63.229144061, 2024-09-14T06:30:30.344 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047453823999999999, 2024-09-14T06:30:30.344 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:30:30.344 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:30:30.344 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:30:30.344 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:30:30.344 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:30:30.344 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:30.344 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 19360 2024-09-14T06:30:30.345 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:30.345 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:30:30.345 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:30.345 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:30:30.345 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.063017+0000", 2024-09-14T06:30:30.345 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:30.345 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:30.345 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:30.345 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:30:30.345 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.063017+0000", 2024-09-14T06:30:30.345 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:30.345 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:30.345 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:30.346 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:30:30.346 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.064977+0000", 2024-09-14T06:30:30.346 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.0019597080000000001 2024-09-14T06:30:30.346 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:30.346 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:30.346 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:30:30.346 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.064987+0000", 2024-09-14T06:30:30.346 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.9620000000000005e-06 2024-09-14T06:30:30.346 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:30.346 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:30.346 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:30:30.346 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065000+0000", 2024-09-14T06:30:30.346 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.3173000000000001e-05 2024-09-14T06:30:30.346 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:30.346 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:30.347 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:30:30.347 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065005+0000", 2024-09-14T06:30:30.347 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.9960000000000001e-06 2024-09-14T06:30:30.347 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:30.347 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:30.347 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:30:30.347 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.110314+0000", 2024-09-14T06:30:30.347 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.04530899 2024-09-14T06:30:30.347 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:30.347 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:30.347 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:30:30.347 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.110398+0000", 2024-09-14T06:30:30.347 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.3539000000000001e-05 2024-09-14T06:30:30.347 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:30.347 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:30.348 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:30:30.348 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.110471+0000", 2024-09-14T06:30:30.348 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.3455999999999996e-05 2024-09-14T06:30:30.348 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:30.348 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:30:30.348 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:30.348 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:30.348 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:30.348 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:19365 3.3 3:c34e5e51:::rbd_data.112d92027590.0000000000000024:f [read 1302528~1024 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:30:30.348 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:27.065058+0000", 2024-09-14T06:30:30.348 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 63.227103391, 2024-09-14T06:30:30.348 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046105526000000001, 2024-09-14T06:30:30.348 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:30:30.349 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:30:30.349 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:30:30.349 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:30:30.349 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:30:30.349 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:30.349 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 19365 2024-09-14T06:30:30.349 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:30.349 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:30:30.349 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:30.349 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:30:30.349 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065058+0000", 2024-09-14T06:30:30.349 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:30.349 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:30.349 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:30.350 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:30:30.350 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065058+0000", 2024-09-14T06:30:30.350 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:30.350 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:30.350 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:30.350 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:30:30.350 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065058+0000", 2024-09-14T06:30:30.350 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.1899999999999999e-07 2024-09-14T06:30:30.350 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:30.350 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:30.350 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:30:30.350 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065059+0000", 2024-09-14T06:30:30.350 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.1299999999999998e-07 2024-09-14T06:30:30.350 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:30.351 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:30.351 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:30:30.351 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065063+0000", 2024-09-14T06:30:30.351 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.1099999999999996e-06 2024-09-14T06:30:30.351 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:30.351 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:30.351 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:30:30.351 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065070+0000", 2024-09-14T06:30:30.351 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.973e-06 2024-09-14T06:30:30.351 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:30.351 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:30.351 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:30:30.351 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.111014+0000", 2024-09-14T06:30:30.351 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.045943373000000003 2024-09-14T06:30:30.351 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:30.352 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:30.352 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:30:30.352 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.111073+0000", 2024-09-14T06:30:30.352 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.8987999999999998e-05 2024-09-14T06:30:30.352 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:30.352 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:30.352 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:30:30.352 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.111163+0000", 2024-09-14T06:30:30.352 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.0950000000000002e-05 2024-09-14T06:30:30.352 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:30.352 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:30:30.352 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:30.352 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:30.352 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:30.353 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:19366 3.3 3:c34e5e51:::rbd_data.112d92027590.0000000000000024:f [read 1312768~1024 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:30:30.353 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:27.065074+0000", 2024-09-14T06:30:30.353 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 63.227087541000003, 2024-09-14T06:30:30.353 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046289858000000003, 2024-09-14T06:30:30.353 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:30:30.353 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:30:30.353 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:30:30.353 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:30:30.353 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:30:30.353 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:30.353 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 19366 2024-09-14T06:30:30.353 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:30.354 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:30:30.354 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:30.354 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:30:30.354 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065074+0000", 2024-09-14T06:30:30.354 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:30.354 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:30.354 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:30.354 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:30:30.354 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065074+0000", 2024-09-14T06:30:30.354 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:30.354 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:30.354 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:30.354 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:30:30.354 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065074+0000", 2024-09-14T06:30:30.355 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.3900000000000001e-07 2024-09-14T06:30:30.355 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:30.355 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:30.355 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:30:30.355 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065078+0000", 2024-09-14T06:30:30.355 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.3229999999999999e-06 2024-09-14T06:30:30.355 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:30.355 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:30.355 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:30:30.355 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065082+0000", 2024-09-14T06:30:30.355 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.8299999999999998e-06 2024-09-14T06:30:30.355 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:30.355 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:30.355 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:30:30.355 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065085+0000", 2024-09-14T06:30:30.356 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.8660000000000002e-06 2024-09-14T06:30:30.356 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:30.356 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:30.356 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:30:30.356 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.111270+0000", 2024-09-14T06:30:30.356 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046185035999999999 2024-09-14T06:30:30.356 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:30.356 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:30.356 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:30:30.356 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.111311+0000", 2024-09-14T06:30:30.356 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.0935e-05 2024-09-14T06:30:30.356 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:30.356 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:30.356 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:30:30.357 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.111364+0000", 2024-09-14T06:30:30.357 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.2629e-05 2024-09-14T06:30:30.357 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:30.357 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:30:30.357 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:30.357 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:30.357 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:30.357 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:19367 3.3 3:c34e5e51:::rbd_data.112d92027590.0000000000000024:f [read 1323008~1024 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:30:30.357 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:27.065115+0000", 2024-09-14T06:30:30.357 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 63.22704598, 2024-09-14T06:30:30.357 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046407003000000002, 2024-09-14T06:30:30.357 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:30:30.357 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:30:30.357 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:30:30.358 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:30:30.358 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:30:30.358 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:30.358 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 19367 2024-09-14T06:30:30.358 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:30.358 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:30:30.358 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:30.358 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:30:30.358 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065115+0000", 2024-09-14T06:30:30.358 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:30.358 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:30.358 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:30.358 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:30:30.358 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065115+0000", 2024-09-14T06:30:30.359 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:30.359 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:30.359 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:30.359 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:30:30.359 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065116+0000", 2024-09-14T06:30:30.359 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.4999999999999998e-07 2024-09-14T06:30:30.359 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:30.359 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:30.359 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:30:30.359 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065117+0000", 2024-09-14T06:30:30.359 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.189e-06 2024-09-14T06:30:30.359 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:30.359 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:30.359 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:30:30.360 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065122+0000", 2024-09-14T06:30:30.360 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.3709999999999999e-06 2024-09-14T06:30:30.360 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:30.360 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:30.360 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:30:30.360 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065125+0000", 2024-09-14T06:30:30.360 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.8940000000000002e-06 2024-09-14T06:30:30.360 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:30.360 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:30.360 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:30:30.360 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.111402+0000", 2024-09-14T06:30:30.360 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046276645999999998 2024-09-14T06:30:30.360 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:30.361 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:30.361 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:30:30.361 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.111445+0000", 2024-09-14T06:30:30.361 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.3288000000000003e-05 2024-09-14T06:30:30.361 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:30.361 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:30.361 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:30:30.361 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.111522+0000", 2024-09-14T06:30:30.361 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.7164999999999997e-05 2024-09-14T06:30:30.361 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:30.361 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:30:30.361 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:30.361 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:30.361 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:30.362 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:19368 3.3 3:c34e5e51:::rbd_data.112d92027590.0000000000000024:f [read 1333248~1024 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:30:30.362 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:27.065130+0000", 2024-09-14T06:30:30.362 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 63.227031689999997, 2024-09-14T06:30:30.362 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046507275000000001, 2024-09-14T06:30:30.362 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:30:30.362 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:30:30.362 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:30:30.362 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:30:30.362 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:30:30.362 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:30.362 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 19368 2024-09-14T06:30:30.362 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:30.362 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:30:30.362 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:30.363 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:30:30.363 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065130+0000", 2024-09-14T06:30:30.363 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:30.363 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:30.363 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:30.363 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:30:30.363 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065130+0000", 2024-09-14T06:30:30.363 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:30.363 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:30.363 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:30.363 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:30:30.363 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065130+0000", 2024-09-14T06:30:30.363 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.0800000000000001e-07 2024-09-14T06:30:30.363 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:30.363 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:30.364 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:30:30.364 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065131+0000", 2024-09-14T06:30:30.364 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.8700000000000004e-07 2024-09-14T06:30:30.364 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:30.364 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:30.364 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:30:30.364 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065134+0000", 2024-09-14T06:30:30.364 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.1039999999999998e-06 2024-09-14T06:30:30.364 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:30.364 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:30.364 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:30:30.364 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065136+0000", 2024-09-14T06:30:30.364 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.3949999999999999e-06 2024-09-14T06:30:30.364 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:30.365 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:30.365 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:30:30.365 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.111541+0000", 2024-09-14T06:30:30.365 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046404206000000003 2024-09-14T06:30:30.365 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:30.365 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:30.365 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:30:30.365 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.111582+0000", 2024-09-14T06:30:30.365 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.1051999999999999e-05 2024-09-14T06:30:30.365 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:30.365 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:30.365 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:30:30.365 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.111637+0000", 2024-09-14T06:30:30.365 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.5322999999999999e-05 2024-09-14T06:30:30.366 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:30.366 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:30:30.366 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:30.366 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:30.366 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:30.366 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:19369 3.3 3:c34e5e51:::rbd_data.112d92027590.0000000000000024:f [read 1343488~1024 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:30:30.366 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:27.065139+0000", 2024-09-14T06:30:30.366 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 63.227021872999998, 2024-09-14T06:30:30.366 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046682169000000003, 2024-09-14T06:30:30.366 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:30:30.366 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:30:30.366 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:30:30.366 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:30:30.367 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:30:30.367 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:30.367 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 19369 2024-09-14T06:30:30.367 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:30.367 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:30:30.367 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:30.367 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:30:30.367 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065139+0000", 2024-09-14T06:30:30.367 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:30.367 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:30.367 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:30.367 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:30:30.367 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065139+0000", 2024-09-14T06:30:30.367 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:30.368 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:30.368 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:30.368 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:30:30.368 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065140+0000", 2024-09-14T06:30:30.368 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.8299999999999998e-07 2024-09-14T06:30:30.368 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:30.368 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:30.368 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:30:30.368 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065141+0000", 2024-09-14T06:30:30.368 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.3900000000000003e-07 2024-09-14T06:30:30.368 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:30.368 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:30.368 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:30:30.368 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065144+0000", 2024-09-14T06:30:30.369 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.7409999999999998e-06 2024-09-14T06:30:30.369 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:30.369 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:30.369 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:30:30.369 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065147+0000", 2024-09-14T06:30:30.369 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.4150000000000002e-06 2024-09-14T06:30:30.369 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:30.369 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:30.369 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:30:30.369 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.111673+0000", 2024-09-14T06:30:30.369 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046526004000000003 2024-09-14T06:30:30.369 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:30.369 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:30.369 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:30:30.370 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.111749+0000", 2024-09-14T06:30:30.370 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.5741999999999994e-05 2024-09-14T06:30:30.370 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:30.370 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:30.370 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:30:30.370 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.111822+0000", 2024-09-14T06:30:30.370 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.3045000000000003e-05 2024-09-14T06:30:30.370 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:30.370 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:30:30.370 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:30.370 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:30.370 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:30.370 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:19370 3.3 3:c34e5e51:::rbd_data.112d92027590.0000000000000024:f [read 1353728~1024 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:30:30.371 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:27.065150+0000", 2024-09-14T06:30:30.371 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 63.227011144000002, 2024-09-14T06:30:30.371 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046897556999999999, 2024-09-14T06:30:30.371 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:30:30.371 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:30:30.371 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:30:30.371 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:30:30.371 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:30:30.371 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:30.371 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 19370 2024-09-14T06:30:30.371 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:30.371 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:30:30.371 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:30.371 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:30:30.372 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065150+0000", 2024-09-14T06:30:30.372 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:30.372 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:30.372 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:30.372 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:30:30.372 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065150+0000", 2024-09-14T06:30:30.372 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:30.372 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:30.372 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:30.372 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:30:30.372 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065150+0000", 2024-09-14T06:30:30.372 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.0200000000000001e-07 2024-09-14T06:30:30.373 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:30.373 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:30.373 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:30:30.373 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065151+0000", 2024-09-14T06:30:30.373 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.1100000000000004e-07 2024-09-14T06:30:30.373 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:30.373 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:30.373 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:30:30.373 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065154+0000", 2024-09-14T06:30:30.373 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.9639999999999999e-06 2024-09-14T06:30:30.373 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:30.373 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:30.373 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:30:30.374 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065157+0000", 2024-09-14T06:30:30.374 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.4260000000000002e-06 2024-09-14T06:30:30.374 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:30.374 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:30.374 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:30:30.374 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.111861+0000", 2024-09-14T06:30:30.374 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046704268 2024-09-14T06:30:30.374 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:30.374 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:30.374 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:30:30.374 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.111939+0000", 2024-09-14T06:30:30.374 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.8231000000000004e-05 2024-09-14T06:30:30.374 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:30.374 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:30.375 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:30:30.375 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.112048+0000", 2024-09-14T06:30:30.375 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00010855500000000001 2024-09-14T06:30:30.375 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:30.375 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:30:30.375 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:30.375 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:30.375 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:30.375 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:19371 3.3 3:c34e5e51:::rbd_data.112d92027590.0000000000000024:f [read 1363968~1024 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:30:30.375 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:27.065161+0000", 2024-09-14T06:30:30.375 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 63.227000494000002, 2024-09-14T06:30:30.375 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047021944000000003, 2024-09-14T06:30:30.375 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:30:30.375 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:30:30.376 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:30:30.376 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:30:30.376 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:30:30.376 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:30.376 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 19371 2024-09-14T06:30:30.376 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:30.376 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:30:30.376 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:30.376 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:30:30.376 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065161+0000", 2024-09-14T06:30:30.376 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:30.376 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:30.376 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:30.377 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:30:30.377 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065161+0000", 2024-09-14T06:30:30.377 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:30.377 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:30.377 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:30.377 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:30:30.377 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065161+0000", 2024-09-14T06:30:30.377 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.0200000000000001e-07 2024-09-14T06:30:30.377 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:30.377 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:30.377 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:30:30.377 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065166+0000", 2024-09-14T06:30:30.377 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.9869999999999999e-06 2024-09-14T06:30:30.377 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:30.378 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:30.378 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:30:30.378 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065169+0000", 2024-09-14T06:30:30.378 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.3799999999999998e-06 2024-09-14T06:30:30.378 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:30.378 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:30.378 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:30:30.378 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065172+0000", 2024-09-14T06:30:30.378 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.4830000000000002e-06 2024-09-14T06:30:30.378 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:30.378 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:30.378 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:30:30.378 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.112071+0000", 2024-09-14T06:30:30.378 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046899529000000002 2024-09-14T06:30:30.378 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:30.379 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:30.379 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:30:30.379 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.112128+0000", 2024-09-14T06:30:30.379 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.6341999999999997e-05 2024-09-14T06:30:30.379 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:30.379 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:30.379 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:30:30.379 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.112183+0000", 2024-09-14T06:30:30.379 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.5021000000000001e-05 2024-09-14T06:30:30.379 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:30.379 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:30:30.379 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:30.379 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:30.380 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:30.380 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:19372 3.3 3:c34e5e51:::rbd_data.112d92027590.0000000000000024:f [read 1374208~1024 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:30:30.380 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:27.065176+0000", 2024-09-14T06:30:30.380 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 63.226985343999999, 2024-09-14T06:30:30.380 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047188889999999997, 2024-09-14T06:30:30.380 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:30:30.380 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:30:30.380 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:30:30.380 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:30:30.380 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:30:30.380 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:30.380 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 19372 2024-09-14T06:30:30.380 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:30.381 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:30:30.381 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:30.381 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:30:30.381 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065176+0000", 2024-09-14T06:30:30.381 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:30.381 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:30.381 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:30.381 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:30:30.381 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065176+0000", 2024-09-14T06:30:30.381 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:30.381 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:30.381 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:30.381 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:30:30.381 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.071117+0000", 2024-09-14T06:30:30.382 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.0059404990000000001 2024-09-14T06:30:30.382 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:30.382 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:30.382 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:30:30.382 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.071120+0000", 2024-09-14T06:30:30.382 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.4340000000000001e-06 2024-09-14T06:30:30.382 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:30.382 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:30.382 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:30:30.382 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.071129+0000", 2024-09-14T06:30:30.382 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.8389999999999996e-06 2024-09-14T06:30:30.382 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:30.382 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:30.382 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:30:30.383 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.071134+0000", 2024-09-14T06:30:30.383 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.412e-06 2024-09-14T06:30:30.383 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:30.383 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:30.383 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:30:30.383 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.112221+0000", 2024-09-14T06:30:30.383 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.041086473999999998 2024-09-14T06:30:30.383 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:30.383 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:30.383 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:30:30.383 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.112299+0000", 2024-09-14T06:30:30.383 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.8746999999999997e-05 2024-09-14T06:30:30.383 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:30.383 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:30.384 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:30:30.384 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.112365+0000", 2024-09-14T06:30:30.384 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.5485000000000001e-05 2024-09-14T06:30:30.384 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:30.384 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:30:30.384 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:30.384 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:30.384 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:30:30.384 INFO:teuthology.orchestra.run.smithi138.stdout:} 2024-09-14T06:30:30.385 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-14T06:30:30.386 INFO:tasks.ceph.osd.3.smithi094.stderr:2024-09-14T06:30:30.384+0000 7feeeafe8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33655) UID: 0 2024-09-14T06:30:30.486 INFO:tasks.ceph.osd.0.smithi094.stderr:2024-09-14T06:30:30.485+0000 7f07c9333640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33636) UID: 0 2024-09-14T06:30:30.490 INFO:teuthology.orchestra.run.smithi138.stdout:{ 2024-09-14T06:30:30.490 INFO:teuthology.orchestra.run.smithi138.stdout: "ops": [], 2024-09-14T06:30:30.490 INFO:teuthology.orchestra.run.smithi138.stdout: "num_ops": 0 2024-09-14T06:30:30.490 INFO:teuthology.orchestra.run.smithi138.stdout:} 2024-09-14T06:30:30.499 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-14T06:30:30.588 INFO:tasks.ceph.osd.5.smithi138.stderr:2024-09-14T06:30:30.586+0000 7fe13746f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33618) UID: 0 2024-09-14T06:30:30.589 INFO:teuthology.orchestra.run.smithi094.stderr:noscrub is unset 2024-09-14T06:30:30.606 INFO:teuthology.orchestra.run.smithi138.stdout:{ 2024-09-14T06:30:30.606 INFO:teuthology.orchestra.run.smithi138.stdout: "ops": [], 2024-09-14T06:30:30.606 INFO:teuthology.orchestra.run.smithi138.stdout: "complaint_time": 30, 2024-09-14T06:30:30.606 INFO:teuthology.orchestra.run.smithi138.stdout: "num_blocked_ops": 0 2024-09-14T06:30:30.606 INFO:teuthology.orchestra.run.smithi138.stdout:} 2024-09-14T06:30:30.615 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-14T06:30:30.700 INFO:tasks.ceph.osd.7.smithi138.stderr:2024-09-14T06:30:30.699+0000 7f3c6a21d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33623) UID: 0 2024-09-14T06:30:30.745 INFO:teuthology.orchestra.run.smithi094.stdout:checking clone #21, image image_client.0-clone21 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent22 2024-09-14T06:30:30.763 INFO:teuthology.orchestra.run.smithi138.stdout:{ 2024-09-14T06:30:30.763 INFO:teuthology.orchestra.run.smithi138.stdout: "size": 20, 2024-09-14T06:30:30.763 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 600, 2024-09-14T06:30:30.763 INFO:teuthology.orchestra.run.smithi138.stdout: "ops": [ 2024-09-14T06:30:30.763 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:30.763 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:11453 3.1d 3:bcad7303:::rbd_data.112d92027590.0000000000000001:f [read 94208~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:30:30.763 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:25.654375+0000", 2024-09-14T06:30:30.763 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 65.106882153000001, 2024-09-14T06:30:30.764 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052449318000000002, 2024-09-14T06:30:30.764 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:30:30.764 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:30:30.764 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:30:30.764 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:30:30.764 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:30:30.764 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:30.764 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 11453 2024-09-14T06:30:30.764 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:30.764 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:30:30.764 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:30.764 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:30:30.764 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654375+0000", 2024-09-14T06:30:30.765 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:30.765 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:30.765 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:30.765 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:30:30.765 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654375+0000", 2024-09-14T06:30:30.765 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:30.765 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:30.765 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:30.765 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:30:30.765 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654496+0000", 2024-09-14T06:30:30.765 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00012081 2024-09-14T06:30:30.765 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:30.765 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:30.765 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:30:30.766 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654497+0000", 2024-09-14T06:30:30.766 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.8789999999999999e-06 2024-09-14T06:30:30.766 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:30.766 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:30.766 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:30:30.766 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654501+0000", 2024-09-14T06:30:30.766 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.8999999999999999e-06 2024-09-14T06:30:30.766 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:30.766 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:30.766 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:30:30.766 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654505+0000", 2024-09-14T06:30:30.766 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.2229999999999999e-06 2024-09-14T06:30:30.766 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:30.766 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:30.767 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:30:30.767 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.705917+0000", 2024-09-14T06:30:30.767 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.051412501999999999 2024-09-14T06:30:30.767 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:30.767 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:30.767 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:30:30.767 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.705999+0000", 2024-09-14T06:30:30.767 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.1551999999999994e-05 2024-09-14T06:30:30.767 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:30.767 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:30.767 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:30:30.767 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.706824+0000", 2024-09-14T06:30:30.767 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00082545199999999996 2024-09-14T06:30:30.767 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:30.768 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:30:30.768 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:30.768 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:30.768 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:30.768 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:11454 3.1d 3:bcad7303:::rbd_data.112d92027590.0000000000000001:f [read 104448~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:30:30.768 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:25.654510+0000", 2024-09-14T06:30:30.768 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 65.106747362999997, 2024-09-14T06:30:30.768 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052610713000000003, 2024-09-14T06:30:30.768 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:30:30.768 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:30:30.768 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:30:30.768 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:30:30.768 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:30:30.768 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:30.769 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 11454 2024-09-14T06:30:30.769 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:30.769 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:30:30.769 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:30.769 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:30:30.769 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654510+0000", 2024-09-14T06:30:30.769 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:30.769 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:30.769 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:30.769 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:30:30.769 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654510+0000", 2024-09-14T06:30:30.769 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:30.769 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:30.769 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:30.770 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:30:30.770 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654825+0000", 2024-09-14T06:30:30.770 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00031522400000000002 2024-09-14T06:30:30.770 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:30.770 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:30.770 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:30:30.770 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654827+0000", 2024-09-14T06:30:30.770 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.9709999999999998e-06 2024-09-14T06:30:30.770 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:30.770 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:30.770 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:30:30.770 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654832+0000", 2024-09-14T06:30:30.771 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.4099999999999999e-06 2024-09-14T06:30:30.771 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:30.771 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:30.771 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:30:30.771 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654835+0000", 2024-09-14T06:30:30.771 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.14e-06 2024-09-14T06:30:30.771 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:30.771 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:30.771 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:30:30.771 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.706851+0000", 2024-09-14T06:30:30.771 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052015761000000001 2024-09-14T06:30:30.772 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:30.772 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:30.772 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:30:30.772 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.706934+0000", 2024-09-14T06:30:30.772 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.2484999999999994e-05 2024-09-14T06:30:30.772 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:30.772 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:30.772 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:30:30.772 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.707120+0000", 2024-09-14T06:30:30.772 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00018672200000000001 2024-09-14T06:30:30.772 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:30.772 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:30:30.772 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:30.772 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:30.773 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:30.773 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:11455 3.1d 3:bcad7303:::rbd_data.112d92027590.0000000000000001:f [read 114688~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:30:30.773 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:25.654841+0000", 2024-09-14T06:30:30.773 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 65.106416034000006, 2024-09-14T06:30:30.773 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052579898, 2024-09-14T06:30:30.773 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:30:30.773 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:30:30.773 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:30:30.773 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:30:30.773 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:30:30.773 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:30.773 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 11455 2024-09-14T06:30:30.773 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:30.773 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:30:30.773 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:30.774 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:30:30.774 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654841+0000", 2024-09-14T06:30:30.774 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:30.774 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:30.774 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:30.774 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:30:30.774 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654841+0000", 2024-09-14T06:30:30.774 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:30.774 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:30.774 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:30.774 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:30:30.774 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654841+0000", 2024-09-14T06:30:30.774 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.7300000000000002e-07 2024-09-14T06:30:30.774 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:30.775 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:30.775 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:30:30.775 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654845+0000", 2024-09-14T06:30:30.775 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.0520000000000003e-06 2024-09-14T06:30:30.775 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:30.775 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:30.775 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:30:30.775 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654849+0000", 2024-09-14T06:30:30.775 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.0459999999999999e-06 2024-09-14T06:30:30.775 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:30.775 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:30.775 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:30:30.775 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654852+0000", 2024-09-14T06:30:30.775 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.9830000000000001e-06 2024-09-14T06:30:30.775 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:30.776 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:30.776 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:30:30.776 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.707159+0000", 2024-09-14T06:30:30.776 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052306826000000001 2024-09-14T06:30:30.776 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:30.776 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:30.776 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:30:30.776 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.707242+0000", 2024-09-14T06:30:30.776 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.2595000000000005e-05 2024-09-14T06:30:30.776 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:30.776 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:30.776 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:30:30.776 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.707421+0000", 2024-09-14T06:30:30.776 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.000179123 2024-09-14T06:30:30.776 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:30.777 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:30:30.777 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:30.777 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:30.777 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:30.777 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:11456 3.1d 3:bcad7303:::rbd_data.112d92027590.0000000000000001:f [read 124928~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:30:30.777 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:25.654856+0000", 2024-09-14T06:30:30.777 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 65.106401188000007, 2024-09-14T06:30:30.777 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052801580000000001, 2024-09-14T06:30:30.777 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:30:30.777 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:30:30.777 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:30:30.777 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:30:30.777 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:30:30.778 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:30.778 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 11456 2024-09-14T06:30:30.778 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:30.778 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:30:30.778 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:30.778 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:30:30.778 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654856+0000", 2024-09-14T06:30:30.778 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:30.778 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:30.778 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:30.778 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:30:30.778 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654856+0000", 2024-09-14T06:30:30.778 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:30.779 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:30.779 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:30.779 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:30:30.779 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654856+0000", 2024-09-14T06:30:30.779 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.29e-07 2024-09-14T06:30:30.779 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:30.779 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:30.779 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:30:30.779 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654857+0000", 2024-09-14T06:30:30.779 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.187e-06 2024-09-14T06:30:30.779 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:30.779 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:30.779 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:30:30.779 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654861+0000", 2024-09-14T06:30:30.780 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.3850000000000002e-06 2024-09-14T06:30:30.780 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:30.780 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:30.780 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:30:30.780 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654863+0000", 2024-09-14T06:30:30.780 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.9500000000000001e-06 2024-09-14T06:30:30.780 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:30.780 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:30.780 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:30:30.780 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.707456+0000", 2024-09-14T06:30:30.780 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052592434 2024-09-14T06:30:30.780 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:30.780 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:30.780 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:30:30.781 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.707516+0000", 2024-09-14T06:30:30.781 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.0167999999999997e-05 2024-09-14T06:30:30.781 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:30.781 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:30.781 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:30:30.781 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.707657+0000", 2024-09-14T06:30:30.781 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00014122700000000001 2024-09-14T06:30:30.781 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:30.781 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:30:30.781 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:30.781 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:30.781 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:30.781 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:11457 3.1d 3:bcad7303:::rbd_data.112d92027590.0000000000000001:f [read 135168~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:30:30.781 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:25.654867+0000", 2024-09-14T06:30:30.782 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 65.106390302999998, 2024-09-14T06:30:30.782 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052921902, 2024-09-14T06:30:30.782 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:30:30.782 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:30:30.782 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:30:30.782 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:30:30.782 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:30:30.782 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:30.782 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 11457 2024-09-14T06:30:30.782 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:30.782 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:30:30.783 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:30.783 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:30:30.783 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654867+0000", 2024-09-14T06:30:30.783 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:30.783 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:30.783 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:30.783 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:30:30.783 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654867+0000", 2024-09-14T06:30:30.783 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:30.783 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:30.784 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:30.784 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:30:30.784 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654972+0000", 2024-09-14T06:30:30.784 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00010584 2024-09-14T06:30:30.784 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:30.784 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:30.784 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:30:30.784 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654975+0000", 2024-09-14T06:30:30.784 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.1169999999999998e-06 2024-09-14T06:30:30.784 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:30.784 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:30.785 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:30:30.785 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654979+0000", 2024-09-14T06:30:30.785 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.1960000000000001e-06 2024-09-14T06:30:30.785 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:30.785 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:30.785 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:30:30.785 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654982+0000", 2024-09-14T06:30:30.785 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.2990000000000001e-06 2024-09-14T06:30:30.785 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:30.785 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:30.785 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:30:30.785 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.707684+0000", 2024-09-14T06:30:30.785 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052701979000000003 2024-09-14T06:30:30.785 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:30.786 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:30.786 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:30:30.786 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.707728+0000", 2024-09-14T06:30:30.786 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.3529999999999998e-05 2024-09-14T06:30:30.786 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:30.786 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:30.786 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:30:30.786 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.707789+0000", 2024-09-14T06:30:30.786 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.0940999999999998e-05 2024-09-14T06:30:30.786 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:30.786 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:30:30.786 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:30.786 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:30.786 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:30.787 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:11458 3.1d 3:bcad7303:::rbd_data.112d92027590.0000000000000001:f [read 145408~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:30:30.787 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:25.654987+0000", 2024-09-14T06:30:30.787 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 65.106269905000005, 2024-09-14T06:30:30.787 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.05291655, 2024-09-14T06:30:30.787 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:30:30.787 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:30:30.787 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:30:30.787 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:30:30.787 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:30:30.787 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:30.788 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 11458 2024-09-14T06:30:30.788 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:30.788 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:30:30.788 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:30.788 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:30:30.788 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654987+0000", 2024-09-14T06:30:30.788 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:30.788 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:30.788 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:30.788 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:30:30.788 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654987+0000", 2024-09-14T06:30:30.788 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:30.788 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:30.788 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:30.789 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:30:30.789 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655077+0000", 2024-09-14T06:30:30.789 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.0253000000000005e-05 2024-09-14T06:30:30.789 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:30.789 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:30.789 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:30:30.789 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655079+0000", 2024-09-14T06:30:30.789 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.9989999999999998e-06 2024-09-14T06:30:30.789 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:30.789 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:30.790 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:30:30.790 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655083+0000", 2024-09-14T06:30:30.790 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.9280000000000003e-06 2024-09-14T06:30:30.790 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:30.790 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:30.791 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:30:30.791 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655086+0000", 2024-09-14T06:30:30.791 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.056e-06 2024-09-14T06:30:30.791 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:30.791 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:30.791 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:30:30.791 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.707805+0000", 2024-09-14T06:30:30.792 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052718627999999997 2024-09-14T06:30:30.792 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:30.792 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:30.792 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:30:30.792 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.707848+0000", 2024-09-14T06:30:30.792 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.2728999999999997e-05 2024-09-14T06:30:30.792 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:30.792 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:30.792 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:30:30.792 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.707904+0000", 2024-09-14T06:30:30.792 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.5957e-05 2024-09-14T06:30:30.792 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:30.792 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:30:30.793 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:30.793 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:30.793 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:30.793 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:11459 3.1d 3:bcad7303:::rbd_data.112d92027590.0000000000000001:f [read 155648~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:30:30.793 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:25.655091+0000", 2024-09-14T06:30:30.793 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 65.106165896999997, 2024-09-14T06:30:30.793 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052944242000000002, 2024-09-14T06:30:30.793 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:30:30.793 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:30:30.793 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:30:30.793 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:30:30.793 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:30:30.793 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:30.793 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 11459 2024-09-14T06:30:30.794 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:30.794 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:30:30.794 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:30.794 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:30:30.794 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655091+0000", 2024-09-14T06:30:30.794 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:30.794 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:30.794 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:30.794 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:30:30.794 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655091+0000", 2024-09-14T06:30:30.794 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:30.794 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:30.794 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:30.794 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:30:30.794 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655237+0000", 2024-09-14T06:30:30.795 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00014570100000000001 2024-09-14T06:30:30.795 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:30.795 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:30.795 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:30:30.795 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655239+0000", 2024-09-14T06:30:30.795 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.8670000000000001e-06 2024-09-14T06:30:30.795 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:30.795 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:30.795 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:30:30.795 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655243+0000", 2024-09-14T06:30:30.795 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.9829999999999998e-06 2024-09-14T06:30:30.795 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:30.795 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:30.795 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:30:30.796 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655246+0000", 2024-09-14T06:30:30.796 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.1540000000000002e-06 2024-09-14T06:30:30.796 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:30.796 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:30.796 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:30:30.796 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.707934+0000", 2024-09-14T06:30:30.796 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052688286000000001 2024-09-14T06:30:30.796 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:30.796 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:30.796 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:30:30.796 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.707977+0000", 2024-09-14T06:30:30.796 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.3291999999999998e-05 2024-09-14T06:30:30.796 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:30.796 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:30.796 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:30:30.797 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.708035+0000", 2024-09-14T06:30:30.797 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.7958999999999999e-05 2024-09-14T06:30:30.797 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:30.797 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:30:30.797 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:30.797 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:30.797 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:30.797 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:11460 3.1d 3:bcad7303:::rbd_data.112d92027590.0000000000000001:f [read 165888~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:30:30.797 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:25.655250+0000", 2024-09-14T06:30:30.797 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 65.106006600000001, 2024-09-14T06:30:30.798 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052910698999999999, 2024-09-14T06:30:30.798 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:30:30.798 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:30:30.798 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:30:30.798 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:30:30.798 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:30:30.798 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:30.798 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 11460 2024-09-14T06:30:30.798 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:30.798 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:30:30.798 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:30.798 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:30:30.798 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655250+0000", 2024-09-14T06:30:30.798 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:30.799 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:30.799 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:30.799 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:30:30.799 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655250+0000", 2024-09-14T06:30:30.799 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:30.799 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:30.799 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:30.799 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:30:30.799 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655356+0000", 2024-09-14T06:30:30.799 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00010535900000000001 2024-09-14T06:30:30.800 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:30.800 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:30.800 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:30:30.800 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655357+0000", 2024-09-14T06:30:30.800 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.8080000000000001e-06 2024-09-14T06:30:30.800 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:30.800 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:30.800 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:30:30.801 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655361+0000", 2024-09-14T06:30:30.801 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.861e-06 2024-09-14T06:30:30.801 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:30.801 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:30.801 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:30:30.801 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655364+0000", 2024-09-14T06:30:30.801 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.1089999999999998e-06 2024-09-14T06:30:30.801 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:30.801 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:30.801 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:30:30.802 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.708065+0000", 2024-09-14T06:30:30.802 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052700595000000003 2024-09-14T06:30:30.802 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:30.802 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:30.802 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:30:30.802 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.708108+0000", 2024-09-14T06:30:30.802 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.2484999999999997e-05 2024-09-14T06:30:30.802 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:30.803 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:30.803 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:30:30.803 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.708161+0000", 2024-09-14T06:30:30.803 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.3482000000000001e-05 2024-09-14T06:30:30.803 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:30.803 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:30:30.803 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:30.803 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:30.803 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:30.803 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:11461 3.1d 3:bcad7303:::rbd_data.112d92027590.0000000000000001:f [read 176128~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:30:30.803 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:25.655370+0000", 2024-09-14T06:30:30.803 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 65.105887318000001, 2024-09-14T06:30:30.803 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052915141999999998, 2024-09-14T06:30:30.803 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:30:30.804 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:30:30.804 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:30:30.804 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:30:30.804 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:30:30.804 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:30.804 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 11461 2024-09-14T06:30:30.804 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:30.804 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:30:30.804 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:30.804 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:30:30.804 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655370+0000", 2024-09-14T06:30:30.804 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:30.804 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:30.804 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:30.805 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:30:30.805 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655370+0000", 2024-09-14T06:30:30.805 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:30.805 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:30.805 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:30.805 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:30:30.805 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655394+0000", 2024-09-14T06:30:30.805 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.4553999999999999e-05 2024-09-14T06:30:30.805 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:30.805 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:30.805 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:30:30.805 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655396+0000", 2024-09-14T06:30:30.805 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.8950000000000001e-06 2024-09-14T06:30:30.805 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:30.805 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:30.806 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:30:30.806 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655401+0000", 2024-09-14T06:30:30.806 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.0089999999999998e-06 2024-09-14T06:30:30.806 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:30.806 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:30.806 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:30:30.806 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655404+0000", 2024-09-14T06:30:30.806 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.3400000000000002e-06 2024-09-14T06:30:30.806 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:30.806 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:30.806 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:30:30.806 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.708186+0000", 2024-09-14T06:30:30.807 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052781726000000001 2024-09-14T06:30:30.807 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:30.807 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:30.807 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:30:30.807 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.708229+0000", 2024-09-14T06:30:30.807 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.2670999999999999e-05 2024-09-14T06:30:30.807 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:30.808 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:30.808 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:30:30.808 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.708285+0000", 2024-09-14T06:30:30.808 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.5946999999999998e-05 2024-09-14T06:30:30.808 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:30.808 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:30:30.808 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:30.808 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:30.808 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:30.808 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:11462 3.1d 3:bcad7303:::rbd_data.112d92027590.0000000000000001:f [read 186368~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:30:30.808 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:25.655410+0000", 2024-09-14T06:30:30.808 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 65.105846839999998, 2024-09-14T06:30:30.808 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052995258000000003, 2024-09-14T06:30:30.808 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:30:30.809 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:30:30.809 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:30:30.809 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:30:30.809 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:30:30.809 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:30.809 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 11462 2024-09-14T06:30:30.809 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:30.809 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:30:30.809 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:30.809 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:30:30.809 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655410+0000", 2024-09-14T06:30:30.810 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:30.810 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:30.810 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:30.810 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:30:30.810 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655410+0000", 2024-09-14T06:30:30.810 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:30.810 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:30.810 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:30.810 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:30:30.810 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655625+0000", 2024-09-14T06:30:30.810 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00021442600000000001 2024-09-14T06:30:30.810 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:30.810 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:30.810 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:30:30.811 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655626+0000", 2024-09-14T06:30:30.811 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.815e-06 2024-09-14T06:30:30.811 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:30.811 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:30.811 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:30:30.811 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655631+0000", 2024-09-14T06:30:30.811 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.651e-06 2024-09-14T06:30:30.811 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:30.811 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:30.811 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:30:30.811 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655634+0000", 2024-09-14T06:30:30.811 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.2449999999999998e-06 2024-09-14T06:30:30.811 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:30.811 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:30.812 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:30:30.812 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.708312+0000", 2024-09-14T06:30:30.812 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052677412 2024-09-14T06:30:30.812 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:30.812 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:30.812 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:30:30.812 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.708352+0000", 2024-09-14T06:30:30.812 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.0729999999999998e-05 2024-09-14T06:30:30.812 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:30.812 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:30.812 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:30:30.812 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.708405+0000", 2024-09-14T06:30:30.812 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.2979000000000002e-05 2024-09-14T06:30:30.812 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:30.813 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:30:30.813 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:30.813 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:30.813 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:30.813 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:11463 3.1d 3:bcad7303:::rbd_data.112d92027590.0000000000000001:f [read 196608~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:30:30.813 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:25.655639+0000", 2024-09-14T06:30:30.813 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 65.105617694000003, 2024-09-14T06:30:30.813 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052888859000000003, 2024-09-14T06:30:30.813 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:30:30.813 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:30:30.814 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:30:30.814 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:30:30.814 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:30:30.814 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:30.814 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 11463 2024-09-14T06:30:30.814 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:30.814 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:30:30.814 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:30.814 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:30:30.814 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655639+0000", 2024-09-14T06:30:30.814 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:30.814 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:30.814 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:30.814 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:30:30.815 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655639+0000", 2024-09-14T06:30:30.815 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:30.815 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:30.815 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:30.815 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:30:30.815 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655640+0000", 2024-09-14T06:30:30.815 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.08e-07 2024-09-14T06:30:30.815 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:30.815 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:30.815 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:30:30.815 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655641+0000", 2024-09-14T06:30:30.815 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.156e-06 2024-09-14T06:30:30.815 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:30.815 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:30.816 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:30:30.816 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655648+0000", 2024-09-14T06:30:30.816 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.8460000000000002e-06 2024-09-14T06:30:30.816 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:30.816 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:30.816 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:30:30.816 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655651+0000", 2024-09-14T06:30:30.816 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.0079999999999998e-06 2024-09-14T06:30:30.816 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:30.816 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:30.816 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:30:30.816 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.708429+0000", 2024-09-14T06:30:30.816 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052778459 2024-09-14T06:30:30.816 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:30.817 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:30.817 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:30:30.817 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.708473+0000", 2024-09-14T06:30:30.817 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.3782999999999997e-05 2024-09-14T06:30:30.817 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:30.817 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:30.817 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:30:30.817 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.708528+0000", 2024-09-14T06:30:30.817 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.5198999999999998e-05 2024-09-14T06:30:30.817 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:30.817 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:30:30.817 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:30.817 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:30.817 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:30.818 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:11464 3.1d 3:bcad7303:::rbd_data.112d92027590.0000000000000001:f [read 206848~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:30:30.818 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:25.655654+0000", 2024-09-14T06:30:30.818 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 65.105602825000005, 2024-09-14T06:30:30.818 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052981845999999999, 2024-09-14T06:30:30.818 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:30:30.818 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:30:30.818 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:30:30.818 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:30:30.818 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:30:30.818 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:30.818 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 11464 2024-09-14T06:30:30.818 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:30.818 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:30:30.818 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:30.818 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:30:30.819 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655654+0000", 2024-09-14T06:30:30.819 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:30.819 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:30.819 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:30.819 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:30:30.819 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655654+0000", 2024-09-14T06:30:30.819 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:30.819 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:30.819 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:30.819 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:30:30.819 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655946+0000", 2024-09-14T06:30:30.819 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00029148200000000003 2024-09-14T06:30:30.819 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:30.819 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:30.819 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:30:30.820 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655948+0000", 2024-09-14T06:30:30.820 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.973e-06 2024-09-14T06:30:30.820 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:30.820 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:30.820 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:30:30.820 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655953+0000", 2024-09-14T06:30:30.820 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.1240000000000004e-06 2024-09-14T06:30:30.820 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:30.820 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:30.820 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:30:30.820 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655956+0000", 2024-09-14T06:30:30.820 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.315e-06 2024-09-14T06:30:30.820 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:30.820 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:30.821 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:30:30.821 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.708543+0000", 2024-09-14T06:30:30.821 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052587416999999997 2024-09-14T06:30:30.821 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:30.821 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:30.821 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:30:30.821 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.708584+0000", 2024-09-14T06:30:30.821 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.1004000000000002e-05 2024-09-14T06:30:30.821 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:30.821 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:30.821 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:30:30.821 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.708636+0000", 2024-09-14T06:30:30.821 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.1530999999999998e-05 2024-09-14T06:30:30.822 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:30.822 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:30:30.822 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:30.822 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:30.822 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:30.822 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:11465 3.1d 3:bcad7303:::rbd_data.112d92027590.0000000000000001:f [read 217088~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:30:30.822 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:25.655961+0000", 2024-09-14T06:30:30.822 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 65.105295673000001, 2024-09-14T06:30:30.822 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052798682999999999, 2024-09-14T06:30:30.822 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:30:30.822 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:30:30.822 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:30:30.822 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:30:30.822 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:30:30.823 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:30.823 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 11465 2024-09-14T06:30:30.823 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:30.823 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:30:30.823 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:30.823 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:30:30.823 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655961+0000", 2024-09-14T06:30:30.823 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:30.823 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:30.823 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:30.823 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:30:30.823 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655961+0000", 2024-09-14T06:30:30.823 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:30.823 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:30.824 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:30.824 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:30:30.824 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.656114+0000", 2024-09-14T06:30:30.824 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.000152745 2024-09-14T06:30:30.824 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:30.824 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:30.824 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:30:30.824 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.656116+0000", 2024-09-14T06:30:30.824 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.804e-06 2024-09-14T06:30:30.824 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:30.824 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:30.824 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:30:30.824 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.656120+0000", 2024-09-14T06:30:30.825 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.4700000000000004e-06 2024-09-14T06:30:30.825 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:30.825 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:30.825 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:30:30.825 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.656123+0000", 2024-09-14T06:30:30.825 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.1269999999999999e-06 2024-09-14T06:30:30.825 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:30.825 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:30.825 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:30:30.825 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.708660+0000", 2024-09-14T06:30:30.826 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052536919000000001 2024-09-14T06:30:30.826 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:30.826 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:30.826 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:30:30.826 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.708703+0000", 2024-09-14T06:30:30.826 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.3077000000000001e-05 2024-09-14T06:30:30.826 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:30.826 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:30.826 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:30:30.826 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.708760+0000", 2024-09-14T06:30:30.826 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.6541e-05 2024-09-14T06:30:30.826 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:30.826 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:30:30.826 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:30.827 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:30.827 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:30.827 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:11466 3.1d 3:bcad7303:::rbd_data.112d92027590.0000000000000001:f [read 227328~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:30:30.827 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:25.656129+0000", 2024-09-14T06:30:30.827 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 65.105128285000006, 2024-09-14T06:30:30.827 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052750987999999999, 2024-09-14T06:30:30.827 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:30:30.827 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:30:30.827 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:30:30.827 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:30:30.827 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:30:30.827 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:30.827 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 11466 2024-09-14T06:30:30.827 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:30.828 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:30:30.828 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:30.828 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:30:30.828 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.656129+0000", 2024-09-14T06:30:30.828 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:30.828 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:30.828 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:30.828 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:30:30.828 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.656129+0000", 2024-09-14T06:30:30.828 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:30.828 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:30.828 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:30.828 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:30:30.828 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.656347+0000", 2024-09-14T06:30:30.829 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.000218054 2024-09-14T06:30:30.829 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:30.829 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:30.829 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:30:30.829 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.656349+0000", 2024-09-14T06:30:30.829 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.9199999999999998e-06 2024-09-14T06:30:30.830 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:30.830 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:30.830 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:30:30.830 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.656353+0000", 2024-09-14T06:30:30.830 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.9210000000000002e-06 2024-09-14T06:30:30.830 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:30.830 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:30.830 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:30:30.830 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.656356+0000", 2024-09-14T06:30:30.830 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.7670000000000001e-06 2024-09-14T06:30:30.830 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:30.830 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:30.830 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:30:30.831 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.708787+0000", 2024-09-14T06:30:30.831 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052430840999999999 2024-09-14T06:30:30.831 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:30.831 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:30.831 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:30:30.831 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.708827+0000", 2024-09-14T06:30:30.831 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.9538e-05 2024-09-14T06:30:30.831 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:30.831 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:30.831 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:30:30.832 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.708880+0000", 2024-09-14T06:30:30.832 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.2947e-05 2024-09-14T06:30:30.832 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:30.832 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:30:30.832 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:30.832 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:30.832 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:30.832 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:11467 3.1d 3:bcad7303:::rbd_data.112d92027590.0000000000000001:f [read 237568~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:30:30.832 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:25.656361+0000", 2024-09-14T06:30:30.832 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 65.104895636999998, 2024-09-14T06:30:30.832 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052643898000000001, 2024-09-14T06:30:30.832 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:30:30.832 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:30:30.833 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:30:30.833 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:30:30.833 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:30:30.833 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:30.833 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 11467 2024-09-14T06:30:30.833 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:30.833 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:30:30.833 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:30.833 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:30:30.833 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.656361+0000", 2024-09-14T06:30:30.833 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:30.833 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:30.833 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:30.833 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:30:30.834 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.656361+0000", 2024-09-14T06:30:30.834 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:30.834 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:30.834 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:30.834 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:30:30.834 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.656855+0000", 2024-09-14T06:30:30.834 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00049372000000000001 2024-09-14T06:30:30.834 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:30.834 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:30.834 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:30:30.834 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.656859+0000", 2024-09-14T06:30:30.834 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.1300000000000003e-06 2024-09-14T06:30:30.834 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:30.834 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:30.834 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:30:30.834 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.656863+0000", 2024-09-14T06:30:30.835 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.2629999999999997e-06 2024-09-14T06:30:30.835 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:30.835 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:30.835 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:30:30.835 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.656867+0000", 2024-09-14T06:30:30.835 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.3220000000000001e-06 2024-09-14T06:30:30.835 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:30.835 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:30.835 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:30:30.835 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.708903+0000", 2024-09-14T06:30:30.835 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052035855999999998 2024-09-14T06:30:30.835 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:30.835 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:30.835 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:30:30.836 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.708949+0000", 2024-09-14T06:30:30.836 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.6276999999999998e-05 2024-09-14T06:30:30.836 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:30.836 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:30.836 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:30:30.836 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.709005+0000", 2024-09-14T06:30:30.836 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.6329999999999998e-05 2024-09-14T06:30:30.836 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:30.836 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:30:30.836 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:30.836 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:30.836 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:30.836 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:11479 3.1d 3:bcad7303:::rbd_data.112d92027590.0000000000000001:f [read 360448~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:30:30.836 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:25.658388+0000", 2024-09-14T06:30:30.836 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 65.102868803999996, 2024-09-14T06:30:30.837 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052408920999999997, 2024-09-14T06:30:30.837 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:30:30.837 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:30:30.837 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:30:30.837 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:30:30.837 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:30:30.837 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:30.837 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 11479 2024-09-14T06:30:30.837 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:30.837 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:30:30.837 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:30.837 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:30:30.837 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.658388+0000", 2024-09-14T06:30:30.837 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:30.837 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:30.838 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:30.838 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:30:30.838 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.658388+0000", 2024-09-14T06:30:30.838 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:30.838 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:30.838 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:30.838 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:30:30.838 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.658534+0000", 2024-09-14T06:30:30.838 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.000145613 2024-09-14T06:30:30.838 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:30.838 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:30.838 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:30:30.838 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.658540+0000", 2024-09-14T06:30:30.839 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.6680000000000004e-06 2024-09-14T06:30:30.839 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:30.839 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:30.839 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:30:30.839 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.658548+0000", 2024-09-14T06:30:30.839 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.7980000000000006e-06 2024-09-14T06:30:30.839 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:30.839 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:30.839 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:30:30.839 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.658554+0000", 2024-09-14T06:30:30.839 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.6869999999999999e-06 2024-09-14T06:30:30.839 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:30.839 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:30.839 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:30:30.840 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.710624+0000", 2024-09-14T06:30:30.840 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052070567999999998 2024-09-14T06:30:30.840 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:30.840 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:30.840 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:30:30.840 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.710666+0000", 2024-09-14T06:30:30.840 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.1467000000000001e-05 2024-09-14T06:30:30.840 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:30.840 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:30.840 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:30:30.840 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.710797+0000", 2024-09-14T06:30:30.840 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00013112 2024-09-14T06:30:30.840 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:30.840 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:30:30.841 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:30.841 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:30.841 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:30.841 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:11480 3.1d 3:bcad7303:::rbd_data.112d92027590.0000000000000001:f [read 370688~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:30:30.841 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:25.658563+0000", 2024-09-14T06:30:30.841 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 65.102693873000007, 2024-09-14T06:30:30.841 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052429164, 2024-09-14T06:30:30.841 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:30:30.841 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:30:30.841 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:30:30.841 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:30:30.841 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:30:30.841 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:30.842 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 11480 2024-09-14T06:30:30.842 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:30.842 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:30:30.842 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:30.842 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:30:30.842 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.658563+0000", 2024-09-14T06:30:30.842 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:30.842 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:30.842 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:30.842 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:30:30.842 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.658563+0000", 2024-09-14T06:30:30.842 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:30.842 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:30.842 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:30.842 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:30:30.843 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.658658+0000", 2024-09-14T06:30:30.843 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.5346999999999994e-05 2024-09-14T06:30:30.843 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:30.843 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:30.843 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:30:30.843 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.658662+0000", 2024-09-14T06:30:30.843 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.14e-06 2024-09-14T06:30:30.843 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:30.843 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:30.843 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:30:30.843 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.658669+0000", 2024-09-14T06:30:30.843 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.7379999999999994e-06 2024-09-14T06:30:30.843 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:30.843 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:30.844 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:30:30.844 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.658675+0000", 2024-09-14T06:30:30.844 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.5430000000000001e-06 2024-09-14T06:30:30.844 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:30.844 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:30.844 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:30:30.844 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.710824+0000", 2024-09-14T06:30:30.844 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052148847999999998 2024-09-14T06:30:30.844 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:30.844 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:30.844 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:30:30.844 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.710864+0000", 2024-09-14T06:30:30.844 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.9925999999999997e-05 2024-09-14T06:30:30.844 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:30.845 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:30.845 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:30:30.848 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.710992+0000", 2024-09-14T06:30:30.848 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.000128622 2024-09-14T06:30:30.848 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:30.849 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:30:30.849 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:30.849 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:30.849 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:30.849 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:11481 3.1d 3:bcad7303:::rbd_data.112d92027590.0000000000000001:f [read 380928~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:30:30.849 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:25.658684+0000", 2024-09-14T06:30:30.849 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 65.102572803000001, 2024-09-14T06:30:30.849 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052508565, 2024-09-14T06:30:30.849 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:30:30.849 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:30:30.849 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:30:30.849 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:30:30.849 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:30:30.849 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:30.850 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 11481 2024-09-14T06:30:30.850 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:30.850 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:30:30.850 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:30.850 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:30:30.850 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.658684+0000", 2024-09-14T06:30:30.850 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:30.850 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:30.850 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:30.850 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:30:30.850 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.658684+0000", 2024-09-14T06:30:30.850 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:30.850 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:30.851 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:30.851 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:30:30.851 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.658715+0000", 2024-09-14T06:30:30.851 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.0703000000000001e-05 2024-09-14T06:30:30.851 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:30.851 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:30.851 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:30:30.851 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.658718+0000", 2024-09-14T06:30:30.851 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.0709999999999999e-06 2024-09-14T06:30:30.851 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:30.851 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:30.851 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:30:30.851 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.658725+0000", 2024-09-14T06:30:30.851 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.2339999999999997e-06 2024-09-14T06:30:30.852 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:30.852 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:30.852 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:30:30.852 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.658731+0000", 2024-09-14T06:30:30.852 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.6450000000000002e-06 2024-09-14T06:30:30.852 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:30.852 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:30.852 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:30:30.852 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.711020+0000", 2024-09-14T06:30:30.852 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052288886 2024-09-14T06:30:30.852 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:30.852 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:30.852 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:30:30.853 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.711061+0000", 2024-09-14T06:30:30.853 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.1084000000000002e-05 2024-09-14T06:30:30.853 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:30.853 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:30.853 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:30:30.853 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.711193+0000", 2024-09-14T06:30:30.853 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00013194199999999999 2024-09-14T06:30:30.853 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:30.853 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:30:30.853 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:30.853 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:30.853 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:30.853 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:11482 3.1d 3:bcad7303:::rbd_data.112d92027590.0000000000000001:f [read 391168~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:30:30.853 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:25.658740+0000", 2024-09-14T06:30:30.853 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 65.102517337999998, 2024-09-14T06:30:30.854 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052637015000000002, 2024-09-14T06:30:30.854 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:30:30.854 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:30:30.854 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:30:30.854 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:30:30.854 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:30:30.854 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:30.854 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 11482 2024-09-14T06:30:30.854 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:30.854 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:30:30.854 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:30.854 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:30:30.854 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.658740+0000", 2024-09-14T06:30:30.854 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:30.855 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:30.855 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:30.855 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:30:30.855 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.658740+0000", 2024-09-14T06:30:30.855 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:30.855 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:30.855 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:30.855 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:30:30.855 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.658880+0000", 2024-09-14T06:30:30.855 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00014036100000000001 2024-09-14T06:30:30.855 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:30.855 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:30.855 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:30:30.855 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.658883+0000", 2024-09-14T06:30:30.855 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.3969999999999998e-06 2024-09-14T06:30:30.855 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:30.856 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:30.856 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:30:30.856 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.658890+0000", 2024-09-14T06:30:30.856 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.1500000000000002e-06 2024-09-14T06:30:30.856 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:30.856 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:30.856 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:30:30.856 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.658897+0000", 2024-09-14T06:30:30.856 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.1260000000000003e-06 2024-09-14T06:30:30.856 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:30.856 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:30.856 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:30:30.856 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.711207+0000", 2024-09-14T06:30:30.856 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052310777000000003 2024-09-14T06:30:30.856 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:30.857 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:30.857 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:30:30.857 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.711249+0000", 2024-09-14T06:30:30.857 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.1786999999999998e-05 2024-09-14T06:30:30.857 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:30.857 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:30.857 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:30:30.857 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.711377+0000", 2024-09-14T06:30:30.857 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.000127417 2024-09-14T06:30:30.857 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:30.857 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:30:30.857 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:30.857 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:30.857 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:30.857 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:11483 3.1d 3:bcad7303:::rbd_data.112d92027590.0000000000000001:f [read 401408~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:30:30.858 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:25.658906+0000", 2024-09-14T06:30:30.858 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 65.102350553999997, 2024-09-14T06:30:30.858 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052595109000000001, 2024-09-14T06:30:30.858 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:30:30.858 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:30:30.858 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:30:30.858 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:30:30.858 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:30:30.858 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:30.858 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 11483 2024-09-14T06:30:30.858 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:30.858 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:30:30.858 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:30.858 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:30:30.859 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.658906+0000", 2024-09-14T06:30:30.859 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:30.859 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:30.859 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:30.859 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:30:30.859 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.658906+0000", 2024-09-14T06:30:30.859 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:30.859 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:30.859 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:30.859 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:30:30.859 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.664985+0000", 2024-09-14T06:30:30.859 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.0060787250000000001 2024-09-14T06:30:30.859 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:30.859 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:30.860 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:30:30.860 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.664991+0000", 2024-09-14T06:30:30.860 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.7030000000000003e-06 2024-09-14T06:30:30.860 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:30.860 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:30.860 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:30:30.860 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.664999+0000", 2024-09-14T06:30:30.860 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.4430000000000003e-06 2024-09-14T06:30:30.860 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:30.861 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:30.861 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:30:30.861 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.665006+0000", 2024-09-14T06:30:30.861 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.0840000000000003e-06 2024-09-14T06:30:30.861 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:30.861 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:30.861 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:30:30.861 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.711402+0000", 2024-09-14T06:30:30.861 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046395714999999997 2024-09-14T06:30:30.861 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:30.861 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:30.861 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:30:30.861 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.711443+0000", 2024-09-14T06:30:30.861 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.0911999999999997e-05 2024-09-14T06:30:30.862 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:30.862 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:30.862 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:30:30.862 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.711501+0000", 2024-09-14T06:30:30.862 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.8526999999999997e-05 2024-09-14T06:30:30.862 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:30.862 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:30:30.862 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:30.862 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:30.862 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:30:30.862 INFO:teuthology.orchestra.run.smithi138.stdout:} 2024-09-14T06:30:30.864 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-14T06:30:30.864 INFO:tasks.ceph.osd.2.smithi094.stderr:2024-09-14T06:30:30.861+0000 7faf3269b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33649) UID: 0 2024-09-14T06:30:30.965 INFO:tasks.ceph.osd.6.smithi138.stderr:2024-09-14T06:30:30.964+0000 7f8cd8849640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33620) UID: 0 2024-09-14T06:30:30.998 INFO:teuthology.orchestra.run.smithi094.stdout:3318 read 0x15e355e thru 0x15e83a7 (0x4e4a bytes) 2024-09-14T06:30:30.999 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-14T06:30:30.999 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-14T06:30:30.999 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-14T06:30:30.999 INFO:teuthology.orchestra.run.smithi094.stdout:3324 writesame 0x36e2615 thru 0x36f11bd data_size 0x2b(0xeba9 bytes) 2024-09-14T06:30:31.004 INFO:teuthology.orchestra.run.smithi094.stdout:3325 write 0xdc53510 thru 0xdc62d08 (0xf7f9 bytes) 2024-09-14T06:30:31.008 INFO:teuthology.orchestra.run.smithi094.stdout:3327 read 0x139ebe3 thru 0x139f9ee (0xe0c bytes) 2024-09-14T06:30:31.009 INFO:teuthology.orchestra.run.smithi138.stdout:{ 2024-09-14T06:30:31.010 INFO:teuthology.orchestra.run.smithi138.stdout: "ops": [ 2024-09-14T06:30:31.010 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:31.010 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:132149 3.10 3.7a888ff0 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e108)", 2024-09-14T06:30:31.010 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:30:31.008803+0000", 2024-09-14T06:30:31.010 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 0.00055190800000000004, 2024-09-14T06:30:31.010 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00056699300000000001, 2024-09-14T06:30:31.010 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:30:31.010 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:30:31.010 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "op must wait for map", 2024-09-14T06:30:31.010 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:30:31.010 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:30:31.010 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:31.010 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 132149 2024-09-14T06:30:31.011 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:31.011 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:30:31.011 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:31.011 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:30:31.011 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:31.008803+0000", 2024-09-14T06:30:31.011 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:31.011 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:31.011 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:31.011 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:30:31.011 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:31.008803+0000", 2024-09-14T06:30:31.011 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:31.011 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:31.011 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:31.012 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:30:31.012 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:31.008804+0000", 2024-09-14T06:30:31.012 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.6270000000000001e-06 2024-09-14T06:30:31.012 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:31.012 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:31.012 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:30:31.012 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:31.008811+0000", 2024-09-14T06:30:31.012 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.4479999999999999e-06 2024-09-14T06:30:31.012 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:31.012 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:31.012 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:30:31.012 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:31.008826+0000", 2024-09-14T06:30:31.012 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.5627000000000001e-05 2024-09-14T06:30:31.012 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:31.013 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:31.013 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:30:31.013 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:31.008834+0000", 2024-09-14T06:30:31.013 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.8630000000000008e-06 2024-09-14T06:30:31.013 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:31.013 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:31.013 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:30:31.013 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:31.008888+0000", 2024-09-14T06:30:31.013 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.3766e-05 2024-09-14T06:30:31.013 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:31.013 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:31.013 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "op must wait for map", 2024-09-14T06:30:31.013 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:31.008892+0000", 2024-09-14T06:30:31.013 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.2420000000000002e-06 2024-09-14T06:30:31.014 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:31.014 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:30:31.014 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:31.014 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:31.014 INFO:teuthology.orchestra.run.smithi138.stdout: ], 2024-09-14T06:30:31.014 INFO:teuthology.orchestra.run.smithi138.stdout: "num_ops": 1 2024-09-14T06:30:31.014 INFO:teuthology.orchestra.run.smithi138.stdout:} 2024-09-14T06:30:31.014 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-14T06:30:31.015 INFO:teuthology.orchestra.run.smithi094.stdout:3329 write 0x7584f1 thru 0x75f2e8 (0x6df8 bytes) 2024-09-14T06:30:31.015 INFO:teuthology.orchestra.run.smithi094.stdout:3330 read 0xac77a40 thru 0xac78336 (0x8f7 bytes) 2024-09-14T06:30:31.020 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-14T06:30:31.022 INFO:teuthology.orchestra.run.smithi094.stdout:3332 trunc from 0xea85cb6 to 0x6985d28 2024-09-14T06:30:31.035 INFO:teuthology.orchestra.run.smithi094.stdout:3333 write 0x3ae159d thru 0x3ae75e3 (0x6047 bytes) 2024-09-14T06:30:31.037 INFO:teuthology.orchestra.run.smithi094.stdout:3334 write 0x4f95d7d thru 0x4f994df (0x3763 bytes) 2024-09-14T06:30:31.039 INFO:teuthology.orchestra.run.smithi094.stdout:3335 punch from 0x212a2ab to 0x213478d, (0xa4e2 bytes) 2024-09-14T06:30:31.039 INFO:teuthology.orchestra.run.smithi094.stdout:3338 write 0x26a87b4 thru 0x26ab759 (0x2fa6 bytes) 2024-09-14T06:30:31.040 INFO:teuthology.orchestra.run.smithi094.stdout:3339 write 0x3f0ea52 thru 0x3f13860 (0x4e0f bytes) 2024-09-14T06:30:31.045 INFO:teuthology.orchestra.run.smithi094.stdout:3340 read 0x48f2bd9 thru 0x4901285 (0xe6ad bytes) 2024-09-14T06:30:31.045 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-14T06:30:31.045 INFO:teuthology.orchestra.run.smithi094.stdout:3342 write 0x18978ae thru 0x18a01f5 (0x8948 bytes) 2024-09-14T06:30:31.058 INFO:teuthology.orchestra.run.smithi094.stdout:3343 writesame 0xdcda591 thru 0xdce17d1 data_size 0x1(0x7241 bytes) 2024-09-14T06:30:31.058 INFO:teuthology.orchestra.run.smithi094.stdout:3344 write 0xcf5525 thru 0xcfd838 (0x8314 bytes) 2024-09-14T06:30:31.060 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-14T06:30:31.060 INFO:teuthology.orchestra.run.smithi094.stdout:3347 write 0x644d2b thru 0x654316 (0xf5ec bytes) 2024-09-14T06:30:31.062 INFO:teuthology.orchestra.run.smithi094.stdout:3348 clone 24 order 20 su 131072 sc 7 2024-09-14T06:30:31.104 INFO:tasks.ceph.osd.5.smithi138.stderr:2024-09-14T06:30:31.103+0000 7fe13746f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33618) UID: 0 2024-09-14T06:30:31.162 INFO:teuthology.orchestra.run.smithi138.stdout:{ 2024-09-14T06:30:31.162 INFO:teuthology.orchestra.run.smithi138.stdout: "ops": [], 2024-09-14T06:30:31.162 INFO:teuthology.orchestra.run.smithi138.stdout: "complaint_time": 30, 2024-09-14T06:30:31.162 INFO:teuthology.orchestra.run.smithi138.stdout: "num_blocked_ops": 0 2024-09-14T06:30:31.162 INFO:teuthology.orchestra.run.smithi138.stdout:} 2024-09-14T06:30:31.167 INFO:tasks.ceph.osd.6.smithi138.stderr:2024-09-14T06:30:31.166+0000 7f8cd8849640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33620) UID: 0 2024-09-14T06:30:31.170 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-14T06:30:31.268 INFO:tasks.ceph.osd.0.smithi094.stderr:2024-09-14T06:30:31.267+0000 7f07c9333640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33636) UID: 0 2024-09-14T06:30:31.316 INFO:teuthology.orchestra.run.smithi138.stdout:{ 2024-09-14T06:30:31.316 INFO:teuthology.orchestra.run.smithi138.stdout: "size": 20, 2024-09-14T06:30:31.316 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 600, 2024-09-14T06:30:31.316 INFO:teuthology.orchestra.run.smithi138.stdout: "ops": [ 2024-09-14T06:30:31.316 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:31.316 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:98505 3.10 3.1bf56f90 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T06:30:31.316 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:30:07.045453+0000", 2024-09-14T06:30:31.317 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 24.269190899000002, 2024-09-14T06:30:31.317 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032144440000000003, 2024-09-14T06:30:31.317 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:30:31.317 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:30:31.317 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "reached pg", 2024-09-14T06:30:31.317 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:30:31.317 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:30:31.317 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:31.317 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 98505 2024-09-14T06:30:31.317 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:31.317 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:30:31.317 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:31.317 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:30:31.317 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045453+0000", 2024-09-14T06:30:31.318 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:31.318 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:31.318 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:31.318 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:30:31.318 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045453+0000", 2024-09-14T06:30:31.318 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:31.318 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:31.318 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:31.318 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:30:31.318 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045453+0000", 2024-09-14T06:30:31.318 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.36e-07 2024-09-14T06:30:31.318 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:31.318 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:31.318 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:30:31.319 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045454+0000", 2024-09-14T06:30:31.319 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.1299999999999998e-07 2024-09-14T06:30:31.319 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:31.319 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:31.319 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:30:31.319 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045457+0000", 2024-09-14T06:30:31.319 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.8559999999999998e-06 2024-09-14T06:30:31.319 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:31.319 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:31.319 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:30:31.319 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045460+0000", 2024-09-14T06:30:31.319 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.8750000000000002e-06 2024-09-14T06:30:31.319 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:31.319 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:31.319 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:30:31.320 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.077592+0000", 2024-09-14T06:30:31.320 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032131727999999998 2024-09-14T06:30:31.320 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:31.320 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:31.320 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:30:31.320 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.077597+0000", 2024-09-14T06:30:31.320 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.8319999999999996e-06 2024-09-14T06:30:31.320 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:31.320 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:30:31.320 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:31.320 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:31.320 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:31.320 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:98506 3.10 3.1bf56f90 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T06:30:31.321 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:30:07.045481+0000", 2024-09-14T06:30:31.321 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 24.269162644000001, 2024-09-14T06:30:31.321 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032142759999999999, 2024-09-14T06:30:31.321 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:30:31.321 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:30:31.321 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "reached pg", 2024-09-14T06:30:31.321 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:30:31.321 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:30:31.321 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:31.321 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 98506 2024-09-14T06:30:31.321 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:31.321 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:30:31.321 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:31.321 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:30:31.322 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045481+0000", 2024-09-14T06:30:31.322 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:31.322 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:31.322 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:31.322 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:30:31.322 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045481+0000", 2024-09-14T06:30:31.322 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:31.322 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:31.322 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:31.322 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:30:31.322 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045481+0000", 2024-09-14T06:30:31.322 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.8500000000000001e-07 2024-09-14T06:30:31.322 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:31.322 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:31.323 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:30:31.323 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045483+0000", 2024-09-14T06:30:31.323 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.8250000000000001e-06 2024-09-14T06:30:31.323 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:31.323 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:31.323 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:30:31.323 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045489+0000", 2024-09-14T06:30:31.323 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.5740000000000003e-06 2024-09-14T06:30:31.323 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:31.323 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:31.323 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:30:31.323 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045495+0000", 2024-09-14T06:30:31.323 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.7690000000000002e-06 2024-09-14T06:30:31.323 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:31.324 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:31.324 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:30:31.324 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.077619+0000", 2024-09-14T06:30:31.324 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032124621999999999 2024-09-14T06:30:31.324 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:31.324 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:31.324 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:30:31.324 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.077624+0000", 2024-09-14T06:30:31.324 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.3850000000000004e-06 2024-09-14T06:30:31.324 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:31.324 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:30:31.324 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:31.324 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:31.324 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:31.325 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:98507 3.10 3.1bf56f90 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T06:30:31.325 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:30:07.045502+0000", 2024-09-14T06:30:31.325 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 24.269141222999998, 2024-09-14T06:30:31.325 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032147219999999997, 2024-09-14T06:30:31.325 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:30:31.325 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:30:31.325 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "reached pg", 2024-09-14T06:30:31.325 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:30:31.325 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:30:31.325 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:31.325 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 98507 2024-09-14T06:30:31.325 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:31.325 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:30:31.325 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:31.326 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:30:31.326 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045502+0000", 2024-09-14T06:30:31.326 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:31.326 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:31.326 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:31.326 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:30:31.326 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045502+0000", 2024-09-14T06:30:31.326 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:31.326 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:31.326 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:31.326 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:30:31.326 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045503+0000", 2024-09-14T06:30:31.326 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.4700000000000002e-07 2024-09-14T06:30:31.327 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:31.327 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:31.327 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:30:31.327 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045509+0000", 2024-09-14T06:30:31.327 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.7329999999999997e-06 2024-09-14T06:30:31.327 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:31.327 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:31.327 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:30:31.327 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045515+0000", 2024-09-14T06:30:31.328 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.8030000000000002e-06 2024-09-14T06:30:31.328 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:31.328 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:31.328 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:30:31.328 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045522+0000", 2024-09-14T06:30:31.328 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.0369999999999997e-06 2024-09-14T06:30:31.328 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:31.328 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:31.328 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:30:31.328 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.077641+0000", 2024-09-14T06:30:31.328 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032119091000000002 2024-09-14T06:30:31.328 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:31.329 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:31.329 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:30:31.329 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.077650+0000", 2024-09-14T06:30:31.329 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.2090000000000005e-06 2024-09-14T06:30:31.329 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:31.329 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:30:31.329 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:31.329 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:31.329 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:31.329 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:98508 3.10 3.1bf56f90 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T06:30:31.329 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:30:07.045530+0000", 2024-09-14T06:30:31.329 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 24.269113606000001, 2024-09-14T06:30:31.329 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032198773999999999, 2024-09-14T06:30:31.329 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:30:31.330 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:30:31.330 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "reached pg", 2024-09-14T06:30:31.330 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:30:31.330 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:30:31.330 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:31.330 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 98508 2024-09-14T06:30:31.330 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:31.330 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:30:31.330 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:31.330 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:30:31.330 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045530+0000", 2024-09-14T06:30:31.330 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:31.330 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:31.330 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:31.330 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:30:31.330 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045530+0000", 2024-09-14T06:30:31.331 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:31.331 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:31.331 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:31.331 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:30:31.331 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045530+0000", 2024-09-14T06:30:31.331 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.3099999999999998e-07 2024-09-14T06:30:31.331 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:31.331 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:31.331 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:30:31.331 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045539+0000", 2024-09-14T06:30:31.331 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.4780000000000008e-06 2024-09-14T06:30:31.331 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:31.331 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:31.331 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:30:31.332 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045546+0000", 2024-09-14T06:30:31.332 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.3259999999999998e-06 2024-09-14T06:30:31.332 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:31.332 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:31.332 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:30:31.332 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045551+0000", 2024-09-14T06:30:31.332 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.9960000000000001e-06 2024-09-14T06:30:31.332 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:31.332 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:31.332 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:30:31.332 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.077716+0000", 2024-09-14T06:30:31.332 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032165081999999998 2024-09-14T06:30:31.332 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:31.332 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:31.333 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:30:31.333 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.077729+0000", 2024-09-14T06:30:31.333 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.2461e-05 2024-09-14T06:30:31.333 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:31.333 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:30:31.333 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:31.333 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:31.333 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:31.333 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:98509 3.10 3.1bf56f90 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T06:30:31.333 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:30:07.045558+0000", 2024-09-14T06:30:31.333 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 24.269085404999998, 2024-09-14T06:30:31.333 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032242198, 2024-09-14T06:30:31.333 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:30:31.333 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:30:31.334 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "reached pg", 2024-09-14T06:30:31.334 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:30:31.334 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:30:31.334 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:31.334 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 98509 2024-09-14T06:30:31.334 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:31.334 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:30:31.334 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:31.334 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:30:31.334 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045558+0000", 2024-09-14T06:30:31.334 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:31.334 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:31.334 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:31.334 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:30:31.334 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045558+0000", 2024-09-14T06:30:31.335 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:31.335 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:31.335 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:31.335 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:30:31.335 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045559+0000", 2024-09-14T06:30:31.335 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.27e-07 2024-09-14T06:30:31.335 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:31.335 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:31.335 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:30:31.335 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045560+0000", 2024-09-14T06:30:31.335 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.6959999999999999e-06 2024-09-14T06:30:31.335 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:31.335 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:31.335 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:30:31.336 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045567+0000", 2024-09-14T06:30:31.336 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.6900000000000003e-06 2024-09-14T06:30:31.336 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:31.336 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:31.336 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:30:31.336 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045571+0000", 2024-09-14T06:30:31.336 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.5750000000000002e-06 2024-09-14T06:30:31.336 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:31.336 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:31.336 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:30:31.336 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.077789+0000", 2024-09-14T06:30:31.336 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032217186000000002 2024-09-14T06:30:31.336 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:31.336 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:31.337 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:30:31.337 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.077800+0000", 2024-09-14T06:30:31.337 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.1623999999999999e-05 2024-09-14T06:30:31.337 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:31.337 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:30:31.337 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:31.337 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:31.337 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:31.337 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:98510 3.10 3.1bf56f90 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T06:30:31.337 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:30:07.045580+0000", 2024-09-14T06:30:31.337 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 24.269063590999998, 2024-09-14T06:30:31.337 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032299845000000001, 2024-09-14T06:30:31.337 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:30:31.337 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:30:31.338 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "reached pg", 2024-09-14T06:30:31.338 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:30:31.338 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:30:31.338 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:31.338 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 98510 2024-09-14T06:30:31.338 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:31.338 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:30:31.338 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:31.338 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:30:31.338 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045580+0000", 2024-09-14T06:30:31.338 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:31.338 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:31.338 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:31.338 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:30:31.339 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045580+0000", 2024-09-14T06:30:31.339 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:31.339 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:31.339 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:31.339 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:30:31.339 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045580+0000", 2024-09-14T06:30:31.339 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.0400000000000002e-07 2024-09-14T06:30:31.339 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:31.339 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:31.339 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:30:31.339 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045582+0000", 2024-09-14T06:30:31.339 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.7600000000000001e-06 2024-09-14T06:30:31.339 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:31.340 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:31.340 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:30:31.340 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045588+0000", 2024-09-14T06:30:31.340 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.5650000000000001e-06 2024-09-14T06:30:31.340 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:31.340 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:31.340 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:30:31.340 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045597+0000", 2024-09-14T06:30:31.340 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.465e-06 2024-09-14T06:30:31.340 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:31.340 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:31.340 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:30:31.340 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.077871+0000", 2024-09-14T06:30:31.340 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032273562999999998 2024-09-14T06:30:31.341 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:31.341 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:31.341 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:30:31.341 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.077880+0000", 2024-09-14T06:30:31.341 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.0880000000000001e-06 2024-09-14T06:30:31.341 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:31.341 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:30:31.341 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:31.341 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:31.341 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:31.341 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:98511 3.10 3.1bf56f90 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T06:30:31.341 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:30:07.045610+0000", 2024-09-14T06:30:31.341 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 24.269033744000001, 2024-09-14T06:30:31.341 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032338894, 2024-09-14T06:30:31.342 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:30:31.342 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:30:31.342 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "reached pg", 2024-09-14T06:30:31.342 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:30:31.342 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:30:31.342 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:31.342 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 98511 2024-09-14T06:30:31.342 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:31.342 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:30:31.342 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:31.342 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:30:31.342 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045610+0000", 2024-09-14T06:30:31.342 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:31.342 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:31.342 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:31.343 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:30:31.343 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045610+0000", 2024-09-14T06:30:31.343 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:31.343 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:31.343 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:31.343 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:30:31.343 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045610+0000", 2024-09-14T06:30:31.343 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.1600000000000002e-07 2024-09-14T06:30:31.343 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:31.343 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:31.343 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:30:31.343 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045612+0000", 2024-09-14T06:30:31.343 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.1150000000000001e-06 2024-09-14T06:30:31.343 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:31.344 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:31.344 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:30:31.344 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045618+0000", 2024-09-14T06:30:31.344 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.0499999999999997e-06 2024-09-14T06:30:31.344 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:31.344 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:31.344 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:30:31.344 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045623+0000", 2024-09-14T06:30:31.344 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.6770000000000003e-06 2024-09-14T06:30:31.344 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:31.344 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:31.344 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:30:31.344 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.077928+0000", 2024-09-14T06:30:31.344 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032305111999999997 2024-09-14T06:30:31.345 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:31.345 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:31.345 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:30:31.345 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.077949+0000", 2024-09-14T06:30:31.345 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.0523999999999999e-05 2024-09-14T06:30:31.345 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:31.345 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:30:31.345 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:31.345 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:31.345 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:31.345 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:98512 3.10 3.1bf56f90 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T06:30:31.345 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:30:07.045630+0000", 2024-09-14T06:30:31.345 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 24.269013738999998, 2024-09-14T06:30:31.346 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032346632, 2024-09-14T06:30:31.346 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:30:31.346 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:30:31.346 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "reached pg", 2024-09-14T06:30:31.346 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:30:31.346 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:30:31.346 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:31.346 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 98512 2024-09-14T06:30:31.346 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:31.346 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:30:31.346 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:31.346 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:30:31.346 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045630+0000", 2024-09-14T06:30:31.346 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:31.346 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:31.347 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:31.347 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:30:31.347 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045630+0000", 2024-09-14T06:30:31.347 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:31.347 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:31.347 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:31.347 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:30:31.347 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045630+0000", 2024-09-14T06:30:31.347 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.8200000000000001e-07 2024-09-14T06:30:31.347 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:31.347 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:31.347 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:30:31.347 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045632+0000", 2024-09-14T06:30:31.347 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.7430000000000001e-06 2024-09-14T06:30:31.348 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:31.348 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:31.348 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:30:31.348 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045637+0000", 2024-09-14T06:30:31.348 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.4550000000000003e-06 2024-09-14T06:30:31.348 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:31.348 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:31.348 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:30:31.348 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045644+0000", 2024-09-14T06:30:31.348 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.1330000000000001e-06 2024-09-14T06:30:31.348 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:31.348 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:31.348 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:30:31.348 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.077968+0000", 2024-09-14T06:30:31.349 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032323401000000002 2024-09-14T06:30:31.349 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:31.349 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:31.349 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:30:31.349 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.077976+0000", 2024-09-14T06:30:31.349 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.5180000000000004e-06 2024-09-14T06:30:31.349 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:31.349 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:30:31.349 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:31.349 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:31.349 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:31.349 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:98513 3.10 3.1bf56f90 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T06:30:31.350 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:30:07.045652+0000", 2024-09-14T06:30:31.350 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 24.268991175, 2024-09-14T06:30:31.350 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032353436999999999, 2024-09-14T06:30:31.350 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:30:31.350 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:30:31.350 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "reached pg", 2024-09-14T06:30:31.350 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:30:31.350 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:30:31.350 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:31.350 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 98513 2024-09-14T06:30:31.350 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:31.350 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:30:31.350 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:31.351 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:30:31.351 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045652+0000", 2024-09-14T06:30:31.351 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:31.351 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:31.351 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:31.351 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:30:31.351 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045652+0000", 2024-09-14T06:30:31.351 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:31.351 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:31.351 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:31.351 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:30:31.351 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045653+0000", 2024-09-14T06:30:31.351 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.3700000000000001e-07 2024-09-14T06:30:31.351 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:31.351 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:31.352 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:30:31.352 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045655+0000", 2024-09-14T06:30:31.352 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.854e-06 2024-09-14T06:30:31.352 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:31.352 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:31.352 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:30:31.352 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045661+0000", 2024-09-14T06:30:31.352 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.5509999999999996e-06 2024-09-14T06:30:31.352 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:31.352 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:31.352 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:30:31.352 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045668+0000", 2024-09-14T06:30:31.352 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.1980000000000004e-06 2024-09-14T06:30:31.352 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:31.353 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:31.353 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:30:31.353 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.077997+0000", 2024-09-14T06:30:31.353 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032328575999999998 2024-09-14T06:30:31.353 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:31.353 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:31.353 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:30:31.353 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.078006+0000", 2024-09-14T06:30:31.353 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.8210000000000008e-06 2024-09-14T06:30:31.353 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:31.353 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:30:31.353 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:31.353 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:31.353 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:31.354 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:98514 3.10 3.1bf56f90 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T06:30:31.354 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:30:07.045699+0000", 2024-09-14T06:30:31.354 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 24.26894484, 2024-09-14T06:30:31.354 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032330223999999998, 2024-09-14T06:30:31.354 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:30:31.354 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:30:31.354 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "reached pg", 2024-09-14T06:30:31.354 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:30:31.354 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:30:31.354 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:31.354 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 98514 2024-09-14T06:30:31.354 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:31.354 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:30:31.354 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:31.354 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:30:31.355 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045699+0000", 2024-09-14T06:30:31.355 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:31.355 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:31.355 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:31.355 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:30:31.355 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045699+0000", 2024-09-14T06:30:31.355 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:31.355 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:31.355 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:31.355 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:30:31.355 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045699+0000", 2024-09-14T06:30:31.355 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.3200000000000001e-07 2024-09-14T06:30:31.355 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:31.355 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:31.356 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:30:31.356 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045701+0000", 2024-09-14T06:30:31.356 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.762e-06 2024-09-14T06:30:31.356 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:31.356 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:31.356 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:30:31.356 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045705+0000", 2024-09-14T06:30:31.356 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.2359999999999998e-06 2024-09-14T06:30:31.356 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:31.356 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:31.356 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:30:31.356 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045708+0000", 2024-09-14T06:30:31.356 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.796e-06 2024-09-14T06:30:31.356 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:31.357 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:31.357 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:30:31.357 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.078023+0000", 2024-09-14T06:30:31.357 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032315213000000002 2024-09-14T06:30:31.357 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:31.357 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:31.357 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:30:31.357 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.078029+0000", 2024-09-14T06:30:31.357 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.8850000000000004e-06 2024-09-14T06:30:31.357 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:31.357 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:30:31.357 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:31.357 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:31.357 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:31.357 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:98515 3.10 3.1bf56f90 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T06:30:31.358 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:30:07.045712+0000", 2024-09-14T06:30:31.358 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 24.268931529, 2024-09-14T06:30:31.358 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032346871999999999, 2024-09-14T06:30:31.358 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:30:31.358 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:30:31.358 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "reached pg", 2024-09-14T06:30:31.358 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:30:31.358 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:30:31.358 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:31.358 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 98515 2024-09-14T06:30:31.358 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:31.358 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:30:31.358 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:31.358 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:30:31.359 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045712+0000", 2024-09-14T06:30:31.359 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:31.359 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:31.359 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:31.359 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:30:31.359 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045712+0000", 2024-09-14T06:30:31.359 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:31.359 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:31.359 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:31.359 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:30:31.359 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045712+0000", 2024-09-14T06:30:31.359 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.2100000000000001e-07 2024-09-14T06:30:31.359 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:31.359 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:31.360 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:30:31.360 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045713+0000", 2024-09-14T06:30:31.360 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.068e-06 2024-09-14T06:30:31.360 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:31.360 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:31.360 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:30:31.360 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045716+0000", 2024-09-14T06:30:31.360 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.971e-06 2024-09-14T06:30:31.360 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:31.360 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:31.360 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:30:31.360 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045719+0000", 2024-09-14T06:30:31.360 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.3879999999999998e-06 2024-09-14T06:30:31.361 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:31.361 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:31.361 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:30:31.361 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.078049+0000", 2024-09-14T06:30:31.361 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032330553999999997 2024-09-14T06:30:31.361 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:31.361 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:31.361 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:30:31.361 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.078059+0000", 2024-09-14T06:30:31.361 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.6700000000000006e-06 2024-09-14T06:30:31.361 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:31.361 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:30:31.361 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:31.361 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:31.362 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:31.362 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:98516 3.10 3.1bf56f90 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T06:30:31.362 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:30:07.045722+0000", 2024-09-14T06:30:31.362 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 24.268921775999999, 2024-09-14T06:30:31.362 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032374334999999997, 2024-09-14T06:30:31.362 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:30:31.362 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:30:31.362 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "reached pg", 2024-09-14T06:30:31.362 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:30:31.362 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:30:31.362 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:31.362 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 98516 2024-09-14T06:30:31.362 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:31.363 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:30:31.363 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:31.363 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:30:31.363 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045722+0000", 2024-09-14T06:30:31.363 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:31.363 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:31.363 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:31.363 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:30:31.363 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045722+0000", 2024-09-14T06:30:31.363 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:31.363 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:31.363 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:31.363 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:30:31.363 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045722+0000", 2024-09-14T06:30:31.364 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.8999999999999998e-07 2024-09-14T06:30:31.364 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:31.364 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:31.364 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:30:31.364 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045723+0000", 2024-09-14T06:30:31.364 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.0049999999999999e-06 2024-09-14T06:30:31.364 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:31.364 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:31.364 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:30:31.364 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045726+0000", 2024-09-14T06:30:31.364 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.9019999999999999e-06 2024-09-14T06:30:31.364 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:31.364 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:31.365 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:30:31.365 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045728+0000", 2024-09-14T06:30:31.365 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.3290000000000001e-06 2024-09-14T06:30:31.365 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:31.365 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:31.365 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:30:31.365 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.078090+0000", 2024-09-14T06:30:31.365 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032362123 2024-09-14T06:30:31.365 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:31.365 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:31.365 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:30:31.365 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.078096+0000", 2024-09-14T06:30:31.365 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.6860000000000003e-06 2024-09-14T06:30:31.365 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:31.366 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:30:31.366 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:31.366 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:31.366 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:31.366 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:98517 3.10 3.1bf56f90 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T06:30:31.366 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:30:07.045731+0000", 2024-09-14T06:30:31.366 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 24.268912045, 2024-09-14T06:30:31.366 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.03241227, 2024-09-14T06:30:31.366 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:30:31.366 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:30:31.366 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "reached pg", 2024-09-14T06:30:31.366 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:30:31.366 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:30:31.367 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:31.367 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 98517 2024-09-14T06:30:31.367 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:31.367 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:30:31.367 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:31.367 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:30:31.367 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045731+0000", 2024-09-14T06:30:31.367 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:31.367 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:31.367 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:31.367 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:30:31.367 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045731+0000", 2024-09-14T06:30:31.367 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:31.367 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:31.367 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:31.368 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:30:31.368 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045732+0000", 2024-09-14T06:30:31.368 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.0699999999999999e-07 2024-09-14T06:30:31.368 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:31.368 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:31.368 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:30:31.368 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045733+0000", 2024-09-14T06:30:31.368 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.9199999999999999e-07 2024-09-14T06:30:31.368 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:31.368 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:31.368 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:30:31.368 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045735+0000", 2024-09-14T06:30:31.368 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.8150000000000002e-06 2024-09-14T06:30:31.369 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:31.369 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:31.369 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:30:31.369 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045738+0000", 2024-09-14T06:30:31.369 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.3640000000000002e-06 2024-09-14T06:30:31.369 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:31.369 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:31.369 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:30:31.369 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.078126+0000", 2024-09-14T06:30:31.369 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032388206000000003 2024-09-14T06:30:31.369 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:31.369 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:31.369 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:30:31.369 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.078144+0000", 2024-09-14T06:30:31.370 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.7686e-05 2024-09-14T06:30:31.370 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:31.370 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:30:31.370 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:31.370 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:31.370 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:31.370 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:98518 3.10 3.1bf56f90 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T06:30:31.370 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:30:07.045741+0000", 2024-09-14T06:30:31.370 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 24.268902650000001, 2024-09-14T06:30:31.370 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032451793999999999, 2024-09-14T06:30:31.370 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:30:31.370 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:30:31.370 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "reached pg", 2024-09-14T06:30:31.370 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:30:31.371 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:30:31.371 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:31.371 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 98518 2024-09-14T06:30:31.371 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:31.371 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:30:31.371 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:31.371 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:30:31.371 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045741+0000", 2024-09-14T06:30:31.371 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:31.371 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:31.371 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:31.371 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:30:31.371 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045741+0000", 2024-09-14T06:30:31.371 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:31.372 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:31.372 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:31.372 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:30:31.372 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045741+0000", 2024-09-14T06:30:31.372 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.05e-07 2024-09-14T06:30:31.372 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:31.372 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:31.372 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:30:31.372 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045746+0000", 2024-09-14T06:30:31.372 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.2259999999999996e-06 2024-09-14T06:30:31.372 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:31.372 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:31.372 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:30:31.372 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045750+0000", 2024-09-14T06:30:31.373 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.2490000000000001e-06 2024-09-14T06:30:31.373 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:31.373 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:31.373 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:30:31.373 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045752+0000", 2024-09-14T06:30:31.373 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.3980000000000002e-06 2024-09-14T06:30:31.373 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:31.373 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:31.373 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:30:31.373 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.078184+0000", 2024-09-14T06:30:31.373 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032431593000000002 2024-09-14T06:30:31.373 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:31.373 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:31.374 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:30:31.374 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.078193+0000", 2024-09-14T06:30:31.374 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.1230000000000006e-06 2024-09-14T06:30:31.374 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:31.374 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:30:31.374 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:31.374 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:31.374 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:31.374 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:98519 3.10 3.1bf56f90 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T06:30:31.374 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:30:07.045755+0000", 2024-09-14T06:30:31.374 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 24.268888225000001, 2024-09-14T06:30:31.374 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032489431999999999, 2024-09-14T06:30:31.374 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:30:31.374 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:30:31.375 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "reached pg", 2024-09-14T06:30:31.375 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:30:31.375 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:30:31.375 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:31.375 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 98519 2024-09-14T06:30:31.375 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:31.375 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:30:31.375 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:31.375 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:30:31.375 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045755+0000", 2024-09-14T06:30:31.375 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:31.375 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:31.375 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:31.375 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:30:31.376 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045755+0000", 2024-09-14T06:30:31.376 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:31.376 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:31.376 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:31.376 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:30:31.376 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045756+0000", 2024-09-14T06:30:31.376 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.23e-07 2024-09-14T06:30:31.376 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:31.376 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:31.376 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:30:31.376 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045757+0000", 2024-09-14T06:30:31.376 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.1060000000000001e-06 2024-09-14T06:30:31.377 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:31.377 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:31.377 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:30:31.377 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045760+0000", 2024-09-14T06:30:31.377 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.988e-06 2024-09-14T06:30:31.377 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:31.377 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:31.377 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:30:31.377 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045765+0000", 2024-09-14T06:30:31.377 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.2689999999999999e-06 2024-09-14T06:30:31.377 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:31.377 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:31.377 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:30:31.377 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.078234+0000", 2024-09-14T06:30:31.377 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032468915000000001 2024-09-14T06:30:31.378 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:31.378 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:31.378 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:30:31.378 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.078245+0000", 2024-09-14T06:30:31.378 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.0930999999999999e-05 2024-09-14T06:30:31.378 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:31.378 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:30:31.378 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:31.378 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:31.378 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:31.378 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:98520 3.10 3.1bf56f90 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T06:30:31.378 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:30:07.045768+0000", 2024-09-14T06:30:31.378 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 24.268875347000002, 2024-09-14T06:30:31.378 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032531379999999999, 2024-09-14T06:30:31.378 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:30:31.379 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:30:31.379 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "reached pg", 2024-09-14T06:30:31.379 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:30:31.379 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:30:31.379 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:31.379 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 98520 2024-09-14T06:30:31.379 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:31.379 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:30:31.379 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:31.379 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:30:31.379 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045768+0000", 2024-09-14T06:30:31.379 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:31.379 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:31.380 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:31.380 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:30:31.380 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045768+0000", 2024-09-14T06:30:31.380 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:31.380 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:31.380 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:31.380 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:30:31.380 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045768+0000", 2024-09-14T06:30:31.380 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.22e-07 2024-09-14T06:30:31.380 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:31.380 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:31.380 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:30:31.380 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045773+0000", 2024-09-14T06:30:31.381 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.3470000000000001e-06 2024-09-14T06:30:31.381 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:31.381 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:31.381 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:30:31.381 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045776+0000", 2024-09-14T06:30:31.381 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.1319999999999998e-06 2024-09-14T06:30:31.381 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:31.381 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:31.381 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:30:31.381 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045779+0000", 2024-09-14T06:30:31.382 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.9749999999999999e-06 2024-09-14T06:30:31.382 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:31.382 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:31.382 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:30:31.382 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.078289+0000", 2024-09-14T06:30:31.382 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032509693999999999 2024-09-14T06:30:31.382 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:31.382 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:31.382 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:30:31.382 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.078300+0000", 2024-09-14T06:30:31.382 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.101e-05 2024-09-14T06:30:31.382 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:31.382 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:30:31.382 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:31.382 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:31.383 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:31.383 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:98521 3.10 3.1bf56f90 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T06:30:31.383 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:30:07.045782+0000", 2024-09-14T06:30:31.383 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 24.268861167000001, 2024-09-14T06:30:31.383 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032566109000000003, 2024-09-14T06:30:31.383 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:30:31.383 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:30:31.383 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "reached pg", 2024-09-14T06:30:31.383 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:30:31.383 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:30:31.383 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:31.383 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 98521 2024-09-14T06:30:31.383 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:31.384 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:30:31.384 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:31.384 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:30:31.384 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045782+0000", 2024-09-14T06:30:31.384 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:31.384 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:31.384 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:31.384 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:30:31.384 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045782+0000", 2024-09-14T06:30:31.384 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:31.384 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:31.384 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:31.384 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:30:31.385 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045783+0000", 2024-09-14T06:30:31.385 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.2499999999999999e-07 2024-09-14T06:30:31.385 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:31.385 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:31.385 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:30:31.385 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045784+0000", 2024-09-14T06:30:31.385 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.0109999999999999e-06 2024-09-14T06:30:31.385 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:31.385 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:31.385 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:30:31.385 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045786+0000", 2024-09-14T06:30:31.385 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.8289999999999999e-06 2024-09-14T06:30:31.385 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:31.385 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:31.386 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:30:31.386 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045793+0000", 2024-09-14T06:30:31.386 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.579e-06 2024-09-14T06:30:31.386 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:31.386 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:31.386 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:30:31.386 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.078338+0000", 2024-09-14T06:30:31.386 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032545389000000001 2024-09-14T06:30:31.386 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:31.386 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:31.386 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:30:31.386 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.078348+0000", 2024-09-14T06:30:31.386 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.0076e-05 2024-09-14T06:30:31.386 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:31.386 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:30:31.387 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:31.387 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:31.387 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:31.387 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:98522 3.10 3.1bf56f90 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T06:30:31.387 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:30:07.045796+0000", 2024-09-14T06:30:31.387 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 24.268847259000001, 2024-09-14T06:30:31.387 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032577846000000001, 2024-09-14T06:30:31.387 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:30:31.387 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:30:31.387 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "reached pg", 2024-09-14T06:30:31.387 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:30:31.387 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:30:31.387 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:31.387 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 98522 2024-09-14T06:30:31.388 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:31.388 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:30:31.388 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:31.388 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:30:31.388 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045796+0000", 2024-09-14T06:30:31.388 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:31.388 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:31.388 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:31.388 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:30:31.388 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045796+0000", 2024-09-14T06:30:31.388 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:31.388 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:31.388 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:31.388 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:30:31.389 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045796+0000", 2024-09-14T06:30:31.389 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.2100000000000001e-07 2024-09-14T06:30:31.389 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:31.389 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:31.389 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:30:31.389 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045797+0000", 2024-09-14T06:30:31.389 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.6899999999999996e-07 2024-09-14T06:30:31.389 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:31.389 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:31.389 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:30:31.389 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045800+0000", 2024-09-14T06:30:31.389 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.9979999999999999e-06 2024-09-14T06:30:31.389 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:31.390 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:31.390 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:30:31.390 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045803+0000", 2024-09-14T06:30:31.390 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.2850000000000001e-06 2024-09-14T06:30:31.390 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:31.390 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:31.390 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:30:31.390 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.078366+0000", 2024-09-14T06:30:31.390 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032562932000000003 2024-09-14T06:30:31.390 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:31.390 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:31.390 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:30:31.390 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.078374+0000", 2024-09-14T06:30:31.391 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.4409999999999993e-06 2024-09-14T06:30:31.391 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:31.391 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:30:31.391 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:31.391 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:31.391 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:31.391 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:98523 3.10 3.1bf56f90 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T06:30:31.391 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:30:07.045806+0000", 2024-09-14T06:30:31.391 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 24.268837716, 2024-09-14T06:30:31.391 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032611167000000003, 2024-09-14T06:30:31.391 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:30:31.392 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:30:31.392 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "reached pg", 2024-09-14T06:30:31.392 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:30:31.392 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:30:31.392 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:31.392 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 98523 2024-09-14T06:30:31.392 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:31.392 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:30:31.392 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:31.392 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:30:31.392 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045806+0000", 2024-09-14T06:30:31.392 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:31.392 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:31.393 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:31.393 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:30:31.393 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045806+0000", 2024-09-14T06:30:31.393 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:31.393 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:31.393 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:31.393 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:30:31.393 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045806+0000", 2024-09-14T06:30:31.393 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.2000000000000001e-07 2024-09-14T06:30:31.393 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:31.393 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:31.393 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:30:31.393 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045807+0000", 2024-09-14T06:30:31.394 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.4e-07 2024-09-14T06:30:31.394 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:31.394 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:31.394 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:30:31.394 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045810+0000", 2024-09-14T06:30:31.394 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.9229999999999998e-06 2024-09-14T06:30:31.394 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:31.394 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:31.394 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:30:31.394 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045812+0000", 2024-09-14T06:30:31.394 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.2220000000000001e-06 2024-09-14T06:30:31.394 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:31.394 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:31.395 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:30:31.395 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.078407+0000", 2024-09-14T06:30:31.395 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032594513999999998 2024-09-14T06:30:31.395 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:31.395 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:31.395 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:30:31.395 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.078417+0000", 2024-09-14T06:30:31.395 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.0348e-05 2024-09-14T06:30:31.395 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:31.395 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:30:31.395 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:31.395 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:31.395 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:31.396 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:98524 3.10 3.1bf56f90 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T06:30:31.396 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:30:07.045815+0000", 2024-09-14T06:30:31.396 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 24.268828348, 2024-09-14T06:30:31.396 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032643208999999999, 2024-09-14T06:30:31.396 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:30:31.396 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:30:31.396 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "reached pg", 2024-09-14T06:30:31.396 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:30:31.396 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:30:31.396 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:31.396 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 98524 2024-09-14T06:30:31.396 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:31.396 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:30:31.396 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:31.397 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:30:31.397 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045815+0000", 2024-09-14T06:30:31.397 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:31.397 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:31.397 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:31.397 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:30:31.397 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045815+0000", 2024-09-14T06:30:31.397 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:31.397 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:31.397 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:31.397 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:30:31.397 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045815+0000", 2024-09-14T06:30:31.397 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.11e-07 2024-09-14T06:30:31.398 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:31.398 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:31.398 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:30:31.398 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045816+0000", 2024-09-14T06:30:31.398 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.9099999999999991e-07 2024-09-14T06:30:31.398 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:31.398 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:31.398 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:30:31.398 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045819+0000", 2024-09-14T06:30:31.398 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.9399999999999998e-06 2024-09-14T06:30:31.398 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:31.398 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:31.398 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:30:31.398 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045822+0000", 2024-09-14T06:30:31.399 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.3470000000000001e-06 2024-09-14T06:30:31.399 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:31.399 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:31.399 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:30:31.399 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.078449+0000", 2024-09-14T06:30:31.399 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032627529000000002 2024-09-14T06:30:31.399 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:31.399 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:31.399 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:30:31.399 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.078458+0000", 2024-09-14T06:30:31.399 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.1910000000000006e-06 2024-09-14T06:30:31.399 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:31.399 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:30:31.400 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:31.400 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:31.400 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:30:31.400 INFO:teuthology.orchestra.run.smithi138.stdout:} 2024-09-14T06:30:31.401 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-14T06:30:31.402 INFO:tasks.ceph.osd.4.smithi138.stderr:2024-09-14T06:30:31.400+0000 7f6c653c8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33615) UID: 0 2024-09-14T06:30:31.501 INFO:tasks.ceph.osd.7.smithi138.stderr:2024-09-14T06:30:31.500+0000 7f3c6a21d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33623) UID: 0 2024-09-14T06:30:31.546 INFO:teuthology.orchestra.run.smithi138.stdout:{ 2024-09-14T06:30:31.546 INFO:teuthology.orchestra.run.smithi138.stdout: "ops": [], 2024-09-14T06:30:31.546 INFO:teuthology.orchestra.run.smithi138.stdout: "num_ops": 0 2024-09-14T06:30:31.546 INFO:teuthology.orchestra.run.smithi138.stdout:} 2024-09-14T06:30:31.554 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-14T06:30:31.638 INFO:tasks.ceph.osd.7.smithi138.stderr:2024-09-14T06:30:31.637+0000 7f3c6a21d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33623) UID: 0 2024-09-14T06:30:31.703 INFO:tasks.ceph.osd.6.smithi138.stderr:2024-09-14T06:30:31.701+0000 7f8cd8849640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33620) UID: 0 2024-09-14T06:30:31.703 INFO:teuthology.orchestra.run.smithi138.stdout:{ 2024-09-14T06:30:31.703 INFO:teuthology.orchestra.run.smithi138.stdout: "ops": [], 2024-09-14T06:30:31.703 INFO:teuthology.orchestra.run.smithi138.stdout: "complaint_time": 30, 2024-09-14T06:30:31.703 INFO:teuthology.orchestra.run.smithi138.stdout: "num_blocked_ops": 0 2024-09-14T06:30:31.703 INFO:teuthology.orchestra.run.smithi138.stdout:} 2024-09-14T06:30:31.712 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-14T06:30:31.803 INFO:tasks.ceph.osd.0.smithi094.stderr:2024-09-14T06:30:31.801+0000 7f07c9333640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33636) UID: 0 2024-09-14T06:30:31.858 INFO:teuthology.orchestra.run.smithi138.stdout:{ 2024-09-14T06:30:31.858 INFO:teuthology.orchestra.run.smithi138.stdout: "size": 20, 2024-09-14T06:30:31.858 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 600, 2024-09-14T06:30:31.858 INFO:teuthology.orchestra.run.smithi138.stdout: "ops": [ 2024-09-14T06:30:31.858 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:31.858 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:53244 3.f 3:f71f51a7:::rbd_data.112dca2f6ae1.0000000000000015:17 [read 1863680~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:30:31.858 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:44.369399+0000", 2024-09-14T06:30:31.858 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 47.486713960000003, 2024-09-14T06:30:31.858 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047291189999999997, 2024-09-14T06:30:31.858 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:30:31.858 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:30:31.858 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:30:31.859 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:30:31.859 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:30:31.859 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:31.859 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 53244 2024-09-14T06:30:31.859 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:31.859 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:30:31.859 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:31.859 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:30:31.859 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369399+0000", 2024-09-14T06:30:31.859 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:31.859 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:31.859 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:31.859 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:30:31.859 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369399+0000", 2024-09-14T06:30:31.859 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:31.860 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:31.860 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:31.860 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:30:31.860 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369400+0000", 2024-09-14T06:30:31.860 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.0699999999999999e-07 2024-09-14T06:30:31.860 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:31.860 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:31.860 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:30:31.860 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369401+0000", 2024-09-14T06:30:31.860 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.4300000000000001e-06 2024-09-14T06:30:31.860 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:31.861 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:31.861 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:30:31.861 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369404+0000", 2024-09-14T06:30:31.861 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.061e-06 2024-09-14T06:30:31.861 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:31.861 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:31.861 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:30:31.861 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369407+0000", 2024-09-14T06:30:31.861 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.6139999999999999e-06 2024-09-14T06:30:31.861 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:31.861 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:31.861 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:30:31.861 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.416517+0000", 2024-09-14T06:30:31.861 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047110338000000002 2024-09-14T06:30:31.861 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:31.862 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:31.862 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:30:31.862 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.416616+0000", 2024-09-14T06:30:31.862 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.8651000000000001e-05 2024-09-14T06:30:31.862 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:31.862 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:31.862 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:30:31.862 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.416691+0000", 2024-09-14T06:30:31.862 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.4888999999999994e-05 2024-09-14T06:30:31.862 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:31.862 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:30:31.862 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:31.862 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:31.862 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:31.863 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:53245 3.f 3:f71f51a7:::rbd_data.112dca2f6ae1.0000000000000015:17 [read 1888256~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:30:31.863 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:44.369410+0000", 2024-09-14T06:30:31.863 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 47.486703671999997, 2024-09-14T06:30:31.863 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047399275999999997, 2024-09-14T06:30:31.863 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:30:31.863 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:30:31.863 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:30:31.863 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:30:31.863 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:30:31.863 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:31.863 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 53245 2024-09-14T06:30:31.863 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:31.863 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:30:31.863 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:31.863 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:30:31.864 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369410+0000", 2024-09-14T06:30:31.864 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:31.864 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:31.864 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:31.864 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:30:31.864 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369410+0000", 2024-09-14T06:30:31.864 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:31.864 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:31.864 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:31.864 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:30:31.864 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369410+0000", 2024-09-14T06:30:31.864 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.1899999999999999e-07 2024-09-14T06:30:31.864 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:31.864 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:31.865 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:30:31.865 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369411+0000", 2024-09-14T06:30:31.865 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.0300000000000001e-06 2024-09-14T06:30:31.865 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:31.865 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:31.865 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:30:31.865 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369414+0000", 2024-09-14T06:30:31.865 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.1099999999999999e-06 2024-09-14T06:30:31.865 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:31.865 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:31.865 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:30:31.865 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369417+0000", 2024-09-14T06:30:31.865 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.6189999999999998e-06 2024-09-14T06:30:31.865 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:31.866 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:31.866 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:30:31.866 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.416718+0000", 2024-09-14T06:30:31.866 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047301322999999999 2024-09-14T06:30:31.866 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:31.866 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:31.866 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:30:31.866 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.416759+0000", 2024-09-14T06:30:31.866 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.1009e-05 2024-09-14T06:30:31.866 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:31.866 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:31.866 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:30:31.866 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.416809+0000", 2024-09-14T06:30:31.866 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.9966000000000002e-05 2024-09-14T06:30:31.866 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:31.867 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:30:31.867 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:31.867 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:31.867 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:31.867 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:53246 3.f 3:f71f51a7:::rbd_data.112dca2f6ae1.0000000000000015:17 [read 1912832~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:30:31.867 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:44.369420+0000", 2024-09-14T06:30:31.867 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 47.486693727999999, 2024-09-14T06:30:31.867 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047529463000000001, 2024-09-14T06:30:31.867 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:30:31.867 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:30:31.867 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:30:31.867 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:30:31.867 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:30:31.867 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:31.868 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 53246 2024-09-14T06:30:31.868 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:31.868 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:30:31.868 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:31.868 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:30:31.868 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369420+0000", 2024-09-14T06:30:31.868 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:31.868 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:31.868 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:31.868 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:30:31.868 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369420+0000", 2024-09-14T06:30:31.868 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:31.868 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:31.868 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:31.868 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:30:31.869 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369420+0000", 2024-09-14T06:30:31.869 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.1299999999999999e-07 2024-09-14T06:30:31.869 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:31.869 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:31.869 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:30:31.869 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369425+0000", 2024-09-14T06:30:31.869 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.0130000000000001e-06 2024-09-14T06:30:31.869 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:31.869 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:31.869 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:30:31.869 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369428+0000", 2024-09-14T06:30:31.869 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.349e-06 2024-09-14T06:30:31.869 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:31.869 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:31.869 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:30:31.870 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369431+0000", 2024-09-14T06:30:31.870 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.7389999999999999e-06 2024-09-14T06:30:31.870 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:31.870 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:31.870 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:30:31.870 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.416834+0000", 2024-09-14T06:30:31.870 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047402669000000001 2024-09-14T06:30:31.870 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:31.870 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:31.870 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:30:31.871 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.416882+0000", 2024-09-14T06:30:31.871 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.8068999999999997e-05 2024-09-14T06:30:31.871 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:31.871 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:31.871 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:30:31.871 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.416949+0000", 2024-09-14T06:30:31.871 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.7410999999999995e-05 2024-09-14T06:30:31.871 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:31.871 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:30:31.871 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:31.871 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:31.871 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:31.871 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:53247 3.f 3:f71f51a7:::rbd_data.112dca2f6ae1.0000000000000015:17 [read 1937408~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:30:31.871 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:44.369434+0000", 2024-09-14T06:30:31.872 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 47.486678963999999, 2024-09-14T06:30:31.872 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047652264999999999, 2024-09-14T06:30:31.872 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:30:31.872 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:30:31.872 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:30:31.872 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:30:31.872 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:30:31.872 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:31.872 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 53247 2024-09-14T06:30:31.872 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:31.872 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:30:31.872 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:31.872 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:30:31.872 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369434+0000", 2024-09-14T06:30:31.873 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:31.873 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:31.873 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:31.873 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:30:31.873 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369434+0000", 2024-09-14T06:30:31.873 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:31.873 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:31.873 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:31.873 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:30:31.873 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369435+0000", 2024-09-14T06:30:31.873 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.2100000000000001e-07 2024-09-14T06:30:31.874 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:31.874 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:31.874 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:30:31.874 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369436+0000", 2024-09-14T06:30:31.874 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.122e-06 2024-09-14T06:30:31.874 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:31.874 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:31.874 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:30:31.874 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369439+0000", 2024-09-14T06:30:31.874 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.0400000000000001e-06 2024-09-14T06:30:31.874 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:31.874 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:31.874 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:30:31.874 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369441+0000", 2024-09-14T06:30:31.875 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.728e-06 2024-09-14T06:30:31.875 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:31.875 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:31.875 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:30:31.875 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.416977+0000", 2024-09-14T06:30:31.875 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047535344 2024-09-14T06:30:31.875 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:31.875 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:31.875 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:30:31.875 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.417029+0000", 2024-09-14T06:30:31.875 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.2037999999999999e-05 2024-09-14T06:30:31.875 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:31.875 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:31.875 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:30:31.875 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.417087+0000", 2024-09-14T06:30:31.876 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.7772000000000002e-05 2024-09-14T06:30:31.876 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:31.876 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:30:31.876 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:31.876 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:31.876 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:31.876 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:53248 3.f 3:f71f51a7:::rbd_data.112dca2f6ae1.0000000000000015:17 [read 1961984~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:30:31.876 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:44.369445+0000", 2024-09-14T06:30:31.876 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 47.486668412, 2024-09-14T06:30:31.876 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047798489, 2024-09-14T06:30:31.876 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:30:31.876 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:30:31.876 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:30:31.876 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:30:31.877 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:30:31.877 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:31.877 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 53248 2024-09-14T06:30:31.877 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:31.877 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:30:31.877 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:31.877 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:30:31.877 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369445+0000", 2024-09-14T06:30:31.877 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:31.877 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:31.877 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:31.877 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:30:31.877 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369445+0000", 2024-09-14T06:30:31.877 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:31.878 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:31.878 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:31.878 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:30:31.878 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369445+0000", 2024-09-14T06:30:31.878 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.0699999999999999e-07 2024-09-14T06:30:31.878 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:31.878 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:31.878 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:30:31.878 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369446+0000", 2024-09-14T06:30:31.878 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.1000000000000001e-06 2024-09-14T06:30:31.878 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:31.878 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:31.878 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:30:31.878 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369449+0000", 2024-09-14T06:30:31.878 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.1039999999999998e-06 2024-09-14T06:30:31.879 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:31.879 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:31.879 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:30:31.879 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369452+0000", 2024-09-14T06:30:31.879 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.6199999999999999e-06 2024-09-14T06:30:31.879 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:31.879 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:31.879 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:30:31.879 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.417114+0000", 2024-09-14T06:30:31.879 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047661901999999999 2024-09-14T06:30:31.879 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:31.880 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:31.880 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:30:31.880 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.417181+0000", 2024-09-14T06:30:31.880 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.7385999999999995e-05 2024-09-14T06:30:31.880 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:31.880 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:31.880 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:30:31.880 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.417243+0000", 2024-09-14T06:30:31.880 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.2169999999999996e-05 2024-09-14T06:30:31.880 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:31.880 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:30:31.880 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:31.881 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:31.881 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:31.881 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:53249 3.f 3:f71f51a7:::rbd_data.112dca2f6ae1.0000000000000015:17 [read 1986560~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:30:31.881 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:44.369455+0000", 2024-09-14T06:30:31.881 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 47.486658101000003, 2024-09-14T06:30:31.881 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047890493999999999, 2024-09-14T06:30:31.881 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:30:31.881 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:30:31.881 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:30:31.881 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:30:31.881 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:30:31.881 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:31.881 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 53249 2024-09-14T06:30:31.881 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:31.881 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:30:31.882 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:31.882 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:30:31.882 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369455+0000", 2024-09-14T06:30:31.882 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:31.882 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:31.882 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:31.882 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:30:31.882 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369455+0000", 2024-09-14T06:30:31.882 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:31.882 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:31.882 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:31.882 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:30:31.882 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369455+0000", 2024-09-14T06:30:31.882 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.0800000000000001e-07 2024-09-14T06:30:31.883 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:31.883 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:31.883 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:30:31.883 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369456+0000", 2024-09-14T06:30:31.883 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.0729999999999999e-06 2024-09-14T06:30:31.883 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:31.883 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:31.883 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:30:31.883 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369460+0000", 2024-09-14T06:30:31.883 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.027e-06 2024-09-14T06:30:31.883 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:31.883 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:31.883 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:30:31.883 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369462+0000", 2024-09-14T06:30:31.883 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.5320000000000001e-06 2024-09-14T06:30:31.884 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:31.884 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:31.884 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:30:31.884 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.417257+0000", 2024-09-14T06:30:31.884 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.04779489 2024-09-14T06:30:31.884 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:31.884 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:31.884 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:30:31.884 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.417296+0000", 2024-09-14T06:30:31.884 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.8658000000000002e-05 2024-09-14T06:30:31.884 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:31.884 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:31.884 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:30:31.884 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.417346+0000", 2024-09-14T06:30:31.884 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.0105999999999998e-05 2024-09-14T06:30:31.885 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:31.885 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:30:31.885 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:31.885 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:31.885 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:31.885 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:53250 3.f 3:f71f51a7:::rbd_data.112dca2f6ae1.0000000000000015:17 [read 2011136~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:30:31.885 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:44.369465+0000", 2024-09-14T06:30:31.885 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 47.486648131000003, 2024-09-14T06:30:31.885 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.048032230000000002, 2024-09-14T06:30:31.885 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:30:31.885 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:30:31.885 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:30:31.885 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:30:31.885 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:30:31.886 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:31.886 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 53250 2024-09-14T06:30:31.886 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:31.886 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:30:31.886 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:31.886 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:30:31.886 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369465+0000", 2024-09-14T06:30:31.886 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:31.886 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:31.886 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:31.886 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:30:31.886 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369465+0000", 2024-09-14T06:30:31.886 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:31.886 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:31.887 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:31.887 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:30:31.887 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369465+0000", 2024-09-14T06:30:31.887 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.2100000000000001e-07 2024-09-14T06:30:31.887 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:31.887 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:31.887 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:30:31.887 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369466+0000", 2024-09-14T06:30:31.887 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.0640000000000001e-06 2024-09-14T06:30:31.887 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:31.887 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:31.887 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:30:31.887 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369469+0000", 2024-09-14T06:30:31.887 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.0170000000000001e-06 2024-09-14T06:30:31.887 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:31.888 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:31.888 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:30:31.888 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369472+0000", 2024-09-14T06:30:31.888 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.6189999999999998e-06 2024-09-14T06:30:31.888 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:31.888 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:31.888 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:30:31.888 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.417383+0000", 2024-09-14T06:30:31.888 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047911092000000002 2024-09-14T06:30:31.888 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:31.888 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:31.888 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:30:31.888 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.417430+0000", 2024-09-14T06:30:31.888 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.6898999999999999e-05 2024-09-14T06:30:31.889 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:31.889 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:31.889 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:30:31.889 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.417497+0000", 2024-09-14T06:30:31.889 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.7317999999999995e-05 2024-09-14T06:30:31.889 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:31.889 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:30:31.889 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:31.889 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:31.889 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:31.889 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:53251 3.f 3:f71f51a7:::rbd_data.112dca2f6ae1.0000000000000015:17 [read 2035712~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:30:31.889 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:44.369475+0000", 2024-09-14T06:30:31.889 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 47.486638135, 2024-09-14T06:30:31.889 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.048154542000000002, 2024-09-14T06:30:31.890 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:30:31.890 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:30:31.890 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:30:31.890 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:30:31.890 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:30:31.890 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:31.890 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 53251 2024-09-14T06:30:31.890 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:31.890 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:30:31.890 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:31.890 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:30:31.890 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369475+0000", 2024-09-14T06:30:31.891 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:31.891 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:31.891 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:31.891 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:30:31.891 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369475+0000", 2024-09-14T06:30:31.891 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:31.892 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:31.892 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:31.892 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:30:31.892 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369475+0000", 2024-09-14T06:30:31.892 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.0100000000000001e-07 2024-09-14T06:30:31.892 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:31.892 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:31.892 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:30:31.892 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369476+0000", 2024-09-14T06:30:31.892 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.0580000000000001e-06 2024-09-14T06:30:31.892 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:31.892 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:31.892 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:30:31.893 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369479+0000", 2024-09-14T06:30:31.893 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.9950000000000001e-06 2024-09-14T06:30:31.893 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:31.893 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:31.893 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:30:31.893 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369482+0000", 2024-09-14T06:30:31.893 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.6340000000000001e-06 2024-09-14T06:30:31.893 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:31.893 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:31.893 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:30:31.893 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.417530+0000", 2024-09-14T06:30:31.893 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.048047535000000002 2024-09-14T06:30:31.893 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:31.894 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:31.894 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:30:31.894 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.417574+0000", 2024-09-14T06:30:31.894 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.4255999999999998e-05 2024-09-14T06:30:31.894 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:31.894 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:31.894 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:30:31.894 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.417630+0000", 2024-09-14T06:30:31.894 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.5862999999999997e-05 2024-09-14T06:30:31.894 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:31.894 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:30:31.894 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:31.894 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:31.894 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:31.895 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:53252 3.f 3:f71f51a7:::rbd_data.112dca2f6ae1.0000000000000015:17 [read 2060288~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:30:31.895 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:44.369485+0000", 2024-09-14T06:30:31.895 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 47.486628287000002, 2024-09-14T06:30:31.895 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.048254116, 2024-09-14T06:30:31.895 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:30:31.895 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:30:31.895 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:30:31.895 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:30:31.895 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:30:31.895 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:31.895 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 53252 2024-09-14T06:30:31.895 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:31.895 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:30:31.895 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:31.896 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:30:31.896 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369485+0000", 2024-09-14T06:30:31.896 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:31.896 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:31.896 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:31.896 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:30:31.896 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369485+0000", 2024-09-14T06:30:31.896 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:31.896 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:31.896 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:31.896 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:30:31.896 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369485+0000", 2024-09-14T06:30:31.896 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.1e-07 2024-09-14T06:30:31.896 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:31.897 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:31.897 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:30:31.897 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369487+0000", 2024-09-14T06:30:31.897 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.42e-06 2024-09-14T06:30:31.897 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:31.897 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:31.897 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:30:31.897 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369490+0000", 2024-09-14T06:30:31.897 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.9639999999999999e-06 2024-09-14T06:30:31.897 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:31.897 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:31.897 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:30:31.897 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369492+0000", 2024-09-14T06:30:31.897 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.5679999999999998e-06 2024-09-14T06:30:31.898 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:31.898 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:31.898 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:30:31.898 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.417645+0000", 2024-09-14T06:30:31.898 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.048152753 2024-09-14T06:30:31.898 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:31.898 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:31.898 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:30:31.898 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.417687+0000", 2024-09-14T06:30:31.898 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.1900999999999997e-05 2024-09-14T06:30:31.898 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:31.898 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:31.898 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:30:31.898 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.417739+0000", 2024-09-14T06:30:31.899 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.2299999999999997e-05 2024-09-14T06:30:31.899 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:31.899 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:30:31.899 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:31.899 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:31.899 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:31.899 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:53253 3.f 3:f71f51a7:::rbd_data.112dca2f6ae1.0000000000000015:17 [read 2084864~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:30:31.899 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:44.369495+0000", 2024-09-14T06:30:31.899 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 47.486618163000003, 2024-09-14T06:30:31.899 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.048366785000000002, 2024-09-14T06:30:31.899 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:30:31.899 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:30:31.899 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:30:31.899 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:30:31.900 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:30:31.900 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:31.900 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 53253 2024-09-14T06:30:31.900 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:31.900 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:30:31.900 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:31.900 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:30:31.900 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369495+0000", 2024-09-14T06:30:31.900 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:31.900 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:31.900 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:31.900 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:30:31.900 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369495+0000", 2024-09-14T06:30:31.901 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:31.901 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:31.901 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:31.901 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:30:31.901 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370388+0000", 2024-09-14T06:30:31.901 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00089287999999999998 2024-09-14T06:30:31.901 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:31.901 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:31.901 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:30:31.901 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370392+0000", 2024-09-14T06:30:31.901 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.2130000000000002e-06 2024-09-14T06:30:31.901 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:31.901 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:31.901 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:30:31.902 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370401+0000", 2024-09-14T06:30:31.902 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.7649999999999999e-06 2024-09-14T06:30:31.902 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:31.902 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:31.902 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:30:31.902 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370406+0000", 2024-09-14T06:30:31.902 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.0159999999999999e-06 2024-09-14T06:30:31.902 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:31.902 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:31.902 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:30:31.902 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.417764+0000", 2024-09-14T06:30:31.902 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047358348000000001 2024-09-14T06:30:31.902 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:31.902 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:31.903 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:30:31.903 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.417806+0000", 2024-09-14T06:30:31.903 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.1149999999999997e-05 2024-09-14T06:30:31.903 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:31.903 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:31.903 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:30:31.903 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.417862+0000", 2024-09-14T06:30:31.903 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.6412999999999997e-05 2024-09-14T06:30:31.903 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:31.903 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:30:31.903 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:31.903 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:31.903 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:31.903 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:53340 3.f 3:f71f51a7:::rbd_data.112dca2f6ae1.0000000000000015:17 [read 1056768~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:30:31.903 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:44.370421+0000", 2024-09-14T06:30:31.904 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 47.485692282999999, 2024-09-14T06:30:31.904 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047566313999999998, 2024-09-14T06:30:31.904 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:30:31.904 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:30:31.904 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:30:31.904 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:30:31.904 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:30:31.904 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:31.904 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 53340 2024-09-14T06:30:31.904 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:31.904 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:30:31.904 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:31.904 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:30:31.905 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370421+0000", 2024-09-14T06:30:31.905 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:31.905 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:31.905 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:31.905 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:30:31.905 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370421+0000", 2024-09-14T06:30:31.905 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:31.905 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:31.905 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:31.905 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:30:31.905 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370422+0000", 2024-09-14T06:30:31.905 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.3799999999999996e-07 2024-09-14T06:30:31.905 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:31.905 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:31.906 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:30:31.906 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370424+0000", 2024-09-14T06:30:31.906 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.6919999999999998e-06 2024-09-14T06:30:31.906 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:31.906 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:31.906 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:30:31.906 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370432+0000", 2024-09-14T06:30:31.906 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.2439999999999996e-06 2024-09-14T06:30:31.906 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:31.906 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:31.906 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:30:31.906 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370437+0000", 2024-09-14T06:30:31.906 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.5389999999999998e-06 2024-09-14T06:30:31.906 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:31.907 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:31.907 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:30:31.907 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.417888+0000", 2024-09-14T06:30:31.907 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047450432000000001 2024-09-14T06:30:31.907 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:31.907 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:31.907 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:30:31.907 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.417935+0000", 2024-09-14T06:30:31.907 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.7454999999999999e-05 2024-09-14T06:30:31.907 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:31.907 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:31.907 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:30:31.907 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.417987+0000", 2024-09-14T06:30:31.907 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.2213999999999998e-05 2024-09-14T06:30:31.908 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:31.908 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:30:31.908 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:31.908 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:31.908 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:31.908 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:53341 3.f 3:f71f51a7:::rbd_data.112dca2f6ae1.0000000000000015:17 [read 1081344~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:30:31.908 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:44.370446+0000", 2024-09-14T06:30:31.908 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 47.48566701, 2024-09-14T06:30:31.908 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047665723, 2024-09-14T06:30:31.908 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:30:31.908 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:30:31.908 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:30:31.908 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:30:31.909 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:30:31.909 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:31.909 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 53341 2024-09-14T06:30:31.909 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:31.909 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:30:31.909 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:31.909 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:30:31.909 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370446+0000", 2024-09-14T06:30:31.909 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:31.909 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:31.909 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:31.909 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:30:31.909 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370446+0000", 2024-09-14T06:30:31.909 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:31.910 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:31.910 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:31.910 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:30:31.910 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370447+0000", 2024-09-14T06:30:31.910 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.6100000000000002e-07 2024-09-14T06:30:31.910 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:31.910 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:31.910 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:30:31.910 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370448+0000", 2024-09-14T06:30:31.910 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.7370000000000001e-06 2024-09-14T06:30:31.910 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:31.910 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:31.910 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:30:31.910 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370462+0000", 2024-09-14T06:30:31.911 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.3227000000000001e-05 2024-09-14T06:30:31.911 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:31.911 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:31.911 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:30:31.911 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370467+0000", 2024-09-14T06:30:31.911 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.6540000000000004e-06 2024-09-14T06:30:31.911 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:31.911 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:31.911 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:30:31.911 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.418013+0000", 2024-09-14T06:30:31.911 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047545325999999999 2024-09-14T06:30:31.911 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:31.911 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:31.911 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:30:31.911 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.418054+0000", 2024-09-14T06:30:31.912 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.1696999999999997e-05 2024-09-14T06:30:31.912 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:31.912 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:31.912 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:30:31.912 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.418112+0000", 2024-09-14T06:30:31.912 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.7720999999999998e-05 2024-09-14T06:30:31.912 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:31.912 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:30:31.912 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:31.912 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:31.912 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:31.912 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:53342 3.f 3:f71f51a7:::rbd_data.112dca2f6ae1.0000000000000015:17 [read 1105920~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:30:31.912 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:44.370475+0000", 2024-09-14T06:30:31.912 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 47.485637949000001, 2024-09-14T06:30:31.913 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047748848000000003, 2024-09-14T06:30:31.913 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:30:31.913 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:30:31.913 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:30:31.913 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:30:31.913 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:30:31.913 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:31.913 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 53342 2024-09-14T06:30:31.913 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:31.913 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:30:31.913 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:31.913 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:30:31.913 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370475+0000", 2024-09-14T06:30:31.914 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:31.914 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:31.914 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:31.914 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:30:31.914 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370475+0000", 2024-09-14T06:30:31.914 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:31.914 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:31.914 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:31.914 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:30:31.914 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370476+0000", 2024-09-14T06:30:31.914 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.4700000000000002e-07 2024-09-14T06:30:31.914 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:31.914 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:31.914 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:30:31.915 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370478+0000", 2024-09-14T06:30:31.915 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.5890000000000001e-06 2024-09-14T06:30:31.915 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:31.915 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:31.915 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:30:31.915 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370486+0000", 2024-09-14T06:30:31.915 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.3470000000000001e-06 2024-09-14T06:30:31.915 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:31.915 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:31.915 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:30:31.915 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370491+0000", 2024-09-14T06:30:31.915 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.5720000000000002e-06 2024-09-14T06:30:31.915 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:31.915 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:31.916 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:30:31.916 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.418135+0000", 2024-09-14T06:30:31.916 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047644094999999997 2024-09-14T06:30:31.916 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:31.916 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:31.916 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:30:31.916 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.418174+0000", 2024-09-14T06:30:31.916 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.8204000000000002e-05 2024-09-14T06:30:31.916 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:31.916 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:31.916 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:30:31.916 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.418224+0000", 2024-09-14T06:30:31.916 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.0593999999999997e-05 2024-09-14T06:30:31.916 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:31.917 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:30:31.917 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:31.917 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:31.917 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:31.917 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:53343 3.f 3:f71f51a7:::rbd_data.112dca2f6ae1.0000000000000015:17 [read 1130496~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:30:31.917 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:44.370498+0000", 2024-09-14T06:30:31.917 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 47.485615072999998, 2024-09-14T06:30:31.917 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047841574999999997, 2024-09-14T06:30:31.917 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:30:31.917 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:30:31.917 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:30:31.917 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:30:31.917 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:30:31.918 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:31.918 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 53343 2024-09-14T06:30:31.918 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:31.918 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:30:31.918 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:31.918 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:30:31.918 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370498+0000", 2024-09-14T06:30:31.918 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:31.918 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:31.918 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:31.918 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:30:31.918 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370498+0000", 2024-09-14T06:30:31.918 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:31.918 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:31.919 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:31.919 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:30:31.919 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370499+0000", 2024-09-14T06:30:31.919 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.9999999999999998e-07 2024-09-14T06:30:31.919 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:31.919 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:31.919 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:30:31.919 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370501+0000", 2024-09-14T06:30:31.919 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.0310000000000001e-06 2024-09-14T06:30:31.919 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:31.919 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:31.919 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:30:31.919 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370507+0000", 2024-09-14T06:30:31.919 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.4169999999999997e-06 2024-09-14T06:30:31.920 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:31.920 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:31.920 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:30:31.920 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370516+0000", 2024-09-14T06:30:31.920 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.7190000000000007e-06 2024-09-14T06:30:31.920 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:31.920 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:31.920 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:30:31.920 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.418247+0000", 2024-09-14T06:30:31.920 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047731190999999999 2024-09-14T06:30:31.920 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:31.920 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:31.920 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:30:31.921 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.418286+0000", 2024-09-14T06:30:31.921 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.8988e-05 2024-09-14T06:30:31.921 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:31.921 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:31.921 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:30:31.921 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.418340+0000", 2024-09-14T06:30:31.921 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.3829000000000002e-05 2024-09-14T06:30:31.921 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:31.921 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:30:31.921 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:31.921 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:31.921 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:31.921 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:53344 3.f 3:f71f51a7:::rbd_data.112dca2f6ae1.0000000000000015:17 [read 1155072~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:30:31.921 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:44.370524+0000", 2024-09-14T06:30:31.922 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 47.485589302999998, 2024-09-14T06:30:31.922 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047972262000000002, 2024-09-14T06:30:31.922 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:30:31.922 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:30:31.922 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:30:31.922 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:30:31.922 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:30:31.922 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:31.922 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 53344 2024-09-14T06:30:31.922 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:31.922 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:30:31.922 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:31.922 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:30:31.922 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370524+0000", 2024-09-14T06:30:31.922 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:31.923 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:31.923 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:31.923 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:30:31.923 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370524+0000", 2024-09-14T06:30:31.923 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:31.923 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:31.923 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:31.923 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:30:31.923 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370618+0000", 2024-09-14T06:30:31.923 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.4425999999999993e-05 2024-09-14T06:30:31.923 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:31.923 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:31.923 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:30:31.923 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370622+0000", 2024-09-14T06:30:31.924 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.3519999999999998e-06 2024-09-14T06:30:31.924 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:31.924 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:31.924 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:30:31.924 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370631+0000", 2024-09-14T06:30:31.924 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.9299999999999992e-06 2024-09-14T06:30:31.924 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:31.924 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:31.924 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:30:31.924 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370637+0000", 2024-09-14T06:30:31.924 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.3269999999999997e-06 2024-09-14T06:30:31.924 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:31.924 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:31.924 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:30:31.925 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.418372+0000", 2024-09-14T06:30:31.925 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047734764999999998 2024-09-14T06:30:31.925 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:31.925 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:31.925 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:30:31.925 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.418431+0000", 2024-09-14T06:30:31.925 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.8696999999999997e-05 2024-09-14T06:30:31.925 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:31.925 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:31.925 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:30:31.925 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.418496+0000", 2024-09-14T06:30:31.925 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.5765000000000005e-05 2024-09-14T06:30:31.925 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:31.925 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:30:31.926 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:31.926 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:31.926 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:31.926 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:53345 3.f 3:f71f51a7:::rbd_data.112dca2f6ae1.0000000000000015:17 [read 1179648~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:30:31.926 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:44.370647+0000", 2024-09-14T06:30:31.926 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 47.485465836000003, 2024-09-14T06:30:31.926 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047953775999999997, 2024-09-14T06:30:31.926 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:30:31.926 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:30:31.926 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:30:31.926 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:30:31.926 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:30:31.926 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:31.926 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 53345 2024-09-14T06:30:31.926 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:31.927 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:30:31.927 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:31.927 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:30:31.927 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370647+0000", 2024-09-14T06:30:31.927 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:31.927 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:31.927 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:31.927 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:30:31.927 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370647+0000", 2024-09-14T06:30:31.927 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:31.927 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:31.927 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:31.927 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:30:31.927 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.372346+0000", 2024-09-14T06:30:31.928 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.001698284 2024-09-14T06:30:31.928 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:31.928 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:31.928 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:30:31.928 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.372351+0000", 2024-09-14T06:30:31.928 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.8890000000000001e-06 2024-09-14T06:30:31.928 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:31.928 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:31.928 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:30:31.928 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.372362+0000", 2024-09-14T06:30:31.928 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.0941000000000001e-05 2024-09-14T06:30:31.928 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:31.928 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:31.928 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:30:31.929 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.372369+0000", 2024-09-14T06:30:31.929 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.362e-06 2024-09-14T06:30:31.929 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:31.929 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:31.929 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:30:31.929 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.418510+0000", 2024-09-14T06:30:31.929 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046140914999999998 2024-09-14T06:30:31.929 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:31.929 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:31.929 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:30:31.929 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.418549+0000", 2024-09-14T06:30:31.929 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.8717999999999998e-05 2024-09-14T06:30:31.929 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:31.929 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:31.930 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:30:31.930 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.418601+0000", 2024-09-14T06:30:31.930 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.2667000000000002e-05 2024-09-14T06:30:31.930 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:31.930 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:30:31.930 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:31.930 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:31.930 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:31.930 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:53351 3.f 3:f71f51a7:::rbd_data.112dca2f6ae1.0000000000000015:17 [read 1327104~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:30:31.930 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:44.372499+0000", 2024-09-14T06:30:31.930 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 47.483614715000002, 2024-09-14T06:30:31.930 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047140596, 2024-09-14T06:30:31.930 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:30:31.931 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:30:31.931 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:30:31.931 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:30:31.931 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:30:31.931 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:31.931 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 53351 2024-09-14T06:30:31.931 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:31.931 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:30:31.931 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:31.931 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:30:31.931 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.372499+0000", 2024-09-14T06:30:31.931 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:31.931 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:31.931 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:31.932 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:30:31.932 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.372499+0000", 2024-09-14T06:30:31.932 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:31.932 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:31.932 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:31.932 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:30:31.932 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.372499+0000", 2024-09-14T06:30:31.932 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.3000000000000001e-07 2024-09-14T06:30:31.932 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:31.932 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:31.932 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:30:31.932 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.372501+0000", 2024-09-14T06:30:31.932 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.0820000000000001e-06 2024-09-14T06:30:31.932 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:31.932 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:31.933 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:30:31.933 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.372513+0000", 2024-09-14T06:30:31.933 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.1610999999999999e-05 2024-09-14T06:30:31.933 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:31.933 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:31.933 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:30:31.933 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.372518+0000", 2024-09-14T06:30:31.933 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.327e-06 2024-09-14T06:30:31.933 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:31.933 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:31.933 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:30:31.933 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.419530+0000", 2024-09-14T06:30:31.933 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047011798 2024-09-14T06:30:31.933 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:31.934 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:31.934 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:30:31.934 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.419578+0000", 2024-09-14T06:30:31.934 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.8446999999999999e-05 2024-09-14T06:30:31.934 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:31.934 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:31.934 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:30:31.934 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.419639+0000", 2024-09-14T06:30:31.934 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.0900999999999998e-05 2024-09-14T06:30:31.934 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:31.934 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:30:31.934 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:31.934 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:31.934 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:31.935 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:53352 3.f 3:f71f51a7:::rbd_data.112dca2f6ae1.0000000000000015:17 [read 1351680~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:30:31.935 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:44.372525+0000", 2024-09-14T06:30:31.935 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 47.483588801000003, 2024-09-14T06:30:31.935 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047259847000000001, 2024-09-14T06:30:31.935 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:30:31.935 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:30:31.935 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:30:31.935 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:30:31.935 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:30:31.935 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:31.935 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 53352 2024-09-14T06:30:31.935 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:31.935 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:30:31.935 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:31.936 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:30:31.936 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.372525+0000", 2024-09-14T06:30:31.936 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:31.936 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:31.936 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:31.936 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:30:31.936 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.372525+0000", 2024-09-14T06:30:31.936 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:31.936 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:31.936 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:31.936 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:30:31.936 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.372525+0000", 2024-09-14T06:30:31.936 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.0600000000000001e-07 2024-09-14T06:30:31.936 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:31.937 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:31.937 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:30:31.937 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.372527+0000", 2024-09-14T06:30:31.937 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.0499999999999999e-06 2024-09-14T06:30:31.937 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:31.937 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:31.937 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:30:31.937 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.372533+0000", 2024-09-14T06:30:31.937 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.4320000000000004e-06 2024-09-14T06:30:31.937 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:31.937 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:31.937 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:30:31.937 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.372538+0000", 2024-09-14T06:30:31.937 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.9130000000000002e-06 2024-09-14T06:30:31.938 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:31.938 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:31.938 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:30:31.938 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.419667+0000", 2024-09-14T06:30:31.938 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047128976000000003 2024-09-14T06:30:31.938 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:31.938 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:31.938 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:30:31.938 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.419715+0000", 2024-09-14T06:30:31.938 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.7868999999999999e-05 2024-09-14T06:30:31.938 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:31.938 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:31.938 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:30:31.938 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.419784+0000", 2024-09-14T06:30:31.938 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.9201000000000004e-05 2024-09-14T06:30:31.939 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:31.939 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:30:31.939 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:31.939 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:31.939 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:31.939 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:53353 3.f 3:f71f51a7:::rbd_data.112dca2f6ae1.0000000000000015:17 [read 1376256~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:30:31.939 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:44.372545+0000", 2024-09-14T06:30:31.939 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 47.483568761000001, 2024-09-14T06:30:31.939 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047364232999999999, 2024-09-14T06:30:31.939 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:30:31.939 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:30:31.939 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:30:31.939 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:30:31.939 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:30:31.940 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:31.940 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 53353 2024-09-14T06:30:31.940 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:31.940 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:30:31.940 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:31.940 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:30:31.940 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.372545+0000", 2024-09-14T06:30:31.940 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:31.940 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:31.940 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:31.940 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:30:31.940 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.372545+0000", 2024-09-14T06:30:31.940 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:31.940 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:31.941 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:31.941 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:30:31.941 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.372545+0000", 2024-09-14T06:30:31.941 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.27e-07 2024-09-14T06:30:31.941 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:31.941 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:31.941 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:30:31.941 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.372547+0000", 2024-09-14T06:30:31.941 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.0600000000000002e-06 2024-09-14T06:30:31.941 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:31.941 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:31.941 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:30:31.941 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.372553+0000", 2024-09-14T06:30:31.941 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.7980000000000002e-06 2024-09-14T06:30:31.942 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:31.942 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:31.942 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:30:31.942 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.372557+0000", 2024-09-14T06:30:31.942 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.7389999999999999e-06 2024-09-14T06:30:31.942 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:31.942 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:31.942 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:30:31.942 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.419811+0000", 2024-09-14T06:30:31.942 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047253310999999999 2024-09-14T06:30:31.942 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:31.942 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:31.942 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:30:31.942 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.419854+0000", 2024-09-14T06:30:31.943 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.3448000000000001e-05 2024-09-14T06:30:31.943 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:31.943 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:31.943 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:30:31.943 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.419909+0000", 2024-09-14T06:30:31.943 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.4549999999999998e-05 2024-09-14T06:30:31.943 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:31.943 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:30:31.943 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:31.943 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:31.943 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:31.943 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:53354 3.f 3:f71f51a7:::rbd_data.112dca2f6ae1.0000000000000015:17 [read 1400832~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:30:31.943 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:44.372563+0000", 2024-09-14T06:30:31.943 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 47.483550323999999, 2024-09-14T06:30:31.944 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047482233999999998, 2024-09-14T06:30:31.944 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:30:31.944 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:30:31.944 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:30:31.944 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:30:31.944 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:30:31.944 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:31.944 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 53354 2024-09-14T06:30:31.944 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:31.944 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:30:31.944 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:31.944 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:30:31.944 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.372563+0000", 2024-09-14T06:30:31.944 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:31.945 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:31.945 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:31.945 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:30:31.945 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.372563+0000", 2024-09-14T06:30:31.945 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:31.945 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:31.945 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:31.945 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:30:31.945 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.375009+0000", 2024-09-14T06:30:31.945 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.0024457680000000001 2024-09-14T06:30:31.945 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:31.945 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:31.945 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:30:31.945 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.375014+0000", 2024-09-14T06:30:31.946 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.0599999999999998e-06 2024-09-14T06:30:31.946 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:31.946 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:31.946 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:30:31.946 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.375026+0000", 2024-09-14T06:30:31.946 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.1783999999999999e-05 2024-09-14T06:30:31.946 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:31.946 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:31.946 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:30:31.946 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.375035+0000", 2024-09-14T06:30:31.946 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.7799999999999995e-06 2024-09-14T06:30:31.946 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:31.946 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:31.946 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:30:31.947 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.419942+0000", 2024-09-14T06:30:31.947 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.044906982999999998 2024-09-14T06:30:31.947 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:31.947 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:31.947 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:30:31.947 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.419989+0000", 2024-09-14T06:30:31.947 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.7104999999999997e-05 2024-09-14T06:30:31.947 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:31.947 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:31.947 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:30:31.947 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.420045+0000", 2024-09-14T06:30:31.947 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.5754000000000002e-05 2024-09-14T06:30:31.947 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:31.947 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:30:31.948 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:31.948 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:31.948 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:30:31.948 INFO:teuthology.orchestra.run.smithi138.stdout:} 2024-09-14T06:30:31.949 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-14T06:30:31.950 INFO:tasks.ceph.osd.3.smithi094.stderr:2024-09-14T06:30:31.948+0000 7feeeafe8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33655) UID: 0 2024-09-14T06:30:32.050 INFO:tasks.ceph.osd.5.smithi138.stderr:2024-09-14T06:30:32.048+0000 7fe13746f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33618) UID: 0 2024-09-14T06:30:32.070 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-14T06:30:32.071 INFO:teuthology.orchestra.run.smithi094.stdout: "ops": [], 2024-09-14T06:30:32.071 INFO:teuthology.orchestra.run.smithi094.stdout: "num_ops": 0 2024-09-14T06:30:32.071 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-14T06:30:32.081 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-14T06:30:32.148 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-14T06:30:32.150 INFO:tasks.ceph.osd.0.smithi094.stderr:2024-09-14T06:30:32.148+0000 7f07c9333640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33636) UID: 0 2024-09-14T06:30:32.189 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-14T06:30:32.190 INFO:teuthology.orchestra.run.smithi094.stdout: "ops": [], 2024-09-14T06:30:32.190 INFO:teuthology.orchestra.run.smithi094.stdout: "complaint_time": 30, 2024-09-14T06:30:32.190 INFO:teuthology.orchestra.run.smithi094.stdout: "num_blocked_ops": 0 2024-09-14T06:30:32.190 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-14T06:30:32.199 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-14T06:30:32.272 INFO:teuthology.orchestra.run.smithi094.stdout:osd.0: {} 2024-09-14T06:30:32.272 INFO:teuthology.orchestra.run.smithi094.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-14T06:30:32.273 INFO:tasks.ceph.osd.3.smithi094.stderr:2024-09-14T06:30:32.270+0000 7feeeafe8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33655) UID: 0 2024-09-14T06:30:32.305 INFO:teuthology.orchestra.run.smithi094.stdout:osd.1: {} 2024-09-14T06:30:32.305 INFO:teuthology.orchestra.run.smithi094.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-14T06:30:32.327 INFO:teuthology.orchestra.run.smithi094.stdout:osd.2: {} 2024-09-14T06:30:32.327 INFO:teuthology.orchestra.run.smithi094.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-14T06:30:32.343 INFO:teuthology.orchestra.run.smithi094.stdout:osd.3: {} 2024-09-14T06:30:32.343 INFO:teuthology.orchestra.run.smithi094.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-14T06:30:32.351 INFO:tasks.ceph.osd.5.smithi138.stderr:2024-09-14T06:30:32.351+0000 7fe13746f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33618) UID: 0 2024-09-14T06:30:32.360 INFO:teuthology.orchestra.run.smithi094.stdout:osd.4: {} 2024-09-14T06:30:32.360 INFO:teuthology.orchestra.run.smithi094.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-14T06:30:32.361 INFO:teuthology.orchestra.run.smithi094.stdout:ERROR: (22) Invalid argument 2024-09-14T06:30:32.361 INFO:teuthology.orchestra.run.smithi094.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:30:32.370 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-14T06:30:32.378 INFO:teuthology.orchestra.run.smithi094.stdout:osd.5: {} 2024-09-14T06:30:32.378 INFO:teuthology.orchestra.run.smithi094.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-14T06:30:32.394 INFO:teuthology.orchestra.run.smithi094.stdout:osd.6: {} 2024-09-14T06:30:32.394 INFO:teuthology.orchestra.run.smithi094.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-14T06:30:32.412 INFO:teuthology.orchestra.run.smithi094.stdout:osd.7: {} 2024-09-14T06:30:32.412 INFO:teuthology.orchestra.run.smithi094.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-14T06:30:32.465 INFO:tasks.ceph.osd.0.smithi094.stderr:2024-09-14T06:30:32.463+0000 7f07c9333640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33636) UID: 0 2024-09-14T06:30:32.483 INFO:teuthology.orchestra.run.smithi094.stdout:ERROR: (22) Invalid argument 2024-09-14T06:30:32.483 INFO:teuthology.orchestra.run.smithi094.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:30:32.492 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-14T06:30:32.576 INFO:tasks.ceph.osd.2.smithi094.stderr:2024-09-14T06:30:32.574+0000 7faf3269b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33649) UID: 0 2024-09-14T06:30:32.606 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-14T06:30:32.637 INFO:teuthology.orchestra.run.smithi094.stdout:ERROR: (22) Invalid argument 2024-09-14T06:30:32.637 INFO:teuthology.orchestra.run.smithi094.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:30:32.647 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-14T06:30:32.653 INFO:tasks.ceph.osd.4.smithi138.stderr:2024-09-14T06:30:32.652+0000 7f6c653c8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33615) UID: 0 2024-09-14T06:30:32.754 INFO:tasks.ceph.osd.4.smithi138.stderr:2024-09-14T06:30:32.753+0000 7f6c653c8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33615) UID: 0 2024-09-14T06:30:32.855 INFO:tasks.ceph.osd.5.smithi138.stderr:2024-09-14T06:30:32.854+0000 7fe13746f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33618) UID: 0 2024-09-14T06:30:32.856 INFO:teuthology.orchestra.run.smithi094.stdout:ERROR: (22) Invalid argument 2024-09-14T06:30:32.856 INFO:teuthology.orchestra.run.smithi094.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:30:32.866 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-14T06:30:32.956 INFO:tasks.ceph.osd.4.smithi138.stderr:2024-09-14T06:30:32.955+0000 7f6c653c8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33615) UID: 0 2024-09-14T06:30:32.970 INFO:teuthology.orchestra.run.smithi094.stdout:ERROR: (22) Invalid argument 2024-09-14T06:30:32.970 INFO:teuthology.orchestra.run.smithi094.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:30:32.979 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-14T06:30:33.058 INFO:tasks.ceph.osd.2.smithi094.stderr:2024-09-14T06:30:33.055+0000 7faf3269b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33649) UID: 0 2024-09-14T06:30:33.081 INFO:teuthology.orchestra.run.smithi094.stdout:ERROR: (22) Invalid argument 2024-09-14T06:30:33.082 INFO:teuthology.orchestra.run.smithi094.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:30:33.090 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-14T06:30:33.174 INFO:tasks.ceph.osd.3.smithi094.stderr:2024-09-14T06:30:33.172+0000 7feeeafe8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33655) UID: 0 2024-09-14T06:30:33.233 INFO:teuthology.orchestra.run.smithi094.stdout:ERROR: (22) Invalid argument 2024-09-14T06:30:33.233 INFO:teuthology.orchestra.run.smithi094.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:30:33.241 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-14T06:30:33.259 INFO:tasks.ceph.osd.5.smithi138.stderr:2024-09-14T06:30:33.259+0000 7fe13746f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33618) UID: 0 2024-09-14T06:30:33.360 INFO:tasks.ceph.osd.5.smithi138.stderr:2024-09-14T06:30:33.359+0000 7fe13746f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33618) UID: 0 2024-09-14T06:30:33.385 INFO:teuthology.orchestra.run.smithi094.stdout:ERROR: (22) Invalid argument 2024-09-14T06:30:33.385 INFO:teuthology.orchestra.run.smithi094.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:30:33.393 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-14T06:30:33.461 INFO:tasks.ceph.osd.7.smithi138.stderr:2024-09-14T06:30:33.461+0000 7f3c6a21d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33623) UID: 0 2024-09-14T06:30:33.495 INFO:teuthology.orchestra.run.smithi094.stdout:ERROR: (22) Invalid argument 2024-09-14T06:30:33.495 INFO:teuthology.orchestra.run.smithi094.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:30:33.503 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-14T06:30:33.562 INFO:tasks.ceph.osd.4.smithi138.stderr:2024-09-14T06:30:33.561+0000 7f6c653c8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33615) UID: 0 2024-09-14T06:30:33.605 INFO:teuthology.orchestra.run.smithi094.stdout:ERROR: (22) Invalid argument 2024-09-14T06:30:33.605 INFO:teuthology.orchestra.run.smithi094.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:30:33.613 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-14T06:30:33.649 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-14T06:30:33.649 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-14T06:30:33.649 INFO:tasks.thrashosds.thrasher:thrash_pg_upmap 2024-09-14T06:30:33.650 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump -f json-pretty 2024-09-14T06:30:33.654 INFO:teuthology.orchestra.run.smithi094.stderr:nodeep-scrub is unset 2024-09-14T06:30:33.662 INFO:tasks.ceph.osd.6.smithi138.stderr:2024-09-14T06:30:33.661+0000 7f8cd8849640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33620) UID: 0 2024-09-14T06:30:33.728 INFO:teuthology.orchestra.run.smithi138.stdout:ERROR: (22) Invalid argument 2024-09-14T06:30:33.728 INFO:teuthology.orchestra.run.smithi138.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:30:33.737 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-14T06:30:33.763 INFO:tasks.ceph.osd.1.smithi094.stderr:2024-09-14T06:30:33.761+0000 7f328e47a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33641) UID: 0 2024-09-14T06:30:33.817 INFO:teuthology.orchestra.run.smithi094.stdout:checking clone #22, image image_client.0-clone22 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent23 2024-09-14T06:30:33.864 INFO:tasks.ceph.osd.1.smithi094.stderr:2024-09-14T06:30:33.862+0000 7f328e47a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33641) UID: 0 2024-09-14T06:30:33.890 INFO:teuthology.orchestra.run.smithi138.stdout:ERROR: (22) Invalid argument 2024-09-14T06:30:33.890 INFO:teuthology.orchestra.run.smithi138.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:30:33.899 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-14T06:30:33.965 INFO:tasks.ceph.osd.4.smithi138.stderr:2024-09-14T06:30:33.963+0000 7f6c653c8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33615) UID: 0 2024-09-14T06:30:33.970 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-14T06:30:33.970 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-14T06:30:33.970 INFO:teuthology.orchestra.run.smithi094.stdout: "epoch": 111, 2024-09-14T06:30:33.970 INFO:teuthology.orchestra.run.smithi094.stdout: "fsid": "200ad3da-bd63-4e04-af74-efb5ca0af75e", 2024-09-14T06:30:33.970 INFO:teuthology.orchestra.run.smithi094.stdout: "created": "2024-09-14T06:28:30.782046+0000", 2024-09-14T06:30:33.970 INFO:teuthology.orchestra.run.smithi094.stdout: "modified": "2024-09-14T06:30:33.646401+0000", 2024-09-14T06:30:33.970 INFO:teuthology.orchestra.run.smithi094.stdout: "last_up_change": "2024-09-14T06:28:42.827239+0000", 2024-09-14T06:30:33.970 INFO:teuthology.orchestra.run.smithi094.stdout: "last_in_change": "2024-09-14T06:28:33.724176+0000", 2024-09-14T06:30:33.970 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2024-09-14T06:30:33.971 INFO:teuthology.orchestra.run.smithi094.stdout: "flags_num": 5799936, 2024-09-14T06:30:33.971 INFO:teuthology.orchestra.run.smithi094.stdout: "flags_set": [ 2024-09-14T06:30:33.971 INFO:teuthology.orchestra.run.smithi094.stdout: "pglog_hardlimit", 2024-09-14T06:30:33.971 INFO:teuthology.orchestra.run.smithi094.stdout: "purged_snapdirs", 2024-09-14T06:30:33.971 INFO:teuthology.orchestra.run.smithi094.stdout: "recovery_deletes", 2024-09-14T06:30:33.971 INFO:teuthology.orchestra.run.smithi094.stdout: "sortbitwise" 2024-09-14T06:30:33.971 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-09-14T06:30:33.971 INFO:teuthology.orchestra.run.smithi094.stdout: "crush_version": 5, 2024-09-14T06:30:33.971 INFO:teuthology.orchestra.run.smithi094.stdout: "full_ratio": 0.94999998807907104, 2024-09-14T06:30:33.971 INFO:teuthology.orchestra.run.smithi094.stdout: "backfillfull_ratio": 0.89999997615814209, 2024-09-14T06:30:33.972 INFO:teuthology.orchestra.run.smithi094.stdout: "nearfull_ratio": 0.85000002384185791, 2024-09-14T06:30:33.972 INFO:teuthology.orchestra.run.smithi094.stdout: "cluster_snapshot": "", 2024-09-14T06:30:33.972 INFO:teuthology.orchestra.run.smithi094.stdout: "pool_max": 3, 2024-09-14T06:30:33.973 INFO:teuthology.orchestra.run.smithi094.stdout: "max_osd": 8, 2024-09-14T06:30:33.973 INFO:teuthology.orchestra.run.smithi094.stdout: "require_min_compat_client": "luminous", 2024-09-14T06:30:33.973 INFO:teuthology.orchestra.run.smithi094.stdout: "min_compat_client": "luminous", 2024-09-14T06:30:33.973 INFO:teuthology.orchestra.run.smithi094.stdout: "require_osd_release": "squid", 2024-09-14T06:30:33.973 INFO:teuthology.orchestra.run.smithi094.stdout: "allow_crimson": false, 2024-09-14T06:30:33.973 INFO:teuthology.orchestra.run.smithi094.stdout: "pools": [ 2024-09-14T06:30:33.973 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:33.973 INFO:teuthology.orchestra.run.smithi094.stdout: "pool": 1, 2024-09-14T06:30:33.973 INFO:teuthology.orchestra.run.smithi094.stdout: "pool_name": ".mgr", 2024-09-14T06:30:33.974 INFO:teuthology.orchestra.run.smithi094.stdout: "create_time": "2024-09-14T06:28:43.295990+0000", 2024-09-14T06:30:33.974 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-14T06:30:33.974 INFO:teuthology.orchestra.run.smithi094.stdout: "flags_names": "hashpspool", 2024-09-14T06:30:33.974 INFO:teuthology.orchestra.run.smithi094.stdout: "type": 1, 2024-09-14T06:30:33.974 INFO:teuthology.orchestra.run.smithi094.stdout: "size": 2, 2024-09-14T06:30:33.974 INFO:teuthology.orchestra.run.smithi094.stdout: "min_size": 1, 2024-09-14T06:30:33.974 INFO:teuthology.orchestra.run.smithi094.stdout: "crush_rule": 0, 2024-09-14T06:30:33.974 INFO:teuthology.orchestra.run.smithi094.stdout: "peering_crush_bucket_count": 0, 2024-09-14T06:30:33.974 INFO:teuthology.orchestra.run.smithi094.stdout: "peering_crush_bucket_target": 0, 2024-09-14T06:30:33.974 INFO:teuthology.orchestra.run.smithi094.stdout: "peering_crush_bucket_barrier": 0, 2024-09-14T06:30:33.974 INFO:teuthology.orchestra.run.smithi094.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-14T06:30:33.974 INFO:teuthology.orchestra.run.smithi094.stdout: "is_stretch_pool": false, 2024-09-14T06:30:33.974 INFO:teuthology.orchestra.run.smithi094.stdout: "object_hash": 2, 2024-09-14T06:30:33.974 INFO:teuthology.orchestra.run.smithi094.stdout: "pg_autoscale_mode": "off", 2024-09-14T06:30:33.975 INFO:teuthology.orchestra.run.smithi094.stdout: "pg_num": 1, 2024-09-14T06:30:33.975 INFO:teuthology.orchestra.run.smithi094.stdout: "pg_placement_num": 1, 2024-09-14T06:30:33.975 INFO:teuthology.orchestra.run.smithi094.stdout: "pg_placement_num_target": 1, 2024-09-14T06:30:33.975 INFO:teuthology.orchestra.run.smithi094.stdout: "pg_num_target": 1, 2024-09-14T06:30:33.975 INFO:teuthology.orchestra.run.smithi094.stdout: "pg_num_pending": 1, 2024-09-14T06:30:33.975 INFO:teuthology.orchestra.run.smithi094.stdout: "last_pg_merge_meta": { 2024-09-14T06:30:33.975 INFO:teuthology.orchestra.run.smithi094.stdout: "source_pgid": "0.0", 2024-09-14T06:30:33.975 INFO:teuthology.orchestra.run.smithi094.stdout: "ready_epoch": 0, 2024-09-14T06:30:33.975 INFO:teuthology.orchestra.run.smithi094.stdout: "last_epoch_started": 0, 2024-09-14T06:30:33.975 INFO:teuthology.orchestra.run.smithi094.stdout: "last_epoch_clean": 0, 2024-09-14T06:30:33.975 INFO:teuthology.orchestra.run.smithi094.stdout: "source_version": "0'0", 2024-09-14T06:30:33.975 INFO:teuthology.orchestra.run.smithi094.stdout: "target_version": "0'0" 2024-09-14T06:30:33.975 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:33.975 INFO:teuthology.orchestra.run.smithi094.stdout: "last_change": "18", 2024-09-14T06:30:33.976 INFO:teuthology.orchestra.run.smithi094.stdout: "last_force_op_resend": "0", 2024-09-14T06:30:33.976 INFO:teuthology.orchestra.run.smithi094.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-14T06:30:33.976 INFO:teuthology.orchestra.run.smithi094.stdout: "last_force_op_resend_preluminous": "0", 2024-09-14T06:30:33.976 INFO:teuthology.orchestra.run.smithi094.stdout: "auid": 0, 2024-09-14T06:30:33.976 INFO:teuthology.orchestra.run.smithi094.stdout: "snap_mode": "selfmanaged", 2024-09-14T06:30:33.976 INFO:teuthology.orchestra.run.smithi094.stdout: "snap_seq": 0, 2024-09-14T06:30:33.976 INFO:teuthology.orchestra.run.smithi094.stdout: "snap_epoch": 0, 2024-09-14T06:30:33.976 INFO:teuthology.orchestra.run.smithi094.stdout: "pool_snaps": [], 2024-09-14T06:30:33.976 INFO:teuthology.orchestra.run.smithi094.stdout: "removed_snaps": "[]", 2024-09-14T06:30:33.976 INFO:teuthology.orchestra.run.smithi094.stdout: "quota_max_bytes": 0, 2024-09-14T06:30:33.976 INFO:teuthology.orchestra.run.smithi094.stdout: "quota_max_objects": 0, 2024-09-14T06:30:33.976 INFO:teuthology.orchestra.run.smithi094.stdout: "tiers": [], 2024-09-14T06:30:33.976 INFO:teuthology.orchestra.run.smithi094.stdout: "tier_of": -1, 2024-09-14T06:30:33.976 INFO:teuthology.orchestra.run.smithi094.stdout: "read_tier": -1, 2024-09-14T06:30:33.977 INFO:teuthology.orchestra.run.smithi094.stdout: "write_tier": -1, 2024-09-14T06:30:33.977 INFO:teuthology.orchestra.run.smithi094.stdout: "cache_mode": "none", 2024-09-14T06:30:33.977 INFO:teuthology.orchestra.run.smithi094.stdout: "target_max_bytes": 0, 2024-09-14T06:30:33.977 INFO:teuthology.orchestra.run.smithi094.stdout: "target_max_objects": 0, 2024-09-14T06:30:33.977 INFO:teuthology.orchestra.run.smithi094.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-14T06:30:33.977 INFO:teuthology.orchestra.run.smithi094.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-14T06:30:33.977 INFO:teuthology.orchestra.run.smithi094.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-14T06:30:33.977 INFO:teuthology.orchestra.run.smithi094.stdout: "cache_min_flush_age": 0, 2024-09-14T06:30:33.977 INFO:teuthology.orchestra.run.smithi094.stdout: "cache_min_evict_age": 0, 2024-09-14T06:30:33.977 INFO:teuthology.orchestra.run.smithi094.stdout: "erasure_code_profile": "", 2024-09-14T06:30:33.977 INFO:teuthology.orchestra.run.smithi094.stdout: "hit_set_params": { 2024-09-14T06:30:33.977 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "none" 2024-09-14T06:30:33.977 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:33.977 INFO:teuthology.orchestra.run.smithi094.stdout: "hit_set_period": 0, 2024-09-14T06:30:33.978 INFO:teuthology.orchestra.run.smithi094.stdout: "hit_set_count": 0, 2024-09-14T06:30:33.978 INFO:teuthology.orchestra.run.smithi094.stdout: "use_gmt_hitset": true, 2024-09-14T06:30:33.978 INFO:teuthology.orchestra.run.smithi094.stdout: "min_read_recency_for_promote": 0, 2024-09-14T06:30:33.978 INFO:teuthology.orchestra.run.smithi094.stdout: "min_write_recency_for_promote": 0, 2024-09-14T06:30:33.978 INFO:teuthology.orchestra.run.smithi094.stdout: "hit_set_grade_decay_rate": 0, 2024-09-14T06:30:33.978 INFO:teuthology.orchestra.run.smithi094.stdout: "hit_set_search_last_n": 0, 2024-09-14T06:30:33.978 INFO:teuthology.orchestra.run.smithi094.stdout: "grade_table": [], 2024-09-14T06:30:33.978 INFO:teuthology.orchestra.run.smithi094.stdout: "stripe_width": 0, 2024-09-14T06:30:33.978 INFO:teuthology.orchestra.run.smithi094.stdout: "expected_num_objects": 0, 2024-09-14T06:30:33.978 INFO:teuthology.orchestra.run.smithi094.stdout: "fast_read": false, 2024-09-14T06:30:33.978 INFO:teuthology.orchestra.run.smithi094.stdout: "options": { 2024-09-14T06:30:33.978 INFO:teuthology.orchestra.run.smithi094.stdout: "pg_num_max": 32, 2024-09-14T06:30:33.978 INFO:teuthology.orchestra.run.smithi094.stdout: "pg_num_min": 1 2024-09-14T06:30:33.979 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:33.979 INFO:teuthology.orchestra.run.smithi094.stdout: "application_metadata": { 2024-09-14T06:30:33.979 INFO:teuthology.orchestra.run.smithi094.stdout: "mgr": {} 2024-09-14T06:30:33.979 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:33.979 INFO:teuthology.orchestra.run.smithi094.stdout: "read_balance": { 2024-09-14T06:30:33.979 INFO:teuthology.orchestra.run.smithi094.stdout: "score_type": "Fair distribution", 2024-09-14T06:30:33.979 INFO:teuthology.orchestra.run.smithi094.stdout: "score_acting": 8, 2024-09-14T06:30:33.979 INFO:teuthology.orchestra.run.smithi094.stdout: "score_stable": 8, 2024-09-14T06:30:33.979 INFO:teuthology.orchestra.run.smithi094.stdout: "optimal_score": 0.25, 2024-09-14T06:30:33.979 INFO:teuthology.orchestra.run.smithi094.stdout: "raw_score_acting": 2, 2024-09-14T06:30:33.979 INFO:teuthology.orchestra.run.smithi094.stdout: "raw_score_stable": 2, 2024-09-14T06:30:33.979 INFO:teuthology.orchestra.run.smithi094.stdout: "primary_affinity_weighted": 1, 2024-09-14T06:30:33.979 INFO:teuthology.orchestra.run.smithi094.stdout: "average_primary_affinity": 1, 2024-09-14T06:30:33.980 INFO:teuthology.orchestra.run.smithi094.stdout: "average_primary_affinity_weighted": 1 2024-09-14T06:30:33.980 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:33.980 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:33.980 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:33.980 INFO:teuthology.orchestra.run.smithi094.stdout: "pool": 2, 2024-09-14T06:30:33.980 INFO:teuthology.orchestra.run.smithi094.stdout: "pool_name": "rbd", 2024-09-14T06:30:33.980 INFO:teuthology.orchestra.run.smithi094.stdout: "create_time": "2024-09-14T06:28:49.451714+0000", 2024-09-14T06:30:33.980 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 8193, 2024-09-14T06:30:33.980 INFO:teuthology.orchestra.run.smithi094.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-09-14T06:30:33.980 INFO:teuthology.orchestra.run.smithi094.stdout: "type": 1, 2024-09-14T06:30:33.980 INFO:teuthology.orchestra.run.smithi094.stdout: "size": 2, 2024-09-14T06:30:33.980 INFO:teuthology.orchestra.run.smithi094.stdout: "min_size": 1, 2024-09-14T06:30:33.980 INFO:teuthology.orchestra.run.smithi094.stdout: "crush_rule": 0, 2024-09-14T06:30:33.981 INFO:teuthology.orchestra.run.smithi094.stdout: "peering_crush_bucket_count": 0, 2024-09-14T06:30:33.981 INFO:teuthology.orchestra.run.smithi094.stdout: "peering_crush_bucket_target": 0, 2024-09-14T06:30:33.981 INFO:teuthology.orchestra.run.smithi094.stdout: "peering_crush_bucket_barrier": 0, 2024-09-14T06:30:33.981 INFO:teuthology.orchestra.run.smithi094.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-14T06:30:33.981 INFO:teuthology.orchestra.run.smithi094.stdout: "is_stretch_pool": false, 2024-09-14T06:30:33.981 INFO:teuthology.orchestra.run.smithi094.stdout: "object_hash": 2, 2024-09-14T06:30:33.981 INFO:teuthology.orchestra.run.smithi094.stdout: "pg_autoscale_mode": "off", 2024-09-14T06:30:33.981 INFO:teuthology.orchestra.run.smithi094.stdout: "pg_num": 8, 2024-09-14T06:30:33.981 INFO:teuthology.orchestra.run.smithi094.stdout: "pg_placement_num": 8, 2024-09-14T06:30:33.981 INFO:teuthology.orchestra.run.smithi094.stdout: "pg_placement_num_target": 8, 2024-09-14T06:30:33.981 INFO:teuthology.orchestra.run.smithi094.stdout: "pg_num_target": 8, 2024-09-14T06:30:33.981 INFO:teuthology.orchestra.run.smithi094.stdout: "pg_num_pending": 8, 2024-09-14T06:30:33.981 INFO:teuthology.orchestra.run.smithi094.stdout: "last_pg_merge_meta": { 2024-09-14T06:30:33.982 INFO:teuthology.orchestra.run.smithi094.stdout: "source_pgid": "0.0", 2024-09-14T06:30:33.982 INFO:teuthology.orchestra.run.smithi094.stdout: "ready_epoch": 0, 2024-09-14T06:30:33.982 INFO:teuthology.orchestra.run.smithi094.stdout: "last_epoch_started": 0, 2024-09-14T06:30:33.982 INFO:teuthology.orchestra.run.smithi094.stdout: "last_epoch_clean": 0, 2024-09-14T06:30:33.982 INFO:teuthology.orchestra.run.smithi094.stdout: "source_version": "0'0", 2024-09-14T06:30:33.982 INFO:teuthology.orchestra.run.smithi094.stdout: "target_version": "0'0" 2024-09-14T06:30:33.982 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:33.982 INFO:teuthology.orchestra.run.smithi094.stdout: "last_change": "22", 2024-09-14T06:30:33.982 INFO:teuthology.orchestra.run.smithi094.stdout: "last_force_op_resend": "0", 2024-09-14T06:30:33.982 INFO:teuthology.orchestra.run.smithi094.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-14T06:30:33.982 INFO:teuthology.orchestra.run.smithi094.stdout: "last_force_op_resend_preluminous": "0", 2024-09-14T06:30:33.982 INFO:teuthology.orchestra.run.smithi094.stdout: "auid": 0, 2024-09-14T06:30:33.982 INFO:teuthology.orchestra.run.smithi094.stdout: "snap_mode": "selfmanaged", 2024-09-14T06:30:33.983 INFO:teuthology.orchestra.run.smithi094.stdout: "snap_seq": 2, 2024-09-14T06:30:33.983 INFO:teuthology.orchestra.run.smithi094.stdout: "snap_epoch": 22, 2024-09-14T06:30:33.983 INFO:teuthology.orchestra.run.smithi094.stdout: "pool_snaps": [], 2024-09-14T06:30:33.983 INFO:teuthology.orchestra.run.smithi094.stdout: "removed_snaps": "[]", 2024-09-14T06:30:33.983 INFO:teuthology.orchestra.run.smithi094.stdout: "quota_max_bytes": 0, 2024-09-14T06:30:33.983 INFO:teuthology.orchestra.run.smithi094.stdout: "quota_max_objects": 0, 2024-09-14T06:30:33.983 INFO:teuthology.orchestra.run.smithi094.stdout: "tiers": [], 2024-09-14T06:30:33.983 INFO:teuthology.orchestra.run.smithi094.stdout: "tier_of": -1, 2024-09-14T06:30:33.983 INFO:teuthology.orchestra.run.smithi094.stdout: "read_tier": -1, 2024-09-14T06:30:33.983 INFO:teuthology.orchestra.run.smithi094.stdout: "write_tier": -1, 2024-09-14T06:30:33.983 INFO:teuthology.orchestra.run.smithi094.stdout: "cache_mode": "none", 2024-09-14T06:30:33.983 INFO:teuthology.orchestra.run.smithi094.stdout: "target_max_bytes": 0, 2024-09-14T06:30:33.983 INFO:teuthology.orchestra.run.smithi094.stdout: "target_max_objects": 0, 2024-09-14T06:30:33.984 INFO:teuthology.orchestra.run.smithi094.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-14T06:30:33.984 INFO:teuthology.orchestra.run.smithi094.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-14T06:30:33.984 INFO:teuthology.orchestra.run.smithi094.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-14T06:30:33.984 INFO:teuthology.orchestra.run.smithi094.stdout: "cache_min_flush_age": 0, 2024-09-14T06:30:33.984 INFO:teuthology.orchestra.run.smithi094.stdout: "cache_min_evict_age": 0, 2024-09-14T06:30:33.984 INFO:teuthology.orchestra.run.smithi094.stdout: "erasure_code_profile": "", 2024-09-14T06:30:33.984 INFO:teuthology.orchestra.run.smithi094.stdout: "hit_set_params": { 2024-09-14T06:30:33.984 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "none" 2024-09-14T06:30:33.984 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:33.984 INFO:teuthology.orchestra.run.smithi094.stdout: "hit_set_period": 0, 2024-09-14T06:30:33.984 INFO:teuthology.orchestra.run.smithi094.stdout: "hit_set_count": 0, 2024-09-14T06:30:33.984 INFO:teuthology.orchestra.run.smithi094.stdout: "use_gmt_hitset": true, 2024-09-14T06:30:33.984 INFO:teuthology.orchestra.run.smithi094.stdout: "min_read_recency_for_promote": 0, 2024-09-14T06:30:33.985 INFO:teuthology.orchestra.run.smithi094.stdout: "min_write_recency_for_promote": 0, 2024-09-14T06:30:33.985 INFO:teuthology.orchestra.run.smithi094.stdout: "hit_set_grade_decay_rate": 0, 2024-09-14T06:30:33.985 INFO:teuthology.orchestra.run.smithi094.stdout: "hit_set_search_last_n": 0, 2024-09-14T06:30:33.985 INFO:teuthology.orchestra.run.smithi094.stdout: "grade_table": [], 2024-09-14T06:30:33.985 INFO:teuthology.orchestra.run.smithi094.stdout: "stripe_width": 0, 2024-09-14T06:30:33.985 INFO:teuthology.orchestra.run.smithi094.stdout: "expected_num_objects": 0, 2024-09-14T06:30:33.985 INFO:teuthology.orchestra.run.smithi094.stdout: "fast_read": false, 2024-09-14T06:30:33.985 INFO:teuthology.orchestra.run.smithi094.stdout: "options": {}, 2024-09-14T06:30:33.985 INFO:teuthology.orchestra.run.smithi094.stdout: "application_metadata": { 2024-09-14T06:30:33.985 INFO:teuthology.orchestra.run.smithi094.stdout: "rbd": {} 2024-09-14T06:30:33.985 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:33.985 INFO:teuthology.orchestra.run.smithi094.stdout: "read_balance": { 2024-09-14T06:30:33.985 INFO:teuthology.orchestra.run.smithi094.stdout: "score_type": "Fair distribution", 2024-09-14T06:30:33.985 INFO:teuthology.orchestra.run.smithi094.stdout: "score_acting": 1.9900000095367432, 2024-09-14T06:30:33.986 INFO:teuthology.orchestra.run.smithi094.stdout: "score_stable": 1.9900000095367432, 2024-09-14T06:30:33.986 INFO:teuthology.orchestra.run.smithi094.stdout: "optimal_score": 0.87999999523162842, 2024-09-14T06:30:33.986 INFO:teuthology.orchestra.run.smithi094.stdout: "raw_score_acting": 1.75, 2024-09-14T06:30:33.986 INFO:teuthology.orchestra.run.smithi094.stdout: "raw_score_stable": 1.75, 2024-09-14T06:30:33.986 INFO:teuthology.orchestra.run.smithi094.stdout: "primary_affinity_weighted": 1, 2024-09-14T06:30:33.986 INFO:teuthology.orchestra.run.smithi094.stdout: "average_primary_affinity": 1, 2024-09-14T06:30:33.986 INFO:teuthology.orchestra.run.smithi094.stdout: "average_primary_affinity_weighted": 1 2024-09-14T06:30:33.986 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:33.986 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:33.986 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:33.986 INFO:teuthology.orchestra.run.smithi094.stdout: "pool": 3, 2024-09-14T06:30:33.986 INFO:teuthology.orchestra.run.smithi094.stdout: "pool_name": "pool_client.0", 2024-09-14T06:30:33.986 INFO:teuthology.orchestra.run.smithi094.stdout: "create_time": "2024-09-14T06:28:59.688535+0000", 2024-09-14T06:30:33.986 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 8193, 2024-09-14T06:30:33.986 INFO:teuthology.orchestra.run.smithi094.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-09-14T06:30:33.987 INFO:teuthology.orchestra.run.smithi094.stdout: "type": 1, 2024-09-14T06:30:33.987 INFO:teuthology.orchestra.run.smithi094.stdout: "size": 2, 2024-09-14T06:30:33.987 INFO:teuthology.orchestra.run.smithi094.stdout: "min_size": 1, 2024-09-14T06:30:33.987 INFO:teuthology.orchestra.run.smithi094.stdout: "crush_rule": 0, 2024-09-14T06:30:33.987 INFO:teuthology.orchestra.run.smithi094.stdout: "peering_crush_bucket_count": 0, 2024-09-14T06:30:33.987 INFO:teuthology.orchestra.run.smithi094.stdout: "peering_crush_bucket_target": 0, 2024-09-14T06:30:33.987 INFO:teuthology.orchestra.run.smithi094.stdout: "peering_crush_bucket_barrier": 0, 2024-09-14T06:30:33.987 INFO:teuthology.orchestra.run.smithi094.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-14T06:30:33.987 INFO:teuthology.orchestra.run.smithi094.stdout: "is_stretch_pool": false, 2024-09-14T06:30:33.987 INFO:teuthology.orchestra.run.smithi094.stdout: "object_hash": 2, 2024-09-14T06:30:33.987 INFO:teuthology.orchestra.run.smithi094.stdout: "pg_autoscale_mode": "off", 2024-09-14T06:30:33.987 INFO:teuthology.orchestra.run.smithi094.stdout: "pg_num": 32, 2024-09-14T06:30:33.987 INFO:teuthology.orchestra.run.smithi094.stdout: "pg_placement_num": 32, 2024-09-14T06:30:33.987 INFO:teuthology.orchestra.run.smithi094.stdout: "pg_placement_num_target": 32, 2024-09-14T06:30:33.987 INFO:teuthology.orchestra.run.smithi094.stdout: "pg_num_target": 32, 2024-09-14T06:30:33.987 INFO:teuthology.orchestra.run.smithi094.stdout: "pg_num_pending": 32, 2024-09-14T06:30:33.988 INFO:teuthology.orchestra.run.smithi094.stdout: "last_pg_merge_meta": { 2024-09-14T06:30:33.988 INFO:teuthology.orchestra.run.smithi094.stdout: "source_pgid": "0.0", 2024-09-14T06:30:33.988 INFO:teuthology.orchestra.run.smithi094.stdout: "ready_epoch": 0, 2024-09-14T06:30:33.988 INFO:teuthology.orchestra.run.smithi094.stdout: "last_epoch_started": 0, 2024-09-14T06:30:33.988 INFO:teuthology.orchestra.run.smithi094.stdout: "last_epoch_clean": 0, 2024-09-14T06:30:33.988 INFO:teuthology.orchestra.run.smithi094.stdout: "source_version": "0'0", 2024-09-14T06:30:33.988 INFO:teuthology.orchestra.run.smithi094.stdout: "target_version": "0'0" 2024-09-14T06:30:33.988 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:33.988 INFO:teuthology.orchestra.run.smithi094.stdout: "last_change": "111", 2024-09-14T06:30:33.988 INFO:teuthology.orchestra.run.smithi094.stdout: "last_force_op_resend": "0", 2024-09-14T06:30:33.988 INFO:teuthology.orchestra.run.smithi094.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-14T06:30:33.988 INFO:teuthology.orchestra.run.smithi094.stdout: "last_force_op_resend_preluminous": "0", 2024-09-14T06:30:33.988 INFO:teuthology.orchestra.run.smithi094.stdout: "auid": 0, 2024-09-14T06:30:33.988 INFO:teuthology.orchestra.run.smithi094.stdout: "snap_mode": "selfmanaged", 2024-09-14T06:30:33.989 INFO:teuthology.orchestra.run.smithi094.stdout: "snap_seq": 50, 2024-09-14T06:30:33.989 INFO:teuthology.orchestra.run.smithi094.stdout: "snap_epoch": 111, 2024-09-14T06:30:33.989 INFO:teuthology.orchestra.run.smithi094.stdout: "pool_snaps": [], 2024-09-14T06:30:33.989 INFO:teuthology.orchestra.run.smithi094.stdout: "removed_snaps": "[]", 2024-09-14T06:30:33.989 INFO:teuthology.orchestra.run.smithi094.stdout: "quota_max_bytes": 0, 2024-09-14T06:30:33.989 INFO:teuthology.orchestra.run.smithi094.stdout: "quota_max_objects": 0, 2024-09-14T06:30:33.989 INFO:teuthology.orchestra.run.smithi094.stdout: "tiers": [], 2024-09-14T06:30:33.989 INFO:teuthology.orchestra.run.smithi094.stdout: "tier_of": -1, 2024-09-14T06:30:33.989 INFO:teuthology.orchestra.run.smithi094.stdout: "read_tier": -1, 2024-09-14T06:30:33.989 INFO:teuthology.orchestra.run.smithi094.stdout: "write_tier": -1, 2024-09-14T06:30:33.989 INFO:teuthology.orchestra.run.smithi094.stdout: "cache_mode": "none", 2024-09-14T06:30:33.989 INFO:teuthology.orchestra.run.smithi094.stdout: "target_max_bytes": 0, 2024-09-14T06:30:33.989 INFO:teuthology.orchestra.run.smithi094.stdout: "target_max_objects": 0, 2024-09-14T06:30:33.989 INFO:teuthology.orchestra.run.smithi094.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-14T06:30:33.990 INFO:teuthology.orchestra.run.smithi094.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-14T06:30:33.990 INFO:teuthology.orchestra.run.smithi094.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-14T06:30:33.990 INFO:teuthology.orchestra.run.smithi094.stdout: "cache_min_flush_age": 0, 2024-09-14T06:30:33.990 INFO:teuthology.orchestra.run.smithi094.stdout: "cache_min_evict_age": 0, 2024-09-14T06:30:33.990 INFO:teuthology.orchestra.run.smithi094.stdout: "erasure_code_profile": "", 2024-09-14T06:30:33.990 INFO:teuthology.orchestra.run.smithi094.stdout: "hit_set_params": { 2024-09-14T06:30:33.990 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "none" 2024-09-14T06:30:33.990 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:33.990 INFO:teuthology.orchestra.run.smithi094.stdout: "hit_set_period": 0, 2024-09-14T06:30:33.990 INFO:teuthology.orchestra.run.smithi094.stdout: "hit_set_count": 0, 2024-09-14T06:30:33.990 INFO:teuthology.orchestra.run.smithi094.stdout: "use_gmt_hitset": true, 2024-09-14T06:30:33.990 INFO:teuthology.orchestra.run.smithi094.stdout: "min_read_recency_for_promote": 0, 2024-09-14T06:30:33.990 INFO:teuthology.orchestra.run.smithi094.stdout: "min_write_recency_for_promote": 0, 2024-09-14T06:30:33.991 INFO:teuthology.orchestra.run.smithi094.stdout: "hit_set_grade_decay_rate": 0, 2024-09-14T06:30:33.991 INFO:teuthology.orchestra.run.smithi094.stdout: "hit_set_search_last_n": 0, 2024-09-14T06:30:33.991 INFO:teuthology.orchestra.run.smithi094.stdout: "grade_table": [], 2024-09-14T06:30:33.991 INFO:teuthology.orchestra.run.smithi094.stdout: "stripe_width": 0, 2024-09-14T06:30:33.991 INFO:teuthology.orchestra.run.smithi094.stdout: "expected_num_objects": 0, 2024-09-14T06:30:33.991 INFO:teuthology.orchestra.run.smithi094.stdout: "fast_read": false, 2024-09-14T06:30:33.991 INFO:teuthology.orchestra.run.smithi094.stdout: "options": {}, 2024-09-14T06:30:33.991 INFO:teuthology.orchestra.run.smithi094.stdout: "application_metadata": { 2024-09-14T06:30:33.991 INFO:teuthology.orchestra.run.smithi094.stdout: "rbd": {} 2024-09-14T06:30:33.991 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:33.991 INFO:teuthology.orchestra.run.smithi094.stdout: "read_balance": { 2024-09-14T06:30:33.991 INFO:teuthology.orchestra.run.smithi094.stdout: "score_type": "Fair distribution", 2024-09-14T06:30:33.991 INFO:teuthology.orchestra.run.smithi094.stdout: "score_acting": 1.75, 2024-09-14T06:30:33.991 INFO:teuthology.orchestra.run.smithi094.stdout: "score_stable": 1.75, 2024-09-14T06:30:33.992 INFO:teuthology.orchestra.run.smithi094.stdout: "optimal_score": 1, 2024-09-14T06:30:33.992 INFO:teuthology.orchestra.run.smithi094.stdout: "raw_score_acting": 1.75, 2024-09-14T06:30:33.992 INFO:teuthology.orchestra.run.smithi094.stdout: "raw_score_stable": 1.75, 2024-09-14T06:30:33.992 INFO:teuthology.orchestra.run.smithi094.stdout: "primary_affinity_weighted": 1, 2024-09-14T06:30:33.992 INFO:teuthology.orchestra.run.smithi094.stdout: "average_primary_affinity": 1, 2024-09-14T06:30:33.992 INFO:teuthology.orchestra.run.smithi094.stdout: "average_primary_affinity_weighted": 1 2024-09-14T06:30:33.992 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:33.992 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:33.992 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-09-14T06:30:33.992 INFO:teuthology.orchestra.run.smithi094.stdout: "osds": [ 2024-09-14T06:30:33.992 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:33.992 INFO:teuthology.orchestra.run.smithi094.stdout: "osd": 0, 2024-09-14T06:30:33.992 INFO:teuthology.orchestra.run.smithi094.stdout: "uuid": "8b22a249-b842-406a-a02c-c0f31f57b635", 2024-09-14T06:30:33.993 INFO:teuthology.orchestra.run.smithi094.stdout: "up": 1, 2024-09-14T06:30:33.993 INFO:teuthology.orchestra.run.smithi094.stdout: "in": 1, 2024-09-14T06:30:33.993 INFO:teuthology.orchestra.run.smithi094.stdout: "weight": 1, 2024-09-14T06:30:33.993 INFO:teuthology.orchestra.run.smithi094.stdout: "primary_affinity": 1, 2024-09-14T06:30:33.993 INFO:teuthology.orchestra.run.smithi094.stdout: "last_clean_begin": 0, 2024-09-14T06:30:33.993 INFO:teuthology.orchestra.run.smithi094.stdout: "last_clean_end": 0, 2024-09-14T06:30:33.993 INFO:teuthology.orchestra.run.smithi094.stdout: "up_from": 14, 2024-09-14T06:30:33.993 INFO:teuthology.orchestra.run.smithi094.stdout: "up_thru": 75, 2024-09-14T06:30:33.993 INFO:teuthology.orchestra.run.smithi094.stdout: "down_at": 0, 2024-09-14T06:30:33.993 INFO:teuthology.orchestra.run.smithi094.stdout: "lost_at": 0, 2024-09-14T06:30:33.993 INFO:teuthology.orchestra.run.smithi094.stdout: "public_addrs": { 2024-09-14T06:30:33.993 INFO:teuthology.orchestra.run.smithi094.stdout: "addrvec": [ 2024-09-14T06:30:33.993 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:33.994 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v2", 2024-09-14T06:30:33.994 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.94:6824", 2024-09-14T06:30:33.994 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 1809564923 2024-09-14T06:30:33.994 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:33.994 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:33.994 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v1", 2024-09-14T06:30:33.994 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.94:6825", 2024-09-14T06:30:33.994 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 1809564923 2024-09-14T06:30:33.994 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:33.994 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:33.994 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:33.994 INFO:teuthology.orchestra.run.smithi094.stdout: "cluster_addrs": { 2024-09-14T06:30:33.994 INFO:teuthology.orchestra.run.smithi094.stdout: "addrvec": [ 2024-09-14T06:30:33.994 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:33.995 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v2", 2024-09-14T06:30:33.995 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.94:6826", 2024-09-14T06:30:33.995 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 1809564923 2024-09-14T06:30:33.995 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:33.995 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:33.995 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v1", 2024-09-14T06:30:33.995 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.94:6827", 2024-09-14T06:30:33.995 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 1809564923 2024-09-14T06:30:33.995 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:33.995 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:33.995 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:33.995 INFO:teuthology.orchestra.run.smithi094.stdout: "heartbeat_back_addrs": { 2024-09-14T06:30:33.995 INFO:teuthology.orchestra.run.smithi094.stdout: "addrvec": [ 2024-09-14T06:30:33.996 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:33.996 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v2", 2024-09-14T06:30:33.996 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.94:6830", 2024-09-14T06:30:33.996 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 1809564923 2024-09-14T06:30:33.996 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:33.996 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:33.996 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v1", 2024-09-14T06:30:33.996 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.94:6831", 2024-09-14T06:30:33.996 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 1809564923 2024-09-14T06:30:33.996 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:33.996 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:33.996 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:33.996 INFO:teuthology.orchestra.run.smithi094.stdout: "heartbeat_front_addrs": { 2024-09-14T06:30:33.996 INFO:teuthology.orchestra.run.smithi094.stdout: "addrvec": [ 2024-09-14T06:30:33.997 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:33.997 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v2", 2024-09-14T06:30:33.997 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.94:6828", 2024-09-14T06:30:33.997 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 1809564923 2024-09-14T06:30:33.997 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:33.997 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:33.997 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v1", 2024-09-14T06:30:33.997 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.94:6829", 2024-09-14T06:30:33.997 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 1809564923 2024-09-14T06:30:33.997 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:33.997 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:33.997 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:33.997 INFO:teuthology.orchestra.run.smithi094.stdout: "public_addr": "172.21.15.94:6825/1809564923", 2024-09-14T06:30:33.997 INFO:teuthology.orchestra.run.smithi094.stdout: "cluster_addr": "172.21.15.94:6827/1809564923", 2024-09-14T06:30:33.998 INFO:teuthology.orchestra.run.smithi094.stdout: "heartbeat_back_addr": "172.21.15.94:6831/1809564923", 2024-09-14T06:30:33.998 INFO:teuthology.orchestra.run.smithi094.stdout: "heartbeat_front_addr": "172.21.15.94:6829/1809564923", 2024-09-14T06:30:33.998 INFO:teuthology.orchestra.run.smithi094.stdout: "state": [ 2024-09-14T06:30:33.998 INFO:teuthology.orchestra.run.smithi094.stdout: "exists", 2024-09-14T06:30:33.998 INFO:teuthology.orchestra.run.smithi094.stdout: "up" 2024-09-14T06:30:33.998 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:33.998 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:33.998 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:33.998 INFO:teuthology.orchestra.run.smithi094.stdout: "osd": 1, 2024-09-14T06:30:33.998 INFO:teuthology.orchestra.run.smithi094.stdout: "uuid": "02251a03-84a0-414d-a980-fac4fd043cb6", 2024-09-14T06:30:33.998 INFO:teuthology.orchestra.run.smithi094.stdout: "up": 1, 2024-09-14T06:30:33.998 INFO:teuthology.orchestra.run.smithi094.stdout: "in": 1, 2024-09-14T06:30:33.998 INFO:teuthology.orchestra.run.smithi094.stdout: "weight": 1, 2024-09-14T06:30:33.999 INFO:teuthology.orchestra.run.smithi094.stdout: "primary_affinity": 1, 2024-09-14T06:30:33.999 INFO:teuthology.orchestra.run.smithi094.stdout: "last_clean_begin": 0, 2024-09-14T06:30:33.999 INFO:teuthology.orchestra.run.smithi094.stdout: "last_clean_end": 0, 2024-09-14T06:30:33.999 INFO:teuthology.orchestra.run.smithi094.stdout: "up_from": 15, 2024-09-14T06:30:33.999 INFO:teuthology.orchestra.run.smithi094.stdout: "up_thru": 23, 2024-09-14T06:30:33.999 INFO:teuthology.orchestra.run.smithi094.stdout: "down_at": 0, 2024-09-14T06:30:33.999 INFO:teuthology.orchestra.run.smithi094.stdout: "lost_at": 0, 2024-09-14T06:30:33.999 INFO:teuthology.orchestra.run.smithi094.stdout: "public_addrs": { 2024-09-14T06:30:33.999 INFO:teuthology.orchestra.run.smithi094.stdout: "addrvec": [ 2024-09-14T06:30:33.999 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:33.999 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v2", 2024-09-14T06:30:33.999 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.94:6816", 2024-09-14T06:30:33.999 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 736402652 2024-09-14T06:30:33.999 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:34.000 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:34.000 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v1", 2024-09-14T06:30:34.000 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.94:6817", 2024-09-14T06:30:34.000 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 736402652 2024-09-14T06:30:34.000 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:34.000 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:34.000 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:34.000 INFO:teuthology.orchestra.run.smithi094.stdout: "cluster_addrs": { 2024-09-14T06:30:34.000 INFO:teuthology.orchestra.run.smithi094.stdout: "addrvec": [ 2024-09-14T06:30:34.000 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:34.000 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v2", 2024-09-14T06:30:34.000 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.94:6818", 2024-09-14T06:30:34.000 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 736402652 2024-09-14T06:30:34.001 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:34.001 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:34.001 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v1", 2024-09-14T06:30:34.001 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.94:6819", 2024-09-14T06:30:34.001 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 736402652 2024-09-14T06:30:34.001 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:34.001 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:34.001 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:34.001 INFO:teuthology.orchestra.run.smithi094.stdout: "heartbeat_back_addrs": { 2024-09-14T06:30:34.001 INFO:teuthology.orchestra.run.smithi094.stdout: "addrvec": [ 2024-09-14T06:30:34.001 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:34.001 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v2", 2024-09-14T06:30:34.001 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.94:6822", 2024-09-14T06:30:34.001 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 736402652 2024-09-14T06:30:34.002 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:34.002 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:34.002 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v1", 2024-09-14T06:30:34.002 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.94:6823", 2024-09-14T06:30:34.002 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 736402652 2024-09-14T06:30:34.002 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:34.002 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:34.002 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:34.002 INFO:teuthology.orchestra.run.smithi094.stdout: "heartbeat_front_addrs": { 2024-09-14T06:30:34.002 INFO:teuthology.orchestra.run.smithi094.stdout: "addrvec": [ 2024-09-14T06:30:34.002 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:34.002 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v2", 2024-09-14T06:30:34.002 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.94:6820", 2024-09-14T06:30:34.003 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 736402652 2024-09-14T06:30:34.003 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:34.003 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:34.003 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v1", 2024-09-14T06:30:34.003 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.94:6821", 2024-09-14T06:30:34.003 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 736402652 2024-09-14T06:30:34.003 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:34.003 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:34.003 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:34.003 INFO:teuthology.orchestra.run.smithi094.stdout: "public_addr": "172.21.15.94:6817/736402652", 2024-09-14T06:30:34.003 INFO:teuthology.orchestra.run.smithi094.stdout: "cluster_addr": "172.21.15.94:6819/736402652", 2024-09-14T06:30:34.003 INFO:teuthology.orchestra.run.smithi094.stdout: "heartbeat_back_addr": "172.21.15.94:6823/736402652", 2024-09-14T06:30:34.003 INFO:teuthology.orchestra.run.smithi094.stdout: "heartbeat_front_addr": "172.21.15.94:6821/736402652", 2024-09-14T06:30:34.003 INFO:teuthology.orchestra.run.smithi094.stdout: "state": [ 2024-09-14T06:30:34.003 INFO:teuthology.orchestra.run.smithi094.stdout: "exists", 2024-09-14T06:30:34.004 INFO:teuthology.orchestra.run.smithi094.stdout: "up" 2024-09-14T06:30:34.004 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:34.004 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:34.004 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:34.004 INFO:teuthology.orchestra.run.smithi094.stdout: "osd": 2, 2024-09-14T06:30:34.004 INFO:teuthology.orchestra.run.smithi094.stdout: "uuid": "d45bf470-bb29-4739-8294-cd122b2b4102", 2024-09-14T06:30:34.004 INFO:teuthology.orchestra.run.smithi094.stdout: "up": 1, 2024-09-14T06:30:34.004 INFO:teuthology.orchestra.run.smithi094.stdout: "in": 1, 2024-09-14T06:30:34.004 INFO:teuthology.orchestra.run.smithi094.stdout: "weight": 1, 2024-09-14T06:30:34.004 INFO:teuthology.orchestra.run.smithi094.stdout: "primary_affinity": 1, 2024-09-14T06:30:34.004 INFO:teuthology.orchestra.run.smithi094.stdout: "last_clean_begin": 0, 2024-09-14T06:30:34.004 INFO:teuthology.orchestra.run.smithi094.stdout: "last_clean_end": 0, 2024-09-14T06:30:34.004 INFO:teuthology.orchestra.run.smithi094.stdout: "up_from": 14, 2024-09-14T06:30:34.005 INFO:teuthology.orchestra.run.smithi094.stdout: "up_thru": 19, 2024-09-14T06:30:34.005 INFO:teuthology.orchestra.run.smithi094.stdout: "down_at": 0, 2024-09-14T06:30:34.005 INFO:teuthology.orchestra.run.smithi094.stdout: "lost_at": 0, 2024-09-14T06:30:34.005 INFO:teuthology.orchestra.run.smithi094.stdout: "public_addrs": { 2024-09-14T06:30:34.005 INFO:teuthology.orchestra.run.smithi094.stdout: "addrvec": [ 2024-09-14T06:30:34.005 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:34.005 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v2", 2024-09-14T06:30:34.005 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.94:6808", 2024-09-14T06:30:34.005 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 1063204782 2024-09-14T06:30:34.005 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:34.005 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:34.005 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v1", 2024-09-14T06:30:34.005 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.94:6809", 2024-09-14T06:30:34.006 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 1063204782 2024-09-14T06:30:34.006 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:34.006 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:34.006 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:34.006 INFO:teuthology.orchestra.run.smithi094.stdout: "cluster_addrs": { 2024-09-14T06:30:34.006 INFO:teuthology.orchestra.run.smithi094.stdout: "addrvec": [ 2024-09-14T06:30:34.006 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:34.006 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v2", 2024-09-14T06:30:34.006 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.94:6810", 2024-09-14T06:30:34.006 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 1063204782 2024-09-14T06:30:34.006 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:34.006 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:34.006 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v1", 2024-09-14T06:30:34.007 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.94:6811", 2024-09-14T06:30:34.007 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 1063204782 2024-09-14T06:30:34.007 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:34.007 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:34.007 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:34.007 INFO:teuthology.orchestra.run.smithi094.stdout: "heartbeat_back_addrs": { 2024-09-14T06:30:34.007 INFO:teuthology.orchestra.run.smithi094.stdout: "addrvec": [ 2024-09-14T06:30:34.007 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:34.007 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v2", 2024-09-14T06:30:34.007 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.94:6814", 2024-09-14T06:30:34.007 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 1063204782 2024-09-14T06:30:34.007 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:34.007 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:34.008 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v1", 2024-09-14T06:30:34.008 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.94:6815", 2024-09-14T06:30:34.008 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 1063204782 2024-09-14T06:30:34.008 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:34.008 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:34.008 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:34.008 INFO:teuthology.orchestra.run.smithi094.stdout: "heartbeat_front_addrs": { 2024-09-14T06:30:34.008 INFO:teuthology.orchestra.run.smithi094.stdout: "addrvec": [ 2024-09-14T06:30:34.008 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:34.008 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v2", 2024-09-14T06:30:34.008 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.94:6812", 2024-09-14T06:30:34.008 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 1063204782 2024-09-14T06:30:34.008 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:34.009 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:34.009 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v1", 2024-09-14T06:30:34.009 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.94:6813", 2024-09-14T06:30:34.009 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 1063204782 2024-09-14T06:30:34.009 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:34.009 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:34.009 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:34.009 INFO:teuthology.orchestra.run.smithi094.stdout: "public_addr": "172.21.15.94:6809/1063204782", 2024-09-14T06:30:34.009 INFO:teuthology.orchestra.run.smithi094.stdout: "cluster_addr": "172.21.15.94:6811/1063204782", 2024-09-14T06:30:34.009 INFO:teuthology.orchestra.run.smithi094.stdout: "heartbeat_back_addr": "172.21.15.94:6815/1063204782", 2024-09-14T06:30:34.009 INFO:teuthology.orchestra.run.smithi094.stdout: "heartbeat_front_addr": "172.21.15.94:6813/1063204782", 2024-09-14T06:30:34.009 INFO:teuthology.orchestra.run.smithi094.stdout: "state": [ 2024-09-14T06:30:34.009 INFO:teuthology.orchestra.run.smithi094.stdout: "exists", 2024-09-14T06:30:34.010 INFO:teuthology.orchestra.run.smithi094.stdout: "up" 2024-09-14T06:30:34.010 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:34.010 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:34.010 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:34.010 INFO:teuthology.orchestra.run.smithi094.stdout: "osd": 3, 2024-09-14T06:30:34.010 INFO:teuthology.orchestra.run.smithi094.stdout: "uuid": "456ec2d9-c438-4243-b931-335704f661eb", 2024-09-14T06:30:34.010 INFO:teuthology.orchestra.run.smithi094.stdout: "up": 1, 2024-09-14T06:30:34.010 INFO:teuthology.orchestra.run.smithi094.stdout: "in": 1, 2024-09-14T06:30:34.010 INFO:teuthology.orchestra.run.smithi094.stdout: "weight": 1, 2024-09-14T06:30:34.010 INFO:teuthology.orchestra.run.smithi094.stdout: "primary_affinity": 1, 2024-09-14T06:30:34.010 INFO:teuthology.orchestra.run.smithi094.stdout: "last_clean_begin": 0, 2024-09-14T06:30:34.010 INFO:teuthology.orchestra.run.smithi094.stdout: "last_clean_end": 0, 2024-09-14T06:30:34.010 INFO:teuthology.orchestra.run.smithi094.stdout: "up_from": 13, 2024-09-14T06:30:34.010 INFO:teuthology.orchestra.run.smithi094.stdout: "up_thru": 74, 2024-09-14T06:30:34.011 INFO:teuthology.orchestra.run.smithi094.stdout: "down_at": 0, 2024-09-14T06:30:34.011 INFO:teuthology.orchestra.run.smithi094.stdout: "lost_at": 0, 2024-09-14T06:30:34.011 INFO:teuthology.orchestra.run.smithi094.stdout: "public_addrs": { 2024-09-14T06:30:34.011 INFO:teuthology.orchestra.run.smithi094.stdout: "addrvec": [ 2024-09-14T06:30:34.011 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:34.011 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v2", 2024-09-14T06:30:34.011 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.94:6800", 2024-09-14T06:30:34.011 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 3382046754 2024-09-14T06:30:34.011 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:34.011 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:34.011 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v1", 2024-09-14T06:30:34.011 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.94:6801", 2024-09-14T06:30:34.011 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 3382046754 2024-09-14T06:30:34.011 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:34.012 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:34.012 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:34.012 INFO:teuthology.orchestra.run.smithi094.stdout: "cluster_addrs": { 2024-09-14T06:30:34.012 INFO:teuthology.orchestra.run.smithi094.stdout: "addrvec": [ 2024-09-14T06:30:34.012 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:34.012 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v2", 2024-09-14T06:30:34.012 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.94:6802", 2024-09-14T06:30:34.012 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 3382046754 2024-09-14T06:30:34.012 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:34.012 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:34.012 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v1", 2024-09-14T06:30:34.012 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.94:6803", 2024-09-14T06:30:34.012 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 3382046754 2024-09-14T06:30:34.012 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:34.013 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:34.013 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:34.013 INFO:teuthology.orchestra.run.smithi094.stdout: "heartbeat_back_addrs": { 2024-09-14T06:30:34.013 INFO:teuthology.orchestra.run.smithi094.stdout: "addrvec": [ 2024-09-14T06:30:34.013 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:34.013 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v2", 2024-09-14T06:30:34.013 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.94:6806", 2024-09-14T06:30:34.013 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 3382046754 2024-09-14T06:30:34.013 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:34.013 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:34.013 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v1", 2024-09-14T06:30:34.013 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.94:6807", 2024-09-14T06:30:34.013 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 3382046754 2024-09-14T06:30:34.014 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:34.014 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:34.014 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:34.014 INFO:teuthology.orchestra.run.smithi094.stdout: "heartbeat_front_addrs": { 2024-09-14T06:30:34.014 INFO:teuthology.orchestra.run.smithi094.stdout: "addrvec": [ 2024-09-14T06:30:34.014 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:34.014 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v2", 2024-09-14T06:30:34.014 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.94:6804", 2024-09-14T06:30:34.014 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 3382046754 2024-09-14T06:30:34.014 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:34.014 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:34.014 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v1", 2024-09-14T06:30:34.014 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.94:6805", 2024-09-14T06:30:34.014 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 3382046754 2024-09-14T06:30:34.015 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:34.015 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:34.015 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:34.015 INFO:teuthology.orchestra.run.smithi094.stdout: "public_addr": "172.21.15.94:6801/3382046754", 2024-09-14T06:30:34.015 INFO:teuthology.orchestra.run.smithi094.stdout: "cluster_addr": "172.21.15.94:6803/3382046754", 2024-09-14T06:30:34.015 INFO:teuthology.orchestra.run.smithi094.stdout: "heartbeat_back_addr": "172.21.15.94:6807/3382046754", 2024-09-14T06:30:34.015 INFO:teuthology.orchestra.run.smithi094.stdout: "heartbeat_front_addr": "172.21.15.94:6805/3382046754", 2024-09-14T06:30:34.015 INFO:teuthology.orchestra.run.smithi094.stdout: "state": [ 2024-09-14T06:30:34.015 INFO:teuthology.orchestra.run.smithi094.stdout: "exists", 2024-09-14T06:30:34.015 INFO:teuthology.orchestra.run.smithi094.stdout: "up" 2024-09-14T06:30:34.015 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:34.015 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:34.015 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:34.015 INFO:teuthology.orchestra.run.smithi094.stdout: "osd": 4, 2024-09-14T06:30:34.016 INFO:teuthology.orchestra.run.smithi094.stdout: "uuid": "2dab948e-c6e4-4437-bf17-017d1ac6821c", 2024-09-14T06:30:34.016 INFO:teuthology.orchestra.run.smithi094.stdout: "up": 1, 2024-09-14T06:30:34.016 INFO:teuthology.orchestra.run.smithi094.stdout: "in": 1, 2024-09-14T06:30:34.016 INFO:teuthology.orchestra.run.smithi094.stdout: "weight": 1, 2024-09-14T06:30:34.016 INFO:teuthology.orchestra.run.smithi094.stdout: "primary_affinity": 1, 2024-09-14T06:30:34.016 INFO:teuthology.orchestra.run.smithi094.stdout: "last_clean_begin": 0, 2024-09-14T06:30:34.016 INFO:teuthology.orchestra.run.smithi094.stdout: "last_clean_end": 0, 2024-09-14T06:30:34.016 INFO:teuthology.orchestra.run.smithi094.stdout: "up_from": 15, 2024-09-14T06:30:34.016 INFO:teuthology.orchestra.run.smithi094.stdout: "up_thru": 23, 2024-09-14T06:30:34.016 INFO:teuthology.orchestra.run.smithi094.stdout: "down_at": 0, 2024-09-14T06:30:34.016 INFO:teuthology.orchestra.run.smithi094.stdout: "lost_at": 0, 2024-09-14T06:30:34.016 INFO:teuthology.orchestra.run.smithi094.stdout: "public_addrs": { 2024-09-14T06:30:34.016 INFO:teuthology.orchestra.run.smithi094.stdout: "addrvec": [ 2024-09-14T06:30:34.016 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:34.017 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v2", 2024-09-14T06:30:34.017 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.138:6810", 2024-09-14T06:30:34.017 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 501726729 2024-09-14T06:30:34.017 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:34.017 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:34.017 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v1", 2024-09-14T06:30:34.017 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.138:6811", 2024-09-14T06:30:34.017 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 501726729 2024-09-14T06:30:34.017 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:34.017 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:34.017 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:34.017 INFO:teuthology.orchestra.run.smithi094.stdout: "cluster_addrs": { 2024-09-14T06:30:34.017 INFO:teuthology.orchestra.run.smithi094.stdout: "addrvec": [ 2024-09-14T06:30:34.018 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:34.018 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v2", 2024-09-14T06:30:34.018 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.138:6812", 2024-09-14T06:30:34.018 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 501726729 2024-09-14T06:30:34.018 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:34.018 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:34.018 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v1", 2024-09-14T06:30:34.018 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.138:6813", 2024-09-14T06:30:34.018 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 501726729 2024-09-14T06:30:34.018 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:34.018 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:34.018 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:34.018 INFO:teuthology.orchestra.run.smithi094.stdout: "heartbeat_back_addrs": { 2024-09-14T06:30:34.019 INFO:teuthology.orchestra.run.smithi094.stdout: "addrvec": [ 2024-09-14T06:30:34.019 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:34.019 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v2", 2024-09-14T06:30:34.019 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.138:6816", 2024-09-14T06:30:34.019 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 501726729 2024-09-14T06:30:34.019 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:34.019 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:34.019 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v1", 2024-09-14T06:30:34.019 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.138:6817", 2024-09-14T06:30:34.019 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 501726729 2024-09-14T06:30:34.019 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:34.019 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:34.019 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:34.020 INFO:teuthology.orchestra.run.smithi094.stdout: "heartbeat_front_addrs": { 2024-09-14T06:30:34.020 INFO:teuthology.orchestra.run.smithi094.stdout: "addrvec": [ 2024-09-14T06:30:34.020 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:34.020 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v2", 2024-09-14T06:30:34.020 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.138:6814", 2024-09-14T06:30:34.020 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 501726729 2024-09-14T06:30:34.020 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:34.020 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:34.020 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v1", 2024-09-14T06:30:34.020 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.138:6815", 2024-09-14T06:30:34.020 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 501726729 2024-09-14T06:30:34.020 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:34.020 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:34.020 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:34.021 INFO:teuthology.orchestra.run.smithi094.stdout: "public_addr": "172.21.15.138:6811/501726729", 2024-09-14T06:30:34.021 INFO:teuthology.orchestra.run.smithi094.stdout: "cluster_addr": "172.21.15.138:6813/501726729", 2024-09-14T06:30:34.021 INFO:teuthology.orchestra.run.smithi094.stdout: "heartbeat_back_addr": "172.21.15.138:6817/501726729", 2024-09-14T06:30:34.021 INFO:teuthology.orchestra.run.smithi094.stdout: "heartbeat_front_addr": "172.21.15.138:6815/501726729", 2024-09-14T06:30:34.021 INFO:teuthology.orchestra.run.smithi094.stdout: "state": [ 2024-09-14T06:30:34.021 INFO:teuthology.orchestra.run.smithi094.stdout: "exists", 2024-09-14T06:30:34.021 INFO:teuthology.orchestra.run.smithi094.stdout: "up" 2024-09-14T06:30:34.021 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:34.021 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:34.021 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:34.021 INFO:teuthology.orchestra.run.smithi094.stdout: "osd": 5, 2024-09-14T06:30:34.021 INFO:teuthology.orchestra.run.smithi094.stdout: "uuid": "8caf3aa3-ad87-4d33-b82e-4c15195176fd", 2024-09-14T06:30:34.021 INFO:teuthology.orchestra.run.smithi094.stdout: "up": 1, 2024-09-14T06:30:34.022 INFO:teuthology.orchestra.run.smithi094.stdout: "in": 1, 2024-09-14T06:30:34.022 INFO:teuthology.orchestra.run.smithi094.stdout: "weight": 1, 2024-09-14T06:30:34.022 INFO:teuthology.orchestra.run.smithi094.stdout: "primary_affinity": 1, 2024-09-14T06:30:34.022 INFO:teuthology.orchestra.run.smithi094.stdout: "last_clean_begin": 0, 2024-09-14T06:30:34.022 INFO:teuthology.orchestra.run.smithi094.stdout: "last_clean_end": 0, 2024-09-14T06:30:34.022 INFO:teuthology.orchestra.run.smithi094.stdout: "up_from": 15, 2024-09-14T06:30:34.022 INFO:teuthology.orchestra.run.smithi094.stdout: "up_thru": 23, 2024-09-14T06:30:34.022 INFO:teuthology.orchestra.run.smithi094.stdout: "down_at": 0, 2024-09-14T06:30:34.022 INFO:teuthology.orchestra.run.smithi094.stdout: "lost_at": 0, 2024-09-14T06:30:34.022 INFO:teuthology.orchestra.run.smithi094.stdout: "public_addrs": { 2024-09-14T06:30:34.022 INFO:teuthology.orchestra.run.smithi094.stdout: "addrvec": [ 2024-09-14T06:30:34.022 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:34.022 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v2", 2024-09-14T06:30:34.022 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.138:6826", 2024-09-14T06:30:34.022 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 1599292735 2024-09-14T06:30:34.023 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:34.023 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:34.023 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v1", 2024-09-14T06:30:34.023 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.138:6827", 2024-09-14T06:30:34.023 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 1599292735 2024-09-14T06:30:34.023 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:34.023 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:34.023 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:34.023 INFO:teuthology.orchestra.run.smithi094.stdout: "cluster_addrs": { 2024-09-14T06:30:34.023 INFO:teuthology.orchestra.run.smithi094.stdout: "addrvec": [ 2024-09-14T06:30:34.023 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:34.023 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v2", 2024-09-14T06:30:34.023 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.138:6828", 2024-09-14T06:30:34.023 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 1599292735 2024-09-14T06:30:34.023 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:34.024 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:34.024 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v1", 2024-09-14T06:30:34.024 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.138:6829", 2024-09-14T06:30:34.024 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 1599292735 2024-09-14T06:30:34.024 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:34.024 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:34.024 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:34.024 INFO:teuthology.orchestra.run.smithi094.stdout: "heartbeat_back_addrs": { 2024-09-14T06:30:34.024 INFO:teuthology.orchestra.run.smithi094.stdout: "addrvec": [ 2024-09-14T06:30:34.024 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:34.024 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v2", 2024-09-14T06:30:34.024 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.138:6832", 2024-09-14T06:30:34.024 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 1599292735 2024-09-14T06:30:34.024 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:34.025 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:34.025 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v1", 2024-09-14T06:30:34.025 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.138:6833", 2024-09-14T06:30:34.025 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 1599292735 2024-09-14T06:30:34.025 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:34.025 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:34.025 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:34.025 INFO:teuthology.orchestra.run.smithi094.stdout: "heartbeat_front_addrs": { 2024-09-14T06:30:34.025 INFO:teuthology.orchestra.run.smithi094.stdout: "addrvec": [ 2024-09-14T06:30:34.025 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:34.025 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v2", 2024-09-14T06:30:34.025 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.138:6830", 2024-09-14T06:30:34.026 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 1599292735 2024-09-14T06:30:34.026 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:34.026 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:34.026 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v1", 2024-09-14T06:30:34.026 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.138:6831", 2024-09-14T06:30:34.026 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 1599292735 2024-09-14T06:30:34.026 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:34.026 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:34.026 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:34.026 INFO:teuthology.orchestra.run.smithi094.stdout: "public_addr": "172.21.15.138:6827/1599292735", 2024-09-14T06:30:34.026 INFO:teuthology.orchestra.run.smithi094.stdout: "cluster_addr": "172.21.15.138:6829/1599292735", 2024-09-14T06:30:34.026 INFO:teuthology.orchestra.run.smithi094.stdout: "heartbeat_back_addr": "172.21.15.138:6833/1599292735", 2024-09-14T06:30:34.026 INFO:teuthology.orchestra.run.smithi094.stdout: "heartbeat_front_addr": "172.21.15.138:6831/1599292735", 2024-09-14T06:30:34.026 INFO:teuthology.orchestra.run.smithi094.stdout: "state": [ 2024-09-14T06:30:34.027 INFO:teuthology.orchestra.run.smithi094.stdout: "exists", 2024-09-14T06:30:34.027 INFO:teuthology.orchestra.run.smithi094.stdout: "up" 2024-09-14T06:30:34.027 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:34.027 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:34.027 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:34.027 INFO:teuthology.orchestra.run.smithi094.stdout: "osd": 6, 2024-09-14T06:30:34.027 INFO:teuthology.orchestra.run.smithi094.stdout: "uuid": "e55d11fd-433d-4462-93a7-d5e84560221b", 2024-09-14T06:30:34.027 INFO:teuthology.orchestra.run.smithi094.stdout: "up": 1, 2024-09-14T06:30:34.027 INFO:teuthology.orchestra.run.smithi094.stdout: "in": 1, 2024-09-14T06:30:34.027 INFO:teuthology.orchestra.run.smithi094.stdout: "weight": 1, 2024-09-14T06:30:34.027 INFO:teuthology.orchestra.run.smithi094.stdout: "primary_affinity": 1, 2024-09-14T06:30:34.027 INFO:teuthology.orchestra.run.smithi094.stdout: "last_clean_begin": 0, 2024-09-14T06:30:34.027 INFO:teuthology.orchestra.run.smithi094.stdout: "last_clean_end": 0, 2024-09-14T06:30:34.027 INFO:teuthology.orchestra.run.smithi094.stdout: "up_from": 15, 2024-09-14T06:30:34.028 INFO:teuthology.orchestra.run.smithi094.stdout: "up_thru": 23, 2024-09-14T06:30:34.028 INFO:teuthology.orchestra.run.smithi094.stdout: "down_at": 0, 2024-09-14T06:30:34.028 INFO:teuthology.orchestra.run.smithi094.stdout: "lost_at": 0, 2024-09-14T06:30:34.028 INFO:teuthology.orchestra.run.smithi094.stdout: "public_addrs": { 2024-09-14T06:30:34.028 INFO:teuthology.orchestra.run.smithi094.stdout: "addrvec": [ 2024-09-14T06:30:34.028 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:34.028 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v2", 2024-09-14T06:30:34.028 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.138:6818", 2024-09-14T06:30:34.028 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 163531479 2024-09-14T06:30:34.028 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:34.028 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:34.028 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v1", 2024-09-14T06:30:34.028 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.138:6819", 2024-09-14T06:30:34.028 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 163531479 2024-09-14T06:30:34.029 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:34.029 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:34.029 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:34.029 INFO:teuthology.orchestra.run.smithi094.stdout: "cluster_addrs": { 2024-09-14T06:30:34.029 INFO:teuthology.orchestra.run.smithi094.stdout: "addrvec": [ 2024-09-14T06:30:34.029 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:34.029 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v2", 2024-09-14T06:30:34.029 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.138:6820", 2024-09-14T06:30:34.029 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 163531479 2024-09-14T06:30:34.029 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:34.029 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:34.029 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v1", 2024-09-14T06:30:34.029 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.138:6821", 2024-09-14T06:30:34.029 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 163531479 2024-09-14T06:30:34.030 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:34.030 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:34.030 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:34.030 INFO:teuthology.orchestra.run.smithi094.stdout: "heartbeat_back_addrs": { 2024-09-14T06:30:34.030 INFO:teuthology.orchestra.run.smithi094.stdout: "addrvec": [ 2024-09-14T06:30:34.030 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:34.030 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v2", 2024-09-14T06:30:34.030 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.138:6824", 2024-09-14T06:30:34.030 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 163531479 2024-09-14T06:30:34.030 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:34.030 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:34.030 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v1", 2024-09-14T06:30:34.030 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.138:6825", 2024-09-14T06:30:34.030 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 163531479 2024-09-14T06:30:34.030 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:34.031 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:34.031 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:34.031 INFO:teuthology.orchestra.run.smithi094.stdout: "heartbeat_front_addrs": { 2024-09-14T06:30:34.031 INFO:teuthology.orchestra.run.smithi094.stdout: "addrvec": [ 2024-09-14T06:30:34.031 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:34.031 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v2", 2024-09-14T06:30:34.031 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.138:6822", 2024-09-14T06:30:34.031 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 163531479 2024-09-14T06:30:34.031 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:34.031 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:34.031 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v1", 2024-09-14T06:30:34.031 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.138:6823", 2024-09-14T06:30:34.031 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 163531479 2024-09-14T06:30:34.031 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:34.032 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:34.032 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:34.032 INFO:teuthology.orchestra.run.smithi094.stdout: "public_addr": "172.21.15.138:6819/163531479", 2024-09-14T06:30:34.032 INFO:teuthology.orchestra.run.smithi094.stdout: "cluster_addr": "172.21.15.138:6821/163531479", 2024-09-14T06:30:34.032 INFO:teuthology.orchestra.run.smithi094.stdout: "heartbeat_back_addr": "172.21.15.138:6825/163531479", 2024-09-14T06:30:34.032 INFO:teuthology.orchestra.run.smithi094.stdout: "heartbeat_front_addr": "172.21.15.138:6823/163531479", 2024-09-14T06:30:34.032 INFO:teuthology.orchestra.run.smithi094.stdout: "state": [ 2024-09-14T06:30:34.032 INFO:teuthology.orchestra.run.smithi094.stdout: "exists", 2024-09-14T06:30:34.032 INFO:teuthology.orchestra.run.smithi094.stdout: "up" 2024-09-14T06:30:34.032 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:34.032 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:34.032 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:34.032 INFO:teuthology.orchestra.run.smithi094.stdout: "osd": 7, 2024-09-14T06:30:34.032 INFO:teuthology.orchestra.run.smithi094.stdout: "uuid": "ba14ba69-2c6c-4d09-81e2-2593dc3a7ea9", 2024-09-14T06:30:34.033 INFO:teuthology.orchestra.run.smithi094.stdout: "up": 1, 2024-09-14T06:30:34.033 INFO:teuthology.orchestra.run.smithi094.stdout: "in": 1, 2024-09-14T06:30:34.033 INFO:teuthology.orchestra.run.smithi094.stdout: "weight": 1, 2024-09-14T06:30:34.033 INFO:teuthology.orchestra.run.smithi094.stdout: "primary_affinity": 1, 2024-09-14T06:30:34.033 INFO:teuthology.orchestra.run.smithi094.stdout: "last_clean_begin": 0, 2024-09-14T06:30:34.033 INFO:teuthology.orchestra.run.smithi094.stdout: "last_clean_end": 0, 2024-09-14T06:30:34.033 INFO:teuthology.orchestra.run.smithi094.stdout: "up_from": 15, 2024-09-14T06:30:34.033 INFO:teuthology.orchestra.run.smithi094.stdout: "up_thru": 23, 2024-09-14T06:30:34.033 INFO:teuthology.orchestra.run.smithi094.stdout: "down_at": 0, 2024-09-14T06:30:34.033 INFO:teuthology.orchestra.run.smithi094.stdout: "lost_at": 0, 2024-09-14T06:30:34.033 INFO:teuthology.orchestra.run.smithi094.stdout: "public_addrs": { 2024-09-14T06:30:34.033 INFO:teuthology.orchestra.run.smithi094.stdout: "addrvec": [ 2024-09-14T06:30:34.033 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:34.034 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v2", 2024-09-14T06:30:34.034 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.138:6800", 2024-09-14T06:30:34.034 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 218507235 2024-09-14T06:30:34.034 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:34.034 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:34.034 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v1", 2024-09-14T06:30:34.034 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.138:6801", 2024-09-14T06:30:34.034 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 218507235 2024-09-14T06:30:34.034 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:34.034 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:34.034 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:34.034 INFO:teuthology.orchestra.run.smithi094.stdout: "cluster_addrs": { 2024-09-14T06:30:34.034 INFO:teuthology.orchestra.run.smithi094.stdout: "addrvec": [ 2024-09-14T06:30:34.035 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:34.035 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v2", 2024-09-14T06:30:34.035 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.138:6802", 2024-09-14T06:30:34.035 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 218507235 2024-09-14T06:30:34.035 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:34.035 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:34.035 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v1", 2024-09-14T06:30:34.035 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.138:6803", 2024-09-14T06:30:34.035 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 218507235 2024-09-14T06:30:34.035 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:34.035 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:34.035 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:34.035 INFO:teuthology.orchestra.run.smithi094.stdout: "heartbeat_back_addrs": { 2024-09-14T06:30:34.036 INFO:teuthology.orchestra.run.smithi094.stdout: "addrvec": [ 2024-09-14T06:30:34.036 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:34.036 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v2", 2024-09-14T06:30:34.036 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.138:6806", 2024-09-14T06:30:34.036 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 218507235 2024-09-14T06:30:34.036 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:34.036 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:34.036 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v1", 2024-09-14T06:30:34.036 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.138:6807", 2024-09-14T06:30:34.036 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 218507235 2024-09-14T06:30:34.036 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:34.036 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:34.036 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:34.037 INFO:teuthology.orchestra.run.smithi094.stdout: "heartbeat_front_addrs": { 2024-09-14T06:30:34.037 INFO:teuthology.orchestra.run.smithi094.stdout: "addrvec": [ 2024-09-14T06:30:34.037 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:34.037 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v2", 2024-09-14T06:30:34.037 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.138:6804", 2024-09-14T06:30:34.037 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 218507235 2024-09-14T06:30:34.037 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:34.037 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:34.037 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v1", 2024-09-14T06:30:34.037 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.138:6805", 2024-09-14T06:30:34.037 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 218507235 2024-09-14T06:30:34.037 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:34.037 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:34.037 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:34.038 INFO:teuthology.orchestra.run.smithi094.stdout: "public_addr": "172.21.15.138:6801/218507235", 2024-09-14T06:30:34.038 INFO:teuthology.orchestra.run.smithi094.stdout: "cluster_addr": "172.21.15.138:6803/218507235", 2024-09-14T06:30:34.038 INFO:teuthology.orchestra.run.smithi094.stdout: "heartbeat_back_addr": "172.21.15.138:6807/218507235", 2024-09-14T06:30:34.038 INFO:teuthology.orchestra.run.smithi094.stdout: "heartbeat_front_addr": "172.21.15.138:6805/218507235", 2024-09-14T06:30:34.038 INFO:teuthology.orchestra.run.smithi094.stdout: "state": [ 2024-09-14T06:30:34.038 INFO:teuthology.orchestra.run.smithi094.stdout: "exists", 2024-09-14T06:30:34.038 INFO:teuthology.orchestra.run.smithi094.stdout: "up" 2024-09-14T06:30:34.038 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:34.038 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:34.038 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-09-14T06:30:34.038 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_xinfo": [ 2024-09-14T06:30:34.038 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:34.038 INFO:teuthology.orchestra.run.smithi094.stdout: "osd": 0, 2024-09-14T06:30:34.038 INFO:teuthology.orchestra.run.smithi094.stdout: "down_stamp": "0.000000", 2024-09-14T06:30:34.039 INFO:teuthology.orchestra.run.smithi094.stdout: "laggy_probability": 0, 2024-09-14T06:30:34.039 INFO:teuthology.orchestra.run.smithi094.stdout: "laggy_interval": 0, 2024-09-14T06:30:34.039 INFO:teuthology.orchestra.run.smithi094.stdout: "features": 4540719139924082687, 2024-09-14T06:30:34.039 INFO:teuthology.orchestra.run.smithi094.stdout: "old_weight": 0, 2024-09-14T06:30:34.039 INFO:teuthology.orchestra.run.smithi094.stdout: "last_purged_snaps_scrub": "2024-09-14T06:28:39.906639+0000", 2024-09-14T06:30:34.039 INFO:teuthology.orchestra.run.smithi094.stdout: "dead_epoch": 0 2024-09-14T06:30:34.039 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:34.039 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:34.039 INFO:teuthology.orchestra.run.smithi094.stdout: "osd": 1, 2024-09-14T06:30:34.039 INFO:teuthology.orchestra.run.smithi094.stdout: "down_stamp": "0.000000", 2024-09-14T06:30:34.039 INFO:teuthology.orchestra.run.smithi094.stdout: "laggy_probability": 0, 2024-09-14T06:30:34.039 INFO:teuthology.orchestra.run.smithi094.stdout: "laggy_interval": 0, 2024-09-14T06:30:34.039 INFO:teuthology.orchestra.run.smithi094.stdout: "features": 4540719139924082687, 2024-09-14T06:30:34.039 INFO:teuthology.orchestra.run.smithi094.stdout: "old_weight": 0, 2024-09-14T06:30:34.040 INFO:teuthology.orchestra.run.smithi094.stdout: "last_purged_snaps_scrub": "2024-09-14T06:28:40.541279+0000", 2024-09-14T06:30:34.040 INFO:teuthology.orchestra.run.smithi094.stdout: "dead_epoch": 0 2024-09-14T06:30:34.040 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:34.040 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:34.040 INFO:teuthology.orchestra.run.smithi094.stdout: "osd": 2, 2024-09-14T06:30:34.040 INFO:teuthology.orchestra.run.smithi094.stdout: "down_stamp": "0.000000", 2024-09-14T06:30:34.040 INFO:teuthology.orchestra.run.smithi094.stdout: "laggy_probability": 0, 2024-09-14T06:30:34.040 INFO:teuthology.orchestra.run.smithi094.stdout: "laggy_interval": 0, 2024-09-14T06:30:34.040 INFO:teuthology.orchestra.run.smithi094.stdout: "features": 4540719139924082687, 2024-09-14T06:30:34.040 INFO:teuthology.orchestra.run.smithi094.stdout: "old_weight": 0, 2024-09-14T06:30:34.040 INFO:teuthology.orchestra.run.smithi094.stdout: "last_purged_snaps_scrub": "2024-09-14T06:28:40.025350+0000", 2024-09-14T06:30:34.040 INFO:teuthology.orchestra.run.smithi094.stdout: "dead_epoch": 0 2024-09-14T06:30:34.040 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:34.040 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:34.041 INFO:teuthology.orchestra.run.smithi094.stdout: "osd": 3, 2024-09-14T06:30:34.041 INFO:teuthology.orchestra.run.smithi094.stdout: "down_stamp": "0.000000", 2024-09-14T06:30:34.041 INFO:teuthology.orchestra.run.smithi094.stdout: "laggy_probability": 0, 2024-09-14T06:30:34.041 INFO:teuthology.orchestra.run.smithi094.stdout: "laggy_interval": 0, 2024-09-14T06:30:34.041 INFO:teuthology.orchestra.run.smithi094.stdout: "features": 4540719139924082687, 2024-09-14T06:30:34.041 INFO:teuthology.orchestra.run.smithi094.stdout: "old_weight": 0, 2024-09-14T06:30:34.041 INFO:teuthology.orchestra.run.smithi094.stdout: "last_purged_snaps_scrub": "2024-09-14T06:28:39.217944+0000", 2024-09-14T06:30:34.041 INFO:teuthology.orchestra.run.smithi094.stdout: "dead_epoch": 0 2024-09-14T06:30:34.041 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:34.041 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:34.041 INFO:teuthology.orchestra.run.smithi094.stdout: "osd": 4, 2024-09-14T06:30:34.041 INFO:teuthology.orchestra.run.smithi094.stdout: "down_stamp": "0.000000", 2024-09-14T06:30:34.041 INFO:teuthology.orchestra.run.smithi094.stdout: "laggy_probability": 0, 2024-09-14T06:30:34.041 INFO:teuthology.orchestra.run.smithi094.stdout: "laggy_interval": 0, 2024-09-14T06:30:34.041 INFO:teuthology.orchestra.run.smithi094.stdout: "features": 4540719139924082687, 2024-09-14T06:30:34.042 INFO:teuthology.orchestra.run.smithi094.stdout: "old_weight": 0, 2024-09-14T06:30:34.042 INFO:teuthology.orchestra.run.smithi094.stdout: "last_purged_snaps_scrub": "2024-09-14T06:28:40.185898+0000", 2024-09-14T06:30:34.042 INFO:teuthology.orchestra.run.smithi094.stdout: "dead_epoch": 0 2024-09-14T06:30:34.042 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:34.042 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:34.042 INFO:teuthology.orchestra.run.smithi094.stdout: "osd": 5, 2024-09-14T06:30:34.042 INFO:teuthology.orchestra.run.smithi094.stdout: "down_stamp": "0.000000", 2024-09-14T06:30:34.042 INFO:teuthology.orchestra.run.smithi094.stdout: "laggy_probability": 0, 2024-09-14T06:30:34.042 INFO:teuthology.orchestra.run.smithi094.stdout: "laggy_interval": 0, 2024-09-14T06:30:34.042 INFO:teuthology.orchestra.run.smithi094.stdout: "features": 4540719139924082687, 2024-09-14T06:30:34.042 INFO:teuthology.orchestra.run.smithi094.stdout: "old_weight": 0, 2024-09-14T06:30:34.042 INFO:teuthology.orchestra.run.smithi094.stdout: "last_purged_snaps_scrub": "2024-09-14T06:28:40.783504+0000", 2024-09-14T06:30:34.042 INFO:teuthology.orchestra.run.smithi094.stdout: "dead_epoch": 0 2024-09-14T06:30:34.043 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:34.043 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:34.043 INFO:teuthology.orchestra.run.smithi094.stdout: "osd": 6, 2024-09-14T06:30:34.043 INFO:teuthology.orchestra.run.smithi094.stdout: "down_stamp": "0.000000", 2024-09-14T06:30:34.043 INFO:teuthology.orchestra.run.smithi094.stdout: "laggy_probability": 0, 2024-09-14T06:30:34.043 INFO:teuthology.orchestra.run.smithi094.stdout: "laggy_interval": 0, 2024-09-14T06:30:34.043 INFO:teuthology.orchestra.run.smithi094.stdout: "features": 4540719139924082687, 2024-09-14T06:30:34.043 INFO:teuthology.orchestra.run.smithi094.stdout: "old_weight": 0, 2024-09-14T06:30:34.043 INFO:teuthology.orchestra.run.smithi094.stdout: "last_purged_snaps_scrub": "2024-09-14T06:28:40.527305+0000", 2024-09-14T06:30:34.043 INFO:teuthology.orchestra.run.smithi094.stdout: "dead_epoch": 0 2024-09-14T06:30:34.043 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:34.043 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:34.043 INFO:teuthology.orchestra.run.smithi094.stdout: "osd": 7, 2024-09-14T06:30:34.043 INFO:teuthology.orchestra.run.smithi094.stdout: "down_stamp": "0.000000", 2024-09-14T06:30:34.044 INFO:teuthology.orchestra.run.smithi094.stdout: "laggy_probability": 0, 2024-09-14T06:30:34.044 INFO:teuthology.orchestra.run.smithi094.stdout: "laggy_interval": 0, 2024-09-14T06:30:34.044 INFO:teuthology.orchestra.run.smithi094.stdout: "features": 4540719139924082687, 2024-09-14T06:30:34.044 INFO:teuthology.orchestra.run.smithi094.stdout: "old_weight": 0, 2024-09-14T06:30:34.044 INFO:teuthology.orchestra.run.smithi094.stdout: "last_purged_snaps_scrub": "2024-09-14T06:28:40.349676+0000", 2024-09-14T06:30:34.044 INFO:teuthology.orchestra.run.smithi094.stdout: "dead_epoch": 0 2024-09-14T06:30:34.044 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:34.044 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-09-14T06:30:34.044 INFO:teuthology.orchestra.run.smithi094.stdout: "pg_upmap": [], 2024-09-14T06:30:34.044 INFO:teuthology.orchestra.run.smithi094.stdout: "pg_upmap_items": [ 2024-09-14T06:30:34.044 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:34.044 INFO:teuthology.orchestra.run.smithi094.stdout: "pgid": "3.7", 2024-09-14T06:30:34.044 INFO:teuthology.orchestra.run.smithi094.stdout: "mappings": [ 2024-09-14T06:30:34.044 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:34.044 INFO:teuthology.orchestra.run.smithi094.stdout: "from": 3, 2024-09-14T06:30:34.045 INFO:teuthology.orchestra.run.smithi094.stdout: "to": 0 2024-09-14T06:30:34.045 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:34.045 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:34.045 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:34.045 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-09-14T06:30:34.045 INFO:teuthology.orchestra.run.smithi094.stdout: "pg_upmap_primaries": [], 2024-09-14T06:30:34.045 INFO:teuthology.orchestra.run.smithi094.stdout: "pg_temp": [], 2024-09-14T06:30:34.045 INFO:teuthology.orchestra.run.smithi094.stdout: "primary_temp": [], 2024-09-14T06:30:34.045 INFO:teuthology.orchestra.run.smithi094.stdout: "blocklist": {}, 2024-09-14T06:30:34.045 INFO:teuthology.orchestra.run.smithi094.stdout: "range_blocklist": {}, 2024-09-14T06:30:34.045 INFO:teuthology.orchestra.run.smithi094.stdout: "erasure_code_profiles": { 2024-09-14T06:30:34.045 INFO:teuthology.orchestra.run.smithi094.stdout: "default": { 2024-09-14T06:30:34.045 INFO:teuthology.orchestra.run.smithi094.stdout: "crush-failure-domain": "osd", 2024-09-14T06:30:34.045 INFO:teuthology.orchestra.run.smithi094.stdout: "k": "2", 2024-09-14T06:30:34.045 INFO:teuthology.orchestra.run.smithi094.stdout: "m": "1", 2024-09-14T06:30:34.046 INFO:teuthology.orchestra.run.smithi094.stdout: "plugin": "jerasure", 2024-09-14T06:30:34.046 INFO:teuthology.orchestra.run.smithi094.stdout: "technique": "reed_sol_van" 2024-09-14T06:30:34.046 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:34.046 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:34.046 INFO:teuthology.orchestra.run.smithi094.stdout: "removed_snaps_queue": [ 2024-09-14T06:30:34.046 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:34.046 INFO:teuthology.orchestra.run.smithi094.stdout: "pool": 3, 2024-09-14T06:30:34.046 INFO:teuthology.orchestra.run.smithi094.stdout: "snaps": [ 2024-09-14T06:30:34.046 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:34.046 INFO:teuthology.orchestra.run.smithi094.stdout: "begin": 46, 2024-09-14T06:30:34.046 INFO:teuthology.orchestra.run.smithi094.stdout: "length": 1 2024-09-14T06:30:34.046 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:34.046 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:34.046 INFO:teuthology.orchestra.run.smithi094.stdout: "begin": 48, 2024-09-14T06:30:34.046 INFO:teuthology.orchestra.run.smithi094.stdout: "length": 1 2024-09-14T06:30:34.047 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:34.047 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:34.047 INFO:teuthology.orchestra.run.smithi094.stdout: "begin": 50, 2024-09-14T06:30:34.047 INFO:teuthology.orchestra.run.smithi094.stdout: "length": 1 2024-09-14T06:30:34.047 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:34.047 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:34.047 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:34.047 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-09-14T06:30:34.047 INFO:teuthology.orchestra.run.smithi094.stdout: "new_removed_snaps": [ 2024-09-14T06:30:34.047 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:34.047 INFO:teuthology.orchestra.run.smithi094.stdout: "pool": 3, 2024-09-14T06:30:34.047 INFO:teuthology.orchestra.run.smithi094.stdout: "snaps": [ 2024-09-14T06:30:34.047 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:34.047 INFO:teuthology.orchestra.run.smithi094.stdout: "begin": 50, 2024-09-14T06:30:34.047 INFO:teuthology.orchestra.run.smithi094.stdout: "length": 1 2024-09-14T06:30:34.047 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:34.048 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:34.048 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:34.048 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-09-14T06:30:34.048 INFO:teuthology.orchestra.run.smithi094.stdout: "new_purged_snaps": [], 2024-09-14T06:30:34.048 INFO:teuthology.orchestra.run.smithi094.stdout: "crush_node_flags": {}, 2024-09-14T06:30:34.048 INFO:teuthology.orchestra.run.smithi094.stdout: "device_class_flags": {}, 2024-09-14T06:30:34.048 INFO:teuthology.orchestra.run.smithi094.stdout: "stretch_mode": { 2024-09-14T06:30:34.048 INFO:teuthology.orchestra.run.smithi094.stdout: "stretch_mode_enabled": false, 2024-09-14T06:30:34.048 INFO:teuthology.orchestra.run.smithi094.stdout: "stretch_bucket_count": 0, 2024-09-14T06:30:34.048 INFO:teuthology.orchestra.run.smithi094.stdout: "degraded_stretch_mode": 0, 2024-09-14T06:30:34.048 INFO:teuthology.orchestra.run.smithi094.stdout: "recovering_stretch_mode": 0, 2024-09-14T06:30:34.048 INFO:teuthology.orchestra.run.smithi094.stdout: "stretch_mode_bucket": 0 2024-09-14T06:30:34.048 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:34.048 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-14T06:30:34.050 INFO:teuthology.orchestra.run.smithi138.stdout:ERROR: (22) Invalid argument 2024-09-14T06:30:34.050 INFO:teuthology.orchestra.run.smithi138.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:30:34.051 INFO:tasks.thrashosds.thrasher:j is {'epoch': 111, 'fsid': '200ad3da-bd63-4e04-af74-efb5ca0af75e', 'created': '2024-09-14T06:28:30.782046+0000', 'modified': '2024-09-14T06:30:33.646401+0000', 'last_up_change': '2024-09-14T06:28:42.827239+0000', 'last_in_change': '2024-09-14T06:28:33.724176+0000', 'flags': 'sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit', 'flags_num': 5799936, 'flags_set': ['pglog_hardlimit', 'purged_snapdirs', 'recovery_deletes', 'sortbitwise'], 'crush_version': 5, 'full_ratio': 0.949999988079071, 'backfillfull_ratio': 0.8999999761581421, 'nearfull_ratio': 0.8500000238418579, 'cluster_snapshot': '', 'pool_max': 3, 'max_osd': 8, 'require_min_compat_client': 'luminous', 'min_compat_client': 'luminous', 'require_osd_release': 'squid', 'allow_crimson': False, 'pools': [{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-14T06:28:43.295990+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2024-09-14T06:28:49.451714+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 2, 'snap_epoch': 22, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.9900000095367432, 'score_stable': 1.9900000095367432, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'pool_client.0', 'create_time': '2024-09-14T06:28:59.688535+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '111', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 50, 'snap_epoch': 111, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}], 'osds': [{'osd': 0, 'uuid': '8b22a249-b842-406a-a02c-c0f31f57b635', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 75, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.94:6824', 'nonce': 1809564923}, {'type': 'v1', 'addr': '172.21.15.94:6825', 'nonce': 1809564923}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.94:6826', 'nonce': 1809564923}, {'type': 'v1', 'addr': '172.21.15.94:6827', 'nonce': 1809564923}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.94:6830', 'nonce': 1809564923}, {'type': 'v1', 'addr': '172.21.15.94:6831', 'nonce': 1809564923}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.94:6828', 'nonce': 1809564923}, {'type': 'v1', 'addr': '172.21.15.94:6829', 'nonce': 1809564923}]}, 'public_addr': '172.21.15.94:6825/1809564923', 'cluster_addr': '172.21.15.94:6827/1809564923', 'heartbeat_back_addr': '172.21.15.94:6831/1809564923', 'heartbeat_front_addr': '172.21.15.94:6829/1809564923', 'state': ['exists', 'up']}, {'osd': 1, 'uuid': '02251a03-84a0-414d-a980-fac4fd043cb6', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 23, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.94:6816', 'nonce': 736402652}, {'type': 'v1', 'addr': '172.21.15.94:6817', 'nonce': 736402652}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.94:6818', 'nonce': 736402652}, {'type': 'v1', 'addr': '172.21.15.94:6819', 'nonce': 736402652}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.94:6822', 'nonce': 736402652}, {'type': 'v1', 'addr': '172.21.15.94:6823', 'nonce': 736402652}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.94:6820', 'nonce': 736402652}, {'type': 'v1', 'addr': '172.21.15.94:6821', 'nonce': 736402652}]}, 'public_addr': '172.21.15.94:6817/736402652', 'cluster_addr': '172.21.15.94:6819/736402652', 'heartbeat_back_addr': '172.21.15.94:6823/736402652', 'heartbeat_front_addr': '172.21.15.94:6821/736402652', 'state': ['exists', 'up']}, {'osd': 2, 'uuid': 'd45bf470-bb29-4739-8294-cd122b2b4102', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 19, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.94:6808', 'nonce': 1063204782}, {'type': 'v1', 'addr': '172.21.15.94:6809', 'nonce': 1063204782}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.94:6810', 'nonce': 1063204782}, {'type': 'v1', 'addr': '172.21.15.94:6811', 'nonce': 1063204782}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.94:6814', 'nonce': 1063204782}, {'type': 'v1', 'addr': '172.21.15.94:6815', 'nonce': 1063204782}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.94:6812', 'nonce': 1063204782}, {'type': 'v1', 'addr': '172.21.15.94:6813', 'nonce': 1063204782}]}, 'public_addr': '172.21.15.94:6809/1063204782', 'cluster_addr': '172.21.15.94:6811/1063204782', 'heartbeat_back_addr': '172.21.15.94:6815/1063204782', 'heartbeat_front_addr': '172.21.15.94:6813/1063204782', 'state': ['exists', 'up']}, {'osd': 3, 'uuid': '456ec2d9-c438-4243-b931-335704f661eb', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 74, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.94:6800', 'nonce': 3382046754}, {'type': 'v1', 'addr': '172.21.15.94:6801', 'nonce': 3382046754}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.94:6802', 'nonce': 3382046754}, {'type': 'v1', 'addr': '172.21.15.94:6803', 'nonce': 3382046754}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.94:6806', 'nonce': 3382046754}, {'type': 'v1', 'addr': '172.21.15.94:6807', 'nonce': 3382046754}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.94:6804', 'nonce': 3382046754}, {'type': 'v1', 'addr': '172.21.15.94:6805', 'nonce': 3382046754}]}, 'public_addr': '172.21.15.94:6801/3382046754', 'cluster_addr': '172.21.15.94:6803/3382046754', 'heartbeat_back_addr': '172.21.15.94:6807/3382046754', 'heartbeat_front_addr': '172.21.15.94:6805/3382046754', 'state': ['exists', 'up']}, {'osd': 4, 'uuid': '2dab948e-c6e4-4437-bf17-017d1ac6821c', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 23, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.138:6810', 'nonce': 501726729}, {'type': 'v1', 'addr': '172.21.15.138:6811', 'nonce': 501726729}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.138:6812', 'nonce': 501726729}, {'type': 'v1', 'addr': '172.21.15.138:6813', 'nonce': 501726729}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.138:6816', 'nonce': 501726729}, {'type': 'v1', 'addr': '172.21.15.138:6817', 'nonce': 501726729}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.138:6814', 'nonce': 501726729}, {'type': 'v1', 'addr': '172.21.15.138:6815', 'nonce': 501726729}]}, 'public_addr': '172.21.15.138:6811/501726729', 'cluster_addr': '172.21.15.138:6813/501726729', 'heartbeat_back_addr': '172.21.15.138:6817/501726729', 'heartbeat_front_addr': '172.21.15.138:6815/501726729', 'state': ['exists', 'up']}, {'osd': 5, 'uuid': '8caf3aa3-ad87-4d33-b82e-4c15195176fd', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 23, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.138:6826', 'nonce': 1599292735}, {'type': 'v1', 'addr': '172.21.15.138:6827', 'nonce': 1599292735}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.138:6828', 'nonce': 1599292735}, {'type': 'v1', 'addr': '172.21.15.138:6829', 'nonce': 1599292735}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.138:6832', 'nonce': 1599292735}, {'type': 'v1', 'addr': '172.21.15.138:6833', 'nonce': 1599292735}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.138:6830', 'nonce': 1599292735}, {'type': 'v1', 'addr': '172.21.15.138:6831', 'nonce': 1599292735}]}, 'public_addr': '172.21.15.138:6827/1599292735', 'cluster_addr': '172.21.15.138:6829/1599292735', 'heartbeat_back_addr': '172.21.15.138:6833/1599292735', 'heartbeat_front_addr': '172.21.15.138:6831/1599292735', 'state': ['exists', 'up']}, {'osd': 6, 'uuid': 'e55d11fd-433d-4462-93a7-d5e84560221b', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 23, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.138:6818', 'nonce': 163531479}, {'type': 'v1', 'addr': '172.21.15.138:6819', 'nonce': 163531479}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.138:6820', 'nonce': 163531479}, {'type': 'v1', 'addr': '172.21.15.138:6821', 'nonce': 163531479}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.138:6824', 'nonce': 163531479}, {'type': 'v1', 'addr': '172.21.15.138:6825', 'nonce': 163531479}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.138:6822', 'nonce': 163531479}, {'type': 'v1', 'addr': '172.21.15.138:6823', 'nonce': 163531479}]}, 'public_addr': '172.21.15.138:6819/163531479', 'cluster_addr': '172.21.15.138:6821/163531479', 'heartbeat_back_addr': '172.21.15.138:6825/163531479', 'heartbeat_front_addr': '172.21.15.138:6823/163531479', 'state': ['exists', 'up']}, {'osd': 7, 'uuid': 'ba14ba69-2c6c-4d09-81e2-2593dc3a7ea9', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 23, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.138:6800', 'nonce': 218507235}, {'type': 'v1', 'addr': '172.21.15.138:6801', 'nonce': 218507235}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.138:6802', 'nonce': 218507235}, {'type': 'v1', 'addr': '172.21.15.138:6803', 'nonce': 218507235}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.138:6806', 'nonce': 218507235}, {'type': 'v1', 'addr': '172.21.15.138:6807', 'nonce': 218507235}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.138:6804', 'nonce': 218507235}, {'type': 'v1', 'addr': '172.21.15.138:6805', 'nonce': 218507235}]}, 'public_addr': '172.21.15.138:6801/218507235', 'cluster_addr': '172.21.15.138:6803/218507235', 'heartbeat_back_addr': '172.21.15.138:6807/218507235', 'heartbeat_front_addr': '172.21.15.138:6805/218507235', 'state': ['exists', 'up']}], 'osd_xinfo': [{'osd': 0, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-14T06:28:39.906639+0000', 'dead_epoch': 0}, {'osd': 1, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-14T06:28:40.541279+0000', 'dead_epoch': 0}, {'osd': 2, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-14T06:28:40.025350+0000', 'dead_epoch': 0}, {'osd': 3, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-14T06:28:39.217944+0000', 'dead_epoch': 0}, {'osd': 4, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-14T06:28:40.185898+0000', 'dead_epoch': 0}, {'osd': 5, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-14T06:28:40.783504+0000', 'dead_epoch': 0}, {'osd': 6, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-14T06:28:40.527305+0000', 'dead_epoch': 0}, {'osd': 7, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-14T06:28:40.349676+0000', 'dead_epoch': 0}], 'pg_upmap': [], 'pg_upmap_items': [{'pgid': '3.7', 'mappings': [{'from': 3, 'to': 0}]}], 'pg_upmap_primaries': [], 'pg_temp': [], 'primary_temp': [], 'blocklist': {}, 'range_blocklist': {}, 'erasure_code_profiles': {'default': {'crush-failure-domain': 'osd', 'k': '2', 'm': '1', 'plugin': 'jerasure', 'technique': 'reed_sol_van'}}, 'removed_snaps_queue': [{'pool': 3, 'snaps': [{'begin': 46, 'length': 1}, {'begin': 48, 'length': 1}, {'begin': 50, 'length': 1}]}], 'new_removed_snaps': [{'pool': 3, 'snaps': [{'begin': 50, 'length': 1}]}], 'new_purged_snaps': [], 'crush_node_flags': {}, 'device_class_flags': {}, 'stretch_mode': {'stretch_mode_enabled': False, 'stretch_bucket_count': 0, 'degraded_stretch_mode': 0, 'recovering_stretch_mode': 0, 'stretch_mode_bucket': 0}} 2024-09-14T06:30:34.051 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-14T06:30:34.052 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-14T06:30:34.091 INFO:tasks.ceph.osd.2.smithi094.stderr:2024-09-14T06:30:34.089+0000 7faf3269b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33649) UID: 0 2024-09-14T06:30:34.099 INFO:teuthology.orchestra.run.smithi094.stdout:3349 trunc from 0xdce17d2 to 0x81d134c 2024-09-14T06:30:34.112 INFO:teuthology.orchestra.run.smithi094.stdout:3350 writesame 0x4317b6c thru 0x431c3a3 data_size 0x4(0x4838 bytes) 2024-09-14T06:30:34.130 INFO:teuthology.orchestra.run.smithi094.stdout:3351 writesame 0xddde8d6 thru 0xddec083 data_size 0x13(0xd7ae bytes) 2024-09-14T06:30:34.131 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-14T06:30:34.131 INFO:teuthology.orchestra.run.smithi094.stdout:3353 write 0xdc14ecf thru 0xdc1bd51 (0x6e83 bytes) 2024-09-14T06:30:34.146 INFO:teuthology.orchestra.run.smithi094.stdout:3354 trunc from 0xddec084 to 0x3e47ef0 2024-09-14T06:30:34.166 INFO:tasks.ceph.osd.4.smithi138.stderr:2024-09-14T06:30:34.165+0000 7f6c653c8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33615) UID: 0 2024-09-14T06:30:34.185 INFO:teuthology.orchestra.run.smithi094.stdout:3355 read 0x336197e thru 0x3369426 (0x7aa9 bytes) 2024-09-14T06:30:34.185 INFO:teuthology.orchestra.run.smithi094.stdout:3356 read 0xa96dd8 thru 0xaa363c (0xc865 bytes) 2024-09-14T06:30:34.193 INFO:teuthology.orchestra.run.smithi094.stdout:3358 writesame 0x5d3b7cd thru 0x5d3f5eb data_size 0x39(0x3e1f bytes) 2024-09-14T06:30:34.202 INFO:teuthology.orchestra.run.smithi094.stdout:3359 write 0x9454b4b thru 0x9458cd2 (0x4188 bytes) 2024-09-14T06:30:34.207 INFO:teuthology.orchestra.run.smithi138.stdout:ERROR: (22) Invalid argument 2024-09-14T06:30:34.208 INFO:teuthology.orchestra.run.smithi138.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:30:34.214 INFO:teuthology.orchestra.run.smithi094.stdout:3360 write 0xc0ad8a2 thru 0xc0b8ac5 (0xb224 bytes) 2024-09-14T06:30:34.218 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-14T06:30:34.218 INFO:teuthology.orchestra.run.smithi094.stdout:3361 writesame 0xaf473cb thru 0xaf4f7c2 data_size 0x29(0x83f8 bytes) 2024-09-14T06:30:34.221 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-14T06:30:34.222 INFO:teuthology.orchestra.run.smithi094.stdout:3363 read 0x451eee4 thru 0x45204c9 (0x15e6 bytes) 2024-09-14T06:30:34.222 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-14T06:30:34.234 INFO:teuthology.orchestra.run.smithi094.stdout:3371 trunc from 0xc0b8ac6 to 0x18624e7 2024-09-14T06:30:34.257 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-14T06:30:34.257 INFO:teuthology.orchestra.run.smithi094.stdout:3374 punch from 0x606ba8 to 0x615476, (0xe8ce bytes) 2024-09-14T06:30:34.257 INFO:teuthology.orchestra.run.smithi094.stdout:3377 punch from 0xf03e3e to 0xf0d897, (0x9a59 bytes) 2024-09-14T06:30:34.265 INFO:teuthology.orchestra.run.smithi094.stdout:3378 writesame 0x3c4d7d1 thru 0x3c4e585 data_size 0xb(0xdb5 bytes) 2024-09-14T06:30:34.267 INFO:teuthology.orchestra.run.smithi094.stdout:3379 trunc from 0x3c4e586 to 0x2b36036 2024-09-14T06:30:34.272 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-14T06:30:34.273 INFO:teuthology.orchestra.run.smithi094.stdout:3384 read 0xa34074 thru 0xa3b05d (0x6fea bytes) 2024-09-14T06:30:34.273 INFO:teuthology.orchestra.run.smithi094.stdout:3385 read 0x7ffa85 thru 0x802202 (0x277e bytes) 2024-09-14T06:30:34.274 INFO:teuthology.orchestra.run.smithi094.stdout:3386 read 0x20177ec thru 0x201a848 (0x305d bytes) 2024-09-14T06:30:34.274 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-14T06:30:34.274 INFO:teuthology.orchestra.run.smithi094.stdout:3393 punch from 0x1a0cdad to 0x1a1a8e7, (0xdb3a bytes) 2024-09-14T06:30:34.283 INFO:teuthology.orchestra.run.smithi094.stdout:3394 writesame 0x4c9d00b thru 0x4ca7c13 data_size 0x1(0xac09 bytes) 2024-09-14T06:30:34.283 INFO:teuthology.orchestra.run.smithi094.stdout:3395 read 0x344de31 thru 0x344f555 (0x1725 bytes) 2024-09-14T06:30:34.283 INFO:teuthology.orchestra.run.smithi094.stdout:3396 writesame 0x17cddd8 thru 0x17cf9a9 data_size 0x3(0x1bd2 bytes) 2024-09-14T06:30:34.285 INFO:teuthology.orchestra.run.smithi094.stdout:3399 punch from 0xdd6848 to 0xdde7f5, (0x7fad bytes) 2024-09-14T06:30:34.285 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-14T06:30:34.302 INFO:tasks.ceph.osd.5.smithi138.stderr:2024-09-14T06:30:34.301+0000 7fe13746f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33618) UID: 0 2024-09-14T06:30:34.302 INFO:teuthology.orchestra.run.smithi094.stdout:3402 write 0xdadc948 thru 0xdadf9da (0x3093 bytes) 2024-09-14T06:30:34.306 INFO:teuthology.orchestra.run.smithi094.stdout:3403 read 0xbebb0a6 thru 0xbec1bca (0x6b25 bytes) 2024-09-14T06:30:34.306 INFO:teuthology.orchestra.run.smithi094.stdout:3405 write 0x8ba1a29 thru 0x8baac32 (0x920a bytes) 2024-09-14T06:30:34.319 INFO:teuthology.orchestra.run.smithi094.stdout:3408 trunc from 0xdadf9db to 0x49e5117 2024-09-14T06:30:34.331 INFO:teuthology.orchestra.run.smithi094.stdout:3409 read 0x4743c00 thru 0x474d78b (0x9b8c bytes) 2024-09-14T06:30:34.331 INFO:teuthology.orchestra.run.smithi094.stdout:3410 punch from 0x2425f34 to 0x242f0c5, (0x9191 bytes) 2024-09-14T06:30:34.331 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-14T06:30:34.340 INFO:teuthology.orchestra.run.smithi094.stdout:3412 write 0x5c40603 thru 0x5c4ed60 (0xe75e bytes) 2024-09-14T06:30:34.346 INFO:teuthology.orchestra.run.smithi094.stdout:3413 trunc from 0x5c4ed61 to 0x390fdba 2024-09-14T06:30:34.358 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-14T06:30:34.359 INFO:teuthology.orchestra.run.smithi094.stdout:3415 trunc from 0x390fdba to 0x48b2d1a 2024-09-14T06:30:34.367 INFO:tasks.ceph.osd.5.smithi138.stderr:2024-09-14T06:30:34.366+0000 7fe13746f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33618) UID: 0 2024-09-14T06:30:34.371 INFO:teuthology.orchestra.run.smithi138.stdout:ERROR: (22) Invalid argument 2024-09-14T06:30:34.371 INFO:teuthology.orchestra.run.smithi138.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:30:34.380 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-14T06:30:34.382 INFO:teuthology.orchestra.run.smithi094.stdout:3416 write 0xe4602ff thru 0xe46eced (0xe9ef bytes) 2024-09-14T06:30:34.397 INFO:teuthology.orchestra.run.smithi094.stdout:3417 trunc from 0xe46ecee to 0x426a2f6 2024-09-14T06:30:34.406 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-14T06:30:34.408 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-14T06:30:34.409 INFO:teuthology.orchestra.run.smithi094.stderr:dumped all 2024-09-14T06:30:34.421 INFO:teuthology.orchestra.run.smithi094.stdout:3420 writesame 0xc225caa thru 0xc2338e5 data_size 0xa(0xdc3c bytes) 2024-09-14T06:30:34.421 INFO:teuthology.orchestra.run.smithi094.stdout:3421 punch from 0x8d3d41e to 0x8d407e0, (0x33c2 bytes) 2024-09-14T06:30:34.422 INFO:teuthology.orchestra.run.smithi094.stdout:3422 write 0x5636033 thru 0x5640ee8 (0xaeb6 bytes) 2024-09-14T06:30:34.423 INFO:teuthology.orchestra.run.smithi094.stdout:{"pg_ready":true,"pg_map":{"version":161,"stamp":"2024-09-14T06:30:33.650465+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1171607514,"num_objects":1375,"num_object_clones":0,"num_object_copies":2750,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1375,"num_whiteouts":0,"num_read":134342,"num_read_kb":1237765,"num_write":15902,"num_write_kb":866889,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":39,"num_bytes_recovered":1400964,"num_keys_recovered":11,"num_objects_omap":25,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":11819,"ondisk_log_size":11819,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":410996,"kb_used_data":86476,"kb_used_omap":12,"kb_used_meta":324275,"kb_avail":754563724,"statfs":{"total":773094113280,"available":772673253376,"internally_reserved":0,"allocated":88551424,"data_stored":780749241,"data_compressed":47533142,"data_compressed_allocated":62562304,"data_compressed_original":763470500,"omap_allocated":12751,"internal_metadata":332058161},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":34162574,"num_objects":29,"num_object_clones":0,"num_object_copies":58,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":29,"num_whiteouts":0,"num_read":1999,"num_read_kb":22852,"num_write":418,"num_write_kb":21152,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.013581"},"pg_stats":[{"pgid":"3.1f","version":"110'458","reported_seq":4057,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-09-14T06:30:32.827143+0000","last_change":"2024-09-14T06:30:30.589009+0000","last_active":"2024-09-14T06:30:32.827143+0000","last_peered":"2024-09-14T06:30:32.827143+0000","last_clean":"2024-09-14T06:30:32.827143+0000","last_became_active":"2024-09-14T06:29:00.579883+0000","last_became_peered":"2024-09-14T06:29:00.579883+0000","last_unstale":"2024-09-14T06:30:32.827143+0000","last_undegraded":"2024-09-14T06:30:32.827143+0000","last_fullsized":"2024-09-14T06:30:32.827143+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":458,"log_dups_size":0,"ondisk_log_size":458,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T16:50:48.782866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010287320000000001,"stat_sum":{"num_bytes":43318263,"num_objects":42,"num_object_clones":0,"num_object_copies":84,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":42,"num_whiteouts":0,"num_read":4950,"num_read_kb":41970,"num_write":696,"num_write_kb":22618,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"2e","length":"1"},{"start":"30","length":"1"}]},{"pgid":"3.1e","version":"105'290","reported_seq":4940,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-09-14T06:30:30.728190+0000","last_change":"2024-09-14T06:30:30.728354+0000","last_active":"2024-09-14T06:30:30.728190+0000","last_peered":"2024-09-14T06:30:30.728190+0000","last_clean":"2024-09-14T06:30:30.728190+0000","last_became_active":"2024-09-14T06:29:00.580507+0000","last_became_peered":"2024-09-14T06:29:00.580507+0000","last_unstale":"2024-09-14T06:30:30.728190+0000","last_undegraded":"2024-09-14T06:30:30.728190+0000","last_fullsized":"2024-09-14T06:30:30.728190+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":290,"log_dups_size":0,"ondisk_log_size":290,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T11:56:53.772195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0016057650000000001,"stat_sum":{"num_bytes":28440908,"num_objects":33,"num_object_clones":0,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":33,"num_whiteouts":0,"num_read":4697,"num_read_kb":34742,"num_write":347,"num_write_kb":25429,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2e","length":"1"},{"start":"30","length":"1"}]},{"pgid":"3.1d","version":"105'486","reported_seq":4991,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-09-14T06:30:28.642326+0000","last_change":"2024-09-14T06:30:27.628397+0000","last_active":"2024-09-14T06:30:28.642326+0000","last_peered":"2024-09-14T06:30:28.642326+0000","last_clean":"2024-09-14T06:30:28.642326+0000","last_became_active":"2024-09-14T06:29:00.579345+0000","last_became_peered":"2024-09-14T06:29:00.579345+0000","last_unstale":"2024-09-14T06:30:28.642326+0000","last_undegraded":"2024-09-14T06:30:28.642326+0000","last_fullsized":"2024-09-14T06:30:28.642326+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":486,"log_dups_size":0,"ondisk_log_size":486,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T13:11:44.710501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066042900000000003,"stat_sum":{"num_bytes":43960317,"num_objects":41,"num_object_clones":0,"num_object_copies":82,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":41,"num_whiteouts":0,"num_read":5628,"num_read_kb":37028,"num_write":728,"num_write_kb":25622,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2a","length":"1"},{"start":"2c","length":"1"},{"start":"2e","length":"1"}]},{"pgid":"3.1c","version":"106'283","reported_seq":2581,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-09-14T06:30:28.642376+0000","last_change":"2024-09-14T06:30:27.628138+0000","last_active":"2024-09-14T06:30:28.642376+0000","last_peered":"2024-09-14T06:30:28.642376+0000","last_clean":"2024-09-14T06:30:28.642376+0000","last_became_active":"2024-09-14T06:29:00.579807+0000","last_became_peered":"2024-09-14T06:29:00.579807+0000","last_unstale":"2024-09-14T06:30:28.642376+0000","last_undegraded":"2024-09-14T06:30:28.642376+0000","last_fullsized":"2024-09-14T06:30:28.642376+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":283,"log_dups_size":0,"ondisk_log_size":283,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T13:43:20.804281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047123500000000002,"stat_sum":{"num_bytes":17988923,"num_objects":45,"num_object_clones":0,"num_object_copies":90,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":45,"num_whiteouts":0,"num_read":2535,"num_read_kb":28399,"num_write":343,"num_write_kb":28800,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2a","length":"1"},{"start":"2c","length":"1"},{"start":"2e","length":"1"}]},{"pgid":"3.1b","version":"110'219","reported_seq":7135,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-09-14T06:30:32.659947+0000","last_change":"2024-09-14T06:30:30.588352+0000","last_active":"2024-09-14T06:30:32.659947+0000","last_peered":"2024-09-14T06:30:32.659947+0000","last_clean":"2024-09-14T06:30:32.659947+0000","last_became_active":"2024-09-14T06:29:00.586183+0000","last_became_peered":"2024-09-14T06:29:00.586183+0000","last_unstale":"2024-09-14T06:30:32.659947+0000","last_undegraded":"2024-09-14T06:30:32.659947+0000","last_fullsized":"2024-09-14T06:30:32.659947+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":219,"log_dups_size":0,"ondisk_log_size":219,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T17:19:38.088152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001303539,"stat_sum":{"num_bytes":56512327,"num_objects":58,"num_object_clones":0,"num_object_copies":116,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":58,"num_whiteouts":0,"num_read":6764,"num_read_kb":68260,"num_write":240,"num_write_kb":33243,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"2e","length":"1"},{"start":"30","length":"1"}]},{"pgid":"3.1a","version":"105'869","reported_seq":5411,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-09-14T06:30:28.664638+0000","last_change":"2024-09-14T06:30:27.779386+0000","last_active":"2024-09-14T06:30:28.664638+0000","last_peered":"2024-09-14T06:30:28.664638+0000","last_clean":"2024-09-14T06:30:28.664638+0000","last_became_active":"2024-09-14T06:29:00.576206+0000","last_became_peered":"2024-09-14T06:29:00.576206+0000","last_unstale":"2024-09-14T06:30:28.664638+0000","last_undegraded":"2024-09-14T06:30:28.664638+0000","last_fullsized":"2024-09-14T06:30:28.664638+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":869,"log_dups_size":0,"ondisk_log_size":869,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T13:11:02.897169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00060567599999999998,"stat_sum":{"num_bytes":39956900,"num_objects":50,"num_object_clones":0,"num_object_copies":100,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":50,"num_whiteouts":0,"num_read":7443,"num_read_kb":42215,"num_write":1230,"num_write_kb":32985,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"2a","length":"1"},{"start":"2c","length":"1"},{"start":"2e","length":"1"}]},{"pgid":"3.19","version":"106'259","reported_seq":2859,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-09-14T06:30:28.678164+0000","last_change":"2024-09-14T06:30:27.628870+0000","last_active":"2024-09-14T06:30:28.678164+0000","last_peered":"2024-09-14T06:30:28.678164+0000","last_clean":"2024-09-14T06:30:28.678164+0000","last_became_active":"2024-09-14T06:29:00.575639+0000","last_became_peered":"2024-09-14T06:29:00.575639+0000","last_unstale":"2024-09-14T06:30:28.678164+0000","last_undegraded":"2024-09-14T06:30:28.678164+0000","last_fullsized":"2024-09-14T06:30:28.678164+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":259,"log_dups_size":0,"ondisk_log_size":259,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T13:48:05.805592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049481099999999995,"stat_sum":{"num_bytes":34593974,"num_objects":44,"num_object_clones":0,"num_object_copies":88,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":44,"num_whiteouts":0,"num_read":2840,"num_read_kb":42011,"num_write":358,"num_write_kb":26631,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2a","length":"1"},{"start":"2c","length":"1"},{"start":"2e","length":"1"}]},{"pgid":"3.18","version":"107'179","reported_seq":2382,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-09-14T06:30:30.727979+0000","last_change":"2024-09-14T06:30:30.728161+0000","last_active":"2024-09-14T06:30:30.727979+0000","last_peered":"2024-09-14T06:30:30.727979+0000","last_clean":"2024-09-14T06:30:30.727979+0000","last_became_active":"2024-09-14T06:29:00.578017+0000","last_became_peered":"2024-09-14T06:29:00.578017+0000","last_unstale":"2024-09-14T06:30:30.727979+0000","last_undegraded":"2024-09-14T06:30:30.727979+0000","last_fullsized":"2024-09-14T06:30:30.727979+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":179,"log_dups_size":0,"ondisk_log_size":179,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T12:25:51.391940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001476783,"stat_sum":{"num_bytes":28762789,"num_objects":43,"num_object_clones":0,"num_object_copies":86,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":43,"num_whiteouts":0,"num_read":1930,"num_read_kb":33159,"num_write":191,"num_write_kb":26635,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2e","length":"1"},{"start":"30","length":"1"}]},{"pgid":"3.17","version":"110'439","reported_seq":6586,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-09-14T06:30:32.818886+0000","last_change":"2024-09-14T06:30:30.589061+0000","last_active":"2024-09-14T06:30:32.818886+0000","last_peered":"2024-09-14T06:30:32.818886+0000","last_clean":"2024-09-14T06:30:32.818886+0000","last_became_active":"2024-09-14T06:29:00.579714+0000","last_became_peered":"2024-09-14T06:29:00.579714+0000","last_unstale":"2024-09-14T06:30:32.818886+0000","last_undegraded":"2024-09-14T06:30:32.818886+0000","last_fullsized":"2024-09-14T06:30:32.818886+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":439,"log_dups_size":0,"ondisk_log_size":439,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T08:21:43.178165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077245199999999997,"stat_sum":{"num_bytes":60827393,"num_objects":61,"num_object_clones":0,"num_object_copies":122,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":61,"num_whiteouts":0,"num_read":7051,"num_read_kb":51191,"num_write":524,"num_write_kb":30879,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"2e","length":"1"},{"start":"30","length":"1"}]},{"pgid":"3.16","version":"105'309","reported_seq":1390,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-09-14T06:30:28.642314+0000","last_change":"2024-09-14T06:30:27.628171+0000","last_active":"2024-09-14T06:30:28.642314+0000","last_peered":"2024-09-14T06:30:28.642314+0000","last_clean":"2024-09-14T06:30:28.642314+0000","last_became_active":"2024-09-14T06:29:00.580070+0000","last_became_peered":"2024-09-14T06:29:00.580070+0000","last_unstale":"2024-09-14T06:30:28.642314+0000","last_undegraded":"2024-09-14T06:30:28.642314+0000","last_fullsized":"2024-09-14T06:30:28.642314+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":309,"log_dups_size":0,"ondisk_log_size":309,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T09:47:19.336529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047747399999999998,"stat_sum":{"num_bytes":17620777,"num_objects":40,"num_object_clones":0,"num_object_copies":80,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":40,"num_whiteouts":0,"num_read":1721,"num_read_kb":11766,"num_write":445,"num_write_kb":17625,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2a","length":"1"},{"start":"2c","length":"1"},{"start":"2e","length":"1"}]},{"pgid":"3.15","version":"105'275","reported_seq":2887,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-14T06:30:28.321543+0000","last_change":"2024-09-14T06:30:27.770341+0000","last_active":"2024-09-14T06:30:28.321543+0000","last_peered":"2024-09-14T06:30:28.321543+0000","last_clean":"2024-09-14T06:30:28.321543+0000","last_became_active":"2024-09-14T06:29:00.578451+0000","last_became_peered":"2024-09-14T06:29:00.578451+0000","last_unstale":"2024-09-14T06:30:28.321543+0000","last_undegraded":"2024-09-14T06:30:28.321543+0000","last_fullsized":"2024-09-14T06:30:28.321543+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":275,"log_dups_size":0,"ondisk_log_size":275,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T13:00:07.138163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047808900000000001,"stat_sum":{"num_bytes":35165078,"num_objects":41,"num_object_clones":0,"num_object_copies":82,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":41,"num_whiteouts":0,"num_read":2460,"num_read_kb":50305,"num_write":290,"num_write_kb":38395,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"2a","length":"1"},{"start":"2c","length":"1"},{"start":"2e","length":"1"}]},{"pgid":"3.14","version":"105'672","reported_seq":3804,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-09-14T06:30:28.633721+0000","last_change":"2024-09-14T06:30:27.779576+0000","last_active":"2024-09-14T06:30:28.633721+0000","last_peered":"2024-09-14T06:30:28.633721+0000","last_clean":"2024-09-14T06:30:28.633721+0000","last_became_active":"2024-09-14T06:29:00.582595+0000","last_became_peered":"2024-09-14T06:29:00.582595+0000","last_unstale":"2024-09-14T06:30:28.633721+0000","last_undegraded":"2024-09-14T06:30:28.633721+0000","last_fullsized":"2024-09-14T06:30:28.633721+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":672,"log_dups_size":0,"ondisk_log_size":672,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T16:25:54.875217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066056399999999999,"stat_sum":{"num_bytes":35195867,"num_objects":35,"num_object_clones":0,"num_object_copies":70,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":35,"num_whiteouts":0,"num_read":6283,"num_read_kb":33434,"num_write":1044,"num_write_kb":15111,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"2a","length":"1"},{"start":"2c","length":"1"},{"start":"2e","length":"1"}]},{"pgid":"2.7","version":"0'0","reported_seq":184,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-14T06:30:27.773482+0000","last_change":"2024-09-14T06:28:53.625193+0000","last_active":"2024-09-14T06:30:27.773482+0000","last_peered":"2024-09-14T06:30:27.773482+0000","last_clean":"2024-09-14T06:30:27.773482+0000","last_became_active":"2024-09-14T06:28:51.223535+0000","last_became_peered":"2024-09-14T06:28:51.223535+0000","last_unstale":"2024-09-14T06:30:27.773482+0000","last_undegraded":"2024-09-14T06:30:27.773482+0000","last_fullsized":"2024-09-14T06:30:27.773482+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:28:50.205350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:28:50.205350+0000","last_clean_scrub_stamp":"2024-09-14T06:28:50.205350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T18:14:22.822245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00052660300000000005,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"110'226","reported_seq":2283,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-09-14T06:30:32.755194+0000","last_change":"2024-09-14T06:30:30.588717+0000","last_active":"2024-09-14T06:30:32.755194+0000","last_peered":"2024-09-14T06:30:32.755194+0000","last_clean":"2024-09-14T06:30:32.755194+0000","last_became_active":"2024-09-14T06:29:00.578283+0000","last_became_peered":"2024-09-14T06:29:00.578283+0000","last_unstale":"2024-09-14T06:30:32.755194+0000","last_undegraded":"2024-09-14T06:30:32.755194+0000","last_fullsized":"2024-09-14T06:30:32.755194+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":226,"log_dups_size":0,"ondisk_log_size":226,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T16:19:46.717729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012352330000000001,"stat_sum":{"num_bytes":27240028,"num_objects":38,"num_object_clones":0,"num_object_copies":76,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":38,"num_whiteouts":0,"num_read":1969,"num_read_kb":20636,"num_write":241,"num_write_kb":19949,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"2e","length":"1"},{"start":"30","length":"1"}]},{"pgid":"2.6","version":"0'0","reported_seq":188,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-09-14T06:30:28.633707+0000","last_change":"2024-09-14T06:28:53.226122+0000","last_active":"2024-09-14T06:30:28.633707+0000","last_peered":"2024-09-14T06:30:28.633707+0000","last_clean":"2024-09-14T06:30:28.633707+0000","last_became_active":"2024-09-14T06:28:51.217205+0000","last_became_peered":"2024-09-14T06:28:51.217205+0000","last_unstale":"2024-09-14T06:30:28.633707+0000","last_undegraded":"2024-09-14T06:30:28.633707+0000","last_fullsized":"2024-09-14T06:30:28.633707+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:28:50.205350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:28:50.205350+0000","last_clean_scrub_stamp":"2024-09-14T06:28:50.205350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T15:29:19.563302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065018200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"110'367","reported_seq":2562,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-09-14T06:30:32.677594+0000","last_change":"2024-09-14T06:30:30.588756+0000","last_active":"2024-09-14T06:30:32.677594+0000","last_peered":"2024-09-14T06:30:32.677594+0000","last_clean":"2024-09-14T06:30:32.677594+0000","last_became_active":"2024-09-14T06:29:56.584268+0000","last_became_peered":"2024-09-14T06:29:56.584268+0000","last_unstale":"2024-09-14T06:30:32.677594+0000","last_undegraded":"2024-09-14T06:30:32.677594+0000","last_fullsized":"2024-09-14T06:30:32.677594+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":367,"log_dups_size":0,"ondisk_log_size":367,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T15:24:23.000833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00089781300000000002,"stat_sum":{"num_bytes":33902660,"num_objects":35,"num_object_clones":0,"num_object_copies":70,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":35,"num_whiteouts":0,"num_read":2627,"num_read_kb":23283,"num_write":503,"num_write_kb":21949,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":39,"num_bytes_recovered":1400964,"num_keys_recovered":11,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"2e","length":"1"},{"start":"30","length":"1"}]},{"pgid":"2.5","version":"0'0","reported_seq":186,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-14T06:30:27.770131+0000","last_change":"2024-09-14T06:28:53.640035+0000","last_active":"2024-09-14T06:30:27.770131+0000","last_peered":"2024-09-14T06:30:27.770131+0000","last_clean":"2024-09-14T06:30:27.770131+0000","last_became_active":"2024-09-14T06:28:51.308058+0000","last_became_peered":"2024-09-14T06:28:51.308058+0000","last_unstale":"2024-09-14T06:30:27.770131+0000","last_undegraded":"2024-09-14T06:30:27.770131+0000","last_fullsized":"2024-09-14T06:30:27.770131+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:28:50.205350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:28:50.205350+0000","last_clean_scrub_stamp":"2024-09-14T06:28:50.205350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T13:35:48.544057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00037824000000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"105'199","reported_seq":1102,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-09-14T06:30:28.634112+0000","last_change":"2024-09-14T06:30:27.628534+0000","last_active":"2024-09-14T06:30:28.634112+0000","last_peered":"2024-09-14T06:30:28.634112+0000","last_clean":"2024-09-14T06:30:28.634112+0000","last_became_active":"2024-09-14T06:29:00.574895+0000","last_became_peered":"2024-09-14T06:29:00.574895+0000","last_unstale":"2024-09-14T06:30:28.634112+0000","last_undegraded":"2024-09-14T06:30:28.634112+0000","last_fullsized":"2024-09-14T06:30:28.634112+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":199,"log_dups_size":0,"ondisk_log_size":199,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T13:20:49.569365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00058354499999999998,"stat_sum":{"num_bytes":13383727,"num_objects":36,"num_object_clones":0,"num_object_copies":72,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":36,"num_whiteouts":0,"num_read":731,"num_read_kb":15418,"num_write":247,"num_write_kb":20235,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2a","length":"1"},{"start":"2c","length":"1"},{"start":"2e","length":"1"}]},{"pgid":"2.4","version":"0'0","reported_seq":188,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-09-14T06:30:28.634187+0000","last_change":"2024-09-14T06:28:53.226072+0000","last_active":"2024-09-14T06:30:28.634187+0000","last_peered":"2024-09-14T06:30:28.634187+0000","last_clean":"2024-09-14T06:30:28.634187+0000","last_became_active":"2024-09-14T06:28:51.308072+0000","last_became_peered":"2024-09-14T06:28:51.308072+0000","last_unstale":"2024-09-14T06:30:28.634187+0000","last_undegraded":"2024-09-14T06:30:28.634187+0000","last_fullsized":"2024-09-14T06:30:28.634187+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:28:50.205350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:28:50.205350+0000","last_clean_scrub_stamp":"2024-09-14T06:28:50.205350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T13:15:09.052328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066197399999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"105'553","reported_seq":2259,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-09-14T06:30:28.642468+0000","last_change":"2024-09-14T06:30:27.628620+0000","last_active":"2024-09-14T06:30:28.642468+0000","last_peered":"2024-09-14T06:30:28.642468+0000","last_clean":"2024-09-14T06:30:28.642468+0000","last_became_active":"2024-09-14T06:29:00.581874+0000","last_became_peered":"2024-09-14T06:29:00.581874+0000","last_unstale":"2024-09-14T06:30:28.642468+0000","last_undegraded":"2024-09-14T06:30:28.642468+0000","last_fullsized":"2024-09-14T06:30:28.642468+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":553,"log_dups_size":0,"ondisk_log_size":553,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T17:27:56.304782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00068477100000000003,"stat_sum":{"num_bytes":27242971,"num_objects":41,"num_object_clones":0,"num_object_copies":82,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":41,"num_whiteouts":0,"num_read":2810,"num_read_kb":19328,"num_write":849,"num_write_kb":31859,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2a","length":"1"},{"start":"2c","length":"1"},{"start":"2e","length":"1"}]},{"pgid":"3.3","version":"105'227","reported_seq":7717,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-09-14T06:30:28.633739+0000","last_change":"2024-09-14T06:30:27.779456+0000","last_active":"2024-09-14T06:30:28.633739+0000","last_peered":"2024-09-14T06:30:28.633739+0000","last_clean":"2024-09-14T06:30:28.633739+0000","last_became_active":"2024-09-14T06:29:00.584678+0000","last_became_peered":"2024-09-14T06:29:00.584678+0000","last_unstale":"2024-09-14T06:30:28.633739+0000","last_undegraded":"2024-09-14T06:30:28.633739+0000","last_fullsized":"2024-09-14T06:30:28.633739+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":227,"log_dups_size":0,"ondisk_log_size":227,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T09:14:52.082777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062621599999999997,"stat_sum":{"num_bytes":49423207,"num_objects":33,"num_object_clones":0,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":33,"num_whiteouts":0,"num_read":7399,"num_read_kb":64280,"num_write":223,"num_write_kb":32792,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"2a","length":"1"},{"start":"2c","length":"1"},{"start":"2e","length":"1"}]},{"pgid":"2.2","version":"22'2","reported_seq":190,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-09-14T06:30:28.642165+0000","last_change":"2024-09-14T06:28:53.231428+0000","last_active":"2024-09-14T06:30:28.642165+0000","last_peered":"2024-09-14T06:30:28.642165+0000","last_clean":"2024-09-14T06:30:28.642165+0000","last_became_active":"2024-09-14T06:28:51.222937+0000","last_became_peered":"2024-09-14T06:28:51.222937+0000","last_unstale":"2024-09-14T06:30:28.642165+0000","last_undegraded":"2024-09-14T06:30:28.642165+0000","last_fullsized":"2024-09-14T06:30:28.642165+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:28:50.205350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:28:50.205350+0000","last_clean_scrub_stamp":"2024-09-14T06:28:50.205350+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T14:27:37.233976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010072970000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.0","version":"105'185","reported_seq":3945,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-09-14T06:30:28.634288+0000","last_change":"2024-09-14T06:30:27.627964+0000","last_active":"2024-09-14T06:30:28.634288+0000","last_peered":"2024-09-14T06:30:28.634288+0000","last_clean":"2024-09-14T06:30:28.634288+0000","last_became_active":"2024-09-14T06:29:00.574873+0000","last_became_peered":"2024-09-14T06:29:00.574873+0000","last_unstale":"2024-09-14T06:30:28.634288+0000","last_undegraded":"2024-09-14T06:30:28.634288+0000","last_fullsized":"2024-09-14T06:30:28.634288+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":185,"log_dups_size":0,"ondisk_log_size":185,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T12:07:22.189466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00052652000000000005,"stat_sum":{"num_bytes":44082536,"num_objects":52,"num_object_clones":0,"num_object_copies":104,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":52,"num_whiteouts":0,"num_read":3439,"num_read_kb":52276,"num_write":231,"num_write_kb":33359,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2a","length":"1"},{"start":"2c","length":"1"},{"start":"2e","length":"1"}]},{"pgid":"2.1","version":"0'0","reported_seq":195,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-09-14T06:30:32.644406+0000","last_change":"2024-09-14T06:28:53.226264+0000","last_active":"2024-09-14T06:30:32.644406+0000","last_peered":"2024-09-14T06:30:32.644406+0000","last_clean":"2024-09-14T06:30:32.644406+0000","last_became_active":"2024-09-14T06:28:51.223252+0000","last_became_peered":"2024-09-14T06:28:51.223252+0000","last_unstale":"2024-09-14T06:30:32.644406+0000","last_undegraded":"2024-09-14T06:30:32.644406+0000","last_fullsized":"2024-09-14T06:30:32.644406+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:28:50.205350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:28:50.205350+0000","last_clean_scrub_stamp":"2024-09-14T06:28:50.205350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T11:56:04.373414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053403100000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1","version":"110'454","reported_seq":7099,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-09-14T06:30:32.659910+0000","last_change":"2024-09-14T06:30:30.588047+0000","last_active":"2024-09-14T06:30:32.659910+0000","last_peered":"2024-09-14T06:30:32.659910+0000","last_clean":"2024-09-14T06:30:32.659910+0000","last_became_active":"2024-09-14T06:29:00.585673+0000","last_became_peered":"2024-09-14T06:29:00.585673+0000","last_unstale":"2024-09-14T06:30:32.659910+0000","last_undegraded":"2024-09-14T06:30:32.659910+0000","last_fullsized":"2024-09-14T06:30:32.659910+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":454,"log_dups_size":0,"ondisk_log_size":454,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T11:42:19.451388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010428970000000001,"stat_sum":{"num_bytes":54984688,"num_objects":41,"num_object_clones":0,"num_object_copies":82,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":41,"num_whiteouts":0,"num_read":7481,"num_read_kb":63640,"num_write":568,"num_write_kb":26042,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"2e","length":"1"},{"start":"30","length":"1"}]},{"pgid":"2.0","version":"0'0","reported_seq":186,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-14T06:30:27.769723+0000","last_change":"2024-09-14T06:28:53.640096+0000","last_active":"2024-09-14T06:30:27.769723+0000","last_peered":"2024-09-14T06:30:27.769723+0000","last_clean":"2024-09-14T06:30:27.769723+0000","last_became_active":"2024-09-14T06:28:51.223069+0000","last_became_peered":"2024-09-14T06:28:51.223069+0000","last_unstale":"2024-09-14T06:30:27.769723+0000","last_undegraded":"2024-09-14T06:30:27.769723+0000","last_fullsized":"2024-09-14T06:30:27.769723+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:28:50.205350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:28:50.205350+0000","last_clean_scrub_stamp":"2024-09-14T06:28:50.205350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T15:13:00.471509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00043714600000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"105'460","reported_seq":4404,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-09-14T06:30:30.727011+0000","last_change":"2024-09-14T06:30:30.727180+0000","last_active":"2024-09-14T06:30:30.727011+0000","last_peered":"2024-09-14T06:30:30.727011+0000","last_clean":"2024-09-14T06:30:30.727011+0000","last_became_active":"2024-09-14T06:29:00.578574+0000","last_became_peered":"2024-09-14T06:29:00.578574+0000","last_unstale":"2024-09-14T06:30:30.727011+0000","last_undegraded":"2024-09-14T06:30:30.727011+0000","last_fullsized":"2024-09-14T06:30:30.727011+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":460,"log_dups_size":0,"ondisk_log_size":460,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T09:41:44.424444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000560974,"stat_sum":{"num_bytes":45461194,"num_objects":56,"num_object_clones":0,"num_object_copies":112,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":56,"num_whiteouts":0,"num_read":4552,"num_read_kb":45712,"num_write":583,"num_write_kb":33990,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2e","length":"1"},{"start":"30","length":"1"}]},{"pgid":"2.3","version":"20'1","reported_seq":189,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-09-14T06:30:28.642155+0000","last_change":"2024-09-14T06:28:53.230933+0000","last_active":"2024-09-14T06:30:28.642155+0000","last_peered":"2024-09-14T06:30:28.642155+0000","last_clean":"2024-09-14T06:30:28.642155+0000","last_became_active":"2024-09-14T06:28:51.222874+0000","last_became_peered":"2024-09-14T06:28:51.222874+0000","last_unstale":"2024-09-14T06:30:28.642155+0000","last_undegraded":"2024-09-14T06:30:28.642155+0000","last_fullsized":"2024-09-14T06:30:28.642155+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:28:50.205350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:28:50.205350+0000","last_clean_scrub_stamp":"2024-09-14T06:28:50.205350+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T12:04:25.190171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050808899999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'72","reported_seq":277,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-14T06:30:27.769825+0000","last_change":"2024-09-14T06:28:44.839885+0000","last_active":"2024-09-14T06:30:27.769825+0000","last_peered":"2024-09-14T06:30:27.769825+0000","last_clean":"2024-09-14T06:30:27.769825+0000","last_became_active":"2024-09-14T06:28:44.839573+0000","last_became_peered":"2024-09-14T06:28:44.839573+0000","last_unstale":"2024-09-14T06:30:27.769825+0000","last_undegraded":"2024-09-14T06:30:27.769825+0000","last_fullsized":"2024-09-14T06:30:27.769825+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:28:43.828494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:28:43.828494+0000","last_clean_scrub_stamp":"2024-09-14T06:28:43.828494+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T14:41:25.787293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.8","version":"107'237","reported_seq":3546,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-09-14T06:30:30.728248+0000","last_change":"2024-09-14T06:30:30.728419+0000","last_active":"2024-09-14T06:30:30.728248+0000","last_peered":"2024-09-14T06:30:30.728248+0000","last_clean":"2024-09-14T06:30:30.728248+0000","last_became_active":"2024-09-14T06:29:00.574458+0000","last_became_peered":"2024-09-14T06:29:00.574458+0000","last_unstale":"2024-09-14T06:30:30.728248+0000","last_undegraded":"2024-09-14T06:30:30.728248+0000","last_fullsized":"2024-09-14T06:30:30.728248+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":237,"log_dups_size":0,"ondisk_log_size":237,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T07:14:36.793062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001538738,"stat_sum":{"num_bytes":50929397,"num_objects":51,"num_object_clones":0,"num_object_copies":102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":51,"num_whiteouts":0,"num_read":3535,"num_read_kb":67767,"num_write":292,"num_write_kb":41960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2e","length":"1"},{"start":"30","length":"1"}]},{"pgid":"3.9","version":"105'147","reported_seq":1338,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-09-14T06:30:28.633625+0000","last_change":"2024-09-14T06:30:27.779173+0000","last_active":"2024-09-14T06:30:28.633625+0000","last_peered":"2024-09-14T06:30:28.633625+0000","last_clean":"2024-09-14T06:30:28.633625+0000","last_became_active":"2024-09-14T06:29:00.573179+0000","last_became_peered":"2024-09-14T06:29:00.573179+0000","last_unstale":"2024-09-14T06:30:28.633625+0000","last_undegraded":"2024-09-14T06:30:28.633625+0000","last_fullsized":"2024-09-14T06:30:28.633625+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":147,"log_dups_size":0,"ondisk_log_size":147,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T12:25:48.769262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041994599999999998,"stat_sum":{"num_bytes":19760470,"num_objects":45,"num_object_clones":0,"num_object_copies":90,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":45,"num_whiteouts":0,"num_read":861,"num_read_kb":29231,"num_write":177,"num_write_kb":23316,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"2a","length":"1"},{"start":"2c","length":"1"},{"start":"2e","length":"1"}]},{"pgid":"3.a","version":"105'186","reported_seq":2220,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-14T06:30:28.208572+0000","last_change":"2024-09-14T06:30:27.773932+0000","last_active":"2024-09-14T06:30:28.208572+0000","last_peered":"2024-09-14T06:30:28.208572+0000","last_clean":"2024-09-14T06:30:28.208572+0000","last_became_active":"2024-09-14T06:29:00.582426+0000","last_became_peered":"2024-09-14T06:29:00.582426+0000","last_unstale":"2024-09-14T06:30:28.208572+0000","last_undegraded":"2024-09-14T06:30:28.208572+0000","last_fullsized":"2024-09-14T06:30:28.208572+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":186,"log_dups_size":0,"ondisk_log_size":186,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T14:02:00.360510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00040753399999999998,"stat_sum":{"num_bytes":38682031,"num_objects":40,"num_object_clones":0,"num_object_copies":80,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":40,"num_whiteouts":0,"num_read":1912,"num_read_kb":36970,"num_write":224,"num_write_kb":29672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"2a","length":"1"},{"start":"2c","length":"1"},{"start":"2e","length":"1"}]},{"pgid":"3.b","version":"108'1160","reported_seq":4743,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-09-14T06:30:30.728464+0000","last_change":"2024-09-14T06:30:30.728139+0000","last_active":"2024-09-14T06:30:30.728464+0000","last_peered":"2024-09-14T06:30:30.728464+0000","last_clean":"2024-09-14T06:30:30.728464+0000","last_became_active":"2024-09-14T06:29:00.578483+0000","last_became_peered":"2024-09-14T06:29:00.578483+0000","last_unstale":"2024-09-14T06:30:30.728464+0000","last_undegraded":"2024-09-14T06:30:30.728464+0000","last_fullsized":"2024-09-14T06:30:30.728464+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":1160,"log_dups_size":0,"ondisk_log_size":1160,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T10:31:40.918533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001457102,"stat_sum":{"num_bytes":23946128,"num_objects":46,"num_object_clones":0,"num_object_copies":92,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":46,"num_whiteouts":0,"num_read":9294,"num_read_kb":20256,"num_write":1913,"num_write_kb":19334,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2e","length":"1"},{"start":"30","length":"1"}]},{"pgid":"3.c","version":"106'274","reported_seq":3153,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-09-14T06:30:28.644486+0000","last_change":"2024-09-14T06:30:27.628421+0000","last_active":"2024-09-14T06:30:28.644486+0000","last_peered":"2024-09-14T06:30:28.644486+0000","last_clean":"2024-09-14T06:30:28.644486+0000","last_became_active":"2024-09-14T06:29:00.583406+0000","last_became_peered":"2024-09-14T06:29:00.583406+0000","last_unstale":"2024-09-14T06:30:28.644486+0000","last_undegraded":"2024-09-14T06:30:28.644486+0000","last_fullsized":"2024-09-14T06:30:28.644486+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":274,"log_dups_size":0,"ondisk_log_size":274,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T08:04:03.633821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050967999999999999,"stat_sum":{"num_bytes":26714859,"num_objects":38,"num_object_clones":0,"num_object_copies":76,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":38,"num_whiteouts":0,"num_read":3046,"num_read_kb":26427,"num_write":383,"num_write_kb":22832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2a","length":"1"},{"start":"2c","length":"1"},{"start":"2e","length":"1"}]},{"pgid":"3.d","version":"105'206","reported_seq":3670,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-14T06:30:28.208536+0000","last_change":"2024-09-14T06:30:27.770507+0000","last_active":"2024-09-14T06:30:28.208536+0000","last_peered":"2024-09-14T06:30:28.208536+0000","last_clean":"2024-09-14T06:30:28.208536+0000","last_became_active":"2024-09-14T06:29:00.580362+0000","last_became_peered":"2024-09-14T06:29:00.580362+0000","last_unstale":"2024-09-14T06:30:28.208536+0000","last_undegraded":"2024-09-14T06:30:28.208536+0000","last_fullsized":"2024-09-14T06:30:28.208536+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":206,"log_dups_size":0,"ondisk_log_size":206,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T08:41:40.900999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00060751499999999999,"stat_sum":{"num_bytes":45657425,"num_objects":42,"num_object_clones":0,"num_object_copies":84,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":42,"num_whiteouts":0,"num_read":3365,"num_read_kb":45716,"num_write":254,"num_write_kb":22522,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"2a","length":"1"},{"start":"2c","length":"1"},{"start":"2e","length":"1"}]},{"pgid":"3.e","version":"105'277","reported_seq":1889,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-14T06:30:28.219534+0000","last_change":"2024-09-14T06:30:27.770721+0000","last_active":"2024-09-14T06:30:28.219534+0000","last_peered":"2024-09-14T06:30:28.219534+0000","last_clean":"2024-09-14T06:30:28.219534+0000","last_became_active":"2024-09-14T06:29:00.586374+0000","last_became_peered":"2024-09-14T06:29:00.586374+0000","last_unstale":"2024-09-14T06:30:28.219534+0000","last_undegraded":"2024-09-14T06:30:28.219534+0000","last_fullsized":"2024-09-14T06:30:28.219534+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":277,"log_dups_size":0,"ondisk_log_size":277,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T11:07:15.426932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061259900000000002,"stat_sum":{"num_bytes":25133180,"num_objects":45,"num_object_clones":0,"num_object_copies":90,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":45,"num_whiteouts":0,"num_read":1700,"num_read_kb":29073,"num_write":403,"num_write_kb":25569,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"2a","length":"1"},{"start":"2c","length":"1"},{"start":"2e","length":"1"}]},{"pgid":"3.f","version":"105'248","reported_seq":3445,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-14T06:30:28.182340+0000","last_change":"2024-09-14T06:30:27.770798+0000","last_active":"2024-09-14T06:30:28.182340+0000","last_peered":"2024-09-14T06:30:28.182340+0000","last_clean":"2024-09-14T06:30:28.182340+0000","last_became_active":"2024-09-14T06:29:00.586491+0000","last_became_peered":"2024-09-14T06:29:00.586491+0000","last_unstale":"2024-09-14T06:30:28.182340+0000","last_undegraded":"2024-09-14T06:30:28.182340+0000","last_fullsized":"2024-09-14T06:30:28.182340+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":248,"log_dups_size":0,"ondisk_log_size":248,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T18:09:18.520308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00068568899999999996,"stat_sum":{"num_bytes":17931678,"num_objects":26,"num_object_clones":0,"num_object_copies":52,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":0,"num_read":3733,"num_read_kb":18278,"num_write":305,"num_write_kb":14546,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"2a","length":"1"},{"start":"2c","length":"1"},{"start":"2e","length":"1"}]},{"pgid":"3.10","version":"105'293","reported_seq":3009,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-14T06:30:28.229429+0000","last_change":"2024-09-14T06:30:27.774075+0000","last_active":"2024-09-14T06:30:28.229429+0000","last_peered":"2024-09-14T06:30:28.229429+0000","last_clean":"2024-09-14T06:30:28.229429+0000","last_became_active":"2024-09-14T06:29:00.579539+0000","last_became_peered":"2024-09-14T06:29:00.579539+0000","last_unstale":"2024-09-14T06:30:28.229429+0000","last_undegraded":"2024-09-14T06:30:28.229429+0000","last_fullsized":"2024-09-14T06:30:28.229429+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":293,"log_dups_size":0,"ondisk_log_size":293,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T07:17:22.340867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053458100000000001,"stat_sum":{"num_bytes":20889312,"num_objects":35,"num_object_clones":0,"num_object_copies":70,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":35,"num_whiteouts":0,"num_read":2719,"num_read_kb":14767,"num_write":316,"num_write_kb":24230,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"2a","length":"1"},{"start":"2c","length":"1"},{"start":"2e","length":"1"}]},{"pgid":"3.11","version":"105'380","reported_seq":7697,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-14T06:30:28.227405+0000","last_change":"2024-09-14T06:30:27.770525+0000","last_active":"2024-09-14T06:30:28.227405+0000","last_peered":"2024-09-14T06:30:28.227405+0000","last_clean":"2024-09-14T06:30:28.227405+0000","last_became_active":"2024-09-14T06:29:00.586480+0000","last_became_peered":"2024-09-14T06:29:00.586480+0000","last_unstale":"2024-09-14T06:30:28.227405+0000","last_undegraded":"2024-09-14T06:30:28.227405+0000","last_fullsized":"2024-09-14T06:30:28.227405+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":380,"log_dups_size":0,"ondisk_log_size":380,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T15:08:26.221009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057463600000000001,"stat_sum":{"num_bytes":73789373,"num_objects":52,"num_object_clones":0,"num_object_copies":104,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":52,"num_whiteouts":0,"num_read":7462,"num_read_kb":81773,"num_write":445,"num_write_kb":35125,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"2a","length":"1"},{"start":"2c","length":"1"},{"start":"2e","length":"1"}]},{"pgid":"3.12","version":"110'462","reported_seq":5526,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-09-14T06:30:32.688424+0000","last_change":"2024-09-14T06:30:30.588471+0000","last_active":"2024-09-14T06:30:32.688424+0000","last_peered":"2024-09-14T06:30:32.688424+0000","last_clean":"2024-09-14T06:30:32.688424+0000","last_became_active":"2024-09-14T06:29:00.579826+0000","last_became_peered":"2024-09-14T06:29:00.579826+0000","last_unstale":"2024-09-14T06:30:32.688424+0000","last_undegraded":"2024-09-14T06:30:32.688424+0000","last_fullsized":"2024-09-14T06:30:32.688424+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":462,"log_dups_size":0,"ondisk_log_size":462,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T12:39:50.427703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00117,"stat_sum":{"num_bytes":35173204,"num_objects":45,"num_object_clones":0,"num_object_copies":90,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":45,"num_whiteouts":0,"num_read":5850,"num_read_kb":42269,"num_write":577,"num_write_kb":29852,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"2e","length":"1"},{"start":"30","length":"1"}]},{"pgid":"3.13","version":"105'465","reported_seq":5376,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-14T06:30:28.333639+0000","last_change":"2024-09-14T06:30:27.770292+0000","last_active":"2024-09-14T06:30:28.333639+0000","last_peered":"2024-09-14T06:30:28.333639+0000","last_clean":"2024-09-14T06:30:28.333639+0000","last_became_active":"2024-09-14T06:29:00.586520+0000","last_became_peered":"2024-09-14T06:29:00.586520+0000","last_unstale":"2024-09-14T06:30:28.333639+0000","last_undegraded":"2024-09-14T06:30:28.333639+0000","last_fullsized":"2024-09-14T06:30:28.333639+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":465,"log_dups_size":0,"ondisk_log_size":465,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T09:50:04.296346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00045950299999999999,"stat_sum":{"num_bytes":54345543,"num_objects":41,"num_object_clones":0,"num_object_copies":82,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":41,"num_whiteouts":0,"num_read":5489,"num_read_kb":46130,"num_write":629,"num_write_kb":32153,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"2a","length":"1"},{"start":"2c","length":"1"},{"start":"2e","length":"1"}]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":1171017127,"num_objects":1371,"num_object_clones":0,"num_object_copies":2742,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1371,"num_whiteouts":0,"num_read":134276,"num_read_kb":1237710,"num_write":15799,"num_write_kb":865259,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":39,"num_bytes_recovered":1400964,"num_keys_recovered":11,"num_objects_omap":25,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":78856192,"data_stored":776050777,"data_compressed":47695838,"data_compressed_allocated":62722048,"data_compressed_original":763351716,"omap_allocated":0,"internal_metadata":0},"log_size":11744,"ondisk_log_size":11744,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":131072,"data_stored":1180736,"data_compressed":65166,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509464,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":38996,"kb_used_data":11892,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94332844,"statfs":{"total":96636764160,"available":96596832256,"internally_reserved":0,"allocated":12177408,"data_stored":105550460,"data_compressed":6344321,"data_compressed_allocated":8302592,"data_compressed_original":102843715,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509463,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":35096,"kb_used_data":3908,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94336744,"statfs":{"total":96636764160,"available":96600825856,"internally_reserved":0,"allocated":4001792,"data_stored":28714312,"data_compressed":1616596,"data_compressed_allocated":2199552,"data_compressed_original":27680295,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509465,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":54872,"kb_used_data":11208,"kb_used_omap":1,"kb_used_meta":43646,"kb_avail":94316968,"statfs":{"total":96636764160,"available":96580575232,"internally_reserved":0,"allocated":11476992,"data_stored":96907161,"data_compressed":5914250,"data_compressed_allocated":7892992,"data_compressed_original":94499664,"omap_allocated":1595,"internal_metadata":44693957},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509464,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":66816,"kb_used_data":17580,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94305024,"statfs":{"total":96636764160,"available":96568344576,"internally_reserved":0,"allocated":18001920,"data_stored":169955701,"data_compressed":10645383,"data_compressed_allocated":13836288,"data_compressed_original":167157691,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509465,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":48160,"kb_used_data":8652,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94323680,"statfs":{"total":96636764160,"available":96587448320,"internally_reserved":0,"allocated":8859648,"data_stored":78885735,"data_compressed":4733561,"data_compressed_allocated":6275072,"data_compressed_original":77274137,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":14,"seq":60129542169,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40768,"kb_used_data":5420,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94331072,"statfs":{"total":96636764160,"available":96595017728,"internally_reserved":0,"allocated":5550080,"data_stored":45569144,"data_compressed":2764337,"data_compressed_allocated":3653632,"data_compressed_original":44518123,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542168,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":63432,"kb_used_data":14196,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94308408,"statfs":{"total":96636764160,"available":96571809792,"internally_reserved":0,"allocated":14536704,"data_stored":127553606,"data_compressed":7784994,"data_compressed_allocated":10145792,"data_compressed_original":124425537,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574873,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":62856,"kb_used_data":13620,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94308984,"statfs":{"total":96636764160,"available":96572399616,"internally_reserved":0,"allocated":13946880,"data_stored":127613122,"data_compressed":7729700,"data_compressed_allocated":10256384,"data_compressed_original":125071338,"omap_allocated":1593,"internal_metadata":50395591},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32583,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32583,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":13934592,"data_stored":133839975,"data_compressed":8273664,"data_compressed_allocated":10739712,"data_compressed_original":131249473,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":7643136,"data_stored":78321406,"data_compressed":4733561,"data_compressed_allocated":6275072,"data_compressed_original":77274137,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4272128,"data_stored":44972439,"data_compressed":2764337,"data_compressed_allocated":3653632,"data_compressed_original":44518123,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12681216,"data_stored":127015130,"data_compressed":7729700,"data_compressed_allocated":10256384,"data_compressed_original":125071338,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":16445440,"data_stored":163065075,"data_compressed":10351992,"data_compressed_allocated":13447168,"data_compressed_original":160788411,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":10235904,"data_stored":96321719,"data_compressed":5914250,"data_compressed_allocated":7892992,"data_compressed_original":94499664,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":2760704,"data_stored":28128737,"data_compressed":1616596,"data_compressed_allocated":2199552,"data_compressed_original":27680295,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":10883072,"data_stored":104386296,"data_compressed":6311738,"data_compressed_allocated":8257536,"data_compressed_original":102270275,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-14T06:30:34.426 INFO:tasks.thrashosds.thrasher:Setting 3.1 to [4, 1] 2024-09-14T06:30:34.426 INFO:tasks.thrashosds.thrasher:cmd ['osd', 'pg-upmap', '3.1', '4', '1'] 2024-09-14T06:30:34.426 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pg-upmap 3.1 4 1 2024-09-14T06:30:34.427 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-14T06:30:34.428 INFO:teuthology.orchestra.run.smithi094.stdout:3423 read 0x5cfe676 thru 0x5d0a84c (0xc1d7 bytes) 2024-09-14T06:30:34.428 INFO:teuthology.orchestra.run.smithi094.stdout:3424 read 0x92ae7f3 thru 0x92b3f1f (0x572d bytes) 2024-09-14T06:30:34.429 INFO:teuthology.orchestra.run.smithi094.stdout:3426 writesame 0x2a95889 thru 0x2aa4d73 data_size 0x5b(0xf4eb bytes) 2024-09-14T06:30:34.433 INFO:teuthology.orchestra.run.smithi094.stdout:3427 write 0x98959f4 thru 0x98a06a3 (0xacb0 bytes) 2024-09-14T06:30:34.438 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-14T06:30:34.438 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-14T06:30:34.438 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-14T06:30:34.438 INFO:teuthology.orchestra.run.smithi094.stdout:3432 clone 25 order 21 su 8192 sc 14 2024-09-14T06:30:34.469 INFO:tasks.ceph.osd.4.smithi138.stderr:2024-09-14T06:30:34.467+0000 7f6c653c8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33615) UID: 0 2024-09-14T06:30:34.525 INFO:teuthology.orchestra.run.smithi138.stdout:ERROR: (22) Invalid argument 2024-09-14T06:30:34.525 INFO:teuthology.orchestra.run.smithi138.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:30:34.534 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-14T06:30:34.553 INFO:teuthology.orchestra.run.smithi094.stdout:osd.0: {} 2024-09-14T06:30:34.553 INFO:teuthology.orchestra.run.smithi094.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-14T06:30:34.569 INFO:teuthology.orchestra.run.smithi094.stdout:osd.1: {} 2024-09-14T06:30:34.569 INFO:teuthology.orchestra.run.smithi094.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-14T06:30:34.575 INFO:tasks.ceph.osd.5.smithi138.stderr:2024-09-14T06:30:34.574+0000 7fe13746f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33618) UID: 0 2024-09-14T06:30:34.585 INFO:teuthology.orchestra.run.smithi094.stdout:osd.2: {} 2024-09-14T06:30:34.586 INFO:teuthology.orchestra.run.smithi094.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-14T06:30:34.603 INFO:teuthology.orchestra.run.smithi094.stdout:osd.3: {} 2024-09-14T06:30:34.603 INFO:teuthology.orchestra.run.smithi094.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-14T06:30:34.619 INFO:teuthology.orchestra.run.smithi094.stdout:osd.4: {} 2024-09-14T06:30:34.619 INFO:teuthology.orchestra.run.smithi094.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-14T06:30:34.635 INFO:teuthology.orchestra.run.smithi094.stdout:osd.5: {} 2024-09-14T06:30:34.635 INFO:teuthology.orchestra.run.smithi094.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-14T06:30:34.652 INFO:teuthology.orchestra.run.smithi094.stdout:osd.6: {} 2024-09-14T06:30:34.652 INFO:teuthology.orchestra.run.smithi094.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-14T06:30:34.670 INFO:tasks.ceph.osd.7.smithi138.stderr:2024-09-14T06:30:34.669+0000 7f3c6a21d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33623) UID: 0 2024-09-14T06:30:34.671 INFO:teuthology.orchestra.run.smithi094.stdout:osd.7: {} 2024-09-14T06:30:34.671 INFO:teuthology.orchestra.run.smithi094.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-14T06:30:34.685 INFO:teuthology.orchestra.run.smithi138.stdout:{ 2024-09-14T06:30:34.685 INFO:teuthology.orchestra.run.smithi138.stdout: "ops": [ 2024-09-14T06:30:34.685 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:34.686 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:135059 3.10 3:0e8a5476:::rbd_header.112d173a9a99:head [create,call rbd.create in=59b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112)", 2024-09-14T06:30:34.686 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:30:34.684354+0000", 2024-09-14T06:30:34.686 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 0.00077664800000000001, 2024-09-14T06:30:34.686 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00079968599999999997, 2024-09-14T06:30:34.686 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:30:34.686 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:30:34.686 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "waiting for sub ops", 2024-09-14T06:30:34.686 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:30:34.686 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:30:34.686 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:34.686 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 135059 2024-09-14T06:30:34.686 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:34.686 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:30:34.687 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:34.687 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:30:34.687 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:34.684354+0000", 2024-09-14T06:30:34.687 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:34.687 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:34.687 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:34.687 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:30:34.687 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:34.684354+0000", 2024-09-14T06:30:34.687 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:34.687 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:34.687 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:34.687 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:30:34.687 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:34.684355+0000", 2024-09-14T06:30:34.688 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.159e-06 2024-09-14T06:30:34.688 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:34.688 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:34.688 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:30:34.688 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:34.684361+0000", 2024-09-14T06:30:34.688 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.614e-06 2024-09-14T06:30:34.688 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:34.688 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:34.688 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:30:34.688 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:34.684372+0000", 2024-09-14T06:30:34.688 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.1598e-05 2024-09-14T06:30:34.688 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:34.688 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:34.688 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:30:34.688 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:34.684378+0000", 2024-09-14T06:30:34.689 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.7880000000000003e-06 2024-09-14T06:30:34.689 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:34.689 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:34.689 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:30:34.689 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:34.684431+0000", 2024-09-14T06:30:34.689 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.3393999999999997e-05 2024-09-14T06:30:34.689 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:34.689 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:34.689 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:30:34.689 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:34.684521+0000", 2024-09-14T06:30:34.689 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.9895999999999994e-05 2024-09-14T06:30:34.689 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:34.689 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:34.690 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "waiting for subops from 0", 2024-09-14T06:30:34.690 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:34.684660+0000", 2024-09-14T06:30:34.690 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00013854899999999999 2024-09-14T06:30:34.690 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:34.690 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:30:34.690 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:34.690 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:34.690 INFO:teuthology.orchestra.run.smithi138.stdout: ], 2024-09-14T06:30:34.690 INFO:teuthology.orchestra.run.smithi138.stdout: "num_ops": 1 2024-09-14T06:30:34.690 INFO:teuthology.orchestra.run.smithi138.stdout:} 2024-09-14T06:30:34.695 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-14T06:30:34.779 INFO:tasks.ceph.osd.4.smithi138.stderr:2024-09-14T06:30:34.778+0000 7f6c653c8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33615) UID: 0 2024-09-14T06:30:34.839 INFO:teuthology.orchestra.run.smithi138.stdout:{ 2024-09-14T06:30:34.839 INFO:teuthology.orchestra.run.smithi138.stdout: "ops": [], 2024-09-14T06:30:34.839 INFO:teuthology.orchestra.run.smithi138.stdout: "complaint_time": 30, 2024-09-14T06:30:34.839 INFO:teuthology.orchestra.run.smithi138.stdout: "num_blocked_ops": 0 2024-09-14T06:30:34.839 INFO:teuthology.orchestra.run.smithi138.stdout:} 2024-09-14T06:30:34.847 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-14T06:30:34.872 INFO:tasks.ceph.osd.1.smithi094.stderr:2024-09-14T06:30:34.871+0000 7f328e47a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33641) UID: 0 2024-09-14T06:30:34.973 INFO:tasks.ceph.osd.3.smithi094.stderr:2024-09-14T06:30:34.971+0000 7feeeafe8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33655) UID: 0 2024-09-14T06:30:34.991 INFO:teuthology.orchestra.run.smithi138.stdout:{ 2024-09-14T06:30:34.991 INFO:teuthology.orchestra.run.smithi138.stdout: "size": 20, 2024-09-14T06:30:34.991 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 600, 2024-09-14T06:30:34.991 INFO:teuthology.orchestra.run.smithi138.stdout: "ops": [ 2024-09-14T06:30:34.992 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:34.992 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:98505 3.10 3.1bf56f90 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T06:30:34.992 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:30:07.045453+0000", 2024-09-14T06:30:34.992 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 27.944238447, 2024-09-14T06:30:34.992 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032144440000000003, 2024-09-14T06:30:34.992 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:30:34.992 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:30:34.992 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "reached pg", 2024-09-14T06:30:34.992 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:30:34.992 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:30:34.992 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:34.992 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 98505 2024-09-14T06:30:34.993 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:34.993 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:30:34.993 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:34.993 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:30:34.993 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045453+0000", 2024-09-14T06:30:34.993 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:34.993 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:34.993 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:34.993 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:30:34.993 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045453+0000", 2024-09-14T06:30:34.993 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:34.993 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:34.993 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:34.993 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:30:34.994 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045453+0000", 2024-09-14T06:30:34.994 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.36e-07 2024-09-14T06:30:34.994 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:34.994 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:34.994 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:30:34.994 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045454+0000", 2024-09-14T06:30:34.994 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.1299999999999998e-07 2024-09-14T06:30:34.994 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:34.994 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:34.994 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:30:34.994 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045457+0000", 2024-09-14T06:30:34.994 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.8559999999999998e-06 2024-09-14T06:30:34.994 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:34.994 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:34.995 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:30:34.995 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045460+0000", 2024-09-14T06:30:34.995 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.8750000000000002e-06 2024-09-14T06:30:34.995 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:34.995 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:34.995 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:30:34.995 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.077592+0000", 2024-09-14T06:30:34.995 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032131727999999998 2024-09-14T06:30:34.995 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:34.995 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:34.995 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:30:34.995 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.077597+0000", 2024-09-14T06:30:34.995 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.8319999999999996e-06 2024-09-14T06:30:34.995 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:34.995 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:30:34.996 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:34.996 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:34.996 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:34.996 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:98506 3.10 3.1bf56f90 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T06:30:34.996 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:30:07.045481+0000", 2024-09-14T06:30:34.996 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 27.944210192, 2024-09-14T06:30:34.996 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032142759999999999, 2024-09-14T06:30:34.996 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:30:34.996 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:30:34.996 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "reached pg", 2024-09-14T06:30:34.996 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:30:34.996 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:30:34.996 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:34.996 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 98506 2024-09-14T06:30:34.997 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:34.997 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:30:34.997 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:34.997 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:30:34.997 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045481+0000", 2024-09-14T06:30:34.997 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:34.997 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:34.997 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:34.997 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:30:34.997 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045481+0000", 2024-09-14T06:30:34.997 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:34.997 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:34.997 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:34.997 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:30:34.998 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045481+0000", 2024-09-14T06:30:34.998 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.8500000000000001e-07 2024-09-14T06:30:34.998 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:34.998 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:34.998 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:30:34.998 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045483+0000", 2024-09-14T06:30:34.998 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.8250000000000001e-06 2024-09-14T06:30:34.998 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:34.998 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:34.998 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:30:34.998 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045489+0000", 2024-09-14T06:30:34.998 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.5740000000000003e-06 2024-09-14T06:30:34.998 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:34.998 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:34.998 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:30:34.999 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045495+0000", 2024-09-14T06:30:34.999 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.7690000000000002e-06 2024-09-14T06:30:34.999 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:34.999 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:34.999 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:30:34.999 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.077619+0000", 2024-09-14T06:30:34.999 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032124621999999999 2024-09-14T06:30:34.999 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:34.999 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:34.999 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:30:34.999 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.077624+0000", 2024-09-14T06:30:34.999 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.3850000000000004e-06 2024-09-14T06:30:34.999 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:34.999 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:30:35.000 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:35.000 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:35.000 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:35.000 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:98507 3.10 3.1bf56f90 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T06:30:35.000 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:30:07.045502+0000", 2024-09-14T06:30:35.000 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 27.944188771, 2024-09-14T06:30:35.000 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032147219999999997, 2024-09-14T06:30:35.000 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:30:35.000 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:30:35.000 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "reached pg", 2024-09-14T06:30:35.000 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:30:35.000 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:30:35.000 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:35.000 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 98507 2024-09-14T06:30:35.001 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:35.001 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:30:35.001 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:35.001 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:30:35.001 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045502+0000", 2024-09-14T06:30:35.001 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:35.001 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:35.001 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:35.001 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:30:35.001 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045502+0000", 2024-09-14T06:30:35.001 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:35.001 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:35.001 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:35.001 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:30:35.001 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045503+0000", 2024-09-14T06:30:35.002 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.4700000000000002e-07 2024-09-14T06:30:35.002 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:35.002 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:35.002 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:30:35.002 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045509+0000", 2024-09-14T06:30:35.002 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.7329999999999997e-06 2024-09-14T06:30:35.002 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:35.002 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:35.002 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:30:35.002 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045515+0000", 2024-09-14T06:30:35.002 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.8030000000000002e-06 2024-09-14T06:30:35.002 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:35.002 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:35.003 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:30:35.003 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045522+0000", 2024-09-14T06:30:35.003 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.0369999999999997e-06 2024-09-14T06:30:35.003 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:35.003 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:35.003 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:30:35.003 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.077641+0000", 2024-09-14T06:30:35.003 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032119091000000002 2024-09-14T06:30:35.003 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:35.003 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:35.003 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:30:35.003 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.077650+0000", 2024-09-14T06:30:35.003 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.2090000000000005e-06 2024-09-14T06:30:35.004 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:35.004 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:30:35.004 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:35.004 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:35.004 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:35.004 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:98508 3.10 3.1bf56f90 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T06:30:35.004 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:30:07.045530+0000", 2024-09-14T06:30:35.004 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 27.944161154, 2024-09-14T06:30:35.004 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032198773999999999, 2024-09-14T06:30:35.004 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:30:35.004 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:30:35.004 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "reached pg", 2024-09-14T06:30:35.004 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:30:35.005 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:30:35.005 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:35.005 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 98508 2024-09-14T06:30:35.005 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:35.005 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:30:35.005 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:35.005 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:30:35.005 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045530+0000", 2024-09-14T06:30:35.005 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:35.005 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:35.005 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:35.005 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:30:35.005 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045530+0000", 2024-09-14T06:30:35.006 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:35.006 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:35.006 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:35.006 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:30:35.006 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045530+0000", 2024-09-14T06:30:35.006 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.3099999999999998e-07 2024-09-14T06:30:35.006 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:35.006 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:35.006 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:30:35.006 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045539+0000", 2024-09-14T06:30:35.006 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.4780000000000008e-06 2024-09-14T06:30:35.007 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:35.007 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:35.007 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:30:35.007 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045546+0000", 2024-09-14T06:30:35.007 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.3259999999999998e-06 2024-09-14T06:30:35.007 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:35.007 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:35.007 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:30:35.007 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045551+0000", 2024-09-14T06:30:35.007 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.9960000000000001e-06 2024-09-14T06:30:35.007 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:35.007 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:35.007 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:30:35.008 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.077716+0000", 2024-09-14T06:30:35.008 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032165081999999998 2024-09-14T06:30:35.008 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:35.008 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:35.008 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:30:35.008 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.077729+0000", 2024-09-14T06:30:35.008 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.2461e-05 2024-09-14T06:30:35.008 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:35.008 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:30:35.008 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:35.008 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:35.008 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:35.008 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:98509 3.10 3.1bf56f90 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T06:30:35.008 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:30:07.045558+0000", 2024-09-14T06:30:35.009 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 27.944132953, 2024-09-14T06:30:35.009 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032242198, 2024-09-14T06:30:35.009 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:30:35.009 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:30:35.009 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "reached pg", 2024-09-14T06:30:35.009 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:30:35.009 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:30:35.009 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:35.009 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 98509 2024-09-14T06:30:35.009 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:35.009 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:30:35.009 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:35.009 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:30:35.009 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045558+0000", 2024-09-14T06:30:35.010 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:35.010 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:35.010 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:35.010 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:30:35.010 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045558+0000", 2024-09-14T06:30:35.010 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:35.010 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:35.010 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:35.010 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:30:35.010 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045559+0000", 2024-09-14T06:30:35.010 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.27e-07 2024-09-14T06:30:35.010 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:35.010 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:35.010 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:30:35.010 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045560+0000", 2024-09-14T06:30:35.011 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.6959999999999999e-06 2024-09-14T06:30:35.011 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:35.011 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:35.011 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:30:35.011 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045567+0000", 2024-09-14T06:30:35.011 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.6900000000000003e-06 2024-09-14T06:30:35.011 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:35.011 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:35.011 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:30:35.011 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045571+0000", 2024-09-14T06:30:35.011 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.5750000000000002e-06 2024-09-14T06:30:35.011 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:35.011 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:35.011 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:30:35.011 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.077789+0000", 2024-09-14T06:30:35.011 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032217186000000002 2024-09-14T06:30:35.012 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:35.012 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:35.012 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:30:35.012 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.077800+0000", 2024-09-14T06:30:35.012 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.1623999999999999e-05 2024-09-14T06:30:35.012 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:35.012 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:30:35.012 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:35.012 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:35.012 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:35.012 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:98510 3.10 3.1bf56f90 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T06:30:35.012 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:30:07.045580+0000", 2024-09-14T06:30:35.012 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 27.944111139, 2024-09-14T06:30:35.012 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032299845000000001, 2024-09-14T06:30:35.013 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:30:35.013 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:30:35.013 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "reached pg", 2024-09-14T06:30:35.013 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:30:35.013 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:30:35.013 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:35.013 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 98510 2024-09-14T06:30:35.013 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:35.013 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:30:35.013 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:35.013 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:30:35.013 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045580+0000", 2024-09-14T06:30:35.013 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:35.013 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:35.013 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:35.014 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:30:35.014 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045580+0000", 2024-09-14T06:30:35.014 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:35.014 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:35.014 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:35.014 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:30:35.014 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045580+0000", 2024-09-14T06:30:35.014 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.0400000000000002e-07 2024-09-14T06:30:35.014 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:35.014 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:35.014 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:30:35.014 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045582+0000", 2024-09-14T06:30:35.014 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.7600000000000001e-06 2024-09-14T06:30:35.014 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:35.014 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:35.015 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:30:35.015 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045588+0000", 2024-09-14T06:30:35.015 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.5650000000000001e-06 2024-09-14T06:30:35.015 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:35.015 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:35.015 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:30:35.015 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045597+0000", 2024-09-14T06:30:35.015 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.465e-06 2024-09-14T06:30:35.015 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:35.015 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:35.015 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:30:35.015 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.077871+0000", 2024-09-14T06:30:35.015 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032273562999999998 2024-09-14T06:30:35.015 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:35.016 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:35.016 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:30:35.016 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.077880+0000", 2024-09-14T06:30:35.016 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.0880000000000001e-06 2024-09-14T06:30:35.016 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:35.016 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:30:35.016 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:35.016 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:35.016 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:35.016 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:98511 3.10 3.1bf56f90 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T06:30:35.016 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:30:07.045610+0000", 2024-09-14T06:30:35.016 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 27.944081292, 2024-09-14T06:30:35.016 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032338894, 2024-09-14T06:30:35.016 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:30:35.017 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:30:35.017 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "reached pg", 2024-09-14T06:30:35.017 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:30:35.017 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:30:35.017 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:35.017 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 98511 2024-09-14T06:30:35.017 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:35.017 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:30:35.017 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:35.017 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:30:35.017 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045610+0000", 2024-09-14T06:30:35.017 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:35.017 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:35.017 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:35.018 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:30:35.018 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045610+0000", 2024-09-14T06:30:35.018 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:35.018 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:35.018 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:35.018 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:30:35.018 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045610+0000", 2024-09-14T06:30:35.018 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.1600000000000002e-07 2024-09-14T06:30:35.018 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:35.018 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:35.018 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:30:35.018 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045612+0000", 2024-09-14T06:30:35.018 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.1150000000000001e-06 2024-09-14T06:30:35.018 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:35.018 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:35.019 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:30:35.019 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045618+0000", 2024-09-14T06:30:35.019 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.0499999999999997e-06 2024-09-14T06:30:35.019 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:35.019 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:35.019 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:30:35.019 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045623+0000", 2024-09-14T06:30:35.019 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.6770000000000003e-06 2024-09-14T06:30:35.019 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:35.020 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:35.020 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:30:35.020 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.077928+0000", 2024-09-14T06:30:35.021 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032305111999999997 2024-09-14T06:30:35.021 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:35.021 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:35.021 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:30:35.021 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.077949+0000", 2024-09-14T06:30:35.021 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.0523999999999999e-05 2024-09-14T06:30:35.021 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:35.021 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:30:35.021 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:35.021 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:35.021 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:35.021 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:98512 3.10 3.1bf56f90 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T06:30:35.022 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:30:07.045630+0000", 2024-09-14T06:30:35.022 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 27.944061287, 2024-09-14T06:30:35.022 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032346632, 2024-09-14T06:30:35.022 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:30:35.022 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:30:35.022 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "reached pg", 2024-09-14T06:30:35.022 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:30:35.022 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:30:35.022 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:35.022 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 98512 2024-09-14T06:30:35.022 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:35.022 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:30:35.022 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:35.022 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:30:35.023 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045630+0000", 2024-09-14T06:30:35.023 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:35.023 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:35.023 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:35.023 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:30:35.023 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045630+0000", 2024-09-14T06:30:35.023 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:35.023 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:35.023 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:35.023 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:30:35.023 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045630+0000", 2024-09-14T06:30:35.023 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.8200000000000001e-07 2024-09-14T06:30:35.023 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:35.023 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:35.024 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:30:35.024 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045632+0000", 2024-09-14T06:30:35.024 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.7430000000000001e-06 2024-09-14T06:30:35.024 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:35.024 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:35.024 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:30:35.024 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045637+0000", 2024-09-14T06:30:35.024 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.4550000000000003e-06 2024-09-14T06:30:35.024 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:35.024 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:35.024 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:30:35.024 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045644+0000", 2024-09-14T06:30:35.024 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.1330000000000001e-06 2024-09-14T06:30:35.025 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:35.025 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:35.025 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:30:35.025 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.077968+0000", 2024-09-14T06:30:35.025 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032323401000000002 2024-09-14T06:30:35.025 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:35.025 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:35.025 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:30:35.025 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.077976+0000", 2024-09-14T06:30:35.025 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.5180000000000004e-06 2024-09-14T06:30:35.025 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:35.025 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:30:35.025 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:35.026 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:35.026 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:35.026 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:98513 3.10 3.1bf56f90 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T06:30:35.026 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:30:07.045652+0000", 2024-09-14T06:30:35.026 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 27.944038722999998, 2024-09-14T06:30:35.026 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032353436999999999, 2024-09-14T06:30:35.026 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:30:35.026 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:30:35.026 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "reached pg", 2024-09-14T06:30:35.026 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:30:35.026 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:30:35.026 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:35.026 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 98513 2024-09-14T06:30:35.026 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:35.027 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:30:35.027 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:35.027 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:30:35.027 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045652+0000", 2024-09-14T06:30:35.027 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:35.027 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:35.027 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:35.027 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:30:35.027 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045652+0000", 2024-09-14T06:30:35.027 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:35.027 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:35.028 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:35.028 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:30:35.028 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045653+0000", 2024-09-14T06:30:35.028 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.3700000000000001e-07 2024-09-14T06:30:35.028 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:35.028 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:35.028 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:30:35.028 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045655+0000", 2024-09-14T06:30:35.028 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.854e-06 2024-09-14T06:30:35.028 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:35.028 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:35.028 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:30:35.029 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045661+0000", 2024-09-14T06:30:35.029 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.5509999999999996e-06 2024-09-14T06:30:35.029 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:35.029 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:35.029 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:30:35.029 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045668+0000", 2024-09-14T06:30:35.029 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.1980000000000004e-06 2024-09-14T06:30:35.029 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:35.029 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:35.029 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:30:35.029 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.077997+0000", 2024-09-14T06:30:35.029 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032328575999999998 2024-09-14T06:30:35.029 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:35.029 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:35.030 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:30:35.030 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.078006+0000", 2024-09-14T06:30:35.030 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.8210000000000008e-06 2024-09-14T06:30:35.030 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:35.030 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:30:35.030 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:35.030 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:35.030 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:35.030 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:98514 3.10 3.1bf56f90 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T06:30:35.030 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:30:07.045699+0000", 2024-09-14T06:30:35.030 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 27.943992388000002, 2024-09-14T06:30:35.030 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032330223999999998, 2024-09-14T06:30:35.030 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:30:35.030 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:30:35.031 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "reached pg", 2024-09-14T06:30:35.031 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:30:35.031 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:30:35.031 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:35.031 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 98514 2024-09-14T06:30:35.031 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:35.031 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:30:35.031 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:35.031 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:30:35.031 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045699+0000", 2024-09-14T06:30:35.031 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:35.031 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:35.031 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:35.032 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:30:35.032 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045699+0000", 2024-09-14T06:30:35.032 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:35.032 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:35.032 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:35.032 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:30:35.032 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045699+0000", 2024-09-14T06:30:35.032 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.3200000000000001e-07 2024-09-14T06:30:35.032 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:35.032 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:35.032 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:30:35.032 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045701+0000", 2024-09-14T06:30:35.032 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.762e-06 2024-09-14T06:30:35.032 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:35.033 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:35.033 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:30:35.033 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045705+0000", 2024-09-14T06:30:35.033 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.2359999999999998e-06 2024-09-14T06:30:35.033 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:35.033 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:35.033 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:30:35.033 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045708+0000", 2024-09-14T06:30:35.033 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.796e-06 2024-09-14T06:30:35.033 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:35.033 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:35.033 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:30:35.033 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.078023+0000", 2024-09-14T06:30:35.033 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032315213000000002 2024-09-14T06:30:35.034 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:35.034 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:35.034 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:30:35.034 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.078029+0000", 2024-09-14T06:30:35.034 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.8850000000000004e-06 2024-09-14T06:30:35.034 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:35.034 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:30:35.034 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:35.034 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:35.034 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:35.034 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:98515 3.10 3.1bf56f90 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T06:30:35.034 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:30:07.045712+0000", 2024-09-14T06:30:35.034 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 27.943979077000002, 2024-09-14T06:30:35.035 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032346871999999999, 2024-09-14T06:30:35.035 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:30:35.035 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:30:35.035 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "reached pg", 2024-09-14T06:30:35.035 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:30:35.035 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:30:35.035 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:35.035 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 98515 2024-09-14T06:30:35.035 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:35.035 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:30:35.035 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:35.035 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:30:35.036 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045712+0000", 2024-09-14T06:30:35.036 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:35.036 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:35.036 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:35.036 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:30:35.036 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045712+0000", 2024-09-14T06:30:35.036 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:35.036 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:35.036 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:35.036 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:30:35.036 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045712+0000", 2024-09-14T06:30:35.036 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.2100000000000001e-07 2024-09-14T06:30:35.037 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:35.037 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:35.037 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:30:35.037 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045713+0000", 2024-09-14T06:30:35.037 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.068e-06 2024-09-14T06:30:35.037 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:35.037 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:35.037 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:30:35.037 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045716+0000", 2024-09-14T06:30:35.037 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.971e-06 2024-09-14T06:30:35.037 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:35.037 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:35.037 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:30:35.038 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045719+0000", 2024-09-14T06:30:35.038 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.3879999999999998e-06 2024-09-14T06:30:35.038 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:35.038 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:35.038 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:30:35.038 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.078049+0000", 2024-09-14T06:30:35.038 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032330553999999997 2024-09-14T06:30:35.038 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:35.038 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:35.038 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:30:35.038 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.078059+0000", 2024-09-14T06:30:35.038 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.6700000000000006e-06 2024-09-14T06:30:35.038 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:35.038 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:30:35.039 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:35.039 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:35.039 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:35.039 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:98516 3.10 3.1bf56f90 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T06:30:35.039 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:30:07.045722+0000", 2024-09-14T06:30:35.039 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 27.943969324000001, 2024-09-14T06:30:35.039 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032374334999999997, 2024-09-14T06:30:35.039 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:30:35.039 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:30:35.039 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "reached pg", 2024-09-14T06:30:35.039 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:30:35.039 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:30:35.039 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:35.040 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 98516 2024-09-14T06:30:35.040 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:35.040 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:30:35.040 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:35.040 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:30:35.040 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045722+0000", 2024-09-14T06:30:35.040 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:35.040 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:35.040 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:35.040 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:30:35.040 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045722+0000", 2024-09-14T06:30:35.040 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:35.040 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:35.040 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:35.041 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:30:35.041 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045722+0000", 2024-09-14T06:30:35.041 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.8999999999999998e-07 2024-09-14T06:30:35.041 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:35.041 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:35.041 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:30:35.041 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045723+0000", 2024-09-14T06:30:35.041 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.0049999999999999e-06 2024-09-14T06:30:35.041 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:35.041 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:35.041 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:30:35.041 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045726+0000", 2024-09-14T06:30:35.041 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.9019999999999999e-06 2024-09-14T06:30:35.041 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:35.042 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:35.042 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:30:35.042 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045728+0000", 2024-09-14T06:30:35.042 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.3290000000000001e-06 2024-09-14T06:30:35.042 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:35.042 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:35.042 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:30:35.042 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.078090+0000", 2024-09-14T06:30:35.042 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032362123 2024-09-14T06:30:35.042 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:35.042 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:35.042 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:30:35.042 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.078096+0000", 2024-09-14T06:30:35.042 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.6860000000000003e-06 2024-09-14T06:30:35.043 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:35.043 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:30:35.043 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:35.043 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:35.043 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:35.043 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:98517 3.10 3.1bf56f90 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T06:30:35.043 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:30:07.045731+0000", 2024-09-14T06:30:35.043 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 27.943959592999999, 2024-09-14T06:30:35.043 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.03241227, 2024-09-14T06:30:35.043 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:30:35.043 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:30:35.043 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "reached pg", 2024-09-14T06:30:35.043 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:30:35.043 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:30:35.044 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:35.044 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 98517 2024-09-14T06:30:35.044 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:35.044 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:30:35.044 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:35.044 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:30:35.044 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045731+0000", 2024-09-14T06:30:35.044 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:35.044 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:35.044 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:35.044 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:30:35.044 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045731+0000", 2024-09-14T06:30:35.044 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:35.044 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:35.045 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:35.045 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:30:35.045 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045732+0000", 2024-09-14T06:30:35.045 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.0699999999999999e-07 2024-09-14T06:30:35.045 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:35.045 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:35.045 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:30:35.045 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045733+0000", 2024-09-14T06:30:35.045 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.9199999999999999e-07 2024-09-14T06:30:35.045 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:35.045 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:35.045 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:30:35.045 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045735+0000", 2024-09-14T06:30:35.045 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.8150000000000002e-06 2024-09-14T06:30:35.046 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:35.046 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:35.046 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:30:35.046 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045738+0000", 2024-09-14T06:30:35.046 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.3640000000000002e-06 2024-09-14T06:30:35.046 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:35.046 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:35.046 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:30:35.046 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.078126+0000", 2024-09-14T06:30:35.046 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032388206000000003 2024-09-14T06:30:35.046 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:35.046 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:35.046 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:30:35.046 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.078144+0000", 2024-09-14T06:30:35.047 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.7686e-05 2024-09-14T06:30:35.047 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:35.047 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:30:35.047 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:35.047 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:35.047 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:35.047 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:98518 3.10 3.1bf56f90 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T06:30:35.047 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:30:07.045741+0000", 2024-09-14T06:30:35.047 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 27.943950198, 2024-09-14T06:30:35.047 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032451793999999999, 2024-09-14T06:30:35.047 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:30:35.047 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:30:35.047 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "reached pg", 2024-09-14T06:30:35.047 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:30:35.048 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:30:35.048 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:35.048 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 98518 2024-09-14T06:30:35.048 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:35.048 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:30:35.048 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:35.048 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:30:35.048 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045741+0000", 2024-09-14T06:30:35.048 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:35.048 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:35.048 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:35.048 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:30:35.048 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045741+0000", 2024-09-14T06:30:35.048 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:35.049 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:35.049 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:35.049 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:30:35.049 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045741+0000", 2024-09-14T06:30:35.049 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.05e-07 2024-09-14T06:30:35.049 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:35.049 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:35.049 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:30:35.049 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045746+0000", 2024-09-14T06:30:35.049 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.2259999999999996e-06 2024-09-14T06:30:35.049 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:35.049 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:35.049 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:30:35.049 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045750+0000", 2024-09-14T06:30:35.050 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.2490000000000001e-06 2024-09-14T06:30:35.050 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:35.050 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:35.050 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:30:35.050 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045752+0000", 2024-09-14T06:30:35.050 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.3980000000000002e-06 2024-09-14T06:30:35.050 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:35.050 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:35.050 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:30:35.050 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.078184+0000", 2024-09-14T06:30:35.050 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032431593000000002 2024-09-14T06:30:35.050 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:35.050 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:35.050 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:30:35.051 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.078193+0000", 2024-09-14T06:30:35.051 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.1230000000000006e-06 2024-09-14T06:30:35.051 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:35.051 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:30:35.051 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:35.051 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:35.051 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:35.051 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:98519 3.10 3.1bf56f90 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T06:30:35.051 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:30:07.045755+0000", 2024-09-14T06:30:35.051 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 27.943935773, 2024-09-14T06:30:35.051 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032489431999999999, 2024-09-14T06:30:35.051 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:30:35.051 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:30:35.051 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "reached pg", 2024-09-14T06:30:35.052 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:30:35.052 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:30:35.052 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:35.052 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 98519 2024-09-14T06:30:35.052 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:35.052 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:30:35.052 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:35.052 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:30:35.052 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045755+0000", 2024-09-14T06:30:35.052 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:35.052 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:35.052 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:35.052 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:30:35.053 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045755+0000", 2024-09-14T06:30:35.053 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:35.053 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:35.053 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:35.053 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:30:35.053 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045756+0000", 2024-09-14T06:30:35.053 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.23e-07 2024-09-14T06:30:35.053 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:35.053 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:35.053 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:30:35.053 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045757+0000", 2024-09-14T06:30:35.053 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.1060000000000001e-06 2024-09-14T06:30:35.053 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:35.053 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:35.054 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:30:35.054 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045760+0000", 2024-09-14T06:30:35.054 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.988e-06 2024-09-14T06:30:35.054 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:35.054 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:35.054 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:30:35.054 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045765+0000", 2024-09-14T06:30:35.054 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.2689999999999999e-06 2024-09-14T06:30:35.054 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:35.054 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:35.054 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:30:35.054 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.078234+0000", 2024-09-14T06:30:35.054 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032468915000000001 2024-09-14T06:30:35.055 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:35.055 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:35.055 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:30:35.055 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.078245+0000", 2024-09-14T06:30:35.055 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.0930999999999999e-05 2024-09-14T06:30:35.055 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:35.055 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:30:35.055 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:35.055 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:35.055 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:35.055 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:98520 3.10 3.1bf56f90 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T06:30:35.055 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:30:07.045768+0000", 2024-09-14T06:30:35.055 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 27.943922895, 2024-09-14T06:30:35.055 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032531379999999999, 2024-09-14T06:30:35.056 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:30:35.056 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:30:35.056 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "reached pg", 2024-09-14T06:30:35.056 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:30:35.056 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:30:35.056 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:35.056 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 98520 2024-09-14T06:30:35.056 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:35.056 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:30:35.056 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:35.056 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:30:35.056 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045768+0000", 2024-09-14T06:30:35.056 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:35.056 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:35.057 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:35.057 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:30:35.057 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045768+0000", 2024-09-14T06:30:35.057 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:35.057 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:35.057 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:35.057 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:30:35.057 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045768+0000", 2024-09-14T06:30:35.057 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.22e-07 2024-09-14T06:30:35.057 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:35.057 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:35.057 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:30:35.057 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045773+0000", 2024-09-14T06:30:35.058 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.3470000000000001e-06 2024-09-14T06:30:35.058 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:35.058 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:35.058 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:30:35.058 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045776+0000", 2024-09-14T06:30:35.058 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.1319999999999998e-06 2024-09-14T06:30:35.058 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:35.058 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:35.058 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:30:35.058 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045779+0000", 2024-09-14T06:30:35.058 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.9749999999999999e-06 2024-09-14T06:30:35.058 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:35.058 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:35.058 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:30:35.059 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.078289+0000", 2024-09-14T06:30:35.059 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032509693999999999 2024-09-14T06:30:35.059 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:35.059 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:35.059 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:30:35.059 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.078300+0000", 2024-09-14T06:30:35.059 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.101e-05 2024-09-14T06:30:35.059 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:35.059 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:30:35.059 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:35.059 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:35.059 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:35.059 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:98521 3.10 3.1bf56f90 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T06:30:35.060 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:30:07.045782+0000", 2024-09-14T06:30:35.060 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 27.943908714999999, 2024-09-14T06:30:35.060 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032566109000000003, 2024-09-14T06:30:35.060 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:30:35.060 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:30:35.060 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "reached pg", 2024-09-14T06:30:35.060 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:30:35.060 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:30:35.060 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:35.060 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 98521 2024-09-14T06:30:35.060 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:35.060 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:30:35.060 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:35.060 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:30:35.061 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045782+0000", 2024-09-14T06:30:35.061 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:35.061 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:35.061 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:35.061 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:30:35.061 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045782+0000", 2024-09-14T06:30:35.061 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:35.061 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:35.061 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:35.061 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:30:35.061 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045783+0000", 2024-09-14T06:30:35.061 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.2499999999999999e-07 2024-09-14T06:30:35.061 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:35.062 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:35.062 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:30:35.062 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045784+0000", 2024-09-14T06:30:35.062 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.0109999999999999e-06 2024-09-14T06:30:35.062 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:35.062 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:35.062 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:30:35.062 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045786+0000", 2024-09-14T06:30:35.062 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.8289999999999999e-06 2024-09-14T06:30:35.062 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:35.062 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:35.062 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:30:35.062 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045793+0000", 2024-09-14T06:30:35.063 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.579e-06 2024-09-14T06:30:35.063 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:35.063 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:35.063 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:30:35.063 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.078338+0000", 2024-09-14T06:30:35.063 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032545389000000001 2024-09-14T06:30:35.063 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:35.063 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:35.063 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:30:35.063 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.078348+0000", 2024-09-14T06:30:35.063 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.0076e-05 2024-09-14T06:30:35.063 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:35.063 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:30:35.064 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:35.064 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:35.064 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:35.064 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:98522 3.10 3.1bf56f90 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T06:30:35.064 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:30:07.045796+0000", 2024-09-14T06:30:35.064 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 27.943894807, 2024-09-14T06:30:35.064 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032577846000000001, 2024-09-14T06:30:35.064 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:30:35.064 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:30:35.064 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "reached pg", 2024-09-14T06:30:35.064 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:30:35.064 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:30:35.065 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:35.065 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 98522 2024-09-14T06:30:35.065 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:35.065 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:30:35.065 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:35.065 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:30:35.065 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045796+0000", 2024-09-14T06:30:35.065 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:35.065 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:35.065 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:35.065 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:30:35.065 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045796+0000", 2024-09-14T06:30:35.065 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:35.066 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:35.066 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:35.066 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:30:35.066 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045796+0000", 2024-09-14T06:30:35.066 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.2100000000000001e-07 2024-09-14T06:30:35.066 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:35.066 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:35.066 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:30:35.066 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045797+0000", 2024-09-14T06:30:35.066 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.6899999999999996e-07 2024-09-14T06:30:35.066 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:35.066 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:35.067 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:30:35.067 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045800+0000", 2024-09-14T06:30:35.067 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.9979999999999999e-06 2024-09-14T06:30:35.067 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:35.067 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:35.067 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:30:35.067 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045803+0000", 2024-09-14T06:30:35.067 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.2850000000000001e-06 2024-09-14T06:30:35.067 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:35.067 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:35.067 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:30:35.067 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.078366+0000", 2024-09-14T06:30:35.067 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032562932000000003 2024-09-14T06:30:35.068 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:35.068 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:35.068 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:30:35.068 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.078374+0000", 2024-09-14T06:30:35.068 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.4409999999999993e-06 2024-09-14T06:30:35.068 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:35.068 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:30:35.068 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:35.068 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:35.068 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:35.068 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:98523 3.10 3.1bf56f90 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T06:30:35.068 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:30:07.045806+0000", 2024-09-14T06:30:35.069 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 27.943885263999999, 2024-09-14T06:30:35.069 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032611167000000003, 2024-09-14T06:30:35.069 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:30:35.069 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:30:35.069 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "reached pg", 2024-09-14T06:30:35.069 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:30:35.069 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:30:35.069 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:35.069 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 98523 2024-09-14T06:30:35.069 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:35.070 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:30:35.070 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:35.070 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:30:35.070 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045806+0000", 2024-09-14T06:30:35.070 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:35.070 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:35.070 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:35.070 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:30:35.070 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045806+0000", 2024-09-14T06:30:35.070 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:35.070 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:35.070 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:35.070 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:30:35.070 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045806+0000", 2024-09-14T06:30:35.071 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.2000000000000001e-07 2024-09-14T06:30:35.071 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:35.071 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:35.071 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:30:35.071 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045807+0000", 2024-09-14T06:30:35.071 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.4e-07 2024-09-14T06:30:35.071 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:35.071 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:35.071 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:30:35.071 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045810+0000", 2024-09-14T06:30:35.071 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.9229999999999998e-06 2024-09-14T06:30:35.071 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:35.071 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:35.071 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:30:35.072 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045812+0000", 2024-09-14T06:30:35.072 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.2220000000000001e-06 2024-09-14T06:30:35.072 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:35.072 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:35.072 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:30:35.072 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.078407+0000", 2024-09-14T06:30:35.072 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032594513999999998 2024-09-14T06:30:35.072 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:35.072 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:35.072 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:30:35.072 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.078417+0000", 2024-09-14T06:30:35.072 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.0348e-05 2024-09-14T06:30:35.072 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:35.072 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:30:35.073 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:35.073 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:35.073 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:35.073 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:98524 3.10 3.1bf56f90 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T06:30:35.073 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:30:07.045815+0000", 2024-09-14T06:30:35.073 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 27.943875896000002, 2024-09-14T06:30:35.073 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032643208999999999, 2024-09-14T06:30:35.073 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:30:35.073 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:30:35.073 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "reached pg", 2024-09-14T06:30:35.073 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:30:35.073 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:30:35.073 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:35.074 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 98524 2024-09-14T06:30:35.074 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:35.074 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:30:35.074 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:35.074 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:30:35.074 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045815+0000", 2024-09-14T06:30:35.074 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:35.074 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:35.074 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:35.074 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:30:35.074 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045815+0000", 2024-09-14T06:30:35.074 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:35.074 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:35.074 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:35.075 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:30:35.075 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045815+0000", 2024-09-14T06:30:35.075 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.11e-07 2024-09-14T06:30:35.075 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:35.075 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:35.075 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:30:35.075 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045816+0000", 2024-09-14T06:30:35.075 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.9099999999999991e-07 2024-09-14T06:30:35.075 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:35.075 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:35.075 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:30:35.075 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045819+0000", 2024-09-14T06:30:35.075 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.9399999999999998e-06 2024-09-14T06:30:35.075 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:35.076 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:35.076 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:30:35.076 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045822+0000", 2024-09-14T06:30:35.076 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.3470000000000001e-06 2024-09-14T06:30:35.076 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:35.076 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:35.076 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:30:35.076 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.078449+0000", 2024-09-14T06:30:35.076 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032627529000000002 2024-09-14T06:30:35.076 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:35.076 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:35.076 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:30:35.076 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.078458+0000", 2024-09-14T06:30:35.077 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.1910000000000006e-06 2024-09-14T06:30:35.077 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:35.077 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:30:35.077 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:35.077 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:35.077 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:30:35.077 INFO:teuthology.orchestra.run.smithi138.stdout:} 2024-09-14T06:30:35.079 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-14T06:30:35.079 INFO:tasks.ceph.osd.6.smithi138.stderr:2024-09-14T06:30:35.077+0000 7f8cd8849640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33620) UID: 0 2024-09-14T06:30:35.180 INFO:tasks.ceph.osd.4.smithi138.stderr:2024-09-14T06:30:35.179+0000 7f6c653c8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33615) UID: 0 2024-09-14T06:30:35.223 INFO:teuthology.orchestra.run.smithi138.stdout:{ 2024-09-14T06:30:35.223 INFO:teuthology.orchestra.run.smithi138.stdout: "ops": [], 2024-09-14T06:30:35.223 INFO:teuthology.orchestra.run.smithi138.stdout: "num_ops": 0 2024-09-14T06:30:35.223 INFO:teuthology.orchestra.run.smithi138.stdout:} 2024-09-14T06:30:35.231 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-14T06:30:35.281 INFO:tasks.ceph.osd.2.smithi094.stderr:2024-09-14T06:30:35.280+0000 7faf3269b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33649) UID: 0 2024-09-14T06:30:35.377 INFO:teuthology.orchestra.run.smithi138.stdout:{ 2024-09-14T06:30:35.377 INFO:teuthology.orchestra.run.smithi138.stdout: "ops": [], 2024-09-14T06:30:35.377 INFO:teuthology.orchestra.run.smithi138.stdout: "complaint_time": 30, 2024-09-14T06:30:35.377 INFO:teuthology.orchestra.run.smithi138.stdout: "num_blocked_ops": 0 2024-09-14T06:30:35.377 INFO:teuthology.orchestra.run.smithi138.stdout:} 2024-09-14T06:30:35.383 INFO:tasks.ceph.osd.2.smithi094.stderr:2024-09-14T06:30:35.381+0000 7faf3269b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33649) UID: 0 2024-09-14T06:30:35.385 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-14T06:30:35.484 INFO:tasks.ceph.osd.0.smithi094.stderr:2024-09-14T06:30:35.483+0000 7f07c9333640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33636) UID: 0 2024-09-14T06:30:35.491 INFO:teuthology.orchestra.run.smithi138.stdout:{ 2024-09-14T06:30:35.491 INFO:teuthology.orchestra.run.smithi138.stdout: "size": 20, 2024-09-14T06:30:35.491 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 600, 2024-09-14T06:30:35.491 INFO:teuthology.orchestra.run.smithi138.stdout: "ops": [ 2024-09-14T06:30:35.491 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:35.491 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:53244 3.f 3:f71f51a7:::rbd_data.112dca2f6ae1.0000000000000015:17 [read 1863680~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:30:35.491 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:44.369399+0000", 2024-09-14T06:30:35.492 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 51.120030968000002, 2024-09-14T06:30:35.492 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047291189999999997, 2024-09-14T06:30:35.492 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:30:35.492 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:30:35.492 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:30:35.492 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:30:35.492 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:30:35.492 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:35.492 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 53244 2024-09-14T06:30:35.492 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:35.492 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:30:35.492 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:35.492 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:30:35.493 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369399+0000", 2024-09-14T06:30:35.493 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:35.493 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:35.493 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:35.493 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:30:35.493 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369399+0000", 2024-09-14T06:30:35.493 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:35.493 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:35.493 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:35.493 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:30:35.493 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369400+0000", 2024-09-14T06:30:35.493 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.0699999999999999e-07 2024-09-14T06:30:35.494 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:35.494 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:35.494 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:30:35.494 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369401+0000", 2024-09-14T06:30:35.494 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.4300000000000001e-06 2024-09-14T06:30:35.494 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:35.494 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:35.494 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:30:35.494 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369404+0000", 2024-09-14T06:30:35.494 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.061e-06 2024-09-14T06:30:35.494 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:35.494 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:35.494 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:30:35.495 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369407+0000", 2024-09-14T06:30:35.495 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.6139999999999999e-06 2024-09-14T06:30:35.495 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:35.495 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:35.495 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:30:35.495 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.416517+0000", 2024-09-14T06:30:35.495 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047110338000000002 2024-09-14T06:30:35.495 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:35.495 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:35.495 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:30:35.495 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.416616+0000", 2024-09-14T06:30:35.495 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.8651000000000001e-05 2024-09-14T06:30:35.496 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:35.496 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:35.496 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:30:35.496 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.416691+0000", 2024-09-14T06:30:35.496 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.4888999999999994e-05 2024-09-14T06:30:35.496 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:35.496 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:30:35.496 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:35.496 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:35.496 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:35.496 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:53245 3.f 3:f71f51a7:::rbd_data.112dca2f6ae1.0000000000000015:17 [read 1888256~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:30:35.496 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:44.369410+0000", 2024-09-14T06:30:35.496 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 51.120020680000003, 2024-09-14T06:30:35.496 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047399275999999997, 2024-09-14T06:30:35.497 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:30:35.497 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:30:35.497 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:30:35.497 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:30:35.497 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:30:35.497 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:35.497 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 53245 2024-09-14T06:30:35.497 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:35.497 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:30:35.497 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:35.497 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:30:35.497 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369410+0000", 2024-09-14T06:30:35.497 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:35.497 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:35.498 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:35.498 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:30:35.498 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369410+0000", 2024-09-14T06:30:35.498 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:35.498 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:35.498 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:35.498 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:30:35.498 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369410+0000", 2024-09-14T06:30:35.498 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.1899999999999999e-07 2024-09-14T06:30:35.498 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:35.498 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:35.498 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:30:35.498 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369411+0000", 2024-09-14T06:30:35.498 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.0300000000000001e-06 2024-09-14T06:30:35.499 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:35.499 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:35.499 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:30:35.499 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369414+0000", 2024-09-14T06:30:35.499 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.1099999999999999e-06 2024-09-14T06:30:35.499 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:35.499 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:35.499 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:30:35.499 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369417+0000", 2024-09-14T06:30:35.499 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.6189999999999998e-06 2024-09-14T06:30:35.499 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:35.499 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:35.499 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:30:35.499 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.416718+0000", 2024-09-14T06:30:35.500 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047301322999999999 2024-09-14T06:30:35.500 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:35.500 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:35.500 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:30:35.500 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.416759+0000", 2024-09-14T06:30:35.500 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.1009e-05 2024-09-14T06:30:35.500 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:35.500 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:35.500 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:30:35.500 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.416809+0000", 2024-09-14T06:30:35.500 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.9966000000000002e-05 2024-09-14T06:30:35.500 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:35.500 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:30:35.500 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:35.501 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:35.501 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:35.501 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:53246 3.f 3:f71f51a7:::rbd_data.112dca2f6ae1.0000000000000015:17 [read 1912832~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:30:35.501 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:44.369420+0000", 2024-09-14T06:30:35.501 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 51.120010735999998, 2024-09-14T06:30:35.501 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047529463000000001, 2024-09-14T06:30:35.501 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:30:35.501 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:30:35.501 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:30:35.501 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:30:35.501 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:30:35.501 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:35.501 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 53246 2024-09-14T06:30:35.502 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:35.502 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:30:35.502 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:35.502 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:30:35.502 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369420+0000", 2024-09-14T06:30:35.502 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:35.502 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:35.502 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:35.502 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:30:35.502 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369420+0000", 2024-09-14T06:30:35.502 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:35.502 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:35.502 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:35.502 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:30:35.503 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369420+0000", 2024-09-14T06:30:35.503 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.1299999999999999e-07 2024-09-14T06:30:35.503 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:35.503 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:35.503 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:30:35.503 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369425+0000", 2024-09-14T06:30:35.503 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.0130000000000001e-06 2024-09-14T06:30:35.503 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:35.503 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:35.503 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:30:35.503 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369428+0000", 2024-09-14T06:30:35.503 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.349e-06 2024-09-14T06:30:35.503 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:35.503 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:35.504 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:30:35.504 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369431+0000", 2024-09-14T06:30:35.504 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.7389999999999999e-06 2024-09-14T06:30:35.504 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:35.504 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:35.504 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:30:35.504 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.416834+0000", 2024-09-14T06:30:35.504 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047402669000000001 2024-09-14T06:30:35.504 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:35.504 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:35.504 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:30:35.504 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.416882+0000", 2024-09-14T06:30:35.504 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.8068999999999997e-05 2024-09-14T06:30:35.504 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:35.505 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:35.505 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:30:35.505 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.416949+0000", 2024-09-14T06:30:35.505 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.7410999999999995e-05 2024-09-14T06:30:35.505 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:35.505 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:30:35.505 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:35.505 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:35.505 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:35.505 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:53247 3.f 3:f71f51a7:::rbd_data.112dca2f6ae1.0000000000000015:17 [read 1937408~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:30:35.505 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:44.369434+0000", 2024-09-14T06:30:35.505 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 51.119995971999998, 2024-09-14T06:30:35.505 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047652264999999999, 2024-09-14T06:30:35.505 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:30:35.506 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:30:35.506 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:30:35.506 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:30:35.506 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:30:35.506 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:35.506 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 53247 2024-09-14T06:30:35.506 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:35.506 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:30:35.506 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:35.506 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:30:35.506 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369434+0000", 2024-09-14T06:30:35.506 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:35.506 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:35.506 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:35.507 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:30:35.507 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369434+0000", 2024-09-14T06:30:35.507 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:35.507 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:35.507 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:35.507 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:30:35.507 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369435+0000", 2024-09-14T06:30:35.507 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.2100000000000001e-07 2024-09-14T06:30:35.507 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:35.507 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:35.507 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:30:35.507 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369436+0000", 2024-09-14T06:30:35.507 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.122e-06 2024-09-14T06:30:35.507 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:35.508 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:35.508 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:30:35.508 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369439+0000", 2024-09-14T06:30:35.508 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.0400000000000001e-06 2024-09-14T06:30:35.508 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:35.508 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:35.508 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:30:35.508 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369441+0000", 2024-09-14T06:30:35.508 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.728e-06 2024-09-14T06:30:35.508 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:35.508 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:35.508 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:30:35.508 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.416977+0000", 2024-09-14T06:30:35.508 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047535344 2024-09-14T06:30:35.509 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:35.509 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:35.509 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:30:35.509 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.417029+0000", 2024-09-14T06:30:35.509 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.2037999999999999e-05 2024-09-14T06:30:35.509 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:35.509 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:35.509 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:30:35.509 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.417087+0000", 2024-09-14T06:30:35.509 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.7772000000000002e-05 2024-09-14T06:30:35.509 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:35.509 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:30:35.509 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:35.509 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:35.510 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:35.510 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:53248 3.f 3:f71f51a7:::rbd_data.112dca2f6ae1.0000000000000015:17 [read 1961984~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:30:35.510 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:44.369445+0000", 2024-09-14T06:30:35.510 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 51.119985419999999, 2024-09-14T06:30:35.510 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047798489, 2024-09-14T06:30:35.510 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:30:35.510 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:30:35.510 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:30:35.510 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:30:35.510 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:30:35.510 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:35.510 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 53248 2024-09-14T06:30:35.510 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:35.510 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:30:35.511 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:35.511 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:30:35.511 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369445+0000", 2024-09-14T06:30:35.511 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:35.511 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:35.511 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:35.511 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:30:35.511 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369445+0000", 2024-09-14T06:30:35.511 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:35.511 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:35.511 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:35.511 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:30:35.511 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369445+0000", 2024-09-14T06:30:35.511 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.0699999999999999e-07 2024-09-14T06:30:35.511 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:35.512 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:35.512 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:30:35.512 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369446+0000", 2024-09-14T06:30:35.512 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.1000000000000001e-06 2024-09-14T06:30:35.512 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:35.512 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:35.512 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:30:35.512 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369449+0000", 2024-09-14T06:30:35.512 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.1039999999999998e-06 2024-09-14T06:30:35.512 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:35.512 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:35.512 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:30:35.512 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369452+0000", 2024-09-14T06:30:35.513 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.6199999999999999e-06 2024-09-14T06:30:35.513 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:35.513 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:35.513 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:30:35.513 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.417114+0000", 2024-09-14T06:30:35.513 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047661901999999999 2024-09-14T06:30:35.513 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:35.513 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:35.513 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:30:35.513 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.417181+0000", 2024-09-14T06:30:35.513 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.7385999999999995e-05 2024-09-14T06:30:35.514 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:35.514 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:35.514 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:30:35.514 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.417243+0000", 2024-09-14T06:30:35.514 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.2169999999999996e-05 2024-09-14T06:30:35.514 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:35.514 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:30:35.514 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:35.514 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:35.514 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:35.514 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:53249 3.f 3:f71f51a7:::rbd_data.112dca2f6ae1.0000000000000015:17 [read 1986560~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:30:35.514 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:44.369455+0000", 2024-09-14T06:30:35.514 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 51.119975109000002, 2024-09-14T06:30:35.514 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047890493999999999, 2024-09-14T06:30:35.515 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:30:35.515 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:30:35.515 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:30:35.515 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:30:35.515 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:30:35.515 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:35.515 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 53249 2024-09-14T06:30:35.515 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:35.515 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:30:35.515 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:35.515 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:30:35.515 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369455+0000", 2024-09-14T06:30:35.515 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:35.515 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:35.515 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:35.516 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:30:35.516 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369455+0000", 2024-09-14T06:30:35.516 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:35.516 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:35.516 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:35.516 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:30:35.516 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369455+0000", 2024-09-14T06:30:35.516 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.0800000000000001e-07 2024-09-14T06:30:35.516 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:35.516 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:35.516 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:30:35.516 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369456+0000", 2024-09-14T06:30:35.516 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.0729999999999999e-06 2024-09-14T06:30:35.516 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:35.517 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:35.517 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:30:35.517 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369460+0000", 2024-09-14T06:30:35.517 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.027e-06 2024-09-14T06:30:35.517 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:35.517 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:35.517 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:30:35.517 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369462+0000", 2024-09-14T06:30:35.517 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.5320000000000001e-06 2024-09-14T06:30:35.517 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:35.517 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:35.517 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:30:35.517 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.417257+0000", 2024-09-14T06:30:35.517 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.04779489 2024-09-14T06:30:35.518 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:35.518 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:35.518 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:30:35.518 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.417296+0000", 2024-09-14T06:30:35.518 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.8658000000000002e-05 2024-09-14T06:30:35.518 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:35.518 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:35.518 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:30:35.518 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.417346+0000", 2024-09-14T06:30:35.518 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.0105999999999998e-05 2024-09-14T06:30:35.518 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:35.518 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:30:35.518 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:35.518 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:35.518 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:35.519 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:53250 3.f 3:f71f51a7:::rbd_data.112dca2f6ae1.0000000000000015:17 [read 2011136~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:30:35.519 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:44.369465+0000", 2024-09-14T06:30:35.519 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 51.119965139000001, 2024-09-14T06:30:35.519 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.048032230000000002, 2024-09-14T06:30:35.519 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:30:35.519 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:30:35.519 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:30:35.519 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:30:35.519 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:30:35.519 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:35.519 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 53250 2024-09-14T06:30:35.519 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:35.519 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:30:35.520 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:35.520 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:30:35.520 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369465+0000", 2024-09-14T06:30:35.520 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:35.520 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:35.520 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:35.520 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:30:35.520 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369465+0000", 2024-09-14T06:30:35.520 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:35.520 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:35.520 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:35.520 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:30:35.520 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369465+0000", 2024-09-14T06:30:35.521 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.2100000000000001e-07 2024-09-14T06:30:35.521 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:35.521 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:35.521 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:30:35.521 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369466+0000", 2024-09-14T06:30:35.521 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.0640000000000001e-06 2024-09-14T06:30:35.521 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:35.521 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:35.521 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:30:35.521 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369469+0000", 2024-09-14T06:30:35.521 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.0170000000000001e-06 2024-09-14T06:30:35.521 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:35.521 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:35.521 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:30:35.522 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369472+0000", 2024-09-14T06:30:35.522 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.6189999999999998e-06 2024-09-14T06:30:35.522 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:35.522 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:35.522 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:30:35.522 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.417383+0000", 2024-09-14T06:30:35.522 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047911092000000002 2024-09-14T06:30:35.522 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:35.522 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:35.522 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:30:35.522 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.417430+0000", 2024-09-14T06:30:35.522 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.6898999999999999e-05 2024-09-14T06:30:35.522 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:35.522 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:35.523 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:30:35.523 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.417497+0000", 2024-09-14T06:30:35.523 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.7317999999999995e-05 2024-09-14T06:30:35.523 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:35.523 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:30:35.523 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:35.523 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:35.523 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:35.523 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:53251 3.f 3:f71f51a7:::rbd_data.112dca2f6ae1.0000000000000015:17 [read 2035712~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:30:35.523 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:44.369475+0000", 2024-09-14T06:30:35.523 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 51.119955142999999, 2024-09-14T06:30:35.523 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.048154542000000002, 2024-09-14T06:30:35.523 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:30:35.523 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:30:35.524 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:30:35.524 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:30:35.524 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:30:35.524 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:35.524 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 53251 2024-09-14T06:30:35.524 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:35.524 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:30:35.524 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:35.524 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:30:35.524 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369475+0000", 2024-09-14T06:30:35.524 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:35.524 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:35.524 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:35.525 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:30:35.525 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369475+0000", 2024-09-14T06:30:35.525 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:35.525 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:35.525 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:35.525 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:30:35.525 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369475+0000", 2024-09-14T06:30:35.525 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.0100000000000001e-07 2024-09-14T06:30:35.525 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:35.525 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:35.525 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:30:35.525 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369476+0000", 2024-09-14T06:30:35.525 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.0580000000000001e-06 2024-09-14T06:30:35.525 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:35.526 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:35.526 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:30:35.526 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369479+0000", 2024-09-14T06:30:35.526 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.9950000000000001e-06 2024-09-14T06:30:35.526 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:35.526 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:35.526 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:30:35.526 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369482+0000", 2024-09-14T06:30:35.526 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.6340000000000001e-06 2024-09-14T06:30:35.526 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:35.526 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:35.526 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:30:35.526 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.417530+0000", 2024-09-14T06:30:35.526 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.048047535000000002 2024-09-14T06:30:35.527 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:35.527 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:35.527 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:30:35.527 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.417574+0000", 2024-09-14T06:30:35.527 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.4255999999999998e-05 2024-09-14T06:30:35.527 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:35.527 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:35.527 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:30:35.527 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.417630+0000", 2024-09-14T06:30:35.527 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.5862999999999997e-05 2024-09-14T06:30:35.527 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:35.527 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:30:35.527 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:35.527 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:35.528 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:35.528 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:53252 3.f 3:f71f51a7:::rbd_data.112dca2f6ae1.0000000000000015:17 [read 2060288~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:30:35.528 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:44.369485+0000", 2024-09-14T06:30:35.528 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 51.119945295000001, 2024-09-14T06:30:35.528 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.048254116, 2024-09-14T06:30:35.528 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:30:35.528 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:30:35.528 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:30:35.528 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:30:35.528 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:30:35.528 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:35.528 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 53252 2024-09-14T06:30:35.528 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:35.528 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:30:35.529 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:35.529 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:30:35.529 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369485+0000", 2024-09-14T06:30:35.529 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:35.529 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:35.529 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:35.529 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:30:35.529 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369485+0000", 2024-09-14T06:30:35.529 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:35.529 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:35.529 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:35.529 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:30:35.529 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369485+0000", 2024-09-14T06:30:35.530 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.1e-07 2024-09-14T06:30:35.530 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:35.530 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:35.530 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:30:35.530 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369487+0000", 2024-09-14T06:30:35.530 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.42e-06 2024-09-14T06:30:35.530 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:35.530 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:35.530 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:30:35.530 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369490+0000", 2024-09-14T06:30:35.531 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.9639999999999999e-06 2024-09-14T06:30:35.531 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:35.531 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:35.531 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:30:35.531 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369492+0000", 2024-09-14T06:30:35.531 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.5679999999999998e-06 2024-09-14T06:30:35.531 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:35.531 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:35.531 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:30:35.531 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.417645+0000", 2024-09-14T06:30:35.531 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.048152753 2024-09-14T06:30:35.531 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:35.531 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:35.531 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:30:35.532 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.417687+0000", 2024-09-14T06:30:35.532 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.1900999999999997e-05 2024-09-14T06:30:35.532 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:35.532 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:35.532 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:30:35.532 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.417739+0000", 2024-09-14T06:30:35.532 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.2299999999999997e-05 2024-09-14T06:30:35.532 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:35.532 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:30:35.532 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:35.532 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:35.532 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:35.532 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:53253 3.f 3:f71f51a7:::rbd_data.112dca2f6ae1.0000000000000015:17 [read 2084864~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:30:35.532 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:44.369495+0000", 2024-09-14T06:30:35.532 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 51.119935171000002, 2024-09-14T06:30:35.533 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.048366785000000002, 2024-09-14T06:30:35.533 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:30:35.533 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:30:35.533 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:30:35.533 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:30:35.533 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:30:35.533 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:35.533 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 53253 2024-09-14T06:30:35.533 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:35.533 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:30:35.534 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:35.534 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:30:35.534 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369495+0000", 2024-09-14T06:30:35.534 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:35.534 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:35.534 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:35.534 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:30:35.534 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369495+0000", 2024-09-14T06:30:35.534 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:35.534 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:35.534 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:35.534 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:30:35.534 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370388+0000", 2024-09-14T06:30:35.535 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00089287999999999998 2024-09-14T06:30:35.535 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:35.535 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:35.535 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:30:35.535 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370392+0000", 2024-09-14T06:30:35.535 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.2130000000000002e-06 2024-09-14T06:30:35.535 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:35.535 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:35.535 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:30:35.535 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370401+0000", 2024-09-14T06:30:35.535 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.7649999999999999e-06 2024-09-14T06:30:35.535 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:35.535 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:35.535 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:30:35.536 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370406+0000", 2024-09-14T06:30:35.536 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.0159999999999999e-06 2024-09-14T06:30:35.536 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:35.536 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:35.536 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:30:35.536 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.417764+0000", 2024-09-14T06:30:35.536 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047358348000000001 2024-09-14T06:30:35.536 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:35.536 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:35.536 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:30:35.536 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.417806+0000", 2024-09-14T06:30:35.537 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.1149999999999997e-05 2024-09-14T06:30:35.537 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:35.537 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:35.537 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:30:35.537 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.417862+0000", 2024-09-14T06:30:35.537 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.6412999999999997e-05 2024-09-14T06:30:35.537 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:35.537 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:30:35.537 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:35.537 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:35.537 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:35.537 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:53340 3.f 3:f71f51a7:::rbd_data.112dca2f6ae1.0000000000000015:17 [read 1056768~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:30:35.537 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:44.370421+0000", 2024-09-14T06:30:35.538 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 51.119009290999998, 2024-09-14T06:30:35.538 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047566313999999998, 2024-09-14T06:30:35.538 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:30:35.538 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:30:35.538 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:30:35.538 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:30:35.538 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:30:35.538 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:35.538 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 53340 2024-09-14T06:30:35.538 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:35.538 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:30:35.538 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:35.538 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:30:35.538 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370421+0000", 2024-09-14T06:30:35.539 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:35.539 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:35.539 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:35.539 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:30:35.539 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370421+0000", 2024-09-14T06:30:35.539 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:35.539 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:35.539 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:35.539 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:30:35.539 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370422+0000", 2024-09-14T06:30:35.539 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.3799999999999996e-07 2024-09-14T06:30:35.539 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:35.539 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:35.539 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:30:35.539 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370424+0000", 2024-09-14T06:30:35.540 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.6919999999999998e-06 2024-09-14T06:30:35.540 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:35.540 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:35.540 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:30:35.540 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370432+0000", 2024-09-14T06:30:35.540 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.2439999999999996e-06 2024-09-14T06:30:35.540 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:35.540 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:35.540 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:30:35.540 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370437+0000", 2024-09-14T06:30:35.540 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.5389999999999998e-06 2024-09-14T06:30:35.541 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:35.541 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:35.541 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:30:35.541 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.417888+0000", 2024-09-14T06:30:35.541 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047450432000000001 2024-09-14T06:30:35.541 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:35.541 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:35.541 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:30:35.541 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.417935+0000", 2024-09-14T06:30:35.541 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.7454999999999999e-05 2024-09-14T06:30:35.541 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:35.541 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:35.541 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:30:35.541 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.417987+0000", 2024-09-14T06:30:35.542 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.2213999999999998e-05 2024-09-14T06:30:35.542 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:35.542 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:30:35.542 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:35.542 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:35.542 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:35.542 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:53341 3.f 3:f71f51a7:::rbd_data.112dca2f6ae1.0000000000000015:17 [read 1081344~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:30:35.542 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:44.370446+0000", 2024-09-14T06:30:35.542 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 51.118984017999999, 2024-09-14T06:30:35.542 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047665723, 2024-09-14T06:30:35.542 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:30:35.542 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:30:35.542 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:30:35.542 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:30:35.542 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:30:35.542 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:35.543 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 53341 2024-09-14T06:30:35.543 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:35.543 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:30:35.543 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:35.543 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:30:35.543 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370446+0000", 2024-09-14T06:30:35.543 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:35.543 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:35.543 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:35.543 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:30:35.543 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370446+0000", 2024-09-14T06:30:35.543 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:35.543 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:35.543 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:35.544 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:30:35.544 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370447+0000", 2024-09-14T06:30:35.544 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.6100000000000002e-07 2024-09-14T06:30:35.544 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:35.544 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:35.544 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:30:35.544 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370448+0000", 2024-09-14T06:30:35.544 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.7370000000000001e-06 2024-09-14T06:30:35.544 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:35.544 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:35.544 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:30:35.544 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370462+0000", 2024-09-14T06:30:35.544 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.3227000000000001e-05 2024-09-14T06:30:35.545 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:35.545 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:35.545 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:30:35.545 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370467+0000", 2024-09-14T06:30:35.545 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.6540000000000004e-06 2024-09-14T06:30:35.545 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:35.545 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:35.545 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:30:35.545 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.418013+0000", 2024-09-14T06:30:35.545 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047545325999999999 2024-09-14T06:30:35.545 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:35.545 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:35.545 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:30:35.545 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.418054+0000", 2024-09-14T06:30:35.546 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.1696999999999997e-05 2024-09-14T06:30:35.546 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:35.546 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:35.546 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:30:35.546 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.418112+0000", 2024-09-14T06:30:35.546 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.7720999999999998e-05 2024-09-14T06:30:35.546 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:35.546 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:30:35.546 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:35.546 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:35.546 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:35.546 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:53342 3.f 3:f71f51a7:::rbd_data.112dca2f6ae1.0000000000000015:17 [read 1105920~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:30:35.546 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:44.370475+0000", 2024-09-14T06:30:35.546 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 51.118954957, 2024-09-14T06:30:35.547 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047748848000000003, 2024-09-14T06:30:35.547 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:30:35.547 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:30:35.547 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:30:35.547 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:30:35.547 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:30:35.547 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:35.547 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 53342 2024-09-14T06:30:35.547 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:35.547 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:30:35.547 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:35.547 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:30:35.547 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370475+0000", 2024-09-14T06:30:35.548 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:35.548 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:35.548 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:35.548 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:30:35.548 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370475+0000", 2024-09-14T06:30:35.548 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:35.548 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:35.548 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:35.548 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:30:35.548 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370476+0000", 2024-09-14T06:30:35.548 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.4700000000000002e-07 2024-09-14T06:30:35.548 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:35.548 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:35.548 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:30:35.549 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370478+0000", 2024-09-14T06:30:35.549 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.5890000000000001e-06 2024-09-14T06:30:35.549 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:35.549 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:35.549 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:30:35.549 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370486+0000", 2024-09-14T06:30:35.549 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.3470000000000001e-06 2024-09-14T06:30:35.549 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:35.549 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:35.549 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:30:35.549 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370491+0000", 2024-09-14T06:30:35.549 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.5720000000000002e-06 2024-09-14T06:30:35.550 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:35.550 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:35.550 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:30:35.550 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.418135+0000", 2024-09-14T06:30:35.550 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047644094999999997 2024-09-14T06:30:35.550 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:35.550 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:35.550 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:30:35.550 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.418174+0000", 2024-09-14T06:30:35.550 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.8204000000000002e-05 2024-09-14T06:30:35.550 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:35.550 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:35.550 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:30:35.551 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.418224+0000", 2024-09-14T06:30:35.551 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.0593999999999997e-05 2024-09-14T06:30:35.551 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:35.551 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:30:35.551 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:35.551 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:35.551 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:35.551 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:53343 3.f 3:f71f51a7:::rbd_data.112dca2f6ae1.0000000000000015:17 [read 1130496~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:30:35.551 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:44.370498+0000", 2024-09-14T06:30:35.551 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 51.118932080999997, 2024-09-14T06:30:35.551 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047841574999999997, 2024-09-14T06:30:35.551 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:30:35.551 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:30:35.551 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:30:35.552 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:30:35.552 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:30:35.552 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:35.552 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 53343 2024-09-14T06:30:35.552 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:35.552 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:30:35.552 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:35.552 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:30:35.552 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370498+0000", 2024-09-14T06:30:35.552 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:35.552 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:35.552 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:35.552 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:30:35.553 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370498+0000", 2024-09-14T06:30:35.553 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:35.553 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:35.553 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:35.553 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:30:35.553 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370499+0000", 2024-09-14T06:30:35.553 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.9999999999999998e-07 2024-09-14T06:30:35.553 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:35.553 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:35.553 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:30:35.553 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370501+0000", 2024-09-14T06:30:35.553 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.0310000000000001e-06 2024-09-14T06:30:35.553 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:35.554 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:35.554 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:30:35.554 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370507+0000", 2024-09-14T06:30:35.554 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.4169999999999997e-06 2024-09-14T06:30:35.554 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:35.554 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:35.554 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:30:35.554 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370516+0000", 2024-09-14T06:30:35.554 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.7190000000000007e-06 2024-09-14T06:30:35.554 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:35.554 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:35.554 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:30:35.554 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.418247+0000", 2024-09-14T06:30:35.555 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047731190999999999 2024-09-14T06:30:35.555 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:35.555 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:35.555 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:30:35.555 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.418286+0000", 2024-09-14T06:30:35.555 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.8988e-05 2024-09-14T06:30:35.555 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:35.555 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:35.555 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:30:35.555 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.418340+0000", 2024-09-14T06:30:35.555 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.3829000000000002e-05 2024-09-14T06:30:35.555 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:35.555 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:30:35.555 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:35.556 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:35.556 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:35.556 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:53344 3.f 3:f71f51a7:::rbd_data.112dca2f6ae1.0000000000000015:17 [read 1155072~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:30:35.556 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:44.370524+0000", 2024-09-14T06:30:35.556 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 51.118906311000003, 2024-09-14T06:30:35.556 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047972262000000002, 2024-09-14T06:30:35.556 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:30:35.556 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:30:35.556 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:30:35.556 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:30:35.556 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:30:35.556 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:35.556 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 53344 2024-09-14T06:30:35.556 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:35.557 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:30:35.557 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:35.557 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:30:35.557 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370524+0000", 2024-09-14T06:30:35.557 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:35.557 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:35.557 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:35.557 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:30:35.557 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370524+0000", 2024-09-14T06:30:35.557 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:35.557 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:35.557 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:35.557 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:30:35.557 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370618+0000", 2024-09-14T06:30:35.558 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.4425999999999993e-05 2024-09-14T06:30:35.558 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:35.558 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:35.558 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:30:35.558 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370622+0000", 2024-09-14T06:30:35.558 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.3519999999999998e-06 2024-09-14T06:30:35.558 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:35.558 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:35.558 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:30:35.558 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370631+0000", 2024-09-14T06:30:35.558 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.9299999999999992e-06 2024-09-14T06:30:35.558 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:35.558 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:35.558 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:30:35.559 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370637+0000", 2024-09-14T06:30:35.559 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.3269999999999997e-06 2024-09-14T06:30:35.559 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:35.559 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:35.559 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:30:35.559 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.418372+0000", 2024-09-14T06:30:35.559 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047734764999999998 2024-09-14T06:30:35.559 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:35.559 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:35.559 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:30:35.559 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.418431+0000", 2024-09-14T06:30:35.559 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.8696999999999997e-05 2024-09-14T06:30:35.560 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:35.560 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:35.560 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:30:35.560 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.418496+0000", 2024-09-14T06:30:35.560 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.5765000000000005e-05 2024-09-14T06:30:35.560 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:35.560 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:30:35.560 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:35.560 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:35.560 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:35.560 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:53345 3.f 3:f71f51a7:::rbd_data.112dca2f6ae1.0000000000000015:17 [read 1179648~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:30:35.560 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:44.370647+0000", 2024-09-14T06:30:35.560 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 51.118782844000002, 2024-09-14T06:30:35.561 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047953775999999997, 2024-09-14T06:30:35.561 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:30:35.561 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:30:35.561 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:30:35.561 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:30:35.561 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:30:35.561 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:35.561 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 53345 2024-09-14T06:30:35.561 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:35.561 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:30:35.561 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:35.561 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:30:35.561 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370647+0000", 2024-09-14T06:30:35.562 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:35.562 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:35.562 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:35.562 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:30:35.562 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370647+0000", 2024-09-14T06:30:35.562 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:35.562 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:35.562 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:35.562 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:30:35.562 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.372346+0000", 2024-09-14T06:30:35.562 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.001698284 2024-09-14T06:30:35.562 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:35.562 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:35.562 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:30:35.563 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.372351+0000", 2024-09-14T06:30:35.563 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.8890000000000001e-06 2024-09-14T06:30:35.563 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:35.563 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:35.563 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:30:35.563 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.372362+0000", 2024-09-14T06:30:35.563 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.0941000000000001e-05 2024-09-14T06:30:35.563 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:35.563 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:35.563 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:30:35.563 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.372369+0000", 2024-09-14T06:30:35.563 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.362e-06 2024-09-14T06:30:35.563 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:35.563 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:35.564 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:30:35.564 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.418510+0000", 2024-09-14T06:30:35.564 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046140914999999998 2024-09-14T06:30:35.564 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:35.564 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:35.564 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:30:35.564 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.418549+0000", 2024-09-14T06:30:35.564 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.8717999999999998e-05 2024-09-14T06:30:35.564 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:35.564 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:35.564 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:30:35.564 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.418601+0000", 2024-09-14T06:30:35.565 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.2667000000000002e-05 2024-09-14T06:30:35.565 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:35.565 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:30:35.565 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:35.565 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:35.565 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:35.565 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:53351 3.f 3:f71f51a7:::rbd_data.112dca2f6ae1.0000000000000015:17 [read 1327104~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:30:35.565 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:44.372499+0000", 2024-09-14T06:30:35.565 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 51.116931723, 2024-09-14T06:30:35.565 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047140596, 2024-09-14T06:30:35.565 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:30:35.565 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:30:35.565 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:30:35.565 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:30:35.566 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:30:35.566 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:35.566 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 53351 2024-09-14T06:30:35.566 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:35.566 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:30:35.566 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:35.566 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:30:35.566 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.372499+0000", 2024-09-14T06:30:35.566 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:35.566 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:35.566 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:35.567 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:30:35.567 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.372499+0000", 2024-09-14T06:30:35.567 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:35.567 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:35.567 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:35.567 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:30:35.567 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.372499+0000", 2024-09-14T06:30:35.567 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.3000000000000001e-07 2024-09-14T06:30:35.567 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:35.567 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:35.567 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:30:35.567 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.372501+0000", 2024-09-14T06:30:35.568 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.0820000000000001e-06 2024-09-14T06:30:35.568 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:35.568 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:35.568 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:30:35.568 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.372513+0000", 2024-09-14T06:30:35.568 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.1610999999999999e-05 2024-09-14T06:30:35.568 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:35.568 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:35.568 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:30:35.568 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.372518+0000", 2024-09-14T06:30:35.568 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.327e-06 2024-09-14T06:30:35.568 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:35.568 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:35.568 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:30:35.569 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.419530+0000", 2024-09-14T06:30:35.569 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047011798 2024-09-14T06:30:35.569 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:35.569 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:35.569 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:30:35.569 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.419578+0000", 2024-09-14T06:30:35.569 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.8446999999999999e-05 2024-09-14T06:30:35.569 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:35.569 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:35.569 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:30:35.569 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.419639+0000", 2024-09-14T06:30:35.569 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.0900999999999998e-05 2024-09-14T06:30:35.569 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:35.569 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:30:35.570 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:35.570 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:35.570 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:35.570 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:53352 3.f 3:f71f51a7:::rbd_data.112dca2f6ae1.0000000000000015:17 [read 1351680~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:30:35.570 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:44.372525+0000", 2024-09-14T06:30:35.570 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 51.116905809000002, 2024-09-14T06:30:35.570 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047259847000000001, 2024-09-14T06:30:35.570 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:30:35.570 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:30:35.570 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:30:35.570 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:30:35.570 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:30:35.570 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:35.570 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 53352 2024-09-14T06:30:35.571 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:35.571 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:30:35.571 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:35.571 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:30:35.571 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.372525+0000", 2024-09-14T06:30:35.571 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:35.571 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:35.571 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:35.571 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:30:35.571 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.372525+0000", 2024-09-14T06:30:35.571 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:35.571 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:35.571 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:35.571 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:30:35.572 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.372525+0000", 2024-09-14T06:30:35.572 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.0600000000000001e-07 2024-09-14T06:30:35.572 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:35.572 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:35.572 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:30:35.572 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.372527+0000", 2024-09-14T06:30:35.572 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.0499999999999999e-06 2024-09-14T06:30:35.572 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:35.572 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:35.572 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:30:35.572 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.372533+0000", 2024-09-14T06:30:35.572 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.4320000000000004e-06 2024-09-14T06:30:35.572 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:35.572 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:35.573 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:30:35.573 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.372538+0000", 2024-09-14T06:30:35.573 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.9130000000000002e-06 2024-09-14T06:30:35.573 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:35.573 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:35.573 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:30:35.573 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.419667+0000", 2024-09-14T06:30:35.573 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047128976000000003 2024-09-14T06:30:35.573 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:35.573 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:35.573 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:30:35.573 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.419715+0000", 2024-09-14T06:30:35.573 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.7868999999999999e-05 2024-09-14T06:30:35.573 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:35.574 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:35.574 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:30:35.574 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.419784+0000", 2024-09-14T06:30:35.574 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.9201000000000004e-05 2024-09-14T06:30:35.574 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:35.574 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:30:35.574 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:35.574 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:35.574 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:35.574 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:53353 3.f 3:f71f51a7:::rbd_data.112dca2f6ae1.0000000000000015:17 [read 1376256~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:30:35.574 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:44.372545+0000", 2024-09-14T06:30:35.574 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 51.116885769, 2024-09-14T06:30:35.574 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047364232999999999, 2024-09-14T06:30:35.574 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:30:35.574 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:30:35.575 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:30:35.575 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:30:35.575 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:30:35.575 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:35.575 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 53353 2024-09-14T06:30:35.575 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:35.575 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:30:35.575 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:35.575 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:30:35.575 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.372545+0000", 2024-09-14T06:30:35.575 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:35.575 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:35.575 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:35.576 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:30:35.576 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.372545+0000", 2024-09-14T06:30:35.576 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:35.576 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:35.576 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:35.576 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:30:35.576 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.372545+0000", 2024-09-14T06:30:35.576 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.27e-07 2024-09-14T06:30:35.576 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:35.576 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:35.576 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:30:35.576 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.372547+0000", 2024-09-14T06:30:35.576 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.0600000000000002e-06 2024-09-14T06:30:35.576 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:35.577 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:35.577 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:30:35.577 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.372553+0000", 2024-09-14T06:30:35.577 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.7980000000000002e-06 2024-09-14T06:30:35.577 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:35.577 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:35.577 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:30:35.577 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.372557+0000", 2024-09-14T06:30:35.577 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.7389999999999999e-06 2024-09-14T06:30:35.577 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:35.577 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:35.577 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:30:35.577 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.419811+0000", 2024-09-14T06:30:35.578 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047253310999999999 2024-09-14T06:30:35.578 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:35.578 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:35.578 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:30:35.578 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.419854+0000", 2024-09-14T06:30:35.578 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.3448000000000001e-05 2024-09-14T06:30:35.578 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:35.578 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:35.578 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:30:35.578 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.419909+0000", 2024-09-14T06:30:35.578 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.4549999999999998e-05 2024-09-14T06:30:35.578 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:35.578 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:30:35.578 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:35.579 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:35.579 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:35.579 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:53354 3.f 3:f71f51a7:::rbd_data.112dca2f6ae1.0000000000000015:17 [read 1400832~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:30:35.579 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:44.372563+0000", 2024-09-14T06:30:35.579 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 51.116867331999998, 2024-09-14T06:30:35.579 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047482233999999998, 2024-09-14T06:30:35.579 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:30:35.579 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:30:35.579 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:30:35.580 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:30:35.580 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:30:35.580 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:35.580 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 53354 2024-09-14T06:30:35.580 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:35.580 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:30:35.580 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:35.580 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:30:35.580 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.372563+0000", 2024-09-14T06:30:35.580 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:35.580 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:35.580 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:35.580 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:30:35.580 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.372563+0000", 2024-09-14T06:30:35.581 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:35.581 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:35.581 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:35.581 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:30:35.581 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.375009+0000", 2024-09-14T06:30:35.581 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.0024457680000000001 2024-09-14T06:30:35.581 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:35.581 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:35.581 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:30:35.581 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.375014+0000", 2024-09-14T06:30:35.581 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.0599999999999998e-06 2024-09-14T06:30:35.581 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:35.581 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:35.581 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:30:35.582 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.375026+0000", 2024-09-14T06:30:35.582 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.1783999999999999e-05 2024-09-14T06:30:35.582 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:35.582 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:35.582 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:30:35.582 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.375035+0000", 2024-09-14T06:30:35.582 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.7799999999999995e-06 2024-09-14T06:30:35.582 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:35.582 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:35.582 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:30:35.582 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.419942+0000", 2024-09-14T06:30:35.582 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.044906982999999998 2024-09-14T06:30:35.582 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:35.582 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:35.583 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:30:35.583 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.419989+0000", 2024-09-14T06:30:35.583 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.7104999999999997e-05 2024-09-14T06:30:35.583 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:35.583 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:35.583 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:30:35.583 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.420045+0000", 2024-09-14T06:30:35.583 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.5754000000000002e-05 2024-09-14T06:30:35.583 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:35.583 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:30:35.583 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:35.583 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:35.583 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:30:35.583 INFO:teuthology.orchestra.run.smithi138.stdout:} 2024-09-14T06:30:35.585 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-14T06:30:35.586 INFO:tasks.ceph.osd.0.smithi094.stderr:2024-09-14T06:30:35.583+0000 7f07c9333640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33636) UID: 0 2024-09-14T06:30:35.589 INFO:teuthology.orchestra.run.smithi094.stderr:set 3.1 pg_upmap mapping to [4,1] 2024-09-14T06:30:35.668 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-14T06:30:35.686 INFO:tasks.ceph.osd.5.smithi138.stderr:2024-09-14T06:30:35.685+0000 7fe13746f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33618) UID: 0 2024-09-14T06:30:35.698 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-14T06:30:35.699 INFO:teuthology.orchestra.run.smithi094.stdout: "ops": [], 2024-09-14T06:30:35.699 INFO:teuthology.orchestra.run.smithi094.stdout: "num_ops": 0 2024-09-14T06:30:35.699 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-14T06:30:35.708 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-14T06:30:35.787 INFO:tasks.ceph.osd.7.smithi138.stderr:2024-09-14T06:30:35.786+0000 7f3c6a21d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33623) UID: 0 2024-09-14T06:30:35.853 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-14T06:30:35.854 INFO:teuthology.orchestra.run.smithi094.stdout: "ops": [], 2024-09-14T06:30:35.854 INFO:teuthology.orchestra.run.smithi094.stdout: "complaint_time": 30, 2024-09-14T06:30:35.854 INFO:teuthology.orchestra.run.smithi094.stdout: "num_blocked_ops": 0 2024-09-14T06:30:35.854 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-14T06:30:35.863 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-14T06:30:35.888 INFO:tasks.ceph.osd.6.smithi138.stderr:2024-09-14T06:30:35.888+0000 7f8cd8849640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33620) UID: 0 2024-09-14T06:30:35.967 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-14T06:30:35.967 INFO:teuthology.orchestra.run.smithi094.stdout: "size": 20, 2024-09-14T06:30:35.968 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 600, 2024-09-14T06:30:35.968 INFO:teuthology.orchestra.run.smithi094.stdout: "ops": [ 2024-09-14T06:30:35.968 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:35.968 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:115193 3.1 3:856be886:::rbd_data.112d4d24c558.0000000000000004:29 [read 1523712~16384 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e96)", 2024-09-14T06:30:35.968 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:30:17.939854+0000", 2024-09-14T06:30:35.968 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 18.026238357, 2024-09-14T06:30:35.968 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.073897516999999996, 2024-09-14T06:30:35.968 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:30:35.968 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:30:35.969 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:30:35.969 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:30:35.969 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:30:35.969 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:35.969 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 115193 2024-09-14T06:30:35.969 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:35.969 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:30:35.969 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:35.969 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:30:35.969 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.939854+0000", 2024-09-14T06:30:35.969 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:35.969 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:35.969 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:35.969 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:30:35.969 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.939854+0000", 2024-09-14T06:30:35.970 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:35.970 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:35.970 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:35.970 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:30:35.970 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.939854+0000", 2024-09-14T06:30:35.970 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.9500000000000002e-07 2024-09-14T06:30:35.970 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:35.970 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:35.970 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:30:35.970 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.939856+0000", 2024-09-14T06:30:35.970 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.0959999999999999e-06 2024-09-14T06:30:35.970 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:35.970 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:35.970 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:30:35.971 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.939863+0000", 2024-09-14T06:30:35.971 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.1129999999999997e-06 2024-09-14T06:30:35.971 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:35.971 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:35.971 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:30:35.971 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.939867+0000", 2024-09-14T06:30:35.971 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.8350000000000003e-06 2024-09-14T06:30:35.971 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:35.971 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:35.972 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:30:35.972 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.013626+0000", 2024-09-14T06:30:35.972 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.073758799999999999 2024-09-14T06:30:35.972 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:35.972 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:35.972 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:30:35.972 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.013690+0000", 2024-09-14T06:30:35.972 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.3918999999999996e-05 2024-09-14T06:30:35.972 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:35.972 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:35.972 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:30:35.972 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.013751+0000", 2024-09-14T06:30:35.972 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.1159000000000002e-05 2024-09-14T06:30:35.972 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:35.972 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:35.973 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:35.973 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:35.973 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:35.973 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:115194 3.1 3:856be886:::rbd_data.112d4d24c558.0000000000000004:29 [read 1769472~16384 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e96)", 2024-09-14T06:30:35.973 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:30:17.939874+0000", 2024-09-14T06:30:35.973 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 18.026218046, 2024-09-14T06:30:35.973 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.074016988000000006, 2024-09-14T06:30:35.973 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:30:35.973 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:30:35.973 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:30:35.974 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:30:35.974 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:30:35.974 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:35.974 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 115194 2024-09-14T06:30:35.974 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:35.974 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:30:35.974 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:35.974 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:30:35.974 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.939874+0000", 2024-09-14T06:30:35.974 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:35.974 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:35.974 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:35.974 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:30:35.974 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.939874+0000", 2024-09-14T06:30:35.974 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:35.975 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:35.975 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:35.975 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:30:35.975 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.939875+0000", 2024-09-14T06:30:35.975 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.2600000000000002e-07 2024-09-14T06:30:35.975 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:35.975 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:35.975 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:30:35.975 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.939877+0000", 2024-09-14T06:30:35.975 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.9420000000000002e-06 2024-09-14T06:30:35.975 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:35.975 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:35.975 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:30:35.975 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.939882+0000", 2024-09-14T06:30:35.975 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.8749999999999997e-06 2024-09-14T06:30:35.975 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:35.976 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:35.976 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:30:35.976 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.939887+0000", 2024-09-14T06:30:35.976 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.741e-06 2024-09-14T06:30:35.976 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:35.976 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:35.976 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:30:35.976 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.013767+0000", 2024-09-14T06:30:35.976 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.073879720999999995 2024-09-14T06:30:35.976 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:35.976 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:35.977 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:30:35.977 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.013832+0000", 2024-09-14T06:30:35.977 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.5312000000000001e-05 2024-09-14T06:30:35.977 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:35.977 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:35.977 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:30:35.977 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.013891+0000", 2024-09-14T06:30:35.977 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.8870999999999999e-05 2024-09-14T06:30:35.977 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:35.977 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:35.977 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:35.977 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:35.977 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:35.977 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:115195 3.1 3:856be886:::rbd_data.112d4d24c558.0000000000000004:29 [read 2015232~16384 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e96)", 2024-09-14T06:30:35.978 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:30:17.939903+0000", 2024-09-14T06:30:35.978 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 18.026188861000001, 2024-09-14T06:30:35.978 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.074128199000000006, 2024-09-14T06:30:35.978 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:30:35.978 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:30:35.978 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:30:35.978 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:30:35.978 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:30:35.978 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:35.978 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 115195 2024-09-14T06:30:35.978 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:35.978 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:30:35.978 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:35.978 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:30:35.979 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.939903+0000", 2024-09-14T06:30:35.979 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:35.979 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:35.979 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:35.979 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:30:35.979 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.939903+0000", 2024-09-14T06:30:35.979 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:35.979 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:35.979 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:35.979 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:30:35.979 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.939904+0000", 2024-09-14T06:30:35.979 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.5799999999999999e-07 2024-09-14T06:30:35.979 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:35.979 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:35.979 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:30:35.980 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.939906+0000", 2024-09-14T06:30:35.980 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.2350000000000002e-06 2024-09-14T06:30:35.980 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:35.980 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:35.980 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:30:35.980 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.939913+0000", 2024-09-14T06:30:35.980 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.5649999999999998e-06 2024-09-14T06:30:35.980 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:35.980 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:35.980 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:30:35.980 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.939918+0000", 2024-09-14T06:30:35.980 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.784e-06 2024-09-14T06:30:35.980 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:35.980 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:35.981 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:30:35.981 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.013921+0000", 2024-09-14T06:30:35.981 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.074002911000000005 2024-09-14T06:30:35.981 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:35.981 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:35.981 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:30:35.981 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.013971+0000", 2024-09-14T06:30:35.981 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.9551e-05 2024-09-14T06:30:35.981 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:35.981 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:35.981 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:30:35.982 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.014031+0000", 2024-09-14T06:30:35.982 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.0594999999999998e-05 2024-09-14T06:30:35.982 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:35.982 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:35.982 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:35.982 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:35.982 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:35.982 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:115196 3.1 3:856be886:::rbd_data.112d4d24c558.0000000000000004:29 [read 2260992~16384 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e96)", 2024-09-14T06:30:35.982 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:30:17.939925+0000", 2024-09-14T06:30:35.982 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 18.026166668999998, 2024-09-14T06:30:35.982 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.074320137999999994, 2024-09-14T06:30:35.982 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:30:35.983 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:30:35.983 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:30:35.983 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:30:35.983 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:30:35.983 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:35.983 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 115196 2024-09-14T06:30:35.983 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:35.983 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:30:35.983 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:35.983 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:30:35.983 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.939925+0000", 2024-09-14T06:30:35.983 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:35.983 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:35.983 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:35.984 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:30:35.984 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.939925+0000", 2024-09-14T06:30:35.984 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:35.984 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:35.984 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:35.984 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:30:35.984 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.939926+0000", 2024-09-14T06:30:35.984 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.9999999999999998e-07 2024-09-14T06:30:35.984 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:35.984 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:35.984 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:30:35.984 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.939928+0000", 2024-09-14T06:30:35.984 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.9549999999999999e-06 2024-09-14T06:30:35.985 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:35.985 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:35.985 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:30:35.985 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.939934+0000", 2024-09-14T06:30:35.985 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.127e-06 2024-09-14T06:30:35.985 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:35.985 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:35.985 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:30:35.985 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.939939+0000", 2024-09-14T06:30:35.985 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.775e-06 2024-09-14T06:30:35.986 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:35.986 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:35.986 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:30:35.986 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.014075+0000", 2024-09-14T06:30:35.986 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.074136598999999997 2024-09-14T06:30:35.986 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:35.986 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:35.986 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:30:35.986 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.014162+0000", 2024-09-14T06:30:35.986 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.6477999999999994e-05 2024-09-14T06:30:35.986 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:35.986 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:35.987 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:30:35.987 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.014246+0000", 2024-09-14T06:30:35.987 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.3703999999999997e-05 2024-09-14T06:30:35.987 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:35.987 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:35.987 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:35.987 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:35.987 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:35.987 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:115197 3.1 3:856be886:::rbd_data.112d4d24c558.0000000000000004:29 [read 2506752~16384 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e96)", 2024-09-14T06:30:35.987 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:30:17.939969+0000", 2024-09-14T06:30:35.987 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 18.026123183999999, 2024-09-14T06:30:35.987 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.074528544000000002, 2024-09-14T06:30:35.987 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:30:35.987 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:30:35.988 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:30:35.988 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:30:35.988 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:30:35.988 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:35.988 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 115197 2024-09-14T06:30:35.988 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:35.988 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:30:35.988 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:35.988 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:30:35.988 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.939969+0000", 2024-09-14T06:30:35.988 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:35.988 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:35.988 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:35.988 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:30:35.988 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.939969+0000", 2024-09-14T06:30:35.989 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:35.989 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:35.989 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:35.989 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:30:35.989 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.939970+0000", 2024-09-14T06:30:35.989 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.3600000000000003e-07 2024-09-14T06:30:35.989 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:35.989 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:35.989 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:30:35.989 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.939972+0000", 2024-09-14T06:30:35.989 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.3180000000000001e-06 2024-09-14T06:30:35.989 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:35.989 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:35.990 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:30:35.990 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.939979+0000", 2024-09-14T06:30:35.990 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.8800000000000002e-06 2024-09-14T06:30:35.990 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:35.990 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:35.990 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:30:35.990 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.939984+0000", 2024-09-14T06:30:35.990 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.3510000000000001e-06 2024-09-14T06:30:35.990 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:35.990 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:35.990 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:30:35.990 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.014302+0000", 2024-09-14T06:30:35.991 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.074317416999999997 2024-09-14T06:30:35.991 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:35.991 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:35.991 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:30:35.991 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.014392+0000", 2024-09-14T06:30:35.991 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.0181999999999998e-05 2024-09-14T06:30:35.991 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:35.991 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:35.991 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:30:35.991 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.014497+0000", 2024-09-14T06:30:35.991 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00010576 2024-09-14T06:30:35.991 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:35.992 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:35.992 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:35.992 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:35.992 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:35.992 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:115198 3.1 3:856be886:::rbd_data.112d4d24c558.0000000000000004:29 [read 2752512~16384 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e96)", 2024-09-14T06:30:35.992 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:30:17.940004+0000", 2024-09-14T06:30:35.992 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 18.026087904000001, 2024-09-14T06:30:35.992 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.074736367999999997, 2024-09-14T06:30:35.992 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:30:35.992 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:30:35.992 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:30:35.992 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:30:35.992 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:30:35.993 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:35.993 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 115198 2024-09-14T06:30:35.993 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:35.993 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:30:35.993 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:35.993 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:30:35.993 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940004+0000", 2024-09-14T06:30:35.993 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:35.993 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:35.993 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:35.993 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:30:35.993 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940004+0000", 2024-09-14T06:30:35.993 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:35.994 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:35.994 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:35.994 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:30:35.994 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940005+0000", 2024-09-14T06:30:35.994 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.2200000000000004e-07 2024-09-14T06:30:35.994 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:35.994 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:35.994 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:30:35.994 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940007+0000", 2024-09-14T06:30:35.994 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.322e-06 2024-09-14T06:30:35.994 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:35.994 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:35.994 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:30:35.994 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940014+0000", 2024-09-14T06:30:35.994 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.6220000000000003e-06 2024-09-14T06:30:35.995 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:35.995 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:35.995 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:30:35.995 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940019+0000", 2024-09-14T06:30:35.995 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.6380000000000001e-06 2024-09-14T06:30:35.995 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:35.995 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:35.995 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:30:35.995 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.014562+0000", 2024-09-14T06:30:35.995 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.074542320999999995 2024-09-14T06:30:35.995 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:35.995 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:35.995 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:30:35.996 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.014646+0000", 2024-09-14T06:30:35.996 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.4079999999999995e-05 2024-09-14T06:30:35.996 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:35.996 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:35.996 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:30:35.996 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.014741+0000", 2024-09-14T06:30:35.996 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.4763000000000001e-05 2024-09-14T06:30:35.996 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:35.996 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:35.996 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:35.996 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:35.996 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:35.996 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:115199 3.1 3:856be886:::rbd_data.112d4d24c558.0000000000000004:29 [read 2998272~16384 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e96)", 2024-09-14T06:30:35.996 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:30:17.940040+0000", 2024-09-14T06:30:35.996 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 18.026052149000002, 2024-09-14T06:30:35.997 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.074957134999999994, 2024-09-14T06:30:35.997 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:30:35.997 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:30:35.997 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:30:35.997 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:30:35.997 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:30:35.997 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:35.997 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 115199 2024-09-14T06:30:35.997 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:35.997 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:30:35.997 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:35.997 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:30:35.998 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940040+0000", 2024-09-14T06:30:35.998 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:35.998 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:35.998 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:35.998 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:30:35.998 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940040+0000", 2024-09-14T06:30:35.998 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:35.998 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:35.998 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:35.998 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:30:35.999 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940041+0000", 2024-09-14T06:30:35.999 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.4000000000000001e-07 2024-09-14T06:30:35.999 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:35.999 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:35.999 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:30:35.999 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940046+0000", 2024-09-14T06:30:35.999 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.7250000000000002e-06 2024-09-14T06:30:35.999 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:35.999 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:35.999 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:30:35.999 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940053+0000", 2024-09-14T06:30:35.999 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.6109999999999999e-06 2024-09-14T06:30:35.999 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:35.999 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:36.000 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:30:36.000 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940058+0000", 2024-09-14T06:30:36.000 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.1590000000000001e-06 2024-09-14T06:30:36.000 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:36.000 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:36.000 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:30:36.000 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.014768+0000", 2024-09-14T06:30:36.000 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.074709624000000002 2024-09-14T06:30:36.000 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:36.000 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:36.000 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:30:36.000 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.014907+0000", 2024-09-14T06:30:36.000 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000139665 2024-09-14T06:30:36.000 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:36.001 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:36.001 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:30:36.001 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.014997+0000", 2024-09-14T06:30:36.001 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.9710999999999995e-05 2024-09-14T06:30:36.001 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:36.001 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:36.001 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:36.001 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:36.001 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:36.001 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:115200 3.1 3:856be886:::rbd_data.112d4d24c558.0000000000000004:29 [read 3244032~16384 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e96)", 2024-09-14T06:30:36.002 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:30:17.940080+0000", 2024-09-14T06:30:36.002 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 18.026011944, 2024-09-14T06:30:36.002 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.075137291999999994, 2024-09-14T06:30:36.002 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:30:36.002 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:30:36.002 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:30:36.002 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:30:36.002 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:30:36.002 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:36.002 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 115200 2024-09-14T06:30:36.002 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:36.002 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:30:36.002 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:36.003 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:30:36.003 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940080+0000", 2024-09-14T06:30:36.003 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:36.003 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:36.003 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:36.003 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:30:36.003 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940080+0000", 2024-09-14T06:30:36.003 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:36.004 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:36.004 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:36.005 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:30:36.005 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940081+0000", 2024-09-14T06:30:36.005 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.0800000000000004e-07 2024-09-14T06:30:36.005 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:36.005 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:36.005 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:30:36.005 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940083+0000", 2024-09-14T06:30:36.005 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.3920000000000001e-06 2024-09-14T06:30:36.005 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:36.005 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:36.005 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:30:36.005 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940091+0000", 2024-09-14T06:30:36.005 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.1389999999999995e-06 2024-09-14T06:30:36.005 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:36.006 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:36.006 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:30:36.006 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940097+0000", 2024-09-14T06:30:36.006 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.5550000000000002e-06 2024-09-14T06:30:36.006 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:36.006 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:36.006 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:30:36.006 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.015025+0000", 2024-09-14T06:30:36.006 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.074928579999999995 2024-09-14T06:30:36.006 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:36.006 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:36.006 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:30:36.006 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.015096+0000", 2024-09-14T06:30:36.006 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.0705999999999997e-05 2024-09-14T06:30:36.006 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:36.006 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:36.007 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:30:36.007 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.015217+0000", 2024-09-14T06:30:36.007 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00012131200000000001 2024-09-14T06:30:36.007 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:36.007 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:36.007 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:36.007 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:36.007 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:36.007 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:115201 3.1 3:856be886:::rbd_data.112d4d24c558.0000000000000004:29 [read 3489792~16384 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e96)", 2024-09-14T06:30:36.007 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:30:17.940113+0000", 2024-09-14T06:30:36.007 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 18.025979170999999, 2024-09-14T06:30:36.007 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.075246899000000006, 2024-09-14T06:30:36.007 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:30:36.007 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:30:36.007 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:30:36.007 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:30:36.008 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:30:36.008 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:36.008 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 115201 2024-09-14T06:30:36.008 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:36.008 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:30:36.008 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:36.008 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:30:36.008 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940113+0000", 2024-09-14T06:30:36.008 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:36.008 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:36.009 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:36.009 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:30:36.009 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940113+0000", 2024-09-14T06:30:36.009 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:36.009 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:36.009 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:36.009 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:30:36.009 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940114+0000", 2024-09-14T06:30:36.009 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.1500000000000004e-07 2024-09-14T06:30:36.009 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:36.009 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:36.009 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:30:36.009 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940116+0000", 2024-09-14T06:30:36.009 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.2809999999999998e-06 2024-09-14T06:30:36.010 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:36.010 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:36.010 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:30:36.010 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940123+0000", 2024-09-14T06:30:36.010 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.9229999999999996e-06 2024-09-14T06:30:36.010 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:36.010 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:36.010 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:30:36.010 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940128+0000", 2024-09-14T06:30:36.010 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.7790000000000003e-06 2024-09-14T06:30:36.010 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:36.010 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:36.010 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:30:36.010 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.015253+0000", 2024-09-14T06:30:36.010 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.075125792999999996 2024-09-14T06:30:36.011 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:36.011 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:36.011 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:30:36.011 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.015297+0000", 2024-09-14T06:30:36.011 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.3847000000000002e-05 2024-09-14T06:30:36.011 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:36.011 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:36.011 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:30:36.013 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.015360+0000", 2024-09-14T06:30:36.013 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.2661000000000002e-05 2024-09-14T06:30:36.013 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:36.013 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:36.013 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:36.013 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:36.014 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:36.014 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:115202 3.1 3:856be886:::rbd_data.112d4d24c558.0000000000000004:29 [read 3735552~16384 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e96)", 2024-09-14T06:30:36.014 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:30:17.940135+0000", 2024-09-14T06:30:36.014 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 18.025957391999999, 2024-09-14T06:30:36.014 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.075451105000000004, 2024-09-14T06:30:36.014 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:30:36.014 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:30:36.014 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:30:36.014 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:30:36.014 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:30:36.014 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:36.015 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 115202 2024-09-14T06:30:36.015 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:36.015 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:30:36.015 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:36.015 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:30:36.015 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940135+0000", 2024-09-14T06:30:36.015 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:36.015 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:36.015 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:36.015 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:30:36.015 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940135+0000", 2024-09-14T06:30:36.015 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:36.015 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:36.016 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:36.016 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:30:36.016 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940135+0000", 2024-09-14T06:30:36.016 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.0999999999999999e-07 2024-09-14T06:30:36.016 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:36.016 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:36.016 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:30:36.016 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940137+0000", 2024-09-14T06:30:36.016 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.0020000000000001e-06 2024-09-14T06:30:36.016 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:36.016 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:36.016 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:30:36.017 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940143+0000", 2024-09-14T06:30:36.017 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.9850000000000003e-06 2024-09-14T06:30:36.017 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:36.017 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:36.017 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:30:36.017 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940148+0000", 2024-09-14T06:30:36.017 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.9880000000000004e-06 2024-09-14T06:30:36.017 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:36.018 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:36.018 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:30:36.018 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.015400+0000", 2024-09-14T06:30:36.018 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.075252186999999998 2024-09-14T06:30:36.018 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:36.018 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:36.018 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:30:36.018 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.015481+0000", 2024-09-14T06:30:36.018 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.0698000000000006e-05 2024-09-14T06:30:36.018 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:36.018 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:36.018 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:30:36.018 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.015586+0000", 2024-09-14T06:30:36.018 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00010483499999999999 2024-09-14T06:30:36.019 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:36.019 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:36.019 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:36.019 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:36.019 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:36.019 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:115203 3.1 3:856be886:::rbd_data.112d4d24c558.0000000000000004:29 [read 3981312~16384 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e96)", 2024-09-14T06:30:36.019 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:30:17.940156+0000", 2024-09-14T06:30:36.019 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 18.025936167000001, 2024-09-14T06:30:36.019 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.075620860999999998, 2024-09-14T06:30:36.019 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:30:36.019 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:30:36.019 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:30:36.020 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:30:36.020 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:30:36.020 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:36.020 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 115203 2024-09-14T06:30:36.020 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:36.020 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:30:36.021 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:36.021 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:30:36.021 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940156+0000", 2024-09-14T06:30:36.021 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:36.021 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:36.021 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:36.021 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:30:36.021 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940156+0000", 2024-09-14T06:30:36.021 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:36.021 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:36.021 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:36.021 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:30:36.021 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940156+0000", 2024-09-14T06:30:36.022 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.1500000000000005e-07 2024-09-14T06:30:36.022 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:36.022 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:36.022 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:30:36.022 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940158+0000", 2024-09-14T06:30:36.022 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.967e-06 2024-09-14T06:30:36.022 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:36.022 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:36.022 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:30:36.022 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940165+0000", 2024-09-14T06:30:36.022 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.6939999999999998e-06 2024-09-14T06:30:36.023 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:36.023 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:36.023 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:30:36.023 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940170+0000", 2024-09-14T06:30:36.023 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.7539999999999997e-06 2024-09-14T06:30:36.023 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:36.023 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:36.023 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:30:36.023 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.015633+0000", 2024-09-14T06:30:36.023 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.075463334000000007 2024-09-14T06:30:36.023 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:36.023 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:36.023 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:30:36.023 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.015693+0000", 2024-09-14T06:30:36.024 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.9367000000000003e-05 2024-09-14T06:30:36.024 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:36.024 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:36.024 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:30:36.024 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.015777+0000", 2024-09-14T06:30:36.024 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.4229999999999998e-05 2024-09-14T06:30:36.024 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:36.024 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:36.024 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:36.024 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:36.024 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:36.024 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:115204 3.1 3:856be886:::rbd_data.112d4d24c558.0000000000000004:29 [read 4227072~16384 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e96)", 2024-09-14T06:30:36.024 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:30:17.940176+0000", 2024-09-14T06:30:36.025 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 18.025915634, 2024-09-14T06:30:36.025 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.075785505000000003, 2024-09-14T06:30:36.025 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:30:36.025 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:30:36.025 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:30:36.025 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:30:36.025 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:30:36.026 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:36.026 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 115204 2024-09-14T06:30:36.026 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:36.026 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:30:36.026 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:36.026 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:30:36.026 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940176+0000", 2024-09-14T06:30:36.026 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:36.026 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:36.026 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:36.026 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:30:36.026 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940176+0000", 2024-09-14T06:30:36.026 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:36.027 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:36.027 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:36.027 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:30:36.027 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940177+0000", 2024-09-14T06:30:36.027 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.7899999999999999e-07 2024-09-14T06:30:36.027 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:36.027 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:36.027 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:30:36.027 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940179+0000", 2024-09-14T06:30:36.027 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.103e-06 2024-09-14T06:30:36.028 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:36.028 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:36.028 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:30:36.028 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940185+0000", 2024-09-14T06:30:36.028 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.7239999999999997e-06 2024-09-14T06:30:36.028 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:36.028 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:36.028 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:30:36.028 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940190+0000", 2024-09-14T06:30:36.028 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.7269999999999998e-06 2024-09-14T06:30:36.028 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:36.028 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:36.028 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:30:36.028 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.015814+0000", 2024-09-14T06:30:36.029 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.075624114000000006 2024-09-14T06:30:36.029 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:36.029 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:36.029 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:30:36.029 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.015862+0000", 2024-09-14T06:30:36.029 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.8770999999999998e-05 2024-09-14T06:30:36.029 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:36.029 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:36.029 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:30:36.029 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.015962+0000", 2024-09-14T06:30:36.029 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.9586999999999994e-05 2024-09-14T06:30:36.029 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:36.029 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:36.029 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:36.029 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:36.030 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:36.030 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:115205 3.1 3:856be886:::rbd_data.112d4d24c558.0000000000000004:29 [read 4472832~16384 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e96)", 2024-09-14T06:30:36.030 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:30:17.940197+0000", 2024-09-14T06:30:36.030 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 18.025894924999999, 2024-09-14T06:30:36.030 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.076068920999999998, 2024-09-14T06:30:36.030 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:30:36.030 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:30:36.030 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:30:36.030 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:30:36.030 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:30:36.030 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:36.030 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 115205 2024-09-14T06:30:36.030 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:36.030 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:30:36.030 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:36.031 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:30:36.031 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940197+0000", 2024-09-14T06:30:36.031 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:36.031 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:36.031 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:36.031 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:30:36.031 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940197+0000", 2024-09-14T06:30:36.031 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:36.031 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:36.031 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:36.031 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:30:36.031 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940198+0000", 2024-09-14T06:30:36.031 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.5999999999999999e-07 2024-09-14T06:30:36.031 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:36.031 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:36.032 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:30:36.032 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940200+0000", 2024-09-14T06:30:36.032 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.0899999999999999e-06 2024-09-14T06:30:36.032 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:36.032 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:36.032 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:30:36.032 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940206+0000", 2024-09-14T06:30:36.032 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.8710000000000002e-06 2024-09-14T06:30:36.032 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:36.032 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:36.032 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:30:36.032 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940211+0000", 2024-09-14T06:30:36.032 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.9230000000000001e-06 2024-09-14T06:30:36.032 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:36.033 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:36.033 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:30:36.033 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.016059+0000", 2024-09-14T06:30:36.033 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.075848051999999999 2024-09-14T06:30:36.033 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:36.033 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:36.033 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:30:36.033 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.016151+0000", 2024-09-14T06:30:36.033 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.1982000000000001e-05 2024-09-14T06:30:36.033 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:36.033 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:36.033 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:30:36.033 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.016266+0000", 2024-09-14T06:30:36.033 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000115543 2024-09-14T06:30:36.034 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:36.034 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:36.034 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:36.034 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:36.034 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:36.034 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:115206 3.1 3:856be886:::rbd_data.112d4d24c558.0000000000000004:29 [read 4718592~16384 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e96)", 2024-09-14T06:30:36.034 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:30:17.940223+0000", 2024-09-14T06:30:36.034 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 18.025868835000001, 2024-09-14T06:30:36.034 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.076265115999999994, 2024-09-14T06:30:36.034 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:30:36.034 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:30:36.034 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:30:36.034 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:30:36.034 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:30:36.034 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:36.034 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 115206 2024-09-14T06:30:36.035 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:36.035 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:30:36.035 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:36.035 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:30:36.035 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940223+0000", 2024-09-14T06:30:36.035 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:36.035 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:36.035 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:36.035 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:30:36.035 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940223+0000", 2024-09-14T06:30:36.035 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:36.035 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:36.035 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:36.035 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:30:36.035 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940224+0000", 2024-09-14T06:30:36.035 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.0399999999999996e-07 2024-09-14T06:30:36.036 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:36.036 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:36.036 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:30:36.036 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940226+0000", 2024-09-14T06:30:36.036 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.1780000000000002e-06 2024-09-14T06:30:36.036 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:36.036 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:36.036 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:30:36.036 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940232+0000", 2024-09-14T06:30:36.036 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.5010000000000001e-06 2024-09-14T06:30:36.036 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:36.036 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:36.036 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:30:36.037 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940237+0000", 2024-09-14T06:30:36.037 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.0250000000000002e-06 2024-09-14T06:30:36.037 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:36.037 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:36.037 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:30:36.037 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.016305+0000", 2024-09-14T06:30:36.037 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.076067262999999996 2024-09-14T06:30:36.037 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:36.037 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:36.037 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:30:36.037 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.016383+0000", 2024-09-14T06:30:36.037 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.8578000000000006e-05 2024-09-14T06:30:36.037 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:36.037 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:36.038 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:30:36.038 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.016488+0000", 2024-09-14T06:30:36.038 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000105067 2024-09-14T06:30:36.038 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:36.038 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:36.038 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:36.038 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:36.038 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:36.038 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:115207 3.1 3:856be886:::rbd_data.112d4d24c558.0000000000000004:29 [read 4964352~16384 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e96)", 2024-09-14T06:30:36.038 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:30:17.940244+0000", 2024-09-14T06:30:36.038 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 18.025847719000001, 2024-09-14T06:30:36.038 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.076371154999999996, 2024-09-14T06:30:36.038 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:30:36.038 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:30:36.038 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:30:36.039 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:30:36.039 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:30:36.039 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:36.039 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 115207 2024-09-14T06:30:36.039 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:36.039 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:30:36.039 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:36.039 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:30:36.039 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940244+0000", 2024-09-14T06:30:36.039 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:36.039 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:36.039 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:36.039 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:30:36.039 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940244+0000", 2024-09-14T06:30:36.039 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:36.040 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:36.040 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:36.040 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:30:36.040 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940245+0000", 2024-09-14T06:30:36.040 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.1200000000000003e-07 2024-09-14T06:30:36.040 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:36.040 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:36.040 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:30:36.040 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940247+0000", 2024-09-14T06:30:36.040 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.0030000000000001e-06 2024-09-14T06:30:36.040 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:36.040 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:36.040 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:30:36.040 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940252+0000", 2024-09-14T06:30:36.040 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.5520000000000004e-06 2024-09-14T06:30:36.041 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:36.041 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:36.041 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:30:36.041 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940257+0000", 2024-09-14T06:30:36.041 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.6759999999999998e-06 2024-09-14T06:30:36.041 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:36.041 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:36.041 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:30:36.041 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.016521+0000", 2024-09-14T06:30:36.041 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.076263499999999998 2024-09-14T06:30:36.041 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:36.041 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:36.041 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:30:36.041 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.016562+0000", 2024-09-14T06:30:36.041 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.0967000000000003e-05 2024-09-14T06:30:36.042 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:36.042 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:36.042 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:30:36.042 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.016616+0000", 2024-09-14T06:30:36.042 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.3944999999999999e-05 2024-09-14T06:30:36.042 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:36.042 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:36.042 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:36.042 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:36.042 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:36.042 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:115208 3.1 3:856be886:::rbd_data.112d4d24c558.0000000000000004:29 [read 5210112~16384 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e96)", 2024-09-14T06:30:36.042 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:30:17.940264+0000", 2024-09-14T06:30:36.042 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 18.025827862, 2024-09-14T06:30:36.043 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.076558288000000002, 2024-09-14T06:30:36.043 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:30:36.043 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:30:36.043 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:30:36.043 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:30:36.043 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:30:36.043 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:36.043 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 115208 2024-09-14T06:30:36.043 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:36.043 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:30:36.043 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:36.043 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:30:36.043 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940264+0000", 2024-09-14T06:30:36.043 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:36.043 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:36.043 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:36.044 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:30:36.044 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940264+0000", 2024-09-14T06:30:36.044 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:36.044 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:36.044 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:36.044 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:30:36.044 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940265+0000", 2024-09-14T06:30:36.044 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.0500000000000004e-07 2024-09-14T06:30:36.044 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:36.044 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:36.044 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:30:36.044 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940267+0000", 2024-09-14T06:30:36.044 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.9810000000000002e-06 2024-09-14T06:30:36.044 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:36.045 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:36.045 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:30:36.045 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940272+0000", 2024-09-14T06:30:36.045 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.6999999999999996e-06 2024-09-14T06:30:36.045 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:36.045 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:36.045 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:30:36.045 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940277+0000", 2024-09-14T06:30:36.045 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.6480000000000002e-06 2024-09-14T06:30:36.045 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:36.045 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:36.045 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:30:36.045 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.016650+0000", 2024-09-14T06:30:36.045 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.076373089000000005 2024-09-14T06:30:36.046 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:36.046 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:36.046 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:30:36.046 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.016724+0000", 2024-09-14T06:30:36.046 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.4105999999999998e-05 2024-09-14T06:30:36.046 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:36.046 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:36.046 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:30:36.046 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.016823+0000", 2024-09-14T06:30:36.046 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.8258999999999996e-05 2024-09-14T06:30:36.046 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:36.046 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:36.046 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:36.046 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:36.046 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:36.046 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:115209 3.1 3:856be886:::rbd_data.112d4d24c558.0000000000000004:29 [read 5455872~16384 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e96)", 2024-09-14T06:30:36.047 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:30:17.940283+0000", 2024-09-14T06:30:36.047 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 18.025808836, 2024-09-14T06:30:36.047 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.076724029999999999, 2024-09-14T06:30:36.047 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:30:36.047 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:30:36.047 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:30:36.047 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:30:36.047 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:30:36.047 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:36.047 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 115209 2024-09-14T06:30:36.047 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:36.047 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:30:36.047 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:36.047 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:30:36.047 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940283+0000", 2024-09-14T06:30:36.047 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:36.048 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:36.048 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:36.048 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:30:36.048 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940283+0000", 2024-09-14T06:30:36.048 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:36.048 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:36.048 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:36.048 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:30:36.048 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940284+0000", 2024-09-14T06:30:36.048 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.6700000000000003e-07 2024-09-14T06:30:36.048 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:36.048 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:36.048 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:30:36.048 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940286+0000", 2024-09-14T06:30:36.048 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.9520000000000001e-06 2024-09-14T06:30:36.048 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:36.049 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:36.049 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:30:36.049 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940292+0000", 2024-09-14T06:30:36.049 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.7339999999999996e-06 2024-09-14T06:30:36.049 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:36.049 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:36.049 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:30:36.049 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940296+0000", 2024-09-14T06:30:36.049 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.7700000000000001e-06 2024-09-14T06:30:36.049 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:36.049 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:36.049 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:30:36.049 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.016859+0000", 2024-09-14T06:30:36.049 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.07656317 2024-09-14T06:30:36.049 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:36.049 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:36.050 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:30:36.050 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.016929+0000", 2024-09-14T06:30:36.050 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.9679999999999997e-05 2024-09-14T06:30:36.050 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:36.050 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:36.050 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:30:36.050 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.017007+0000", 2024-09-14T06:30:36.050 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.8157000000000004e-05 2024-09-14T06:30:36.050 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:36.050 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:36.050 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:36.050 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:36.050 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:36.050 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:115210 3.1 3:856be886:::rbd_data.112d4d24c558.0000000000000004:29 [read 5701632~16384 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e96)", 2024-09-14T06:30:36.050 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:30:17.940303+0000", 2024-09-14T06:30:36.050 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 18.025789319000001, 2024-09-14T06:30:36.051 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.076821358000000006, 2024-09-14T06:30:36.051 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:30:36.051 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:30:36.051 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:30:36.051 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:30:36.051 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:30:36.051 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:36.051 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 115210 2024-09-14T06:30:36.051 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:36.051 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:30:36.051 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:36.051 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:30:36.051 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940303+0000", 2024-09-14T06:30:36.051 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:36.052 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:36.052 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:36.052 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:30:36.052 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940303+0000", 2024-09-14T06:30:36.052 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:36.052 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:36.052 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:36.052 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:30:36.052 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940303+0000", 2024-09-14T06:30:36.052 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.3000000000000001e-07 2024-09-14T06:30:36.052 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:36.052 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:36.052 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:30:36.052 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940306+0000", 2024-09-14T06:30:36.053 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.255e-06 2024-09-14T06:30:36.053 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:36.053 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:36.053 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:30:36.053 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940312+0000", 2024-09-14T06:30:36.053 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.9710000000000001e-06 2024-09-14T06:30:36.053 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:36.053 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:36.053 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:30:36.053 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940316+0000", 2024-09-14T06:30:36.053 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.5340000000000001e-06 2024-09-14T06:30:36.053 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:36.053 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:36.053 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:30:36.054 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.017033+0000", 2024-09-14T06:30:36.054 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.076717169000000002 2024-09-14T06:30:36.054 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:36.054 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:36.054 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:30:36.054 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.017073+0000", 2024-09-14T06:30:36.054 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.9582000000000002e-05 2024-09-14T06:30:36.054 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:36.054 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:36.054 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:30:36.054 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.017124+0000", 2024-09-14T06:30:36.054 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.1316999999999997e-05 2024-09-14T06:30:36.054 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:36.054 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:36.055 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:36.055 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:36.055 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:36.055 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:115219 3.1 3:856be886:::rbd_data.112d4d24c558.0000000000000004:29 [read 65536~16384 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e96)", 2024-09-14T06:30:36.055 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:30:17.940323+0000", 2024-09-14T06:30:36.055 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 18.025769457999999, 2024-09-14T06:30:36.055 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.076915205, 2024-09-14T06:30:36.055 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:30:36.055 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:30:36.055 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:30:36.055 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:30:36.055 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:30:36.055 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:36.055 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 115219 2024-09-14T06:30:36.055 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:36.055 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:30:36.056 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:36.056 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:30:36.056 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940323+0000", 2024-09-14T06:30:36.056 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:36.056 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:36.056 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:36.056 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:30:36.056 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940323+0000", 2024-09-14T06:30:36.056 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:36.056 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:36.056 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:36.057 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:30:36.057 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940323+0000", 2024-09-14T06:30:36.057 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.0800000000000005e-07 2024-09-14T06:30:36.057 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:36.057 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:36.057 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:30:36.057 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940328+0000", 2024-09-14T06:30:36.057 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.887e-06 2024-09-14T06:30:36.057 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:36.057 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:36.057 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:30:36.057 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940334+0000", 2024-09-14T06:30:36.057 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.3269999999999997e-06 2024-09-14T06:30:36.057 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:36.058 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:36.058 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:30:36.058 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940339+0000", 2024-09-14T06:30:36.058 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.7960000000000003e-06 2024-09-14T06:30:36.058 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:36.058 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:36.058 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:30:36.058 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.017148+0000", 2024-09-14T06:30:36.058 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.076809125000000006 2024-09-14T06:30:36.058 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:36.058 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:36.058 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:30:36.058 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.017186+0000", 2024-09-14T06:30:36.058 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.7283000000000002e-05 2024-09-14T06:30:36.058 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:36.059 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:36.059 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:30:36.059 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.017238+0000", 2024-09-14T06:30:36.059 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.2278999999999999e-05 2024-09-14T06:30:36.059 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:36.059 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:36.059 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:36.059 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:36.059 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:36.059 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:115220 3.1 3:856be886:::rbd_data.112d4d24c558.0000000000000004:29 [read 311296~16384 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e96)", 2024-09-14T06:30:36.060 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:30:17.940346+0000", 2024-09-14T06:30:36.060 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 18.025746329, 2024-09-14T06:30:36.060 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.077033293000000003, 2024-09-14T06:30:36.060 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:30:36.060 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:30:36.060 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:30:36.060 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:30:36.060 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:30:36.060 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:36.060 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 115220 2024-09-14T06:30:36.060 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:36.060 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:30:36.061 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:36.061 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:30:36.061 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940346+0000", 2024-09-14T06:30:36.061 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:36.061 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:36.061 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:36.061 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:30:36.061 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940346+0000", 2024-09-14T06:30:36.061 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:36.061 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:36.061 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:36.061 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:30:36.061 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940346+0000", 2024-09-14T06:30:36.061 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.4300000000000003e-07 2024-09-14T06:30:36.062 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:36.062 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:36.062 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:30:36.062 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940348+0000", 2024-09-14T06:30:36.062 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.0030000000000001e-06 2024-09-14T06:30:36.062 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:36.062 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:36.062 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:30:36.062 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.004655+0000", 2024-09-14T06:30:36.062 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.064306323999999998 2024-09-14T06:30:36.062 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:36.062 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:36.062 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:30:36.062 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.004671+0000", 2024-09-14T06:30:36.062 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.6589999999999999e-05 2024-09-14T06:30:36.062 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:36.063 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:36.063 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:30:36.063 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.017262+0000", 2024-09-14T06:30:36.063 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.012590570000000001 2024-09-14T06:30:36.063 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:36.063 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:36.063 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:30:36.063 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.017321+0000", 2024-09-14T06:30:36.063 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.9210999999999999e-05 2024-09-14T06:30:36.063 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:36.063 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:36.063 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:30:36.063 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.017379+0000", 2024-09-14T06:30:36.063 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.8051999999999999e-05 2024-09-14T06:30:36.063 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:36.064 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:36.064 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:36.064 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:36.064 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:36.064 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-14T06:30:36.065 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-14T06:30:36.066 INFO:tasks.ceph.osd.5.smithi138.stderr:2024-09-14T06:30:36.064+0000 7fe13746f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33618) UID: 0 2024-09-14T06:30:36.166 INFO:tasks.ceph.osd.1.smithi094.stderr:2024-09-14T06:30:36.164+0000 7f328e47a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33641) UID: 0 2024-09-14T06:30:36.168 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-14T06:30:36.168 INFO:teuthology.orchestra.run.smithi094.stdout: "ops": [], 2024-09-14T06:30:36.168 INFO:teuthology.orchestra.run.smithi094.stdout: "num_ops": 0 2024-09-14T06:30:36.168 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-14T06:30:36.176 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-14T06:30:36.267 INFO:tasks.ceph.osd.5.smithi138.stderr:2024-09-14T06:30:36.267+0000 7fe13746f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33618) UID: 0 2024-09-14T06:30:36.321 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-14T06:30:36.321 INFO:teuthology.orchestra.run.smithi094.stdout: "ops": [], 2024-09-14T06:30:36.321 INFO:teuthology.orchestra.run.smithi094.stdout: "complaint_time": 30, 2024-09-14T06:30:36.321 INFO:teuthology.orchestra.run.smithi094.stdout: "num_blocked_ops": 0 2024-09-14T06:30:36.322 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-14T06:30:36.330 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-14T06:30:36.368 INFO:tasks.ceph.osd.5.smithi138.stderr:2024-09-14T06:30:36.368+0000 7fe13746f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33618) UID: 0 2024-09-14T06:30:36.434 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-14T06:30:36.434 INFO:teuthology.orchestra.run.smithi094.stdout: "size": 20, 2024-09-14T06:30:36.434 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 600, 2024-09-14T06:30:36.434 INFO:teuthology.orchestra.run.smithi094.stdout: "ops": [ 2024-09-14T06:30:36.435 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:36.435 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:14918 3.0 3:05c39535:::rbd_data.112d92027590.0000000000000009:f [read 795648~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:30:36.435 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:26.083834+0000", 2024-09-14T06:30:36.435 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 70.348751242000006, 2024-09-14T06:30:36.435 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.041776287000000002, 2024-09-14T06:30:36.435 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:30:36.435 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:30:36.435 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:30:36.435 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:30:36.435 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:30:36.435 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:36.435 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 14918 2024-09-14T06:30:36.435 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:36.435 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:30:36.436 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:36.436 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:30:36.436 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.083834+0000", 2024-09-14T06:30:36.436 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:36.436 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:36.436 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:36.436 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:30:36.436 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.083834+0000", 2024-09-14T06:30:36.436 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:36.436 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:36.436 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:36.436 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:30:36.436 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.083834+0000", 2024-09-14T06:30:36.436 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.0100000000000001e-07 2024-09-14T06:30:36.436 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:36.437 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:36.437 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:30:36.437 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.083835+0000", 2024-09-14T06:30:36.437 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.3900000000000003e-07 2024-09-14T06:30:36.437 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:36.437 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:36.437 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:30:36.437 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.083838+0000", 2024-09-14T06:30:36.437 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.9830000000000001e-06 2024-09-14T06:30:36.437 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:36.437 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:36.437 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:30:36.437 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.083840+0000", 2024-09-14T06:30:36.437 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.4679999999999999e-06 2024-09-14T06:30:36.438 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:36.438 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:36.438 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:30:36.438 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.125455+0000", 2024-09-14T06:30:36.438 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.041614578999999999 2024-09-14T06:30:36.438 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:36.438 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:36.438 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:30:36.438 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.125496+0000", 2024-09-14T06:30:36.438 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.1047000000000002e-05 2024-09-14T06:30:36.438 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:36.438 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:36.438 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:30:36.438 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.125610+0000", 2024-09-14T06:30:36.439 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00011407 2024-09-14T06:30:36.439 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:36.439 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:36.439 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:36.439 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:36.439 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:36.439 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:14919 3.0 3:05c39535:::rbd_data.112d92027590.0000000000000009:f [read 805888~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:30:36.439 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:26.083843+0000", 2024-09-14T06:30:36.439 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 70.348741493999995, 2024-09-14T06:30:36.439 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.041992356000000002, 2024-09-14T06:30:36.439 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:30:36.439 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:30:36.439 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:30:36.439 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:30:36.440 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:30:36.440 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:36.440 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 14919 2024-09-14T06:30:36.440 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:36.440 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:30:36.440 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:36.440 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:30:36.440 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.083843+0000", 2024-09-14T06:30:36.440 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:36.440 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:36.440 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:36.440 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:30:36.440 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.083843+0000", 2024-09-14T06:30:36.441 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:36.441 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:36.441 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:36.441 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:30:36.441 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.083844+0000", 2024-09-14T06:30:36.441 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.3300000000000001e-07 2024-09-14T06:30:36.441 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:36.441 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:36.441 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:30:36.441 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.083845+0000", 2024-09-14T06:30:36.441 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.4729999999999999e-06 2024-09-14T06:30:36.441 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:36.441 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:36.442 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:30:36.442 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.083850+0000", 2024-09-14T06:30:36.442 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.0130000000000001e-06 2024-09-14T06:30:36.442 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:36.442 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:36.442 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:30:36.442 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.083854+0000", 2024-09-14T06:30:36.442 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.4759999999999998e-06 2024-09-14T06:30:36.442 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:36.442 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:36.442 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:30:36.442 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.125636+0000", 2024-09-14T06:30:36.442 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.041782683000000001 2024-09-14T06:30:36.443 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:36.443 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:36.443 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:30:36.443 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.125680+0000", 2024-09-14T06:30:36.443 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.3614e-05 2024-09-14T06:30:36.443 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:36.443 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:36.443 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:30:36.443 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.125836+0000", 2024-09-14T06:30:36.443 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000155864 2024-09-14T06:30:36.443 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:36.443 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:36.443 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:36.443 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:36.444 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:36.444 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:14920 3.0 3:05c39535:::rbd_data.112d92027590.0000000000000009:f [read 816128~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:30:36.444 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:26.083868+0000", 2024-09-14T06:30:36.444 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 70.348716832999997, 2024-09-14T06:30:36.444 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.042166771999999998, 2024-09-14T06:30:36.444 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:30:36.444 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:30:36.444 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:30:36.444 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:30:36.444 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:30:36.444 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:36.444 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 14920 2024-09-14T06:30:36.444 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:36.445 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:30:36.445 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:36.445 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:30:36.445 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.083868+0000", 2024-09-14T06:30:36.445 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:36.445 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:36.445 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:36.445 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:30:36.445 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.083868+0000", 2024-09-14T06:30:36.445 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:36.445 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:36.445 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:36.445 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:30:36.446 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.083869+0000", 2024-09-14T06:30:36.446 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.99e-07 2024-09-14T06:30:36.446 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:36.446 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:36.446 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:30:36.446 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.083870+0000", 2024-09-14T06:30:36.446 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.782e-06 2024-09-14T06:30:36.446 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:36.446 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:36.446 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:30:36.446 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.083876+0000", 2024-09-14T06:30:36.446 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.3809999999999998e-06 2024-09-14T06:30:36.446 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:36.446 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:36.446 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:30:36.447 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.083882+0000", 2024-09-14T06:30:36.447 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.9479999999999996e-06 2024-09-14T06:30:36.447 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:36.447 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:36.447 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:30:36.447 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.125878+0000", 2024-09-14T06:30:36.447 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.041996264999999998 2024-09-14T06:30:36.447 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:36.447 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:36.447 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:30:36.447 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.125929+0000", 2024-09-14T06:30:36.447 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.0498999999999999e-05 2024-09-14T06:30:36.447 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:36.447 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:36.447 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:30:36.448 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.126035+0000", 2024-09-14T06:30:36.448 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000106298 2024-09-14T06:30:36.448 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:36.448 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:36.448 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:36.448 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:36.448 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:36.448 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:14921 3.0 3:05c39535:::rbd_data.112d92027590.0000000000000009:f [read 826368~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:30:36.448 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:26.083889+0000", 2024-09-14T06:30:36.448 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 70.348695823, 2024-09-14T06:30:36.448 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.042435309999999997, 2024-09-14T06:30:36.448 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:30:36.448 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:30:36.448 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:30:36.449 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:30:36.449 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:30:36.449 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:36.449 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 14921 2024-09-14T06:30:36.449 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:36.449 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:30:36.449 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:36.449 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:30:36.449 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.083889+0000", 2024-09-14T06:30:36.449 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:36.449 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:36.449 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:36.449 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:30:36.450 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.083889+0000", 2024-09-14T06:30:36.450 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:36.450 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:36.450 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:36.450 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:30:36.450 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.086033+0000", 2024-09-14T06:30:36.450 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0021444179999999999 2024-09-14T06:30:36.450 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:36.450 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:36.450 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:30:36.450 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.086038+0000", 2024-09-14T06:30:36.450 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.0969999999999999e-06 2024-09-14T06:30:36.450 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:36.450 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:36.451 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:30:36.451 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.086046+0000", 2024-09-14T06:30:36.451 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.9440000000000005e-06 2024-09-14T06:30:36.451 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:36.451 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:36.451 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:30:36.451 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.086050+0000", 2024-09-14T06:30:36.451 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.69e-06 2024-09-14T06:30:36.451 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:36.451 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:36.451 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:30:36.451 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.126080+0000", 2024-09-14T06:30:36.451 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.040030237000000003 2024-09-14T06:30:36.451 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:36.452 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:36.452 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:30:36.452 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.126163+0000", 2024-09-14T06:30:36.452 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.2071999999999996e-05 2024-09-14T06:30:36.452 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:36.452 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:36.452 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:30:36.452 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.126324+0000", 2024-09-14T06:30:36.452 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00016185200000000001 2024-09-14T06:30:36.452 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:36.452 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:36.452 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:36.452 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:36.452 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:36.453 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:55627 3.0 3:06490baf:::rbd_data.112dca2f6ae1.0000000000000030:17 [read 2002944~4096 [fadvise_nocache]] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:30:36.453 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:44.599966+0000", 2024-09-14T06:30:36.453 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 51.832618875000001, 2024-09-14T06:30:36.453 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.041868546999999999, 2024-09-14T06:30:36.453 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:30:36.453 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:30:36.453 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:30:36.453 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:30:36.453 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:30:36.453 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:36.453 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 55627 2024-09-14T06:30:36.453 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:36.453 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:30:36.454 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:36.454 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:30:36.454 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.599966+0000", 2024-09-14T06:30:36.454 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:36.454 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:36.454 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:36.454 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:30:36.454 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.599966+0000", 2024-09-14T06:30:36.454 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:36.454 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:36.454 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:36.454 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:30:36.454 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.599966+0000", 2024-09-14T06:30:36.454 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.0900000000000001e-07 2024-09-14T06:30:36.455 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:36.455 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:36.455 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:30:36.455 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.599967+0000", 2024-09-14T06:30:36.455 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.64e-07 2024-09-14T06:30:36.455 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:36.455 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:36.455 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:30:36.455 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.599970+0000", 2024-09-14T06:30:36.455 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.9670000000000001e-06 2024-09-14T06:30:36.455 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:36.455 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:36.455 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:30:36.455 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.599973+0000", 2024-09-14T06:30:36.455 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.632e-06 2024-09-14T06:30:36.455 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:36.456 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:36.456 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:30:36.456 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.641665+0000", 2024-09-14T06:30:36.456 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.041692547000000003 2024-09-14T06:30:36.456 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:36.456 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:36.456 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:30:36.456 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.641740+0000", 2024-09-14T06:30:36.456 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.4559000000000002e-05 2024-09-14T06:30:36.456 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:36.456 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:36.456 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:30:36.456 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.641835+0000", 2024-09-14T06:30:36.456 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.4668999999999998e-05 2024-09-14T06:30:36.457 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:36.457 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:36.457 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:36.457 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:36.457 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:36.457 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:55628 3.0 3:06490baf:::rbd_data.112dca2f6ae1.0000000000000030:17 [read 2027520~4096 [fadvise_nocache]] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:30:36.457 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:44.599976+0000", 2024-09-14T06:30:36.457 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 51.832609087999998, 2024-09-14T06:30:36.457 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.041991621, 2024-09-14T06:30:36.457 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:30:36.457 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:30:36.457 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:30:36.457 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:30:36.457 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:30:36.458 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:36.458 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 55628 2024-09-14T06:30:36.458 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:36.458 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:30:36.458 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:36.458 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:30:36.458 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.599976+0000", 2024-09-14T06:30:36.458 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:36.458 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:36.458 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:36.458 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:30:36.458 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.599976+0000", 2024-09-14T06:30:36.458 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:36.458 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:36.458 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:36.459 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:30:36.459 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.599976+0000", 2024-09-14T06:30:36.459 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.1299999999999999e-07 2024-09-14T06:30:36.459 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:36.459 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:36.459 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:30:36.459 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.599977+0000", 2024-09-14T06:30:36.459 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.7999999999999993e-07 2024-09-14T06:30:36.459 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:36.459 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:36.459 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:30:36.459 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.599980+0000", 2024-09-14T06:30:36.459 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.01e-06 2024-09-14T06:30:36.459 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:36.459 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:36.460 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:30:36.460 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.599983+0000", 2024-09-14T06:30:36.460 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.5409999999999999e-06 2024-09-14T06:30:36.460 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:36.460 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:36.460 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:30:36.460 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.641869+0000", 2024-09-14T06:30:36.460 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.041886235000000001 2024-09-14T06:30:36.460 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:36.460 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:36.460 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:30:36.460 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.641913+0000", 2024-09-14T06:30:36.460 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.4357000000000002e-05 2024-09-14T06:30:36.461 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:36.461 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:36.461 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:30:36.461 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.641967+0000", 2024-09-14T06:30:36.461 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.4284999999999999e-05 2024-09-14T06:30:36.461 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:36.461 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:36.461 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:36.461 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:36.461 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:36.461 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:55629 3.0 3:06490baf:::rbd_data.112dca2f6ae1.0000000000000030:17 [read 2052096~4096 [fadvise_nocache]] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:30:36.461 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:44.599985+0000", 2024-09-14T06:30:36.462 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 51.832599381999998, 2024-09-14T06:30:36.462 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.042183375000000002, 2024-09-14T06:30:36.462 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:30:36.462 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:30:36.462 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:30:36.462 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:30:36.462 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:30:36.462 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:36.462 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 55629 2024-09-14T06:30:36.462 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:36.462 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:30:36.462 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:36.463 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:30:36.463 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.599985+0000", 2024-09-14T06:30:36.463 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:36.463 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:36.463 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:36.463 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:30:36.463 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.599985+0000", 2024-09-14T06:30:36.463 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:36.463 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:36.463 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:36.463 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:30:36.463 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.599986+0000", 2024-09-14T06:30:36.463 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.05e-07 2024-09-14T06:30:36.464 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:36.464 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:36.464 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:30:36.464 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.599987+0000", 2024-09-14T06:30:36.464 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.8400000000000002e-07 2024-09-14T06:30:36.464 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:36.464 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:36.464 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:30:36.464 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.599990+0000", 2024-09-14T06:30:36.464 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.5640000000000001e-06 2024-09-14T06:30:36.464 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:36.464 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:36.464 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:30:36.464 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.599993+0000", 2024-09-14T06:30:36.465 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.615e-06 2024-09-14T06:30:36.465 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:36.465 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:36.465 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:30:36.465 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.642003+0000", 2024-09-14T06:30:36.465 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.042010345999999997 2024-09-14T06:30:36.465 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:36.465 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:36.465 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:30:36.465 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.642080+0000", 2024-09-14T06:30:36.465 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.6538999999999993e-05 2024-09-14T06:30:36.465 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:36.465 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:36.465 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:30:36.466 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.642169+0000", 2024-09-14T06:30:36.466 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.9122000000000005e-05 2024-09-14T06:30:36.466 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:36.466 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:36.466 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:36.466 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:36.466 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:36.466 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:55630 3.0 3:06490baf:::rbd_data.112dca2f6ae1.0000000000000030:17 [read 2076672~4096 [fadvise_nocache]] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:30:36.466 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:44.599996+0000", 2024-09-14T06:30:36.466 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 51.832588903999998, 2024-09-14T06:30:36.466 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.042299615999999998, 2024-09-14T06:30:36.466 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:30:36.466 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:30:36.466 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:30:36.467 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:30:36.467 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:30:36.467 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:36.467 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 55630 2024-09-14T06:30:36.467 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:36.467 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:30:36.467 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:36.467 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:30:36.467 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.599996+0000", 2024-09-14T06:30:36.467 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:36.467 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:36.467 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:36.467 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:30:36.467 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.599996+0000", 2024-09-14T06:30:36.468 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:36.468 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:36.468 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:36.468 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:30:36.468 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.599996+0000", 2024-09-14T06:30:36.468 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.1500000000000001e-07 2024-09-14T06:30:36.468 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:36.468 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:36.468 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:30:36.468 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.599997+0000", 2024-09-14T06:30:36.468 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.33e-07 2024-09-14T06:30:36.468 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:36.468 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:36.468 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:30:36.469 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600000+0000", 2024-09-14T06:30:36.469 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.0240000000000002e-06 2024-09-14T06:30:36.469 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:36.469 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:36.469 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:30:36.469 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600003+0000", 2024-09-14T06:30:36.469 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.3640000000000002e-06 2024-09-14T06:30:36.469 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:36.469 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:36.469 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:30:36.469 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.642200+0000", 2024-09-14T06:30:36.469 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.042197571000000003 2024-09-14T06:30:36.469 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:36.470 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:36.470 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:30:36.470 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.642243+0000", 2024-09-14T06:30:36.470 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.2765000000000001e-05 2024-09-14T06:30:36.470 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:36.470 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:36.470 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:30:36.470 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.642296+0000", 2024-09-14T06:30:36.470 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.2744000000000002e-05 2024-09-14T06:30:36.470 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:36.470 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:36.470 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:36.470 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:36.471 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:36.471 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:55631 3.0 3:06490baf:::rbd_data.112dca2f6ae1.0000000000000030:17 [read 1048576~4096 [fadvise_nocache]] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:30:36.471 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:44.600006+0000", 2024-09-14T06:30:36.471 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 51.832579316, 2024-09-14T06:30:36.471 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.042487905999999999, 2024-09-14T06:30:36.471 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:30:36.471 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:30:36.471 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:30:36.471 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:30:36.471 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:30:36.471 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:36.471 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 55631 2024-09-14T06:30:36.471 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:36.471 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:30:36.472 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:36.472 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:30:36.472 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600006+0000", 2024-09-14T06:30:36.472 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:36.472 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:36.472 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:36.472 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:30:36.472 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600006+0000", 2024-09-14T06:30:36.472 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:36.472 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:36.472 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:36.472 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:30:36.472 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600006+0000", 2024-09-14T06:30:36.473 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.0800000000000001e-07 2024-09-14T06:30:36.473 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:36.473 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:36.473 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:30:36.473 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600007+0000", 2024-09-14T06:30:36.473 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.7199999999999997e-07 2024-09-14T06:30:36.473 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:36.473 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:36.473 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:30:36.473 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600010+0000", 2024-09-14T06:30:36.473 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.0280000000000001e-06 2024-09-14T06:30:36.473 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:36.473 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:36.473 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:30:36.474 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600012+0000", 2024-09-14T06:30:36.474 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.475e-06 2024-09-14T06:30:36.474 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:36.474 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:36.474 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:30:36.474 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.642329+0000", 2024-09-14T06:30:36.474 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.042317157000000001 2024-09-14T06:30:36.474 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:36.474 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:36.474 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:30:36.474 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.642403+0000", 2024-09-14T06:30:36.474 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.3941000000000002e-05 2024-09-14T06:30:36.474 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:36.474 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:36.475 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:30:36.475 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.642493+0000", 2024-09-14T06:30:36.475 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.0124999999999995e-05 2024-09-14T06:30:36.475 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:36.475 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:36.475 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:36.475 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:36.475 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:36.475 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:55632 3.0 3:06490baf:::rbd_data.112dca2f6ae1.0000000000000030:17 [read 1073152~4096 [fadvise_nocache]] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:30:36.475 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:44.600015+0000", 2024-09-14T06:30:36.475 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 51.832569563, 2024-09-14T06:30:36.475 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.042604922000000003, 2024-09-14T06:30:36.475 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:30:36.475 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:30:36.476 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:30:36.476 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:30:36.476 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:30:36.476 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:36.476 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 55632 2024-09-14T06:30:36.476 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:36.476 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:30:36.476 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:36.476 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:30:36.476 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600015+0000", 2024-09-14T06:30:36.476 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:36.476 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:36.477 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:36.477 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:30:36.477 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600015+0000", 2024-09-14T06:30:36.477 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:36.477 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:36.477 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:36.477 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:30:36.477 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600016+0000", 2024-09-14T06:30:36.477 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.1e-07 2024-09-14T06:30:36.477 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:36.477 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:36.477 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:30:36.478 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600017+0000", 2024-09-14T06:30:36.478 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.9900000000000009e-07 2024-09-14T06:30:36.478 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:36.478 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:36.478 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:30:36.478 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600020+0000", 2024-09-14T06:30:36.478 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.4170000000000001e-06 2024-09-14T06:30:36.478 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:36.478 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:36.478 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:30:36.478 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600023+0000", 2024-09-14T06:30:36.478 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.588e-06 2024-09-14T06:30:36.479 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:36.479 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:36.479 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:30:36.479 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.642526+0000", 2024-09-14T06:30:36.479 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.042503236999999999 2024-09-14T06:30:36.479 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:36.479 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:36.479 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:30:36.479 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.642568+0000", 2024-09-14T06:30:36.479 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.1925000000000003e-05 2024-09-14T06:30:36.479 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:36.479 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:36.479 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:30:36.479 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.642620+0000", 2024-09-14T06:30:36.480 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.2546000000000001e-05 2024-09-14T06:30:36.480 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:36.480 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:36.480 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:36.480 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:36.480 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:36.480 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:55633 3.0 3:06490baf:::rbd_data.112dca2f6ae1.0000000000000030:17 [read 1097728~4096 [fadvise_nocache]] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:30:36.480 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:44.600026+0000", 2024-09-14T06:30:36.480 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 51.832559293999999, 2024-09-14T06:30:36.480 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.042797474000000002, 2024-09-14T06:30:36.480 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:30:36.480 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:30:36.480 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:30:36.481 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:30:36.481 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:30:36.481 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:36.481 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 55633 2024-09-14T06:30:36.481 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:36.481 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:30:36.481 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:36.481 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:30:36.481 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600026+0000", 2024-09-14T06:30:36.481 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:36.481 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:36.481 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:36.481 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:30:36.481 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600026+0000", 2024-09-14T06:30:36.482 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:36.482 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:36.482 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:36.482 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:30:36.482 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600026+0000", 2024-09-14T06:30:36.482 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.3099999999999999e-07 2024-09-14T06:30:36.482 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:36.482 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:36.482 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:30:36.482 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600029+0000", 2024-09-14T06:30:36.482 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.304e-06 2024-09-14T06:30:36.482 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:36.482 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:36.482 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:30:36.482 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600034+0000", 2024-09-14T06:30:36.483 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.1429999999999997e-06 2024-09-14T06:30:36.483 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:36.483 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:36.483 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:30:36.483 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600038+0000", 2024-09-14T06:30:36.483 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.692e-06 2024-09-14T06:30:36.483 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:36.483 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:36.483 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:30:36.483 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.642654+0000", 2024-09-14T06:30:36.483 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.042615564000000002 2024-09-14T06:30:36.483 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:36.483 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:36.483 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:30:36.484 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.642728+0000", 2024-09-14T06:30:36.484 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.4018999999999997e-05 2024-09-14T06:30:36.484 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:36.484 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:36.484 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:30:36.484 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.642823+0000", 2024-09-14T06:30:36.484 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.5520999999999996e-05 2024-09-14T06:30:36.484 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:36.484 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:36.484 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:36.484 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:36.484 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:36.484 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:55634 3.0 3:06490baf:::rbd_data.112dca2f6ae1.0000000000000030:17 [read 1122304~4096 [fadvise_nocache]] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:30:36.485 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:44.600042+0000", 2024-09-14T06:30:36.485 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 51.832542816999997, 2024-09-14T06:30:36.485 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.042903729000000002, 2024-09-14T06:30:36.485 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:30:36.485 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:30:36.485 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:30:36.485 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:30:36.485 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:30:36.485 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:36.485 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 55634 2024-09-14T06:30:36.485 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:36.485 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:30:36.485 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:36.486 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:30:36.486 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600042+0000", 2024-09-14T06:30:36.486 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:36.486 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:36.486 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:36.486 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:30:36.486 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600042+0000", 2024-09-14T06:30:36.486 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:36.486 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:36.486 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:36.486 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:30:36.486 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600043+0000", 2024-09-14T06:30:36.486 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.7000000000000005e-07 2024-09-14T06:30:36.486 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:36.487 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:36.487 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:30:36.487 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600044+0000", 2024-09-14T06:30:36.487 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.742e-06 2024-09-14T06:30:36.487 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:36.487 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:36.487 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:30:36.487 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600050+0000", 2024-09-14T06:30:36.487 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.0800000000000002e-06 2024-09-14T06:30:36.487 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:36.487 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:36.487 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:30:36.487 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600055+0000", 2024-09-14T06:30:36.487 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.3889999999999998e-06 2024-09-14T06:30:36.488 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:36.488 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:36.488 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:30:36.488 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.642845+0000", 2024-09-14T06:30:36.488 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.042790040000000001 2024-09-14T06:30:36.488 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:36.488 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:36.488 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:30:36.488 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.642888+0000", 2024-09-14T06:30:36.488 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.2790000000000002e-05 2024-09-14T06:30:36.488 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:36.488 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:36.488 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:30:36.488 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.642946+0000", 2024-09-14T06:30:36.489 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.8118000000000002e-05 2024-09-14T06:30:36.489 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:36.489 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:36.489 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:36.489 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:36.489 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:36.489 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:55635 3.0 3:06490baf:::rbd_data.112dca2f6ae1.0000000000000030:17 [read 1146880~4096 [fadvise_nocache]] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:30:36.489 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:44.600061+0000", 2024-09-14T06:30:36.489 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 51.832524245999998, 2024-09-14T06:30:36.489 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.043092695, 2024-09-14T06:30:36.489 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:30:36.489 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:30:36.489 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:30:36.490 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:30:36.490 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:30:36.490 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:36.490 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 55635 2024-09-14T06:30:36.490 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:36.490 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:30:36.490 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:36.490 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:30:36.490 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600061+0000", 2024-09-14T06:30:36.490 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:36.490 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:36.490 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:36.490 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:30:36.490 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600061+0000", 2024-09-14T06:30:36.491 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:36.491 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:36.491 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:36.491 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:30:36.491 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600061+0000", 2024-09-14T06:30:36.491 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.3099999999999999e-07 2024-09-14T06:30:36.491 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:36.491 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:36.491 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:30:36.491 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600063+0000", 2024-09-14T06:30:36.491 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.6220000000000001e-06 2024-09-14T06:30:36.491 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:36.491 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:36.492 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:30:36.492 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600068+0000", 2024-09-14T06:30:36.492 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.4790000000000004e-06 2024-09-14T06:30:36.492 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:36.492 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:36.492 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:30:36.492 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600072+0000", 2024-09-14T06:30:36.492 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.3100000000000002e-06 2024-09-14T06:30:36.492 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:36.492 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:36.492 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:30:36.492 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.642987+0000", 2024-09-14T06:30:36.492 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.042914489 2024-09-14T06:30:36.493 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:36.493 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:36.493 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:30:36.493 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.643062+0000", 2024-09-14T06:30:36.493 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.5011000000000004e-05 2024-09-14T06:30:36.493 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:36.493 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:36.493 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:30:36.493 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.643153+0000", 2024-09-14T06:30:36.493 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.1452999999999993e-05 2024-09-14T06:30:36.493 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:36.493 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:36.493 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:36.494 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:36.494 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:36.494 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:55636 3.0 3:06490baf:::rbd_data.112dca2f6ae1.0000000000000030:17 [read 1171456~4096 [fadvise_nocache]] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:30:36.494 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:44.600078+0000", 2024-09-14T06:30:36.494 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 51.832506834, 2024-09-14T06:30:36.494 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.043207966, 2024-09-14T06:30:36.494 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:30:36.494 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:30:36.494 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:30:36.494 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:30:36.494 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:30:36.494 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:36.494 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 55636 2024-09-14T06:30:36.494 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:36.495 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:30:36.495 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:36.495 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:30:36.495 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600078+0000", 2024-09-14T06:30:36.495 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:36.495 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:36.495 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:36.495 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:30:36.495 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600078+0000", 2024-09-14T06:30:36.495 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:36.495 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:36.495 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:36.495 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:30:36.496 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600078+0000", 2024-09-14T06:30:36.496 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.34e-07 2024-09-14T06:30:36.496 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:36.496 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:36.496 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:30:36.496 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600080+0000", 2024-09-14T06:30:36.496 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.641e-06 2024-09-14T06:30:36.496 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:36.496 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:36.496 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:30:36.496 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600086+0000", 2024-09-14T06:30:36.496 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.3860000000000004e-06 2024-09-14T06:30:36.496 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:36.496 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:36.497 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:30:36.497 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600091+0000", 2024-09-14T06:30:36.497 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.5970000000000002e-06 2024-09-14T06:30:36.497 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:36.497 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:36.497 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:30:36.497 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.643184+0000", 2024-09-14T06:30:36.497 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.043092893 2024-09-14T06:30:36.497 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:36.497 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:36.497 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:30:36.497 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.643228+0000", 2024-09-14T06:30:36.497 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.4536999999999997e-05 2024-09-14T06:30:36.497 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:36.498 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:36.498 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:30:36.498 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.643286+0000", 2024-09-14T06:30:36.498 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.7578000000000003e-05 2024-09-14T06:30:36.498 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:36.498 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:36.498 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:36.498 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:36.498 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:36.498 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:55637 3.0 3:06490baf:::rbd_data.112dca2f6ae1.0000000000000030:17 [read 1196032~4096 [fadvise_nocache]] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:30:36.498 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:44.600097+0000", 2024-09-14T06:30:36.498 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 51.832488269999999, 2024-09-14T06:30:36.499 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.043399124999999997, 2024-09-14T06:30:36.499 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:30:36.499 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:30:36.499 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:30:36.499 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:30:36.499 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:30:36.499 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:36.499 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 55637 2024-09-14T06:30:36.499 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:36.499 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:30:36.499 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:36.499 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:30:36.499 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600097+0000", 2024-09-14T06:30:36.499 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:36.500 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:36.500 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:36.500 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:30:36.500 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600097+0000", 2024-09-14T06:30:36.500 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:36.500 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:36.500 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:36.500 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:30:36.500 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.604372+0000", 2024-09-14T06:30:36.500 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0042751489999999998 2024-09-14T06:30:36.500 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:36.500 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:36.500 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:30:36.501 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.604377+0000", 2024-09-14T06:30:36.501 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.7899999999999999e-06 2024-09-14T06:30:36.501 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:36.501 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:36.501 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:30:36.501 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.604389+0000", 2024-09-14T06:30:36.501 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.2197e-05 2024-09-14T06:30:36.501 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:36.501 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:36.501 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:30:36.501 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.604397+0000", 2024-09-14T06:30:36.501 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.8730000000000007e-06 2024-09-14T06:30:36.501 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:36.501 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:36.502 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:30:36.502 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.643324+0000", 2024-09-14T06:30:36.502 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.038926897000000002 2024-09-14T06:30:36.502 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:36.502 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:36.502 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:30:36.502 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.643402+0000", 2024-09-14T06:30:36.502 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.8063000000000001e-05 2024-09-14T06:30:36.502 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:36.502 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:36.502 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:30:36.502 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.643496+0000", 2024-09-14T06:30:36.502 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.4155999999999998e-05 2024-09-14T06:30:36.503 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:36.503 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:36.503 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:36.503 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:36.503 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:36.503 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:55657 3.0 3:06490baf:::rbd_data.112dca2f6ae1.0000000000000030:17 [read 1687552~4096 [fadvise_nocache]] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:30:36.503 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:44.604841+0000", 2024-09-14T06:30:36.503 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 51.827743540999997, 2024-09-14T06:30:36.503 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.041890087999999999, 2024-09-14T06:30:36.503 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:30:36.503 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:30:36.503 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:30:36.503 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:30:36.503 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:30:36.504 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:36.504 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 55657 2024-09-14T06:30:36.504 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:36.504 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:30:36.504 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:36.504 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:30:36.504 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.604841+0000", 2024-09-14T06:30:36.504 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:36.504 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:36.504 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:36.504 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:30:36.504 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.604841+0000", 2024-09-14T06:30:36.504 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:36.505 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:36.505 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:36.505 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:30:36.505 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.604842+0000", 2024-09-14T06:30:36.505 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.0900000000000001e-07 2024-09-14T06:30:36.505 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:36.505 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:36.505 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:30:36.505 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.604846+0000", 2024-09-14T06:30:36.505 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.5759999999999999e-06 2024-09-14T06:30:36.505 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:36.505 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:36.505 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:30:36.506 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.604850+0000", 2024-09-14T06:30:36.506 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.405e-06 2024-09-14T06:30:36.506 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:36.506 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:36.506 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:30:36.506 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.604852+0000", 2024-09-14T06:30:36.506 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.8590000000000001e-06 2024-09-14T06:30:36.506 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:36.506 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:36.506 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:30:36.506 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.646633+0000", 2024-09-14T06:30:36.506 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.041780772000000001 2024-09-14T06:30:36.506 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:36.506 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:36.507 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:30:36.507 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.646677+0000", 2024-09-14T06:30:36.507 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.4115e-05 2024-09-14T06:30:36.507 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:36.507 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:36.507 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:30:36.507 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.646731+0000", 2024-09-14T06:30:36.507 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.4151999999999999e-05 2024-09-14T06:30:36.507 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:36.507 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:36.507 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:36.507 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:36.507 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:36.507 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:55658 3.0 3:06490baf:::rbd_data.112dca2f6ae1.0000000000000030:17 [read 1712128~4096 [fadvise_nocache]] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:30:36.508 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:44.604856+0000", 2024-09-14T06:30:36.508 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 51.827728940999997, 2024-09-14T06:30:36.508 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.041986227000000001, 2024-09-14T06:30:36.508 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:30:36.508 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:30:36.508 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:30:36.508 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:30:36.508 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:30:36.508 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:36.508 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 55658 2024-09-14T06:30:36.508 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:36.508 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:30:36.508 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:36.509 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:30:36.509 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.604856+0000", 2024-09-14T06:30:36.509 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:36.509 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:36.509 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:36.509 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:30:36.509 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.604856+0000", 2024-09-14T06:30:36.509 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:36.509 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:36.509 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:36.509 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:30:36.509 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.604856+0000", 2024-09-14T06:30:36.509 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.23e-07 2024-09-14T06:30:36.509 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:36.510 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:36.510 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:30:36.510 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.604857+0000", 2024-09-14T06:30:36.510 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.9999999999999995e-07 2024-09-14T06:30:36.510 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:36.510 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:36.510 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:30:36.510 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.604861+0000", 2024-09-14T06:30:36.510 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.9890000000000003e-06 2024-09-14T06:30:36.510 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:36.510 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:36.510 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:30:36.510 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.604864+0000", 2024-09-14T06:30:36.511 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.7429999999999998e-06 2024-09-14T06:30:36.511 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:36.511 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:36.511 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:30:36.511 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.646744+0000", 2024-09-14T06:30:36.511 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.041880325000000003 2024-09-14T06:30:36.511 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:36.511 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:36.511 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:30:36.511 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.646784+0000", 2024-09-14T06:30:36.511 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.9604000000000002e-05 2024-09-14T06:30:36.511 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:36.511 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:36.511 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:30:36.511 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.646842+0000", 2024-09-14T06:30:36.512 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.8343e-05 2024-09-14T06:30:36.512 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:36.512 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:36.512 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:36.512 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:36.512 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:36.512 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:55659 3.0 3:06490baf:::rbd_data.112dca2f6ae1.0000000000000030:17 [read 1736704~4096 [fadvise_nocache]] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:30:36.512 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:44.604867+0000", 2024-09-14T06:30:36.512 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 51.827717788000001, 2024-09-14T06:30:36.512 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.042097082000000001, 2024-09-14T06:30:36.512 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:30:36.512 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:30:36.512 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:30:36.512 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:30:36.513 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:30:36.513 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:36.513 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 55659 2024-09-14T06:30:36.513 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:36.513 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:30:36.513 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:36.513 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:30:36.513 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.604867+0000", 2024-09-14T06:30:36.513 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:36.513 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:36.513 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:36.513 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:30:36.513 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.604867+0000", 2024-09-14T06:30:36.513 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:36.514 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:36.514 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:36.514 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:30:36.514 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.604867+0000", 2024-09-14T06:30:36.514 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.2000000000000001e-07 2024-09-14T06:30:36.514 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:36.514 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:36.514 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:30:36.514 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.604868+0000", 2024-09-14T06:30:36.514 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.04e-06 2024-09-14T06:30:36.514 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:36.514 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:36.514 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:30:36.514 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.604871+0000", 2024-09-14T06:30:36.515 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.9689999999999999e-06 2024-09-14T06:30:36.515 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:36.515 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:36.515 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:30:36.515 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.604874+0000", 2024-09-14T06:30:36.515 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.5069999999999999e-06 2024-09-14T06:30:36.515 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:36.515 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:36.515 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:30:36.515 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.646874+0000", 2024-09-14T06:30:36.515 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.042000513000000003 2024-09-14T06:30:36.515 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:36.515 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:36.515 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:30:36.515 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.646914+0000", 2024-09-14T06:30:36.516 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.9443999999999997e-05 2024-09-14T06:30:36.516 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:36.516 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:36.516 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:30:36.516 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.646964+0000", 2024-09-14T06:30:36.516 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.0389000000000002e-05 2024-09-14T06:30:36.516 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:36.516 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:36.516 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:36.516 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:36.516 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:36.516 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:55660 3.0 3:06490baf:::rbd_data.112dca2f6ae1.0000000000000030:17 [read 1761280~4096 [fadvise_nocache]] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:30:36.516 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:44.604877+0000", 2024-09-14T06:30:36.516 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 51.827708039000001, 2024-09-14T06:30:36.516 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.042196349000000001, 2024-09-14T06:30:36.517 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:30:36.517 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:30:36.517 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:30:36.517 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:30:36.517 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:30:36.517 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:36.517 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 55660 2024-09-14T06:30:36.517 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:36.517 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:30:36.517 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:36.517 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:30:36.517 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.604877+0000", 2024-09-14T06:30:36.517 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:36.517 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:36.518 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:36.518 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:30:36.518 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.604877+0000", 2024-09-14T06:30:36.518 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:36.518 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:36.518 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:36.518 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:30:36.518 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.605810+0000", 2024-09-14T06:30:36.518 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00093336899999999997 2024-09-14T06:30:36.518 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:36.518 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:36.518 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:30:36.518 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.605815+0000", 2024-09-14T06:30:36.519 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.6140000000000002e-06 2024-09-14T06:30:36.519 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:36.519 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:36.519 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:30:36.519 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.605825+0000", 2024-09-14T06:30:36.519 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.8889999999999997e-06 2024-09-14T06:30:36.519 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:36.519 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:36.519 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:30:36.519 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.605831+0000", 2024-09-14T06:30:36.519 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.9800000000000003e-06 2024-09-14T06:30:36.519 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:36.519 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:36.519 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:30:36.520 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.646989+0000", 2024-09-14T06:30:36.520 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.041157878000000002 2024-09-14T06:30:36.520 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:36.520 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:36.520 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:30:36.520 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.647026+0000", 2024-09-14T06:30:36.520 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.7305000000000003e-05 2024-09-14T06:30:36.520 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:36.520 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:36.520 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:30:36.520 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.647073+0000", 2024-09-14T06:30:36.520 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.7314000000000001e-05 2024-09-14T06:30:36.521 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:36.521 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:36.521 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:36.521 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:36.521 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:36.521 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:55666 3.0 3:06490baf:::rbd_data.112dca2f6ae1.0000000000000030:17 [read 1908736~4096 [fadvise_nocache]] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:30:36.521 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:44.605898+0000", 2024-09-14T06:30:36.521 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 51.826687292999999, 2024-09-14T06:30:36.521 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.041862887000000001, 2024-09-14T06:30:36.521 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:30:36.521 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:30:36.521 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:30:36.521 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:30:36.521 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:30:36.522 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:36.522 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 55666 2024-09-14T06:30:36.522 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:36.522 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:30:36.522 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:36.522 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:30:36.522 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.605898+0000", 2024-09-14T06:30:36.522 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:36.522 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:36.522 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:36.522 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:30:36.522 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.605898+0000", 2024-09-14T06:30:36.522 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:36.522 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:36.522 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:36.523 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:30:36.523 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.611874+0000", 2024-09-14T06:30:36.523 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0059764049999999997 2024-09-14T06:30:36.523 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:36.523 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:36.523 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:30:36.523 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.611879+0000", 2024-09-14T06:30:36.523 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.8749999999999999e-06 2024-09-14T06:30:36.523 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:36.523 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:36.523 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:30:36.523 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.611887+0000", 2024-09-14T06:30:36.523 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.4109999999999996e-06 2024-09-14T06:30:36.524 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:36.524 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:36.524 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:30:36.524 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.611893+0000", 2024-09-14T06:30:36.524 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.2959999999999998e-06 2024-09-14T06:30:36.524 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:36.524 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:36.524 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:30:36.524 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.647674+0000", 2024-09-14T06:30:36.524 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.035781281999999998 2024-09-14T06:30:36.524 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:36.524 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:36.524 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:30:36.525 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.647711+0000", 2024-09-14T06:30:36.525 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.7434000000000001e-05 2024-09-14T06:30:36.525 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:36.525 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:36.525 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:30:36.525 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.647760+0000", 2024-09-14T06:30:36.525 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.9184000000000002e-05 2024-09-14T06:30:36.525 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:36.525 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:36.525 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:36.525 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:36.525 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:36.525 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-14T06:30:36.527 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-14T06:30:36.528 INFO:tasks.ceph.osd.7.smithi138.stderr:2024-09-14T06:30:36.525+0000 7f3c6a21d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33623) UID: 0 2024-09-14T06:30:36.587 INFO:teuthology.orchestra.run.smithi094.stderr:noscrub is set 2024-09-14T06:30:36.627 INFO:tasks.ceph.osd.2.smithi094.stderr:2024-09-14T06:30:36.625+0000 7faf3269b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33649) UID: 0 2024-09-14T06:30:36.635 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-14T06:30:36.635 INFO:teuthology.orchestra.run.smithi094.stdout: "ops": [], 2024-09-14T06:30:36.635 INFO:teuthology.orchestra.run.smithi094.stdout: "num_ops": 0 2024-09-14T06:30:36.635 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-14T06:30:36.643 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-14T06:30:36.684 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-14T06:30:36.727 INFO:tasks.ceph.osd.3.smithi094.stderr:2024-09-14T06:30:36.725+0000 7feeeafe8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33655) UID: 0 2024-09-14T06:30:36.790 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-14T06:30:36.791 INFO:teuthology.orchestra.run.smithi094.stdout: "ops": [], 2024-09-14T06:30:36.791 INFO:teuthology.orchestra.run.smithi094.stdout: "complaint_time": 30, 2024-09-14T06:30:36.791 INFO:teuthology.orchestra.run.smithi094.stdout: "num_blocked_ops": 0 2024-09-14T06:30:36.791 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-14T06:30:36.800 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-14T06:30:36.807 INFO:teuthology.orchestra.run.smithi094.stdout:osd.0: {} 2024-09-14T06:30:36.807 INFO:teuthology.orchestra.run.smithi094.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-14T06:30:36.826 INFO:teuthology.orchestra.run.smithi094.stdout:osd.1: {} 2024-09-14T06:30:36.827 INFO:teuthology.orchestra.run.smithi094.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-14T06:30:36.828 INFO:tasks.ceph.osd.3.smithi094.stderr:2024-09-14T06:30:36.826+0000 7feeeafe8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33655) UID: 0 2024-09-14T06:30:36.843 INFO:teuthology.orchestra.run.smithi094.stdout:osd.2: {} 2024-09-14T06:30:36.843 INFO:teuthology.orchestra.run.smithi094.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-14T06:30:36.860 INFO:teuthology.orchestra.run.smithi094.stdout:osd.3: {} 2024-09-14T06:30:36.860 INFO:teuthology.orchestra.run.smithi094.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-14T06:30:36.877 INFO:teuthology.orchestra.run.smithi094.stdout:osd.4: {} 2024-09-14T06:30:36.877 INFO:teuthology.orchestra.run.smithi094.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-14T06:30:36.894 INFO:teuthology.orchestra.run.smithi094.stdout:osd.5: {} 2024-09-14T06:30:36.894 INFO:teuthology.orchestra.run.smithi094.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-14T06:30:36.911 INFO:teuthology.orchestra.run.smithi094.stdout:osd.6: {} 2024-09-14T06:30:36.911 INFO:teuthology.orchestra.run.smithi094.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-14T06:30:36.913 INFO:teuthology.orchestra.run.smithi094.stdout:ERROR: (22) Invalid argument 2024-09-14T06:30:36.914 INFO:teuthology.orchestra.run.smithi094.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:30:36.923 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-14T06:30:36.928 INFO:teuthology.orchestra.run.smithi094.stdout:osd.7: {} 2024-09-14T06:30:36.929 INFO:teuthology.orchestra.run.smithi094.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-14T06:30:36.929 INFO:tasks.ceph.osd.0.smithi094.stderr:2024-09-14T06:30:36.927+0000 7f07c9333640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33636) UID: 0 2024-09-14T06:30:37.029 INFO:tasks.ceph.osd.4.smithi138.stderr:2024-09-14T06:30:37.028+0000 7f6c653c8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33615) UID: 0 2024-09-14T06:30:37.030 INFO:teuthology.orchestra.run.smithi094.stdout:ERROR: (22) Invalid argument 2024-09-14T06:30:37.030 INFO:teuthology.orchestra.run.smithi094.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:30:37.038 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-14T06:30:37.131 INFO:tasks.ceph.osd.0.smithi094.stderr:2024-09-14T06:30:37.129+0000 7f07c9333640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33636) UID: 0 2024-09-14T06:30:37.141 INFO:teuthology.orchestra.run.smithi094.stdout:ERROR: (22) Invalid argument 2024-09-14T06:30:37.141 INFO:teuthology.orchestra.run.smithi094.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:30:37.149 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-14T06:30:37.234 INFO:tasks.ceph.osd.2.smithi094.stderr:2024-09-14T06:30:37.232+0000 7faf3269b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33649) UID: 0 2024-09-14T06:30:37.293 INFO:teuthology.orchestra.run.smithi094.stdout:ERROR: (22) Invalid argument 2024-09-14T06:30:37.293 INFO:teuthology.orchestra.run.smithi094.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:30:37.301 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-14T06:30:37.342 INFO:tasks.ceph.osd.2.smithi094.stderr:2024-09-14T06:30:37.341+0000 7faf3269b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33649) UID: 0 2024-09-14T06:30:37.415 INFO:teuthology.orchestra.run.smithi138.stdout:ERROR: (22) Invalid argument 2024-09-14T06:30:37.416 INFO:teuthology.orchestra.run.smithi138.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:30:37.424 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-14T06:30:37.469 INFO:tasks.ceph.osd.6.smithi138.stderr:2024-09-14T06:30:37.467+0000 7f8cd8849640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33620) UID: 0 2024-09-14T06:30:37.528 INFO:teuthology.orchestra.run.smithi138.stdout:ERROR: (22) Invalid argument 2024-09-14T06:30:37.528 INFO:teuthology.orchestra.run.smithi138.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:30:37.535 INFO:tasks.ceph.osd.1.smithi094.stderr:2024-09-14T06:30:37.533+0000 7f328e47a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33641) UID: 0 2024-09-14T06:30:37.537 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-14T06:30:37.636 INFO:tasks.ceph.osd.4.smithi138.stderr:2024-09-14T06:30:37.634+0000 7f6c653c8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33615) UID: 0 2024-09-14T06:30:37.733 INFO:teuthology.orchestra.run.smithi138.stdout:ERROR: (22) Invalid argument 2024-09-14T06:30:37.733 INFO:teuthology.orchestra.run.smithi138.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:30:37.736 INFO:tasks.ceph.osd.1.smithi094.stderr:2024-09-14T06:30:37.734+0000 7f328e47a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33641) UID: 0 2024-09-14T06:30:37.745 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-14T06:30:37.837 INFO:tasks.ceph.osd.4.smithi138.stderr:2024-09-14T06:30:37.835+0000 7f6c653c8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33615) UID: 0 2024-09-14T06:30:37.913 INFO:teuthology.orchestra.run.smithi138.stdout:ERROR: (22) Invalid argument 2024-09-14T06:30:37.913 INFO:teuthology.orchestra.run.smithi138.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:30:37.922 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-14T06:30:37.938 INFO:tasks.ceph.osd.0.smithi094.stderr:2024-09-14T06:30:37.937+0000 7f07c9333640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33636) UID: 0 2024-09-14T06:30:38.039 INFO:tasks.ceph.osd.1.smithi094.stderr:2024-09-14T06:30:38.038+0000 7f328e47a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33641) UID: 0 2024-09-14T06:30:38.064 INFO:teuthology.orchestra.run.smithi138.stdout:ERROR: (22) Invalid argument 2024-09-14T06:30:38.064 INFO:teuthology.orchestra.run.smithi138.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:30:38.072 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-14T06:30:38.141 INFO:tasks.ceph.osd.6.smithi138.stderr:2024-09-14T06:30:38.139+0000 7f8cd8849640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33620) UID: 0 2024-09-14T06:30:38.175 INFO:teuthology.orchestra.run.smithi138.stdout:ERROR: (22) Invalid argument 2024-09-14T06:30:38.175 INFO:teuthology.orchestra.run.smithi138.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:30:38.184 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-14T06:30:38.268 INFO:tasks.ceph.osd.7.smithi138.stderr:2024-09-14T06:30:38.266+0000 7f3c6a21d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33623) UID: 0 2024-09-14T06:30:38.328 INFO:teuthology.orchestra.run.smithi138.stdout:ERROR: (22) Invalid argument 2024-09-14T06:30:38.328 INFO:teuthology.orchestra.run.smithi138.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:30:38.337 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-14T06:30:38.343 INFO:tasks.ceph.osd.2.smithi094.stderr:2024-09-14T06:30:38.342+0000 7faf3269b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33649) UID: 0 2024-09-14T06:30:38.444 INFO:tasks.ceph.osd.3.smithi094.stderr:2024-09-14T06:30:38.443+0000 7feeeafe8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33655) UID: 0 2024-09-14T06:30:38.480 INFO:teuthology.orchestra.run.smithi138.stdout:ERROR: (22) Invalid argument 2024-09-14T06:30:38.480 INFO:teuthology.orchestra.run.smithi138.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:30:38.489 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-14T06:30:38.545 INFO:tasks.ceph.osd.1.smithi094.stderr:2024-09-14T06:30:38.543+0000 7f328e47a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33641) UID: 0 2024-09-14T06:30:38.592 INFO:teuthology.orchestra.run.smithi138.stdout:ERROR: (22) Invalid argument 2024-09-14T06:30:38.592 INFO:teuthology.orchestra.run.smithi138.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:30:38.601 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-14T06:30:38.603 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-14T06:30:38.647 INFO:tasks.ceph.osd.1.smithi094.stderr:2024-09-14T06:30:38.645+0000 7f328e47a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33641) UID: 0 2024-09-14T06:30:38.708 INFO:teuthology.orchestra.run.smithi138.stdout:ERROR: (22) Invalid argument 2024-09-14T06:30:38.709 INFO:teuthology.orchestra.run.smithi138.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:30:38.717 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-14T06:30:38.761 INFO:tasks.ceph.osd.5.smithi138.stderr:2024-09-14T06:30:38.759+0000 7fe13746f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33618) UID: 0 2024-09-14T06:30:38.808 INFO:teuthology.orchestra.run.smithi094.stdout:checking clone #23, image image_client.0-clone23 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent24 2024-09-14T06:30:38.821 INFO:teuthology.orchestra.run.smithi138.stdout:ERROR: (22) Invalid argument 2024-09-14T06:30:38.821 INFO:teuthology.orchestra.run.smithi138.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:30:38.830 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-14T06:30:38.873 INFO:tasks.ceph.osd.6.smithi138.stderr:2024-09-14T06:30:38.871+0000 7f8cd8849640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33620) UID: 0 2024-09-14T06:30:38.940 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-14T06:30:38.947 INFO:tasks.ceph.osd.7.smithi138.stderr:2024-09-14T06:30:38.945+0000 7f3c6a21d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33623) UID: 0 2024-09-14T06:30:38.976 INFO:teuthology.orchestra.run.smithi138.stdout:ERROR: (22) Invalid argument 2024-09-14T06:30:38.976 INFO:teuthology.orchestra.run.smithi138.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:30:38.985 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-14T06:30:39.049 INFO:tasks.ceph.osd.1.smithi094.stderr:2024-09-14T06:30:39.047+0000 7f328e47a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33641) UID: 0 2024-09-14T06:30:39.063 INFO:teuthology.orchestra.run.smithi094.stdout:osd.0: {} 2024-09-14T06:30:39.064 INFO:teuthology.orchestra.run.smithi094.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-14T06:30:39.082 INFO:teuthology.orchestra.run.smithi094.stdout:osd.1: {} 2024-09-14T06:30:39.082 INFO:teuthology.orchestra.run.smithi094.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-14T06:30:39.093 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-14T06:30:39.093 INFO:teuthology.orchestra.run.smithi094.stdout: "ops": [], 2024-09-14T06:30:39.093 INFO:teuthology.orchestra.run.smithi094.stdout: "num_ops": 0 2024-09-14T06:30:39.093 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-14T06:30:39.098 INFO:teuthology.orchestra.run.smithi094.stdout:osd.2: {} 2024-09-14T06:30:39.098 INFO:teuthology.orchestra.run.smithi094.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-14T06:30:39.102 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-14T06:30:39.114 INFO:teuthology.orchestra.run.smithi094.stdout:osd.3: {} 2024-09-14T06:30:39.115 INFO:teuthology.orchestra.run.smithi094.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-14T06:30:39.131 INFO:teuthology.orchestra.run.smithi094.stdout:osd.4: {} 2024-09-14T06:30:39.132 INFO:teuthology.orchestra.run.smithi094.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-14T06:30:39.148 INFO:teuthology.orchestra.run.smithi094.stdout:osd.5: {} 2024-09-14T06:30:39.148 INFO:teuthology.orchestra.run.smithi094.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-14T06:30:39.149 INFO:tasks.ceph.osd.6.smithi138.stderr:2024-09-14T06:30:39.147+0000 7f8cd8849640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33620) UID: 0 2024-09-14T06:30:39.165 INFO:teuthology.orchestra.run.smithi094.stdout:osd.6: {} 2024-09-14T06:30:39.165 INFO:teuthology.orchestra.run.smithi094.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-14T06:30:39.182 INFO:teuthology.orchestra.run.smithi094.stdout:osd.7: {} 2024-09-14T06:30:39.182 INFO:teuthology.orchestra.run.smithi094.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-14T06:30:39.219 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-14T06:30:39.219 INFO:teuthology.orchestra.run.smithi094.stdout: "ops": [], 2024-09-14T06:30:39.219 INFO:teuthology.orchestra.run.smithi094.stdout: "complaint_time": 30, 2024-09-14T06:30:39.219 INFO:teuthology.orchestra.run.smithi094.stdout: "num_blocked_ops": 0 2024-09-14T06:30:39.219 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-14T06:30:39.228 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-14T06:30:39.251 INFO:tasks.ceph.osd.7.smithi138.stderr:2024-09-14T06:30:39.249+0000 7f3c6a21d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33623) UID: 0 2024-09-14T06:30:39.351 INFO:tasks.ceph.osd.1.smithi094.stderr:2024-09-14T06:30:39.349+0000 7f328e47a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33641) UID: 0 2024-09-14T06:30:39.373 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-14T06:30:39.373 INFO:teuthology.orchestra.run.smithi094.stdout: "size": 20, 2024-09-14T06:30:39.374 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 600, 2024-09-14T06:30:39.374 INFO:teuthology.orchestra.run.smithi094.stdout: "ops": [ 2024-09-14T06:30:39.374 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:39.374 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:115193 3.1 3:856be886:::rbd_data.112d4d24c558.0000000000000004:29 [read 1523712~16384 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e96)", 2024-09-14T06:30:39.374 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:30:17.939854+0000", 2024-09-14T06:30:39.374 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 21.431979509000001, 2024-09-14T06:30:39.374 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.073897516999999996, 2024-09-14T06:30:39.374 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:30:39.374 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:30:39.374 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:30:39.374 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:30:39.375 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:30:39.375 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:39.375 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 115193 2024-09-14T06:30:39.375 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:39.375 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:30:39.375 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:39.375 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:30:39.375 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.939854+0000", 2024-09-14T06:30:39.375 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:39.375 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:39.375 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:39.375 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:30:39.375 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.939854+0000", 2024-09-14T06:30:39.376 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:39.376 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:39.376 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:39.376 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:30:39.376 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.939854+0000", 2024-09-14T06:30:39.376 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.9500000000000002e-07 2024-09-14T06:30:39.376 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:39.376 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:39.376 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:30:39.376 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.939856+0000", 2024-09-14T06:30:39.376 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.0959999999999999e-06 2024-09-14T06:30:39.376 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:39.377 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:39.377 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:30:39.377 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.939863+0000", 2024-09-14T06:30:39.377 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.1129999999999997e-06 2024-09-14T06:30:39.377 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:39.377 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:39.377 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:30:39.377 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.939867+0000", 2024-09-14T06:30:39.377 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.8350000000000003e-06 2024-09-14T06:30:39.377 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:39.377 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:39.377 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:30:39.378 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.013626+0000", 2024-09-14T06:30:39.378 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.073758799999999999 2024-09-14T06:30:39.378 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:39.378 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:39.378 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:30:39.378 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.013690+0000", 2024-09-14T06:30:39.378 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.3918999999999996e-05 2024-09-14T06:30:39.378 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:39.378 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:39.378 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:30:39.378 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.013751+0000", 2024-09-14T06:30:39.379 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.1159000000000002e-05 2024-09-14T06:30:39.379 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:39.379 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:39.379 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:39.379 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:39.379 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:39.379 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:115194 3.1 3:856be886:::rbd_data.112d4d24c558.0000000000000004:29 [read 1769472~16384 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e96)", 2024-09-14T06:30:39.379 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:30:17.939874+0000", 2024-09-14T06:30:39.379 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 21.431959198000001, 2024-09-14T06:30:39.379 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.074016988000000006, 2024-09-14T06:30:39.379 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:30:39.380 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:30:39.380 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:30:39.380 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:30:39.380 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:30:39.380 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:39.380 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 115194 2024-09-14T06:30:39.380 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:39.380 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:30:39.380 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:39.380 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:30:39.380 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.939874+0000", 2024-09-14T06:30:39.381 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:39.381 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:39.381 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:39.381 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:30:39.381 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.939874+0000", 2024-09-14T06:30:39.381 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:39.381 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:39.381 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:39.381 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:30:39.381 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.939875+0000", 2024-09-14T06:30:39.381 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.2600000000000002e-07 2024-09-14T06:30:39.381 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:39.382 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:39.382 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:30:39.382 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.939877+0000", 2024-09-14T06:30:39.382 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.9420000000000002e-06 2024-09-14T06:30:39.382 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:39.382 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:39.382 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:30:39.382 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.939882+0000", 2024-09-14T06:30:39.382 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.8749999999999997e-06 2024-09-14T06:30:39.382 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:39.382 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:39.382 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:30:39.383 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.939887+0000", 2024-09-14T06:30:39.383 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.741e-06 2024-09-14T06:30:39.383 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:39.383 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:39.383 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:30:39.383 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.013767+0000", 2024-09-14T06:30:39.383 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.073879720999999995 2024-09-14T06:30:39.383 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:39.383 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:39.383 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:30:39.383 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.013832+0000", 2024-09-14T06:30:39.383 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.5312000000000001e-05 2024-09-14T06:30:39.383 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:39.384 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:39.384 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:30:39.384 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.013891+0000", 2024-09-14T06:30:39.384 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.8870999999999999e-05 2024-09-14T06:30:39.384 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:39.384 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:39.384 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:39.384 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:39.384 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:39.384 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:115195 3.1 3:856be886:::rbd_data.112d4d24c558.0000000000000004:29 [read 2015232~16384 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e96)", 2024-09-14T06:30:39.384 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:30:17.939903+0000", 2024-09-14T06:30:39.385 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 21.431930012999999, 2024-09-14T06:30:39.385 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.074128199000000006, 2024-09-14T06:30:39.385 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:30:39.385 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:30:39.385 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:30:39.385 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:30:39.385 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:30:39.385 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:39.385 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 115195 2024-09-14T06:30:39.385 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:39.385 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:30:39.386 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:39.386 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:30:39.386 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.939903+0000", 2024-09-14T06:30:39.386 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:39.386 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:39.386 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:39.386 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:30:39.386 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.939903+0000", 2024-09-14T06:30:39.386 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:39.386 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:39.386 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:39.386 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:30:39.387 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.939904+0000", 2024-09-14T06:30:39.387 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.5799999999999999e-07 2024-09-14T06:30:39.387 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:39.387 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:39.387 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:30:39.387 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.939906+0000", 2024-09-14T06:30:39.387 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.2350000000000002e-06 2024-09-14T06:30:39.387 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:39.387 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:39.387 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:30:39.387 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.939913+0000", 2024-09-14T06:30:39.388 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.5649999999999998e-06 2024-09-14T06:30:39.388 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:39.388 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:39.388 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:30:39.388 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.939918+0000", 2024-09-14T06:30:39.388 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.784e-06 2024-09-14T06:30:39.388 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:39.388 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:39.388 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:30:39.388 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.013921+0000", 2024-09-14T06:30:39.389 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.074002911000000005 2024-09-14T06:30:39.389 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:39.389 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:39.389 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:30:39.389 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.013971+0000", 2024-09-14T06:30:39.389 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.9551e-05 2024-09-14T06:30:39.389 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:39.389 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:39.389 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:30:39.389 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.014031+0000", 2024-09-14T06:30:39.389 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.0594999999999998e-05 2024-09-14T06:30:39.389 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:39.390 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:39.390 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:39.390 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:39.390 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:39.390 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:115196 3.1 3:856be886:::rbd_data.112d4d24c558.0000000000000004:29 [read 2260992~16384 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e96)", 2024-09-14T06:30:39.390 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:30:17.939925+0000", 2024-09-14T06:30:39.390 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 21.431907820999999, 2024-09-14T06:30:39.390 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.074320137999999994, 2024-09-14T06:30:39.390 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:30:39.390 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:30:39.390 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:30:39.391 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:30:39.391 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:30:39.391 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:39.391 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 115196 2024-09-14T06:30:39.391 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:39.391 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:30:39.391 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:39.391 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:30:39.391 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.939925+0000", 2024-09-14T06:30:39.391 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:39.391 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:39.392 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:39.392 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:30:39.392 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.939925+0000", 2024-09-14T06:30:39.392 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:39.392 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:39.392 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:39.392 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:30:39.392 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.939926+0000", 2024-09-14T06:30:39.392 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.9999999999999998e-07 2024-09-14T06:30:39.392 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:39.392 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:39.392 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:30:39.393 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.939928+0000", 2024-09-14T06:30:39.393 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.9549999999999999e-06 2024-09-14T06:30:39.393 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:39.393 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:39.393 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:30:39.393 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.939934+0000", 2024-09-14T06:30:39.393 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.127e-06 2024-09-14T06:30:39.393 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:39.393 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:39.393 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:30:39.393 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.939939+0000", 2024-09-14T06:30:39.394 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.775e-06 2024-09-14T06:30:39.394 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:39.394 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:39.394 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:30:39.394 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.014075+0000", 2024-09-14T06:30:39.394 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.074136598999999997 2024-09-14T06:30:39.394 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:39.394 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:39.394 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:30:39.394 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.014162+0000", 2024-09-14T06:30:39.394 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.6477999999999994e-05 2024-09-14T06:30:39.394 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:39.395 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:39.395 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:30:39.395 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.014246+0000", 2024-09-14T06:30:39.395 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.3703999999999997e-05 2024-09-14T06:30:39.395 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:39.395 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:39.395 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:39.395 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:39.395 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:39.395 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:115197 3.1 3:856be886:::rbd_data.112d4d24c558.0000000000000004:29 [read 2506752~16384 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e96)", 2024-09-14T06:30:39.395 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:30:17.939969+0000", 2024-09-14T06:30:39.395 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 21.431864336, 2024-09-14T06:30:39.395 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.074528544000000002, 2024-09-14T06:30:39.396 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:30:39.396 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:30:39.396 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:30:39.396 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:30:39.396 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:30:39.396 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:39.396 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 115197 2024-09-14T06:30:39.396 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:39.396 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:30:39.396 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:39.396 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:30:39.396 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.939969+0000", 2024-09-14T06:30:39.397 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:39.397 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:39.397 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:39.397 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:30:39.397 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.939969+0000", 2024-09-14T06:30:39.397 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:39.397 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:39.397 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:39.397 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:30:39.397 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.939970+0000", 2024-09-14T06:30:39.397 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.3600000000000003e-07 2024-09-14T06:30:39.398 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:39.398 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:39.398 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:30:39.398 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.939972+0000", 2024-09-14T06:30:39.398 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.3180000000000001e-06 2024-09-14T06:30:39.398 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:39.398 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:39.398 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:30:39.398 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.939979+0000", 2024-09-14T06:30:39.398 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.8800000000000002e-06 2024-09-14T06:30:39.398 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:39.398 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:39.399 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:30:39.399 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.939984+0000", 2024-09-14T06:30:39.399 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.3510000000000001e-06 2024-09-14T06:30:39.399 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:39.399 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:39.399 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:30:39.399 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.014302+0000", 2024-09-14T06:30:39.399 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.074317416999999997 2024-09-14T06:30:39.399 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:39.399 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:39.399 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:30:39.400 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.014392+0000", 2024-09-14T06:30:39.400 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.0181999999999998e-05 2024-09-14T06:30:39.400 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:39.400 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:39.400 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:30:39.400 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.014497+0000", 2024-09-14T06:30:39.400 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00010576 2024-09-14T06:30:39.400 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:39.400 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:39.400 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:39.400 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:39.400 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:39.401 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:115198 3.1 3:856be886:::rbd_data.112d4d24c558.0000000000000004:29 [read 2752512~16384 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e96)", 2024-09-14T06:30:39.401 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:30:17.940004+0000", 2024-09-14T06:30:39.401 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 21.431829056000002, 2024-09-14T06:30:39.401 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.074736367999999997, 2024-09-14T06:30:39.401 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:30:39.401 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:30:39.401 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:30:39.401 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:30:39.401 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:30:39.401 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:39.401 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 115198 2024-09-14T06:30:39.401 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:39.401 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:30:39.402 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:39.402 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:30:39.402 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940004+0000", 2024-09-14T06:30:39.402 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:39.402 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:39.402 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:39.402 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:30:39.402 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940004+0000", 2024-09-14T06:30:39.402 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:39.402 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:39.402 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:39.402 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:30:39.403 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940005+0000", 2024-09-14T06:30:39.403 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.2200000000000004e-07 2024-09-14T06:30:39.403 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:39.403 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:39.403 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:30:39.403 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940007+0000", 2024-09-14T06:30:39.403 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.322e-06 2024-09-14T06:30:39.403 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:39.403 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:39.403 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:30:39.403 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940014+0000", 2024-09-14T06:30:39.403 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.6220000000000003e-06 2024-09-14T06:30:39.404 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:39.404 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:39.404 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:30:39.404 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940019+0000", 2024-09-14T06:30:39.404 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.6380000000000001e-06 2024-09-14T06:30:39.404 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:39.404 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:39.405 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:30:39.405 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.014562+0000", 2024-09-14T06:30:39.405 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.074542320999999995 2024-09-14T06:30:39.405 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:39.405 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:39.405 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:30:39.405 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.014646+0000", 2024-09-14T06:30:39.405 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.4079999999999995e-05 2024-09-14T06:30:39.405 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:39.405 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:39.405 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:30:39.405 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.014741+0000", 2024-09-14T06:30:39.406 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.4763000000000001e-05 2024-09-14T06:30:39.406 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:39.406 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:39.406 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:39.406 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:39.406 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:39.406 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:115199 3.1 3:856be886:::rbd_data.112d4d24c558.0000000000000004:29 [read 2998272~16384 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e96)", 2024-09-14T06:30:39.406 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:30:17.940040+0000", 2024-09-14T06:30:39.406 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 21.431793300999999, 2024-09-14T06:30:39.406 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.074957134999999994, 2024-09-14T06:30:39.406 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:30:39.407 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:30:39.407 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:30:39.407 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:30:39.407 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:30:39.407 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:39.407 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 115199 2024-09-14T06:30:39.407 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:39.407 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:30:39.407 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:39.407 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:30:39.407 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940040+0000", 2024-09-14T06:30:39.407 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:39.408 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:39.408 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:39.408 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:30:39.408 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940040+0000", 2024-09-14T06:30:39.408 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:39.408 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:39.408 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:39.408 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:30:39.408 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940041+0000", 2024-09-14T06:30:39.408 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.4000000000000001e-07 2024-09-14T06:30:39.408 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:39.408 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:39.409 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:30:39.409 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940046+0000", 2024-09-14T06:30:39.409 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.7250000000000002e-06 2024-09-14T06:30:39.409 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:39.409 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:39.409 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:30:39.409 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940053+0000", 2024-09-14T06:30:39.409 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.6109999999999999e-06 2024-09-14T06:30:39.409 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:39.409 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:39.409 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:30:39.409 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940058+0000", 2024-09-14T06:30:39.410 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.1590000000000001e-06 2024-09-14T06:30:39.410 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:39.410 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:39.410 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:30:39.410 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.014768+0000", 2024-09-14T06:30:39.410 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.074709624000000002 2024-09-14T06:30:39.410 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:39.410 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:39.410 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:30:39.410 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.014907+0000", 2024-09-14T06:30:39.410 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000139665 2024-09-14T06:30:39.411 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:39.411 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:39.411 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:30:39.411 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.014997+0000", 2024-09-14T06:30:39.411 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.9710999999999995e-05 2024-09-14T06:30:39.411 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:39.411 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:39.411 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:39.411 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:39.411 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:39.411 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:115200 3.1 3:856be886:::rbd_data.112d4d24c558.0000000000000004:29 [read 3244032~16384 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e96)", 2024-09-14T06:30:39.411 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:30:17.940080+0000", 2024-09-14T06:30:39.412 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 21.431753096000001, 2024-09-14T06:30:39.412 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.075137291999999994, 2024-09-14T06:30:39.412 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:30:39.412 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:30:39.412 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:30:39.412 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:30:39.412 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:30:39.412 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:39.412 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 115200 2024-09-14T06:30:39.412 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:39.412 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:30:39.413 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:39.413 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:30:39.413 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940080+0000", 2024-09-14T06:30:39.413 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:39.413 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:39.413 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:39.413 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:30:39.413 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940080+0000", 2024-09-14T06:30:39.413 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:39.413 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:39.413 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:39.413 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:30:39.414 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940081+0000", 2024-09-14T06:30:39.414 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.0800000000000004e-07 2024-09-14T06:30:39.414 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:39.414 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:39.414 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:30:39.414 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940083+0000", 2024-09-14T06:30:39.414 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.3920000000000001e-06 2024-09-14T06:30:39.414 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:39.414 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:39.414 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:30:39.414 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940091+0000", 2024-09-14T06:30:39.415 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.1389999999999995e-06 2024-09-14T06:30:39.415 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:39.415 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:39.415 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:30:39.415 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940097+0000", 2024-09-14T06:30:39.415 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.5550000000000002e-06 2024-09-14T06:30:39.415 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:39.415 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:39.415 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:30:39.415 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.015025+0000", 2024-09-14T06:30:39.415 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.074928579999999995 2024-09-14T06:30:39.415 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:39.416 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:39.416 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:30:39.416 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.015096+0000", 2024-09-14T06:30:39.416 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.0705999999999997e-05 2024-09-14T06:30:39.416 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:39.416 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:39.416 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:30:39.416 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.015217+0000", 2024-09-14T06:30:39.416 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00012131200000000001 2024-09-14T06:30:39.416 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:39.417 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:39.417 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:39.417 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:39.417 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:39.417 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:115201 3.1 3:856be886:::rbd_data.112d4d24c558.0000000000000004:29 [read 3489792~16384 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e96)", 2024-09-14T06:30:39.417 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:30:17.940113+0000", 2024-09-14T06:30:39.417 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 21.431720323, 2024-09-14T06:30:39.417 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.075246899000000006, 2024-09-14T06:30:39.417 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:30:39.417 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:30:39.417 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:30:39.417 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:30:39.418 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:30:39.418 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:39.418 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 115201 2024-09-14T06:30:39.418 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:39.418 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:30:39.418 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:39.418 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:30:39.418 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940113+0000", 2024-09-14T06:30:39.418 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:39.418 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:39.418 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:39.419 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:30:39.419 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940113+0000", 2024-09-14T06:30:39.419 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:39.419 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:39.419 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:39.419 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:30:39.419 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940114+0000", 2024-09-14T06:30:39.419 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.1500000000000004e-07 2024-09-14T06:30:39.419 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:39.419 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:39.419 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:30:39.419 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940116+0000", 2024-09-14T06:30:39.420 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.2809999999999998e-06 2024-09-14T06:30:39.420 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:39.420 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:39.420 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:30:39.420 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940123+0000", 2024-09-14T06:30:39.420 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.9229999999999996e-06 2024-09-14T06:30:39.420 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:39.420 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:39.420 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:30:39.420 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940128+0000", 2024-09-14T06:30:39.420 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.7790000000000003e-06 2024-09-14T06:30:39.421 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:39.421 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:39.421 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:30:39.421 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.015253+0000", 2024-09-14T06:30:39.421 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.075125792999999996 2024-09-14T06:30:39.421 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:39.421 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:39.421 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:30:39.421 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.015297+0000", 2024-09-14T06:30:39.421 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.3847000000000002e-05 2024-09-14T06:30:39.421 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:39.421 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:39.422 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:30:39.422 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.015360+0000", 2024-09-14T06:30:39.422 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.2661000000000002e-05 2024-09-14T06:30:39.422 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:39.422 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:39.422 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:39.422 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:39.422 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:39.422 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:115202 3.1 3:856be886:::rbd_data.112d4d24c558.0000000000000004:29 [read 3735552~16384 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e96)", 2024-09-14T06:30:39.422 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:30:17.940135+0000", 2024-09-14T06:30:39.422 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 21.431698544, 2024-09-14T06:30:39.422 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.075451105000000004, 2024-09-14T06:30:39.423 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:30:39.423 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:30:39.423 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:30:39.423 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:30:39.423 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:30:39.423 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:39.423 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 115202 2024-09-14T06:30:39.423 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:39.423 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:30:39.423 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:39.424 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:30:39.424 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940135+0000", 2024-09-14T06:30:39.424 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:39.424 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:39.424 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:39.424 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:30:39.424 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940135+0000", 2024-09-14T06:30:39.425 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:39.425 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:39.425 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:39.425 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:30:39.425 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940135+0000", 2024-09-14T06:30:39.425 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.0999999999999999e-07 2024-09-14T06:30:39.425 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:39.425 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:39.425 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:30:39.425 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940137+0000", 2024-09-14T06:30:39.425 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.0020000000000001e-06 2024-09-14T06:30:39.426 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:39.426 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:39.426 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:30:39.426 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940143+0000", 2024-09-14T06:30:39.426 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.9850000000000003e-06 2024-09-14T06:30:39.426 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:39.426 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:39.426 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:30:39.426 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940148+0000", 2024-09-14T06:30:39.426 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.9880000000000004e-06 2024-09-14T06:30:39.426 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:39.426 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:39.427 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:30:39.427 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.015400+0000", 2024-09-14T06:30:39.427 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.075252186999999998 2024-09-14T06:30:39.427 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:39.427 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:39.427 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:30:39.427 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.015481+0000", 2024-09-14T06:30:39.427 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.0698000000000006e-05 2024-09-14T06:30:39.427 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:39.427 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:39.427 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:30:39.428 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.015586+0000", 2024-09-14T06:30:39.428 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00010483499999999999 2024-09-14T06:30:39.428 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:39.428 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:39.428 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:39.428 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:39.428 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:39.428 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:115203 3.1 3:856be886:::rbd_data.112d4d24c558.0000000000000004:29 [read 3981312~16384 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e96)", 2024-09-14T06:30:39.428 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:30:17.940156+0000", 2024-09-14T06:30:39.428 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 21.431677318999999, 2024-09-14T06:30:39.429 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.075620860999999998, 2024-09-14T06:30:39.429 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:30:39.429 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:30:39.429 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:30:39.429 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:30:39.429 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:30:39.429 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:39.429 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 115203 2024-09-14T06:30:39.429 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:39.429 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:30:39.429 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:39.430 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:30:39.430 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940156+0000", 2024-09-14T06:30:39.430 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:39.430 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:39.430 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:39.430 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:30:39.430 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940156+0000", 2024-09-14T06:30:39.430 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:39.430 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:39.430 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:39.430 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:30:39.430 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940156+0000", 2024-09-14T06:30:39.431 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.1500000000000005e-07 2024-09-14T06:30:39.431 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:39.431 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:39.431 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:30:39.431 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940158+0000", 2024-09-14T06:30:39.431 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.967e-06 2024-09-14T06:30:39.431 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:39.431 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:39.431 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:30:39.431 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940165+0000", 2024-09-14T06:30:39.432 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.6939999999999998e-06 2024-09-14T06:30:39.432 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:39.432 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:39.432 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:30:39.432 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940170+0000", 2024-09-14T06:30:39.432 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.7539999999999997e-06 2024-09-14T06:30:39.432 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:39.432 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:39.432 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:30:39.432 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.015633+0000", 2024-09-14T06:30:39.432 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.075463334000000007 2024-09-14T06:30:39.433 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:39.433 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:39.433 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:30:39.433 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.015693+0000", 2024-09-14T06:30:39.433 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.9367000000000003e-05 2024-09-14T06:30:39.433 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:39.433 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:39.433 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:30:39.433 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.015777+0000", 2024-09-14T06:30:39.433 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.4229999999999998e-05 2024-09-14T06:30:39.433 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:39.433 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:39.434 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:39.434 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:39.434 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:39.434 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:115204 3.1 3:856be886:::rbd_data.112d4d24c558.0000000000000004:29 [read 4227072~16384 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e96)", 2024-09-14T06:30:39.434 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:30:17.940176+0000", 2024-09-14T06:30:39.434 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 21.431656786000001, 2024-09-14T06:30:39.434 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.075785505000000003, 2024-09-14T06:30:39.434 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:30:39.434 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:30:39.434 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:30:39.434 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:30:39.435 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:30:39.435 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:39.435 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 115204 2024-09-14T06:30:39.435 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:39.435 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:30:39.435 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:39.435 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:30:39.435 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940176+0000", 2024-09-14T06:30:39.435 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:39.435 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:39.435 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:39.435 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:30:39.436 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940176+0000", 2024-09-14T06:30:39.436 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:39.436 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:39.436 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:39.436 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:30:39.436 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940177+0000", 2024-09-14T06:30:39.436 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.7899999999999999e-07 2024-09-14T06:30:39.436 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:39.436 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:39.436 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:30:39.436 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940179+0000", 2024-09-14T06:30:39.436 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.103e-06 2024-09-14T06:30:39.437 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:39.437 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:39.437 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:30:39.437 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940185+0000", 2024-09-14T06:30:39.437 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.7239999999999997e-06 2024-09-14T06:30:39.437 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:39.437 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:39.437 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:30:39.437 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940190+0000", 2024-09-14T06:30:39.437 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.7269999999999998e-06 2024-09-14T06:30:39.437 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:39.438 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:39.438 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:30:39.438 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.015814+0000", 2024-09-14T06:30:39.438 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.075624114000000006 2024-09-14T06:30:39.438 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:39.438 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:39.438 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:30:39.438 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.015862+0000", 2024-09-14T06:30:39.438 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.8770999999999998e-05 2024-09-14T06:30:39.438 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:39.438 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:39.438 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:30:39.439 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.015962+0000", 2024-09-14T06:30:39.439 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.9586999999999994e-05 2024-09-14T06:30:39.439 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:39.439 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:39.439 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:39.439 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:39.439 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:39.439 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:115205 3.1 3:856be886:::rbd_data.112d4d24c558.0000000000000004:29 [read 4472832~16384 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e96)", 2024-09-14T06:30:39.439 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:30:17.940197+0000", 2024-09-14T06:30:39.439 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 21.431636077, 2024-09-14T06:30:39.439 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.076068920999999998, 2024-09-14T06:30:39.439 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:30:39.439 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:30:39.439 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:30:39.440 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:30:39.440 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:30:39.440 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:39.440 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 115205 2024-09-14T06:30:39.440 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:39.440 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:30:39.440 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:39.440 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:30:39.440 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940197+0000", 2024-09-14T06:30:39.440 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:39.440 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:39.440 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:39.440 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:30:39.441 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940197+0000", 2024-09-14T06:30:39.441 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:39.441 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:39.441 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:39.441 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:30:39.441 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940198+0000", 2024-09-14T06:30:39.441 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.5999999999999999e-07 2024-09-14T06:30:39.441 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:39.441 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:39.441 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:30:39.441 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940200+0000", 2024-09-14T06:30:39.442 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.0899999999999999e-06 2024-09-14T06:30:39.442 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:39.442 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:39.442 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:30:39.442 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940206+0000", 2024-09-14T06:30:39.442 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.8710000000000002e-06 2024-09-14T06:30:39.442 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:39.442 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:39.442 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:30:39.442 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940211+0000", 2024-09-14T06:30:39.442 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.9230000000000001e-06 2024-09-14T06:30:39.442 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:39.442 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:39.443 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:30:39.443 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.016059+0000", 2024-09-14T06:30:39.443 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.075848051999999999 2024-09-14T06:30:39.443 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:39.443 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:39.443 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:30:39.443 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.016151+0000", 2024-09-14T06:30:39.443 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.1982000000000001e-05 2024-09-14T06:30:39.443 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:39.443 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:39.443 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:30:39.443 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.016266+0000", 2024-09-14T06:30:39.444 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000115543 2024-09-14T06:30:39.444 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:39.444 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:39.444 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:39.444 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:39.444 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:39.444 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:115206 3.1 3:856be886:::rbd_data.112d4d24c558.0000000000000004:29 [read 4718592~16384 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e96)", 2024-09-14T06:30:39.444 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:30:17.940223+0000", 2024-09-14T06:30:39.444 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 21.431609987000002, 2024-09-14T06:30:39.444 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.076265115999999994, 2024-09-14T06:30:39.444 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:30:39.444 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:30:39.445 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:30:39.445 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:30:39.445 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:30:39.445 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:39.445 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 115206 2024-09-14T06:30:39.445 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:39.445 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:30:39.445 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:39.445 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:30:39.445 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940223+0000", 2024-09-14T06:30:39.445 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:39.446 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:39.446 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:39.446 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:30:39.446 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940223+0000", 2024-09-14T06:30:39.446 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:39.446 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:39.446 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:39.446 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:30:39.446 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940224+0000", 2024-09-14T06:30:39.446 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.0399999999999996e-07 2024-09-14T06:30:39.446 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:39.447 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:39.447 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:30:39.447 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940226+0000", 2024-09-14T06:30:39.447 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.1780000000000002e-06 2024-09-14T06:30:39.447 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:39.447 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:39.447 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:30:39.447 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940232+0000", 2024-09-14T06:30:39.447 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.5010000000000001e-06 2024-09-14T06:30:39.447 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:39.447 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:39.448 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:30:39.448 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940237+0000", 2024-09-14T06:30:39.448 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.0250000000000002e-06 2024-09-14T06:30:39.448 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:39.448 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:39.448 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:30:39.448 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.016305+0000", 2024-09-14T06:30:39.448 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.076067262999999996 2024-09-14T06:30:39.448 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:39.448 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:39.448 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:30:39.448 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.016383+0000", 2024-09-14T06:30:39.449 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.8578000000000006e-05 2024-09-14T06:30:39.449 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:39.449 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:39.449 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:30:39.449 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.016488+0000", 2024-09-14T06:30:39.449 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000105067 2024-09-14T06:30:39.449 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:39.449 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:39.449 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:39.449 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:39.449 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:39.450 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:115207 3.1 3:856be886:::rbd_data.112d4d24c558.0000000000000004:29 [read 4964352~16384 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e96)", 2024-09-14T06:30:39.450 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:30:17.940244+0000", 2024-09-14T06:30:39.450 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 21.431588870999999, 2024-09-14T06:30:39.450 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.076371154999999996, 2024-09-14T06:30:39.450 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:30:39.450 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:30:39.450 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:30:39.450 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:30:39.450 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:30:39.450 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:39.450 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 115207 2024-09-14T06:30:39.450 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:39.451 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:30:39.451 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:39.451 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:30:39.451 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940244+0000", 2024-09-14T06:30:39.451 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:39.451 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:39.451 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:39.451 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:30:39.451 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940244+0000", 2024-09-14T06:30:39.451 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:39.451 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:39.452 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:39.452 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:30:39.452 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940245+0000", 2024-09-14T06:30:39.452 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.1200000000000003e-07 2024-09-14T06:30:39.452 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:39.452 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:39.452 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:30:39.452 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940247+0000", 2024-09-14T06:30:39.452 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.0030000000000001e-06 2024-09-14T06:30:39.452 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:39.452 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:39.452 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:30:39.452 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940252+0000", 2024-09-14T06:30:39.453 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.5520000000000004e-06 2024-09-14T06:30:39.453 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:39.453 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:39.453 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:30:39.453 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940257+0000", 2024-09-14T06:30:39.453 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.6759999999999998e-06 2024-09-14T06:30:39.453 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:39.453 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:39.453 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:30:39.453 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.016521+0000", 2024-09-14T06:30:39.453 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.076263499999999998 2024-09-14T06:30:39.453 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:39.454 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:39.454 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:30:39.454 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.016562+0000", 2024-09-14T06:30:39.454 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.0967000000000003e-05 2024-09-14T06:30:39.454 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:39.454 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:39.454 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:30:39.456 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.016616+0000", 2024-09-14T06:30:39.456 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.3944999999999999e-05 2024-09-14T06:30:39.456 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:39.456 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:39.456 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:39.456 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:39.456 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:39.456 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:115208 3.1 3:856be886:::rbd_data.112d4d24c558.0000000000000004:29 [read 5210112~16384 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e96)", 2024-09-14T06:30:39.456 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:30:17.940264+0000", 2024-09-14T06:30:39.456 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 21.431569014000001, 2024-09-14T06:30:39.456 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.076558288000000002, 2024-09-14T06:30:39.457 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:30:39.457 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:30:39.457 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:30:39.457 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:30:39.457 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:30:39.457 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:39.457 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 115208 2024-09-14T06:30:39.457 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:39.457 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:30:39.457 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:39.457 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:30:39.457 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940264+0000", 2024-09-14T06:30:39.458 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:39.458 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:39.458 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:39.458 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:30:39.458 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940264+0000", 2024-09-14T06:30:39.458 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:39.458 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:39.458 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:39.458 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:30:39.458 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940265+0000", 2024-09-14T06:30:39.458 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.0500000000000004e-07 2024-09-14T06:30:39.459 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:39.459 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:39.459 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:30:39.459 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940267+0000", 2024-09-14T06:30:39.459 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.9810000000000002e-06 2024-09-14T06:30:39.459 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:39.459 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:39.459 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:30:39.459 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940272+0000", 2024-09-14T06:30:39.459 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.6999999999999996e-06 2024-09-14T06:30:39.459 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:39.459 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:39.460 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:30:39.460 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940277+0000", 2024-09-14T06:30:39.460 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.6480000000000002e-06 2024-09-14T06:30:39.460 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:39.460 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:39.460 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:30:39.460 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.016650+0000", 2024-09-14T06:30:39.460 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.076373089000000005 2024-09-14T06:30:39.460 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:39.460 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:39.460 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:30:39.460 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.016724+0000", 2024-09-14T06:30:39.461 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.4105999999999998e-05 2024-09-14T06:30:39.461 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:39.461 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:39.461 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:30:39.461 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.016823+0000", 2024-09-14T06:30:39.461 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.8258999999999996e-05 2024-09-14T06:30:39.461 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:39.461 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:39.461 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:39.461 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:39.461 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:39.462 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:115209 3.1 3:856be886:::rbd_data.112d4d24c558.0000000000000004:29 [read 5455872~16384 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e96)", 2024-09-14T06:30:39.462 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:30:17.940283+0000", 2024-09-14T06:30:39.462 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 21.431549988, 2024-09-14T06:30:39.462 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.076724029999999999, 2024-09-14T06:30:39.462 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:30:39.462 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:30:39.462 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:30:39.462 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:30:39.462 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:30:39.462 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:39.462 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 115209 2024-09-14T06:30:39.463 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:39.463 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:30:39.463 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:39.463 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:30:39.463 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940283+0000", 2024-09-14T06:30:39.463 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:39.463 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:39.463 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:39.463 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:30:39.463 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940283+0000", 2024-09-14T06:30:39.463 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:39.463 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:39.463 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:39.464 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:30:39.464 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940284+0000", 2024-09-14T06:30:39.464 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.6700000000000003e-07 2024-09-14T06:30:39.464 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:39.464 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:39.464 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:30:39.464 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940286+0000", 2024-09-14T06:30:39.464 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.9520000000000001e-06 2024-09-14T06:30:39.464 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:39.464 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:39.464 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:30:39.465 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940292+0000", 2024-09-14T06:30:39.465 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.7339999999999996e-06 2024-09-14T06:30:39.465 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:39.465 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:39.465 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:30:39.465 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940296+0000", 2024-09-14T06:30:39.465 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.7700000000000001e-06 2024-09-14T06:30:39.465 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:39.465 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:39.465 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:30:39.465 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.016859+0000", 2024-09-14T06:30:39.465 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.07656317 2024-09-14T06:30:39.466 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:39.466 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:39.466 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:30:39.466 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.016929+0000", 2024-09-14T06:30:39.466 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.9679999999999997e-05 2024-09-14T06:30:39.466 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:39.466 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:39.466 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:30:39.466 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.017007+0000", 2024-09-14T06:30:39.466 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.8157000000000004e-05 2024-09-14T06:30:39.466 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:39.466 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:39.467 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:39.467 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:39.467 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:39.467 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:115210 3.1 3:856be886:::rbd_data.112d4d24c558.0000000000000004:29 [read 5701632~16384 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e96)", 2024-09-14T06:30:39.467 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:30:17.940303+0000", 2024-09-14T06:30:39.467 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 21.431530470999999, 2024-09-14T06:30:39.467 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.076821358000000006, 2024-09-14T06:30:39.467 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:30:39.467 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:30:39.467 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:30:39.467 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:30:39.468 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:30:39.468 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:39.468 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 115210 2024-09-14T06:30:39.468 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:39.468 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:30:39.468 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:39.468 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:30:39.468 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940303+0000", 2024-09-14T06:30:39.468 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:39.468 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:39.468 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:39.468 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:30:39.469 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940303+0000", 2024-09-14T06:30:39.469 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:39.469 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:39.469 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:39.469 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:30:39.469 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940303+0000", 2024-09-14T06:30:39.469 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.3000000000000001e-07 2024-09-14T06:30:39.469 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:39.469 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:39.469 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:30:39.469 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940306+0000", 2024-09-14T06:30:39.469 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.255e-06 2024-09-14T06:30:39.470 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:39.470 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:39.470 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:30:39.470 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940312+0000", 2024-09-14T06:30:39.470 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.9710000000000001e-06 2024-09-14T06:30:39.470 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:39.470 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:39.470 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:30:39.470 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940316+0000", 2024-09-14T06:30:39.470 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.5340000000000001e-06 2024-09-14T06:30:39.470 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:39.471 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:39.471 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:30:39.471 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.017033+0000", 2024-09-14T06:30:39.471 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.076717169000000002 2024-09-14T06:30:39.471 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:39.471 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:39.471 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:30:39.471 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.017073+0000", 2024-09-14T06:30:39.471 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.9582000000000002e-05 2024-09-14T06:30:39.471 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:39.471 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:39.471 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:30:39.472 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.017124+0000", 2024-09-14T06:30:39.472 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.1316999999999997e-05 2024-09-14T06:30:39.472 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:39.472 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:39.472 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:39.472 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:39.472 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:39.472 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:115219 3.1 3:856be886:::rbd_data.112d4d24c558.0000000000000004:29 [read 65536~16384 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e96)", 2024-09-14T06:30:39.472 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:30:17.940323+0000", 2024-09-14T06:30:39.472 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 21.43151061, 2024-09-14T06:30:39.472 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.076915205, 2024-09-14T06:30:39.473 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:30:39.473 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:30:39.473 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:30:39.473 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:30:39.473 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:30:39.473 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:39.473 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 115219 2024-09-14T06:30:39.473 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:39.473 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:30:39.473 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:39.473 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:30:39.473 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940323+0000", 2024-09-14T06:30:39.474 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:39.474 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:39.474 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:39.474 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:30:39.474 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940323+0000", 2024-09-14T06:30:39.474 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:39.474 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:39.474 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:39.474 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:30:39.474 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940323+0000", 2024-09-14T06:30:39.474 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.0800000000000005e-07 2024-09-14T06:30:39.474 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:39.475 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:39.475 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:30:39.475 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940328+0000", 2024-09-14T06:30:39.475 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.887e-06 2024-09-14T06:30:39.475 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:39.475 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:39.475 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:30:39.475 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940334+0000", 2024-09-14T06:30:39.475 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.3269999999999997e-06 2024-09-14T06:30:39.475 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:39.475 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:39.475 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:30:39.476 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940339+0000", 2024-09-14T06:30:39.476 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.7960000000000003e-06 2024-09-14T06:30:39.476 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:39.476 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:39.476 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:30:39.476 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.017148+0000", 2024-09-14T06:30:39.476 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.076809125000000006 2024-09-14T06:30:39.476 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:39.476 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:39.476 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:30:39.476 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.017186+0000", 2024-09-14T06:30:39.476 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.7283000000000002e-05 2024-09-14T06:30:39.477 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:39.477 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:39.477 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:30:39.477 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.017238+0000", 2024-09-14T06:30:39.477 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.2278999999999999e-05 2024-09-14T06:30:39.477 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:39.477 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:39.477 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:39.477 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:39.477 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:39.477 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:115220 3.1 3:856be886:::rbd_data.112d4d24c558.0000000000000004:29 [read 311296~16384 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e96)", 2024-09-14T06:30:39.478 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:30:17.940346+0000", 2024-09-14T06:30:39.478 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 21.431487481000001, 2024-09-14T06:30:39.478 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.077033293000000003, 2024-09-14T06:30:39.478 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:30:39.478 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:30:39.478 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:30:39.478 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:30:39.478 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:30:39.478 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:39.478 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 115220 2024-09-14T06:30:39.478 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:39.478 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:30:39.479 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:39.479 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:30:39.479 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940346+0000", 2024-09-14T06:30:39.479 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:39.479 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:39.479 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:39.479 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:30:39.479 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940346+0000", 2024-09-14T06:30:39.479 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:39.479 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:39.479 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:39.479 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:30:39.480 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940346+0000", 2024-09-14T06:30:39.480 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.4300000000000003e-07 2024-09-14T06:30:39.480 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:39.480 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:39.480 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:30:39.480 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940348+0000", 2024-09-14T06:30:39.480 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.0030000000000001e-06 2024-09-14T06:30:39.480 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:39.480 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:39.480 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:30:39.480 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.004655+0000", 2024-09-14T06:30:39.481 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.064306323999999998 2024-09-14T06:30:39.481 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:39.481 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:39.481 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:30:39.481 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.004671+0000", 2024-09-14T06:30:39.481 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.6589999999999999e-05 2024-09-14T06:30:39.481 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:39.481 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:39.481 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:30:39.481 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.017262+0000", 2024-09-14T06:30:39.481 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.012590570000000001 2024-09-14T06:30:39.481 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:39.482 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:39.482 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:30:39.482 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.017321+0000", 2024-09-14T06:30:39.482 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.9210999999999999e-05 2024-09-14T06:30:39.482 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:39.482 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:39.482 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:30:39.482 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.017379+0000", 2024-09-14T06:30:39.482 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.8051999999999999e-05 2024-09-14T06:30:39.482 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:39.482 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:39.482 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:39.483 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:39.483 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:39.483 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-14T06:30:39.484 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-14T06:30:39.485 INFO:tasks.ceph.osd.3.smithi094.stderr:2024-09-14T06:30:39.483+0000 7feeeafe8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33655) UID: 0 2024-09-14T06:30:39.586 INFO:tasks.ceph.osd.6.smithi138.stderr:2024-09-14T06:30:39.584+0000 7f8cd8849640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33620) UID: 0 2024-09-14T06:30:39.627 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-14T06:30:39.627 INFO:teuthology.orchestra.run.smithi094.stdout: "ops": [], 2024-09-14T06:30:39.627 INFO:teuthology.orchestra.run.smithi094.stdout: "num_ops": 0 2024-09-14T06:30:39.627 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-14T06:30:39.636 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-14T06:30:39.677 INFO:teuthology.orchestra.run.smithi094.stderr:nodeep-scrub is set 2024-09-14T06:30:39.686 INFO:tasks.ceph.osd.2.smithi094.stderr:2024-09-14T06:30:39.684+0000 7faf3269b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33649) UID: 0 2024-09-14T06:30:39.750 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-14T06:30:39.750 INFO:teuthology.orchestra.run.smithi094.stdout: "ops": [], 2024-09-14T06:30:39.750 INFO:teuthology.orchestra.run.smithi094.stdout: "complaint_time": 30, 2024-09-14T06:30:39.751 INFO:teuthology.orchestra.run.smithi094.stdout: "num_blocked_ops": 0 2024-09-14T06:30:39.751 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-14T06:30:39.759 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-14T06:30:39.803 INFO:tasks.ceph.osd.0.smithi094.stderr:2024-09-14T06:30:39.801+0000 7f07c9333640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33636) UID: 0 2024-09-14T06:30:39.869 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-14T06:30:39.869 INFO:teuthology.orchestra.run.smithi094.stdout: "size": 20, 2024-09-14T06:30:39.870 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 600, 2024-09-14T06:30:39.870 INFO:teuthology.orchestra.run.smithi094.stdout: "ops": [ 2024-09-14T06:30:39.870 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:39.870 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:14918 3.0 3:05c39535:::rbd_data.112d92027590.0000000000000009:f [read 795648~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:30:39.870 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:26.083834+0000", 2024-09-14T06:30:39.870 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 73.783654988999999, 2024-09-14T06:30:39.870 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.041776287000000002, 2024-09-14T06:30:39.870 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:30:39.870 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:30:39.870 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:30:39.870 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:30:39.870 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:30:39.870 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:39.870 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 14918 2024-09-14T06:30:39.871 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:39.871 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:30:39.871 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:39.871 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:30:39.871 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.083834+0000", 2024-09-14T06:30:39.871 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:39.871 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:39.871 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:39.871 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:30:39.871 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.083834+0000", 2024-09-14T06:30:39.871 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:39.871 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:39.871 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:39.871 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:30:39.872 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.083834+0000", 2024-09-14T06:30:39.872 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.0100000000000001e-07 2024-09-14T06:30:39.872 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:39.872 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:39.872 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:30:39.872 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.083835+0000", 2024-09-14T06:30:39.872 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.3900000000000003e-07 2024-09-14T06:30:39.872 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:39.872 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:39.872 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:30:39.872 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.083838+0000", 2024-09-14T06:30:39.872 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.9830000000000001e-06 2024-09-14T06:30:39.872 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:39.873 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:39.873 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:30:39.873 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.083840+0000", 2024-09-14T06:30:39.873 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.4679999999999999e-06 2024-09-14T06:30:39.873 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:39.873 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:39.873 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:30:39.873 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.125455+0000", 2024-09-14T06:30:39.873 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.041614578999999999 2024-09-14T06:30:39.873 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:39.873 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:39.873 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:30:39.873 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.125496+0000", 2024-09-14T06:30:39.873 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.1047000000000002e-05 2024-09-14T06:30:39.874 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:39.874 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:39.874 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:30:39.874 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.125610+0000", 2024-09-14T06:30:39.874 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00011407 2024-09-14T06:30:39.874 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:39.874 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:39.874 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:39.874 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:39.874 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:39.874 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:14919 3.0 3:05c39535:::rbd_data.112d92027590.0000000000000009:f [read 805888~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:30:39.874 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:26.083843+0000", 2024-09-14T06:30:39.874 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 73.783645241000002, 2024-09-14T06:30:39.874 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.041992356000000002, 2024-09-14T06:30:39.875 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:30:39.875 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:30:39.875 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:30:39.875 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:30:39.875 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:30:39.875 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:39.875 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 14919 2024-09-14T06:30:39.875 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:39.875 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:30:39.875 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:39.875 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:30:39.875 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.083843+0000", 2024-09-14T06:30:39.875 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:39.875 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:39.876 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:39.876 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:30:39.876 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.083843+0000", 2024-09-14T06:30:39.876 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:39.876 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:39.876 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:39.876 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:30:39.876 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.083844+0000", 2024-09-14T06:30:39.876 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.3300000000000001e-07 2024-09-14T06:30:39.876 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:39.876 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:39.876 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:30:39.876 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.083845+0000", 2024-09-14T06:30:39.877 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.4729999999999999e-06 2024-09-14T06:30:39.877 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:39.877 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:39.877 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:30:39.877 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.083850+0000", 2024-09-14T06:30:39.877 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.0130000000000001e-06 2024-09-14T06:30:39.877 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:39.877 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:39.877 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:30:39.877 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.083854+0000", 2024-09-14T06:30:39.877 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.4759999999999998e-06 2024-09-14T06:30:39.877 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:39.877 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:39.877 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:30:39.878 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.125636+0000", 2024-09-14T06:30:39.878 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.041782683000000001 2024-09-14T06:30:39.878 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:39.878 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:39.878 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:30:39.878 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.125680+0000", 2024-09-14T06:30:39.878 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.3614e-05 2024-09-14T06:30:39.878 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:39.878 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:39.878 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:30:39.878 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.125836+0000", 2024-09-14T06:30:39.878 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000155864 2024-09-14T06:30:39.878 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:39.878 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:39.879 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:39.879 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:39.879 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:39.879 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:14920 3.0 3:05c39535:::rbd_data.112d92027590.0000000000000009:f [read 816128~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:30:39.879 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:26.083868+0000", 2024-09-14T06:30:39.879 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 73.783620580000004, 2024-09-14T06:30:39.879 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.042166771999999998, 2024-09-14T06:30:39.879 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:30:39.879 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:30:39.879 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:30:39.879 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:30:39.879 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:30:39.879 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:39.879 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 14920 2024-09-14T06:30:39.879 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:39.880 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:30:39.880 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:39.880 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:30:39.880 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.083868+0000", 2024-09-14T06:30:39.880 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:39.880 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:39.880 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:39.880 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:30:39.880 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.083868+0000", 2024-09-14T06:30:39.880 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:39.880 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:39.880 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:39.880 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:30:39.880 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.083869+0000", 2024-09-14T06:30:39.881 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.99e-07 2024-09-14T06:30:39.881 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:39.881 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:39.881 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:30:39.881 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.083870+0000", 2024-09-14T06:30:39.881 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.782e-06 2024-09-14T06:30:39.881 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:39.881 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:39.881 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:30:39.881 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.083876+0000", 2024-09-14T06:30:39.881 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.3809999999999998e-06 2024-09-14T06:30:39.881 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:39.881 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:39.882 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:30:39.882 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.083882+0000", 2024-09-14T06:30:39.882 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.9479999999999996e-06 2024-09-14T06:30:39.882 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:39.882 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:39.882 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:30:39.882 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.125878+0000", 2024-09-14T06:30:39.882 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.041996264999999998 2024-09-14T06:30:39.882 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:39.882 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:39.883 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:30:39.883 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.125929+0000", 2024-09-14T06:30:39.883 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.0498999999999999e-05 2024-09-14T06:30:39.883 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:39.883 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:39.883 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:30:39.883 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.126035+0000", 2024-09-14T06:30:39.883 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000106298 2024-09-14T06:30:39.883 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:39.883 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:39.883 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:39.883 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:39.883 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:39.884 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:14921 3.0 3:05c39535:::rbd_data.112d92027590.0000000000000009:f [read 826368~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:30:39.884 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:26.083889+0000", 2024-09-14T06:30:39.884 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 73.783599570000007, 2024-09-14T06:30:39.884 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.042435309999999997, 2024-09-14T06:30:39.884 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:30:39.884 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:30:39.884 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:30:39.884 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:30:39.884 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:30:39.884 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:39.884 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 14921 2024-09-14T06:30:39.884 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:39.884 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:30:39.885 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:39.885 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:30:39.885 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.083889+0000", 2024-09-14T06:30:39.885 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:39.885 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:39.885 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:39.885 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:30:39.885 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.083889+0000", 2024-09-14T06:30:39.885 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:39.885 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:39.885 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:39.885 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:30:39.885 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.086033+0000", 2024-09-14T06:30:39.885 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0021444179999999999 2024-09-14T06:30:39.885 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:39.886 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:39.886 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:30:39.886 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.086038+0000", 2024-09-14T06:30:39.886 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.0969999999999999e-06 2024-09-14T06:30:39.886 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:39.886 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:39.886 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:30:39.886 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.086046+0000", 2024-09-14T06:30:39.886 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.9440000000000005e-06 2024-09-14T06:30:39.886 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:39.886 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:39.886 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:30:39.886 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.086050+0000", 2024-09-14T06:30:39.887 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.69e-06 2024-09-14T06:30:39.887 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:39.887 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:39.887 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:30:39.887 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.126080+0000", 2024-09-14T06:30:39.887 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.040030237000000003 2024-09-14T06:30:39.887 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:39.887 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:39.887 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:30:39.887 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.126163+0000", 2024-09-14T06:30:39.887 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.2071999999999996e-05 2024-09-14T06:30:39.887 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:39.887 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:39.888 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:30:39.888 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.126324+0000", 2024-09-14T06:30:39.888 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00016185200000000001 2024-09-14T06:30:39.888 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:39.888 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:39.888 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:39.888 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:39.888 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:39.888 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:55627 3.0 3:06490baf:::rbd_data.112dca2f6ae1.0000000000000030:17 [read 2002944~4096 [fadvise_nocache]] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:30:39.888 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:44.599966+0000", 2024-09-14T06:30:39.888 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 55.267522622000001, 2024-09-14T06:30:39.888 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.041868546999999999, 2024-09-14T06:30:39.888 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:30:39.889 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:30:39.889 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:30:39.889 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:30:39.889 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:30:39.889 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:39.889 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 55627 2024-09-14T06:30:39.889 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:39.889 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:30:39.889 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:39.889 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:30:39.889 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.599966+0000", 2024-09-14T06:30:39.889 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:39.889 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:39.889 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:39.890 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:30:39.890 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.599966+0000", 2024-09-14T06:30:39.890 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:39.890 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:39.890 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:39.890 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:30:39.890 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.599966+0000", 2024-09-14T06:30:39.890 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.0900000000000001e-07 2024-09-14T06:30:39.890 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:39.890 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:39.890 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:30:39.890 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.599967+0000", 2024-09-14T06:30:39.890 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.64e-07 2024-09-14T06:30:39.890 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:39.890 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:39.890 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:30:39.891 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.599970+0000", 2024-09-14T06:30:39.891 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.9670000000000001e-06 2024-09-14T06:30:39.891 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:39.891 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:39.891 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:30:39.891 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.599973+0000", 2024-09-14T06:30:39.891 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.632e-06 2024-09-14T06:30:39.891 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:39.891 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:39.891 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:30:39.891 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.641665+0000", 2024-09-14T06:30:39.891 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.041692547000000003 2024-09-14T06:30:39.891 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:39.891 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:39.892 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:30:39.892 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.641740+0000", 2024-09-14T06:30:39.892 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.4559000000000002e-05 2024-09-14T06:30:39.892 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:39.892 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:39.892 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:30:39.892 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.641835+0000", 2024-09-14T06:30:39.892 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.4668999999999998e-05 2024-09-14T06:30:39.892 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:39.892 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:39.892 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:39.892 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:39.892 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:39.893 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:55628 3.0 3:06490baf:::rbd_data.112dca2f6ae1.0000000000000030:17 [read 2027520~4096 [fadvise_nocache]] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:30:39.893 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:44.599976+0000", 2024-09-14T06:30:39.893 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 55.267512834999998, 2024-09-14T06:30:39.893 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.041991621, 2024-09-14T06:30:39.893 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:30:39.893 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:30:39.893 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:30:39.893 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:30:39.893 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:30:39.893 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:39.893 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 55628 2024-09-14T06:30:39.893 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:39.893 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:30:39.894 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:39.894 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:30:39.894 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.599976+0000", 2024-09-14T06:30:39.894 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:39.894 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:39.894 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:39.894 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:30:39.894 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.599976+0000", 2024-09-14T06:30:39.894 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:39.894 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:39.894 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:39.894 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:30:39.894 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.599976+0000", 2024-09-14T06:30:39.894 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.1299999999999999e-07 2024-09-14T06:30:39.895 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:39.895 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:39.895 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:30:39.895 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.599977+0000", 2024-09-14T06:30:39.895 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.7999999999999993e-07 2024-09-14T06:30:39.895 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:39.895 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:39.895 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:30:39.895 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.599980+0000", 2024-09-14T06:30:39.895 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.01e-06 2024-09-14T06:30:39.895 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:39.895 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:39.895 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:30:39.895 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.599983+0000", 2024-09-14T06:30:39.895 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.5409999999999999e-06 2024-09-14T06:30:39.896 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:39.896 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:39.896 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:30:39.896 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.641869+0000", 2024-09-14T06:30:39.896 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.041886235000000001 2024-09-14T06:30:39.896 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:39.896 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:39.896 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:30:39.896 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.641913+0000", 2024-09-14T06:30:39.896 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.4357000000000002e-05 2024-09-14T06:30:39.896 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:39.896 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:39.896 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:30:39.897 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.641967+0000", 2024-09-14T06:30:39.897 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.4284999999999999e-05 2024-09-14T06:30:39.897 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:39.897 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:39.897 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:39.897 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:39.897 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:39.897 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:55629 3.0 3:06490baf:::rbd_data.112dca2f6ae1.0000000000000030:17 [read 2052096~4096 [fadvise_nocache]] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:30:39.897 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:44.599985+0000", 2024-09-14T06:30:39.897 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 55.267503128999998, 2024-09-14T06:30:39.897 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.042183375000000002, 2024-09-14T06:30:39.897 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:30:39.897 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:30:39.897 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:30:39.898 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:30:39.898 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:30:39.898 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:39.898 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 55629 2024-09-14T06:30:39.898 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:39.898 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:30:39.898 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:39.898 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:30:39.898 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.599985+0000", 2024-09-14T06:30:39.898 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:39.898 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:39.898 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:39.898 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:30:39.898 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.599985+0000", 2024-09-14T06:30:39.899 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:39.899 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:39.899 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:39.899 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:30:39.899 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.599986+0000", 2024-09-14T06:30:39.899 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.05e-07 2024-09-14T06:30:39.899 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:39.899 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:39.899 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:30:39.899 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.599987+0000", 2024-09-14T06:30:39.899 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.8400000000000002e-07 2024-09-14T06:30:39.899 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:39.899 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:39.900 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:30:39.900 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.599990+0000", 2024-09-14T06:30:39.900 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.5640000000000001e-06 2024-09-14T06:30:39.900 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:39.900 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:39.900 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:30:39.900 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.599993+0000", 2024-09-14T06:30:39.900 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.615e-06 2024-09-14T06:30:39.900 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:39.900 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:39.900 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:30:39.900 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.642003+0000", 2024-09-14T06:30:39.900 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.042010345999999997 2024-09-14T06:30:39.901 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:39.901 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:39.901 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:30:39.901 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.642080+0000", 2024-09-14T06:30:39.901 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.6538999999999993e-05 2024-09-14T06:30:39.901 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:39.901 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:39.901 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:30:39.901 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.642169+0000", 2024-09-14T06:30:39.901 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.9122000000000005e-05 2024-09-14T06:30:39.901 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:39.901 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:39.901 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:39.901 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:39.902 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:39.902 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:55630 3.0 3:06490baf:::rbd_data.112dca2f6ae1.0000000000000030:17 [read 2076672~4096 [fadvise_nocache]] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:30:39.902 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:44.599996+0000", 2024-09-14T06:30:39.902 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 55.267492650999998, 2024-09-14T06:30:39.902 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.042299615999999998, 2024-09-14T06:30:39.902 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:30:39.902 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:30:39.902 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:30:39.902 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:30:39.902 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:30:39.902 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:39.902 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 55630 2024-09-14T06:30:39.902 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:39.902 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:30:39.903 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:39.903 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:30:39.903 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.599996+0000", 2024-09-14T06:30:39.903 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:39.903 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:39.903 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:39.903 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:30:39.903 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.599996+0000", 2024-09-14T06:30:39.903 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:39.903 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:39.903 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:39.903 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:30:39.903 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.599996+0000", 2024-09-14T06:30:39.903 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.1500000000000001e-07 2024-09-14T06:30:39.904 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:39.904 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:39.904 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:30:39.904 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.599997+0000", 2024-09-14T06:30:39.904 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.33e-07 2024-09-14T06:30:39.904 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:39.904 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:39.904 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:30:39.904 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600000+0000", 2024-09-14T06:30:39.904 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.0240000000000002e-06 2024-09-14T06:30:39.904 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:39.905 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:39.905 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:30:39.905 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600003+0000", 2024-09-14T06:30:39.905 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.3640000000000002e-06 2024-09-14T06:30:39.905 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:39.905 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:39.905 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:30:39.905 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.642200+0000", 2024-09-14T06:30:39.905 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.042197571000000003 2024-09-14T06:30:39.905 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:39.905 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:39.905 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:30:39.906 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.642243+0000", 2024-09-14T06:30:39.906 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.2765000000000001e-05 2024-09-14T06:30:39.906 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:39.906 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:39.906 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:30:39.906 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.642296+0000", 2024-09-14T06:30:39.906 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.2744000000000002e-05 2024-09-14T06:30:39.906 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:39.906 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:39.906 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:39.906 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:39.906 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:39.906 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:55631 3.0 3:06490baf:::rbd_data.112dca2f6ae1.0000000000000030:17 [read 1048576~4096 [fadvise_nocache]] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:30:39.907 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:44.600006+0000", 2024-09-14T06:30:39.907 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 55.267483063, 2024-09-14T06:30:39.907 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.042487905999999999, 2024-09-14T06:30:39.907 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:30:39.907 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:30:39.907 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:30:39.907 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:30:39.907 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:30:39.907 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:39.907 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 55631 2024-09-14T06:30:39.907 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:39.907 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:30:39.907 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:39.907 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:30:39.908 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600006+0000", 2024-09-14T06:30:39.908 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:39.908 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:39.908 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:39.908 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:30:39.908 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600006+0000", 2024-09-14T06:30:39.908 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:39.908 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:39.908 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:39.908 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:30:39.908 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600006+0000", 2024-09-14T06:30:39.908 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.0800000000000001e-07 2024-09-14T06:30:39.908 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:39.909 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:39.909 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:30:39.909 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600007+0000", 2024-09-14T06:30:39.909 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.7199999999999997e-07 2024-09-14T06:30:39.909 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:39.909 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:39.909 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:30:39.909 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600010+0000", 2024-09-14T06:30:39.909 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.0280000000000001e-06 2024-09-14T06:30:39.909 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:39.909 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:39.909 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:30:39.909 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600012+0000", 2024-09-14T06:30:39.909 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.475e-06 2024-09-14T06:30:39.910 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:39.910 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:39.910 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:30:39.910 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.642329+0000", 2024-09-14T06:30:39.910 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.042317157000000001 2024-09-14T06:30:39.910 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:39.910 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:39.910 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:30:39.910 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.642403+0000", 2024-09-14T06:30:39.910 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.3941000000000002e-05 2024-09-14T06:30:39.910 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:39.910 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:39.910 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:30:39.910 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.642493+0000", 2024-09-14T06:30:39.911 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.0124999999999995e-05 2024-09-14T06:30:39.911 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:39.911 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:39.911 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:39.911 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:39.911 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:39.911 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:55632 3.0 3:06490baf:::rbd_data.112dca2f6ae1.0000000000000030:17 [read 1073152~4096 [fadvise_nocache]] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:30:39.911 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:44.600015+0000", 2024-09-14T06:30:39.911 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 55.26747331, 2024-09-14T06:30:39.911 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.042604922000000003, 2024-09-14T06:30:39.911 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:30:39.911 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:30:39.911 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:30:39.911 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:30:39.912 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:30:39.912 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:39.912 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 55632 2024-09-14T06:30:39.912 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:39.912 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:30:39.912 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:39.912 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:30:39.912 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600015+0000", 2024-09-14T06:30:39.912 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:39.912 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:39.912 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:39.912 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:30:39.912 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600015+0000", 2024-09-14T06:30:39.912 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:39.913 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:39.913 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:39.913 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:30:39.913 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600016+0000", 2024-09-14T06:30:39.913 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.1e-07 2024-09-14T06:30:39.913 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:39.913 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:39.913 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:30:39.913 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600017+0000", 2024-09-14T06:30:39.913 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.9900000000000009e-07 2024-09-14T06:30:39.913 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:39.913 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:39.913 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:30:39.914 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600020+0000", 2024-09-14T06:30:39.914 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.4170000000000001e-06 2024-09-14T06:30:39.914 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:39.914 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:39.914 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:30:39.914 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600023+0000", 2024-09-14T06:30:39.914 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.588e-06 2024-09-14T06:30:39.914 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:39.914 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:39.914 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:30:39.914 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.642526+0000", 2024-09-14T06:30:39.914 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.042503236999999999 2024-09-14T06:30:39.914 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:39.914 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:39.914 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:30:39.915 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.642568+0000", 2024-09-14T06:30:39.915 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.1925000000000003e-05 2024-09-14T06:30:39.915 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:39.915 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:39.915 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:30:39.915 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.642620+0000", 2024-09-14T06:30:39.915 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.2546000000000001e-05 2024-09-14T06:30:39.915 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:39.915 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:39.915 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:39.915 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:39.915 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:39.915 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:55633 3.0 3:06490baf:::rbd_data.112dca2f6ae1.0000000000000030:17 [read 1097728~4096 [fadvise_nocache]] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:30:39.915 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:44.600026+0000", 2024-09-14T06:30:39.916 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 55.267463040999999, 2024-09-14T06:30:39.916 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.042797474000000002, 2024-09-14T06:30:39.916 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:30:39.916 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:30:39.916 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:30:39.916 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:30:39.916 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:30:39.916 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:39.916 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 55633 2024-09-14T06:30:39.916 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:39.916 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:30:39.916 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:39.916 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:30:39.917 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600026+0000", 2024-09-14T06:30:39.917 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:39.917 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:39.917 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:39.917 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:30:39.917 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600026+0000", 2024-09-14T06:30:39.917 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:39.917 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:39.917 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:39.917 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:30:39.917 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600026+0000", 2024-09-14T06:30:39.917 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.3099999999999999e-07 2024-09-14T06:30:39.917 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:39.917 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:39.918 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:30:39.918 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600029+0000", 2024-09-14T06:30:39.918 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.304e-06 2024-09-14T06:30:39.918 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:39.918 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:39.918 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:30:39.918 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600034+0000", 2024-09-14T06:30:39.918 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.1429999999999997e-06 2024-09-14T06:30:39.918 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:39.918 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:39.918 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:30:39.918 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600038+0000", 2024-09-14T06:30:39.918 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.692e-06 2024-09-14T06:30:39.918 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:39.919 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:39.919 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:30:39.919 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.642654+0000", 2024-09-14T06:30:39.919 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.042615564000000002 2024-09-14T06:30:39.919 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:39.919 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:39.919 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:30:39.919 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.642728+0000", 2024-09-14T06:30:39.919 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.4018999999999997e-05 2024-09-14T06:30:39.919 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:39.919 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:39.919 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:30:39.919 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.642823+0000", 2024-09-14T06:30:39.919 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.5520999999999996e-05 2024-09-14T06:30:39.919 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:39.920 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:39.920 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:39.920 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:39.920 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:39.920 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:55634 3.0 3:06490baf:::rbd_data.112dca2f6ae1.0000000000000030:17 [read 1122304~4096 [fadvise_nocache]] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:30:39.920 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:44.600042+0000", 2024-09-14T06:30:39.920 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 55.267446563999997, 2024-09-14T06:30:39.920 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.042903729000000002, 2024-09-14T06:30:39.920 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:30:39.920 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:30:39.920 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:30:39.920 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:30:39.920 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:30:39.920 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:39.921 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 55634 2024-09-14T06:30:39.921 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:39.921 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:30:39.921 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:39.921 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:30:39.921 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600042+0000", 2024-09-14T06:30:39.921 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:39.921 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:39.921 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:39.921 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:30:39.921 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600042+0000", 2024-09-14T06:30:39.921 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:39.921 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:39.921 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:39.922 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:30:39.922 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600043+0000", 2024-09-14T06:30:39.922 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.7000000000000005e-07 2024-09-14T06:30:39.922 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:39.922 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:39.922 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:30:39.922 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600044+0000", 2024-09-14T06:30:39.922 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.742e-06 2024-09-14T06:30:39.922 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:39.922 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:39.922 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:30:39.922 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600050+0000", 2024-09-14T06:30:39.922 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.0800000000000002e-06 2024-09-14T06:30:39.922 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:39.923 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:39.923 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:30:39.923 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600055+0000", 2024-09-14T06:30:39.923 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.3889999999999998e-06 2024-09-14T06:30:39.923 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:39.923 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:39.923 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:30:39.923 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.642845+0000", 2024-09-14T06:30:39.923 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.042790040000000001 2024-09-14T06:30:39.923 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:39.923 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:39.923 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:30:39.923 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.642888+0000", 2024-09-14T06:30:39.924 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.2790000000000002e-05 2024-09-14T06:30:39.924 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:39.924 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:39.924 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:30:39.924 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.642946+0000", 2024-09-14T06:30:39.924 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.8118000000000002e-05 2024-09-14T06:30:39.924 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:39.924 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:39.924 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:39.924 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:39.924 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:39.924 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:55635 3.0 3:06490baf:::rbd_data.112dca2f6ae1.0000000000000030:17 [read 1146880~4096 [fadvise_nocache]] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:30:39.924 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:44.600061+0000", 2024-09-14T06:30:39.924 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 55.267427992999998, 2024-09-14T06:30:39.925 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.043092695, 2024-09-14T06:30:39.925 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:30:39.925 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:30:39.925 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:30:39.925 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:30:39.925 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:30:39.925 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:39.925 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 55635 2024-09-14T06:30:39.925 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:39.925 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:30:39.925 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:39.925 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:30:39.925 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600061+0000", 2024-09-14T06:30:39.925 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:39.926 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:39.926 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:39.926 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:30:39.926 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600061+0000", 2024-09-14T06:30:39.926 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:39.926 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:39.926 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:39.926 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:30:39.926 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600061+0000", 2024-09-14T06:30:39.926 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.3099999999999999e-07 2024-09-14T06:30:39.926 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:39.926 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:39.926 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:30:39.927 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600063+0000", 2024-09-14T06:30:39.927 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.6220000000000001e-06 2024-09-14T06:30:39.927 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:39.927 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:39.927 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:30:39.927 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600068+0000", 2024-09-14T06:30:39.927 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.4790000000000004e-06 2024-09-14T06:30:39.927 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:39.927 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:39.927 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:30:39.927 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600072+0000", 2024-09-14T06:30:39.927 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.3100000000000002e-06 2024-09-14T06:30:39.927 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:39.927 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:39.927 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:30:39.928 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.642987+0000", 2024-09-14T06:30:39.928 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.042914489 2024-09-14T06:30:39.928 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:39.928 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:39.928 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:30:39.928 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.643062+0000", 2024-09-14T06:30:39.928 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.5011000000000004e-05 2024-09-14T06:30:39.928 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:39.928 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:39.928 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:30:39.928 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.643153+0000", 2024-09-14T06:30:39.928 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.1452999999999993e-05 2024-09-14T06:30:39.928 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:39.928 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:39.929 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:39.929 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:39.929 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:39.929 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:55636 3.0 3:06490baf:::rbd_data.112dca2f6ae1.0000000000000030:17 [read 1171456~4096 [fadvise_nocache]] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:30:39.929 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:44.600078+0000", 2024-09-14T06:30:39.929 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 55.267410581, 2024-09-14T06:30:39.929 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.043207966, 2024-09-14T06:30:39.929 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:30:39.929 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:30:39.929 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:30:39.929 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:30:39.929 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:30:39.929 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:39.929 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 55636 2024-09-14T06:30:39.930 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:39.930 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:30:39.930 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:39.930 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:30:39.930 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600078+0000", 2024-09-14T06:30:39.930 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:39.930 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:39.930 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:39.930 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:30:39.930 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600078+0000", 2024-09-14T06:30:39.930 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:39.930 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:39.930 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:39.930 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:30:39.931 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600078+0000", 2024-09-14T06:30:39.931 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.34e-07 2024-09-14T06:30:39.931 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:39.931 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:39.931 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:30:39.931 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600080+0000", 2024-09-14T06:30:39.931 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.641e-06 2024-09-14T06:30:39.931 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:39.931 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:39.931 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:30:39.931 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600086+0000", 2024-09-14T06:30:39.931 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.3860000000000004e-06 2024-09-14T06:30:39.931 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:39.931 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:39.932 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:30:39.932 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600091+0000", 2024-09-14T06:30:39.932 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.5970000000000002e-06 2024-09-14T06:30:39.932 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:39.932 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:39.932 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:30:39.932 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.643184+0000", 2024-09-14T06:30:39.932 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.043092893 2024-09-14T06:30:39.932 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:39.932 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:39.932 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:30:39.932 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.643228+0000", 2024-09-14T06:30:39.932 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.4536999999999997e-05 2024-09-14T06:30:39.933 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:39.933 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:39.933 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:30:39.933 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.643286+0000", 2024-09-14T06:30:39.933 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.7578000000000003e-05 2024-09-14T06:30:39.933 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:39.933 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:39.933 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:39.933 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:39.933 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:39.933 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:55637 3.0 3:06490baf:::rbd_data.112dca2f6ae1.0000000000000030:17 [read 1196032~4096 [fadvise_nocache]] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:30:39.933 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:44.600097+0000", 2024-09-14T06:30:39.933 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 55.267392016999999, 2024-09-14T06:30:39.933 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.043399124999999997, 2024-09-14T06:30:39.934 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:30:39.934 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:30:39.934 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:30:39.934 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:30:39.934 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:30:39.934 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:39.934 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 55637 2024-09-14T06:30:39.934 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:39.934 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:30:39.935 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:39.935 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:30:39.935 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600097+0000", 2024-09-14T06:30:39.935 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:39.935 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:39.935 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:39.935 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:30:39.935 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600097+0000", 2024-09-14T06:30:39.935 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:39.935 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:39.935 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:39.935 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:30:39.935 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.604372+0000", 2024-09-14T06:30:39.936 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0042751489999999998 2024-09-14T06:30:39.936 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:39.936 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:39.936 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:30:39.936 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.604377+0000", 2024-09-14T06:30:39.936 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.7899999999999999e-06 2024-09-14T06:30:39.936 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:39.936 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:39.936 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:30:39.936 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.604389+0000", 2024-09-14T06:30:39.936 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.2197e-05 2024-09-14T06:30:39.936 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:39.936 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:39.936 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:30:39.937 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.604397+0000", 2024-09-14T06:30:39.937 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.8730000000000007e-06 2024-09-14T06:30:39.937 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:39.937 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:39.937 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:30:39.937 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.643324+0000", 2024-09-14T06:30:39.937 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.038926897000000002 2024-09-14T06:30:39.937 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:39.937 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:39.937 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:30:39.937 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.643402+0000", 2024-09-14T06:30:39.937 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.8063000000000001e-05 2024-09-14T06:30:39.937 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:39.937 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:39.938 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:30:39.938 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.643496+0000", 2024-09-14T06:30:39.938 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.4155999999999998e-05 2024-09-14T06:30:39.938 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:39.938 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:39.938 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:39.938 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:39.938 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:39.938 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:55657 3.0 3:06490baf:::rbd_data.112dca2f6ae1.0000000000000030:17 [read 1687552~4096 [fadvise_nocache]] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:30:39.938 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:44.604841+0000", 2024-09-14T06:30:39.938 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 55.262647287999997, 2024-09-14T06:30:39.938 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.041890087999999999, 2024-09-14T06:30:39.938 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:30:39.938 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:30:39.938 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:30:39.939 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:30:39.939 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:30:39.939 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:39.939 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 55657 2024-09-14T06:30:39.939 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:39.939 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:30:39.939 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:39.939 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:30:39.939 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.604841+0000", 2024-09-14T06:30:39.939 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:39.939 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:39.939 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:39.939 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:30:39.939 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.604841+0000", 2024-09-14T06:30:39.939 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:39.940 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:39.940 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:39.940 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:30:39.940 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.604842+0000", 2024-09-14T06:30:39.940 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.0900000000000001e-07 2024-09-14T06:30:39.940 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:39.940 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:39.940 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:30:39.940 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.604846+0000", 2024-09-14T06:30:39.940 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.5759999999999999e-06 2024-09-14T06:30:39.940 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:39.940 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:39.940 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:30:39.940 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.604850+0000", 2024-09-14T06:30:39.941 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.405e-06 2024-09-14T06:30:39.941 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:39.941 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:39.941 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:30:39.941 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.604852+0000", 2024-09-14T06:30:39.941 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.8590000000000001e-06 2024-09-14T06:30:39.941 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:39.941 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:39.941 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:30:39.941 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.646633+0000", 2024-09-14T06:30:39.941 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.041780772000000001 2024-09-14T06:30:39.941 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:39.941 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:39.942 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:30:39.942 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.646677+0000", 2024-09-14T06:30:39.942 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.4115e-05 2024-09-14T06:30:39.942 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:39.942 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:39.942 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:30:39.942 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.646731+0000", 2024-09-14T06:30:39.942 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.4151999999999999e-05 2024-09-14T06:30:39.942 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:39.942 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:39.942 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:39.942 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:39.942 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:39.942 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:55658 3.0 3:06490baf:::rbd_data.112dca2f6ae1.0000000000000030:17 [read 1712128~4096 [fadvise_nocache]] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:30:39.942 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:44.604856+0000", 2024-09-14T06:30:39.943 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 55.262632687999997, 2024-09-14T06:30:39.943 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.041986227000000001, 2024-09-14T06:30:39.943 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:30:39.943 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:30:39.943 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:30:39.943 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:30:39.943 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:30:39.943 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:39.943 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 55658 2024-09-14T06:30:39.943 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:39.943 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:30:39.943 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:39.943 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:30:39.943 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.604856+0000", 2024-09-14T06:30:39.944 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:39.944 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:39.944 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:39.944 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:30:39.944 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.604856+0000", 2024-09-14T06:30:39.944 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:39.944 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:39.944 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:39.944 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:30:39.944 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.604856+0000", 2024-09-14T06:30:39.944 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.23e-07 2024-09-14T06:30:39.944 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:39.944 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:39.944 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:30:39.945 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.604857+0000", 2024-09-14T06:30:39.945 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.9999999999999995e-07 2024-09-14T06:30:39.945 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:39.945 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:39.945 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:30:39.945 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.604861+0000", 2024-09-14T06:30:39.945 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.9890000000000003e-06 2024-09-14T06:30:39.945 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:39.945 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:39.945 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:30:39.945 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.604864+0000", 2024-09-14T06:30:39.945 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.7429999999999998e-06 2024-09-14T06:30:39.945 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:39.946 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:39.946 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:30:39.946 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.646744+0000", 2024-09-14T06:30:39.946 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.041880325000000003 2024-09-14T06:30:39.946 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:39.946 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:39.946 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:30:39.946 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.646784+0000", 2024-09-14T06:30:39.947 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.9604000000000002e-05 2024-09-14T06:30:39.947 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:39.947 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:39.947 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:30:39.947 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.646842+0000", 2024-09-14T06:30:39.947 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.8343e-05 2024-09-14T06:30:39.947 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:39.947 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:39.947 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:39.947 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:39.947 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:39.947 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:55659 3.0 3:06490baf:::rbd_data.112dca2f6ae1.0000000000000030:17 [read 1736704~4096 [fadvise_nocache]] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:30:39.947 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:44.604867+0000", 2024-09-14T06:30:39.947 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 55.262621535000001, 2024-09-14T06:30:39.948 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.042097082000000001, 2024-09-14T06:30:39.948 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:30:39.948 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:30:39.948 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:30:39.948 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:30:39.948 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:30:39.948 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:39.948 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 55659 2024-09-14T06:30:39.948 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:39.948 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:30:39.948 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:39.948 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:30:39.948 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.604867+0000", 2024-09-14T06:30:39.948 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:39.948 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:39.949 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:39.949 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:30:39.949 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.604867+0000", 2024-09-14T06:30:39.949 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:39.949 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:39.949 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:39.949 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:30:39.949 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.604867+0000", 2024-09-14T06:30:39.949 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.2000000000000001e-07 2024-09-14T06:30:39.949 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:39.949 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:39.949 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:30:39.949 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.604868+0000", 2024-09-14T06:30:39.950 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.04e-06 2024-09-14T06:30:39.950 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:39.950 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:39.950 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:30:39.950 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.604871+0000", 2024-09-14T06:30:39.950 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.9689999999999999e-06 2024-09-14T06:30:39.950 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:39.950 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:39.950 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:30:39.950 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.604874+0000", 2024-09-14T06:30:39.950 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.5069999999999999e-06 2024-09-14T06:30:39.950 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:39.950 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:39.951 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:30:39.951 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.646874+0000", 2024-09-14T06:30:39.951 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.042000513000000003 2024-09-14T06:30:39.951 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:39.951 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:39.951 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:30:39.951 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.646914+0000", 2024-09-14T06:30:39.951 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.9443999999999997e-05 2024-09-14T06:30:39.951 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:39.951 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:39.951 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:30:39.951 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.646964+0000", 2024-09-14T06:30:39.951 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.0389000000000002e-05 2024-09-14T06:30:39.951 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:39.952 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:39.952 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:39.952 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:39.952 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:39.952 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:55660 3.0 3:06490baf:::rbd_data.112dca2f6ae1.0000000000000030:17 [read 1761280~4096 [fadvise_nocache]] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:30:39.952 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:44.604877+0000", 2024-09-14T06:30:39.952 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 55.262611786000001, 2024-09-14T06:30:39.952 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.042196349000000001, 2024-09-14T06:30:39.952 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:30:39.952 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:30:39.952 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:30:39.952 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:30:39.952 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:30:39.953 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:39.953 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 55660 2024-09-14T06:30:39.953 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:39.953 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:30:39.953 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:39.953 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:30:39.953 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.604877+0000", 2024-09-14T06:30:39.953 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:39.953 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:39.953 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:39.953 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:30:39.953 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.604877+0000", 2024-09-14T06:30:39.953 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:39.953 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:39.954 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:39.954 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:30:39.954 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.605810+0000", 2024-09-14T06:30:39.954 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00093336899999999997 2024-09-14T06:30:39.954 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:39.954 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:39.954 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:30:39.954 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.605815+0000", 2024-09-14T06:30:39.954 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.6140000000000002e-06 2024-09-14T06:30:39.954 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:39.954 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:39.954 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:30:39.954 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.605825+0000", 2024-09-14T06:30:39.954 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.8889999999999997e-06 2024-09-14T06:30:39.954 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:39.955 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:39.955 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:30:39.955 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.605831+0000", 2024-09-14T06:30:39.955 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.9800000000000003e-06 2024-09-14T06:30:39.955 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:39.955 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:39.955 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:30:39.955 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.646989+0000", 2024-09-14T06:30:39.955 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.041157878000000002 2024-09-14T06:30:39.955 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:39.955 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:39.955 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:30:39.955 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.647026+0000", 2024-09-14T06:30:39.956 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.7305000000000003e-05 2024-09-14T06:30:39.956 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:39.956 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:39.956 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:30:39.956 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.647073+0000", 2024-09-14T06:30:39.956 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.7314000000000001e-05 2024-09-14T06:30:39.956 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:39.956 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:39.956 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:39.956 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:39.956 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:39.956 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:55666 3.0 3:06490baf:::rbd_data.112dca2f6ae1.0000000000000030:17 [read 1908736~4096 [fadvise_nocache]] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:30:39.956 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:44.605898+0000", 2024-09-14T06:30:39.956 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 55.261591039999999, 2024-09-14T06:30:39.957 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.041862887000000001, 2024-09-14T06:30:39.957 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:30:39.957 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:30:39.957 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:30:39.957 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:30:39.957 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:30:39.957 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:39.957 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 55666 2024-09-14T06:30:39.957 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:39.957 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:30:39.957 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:39.957 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:30:39.957 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.605898+0000", 2024-09-14T06:30:39.958 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:39.958 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:39.958 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:39.958 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:30:39.958 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.605898+0000", 2024-09-14T06:30:39.958 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:39.958 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:39.958 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:39.958 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:30:39.958 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.611874+0000", 2024-09-14T06:30:39.958 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0059764049999999997 2024-09-14T06:30:39.958 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:39.958 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:39.958 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:30:39.959 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.611879+0000", 2024-09-14T06:30:39.959 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.8749999999999999e-06 2024-09-14T06:30:39.959 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:39.959 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:39.959 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:30:39.959 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.611887+0000", 2024-09-14T06:30:39.959 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.4109999999999996e-06 2024-09-14T06:30:39.959 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:39.959 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:39.959 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:30:39.959 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.611893+0000", 2024-09-14T06:30:39.959 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.2959999999999998e-06 2024-09-14T06:30:39.959 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:39.959 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:39.959 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:30:39.960 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.647674+0000", 2024-09-14T06:30:39.960 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.035781281999999998 2024-09-14T06:30:39.960 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:39.960 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:39.960 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:30:39.960 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.647711+0000", 2024-09-14T06:30:39.960 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.7434000000000001e-05 2024-09-14T06:30:39.960 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:39.960 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:39.960 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:30:39.960 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.647760+0000", 2024-09-14T06:30:39.960 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.9184000000000002e-05 2024-09-14T06:30:39.960 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:39.960 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:39.961 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:39.961 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:39.961 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:39.961 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-14T06:30:39.962 INFO:teuthology.orchestra.run.smithi094.stdout:3433 writesame 0x52d0 thru 0xd42e data_size 0x1(0x815f bytes) 2024-09-14T06:30:39.963 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-14T06:30:39.963 INFO:teuthology.orchestra.run.smithi094.stdout:3434 write 0xea6b1c4 thru 0xea72dc8 (0x7c05 bytes) 2024-09-14T06:30:39.964 INFO:teuthology.orchestra.run.smithi094.stdout:3436 write 0x5a33db1 thru 0x5a419bd (0xdc0d bytes) 2024-09-14T06:30:39.964 INFO:teuthology.orchestra.run.smithi094.stdout:3437 read 0x7853d62 thru 0x78573c0 (0x365f bytes) 2024-09-14T06:30:39.964 INFO:teuthology.orchestra.run.smithi094.stdout:3438 read 0x50a0c75 thru 0x50a13b7 (0x743 bytes) 2024-09-14T06:30:39.964 INFO:teuthology.orchestra.run.smithi094.stdout:3440 trunc from 0xea72dc9 to 0x2da79a6 2024-09-14T06:30:39.964 INFO:teuthology.orchestra.run.smithi094.stdout:3442 read 0x24e62da thru 0x24ed63d (0x7364 bytes) 2024-09-14T06:30:39.964 INFO:teuthology.orchestra.run.smithi094.stdout:3443 punch from 0x2772c61 to 0x2780e51, (0xe1f0 bytes) 2024-09-14T06:30:39.964 INFO:tasks.ceph.osd.3.smithi094.stderr:2024-09-14T06:30:39.960+0000 7feeeafe8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33655) UID: 0 2024-09-14T06:30:39.970 INFO:teuthology.orchestra.run.smithi094.stdout:3444 write 0x9fcc5e9 thru 0x9fd0f2f (0x4947 bytes) 2024-09-14T06:30:39.978 INFO:teuthology.orchestra.run.smithi094.stdout:3446 write 0x2a8bd43 thru 0x2a992df (0xd59d bytes) 2024-09-14T06:30:39.989 INFO:teuthology.orchestra.run.smithi094.stdout:3447 write 0xbe2ab07 thru 0xbe36af1 (0xbfeb bytes) 2024-09-14T06:30:39.996 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-14T06:30:39.996 INFO:teuthology.orchestra.run.smithi094.stdout:3449 write 0xb1d9822 thru 0xb1dc749 (0x2f28 bytes) 2024-09-14T06:30:39.999 INFO:teuthology.orchestra.run.smithi094.stdout:3450 punch from 0x1b0847a to 0x1b1416f, (0xbcf5 bytes) 2024-09-14T06:30:39.999 INFO:teuthology.orchestra.run.smithi094.stdout:3451 write 0x3b37662 thru 0x3b44d0f (0xd6ae bytes) 2024-09-14T06:30:40.015 INFO:teuthology.orchestra.run.smithi094.stdout:3452 trunc from 0xbe36af2 to 0x6c8a634 2024-09-14T06:30:40.027 INFO:teuthology.orchestra.run.smithi094.stdout:3453 writesame 0x4300363 thru 0x4301ec7 data_size 0x1(0x1b65 bytes) 2024-09-14T06:30:40.042 INFO:teuthology.orchestra.run.smithi094.stdout:3455 write 0xc76c7f4 thru 0xc77351e (0x6d2b bytes) 2024-09-14T06:30:40.048 INFO:teuthology.orchestra.run.smithi094.stdout:3456 writesame 0xbb8f686 thru 0xbb9ca71 data_size 0xc(0xd3ec bytes) 2024-09-14T06:30:40.058 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-14T06:30:40.058 INFO:teuthology.orchestra.run.smithi094.stdout:3458 punch from 0x81298d6 to 0x812ff52, (0x667c bytes) 2024-09-14T06:30:40.063 INFO:tasks.ceph.osd.3.smithi094.stderr:2024-09-14T06:30:40.061+0000 7feeeafe8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33655) UID: 0 2024-09-14T06:30:40.067 INFO:teuthology.orchestra.run.smithi094.stdout:3459 trunc from 0xc77351f to 0x4fe0ec1 2024-09-14T06:30:40.087 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-14T06:30:40.087 INFO:teuthology.orchestra.run.smithi094.stdout: "ops": [], 2024-09-14T06:30:40.087 INFO:teuthology.orchestra.run.smithi094.stdout: "num_ops": 0 2024-09-14T06:30:40.087 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-14T06:30:40.099 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-14T06:30:40.099 INFO:teuthology.orchestra.run.smithi094.stdout:3460 write 0xa36e495 thru 0xa3713d4 (0x2f40 bytes) 2024-09-14T06:30:40.106 INFO:teuthology.orchestra.run.smithi094.stdout:3461 read 0x1780c23 thru 0x178bd24 (0xb102 bytes) 2024-09-14T06:30:40.108 INFO:teuthology.orchestra.run.smithi094.stdout:3463 punch from 0x7e7dc9a to 0x7e81e78, (0x41de bytes) 2024-09-14T06:30:40.108 INFO:teuthology.orchestra.run.smithi094.stdout:3464 read 0x9a1f3b0 thru 0x9a2dd58 (0xe9a9 bytes) 2024-09-14T06:30:40.117 INFO:teuthology.orchestra.run.smithi094.stdout:3465 trunc from 0xa3713d5 to 0x1ae45f4 2024-09-14T06:30:40.143 INFO:teuthology.orchestra.run.smithi094.stdout:3466 write 0x25e6243 thru 0x25f47d3 (0xe591 bytes) 2024-09-14T06:30:40.155 INFO:teuthology.orchestra.run.smithi094.stdout:3467 read 0xdedd0e thru 0xdef0b6 (0x13a9 bytes) 2024-09-14T06:30:40.164 INFO:tasks.ceph.osd.0.smithi094.stderr:2024-09-14T06:30:40.162+0000 7f07c9333640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33636) UID: 0 2024-09-14T06:30:40.171 INFO:teuthology.orchestra.run.smithi094.stdout:3468 write 0xa6c1485 thru 0xa6c9165 (0x7ce1 bytes) 2024-09-14T06:30:40.180 INFO:teuthology.orchestra.run.smithi094.stdout:3469 clone 26 order 19 su 131072 sc 15 2024-09-14T06:30:40.218 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-14T06:30:40.218 INFO:teuthology.orchestra.run.smithi094.stdout: "ops": [], 2024-09-14T06:30:40.218 INFO:teuthology.orchestra.run.smithi094.stdout: "complaint_time": 30, 2024-09-14T06:30:40.218 INFO:teuthology.orchestra.run.smithi094.stdout: "num_blocked_ops": 0 2024-09-14T06:30:40.218 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-14T06:30:40.226 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-14T06:30:40.268 INFO:tasks.ceph.osd.2.smithi094.stderr:2024-09-14T06:30:40.267+0000 7faf3269b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33649) UID: 0 2024-09-14T06:30:40.366 INFO:tasks.ceph.osd.6.smithi138.stderr:2024-09-14T06:30:40.364+0000 7f8cd8849640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33620) UID: 0 2024-09-14T06:30:40.371 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-14T06:30:40.371 INFO:teuthology.orchestra.run.smithi094.stdout: "size": 20, 2024-09-14T06:30:40.371 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 600, 2024-09-14T06:30:40.371 INFO:teuthology.orchestra.run.smithi094.stdout: "ops": [ 2024-09-14T06:30:40.371 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:40.371 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4397.0:1848 3.4 e32/23 3:27620a21:::rbd_data.112dcdddac74.000000000000000b:head v 32'40, mlcod=30'38)", 2024-09-14T06:30:40.371 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:08.658649+0000", 2024-09-14T06:30:40.371 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 91.710730153, 2024-09-14T06:30:40.371 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.001862867, 2024-09-14T06:30:40.372 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:30:40.372 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:30:40.372 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:30:40.372 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:30:40.372 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:40.372 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:30:40.372 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.658649+0000", 2024-09-14T06:30:40.372 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:40.372 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:40.372 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:40.372 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:30:40.372 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.658649+0000", 2024-09-14T06:30:40.373 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:40.373 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:40.373 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:40.373 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:30:40.373 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.658651+0000", 2024-09-14T06:30:40.373 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.5650000000000001e-06 2024-09-14T06:30:40.373 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:40.373 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:40.373 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:30:40.373 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.658709+0000", 2024-09-14T06:30:40.373 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.8442000000000001e-05 2024-09-14T06:30:40.373 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:40.373 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:40.373 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:30:40.374 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.658730+0000", 2024-09-14T06:30:40.374 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.0509999999999998e-05 2024-09-14T06:30:40.374 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:40.374 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:40.374 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:30:40.374 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.658740+0000", 2024-09-14T06:30:40.374 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.8619999999999998e-06 2024-09-14T06:30:40.374 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:40.374 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:40.374 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:30:40.374 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.658813+0000", 2024-09-14T06:30:40.374 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.3578000000000006e-05 2024-09-14T06:30:40.374 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:40.374 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:40.375 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:30:40.375 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.658836+0000", 2024-09-14T06:30:40.375 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.2677e-05 2024-09-14T06:30:40.375 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:40.375 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:40.375 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-14T06:30:40.375 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.660462+0000", 2024-09-14T06:30:40.375 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00162637 2024-09-14T06:30:40.375 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:40.375 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:40.375 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:30:40.375 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.660512+0000", 2024-09-14T06:30:40.375 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.9863e-05 2024-09-14T06:30:40.375 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:40.376 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:40.376 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:40.376 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:40.376 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:40.376 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4397.0:2199 3.0 e32/23 3:05f3e980:::rbd_data.112dcdddac74.000000000000000f:head v 32'3, mlcod=31'1)", 2024-09-14T06:30:40.376 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:08.680236+0000", 2024-09-14T06:30:40.376 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 91.689143787999996, 2024-09-14T06:30:40.376 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0019068990000000001, 2024-09-14T06:30:40.376 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:30:40.376 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:30:40.376 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:30:40.376 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:30:40.376 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:40.376 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:30:40.376 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.680236+0000", 2024-09-14T06:30:40.377 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:40.377 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:40.377 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:40.377 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:30:40.377 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.680236+0000", 2024-09-14T06:30:40.377 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:40.377 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:40.377 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:40.377 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:30:40.377 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.680237+0000", 2024-09-14T06:30:40.377 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.4980000000000001e-06 2024-09-14T06:30:40.377 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:40.377 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:40.377 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:30:40.377 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.680324+0000", 2024-09-14T06:30:40.378 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.7310000000000003e-05 2024-09-14T06:30:40.378 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:40.378 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:40.378 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:30:40.378 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.680354+0000", 2024-09-14T06:30:40.378 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.97e-05 2024-09-14T06:30:40.378 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:40.378 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:40.378 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:30:40.378 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.680366+0000", 2024-09-14T06:30:40.378 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.2048999999999999e-05 2024-09-14T06:30:40.378 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:40.378 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:40.378 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:30:40.378 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.680439+0000", 2024-09-14T06:30:40.378 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.2910000000000005e-05 2024-09-14T06:30:40.379 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:40.379 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:40.379 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:30:40.379 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.680465+0000", 2024-09-14T06:30:40.379 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.5812999999999998e-05 2024-09-14T06:30:40.379 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:40.379 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:40.379 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-14T06:30:40.379 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.682094+0000", 2024-09-14T06:30:40.379 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.001628876 2024-09-14T06:30:40.379 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:40.379 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:40.379 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:30:40.379 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.682142+0000", 2024-09-14T06:30:40.379 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.8742999999999997e-05 2024-09-14T06:30:40.380 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:40.380 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:40.380 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:40.380 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:40.380 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:40.380 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4397.0:3292 3.9 e32/23 3:91dbce57:::rbd_data.112dcdddac74.0000000000000036:head v 32'10, mlcod=32'8)", 2024-09-14T06:30:40.380 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:08.740506+0000", 2024-09-14T06:30:40.380 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 91.628873076000005, 2024-09-14T06:30:40.380 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.001864166, 2024-09-14T06:30:40.380 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:30:40.380 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:30:40.380 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:30:40.380 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:30:40.381 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:40.381 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:30:40.381 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.740506+0000", 2024-09-14T06:30:40.381 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:40.381 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:40.381 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:40.381 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:30:40.381 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.740506+0000", 2024-09-14T06:30:40.381 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:40.381 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:40.381 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:40.381 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:30:40.381 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.740514+0000", 2024-09-14T06:30:40.382 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.2109999999999998e-06 2024-09-14T06:30:40.382 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:40.382 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:40.382 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:30:40.382 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.740745+0000", 2024-09-14T06:30:40.382 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000230194 2024-09-14T06:30:40.382 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:40.382 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:40.382 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:30:40.382 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.740764+0000", 2024-09-14T06:30:40.382 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.9330999999999998e-05 2024-09-14T06:30:40.382 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:40.382 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:40.383 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:30:40.383 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.740774+0000", 2024-09-14T06:30:40.383 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.6260000000000007e-06 2024-09-14T06:30:40.383 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:40.383 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:40.383 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:30:40.383 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.740839+0000", 2024-09-14T06:30:40.383 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.5314000000000005e-05 2024-09-14T06:30:40.383 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:40.383 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:40.383 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:30:40.383 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.740860+0000", 2024-09-14T06:30:40.383 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.0868000000000001e-05 2024-09-14T06:30:40.383 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:40.384 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:40.384 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-14T06:30:40.384 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.742327+0000", 2024-09-14T06:30:40.384 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0014672909999999999 2024-09-14T06:30:40.384 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:40.384 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:40.384 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:30:40.384 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.742370+0000", 2024-09-14T06:30:40.384 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.3331000000000002e-05 2024-09-14T06:30:40.385 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:40.385 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:40.385 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:40.385 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:40.385 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:40.385 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4397.0:26415 3.0 e53/23 3:05fd9be0:::rbd_data.112d38768680.0000000000000006:head v 53'21, mlcod=50'19)", 2024-09-14T06:30:40.385 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:30.721610+0000", 2024-09-14T06:30:40.385 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 69.647769006999994, 2024-09-14T06:30:40.385 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0021763160000000002, 2024-09-14T06:30:40.385 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:30:40.385 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:30:40.385 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:30:40.385 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:30:40.385 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:40.386 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:30:40.386 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:30.721610+0000", 2024-09-14T06:30:40.386 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:40.386 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:40.386 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:40.386 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:30:40.386 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:30.721610+0000", 2024-09-14T06:30:40.386 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:40.386 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:40.386 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:40.386 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:30:40.386 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:30.721611+0000", 2024-09-14T06:30:40.386 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.41e-07 2024-09-14T06:30:40.386 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:40.386 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:40.386 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:30:40.387 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:30.721905+0000", 2024-09-14T06:30:40.387 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00029394300000000001 2024-09-14T06:30:40.387 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:40.387 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:40.387 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:30:40.387 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:30.721921+0000", 2024-09-14T06:30:40.387 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.6609999999999999e-05 2024-09-14T06:30:40.387 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:40.387 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:40.387 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:30:40.387 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:30.721931+0000", 2024-09-14T06:30:40.387 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.6739999999999992e-06 2024-09-14T06:30:40.387 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:40.387 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:40.388 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:30:40.388 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:30.722001+0000", 2024-09-14T06:30:40.388 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.0549000000000005e-05 2024-09-14T06:30:40.388 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:40.388 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:40.388 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:30:40.388 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:30.722029+0000", 2024-09-14T06:30:40.388 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.7327000000000001e-05 2024-09-14T06:30:40.388 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:40.388 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:40.388 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-14T06:30:40.388 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:30.723727+0000", 2024-09-14T06:30:40.388 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0016977310000000001 2024-09-14T06:30:40.388 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:40.388 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:40.389 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:30:40.389 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:30.723787+0000", 2024-09-14T06:30:40.389 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.0140999999999999e-05 2024-09-14T06:30:40.389 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:40.389 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:40.389 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:40.389 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:40.389 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:40.389 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4397.0:36084 3.0 e56/23 3:05a48ef4:::rbd_data.112dc58639fa.0000000000000058:head v 56'29, mlcod=56'27)", 2024-09-14T06:30:40.389 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:34.508855+0000", 2024-09-14T06:30:40.389 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 65.860524252999994, 2024-09-14T06:30:40.389 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0020801140000000001, 2024-09-14T06:30:40.389 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:30:40.390 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:30:40.390 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:30:40.390 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:30:40.390 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:40.390 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:30:40.390 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:34.508855+0000", 2024-09-14T06:30:40.390 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:40.390 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:40.390 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:40.390 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:30:40.390 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:34.508855+0000", 2024-09-14T06:30:40.390 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:40.390 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:40.391 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:40.391 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:30:40.391 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:34.508856+0000", 2024-09-14T06:30:40.391 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.1909999999999999e-06 2024-09-14T06:30:40.391 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:40.391 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:40.391 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:30:40.391 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:34.509203+0000", 2024-09-14T06:30:40.391 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00034625900000000002 2024-09-14T06:30:40.391 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:40.391 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:40.391 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:30:40.391 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:34.509218+0000", 2024-09-14T06:30:40.391 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.5091e-05 2024-09-14T06:30:40.391 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:40.392 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:40.392 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:30:40.392 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:34.509227+0000", 2024-09-14T06:30:40.392 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.0850000000000003e-06 2024-09-14T06:30:40.392 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:40.392 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:40.392 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:30:40.392 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:34.509311+0000", 2024-09-14T06:30:40.392 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.4436000000000003e-05 2024-09-14T06:30:40.392 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:40.392 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:40.392 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:30:40.392 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:34.509335+0000", 2024-09-14T06:30:40.392 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.3577000000000001e-05 2024-09-14T06:30:40.392 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:40.393 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:40.393 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-14T06:30:40.393 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:34.510890+0000", 2024-09-14T06:30:40.393 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0015554259999999999 2024-09-14T06:30:40.393 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:40.393 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:40.393 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:30:40.393 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:34.510935+0000", 2024-09-14T06:30:40.393 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.5049000000000002e-05 2024-09-14T06:30:40.393 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:40.393 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:40.393 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:40.393 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:40.393 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:40.394 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4397.0:36966 3.4 e56/23 3:22008318:::rbd_data.112dc58639fa.0000000000000066:head v 56'56, mlcod=50'54)", 2024-09-14T06:30:40.394 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:34.565614+0000", 2024-09-14T06:30:40.394 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 65.803765816999999, 2024-09-14T06:30:40.394 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0018691929999999999, 2024-09-14T06:30:40.394 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:30:40.394 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:30:40.394 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:30:40.394 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:30:40.394 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:40.394 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:30:40.394 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:34.565614+0000", 2024-09-14T06:30:40.394 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:40.394 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:40.394 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:40.394 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:30:40.394 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:34.565614+0000", 2024-09-14T06:30:40.395 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:40.395 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:40.395 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:40.395 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:30:40.395 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:34.565615+0000", 2024-09-14T06:30:40.395 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.784e-06 2024-09-14T06:30:40.395 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:40.395 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:40.395 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:30:40.395 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:34.565919+0000", 2024-09-14T06:30:40.395 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00030351599999999999 2024-09-14T06:30:40.395 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:40.395 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:40.395 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:30:40.395 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:34.565933+0000", 2024-09-14T06:30:40.395 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.4465e-05 2024-09-14T06:30:40.396 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:40.396 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:40.396 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:30:40.396 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:34.565943+0000", 2024-09-14T06:30:40.396 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.7920000000000004e-06 2024-09-14T06:30:40.396 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:40.396 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:40.396 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:30:40.396 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:34.565999+0000", 2024-09-14T06:30:40.396 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.5643000000000002e-05 2024-09-14T06:30:40.396 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:40.396 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:40.396 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:30:40.396 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:34.566025+0000", 2024-09-14T06:30:40.396 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.6040000000000001e-05 2024-09-14T06:30:40.397 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:40.397 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:40.397 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-14T06:30:40.397 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:34.567430+0000", 2024-09-14T06:30:40.397 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.001405653 2024-09-14T06:30:40.397 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:40.397 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:40.397 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:30:40.397 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:34.567483+0000", 2024-09-14T06:30:40.397 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.2299999999999997e-05 2024-09-14T06:30:40.397 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:40.397 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:40.397 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:40.397 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:40.397 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:40.398 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4397.0:45722 3.9 e64/23 3:91fed098:::rbd_data.112d5a737e72.0000000000000001:head v 64'61, mlcod=62'59)", 2024-09-14T06:30:40.398 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:43.864269+0000", 2024-09-14T06:30:40.398 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 56.505110178000002, 2024-09-14T06:30:40.398 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.003055224, 2024-09-14T06:30:40.398 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:30:40.398 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:30:40.398 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:30:40.398 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:30:40.398 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:40.398 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:30:40.398 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.864269+0000", 2024-09-14T06:30:40.398 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:40.398 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:40.398 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:40.398 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:30:40.398 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.864269+0000", 2024-09-14T06:30:40.399 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:40.399 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:40.399 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:40.399 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:30:40.399 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.864269+0000", 2024-09-14T06:30:40.399 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.41e-07 2024-09-14T06:30:40.399 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:40.399 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:40.399 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:30:40.399 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.865490+0000", 2024-09-14T06:30:40.399 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.001220552 2024-09-14T06:30:40.399 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:40.399 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:40.399 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:30:40.399 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.865506+0000", 2024-09-14T06:30:40.400 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.6263999999999999e-05 2024-09-14T06:30:40.400 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:40.400 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:40.400 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:30:40.400 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.865516+0000", 2024-09-14T06:30:40.400 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.0091999999999999e-05 2024-09-14T06:30:40.400 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:40.400 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:40.400 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:30:40.400 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.865583+0000", 2024-09-14T06:30:40.400 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.6676999999999999e-05 2024-09-14T06:30:40.400 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:40.400 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:40.400 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:30:40.400 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.865610+0000", 2024-09-14T06:30:40.401 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.6529999999999998e-05 2024-09-14T06:30:40.401 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:40.401 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:40.401 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-14T06:30:40.401 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.867281+0000", 2024-09-14T06:30:40.401 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.001671203 2024-09-14T06:30:40.401 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:40.401 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:40.401 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:30:40.401 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.867324+0000", 2024-09-14T06:30:40.401 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.3665000000000003e-05 2024-09-14T06:30:40.401 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:40.401 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:40.401 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:40.401 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:40.402 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:40.402 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4397.0:49627 3.4 e64/23 3:21e30d4b:::rbd_data.112d5a737e72.0000000000000017:head v 64'69, mlcod=61'67)", 2024-09-14T06:30:40.402 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:44.206446+0000", 2024-09-14T06:30:40.402 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 56.162933539999997, 2024-09-14T06:30:40.402 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0018142220000000001, 2024-09-14T06:30:40.402 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:30:40.402 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:30:40.402 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:30:40.402 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:30:40.402 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:40.402 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:30:40.402 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.206446+0000", 2024-09-14T06:30:40.402 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:40.402 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:40.402 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:40.403 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:30:40.403 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.206446+0000", 2024-09-14T06:30:40.403 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:40.403 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:40.403 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:40.403 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:30:40.403 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.206446+0000", 2024-09-14T06:30:40.403 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.4900000000000001e-07 2024-09-14T06:30:40.403 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:40.403 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:40.403 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:30:40.403 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.206797+0000", 2024-09-14T06:30:40.403 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00035112799999999998 2024-09-14T06:30:40.403 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:40.403 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:40.403 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:30:40.404 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.206816+0000", 2024-09-14T06:30:40.404 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.8977000000000001e-05 2024-09-14T06:30:40.404 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:40.404 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:40.404 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:30:40.404 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.206825+0000", 2024-09-14T06:30:40.404 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.8279999999999992e-06 2024-09-14T06:30:40.404 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:40.404 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:40.404 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:30:40.404 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.206892+0000", 2024-09-14T06:30:40.404 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.6809000000000004e-05 2024-09-14T06:30:40.404 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:40.404 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:40.404 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:30:40.405 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.206915+0000", 2024-09-14T06:30:40.405 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.315e-05 2024-09-14T06:30:40.405 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:40.405 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:40.405 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-14T06:30:40.405 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.208201+0000", 2024-09-14T06:30:40.405 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0012858819999999999 2024-09-14T06:30:40.405 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:40.405 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:40.405 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:30:40.405 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.208260+0000", 2024-09-14T06:30:40.405 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.9098999999999998e-05 2024-09-14T06:30:40.405 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:40.406 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:40.406 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:40.406 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:40.406 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:40.406 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4397.0:50003 3.9 e64/23 3:94f26c4b:::rbd_data.112d5a737e72.000000000000001f:head v 64'62, mlcod=62'60)", 2024-09-14T06:30:40.406 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:44.220413+0000", 2024-09-14T06:30:40.406 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 56.148965861999997, 2024-09-14T06:30:40.406 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0019210430000000001, 2024-09-14T06:30:40.407 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:30:40.407 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:30:40.407 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:30:40.407 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:30:40.407 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:40.407 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:30:40.407 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.220413+0000", 2024-09-14T06:30:40.407 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:40.407 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:40.407 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:40.407 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:30:40.407 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.220413+0000", 2024-09-14T06:30:40.407 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:40.407 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:40.407 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:40.408 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:30:40.408 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.220415+0000", 2024-09-14T06:30:40.408 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.575e-06 2024-09-14T06:30:40.408 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:40.408 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:40.408 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:30:40.408 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.220984+0000", 2024-09-14T06:30:40.408 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000568624 2024-09-14T06:30:40.408 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:40.408 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:40.408 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:30:40.408 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.221007+0000", 2024-09-14T06:30:40.408 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.3112000000000002e-05 2024-09-14T06:30:40.408 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:40.409 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:40.409 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:30:40.409 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.221017+0000", 2024-09-14T06:30:40.409 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.0098999999999999e-05 2024-09-14T06:30:40.409 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:40.409 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:40.409 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:30:40.409 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.221110+0000", 2024-09-14T06:30:40.409 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.3268000000000002e-05 2024-09-14T06:30:40.409 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:40.409 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:40.409 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:30:40.409 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.221131+0000", 2024-09-14T06:30:40.409 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.0905999999999999e-05 2024-09-14T06:30:40.409 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:40.410 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:40.410 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-14T06:30:40.410 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.222290+0000", 2024-09-14T06:30:40.410 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.001159408 2024-09-14T06:30:40.410 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:40.410 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:40.410 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:30:40.410 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.222335+0000", 2024-09-14T06:30:40.410 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.4051000000000002e-05 2024-09-14T06:30:40.410 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:40.410 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:40.410 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:40.410 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:40.410 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:40.410 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4397.0:50006 3.4 e64/23 3:20cfe300:::rbd_data.112d5a737e72.0000000000000019:head v 64'70, mlcod=61'68)", 2024-09-14T06:30:40.410 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:44.224218+0000", 2024-09-14T06:30:40.411 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 56.145160980999997, 2024-09-14T06:30:40.411 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0021104320000000002, 2024-09-14T06:30:40.411 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:30:40.411 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:30:40.411 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:30:40.411 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:30:40.411 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:40.411 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:30:40.411 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.224218+0000", 2024-09-14T06:30:40.411 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:40.411 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:40.411 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:40.411 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:30:40.411 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.224218+0000", 2024-09-14T06:30:40.411 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:40.411 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:40.412 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:40.412 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:30:40.412 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.224219+0000", 2024-09-14T06:30:40.412 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.0049999999999999e-06 2024-09-14T06:30:40.412 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:40.412 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:40.412 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:30:40.412 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.224326+0000", 2024-09-14T06:30:40.412 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000106848 2024-09-14T06:30:40.412 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:40.412 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:40.412 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:30:40.412 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.224345+0000", 2024-09-14T06:30:40.412 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.8587999999999999e-05 2024-09-14T06:30:40.412 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:40.413 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:40.413 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:30:40.413 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.224354+0000", 2024-09-14T06:30:40.413 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.7220000000000005e-06 2024-09-14T06:30:40.413 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:40.413 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:40.413 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:30:40.413 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.224416+0000", 2024-09-14T06:30:40.413 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.2738999999999997e-05 2024-09-14T06:30:40.413 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:40.413 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:40.413 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:30:40.413 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.224441+0000", 2024-09-14T06:30:40.413 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.4408000000000001e-05 2024-09-14T06:30:40.413 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:40.413 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:40.414 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-14T06:30:40.414 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.226295+0000", 2024-09-14T06:30:40.414 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00185428 2024-09-14T06:30:40.414 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:40.414 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:40.414 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:30:40.414 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.226329+0000", 2024-09-14T06:30:40.414 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.3841999999999999e-05 2024-09-14T06:30:40.414 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:40.414 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:40.414 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:40.414 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:40.414 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:40.415 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4397.0:51816 3.4 e64/23 3:21a826f9:::rbd_data.112d5a737e72.000000000000002d:head v 64'72, mlcod=64'70)", 2024-09-14T06:30:40.415 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:44.312072+0000", 2024-09-14T06:30:40.415 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 56.057307508000001, 2024-09-14T06:30:40.415 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0019555240000000002, 2024-09-14T06:30:40.415 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:30:40.415 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:30:40.415 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:30:40.415 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:30:40.415 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:40.415 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:30:40.415 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.312072+0000", 2024-09-14T06:30:40.415 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:40.415 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:40.415 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:40.415 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:30:40.416 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.312072+0000", 2024-09-14T06:30:40.416 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:40.416 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:40.416 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:40.416 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:30:40.416 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.312074+0000", 2024-09-14T06:30:40.416 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.9530000000000002e-06 2024-09-14T06:30:40.416 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:40.416 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:40.416 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:30:40.416 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.312378+0000", 2024-09-14T06:30:40.416 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00030392 2024-09-14T06:30:40.416 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:40.416 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:40.416 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:30:40.417 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.312399+0000", 2024-09-14T06:30:40.417 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.1134000000000001e-05 2024-09-14T06:30:40.417 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:40.417 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:40.417 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:30:40.417 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.312408+0000", 2024-09-14T06:30:40.417 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.2879999999999998e-06 2024-09-14T06:30:40.417 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:40.417 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:40.417 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:30:40.417 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.312494+0000", 2024-09-14T06:30:40.417 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.6280999999999996e-05 2024-09-14T06:30:40.417 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:40.417 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:40.417 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:30:40.417 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.312518+0000", 2024-09-14T06:30:40.418 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.3149000000000001e-05 2024-09-14T06:30:40.418 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:40.418 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:40.418 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-14T06:30:40.418 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.313964+0000", 2024-09-14T06:30:40.418 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.001446754 2024-09-14T06:30:40.418 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:40.418 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:40.418 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:30:40.418 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.314027+0000", 2024-09-14T06:30:40.418 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.3045000000000004e-05 2024-09-14T06:30:40.418 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:40.418 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:40.418 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:40.418 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:40.419 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:40.419 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4397.0:74241 3.0 e73/23 3:06e93fb9:::rbd_data.112d80ca42af.0000000000000000:head v 73'102, mlcod=71'100)", 2024-09-14T06:30:40.419 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:53.876730+0000", 2024-09-14T06:30:40.419 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 46.492649077999999, 2024-09-14T06:30:40.419 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0079470329999999992, 2024-09-14T06:30:40.419 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:30:40.419 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:30:40.419 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:30:40.419 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:30:40.419 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:40.419 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:30:40.419 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:53.876730+0000", 2024-09-14T06:30:40.419 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:40.419 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:40.419 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:40.419 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:30:40.420 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:53.876730+0000", 2024-09-14T06:30:40.420 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:40.420 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:40.420 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:40.420 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:30:40.420 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:53.876731+0000", 2024-09-14T06:30:40.420 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.0500000000000004e-07 2024-09-14T06:30:40.420 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:40.420 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:40.420 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:30:40.420 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:53.879616+0000", 2024-09-14T06:30:40.420 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0028856680000000001 2024-09-14T06:30:40.420 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:40.420 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:40.421 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:30:40.421 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:53.879643+0000", 2024-09-14T06:30:40.421 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.6120999999999999e-05 2024-09-14T06:30:40.421 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:40.421 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:40.421 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:30:40.421 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:53.879654+0000", 2024-09-14T06:30:40.421 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.1586999999999999e-05 2024-09-14T06:30:40.421 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:40.421 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:40.421 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:30:40.421 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:53.879756+0000", 2024-09-14T06:30:40.421 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00010223899999999999 2024-09-14T06:30:40.421 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:40.422 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:40.422 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:30:40.422 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:53.879783+0000", 2024-09-14T06:30:40.422 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.6913000000000001e-05 2024-09-14T06:30:40.422 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:40.422 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:40.422 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-14T06:30:40.422 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:53.884594+0000", 2024-09-14T06:30:40.422 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0048110729999999999 2024-09-14T06:30:40.422 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:40.422 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:40.422 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:30:40.422 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:53.884677+0000", 2024-09-14T06:30:40.422 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.2927000000000001e-05 2024-09-14T06:30:40.422 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:40.423 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:40.423 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:40.423 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:40.423 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:40.423 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4397.0:74243 3.9 e73/23 3:94977f81:::rbd_data.112d80ca42af.0000000000000002:head v 73'94, mlcod=71'92)", 2024-09-14T06:30:40.423 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:53.885156+0000", 2024-09-14T06:30:40.423 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 46.484223159999999, 2024-09-14T06:30:40.423 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0096009860000000006, 2024-09-14T06:30:40.423 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:30:40.423 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:30:40.423 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:30:40.423 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:30:40.423 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:40.423 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:30:40.423 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:53.885156+0000", 2024-09-14T06:30:40.423 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:40.424 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:40.424 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:40.424 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:30:40.424 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:53.885156+0000", 2024-09-14T06:30:40.424 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:40.424 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:40.424 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:40.424 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:30:40.424 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:53.885156+0000", 2024-09-14T06:30:40.424 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.23e-07 2024-09-14T06:30:40.424 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:40.424 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:40.424 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:30:40.424 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:53.889694+0000", 2024-09-14T06:30:40.424 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0045378709999999997 2024-09-14T06:30:40.425 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:40.425 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:40.425 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:30:40.425 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:53.889720+0000", 2024-09-14T06:30:40.425 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.5259e-05 2024-09-14T06:30:40.425 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:40.425 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:40.425 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:30:40.425 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:53.889731+0000", 2024-09-14T06:30:40.425 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.1739000000000001e-05 2024-09-14T06:30:40.425 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:40.425 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:40.425 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:30:40.426 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:53.889822+0000", 2024-09-14T06:30:40.426 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.1200999999999996e-05 2024-09-14T06:30:40.426 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:40.426 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:40.426 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:30:40.426 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:53.889850+0000", 2024-09-14T06:30:40.426 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.7084999999999999e-05 2024-09-14T06:30:40.426 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:40.426 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:40.426 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-14T06:30:40.426 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:53.894703+0000", 2024-09-14T06:30:40.426 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0048532690000000003 2024-09-14T06:30:40.426 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:40.426 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:40.426 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:30:40.427 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:53.894757+0000", 2024-09-14T06:30:40.427 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.4339000000000003e-05 2024-09-14T06:30:40.427 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:40.427 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:40.427 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:40.427 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:40.427 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:40.427 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4397.0:74249 3.0 e73/23 3:009b0225:::rbd_data.112d80ca42af.0000000000000009:head v 73'103, mlcod=71'101)", 2024-09-14T06:30:40.427 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:53.893219+0000", 2024-09-14T06:30:40.427 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 46.476160682, 2024-09-14T06:30:40.427 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0065381980000000003, 2024-09-14T06:30:40.427 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:30:40.427 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:30:40.427 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:30:40.427 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:30:40.428 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:40.428 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:30:40.428 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:53.893219+0000", 2024-09-14T06:30:40.428 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:40.428 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:40.428 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:40.428 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:30:40.428 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:53.893219+0000", 2024-09-14T06:30:40.428 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:40.428 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:40.428 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:40.428 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:30:40.428 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:53.893221+0000", 2024-09-14T06:30:40.428 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.4329999999999998e-06 2024-09-14T06:30:40.428 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:40.429 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:40.429 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:30:40.429 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:53.895968+0000", 2024-09-14T06:30:40.429 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.002746746 2024-09-14T06:30:40.429 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:40.429 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:40.429 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:30:40.429 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:53.895988+0000", 2024-09-14T06:30:40.429 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.0227000000000001e-05 2024-09-14T06:30:40.429 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:40.429 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:40.429 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:30:40.429 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:53.896000+0000", 2024-09-14T06:30:40.429 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.2359e-05 2024-09-14T06:30:40.430 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:40.430 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:40.430 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:30:40.430 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:53.896084+0000", 2024-09-14T06:30:40.430 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.3659999999999995e-05 2024-09-14T06:30:40.430 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:40.430 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:40.430 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:30:40.430 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:53.896109+0000", 2024-09-14T06:30:40.430 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.5259999999999999e-05 2024-09-14T06:30:40.430 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:40.430 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:40.430 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-14T06:30:40.430 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:53.899704+0000", 2024-09-14T06:30:40.430 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0035942209999999999 2024-09-14T06:30:40.431 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:40.431 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:40.431 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:30:40.431 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:53.899757+0000", 2024-09-14T06:30:40.431 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.3291999999999997e-05 2024-09-14T06:30:40.431 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:40.431 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:40.431 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:40.431 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:40.431 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:40.431 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4397.0:78263 3.0 e77/23 3:00899b22:::rbd_data.112ddfbec471.0000000000000016:head v 77'104, mlcod=73'102)", 2024-09-14T06:30:40.431 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:57.634578+0000", 2024-09-14T06:30:40.431 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 42.73480163, 2024-09-14T06:30:40.431 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.002372454, 2024-09-14T06:30:40.432 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:30:40.432 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:30:40.432 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:30:40.432 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:30:40.432 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:40.432 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:30:40.432 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:57.634578+0000", 2024-09-14T06:30:40.432 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:40.432 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:40.432 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:40.432 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:30:40.432 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:57.634578+0000", 2024-09-14T06:30:40.432 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:40.432 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:40.432 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:40.433 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:30:40.433 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:57.634578+0000", 2024-09-14T06:30:40.433 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.5699999999999998e-07 2024-09-14T06:30:40.433 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:40.433 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:40.433 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:30:40.433 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:57.635347+0000", 2024-09-14T06:30:40.433 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00076848300000000003 2024-09-14T06:30:40.433 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:40.433 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:40.433 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:30:40.433 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:57.635361+0000", 2024-09-14T06:30:40.433 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.4238e-05 2024-09-14T06:30:40.433 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:40.434 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:40.434 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:30:40.434 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:57.635371+0000", 2024-09-14T06:30:40.434 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.0087e-05 2024-09-14T06:30:40.434 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:40.434 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:40.434 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:30:40.434 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:57.635439+0000", 2024-09-14T06:30:40.434 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.7953000000000005e-05 2024-09-14T06:30:40.434 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:40.434 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:40.434 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:30:40.434 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:57.635464+0000", 2024-09-14T06:30:40.434 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.5062000000000002e-05 2024-09-14T06:30:40.434 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:40.434 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:40.435 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-14T06:30:40.435 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:57.636914+0000", 2024-09-14T06:30:40.435 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.001449625 2024-09-14T06:30:40.435 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:40.435 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:40.435 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:30:40.435 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:57.636950+0000", 2024-09-14T06:30:40.435 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.6548999999999999e-05 2024-09-14T06:30:40.435 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:40.435 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:40.435 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:40.435 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:40.435 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:40.435 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4397.0:78604 3.9 e77/23 3:94c62a0e:::rbd_data.112ddfbec471.000000000000002c:head v 77'96, mlcod=73'94)", 2024-09-14T06:30:40.436 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:57.677568+0000", 2024-09-14T06:30:40.436 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 42.691811242, 2024-09-14T06:30:40.436 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.002564846, 2024-09-14T06:30:40.436 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:30:40.436 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:30:40.436 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:30:40.436 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:30:40.436 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:40.436 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:30:40.436 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:57.677568+0000", 2024-09-14T06:30:40.436 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:40.436 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:40.437 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:40.437 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:30:40.437 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:57.677568+0000", 2024-09-14T06:30:40.437 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:40.437 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:40.437 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:40.437 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:30:40.437 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:57.677568+0000", 2024-09-14T06:30:40.437 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.8900000000000001e-07 2024-09-14T06:30:40.437 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:40.437 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:40.437 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:30:40.437 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:57.678488+0000", 2024-09-14T06:30:40.437 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00091963899999999998 2024-09-14T06:30:40.437 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:40.438 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:40.438 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:30:40.438 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:57.678500+0000", 2024-09-14T06:30:40.438 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.1902000000000001e-05 2024-09-14T06:30:40.438 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:40.438 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:40.438 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:30:40.438 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:57.678507+0000", 2024-09-14T06:30:40.438 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.7739999999999999e-06 2024-09-14T06:30:40.438 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:40.438 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:40.438 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:30:40.438 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:57.678586+0000", 2024-09-14T06:30:40.438 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.9103000000000005e-05 2024-09-14T06:30:40.439 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:40.439 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:40.439 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:30:40.439 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:57.678614+0000", 2024-09-14T06:30:40.439 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.7912e-05 2024-09-14T06:30:40.439 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:40.439 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:40.439 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-14T06:30:40.439 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:57.680075+0000", 2024-09-14T06:30:40.439 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0014613370000000001 2024-09-14T06:30:40.439 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:40.439 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:40.439 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:30:40.439 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:57.680133+0000", 2024-09-14T06:30:40.439 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.7890000000000003e-05 2024-09-14T06:30:40.440 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:40.440 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:40.440 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:40.440 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:40.440 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:40.440 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4397.0:78641 3.9 e77/23 3:914b72e4:::rbd_data.112ddfbec471.0000000000000030:head v 77'98, mlcod=73'95)", 2024-09-14T06:30:40.440 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:57.682289+0000", 2024-09-14T06:30:40.440 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 42.687090560999998, 2024-09-14T06:30:40.440 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0020171400000000002, 2024-09-14T06:30:40.440 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:30:40.440 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:30:40.440 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:30:40.440 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:30:40.440 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:40.441 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:30:40.441 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:57.682289+0000", 2024-09-14T06:30:40.441 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:40.441 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:40.441 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:40.441 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:30:40.441 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:57.682289+0000", 2024-09-14T06:30:40.441 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:40.441 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:40.441 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:40.441 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:30:40.441 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:57.682290+0000", 2024-09-14T06:30:40.441 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.0100000000000003e-07 2024-09-14T06:30:40.441 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:40.442 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:40.442 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:30:40.442 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:57.683230+0000", 2024-09-14T06:30:40.442 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00094047599999999999 2024-09-14T06:30:40.442 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:40.442 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:40.442 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:30:40.442 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:57.683243+0000", 2024-09-14T06:30:40.442 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.2926e-05 2024-09-14T06:30:40.442 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:40.442 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:40.442 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:30:40.442 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:57.683250+0000", 2024-09-14T06:30:40.442 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.6109999999999999e-06 2024-09-14T06:30:40.442 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:40.443 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:40.443 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:30:40.443 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:57.683313+0000", 2024-09-14T06:30:40.443 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.2943999999999999e-05 2024-09-14T06:30:40.443 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:40.443 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:40.443 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:30:40.443 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:57.683337+0000", 2024-09-14T06:30:40.443 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.4377999999999999e-05 2024-09-14T06:30:40.443 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:40.443 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:40.443 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-14T06:30:40.443 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:57.684264+0000", 2024-09-14T06:30:40.443 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00092733099999999999 2024-09-14T06:30:40.443 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:40.443 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:40.444 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:30:40.444 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:57.684306+0000", 2024-09-14T06:30:40.444 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.1573000000000003e-05 2024-09-14T06:30:40.444 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:40.444 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:40.444 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:40.444 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:40.444 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:40.444 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4397.0:113444 3.9 e96/23 3:9619bd5a:::rbd_data.112d4f226fa.0000000000000013:head v 96'128, mlcod=90'126)", 2024-09-14T06:30:40.444 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:30:17.730149+0000", 2024-09-14T06:30:40.444 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 22.639230449999999, 2024-09-14T06:30:40.444 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0019322790000000001, 2024-09-14T06:30:40.444 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:30:40.444 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:30:40.445 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:30:40.445 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:30:40.445 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:40.445 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:30:40.445 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.730149+0000", 2024-09-14T06:30:40.445 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:40.445 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:40.445 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:40.445 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:30:40.445 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.730149+0000", 2024-09-14T06:30:40.445 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:40.445 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:40.445 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:40.445 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:30:40.445 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.730149+0000", 2024-09-14T06:30:40.446 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.0499999999999999e-07 2024-09-14T06:30:40.446 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:40.446 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:40.446 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:30:40.446 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.730852+0000", 2024-09-14T06:30:40.446 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00070294699999999999 2024-09-14T06:30:40.446 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:40.446 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:40.446 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:30:40.446 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.730867+0000", 2024-09-14T06:30:40.446 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.5294999999999998e-05 2024-09-14T06:30:40.446 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:40.446 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:40.446 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:30:40.446 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.730877+0000", 2024-09-14T06:30:40.447 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.3519999999999996e-06 2024-09-14T06:30:40.447 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:40.447 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:40.447 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:30:40.447 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.730947+0000", 2024-09-14T06:30:40.447 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.0162999999999999e-05 2024-09-14T06:30:40.447 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:40.447 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:40.447 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:30:40.447 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.730972+0000", 2024-09-14T06:30:40.447 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.4589000000000001e-05 2024-09-14T06:30:40.447 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:40.447 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:40.447 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-14T06:30:40.447 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.732034+0000", 2024-09-14T06:30:40.448 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.001062508 2024-09-14T06:30:40.448 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:40.448 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:40.448 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:30:40.448 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.732081+0000", 2024-09-14T06:30:40.448 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.7120000000000003e-05 2024-09-14T06:30:40.448 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:40.448 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:40.448 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:40.448 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:40.448 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:40.448 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4397.0:124884 3.9 e104/23 3:93946509:::rbd_data.112d97471187.000000000000000d:head v 104'144, mlcod=98'142)", 2024-09-14T06:30:40.448 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:30:25.843253+0000", 2024-09-14T06:30:40.448 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 14.526126423999999, 2024-09-14T06:30:40.449 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.001767097, 2024-09-14T06:30:40.449 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:30:40.449 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:30:40.449 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:30:40.449 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:30:40.449 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:40.449 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:30:40.449 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:25.843253+0000", 2024-09-14T06:30:40.449 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:40.449 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:40.449 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:40.449 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:30:40.449 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:25.843253+0000", 2024-09-14T06:30:40.449 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:40.449 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:40.450 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:40.450 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:30:40.450 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:25.843253+0000", 2024-09-14T06:30:40.450 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.1400000000000001e-07 2024-09-14T06:30:40.450 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:40.450 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:40.450 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:30:40.450 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:25.843901+0000", 2024-09-14T06:30:40.450 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000647509 2024-09-14T06:30:40.450 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:40.450 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:40.450 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:30:40.450 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:25.843912+0000", 2024-09-14T06:30:40.450 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.0951999999999999e-05 2024-09-14T06:30:40.450 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:40.451 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:40.451 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:30:40.451 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:25.843918+0000", 2024-09-14T06:30:40.451 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.5390000000000004e-06 2024-09-14T06:30:40.451 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:40.451 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:40.451 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:30:40.451 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:25.843981+0000", 2024-09-14T06:30:40.451 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.2859000000000002e-05 2024-09-14T06:30:40.451 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:40.451 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:40.451 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:30:40.451 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:25.844006+0000", 2024-09-14T06:30:40.451 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.5363000000000001e-05 2024-09-14T06:30:40.451 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:40.451 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:40.452 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-14T06:30:40.452 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:25.844982+0000", 2024-09-14T06:30:40.452 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00097514300000000004 2024-09-14T06:30:40.452 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:40.452 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:40.452 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:30:40.452 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:25.845020+0000", 2024-09-14T06:30:40.452 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.8518e-05 2024-09-14T06:30:40.452 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:40.452 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:40.452 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:40.452 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:40.452 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:40.452 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4397.0:126010 3.9 e104/23 3:9120f7ff:::rbd_data.112d97471187.000000000000001b:head v 104'145, mlcod=99'143)", 2024-09-14T06:30:40.453 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:30:25.990254+0000", 2024-09-14T06:30:40.453 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 14.379125753, 2024-09-14T06:30:40.453 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0020629699999999999, 2024-09-14T06:30:40.453 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:30:40.453 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:30:40.453 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:30:40.453 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:30:40.453 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:40.453 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:30:40.453 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:25.990254+0000", 2024-09-14T06:30:40.453 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:40.453 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:40.453 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:40.453 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:30:40.454 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:25.990254+0000", 2024-09-14T06:30:40.454 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:40.454 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:40.454 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:40.454 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:30:40.454 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:25.990255+0000", 2024-09-14T06:30:40.454 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.3349999999999999e-06 2024-09-14T06:30:40.454 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:40.454 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:40.454 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:30:40.455 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:25.990966+0000", 2024-09-14T06:30:40.455 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00071085199999999999 2024-09-14T06:30:40.455 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:40.455 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:40.455 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:30:40.455 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:25.990986+0000", 2024-09-14T06:30:40.455 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.0054000000000001e-05 2024-09-14T06:30:40.455 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:40.455 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:40.455 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:30:40.455 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:25.990995+0000", 2024-09-14T06:30:40.455 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.166e-06 2024-09-14T06:30:40.455 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:40.455 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:40.456 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:30:40.456 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:25.991055+0000", 2024-09-14T06:30:40.456 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.0093000000000002e-05 2024-09-14T06:30:40.456 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:40.456 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:40.456 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:30:40.456 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:25.991082+0000", 2024-09-14T06:30:40.456 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.6505000000000001e-05 2024-09-14T06:30:40.456 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:40.456 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:40.456 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-14T06:30:40.456 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:25.992270+0000", 2024-09-14T06:30:40.456 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0011880479999999999 2024-09-14T06:30:40.456 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:40.457 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:40.457 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:30:40.457 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:25.992317+0000", 2024-09-14T06:30:40.457 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.6916999999999998e-05 2024-09-14T06:30:40.457 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:40.457 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:40.457 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:40.457 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:40.457 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:40.457 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-14T06:30:40.458 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-14T06:30:40.467 INFO:tasks.ceph.osd.7.smithi138.stderr:2024-09-14T06:30:40.466+0000 7f3c6a21d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33623) UID: 0 2024-09-14T06:30:40.561 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-14T06:30:40.561 INFO:teuthology.orchestra.run.smithi094.stdout: "ops": [], 2024-09-14T06:30:40.561 INFO:teuthology.orchestra.run.smithi094.stdout: "num_ops": 0 2024-09-14T06:30:40.561 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-14T06:30:40.567 INFO:tasks.ceph.osd.4.smithi138.stderr:2024-09-14T06:30:40.566+0000 7f6c653c8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33615) UID: 0 2024-09-14T06:30:40.569 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-14T06:30:40.609 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-14T06:30:40.609 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-14T06:30:40.609 INFO:tasks.thrashosds.thrasher:reset_purged_snaps_last 2024-09-14T06:30:40.609 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 reset_purged_snaps_last 2024-09-14T06:30:40.668 INFO:tasks.ceph.osd.6.smithi138.stderr:2024-09-14T06:30:40.667+0000 7f8cd8849640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33620) UID: 0 2024-09-14T06:30:40.680 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-14T06:30:40.680 INFO:teuthology.orchestra.run.smithi094.stdout: "ops": [], 2024-09-14T06:30:40.680 INFO:teuthology.orchestra.run.smithi094.stdout: "complaint_time": 30, 2024-09-14T06:30:40.680 INFO:teuthology.orchestra.run.smithi094.stdout: "num_blocked_ops": 0 2024-09-14T06:30:40.680 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-14T06:30:40.690 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-14T06:30:40.756 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 reset_purged_snaps_last 2024-09-14T06:30:40.769 INFO:tasks.ceph.osd.6.smithi138.stderr:2024-09-14T06:30:40.768+0000 7f8cd8849640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33620) UID: 0 2024-09-14T06:30:40.803 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-14T06:30:40.803 INFO:teuthology.orchestra.run.smithi094.stdout: "size": 20, 2024-09-14T06:30:40.803 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 600, 2024-09-14T06:30:40.803 INFO:teuthology.orchestra.run.smithi094.stdout: "ops": [ 2024-09-14T06:30:40.803 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:40.803 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:44425 3.2 3:4603ad7f:::rbd_data.112dca2f6ae1.0000000000000005:17 [read 868352~4096 [fadvise_nocache]] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:30:40.803 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:43.854967+0000", 2024-09-14T06:30:40.803 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 56.946420547000002, 2024-09-14T06:30:40.803 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.063146025999999994, 2024-09-14T06:30:40.803 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:30:40.804 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:30:40.804 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:30:40.804 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:30:40.804 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:30:40.804 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:40.804 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 44425 2024-09-14T06:30:40.804 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:40.804 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:30:40.804 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:40.804 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:30:40.804 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.854967+0000", 2024-09-14T06:30:40.804 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:40.804 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:40.804 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:40.804 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:30:40.805 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.854967+0000", 2024-09-14T06:30:40.805 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:40.805 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:40.805 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:40.805 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:30:40.805 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.854967+0000", 2024-09-14T06:30:40.805 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.16e-07 2024-09-14T06:30:40.805 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:40.805 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:40.805 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:30:40.805 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.854968+0000", 2024-09-14T06:30:40.805 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.006e-06 2024-09-14T06:30:40.805 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:40.805 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:40.806 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:30:40.806 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.854971+0000", 2024-09-14T06:30:40.806 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.9359999999999999e-06 2024-09-14T06:30:40.806 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:40.806 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:40.806 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:30:40.806 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.854974+0000", 2024-09-14T06:30:40.806 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.1840000000000002e-06 2024-09-14T06:30:40.806 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:40.806 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:40.806 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:30:40.806 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.917945+0000", 2024-09-14T06:30:40.806 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.062970925999999997 2024-09-14T06:30:40.807 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:40.807 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:40.807 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:30:40.807 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.918030+0000", 2024-09-14T06:30:40.807 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.5863000000000001e-05 2024-09-14T06:30:40.807 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:40.807 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:40.807 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:30:40.807 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.918113+0000", 2024-09-14T06:30:40.807 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.2894999999999998e-05 2024-09-14T06:30:40.807 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:40.807 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:40.808 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:40.808 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:40.808 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:40.808 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:44426 3.2 3:4603ad7f:::rbd_data.112dca2f6ae1.0000000000000005:17 [read 892928~4096 [fadvise_nocache]] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:30:40.808 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:43.854977+0000", 2024-09-14T06:30:40.808 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 56.946411136999998, 2024-09-14T06:30:40.808 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.063369237999999994, 2024-09-14T06:30:40.808 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:30:40.808 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:30:40.808 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:30:40.808 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:30:40.808 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:30:40.809 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:40.809 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 44426 2024-09-14T06:30:40.809 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:40.809 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:30:40.809 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:40.809 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:30:40.809 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.854977+0000", 2024-09-14T06:30:40.809 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:40.809 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:40.809 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:40.809 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:30:40.809 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.854977+0000", 2024-09-14T06:30:40.809 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:40.809 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:40.810 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:40.810 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:30:40.810 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.854977+0000", 2024-09-14T06:30:40.810 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.1199999999999999e-07 2024-09-14T06:30:40.810 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:40.810 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:40.810 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:30:40.810 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.854980+0000", 2024-09-14T06:30:40.810 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.27e-06 2024-09-14T06:30:40.810 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:40.810 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:40.810 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:30:40.810 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.854983+0000", 2024-09-14T06:30:40.810 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.1650000000000002e-06 2024-09-14T06:30:40.810 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:40.811 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:40.811 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:30:40.811 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.854989+0000", 2024-09-14T06:30:40.811 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.6520000000000003e-06 2024-09-14T06:30:40.811 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:40.811 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:40.811 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:30:40.811 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.918143+0000", 2024-09-14T06:30:40.811 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.063154541999999994 2024-09-14T06:30:40.811 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:40.811 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:40.811 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:30:40.811 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.918209+0000", 2024-09-14T06:30:40.811 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.5529000000000002e-05 2024-09-14T06:30:40.811 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:40.811 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:40.812 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:30:40.812 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.918346+0000", 2024-09-14T06:30:40.812 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000136868 2024-09-14T06:30:40.812 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:40.812 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:40.812 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:40.812 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:40.812 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:40.812 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:44427 3.2 3:4603ad7f:::rbd_data.112dca2f6ae1.0000000000000005:17 [read 917504~4096 [fadvise_nocache]] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:30:40.812 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:43.854992+0000", 2024-09-14T06:30:40.812 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 56.946395490999997, 2024-09-14T06:30:40.812 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.063607792999999996, 2024-09-14T06:30:40.812 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:30:40.812 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:30:40.812 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:30:40.813 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:30:40.813 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:30:40.813 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:40.813 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 44427 2024-09-14T06:30:40.813 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:40.813 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:30:40.813 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:40.813 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:30:40.813 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.854992+0000", 2024-09-14T06:30:40.813 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:40.813 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:40.813 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:40.813 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:30:40.813 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.854992+0000", 2024-09-14T06:30:40.813 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:40.814 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:40.814 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:40.814 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:30:40.814 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.854993+0000", 2024-09-14T06:30:40.814 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.1400000000000001e-07 2024-09-14T06:30:40.814 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:40.814 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:40.814 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:30:40.814 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.854994+0000", 2024-09-14T06:30:40.814 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.091e-06 2024-09-14T06:30:40.814 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:40.814 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:40.814 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:30:40.814 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855001+0000", 2024-09-14T06:30:40.815 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.204e-06 2024-09-14T06:30:40.815 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:40.815 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:40.815 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:30:40.815 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855003+0000", 2024-09-14T06:30:40.815 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.514e-06 2024-09-14T06:30:40.815 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:40.815 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:40.815 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:30:40.815 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.918380+0000", 2024-09-14T06:30:40.815 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.063376738000000002 2024-09-14T06:30:40.815 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:40.815 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:40.815 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:30:40.815 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.918453+0000", 2024-09-14T06:30:40.815 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.3291e-05 2024-09-14T06:30:40.816 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:40.816 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:40.816 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:30:40.816 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.918600+0000", 2024-09-14T06:30:40.816 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00014674100000000001 2024-09-14T06:30:40.816 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:40.816 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:40.816 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:40.816 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:40.816 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:40.816 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:44428 3.2 3:4603ad7f:::rbd_data.112dca2f6ae1.0000000000000005:17 [read 942080~4096 [fadvise_nocache]] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:30:40.816 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:43.855006+0000", 2024-09-14T06:30:40.816 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 56.946381441, 2024-09-14T06:30:40.817 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.063846321999999997, 2024-09-14T06:30:40.817 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:30:40.817 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:30:40.817 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:30:40.817 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:30:40.817 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:30:40.817 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:40.817 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 44428 2024-09-14T06:30:40.817 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:40.817 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:30:40.817 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:40.817 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:30:40.817 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855006+0000", 2024-09-14T06:30:40.817 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:40.818 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:40.818 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:40.818 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:30:40.818 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855006+0000", 2024-09-14T06:30:40.818 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:40.818 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:40.818 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:40.818 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:30:40.818 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855007+0000", 2024-09-14T06:30:40.818 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.1199999999999999e-07 2024-09-14T06:30:40.818 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:40.818 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:40.818 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:30:40.818 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855011+0000", 2024-09-14T06:30:40.818 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.9539999999999998e-06 2024-09-14T06:30:40.819 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:40.819 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:40.819 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:30:40.819 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855014+0000", 2024-09-14T06:30:40.819 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.2930000000000001e-06 2024-09-14T06:30:40.819 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:40.819 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:40.819 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:30:40.819 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855016+0000", 2024-09-14T06:30:40.819 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.3039999999999999e-06 2024-09-14T06:30:40.819 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:40.819 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:40.819 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:30:40.819 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.918634+0000", 2024-09-14T06:30:40.820 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.063617928000000004 2024-09-14T06:30:40.820 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:40.820 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:40.820 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:30:40.820 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.918711+0000", 2024-09-14T06:30:40.820 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.6545000000000007e-05 2024-09-14T06:30:40.820 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:40.820 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:40.820 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:30:40.820 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.918853+0000", 2024-09-14T06:30:40.820 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00014208600000000001 2024-09-14T06:30:40.820 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:40.820 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:40.820 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:40.820 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:40.821 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:40.821 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:44429 3.2 3:4603ad7f:::rbd_data.112dca2f6ae1.0000000000000005:17 [read 966656~4096 [fadvise_nocache]] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:30:40.821 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:43.855019+0000", 2024-09-14T06:30:40.821 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 56.946368767999999, 2024-09-14T06:30:40.821 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.064059226999999996, 2024-09-14T06:30:40.821 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:30:40.821 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:30:40.821 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:30:40.821 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:30:40.821 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:30:40.821 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:40.821 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 44429 2024-09-14T06:30:40.821 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:40.821 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:30:40.821 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:40.822 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:30:40.822 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855019+0000", 2024-09-14T06:30:40.822 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:40.822 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:40.822 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:40.822 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:30:40.822 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855019+0000", 2024-09-14T06:30:40.822 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:40.822 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:40.822 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:40.822 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:30:40.822 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855019+0000", 2024-09-14T06:30:40.822 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.28e-07 2024-09-14T06:30:40.822 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:40.823 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:40.823 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:30:40.823 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855020+0000", 2024-09-14T06:30:40.823 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.0330000000000001e-06 2024-09-14T06:30:40.823 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:40.823 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:40.823 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:30:40.823 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855023+0000", 2024-09-14T06:30:40.823 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.79e-06 2024-09-14T06:30:40.823 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:40.823 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:40.823 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:30:40.823 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855025+0000", 2024-09-14T06:30:40.823 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.216e-06 2024-09-14T06:30:40.824 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:40.824 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:40.824 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:30:40.824 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.918886+0000", 2024-09-14T06:30:40.824 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.063861037999999995 2024-09-14T06:30:40.824 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:40.824 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:40.824 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:30:40.824 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.918952+0000", 2024-09-14T06:30:40.824 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.5993000000000003e-05 2024-09-14T06:30:40.824 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:40.824 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:40.824 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:30:40.824 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.919078+0000", 2024-09-14T06:30:40.825 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00012592899999999999 2024-09-14T06:30:40.825 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:40.825 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:40.825 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:40.825 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:40.825 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:40.825 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:44430 3.2 3:4603ad7f:::rbd_data.112dca2f6ae1.0000000000000005:17 [read 991232~4096 [fadvise_nocache]] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:30:40.825 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:43.855028+0000", 2024-09-14T06:30:40.825 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 56.946359758, 2024-09-14T06:30:40.825 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.064284346000000006, 2024-09-14T06:30:40.825 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:30:40.825 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:30:40.825 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:30:40.825 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:30:40.825 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:30:40.825 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:40.826 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 44430 2024-09-14T06:30:40.826 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:40.826 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:30:40.826 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:40.826 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:30:40.826 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855028+0000", 2024-09-14T06:30:40.826 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:40.826 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:40.826 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:40.826 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:30:40.826 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855028+0000", 2024-09-14T06:30:40.826 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:40.826 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:40.826 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:40.826 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:30:40.827 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855028+0000", 2024-09-14T06:30:40.827 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.1199999999999999e-07 2024-09-14T06:30:40.827 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:40.827 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:40.827 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:30:40.827 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855029+0000", 2024-09-14T06:30:40.827 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.1100000000000004e-07 2024-09-14T06:30:40.827 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:40.827 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:40.827 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:30:40.827 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855032+0000", 2024-09-14T06:30:40.827 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.7889999999999999e-06 2024-09-14T06:30:40.827 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:40.828 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:40.828 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:30:40.828 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855034+0000", 2024-09-14T06:30:40.828 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.125e-06 2024-09-14T06:30:40.828 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:40.828 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:40.828 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:30:40.828 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.919110+0000", 2024-09-14T06:30:40.828 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.064076413999999998 2024-09-14T06:30:40.828 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:40.828 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:40.828 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:30:40.828 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.919186+0000", 2024-09-14T06:30:40.828 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.5784999999999994e-05 2024-09-14T06:30:40.829 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:40.829 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:40.829 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:30:40.829 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.919312+0000", 2024-09-14T06:30:40.829 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00012611 2024-09-14T06:30:40.829 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:40.829 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:40.829 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:40.829 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:40.829 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:40.829 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:44431 3.2 3:4603ad7f:::rbd_data.112dca2f6ae1.0000000000000005:17 [read 1015808~4096 [fadvise_nocache]] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:30:40.829 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:43.855037+0000", 2024-09-14T06:30:40.829 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 56.946350889999998, 2024-09-14T06:30:40.829 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.064469755000000004, 2024-09-14T06:30:40.830 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:30:40.830 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:30:40.830 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:30:40.830 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:30:40.830 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:30:40.830 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:40.830 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 44431 2024-09-14T06:30:40.830 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:40.830 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:30:40.830 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:40.830 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:30:40.830 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855037+0000", 2024-09-14T06:30:40.830 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:40.830 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:40.830 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:40.830 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:30:40.831 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855037+0000", 2024-09-14T06:30:40.831 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:40.831 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:40.831 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:40.831 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:30:40.831 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855037+0000", 2024-09-14T06:30:40.831 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.1e-07 2024-09-14T06:30:40.831 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:40.831 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:40.831 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:30:40.831 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855038+0000", 2024-09-14T06:30:40.831 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.9400000000000004e-07 2024-09-14T06:30:40.831 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:40.831 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:40.831 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:30:40.832 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855041+0000", 2024-09-14T06:30:40.832 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.9639999999999999e-06 2024-09-14T06:30:40.832 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:40.832 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:40.832 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:30:40.832 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855043+0000", 2024-09-14T06:30:40.832 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.2259999999999999e-06 2024-09-14T06:30:40.832 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:40.832 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:40.832 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:30:40.832 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.919347+0000", 2024-09-14T06:30:40.832 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.064304091999999993 2024-09-14T06:30:40.832 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:40.832 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:40.832 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:30:40.833 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.919393+0000", 2024-09-14T06:30:40.833 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.5340999999999998e-05 2024-09-14T06:30:40.833 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:40.833 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:40.833 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:30:40.833 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.919507+0000", 2024-09-14T06:30:40.833 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000114028 2024-09-14T06:30:40.833 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:40.833 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:40.833 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:40.833 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:40.833 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:40.833 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:44432 3.2 3:4603ad7f:::rbd_data.112dca2f6ae1.0000000000000005:17 [read 1040384~4096 [fadvise_nocache]] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:30:40.833 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:43.855046+0000", 2024-09-14T06:30:40.834 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 56.946341822000001, 2024-09-14T06:30:40.834 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.064639253999999993, 2024-09-14T06:30:40.834 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:30:40.834 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:30:40.834 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:30:40.834 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:30:40.834 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:30:40.834 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:40.834 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 44432 2024-09-14T06:30:40.834 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:40.834 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:30:40.834 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:40.834 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:30:40.834 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855046+0000", 2024-09-14T06:30:40.835 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:40.835 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:40.835 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:40.835 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:30:40.835 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855046+0000", 2024-09-14T06:30:40.835 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:40.835 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:40.835 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:40.835 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:30:40.835 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855046+0000", 2024-09-14T06:30:40.835 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.05e-07 2024-09-14T06:30:40.835 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:40.835 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:40.835 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:30:40.836 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855047+0000", 2024-09-14T06:30:40.836 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.879999999999999e-07 2024-09-14T06:30:40.836 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:40.836 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:40.836 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:30:40.836 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855050+0000", 2024-09-14T06:30:40.836 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.8459999999999999e-06 2024-09-14T06:30:40.836 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:40.836 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:40.836 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:30:40.836 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855052+0000", 2024-09-14T06:30:40.836 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.136e-06 2024-09-14T06:30:40.836 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:40.836 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:40.837 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:30:40.837 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.919536+0000", 2024-09-14T06:30:40.837 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.064484132 2024-09-14T06:30:40.837 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:40.837 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:40.837 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:30:40.837 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.919611+0000", 2024-09-14T06:30:40.837 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.4407999999999996e-05 2024-09-14T06:30:40.837 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:40.837 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:40.838 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:30:40.838 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.919685+0000", 2024-09-14T06:30:40.838 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.4538999999999999e-05 2024-09-14T06:30:40.838 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:40.838 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:40.838 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:40.838 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:40.838 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:40.838 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:44518 3.2 3:4190d882:::rbd_data.112dca2f6ae1.0000000000000002:17 [read 12288~4096 [fadvise_nocache]] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:30:40.838 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:43.855055+0000", 2024-09-14T06:30:40.838 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 56.946332937999998, 2024-09-14T06:30:40.838 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.064894317000000007, 2024-09-14T06:30:40.838 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:30:40.839 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:30:40.839 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:30:40.839 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:30:40.839 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:30:40.839 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:40.839 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 44518 2024-09-14T06:30:40.839 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:40.839 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:30:40.839 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:40.839 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:30:40.839 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855055+0000", 2024-09-14T06:30:40.839 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:40.839 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:40.839 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:40.839 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:30:40.840 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855055+0000", 2024-09-14T06:30:40.840 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:40.840 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:40.840 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:40.840 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:30:40.840 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855055+0000", 2024-09-14T06:30:40.840 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.0699999999999999e-07 2024-09-14T06:30:40.840 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:40.840 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:40.840 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:30:40.840 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855056+0000", 2024-09-14T06:30:40.840 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.9400000000000004e-07 2024-09-14T06:30:40.840 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:40.840 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:40.841 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:30:40.841 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855059+0000", 2024-09-14T06:30:40.841 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.8870000000000001e-06 2024-09-14T06:30:40.841 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:40.841 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:40.841 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:30:40.841 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855061+0000", 2024-09-14T06:30:40.841 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.1919999999999999e-06 2024-09-14T06:30:40.841 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:40.841 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:40.841 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:30:40.841 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.919719+0000", 2024-09-14T06:30:40.841 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.064658013 2024-09-14T06:30:40.841 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:40.842 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:40.842 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:30:40.842 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.919788+0000", 2024-09-14T06:30:40.842 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.9282000000000005e-05 2024-09-14T06:30:40.842 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:40.842 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:40.842 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:30:40.842 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.919949+0000", 2024-09-14T06:30:40.842 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00016084199999999999 2024-09-14T06:30:40.842 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:40.842 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:40.842 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:40.842 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:40.842 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:40.843 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:44519 3.2 3:4190d882:::rbd_data.112dca2f6ae1.0000000000000002:17 [read 36864~4096 [fadvise_nocache]] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:30:40.843 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:43.855064+0000", 2024-09-14T06:30:40.843 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 56.946323960999997, 2024-09-14T06:30:40.843 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.065081865000000003, 2024-09-14T06:30:40.843 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:30:40.843 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:30:40.843 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:30:40.843 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:30:40.843 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:30:40.843 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:40.843 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 44519 2024-09-14T06:30:40.843 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:40.843 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:30:40.843 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:40.843 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:30:40.844 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855064+0000", 2024-09-14T06:30:40.844 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:40.844 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:40.844 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:40.844 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:30:40.844 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855064+0000", 2024-09-14T06:30:40.844 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:40.844 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:40.844 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:40.844 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:30:40.844 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855064+0000", 2024-09-14T06:30:40.844 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.1e-07 2024-09-14T06:30:40.844 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:40.844 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:40.845 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:30:40.845 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855065+0000", 2024-09-14T06:30:40.845 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.035e-06 2024-09-14T06:30:40.845 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:40.845 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:40.845 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:30:40.845 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855068+0000", 2024-09-14T06:30:40.845 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.8339999999999999e-06 2024-09-14T06:30:40.845 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:40.845 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:40.845 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:30:40.845 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855071+0000", 2024-09-14T06:30:40.845 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.3450000000000002e-06 2024-09-14T06:30:40.845 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:40.846 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:40.846 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:30:40.846 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.919979+0000", 2024-09-14T06:30:40.846 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.064907647999999998 2024-09-14T06:30:40.846 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:40.846 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:40.846 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:30:40.846 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.920026+0000", 2024-09-14T06:30:40.846 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.7595000000000001e-05 2024-09-14T06:30:40.846 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:40.846 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:40.846 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:30:40.846 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.920146+0000", 2024-09-14T06:30:40.846 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00011919800000000001 2024-09-14T06:30:40.846 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:40.846 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:40.847 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:40.847 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:40.847 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:40.847 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:44520 3.2 3:4190d882:::rbd_data.112dca2f6ae1.0000000000000002:17 [read 61440~4096 [fadvise_nocache]] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:30:40.847 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:43.855075+0000", 2024-09-14T06:30:40.847 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 56.946312910000003, 2024-09-14T06:30:40.847 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.065335078000000005, 2024-09-14T06:30:40.847 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:30:40.847 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:30:40.847 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:30:40.847 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:30:40.847 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:30:40.847 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:40.847 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 44520 2024-09-14T06:30:40.848 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:40.848 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:30:40.848 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:40.848 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:30:40.848 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855075+0000", 2024-09-14T06:30:40.848 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:40.848 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:40.848 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:40.848 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:30:40.848 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855075+0000", 2024-09-14T06:30:40.848 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:40.848 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:40.848 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:40.848 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:30:40.849 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855075+0000", 2024-09-14T06:30:40.849 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.1e-07 2024-09-14T06:30:40.849 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:40.849 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:40.849 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:30:40.849 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855076+0000", 2024-09-14T06:30:40.849 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.0839999999999999e-06 2024-09-14T06:30:40.849 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:40.849 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:40.849 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:30:40.849 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855079+0000", 2024-09-14T06:30:40.849 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.9359999999999999e-06 2024-09-14T06:30:40.849 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:40.849 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:40.849 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:30:40.850 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855084+0000", 2024-09-14T06:30:40.850 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.1730000000000003e-06 2024-09-14T06:30:40.850 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:40.850 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:40.850 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:30:40.850 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.920175+0000", 2024-09-14T06:30:40.850 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.065090234999999996 2024-09-14T06:30:40.850 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:40.850 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:40.850 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:30:40.850 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.920250+0000", 2024-09-14T06:30:40.850 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.5807000000000001e-05 2024-09-14T06:30:40.850 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:40.850 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:40.850 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:30:40.851 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.920410+0000", 2024-09-14T06:30:40.851 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00015963300000000001 2024-09-14T06:30:40.851 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:40.851 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:40.851 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:40.851 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:40.851 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:40.851 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:44521 3.2 3:4190d882:::rbd_data.112dca2f6ae1.0000000000000002:17 [read 86016~4096 [fadvise_nocache]] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:30:40.851 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:43.855087+0000", 2024-09-14T06:30:40.851 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 56.946300428999997, 2024-09-14T06:30:40.851 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.065563264999999996, 2024-09-14T06:30:40.851 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:30:40.851 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:30:40.851 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:30:40.851 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:30:40.851 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:30:40.852 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:40.852 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 44521 2024-09-14T06:30:40.852 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:40.852 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:30:40.852 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:40.852 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:30:40.852 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855087+0000", 2024-09-14T06:30:40.852 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:40.852 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:40.852 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:40.852 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:30:40.852 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855087+0000", 2024-09-14T06:30:40.852 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:40.852 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:40.853 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:40.853 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:30:40.853 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855088+0000", 2024-09-14T06:30:40.853 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.3699999999999999e-07 2024-09-14T06:30:40.853 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:40.853 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:40.853 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:30:40.853 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855089+0000", 2024-09-14T06:30:40.853 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.9900000000000009e-07 2024-09-14T06:30:40.853 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:40.853 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:40.853 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:30:40.853 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855091+0000", 2024-09-14T06:30:40.853 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.8940000000000002e-06 2024-09-14T06:30:40.853 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:40.854 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:40.854 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:30:40.854 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855094+0000", 2024-09-14T06:30:40.854 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.351e-06 2024-09-14T06:30:40.854 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:40.854 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:40.854 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:30:40.854 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.920437+0000", 2024-09-14T06:30:40.854 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.065343221000000007 2024-09-14T06:30:40.854 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:40.854 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:40.854 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:30:40.854 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.920481+0000", 2024-09-14T06:30:40.854 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.4431000000000002e-05 2024-09-14T06:30:40.855 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:40.855 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:40.855 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:30:40.855 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.920651+0000", 2024-09-14T06:30:40.855 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00016913200000000001 2024-09-14T06:30:40.855 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:40.855 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:40.855 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:40.855 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:40.855 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:40.855 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:44522 3.2 3:4190d882:::rbd_data.112dca2f6ae1.0000000000000002:17 [read 110592~4096 [fadvise_nocache]] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:30:40.855 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:43.855098+0000", 2024-09-14T06:30:40.855 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 56.946290202, 2024-09-14T06:30:40.855 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.065827628999999999, 2024-09-14T06:30:40.856 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:30:40.856 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:30:40.856 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:30:40.856 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:30:40.856 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:30:40.856 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:40.856 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 44522 2024-09-14T06:30:40.856 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:40.856 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:30:40.856 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:40.856 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:30:40.856 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855098+0000", 2024-09-14T06:30:40.856 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:40.857 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:40.857 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:40.857 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:30:40.857 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855098+0000", 2024-09-14T06:30:40.857 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:40.857 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:40.857 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:40.857 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:30:40.857 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855098+0000", 2024-09-14T06:30:40.857 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.1e-07 2024-09-14T06:30:40.857 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:40.857 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:40.857 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:30:40.857 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855099+0000", 2024-09-14T06:30:40.858 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.012e-06 2024-09-14T06:30:40.858 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:40.858 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:40.858 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:30:40.858 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855102+0000", 2024-09-14T06:30:40.858 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.9380000000000001e-06 2024-09-14T06:30:40.858 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:40.858 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:40.858 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:30:40.858 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855108+0000", 2024-09-14T06:30:40.858 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.4339999999999997e-06 2024-09-14T06:30:40.858 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:40.858 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:40.858 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:30:40.859 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.920681+0000", 2024-09-14T06:30:40.859 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.065572943999999994 2024-09-14T06:30:40.859 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:40.859 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:40.859 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:30:40.859 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.920724+0000", 2024-09-14T06:30:40.859 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.2877000000000003e-05 2024-09-14T06:30:40.859 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:40.859 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:40.859 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:30:40.859 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.920925+0000", 2024-09-14T06:30:40.859 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00020121399999999999 2024-09-14T06:30:40.859 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:40.859 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:40.860 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:40.860 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:40.860 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:40.860 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:44523 3.2 3:4190d882:::rbd_data.112dca2f6ae1.0000000000000002:17 [read 135168~4096 [fadvise_nocache]] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:30:40.860 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:43.855111+0000", 2024-09-14T06:30:40.860 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 56.946276480000002, 2024-09-14T06:30:40.860 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.066028198999999996, 2024-09-14T06:30:40.860 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:30:40.860 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:30:40.860 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:30:40.860 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:30:40.860 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:30:40.860 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:40.860 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 44523 2024-09-14T06:30:40.861 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:40.861 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:30:40.861 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:40.861 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:30:40.861 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855111+0000", 2024-09-14T06:30:40.861 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:40.861 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:40.861 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:40.861 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:30:40.861 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855111+0000", 2024-09-14T06:30:40.861 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:40.861 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:40.861 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:40.861 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:30:40.861 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855112+0000", 2024-09-14T06:30:40.862 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.1199999999999999e-07 2024-09-14T06:30:40.862 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:40.862 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:40.862 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:30:40.862 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855115+0000", 2024-09-14T06:30:40.862 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.3380000000000001e-06 2024-09-14T06:30:40.862 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:40.862 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:40.862 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:30:40.862 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855118+0000", 2024-09-14T06:30:40.862 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.366e-06 2024-09-14T06:30:40.862 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:40.862 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:40.862 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:30:40.863 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855121+0000", 2024-09-14T06:30:40.863 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.8329999999999998e-06 2024-09-14T06:30:40.863 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:40.863 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:40.863 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:30:40.863 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.920956+0000", 2024-09-14T06:30:40.863 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.065835251999999997 2024-09-14T06:30:40.863 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:40.863 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:40.863 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:30:40.863 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.921001+0000", 2024-09-14T06:30:40.863 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.4778999999999999e-05 2024-09-14T06:30:40.863 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:40.863 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:40.863 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:30:40.864 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.921139+0000", 2024-09-14T06:30:40.864 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000138419 2024-09-14T06:30:40.864 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:40.864 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:40.864 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:40.864 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:40.864 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:40.864 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:44524 3.2 3:4190d882:::rbd_data.112dca2f6ae1.0000000000000002:17 [read 159744~4096 [fadvise_nocache]] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:30:40.864 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:43.855124+0000", 2024-09-14T06:30:40.864 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 56.946263500999997, 2024-09-14T06:30:40.864 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.066249261000000004, 2024-09-14T06:30:40.864 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:30:40.864 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:30:40.864 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:30:40.865 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:30:40.865 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:30:40.865 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:40.865 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 44524 2024-09-14T06:30:40.865 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:40.865 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:30:40.865 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:40.865 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:30:40.865 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855124+0000", 2024-09-14T06:30:40.865 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:40.865 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:40.865 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:40.865 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:30:40.865 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855124+0000", 2024-09-14T06:30:40.866 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:40.866 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:40.866 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:40.866 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:30:40.866 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861086+0000", 2024-09-14T06:30:40.866 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0059613829999999998 2024-09-14T06:30:40.866 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:40.866 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:40.866 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:30:40.866 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861100+0000", 2024-09-14T06:30:40.866 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.4644000000000001e-05 2024-09-14T06:30:40.866 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:40.866 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:40.866 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:30:40.866 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861109+0000", 2024-09-14T06:30:40.867 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.541e-06 2024-09-14T06:30:40.867 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:40.867 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:40.867 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:30:40.867 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861117+0000", 2024-09-14T06:30:40.867 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.1049999999999995e-06 2024-09-14T06:30:40.867 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:40.867 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:40.867 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:30:40.867 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.921169+0000", 2024-09-14T06:30:40.867 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.060052405000000003 2024-09-14T06:30:40.867 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:40.867 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:40.867 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:30:40.867 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.921213+0000", 2024-09-14T06:30:40.868 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.3565e-05 2024-09-14T06:30:40.868 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:40.868 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:40.868 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:30:40.868 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.921374+0000", 2024-09-14T06:30:40.868 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00016061800000000001 2024-09-14T06:30:40.868 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:40.868 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:40.868 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:40.868 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:40.868 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:40.868 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:44536 3.2 3:4190d882:::rbd_data.112dca2f6ae1.0000000000000002:17 [read 454656~4096 [fadvise_nocache]] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:30:40.868 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:43.861384+0000", 2024-09-14T06:30:40.868 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 56.940003943999997, 2024-09-14T06:30:40.869 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.063058528000000003, 2024-09-14T06:30:40.869 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:30:40.869 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:30:40.869 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:30:40.869 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:30:40.869 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:30:40.869 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:40.869 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 44536 2024-09-14T06:30:40.869 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:40.869 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:30:40.869 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:40.869 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:30:40.869 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861384+0000", 2024-09-14T06:30:40.869 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:40.869 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:40.870 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:40.870 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:30:40.870 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861384+0000", 2024-09-14T06:30:40.870 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:40.870 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:40.870 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:40.870 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:30:40.870 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861384+0000", 2024-09-14T06:30:40.870 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.4000000000000002e-07 2024-09-14T06:30:40.870 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:40.870 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:40.870 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:30:40.870 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861386+0000", 2024-09-14T06:30:40.870 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.891e-06 2024-09-14T06:30:40.871 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:40.871 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:40.871 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:30:40.871 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861392+0000", 2024-09-14T06:30:40.871 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.9939999999999997e-06 2024-09-14T06:30:40.871 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:40.871 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:40.871 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:30:40.871 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861398+0000", 2024-09-14T06:30:40.871 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.507e-06 2024-09-14T06:30:40.871 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:40.871 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:40.871 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:30:40.871 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.924149+0000", 2024-09-14T06:30:40.871 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.062751135999999999 2024-09-14T06:30:40.872 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:40.872 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:40.872 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:30:40.872 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.924226+0000", 2024-09-14T06:30:40.872 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.7517999999999999e-05 2024-09-14T06:30:40.872 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:40.872 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:40.872 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:30:40.872 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.924442+0000", 2024-09-14T06:30:40.872 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000216042 2024-09-14T06:30:40.872 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:40.872 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:40.872 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:40.872 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:40.873 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:40.873 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:44537 3.2 3:4190d882:::rbd_data.112dca2f6ae1.0000000000000002:17 [read 479232~4096 [fadvise_nocache]] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:30:40.873 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:43.861405+0000", 2024-09-14T06:30:40.873 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 56.939982897999997, 2024-09-14T06:30:40.873 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.063351795000000002, 2024-09-14T06:30:40.873 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:30:40.873 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:30:40.873 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:30:40.873 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:30:40.873 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:30:40.873 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:40.873 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 44537 2024-09-14T06:30:40.874 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:40.874 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:30:40.874 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:40.874 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:30:40.874 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861405+0000", 2024-09-14T06:30:40.874 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:40.874 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:40.874 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:40.874 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:30:40.874 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861405+0000", 2024-09-14T06:30:40.875 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:40.875 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:40.875 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:40.875 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:30:40.875 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861405+0000", 2024-09-14T06:30:40.875 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.9099999999999999e-07 2024-09-14T06:30:40.875 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:40.875 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:40.875 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:30:40.875 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861407+0000", 2024-09-14T06:30:40.875 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.075e-06 2024-09-14T06:30:40.875 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:40.875 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:40.875 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:30:40.876 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861414+0000", 2024-09-14T06:30:40.876 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.4659999999999996e-06 2024-09-14T06:30:40.876 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:40.876 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:40.876 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:30:40.876 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861418+0000", 2024-09-14T06:30:40.876 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.5519999999999998e-06 2024-09-14T06:30:40.876 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:40.876 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:40.876 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:30:40.876 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.924474+0000", 2024-09-14T06:30:40.876 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.063056012999999994 2024-09-14T06:30:40.876 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:40.876 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:40.877 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:30:40.877 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.924556+0000", 2024-09-14T06:30:40.877 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.1181999999999996e-05 2024-09-14T06:30:40.877 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:40.877 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:40.877 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:30:40.877 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.924757+0000", 2024-09-14T06:30:40.877 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00020111599999999999 2024-09-14T06:30:40.877 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:40.877 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:40.878 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:40.878 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:40.878 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:40.878 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:44538 3.2 3:4190d882:::rbd_data.112dca2f6ae1.0000000000000002:17 [read 503808~4096 [fadvise_nocache]] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:30:40.878 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:43.861425+0000", 2024-09-14T06:30:40.878 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 56.939962856999998, 2024-09-14T06:30:40.878 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.063627125000000007, 2024-09-14T06:30:40.878 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:30:40.878 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:30:40.878 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:30:40.878 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:30:40.878 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:30:40.878 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:40.878 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 44538 2024-09-14T06:30:40.878 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:40.879 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:30:40.879 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:40.879 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:30:40.879 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861425+0000", 2024-09-14T06:30:40.879 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:40.879 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:40.879 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:40.879 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:30:40.879 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861425+0000", 2024-09-14T06:30:40.879 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:40.879 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:40.879 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:40.879 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:30:40.879 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861425+0000", 2024-09-14T06:30:40.880 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.9999999999999998e-07 2024-09-14T06:30:40.880 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:40.880 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:40.880 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:30:40.880 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861427+0000", 2024-09-14T06:30:40.880 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.9829999999999999e-06 2024-09-14T06:30:40.880 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:40.880 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:40.880 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:30:40.880 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861433+0000", 2024-09-14T06:30:40.880 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.2889999999999997e-06 2024-09-14T06:30:40.880 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:40.880 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:40.880 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:30:40.880 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861454+0000", 2024-09-14T06:30:40.881 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.1844999999999998e-05 2024-09-14T06:30:40.881 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:40.881 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:40.881 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:30:40.881 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.924786+0000", 2024-09-14T06:30:40.881 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.063331497 2024-09-14T06:30:40.881 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:40.881 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:40.881 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:30:40.881 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.924864+0000", 2024-09-14T06:30:40.883 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.7744000000000006e-05 2024-09-14T06:30:40.884 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:40.884 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:40.884 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:30:40.884 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.925052+0000", 2024-09-14T06:30:40.884 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00018836699999999999 2024-09-14T06:30:40.884 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:40.884 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:40.884 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:40.884 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:40.884 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:40.884 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:44539 3.2 3:4190d882:::rbd_data.112dca2f6ae1.0000000000000002:17 [read 528384~4096 [fadvise_nocache]] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:30:40.884 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:43.861462+0000", 2024-09-14T06:30:40.884 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 56.939925529999996, 2024-09-14T06:30:40.885 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.063855705999999998, 2024-09-14T06:30:40.885 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:30:40.885 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:30:40.885 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:30:40.885 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:30:40.885 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:30:40.885 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:40.885 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 44539 2024-09-14T06:30:40.885 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:40.885 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:30:40.885 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:40.886 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:30:40.886 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861462+0000", 2024-09-14T06:30:40.886 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:40.886 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:40.886 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:40.886 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:30:40.886 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861462+0000", 2024-09-14T06:30:40.886 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:40.886 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:40.886 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:40.886 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:30:40.886 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861463+0000", 2024-09-14T06:30:40.886 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.5200000000000002e-07 2024-09-14T06:30:40.887 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:40.887 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:40.887 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:30:40.887 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861465+0000", 2024-09-14T06:30:40.887 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.5189999999999999e-06 2024-09-14T06:30:40.887 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:40.887 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:40.887 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:30:40.887 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861471+0000", 2024-09-14T06:30:40.887 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.6899999999999997e-06 2024-09-14T06:30:40.887 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:40.887 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:40.887 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:30:40.887 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861475+0000", 2024-09-14T06:30:40.888 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.5730000000000001e-06 2024-09-14T06:30:40.888 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:40.888 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:40.888 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:30:40.888 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.925085+0000", 2024-09-14T06:30:40.888 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.063609902999999995 2024-09-14T06:30:40.888 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:40.888 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:40.888 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:30:40.888 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.925145+0000", 2024-09-14T06:30:40.888 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.9383999999999999e-05 2024-09-14T06:30:40.888 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:40.888 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:40.888 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:30:40.889 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.925318+0000", 2024-09-14T06:30:40.889 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00017318499999999999 2024-09-14T06:30:40.889 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:40.889 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:40.889 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:40.889 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:40.889 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:40.889 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:44540 3.2 3:4190d882:::rbd_data.112dca2f6ae1.0000000000000002:17 [read 552960~4096 [fadvise_nocache]] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:30:40.889 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:43.861483+0000", 2024-09-14T06:30:40.889 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 56.939905099000001, 2024-09-14T06:30:40.889 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.064065681999999999, 2024-09-14T06:30:40.889 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:30:40.889 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:30:40.889 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:30:40.890 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:30:40.890 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:30:40.890 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:40.890 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 44540 2024-09-14T06:30:40.890 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:40.890 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:30:40.890 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:40.890 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:30:40.890 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861483+0000", 2024-09-14T06:30:40.890 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:40.890 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:40.890 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:40.890 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:30:40.890 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861483+0000", 2024-09-14T06:30:40.891 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:40.891 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:40.891 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:40.891 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:30:40.891 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.867079+0000", 2024-09-14T06:30:40.891 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0055966480000000001 2024-09-14T06:30:40.891 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:40.891 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:40.891 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:30:40.891 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.867086+0000", 2024-09-14T06:30:40.891 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.387e-06 2024-09-14T06:30:40.891 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:40.891 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:40.891 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:30:40.891 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.867094+0000", 2024-09-14T06:30:40.892 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.9419999999999995e-06 2024-09-14T06:30:40.892 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:40.892 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:40.892 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:30:40.892 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.867109+0000", 2024-09-14T06:30:40.892 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.5818999999999999e-05 2024-09-14T06:30:40.892 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:40.892 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:40.892 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:30:40.892 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.925354+0000", 2024-09-14T06:30:40.892 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.058244047 2024-09-14T06:30:40.892 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:40.892 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:40.892 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:30:40.893 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.925406+0000", 2024-09-14T06:30:40.893 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.2661999999999998e-05 2024-09-14T06:30:40.893 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:40.893 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:40.893 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:30:40.893 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.925548+0000", 2024-09-14T06:30:40.893 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000142177 2024-09-14T06:30:40.893 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:40.893 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:40.893 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:40.893 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:40.893 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:40.893 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-14T06:30:40.895 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-14T06:30:40.896 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 reset_purged_snaps_last 2024-09-14T06:30:40.896 INFO:tasks.ceph.osd.2.smithi094.stderr:2024-09-14T06:30:40.894+0000 7faf3269b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33649) UID: 0 2024-09-14T06:30:40.996 INFO:tasks.ceph.osd.1.smithi094.stderr:2024-09-14T06:30:40.995+0000 7f328e47a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33641) UID: 0 2024-09-14T06:30:40.999 INFO:teuthology.orchestra.run.smithi138.stdout:{ 2024-09-14T06:30:40.999 INFO:teuthology.orchestra.run.smithi138.stdout: "ops": [], 2024-09-14T06:30:40.999 INFO:teuthology.orchestra.run.smithi138.stdout: "num_ops": 0 2024-09-14T06:30:41.000 INFO:teuthology.orchestra.run.smithi138.stdout:} 2024-09-14T06:30:41.008 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-14T06:30:41.068 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 reset_purged_snaps_last 2024-09-14T06:30:41.096 INFO:tasks.ceph.osd.6.smithi138.stderr:2024-09-14T06:30:41.095+0000 7f8cd8849640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33620) UID: 0 2024-09-14T06:30:41.111 INFO:teuthology.orchestra.run.smithi138.stdout:{ 2024-09-14T06:30:41.111 INFO:teuthology.orchestra.run.smithi138.stdout: "ops": [], 2024-09-14T06:30:41.111 INFO:teuthology.orchestra.run.smithi138.stdout: "complaint_time": 30, 2024-09-14T06:30:41.111 INFO:teuthology.orchestra.run.smithi138.stdout: "num_blocked_ops": 0 2024-09-14T06:30:41.111 INFO:teuthology.orchestra.run.smithi138.stdout:} 2024-09-14T06:30:41.119 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-14T06:30:41.194 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-14T06:30:41.203 INFO:tasks.ceph.osd.4.smithi138.stderr:2024-09-14T06:30:41.202+0000 7f6c653c8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33615) UID: 0 2024-09-14T06:30:41.242 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 reset_purged_snaps_last 2024-09-14T06:30:41.264 INFO:teuthology.orchestra.run.smithi138.stdout:{ 2024-09-14T06:30:41.264 INFO:teuthology.orchestra.run.smithi138.stdout: "size": 20, 2024-09-14T06:30:41.264 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 600, 2024-09-14T06:30:41.264 INFO:teuthology.orchestra.run.smithi138.stdout: "ops": [ 2024-09-14T06:30:41.264 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:41.264 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:19328 3.3 3:c34e5e51:::rbd_data.112d92027590.0000000000000024:f [read 1926144~1024 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:30:41.264 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:27.059768+0000", 2024-09-14T06:30:41.264 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 74.202484491000007, 2024-09-14T06:30:41.264 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046369147999999999, 2024-09-14T06:30:41.265 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:30:41.265 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:30:41.265 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:30:41.265 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:30:41.265 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:30:41.265 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:41.265 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 19328 2024-09-14T06:30:41.265 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:41.265 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:30:41.265 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:41.265 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:30:41.265 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.059768+0000", 2024-09-14T06:30:41.265 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:41.265 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:41.265 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:41.266 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:30:41.266 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.059768+0000", 2024-09-14T06:30:41.266 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:41.266 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:41.266 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:41.266 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:30:41.266 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.059874+0000", 2024-09-14T06:30:41.266 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00010551400000000001 2024-09-14T06:30:41.266 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:41.266 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:41.266 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:30:41.266 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.059877+0000", 2024-09-14T06:30:41.266 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.8279999999999999e-06 2024-09-14T06:30:41.266 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:41.266 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:41.267 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:30:41.267 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.059884+0000", 2024-09-14T06:30:41.267 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.6389999999999992e-06 2024-09-14T06:30:41.267 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:41.267 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:41.267 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:30:41.267 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.059890+0000", 2024-09-14T06:30:41.267 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.49e-06 2024-09-14T06:30:41.267 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:41.267 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:41.267 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:30:41.267 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.105804+0000", 2024-09-14T06:30:41.268 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.045914242000000001 2024-09-14T06:30:41.268 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:41.268 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:41.268 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:30:41.268 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.105886+0000", 2024-09-14T06:30:41.268 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.2305999999999994e-05 2024-09-14T06:30:41.268 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:41.268 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:41.268 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:30:41.268 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.106138+0000", 2024-09-14T06:30:41.268 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.000251129 2024-09-14T06:30:41.268 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:41.268 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:30:41.268 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:41.269 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:41.269 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:41.269 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:19329 3.3 3:c34e5e51:::rbd_data.112d92027590.0000000000000024:f [read 1936384~1024 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:30:41.269 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:27.059899+0000", 2024-09-14T06:30:41.269 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 74.202353967999997, 2024-09-14T06:30:41.269 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046572122000000001, 2024-09-14T06:30:41.269 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:30:41.269 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:30:41.269 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:30:41.269 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:30:41.269 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:30:41.269 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:41.269 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 19329 2024-09-14T06:30:41.270 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:41.270 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:30:41.270 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:41.270 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:30:41.270 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.059899+0000", 2024-09-14T06:30:41.270 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:41.270 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:41.270 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:41.270 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:30:41.270 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.059899+0000", 2024-09-14T06:30:41.270 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:41.270 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:41.270 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:41.270 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:30:41.271 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062501+0000", 2024-09-14T06:30:41.271 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.0026016419999999999 2024-09-14T06:30:41.271 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:41.271 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:41.271 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:30:41.271 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062514+0000", 2024-09-14T06:30:41.271 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.2947e-05 2024-09-14T06:30:41.271 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:41.271 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:41.271 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:30:41.271 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062523+0000", 2024-09-14T06:30:41.271 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.5519999999999993e-06 2024-09-14T06:30:41.271 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:41.271 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:41.272 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:30:41.272 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062534+0000", 2024-09-14T06:30:41.272 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.1416e-05 2024-09-14T06:30:41.272 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:41.272 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:41.272 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:30:41.272 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.106179+0000", 2024-09-14T06:30:41.272 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.043644049999999997 2024-09-14T06:30:41.272 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:41.272 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:41.272 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:30:41.272 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.106262+0000", 2024-09-14T06:30:41.272 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.3178999999999998e-05 2024-09-14T06:30:41.272 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:41.273 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:41.273 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:30:41.273 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.106471+0000", 2024-09-14T06:30:41.273 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00020933599999999999 2024-09-14T06:30:41.273 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:41.273 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:30:41.273 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:41.273 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:41.273 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:41.273 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:19351 3.3 3:c34e5e51:::rbd_data.112d92027590.0000000000000024:f [read 1159168~1024 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:30:41.273 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:27.062766+0000", 2024-09-14T06:30:41.273 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 74.199486742999994, 2024-09-14T06:30:41.273 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046099454999999998, 2024-09-14T06:30:41.273 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:30:41.273 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:30:41.274 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:30:41.274 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:30:41.274 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:30:41.274 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:41.274 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 19351 2024-09-14T06:30:41.274 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:41.274 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:30:41.274 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:41.274 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:30:41.274 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062766+0000", 2024-09-14T06:30:41.274 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:41.274 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:41.274 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:41.274 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:30:41.274 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062766+0000", 2024-09-14T06:30:41.275 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:41.275 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:41.275 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:41.275 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:30:41.275 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062766+0000", 2024-09-14T06:30:41.275 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.6e-07 2024-09-14T06:30:41.275 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:41.275 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:41.275 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:30:41.275 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062767+0000", 2024-09-14T06:30:41.275 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.0070000000000001e-06 2024-09-14T06:30:41.275 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:41.275 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:41.275 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:30:41.276 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062785+0000", 2024-09-14T06:30:41.276 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.7884999999999999e-05 2024-09-14T06:30:41.276 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:41.276 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:41.276 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:30:41.276 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062790+0000", 2024-09-14T06:30:41.276 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.0900000000000004e-06 2024-09-14T06:30:41.276 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:41.276 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:41.276 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:30:41.276 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.108735+0000", 2024-09-14T06:30:41.276 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.045944411999999997 2024-09-14T06:30:41.276 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:41.276 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:41.276 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:30:41.276 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.108809+0000", 2024-09-14T06:30:41.277 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.4157000000000002e-05 2024-09-14T06:30:41.277 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:41.277 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:41.277 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:30:41.277 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.108866+0000", 2024-09-14T06:30:41.277 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.6644000000000002e-05 2024-09-14T06:30:41.277 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:41.277 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:30:41.277 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:41.277 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:41.277 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:41.277 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:19352 3.3 3:c34e5e51:::rbd_data.112d92027590.0000000000000024:f [read 1169408~1024 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:30:41.277 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:27.062798+0000", 2024-09-14T06:30:41.277 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 74.199455331999999, 2024-09-14T06:30:41.277 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046255900000000003, 2024-09-14T06:30:41.278 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:30:41.278 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:30:41.278 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:30:41.278 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:30:41.278 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:30:41.278 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:41.278 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 19352 2024-09-14T06:30:41.278 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:41.278 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:30:41.278 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:41.278 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:30:41.278 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062798+0000", 2024-09-14T06:30:41.278 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:41.278 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:41.278 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:41.279 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:30:41.279 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062798+0000", 2024-09-14T06:30:41.279 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:41.279 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:41.279 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:41.279 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:30:41.279 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062798+0000", 2024-09-14T06:30:41.279 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.37e-07 2024-09-14T06:30:41.279 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:41.279 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:41.279 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:30:41.279 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062800+0000", 2024-09-14T06:30:41.279 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.0269999999999998e-06 2024-09-14T06:30:41.279 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:41.280 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:41.280 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:30:41.280 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062806+0000", 2024-09-14T06:30:41.280 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.6690000000000003e-06 2024-09-14T06:30:41.280 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:41.280 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:41.280 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:30:41.280 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062810+0000", 2024-09-14T06:30:41.280 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.3449999999999999e-06 2024-09-14T06:30:41.280 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:41.280 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:41.280 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:30:41.280 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.108892+0000", 2024-09-14T06:30:41.280 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046081715000000002 2024-09-14T06:30:41.281 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:41.281 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:41.281 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:30:41.281 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.108940+0000", 2024-09-14T06:30:41.281 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.7815999999999998e-05 2024-09-14T06:30:41.281 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:41.281 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:41.281 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:30:41.281 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.109053+0000", 2024-09-14T06:30:41.281 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.000113791 2024-09-14T06:30:41.281 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:41.281 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:30:41.281 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:41.281 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:41.281 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:41.281 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:19353 3.3 3:c34e5e51:::rbd_data.112d92027590.0000000000000024:f [read 1179648~1024 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:30:41.282 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:27.062816+0000", 2024-09-14T06:30:41.282 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 74.199436813000005, 2024-09-14T06:30:41.282 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046481256999999998, 2024-09-14T06:30:41.282 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:30:41.282 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:30:41.282 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:30:41.282 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:30:41.282 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:30:41.282 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:41.282 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 19353 2024-09-14T06:30:41.282 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:41.282 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:30:41.282 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:41.282 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:30:41.282 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062816+0000", 2024-09-14T06:30:41.282 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:41.283 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:41.283 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:41.283 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:30:41.283 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062816+0000", 2024-09-14T06:30:41.283 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:41.283 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:41.283 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:41.283 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:30:41.283 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062816+0000", 2024-09-14T06:30:41.283 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.2899999999999999e-07 2024-09-14T06:30:41.283 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:41.283 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:41.283 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:30:41.283 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062818+0000", 2024-09-14T06:30:41.283 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.699e-06 2024-09-14T06:30:41.283 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:41.284 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:41.284 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:30:41.284 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062825+0000", 2024-09-14T06:30:41.284 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.878e-06 2024-09-14T06:30:41.284 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:41.284 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:41.284 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:30:41.284 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062830+0000", 2024-09-14T06:30:41.284 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.5410000000000002e-06 2024-09-14T06:30:41.284 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:41.284 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:41.284 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:30:41.284 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.109096+0000", 2024-09-14T06:30:41.284 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046266517 2024-09-14T06:30:41.285 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:41.285 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:41.285 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:30:41.285 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.109190+0000", 2024-09-14T06:30:41.285 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.4000999999999997e-05 2024-09-14T06:30:41.285 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:41.285 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:41.285 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:30:41.285 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.109297+0000", 2024-09-14T06:30:41.285 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.000107292 2024-09-14T06:30:41.285 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:41.285 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:30:41.285 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:41.285 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:41.285 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:41.285 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:19354 3.3 3:c34e5e51:::rbd_data.112d92027590.0000000000000024:f [read 1189888~1024 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:30:41.286 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:27.062842+0000", 2024-09-14T06:30:41.286 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 74.199411358000006, 2024-09-14T06:30:41.286 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046594257, 2024-09-14T06:30:41.286 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:30:41.286 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:30:41.286 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:30:41.286 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:30:41.286 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:30:41.286 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:41.286 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 19354 2024-09-14T06:30:41.286 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:41.286 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:30:41.286 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:41.286 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:30:41.287 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062842+0000", 2024-09-14T06:30:41.287 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:41.287 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:41.287 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:41.287 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:30:41.287 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062842+0000", 2024-09-14T06:30:41.287 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:41.287 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:41.287 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:41.287 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:30:41.287 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062842+0000", 2024-09-14T06:30:41.287 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.7399999999999999e-07 2024-09-14T06:30:41.287 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:41.287 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:41.287 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:30:41.287 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062859+0000", 2024-09-14T06:30:41.288 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.7010000000000001e-05 2024-09-14T06:30:41.288 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:41.288 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:41.288 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:30:41.288 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062866+0000", 2024-09-14T06:30:41.288 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.9990000000000002e-06 2024-09-14T06:30:41.288 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:41.288 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:41.288 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:30:41.288 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062873+0000", 2024-09-14T06:30:41.288 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.8569999999999997e-06 2024-09-14T06:30:41.288 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:41.288 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:41.288 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:30:41.288 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.109335+0000", 2024-09-14T06:30:41.289 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046461943999999998 2024-09-14T06:30:41.289 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:41.289 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:41.289 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:30:41.289 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.109377+0000", 2024-09-14T06:30:41.289 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.2558999999999997e-05 2024-09-14T06:30:41.289 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:41.289 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:41.289 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:30:41.289 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.109436+0000", 2024-09-14T06:30:41.289 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.8514000000000003e-05 2024-09-14T06:30:41.289 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:41.289 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:30:41.289 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:41.290 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:41.290 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:41.290 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:19355 3.3 3:c34e5e51:::rbd_data.112d92027590.0000000000000024:f [read 1200128~1024 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:30:41.290 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:27.062884+0000", 2024-09-14T06:30:41.290 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 74.199368684999996, 2024-09-14T06:30:41.290 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046681650999999998, 2024-09-14T06:30:41.290 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:30:41.290 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:30:41.290 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:30:41.290 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:30:41.290 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:30:41.290 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:41.290 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 19355 2024-09-14T06:30:41.290 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:41.291 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:30:41.291 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:41.291 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:30:41.291 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062884+0000", 2024-09-14T06:30:41.291 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:41.291 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:41.291 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:41.291 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:30:41.291 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062884+0000", 2024-09-14T06:30:41.291 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:41.291 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:41.291 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:41.291 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:30:41.292 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062885+0000", 2024-09-14T06:30:41.292 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.8000000000000001e-07 2024-09-14T06:30:41.292 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:41.292 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:41.292 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:30:41.292 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062886+0000", 2024-09-14T06:30:41.292 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.8920000000000001e-06 2024-09-14T06:30:41.292 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:41.292 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:41.292 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:30:41.292 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062892+0000", 2024-09-14T06:30:41.292 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.84e-06 2024-09-14T06:30:41.292 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:41.292 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:41.293 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:30:41.293 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062899+0000", 2024-09-14T06:30:41.293 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.5339999999999996e-06 2024-09-14T06:30:41.293 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:41.293 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:41.293 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:30:41.293 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.109466+0000", 2024-09-14T06:30:41.293 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046566919999999998 2024-09-14T06:30:41.293 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:41.293 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:41.293 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:30:41.293 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.109510+0000", 2024-09-14T06:30:41.293 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.4153000000000002e-05 2024-09-14T06:30:41.293 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:41.293 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:41.294 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:30:41.294 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.109566+0000", 2024-09-14T06:30:41.294 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.5931999999999999e-05 2024-09-14T06:30:41.294 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:41.294 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:30:41.294 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:41.294 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:41.294 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:41.294 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:19356 3.3 3:c34e5e51:::rbd_data.112d92027590.0000000000000024:f [read 1210368~1024 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:30:41.294 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:27.062908+0000", 2024-09-14T06:30:41.294 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 74.199345248, 2024-09-14T06:30:41.294 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046807498000000003, 2024-09-14T06:30:41.294 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:30:41.294 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:30:41.295 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:30:41.295 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:30:41.295 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:30:41.295 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:41.295 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 19356 2024-09-14T06:30:41.295 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:41.295 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:30:41.295 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:41.295 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:30:41.295 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062908+0000", 2024-09-14T06:30:41.295 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:41.295 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:41.295 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:41.295 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:30:41.296 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062908+0000", 2024-09-14T06:30:41.296 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:41.296 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:41.296 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:41.296 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:30:41.296 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062908+0000", 2024-09-14T06:30:41.296 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.5699999999999998e-07 2024-09-14T06:30:41.296 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:41.296 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:41.296 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:30:41.296 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062910+0000", 2024-09-14T06:30:41.296 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.7710000000000001e-06 2024-09-14T06:30:41.296 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:41.297 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:41.297 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:30:41.297 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062920+0000", 2024-09-14T06:30:41.297 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.0654e-05 2024-09-14T06:30:41.297 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:41.297 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:41.297 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:30:41.297 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062931+0000", 2024-09-14T06:30:41.297 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.075e-05 2024-09-14T06:30:41.297 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:41.297 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:41.297 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:30:41.297 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.109595+0000", 2024-09-14T06:30:41.297 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046663713000000002 2024-09-14T06:30:41.298 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:41.298 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:41.298 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:30:41.298 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.109632+0000", 2024-09-14T06:30:41.298 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.7505000000000001e-05 2024-09-14T06:30:41.298 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:41.298 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:41.298 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:30:41.298 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.109715+0000", 2024-09-14T06:30:41.298 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.2748000000000001e-05 2024-09-14T06:30:41.298 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:41.298 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:30:41.298 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:41.298 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:41.298 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:41.298 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:19357 3.3 3:c34e5e51:::rbd_data.112d92027590.0000000000000024:f [read 1220608~1024 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:30:41.299 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:27.062939+0000", 2024-09-14T06:30:41.299 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 74.199313799999999, 2024-09-14T06:30:41.299 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046920508, 2024-09-14T06:30:41.299 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:30:41.299 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:30:41.299 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:30:41.299 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:30:41.299 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:30:41.299 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:41.299 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 19357 2024-09-14T06:30:41.299 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:41.299 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:30:41.299 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:41.299 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:30:41.299 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062939+0000", 2024-09-14T06:30:41.300 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:41.300 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:41.300 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:41.300 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:30:41.300 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062939+0000", 2024-09-14T06:30:41.300 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:41.300 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:41.300 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:41.300 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:30:41.300 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062940+0000", 2024-09-14T06:30:41.300 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.32e-07 2024-09-14T06:30:41.300 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:41.300 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:41.301 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:30:41.301 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062942+0000", 2024-09-14T06:30:41.301 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.1919999999999999e-06 2024-09-14T06:30:41.301 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:41.301 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:41.301 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:30:41.301 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062948+0000", 2024-09-14T06:30:41.301 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.1149999999999999e-06 2024-09-14T06:30:41.301 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:41.301 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:41.301 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:30:41.301 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062960+0000", 2024-09-14T06:30:41.301 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.1895e-05 2024-09-14T06:30:41.301 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:41.302 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:41.302 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:30:41.302 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.109704+0000", 2024-09-14T06:30:41.302 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046744584999999998 2024-09-14T06:30:41.302 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:41.302 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:41.302 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:30:41.302 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.109777+0000", 2024-09-14T06:30:41.302 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.2282999999999999e-05 2024-09-14T06:30:41.302 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:41.302 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:41.302 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:30:41.302 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.109860+0000", 2024-09-14T06:30:41.302 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.3005999999999998e-05 2024-09-14T06:30:41.303 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:41.303 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:30:41.303 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:41.303 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:41.303 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:41.303 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:19358 3.3 3:c34e5e51:::rbd_data.112d92027590.0000000000000024:f [read 1230848~1024 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:30:41.303 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:27.062967+0000", 2024-09-14T06:30:41.303 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 74.199286169999993, 2024-09-14T06:30:41.303 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047144471, 2024-09-14T06:30:41.303 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:30:41.303 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:30:41.303 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:30:41.303 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:30:41.303 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:30:41.304 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:41.304 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 19358 2024-09-14T06:30:41.304 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:41.304 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:30:41.304 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:41.304 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:30:41.304 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062967+0000", 2024-09-14T06:30:41.304 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:41.304 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:41.304 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:41.304 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:30:41.304 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062967+0000", 2024-09-14T06:30:41.304 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:41.305 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:41.305 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:41.305 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:30:41.305 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062967+0000", 2024-09-14T06:30:41.305 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.89e-07 2024-09-14T06:30:41.305 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:41.305 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:41.305 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:30:41.305 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062969+0000", 2024-09-14T06:30:41.305 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.9929999999999998e-06 2024-09-14T06:30:41.305 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:41.305 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:41.305 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:30:41.305 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062975+0000", 2024-09-14T06:30:41.306 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.9030000000000001e-06 2024-09-14T06:30:41.306 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:41.306 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:41.306 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:30:41.306 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062979+0000", 2024-09-14T06:30:41.306 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.2250000000000002e-06 2024-09-14T06:30:41.306 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:41.306 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:41.306 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:30:41.306 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.109934+0000", 2024-09-14T06:30:41.306 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046954410000000002 2024-09-14T06:30:41.306 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:41.306 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:41.306 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:30:41.307 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.110007+0000", 2024-09-14T06:30:41.307 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.3829000000000001e-05 2024-09-14T06:30:41.307 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:41.307 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:41.307 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:30:41.307 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.110111+0000", 2024-09-14T06:30:41.307 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00010372199999999999 2024-09-14T06:30:41.307 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:41.307 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:30:41.307 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:41.307 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:41.307 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:41.307 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:19359 3.3 3:c34e5e51:::rbd_data.112d92027590.0000000000000024:f [read 1241088~1024 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:30:41.307 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:27.062987+0000", 2024-09-14T06:30:41.307 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 74.199266080000001, 2024-09-14T06:30:41.308 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047297182, 2024-09-14T06:30:41.308 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:30:41.308 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:30:41.308 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:30:41.308 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:30:41.308 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:30:41.308 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:41.308 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 19359 2024-09-14T06:30:41.308 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:41.308 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:30:41.308 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:41.308 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:30:41.308 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062987+0000", 2024-09-14T06:30:41.309 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:41.309 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:41.309 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:41.309 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:30:41.309 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062987+0000", 2024-09-14T06:30:41.309 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:41.309 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:41.309 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:41.309 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:30:41.309 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062987+0000", 2024-09-14T06:30:41.309 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.2500000000000001e-07 2024-09-14T06:30:41.309 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:41.309 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:41.310 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:30:41.310 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062989+0000", 2024-09-14T06:30:41.310 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.6080000000000001e-06 2024-09-14T06:30:41.310 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:41.310 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:41.310 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:30:41.310 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062994+0000", 2024-09-14T06:30:41.310 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.5520000000000004e-06 2024-09-14T06:30:41.310 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:41.310 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:41.310 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:30:41.310 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062999+0000", 2024-09-14T06:30:41.310 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.3930000000000001e-06 2024-09-14T06:30:41.310 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:41.311 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:41.311 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:30:41.311 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.110142+0000", 2024-09-14T06:30:41.311 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047143154 2024-09-14T06:30:41.311 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:41.311 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:41.311 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:30:41.311 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.110219+0000", 2024-09-14T06:30:41.311 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.6737999999999996e-05 2024-09-14T06:30:41.311 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:41.311 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:41.311 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:30:41.311 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.110284+0000", 2024-09-14T06:30:41.311 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.5412000000000003e-05 2024-09-14T06:30:41.312 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:41.312 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:30:41.312 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:41.312 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:41.312 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:41.312 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:19360 3.3 3:c34e5e51:::rbd_data.112d92027590.0000000000000024:f [read 1251328~1024 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:30:41.312 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:27.063017+0000", 2024-09-14T06:30:41.312 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 74.199235619000007, 2024-09-14T06:30:41.312 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047453823999999999, 2024-09-14T06:30:41.312 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:30:41.312 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:30:41.312 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:30:41.312 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:30:41.312 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:30:41.313 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:41.313 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 19360 2024-09-14T06:30:41.313 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:41.313 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:30:41.313 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:41.313 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:30:41.313 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.063017+0000", 2024-09-14T06:30:41.313 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:41.313 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:41.313 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:41.313 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:30:41.313 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.063017+0000", 2024-09-14T06:30:41.313 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:41.313 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:41.314 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:41.314 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:30:41.314 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.064977+0000", 2024-09-14T06:30:41.314 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.0019597080000000001 2024-09-14T06:30:41.314 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:41.314 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:41.314 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:30:41.314 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.064987+0000", 2024-09-14T06:30:41.314 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.9620000000000005e-06 2024-09-14T06:30:41.314 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:41.314 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:41.314 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:30:41.314 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065000+0000", 2024-09-14T06:30:41.314 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.3173000000000001e-05 2024-09-14T06:30:41.314 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:41.315 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:41.315 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:30:41.315 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065005+0000", 2024-09-14T06:30:41.315 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.9960000000000001e-06 2024-09-14T06:30:41.315 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:41.315 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:41.315 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:30:41.315 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.110314+0000", 2024-09-14T06:30:41.315 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.04530899 2024-09-14T06:30:41.315 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:41.315 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:41.315 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:30:41.315 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.110398+0000", 2024-09-14T06:30:41.315 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.3539000000000001e-05 2024-09-14T06:30:41.316 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:41.316 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:41.316 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:30:41.316 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.110471+0000", 2024-09-14T06:30:41.316 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.3455999999999996e-05 2024-09-14T06:30:41.316 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:41.316 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:30:41.316 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:41.316 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:41.316 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:41.316 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:19365 3.3 3:c34e5e51:::rbd_data.112d92027590.0000000000000024:f [read 1302528~1024 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:30:41.316 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:27.065058+0000", 2024-09-14T06:30:41.316 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 74.197194948999993, 2024-09-14T06:30:41.316 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046105526000000001, 2024-09-14T06:30:41.317 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:30:41.317 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:30:41.317 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:30:41.317 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:30:41.317 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:30:41.317 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:41.317 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 19365 2024-09-14T06:30:41.317 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:41.317 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:30:41.317 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:41.317 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:30:41.317 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065058+0000", 2024-09-14T06:30:41.317 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:41.317 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:41.318 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:41.318 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:30:41.318 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065058+0000", 2024-09-14T06:30:41.318 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:41.318 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:41.318 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:41.318 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:30:41.318 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065058+0000", 2024-09-14T06:30:41.318 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.1899999999999999e-07 2024-09-14T06:30:41.318 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:41.318 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:41.318 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:30:41.318 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065059+0000", 2024-09-14T06:30:41.318 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.1299999999999998e-07 2024-09-14T06:30:41.319 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:41.319 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:41.319 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:30:41.319 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065063+0000", 2024-09-14T06:30:41.319 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.1099999999999996e-06 2024-09-14T06:30:41.319 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:41.319 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:41.319 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:30:41.319 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065070+0000", 2024-09-14T06:30:41.319 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.973e-06 2024-09-14T06:30:41.319 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:41.319 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:41.319 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:30:41.319 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.111014+0000", 2024-09-14T06:30:41.319 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.045943373000000003 2024-09-14T06:30:41.320 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:41.320 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:41.320 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:30:41.320 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.111073+0000", 2024-09-14T06:30:41.320 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.8987999999999998e-05 2024-09-14T06:30:41.320 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:41.320 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:41.320 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:30:41.320 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.111163+0000", 2024-09-14T06:30:41.320 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.0950000000000002e-05 2024-09-14T06:30:41.320 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:41.320 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:30:41.320 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:41.320 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:41.321 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:41.321 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:19366 3.3 3:c34e5e51:::rbd_data.112d92027590.0000000000000024:f [read 1312768~1024 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:30:41.321 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:27.065074+0000", 2024-09-14T06:30:41.321 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 74.197179098999996, 2024-09-14T06:30:41.321 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046289858000000003, 2024-09-14T06:30:41.321 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:30:41.321 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:30:41.321 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:30:41.321 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:30:41.321 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:30:41.321 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:41.321 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 19366 2024-09-14T06:30:41.321 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:41.321 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:30:41.321 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:41.321 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:30:41.322 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065074+0000", 2024-09-14T06:30:41.322 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:41.322 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:41.322 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:41.322 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:30:41.322 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065074+0000", 2024-09-14T06:30:41.322 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:41.322 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:41.322 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:41.322 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:30:41.322 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065074+0000", 2024-09-14T06:30:41.322 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.3900000000000001e-07 2024-09-14T06:30:41.322 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:41.322 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:41.322 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:30:41.322 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065078+0000", 2024-09-14T06:30:41.323 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.3229999999999999e-06 2024-09-14T06:30:41.323 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:41.323 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:41.323 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:30:41.323 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065082+0000", 2024-09-14T06:30:41.323 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.8299999999999998e-06 2024-09-14T06:30:41.323 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:41.323 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:41.323 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:30:41.323 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065085+0000", 2024-09-14T06:30:41.323 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.8660000000000002e-06 2024-09-14T06:30:41.323 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:41.323 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:41.324 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:30:41.324 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.111270+0000", 2024-09-14T06:30:41.324 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046185035999999999 2024-09-14T06:30:41.324 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:41.324 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:41.324 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:30:41.324 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.111311+0000", 2024-09-14T06:30:41.324 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.0935e-05 2024-09-14T06:30:41.324 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:41.324 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:41.324 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:30:41.324 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.111364+0000", 2024-09-14T06:30:41.324 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.2629e-05 2024-09-14T06:30:41.324 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:41.325 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:30:41.325 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:41.325 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:41.325 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:41.325 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:19367 3.3 3:c34e5e51:::rbd_data.112d92027590.0000000000000024:f [read 1323008~1024 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:30:41.325 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:27.065115+0000", 2024-09-14T06:30:41.325 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 74.197137538000007, 2024-09-14T06:30:41.325 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046407003000000002, 2024-09-14T06:30:41.325 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:30:41.325 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:30:41.325 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:30:41.325 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:30:41.325 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:30:41.326 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:41.326 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 19367 2024-09-14T06:30:41.326 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:41.326 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:30:41.326 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:41.326 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:30:41.326 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065115+0000", 2024-09-14T06:30:41.326 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:41.326 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:41.326 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:41.326 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:30:41.326 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065115+0000", 2024-09-14T06:30:41.326 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:41.326 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:41.327 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:41.327 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:30:41.327 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065116+0000", 2024-09-14T06:30:41.327 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.4999999999999998e-07 2024-09-14T06:30:41.327 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:41.327 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:41.327 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:30:41.327 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065117+0000", 2024-09-14T06:30:41.327 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.189e-06 2024-09-14T06:30:41.327 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:41.327 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:41.327 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:30:41.327 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065122+0000", 2024-09-14T06:30:41.327 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.3709999999999999e-06 2024-09-14T06:30:41.327 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:41.328 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:41.328 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:30:41.328 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065125+0000", 2024-09-14T06:30:41.328 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.8940000000000002e-06 2024-09-14T06:30:41.328 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:41.328 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:41.328 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:30:41.328 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.111402+0000", 2024-09-14T06:30:41.328 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046276645999999998 2024-09-14T06:30:41.328 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:41.328 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:41.328 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:30:41.328 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.111445+0000", 2024-09-14T06:30:41.328 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.3288000000000003e-05 2024-09-14T06:30:41.328 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:41.329 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:41.329 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:30:41.329 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.111522+0000", 2024-09-14T06:30:41.329 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.7164999999999997e-05 2024-09-14T06:30:41.329 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:41.329 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:30:41.329 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:41.329 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:41.329 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:41.329 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:19368 3.3 3:c34e5e51:::rbd_data.112d92027590.0000000000000024:f [read 1333248~1024 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:30:41.329 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:27.065130+0000", 2024-09-14T06:30:41.329 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 74.197123247999997, 2024-09-14T06:30:41.329 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046507275000000001, 2024-09-14T06:30:41.329 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:30:41.329 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:30:41.330 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:30:41.330 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:30:41.330 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:30:41.330 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:41.330 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 19368 2024-09-14T06:30:41.330 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:41.330 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:30:41.330 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:41.330 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:30:41.330 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065130+0000", 2024-09-14T06:30:41.330 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:41.330 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:41.330 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:41.330 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:30:41.331 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065130+0000", 2024-09-14T06:30:41.331 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:41.331 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:41.331 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:41.331 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:30:41.331 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065130+0000", 2024-09-14T06:30:41.331 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.0800000000000001e-07 2024-09-14T06:30:41.331 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:41.331 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:41.331 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:30:41.331 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065131+0000", 2024-09-14T06:30:41.331 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.8700000000000004e-07 2024-09-14T06:30:41.332 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:41.332 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:41.332 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:30:41.332 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065134+0000", 2024-09-14T06:30:41.332 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.1039999999999998e-06 2024-09-14T06:30:41.332 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:41.332 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:41.332 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:30:41.332 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065136+0000", 2024-09-14T06:30:41.332 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.3949999999999999e-06 2024-09-14T06:30:41.332 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:41.332 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:41.332 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:30:41.333 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.111541+0000", 2024-09-14T06:30:41.333 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046404206000000003 2024-09-14T06:30:41.333 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:41.333 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:41.333 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:30:41.333 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.111582+0000", 2024-09-14T06:30:41.333 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.1051999999999999e-05 2024-09-14T06:30:41.333 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:41.333 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:41.333 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:30:41.333 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.111637+0000", 2024-09-14T06:30:41.333 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.5322999999999999e-05 2024-09-14T06:30:41.333 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:41.334 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:30:41.334 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:41.334 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:41.334 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:41.334 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:19369 3.3 3:c34e5e51:::rbd_data.112d92027590.0000000000000024:f [read 1343488~1024 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:30:41.334 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:27.065139+0000", 2024-09-14T06:30:41.334 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 74.197113431000005, 2024-09-14T06:30:41.334 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046682169000000003, 2024-09-14T06:30:41.334 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:30:41.334 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:30:41.334 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:30:41.334 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:30:41.334 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:30:41.335 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:41.335 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 19369 2024-09-14T06:30:41.335 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:41.335 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:30:41.335 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:41.335 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:30:41.335 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065139+0000", 2024-09-14T06:30:41.335 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:41.335 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:41.335 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:41.335 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:30:41.335 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065139+0000", 2024-09-14T06:30:41.335 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:41.335 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:41.336 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:41.336 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:30:41.336 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065140+0000", 2024-09-14T06:30:41.336 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.8299999999999998e-07 2024-09-14T06:30:41.336 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:41.336 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:41.336 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:30:41.336 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065141+0000", 2024-09-14T06:30:41.336 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.3900000000000003e-07 2024-09-14T06:30:41.336 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:41.336 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:41.336 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:30:41.336 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065144+0000", 2024-09-14T06:30:41.337 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.7409999999999998e-06 2024-09-14T06:30:41.337 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:41.337 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:41.337 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:30:41.337 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065147+0000", 2024-09-14T06:30:41.337 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.4150000000000002e-06 2024-09-14T06:30:41.337 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:41.337 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:41.337 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:30:41.337 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.111673+0000", 2024-09-14T06:30:41.337 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046526004000000003 2024-09-14T06:30:41.337 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:41.337 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:41.337 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:30:41.338 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.111749+0000", 2024-09-14T06:30:41.338 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.5741999999999994e-05 2024-09-14T06:30:41.338 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:41.338 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:41.338 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:30:41.338 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.111822+0000", 2024-09-14T06:30:41.338 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.3045000000000003e-05 2024-09-14T06:30:41.338 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:41.338 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:30:41.338 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:41.338 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:41.338 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:41.338 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:19370 3.3 3:c34e5e51:::rbd_data.112d92027590.0000000000000024:f [read 1353728~1024 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:30:41.338 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:27.065150+0000", 2024-09-14T06:30:41.339 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 74.197102701999995, 2024-09-14T06:30:41.339 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046897556999999999, 2024-09-14T06:30:41.339 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:30:41.339 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:30:41.339 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:30:41.339 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:30:41.339 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:30:41.339 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:41.339 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 19370 2024-09-14T06:30:41.339 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:41.339 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:30:41.339 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:41.339 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:30:41.339 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065150+0000", 2024-09-14T06:30:41.339 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:41.340 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:41.340 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:41.340 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:30:41.340 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065150+0000", 2024-09-14T06:30:41.340 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:41.340 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:41.340 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:41.340 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:30:41.340 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065150+0000", 2024-09-14T06:30:41.340 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.0200000000000001e-07 2024-09-14T06:30:41.340 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:41.340 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:41.340 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:30:41.341 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065151+0000", 2024-09-14T06:30:41.341 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.1100000000000004e-07 2024-09-14T06:30:41.341 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:41.341 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:41.341 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:30:41.341 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065154+0000", 2024-09-14T06:30:41.341 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.9639999999999999e-06 2024-09-14T06:30:41.341 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:41.341 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:41.341 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:30:41.341 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065157+0000", 2024-09-14T06:30:41.341 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.4260000000000002e-06 2024-09-14T06:30:41.341 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:41.341 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:41.342 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:30:41.342 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.111861+0000", 2024-09-14T06:30:41.342 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046704268 2024-09-14T06:30:41.342 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:41.342 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:41.342 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:30:41.342 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.111939+0000", 2024-09-14T06:30:41.342 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.8231000000000004e-05 2024-09-14T06:30:41.342 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:41.342 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:41.342 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:30:41.342 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.112048+0000", 2024-09-14T06:30:41.342 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00010855500000000001 2024-09-14T06:30:41.342 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:41.343 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:30:41.343 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:41.343 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:41.343 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:41.343 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:19371 3.3 3:c34e5e51:::rbd_data.112d92027590.0000000000000024:f [read 1363968~1024 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:30:41.343 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:27.065161+0000", 2024-09-14T06:30:41.343 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 74.197092052000002, 2024-09-14T06:30:41.343 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047021944000000003, 2024-09-14T06:30:41.343 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:30:41.343 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:30:41.343 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:30:41.343 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:30:41.343 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:30:41.343 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:41.343 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 19371 2024-09-14T06:30:41.344 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:41.344 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:30:41.344 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:41.344 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:30:41.344 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065161+0000", 2024-09-14T06:30:41.344 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:41.344 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:41.344 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:41.344 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:30:41.344 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065161+0000", 2024-09-14T06:30:41.344 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:41.344 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:41.345 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:41.345 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:30:41.345 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065161+0000", 2024-09-14T06:30:41.345 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.0200000000000001e-07 2024-09-14T06:30:41.345 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:41.345 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:41.345 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:30:41.345 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065166+0000", 2024-09-14T06:30:41.345 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.9869999999999999e-06 2024-09-14T06:30:41.345 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:41.345 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:41.345 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:30:41.345 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065169+0000", 2024-09-14T06:30:41.346 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.3799999999999998e-06 2024-09-14T06:30:41.346 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:41.346 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:41.346 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:30:41.346 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065172+0000", 2024-09-14T06:30:41.346 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.4830000000000002e-06 2024-09-14T06:30:41.346 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:41.346 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:41.346 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:30:41.346 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.112071+0000", 2024-09-14T06:30:41.346 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046899529000000002 2024-09-14T06:30:41.346 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:41.346 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:41.346 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:30:41.347 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.112128+0000", 2024-09-14T06:30:41.347 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.6341999999999997e-05 2024-09-14T06:30:41.347 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:41.347 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:41.347 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:30:41.347 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.112183+0000", 2024-09-14T06:30:41.347 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.5021000000000001e-05 2024-09-14T06:30:41.347 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:41.347 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:30:41.347 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:41.347 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:41.347 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:41.347 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:19372 3.3 3:c34e5e51:::rbd_data.112d92027590.0000000000000024:f [read 1374208~1024 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:30:41.348 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:27.065176+0000", 2024-09-14T06:30:41.348 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 74.197076902000006, 2024-09-14T06:30:41.348 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047188889999999997, 2024-09-14T06:30:41.348 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:30:41.348 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:30:41.348 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:30:41.348 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:30:41.348 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:30:41.348 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:41.348 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 19372 2024-09-14T06:30:41.348 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:41.348 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:30:41.348 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:41.349 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:30:41.349 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065176+0000", 2024-09-14T06:30:41.349 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:41.349 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:41.349 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:41.349 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:30:41.349 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065176+0000", 2024-09-14T06:30:41.349 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:41.349 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:41.349 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:41.349 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:30:41.349 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.071117+0000", 2024-09-14T06:30:41.349 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.0059404990000000001 2024-09-14T06:30:41.349 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:41.350 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:41.350 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:30:41.350 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.071120+0000", 2024-09-14T06:30:41.350 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.4340000000000001e-06 2024-09-14T06:30:41.350 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:41.350 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:41.350 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:30:41.350 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.071129+0000", 2024-09-14T06:30:41.350 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.8389999999999996e-06 2024-09-14T06:30:41.350 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:41.350 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:41.350 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:30:41.350 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.071134+0000", 2024-09-14T06:30:41.350 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.412e-06 2024-09-14T06:30:41.351 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:41.351 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:41.351 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:30:41.351 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.112221+0000", 2024-09-14T06:30:41.351 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.041086473999999998 2024-09-14T06:30:41.351 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:41.351 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:41.351 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:30:41.351 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.112299+0000", 2024-09-14T06:30:41.351 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.8746999999999997e-05 2024-09-14T06:30:41.351 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:41.351 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:41.351 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:30:41.351 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.112365+0000", 2024-09-14T06:30:41.352 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.5485000000000001e-05 2024-09-14T06:30:41.352 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:41.352 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:30:41.352 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:41.352 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:41.352 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:30:41.352 INFO:teuthology.orchestra.run.smithi138.stdout:} 2024-09-14T06:30:41.353 INFO:teuthology.orchestra.run.smithi094.stdout:osd.0: {} 2024-09-14T06:30:41.353 INFO:teuthology.orchestra.run.smithi094.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-14T06:30:41.354 INFO:teuthology.orchestra.run.smithi094.stdout:osd.1: {} 2024-09-14T06:30:41.354 INFO:teuthology.orchestra.run.smithi094.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-14T06:30:41.355 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-14T06:30:41.355 INFO:tasks.ceph.osd.6.smithi138.stderr:2024-09-14T06:30:41.352+0000 7f8cd8849640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33620) UID: 0 2024-09-14T06:30:41.355 INFO:teuthology.orchestra.run.smithi094.stdout:osd.2: {} 2024-09-14T06:30:41.356 INFO:teuthology.orchestra.run.smithi094.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-14T06:30:41.372 INFO:teuthology.orchestra.run.smithi094.stdout:osd.3: {} 2024-09-14T06:30:41.372 INFO:teuthology.orchestra.run.smithi094.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-14T06:30:41.383 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 reset_purged_snaps_last 2024-09-14T06:30:41.389 INFO:teuthology.orchestra.run.smithi094.stdout:osd.4: {} 2024-09-14T06:30:41.389 INFO:teuthology.orchestra.run.smithi094.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-14T06:30:41.406 INFO:teuthology.orchestra.run.smithi094.stdout:osd.5: {} 2024-09-14T06:30:41.406 INFO:teuthology.orchestra.run.smithi094.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-14T06:30:41.423 INFO:teuthology.orchestra.run.smithi094.stdout:osd.6: {} 2024-09-14T06:30:41.423 INFO:teuthology.orchestra.run.smithi094.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-14T06:30:41.439 INFO:teuthology.orchestra.run.smithi094.stdout:osd.7: {} 2024-09-14T06:30:41.439 INFO:teuthology.orchestra.run.smithi094.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-14T06:30:41.492 INFO:tasks.ceph.osd.0.smithi094.stderr:2024-09-14T06:30:41.491+0000 7f07c9333640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33636) UID: 0 2024-09-14T06:30:41.499 INFO:teuthology.orchestra.run.smithi138.stdout:ERROR: (22) Invalid argument 2024-09-14T06:30:41.499 INFO:teuthology.orchestra.run.smithi138.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:30:41.508 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-14T06:30:41.517 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 reset_purged_snaps_last 2024-09-14T06:30:41.555 INFO:tasks.ceph.osd.7.smithi138.stderr:2024-09-14T06:30:41.554+0000 7f3c6a21d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33623) UID: 0 2024-09-14T06:30:41.620 INFO:teuthology.orchestra.run.smithi138.stdout:ERROR: (22) Invalid argument 2024-09-14T06:30:41.621 INFO:teuthology.orchestra.run.smithi138.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:30:41.632 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-14T06:30:41.656 INFO:tasks.ceph.osd.2.smithi094.stderr:2024-09-14T06:30:41.655+0000 7faf3269b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33649) UID: 0 2024-09-14T06:30:41.700 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-14T06:30:41.757 INFO:tasks.ceph.osd.6.smithi138.stderr:2024-09-14T06:30:41.756+0000 7f8cd8849640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33620) UID: 0 2024-09-14T06:30:41.779 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 reset_purged_snaps_last 2024-09-14T06:30:41.817 INFO:teuthology.orchestra.run.smithi138.stdout:ERROR: (22) Invalid argument 2024-09-14T06:30:41.817 INFO:teuthology.orchestra.run.smithi138.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:30:41.828 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-14T06:30:41.869 INFO:tasks.ceph.osd.6.smithi138.stderr:2024-09-14T06:30:41.868+0000 7f8cd8849640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33620) UID: 0 2024-09-14T06:30:41.960 INFO:tasks.ceph.osd.6.smithi138.stderr:2024-09-14T06:30:41.958+0000 7f8cd8849640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33620) UID: 0 2024-09-14T06:30:42.036 INFO:teuthology.orchestra.run.smithi138.stdout:ERROR: (22) Invalid argument 2024-09-14T06:30:42.037 INFO:teuthology.orchestra.run.smithi138.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:30:42.050 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-14T06:30:42.091 INFO:tasks.ceph.osd.4.smithi138.stderr:2024-09-14T06:30:42.090+0000 7f6c653c8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33615) UID: 0 2024-09-14T06:30:42.162 INFO:tasks.ceph.osd.5.smithi138.stderr:2024-09-14T06:30:42.160+0000 7fe13746f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33618) UID: 0 2024-09-14T06:30:42.192 INFO:teuthology.orchestra.run.smithi138.stdout:ERROR: (22) Invalid argument 2024-09-14T06:30:42.192 INFO:teuthology.orchestra.run.smithi138.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:30:42.200 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-14T06:30:42.263 INFO:tasks.ceph.osd.1.smithi094.stderr:2024-09-14T06:30:42.262+0000 7f328e47a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33641) UID: 0 2024-09-14T06:30:42.343 INFO:teuthology.orchestra.run.smithi138.stdout:ERROR: (22) Invalid argument 2024-09-14T06:30:42.343 INFO:teuthology.orchestra.run.smithi138.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:30:42.352 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-14T06:30:42.363 INFO:tasks.ceph.osd.1.smithi094.stderr:2024-09-14T06:30:42.362+0000 7f328e47a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33641) UID: 0 2024-09-14T06:30:42.455 INFO:teuthology.orchestra.run.smithi138.stdout:ERROR: (22) Invalid argument 2024-09-14T06:30:42.455 INFO:teuthology.orchestra.run.smithi138.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:30:42.464 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-14T06:30:42.464 INFO:tasks.ceph.osd.6.smithi138.stderr:2024-09-14T06:30:42.462+0000 7f8cd8849640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33620) UID: 0 2024-09-14T06:30:42.564 INFO:tasks.ceph.osd.6.smithi138.stderr:2024-09-14T06:30:42.563+0000 7f8cd8849640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33620) UID: 0 2024-09-14T06:30:42.607 INFO:teuthology.orchestra.run.smithi138.stdout:ERROR: (22) Invalid argument 2024-09-14T06:30:42.607 INFO:teuthology.orchestra.run.smithi138.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:30:42.616 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-14T06:30:42.694 INFO:teuthology.orchestra.run.smithi094.stderr:noscrub is unset 2024-09-14T06:30:42.701 INFO:tasks.ceph.osd.5.smithi138.stderr:2024-09-14T06:30:42.700+0000 7fe13746f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33618) UID: 0 2024-09-14T06:30:42.762 INFO:teuthology.orchestra.run.smithi138.stdout:ERROR: (22) Invalid argument 2024-09-14T06:30:42.762 INFO:teuthology.orchestra.run.smithi138.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:30:42.765 INFO:tasks.ceph.osd.4.smithi138.stderr:2024-09-14T06:30:42.764+0000 7f6c653c8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33615) UID: 0 2024-09-14T06:30:42.771 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-14T06:30:42.819 INFO:teuthology.orchestra.run.smithi094.stdout:checking clone #24, image image_client.0-clone24 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent25 2024-09-14T06:30:42.867 INFO:tasks.ceph.osd.7.smithi138.stderr:2024-09-14T06:30:42.866+0000 7f3c6a21d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33623) UID: 0 2024-09-14T06:30:42.906 INFO:teuthology.orchestra.run.smithi094.stdout:ERROR: (22) Invalid argument 2024-09-14T06:30:42.906 INFO:teuthology.orchestra.run.smithi094.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:30:42.917 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-14T06:30:42.968 INFO:tasks.ceph.osd.1.smithi094.stderr:2024-09-14T06:30:42.966+0000 7f328e47a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33641) UID: 0 2024-09-14T06:30:43.024 INFO:teuthology.orchestra.run.smithi094.stdout:ERROR: (22) Invalid argument 2024-09-14T06:30:43.024 INFO:teuthology.orchestra.run.smithi094.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:30:43.033 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-14T06:30:43.050 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-14T06:30:43.051 INFO:teuthology.orchestra.run.smithi094.stdout:3472 write 0x853179 thru 0x85c056 (0x8ede bytes) 2024-09-14T06:30:43.056 INFO:teuthology.orchestra.run.smithi094.stdout:3474 read 0x2655fae thru 0x265fa95 (0x9ae8 bytes) 2024-09-14T06:30:43.056 INFO:teuthology.orchestra.run.smithi094.stdout:3476 read 0x9ecb2ad thru 0x9ed43a3 (0x90f7 bytes) 2024-09-14T06:30:43.057 INFO:teuthology.orchestra.run.smithi094.stdout:3477 punch from 0x9606ac0 to 0x960b828, (0x4d68 bytes) 2024-09-14T06:30:43.057 INFO:teuthology.orchestra.run.smithi094.stdout:3479 read 0x579d29f thru 0x57a9738 (0xc49a bytes) 2024-09-14T06:30:43.057 INFO:teuthology.orchestra.run.smithi094.stdout:3480 read 0x6c9268a thru 0x6c9a0e0 (0x7a57 bytes) 2024-09-14T06:30:43.067 INFO:teuthology.orchestra.run.smithi094.stdout:3481 write 0xe584ab1 thru 0xe5901a4 (0xb6f4 bytes) 2024-09-14T06:30:43.068 INFO:tasks.ceph.osd.3.smithi094.stderr:2024-09-14T06:30:43.067+0000 7feeeafe8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33655) UID: 0 2024-09-14T06:30:43.071 INFO:teuthology.orchestra.run.smithi094.stdout:3482 write 0x7e243a1 thru 0x7e34286 (0xfee6 bytes) 2024-09-14T06:30:43.074 INFO:teuthology.orchestra.run.smithi094.stdout:3483 write 0x731d945 thru 0x7327595 (0x9c51 bytes) 2024-09-14T06:30:43.078 INFO:teuthology.orchestra.run.smithi094.stdout:3484 writesame 0xcd5a049 thru 0xcd5a23f data_size 0x1(0x1f7 bytes) 2024-09-14T06:30:43.081 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-14T06:30:43.081 INFO:teuthology.orchestra.run.smithi094.stdout:3486 writesame 0x50dec88 thru 0x50e6550 data_size 0xb(0x78c9 bytes) 2024-09-14T06:30:43.096 INFO:teuthology.orchestra.run.smithi094.stdout:3487 trunc from 0xe5901a5 to 0x5f4dd5f 2024-09-14T06:30:43.128 INFO:teuthology.orchestra.run.smithi094.stdout:3488 trunc from 0x5f4dd5f to 0x565fc5b 2024-09-14T06:30:43.148 INFO:teuthology.orchestra.run.smithi094.stdout:3489 write 0xc00af96 thru 0xc01586b (0xa8d6 bytes) 2024-09-14T06:30:43.151 INFO:teuthology.orchestra.run.smithi094.stdout:3490 write 0x270d520 thru 0x2714cda (0x77bb bytes) 2024-09-14T06:30:43.157 INFO:teuthology.orchestra.run.smithi094.stdout:3491 trunc from 0xc01586c to 0x9bd4ac3 2024-09-14T06:30:43.169 INFO:tasks.ceph.osd.4.smithi138.stderr:2024-09-14T06:30:43.168+0000 7f6c653c8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33615) UID: 0 2024-09-14T06:30:43.174 INFO:teuthology.orchestra.run.smithi094.stdout:ERROR: (22) Invalid argument 2024-09-14T06:30:43.174 INFO:teuthology.orchestra.run.smithi094.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:30:43.176 INFO:teuthology.orchestra.run.smithi094.stdout:3493 writesame 0xa735f3f thru 0xa73ddef data_size 0x13(0x7eb1 bytes) 2024-09-14T06:30:43.176 INFO:teuthology.orchestra.run.smithi094.stdout:3494 read 0x530c60b thru 0x53119a9 (0x539f bytes) 2024-09-14T06:30:43.176 INFO:teuthology.orchestra.run.smithi094.stdout:3495 write 0x7915410 thru 0x79236ec (0xe2dd bytes) 2024-09-14T06:30:43.181 INFO:teuthology.orchestra.run.smithi094.stdout:3497 trunc from 0xa73ddf0 to 0xb6ea8e6 2024-09-14T06:30:43.185 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-14T06:30:43.187 INFO:teuthology.orchestra.run.smithi094.stdout:3498 punch from 0x8b9a9cf to 0x8b9b2eb, (0x91c bytes) 2024-09-14T06:30:43.187 INFO:teuthology.orchestra.run.smithi094.stdout:3499 punch from 0xa9fe833 to 0xaa08964, (0xa131 bytes) 2024-09-14T06:30:43.189 INFO:teuthology.orchestra.run.smithi094.stdout:3500 trunc from 0xb6ea8e6 to 0xd41baf6 2024-09-14T06:30:43.194 INFO:teuthology.orchestra.run.smithi094.stdout:3501 write 0x1b7426e thru 0x1b74cc9 (0xa5c bytes) 2024-09-14T06:30:43.196 INFO:teuthology.orchestra.run.smithi094.stdout:3503 write 0xd0715e2 thru 0xd07919b (0x7bba bytes) 2024-09-14T06:30:43.200 INFO:teuthology.orchestra.run.smithi094.stdout:3504 punch from 0x160b67f to 0x160ec56, (0x35d7 bytes) 2024-09-14T06:30:43.200 INFO:teuthology.orchestra.run.smithi094.stdout:3505 punch from 0x266ed7f to 0x267c5c6, (0xd847 bytes) 2024-09-14T06:30:43.200 INFO:teuthology.orchestra.run.smithi094.stdout:3506 write 0x76b8ff1 thru 0x76c64c4 (0xd4d4 bytes) 2024-09-14T06:30:43.204 INFO:teuthology.orchestra.run.smithi094.stdout:3508 write 0xa50dde2 thru 0xa51c78f (0xe9ae bytes) 2024-09-14T06:30:43.207 INFO:teuthology.orchestra.run.smithi094.stdout:3509 writesame 0x7d6d935 thru 0x7d7b590 data_size 0x12(0xdc5c bytes) 2024-09-14T06:30:43.211 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-14T06:30:43.211 INFO:teuthology.orchestra.run.smithi094.stdout:3511 writesame 0xc3e9ffb thru 0xc3f0cfb data_size 0x1(0x6d01 bytes) 2024-09-14T06:30:43.226 INFO:teuthology.orchestra.run.smithi094.stdout:3512 clone 27 order 19 su 32768 sc 3 2024-09-14T06:30:43.269 INFO:tasks.ceph.osd.3.smithi094.stderr:2024-09-14T06:30:43.268+0000 7feeeafe8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33655) UID: 0 2024-09-14T06:30:43.291 INFO:teuthology.orchestra.run.smithi094.stdout:ERROR: (22) Invalid argument 2024-09-14T06:30:43.291 INFO:teuthology.orchestra.run.smithi094.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:30:43.300 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-14T06:30:43.371 INFO:tasks.ceph.osd.7.smithi138.stderr:2024-09-14T06:30:43.370+0000 7f3c6a21d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33623) UID: 0 2024-09-14T06:30:43.443 INFO:teuthology.orchestra.run.smithi094.stdout:ERROR: (22) Invalid argument 2024-09-14T06:30:43.443 INFO:teuthology.orchestra.run.smithi094.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:30:43.451 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-14T06:30:43.452 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-14T06:30:43.472 INFO:tasks.ceph.osd.5.smithi138.stderr:2024-09-14T06:30:43.471+0000 7fe13746f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33618) UID: 0 2024-09-14T06:30:43.573 INFO:tasks.ceph.osd.5.smithi138.stderr:2024-09-14T06:30:43.571+0000 7fe13746f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33618) UID: 0 2024-09-14T06:30:43.599 INFO:teuthology.orchestra.run.smithi094.stdout:ERROR: (22) Invalid argument 2024-09-14T06:30:43.599 INFO:teuthology.orchestra.run.smithi094.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:30:43.609 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-14T06:30:43.615 INFO:teuthology.orchestra.run.smithi094.stdout:osd.0: {} 2024-09-14T06:30:43.615 INFO:teuthology.orchestra.run.smithi094.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-14T06:30:43.634 INFO:teuthology.orchestra.run.smithi094.stdout:osd.1: {} 2024-09-14T06:30:43.634 INFO:teuthology.orchestra.run.smithi094.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-14T06:30:43.650 INFO:teuthology.orchestra.run.smithi094.stdout:osd.2: {} 2024-09-14T06:30:43.650 INFO:teuthology.orchestra.run.smithi094.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-14T06:30:43.665 INFO:teuthology.orchestra.run.smithi094.stdout:osd.3: {} 2024-09-14T06:30:43.666 INFO:teuthology.orchestra.run.smithi094.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-14T06:30:43.673 INFO:tasks.ceph.osd.0.smithi094.stderr:2024-09-14T06:30:43.672+0000 7f07c9333640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33636) UID: 0 2024-09-14T06:30:43.683 INFO:teuthology.orchestra.run.smithi094.stdout:osd.4: {} 2024-09-14T06:30:43.683 INFO:teuthology.orchestra.run.smithi094.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-14T06:30:43.700 INFO:teuthology.orchestra.run.smithi094.stdout:osd.5: {} 2024-09-14T06:30:43.700 INFO:teuthology.orchestra.run.smithi094.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-14T06:30:43.719 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-14T06:30:43.719 INFO:teuthology.orchestra.run.smithi094.stdout: "ops": [], 2024-09-14T06:30:43.719 INFO:teuthology.orchestra.run.smithi094.stdout: "num_ops": 0 2024-09-14T06:30:43.719 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-14T06:30:43.720 INFO:teuthology.orchestra.run.smithi094.stdout:osd.6: {} 2024-09-14T06:30:43.720 INFO:teuthology.orchestra.run.smithi094.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-14T06:30:43.728 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-14T06:30:43.738 INFO:teuthology.orchestra.run.smithi094.stdout:osd.7: {} 2024-09-14T06:30:43.739 INFO:teuthology.orchestra.run.smithi094.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-14T06:30:43.791 INFO:tasks.ceph.osd.0.smithi094.stderr:2024-09-14T06:30:43.790+0000 7f07c9333640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33636) UID: 0 2024-09-14T06:30:43.842 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-14T06:30:43.842 INFO:teuthology.orchestra.run.smithi094.stdout: "ops": [], 2024-09-14T06:30:43.842 INFO:teuthology.orchestra.run.smithi094.stdout: "complaint_time": 30, 2024-09-14T06:30:43.842 INFO:teuthology.orchestra.run.smithi094.stdout: "num_blocked_ops": 0 2024-09-14T06:30:43.843 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-14T06:30:43.851 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-14T06:30:43.892 INFO:tasks.ceph.osd.0.smithi094.stderr:2024-09-14T06:30:43.891+0000 7f07c9333640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33636) UID: 0 2024-09-14T06:30:43.975 INFO:tasks.ceph.osd.7.smithi138.stderr:2024-09-14T06:30:43.974+0000 7f3c6a21d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33623) UID: 0 2024-09-14T06:30:43.995 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-14T06:30:43.995 INFO:teuthology.orchestra.run.smithi094.stdout: "size": 20, 2024-09-14T06:30:43.995 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 600, 2024-09-14T06:30:43.995 INFO:teuthology.orchestra.run.smithi094.stdout: "ops": [ 2024-09-14T06:30:43.995 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:43.995 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4397.0:1848 3.4 e32/23 3:27620a21:::rbd_data.112dcdddac74.000000000000000b:head v 32'40, mlcod=30'38)", 2024-09-14T06:30:43.995 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:08.658649+0000", 2024-09-14T06:30:43.995 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 95.334797780000002, 2024-09-14T06:30:43.996 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.001862867, 2024-09-14T06:30:43.996 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:30:43.996 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:30:43.996 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:30:43.996 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:30:43.996 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:43.996 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:30:43.996 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.658649+0000", 2024-09-14T06:30:43.996 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:43.996 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:43.996 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:43.996 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:30:43.997 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.658649+0000", 2024-09-14T06:30:43.997 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:43.997 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:43.997 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:43.997 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:30:43.997 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.658651+0000", 2024-09-14T06:30:43.997 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.5650000000000001e-06 2024-09-14T06:30:43.997 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:43.997 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:43.997 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:30:43.997 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.658709+0000", 2024-09-14T06:30:43.997 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.8442000000000001e-05 2024-09-14T06:30:43.997 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:43.997 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:43.998 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:30:43.998 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.658730+0000", 2024-09-14T06:30:43.998 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.0509999999999998e-05 2024-09-14T06:30:43.998 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:43.998 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:43.998 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:30:43.998 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.658740+0000", 2024-09-14T06:30:43.998 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.8619999999999998e-06 2024-09-14T06:30:43.998 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:43.998 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:43.998 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:30:43.998 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.658813+0000", 2024-09-14T06:30:43.998 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.3578000000000006e-05 2024-09-14T06:30:43.999 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:43.999 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:43.999 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:30:43.999 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.658836+0000", 2024-09-14T06:30:43.999 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.2677e-05 2024-09-14T06:30:43.999 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:43.999 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:43.999 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-14T06:30:43.999 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.660462+0000", 2024-09-14T06:30:43.999 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00162637 2024-09-14T06:30:43.999 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:43.999 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:43.999 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:30:44.000 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.660512+0000", 2024-09-14T06:30:44.000 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.9863e-05 2024-09-14T06:30:44.000 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:44.000 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:44.000 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:44.000 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:44.000 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:44.000 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4397.0:2199 3.0 e32/23 3:05f3e980:::rbd_data.112dcdddac74.000000000000000f:head v 32'3, mlcod=31'1)", 2024-09-14T06:30:44.000 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:08.680236+0000", 2024-09-14T06:30:44.000 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 95.313211414999998, 2024-09-14T06:30:44.000 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0019068990000000001, 2024-09-14T06:30:44.000 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:30:44.000 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:30:44.000 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:30:44.001 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:30:44.001 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:44.001 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:30:44.001 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.680236+0000", 2024-09-14T06:30:44.001 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:44.001 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:44.001 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:44.001 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:30:44.001 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.680236+0000", 2024-09-14T06:30:44.001 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:44.001 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:44.001 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:44.001 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:30:44.002 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.680237+0000", 2024-09-14T06:30:44.002 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.4980000000000001e-06 2024-09-14T06:30:44.002 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:44.002 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:44.002 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:30:44.002 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.680324+0000", 2024-09-14T06:30:44.002 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.7310000000000003e-05 2024-09-14T06:30:44.002 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:44.002 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:44.002 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:30:44.002 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.680354+0000", 2024-09-14T06:30:44.002 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.97e-05 2024-09-14T06:30:44.002 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:44.002 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:44.002 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:30:44.003 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.680366+0000", 2024-09-14T06:30:44.003 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.2048999999999999e-05 2024-09-14T06:30:44.003 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:44.003 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:44.003 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:30:44.003 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.680439+0000", 2024-09-14T06:30:44.003 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.2910000000000005e-05 2024-09-14T06:30:44.003 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:44.003 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:44.003 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:30:44.003 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.680465+0000", 2024-09-14T06:30:44.003 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.5812999999999998e-05 2024-09-14T06:30:44.003 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:44.003 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:44.004 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-14T06:30:44.004 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.682094+0000", 2024-09-14T06:30:44.004 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.001628876 2024-09-14T06:30:44.004 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:44.004 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:44.004 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:30:44.004 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.682142+0000", 2024-09-14T06:30:44.004 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.8742999999999997e-05 2024-09-14T06:30:44.004 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:44.004 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:44.004 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:44.004 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:44.004 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:44.005 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4397.0:3292 3.9 e32/23 3:91dbce57:::rbd_data.112dcdddac74.0000000000000036:head v 32'10, mlcod=32'8)", 2024-09-14T06:30:44.005 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:08.740506+0000", 2024-09-14T06:30:44.005 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 95.252940702999993, 2024-09-14T06:30:44.005 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.001864166, 2024-09-14T06:30:44.005 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:30:44.005 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:30:44.005 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:30:44.005 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:30:44.005 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:44.005 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:30:44.005 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.740506+0000", 2024-09-14T06:30:44.005 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:44.005 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:44.005 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:44.005 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:30:44.006 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.740506+0000", 2024-09-14T06:30:44.006 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:44.006 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:44.006 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:44.006 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:30:44.006 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.740514+0000", 2024-09-14T06:30:44.006 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.2109999999999998e-06 2024-09-14T06:30:44.006 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:44.006 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:44.006 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:30:44.006 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.740745+0000", 2024-09-14T06:30:44.006 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000230194 2024-09-14T06:30:44.006 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:44.006 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:44.006 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:30:44.006 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.740764+0000", 2024-09-14T06:30:44.007 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.9330999999999998e-05 2024-09-14T06:30:44.007 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:44.007 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:44.007 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:30:44.007 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.740774+0000", 2024-09-14T06:30:44.007 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.6260000000000007e-06 2024-09-14T06:30:44.007 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:44.007 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:44.007 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:30:44.007 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.740839+0000", 2024-09-14T06:30:44.007 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.5314000000000005e-05 2024-09-14T06:30:44.007 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:44.007 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:44.007 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:30:44.008 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.740860+0000", 2024-09-14T06:30:44.008 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.0868000000000001e-05 2024-09-14T06:30:44.008 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:44.008 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:44.008 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-14T06:30:44.008 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.742327+0000", 2024-09-14T06:30:44.008 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0014672909999999999 2024-09-14T06:30:44.008 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:44.008 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:44.008 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:30:44.008 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:08.742370+0000", 2024-09-14T06:30:44.008 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.3331000000000002e-05 2024-09-14T06:30:44.008 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:44.008 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:44.009 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:44.009 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:44.009 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:44.009 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4397.0:26415 3.0 e53/23 3:05fd9be0:::rbd_data.112d38768680.0000000000000006:head v 53'21, mlcod=50'19)", 2024-09-14T06:30:44.009 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:30.721610+0000", 2024-09-14T06:30:44.009 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 73.271836633999996, 2024-09-14T06:30:44.009 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0021763160000000002, 2024-09-14T06:30:44.009 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:30:44.009 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:30:44.009 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:30:44.010 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:30:44.010 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:44.010 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:30:44.010 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:30.721610+0000", 2024-09-14T06:30:44.010 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:44.010 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:44.010 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:44.010 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:30:44.010 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:30.721610+0000", 2024-09-14T06:30:44.010 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:44.010 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:44.010 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:44.010 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:30:44.010 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:30.721611+0000", 2024-09-14T06:30:44.011 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.41e-07 2024-09-14T06:30:44.011 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:44.011 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:44.011 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:30:44.011 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:30.721905+0000", 2024-09-14T06:30:44.011 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00029394300000000001 2024-09-14T06:30:44.011 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:44.011 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:44.011 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:30:44.011 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:30.721921+0000", 2024-09-14T06:30:44.011 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.6609999999999999e-05 2024-09-14T06:30:44.011 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:44.011 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:44.011 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:30:44.011 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:30.721931+0000", 2024-09-14T06:30:44.011 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.6739999999999992e-06 2024-09-14T06:30:44.012 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:44.012 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:44.012 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:30:44.012 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:30.722001+0000", 2024-09-14T06:30:44.012 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.0549000000000005e-05 2024-09-14T06:30:44.012 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:44.012 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:44.012 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:30:44.012 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:30.722029+0000", 2024-09-14T06:30:44.012 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.7327000000000001e-05 2024-09-14T06:30:44.012 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:44.012 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:44.012 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-14T06:30:44.012 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:30.723727+0000", 2024-09-14T06:30:44.013 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0016977310000000001 2024-09-14T06:30:44.013 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:44.013 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:44.013 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:30:44.013 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:30.723787+0000", 2024-09-14T06:30:44.013 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.0140999999999999e-05 2024-09-14T06:30:44.013 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:44.013 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:44.013 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:44.013 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:44.013 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:44.013 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4397.0:36084 3.0 e56/23 3:05a48ef4:::rbd_data.112dc58639fa.0000000000000058:head v 56'29, mlcod=56'27)", 2024-09-14T06:30:44.013 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:34.508855+0000", 2024-09-14T06:30:44.013 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 69.484591879999996, 2024-09-14T06:30:44.014 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0020801140000000001, 2024-09-14T06:30:44.014 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:30:44.014 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:30:44.014 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:30:44.014 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:30:44.014 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:44.014 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:30:44.014 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:34.508855+0000", 2024-09-14T06:30:44.014 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:44.014 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:44.014 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:44.014 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:30:44.014 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:34.508855+0000", 2024-09-14T06:30:44.014 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:44.015 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:44.015 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:44.015 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:30:44.015 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:34.508856+0000", 2024-09-14T06:30:44.015 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.1909999999999999e-06 2024-09-14T06:30:44.015 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:44.015 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:44.015 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:30:44.015 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:34.509203+0000", 2024-09-14T06:30:44.015 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00034625900000000002 2024-09-14T06:30:44.015 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:44.015 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:44.015 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:30:44.016 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:34.509218+0000", 2024-09-14T06:30:44.016 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.5091e-05 2024-09-14T06:30:44.016 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:44.016 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:44.016 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:30:44.016 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:34.509227+0000", 2024-09-14T06:30:44.016 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.0850000000000003e-06 2024-09-14T06:30:44.016 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:44.016 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:44.016 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:30:44.016 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:34.509311+0000", 2024-09-14T06:30:44.016 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.4436000000000003e-05 2024-09-14T06:30:44.016 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:44.016 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:44.017 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:30:44.017 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:34.509335+0000", 2024-09-14T06:30:44.017 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.3577000000000001e-05 2024-09-14T06:30:44.017 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:44.017 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:44.017 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-14T06:30:44.017 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:34.510890+0000", 2024-09-14T06:30:44.017 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0015554259999999999 2024-09-14T06:30:44.017 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:44.017 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:44.017 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:30:44.017 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:34.510935+0000", 2024-09-14T06:30:44.017 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.5049000000000002e-05 2024-09-14T06:30:44.018 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:44.018 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:44.018 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:44.018 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:44.018 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:44.018 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4397.0:36966 3.4 e56/23 3:22008318:::rbd_data.112dc58639fa.0000000000000066:head v 56'56, mlcod=50'54)", 2024-09-14T06:30:44.018 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:34.565614+0000", 2024-09-14T06:30:44.018 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 69.427833444000001, 2024-09-14T06:30:44.018 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0018691929999999999, 2024-09-14T06:30:44.018 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:30:44.018 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:30:44.018 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:30:44.019 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:30:44.019 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:44.019 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:30:44.019 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:34.565614+0000", 2024-09-14T06:30:44.019 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:44.019 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:44.019 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:44.019 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:30:44.019 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:34.565614+0000", 2024-09-14T06:30:44.019 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:44.019 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:44.020 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:44.020 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:30:44.020 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:34.565615+0000", 2024-09-14T06:30:44.020 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.784e-06 2024-09-14T06:30:44.020 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:44.020 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:44.020 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:30:44.020 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:34.565919+0000", 2024-09-14T06:30:44.020 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00030351599999999999 2024-09-14T06:30:44.020 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:44.020 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:44.020 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:30:44.020 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:34.565933+0000", 2024-09-14T06:30:44.021 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.4465e-05 2024-09-14T06:30:44.021 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:44.021 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:44.021 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:30:44.021 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:34.565943+0000", 2024-09-14T06:30:44.021 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.7920000000000004e-06 2024-09-14T06:30:44.021 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:44.021 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:44.021 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:30:44.021 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:34.565999+0000", 2024-09-14T06:30:44.021 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.5643000000000002e-05 2024-09-14T06:30:44.021 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:44.021 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:44.021 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:30:44.022 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:34.566025+0000", 2024-09-14T06:30:44.022 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.6040000000000001e-05 2024-09-14T06:30:44.022 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:44.022 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:44.022 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-14T06:30:44.022 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:34.567430+0000", 2024-09-14T06:30:44.022 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.001405653 2024-09-14T06:30:44.022 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:44.022 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:44.022 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:30:44.022 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:34.567483+0000", 2024-09-14T06:30:44.022 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.2299999999999997e-05 2024-09-14T06:30:44.022 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:44.022 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:44.023 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:44.023 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:44.023 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:44.023 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4397.0:45722 3.9 e64/23 3:91fed098:::rbd_data.112d5a737e72.0000000000000001:head v 64'61, mlcod=62'59)", 2024-09-14T06:30:44.023 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:43.864269+0000", 2024-09-14T06:30:44.023 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 60.129177804999998, 2024-09-14T06:30:44.023 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.003055224, 2024-09-14T06:30:44.023 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:30:44.023 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:30:44.023 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:30:44.023 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:30:44.023 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:44.023 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:30:44.023 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.864269+0000", 2024-09-14T06:30:44.024 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:44.024 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:44.024 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:44.024 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:30:44.024 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.864269+0000", 2024-09-14T06:30:44.024 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:44.024 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:44.024 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:44.024 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:30:44.024 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.864269+0000", 2024-09-14T06:30:44.024 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.41e-07 2024-09-14T06:30:44.024 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:44.024 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:44.024 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:30:44.025 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.865490+0000", 2024-09-14T06:30:44.025 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.001220552 2024-09-14T06:30:44.025 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:44.025 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:44.025 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:30:44.025 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.865506+0000", 2024-09-14T06:30:44.025 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.6263999999999999e-05 2024-09-14T06:30:44.025 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:44.025 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:44.025 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:30:44.025 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.865516+0000", 2024-09-14T06:30:44.025 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.0091999999999999e-05 2024-09-14T06:30:44.025 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:44.026 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:44.026 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:30:44.026 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.865583+0000", 2024-09-14T06:30:44.026 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.6676999999999999e-05 2024-09-14T06:30:44.026 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:44.026 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:44.026 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:30:44.026 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.865610+0000", 2024-09-14T06:30:44.026 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.6529999999999998e-05 2024-09-14T06:30:44.026 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:44.026 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:44.026 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-14T06:30:44.026 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.867281+0000", 2024-09-14T06:30:44.027 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.001671203 2024-09-14T06:30:44.027 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:44.027 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:44.027 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:30:44.027 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.867324+0000", 2024-09-14T06:30:44.027 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.3665000000000003e-05 2024-09-14T06:30:44.027 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:44.027 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:44.027 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:44.027 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:44.027 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:44.027 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4397.0:49627 3.4 e64/23 3:21e30d4b:::rbd_data.112d5a737e72.0000000000000017:head v 64'69, mlcod=61'67)", 2024-09-14T06:30:44.027 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:44.206446+0000", 2024-09-14T06:30:44.027 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 59.787001167, 2024-09-14T06:30:44.028 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0018142220000000001, 2024-09-14T06:30:44.028 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:30:44.028 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:30:44.028 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:30:44.028 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:30:44.028 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:44.028 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:30:44.028 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.206446+0000", 2024-09-14T06:30:44.028 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:44.028 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:44.028 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:44.028 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:30:44.028 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.206446+0000", 2024-09-14T06:30:44.029 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:44.029 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:44.029 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:44.029 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:30:44.029 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.206446+0000", 2024-09-14T06:30:44.029 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.4900000000000001e-07 2024-09-14T06:30:44.029 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:44.029 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:44.029 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:30:44.029 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.206797+0000", 2024-09-14T06:30:44.029 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00035112799999999998 2024-09-14T06:30:44.029 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:44.029 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:44.029 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:30:44.030 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.206816+0000", 2024-09-14T06:30:44.030 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.8977000000000001e-05 2024-09-14T06:30:44.030 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:44.030 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:44.030 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:30:44.030 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.206825+0000", 2024-09-14T06:30:44.030 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.8279999999999992e-06 2024-09-14T06:30:44.030 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:44.030 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:44.030 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:30:44.030 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.206892+0000", 2024-09-14T06:30:44.030 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.6809000000000004e-05 2024-09-14T06:30:44.030 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:44.030 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:44.030 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:30:44.031 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.206915+0000", 2024-09-14T06:30:44.031 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.315e-05 2024-09-14T06:30:44.031 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:44.031 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:44.031 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-14T06:30:44.031 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.208201+0000", 2024-09-14T06:30:44.031 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0012858819999999999 2024-09-14T06:30:44.031 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:44.031 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:44.031 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:30:44.031 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.208260+0000", 2024-09-14T06:30:44.031 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.9098999999999998e-05 2024-09-14T06:30:44.031 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:44.031 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:44.031 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:44.032 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:44.032 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:44.032 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4397.0:50003 3.9 e64/23 3:94f26c4b:::rbd_data.112d5a737e72.000000000000001f:head v 64'62, mlcod=62'60)", 2024-09-14T06:30:44.032 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:44.220413+0000", 2024-09-14T06:30:44.032 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 59.773033488999999, 2024-09-14T06:30:44.032 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0019210430000000001, 2024-09-14T06:30:44.032 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:30:44.032 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:30:44.032 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:30:44.032 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:30:44.032 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:44.032 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:30:44.032 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.220413+0000", 2024-09-14T06:30:44.032 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:44.032 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:44.033 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:44.033 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:30:44.033 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.220413+0000", 2024-09-14T06:30:44.033 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:44.033 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:44.033 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:44.033 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:30:44.033 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.220415+0000", 2024-09-14T06:30:44.033 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.575e-06 2024-09-14T06:30:44.033 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:44.033 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:44.033 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:30:44.033 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.220984+0000", 2024-09-14T06:30:44.034 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000568624 2024-09-14T06:30:44.034 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:44.034 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:44.034 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:30:44.034 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.221007+0000", 2024-09-14T06:30:44.034 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.3112000000000002e-05 2024-09-14T06:30:44.034 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:44.034 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:44.034 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:30:44.034 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.221017+0000", 2024-09-14T06:30:44.034 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.0098999999999999e-05 2024-09-14T06:30:44.034 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:44.034 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:44.034 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:30:44.034 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.221110+0000", 2024-09-14T06:30:44.035 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.3268000000000002e-05 2024-09-14T06:30:44.035 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:44.035 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:44.035 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:30:44.035 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.221131+0000", 2024-09-14T06:30:44.035 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.0905999999999999e-05 2024-09-14T06:30:44.035 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:44.035 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:44.035 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-14T06:30:44.035 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.222290+0000", 2024-09-14T06:30:44.035 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.001159408 2024-09-14T06:30:44.035 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:44.035 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:44.035 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:30:44.036 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.222335+0000", 2024-09-14T06:30:44.036 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.4051000000000002e-05 2024-09-14T06:30:44.036 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:44.036 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:44.036 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:44.036 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:44.036 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:44.036 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4397.0:50006 3.4 e64/23 3:20cfe300:::rbd_data.112d5a737e72.0000000000000019:head v 64'70, mlcod=61'68)", 2024-09-14T06:30:44.036 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:44.224218+0000", 2024-09-14T06:30:44.036 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 59.769228607999999, 2024-09-14T06:30:44.036 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0021104320000000002, 2024-09-14T06:30:44.036 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:30:44.036 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:30:44.036 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:30:44.037 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:30:44.037 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:44.037 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:30:44.037 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.224218+0000", 2024-09-14T06:30:44.037 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:44.037 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:44.037 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:44.037 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:30:44.037 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.224218+0000", 2024-09-14T06:30:44.037 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:44.037 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:44.037 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:44.037 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:30:44.038 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.224219+0000", 2024-09-14T06:30:44.038 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.0049999999999999e-06 2024-09-14T06:30:44.038 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:44.038 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:44.038 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:30:44.038 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.224326+0000", 2024-09-14T06:30:44.038 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000106848 2024-09-14T06:30:44.038 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:44.038 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:44.038 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:30:44.038 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.224345+0000", 2024-09-14T06:30:44.038 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.8587999999999999e-05 2024-09-14T06:30:44.038 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:44.039 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:44.039 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:30:44.039 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.224354+0000", 2024-09-14T06:30:44.039 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.7220000000000005e-06 2024-09-14T06:30:44.039 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:44.039 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:44.039 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:30:44.039 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.224416+0000", 2024-09-14T06:30:44.039 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.2738999999999997e-05 2024-09-14T06:30:44.039 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:44.039 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:44.039 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:30:44.039 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.224441+0000", 2024-09-14T06:30:44.039 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.4408000000000001e-05 2024-09-14T06:30:44.040 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:44.040 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:44.040 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-14T06:30:44.040 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.226295+0000", 2024-09-14T06:30:44.040 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00185428 2024-09-14T06:30:44.040 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:44.040 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:44.040 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:30:44.040 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.226329+0000", 2024-09-14T06:30:44.040 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.3841999999999999e-05 2024-09-14T06:30:44.040 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:44.040 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:44.040 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:44.041 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:44.041 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:44.041 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4397.0:51816 3.4 e64/23 3:21a826f9:::rbd_data.112d5a737e72.000000000000002d:head v 64'72, mlcod=64'70)", 2024-09-14T06:30:44.041 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:44.312072+0000", 2024-09-14T06:30:44.041 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 59.681375135000003, 2024-09-14T06:30:44.041 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0019555240000000002, 2024-09-14T06:30:44.041 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:30:44.041 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:30:44.041 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:30:44.041 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:30:44.041 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:44.041 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:30:44.041 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.312072+0000", 2024-09-14T06:30:44.041 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:44.042 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:44.042 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:44.042 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:30:44.042 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.312072+0000", 2024-09-14T06:30:44.042 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:44.042 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:44.042 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:44.042 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:30:44.042 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.312074+0000", 2024-09-14T06:30:44.042 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.9530000000000002e-06 2024-09-14T06:30:44.042 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:44.042 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:44.043 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:30:44.043 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.312378+0000", 2024-09-14T06:30:44.043 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00030392 2024-09-14T06:30:44.043 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:44.043 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:44.043 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:30:44.043 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.312399+0000", 2024-09-14T06:30:44.043 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.1134000000000001e-05 2024-09-14T06:30:44.043 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:44.043 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:44.043 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:30:44.043 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.312408+0000", 2024-09-14T06:30:44.043 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.2879999999999998e-06 2024-09-14T06:30:44.043 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:44.044 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:44.044 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:30:44.044 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.312494+0000", 2024-09-14T06:30:44.044 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.6280999999999996e-05 2024-09-14T06:30:44.044 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:44.044 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:44.044 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:30:44.044 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.312518+0000", 2024-09-14T06:30:44.044 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.3149000000000001e-05 2024-09-14T06:30:44.044 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:44.044 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:44.044 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-14T06:30:44.044 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.313964+0000", 2024-09-14T06:30:44.044 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.001446754 2024-09-14T06:30:44.044 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:44.045 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:44.045 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:30:44.045 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.314027+0000", 2024-09-14T06:30:44.045 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.3045000000000004e-05 2024-09-14T06:30:44.045 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:44.045 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:44.045 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:44.045 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:44.045 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:44.045 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4397.0:74241 3.0 e73/23 3:06e93fb9:::rbd_data.112d80ca42af.0000000000000000:head v 73'102, mlcod=71'100)", 2024-09-14T06:30:44.045 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:53.876730+0000", 2024-09-14T06:30:44.045 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 50.116716705000002, 2024-09-14T06:30:44.046 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0079470329999999992, 2024-09-14T06:30:44.046 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:30:44.046 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:30:44.046 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:30:44.046 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:30:44.046 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:44.046 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:30:44.046 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:53.876730+0000", 2024-09-14T06:30:44.046 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:44.046 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:44.046 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:44.046 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:30:44.046 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:53.876730+0000", 2024-09-14T06:30:44.046 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:44.046 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:44.047 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:44.047 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:30:44.047 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:53.876731+0000", 2024-09-14T06:30:44.047 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.0500000000000004e-07 2024-09-14T06:30:44.047 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:44.047 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:44.047 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:30:44.047 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:53.879616+0000", 2024-09-14T06:30:44.047 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0028856680000000001 2024-09-14T06:30:44.047 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:44.047 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:44.047 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:30:44.047 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:53.879643+0000", 2024-09-14T06:30:44.047 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.6120999999999999e-05 2024-09-14T06:30:44.048 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:44.048 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:44.048 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:30:44.048 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:53.879654+0000", 2024-09-14T06:30:44.048 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.1586999999999999e-05 2024-09-14T06:30:44.048 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:44.048 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:44.048 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:30:44.048 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:53.879756+0000", 2024-09-14T06:30:44.048 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00010223899999999999 2024-09-14T06:30:44.048 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:44.048 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:44.048 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:30:44.048 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:53.879783+0000", 2024-09-14T06:30:44.049 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.6913000000000001e-05 2024-09-14T06:30:44.049 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:44.049 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:44.049 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-14T06:30:44.049 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:53.884594+0000", 2024-09-14T06:30:44.049 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0048110729999999999 2024-09-14T06:30:44.049 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:44.049 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:44.049 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:30:44.049 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:53.884677+0000", 2024-09-14T06:30:44.049 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.2927000000000001e-05 2024-09-14T06:30:44.049 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:44.049 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:44.049 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:44.050 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:44.050 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:44.050 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4397.0:74243 3.9 e73/23 3:94977f81:::rbd_data.112d80ca42af.0000000000000002:head v 73'94, mlcod=71'92)", 2024-09-14T06:30:44.050 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:53.885156+0000", 2024-09-14T06:30:44.050 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 50.108290787000001, 2024-09-14T06:30:44.050 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0096009860000000006, 2024-09-14T06:30:44.050 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:30:44.050 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:30:44.050 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:30:44.050 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:30:44.050 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:44.050 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:30:44.050 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:53.885156+0000", 2024-09-14T06:30:44.050 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:44.050 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:44.050 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:44.051 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:30:44.051 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:53.885156+0000", 2024-09-14T06:30:44.051 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:44.051 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:44.051 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:44.051 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:30:44.051 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:53.885156+0000", 2024-09-14T06:30:44.051 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.23e-07 2024-09-14T06:30:44.051 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:44.051 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:44.051 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:30:44.051 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:53.889694+0000", 2024-09-14T06:30:44.051 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0045378709999999997 2024-09-14T06:30:44.051 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:44.051 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:44.052 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:30:44.052 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:53.889720+0000", 2024-09-14T06:30:44.052 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.5259e-05 2024-09-14T06:30:44.052 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:44.052 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:44.052 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:30:44.052 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:53.889731+0000", 2024-09-14T06:30:44.052 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.1739000000000001e-05 2024-09-14T06:30:44.052 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:44.052 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:44.052 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:30:44.052 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:53.889822+0000", 2024-09-14T06:30:44.052 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.1200999999999996e-05 2024-09-14T06:30:44.052 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:44.053 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:44.053 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:30:44.053 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:53.889850+0000", 2024-09-14T06:30:44.053 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.7084999999999999e-05 2024-09-14T06:30:44.053 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:44.053 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:44.053 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-14T06:30:44.053 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:53.894703+0000", 2024-09-14T06:30:44.053 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0048532690000000003 2024-09-14T06:30:44.053 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:44.053 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:44.053 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:30:44.053 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:53.894757+0000", 2024-09-14T06:30:44.054 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.4339000000000003e-05 2024-09-14T06:30:44.054 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:44.054 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:44.054 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:44.054 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:44.054 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:44.054 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4397.0:74249 3.0 e73/23 3:009b0225:::rbd_data.112d80ca42af.0000000000000009:head v 73'103, mlcod=71'101)", 2024-09-14T06:30:44.054 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:53.893219+0000", 2024-09-14T06:30:44.054 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 50.100228309000002, 2024-09-14T06:30:44.055 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0065381980000000003, 2024-09-14T06:30:44.055 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:30:44.055 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:30:44.055 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:30:44.055 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:30:44.055 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:44.055 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:30:44.055 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:53.893219+0000", 2024-09-14T06:30:44.055 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:44.055 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:44.055 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:44.055 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:30:44.055 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:53.893219+0000", 2024-09-14T06:30:44.055 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:44.056 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:44.056 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:44.056 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:30:44.056 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:53.893221+0000", 2024-09-14T06:30:44.056 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.4329999999999998e-06 2024-09-14T06:30:44.056 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:44.056 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:44.056 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:30:44.056 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:53.895968+0000", 2024-09-14T06:30:44.056 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.002746746 2024-09-14T06:30:44.056 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:44.056 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:44.056 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:30:44.056 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:53.895988+0000", 2024-09-14T06:30:44.057 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.0227000000000001e-05 2024-09-14T06:30:44.057 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:44.057 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:44.057 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:30:44.057 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:53.896000+0000", 2024-09-14T06:30:44.057 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.2359e-05 2024-09-14T06:30:44.057 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:44.057 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:44.057 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:30:44.057 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:53.896084+0000", 2024-09-14T06:30:44.057 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.3659999999999995e-05 2024-09-14T06:30:44.057 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:44.057 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:44.058 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:30:44.058 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:53.896109+0000", 2024-09-14T06:30:44.058 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.5259999999999999e-05 2024-09-14T06:30:44.058 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:44.058 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:44.058 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-14T06:30:44.058 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:53.899704+0000", 2024-09-14T06:30:44.058 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0035942209999999999 2024-09-14T06:30:44.058 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:44.058 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:44.058 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:30:44.058 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:53.899757+0000", 2024-09-14T06:30:44.058 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.3291999999999997e-05 2024-09-14T06:30:44.058 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:44.058 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:44.059 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:44.059 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:44.059 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:44.059 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4397.0:78263 3.0 e77/23 3:00899b22:::rbd_data.112ddfbec471.0000000000000016:head v 77'104, mlcod=73'102)", 2024-09-14T06:30:44.059 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:57.634578+0000", 2024-09-14T06:30:44.059 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 46.358869257000002, 2024-09-14T06:30:44.059 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.002372454, 2024-09-14T06:30:44.059 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:30:44.059 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:30:44.059 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:30:44.059 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:30:44.059 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:44.059 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:30:44.059 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:57.634578+0000", 2024-09-14T06:30:44.059 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:44.060 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:44.060 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:44.060 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:30:44.060 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:57.634578+0000", 2024-09-14T06:30:44.060 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:44.060 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:44.060 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:44.060 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:30:44.060 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:57.634578+0000", 2024-09-14T06:30:44.060 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.5699999999999998e-07 2024-09-14T06:30:44.060 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:44.060 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:44.060 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:30:44.061 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:57.635347+0000", 2024-09-14T06:30:44.061 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00076848300000000003 2024-09-14T06:30:44.061 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:44.061 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:44.061 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:30:44.061 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:57.635361+0000", 2024-09-14T06:30:44.061 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.4238e-05 2024-09-14T06:30:44.061 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:44.061 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:44.061 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:30:44.061 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:57.635371+0000", 2024-09-14T06:30:44.061 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.0087e-05 2024-09-14T06:30:44.061 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:44.061 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:44.061 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:30:44.062 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:57.635439+0000", 2024-09-14T06:30:44.062 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.7953000000000005e-05 2024-09-14T06:30:44.062 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:44.062 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:44.062 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:30:44.062 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:57.635464+0000", 2024-09-14T06:30:44.062 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.5062000000000002e-05 2024-09-14T06:30:44.062 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:44.062 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:44.062 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-14T06:30:44.062 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:57.636914+0000", 2024-09-14T06:30:44.062 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.001449625 2024-09-14T06:30:44.062 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:44.062 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:44.063 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:30:44.063 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:57.636950+0000", 2024-09-14T06:30:44.063 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.6548999999999999e-05 2024-09-14T06:30:44.063 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:44.063 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:44.063 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:44.063 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:44.063 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:44.063 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4397.0:78604 3.9 e77/23 3:94c62a0e:::rbd_data.112ddfbec471.000000000000002c:head v 77'96, mlcod=73'94)", 2024-09-14T06:30:44.063 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:57.677568+0000", 2024-09-14T06:30:44.063 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 46.315878869000002, 2024-09-14T06:30:44.063 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.002564846, 2024-09-14T06:30:44.063 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:30:44.063 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:30:44.063 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:30:44.064 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:30:44.064 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:44.064 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:30:44.064 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:57.677568+0000", 2024-09-14T06:30:44.064 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:44.064 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:44.064 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:44.064 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:30:44.064 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:57.677568+0000", 2024-09-14T06:30:44.064 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:44.064 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:44.064 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:44.064 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:30:44.064 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:57.677568+0000", 2024-09-14T06:30:44.065 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.8900000000000001e-07 2024-09-14T06:30:44.065 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:44.065 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:44.065 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:30:44.065 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:57.678488+0000", 2024-09-14T06:30:44.065 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00091963899999999998 2024-09-14T06:30:44.065 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:44.065 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:44.065 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:30:44.065 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:57.678500+0000", 2024-09-14T06:30:44.065 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.1902000000000001e-05 2024-09-14T06:30:44.065 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:44.065 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:44.065 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:30:44.066 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:57.678507+0000", 2024-09-14T06:30:44.066 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.7739999999999999e-06 2024-09-14T06:30:44.066 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:44.066 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:44.066 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:30:44.066 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:57.678586+0000", 2024-09-14T06:30:44.066 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.9103000000000005e-05 2024-09-14T06:30:44.066 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:44.066 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:44.066 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:30:44.066 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:57.678614+0000", 2024-09-14T06:30:44.066 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.7912e-05 2024-09-14T06:30:44.066 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:44.066 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:44.066 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-14T06:30:44.067 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:57.680075+0000", 2024-09-14T06:30:44.067 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0014613370000000001 2024-09-14T06:30:44.067 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:44.067 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:44.067 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:30:44.067 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:57.680133+0000", 2024-09-14T06:30:44.067 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.7890000000000003e-05 2024-09-14T06:30:44.067 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:44.067 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:44.067 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:44.067 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:44.067 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:44.067 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4397.0:78641 3.9 e77/23 3:914b72e4:::rbd_data.112ddfbec471.0000000000000030:head v 77'98, mlcod=73'95)", 2024-09-14T06:30:44.068 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:57.682289+0000", 2024-09-14T06:30:44.068 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 46.311158188, 2024-09-14T06:30:44.068 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0020171400000000002, 2024-09-14T06:30:44.068 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:30:44.068 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:30:44.068 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:30:44.068 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:30:44.068 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:44.068 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:30:44.068 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:57.682289+0000", 2024-09-14T06:30:44.068 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:44.068 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:44.068 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:44.069 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:30:44.069 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:57.682289+0000", 2024-09-14T06:30:44.069 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:44.069 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:44.069 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:44.069 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:30:44.069 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:57.682290+0000", 2024-09-14T06:30:44.069 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.0100000000000003e-07 2024-09-14T06:30:44.069 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:44.069 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:44.069 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:30:44.069 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:57.683230+0000", 2024-09-14T06:30:44.069 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00094047599999999999 2024-09-14T06:30:44.070 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:44.070 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:44.070 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:30:44.070 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:57.683243+0000", 2024-09-14T06:30:44.070 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.2926e-05 2024-09-14T06:30:44.070 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:44.070 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:44.070 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:30:44.070 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:57.683250+0000", 2024-09-14T06:30:44.070 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.6109999999999999e-06 2024-09-14T06:30:44.070 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:44.070 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:44.070 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:30:44.070 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:57.683313+0000", 2024-09-14T06:30:44.071 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.2943999999999999e-05 2024-09-14T06:30:44.071 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:44.071 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:44.071 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:30:44.071 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:57.683337+0000", 2024-09-14T06:30:44.071 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.4377999999999999e-05 2024-09-14T06:30:44.071 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:44.071 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:44.071 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-14T06:30:44.071 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:57.684264+0000", 2024-09-14T06:30:44.071 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00092733099999999999 2024-09-14T06:30:44.071 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:44.071 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:44.071 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:30:44.072 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:57.684306+0000", 2024-09-14T06:30:44.072 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.1573000000000003e-05 2024-09-14T06:30:44.072 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:44.072 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:44.072 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:44.072 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:44.072 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:44.072 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4397.0:113444 3.9 e96/23 3:9619bd5a:::rbd_data.112d4f226fa.0000000000000013:head v 96'128, mlcod=90'126)", 2024-09-14T06:30:44.072 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:30:17.730149+0000", 2024-09-14T06:30:44.072 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 26.263298077000002, 2024-09-14T06:30:44.072 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0019322790000000001, 2024-09-14T06:30:44.072 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:30:44.072 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:30:44.073 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:30:44.073 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:30:44.073 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:44.073 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:30:44.073 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.730149+0000", 2024-09-14T06:30:44.073 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:44.073 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:44.073 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:44.073 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:30:44.073 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.730149+0000", 2024-09-14T06:30:44.073 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:44.073 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:44.073 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:44.074 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:30:44.074 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.730149+0000", 2024-09-14T06:30:44.074 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.0499999999999999e-07 2024-09-14T06:30:44.074 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:44.074 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:44.074 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:30:44.074 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.730852+0000", 2024-09-14T06:30:44.074 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00070294699999999999 2024-09-14T06:30:44.074 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:44.074 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:44.074 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:30:44.074 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.730867+0000", 2024-09-14T06:30:44.074 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.5294999999999998e-05 2024-09-14T06:30:44.075 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:44.075 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:44.075 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:30:44.075 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.730877+0000", 2024-09-14T06:30:44.075 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.3519999999999996e-06 2024-09-14T06:30:44.075 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:44.075 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:44.075 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:30:44.075 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.730947+0000", 2024-09-14T06:30:44.075 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.0162999999999999e-05 2024-09-14T06:30:44.075 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:44.075 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:44.075 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:30:44.075 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.730972+0000", 2024-09-14T06:30:44.076 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.4589000000000001e-05 2024-09-14T06:30:44.076 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:44.076 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:44.076 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-14T06:30:44.076 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.732034+0000", 2024-09-14T06:30:44.076 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.001062508 2024-09-14T06:30:44.076 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:44.076 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:44.076 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:30:44.076 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.732081+0000", 2024-09-14T06:30:44.076 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.7120000000000003e-05 2024-09-14T06:30:44.076 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:44.076 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:44.076 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:44.077 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:44.077 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:44.077 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4397.0:124884 3.9 e104/23 3:93946509:::rbd_data.112d97471187.000000000000000d:head v 104'144, mlcod=98'142)", 2024-09-14T06:30:44.077 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:30:25.843253+0000", 2024-09-14T06:30:44.077 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 18.150194051, 2024-09-14T06:30:44.077 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.001767097, 2024-09-14T06:30:44.077 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:30:44.077 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:30:44.077 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:30:44.077 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:30:44.077 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:44.077 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:30:44.078 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:25.843253+0000", 2024-09-14T06:30:44.078 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:44.078 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:44.078 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:44.078 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:30:44.078 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:25.843253+0000", 2024-09-14T06:30:44.078 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:44.078 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:44.078 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:44.078 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:30:44.078 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:25.843253+0000", 2024-09-14T06:30:44.078 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.1400000000000001e-07 2024-09-14T06:30:44.078 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:44.078 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:44.078 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:30:44.079 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:25.843901+0000", 2024-09-14T06:30:44.079 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000647509 2024-09-14T06:30:44.079 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:44.079 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:44.079 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:30:44.079 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:25.843912+0000", 2024-09-14T06:30:44.079 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.0951999999999999e-05 2024-09-14T06:30:44.079 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:44.079 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:44.079 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:30:44.079 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:25.843918+0000", 2024-09-14T06:30:44.079 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.5390000000000004e-06 2024-09-14T06:30:44.079 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:44.079 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:44.080 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:30:44.080 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:25.843981+0000", 2024-09-14T06:30:44.080 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.2859000000000002e-05 2024-09-14T06:30:44.080 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:44.080 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:44.080 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:30:44.080 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:25.844006+0000", 2024-09-14T06:30:44.080 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.5363000000000001e-05 2024-09-14T06:30:44.080 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:44.080 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:44.080 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-14T06:30:44.080 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:25.844982+0000", 2024-09-14T06:30:44.080 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00097514300000000004 2024-09-14T06:30:44.080 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:44.081 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:44.081 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:30:44.081 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:25.845020+0000", 2024-09-14T06:30:44.081 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.8518e-05 2024-09-14T06:30:44.081 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:44.081 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:44.081 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:44.081 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:44.081 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:44.081 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4397.0:126010 3.9 e104/23 3:9120f7ff:::rbd_data.112d97471187.000000000000001b:head v 104'145, mlcod=99'143)", 2024-09-14T06:30:44.081 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:30:25.990254+0000", 2024-09-14T06:30:44.081 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 18.003193379999999, 2024-09-14T06:30:44.081 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0020629699999999999, 2024-09-14T06:30:44.082 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:30:44.082 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:30:44.082 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:30:44.082 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:30:44.082 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:44.082 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:30:44.082 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:25.990254+0000", 2024-09-14T06:30:44.082 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:44.082 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:44.082 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:44.082 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:30:44.082 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:25.990254+0000", 2024-09-14T06:30:44.082 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:44.082 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:44.082 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:44.083 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:30:44.083 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:25.990255+0000", 2024-09-14T06:30:44.083 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.3349999999999999e-06 2024-09-14T06:30:44.083 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:44.083 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:44.083 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:30:44.083 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:25.990966+0000", 2024-09-14T06:30:44.083 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00071085199999999999 2024-09-14T06:30:44.083 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:44.083 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:44.083 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:30:44.083 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:25.990986+0000", 2024-09-14T06:30:44.083 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.0054000000000001e-05 2024-09-14T06:30:44.083 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:44.083 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:44.083 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:30:44.084 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:25.990995+0000", 2024-09-14T06:30:44.084 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.166e-06 2024-09-14T06:30:44.084 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:44.084 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:44.084 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:30:44.084 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:25.991055+0000", 2024-09-14T06:30:44.084 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.0093000000000002e-05 2024-09-14T06:30:44.084 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:44.084 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:44.084 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:30:44.084 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:25.991082+0000", 2024-09-14T06:30:44.084 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.6505000000000001e-05 2024-09-14T06:30:44.084 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:44.084 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:44.084 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-14T06:30:44.084 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:25.992270+0000", 2024-09-14T06:30:44.085 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0011880479999999999 2024-09-14T06:30:44.085 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:44.085 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:44.085 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:30:44.085 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:25.992317+0000", 2024-09-14T06:30:44.085 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.6916999999999998e-05 2024-09-14T06:30:44.085 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:44.085 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:44.085 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:44.085 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:44.085 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:44.085 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-14T06:30:44.087 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-14T06:30:44.087 INFO:tasks.ceph.osd.3.smithi094.stderr:2024-09-14T06:30:44.085+0000 7feeeafe8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33655) UID: 0 2024-09-14T06:30:44.187 INFO:tasks.ceph.osd.4.smithi138.stderr:2024-09-14T06:30:44.186+0000 7f6c653c8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33615) UID: 0 2024-09-14T06:30:44.231 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-14T06:30:44.231 INFO:teuthology.orchestra.run.smithi094.stdout: "ops": [], 2024-09-14T06:30:44.231 INFO:teuthology.orchestra.run.smithi094.stdout: "num_ops": 0 2024-09-14T06:30:44.231 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-14T06:30:44.239 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-14T06:30:44.288 INFO:tasks.ceph.osd.2.smithi094.stderr:2024-09-14T06:30:44.287+0000 7faf3269b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33649) UID: 0 2024-09-14T06:30:44.341 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-14T06:30:44.342 INFO:teuthology.orchestra.run.smithi094.stdout: "ops": [], 2024-09-14T06:30:44.342 INFO:teuthology.orchestra.run.smithi094.stdout: "complaint_time": 30, 2024-09-14T06:30:44.342 INFO:teuthology.orchestra.run.smithi094.stdout: "num_blocked_ops": 0 2024-09-14T06:30:44.342 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-14T06:30:44.350 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-14T06:30:44.391 INFO:tasks.ceph.osd.0.smithi094.stderr:2024-09-14T06:30:44.390+0000 7f07c9333640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33636) UID: 0 2024-09-14T06:30:44.490 INFO:tasks.ceph.osd.1.smithi094.stderr:2024-09-14T06:30:44.489+0000 7f328e47a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33641) UID: 0 2024-09-14T06:30:44.494 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-14T06:30:44.494 INFO:teuthology.orchestra.run.smithi094.stdout: "size": 20, 2024-09-14T06:30:44.494 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 600, 2024-09-14T06:30:44.494 INFO:teuthology.orchestra.run.smithi094.stdout: "ops": [ 2024-09-14T06:30:44.495 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:44.495 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:44425 3.2 3:4603ad7f:::rbd_data.112dca2f6ae1.0000000000000005:17 [read 868352~4096 [fadvise_nocache]] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:30:44.495 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:43.854967+0000", 2024-09-14T06:30:44.495 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 60.637888484000001, 2024-09-14T06:30:44.495 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.063146025999999994, 2024-09-14T06:30:44.495 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:30:44.495 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:30:44.495 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:30:44.495 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:30:44.495 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:30:44.495 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:44.495 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 44425 2024-09-14T06:30:44.495 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:44.496 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:30:44.496 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:44.496 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:30:44.496 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.854967+0000", 2024-09-14T06:30:44.496 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:44.496 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:44.496 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:44.496 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:30:44.496 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.854967+0000", 2024-09-14T06:30:44.496 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:44.496 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:44.496 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:44.496 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:30:44.496 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.854967+0000", 2024-09-14T06:30:44.497 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.16e-07 2024-09-14T06:30:44.497 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:44.497 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:44.497 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:30:44.497 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.854968+0000", 2024-09-14T06:30:44.497 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.006e-06 2024-09-14T06:30:44.497 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:44.497 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:44.497 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:30:44.497 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.854971+0000", 2024-09-14T06:30:44.497 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.9359999999999999e-06 2024-09-14T06:30:44.497 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:44.497 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:44.497 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:30:44.498 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.854974+0000", 2024-09-14T06:30:44.498 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.1840000000000002e-06 2024-09-14T06:30:44.498 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:44.498 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:44.498 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:30:44.498 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.917945+0000", 2024-09-14T06:30:44.498 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.062970925999999997 2024-09-14T06:30:44.498 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:44.498 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:44.498 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:30:44.498 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.918030+0000", 2024-09-14T06:30:44.498 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.5863000000000001e-05 2024-09-14T06:30:44.498 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:44.498 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:44.498 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:30:44.499 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.918113+0000", 2024-09-14T06:30:44.499 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.2894999999999998e-05 2024-09-14T06:30:44.499 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:44.499 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:44.499 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:44.499 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:44.499 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:44.499 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:44426 3.2 3:4603ad7f:::rbd_data.112dca2f6ae1.0000000000000005:17 [read 892928~4096 [fadvise_nocache]] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:30:44.499 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:43.854977+0000", 2024-09-14T06:30:44.499 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 60.637879073999997, 2024-09-14T06:30:44.499 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.063369237999999994, 2024-09-14T06:30:44.499 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:30:44.499 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:30:44.499 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:30:44.500 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:30:44.500 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:30:44.500 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:44.500 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 44426 2024-09-14T06:30:44.500 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:44.500 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:30:44.500 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:44.500 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:30:44.500 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.854977+0000", 2024-09-14T06:30:44.500 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:44.500 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:44.500 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:44.500 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:30:44.501 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.854977+0000", 2024-09-14T06:30:44.501 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:44.501 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:44.501 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:44.501 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:30:44.501 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.854977+0000", 2024-09-14T06:30:44.501 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.1199999999999999e-07 2024-09-14T06:30:44.501 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:44.501 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:44.501 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:30:44.501 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.854980+0000", 2024-09-14T06:30:44.501 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.27e-06 2024-09-14T06:30:44.501 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:44.502 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:44.502 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:30:44.502 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.854983+0000", 2024-09-14T06:30:44.502 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.1650000000000002e-06 2024-09-14T06:30:44.502 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:44.502 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:44.502 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:30:44.502 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.854989+0000", 2024-09-14T06:30:44.502 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.6520000000000003e-06 2024-09-14T06:30:44.502 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:44.502 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:44.502 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:30:44.502 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.918143+0000", 2024-09-14T06:30:44.502 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.063154541999999994 2024-09-14T06:30:44.502 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:44.503 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:44.503 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:30:44.503 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.918209+0000", 2024-09-14T06:30:44.503 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.5529000000000002e-05 2024-09-14T06:30:44.503 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:44.503 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:44.503 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:30:44.503 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.918346+0000", 2024-09-14T06:30:44.503 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000136868 2024-09-14T06:30:44.503 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:44.503 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:44.503 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:44.503 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:44.503 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:44.503 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:44427 3.2 3:4603ad7f:::rbd_data.112dca2f6ae1.0000000000000005:17 [read 917504~4096 [fadvise_nocache]] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:30:44.503 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:43.854992+0000", 2024-09-14T06:30:44.504 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 60.637863428000003, 2024-09-14T06:30:44.504 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.063607792999999996, 2024-09-14T06:30:44.504 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:30:44.504 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:30:44.504 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:30:44.504 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:30:44.504 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:30:44.504 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:44.504 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 44427 2024-09-14T06:30:44.504 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:44.504 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:30:44.504 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:44.504 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:30:44.504 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.854992+0000", 2024-09-14T06:30:44.505 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:44.505 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:44.505 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:44.505 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:30:44.505 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.854992+0000", 2024-09-14T06:30:44.505 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:44.505 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:44.505 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:44.505 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:30:44.505 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.854993+0000", 2024-09-14T06:30:44.505 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.1400000000000001e-07 2024-09-14T06:30:44.505 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:44.505 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:44.505 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:30:44.505 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.854994+0000", 2024-09-14T06:30:44.506 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.091e-06 2024-09-14T06:30:44.506 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:44.506 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:44.506 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:30:44.506 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855001+0000", 2024-09-14T06:30:44.506 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.204e-06 2024-09-14T06:30:44.506 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:44.506 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:44.506 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:30:44.506 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855003+0000", 2024-09-14T06:30:44.506 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.514e-06 2024-09-14T06:30:44.506 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:44.506 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:44.506 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:30:44.506 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.918380+0000", 2024-09-14T06:30:44.507 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.063376738000000002 2024-09-14T06:30:44.507 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:44.507 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:44.507 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:30:44.507 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.918453+0000", 2024-09-14T06:30:44.507 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.3291e-05 2024-09-14T06:30:44.507 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:44.507 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:44.507 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:30:44.507 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.918600+0000", 2024-09-14T06:30:44.507 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00014674100000000001 2024-09-14T06:30:44.507 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:44.507 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:44.507 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:44.508 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:44.508 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:44.508 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:44428 3.2 3:4603ad7f:::rbd_data.112dca2f6ae1.0000000000000005:17 [read 942080~4096 [fadvise_nocache]] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:30:44.508 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:43.855006+0000", 2024-09-14T06:30:44.508 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 60.637849377999999, 2024-09-14T06:30:44.508 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.063846321999999997, 2024-09-14T06:30:44.508 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:30:44.508 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:30:44.508 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:30:44.508 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:30:44.508 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:30:44.508 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:44.508 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 44428 2024-09-14T06:30:44.508 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:44.509 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:30:44.509 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:44.509 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:30:44.509 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855006+0000", 2024-09-14T06:30:44.509 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:44.509 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:44.509 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:44.509 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:30:44.509 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855006+0000", 2024-09-14T06:30:44.509 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:44.509 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:44.509 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:44.509 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:30:44.509 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855007+0000", 2024-09-14T06:30:44.509 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.1199999999999999e-07 2024-09-14T06:30:44.510 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:44.510 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:44.510 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:30:44.510 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855011+0000", 2024-09-14T06:30:44.510 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.9539999999999998e-06 2024-09-14T06:30:44.510 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:44.510 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:44.510 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:30:44.510 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855014+0000", 2024-09-14T06:30:44.510 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.2930000000000001e-06 2024-09-14T06:30:44.510 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:44.510 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:44.510 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:30:44.510 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855016+0000", 2024-09-14T06:30:44.511 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.3039999999999999e-06 2024-09-14T06:30:44.511 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:44.511 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:44.511 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:30:44.511 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.918634+0000", 2024-09-14T06:30:44.511 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.063617928000000004 2024-09-14T06:30:44.511 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:44.511 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:44.511 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:30:44.511 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.918711+0000", 2024-09-14T06:30:44.511 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.6545000000000007e-05 2024-09-14T06:30:44.511 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:44.511 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:44.511 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:30:44.511 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.918853+0000", 2024-09-14T06:30:44.511 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00014208600000000001 2024-09-14T06:30:44.512 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:44.512 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:44.512 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:44.512 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:44.512 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:44.512 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:44429 3.2 3:4603ad7f:::rbd_data.112dca2f6ae1.0000000000000005:17 [read 966656~4096 [fadvise_nocache]] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:30:44.512 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:43.855019+0000", 2024-09-14T06:30:44.512 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 60.637836704999998, 2024-09-14T06:30:44.512 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.064059226999999996, 2024-09-14T06:30:44.512 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:30:44.512 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:30:44.512 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:30:44.512 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:30:44.512 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:30:44.512 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:44.513 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 44429 2024-09-14T06:30:44.513 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:44.513 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:30:44.513 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:44.513 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:30:44.513 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855019+0000", 2024-09-14T06:30:44.513 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:44.513 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:44.513 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:44.513 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:30:44.513 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855019+0000", 2024-09-14T06:30:44.513 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:44.513 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:44.513 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:44.513 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:30:44.513 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855019+0000", 2024-09-14T06:30:44.514 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.28e-07 2024-09-14T06:30:44.514 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:44.514 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:44.514 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:30:44.514 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855020+0000", 2024-09-14T06:30:44.514 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.0330000000000001e-06 2024-09-14T06:30:44.514 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:44.514 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:44.514 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:30:44.514 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855023+0000", 2024-09-14T06:30:44.514 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.79e-06 2024-09-14T06:30:44.514 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:44.514 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:44.514 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:30:44.514 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855025+0000", 2024-09-14T06:30:44.515 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.216e-06 2024-09-14T06:30:44.515 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:44.515 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:44.515 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:30:44.515 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.918886+0000", 2024-09-14T06:30:44.515 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.063861037999999995 2024-09-14T06:30:44.515 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:44.515 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:44.515 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:30:44.515 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.918952+0000", 2024-09-14T06:30:44.515 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.5993000000000003e-05 2024-09-14T06:30:44.515 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:44.515 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:44.516 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:30:44.516 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.919078+0000", 2024-09-14T06:30:44.516 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00012592899999999999 2024-09-14T06:30:44.516 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:44.516 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:44.516 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:44.516 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:44.516 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:44.516 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:44430 3.2 3:4603ad7f:::rbd_data.112dca2f6ae1.0000000000000005:17 [read 991232~4096 [fadvise_nocache]] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:30:44.516 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:43.855028+0000", 2024-09-14T06:30:44.516 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 60.637827694999999, 2024-09-14T06:30:44.516 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.064284346000000006, 2024-09-14T06:30:44.516 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:30:44.516 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:30:44.517 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:30:44.517 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:30:44.517 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:30:44.517 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:44.517 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 44430 2024-09-14T06:30:44.517 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:44.517 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:30:44.517 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:44.517 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:30:44.517 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855028+0000", 2024-09-14T06:30:44.517 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:44.517 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:44.517 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:44.517 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:30:44.518 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855028+0000", 2024-09-14T06:30:44.518 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:44.518 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:44.518 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:44.518 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:30:44.518 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855028+0000", 2024-09-14T06:30:44.518 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.1199999999999999e-07 2024-09-14T06:30:44.518 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:44.518 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:44.518 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:30:44.518 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855029+0000", 2024-09-14T06:30:44.518 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.1100000000000004e-07 2024-09-14T06:30:44.518 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:44.518 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:44.519 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:30:44.519 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855032+0000", 2024-09-14T06:30:44.519 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.7889999999999999e-06 2024-09-14T06:30:44.519 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:44.519 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:44.519 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:30:44.519 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855034+0000", 2024-09-14T06:30:44.519 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.125e-06 2024-09-14T06:30:44.519 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:44.519 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:44.519 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:30:44.519 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.919110+0000", 2024-09-14T06:30:44.519 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.064076413999999998 2024-09-14T06:30:44.519 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:44.520 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:44.520 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:30:44.520 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.919186+0000", 2024-09-14T06:30:44.520 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.5784999999999994e-05 2024-09-14T06:30:44.520 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:44.520 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:44.520 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:30:44.520 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.919312+0000", 2024-09-14T06:30:44.520 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00012611 2024-09-14T06:30:44.520 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:44.520 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:44.520 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:44.520 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:44.520 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:44.521 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:44431 3.2 3:4603ad7f:::rbd_data.112dca2f6ae1.0000000000000005:17 [read 1015808~4096 [fadvise_nocache]] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:30:44.521 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:43.855037+0000", 2024-09-14T06:30:44.521 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 60.637818826999997, 2024-09-14T06:30:44.521 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.064469755000000004, 2024-09-14T06:30:44.521 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:30:44.521 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:30:44.521 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:30:44.521 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:30:44.521 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:30:44.521 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:44.521 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 44431 2024-09-14T06:30:44.521 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:44.521 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:30:44.522 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:44.522 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:30:44.522 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855037+0000", 2024-09-14T06:30:44.522 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:44.522 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:44.522 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:44.522 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:30:44.522 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855037+0000", 2024-09-14T06:30:44.522 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:44.522 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:44.522 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:44.522 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:30:44.522 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855037+0000", 2024-09-14T06:30:44.522 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.1e-07 2024-09-14T06:30:44.523 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:44.523 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:44.523 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:30:44.523 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855038+0000", 2024-09-14T06:30:44.523 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.9400000000000004e-07 2024-09-14T06:30:44.523 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:44.523 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:44.523 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:30:44.523 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855041+0000", 2024-09-14T06:30:44.523 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.9639999999999999e-06 2024-09-14T06:30:44.523 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:44.523 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:44.523 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:30:44.523 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855043+0000", 2024-09-14T06:30:44.524 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.2259999999999999e-06 2024-09-14T06:30:44.524 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:44.524 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:44.524 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:30:44.524 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.919347+0000", 2024-09-14T06:30:44.524 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.064304091999999993 2024-09-14T06:30:44.524 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:44.524 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:44.524 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:30:44.524 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.919393+0000", 2024-09-14T06:30:44.524 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.5340999999999998e-05 2024-09-14T06:30:44.524 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:44.524 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:44.524 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:30:44.525 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.919507+0000", 2024-09-14T06:30:44.525 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000114028 2024-09-14T06:30:44.525 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:44.525 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:44.525 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:44.525 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:44.525 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:44.525 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:44432 3.2 3:4603ad7f:::rbd_data.112dca2f6ae1.0000000000000005:17 [read 1040384~4096 [fadvise_nocache]] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:30:44.525 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:43.855046+0000", 2024-09-14T06:30:44.525 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 60.637809759, 2024-09-14T06:30:44.525 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.064639253999999993, 2024-09-14T06:30:44.525 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:30:44.525 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:30:44.525 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:30:44.526 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:30:44.526 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:30:44.526 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:44.526 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 44432 2024-09-14T06:30:44.526 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:44.526 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:30:44.526 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:44.526 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:30:44.526 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855046+0000", 2024-09-14T06:30:44.526 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:44.526 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:44.526 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:44.526 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:30:44.526 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855046+0000", 2024-09-14T06:30:44.527 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:44.527 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:44.527 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:44.527 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:30:44.527 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855046+0000", 2024-09-14T06:30:44.527 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.05e-07 2024-09-14T06:30:44.527 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:44.527 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:44.527 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:30:44.527 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855047+0000", 2024-09-14T06:30:44.527 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.879999999999999e-07 2024-09-14T06:30:44.527 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:44.527 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:44.527 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:30:44.527 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855050+0000", 2024-09-14T06:30:44.528 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.8459999999999999e-06 2024-09-14T06:30:44.528 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:44.528 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:44.528 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:30:44.528 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855052+0000", 2024-09-14T06:30:44.528 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.136e-06 2024-09-14T06:30:44.528 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:44.528 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:44.528 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:30:44.528 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.919536+0000", 2024-09-14T06:30:44.528 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.064484132 2024-09-14T06:30:44.528 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:44.528 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:44.529 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:30:44.529 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.919611+0000", 2024-09-14T06:30:44.529 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.4407999999999996e-05 2024-09-14T06:30:44.529 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:44.529 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:44.529 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:30:44.529 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.919685+0000", 2024-09-14T06:30:44.529 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.4538999999999999e-05 2024-09-14T06:30:44.529 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:44.529 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:44.529 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:44.529 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:44.529 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:44.529 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:44518 3.2 3:4190d882:::rbd_data.112dca2f6ae1.0000000000000002:17 [read 12288~4096 [fadvise_nocache]] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:30:44.530 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:43.855055+0000", 2024-09-14T06:30:44.530 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 60.637800875000003, 2024-09-14T06:30:44.530 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.064894317000000007, 2024-09-14T06:30:44.530 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:30:44.530 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:30:44.530 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:30:44.530 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:30:44.530 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:30:44.530 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:44.530 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 44518 2024-09-14T06:30:44.530 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:44.530 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:30:44.530 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:44.530 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:30:44.531 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855055+0000", 2024-09-14T06:30:44.531 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:44.531 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:44.531 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:44.531 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:30:44.531 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855055+0000", 2024-09-14T06:30:44.531 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:44.531 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:44.531 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:44.531 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:30:44.531 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855055+0000", 2024-09-14T06:30:44.531 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.0699999999999999e-07 2024-09-14T06:30:44.531 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:44.531 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:44.532 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:30:44.532 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855056+0000", 2024-09-14T06:30:44.532 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.9400000000000004e-07 2024-09-14T06:30:44.532 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:44.532 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:44.532 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:30:44.532 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855059+0000", 2024-09-14T06:30:44.532 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.8870000000000001e-06 2024-09-14T06:30:44.532 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:44.532 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:44.532 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:30:44.532 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855061+0000", 2024-09-14T06:30:44.532 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.1919999999999999e-06 2024-09-14T06:30:44.532 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:44.533 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:44.533 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:30:44.533 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.919719+0000", 2024-09-14T06:30:44.533 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.064658013 2024-09-14T06:30:44.533 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:44.533 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:44.533 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:30:44.533 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.919788+0000", 2024-09-14T06:30:44.533 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.9282000000000005e-05 2024-09-14T06:30:44.533 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:44.533 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:44.533 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:30:44.533 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.919949+0000", 2024-09-14T06:30:44.533 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00016084199999999999 2024-09-14T06:30:44.534 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:44.534 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:44.534 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:44.534 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:44.534 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:44.534 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:44519 3.2 3:4190d882:::rbd_data.112dca2f6ae1.0000000000000002:17 [read 36864~4096 [fadvise_nocache]] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:30:44.534 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:43.855064+0000", 2024-09-14T06:30:44.534 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 60.637791898000003, 2024-09-14T06:30:44.534 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.065081865000000003, 2024-09-14T06:30:44.534 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:30:44.534 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:30:44.534 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:30:44.534 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:30:44.534 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:30:44.534 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:44.534 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 44519 2024-09-14T06:30:44.535 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:44.535 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:30:44.535 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:44.535 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:30:44.535 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855064+0000", 2024-09-14T06:30:44.535 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:44.535 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:44.535 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:44.535 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:30:44.535 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855064+0000", 2024-09-14T06:30:44.535 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:44.535 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:44.535 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:44.535 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:30:44.536 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855064+0000", 2024-09-14T06:30:44.536 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.1e-07 2024-09-14T06:30:44.536 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:44.536 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:44.536 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:30:44.536 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855065+0000", 2024-09-14T06:30:44.536 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.035e-06 2024-09-14T06:30:44.536 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:44.536 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:44.536 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:30:44.536 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855068+0000", 2024-09-14T06:30:44.536 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.8339999999999999e-06 2024-09-14T06:30:44.536 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:44.536 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:44.537 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:30:44.537 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855071+0000", 2024-09-14T06:30:44.537 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.3450000000000002e-06 2024-09-14T06:30:44.537 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:44.537 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:44.537 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:30:44.537 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.919979+0000", 2024-09-14T06:30:44.537 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.064907647999999998 2024-09-14T06:30:44.537 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:44.537 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:44.537 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:30:44.537 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.920026+0000", 2024-09-14T06:30:44.537 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.7595000000000001e-05 2024-09-14T06:30:44.538 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:44.538 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:44.538 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:30:44.538 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.920146+0000", 2024-09-14T06:30:44.538 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00011919800000000001 2024-09-14T06:30:44.538 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:44.538 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:44.538 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:44.538 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:44.538 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:44.538 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:44520 3.2 3:4190d882:::rbd_data.112dca2f6ae1.0000000000000002:17 [read 61440~4096 [fadvise_nocache]] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:30:44.538 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:43.855075+0000", 2024-09-14T06:30:44.538 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 60.637780847000002, 2024-09-14T06:30:44.538 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.065335078000000005, 2024-09-14T06:30:44.539 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:30:44.539 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:30:44.539 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:30:44.539 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:30:44.539 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:30:44.539 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:44.539 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 44520 2024-09-14T06:30:44.539 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:44.539 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:30:44.539 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:44.539 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:30:44.539 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855075+0000", 2024-09-14T06:30:44.539 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:44.539 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:44.539 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:44.540 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:30:44.540 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855075+0000", 2024-09-14T06:30:44.540 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:44.540 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:44.540 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:44.540 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:30:44.540 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855075+0000", 2024-09-14T06:30:44.540 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.1e-07 2024-09-14T06:30:44.540 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:44.540 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:44.540 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:30:44.540 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855076+0000", 2024-09-14T06:30:44.540 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.0839999999999999e-06 2024-09-14T06:30:44.541 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:44.541 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:44.541 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:30:44.541 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855079+0000", 2024-09-14T06:30:44.541 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.9359999999999999e-06 2024-09-14T06:30:44.541 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:44.541 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:44.541 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:30:44.541 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855084+0000", 2024-09-14T06:30:44.541 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.1730000000000003e-06 2024-09-14T06:30:44.541 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:44.541 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:44.541 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:30:44.541 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.920175+0000", 2024-09-14T06:30:44.542 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.065090234999999996 2024-09-14T06:30:44.542 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:44.542 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:44.542 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:30:44.542 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.920250+0000", 2024-09-14T06:30:44.542 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.5807000000000001e-05 2024-09-14T06:30:44.542 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:44.542 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:44.542 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:30:44.542 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.920410+0000", 2024-09-14T06:30:44.542 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00015963300000000001 2024-09-14T06:30:44.542 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:44.542 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:44.542 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:44.543 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:44.543 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:44.543 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:44521 3.2 3:4190d882:::rbd_data.112dca2f6ae1.0000000000000002:17 [read 86016~4096 [fadvise_nocache]] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:30:44.543 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:43.855087+0000", 2024-09-14T06:30:44.543 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 60.637768366000003, 2024-09-14T06:30:44.543 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.065563264999999996, 2024-09-14T06:30:44.543 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:30:44.543 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:30:44.543 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:30:44.543 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:30:44.543 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:30:44.543 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:44.543 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 44521 2024-09-14T06:30:44.543 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:44.543 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:30:44.544 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:44.544 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:30:44.544 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855087+0000", 2024-09-14T06:30:44.544 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:44.544 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:44.544 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:44.544 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:30:44.544 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855087+0000", 2024-09-14T06:30:44.544 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:44.544 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:44.544 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:44.544 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:30:44.544 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855088+0000", 2024-09-14T06:30:44.544 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.3699999999999999e-07 2024-09-14T06:30:44.545 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:44.545 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:44.545 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:30:44.545 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855089+0000", 2024-09-14T06:30:44.545 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.9900000000000009e-07 2024-09-14T06:30:44.545 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:44.545 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:44.545 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:30:44.545 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855091+0000", 2024-09-14T06:30:44.545 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.8940000000000002e-06 2024-09-14T06:30:44.545 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:44.545 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:44.545 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:30:44.546 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855094+0000", 2024-09-14T06:30:44.546 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.351e-06 2024-09-14T06:30:44.546 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:44.546 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:44.546 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:30:44.546 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.920437+0000", 2024-09-14T06:30:44.546 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.065343221000000007 2024-09-14T06:30:44.546 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:44.546 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:44.546 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:30:44.546 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.920481+0000", 2024-09-14T06:30:44.546 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.4431000000000002e-05 2024-09-14T06:30:44.546 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:44.546 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:44.546 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:30:44.546 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.920651+0000", 2024-09-14T06:30:44.547 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00016913200000000001 2024-09-14T06:30:44.547 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:44.547 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:44.547 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:44.547 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:44.547 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:44.547 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:44522 3.2 3:4190d882:::rbd_data.112dca2f6ae1.0000000000000002:17 [read 110592~4096 [fadvise_nocache]] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:30:44.547 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:43.855098+0000", 2024-09-14T06:30:44.547 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 60.637758138999999, 2024-09-14T06:30:44.547 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.065827628999999999, 2024-09-14T06:30:44.547 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:30:44.547 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:30:44.547 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:30:44.547 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:30:44.548 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:30:44.548 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:44.548 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 44522 2024-09-14T06:30:44.548 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:44.548 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:30:44.548 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:44.548 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:30:44.548 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855098+0000", 2024-09-14T06:30:44.548 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:44.548 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:44.548 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:44.548 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:30:44.548 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855098+0000", 2024-09-14T06:30:44.549 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:44.549 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:44.549 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:44.549 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:30:44.549 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855098+0000", 2024-09-14T06:30:44.549 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.1e-07 2024-09-14T06:30:44.549 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:44.549 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:44.549 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:30:44.549 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855099+0000", 2024-09-14T06:30:44.549 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.012e-06 2024-09-14T06:30:44.549 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:44.549 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:44.549 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:30:44.550 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855102+0000", 2024-09-14T06:30:44.550 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.9380000000000001e-06 2024-09-14T06:30:44.550 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:44.550 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:44.550 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:30:44.550 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855108+0000", 2024-09-14T06:30:44.550 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.4339999999999997e-06 2024-09-14T06:30:44.550 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:44.550 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:44.550 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:30:44.550 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.920681+0000", 2024-09-14T06:30:44.550 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.065572943999999994 2024-09-14T06:30:44.550 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:44.550 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:44.551 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:30:44.551 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.920724+0000", 2024-09-14T06:30:44.551 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.2877000000000003e-05 2024-09-14T06:30:44.551 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:44.551 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:44.551 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:30:44.551 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.920925+0000", 2024-09-14T06:30:44.551 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00020121399999999999 2024-09-14T06:30:44.551 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:44.551 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:44.551 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:44.551 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:44.551 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:44.551 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:44523 3.2 3:4190d882:::rbd_data.112dca2f6ae1.0000000000000002:17 [read 135168~4096 [fadvise_nocache]] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:30:44.552 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:43.855111+0000", 2024-09-14T06:30:44.552 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 60.637744417, 2024-09-14T06:30:44.552 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.066028198999999996, 2024-09-14T06:30:44.552 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:30:44.552 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:30:44.552 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:30:44.552 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:30:44.552 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:30:44.552 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:44.552 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 44523 2024-09-14T06:30:44.552 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:44.552 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:30:44.552 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:44.552 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:30:44.553 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855111+0000", 2024-09-14T06:30:44.553 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:44.553 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:44.553 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:44.553 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:30:44.553 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855111+0000", 2024-09-14T06:30:44.553 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:44.553 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:44.553 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:44.553 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:30:44.553 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855112+0000", 2024-09-14T06:30:44.553 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.1199999999999999e-07 2024-09-14T06:30:44.553 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:44.553 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:44.554 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:30:44.554 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855115+0000", 2024-09-14T06:30:44.554 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.3380000000000001e-06 2024-09-14T06:30:44.554 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:44.554 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:44.554 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:30:44.554 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855118+0000", 2024-09-14T06:30:44.554 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.366e-06 2024-09-14T06:30:44.554 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:44.554 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:44.554 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:30:44.554 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855121+0000", 2024-09-14T06:30:44.554 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.8329999999999998e-06 2024-09-14T06:30:44.554 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:44.555 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:44.555 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:30:44.555 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.920956+0000", 2024-09-14T06:30:44.555 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.065835251999999997 2024-09-14T06:30:44.555 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:44.555 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:44.555 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:30:44.555 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.921001+0000", 2024-09-14T06:30:44.555 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.4778999999999999e-05 2024-09-14T06:30:44.555 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:44.555 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:44.555 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:30:44.555 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.921139+0000", 2024-09-14T06:30:44.555 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000138419 2024-09-14T06:30:44.555 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:44.556 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:44.556 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:44.556 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:44.556 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:44.556 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:44524 3.2 3:4190d882:::rbd_data.112dca2f6ae1.0000000000000002:17 [read 159744~4096 [fadvise_nocache]] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:30:44.556 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:43.855124+0000", 2024-09-14T06:30:44.556 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 60.637731438000003, 2024-09-14T06:30:44.556 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.066249261000000004, 2024-09-14T06:30:44.556 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:30:44.556 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:30:44.556 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:30:44.556 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:30:44.556 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:30:44.557 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:44.557 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 44524 2024-09-14T06:30:44.557 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:44.557 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:30:44.557 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:44.557 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:30:44.557 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855124+0000", 2024-09-14T06:30:44.557 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:44.557 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:44.557 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:44.557 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:30:44.557 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855124+0000", 2024-09-14T06:30:44.557 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:44.557 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:44.557 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:44.558 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:30:44.558 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861086+0000", 2024-09-14T06:30:44.558 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0059613829999999998 2024-09-14T06:30:44.558 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:44.558 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:44.558 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:30:44.558 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861100+0000", 2024-09-14T06:30:44.558 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.4644000000000001e-05 2024-09-14T06:30:44.558 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:44.558 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:44.558 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:30:44.558 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861109+0000", 2024-09-14T06:30:44.558 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.541e-06 2024-09-14T06:30:44.558 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:44.559 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:44.559 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:30:44.559 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861117+0000", 2024-09-14T06:30:44.559 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.1049999999999995e-06 2024-09-14T06:30:44.559 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:44.559 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:44.559 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:30:44.559 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.921169+0000", 2024-09-14T06:30:44.559 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.060052405000000003 2024-09-14T06:30:44.559 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:44.559 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:44.559 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:30:44.559 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.921213+0000", 2024-09-14T06:30:44.559 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.3565e-05 2024-09-14T06:30:44.560 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:44.560 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:44.560 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:30:44.560 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.921374+0000", 2024-09-14T06:30:44.560 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00016061800000000001 2024-09-14T06:30:44.560 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:44.560 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:44.560 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:44.560 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:44.560 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:44.560 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:44536 3.2 3:4190d882:::rbd_data.112dca2f6ae1.0000000000000002:17 [read 454656~4096 [fadvise_nocache]] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:30:44.560 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:43.861384+0000", 2024-09-14T06:30:44.560 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 60.631471881000003, 2024-09-14T06:30:44.560 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.063058528000000003, 2024-09-14T06:30:44.561 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:30:44.561 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:30:44.561 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:30:44.561 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:30:44.561 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:30:44.561 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:44.561 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 44536 2024-09-14T06:30:44.561 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:44.561 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:30:44.561 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:44.561 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:30:44.561 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861384+0000", 2024-09-14T06:30:44.561 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:44.561 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:44.562 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:44.562 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:30:44.562 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861384+0000", 2024-09-14T06:30:44.562 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:44.562 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:44.562 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:44.562 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:30:44.562 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861384+0000", 2024-09-14T06:30:44.562 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.4000000000000002e-07 2024-09-14T06:30:44.562 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:44.562 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:44.562 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:30:44.562 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861386+0000", 2024-09-14T06:30:44.563 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.891e-06 2024-09-14T06:30:44.563 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:44.563 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:44.563 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:30:44.563 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861392+0000", 2024-09-14T06:30:44.563 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.9939999999999997e-06 2024-09-14T06:30:44.563 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:44.563 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:44.563 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:30:44.563 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861398+0000", 2024-09-14T06:30:44.563 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.507e-06 2024-09-14T06:30:44.563 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:44.563 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:44.563 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:30:44.564 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.924149+0000", 2024-09-14T06:30:44.564 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.062751135999999999 2024-09-14T06:30:44.564 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:44.564 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:44.564 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:30:44.564 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.924226+0000", 2024-09-14T06:30:44.564 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.7517999999999999e-05 2024-09-14T06:30:44.564 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:44.564 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:44.564 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:30:44.564 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.924442+0000", 2024-09-14T06:30:44.564 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000216042 2024-09-14T06:30:44.564 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:44.564 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:44.565 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:44.565 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:44.565 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:44.565 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:44537 3.2 3:4190d882:::rbd_data.112dca2f6ae1.0000000000000002:17 [read 479232~4096 [fadvise_nocache]] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:30:44.565 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:43.861405+0000", 2024-09-14T06:30:44.565 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 60.631450835000003, 2024-09-14T06:30:44.565 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.063351795000000002, 2024-09-14T06:30:44.565 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:30:44.565 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:30:44.565 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:30:44.565 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:30:44.565 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:30:44.565 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:44.565 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 44537 2024-09-14T06:30:44.566 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:44.566 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:30:44.566 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:44.566 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:30:44.566 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861405+0000", 2024-09-14T06:30:44.566 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:44.566 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:44.566 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:44.566 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:30:44.566 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861405+0000", 2024-09-14T06:30:44.566 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:44.566 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:44.566 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:44.566 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:30:44.566 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861405+0000", 2024-09-14T06:30:44.567 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.9099999999999999e-07 2024-09-14T06:30:44.567 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:44.567 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:44.567 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:30:44.567 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861407+0000", 2024-09-14T06:30:44.567 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.075e-06 2024-09-14T06:30:44.567 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:44.567 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:44.567 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:30:44.567 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861414+0000", 2024-09-14T06:30:44.567 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.4659999999999996e-06 2024-09-14T06:30:44.567 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:44.567 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:44.567 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:30:44.568 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861418+0000", 2024-09-14T06:30:44.568 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.5519999999999998e-06 2024-09-14T06:30:44.568 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:44.568 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:44.568 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:30:44.568 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.924474+0000", 2024-09-14T06:30:44.568 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.063056012999999994 2024-09-14T06:30:44.568 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:44.568 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:44.568 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:30:44.568 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.924556+0000", 2024-09-14T06:30:44.568 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.1181999999999996e-05 2024-09-14T06:30:44.568 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:44.568 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:44.569 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:30:44.569 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.924757+0000", 2024-09-14T06:30:44.569 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00020111599999999999 2024-09-14T06:30:44.569 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:44.569 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:44.569 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:44.569 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:44.569 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:44.569 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:44538 3.2 3:4190d882:::rbd_data.112dca2f6ae1.0000000000000002:17 [read 503808~4096 [fadvise_nocache]] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:30:44.569 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:43.861425+0000", 2024-09-14T06:30:44.569 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 60.631430794000003, 2024-09-14T06:30:44.569 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.063627125000000007, 2024-09-14T06:30:44.569 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:30:44.570 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:30:44.570 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:30:44.570 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:30:44.570 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:30:44.570 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:44.570 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 44538 2024-09-14T06:30:44.570 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:44.570 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:30:44.570 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:44.570 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:30:44.570 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861425+0000", 2024-09-14T06:30:44.570 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:44.570 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:44.570 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:44.571 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:30:44.571 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861425+0000", 2024-09-14T06:30:44.571 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:44.571 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:44.571 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:44.571 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:30:44.571 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861425+0000", 2024-09-14T06:30:44.571 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.9999999999999998e-07 2024-09-14T06:30:44.571 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:44.571 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:44.571 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:30:44.571 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861427+0000", 2024-09-14T06:30:44.571 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.9829999999999999e-06 2024-09-14T06:30:44.571 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:44.572 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:44.572 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:30:44.572 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861433+0000", 2024-09-14T06:30:44.572 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.2889999999999997e-06 2024-09-14T06:30:44.572 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:44.572 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:44.572 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:30:44.572 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861454+0000", 2024-09-14T06:30:44.572 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.1844999999999998e-05 2024-09-14T06:30:44.572 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:44.572 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:44.572 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:30:44.572 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.924786+0000", 2024-09-14T06:30:44.572 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.063331497 2024-09-14T06:30:44.573 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:44.573 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:44.573 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:30:44.573 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.924864+0000", 2024-09-14T06:30:44.573 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.7744000000000006e-05 2024-09-14T06:30:44.573 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:44.573 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:44.573 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:30:44.573 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.925052+0000", 2024-09-14T06:30:44.573 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00018836699999999999 2024-09-14T06:30:44.573 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:44.573 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:44.573 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:44.573 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:44.574 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:44.574 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:44539 3.2 3:4190d882:::rbd_data.112dca2f6ae1.0000000000000002:17 [read 528384~4096 [fadvise_nocache]] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:30:44.574 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:43.861462+0000", 2024-09-14T06:30:44.574 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 60.631393467000002, 2024-09-14T06:30:44.574 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.063855705999999998, 2024-09-14T06:30:44.574 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:30:44.574 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:30:44.574 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:30:44.574 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:30:44.574 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:30:44.574 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:44.574 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 44539 2024-09-14T06:30:44.574 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:44.574 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:30:44.575 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:44.575 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:30:44.575 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861462+0000", 2024-09-14T06:30:44.575 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:44.575 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:44.575 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:44.575 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:30:44.575 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861462+0000", 2024-09-14T06:30:44.575 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:44.575 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:44.575 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:44.575 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:30:44.575 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861463+0000", 2024-09-14T06:30:44.575 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.5200000000000002e-07 2024-09-14T06:30:44.576 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:44.576 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:44.576 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:30:44.576 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861465+0000", 2024-09-14T06:30:44.576 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.5189999999999999e-06 2024-09-14T06:30:44.576 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:44.576 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:44.576 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:30:44.576 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861471+0000", 2024-09-14T06:30:44.576 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.6899999999999997e-06 2024-09-14T06:30:44.576 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:44.576 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:44.576 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:30:44.577 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861475+0000", 2024-09-14T06:30:44.577 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.5730000000000001e-06 2024-09-14T06:30:44.577 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:44.577 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:44.577 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:30:44.577 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.925085+0000", 2024-09-14T06:30:44.577 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.063609902999999995 2024-09-14T06:30:44.577 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:44.577 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:44.577 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:30:44.577 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.925145+0000", 2024-09-14T06:30:44.577 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.9383999999999999e-05 2024-09-14T06:30:44.577 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:44.578 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:44.578 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:30:44.578 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.925318+0000", 2024-09-14T06:30:44.578 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00017318499999999999 2024-09-14T06:30:44.578 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:44.578 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:44.578 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:44.578 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:44.578 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:44.578 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:44540 3.2 3:4190d882:::rbd_data.112dca2f6ae1.0000000000000002:17 [read 552960~4096 [fadvise_nocache]] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:30:44.578 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:43.861483+0000", 2024-09-14T06:30:44.578 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 60.631373035999999, 2024-09-14T06:30:44.578 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.064065681999999999, 2024-09-14T06:30:44.578 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:30:44.578 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:30:44.579 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:30:44.579 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:30:44.579 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:30:44.579 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:44.579 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 44540 2024-09-14T06:30:44.579 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:44.579 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:30:44.579 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:44.579 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:30:44.579 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861483+0000", 2024-09-14T06:30:44.579 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:44.579 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:44.579 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:44.580 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:30:44.580 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861483+0000", 2024-09-14T06:30:44.580 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:44.580 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:44.580 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:44.580 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:30:44.580 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.867079+0000", 2024-09-14T06:30:44.580 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0055966480000000001 2024-09-14T06:30:44.580 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:44.580 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:44.580 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:30:44.580 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.867086+0000", 2024-09-14T06:30:44.580 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.387e-06 2024-09-14T06:30:44.580 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:44.581 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:44.581 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:30:44.581 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.867094+0000", 2024-09-14T06:30:44.581 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.9419999999999995e-06 2024-09-14T06:30:44.581 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:44.581 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:44.581 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:30:44.581 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.867109+0000", 2024-09-14T06:30:44.581 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.5818999999999999e-05 2024-09-14T06:30:44.581 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:44.581 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:44.581 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:30:44.581 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.925354+0000", 2024-09-14T06:30:44.581 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.058244047 2024-09-14T06:30:44.582 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:44.582 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:44.582 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:30:44.582 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.925406+0000", 2024-09-14T06:30:44.582 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.2661999999999998e-05 2024-09-14T06:30:44.582 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:44.582 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:44.582 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:30:44.582 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.925548+0000", 2024-09-14T06:30:44.582 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000142177 2024-09-14T06:30:44.582 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:44.582 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:44.582 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:44.582 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:44.583 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:44.583 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-14T06:30:44.584 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-14T06:30:44.625 INFO:tasks.ceph.osd.3.smithi094.stderr:2024-09-14T06:30:44.624+0000 7feeeafe8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33655) UID: 0 2024-09-14T06:30:44.692 INFO:tasks.ceph.osd.3.smithi094.stderr:2024-09-14T06:30:44.691+0000 7feeeafe8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33655) UID: 0 2024-09-14T06:30:44.698 INFO:teuthology.orchestra.run.smithi138.stdout:{ 2024-09-14T06:30:44.698 INFO:teuthology.orchestra.run.smithi138.stdout: "ops": [], 2024-09-14T06:30:44.698 INFO:teuthology.orchestra.run.smithi138.stdout: "num_ops": 0 2024-09-14T06:30:44.698 INFO:teuthology.orchestra.run.smithi138.stdout:} 2024-09-14T06:30:44.709 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-14T06:30:44.712 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-14T06:30:44.793 INFO:tasks.ceph.osd.1.smithi094.stderr:2024-09-14T06:30:44.792+0000 7f328e47a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33641) UID: 0 2024-09-14T06:30:44.876 INFO:teuthology.orchestra.run.smithi138.stdout:{ 2024-09-14T06:30:44.876 INFO:teuthology.orchestra.run.smithi138.stdout: "ops": [], 2024-09-14T06:30:44.876 INFO:teuthology.orchestra.run.smithi138.stdout: "complaint_time": 30, 2024-09-14T06:30:44.876 INFO:teuthology.orchestra.run.smithi138.stdout: "num_blocked_ops": 0 2024-09-14T06:30:44.876 INFO:teuthology.orchestra.run.smithi138.stdout:} 2024-09-14T06:30:44.890 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-14T06:30:44.935 INFO:tasks.ceph.osd.5.smithi138.stderr:2024-09-14T06:30:44.933+0000 7fe13746f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33618) UID: 0 2024-09-14T06:30:44.994 INFO:tasks.ceph.osd.3.smithi094.stderr:2024-09-14T06:30:44.993+0000 7feeeafe8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33655) UID: 0 2024-09-14T06:30:45.022 INFO:teuthology.orchestra.run.smithi138.stdout:{ 2024-09-14T06:30:45.022 INFO:teuthology.orchestra.run.smithi138.stdout: "size": 20, 2024-09-14T06:30:45.022 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 600, 2024-09-14T06:30:45.022 INFO:teuthology.orchestra.run.smithi138.stdout: "ops": [ 2024-09-14T06:30:45.022 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:45.022 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:19328 3.3 3:c34e5e51:::rbd_data.112d92027590.0000000000000024:f [read 1926144~1024 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:30:45.022 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:27.059768+0000", 2024-09-14T06:30:45.022 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 77.960221211999993, 2024-09-14T06:30:45.022 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046369147999999999, 2024-09-14T06:30:45.022 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:30:45.023 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:30:45.023 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:30:45.023 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:30:45.023 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:30:45.023 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:45.023 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 19328 2024-09-14T06:30:45.023 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:45.023 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:30:45.023 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:45.023 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:30:45.023 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.059768+0000", 2024-09-14T06:30:45.023 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:45.023 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:45.023 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:45.024 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:30:45.024 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.059768+0000", 2024-09-14T06:30:45.024 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:45.024 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:45.024 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:45.024 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:30:45.024 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.059874+0000", 2024-09-14T06:30:45.024 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00010551400000000001 2024-09-14T06:30:45.024 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:45.024 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:45.024 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:30:45.024 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.059877+0000", 2024-09-14T06:30:45.024 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.8279999999999999e-06 2024-09-14T06:30:45.024 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:45.025 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:45.025 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:30:45.025 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.059884+0000", 2024-09-14T06:30:45.025 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.6389999999999992e-06 2024-09-14T06:30:45.025 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:45.025 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:45.025 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:30:45.025 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.059890+0000", 2024-09-14T06:30:45.025 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.49e-06 2024-09-14T06:30:45.025 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:45.025 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:45.025 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:30:45.025 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.105804+0000", 2024-09-14T06:30:45.025 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.045914242000000001 2024-09-14T06:30:45.026 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:45.026 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:45.026 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:30:45.026 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.105886+0000", 2024-09-14T06:30:45.026 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.2305999999999994e-05 2024-09-14T06:30:45.026 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:45.026 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:45.026 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:30:45.026 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.106138+0000", 2024-09-14T06:30:45.026 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.000251129 2024-09-14T06:30:45.026 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:45.026 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:30:45.026 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:45.026 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:45.026 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:45.027 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:19329 3.3 3:c34e5e51:::rbd_data.112d92027590.0000000000000024:f [read 1936384~1024 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:30:45.027 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:27.059899+0000", 2024-09-14T06:30:45.027 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 77.960090688999998, 2024-09-14T06:30:45.027 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046572122000000001, 2024-09-14T06:30:45.027 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:30:45.027 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:30:45.027 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:30:45.027 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:30:45.027 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:30:45.027 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:45.027 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 19329 2024-09-14T06:30:45.027 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:45.027 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:30:45.028 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:45.028 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:30:45.028 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.059899+0000", 2024-09-14T06:30:45.028 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:45.028 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:45.028 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:45.028 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:30:45.028 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.059899+0000", 2024-09-14T06:30:45.028 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:45.028 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:45.028 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:45.028 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:30:45.028 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062501+0000", 2024-09-14T06:30:45.028 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.0026016419999999999 2024-09-14T06:30:45.029 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:45.029 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:45.029 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:30:45.029 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062514+0000", 2024-09-14T06:30:45.029 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.2947e-05 2024-09-14T06:30:45.029 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:45.029 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:45.029 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:30:45.029 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062523+0000", 2024-09-14T06:30:45.029 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.5519999999999993e-06 2024-09-14T06:30:45.029 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:45.029 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:45.029 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:30:45.030 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062534+0000", 2024-09-14T06:30:45.030 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.1416e-05 2024-09-14T06:30:45.030 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:45.030 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:45.030 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:30:45.030 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.106179+0000", 2024-09-14T06:30:45.030 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.043644049999999997 2024-09-14T06:30:45.030 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:45.030 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:45.030 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:30:45.030 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.106262+0000", 2024-09-14T06:30:45.030 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.3178999999999998e-05 2024-09-14T06:30:45.030 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:45.030 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:45.031 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:30:45.031 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.106471+0000", 2024-09-14T06:30:45.031 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00020933599999999999 2024-09-14T06:30:45.031 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:45.031 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:30:45.031 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:45.031 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:45.031 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:45.031 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:19351 3.3 3:c34e5e51:::rbd_data.112d92027590.0000000000000024:f [read 1159168~1024 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:30:45.031 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:27.062766+0000", 2024-09-14T06:30:45.031 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 77.957223463999995, 2024-09-14T06:30:45.031 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046099454999999998, 2024-09-14T06:30:45.031 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:30:45.031 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:30:45.032 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:30:45.032 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:30:45.032 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:30:45.032 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:45.032 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 19351 2024-09-14T06:30:45.032 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:45.032 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:30:45.032 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:45.032 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:30:45.032 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062766+0000", 2024-09-14T06:30:45.032 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:45.032 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:45.032 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:45.033 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:30:45.033 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062766+0000", 2024-09-14T06:30:45.033 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:45.033 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:45.033 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:45.033 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:30:45.033 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062766+0000", 2024-09-14T06:30:45.033 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.6e-07 2024-09-14T06:30:45.033 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:45.033 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:45.033 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:30:45.033 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062767+0000", 2024-09-14T06:30:45.033 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.0070000000000001e-06 2024-09-14T06:30:45.033 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:45.033 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:45.034 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:30:45.034 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062785+0000", 2024-09-14T06:30:45.034 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.7884999999999999e-05 2024-09-14T06:30:45.034 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:45.034 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:45.034 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:30:45.034 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062790+0000", 2024-09-14T06:30:45.034 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.0900000000000004e-06 2024-09-14T06:30:45.034 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:45.034 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:45.034 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:30:45.034 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.108735+0000", 2024-09-14T06:30:45.034 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.045944411999999997 2024-09-14T06:30:45.034 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:45.035 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:45.035 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:30:45.035 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.108809+0000", 2024-09-14T06:30:45.035 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.4157000000000002e-05 2024-09-14T06:30:45.035 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:45.035 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:45.035 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:30:45.035 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.108866+0000", 2024-09-14T06:30:45.035 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.6644000000000002e-05 2024-09-14T06:30:45.035 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:45.035 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:30:45.035 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:45.035 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:45.035 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:45.035 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:19352 3.3 3:c34e5e51:::rbd_data.112d92027590.0000000000000024:f [read 1169408~1024 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:30:45.036 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:27.062798+0000", 2024-09-14T06:30:45.036 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 77.957192053, 2024-09-14T06:30:45.036 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046255900000000003, 2024-09-14T06:30:45.036 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:30:45.036 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:30:45.036 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:30:45.036 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:30:45.036 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:30:45.036 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:45.036 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 19352 2024-09-14T06:30:45.036 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:45.036 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:30:45.036 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:45.036 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:30:45.036 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062798+0000", 2024-09-14T06:30:45.037 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:45.037 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:45.037 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:45.037 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:30:45.037 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062798+0000", 2024-09-14T06:30:45.037 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:45.037 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:45.037 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:45.037 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:30:45.037 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062798+0000", 2024-09-14T06:30:45.037 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.37e-07 2024-09-14T06:30:45.037 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:45.037 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:45.037 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:30:45.037 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062800+0000", 2024-09-14T06:30:45.038 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.0269999999999998e-06 2024-09-14T06:30:45.038 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:45.038 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:45.038 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:30:45.038 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062806+0000", 2024-09-14T06:30:45.038 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.6690000000000003e-06 2024-09-14T06:30:45.038 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:45.038 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:45.038 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:30:45.038 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062810+0000", 2024-09-14T06:30:45.038 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.3449999999999999e-06 2024-09-14T06:30:45.038 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:45.038 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:45.039 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:30:45.039 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.108892+0000", 2024-09-14T06:30:45.039 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046081715000000002 2024-09-14T06:30:45.039 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:45.039 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:45.039 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:30:45.039 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.108940+0000", 2024-09-14T06:30:45.039 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.7815999999999998e-05 2024-09-14T06:30:45.039 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:45.039 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:45.039 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:30:45.039 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.109053+0000", 2024-09-14T06:30:45.039 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.000113791 2024-09-14T06:30:45.039 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:45.040 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:30:45.040 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:45.040 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:45.040 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:45.040 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:19353 3.3 3:c34e5e51:::rbd_data.112d92027590.0000000000000024:f [read 1179648~1024 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:30:45.040 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:27.062816+0000", 2024-09-14T06:30:45.040 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 77.957173534000006, 2024-09-14T06:30:45.040 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046481256999999998, 2024-09-14T06:30:45.040 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:30:45.040 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:30:45.040 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:30:45.041 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:30:45.041 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:30:45.041 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:45.041 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 19353 2024-09-14T06:30:45.041 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:45.041 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:30:45.041 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:45.041 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:30:45.041 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062816+0000", 2024-09-14T06:30:45.041 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:45.041 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:45.041 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:45.041 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:30:45.042 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062816+0000", 2024-09-14T06:30:45.042 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:45.042 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:45.042 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:45.042 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:30:45.042 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062816+0000", 2024-09-14T06:30:45.042 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.2899999999999999e-07 2024-09-14T06:30:45.042 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:45.042 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:45.042 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:30:45.042 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062818+0000", 2024-09-14T06:30:45.042 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.699e-06 2024-09-14T06:30:45.042 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:45.042 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:45.042 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:30:45.043 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062825+0000", 2024-09-14T06:30:45.043 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.878e-06 2024-09-14T06:30:45.043 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:45.043 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:45.043 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:30:45.043 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062830+0000", 2024-09-14T06:30:45.043 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.5410000000000002e-06 2024-09-14T06:30:45.043 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:45.043 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:45.043 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:30:45.043 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.109096+0000", 2024-09-14T06:30:45.043 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046266517 2024-09-14T06:30:45.043 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:45.043 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:45.044 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:30:45.044 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.109190+0000", 2024-09-14T06:30:45.044 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.4000999999999997e-05 2024-09-14T06:30:45.044 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:45.044 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:45.044 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:30:45.044 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.109297+0000", 2024-09-14T06:30:45.044 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.000107292 2024-09-14T06:30:45.044 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:45.044 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:30:45.044 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:45.044 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:45.045 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:45.045 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:19354 3.3 3:c34e5e51:::rbd_data.112d92027590.0000000000000024:f [read 1189888~1024 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:30:45.045 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:27.062842+0000", 2024-09-14T06:30:45.045 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 77.957148079000007, 2024-09-14T06:30:45.045 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046594257, 2024-09-14T06:30:45.045 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:30:45.045 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:30:45.045 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:30:45.045 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:30:45.045 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:30:45.045 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:45.045 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 19354 2024-09-14T06:30:45.045 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:45.046 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:30:45.046 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:45.046 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:30:45.046 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062842+0000", 2024-09-14T06:30:45.046 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:45.046 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:45.046 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:45.046 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:30:45.046 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062842+0000", 2024-09-14T06:30:45.046 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:45.046 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:45.046 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:45.046 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:30:45.046 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062842+0000", 2024-09-14T06:30:45.047 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.7399999999999999e-07 2024-09-14T06:30:45.047 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:45.047 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:45.047 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:30:45.047 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062859+0000", 2024-09-14T06:30:45.047 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.7010000000000001e-05 2024-09-14T06:30:45.047 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:45.047 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:45.047 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:30:45.047 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062866+0000", 2024-09-14T06:30:45.047 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.9990000000000002e-06 2024-09-14T06:30:45.047 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:45.047 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:45.047 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:30:45.048 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062873+0000", 2024-09-14T06:30:45.048 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.8569999999999997e-06 2024-09-14T06:30:45.048 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:45.048 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:45.048 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:30:45.048 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.109335+0000", 2024-09-14T06:30:45.048 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046461943999999998 2024-09-14T06:30:45.048 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:45.048 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:45.048 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:30:45.048 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.109377+0000", 2024-09-14T06:30:45.048 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.2558999999999997e-05 2024-09-14T06:30:45.048 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:45.048 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:45.049 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:30:45.049 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.109436+0000", 2024-09-14T06:30:45.049 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.8514000000000003e-05 2024-09-14T06:30:45.049 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:45.049 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:30:45.049 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:45.049 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:45.049 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:45.049 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:19355 3.3 3:c34e5e51:::rbd_data.112d92027590.0000000000000024:f [read 1200128~1024 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:30:45.049 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:27.062884+0000", 2024-09-14T06:30:45.049 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 77.957105405999997, 2024-09-14T06:30:45.049 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046681650999999998, 2024-09-14T06:30:45.049 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:30:45.050 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:30:45.050 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:30:45.050 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:30:45.050 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:30:45.050 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:45.050 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 19355 2024-09-14T06:30:45.050 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:45.050 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:30:45.050 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:45.050 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:30:45.050 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062884+0000", 2024-09-14T06:30:45.051 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:45.051 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:45.051 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:45.051 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:30:45.051 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062884+0000", 2024-09-14T06:30:45.051 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:45.051 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:45.051 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:45.051 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:30:45.051 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062885+0000", 2024-09-14T06:30:45.051 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.8000000000000001e-07 2024-09-14T06:30:45.051 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:45.051 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:45.051 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:30:45.052 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062886+0000", 2024-09-14T06:30:45.052 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.8920000000000001e-06 2024-09-14T06:30:45.052 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:45.052 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:45.052 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:30:45.052 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062892+0000", 2024-09-14T06:30:45.052 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.84e-06 2024-09-14T06:30:45.052 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:45.052 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:45.052 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:30:45.052 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062899+0000", 2024-09-14T06:30:45.052 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.5339999999999996e-06 2024-09-14T06:30:45.052 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:45.052 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:45.053 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:30:45.053 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.109466+0000", 2024-09-14T06:30:45.053 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046566919999999998 2024-09-14T06:30:45.053 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:45.053 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:45.053 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:30:45.053 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.109510+0000", 2024-09-14T06:30:45.053 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.4153000000000002e-05 2024-09-14T06:30:45.053 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:45.053 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:45.053 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:30:45.053 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.109566+0000", 2024-09-14T06:30:45.053 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.5931999999999999e-05 2024-09-14T06:30:45.054 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:45.054 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:30:45.054 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:45.054 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:45.054 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:45.054 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:19356 3.3 3:c34e5e51:::rbd_data.112d92027590.0000000000000024:f [read 1210368~1024 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:30:45.054 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:27.062908+0000", 2024-09-14T06:30:45.054 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 77.957081969000001, 2024-09-14T06:30:45.054 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046807498000000003, 2024-09-14T06:30:45.054 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:30:45.054 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:30:45.054 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:30:45.054 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:30:45.054 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:30:45.054 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:45.055 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 19356 2024-09-14T06:30:45.055 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:45.055 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:30:45.055 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:45.055 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:30:45.055 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062908+0000", 2024-09-14T06:30:45.055 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:45.055 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:45.055 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:45.055 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:30:45.055 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062908+0000", 2024-09-14T06:30:45.055 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:45.055 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:45.056 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:45.056 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:30:45.056 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062908+0000", 2024-09-14T06:30:45.056 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.5699999999999998e-07 2024-09-14T06:30:45.056 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:45.056 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:45.056 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:30:45.056 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062910+0000", 2024-09-14T06:30:45.056 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.7710000000000001e-06 2024-09-14T06:30:45.056 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:45.056 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:45.057 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:30:45.057 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062920+0000", 2024-09-14T06:30:45.057 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.0654e-05 2024-09-14T06:30:45.057 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:45.057 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:45.057 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:30:45.057 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062931+0000", 2024-09-14T06:30:45.057 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.075e-05 2024-09-14T06:30:45.057 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:45.057 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:45.057 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:30:45.057 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.109595+0000", 2024-09-14T06:30:45.057 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046663713000000002 2024-09-14T06:30:45.057 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:45.057 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:45.058 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:30:45.058 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.109632+0000", 2024-09-14T06:30:45.058 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.7505000000000001e-05 2024-09-14T06:30:45.058 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:45.058 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:45.058 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:30:45.058 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.109715+0000", 2024-09-14T06:30:45.058 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.2748000000000001e-05 2024-09-14T06:30:45.058 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:45.058 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:30:45.058 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:45.058 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:45.058 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:45.058 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:19357 3.3 3:c34e5e51:::rbd_data.112d92027590.0000000000000024:f [read 1220608~1024 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:30:45.059 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:27.062939+0000", 2024-09-14T06:30:45.059 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 77.957050520999999, 2024-09-14T06:30:45.059 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046920508, 2024-09-14T06:30:45.059 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:30:45.059 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:30:45.059 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:30:45.059 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:30:45.059 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:30:45.059 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:45.059 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 19357 2024-09-14T06:30:45.059 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:45.059 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:30:45.059 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:45.059 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:30:45.059 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062939+0000", 2024-09-14T06:30:45.060 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:45.060 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:45.060 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:45.060 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:30:45.060 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062939+0000", 2024-09-14T06:30:45.060 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:45.060 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:45.060 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:45.060 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:30:45.060 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062940+0000", 2024-09-14T06:30:45.060 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.32e-07 2024-09-14T06:30:45.060 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:45.060 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:45.061 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:30:45.061 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062942+0000", 2024-09-14T06:30:45.061 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.1919999999999999e-06 2024-09-14T06:30:45.061 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:45.061 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:45.061 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:30:45.061 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062948+0000", 2024-09-14T06:30:45.061 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.1149999999999999e-06 2024-09-14T06:30:45.061 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:45.061 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:45.061 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:30:45.061 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062960+0000", 2024-09-14T06:30:45.061 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.1895e-05 2024-09-14T06:30:45.062 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:45.062 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:45.062 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:30:45.062 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.109704+0000", 2024-09-14T06:30:45.062 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046744584999999998 2024-09-14T06:30:45.062 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:45.062 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:45.062 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:30:45.062 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.109777+0000", 2024-09-14T06:30:45.062 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.2282999999999999e-05 2024-09-14T06:30:45.062 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:45.062 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:45.062 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:30:45.063 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.109860+0000", 2024-09-14T06:30:45.063 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.3005999999999998e-05 2024-09-14T06:30:45.063 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:45.063 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:30:45.063 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:45.063 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:45.063 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:45.063 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:19358 3.3 3:c34e5e51:::rbd_data.112d92027590.0000000000000024:f [read 1230848~1024 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:30:45.063 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:27.062967+0000", 2024-09-14T06:30:45.063 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 77.957022890999994, 2024-09-14T06:30:45.063 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047144471, 2024-09-14T06:30:45.063 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:30:45.064 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:30:45.064 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:30:45.064 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:30:45.064 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:30:45.064 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:45.064 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 19358 2024-09-14T06:30:45.064 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:45.064 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:30:45.064 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:45.064 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:30:45.064 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062967+0000", 2024-09-14T06:30:45.064 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:45.064 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:45.064 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:45.065 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:30:45.065 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062967+0000", 2024-09-14T06:30:45.065 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:45.065 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:45.065 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:45.065 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:30:45.065 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062967+0000", 2024-09-14T06:30:45.065 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.89e-07 2024-09-14T06:30:45.065 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:45.065 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:45.065 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:30:45.065 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062969+0000", 2024-09-14T06:30:45.066 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.9929999999999998e-06 2024-09-14T06:30:45.066 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:45.066 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:45.066 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:30:45.066 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062975+0000", 2024-09-14T06:30:45.066 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.9030000000000001e-06 2024-09-14T06:30:45.066 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:45.066 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:45.066 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:30:45.066 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062979+0000", 2024-09-14T06:30:45.066 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.2250000000000002e-06 2024-09-14T06:30:45.066 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:45.066 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:45.066 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:30:45.067 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.109934+0000", 2024-09-14T06:30:45.067 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046954410000000002 2024-09-14T06:30:45.067 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:45.067 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:45.067 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:30:45.067 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.110007+0000", 2024-09-14T06:30:45.067 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.3829000000000001e-05 2024-09-14T06:30:45.067 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:45.067 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:45.067 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:30:45.067 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.110111+0000", 2024-09-14T06:30:45.067 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00010372199999999999 2024-09-14T06:30:45.068 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:45.068 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:30:45.068 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:45.068 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:45.068 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:45.068 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:19359 3.3 3:c34e5e51:::rbd_data.112d92027590.0000000000000024:f [read 1241088~1024 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:30:45.068 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:27.062987+0000", 2024-09-14T06:30:45.068 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 77.957002801000002, 2024-09-14T06:30:45.068 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047297182, 2024-09-14T06:30:45.068 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:30:45.068 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:30:45.068 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:30:45.068 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:30:45.068 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:30:45.069 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:45.069 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 19359 2024-09-14T06:30:45.069 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:45.069 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:30:45.069 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:45.069 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:30:45.069 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062987+0000", 2024-09-14T06:30:45.069 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:45.069 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:45.069 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:45.069 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:30:45.069 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062987+0000", 2024-09-14T06:30:45.069 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:45.069 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:45.070 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:45.070 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:30:45.070 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062987+0000", 2024-09-14T06:30:45.070 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.2500000000000001e-07 2024-09-14T06:30:45.070 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:45.070 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:45.070 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:30:45.070 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062989+0000", 2024-09-14T06:30:45.070 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.6080000000000001e-06 2024-09-14T06:30:45.070 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:45.070 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:45.070 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:30:45.070 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062994+0000", 2024-09-14T06:30:45.071 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.5520000000000004e-06 2024-09-14T06:30:45.071 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:45.071 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:45.071 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:30:45.071 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062999+0000", 2024-09-14T06:30:45.071 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.3930000000000001e-06 2024-09-14T06:30:45.071 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:45.071 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:45.071 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:30:45.071 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.110142+0000", 2024-09-14T06:30:45.071 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047143154 2024-09-14T06:30:45.071 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:45.071 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:45.071 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:30:45.071 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.110219+0000", 2024-09-14T06:30:45.072 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.6737999999999996e-05 2024-09-14T06:30:45.072 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:45.072 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:45.072 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:30:45.072 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.110284+0000", 2024-09-14T06:30:45.072 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.5412000000000003e-05 2024-09-14T06:30:45.072 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:45.072 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:30:45.072 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:45.072 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:45.072 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:45.072 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:19360 3.3 3:c34e5e51:::rbd_data.112d92027590.0000000000000024:f [read 1251328~1024 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:30:45.072 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:27.063017+0000", 2024-09-14T06:30:45.072 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 77.956972339999993, 2024-09-14T06:30:45.072 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047453823999999999, 2024-09-14T06:30:45.073 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:30:45.073 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:30:45.073 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:30:45.073 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:30:45.073 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:30:45.073 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:45.073 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 19360 2024-09-14T06:30:45.073 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:45.073 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:30:45.073 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:45.073 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:30:45.073 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.063017+0000", 2024-09-14T06:30:45.073 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:45.073 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:45.074 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:45.074 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:30:45.074 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.063017+0000", 2024-09-14T06:30:45.074 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:45.074 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:45.074 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:45.074 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:30:45.074 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.064977+0000", 2024-09-14T06:30:45.074 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.0019597080000000001 2024-09-14T06:30:45.074 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:45.074 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:45.074 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:30:45.074 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.064987+0000", 2024-09-14T06:30:45.074 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.9620000000000005e-06 2024-09-14T06:30:45.074 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:45.075 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:45.075 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:30:45.075 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065000+0000", 2024-09-14T06:30:45.075 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.3173000000000001e-05 2024-09-14T06:30:45.075 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:45.075 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:45.075 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:30:45.075 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065005+0000", 2024-09-14T06:30:45.075 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.9960000000000001e-06 2024-09-14T06:30:45.075 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:45.075 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:45.075 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:30:45.075 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.110314+0000", 2024-09-14T06:30:45.075 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.04530899 2024-09-14T06:30:45.076 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:45.076 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:45.076 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:30:45.076 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.110398+0000", 2024-09-14T06:30:45.076 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.3539000000000001e-05 2024-09-14T06:30:45.076 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:45.076 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:45.076 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:30:45.076 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.110471+0000", 2024-09-14T06:30:45.076 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.3455999999999996e-05 2024-09-14T06:30:45.076 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:45.076 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:30:45.076 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:45.076 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:45.076 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:45.077 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:19365 3.3 3:c34e5e51:::rbd_data.112d92027590.0000000000000024:f [read 1302528~1024 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:30:45.077 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:27.065058+0000", 2024-09-14T06:30:45.077 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 77.954931669999993, 2024-09-14T06:30:45.077 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046105526000000001, 2024-09-14T06:30:45.077 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:30:45.077 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:30:45.077 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:30:45.077 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:30:45.077 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:30:45.077 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:45.077 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 19365 2024-09-14T06:30:45.077 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:45.077 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:30:45.078 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:45.078 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:30:45.078 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065058+0000", 2024-09-14T06:30:45.078 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:45.078 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:45.078 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:45.078 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:30:45.078 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065058+0000", 2024-09-14T06:30:45.078 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:45.078 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:45.078 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:45.078 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:30:45.078 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065058+0000", 2024-09-14T06:30:45.078 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.1899999999999999e-07 2024-09-14T06:30:45.079 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:45.079 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:45.079 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:30:45.079 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065059+0000", 2024-09-14T06:30:45.079 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.1299999999999998e-07 2024-09-14T06:30:45.079 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:45.079 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:45.079 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:30:45.079 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065063+0000", 2024-09-14T06:30:45.079 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.1099999999999996e-06 2024-09-14T06:30:45.079 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:45.079 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:45.079 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:30:45.079 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065070+0000", 2024-09-14T06:30:45.079 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.973e-06 2024-09-14T06:30:45.080 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:45.080 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:45.080 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:30:45.080 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.111014+0000", 2024-09-14T06:30:45.080 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.045943373000000003 2024-09-14T06:30:45.080 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:45.080 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:45.080 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:30:45.080 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.111073+0000", 2024-09-14T06:30:45.080 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.8987999999999998e-05 2024-09-14T06:30:45.080 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:45.080 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:45.080 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:30:45.080 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.111163+0000", 2024-09-14T06:30:45.081 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.0950000000000002e-05 2024-09-14T06:30:45.081 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:45.081 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:30:45.081 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:45.081 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:45.081 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:45.081 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:19366 3.3 3:c34e5e51:::rbd_data.112d92027590.0000000000000024:f [read 1312768~1024 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:30:45.081 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:27.065074+0000", 2024-09-14T06:30:45.081 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 77.954915819999997, 2024-09-14T06:30:45.081 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046289858000000003, 2024-09-14T06:30:45.081 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:30:45.081 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:30:45.081 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:30:45.081 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:30:45.082 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:30:45.082 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:45.082 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 19366 2024-09-14T06:30:45.082 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:45.082 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:30:45.082 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:45.082 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:30:45.082 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065074+0000", 2024-09-14T06:30:45.082 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:45.082 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:45.082 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:45.082 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:30:45.082 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065074+0000", 2024-09-14T06:30:45.082 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:45.082 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:45.083 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:45.083 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:30:45.083 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065074+0000", 2024-09-14T06:30:45.083 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.3900000000000001e-07 2024-09-14T06:30:45.083 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:45.083 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:45.083 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:30:45.083 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065078+0000", 2024-09-14T06:30:45.083 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.3229999999999999e-06 2024-09-14T06:30:45.083 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:45.083 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:45.083 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:30:45.083 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065082+0000", 2024-09-14T06:30:45.084 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.8299999999999998e-06 2024-09-14T06:30:45.084 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:45.084 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:45.084 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:30:45.084 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065085+0000", 2024-09-14T06:30:45.084 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.8660000000000002e-06 2024-09-14T06:30:45.084 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:45.084 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:45.084 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:30:45.084 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.111270+0000", 2024-09-14T06:30:45.084 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046185035999999999 2024-09-14T06:30:45.084 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:45.084 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:45.085 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:30:45.085 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.111311+0000", 2024-09-14T06:30:45.085 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.0935e-05 2024-09-14T06:30:45.085 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:45.085 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:45.085 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:30:45.085 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.111364+0000", 2024-09-14T06:30:45.085 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.2629e-05 2024-09-14T06:30:45.085 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:45.085 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:30:45.085 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:45.085 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:45.085 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:45.085 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:19367 3.3 3:c34e5e51:::rbd_data.112d92027590.0000000000000024:f [read 1323008~1024 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:30:45.086 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:27.065115+0000", 2024-09-14T06:30:45.086 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 77.954874258999993, 2024-09-14T06:30:45.086 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046407003000000002, 2024-09-14T06:30:45.086 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:30:45.086 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:30:45.086 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:30:45.086 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:30:45.086 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:30:45.086 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:45.086 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 19367 2024-09-14T06:30:45.086 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:45.086 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:30:45.086 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:45.086 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:30:45.086 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065115+0000", 2024-09-14T06:30:45.087 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:45.087 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:45.087 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:45.087 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:30:45.087 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065115+0000", 2024-09-14T06:30:45.087 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:45.087 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:45.087 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:45.087 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:30:45.087 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065116+0000", 2024-09-14T06:30:45.087 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.4999999999999998e-07 2024-09-14T06:30:45.087 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:45.088 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:45.088 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:30:45.088 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065117+0000", 2024-09-14T06:30:45.088 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.189e-06 2024-09-14T06:30:45.088 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:45.088 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:45.088 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:30:45.088 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065122+0000", 2024-09-14T06:30:45.088 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.3709999999999999e-06 2024-09-14T06:30:45.088 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:45.088 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:45.088 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:30:45.088 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065125+0000", 2024-09-14T06:30:45.088 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.8940000000000002e-06 2024-09-14T06:30:45.089 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:45.089 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:45.089 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:30:45.089 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.111402+0000", 2024-09-14T06:30:45.089 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046276645999999998 2024-09-14T06:30:45.089 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:45.089 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:45.089 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:30:45.089 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.111445+0000", 2024-09-14T06:30:45.089 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.3288000000000003e-05 2024-09-14T06:30:45.089 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:45.089 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:45.089 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:30:45.090 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.111522+0000", 2024-09-14T06:30:45.090 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.7164999999999997e-05 2024-09-14T06:30:45.090 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:45.090 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:30:45.090 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:45.090 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:45.090 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:45.090 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:19368 3.3 3:c34e5e51:::rbd_data.112d92027590.0000000000000024:f [read 1333248~1024 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:30:45.090 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:27.065130+0000", 2024-09-14T06:30:45.090 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 77.954859968999997, 2024-09-14T06:30:45.090 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046507275000000001, 2024-09-14T06:30:45.090 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:30:45.091 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:30:45.091 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:30:45.091 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:30:45.091 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:30:45.091 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:45.091 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 19368 2024-09-14T06:30:45.091 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:45.091 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:30:45.091 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:45.091 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:30:45.091 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065130+0000", 2024-09-14T06:30:45.091 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:45.091 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:45.092 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:45.092 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:30:45.092 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065130+0000", 2024-09-14T06:30:45.092 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:45.092 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:45.092 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:45.092 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:30:45.092 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065130+0000", 2024-09-14T06:30:45.092 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.0800000000000001e-07 2024-09-14T06:30:45.092 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:45.092 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:45.092 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:30:45.092 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065131+0000", 2024-09-14T06:30:45.093 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.8700000000000004e-07 2024-09-14T06:30:45.093 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:45.093 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:45.093 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:30:45.093 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065134+0000", 2024-09-14T06:30:45.093 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.1039999999999998e-06 2024-09-14T06:30:45.093 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:45.093 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:45.093 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:30:45.093 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065136+0000", 2024-09-14T06:30:45.093 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.3949999999999999e-06 2024-09-14T06:30:45.093 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:45.093 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:45.094 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:30:45.094 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.111541+0000", 2024-09-14T06:30:45.094 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046404206000000003 2024-09-14T06:30:45.094 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:45.094 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:45.094 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:30:45.094 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.111582+0000", 2024-09-14T06:30:45.094 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.1051999999999999e-05 2024-09-14T06:30:45.094 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:45.094 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:45.094 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:30:45.094 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.111637+0000", 2024-09-14T06:30:45.094 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.5322999999999999e-05 2024-09-14T06:30:45.095 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:45.095 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:30:45.095 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:45.095 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:45.095 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:45.095 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:19369 3.3 3:c34e5e51:::rbd_data.112d92027590.0000000000000024:f [read 1343488~1024 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:30:45.095 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:27.065139+0000", 2024-09-14T06:30:45.095 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 77.954850152000006, 2024-09-14T06:30:45.095 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046682169000000003, 2024-09-14T06:30:45.095 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:30:45.095 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:30:45.095 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:30:45.095 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:30:45.096 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:30:45.096 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:45.096 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 19369 2024-09-14T06:30:45.096 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:45.096 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:30:45.096 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:45.096 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:30:45.096 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065139+0000", 2024-09-14T06:30:45.096 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:45.096 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:45.096 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:45.096 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:30:45.096 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065139+0000", 2024-09-14T06:30:45.097 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:45.097 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:45.097 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:45.097 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:30:45.097 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065140+0000", 2024-09-14T06:30:45.097 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.8299999999999998e-07 2024-09-14T06:30:45.097 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:45.097 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:45.097 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:30:45.097 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065141+0000", 2024-09-14T06:30:45.097 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.3900000000000003e-07 2024-09-14T06:30:45.097 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:45.097 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:45.097 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:30:45.098 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065144+0000", 2024-09-14T06:30:45.098 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.7409999999999998e-06 2024-09-14T06:30:45.098 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:45.098 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:45.098 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:30:45.098 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065147+0000", 2024-09-14T06:30:45.098 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.4150000000000002e-06 2024-09-14T06:30:45.098 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:45.098 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:45.098 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:30:45.098 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.111673+0000", 2024-09-14T06:30:45.098 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046526004000000003 2024-09-14T06:30:45.098 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:45.098 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:45.099 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:30:45.099 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.111749+0000", 2024-09-14T06:30:45.099 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.5741999999999994e-05 2024-09-14T06:30:45.099 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:45.099 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:45.099 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:30:45.099 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.111822+0000", 2024-09-14T06:30:45.099 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.3045000000000003e-05 2024-09-14T06:30:45.099 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:45.099 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:30:45.099 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:45.099 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:45.099 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:45.099 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:19370 3.3 3:c34e5e51:::rbd_data.112d92027590.0000000000000024:f [read 1353728~1024 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:30:45.099 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:27.065150+0000", 2024-09-14T06:30:45.100 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 77.954839422999996, 2024-09-14T06:30:45.100 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046897556999999999, 2024-09-14T06:30:45.100 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:30:45.100 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:30:45.100 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:30:45.100 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:30:45.100 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:30:45.100 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:45.100 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 19370 2024-09-14T06:30:45.100 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:45.100 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:30:45.100 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:45.100 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:30:45.100 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065150+0000", 2024-09-14T06:30:45.101 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:45.101 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:45.101 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:45.101 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:30:45.101 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065150+0000", 2024-09-14T06:30:45.101 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:45.101 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:45.101 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:45.101 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:30:45.101 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065150+0000", 2024-09-14T06:30:45.101 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.0200000000000001e-07 2024-09-14T06:30:45.101 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:45.102 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:45.102 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:30:45.102 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065151+0000", 2024-09-14T06:30:45.102 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.1100000000000004e-07 2024-09-14T06:30:45.102 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:45.102 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:45.102 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:30:45.102 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065154+0000", 2024-09-14T06:30:45.102 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.9639999999999999e-06 2024-09-14T06:30:45.102 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:45.102 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:45.102 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:30:45.102 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065157+0000", 2024-09-14T06:30:45.102 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.4260000000000002e-06 2024-09-14T06:30:45.103 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:45.103 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:45.103 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:30:45.103 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.111861+0000", 2024-09-14T06:30:45.103 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046704268 2024-09-14T06:30:45.103 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:45.103 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:45.103 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:30:45.103 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.111939+0000", 2024-09-14T06:30:45.103 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.8231000000000004e-05 2024-09-14T06:30:45.103 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:45.103 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:45.103 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:30:45.103 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.112048+0000", 2024-09-14T06:30:45.103 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00010855500000000001 2024-09-14T06:30:45.104 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:45.104 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:30:45.104 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:45.104 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:45.104 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:45.104 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:19371 3.3 3:c34e5e51:::rbd_data.112d92027590.0000000000000024:f [read 1363968~1024 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:30:45.104 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:27.065161+0000", 2024-09-14T06:30:45.104 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 77.954828773000003, 2024-09-14T06:30:45.104 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047021944000000003, 2024-09-14T06:30:45.104 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:30:45.104 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:30:45.104 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:30:45.104 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:30:45.104 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:30:45.104 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:45.105 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 19371 2024-09-14T06:30:45.105 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:45.105 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:30:45.105 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:45.105 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:30:45.105 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065161+0000", 2024-09-14T06:30:45.105 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:45.105 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:45.105 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:45.105 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:30:45.105 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065161+0000", 2024-09-14T06:30:45.105 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:45.105 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:45.105 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:45.106 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:30:45.106 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065161+0000", 2024-09-14T06:30:45.106 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.0200000000000001e-07 2024-09-14T06:30:45.106 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:45.106 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:45.106 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:30:45.106 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065166+0000", 2024-09-14T06:30:45.106 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.9869999999999999e-06 2024-09-14T06:30:45.106 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:45.106 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:45.106 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:30:45.106 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065169+0000", 2024-09-14T06:30:45.106 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.3799999999999998e-06 2024-09-14T06:30:45.106 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:45.106 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:45.107 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:30:45.107 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065172+0000", 2024-09-14T06:30:45.107 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.4830000000000002e-06 2024-09-14T06:30:45.107 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:45.107 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:45.107 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:30:45.107 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.112071+0000", 2024-09-14T06:30:45.107 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046899529000000002 2024-09-14T06:30:45.107 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:45.107 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:45.107 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:30:45.107 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.112128+0000", 2024-09-14T06:30:45.107 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.6341999999999997e-05 2024-09-14T06:30:45.107 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:45.108 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:45.108 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:30:45.108 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.112183+0000", 2024-09-14T06:30:45.108 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.5021000000000001e-05 2024-09-14T06:30:45.108 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:45.108 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:30:45.108 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:45.108 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:45.108 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:45.108 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:19372 3.3 3:c34e5e51:::rbd_data.112d92027590.0000000000000024:f [read 1374208~1024 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:30:45.108 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:27.065176+0000", 2024-09-14T06:30:45.108 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 77.954813623000007, 2024-09-14T06:30:45.108 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047188889999999997, 2024-09-14T06:30:45.109 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:30:45.109 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:30:45.109 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:30:45.109 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:30:45.109 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:30:45.109 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:45.109 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 19372 2024-09-14T06:30:45.109 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:45.109 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:30:45.109 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:45.109 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:30:45.109 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065176+0000", 2024-09-14T06:30:45.110 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:45.110 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:45.110 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:45.110 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:30:45.110 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065176+0000", 2024-09-14T06:30:45.110 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:45.110 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:45.110 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:45.110 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:30:45.110 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.071117+0000", 2024-09-14T06:30:45.110 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.0059404990000000001 2024-09-14T06:30:45.111 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:45.111 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:45.111 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:30:45.111 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.071120+0000", 2024-09-14T06:30:45.111 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.4340000000000001e-06 2024-09-14T06:30:45.111 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:45.111 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:45.111 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:30:45.111 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.071129+0000", 2024-09-14T06:30:45.111 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.8389999999999996e-06 2024-09-14T06:30:45.111 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:45.111 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:45.111 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:30:45.112 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.071134+0000", 2024-09-14T06:30:45.112 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.412e-06 2024-09-14T06:30:45.112 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:45.112 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:45.112 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:30:45.112 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.112221+0000", 2024-09-14T06:30:45.112 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.041086473999999998 2024-09-14T06:30:45.112 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:45.112 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:45.112 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:30:45.112 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.112299+0000", 2024-09-14T06:30:45.112 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.8746999999999997e-05 2024-09-14T06:30:45.112 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:45.113 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:45.113 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:30:45.113 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.112365+0000", 2024-09-14T06:30:45.113 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.5485000000000001e-05 2024-09-14T06:30:45.113 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:45.113 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:30:45.113 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:45.113 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:45.113 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:30:45.113 INFO:teuthology.orchestra.run.smithi138.stdout:} 2024-09-14T06:30:45.115 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-14T06:30:45.115 INFO:tasks.ceph.osd.2.smithi094.stderr:2024-09-14T06:30:45.113+0000 7faf3269b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33649) UID: 0 2024-09-14T06:30:45.215 INFO:tasks.ceph.osd.6.smithi138.stderr:2024-09-14T06:30:45.214+0000 7f8cd8849640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33620) UID: 0 2024-09-14T06:30:45.258 INFO:teuthology.orchestra.run.smithi138.stdout:{ 2024-09-14T06:30:45.258 INFO:teuthology.orchestra.run.smithi138.stdout: "ops": [], 2024-09-14T06:30:45.258 INFO:teuthology.orchestra.run.smithi138.stdout: "num_ops": 0 2024-09-14T06:30:45.258 INFO:teuthology.orchestra.run.smithi138.stdout:} 2024-09-14T06:30:45.266 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-14T06:30:45.350 INFO:tasks.ceph.osd.6.smithi138.stderr:2024-09-14T06:30:45.349+0000 7f8cd8849640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33620) UID: 0 2024-09-14T06:30:45.408 INFO:teuthology.orchestra.run.smithi138.stdout:{ 2024-09-14T06:30:45.408 INFO:teuthology.orchestra.run.smithi138.stdout: "ops": [], 2024-09-14T06:30:45.408 INFO:teuthology.orchestra.run.smithi138.stdout: "complaint_time": 30, 2024-09-14T06:30:45.408 INFO:teuthology.orchestra.run.smithi138.stdout: "num_blocked_ops": 0 2024-09-14T06:30:45.408 INFO:teuthology.orchestra.run.smithi138.stdout:} 2024-09-14T06:30:45.416 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-14T06:30:45.418 INFO:tasks.ceph.osd.1.smithi094.stderr:2024-09-14T06:30:45.416+0000 7f328e47a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33641) UID: 0 2024-09-14T06:30:45.519 INFO:tasks.ceph.osd.6.smithi138.stderr:2024-09-14T06:30:45.518+0000 7f8cd8849640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33620) UID: 0 2024-09-14T06:30:45.560 INFO:teuthology.orchestra.run.smithi138.stdout:{ 2024-09-14T06:30:45.561 INFO:teuthology.orchestra.run.smithi138.stdout: "size": 20, 2024-09-14T06:30:45.561 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 600, 2024-09-14T06:30:45.561 INFO:teuthology.orchestra.run.smithi138.stdout: "ops": [ 2024-09-14T06:30:45.561 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:45.561 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:11453 3.1d 3:bcad7303:::rbd_data.112d92027590.0000000000000001:f [read 94208~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:30:45.561 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:25.654375+0000", 2024-09-14T06:30:45.561 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 79.904288418999997, 2024-09-14T06:30:45.561 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052449318000000002, 2024-09-14T06:30:45.561 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:30:45.561 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:30:45.561 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:30:45.561 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:30:45.561 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:30:45.562 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:45.562 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 11453 2024-09-14T06:30:45.562 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:45.562 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:30:45.562 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:45.562 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:30:45.562 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654375+0000", 2024-09-14T06:30:45.562 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:45.562 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:45.562 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:45.562 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:30:45.562 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654375+0000", 2024-09-14T06:30:45.562 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:45.562 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:45.563 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:45.563 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:30:45.563 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654496+0000", 2024-09-14T06:30:45.563 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00012081 2024-09-14T06:30:45.563 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:45.563 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:45.563 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:30:45.563 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654497+0000", 2024-09-14T06:30:45.563 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.8789999999999999e-06 2024-09-14T06:30:45.563 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:45.563 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:45.563 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:30:45.563 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654501+0000", 2024-09-14T06:30:45.564 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.8999999999999999e-06 2024-09-14T06:30:45.564 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:45.564 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:45.564 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:30:45.564 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654505+0000", 2024-09-14T06:30:45.564 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.2229999999999999e-06 2024-09-14T06:30:45.564 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:45.564 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:45.564 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:30:45.564 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.705917+0000", 2024-09-14T06:30:45.564 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.051412501999999999 2024-09-14T06:30:45.564 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:45.564 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:45.564 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:30:45.565 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.705999+0000", 2024-09-14T06:30:45.565 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.1551999999999994e-05 2024-09-14T06:30:45.565 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:45.565 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:45.565 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:30:45.565 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.706824+0000", 2024-09-14T06:30:45.565 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00082545199999999996 2024-09-14T06:30:45.565 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:45.565 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:30:45.565 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:45.565 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:45.565 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:45.566 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:11454 3.1d 3:bcad7303:::rbd_data.112d92027590.0000000000000001:f [read 104448~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:30:45.566 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:25.654510+0000", 2024-09-14T06:30:45.566 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 79.904153629000007, 2024-09-14T06:30:45.566 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052610713000000003, 2024-09-14T06:30:45.566 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:30:45.566 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:30:45.566 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:30:45.566 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:30:45.566 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:30:45.566 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:45.566 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 11454 2024-09-14T06:30:45.566 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:45.566 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:30:45.567 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:45.567 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:30:45.567 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654510+0000", 2024-09-14T06:30:45.567 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:45.567 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:45.567 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:45.567 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:30:45.567 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654510+0000", 2024-09-14T06:30:45.567 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:45.567 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:45.567 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:45.567 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:30:45.568 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654825+0000", 2024-09-14T06:30:45.568 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00031522400000000002 2024-09-14T06:30:45.568 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:45.568 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:45.568 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:30:45.568 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654827+0000", 2024-09-14T06:30:45.568 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.9709999999999998e-06 2024-09-14T06:30:45.568 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:45.568 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:45.568 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:30:45.568 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654832+0000", 2024-09-14T06:30:45.568 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.4099999999999999e-06 2024-09-14T06:30:45.569 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:45.569 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:45.569 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:30:45.569 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654835+0000", 2024-09-14T06:30:45.569 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.14e-06 2024-09-14T06:30:45.569 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:45.569 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:45.569 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:30:45.569 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.706851+0000", 2024-09-14T06:30:45.569 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052015761000000001 2024-09-14T06:30:45.569 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:45.569 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:45.570 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:30:45.570 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.706934+0000", 2024-09-14T06:30:45.570 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.2484999999999994e-05 2024-09-14T06:30:45.570 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:45.570 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:45.570 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:30:45.570 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.707120+0000", 2024-09-14T06:30:45.570 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00018672200000000001 2024-09-14T06:30:45.570 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:45.570 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:30:45.570 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:45.570 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:45.570 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:45.570 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:11455 3.1d 3:bcad7303:::rbd_data.112d92027590.0000000000000001:f [read 114688~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:30:45.571 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:25.654841+0000", 2024-09-14T06:30:45.571 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 79.903822300000002, 2024-09-14T06:30:45.571 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052579898, 2024-09-14T06:30:45.571 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:30:45.571 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:30:45.571 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:30:45.571 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:30:45.571 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:30:45.571 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:45.571 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 11455 2024-09-14T06:30:45.571 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:45.571 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:30:45.571 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:45.572 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:30:45.572 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654841+0000", 2024-09-14T06:30:45.572 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:45.572 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:45.572 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:45.572 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:30:45.572 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654841+0000", 2024-09-14T06:30:45.572 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:45.572 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:45.572 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:45.572 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:30:45.572 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654841+0000", 2024-09-14T06:30:45.572 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.7300000000000002e-07 2024-09-14T06:30:45.572 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:45.573 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:45.573 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:30:45.573 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654845+0000", 2024-09-14T06:30:45.573 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.0520000000000003e-06 2024-09-14T06:30:45.573 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:45.573 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:45.573 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:30:45.573 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654849+0000", 2024-09-14T06:30:45.573 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.0459999999999999e-06 2024-09-14T06:30:45.573 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:45.573 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:45.573 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:30:45.573 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654852+0000", 2024-09-14T06:30:45.573 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.9830000000000001e-06 2024-09-14T06:30:45.574 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:45.574 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:45.574 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:30:45.574 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.707159+0000", 2024-09-14T06:30:45.574 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052306826000000001 2024-09-14T06:30:45.574 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:45.574 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:45.574 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:30:45.574 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.707242+0000", 2024-09-14T06:30:45.574 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.2595000000000005e-05 2024-09-14T06:30:45.574 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:45.574 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:45.574 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:30:45.574 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.707421+0000", 2024-09-14T06:30:45.575 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.000179123 2024-09-14T06:30:45.575 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:45.575 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:30:45.575 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:45.575 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:45.575 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:45.575 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:11456 3.1d 3:bcad7303:::rbd_data.112d92027590.0000000000000001:f [read 124928~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:30:45.575 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:25.654856+0000", 2024-09-14T06:30:45.575 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 79.903807454000003, 2024-09-14T06:30:45.575 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052801580000000001, 2024-09-14T06:30:45.575 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:30:45.575 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:30:45.575 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:30:45.575 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:30:45.576 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:30:45.576 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:45.576 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 11456 2024-09-14T06:30:45.576 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:45.576 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:30:45.576 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:45.576 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:30:45.576 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654856+0000", 2024-09-14T06:30:45.576 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:45.576 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:45.576 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:45.576 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:30:45.576 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654856+0000", 2024-09-14T06:30:45.576 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:45.577 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:45.577 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:45.577 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:30:45.577 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654856+0000", 2024-09-14T06:30:45.577 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.29e-07 2024-09-14T06:30:45.577 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:45.577 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:45.577 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:30:45.577 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654857+0000", 2024-09-14T06:30:45.577 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.187e-06 2024-09-14T06:30:45.577 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:45.577 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:45.577 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:30:45.578 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654861+0000", 2024-09-14T06:30:45.578 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.3850000000000002e-06 2024-09-14T06:30:45.578 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:45.578 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:45.578 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:30:45.578 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654863+0000", 2024-09-14T06:30:45.578 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.9500000000000001e-06 2024-09-14T06:30:45.578 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:45.578 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:45.578 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:30:45.578 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.707456+0000", 2024-09-14T06:30:45.578 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052592434 2024-09-14T06:30:45.578 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:45.578 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:45.579 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:30:45.579 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.707516+0000", 2024-09-14T06:30:45.579 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.0167999999999997e-05 2024-09-14T06:30:45.579 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:45.579 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:45.579 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:30:45.579 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.707657+0000", 2024-09-14T06:30:45.579 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00014122700000000001 2024-09-14T06:30:45.579 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:45.579 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:30:45.579 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:45.579 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:45.579 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:45.579 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:11457 3.1d 3:bcad7303:::rbd_data.112d92027590.0000000000000001:f [read 135168~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:30:45.580 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:25.654867+0000", 2024-09-14T06:30:45.580 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 79.903796568999994, 2024-09-14T06:30:45.580 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052921902, 2024-09-14T06:30:45.580 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:30:45.580 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:30:45.580 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:30:45.580 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:30:45.580 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:30:45.580 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:45.580 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 11457 2024-09-14T06:30:45.580 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:45.580 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:30:45.580 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:45.580 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:30:45.581 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654867+0000", 2024-09-14T06:30:45.581 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:45.581 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:45.581 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:45.581 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:30:45.581 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654867+0000", 2024-09-14T06:30:45.581 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:45.581 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:45.581 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:45.581 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:30:45.581 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654972+0000", 2024-09-14T06:30:45.581 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00010584 2024-09-14T06:30:45.581 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:45.581 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:45.581 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:30:45.582 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654975+0000", 2024-09-14T06:30:45.582 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.1169999999999998e-06 2024-09-14T06:30:45.582 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:45.582 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:45.582 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:30:45.582 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654979+0000", 2024-09-14T06:30:45.582 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.1960000000000001e-06 2024-09-14T06:30:45.582 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:45.582 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:45.582 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:30:45.582 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654982+0000", 2024-09-14T06:30:45.582 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.2990000000000001e-06 2024-09-14T06:30:45.582 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:45.582 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:45.582 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:30:45.582 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.707684+0000", 2024-09-14T06:30:45.583 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052701979000000003 2024-09-14T06:30:45.583 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:45.583 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:45.583 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:30:45.583 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.707728+0000", 2024-09-14T06:30:45.583 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.3529999999999998e-05 2024-09-14T06:30:45.583 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:45.583 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:45.583 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:30:45.583 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.707789+0000", 2024-09-14T06:30:45.583 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.0940999999999998e-05 2024-09-14T06:30:45.583 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:45.583 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:30:45.583 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:45.583 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:45.584 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:45.584 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:11458 3.1d 3:bcad7303:::rbd_data.112d92027590.0000000000000001:f [read 145408~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:30:45.584 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:25.654987+0000", 2024-09-14T06:30:45.584 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 79.903676171000001, 2024-09-14T06:30:45.584 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.05291655, 2024-09-14T06:30:45.584 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:30:45.584 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:30:45.584 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:30:45.584 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:30:45.584 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:30:45.584 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:45.584 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 11458 2024-09-14T06:30:45.584 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:45.584 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:30:45.584 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:45.585 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:30:45.585 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654987+0000", 2024-09-14T06:30:45.585 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:45.585 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:45.585 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:45.585 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:30:45.585 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654987+0000", 2024-09-14T06:30:45.585 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:45.585 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:45.585 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:45.585 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:30:45.585 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655077+0000", 2024-09-14T06:30:45.585 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.0253000000000005e-05 2024-09-14T06:30:45.585 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:45.585 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:45.586 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:30:45.586 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655079+0000", 2024-09-14T06:30:45.586 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.9989999999999998e-06 2024-09-14T06:30:45.586 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:45.586 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:45.586 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:30:45.586 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655083+0000", 2024-09-14T06:30:45.586 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.9280000000000003e-06 2024-09-14T06:30:45.586 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:45.586 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:45.586 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:30:45.586 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655086+0000", 2024-09-14T06:30:45.586 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.056e-06 2024-09-14T06:30:45.586 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:45.586 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:45.587 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:30:45.587 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.707805+0000", 2024-09-14T06:30:45.587 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052718627999999997 2024-09-14T06:30:45.587 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:45.587 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:45.587 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:30:45.587 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.707848+0000", 2024-09-14T06:30:45.587 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.2728999999999997e-05 2024-09-14T06:30:45.587 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:45.587 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:45.587 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:30:45.587 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.707904+0000", 2024-09-14T06:30:45.587 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.5957e-05 2024-09-14T06:30:45.587 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:45.588 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:30:45.588 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:45.588 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:45.588 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:45.588 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:11459 3.1d 3:bcad7303:::rbd_data.112d92027590.0000000000000001:f [read 155648~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:30:45.588 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:25.655091+0000", 2024-09-14T06:30:45.588 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 79.903572163000007, 2024-09-14T06:30:45.588 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052944242000000002, 2024-09-14T06:30:45.588 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:30:45.588 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:30:45.588 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:30:45.588 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:30:45.588 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:30:45.588 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:45.588 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 11459 2024-09-14T06:30:45.589 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:45.589 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:30:45.589 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:45.589 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:30:45.589 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655091+0000", 2024-09-14T06:30:45.589 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:45.589 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:45.589 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:45.589 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:30:45.589 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655091+0000", 2024-09-14T06:30:45.589 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:45.589 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:45.589 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:45.589 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:30:45.589 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655237+0000", 2024-09-14T06:30:45.590 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00014570100000000001 2024-09-14T06:30:45.590 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:45.590 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:45.590 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:30:45.590 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655239+0000", 2024-09-14T06:30:45.590 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.8670000000000001e-06 2024-09-14T06:30:45.590 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:45.590 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:45.590 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:30:45.590 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655243+0000", 2024-09-14T06:30:45.590 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.9829999999999998e-06 2024-09-14T06:30:45.590 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:45.590 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:45.590 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:30:45.591 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655246+0000", 2024-09-14T06:30:45.591 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.1540000000000002e-06 2024-09-14T06:30:45.591 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:45.591 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:45.591 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:30:45.591 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.707934+0000", 2024-09-14T06:30:45.591 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052688286000000001 2024-09-14T06:30:45.591 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:45.591 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:45.591 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:30:45.591 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.707977+0000", 2024-09-14T06:30:45.591 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.3291999999999998e-05 2024-09-14T06:30:45.591 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:45.591 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:45.592 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:30:45.592 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.708035+0000", 2024-09-14T06:30:45.592 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.7958999999999999e-05 2024-09-14T06:30:45.592 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:45.592 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:30:45.592 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:45.592 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:45.592 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:45.592 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:11460 3.1d 3:bcad7303:::rbd_data.112d92027590.0000000000000001:f [read 165888~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:30:45.592 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:25.655250+0000", 2024-09-14T06:30:45.592 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 79.903412865999996, 2024-09-14T06:30:45.592 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052910698999999999, 2024-09-14T06:30:45.592 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:30:45.592 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:30:45.592 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:30:45.593 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:30:45.593 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:30:45.593 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:45.593 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 11460 2024-09-14T06:30:45.593 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:45.593 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:30:45.593 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:45.593 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:30:45.593 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655250+0000", 2024-09-14T06:30:45.593 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:45.593 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:45.593 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:45.593 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:30:45.593 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655250+0000", 2024-09-14T06:30:45.593 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:45.594 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:45.594 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:45.594 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:30:45.594 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655356+0000", 2024-09-14T06:30:45.594 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00010535900000000001 2024-09-14T06:30:45.594 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:45.595 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:45.595 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:30:45.595 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655357+0000", 2024-09-14T06:30:45.595 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.8080000000000001e-06 2024-09-14T06:30:45.596 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:45.596 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:45.596 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:30:45.596 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655361+0000", 2024-09-14T06:30:45.596 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.861e-06 2024-09-14T06:30:45.596 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:45.596 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:45.596 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:30:45.596 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655364+0000", 2024-09-14T06:30:45.596 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.1089999999999998e-06 2024-09-14T06:30:45.596 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:45.596 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:45.596 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:30:45.596 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.708065+0000", 2024-09-14T06:30:45.597 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052700595000000003 2024-09-14T06:30:45.597 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:45.597 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:45.597 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:30:45.597 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.708108+0000", 2024-09-14T06:30:45.597 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.2484999999999997e-05 2024-09-14T06:30:45.597 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:45.597 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:45.597 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:30:45.597 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.708161+0000", 2024-09-14T06:30:45.597 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.3482000000000001e-05 2024-09-14T06:30:45.597 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:45.597 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:30:45.597 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:45.598 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:45.598 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:45.598 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:11461 3.1d 3:bcad7303:::rbd_data.112d92027590.0000000000000001:f [read 176128~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:30:45.598 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:25.655370+0000", 2024-09-14T06:30:45.598 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 79.903293583999996, 2024-09-14T06:30:45.598 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052915141999999998, 2024-09-14T06:30:45.598 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:30:45.598 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:30:45.598 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:30:45.598 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:30:45.598 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:30:45.598 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:45.598 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 11461 2024-09-14T06:30:45.598 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:45.598 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:30:45.599 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:45.599 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:30:45.599 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655370+0000", 2024-09-14T06:30:45.599 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:45.599 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:45.599 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:45.599 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:30:45.599 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655370+0000", 2024-09-14T06:30:45.599 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:45.599 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:45.599 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:45.599 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:30:45.599 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655394+0000", 2024-09-14T06:30:45.599 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.4553999999999999e-05 2024-09-14T06:30:45.600 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:45.600 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:45.600 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:30:45.600 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655396+0000", 2024-09-14T06:30:45.600 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.8950000000000001e-06 2024-09-14T06:30:45.600 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:45.600 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:45.600 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:30:45.600 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655401+0000", 2024-09-14T06:30:45.600 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.0089999999999998e-06 2024-09-14T06:30:45.600 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:45.600 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:45.600 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:30:45.600 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655404+0000", 2024-09-14T06:30:45.601 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.3400000000000002e-06 2024-09-14T06:30:45.601 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:45.601 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:45.601 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:30:45.601 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.708186+0000", 2024-09-14T06:30:45.601 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052781726000000001 2024-09-14T06:30:45.601 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:45.601 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:45.601 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:30:45.601 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.708229+0000", 2024-09-14T06:30:45.601 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.2670999999999999e-05 2024-09-14T06:30:45.601 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:45.601 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:45.601 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:30:45.601 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.708285+0000", 2024-09-14T06:30:45.602 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.5946999999999998e-05 2024-09-14T06:30:45.602 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:45.602 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:30:45.602 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:45.602 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:45.602 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:45.602 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:11462 3.1d 3:bcad7303:::rbd_data.112d92027590.0000000000000001:f [read 186368~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:30:45.602 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:25.655410+0000", 2024-09-14T06:30:45.602 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 79.903253105999994, 2024-09-14T06:30:45.602 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052995258000000003, 2024-09-14T06:30:45.602 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:30:45.602 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:30:45.602 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:30:45.602 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:30:45.602 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:30:45.603 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:45.603 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 11462 2024-09-14T06:30:45.603 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:45.603 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:30:45.603 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:45.603 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:30:45.603 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655410+0000", 2024-09-14T06:30:45.603 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:45.603 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:45.603 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:45.603 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:30:45.603 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655410+0000", 2024-09-14T06:30:45.603 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:45.603 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:45.603 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:45.604 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:30:45.604 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655625+0000", 2024-09-14T06:30:45.604 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00021442600000000001 2024-09-14T06:30:45.604 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:45.604 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:45.604 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:30:45.604 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655626+0000", 2024-09-14T06:30:45.604 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.815e-06 2024-09-14T06:30:45.604 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:45.604 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:45.604 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:30:45.604 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655631+0000", 2024-09-14T06:30:45.604 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.651e-06 2024-09-14T06:30:45.604 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:45.605 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:45.605 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:30:45.605 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655634+0000", 2024-09-14T06:30:45.605 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.2449999999999998e-06 2024-09-14T06:30:45.605 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:45.605 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:45.605 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:30:45.605 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.708312+0000", 2024-09-14T06:30:45.605 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052677412 2024-09-14T06:30:45.605 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:45.605 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:45.605 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:30:45.606 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.708352+0000", 2024-09-14T06:30:45.606 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.0729999999999998e-05 2024-09-14T06:30:45.606 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:45.606 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:45.606 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:30:45.606 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.708405+0000", 2024-09-14T06:30:45.606 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.2979000000000002e-05 2024-09-14T06:30:45.606 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:45.606 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:30:45.606 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:45.606 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:45.606 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:45.606 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:11463 3.1d 3:bcad7303:::rbd_data.112d92027590.0000000000000001:f [read 196608~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:30:45.606 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:25.655639+0000", 2024-09-14T06:30:45.607 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 79.903023959999999, 2024-09-14T06:30:45.607 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052888859000000003, 2024-09-14T06:30:45.607 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:30:45.607 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:30:45.607 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:30:45.607 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:30:45.607 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:30:45.607 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:45.607 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 11463 2024-09-14T06:30:45.607 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:45.607 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:30:45.607 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:45.607 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:30:45.607 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655639+0000", 2024-09-14T06:30:45.607 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:45.608 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:45.608 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:45.608 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:30:45.608 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655639+0000", 2024-09-14T06:30:45.608 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:45.608 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:45.608 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:45.608 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:30:45.608 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655640+0000", 2024-09-14T06:30:45.608 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.08e-07 2024-09-14T06:30:45.608 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:45.608 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:45.608 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:30:45.608 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655641+0000", 2024-09-14T06:30:45.608 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.156e-06 2024-09-14T06:30:45.609 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:45.609 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:45.609 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:30:45.609 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655648+0000", 2024-09-14T06:30:45.609 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.8460000000000002e-06 2024-09-14T06:30:45.609 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:45.609 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:45.609 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:30:45.609 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655651+0000", 2024-09-14T06:30:45.609 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.0079999999999998e-06 2024-09-14T06:30:45.609 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:45.609 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:45.609 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:30:45.610 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.708429+0000", 2024-09-14T06:30:45.610 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052778459 2024-09-14T06:30:45.610 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:45.610 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:45.610 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:30:45.610 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.708473+0000", 2024-09-14T06:30:45.610 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.3782999999999997e-05 2024-09-14T06:30:45.610 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:45.610 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:45.610 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:30:45.610 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.708528+0000", 2024-09-14T06:30:45.610 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.5198999999999998e-05 2024-09-14T06:30:45.610 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:45.611 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:30:45.611 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:45.611 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:45.611 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:45.611 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:11464 3.1d 3:bcad7303:::rbd_data.112d92027590.0000000000000001:f [read 206848~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:30:45.611 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:25.655654+0000", 2024-09-14T06:30:45.611 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 79.903009091000001, 2024-09-14T06:30:45.611 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052981845999999999, 2024-09-14T06:30:45.611 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:30:45.611 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:30:45.611 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:30:45.611 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:30:45.611 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:30:45.611 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:45.612 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 11464 2024-09-14T06:30:45.612 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:45.612 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:30:45.612 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:45.612 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:30:45.612 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655654+0000", 2024-09-14T06:30:45.612 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:45.612 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:45.612 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:45.612 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:30:45.612 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655654+0000", 2024-09-14T06:30:45.612 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:45.612 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:45.612 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:45.613 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:30:45.613 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655946+0000", 2024-09-14T06:30:45.613 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00029148200000000003 2024-09-14T06:30:45.613 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:45.613 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:45.613 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:30:45.613 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655948+0000", 2024-09-14T06:30:45.613 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.973e-06 2024-09-14T06:30:45.613 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:45.613 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:45.613 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:30:45.613 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655953+0000", 2024-09-14T06:30:45.613 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.1240000000000004e-06 2024-09-14T06:30:45.613 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:45.613 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:45.614 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:30:45.614 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655956+0000", 2024-09-14T06:30:45.614 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.315e-06 2024-09-14T06:30:45.614 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:45.614 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:45.614 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:30:45.614 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.708543+0000", 2024-09-14T06:30:45.614 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052587416999999997 2024-09-14T06:30:45.614 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:45.614 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:45.614 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:30:45.614 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.708584+0000", 2024-09-14T06:30:45.614 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.1004000000000002e-05 2024-09-14T06:30:45.614 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:45.614 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:45.614 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:30:45.615 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.708636+0000", 2024-09-14T06:30:45.615 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.1530999999999998e-05 2024-09-14T06:30:45.615 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:45.615 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:30:45.615 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:45.615 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:45.615 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:45.615 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:11465 3.1d 3:bcad7303:::rbd_data.112d92027590.0000000000000001:f [read 217088~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:30:45.615 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:25.655961+0000", 2024-09-14T06:30:45.615 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 79.902701938999996, 2024-09-14T06:30:45.615 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052798682999999999, 2024-09-14T06:30:45.615 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:30:45.615 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:30:45.615 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:30:45.615 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:30:45.616 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:30:45.616 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:45.616 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 11465 2024-09-14T06:30:45.616 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:45.616 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:30:45.616 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:45.616 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:30:45.616 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655961+0000", 2024-09-14T06:30:45.616 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:45.616 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:45.616 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:45.616 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:30:45.616 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655961+0000", 2024-09-14T06:30:45.616 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:45.617 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:45.617 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:45.617 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:30:45.617 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.656114+0000", 2024-09-14T06:30:45.617 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.000152745 2024-09-14T06:30:45.617 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:45.617 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:45.617 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:30:45.617 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.656116+0000", 2024-09-14T06:30:45.617 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.804e-06 2024-09-14T06:30:45.617 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:45.617 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:45.617 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:30:45.617 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.656120+0000", 2024-09-14T06:30:45.618 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.4700000000000004e-06 2024-09-14T06:30:45.618 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:45.618 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:45.618 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:30:45.618 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.656123+0000", 2024-09-14T06:30:45.618 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.1269999999999999e-06 2024-09-14T06:30:45.618 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:45.618 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:45.618 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:30:45.618 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.708660+0000", 2024-09-14T06:30:45.618 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052536919000000001 2024-09-14T06:30:45.618 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:45.618 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:45.618 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:30:45.619 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.708703+0000", 2024-09-14T06:30:45.619 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.3077000000000001e-05 2024-09-14T06:30:45.619 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:45.619 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:45.619 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:30:45.619 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.708760+0000", 2024-09-14T06:30:45.619 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.6541e-05 2024-09-14T06:30:45.619 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:45.619 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:30:45.619 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:45.619 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:45.619 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:45.619 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:11466 3.1d 3:bcad7303:::rbd_data.112d92027590.0000000000000001:f [read 227328~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:30:45.619 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:25.656129+0000", 2024-09-14T06:30:45.619 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 79.902534551000002, 2024-09-14T06:30:45.620 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052750987999999999, 2024-09-14T06:30:45.620 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:30:45.620 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:30:45.620 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:30:45.620 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:30:45.620 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:30:45.620 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:45.620 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 11466 2024-09-14T06:30:45.620 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:45.620 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:30:45.620 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:45.620 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:30:45.620 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.656129+0000", 2024-09-14T06:30:45.620 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:45.621 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:45.621 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:45.621 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:30:45.621 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.656129+0000", 2024-09-14T06:30:45.621 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:45.621 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:45.621 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:45.621 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:30:45.621 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.656347+0000", 2024-09-14T06:30:45.621 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.000218054 2024-09-14T06:30:45.621 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:45.621 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:45.621 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:30:45.622 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.656349+0000", 2024-09-14T06:30:45.622 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.9199999999999998e-06 2024-09-14T06:30:45.622 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:45.622 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:45.622 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:30:45.622 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.656353+0000", 2024-09-14T06:30:45.622 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.9210000000000002e-06 2024-09-14T06:30:45.622 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:45.622 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:45.622 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:30:45.622 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.656356+0000", 2024-09-14T06:30:45.622 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.7670000000000001e-06 2024-09-14T06:30:45.622 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:45.622 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:45.623 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:30:45.623 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.708787+0000", 2024-09-14T06:30:45.623 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052430840999999999 2024-09-14T06:30:45.623 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:45.623 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:45.623 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:30:45.623 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.708827+0000", 2024-09-14T06:30:45.623 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.9538e-05 2024-09-14T06:30:45.623 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:45.623 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:45.623 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:30:45.623 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.708880+0000", 2024-09-14T06:30:45.623 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.2947e-05 2024-09-14T06:30:45.623 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:45.624 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:30:45.624 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:45.624 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:45.624 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:45.624 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:11467 3.1d 3:bcad7303:::rbd_data.112d92027590.0000000000000001:f [read 237568~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:30:45.624 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:25.656361+0000", 2024-09-14T06:30:45.624 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 79.902301902999994, 2024-09-14T06:30:45.624 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052643898000000001, 2024-09-14T06:30:45.624 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:30:45.624 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:30:45.624 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:30:45.624 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:30:45.624 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:30:45.624 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:45.625 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 11467 2024-09-14T06:30:45.625 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:45.625 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:30:45.625 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:45.625 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:30:45.625 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.656361+0000", 2024-09-14T06:30:45.625 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:45.625 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:45.625 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:45.625 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:30:45.625 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.656361+0000", 2024-09-14T06:30:45.625 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:45.625 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:45.625 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:45.626 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:30:45.626 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.656855+0000", 2024-09-14T06:30:45.626 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00049372000000000001 2024-09-14T06:30:45.626 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:45.626 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:45.626 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:30:45.626 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.656859+0000", 2024-09-14T06:30:45.626 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.1300000000000003e-06 2024-09-14T06:30:45.626 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:45.626 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:45.626 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:30:45.626 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.656863+0000", 2024-09-14T06:30:45.626 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.2629999999999997e-06 2024-09-14T06:30:45.626 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:45.627 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:45.627 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:30:45.627 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.656867+0000", 2024-09-14T06:30:45.627 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.3220000000000001e-06 2024-09-14T06:30:45.627 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:45.627 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:45.627 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:30:45.627 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.708903+0000", 2024-09-14T06:30:45.627 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052035855999999998 2024-09-14T06:30:45.627 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:45.627 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:45.627 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:30:45.627 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.708949+0000", 2024-09-14T06:30:45.627 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.6276999999999998e-05 2024-09-14T06:30:45.627 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:45.628 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:45.628 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:30:45.628 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.709005+0000", 2024-09-14T06:30:45.628 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.6329999999999998e-05 2024-09-14T06:30:45.628 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:45.628 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:30:45.628 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:45.628 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:45.628 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:45.628 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:11479 3.1d 3:bcad7303:::rbd_data.112d92027590.0000000000000001:f [read 360448~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:30:45.628 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:25.658388+0000", 2024-09-14T06:30:45.628 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 79.900275070000006, 2024-09-14T06:30:45.628 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052408920999999997, 2024-09-14T06:30:45.629 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:30:45.629 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:30:45.629 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:30:45.629 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:30:45.629 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:30:45.629 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:45.629 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 11479 2024-09-14T06:30:45.629 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:45.629 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:30:45.629 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:45.629 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:30:45.629 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.658388+0000", 2024-09-14T06:30:45.629 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:45.630 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:45.630 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:45.630 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:30:45.630 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.658388+0000", 2024-09-14T06:30:45.630 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:45.630 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:45.630 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:45.630 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:30:45.630 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.658534+0000", 2024-09-14T06:30:45.630 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.000145613 2024-09-14T06:30:45.630 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:45.630 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:45.630 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:30:45.630 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.658540+0000", 2024-09-14T06:30:45.631 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.6680000000000004e-06 2024-09-14T06:30:45.631 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:45.631 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:45.631 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:30:45.631 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.658548+0000", 2024-09-14T06:30:45.631 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.7980000000000006e-06 2024-09-14T06:30:45.631 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:45.631 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:45.631 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:30:45.631 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.658554+0000", 2024-09-14T06:30:45.631 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.6869999999999999e-06 2024-09-14T06:30:45.631 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:45.631 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:45.632 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:30:45.632 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.710624+0000", 2024-09-14T06:30:45.632 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052070567999999998 2024-09-14T06:30:45.632 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:45.632 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:45.632 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:30:45.632 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.710666+0000", 2024-09-14T06:30:45.632 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.1467000000000001e-05 2024-09-14T06:30:45.632 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:45.632 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:45.632 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:30:45.632 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.710797+0000", 2024-09-14T06:30:45.632 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00013112 2024-09-14T06:30:45.633 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:45.633 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:30:45.633 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:45.633 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:45.633 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:45.633 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:11480 3.1d 3:bcad7303:::rbd_data.112d92027590.0000000000000001:f [read 370688~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:30:45.633 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:25.658563+0000", 2024-09-14T06:30:45.633 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 79.900100139000003, 2024-09-14T06:30:45.633 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052429164, 2024-09-14T06:30:45.633 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:30:45.633 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:30:45.634 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:30:45.634 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:30:45.634 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:30:45.634 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:45.634 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 11480 2024-09-14T06:30:45.634 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:45.634 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:30:45.634 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:45.634 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:30:45.634 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.658563+0000", 2024-09-14T06:30:45.634 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:45.634 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:45.634 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:45.634 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:30:45.635 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.658563+0000", 2024-09-14T06:30:45.635 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:45.635 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:45.635 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:45.635 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:30:45.635 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.658658+0000", 2024-09-14T06:30:45.635 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.5346999999999994e-05 2024-09-14T06:30:45.635 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:45.635 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:45.635 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:30:45.635 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.658662+0000", 2024-09-14T06:30:45.635 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.14e-06 2024-09-14T06:30:45.635 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:45.635 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:45.636 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:30:45.636 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.658669+0000", 2024-09-14T06:30:45.636 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.7379999999999994e-06 2024-09-14T06:30:45.636 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:45.636 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:45.636 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:30:45.636 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.658675+0000", 2024-09-14T06:30:45.636 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.5430000000000001e-06 2024-09-14T06:30:45.636 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:45.636 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:45.636 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:30:45.636 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.710824+0000", 2024-09-14T06:30:45.636 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052148847999999998 2024-09-14T06:30:45.636 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:45.637 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:45.637 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:30:45.637 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.710864+0000", 2024-09-14T06:30:45.637 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.9925999999999997e-05 2024-09-14T06:30:45.637 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:45.637 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:45.637 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:30:45.637 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.710992+0000", 2024-09-14T06:30:45.637 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.000128622 2024-09-14T06:30:45.637 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:45.637 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:30:45.637 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:45.637 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:45.637 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:45.638 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:11481 3.1d 3:bcad7303:::rbd_data.112d92027590.0000000000000001:f [read 380928~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:30:45.638 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:25.658684+0000", 2024-09-14T06:30:45.638 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 79.899979068999997, 2024-09-14T06:30:45.638 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052508565, 2024-09-14T06:30:45.638 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:30:45.638 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:30:45.638 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:30:45.638 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:30:45.638 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:30:45.638 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:45.638 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 11481 2024-09-14T06:30:45.638 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:45.638 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:30:45.638 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:45.639 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:30:45.639 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.658684+0000", 2024-09-14T06:30:45.639 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:45.639 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:45.639 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:45.639 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:30:45.639 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.658684+0000", 2024-09-14T06:30:45.639 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:45.639 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:45.639 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:45.639 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:30:45.639 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.658715+0000", 2024-09-14T06:30:45.640 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.0703000000000001e-05 2024-09-14T06:30:45.640 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:45.640 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:45.640 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:30:45.640 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.658718+0000", 2024-09-14T06:30:45.640 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.0709999999999999e-06 2024-09-14T06:30:45.640 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:45.640 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:45.640 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:30:45.640 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.658725+0000", 2024-09-14T06:30:45.640 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.2339999999999997e-06 2024-09-14T06:30:45.640 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:45.641 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:45.641 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:30:45.641 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.658731+0000", 2024-09-14T06:30:45.641 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.6450000000000002e-06 2024-09-14T06:30:45.641 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:45.641 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:45.641 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:30:45.641 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.711020+0000", 2024-09-14T06:30:45.641 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052288886 2024-09-14T06:30:45.641 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:45.641 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:45.641 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:30:45.641 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.711061+0000", 2024-09-14T06:30:45.641 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.1084000000000002e-05 2024-09-14T06:30:45.642 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:45.642 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:45.642 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:30:45.642 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.711193+0000", 2024-09-14T06:30:45.642 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00013194199999999999 2024-09-14T06:30:45.642 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:45.642 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:30:45.642 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:45.642 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:45.642 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:45.642 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:11482 3.1d 3:bcad7303:::rbd_data.112d92027590.0000000000000001:f [read 391168~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:30:45.642 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:25.658740+0000", 2024-09-14T06:30:45.642 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 79.899923603999994, 2024-09-14T06:30:45.642 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052637015000000002, 2024-09-14T06:30:45.642 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:30:45.643 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:30:45.643 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:30:45.643 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:30:45.643 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:30:45.643 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:45.643 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 11482 2024-09-14T06:30:45.643 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:45.643 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:30:45.643 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:45.643 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:30:45.643 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.658740+0000", 2024-09-14T06:30:45.643 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:45.643 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:45.643 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:45.643 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:30:45.643 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.658740+0000", 2024-09-14T06:30:45.644 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:45.644 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:45.644 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:45.644 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:30:45.644 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.658880+0000", 2024-09-14T06:30:45.644 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00014036100000000001 2024-09-14T06:30:45.644 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:45.644 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:45.644 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:30:45.644 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.658883+0000", 2024-09-14T06:30:45.644 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.3969999999999998e-06 2024-09-14T06:30:45.644 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:45.644 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:45.644 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:30:45.645 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.658890+0000", 2024-09-14T06:30:45.645 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.1500000000000002e-06 2024-09-14T06:30:45.645 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:45.645 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:45.645 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:30:45.645 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.658897+0000", 2024-09-14T06:30:45.645 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.1260000000000003e-06 2024-09-14T06:30:45.645 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:45.645 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:45.645 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:30:45.645 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.711207+0000", 2024-09-14T06:30:45.645 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052310777000000003 2024-09-14T06:30:45.645 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:45.645 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:45.646 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:30:45.646 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.711249+0000", 2024-09-14T06:30:45.646 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.1786999999999998e-05 2024-09-14T06:30:45.646 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:45.646 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:45.646 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:30:45.646 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.711377+0000", 2024-09-14T06:30:45.646 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.000127417 2024-09-14T06:30:45.646 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:45.646 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:30:45.646 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:45.646 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:45.647 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:45.647 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:11483 3.1d 3:bcad7303:::rbd_data.112d92027590.0000000000000001:f [read 401408~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:30:45.647 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:25.658906+0000", 2024-09-14T06:30:45.647 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 79.899756819999993, 2024-09-14T06:30:45.647 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052595109000000001, 2024-09-14T06:30:45.647 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:30:45.647 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:30:45.647 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:30:45.647 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:30:45.647 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:30:45.647 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:45.647 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 11483 2024-09-14T06:30:45.647 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:45.647 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:30:45.647 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:45.648 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:30:45.648 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.658906+0000", 2024-09-14T06:30:45.648 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:45.648 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:45.648 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:45.648 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:30:45.648 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.658906+0000", 2024-09-14T06:30:45.648 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:45.648 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:45.648 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:45.648 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:30:45.648 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.664985+0000", 2024-09-14T06:30:45.648 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.0060787250000000001 2024-09-14T06:30:45.648 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:45.649 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:45.649 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:30:45.649 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.664991+0000", 2024-09-14T06:30:45.649 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.7030000000000003e-06 2024-09-14T06:30:45.649 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:45.649 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:45.649 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:30:45.649 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.664999+0000", 2024-09-14T06:30:45.649 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.4430000000000003e-06 2024-09-14T06:30:45.649 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:45.649 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:45.649 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:30:45.649 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.665006+0000", 2024-09-14T06:30:45.649 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.0840000000000003e-06 2024-09-14T06:30:45.650 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:45.650 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:45.650 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:30:45.650 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.711402+0000", 2024-09-14T06:30:45.650 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046395714999999997 2024-09-14T06:30:45.650 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:45.650 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:45.650 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:30:45.650 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.711443+0000", 2024-09-14T06:30:45.650 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.0911999999999997e-05 2024-09-14T06:30:45.650 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:45.650 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:45.651 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:30:45.651 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.711501+0000", 2024-09-14T06:30:45.651 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.8526999999999997e-05 2024-09-14T06:30:45.651 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:45.651 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:30:45.651 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:45.651 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:45.651 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:30:45.651 INFO:teuthology.orchestra.run.smithi138.stdout:} 2024-09-14T06:30:45.653 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-14T06:30:45.653 INFO:tasks.ceph.osd.5.smithi138.stderr:2024-09-14T06:30:45.651+0000 7fe13746f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33618) UID: 0 2024-09-14T06:30:45.710 INFO:teuthology.orchestra.run.smithi094.stderr:nodeep-scrub is unset 2024-09-14T06:30:45.751 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-14T06:30:45.753 INFO:tasks.ceph.osd.5.smithi138.stderr:2024-09-14T06:30:45.752+0000 7fe13746f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33618) UID: 0 2024-09-14T06:30:45.799 INFO:teuthology.orchestra.run.smithi138.stdout:{ 2024-09-14T06:30:45.800 INFO:teuthology.orchestra.run.smithi138.stdout: "ops": [], 2024-09-14T06:30:45.800 INFO:teuthology.orchestra.run.smithi138.stdout: "num_ops": 0 2024-09-14T06:30:45.800 INFO:teuthology.orchestra.run.smithi138.stdout:} 2024-09-14T06:30:45.808 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-14T06:30:45.858 INFO:teuthology.orchestra.run.smithi094.stdout:checking clone #25, image image_client.0-clone25 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent26 2024-09-14T06:30:45.892 INFO:tasks.ceph.osd.7.smithi138.stderr:2024-09-14T06:30:45.891+0000 7f3c6a21d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33623) UID: 0 2024-09-14T06:30:45.903 INFO:teuthology.orchestra.run.smithi094.stdout:osd.0: {} 2024-09-14T06:30:45.904 INFO:teuthology.orchestra.run.smithi094.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-14T06:30:45.925 INFO:teuthology.orchestra.run.smithi094.stdout:osd.1: {} 2024-09-14T06:30:45.925 INFO:teuthology.orchestra.run.smithi094.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-14T06:30:45.943 INFO:teuthology.orchestra.run.smithi094.stdout:osd.2: {} 2024-09-14T06:30:45.943 INFO:teuthology.orchestra.run.smithi094.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-14T06:30:45.954 INFO:teuthology.orchestra.run.smithi138.stdout:{ 2024-09-14T06:30:45.955 INFO:teuthology.orchestra.run.smithi138.stdout: "ops": [], 2024-09-14T06:30:45.955 INFO:teuthology.orchestra.run.smithi138.stdout: "complaint_time": 30, 2024-09-14T06:30:45.955 INFO:teuthology.orchestra.run.smithi138.stdout: "num_blocked_ops": 0 2024-09-14T06:30:45.955 INFO:teuthology.orchestra.run.smithi138.stdout:} 2024-09-14T06:30:45.955 INFO:tasks.ceph.osd.3.smithi094.stderr:2024-09-14T06:30:45.954+0000 7feeeafe8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33655) UID: 0 2024-09-14T06:30:45.963 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-14T06:30:45.969 INFO:teuthology.orchestra.run.smithi094.stdout:osd.3: {} 2024-09-14T06:30:45.969 INFO:teuthology.orchestra.run.smithi094.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-14T06:30:45.986 INFO:teuthology.orchestra.run.smithi094.stdout:osd.4: {} 2024-09-14T06:30:45.986 INFO:teuthology.orchestra.run.smithi094.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-14T06:30:46.003 INFO:teuthology.orchestra.run.smithi094.stdout:osd.5: {} 2024-09-14T06:30:46.004 INFO:teuthology.orchestra.run.smithi094.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-14T06:30:46.020 INFO:teuthology.orchestra.run.smithi094.stdout:osd.6: {} 2024-09-14T06:30:46.020 INFO:teuthology.orchestra.run.smithi094.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-14T06:30:46.037 INFO:teuthology.orchestra.run.smithi094.stdout:osd.7: {} 2024-09-14T06:30:46.037 INFO:teuthology.orchestra.run.smithi094.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-14T06:30:46.068 INFO:teuthology.orchestra.run.smithi094.stdout:3513 write 0x1f792c8 thru 0x1f83b5e (0xa897 bytes) 2024-09-14T06:30:46.069 INFO:tasks.ceph.osd.1.smithi094.stderr:2024-09-14T06:30:46.068+0000 7f328e47a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33641) UID: 0 2024-09-14T06:30:46.075 INFO:teuthology.orchestra.run.smithi094.stdout:3514 punch from 0x20865d4 to 0x20898ed, (0x3319 bytes) 2024-09-14T06:30:46.075 INFO:teuthology.orchestra.run.smithi094.stdout:3515 read 0x13cf750 thru 0x13d121d (0x1ace bytes) 2024-09-14T06:30:46.075 INFO:teuthology.orchestra.run.smithi094.stdout:3517 read 0x3b3cbbc thru 0x3b4aeb6 (0xe2fb bytes) 2024-09-14T06:30:46.075 INFO:teuthology.orchestra.run.smithi094.stdout:3519 read 0x59f5122 thru 0x59f78c7 (0x27a6 bytes) 2024-09-14T06:30:46.076 INFO:teuthology.orchestra.run.smithi094.stdout:3521 write 0x221df8c thru 0x2228b58 (0xabcd bytes) 2024-09-14T06:30:46.077 INFO:teuthology.orchestra.run.smithi094.stdout:3522 read 0x12bbd82 thru 0x12c339b (0x761a bytes) 2024-09-14T06:30:46.078 INFO:teuthology.orchestra.run.smithi094.stdout:3523 writesame 0x2c9c121 thru 0x2c9ef7e data_size 0x5(0x2e5e bytes) 2024-09-14T06:30:46.082 INFO:teuthology.orchestra.run.smithi094.stdout:3524 write 0xc5f56b7 thru 0xc5fdc12 (0x855c bytes) 2024-09-14T06:30:46.086 INFO:teuthology.orchestra.run.smithi094.stdout:3525 write 0x6fe3194 thru 0x6ff3009 (0xfe76 bytes) 2024-09-14T06:30:46.090 INFO:teuthology.orchestra.run.smithi094.stdout:3527 write 0xa6beaa8 thru 0xa6c52f8 (0x6851 bytes) 2024-09-14T06:30:46.095 INFO:teuthology.orchestra.run.smithi094.stdout:3528 trunc from 0xd41baf6 to 0xc68da12 2024-09-14T06:30:46.110 INFO:teuthology.orchestra.run.smithi138.stdout:ERROR: (22) Invalid argument 2024-09-14T06:30:46.110 INFO:teuthology.orchestra.run.smithi138.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:30:46.111 INFO:teuthology.orchestra.run.smithi094.stdout:3529 read 0x92217ac thru 0x922fd41 (0xe596 bytes) 2024-09-14T06:30:46.112 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-14T06:30:46.112 INFO:teuthology.orchestra.run.smithi094.stdout:3531 punch from 0x83c48a to 0x845acf, (0x9645 bytes) 2024-09-14T06:30:46.120 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-14T06:30:46.122 INFO:teuthology.orchestra.run.smithi094.stdout:3533 trunc from 0xc68da12 to 0x30a4af9 2024-09-14T06:30:46.164 INFO:tasks.ceph.osd.5.smithi138.stderr:2024-09-14T06:30:46.163+0000 7fe13746f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33618) UID: 0 2024-09-14T06:30:46.169 INFO:teuthology.orchestra.run.smithi094.stdout:3534 writesame 0xe6ede03 thru 0xe6fa9cb data_size 0x1(0xcbc9 bytes) 2024-09-14T06:30:46.169 INFO:teuthology.orchestra.run.smithi094.stdout:3535 punch from 0x4cb79a6 to 0x4cc3f4b, (0xc5a5 bytes) 2024-09-14T06:30:46.169 INFO:teuthology.orchestra.run.smithi094.stdout:3536 read 0x824b4c5 thru 0x825416e (0x8caa bytes) 2024-09-14T06:30:46.169 INFO:teuthology.orchestra.run.smithi094.stdout:3540 read 0xa2eb53 thru 0xa39ab6 (0xaf64 bytes) 2024-09-14T06:30:46.169 INFO:teuthology.orchestra.run.smithi094.stdout:3543 read 0x30b7992 thru 0x30bf4fb (0x7b6a bytes) 2024-09-14T06:30:46.169 INFO:teuthology.orchestra.run.smithi094.stdout:3544 write 0xdcf1a22 thru 0xdd001ce (0xe7ad bytes) 2024-09-14T06:30:46.179 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-14T06:30:46.192 INFO:teuthology.orchestra.run.smithi094.stdout:3547 trunc from 0xe6fa9cc to 0x37cce3d 2024-09-14T06:30:46.219 INFO:teuthology.orchestra.run.smithi094.stdout:3548 write 0x9791673 thru 0x979dde7 (0xc775 bytes) 2024-09-14T06:30:46.222 INFO:teuthology.orchestra.run.smithi094.stdout:3549 clone 28 order 25 su 1048576 sc 8 2024-09-14T06:30:46.230 INFO:teuthology.orchestra.run.smithi138.stdout:ERROR: (22) Invalid argument 2024-09-14T06:30:46.231 INFO:teuthology.orchestra.run.smithi138.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:30:46.239 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-14T06:30:46.257 INFO:tasks.ceph.osd.2.smithi094.stderr:2024-09-14T06:30:46.256+0000 7faf3269b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33649) UID: 0 2024-09-14T06:30:46.358 INFO:tasks.ceph.osd.1.smithi094.stderr:2024-09-14T06:30:46.357+0000 7f328e47a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33641) UID: 0 2024-09-14T06:30:46.383 INFO:teuthology.orchestra.run.smithi138.stdout:ERROR: (22) Invalid argument 2024-09-14T06:30:46.384 INFO:teuthology.orchestra.run.smithi138.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:30:46.392 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-14T06:30:46.459 INFO:tasks.ceph.osd.0.smithi094.stderr:2024-09-14T06:30:46.458+0000 7f07c9333640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33636) UID: 0 2024-09-14T06:30:46.495 INFO:teuthology.orchestra.run.smithi138.stdout:ERROR: (22) Invalid argument 2024-09-14T06:30:46.495 INFO:teuthology.orchestra.run.smithi138.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:30:46.505 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-14T06:30:46.560 INFO:tasks.ceph.osd.4.smithi138.stderr:2024-09-14T06:30:46.559+0000 7f6c653c8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33615) UID: 0 2024-09-14T06:30:46.608 INFO:teuthology.orchestra.run.smithi094.stdout:ERROR: (22) Invalid argument 2024-09-14T06:30:46.608 INFO:teuthology.orchestra.run.smithi094.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:30:46.617 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-14T06:30:46.660 INFO:tasks.ceph.osd.5.smithi138.stderr:2024-09-14T06:30:46.659+0000 7fe13746f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33618) UID: 0 2024-09-14T06:30:46.723 INFO:teuthology.orchestra.run.smithi094.stdout:ERROR: (22) Invalid argument 2024-09-14T06:30:46.723 INFO:teuthology.orchestra.run.smithi094.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:30:46.731 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-14T06:30:46.761 INFO:tasks.ceph.osd.6.smithi138.stderr:2024-09-14T06:30:46.760+0000 7f8cd8849640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33620) UID: 0 2024-09-14T06:30:46.835 INFO:teuthology.orchestra.run.smithi094.stdout:ERROR: (22) Invalid argument 2024-09-14T06:30:46.835 INFO:teuthology.orchestra.run.smithi094.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:30:46.844 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-14T06:30:46.862 INFO:tasks.ceph.osd.7.smithi138.stderr:2024-09-14T06:30:46.861+0000 7f3c6a21d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33623) UID: 0 2024-09-14T06:30:46.946 INFO:teuthology.orchestra.run.smithi094.stdout:ERROR: (22) Invalid argument 2024-09-14T06:30:46.947 INFO:teuthology.orchestra.run.smithi094.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:30:46.956 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-14T06:30:46.997 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-14T06:30:46.997 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-14T06:30:46.998 INFO:tasks.thrashosds.thrasher:inject_pause on osd.3 2024-09-14T06:30:46.998 INFO:tasks.thrashosds.thrasher:Testing filestore_inject_stall pause injection for duration 3 2024-09-14T06:30:46.998 INFO:tasks.thrashosds.thrasher:Checking after 0, should_be_down=False 2024-09-14T06:30:46.998 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok config set filestore_inject_stall 3 2024-09-14T06:30:47.000 INFO:tasks.ceph.osd.1.smithi094.stderr:2024-09-14T06:30:46.999+0000 7f328e47a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33641) UID: 0 2024-09-14T06:30:47.061 INFO:teuthology.orchestra.run.smithi094.stdout:ERROR: (22) Invalid argument 2024-09-14T06:30:47.061 INFO:teuthology.orchestra.run.smithi094.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:30:47.063 INFO:tasks.ceph.osd.1.smithi094.stderr:2024-09-14T06:30:47.062+0000 7f328e47a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33641) UID: 0 2024-09-14T06:30:47.070 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-14T06:30:47.104 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-14T06:30:47.104 INFO:teuthology.orchestra.run.smithi094.stdout: "success": "filestore_inject_stall = '' (not observed, change may require restart) " 2024-09-14T06:30:47.104 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-14T06:30:47.164 INFO:tasks.ceph.osd.0.smithi094.stderr:2024-09-14T06:30:47.163+0000 7f07c9333640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33636) UID: 0 2024-09-14T06:30:47.206 INFO:teuthology.orchestra.run.smithi094.stdout:ERROR: (22) Invalid argument 2024-09-14T06:30:47.206 INFO:teuthology.orchestra.run.smithi094.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:30:47.215 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-14T06:30:47.299 INFO:tasks.ceph.osd.2.smithi094.stderr:2024-09-14T06:30:47.298+0000 7faf3269b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33649) UID: 0 2024-09-14T06:30:47.358 INFO:teuthology.orchestra.run.smithi094.stdout:ERROR: (22) Invalid argument 2024-09-14T06:30:47.358 INFO:teuthology.orchestra.run.smithi094.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:30:47.366 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-14T06:30:47.367 INFO:tasks.ceph.osd.0.smithi094.stderr:2024-09-14T06:30:47.365+0000 7f07c9333640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33636) UID: 0 2024-09-14T06:30:47.467 INFO:tasks.ceph.osd.1.smithi094.stderr:2024-09-14T06:30:47.466+0000 7f328e47a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33641) UID: 0 2024-09-14T06:30:47.508 INFO:teuthology.orchestra.run.smithi094.stdout:ERROR: (22) Invalid argument 2024-09-14T06:30:47.508 INFO:teuthology.orchestra.run.smithi094.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:30:47.516 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-14T06:30:47.601 INFO:tasks.ceph.osd.3.smithi094.stderr:2024-09-14T06:30:47.600+0000 7feeeafe8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33655) UID: 0 2024-09-14T06:30:47.659 INFO:teuthology.orchestra.run.smithi094.stdout:ERROR: (22) Invalid argument 2024-09-14T06:30:47.659 INFO:teuthology.orchestra.run.smithi094.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:30:47.668 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-14T06:30:47.709 INFO:tasks.ceph.osd.1.smithi094.stderr:2024-09-14T06:30:47.708+0000 7f328e47a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33641) UID: 0 2024-09-14T06:30:47.726 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-14T06:30:47.770 INFO:tasks.ceph.osd.7.smithi138.stderr:2024-09-14T06:30:47.769+0000 7f3c6a21d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33623) UID: 0 2024-09-14T06:30:47.850 INFO:teuthology.orchestra.run.smithi094.stdout:ERROR: (22) Invalid argument 2024-09-14T06:30:47.850 INFO:teuthology.orchestra.run.smithi094.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:30:47.859 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-14T06:30:47.903 INFO:tasks.ceph.osd.1.smithi094.stderr:2024-09-14T06:30:47.902+0000 7f328e47a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33641) UID: 0 2024-09-14T06:30:47.967 INFO:teuthology.orchestra.run.smithi094.stdout:ERROR: (22) Invalid argument 2024-09-14T06:30:47.967 INFO:teuthology.orchestra.run.smithi094.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:30:47.972 INFO:tasks.ceph.osd.1.smithi094.stderr:2024-09-14T06:30:47.971+0000 7f328e47a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33641) UID: 0 2024-09-14T06:30:47.976 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-14T06:30:48.049 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-14T06:30:48.104 INFO:tasks.ceph.osd.2.smithi094.stderr:2024-09-14T06:30:48.103+0000 7faf3269b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33649) UID: 0 2024-09-14T06:30:48.121 INFO:teuthology.orchestra.run.smithi094.stdout:ERROR: (22) Invalid argument 2024-09-14T06:30:48.121 INFO:teuthology.orchestra.run.smithi094.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:30:48.130 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-14T06:30:48.176 INFO:tasks.ceph.osd.5.smithi138.stderr:2024-09-14T06:30:48.175+0000 7fe13746f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33618) UID: 0 2024-09-14T06:30:48.180 INFO:teuthology.orchestra.run.smithi094.stdout:osd.0: {} 2024-09-14T06:30:48.181 INFO:teuthology.orchestra.run.smithi094.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-14T06:30:48.197 INFO:teuthology.orchestra.run.smithi094.stdout:osd.1: {} 2024-09-14T06:30:48.198 INFO:teuthology.orchestra.run.smithi094.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-14T06:30:48.213 INFO:teuthology.orchestra.run.smithi094.stdout:osd.2: {} 2024-09-14T06:30:48.213 INFO:teuthology.orchestra.run.smithi094.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-14T06:30:48.229 INFO:teuthology.orchestra.run.smithi094.stdout:osd.3: {} 2024-09-14T06:30:48.229 INFO:teuthology.orchestra.run.smithi094.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-14T06:30:48.234 INFO:teuthology.orchestra.run.smithi138.stdout:ERROR: (22) Invalid argument 2024-09-14T06:30:48.234 INFO:teuthology.orchestra.run.smithi138.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:30:48.243 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-14T06:30:48.276 INFO:tasks.ceph.osd.3.smithi094.stderr:2024-09-14T06:30:48.275+0000 7feeeafe8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33655) UID: 0 2024-09-14T06:30:48.377 INFO:tasks.ceph.osd.5.smithi138.stderr:2024-09-14T06:30:48.376+0000 7fe13746f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33618) UID: 0 2024-09-14T06:30:48.385 INFO:teuthology.orchestra.run.smithi138.stdout:ERROR: (22) Invalid argument 2024-09-14T06:30:48.385 INFO:teuthology.orchestra.run.smithi138.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:30:48.393 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-14T06:30:48.447 INFO:teuthology.orchestra.run.smithi094.stdout:osd.4: {} 2024-09-14T06:30:48.447 INFO:teuthology.orchestra.run.smithi094.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-14T06:30:48.464 INFO:teuthology.orchestra.run.smithi094.stdout:osd.5: {} 2024-09-14T06:30:48.464 INFO:teuthology.orchestra.run.smithi094.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-14T06:30:48.478 INFO:tasks.ceph.osd.5.smithi138.stderr:2024-09-14T06:30:48.477+0000 7fe13746f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33618) UID: 0 2024-09-14T06:30:48.480 INFO:teuthology.orchestra.run.smithi094.stdout:osd.6: {} 2024-09-14T06:30:48.480 INFO:teuthology.orchestra.run.smithi094.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-14T06:30:48.496 INFO:teuthology.orchestra.run.smithi094.stdout:osd.7: {} 2024-09-14T06:30:48.496 INFO:teuthology.orchestra.run.smithi094.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-14T06:30:48.539 INFO:teuthology.orchestra.run.smithi138.stdout:{ 2024-09-14T06:30:48.539 INFO:teuthology.orchestra.run.smithi138.stdout: "size": 20, 2024-09-14T06:30:48.539 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 600, 2024-09-14T06:30:48.539 INFO:teuthology.orchestra.run.smithi138.stdout: "ops": [ 2024-09-14T06:30:48.539 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:48.539 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:19328 3.3 3:c34e5e51:::rbd_data.112d92027590.0000000000000024:f [read 1926144~1024 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:30:48.539 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:27.059768+0000", 2024-09-14T06:30:48.540 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 81.477414269999997, 2024-09-14T06:30:48.540 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046369147999999999, 2024-09-14T06:30:48.540 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:30:48.540 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:30:48.540 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:30:48.540 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:30:48.540 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:30:48.540 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:48.540 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 19328 2024-09-14T06:30:48.540 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:48.540 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:30:48.540 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:48.541 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:30:48.541 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.059768+0000", 2024-09-14T06:30:48.541 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:48.541 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:48.541 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:48.541 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:30:48.541 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.059768+0000", 2024-09-14T06:30:48.541 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:48.541 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:48.541 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:48.541 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:30:48.541 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.059874+0000", 2024-09-14T06:30:48.541 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00010551400000000001 2024-09-14T06:30:48.542 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:48.542 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:48.542 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:30:48.542 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.059877+0000", 2024-09-14T06:30:48.542 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.8279999999999999e-06 2024-09-14T06:30:48.542 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:48.542 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:48.542 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:30:48.542 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.059884+0000", 2024-09-14T06:30:48.542 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.6389999999999992e-06 2024-09-14T06:30:48.542 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:48.542 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:48.542 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:30:48.542 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.059890+0000", 2024-09-14T06:30:48.542 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.49e-06 2024-09-14T06:30:48.542 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:48.543 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:48.543 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:30:48.543 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.105804+0000", 2024-09-14T06:30:48.543 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.045914242000000001 2024-09-14T06:30:48.543 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:48.543 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:48.543 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:30:48.543 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.105886+0000", 2024-09-14T06:30:48.543 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.2305999999999994e-05 2024-09-14T06:30:48.543 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:48.543 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:48.543 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:30:48.543 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.106138+0000", 2024-09-14T06:30:48.543 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.000251129 2024-09-14T06:30:48.543 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:48.543 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:30:48.544 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:48.544 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:48.544 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:48.544 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:19329 3.3 3:c34e5e51:::rbd_data.112d92027590.0000000000000024:f [read 1936384~1024 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:30:48.544 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:27.059899+0000", 2024-09-14T06:30:48.544 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 81.477283747000001, 2024-09-14T06:30:48.544 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046572122000000001, 2024-09-14T06:30:48.544 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:30:48.544 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:30:48.544 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:30:48.544 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:30:48.544 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:30:48.544 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:48.544 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 19329 2024-09-14T06:30:48.545 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:48.545 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:30:48.545 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:48.545 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:30:48.545 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.059899+0000", 2024-09-14T06:30:48.545 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:48.545 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:48.545 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:48.545 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:30:48.545 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.059899+0000", 2024-09-14T06:30:48.545 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:48.545 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:48.545 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:48.546 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:30:48.546 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062501+0000", 2024-09-14T06:30:48.546 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.0026016419999999999 2024-09-14T06:30:48.546 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:48.546 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:48.546 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:30:48.546 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062514+0000", 2024-09-14T06:30:48.546 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.2947e-05 2024-09-14T06:30:48.546 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:48.546 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:48.546 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:30:48.546 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062523+0000", 2024-09-14T06:30:48.546 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.5519999999999993e-06 2024-09-14T06:30:48.546 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:48.547 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:48.547 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:30:48.547 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062534+0000", 2024-09-14T06:30:48.547 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.1416e-05 2024-09-14T06:30:48.547 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:48.547 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:48.547 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:30:48.547 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.106179+0000", 2024-09-14T06:30:48.547 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.043644049999999997 2024-09-14T06:30:48.547 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:48.547 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:48.547 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:30:48.547 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.106262+0000", 2024-09-14T06:30:48.547 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.3178999999999998e-05 2024-09-14T06:30:48.547 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:48.547 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:48.548 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:30:48.548 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.106471+0000", 2024-09-14T06:30:48.548 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00020933599999999999 2024-09-14T06:30:48.548 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:48.548 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:30:48.548 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:48.548 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:48.548 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:48.548 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:19351 3.3 3:c34e5e51:::rbd_data.112d92027590.0000000000000024:f [read 1159168~1024 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:30:48.548 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:27.062766+0000", 2024-09-14T06:30:48.548 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 81.474416521999999, 2024-09-14T06:30:48.548 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046099454999999998, 2024-09-14T06:30:48.548 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:30:48.548 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:30:48.548 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:30:48.548 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:30:48.549 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:30:48.549 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:48.549 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 19351 2024-09-14T06:30:48.549 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:48.549 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:30:48.549 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:48.549 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:30:48.549 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062766+0000", 2024-09-14T06:30:48.549 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:48.549 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:48.549 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:48.549 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:30:48.549 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062766+0000", 2024-09-14T06:30:48.549 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:48.549 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:48.550 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:48.550 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:30:48.550 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062766+0000", 2024-09-14T06:30:48.550 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.6e-07 2024-09-14T06:30:48.550 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:48.550 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:48.550 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:30:48.550 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062767+0000", 2024-09-14T06:30:48.550 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.0070000000000001e-06 2024-09-14T06:30:48.550 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:48.550 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:48.550 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:30:48.550 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062785+0000", 2024-09-14T06:30:48.550 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.7884999999999999e-05 2024-09-14T06:30:48.550 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:48.550 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:48.551 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:30:48.551 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062790+0000", 2024-09-14T06:30:48.551 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.0900000000000004e-06 2024-09-14T06:30:48.551 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:48.551 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:48.551 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:30:48.551 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.108735+0000", 2024-09-14T06:30:48.551 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.045944411999999997 2024-09-14T06:30:48.551 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:48.551 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:48.551 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:30:48.551 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.108809+0000", 2024-09-14T06:30:48.551 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.4157000000000002e-05 2024-09-14T06:30:48.551 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:48.551 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:48.552 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:30:48.552 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.108866+0000", 2024-09-14T06:30:48.552 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.6644000000000002e-05 2024-09-14T06:30:48.552 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:48.552 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:30:48.552 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:48.552 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:48.552 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:48.552 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:19352 3.3 3:c34e5e51:::rbd_data.112d92027590.0000000000000024:f [read 1169408~1024 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:30:48.552 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:27.062798+0000", 2024-09-14T06:30:48.552 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 81.474385111000004, 2024-09-14T06:30:48.552 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046255900000000003, 2024-09-14T06:30:48.552 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:30:48.552 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:30:48.552 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:30:48.553 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:30:48.553 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:30:48.553 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:48.553 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 19352 2024-09-14T06:30:48.553 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:48.553 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:30:48.553 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:48.553 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:30:48.553 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062798+0000", 2024-09-14T06:30:48.553 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:48.553 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:48.553 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:48.553 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:30:48.553 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062798+0000", 2024-09-14T06:30:48.553 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:48.553 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:48.554 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:48.554 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:30:48.554 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062798+0000", 2024-09-14T06:30:48.554 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.37e-07 2024-09-14T06:30:48.554 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:48.554 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:48.554 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:30:48.554 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062800+0000", 2024-09-14T06:30:48.554 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.0269999999999998e-06 2024-09-14T06:30:48.554 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:48.554 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:48.554 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:30:48.554 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062806+0000", 2024-09-14T06:30:48.554 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.6690000000000003e-06 2024-09-14T06:30:48.554 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:48.554 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:48.555 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:30:48.555 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062810+0000", 2024-09-14T06:30:48.555 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.3449999999999999e-06 2024-09-14T06:30:48.555 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:48.555 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:48.555 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:30:48.555 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.108892+0000", 2024-09-14T06:30:48.555 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046081715000000002 2024-09-14T06:30:48.555 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:48.555 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:48.555 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:30:48.555 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.108940+0000", 2024-09-14T06:30:48.555 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.7815999999999998e-05 2024-09-14T06:30:48.555 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:48.555 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:48.555 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:30:48.556 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.109053+0000", 2024-09-14T06:30:48.556 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.000113791 2024-09-14T06:30:48.556 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:48.556 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:30:48.556 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:48.556 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:48.556 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:48.556 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:19353 3.3 3:c34e5e51:::rbd_data.112d92027590.0000000000000024:f [read 1179648~1024 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:30:48.556 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:27.062816+0000", 2024-09-14T06:30:48.556 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 81.474366591999996, 2024-09-14T06:30:48.556 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046481256999999998, 2024-09-14T06:30:48.556 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:30:48.556 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:30:48.556 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:30:48.557 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:30:48.557 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:30:48.557 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:48.557 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 19353 2024-09-14T06:30:48.557 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:48.557 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:30:48.557 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:48.557 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:30:48.557 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062816+0000", 2024-09-14T06:30:48.557 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:48.557 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:48.557 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:48.557 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:30:48.557 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062816+0000", 2024-09-14T06:30:48.558 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:48.558 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:48.558 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:48.558 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:30:48.558 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062816+0000", 2024-09-14T06:30:48.558 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.2899999999999999e-07 2024-09-14T06:30:48.558 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:48.558 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:48.558 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:30:48.558 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062818+0000", 2024-09-14T06:30:48.559 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.699e-06 2024-09-14T06:30:48.559 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:48.559 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:48.559 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:30:48.559 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062825+0000", 2024-09-14T06:30:48.559 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.878e-06 2024-09-14T06:30:48.559 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:48.559 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:48.559 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:30:48.559 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062830+0000", 2024-09-14T06:30:48.559 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.5410000000000002e-06 2024-09-14T06:30:48.559 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:48.559 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:48.559 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:30:48.559 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.109096+0000", 2024-09-14T06:30:48.560 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046266517 2024-09-14T06:30:48.560 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:48.560 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:48.560 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:30:48.560 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.109190+0000", 2024-09-14T06:30:48.560 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.4000999999999997e-05 2024-09-14T06:30:48.560 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:48.560 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:48.560 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:30:48.560 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.109297+0000", 2024-09-14T06:30:48.560 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.000107292 2024-09-14T06:30:48.560 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:48.560 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:30:48.560 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:48.561 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:48.561 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:48.561 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:19354 3.3 3:c34e5e51:::rbd_data.112d92027590.0000000000000024:f [read 1189888~1024 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:30:48.561 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:27.062842+0000", 2024-09-14T06:30:48.561 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 81.474341136999996, 2024-09-14T06:30:48.561 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046594257, 2024-09-14T06:30:48.561 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:30:48.561 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:30:48.561 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:30:48.561 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:30:48.561 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:30:48.561 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:48.561 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 19354 2024-09-14T06:30:48.561 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:48.562 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:30:48.562 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:48.562 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:30:48.562 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062842+0000", 2024-09-14T06:30:48.562 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:48.562 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:48.562 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:48.562 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:30:48.562 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062842+0000", 2024-09-14T06:30:48.562 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:48.562 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:48.562 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:48.562 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:30:48.562 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062842+0000", 2024-09-14T06:30:48.563 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.7399999999999999e-07 2024-09-14T06:30:48.563 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:48.563 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:48.563 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:30:48.563 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062859+0000", 2024-09-14T06:30:48.563 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.7010000000000001e-05 2024-09-14T06:30:48.563 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:48.563 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:48.563 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:30:48.563 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062866+0000", 2024-09-14T06:30:48.563 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.9990000000000002e-06 2024-09-14T06:30:48.564 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:48.564 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:48.564 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:30:48.564 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062873+0000", 2024-09-14T06:30:48.564 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.8569999999999997e-06 2024-09-14T06:30:48.564 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:48.564 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:48.564 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:30:48.564 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.109335+0000", 2024-09-14T06:30:48.564 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046461943999999998 2024-09-14T06:30:48.564 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:48.564 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:48.564 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:30:48.565 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.109377+0000", 2024-09-14T06:30:48.565 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.2558999999999997e-05 2024-09-14T06:30:48.565 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:48.565 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:48.565 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:30:48.565 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.109436+0000", 2024-09-14T06:30:48.565 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.8514000000000003e-05 2024-09-14T06:30:48.565 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:48.565 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:30:48.565 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:48.565 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:48.565 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:48.565 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:19355 3.3 3:c34e5e51:::rbd_data.112d92027590.0000000000000024:f [read 1200128~1024 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:30:48.565 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:27.062884+0000", 2024-09-14T06:30:48.565 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 81.474298464, 2024-09-14T06:30:48.566 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046681650999999998, 2024-09-14T06:30:48.566 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:30:48.566 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:30:48.566 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:30:48.566 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:30:48.566 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:30:48.566 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:48.566 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 19355 2024-09-14T06:30:48.566 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:48.566 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:30:48.566 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:48.566 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:30:48.566 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062884+0000", 2024-09-14T06:30:48.566 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:48.567 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:48.567 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:48.567 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:30:48.567 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062884+0000", 2024-09-14T06:30:48.567 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:48.567 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:48.567 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:48.567 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:30:48.567 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062885+0000", 2024-09-14T06:30:48.567 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.8000000000000001e-07 2024-09-14T06:30:48.567 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:48.567 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:48.567 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:30:48.567 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062886+0000", 2024-09-14T06:30:48.567 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.8920000000000001e-06 2024-09-14T06:30:48.567 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:48.568 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:48.568 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:30:48.568 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062892+0000", 2024-09-14T06:30:48.568 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.84e-06 2024-09-14T06:30:48.568 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:48.568 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:48.568 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:30:48.568 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062899+0000", 2024-09-14T06:30:48.568 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.5339999999999996e-06 2024-09-14T06:30:48.568 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:48.568 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:48.568 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:30:48.568 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.109466+0000", 2024-09-14T06:30:48.568 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046566919999999998 2024-09-14T06:30:48.569 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:48.569 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:48.569 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:30:48.569 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.109510+0000", 2024-09-14T06:30:48.569 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.4153000000000002e-05 2024-09-14T06:30:48.569 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:48.569 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:48.569 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:30:48.569 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.109566+0000", 2024-09-14T06:30:48.569 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.5931999999999999e-05 2024-09-14T06:30:48.569 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:48.569 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:30:48.569 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:48.569 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:48.570 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:48.570 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:19356 3.3 3:c34e5e51:::rbd_data.112d92027590.0000000000000024:f [read 1210368~1024 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:30:48.570 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:27.062908+0000", 2024-09-14T06:30:48.570 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 81.474275027000004, 2024-09-14T06:30:48.570 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046807498000000003, 2024-09-14T06:30:48.570 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:30:48.570 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:30:48.570 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:30:48.570 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:30:48.570 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:30:48.570 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:48.570 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 19356 2024-09-14T06:30:48.570 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:48.570 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:30:48.570 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:48.571 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:30:48.571 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062908+0000", 2024-09-14T06:30:48.571 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:48.571 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:48.571 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:48.571 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:30:48.571 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062908+0000", 2024-09-14T06:30:48.571 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:48.571 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:48.571 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:48.571 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:30:48.571 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062908+0000", 2024-09-14T06:30:48.571 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.5699999999999998e-07 2024-09-14T06:30:48.571 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:48.572 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:48.572 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:30:48.572 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062910+0000", 2024-09-14T06:30:48.572 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.7710000000000001e-06 2024-09-14T06:30:48.572 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:48.572 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:48.572 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:30:48.572 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062920+0000", 2024-09-14T06:30:48.572 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.0654e-05 2024-09-14T06:30:48.572 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:48.572 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:48.572 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:30:48.572 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062931+0000", 2024-09-14T06:30:48.572 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.075e-05 2024-09-14T06:30:48.573 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:48.573 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:48.573 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:30:48.573 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.109595+0000", 2024-09-14T06:30:48.573 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046663713000000002 2024-09-14T06:30:48.573 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:48.573 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:48.573 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:30:48.573 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.109632+0000", 2024-09-14T06:30:48.573 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.7505000000000001e-05 2024-09-14T06:30:48.573 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:48.573 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:48.573 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:30:48.574 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.109715+0000", 2024-09-14T06:30:48.574 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.2748000000000001e-05 2024-09-14T06:30:48.574 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:48.574 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:30:48.574 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:48.574 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:48.574 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:48.574 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:19357 3.3 3:c34e5e51:::rbd_data.112d92027590.0000000000000024:f [read 1220608~1024 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:30:48.574 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:27.062939+0000", 2024-09-14T06:30:48.574 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 81.474243579000003, 2024-09-14T06:30:48.574 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046920508, 2024-09-14T06:30:48.574 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:30:48.574 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:30:48.574 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:30:48.575 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:30:48.575 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:30:48.575 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:48.575 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 19357 2024-09-14T06:30:48.575 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:48.575 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:30:48.575 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:48.575 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:30:48.575 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062939+0000", 2024-09-14T06:30:48.575 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:48.575 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:48.575 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:48.575 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:30:48.575 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062939+0000", 2024-09-14T06:30:48.575 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:48.575 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:48.575 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:48.576 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:30:48.576 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062940+0000", 2024-09-14T06:30:48.576 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.32e-07 2024-09-14T06:30:48.576 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:48.576 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:48.576 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:30:48.576 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062942+0000", 2024-09-14T06:30:48.576 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.1919999999999999e-06 2024-09-14T06:30:48.576 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:48.576 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:48.576 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:30:48.576 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062948+0000", 2024-09-14T06:30:48.576 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.1149999999999999e-06 2024-09-14T06:30:48.576 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:48.577 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:48.577 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:30:48.577 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062960+0000", 2024-09-14T06:30:48.577 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.1895e-05 2024-09-14T06:30:48.577 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:48.577 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:48.577 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:30:48.577 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.109704+0000", 2024-09-14T06:30:48.577 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046744584999999998 2024-09-14T06:30:48.577 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:48.577 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:48.577 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:30:48.577 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.109777+0000", 2024-09-14T06:30:48.577 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.2282999999999999e-05 2024-09-14T06:30:48.577 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:48.578 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:48.578 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:30:48.578 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.109860+0000", 2024-09-14T06:30:48.578 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.3005999999999998e-05 2024-09-14T06:30:48.578 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:48.578 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:30:48.578 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:48.578 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:48.578 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:48.578 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:19358 3.3 3:c34e5e51:::rbd_data.112d92027590.0000000000000024:f [read 1230848~1024 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:30:48.578 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:27.062967+0000", 2024-09-14T06:30:48.578 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 81.474215948999998, 2024-09-14T06:30:48.578 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047144471, 2024-09-14T06:30:48.578 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:30:48.579 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:30:48.579 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:30:48.579 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:30:48.579 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:30:48.579 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:48.579 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 19358 2024-09-14T06:30:48.579 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:48.579 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:30:48.579 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:48.579 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:30:48.579 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062967+0000", 2024-09-14T06:30:48.579 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:48.579 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:48.579 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:48.579 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:30:48.580 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062967+0000", 2024-09-14T06:30:48.580 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:48.580 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:48.580 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:48.580 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:30:48.580 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062967+0000", 2024-09-14T06:30:48.580 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.89e-07 2024-09-14T06:30:48.580 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:48.580 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:48.580 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:30:48.580 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062969+0000", 2024-09-14T06:30:48.580 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.9929999999999998e-06 2024-09-14T06:30:48.580 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:48.580 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:48.581 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:30:48.581 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062975+0000", 2024-09-14T06:30:48.581 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.9030000000000001e-06 2024-09-14T06:30:48.581 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:48.581 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:48.581 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:30:48.581 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062979+0000", 2024-09-14T06:30:48.582 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.2250000000000002e-06 2024-09-14T06:30:48.582 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:48.582 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:48.582 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:30:48.582 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.109934+0000", 2024-09-14T06:30:48.582 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046954410000000002 2024-09-14T06:30:48.582 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:48.582 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:48.582 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:30:48.582 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.110007+0000", 2024-09-14T06:30:48.582 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.3829000000000001e-05 2024-09-14T06:30:48.582 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:48.582 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:48.582 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:30:48.583 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.110111+0000", 2024-09-14T06:30:48.583 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00010372199999999999 2024-09-14T06:30:48.583 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:48.583 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:30:48.583 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:48.583 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:48.583 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:48.583 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:19359 3.3 3:c34e5e51:::rbd_data.112d92027590.0000000000000024:f [read 1241088~1024 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:30:48.583 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:27.062987+0000", 2024-09-14T06:30:48.583 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 81.474195859000005, 2024-09-14T06:30:48.583 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047297182, 2024-09-14T06:30:48.583 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:30:48.583 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:30:48.583 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:30:48.583 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:30:48.584 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:30:48.584 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:48.584 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 19359 2024-09-14T06:30:48.584 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:48.584 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:30:48.584 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:48.584 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:30:48.584 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062987+0000", 2024-09-14T06:30:48.584 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:48.584 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:48.585 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:48.585 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:30:48.585 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062987+0000", 2024-09-14T06:30:48.585 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:48.585 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:48.585 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:48.585 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:30:48.585 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062987+0000", 2024-09-14T06:30:48.585 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.2500000000000001e-07 2024-09-14T06:30:48.585 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:48.585 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:48.585 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:30:48.585 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062989+0000", 2024-09-14T06:30:48.586 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.6080000000000001e-06 2024-09-14T06:30:48.586 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:48.586 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:48.586 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:30:48.586 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062994+0000", 2024-09-14T06:30:48.586 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.5520000000000004e-06 2024-09-14T06:30:48.586 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:48.586 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:48.586 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:30:48.586 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062999+0000", 2024-09-14T06:30:48.586 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.3930000000000001e-06 2024-09-14T06:30:48.586 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:48.586 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:48.586 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:30:48.586 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.110142+0000", 2024-09-14T06:30:48.587 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047143154 2024-09-14T06:30:48.587 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:48.587 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:48.587 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:30:48.587 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.110219+0000", 2024-09-14T06:30:48.587 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.6737999999999996e-05 2024-09-14T06:30:48.587 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:48.587 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:48.587 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:30:48.587 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.110284+0000", 2024-09-14T06:30:48.587 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.5412000000000003e-05 2024-09-14T06:30:48.587 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:48.587 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:30:48.588 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:48.588 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:48.588 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:48.588 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:19360 3.3 3:c34e5e51:::rbd_data.112d92027590.0000000000000024:f [read 1251328~1024 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:30:48.588 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:27.063017+0000", 2024-09-14T06:30:48.588 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 81.474165397999997, 2024-09-14T06:30:48.588 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047453823999999999, 2024-09-14T06:30:48.588 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:30:48.588 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:30:48.588 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:30:48.588 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:30:48.588 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:30:48.589 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:48.589 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 19360 2024-09-14T06:30:48.589 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:48.589 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:30:48.589 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:48.589 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:30:48.589 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.063017+0000", 2024-09-14T06:30:48.589 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:48.589 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:48.589 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:48.589 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:30:48.590 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.063017+0000", 2024-09-14T06:30:48.590 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:48.590 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:48.590 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:48.590 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:30:48.590 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.064977+0000", 2024-09-14T06:30:48.590 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.0019597080000000001 2024-09-14T06:30:48.590 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:48.590 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:48.590 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:30:48.590 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.064987+0000", 2024-09-14T06:30:48.590 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.9620000000000005e-06 2024-09-14T06:30:48.590 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:48.590 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:48.591 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:30:48.591 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065000+0000", 2024-09-14T06:30:48.591 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.3173000000000001e-05 2024-09-14T06:30:48.591 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:48.591 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:48.591 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:30:48.591 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065005+0000", 2024-09-14T06:30:48.591 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.9960000000000001e-06 2024-09-14T06:30:48.591 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:48.591 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:48.591 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:30:48.591 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.110314+0000", 2024-09-14T06:30:48.591 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.04530899 2024-09-14T06:30:48.591 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:48.592 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:48.592 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:30:48.592 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.110398+0000", 2024-09-14T06:30:48.592 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.3539000000000001e-05 2024-09-14T06:30:48.592 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:48.592 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:48.592 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:30:48.592 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.110471+0000", 2024-09-14T06:30:48.592 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.3455999999999996e-05 2024-09-14T06:30:48.592 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:48.592 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:30:48.593 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:48.593 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:48.593 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:48.593 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:19365 3.3 3:c34e5e51:::rbd_data.112d92027590.0000000000000024:f [read 1302528~1024 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:30:48.593 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:27.065058+0000", 2024-09-14T06:30:48.593 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 81.472124727999997, 2024-09-14T06:30:48.593 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046105526000000001, 2024-09-14T06:30:48.593 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:30:48.593 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:30:48.593 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:30:48.593 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:30:48.593 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:30:48.593 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:48.594 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 19365 2024-09-14T06:30:48.594 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:48.594 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:30:48.594 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:48.594 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:30:48.594 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065058+0000", 2024-09-14T06:30:48.594 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:48.594 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:48.594 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:48.594 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:30:48.594 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065058+0000", 2024-09-14T06:30:48.594 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:48.594 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:48.594 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:48.594 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:30:48.595 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065058+0000", 2024-09-14T06:30:48.595 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.1899999999999999e-07 2024-09-14T06:30:48.595 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:48.595 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:48.595 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:30:48.595 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065059+0000", 2024-09-14T06:30:48.595 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.1299999999999998e-07 2024-09-14T06:30:48.595 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:48.595 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:48.595 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:30:48.595 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065063+0000", 2024-09-14T06:30:48.595 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.1099999999999996e-06 2024-09-14T06:30:48.595 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:48.595 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:48.595 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:30:48.596 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065070+0000", 2024-09-14T06:30:48.596 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.973e-06 2024-09-14T06:30:48.596 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:48.596 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:48.596 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:30:48.596 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.111014+0000", 2024-09-14T06:30:48.596 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.045943373000000003 2024-09-14T06:30:48.596 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:48.596 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:48.596 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:30:48.596 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.111073+0000", 2024-09-14T06:30:48.596 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.8987999999999998e-05 2024-09-14T06:30:48.596 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:48.596 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:48.596 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:30:48.597 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.111163+0000", 2024-09-14T06:30:48.597 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.0950000000000002e-05 2024-09-14T06:30:48.597 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:48.597 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:30:48.597 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:48.597 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:48.597 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:48.597 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:19366 3.3 3:c34e5e51:::rbd_data.112d92027590.0000000000000024:f [read 1312768~1024 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:30:48.597 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:27.065074+0000", 2024-09-14T06:30:48.597 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 81.472108878, 2024-09-14T06:30:48.597 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046289858000000003, 2024-09-14T06:30:48.597 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:30:48.597 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:30:48.597 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:30:48.598 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:30:48.598 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:30:48.598 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:48.598 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 19366 2024-09-14T06:30:48.598 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:48.598 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:30:48.598 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:48.598 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:30:48.598 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065074+0000", 2024-09-14T06:30:48.598 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:48.598 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:48.598 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:48.598 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:30:48.599 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065074+0000", 2024-09-14T06:30:48.599 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:48.599 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:48.599 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:48.599 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:30:48.599 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065074+0000", 2024-09-14T06:30:48.599 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.3900000000000001e-07 2024-09-14T06:30:48.599 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:48.599 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:48.599 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:30:48.599 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065078+0000", 2024-09-14T06:30:48.599 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.3229999999999999e-06 2024-09-14T06:30:48.599 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:48.599 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:48.599 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:30:48.600 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065082+0000", 2024-09-14T06:30:48.600 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.8299999999999998e-06 2024-09-14T06:30:48.600 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:48.600 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:48.600 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:30:48.600 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065085+0000", 2024-09-14T06:30:48.600 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.8660000000000002e-06 2024-09-14T06:30:48.600 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:48.600 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:48.600 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:30:48.600 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.111270+0000", 2024-09-14T06:30:48.600 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046185035999999999 2024-09-14T06:30:48.601 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:48.601 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:48.601 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:30:48.601 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.111311+0000", 2024-09-14T06:30:48.601 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.0935e-05 2024-09-14T06:30:48.601 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:48.601 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:48.601 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:30:48.601 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.111364+0000", 2024-09-14T06:30:48.602 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.2629e-05 2024-09-14T06:30:48.602 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:48.602 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:30:48.602 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:48.602 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:48.602 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:48.602 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:19367 3.3 3:c34e5e51:::rbd_data.112d92027590.0000000000000024:f [read 1323008~1024 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:30:48.602 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:27.065115+0000", 2024-09-14T06:30:48.602 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 81.472067316999997, 2024-09-14T06:30:48.602 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046407003000000002, 2024-09-14T06:30:48.602 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:30:48.602 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:30:48.603 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:30:48.603 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:30:48.603 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:30:48.603 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:48.603 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 19367 2024-09-14T06:30:48.603 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:48.603 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:30:48.603 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:48.603 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:30:48.603 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065115+0000", 2024-09-14T06:30:48.604 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:48.604 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:48.604 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:48.604 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:30:48.604 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065115+0000", 2024-09-14T06:30:48.604 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:48.604 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:48.604 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:48.604 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:30:48.604 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065116+0000", 2024-09-14T06:30:48.604 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.4999999999999998e-07 2024-09-14T06:30:48.604 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:48.604 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:48.604 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:30:48.605 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065117+0000", 2024-09-14T06:30:48.605 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.189e-06 2024-09-14T06:30:48.605 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:48.605 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:48.605 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:30:48.605 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065122+0000", 2024-09-14T06:30:48.605 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.3709999999999999e-06 2024-09-14T06:30:48.605 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:48.605 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:48.605 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:30:48.605 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065125+0000", 2024-09-14T06:30:48.605 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.8940000000000002e-06 2024-09-14T06:30:48.605 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:48.605 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:48.606 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:30:48.606 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.111402+0000", 2024-09-14T06:30:48.606 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046276645999999998 2024-09-14T06:30:48.606 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:48.606 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:48.606 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:30:48.606 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.111445+0000", 2024-09-14T06:30:48.606 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.3288000000000003e-05 2024-09-14T06:30:48.606 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:48.606 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:48.606 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:30:48.606 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.111522+0000", 2024-09-14T06:30:48.606 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.7164999999999997e-05 2024-09-14T06:30:48.606 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:48.606 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:30:48.606 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:48.607 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:48.607 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:48.607 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:19368 3.3 3:c34e5e51:::rbd_data.112d92027590.0000000000000024:f [read 1333248~1024 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:30:48.607 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:27.065130+0000", 2024-09-14T06:30:48.607 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 81.472053027000001, 2024-09-14T06:30:48.607 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046507275000000001, 2024-09-14T06:30:48.607 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:30:48.607 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:30:48.607 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:30:48.607 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:30:48.607 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:30:48.607 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:48.607 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 19368 2024-09-14T06:30:48.608 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:48.608 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:30:48.608 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:48.608 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:30:48.608 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065130+0000", 2024-09-14T06:30:48.608 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:48.608 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:48.608 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:48.608 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:30:48.608 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065130+0000", 2024-09-14T06:30:48.608 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:48.608 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:48.609 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:48.609 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:30:48.609 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065130+0000", 2024-09-14T06:30:48.609 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.0800000000000001e-07 2024-09-14T06:30:48.609 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:48.609 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:48.609 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:30:48.609 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065131+0000", 2024-09-14T06:30:48.609 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.8700000000000004e-07 2024-09-14T06:30:48.609 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:48.609 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:48.609 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:30:48.610 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065134+0000", 2024-09-14T06:30:48.610 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.1039999999999998e-06 2024-09-14T06:30:48.610 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:48.610 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:48.610 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:30:48.610 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065136+0000", 2024-09-14T06:30:48.610 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.3949999999999999e-06 2024-09-14T06:30:48.610 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:48.610 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:48.610 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:30:48.610 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.111541+0000", 2024-09-14T06:30:48.610 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046404206000000003 2024-09-14T06:30:48.610 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:48.611 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:48.611 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:30:48.611 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.111582+0000", 2024-09-14T06:30:48.611 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.1051999999999999e-05 2024-09-14T06:30:48.611 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:48.611 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:48.611 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:30:48.611 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.111637+0000", 2024-09-14T06:30:48.611 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.5322999999999999e-05 2024-09-14T06:30:48.611 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:48.611 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:30:48.611 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:48.612 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:48.612 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:48.612 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:19369 3.3 3:c34e5e51:::rbd_data.112d92027590.0000000000000024:f [read 1343488~1024 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:30:48.612 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:27.065139+0000", 2024-09-14T06:30:48.612 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 81.472043209999995, 2024-09-14T06:30:48.612 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046682169000000003, 2024-09-14T06:30:48.612 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:30:48.612 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:30:48.612 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:30:48.612 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:30:48.612 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:30:48.612 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:48.612 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 19369 2024-09-14T06:30:48.613 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:48.613 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:30:48.613 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:48.613 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:30:48.613 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065139+0000", 2024-09-14T06:30:48.613 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:48.613 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:48.613 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:48.613 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:30:48.613 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065139+0000", 2024-09-14T06:30:48.613 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:48.613 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:48.613 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:48.613 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:30:48.614 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065140+0000", 2024-09-14T06:30:48.614 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.8299999999999998e-07 2024-09-14T06:30:48.614 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:48.614 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:48.614 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:30:48.614 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065141+0000", 2024-09-14T06:30:48.614 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.3900000000000003e-07 2024-09-14T06:30:48.614 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:48.614 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:48.614 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:30:48.614 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065144+0000", 2024-09-14T06:30:48.614 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.7409999999999998e-06 2024-09-14T06:30:48.614 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:48.615 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:48.615 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:30:48.615 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065147+0000", 2024-09-14T06:30:48.615 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.4150000000000002e-06 2024-09-14T06:30:48.615 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:48.615 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:48.615 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:30:48.615 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.111673+0000", 2024-09-14T06:30:48.615 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046526004000000003 2024-09-14T06:30:48.615 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:48.615 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:48.615 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:30:48.615 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.111749+0000", 2024-09-14T06:30:48.615 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.5741999999999994e-05 2024-09-14T06:30:48.615 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:48.616 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:48.616 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:30:48.616 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.111822+0000", 2024-09-14T06:30:48.616 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.3045000000000003e-05 2024-09-14T06:30:48.616 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:48.616 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:30:48.616 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:48.616 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:48.616 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:48.616 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:19370 3.3 3:c34e5e51:::rbd_data.112d92027590.0000000000000024:f [read 1353728~1024 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:30:48.616 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:27.065150+0000", 2024-09-14T06:30:48.616 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 81.472032480999999, 2024-09-14T06:30:48.616 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046897556999999999, 2024-09-14T06:30:48.617 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:30:48.617 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:30:48.617 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:30:48.617 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:30:48.617 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:30:48.617 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:48.617 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 19370 2024-09-14T06:30:48.617 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:48.617 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:30:48.617 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:48.618 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:30:48.618 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065150+0000", 2024-09-14T06:30:48.618 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:48.618 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:48.618 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:48.618 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:30:48.618 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065150+0000", 2024-09-14T06:30:48.618 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:48.618 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:48.618 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:48.618 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:30:48.618 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065150+0000", 2024-09-14T06:30:48.619 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.0200000000000001e-07 2024-09-14T06:30:48.619 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:48.619 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:48.619 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:30:48.619 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065151+0000", 2024-09-14T06:30:48.619 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.1100000000000004e-07 2024-09-14T06:30:48.619 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:48.619 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:48.619 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:30:48.619 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065154+0000", 2024-09-14T06:30:48.619 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.9639999999999999e-06 2024-09-14T06:30:48.620 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:48.620 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:48.620 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:30:48.620 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065157+0000", 2024-09-14T06:30:48.620 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.4260000000000002e-06 2024-09-14T06:30:48.620 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:48.620 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:48.620 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:30:48.620 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.111861+0000", 2024-09-14T06:30:48.620 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046704268 2024-09-14T06:30:48.620 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:48.620 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:48.621 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:30:48.621 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.111939+0000", 2024-09-14T06:30:48.621 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.8231000000000004e-05 2024-09-14T06:30:48.621 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:48.621 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:48.621 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:30:48.621 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.112048+0000", 2024-09-14T06:30:48.621 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00010855500000000001 2024-09-14T06:30:48.621 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:48.621 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:30:48.621 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:48.622 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:48.622 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:48.622 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:19371 3.3 3:c34e5e51:::rbd_data.112d92027590.0000000000000024:f [read 1363968~1024 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:30:48.622 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:27.065161+0000", 2024-09-14T06:30:48.622 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 81.472021831000006, 2024-09-14T06:30:48.622 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047021944000000003, 2024-09-14T06:30:48.622 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:30:48.622 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:30:48.622 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:30:48.622 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:30:48.622 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:30:48.622 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:48.622 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 19371 2024-09-14T06:30:48.622 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:48.622 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:30:48.623 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:48.623 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:30:48.623 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065161+0000", 2024-09-14T06:30:48.623 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:48.623 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:48.623 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:48.623 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:30:48.623 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065161+0000", 2024-09-14T06:30:48.623 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:48.623 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:48.623 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:48.623 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:30:48.623 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065161+0000", 2024-09-14T06:30:48.623 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.0200000000000001e-07 2024-09-14T06:30:48.624 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:48.624 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:48.624 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:30:48.624 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065166+0000", 2024-09-14T06:30:48.624 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.9869999999999999e-06 2024-09-14T06:30:48.624 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:48.624 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:48.624 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:30:48.624 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065169+0000", 2024-09-14T06:30:48.624 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.3799999999999998e-06 2024-09-14T06:30:48.624 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:48.624 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:48.624 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:30:48.624 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065172+0000", 2024-09-14T06:30:48.625 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.4830000000000002e-06 2024-09-14T06:30:48.625 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:48.625 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:48.625 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:30:48.625 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.112071+0000", 2024-09-14T06:30:48.625 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046899529000000002 2024-09-14T06:30:48.625 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:48.625 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:48.625 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:30:48.625 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.112128+0000", 2024-09-14T06:30:48.625 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.6341999999999997e-05 2024-09-14T06:30:48.625 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:48.625 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:48.625 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:30:48.625 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.112183+0000", 2024-09-14T06:30:48.626 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.5021000000000001e-05 2024-09-14T06:30:48.626 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:48.626 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:30:48.626 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:48.626 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:48.626 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:48.626 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:19372 3.3 3:c34e5e51:::rbd_data.112d92027590.0000000000000024:f [read 1374208~1024 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:30:48.626 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:27.065176+0000", 2024-09-14T06:30:48.626 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 81.472006680999996, 2024-09-14T06:30:48.626 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047188889999999997, 2024-09-14T06:30:48.626 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:30:48.626 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:30:48.626 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:30:48.626 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:30:48.626 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:30:48.627 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:48.627 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 19372 2024-09-14T06:30:48.627 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:48.627 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:30:48.627 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:48.627 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:30:48.627 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065176+0000", 2024-09-14T06:30:48.627 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:48.627 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:48.627 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:48.627 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:30:48.627 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065176+0000", 2024-09-14T06:30:48.627 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:48.627 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:48.628 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:48.628 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:30:48.628 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.071117+0000", 2024-09-14T06:30:48.628 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.0059404990000000001 2024-09-14T06:30:48.628 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:48.628 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:48.628 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:30:48.628 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.071120+0000", 2024-09-14T06:30:48.628 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.4340000000000001e-06 2024-09-14T06:30:48.628 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:48.628 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:48.628 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:30:48.628 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.071129+0000", 2024-09-14T06:30:48.628 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.8389999999999996e-06 2024-09-14T06:30:48.628 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:48.629 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:48.629 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:30:48.629 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.071134+0000", 2024-09-14T06:30:48.629 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.412e-06 2024-09-14T06:30:48.629 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:48.629 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:48.629 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:30:48.629 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.112221+0000", 2024-09-14T06:30:48.629 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.041086473999999998 2024-09-14T06:30:48.629 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:48.629 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:48.629 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:30:48.629 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.112299+0000", 2024-09-14T06:30:48.630 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.8746999999999997e-05 2024-09-14T06:30:48.630 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:48.630 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:48.630 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:30:48.630 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.112365+0000", 2024-09-14T06:30:48.630 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.5485000000000001e-05 2024-09-14T06:30:48.630 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:48.630 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:30:48.630 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:48.630 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:48.630 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:30:48.630 INFO:teuthology.orchestra.run.smithi138.stdout:} 2024-09-14T06:30:48.632 INFO:tasks.ceph.osd.0.smithi094.stderr:2024-09-14T06:30:48.630+0000 7f07c9333640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33636) UID: 0 2024-09-14T06:30:48.632 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-14T06:30:48.721 INFO:teuthology.orchestra.run.smithi094.stderr:noscrub is set 2024-09-14T06:30:48.732 INFO:tasks.ceph.osd.0.smithi094.stderr:2024-09-14T06:30:48.731+0000 7f07c9333640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33636) UID: 0 2024-09-14T06:30:48.779 INFO:teuthology.orchestra.run.smithi138.stdout:{ 2024-09-14T06:30:48.780 INFO:teuthology.orchestra.run.smithi138.stdout: "ops": [], 2024-09-14T06:30:48.780 INFO:teuthology.orchestra.run.smithi138.stdout: "num_ops": 0 2024-09-14T06:30:48.780 INFO:teuthology.orchestra.run.smithi138.stdout:} 2024-09-14T06:30:48.789 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-14T06:30:48.832 INFO:teuthology.orchestra.run.smithi094.stdout:checking clone #26, image image_client.0-clone26 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent27 2024-09-14T06:30:48.833 INFO:tasks.ceph.osd.7.smithi138.stderr:2024-09-14T06:30:48.832+0000 7f3c6a21d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33623) UID: 0 2024-09-14T06:30:48.903 INFO:teuthology.orchestra.run.smithi138.stdout:{ 2024-09-14T06:30:48.903 INFO:teuthology.orchestra.run.smithi138.stdout: "ops": [], 2024-09-14T06:30:48.903 INFO:teuthology.orchestra.run.smithi138.stdout: "complaint_time": 30, 2024-09-14T06:30:48.903 INFO:teuthology.orchestra.run.smithi138.stdout: "num_blocked_ops": 0 2024-09-14T06:30:48.903 INFO:teuthology.orchestra.run.smithi138.stdout:} 2024-09-14T06:30:48.912 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-14T06:30:48.953 INFO:tasks.ceph.osd.6.smithi138.stderr:2024-09-14T06:30:48.952+0000 7f8cd8849640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33620) UID: 0 2024-09-14T06:30:49.034 INFO:tasks.ceph.osd.7.smithi138.stderr:2024-09-14T06:30:49.033+0000 7f3c6a21d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33623) UID: 0 2024-09-14T06:30:49.057 INFO:teuthology.orchestra.run.smithi138.stdout:{ 2024-09-14T06:30:49.057 INFO:teuthology.orchestra.run.smithi138.stdout: "size": 20, 2024-09-14T06:30:49.057 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 600, 2024-09-14T06:30:49.057 INFO:teuthology.orchestra.run.smithi138.stdout: "ops": [ 2024-09-14T06:30:49.058 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:49.058 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:11453 3.1d 3:bcad7303:::rbd_data.112d92027590.0000000000000001:f [read 94208~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:30:49.058 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:25.654375+0000", 2024-09-14T06:30:49.058 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 83.401220584000001, 2024-09-14T06:30:49.058 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052449318000000002, 2024-09-14T06:30:49.058 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:30:49.058 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:30:49.058 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:30:49.058 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:30:49.058 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:30:49.058 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:49.059 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 11453 2024-09-14T06:30:49.059 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:49.059 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:30:49.059 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:49.059 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:30:49.059 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654375+0000", 2024-09-14T06:30:49.059 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:49.059 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:49.059 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:49.059 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:30:49.059 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654375+0000", 2024-09-14T06:30:49.059 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:49.059 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:49.060 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:49.060 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:30:49.060 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654496+0000", 2024-09-14T06:30:49.060 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00012081 2024-09-14T06:30:49.060 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:49.060 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:49.060 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:30:49.060 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654497+0000", 2024-09-14T06:30:49.060 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.8789999999999999e-06 2024-09-14T06:30:49.060 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:49.060 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:49.060 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:30:49.060 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654501+0000", 2024-09-14T06:30:49.061 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.8999999999999999e-06 2024-09-14T06:30:49.061 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:49.061 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:49.061 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:30:49.061 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654505+0000", 2024-09-14T06:30:49.061 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.2229999999999999e-06 2024-09-14T06:30:49.061 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:49.061 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:49.061 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:30:49.061 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.705917+0000", 2024-09-14T06:30:49.061 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.051412501999999999 2024-09-14T06:30:49.062 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:49.062 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:49.062 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:30:49.062 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.705999+0000", 2024-09-14T06:30:49.062 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.1551999999999994e-05 2024-09-14T06:30:49.062 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:49.062 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:49.062 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:30:49.062 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.706824+0000", 2024-09-14T06:30:49.062 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00082545199999999996 2024-09-14T06:30:49.062 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:49.063 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:30:49.063 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:49.063 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:49.063 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:49.063 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:11454 3.1d 3:bcad7303:::rbd_data.112d92027590.0000000000000001:f [read 104448~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:30:49.063 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:25.654510+0000", 2024-09-14T06:30:49.063 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 83.401085793999997, 2024-09-14T06:30:49.063 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052610713000000003, 2024-09-14T06:30:49.063 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:30:49.063 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:30:49.063 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:30:49.064 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:30:49.064 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:30:49.064 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:49.064 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 11454 2024-09-14T06:30:49.064 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:49.064 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:30:49.064 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:49.064 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:30:49.064 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654510+0000", 2024-09-14T06:30:49.065 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:49.065 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:49.065 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:49.065 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:30:49.065 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654510+0000", 2024-09-14T06:30:49.065 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:49.065 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:49.065 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:49.065 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:30:49.065 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654825+0000", 2024-09-14T06:30:49.065 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00031522400000000002 2024-09-14T06:30:49.065 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:49.066 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:49.066 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:30:49.066 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654827+0000", 2024-09-14T06:30:49.066 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.9709999999999998e-06 2024-09-14T06:30:49.066 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:49.066 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:49.066 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:30:49.066 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654832+0000", 2024-09-14T06:30:49.066 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.4099999999999999e-06 2024-09-14T06:30:49.066 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:49.066 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:49.066 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:30:49.067 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654835+0000", 2024-09-14T06:30:49.067 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.14e-06 2024-09-14T06:30:49.067 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:49.067 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:49.067 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:30:49.067 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.706851+0000", 2024-09-14T06:30:49.067 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052015761000000001 2024-09-14T06:30:49.067 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:49.067 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:49.067 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:30:49.067 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.706934+0000", 2024-09-14T06:30:49.067 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.2484999999999994e-05 2024-09-14T06:30:49.068 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:49.068 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:49.068 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:30:49.068 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.707120+0000", 2024-09-14T06:30:49.068 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00018672200000000001 2024-09-14T06:30:49.068 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:49.068 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:30:49.068 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:49.068 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:49.068 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:49.068 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:11455 3.1d 3:bcad7303:::rbd_data.112d92027590.0000000000000001:f [read 114688~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:30:49.068 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:25.654841+0000", 2024-09-14T06:30:49.068 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 83.400754465000006, 2024-09-14T06:30:49.068 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052579898, 2024-09-14T06:30:49.069 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:30:49.069 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:30:49.069 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:30:49.069 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:30:49.069 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:30:49.069 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:49.069 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 11455 2024-09-14T06:30:49.069 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:49.069 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:30:49.069 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:49.069 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:30:49.069 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654841+0000", 2024-09-14T06:30:49.069 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:49.069 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:49.070 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:49.070 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:30:49.070 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654841+0000", 2024-09-14T06:30:49.070 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:49.070 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:49.070 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:49.070 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:30:49.070 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654841+0000", 2024-09-14T06:30:49.070 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.7300000000000002e-07 2024-09-14T06:30:49.070 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:49.070 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:49.070 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:30:49.070 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654845+0000", 2024-09-14T06:30:49.071 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.0520000000000003e-06 2024-09-14T06:30:49.071 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:49.071 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:49.071 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:30:49.071 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654849+0000", 2024-09-14T06:30:49.071 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.0459999999999999e-06 2024-09-14T06:30:49.071 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:49.071 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:49.071 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:30:49.071 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654852+0000", 2024-09-14T06:30:49.071 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.9830000000000001e-06 2024-09-14T06:30:49.071 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:49.071 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:49.071 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:30:49.071 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.707159+0000", 2024-09-14T06:30:49.072 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052306826000000001 2024-09-14T06:30:49.072 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:49.072 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:49.072 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:30:49.072 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.707242+0000", 2024-09-14T06:30:49.072 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.2595000000000005e-05 2024-09-14T06:30:49.072 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:49.072 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:49.072 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:30:49.072 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.707421+0000", 2024-09-14T06:30:49.072 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.000179123 2024-09-14T06:30:49.072 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:49.072 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:30:49.072 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:49.073 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:49.073 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:49.073 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:11456 3.1d 3:bcad7303:::rbd_data.112d92027590.0000000000000001:f [read 124928~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:30:49.073 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:25.654856+0000", 2024-09-14T06:30:49.073 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 83.400739619000007, 2024-09-14T06:30:49.073 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052801580000000001, 2024-09-14T06:30:49.073 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:30:49.073 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:30:49.073 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:30:49.073 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:30:49.073 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:30:49.073 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:49.073 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 11456 2024-09-14T06:30:49.073 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:49.074 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:30:49.074 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:49.074 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:30:49.074 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654856+0000", 2024-09-14T06:30:49.074 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:49.074 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:49.074 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:49.074 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:30:49.074 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654856+0000", 2024-09-14T06:30:49.074 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:49.074 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:49.074 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:49.074 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:30:49.075 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654856+0000", 2024-09-14T06:30:49.075 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.29e-07 2024-09-14T06:30:49.075 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:49.075 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:49.075 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:30:49.075 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654857+0000", 2024-09-14T06:30:49.075 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.187e-06 2024-09-14T06:30:49.075 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:49.075 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:49.075 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:30:49.075 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654861+0000", 2024-09-14T06:30:49.075 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.3850000000000002e-06 2024-09-14T06:30:49.075 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:49.075 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:49.076 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:30:49.076 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654863+0000", 2024-09-14T06:30:49.076 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.9500000000000001e-06 2024-09-14T06:30:49.076 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:49.076 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:49.076 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:30:49.076 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.707456+0000", 2024-09-14T06:30:49.076 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052592434 2024-09-14T06:30:49.076 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:49.076 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:49.076 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:30:49.076 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.707516+0000", 2024-09-14T06:30:49.076 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.0167999999999997e-05 2024-09-14T06:30:49.077 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:49.077 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:49.077 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:30:49.077 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.707657+0000", 2024-09-14T06:30:49.077 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00014122700000000001 2024-09-14T06:30:49.077 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:49.077 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:30:49.077 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:49.077 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:49.077 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:49.077 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:11457 3.1d 3:bcad7303:::rbd_data.112d92027590.0000000000000001:f [read 135168~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:30:49.077 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:25.654867+0000", 2024-09-14T06:30:49.077 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 83.400728733999998, 2024-09-14T06:30:49.077 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052921902, 2024-09-14T06:30:49.078 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:30:49.078 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:30:49.078 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:30:49.078 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:30:49.078 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:30:49.078 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:49.078 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 11457 2024-09-14T06:30:49.078 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:49.078 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:30:49.078 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:49.078 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:30:49.078 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654867+0000", 2024-09-14T06:30:49.078 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:49.079 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:49.079 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:49.079 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:30:49.079 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654867+0000", 2024-09-14T06:30:49.079 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:49.079 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:49.079 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:49.079 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:30:49.079 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654972+0000", 2024-09-14T06:30:49.079 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00010584 2024-09-14T06:30:49.079 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:49.079 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:49.079 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:30:49.080 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654975+0000", 2024-09-14T06:30:49.080 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.1169999999999998e-06 2024-09-14T06:30:49.080 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:49.080 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:49.080 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:30:49.080 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654979+0000", 2024-09-14T06:30:49.080 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.1960000000000001e-06 2024-09-14T06:30:49.080 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:49.080 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:49.080 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:30:49.080 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654982+0000", 2024-09-14T06:30:49.080 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.2990000000000001e-06 2024-09-14T06:30:49.081 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:49.081 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:49.081 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:30:49.081 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.707684+0000", 2024-09-14T06:30:49.081 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052701979000000003 2024-09-14T06:30:49.081 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:49.081 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:49.081 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:30:49.081 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.707728+0000", 2024-09-14T06:30:49.081 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.3529999999999998e-05 2024-09-14T06:30:49.081 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:49.081 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:49.081 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:30:49.082 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.707789+0000", 2024-09-14T06:30:49.082 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.0940999999999998e-05 2024-09-14T06:30:49.082 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:49.082 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:30:49.082 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:49.082 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:49.082 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:49.082 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:11458 3.1d 3:bcad7303:::rbd_data.112d92027590.0000000000000001:f [read 145408~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:30:49.082 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:25.654987+0000", 2024-09-14T06:30:49.082 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 83.400608336000005, 2024-09-14T06:30:49.082 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.05291655, 2024-09-14T06:30:49.083 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:30:49.083 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:30:49.083 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:30:49.083 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:30:49.083 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:30:49.083 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:49.083 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 11458 2024-09-14T06:30:49.083 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:49.083 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:30:49.083 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:49.083 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:30:49.083 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654987+0000", 2024-09-14T06:30:49.083 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:49.083 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:49.083 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:49.084 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:30:49.084 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654987+0000", 2024-09-14T06:30:49.084 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:49.084 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:49.084 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:49.084 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:30:49.084 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655077+0000", 2024-09-14T06:30:49.084 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.0253000000000005e-05 2024-09-14T06:30:49.084 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:49.084 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:49.084 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:30:49.084 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655079+0000", 2024-09-14T06:30:49.084 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.9989999999999998e-06 2024-09-14T06:30:49.085 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:49.085 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:49.085 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:30:49.085 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655083+0000", 2024-09-14T06:30:49.085 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.9280000000000003e-06 2024-09-14T06:30:49.085 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:49.085 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:49.085 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:30:49.085 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655086+0000", 2024-09-14T06:30:49.085 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.056e-06 2024-09-14T06:30:49.085 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:49.086 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:49.086 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:30:49.086 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.707805+0000", 2024-09-14T06:30:49.086 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052718627999999997 2024-09-14T06:30:49.086 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:49.086 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:49.086 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:30:49.086 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.707848+0000", 2024-09-14T06:30:49.086 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.2728999999999997e-05 2024-09-14T06:30:49.086 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:49.086 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:49.086 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:30:49.086 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.707904+0000", 2024-09-14T06:30:49.086 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.5957e-05 2024-09-14T06:30:49.086 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:49.087 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:30:49.087 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:49.087 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:49.087 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:49.087 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:11459 3.1d 3:bcad7303:::rbd_data.112d92027590.0000000000000001:f [read 155648~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:30:49.087 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:25.655091+0000", 2024-09-14T06:30:49.087 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 83.400504327999997, 2024-09-14T06:30:49.087 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052944242000000002, 2024-09-14T06:30:49.087 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:30:49.087 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:30:49.087 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:30:49.087 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:30:49.087 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:30:49.087 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:49.088 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 11459 2024-09-14T06:30:49.088 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:49.088 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:30:49.088 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:49.088 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:30:49.088 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655091+0000", 2024-09-14T06:30:49.088 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:49.088 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:49.088 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:49.088 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:30:49.088 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655091+0000", 2024-09-14T06:30:49.088 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:49.088 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:49.088 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:49.089 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:30:49.089 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655237+0000", 2024-09-14T06:30:49.089 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00014570100000000001 2024-09-14T06:30:49.089 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:49.089 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:49.089 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:30:49.089 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655239+0000", 2024-09-14T06:30:49.089 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.8670000000000001e-06 2024-09-14T06:30:49.089 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:49.089 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:49.089 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:30:49.089 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655243+0000", 2024-09-14T06:30:49.089 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.9829999999999998e-06 2024-09-14T06:30:49.089 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:49.090 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:49.090 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:30:49.090 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655246+0000", 2024-09-14T06:30:49.090 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.1540000000000002e-06 2024-09-14T06:30:49.090 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:49.090 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:49.090 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:30:49.090 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.707934+0000", 2024-09-14T06:30:49.090 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052688286000000001 2024-09-14T06:30:49.090 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:49.090 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:49.090 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:30:49.090 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.707977+0000", 2024-09-14T06:30:49.090 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.3291999999999998e-05 2024-09-14T06:30:49.090 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:49.091 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:49.091 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:30:49.091 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.708035+0000", 2024-09-14T06:30:49.091 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.7958999999999999e-05 2024-09-14T06:30:49.091 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:49.091 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:30:49.091 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:49.091 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:49.091 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:49.091 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:11460 3.1d 3:bcad7303:::rbd_data.112d92027590.0000000000000001:f [read 165888~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:30:49.091 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:25.655250+0000", 2024-09-14T06:30:49.091 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 83.400345031000001, 2024-09-14T06:30:49.091 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052910698999999999, 2024-09-14T06:30:49.091 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:30:49.091 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:30:49.092 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:30:49.092 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:30:49.092 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:30:49.092 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:49.092 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 11460 2024-09-14T06:30:49.092 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:49.092 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:30:49.092 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:49.092 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:30:49.092 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655250+0000", 2024-09-14T06:30:49.092 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:49.092 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:49.092 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:49.093 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:30:49.093 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655250+0000", 2024-09-14T06:30:49.093 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:49.093 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:49.093 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:49.093 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:30:49.093 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655356+0000", 2024-09-14T06:30:49.093 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00010535900000000001 2024-09-14T06:30:49.093 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:49.093 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:49.093 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:30:49.094 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655357+0000", 2024-09-14T06:30:49.094 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.8080000000000001e-06 2024-09-14T06:30:49.094 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:49.094 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:49.094 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:30:49.094 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655361+0000", 2024-09-14T06:30:49.094 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.861e-06 2024-09-14T06:30:49.094 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:49.094 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:49.094 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:30:49.094 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655364+0000", 2024-09-14T06:30:49.094 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.1089999999999998e-06 2024-09-14T06:30:49.094 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:49.095 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:49.095 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:30:49.095 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.708065+0000", 2024-09-14T06:30:49.095 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052700595000000003 2024-09-14T06:30:49.095 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:49.095 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:49.095 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:30:49.095 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.708108+0000", 2024-09-14T06:30:49.095 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.2484999999999997e-05 2024-09-14T06:30:49.095 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:49.095 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:49.095 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:30:49.095 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.708161+0000", 2024-09-14T06:30:49.095 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.3482000000000001e-05 2024-09-14T06:30:49.096 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:49.096 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:30:49.096 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:49.096 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:49.096 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:49.096 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:11461 3.1d 3:bcad7303:::rbd_data.112d92027590.0000000000000001:f [read 176128~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:30:49.096 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:25.655370+0000", 2024-09-14T06:30:49.096 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 83.400225749000001, 2024-09-14T06:30:49.096 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052915141999999998, 2024-09-14T06:30:49.096 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:30:49.096 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:30:49.096 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:30:49.096 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:30:49.096 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:30:49.096 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:49.097 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 11461 2024-09-14T06:30:49.097 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:49.097 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:30:49.097 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:49.097 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:30:49.097 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655370+0000", 2024-09-14T06:30:49.097 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:49.097 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:49.097 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:49.097 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:30:49.097 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655370+0000", 2024-09-14T06:30:49.097 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:49.097 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:49.097 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:49.097 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:30:49.098 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655394+0000", 2024-09-14T06:30:49.098 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.4553999999999999e-05 2024-09-14T06:30:49.098 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:49.098 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:49.098 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:30:49.098 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655396+0000", 2024-09-14T06:30:49.098 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.8950000000000001e-06 2024-09-14T06:30:49.098 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:49.098 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:49.098 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:30:49.098 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655401+0000", 2024-09-14T06:30:49.099 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.0089999999999998e-06 2024-09-14T06:30:49.099 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:49.099 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:49.099 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:30:49.099 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655404+0000", 2024-09-14T06:30:49.099 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.3400000000000002e-06 2024-09-14T06:30:49.099 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:49.099 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:49.099 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:30:49.099 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.708186+0000", 2024-09-14T06:30:49.099 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052781726000000001 2024-09-14T06:30:49.099 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:49.100 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:49.100 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:30:49.100 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.708229+0000", 2024-09-14T06:30:49.100 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.2670999999999999e-05 2024-09-14T06:30:49.100 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:49.100 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:49.100 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:30:49.100 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.708285+0000", 2024-09-14T06:30:49.101 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.5946999999999998e-05 2024-09-14T06:30:49.101 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:49.101 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:30:49.101 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:49.101 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:49.101 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:49.101 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:11462 3.1d 3:bcad7303:::rbd_data.112d92027590.0000000000000001:f [read 186368~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:30:49.101 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:25.655410+0000", 2024-09-14T06:30:49.101 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 83.400185270999998, 2024-09-14T06:30:49.101 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052995258000000003, 2024-09-14T06:30:49.101 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:30:49.101 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:30:49.101 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:30:49.102 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:30:49.102 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:30:49.102 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:49.102 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 11462 2024-09-14T06:30:49.102 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:49.102 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:30:49.102 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:49.102 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:30:49.102 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655410+0000", 2024-09-14T06:30:49.102 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:49.102 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:49.102 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:49.102 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:30:49.103 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655410+0000", 2024-09-14T06:30:49.103 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:49.103 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:49.103 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:49.103 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:30:49.103 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655625+0000", 2024-09-14T06:30:49.103 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00021442600000000001 2024-09-14T06:30:49.103 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:49.103 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:49.103 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:30:49.103 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655626+0000", 2024-09-14T06:30:49.104 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.815e-06 2024-09-14T06:30:49.104 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:49.104 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:49.104 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:30:49.104 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655631+0000", 2024-09-14T06:30:49.104 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.651e-06 2024-09-14T06:30:49.104 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:49.104 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:49.104 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:30:49.104 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655634+0000", 2024-09-14T06:30:49.104 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.2449999999999998e-06 2024-09-14T06:30:49.104 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:49.105 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:49.105 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:30:49.105 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.708312+0000", 2024-09-14T06:30:49.105 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052677412 2024-09-14T06:30:49.105 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:49.105 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:49.105 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:30:49.105 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.708352+0000", 2024-09-14T06:30:49.105 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.0729999999999998e-05 2024-09-14T06:30:49.105 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:49.105 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:49.105 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:30:49.105 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.708405+0000", 2024-09-14T06:30:49.106 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.2979000000000002e-05 2024-09-14T06:30:49.106 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:49.106 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:30:49.106 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:49.106 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:49.106 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:49.106 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:11463 3.1d 3:bcad7303:::rbd_data.112d92027590.0000000000000001:f [read 196608~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:30:49.106 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:25.655639+0000", 2024-09-14T06:30:49.106 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 83.399956125000003, 2024-09-14T06:30:49.106 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052888859000000003, 2024-09-14T06:30:49.106 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:30:49.107 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:30:49.107 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:30:49.107 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:30:49.107 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:30:49.107 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:49.107 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 11463 2024-09-14T06:30:49.107 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:49.107 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:30:49.107 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:49.107 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:30:49.107 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655639+0000", 2024-09-14T06:30:49.107 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:49.107 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:49.108 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:49.108 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:30:49.108 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655639+0000", 2024-09-14T06:30:49.108 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:49.108 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:49.108 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:49.108 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:30:49.108 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655640+0000", 2024-09-14T06:30:49.108 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.08e-07 2024-09-14T06:30:49.108 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:49.108 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:49.108 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:30:49.109 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655641+0000", 2024-09-14T06:30:49.109 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.156e-06 2024-09-14T06:30:49.109 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:49.109 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:49.109 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:30:49.109 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655648+0000", 2024-09-14T06:30:49.109 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.8460000000000002e-06 2024-09-14T06:30:49.109 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:49.109 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:49.109 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:30:49.109 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655651+0000", 2024-09-14T06:30:49.109 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.0079999999999998e-06 2024-09-14T06:30:49.110 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:49.110 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:49.110 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:30:49.110 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.708429+0000", 2024-09-14T06:30:49.110 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052778459 2024-09-14T06:30:49.110 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:49.110 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:49.110 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:30:49.110 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.708473+0000", 2024-09-14T06:30:49.110 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.3782999999999997e-05 2024-09-14T06:30:49.110 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:49.110 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:49.110 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:30:49.111 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.708528+0000", 2024-09-14T06:30:49.111 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.5198999999999998e-05 2024-09-14T06:30:49.111 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:49.111 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:30:49.111 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:49.111 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:49.111 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:49.111 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:11464 3.1d 3:bcad7303:::rbd_data.112d92027590.0000000000000001:f [read 206848~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:30:49.111 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:25.655654+0000", 2024-09-14T06:30:49.111 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 83.399941256000005, 2024-09-14T06:30:49.111 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052981845999999999, 2024-09-14T06:30:49.111 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:30:49.111 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:30:49.111 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:30:49.111 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:30:49.112 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:30:49.112 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:49.112 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 11464 2024-09-14T06:30:49.112 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:49.112 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:30:49.112 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:49.112 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:30:49.112 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655654+0000", 2024-09-14T06:30:49.112 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:49.112 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:49.112 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:49.112 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:30:49.112 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655654+0000", 2024-09-14T06:30:49.113 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:49.113 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:49.113 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:49.113 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:30:49.113 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655946+0000", 2024-09-14T06:30:49.113 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00029148200000000003 2024-09-14T06:30:49.113 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:49.113 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:49.113 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:30:49.113 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655948+0000", 2024-09-14T06:30:49.113 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.973e-06 2024-09-14T06:30:49.113 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:49.113 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:49.113 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:30:49.113 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655953+0000", 2024-09-14T06:30:49.114 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.1240000000000004e-06 2024-09-14T06:30:49.114 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:49.114 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:49.114 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:30:49.114 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655956+0000", 2024-09-14T06:30:49.114 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.315e-06 2024-09-14T06:30:49.114 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:49.114 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:49.114 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:30:49.114 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.708543+0000", 2024-09-14T06:30:49.114 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052587416999999997 2024-09-14T06:30:49.114 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:49.114 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:49.114 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:30:49.114 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.708584+0000", 2024-09-14T06:30:49.115 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.1004000000000002e-05 2024-09-14T06:30:49.115 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:49.115 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:49.115 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:30:49.115 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.708636+0000", 2024-09-14T06:30:49.115 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.1530999999999998e-05 2024-09-14T06:30:49.115 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:49.115 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:30:49.115 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:49.115 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:49.115 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:49.115 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:11465 3.1d 3:bcad7303:::rbd_data.112d92027590.0000000000000001:f [read 217088~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:30:49.115 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:25.655961+0000", 2024-09-14T06:30:49.115 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 83.399634104, 2024-09-14T06:30:49.115 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052798682999999999, 2024-09-14T06:30:49.116 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:30:49.116 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:30:49.116 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:30:49.116 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:30:49.116 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:30:49.116 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:49.116 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 11465 2024-09-14T06:30:49.116 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:49.116 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:30:49.116 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:49.116 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:30:49.116 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655961+0000", 2024-09-14T06:30:49.117 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:49.117 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:49.117 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:49.117 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:30:49.117 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655961+0000", 2024-09-14T06:30:49.117 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:49.117 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:49.117 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:49.117 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:30:49.117 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.656114+0000", 2024-09-14T06:30:49.117 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.000152745 2024-09-14T06:30:49.117 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:49.117 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:49.118 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:30:49.118 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.656116+0000", 2024-09-14T06:30:49.118 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.804e-06 2024-09-14T06:30:49.118 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:49.118 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:49.118 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:30:49.118 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.656120+0000", 2024-09-14T06:30:49.118 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.4700000000000004e-06 2024-09-14T06:30:49.118 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:49.118 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:49.118 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:30:49.118 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.656123+0000", 2024-09-14T06:30:49.118 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.1269999999999999e-06 2024-09-14T06:30:49.118 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:49.119 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:49.119 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:30:49.119 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.708660+0000", 2024-09-14T06:30:49.119 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052536919000000001 2024-09-14T06:30:49.119 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:49.119 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:49.119 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:30:49.119 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.708703+0000", 2024-09-14T06:30:49.119 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.3077000000000001e-05 2024-09-14T06:30:49.119 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:49.119 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:49.119 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:30:49.119 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.708760+0000", 2024-09-14T06:30:49.119 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.6541e-05 2024-09-14T06:30:49.119 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:49.119 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:30:49.120 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:49.120 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:49.120 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:49.120 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:11466 3.1d 3:bcad7303:::rbd_data.112d92027590.0000000000000001:f [read 227328~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:30:49.120 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:25.656129+0000", 2024-09-14T06:30:49.120 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 83.399466716000006, 2024-09-14T06:30:49.120 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052750987999999999, 2024-09-14T06:30:49.120 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:30:49.120 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:30:49.120 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:30:49.120 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:30:49.120 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:30:49.120 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:49.120 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 11466 2024-09-14T06:30:49.120 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:49.121 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:30:49.121 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:49.121 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:30:49.121 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.656129+0000", 2024-09-14T06:30:49.121 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:49.121 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:49.121 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:49.121 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:30:49.121 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.656129+0000", 2024-09-14T06:30:49.121 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:49.121 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:49.121 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:49.121 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:30:49.121 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.656347+0000", 2024-09-14T06:30:49.121 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.000218054 2024-09-14T06:30:49.122 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:49.122 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:49.122 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:30:49.122 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.656349+0000", 2024-09-14T06:30:49.122 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.9199999999999998e-06 2024-09-14T06:30:49.122 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:49.122 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:49.122 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:30:49.122 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.656353+0000", 2024-09-14T06:30:49.122 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.9210000000000002e-06 2024-09-14T06:30:49.122 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:49.122 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:49.122 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:30:49.122 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.656356+0000", 2024-09-14T06:30:49.122 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.7670000000000001e-06 2024-09-14T06:30:49.123 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:49.123 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:49.123 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:30:49.123 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.708787+0000", 2024-09-14T06:30:49.123 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052430840999999999 2024-09-14T06:30:49.123 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:49.123 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:49.123 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:30:49.123 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.708827+0000", 2024-09-14T06:30:49.123 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.9538e-05 2024-09-14T06:30:49.123 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:49.123 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:49.123 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:30:49.124 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.708880+0000", 2024-09-14T06:30:49.124 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.2947e-05 2024-09-14T06:30:49.124 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:49.124 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:30:49.124 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:49.124 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:49.124 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:49.124 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:11467 3.1d 3:bcad7303:::rbd_data.112d92027590.0000000000000001:f [read 237568~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:30:49.124 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:25.656361+0000", 2024-09-14T06:30:49.124 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 83.399234067999998, 2024-09-14T06:30:49.124 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052643898000000001, 2024-09-14T06:30:49.124 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:30:49.124 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:30:49.124 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:30:49.124 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:30:49.125 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:30:49.125 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:49.125 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 11467 2024-09-14T06:30:49.125 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:49.125 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:30:49.125 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:49.125 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:30:49.125 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.656361+0000", 2024-09-14T06:30:49.125 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:49.125 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:49.125 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:49.125 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:30:49.125 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.656361+0000", 2024-09-14T06:30:49.125 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:49.125 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:49.126 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:49.126 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:30:49.126 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.656855+0000", 2024-09-14T06:30:49.126 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00049372000000000001 2024-09-14T06:30:49.126 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:49.126 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:49.126 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:30:49.126 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.656859+0000", 2024-09-14T06:30:49.126 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.1300000000000003e-06 2024-09-14T06:30:49.126 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:49.126 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:49.126 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:30:49.126 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.656863+0000", 2024-09-14T06:30:49.126 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.2629999999999997e-06 2024-09-14T06:30:49.126 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:49.126 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:49.127 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:30:49.127 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.656867+0000", 2024-09-14T06:30:49.127 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.3220000000000001e-06 2024-09-14T06:30:49.127 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:49.127 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:49.127 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:30:49.127 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.708903+0000", 2024-09-14T06:30:49.127 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052035855999999998 2024-09-14T06:30:49.127 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:49.127 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:49.127 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:30:49.127 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.708949+0000", 2024-09-14T06:30:49.127 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.6276999999999998e-05 2024-09-14T06:30:49.127 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:49.127 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:49.128 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:30:49.128 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.709005+0000", 2024-09-14T06:30:49.128 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.6329999999999998e-05 2024-09-14T06:30:49.128 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:49.128 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:30:49.128 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:49.128 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:49.128 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:49.128 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:11479 3.1d 3:bcad7303:::rbd_data.112d92027590.0000000000000001:f [read 360448~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:30:49.128 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:25.658388+0000", 2024-09-14T06:30:49.128 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 83.397207234999996, 2024-09-14T06:30:49.128 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052408920999999997, 2024-09-14T06:30:49.128 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:30:49.128 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:30:49.128 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:30:49.128 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:30:49.129 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:30:49.129 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:49.129 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 11479 2024-09-14T06:30:49.129 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:49.129 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:30:49.129 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:49.129 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:30:49.129 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.658388+0000", 2024-09-14T06:30:49.129 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:49.129 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:49.129 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:49.129 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:30:49.129 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.658388+0000", 2024-09-14T06:30:49.130 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:49.130 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:49.130 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:49.130 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:30:49.130 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.658534+0000", 2024-09-14T06:30:49.130 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.000145613 2024-09-14T06:30:49.130 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:49.130 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:49.130 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:30:49.130 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.658540+0000", 2024-09-14T06:30:49.130 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.6680000000000004e-06 2024-09-14T06:30:49.130 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:49.130 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:49.130 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:30:49.130 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.658548+0000", 2024-09-14T06:30:49.130 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.7980000000000006e-06 2024-09-14T06:30:49.131 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:49.131 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:49.131 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:30:49.131 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.658554+0000", 2024-09-14T06:30:49.131 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.6869999999999999e-06 2024-09-14T06:30:49.131 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:49.131 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:49.131 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:30:49.131 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.710624+0000", 2024-09-14T06:30:49.131 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052070567999999998 2024-09-14T06:30:49.131 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:49.131 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:49.131 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:30:49.131 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.710666+0000", 2024-09-14T06:30:49.131 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.1467000000000001e-05 2024-09-14T06:30:49.132 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:49.132 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:49.132 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:30:49.132 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.710797+0000", 2024-09-14T06:30:49.132 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00013112 2024-09-14T06:30:49.132 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:49.132 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:30:49.132 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:49.132 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:49.132 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:49.132 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:11480 3.1d 3:bcad7303:::rbd_data.112d92027590.0000000000000001:f [read 370688~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:30:49.132 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:25.658563+0000", 2024-09-14T06:30:49.132 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 83.397032304000007, 2024-09-14T06:30:49.132 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052429164, 2024-09-14T06:30:49.132 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:30:49.133 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:30:49.133 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:30:49.133 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:30:49.133 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:30:49.133 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:49.133 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 11480 2024-09-14T06:30:49.133 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:49.133 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:30:49.133 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:49.133 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:30:49.133 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.658563+0000", 2024-09-14T06:30:49.133 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:49.133 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:49.134 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:49.134 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:30:49.134 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.658563+0000", 2024-09-14T06:30:49.134 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:49.134 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:49.134 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:49.134 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:30:49.134 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.658658+0000", 2024-09-14T06:30:49.134 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.5346999999999994e-05 2024-09-14T06:30:49.134 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:49.134 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:49.134 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:30:49.134 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.658662+0000", 2024-09-14T06:30:49.134 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.14e-06 2024-09-14T06:30:49.134 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:49.135 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:49.135 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:30:49.135 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.658669+0000", 2024-09-14T06:30:49.135 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.7379999999999994e-06 2024-09-14T06:30:49.135 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:49.135 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:49.135 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:30:49.135 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.658675+0000", 2024-09-14T06:30:49.135 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.5430000000000001e-06 2024-09-14T06:30:49.135 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:49.135 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:49.135 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:30:49.135 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.710824+0000", 2024-09-14T06:30:49.136 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052148847999999998 2024-09-14T06:30:49.136 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:49.136 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:49.136 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:30:49.136 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.710864+0000", 2024-09-14T06:30:49.136 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.9925999999999997e-05 2024-09-14T06:30:49.136 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:49.136 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:49.136 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:30:49.136 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.710992+0000", 2024-09-14T06:30:49.136 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.000128622 2024-09-14T06:30:49.136 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:49.137 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:30:49.137 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:49.137 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:49.137 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:49.137 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:11481 3.1d 3:bcad7303:::rbd_data.112d92027590.0000000000000001:f [read 380928~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:30:49.137 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:25.658684+0000", 2024-09-14T06:30:49.137 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 83.396911234000001, 2024-09-14T06:30:49.137 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052508565, 2024-09-14T06:30:49.137 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:30:49.137 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:30:49.137 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:30:49.137 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:30:49.138 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:30:49.138 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:49.138 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 11481 2024-09-14T06:30:49.138 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:49.138 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:30:49.138 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:49.138 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:30:49.138 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.658684+0000", 2024-09-14T06:30:49.138 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:49.138 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:49.138 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:49.138 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:30:49.138 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.658684+0000", 2024-09-14T06:30:49.138 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:49.139 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:49.139 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:49.139 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:30:49.139 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.658715+0000", 2024-09-14T06:30:49.139 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.0703000000000001e-05 2024-09-14T06:30:49.139 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:49.139 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:49.139 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:30:49.139 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.658718+0000", 2024-09-14T06:30:49.139 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.0709999999999999e-06 2024-09-14T06:30:49.139 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:49.139 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:49.139 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:30:49.140 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.658725+0000", 2024-09-14T06:30:49.140 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.2339999999999997e-06 2024-09-14T06:30:49.140 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:49.140 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:49.140 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:30:49.140 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.658731+0000", 2024-09-14T06:30:49.140 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.6450000000000002e-06 2024-09-14T06:30:49.140 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:49.140 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:49.140 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:30:49.140 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.711020+0000", 2024-09-14T06:30:49.140 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052288886 2024-09-14T06:30:49.140 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:49.141 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:49.141 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:30:49.141 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.711061+0000", 2024-09-14T06:30:49.141 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.1084000000000002e-05 2024-09-14T06:30:49.141 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:49.141 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:49.141 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:30:49.141 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.711193+0000", 2024-09-14T06:30:49.141 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00013194199999999999 2024-09-14T06:30:49.141 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:49.141 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:30:49.141 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:49.141 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:49.142 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:49.142 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:11482 3.1d 3:bcad7303:::rbd_data.112d92027590.0000000000000001:f [read 391168~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:30:49.142 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:25.658740+0000", 2024-09-14T06:30:49.142 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 83.396855768999998, 2024-09-14T06:30:49.142 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052637015000000002, 2024-09-14T06:30:49.142 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:30:49.142 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:30:49.142 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:30:49.142 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:30:49.142 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:30:49.142 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:49.142 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 11482 2024-09-14T06:30:49.142 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:49.142 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:30:49.143 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:49.143 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:30:49.143 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.658740+0000", 2024-09-14T06:30:49.143 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:49.143 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:49.143 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:49.143 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:30:49.143 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.658740+0000", 2024-09-14T06:30:49.143 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:49.143 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:49.143 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:49.143 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:30:49.143 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.658880+0000", 2024-09-14T06:30:49.143 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00014036100000000001 2024-09-14T06:30:49.143 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:49.143 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:49.144 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:30:49.144 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.658883+0000", 2024-09-14T06:30:49.144 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.3969999999999998e-06 2024-09-14T06:30:49.144 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:49.144 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:49.144 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:30:49.144 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.658890+0000", 2024-09-14T06:30:49.144 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.1500000000000002e-06 2024-09-14T06:30:49.144 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:49.144 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:49.144 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:30:49.144 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.658897+0000", 2024-09-14T06:30:49.144 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.1260000000000003e-06 2024-09-14T06:30:49.145 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:49.145 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:49.145 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:30:49.145 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.711207+0000", 2024-09-14T06:30:49.145 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052310777000000003 2024-09-14T06:30:49.145 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:49.145 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:49.145 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:30:49.145 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.711249+0000", 2024-09-14T06:30:49.145 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.1786999999999998e-05 2024-09-14T06:30:49.145 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:49.145 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:49.145 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:30:49.145 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.711377+0000", 2024-09-14T06:30:49.146 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.000127417 2024-09-14T06:30:49.146 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:49.146 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:30:49.146 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:49.146 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:49.146 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:49.146 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:11483 3.1d 3:bcad7303:::rbd_data.112d92027590.0000000000000001:f [read 401408~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:30:49.146 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:25.658906+0000", 2024-09-14T06:30:49.146 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 83.396688984999997, 2024-09-14T06:30:49.146 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052595109000000001, 2024-09-14T06:30:49.146 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:30:49.146 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:30:49.147 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:30:49.147 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:30:49.147 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:30:49.147 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:49.147 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 11483 2024-09-14T06:30:49.147 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:49.147 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:30:49.147 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:49.147 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:30:49.147 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.658906+0000", 2024-09-14T06:30:49.147 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:49.147 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:49.147 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:49.147 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:30:49.148 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.658906+0000", 2024-09-14T06:30:49.148 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:49.148 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:49.148 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:49.148 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:30:49.148 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.664985+0000", 2024-09-14T06:30:49.148 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.0060787250000000001 2024-09-14T06:30:49.148 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:49.148 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:49.148 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:30:49.148 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.664991+0000", 2024-09-14T06:30:49.148 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.7030000000000003e-06 2024-09-14T06:30:49.148 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:49.148 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:49.148 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:30:49.149 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.664999+0000", 2024-09-14T06:30:49.149 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.4430000000000003e-06 2024-09-14T06:30:49.149 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:49.149 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:49.149 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:30:49.149 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.665006+0000", 2024-09-14T06:30:49.149 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.0840000000000003e-06 2024-09-14T06:30:49.149 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:49.149 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:49.149 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:30:49.149 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.711402+0000", 2024-09-14T06:30:49.149 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046395714999999997 2024-09-14T06:30:49.149 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:49.149 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:49.150 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:30:49.150 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.711443+0000", 2024-09-14T06:30:49.150 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.0911999999999997e-05 2024-09-14T06:30:49.150 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:49.150 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:49.150 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:30:49.150 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.711501+0000", 2024-09-14T06:30:49.150 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.8526999999999997e-05 2024-09-14T06:30:49.150 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:49.150 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:30:49.150 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:49.150 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:49.150 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:30:49.150 INFO:teuthology.orchestra.run.smithi138.stdout:} 2024-09-14T06:30:49.152 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-14T06:30:49.153 INFO:tasks.ceph.osd.6.smithi138.stderr:2024-09-14T06:30:49.150+0000 7f8cd8849640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33620) UID: 0 2024-09-14T06:30:49.154 INFO:teuthology.orchestra.run.smithi094.stdout:3551 writesame 0x67879da thru 0x678d16b data_size 0xb(0x5792 bytes) 2024-09-14T06:30:49.154 INFO:teuthology.orchestra.run.smithi094.stdout:3552 write 0xd19aaff thru 0xd1a3865 (0x8d67 bytes) 2024-09-14T06:30:49.154 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-14T06:30:49.154 INFO:teuthology.orchestra.run.smithi094.stdout:3554 read 0x6ff5609 thru 0x7004fc7 (0xf9bf bytes) 2024-09-14T06:30:49.154 INFO:teuthology.orchestra.run.smithi094.stdout:3556 read 0x56ff6e8 thru 0x570a162 (0xaa7b bytes) 2024-09-14T06:30:49.154 INFO:teuthology.orchestra.run.smithi094.stdout:3557 write 0xa08ee9d thru 0xa0932ed (0x4451 bytes) 2024-09-14T06:30:49.154 INFO:teuthology.orchestra.run.smithi094.stdout:3558 punch from 0x15eda31 to 0x15f22d9, (0x48a8 bytes) 2024-09-14T06:30:49.155 INFO:teuthology.orchestra.run.smithi094.stdout:3559 write 0x6eaf23f thru 0x6eb3eec (0x4cae bytes) 2024-09-14T06:30:49.155 INFO:teuthology.orchestra.run.smithi094.stdout:3560 write 0x63f7cf7 thru 0x63fc748 (0x4a52 bytes) 2024-09-14T06:30:49.155 INFO:teuthology.orchestra.run.smithi094.stdout:3561 read 0x6aa861c thru 0x6ab6fa0 (0xe985 bytes) 2024-09-14T06:30:49.155 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-14T06:30:49.155 INFO:teuthology.orchestra.run.smithi094.stdout:3563 writesame 0x3107d61 thru 0x310bd81 data_size 0x1(0x4021 bytes) 2024-09-14T06:30:49.155 INFO:teuthology.orchestra.run.smithi094.stdout:3565 read 0x325fc3f thru 0x326abf6 (0xafb8 bytes) 2024-09-14T06:30:49.155 INFO:teuthology.orchestra.run.smithi094.stdout:3566 write 0x87c042d thru 0x87c0cb5 (0x889 bytes) 2024-09-14T06:30:49.155 INFO:teuthology.orchestra.run.smithi094.stdout:3567 punch from 0x4179b57 to 0x4181220, (0x76c9 bytes) 2024-09-14T06:30:49.155 INFO:teuthology.orchestra.run.smithi094.stdout:3571 punch from 0x464570d to 0x4654d59, (0xf64c bytes) 2024-09-14T06:30:49.155 INFO:teuthology.orchestra.run.smithi094.stdout:3572 punch from 0x4651698 to 0x4655892, (0x41fa bytes) 2024-09-14T06:30:49.155 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-14T06:30:49.155 INFO:teuthology.orchestra.run.smithi094.stdout:3575 writesame 0x1887991 thru 0x1895700 data_size 0x18(0xdd70 bytes) 2024-09-14T06:30:49.155 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-14T06:30:49.155 INFO:teuthology.orchestra.run.smithi094.stdout:3577 trunc from 0xd1a3866 to 0x5913313 2024-09-14T06:30:49.156 INFO:teuthology.orchestra.run.smithi094.stdout:3579 write 0xb65fabf thru 0xb66ad7e (0xb2c0 bytes) 2024-09-14T06:30:49.156 INFO:teuthology.orchestra.run.smithi094.stdout:3580 punch from 0x6cda79c to 0x6ce22d4, (0x7b38 bytes) 2024-09-14T06:30:49.156 INFO:teuthology.orchestra.run.smithi094.stdout:3581 write 0xa8caebb thru 0xa8d628a (0xb3d0 bytes) 2024-09-14T06:30:49.158 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-14T06:30:49.158 INFO:teuthology.orchestra.run.smithi094.stdout:3583 read 0x91d1c7f thru 0x91d8640 (0x69c2 bytes) 2024-09-14T06:30:49.159 INFO:teuthology.orchestra.run.smithi094.stdout:3584 writesame 0x9261e8b thru 0x92626d3 data_size 0x7(0x849 bytes) 2024-09-14T06:30:49.168 INFO:teuthology.orchestra.run.smithi094.stdout:3585 trunc from 0xb66ad7f to 0x4635733 2024-09-14T06:30:49.178 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-14T06:30:49.178 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-14T06:30:49.178 INFO:teuthology.orchestra.run.smithi094.stdout:3590 read 0x647323 thru 0x64ec76 (0x7954 bytes) 2024-09-14T06:30:49.190 INFO:teuthology.orchestra.run.smithi094.stdout:3592 writesame 0x8d8cfa5 thru 0x8d8d857 data_size 0x1(0x8b3 bytes) 2024-09-14T06:30:49.191 INFO:teuthology.orchestra.run.smithi094.stdout:3593 read 0x7c15bc7 thru 0x7c1d4ce (0x7908 bytes) 2024-09-14T06:30:49.192 INFO:teuthology.orchestra.run.smithi094.stdout:3594 writesame 0x2cd409e thru 0x2cdd0fa data_size 0x3(0x905d bytes) 2024-09-14T06:30:49.208 INFO:teuthology.orchestra.run.smithi094.stdout:3595 write 0xc13fff2 thru 0xc14ab13 (0xab22 bytes) 2024-09-14T06:30:49.210 INFO:teuthology.orchestra.run.smithi094.stdout:3597 read 0x4bcdb8d thru 0x4bdac4b (0xd0bf bytes) 2024-09-14T06:30:49.210 INFO:teuthology.orchestra.run.smithi094.stdout:3600 write 0x8b282d2 thru 0x8b29ea7 (0x1bd6 bytes) 2024-09-14T06:30:49.212 INFO:teuthology.orchestra.run.smithi094.stdout:3601 read 0x4a731e5 thru 0x4a7b623 (0x843f bytes) 2024-09-14T06:30:49.212 INFO:teuthology.orchestra.run.smithi094.stdout:3602 writesame 0x2d0235 thru 0x2d444c data_size 0x1e(0x4218 bytes) 2024-09-14T06:30:49.221 INFO:teuthology.orchestra.run.smithi094.stdout:3603 write 0xcd314e6 thru 0xcd39cfd (0x8818 bytes) 2024-09-14T06:30:49.223 INFO:teuthology.orchestra.run.smithi094.stdout:3606 write 0x1159f13 thru 0x115edfc (0x4eea bytes) 2024-09-14T06:30:49.225 INFO:teuthology.orchestra.run.smithi094.stdout:3607 write 0x8919ba2 thru 0x892406c (0xa4cb bytes) 2024-09-14T06:30:49.226 INFO:teuthology.orchestra.run.smithi094.stdout:3609 write 0x3f034be thru 0x3f06a19 (0x355c bytes) 2024-09-14T06:30:49.228 INFO:teuthology.orchestra.run.smithi094.stdout:3610 write 0x2910d72 thru 0x291d4f0 (0xc77f bytes) 2024-09-14T06:30:49.229 INFO:teuthology.orchestra.run.smithi094.stdout:3611 write 0x1327c7f thru 0x1331d18 (0xa09a bytes) 2024-09-14T06:30:49.231 INFO:teuthology.orchestra.run.smithi094.stdout:3612 punch from 0x42d1a7 to 0x42f5cf, (0x2428 bytes) 2024-09-14T06:30:49.231 INFO:teuthology.orchestra.run.smithi094.stdout:3613 writesame 0x6180c81 thru 0x618d35c data_size 0x59(0xc6dc bytes) 2024-09-14T06:30:49.233 INFO:teuthology.orchestra.run.smithi094.stdout:3614 write 0x4501ccf thru 0x4504429 (0x275b bytes) 2024-09-14T06:30:49.234 INFO:teuthology.orchestra.run.smithi094.stdout:3615 writesame 0x2e1b4f thru 0x2e2ee7 data_size 0x1(0x1399 bytes) 2024-09-14T06:30:49.237 INFO:teuthology.orchestra.run.smithi094.stdout:3616 punch from 0x12c0fb1 to 0x12c15a3, (0x5f2 bytes) 2024-09-14T06:30:49.237 INFO:teuthology.orchestra.run.smithi094.stdout:3617 write 0x760ce0c thru 0x76193ed (0xc5e2 bytes) 2024-09-14T06:30:49.239 INFO:teuthology.orchestra.run.smithi094.stdout:3618 writesame 0x2b0f0cb thru 0x2b1aadb data_size 0x17(0xba11 bytes) 2024-09-14T06:30:49.248 INFO:teuthology.orchestra.run.smithi094.stdout:3619 write 0xd6e4ecc thru 0xd6e5b75 (0xcaa bytes) 2024-09-14T06:30:49.249 INFO:teuthology.orchestra.run.smithi094.stdout:3620 write 0x356b054 thru 0x356d901 (0x28ae bytes) 2024-09-14T06:30:49.252 INFO:teuthology.orchestra.run.smithi094.stdout:3621 write 0x1ec24d8 thru 0x1ec5d1e (0x3847 bytes) 2024-09-14T06:30:49.252 INFO:tasks.ceph.osd.2.smithi094.stderr:2024-09-14T06:30:49.251+0000 7faf3269b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33649) UID: 0 2024-09-14T06:30:49.253 INFO:teuthology.orchestra.run.smithi094.stdout:3622 write 0x51163a9 thru 0x511ff43 (0x9b9b bytes) 2024-09-14T06:30:49.255 INFO:teuthology.orchestra.run.smithi094.stdout:3623 read 0x8a30185 thru 0x8a3811b (0x7f97 bytes) 2024-09-14T06:30:49.255 INFO:teuthology.orchestra.run.smithi094.stdout:3624 read 0x159525b thru 0x159ee70 (0x9c16 bytes) 2024-09-14T06:30:49.256 INFO:teuthology.orchestra.run.smithi094.stdout:3625 write 0x9e3e22c thru 0x9e44562 (0x6337 bytes) 2024-09-14T06:30:49.257 INFO:teuthology.orchestra.run.smithi094.stdout:3630 read 0xa2afe15 thru 0xa2b2aac (0x2c98 bytes) 2024-09-14T06:30:49.257 INFO:teuthology.orchestra.run.smithi094.stdout:3631 writesame 0x909adf1 thru 0x90a0896 data_size 0x29(0x5aa6 bytes) 2024-09-14T06:30:49.259 INFO:teuthology.orchestra.run.smithi094.stdout:3632 writesame 0x1d98909 thru 0x1da29c7 data_size 0x57(0xa0bf bytes) 2024-09-14T06:30:49.261 INFO:teuthology.orchestra.run.smithi094.stdout:3633 punch from 0x7666f64 to 0x766df91, (0x702d bytes) 2024-09-14T06:30:49.261 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-14T06:30:49.262 INFO:teuthology.orchestra.run.smithi094.stdout:3635 write 0x9caf076 thru 0x9cb0001 (0xf8c bytes) 2024-09-14T06:30:49.262 INFO:teuthology.orchestra.run.smithi094.stdout:3636 read 0x3ab61cf thru 0x3abc5eb (0x641d bytes) 2024-09-14T06:30:49.263 INFO:teuthology.orchestra.run.smithi094.stdout:3638 read 0x55c73a7 thru 0x55d6932 (0xf58c bytes) 2024-09-14T06:30:49.263 INFO:teuthology.orchestra.run.smithi094.stdout:3639 writesame 0xcb70066 thru 0xcb7f247 data_size 0x7(0xf1e2 bytes) 2024-09-14T06:30:49.269 INFO:teuthology.orchestra.run.smithi094.stdout:3640 read 0x9f19a8f thru 0x9f215d3 (0x7b45 bytes) 2024-09-14T06:30:49.269 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-14T06:30:49.269 INFO:teuthology.orchestra.run.smithi094.stdout:3643 writesame 0xc7c965 thru 0xc866f9 data_size 0x71(0x9d95 bytes) 2024-09-14T06:30:49.270 INFO:teuthology.orchestra.run.smithi094.stdout:3644 write 0x94b6f0f thru 0x94c076c (0x985e bytes) 2024-09-14T06:30:49.271 INFO:teuthology.orchestra.run.smithi094.stdout:3646 read 0x9c8e1bf thru 0x9c927f8 (0x463a bytes) 2024-09-14T06:30:49.281 INFO:teuthology.orchestra.run.smithi094.stdout:3647 write 0xe4c7813 thru 0xe4c888c (0x107a bytes) 2024-09-14T06:30:49.284 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-14T06:30:49.284 INFO:teuthology.orchestra.run.smithi094.stdout:3649 write 0x3b28f99 thru 0x3b30d4a (0x7db2 bytes) 2024-09-14T06:30:49.284 INFO:teuthology.orchestra.run.smithi094.stdout:3650 write 0x92fc9e2 thru 0x93033c0 (0x69df bytes) 2024-09-14T06:30:49.286 INFO:teuthology.orchestra.run.smithi094.stdout:3652 write 0x4e120b6 thru 0x4e1b746 (0x9691 bytes) 2024-09-14T06:30:49.287 INFO:teuthology.orchestra.run.smithi094.stdout:3653 read 0x56d7fa3 thru 0x56e7cdb (0xfd39 bytes) 2024-09-14T06:30:49.288 INFO:teuthology.orchestra.run.smithi094.stdout:3654 writesame 0x47c225 thru 0x485055 data_size 0x1(0x8e31 bytes) 2024-09-14T06:30:49.302 INFO:teuthology.orchestra.run.smithi138.stdout:{ 2024-09-14T06:30:49.302 INFO:teuthology.orchestra.run.smithi138.stdout: "ops": [], 2024-09-14T06:30:49.302 INFO:teuthology.orchestra.run.smithi138.stdout: "num_ops": 0 2024-09-14T06:30:49.302 INFO:teuthology.orchestra.run.smithi138.stdout:} 2024-09-14T06:30:49.311 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-14T06:30:49.318 INFO:teuthology.orchestra.run.smithi094.stdout:3655 trunc from 0xe4c888d to 0x28b9e41 2024-09-14T06:30:49.349 INFO:teuthology.orchestra.run.smithi094.stdout:3656 write 0xd2096b1 thru 0xd211c0f (0x855f bytes) 2024-09-14T06:30:49.351 INFO:teuthology.orchestra.run.smithi094.stdout:3657 read 0x8f1db0a thru 0x8f28cf7 (0xb1ee bytes) 2024-09-14T06:30:49.353 INFO:tasks.ceph.osd.1.smithi094.stderr:2024-09-14T06:30:49.352+0000 7f328e47a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33641) UID: 0 2024-09-14T06:30:49.366 INFO:teuthology.orchestra.run.smithi094.stdout:3658 trunc from 0xd211c10 to 0x9014f2 2024-09-14T06:30:49.408 INFO:teuthology.orchestra.run.smithi094.stdout:3659 trunc from 0x9014f2 to 0x8dd4e7b 2024-09-14T06:30:49.530 INFO:teuthology.orchestra.run.smithi138.stdout:{ 2024-09-14T06:30:49.530 INFO:teuthology.orchestra.run.smithi138.stdout: "ops": [], 2024-09-14T06:30:49.530 INFO:teuthology.orchestra.run.smithi138.stdout: "complaint_time": 30, 2024-09-14T06:30:49.530 INFO:teuthology.orchestra.run.smithi138.stdout: "num_blocked_ops": 0 2024-09-14T06:30:49.530 INFO:teuthology.orchestra.run.smithi138.stdout:} 2024-09-14T06:30:49.531 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-14T06:30:49.531 INFO:tasks.ceph.osd.5.smithi138.stderr:2024-09-14T06:30:49.528+0000 7fe13746f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33618) UID: 0 2024-09-14T06:30:49.533 INFO:teuthology.orchestra.run.smithi094.stdout:3660 punch from 0x527aba5 to 0x527f61b, (0x4a76 bytes) 2024-09-14T06:30:49.533 INFO:teuthology.orchestra.run.smithi094.stdout:3661 write 0xa86edd1 thru 0xa873fdf (0x520f bytes) 2024-09-14T06:30:49.533 INFO:teuthology.orchestra.run.smithi094.stdout:3664 punch from 0x106fac5 to 0x107af20, (0xb45b bytes) 2024-09-14T06:30:49.533 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-14T06:30:49.534 INFO:teuthology.orchestra.run.smithi094.stdout:3668 punch from 0x513c6c2 to 0x513de5d, (0x179b bytes) 2024-09-14T06:30:49.534 INFO:teuthology.orchestra.run.smithi094.stdout:3669 writesame 0x3550271 thru 0x355c398 data_size 0x1c(0xc128 bytes) 2024-09-14T06:30:49.534 INFO:teuthology.orchestra.run.smithi094.stdout:3671 trunc from 0xa873fe0 to 0xa54171c 2024-09-14T06:30:49.534 INFO:teuthology.orchestra.run.smithi094.stdout:3672 trunc from 0xa54171c to 0x39e6f86 2024-09-14T06:30:49.534 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-14T06:30:49.534 INFO:teuthology.orchestra.run.smithi094.stdout:3674 trunc from 0x39e6f86 to 0x7a51f59 2024-09-14T06:30:49.534 INFO:teuthology.orchestra.run.smithi094.stdout:3675 read 0x7345343 thru 0x7352fb8 (0xdc76 bytes) 2024-09-14T06:30:49.534 INFO:teuthology.orchestra.run.smithi094.stdout:3676 write 0xc2394ed thru 0xc23d49f (0x3fb3 bytes) 2024-09-14T06:30:49.534 INFO:teuthology.orchestra.run.smithi094.stdout:3677 punch from 0x9d53209 to 0x9d6208b, (0xee82 bytes) 2024-09-14T06:30:49.534 INFO:teuthology.orchestra.run.smithi094.stdout:3678 write 0x45f5327 thru 0x45f93ff (0x40d9 bytes) 2024-09-14T06:30:49.534 INFO:teuthology.orchestra.run.smithi094.stdout:3679 write 0xd7911c3 thru 0xd7a09d5 (0xf813 bytes) 2024-09-14T06:30:49.534 INFO:teuthology.orchestra.run.smithi094.stdout:3680 punch from 0x8379bf8 to 0x8386f63, (0xd36b bytes) 2024-09-14T06:30:49.534 INFO:teuthology.orchestra.run.smithi094.stdout:3682 read 0x8d9f0 thru 0x97e6a (0xa47b bytes) 2024-09-14T06:30:49.534 INFO:teuthology.orchestra.run.smithi094.stdout:3683 read 0xc694ccc thru 0xc6a17cb (0xcb00 bytes) 2024-09-14T06:30:49.535 INFO:teuthology.orchestra.run.smithi094.stdout:3688 writesame 0x205bfb3 thru 0x205d4ac data_size 0xf(0x14fa bytes) 2024-09-14T06:30:49.535 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-14T06:30:49.535 INFO:teuthology.orchestra.run.smithi094.stdout:3691 writesame 0xb0e5639 thru 0xb0e6d9b data_size 0x1(0x1763 bytes) 2024-09-14T06:30:49.535 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-14T06:30:49.535 INFO:teuthology.orchestra.run.smithi094.stdout:3693 read 0x70d76dd thru 0x70daf58 (0x387c bytes) 2024-09-14T06:30:49.535 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-14T06:30:49.535 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-14T06:30:49.535 INFO:teuthology.orchestra.run.smithi094.stdout:3696 punch from 0xc51eb7f to 0xc52bd93, (0xd214 bytes) 2024-09-14T06:30:49.535 INFO:teuthology.orchestra.run.smithi094.stdout:3698 writesame 0x752234b thru 0x753094f data_size 0x1(0xe605 bytes) 2024-09-14T06:30:49.535 INFO:teuthology.orchestra.run.smithi094.stdout:3700 write 0x5e6f76 thru 0x5ee2c8 (0x7353 bytes) 2024-09-14T06:30:49.535 INFO:teuthology.orchestra.run.smithi094.stdout:3702 read 0x827e8cb thru 0x828914e (0xa884 bytes) 2024-09-14T06:30:49.535 INFO:teuthology.orchestra.run.smithi094.stdout:3705 read 0xb471549 thru 0xb475923 (0x43db bytes) 2024-09-14T06:30:49.535 INFO:teuthology.orchestra.run.smithi094.stdout:3706 read 0xd77dae8 thru 0xd78cd40 (0xf259 bytes) 2024-09-14T06:30:49.535 INFO:teuthology.orchestra.run.smithi094.stdout:3707 read 0x933104f thru 0x933b253 (0xa205 bytes) 2024-09-14T06:30:49.535 INFO:teuthology.orchestra.run.smithi094.stdout:3710 writesame 0x9f9e65b thru 0x9fac624 data_size 0x55(0xdfca bytes) 2024-09-14T06:30:49.535 INFO:teuthology.orchestra.run.smithi094.stdout:3711 write 0x8b15e94 thru 0x8b16006 (0x173 bytes) 2024-09-14T06:30:49.540 INFO:teuthology.orchestra.run.smithi094.stdout:3712 writesame 0xdad1520 thru 0xdad8c8a data_size 0x1(0x776b bytes) 2024-09-14T06:30:49.541 INFO:teuthology.orchestra.run.smithi094.stdout:3713 read 0xd2af882 thru 0xd2b8474 (0x8bf3 bytes) 2024-09-14T06:30:49.555 INFO:teuthology.orchestra.run.smithi094.stdout:3714 trunc from 0xdad8c8b to 0x24c7358 2024-09-14T06:30:49.577 INFO:teuthology.orchestra.run.smithi094.stdout:3715 write 0x6329de7 thru 0x632e7cc (0x49e6 bytes) 2024-09-14T06:30:49.579 INFO:teuthology.orchestra.run.smithi094.stdout:3716 read 0x338086b thru 0x3388976 (0x810c bytes) 2024-09-14T06:30:49.579 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-14T06:30:49.579 INFO:teuthology.orchestra.run.smithi094.stdout:3719 read 0x54de211 thru 0x54e1ef8 (0x3ce8 bytes) 2024-09-14T06:30:49.579 INFO:teuthology.orchestra.run.smithi094.stdout:3720 write 0x5b4cc38 thru 0x5b507ce (0x3b97 bytes) 2024-09-14T06:30:49.581 INFO:teuthology.orchestra.run.smithi094.stdout:3721 read 0x25bf57 thru 0x26b36e (0xf418 bytes) 2024-09-14T06:30:49.581 INFO:teuthology.orchestra.run.smithi094.stdout:3723 writesame 0x576f179 thru 0x5774bf2 data_size 0x25(0x5a7a bytes) 2024-09-14T06:30:49.596 INFO:teuthology.orchestra.run.smithi094.stdout:3724 writesame 0xb885259 thru 0xb88dcda data_size 0x1(0x8a82 bytes) 2024-09-14T06:30:49.597 INFO:teuthology.orchestra.run.smithi094.stdout:3725 read 0x720f68a thru 0x721dedd (0xe854 bytes) 2024-09-14T06:30:49.598 INFO:teuthology.orchestra.run.smithi094.stdout:3726 read 0x400f292 thru 0x4011dba (0x2b29 bytes) 2024-09-14T06:30:49.598 INFO:teuthology.orchestra.run.smithi094.stdout:3727 read 0x6fabc9c thru 0x6fb809d (0xc402 bytes) 2024-09-14T06:30:49.598 INFO:teuthology.orchestra.run.smithi094.stdout:3728 read 0x828a8d thru 0x830645 (0x7bb9 bytes) 2024-09-14T06:30:49.599 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-14T06:30:49.600 INFO:teuthology.orchestra.run.smithi094.stdout:3730 write 0x773c09a thru 0x7741a40 (0x59a7 bytes) 2024-09-14T06:30:49.609 INFO:teuthology.orchestra.run.smithi094.stdout:3732 trunc from 0xb88dcdb to 0x5069249 2024-09-14T06:30:49.629 INFO:tasks.ceph.osd.2.smithi094.stderr:2024-09-14T06:30:49.628+0000 7faf3269b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33649) UID: 0 2024-09-14T06:30:49.633 INFO:teuthology.orchestra.run.smithi094.stdout:3734 write 0xb1153f1 thru 0xb120b2c (0xb73c bytes) 2024-09-14T06:30:49.634 INFO:teuthology.orchestra.run.smithi094.stdout:3736 write 0x157a07b thru 0x1580603 (0x6589 bytes) 2024-09-14T06:30:49.637 INFO:teuthology.orchestra.run.smithi094.stdout:3737 read 0x9fbe59 thru 0xa07396 (0xb53e bytes) 2024-09-14T06:30:49.637 INFO:teuthology.orchestra.run.smithi094.stdout:3738 writesame 0x20fc78b thru 0x20ffb95 data_size 0x1(0x340b bytes) 2024-09-14T06:30:49.644 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-14T06:30:49.653 INFO:teuthology.orchestra.run.smithi094.stdout:3742 writesame 0xca3e3f8 thru 0xca46950 data_size 0x3(0x8559 bytes) 2024-09-14T06:30:49.654 INFO:teuthology.orchestra.run.smithi094.stdout:3743 write 0x4bf52cf thru 0x4c01117 (0xbe49 bytes) 2024-09-14T06:30:49.656 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-14T06:30:49.656 INFO:teuthology.orchestra.run.smithi094.stdout:3748 write 0x3a91b8f thru 0x3a92624 (0xa96 bytes) 2024-09-14T06:30:49.657 INFO:teuthology.orchestra.run.smithi094.stdout:3749 writesame 0x9aad196 thru 0x9abbf8b data_size 0x4e(0xedf6 bytes) 2024-09-14T06:30:49.658 INFO:teuthology.orchestra.run.smithi094.stdout:3751 read 0x16a7960 thru 0x16ab33c (0x39dd bytes) 2024-09-14T06:30:49.659 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-14T06:30:49.659 INFO:teuthology.orchestra.run.smithi094.stdout:3754 write 0x55edc27 thru 0x55f463b (0x6a15 bytes) 2024-09-14T06:30:49.660 INFO:teuthology.orchestra.run.smithi094.stdout:3755 writesame 0x35b4b4e thru 0x35bd79a data_size 0x7(0x8c4d bytes) 2024-09-14T06:30:49.663 INFO:teuthology.orchestra.run.smithi094.stdout:3757 writesame 0x736f4b7 thru 0x7372448 data_size 0x1(0x2f92 bytes) 2024-09-14T06:30:49.669 INFO:teuthology.orchestra.run.smithi094.stdout:3760 read 0x3c49fb7 thru 0x3c4bbd6 (0x1c20 bytes) 2024-09-14T06:30:49.670 INFO:teuthology.orchestra.run.smithi094.stdout:3762 write 0x370ae29 thru 0x37140a2 (0x927a bytes) 2024-09-14T06:30:49.672 INFO:teuthology.orchestra.run.smithi094.stdout:3763 read 0x585dd03 thru 0x5860470 (0x276e bytes) 2024-09-14T06:30:49.680 INFO:teuthology.orchestra.run.smithi138.stdout:{ 2024-09-14T06:30:49.680 INFO:teuthology.orchestra.run.smithi138.stdout: "size": 20, 2024-09-14T06:30:49.680 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 600, 2024-09-14T06:30:49.680 INFO:teuthology.orchestra.run.smithi138.stdout: "ops": [ 2024-09-14T06:30:49.680 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:49.680 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:98505 3.10 3.1bf56f90 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T06:30:49.680 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:30:07.045453+0000", 2024-09-14T06:30:49.681 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 42.633162935999998, 2024-09-14T06:30:49.681 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032144440000000003, 2024-09-14T06:30:49.681 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:30:49.681 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:30:49.681 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "reached pg", 2024-09-14T06:30:49.681 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:30:49.681 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:30:49.681 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:49.681 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 98505 2024-09-14T06:30:49.681 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:49.681 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:30:49.681 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:49.681 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:30:49.682 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045453+0000", 2024-09-14T06:30:49.682 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:49.682 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:49.682 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:49.682 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:30:49.682 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045453+0000", 2024-09-14T06:30:49.682 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:49.682 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:49.682 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:49.682 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:30:49.682 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045453+0000", 2024-09-14T06:30:49.682 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.36e-07 2024-09-14T06:30:49.682 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:49.683 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:49.683 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:30:49.683 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045454+0000", 2024-09-14T06:30:49.683 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.1299999999999998e-07 2024-09-14T06:30:49.683 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:49.683 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:49.683 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:30:49.683 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045457+0000", 2024-09-14T06:30:49.683 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.8559999999999998e-06 2024-09-14T06:30:49.683 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:49.683 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:49.683 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:30:49.683 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045460+0000", 2024-09-14T06:30:49.683 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.8750000000000002e-06 2024-09-14T06:30:49.683 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:49.684 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:49.684 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:30:49.684 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.077592+0000", 2024-09-14T06:30:49.684 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032131727999999998 2024-09-14T06:30:49.684 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:49.684 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:49.684 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:30:49.684 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.077597+0000", 2024-09-14T06:30:49.684 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.8319999999999996e-06 2024-09-14T06:30:49.684 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:49.684 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:30:49.684 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:49.684 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:49.684 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:49.684 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:98506 3.10 3.1bf56f90 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T06:30:49.685 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:30:07.045481+0000", 2024-09-14T06:30:49.685 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 42.633134681000001, 2024-09-14T06:30:49.685 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032142759999999999, 2024-09-14T06:30:49.685 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:30:49.685 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:30:49.685 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "reached pg", 2024-09-14T06:30:49.685 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:30:49.685 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:30:49.685 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:49.685 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 98506 2024-09-14T06:30:49.685 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:49.685 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:30:49.686 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:49.686 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:30:49.686 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045481+0000", 2024-09-14T06:30:49.686 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:49.686 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:49.686 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:49.686 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:30:49.686 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045481+0000", 2024-09-14T06:30:49.686 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:49.686 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:49.686 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:49.686 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:30:49.686 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045481+0000", 2024-09-14T06:30:49.687 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.8500000000000001e-07 2024-09-14T06:30:49.687 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:49.687 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:49.687 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:30:49.687 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045483+0000", 2024-09-14T06:30:49.687 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.8250000000000001e-06 2024-09-14T06:30:49.687 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:49.687 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:49.687 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:30:49.687 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045489+0000", 2024-09-14T06:30:49.687 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.5740000000000003e-06 2024-09-14T06:30:49.687 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:49.687 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:49.687 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:30:49.687 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045495+0000", 2024-09-14T06:30:49.688 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.7690000000000002e-06 2024-09-14T06:30:49.688 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:49.688 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:49.688 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:30:49.688 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.077619+0000", 2024-09-14T06:30:49.688 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032124621999999999 2024-09-14T06:30:49.688 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:49.688 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:49.688 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:30:49.688 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.077624+0000", 2024-09-14T06:30:49.688 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.3850000000000004e-06 2024-09-14T06:30:49.688 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:49.688 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:30:49.689 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:49.689 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:49.689 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:49.689 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:98507 3.10 3.1bf56f90 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T06:30:49.689 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:30:07.045502+0000", 2024-09-14T06:30:49.689 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 42.633113260000002, 2024-09-14T06:30:49.689 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032147219999999997, 2024-09-14T06:30:49.689 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:30:49.689 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:30:49.689 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "reached pg", 2024-09-14T06:30:49.689 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:30:49.689 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:30:49.689 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:49.690 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 98507 2024-09-14T06:30:49.690 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:49.690 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:30:49.690 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:49.690 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:30:49.690 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045502+0000", 2024-09-14T06:30:49.692 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:49.692 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:49.692 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:49.692 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:30:49.692 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045502+0000", 2024-09-14T06:30:49.692 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:49.692 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:49.692 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:49.692 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:30:49.692 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045503+0000", 2024-09-14T06:30:49.692 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.4700000000000002e-07 2024-09-14T06:30:49.692 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:49.693 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:49.693 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:30:49.693 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045509+0000", 2024-09-14T06:30:49.693 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.7329999999999997e-06 2024-09-14T06:30:49.693 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:49.693 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:49.693 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:30:49.693 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045515+0000", 2024-09-14T06:30:49.693 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.8030000000000002e-06 2024-09-14T06:30:49.693 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:49.693 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:49.693 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:30:49.693 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045522+0000", 2024-09-14T06:30:49.693 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.0369999999999997e-06 2024-09-14T06:30:49.693 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:49.694 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:49.694 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:30:49.694 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.077641+0000", 2024-09-14T06:30:49.694 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032119091000000002 2024-09-14T06:30:49.694 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:49.694 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:49.694 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:30:49.694 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.077650+0000", 2024-09-14T06:30:49.694 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.2090000000000005e-06 2024-09-14T06:30:49.694 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:49.694 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:30:49.694 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:49.694 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:49.694 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:49.694 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:98508 3.10 3.1bf56f90 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T06:30:49.695 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:30:07.045530+0000", 2024-09-14T06:30:49.695 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 42.633085643000001, 2024-09-14T06:30:49.695 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032198773999999999, 2024-09-14T06:30:49.695 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:30:49.695 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:30:49.695 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "reached pg", 2024-09-14T06:30:49.695 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:30:49.695 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:30:49.695 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:49.695 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 98508 2024-09-14T06:30:49.695 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:49.695 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:30:49.695 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:49.696 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:30:49.696 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045530+0000", 2024-09-14T06:30:49.696 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:49.696 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:49.696 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:49.696 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:30:49.696 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045530+0000", 2024-09-14T06:30:49.696 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:49.696 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:49.696 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:49.696 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:30:49.696 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045530+0000", 2024-09-14T06:30:49.696 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.3099999999999998e-07 2024-09-14T06:30:49.697 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:49.697 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:49.697 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:30:49.697 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045539+0000", 2024-09-14T06:30:49.697 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.4780000000000008e-06 2024-09-14T06:30:49.697 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:49.697 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:49.697 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:30:49.697 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045546+0000", 2024-09-14T06:30:49.697 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.3259999999999998e-06 2024-09-14T06:30:49.697 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:49.697 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:49.697 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:30:49.697 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045551+0000", 2024-09-14T06:30:49.698 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.9960000000000001e-06 2024-09-14T06:30:49.698 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:49.698 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:49.698 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:30:49.698 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.077716+0000", 2024-09-14T06:30:49.698 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032165081999999998 2024-09-14T06:30:49.698 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:49.698 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:49.698 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:30:49.698 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.077729+0000", 2024-09-14T06:30:49.698 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.2461e-05 2024-09-14T06:30:49.698 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:49.698 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:30:49.698 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:49.698 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:49.699 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:49.699 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:98509 3.10 3.1bf56f90 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T06:30:49.699 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:30:07.045558+0000", 2024-09-14T06:30:49.699 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 42.633057442000002, 2024-09-14T06:30:49.699 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032242198, 2024-09-14T06:30:49.699 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:30:49.699 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:30:49.699 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "reached pg", 2024-09-14T06:30:49.699 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:30:49.699 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:30:49.699 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:49.699 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 98509 2024-09-14T06:30:49.699 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:49.699 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:30:49.700 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:49.700 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:30:49.700 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045558+0000", 2024-09-14T06:30:49.700 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:49.700 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:49.700 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:49.700 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:30:49.700 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045558+0000", 2024-09-14T06:30:49.700 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:49.700 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:49.700 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:49.700 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:30:49.700 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045559+0000", 2024-09-14T06:30:49.700 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.27e-07 2024-09-14T06:30:49.701 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:49.701 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:49.701 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:30:49.701 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045560+0000", 2024-09-14T06:30:49.701 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.6959999999999999e-06 2024-09-14T06:30:49.701 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:49.701 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:49.701 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:30:49.701 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045567+0000", 2024-09-14T06:30:49.701 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.6900000000000003e-06 2024-09-14T06:30:49.701 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:49.701 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:49.701 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:30:49.701 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045571+0000", 2024-09-14T06:30:49.702 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.5750000000000002e-06 2024-09-14T06:30:49.702 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:49.702 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:49.702 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:30:49.702 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.077789+0000", 2024-09-14T06:30:49.702 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032217186000000002 2024-09-14T06:30:49.702 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:49.702 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:49.702 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:30:49.702 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.077800+0000", 2024-09-14T06:30:49.702 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.1623999999999999e-05 2024-09-14T06:30:49.702 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:49.702 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:30:49.702 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:49.702 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:49.702 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:49.703 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:98510 3.10 3.1bf56f90 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T06:30:49.703 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:30:07.045580+0000", 2024-09-14T06:30:49.703 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 42.633035628000002, 2024-09-14T06:30:49.703 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032299845000000001, 2024-09-14T06:30:49.703 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:30:49.703 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:30:49.703 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "reached pg", 2024-09-14T06:30:49.703 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:30:49.703 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:30:49.703 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:49.703 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 98510 2024-09-14T06:30:49.703 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:49.703 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:30:49.703 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:49.703 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:30:49.704 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045580+0000", 2024-09-14T06:30:49.704 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:49.704 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:49.704 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:49.704 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:30:49.704 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045580+0000", 2024-09-14T06:30:49.704 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:49.704 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:49.704 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:49.704 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:30:49.704 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045580+0000", 2024-09-14T06:30:49.704 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.0400000000000002e-07 2024-09-14T06:30:49.704 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:49.704 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:49.704 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:30:49.705 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045582+0000", 2024-09-14T06:30:49.705 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.7600000000000001e-06 2024-09-14T06:30:49.705 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:49.705 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:49.705 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:30:49.705 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045588+0000", 2024-09-14T06:30:49.705 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.5650000000000001e-06 2024-09-14T06:30:49.705 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:49.705 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:49.705 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:30:49.705 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045597+0000", 2024-09-14T06:30:49.705 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.465e-06 2024-09-14T06:30:49.705 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:49.706 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:49.706 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:30:49.706 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.077871+0000", 2024-09-14T06:30:49.706 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032273562999999998 2024-09-14T06:30:49.706 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:49.706 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:49.706 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:30:49.706 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.077880+0000", 2024-09-14T06:30:49.706 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.0880000000000001e-06 2024-09-14T06:30:49.706 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:49.706 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:30:49.706 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:49.706 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:49.706 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:49.706 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:98511 3.10 3.1bf56f90 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T06:30:49.706 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:30:07.045610+0000", 2024-09-14T06:30:49.707 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 42.633005781000001, 2024-09-14T06:30:49.707 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032338894, 2024-09-14T06:30:49.707 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:30:49.707 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:30:49.707 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "reached pg", 2024-09-14T06:30:49.707 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:30:49.707 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:30:49.707 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:49.707 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 98511 2024-09-14T06:30:49.707 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:49.707 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:30:49.707 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:49.707 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:30:49.707 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045610+0000", 2024-09-14T06:30:49.707 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:49.708 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:49.708 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:49.708 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:30:49.708 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045610+0000", 2024-09-14T06:30:49.708 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:49.708 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:49.708 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:49.708 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:30:49.708 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045610+0000", 2024-09-14T06:30:49.708 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.1600000000000002e-07 2024-09-14T06:30:49.708 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:49.709 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:49.709 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:30:49.709 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045612+0000", 2024-09-14T06:30:49.709 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.1150000000000001e-06 2024-09-14T06:30:49.709 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:49.709 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:49.709 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:30:49.709 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045618+0000", 2024-09-14T06:30:49.709 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.0499999999999997e-06 2024-09-14T06:30:49.709 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:49.709 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:49.709 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:30:49.709 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045623+0000", 2024-09-14T06:30:49.709 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.6770000000000003e-06 2024-09-14T06:30:49.709 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:49.710 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:49.710 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:30:49.710 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.077928+0000", 2024-09-14T06:30:49.710 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032305111999999997 2024-09-14T06:30:49.710 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:49.710 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:49.710 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:30:49.710 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.077949+0000", 2024-09-14T06:30:49.710 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.0523999999999999e-05 2024-09-14T06:30:49.710 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:49.710 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:30:49.710 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:49.710 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:49.711 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:49.711 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:98512 3.10 3.1bf56f90 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T06:30:49.711 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:30:07.045630+0000", 2024-09-14T06:30:49.711 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 42.632985775999998, 2024-09-14T06:30:49.711 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032346632, 2024-09-14T06:30:49.711 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:30:49.711 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:30:49.711 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "reached pg", 2024-09-14T06:30:49.711 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:30:49.711 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:30:49.711 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:49.712 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 98512 2024-09-14T06:30:49.712 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:49.712 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:30:49.712 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:49.712 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:30:49.712 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045630+0000", 2024-09-14T06:30:49.712 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:49.712 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:49.712 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:49.712 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:30:49.712 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045630+0000", 2024-09-14T06:30:49.712 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:49.712 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:49.713 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:49.713 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:30:49.713 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045630+0000", 2024-09-14T06:30:49.713 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.8200000000000001e-07 2024-09-14T06:30:49.713 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:49.713 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:49.713 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:30:49.713 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045632+0000", 2024-09-14T06:30:49.713 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.7430000000000001e-06 2024-09-14T06:30:49.714 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:49.714 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:49.714 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:30:49.714 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045637+0000", 2024-09-14T06:30:49.714 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.4550000000000003e-06 2024-09-14T06:30:49.714 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:49.714 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:49.714 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:30:49.714 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045644+0000", 2024-09-14T06:30:49.714 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.1330000000000001e-06 2024-09-14T06:30:49.714 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:49.714 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:49.714 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:30:49.714 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.077968+0000", 2024-09-14T06:30:49.714 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032323401000000002 2024-09-14T06:30:49.714 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:49.715 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:49.715 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:30:49.715 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.077976+0000", 2024-09-14T06:30:49.715 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.5180000000000004e-06 2024-09-14T06:30:49.715 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:49.715 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:30:49.715 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:49.715 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:49.715 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:49.715 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:98513 3.10 3.1bf56f90 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T06:30:49.715 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:30:07.045652+0000", 2024-09-14T06:30:49.715 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 42.632963212, 2024-09-14T06:30:49.716 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032353436999999999, 2024-09-14T06:30:49.716 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:30:49.716 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:30:49.716 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "reached pg", 2024-09-14T06:30:49.716 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:30:49.716 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:30:49.716 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:49.716 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 98513 2024-09-14T06:30:49.716 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:49.716 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:30:49.716 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:49.716 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:30:49.716 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045652+0000", 2024-09-14T06:30:49.716 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:49.716 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:49.717 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:49.717 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:30:49.717 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045652+0000", 2024-09-14T06:30:49.717 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:49.717 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:49.717 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:49.717 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:30:49.717 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045653+0000", 2024-09-14T06:30:49.717 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.3700000000000001e-07 2024-09-14T06:30:49.717 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:49.717 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:49.717 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:30:49.717 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045655+0000", 2024-09-14T06:30:49.717 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.854e-06 2024-09-14T06:30:49.717 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:49.717 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:49.718 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:30:49.718 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045661+0000", 2024-09-14T06:30:49.718 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.5509999999999996e-06 2024-09-14T06:30:49.718 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:49.718 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:49.718 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:30:49.718 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045668+0000", 2024-09-14T06:30:49.718 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.1980000000000004e-06 2024-09-14T06:30:49.718 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:49.718 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:49.718 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:30:49.718 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.077997+0000", 2024-09-14T06:30:49.718 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032328575999999998 2024-09-14T06:30:49.718 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:49.718 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:49.718 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:30:49.719 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.078006+0000", 2024-09-14T06:30:49.719 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.8210000000000008e-06 2024-09-14T06:30:49.719 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:49.719 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:30:49.719 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:49.719 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:49.719 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:49.719 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:98514 3.10 3.1bf56f90 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T06:30:49.719 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:30:07.045699+0000", 2024-09-14T06:30:49.719 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 42.632916877, 2024-09-14T06:30:49.719 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032330223999999998, 2024-09-14T06:30:49.719 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:30:49.719 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:30:49.719 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "reached pg", 2024-09-14T06:30:49.719 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:30:49.720 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:30:49.720 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:49.720 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 98514 2024-09-14T06:30:49.720 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:49.720 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:30:49.720 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:49.720 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:30:49.720 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045699+0000", 2024-09-14T06:30:49.720 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:49.720 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:49.720 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:49.720 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:30:49.720 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045699+0000", 2024-09-14T06:30:49.720 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:49.721 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:49.721 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:49.721 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:30:49.721 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045699+0000", 2024-09-14T06:30:49.721 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.3200000000000001e-07 2024-09-14T06:30:49.721 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:49.721 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:49.721 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:30:49.721 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045701+0000", 2024-09-14T06:30:49.721 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.762e-06 2024-09-14T06:30:49.721 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:49.721 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:49.721 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:30:49.721 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045705+0000", 2024-09-14T06:30:49.722 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.2359999999999998e-06 2024-09-14T06:30:49.722 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:49.722 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:49.722 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:30:49.722 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045708+0000", 2024-09-14T06:30:49.722 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.796e-06 2024-09-14T06:30:49.722 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:49.722 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:49.722 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:30:49.722 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.078023+0000", 2024-09-14T06:30:49.722 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032315213000000002 2024-09-14T06:30:49.722 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:49.722 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:49.723 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:30:49.723 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.078029+0000", 2024-09-14T06:30:49.723 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.8850000000000004e-06 2024-09-14T06:30:49.723 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:49.723 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:30:49.723 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:49.723 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:49.723 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:49.723 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:98515 3.10 3.1bf56f90 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T06:30:49.723 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:30:07.045712+0000", 2024-09-14T06:30:49.723 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 42.632903566000003, 2024-09-14T06:30:49.723 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032346871999999999, 2024-09-14T06:30:49.723 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:30:49.724 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:30:49.724 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "reached pg", 2024-09-14T06:30:49.724 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:30:49.724 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:30:49.724 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:49.724 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 98515 2024-09-14T06:30:49.724 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:49.724 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:30:49.724 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:49.724 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:30:49.724 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045712+0000", 2024-09-14T06:30:49.724 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:49.724 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:49.724 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:49.725 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:30:49.725 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045712+0000", 2024-09-14T06:30:49.725 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:49.725 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:49.725 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:49.725 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:30:49.725 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045712+0000", 2024-09-14T06:30:49.725 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.2100000000000001e-07 2024-09-14T06:30:49.725 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:49.725 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:49.725 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:30:49.725 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045713+0000", 2024-09-14T06:30:49.725 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.068e-06 2024-09-14T06:30:49.725 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:49.725 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:49.726 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:30:49.726 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045716+0000", 2024-09-14T06:30:49.726 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.971e-06 2024-09-14T06:30:49.726 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:49.726 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:49.726 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:30:49.726 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045719+0000", 2024-09-14T06:30:49.726 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.3879999999999998e-06 2024-09-14T06:30:49.726 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:49.726 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:49.726 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:30:49.726 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.078049+0000", 2024-09-14T06:30:49.726 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032330553999999997 2024-09-14T06:30:49.726 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:49.726 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:49.727 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:30:49.727 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.078059+0000", 2024-09-14T06:30:49.727 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.6700000000000006e-06 2024-09-14T06:30:49.727 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:49.727 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:30:49.727 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:49.727 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:49.727 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:49.727 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:98516 3.10 3.1bf56f90 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T06:30:49.727 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:30:07.045722+0000", 2024-09-14T06:30:49.727 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 42.632893813000003, 2024-09-14T06:30:49.727 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032374334999999997, 2024-09-14T06:30:49.727 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:30:49.727 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:30:49.727 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "reached pg", 2024-09-14T06:30:49.728 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:30:49.728 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:30:49.728 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:49.728 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 98516 2024-09-14T06:30:49.728 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:49.728 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:30:49.728 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:49.728 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:30:49.728 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045722+0000", 2024-09-14T06:30:49.728 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:49.728 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:49.728 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:49.728 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:30:49.729 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045722+0000", 2024-09-14T06:30:49.729 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:49.729 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:49.729 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:49.729 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:30:49.729 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045722+0000", 2024-09-14T06:30:49.729 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.8999999999999998e-07 2024-09-14T06:30:49.729 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:49.729 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:49.729 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:30:49.729 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045723+0000", 2024-09-14T06:30:49.729 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.0049999999999999e-06 2024-09-14T06:30:49.729 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:49.730 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:49.730 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:30:49.730 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045726+0000", 2024-09-14T06:30:49.730 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.9019999999999999e-06 2024-09-14T06:30:49.730 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:49.730 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:49.730 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:30:49.730 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045728+0000", 2024-09-14T06:30:49.730 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.3290000000000001e-06 2024-09-14T06:30:49.730 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:49.730 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:49.730 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:30:49.730 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.078090+0000", 2024-09-14T06:30:49.730 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032362123 2024-09-14T06:30:49.731 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:49.731 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:49.731 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:30:49.731 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.078096+0000", 2024-09-14T06:30:49.731 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.6860000000000003e-06 2024-09-14T06:30:49.731 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:49.731 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:30:49.731 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:49.731 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:49.731 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:49.731 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:98517 3.10 3.1bf56f90 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T06:30:49.731 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:30:07.045731+0000", 2024-09-14T06:30:49.731 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 42.632884081999997, 2024-09-14T06:30:49.731 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.03241227, 2024-09-14T06:30:49.731 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:30:49.732 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:30:49.732 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "reached pg", 2024-09-14T06:30:49.732 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:30:49.732 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:30:49.732 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:49.732 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 98517 2024-09-14T06:30:49.732 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:49.732 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:30:49.732 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:49.732 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:30:49.732 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045731+0000", 2024-09-14T06:30:49.732 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:49.732 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:49.733 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:49.733 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:30:49.733 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045731+0000", 2024-09-14T06:30:49.733 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:49.733 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:49.733 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:49.733 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:30:49.733 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045732+0000", 2024-09-14T06:30:49.733 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.0699999999999999e-07 2024-09-14T06:30:49.733 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:49.733 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:49.733 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:30:49.733 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045733+0000", 2024-09-14T06:30:49.733 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.9199999999999999e-07 2024-09-14T06:30:49.733 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:49.734 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:49.734 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:30:49.734 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045735+0000", 2024-09-14T06:30:49.734 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.8150000000000002e-06 2024-09-14T06:30:49.734 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:49.734 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:49.734 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:30:49.734 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045738+0000", 2024-09-14T06:30:49.734 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.3640000000000002e-06 2024-09-14T06:30:49.734 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:49.734 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:49.734 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:30:49.734 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.078126+0000", 2024-09-14T06:30:49.734 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032388206000000003 2024-09-14T06:30:49.734 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:49.735 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:49.735 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:30:49.735 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.078144+0000", 2024-09-14T06:30:49.735 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.7686e-05 2024-09-14T06:30:49.735 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:49.735 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:30:49.735 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:49.735 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:49.735 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:49.735 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:98518 3.10 3.1bf56f90 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T06:30:49.735 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:30:07.045741+0000", 2024-09-14T06:30:49.735 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 42.632874686999997, 2024-09-14T06:30:49.735 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032451793999999999, 2024-09-14T06:30:49.735 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:30:49.735 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:30:49.736 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "reached pg", 2024-09-14T06:30:49.736 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:30:49.736 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:30:49.736 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:49.736 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 98518 2024-09-14T06:30:49.736 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:49.736 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:30:49.736 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:49.736 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:30:49.736 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045741+0000", 2024-09-14T06:30:49.736 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:49.736 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:49.736 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:49.736 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:30:49.736 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045741+0000", 2024-09-14T06:30:49.737 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:49.737 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:49.737 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:49.737 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:30:49.737 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045741+0000", 2024-09-14T06:30:49.737 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.05e-07 2024-09-14T06:30:49.737 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:49.737 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:49.737 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:30:49.737 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045746+0000", 2024-09-14T06:30:49.737 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.2259999999999996e-06 2024-09-14T06:30:49.737 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:49.737 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:49.737 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:30:49.737 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045750+0000", 2024-09-14T06:30:49.737 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.2490000000000001e-06 2024-09-14T06:30:49.738 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:49.738 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:49.738 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:30:49.738 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045752+0000", 2024-09-14T06:30:49.738 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.3980000000000002e-06 2024-09-14T06:30:49.738 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:49.738 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:49.738 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:30:49.738 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.078184+0000", 2024-09-14T06:30:49.738 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032431593000000002 2024-09-14T06:30:49.738 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:49.738 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:49.738 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:30:49.739 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.078193+0000", 2024-09-14T06:30:49.739 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.1230000000000006e-06 2024-09-14T06:30:49.739 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:49.739 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:30:49.739 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:49.739 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:49.739 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:49.739 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:98519 3.10 3.1bf56f90 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T06:30:49.739 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:30:07.045755+0000", 2024-09-14T06:30:49.739 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 42.632860262000001, 2024-09-14T06:30:49.739 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032489431999999999, 2024-09-14T06:30:49.739 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:30:49.739 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:30:49.740 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "reached pg", 2024-09-14T06:30:49.740 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:30:49.740 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:30:49.740 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:49.740 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 98519 2024-09-14T06:30:49.740 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:49.740 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:30:49.740 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:49.740 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:30:49.740 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045755+0000", 2024-09-14T06:30:49.740 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:49.740 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:49.740 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:49.741 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:30:49.741 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045755+0000", 2024-09-14T06:30:49.741 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:49.741 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:49.741 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:49.741 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:30:49.741 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045756+0000", 2024-09-14T06:30:49.741 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.23e-07 2024-09-14T06:30:49.741 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:49.741 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:49.741 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:30:49.741 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045757+0000", 2024-09-14T06:30:49.741 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.1060000000000001e-06 2024-09-14T06:30:49.742 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:49.742 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:49.742 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:30:49.742 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045760+0000", 2024-09-14T06:30:49.742 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.988e-06 2024-09-14T06:30:49.742 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:49.742 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:49.742 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:30:49.742 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045765+0000", 2024-09-14T06:30:49.742 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.2689999999999999e-06 2024-09-14T06:30:49.742 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:49.742 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:49.742 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:30:49.742 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.078234+0000", 2024-09-14T06:30:49.743 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032468915000000001 2024-09-14T06:30:49.743 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:49.743 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:49.743 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:30:49.743 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.078245+0000", 2024-09-14T06:30:49.743 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.0930999999999999e-05 2024-09-14T06:30:49.743 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:49.743 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:30:49.743 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:49.743 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:49.743 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:49.743 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:98520 3.10 3.1bf56f90 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T06:30:49.743 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:30:07.045768+0000", 2024-09-14T06:30:49.743 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 42.632847384000002, 2024-09-14T06:30:49.743 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032531379999999999, 2024-09-14T06:30:49.744 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:30:49.744 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:30:49.744 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "reached pg", 2024-09-14T06:30:49.744 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:30:49.744 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:30:49.744 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:49.744 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 98520 2024-09-14T06:30:49.744 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:49.744 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:30:49.744 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:49.744 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:30:49.744 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045768+0000", 2024-09-14T06:30:49.744 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:49.744 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:49.745 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:49.745 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:30:49.745 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045768+0000", 2024-09-14T06:30:49.745 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:49.745 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:49.745 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:49.745 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:30:49.745 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045768+0000", 2024-09-14T06:30:49.745 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.22e-07 2024-09-14T06:30:49.745 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:49.745 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:49.745 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:30:49.745 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045773+0000", 2024-09-14T06:30:49.745 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.3470000000000001e-06 2024-09-14T06:30:49.745 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:49.746 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:49.746 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:30:49.746 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045776+0000", 2024-09-14T06:30:49.746 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.1319999999999998e-06 2024-09-14T06:30:49.746 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:49.746 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:49.746 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:30:49.746 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045779+0000", 2024-09-14T06:30:49.746 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.9749999999999999e-06 2024-09-14T06:30:49.746 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:49.746 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:49.746 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:30:49.746 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.078289+0000", 2024-09-14T06:30:49.746 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032509693999999999 2024-09-14T06:30:49.746 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:49.747 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:49.747 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:30:49.747 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.078300+0000", 2024-09-14T06:30:49.747 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.101e-05 2024-09-14T06:30:49.747 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:49.747 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:30:49.747 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:49.747 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:49.747 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:49.747 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:98521 3.10 3.1bf56f90 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T06:30:49.747 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:30:07.045782+0000", 2024-09-14T06:30:49.747 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 42.632833204000001, 2024-09-14T06:30:49.747 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032566109000000003, 2024-09-14T06:30:49.747 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:30:49.747 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:30:49.747 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "reached pg", 2024-09-14T06:30:49.748 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:30:49.748 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:30:49.748 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:49.748 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 98521 2024-09-14T06:30:49.748 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:49.748 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:30:49.748 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:49.748 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:30:49.748 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045782+0000", 2024-09-14T06:30:49.748 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:49.748 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:49.748 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:49.748 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:30:49.748 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045782+0000", 2024-09-14T06:30:49.749 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:49.749 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:49.749 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:49.749 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:30:49.749 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045783+0000", 2024-09-14T06:30:49.749 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.2499999999999999e-07 2024-09-14T06:30:49.749 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:49.749 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:49.749 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:30:49.749 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045784+0000", 2024-09-14T06:30:49.749 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.0109999999999999e-06 2024-09-14T06:30:49.749 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:49.749 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:49.750 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:30:49.750 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045786+0000", 2024-09-14T06:30:49.750 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.8289999999999999e-06 2024-09-14T06:30:49.750 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:49.750 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:49.750 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:30:49.750 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045793+0000", 2024-09-14T06:30:49.750 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.579e-06 2024-09-14T06:30:49.750 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:49.750 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:49.750 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:30:49.750 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.078338+0000", 2024-09-14T06:30:49.750 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032545389000000001 2024-09-14T06:30:49.750 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:49.751 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:49.751 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:30:49.751 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.078348+0000", 2024-09-14T06:30:49.751 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.0076e-05 2024-09-14T06:30:49.751 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:49.751 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:30:49.751 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:49.751 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:49.751 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:49.751 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:98522 3.10 3.1bf56f90 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T06:30:49.751 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:30:07.045796+0000", 2024-09-14T06:30:49.751 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 42.632819296000001, 2024-09-14T06:30:49.751 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032577846000000001, 2024-09-14T06:30:49.751 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:30:49.752 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:30:49.752 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "reached pg", 2024-09-14T06:30:49.752 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:30:49.752 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:30:49.752 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:49.752 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 98522 2024-09-14T06:30:49.752 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:49.752 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:30:49.752 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:49.752 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:30:49.752 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045796+0000", 2024-09-14T06:30:49.752 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:49.752 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:49.753 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:49.753 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:30:49.753 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045796+0000", 2024-09-14T06:30:49.753 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:49.753 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:49.753 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:49.753 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:30:49.753 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045796+0000", 2024-09-14T06:30:49.753 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.2100000000000001e-07 2024-09-14T06:30:49.753 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:49.753 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:49.753 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:30:49.754 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045797+0000", 2024-09-14T06:30:49.754 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.6899999999999996e-07 2024-09-14T06:30:49.754 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:49.754 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:49.754 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:30:49.754 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045800+0000", 2024-09-14T06:30:49.754 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.9979999999999999e-06 2024-09-14T06:30:49.754 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:49.754 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:49.754 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:30:49.754 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045803+0000", 2024-09-14T06:30:49.754 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.2850000000000001e-06 2024-09-14T06:30:49.754 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:49.754 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:49.755 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:30:49.755 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.078366+0000", 2024-09-14T06:30:49.755 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032562932000000003 2024-09-14T06:30:49.755 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:49.755 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:49.755 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:30:49.755 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.078374+0000", 2024-09-14T06:30:49.755 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.4409999999999993e-06 2024-09-14T06:30:49.755 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:49.755 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:30:49.755 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:49.755 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:49.755 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:49.755 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:98523 3.10 3.1bf56f90 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T06:30:49.756 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:30:07.045806+0000", 2024-09-14T06:30:49.756 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 42.632809752999997, 2024-09-14T06:30:49.756 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032611167000000003, 2024-09-14T06:30:49.756 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:30:49.756 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:30:49.756 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "reached pg", 2024-09-14T06:30:49.756 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:30:49.756 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:30:49.756 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:49.756 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 98523 2024-09-14T06:30:49.756 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:49.756 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:30:49.756 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:49.756 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:30:49.757 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045806+0000", 2024-09-14T06:30:49.757 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:49.757 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:49.757 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:49.757 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:30:49.757 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045806+0000", 2024-09-14T06:30:49.757 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:49.757 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:49.757 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:49.757 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:30:49.757 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045806+0000", 2024-09-14T06:30:49.757 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.2000000000000001e-07 2024-09-14T06:30:49.757 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:49.758 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:49.758 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:30:49.758 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045807+0000", 2024-09-14T06:30:49.758 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.4e-07 2024-09-14T06:30:49.758 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:49.758 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:49.758 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:30:49.758 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045810+0000", 2024-09-14T06:30:49.758 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.9229999999999998e-06 2024-09-14T06:30:49.758 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:49.758 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:49.758 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:30:49.758 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045812+0000", 2024-09-14T06:30:49.758 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.2220000000000001e-06 2024-09-14T06:30:49.759 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:49.759 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:49.759 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:30:49.759 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.078407+0000", 2024-09-14T06:30:49.759 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032594513999999998 2024-09-14T06:30:49.759 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:49.759 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:49.759 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:30:49.759 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.078417+0000", 2024-09-14T06:30:49.759 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.0348e-05 2024-09-14T06:30:49.759 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:49.759 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:30:49.759 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:49.759 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:49.759 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:49.760 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:98524 3.10 3.1bf56f90 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T06:30:49.760 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:30:07.045815+0000", 2024-09-14T06:30:49.760 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 42.632800385000003, 2024-09-14T06:30:49.760 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032643208999999999, 2024-09-14T06:30:49.760 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:30:49.760 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:30:49.760 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "reached pg", 2024-09-14T06:30:49.760 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:30:49.760 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:30:49.760 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:49.760 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 98524 2024-09-14T06:30:49.760 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:49.761 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:30:49.761 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:49.761 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:30:49.761 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045815+0000", 2024-09-14T06:30:49.761 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:49.761 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:49.761 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:49.761 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:30:49.761 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045815+0000", 2024-09-14T06:30:49.761 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:49.761 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:49.761 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:49.761 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:30:49.762 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045815+0000", 2024-09-14T06:30:49.762 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.11e-07 2024-09-14T06:30:49.762 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:49.762 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:49.762 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:30:49.762 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045816+0000", 2024-09-14T06:30:49.762 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.9099999999999991e-07 2024-09-14T06:30:49.762 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:49.762 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:49.762 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:30:49.762 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045819+0000", 2024-09-14T06:30:49.762 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.9399999999999998e-06 2024-09-14T06:30:49.762 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:49.762 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:49.762 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:30:49.762 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045822+0000", 2024-09-14T06:30:49.763 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.3470000000000001e-06 2024-09-14T06:30:49.763 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:49.763 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:49.763 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:30:49.763 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.078449+0000", 2024-09-14T06:30:49.763 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032627529000000002 2024-09-14T06:30:49.763 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:49.763 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:49.763 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:30:49.763 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.078458+0000", 2024-09-14T06:30:49.763 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.1910000000000006e-06 2024-09-14T06:30:49.763 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:49.763 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:30:49.763 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:49.764 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:49.764 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:30:49.764 INFO:teuthology.orchestra.run.smithi138.stdout:} 2024-09-14T06:30:49.766 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-14T06:30:49.768 INFO:teuthology.orchestra.run.smithi094.stdout:3764 write 0xce7c220 thru 0xce7ea26 (0x2807 bytes) 2024-09-14T06:30:49.768 INFO:teuthology.orchestra.run.smithi094.stdout:3766 read 0x6648097 thru 0x6654600 (0xc56a bytes) 2024-09-14T06:30:49.768 INFO:teuthology.orchestra.run.smithi094.stdout:3767 write 0xb6f856c thru 0xb6f991c (0x13b1 bytes) 2024-09-14T06:30:49.768 INFO:teuthology.orchestra.run.smithi094.stdout:3769 write 0x53e413 thru 0x5452aa (0x6e98 bytes) 2024-09-14T06:30:49.768 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-14T06:30:49.768 INFO:teuthology.orchestra.run.smithi094.stdout:3773 read 0x3131fe6 thru 0x3132891 (0x8ac bytes) 2024-09-14T06:30:49.768 INFO:teuthology.orchestra.run.smithi094.stdout:3774 punch from 0x3cd2f07 to 0x3cdfa6e, (0xcb67 bytes) 2024-09-14T06:30:49.768 INFO:teuthology.orchestra.run.smithi094.stdout:3775 read 0x9c43f67 thru 0x9c50e2e (0xcec8 bytes) 2024-09-14T06:30:49.768 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-14T06:30:49.768 INFO:teuthology.orchestra.run.smithi094.stdout:3778 punch from 0x896abe4 to 0x896dce7, (0x3103 bytes) 2024-09-14T06:30:49.768 INFO:teuthology.orchestra.run.smithi094.stdout:3779 trunc from 0xce7ea27 to 0xbbea827 2024-09-14T06:30:49.769 INFO:teuthology.orchestra.run.smithi094.stdout:3780 write 0x5a3d319 thru 0x5a4ae0e (0xdaf6 bytes) 2024-09-14T06:30:49.769 INFO:teuthology.orchestra.run.smithi094.stdout:3782 write 0x53bc907 thru 0x53cac30 (0xe32a bytes) 2024-09-14T06:30:49.769 INFO:teuthology.orchestra.run.smithi094.stdout:3783 write 0xe321521 thru 0xe330ac1 (0xf5a1 bytes) 2024-09-14T06:30:49.769 INFO:teuthology.orchestra.run.smithi094.stdout:3784 write 0x9954b13 thru 0x995ff41 (0xb42f bytes) 2024-09-14T06:30:49.769 INFO:teuthology.orchestra.run.smithi094.stdout:3785 punch from 0x8a7a8cb to 0x8a8482e, (0x9f63 bytes) 2024-09-14T06:30:49.769 INFO:teuthology.orchestra.run.smithi094.stdout:3786 writesame 0x8377b31 thru 0x83811dc data_size 0x2(0x96ac bytes) 2024-09-14T06:30:49.769 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-14T06:30:49.769 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-14T06:30:49.769 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-14T06:30:49.769 INFO:teuthology.orchestra.run.smithi094.stdout:3791 write 0x8286872 thru 0x8288e3c (0x25cb bytes) 2024-09-14T06:30:49.769 INFO:teuthology.orchestra.run.smithi094.stdout:3792 read 0xc451f04 thru 0xc4570ed (0x51ea bytes) 2024-09-14T06:30:49.769 INFO:teuthology.orchestra.run.smithi094.stdout:3794 write 0x3effda2 thru 0x3f04475 (0x46d4 bytes) 2024-09-14T06:30:49.769 INFO:teuthology.orchestra.run.smithi094.stdout:3795 writesame 0xd54cc5b thru 0xd55a533 data_size 0x1(0xd8d9 bytes) 2024-09-14T06:30:49.769 INFO:teuthology.orchestra.run.smithi094.stdout:3796 trunc from 0xe330ac2 to 0x62c48c7 2024-09-14T06:30:49.770 INFO:tasks.ceph.osd.0.smithi094.stderr:2024-09-14T06:30:49.764+0000 7f07c9333640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33636) UID: 0 2024-09-14T06:30:49.773 INFO:teuthology.orchestra.run.smithi094.stdout:3797 punch from 0x3c8aeac to 0x3c9443f, (0x9593 bytes) 2024-09-14T06:30:49.773 INFO:teuthology.orchestra.run.smithi094.stdout:3798 clone 29 order 18 su 32768 sc 15 2024-09-14T06:30:49.866 INFO:tasks.ceph.osd.2.smithi094.stderr:2024-09-14T06:30:49.864+0000 7faf3269b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33649) UID: 0 2024-09-14T06:30:49.872 INFO:teuthology.orchestra.run.smithi138.stdout:{ 2024-09-14T06:30:49.872 INFO:teuthology.orchestra.run.smithi138.stdout: "ops": [], 2024-09-14T06:30:49.872 INFO:teuthology.orchestra.run.smithi138.stdout: "num_ops": 0 2024-09-14T06:30:49.872 INFO:teuthology.orchestra.run.smithi138.stdout:} 2024-09-14T06:30:49.880 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-14T06:30:49.967 INFO:tasks.ceph.osd.2.smithi094.stderr:2024-09-14T06:30:49.966+0000 7faf3269b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33649) UID: 0 2024-09-14T06:30:49.982 INFO:teuthology.orchestra.run.smithi138.stdout:{ 2024-09-14T06:30:49.982 INFO:teuthology.orchestra.run.smithi138.stdout: "ops": [], 2024-09-14T06:30:49.982 INFO:teuthology.orchestra.run.smithi138.stdout: "complaint_time": 30, 2024-09-14T06:30:49.982 INFO:teuthology.orchestra.run.smithi138.stdout: "num_blocked_ops": 0 2024-09-14T06:30:49.982 INFO:teuthology.orchestra.run.smithi138.stdout:} 2024-09-14T06:30:49.991 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-14T06:30:50.068 INFO:tasks.ceph.osd.5.smithi138.stderr:2024-09-14T06:30:50.066+0000 7fe13746f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33618) UID: 0 2024-09-14T06:30:50.095 INFO:teuthology.orchestra.run.smithi138.stdout:{ 2024-09-14T06:30:50.095 INFO:teuthology.orchestra.run.smithi138.stdout: "size": 20, 2024-09-14T06:30:50.095 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 600, 2024-09-14T06:30:50.095 INFO:teuthology.orchestra.run.smithi138.stdout: "ops": [ 2024-09-14T06:30:50.095 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:50.095 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:53244 3.f 3:f71f51a7:::rbd_data.112dca2f6ae1.0000000000000015:17 [read 1863680~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:30:50.095 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:44.369399+0000", 2024-09-14T06:30:50.095 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 65.723960634999997, 2024-09-14T06:30:50.095 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047291189999999997, 2024-09-14T06:30:50.096 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:30:50.096 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:30:50.096 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:30:50.096 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:30:50.096 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:30:50.096 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:50.096 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 53244 2024-09-14T06:30:50.096 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:50.096 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:30:50.096 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:50.096 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:30:50.096 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369399+0000", 2024-09-14T06:30:50.096 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:50.097 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:50.097 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:50.097 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:30:50.097 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369399+0000", 2024-09-14T06:30:50.097 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:50.097 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:50.097 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:50.097 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:30:50.097 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369400+0000", 2024-09-14T06:30:50.097 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.0699999999999999e-07 2024-09-14T06:30:50.097 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:50.097 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:50.097 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:30:50.098 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369401+0000", 2024-09-14T06:30:50.098 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.4300000000000001e-06 2024-09-14T06:30:50.098 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:50.098 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:50.098 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:30:50.098 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369404+0000", 2024-09-14T06:30:50.098 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.061e-06 2024-09-14T06:30:50.098 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:50.098 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:50.099 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:30:50.100 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369407+0000", 2024-09-14T06:30:50.100 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.6139999999999999e-06 2024-09-14T06:30:50.100 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:50.100 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:50.100 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:30:50.100 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.416517+0000", 2024-09-14T06:30:50.100 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047110338000000002 2024-09-14T06:30:50.100 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:50.100 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:50.100 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:30:50.100 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.416616+0000", 2024-09-14T06:30:50.100 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.8651000000000001e-05 2024-09-14T06:30:50.100 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:50.100 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:50.100 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:30:50.101 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.416691+0000", 2024-09-14T06:30:50.101 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.4888999999999994e-05 2024-09-14T06:30:50.101 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:50.101 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:30:50.101 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:50.101 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:50.101 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:50.101 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:53245 3.f 3:f71f51a7:::rbd_data.112dca2f6ae1.0000000000000015:17 [read 1888256~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:30:50.101 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:44.369410+0000", 2024-09-14T06:30:50.101 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 65.723950346999999, 2024-09-14T06:30:50.101 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047399275999999997, 2024-09-14T06:30:50.101 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:30:50.101 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:30:50.101 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:30:50.101 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:30:50.102 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:30:50.102 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:50.102 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 53245 2024-09-14T06:30:50.102 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:50.102 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:30:50.102 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:50.102 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:30:50.102 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369410+0000", 2024-09-14T06:30:50.102 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:50.102 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:50.102 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:50.102 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:30:50.102 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369410+0000", 2024-09-14T06:30:50.102 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:50.102 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:50.103 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:50.103 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:30:50.103 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369410+0000", 2024-09-14T06:30:50.103 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.1899999999999999e-07 2024-09-14T06:30:50.103 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:50.103 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:50.103 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:30:50.103 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369411+0000", 2024-09-14T06:30:50.103 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.0300000000000001e-06 2024-09-14T06:30:50.103 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:50.103 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:50.103 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:30:50.103 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369414+0000", 2024-09-14T06:30:50.103 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.1099999999999999e-06 2024-09-14T06:30:50.103 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:50.103 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:50.104 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:30:50.104 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369417+0000", 2024-09-14T06:30:50.104 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.6189999999999998e-06 2024-09-14T06:30:50.104 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:50.104 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:50.104 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:30:50.104 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.416718+0000", 2024-09-14T06:30:50.104 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047301322999999999 2024-09-14T06:30:50.104 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:50.104 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:50.104 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:30:50.104 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.416759+0000", 2024-09-14T06:30:50.104 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.1009e-05 2024-09-14T06:30:50.104 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:50.105 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:50.105 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:30:50.105 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.416809+0000", 2024-09-14T06:30:50.105 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.9966000000000002e-05 2024-09-14T06:30:50.105 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:50.105 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:30:50.105 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:50.105 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:50.105 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:50.105 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:53246 3.f 3:f71f51a7:::rbd_data.112dca2f6ae1.0000000000000015:17 [read 1912832~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:30:50.105 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:44.369420+0000", 2024-09-14T06:30:50.105 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 65.723940403, 2024-09-14T06:30:50.105 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047529463000000001, 2024-09-14T06:30:50.105 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:30:50.105 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:30:50.106 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:30:50.106 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:30:50.106 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:30:50.106 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:50.106 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 53246 2024-09-14T06:30:50.106 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:50.106 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:30:50.106 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:50.106 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:30:50.106 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369420+0000", 2024-09-14T06:30:50.106 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:50.106 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:50.106 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:50.106 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:30:50.107 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369420+0000", 2024-09-14T06:30:50.107 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:50.107 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:50.107 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:50.107 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:30:50.107 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369420+0000", 2024-09-14T06:30:50.107 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.1299999999999999e-07 2024-09-14T06:30:50.107 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:50.107 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:50.107 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:30:50.107 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369425+0000", 2024-09-14T06:30:50.107 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.0130000000000001e-06 2024-09-14T06:30:50.107 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:50.107 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:50.107 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:30:50.108 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369428+0000", 2024-09-14T06:30:50.108 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.349e-06 2024-09-14T06:30:50.108 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:50.108 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:50.108 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:30:50.108 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369431+0000", 2024-09-14T06:30:50.108 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.7389999999999999e-06 2024-09-14T06:30:50.108 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:50.108 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:50.108 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:30:50.108 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.416834+0000", 2024-09-14T06:30:50.108 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047402669000000001 2024-09-14T06:30:50.108 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:50.108 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:50.109 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:30:50.109 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.416882+0000", 2024-09-14T06:30:50.109 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.8068999999999997e-05 2024-09-14T06:30:50.109 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:50.109 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:50.109 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:30:50.109 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.416949+0000", 2024-09-14T06:30:50.109 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.7410999999999995e-05 2024-09-14T06:30:50.109 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:50.109 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:30:50.109 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:50.109 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:50.109 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:50.109 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:53247 3.f 3:f71f51a7:::rbd_data.112dca2f6ae1.0000000000000015:17 [read 1937408~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:30:50.109 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:44.369434+0000", 2024-09-14T06:30:50.110 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 65.723925639000001, 2024-09-14T06:30:50.110 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047652264999999999, 2024-09-14T06:30:50.110 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:30:50.110 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:30:50.110 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:30:50.110 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:30:50.110 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:30:50.110 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:50.110 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 53247 2024-09-14T06:30:50.110 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:50.111 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:30:50.111 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:50.111 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:30:50.111 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369434+0000", 2024-09-14T06:30:50.111 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:50.111 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:50.111 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:50.111 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:30:50.111 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369434+0000", 2024-09-14T06:30:50.111 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:50.111 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:50.111 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:50.111 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:30:50.111 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369435+0000", 2024-09-14T06:30:50.111 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.2100000000000001e-07 2024-09-14T06:30:50.112 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:50.112 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:50.112 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:30:50.112 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369436+0000", 2024-09-14T06:30:50.112 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.122e-06 2024-09-14T06:30:50.112 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:50.112 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:50.112 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:30:50.112 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369439+0000", 2024-09-14T06:30:50.112 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.0400000000000001e-06 2024-09-14T06:30:50.112 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:50.112 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:50.112 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:30:50.112 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369441+0000", 2024-09-14T06:30:50.113 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.728e-06 2024-09-14T06:30:50.113 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:50.113 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:50.113 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:30:50.113 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.416977+0000", 2024-09-14T06:30:50.113 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047535344 2024-09-14T06:30:50.113 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:50.113 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:50.113 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:30:50.113 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.417029+0000", 2024-09-14T06:30:50.113 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.2037999999999999e-05 2024-09-14T06:30:50.113 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:50.113 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:50.114 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:30:50.114 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.417087+0000", 2024-09-14T06:30:50.114 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.7772000000000002e-05 2024-09-14T06:30:50.114 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:50.114 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:30:50.114 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:50.114 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:50.114 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:50.114 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:53248 3.f 3:f71f51a7:::rbd_data.112dca2f6ae1.0000000000000015:17 [read 1961984~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:30:50.114 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:44.369445+0000", 2024-09-14T06:30:50.114 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 65.723915086999995, 2024-09-14T06:30:50.114 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047798489, 2024-09-14T06:30:50.114 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:30:50.114 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:30:50.115 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:30:50.115 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:30:50.115 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:30:50.115 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:50.115 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 53248 2024-09-14T06:30:50.115 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:50.115 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:30:50.115 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:50.115 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:30:50.115 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369445+0000", 2024-09-14T06:30:50.115 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:50.115 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:50.115 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:50.116 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:30:50.116 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369445+0000", 2024-09-14T06:30:50.116 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:50.116 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:50.116 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:50.116 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:30:50.116 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369445+0000", 2024-09-14T06:30:50.116 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.0699999999999999e-07 2024-09-14T06:30:50.116 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:50.116 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:50.116 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:30:50.116 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369446+0000", 2024-09-14T06:30:50.116 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.1000000000000001e-06 2024-09-14T06:30:50.116 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:50.117 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:50.117 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:30:50.117 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369449+0000", 2024-09-14T06:30:50.117 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.1039999999999998e-06 2024-09-14T06:30:50.117 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:50.117 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:50.117 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:30:50.117 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369452+0000", 2024-09-14T06:30:50.117 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.6199999999999999e-06 2024-09-14T06:30:50.117 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:50.117 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:50.117 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:30:50.117 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.417114+0000", 2024-09-14T06:30:50.117 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047661901999999999 2024-09-14T06:30:50.117 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:50.118 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:50.118 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:30:50.118 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.417181+0000", 2024-09-14T06:30:50.118 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.7385999999999995e-05 2024-09-14T06:30:50.118 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:50.118 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:50.118 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:30:50.118 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.417243+0000", 2024-09-14T06:30:50.118 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.2169999999999996e-05 2024-09-14T06:30:50.118 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:50.118 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:30:50.118 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:50.118 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:50.118 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:50.118 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:53249 3.f 3:f71f51a7:::rbd_data.112dca2f6ae1.0000000000000015:17 [read 1986560~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:30:50.119 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:44.369455+0000", 2024-09-14T06:30:50.119 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 65.723904775999998, 2024-09-14T06:30:50.119 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047890493999999999, 2024-09-14T06:30:50.119 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:30:50.119 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:30:50.119 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:30:50.119 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:30:50.119 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:30:50.119 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:50.119 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 53249 2024-09-14T06:30:50.119 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:50.119 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:30:50.119 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:50.119 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:30:50.119 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369455+0000", 2024-09-14T06:30:50.119 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:50.119 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:50.120 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:50.120 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:30:50.120 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369455+0000", 2024-09-14T06:30:50.120 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:50.120 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:50.120 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:50.120 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:30:50.120 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369455+0000", 2024-09-14T06:30:50.120 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.0800000000000001e-07 2024-09-14T06:30:50.120 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:50.120 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:50.120 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:30:50.120 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369456+0000", 2024-09-14T06:30:50.120 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.0729999999999999e-06 2024-09-14T06:30:50.121 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:50.121 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:50.121 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:30:50.121 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369460+0000", 2024-09-14T06:30:50.121 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.027e-06 2024-09-14T06:30:50.121 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:50.121 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:50.121 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:30:50.121 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369462+0000", 2024-09-14T06:30:50.121 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.5320000000000001e-06 2024-09-14T06:30:50.121 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:50.121 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:50.121 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:30:50.121 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.417257+0000", 2024-09-14T06:30:50.121 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.04779489 2024-09-14T06:30:50.121 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:50.122 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:50.122 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:30:50.122 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.417296+0000", 2024-09-14T06:30:50.122 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.8658000000000002e-05 2024-09-14T06:30:50.122 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:50.122 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:50.122 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:30:50.122 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.417346+0000", 2024-09-14T06:30:50.122 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.0105999999999998e-05 2024-09-14T06:30:50.122 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:50.122 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:30:50.122 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:50.122 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:50.122 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:50.122 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:53250 3.f 3:f71f51a7:::rbd_data.112dca2f6ae1.0000000000000015:17 [read 2011136~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:30:50.122 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:44.369465+0000", 2024-09-14T06:30:50.123 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 65.723894806000004, 2024-09-14T06:30:50.123 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.048032230000000002, 2024-09-14T06:30:50.123 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:30:50.123 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:30:50.123 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:30:50.123 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:30:50.123 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:30:50.123 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:50.123 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 53250 2024-09-14T06:30:50.123 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:50.123 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:30:50.123 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:50.123 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:30:50.123 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369465+0000", 2024-09-14T06:30:50.123 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:50.123 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:50.124 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:50.124 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:30:50.124 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369465+0000", 2024-09-14T06:30:50.124 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:50.124 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:50.124 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:50.124 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:30:50.124 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369465+0000", 2024-09-14T06:30:50.124 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.2100000000000001e-07 2024-09-14T06:30:50.124 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:50.124 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:50.124 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:30:50.124 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369466+0000", 2024-09-14T06:30:50.124 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.0640000000000001e-06 2024-09-14T06:30:50.124 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:50.125 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:50.125 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:30:50.125 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369469+0000", 2024-09-14T06:30:50.125 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.0170000000000001e-06 2024-09-14T06:30:50.125 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:50.125 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:50.125 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:30:50.125 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369472+0000", 2024-09-14T06:30:50.125 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.6189999999999998e-06 2024-09-14T06:30:50.125 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:50.125 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:50.125 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:30:50.125 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.417383+0000", 2024-09-14T06:30:50.125 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047911092000000002 2024-09-14T06:30:50.126 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:50.126 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:50.126 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:30:50.126 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.417430+0000", 2024-09-14T06:30:50.126 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.6898999999999999e-05 2024-09-14T06:30:50.126 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:50.126 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:50.126 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:30:50.126 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.417497+0000", 2024-09-14T06:30:50.126 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.7317999999999995e-05 2024-09-14T06:30:50.126 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:50.126 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:30:50.126 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:50.126 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:50.126 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:50.127 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:53251 3.f 3:f71f51a7:::rbd_data.112dca2f6ae1.0000000000000015:17 [read 2035712~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:30:50.127 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:44.369475+0000", 2024-09-14T06:30:50.127 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 65.723884810000001, 2024-09-14T06:30:50.127 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.048154542000000002, 2024-09-14T06:30:50.127 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:30:50.127 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:30:50.127 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:30:50.127 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:30:50.127 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:30:50.127 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:50.127 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 53251 2024-09-14T06:30:50.127 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:50.127 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:30:50.127 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:50.127 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:30:50.127 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369475+0000", 2024-09-14T06:30:50.128 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:50.128 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:50.128 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:50.128 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:30:50.128 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369475+0000", 2024-09-14T06:30:50.128 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:50.128 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:50.128 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:50.128 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:30:50.128 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369475+0000", 2024-09-14T06:30:50.128 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.0100000000000001e-07 2024-09-14T06:30:50.128 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:50.128 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:50.128 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:30:50.128 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369476+0000", 2024-09-14T06:30:50.129 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.0580000000000001e-06 2024-09-14T06:30:50.129 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:50.129 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:50.129 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:30:50.129 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369479+0000", 2024-09-14T06:30:50.129 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.9950000000000001e-06 2024-09-14T06:30:50.129 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:50.129 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:50.129 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:30:50.129 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369482+0000", 2024-09-14T06:30:50.129 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.6340000000000001e-06 2024-09-14T06:30:50.129 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:50.129 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:50.129 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:30:50.130 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.417530+0000", 2024-09-14T06:30:50.130 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.048047535000000002 2024-09-14T06:30:50.130 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:50.130 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:50.130 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:30:50.130 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.417574+0000", 2024-09-14T06:30:50.130 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.4255999999999998e-05 2024-09-14T06:30:50.130 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:50.130 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:50.130 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:30:50.130 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.417630+0000", 2024-09-14T06:30:50.130 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.5862999999999997e-05 2024-09-14T06:30:50.130 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:50.130 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:30:50.130 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:50.130 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:50.131 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:50.131 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:53252 3.f 3:f71f51a7:::rbd_data.112dca2f6ae1.0000000000000015:17 [read 2060288~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:30:50.131 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:44.369485+0000", 2024-09-14T06:30:50.131 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 65.723874961999996, 2024-09-14T06:30:50.131 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.048254116, 2024-09-14T06:30:50.131 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:30:50.131 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:30:50.131 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:30:50.131 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:30:50.131 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:30:50.131 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:50.131 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 53252 2024-09-14T06:30:50.131 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:50.131 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:30:50.131 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:50.132 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:30:50.132 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369485+0000", 2024-09-14T06:30:50.132 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:50.132 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:50.132 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:50.132 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:30:50.132 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369485+0000", 2024-09-14T06:30:50.132 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:50.132 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:50.132 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:50.132 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:30:50.132 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369485+0000", 2024-09-14T06:30:50.132 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.1e-07 2024-09-14T06:30:50.132 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:50.132 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:50.133 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:30:50.133 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369487+0000", 2024-09-14T06:30:50.133 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.42e-06 2024-09-14T06:30:50.133 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:50.133 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:50.133 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:30:50.133 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369490+0000", 2024-09-14T06:30:50.133 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.9639999999999999e-06 2024-09-14T06:30:50.133 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:50.133 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:50.133 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:30:50.133 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369492+0000", 2024-09-14T06:30:50.133 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.5679999999999998e-06 2024-09-14T06:30:50.133 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:50.133 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:50.134 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:30:50.134 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.417645+0000", 2024-09-14T06:30:50.134 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.048152753 2024-09-14T06:30:50.134 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:50.134 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:50.134 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:30:50.134 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.417687+0000", 2024-09-14T06:30:50.134 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.1900999999999997e-05 2024-09-14T06:30:50.134 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:50.134 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:50.134 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:30:50.134 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.417739+0000", 2024-09-14T06:30:50.134 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.2299999999999997e-05 2024-09-14T06:30:50.134 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:50.134 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:30:50.135 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:50.135 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:50.135 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:50.135 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:53253 3.f 3:f71f51a7:::rbd_data.112dca2f6ae1.0000000000000015:17 [read 2084864~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:30:50.135 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:44.369495+0000", 2024-09-14T06:30:50.135 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 65.723864837999997, 2024-09-14T06:30:50.135 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.048366785000000002, 2024-09-14T06:30:50.135 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:30:50.135 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:30:50.135 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:30:50.135 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:30:50.135 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:30:50.135 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:50.135 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 53253 2024-09-14T06:30:50.135 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:50.136 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:30:50.136 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:50.136 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:30:50.136 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369495+0000", 2024-09-14T06:30:50.136 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:50.136 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:50.136 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:50.136 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:30:50.136 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369495+0000", 2024-09-14T06:30:50.136 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:50.136 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:50.136 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:50.136 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:30:50.136 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370388+0000", 2024-09-14T06:30:50.137 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00089287999999999998 2024-09-14T06:30:50.137 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:50.137 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:50.137 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:30:50.137 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370392+0000", 2024-09-14T06:30:50.137 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.2130000000000002e-06 2024-09-14T06:30:50.137 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:50.137 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:50.137 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:30:50.137 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370401+0000", 2024-09-14T06:30:50.137 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.7649999999999999e-06 2024-09-14T06:30:50.137 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:50.137 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:50.137 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:30:50.137 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370406+0000", 2024-09-14T06:30:50.138 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.0159999999999999e-06 2024-09-14T06:30:50.138 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:50.138 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:50.138 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:30:50.138 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.417764+0000", 2024-09-14T06:30:50.138 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047358348000000001 2024-09-14T06:30:50.138 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:50.138 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:50.138 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:30:50.138 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.417806+0000", 2024-09-14T06:30:50.138 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.1149999999999997e-05 2024-09-14T06:30:50.138 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:50.138 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:50.138 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:30:50.138 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.417862+0000", 2024-09-14T06:30:50.138 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.6412999999999997e-05 2024-09-14T06:30:50.139 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:50.139 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:30:50.139 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:50.139 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:50.139 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:50.139 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:53340 3.f 3:f71f51a7:::rbd_data.112dca2f6ae1.0000000000000015:17 [read 1056768~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:30:50.139 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:44.370421+0000", 2024-09-14T06:30:50.139 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 65.722938958, 2024-09-14T06:30:50.139 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047566313999999998, 2024-09-14T06:30:50.139 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:30:50.139 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:30:50.139 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:30:50.139 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:30:50.139 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:30:50.139 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:50.139 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 53340 2024-09-14T06:30:50.140 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:50.140 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:30:50.140 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:50.140 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:30:50.140 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370421+0000", 2024-09-14T06:30:50.140 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:50.140 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:50.140 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:50.140 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:30:50.140 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370421+0000", 2024-09-14T06:30:50.140 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:50.140 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:50.140 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:50.141 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:30:50.141 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370422+0000", 2024-09-14T06:30:50.141 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.3799999999999996e-07 2024-09-14T06:30:50.141 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:50.141 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:50.141 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:30:50.141 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370424+0000", 2024-09-14T06:30:50.141 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.6919999999999998e-06 2024-09-14T06:30:50.141 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:50.141 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:50.141 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:30:50.141 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370432+0000", 2024-09-14T06:30:50.141 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.2439999999999996e-06 2024-09-14T06:30:50.141 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:50.141 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:50.142 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:30:50.142 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370437+0000", 2024-09-14T06:30:50.142 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.5389999999999998e-06 2024-09-14T06:30:50.142 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:50.142 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:50.142 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:30:50.142 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.417888+0000", 2024-09-14T06:30:50.142 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047450432000000001 2024-09-14T06:30:50.142 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:50.142 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:50.142 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:30:50.142 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.417935+0000", 2024-09-14T06:30:50.142 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.7454999999999999e-05 2024-09-14T06:30:50.142 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:50.142 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:50.143 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:30:50.143 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.417987+0000", 2024-09-14T06:30:50.143 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.2213999999999998e-05 2024-09-14T06:30:50.143 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:50.143 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:30:50.143 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:50.143 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:50.143 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:50.143 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:53341 3.f 3:f71f51a7:::rbd_data.112dca2f6ae1.0000000000000015:17 [read 1081344~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:30:50.143 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:44.370446+0000", 2024-09-14T06:30:50.143 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 65.722913684999995, 2024-09-14T06:30:50.143 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047665723, 2024-09-14T06:30:50.143 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:30:50.143 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:30:50.143 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:30:50.144 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:30:50.144 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:30:50.144 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:50.144 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 53341 2024-09-14T06:30:50.144 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:50.144 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:30:50.144 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:50.144 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:30:50.144 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370446+0000", 2024-09-14T06:30:50.144 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:50.144 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:50.144 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:50.144 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:30:50.144 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370446+0000", 2024-09-14T06:30:50.144 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:50.145 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:50.145 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:50.145 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:30:50.145 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370447+0000", 2024-09-14T06:30:50.145 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.6100000000000002e-07 2024-09-14T06:30:50.145 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:50.145 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:50.145 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:30:50.145 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370448+0000", 2024-09-14T06:30:50.145 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.7370000000000001e-06 2024-09-14T06:30:50.145 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:50.145 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:50.145 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:30:50.145 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370462+0000", 2024-09-14T06:30:50.146 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.3227000000000001e-05 2024-09-14T06:30:50.146 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:50.146 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:50.146 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:30:50.146 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370467+0000", 2024-09-14T06:30:50.146 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.6540000000000004e-06 2024-09-14T06:30:50.146 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:50.146 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:50.146 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:30:50.146 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.418013+0000", 2024-09-14T06:30:50.146 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047545325999999999 2024-09-14T06:30:50.146 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:50.146 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:50.146 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:30:50.146 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.418054+0000", 2024-09-14T06:30:50.147 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.1696999999999997e-05 2024-09-14T06:30:50.147 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:50.147 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:50.147 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:30:50.147 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.418112+0000", 2024-09-14T06:30:50.147 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.7720999999999998e-05 2024-09-14T06:30:50.147 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:50.147 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:30:50.147 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:50.147 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:50.147 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:50.147 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:53342 3.f 3:f71f51a7:::rbd_data.112dca2f6ae1.0000000000000015:17 [read 1105920~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:30:50.147 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:44.370475+0000", 2024-09-14T06:30:50.147 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 65.722884624000002, 2024-09-14T06:30:50.147 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047748848000000003, 2024-09-14T06:30:50.148 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:30:50.148 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:30:50.148 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:30:50.148 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:30:50.148 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:30:50.148 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:50.148 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 53342 2024-09-14T06:30:50.148 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:50.148 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:30:50.148 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:50.148 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:30:50.148 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370475+0000", 2024-09-14T06:30:50.148 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:50.148 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:50.148 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:50.148 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:30:50.149 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370475+0000", 2024-09-14T06:30:50.149 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:50.149 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:50.149 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:50.149 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:30:50.149 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370476+0000", 2024-09-14T06:30:50.149 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.4700000000000002e-07 2024-09-14T06:30:50.149 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:50.149 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:50.149 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:30:50.149 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370478+0000", 2024-09-14T06:30:50.149 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.5890000000000001e-06 2024-09-14T06:30:50.149 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:50.149 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:50.149 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:30:50.150 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370486+0000", 2024-09-14T06:30:50.150 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.3470000000000001e-06 2024-09-14T06:30:50.150 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:50.150 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:50.150 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:30:50.150 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370491+0000", 2024-09-14T06:30:50.150 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.5720000000000002e-06 2024-09-14T06:30:50.150 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:50.150 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:50.150 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:30:50.150 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.418135+0000", 2024-09-14T06:30:50.150 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047644094999999997 2024-09-14T06:30:50.150 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:50.150 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:50.150 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:30:50.151 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.418174+0000", 2024-09-14T06:30:50.151 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.8204000000000002e-05 2024-09-14T06:30:50.151 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:50.151 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:50.151 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:30:50.151 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.418224+0000", 2024-09-14T06:30:50.151 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.0593999999999997e-05 2024-09-14T06:30:50.151 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:50.151 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:30:50.151 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:50.151 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:50.151 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:50.151 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:53343 3.f 3:f71f51a7:::rbd_data.112dca2f6ae1.0000000000000015:17 [read 1130496~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:30:50.151 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:44.370498+0000", 2024-09-14T06:30:50.151 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 65.722861748, 2024-09-14T06:30:50.152 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047841574999999997, 2024-09-14T06:30:50.152 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:30:50.152 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:30:50.152 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:30:50.152 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:30:50.152 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:30:50.152 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:50.152 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 53343 2024-09-14T06:30:50.152 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:50.152 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:30:50.152 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:50.152 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:30:50.152 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370498+0000", 2024-09-14T06:30:50.152 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:50.152 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:50.153 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:50.153 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:30:50.153 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370498+0000", 2024-09-14T06:30:50.153 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:50.153 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:50.153 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:50.153 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:30:50.153 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370499+0000", 2024-09-14T06:30:50.153 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.9999999999999998e-07 2024-09-14T06:30:50.153 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:50.153 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:50.153 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:30:50.153 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370501+0000", 2024-09-14T06:30:50.153 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.0310000000000001e-06 2024-09-14T06:30:50.153 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:50.154 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:50.154 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:30:50.154 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370507+0000", 2024-09-14T06:30:50.154 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.4169999999999997e-06 2024-09-14T06:30:50.154 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:50.154 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:50.154 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:30:50.154 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370516+0000", 2024-09-14T06:30:50.154 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.7190000000000007e-06 2024-09-14T06:30:50.154 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:50.154 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:50.154 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:30:50.154 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.418247+0000", 2024-09-14T06:30:50.154 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047731190999999999 2024-09-14T06:30:50.154 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:50.154 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:50.155 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:30:50.155 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.418286+0000", 2024-09-14T06:30:50.155 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.8988e-05 2024-09-14T06:30:50.155 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:50.155 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:50.155 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:30:50.155 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.418340+0000", 2024-09-14T06:30:50.155 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.3829000000000002e-05 2024-09-14T06:30:50.155 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:50.155 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:30:50.155 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:50.155 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:50.155 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:50.155 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:53344 3.f 3:f71f51a7:::rbd_data.112dca2f6ae1.0000000000000015:17 [read 1155072~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:30:50.155 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:44.370524+0000", 2024-09-14T06:30:50.156 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 65.722835978000006, 2024-09-14T06:30:50.156 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047972262000000002, 2024-09-14T06:30:50.156 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:30:50.156 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:30:50.156 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:30:50.156 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:30:50.156 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:30:50.156 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:50.156 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 53344 2024-09-14T06:30:50.156 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:50.156 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:30:50.156 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:50.156 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:30:50.156 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370524+0000", 2024-09-14T06:30:50.156 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:50.156 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:50.157 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:50.157 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:30:50.157 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370524+0000", 2024-09-14T06:30:50.157 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:50.157 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:50.157 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:50.157 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:30:50.157 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370618+0000", 2024-09-14T06:30:50.157 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.4425999999999993e-05 2024-09-14T06:30:50.157 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:50.157 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:50.157 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:30:50.157 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370622+0000", 2024-09-14T06:30:50.157 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.3519999999999998e-06 2024-09-14T06:30:50.157 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:50.158 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:50.158 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:30:50.158 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370631+0000", 2024-09-14T06:30:50.158 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.9299999999999992e-06 2024-09-14T06:30:50.158 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:50.158 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:50.158 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:30:50.158 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370637+0000", 2024-09-14T06:30:50.158 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.3269999999999997e-06 2024-09-14T06:30:50.158 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:50.158 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:50.158 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:30:50.158 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.418372+0000", 2024-09-14T06:30:50.158 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047734764999999998 2024-09-14T06:30:50.158 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:50.159 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:50.159 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:30:50.159 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.418431+0000", 2024-09-14T06:30:50.159 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.8696999999999997e-05 2024-09-14T06:30:50.159 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:50.159 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:50.159 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:30:50.159 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.418496+0000", 2024-09-14T06:30:50.159 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.5765000000000005e-05 2024-09-14T06:30:50.159 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:50.159 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:30:50.159 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:50.159 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:50.159 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:50.159 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:53345 3.f 3:f71f51a7:::rbd_data.112dca2f6ae1.0000000000000015:17 [read 1179648~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:30:50.159 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:44.370647+0000", 2024-09-14T06:30:50.159 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 65.722712510999997, 2024-09-14T06:30:50.160 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047953775999999997, 2024-09-14T06:30:50.160 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:30:50.160 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:30:50.160 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:30:50.160 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:30:50.160 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:30:50.160 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:50.160 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 53345 2024-09-14T06:30:50.160 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:50.160 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:30:50.160 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:50.160 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:30:50.160 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370647+0000", 2024-09-14T06:30:50.160 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:50.160 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:50.161 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:50.161 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:30:50.161 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370647+0000", 2024-09-14T06:30:50.161 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:50.161 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:50.161 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:50.161 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:30:50.161 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.372346+0000", 2024-09-14T06:30:50.161 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.001698284 2024-09-14T06:30:50.161 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:50.161 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:50.161 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:30:50.161 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.372351+0000", 2024-09-14T06:30:50.162 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.8890000000000001e-06 2024-09-14T06:30:50.162 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:50.162 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:50.162 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:30:50.162 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.372362+0000", 2024-09-14T06:30:50.162 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.0941000000000001e-05 2024-09-14T06:30:50.162 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:50.162 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:50.162 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:30:50.162 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.372369+0000", 2024-09-14T06:30:50.162 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.362e-06 2024-09-14T06:30:50.162 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:50.162 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:50.162 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:30:50.163 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.418510+0000", 2024-09-14T06:30:50.163 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046140914999999998 2024-09-14T06:30:50.163 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:50.163 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:50.163 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:30:50.163 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.418549+0000", 2024-09-14T06:30:50.163 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.8717999999999998e-05 2024-09-14T06:30:50.163 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:50.163 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:50.163 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:30:50.163 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.418601+0000", 2024-09-14T06:30:50.163 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.2667000000000002e-05 2024-09-14T06:30:50.163 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:50.163 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:30:50.163 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:50.164 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:50.164 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:50.164 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:53351 3.f 3:f71f51a7:::rbd_data.112dca2f6ae1.0000000000000015:17 [read 1327104~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:30:50.164 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:44.372499+0000", 2024-09-14T06:30:50.164 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 65.720861389999996, 2024-09-14T06:30:50.164 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047140596, 2024-09-14T06:30:50.164 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:30:50.164 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:30:50.164 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:30:50.164 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:30:50.164 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:30:50.164 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:50.164 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 53351 2024-09-14T06:30:50.164 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:50.164 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:30:50.164 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:50.165 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:30:50.165 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.372499+0000", 2024-09-14T06:30:50.165 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:50.165 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:50.165 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:50.165 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:30:50.165 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.372499+0000", 2024-09-14T06:30:50.165 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:50.165 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:50.165 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:50.165 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:30:50.165 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.372499+0000", 2024-09-14T06:30:50.165 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.3000000000000001e-07 2024-09-14T06:30:50.165 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:50.165 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:50.165 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:30:50.166 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.372501+0000", 2024-09-14T06:30:50.166 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.0820000000000001e-06 2024-09-14T06:30:50.166 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:50.166 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:50.166 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:30:50.166 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.372513+0000", 2024-09-14T06:30:50.166 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.1610999999999999e-05 2024-09-14T06:30:50.166 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:50.166 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:50.166 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:30:50.166 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.372518+0000", 2024-09-14T06:30:50.166 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.327e-06 2024-09-14T06:30:50.166 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:50.166 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:50.166 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:30:50.167 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.419530+0000", 2024-09-14T06:30:50.167 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047011798 2024-09-14T06:30:50.167 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:50.167 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:50.167 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:30:50.167 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.419578+0000", 2024-09-14T06:30:50.167 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.8446999999999999e-05 2024-09-14T06:30:50.167 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:50.167 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:50.167 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:30:50.167 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.419639+0000", 2024-09-14T06:30:50.167 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.0900999999999998e-05 2024-09-14T06:30:50.167 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:50.167 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:30:50.168 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:50.168 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:50.168 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:50.168 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:53352 3.f 3:f71f51a7:::rbd_data.112dca2f6ae1.0000000000000015:17 [read 1351680~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:30:50.168 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:44.372525+0000", 2024-09-14T06:30:50.168 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 65.720835476000005, 2024-09-14T06:30:50.168 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047259847000000001, 2024-09-14T06:30:50.168 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:30:50.168 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:30:50.168 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:30:50.168 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:30:50.168 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:30:50.168 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:50.168 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 53352 2024-09-14T06:30:50.168 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:50.168 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:30:50.169 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:50.169 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:30:50.169 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.372525+0000", 2024-09-14T06:30:50.169 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:50.169 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:50.169 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:50.169 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:30:50.169 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.372525+0000", 2024-09-14T06:30:50.169 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:50.169 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:50.169 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:50.169 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:30:50.169 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.372525+0000", 2024-09-14T06:30:50.169 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.0600000000000001e-07 2024-09-14T06:30:50.169 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:50.170 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:50.170 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:30:50.170 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.372527+0000", 2024-09-14T06:30:50.170 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.0499999999999999e-06 2024-09-14T06:30:50.170 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:50.170 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:50.170 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:30:50.170 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.372533+0000", 2024-09-14T06:30:50.170 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.4320000000000004e-06 2024-09-14T06:30:50.170 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:50.170 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:50.170 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:30:50.170 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.372538+0000", 2024-09-14T06:30:50.170 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.9130000000000002e-06 2024-09-14T06:30:50.171 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:50.171 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:50.171 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:30:50.171 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.419667+0000", 2024-09-14T06:30:50.171 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047128976000000003 2024-09-14T06:30:50.171 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:50.171 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:50.171 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:30:50.171 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.419715+0000", 2024-09-14T06:30:50.171 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.7868999999999999e-05 2024-09-14T06:30:50.171 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:50.171 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:50.171 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:30:50.171 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.419784+0000", 2024-09-14T06:30:50.172 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.9201000000000004e-05 2024-09-14T06:30:50.172 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:50.172 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:30:50.172 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:50.172 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:50.172 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:50.172 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:53353 3.f 3:f71f51a7:::rbd_data.112dca2f6ae1.0000000000000015:17 [read 1376256~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:30:50.172 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:44.372545+0000", 2024-09-14T06:30:50.172 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 65.720815435999995, 2024-09-14T06:30:50.172 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047364232999999999, 2024-09-14T06:30:50.172 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:30:50.172 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:30:50.172 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:30:50.172 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:30:50.172 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:30:50.173 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:50.173 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 53353 2024-09-14T06:30:50.173 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:50.173 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:30:50.173 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:50.173 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:30:50.173 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.372545+0000", 2024-09-14T06:30:50.173 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:50.173 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:50.173 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:50.173 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:30:50.173 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.372545+0000", 2024-09-14T06:30:50.173 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:50.174 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:50.174 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:50.174 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:30:50.174 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.372545+0000", 2024-09-14T06:30:50.174 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.27e-07 2024-09-14T06:30:50.174 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:50.174 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:50.174 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:30:50.174 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.372547+0000", 2024-09-14T06:30:50.174 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.0600000000000002e-06 2024-09-14T06:30:50.174 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:50.174 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:50.174 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:30:50.174 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.372553+0000", 2024-09-14T06:30:50.175 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.7980000000000002e-06 2024-09-14T06:30:50.175 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:50.175 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:50.175 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:30:50.175 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.372557+0000", 2024-09-14T06:30:50.175 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.7389999999999999e-06 2024-09-14T06:30:50.175 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:50.175 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:50.175 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:30:50.175 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.419811+0000", 2024-09-14T06:30:50.175 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047253310999999999 2024-09-14T06:30:50.175 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:50.175 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:50.175 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:30:50.175 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.419854+0000", 2024-09-14T06:30:50.176 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.3448000000000001e-05 2024-09-14T06:30:50.176 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:50.176 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:50.176 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:30:50.176 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.419909+0000", 2024-09-14T06:30:50.176 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.4549999999999998e-05 2024-09-14T06:30:50.176 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:50.176 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:30:50.176 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:50.176 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:50.176 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:50.176 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:53354 3.f 3:f71f51a7:::rbd_data.112dca2f6ae1.0000000000000015:17 [read 1400832~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:30:50.176 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:44.372563+0000", 2024-09-14T06:30:50.176 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 65.720796999000001, 2024-09-14T06:30:50.177 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047482233999999998, 2024-09-14T06:30:50.177 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:30:50.177 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:30:50.177 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:30:50.177 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:30:50.177 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:30:50.177 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:50.177 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 53354 2024-09-14T06:30:50.177 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:50.177 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:30:50.177 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:50.178 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:30:50.178 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.372563+0000", 2024-09-14T06:30:50.178 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:50.178 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:50.178 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:50.178 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:30:50.178 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.372563+0000", 2024-09-14T06:30:50.178 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:50.178 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:50.178 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:50.178 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:30:50.178 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.375009+0000", 2024-09-14T06:30:50.178 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.0024457680000000001 2024-09-14T06:30:50.179 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:50.179 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:50.179 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:30:50.179 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.375014+0000", 2024-09-14T06:30:50.179 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.0599999999999998e-06 2024-09-14T06:30:50.179 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:50.179 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:50.179 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:30:50.179 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.375026+0000", 2024-09-14T06:30:50.179 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.1783999999999999e-05 2024-09-14T06:30:50.179 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:50.179 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:50.179 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:30:50.179 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.375035+0000", 2024-09-14T06:30:50.179 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.7799999999999995e-06 2024-09-14T06:30:50.180 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:50.180 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:50.180 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:30:50.180 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.419942+0000", 2024-09-14T06:30:50.180 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.044906982999999998 2024-09-14T06:30:50.180 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:50.180 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:50.180 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:30:50.180 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.419989+0000", 2024-09-14T06:30:50.180 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.7104999999999997e-05 2024-09-14T06:30:50.180 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:50.180 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:50.180 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:30:50.180 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.420045+0000", 2024-09-14T06:30:50.180 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.5754000000000002e-05 2024-09-14T06:30:50.181 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:50.181 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:30:50.181 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:50.181 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:50.181 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:30:50.181 INFO:teuthology.orchestra.run.smithi138.stdout:} 2024-09-14T06:30:50.182 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-14T06:30:50.183 INFO:tasks.ceph.osd.1.smithi094.stderr:2024-09-14T06:30:50.181+0000 7f328e47a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33641) UID: 0 2024-09-14T06:30:50.283 INFO:tasks.ceph.osd.5.smithi138.stderr:2024-09-14T06:30:50.282+0000 7fe13746f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33618) UID: 0 2024-09-14T06:30:50.299 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-14T06:30:50.299 INFO:teuthology.orchestra.run.smithi094.stdout: "ops": [], 2024-09-14T06:30:50.299 INFO:teuthology.orchestra.run.smithi094.stdout: "num_ops": 0 2024-09-14T06:30:50.299 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-14T06:30:50.308 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-14T06:30:50.383 INFO:tasks.ceph.osd.6.smithi138.stderr:2024-09-14T06:30:50.382+0000 7f8cd8849640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33620) UID: 0 2024-09-14T06:30:50.412 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-14T06:30:50.412 INFO:teuthology.orchestra.run.smithi094.stdout: "ops": [], 2024-09-14T06:30:50.412 INFO:teuthology.orchestra.run.smithi094.stdout: "complaint_time": 30, 2024-09-14T06:30:50.412 INFO:teuthology.orchestra.run.smithi094.stdout: "num_blocked_ops": 0 2024-09-14T06:30:50.412 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-14T06:30:50.420 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-14T06:30:50.485 INFO:tasks.ceph.osd.4.smithi138.stderr:2024-09-14T06:30:50.483+0000 7f6c653c8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33615) UID: 0 2024-09-14T06:30:50.507 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-14T06:30:50.525 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-14T06:30:50.526 INFO:teuthology.orchestra.run.smithi094.stdout: "size": 20, 2024-09-14T06:30:50.526 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 600, 2024-09-14T06:30:50.526 INFO:teuthology.orchestra.run.smithi094.stdout: "ops": [ 2024-09-14T06:30:50.526 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:50.526 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:115193 3.1 3:856be886:::rbd_data.112d4d24c558.0000000000000004:29 [read 1523712~16384 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e96)", 2024-09-14T06:30:50.526 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:30:17.939854+0000", 2024-09-14T06:30:50.526 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 32.583558291999999, 2024-09-14T06:30:50.526 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.073897516999999996, 2024-09-14T06:30:50.526 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:30:50.526 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:30:50.526 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:30:50.526 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:30:50.527 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:30:50.527 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:50.527 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 115193 2024-09-14T06:30:50.527 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:50.527 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:30:50.527 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:50.527 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:30:50.527 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.939854+0000", 2024-09-14T06:30:50.527 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:50.527 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:50.527 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:50.527 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:30:50.527 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.939854+0000", 2024-09-14T06:30:50.527 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:50.528 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:50.528 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:50.528 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:30:50.528 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.939854+0000", 2024-09-14T06:30:50.528 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.9500000000000002e-07 2024-09-14T06:30:50.528 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:50.528 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:50.528 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:30:50.528 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.939856+0000", 2024-09-14T06:30:50.528 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.0959999999999999e-06 2024-09-14T06:30:50.528 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:50.528 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:50.528 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:30:50.528 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.939863+0000", 2024-09-14T06:30:50.528 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.1129999999999997e-06 2024-09-14T06:30:50.529 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:50.529 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:50.529 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:30:50.529 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.939867+0000", 2024-09-14T06:30:50.529 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.8350000000000003e-06 2024-09-14T06:30:50.529 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:50.529 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:50.529 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:30:50.529 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.013626+0000", 2024-09-14T06:30:50.529 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.073758799999999999 2024-09-14T06:30:50.529 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:50.529 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:50.529 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:30:50.529 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.013690+0000", 2024-09-14T06:30:50.530 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.3918999999999996e-05 2024-09-14T06:30:50.530 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:50.530 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:50.530 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:30:50.530 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.013751+0000", 2024-09-14T06:30:50.530 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.1159000000000002e-05 2024-09-14T06:30:50.530 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:50.530 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:50.530 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:50.530 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:50.530 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:50.530 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:115194 3.1 3:856be886:::rbd_data.112d4d24c558.0000000000000004:29 [read 1769472~16384 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e96)", 2024-09-14T06:30:50.530 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:30:17.939874+0000", 2024-09-14T06:30:50.530 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 32.583537980999999, 2024-09-14T06:30:50.530 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.074016988000000006, 2024-09-14T06:30:50.531 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:30:50.531 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:30:50.531 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:30:50.531 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:30:50.531 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:30:50.531 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:50.531 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 115194 2024-09-14T06:30:50.531 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:50.531 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:30:50.531 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:50.531 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:30:50.531 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.939874+0000", 2024-09-14T06:30:50.531 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:50.532 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:50.532 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:50.532 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:30:50.532 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.939874+0000", 2024-09-14T06:30:50.532 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:50.532 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:50.532 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:50.532 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:30:50.532 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.939875+0000", 2024-09-14T06:30:50.532 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.2600000000000002e-07 2024-09-14T06:30:50.532 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:50.532 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:50.532 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:30:50.533 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.939877+0000", 2024-09-14T06:30:50.533 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.9420000000000002e-06 2024-09-14T06:30:50.533 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:50.533 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:50.533 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:30:50.533 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.939882+0000", 2024-09-14T06:30:50.533 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.8749999999999997e-06 2024-09-14T06:30:50.533 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:50.533 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:50.533 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:30:50.533 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.939887+0000", 2024-09-14T06:30:50.533 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.741e-06 2024-09-14T06:30:50.534 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:50.534 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:50.534 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:30:50.534 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.013767+0000", 2024-09-14T06:30:50.534 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.073879720999999995 2024-09-14T06:30:50.534 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:50.534 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:50.534 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:30:50.534 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.013832+0000", 2024-09-14T06:30:50.534 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.5312000000000001e-05 2024-09-14T06:30:50.535 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:50.535 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:50.535 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:30:50.535 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.013891+0000", 2024-09-14T06:30:50.535 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.8870999999999999e-05 2024-09-14T06:30:50.535 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:50.535 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:50.535 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:50.535 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:50.535 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:50.535 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:115195 3.1 3:856be886:::rbd_data.112d4d24c558.0000000000000004:29 [read 2015232~16384 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e96)", 2024-09-14T06:30:50.535 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:30:17.939903+0000", 2024-09-14T06:30:50.535 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 32.583508795999997, 2024-09-14T06:30:50.536 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.074128199000000006, 2024-09-14T06:30:50.536 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:30:50.536 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:30:50.536 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:30:50.536 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:30:50.536 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:30:50.536 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:50.536 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 115195 2024-09-14T06:30:50.536 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:50.536 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:30:50.536 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:50.536 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:30:50.536 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.939903+0000", 2024-09-14T06:30:50.536 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:50.536 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:50.536 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:50.537 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:30:50.537 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.939903+0000", 2024-09-14T06:30:50.537 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:50.537 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:50.537 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:50.537 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:30:50.537 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.939904+0000", 2024-09-14T06:30:50.537 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.5799999999999999e-07 2024-09-14T06:30:50.537 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:50.537 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:50.537 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:30:50.537 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.939906+0000", 2024-09-14T06:30:50.537 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.2350000000000002e-06 2024-09-14T06:30:50.537 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:50.537 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:50.538 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:30:50.538 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.939913+0000", 2024-09-14T06:30:50.538 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.5649999999999998e-06 2024-09-14T06:30:50.538 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:50.538 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:50.538 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:30:50.538 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.939918+0000", 2024-09-14T06:30:50.538 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.784e-06 2024-09-14T06:30:50.538 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:50.538 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:50.538 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:30:50.538 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.013921+0000", 2024-09-14T06:30:50.538 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.074002911000000005 2024-09-14T06:30:50.538 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:50.538 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:50.538 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:30:50.539 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.013971+0000", 2024-09-14T06:30:50.539 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.9551e-05 2024-09-14T06:30:50.539 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:50.539 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:50.539 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:30:50.539 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.014031+0000", 2024-09-14T06:30:50.539 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.0594999999999998e-05 2024-09-14T06:30:50.539 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:50.539 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:50.539 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:50.539 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:50.539 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:50.539 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:115196 3.1 3:856be886:::rbd_data.112d4d24c558.0000000000000004:29 [read 2260992~16384 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e96)", 2024-09-14T06:30:50.539 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:30:17.939925+0000", 2024-09-14T06:30:50.539 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 32.583486604000001, 2024-09-14T06:30:50.540 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.074320137999999994, 2024-09-14T06:30:50.540 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:30:50.540 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:30:50.540 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:30:50.540 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:30:50.540 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:30:50.540 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:50.540 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 115196 2024-09-14T06:30:50.540 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:50.540 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:30:50.540 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:50.540 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:30:50.540 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.939925+0000", 2024-09-14T06:30:50.540 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:50.540 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:50.541 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:50.541 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:30:50.541 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.939925+0000", 2024-09-14T06:30:50.541 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:50.541 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:50.541 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:50.541 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:30:50.541 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.939926+0000", 2024-09-14T06:30:50.541 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.9999999999999998e-07 2024-09-14T06:30:50.541 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:50.541 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:50.541 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:30:50.541 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.939928+0000", 2024-09-14T06:30:50.541 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.9549999999999999e-06 2024-09-14T06:30:50.542 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:50.542 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:50.542 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:30:50.542 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.939934+0000", 2024-09-14T06:30:50.542 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.127e-06 2024-09-14T06:30:50.542 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:50.542 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:50.542 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:30:50.542 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.939939+0000", 2024-09-14T06:30:50.542 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.775e-06 2024-09-14T06:30:50.542 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:50.542 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:50.542 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:30:50.542 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.014075+0000", 2024-09-14T06:30:50.542 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.074136598999999997 2024-09-14T06:30:50.543 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:50.543 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:50.543 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:30:50.543 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.014162+0000", 2024-09-14T06:30:50.543 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.6477999999999994e-05 2024-09-14T06:30:50.543 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:50.543 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:50.543 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:30:50.543 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.014246+0000", 2024-09-14T06:30:50.543 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.3703999999999997e-05 2024-09-14T06:30:50.543 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:50.543 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:50.543 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:50.543 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:50.543 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:50.544 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:115197 3.1 3:856be886:::rbd_data.112d4d24c558.0000000000000004:29 [read 2506752~16384 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e96)", 2024-09-14T06:30:50.544 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:30:17.939969+0000", 2024-09-14T06:30:50.544 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 32.583443119000002, 2024-09-14T06:30:50.544 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.074528544000000002, 2024-09-14T06:30:50.544 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:30:50.544 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:30:50.544 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:30:50.544 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:30:50.544 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:30:50.544 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:50.544 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 115197 2024-09-14T06:30:50.544 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:50.544 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:30:50.544 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:50.545 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:30:50.545 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.939969+0000", 2024-09-14T06:30:50.545 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:50.545 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:50.545 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:50.545 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:30:50.545 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.939969+0000", 2024-09-14T06:30:50.545 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:50.545 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:50.545 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:50.545 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:30:50.545 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.939970+0000", 2024-09-14T06:30:50.545 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.3600000000000003e-07 2024-09-14T06:30:50.545 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:50.545 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:50.546 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:30:50.546 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.939972+0000", 2024-09-14T06:30:50.546 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.3180000000000001e-06 2024-09-14T06:30:50.546 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:50.546 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:50.546 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:30:50.546 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.939979+0000", 2024-09-14T06:30:50.546 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.8800000000000002e-06 2024-09-14T06:30:50.546 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:50.546 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:50.546 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:30:50.546 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.939984+0000", 2024-09-14T06:30:50.546 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.3510000000000001e-06 2024-09-14T06:30:50.546 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:50.546 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:50.546 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:30:50.547 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.014302+0000", 2024-09-14T06:30:50.547 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.074317416999999997 2024-09-14T06:30:50.547 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:50.547 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:50.547 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:30:50.547 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.014392+0000", 2024-09-14T06:30:50.547 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.0181999999999998e-05 2024-09-14T06:30:50.547 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:50.547 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:50.547 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:30:50.547 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.014497+0000", 2024-09-14T06:30:50.547 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00010576 2024-09-14T06:30:50.547 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:50.548 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:50.548 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:50.548 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:50.548 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:50.548 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:115198 3.1 3:856be886:::rbd_data.112d4d24c558.0000000000000004:29 [read 2752512~16384 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e96)", 2024-09-14T06:30:50.548 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:30:17.940004+0000", 2024-09-14T06:30:50.548 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 32.583407839000003, 2024-09-14T06:30:50.548 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.074736367999999997, 2024-09-14T06:30:50.548 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:30:50.548 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:30:50.548 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:30:50.548 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:30:50.548 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:30:50.548 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:50.548 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 115198 2024-09-14T06:30:50.549 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:50.549 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:30:50.549 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:50.549 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:30:50.549 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940004+0000", 2024-09-14T06:30:50.549 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:50.549 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:50.549 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:50.549 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:30:50.549 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940004+0000", 2024-09-14T06:30:50.549 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:50.549 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:50.549 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:50.549 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:30:50.550 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940005+0000", 2024-09-14T06:30:50.550 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.2200000000000004e-07 2024-09-14T06:30:50.550 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:50.550 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:50.550 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:30:50.550 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940007+0000", 2024-09-14T06:30:50.550 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.322e-06 2024-09-14T06:30:50.550 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:50.550 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:50.550 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:30:50.550 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940014+0000", 2024-09-14T06:30:50.550 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.6220000000000003e-06 2024-09-14T06:30:50.551 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:50.551 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:50.551 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:30:50.551 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940019+0000", 2024-09-14T06:30:50.551 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.6380000000000001e-06 2024-09-14T06:30:50.551 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:50.551 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:50.551 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:30:50.551 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.014562+0000", 2024-09-14T06:30:50.551 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.074542320999999995 2024-09-14T06:30:50.551 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:50.551 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:50.551 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:30:50.551 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.014646+0000", 2024-09-14T06:30:50.552 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.4079999999999995e-05 2024-09-14T06:30:50.552 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:50.552 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:50.552 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:30:50.552 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.014741+0000", 2024-09-14T06:30:50.552 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.4763000000000001e-05 2024-09-14T06:30:50.552 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:50.552 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:50.552 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:50.552 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:50.552 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:50.552 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:115199 3.1 3:856be886:::rbd_data.112d4d24c558.0000000000000004:29 [read 2998272~16384 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e96)", 2024-09-14T06:30:50.552 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:30:17.940040+0000", 2024-09-14T06:30:50.552 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 32.583372083999997, 2024-09-14T06:30:50.553 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.074957134999999994, 2024-09-14T06:30:50.553 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:30:50.553 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:30:50.553 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:30:50.553 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:30:50.553 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:30:50.553 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:50.553 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 115199 2024-09-14T06:30:50.553 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:50.553 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:30:50.553 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:50.553 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:30:50.553 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940040+0000", 2024-09-14T06:30:50.554 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:50.554 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:50.554 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:50.554 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:30:50.554 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940040+0000", 2024-09-14T06:30:50.554 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:50.554 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:50.554 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:50.554 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:30:50.554 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940041+0000", 2024-09-14T06:30:50.554 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.4000000000000001e-07 2024-09-14T06:30:50.554 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:50.554 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:50.554 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:30:50.554 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940046+0000", 2024-09-14T06:30:50.554 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.7250000000000002e-06 2024-09-14T06:30:50.555 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:50.555 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:50.555 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:30:50.555 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940053+0000", 2024-09-14T06:30:50.555 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.6109999999999999e-06 2024-09-14T06:30:50.555 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:50.555 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:50.555 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:30:50.555 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940058+0000", 2024-09-14T06:30:50.555 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.1590000000000001e-06 2024-09-14T06:30:50.555 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:50.555 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:50.555 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:30:50.555 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.014768+0000", 2024-09-14T06:30:50.556 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.074709624000000002 2024-09-14T06:30:50.556 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:50.556 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:50.556 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:30:50.556 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.014907+0000", 2024-09-14T06:30:50.556 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000139665 2024-09-14T06:30:50.556 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:50.556 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:50.556 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:30:50.556 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.014997+0000", 2024-09-14T06:30:50.556 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.9710999999999995e-05 2024-09-14T06:30:50.556 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:50.556 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:50.556 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:50.556 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:50.556 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:50.557 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:115200 3.1 3:856be886:::rbd_data.112d4d24c558.0000000000000004:29 [read 3244032~16384 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e96)", 2024-09-14T06:30:50.557 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:30:17.940080+0000", 2024-09-14T06:30:50.557 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 32.583331878999999, 2024-09-14T06:30:50.557 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.075137291999999994, 2024-09-14T06:30:50.557 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:30:50.557 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:30:50.557 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:30:50.557 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:30:50.557 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:30:50.557 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:50.557 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 115200 2024-09-14T06:30:50.557 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:50.557 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:30:50.557 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:50.557 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:30:50.557 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940080+0000", 2024-09-14T06:30:50.558 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:50.558 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:50.558 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:50.558 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:30:50.558 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940080+0000", 2024-09-14T06:30:50.558 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:50.558 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:50.558 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:50.558 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:30:50.558 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940081+0000", 2024-09-14T06:30:50.558 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.0800000000000004e-07 2024-09-14T06:30:50.558 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:50.558 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:50.558 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:30:50.558 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940083+0000", 2024-09-14T06:30:50.558 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.3920000000000001e-06 2024-09-14T06:30:50.559 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:50.559 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:50.559 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:30:50.559 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940091+0000", 2024-09-14T06:30:50.559 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.1389999999999995e-06 2024-09-14T06:30:50.559 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:50.559 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:50.559 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:30:50.559 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940097+0000", 2024-09-14T06:30:50.559 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.5550000000000002e-06 2024-09-14T06:30:50.559 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:50.559 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:50.559 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:30:50.559 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.015025+0000", 2024-09-14T06:30:50.559 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.074928579999999995 2024-09-14T06:30:50.559 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:50.560 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:50.560 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:30:50.560 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.015096+0000", 2024-09-14T06:30:50.560 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.0705999999999997e-05 2024-09-14T06:30:50.560 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:50.560 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:50.560 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:30:50.560 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.015217+0000", 2024-09-14T06:30:50.560 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00012131200000000001 2024-09-14T06:30:50.560 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:50.560 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:50.560 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:50.560 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:50.560 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:50.561 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:115201 3.1 3:856be886:::rbd_data.112d4d24c558.0000000000000004:29 [read 3489792~16384 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e96)", 2024-09-14T06:30:50.561 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:30:17.940113+0000", 2024-09-14T06:30:50.561 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 32.583299105999998, 2024-09-14T06:30:50.561 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.075246899000000006, 2024-09-14T06:30:50.561 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:30:50.561 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:30:50.561 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:30:50.561 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:30:50.561 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:30:50.561 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:50.561 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 115201 2024-09-14T06:30:50.561 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:50.561 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:30:50.561 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:50.561 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:30:50.562 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940113+0000", 2024-09-14T06:30:50.562 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:50.562 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:50.562 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:50.562 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:30:50.562 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940113+0000", 2024-09-14T06:30:50.562 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:50.562 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:50.562 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:50.562 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:30:50.562 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940114+0000", 2024-09-14T06:30:50.562 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.1500000000000004e-07 2024-09-14T06:30:50.562 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:50.562 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:50.563 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:30:50.563 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940116+0000", 2024-09-14T06:30:50.563 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.2809999999999998e-06 2024-09-14T06:30:50.563 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:50.563 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:50.563 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:30:50.563 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940123+0000", 2024-09-14T06:30:50.563 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.9229999999999996e-06 2024-09-14T06:30:50.563 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:50.563 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:50.564 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:30:50.564 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940128+0000", 2024-09-14T06:30:50.564 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.7790000000000003e-06 2024-09-14T06:30:50.564 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:50.564 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:50.564 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:30:50.564 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.015253+0000", 2024-09-14T06:30:50.564 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.075125792999999996 2024-09-14T06:30:50.564 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:50.564 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:50.564 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:30:50.564 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.015297+0000", 2024-09-14T06:30:50.564 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.3847000000000002e-05 2024-09-14T06:30:50.565 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:50.565 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:50.565 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:30:50.565 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.015360+0000", 2024-09-14T06:30:50.565 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.2661000000000002e-05 2024-09-14T06:30:50.565 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:50.565 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:50.565 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:50.565 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:50.565 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:50.565 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:115202 3.1 3:856be886:::rbd_data.112d4d24c558.0000000000000004:29 [read 3735552~16384 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e96)", 2024-09-14T06:30:50.565 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:30:17.940135+0000", 2024-09-14T06:30:50.565 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 32.583277326999998, 2024-09-14T06:30:50.565 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.075451105000000004, 2024-09-14T06:30:50.565 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:30:50.566 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:30:50.566 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:30:50.566 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:30:50.566 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:30:50.566 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:50.566 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 115202 2024-09-14T06:30:50.566 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:50.566 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:30:50.566 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:50.566 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:30:50.566 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940135+0000", 2024-09-14T06:30:50.566 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:50.566 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:50.567 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:50.567 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:30:50.567 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940135+0000", 2024-09-14T06:30:50.567 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:50.567 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:50.567 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:50.567 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:30:50.567 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940135+0000", 2024-09-14T06:30:50.567 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.0999999999999999e-07 2024-09-14T06:30:50.567 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:50.567 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:50.567 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:30:50.567 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940137+0000", 2024-09-14T06:30:50.567 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.0020000000000001e-06 2024-09-14T06:30:50.567 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:50.568 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:50.568 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:30:50.568 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940143+0000", 2024-09-14T06:30:50.568 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.9850000000000003e-06 2024-09-14T06:30:50.568 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:50.568 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:50.568 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:30:50.568 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940148+0000", 2024-09-14T06:30:50.568 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.9880000000000004e-06 2024-09-14T06:30:50.568 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:50.568 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:50.568 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:30:50.568 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.015400+0000", 2024-09-14T06:30:50.568 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.075252186999999998 2024-09-14T06:30:50.568 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:50.569 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:50.569 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:30:50.569 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.015481+0000", 2024-09-14T06:30:50.569 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.0698000000000006e-05 2024-09-14T06:30:50.569 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:50.569 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:50.569 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:30:50.569 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.015586+0000", 2024-09-14T06:30:50.569 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00010483499999999999 2024-09-14T06:30:50.569 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:50.569 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:50.569 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:50.569 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:50.570 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:50.570 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:115203 3.1 3:856be886:::rbd_data.112d4d24c558.0000000000000004:29 [read 3981312~16384 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e96)", 2024-09-14T06:30:50.570 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:30:17.940156+0000", 2024-09-14T06:30:50.570 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 32.583256102, 2024-09-14T06:30:50.570 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.075620860999999998, 2024-09-14T06:30:50.570 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:30:50.570 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:30:50.570 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:30:50.570 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:30:50.570 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:30:50.570 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:50.570 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 115203 2024-09-14T06:30:50.570 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:50.570 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:30:50.571 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:50.571 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:30:50.571 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940156+0000", 2024-09-14T06:30:50.571 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:50.571 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:50.571 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:50.571 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:30:50.571 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940156+0000", 2024-09-14T06:30:50.571 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:50.571 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:50.571 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:50.572 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:30:50.572 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940156+0000", 2024-09-14T06:30:50.572 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.1500000000000005e-07 2024-09-14T06:30:50.572 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:50.572 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:50.572 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:30:50.572 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940158+0000", 2024-09-14T06:30:50.572 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.967e-06 2024-09-14T06:30:50.572 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:50.572 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:50.573 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:30:50.573 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940165+0000", 2024-09-14T06:30:50.573 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.6939999999999998e-06 2024-09-14T06:30:50.573 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:50.573 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:50.573 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:30:50.573 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940170+0000", 2024-09-14T06:30:50.573 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.7539999999999997e-06 2024-09-14T06:30:50.573 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:50.573 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:50.573 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:30:50.573 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.015633+0000", 2024-09-14T06:30:50.573 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.075463334000000007 2024-09-14T06:30:50.573 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:50.574 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:50.574 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:30:50.574 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.015693+0000", 2024-09-14T06:30:50.574 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.9367000000000003e-05 2024-09-14T06:30:50.574 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:50.574 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:50.574 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:30:50.574 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.015777+0000", 2024-09-14T06:30:50.574 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.4229999999999998e-05 2024-09-14T06:30:50.574 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:50.574 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:50.574 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:50.574 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:50.574 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:50.574 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:115204 3.1 3:856be886:::rbd_data.112d4d24c558.0000000000000004:29 [read 4227072~16384 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e96)", 2024-09-14T06:30:50.575 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:30:17.940176+0000", 2024-09-14T06:30:50.575 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 32.583235569000003, 2024-09-14T06:30:50.575 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.075785505000000003, 2024-09-14T06:30:50.575 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:30:50.575 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:30:50.575 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:30:50.575 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:30:50.575 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:30:50.575 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:50.575 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 115204 2024-09-14T06:30:50.575 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:50.575 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:30:50.575 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:50.575 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:30:50.576 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940176+0000", 2024-09-14T06:30:50.576 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:50.576 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:50.576 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:50.576 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:30:50.576 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940176+0000", 2024-09-14T06:30:50.576 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:50.576 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:50.576 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:50.576 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:30:50.576 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940177+0000", 2024-09-14T06:30:50.576 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.7899999999999999e-07 2024-09-14T06:30:50.576 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:50.576 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:50.577 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:30:50.577 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940179+0000", 2024-09-14T06:30:50.577 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.103e-06 2024-09-14T06:30:50.577 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:50.577 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:50.577 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:30:50.577 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940185+0000", 2024-09-14T06:30:50.577 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.7239999999999997e-06 2024-09-14T06:30:50.577 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:50.577 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:50.577 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:30:50.577 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940190+0000", 2024-09-14T06:30:50.577 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.7269999999999998e-06 2024-09-14T06:30:50.577 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:50.578 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:50.578 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:30:50.578 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.015814+0000", 2024-09-14T06:30:50.578 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.075624114000000006 2024-09-14T06:30:50.578 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:50.578 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:50.578 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:30:50.578 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.015862+0000", 2024-09-14T06:30:50.578 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.8770999999999998e-05 2024-09-14T06:30:50.578 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:50.578 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:50.578 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:30:50.578 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.015962+0000", 2024-09-14T06:30:50.578 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.9586999999999994e-05 2024-09-14T06:30:50.578 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:50.578 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:50.579 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:50.579 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:50.579 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:50.579 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:115205 3.1 3:856be886:::rbd_data.112d4d24c558.0000000000000004:29 [read 4472832~16384 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e96)", 2024-09-14T06:30:50.579 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:30:17.940197+0000", 2024-09-14T06:30:50.579 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 32.583214859999998, 2024-09-14T06:30:50.579 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.076068920999999998, 2024-09-14T06:30:50.579 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:30:50.579 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:30:50.579 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:30:50.579 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:30:50.579 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:30:50.579 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:50.579 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 115205 2024-09-14T06:30:50.580 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:50.580 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:30:50.580 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:50.580 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:30:50.580 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940197+0000", 2024-09-14T06:30:50.580 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:50.580 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:50.580 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:50.580 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:30:50.580 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940197+0000", 2024-09-14T06:30:50.580 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:50.580 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:50.580 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:50.580 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:30:50.581 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940198+0000", 2024-09-14T06:30:50.581 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.5999999999999999e-07 2024-09-14T06:30:50.581 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:50.581 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:50.581 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:30:50.581 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940200+0000", 2024-09-14T06:30:50.581 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.0899999999999999e-06 2024-09-14T06:30:50.581 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:50.581 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:50.581 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:30:50.581 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940206+0000", 2024-09-14T06:30:50.581 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.8710000000000002e-06 2024-09-14T06:30:50.581 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:50.581 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:50.582 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:30:50.582 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940211+0000", 2024-09-14T06:30:50.582 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.9230000000000001e-06 2024-09-14T06:30:50.582 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:50.582 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:50.582 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:30:50.582 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.016059+0000", 2024-09-14T06:30:50.582 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.075848051999999999 2024-09-14T06:30:50.582 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:50.582 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:50.582 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:30:50.582 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.016151+0000", 2024-09-14T06:30:50.582 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.1982000000000001e-05 2024-09-14T06:30:50.582 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:50.583 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:50.583 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:30:50.583 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.016266+0000", 2024-09-14T06:30:50.583 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000115543 2024-09-14T06:30:50.583 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:50.583 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:50.583 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:50.583 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:50.583 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:50.583 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:115206 3.1 3:856be886:::rbd_data.112d4d24c558.0000000000000004:29 [read 4718592~16384 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e96)", 2024-09-14T06:30:50.583 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:30:17.940223+0000", 2024-09-14T06:30:50.583 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 32.58318877, 2024-09-14T06:30:50.583 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.076265115999999994, 2024-09-14T06:30:50.583 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:30:50.584 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:30:50.584 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:30:50.584 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:30:50.584 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:30:50.584 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:50.584 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 115206 2024-09-14T06:30:50.584 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:50.584 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:30:50.584 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:50.584 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:30:50.584 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940223+0000", 2024-09-14T06:30:50.584 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:50.584 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:50.584 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:50.584 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:30:50.585 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940223+0000", 2024-09-14T06:30:50.585 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:50.585 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:50.585 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:50.585 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:30:50.585 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940224+0000", 2024-09-14T06:30:50.585 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.0399999999999996e-07 2024-09-14T06:30:50.585 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:50.585 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:50.585 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:30:50.585 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940226+0000", 2024-09-14T06:30:50.585 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.1780000000000002e-06 2024-09-14T06:30:50.585 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:50.585 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:50.585 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:30:50.586 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940232+0000", 2024-09-14T06:30:50.586 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.5010000000000001e-06 2024-09-14T06:30:50.586 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:50.586 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:50.586 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:30:50.586 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940237+0000", 2024-09-14T06:30:50.586 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.0250000000000002e-06 2024-09-14T06:30:50.586 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:50.586 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:50.586 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:30:50.586 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.016305+0000", 2024-09-14T06:30:50.586 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.076067262999999996 2024-09-14T06:30:50.586 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:50.586 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:50.586 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:30:50.586 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.016383+0000", 2024-09-14T06:30:50.587 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.8578000000000006e-05 2024-09-14T06:30:50.587 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:50.587 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:50.587 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:30:50.587 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.016488+0000", 2024-09-14T06:30:50.587 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000105067 2024-09-14T06:30:50.587 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:50.587 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:50.587 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:50.587 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:50.587 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:50.587 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:115207 3.1 3:856be886:::rbd_data.112d4d24c558.0000000000000004:29 [read 4964352~16384 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e96)", 2024-09-14T06:30:50.587 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:30:17.940244+0000", 2024-09-14T06:30:50.587 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 32.583167654, 2024-09-14T06:30:50.587 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.076371154999999996, 2024-09-14T06:30:50.588 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:30:50.588 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:30:50.588 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:30:50.588 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:30:50.588 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:30:50.588 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:50.588 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 115207 2024-09-14T06:30:50.588 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:50.588 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:30:50.588 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:50.588 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:30:50.588 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940244+0000", 2024-09-14T06:30:50.588 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:50.588 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:50.589 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:50.589 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:30:50.589 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940244+0000", 2024-09-14T06:30:50.589 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:50.589 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:50.589 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:50.589 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:30:50.589 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940245+0000", 2024-09-14T06:30:50.589 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.1200000000000003e-07 2024-09-14T06:30:50.589 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:50.589 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:50.589 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:30:50.589 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940247+0000", 2024-09-14T06:30:50.590 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.0030000000000001e-06 2024-09-14T06:30:50.590 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:50.590 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:50.590 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:30:50.590 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940252+0000", 2024-09-14T06:30:50.590 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.5520000000000004e-06 2024-09-14T06:30:50.590 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:50.590 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:50.590 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:30:50.590 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940257+0000", 2024-09-14T06:30:50.590 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.6759999999999998e-06 2024-09-14T06:30:50.590 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:50.590 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:50.590 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:30:50.590 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.016521+0000", 2024-09-14T06:30:50.591 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.076263499999999998 2024-09-14T06:30:50.591 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:50.591 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:50.591 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:30:50.591 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.016562+0000", 2024-09-14T06:30:50.591 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.0967000000000003e-05 2024-09-14T06:30:50.591 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:50.591 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:50.591 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:30:50.591 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.016616+0000", 2024-09-14T06:30:50.591 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.3944999999999999e-05 2024-09-14T06:30:50.591 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:50.591 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:50.591 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:50.592 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:50.592 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:50.592 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:115208 3.1 3:856be886:::rbd_data.112d4d24c558.0000000000000004:29 [read 5210112~16384 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e96)", 2024-09-14T06:30:50.592 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:30:17.940264+0000", 2024-09-14T06:30:50.592 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 32.583147797000002, 2024-09-14T06:30:50.592 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.076558288000000002, 2024-09-14T06:30:50.592 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:30:50.592 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:30:50.592 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:30:50.592 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:30:50.592 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:30:50.592 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:50.592 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 115208 2024-09-14T06:30:50.593 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:50.593 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:30:50.593 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:50.593 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:30:50.593 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940264+0000", 2024-09-14T06:30:50.593 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:50.593 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:50.593 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:50.593 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:30:50.593 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940264+0000", 2024-09-14T06:30:50.593 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:50.593 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:50.593 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:50.593 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:30:50.594 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940265+0000", 2024-09-14T06:30:50.594 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.0500000000000004e-07 2024-09-14T06:30:50.594 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:50.594 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:50.594 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:30:50.594 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940267+0000", 2024-09-14T06:30:50.594 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.9810000000000002e-06 2024-09-14T06:30:50.594 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:50.594 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:50.594 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:30:50.594 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940272+0000", 2024-09-14T06:30:50.594 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.6999999999999996e-06 2024-09-14T06:30:50.594 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:50.595 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:50.595 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:30:50.595 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940277+0000", 2024-09-14T06:30:50.595 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.6480000000000002e-06 2024-09-14T06:30:50.595 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:50.595 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:50.595 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:30:50.595 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.016650+0000", 2024-09-14T06:30:50.595 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.076373089000000005 2024-09-14T06:30:50.595 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:50.595 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:50.596 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:30:50.596 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.016724+0000", 2024-09-14T06:30:50.596 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.4105999999999998e-05 2024-09-14T06:30:50.596 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:50.596 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:50.596 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:30:50.596 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.016823+0000", 2024-09-14T06:30:50.596 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.8258999999999996e-05 2024-09-14T06:30:50.596 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:50.596 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:50.596 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:50.596 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:50.596 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:50.596 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:115209 3.1 3:856be886:::rbd_data.112d4d24c558.0000000000000004:29 [read 5455872~16384 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e96)", 2024-09-14T06:30:50.596 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:30:17.940283+0000", 2024-09-14T06:30:50.596 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 32.583128770999998, 2024-09-14T06:30:50.597 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.076724029999999999, 2024-09-14T06:30:50.597 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:30:50.597 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:30:50.597 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:30:50.597 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:30:50.597 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:30:50.597 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:50.597 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 115209 2024-09-14T06:30:50.597 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:50.597 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:30:50.597 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:50.597 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:30:50.597 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940283+0000", 2024-09-14T06:30:50.597 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:50.598 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:50.598 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:50.598 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:30:50.598 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940283+0000", 2024-09-14T06:30:50.598 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:50.598 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:50.598 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:50.598 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:30:50.598 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940284+0000", 2024-09-14T06:30:50.598 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.6700000000000003e-07 2024-09-14T06:30:50.598 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:50.598 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:50.598 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:30:50.598 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940286+0000", 2024-09-14T06:30:50.598 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.9520000000000001e-06 2024-09-14T06:30:50.599 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:50.599 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:50.599 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:30:50.599 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940292+0000", 2024-09-14T06:30:50.599 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.7339999999999996e-06 2024-09-14T06:30:50.599 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:50.599 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:50.599 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:30:50.599 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940296+0000", 2024-09-14T06:30:50.599 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.7700000000000001e-06 2024-09-14T06:30:50.599 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:50.599 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:50.599 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:30:50.599 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.016859+0000", 2024-09-14T06:30:50.599 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.07656317 2024-09-14T06:30:50.599 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:50.600 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:50.600 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:30:50.600 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.016929+0000", 2024-09-14T06:30:50.600 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.9679999999999997e-05 2024-09-14T06:30:50.600 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:50.600 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:50.600 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:30:50.600 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.017007+0000", 2024-09-14T06:30:50.600 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.8157000000000004e-05 2024-09-14T06:30:50.600 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:50.600 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:50.600 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:50.600 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:50.600 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:50.600 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:115210 3.1 3:856be886:::rbd_data.112d4d24c558.0000000000000004:29 [read 5701632~16384 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e96)", 2024-09-14T06:30:50.600 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:30:17.940303+0000", 2024-09-14T06:30:50.601 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 32.583109254, 2024-09-14T06:30:50.601 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.076821358000000006, 2024-09-14T06:30:50.601 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:30:50.601 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:30:50.601 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:30:50.601 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:30:50.601 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:30:50.601 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:50.601 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 115210 2024-09-14T06:30:50.601 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:50.601 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:30:50.601 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:50.601 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:30:50.601 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940303+0000", 2024-09-14T06:30:50.601 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:50.602 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:50.602 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:50.602 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:30:50.602 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940303+0000", 2024-09-14T06:30:50.602 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:50.602 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:50.602 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:50.602 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:30:50.602 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940303+0000", 2024-09-14T06:30:50.602 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.3000000000000001e-07 2024-09-14T06:30:50.602 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:50.602 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:50.602 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:30:50.602 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940306+0000", 2024-09-14T06:30:50.602 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.255e-06 2024-09-14T06:30:50.602 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:50.603 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:50.603 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:30:50.603 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940312+0000", 2024-09-14T06:30:50.603 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.9710000000000001e-06 2024-09-14T06:30:50.603 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:50.603 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:50.603 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:30:50.603 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940316+0000", 2024-09-14T06:30:50.603 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.5340000000000001e-06 2024-09-14T06:30:50.603 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:50.603 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:50.603 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:30:50.603 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.017033+0000", 2024-09-14T06:30:50.603 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.076717169000000002 2024-09-14T06:30:50.604 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:50.604 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:50.604 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:30:50.604 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.017073+0000", 2024-09-14T06:30:50.604 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.9582000000000002e-05 2024-09-14T06:30:50.604 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:50.604 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:50.604 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:30:50.604 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.017124+0000", 2024-09-14T06:30:50.604 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.1316999999999997e-05 2024-09-14T06:30:50.604 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:50.604 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:50.604 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:50.604 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:50.604 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:50.604 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:115219 3.1 3:856be886:::rbd_data.112d4d24c558.0000000000000004:29 [read 65536~16384 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e96)", 2024-09-14T06:30:50.605 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:30:17.940323+0000", 2024-09-14T06:30:50.605 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 32.583089393000002, 2024-09-14T06:30:50.605 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.076915205, 2024-09-14T06:30:50.605 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:30:50.605 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:30:50.605 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:30:50.605 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:30:50.605 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:30:50.605 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:50.605 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 115219 2024-09-14T06:30:50.605 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:50.605 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:30:50.606 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:50.606 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:30:50.606 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940323+0000", 2024-09-14T06:30:50.606 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:50.606 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:50.606 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:50.606 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:30:50.606 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940323+0000", 2024-09-14T06:30:50.606 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:50.606 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:50.606 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:50.606 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:30:50.606 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940323+0000", 2024-09-14T06:30:50.606 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.0800000000000005e-07 2024-09-14T06:30:50.606 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:50.607 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:50.607 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:30:50.607 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940328+0000", 2024-09-14T06:30:50.607 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.887e-06 2024-09-14T06:30:50.607 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:50.607 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:50.607 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:30:50.607 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940334+0000", 2024-09-14T06:30:50.607 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.3269999999999997e-06 2024-09-14T06:30:50.607 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:50.607 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:50.607 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:30:50.607 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940339+0000", 2024-09-14T06:30:50.608 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.7960000000000003e-06 2024-09-14T06:30:50.608 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:50.608 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:50.608 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:30:50.608 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.017148+0000", 2024-09-14T06:30:50.608 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.076809125000000006 2024-09-14T06:30:50.608 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:50.608 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:50.608 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:30:50.608 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.017186+0000", 2024-09-14T06:30:50.608 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.7283000000000002e-05 2024-09-14T06:30:50.608 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:50.608 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:50.608 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:30:50.609 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.017238+0000", 2024-09-14T06:30:50.609 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.2278999999999999e-05 2024-09-14T06:30:50.609 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:50.609 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:50.609 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:50.609 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:50.609 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:50.609 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:115220 3.1 3:856be886:::rbd_data.112d4d24c558.0000000000000004:29 [read 311296~16384 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e96)", 2024-09-14T06:30:50.609 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:30:17.940346+0000", 2024-09-14T06:30:50.609 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 32.583066264000003, 2024-09-14T06:30:50.609 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.077033293000000003, 2024-09-14T06:30:50.609 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:30:50.609 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:30:50.609 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:30:50.610 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:30:50.610 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:30:50.610 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:50.610 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 115220 2024-09-14T06:30:50.610 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:50.610 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:30:50.610 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:50.610 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:30:50.610 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940346+0000", 2024-09-14T06:30:50.610 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:50.610 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:50.610 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:50.610 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:30:50.610 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940346+0000", 2024-09-14T06:30:50.610 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:50.610 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:50.611 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:50.611 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:30:50.611 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940346+0000", 2024-09-14T06:30:50.611 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.4300000000000003e-07 2024-09-14T06:30:50.611 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:50.611 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:50.611 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:30:50.611 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940348+0000", 2024-09-14T06:30:50.611 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.0030000000000001e-06 2024-09-14T06:30:50.611 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:50.611 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:50.611 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:30:50.611 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.004655+0000", 2024-09-14T06:30:50.612 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.064306323999999998 2024-09-14T06:30:50.612 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:50.612 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:50.612 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:30:50.612 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.004671+0000", 2024-09-14T06:30:50.612 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.6589999999999999e-05 2024-09-14T06:30:50.612 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:50.612 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:50.612 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:30:50.612 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.017262+0000", 2024-09-14T06:30:50.612 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.012590570000000001 2024-09-14T06:30:50.612 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:50.612 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:50.613 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:30:50.613 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.017321+0000", 2024-09-14T06:30:50.613 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.9210999999999999e-05 2024-09-14T06:30:50.613 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:50.613 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:50.613 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:30:50.613 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.017379+0000", 2024-09-14T06:30:50.613 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.8051999999999999e-05 2024-09-14T06:30:50.613 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:50.613 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:50.613 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:50.613 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:50.613 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:50.614 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-14T06:30:50.615 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-14T06:30:50.615 INFO:tasks.ceph.osd.6.smithi138.stderr:2024-09-14T06:30:50.613+0000 7f8cd8849640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33620) UID: 0 2024-09-14T06:30:50.663 INFO:teuthology.orchestra.run.smithi094.stdout:osd.0: {} 2024-09-14T06:30:50.663 INFO:teuthology.orchestra.run.smithi094.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-14T06:30:50.680 INFO:teuthology.orchestra.run.smithi094.stdout:osd.1: {} 2024-09-14T06:30:50.680 INFO:teuthology.orchestra.run.smithi094.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-14T06:30:50.696 INFO:teuthology.orchestra.run.smithi094.stdout:osd.2: {} 2024-09-14T06:30:50.697 INFO:teuthology.orchestra.run.smithi094.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-14T06:30:50.712 INFO:teuthology.orchestra.run.smithi094.stdout:osd.3: {} 2024-09-14T06:30:50.713 INFO:teuthology.orchestra.run.smithi094.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-14T06:30:50.716 INFO:tasks.ceph.osd.3.smithi094.stderr:2024-09-14T06:30:50.715+0000 7feeeafe8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33655) UID: 0 2024-09-14T06:30:50.731 INFO:teuthology.orchestra.run.smithi094.stdout:osd.4: {} 2024-09-14T06:30:50.731 INFO:teuthology.orchestra.run.smithi094.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-14T06:30:50.735 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-14T06:30:50.748 INFO:teuthology.orchestra.run.smithi094.stdout:osd.5: {} 2024-09-14T06:30:50.748 INFO:teuthology.orchestra.run.smithi094.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-14T06:30:50.761 INFO:teuthology.orchestra.run.smithi094.stdout:ERROR: (22) Invalid argument 2024-09-14T06:30:50.761 INFO:teuthology.orchestra.run.smithi094.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:30:50.765 INFO:teuthology.orchestra.run.smithi094.stdout:osd.6: {} 2024-09-14T06:30:50.765 INFO:teuthology.orchestra.run.smithi094.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-14T06:30:50.770 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-14T06:30:50.781 INFO:teuthology.orchestra.run.smithi094.stdout:osd.7: {} 2024-09-14T06:30:50.782 INFO:teuthology.orchestra.run.smithi094.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-14T06:30:50.818 INFO:tasks.ceph.osd.5.smithi138.stderr:2024-09-14T06:30:50.816+0000 7fe13746f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33618) UID: 0 2024-09-14T06:30:50.887 INFO:teuthology.orchestra.run.smithi094.stdout:ERROR: (22) Invalid argument 2024-09-14T06:30:50.887 INFO:teuthology.orchestra.run.smithi094.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:30:50.896 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-14T06:30:50.918 INFO:tasks.ceph.osd.7.smithi138.stderr:2024-09-14T06:30:50.917+0000 7f3c6a21d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33623) UID: 0 2024-09-14T06:30:50.999 INFO:teuthology.orchestra.run.smithi094.stdout:ERROR: (22) Invalid argument 2024-09-14T06:30:51.000 INFO:teuthology.orchestra.run.smithi094.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:30:51.008 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-14T06:30:51.019 INFO:tasks.ceph.osd.5.smithi138.stderr:2024-09-14T06:30:51.018+0000 7fe13746f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33618) UID: 0 2024-09-14T06:30:51.110 INFO:teuthology.orchestra.run.smithi094.stdout:ERROR: (22) Invalid argument 2024-09-14T06:30:51.110 INFO:teuthology.orchestra.run.smithi094.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:30:51.119 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-14T06:30:51.120 INFO:tasks.ceph.osd.5.smithi138.stderr:2024-09-14T06:30:51.119+0000 7fe13746f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33618) UID: 0 2024-09-14T06:30:51.221 INFO:tasks.ceph.osd.6.smithi138.stderr:2024-09-14T06:30:51.220+0000 7f8cd8849640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33620) UID: 0 2024-09-14T06:30:51.222 INFO:teuthology.orchestra.run.smithi094.stdout:ERROR: (22) Invalid argument 2024-09-14T06:30:51.222 INFO:teuthology.orchestra.run.smithi094.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:30:51.230 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-14T06:30:51.322 INFO:tasks.ceph.osd.0.smithi094.stderr:2024-09-14T06:30:51.321+0000 7f07c9333640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33636) UID: 0 2024-09-14T06:30:51.333 INFO:teuthology.orchestra.run.smithi094.stdout:ERROR: (22) Invalid argument 2024-09-14T06:30:51.333 INFO:teuthology.orchestra.run.smithi094.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:30:51.342 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-14T06:30:51.423 INFO:tasks.ceph.osd.6.smithi138.stderr:2024-09-14T06:30:51.422+0000 7f8cd8849640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33620) UID: 0 2024-09-14T06:30:51.486 INFO:teuthology.orchestra.run.smithi094.stdout:ERROR: (22) Invalid argument 2024-09-14T06:30:51.486 INFO:teuthology.orchestra.run.smithi094.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:30:51.494 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-14T06:30:51.539 INFO:tasks.ceph.osd.2.smithi094.stderr:2024-09-14T06:30:51.538+0000 7faf3269b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33649) UID: 0 2024-09-14T06:30:51.590 INFO:teuthology.orchestra.run.smithi094.stderr:nodeep-scrub is set 2024-09-14T06:30:51.599 INFO:teuthology.orchestra.run.smithi094.stdout:ERROR: (22) Invalid argument 2024-09-14T06:30:51.599 INFO:teuthology.orchestra.run.smithi094.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:30:51.611 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-14T06:30:51.653 INFO:tasks.ceph.osd.0.smithi094.stderr:2024-09-14T06:30:51.652+0000 7f07c9333640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33636) UID: 0 2024-09-14T06:30:51.725 INFO:tasks.ceph.osd.7.smithi138.stderr:2024-09-14T06:30:51.724+0000 7f3c6a21d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33623) UID: 0 2024-09-14T06:30:51.820 INFO:teuthology.orchestra.run.smithi094.stdout:ERROR: (22) Invalid argument 2024-09-14T06:30:51.820 INFO:teuthology.orchestra.run.smithi094.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:30:51.826 INFO:tasks.ceph.osd.0.smithi094.stderr:2024-09-14T06:30:51.825+0000 7f07c9333640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33636) UID: 0 2024-09-14T06:30:51.833 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-14T06:30:51.927 INFO:tasks.ceph.osd.3.smithi094.stderr:2024-09-14T06:30:51.926+0000 7feeeafe8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33655) UID: 0 2024-09-14T06:30:51.980 INFO:teuthology.orchestra.run.smithi138.stdout:ERROR: (22) Invalid argument 2024-09-14T06:30:51.980 INFO:teuthology.orchestra.run.smithi138.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:30:51.994 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-14T06:30:52.028 INFO:tasks.ceph.osd.0.smithi094.stderr:2024-09-14T06:30:52.027+0000 7f07c9333640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33636) UID: 0 2024-09-14T06:30:52.112 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-14T06:30:52.113 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-14T06:30:52.113 INFO:tasks.thrashosds.thrasher:thrash_pg_upmap 2024-09-14T06:30:52.113 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump -f json-pretty 2024-09-14T06:30:52.128 INFO:tasks.ceph.osd.4.smithi138.stderr:2024-09-14T06:30:52.127+0000 7f6c653c8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33615) UID: 0 2024-09-14T06:30:52.143 INFO:teuthology.orchestra.run.smithi138.stdout:ERROR: (22) Invalid argument 2024-09-14T06:30:52.143 INFO:teuthology.orchestra.run.smithi138.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:30:52.153 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-14T06:30:52.237 INFO:tasks.ceph.osd.7.smithi138.stderr:2024-09-14T06:30:52.236+0000 7f3c6a21d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33623) UID: 0 2024-09-14T06:30:52.297 INFO:teuthology.orchestra.run.smithi138.stdout:ERROR: (22) Invalid argument 2024-09-14T06:30:52.297 INFO:teuthology.orchestra.run.smithi138.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:30:52.306 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-14T06:30:52.346 INFO:tasks.ceph.osd.7.smithi138.stderr:2024-09-14T06:30:52.345+0000 7f3c6a21d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33623) UID: 0 2024-09-14T06:30:52.409 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-14T06:30:52.410 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-14T06:30:52.410 INFO:teuthology.orchestra.run.smithi094.stdout: "epoch": 128, 2024-09-14T06:30:52.410 INFO:teuthology.orchestra.run.smithi094.stdout: "fsid": "200ad3da-bd63-4e04-af74-efb5ca0af75e", 2024-09-14T06:30:52.410 INFO:teuthology.orchestra.run.smithi094.stdout: "created": "2024-09-14T06:28:30.782046+0000", 2024-09-14T06:30:52.410 INFO:teuthology.orchestra.run.smithi094.stdout: "modified": "2024-09-14T06:30:51.588445+0000", 2024-09-14T06:30:52.410 INFO:teuthology.orchestra.run.smithi094.stdout: "last_up_change": "2024-09-14T06:28:42.827239+0000", 2024-09-14T06:30:52.410 INFO:teuthology.orchestra.run.smithi094.stdout: "last_in_change": "2024-09-14T06:28:33.724176+0000", 2024-09-14T06:30:52.410 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": "noscrub,nodeep-scrub,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2024-09-14T06:30:52.410 INFO:teuthology.orchestra.run.smithi094.stdout: "flags_num": 5806080, 2024-09-14T06:30:52.410 INFO:teuthology.orchestra.run.smithi094.stdout: "flags_set": [ 2024-09-14T06:30:52.410 INFO:teuthology.orchestra.run.smithi094.stdout: "nodeep-scrub", 2024-09-14T06:30:52.410 INFO:teuthology.orchestra.run.smithi094.stdout: "noscrub", 2024-09-14T06:30:52.410 INFO:teuthology.orchestra.run.smithi094.stdout: "pglog_hardlimit", 2024-09-14T06:30:52.411 INFO:teuthology.orchestra.run.smithi094.stdout: "purged_snapdirs", 2024-09-14T06:30:52.411 INFO:teuthology.orchestra.run.smithi094.stdout: "recovery_deletes", 2024-09-14T06:30:52.411 INFO:teuthology.orchestra.run.smithi094.stdout: "sortbitwise" 2024-09-14T06:30:52.411 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-09-14T06:30:52.411 INFO:teuthology.orchestra.run.smithi094.stdout: "crush_version": 5, 2024-09-14T06:30:52.411 INFO:teuthology.orchestra.run.smithi094.stdout: "full_ratio": 0.94999998807907104, 2024-09-14T06:30:52.411 INFO:teuthology.orchestra.run.smithi094.stdout: "backfillfull_ratio": 0.89999997615814209, 2024-09-14T06:30:52.411 INFO:teuthology.orchestra.run.smithi094.stdout: "nearfull_ratio": 0.85000002384185791, 2024-09-14T06:30:52.411 INFO:teuthology.orchestra.run.smithi094.stdout: "cluster_snapshot": "", 2024-09-14T06:30:52.411 INFO:teuthology.orchestra.run.smithi094.stdout: "pool_max": 3, 2024-09-14T06:30:52.411 INFO:teuthology.orchestra.run.smithi094.stdout: "max_osd": 8, 2024-09-14T06:30:52.411 INFO:teuthology.orchestra.run.smithi094.stdout: "require_min_compat_client": "luminous", 2024-09-14T06:30:52.411 INFO:teuthology.orchestra.run.smithi094.stdout: "min_compat_client": "luminous", 2024-09-14T06:30:52.411 INFO:teuthology.orchestra.run.smithi094.stdout: "require_osd_release": "squid", 2024-09-14T06:30:52.412 INFO:teuthology.orchestra.run.smithi094.stdout: "allow_crimson": false, 2024-09-14T06:30:52.412 INFO:teuthology.orchestra.run.smithi094.stdout: "pools": [ 2024-09-14T06:30:52.412 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:52.412 INFO:teuthology.orchestra.run.smithi094.stdout: "pool": 1, 2024-09-14T06:30:52.412 INFO:teuthology.orchestra.run.smithi094.stdout: "pool_name": ".mgr", 2024-09-14T06:30:52.412 INFO:teuthology.orchestra.run.smithi094.stdout: "create_time": "2024-09-14T06:28:43.295990+0000", 2024-09-14T06:30:52.412 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-14T06:30:52.412 INFO:teuthology.orchestra.run.smithi094.stdout: "flags_names": "hashpspool", 2024-09-14T06:30:52.412 INFO:teuthology.orchestra.run.smithi094.stdout: "type": 1, 2024-09-14T06:30:52.412 INFO:teuthology.orchestra.run.smithi094.stdout: "size": 2, 2024-09-14T06:30:52.412 INFO:teuthology.orchestra.run.smithi094.stdout: "min_size": 1, 2024-09-14T06:30:52.412 INFO:teuthology.orchestra.run.smithi094.stdout: "crush_rule": 0, 2024-09-14T06:30:52.412 INFO:teuthology.orchestra.run.smithi094.stdout: "peering_crush_bucket_count": 0, 2024-09-14T06:30:52.413 INFO:teuthology.orchestra.run.smithi094.stdout: "peering_crush_bucket_target": 0, 2024-09-14T06:30:52.413 INFO:teuthology.orchestra.run.smithi094.stdout: "peering_crush_bucket_barrier": 0, 2024-09-14T06:30:52.413 INFO:teuthology.orchestra.run.smithi094.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-14T06:30:52.413 INFO:teuthology.orchestra.run.smithi094.stdout: "is_stretch_pool": false, 2024-09-14T06:30:52.413 INFO:teuthology.orchestra.run.smithi094.stdout: "object_hash": 2, 2024-09-14T06:30:52.413 INFO:teuthology.orchestra.run.smithi094.stdout: "pg_autoscale_mode": "off", 2024-09-14T06:30:52.413 INFO:teuthology.orchestra.run.smithi094.stdout: "pg_num": 1, 2024-09-14T06:30:52.413 INFO:teuthology.orchestra.run.smithi094.stdout: "pg_placement_num": 1, 2024-09-14T06:30:52.413 INFO:teuthology.orchestra.run.smithi094.stdout: "pg_placement_num_target": 1, 2024-09-14T06:30:52.413 INFO:teuthology.orchestra.run.smithi094.stdout: "pg_num_target": 1, 2024-09-14T06:30:52.413 INFO:teuthology.orchestra.run.smithi094.stdout: "pg_num_pending": 1, 2024-09-14T06:30:52.413 INFO:teuthology.orchestra.run.smithi094.stdout: "last_pg_merge_meta": { 2024-09-14T06:30:52.413 INFO:teuthology.orchestra.run.smithi094.stdout: "source_pgid": "0.0", 2024-09-14T06:30:52.414 INFO:teuthology.orchestra.run.smithi094.stdout: "ready_epoch": 0, 2024-09-14T06:30:52.414 INFO:teuthology.orchestra.run.smithi094.stdout: "last_epoch_started": 0, 2024-09-14T06:30:52.414 INFO:teuthology.orchestra.run.smithi094.stdout: "last_epoch_clean": 0, 2024-09-14T06:30:52.414 INFO:teuthology.orchestra.run.smithi094.stdout: "source_version": "0'0", 2024-09-14T06:30:52.414 INFO:teuthology.orchestra.run.smithi094.stdout: "target_version": "0'0" 2024-09-14T06:30:52.414 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:52.414 INFO:teuthology.orchestra.run.smithi094.stdout: "last_change": "18", 2024-09-14T06:30:52.414 INFO:teuthology.orchestra.run.smithi094.stdout: "last_force_op_resend": "0", 2024-09-14T06:30:52.414 INFO:teuthology.orchestra.run.smithi094.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-14T06:30:52.414 INFO:teuthology.orchestra.run.smithi094.stdout: "last_force_op_resend_preluminous": "0", 2024-09-14T06:30:52.414 INFO:teuthology.orchestra.run.smithi094.stdout: "auid": 0, 2024-09-14T06:30:52.414 INFO:teuthology.orchestra.run.smithi094.stdout: "snap_mode": "selfmanaged", 2024-09-14T06:30:52.414 INFO:teuthology.orchestra.run.smithi094.stdout: "snap_seq": 0, 2024-09-14T06:30:52.414 INFO:teuthology.orchestra.run.smithi094.stdout: "snap_epoch": 0, 2024-09-14T06:30:52.415 INFO:teuthology.orchestra.run.smithi094.stdout: "pool_snaps": [], 2024-09-14T06:30:52.415 INFO:teuthology.orchestra.run.smithi094.stdout: "removed_snaps": "[]", 2024-09-14T06:30:52.415 INFO:teuthology.orchestra.run.smithi094.stdout: "quota_max_bytes": 0, 2024-09-14T06:30:52.415 INFO:teuthology.orchestra.run.smithi094.stdout: "quota_max_objects": 0, 2024-09-14T06:30:52.415 INFO:teuthology.orchestra.run.smithi094.stdout: "tiers": [], 2024-09-14T06:30:52.415 INFO:teuthology.orchestra.run.smithi094.stdout: "tier_of": -1, 2024-09-14T06:30:52.415 INFO:teuthology.orchestra.run.smithi094.stdout: "read_tier": -1, 2024-09-14T06:30:52.415 INFO:teuthology.orchestra.run.smithi094.stdout: "write_tier": -1, 2024-09-14T06:30:52.415 INFO:teuthology.orchestra.run.smithi094.stdout: "cache_mode": "none", 2024-09-14T06:30:52.415 INFO:teuthology.orchestra.run.smithi094.stdout: "target_max_bytes": 0, 2024-09-14T06:30:52.415 INFO:teuthology.orchestra.run.smithi094.stdout: "target_max_objects": 0, 2024-09-14T06:30:52.415 INFO:teuthology.orchestra.run.smithi094.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-14T06:30:52.415 INFO:teuthology.orchestra.run.smithi094.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-14T06:30:52.415 INFO:teuthology.orchestra.run.smithi094.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-14T06:30:52.415 INFO:teuthology.orchestra.run.smithi094.stdout: "cache_min_flush_age": 0, 2024-09-14T06:30:52.416 INFO:teuthology.orchestra.run.smithi094.stdout: "cache_min_evict_age": 0, 2024-09-14T06:30:52.416 INFO:teuthology.orchestra.run.smithi094.stdout: "erasure_code_profile": "", 2024-09-14T06:30:52.416 INFO:teuthology.orchestra.run.smithi094.stdout: "hit_set_params": { 2024-09-14T06:30:52.416 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "none" 2024-09-14T06:30:52.416 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:52.416 INFO:teuthology.orchestra.run.smithi094.stdout: "hit_set_period": 0, 2024-09-14T06:30:52.416 INFO:teuthology.orchestra.run.smithi094.stdout: "hit_set_count": 0, 2024-09-14T06:30:52.416 INFO:teuthology.orchestra.run.smithi094.stdout: "use_gmt_hitset": true, 2024-09-14T06:30:52.416 INFO:teuthology.orchestra.run.smithi094.stdout: "min_read_recency_for_promote": 0, 2024-09-14T06:30:52.416 INFO:teuthology.orchestra.run.smithi094.stdout: "min_write_recency_for_promote": 0, 2024-09-14T06:30:52.416 INFO:teuthology.orchestra.run.smithi094.stdout: "hit_set_grade_decay_rate": 0, 2024-09-14T06:30:52.416 INFO:teuthology.orchestra.run.smithi094.stdout: "hit_set_search_last_n": 0, 2024-09-14T06:30:52.416 INFO:teuthology.orchestra.run.smithi094.stdout: "grade_table": [], 2024-09-14T06:30:52.417 INFO:teuthology.orchestra.run.smithi094.stdout: "stripe_width": 0, 2024-09-14T06:30:52.417 INFO:teuthology.orchestra.run.smithi094.stdout: "expected_num_objects": 0, 2024-09-14T06:30:52.417 INFO:teuthology.orchestra.run.smithi094.stdout: "fast_read": false, 2024-09-14T06:30:52.417 INFO:teuthology.orchestra.run.smithi094.stdout: "options": { 2024-09-14T06:30:52.417 INFO:teuthology.orchestra.run.smithi094.stdout: "pg_num_max": 32, 2024-09-14T06:30:52.417 INFO:teuthology.orchestra.run.smithi094.stdout: "pg_num_min": 1 2024-09-14T06:30:52.417 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:52.417 INFO:teuthology.orchestra.run.smithi094.stdout: "application_metadata": { 2024-09-14T06:30:52.417 INFO:teuthology.orchestra.run.smithi094.stdout: "mgr": {} 2024-09-14T06:30:52.417 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:52.417 INFO:teuthology.orchestra.run.smithi094.stdout: "read_balance": { 2024-09-14T06:30:52.417 INFO:teuthology.orchestra.run.smithi094.stdout: "score_type": "Fair distribution", 2024-09-14T06:30:52.417 INFO:teuthology.orchestra.run.smithi094.stdout: "score_acting": 8, 2024-09-14T06:30:52.417 INFO:teuthology.orchestra.run.smithi094.stdout: "score_stable": 8, 2024-09-14T06:30:52.418 INFO:teuthology.orchestra.run.smithi094.stdout: "optimal_score": 0.25, 2024-09-14T06:30:52.418 INFO:teuthology.orchestra.run.smithi094.stdout: "raw_score_acting": 2, 2024-09-14T06:30:52.418 INFO:teuthology.orchestra.run.smithi094.stdout: "raw_score_stable": 2, 2024-09-14T06:30:52.418 INFO:teuthology.orchestra.run.smithi094.stdout: "primary_affinity_weighted": 1, 2024-09-14T06:30:52.418 INFO:teuthology.orchestra.run.smithi094.stdout: "average_primary_affinity": 1, 2024-09-14T06:30:52.418 INFO:teuthology.orchestra.run.smithi094.stdout: "average_primary_affinity_weighted": 1 2024-09-14T06:30:52.418 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:52.418 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:52.418 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:52.418 INFO:teuthology.orchestra.run.smithi094.stdout: "pool": 2, 2024-09-14T06:30:52.418 INFO:teuthology.orchestra.run.smithi094.stdout: "pool_name": "rbd", 2024-09-14T06:30:52.419 INFO:teuthology.orchestra.run.smithi094.stdout: "create_time": "2024-09-14T06:28:49.451714+0000", 2024-09-14T06:30:52.419 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 8193, 2024-09-14T06:30:52.419 INFO:teuthology.orchestra.run.smithi094.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-09-14T06:30:52.419 INFO:teuthology.orchestra.run.smithi094.stdout: "type": 1, 2024-09-14T06:30:52.419 INFO:teuthology.orchestra.run.smithi094.stdout: "size": 2, 2024-09-14T06:30:52.419 INFO:teuthology.orchestra.run.smithi094.stdout: "min_size": 1, 2024-09-14T06:30:52.419 INFO:teuthology.orchestra.run.smithi094.stdout: "crush_rule": 0, 2024-09-14T06:30:52.419 INFO:teuthology.orchestra.run.smithi094.stdout: "peering_crush_bucket_count": 0, 2024-09-14T06:30:52.419 INFO:teuthology.orchestra.run.smithi094.stdout: "peering_crush_bucket_target": 0, 2024-09-14T06:30:52.419 INFO:teuthology.orchestra.run.smithi094.stdout: "peering_crush_bucket_barrier": 0, 2024-09-14T06:30:52.419 INFO:teuthology.orchestra.run.smithi094.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-14T06:30:52.419 INFO:teuthology.orchestra.run.smithi094.stdout: "is_stretch_pool": false, 2024-09-14T06:30:52.420 INFO:teuthology.orchestra.run.smithi094.stdout: "object_hash": 2, 2024-09-14T06:30:52.420 INFO:teuthology.orchestra.run.smithi094.stdout: "pg_autoscale_mode": "off", 2024-09-14T06:30:52.420 INFO:teuthology.orchestra.run.smithi094.stdout: "pg_num": 8, 2024-09-14T06:30:52.420 INFO:teuthology.orchestra.run.smithi094.stdout: "pg_placement_num": 8, 2024-09-14T06:30:52.420 INFO:teuthology.orchestra.run.smithi094.stdout: "pg_placement_num_target": 8, 2024-09-14T06:30:52.420 INFO:teuthology.orchestra.run.smithi094.stdout: "pg_num_target": 8, 2024-09-14T06:30:52.420 INFO:teuthology.orchestra.run.smithi094.stdout: "pg_num_pending": 8, 2024-09-14T06:30:52.420 INFO:teuthology.orchestra.run.smithi094.stdout: "last_pg_merge_meta": { 2024-09-14T06:30:52.420 INFO:teuthology.orchestra.run.smithi094.stdout: "source_pgid": "0.0", 2024-09-14T06:30:52.420 INFO:teuthology.orchestra.run.smithi094.stdout: "ready_epoch": 0, 2024-09-14T06:30:52.420 INFO:teuthology.orchestra.run.smithi094.stdout: "last_epoch_started": 0, 2024-09-14T06:30:52.420 INFO:teuthology.orchestra.run.smithi094.stdout: "last_epoch_clean": 0, 2024-09-14T06:30:52.420 INFO:teuthology.orchestra.run.smithi094.stdout: "source_version": "0'0", 2024-09-14T06:30:52.421 INFO:teuthology.orchestra.run.smithi094.stdout: "target_version": "0'0" 2024-09-14T06:30:52.421 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:52.421 INFO:teuthology.orchestra.run.smithi094.stdout: "last_change": "22", 2024-09-14T06:30:52.421 INFO:teuthology.orchestra.run.smithi094.stdout: "last_force_op_resend": "0", 2024-09-14T06:30:52.421 INFO:teuthology.orchestra.run.smithi094.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-14T06:30:52.421 INFO:teuthology.orchestra.run.smithi094.stdout: "last_force_op_resend_preluminous": "0", 2024-09-14T06:30:52.421 INFO:teuthology.orchestra.run.smithi094.stdout: "auid": 0, 2024-09-14T06:30:52.421 INFO:teuthology.orchestra.run.smithi094.stdout: "snap_mode": "selfmanaged", 2024-09-14T06:30:52.421 INFO:teuthology.orchestra.run.smithi094.stdout: "snap_seq": 2, 2024-09-14T06:30:52.421 INFO:teuthology.orchestra.run.smithi094.stdout: "snap_epoch": 22, 2024-09-14T06:30:52.421 INFO:teuthology.orchestra.run.smithi094.stdout: "pool_snaps": [], 2024-09-14T06:30:52.421 INFO:teuthology.orchestra.run.smithi094.stdout: "removed_snaps": "[]", 2024-09-14T06:30:52.421 INFO:teuthology.orchestra.run.smithi094.stdout: "quota_max_bytes": 0, 2024-09-14T06:30:52.421 INFO:teuthology.orchestra.run.smithi094.stdout: "quota_max_objects": 0, 2024-09-14T06:30:52.422 INFO:teuthology.orchestra.run.smithi094.stdout: "tiers": [], 2024-09-14T06:30:52.422 INFO:teuthology.orchestra.run.smithi094.stdout: "tier_of": -1, 2024-09-14T06:30:52.422 INFO:teuthology.orchestra.run.smithi094.stdout: "read_tier": -1, 2024-09-14T06:30:52.422 INFO:teuthology.orchestra.run.smithi094.stdout: "write_tier": -1, 2024-09-14T06:30:52.422 INFO:teuthology.orchestra.run.smithi094.stdout: "cache_mode": "none", 2024-09-14T06:30:52.422 INFO:teuthology.orchestra.run.smithi094.stdout: "target_max_bytes": 0, 2024-09-14T06:30:52.422 INFO:teuthology.orchestra.run.smithi094.stdout: "target_max_objects": 0, 2024-09-14T06:30:52.422 INFO:teuthology.orchestra.run.smithi094.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-14T06:30:52.422 INFO:teuthology.orchestra.run.smithi094.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-14T06:30:52.422 INFO:teuthology.orchestra.run.smithi094.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-14T06:30:52.422 INFO:teuthology.orchestra.run.smithi094.stdout: "cache_min_flush_age": 0, 2024-09-14T06:30:52.422 INFO:teuthology.orchestra.run.smithi094.stdout: "cache_min_evict_age": 0, 2024-09-14T06:30:52.422 INFO:teuthology.orchestra.run.smithi094.stdout: "erasure_code_profile": "", 2024-09-14T06:30:52.422 INFO:teuthology.orchestra.run.smithi094.stdout: "hit_set_params": { 2024-09-14T06:30:52.422 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "none" 2024-09-14T06:30:52.423 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:52.423 INFO:teuthology.orchestra.run.smithi094.stdout: "hit_set_period": 0, 2024-09-14T06:30:52.423 INFO:teuthology.orchestra.run.smithi094.stdout: "hit_set_count": 0, 2024-09-14T06:30:52.423 INFO:teuthology.orchestra.run.smithi094.stdout: "use_gmt_hitset": true, 2024-09-14T06:30:52.423 INFO:teuthology.orchestra.run.smithi094.stdout: "min_read_recency_for_promote": 0, 2024-09-14T06:30:52.423 INFO:teuthology.orchestra.run.smithi094.stdout: "min_write_recency_for_promote": 0, 2024-09-14T06:30:52.423 INFO:teuthology.orchestra.run.smithi094.stdout: "hit_set_grade_decay_rate": 0, 2024-09-14T06:30:52.423 INFO:teuthology.orchestra.run.smithi094.stdout: "hit_set_search_last_n": 0, 2024-09-14T06:30:52.423 INFO:teuthology.orchestra.run.smithi094.stdout: "grade_table": [], 2024-09-14T06:30:52.423 INFO:teuthology.orchestra.run.smithi094.stdout: "stripe_width": 0, 2024-09-14T06:30:52.423 INFO:teuthology.orchestra.run.smithi094.stdout: "expected_num_objects": 0, 2024-09-14T06:30:52.423 INFO:teuthology.orchestra.run.smithi094.stdout: "fast_read": false, 2024-09-14T06:30:52.423 INFO:teuthology.orchestra.run.smithi094.stdout: "options": {}, 2024-09-14T06:30:52.423 INFO:teuthology.orchestra.run.smithi094.stdout: "application_metadata": { 2024-09-14T06:30:52.424 INFO:teuthology.orchestra.run.smithi094.stdout: "rbd": {} 2024-09-14T06:30:52.424 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:52.424 INFO:teuthology.orchestra.run.smithi094.stdout: "read_balance": { 2024-09-14T06:30:52.424 INFO:teuthology.orchestra.run.smithi094.stdout: "score_type": "Fair distribution", 2024-09-14T06:30:52.424 INFO:teuthology.orchestra.run.smithi094.stdout: "score_acting": 1.9900000095367432, 2024-09-14T06:30:52.424 INFO:teuthology.orchestra.run.smithi094.stdout: "score_stable": 1.9900000095367432, 2024-09-14T06:30:52.424 INFO:teuthology.orchestra.run.smithi094.stdout: "optimal_score": 0.87999999523162842, 2024-09-14T06:30:52.424 INFO:teuthology.orchestra.run.smithi094.stdout: "raw_score_acting": 1.75, 2024-09-14T06:30:52.424 INFO:teuthology.orchestra.run.smithi094.stdout: "raw_score_stable": 1.75, 2024-09-14T06:30:52.424 INFO:teuthology.orchestra.run.smithi094.stdout: "primary_affinity_weighted": 1, 2024-09-14T06:30:52.424 INFO:teuthology.orchestra.run.smithi094.stdout: "average_primary_affinity": 1, 2024-09-14T06:30:52.424 INFO:teuthology.orchestra.run.smithi094.stdout: "average_primary_affinity_weighted": 1 2024-09-14T06:30:52.424 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:52.424 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:52.425 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:52.425 INFO:teuthology.orchestra.run.smithi094.stdout: "pool": 3, 2024-09-14T06:30:52.425 INFO:teuthology.orchestra.run.smithi094.stdout: "pool_name": "pool_client.0", 2024-09-14T06:30:52.425 INFO:teuthology.orchestra.run.smithi094.stdout: "create_time": "2024-09-14T06:28:59.688535+0000", 2024-09-14T06:30:52.425 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 8193, 2024-09-14T06:30:52.425 INFO:teuthology.orchestra.run.smithi094.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-09-14T06:30:52.425 INFO:teuthology.orchestra.run.smithi094.stdout: "type": 1, 2024-09-14T06:30:52.425 INFO:teuthology.orchestra.run.smithi094.stdout: "size": 2, 2024-09-14T06:30:52.425 INFO:teuthology.orchestra.run.smithi094.stdout: "min_size": 1, 2024-09-14T06:30:52.425 INFO:teuthology.orchestra.run.smithi094.stdout: "crush_rule": 0, 2024-09-14T06:30:52.425 INFO:teuthology.orchestra.run.smithi094.stdout: "peering_crush_bucket_count": 0, 2024-09-14T06:30:52.425 INFO:teuthology.orchestra.run.smithi094.stdout: "peering_crush_bucket_target": 0, 2024-09-14T06:30:52.425 INFO:teuthology.orchestra.run.smithi094.stdout: "peering_crush_bucket_barrier": 0, 2024-09-14T06:30:52.426 INFO:teuthology.orchestra.run.smithi094.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-14T06:30:52.426 INFO:teuthology.orchestra.run.smithi094.stdout: "is_stretch_pool": false, 2024-09-14T06:30:52.426 INFO:teuthology.orchestra.run.smithi094.stdout: "object_hash": 2, 2024-09-14T06:30:52.426 INFO:teuthology.orchestra.run.smithi094.stdout: "pg_autoscale_mode": "off", 2024-09-14T06:30:52.426 INFO:teuthology.orchestra.run.smithi094.stdout: "pg_num": 32, 2024-09-14T06:30:52.426 INFO:teuthology.orchestra.run.smithi094.stdout: "pg_placement_num": 32, 2024-09-14T06:30:52.426 INFO:teuthology.orchestra.run.smithi094.stdout: "pg_placement_num_target": 32, 2024-09-14T06:30:52.426 INFO:teuthology.orchestra.run.smithi094.stdout: "pg_num_target": 32, 2024-09-14T06:30:52.426 INFO:teuthology.orchestra.run.smithi094.stdout: "pg_num_pending": 32, 2024-09-14T06:30:52.426 INFO:teuthology.orchestra.run.smithi094.stdout: "last_pg_merge_meta": { 2024-09-14T06:30:52.426 INFO:teuthology.orchestra.run.smithi094.stdout: "source_pgid": "0.0", 2024-09-14T06:30:52.426 INFO:teuthology.orchestra.run.smithi094.stdout: "ready_epoch": 0, 2024-09-14T06:30:52.426 INFO:teuthology.orchestra.run.smithi094.stdout: "last_epoch_started": 0, 2024-09-14T06:30:52.426 INFO:teuthology.orchestra.run.smithi094.stdout: "last_epoch_clean": 0, 2024-09-14T06:30:52.427 INFO:teuthology.orchestra.run.smithi094.stdout: "source_version": "0'0", 2024-09-14T06:30:52.427 INFO:teuthology.orchestra.run.smithi094.stdout: "target_version": "0'0" 2024-09-14T06:30:52.427 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:52.427 INFO:teuthology.orchestra.run.smithi094.stdout: "last_change": "128", 2024-09-14T06:30:52.427 INFO:teuthology.orchestra.run.smithi094.stdout: "last_force_op_resend": "0", 2024-09-14T06:30:52.427 INFO:teuthology.orchestra.run.smithi094.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-14T06:30:52.427 INFO:teuthology.orchestra.run.smithi094.stdout: "last_force_op_resend_preluminous": "0", 2024-09-14T06:30:52.427 INFO:teuthology.orchestra.run.smithi094.stdout: "auid": 0, 2024-09-14T06:30:52.427 INFO:teuthology.orchestra.run.smithi094.stdout: "snap_mode": "selfmanaged", 2024-09-14T06:30:52.427 INFO:teuthology.orchestra.run.smithi094.stdout: "snap_seq": 60, 2024-09-14T06:30:52.427 INFO:teuthology.orchestra.run.smithi094.stdout: "snap_epoch": 128, 2024-09-14T06:30:52.427 INFO:teuthology.orchestra.run.smithi094.stdout: "pool_snaps": [], 2024-09-14T06:30:52.427 INFO:teuthology.orchestra.run.smithi094.stdout: "removed_snaps": "[]", 2024-09-14T06:30:52.427 INFO:teuthology.orchestra.run.smithi094.stdout: "quota_max_bytes": 0, 2024-09-14T06:30:52.428 INFO:teuthology.orchestra.run.smithi094.stdout: "quota_max_objects": 0, 2024-09-14T06:30:52.428 INFO:teuthology.orchestra.run.smithi094.stdout: "tiers": [], 2024-09-14T06:30:52.428 INFO:teuthology.orchestra.run.smithi094.stdout: "tier_of": -1, 2024-09-14T06:30:52.428 INFO:teuthology.orchestra.run.smithi094.stdout: "read_tier": -1, 2024-09-14T06:30:52.428 INFO:teuthology.orchestra.run.smithi094.stdout: "write_tier": -1, 2024-09-14T06:30:52.428 INFO:teuthology.orchestra.run.smithi094.stdout: "cache_mode": "none", 2024-09-14T06:30:52.428 INFO:teuthology.orchestra.run.smithi094.stdout: "target_max_bytes": 0, 2024-09-14T06:30:52.428 INFO:teuthology.orchestra.run.smithi094.stdout: "target_max_objects": 0, 2024-09-14T06:30:52.428 INFO:teuthology.orchestra.run.smithi094.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-14T06:30:52.428 INFO:teuthology.orchestra.run.smithi094.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-14T06:30:52.428 INFO:teuthology.orchestra.run.smithi094.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-14T06:30:52.428 INFO:teuthology.orchestra.run.smithi094.stdout: "cache_min_flush_age": 0, 2024-09-14T06:30:52.428 INFO:teuthology.orchestra.run.smithi094.stdout: "cache_min_evict_age": 0, 2024-09-14T06:30:52.428 INFO:teuthology.orchestra.run.smithi094.stdout: "erasure_code_profile": "", 2024-09-14T06:30:52.429 INFO:teuthology.orchestra.run.smithi094.stdout: "hit_set_params": { 2024-09-14T06:30:52.429 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "none" 2024-09-14T06:30:52.429 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:52.429 INFO:teuthology.orchestra.run.smithi094.stdout: "hit_set_period": 0, 2024-09-14T06:30:52.429 INFO:teuthology.orchestra.run.smithi094.stdout: "hit_set_count": 0, 2024-09-14T06:30:52.429 INFO:teuthology.orchestra.run.smithi094.stdout: "use_gmt_hitset": true, 2024-09-14T06:30:52.429 INFO:teuthology.orchestra.run.smithi094.stdout: "min_read_recency_for_promote": 0, 2024-09-14T06:30:52.429 INFO:teuthology.orchestra.run.smithi094.stdout: "min_write_recency_for_promote": 0, 2024-09-14T06:30:52.429 INFO:teuthology.orchestra.run.smithi094.stdout: "hit_set_grade_decay_rate": 0, 2024-09-14T06:30:52.429 INFO:teuthology.orchestra.run.smithi094.stdout: "hit_set_search_last_n": 0, 2024-09-14T06:30:52.429 INFO:teuthology.orchestra.run.smithi094.stdout: "grade_table": [], 2024-09-14T06:30:52.429 INFO:teuthology.orchestra.run.smithi094.stdout: "stripe_width": 0, 2024-09-14T06:30:52.429 INFO:teuthology.orchestra.run.smithi094.stdout: "expected_num_objects": 0, 2024-09-14T06:30:52.430 INFO:teuthology.orchestra.run.smithi094.stdout: "fast_read": false, 2024-09-14T06:30:52.430 INFO:teuthology.orchestra.run.smithi094.stdout: "options": {}, 2024-09-14T06:30:52.430 INFO:teuthology.orchestra.run.smithi094.stdout: "application_metadata": { 2024-09-14T06:30:52.430 INFO:teuthology.orchestra.run.smithi094.stdout: "rbd": {} 2024-09-14T06:30:52.430 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:52.430 INFO:teuthology.orchestra.run.smithi094.stdout: "read_balance": { 2024-09-14T06:30:52.430 INFO:teuthology.orchestra.run.smithi094.stdout: "score_type": "Fair distribution", 2024-09-14T06:30:52.430 INFO:teuthology.orchestra.run.smithi094.stdout: "score_acting": 1.5, 2024-09-14T06:30:52.430 INFO:teuthology.orchestra.run.smithi094.stdout: "score_stable": 1.5, 2024-09-14T06:30:52.430 INFO:teuthology.orchestra.run.smithi094.stdout: "optimal_score": 1, 2024-09-14T06:30:52.430 INFO:teuthology.orchestra.run.smithi094.stdout: "raw_score_acting": 1.5, 2024-09-14T06:30:52.430 INFO:teuthology.orchestra.run.smithi094.stdout: "raw_score_stable": 1.5, 2024-09-14T06:30:52.430 INFO:teuthology.orchestra.run.smithi094.stdout: "primary_affinity_weighted": 1, 2024-09-14T06:30:52.430 INFO:teuthology.orchestra.run.smithi094.stdout: "average_primary_affinity": 1, 2024-09-14T06:30:52.430 INFO:teuthology.orchestra.run.smithi094.stdout: "average_primary_affinity_weighted": 1 2024-09-14T06:30:52.431 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:52.431 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:52.431 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-09-14T06:30:52.431 INFO:teuthology.orchestra.run.smithi094.stdout: "osds": [ 2024-09-14T06:30:52.431 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:52.431 INFO:teuthology.orchestra.run.smithi094.stdout: "osd": 0, 2024-09-14T06:30:52.431 INFO:teuthology.orchestra.run.smithi094.stdout: "uuid": "8b22a249-b842-406a-a02c-c0f31f57b635", 2024-09-14T06:30:52.431 INFO:teuthology.orchestra.run.smithi094.stdout: "up": 1, 2024-09-14T06:30:52.431 INFO:teuthology.orchestra.run.smithi094.stdout: "in": 1, 2024-09-14T06:30:52.431 INFO:teuthology.orchestra.run.smithi094.stdout: "weight": 1, 2024-09-14T06:30:52.431 INFO:teuthology.orchestra.run.smithi094.stdout: "primary_affinity": 1, 2024-09-14T06:30:52.431 INFO:teuthology.orchestra.run.smithi094.stdout: "last_clean_begin": 0, 2024-09-14T06:30:52.431 INFO:teuthology.orchestra.run.smithi094.stdout: "last_clean_end": 0, 2024-09-14T06:30:52.432 INFO:teuthology.orchestra.run.smithi094.stdout: "up_from": 14, 2024-09-14T06:30:52.432 INFO:teuthology.orchestra.run.smithi094.stdout: "up_thru": 75, 2024-09-14T06:30:52.432 INFO:teuthology.orchestra.run.smithi094.stdout: "down_at": 0, 2024-09-14T06:30:52.432 INFO:teuthology.orchestra.run.smithi094.stdout: "lost_at": 0, 2024-09-14T06:30:52.432 INFO:teuthology.orchestra.run.smithi094.stdout: "public_addrs": { 2024-09-14T06:30:52.432 INFO:teuthology.orchestra.run.smithi094.stdout: "addrvec": [ 2024-09-14T06:30:52.432 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:52.432 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v2", 2024-09-14T06:30:52.432 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.94:6824", 2024-09-14T06:30:52.432 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 1809564923 2024-09-14T06:30:52.432 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:52.432 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:52.432 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v1", 2024-09-14T06:30:52.432 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.94:6825", 2024-09-14T06:30:52.433 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 1809564923 2024-09-14T06:30:52.433 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:52.433 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:52.433 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:52.433 INFO:teuthology.orchestra.run.smithi094.stdout: "cluster_addrs": { 2024-09-14T06:30:52.433 INFO:teuthology.orchestra.run.smithi094.stdout: "addrvec": [ 2024-09-14T06:30:52.433 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:52.433 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v2", 2024-09-14T06:30:52.433 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.94:6826", 2024-09-14T06:30:52.433 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 1809564923 2024-09-14T06:30:52.433 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:52.433 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:52.433 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v1", 2024-09-14T06:30:52.433 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.94:6827", 2024-09-14T06:30:52.434 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 1809564923 2024-09-14T06:30:52.434 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:52.434 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:52.434 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:52.434 INFO:teuthology.orchestra.run.smithi094.stdout: "heartbeat_back_addrs": { 2024-09-14T06:30:52.434 INFO:teuthology.orchestra.run.smithi094.stdout: "addrvec": [ 2024-09-14T06:30:52.434 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:52.434 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v2", 2024-09-14T06:30:52.434 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.94:6830", 2024-09-14T06:30:52.434 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 1809564923 2024-09-14T06:30:52.434 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:52.434 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:52.434 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v1", 2024-09-14T06:30:52.434 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.94:6831", 2024-09-14T06:30:52.434 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 1809564923 2024-09-14T06:30:52.435 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:52.435 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:52.435 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:52.435 INFO:teuthology.orchestra.run.smithi094.stdout: "heartbeat_front_addrs": { 2024-09-14T06:30:52.435 INFO:teuthology.orchestra.run.smithi094.stdout: "addrvec": [ 2024-09-14T06:30:52.435 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:52.435 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v2", 2024-09-14T06:30:52.435 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.94:6828", 2024-09-14T06:30:52.435 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 1809564923 2024-09-14T06:30:52.435 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:52.435 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:52.435 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v1", 2024-09-14T06:30:52.435 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.94:6829", 2024-09-14T06:30:52.435 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 1809564923 2024-09-14T06:30:52.436 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:52.436 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:52.436 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:52.436 INFO:teuthology.orchestra.run.smithi094.stdout: "public_addr": "172.21.15.94:6825/1809564923", 2024-09-14T06:30:52.436 INFO:teuthology.orchestra.run.smithi094.stdout: "cluster_addr": "172.21.15.94:6827/1809564923", 2024-09-14T06:30:52.436 INFO:teuthology.orchestra.run.smithi094.stdout: "heartbeat_back_addr": "172.21.15.94:6831/1809564923", 2024-09-14T06:30:52.436 INFO:teuthology.orchestra.run.smithi094.stdout: "heartbeat_front_addr": "172.21.15.94:6829/1809564923", 2024-09-14T06:30:52.436 INFO:teuthology.orchestra.run.smithi094.stdout: "state": [ 2024-09-14T06:30:52.436 INFO:teuthology.orchestra.run.smithi094.stdout: "exists", 2024-09-14T06:30:52.436 INFO:teuthology.orchestra.run.smithi094.stdout: "up" 2024-09-14T06:30:52.436 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:52.436 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:52.436 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:52.436 INFO:teuthology.orchestra.run.smithi094.stdout: "osd": 1, 2024-09-14T06:30:52.437 INFO:teuthology.orchestra.run.smithi094.stdout: "uuid": "02251a03-84a0-414d-a980-fac4fd043cb6", 2024-09-14T06:30:52.437 INFO:teuthology.orchestra.run.smithi094.stdout: "up": 1, 2024-09-14T06:30:52.437 INFO:teuthology.orchestra.run.smithi094.stdout: "in": 1, 2024-09-14T06:30:52.437 INFO:teuthology.orchestra.run.smithi094.stdout: "weight": 1, 2024-09-14T06:30:52.437 INFO:teuthology.orchestra.run.smithi094.stdout: "primary_affinity": 1, 2024-09-14T06:30:52.437 INFO:teuthology.orchestra.run.smithi094.stdout: "last_clean_begin": 0, 2024-09-14T06:30:52.437 INFO:teuthology.orchestra.run.smithi094.stdout: "last_clean_end": 0, 2024-09-14T06:30:52.437 INFO:teuthology.orchestra.run.smithi094.stdout: "up_from": 15, 2024-09-14T06:30:52.437 INFO:teuthology.orchestra.run.smithi094.stdout: "up_thru": 23, 2024-09-14T06:30:52.437 INFO:teuthology.orchestra.run.smithi094.stdout: "down_at": 0, 2024-09-14T06:30:52.437 INFO:teuthology.orchestra.run.smithi094.stdout: "lost_at": 0, 2024-09-14T06:30:52.437 INFO:teuthology.orchestra.run.smithi094.stdout: "public_addrs": { 2024-09-14T06:30:52.437 INFO:teuthology.orchestra.run.smithi094.stdout: "addrvec": [ 2024-09-14T06:30:52.438 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:52.438 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v2", 2024-09-14T06:30:52.438 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.94:6816", 2024-09-14T06:30:52.438 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 736402652 2024-09-14T06:30:52.438 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:52.438 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:52.438 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v1", 2024-09-14T06:30:52.438 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.94:6817", 2024-09-14T06:30:52.438 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 736402652 2024-09-14T06:30:52.438 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:52.438 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:52.438 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:52.438 INFO:teuthology.orchestra.run.smithi094.stdout: "cluster_addrs": { 2024-09-14T06:30:52.438 INFO:teuthology.orchestra.run.smithi094.stdout: "addrvec": [ 2024-09-14T06:30:52.439 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:52.439 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v2", 2024-09-14T06:30:52.439 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.94:6818", 2024-09-14T06:30:52.439 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 736402652 2024-09-14T06:30:52.439 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:52.439 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:52.439 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v1", 2024-09-14T06:30:52.439 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.94:6819", 2024-09-14T06:30:52.439 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 736402652 2024-09-14T06:30:52.439 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:52.439 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:52.440 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:52.440 INFO:teuthology.orchestra.run.smithi094.stdout: "heartbeat_back_addrs": { 2024-09-14T06:30:52.440 INFO:teuthology.orchestra.run.smithi094.stdout: "addrvec": [ 2024-09-14T06:30:52.440 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:52.440 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v2", 2024-09-14T06:30:52.440 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.94:6822", 2024-09-14T06:30:52.440 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 736402652 2024-09-14T06:30:52.440 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:52.440 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:52.440 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v1", 2024-09-14T06:30:52.440 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.94:6823", 2024-09-14T06:30:52.440 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 736402652 2024-09-14T06:30:52.440 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:52.440 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:52.441 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:52.441 INFO:teuthology.orchestra.run.smithi094.stdout: "heartbeat_front_addrs": { 2024-09-14T06:30:52.441 INFO:teuthology.orchestra.run.smithi094.stdout: "addrvec": [ 2024-09-14T06:30:52.441 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:52.441 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v2", 2024-09-14T06:30:52.441 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.94:6820", 2024-09-14T06:30:52.441 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 736402652 2024-09-14T06:30:52.441 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:52.441 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:52.441 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v1", 2024-09-14T06:30:52.441 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.94:6821", 2024-09-14T06:30:52.441 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 736402652 2024-09-14T06:30:52.441 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:52.441 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:52.442 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:52.442 INFO:teuthology.orchestra.run.smithi094.stdout: "public_addr": "172.21.15.94:6817/736402652", 2024-09-14T06:30:52.442 INFO:teuthology.orchestra.run.smithi094.stdout: "cluster_addr": "172.21.15.94:6819/736402652", 2024-09-14T06:30:52.442 INFO:teuthology.orchestra.run.smithi094.stdout: "heartbeat_back_addr": "172.21.15.94:6823/736402652", 2024-09-14T06:30:52.442 INFO:teuthology.orchestra.run.smithi094.stdout: "heartbeat_front_addr": "172.21.15.94:6821/736402652", 2024-09-14T06:30:52.442 INFO:teuthology.orchestra.run.smithi094.stdout: "state": [ 2024-09-14T06:30:52.442 INFO:teuthology.orchestra.run.smithi094.stdout: "exists", 2024-09-14T06:30:52.442 INFO:teuthology.orchestra.run.smithi094.stdout: "up" 2024-09-14T06:30:52.442 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:52.442 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:52.442 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:52.442 INFO:teuthology.orchestra.run.smithi094.stdout: "osd": 2, 2024-09-14T06:30:52.442 INFO:teuthology.orchestra.run.smithi094.stdout: "uuid": "d45bf470-bb29-4739-8294-cd122b2b4102", 2024-09-14T06:30:52.443 INFO:teuthology.orchestra.run.smithi094.stdout: "up": 1, 2024-09-14T06:30:52.443 INFO:teuthology.orchestra.run.smithi094.stdout: "in": 1, 2024-09-14T06:30:52.443 INFO:teuthology.orchestra.run.smithi094.stdout: "weight": 1, 2024-09-14T06:30:52.443 INFO:teuthology.orchestra.run.smithi094.stdout: "primary_affinity": 1, 2024-09-14T06:30:52.443 INFO:teuthology.orchestra.run.smithi094.stdout: "last_clean_begin": 0, 2024-09-14T06:30:52.443 INFO:teuthology.orchestra.run.smithi094.stdout: "last_clean_end": 0, 2024-09-14T06:30:52.443 INFO:teuthology.orchestra.run.smithi094.stdout: "up_from": 14, 2024-09-14T06:30:52.443 INFO:teuthology.orchestra.run.smithi094.stdout: "up_thru": 19, 2024-09-14T06:30:52.443 INFO:teuthology.orchestra.run.smithi094.stdout: "down_at": 0, 2024-09-14T06:30:52.443 INFO:teuthology.orchestra.run.smithi094.stdout: "lost_at": 0, 2024-09-14T06:30:52.443 INFO:teuthology.orchestra.run.smithi094.stdout: "public_addrs": { 2024-09-14T06:30:52.443 INFO:teuthology.orchestra.run.smithi094.stdout: "addrvec": [ 2024-09-14T06:30:52.443 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:52.443 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v2", 2024-09-14T06:30:52.443 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.94:6808", 2024-09-14T06:30:52.444 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 1063204782 2024-09-14T06:30:52.444 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:52.444 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:52.444 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v1", 2024-09-14T06:30:52.444 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.94:6809", 2024-09-14T06:30:52.444 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 1063204782 2024-09-14T06:30:52.444 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:52.444 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:52.444 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:52.444 INFO:teuthology.orchestra.run.smithi094.stdout: "cluster_addrs": { 2024-09-14T06:30:52.444 INFO:teuthology.orchestra.run.smithi094.stdout: "addrvec": [ 2024-09-14T06:30:52.444 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:52.444 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v2", 2024-09-14T06:30:52.444 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.94:6810", 2024-09-14T06:30:52.445 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 1063204782 2024-09-14T06:30:52.445 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:52.445 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:52.445 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v1", 2024-09-14T06:30:52.445 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.94:6811", 2024-09-14T06:30:52.445 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 1063204782 2024-09-14T06:30:52.445 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:52.445 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:52.445 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:52.445 INFO:teuthology.orchestra.run.smithi094.stdout: "heartbeat_back_addrs": { 2024-09-14T06:30:52.445 INFO:teuthology.orchestra.run.smithi094.stdout: "addrvec": [ 2024-09-14T06:30:52.445 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:52.445 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v2", 2024-09-14T06:30:52.446 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.94:6814", 2024-09-14T06:30:52.446 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 1063204782 2024-09-14T06:30:52.446 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:52.446 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:52.446 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v1", 2024-09-14T06:30:52.446 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.94:6815", 2024-09-14T06:30:52.446 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 1063204782 2024-09-14T06:30:52.446 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:52.446 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:52.446 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:52.446 INFO:teuthology.orchestra.run.smithi094.stdout: "heartbeat_front_addrs": { 2024-09-14T06:30:52.446 INFO:teuthology.orchestra.run.smithi094.stdout: "addrvec": [ 2024-09-14T06:30:52.446 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:52.446 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v2", 2024-09-14T06:30:52.447 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.94:6812", 2024-09-14T06:30:52.447 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 1063204782 2024-09-14T06:30:52.447 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:52.447 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:52.447 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v1", 2024-09-14T06:30:52.447 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.94:6813", 2024-09-14T06:30:52.447 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 1063204782 2024-09-14T06:30:52.447 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:52.447 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:52.447 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:52.447 INFO:teuthology.orchestra.run.smithi094.stdout: "public_addr": "172.21.15.94:6809/1063204782", 2024-09-14T06:30:52.447 INFO:teuthology.orchestra.run.smithi094.stdout: "cluster_addr": "172.21.15.94:6811/1063204782", 2024-09-14T06:30:52.447 INFO:teuthology.orchestra.run.smithi094.stdout: "heartbeat_back_addr": "172.21.15.94:6815/1063204782", 2024-09-14T06:30:52.447 INFO:teuthology.orchestra.run.smithi094.stdout: "heartbeat_front_addr": "172.21.15.94:6813/1063204782", 2024-09-14T06:30:52.448 INFO:teuthology.orchestra.run.smithi094.stdout: "state": [ 2024-09-14T06:30:52.448 INFO:teuthology.orchestra.run.smithi094.stdout: "exists", 2024-09-14T06:30:52.448 INFO:teuthology.orchestra.run.smithi094.stdout: "up" 2024-09-14T06:30:52.448 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:52.448 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:52.448 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:52.448 INFO:teuthology.orchestra.run.smithi094.stdout: "osd": 3, 2024-09-14T06:30:52.448 INFO:teuthology.orchestra.run.smithi094.stdout: "uuid": "456ec2d9-c438-4243-b931-335704f661eb", 2024-09-14T06:30:52.448 INFO:teuthology.orchestra.run.smithi094.stdout: "up": 1, 2024-09-14T06:30:52.448 INFO:teuthology.orchestra.run.smithi094.stdout: "in": 1, 2024-09-14T06:30:52.448 INFO:teuthology.orchestra.run.smithi094.stdout: "weight": 1, 2024-09-14T06:30:52.448 INFO:teuthology.orchestra.run.smithi094.stdout: "primary_affinity": 1, 2024-09-14T06:30:52.448 INFO:teuthology.orchestra.run.smithi094.stdout: "last_clean_begin": 0, 2024-09-14T06:30:52.449 INFO:teuthology.orchestra.run.smithi094.stdout: "last_clean_end": 0, 2024-09-14T06:30:52.449 INFO:teuthology.orchestra.run.smithi094.stdout: "up_from": 13, 2024-09-14T06:30:52.449 INFO:teuthology.orchestra.run.smithi094.stdout: "up_thru": 74, 2024-09-14T06:30:52.449 INFO:teuthology.orchestra.run.smithi094.stdout: "down_at": 0, 2024-09-14T06:30:52.449 INFO:teuthology.orchestra.run.smithi094.stdout: "lost_at": 0, 2024-09-14T06:30:52.449 INFO:teuthology.orchestra.run.smithi094.stdout: "public_addrs": { 2024-09-14T06:30:52.449 INFO:teuthology.orchestra.run.smithi094.stdout: "addrvec": [ 2024-09-14T06:30:52.449 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:52.449 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v2", 2024-09-14T06:30:52.449 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.94:6800", 2024-09-14T06:30:52.449 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 3382046754 2024-09-14T06:30:52.449 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:52.449 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:52.450 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v1", 2024-09-14T06:30:52.450 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.94:6801", 2024-09-14T06:30:52.450 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 3382046754 2024-09-14T06:30:52.450 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:52.450 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:52.450 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:52.450 INFO:teuthology.orchestra.run.smithi094.stdout: "cluster_addrs": { 2024-09-14T06:30:52.450 INFO:teuthology.orchestra.run.smithi094.stdout: "addrvec": [ 2024-09-14T06:30:52.450 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:52.450 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v2", 2024-09-14T06:30:52.450 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.94:6802", 2024-09-14T06:30:52.450 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 3382046754 2024-09-14T06:30:52.450 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:52.450 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:52.451 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v1", 2024-09-14T06:30:52.451 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.94:6803", 2024-09-14T06:30:52.451 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 3382046754 2024-09-14T06:30:52.451 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:52.451 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:52.451 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:52.451 INFO:teuthology.orchestra.run.smithi094.stdout: "heartbeat_back_addrs": { 2024-09-14T06:30:52.451 INFO:teuthology.orchestra.run.smithi094.stdout: "addrvec": [ 2024-09-14T06:30:52.451 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:52.451 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v2", 2024-09-14T06:30:52.451 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.94:6806", 2024-09-14T06:30:52.451 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 3382046754 2024-09-14T06:30:52.451 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:52.451 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:52.451 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v1", 2024-09-14T06:30:52.452 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.94:6807", 2024-09-14T06:30:52.452 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 3382046754 2024-09-14T06:30:52.452 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:52.452 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:52.452 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:52.452 INFO:teuthology.orchestra.run.smithi094.stdout: "heartbeat_front_addrs": { 2024-09-14T06:30:52.452 INFO:teuthology.orchestra.run.smithi094.stdout: "addrvec": [ 2024-09-14T06:30:52.452 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:52.452 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v2", 2024-09-14T06:30:52.452 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.94:6804", 2024-09-14T06:30:52.452 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 3382046754 2024-09-14T06:30:52.452 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:52.452 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:52.453 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v1", 2024-09-14T06:30:52.453 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.94:6805", 2024-09-14T06:30:52.453 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 3382046754 2024-09-14T06:30:52.453 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:52.453 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:52.453 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:52.453 INFO:teuthology.orchestra.run.smithi094.stdout: "public_addr": "172.21.15.94:6801/3382046754", 2024-09-14T06:30:52.453 INFO:teuthology.orchestra.run.smithi094.stdout: "cluster_addr": "172.21.15.94:6803/3382046754", 2024-09-14T06:30:52.453 INFO:teuthology.orchestra.run.smithi094.stdout: "heartbeat_back_addr": "172.21.15.94:6807/3382046754", 2024-09-14T06:30:52.453 INFO:teuthology.orchestra.run.smithi094.stdout: "heartbeat_front_addr": "172.21.15.94:6805/3382046754", 2024-09-14T06:30:52.453 INFO:teuthology.orchestra.run.smithi094.stdout: "state": [ 2024-09-14T06:30:52.453 INFO:teuthology.orchestra.run.smithi094.stdout: "exists", 2024-09-14T06:30:52.453 INFO:teuthology.orchestra.run.smithi094.stdout: "up" 2024-09-14T06:30:52.453 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:52.453 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:52.454 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:52.454 INFO:teuthology.orchestra.run.smithi094.stdout: "osd": 4, 2024-09-14T06:30:52.454 INFO:teuthology.orchestra.run.smithi094.stdout: "uuid": "2dab948e-c6e4-4437-bf17-017d1ac6821c", 2024-09-14T06:30:52.454 INFO:teuthology.orchestra.run.smithi094.stdout: "up": 1, 2024-09-14T06:30:52.454 INFO:teuthology.orchestra.run.smithi094.stdout: "in": 1, 2024-09-14T06:30:52.454 INFO:teuthology.orchestra.run.smithi094.stdout: "weight": 1, 2024-09-14T06:30:52.454 INFO:teuthology.orchestra.run.smithi094.stdout: "primary_affinity": 1, 2024-09-14T06:30:52.454 INFO:teuthology.orchestra.run.smithi094.stdout: "last_clean_begin": 0, 2024-09-14T06:30:52.454 INFO:teuthology.orchestra.run.smithi094.stdout: "last_clean_end": 0, 2024-09-14T06:30:52.454 INFO:teuthology.orchestra.run.smithi094.stdout: "up_from": 15, 2024-09-14T06:30:52.454 INFO:teuthology.orchestra.run.smithi094.stdout: "up_thru": 116, 2024-09-14T06:30:52.454 INFO:teuthology.orchestra.run.smithi094.stdout: "down_at": 0, 2024-09-14T06:30:52.454 INFO:teuthology.orchestra.run.smithi094.stdout: "lost_at": 0, 2024-09-14T06:30:52.454 INFO:teuthology.orchestra.run.smithi094.stdout: "public_addrs": { 2024-09-14T06:30:52.454 INFO:teuthology.orchestra.run.smithi094.stdout: "addrvec": [ 2024-09-14T06:30:52.454 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:52.455 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v2", 2024-09-14T06:30:52.455 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.138:6810", 2024-09-14T06:30:52.455 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 501726729 2024-09-14T06:30:52.455 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:52.455 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:52.455 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v1", 2024-09-14T06:30:52.455 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.138:6811", 2024-09-14T06:30:52.455 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 501726729 2024-09-14T06:30:52.455 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:52.455 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:52.455 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:52.455 INFO:teuthology.orchestra.run.smithi094.stdout: "cluster_addrs": { 2024-09-14T06:30:52.455 INFO:teuthology.orchestra.run.smithi094.stdout: "addrvec": [ 2024-09-14T06:30:52.455 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:52.455 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v2", 2024-09-14T06:30:52.455 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.138:6812", 2024-09-14T06:30:52.456 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 501726729 2024-09-14T06:30:52.456 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:52.456 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:52.456 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v1", 2024-09-14T06:30:52.456 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.138:6813", 2024-09-14T06:30:52.456 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 501726729 2024-09-14T06:30:52.456 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:52.456 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:52.456 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:52.456 INFO:teuthology.orchestra.run.smithi094.stdout: "heartbeat_back_addrs": { 2024-09-14T06:30:52.456 INFO:teuthology.orchestra.run.smithi094.stdout: "addrvec": [ 2024-09-14T06:30:52.456 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:52.456 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v2", 2024-09-14T06:30:52.456 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.138:6816", 2024-09-14T06:30:52.456 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 501726729 2024-09-14T06:30:52.457 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:52.457 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:52.457 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v1", 2024-09-14T06:30:52.457 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.138:6817", 2024-09-14T06:30:52.457 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 501726729 2024-09-14T06:30:52.457 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:52.457 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:52.457 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:52.457 INFO:teuthology.orchestra.run.smithi094.stdout: "heartbeat_front_addrs": { 2024-09-14T06:30:52.457 INFO:teuthology.orchestra.run.smithi094.stdout: "addrvec": [ 2024-09-14T06:30:52.457 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:52.457 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v2", 2024-09-14T06:30:52.457 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.138:6814", 2024-09-14T06:30:52.458 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 501726729 2024-09-14T06:30:52.458 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:52.458 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:52.458 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v1", 2024-09-14T06:30:52.458 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.138:6815", 2024-09-14T06:30:52.458 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 501726729 2024-09-14T06:30:52.458 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:52.458 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:52.458 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:52.458 INFO:teuthology.orchestra.run.smithi094.stdout: "public_addr": "172.21.15.138:6811/501726729", 2024-09-14T06:30:52.458 INFO:teuthology.orchestra.run.smithi094.stdout: "cluster_addr": "172.21.15.138:6813/501726729", 2024-09-14T06:30:52.458 INFO:teuthology.orchestra.run.smithi094.stdout: "heartbeat_back_addr": "172.21.15.138:6817/501726729", 2024-09-14T06:30:52.458 INFO:teuthology.orchestra.run.smithi094.stdout: "heartbeat_front_addr": "172.21.15.138:6815/501726729", 2024-09-14T06:30:52.458 INFO:teuthology.orchestra.run.smithi094.stdout: "state": [ 2024-09-14T06:30:52.458 INFO:teuthology.orchestra.run.smithi094.stdout: "exists", 2024-09-14T06:30:52.459 INFO:teuthology.orchestra.run.smithi094.stdout: "up" 2024-09-14T06:30:52.459 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:52.459 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:52.459 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:52.459 INFO:teuthology.orchestra.run.smithi094.stdout: "osd": 5, 2024-09-14T06:30:52.459 INFO:teuthology.orchestra.run.smithi094.stdout: "uuid": "8caf3aa3-ad87-4d33-b82e-4c15195176fd", 2024-09-14T06:30:52.459 INFO:teuthology.orchestra.run.smithi094.stdout: "up": 1, 2024-09-14T06:30:52.459 INFO:teuthology.orchestra.run.smithi094.stdout: "in": 1, 2024-09-14T06:30:52.459 INFO:teuthology.orchestra.run.smithi094.stdout: "weight": 1, 2024-09-14T06:30:52.459 INFO:teuthology.orchestra.run.smithi094.stdout: "primary_affinity": 1, 2024-09-14T06:30:52.459 INFO:teuthology.orchestra.run.smithi094.stdout: "last_clean_begin": 0, 2024-09-14T06:30:52.459 INFO:teuthology.orchestra.run.smithi094.stdout: "last_clean_end": 0, 2024-09-14T06:30:52.459 INFO:teuthology.orchestra.run.smithi094.stdout: "up_from": 15, 2024-09-14T06:30:52.459 INFO:teuthology.orchestra.run.smithi094.stdout: "up_thru": 23, 2024-09-14T06:30:52.460 INFO:teuthology.orchestra.run.smithi094.stdout: "down_at": 0, 2024-09-14T06:30:52.460 INFO:teuthology.orchestra.run.smithi094.stdout: "lost_at": 0, 2024-09-14T06:30:52.460 INFO:teuthology.orchestra.run.smithi094.stdout: "public_addrs": { 2024-09-14T06:30:52.460 INFO:teuthology.orchestra.run.smithi094.stdout: "addrvec": [ 2024-09-14T06:30:52.460 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:52.460 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v2", 2024-09-14T06:30:52.460 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.138:6826", 2024-09-14T06:30:52.460 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 1599292735 2024-09-14T06:30:52.460 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:52.460 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:52.460 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v1", 2024-09-14T06:30:52.460 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.138:6827", 2024-09-14T06:30:52.460 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 1599292735 2024-09-14T06:30:52.461 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:52.461 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:52.461 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:52.461 INFO:teuthology.orchestra.run.smithi094.stdout: "cluster_addrs": { 2024-09-14T06:30:52.461 INFO:teuthology.orchestra.run.smithi094.stdout: "addrvec": [ 2024-09-14T06:30:52.461 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:52.461 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v2", 2024-09-14T06:30:52.461 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.138:6828", 2024-09-14T06:30:52.461 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 1599292735 2024-09-14T06:30:52.461 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:52.461 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:52.461 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v1", 2024-09-14T06:30:52.461 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.138:6829", 2024-09-14T06:30:52.461 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 1599292735 2024-09-14T06:30:52.462 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:52.462 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:52.462 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:52.462 INFO:teuthology.orchestra.run.smithi094.stdout: "heartbeat_back_addrs": { 2024-09-14T06:30:52.462 INFO:teuthology.orchestra.run.smithi094.stdout: "addrvec": [ 2024-09-14T06:30:52.462 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:52.462 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v2", 2024-09-14T06:30:52.462 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.138:6832", 2024-09-14T06:30:52.462 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 1599292735 2024-09-14T06:30:52.462 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:52.462 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:52.462 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v1", 2024-09-14T06:30:52.462 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.138:6833", 2024-09-14T06:30:52.462 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 1599292735 2024-09-14T06:30:52.462 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:52.463 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:52.463 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:52.463 INFO:teuthology.orchestra.run.smithi094.stdout: "heartbeat_front_addrs": { 2024-09-14T06:30:52.463 INFO:teuthology.orchestra.run.smithi094.stdout: "addrvec": [ 2024-09-14T06:30:52.463 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:52.463 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v2", 2024-09-14T06:30:52.463 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.138:6830", 2024-09-14T06:30:52.463 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 1599292735 2024-09-14T06:30:52.463 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:52.463 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:52.463 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v1", 2024-09-14T06:30:52.463 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.138:6831", 2024-09-14T06:30:52.463 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 1599292735 2024-09-14T06:30:52.463 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:52.464 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:52.464 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:52.464 INFO:teuthology.orchestra.run.smithi094.stdout: "public_addr": "172.21.15.138:6827/1599292735", 2024-09-14T06:30:52.464 INFO:teuthology.orchestra.run.smithi094.stdout: "cluster_addr": "172.21.15.138:6829/1599292735", 2024-09-14T06:30:52.464 INFO:teuthology.orchestra.run.smithi094.stdout: "heartbeat_back_addr": "172.21.15.138:6833/1599292735", 2024-09-14T06:30:52.464 INFO:teuthology.orchestra.run.smithi094.stdout: "heartbeat_front_addr": "172.21.15.138:6831/1599292735", 2024-09-14T06:30:52.464 INFO:teuthology.orchestra.run.smithi094.stdout: "state": [ 2024-09-14T06:30:52.464 INFO:teuthology.orchestra.run.smithi094.stdout: "exists", 2024-09-14T06:30:52.464 INFO:teuthology.orchestra.run.smithi094.stdout: "up" 2024-09-14T06:30:52.464 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:52.464 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:52.464 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:52.464 INFO:teuthology.orchestra.run.smithi094.stdout: "osd": 6, 2024-09-14T06:30:52.464 INFO:teuthology.orchestra.run.smithi094.stdout: "uuid": "e55d11fd-433d-4462-93a7-d5e84560221b", 2024-09-14T06:30:52.465 INFO:teuthology.orchestra.run.smithi094.stdout: "up": 1, 2024-09-14T06:30:52.465 INFO:teuthology.orchestra.run.smithi094.stdout: "in": 1, 2024-09-14T06:30:52.465 INFO:teuthology.orchestra.run.smithi094.stdout: "weight": 1, 2024-09-14T06:30:52.465 INFO:teuthology.orchestra.run.smithi094.stdout: "primary_affinity": 1, 2024-09-14T06:30:52.465 INFO:teuthology.orchestra.run.smithi094.stdout: "last_clean_begin": 0, 2024-09-14T06:30:52.465 INFO:teuthology.orchestra.run.smithi094.stdout: "last_clean_end": 0, 2024-09-14T06:30:52.465 INFO:teuthology.orchestra.run.smithi094.stdout: "up_from": 15, 2024-09-14T06:30:52.465 INFO:teuthology.orchestra.run.smithi094.stdout: "up_thru": 23, 2024-09-14T06:30:52.465 INFO:teuthology.orchestra.run.smithi094.stdout: "down_at": 0, 2024-09-14T06:30:52.465 INFO:teuthology.orchestra.run.smithi094.stdout: "lost_at": 0, 2024-09-14T06:30:52.465 INFO:teuthology.orchestra.run.smithi094.stdout: "public_addrs": { 2024-09-14T06:30:52.465 INFO:teuthology.orchestra.run.smithi094.stdout: "addrvec": [ 2024-09-14T06:30:52.465 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:52.465 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v2", 2024-09-14T06:30:52.466 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.138:6818", 2024-09-14T06:30:52.466 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 163531479 2024-09-14T06:30:52.466 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:52.466 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:52.466 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v1", 2024-09-14T06:30:52.466 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.138:6819", 2024-09-14T06:30:52.466 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 163531479 2024-09-14T06:30:52.466 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:52.466 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:52.466 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:52.466 INFO:teuthology.orchestra.run.smithi094.stdout: "cluster_addrs": { 2024-09-14T06:30:52.466 INFO:teuthology.orchestra.run.smithi094.stdout: "addrvec": [ 2024-09-14T06:30:52.466 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:52.466 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v2", 2024-09-14T06:30:52.466 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.138:6820", 2024-09-14T06:30:52.467 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 163531479 2024-09-14T06:30:52.467 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:52.467 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:52.467 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v1", 2024-09-14T06:30:52.467 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.138:6821", 2024-09-14T06:30:52.467 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 163531479 2024-09-14T06:30:52.467 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:52.467 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:52.467 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:52.467 INFO:teuthology.orchestra.run.smithi094.stdout: "heartbeat_back_addrs": { 2024-09-14T06:30:52.467 INFO:teuthology.orchestra.run.smithi094.stdout: "addrvec": [ 2024-09-14T06:30:52.467 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:52.467 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v2", 2024-09-14T06:30:52.467 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.138:6824", 2024-09-14T06:30:52.467 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 163531479 2024-09-14T06:30:52.468 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:52.468 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:52.468 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v1", 2024-09-14T06:30:52.468 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.138:6825", 2024-09-14T06:30:52.468 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 163531479 2024-09-14T06:30:52.468 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:52.468 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:52.468 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:52.468 INFO:teuthology.orchestra.run.smithi094.stdout: "heartbeat_front_addrs": { 2024-09-14T06:30:52.468 INFO:teuthology.orchestra.run.smithi094.stdout: "addrvec": [ 2024-09-14T06:30:52.468 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:52.468 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v2", 2024-09-14T06:30:52.468 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.138:6822", 2024-09-14T06:30:52.468 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 163531479 2024-09-14T06:30:52.469 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:52.469 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:52.469 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v1", 2024-09-14T06:30:52.469 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.138:6823", 2024-09-14T06:30:52.469 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 163531479 2024-09-14T06:30:52.469 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:52.469 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:52.469 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:52.469 INFO:teuthology.orchestra.run.smithi094.stdout: "public_addr": "172.21.15.138:6819/163531479", 2024-09-14T06:30:52.469 INFO:teuthology.orchestra.run.smithi094.stdout: "cluster_addr": "172.21.15.138:6821/163531479", 2024-09-14T06:30:52.469 INFO:teuthology.orchestra.run.smithi094.stdout: "heartbeat_back_addr": "172.21.15.138:6825/163531479", 2024-09-14T06:30:52.469 INFO:teuthology.orchestra.run.smithi094.stdout: "heartbeat_front_addr": "172.21.15.138:6823/163531479", 2024-09-14T06:30:52.469 INFO:teuthology.orchestra.run.smithi094.stdout: "state": [ 2024-09-14T06:30:52.469 INFO:teuthology.orchestra.run.smithi094.stdout: "exists", 2024-09-14T06:30:52.470 INFO:teuthology.orchestra.run.smithi094.stdout: "up" 2024-09-14T06:30:52.470 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:52.470 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:52.470 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:52.470 INFO:teuthology.orchestra.run.smithi094.stdout: "osd": 7, 2024-09-14T06:30:52.470 INFO:teuthology.orchestra.run.smithi094.stdout: "uuid": "ba14ba69-2c6c-4d09-81e2-2593dc3a7ea9", 2024-09-14T06:30:52.470 INFO:teuthology.orchestra.run.smithi094.stdout: "up": 1, 2024-09-14T06:30:52.470 INFO:teuthology.orchestra.run.smithi094.stdout: "in": 1, 2024-09-14T06:30:52.470 INFO:teuthology.orchestra.run.smithi094.stdout: "weight": 1, 2024-09-14T06:30:52.470 INFO:teuthology.orchestra.run.smithi094.stdout: "primary_affinity": 1, 2024-09-14T06:30:52.470 INFO:teuthology.orchestra.run.smithi094.stdout: "last_clean_begin": 0, 2024-09-14T06:30:52.470 INFO:teuthology.orchestra.run.smithi094.stdout: "last_clean_end": 0, 2024-09-14T06:30:52.470 INFO:teuthology.orchestra.run.smithi094.stdout: "up_from": 15, 2024-09-14T06:30:52.471 INFO:teuthology.orchestra.run.smithi094.stdout: "up_thru": 23, 2024-09-14T06:30:52.471 INFO:teuthology.orchestra.run.smithi094.stdout: "down_at": 0, 2024-09-14T06:30:52.471 INFO:teuthology.orchestra.run.smithi094.stdout: "lost_at": 0, 2024-09-14T06:30:52.471 INFO:teuthology.orchestra.run.smithi094.stdout: "public_addrs": { 2024-09-14T06:30:52.471 INFO:teuthology.orchestra.run.smithi094.stdout: "addrvec": [ 2024-09-14T06:30:52.471 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:52.471 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v2", 2024-09-14T06:30:52.471 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.138:6800", 2024-09-14T06:30:52.471 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 218507235 2024-09-14T06:30:52.471 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:52.471 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:52.471 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v1", 2024-09-14T06:30:52.471 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.138:6801", 2024-09-14T06:30:52.471 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 218507235 2024-09-14T06:30:52.472 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:52.472 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:52.472 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:52.472 INFO:teuthology.orchestra.run.smithi094.stdout: "cluster_addrs": { 2024-09-14T06:30:52.472 INFO:teuthology.orchestra.run.smithi094.stdout: "addrvec": [ 2024-09-14T06:30:52.472 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:52.472 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v2", 2024-09-14T06:30:52.472 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.138:6802", 2024-09-14T06:30:52.472 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 218507235 2024-09-14T06:30:52.472 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:52.472 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:52.472 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v1", 2024-09-14T06:30:52.472 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.138:6803", 2024-09-14T06:30:52.473 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 218507235 2024-09-14T06:30:52.473 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:52.473 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:52.473 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:52.473 INFO:teuthology.orchestra.run.smithi094.stdout: "heartbeat_back_addrs": { 2024-09-14T06:30:52.473 INFO:teuthology.orchestra.run.smithi094.stdout: "addrvec": [ 2024-09-14T06:30:52.473 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:52.473 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v2", 2024-09-14T06:30:52.473 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.138:6806", 2024-09-14T06:30:52.473 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 218507235 2024-09-14T06:30:52.473 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:52.473 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:52.473 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v1", 2024-09-14T06:30:52.473 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.138:6807", 2024-09-14T06:30:52.474 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 218507235 2024-09-14T06:30:52.474 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:52.474 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:52.474 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:52.474 INFO:teuthology.orchestra.run.smithi094.stdout: "heartbeat_front_addrs": { 2024-09-14T06:30:52.474 INFO:teuthology.orchestra.run.smithi094.stdout: "addrvec": [ 2024-09-14T06:30:52.474 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:52.474 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v2", 2024-09-14T06:30:52.474 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.138:6804", 2024-09-14T06:30:52.474 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 218507235 2024-09-14T06:30:52.474 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:52.474 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:52.474 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v1", 2024-09-14T06:30:52.474 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.138:6805", 2024-09-14T06:30:52.474 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 218507235 2024-09-14T06:30:52.475 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:52.475 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:52.475 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:52.475 INFO:teuthology.orchestra.run.smithi094.stdout: "public_addr": "172.21.15.138:6801/218507235", 2024-09-14T06:30:52.475 INFO:teuthology.orchestra.run.smithi094.stdout: "cluster_addr": "172.21.15.138:6803/218507235", 2024-09-14T06:30:52.475 INFO:teuthology.orchestra.run.smithi094.stdout: "heartbeat_back_addr": "172.21.15.138:6807/218507235", 2024-09-14T06:30:52.475 INFO:teuthology.orchestra.run.smithi094.stdout: "heartbeat_front_addr": "172.21.15.138:6805/218507235", 2024-09-14T06:30:52.475 INFO:teuthology.orchestra.run.smithi094.stdout: "state": [ 2024-09-14T06:30:52.475 INFO:teuthology.orchestra.run.smithi094.stdout: "exists", 2024-09-14T06:30:52.475 INFO:teuthology.orchestra.run.smithi094.stdout: "up" 2024-09-14T06:30:52.475 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:52.475 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:52.475 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-09-14T06:30:52.475 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_xinfo": [ 2024-09-14T06:30:52.476 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:52.476 INFO:teuthology.orchestra.run.smithi094.stdout: "osd": 0, 2024-09-14T06:30:52.476 INFO:teuthology.orchestra.run.smithi094.stdout: "down_stamp": "0.000000", 2024-09-14T06:30:52.476 INFO:teuthology.orchestra.run.smithi094.stdout: "laggy_probability": 0, 2024-09-14T06:30:52.476 INFO:teuthology.orchestra.run.smithi094.stdout: "laggy_interval": 0, 2024-09-14T06:30:52.476 INFO:teuthology.orchestra.run.smithi094.stdout: "features": 4540719139924082687, 2024-09-14T06:30:52.476 INFO:teuthology.orchestra.run.smithi094.stdout: "old_weight": 0, 2024-09-14T06:30:52.476 INFO:teuthology.orchestra.run.smithi094.stdout: "last_purged_snaps_scrub": "2024-09-14T06:28:39.906639+0000", 2024-09-14T06:30:52.476 INFO:teuthology.orchestra.run.smithi094.stdout: "dead_epoch": 0 2024-09-14T06:30:52.476 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:52.476 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:52.476 INFO:teuthology.orchestra.run.smithi094.stdout: "osd": 1, 2024-09-14T06:30:52.476 INFO:teuthology.orchestra.run.smithi094.stdout: "down_stamp": "0.000000", 2024-09-14T06:30:52.476 INFO:teuthology.orchestra.run.smithi094.stdout: "laggy_probability": 0, 2024-09-14T06:30:52.477 INFO:teuthology.orchestra.run.smithi094.stdout: "laggy_interval": 0, 2024-09-14T06:30:52.477 INFO:teuthology.orchestra.run.smithi094.stdout: "features": 4540719139924082687, 2024-09-14T06:30:52.477 INFO:teuthology.orchestra.run.smithi094.stdout: "old_weight": 0, 2024-09-14T06:30:52.477 INFO:teuthology.orchestra.run.smithi094.stdout: "last_purged_snaps_scrub": "2024-09-14T06:28:40.541279+0000", 2024-09-14T06:30:52.477 INFO:teuthology.orchestra.run.smithi094.stdout: "dead_epoch": 0 2024-09-14T06:30:52.477 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:52.477 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:52.477 INFO:teuthology.orchestra.run.smithi094.stdout: "osd": 2, 2024-09-14T06:30:52.477 INFO:teuthology.orchestra.run.smithi094.stdout: "down_stamp": "0.000000", 2024-09-14T06:30:52.477 INFO:teuthology.orchestra.run.smithi094.stdout: "laggy_probability": 0, 2024-09-14T06:30:52.477 INFO:teuthology.orchestra.run.smithi094.stdout: "laggy_interval": 0, 2024-09-14T06:30:52.477 INFO:teuthology.orchestra.run.smithi094.stdout: "features": 4540719139924082687, 2024-09-14T06:30:52.477 INFO:teuthology.orchestra.run.smithi094.stdout: "old_weight": 0, 2024-09-14T06:30:52.478 INFO:teuthology.orchestra.run.smithi094.stdout: "last_purged_snaps_scrub": "2024-09-14T06:28:40.025350+0000", 2024-09-14T06:30:52.478 INFO:teuthology.orchestra.run.smithi094.stdout: "dead_epoch": 0 2024-09-14T06:30:52.478 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:52.478 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:52.478 INFO:teuthology.orchestra.run.smithi094.stdout: "osd": 3, 2024-09-14T06:30:52.478 INFO:teuthology.orchestra.run.smithi094.stdout: "down_stamp": "0.000000", 2024-09-14T06:30:52.478 INFO:teuthology.orchestra.run.smithi094.stdout: "laggy_probability": 0, 2024-09-14T06:30:52.478 INFO:teuthology.orchestra.run.smithi094.stdout: "laggy_interval": 0, 2024-09-14T06:30:52.478 INFO:teuthology.orchestra.run.smithi094.stdout: "features": 4540719139924082687, 2024-09-14T06:30:52.478 INFO:teuthology.orchestra.run.smithi094.stdout: "old_weight": 0, 2024-09-14T06:30:52.479 INFO:teuthology.orchestra.run.smithi094.stdout: "last_purged_snaps_scrub": "2024-09-14T06:28:39.217944+0000", 2024-09-14T06:30:52.479 INFO:teuthology.orchestra.run.smithi094.stdout: "dead_epoch": 0 2024-09-14T06:30:52.479 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:52.479 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:52.479 INFO:teuthology.orchestra.run.smithi094.stdout: "osd": 4, 2024-09-14T06:30:52.479 INFO:teuthology.orchestra.run.smithi094.stdout: "down_stamp": "0.000000", 2024-09-14T06:30:52.479 INFO:teuthology.orchestra.run.smithi094.stdout: "laggy_probability": 0, 2024-09-14T06:30:52.479 INFO:teuthology.orchestra.run.smithi094.stdout: "laggy_interval": 0, 2024-09-14T06:30:52.479 INFO:teuthology.orchestra.run.smithi094.stdout: "features": 4540719139924082687, 2024-09-14T06:30:52.479 INFO:teuthology.orchestra.run.smithi094.stdout: "old_weight": 0, 2024-09-14T06:30:52.479 INFO:teuthology.orchestra.run.smithi094.stdout: "last_purged_snaps_scrub": "2024-09-14T06:28:40.185898+0000", 2024-09-14T06:30:52.479 INFO:teuthology.orchestra.run.smithi094.stdout: "dead_epoch": 0 2024-09-14T06:30:52.479 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:52.480 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:52.480 INFO:teuthology.orchestra.run.smithi094.stdout: "osd": 5, 2024-09-14T06:30:52.480 INFO:teuthology.orchestra.run.smithi094.stdout: "down_stamp": "0.000000", 2024-09-14T06:30:52.480 INFO:teuthology.orchestra.run.smithi094.stdout: "laggy_probability": 0, 2024-09-14T06:30:52.480 INFO:teuthology.orchestra.run.smithi094.stdout: "laggy_interval": 0, 2024-09-14T06:30:52.480 INFO:teuthology.orchestra.run.smithi094.stdout: "features": 4540719139924082687, 2024-09-14T06:30:52.480 INFO:teuthology.orchestra.run.smithi094.stdout: "old_weight": 0, 2024-09-14T06:30:52.480 INFO:teuthology.orchestra.run.smithi094.stdout: "last_purged_snaps_scrub": "2024-09-14T06:28:40.783504+0000", 2024-09-14T06:30:52.480 INFO:teuthology.orchestra.run.smithi094.stdout: "dead_epoch": 0 2024-09-14T06:30:52.480 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:52.480 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:52.480 INFO:teuthology.orchestra.run.smithi094.stdout: "osd": 6, 2024-09-14T06:30:52.480 INFO:teuthology.orchestra.run.smithi094.stdout: "down_stamp": "0.000000", 2024-09-14T06:30:52.480 INFO:teuthology.orchestra.run.smithi094.stdout: "laggy_probability": 0, 2024-09-14T06:30:52.481 INFO:teuthology.orchestra.run.smithi094.stdout: "laggy_interval": 0, 2024-09-14T06:30:52.481 INFO:teuthology.orchestra.run.smithi094.stdout: "features": 4540719139924082687, 2024-09-14T06:30:52.481 INFO:teuthology.orchestra.run.smithi094.stdout: "old_weight": 0, 2024-09-14T06:30:52.481 INFO:teuthology.orchestra.run.smithi094.stdout: "last_purged_snaps_scrub": "2024-09-14T06:28:40.527305+0000", 2024-09-14T06:30:52.481 INFO:teuthology.orchestra.run.smithi094.stdout: "dead_epoch": 0 2024-09-14T06:30:52.481 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:52.481 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:52.481 INFO:teuthology.orchestra.run.smithi094.stdout: "osd": 7, 2024-09-14T06:30:52.481 INFO:teuthology.orchestra.run.smithi094.stdout: "down_stamp": "0.000000", 2024-09-14T06:30:52.481 INFO:teuthology.orchestra.run.smithi094.stdout: "laggy_probability": 0, 2024-09-14T06:30:52.481 INFO:teuthology.orchestra.run.smithi094.stdout: "laggy_interval": 0, 2024-09-14T06:30:52.481 INFO:teuthology.orchestra.run.smithi094.stdout: "features": 4540719139924082687, 2024-09-14T06:30:52.481 INFO:teuthology.orchestra.run.smithi094.stdout: "old_weight": 0, 2024-09-14T06:30:52.481 INFO:teuthology.orchestra.run.smithi094.stdout: "last_purged_snaps_scrub": "2024-09-14T06:28:40.349676+0000", 2024-09-14T06:30:52.482 INFO:teuthology.orchestra.run.smithi094.stdout: "dead_epoch": 0 2024-09-14T06:30:52.482 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:52.482 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-09-14T06:30:52.482 INFO:teuthology.orchestra.run.smithi094.stdout: "pg_upmap": [ 2024-09-14T06:30:52.482 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:52.482 INFO:teuthology.orchestra.run.smithi094.stdout: "pgid": "3.1", 2024-09-14T06:30:52.482 INFO:teuthology.orchestra.run.smithi094.stdout: "osds": [ 2024-09-14T06:30:52.482 INFO:teuthology.orchestra.run.smithi094.stdout: 4, 2024-09-14T06:30:52.482 INFO:teuthology.orchestra.run.smithi094.stdout: 1 2024-09-14T06:30:52.482 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:52.482 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:52.482 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-09-14T06:30:52.482 INFO:teuthology.orchestra.run.smithi094.stdout: "pg_upmap_items": [ 2024-09-14T06:30:52.482 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:52.483 INFO:teuthology.orchestra.run.smithi094.stdout: "pgid": "3.7", 2024-09-14T06:30:52.483 INFO:teuthology.orchestra.run.smithi094.stdout: "mappings": [ 2024-09-14T06:30:52.483 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:52.483 INFO:teuthology.orchestra.run.smithi094.stdout: "from": 3, 2024-09-14T06:30:52.483 INFO:teuthology.orchestra.run.smithi094.stdout: "to": 0 2024-09-14T06:30:52.483 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:52.483 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:52.483 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:52.483 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-09-14T06:30:52.483 INFO:teuthology.orchestra.run.smithi094.stdout: "pg_upmap_primaries": [], 2024-09-14T06:30:52.483 INFO:teuthology.orchestra.run.smithi094.stdout: "pg_temp": [], 2024-09-14T06:30:52.483 INFO:teuthology.orchestra.run.smithi094.stdout: "primary_temp": [], 2024-09-14T06:30:52.483 INFO:teuthology.orchestra.run.smithi094.stdout: "blocklist": {}, 2024-09-14T06:30:52.483 INFO:teuthology.orchestra.run.smithi094.stdout: "range_blocklist": {}, 2024-09-14T06:30:52.484 INFO:teuthology.orchestra.run.smithi094.stdout: "erasure_code_profiles": { 2024-09-14T06:30:52.484 INFO:teuthology.orchestra.run.smithi094.stdout: "default": { 2024-09-14T06:30:52.484 INFO:teuthology.orchestra.run.smithi094.stdout: "crush-failure-domain": "osd", 2024-09-14T06:30:52.484 INFO:teuthology.orchestra.run.smithi094.stdout: "k": "2", 2024-09-14T06:30:52.484 INFO:teuthology.orchestra.run.smithi094.stdout: "m": "1", 2024-09-14T06:30:52.484 INFO:teuthology.orchestra.run.smithi094.stdout: "plugin": "jerasure", 2024-09-14T06:30:52.484 INFO:teuthology.orchestra.run.smithi094.stdout: "technique": "reed_sol_van" 2024-09-14T06:30:52.484 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:52.484 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:52.484 INFO:teuthology.orchestra.run.smithi094.stdout: "removed_snaps_queue": [ 2024-09-14T06:30:52.484 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:52.484 INFO:teuthology.orchestra.run.smithi094.stdout: "pool": 3, 2024-09-14T06:30:52.484 INFO:teuthology.orchestra.run.smithi094.stdout: "snaps": [ 2024-09-14T06:30:52.484 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:52.485 INFO:teuthology.orchestra.run.smithi094.stdout: "begin": 56, 2024-09-14T06:30:52.485 INFO:teuthology.orchestra.run.smithi094.stdout: "length": 1 2024-09-14T06:30:52.485 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:52.485 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:52.485 INFO:teuthology.orchestra.run.smithi094.stdout: "begin": 58, 2024-09-14T06:30:52.485 INFO:teuthology.orchestra.run.smithi094.stdout: "length": 1 2024-09-14T06:30:52.485 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:52.485 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:52.485 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:52.485 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-09-14T06:30:52.485 INFO:teuthology.orchestra.run.smithi094.stdout: "new_removed_snaps": [], 2024-09-14T06:30:52.485 INFO:teuthology.orchestra.run.smithi094.stdout: "new_purged_snaps": [], 2024-09-14T06:30:52.485 INFO:teuthology.orchestra.run.smithi094.stdout: "crush_node_flags": {}, 2024-09-14T06:30:52.486 INFO:teuthology.orchestra.run.smithi094.stdout: "device_class_flags": {}, 2024-09-14T06:30:52.486 INFO:teuthology.orchestra.run.smithi094.stdout: "stretch_mode": { 2024-09-14T06:30:52.486 INFO:teuthology.orchestra.run.smithi094.stdout: "stretch_mode_enabled": false, 2024-09-14T06:30:52.486 INFO:teuthology.orchestra.run.smithi094.stdout: "stretch_bucket_count": 0, 2024-09-14T06:30:52.486 INFO:teuthology.orchestra.run.smithi094.stdout: "degraded_stretch_mode": 0, 2024-09-14T06:30:52.486 INFO:teuthology.orchestra.run.smithi094.stdout: "recovering_stretch_mode": 0, 2024-09-14T06:30:52.486 INFO:teuthology.orchestra.run.smithi094.stdout: "stretch_mode_bucket": 0 2024-09-14T06:30:52.486 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:52.486 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-14T06:30:52.488 INFO:teuthology.orchestra.run.smithi138.stdout:ERROR: (22) Invalid argument 2024-09-14T06:30:52.488 INFO:teuthology.orchestra.run.smithi138.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:30:52.489 INFO:tasks.thrashosds.thrasher:j is {'epoch': 128, 'fsid': '200ad3da-bd63-4e04-af74-efb5ca0af75e', 'created': '2024-09-14T06:28:30.782046+0000', 'modified': '2024-09-14T06:30:51.588445+0000', 'last_up_change': '2024-09-14T06:28:42.827239+0000', 'last_in_change': '2024-09-14T06:28:33.724176+0000', 'flags': 'noscrub,nodeep-scrub,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit', 'flags_num': 5806080, 'flags_set': ['nodeep-scrub', 'noscrub', 'pglog_hardlimit', 'purged_snapdirs', 'recovery_deletes', 'sortbitwise'], 'crush_version': 5, 'full_ratio': 0.949999988079071, 'backfillfull_ratio': 0.8999999761581421, 'nearfull_ratio': 0.8500000238418579, 'cluster_snapshot': '', 'pool_max': 3, 'max_osd': 8, 'require_min_compat_client': 'luminous', 'min_compat_client': 'luminous', 'require_osd_release': 'squid', 'allow_crimson': False, 'pools': [{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-14T06:28:43.295990+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2024-09-14T06:28:49.451714+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 2, 'snap_epoch': 22, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.9900000095367432, 'score_stable': 1.9900000095367432, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'pool_client.0', 'create_time': '2024-09-14T06:28:59.688535+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '128', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 60, 'snap_epoch': 128, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}], 'osds': [{'osd': 0, 'uuid': '8b22a249-b842-406a-a02c-c0f31f57b635', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 75, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.94:6824', 'nonce': 1809564923}, {'type': 'v1', 'addr': '172.21.15.94:6825', 'nonce': 1809564923}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.94:6826', 'nonce': 1809564923}, {'type': 'v1', 'addr': '172.21.15.94:6827', 'nonce': 1809564923}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.94:6830', 'nonce': 1809564923}, {'type': 'v1', 'addr': '172.21.15.94:6831', 'nonce': 1809564923}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.94:6828', 'nonce': 1809564923}, {'type': 'v1', 'addr': '172.21.15.94:6829', 'nonce': 1809564923}]}, 'public_addr': '172.21.15.94:6825/1809564923', 'cluster_addr': '172.21.15.94:6827/1809564923', 'heartbeat_back_addr': '172.21.15.94:6831/1809564923', 'heartbeat_front_addr': '172.21.15.94:6829/1809564923', 'state': ['exists', 'up']}, {'osd': 1, 'uuid': '02251a03-84a0-414d-a980-fac4fd043cb6', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 23, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.94:6816', 'nonce': 736402652}, {'type': 'v1', 'addr': '172.21.15.94:6817', 'nonce': 736402652}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.94:6818', 'nonce': 736402652}, {'type': 'v1', 'addr': '172.21.15.94:6819', 'nonce': 736402652}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.94:6822', 'nonce': 736402652}, {'type': 'v1', 'addr': '172.21.15.94:6823', 'nonce': 736402652}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.94:6820', 'nonce': 736402652}, {'type': 'v1', 'addr': '172.21.15.94:6821', 'nonce': 736402652}]}, 'public_addr': '172.21.15.94:6817/736402652', 'cluster_addr': '172.21.15.94:6819/736402652', 'heartbeat_back_addr': '172.21.15.94:6823/736402652', 'heartbeat_front_addr': '172.21.15.94:6821/736402652', 'state': ['exists', 'up']}, {'osd': 2, 'uuid': 'd45bf470-bb29-4739-8294-cd122b2b4102', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 19, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.94:6808', 'nonce': 1063204782}, {'type': 'v1', 'addr': '172.21.15.94:6809', 'nonce': 1063204782}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.94:6810', 'nonce': 1063204782}, {'type': 'v1', 'addr': '172.21.15.94:6811', 'nonce': 1063204782}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.94:6814', 'nonce': 1063204782}, {'type': 'v1', 'addr': '172.21.15.94:6815', 'nonce': 1063204782}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.94:6812', 'nonce': 1063204782}, {'type': 'v1', 'addr': '172.21.15.94:6813', 'nonce': 1063204782}]}, 'public_addr': '172.21.15.94:6809/1063204782', 'cluster_addr': '172.21.15.94:6811/1063204782', 'heartbeat_back_addr': '172.21.15.94:6815/1063204782', 'heartbeat_front_addr': '172.21.15.94:6813/1063204782', 'state': ['exists', 'up']}, {'osd': 3, 'uuid': '456ec2d9-c438-4243-b931-335704f661eb', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 74, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.94:6800', 'nonce': 3382046754}, {'type': 'v1', 'addr': '172.21.15.94:6801', 'nonce': 3382046754}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.94:6802', 'nonce': 3382046754}, {'type': 'v1', 'addr': '172.21.15.94:6803', 'nonce': 3382046754}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.94:6806', 'nonce': 3382046754}, {'type': 'v1', 'addr': '172.21.15.94:6807', 'nonce': 3382046754}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.94:6804', 'nonce': 3382046754}, {'type': 'v1', 'addr': '172.21.15.94:6805', 'nonce': 3382046754}]}, 'public_addr': '172.21.15.94:6801/3382046754', 'cluster_addr': '172.21.15.94:6803/3382046754', 'heartbeat_back_addr': '172.21.15.94:6807/3382046754', 'heartbeat_front_addr': '172.21.15.94:6805/3382046754', 'state': ['exists', 'up']}, {'osd': 4, 'uuid': '2dab948e-c6e4-4437-bf17-017d1ac6821c', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 116, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.138:6810', 'nonce': 501726729}, {'type': 'v1', 'addr': '172.21.15.138:6811', 'nonce': 501726729}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.138:6812', 'nonce': 501726729}, {'type': 'v1', 'addr': '172.21.15.138:6813', 'nonce': 501726729}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.138:6816', 'nonce': 501726729}, {'type': 'v1', 'addr': '172.21.15.138:6817', 'nonce': 501726729}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.138:6814', 'nonce': 501726729}, {'type': 'v1', 'addr': '172.21.15.138:6815', 'nonce': 501726729}]}, 'public_addr': '172.21.15.138:6811/501726729', 'cluster_addr': '172.21.15.138:6813/501726729', 'heartbeat_back_addr': '172.21.15.138:6817/501726729', 'heartbeat_front_addr': '172.21.15.138:6815/501726729', 'state': ['exists', 'up']}, {'osd': 5, 'uuid': '8caf3aa3-ad87-4d33-b82e-4c15195176fd', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 23, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.138:6826', 'nonce': 1599292735}, {'type': 'v1', 'addr': '172.21.15.138:6827', 'nonce': 1599292735}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.138:6828', 'nonce': 1599292735}, {'type': 'v1', 'addr': '172.21.15.138:6829', 'nonce': 1599292735}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.138:6832', 'nonce': 1599292735}, {'type': 'v1', 'addr': '172.21.15.138:6833', 'nonce': 1599292735}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.138:6830', 'nonce': 1599292735}, {'type': 'v1', 'addr': '172.21.15.138:6831', 'nonce': 1599292735}]}, 'public_addr': '172.21.15.138:6827/1599292735', 'cluster_addr': '172.21.15.138:6829/1599292735', 'heartbeat_back_addr': '172.21.15.138:6833/1599292735', 'heartbeat_front_addr': '172.21.15.138:6831/1599292735', 'state': ['exists', 'up']}, {'osd': 6, 'uuid': 'e55d11fd-433d-4462-93a7-d5e84560221b', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 23, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.138:6818', 'nonce': 163531479}, {'type': 'v1', 'addr': '172.21.15.138:6819', 'nonce': 163531479}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.138:6820', 'nonce': 163531479}, {'type': 'v1', 'addr': '172.21.15.138:6821', 'nonce': 163531479}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.138:6824', 'nonce': 163531479}, {'type': 'v1', 'addr': '172.21.15.138:6825', 'nonce': 163531479}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.138:6822', 'nonce': 163531479}, {'type': 'v1', 'addr': '172.21.15.138:6823', 'nonce': 163531479}]}, 'public_addr': '172.21.15.138:6819/163531479', 'cluster_addr': '172.21.15.138:6821/163531479', 'heartbeat_back_addr': '172.21.15.138:6825/163531479', 'heartbeat_front_addr': '172.21.15.138:6823/163531479', 'state': ['exists', 'up']}, {'osd': 7, 'uuid': 'ba14ba69-2c6c-4d09-81e2-2593dc3a7ea9', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 23, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.138:6800', 'nonce': 218507235}, {'type': 'v1', 'addr': '172.21.15.138:6801', 'nonce': 218507235}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.138:6802', 'nonce': 218507235}, {'type': 'v1', 'addr': '172.21.15.138:6803', 'nonce': 218507235}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.138:6806', 'nonce': 218507235}, {'type': 'v1', 'addr': '172.21.15.138:6807', 'nonce': 218507235}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.138:6804', 'nonce': 218507235}, {'type': 'v1', 'addr': '172.21.15.138:6805', 'nonce': 218507235}]}, 'public_addr': '172.21.15.138:6801/218507235', 'cluster_addr': '172.21.15.138:6803/218507235', 'heartbeat_back_addr': '172.21.15.138:6807/218507235', 'heartbeat_front_addr': '172.21.15.138:6805/218507235', 'state': ['exists', 'up']}], 'osd_xinfo': [{'osd': 0, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-14T06:28:39.906639+0000', 'dead_epoch': 0}, {'osd': 1, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-14T06:28:40.541279+0000', 'dead_epoch': 0}, {'osd': 2, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-14T06:28:40.025350+0000', 'dead_epoch': 0}, {'osd': 3, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-14T06:28:39.217944+0000', 'dead_epoch': 0}, {'osd': 4, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-14T06:28:40.185898+0000', 'dead_epoch': 0}, {'osd': 5, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-14T06:28:40.783504+0000', 'dead_epoch': 0}, {'osd': 6, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-14T06:28:40.527305+0000', 'dead_epoch': 0}, {'osd': 7, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-14T06:28:40.349676+0000', 'dead_epoch': 0}], 'pg_upmap': [{'pgid': '3.1', 'osds': [4, 1]}], 'pg_upmap_items': [{'pgid': '3.7', 'mappings': [{'from': 3, 'to': 0}]}], 'pg_upmap_primaries': [], 'pg_temp': [], 'primary_temp': [], 'blocklist': {}, 'range_blocklist': {}, 'erasure_code_profiles': {'default': {'crush-failure-domain': 'osd', 'k': '2', 'm': '1', 'plugin': 'jerasure', 'technique': 'reed_sol_van'}}, 'removed_snaps_queue': [{'pool': 3, 'snaps': [{'begin': 56, 'length': 1}, {'begin': 58, 'length': 1}]}], 'new_removed_snaps': [], 'new_purged_snaps': [], 'crush_node_flags': {}, 'device_class_flags': {}, 'stretch_mode': {'stretch_mode_enabled': False, 'stretch_bucket_count': 0, 'degraded_stretch_mode': 0, 'recovering_stretch_mode': 0, 'stretch_mode_bucket': 0}} 2024-09-14T06:30:52.489 INFO:tasks.thrashosds.thrasher:Clearing pg_upmap on 3.1 2024-09-14T06:30:52.489 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd rm-pg-upmap 3.1 2024-09-14T06:30:52.490 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-14T06:30:52.490 INFO:tasks.ceph.osd.5.smithi138.stderr:2024-09-14T06:30:52.486+0000 7fe13746f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33618) UID: 0 2024-09-14T06:30:52.588 INFO:tasks.ceph.osd.4.smithi138.stderr:2024-09-14T06:30:52.587+0000 7f6c653c8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33615) UID: 0 2024-09-14T06:30:52.632 INFO:teuthology.orchestra.run.smithi138.stdout:ERROR: (22) Invalid argument 2024-09-14T06:30:52.633 INFO:teuthology.orchestra.run.smithi138.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:30:52.641 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-14T06:30:52.689 INFO:tasks.ceph.osd.2.smithi094.stderr:2024-09-14T06:30:52.688+0000 7faf3269b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33649) UID: 0 2024-09-14T06:30:52.789 INFO:teuthology.orchestra.run.smithi138.stdout:ERROR: (22) Invalid argument 2024-09-14T06:30:52.789 INFO:teuthology.orchestra.run.smithi138.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:30:52.789 INFO:tasks.ceph.osd.4.smithi138.stderr:2024-09-14T06:30:52.788+0000 7f6c653c8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33615) UID: 0 2024-09-14T06:30:52.794 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-14T06:30:52.798 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-14T06:30:52.837 INFO:teuthology.orchestra.run.smithi094.stdout:checking clone #27, image image_client.0-clone27 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent28 2024-09-14T06:30:52.891 INFO:tasks.ceph.osd.3.smithi094.stderr:2024-09-14T06:30:52.889+0000 7feeeafe8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33655) UID: 0 2024-09-14T06:30:52.950 INFO:teuthology.orchestra.run.smithi138.stdout:ERROR: (22) Invalid argument 2024-09-14T06:30:52.950 INFO:teuthology.orchestra.run.smithi138.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:30:52.951 INFO:teuthology.orchestra.run.smithi094.stdout:osd.0: {} 2024-09-14T06:30:52.952 INFO:teuthology.orchestra.run.smithi094.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-14T06:30:52.959 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-14T06:30:52.969 INFO:teuthology.orchestra.run.smithi094.stdout:osd.1: {} 2024-09-14T06:30:52.969 INFO:teuthology.orchestra.run.smithi094.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-14T06:30:52.985 INFO:teuthology.orchestra.run.smithi094.stdout:osd.2: {} 2024-09-14T06:30:52.986 INFO:teuthology.orchestra.run.smithi094.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-14T06:30:52.992 INFO:tasks.ceph.osd.0.smithi094.stderr:2024-09-14T06:30:52.991+0000 7f07c9333640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33636) UID: 0 2024-09-14T06:30:53.003 INFO:teuthology.orchestra.run.smithi094.stdout:osd.3: {} 2024-09-14T06:30:53.003 INFO:teuthology.orchestra.run.smithi094.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-14T06:30:53.020 INFO:teuthology.orchestra.run.smithi094.stdout:osd.4: {} 2024-09-14T06:30:53.020 INFO:teuthology.orchestra.run.smithi094.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-14T06:30:53.037 INFO:teuthology.orchestra.run.smithi094.stdout:osd.5: {} 2024-09-14T06:30:53.037 INFO:teuthology.orchestra.run.smithi094.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-14T06:30:53.045 INFO:teuthology.orchestra.run.smithi094.stdout:3799 write 0xa605924 thru 0xa6136fd (0xddda bytes) 2024-09-14T06:30:53.054 INFO:teuthology.orchestra.run.smithi094.stdout:osd.6: {} 2024-09-14T06:30:53.054 INFO:teuthology.orchestra.run.smithi094.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-14T06:30:53.055 INFO:teuthology.orchestra.run.smithi094.stdout:3800 writesame 0x27521fc thru 0x27599d7 data_size 0x6(0x77dc bytes) 2024-09-14T06:30:53.058 INFO:teuthology.orchestra.run.smithi094.stdout:3801 read 0x7ca3828 thru 0x7ca9490 (0x5c69 bytes) 2024-09-14T06:30:53.058 INFO:teuthology.orchestra.run.smithi094.stdout:3802 punch from 0x8eda78b to 0x8ee6914, (0xc189 bytes) 2024-09-14T06:30:53.058 INFO:teuthology.orchestra.run.smithi094.stdout:3803 read 0x9666ab8 thru 0x9668268 (0x17b1 bytes) 2024-09-14T06:30:53.058 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-14T06:30:53.058 INFO:teuthology.orchestra.run.smithi094.stdout:3807 read 0x61f8be1 thru 0x61ffb55 (0x6f75 bytes) 2024-09-14T06:30:53.059 INFO:teuthology.orchestra.run.smithi094.stdout:3808 write 0x9e3b79c thru 0x9e41bda (0x643f bytes) 2024-09-14T06:30:53.068 INFO:teuthology.orchestra.run.smithi138.stdout:{ 2024-09-14T06:30:53.068 INFO:teuthology.orchestra.run.smithi138.stdout: "ops": [], 2024-09-14T06:30:53.068 INFO:teuthology.orchestra.run.smithi138.stdout: "complaint_time": 30, 2024-09-14T06:30:53.068 INFO:teuthology.orchestra.run.smithi138.stdout: "num_blocked_ops": 0 2024-09-14T06:30:53.068 INFO:teuthology.orchestra.run.smithi138.stdout:} 2024-09-14T06:30:53.072 INFO:teuthology.orchestra.run.smithi094.stdout:osd.7: {} 2024-09-14T06:30:53.072 INFO:teuthology.orchestra.run.smithi094.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-14T06:30:53.075 INFO:teuthology.orchestra.run.smithi094.stdout:3810 trunc from 0xa6136fe to 0x204f725 2024-09-14T06:30:53.077 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-14T06:30:53.129 INFO:tasks.ceph.osd.2.smithi094.stderr:2024-09-14T06:30:53.128+0000 7faf3269b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33649) UID: 0 2024-09-14T06:30:53.194 INFO:tasks.ceph.osd.2.smithi094.stderr:2024-09-14T06:30:53.193+0000 7faf3269b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33649) UID: 0 2024-09-14T06:30:53.209 INFO:teuthology.orchestra.run.smithi138.stdout:{ 2024-09-14T06:30:53.209 INFO:teuthology.orchestra.run.smithi138.stdout: "size": 20, 2024-09-14T06:30:53.209 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 600, 2024-09-14T06:30:53.209 INFO:teuthology.orchestra.run.smithi138.stdout: "ops": [ 2024-09-14T06:30:53.210 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:53.210 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:98505 3.10 3.1bf56f90 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T06:30:53.210 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:30:07.045453+0000", 2024-09-14T06:30:53.210 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 46.162105554999997, 2024-09-14T06:30:53.210 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032144440000000003, 2024-09-14T06:30:53.210 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:30:53.210 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:30:53.210 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "reached pg", 2024-09-14T06:30:53.210 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:30:53.210 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:30:53.210 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:53.210 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 98505 2024-09-14T06:30:53.210 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:53.210 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:30:53.211 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:53.211 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:30:53.211 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045453+0000", 2024-09-14T06:30:53.211 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:53.211 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:53.211 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:53.211 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:30:53.211 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045453+0000", 2024-09-14T06:30:53.211 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:53.211 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:53.211 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:53.211 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:30:53.211 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045453+0000", 2024-09-14T06:30:53.212 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.36e-07 2024-09-14T06:30:53.212 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:53.212 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:53.212 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:30:53.212 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045454+0000", 2024-09-14T06:30:53.212 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.1299999999999998e-07 2024-09-14T06:30:53.212 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:53.212 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:53.212 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:30:53.212 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045457+0000", 2024-09-14T06:30:53.212 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.8559999999999998e-06 2024-09-14T06:30:53.213 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:53.213 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:53.213 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:30:53.213 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045460+0000", 2024-09-14T06:30:53.213 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.8750000000000002e-06 2024-09-14T06:30:53.213 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:53.213 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:53.213 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:30:53.213 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.077592+0000", 2024-09-14T06:30:53.213 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032131727999999998 2024-09-14T06:30:53.213 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:53.213 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:53.214 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:30:53.214 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.077597+0000", 2024-09-14T06:30:53.214 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.8319999999999996e-06 2024-09-14T06:30:53.214 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:53.214 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:30:53.214 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:53.214 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:53.214 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:53.214 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:98506 3.10 3.1bf56f90 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T06:30:53.214 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:30:07.045481+0000", 2024-09-14T06:30:53.214 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 46.1620773, 2024-09-14T06:30:53.214 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032142759999999999, 2024-09-14T06:30:53.214 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:30:53.214 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:30:53.215 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "reached pg", 2024-09-14T06:30:53.215 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:30:53.215 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:30:53.215 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:53.215 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 98506 2024-09-14T06:30:53.215 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:53.215 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:30:53.215 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:53.215 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:30:53.215 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045481+0000", 2024-09-14T06:30:53.215 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:53.215 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:53.215 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:53.216 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:30:53.216 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045481+0000", 2024-09-14T06:30:53.216 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:53.216 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:53.216 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:53.216 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:30:53.216 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045481+0000", 2024-09-14T06:30:53.216 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.8500000000000001e-07 2024-09-14T06:30:53.216 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:53.216 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:53.216 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:30:53.216 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045483+0000", 2024-09-14T06:30:53.216 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.8250000000000001e-06 2024-09-14T06:30:53.216 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:53.217 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:53.217 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:30:53.217 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045489+0000", 2024-09-14T06:30:53.217 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.5740000000000003e-06 2024-09-14T06:30:53.217 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:53.217 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:53.217 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:30:53.217 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045495+0000", 2024-09-14T06:30:53.217 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.7690000000000002e-06 2024-09-14T06:30:53.217 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:53.217 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:53.217 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:30:53.217 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.077619+0000", 2024-09-14T06:30:53.218 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032124621999999999 2024-09-14T06:30:53.218 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:53.218 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:53.218 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:30:53.218 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.077624+0000", 2024-09-14T06:30:53.218 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.3850000000000004e-06 2024-09-14T06:30:53.218 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:53.218 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:30:53.218 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:53.218 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:53.218 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:53.218 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:98507 3.10 3.1bf56f90 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T06:30:53.218 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:30:07.045502+0000", 2024-09-14T06:30:53.218 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 46.162055879, 2024-09-14T06:30:53.219 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032147219999999997, 2024-09-14T06:30:53.219 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:30:53.219 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:30:53.219 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "reached pg", 2024-09-14T06:30:53.219 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:30:53.219 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:30:53.219 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:53.219 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 98507 2024-09-14T06:30:53.219 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:53.219 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:30:53.219 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:53.219 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:30:53.219 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045502+0000", 2024-09-14T06:30:53.219 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:53.220 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:53.220 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:53.220 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:30:53.220 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045502+0000", 2024-09-14T06:30:53.220 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:53.220 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:53.220 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:53.220 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:30:53.220 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045503+0000", 2024-09-14T06:30:53.220 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.4700000000000002e-07 2024-09-14T06:30:53.220 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:53.220 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:53.220 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:30:53.221 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045509+0000", 2024-09-14T06:30:53.221 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.7329999999999997e-06 2024-09-14T06:30:53.221 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:53.221 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:53.221 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:30:53.221 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045515+0000", 2024-09-14T06:30:53.221 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.8030000000000002e-06 2024-09-14T06:30:53.221 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:53.221 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:53.221 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:30:53.221 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045522+0000", 2024-09-14T06:30:53.221 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.0369999999999997e-06 2024-09-14T06:30:53.221 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:53.222 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:53.222 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:30:53.222 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.077641+0000", 2024-09-14T06:30:53.222 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032119091000000002 2024-09-14T06:30:53.222 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:53.222 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:53.222 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:30:53.222 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.077650+0000", 2024-09-14T06:30:53.222 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.2090000000000005e-06 2024-09-14T06:30:53.222 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:53.222 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:30:53.222 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:53.222 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:53.222 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:53.223 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:98508 3.10 3.1bf56f90 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T06:30:53.223 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:30:07.045530+0000", 2024-09-14T06:30:53.223 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 46.162028262, 2024-09-14T06:30:53.223 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032198773999999999, 2024-09-14T06:30:53.223 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:30:53.223 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:30:53.223 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "reached pg", 2024-09-14T06:30:53.223 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:30:53.223 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:30:53.223 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:53.223 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 98508 2024-09-14T06:30:53.223 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:53.224 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:30:53.224 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:53.224 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:30:53.224 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045530+0000", 2024-09-14T06:30:53.224 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:53.224 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:53.224 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:53.224 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:30:53.224 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045530+0000", 2024-09-14T06:30:53.224 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:53.224 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:53.224 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:53.224 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:30:53.224 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045530+0000", 2024-09-14T06:30:53.225 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.3099999999999998e-07 2024-09-14T06:30:53.225 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:53.225 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:53.225 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:30:53.225 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045539+0000", 2024-09-14T06:30:53.225 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.4780000000000008e-06 2024-09-14T06:30:53.225 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:53.225 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:53.225 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:30:53.225 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045546+0000", 2024-09-14T06:30:53.225 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.3259999999999998e-06 2024-09-14T06:30:53.225 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:53.225 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:53.225 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:30:53.226 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045551+0000", 2024-09-14T06:30:53.226 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.9960000000000001e-06 2024-09-14T06:30:53.226 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:53.226 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:53.226 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:30:53.226 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.077716+0000", 2024-09-14T06:30:53.226 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032165081999999998 2024-09-14T06:30:53.226 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:53.226 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:53.226 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:30:53.226 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.077729+0000", 2024-09-14T06:30:53.226 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.2461e-05 2024-09-14T06:30:53.226 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:53.226 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:30:53.226 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:53.226 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:53.227 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:53.227 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:98509 3.10 3.1bf56f90 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T06:30:53.227 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:30:07.045558+0000", 2024-09-14T06:30:53.227 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 46.162000061000001, 2024-09-14T06:30:53.227 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032242198, 2024-09-14T06:30:53.227 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:30:53.227 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:30:53.227 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "reached pg", 2024-09-14T06:30:53.227 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:30:53.227 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:30:53.227 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:53.227 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 98509 2024-09-14T06:30:53.227 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:53.227 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:30:53.228 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:53.228 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:30:53.228 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045558+0000", 2024-09-14T06:30:53.228 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:53.228 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:53.228 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:53.228 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:30:53.228 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045558+0000", 2024-09-14T06:30:53.228 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:53.228 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:53.228 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:53.228 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:30:53.228 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045559+0000", 2024-09-14T06:30:53.229 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.27e-07 2024-09-14T06:30:53.229 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:53.229 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:53.229 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:30:53.229 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045560+0000", 2024-09-14T06:30:53.229 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.6959999999999999e-06 2024-09-14T06:30:53.229 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:53.229 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:53.229 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:30:53.229 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045567+0000", 2024-09-14T06:30:53.229 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.6900000000000003e-06 2024-09-14T06:30:53.229 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:53.229 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:53.229 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:30:53.230 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045571+0000", 2024-09-14T06:30:53.230 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.5750000000000002e-06 2024-09-14T06:30:53.230 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:53.230 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:53.230 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:30:53.230 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.077789+0000", 2024-09-14T06:30:53.230 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032217186000000002 2024-09-14T06:30:53.230 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:53.230 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:53.230 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:30:53.230 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.077800+0000", 2024-09-14T06:30:53.230 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.1623999999999999e-05 2024-09-14T06:30:53.230 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:53.231 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:30:53.231 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:53.231 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:53.231 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:53.231 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:98510 3.10 3.1bf56f90 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T06:30:53.231 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:30:07.045580+0000", 2024-09-14T06:30:53.231 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 46.161978247, 2024-09-14T06:30:53.231 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032299845000000001, 2024-09-14T06:30:53.231 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:30:53.231 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:30:53.231 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "reached pg", 2024-09-14T06:30:53.231 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:30:53.231 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:30:53.231 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:53.232 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 98510 2024-09-14T06:30:53.232 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:53.232 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:30:53.232 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:53.232 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:30:53.232 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045580+0000", 2024-09-14T06:30:53.232 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:53.232 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:53.232 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:53.232 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:30:53.232 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045580+0000", 2024-09-14T06:30:53.232 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:53.232 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:53.233 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:53.233 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:30:53.233 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045580+0000", 2024-09-14T06:30:53.233 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.0400000000000002e-07 2024-09-14T06:30:53.233 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:53.233 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:53.233 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:30:53.233 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045582+0000", 2024-09-14T06:30:53.233 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.7600000000000001e-06 2024-09-14T06:30:53.233 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:53.233 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:53.233 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:30:53.233 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045588+0000", 2024-09-14T06:30:53.233 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.5650000000000001e-06 2024-09-14T06:30:53.234 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:53.234 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:53.234 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:30:53.234 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045597+0000", 2024-09-14T06:30:53.234 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.465e-06 2024-09-14T06:30:53.234 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:53.234 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:53.234 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:30:53.234 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.077871+0000", 2024-09-14T06:30:53.234 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032273562999999998 2024-09-14T06:30:53.234 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:53.234 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:53.234 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:30:53.234 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.077880+0000", 2024-09-14T06:30:53.234 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.0880000000000001e-06 2024-09-14T06:30:53.234 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:53.235 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:30:53.235 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:53.235 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:53.235 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:53.235 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:98511 3.10 3.1bf56f90 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T06:30:53.235 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:30:07.045610+0000", 2024-09-14T06:30:53.235 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 46.1619484, 2024-09-14T06:30:53.235 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032338894, 2024-09-14T06:30:53.235 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:30:53.235 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:30:53.235 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "reached pg", 2024-09-14T06:30:53.235 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:30:53.235 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:30:53.235 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:53.235 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 98511 2024-09-14T06:30:53.236 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:53.236 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:30:53.236 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:53.236 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:30:53.236 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045610+0000", 2024-09-14T06:30:53.236 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:53.236 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:53.236 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:53.236 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:30:53.236 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045610+0000", 2024-09-14T06:30:53.236 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:53.236 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:53.236 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:53.236 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:30:53.236 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045610+0000", 2024-09-14T06:30:53.237 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.1600000000000002e-07 2024-09-14T06:30:53.237 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:53.237 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:53.237 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:30:53.237 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045612+0000", 2024-09-14T06:30:53.237 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.1150000000000001e-06 2024-09-14T06:30:53.237 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:53.237 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:53.237 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:30:53.237 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045618+0000", 2024-09-14T06:30:53.237 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.0499999999999997e-06 2024-09-14T06:30:53.237 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:53.237 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:53.238 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:30:53.238 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045623+0000", 2024-09-14T06:30:53.238 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.6770000000000003e-06 2024-09-14T06:30:53.238 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:53.238 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:53.238 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:30:53.238 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.077928+0000", 2024-09-14T06:30:53.238 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032305111999999997 2024-09-14T06:30:53.238 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:53.238 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:53.238 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:30:53.238 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.077949+0000", 2024-09-14T06:30:53.238 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.0523999999999999e-05 2024-09-14T06:30:53.238 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:53.239 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:30:53.239 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:53.239 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:53.239 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:53.239 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:98512 3.10 3.1bf56f90 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T06:30:53.239 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:30:07.045630+0000", 2024-09-14T06:30:53.239 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 46.161928394999997, 2024-09-14T06:30:53.239 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032346632, 2024-09-14T06:30:53.239 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:30:53.239 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:30:53.239 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "reached pg", 2024-09-14T06:30:53.239 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:30:53.239 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:30:53.239 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:53.240 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 98512 2024-09-14T06:30:53.240 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:53.240 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:30:53.240 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:53.240 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:30:53.240 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045630+0000", 2024-09-14T06:30:53.240 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:53.240 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:53.240 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:53.240 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:30:53.240 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045630+0000", 2024-09-14T06:30:53.240 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:53.240 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:53.240 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:53.240 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:30:53.241 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045630+0000", 2024-09-14T06:30:53.241 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.8200000000000001e-07 2024-09-14T06:30:53.241 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:53.241 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:53.241 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:30:53.241 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045632+0000", 2024-09-14T06:30:53.241 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.7430000000000001e-06 2024-09-14T06:30:53.241 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:53.241 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:53.241 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:30:53.241 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045637+0000", 2024-09-14T06:30:53.241 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.4550000000000003e-06 2024-09-14T06:30:53.241 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:53.241 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:53.241 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:30:53.242 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045644+0000", 2024-09-14T06:30:53.242 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.1330000000000001e-06 2024-09-14T06:30:53.242 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:53.242 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:53.242 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:30:53.242 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.077968+0000", 2024-09-14T06:30:53.242 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032323401000000002 2024-09-14T06:30:53.242 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:53.242 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:53.242 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:30:53.242 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.077976+0000", 2024-09-14T06:30:53.242 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.5180000000000004e-06 2024-09-14T06:30:53.242 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:53.242 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:30:53.242 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:53.243 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:53.243 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:53.243 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:98513 3.10 3.1bf56f90 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T06:30:53.243 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:30:07.045652+0000", 2024-09-14T06:30:53.243 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 46.161905830999999, 2024-09-14T06:30:53.243 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032353436999999999, 2024-09-14T06:30:53.243 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:30:53.243 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:30:53.243 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "reached pg", 2024-09-14T06:30:53.243 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:30:53.243 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:30:53.243 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:53.243 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 98513 2024-09-14T06:30:53.243 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:53.243 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:30:53.243 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:53.244 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:30:53.244 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045652+0000", 2024-09-14T06:30:53.244 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:53.244 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:53.244 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:53.244 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:30:53.244 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045652+0000", 2024-09-14T06:30:53.244 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:53.244 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:53.244 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:53.244 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:30:53.244 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045653+0000", 2024-09-14T06:30:53.244 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.3700000000000001e-07 2024-09-14T06:30:53.244 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:53.245 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:53.245 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:30:53.245 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045655+0000", 2024-09-14T06:30:53.245 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.854e-06 2024-09-14T06:30:53.245 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:53.245 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:53.245 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:30:53.245 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045661+0000", 2024-09-14T06:30:53.245 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.5509999999999996e-06 2024-09-14T06:30:53.245 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:53.245 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:53.245 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:30:53.245 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045668+0000", 2024-09-14T06:30:53.245 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.1980000000000004e-06 2024-09-14T06:30:53.246 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:53.246 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:53.246 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:30:53.246 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.077997+0000", 2024-09-14T06:30:53.246 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032328575999999998 2024-09-14T06:30:53.246 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:53.246 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:53.246 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:30:53.246 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.078006+0000", 2024-09-14T06:30:53.246 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.8210000000000008e-06 2024-09-14T06:30:53.246 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:53.246 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:30:53.246 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:53.246 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:53.247 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:53.247 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:98514 3.10 3.1bf56f90 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T06:30:53.247 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:30:07.045699+0000", 2024-09-14T06:30:53.247 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 46.161859495999998, 2024-09-14T06:30:53.247 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032330223999999998, 2024-09-14T06:30:53.247 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:30:53.247 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:30:53.247 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "reached pg", 2024-09-14T06:30:53.247 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:30:53.247 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:30:53.247 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:53.247 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 98514 2024-09-14T06:30:53.247 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:53.247 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:30:53.247 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:53.248 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:30:53.248 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045699+0000", 2024-09-14T06:30:53.248 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:53.248 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:53.248 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:53.248 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:30:53.248 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045699+0000", 2024-09-14T06:30:53.248 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:53.248 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:53.248 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:53.248 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:30:53.248 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045699+0000", 2024-09-14T06:30:53.248 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.3200000000000001e-07 2024-09-14T06:30:53.249 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:53.249 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:53.249 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:30:53.249 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045701+0000", 2024-09-14T06:30:53.249 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.762e-06 2024-09-14T06:30:53.249 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:53.249 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:53.249 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:30:53.249 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045705+0000", 2024-09-14T06:30:53.249 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.2359999999999998e-06 2024-09-14T06:30:53.249 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:53.249 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:53.249 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:30:53.249 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045708+0000", 2024-09-14T06:30:53.250 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.796e-06 2024-09-14T06:30:53.250 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:53.250 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:53.250 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:30:53.250 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.078023+0000", 2024-09-14T06:30:53.250 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032315213000000002 2024-09-14T06:30:53.250 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:53.250 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:53.250 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:30:53.250 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.078029+0000", 2024-09-14T06:30:53.250 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.8850000000000004e-06 2024-09-14T06:30:53.250 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:53.250 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:30:53.250 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:53.251 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:53.251 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:53.251 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:98515 3.10 3.1bf56f90 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T06:30:53.251 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:30:07.045712+0000", 2024-09-14T06:30:53.251 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 46.161846185000002, 2024-09-14T06:30:53.251 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032346871999999999, 2024-09-14T06:30:53.251 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:30:53.251 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:30:53.251 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "reached pg", 2024-09-14T06:30:53.251 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:30:53.251 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:30:53.251 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:53.251 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 98515 2024-09-14T06:30:53.251 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:53.252 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:30:53.252 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:53.252 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:30:53.252 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045712+0000", 2024-09-14T06:30:53.252 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:53.252 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:53.252 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:53.252 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:30:53.252 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045712+0000", 2024-09-14T06:30:53.252 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:53.252 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:53.252 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:53.252 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:30:53.252 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045712+0000", 2024-09-14T06:30:53.253 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.2100000000000001e-07 2024-09-14T06:30:53.253 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:53.253 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:53.253 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:30:53.253 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045713+0000", 2024-09-14T06:30:53.253 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.068e-06 2024-09-14T06:30:53.253 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:53.253 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:53.253 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:30:53.253 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045716+0000", 2024-09-14T06:30:53.253 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.971e-06 2024-09-14T06:30:53.253 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:53.253 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:53.254 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:30:53.254 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045719+0000", 2024-09-14T06:30:53.254 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.3879999999999998e-06 2024-09-14T06:30:53.254 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:53.254 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:53.254 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:30:53.254 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.078049+0000", 2024-09-14T06:30:53.254 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032330553999999997 2024-09-14T06:30:53.254 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:53.254 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:53.254 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:30:53.255 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.078059+0000", 2024-09-14T06:30:53.255 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.6700000000000006e-06 2024-09-14T06:30:53.255 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:53.255 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:30:53.255 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:53.255 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:53.255 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:53.255 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:98516 3.10 3.1bf56f90 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T06:30:53.255 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:30:07.045722+0000", 2024-09-14T06:30:53.255 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 46.161836432000001, 2024-09-14T06:30:53.255 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032374334999999997, 2024-09-14T06:30:53.255 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:30:53.255 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:30:53.256 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "reached pg", 2024-09-14T06:30:53.256 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:30:53.256 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:30:53.256 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:53.256 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 98516 2024-09-14T06:30:53.256 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:53.256 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:30:53.256 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:53.256 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:30:53.256 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045722+0000", 2024-09-14T06:30:53.256 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:53.256 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:53.256 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:53.256 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:30:53.257 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045722+0000", 2024-09-14T06:30:53.257 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:53.257 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:53.257 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:53.257 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:30:53.257 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045722+0000", 2024-09-14T06:30:53.257 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.8999999999999998e-07 2024-09-14T06:30:53.257 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:53.257 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:53.257 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:30:53.257 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045723+0000", 2024-09-14T06:30:53.257 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.0049999999999999e-06 2024-09-14T06:30:53.257 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:53.258 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:53.258 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:30:53.258 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045726+0000", 2024-09-14T06:30:53.258 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.9019999999999999e-06 2024-09-14T06:30:53.258 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:53.258 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:53.258 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:30:53.258 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045728+0000", 2024-09-14T06:30:53.258 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.3290000000000001e-06 2024-09-14T06:30:53.258 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:53.258 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:53.258 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:30:53.258 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.078090+0000", 2024-09-14T06:30:53.258 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032362123 2024-09-14T06:30:53.259 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:53.259 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:53.259 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:30:53.259 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.078096+0000", 2024-09-14T06:30:53.259 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.6860000000000003e-06 2024-09-14T06:30:53.259 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:53.259 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:30:53.259 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:53.259 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:53.259 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:53.259 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:98517 3.10 3.1bf56f90 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T06:30:53.259 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:30:07.045731+0000", 2024-09-14T06:30:53.259 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 46.161826701000003, 2024-09-14T06:30:53.259 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.03241227, 2024-09-14T06:30:53.259 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:30:53.260 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:30:53.260 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "reached pg", 2024-09-14T06:30:53.260 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:30:53.260 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:30:53.260 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:53.260 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 98517 2024-09-14T06:30:53.260 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:53.260 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:30:53.260 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:53.260 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:30:53.260 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045731+0000", 2024-09-14T06:30:53.260 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:53.260 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:53.260 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:53.261 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:30:53.261 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045731+0000", 2024-09-14T06:30:53.261 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:53.261 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:53.261 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:53.261 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:30:53.261 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045732+0000", 2024-09-14T06:30:53.261 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.0699999999999999e-07 2024-09-14T06:30:53.261 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:53.261 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:53.261 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:30:53.261 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045733+0000", 2024-09-14T06:30:53.261 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.9199999999999999e-07 2024-09-14T06:30:53.262 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:53.262 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:53.262 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:30:53.262 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045735+0000", 2024-09-14T06:30:53.262 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.8150000000000002e-06 2024-09-14T06:30:53.262 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:53.262 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:53.262 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:30:53.262 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045738+0000", 2024-09-14T06:30:53.262 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.3640000000000002e-06 2024-09-14T06:30:53.262 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:53.262 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:53.262 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:30:53.263 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.078126+0000", 2024-09-14T06:30:53.263 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032388206000000003 2024-09-14T06:30:53.263 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:53.263 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:53.263 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:30:53.263 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.078144+0000", 2024-09-14T06:30:53.263 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.7686e-05 2024-09-14T06:30:53.263 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:53.263 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:30:53.263 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:53.263 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:53.263 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:53.263 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:98518 3.10 3.1bf56f90 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T06:30:53.263 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:30:07.045741+0000", 2024-09-14T06:30:53.263 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 46.161817306000003, 2024-09-14T06:30:53.264 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032451793999999999, 2024-09-14T06:30:53.264 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:30:53.264 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:30:53.264 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "reached pg", 2024-09-14T06:30:53.264 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:30:53.264 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:30:53.264 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:53.264 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 98518 2024-09-14T06:30:53.264 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:53.264 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:30:53.264 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:53.264 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:30:53.264 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045741+0000", 2024-09-14T06:30:53.264 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:53.265 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:53.265 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:53.265 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:30:53.265 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045741+0000", 2024-09-14T06:30:53.265 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:53.265 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:53.265 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:53.265 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:30:53.265 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045741+0000", 2024-09-14T06:30:53.265 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.05e-07 2024-09-14T06:30:53.265 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:53.265 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:53.265 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:30:53.265 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045746+0000", 2024-09-14T06:30:53.266 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.2259999999999996e-06 2024-09-14T06:30:53.266 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:53.266 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:53.266 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:30:53.266 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045750+0000", 2024-09-14T06:30:53.266 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.2490000000000001e-06 2024-09-14T06:30:53.266 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:53.266 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:53.266 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:30:53.266 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045752+0000", 2024-09-14T06:30:53.266 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.3980000000000002e-06 2024-09-14T06:30:53.266 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:53.266 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:53.266 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:30:53.267 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.078184+0000", 2024-09-14T06:30:53.267 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032431593000000002 2024-09-14T06:30:53.267 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:53.267 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:53.267 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:30:53.267 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.078193+0000", 2024-09-14T06:30:53.267 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.1230000000000006e-06 2024-09-14T06:30:53.267 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:53.267 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:30:53.267 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:53.267 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:53.267 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:53.267 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:98519 3.10 3.1bf56f90 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T06:30:53.267 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:30:07.045755+0000", 2024-09-14T06:30:53.268 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 46.161802881, 2024-09-14T06:30:53.268 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032489431999999999, 2024-09-14T06:30:53.268 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:30:53.268 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:30:53.268 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "reached pg", 2024-09-14T06:30:53.268 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:30:53.268 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:30:53.268 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:53.268 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 98519 2024-09-14T06:30:53.268 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:53.268 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:30:53.268 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:53.268 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:30:53.269 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045755+0000", 2024-09-14T06:30:53.269 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:53.269 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:53.269 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:53.269 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:30:53.269 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045755+0000", 2024-09-14T06:30:53.269 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:53.269 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:53.269 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:53.269 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:30:53.269 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045756+0000", 2024-09-14T06:30:53.269 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.23e-07 2024-09-14T06:30:53.269 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:53.269 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:53.269 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:30:53.270 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045757+0000", 2024-09-14T06:30:53.270 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.1060000000000001e-06 2024-09-14T06:30:53.270 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:53.270 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:53.270 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:30:53.270 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045760+0000", 2024-09-14T06:30:53.270 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.988e-06 2024-09-14T06:30:53.270 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:53.270 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:53.270 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:30:53.270 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045765+0000", 2024-09-14T06:30:53.270 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.2689999999999999e-06 2024-09-14T06:30:53.270 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:53.270 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:53.271 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:30:53.271 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.078234+0000", 2024-09-14T06:30:53.271 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032468915000000001 2024-09-14T06:30:53.271 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:53.271 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:53.271 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:30:53.271 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.078245+0000", 2024-09-14T06:30:53.271 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.0930999999999999e-05 2024-09-14T06:30:53.271 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:53.271 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:30:53.271 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:53.271 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:53.271 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:53.271 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:98520 3.10 3.1bf56f90 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T06:30:53.272 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:30:07.045768+0000", 2024-09-14T06:30:53.272 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 46.161790003, 2024-09-14T06:30:53.272 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032531379999999999, 2024-09-14T06:30:53.272 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:30:53.272 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:30:53.272 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "reached pg", 2024-09-14T06:30:53.272 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:30:53.272 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:30:53.272 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:53.272 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 98520 2024-09-14T06:30:53.272 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:53.272 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:30:53.272 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:53.272 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:30:53.272 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045768+0000", 2024-09-14T06:30:53.273 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:53.273 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:53.273 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:53.273 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:30:53.273 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045768+0000", 2024-09-14T06:30:53.273 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:53.273 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:53.273 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:53.273 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:30:53.273 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045768+0000", 2024-09-14T06:30:53.273 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.22e-07 2024-09-14T06:30:53.273 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:53.273 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:53.273 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:30:53.274 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045773+0000", 2024-09-14T06:30:53.274 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.3470000000000001e-06 2024-09-14T06:30:53.274 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:53.274 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:53.274 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:30:53.274 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045776+0000", 2024-09-14T06:30:53.274 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.1319999999999998e-06 2024-09-14T06:30:53.274 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:53.274 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:53.274 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:30:53.274 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045779+0000", 2024-09-14T06:30:53.274 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.9749999999999999e-06 2024-09-14T06:30:53.274 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:53.274 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:53.275 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:30:53.275 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.078289+0000", 2024-09-14T06:30:53.275 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032509693999999999 2024-09-14T06:30:53.275 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:53.275 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:53.275 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:30:53.275 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.078300+0000", 2024-09-14T06:30:53.275 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.101e-05 2024-09-14T06:30:53.275 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:53.275 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:30:53.275 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:53.275 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:53.275 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:53.275 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:98521 3.10 3.1bf56f90 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T06:30:53.275 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:30:07.045782+0000", 2024-09-14T06:30:53.276 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 46.161775822999999, 2024-09-14T06:30:53.276 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032566109000000003, 2024-09-14T06:30:53.276 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:30:53.276 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:30:53.276 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "reached pg", 2024-09-14T06:30:53.276 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:30:53.276 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:30:53.276 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:53.276 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 98521 2024-09-14T06:30:53.276 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:53.276 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:30:53.276 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:53.276 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:30:53.276 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045782+0000", 2024-09-14T06:30:53.276 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:53.277 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:53.277 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:53.277 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:30:53.277 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045782+0000", 2024-09-14T06:30:53.277 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:53.277 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:53.277 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:53.277 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:30:53.277 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045783+0000", 2024-09-14T06:30:53.277 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.2499999999999999e-07 2024-09-14T06:30:53.277 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:53.277 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:53.277 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:30:53.277 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045784+0000", 2024-09-14T06:30:53.278 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.0109999999999999e-06 2024-09-14T06:30:53.278 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:53.278 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:53.278 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:30:53.278 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045786+0000", 2024-09-14T06:30:53.278 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.8289999999999999e-06 2024-09-14T06:30:53.278 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:53.278 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:53.278 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:30:53.278 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045793+0000", 2024-09-14T06:30:53.278 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.579e-06 2024-09-14T06:30:53.278 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:53.278 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:53.278 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:30:53.278 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.078338+0000", 2024-09-14T06:30:53.279 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032545389000000001 2024-09-14T06:30:53.279 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:53.279 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:53.279 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:30:53.279 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.078348+0000", 2024-09-14T06:30:53.279 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.0076e-05 2024-09-14T06:30:53.279 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:53.279 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:30:53.279 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:53.279 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:53.279 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:53.279 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:98522 3.10 3.1bf56f90 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T06:30:53.279 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:30:07.045796+0000", 2024-09-14T06:30:53.280 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 46.161761915, 2024-09-14T06:30:53.280 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032577846000000001, 2024-09-14T06:30:53.280 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:30:53.280 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:30:53.280 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "reached pg", 2024-09-14T06:30:53.280 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:30:53.280 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:30:53.280 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:53.280 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 98522 2024-09-14T06:30:53.280 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:53.280 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:30:53.280 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:53.280 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:30:53.280 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045796+0000", 2024-09-14T06:30:53.281 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:53.281 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:53.281 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:53.281 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:30:53.281 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045796+0000", 2024-09-14T06:30:53.281 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:53.281 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:53.281 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:53.281 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:30:53.281 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045796+0000", 2024-09-14T06:30:53.281 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.2100000000000001e-07 2024-09-14T06:30:53.281 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:53.281 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:53.282 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:30:53.282 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045797+0000", 2024-09-14T06:30:53.282 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.6899999999999996e-07 2024-09-14T06:30:53.282 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:53.282 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:53.282 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:30:53.282 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045800+0000", 2024-09-14T06:30:53.282 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.9979999999999999e-06 2024-09-14T06:30:53.282 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:53.282 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:53.282 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:30:53.282 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045803+0000", 2024-09-14T06:30:53.282 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.2850000000000001e-06 2024-09-14T06:30:53.282 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:53.283 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:53.283 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:30:53.283 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.078366+0000", 2024-09-14T06:30:53.283 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032562932000000003 2024-09-14T06:30:53.283 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:53.283 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:53.283 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:30:53.283 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.078374+0000", 2024-09-14T06:30:53.283 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.4409999999999993e-06 2024-09-14T06:30:53.283 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:53.283 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:30:53.283 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:53.283 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:53.284 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:53.284 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:98523 3.10 3.1bf56f90 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T06:30:53.284 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:30:07.045806+0000", 2024-09-14T06:30:53.284 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 46.161752372000002, 2024-09-14T06:30:53.284 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032611167000000003, 2024-09-14T06:30:53.284 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:30:53.284 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:30:53.284 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "reached pg", 2024-09-14T06:30:53.284 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:30:53.284 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:30:53.284 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:53.284 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 98523 2024-09-14T06:30:53.284 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:53.284 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:30:53.285 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:53.285 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:30:53.285 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045806+0000", 2024-09-14T06:30:53.285 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:53.285 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:53.285 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:53.285 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:30:53.285 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045806+0000", 2024-09-14T06:30:53.285 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:53.285 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:53.285 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:53.285 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:30:53.285 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045806+0000", 2024-09-14T06:30:53.286 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.2000000000000001e-07 2024-09-14T06:30:53.286 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:53.286 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:53.286 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:30:53.286 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045807+0000", 2024-09-14T06:30:53.286 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.4e-07 2024-09-14T06:30:53.286 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:53.286 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:53.286 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:30:53.286 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045810+0000", 2024-09-14T06:30:53.286 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.9229999999999998e-06 2024-09-14T06:30:53.286 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:53.286 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:53.286 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:30:53.286 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045812+0000", 2024-09-14T06:30:53.287 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.2220000000000001e-06 2024-09-14T06:30:53.287 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:53.287 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:53.287 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:30:53.287 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.078407+0000", 2024-09-14T06:30:53.287 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032594513999999998 2024-09-14T06:30:53.287 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:53.287 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:53.287 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:30:53.287 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.078417+0000", 2024-09-14T06:30:53.287 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.0348e-05 2024-09-14T06:30:53.287 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:53.287 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:30:53.287 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:53.287 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:53.288 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:53.288 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:98524 3.10 3.1bf56f90 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T06:30:53.288 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:30:07.045815+0000", 2024-09-14T06:30:53.288 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 46.161743004000002, 2024-09-14T06:30:53.288 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032643208999999999, 2024-09-14T06:30:53.288 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:30:53.288 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:30:53.288 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "reached pg", 2024-09-14T06:30:53.288 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:30:53.288 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:30:53.288 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:53.288 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 98524 2024-09-14T06:30:53.288 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:53.288 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:30:53.289 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:53.289 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:30:53.289 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045815+0000", 2024-09-14T06:30:53.289 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:53.289 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:53.289 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:53.289 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:30:53.289 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045815+0000", 2024-09-14T06:30:53.289 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:53.289 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:53.289 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:53.289 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:30:53.289 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045815+0000", 2024-09-14T06:30:53.289 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.11e-07 2024-09-14T06:30:53.290 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:53.290 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:53.290 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:30:53.290 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045816+0000", 2024-09-14T06:30:53.290 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.9099999999999991e-07 2024-09-14T06:30:53.290 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:53.290 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:53.290 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:30:53.290 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045819+0000", 2024-09-14T06:30:53.290 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.9399999999999998e-06 2024-09-14T06:30:53.290 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:53.290 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:53.290 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:30:53.291 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045822+0000", 2024-09-14T06:30:53.291 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.3470000000000001e-06 2024-09-14T06:30:53.291 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:53.291 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:53.291 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:30:53.291 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.078449+0000", 2024-09-14T06:30:53.291 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032627529000000002 2024-09-14T06:30:53.291 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:53.291 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:53.291 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:30:53.291 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.078458+0000", 2024-09-14T06:30:53.291 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.1910000000000006e-06 2024-09-14T06:30:53.291 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:53.291 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:30:53.291 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:53.291 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:53.292 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:30:53.292 INFO:teuthology.orchestra.run.smithi138.stdout:} 2024-09-14T06:30:53.294 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-14T06:30:53.295 INFO:teuthology.orchestra.run.smithi094.stdout:3811 write 0xdd706c9 thru 0xdd71601 (0xf39 bytes) 2024-09-14T06:30:53.295 INFO:teuthology.orchestra.run.smithi094.stdout:3812 write 0x1a37ca4 thru 0x1a466dc (0xea39 bytes) 2024-09-14T06:30:53.295 INFO:teuthology.orchestra.run.smithi094.stdout:3813 read 0x199abe9 thru 0x19a0099 (0x54b1 bytes) 2024-09-14T06:30:53.295 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-14T06:30:53.295 INFO:teuthology.orchestra.run.smithi094.stdout:3816 writesame 0x63d53bf thru 0x63dcd3f data_size 0x1(0x7981 bytes) 2024-09-14T06:30:53.295 INFO:teuthology.orchestra.run.smithi094.stdout:3817 read 0x1f13bfc thru 0x1f1850c (0x4911 bytes) 2024-09-14T06:30:53.295 INFO:teuthology.orchestra.run.smithi094.stdout:3818 writesame 0xcf77f28 thru 0xcf79c52 data_size 0x3(0x1d2b bytes) 2024-09-14T06:30:53.295 INFO:teuthology.orchestra.run.smithi094.stdout:3820 trunc from 0xdd71602 to 0xcaba3f7 2024-09-14T06:30:53.295 INFO:teuthology.orchestra.run.smithi094.stdout:3821 write 0x650a265 thru 0x6513287 (0x9023 bytes) 2024-09-14T06:30:53.295 INFO:teuthology.orchestra.run.smithi094.stdout:3822 punch from 0xb8e7077 to 0xb8ede75, (0x6dfe bytes) 2024-09-14T06:30:53.295 INFO:teuthology.orchestra.run.smithi094.stdout:3823 trunc from 0xcaba3f7 to 0x296193e 2024-09-14T06:30:53.295 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-14T06:30:53.296 INFO:teuthology.orchestra.run.smithi094.stdout:3826 read 0xf27970 thru 0xf28221 (0x8b2 bytes) 2024-09-14T06:30:53.296 INFO:teuthology.orchestra.run.smithi094.stdout:3827 punch from 0x1c7b157 to 0x1c80449, (0x52f2 bytes) 2024-09-14T06:30:53.297 INFO:teuthology.orchestra.run.smithi094.stdout:3828 writesame 0x856a81d thru 0x8577b4e data_size 0x3(0xd332 bytes) 2024-09-14T06:30:53.297 INFO:teuthology.orchestra.run.smithi094.stdout:3829 clone 30 order 24 su 131072 sc 8 2024-09-14T06:30:53.297 INFO:tasks.ceph.osd.0.smithi094.stderr:2024-09-14T06:30:53.296+0000 7f07c9333640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33636) UID: 0 2024-09-14T06:30:53.397 INFO:tasks.ceph.osd.7.smithi138.stderr:2024-09-14T06:30:53.396+0000 7f3c6a21d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33623) UID: 0 2024-09-14T06:30:53.398 INFO:teuthology.orchestra.run.smithi138.stdout:{ 2024-09-14T06:30:53.398 INFO:teuthology.orchestra.run.smithi138.stdout: "ops": [], 2024-09-14T06:30:53.399 INFO:teuthology.orchestra.run.smithi138.stdout: "num_ops": 0 2024-09-14T06:30:53.399 INFO:teuthology.orchestra.run.smithi138.stdout:} 2024-09-14T06:30:53.407 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-14T06:30:53.498 INFO:tasks.ceph.osd.3.smithi094.stderr:2024-09-14T06:30:53.497+0000 7feeeafe8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33655) UID: 0 2024-09-14T06:30:53.551 INFO:teuthology.orchestra.run.smithi138.stdout:{ 2024-09-14T06:30:53.551 INFO:teuthology.orchestra.run.smithi138.stdout: "ops": [], 2024-09-14T06:30:53.551 INFO:teuthology.orchestra.run.smithi138.stdout: "complaint_time": 30, 2024-09-14T06:30:53.551 INFO:teuthology.orchestra.run.smithi138.stdout: "num_blocked_ops": 0 2024-09-14T06:30:53.551 INFO:teuthology.orchestra.run.smithi138.stdout:} 2024-09-14T06:30:53.559 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-14T06:30:53.598 INFO:tasks.ceph.osd.2.smithi094.stderr:2024-09-14T06:30:53.597+0000 7faf3269b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33649) UID: 0 2024-09-14T06:30:53.605 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-14T06:30:53.663 INFO:teuthology.orchestra.run.smithi138.stdout:{ 2024-09-14T06:30:53.663 INFO:teuthology.orchestra.run.smithi138.stdout: "size": 20, 2024-09-14T06:30:53.663 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 600, 2024-09-14T06:30:53.664 INFO:teuthology.orchestra.run.smithi138.stdout: "ops": [ 2024-09-14T06:30:53.664 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:53.664 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:53244 3.f 3:f71f51a7:::rbd_data.112dca2f6ae1.0000000000000015:17 [read 1863680~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:30:53.664 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:44.369399+0000", 2024-09-14T06:30:53.664 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 69.292245707000006, 2024-09-14T06:30:53.664 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047291189999999997, 2024-09-14T06:30:53.664 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:30:53.664 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:30:53.664 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:30:53.664 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:30:53.664 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:30:53.664 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:53.664 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 53244 2024-09-14T06:30:53.665 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:53.665 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:30:53.665 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:53.665 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:30:53.665 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369399+0000", 2024-09-14T06:30:53.665 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:53.665 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:53.665 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:53.665 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:30:53.665 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369399+0000", 2024-09-14T06:30:53.665 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:53.665 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:53.665 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:53.665 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:30:53.666 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369400+0000", 2024-09-14T06:30:53.666 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.0699999999999999e-07 2024-09-14T06:30:53.666 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:53.666 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:53.666 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:30:53.666 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369401+0000", 2024-09-14T06:30:53.666 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.4300000000000001e-06 2024-09-14T06:30:53.666 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:53.666 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:53.666 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:30:53.666 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369404+0000", 2024-09-14T06:30:53.666 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.061e-06 2024-09-14T06:30:53.666 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:53.666 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:53.666 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:30:53.666 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369407+0000", 2024-09-14T06:30:53.667 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.6139999999999999e-06 2024-09-14T06:30:53.667 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:53.667 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:53.667 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:30:53.667 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.416517+0000", 2024-09-14T06:30:53.667 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047110338000000002 2024-09-14T06:30:53.667 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:53.667 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:53.667 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:30:53.667 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.416616+0000", 2024-09-14T06:30:53.667 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.8651000000000001e-05 2024-09-14T06:30:53.667 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:53.667 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:53.667 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:30:53.667 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.416691+0000", 2024-09-14T06:30:53.667 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.4888999999999994e-05 2024-09-14T06:30:53.668 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:53.668 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:30:53.668 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:53.668 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:53.668 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:53.668 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:53245 3.f 3:f71f51a7:::rbd_data.112dca2f6ae1.0000000000000015:17 [read 1888256~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:30:53.668 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:44.369410+0000", 2024-09-14T06:30:53.668 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 69.292235418999994, 2024-09-14T06:30:53.668 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047399275999999997, 2024-09-14T06:30:53.668 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:30:53.668 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:30:53.668 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:30:53.668 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:30:53.668 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:30:53.668 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:53.669 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 53245 2024-09-14T06:30:53.669 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:53.669 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:30:53.669 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:53.669 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:30:53.669 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369410+0000", 2024-09-14T06:30:53.669 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:53.669 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:53.669 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:53.669 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:30:53.669 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369410+0000", 2024-09-14T06:30:53.669 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:53.669 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:53.669 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:53.670 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:30:53.670 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369410+0000", 2024-09-14T06:30:53.670 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.1899999999999999e-07 2024-09-14T06:30:53.670 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:53.670 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:53.670 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:30:53.670 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369411+0000", 2024-09-14T06:30:53.670 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.0300000000000001e-06 2024-09-14T06:30:53.670 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:53.670 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:53.670 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:30:53.670 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369414+0000", 2024-09-14T06:30:53.670 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.1099999999999999e-06 2024-09-14T06:30:53.670 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:53.671 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:53.671 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:30:53.671 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369417+0000", 2024-09-14T06:30:53.671 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.6189999999999998e-06 2024-09-14T06:30:53.671 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:53.671 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:53.671 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:30:53.671 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.416718+0000", 2024-09-14T06:30:53.671 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047301322999999999 2024-09-14T06:30:53.671 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:53.671 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:53.671 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:30:53.671 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.416759+0000", 2024-09-14T06:30:53.671 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.1009e-05 2024-09-14T06:30:53.671 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:53.672 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:53.672 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:30:53.672 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.416809+0000", 2024-09-14T06:30:53.672 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.9966000000000002e-05 2024-09-14T06:30:53.672 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:53.672 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:30:53.672 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:53.672 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:53.672 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:53.672 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:53246 3.f 3:f71f51a7:::rbd_data.112dca2f6ae1.0000000000000015:17 [read 1912832~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:30:53.672 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:44.369420+0000", 2024-09-14T06:30:53.672 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 69.292225474999995, 2024-09-14T06:30:53.672 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047529463000000001, 2024-09-14T06:30:53.672 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:30:53.673 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:30:53.673 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:30:53.673 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:30:53.673 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:30:53.673 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:53.673 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 53246 2024-09-14T06:30:53.673 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:53.673 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:30:53.673 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:53.673 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:30:53.673 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369420+0000", 2024-09-14T06:30:53.673 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:53.673 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:53.674 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:53.674 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:30:53.674 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369420+0000", 2024-09-14T06:30:53.674 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:53.674 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:53.674 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:53.674 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:30:53.674 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369420+0000", 2024-09-14T06:30:53.674 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.1299999999999999e-07 2024-09-14T06:30:53.674 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:53.674 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:53.674 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:30:53.674 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369425+0000", 2024-09-14T06:30:53.674 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.0130000000000001e-06 2024-09-14T06:30:53.674 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:53.675 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:53.675 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:30:53.675 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369428+0000", 2024-09-14T06:30:53.675 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.349e-06 2024-09-14T06:30:53.675 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:53.675 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:53.675 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:30:53.675 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369431+0000", 2024-09-14T06:30:53.675 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.7389999999999999e-06 2024-09-14T06:30:53.675 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:53.675 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:53.675 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:30:53.675 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.416834+0000", 2024-09-14T06:30:53.675 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047402669000000001 2024-09-14T06:30:53.675 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:53.676 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:53.676 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:30:53.676 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.416882+0000", 2024-09-14T06:30:53.676 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.8068999999999997e-05 2024-09-14T06:30:53.676 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:53.676 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:53.676 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:30:53.676 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.416949+0000", 2024-09-14T06:30:53.676 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.7410999999999995e-05 2024-09-14T06:30:53.676 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:53.676 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:30:53.676 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:53.676 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:53.676 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:53.677 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:53247 3.f 3:f71f51a7:::rbd_data.112dca2f6ae1.0000000000000015:17 [read 1937408~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:30:53.677 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:44.369434+0000", 2024-09-14T06:30:53.677 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 69.292210710999996, 2024-09-14T06:30:53.677 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047652264999999999, 2024-09-14T06:30:53.677 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:30:53.677 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:30:53.677 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:30:53.677 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:30:53.677 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:30:53.677 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:53.677 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 53247 2024-09-14T06:30:53.677 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:53.678 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:30:53.678 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:53.678 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:30:53.678 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369434+0000", 2024-09-14T06:30:53.678 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:53.678 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:53.678 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:53.678 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:30:53.678 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369434+0000", 2024-09-14T06:30:53.678 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:53.678 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:53.678 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:53.678 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:30:53.678 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369435+0000", 2024-09-14T06:30:53.679 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.2100000000000001e-07 2024-09-14T06:30:53.679 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:53.679 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:53.679 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:30:53.679 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369436+0000", 2024-09-14T06:30:53.679 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.122e-06 2024-09-14T06:30:53.679 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:53.679 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:53.679 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:30:53.679 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369439+0000", 2024-09-14T06:30:53.679 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.0400000000000001e-06 2024-09-14T06:30:53.679 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:53.679 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:53.679 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:30:53.679 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369441+0000", 2024-09-14T06:30:53.680 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.728e-06 2024-09-14T06:30:53.680 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:53.680 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:53.680 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:30:53.680 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.416977+0000", 2024-09-14T06:30:53.680 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047535344 2024-09-14T06:30:53.680 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:53.680 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:53.680 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:30:53.680 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.417029+0000", 2024-09-14T06:30:53.680 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.2037999999999999e-05 2024-09-14T06:30:53.680 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:53.680 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:53.680 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:30:53.681 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.417087+0000", 2024-09-14T06:30:53.681 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.7772000000000002e-05 2024-09-14T06:30:53.681 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:53.681 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:30:53.681 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:53.681 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:53.681 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:53.681 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:53248 3.f 3:f71f51a7:::rbd_data.112dca2f6ae1.0000000000000015:17 [read 1961984~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:30:53.681 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:44.369445+0000", 2024-09-14T06:30:53.681 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 69.292200159000004, 2024-09-14T06:30:53.681 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047798489, 2024-09-14T06:30:53.681 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:30:53.681 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:30:53.682 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:30:53.682 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:30:53.682 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:30:53.682 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:53.682 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 53248 2024-09-14T06:30:53.682 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:53.682 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:30:53.682 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:53.682 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:30:53.682 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369445+0000", 2024-09-14T06:30:53.682 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:53.682 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:53.682 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:53.682 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:30:53.683 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369445+0000", 2024-09-14T06:30:53.683 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:53.683 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:53.683 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:53.683 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:30:53.683 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369445+0000", 2024-09-14T06:30:53.683 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.0699999999999999e-07 2024-09-14T06:30:53.683 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:53.683 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:53.683 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:30:53.683 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369446+0000", 2024-09-14T06:30:53.683 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.1000000000000001e-06 2024-09-14T06:30:53.683 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:53.684 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:53.684 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:30:53.684 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369449+0000", 2024-09-14T06:30:53.684 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.1039999999999998e-06 2024-09-14T06:30:53.684 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:53.684 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:53.684 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:30:53.684 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369452+0000", 2024-09-14T06:30:53.684 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.6199999999999999e-06 2024-09-14T06:30:53.684 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:53.684 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:53.684 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:30:53.684 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.417114+0000", 2024-09-14T06:30:53.685 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047661901999999999 2024-09-14T06:30:53.685 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:53.685 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:53.685 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:30:53.685 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.417181+0000", 2024-09-14T06:30:53.685 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.7385999999999995e-05 2024-09-14T06:30:53.685 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:53.685 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:53.685 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:30:53.685 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.417243+0000", 2024-09-14T06:30:53.685 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.2169999999999996e-05 2024-09-14T06:30:53.685 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:53.685 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:30:53.685 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:53.686 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:53.686 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:53.686 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:53249 3.f 3:f71f51a7:::rbd_data.112dca2f6ae1.0000000000000015:17 [read 1986560~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:30:53.686 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:44.369455+0000", 2024-09-14T06:30:53.686 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 69.292189848000007, 2024-09-14T06:30:53.686 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047890493999999999, 2024-09-14T06:30:53.686 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:30:53.686 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:30:53.686 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:30:53.686 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:30:53.686 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:30:53.686 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:53.686 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 53249 2024-09-14T06:30:53.686 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:53.686 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:30:53.687 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:53.687 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:30:53.687 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369455+0000", 2024-09-14T06:30:53.687 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:53.687 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:53.687 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:53.687 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:30:53.687 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369455+0000", 2024-09-14T06:30:53.687 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:53.687 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:53.687 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:53.687 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:30:53.687 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369455+0000", 2024-09-14T06:30:53.687 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.0800000000000001e-07 2024-09-14T06:30:53.688 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:53.688 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:53.688 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:30:53.688 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369456+0000", 2024-09-14T06:30:53.688 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.0729999999999999e-06 2024-09-14T06:30:53.688 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:53.688 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:53.688 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:30:53.688 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369460+0000", 2024-09-14T06:30:53.688 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.027e-06 2024-09-14T06:30:53.688 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:53.688 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:53.688 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:30:53.688 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369462+0000", 2024-09-14T06:30:53.688 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.5320000000000001e-06 2024-09-14T06:30:53.689 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:53.689 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:53.689 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:30:53.689 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.417257+0000", 2024-09-14T06:30:53.689 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.04779489 2024-09-14T06:30:53.689 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:53.689 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:53.689 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:30:53.689 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.417296+0000", 2024-09-14T06:30:53.689 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.8658000000000002e-05 2024-09-14T06:30:53.689 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:53.689 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:53.689 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:30:53.689 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.417346+0000", 2024-09-14T06:30:53.690 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.0105999999999998e-05 2024-09-14T06:30:53.690 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:53.690 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:30:53.690 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:53.690 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:53.690 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:53.690 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:53250 3.f 3:f71f51a7:::rbd_data.112dca2f6ae1.0000000000000015:17 [read 2011136~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:30:53.690 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:44.369465+0000", 2024-09-14T06:30:53.690 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 69.292179877999999, 2024-09-14T06:30:53.690 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.048032230000000002, 2024-09-14T06:30:53.690 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:30:53.690 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:30:53.690 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:30:53.690 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:30:53.690 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:30:53.691 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:53.691 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 53250 2024-09-14T06:30:53.691 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:53.691 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:30:53.691 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:53.691 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:30:53.691 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369465+0000", 2024-09-14T06:30:53.691 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:53.691 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:53.691 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:53.691 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:30:53.691 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369465+0000", 2024-09-14T06:30:53.691 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:53.691 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:53.692 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:53.692 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:30:53.692 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369465+0000", 2024-09-14T06:30:53.692 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.2100000000000001e-07 2024-09-14T06:30:53.692 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:53.692 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:53.692 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:30:53.692 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369466+0000", 2024-09-14T06:30:53.692 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.0640000000000001e-06 2024-09-14T06:30:53.692 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:53.692 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:53.692 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:30:53.692 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369469+0000", 2024-09-14T06:30:53.692 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.0170000000000001e-06 2024-09-14T06:30:53.693 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:53.693 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:53.693 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:30:53.693 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369472+0000", 2024-09-14T06:30:53.693 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.6189999999999998e-06 2024-09-14T06:30:53.693 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:53.693 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:53.693 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:30:53.693 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.417383+0000", 2024-09-14T06:30:53.693 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047911092000000002 2024-09-14T06:30:53.693 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:53.693 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:53.693 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:30:53.693 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.417430+0000", 2024-09-14T06:30:53.694 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.6898999999999999e-05 2024-09-14T06:30:53.694 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:53.694 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:53.694 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:30:53.694 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.417497+0000", 2024-09-14T06:30:53.694 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.7317999999999995e-05 2024-09-14T06:30:53.694 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:53.694 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:30:53.694 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:53.694 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:53.694 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:53.694 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:53251 3.f 3:f71f51a7:::rbd_data.112dca2f6ae1.0000000000000015:17 [read 2035712~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:30:53.694 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:44.369475+0000", 2024-09-14T06:30:53.694 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 69.292169881999996, 2024-09-14T06:30:53.694 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.048154542000000002, 2024-09-14T06:30:53.695 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:30:53.695 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:30:53.695 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:30:53.695 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:30:53.695 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:30:53.695 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:53.695 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 53251 2024-09-14T06:30:53.695 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:53.695 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:30:53.695 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:53.695 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:30:53.695 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369475+0000", 2024-09-14T06:30:53.695 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:53.695 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:53.695 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:53.695 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:30:53.696 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369475+0000", 2024-09-14T06:30:53.696 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:53.696 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:53.696 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:53.696 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:30:53.696 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369475+0000", 2024-09-14T06:30:53.696 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.0100000000000001e-07 2024-09-14T06:30:53.696 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:53.696 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:53.696 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:30:53.696 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369476+0000", 2024-09-14T06:30:53.696 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.0580000000000001e-06 2024-09-14T06:30:53.696 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:53.697 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:53.697 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:30:53.697 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369479+0000", 2024-09-14T06:30:53.697 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.9950000000000001e-06 2024-09-14T06:30:53.697 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:53.697 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:53.697 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:30:53.697 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369482+0000", 2024-09-14T06:30:53.697 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.6340000000000001e-06 2024-09-14T06:30:53.697 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:53.697 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:53.697 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:30:53.697 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.417530+0000", 2024-09-14T06:30:53.697 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.048047535000000002 2024-09-14T06:30:53.698 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:53.698 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:53.698 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:30:53.698 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.417574+0000", 2024-09-14T06:30:53.698 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.4255999999999998e-05 2024-09-14T06:30:53.698 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:53.698 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:53.698 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:30:53.698 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.417630+0000", 2024-09-14T06:30:53.698 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.5862999999999997e-05 2024-09-14T06:30:53.698 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:53.698 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:30:53.698 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:53.699 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:53.699 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:53.699 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:53252 3.f 3:f71f51a7:::rbd_data.112dca2f6ae1.0000000000000015:17 [read 2060288~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:30:53.699 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:44.369485+0000", 2024-09-14T06:30:53.699 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 69.292160034000005, 2024-09-14T06:30:53.699 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.048254116, 2024-09-14T06:30:53.699 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:30:53.699 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:30:53.699 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:30:53.699 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:30:53.699 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:30:53.699 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:53.699 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 53252 2024-09-14T06:30:53.699 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:53.699 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:30:53.700 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:53.700 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:30:53.700 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369485+0000", 2024-09-14T06:30:53.700 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:53.700 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:53.700 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:53.700 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:30:53.700 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369485+0000", 2024-09-14T06:30:53.700 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:53.700 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:53.700 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:53.700 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:30:53.700 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369485+0000", 2024-09-14T06:30:53.700 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.1e-07 2024-09-14T06:30:53.701 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:53.701 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:53.701 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:30:53.701 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369487+0000", 2024-09-14T06:30:53.701 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.42e-06 2024-09-14T06:30:53.701 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:53.701 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:53.701 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:30:53.701 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369490+0000", 2024-09-14T06:30:53.701 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.9639999999999999e-06 2024-09-14T06:30:53.701 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:53.701 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:53.701 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:30:53.701 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369492+0000", 2024-09-14T06:30:53.702 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.5679999999999998e-06 2024-09-14T06:30:53.702 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:53.702 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:53.702 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:30:53.702 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.417645+0000", 2024-09-14T06:30:53.702 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.048152753 2024-09-14T06:30:53.702 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:53.702 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:53.702 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:30:53.702 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.417687+0000", 2024-09-14T06:30:53.702 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.1900999999999997e-05 2024-09-14T06:30:53.702 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:53.702 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:53.703 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:30:53.703 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.417739+0000", 2024-09-14T06:30:53.703 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.2299999999999997e-05 2024-09-14T06:30:53.703 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:53.703 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:30:53.703 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:53.703 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:53.703 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:53.703 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:53253 3.f 3:f71f51a7:::rbd_data.112dca2f6ae1.0000000000000015:17 [read 2084864~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:30:53.703 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:44.369495+0000", 2024-09-14T06:30:53.703 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 69.292149910000006, 2024-09-14T06:30:53.703 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.048366785000000002, 2024-09-14T06:30:53.703 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:30:53.703 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:30:53.704 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:30:53.704 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:30:53.704 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:30:53.704 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:53.704 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 53253 2024-09-14T06:30:53.704 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:53.704 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:30:53.704 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:53.704 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:30:53.704 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369495+0000", 2024-09-14T06:30:53.704 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:53.704 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:53.704 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:53.704 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:30:53.705 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369495+0000", 2024-09-14T06:30:53.705 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:53.705 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:53.705 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:53.705 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:30:53.705 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370388+0000", 2024-09-14T06:30:53.705 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00089287999999999998 2024-09-14T06:30:53.705 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:53.705 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:53.705 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:30:53.705 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370392+0000", 2024-09-14T06:30:53.705 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.2130000000000002e-06 2024-09-14T06:30:53.705 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:53.705 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:53.706 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:30:53.706 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370401+0000", 2024-09-14T06:30:53.706 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.7649999999999999e-06 2024-09-14T06:30:53.706 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:53.706 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:53.706 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:30:53.706 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370406+0000", 2024-09-14T06:30:53.706 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.0159999999999999e-06 2024-09-14T06:30:53.706 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:53.706 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:53.706 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:30:53.706 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.417764+0000", 2024-09-14T06:30:53.706 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047358348000000001 2024-09-14T06:30:53.706 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:53.707 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:53.707 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:30:53.707 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.417806+0000", 2024-09-14T06:30:53.707 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.1149999999999997e-05 2024-09-14T06:30:53.707 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:53.707 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:53.707 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:30:53.707 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.417862+0000", 2024-09-14T06:30:53.707 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.6412999999999997e-05 2024-09-14T06:30:53.707 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:53.707 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:30:53.707 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:53.707 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:53.707 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:53.707 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:53340 3.f 3:f71f51a7:::rbd_data.112dca2f6ae1.0000000000000015:17 [read 1056768~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:30:53.708 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:44.370421+0000", 2024-09-14T06:30:53.708 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 69.291224029999995, 2024-09-14T06:30:53.708 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047566313999999998, 2024-09-14T06:30:53.708 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:30:53.708 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:30:53.708 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:30:53.708 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:30:53.708 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:30:53.708 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:53.708 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 53340 2024-09-14T06:30:53.708 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:53.708 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:30:53.708 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:53.708 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:30:53.708 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370421+0000", 2024-09-14T06:30:53.709 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:53.709 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:53.709 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:53.709 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:30:53.709 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370421+0000", 2024-09-14T06:30:53.709 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:53.709 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:53.709 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:53.709 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:30:53.709 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370422+0000", 2024-09-14T06:30:53.709 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.3799999999999996e-07 2024-09-14T06:30:53.709 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:53.709 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:53.709 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:30:53.710 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370424+0000", 2024-09-14T06:30:53.710 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.6919999999999998e-06 2024-09-14T06:30:53.710 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:53.710 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:53.710 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:30:53.710 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370432+0000", 2024-09-14T06:30:53.710 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.2439999999999996e-06 2024-09-14T06:30:53.710 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:53.710 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:53.710 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:30:53.710 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370437+0000", 2024-09-14T06:30:53.710 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.5389999999999998e-06 2024-09-14T06:30:53.710 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:53.711 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:53.711 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:30:53.711 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.417888+0000", 2024-09-14T06:30:53.711 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047450432000000001 2024-09-14T06:30:53.711 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:53.711 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:53.711 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:30:53.711 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.417935+0000", 2024-09-14T06:30:53.711 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.7454999999999999e-05 2024-09-14T06:30:53.711 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:53.711 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:53.711 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:30:53.711 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.417987+0000", 2024-09-14T06:30:53.711 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.2213999999999998e-05 2024-09-14T06:30:53.711 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:53.711 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:30:53.711 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:53.712 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:53.712 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:53.712 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:53341 3.f 3:f71f51a7:::rbd_data.112dca2f6ae1.0000000000000015:17 [read 1081344~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:30:53.712 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:44.370446+0000", 2024-09-14T06:30:53.712 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 69.291198757000004, 2024-09-14T06:30:53.712 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047665723, 2024-09-14T06:30:53.712 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:30:53.712 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:30:53.712 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:30:53.712 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:30:53.712 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:30:53.712 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:53.712 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 53341 2024-09-14T06:30:53.712 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:53.713 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:30:53.713 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:53.713 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:30:53.713 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370446+0000", 2024-09-14T06:30:53.713 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:53.713 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:53.713 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:53.713 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:30:53.713 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370446+0000", 2024-09-14T06:30:53.713 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:53.713 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:53.713 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:53.713 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:30:53.714 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370447+0000", 2024-09-14T06:30:53.714 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.6100000000000002e-07 2024-09-14T06:30:53.714 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:53.714 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:53.714 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:30:53.714 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370448+0000", 2024-09-14T06:30:53.714 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.7370000000000001e-06 2024-09-14T06:30:53.714 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:53.714 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:53.714 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:30:53.714 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370462+0000", 2024-09-14T06:30:53.714 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.3227000000000001e-05 2024-09-14T06:30:53.714 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:53.715 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:53.715 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:30:53.715 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370467+0000", 2024-09-14T06:30:53.715 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.6540000000000004e-06 2024-09-14T06:30:53.715 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:53.715 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:53.715 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:30:53.715 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.418013+0000", 2024-09-14T06:30:53.715 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047545325999999999 2024-09-14T06:30:53.715 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:53.715 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:53.715 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:30:53.715 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.418054+0000", 2024-09-14T06:30:53.716 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.1696999999999997e-05 2024-09-14T06:30:53.716 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:53.716 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:53.716 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:30:53.716 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.418112+0000", 2024-09-14T06:30:53.716 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.7720999999999998e-05 2024-09-14T06:30:53.716 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:53.716 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:30:53.716 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:53.716 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:53.716 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:53.716 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:53342 3.f 3:f71f51a7:::rbd_data.112dca2f6ae1.0000000000000015:17 [read 1105920~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:30:53.716 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:44.370475+0000", 2024-09-14T06:30:53.717 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 69.291169695999997, 2024-09-14T06:30:53.717 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047748848000000003, 2024-09-14T06:30:53.717 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:30:53.717 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:30:53.717 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:30:53.717 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:30:53.717 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:30:53.717 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:53.717 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 53342 2024-09-14T06:30:53.717 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:53.717 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:30:53.717 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:53.717 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:30:53.718 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370475+0000", 2024-09-14T06:30:53.718 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:53.718 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:53.718 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:53.718 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:30:53.718 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370475+0000", 2024-09-14T06:30:53.718 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:53.718 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:53.718 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:53.718 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:30:53.718 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370476+0000", 2024-09-14T06:30:53.718 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.4700000000000002e-07 2024-09-14T06:30:53.718 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:53.719 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:53.719 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:30:53.719 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370478+0000", 2024-09-14T06:30:53.719 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.5890000000000001e-06 2024-09-14T06:30:53.719 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:53.719 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:53.719 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:30:53.719 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370486+0000", 2024-09-14T06:30:53.719 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.3470000000000001e-06 2024-09-14T06:30:53.719 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:53.719 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:53.719 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:30:53.719 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370491+0000", 2024-09-14T06:30:53.720 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.5720000000000002e-06 2024-09-14T06:30:53.720 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:53.720 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:53.720 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:30:53.720 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.418135+0000", 2024-09-14T06:30:53.720 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047644094999999997 2024-09-14T06:30:53.720 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:53.720 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:53.720 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:30:53.720 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.418174+0000", 2024-09-14T06:30:53.720 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.8204000000000002e-05 2024-09-14T06:30:53.720 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:53.720 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:53.720 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:30:53.721 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.418224+0000", 2024-09-14T06:30:53.721 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.0593999999999997e-05 2024-09-14T06:30:53.721 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:53.721 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:30:53.721 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:53.721 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:53.721 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:53.721 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:53343 3.f 3:f71f51a7:::rbd_data.112dca2f6ae1.0000000000000015:17 [read 1130496~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:30:53.721 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:44.370498+0000", 2024-09-14T06:30:53.721 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 69.291146819999994, 2024-09-14T06:30:53.721 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047841574999999997, 2024-09-14T06:30:53.721 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:30:53.721 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:30:53.722 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:30:53.722 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:30:53.722 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:30:53.722 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:53.722 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 53343 2024-09-14T06:30:53.722 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:53.722 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:30:53.722 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:53.722 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:30:53.722 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370498+0000", 2024-09-14T06:30:53.722 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:53.722 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:53.722 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:53.723 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:30:53.723 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370498+0000", 2024-09-14T06:30:53.723 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:53.723 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:53.723 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:53.723 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:30:53.723 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370499+0000", 2024-09-14T06:30:53.723 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.9999999999999998e-07 2024-09-14T06:30:53.723 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:53.723 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:53.723 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:30:53.723 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370501+0000", 2024-09-14T06:30:53.724 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.0310000000000001e-06 2024-09-14T06:30:53.724 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:53.724 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:53.724 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:30:53.724 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370507+0000", 2024-09-14T06:30:53.724 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.4169999999999997e-06 2024-09-14T06:30:53.724 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:53.724 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:53.724 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:30:53.724 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370516+0000", 2024-09-14T06:30:53.724 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.7190000000000007e-06 2024-09-14T06:30:53.724 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:53.724 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:53.725 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:30:53.725 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.418247+0000", 2024-09-14T06:30:53.725 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047731190999999999 2024-09-14T06:30:53.725 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:53.725 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:53.725 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:30:53.725 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.418286+0000", 2024-09-14T06:30:53.725 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.8988e-05 2024-09-14T06:30:53.725 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:53.725 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:53.725 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:30:53.725 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.418340+0000", 2024-09-14T06:30:53.725 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.3829000000000002e-05 2024-09-14T06:30:53.726 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:53.726 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:30:53.726 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:53.726 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:53.726 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:53.726 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:53344 3.f 3:f71f51a7:::rbd_data.112dca2f6ae1.0000000000000015:17 [read 1155072~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:30:53.726 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:44.370524+0000", 2024-09-14T06:30:53.726 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 69.291121050000001, 2024-09-14T06:30:53.726 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047972262000000002, 2024-09-14T06:30:53.726 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:30:53.726 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:30:53.726 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:30:53.726 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:30:53.726 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:30:53.727 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:53.727 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 53344 2024-09-14T06:30:53.727 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:53.727 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:30:53.727 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:53.727 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:30:53.727 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370524+0000", 2024-09-14T06:30:53.727 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:53.727 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:53.727 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:53.727 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:30:53.727 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370524+0000", 2024-09-14T06:30:53.727 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:53.727 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:53.728 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:53.728 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:30:53.728 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370618+0000", 2024-09-14T06:30:53.728 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.4425999999999993e-05 2024-09-14T06:30:53.728 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:53.728 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:53.728 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:30:53.728 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370622+0000", 2024-09-14T06:30:53.728 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.3519999999999998e-06 2024-09-14T06:30:53.728 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:53.728 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:53.728 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:30:53.728 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370631+0000", 2024-09-14T06:30:53.728 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.9299999999999992e-06 2024-09-14T06:30:53.729 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:53.729 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:53.729 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:30:53.729 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370637+0000", 2024-09-14T06:30:53.729 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.3269999999999997e-06 2024-09-14T06:30:53.729 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:53.729 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:53.729 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:30:53.729 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.418372+0000", 2024-09-14T06:30:53.729 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047734764999999998 2024-09-14T06:30:53.729 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:53.729 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:53.729 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:30:53.729 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.418431+0000", 2024-09-14T06:30:53.730 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.8696999999999997e-05 2024-09-14T06:30:53.730 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:53.730 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:53.730 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:30:53.730 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.418496+0000", 2024-09-14T06:30:53.730 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.5765000000000005e-05 2024-09-14T06:30:53.730 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:53.730 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:30:53.730 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:53.730 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:53.730 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:53.730 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:53345 3.f 3:f71f51a7:::rbd_data.112dca2f6ae1.0000000000000015:17 [read 1179648~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:30:53.730 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:44.370647+0000", 2024-09-14T06:30:53.731 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 69.290997583000006, 2024-09-14T06:30:53.731 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047953775999999997, 2024-09-14T06:30:53.731 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:30:53.731 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:30:53.731 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:30:53.731 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:30:53.731 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:30:53.731 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:53.731 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 53345 2024-09-14T06:30:53.731 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:53.731 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:30:53.731 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:53.731 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:30:53.731 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370647+0000", 2024-09-14T06:30:53.731 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:53.732 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:53.732 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:53.732 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:30:53.732 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370647+0000", 2024-09-14T06:30:53.732 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:53.732 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:53.732 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:53.732 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:30:53.732 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.372346+0000", 2024-09-14T06:30:53.732 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.001698284 2024-09-14T06:30:53.732 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:53.732 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:53.732 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:30:53.733 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.372351+0000", 2024-09-14T06:30:53.733 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.8890000000000001e-06 2024-09-14T06:30:53.733 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:53.733 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:53.733 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:30:53.733 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.372362+0000", 2024-09-14T06:30:53.733 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.0941000000000001e-05 2024-09-14T06:30:53.733 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:53.733 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:53.733 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:30:53.733 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.372369+0000", 2024-09-14T06:30:53.733 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.362e-06 2024-09-14T06:30:53.733 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:53.733 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:53.734 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:30:53.734 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.418510+0000", 2024-09-14T06:30:53.734 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046140914999999998 2024-09-14T06:30:53.734 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:53.734 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:53.734 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:30:53.734 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.418549+0000", 2024-09-14T06:30:53.734 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.8717999999999998e-05 2024-09-14T06:30:53.734 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:53.734 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:53.734 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:30:53.734 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.418601+0000", 2024-09-14T06:30:53.735 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.2667000000000002e-05 2024-09-14T06:30:53.735 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:53.735 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:30:53.735 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:53.735 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:53.735 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:53.735 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:53351 3.f 3:f71f51a7:::rbd_data.112dca2f6ae1.0000000000000015:17 [read 1327104~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:30:53.735 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:44.372499+0000", 2024-09-14T06:30:53.735 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 69.289146462000005, 2024-09-14T06:30:53.735 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047140596, 2024-09-14T06:30:53.735 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:30:53.735 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:30:53.736 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:30:53.736 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:30:53.736 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:30:53.736 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:53.736 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 53351 2024-09-14T06:30:53.736 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:53.736 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:30:53.736 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:53.736 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:30:53.736 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.372499+0000", 2024-09-14T06:30:53.736 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:53.736 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:53.736 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:53.736 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:30:53.737 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.372499+0000", 2024-09-14T06:30:53.737 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:53.737 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:53.737 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:53.737 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:30:53.737 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.372499+0000", 2024-09-14T06:30:53.737 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.3000000000000001e-07 2024-09-14T06:30:53.737 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:53.737 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:53.737 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:30:53.737 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.372501+0000", 2024-09-14T06:30:53.737 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.0820000000000001e-06 2024-09-14T06:30:53.737 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:53.737 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:53.738 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:30:53.738 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.372513+0000", 2024-09-14T06:30:53.738 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.1610999999999999e-05 2024-09-14T06:30:53.738 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:53.738 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:53.738 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:30:53.738 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.372518+0000", 2024-09-14T06:30:53.738 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.327e-06 2024-09-14T06:30:53.738 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:53.738 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:53.738 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:30:53.738 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.419530+0000", 2024-09-14T06:30:53.738 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047011798 2024-09-14T06:30:53.739 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:53.739 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:53.739 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:30:53.739 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.419578+0000", 2024-09-14T06:30:53.739 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.8446999999999999e-05 2024-09-14T06:30:53.739 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:53.739 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:53.739 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:30:53.739 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.419639+0000", 2024-09-14T06:30:53.739 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.0900999999999998e-05 2024-09-14T06:30:53.739 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:53.739 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:30:53.739 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:53.739 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:53.740 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:53.740 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:53352 3.f 3:f71f51a7:::rbd_data.112dca2f6ae1.0000000000000015:17 [read 1351680~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:30:53.740 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:44.372525+0000", 2024-09-14T06:30:53.740 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 69.289120548, 2024-09-14T06:30:53.740 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047259847000000001, 2024-09-14T06:30:53.740 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:30:53.740 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:30:53.740 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:30:53.740 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:30:53.740 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:30:53.740 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:53.740 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 53352 2024-09-14T06:30:53.740 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:53.741 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:30:53.741 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:53.741 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:30:53.741 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.372525+0000", 2024-09-14T06:30:53.741 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:53.741 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:53.741 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:53.741 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:30:53.741 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.372525+0000", 2024-09-14T06:30:53.741 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:53.741 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:53.741 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:53.741 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:30:53.741 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.372525+0000", 2024-09-14T06:30:53.742 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.0600000000000001e-07 2024-09-14T06:30:53.742 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:53.742 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:53.742 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:30:53.742 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.372527+0000", 2024-09-14T06:30:53.742 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.0499999999999999e-06 2024-09-14T06:30:53.742 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:53.742 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:53.742 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:30:53.742 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.372533+0000", 2024-09-14T06:30:53.742 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.4320000000000004e-06 2024-09-14T06:30:53.742 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:53.742 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:53.742 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:30:53.743 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.372538+0000", 2024-09-14T06:30:53.743 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.9130000000000002e-06 2024-09-14T06:30:53.743 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:53.743 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:53.743 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:30:53.743 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.419667+0000", 2024-09-14T06:30:53.743 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047128976000000003 2024-09-14T06:30:53.743 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:53.743 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:53.743 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:30:53.743 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.419715+0000", 2024-09-14T06:30:53.743 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.7868999999999999e-05 2024-09-14T06:30:53.743 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:53.743 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:53.743 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:30:53.744 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.419784+0000", 2024-09-14T06:30:53.744 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.9201000000000004e-05 2024-09-14T06:30:53.744 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:53.744 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:30:53.744 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:53.744 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:53.744 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:53.744 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:53353 3.f 3:f71f51a7:::rbd_data.112dca2f6ae1.0000000000000015:17 [read 1376256~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:30:53.744 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:44.372545+0000", 2024-09-14T06:30:53.744 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 69.289100508000004, 2024-09-14T06:30:53.744 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047364232999999999, 2024-09-14T06:30:53.744 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:30:53.744 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:30:53.745 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:30:53.745 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:30:53.745 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:30:53.745 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:53.745 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 53353 2024-09-14T06:30:53.745 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:53.745 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:30:53.745 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:53.745 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:30:53.745 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.372545+0000", 2024-09-14T06:30:53.745 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:53.745 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:53.745 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:53.746 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:30:53.746 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.372545+0000", 2024-09-14T06:30:53.746 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:53.746 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:53.746 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:53.746 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:30:53.746 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.372545+0000", 2024-09-14T06:30:53.746 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.27e-07 2024-09-14T06:30:53.746 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:53.746 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:53.746 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:30:53.746 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.372547+0000", 2024-09-14T06:30:53.746 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.0600000000000002e-06 2024-09-14T06:30:53.746 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:53.746 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:53.746 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:30:53.747 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.372553+0000", 2024-09-14T06:30:53.747 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.7980000000000002e-06 2024-09-14T06:30:53.747 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:53.747 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:53.747 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:30:53.747 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.372557+0000", 2024-09-14T06:30:53.747 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.7389999999999999e-06 2024-09-14T06:30:53.747 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:53.747 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:53.747 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:30:53.747 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.419811+0000", 2024-09-14T06:30:53.747 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047253310999999999 2024-09-14T06:30:53.747 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:53.747 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:53.747 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:30:53.747 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.419854+0000", 2024-09-14T06:30:53.748 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.3448000000000001e-05 2024-09-14T06:30:53.748 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:53.748 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:53.748 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:30:53.748 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.419909+0000", 2024-09-14T06:30:53.748 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.4549999999999998e-05 2024-09-14T06:30:53.748 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:53.748 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:30:53.748 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:53.748 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:53.748 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:53.748 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:53354 3.f 3:f71f51a7:::rbd_data.112dca2f6ae1.0000000000000015:17 [read 1400832~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:30:53.748 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:44.372563+0000", 2024-09-14T06:30:53.748 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 69.289082070999996, 2024-09-14T06:30:53.748 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047482233999999998, 2024-09-14T06:30:53.749 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:30:53.749 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:30:53.749 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:30:53.749 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:30:53.749 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:30:53.749 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:53.749 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 53354 2024-09-14T06:30:53.749 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:53.749 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:30:53.749 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:53.749 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:30:53.749 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.372563+0000", 2024-09-14T06:30:53.749 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:53.749 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:53.749 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:53.749 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:30:53.750 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.372563+0000", 2024-09-14T06:30:53.750 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:53.750 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:53.750 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:53.750 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:30:53.750 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.375009+0000", 2024-09-14T06:30:53.750 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.0024457680000000001 2024-09-14T06:30:53.750 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:53.750 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:53.750 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:30:53.750 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.375014+0000", 2024-09-14T06:30:53.750 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.0599999999999998e-06 2024-09-14T06:30:53.750 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:53.750 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:53.750 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:30:53.750 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.375026+0000", 2024-09-14T06:30:53.751 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.1783999999999999e-05 2024-09-14T06:30:53.751 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:53.751 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:53.751 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:30:53.751 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.375035+0000", 2024-09-14T06:30:53.751 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.7799999999999995e-06 2024-09-14T06:30:53.751 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:53.751 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:53.751 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:30:53.751 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.419942+0000", 2024-09-14T06:30:53.751 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.044906982999999998 2024-09-14T06:30:53.751 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:53.751 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:53.751 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:30:53.751 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.419989+0000", 2024-09-14T06:30:53.751 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.7104999999999997e-05 2024-09-14T06:30:53.752 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:53.752 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:53.752 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:30:53.752 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.420045+0000", 2024-09-14T06:30:53.752 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.5754000000000002e-05 2024-09-14T06:30:53.752 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:53.752 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:30:53.752 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:53.752 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:53.752 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:30:53.752 INFO:teuthology.orchestra.run.smithi138.stdout:} 2024-09-14T06:30:53.753 INFO:teuthology.orchestra.run.smithi094.stderr:clear 3.1 pg_upmap mapping 2024-09-14T06:30:53.754 INFO:tasks.ceph.osd.2.smithi094.stderr:2024-09-14T06:30:53.752+0000 7faf3269b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33649) UID: 0 2024-09-14T06:30:53.754 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-14T06:30:53.855 INFO:tasks.ceph.osd.5.smithi138.stderr:2024-09-14T06:30:53.854+0000 7fe13746f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33618) UID: 0 2024-09-14T06:30:53.861 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-14T06:30:53.861 INFO:teuthology.orchestra.run.smithi094.stdout: "ops": [], 2024-09-14T06:30:53.861 INFO:teuthology.orchestra.run.smithi094.stdout: "num_ops": 0 2024-09-14T06:30:53.861 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-14T06:30:53.870 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-14T06:30:53.956 INFO:tasks.ceph.osd.6.smithi138.stderr:2024-09-14T06:30:53.954+0000 7f8cd8849640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33620) UID: 0 2024-09-14T06:30:53.972 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-14T06:30:53.972 INFO:teuthology.orchestra.run.smithi094.stdout: "ops": [], 2024-09-14T06:30:53.972 INFO:teuthology.orchestra.run.smithi094.stdout: "complaint_time": 30, 2024-09-14T06:30:53.972 INFO:teuthology.orchestra.run.smithi094.stdout: "num_blocked_ops": 0 2024-09-14T06:30:53.973 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-14T06:30:53.981 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-14T06:30:54.056 INFO:tasks.ceph.osd.1.smithi094.stderr:2024-09-14T06:30:54.055+0000 7f328e47a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33641) UID: 0 2024-09-14T06:30:54.084 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-14T06:30:54.085 INFO:teuthology.orchestra.run.smithi094.stdout: "size": 20, 2024-09-14T06:30:54.085 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 600, 2024-09-14T06:30:54.085 INFO:teuthology.orchestra.run.smithi094.stdout: "ops": [ 2024-09-14T06:30:54.085 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:54.085 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:115193 3.1 3:856be886:::rbd_data.112d4d24c558.0000000000000004:29 [read 1523712~16384 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e96)", 2024-09-14T06:30:54.085 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:30:17.939854+0000", 2024-09-14T06:30:54.085 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 36.143128289000003, 2024-09-14T06:30:54.085 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.073897516999999996, 2024-09-14T06:30:54.085 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:30:54.085 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:30:54.085 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:30:54.086 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:30:54.086 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:30:54.086 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:54.086 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 115193 2024-09-14T06:30:54.086 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:54.086 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:30:54.086 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:54.086 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:30:54.086 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.939854+0000", 2024-09-14T06:30:54.086 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:54.086 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:54.086 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:54.086 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:30:54.086 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.939854+0000", 2024-09-14T06:30:54.086 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:54.087 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:54.087 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:54.087 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:30:54.087 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.939854+0000", 2024-09-14T06:30:54.087 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.9500000000000002e-07 2024-09-14T06:30:54.087 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:54.087 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:54.087 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:30:54.087 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.939856+0000", 2024-09-14T06:30:54.087 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.0959999999999999e-06 2024-09-14T06:30:54.087 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:54.087 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:54.087 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:30:54.087 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.939863+0000", 2024-09-14T06:30:54.087 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.1129999999999997e-06 2024-09-14T06:30:54.087 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:54.088 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:54.088 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:30:54.088 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.939867+0000", 2024-09-14T06:30:54.088 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.8350000000000003e-06 2024-09-14T06:30:54.088 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:54.088 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:54.088 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:30:54.088 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.013626+0000", 2024-09-14T06:30:54.088 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.073758799999999999 2024-09-14T06:30:54.088 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:54.088 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:54.088 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:30:54.089 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.013690+0000", 2024-09-14T06:30:54.089 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.3918999999999996e-05 2024-09-14T06:30:54.089 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:54.089 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:54.089 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:30:54.089 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.013751+0000", 2024-09-14T06:30:54.089 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.1159000000000002e-05 2024-09-14T06:30:54.089 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:54.089 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:54.089 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:54.089 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:54.089 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:54.089 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:115194 3.1 3:856be886:::rbd_data.112d4d24c558.0000000000000004:29 [read 1769472~16384 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e96)", 2024-09-14T06:30:54.090 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:30:17.939874+0000", 2024-09-14T06:30:54.090 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 36.143107978000003, 2024-09-14T06:30:54.090 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.074016988000000006, 2024-09-14T06:30:54.090 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:30:54.090 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:30:54.090 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:30:54.090 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:30:54.090 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:30:54.090 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:54.090 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 115194 2024-09-14T06:30:54.090 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:54.090 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:30:54.090 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:54.090 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:30:54.090 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.939874+0000", 2024-09-14T06:30:54.090 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:54.091 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:54.091 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:54.091 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:30:54.091 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.939874+0000", 2024-09-14T06:30:54.091 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:54.091 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:54.091 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:54.091 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:30:54.091 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.939875+0000", 2024-09-14T06:30:54.091 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.2600000000000002e-07 2024-09-14T06:30:54.091 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:54.091 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:54.091 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:30:54.092 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.939877+0000", 2024-09-14T06:30:54.092 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.9420000000000002e-06 2024-09-14T06:30:54.092 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:54.092 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:54.092 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:30:54.092 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.939882+0000", 2024-09-14T06:30:54.092 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.8749999999999997e-06 2024-09-14T06:30:54.092 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:54.092 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:54.092 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:30:54.092 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.939887+0000", 2024-09-14T06:30:54.092 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.741e-06 2024-09-14T06:30:54.092 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:54.092 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:54.093 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:30:54.093 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.013767+0000", 2024-09-14T06:30:54.093 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.073879720999999995 2024-09-14T06:30:54.093 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:54.093 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:54.093 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:30:54.093 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.013832+0000", 2024-09-14T06:30:54.093 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.5312000000000001e-05 2024-09-14T06:30:54.093 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:54.093 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:54.093 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:30:54.094 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.013891+0000", 2024-09-14T06:30:54.094 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.8870999999999999e-05 2024-09-14T06:30:54.094 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:54.094 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:54.094 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:54.094 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:54.094 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:54.094 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:115195 3.1 3:856be886:::rbd_data.112d4d24c558.0000000000000004:29 [read 2015232~16384 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e96)", 2024-09-14T06:30:54.094 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:30:17.939903+0000", 2024-09-14T06:30:54.094 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 36.143078793000001, 2024-09-14T06:30:54.094 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.074128199000000006, 2024-09-14T06:30:54.094 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:30:54.094 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:30:54.094 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:30:54.094 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:30:54.094 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:30:54.095 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:54.095 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 115195 2024-09-14T06:30:54.095 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:54.095 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:30:54.095 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:54.095 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:30:54.095 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.939903+0000", 2024-09-14T06:30:54.095 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:54.095 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:54.095 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:54.095 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:30:54.095 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.939903+0000", 2024-09-14T06:30:54.095 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:54.095 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:54.095 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:54.095 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:30:54.095 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.939904+0000", 2024-09-14T06:30:54.096 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.5799999999999999e-07 2024-09-14T06:30:54.096 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:54.096 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:54.096 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:30:54.096 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.939906+0000", 2024-09-14T06:30:54.096 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.2350000000000002e-06 2024-09-14T06:30:54.096 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:54.096 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:54.096 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:30:54.096 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.939913+0000", 2024-09-14T06:30:54.096 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.5649999999999998e-06 2024-09-14T06:30:54.096 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:54.096 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:54.096 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:30:54.096 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.939918+0000", 2024-09-14T06:30:54.096 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.784e-06 2024-09-14T06:30:54.097 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:54.097 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:54.097 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:30:54.097 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.013921+0000", 2024-09-14T06:30:54.097 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.074002911000000005 2024-09-14T06:30:54.097 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:54.097 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:54.097 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:30:54.097 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.013971+0000", 2024-09-14T06:30:54.097 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.9551e-05 2024-09-14T06:30:54.097 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:54.097 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:54.097 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:30:54.097 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.014031+0000", 2024-09-14T06:30:54.097 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.0594999999999998e-05 2024-09-14T06:30:54.098 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:54.098 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:54.098 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:54.098 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:54.098 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:54.098 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:115196 3.1 3:856be886:::rbd_data.112d4d24c558.0000000000000004:29 [read 2260992~16384 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e96)", 2024-09-14T06:30:54.098 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:30:17.939925+0000", 2024-09-14T06:30:54.098 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 36.143056600999998, 2024-09-14T06:30:54.098 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.074320137999999994, 2024-09-14T06:30:54.098 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:30:54.098 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:30:54.098 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:30:54.098 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:30:54.098 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:30:54.099 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:54.099 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 115196 2024-09-14T06:30:54.099 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:54.099 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:30:54.099 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:54.099 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:30:54.099 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.939925+0000", 2024-09-14T06:30:54.099 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:54.099 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:54.099 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:54.099 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:30:54.100 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.939925+0000", 2024-09-14T06:30:54.100 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:54.100 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:54.100 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:54.100 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:30:54.100 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.939926+0000", 2024-09-14T06:30:54.100 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.9999999999999998e-07 2024-09-14T06:30:54.100 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:54.100 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:54.100 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:30:54.100 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.939928+0000", 2024-09-14T06:30:54.100 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.9549999999999999e-06 2024-09-14T06:30:54.100 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:54.100 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:54.100 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:30:54.101 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.939934+0000", 2024-09-14T06:30:54.101 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.127e-06 2024-09-14T06:30:54.101 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:54.101 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:54.101 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:30:54.101 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.939939+0000", 2024-09-14T06:30:54.101 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.775e-06 2024-09-14T06:30:54.101 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:54.101 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:54.101 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:30:54.101 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.014075+0000", 2024-09-14T06:30:54.101 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.074136598999999997 2024-09-14T06:30:54.101 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:54.102 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:54.102 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:30:54.102 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.014162+0000", 2024-09-14T06:30:54.102 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.6477999999999994e-05 2024-09-14T06:30:54.102 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:54.102 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:54.102 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:30:54.102 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.014246+0000", 2024-09-14T06:30:54.102 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.3703999999999997e-05 2024-09-14T06:30:54.102 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:54.102 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:54.102 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:54.102 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:54.102 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:54.103 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:115197 3.1 3:856be886:::rbd_data.112d4d24c558.0000000000000004:29 [read 2506752~16384 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e96)", 2024-09-14T06:30:54.103 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:30:17.939969+0000", 2024-09-14T06:30:54.103 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 36.143013115999999, 2024-09-14T06:30:54.103 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.074528544000000002, 2024-09-14T06:30:54.103 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:30:54.103 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:30:54.103 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:30:54.103 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:30:54.103 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:30:54.103 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:54.103 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 115197 2024-09-14T06:30:54.103 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:54.103 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:30:54.103 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:54.104 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:30:54.104 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.939969+0000", 2024-09-14T06:30:54.104 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:54.104 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:54.104 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:54.104 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:30:54.104 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.939969+0000", 2024-09-14T06:30:54.104 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:54.104 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:54.104 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:54.104 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:30:54.104 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.939970+0000", 2024-09-14T06:30:54.104 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.3600000000000003e-07 2024-09-14T06:30:54.104 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:54.105 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:54.105 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:30:54.105 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.939972+0000", 2024-09-14T06:30:54.105 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.3180000000000001e-06 2024-09-14T06:30:54.105 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:54.105 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:54.105 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:30:54.105 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.939979+0000", 2024-09-14T06:30:54.105 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.8800000000000002e-06 2024-09-14T06:30:54.105 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:54.105 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:54.106 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:30:54.106 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.939984+0000", 2024-09-14T06:30:54.106 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.3510000000000001e-06 2024-09-14T06:30:54.106 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:54.106 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:54.106 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:30:54.106 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.014302+0000", 2024-09-14T06:30:54.106 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.074317416999999997 2024-09-14T06:30:54.106 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:54.106 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:54.106 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:30:54.106 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.014392+0000", 2024-09-14T06:30:54.106 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.0181999999999998e-05 2024-09-14T06:30:54.107 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:54.107 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:54.107 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:30:54.107 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.014497+0000", 2024-09-14T06:30:54.107 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00010576 2024-09-14T06:30:54.107 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:54.107 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:54.107 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:54.107 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:54.107 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:54.107 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:115198 3.1 3:856be886:::rbd_data.112d4d24c558.0000000000000004:29 [read 2752512~16384 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e96)", 2024-09-14T06:30:54.107 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:30:17.940004+0000", 2024-09-14T06:30:54.108 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 36.142977836, 2024-09-14T06:30:54.108 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.074736367999999997, 2024-09-14T06:30:54.108 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:30:54.108 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:30:54.108 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:30:54.108 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:30:54.108 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:30:54.108 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:54.108 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 115198 2024-09-14T06:30:54.108 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:54.108 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:30:54.108 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:54.108 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:30:54.108 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940004+0000", 2024-09-14T06:30:54.108 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:54.109 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:54.109 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:54.109 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:30:54.109 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940004+0000", 2024-09-14T06:30:54.109 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:54.109 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:54.109 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:54.109 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:30:54.109 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940005+0000", 2024-09-14T06:30:54.109 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.2200000000000004e-07 2024-09-14T06:30:54.109 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:54.109 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:54.109 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:30:54.110 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940007+0000", 2024-09-14T06:30:54.110 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.322e-06 2024-09-14T06:30:54.110 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:54.110 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:54.110 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:30:54.110 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940014+0000", 2024-09-14T06:30:54.110 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.6220000000000003e-06 2024-09-14T06:30:54.110 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:54.110 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:54.110 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:30:54.110 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940019+0000", 2024-09-14T06:30:54.110 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.6380000000000001e-06 2024-09-14T06:30:54.110 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:54.110 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:54.110 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:30:54.111 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.014562+0000", 2024-09-14T06:30:54.111 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.074542320999999995 2024-09-14T06:30:54.111 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:54.111 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:54.111 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:30:54.111 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.014646+0000", 2024-09-14T06:30:54.111 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.4079999999999995e-05 2024-09-14T06:30:54.111 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:54.111 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:54.111 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:30:54.111 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.014741+0000", 2024-09-14T06:30:54.111 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.4763000000000001e-05 2024-09-14T06:30:54.111 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:54.112 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:54.112 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:54.112 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:54.112 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:54.112 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:115199 3.1 3:856be886:::rbd_data.112d4d24c558.0000000000000004:29 [read 2998272~16384 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e96)", 2024-09-14T06:30:54.112 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:30:17.940040+0000", 2024-09-14T06:30:54.112 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 36.142942081000001, 2024-09-14T06:30:54.112 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.074957134999999994, 2024-09-14T06:30:54.112 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:30:54.112 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:30:54.112 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:30:54.112 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:30:54.112 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:30:54.113 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:54.113 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 115199 2024-09-14T06:30:54.113 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:54.113 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:30:54.113 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:54.113 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:30:54.113 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940040+0000", 2024-09-14T06:30:54.113 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:54.113 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:54.113 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:54.113 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:30:54.113 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940040+0000", 2024-09-14T06:30:54.113 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:54.114 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:54.114 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:54.114 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:30:54.114 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940041+0000", 2024-09-14T06:30:54.114 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.4000000000000001e-07 2024-09-14T06:30:54.114 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:54.114 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:54.114 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:30:54.114 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940046+0000", 2024-09-14T06:30:54.114 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.7250000000000002e-06 2024-09-14T06:30:54.114 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:54.114 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:54.114 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:30:54.114 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940053+0000", 2024-09-14T06:30:54.115 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.6109999999999999e-06 2024-09-14T06:30:54.115 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:54.115 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:54.115 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:30:54.115 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940058+0000", 2024-09-14T06:30:54.115 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.1590000000000001e-06 2024-09-14T06:30:54.115 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:54.115 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:54.115 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:30:54.115 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.014768+0000", 2024-09-14T06:30:54.115 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.074709624000000002 2024-09-14T06:30:54.115 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:54.115 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:54.115 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:30:54.115 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.014907+0000", 2024-09-14T06:30:54.116 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000139665 2024-09-14T06:30:54.116 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:54.116 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:54.116 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:30:54.116 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.014997+0000", 2024-09-14T06:30:54.116 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.9710999999999995e-05 2024-09-14T06:30:54.116 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:54.116 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:54.116 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:54.116 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:54.116 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:54.116 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:115200 3.1 3:856be886:::rbd_data.112d4d24c558.0000000000000004:29 [read 3244032~16384 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e96)", 2024-09-14T06:30:54.116 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:30:17.940080+0000", 2024-09-14T06:30:54.116 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 36.142901876000003, 2024-09-14T06:30:54.116 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.075137291999999994, 2024-09-14T06:30:54.117 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:30:54.117 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:30:54.117 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:30:54.117 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:30:54.117 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:30:54.117 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:54.117 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 115200 2024-09-14T06:30:54.117 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:54.117 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:30:54.117 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:54.117 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:30:54.117 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940080+0000", 2024-09-14T06:30:54.117 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:54.117 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:54.117 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:54.118 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:30:54.118 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940080+0000", 2024-09-14T06:30:54.118 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:54.118 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:54.118 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:54.118 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:30:54.118 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940081+0000", 2024-09-14T06:30:54.118 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.0800000000000004e-07 2024-09-14T06:30:54.118 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:54.118 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:54.118 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:30:54.118 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940083+0000", 2024-09-14T06:30:54.118 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.3920000000000001e-06 2024-09-14T06:30:54.119 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:54.119 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:54.119 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:30:54.119 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940091+0000", 2024-09-14T06:30:54.119 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.1389999999999995e-06 2024-09-14T06:30:54.119 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:54.119 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:54.119 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:30:54.119 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940097+0000", 2024-09-14T06:30:54.119 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.5550000000000002e-06 2024-09-14T06:30:54.119 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:54.119 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:54.119 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:30:54.119 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.015025+0000", 2024-09-14T06:30:54.120 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.074928579999999995 2024-09-14T06:30:54.120 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:54.120 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:54.120 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:30:54.120 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.015096+0000", 2024-09-14T06:30:54.120 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.0705999999999997e-05 2024-09-14T06:30:54.120 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:54.120 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:54.120 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:30:54.120 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.015217+0000", 2024-09-14T06:30:54.120 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00012131200000000001 2024-09-14T06:30:54.120 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:54.120 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:54.120 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:54.121 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:54.121 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:54.121 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:115201 3.1 3:856be886:::rbd_data.112d4d24c558.0000000000000004:29 [read 3489792~16384 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e96)", 2024-09-14T06:30:54.121 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:30:17.940113+0000", 2024-09-14T06:30:54.121 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 36.142869103000002, 2024-09-14T06:30:54.121 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.075246899000000006, 2024-09-14T06:30:54.121 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:30:54.121 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:30:54.121 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:30:54.121 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:30:54.121 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:30:54.121 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:54.121 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 115201 2024-09-14T06:30:54.122 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:54.122 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:30:54.122 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:54.122 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:30:54.122 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940113+0000", 2024-09-14T06:30:54.122 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:54.122 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:54.122 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:54.122 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:30:54.122 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940113+0000", 2024-09-14T06:30:54.122 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:54.122 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:54.122 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:54.122 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:30:54.122 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940114+0000", 2024-09-14T06:30:54.123 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.1500000000000004e-07 2024-09-14T06:30:54.123 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:54.123 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:54.123 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:30:54.123 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940116+0000", 2024-09-14T06:30:54.123 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.2809999999999998e-06 2024-09-14T06:30:54.123 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:54.123 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:54.123 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:30:54.123 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940123+0000", 2024-09-14T06:30:54.123 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.9229999999999996e-06 2024-09-14T06:30:54.123 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:54.123 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:54.123 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:30:54.123 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940128+0000", 2024-09-14T06:30:54.124 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.7790000000000003e-06 2024-09-14T06:30:54.124 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:54.124 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:54.124 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:30:54.124 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.015253+0000", 2024-09-14T06:30:54.124 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.075125792999999996 2024-09-14T06:30:54.124 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:54.124 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:54.124 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:30:54.124 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.015297+0000", 2024-09-14T06:30:54.124 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.3847000000000002e-05 2024-09-14T06:30:54.124 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:54.124 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:54.124 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:30:54.125 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.015360+0000", 2024-09-14T06:30:54.125 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.2661000000000002e-05 2024-09-14T06:30:54.125 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:54.125 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:54.125 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:54.125 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:54.125 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:54.125 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:115202 3.1 3:856be886:::rbd_data.112d4d24c558.0000000000000004:29 [read 3735552~16384 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e96)", 2024-09-14T06:30:54.125 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:30:17.940135+0000", 2024-09-14T06:30:54.125 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 36.142847324000002, 2024-09-14T06:30:54.125 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.075451105000000004, 2024-09-14T06:30:54.126 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:30:54.126 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:30:54.126 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:30:54.126 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:30:54.126 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:30:54.126 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:54.126 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 115202 2024-09-14T06:30:54.126 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:54.126 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:30:54.126 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:54.126 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:30:54.126 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940135+0000", 2024-09-14T06:30:54.126 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:54.126 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:54.126 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:54.127 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:30:54.127 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940135+0000", 2024-09-14T06:30:54.127 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:54.127 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:54.127 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:54.127 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:30:54.127 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940135+0000", 2024-09-14T06:30:54.127 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.0999999999999999e-07 2024-09-14T06:30:54.127 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:54.127 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:54.127 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:30:54.127 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940137+0000", 2024-09-14T06:30:54.127 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.0020000000000001e-06 2024-09-14T06:30:54.127 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:54.127 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:54.127 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:30:54.128 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940143+0000", 2024-09-14T06:30:54.128 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.9850000000000003e-06 2024-09-14T06:30:54.128 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:54.128 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:54.128 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:30:54.128 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940148+0000", 2024-09-14T06:30:54.128 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.9880000000000004e-06 2024-09-14T06:30:54.128 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:54.128 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:54.128 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:30:54.128 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.015400+0000", 2024-09-14T06:30:54.128 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.075252186999999998 2024-09-14T06:30:54.128 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:54.128 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:54.128 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:30:54.129 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.015481+0000", 2024-09-14T06:30:54.129 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.0698000000000006e-05 2024-09-14T06:30:54.129 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:54.129 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:54.129 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:30:54.129 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.015586+0000", 2024-09-14T06:30:54.129 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00010483499999999999 2024-09-14T06:30:54.129 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:54.129 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:54.129 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:54.129 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:54.129 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:54.129 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:115203 3.1 3:856be886:::rbd_data.112d4d24c558.0000000000000004:29 [read 3981312~16384 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e96)", 2024-09-14T06:30:54.129 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:30:17.940156+0000", 2024-09-14T06:30:54.129 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 36.142826098999997, 2024-09-14T06:30:54.130 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.075620860999999998, 2024-09-14T06:30:54.130 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:30:54.130 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:30:54.130 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:30:54.130 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:30:54.130 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:30:54.130 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:54.130 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 115203 2024-09-14T06:30:54.130 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:54.130 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:30:54.130 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:54.130 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:30:54.131 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940156+0000", 2024-09-14T06:30:54.131 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:54.131 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:54.131 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:54.131 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:30:54.131 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940156+0000", 2024-09-14T06:30:54.131 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:54.131 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:54.131 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:54.131 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:30:54.131 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940156+0000", 2024-09-14T06:30:54.131 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.1500000000000005e-07 2024-09-14T06:30:54.131 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:54.131 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:54.131 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:30:54.132 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940158+0000", 2024-09-14T06:30:54.132 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.967e-06 2024-09-14T06:30:54.132 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:54.132 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:54.132 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:30:54.132 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940165+0000", 2024-09-14T06:30:54.132 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.6939999999999998e-06 2024-09-14T06:30:54.132 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:54.132 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:54.132 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:30:54.132 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940170+0000", 2024-09-14T06:30:54.132 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.7539999999999997e-06 2024-09-14T06:30:54.132 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:54.132 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:54.133 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:30:54.133 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.015633+0000", 2024-09-14T06:30:54.133 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.075463334000000007 2024-09-14T06:30:54.133 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:54.133 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:54.133 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:30:54.133 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.015693+0000", 2024-09-14T06:30:54.133 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.9367000000000003e-05 2024-09-14T06:30:54.133 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:54.133 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:54.133 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:30:54.133 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.015777+0000", 2024-09-14T06:30:54.134 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.4229999999999998e-05 2024-09-14T06:30:54.134 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:54.134 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:54.134 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:54.134 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:54.134 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:54.134 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:115204 3.1 3:856be886:::rbd_data.112d4d24c558.0000000000000004:29 [read 4227072~16384 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e96)", 2024-09-14T06:30:54.134 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:30:17.940176+0000", 2024-09-14T06:30:54.135 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 36.142805566, 2024-09-14T06:30:54.135 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.075785505000000003, 2024-09-14T06:30:54.135 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:30:54.135 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:30:54.135 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:30:54.135 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:30:54.135 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:30:54.135 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:54.135 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 115204 2024-09-14T06:30:54.135 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:54.135 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:30:54.135 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:54.135 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:30:54.135 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940176+0000", 2024-09-14T06:30:54.136 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:54.136 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:54.136 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:54.136 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:30:54.136 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940176+0000", 2024-09-14T06:30:54.136 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:54.136 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:54.136 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:54.137 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:30:54.137 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940177+0000", 2024-09-14T06:30:54.137 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.7899999999999999e-07 2024-09-14T06:30:54.137 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:54.137 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:54.137 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:30:54.137 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940179+0000", 2024-09-14T06:30:54.137 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.103e-06 2024-09-14T06:30:54.137 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:54.137 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:54.137 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:30:54.137 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940185+0000", 2024-09-14T06:30:54.137 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.7239999999999997e-06 2024-09-14T06:30:54.138 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:54.138 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:54.138 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:30:54.138 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940190+0000", 2024-09-14T06:30:54.138 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.7269999999999998e-06 2024-09-14T06:30:54.138 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:54.138 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:54.138 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:30:54.138 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.015814+0000", 2024-09-14T06:30:54.138 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.075624114000000006 2024-09-14T06:30:54.138 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:54.138 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:54.138 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:30:54.138 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.015862+0000", 2024-09-14T06:30:54.139 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.8770999999999998e-05 2024-09-14T06:30:54.139 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:54.139 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:54.139 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:30:54.139 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.015962+0000", 2024-09-14T06:30:54.139 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.9586999999999994e-05 2024-09-14T06:30:54.139 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:54.139 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:54.139 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:54.139 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:54.139 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:54.139 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:115205 3.1 3:856be886:::rbd_data.112d4d24c558.0000000000000004:29 [read 4472832~16384 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e96)", 2024-09-14T06:30:54.139 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:30:17.940197+0000", 2024-09-14T06:30:54.139 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 36.142784857000002, 2024-09-14T06:30:54.140 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.076068920999999998, 2024-09-14T06:30:54.140 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:30:54.140 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:30:54.140 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:30:54.140 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:30:54.140 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:30:54.140 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:54.140 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 115205 2024-09-14T06:30:54.140 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:54.140 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:30:54.140 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:54.140 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:30:54.140 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940197+0000", 2024-09-14T06:30:54.140 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:54.140 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:54.141 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:54.141 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:30:54.141 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940197+0000", 2024-09-14T06:30:54.141 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:54.141 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:54.141 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:54.141 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:30:54.141 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940198+0000", 2024-09-14T06:30:54.141 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.5999999999999999e-07 2024-09-14T06:30:54.141 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:54.141 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:54.141 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:30:54.141 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940200+0000", 2024-09-14T06:30:54.142 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.0899999999999999e-06 2024-09-14T06:30:54.142 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:54.142 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:54.142 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:30:54.142 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940206+0000", 2024-09-14T06:30:54.142 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.8710000000000002e-06 2024-09-14T06:30:54.142 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:54.142 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:54.142 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:30:54.142 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940211+0000", 2024-09-14T06:30:54.142 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.9230000000000001e-06 2024-09-14T06:30:54.142 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:54.142 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:54.142 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:30:54.143 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.016059+0000", 2024-09-14T06:30:54.143 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.075848051999999999 2024-09-14T06:30:54.143 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:54.143 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:54.143 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:30:54.143 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.016151+0000", 2024-09-14T06:30:54.143 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.1982000000000001e-05 2024-09-14T06:30:54.143 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:54.143 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:54.143 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:30:54.143 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.016266+0000", 2024-09-14T06:30:54.143 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000115543 2024-09-14T06:30:54.143 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:54.143 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:54.144 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:54.144 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:54.144 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:54.144 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:115206 3.1 3:856be886:::rbd_data.112d4d24c558.0000000000000004:29 [read 4718592~16384 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e96)", 2024-09-14T06:30:54.144 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:30:17.940223+0000", 2024-09-14T06:30:54.144 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 36.142758766999997, 2024-09-14T06:30:54.144 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.076265115999999994, 2024-09-14T06:30:54.144 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:30:54.144 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:30:54.144 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:30:54.144 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:30:54.144 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:30:54.144 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:54.144 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 115206 2024-09-14T06:30:54.144 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:54.145 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:30:54.145 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:54.145 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:30:54.145 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940223+0000", 2024-09-14T06:30:54.145 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:54.145 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:54.145 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:54.145 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:30:54.145 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940223+0000", 2024-09-14T06:30:54.145 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:54.145 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:54.145 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:54.145 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:30:54.146 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940224+0000", 2024-09-14T06:30:54.146 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.0399999999999996e-07 2024-09-14T06:30:54.146 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:54.146 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:54.146 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:30:54.146 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940226+0000", 2024-09-14T06:30:54.146 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.1780000000000002e-06 2024-09-14T06:30:54.146 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:54.146 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:54.146 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:30:54.146 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940232+0000", 2024-09-14T06:30:54.146 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.5010000000000001e-06 2024-09-14T06:30:54.146 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:54.146 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:54.147 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:30:54.147 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940237+0000", 2024-09-14T06:30:54.147 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.0250000000000002e-06 2024-09-14T06:30:54.147 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:54.147 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:54.147 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:30:54.147 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.016305+0000", 2024-09-14T06:30:54.147 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.076067262999999996 2024-09-14T06:30:54.147 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:54.147 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:54.147 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:30:54.147 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.016383+0000", 2024-09-14T06:30:54.147 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.8578000000000006e-05 2024-09-14T06:30:54.147 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:54.147 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:54.148 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:30:54.148 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.016488+0000", 2024-09-14T06:30:54.148 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000105067 2024-09-14T06:30:54.148 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:54.148 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:54.148 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:54.148 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:54.148 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:54.148 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:115207 3.1 3:856be886:::rbd_data.112d4d24c558.0000000000000004:29 [read 4964352~16384 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e96)", 2024-09-14T06:30:54.148 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:30:17.940244+0000", 2024-09-14T06:30:54.148 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 36.142737650999997, 2024-09-14T06:30:54.148 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.076371154999999996, 2024-09-14T06:30:54.148 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:30:54.148 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:30:54.148 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:30:54.149 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:30:54.149 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:30:54.149 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:54.149 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 115207 2024-09-14T06:30:54.149 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:54.149 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:30:54.149 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:54.149 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:30:54.149 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940244+0000", 2024-09-14T06:30:54.149 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:54.149 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:54.149 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:54.149 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:30:54.149 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940244+0000", 2024-09-14T06:30:54.150 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:54.150 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:54.150 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:54.150 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:30:54.150 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940245+0000", 2024-09-14T06:30:54.150 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.1200000000000003e-07 2024-09-14T06:30:54.150 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:54.150 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:54.150 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:30:54.150 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940247+0000", 2024-09-14T06:30:54.150 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.0030000000000001e-06 2024-09-14T06:30:54.150 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:54.150 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:54.150 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:30:54.150 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940252+0000", 2024-09-14T06:30:54.151 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.5520000000000004e-06 2024-09-14T06:30:54.151 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:54.151 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:54.151 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:30:54.151 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940257+0000", 2024-09-14T06:30:54.151 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.6759999999999998e-06 2024-09-14T06:30:54.151 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:54.151 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:54.151 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:30:54.151 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.016521+0000", 2024-09-14T06:30:54.151 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.076263499999999998 2024-09-14T06:30:54.151 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:54.151 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:54.151 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:30:54.152 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.016562+0000", 2024-09-14T06:30:54.152 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.0967000000000003e-05 2024-09-14T06:30:54.152 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:54.152 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:54.152 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:30:54.152 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.016616+0000", 2024-09-14T06:30:54.152 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.3944999999999999e-05 2024-09-14T06:30:54.152 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:54.152 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:54.152 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:54.152 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:54.152 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:54.152 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:115208 3.1 3:856be886:::rbd_data.112d4d24c558.0000000000000004:29 [read 5210112~16384 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e96)", 2024-09-14T06:30:54.152 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:30:17.940264+0000", 2024-09-14T06:30:54.153 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 36.142717793999999, 2024-09-14T06:30:54.153 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.076558288000000002, 2024-09-14T06:30:54.153 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:30:54.153 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:30:54.153 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:30:54.153 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:30:54.153 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:30:54.153 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:54.153 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 115208 2024-09-14T06:30:54.153 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:54.153 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:30:54.153 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:54.153 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:30:54.154 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940264+0000", 2024-09-14T06:30:54.154 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:54.154 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:54.154 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:54.154 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:30:54.154 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940264+0000", 2024-09-14T06:30:54.154 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:54.154 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:54.154 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:54.154 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:30:54.154 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940265+0000", 2024-09-14T06:30:54.154 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.0500000000000004e-07 2024-09-14T06:30:54.154 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:54.155 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:54.155 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:30:54.155 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940267+0000", 2024-09-14T06:30:54.155 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.9810000000000002e-06 2024-09-14T06:30:54.155 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:54.155 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:54.155 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:30:54.155 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940272+0000", 2024-09-14T06:30:54.155 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.6999999999999996e-06 2024-09-14T06:30:54.155 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:54.155 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:54.156 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:30:54.156 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940277+0000", 2024-09-14T06:30:54.156 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.6480000000000002e-06 2024-09-14T06:30:54.156 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:54.156 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:54.156 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:30:54.156 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.016650+0000", 2024-09-14T06:30:54.156 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.076373089000000005 2024-09-14T06:30:54.156 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:54.156 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:54.156 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:30:54.156 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.016724+0000", 2024-09-14T06:30:54.156 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.4105999999999998e-05 2024-09-14T06:30:54.156 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:54.157 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:54.157 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:30:54.157 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.016823+0000", 2024-09-14T06:30:54.157 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.8258999999999996e-05 2024-09-14T06:30:54.157 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:54.157 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:54.157 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:54.157 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:54.157 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:54.157 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:115209 3.1 3:856be886:::rbd_data.112d4d24c558.0000000000000004:29 [read 5455872~16384 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e96)", 2024-09-14T06:30:54.157 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:30:17.940283+0000", 2024-09-14T06:30:54.157 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 36.142698768000002, 2024-09-14T06:30:54.157 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.076724029999999999, 2024-09-14T06:30:54.157 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:30:54.158 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:30:54.158 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:30:54.158 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:30:54.158 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:30:54.158 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:54.158 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 115209 2024-09-14T06:30:54.158 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:54.158 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:30:54.158 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:54.158 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:30:54.158 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940283+0000", 2024-09-14T06:30:54.158 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:54.158 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:54.158 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:54.159 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:30:54.159 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940283+0000", 2024-09-14T06:30:54.159 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:54.159 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:54.159 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:54.159 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:30:54.159 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940284+0000", 2024-09-14T06:30:54.159 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.6700000000000003e-07 2024-09-14T06:30:54.159 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:54.159 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:54.159 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:30:54.159 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940286+0000", 2024-09-14T06:30:54.159 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.9520000000000001e-06 2024-09-14T06:30:54.159 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:54.159 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:54.159 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:30:54.160 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940292+0000", 2024-09-14T06:30:54.160 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.7339999999999996e-06 2024-09-14T06:30:54.160 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:54.160 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:54.160 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:30:54.160 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940296+0000", 2024-09-14T06:30:54.160 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.7700000000000001e-06 2024-09-14T06:30:54.160 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:54.160 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:54.160 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:30:54.160 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.016859+0000", 2024-09-14T06:30:54.160 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.07656317 2024-09-14T06:30:54.160 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:54.160 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:54.160 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:30:54.161 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.016929+0000", 2024-09-14T06:30:54.161 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.9679999999999997e-05 2024-09-14T06:30:54.161 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:54.161 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:54.161 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:30:54.161 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.017007+0000", 2024-09-14T06:30:54.161 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.8157000000000004e-05 2024-09-14T06:30:54.161 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:54.161 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:54.161 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:54.161 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:54.161 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:54.161 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:115210 3.1 3:856be886:::rbd_data.112d4d24c558.0000000000000004:29 [read 5701632~16384 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e96)", 2024-09-14T06:30:54.162 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:30:17.940303+0000", 2024-09-14T06:30:54.162 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 36.142679250999997, 2024-09-14T06:30:54.162 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.076821358000000006, 2024-09-14T06:30:54.162 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:30:54.162 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:30:54.162 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:30:54.162 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:30:54.162 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:30:54.162 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:54.163 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 115210 2024-09-14T06:30:54.163 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:54.163 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:30:54.163 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:54.163 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:30:54.163 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940303+0000", 2024-09-14T06:30:54.163 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:54.163 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:54.164 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:54.164 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:30:54.164 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940303+0000", 2024-09-14T06:30:54.164 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:54.164 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:54.164 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:54.164 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:30:54.164 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940303+0000", 2024-09-14T06:30:54.164 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.3000000000000001e-07 2024-09-14T06:30:54.164 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:54.165 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:54.165 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:30:54.165 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940306+0000", 2024-09-14T06:30:54.165 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.255e-06 2024-09-14T06:30:54.165 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:54.165 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:54.165 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:30:54.165 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940312+0000", 2024-09-14T06:30:54.166 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.9710000000000001e-06 2024-09-14T06:30:54.166 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:54.166 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:54.166 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:30:54.166 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940316+0000", 2024-09-14T06:30:54.166 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.5340000000000001e-06 2024-09-14T06:30:54.166 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:54.166 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:54.167 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:30:54.167 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.017033+0000", 2024-09-14T06:30:54.167 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.076717169000000002 2024-09-14T06:30:54.167 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:54.167 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:54.167 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:30:54.167 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.017073+0000", 2024-09-14T06:30:54.167 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.9582000000000002e-05 2024-09-14T06:30:54.167 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:54.168 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:54.168 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:30:54.168 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.017124+0000", 2024-09-14T06:30:54.168 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.1316999999999997e-05 2024-09-14T06:30:54.168 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:54.168 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:54.168 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:54.168 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:54.168 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:54.169 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:115219 3.1 3:856be886:::rbd_data.112d4d24c558.0000000000000004:29 [read 65536~16384 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e96)", 2024-09-14T06:30:54.169 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:30:17.940323+0000", 2024-09-14T06:30:54.169 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 36.142659389999999, 2024-09-14T06:30:54.169 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.076915205, 2024-09-14T06:30:54.169 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:30:54.169 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:30:54.169 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:30:54.169 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:30:54.170 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:30:54.170 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:54.170 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 115219 2024-09-14T06:30:54.170 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:54.170 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:30:54.170 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:54.170 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:30:54.170 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940323+0000", 2024-09-14T06:30:54.170 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:54.171 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:54.171 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:54.171 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:30:54.171 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940323+0000", 2024-09-14T06:30:54.171 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:54.171 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:54.171 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:54.171 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:30:54.171 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940323+0000", 2024-09-14T06:30:54.171 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.0800000000000005e-07 2024-09-14T06:30:54.172 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:54.172 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:54.172 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:30:54.172 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940328+0000", 2024-09-14T06:30:54.172 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.887e-06 2024-09-14T06:30:54.172 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:54.172 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:54.172 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:30:54.173 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940334+0000", 2024-09-14T06:30:54.173 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.3269999999999997e-06 2024-09-14T06:30:54.173 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:54.173 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:54.173 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:30:54.173 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940339+0000", 2024-09-14T06:30:54.173 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.7960000000000003e-06 2024-09-14T06:30:54.173 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:54.174 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:54.174 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:30:54.174 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.017148+0000", 2024-09-14T06:30:54.174 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.076809125000000006 2024-09-14T06:30:54.174 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:54.174 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:54.174 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:30:54.174 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.017186+0000", 2024-09-14T06:30:54.174 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.7283000000000002e-05 2024-09-14T06:30:54.175 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:54.175 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:54.175 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:30:54.175 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.017238+0000", 2024-09-14T06:30:54.175 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.2278999999999999e-05 2024-09-14T06:30:54.175 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:54.175 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:54.175 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:54.175 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:54.176 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:54.176 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:115220 3.1 3:856be886:::rbd_data.112d4d24c558.0000000000000004:29 [read 311296~16384 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e96)", 2024-09-14T06:30:54.176 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:30:17.940346+0000", 2024-09-14T06:30:54.176 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 36.142636261, 2024-09-14T06:30:54.176 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.077033293000000003, 2024-09-14T06:30:54.176 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:30:54.176 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:30:54.176 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:30:54.177 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:30:54.177 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:30:54.177 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:54.177 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 115220 2024-09-14T06:30:54.177 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:54.177 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:30:54.177 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:54.177 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:30:54.177 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940346+0000", 2024-09-14T06:30:54.177 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:54.177 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:54.177 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:54.177 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:30:54.177 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940346+0000", 2024-09-14T06:30:54.178 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:54.178 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:54.178 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:54.178 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:30:54.178 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940346+0000", 2024-09-14T06:30:54.178 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.4300000000000003e-07 2024-09-14T06:30:54.178 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:54.178 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:54.178 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:30:54.178 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940348+0000", 2024-09-14T06:30:54.178 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.0030000000000001e-06 2024-09-14T06:30:54.178 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:54.178 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:54.178 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:30:54.178 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.004655+0000", 2024-09-14T06:30:54.178 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.064306323999999998 2024-09-14T06:30:54.179 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:54.179 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:54.179 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:30:54.179 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.004671+0000", 2024-09-14T06:30:54.179 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.6589999999999999e-05 2024-09-14T06:30:54.179 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:54.179 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:54.179 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:30:54.179 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.017262+0000", 2024-09-14T06:30:54.179 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.012590570000000001 2024-09-14T06:30:54.179 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:54.179 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:54.179 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:30:54.179 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.017321+0000", 2024-09-14T06:30:54.180 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.9210999999999999e-05 2024-09-14T06:30:54.180 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:54.180 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:54.180 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:30:54.180 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.017379+0000", 2024-09-14T06:30:54.180 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.8051999999999999e-05 2024-09-14T06:30:54.180 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:54.180 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:54.180 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:54.180 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:54.180 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:54.180 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-14T06:30:54.182 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-14T06:30:54.182 INFO:tasks.ceph.osd.6.smithi138.stderr:2024-09-14T06:30:54.180+0000 7f8cd8849640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33620) UID: 0 2024-09-14T06:30:54.282 INFO:tasks.ceph.osd.2.smithi094.stderr:2024-09-14T06:30:54.281+0000 7faf3269b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33649) UID: 0 2024-09-14T06:30:54.326 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-14T06:30:54.326 INFO:teuthology.orchestra.run.smithi094.stdout: "ops": [], 2024-09-14T06:30:54.326 INFO:teuthology.orchestra.run.smithi094.stdout: "num_ops": 0 2024-09-14T06:30:54.326 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-14T06:30:54.334 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-14T06:30:54.419 INFO:tasks.ceph.osd.2.smithi094.stderr:2024-09-14T06:30:54.418+0000 7faf3269b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33649) UID: 0 2024-09-14T06:30:54.477 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-14T06:30:54.477 INFO:teuthology.orchestra.run.smithi094.stdout: "ops": [], 2024-09-14T06:30:54.477 INFO:teuthology.orchestra.run.smithi094.stdout: "complaint_time": 30, 2024-09-14T06:30:54.477 INFO:teuthology.orchestra.run.smithi094.stdout: "num_blocked_ops": 0 2024-09-14T06:30:54.477 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-14T06:30:54.483 INFO:tasks.ceph.osd.5.smithi138.stderr:2024-09-14T06:30:54.482+0000 7fe13746f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33618) UID: 0 2024-09-14T06:30:54.485 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-14T06:30:54.585 INFO:tasks.ceph.osd.2.smithi094.stderr:2024-09-14T06:30:54.584+0000 7faf3269b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33649) UID: 0 2024-09-14T06:30:54.628 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-14T06:30:54.629 INFO:teuthology.orchestra.run.smithi094.stdout: "size": 20, 2024-09-14T06:30:54.629 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 600, 2024-09-14T06:30:54.629 INFO:teuthology.orchestra.run.smithi094.stdout: "ops": [ 2024-09-14T06:30:54.629 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:54.629 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:14918 3.0 3:05c39535:::rbd_data.112d92027590.0000000000000009:f [read 795648~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:30:54.630 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:26.083834+0000", 2024-09-14T06:30:54.630 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 88.543110239000001, 2024-09-14T06:30:54.630 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.041776287000000002, 2024-09-14T06:30:54.630 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:30:54.630 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:30:54.630 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:30:54.631 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:30:54.631 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:30:54.631 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:54.631 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 14918 2024-09-14T06:30:54.631 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:54.631 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:30:54.631 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:54.632 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:30:54.632 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.083834+0000", 2024-09-14T06:30:54.632 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:54.632 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:54.632 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:54.632 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:30:54.632 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.083834+0000", 2024-09-14T06:30:54.633 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:54.633 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:54.633 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:54.633 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:30:54.633 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.083834+0000", 2024-09-14T06:30:54.633 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.0100000000000001e-07 2024-09-14T06:30:54.633 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:54.633 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:54.634 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:30:54.634 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.083835+0000", 2024-09-14T06:30:54.634 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.3900000000000003e-07 2024-09-14T06:30:54.634 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:54.634 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:54.634 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:30:54.634 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.083838+0000", 2024-09-14T06:30:54.634 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.9830000000000001e-06 2024-09-14T06:30:54.634 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:54.635 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:54.635 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:30:54.635 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.083840+0000", 2024-09-14T06:30:54.635 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.4679999999999999e-06 2024-09-14T06:30:54.635 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:54.635 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:54.635 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:30:54.635 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.125455+0000", 2024-09-14T06:30:54.636 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.041614578999999999 2024-09-14T06:30:54.636 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:54.636 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:54.636 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:30:54.636 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.125496+0000", 2024-09-14T06:30:54.636 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.1047000000000002e-05 2024-09-14T06:30:54.636 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:54.637 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:54.637 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:30:54.637 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.125610+0000", 2024-09-14T06:30:54.637 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00011407 2024-09-14T06:30:54.637 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:54.637 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:54.637 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:54.638 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:54.638 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:54.638 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:14919 3.0 3:05c39535:::rbd_data.112d92027590.0000000000000009:f [read 805888~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:30:54.638 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:26.083843+0000", 2024-09-14T06:30:54.638 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 88.543100491000004, 2024-09-14T06:30:54.638 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.041992356000000002, 2024-09-14T06:30:54.638 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:30:54.638 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:30:54.639 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:30:54.639 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:30:54.639 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:30:54.639 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:54.639 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 14919 2024-09-14T06:30:54.639 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:54.639 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:30:54.639 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:54.640 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:30:54.640 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.083843+0000", 2024-09-14T06:30:54.640 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:54.640 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:54.640 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:54.640 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:30:54.640 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.083843+0000", 2024-09-14T06:30:54.641 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:54.641 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:54.641 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:54.641 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:30:54.641 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.083844+0000", 2024-09-14T06:30:54.641 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.3300000000000001e-07 2024-09-14T06:30:54.641 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:54.642 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:54.642 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:30:54.642 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.083845+0000", 2024-09-14T06:30:54.642 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.4729999999999999e-06 2024-09-14T06:30:54.642 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:54.642 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:54.642 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:30:54.642 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.083850+0000", 2024-09-14T06:30:54.642 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.0130000000000001e-06 2024-09-14T06:30:54.642 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:54.642 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:54.642 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:30:54.643 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.083854+0000", 2024-09-14T06:30:54.643 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.4759999999999998e-06 2024-09-14T06:30:54.643 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:54.643 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:54.643 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:30:54.643 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.125636+0000", 2024-09-14T06:30:54.643 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.041782683000000001 2024-09-14T06:30:54.643 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:54.643 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:54.643 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:30:54.643 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.125680+0000", 2024-09-14T06:30:54.643 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.3614e-05 2024-09-14T06:30:54.643 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:54.643 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:54.644 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:30:54.644 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.125836+0000", 2024-09-14T06:30:54.644 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000155864 2024-09-14T06:30:54.644 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:54.644 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:54.644 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:54.645 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:54.645 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:54.645 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:14920 3.0 3:05c39535:::rbd_data.112d92027590.0000000000000009:f [read 816128~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:30:54.645 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:26.083868+0000", 2024-09-14T06:30:54.645 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 88.543075830000006, 2024-09-14T06:30:54.645 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.042166771999999998, 2024-09-14T06:30:54.645 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:30:54.645 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:30:54.645 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:30:54.645 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:30:54.645 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:30:54.645 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:54.645 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 14920 2024-09-14T06:30:54.646 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:54.646 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:30:54.646 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:54.646 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:30:54.646 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.083868+0000", 2024-09-14T06:30:54.646 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:54.646 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:54.646 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:54.646 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:30:54.646 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.083868+0000", 2024-09-14T06:30:54.646 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:54.646 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:54.646 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:54.646 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:30:54.647 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.083869+0000", 2024-09-14T06:30:54.647 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.99e-07 2024-09-14T06:30:54.647 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:54.647 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:54.647 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:30:54.647 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.083870+0000", 2024-09-14T06:30:54.647 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.782e-06 2024-09-14T06:30:54.647 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:54.647 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:54.647 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:30:54.647 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.083876+0000", 2024-09-14T06:30:54.647 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.3809999999999998e-06 2024-09-14T06:30:54.647 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:54.647 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:54.648 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:30:54.648 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.083882+0000", 2024-09-14T06:30:54.648 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.9479999999999996e-06 2024-09-14T06:30:54.648 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:54.648 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:54.648 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:30:54.648 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.125878+0000", 2024-09-14T06:30:54.648 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.041996264999999998 2024-09-14T06:30:54.648 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:54.648 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:54.648 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:30:54.648 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.125929+0000", 2024-09-14T06:30:54.648 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.0498999999999999e-05 2024-09-14T06:30:54.648 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:54.649 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:54.649 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:30:54.649 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.126035+0000", 2024-09-14T06:30:54.649 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000106298 2024-09-14T06:30:54.649 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:54.649 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:54.649 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:54.649 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:54.649 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:54.649 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:14921 3.0 3:05c39535:::rbd_data.112d92027590.0000000000000009:f [read 826368~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:30:54.649 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:26.083889+0000", 2024-09-14T06:30:54.649 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 88.543054819999995, 2024-09-14T06:30:54.649 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.042435309999999997, 2024-09-14T06:30:54.650 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:30:54.650 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:30:54.650 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:30:54.650 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:30:54.650 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:30:54.650 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:54.650 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 14921 2024-09-14T06:30:54.650 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:54.650 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:30:54.650 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:54.650 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:30:54.650 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.083889+0000", 2024-09-14T06:30:54.650 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:54.650 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:54.651 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:54.651 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:30:54.651 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.083889+0000", 2024-09-14T06:30:54.651 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:54.651 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:54.651 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:54.651 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:30:54.651 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.086033+0000", 2024-09-14T06:30:54.651 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0021444179999999999 2024-09-14T06:30:54.651 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:54.651 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:54.651 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:30:54.651 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.086038+0000", 2024-09-14T06:30:54.651 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.0969999999999999e-06 2024-09-14T06:30:54.651 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:54.652 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:54.652 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:30:54.652 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.086046+0000", 2024-09-14T06:30:54.652 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.9440000000000005e-06 2024-09-14T06:30:54.652 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:54.652 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:54.652 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:30:54.652 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.086050+0000", 2024-09-14T06:30:54.652 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.69e-06 2024-09-14T06:30:54.652 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:54.652 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:54.652 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:30:54.652 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.126080+0000", 2024-09-14T06:30:54.652 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.040030237000000003 2024-09-14T06:30:54.653 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:54.653 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:54.653 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:30:54.653 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.126163+0000", 2024-09-14T06:30:54.653 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.2071999999999996e-05 2024-09-14T06:30:54.653 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:54.653 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:54.653 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:30:54.653 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.126324+0000", 2024-09-14T06:30:54.653 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00016185200000000001 2024-09-14T06:30:54.653 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:54.653 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:54.653 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:54.653 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:54.654 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:54.654 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:55627 3.0 3:06490baf:::rbd_data.112dca2f6ae1.0000000000000030:17 [read 2002944~4096 [fadvise_nocache]] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:30:54.654 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:44.599966+0000", 2024-09-14T06:30:54.654 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 70.026977872000003, 2024-09-14T06:30:54.654 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.041868546999999999, 2024-09-14T06:30:54.654 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:30:54.654 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:30:54.654 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:30:54.654 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:30:54.654 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:30:54.654 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:54.654 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 55627 2024-09-14T06:30:54.654 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:54.654 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:30:54.654 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:54.655 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:30:54.655 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.599966+0000", 2024-09-14T06:30:54.655 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:54.655 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:54.655 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:54.655 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:30:54.655 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.599966+0000", 2024-09-14T06:30:54.655 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:54.655 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:54.655 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:54.655 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:30:54.655 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.599966+0000", 2024-09-14T06:30:54.655 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.0900000000000001e-07 2024-09-14T06:30:54.656 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:54.656 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:54.656 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:30:54.656 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.599967+0000", 2024-09-14T06:30:54.656 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.64e-07 2024-09-14T06:30:54.656 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:54.656 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:54.656 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:30:54.656 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.599970+0000", 2024-09-14T06:30:54.656 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.9670000000000001e-06 2024-09-14T06:30:54.656 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:54.656 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:54.656 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:30:54.656 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.599973+0000", 2024-09-14T06:30:54.657 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.632e-06 2024-09-14T06:30:54.657 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:54.657 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:54.657 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:30:54.657 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.641665+0000", 2024-09-14T06:30:54.657 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.041692547000000003 2024-09-14T06:30:54.657 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:54.657 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:54.657 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:30:54.657 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.641740+0000", 2024-09-14T06:30:54.657 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.4559000000000002e-05 2024-09-14T06:30:54.657 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:54.657 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:54.658 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:30:54.658 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.641835+0000", 2024-09-14T06:30:54.658 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.4668999999999998e-05 2024-09-14T06:30:54.658 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:54.658 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:54.658 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:54.658 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:54.658 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:54.658 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:55628 3.0 3:06490baf:::rbd_data.112dca2f6ae1.0000000000000030:17 [read 2027520~4096 [fadvise_nocache]] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:30:54.658 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:44.599976+0000", 2024-09-14T06:30:54.658 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 70.026968084999993, 2024-09-14T06:30:54.658 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.041991621, 2024-09-14T06:30:54.658 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:30:54.659 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:30:54.659 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:30:54.659 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:30:54.659 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:30:54.659 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:54.659 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 55628 2024-09-14T06:30:54.659 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:54.659 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:30:54.659 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:54.659 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:30:54.659 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.599976+0000", 2024-09-14T06:30:54.659 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:54.659 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:54.659 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:54.660 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:30:54.660 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.599976+0000", 2024-09-14T06:30:54.660 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:54.660 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:54.660 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:54.660 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:30:54.660 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.599976+0000", 2024-09-14T06:30:54.660 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.1299999999999999e-07 2024-09-14T06:30:54.660 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:54.660 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:54.660 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:30:54.660 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.599977+0000", 2024-09-14T06:30:54.660 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.7999999999999993e-07 2024-09-14T06:30:54.660 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:54.661 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:54.661 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:30:54.661 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.599980+0000", 2024-09-14T06:30:54.661 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.01e-06 2024-09-14T06:30:54.661 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:54.661 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:54.661 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:30:54.661 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.599983+0000", 2024-09-14T06:30:54.661 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.5409999999999999e-06 2024-09-14T06:30:54.661 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:54.661 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:54.661 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:30:54.661 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.641869+0000", 2024-09-14T06:30:54.661 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.041886235000000001 2024-09-14T06:30:54.662 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:54.662 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:54.662 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:30:54.662 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.641913+0000", 2024-09-14T06:30:54.662 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.4357000000000002e-05 2024-09-14T06:30:54.662 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:54.662 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:54.662 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:30:54.662 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.641967+0000", 2024-09-14T06:30:54.662 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.4284999999999999e-05 2024-09-14T06:30:54.662 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:54.662 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:54.662 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:54.662 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:54.663 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:54.663 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:55629 3.0 3:06490baf:::rbd_data.112dca2f6ae1.0000000000000030:17 [read 2052096~4096 [fadvise_nocache]] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:30:54.663 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:44.599985+0000", 2024-09-14T06:30:54.663 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 70.026958379000007, 2024-09-14T06:30:54.663 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.042183375000000002, 2024-09-14T06:30:54.663 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:30:54.663 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:30:54.663 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:30:54.663 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:30:54.663 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:30:54.663 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:54.663 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 55629 2024-09-14T06:30:54.663 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:54.664 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:30:54.664 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:54.664 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:30:54.664 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.599985+0000", 2024-09-14T06:30:54.664 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:54.664 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:54.664 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:54.664 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:30:54.664 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.599985+0000", 2024-09-14T06:30:54.664 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:54.664 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:54.664 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:54.664 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:30:54.665 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.599986+0000", 2024-09-14T06:30:54.665 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.05e-07 2024-09-14T06:30:54.665 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:54.665 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:54.665 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:30:54.665 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.599987+0000", 2024-09-14T06:30:54.665 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.8400000000000002e-07 2024-09-14T06:30:54.665 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:54.665 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:54.665 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:30:54.665 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.599990+0000", 2024-09-14T06:30:54.665 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.5640000000000001e-06 2024-09-14T06:30:54.665 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:54.665 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:54.666 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:30:54.666 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.599993+0000", 2024-09-14T06:30:54.666 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.615e-06 2024-09-14T06:30:54.666 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:54.666 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:54.666 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:30:54.666 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.642003+0000", 2024-09-14T06:30:54.666 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.042010345999999997 2024-09-14T06:30:54.666 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:54.666 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:54.666 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:30:54.666 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.642080+0000", 2024-09-14T06:30:54.666 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.6538999999999993e-05 2024-09-14T06:30:54.667 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:54.667 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:54.667 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:30:54.667 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.642169+0000", 2024-09-14T06:30:54.667 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.9122000000000005e-05 2024-09-14T06:30:54.667 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:54.667 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:54.667 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:54.667 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:54.667 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:54.667 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:55630 3.0 3:06490baf:::rbd_data.112dca2f6ae1.0000000000000030:17 [read 2076672~4096 [fadvise_nocache]] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:30:54.667 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:44.599996+0000", 2024-09-14T06:30:54.667 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 70.026947901, 2024-09-14T06:30:54.668 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.042299615999999998, 2024-09-14T06:30:54.668 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:30:54.668 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:30:54.668 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:30:54.668 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:30:54.668 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:30:54.668 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:54.668 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 55630 2024-09-14T06:30:54.668 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:54.668 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:30:54.668 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:54.668 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:30:54.669 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.599996+0000", 2024-09-14T06:30:54.669 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:54.669 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:54.669 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:54.669 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:30:54.669 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.599996+0000", 2024-09-14T06:30:54.669 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:54.669 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:54.669 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:54.669 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:30:54.669 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.599996+0000", 2024-09-14T06:30:54.669 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.1500000000000001e-07 2024-09-14T06:30:54.669 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:54.669 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:54.670 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:30:54.670 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.599997+0000", 2024-09-14T06:30:54.670 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.33e-07 2024-09-14T06:30:54.670 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:54.670 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:54.670 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:30:54.670 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600000+0000", 2024-09-14T06:30:54.670 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.0240000000000002e-06 2024-09-14T06:30:54.670 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:54.670 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:54.670 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:30:54.670 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600003+0000", 2024-09-14T06:30:54.670 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.3640000000000002e-06 2024-09-14T06:30:54.670 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:54.671 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:54.671 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:30:54.671 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.642200+0000", 2024-09-14T06:30:54.671 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.042197571000000003 2024-09-14T06:30:54.671 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:54.671 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:54.671 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:30:54.671 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.642243+0000", 2024-09-14T06:30:54.671 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.2765000000000001e-05 2024-09-14T06:30:54.671 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:54.671 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:54.671 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:30:54.671 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.642296+0000", 2024-09-14T06:30:54.671 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.2744000000000002e-05 2024-09-14T06:30:54.672 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:54.672 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:54.672 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:54.672 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:54.672 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:54.672 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:55631 3.0 3:06490baf:::rbd_data.112dca2f6ae1.0000000000000030:17 [read 1048576~4096 [fadvise_nocache]] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:30:54.672 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:44.600006+0000", 2024-09-14T06:30:54.672 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 70.026938313000002, 2024-09-14T06:30:54.672 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.042487905999999999, 2024-09-14T06:30:54.672 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:30:54.672 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:30:54.672 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:30:54.672 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:30:54.673 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:30:54.673 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:54.673 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 55631 2024-09-14T06:30:54.673 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:54.673 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:30:54.673 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:54.673 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:30:54.673 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600006+0000", 2024-09-14T06:30:54.673 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:54.673 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:54.673 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:54.673 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:30:54.673 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600006+0000", 2024-09-14T06:30:54.674 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:54.674 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:54.674 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:54.674 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:30:54.674 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600006+0000", 2024-09-14T06:30:54.674 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.0800000000000001e-07 2024-09-14T06:30:54.674 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:54.674 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:54.674 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:30:54.674 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600007+0000", 2024-09-14T06:30:54.674 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.7199999999999997e-07 2024-09-14T06:30:54.674 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:54.674 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:54.674 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:30:54.675 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600010+0000", 2024-09-14T06:30:54.675 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.0280000000000001e-06 2024-09-14T06:30:54.675 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:54.675 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:54.675 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:30:54.675 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600012+0000", 2024-09-14T06:30:54.675 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.475e-06 2024-09-14T06:30:54.675 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:54.675 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:54.675 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:30:54.675 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.642329+0000", 2024-09-14T06:30:54.675 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.042317157000000001 2024-09-14T06:30:54.675 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:54.675 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:54.675 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:30:54.676 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.642403+0000", 2024-09-14T06:30:54.676 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.3941000000000002e-05 2024-09-14T06:30:54.676 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:54.676 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:54.676 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:30:54.676 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.642493+0000", 2024-09-14T06:30:54.676 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.0124999999999995e-05 2024-09-14T06:30:54.676 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:54.676 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:54.676 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:54.676 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:54.676 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:54.676 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:55632 3.0 3:06490baf:::rbd_data.112dca2f6ae1.0000000000000030:17 [read 1073152~4096 [fadvise_nocache]] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:30:54.676 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:44.600015+0000", 2024-09-14T06:30:54.677 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 70.026928560000002, 2024-09-14T06:30:54.677 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.042604922000000003, 2024-09-14T06:30:54.677 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:30:54.677 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:30:54.677 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:30:54.677 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:30:54.677 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:30:54.677 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:54.677 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 55632 2024-09-14T06:30:54.677 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:54.677 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:30:54.677 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:54.677 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:30:54.678 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600015+0000", 2024-09-14T06:30:54.678 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:54.678 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:54.678 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:54.678 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:30:54.678 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600015+0000", 2024-09-14T06:30:54.678 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:54.678 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:54.678 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:54.678 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:30:54.678 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600016+0000", 2024-09-14T06:30:54.678 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.1e-07 2024-09-14T06:30:54.678 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:54.679 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:54.679 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:30:54.679 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600017+0000", 2024-09-14T06:30:54.679 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.9900000000000009e-07 2024-09-14T06:30:54.679 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:54.679 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:54.679 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:30:54.679 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600020+0000", 2024-09-14T06:30:54.679 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.4170000000000001e-06 2024-09-14T06:30:54.680 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:54.680 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:54.680 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:30:54.680 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600023+0000", 2024-09-14T06:30:54.680 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.588e-06 2024-09-14T06:30:54.680 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:54.680 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:54.680 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:30:54.680 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.642526+0000", 2024-09-14T06:30:54.680 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.042503236999999999 2024-09-14T06:30:54.680 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:54.680 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:54.680 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:30:54.681 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.642568+0000", 2024-09-14T06:30:54.681 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.1925000000000003e-05 2024-09-14T06:30:54.681 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:54.681 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:54.681 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:30:54.681 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.642620+0000", 2024-09-14T06:30:54.681 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.2546000000000001e-05 2024-09-14T06:30:54.681 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:54.681 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:54.681 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:54.681 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:54.681 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:54.681 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:55633 3.0 3:06490baf:::rbd_data.112dca2f6ae1.0000000000000030:17 [read 1097728~4096 [fadvise_nocache]] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:30:54.682 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:44.600026+0000", 2024-09-14T06:30:54.682 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 70.026918291000001, 2024-09-14T06:30:54.682 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.042797474000000002, 2024-09-14T06:30:54.682 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:30:54.682 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:30:54.682 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:30:54.682 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:30:54.682 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:30:54.682 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:54.682 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 55633 2024-09-14T06:30:54.682 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:54.682 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:30:54.682 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:54.682 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:30:54.683 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600026+0000", 2024-09-14T06:30:54.683 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:54.683 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:54.683 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:54.683 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:30:54.683 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600026+0000", 2024-09-14T06:30:54.683 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:54.683 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:54.683 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:54.683 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:30:54.683 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600026+0000", 2024-09-14T06:30:54.683 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.3099999999999999e-07 2024-09-14T06:30:54.683 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:54.684 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:54.684 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:30:54.684 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600029+0000", 2024-09-14T06:30:54.684 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.304e-06 2024-09-14T06:30:54.684 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:54.684 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:54.684 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:30:54.684 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600034+0000", 2024-09-14T06:30:54.684 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.1429999999999997e-06 2024-09-14T06:30:54.684 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:54.684 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:54.684 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:30:54.684 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600038+0000", 2024-09-14T06:30:54.684 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.692e-06 2024-09-14T06:30:54.685 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:54.685 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:54.685 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:30:54.685 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.642654+0000", 2024-09-14T06:30:54.685 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.042615564000000002 2024-09-14T06:30:54.685 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:54.685 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:54.685 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:30:54.685 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.642728+0000", 2024-09-14T06:30:54.685 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.4018999999999997e-05 2024-09-14T06:30:54.685 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:54.685 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:54.685 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:30:54.686 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.642823+0000", 2024-09-14T06:30:54.686 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.5520999999999996e-05 2024-09-14T06:30:54.686 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:54.686 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:54.686 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:54.686 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:54.686 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:54.686 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:55634 3.0 3:06490baf:::rbd_data.112dca2f6ae1.0000000000000030:17 [read 1122304~4096 [fadvise_nocache]] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:30:54.686 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:44.600042+0000", 2024-09-14T06:30:54.686 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 70.026901813999999, 2024-09-14T06:30:54.687 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.042903729000000002, 2024-09-14T06:30:54.687 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:30:54.687 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:30:54.687 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:30:54.687 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:30:54.687 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:30:54.687 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:54.687 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 55634 2024-09-14T06:30:54.687 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:54.687 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:30:54.687 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:54.687 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:30:54.687 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600042+0000", 2024-09-14T06:30:54.688 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:54.688 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:54.688 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:54.688 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:30:54.688 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600042+0000", 2024-09-14T06:30:54.688 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:54.688 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:54.688 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:54.688 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:30:54.688 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600043+0000", 2024-09-14T06:30:54.688 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.7000000000000005e-07 2024-09-14T06:30:54.688 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:54.688 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:54.689 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:30:54.689 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600044+0000", 2024-09-14T06:30:54.689 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.742e-06 2024-09-14T06:30:54.689 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:54.689 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:54.689 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:30:54.689 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600050+0000", 2024-09-14T06:30:54.689 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.0800000000000002e-06 2024-09-14T06:30:54.689 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:54.689 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:54.689 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:30:54.689 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600055+0000", 2024-09-14T06:30:54.689 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.3889999999999998e-06 2024-09-14T06:30:54.689 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:54.690 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:54.690 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:30:54.690 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.642845+0000", 2024-09-14T06:30:54.690 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.042790040000000001 2024-09-14T06:30:54.690 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:54.690 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:54.690 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:30:54.690 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.642888+0000", 2024-09-14T06:30:54.690 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.2790000000000002e-05 2024-09-14T06:30:54.690 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:54.690 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:54.690 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:30:54.690 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.642946+0000", 2024-09-14T06:30:54.690 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.8118000000000002e-05 2024-09-14T06:30:54.691 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:54.691 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:54.691 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:54.691 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:54.691 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:54.691 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:55635 3.0 3:06490baf:::rbd_data.112dca2f6ae1.0000000000000030:17 [read 1146880~4096 [fadvise_nocache]] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:30:54.691 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:44.600061+0000", 2024-09-14T06:30:54.691 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 70.026883243, 2024-09-14T06:30:54.691 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.043092695, 2024-09-14T06:30:54.691 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:30:54.691 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:30:54.691 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:30:54.691 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:30:54.691 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:30:54.692 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:54.692 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 55635 2024-09-14T06:30:54.692 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:54.692 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:30:54.692 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:54.692 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:30:54.692 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600061+0000", 2024-09-14T06:30:54.692 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:54.692 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:54.692 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:54.692 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:30:54.692 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600061+0000", 2024-09-14T06:30:54.692 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:54.693 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:54.693 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:54.693 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:30:54.693 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600061+0000", 2024-09-14T06:30:54.693 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.3099999999999999e-07 2024-09-14T06:30:54.693 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:54.693 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:54.693 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:30:54.693 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600063+0000", 2024-09-14T06:30:54.693 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.6220000000000001e-06 2024-09-14T06:30:54.693 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:54.693 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:54.693 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:30:54.694 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600068+0000", 2024-09-14T06:30:54.694 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.4790000000000004e-06 2024-09-14T06:30:54.694 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:54.694 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:54.694 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:30:54.694 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600072+0000", 2024-09-14T06:30:54.694 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.3100000000000002e-06 2024-09-14T06:30:54.694 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:54.694 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:54.694 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:30:54.694 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.642987+0000", 2024-09-14T06:30:54.694 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.042914489 2024-09-14T06:30:54.694 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:54.695 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:54.695 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:30:54.695 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.643062+0000", 2024-09-14T06:30:54.695 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.5011000000000004e-05 2024-09-14T06:30:54.695 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:54.695 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:54.695 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:30:54.695 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.643153+0000", 2024-09-14T06:30:54.695 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.1452999999999993e-05 2024-09-14T06:30:54.695 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:54.695 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:54.695 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:54.695 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:54.695 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:54.696 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:55636 3.0 3:06490baf:::rbd_data.112dca2f6ae1.0000000000000030:17 [read 1171456~4096 [fadvise_nocache]] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:30:54.696 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:44.600078+0000", 2024-09-14T06:30:54.696 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 70.026865830999995, 2024-09-14T06:30:54.696 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.043207966, 2024-09-14T06:30:54.696 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:30:54.696 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:30:54.696 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:30:54.696 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:30:54.696 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:30:54.696 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:54.696 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 55636 2024-09-14T06:30:54.696 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:54.696 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:30:54.697 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:54.697 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:30:54.697 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600078+0000", 2024-09-14T06:30:54.697 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:54.697 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:54.697 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:54.697 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:30:54.697 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600078+0000", 2024-09-14T06:30:54.697 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:54.697 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:54.697 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:54.697 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:30:54.697 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600078+0000", 2024-09-14T06:30:54.697 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.34e-07 2024-09-14T06:30:54.698 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:54.698 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:54.698 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:30:54.698 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600080+0000", 2024-09-14T06:30:54.698 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.641e-06 2024-09-14T06:30:54.698 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:54.698 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:54.698 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:30:54.698 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600086+0000", 2024-09-14T06:30:54.698 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.3860000000000004e-06 2024-09-14T06:30:54.698 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:54.698 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:54.698 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:30:54.698 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600091+0000", 2024-09-14T06:30:54.698 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.5970000000000002e-06 2024-09-14T06:30:54.699 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:54.699 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:54.699 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:30:54.699 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.643184+0000", 2024-09-14T06:30:54.699 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.043092893 2024-09-14T06:30:54.699 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:54.699 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:54.699 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:30:54.699 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.643228+0000", 2024-09-14T06:30:54.699 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.4536999999999997e-05 2024-09-14T06:30:54.699 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:54.699 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:54.699 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:30:54.699 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.643286+0000", 2024-09-14T06:30:54.699 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.7578000000000003e-05 2024-09-14T06:30:54.699 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:54.700 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:54.700 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:54.700 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:54.700 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:54.700 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:55637 3.0 3:06490baf:::rbd_data.112dca2f6ae1.0000000000000030:17 [read 1196032~4096 [fadvise_nocache]] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:30:54.700 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:44.600097+0000", 2024-09-14T06:30:54.700 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 70.026847266999994, 2024-09-14T06:30:54.700 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.043399124999999997, 2024-09-14T06:30:54.700 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:30:54.700 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:30:54.700 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:30:54.700 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:30:54.700 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:30:54.700 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:54.701 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 55637 2024-09-14T06:30:54.701 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:54.701 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:30:54.701 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:54.701 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:30:54.701 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600097+0000", 2024-09-14T06:30:54.701 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:54.701 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:54.701 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:54.701 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:30:54.701 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600097+0000", 2024-09-14T06:30:54.701 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:54.701 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:54.702 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:54.702 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:30:54.702 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.604372+0000", 2024-09-14T06:30:54.702 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0042751489999999998 2024-09-14T06:30:54.702 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:54.702 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:54.702 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:30:54.702 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.604377+0000", 2024-09-14T06:30:54.702 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.7899999999999999e-06 2024-09-14T06:30:54.702 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:54.702 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:54.702 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:30:54.702 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.604389+0000", 2024-09-14T06:30:54.702 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.2197e-05 2024-09-14T06:30:54.703 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:54.703 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:54.703 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:30:54.703 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.604397+0000", 2024-09-14T06:30:54.703 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.8730000000000007e-06 2024-09-14T06:30:54.703 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:54.703 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:54.703 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:30:54.703 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.643324+0000", 2024-09-14T06:30:54.703 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.038926897000000002 2024-09-14T06:30:54.703 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:54.703 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:54.703 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:30:54.704 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.643402+0000", 2024-09-14T06:30:54.704 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.8063000000000001e-05 2024-09-14T06:30:54.704 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:54.704 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:54.704 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:30:54.704 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.643496+0000", 2024-09-14T06:30:54.704 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.4155999999999998e-05 2024-09-14T06:30:54.704 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:54.704 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:54.704 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:54.704 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:54.704 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:54.704 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:55657 3.0 3:06490baf:::rbd_data.112dca2f6ae1.0000000000000030:17 [read 1687552~4096 [fadvise_nocache]] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:30:54.704 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:44.604841+0000", 2024-09-14T06:30:54.705 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 70.022102537999999, 2024-09-14T06:30:54.705 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.041890087999999999, 2024-09-14T06:30:54.705 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:30:54.705 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:30:54.705 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:30:54.705 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:30:54.705 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:30:54.705 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:54.705 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 55657 2024-09-14T06:30:54.705 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:54.705 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:30:54.705 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:54.705 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:30:54.705 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.604841+0000", 2024-09-14T06:30:54.706 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:54.706 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:54.706 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:54.706 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:30:54.706 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.604841+0000", 2024-09-14T06:30:54.706 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:54.706 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:54.706 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:54.706 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:30:54.706 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.604842+0000", 2024-09-14T06:30:54.706 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.0900000000000001e-07 2024-09-14T06:30:54.706 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:54.706 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:54.706 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:30:54.706 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.604846+0000", 2024-09-14T06:30:54.706 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.5759999999999999e-06 2024-09-14T06:30:54.707 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:54.707 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:54.707 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:30:54.707 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.604850+0000", 2024-09-14T06:30:54.707 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.405e-06 2024-09-14T06:30:54.707 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:54.707 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:54.707 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:30:54.707 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.604852+0000", 2024-09-14T06:30:54.707 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.8590000000000001e-06 2024-09-14T06:30:54.707 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:54.707 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:54.707 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:30:54.708 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.646633+0000", 2024-09-14T06:30:54.708 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.041780772000000001 2024-09-14T06:30:54.708 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:54.708 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:54.708 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:30:54.708 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.646677+0000", 2024-09-14T06:30:54.708 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.4115e-05 2024-09-14T06:30:54.708 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:54.708 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:54.708 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:30:54.708 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.646731+0000", 2024-09-14T06:30:54.708 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.4151999999999999e-05 2024-09-14T06:30:54.708 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:54.708 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:54.709 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:54.709 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:54.709 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:54.709 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:55658 3.0 3:06490baf:::rbd_data.112dca2f6ae1.0000000000000030:17 [read 1712128~4096 [fadvise_nocache]] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:30:54.709 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:44.604856+0000", 2024-09-14T06:30:54.709 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 70.022087937999999, 2024-09-14T06:30:54.709 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.041986227000000001, 2024-09-14T06:30:54.709 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:30:54.709 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:30:54.709 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:30:54.710 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:30:54.710 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:30:54.710 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:54.710 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 55658 2024-09-14T06:30:54.710 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:54.710 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:30:54.710 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:54.710 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:30:54.710 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.604856+0000", 2024-09-14T06:30:54.710 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:54.710 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:54.710 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:54.710 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:30:54.710 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.604856+0000", 2024-09-14T06:30:54.710 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:54.710 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:54.711 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:54.711 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:30:54.711 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.604856+0000", 2024-09-14T06:30:54.711 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.23e-07 2024-09-14T06:30:54.711 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:54.711 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:54.711 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:30:54.711 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.604857+0000", 2024-09-14T06:30:54.711 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.9999999999999995e-07 2024-09-14T06:30:54.711 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:54.711 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:54.711 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:30:54.711 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.604861+0000", 2024-09-14T06:30:54.711 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.9890000000000003e-06 2024-09-14T06:30:54.711 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:54.712 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:54.712 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:30:54.712 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.604864+0000", 2024-09-14T06:30:54.712 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.7429999999999998e-06 2024-09-14T06:30:54.712 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:54.712 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:54.712 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:30:54.712 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.646744+0000", 2024-09-14T06:30:54.712 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.041880325000000003 2024-09-14T06:30:54.712 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:54.712 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:54.712 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:30:54.712 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.646784+0000", 2024-09-14T06:30:54.713 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.9604000000000002e-05 2024-09-14T06:30:54.713 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:54.713 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:54.713 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:30:54.713 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.646842+0000", 2024-09-14T06:30:54.713 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.8343e-05 2024-09-14T06:30:54.713 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:54.713 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:54.713 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:54.713 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:54.713 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:54.713 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:55659 3.0 3:06490baf:::rbd_data.112dca2f6ae1.0000000000000030:17 [read 1736704~4096 [fadvise_nocache]] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:30:54.713 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:44.604867+0000", 2024-09-14T06:30:54.714 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 70.022076784999996, 2024-09-14T06:30:54.714 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.042097082000000001, 2024-09-14T06:30:54.714 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:30:54.714 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:30:54.714 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:30:54.714 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:30:54.714 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:30:54.714 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:54.714 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 55659 2024-09-14T06:30:54.714 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:54.714 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:30:54.714 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:54.714 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:30:54.714 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.604867+0000", 2024-09-14T06:30:54.714 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:54.715 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:54.715 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:54.715 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:30:54.715 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.604867+0000", 2024-09-14T06:30:54.715 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:54.715 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:54.715 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:54.715 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:30:54.715 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.604867+0000", 2024-09-14T06:30:54.715 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.2000000000000001e-07 2024-09-14T06:30:54.715 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:54.715 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:54.715 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:30:54.715 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.604868+0000", 2024-09-14T06:30:54.716 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.04e-06 2024-09-14T06:30:54.716 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:54.716 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:54.716 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:30:54.716 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.604871+0000", 2024-09-14T06:30:54.716 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.9689999999999999e-06 2024-09-14T06:30:54.716 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:54.716 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:54.716 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:30:54.716 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.604874+0000", 2024-09-14T06:30:54.716 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.5069999999999999e-06 2024-09-14T06:30:54.716 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:54.716 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:54.716 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:30:54.716 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.646874+0000", 2024-09-14T06:30:54.717 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.042000513000000003 2024-09-14T06:30:54.717 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:54.717 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:54.717 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:30:54.717 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.646914+0000", 2024-09-14T06:30:54.717 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.9443999999999997e-05 2024-09-14T06:30:54.717 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:54.717 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:54.717 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:30:54.717 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.646964+0000", 2024-09-14T06:30:54.717 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.0389000000000002e-05 2024-09-14T06:30:54.717 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:54.717 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:54.717 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:54.718 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:54.718 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:54.718 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:55660 3.0 3:06490baf:::rbd_data.112dca2f6ae1.0000000000000030:17 [read 1761280~4096 [fadvise_nocache]] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:30:54.718 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:44.604877+0000", 2024-09-14T06:30:54.718 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 70.022067035999996, 2024-09-14T06:30:54.718 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.042196349000000001, 2024-09-14T06:30:54.718 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:30:54.718 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:30:54.718 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:30:54.718 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:30:54.718 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:30:54.718 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:54.718 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 55660 2024-09-14T06:30:54.718 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:54.718 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:30:54.719 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:54.719 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:30:54.719 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.604877+0000", 2024-09-14T06:30:54.719 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:54.719 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:54.719 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:54.719 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:30:54.719 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.604877+0000", 2024-09-14T06:30:54.719 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:54.719 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:54.719 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:54.719 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:30:54.719 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.605810+0000", 2024-09-14T06:30:54.719 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00093336899999999997 2024-09-14T06:30:54.719 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:54.719 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:54.720 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:30:54.720 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.605815+0000", 2024-09-14T06:30:54.720 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.6140000000000002e-06 2024-09-14T06:30:54.720 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:54.720 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:54.720 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:30:54.720 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.605825+0000", 2024-09-14T06:30:54.720 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.8889999999999997e-06 2024-09-14T06:30:54.720 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:54.720 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:54.720 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:30:54.720 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.605831+0000", 2024-09-14T06:30:54.720 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.9800000000000003e-06 2024-09-14T06:30:54.720 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:54.721 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:54.721 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:30:54.721 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.646989+0000", 2024-09-14T06:30:54.721 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.041157878000000002 2024-09-14T06:30:54.721 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:54.721 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:54.721 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:30:54.721 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.647026+0000", 2024-09-14T06:30:54.721 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.7305000000000003e-05 2024-09-14T06:30:54.721 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:54.721 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:54.721 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:30:54.721 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.647073+0000", 2024-09-14T06:30:54.722 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.7314000000000001e-05 2024-09-14T06:30:54.722 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:54.722 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:54.722 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:54.722 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:54.722 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:54.722 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:55666 3.0 3:06490baf:::rbd_data.112dca2f6ae1.0000000000000030:17 [read 1908736~4096 [fadvise_nocache]] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:30:54.722 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:44.605898+0000", 2024-09-14T06:30:54.722 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 70.021046290000001, 2024-09-14T06:30:54.722 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.041862887000000001, 2024-09-14T06:30:54.722 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:30:54.722 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:30:54.722 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:30:54.723 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:30:54.723 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:30:54.723 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:54.723 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 55666 2024-09-14T06:30:54.723 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:54.723 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:30:54.723 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:54.723 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:30:54.723 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.605898+0000", 2024-09-14T06:30:54.723 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:54.723 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:54.723 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:54.723 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:30:54.723 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.605898+0000", 2024-09-14T06:30:54.724 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:54.724 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:54.724 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:54.724 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:30:54.724 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.611874+0000", 2024-09-14T06:30:54.724 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0059764049999999997 2024-09-14T06:30:54.724 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:54.724 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:54.724 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:30:54.724 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.611879+0000", 2024-09-14T06:30:54.724 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.8749999999999999e-06 2024-09-14T06:30:54.724 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:54.724 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:54.725 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:30:54.725 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.611887+0000", 2024-09-14T06:30:54.725 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.4109999999999996e-06 2024-09-14T06:30:54.725 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:54.725 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:54.725 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:30:54.725 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.611893+0000", 2024-09-14T06:30:54.725 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.2959999999999998e-06 2024-09-14T06:30:54.725 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:54.725 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:54.725 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:30:54.725 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.647674+0000", 2024-09-14T06:30:54.725 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.035781281999999998 2024-09-14T06:30:54.725 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:54.726 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:54.726 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:30:54.726 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.647711+0000", 2024-09-14T06:30:54.726 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.7434000000000001e-05 2024-09-14T06:30:54.726 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:54.726 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:54.726 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:30:54.726 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.647760+0000", 2024-09-14T06:30:54.726 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.9184000000000002e-05 2024-09-14T06:30:54.726 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:54.726 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:54.726 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:54.726 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:54.726 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:54.726 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-14T06:30:54.728 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-14T06:30:54.729 INFO:tasks.ceph.osd.7.smithi138.stderr:2024-09-14T06:30:54.727+0000 7f3c6a21d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33623) UID: 0 2024-09-14T06:30:54.747 INFO:teuthology.orchestra.run.smithi094.stderr:noscrub is unset 2024-09-14T06:30:54.828 INFO:tasks.ceph.osd.3.smithi094.stderr:2024-09-14T06:30:54.827+0000 7feeeafe8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33655) UID: 0 2024-09-14T06:30:54.884 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-14T06:30:54.884 INFO:teuthology.orchestra.run.smithi094.stdout: "ops": [], 2024-09-14T06:30:54.884 INFO:teuthology.orchestra.run.smithi094.stdout: "num_ops": 0 2024-09-14T06:30:54.885 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-14T06:30:54.893 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-14T06:30:54.929 INFO:tasks.ceph.osd.6.smithi138.stderr:2024-09-14T06:30:54.928+0000 7f8cd8849640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33620) UID: 0 2024-09-14T06:30:55.030 INFO:tasks.ceph.osd.6.smithi138.stderr:2024-09-14T06:30:55.029+0000 7f8cd8849640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33620) UID: 0 2024-09-14T06:30:55.035 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-14T06:30:55.035 INFO:teuthology.orchestra.run.smithi094.stdout: "ops": [], 2024-09-14T06:30:55.036 INFO:teuthology.orchestra.run.smithi094.stdout: "complaint_time": 30, 2024-09-14T06:30:55.036 INFO:teuthology.orchestra.run.smithi094.stdout: "num_blocked_ops": 0 2024-09-14T06:30:55.036 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-14T06:30:55.044 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-14T06:30:55.088 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-14T06:30:55.130 INFO:tasks.ceph.osd.7.smithi138.stderr:2024-09-14T06:30:55.129+0000 7f3c6a21d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33623) UID: 0 2024-09-14T06:30:55.151 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-14T06:30:55.151 INFO:teuthology.orchestra.run.smithi094.stdout: "size": 20, 2024-09-14T06:30:55.151 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 600, 2024-09-14T06:30:55.151 INFO:teuthology.orchestra.run.smithi094.stdout: "ops": [ 2024-09-14T06:30:55.151 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:55.151 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4397.0:26415 3.0 e53/23 3:05fd9be0:::rbd_data.112d38768680.0000000000000006:head v 53'21, mlcod=50'19)", 2024-09-14T06:30:55.151 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:30.721610+0000", 2024-09-14T06:30:55.152 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 84.427227152, 2024-09-14T06:30:55.152 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0021763160000000002, 2024-09-14T06:30:55.152 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:30:55.152 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:30:55.152 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:30:55.152 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:30:55.152 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:55.152 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:30:55.152 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:30.721610+0000", 2024-09-14T06:30:55.152 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:55.152 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:55.152 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:55.152 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:30:55.152 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:30.721610+0000", 2024-09-14T06:30:55.153 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:55.153 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:55.153 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:55.153 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:30:55.153 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:30.721611+0000", 2024-09-14T06:30:55.153 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.41e-07 2024-09-14T06:30:55.153 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:55.153 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:55.153 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:30:55.153 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:30.721905+0000", 2024-09-14T06:30:55.153 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00029394300000000001 2024-09-14T06:30:55.153 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:55.153 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:55.153 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:30:55.153 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:30.721921+0000", 2024-09-14T06:30:55.153 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.6609999999999999e-05 2024-09-14T06:30:55.154 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:55.154 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:55.154 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:30:55.154 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:30.721931+0000", 2024-09-14T06:30:55.154 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.6739999999999992e-06 2024-09-14T06:30:55.154 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:55.154 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:55.154 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:30:55.154 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:30.722001+0000", 2024-09-14T06:30:55.154 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.0549000000000005e-05 2024-09-14T06:30:55.154 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:55.154 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:55.154 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:30:55.154 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:30.722029+0000", 2024-09-14T06:30:55.154 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.7327000000000001e-05 2024-09-14T06:30:55.154 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:55.154 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:55.155 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-14T06:30:55.155 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:30.723727+0000", 2024-09-14T06:30:55.155 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0016977310000000001 2024-09-14T06:30:55.155 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:55.155 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:55.155 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:30:55.155 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:30.723787+0000", 2024-09-14T06:30:55.155 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.0140999999999999e-05 2024-09-14T06:30:55.155 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:55.155 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:55.155 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:55.155 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:55.155 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:55.155 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4397.0:36084 3.0 e56/23 3:05a48ef4:::rbd_data.112dc58639fa.0000000000000058:head v 56'29, mlcod=56'27)", 2024-09-14T06:30:55.156 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:34.508855+0000", 2024-09-14T06:30:55.156 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 80.639982398000001, 2024-09-14T06:30:55.156 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0020801140000000001, 2024-09-14T06:30:55.156 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:30:55.156 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:30:55.156 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:30:55.156 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:30:55.156 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:55.156 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:30:55.156 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:34.508855+0000", 2024-09-14T06:30:55.156 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:55.156 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:55.156 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:55.156 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:30:55.156 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:34.508855+0000", 2024-09-14T06:30:55.156 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:55.157 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:55.157 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:55.157 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:30:55.157 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:34.508856+0000", 2024-09-14T06:30:55.157 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.1909999999999999e-06 2024-09-14T06:30:55.157 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:55.157 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:55.157 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:30:55.157 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:34.509203+0000", 2024-09-14T06:30:55.157 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00034625900000000002 2024-09-14T06:30:55.157 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:55.157 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:55.157 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:30:55.157 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:34.509218+0000", 2024-09-14T06:30:55.157 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.5091e-05 2024-09-14T06:30:55.157 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:55.158 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:55.158 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:30:55.158 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:34.509227+0000", 2024-09-14T06:30:55.158 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.0850000000000003e-06 2024-09-14T06:30:55.158 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:55.158 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:55.158 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:30:55.158 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:34.509311+0000", 2024-09-14T06:30:55.158 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.4436000000000003e-05 2024-09-14T06:30:55.158 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:55.158 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:55.158 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:30:55.158 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:34.509335+0000", 2024-09-14T06:30:55.158 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.3577000000000001e-05 2024-09-14T06:30:55.159 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:55.159 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:55.159 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-14T06:30:55.159 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:34.510890+0000", 2024-09-14T06:30:55.159 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0015554259999999999 2024-09-14T06:30:55.159 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:55.159 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:55.159 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:30:55.159 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:34.510935+0000", 2024-09-14T06:30:55.159 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.5049000000000002e-05 2024-09-14T06:30:55.159 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:55.159 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:55.159 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:55.159 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:55.160 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:55.160 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4397.0:45722 3.9 e64/23 3:91fed098:::rbd_data.112d5a737e72.0000000000000001:head v 64'61, mlcod=62'59)", 2024-09-14T06:30:55.160 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:43.864269+0000", 2024-09-14T06:30:55.160 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 71.284568323000002, 2024-09-14T06:30:55.160 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.003055224, 2024-09-14T06:30:55.160 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:30:55.160 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:30:55.160 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:30:55.160 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:30:55.160 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:55.160 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:30:55.160 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.864269+0000", 2024-09-14T06:30:55.160 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:55.160 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:55.160 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:55.161 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:30:55.161 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.864269+0000", 2024-09-14T06:30:55.161 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:55.161 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:55.161 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:55.161 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:30:55.161 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.864269+0000", 2024-09-14T06:30:55.161 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.41e-07 2024-09-14T06:30:55.161 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:55.161 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:55.161 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:30:55.161 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.865490+0000", 2024-09-14T06:30:55.161 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.001220552 2024-09-14T06:30:55.161 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:55.161 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:55.162 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:30:55.162 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.865506+0000", 2024-09-14T06:30:55.162 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.6263999999999999e-05 2024-09-14T06:30:55.162 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:55.162 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:55.162 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:30:55.162 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.865516+0000", 2024-09-14T06:30:55.162 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.0091999999999999e-05 2024-09-14T06:30:55.162 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:55.162 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:55.162 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:30:55.162 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.865583+0000", 2024-09-14T06:30:55.162 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.6676999999999999e-05 2024-09-14T06:30:55.162 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:55.162 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:55.163 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:30:55.163 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.865610+0000", 2024-09-14T06:30:55.163 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.6529999999999998e-05 2024-09-14T06:30:55.163 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:55.163 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:55.163 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-14T06:30:55.163 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.867281+0000", 2024-09-14T06:30:55.163 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.001671203 2024-09-14T06:30:55.163 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:55.163 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:55.163 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:30:55.163 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.867324+0000", 2024-09-14T06:30:55.163 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.3665000000000003e-05 2024-09-14T06:30:55.163 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:55.163 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:55.163 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:55.163 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:55.164 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:55.164 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4397.0:50006 3.4 e64/23 3:20cfe300:::rbd_data.112d5a737e72.0000000000000019:head v 64'70, mlcod=61'68)", 2024-09-14T06:30:55.164 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:44.224218+0000", 2024-09-14T06:30:55.164 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 70.924619125999996, 2024-09-14T06:30:55.164 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0021104320000000002, 2024-09-14T06:30:55.164 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:30:55.164 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:30:55.164 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:30:55.164 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:30:55.164 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:55.164 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:30:55.164 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.224218+0000", 2024-09-14T06:30:55.164 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:55.164 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:55.164 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:55.165 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:30:55.165 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.224218+0000", 2024-09-14T06:30:55.165 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:55.165 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:55.165 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:55.165 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:30:55.165 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.224219+0000", 2024-09-14T06:30:55.165 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.0049999999999999e-06 2024-09-14T06:30:55.165 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:55.165 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:55.165 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:30:55.165 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.224326+0000", 2024-09-14T06:30:55.165 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000106848 2024-09-14T06:30:55.165 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:55.165 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:55.165 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:30:55.165 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.224345+0000", 2024-09-14T06:30:55.166 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.8587999999999999e-05 2024-09-14T06:30:55.166 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:55.166 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:55.166 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:30:55.166 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.224354+0000", 2024-09-14T06:30:55.166 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.7220000000000005e-06 2024-09-14T06:30:55.166 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:55.166 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:55.166 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:30:55.166 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.224416+0000", 2024-09-14T06:30:55.166 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.2738999999999997e-05 2024-09-14T06:30:55.166 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:55.166 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:55.166 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:30:55.166 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.224441+0000", 2024-09-14T06:30:55.166 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.4408000000000001e-05 2024-09-14T06:30:55.167 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:55.167 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:55.167 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-14T06:30:55.167 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.226295+0000", 2024-09-14T06:30:55.167 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00185428 2024-09-14T06:30:55.167 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:55.167 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:55.167 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:30:55.167 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.226329+0000", 2024-09-14T06:30:55.167 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.3841999999999999e-05 2024-09-14T06:30:55.167 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:55.167 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:55.167 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:55.167 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:55.167 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:55.167 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4397.0:74241 3.0 e73/23 3:06e93fb9:::rbd_data.112d80ca42af.0000000000000000:head v 73'102, mlcod=71'100)", 2024-09-14T06:30:55.168 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:53.876730+0000", 2024-09-14T06:30:55.168 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 61.272107222999999, 2024-09-14T06:30:55.168 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0079470329999999992, 2024-09-14T06:30:55.168 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:30:55.168 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:30:55.168 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:30:55.168 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:30:55.168 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:55.168 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:30:55.168 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:53.876730+0000", 2024-09-14T06:30:55.168 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:55.168 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:55.168 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:55.168 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:30:55.168 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:53.876730+0000", 2024-09-14T06:30:55.169 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:55.169 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:55.169 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:55.169 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:30:55.169 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:53.876731+0000", 2024-09-14T06:30:55.169 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.0500000000000004e-07 2024-09-14T06:30:55.169 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:55.169 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:55.169 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:30:55.169 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:53.879616+0000", 2024-09-14T06:30:55.169 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0028856680000000001 2024-09-14T06:30:55.169 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:55.169 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:55.169 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:30:55.170 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:53.879643+0000", 2024-09-14T06:30:55.170 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.6120999999999999e-05 2024-09-14T06:30:55.170 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:55.170 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:55.170 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:30:55.170 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:53.879654+0000", 2024-09-14T06:30:55.170 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.1586999999999999e-05 2024-09-14T06:30:55.170 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:55.170 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:55.170 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:30:55.170 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:53.879756+0000", 2024-09-14T06:30:55.170 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00010223899999999999 2024-09-14T06:30:55.170 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:55.170 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:55.170 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:30:55.171 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:53.879783+0000", 2024-09-14T06:30:55.171 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.6913000000000001e-05 2024-09-14T06:30:55.171 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:55.171 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:55.171 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-14T06:30:55.171 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:53.884594+0000", 2024-09-14T06:30:55.171 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0048110729999999999 2024-09-14T06:30:55.171 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:55.171 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:55.171 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:30:55.171 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:53.884677+0000", 2024-09-14T06:30:55.171 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.2927000000000001e-05 2024-09-14T06:30:55.171 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:55.171 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:55.171 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:55.172 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:55.172 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:55.172 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4397.0:74243 3.9 e73/23 3:94977f81:::rbd_data.112d80ca42af.0000000000000002:head v 73'94, mlcod=71'92)", 2024-09-14T06:30:55.172 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:53.885156+0000", 2024-09-14T06:30:55.172 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 61.263681304999999, 2024-09-14T06:30:55.172 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0096009860000000006, 2024-09-14T06:30:55.172 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:30:55.172 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:30:55.172 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:30:55.172 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:30:55.172 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:55.172 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:30:55.172 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:53.885156+0000", 2024-09-14T06:30:55.172 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:55.172 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:55.173 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:55.173 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:30:55.173 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:53.885156+0000", 2024-09-14T06:30:55.173 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:55.173 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:55.173 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:55.173 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:30:55.173 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:53.885156+0000", 2024-09-14T06:30:55.173 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.23e-07 2024-09-14T06:30:55.173 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:55.173 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:55.173 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:30:55.173 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:53.889694+0000", 2024-09-14T06:30:55.173 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0045378709999999997 2024-09-14T06:30:55.174 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:55.174 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:55.174 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:30:55.174 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:53.889720+0000", 2024-09-14T06:30:55.174 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.5259e-05 2024-09-14T06:30:55.174 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:55.174 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:55.174 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:30:55.174 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:53.889731+0000", 2024-09-14T06:30:55.174 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.1739000000000001e-05 2024-09-14T06:30:55.174 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:55.174 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:55.174 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:30:55.174 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:53.889822+0000", 2024-09-14T06:30:55.174 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.1200999999999996e-05 2024-09-14T06:30:55.175 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:55.175 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:55.175 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:30:55.175 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:53.889850+0000", 2024-09-14T06:30:55.175 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.7084999999999999e-05 2024-09-14T06:30:55.175 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:55.175 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:55.175 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-14T06:30:55.175 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:53.894703+0000", 2024-09-14T06:30:55.175 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0048532690000000003 2024-09-14T06:30:55.175 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:55.175 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:55.175 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:30:55.175 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:53.894757+0000", 2024-09-14T06:30:55.175 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.4339000000000003e-05 2024-09-14T06:30:55.176 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:55.176 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:55.176 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:55.176 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:55.176 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:55.176 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4397.0:74249 3.0 e73/23 3:009b0225:::rbd_data.112d80ca42af.0000000000000009:head v 73'103, mlcod=71'101)", 2024-09-14T06:30:55.176 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:53.893219+0000", 2024-09-14T06:30:55.176 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 61.255618826999999, 2024-09-14T06:30:55.176 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0065381980000000003, 2024-09-14T06:30:55.176 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:30:55.176 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:30:55.176 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:30:55.176 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:30:55.176 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:55.176 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:30:55.177 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:53.893219+0000", 2024-09-14T06:30:55.177 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:55.177 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:55.177 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:55.177 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:30:55.177 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:53.893219+0000", 2024-09-14T06:30:55.177 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:55.177 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:55.177 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:55.177 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:30:55.177 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:53.893221+0000", 2024-09-14T06:30:55.177 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.4329999999999998e-06 2024-09-14T06:30:55.177 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:55.178 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:55.178 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:30:55.178 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:53.895968+0000", 2024-09-14T06:30:55.178 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.002746746 2024-09-14T06:30:55.178 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:55.178 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:55.178 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:30:55.178 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:53.895988+0000", 2024-09-14T06:30:55.178 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.0227000000000001e-05 2024-09-14T06:30:55.178 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:55.178 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:55.178 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:30:55.178 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:53.896000+0000", 2024-09-14T06:30:55.178 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.2359e-05 2024-09-14T06:30:55.178 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:55.178 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:55.178 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:30:55.179 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:53.896084+0000", 2024-09-14T06:30:55.179 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.3659999999999995e-05 2024-09-14T06:30:55.179 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:55.179 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:55.179 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:30:55.179 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:53.896109+0000", 2024-09-14T06:30:55.179 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.5259999999999999e-05 2024-09-14T06:30:55.179 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:55.179 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:55.179 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-14T06:30:55.179 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:53.899704+0000", 2024-09-14T06:30:55.179 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0035942209999999999 2024-09-14T06:30:55.179 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:55.179 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:55.179 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:30:55.180 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:53.899757+0000", 2024-09-14T06:30:55.180 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.3291999999999997e-05 2024-09-14T06:30:55.180 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:55.180 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:55.180 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:55.180 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:55.180 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:55.180 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4397.0:78263 3.0 e77/23 3:00899b22:::rbd_data.112ddfbec471.0000000000000016:head v 77'104, mlcod=73'102)", 2024-09-14T06:30:55.180 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:57.634578+0000", 2024-09-14T06:30:55.180 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 57.514259774999999, 2024-09-14T06:30:55.180 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.002372454, 2024-09-14T06:30:55.180 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:30:55.180 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:30:55.180 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:30:55.180 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:30:55.181 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:55.181 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:30:55.181 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:57.634578+0000", 2024-09-14T06:30:55.181 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:55.181 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:55.181 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:55.181 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:30:55.181 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:57.634578+0000", 2024-09-14T06:30:55.181 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:55.181 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:55.181 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:55.181 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:30:55.181 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:57.634578+0000", 2024-09-14T06:30:55.181 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.5699999999999998e-07 2024-09-14T06:30:55.182 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:55.182 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:55.182 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:30:55.182 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:57.635347+0000", 2024-09-14T06:30:55.182 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00076848300000000003 2024-09-14T06:30:55.182 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:55.182 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:55.182 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:30:55.182 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:57.635361+0000", 2024-09-14T06:30:55.182 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.4238e-05 2024-09-14T06:30:55.182 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:55.182 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:55.182 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:30:55.182 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:57.635371+0000", 2024-09-14T06:30:55.182 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.0087e-05 2024-09-14T06:30:55.182 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:55.183 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:55.183 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:30:55.183 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:57.635439+0000", 2024-09-14T06:30:55.183 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.7953000000000005e-05 2024-09-14T06:30:55.183 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:55.183 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:55.183 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:30:55.183 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:57.635464+0000", 2024-09-14T06:30:55.183 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.5062000000000002e-05 2024-09-14T06:30:55.183 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:55.183 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:55.183 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-14T06:30:55.183 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:57.636914+0000", 2024-09-14T06:30:55.183 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.001449625 2024-09-14T06:30:55.183 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:55.184 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:55.184 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:30:55.184 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:57.636950+0000", 2024-09-14T06:30:55.184 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.6548999999999999e-05 2024-09-14T06:30:55.184 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:55.184 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:55.184 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:55.184 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:55.184 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:55.184 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4397.0:78604 3.9 e77/23 3:94c62a0e:::rbd_data.112ddfbec471.000000000000002c:head v 77'96, mlcod=73'94)", 2024-09-14T06:30:55.184 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:57.677568+0000", 2024-09-14T06:30:55.184 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 57.471269387, 2024-09-14T06:30:55.184 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.002564846, 2024-09-14T06:30:55.184 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:30:55.185 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:30:55.185 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:30:55.185 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:30:55.185 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:55.185 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:30:55.185 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:57.677568+0000", 2024-09-14T06:30:55.185 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:55.185 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:55.185 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:55.185 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:30:55.185 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:57.677568+0000", 2024-09-14T06:30:55.185 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:55.185 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:55.185 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:55.185 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:30:55.186 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:57.677568+0000", 2024-09-14T06:30:55.186 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.8900000000000001e-07 2024-09-14T06:30:55.186 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:55.186 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:55.186 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:30:55.186 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:57.678488+0000", 2024-09-14T06:30:55.186 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00091963899999999998 2024-09-14T06:30:55.186 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:55.186 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:55.186 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:30:55.186 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:57.678500+0000", 2024-09-14T06:30:55.186 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.1902000000000001e-05 2024-09-14T06:30:55.186 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:55.186 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:55.186 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:30:55.186 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:57.678507+0000", 2024-09-14T06:30:55.187 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.7739999999999999e-06 2024-09-14T06:30:55.187 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:55.187 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:55.187 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:30:55.187 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:57.678586+0000", 2024-09-14T06:30:55.187 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.9103000000000005e-05 2024-09-14T06:30:55.187 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:55.187 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:55.187 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:30:55.187 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:57.678614+0000", 2024-09-14T06:30:55.187 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.7912e-05 2024-09-14T06:30:55.187 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:55.187 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:55.187 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-14T06:30:55.187 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:57.680075+0000", 2024-09-14T06:30:55.188 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0014613370000000001 2024-09-14T06:30:55.188 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:55.188 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:55.188 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:30:55.188 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:57.680133+0000", 2024-09-14T06:30:55.188 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.7890000000000003e-05 2024-09-14T06:30:55.188 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:55.188 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:55.188 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:55.188 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:55.188 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:55.188 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4397.0:78641 3.9 e77/23 3:914b72e4:::rbd_data.112ddfbec471.0000000000000030:head v 77'98, mlcod=73'95)", 2024-09-14T06:30:55.188 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:57.682289+0000", 2024-09-14T06:30:55.188 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 57.466548705999998, 2024-09-14T06:30:55.189 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0020171400000000002, 2024-09-14T06:30:55.189 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:30:55.189 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:30:55.189 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:30:55.189 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:30:55.189 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:55.189 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:30:55.189 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:57.682289+0000", 2024-09-14T06:30:55.189 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:55.189 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:55.189 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:55.189 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:30:55.189 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:57.682289+0000", 2024-09-14T06:30:55.189 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:55.189 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:55.190 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:55.190 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:30:55.190 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:57.682290+0000", 2024-09-14T06:30:55.190 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.0100000000000003e-07 2024-09-14T06:30:55.190 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:55.190 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:55.190 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:30:55.190 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:57.683230+0000", 2024-09-14T06:30:55.190 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00094047599999999999 2024-09-14T06:30:55.190 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:55.190 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:55.190 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:30:55.190 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:57.683243+0000", 2024-09-14T06:30:55.190 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.2926e-05 2024-09-14T06:30:55.191 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:55.191 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:55.191 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:30:55.191 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:57.683250+0000", 2024-09-14T06:30:55.191 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.6109999999999999e-06 2024-09-14T06:30:55.191 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:55.191 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:55.191 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:30:55.191 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:57.683313+0000", 2024-09-14T06:30:55.191 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.2943999999999999e-05 2024-09-14T06:30:55.191 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:55.191 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:55.191 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:30:55.191 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:57.683337+0000", 2024-09-14T06:30:55.191 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.4377999999999999e-05 2024-09-14T06:30:55.192 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:55.192 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:55.192 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-14T06:30:55.192 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:57.684264+0000", 2024-09-14T06:30:55.192 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00092733099999999999 2024-09-14T06:30:55.192 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:55.192 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:55.192 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:30:55.192 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:57.684306+0000", 2024-09-14T06:30:55.192 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.1573000000000003e-05 2024-09-14T06:30:55.192 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:55.192 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:55.192 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:55.192 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:55.193 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:55.193 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4397.0:126010 3.9 e104/23 3:9120f7ff:::rbd_data.112d97471187.000000000000001b:head v 104'145, mlcod=99'143)", 2024-09-14T06:30:55.193 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:30:25.990254+0000", 2024-09-14T06:30:55.193 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 29.158583898, 2024-09-14T06:30:55.193 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0020629699999999999, 2024-09-14T06:30:55.193 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:30:55.193 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:30:55.193 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:30:55.193 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:30:55.193 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:55.193 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:30:55.193 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:25.990254+0000", 2024-09-14T06:30:55.193 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:55.193 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:55.193 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:55.194 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:30:55.194 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:25.990254+0000", 2024-09-14T06:30:55.194 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:55.194 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:55.194 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:55.194 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:30:55.194 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:25.990255+0000", 2024-09-14T06:30:55.194 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.3349999999999999e-06 2024-09-14T06:30:55.194 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:55.194 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:55.194 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:30:55.194 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:25.990966+0000", 2024-09-14T06:30:55.194 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00071085199999999999 2024-09-14T06:30:55.194 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:55.194 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:55.195 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:30:55.195 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:25.990986+0000", 2024-09-14T06:30:55.195 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.0054000000000001e-05 2024-09-14T06:30:55.195 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:55.195 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:55.195 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:30:55.195 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:25.990995+0000", 2024-09-14T06:30:55.195 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.166e-06 2024-09-14T06:30:55.195 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:55.195 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:55.195 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:30:55.195 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:25.991055+0000", 2024-09-14T06:30:55.195 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.0093000000000002e-05 2024-09-14T06:30:55.195 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:55.195 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:55.195 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:30:55.196 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:25.991082+0000", 2024-09-14T06:30:55.196 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.6505000000000001e-05 2024-09-14T06:30:55.196 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:55.196 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:55.196 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-14T06:30:55.196 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:25.992270+0000", 2024-09-14T06:30:55.196 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0011880479999999999 2024-09-14T06:30:55.196 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:55.196 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:55.196 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:30:55.196 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:25.992317+0000", 2024-09-14T06:30:55.196 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.6916999999999998e-05 2024-09-14T06:30:55.196 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:55.196 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:55.196 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:55.196 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:55.197 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:55.197 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4397.0:149885 3.9 e122/23 3:951acef3:::rbd_data.112d1bb7b269.0000000000000005:head v 122'176, mlcod=122'172)", 2024-09-14T06:30:55.197 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:30:44.739092+0000", 2024-09-14T06:30:55.197 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 10.409745077, 2024-09-14T06:30:55.197 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0028236659999999998, 2024-09-14T06:30:55.197 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:30:55.197 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:30:55.197 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:30:55.197 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:30:55.197 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:55.197 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:30:55.197 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.739092+0000", 2024-09-14T06:30:55.197 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:55.197 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:55.197 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:55.198 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:30:55.198 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.739092+0000", 2024-09-14T06:30:55.198 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:55.198 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:55.198 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:55.198 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:30:55.198 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.739093+0000", 2024-09-14T06:30:55.198 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.0109999999999999e-06 2024-09-14T06:30:55.198 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:55.198 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:55.198 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:30:55.198 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.740189+0000", 2024-09-14T06:30:55.198 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00109527 2024-09-14T06:30:55.198 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:55.198 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:55.198 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:30:55.198 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.740199+0000", 2024-09-14T06:30:55.199 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.0458999999999999e-05 2024-09-14T06:30:55.199 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:55.199 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:55.199 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:30:55.199 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.740207+0000", 2024-09-14T06:30:55.199 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.4499999999999998e-06 2024-09-14T06:30:55.199 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:55.199 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:55.199 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:30:55.199 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.740295+0000", 2024-09-14T06:30:55.199 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.8740999999999996e-05 2024-09-14T06:30:55.199 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:55.199 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:55.199 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:30:55.199 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.740317+0000", 2024-09-14T06:30:55.200 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.1523e-05 2024-09-14T06:30:55.200 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:55.200 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:55.200 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-14T06:30:55.200 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.741865+0000", 2024-09-14T06:30:55.200 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.001548315 2024-09-14T06:30:55.200 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:55.200 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:55.200 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:30:55.200 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.741916+0000", 2024-09-14T06:30:55.200 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.0896999999999997e-05 2024-09-14T06:30:55.200 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:55.200 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:55.200 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:55.200 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:55.201 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:55.201 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4397.0:149886 3.4 e122/23 3:2033c3d0:::rbd_data.112d1bb7b269.0000000000000006:head v 122'220, mlcod=120'217)", 2024-09-14T06:30:55.201 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:30:44.741736+0000", 2024-09-14T06:30:55.201 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 10.40710166, 2024-09-14T06:30:55.201 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0020268999999999999, 2024-09-14T06:30:55.201 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:30:55.201 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:30:55.201 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:30:55.201 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:30:55.201 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:55.201 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:30:55.201 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.741736+0000", 2024-09-14T06:30:55.201 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:55.201 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:55.202 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:55.202 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:30:55.202 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.741736+0000", 2024-09-14T06:30:55.202 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:55.202 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:55.202 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:55.202 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:30:55.202 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.741737+0000", 2024-09-14T06:30:55.202 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.9000000000000005e-07 2024-09-14T06:30:55.202 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:55.202 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:55.202 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:30:55.202 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.741918+0000", 2024-09-14T06:30:55.202 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00018120300000000001 2024-09-14T06:30:55.202 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:55.203 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:55.203 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:30:55.203 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.741930+0000", 2024-09-14T06:30:55.203 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.1875999999999999e-05 2024-09-14T06:30:55.203 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:55.203 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:55.203 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:30:55.203 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.741938+0000", 2024-09-14T06:30:55.203 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.8040000000000002e-06 2024-09-14T06:30:55.203 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:55.203 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:55.203 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:30:55.203 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.742185+0000", 2024-09-14T06:30:55.203 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00024747199999999999 2024-09-14T06:30:55.203 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:55.204 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:55.204 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:30:55.204 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.742207+0000", 2024-09-14T06:30:55.204 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.1416999999999998e-05 2024-09-14T06:30:55.204 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:55.204 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:55.204 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-14T06:30:55.204 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.743715+0000", 2024-09-14T06:30:55.204 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00150884 2024-09-14T06:30:55.204 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:55.204 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:55.204 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:30:55.204 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.743763+0000", 2024-09-14T06:30:55.204 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.7298e-05 2024-09-14T06:30:55.204 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:55.204 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:55.205 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:55.205 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:55.205 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:55.205 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4397.0:149933 3.4 e122/23 3:234f5229:::rbd_data.112d1bb7b269.0000000000000009:head v 122'221, mlcod=120'218)", 2024-09-14T06:30:55.205 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:30:44.743849+0000", 2024-09-14T06:30:55.205 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 10.404988660000001, 2024-09-14T06:30:55.205 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0031479519999999999, 2024-09-14T06:30:55.205 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:30:55.205 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:30:55.205 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:30:55.205 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:30:55.205 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:55.205 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:30:55.205 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.743849+0000", 2024-09-14T06:30:55.206 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:55.206 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:55.206 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:55.206 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:30:55.206 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.743849+0000", 2024-09-14T06:30:55.206 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:55.206 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:55.206 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:55.206 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:30:55.206 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.743850+0000", 2024-09-14T06:30:55.206 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.279e-06 2024-09-14T06:30:55.206 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:55.206 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:55.206 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:30:55.206 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.745151+0000", 2024-09-14T06:30:55.207 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.001300495 2024-09-14T06:30:55.207 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:55.207 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:55.207 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:30:55.207 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.745167+0000", 2024-09-14T06:30:55.207 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.6478e-05 2024-09-14T06:30:55.207 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:55.207 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:55.207 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:30:55.207 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.745175+0000", 2024-09-14T06:30:55.207 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.2679999999999994e-06 2024-09-14T06:30:55.207 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:55.207 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:55.207 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:30:55.207 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.745226+0000", 2024-09-14T06:30:55.208 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.0192999999999999e-05 2024-09-14T06:30:55.208 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:55.208 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:55.208 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:30:55.208 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.745247+0000", 2024-09-14T06:30:55.208 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.1767e-05 2024-09-14T06:30:55.208 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:55.208 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:55.208 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-14T06:30:55.208 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.746952+0000", 2024-09-14T06:30:55.208 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0017042769999999999 2024-09-14T06:30:55.208 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:55.208 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:55.208 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:30:55.209 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.746997+0000", 2024-09-14T06:30:55.209 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.5194999999999997e-05 2024-09-14T06:30:55.209 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:55.209 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:55.209 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:55.209 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:55.209 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:55.209 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4397.0:150494 3.9 e122/23 3:955152a4:::rbd_object_map.112d1bb7b269.0000000000000038:head v 122'194, mlcod=122'190)", 2024-09-14T06:30:55.209 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:30:44.781585+0000", 2024-09-14T06:30:55.209 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 10.367252768, 2024-09-14T06:30:55.209 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0026957499999999998, 2024-09-14T06:30:55.209 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:30:55.209 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:30:55.209 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:30:55.209 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:30:55.210 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:55.210 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:30:55.210 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.781585+0000", 2024-09-14T06:30:55.210 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:55.210 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:55.210 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:55.210 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:30:55.210 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.781585+0000", 2024-09-14T06:30:55.210 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:55.210 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:55.210 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:55.210 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:30:55.210 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.781585+0000", 2024-09-14T06:30:55.211 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.5000000000000002e-07 2024-09-14T06:30:55.211 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:55.211 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:55.211 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:30:55.211 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.781594+0000", 2024-09-14T06:30:55.211 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.4640000000000006e-06 2024-09-14T06:30:55.211 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:55.211 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:55.211 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:30:55.211 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.781604+0000", 2024-09-14T06:30:55.211 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.0071999999999999e-05 2024-09-14T06:30:55.211 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:55.212 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:55.212 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:30:55.212 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.781610+0000", 2024-09-14T06:30:55.212 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.3609999999999997e-06 2024-09-14T06:30:55.212 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:55.212 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:55.212 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:30:55.212 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.781676+0000", 2024-09-14T06:30:55.212 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.5963000000000005e-05 2024-09-14T06:30:55.212 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:55.212 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:55.212 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:30:55.212 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.781696+0000", 2024-09-14T06:30:55.212 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.9278000000000001e-05 2024-09-14T06:30:55.213 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:55.213 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:55.213 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-14T06:30:55.213 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.784231+0000", 2024-09-14T06:30:55.213 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.002535093 2024-09-14T06:30:55.213 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:55.213 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:55.213 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:30:55.213 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.784280+0000", 2024-09-14T06:30:55.213 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.9869e-05 2024-09-14T06:30:55.213 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:55.213 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:55.213 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:55.213 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:55.213 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:55.214 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4397.0:150519 3.9 e122/23 3:955152a4:::rbd_object_map.112d1bb7b269.0000000000000038:head v 122'195, mlcod=122'192)", 2024-09-14T06:30:55.214 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:30:44.782445+0000", 2024-09-14T06:30:55.214 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 10.366392376, 2024-09-14T06:30:55.214 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.002158227, 2024-09-14T06:30:55.214 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:30:55.214 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:30:55.214 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:30:55.214 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:30:55.214 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:55.214 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:30:55.214 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.782445+0000", 2024-09-14T06:30:55.214 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:55.214 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:55.214 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:55.215 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:30:55.215 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.782445+0000", 2024-09-14T06:30:55.215 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:55.215 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:55.215 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:55.215 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:30:55.215 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.782446+0000", 2024-09-14T06:30:55.215 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.6499999999999998e-07 2024-09-14T06:30:55.215 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:55.215 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:55.215 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:30:55.215 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.782455+0000", 2024-09-14T06:30:55.215 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.1940000000000004e-06 2024-09-14T06:30:55.215 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:55.215 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:55.216 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:30:55.216 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.782464+0000", 2024-09-14T06:30:55.216 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.7220000000000005e-06 2024-09-14T06:30:55.216 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:55.216 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:55.216 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:30:55.216 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.782470+0000", 2024-09-14T06:30:55.216 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.9580000000000004e-06 2024-09-14T06:30:55.216 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:55.216 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:55.216 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:30:55.216 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.782514+0000", 2024-09-14T06:30:55.216 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.3931999999999999e-05 2024-09-14T06:30:55.216 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:55.217 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:55.217 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:30:55.217 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.782534+0000", 2024-09-14T06:30:55.217 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.0237e-05 2024-09-14T06:30:55.217 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:55.217 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:55.217 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-14T06:30:55.217 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.784560+0000", 2024-09-14T06:30:55.217 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.002025813 2024-09-14T06:30:55.217 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:55.217 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:55.217 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:30:55.217 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.784603+0000", 2024-09-14T06:30:55.217 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.3606000000000002e-05 2024-09-14T06:30:55.217 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:55.218 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:55.218 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:55.218 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:55.218 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:55.218 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4397.0:150898 3.9 e122/23 3:97c98f90:::rbd_data.112d1bb7b269.0000000000000025:head v 122'206, mlcod=122'203)", 2024-09-14T06:30:55.218 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:30:44.807232+0000", 2024-09-14T06:30:55.218 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 10.341605851000001, 2024-09-14T06:30:55.218 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0022297889999999998, 2024-09-14T06:30:55.219 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:30:55.219 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:30:55.220 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:30:55.220 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:30:55.220 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:55.220 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:30:55.220 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.807232+0000", 2024-09-14T06:30:55.220 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:55.220 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:55.220 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:55.220 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:30:55.220 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.807232+0000", 2024-09-14T06:30:55.220 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:55.220 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:55.220 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:55.221 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:30:55.221 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.807233+0000", 2024-09-14T06:30:55.221 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.5710000000000001e-06 2024-09-14T06:30:55.221 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:55.221 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:55.221 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:30:55.221 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.807944+0000", 2024-09-14T06:30:55.221 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00071056299999999999 2024-09-14T06:30:55.221 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:55.221 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:55.221 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:30:55.221 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.807961+0000", 2024-09-14T06:30:55.221 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.7689e-05 2024-09-14T06:30:55.221 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:55.221 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:55.222 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:30:55.222 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.807971+0000", 2024-09-14T06:30:55.222 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.2e-06 2024-09-14T06:30:55.222 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:55.222 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:55.222 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:30:55.222 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.808050+0000", 2024-09-14T06:30:55.222 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.9308999999999996e-05 2024-09-14T06:30:55.222 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:55.222 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:55.222 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:30:55.222 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.808074+0000", 2024-09-14T06:30:55.222 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.4147000000000001e-05 2024-09-14T06:30:55.222 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:55.223 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:55.223 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-14T06:30:55.223 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.809415+0000", 2024-09-14T06:30:55.223 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.001341257 2024-09-14T06:30:55.223 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:55.223 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:55.223 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:30:55.223 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.809461+0000", 2024-09-14T06:30:55.223 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.6053000000000001e-05 2024-09-14T06:30:55.223 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:55.223 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:55.223 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:55.223 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:55.223 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:55.224 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4397.0:152325 3.9 e122/23 3:955152a4:::rbd_object_map.112d1bb7b269.0000000000000038:head v 122'268, mlcod=122'265)", 2024-09-14T06:30:55.224 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:30:44.914377+0000", 2024-09-14T06:30:55.224 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 10.234460545999999, 2024-09-14T06:30:55.224 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0020208840000000001, 2024-09-14T06:30:55.224 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:30:55.224 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:30:55.224 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:30:55.224 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:30:55.224 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:55.224 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:30:55.224 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.914377+0000", 2024-09-14T06:30:55.224 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:55.224 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:55.224 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:55.225 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:30:55.225 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.914377+0000", 2024-09-14T06:30:55.225 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:55.225 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:55.225 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:55.225 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:30:55.225 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.914378+0000", 2024-09-14T06:30:55.225 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.3600000000000003e-07 2024-09-14T06:30:55.225 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:55.225 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:55.225 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:30:55.225 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.914383+0000", 2024-09-14T06:30:55.225 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.7230000000000001e-06 2024-09-14T06:30:55.225 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:55.226 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:55.226 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:30:55.226 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.914393+0000", 2024-09-14T06:30:55.226 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.8179999999999999e-06 2024-09-14T06:30:55.226 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:55.226 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:55.226 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:30:55.226 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.914400+0000", 2024-09-14T06:30:55.226 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.968e-06 2024-09-14T06:30:55.226 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:55.226 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:55.226 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:30:55.226 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.914447+0000", 2024-09-14T06:30:55.226 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.7122e-05 2024-09-14T06:30:55.227 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:55.227 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:55.227 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:30:55.227 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.914466+0000", 2024-09-14T06:30:55.227 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.8958999999999999e-05 2024-09-14T06:30:55.227 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:55.227 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:55.227 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-14T06:30:55.227 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.916356+0000", 2024-09-14T06:30:55.227 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.001890121 2024-09-14T06:30:55.227 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:55.227 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:55.227 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:30:55.227 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.916398+0000", 2024-09-14T06:30:55.228 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.1437000000000003e-05 2024-09-14T06:30:55.228 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:55.228 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:55.228 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:55.228 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:55.228 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:55.228 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4397.0:152449 3.9 e122/23 3:955152a4:::rbd_object_map.112d1bb7b269.0000000000000038:head v 122'274, mlcod=122'268)", 2024-09-14T06:30:55.228 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:30:44.924063+0000", 2024-09-14T06:30:55.228 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 10.224774034999999, 2024-09-14T06:30:55.228 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.002131764, 2024-09-14T06:30:55.228 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:30:55.228 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:30:55.228 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:30:55.228 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:30:55.228 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:55.229 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:30:55.229 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.924063+0000", 2024-09-14T06:30:55.229 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:55.229 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:55.229 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:55.229 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:30:55.229 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.924063+0000", 2024-09-14T06:30:55.229 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:55.229 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:55.229 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:55.229 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:30:55.229 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.924064+0000", 2024-09-14T06:30:55.230 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.72e-07 2024-09-14T06:30:55.230 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:55.230 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:55.230 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:30:55.230 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.924071+0000", 2024-09-14T06:30:55.230 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.0999999999999998e-06 2024-09-14T06:30:55.230 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:55.230 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:55.230 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:30:55.230 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.924077+0000", 2024-09-14T06:30:55.230 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.268e-06 2024-09-14T06:30:55.230 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:55.230 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:55.230 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:30:55.231 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.924082+0000", 2024-09-14T06:30:55.231 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.7110000000000003e-06 2024-09-14T06:30:55.231 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:55.231 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:55.231 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:30:55.231 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.924397+0000", 2024-09-14T06:30:55.231 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00031503500000000002 2024-09-14T06:30:55.231 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:55.231 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:55.231 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:30:55.231 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.924418+0000", 2024-09-14T06:30:55.231 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.1044999999999999e-05 2024-09-14T06:30:55.231 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:55.231 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:55.231 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-14T06:30:55.231 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.926158+0000", 2024-09-14T06:30:55.232 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0017404860000000001 2024-09-14T06:30:55.232 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:55.232 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:55.232 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:30:55.232 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.926195+0000", 2024-09-14T06:30:55.232 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.6847000000000001e-05 2024-09-14T06:30:55.232 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:55.232 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:55.232 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:55.232 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:55.232 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:55.232 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4397.0:152452 3.9 e122/23 3:955152a4:::rbd_object_map.112d1bb7b269.0000000000000038:head v 122'275, mlcod=122'268)", 2024-09-14T06:30:55.232 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:30:44.924090+0000", 2024-09-14T06:30:55.232 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 10.224747714999999, 2024-09-14T06:30:55.232 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0021362400000000002, 2024-09-14T06:30:55.233 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:30:55.233 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:30:55.233 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:30:55.233 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:30:55.233 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:55.233 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:30:55.233 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.924090+0000", 2024-09-14T06:30:55.233 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:55.233 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:55.233 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:55.233 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:30:55.233 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.924090+0000", 2024-09-14T06:30:55.233 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:55.233 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:55.234 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:55.234 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:30:55.234 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.924090+0000", 2024-09-14T06:30:55.234 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.4699999999999998e-07 2024-09-14T06:30:55.234 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:55.234 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:55.234 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:30:55.234 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.924658+0000", 2024-09-14T06:30:55.234 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00056802700000000003 2024-09-14T06:30:55.234 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:55.234 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:55.234 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:30:55.234 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.924664+0000", 2024-09-14T06:30:55.234 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.3570000000000003e-06 2024-09-14T06:30:55.234 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:55.235 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:55.235 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:30:55.235 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.924669+0000", 2024-09-14T06:30:55.235 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.2300000000000002e-06 2024-09-14T06:30:55.235 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:55.235 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:55.235 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:30:55.235 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.924802+0000", 2024-09-14T06:30:55.235 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00013362500000000001 2024-09-14T06:30:55.235 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:55.235 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:55.235 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:30:55.235 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.924824+0000", 2024-09-14T06:30:55.235 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.1449000000000001e-05 2024-09-14T06:30:55.235 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:55.236 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:55.236 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-14T06:30:55.236 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.926197+0000", 2024-09-14T06:30:55.236 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.001373354 2024-09-14T06:30:55.236 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:55.236 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:55.236 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:30:55.236 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.926226+0000", 2024-09-14T06:30:55.236 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.8951000000000001e-05 2024-09-14T06:30:55.236 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:55.236 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:55.236 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:55.236 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:55.236 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:55.237 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-14T06:30:55.238 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-14T06:30:55.238 INFO:teuthology.orchestra.run.smithi094.stdout:osd.0: {} 2024-09-14T06:30:55.239 INFO:teuthology.orchestra.run.smithi094.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-14T06:30:55.239 INFO:teuthology.orchestra.run.smithi094.stdout:osd.1: {} 2024-09-14T06:30:55.239 INFO:teuthology.orchestra.run.smithi094.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-14T06:30:55.240 INFO:tasks.ceph.osd.7.smithi138.stderr:2024-09-14T06:30:55.237+0000 7f3c6a21d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33623) UID: 0 2024-09-14T06:30:55.241 INFO:teuthology.orchestra.run.smithi094.stdout:osd.2: {} 2024-09-14T06:30:55.242 INFO:teuthology.orchestra.run.smithi094.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-14T06:30:55.258 INFO:teuthology.orchestra.run.smithi094.stdout:osd.3: {} 2024-09-14T06:30:55.258 INFO:teuthology.orchestra.run.smithi094.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-14T06:30:55.275 INFO:teuthology.orchestra.run.smithi094.stdout:osd.4: {} 2024-09-14T06:30:55.275 INFO:teuthology.orchestra.run.smithi094.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-14T06:30:55.293 INFO:teuthology.orchestra.run.smithi094.stdout:osd.5: {} 2024-09-14T06:30:55.293 INFO:teuthology.orchestra.run.smithi094.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-14T06:30:55.310 INFO:teuthology.orchestra.run.smithi094.stdout:osd.6: {} 2024-09-14T06:30:55.310 INFO:teuthology.orchestra.run.smithi094.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-14T06:30:55.327 INFO:teuthology.orchestra.run.smithi094.stdout:osd.7: {} 2024-09-14T06:30:55.327 INFO:teuthology.orchestra.run.smithi094.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-14T06:30:55.339 INFO:tasks.ceph.osd.5.smithi138.stderr:2024-09-14T06:30:55.338+0000 7fe13746f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33618) UID: 0 2024-09-14T06:30:55.345 INFO:teuthology.orchestra.run.smithi094.stdout:ERROR: (22) Invalid argument 2024-09-14T06:30:55.345 INFO:teuthology.orchestra.run.smithi094.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:30:55.354 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-14T06:30:55.440 INFO:tasks.ceph.osd.5.smithi138.stderr:2024-09-14T06:30:55.439+0000 7fe13746f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33618) UID: 0 2024-09-14T06:30:55.497 INFO:teuthology.orchestra.run.smithi094.stdout:ERROR: (22) Invalid argument 2024-09-14T06:30:55.497 INFO:teuthology.orchestra.run.smithi094.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:30:55.505 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-14T06:30:55.541 INFO:tasks.ceph.osd.6.smithi138.stderr:2024-09-14T06:30:55.539+0000 7f8cd8849640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33620) UID: 0 2024-09-14T06:30:55.621 INFO:teuthology.orchestra.run.smithi094.stdout:ERROR: (22) Invalid argument 2024-09-14T06:30:55.621 INFO:teuthology.orchestra.run.smithi094.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:30:55.641 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-14T06:30:55.642 INFO:tasks.ceph.osd.1.smithi094.stderr:2024-09-14T06:30:55.640+0000 7f328e47a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33641) UID: 0 2024-09-14T06:30:55.741 INFO:tasks.ceph.osd.2.smithi094.stderr:2024-09-14T06:30:55.740+0000 7faf3269b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33649) UID: 0 2024-09-14T06:30:55.780 INFO:teuthology.orchestra.run.smithi138.stdout:ERROR: (22) Invalid argument 2024-09-14T06:30:55.781 INFO:teuthology.orchestra.run.smithi138.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:30:55.789 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-14T06:30:55.843 INFO:tasks.ceph.osd.1.smithi094.stderr:2024-09-14T06:30:55.842+0000 7f328e47a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33641) UID: 0 2024-09-14T06:30:55.892 INFO:teuthology.orchestra.run.smithi138.stdout:ERROR: (22) Invalid argument 2024-09-14T06:30:55.892 INFO:teuthology.orchestra.run.smithi138.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:30:55.901 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-14T06:30:55.945 INFO:tasks.ceph.osd.1.smithi094.stderr:2024-09-14T06:30:55.943+0000 7f328e47a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33641) UID: 0 2024-09-14T06:30:56.002 INFO:teuthology.orchestra.run.smithi138.stdout:ERROR: (22) Invalid argument 2024-09-14T06:30:56.002 INFO:teuthology.orchestra.run.smithi138.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:30:56.011 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-14T06:30:56.045 INFO:tasks.ceph.osd.0.smithi094.stderr:2024-09-14T06:30:56.044+0000 7f07c9333640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33636) UID: 0 2024-09-14T06:30:56.113 INFO:teuthology.orchestra.run.smithi138.stdout:ERROR: (22) Invalid argument 2024-09-14T06:30:56.113 INFO:teuthology.orchestra.run.smithi138.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:30:56.121 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-14T06:30:56.146 INFO:tasks.ceph.osd.3.smithi094.stderr:2024-09-14T06:30:56.145+0000 7feeeafe8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33655) UID: 0 2024-09-14T06:30:56.224 INFO:teuthology.orchestra.run.smithi138.stdout:ERROR: (22) Invalid argument 2024-09-14T06:30:56.224 INFO:teuthology.orchestra.run.smithi138.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:30:56.233 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-14T06:30:56.277 INFO:tasks.ceph.osd.6.smithi138.stderr:2024-09-14T06:30:56.276+0000 7f8cd8849640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33620) UID: 0 2024-09-14T06:30:56.336 INFO:teuthology.orchestra.run.smithi138.stdout:ERROR: (22) Invalid argument 2024-09-14T06:30:56.337 INFO:teuthology.orchestra.run.smithi138.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:30:56.345 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-14T06:30:56.347 INFO:tasks.ceph.osd.1.smithi094.stderr:2024-09-14T06:30:56.346+0000 7f328e47a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33641) UID: 0 2024-09-14T06:30:56.448 INFO:tasks.ceph.osd.0.smithi094.stderr:2024-09-14T06:30:56.447+0000 7f07c9333640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33636) UID: 0 2024-09-14T06:30:56.489 INFO:teuthology.orchestra.run.smithi138.stdout:ERROR: (22) Invalid argument 2024-09-14T06:30:56.489 INFO:teuthology.orchestra.run.smithi138.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:30:56.497 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-14T06:30:56.549 INFO:tasks.ceph.osd.5.smithi138.stderr:2024-09-14T06:30:56.548+0000 7fe13746f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33618) UID: 0 2024-09-14T06:30:56.600 INFO:teuthology.orchestra.run.smithi138.stdout:ERROR: (22) Invalid argument 2024-09-14T06:30:56.600 INFO:teuthology.orchestra.run.smithi138.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:30:56.609 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-14T06:30:56.649 INFO:tasks.ceph.osd.2.smithi094.stderr:2024-09-14T06:30:56.648+0000 7faf3269b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33649) UID: 0 2024-09-14T06:30:56.750 INFO:tasks.ceph.osd.2.smithi094.stderr:2024-09-14T06:30:56.749+0000 7faf3269b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33649) UID: 0 2024-09-14T06:30:56.752 INFO:teuthology.orchestra.run.smithi138.stdout:ERROR: (22) Invalid argument 2024-09-14T06:30:56.752 INFO:teuthology.orchestra.run.smithi138.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:30:56.761 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-14T06:30:56.766 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-14T06:30:56.851 INFO:tasks.ceph.osd.4.smithi138.stderr:2024-09-14T06:30:56.850+0000 7f6c653c8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33615) UID: 0 2024-09-14T06:30:56.867 INFO:teuthology.orchestra.run.smithi138.stdout:ERROR: (22) Invalid argument 2024-09-14T06:30:56.868 INFO:teuthology.orchestra.run.smithi138.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:30:56.869 INFO:teuthology.orchestra.run.smithi094.stdout:checking clone #28, image image_client.0-clone28 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent29 2024-09-14T06:30:56.877 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-14T06:30:56.961 INFO:tasks.ceph.osd.7.smithi138.stderr:2024-09-14T06:30:56.960+0000 7f3c6a21d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33623) UID: 0 2024-09-14T06:30:57.019 INFO:teuthology.orchestra.run.smithi138.stdout:ERROR: (22) Invalid argument 2024-09-14T06:30:57.019 INFO:teuthology.orchestra.run.smithi138.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:30:57.028 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-14T06:30:57.052 INFO:tasks.ceph.osd.2.smithi094.stderr:2024-09-14T06:30:57.051+0000 7faf3269b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33649) UID: 0 2024-09-14T06:30:57.154 INFO:tasks.ceph.osd.2.smithi094.stderr:2024-09-14T06:30:57.152+0000 7faf3269b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33649) UID: 0 2024-09-14T06:30:57.171 INFO:teuthology.orchestra.run.smithi138.stdout:ERROR: (22) Invalid argument 2024-09-14T06:30:57.171 INFO:teuthology.orchestra.run.smithi138.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:30:57.181 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-14T06:30:57.255 INFO:tasks.ceph.osd.4.smithi138.stderr:2024-09-14T06:30:57.253+0000 7f6c653c8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33615) UID: 0 2024-09-14T06:30:57.284 INFO:teuthology.orchestra.run.smithi094.stdout:ERROR: (22) Invalid argument 2024-09-14T06:30:57.284 INFO:teuthology.orchestra.run.smithi094.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:30:57.292 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-14T06:30:57.339 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-14T06:30:57.356 INFO:tasks.ceph.osd.5.smithi138.stderr:2024-09-14T06:30:57.355+0000 7fe13746f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33618) UID: 0 2024-09-14T06:30:57.397 INFO:teuthology.orchestra.run.smithi094.stdout:ERROR: (22) Invalid argument 2024-09-14T06:30:57.397 INFO:teuthology.orchestra.run.smithi094.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:30:57.406 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-14T06:30:57.457 INFO:tasks.ceph.osd.6.smithi138.stderr:2024-09-14T06:30:57.456+0000 7f8cd8849640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33620) UID: 0 2024-09-14T06:30:57.465 INFO:teuthology.orchestra.run.smithi094.stdout:osd.0: {} 2024-09-14T06:30:57.465 INFO:teuthology.orchestra.run.smithi094.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-14T06:30:57.483 INFO:teuthology.orchestra.run.smithi094.stdout:osd.1: {} 2024-09-14T06:30:57.483 INFO:teuthology.orchestra.run.smithi094.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-14T06:30:57.499 INFO:teuthology.orchestra.run.smithi094.stdout:osd.2: {} 2024-09-14T06:30:57.499 INFO:teuthology.orchestra.run.smithi094.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-14T06:30:57.515 INFO:teuthology.orchestra.run.smithi094.stdout:osd.3: {} 2024-09-14T06:30:57.516 INFO:teuthology.orchestra.run.smithi094.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-14T06:30:57.517 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-14T06:30:57.517 INFO:teuthology.orchestra.run.smithi094.stdout: "size": 20, 2024-09-14T06:30:57.517 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 600, 2024-09-14T06:30:57.517 INFO:teuthology.orchestra.run.smithi094.stdout: "ops": [ 2024-09-14T06:30:57.517 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:57.517 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:115193 3.1 3:856be886:::rbd_data.112d4d24c558.0000000000000004:29 [read 1523712~16384 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e96)", 2024-09-14T06:30:57.517 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:30:17.939854+0000", 2024-09-14T06:30:57.518 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 39.574920484000003, 2024-09-14T06:30:57.518 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.073897516999999996, 2024-09-14T06:30:57.518 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:30:57.518 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:30:57.518 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:30:57.518 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:30:57.518 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:30:57.518 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:57.518 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 115193 2024-09-14T06:30:57.518 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:57.519 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:30:57.519 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:57.519 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:30:57.519 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.939854+0000", 2024-09-14T06:30:57.519 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:57.519 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:57.519 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:57.519 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:30:57.519 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.939854+0000", 2024-09-14T06:30:57.519 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:57.519 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:57.520 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:57.520 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:30:57.520 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.939854+0000", 2024-09-14T06:30:57.520 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.9500000000000002e-07 2024-09-14T06:30:57.520 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:57.520 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:57.520 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:30:57.520 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.939856+0000", 2024-09-14T06:30:57.520 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.0959999999999999e-06 2024-09-14T06:30:57.521 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:57.521 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:57.521 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:30:57.521 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.939863+0000", 2024-09-14T06:30:57.521 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.1129999999999997e-06 2024-09-14T06:30:57.521 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:57.521 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:57.521 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:30:57.521 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.939867+0000", 2024-09-14T06:30:57.521 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.8350000000000003e-06 2024-09-14T06:30:57.522 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:57.522 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:57.522 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:30:57.522 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.013626+0000", 2024-09-14T06:30:57.522 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.073758799999999999 2024-09-14T06:30:57.522 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:57.522 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:57.522 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:30:57.522 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.013690+0000", 2024-09-14T06:30:57.522 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.3918999999999996e-05 2024-09-14T06:30:57.522 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:57.522 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:57.522 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:30:57.523 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.013751+0000", 2024-09-14T06:30:57.523 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.1159000000000002e-05 2024-09-14T06:30:57.523 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:57.523 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:57.523 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:57.523 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:57.523 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:57.523 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:115194 3.1 3:856be886:::rbd_data.112d4d24c558.0000000000000004:29 [read 1769472~16384 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e96)", 2024-09-14T06:30:57.523 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:30:17.939874+0000", 2024-09-14T06:30:57.523 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 39.574900173000003, 2024-09-14T06:30:57.523 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.074016988000000006, 2024-09-14T06:30:57.523 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:30:57.523 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:30:57.524 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:30:57.524 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:30:57.524 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:30:57.524 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:57.524 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 115194 2024-09-14T06:30:57.524 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:57.524 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:30:57.524 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:57.524 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:30:57.524 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.939874+0000", 2024-09-14T06:30:57.524 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:57.524 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:57.524 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:57.525 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:30:57.525 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.939874+0000", 2024-09-14T06:30:57.525 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:57.525 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:57.525 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:57.525 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:30:57.525 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.939875+0000", 2024-09-14T06:30:57.525 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.2600000000000002e-07 2024-09-14T06:30:57.525 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:57.525 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:57.525 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:30:57.525 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.939877+0000", 2024-09-14T06:30:57.525 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.9420000000000002e-06 2024-09-14T06:30:57.526 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:57.526 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:57.526 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:30:57.526 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.939882+0000", 2024-09-14T06:30:57.526 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.8749999999999997e-06 2024-09-14T06:30:57.526 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:57.526 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:57.526 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:30:57.526 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.939887+0000", 2024-09-14T06:30:57.526 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.741e-06 2024-09-14T06:30:57.526 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:57.526 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:57.526 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:30:57.527 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.013767+0000", 2024-09-14T06:30:57.527 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.073879720999999995 2024-09-14T06:30:57.527 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:57.527 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:57.527 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:30:57.527 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.013832+0000", 2024-09-14T06:30:57.527 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.5312000000000001e-05 2024-09-14T06:30:57.527 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:57.527 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:57.528 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:30:57.528 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.013891+0000", 2024-09-14T06:30:57.528 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.8870999999999999e-05 2024-09-14T06:30:57.528 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:57.528 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:57.528 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:57.528 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:57.528 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:57.528 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:115195 3.1 3:856be886:::rbd_data.112d4d24c558.0000000000000004:29 [read 2015232~16384 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e96)", 2024-09-14T06:30:57.528 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:30:17.939903+0000", 2024-09-14T06:30:57.528 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 39.574870988000001, 2024-09-14T06:30:57.528 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.074128199000000006, 2024-09-14T06:30:57.529 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:30:57.529 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:30:57.529 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:30:57.529 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:30:57.529 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:30:57.529 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:57.529 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 115195 2024-09-14T06:30:57.529 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:57.529 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:30:57.529 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:57.530 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:30:57.530 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.939903+0000", 2024-09-14T06:30:57.530 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:57.530 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:57.530 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:57.530 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:30:57.530 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.939903+0000", 2024-09-14T06:30:57.530 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:57.530 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:57.530 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:57.530 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:30:57.530 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.939904+0000", 2024-09-14T06:30:57.530 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.5799999999999999e-07 2024-09-14T06:30:57.531 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:57.531 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:57.531 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:30:57.531 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.939906+0000", 2024-09-14T06:30:57.531 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.2350000000000002e-06 2024-09-14T06:30:57.531 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:57.531 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:57.531 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:30:57.531 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.939913+0000", 2024-09-14T06:30:57.531 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.5649999999999998e-06 2024-09-14T06:30:57.531 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:57.532 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:57.532 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:30:57.532 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.939918+0000", 2024-09-14T06:30:57.532 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.784e-06 2024-09-14T06:30:57.532 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:57.532 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:57.532 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:30:57.532 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.013921+0000", 2024-09-14T06:30:57.532 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.074002911000000005 2024-09-14T06:30:57.532 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:57.532 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:57.532 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:30:57.532 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.013971+0000", 2024-09-14T06:30:57.533 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.9551e-05 2024-09-14T06:30:57.533 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:57.533 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:57.533 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:30:57.533 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.014031+0000", 2024-09-14T06:30:57.533 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.0594999999999998e-05 2024-09-14T06:30:57.533 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:57.533 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:57.533 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:57.533 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:57.533 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:57.533 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:115196 3.1 3:856be886:::rbd_data.112d4d24c558.0000000000000004:29 [read 2260992~16384 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e96)", 2024-09-14T06:30:57.533 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:30:17.939925+0000", 2024-09-14T06:30:57.534 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 39.574848795999998, 2024-09-14T06:30:57.534 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.074320137999999994, 2024-09-14T06:30:57.534 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:30:57.534 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:30:57.534 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:30:57.534 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:30:57.534 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:30:57.534 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:57.534 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 115196 2024-09-14T06:30:57.534 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:57.534 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:30:57.535 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:57.535 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:30:57.535 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.939925+0000", 2024-09-14T06:30:57.535 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:57.535 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:57.535 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:57.535 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:30:57.535 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.939925+0000", 2024-09-14T06:30:57.535 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:57.535 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:57.535 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:57.536 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:30:57.536 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.939926+0000", 2024-09-14T06:30:57.536 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.9999999999999998e-07 2024-09-14T06:30:57.536 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:57.536 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:57.536 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:30:57.536 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.939928+0000", 2024-09-14T06:30:57.536 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.9549999999999999e-06 2024-09-14T06:30:57.536 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:57.536 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:57.536 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:30:57.536 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.939934+0000", 2024-09-14T06:30:57.537 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.127e-06 2024-09-14T06:30:57.537 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:57.537 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:57.537 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:30:57.537 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.939939+0000", 2024-09-14T06:30:57.537 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.775e-06 2024-09-14T06:30:57.537 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:57.537 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:57.537 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:30:57.538 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.014075+0000", 2024-09-14T06:30:57.538 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.074136598999999997 2024-09-14T06:30:57.538 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:57.538 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:57.538 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:30:57.538 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.014162+0000", 2024-09-14T06:30:57.538 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.6477999999999994e-05 2024-09-14T06:30:57.538 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:57.538 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:57.538 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:30:57.538 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.014246+0000", 2024-09-14T06:30:57.538 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.3703999999999997e-05 2024-09-14T06:30:57.539 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:57.539 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:57.539 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:57.539 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:57.539 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:57.539 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:115197 3.1 3:856be886:::rbd_data.112d4d24c558.0000000000000004:29 [read 2506752~16384 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e96)", 2024-09-14T06:30:57.539 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:30:17.939969+0000", 2024-09-14T06:30:57.539 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 39.574805310999999, 2024-09-14T06:30:57.539 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.074528544000000002, 2024-09-14T06:30:57.540 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:30:57.540 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:30:57.540 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:30:57.540 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:30:57.540 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:30:57.541 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:57.541 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 115197 2024-09-14T06:30:57.541 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:57.541 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:30:57.541 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:57.541 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:30:57.541 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.939969+0000", 2024-09-14T06:30:57.541 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:57.541 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:57.541 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:57.541 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:30:57.541 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.939969+0000", 2024-09-14T06:30:57.541 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:57.542 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:57.542 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:57.542 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:30:57.542 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.939970+0000", 2024-09-14T06:30:57.542 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.3600000000000003e-07 2024-09-14T06:30:57.542 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:57.542 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:57.542 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:30:57.542 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.939972+0000", 2024-09-14T06:30:57.542 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.3180000000000001e-06 2024-09-14T06:30:57.542 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:57.542 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:57.543 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:30:57.543 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.939979+0000", 2024-09-14T06:30:57.543 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.8800000000000002e-06 2024-09-14T06:30:57.543 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:57.543 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:57.543 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:30:57.543 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.939984+0000", 2024-09-14T06:30:57.543 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.3510000000000001e-06 2024-09-14T06:30:57.543 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:57.543 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:57.543 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:30:57.543 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.014302+0000", 2024-09-14T06:30:57.543 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.074317416999999997 2024-09-14T06:30:57.543 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:57.544 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:57.544 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:30:57.544 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.014392+0000", 2024-09-14T06:30:57.544 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.0181999999999998e-05 2024-09-14T06:30:57.544 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:57.544 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:57.544 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:30:57.544 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.014497+0000", 2024-09-14T06:30:57.544 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00010576 2024-09-14T06:30:57.544 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:57.544 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:57.544 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:57.544 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:57.545 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:57.545 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:115198 3.1 3:856be886:::rbd_data.112d4d24c558.0000000000000004:29 [read 2752512~16384 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e96)", 2024-09-14T06:30:57.545 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:30:17.940004+0000", 2024-09-14T06:30:57.545 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 39.574770031, 2024-09-14T06:30:57.545 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.074736367999999997, 2024-09-14T06:30:57.545 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:30:57.545 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:30:57.545 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:30:57.545 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:30:57.545 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:30:57.545 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:57.545 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 115198 2024-09-14T06:30:57.545 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:57.546 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:30:57.546 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:57.546 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:30:57.546 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940004+0000", 2024-09-14T06:30:57.546 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:57.546 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:57.546 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:57.546 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:30:57.546 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940004+0000", 2024-09-14T06:30:57.546 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:57.546 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:57.546 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:57.546 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:30:57.546 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940005+0000", 2024-09-14T06:30:57.547 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.2200000000000004e-07 2024-09-14T06:30:57.547 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:57.547 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:57.547 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:30:57.547 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940007+0000", 2024-09-14T06:30:57.547 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.322e-06 2024-09-14T06:30:57.547 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:57.547 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:57.547 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:30:57.547 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940014+0000", 2024-09-14T06:30:57.547 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.6220000000000003e-06 2024-09-14T06:30:57.547 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:57.547 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:57.547 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:30:57.548 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940019+0000", 2024-09-14T06:30:57.548 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.6380000000000001e-06 2024-09-14T06:30:57.548 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:57.548 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:57.548 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:30:57.548 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.014562+0000", 2024-09-14T06:30:57.548 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.074542320999999995 2024-09-14T06:30:57.548 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:57.548 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:57.548 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:30:57.548 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.014646+0000", 2024-09-14T06:30:57.548 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.4079999999999995e-05 2024-09-14T06:30:57.548 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:57.549 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:57.549 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:30:57.549 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.014741+0000", 2024-09-14T06:30:57.549 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.4763000000000001e-05 2024-09-14T06:30:57.549 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:57.549 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:57.549 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:57.549 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:57.549 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:57.549 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:115199 3.1 3:856be886:::rbd_data.112d4d24c558.0000000000000004:29 [read 2998272~16384 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e96)", 2024-09-14T06:30:57.549 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:30:17.940040+0000", 2024-09-14T06:30:57.549 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 39.574734276000001, 2024-09-14T06:30:57.549 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.074957134999999994, 2024-09-14T06:30:57.550 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:30:57.550 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:30:57.550 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:30:57.550 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:30:57.550 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:30:57.550 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:57.550 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 115199 2024-09-14T06:30:57.550 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:57.550 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:30:57.550 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:57.550 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:30:57.550 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940040+0000", 2024-09-14T06:30:57.550 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:57.551 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:57.551 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:57.551 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:30:57.551 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940040+0000", 2024-09-14T06:30:57.551 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:57.551 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:57.551 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:57.551 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:30:57.551 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940041+0000", 2024-09-14T06:30:57.551 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.4000000000000001e-07 2024-09-14T06:30:57.551 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:57.551 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:57.551 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:30:57.552 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940046+0000", 2024-09-14T06:30:57.552 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.7250000000000002e-06 2024-09-14T06:30:57.552 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:57.552 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:57.552 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:30:57.552 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940053+0000", 2024-09-14T06:30:57.552 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.6109999999999999e-06 2024-09-14T06:30:57.552 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:57.552 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:57.552 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:30:57.552 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940058+0000", 2024-09-14T06:30:57.552 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.1590000000000001e-06 2024-09-14T06:30:57.552 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:57.553 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:57.553 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:30:57.553 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.014768+0000", 2024-09-14T06:30:57.553 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.074709624000000002 2024-09-14T06:30:57.553 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:57.553 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:57.553 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:30:57.553 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.014907+0000", 2024-09-14T06:30:57.553 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000139665 2024-09-14T06:30:57.553 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:57.553 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:57.553 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:30:57.553 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.014997+0000", 2024-09-14T06:30:57.554 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.9710999999999995e-05 2024-09-14T06:30:57.554 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:57.554 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:57.554 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:57.554 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:57.554 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:57.554 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:115200 3.1 3:856be886:::rbd_data.112d4d24c558.0000000000000004:29 [read 3244032~16384 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e96)", 2024-09-14T06:30:57.554 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:30:17.940080+0000", 2024-09-14T06:30:57.554 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 39.574694071000003, 2024-09-14T06:30:57.554 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.075137291999999994, 2024-09-14T06:30:57.554 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:30:57.554 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:30:57.554 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:30:57.555 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:30:57.555 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:30:57.555 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:57.555 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 115200 2024-09-14T06:30:57.555 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:57.555 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:30:57.555 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:57.555 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:30:57.555 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940080+0000", 2024-09-14T06:30:57.555 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:57.555 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:57.555 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:57.555 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:30:57.555 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940080+0000", 2024-09-14T06:30:57.555 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:57.556 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:57.556 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:57.556 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:30:57.556 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940081+0000", 2024-09-14T06:30:57.556 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.0800000000000004e-07 2024-09-14T06:30:57.556 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:57.556 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:57.556 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:30:57.556 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940083+0000", 2024-09-14T06:30:57.556 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.3920000000000001e-06 2024-09-14T06:30:57.556 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:57.556 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:57.556 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:30:57.556 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940091+0000", 2024-09-14T06:30:57.557 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.1389999999999995e-06 2024-09-14T06:30:57.557 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:57.557 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:57.557 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:30:57.557 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940097+0000", 2024-09-14T06:30:57.557 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.5550000000000002e-06 2024-09-14T06:30:57.557 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:57.557 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:57.557 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:30:57.557 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.015025+0000", 2024-09-14T06:30:57.557 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.074928579999999995 2024-09-14T06:30:57.557 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:57.557 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:57.557 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:30:57.557 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.015096+0000", 2024-09-14T06:30:57.558 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.0705999999999997e-05 2024-09-14T06:30:57.558 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:57.558 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:57.558 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:30:57.558 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.015217+0000", 2024-09-14T06:30:57.558 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00012131200000000001 2024-09-14T06:30:57.558 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:57.558 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:57.558 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:57.558 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:57.558 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:57.558 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:115201 3.1 3:856be886:::rbd_data.112d4d24c558.0000000000000004:29 [read 3489792~16384 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e96)", 2024-09-14T06:30:57.558 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:30:17.940113+0000", 2024-09-14T06:30:57.558 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 39.574661298000002, 2024-09-14T06:30:57.558 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.075246899000000006, 2024-09-14T06:30:57.559 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:30:57.559 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:30:57.559 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:30:57.559 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:30:57.559 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:30:57.559 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:57.559 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 115201 2024-09-14T06:30:57.559 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:57.559 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:30:57.559 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:57.559 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:30:57.559 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940113+0000", 2024-09-14T06:30:57.559 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:57.559 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:57.559 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:57.560 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:30:57.560 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940113+0000", 2024-09-14T06:30:57.560 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:57.560 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:57.560 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:57.560 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:30:57.560 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940114+0000", 2024-09-14T06:30:57.560 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.1500000000000004e-07 2024-09-14T06:30:57.560 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:57.560 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:57.560 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:30:57.560 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940116+0000", 2024-09-14T06:30:57.560 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.2809999999999998e-06 2024-09-14T06:30:57.560 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:57.561 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:57.561 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:30:57.561 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940123+0000", 2024-09-14T06:30:57.561 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.9229999999999996e-06 2024-09-14T06:30:57.561 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:57.561 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:57.561 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:30:57.561 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940128+0000", 2024-09-14T06:30:57.561 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.7790000000000003e-06 2024-09-14T06:30:57.561 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:57.561 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:57.562 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:30:57.562 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.015253+0000", 2024-09-14T06:30:57.562 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.075125792999999996 2024-09-14T06:30:57.562 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:57.562 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:57.562 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:30:57.562 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.015297+0000", 2024-09-14T06:30:57.562 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.3847000000000002e-05 2024-09-14T06:30:57.562 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:57.562 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:57.562 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:30:57.562 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.015360+0000", 2024-09-14T06:30:57.562 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.2661000000000002e-05 2024-09-14T06:30:57.562 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:57.563 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:57.563 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:57.563 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:57.563 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:57.563 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:115202 3.1 3:856be886:::rbd_data.112d4d24c558.0000000000000004:29 [read 3735552~16384 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e96)", 2024-09-14T06:30:57.563 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:30:17.940135+0000", 2024-09-14T06:30:57.563 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 39.574639519000002, 2024-09-14T06:30:57.563 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.075451105000000004, 2024-09-14T06:30:57.563 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:30:57.563 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:30:57.563 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:30:57.563 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:30:57.563 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:30:57.564 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:57.564 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 115202 2024-09-14T06:30:57.564 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:57.564 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:30:57.564 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:57.564 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:30:57.564 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940135+0000", 2024-09-14T06:30:57.564 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:57.564 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:57.564 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:57.564 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:30:57.564 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940135+0000", 2024-09-14T06:30:57.564 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:57.565 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:57.565 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:57.565 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:30:57.565 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940135+0000", 2024-09-14T06:30:57.565 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.0999999999999999e-07 2024-09-14T06:30:57.565 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:57.565 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:57.565 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:30:57.565 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940137+0000", 2024-09-14T06:30:57.565 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.0020000000000001e-06 2024-09-14T06:30:57.565 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:57.566 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:57.566 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:30:57.566 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940143+0000", 2024-09-14T06:30:57.566 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.9850000000000003e-06 2024-09-14T06:30:57.566 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:57.566 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:57.566 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:30:57.566 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940148+0000", 2024-09-14T06:30:57.566 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.9880000000000004e-06 2024-09-14T06:30:57.566 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:57.566 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:57.567 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:30:57.567 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.015400+0000", 2024-09-14T06:30:57.567 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.075252186999999998 2024-09-14T06:30:57.567 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:57.567 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:57.567 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:30:57.567 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.015481+0000", 2024-09-14T06:30:57.567 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.0698000000000006e-05 2024-09-14T06:30:57.567 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:57.567 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:57.567 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:30:57.567 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.015586+0000", 2024-09-14T06:30:57.568 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00010483499999999999 2024-09-14T06:30:57.568 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:57.568 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:57.568 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:57.568 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:57.568 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:57.568 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:115203 3.1 3:856be886:::rbd_data.112d4d24c558.0000000000000004:29 [read 3981312~16384 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e96)", 2024-09-14T06:30:57.568 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:30:17.940156+0000", 2024-09-14T06:30:57.568 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 39.574618293999997, 2024-09-14T06:30:57.569 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.075620860999999998, 2024-09-14T06:30:57.569 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:30:57.569 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:30:57.569 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:30:57.569 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:30:57.569 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:30:57.569 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:57.569 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 115203 2024-09-14T06:30:57.569 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:57.569 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:30:57.569 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:57.569 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:30:57.569 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940156+0000", 2024-09-14T06:30:57.569 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:57.570 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:57.570 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:57.570 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:30:57.570 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940156+0000", 2024-09-14T06:30:57.570 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:57.570 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:57.570 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:57.570 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:30:57.570 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940156+0000", 2024-09-14T06:30:57.571 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.1500000000000005e-07 2024-09-14T06:30:57.571 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:57.571 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:57.571 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:30:57.571 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940158+0000", 2024-09-14T06:30:57.571 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.967e-06 2024-09-14T06:30:57.571 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:57.571 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:57.571 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:30:57.571 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940165+0000", 2024-09-14T06:30:57.571 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.6939999999999998e-06 2024-09-14T06:30:57.572 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:57.572 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:57.572 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:30:57.572 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940170+0000", 2024-09-14T06:30:57.572 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.7539999999999997e-06 2024-09-14T06:30:57.572 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:57.572 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:57.572 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:30:57.572 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.015633+0000", 2024-09-14T06:30:57.573 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.075463334000000007 2024-09-14T06:30:57.573 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:57.573 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:57.573 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:30:57.573 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.015693+0000", 2024-09-14T06:30:57.573 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.9367000000000003e-05 2024-09-14T06:30:57.573 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:57.573 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:57.573 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:30:57.573 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.015777+0000", 2024-09-14T06:30:57.573 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.4229999999999998e-05 2024-09-14T06:30:57.574 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:57.574 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:57.574 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:57.574 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:57.574 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:57.574 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:115204 3.1 3:856be886:::rbd_data.112d4d24c558.0000000000000004:29 [read 4227072~16384 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e96)", 2024-09-14T06:30:57.574 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:30:17.940176+0000", 2024-09-14T06:30:57.574 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 39.574597761, 2024-09-14T06:30:57.574 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.075785505000000003, 2024-09-14T06:30:57.575 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:30:57.575 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:30:57.575 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:30:57.575 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:30:57.575 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:30:57.575 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:57.575 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 115204 2024-09-14T06:30:57.575 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:57.575 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:30:57.575 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:57.575 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:30:57.576 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940176+0000", 2024-09-14T06:30:57.576 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:57.576 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:57.576 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:57.576 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:30:57.576 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940176+0000", 2024-09-14T06:30:57.576 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:57.577 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:57.577 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:57.577 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:30:57.577 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940177+0000", 2024-09-14T06:30:57.577 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.7899999999999999e-07 2024-09-14T06:30:57.577 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:57.577 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:57.577 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:30:57.578 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940179+0000", 2024-09-14T06:30:57.578 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.103e-06 2024-09-14T06:30:57.578 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:57.578 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:57.578 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:30:57.578 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940185+0000", 2024-09-14T06:30:57.578 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.7239999999999997e-06 2024-09-14T06:30:57.578 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:57.578 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:57.578 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:30:57.578 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940190+0000", 2024-09-14T06:30:57.578 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.7269999999999998e-06 2024-09-14T06:30:57.578 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:57.579 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:57.579 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:30:57.579 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.015814+0000", 2024-09-14T06:30:57.579 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.075624114000000006 2024-09-14T06:30:57.579 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:57.579 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:57.579 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:30:57.579 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.015862+0000", 2024-09-14T06:30:57.579 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.8770999999999998e-05 2024-09-14T06:30:57.579 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:57.579 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:57.579 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:30:57.579 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.015962+0000", 2024-09-14T06:30:57.579 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.9586999999999994e-05 2024-09-14T06:30:57.580 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:57.580 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:57.580 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:57.580 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:57.580 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:57.580 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:115205 3.1 3:856be886:::rbd_data.112d4d24c558.0000000000000004:29 [read 4472832~16384 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e96)", 2024-09-14T06:30:57.580 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:30:17.940197+0000", 2024-09-14T06:30:57.580 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 39.574577052000002, 2024-09-14T06:30:57.580 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.076068920999999998, 2024-09-14T06:30:57.580 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:30:57.580 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:30:57.580 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:30:57.580 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:30:57.581 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:30:57.581 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:57.581 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 115205 2024-09-14T06:30:57.581 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:57.581 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:30:57.581 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:57.581 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:30:57.581 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940197+0000", 2024-09-14T06:30:57.581 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:57.581 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:57.581 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:57.581 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:30:57.581 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940197+0000", 2024-09-14T06:30:57.581 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:57.582 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:57.582 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:57.582 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:30:57.582 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940198+0000", 2024-09-14T06:30:57.582 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.5999999999999999e-07 2024-09-14T06:30:57.582 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:57.582 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:57.582 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:30:57.582 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940200+0000", 2024-09-14T06:30:57.582 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.0899999999999999e-06 2024-09-14T06:30:57.582 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:57.582 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:57.582 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:30:57.582 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940206+0000", 2024-09-14T06:30:57.583 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.8710000000000002e-06 2024-09-14T06:30:57.583 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:57.583 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:57.583 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:30:57.583 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940211+0000", 2024-09-14T06:30:57.583 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.9230000000000001e-06 2024-09-14T06:30:57.583 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:57.583 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:57.583 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:30:57.583 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.016059+0000", 2024-09-14T06:30:57.583 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.075848051999999999 2024-09-14T06:30:57.583 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:57.583 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:57.584 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:30:57.584 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.016151+0000", 2024-09-14T06:30:57.584 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.1982000000000001e-05 2024-09-14T06:30:57.584 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:57.584 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:57.584 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:30:57.584 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.016266+0000", 2024-09-14T06:30:57.584 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000115543 2024-09-14T06:30:57.584 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:57.584 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:57.584 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:57.584 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:57.584 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:57.584 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:115206 3.1 3:856be886:::rbd_data.112d4d24c558.0000000000000004:29 [read 4718592~16384 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e96)", 2024-09-14T06:30:57.584 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:30:17.940223+0000", 2024-09-14T06:30:57.585 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 39.574550962000004, 2024-09-14T06:30:57.585 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.076265115999999994, 2024-09-14T06:30:57.585 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:30:57.585 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:30:57.585 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:30:57.585 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:30:57.585 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:30:57.585 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:57.585 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 115206 2024-09-14T06:30:57.585 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:57.585 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:30:57.585 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:57.585 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:30:57.585 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940223+0000", 2024-09-14T06:30:57.586 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:57.586 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:57.586 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:57.586 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:30:57.586 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940223+0000", 2024-09-14T06:30:57.586 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:57.586 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:57.586 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:57.586 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:30:57.586 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940224+0000", 2024-09-14T06:30:57.586 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.0399999999999996e-07 2024-09-14T06:30:57.586 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:57.586 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:57.586 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:30:57.586 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940226+0000", 2024-09-14T06:30:57.587 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.1780000000000002e-06 2024-09-14T06:30:57.587 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:57.587 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:57.587 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:30:57.587 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940232+0000", 2024-09-14T06:30:57.587 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.5010000000000001e-06 2024-09-14T06:30:57.587 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:57.587 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:57.587 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:30:57.587 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940237+0000", 2024-09-14T06:30:57.587 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.0250000000000002e-06 2024-09-14T06:30:57.587 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:57.587 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:57.587 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:30:57.588 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.016305+0000", 2024-09-14T06:30:57.588 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.076067262999999996 2024-09-14T06:30:57.588 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:57.588 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:57.588 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:30:57.588 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.016383+0000", 2024-09-14T06:30:57.588 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.8578000000000006e-05 2024-09-14T06:30:57.588 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:57.588 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:57.588 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:30:57.588 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.016488+0000", 2024-09-14T06:30:57.588 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000105067 2024-09-14T06:30:57.588 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:57.589 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:57.589 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:57.589 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:57.589 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:57.589 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:115207 3.1 3:856be886:::rbd_data.112d4d24c558.0000000000000004:29 [read 4964352~16384 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e96)", 2024-09-14T06:30:57.589 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:30:17.940244+0000", 2024-09-14T06:30:57.589 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 39.574529845999997, 2024-09-14T06:30:57.589 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.076371154999999996, 2024-09-14T06:30:57.589 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:30:57.589 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:30:57.589 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:30:57.589 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:30:57.589 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:30:57.589 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:57.589 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 115207 2024-09-14T06:30:57.590 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:57.590 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:30:57.590 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:57.590 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:30:57.590 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940244+0000", 2024-09-14T06:30:57.590 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:57.590 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:57.590 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:57.590 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:30:57.590 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940244+0000", 2024-09-14T06:30:57.590 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:57.590 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:57.590 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:57.591 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:30:57.591 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940245+0000", 2024-09-14T06:30:57.591 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.1200000000000003e-07 2024-09-14T06:30:57.591 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:57.591 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:57.591 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:30:57.591 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940247+0000", 2024-09-14T06:30:57.591 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.0030000000000001e-06 2024-09-14T06:30:57.591 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:57.591 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:57.591 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:30:57.591 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940252+0000", 2024-09-14T06:30:57.591 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.5520000000000004e-06 2024-09-14T06:30:57.591 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:57.591 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:57.592 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:30:57.592 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940257+0000", 2024-09-14T06:30:57.592 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.6759999999999998e-06 2024-09-14T06:30:57.592 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:57.592 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:57.592 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:30:57.592 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.016521+0000", 2024-09-14T06:30:57.592 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.076263499999999998 2024-09-14T06:30:57.592 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:57.592 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:57.592 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:30:57.592 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.016562+0000", 2024-09-14T06:30:57.592 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.0967000000000003e-05 2024-09-14T06:30:57.592 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:57.592 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:57.593 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:30:57.593 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.016616+0000", 2024-09-14T06:30:57.593 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.3944999999999999e-05 2024-09-14T06:30:57.593 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:57.593 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:57.593 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:57.593 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:57.593 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:57.593 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:115208 3.1 3:856be886:::rbd_data.112d4d24c558.0000000000000004:29 [read 5210112~16384 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e96)", 2024-09-14T06:30:57.593 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:30:17.940264+0000", 2024-09-14T06:30:57.593 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 39.574509988999999, 2024-09-14T06:30:57.593 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.076558288000000002, 2024-09-14T06:30:57.593 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:30:57.593 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:30:57.594 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:30:57.594 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:30:57.594 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:30:57.594 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:57.594 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 115208 2024-09-14T06:30:57.594 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:57.594 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:30:57.594 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:57.594 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:30:57.594 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940264+0000", 2024-09-14T06:30:57.594 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:57.594 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:57.594 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:57.594 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:30:57.595 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940264+0000", 2024-09-14T06:30:57.595 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:57.595 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:57.595 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:57.595 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:30:57.595 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940265+0000", 2024-09-14T06:30:57.595 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.0500000000000004e-07 2024-09-14T06:30:57.595 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:57.595 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:57.595 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:30:57.595 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940267+0000", 2024-09-14T06:30:57.595 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.9810000000000002e-06 2024-09-14T06:30:57.595 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:57.595 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:57.596 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:30:57.596 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940272+0000", 2024-09-14T06:30:57.596 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.6999999999999996e-06 2024-09-14T06:30:57.596 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:57.596 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:57.596 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:30:57.596 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940277+0000", 2024-09-14T06:30:57.596 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.6480000000000002e-06 2024-09-14T06:30:57.596 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:57.596 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:57.596 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:30:57.596 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.016650+0000", 2024-09-14T06:30:57.596 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.076373089000000005 2024-09-14T06:30:57.596 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:57.596 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:57.597 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:30:57.597 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.016724+0000", 2024-09-14T06:30:57.597 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.4105999999999998e-05 2024-09-14T06:30:57.597 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:57.597 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:57.597 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:30:57.597 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.016823+0000", 2024-09-14T06:30:57.597 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.8258999999999996e-05 2024-09-14T06:30:57.597 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:57.597 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:57.597 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:57.598 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:57.598 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:57.598 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:115209 3.1 3:856be886:::rbd_data.112d4d24c558.0000000000000004:29 [read 5455872~16384 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e96)", 2024-09-14T06:30:57.598 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:30:17.940283+0000", 2024-09-14T06:30:57.598 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 39.574490963000002, 2024-09-14T06:30:57.598 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.076724029999999999, 2024-09-14T06:30:57.598 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:30:57.598 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:30:57.598 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:30:57.598 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:30:57.598 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:30:57.598 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:57.598 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 115209 2024-09-14T06:30:57.598 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:57.598 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:30:57.599 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:57.599 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:30:57.599 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940283+0000", 2024-09-14T06:30:57.599 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:57.599 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:57.599 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:57.599 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:30:57.599 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940283+0000", 2024-09-14T06:30:57.599 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:57.599 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:57.599 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:57.599 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:30:57.599 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940284+0000", 2024-09-14T06:30:57.599 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.6700000000000003e-07 2024-09-14T06:30:57.600 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:57.600 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:57.600 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:30:57.600 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940286+0000", 2024-09-14T06:30:57.600 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.9520000000000001e-06 2024-09-14T06:30:57.600 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:57.600 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:57.600 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:30:57.600 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940292+0000", 2024-09-14T06:30:57.600 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.7339999999999996e-06 2024-09-14T06:30:57.600 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:57.600 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:57.600 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:30:57.600 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940296+0000", 2024-09-14T06:30:57.601 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.7700000000000001e-06 2024-09-14T06:30:57.601 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:57.601 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:57.601 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:30:57.601 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.016859+0000", 2024-09-14T06:30:57.601 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.07656317 2024-09-14T06:30:57.601 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:57.601 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:57.601 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:30:57.601 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.016929+0000", 2024-09-14T06:30:57.601 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.9679999999999997e-05 2024-09-14T06:30:57.601 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:57.601 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:57.602 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:30:57.602 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.017007+0000", 2024-09-14T06:30:57.602 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.8157000000000004e-05 2024-09-14T06:30:57.602 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:57.602 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:57.602 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:57.602 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:57.602 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:57.602 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:115210 3.1 3:856be886:::rbd_data.112d4d24c558.0000000000000004:29 [read 5701632~16384 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e96)", 2024-09-14T06:30:57.602 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:30:17.940303+0000", 2024-09-14T06:30:57.602 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 39.574471445999997, 2024-09-14T06:30:57.602 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.076821358000000006, 2024-09-14T06:30:57.602 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:30:57.603 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:30:57.603 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:30:57.603 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:30:57.603 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:30:57.603 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:57.603 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 115210 2024-09-14T06:30:57.603 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:57.603 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:30:57.603 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:57.603 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:30:57.603 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940303+0000", 2024-09-14T06:30:57.604 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:57.604 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:57.604 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:57.604 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:30:57.604 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940303+0000", 2024-09-14T06:30:57.604 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:57.604 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:57.604 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:57.604 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:30:57.605 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940303+0000", 2024-09-14T06:30:57.605 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.3000000000000001e-07 2024-09-14T06:30:57.605 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:57.605 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:57.605 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:30:57.605 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940306+0000", 2024-09-14T06:30:57.605 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.255e-06 2024-09-14T06:30:57.605 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:57.605 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:57.605 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:30:57.605 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940312+0000", 2024-09-14T06:30:57.605 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.9710000000000001e-06 2024-09-14T06:30:57.606 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:57.606 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:57.606 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:30:57.606 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940316+0000", 2024-09-14T06:30:57.606 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.5340000000000001e-06 2024-09-14T06:30:57.606 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:57.606 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:57.606 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:30:57.606 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.017033+0000", 2024-09-14T06:30:57.606 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.076717169000000002 2024-09-14T06:30:57.606 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:57.607 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:57.607 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:30:57.607 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.017073+0000", 2024-09-14T06:30:57.607 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.9582000000000002e-05 2024-09-14T06:30:57.607 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:57.607 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:57.607 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:30:57.607 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.017124+0000", 2024-09-14T06:30:57.607 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.1316999999999997e-05 2024-09-14T06:30:57.607 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:57.607 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:57.607 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:57.608 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:57.608 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:57.608 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:115219 3.1 3:856be886:::rbd_data.112d4d24c558.0000000000000004:29 [read 65536~16384 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e96)", 2024-09-14T06:30:57.608 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:30:17.940323+0000", 2024-09-14T06:30:57.608 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 39.574451584999998, 2024-09-14T06:30:57.608 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.076915205, 2024-09-14T06:30:57.608 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:30:57.608 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:30:57.608 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:30:57.608 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:30:57.608 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:30:57.609 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:57.609 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 115219 2024-09-14T06:30:57.609 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:57.609 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:30:57.609 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:57.609 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:30:57.609 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940323+0000", 2024-09-14T06:30:57.609 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:57.609 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:57.609 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:57.609 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:30:57.610 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940323+0000", 2024-09-14T06:30:57.610 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:57.610 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:57.610 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:57.610 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:30:57.610 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940323+0000", 2024-09-14T06:30:57.610 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.0800000000000005e-07 2024-09-14T06:30:57.610 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:57.610 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:57.611 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:30:57.611 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940328+0000", 2024-09-14T06:30:57.611 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.887e-06 2024-09-14T06:30:57.611 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:57.611 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:57.611 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:30:57.611 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940334+0000", 2024-09-14T06:30:57.611 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.3269999999999997e-06 2024-09-14T06:30:57.611 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:57.611 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:57.611 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:30:57.612 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940339+0000", 2024-09-14T06:30:57.612 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.7960000000000003e-06 2024-09-14T06:30:57.612 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:57.612 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:57.612 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:30:57.612 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.017148+0000", 2024-09-14T06:30:57.612 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.076809125000000006 2024-09-14T06:30:57.612 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:57.612 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:57.612 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:30:57.613 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.017186+0000", 2024-09-14T06:30:57.613 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.7283000000000002e-05 2024-09-14T06:30:57.613 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:57.613 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:57.613 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:30:57.613 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.017238+0000", 2024-09-14T06:30:57.613 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.2278999999999999e-05 2024-09-14T06:30:57.613 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:57.613 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:57.614 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:57.614 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:57.614 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:57.614 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:115220 3.1 3:856be886:::rbd_data.112d4d24c558.0000000000000004:29 [read 311296~16384 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e96)", 2024-09-14T06:30:57.614 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:30:17.940346+0000", 2024-09-14T06:30:57.614 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 39.574428456, 2024-09-14T06:30:57.614 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.077033293000000003, 2024-09-14T06:30:57.614 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:30:57.615 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:30:57.615 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:30:57.615 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:30:57.615 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:30:57.615 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:57.615 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 115220 2024-09-14T06:30:57.615 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:57.615 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:30:57.615 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:57.615 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:30:57.615 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940346+0000", 2024-09-14T06:30:57.616 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:57.616 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:57.616 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:57.616 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:30:57.616 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940346+0000", 2024-09-14T06:30:57.616 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:57.616 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:57.616 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:57.616 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:30:57.616 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940346+0000", 2024-09-14T06:30:57.616 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.4300000000000003e-07 2024-09-14T06:30:57.616 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:57.616 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:57.616 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:30:57.616 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940348+0000", 2024-09-14T06:30:57.617 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.0030000000000001e-06 2024-09-14T06:30:57.617 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:57.617 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:57.617 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:30:57.617 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.004655+0000", 2024-09-14T06:30:57.617 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.064306323999999998 2024-09-14T06:30:57.617 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:57.617 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:57.617 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:30:57.617 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.004671+0000", 2024-09-14T06:30:57.617 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.6589999999999999e-05 2024-09-14T06:30:57.617 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:57.617 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:57.618 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:30:57.618 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.017262+0000", 2024-09-14T06:30:57.618 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.012590570000000001 2024-09-14T06:30:57.618 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:57.618 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:57.618 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:30:57.618 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.017321+0000", 2024-09-14T06:30:57.618 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.9210999999999999e-05 2024-09-14T06:30:57.618 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:57.618 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:57.618 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:30:57.618 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.017379+0000", 2024-09-14T06:30:57.618 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.8051999999999999e-05 2024-09-14T06:30:57.618 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:57.618 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:57.619 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:57.619 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:57.619 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:57.619 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-14T06:30:57.621 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-14T06:30:57.621 INFO:teuthology.orchestra.run.smithi094.stdout:osd.4: {} 2024-09-14T06:30:57.621 INFO:teuthology.orchestra.run.smithi094.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-14T06:30:57.622 INFO:teuthology.orchestra.run.smithi094.stdout:osd.5: {} 2024-09-14T06:30:57.622 INFO:teuthology.orchestra.run.smithi094.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-14T06:30:57.622 INFO:teuthology.orchestra.run.smithi094.stdout:osd.6: {} 2024-09-14T06:30:57.622 INFO:teuthology.orchestra.run.smithi094.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-14T06:30:57.623 INFO:teuthology.orchestra.run.smithi094.stdout:osd.7: {} 2024-09-14T06:30:57.623 INFO:teuthology.orchestra.run.smithi094.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-14T06:30:57.624 INFO:tasks.ceph.osd.6.smithi138.stderr:2024-09-14T06:30:57.619+0000 7f8cd8849640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33620) UID: 0 2024-09-14T06:30:57.721 INFO:tasks.ceph.osd.0.smithi094.stderr:2024-09-14T06:30:57.720+0000 7f07c9333640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33636) UID: 0 2024-09-14T06:30:57.767 INFO:teuthology.orchestra.run.smithi094.stderr:nodeep-scrub is unset 2024-09-14T06:30:57.768 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-14T06:30:57.769 INFO:teuthology.orchestra.run.smithi094.stdout: "ops": [], 2024-09-14T06:30:57.769 INFO:teuthology.orchestra.run.smithi094.stdout: "num_ops": 0 2024-09-14T06:30:57.769 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-14T06:30:57.783 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-14T06:30:57.821 INFO:tasks.ceph.osd.4.smithi138.stderr:2024-09-14T06:30:57.820+0000 7f6c653c8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33615) UID: 0 2024-09-14T06:30:57.861 INFO:teuthology.orchestra.run.smithi094.stdout:3830 write 0x60819cd thru 0x60828ba (0xeee bytes) 2024-09-14T06:30:57.877 INFO:teuthology.orchestra.run.smithi094.stdout:3834 writesame 0xad1d702 thru 0xad28b9f data_size 0x2(0xb49e bytes) 2024-09-14T06:30:57.877 INFO:teuthology.orchestra.run.smithi094.stdout:3835 trunc from 0xad28ba0 to 0xb5b8cd0 2024-09-14T06:30:57.882 INFO:teuthology.orchestra.run.smithi094.stdout:3836 punch from 0xa0edd97 to 0xa0f4598, (0x6801 bytes) 2024-09-14T06:30:57.882 INFO:teuthology.orchestra.run.smithi094.stdout:3837 read 0x2a408ac thru 0x2a42072 (0x17c7 bytes) 2024-09-14T06:30:57.883 INFO:teuthology.orchestra.run.smithi094.stdout:3839 punch from 0x7cb1ebe to 0x7cb5e00, (0x3f42 bytes) 2024-09-14T06:30:57.883 INFO:teuthology.orchestra.run.smithi094.stdout:3840 read 0x9ec139e thru 0x9ec9472 (0x80d5 bytes) 2024-09-14T06:30:57.886 INFO:teuthology.orchestra.run.smithi094.stdout:3842 trunc from 0xb5b8cd0 to 0xdfbda46 2024-09-14T06:30:57.892 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-14T06:30:57.892 INFO:teuthology.orchestra.run.smithi094.stdout:3845 punch from 0x15186e8 to 0x1524360, (0xbc78 bytes) 2024-09-14T06:30:57.892 INFO:teuthology.orchestra.run.smithi094.stdout:3846 punch from 0x966403a to 0x9669655, (0x561b bytes) 2024-09-14T06:30:57.892 INFO:teuthology.orchestra.run.smithi094.stdout:3848 write 0x4eac509 thru 0x4eb4c28 (0x8720 bytes) 2024-09-14T06:30:57.896 INFO:teuthology.orchestra.run.smithi094.stdout:3849 trunc from 0xdfbda46 to 0xc022151 2024-09-14T06:30:57.905 INFO:teuthology.orchestra.run.smithi094.stdout:3850 trunc from 0xc022151 to 0xda52dc5 2024-09-14T06:30:57.911 INFO:teuthology.orchestra.run.smithi094.stdout:3852 write 0x2ba1927 thru 0x2ba1fee (0x6c8 bytes) 2024-09-14T06:30:57.912 INFO:teuthology.orchestra.run.smithi094.stdout:3855 read 0x969a178 thru 0x96a7da3 (0xdc2c bytes) 2024-09-14T06:30:57.919 INFO:teuthology.orchestra.run.smithi094.stdout:3856 writesame 0xec8c537 thru 0xec990b6 data_size 0x58(0xcb80 bytes) 2024-09-14T06:30:57.919 INFO:teuthology.orchestra.run.smithi094.stdout:3859 writesame 0xb87417e thru 0xb87a7fb data_size 0x3(0x667e bytes) 2024-09-14T06:30:57.922 INFO:tasks.ceph.osd.1.smithi094.stderr:2024-09-14T06:30:57.921+0000 7f328e47a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33641) UID: 0 2024-09-14T06:30:57.927 INFO:teuthology.orchestra.run.smithi094.stdout:3862 write 0x2689e01 thru 0x268e7cf (0x49cf bytes) 2024-09-14T06:30:57.929 INFO:teuthology.orchestra.run.smithi094.stdout:3863 write 0x3a79b28 thru 0x3a87372 (0xd84b bytes) 2024-09-14T06:30:57.931 INFO:teuthology.orchestra.run.smithi094.stdout:3864 writesame 0xd69bdb3 thru 0xd6a1b42 data_size 0x4(0x5d90 bytes) 2024-09-14T06:30:57.938 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-14T06:30:57.938 INFO:teuthology.orchestra.run.smithi094.stdout:3867 writesame 0x2618656 thru 0x2626d5e data_size 0x5(0xe709 bytes) 2024-09-14T06:30:57.943 INFO:teuthology.orchestra.run.smithi094.stdout:3868 read 0x64ceff6 thru 0x64d6f4e (0x7f59 bytes) 2024-09-14T06:30:57.950 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-14T06:30:57.950 INFO:teuthology.orchestra.run.smithi094.stdout: "ops": [], 2024-09-14T06:30:57.951 INFO:teuthology.orchestra.run.smithi094.stdout: "complaint_time": 30, 2024-09-14T06:30:57.951 INFO:teuthology.orchestra.run.smithi094.stdout: "num_blocked_ops": 0 2024-09-14T06:30:57.951 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-14T06:30:57.951 INFO:teuthology.orchestra.run.smithi094.stdout:3869 trunc from 0xec990b7 to 0x91e0004 2024-09-14T06:30:57.959 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-14T06:30:57.960 INFO:teuthology.orchestra.run.smithi094.stdout:3873 read 0x5e80122 thru 0x5e81816 (0x16f5 bytes) 2024-09-14T06:30:57.961 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-14T06:30:57.972 INFO:teuthology.orchestra.run.smithi094.stdout:3874 write 0xe181e62 thru 0xe1917e6 (0xf985 bytes) 2024-09-14T06:30:57.974 INFO:teuthology.orchestra.run.smithi094.stdout:3877 read 0xd95b726 thru 0xd95f001 (0x38dc bytes) 2024-09-14T06:30:57.974 INFO:teuthology.orchestra.run.smithi094.stdout:3878 clone 31 order 20 su 8192 sc 5 2024-09-14T06:30:58.023 INFO:tasks.ceph.osd.5.smithi138.stderr:2024-09-14T06:30:58.022+0000 7fe13746f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33618) UID: 0 2024-09-14T06:30:58.076 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-14T06:30:58.076 INFO:teuthology.orchestra.run.smithi094.stdout: "size": 20, 2024-09-14T06:30:58.076 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 600, 2024-09-14T06:30:58.076 INFO:teuthology.orchestra.run.smithi094.stdout: "ops": [ 2024-09-14T06:30:58.076 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:58.076 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:14918 3.0 3:05c39535:::rbd_data.112d92027590.0000000000000009:f [read 795648~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:30:58.076 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:26.083834+0000", 2024-09-14T06:30:58.077 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 91.99026594, 2024-09-14T06:30:58.077 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.041776287000000002, 2024-09-14T06:30:58.077 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:30:58.077 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:30:58.077 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:30:58.077 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:30:58.077 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:30:58.077 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:58.077 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 14918 2024-09-14T06:30:58.077 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:58.077 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:30:58.077 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:58.077 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:30:58.078 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.083834+0000", 2024-09-14T06:30:58.078 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:58.078 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:58.078 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:58.078 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:30:58.078 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.083834+0000", 2024-09-14T06:30:58.078 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:58.078 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:58.078 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:58.078 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:30:58.078 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.083834+0000", 2024-09-14T06:30:58.078 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.0100000000000001e-07 2024-09-14T06:30:58.078 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:58.078 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:58.079 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:30:58.079 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.083835+0000", 2024-09-14T06:30:58.079 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.3900000000000003e-07 2024-09-14T06:30:58.079 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:58.079 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:58.079 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:30:58.079 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.083838+0000", 2024-09-14T06:30:58.079 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.9830000000000001e-06 2024-09-14T06:30:58.079 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:58.079 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:58.079 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:30:58.079 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.083840+0000", 2024-09-14T06:30:58.079 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.4679999999999999e-06 2024-09-14T06:30:58.079 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:58.079 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:58.080 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:30:58.080 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.125455+0000", 2024-09-14T06:30:58.080 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.041614578999999999 2024-09-14T06:30:58.080 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:58.080 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:58.080 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:30:58.080 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.125496+0000", 2024-09-14T06:30:58.080 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.1047000000000002e-05 2024-09-14T06:30:58.081 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:58.081 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:58.081 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:30:58.081 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.125610+0000", 2024-09-14T06:30:58.081 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00011407 2024-09-14T06:30:58.084 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:58.084 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:58.084 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:58.084 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:58.084 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:58.085 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:14919 3.0 3:05c39535:::rbd_data.112d92027590.0000000000000009:f [read 805888~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:30:58.085 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:26.083843+0000", 2024-09-14T06:30:58.085 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 91.990256192000004, 2024-09-14T06:30:58.085 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.041992356000000002, 2024-09-14T06:30:58.085 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:30:58.085 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:30:58.085 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:30:58.085 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:30:58.085 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:30:58.085 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:58.085 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 14919 2024-09-14T06:30:58.085 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:58.085 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:30:58.085 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:58.086 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:30:58.086 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.083843+0000", 2024-09-14T06:30:58.086 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:58.086 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:58.086 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:58.086 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:30:58.086 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.083843+0000", 2024-09-14T06:30:58.086 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:58.086 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:58.086 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:58.086 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:30:58.086 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.083844+0000", 2024-09-14T06:30:58.086 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.3300000000000001e-07 2024-09-14T06:30:58.086 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:58.087 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:58.087 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:30:58.087 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.083845+0000", 2024-09-14T06:30:58.087 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.4729999999999999e-06 2024-09-14T06:30:58.087 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:58.087 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:58.087 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:30:58.087 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.083850+0000", 2024-09-14T06:30:58.087 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.0130000000000001e-06 2024-09-14T06:30:58.087 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:58.087 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:58.088 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:30:58.088 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.083854+0000", 2024-09-14T06:30:58.088 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.4759999999999998e-06 2024-09-14T06:30:58.088 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:58.088 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:58.088 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:30:58.089 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.125636+0000", 2024-09-14T06:30:58.089 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.041782683000000001 2024-09-14T06:30:58.089 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:58.089 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:58.089 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:30:58.089 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.125680+0000", 2024-09-14T06:30:58.089 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.3614e-05 2024-09-14T06:30:58.090 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:58.090 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:58.090 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:30:58.090 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.125836+0000", 2024-09-14T06:30:58.090 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000155864 2024-09-14T06:30:58.090 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:58.090 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:58.090 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:58.090 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:58.090 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:58.090 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:14920 3.0 3:05c39535:::rbd_data.112d92027590.0000000000000009:f [read 816128~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:30:58.090 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:26.083868+0000", 2024-09-14T06:30:58.091 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 91.990231531000006, 2024-09-14T06:30:58.091 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.042166771999999998, 2024-09-14T06:30:58.091 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:30:58.091 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:30:58.091 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:30:58.091 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:30:58.091 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:30:58.092 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:58.092 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 14920 2024-09-14T06:30:58.092 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:58.092 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:30:58.092 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:58.092 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:30:58.092 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.083868+0000", 2024-09-14T06:30:58.092 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:58.092 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:58.092 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:58.092 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:30:58.093 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.083868+0000", 2024-09-14T06:30:58.093 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:58.093 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:58.093 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:58.093 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:30:58.093 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.083869+0000", 2024-09-14T06:30:58.093 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.99e-07 2024-09-14T06:30:58.093 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:58.093 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:58.093 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:30:58.094 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.083870+0000", 2024-09-14T06:30:58.094 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.782e-06 2024-09-14T06:30:58.094 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:58.094 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:58.094 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:30:58.094 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.083876+0000", 2024-09-14T06:30:58.094 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.3809999999999998e-06 2024-09-14T06:30:58.094 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:58.094 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:58.094 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:30:58.094 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.083882+0000", 2024-09-14T06:30:58.094 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.9479999999999996e-06 2024-09-14T06:30:58.094 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:58.095 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:58.095 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:30:58.095 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.125878+0000", 2024-09-14T06:30:58.095 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.041996264999999998 2024-09-14T06:30:58.095 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:58.095 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:58.095 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:30:58.095 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.125929+0000", 2024-09-14T06:30:58.095 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.0498999999999999e-05 2024-09-14T06:30:58.095 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:58.095 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:58.095 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:30:58.095 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.126035+0000", 2024-09-14T06:30:58.096 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000106298 2024-09-14T06:30:58.096 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:58.096 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:58.096 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:58.096 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:58.096 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:58.096 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:14921 3.0 3:05c39535:::rbd_data.112d92027590.0000000000000009:f [read 826368~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:30:58.096 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:26.083889+0000", 2024-09-14T06:30:58.096 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 91.990210520999995, 2024-09-14T06:30:58.096 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.042435309999999997, 2024-09-14T06:30:58.097 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:30:58.097 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:30:58.097 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:30:58.097 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:30:58.097 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:30:58.097 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:58.097 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 14921 2024-09-14T06:30:58.097 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:58.097 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:30:58.097 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:58.097 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:30:58.097 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.083889+0000", 2024-09-14T06:30:58.097 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:58.098 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:58.098 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:58.098 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:30:58.098 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.083889+0000", 2024-09-14T06:30:58.098 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:58.098 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:58.098 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:58.098 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:30:58.098 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.086033+0000", 2024-09-14T06:30:58.098 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0021444179999999999 2024-09-14T06:30:58.098 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:58.098 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:58.098 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:30:58.098 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.086038+0000", 2024-09-14T06:30:58.099 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.0969999999999999e-06 2024-09-14T06:30:58.099 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:58.099 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:58.099 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:30:58.099 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.086046+0000", 2024-09-14T06:30:58.099 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.9440000000000005e-06 2024-09-14T06:30:58.099 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:58.099 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:58.099 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:30:58.099 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.086050+0000", 2024-09-14T06:30:58.099 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.69e-06 2024-09-14T06:30:58.100 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:58.100 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:58.100 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:30:58.100 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.126080+0000", 2024-09-14T06:30:58.100 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.040030237000000003 2024-09-14T06:30:58.100 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:58.100 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:58.100 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:30:58.100 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.126163+0000", 2024-09-14T06:30:58.100 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.2071999999999996e-05 2024-09-14T06:30:58.101 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:58.101 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:58.101 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:30:58.101 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.126324+0000", 2024-09-14T06:30:58.101 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00016185200000000001 2024-09-14T06:30:58.101 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:58.101 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:58.101 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:58.101 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:58.101 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:58.101 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:55627 3.0 3:06490baf:::rbd_data.112dca2f6ae1.0000000000000030:17 [read 2002944~4096 [fadvise_nocache]] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:30:58.101 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:44.599966+0000", 2024-09-14T06:30:58.101 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 73.474133573000003, 2024-09-14T06:30:58.102 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.041868546999999999, 2024-09-14T06:30:58.102 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:30:58.102 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:30:58.102 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:30:58.102 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:30:58.102 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:30:58.102 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:58.102 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 55627 2024-09-14T06:30:58.102 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:58.102 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:30:58.102 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:58.102 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:30:58.103 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.599966+0000", 2024-09-14T06:30:58.103 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:58.103 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:58.103 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:58.103 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:30:58.103 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.599966+0000", 2024-09-14T06:30:58.103 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:58.103 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:58.103 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:58.103 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:30:58.103 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.599966+0000", 2024-09-14T06:30:58.103 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.0900000000000001e-07 2024-09-14T06:30:58.103 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:58.104 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:58.104 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:30:58.104 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.599967+0000", 2024-09-14T06:30:58.104 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.64e-07 2024-09-14T06:30:58.104 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:58.104 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:58.104 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:30:58.104 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.599970+0000", 2024-09-14T06:30:58.104 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.9670000000000001e-06 2024-09-14T06:30:58.104 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:58.104 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:58.105 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:30:58.105 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.599973+0000", 2024-09-14T06:30:58.105 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.632e-06 2024-09-14T06:30:58.105 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:58.105 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:58.105 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:30:58.105 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.641665+0000", 2024-09-14T06:30:58.105 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.041692547000000003 2024-09-14T06:30:58.105 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:58.105 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:58.105 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:30:58.105 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.641740+0000", 2024-09-14T06:30:58.105 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.4559000000000002e-05 2024-09-14T06:30:58.105 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:58.106 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:58.106 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:30:58.106 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.641835+0000", 2024-09-14T06:30:58.106 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.4668999999999998e-05 2024-09-14T06:30:58.106 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:58.106 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:58.106 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:58.106 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:58.106 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:58.106 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:55628 3.0 3:06490baf:::rbd_data.112dca2f6ae1.0000000000000030:17 [read 2027520~4096 [fadvise_nocache]] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:30:58.106 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:44.599976+0000", 2024-09-14T06:30:58.106 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 73.474123786000007, 2024-09-14T06:30:58.106 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.041991621, 2024-09-14T06:30:58.106 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:30:58.106 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:30:58.107 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:30:58.107 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:30:58.107 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:30:58.107 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:58.107 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 55628 2024-09-14T06:30:58.107 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:58.107 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:30:58.107 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:58.107 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:30:58.107 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.599976+0000", 2024-09-14T06:30:58.107 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:58.108 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:58.108 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:58.108 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:30:58.108 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.599976+0000", 2024-09-14T06:30:58.108 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:58.108 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:58.108 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:58.108 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:30:58.108 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.599976+0000", 2024-09-14T06:30:58.108 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.1299999999999999e-07 2024-09-14T06:30:58.108 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:58.108 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:58.108 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:30:58.109 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.599977+0000", 2024-09-14T06:30:58.109 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.7999999999999993e-07 2024-09-14T06:30:58.109 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:58.109 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:58.109 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:30:58.109 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.599980+0000", 2024-09-14T06:30:58.109 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.01e-06 2024-09-14T06:30:58.109 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:58.109 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:58.109 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:30:58.109 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.599983+0000", 2024-09-14T06:30:58.109 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.5409999999999999e-06 2024-09-14T06:30:58.109 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:58.109 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:58.110 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:30:58.110 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.641869+0000", 2024-09-14T06:30:58.110 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.041886235000000001 2024-09-14T06:30:58.110 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:58.110 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:58.110 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:30:58.110 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.641913+0000", 2024-09-14T06:30:58.110 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.4357000000000002e-05 2024-09-14T06:30:58.110 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:58.110 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:58.110 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:30:58.110 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.641967+0000", 2024-09-14T06:30:58.110 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.4284999999999999e-05 2024-09-14T06:30:58.110 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:58.111 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:58.111 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:58.111 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:58.111 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:58.111 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:55629 3.0 3:06490baf:::rbd_data.112dca2f6ae1.0000000000000030:17 [read 2052096~4096 [fadvise_nocache]] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:30:58.111 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:44.599985+0000", 2024-09-14T06:30:58.111 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 73.474114080000007, 2024-09-14T06:30:58.111 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.042183375000000002, 2024-09-14T06:30:58.111 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:30:58.111 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:30:58.111 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:30:58.111 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:30:58.111 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:30:58.112 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:58.112 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 55629 2024-09-14T06:30:58.112 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:58.112 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:30:58.112 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:58.112 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:30:58.112 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.599985+0000", 2024-09-14T06:30:58.112 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:58.112 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:58.112 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:58.112 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:30:58.112 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.599985+0000", 2024-09-14T06:30:58.112 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:58.112 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:58.113 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:58.113 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:30:58.113 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.599986+0000", 2024-09-14T06:30:58.113 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.05e-07 2024-09-14T06:30:58.113 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:58.113 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:58.113 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:30:58.113 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.599987+0000", 2024-09-14T06:30:58.113 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.8400000000000002e-07 2024-09-14T06:30:58.113 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:58.113 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:58.113 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:30:58.114 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.599990+0000", 2024-09-14T06:30:58.114 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.5640000000000001e-06 2024-09-14T06:30:58.114 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:58.114 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:58.114 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:30:58.114 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.599993+0000", 2024-09-14T06:30:58.114 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.615e-06 2024-09-14T06:30:58.114 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:58.114 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:58.114 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:30:58.114 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.642003+0000", 2024-09-14T06:30:58.114 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.042010345999999997 2024-09-14T06:30:58.115 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:58.115 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:58.115 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:30:58.115 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.642080+0000", 2024-09-14T06:30:58.115 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.6538999999999993e-05 2024-09-14T06:30:58.115 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:58.115 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:58.115 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:30:58.115 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.642169+0000", 2024-09-14T06:30:58.115 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.9122000000000005e-05 2024-09-14T06:30:58.115 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:58.115 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:58.115 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:58.115 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:58.116 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:58.116 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:55630 3.0 3:06490baf:::rbd_data.112dca2f6ae1.0000000000000030:17 [read 2076672~4096 [fadvise_nocache]] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:30:58.116 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:44.599996+0000", 2024-09-14T06:30:58.116 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 73.474103602, 2024-09-14T06:30:58.116 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.042299615999999998, 2024-09-14T06:30:58.116 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:30:58.116 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:30:58.116 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:30:58.116 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:30:58.116 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:30:58.116 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:58.116 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 55630 2024-09-14T06:30:58.116 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:58.116 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:30:58.117 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:58.117 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:30:58.117 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.599996+0000", 2024-09-14T06:30:58.117 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:58.117 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:58.117 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:58.117 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:30:58.117 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.599996+0000", 2024-09-14T06:30:58.117 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:58.117 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:58.117 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:58.117 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:30:58.117 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.599996+0000", 2024-09-14T06:30:58.118 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.1500000000000001e-07 2024-09-14T06:30:58.118 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:58.118 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:58.118 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:30:58.118 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.599997+0000", 2024-09-14T06:30:58.118 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.33e-07 2024-09-14T06:30:58.118 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:58.118 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:58.118 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:30:58.118 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600000+0000", 2024-09-14T06:30:58.118 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.0240000000000002e-06 2024-09-14T06:30:58.119 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:58.119 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:58.119 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:30:58.119 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600003+0000", 2024-09-14T06:30:58.119 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.3640000000000002e-06 2024-09-14T06:30:58.119 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:58.119 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:58.119 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:30:58.119 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.642200+0000", 2024-09-14T06:30:58.119 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.042197571000000003 2024-09-14T06:30:58.119 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:58.119 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:58.120 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:30:58.120 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.642243+0000", 2024-09-14T06:30:58.120 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.2765000000000001e-05 2024-09-14T06:30:58.120 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:58.120 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:58.120 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:30:58.120 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.642296+0000", 2024-09-14T06:30:58.120 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.2744000000000002e-05 2024-09-14T06:30:58.120 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:58.120 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:58.120 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:58.120 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:58.120 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:58.120 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:55631 3.0 3:06490baf:::rbd_data.112dca2f6ae1.0000000000000030:17 [read 1048576~4096 [fadvise_nocache]] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:30:58.121 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:44.600006+0000", 2024-09-14T06:30:58.121 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 73.474094014000002, 2024-09-14T06:30:58.121 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.042487905999999999, 2024-09-14T06:30:58.121 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:30:58.121 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:30:58.121 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:30:58.121 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:30:58.121 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:30:58.121 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:58.121 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 55631 2024-09-14T06:30:58.121 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:58.121 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:30:58.121 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:58.121 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:30:58.122 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600006+0000", 2024-09-14T06:30:58.122 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:58.122 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:58.122 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:58.122 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:30:58.122 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600006+0000", 2024-09-14T06:30:58.122 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:58.122 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:58.122 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:58.122 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:30:58.122 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600006+0000", 2024-09-14T06:30:58.122 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.0800000000000001e-07 2024-09-14T06:30:58.122 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:58.122 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:58.123 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:30:58.123 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600007+0000", 2024-09-14T06:30:58.123 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.7199999999999997e-07 2024-09-14T06:30:58.123 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:58.123 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:58.123 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:30:58.123 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600010+0000", 2024-09-14T06:30:58.123 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.0280000000000001e-06 2024-09-14T06:30:58.123 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:58.123 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:58.123 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:30:58.123 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600012+0000", 2024-09-14T06:30:58.123 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.475e-06 2024-09-14T06:30:58.123 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:58.123 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:58.124 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:30:58.124 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.642329+0000", 2024-09-14T06:30:58.124 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.042317157000000001 2024-09-14T06:30:58.124 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:58.124 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:58.124 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:30:58.124 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.642403+0000", 2024-09-14T06:30:58.124 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.3941000000000002e-05 2024-09-14T06:30:58.124 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:58.124 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:58.124 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:30:58.124 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.642493+0000", 2024-09-14T06:30:58.124 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.0124999999999995e-05 2024-09-14T06:30:58.124 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:58.125 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:58.125 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:58.125 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:58.125 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:58.125 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:55632 3.0 3:06490baf:::rbd_data.112dca2f6ae1.0000000000000030:17 [read 1073152~4096 [fadvise_nocache]] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:30:58.125 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:44.600015+0000", 2024-09-14T06:30:58.125 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 73.474084261000002, 2024-09-14T06:30:58.125 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.042604922000000003, 2024-09-14T06:30:58.125 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:30:58.125 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:30:58.125 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:30:58.125 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:30:58.125 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:30:58.125 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:58.126 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 55632 2024-09-14T06:30:58.126 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:58.126 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:30:58.126 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:58.126 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:30:58.126 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600015+0000", 2024-09-14T06:30:58.126 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:58.126 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:58.126 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:58.126 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:30:58.126 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600015+0000", 2024-09-14T06:30:58.126 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:58.126 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:58.126 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:58.127 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:30:58.127 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600016+0000", 2024-09-14T06:30:58.127 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.1e-07 2024-09-14T06:30:58.127 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:58.127 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:58.127 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:30:58.127 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600017+0000", 2024-09-14T06:30:58.127 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.9900000000000009e-07 2024-09-14T06:30:58.127 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:58.127 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:58.127 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:30:58.127 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600020+0000", 2024-09-14T06:30:58.127 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.4170000000000001e-06 2024-09-14T06:30:58.127 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:58.127 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:58.128 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:30:58.128 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600023+0000", 2024-09-14T06:30:58.128 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.588e-06 2024-09-14T06:30:58.128 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:58.128 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:58.128 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:30:58.128 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.642526+0000", 2024-09-14T06:30:58.128 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.042503236999999999 2024-09-14T06:30:58.128 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:58.128 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:58.128 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:30:58.128 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.642568+0000", 2024-09-14T06:30:58.128 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.1925000000000003e-05 2024-09-14T06:30:58.129 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:58.129 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:58.129 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:30:58.129 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.642620+0000", 2024-09-14T06:30:58.129 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.2546000000000001e-05 2024-09-14T06:30:58.129 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:58.129 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:58.129 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:58.129 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:58.129 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:58.129 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:55633 3.0 3:06490baf:::rbd_data.112dca2f6ae1.0000000000000030:17 [read 1097728~4096 [fadvise_nocache]] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:30:58.129 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:44.600026+0000", 2024-09-14T06:30:58.129 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 73.474073992000001, 2024-09-14T06:30:58.129 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.042797474000000002, 2024-09-14T06:30:58.130 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:30:58.130 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:30:58.130 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:30:58.130 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:30:58.130 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:30:58.130 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:58.130 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 55633 2024-09-14T06:30:58.130 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:58.130 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:30:58.130 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:58.130 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:30:58.130 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600026+0000", 2024-09-14T06:30:58.130 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:58.131 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:58.131 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:58.131 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:30:58.131 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600026+0000", 2024-09-14T06:30:58.131 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:58.131 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:58.131 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:58.131 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:30:58.131 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600026+0000", 2024-09-14T06:30:58.131 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.3099999999999999e-07 2024-09-14T06:30:58.131 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:58.131 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:58.131 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:30:58.132 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600029+0000", 2024-09-14T06:30:58.132 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.304e-06 2024-09-14T06:30:58.132 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:58.132 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:58.132 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:30:58.132 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600034+0000", 2024-09-14T06:30:58.132 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.1429999999999997e-06 2024-09-14T06:30:58.132 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:58.132 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:58.132 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:30:58.132 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600038+0000", 2024-09-14T06:30:58.133 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.692e-06 2024-09-14T06:30:58.133 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:58.133 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:58.133 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:30:58.133 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.642654+0000", 2024-09-14T06:30:58.133 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.042615564000000002 2024-09-14T06:30:58.133 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:58.133 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:58.133 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:30:58.133 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.642728+0000", 2024-09-14T06:30:58.133 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.4018999999999997e-05 2024-09-14T06:30:58.133 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:58.133 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:58.133 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:30:58.134 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.642823+0000", 2024-09-14T06:30:58.134 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.5520999999999996e-05 2024-09-14T06:30:58.134 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:58.134 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:58.134 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:58.134 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:58.134 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:58.134 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:55634 3.0 3:06490baf:::rbd_data.112dca2f6ae1.0000000000000030:17 [read 1122304~4096 [fadvise_nocache]] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:30:58.134 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:44.600042+0000", 2024-09-14T06:30:58.134 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 73.474057514999998, 2024-09-14T06:30:58.134 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.042903729000000002, 2024-09-14T06:30:58.134 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:30:58.134 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:30:58.134 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:30:58.135 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:30:58.135 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:30:58.135 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:58.135 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 55634 2024-09-14T06:30:58.135 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:58.135 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:30:58.135 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:58.135 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:30:58.135 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600042+0000", 2024-09-14T06:30:58.135 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:58.135 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:58.135 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:58.135 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:30:58.135 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600042+0000", 2024-09-14T06:30:58.135 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:58.136 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:58.136 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:58.136 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:30:58.136 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600043+0000", 2024-09-14T06:30:58.136 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.7000000000000005e-07 2024-09-14T06:30:58.136 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:58.136 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:58.136 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:30:58.136 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600044+0000", 2024-09-14T06:30:58.136 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.742e-06 2024-09-14T06:30:58.136 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:58.137 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:58.137 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:30:58.137 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600050+0000", 2024-09-14T06:30:58.137 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.0800000000000002e-06 2024-09-14T06:30:58.137 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:58.137 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:58.137 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:30:58.137 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600055+0000", 2024-09-14T06:30:58.137 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.3889999999999998e-06 2024-09-14T06:30:58.137 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:58.137 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:58.137 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:30:58.137 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.642845+0000", 2024-09-14T06:30:58.137 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.042790040000000001 2024-09-14T06:30:58.138 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:58.138 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:58.138 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:30:58.138 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.642888+0000", 2024-09-14T06:30:58.138 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.2790000000000002e-05 2024-09-14T06:30:58.138 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:58.138 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:58.138 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:30:58.138 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.642946+0000", 2024-09-14T06:30:58.138 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.8118000000000002e-05 2024-09-14T06:30:58.138 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:58.138 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:58.139 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:58.139 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:58.139 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:58.139 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:55635 3.0 3:06490baf:::rbd_data.112dca2f6ae1.0000000000000030:17 [read 1146880~4096 [fadvise_nocache]] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:30:58.139 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:44.600061+0000", 2024-09-14T06:30:58.139 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 73.474038944, 2024-09-14T06:30:58.139 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.043092695, 2024-09-14T06:30:58.139 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:30:58.139 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:30:58.139 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:30:58.139 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:30:58.139 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:30:58.139 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:58.139 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 55635 2024-09-14T06:30:58.140 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:58.140 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:30:58.140 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:58.140 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:30:58.140 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600061+0000", 2024-09-14T06:30:58.140 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:58.140 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:58.140 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:58.140 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:30:58.140 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600061+0000", 2024-09-14T06:30:58.140 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:58.140 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:58.140 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:58.141 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:30:58.141 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600061+0000", 2024-09-14T06:30:58.141 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.3099999999999999e-07 2024-09-14T06:30:58.141 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:58.141 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:58.141 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:30:58.141 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600063+0000", 2024-09-14T06:30:58.141 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.6220000000000001e-06 2024-09-14T06:30:58.141 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:58.141 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:58.141 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:30:58.141 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600068+0000", 2024-09-14T06:30:58.141 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.4790000000000004e-06 2024-09-14T06:30:58.141 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:58.142 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:58.142 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:30:58.142 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600072+0000", 2024-09-14T06:30:58.142 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.3100000000000002e-06 2024-09-14T06:30:58.142 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:58.142 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:58.142 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:30:58.142 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.642987+0000", 2024-09-14T06:30:58.142 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.042914489 2024-09-14T06:30:58.142 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:58.142 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:58.142 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:30:58.143 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.643062+0000", 2024-09-14T06:30:58.143 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.5011000000000004e-05 2024-09-14T06:30:58.143 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:58.143 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:58.143 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:30:58.143 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.643153+0000", 2024-09-14T06:30:58.143 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.1452999999999993e-05 2024-09-14T06:30:58.144 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:58.144 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:58.144 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:58.144 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:58.144 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:58.144 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:55636 3.0 3:06490baf:::rbd_data.112dca2f6ae1.0000000000000030:17 [read 1171456~4096 [fadvise_nocache]] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:30:58.144 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:44.600078+0000", 2024-09-14T06:30:58.144 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 73.474021531999995, 2024-09-14T06:30:58.145 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.043207966, 2024-09-14T06:30:58.145 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:30:58.145 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:30:58.145 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:30:58.145 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:30:58.145 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:30:58.145 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:58.145 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 55636 2024-09-14T06:30:58.145 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:58.145 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:30:58.145 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:58.146 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:30:58.146 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600078+0000", 2024-09-14T06:30:58.146 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:58.146 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:58.146 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:58.146 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:30:58.146 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600078+0000", 2024-09-14T06:30:58.147 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:58.147 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:58.147 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:58.147 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:30:58.147 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600078+0000", 2024-09-14T06:30:58.147 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.34e-07 2024-09-14T06:30:58.147 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:58.148 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:58.148 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:30:58.148 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600080+0000", 2024-09-14T06:30:58.148 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.641e-06 2024-09-14T06:30:58.148 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:58.148 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:58.149 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:30:58.149 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600086+0000", 2024-09-14T06:30:58.149 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.3860000000000004e-06 2024-09-14T06:30:58.149 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:58.149 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:58.149 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:30:58.150 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600091+0000", 2024-09-14T06:30:58.150 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.5970000000000002e-06 2024-09-14T06:30:58.150 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:58.150 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:58.150 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:30:58.150 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.643184+0000", 2024-09-14T06:30:58.151 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.043092893 2024-09-14T06:30:58.151 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:58.151 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:58.151 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:30:58.151 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.643228+0000", 2024-09-14T06:30:58.151 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.4536999999999997e-05 2024-09-14T06:30:58.152 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:58.152 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:58.152 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:30:58.152 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.643286+0000", 2024-09-14T06:30:58.152 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.7578000000000003e-05 2024-09-14T06:30:58.152 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:58.153 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:58.153 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:58.153 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:58.153 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:58.153 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:55637 3.0 3:06490baf:::rbd_data.112dca2f6ae1.0000000000000030:17 [read 1196032~4096 [fadvise_nocache]] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:30:58.153 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:44.600097+0000", 2024-09-14T06:30:58.154 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 73.474002967999994, 2024-09-14T06:30:58.154 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.043399124999999997, 2024-09-14T06:30:58.154 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:30:58.154 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:30:58.154 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:30:58.154 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:30:58.155 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:30:58.155 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:58.155 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 55637 2024-09-14T06:30:58.155 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:58.155 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:30:58.155 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:58.156 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:30:58.156 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600097+0000", 2024-09-14T06:30:58.156 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:58.156 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:58.156 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:58.156 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:30:58.157 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600097+0000", 2024-09-14T06:30:58.157 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:58.157 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:58.157 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:58.157 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:30:58.157 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.604372+0000", 2024-09-14T06:30:58.158 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0042751489999999998 2024-09-14T06:30:58.158 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:58.158 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:58.158 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:30:58.158 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.604377+0000", 2024-09-14T06:30:58.159 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.7899999999999999e-06 2024-09-14T06:30:58.159 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:58.159 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:58.159 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:30:58.159 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.604389+0000", 2024-09-14T06:30:58.159 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.2197e-05 2024-09-14T06:30:58.160 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:58.160 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:58.160 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:30:58.160 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.604397+0000", 2024-09-14T06:30:58.160 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.8730000000000007e-06 2024-09-14T06:30:58.160 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:58.161 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:58.161 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:30:58.161 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.643324+0000", 2024-09-14T06:30:58.161 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.038926897000000002 2024-09-14T06:30:58.161 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:58.161 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:58.162 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:30:58.162 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.643402+0000", 2024-09-14T06:30:58.162 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.8063000000000001e-05 2024-09-14T06:30:58.162 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:58.162 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:58.162 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:30:58.163 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.643496+0000", 2024-09-14T06:30:58.163 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.4155999999999998e-05 2024-09-14T06:30:58.163 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:58.163 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:58.163 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:58.163 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:58.164 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:58.164 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:55657 3.0 3:06490baf:::rbd_data.112dca2f6ae1.0000000000000030:17 [read 1687552~4096 [fadvise_nocache]] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:30:58.164 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:44.604841+0000", 2024-09-14T06:30:58.164 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 73.469258238999998, 2024-09-14T06:30:58.164 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.041890087999999999, 2024-09-14T06:30:58.164 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:30:58.165 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:30:58.165 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:30:58.165 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:30:58.165 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:30:58.165 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:58.165 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 55657 2024-09-14T06:30:58.166 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:58.166 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:30:58.166 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:58.166 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:30:58.166 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.604841+0000", 2024-09-14T06:30:58.166 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:58.167 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:58.167 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:58.167 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:30:58.167 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.604841+0000", 2024-09-14T06:30:58.167 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:58.167 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:58.168 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:58.168 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:30:58.168 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.604842+0000", 2024-09-14T06:30:58.168 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.0900000000000001e-07 2024-09-14T06:30:58.168 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:58.169 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:58.169 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:30:58.169 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.604846+0000", 2024-09-14T06:30:58.169 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.5759999999999999e-06 2024-09-14T06:30:58.169 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:58.169 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:58.170 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:30:58.170 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.604850+0000", 2024-09-14T06:30:58.170 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.405e-06 2024-09-14T06:30:58.170 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:58.170 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:58.170 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:30:58.171 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.604852+0000", 2024-09-14T06:30:58.171 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.8590000000000001e-06 2024-09-14T06:30:58.171 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:58.171 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:58.171 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:30:58.172 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.646633+0000", 2024-09-14T06:30:58.172 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.041780772000000001 2024-09-14T06:30:58.172 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:58.172 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:58.172 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:30:58.172 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.646677+0000", 2024-09-14T06:30:58.173 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.4115e-05 2024-09-14T06:30:58.173 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:58.173 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:58.173 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:30:58.173 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.646731+0000", 2024-09-14T06:30:58.173 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.4151999999999999e-05 2024-09-14T06:30:58.173 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:58.174 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:58.174 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:58.174 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:58.174 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:58.174 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:55658 3.0 3:06490baf:::rbd_data.112dca2f6ae1.0000000000000030:17 [read 1712128~4096 [fadvise_nocache]] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:30:58.174 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:44.604856+0000", 2024-09-14T06:30:58.175 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 73.469243638999998, 2024-09-14T06:30:58.175 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.041986227000000001, 2024-09-14T06:30:58.175 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:30:58.175 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:30:58.175 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:30:58.175 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:30:58.175 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:30:58.175 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:58.175 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 55658 2024-09-14T06:30:58.175 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:58.176 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:30:58.176 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:58.176 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:30:58.176 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.604856+0000", 2024-09-14T06:30:58.176 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:58.176 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:58.176 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:58.176 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:30:58.176 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.604856+0000", 2024-09-14T06:30:58.176 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:58.176 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:58.176 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:58.177 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:30:58.177 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.604856+0000", 2024-09-14T06:30:58.177 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.23e-07 2024-09-14T06:30:58.177 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:58.177 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:58.177 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:30:58.177 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.604857+0000", 2024-09-14T06:30:58.177 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.9999999999999995e-07 2024-09-14T06:30:58.177 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:58.177 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:58.177 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:30:58.178 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.604861+0000", 2024-09-14T06:30:58.178 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.9890000000000003e-06 2024-09-14T06:30:58.178 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:58.178 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:58.178 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:30:58.178 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.604864+0000", 2024-09-14T06:30:58.178 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.7429999999999998e-06 2024-09-14T06:30:58.178 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:58.178 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:58.178 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:30:58.178 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.646744+0000", 2024-09-14T06:30:58.178 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.041880325000000003 2024-09-14T06:30:58.178 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:58.178 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:58.178 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:30:58.179 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.646784+0000", 2024-09-14T06:30:58.179 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.9604000000000002e-05 2024-09-14T06:30:58.179 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:58.179 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:58.179 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:30:58.179 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.646842+0000", 2024-09-14T06:30:58.179 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.8343e-05 2024-09-14T06:30:58.179 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:58.179 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:58.179 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:58.179 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:58.179 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:58.179 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:55659 3.0 3:06490baf:::rbd_data.112dca2f6ae1.0000000000000030:17 [read 1736704~4096 [fadvise_nocache]] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:30:58.179 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:44.604867+0000", 2024-09-14T06:30:58.180 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 73.469232485999996, 2024-09-14T06:30:58.180 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.042097082000000001, 2024-09-14T06:30:58.180 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:30:58.180 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:30:58.180 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:30:58.180 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:30:58.180 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:30:58.180 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:58.180 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 55659 2024-09-14T06:30:58.180 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:58.180 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:30:58.180 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:58.181 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:30:58.181 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.604867+0000", 2024-09-14T06:30:58.181 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:58.181 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:58.181 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:58.181 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:30:58.181 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.604867+0000", 2024-09-14T06:30:58.181 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:58.181 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:58.181 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:58.181 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:30:58.181 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.604867+0000", 2024-09-14T06:30:58.181 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.2000000000000001e-07 2024-09-14T06:30:58.181 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:58.182 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:58.182 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:30:58.182 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.604868+0000", 2024-09-14T06:30:58.182 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.04e-06 2024-09-14T06:30:58.182 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:58.182 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:58.182 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:30:58.182 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.604871+0000", 2024-09-14T06:30:58.182 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.9689999999999999e-06 2024-09-14T06:30:58.182 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:58.182 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:58.182 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:30:58.182 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.604874+0000", 2024-09-14T06:30:58.182 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.5069999999999999e-06 2024-09-14T06:30:58.182 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:58.182 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:58.183 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:30:58.183 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.646874+0000", 2024-09-14T06:30:58.183 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.042000513000000003 2024-09-14T06:30:58.183 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:58.183 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:58.183 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:30:58.183 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.646914+0000", 2024-09-14T06:30:58.183 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.9443999999999997e-05 2024-09-14T06:30:58.183 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:58.183 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:58.183 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:30:58.183 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.646964+0000", 2024-09-14T06:30:58.183 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.0389000000000002e-05 2024-09-14T06:30:58.184 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:58.184 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:58.184 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:58.184 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:58.184 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:58.184 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:55660 3.0 3:06490baf:::rbd_data.112dca2f6ae1.0000000000000030:17 [read 1761280~4096 [fadvise_nocache]] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:30:58.184 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:44.604877+0000", 2024-09-14T06:30:58.184 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 73.469222736999996, 2024-09-14T06:30:58.184 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.042196349000000001, 2024-09-14T06:30:58.184 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:30:58.185 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:30:58.185 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:30:58.185 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:30:58.185 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:30:58.185 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:58.185 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 55660 2024-09-14T06:30:58.185 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:58.185 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:30:58.186 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:58.186 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:30:58.186 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.604877+0000", 2024-09-14T06:30:58.186 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:58.186 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:58.186 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:58.186 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:30:58.186 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.604877+0000", 2024-09-14T06:30:58.186 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:58.186 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:58.186 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:58.187 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:30:58.187 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.605810+0000", 2024-09-14T06:30:58.187 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00093336899999999997 2024-09-14T06:30:58.187 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:58.187 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:58.187 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:30:58.187 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.605815+0000", 2024-09-14T06:30:58.187 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.6140000000000002e-06 2024-09-14T06:30:58.187 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:58.187 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:58.187 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:30:58.187 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.605825+0000", 2024-09-14T06:30:58.187 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.8889999999999997e-06 2024-09-14T06:30:58.187 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:58.188 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:58.188 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:30:58.188 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.605831+0000", 2024-09-14T06:30:58.188 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.9800000000000003e-06 2024-09-14T06:30:58.188 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:58.188 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:58.188 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:30:58.188 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.646989+0000", 2024-09-14T06:30:58.188 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.041157878000000002 2024-09-14T06:30:58.188 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:58.188 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:58.188 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:30:58.189 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.647026+0000", 2024-09-14T06:30:58.189 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.7305000000000003e-05 2024-09-14T06:30:58.189 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:58.189 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:58.189 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:30:58.189 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.647073+0000", 2024-09-14T06:30:58.189 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.7314000000000001e-05 2024-09-14T06:30:58.189 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:58.189 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:58.189 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:58.189 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:58.189 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:58.189 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:55666 3.0 3:06490baf:::rbd_data.112dca2f6ae1.0000000000000030:17 [read 1908736~4096 [fadvise_nocache]] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:30:58.190 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:44.605898+0000", 2024-09-14T06:30:58.190 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 73.468201991000001, 2024-09-14T06:30:58.190 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.041862887000000001, 2024-09-14T06:30:58.190 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:30:58.190 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:30:58.190 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:30:58.190 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:30:58.190 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:30:58.190 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:58.190 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 55666 2024-09-14T06:30:58.190 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:58.190 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:30:58.190 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:58.190 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:30:58.190 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.605898+0000", 2024-09-14T06:30:58.191 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:58.191 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:58.191 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:58.191 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:30:58.191 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.605898+0000", 2024-09-14T06:30:58.191 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:58.191 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:58.191 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:58.191 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:30:58.191 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.611874+0000", 2024-09-14T06:30:58.191 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0059764049999999997 2024-09-14T06:30:58.191 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:58.191 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:58.191 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:30:58.192 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.611879+0000", 2024-09-14T06:30:58.192 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.8749999999999999e-06 2024-09-14T06:30:58.192 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:58.192 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:58.192 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:30:58.192 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.611887+0000", 2024-09-14T06:30:58.192 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.4109999999999996e-06 2024-09-14T06:30:58.192 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:58.192 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:58.192 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:30:58.192 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.611893+0000", 2024-09-14T06:30:58.192 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.2959999999999998e-06 2024-09-14T06:30:58.192 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:58.193 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:58.193 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:30:58.193 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.647674+0000", 2024-09-14T06:30:58.193 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.035781281999999998 2024-09-14T06:30:58.193 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:58.193 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:58.193 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:30:58.193 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.647711+0000", 2024-09-14T06:30:58.193 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.7434000000000001e-05 2024-09-14T06:30:58.194 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:58.194 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:58.194 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:30:58.194 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.647760+0000", 2024-09-14T06:30:58.194 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.9184000000000002e-05 2024-09-14T06:30:58.194 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:58.194 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:58.194 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:58.194 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:58.194 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:58.194 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-14T06:30:58.196 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-14T06:30:58.197 INFO:tasks.ceph.osd.5.smithi138.stderr:2024-09-14T06:30:58.195+0000 7fe13746f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33618) UID: 0 2024-09-14T06:30:58.296 INFO:tasks.ceph.osd.0.smithi094.stderr:2024-09-14T06:30:58.295+0000 7f07c9333640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33636) UID: 0 2024-09-14T06:30:58.300 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-14T06:30:58.300 INFO:teuthology.orchestra.run.smithi094.stdout: "ops": [], 2024-09-14T06:30:58.300 INFO:teuthology.orchestra.run.smithi094.stdout: "num_ops": 0 2024-09-14T06:30:58.300 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-14T06:30:58.308 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-14T06:30:58.398 INFO:tasks.ceph.osd.4.smithi138.stderr:2024-09-14T06:30:58.396+0000 7f6c653c8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33615) UID: 0 2024-09-14T06:30:58.451 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-14T06:30:58.451 INFO:teuthology.orchestra.run.smithi094.stdout: "ops": [], 2024-09-14T06:30:58.451 INFO:teuthology.orchestra.run.smithi094.stdout: "complaint_time": 30, 2024-09-14T06:30:58.451 INFO:teuthology.orchestra.run.smithi094.stdout: "num_blocked_ops": 0 2024-09-14T06:30:58.452 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-14T06:30:58.459 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-14T06:30:58.499 INFO:tasks.ceph.osd.4.smithi138.stderr:2024-09-14T06:30:58.497+0000 7f6c653c8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33615) UID: 0 2024-09-14T06:30:58.563 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-14T06:30:58.564 INFO:teuthology.orchestra.run.smithi094.stdout: "size": 20, 2024-09-14T06:30:58.564 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 600, 2024-09-14T06:30:58.564 INFO:teuthology.orchestra.run.smithi094.stdout: "ops": [ 2024-09-14T06:30:58.564 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:58.564 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4397.0:26415 3.0 e53/23 3:05fd9be0:::rbd_data.112d38768680.0000000000000006:head v 53'21, mlcod=50'19)", 2024-09-14T06:30:58.565 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:30.721610+0000", 2024-09-14T06:30:58.565 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 87.840258449000004, 2024-09-14T06:30:58.565 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0021763160000000002, 2024-09-14T06:30:58.565 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:30:58.565 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:30:58.565 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:30:58.566 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:30:58.566 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:58.566 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:30:58.566 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:30.721610+0000", 2024-09-14T06:30:58.566 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:58.567 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:58.567 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:58.567 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:30:58.567 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:30.721610+0000", 2024-09-14T06:30:58.567 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:58.567 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:58.567 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:58.568 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:30:58.568 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:30.721611+0000", 2024-09-14T06:30:58.568 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.41e-07 2024-09-14T06:30:58.568 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:58.568 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:58.568 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:30:58.569 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:30.721905+0000", 2024-09-14T06:30:58.569 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00029394300000000001 2024-09-14T06:30:58.569 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:58.569 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:58.569 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:30:58.569 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:30.721921+0000", 2024-09-14T06:30:58.569 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.6609999999999999e-05 2024-09-14T06:30:58.570 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:58.570 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:58.570 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:30:58.570 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:30.721931+0000", 2024-09-14T06:30:58.570 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.6739999999999992e-06 2024-09-14T06:30:58.570 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:58.570 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:58.571 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:30:58.571 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:30.722001+0000", 2024-09-14T06:30:58.571 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.0549000000000005e-05 2024-09-14T06:30:58.571 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:58.571 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:58.571 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:30:58.571 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:30.722029+0000", 2024-09-14T06:30:58.572 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.7327000000000001e-05 2024-09-14T06:30:58.572 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:58.572 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:58.572 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-14T06:30:58.572 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:30.723727+0000", 2024-09-14T06:30:58.572 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0016977310000000001 2024-09-14T06:30:58.573 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:58.573 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:58.573 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:30:58.573 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:30.723787+0000", 2024-09-14T06:30:58.573 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.0140999999999999e-05 2024-09-14T06:30:58.573 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:58.574 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:58.574 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:58.574 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:58.574 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:58.574 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4397.0:36084 3.0 e56/23 3:05a48ef4:::rbd_data.112dc58639fa.0000000000000058:head v 56'29, mlcod=56'27)", 2024-09-14T06:30:58.574 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:34.508855+0000", 2024-09-14T06:30:58.574 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 84.053013695000004, 2024-09-14T06:30:58.575 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0020801140000000001, 2024-09-14T06:30:58.575 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:30:58.575 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:30:58.575 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:30:58.575 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:30:58.575 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:58.576 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:30:58.576 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:34.508855+0000", 2024-09-14T06:30:58.576 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:58.576 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:58.576 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:58.576 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:30:58.577 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:34.508855+0000", 2024-09-14T06:30:58.577 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:58.577 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:58.577 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:58.577 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:30:58.577 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:34.508856+0000", 2024-09-14T06:30:58.578 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.1909999999999999e-06 2024-09-14T06:30:58.578 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:58.578 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:58.578 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:30:58.578 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:34.509203+0000", 2024-09-14T06:30:58.578 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00034625900000000002 2024-09-14T06:30:58.579 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:58.579 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:58.579 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:30:58.579 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:34.509218+0000", 2024-09-14T06:30:58.579 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.5091e-05 2024-09-14T06:30:58.579 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:58.579 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:58.580 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:30:58.580 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:34.509227+0000", 2024-09-14T06:30:58.580 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.0850000000000003e-06 2024-09-14T06:30:58.580 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:58.580 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:58.580 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:30:58.580 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:34.509311+0000", 2024-09-14T06:30:58.580 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.4436000000000003e-05 2024-09-14T06:30:58.581 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:58.581 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:58.581 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:30:58.581 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:34.509335+0000", 2024-09-14T06:30:58.581 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.3577000000000001e-05 2024-09-14T06:30:58.581 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:58.581 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:58.582 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-14T06:30:58.582 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:34.510890+0000", 2024-09-14T06:30:58.582 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0015554259999999999 2024-09-14T06:30:58.582 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:58.582 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:58.582 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:30:58.582 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:34.510935+0000", 2024-09-14T06:30:58.583 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.5049000000000002e-05 2024-09-14T06:30:58.583 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:58.583 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:58.583 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:58.583 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:58.583 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:58.583 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4397.0:45722 3.9 e64/23 3:91fed098:::rbd_data.112d5a737e72.0000000000000001:head v 64'61, mlcod=62'59)", 2024-09-14T06:30:58.584 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:43.864269+0000", 2024-09-14T06:30:58.584 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 74.697599620000005, 2024-09-14T06:30:58.584 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.003055224, 2024-09-14T06:30:58.584 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:30:58.584 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:30:58.585 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:30:58.585 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:30:58.585 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:58.585 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:30:58.585 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.864269+0000", 2024-09-14T06:30:58.585 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:58.586 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:58.586 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:58.586 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:30:58.586 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.864269+0000", 2024-09-14T06:30:58.586 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:58.587 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:58.587 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:58.587 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:30:58.587 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.864269+0000", 2024-09-14T06:30:58.587 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.41e-07 2024-09-14T06:30:58.587 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:58.587 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:58.588 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:30:58.588 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.865490+0000", 2024-09-14T06:30:58.588 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.001220552 2024-09-14T06:30:58.588 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:58.588 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:58.588 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:30:58.588 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.865506+0000", 2024-09-14T06:30:58.588 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.6263999999999999e-05 2024-09-14T06:30:58.589 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:58.589 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:58.589 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:30:58.589 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.865516+0000", 2024-09-14T06:30:58.589 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.0091999999999999e-05 2024-09-14T06:30:58.589 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:58.589 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:58.589 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:30:58.590 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.865583+0000", 2024-09-14T06:30:58.590 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.6676999999999999e-05 2024-09-14T06:30:58.590 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:58.590 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:58.590 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:30:58.590 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.865610+0000", 2024-09-14T06:30:58.590 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.6529999999999998e-05 2024-09-14T06:30:58.590 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:58.591 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:58.591 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-14T06:30:58.591 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.867281+0000", 2024-09-14T06:30:58.591 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.001671203 2024-09-14T06:30:58.591 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:58.591 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:58.591 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:30:58.592 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.867324+0000", 2024-09-14T06:30:58.592 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.3665000000000003e-05 2024-09-14T06:30:58.592 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:58.592 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:58.592 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:58.592 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:58.593 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:58.593 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4397.0:50006 3.4 e64/23 3:20cfe300:::rbd_data.112d5a737e72.0000000000000019:head v 64'70, mlcod=61'68)", 2024-09-14T06:30:58.593 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:44.224218+0000", 2024-09-14T06:30:58.593 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 74.337650422999999, 2024-09-14T06:30:58.593 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0021104320000000002, 2024-09-14T06:30:58.593 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:30:58.593 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:30:58.593 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:30:58.593 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:30:58.593 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:58.593 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:30:58.593 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.224218+0000", 2024-09-14T06:30:58.594 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:58.594 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:58.594 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:58.594 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:30:58.594 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.224218+0000", 2024-09-14T06:30:58.594 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:58.594 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:58.594 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:58.594 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:30:58.594 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.224219+0000", 2024-09-14T06:30:58.594 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.0049999999999999e-06 2024-09-14T06:30:58.594 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:58.594 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:58.594 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:30:58.595 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.224326+0000", 2024-09-14T06:30:58.595 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000106848 2024-09-14T06:30:58.595 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:58.595 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:58.595 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:30:58.595 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.224345+0000", 2024-09-14T06:30:58.595 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.8587999999999999e-05 2024-09-14T06:30:58.595 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:58.595 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:58.595 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:30:58.595 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.224354+0000", 2024-09-14T06:30:58.595 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.7220000000000005e-06 2024-09-14T06:30:58.595 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:58.595 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:58.596 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:30:58.596 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.224416+0000", 2024-09-14T06:30:58.596 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.2738999999999997e-05 2024-09-14T06:30:58.596 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:58.596 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:58.596 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:30:58.596 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.224441+0000", 2024-09-14T06:30:58.596 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.4408000000000001e-05 2024-09-14T06:30:58.596 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:58.596 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:58.597 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-14T06:30:58.597 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.226295+0000", 2024-09-14T06:30:58.597 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00185428 2024-09-14T06:30:58.597 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:58.597 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:58.597 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:30:58.597 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.226329+0000", 2024-09-14T06:30:58.597 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.3841999999999999e-05 2024-09-14T06:30:58.597 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:58.597 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:58.597 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:58.597 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:58.597 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:58.597 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4397.0:74241 3.0 e73/23 3:06e93fb9:::rbd_data.112d80ca42af.0000000000000000:head v 73'102, mlcod=71'100)", 2024-09-14T06:30:58.598 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:53.876730+0000", 2024-09-14T06:30:58.598 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 64.685138519999995, 2024-09-14T06:30:58.598 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0079470329999999992, 2024-09-14T06:30:58.598 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:30:58.598 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:30:58.598 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:30:58.598 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:30:58.598 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:58.598 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:30:58.598 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:53.876730+0000", 2024-09-14T06:30:58.598 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:58.598 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:58.598 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:58.598 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:30:58.598 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:53.876730+0000", 2024-09-14T06:30:58.599 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:58.599 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:58.599 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:58.599 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:30:58.599 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:53.876731+0000", 2024-09-14T06:30:58.599 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.0500000000000004e-07 2024-09-14T06:30:58.599 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:58.599 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:58.599 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:30:58.599 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:53.879616+0000", 2024-09-14T06:30:58.599 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0028856680000000001 2024-09-14T06:30:58.599 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:58.599 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:58.599 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:30:58.600 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:53.879643+0000", 2024-09-14T06:30:58.600 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.6120999999999999e-05 2024-09-14T06:30:58.600 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:58.600 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:58.600 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:30:58.600 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:53.879654+0000", 2024-09-14T06:30:58.600 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.1586999999999999e-05 2024-09-14T06:30:58.600 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:58.600 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:58.600 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:30:58.600 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:53.879756+0000", 2024-09-14T06:30:58.600 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00010223899999999999 2024-09-14T06:30:58.601 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:58.601 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:58.601 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:30:58.601 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:53.879783+0000", 2024-09-14T06:30:58.601 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.6913000000000001e-05 2024-09-14T06:30:58.601 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:58.601 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:58.601 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-14T06:30:58.601 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:53.884594+0000", 2024-09-14T06:30:58.601 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0048110729999999999 2024-09-14T06:30:58.601 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:58.602 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:58.602 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:30:58.602 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:53.884677+0000", 2024-09-14T06:30:58.602 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.2927000000000001e-05 2024-09-14T06:30:58.602 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:58.602 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:58.602 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:58.602 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:58.602 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:58.602 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4397.0:74243 3.9 e73/23 3:94977f81:::rbd_data.112d80ca42af.0000000000000002:head v 73'94, mlcod=71'92)", 2024-09-14T06:30:58.602 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:53.885156+0000", 2024-09-14T06:30:58.602 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 64.676712601999995, 2024-09-14T06:30:58.603 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0096009860000000006, 2024-09-14T06:30:58.603 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:30:58.603 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:30:58.603 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:30:58.603 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:30:58.603 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:58.603 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:30:58.603 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:53.885156+0000", 2024-09-14T06:30:58.604 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:58.604 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:58.604 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:58.604 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:30:58.604 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:53.885156+0000", 2024-09-14T06:30:58.604 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:58.604 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:58.604 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:58.605 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:30:58.605 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:53.885156+0000", 2024-09-14T06:30:58.605 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.23e-07 2024-09-14T06:30:58.605 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:58.605 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:58.605 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:30:58.605 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:53.889694+0000", 2024-09-14T06:30:58.605 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0045378709999999997 2024-09-14T06:30:58.605 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:58.605 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:58.605 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:30:58.605 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:53.889720+0000", 2024-09-14T06:30:58.606 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.5259e-05 2024-09-14T06:30:58.606 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:58.606 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:58.606 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:30:58.606 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:53.889731+0000", 2024-09-14T06:30:58.606 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.1739000000000001e-05 2024-09-14T06:30:58.606 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:58.606 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:58.606 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:30:58.606 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:53.889822+0000", 2024-09-14T06:30:58.606 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.1200999999999996e-05 2024-09-14T06:30:58.606 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:58.606 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:58.606 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:30:58.606 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:53.889850+0000", 2024-09-14T06:30:58.606 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.7084999999999999e-05 2024-09-14T06:30:58.607 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:58.607 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:58.607 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-14T06:30:58.607 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:53.894703+0000", 2024-09-14T06:30:58.607 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0048532690000000003 2024-09-14T06:30:58.607 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:58.607 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:58.607 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:30:58.607 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:53.894757+0000", 2024-09-14T06:30:58.607 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.4339000000000003e-05 2024-09-14T06:30:58.607 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:58.607 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:58.607 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:58.607 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:58.607 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:58.608 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4397.0:74249 3.0 e73/23 3:009b0225:::rbd_data.112d80ca42af.0000000000000009:head v 73'103, mlcod=71'101)", 2024-09-14T06:30:58.608 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:53.893219+0000", 2024-09-14T06:30:58.608 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 64.668650123999996, 2024-09-14T06:30:58.608 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0065381980000000003, 2024-09-14T06:30:58.608 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:30:58.608 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:30:58.608 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:30:58.608 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:30:58.608 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:58.608 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:30:58.608 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:53.893219+0000", 2024-09-14T06:30:58.608 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:58.608 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:58.608 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:58.609 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:30:58.609 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:53.893219+0000", 2024-09-14T06:30:58.609 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:58.609 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:58.609 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:58.609 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:30:58.609 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:53.893221+0000", 2024-09-14T06:30:58.609 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.4329999999999998e-06 2024-09-14T06:30:58.609 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:58.609 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:58.609 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:30:58.609 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:53.895968+0000", 2024-09-14T06:30:58.609 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.002746746 2024-09-14T06:30:58.610 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:58.610 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:58.610 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:30:58.610 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:53.895988+0000", 2024-09-14T06:30:58.610 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.0227000000000001e-05 2024-09-14T06:30:58.610 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:58.610 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:58.610 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:30:58.610 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:53.896000+0000", 2024-09-14T06:30:58.610 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.2359e-05 2024-09-14T06:30:58.610 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:58.610 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:58.611 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:30:58.611 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:53.896084+0000", 2024-09-14T06:30:58.611 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.3659999999999995e-05 2024-09-14T06:30:58.611 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:58.611 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:58.611 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:30:58.611 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:53.896109+0000", 2024-09-14T06:30:58.611 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.5259999999999999e-05 2024-09-14T06:30:58.611 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:58.611 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:58.611 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-14T06:30:58.611 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:53.899704+0000", 2024-09-14T06:30:58.611 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0035942209999999999 2024-09-14T06:30:58.611 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:58.612 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:58.612 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:30:58.612 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:53.899757+0000", 2024-09-14T06:30:58.612 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.3291999999999997e-05 2024-09-14T06:30:58.612 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:58.612 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:58.612 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:58.612 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:58.612 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:58.612 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4397.0:78263 3.0 e77/23 3:00899b22:::rbd_data.112ddfbec471.0000000000000016:head v 77'104, mlcod=73'102)", 2024-09-14T06:30:58.612 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:57.634578+0000", 2024-09-14T06:30:58.612 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 60.927291072000003, 2024-09-14T06:30:58.612 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.002372454, 2024-09-14T06:30:58.612 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:30:58.613 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:30:58.613 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:30:58.613 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:30:58.613 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:58.613 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:30:58.613 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:57.634578+0000", 2024-09-14T06:30:58.613 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:58.613 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:58.613 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:58.613 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:30:58.613 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:57.634578+0000", 2024-09-14T06:30:58.613 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:58.613 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:58.613 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:58.614 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:30:58.614 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:57.634578+0000", 2024-09-14T06:30:58.614 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.5699999999999998e-07 2024-09-14T06:30:58.614 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:58.614 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:58.614 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:30:58.614 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:57.635347+0000", 2024-09-14T06:30:58.614 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00076848300000000003 2024-09-14T06:30:58.614 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:58.615 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:58.615 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:30:58.615 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:57.635361+0000", 2024-09-14T06:30:58.615 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.4238e-05 2024-09-14T06:30:58.615 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:58.615 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:58.615 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:30:58.615 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:57.635371+0000", 2024-09-14T06:30:58.615 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.0087e-05 2024-09-14T06:30:58.615 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:58.615 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:58.615 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:30:58.615 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:57.635439+0000", 2024-09-14T06:30:58.615 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.7953000000000005e-05 2024-09-14T06:30:58.615 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:58.616 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:58.616 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:30:58.616 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:57.635464+0000", 2024-09-14T06:30:58.616 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.5062000000000002e-05 2024-09-14T06:30:58.616 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:58.616 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:58.616 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-14T06:30:58.616 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:57.636914+0000", 2024-09-14T06:30:58.616 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.001449625 2024-09-14T06:30:58.616 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:58.616 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:58.617 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:30:58.617 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:57.636950+0000", 2024-09-14T06:30:58.617 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.6548999999999999e-05 2024-09-14T06:30:58.617 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:58.617 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:58.617 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:58.617 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:58.617 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:58.617 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4397.0:78604 3.9 e77/23 3:94c62a0e:::rbd_data.112ddfbec471.000000000000002c:head v 77'96, mlcod=73'94)", 2024-09-14T06:30:58.617 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:57.677568+0000", 2024-09-14T06:30:58.617 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 60.884300684000003, 2024-09-14T06:30:58.617 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.002564846, 2024-09-14T06:30:58.617 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:30:58.617 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:30:58.618 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:30:58.618 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:30:58.618 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:58.618 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:30:58.618 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:57.677568+0000", 2024-09-14T06:30:58.618 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:58.619 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:58.619 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:58.619 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:30:58.619 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:57.677568+0000", 2024-09-14T06:30:58.619 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:58.619 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:58.619 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:58.620 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:30:58.620 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:57.677568+0000", 2024-09-14T06:30:58.620 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.8900000000000001e-07 2024-09-14T06:30:58.620 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:58.620 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:58.620 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:30:58.620 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:57.678488+0000", 2024-09-14T06:30:58.620 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00091963899999999998 2024-09-14T06:30:58.620 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:58.620 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:58.620 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:30:58.620 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:57.678500+0000", 2024-09-14T06:30:58.620 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.1902000000000001e-05 2024-09-14T06:30:58.620 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:58.621 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:58.621 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:30:58.621 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:57.678507+0000", 2024-09-14T06:30:58.621 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.7739999999999999e-06 2024-09-14T06:30:58.621 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:58.621 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:58.621 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:30:58.621 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:57.678586+0000", 2024-09-14T06:30:58.621 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.9103000000000005e-05 2024-09-14T06:30:58.621 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:58.621 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:58.621 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:30:58.622 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:57.678614+0000", 2024-09-14T06:30:58.622 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.7912e-05 2024-09-14T06:30:58.622 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:58.622 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:58.622 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-14T06:30:58.622 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:57.680075+0000", 2024-09-14T06:30:58.622 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0014613370000000001 2024-09-14T06:30:58.622 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:58.622 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:58.622 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:30:58.622 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:57.680133+0000", 2024-09-14T06:30:58.622 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.7890000000000003e-05 2024-09-14T06:30:58.623 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:58.623 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:58.623 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:58.623 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:58.623 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:58.623 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4397.0:78641 3.9 e77/23 3:914b72e4:::rbd_data.112ddfbec471.0000000000000030:head v 77'98, mlcod=73'95)", 2024-09-14T06:30:58.623 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:57.682289+0000", 2024-09-14T06:30:58.623 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 60.879580003000001, 2024-09-14T06:30:58.623 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0020171400000000002, 2024-09-14T06:30:58.623 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:30:58.623 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:30:58.623 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:30:58.623 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:30:58.623 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:58.624 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:30:58.624 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:57.682289+0000", 2024-09-14T06:30:58.624 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:58.624 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:58.624 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:58.624 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:30:58.624 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:57.682289+0000", 2024-09-14T06:30:58.624 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:58.624 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:58.624 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:58.624 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:30:58.624 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:57.682290+0000", 2024-09-14T06:30:58.625 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.0100000000000003e-07 2024-09-14T06:30:58.625 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:58.625 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:58.625 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:30:58.625 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:57.683230+0000", 2024-09-14T06:30:58.625 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00094047599999999999 2024-09-14T06:30:58.626 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:58.626 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:58.626 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:30:58.626 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:57.683243+0000", 2024-09-14T06:30:58.626 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.2926e-05 2024-09-14T06:30:58.626 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:58.626 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:58.626 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:30:58.626 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:57.683250+0000", 2024-09-14T06:30:58.626 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.6109999999999999e-06 2024-09-14T06:30:58.626 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:58.626 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:58.627 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:30:58.627 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:57.683313+0000", 2024-09-14T06:30:58.627 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.2943999999999999e-05 2024-09-14T06:30:58.627 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:58.627 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:58.627 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:30:58.627 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:57.683337+0000", 2024-09-14T06:30:58.627 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.4377999999999999e-05 2024-09-14T06:30:58.627 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:58.627 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:58.627 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-14T06:30:58.627 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:57.684264+0000", 2024-09-14T06:30:58.627 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00092733099999999999 2024-09-14T06:30:58.627 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:58.628 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:58.628 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:30:58.628 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:57.684306+0000", 2024-09-14T06:30:58.628 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.1573000000000003e-05 2024-09-14T06:30:58.628 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:58.628 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:58.628 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:58.628 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:58.628 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:58.628 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4397.0:126010 3.9 e104/23 3:9120f7ff:::rbd_data.112d97471187.000000000000001b:head v 104'145, mlcod=99'143)", 2024-09-14T06:30:58.628 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:30:25.990254+0000", 2024-09-14T06:30:58.628 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 32.571615195, 2024-09-14T06:30:58.628 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0020629699999999999, 2024-09-14T06:30:58.628 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:30:58.629 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:30:58.629 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:30:58.629 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:30:58.629 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:58.629 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:30:58.629 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:25.990254+0000", 2024-09-14T06:30:58.629 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:58.629 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:58.629 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:58.629 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:30:58.629 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:25.990254+0000", 2024-09-14T06:30:58.629 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:58.629 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:58.630 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:58.630 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:30:58.630 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:25.990255+0000", 2024-09-14T06:30:58.630 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.3349999999999999e-06 2024-09-14T06:30:58.630 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:58.630 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:58.630 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:30:58.630 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:25.990966+0000", 2024-09-14T06:30:58.630 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00071085199999999999 2024-09-14T06:30:58.630 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:58.630 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:58.630 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:30:58.630 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:25.990986+0000", 2024-09-14T06:30:58.630 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.0054000000000001e-05 2024-09-14T06:30:58.630 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:58.631 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:58.631 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:30:58.631 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:25.990995+0000", 2024-09-14T06:30:58.631 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.166e-06 2024-09-14T06:30:58.631 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:58.631 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:58.631 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:30:58.631 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:25.991055+0000", 2024-09-14T06:30:58.631 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.0093000000000002e-05 2024-09-14T06:30:58.631 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:58.631 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:58.631 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:30:58.631 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:25.991082+0000", 2024-09-14T06:30:58.631 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.6505000000000001e-05 2024-09-14T06:30:58.631 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:58.632 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:58.632 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-14T06:30:58.632 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:25.992270+0000", 2024-09-14T06:30:58.632 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0011880479999999999 2024-09-14T06:30:58.632 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:58.632 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:58.632 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:30:58.632 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:25.992317+0000", 2024-09-14T06:30:58.632 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.6916999999999998e-05 2024-09-14T06:30:58.632 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:58.632 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:58.632 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:58.632 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:58.632 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:58.633 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4397.0:149885 3.9 e122/23 3:951acef3:::rbd_data.112d1bb7b269.0000000000000005:head v 122'176, mlcod=122'172)", 2024-09-14T06:30:58.633 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:30:44.739092+0000", 2024-09-14T06:30:58.633 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 13.822776374, 2024-09-14T06:30:58.633 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0028236659999999998, 2024-09-14T06:30:58.633 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:30:58.633 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:30:58.633 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:30:58.633 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:30:58.633 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:58.633 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:30:58.633 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.739092+0000", 2024-09-14T06:30:58.633 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:58.633 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:58.633 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:58.634 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:30:58.634 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.739092+0000", 2024-09-14T06:30:58.634 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:58.634 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:58.634 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:58.634 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:30:58.634 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.739093+0000", 2024-09-14T06:30:58.634 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.0109999999999999e-06 2024-09-14T06:30:58.634 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:58.634 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:58.634 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:30:58.634 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.740189+0000", 2024-09-14T06:30:58.634 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00109527 2024-09-14T06:30:58.634 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:58.634 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:58.635 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:30:58.635 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.740199+0000", 2024-09-14T06:30:58.635 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.0458999999999999e-05 2024-09-14T06:30:58.635 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:58.635 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:58.635 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:30:58.635 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.740207+0000", 2024-09-14T06:30:58.635 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.4499999999999998e-06 2024-09-14T06:30:58.635 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:58.635 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:58.635 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:30:58.635 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.740295+0000", 2024-09-14T06:30:58.635 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.8740999999999996e-05 2024-09-14T06:30:58.635 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:58.636 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:58.636 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:30:58.636 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.740317+0000", 2024-09-14T06:30:58.636 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.1523e-05 2024-09-14T06:30:58.636 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:58.636 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:58.636 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-14T06:30:58.636 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.741865+0000", 2024-09-14T06:30:58.637 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.001548315 2024-09-14T06:30:58.637 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:58.637 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:58.637 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:30:58.637 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.741916+0000", 2024-09-14T06:30:58.637 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.0896999999999997e-05 2024-09-14T06:30:58.638 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:58.638 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:58.638 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:58.638 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:58.638 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:58.638 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4397.0:149886 3.4 e122/23 3:2033c3d0:::rbd_data.112d1bb7b269.0000000000000006:head v 122'220, mlcod=120'217)", 2024-09-14T06:30:58.638 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:30:44.741736+0000", 2024-09-14T06:30:58.639 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 13.820132957, 2024-09-14T06:30:58.639 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0020268999999999999, 2024-09-14T06:30:58.639 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:30:58.639 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:30:58.639 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:30:58.639 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:30:58.639 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:58.639 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:30:58.640 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.741736+0000", 2024-09-14T06:30:58.640 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:58.640 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:58.640 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:58.640 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:30:58.640 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.741736+0000", 2024-09-14T06:30:58.640 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:58.641 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:58.641 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:58.641 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:30:58.641 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.741737+0000", 2024-09-14T06:30:58.641 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.9000000000000005e-07 2024-09-14T06:30:58.641 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:58.641 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:58.642 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:30:58.642 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.741918+0000", 2024-09-14T06:30:58.642 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00018120300000000001 2024-09-14T06:30:58.642 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:58.642 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:58.642 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:30:58.643 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.741930+0000", 2024-09-14T06:30:58.643 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.1875999999999999e-05 2024-09-14T06:30:58.643 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:58.643 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:58.643 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:30:58.643 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.741938+0000", 2024-09-14T06:30:58.644 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.8040000000000002e-06 2024-09-14T06:30:58.644 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:58.644 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:58.644 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:30:58.644 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.742185+0000", 2024-09-14T06:30:58.644 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00024747199999999999 2024-09-14T06:30:58.644 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:58.644 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:58.645 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:30:58.645 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.742207+0000", 2024-09-14T06:30:58.645 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.1416999999999998e-05 2024-09-14T06:30:58.645 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:58.645 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:58.645 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-14T06:30:58.645 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.743715+0000", 2024-09-14T06:30:58.645 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00150884 2024-09-14T06:30:58.646 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:58.646 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:58.646 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:30:58.646 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.743763+0000", 2024-09-14T06:30:58.646 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.7298e-05 2024-09-14T06:30:58.646 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:58.646 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:58.646 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:58.647 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:58.647 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:58.647 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4397.0:149933 3.4 e122/23 3:234f5229:::rbd_data.112d1bb7b269.0000000000000009:head v 122'221, mlcod=120'218)", 2024-09-14T06:30:58.647 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:30:44.743849+0000", 2024-09-14T06:30:58.647 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 13.818019957000001, 2024-09-14T06:30:58.647 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0031479519999999999, 2024-09-14T06:30:58.647 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:30:58.647 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:30:58.648 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:30:58.648 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:30:58.648 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:58.648 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:30:58.648 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.743849+0000", 2024-09-14T06:30:58.648 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:58.648 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:58.649 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:58.649 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:30:58.649 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.743849+0000", 2024-09-14T06:30:58.649 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:58.649 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:58.650 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:58.650 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:30:58.650 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.743850+0000", 2024-09-14T06:30:58.650 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.279e-06 2024-09-14T06:30:58.650 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:58.650 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:58.650 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:30:58.651 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.745151+0000", 2024-09-14T06:30:58.651 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.001300495 2024-09-14T06:30:58.651 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:58.651 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:58.651 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:30:58.652 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.745167+0000", 2024-09-14T06:30:58.652 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.6478e-05 2024-09-14T06:30:58.652 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:58.652 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:58.652 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:30:58.652 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.745175+0000", 2024-09-14T06:30:58.652 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.2679999999999994e-06 2024-09-14T06:30:58.653 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:58.653 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:58.653 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:30:58.653 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.745226+0000", 2024-09-14T06:30:58.653 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.0192999999999999e-05 2024-09-14T06:30:58.653 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:58.653 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:58.654 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:30:58.654 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.745247+0000", 2024-09-14T06:30:58.654 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.1767e-05 2024-09-14T06:30:58.654 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:58.654 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:58.654 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-14T06:30:58.654 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.746952+0000", 2024-09-14T06:30:58.655 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0017042769999999999 2024-09-14T06:30:58.655 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:58.655 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:58.655 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:30:58.655 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.746997+0000", 2024-09-14T06:30:58.655 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.5194999999999997e-05 2024-09-14T06:30:58.655 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:58.656 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:58.656 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:58.656 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:58.656 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:58.656 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4397.0:150494 3.9 e122/23 3:955152a4:::rbd_object_map.112d1bb7b269.0000000000000038:head v 122'194, mlcod=122'190)", 2024-09-14T06:30:58.656 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:30:44.781585+0000", 2024-09-14T06:30:58.656 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 13.780284065, 2024-09-14T06:30:58.657 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0026957499999999998, 2024-09-14T06:30:58.657 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:30:58.657 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:30:58.657 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:30:58.657 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:30:58.657 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:58.657 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:30:58.657 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.781585+0000", 2024-09-14T06:30:58.657 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:58.657 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:58.657 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:58.657 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:30:58.657 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.781585+0000", 2024-09-14T06:30:58.657 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:58.658 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:58.658 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:58.658 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:30:58.658 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.781585+0000", 2024-09-14T06:30:58.658 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.5000000000000002e-07 2024-09-14T06:30:58.658 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:58.658 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:58.658 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:30:58.658 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.781594+0000", 2024-09-14T06:30:58.658 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.4640000000000006e-06 2024-09-14T06:30:58.659 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:58.659 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:58.659 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:30:58.659 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.781604+0000", 2024-09-14T06:30:58.659 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.0071999999999999e-05 2024-09-14T06:30:58.659 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:58.659 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:58.659 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:30:58.660 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.781610+0000", 2024-09-14T06:30:58.660 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.3609999999999997e-06 2024-09-14T06:30:58.660 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:58.660 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:58.660 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:30:58.660 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.781676+0000", 2024-09-14T06:30:58.660 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.5963000000000005e-05 2024-09-14T06:30:58.660 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:58.660 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:58.661 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:30:58.661 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.781696+0000", 2024-09-14T06:30:58.661 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.9278000000000001e-05 2024-09-14T06:30:58.661 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:58.661 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:58.661 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-14T06:30:58.661 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.784231+0000", 2024-09-14T06:30:58.661 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.002535093 2024-09-14T06:30:58.662 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:58.662 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:58.662 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:30:58.662 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.784280+0000", 2024-09-14T06:30:58.662 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.9869e-05 2024-09-14T06:30:58.662 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:58.662 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:58.662 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:58.663 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:58.663 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:58.663 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4397.0:150519 3.9 e122/23 3:955152a4:::rbd_object_map.112d1bb7b269.0000000000000038:head v 122'195, mlcod=122'192)", 2024-09-14T06:30:58.663 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:30:44.782445+0000", 2024-09-14T06:30:58.663 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 13.779423673, 2024-09-14T06:30:58.663 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.002158227, 2024-09-14T06:30:58.663 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:30:58.663 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:30:58.663 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:30:58.664 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:30:58.664 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:58.664 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:30:58.664 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.782445+0000", 2024-09-14T06:30:58.664 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:58.664 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:58.664 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:58.664 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:30:58.665 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.782445+0000", 2024-09-14T06:30:58.665 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:58.665 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:58.665 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:58.665 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:30:58.665 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.782446+0000", 2024-09-14T06:30:58.665 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.6499999999999998e-07 2024-09-14T06:30:58.665 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:58.666 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:58.666 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:30:58.666 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.782455+0000", 2024-09-14T06:30:58.666 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.1940000000000004e-06 2024-09-14T06:30:58.666 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:58.666 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:58.666 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:30:58.666 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.782464+0000", 2024-09-14T06:30:58.666 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.7220000000000005e-06 2024-09-14T06:30:58.667 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:58.667 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:58.667 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:30:58.667 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.782470+0000", 2024-09-14T06:30:58.667 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.9580000000000004e-06 2024-09-14T06:30:58.667 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:58.667 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:58.667 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:30:58.668 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.782514+0000", 2024-09-14T06:30:58.668 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.3931999999999999e-05 2024-09-14T06:30:58.668 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:58.668 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:58.668 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:30:58.668 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.782534+0000", 2024-09-14T06:30:58.668 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.0237e-05 2024-09-14T06:30:58.668 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:58.669 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:58.669 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-14T06:30:58.669 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.784560+0000", 2024-09-14T06:30:58.669 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.002025813 2024-09-14T06:30:58.669 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:58.669 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:58.669 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:30:58.670 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.784603+0000", 2024-09-14T06:30:58.670 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.3606000000000002e-05 2024-09-14T06:30:58.670 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:58.670 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:58.670 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:58.670 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:58.670 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:58.670 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4397.0:150898 3.9 e122/23 3:97c98f90:::rbd_data.112d1bb7b269.0000000000000025:head v 122'206, mlcod=122'203)", 2024-09-14T06:30:58.670 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:30:44.807232+0000", 2024-09-14T06:30:58.670 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 13.754637148, 2024-09-14T06:30:58.671 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0022297889999999998, 2024-09-14T06:30:58.671 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:30:58.671 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:30:58.671 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:30:58.671 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:30:58.671 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:58.671 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:30:58.671 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.807232+0000", 2024-09-14T06:30:58.671 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:58.671 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:58.671 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:58.671 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:30:58.671 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.807232+0000", 2024-09-14T06:30:58.671 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:58.672 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:58.672 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:58.672 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:30:58.672 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.807233+0000", 2024-09-14T06:30:58.672 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.5710000000000001e-06 2024-09-14T06:30:58.672 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:58.672 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:58.672 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:30:58.672 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.807944+0000", 2024-09-14T06:30:58.672 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00071056299999999999 2024-09-14T06:30:58.673 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:58.673 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:58.673 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:30:58.673 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.807961+0000", 2024-09-14T06:30:58.673 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.7689e-05 2024-09-14T06:30:58.673 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:58.673 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:58.674 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:30:58.674 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.807971+0000", 2024-09-14T06:30:58.674 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.2e-06 2024-09-14T06:30:58.674 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:58.674 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:58.674 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:30:58.674 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.808050+0000", 2024-09-14T06:30:58.675 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.9308999999999996e-05 2024-09-14T06:30:58.675 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:58.675 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:58.675 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:30:58.675 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.808074+0000", 2024-09-14T06:30:58.675 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.4147000000000001e-05 2024-09-14T06:30:58.675 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:58.675 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:58.675 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-14T06:30:58.675 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.809415+0000", 2024-09-14T06:30:58.675 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.001341257 2024-09-14T06:30:58.676 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:58.676 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:58.676 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:30:58.676 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.809461+0000", 2024-09-14T06:30:58.676 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.6053000000000001e-05 2024-09-14T06:30:58.676 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:58.676 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:58.676 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:58.676 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:58.676 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:58.676 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4397.0:152325 3.9 e122/23 3:955152a4:::rbd_object_map.112d1bb7b269.0000000000000038:head v 122'268, mlcod=122'265)", 2024-09-14T06:30:58.676 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:30:44.914377+0000", 2024-09-14T06:30:58.676 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 13.647491842999999, 2024-09-14T06:30:58.677 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0020208840000000001, 2024-09-14T06:30:58.677 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:30:58.677 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:30:58.677 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:30:58.677 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:30:58.677 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:58.677 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:30:58.677 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.914377+0000", 2024-09-14T06:30:58.677 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:58.677 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:58.677 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:58.677 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:30:58.678 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.914377+0000", 2024-09-14T06:30:58.678 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:58.678 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:58.678 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:58.678 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:30:58.678 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.914378+0000", 2024-09-14T06:30:58.678 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.3600000000000003e-07 2024-09-14T06:30:58.678 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:58.678 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:58.678 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:30:58.678 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.914383+0000", 2024-09-14T06:30:58.678 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.7230000000000001e-06 2024-09-14T06:30:58.678 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:58.678 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:58.678 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:30:58.678 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.914393+0000", 2024-09-14T06:30:58.679 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.8179999999999999e-06 2024-09-14T06:30:58.679 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:58.679 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:58.679 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:30:58.679 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.914400+0000", 2024-09-14T06:30:58.679 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.968e-06 2024-09-14T06:30:58.679 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:58.679 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:58.679 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:30:58.680 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.914447+0000", 2024-09-14T06:30:58.680 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.7122e-05 2024-09-14T06:30:58.680 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:58.680 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:58.680 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:30:58.680 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.914466+0000", 2024-09-14T06:30:58.681 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.8958999999999999e-05 2024-09-14T06:30:58.681 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:58.681 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:58.681 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-14T06:30:58.681 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.916356+0000", 2024-09-14T06:30:58.681 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.001890121 2024-09-14T06:30:58.681 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:58.681 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:58.681 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:30:58.681 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.916398+0000", 2024-09-14T06:30:58.681 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.1437000000000003e-05 2024-09-14T06:30:58.682 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:58.682 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:58.682 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:58.682 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:58.682 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:58.682 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4397.0:152449 3.9 e122/23 3:955152a4:::rbd_object_map.112d1bb7b269.0000000000000038:head v 122'274, mlcod=122'268)", 2024-09-14T06:30:58.682 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:30:44.924063+0000", 2024-09-14T06:30:58.682 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 13.637805331999999, 2024-09-14T06:30:58.682 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.002131764, 2024-09-14T06:30:58.682 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:30:58.682 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:30:58.683 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:30:58.683 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:30:58.683 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:58.683 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:30:58.683 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.924063+0000", 2024-09-14T06:30:58.683 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:58.683 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:58.683 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:58.683 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:30:58.683 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.924063+0000", 2024-09-14T06:30:58.683 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:58.683 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:58.683 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:58.683 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:30:58.683 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.924064+0000", 2024-09-14T06:30:58.684 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.72e-07 2024-09-14T06:30:58.684 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:58.684 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:58.684 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:30:58.684 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.924071+0000", 2024-09-14T06:30:58.684 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.0999999999999998e-06 2024-09-14T06:30:58.684 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:58.684 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:58.684 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:30:58.684 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.924077+0000", 2024-09-14T06:30:58.684 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.268e-06 2024-09-14T06:30:58.684 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:58.684 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:58.684 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:30:58.685 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.924082+0000", 2024-09-14T06:30:58.685 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.7110000000000003e-06 2024-09-14T06:30:58.685 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:58.685 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:58.685 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:30:58.685 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.924397+0000", 2024-09-14T06:30:58.685 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00031503500000000002 2024-09-14T06:30:58.685 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:58.685 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:58.685 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:30:58.685 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.924418+0000", 2024-09-14T06:30:58.685 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.1044999999999999e-05 2024-09-14T06:30:58.685 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:58.686 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:58.686 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-14T06:30:58.686 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.926158+0000", 2024-09-14T06:30:58.686 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0017404860000000001 2024-09-14T06:30:58.686 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:58.686 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:58.686 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:30:58.686 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.926195+0000", 2024-09-14T06:30:58.686 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.6847000000000001e-05 2024-09-14T06:30:58.686 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:58.686 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:58.686 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:58.686 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:58.686 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:58.687 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4397.0:152452 3.9 e122/23 3:955152a4:::rbd_object_map.112d1bb7b269.0000000000000038:head v 122'275, mlcod=122'268)", 2024-09-14T06:30:58.687 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:30:44.924090+0000", 2024-09-14T06:30:58.687 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 13.637779011999999, 2024-09-14T06:30:58.687 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0021362400000000002, 2024-09-14T06:30:58.687 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:30:58.687 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:30:58.687 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:30:58.687 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:30:58.687 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:58.687 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:30:58.687 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.924090+0000", 2024-09-14T06:30:58.687 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:58.687 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:58.687 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:58.687 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:30:58.687 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.924090+0000", 2024-09-14T06:30:58.688 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:58.688 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:58.688 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:58.688 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:30:58.688 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.924090+0000", 2024-09-14T06:30:58.688 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.4699999999999998e-07 2024-09-14T06:30:58.688 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:58.688 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:58.688 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:30:58.688 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.924658+0000", 2024-09-14T06:30:58.688 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00056802700000000003 2024-09-14T06:30:58.688 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:58.689 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:58.689 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:30:58.689 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.924664+0000", 2024-09-14T06:30:58.689 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.3570000000000003e-06 2024-09-14T06:30:58.689 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:58.689 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:58.689 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:30:58.689 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.924669+0000", 2024-09-14T06:30:58.689 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.2300000000000002e-06 2024-09-14T06:30:58.689 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:58.689 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:58.689 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:30:58.689 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.924802+0000", 2024-09-14T06:30:58.689 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00013362500000000001 2024-09-14T06:30:58.690 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:58.690 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:58.690 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:30:58.690 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.924824+0000", 2024-09-14T06:30:58.690 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.1449000000000001e-05 2024-09-14T06:30:58.690 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:58.690 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:58.690 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-14T06:30:58.690 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.926197+0000", 2024-09-14T06:30:58.690 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.001373354 2024-09-14T06:30:58.690 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:58.690 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:58.690 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:30:58.690 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.926226+0000", 2024-09-14T06:30:58.690 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.8951000000000001e-05 2024-09-14T06:30:58.691 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:58.691 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:58.691 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:58.691 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:58.691 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:58.691 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-14T06:30:58.692 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-14T06:30:58.693 INFO:tasks.ceph.osd.1.smithi094.stderr:2024-09-14T06:30:58.691+0000 7f328e47a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33641) UID: 0 2024-09-14T06:30:58.760 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-14T06:30:58.760 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 0 1 2024-09-14T06:30:58.793 INFO:tasks.ceph.osd.4.smithi138.stderr:2024-09-14T06:30:58.791+0000 7f6c653c8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33615) UID: 0 2024-09-14T06:30:58.846 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-14T06:30:58.847 INFO:teuthology.orchestra.run.smithi094.stdout: "ops": [], 2024-09-14T06:30:58.847 INFO:teuthology.orchestra.run.smithi094.stdout: "num_ops": 0 2024-09-14T06:30:58.847 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-14T06:30:58.856 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-14T06:30:58.899 INFO:tasks.ceph.osd.0.smithi094.stderr:2024-09-14T06:30:58.898+0000 7f07c9333640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33636) UID: 0 2024-09-14T06:30:58.961 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-14T06:30:58.961 INFO:teuthology.orchestra.run.smithi094.stdout: "ops": [], 2024-09-14T06:30:58.961 INFO:teuthology.orchestra.run.smithi094.stdout: "complaint_time": 30, 2024-09-14T06:30:58.961 INFO:teuthology.orchestra.run.smithi094.stdout: "num_blocked_ops": 0 2024-09-14T06:30:58.962 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-14T06:30:58.970 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-14T06:30:58.995 INFO:tasks.ceph.osd.4.smithi138.stderr:2024-09-14T06:30:58.994+0000 7f6c653c8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33615) UID: 0 2024-09-14T06:30:59.096 INFO:tasks.ceph.osd.6.smithi138.stderr:2024-09-14T06:30:59.095+0000 7f8cd8849640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33620) UID: 0 2024-09-14T06:30:59.115 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-14T06:30:59.115 INFO:teuthology.orchestra.run.smithi094.stdout: "size": 20, 2024-09-14T06:30:59.115 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 600, 2024-09-14T06:30:59.115 INFO:teuthology.orchestra.run.smithi094.stdout: "ops": [ 2024-09-14T06:30:59.115 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:59.115 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:44425 3.2 3:4603ad7f:::rbd_data.112dca2f6ae1.0000000000000005:17 [read 868352~4096 [fadvise_nocache]] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:30:59.115 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:43.854967+0000", 2024-09-14T06:30:59.115 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 75.258359493, 2024-09-14T06:30:59.115 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.063146025999999994, 2024-09-14T06:30:59.116 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:30:59.116 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:30:59.116 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:30:59.116 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:30:59.116 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:30:59.116 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:59.116 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 44425 2024-09-14T06:30:59.116 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:59.116 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:30:59.116 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:59.116 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:30:59.116 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.854967+0000", 2024-09-14T06:30:59.116 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:59.116 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:59.116 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:59.117 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:30:59.117 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.854967+0000", 2024-09-14T06:30:59.117 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:59.117 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:59.117 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:59.117 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:30:59.117 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.854967+0000", 2024-09-14T06:30:59.117 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.16e-07 2024-09-14T06:30:59.117 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:59.117 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:59.117 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:30:59.117 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.854968+0000", 2024-09-14T06:30:59.117 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.006e-06 2024-09-14T06:30:59.117 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:59.118 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:59.118 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:30:59.118 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.854971+0000", 2024-09-14T06:30:59.118 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.9359999999999999e-06 2024-09-14T06:30:59.118 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:59.118 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:59.118 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:30:59.118 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.854974+0000", 2024-09-14T06:30:59.118 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.1840000000000002e-06 2024-09-14T06:30:59.118 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:59.118 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:59.118 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:30:59.118 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.917945+0000", 2024-09-14T06:30:59.119 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.062970925999999997 2024-09-14T06:30:59.119 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:59.119 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:59.119 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:30:59.119 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.918030+0000", 2024-09-14T06:30:59.119 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.5863000000000001e-05 2024-09-14T06:30:59.119 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:59.119 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:59.119 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:30:59.119 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.918113+0000", 2024-09-14T06:30:59.119 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.2894999999999998e-05 2024-09-14T06:30:59.119 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:59.119 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:59.119 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:59.120 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:59.120 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:59.120 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:44426 3.2 3:4603ad7f:::rbd_data.112dca2f6ae1.0000000000000005:17 [read 892928~4096 [fadvise_nocache]] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:30:59.120 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:43.854977+0000", 2024-09-14T06:30:59.120 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 75.258350082999996, 2024-09-14T06:30:59.120 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.063369237999999994, 2024-09-14T06:30:59.120 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:30:59.120 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:30:59.120 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:30:59.120 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:30:59.120 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:30:59.120 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:59.120 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 44426 2024-09-14T06:30:59.120 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:59.121 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:30:59.121 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:59.121 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:30:59.121 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.854977+0000", 2024-09-14T06:30:59.121 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:59.121 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:59.121 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:59.121 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:30:59.121 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.854977+0000", 2024-09-14T06:30:59.121 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:59.121 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:59.121 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:59.121 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:30:59.121 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.854977+0000", 2024-09-14T06:30:59.122 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.1199999999999999e-07 2024-09-14T06:30:59.122 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:59.122 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:59.122 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:30:59.122 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.854980+0000", 2024-09-14T06:30:59.122 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.27e-06 2024-09-14T06:30:59.122 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:59.122 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:59.122 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:30:59.122 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.854983+0000", 2024-09-14T06:30:59.122 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.1650000000000002e-06 2024-09-14T06:30:59.122 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:59.122 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:59.122 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:30:59.123 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.854989+0000", 2024-09-14T06:30:59.123 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.6520000000000003e-06 2024-09-14T06:30:59.123 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:59.123 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:59.123 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:30:59.123 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.918143+0000", 2024-09-14T06:30:59.123 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.063154541999999994 2024-09-14T06:30:59.123 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:59.123 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:59.123 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:30:59.123 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.918209+0000", 2024-09-14T06:30:59.123 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.5529000000000002e-05 2024-09-14T06:30:59.123 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:59.123 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:59.124 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:30:59.124 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.918346+0000", 2024-09-14T06:30:59.124 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000136868 2024-09-14T06:30:59.124 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:59.124 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:59.124 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:59.124 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:59.124 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:59.124 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:44427 3.2 3:4603ad7f:::rbd_data.112dca2f6ae1.0000000000000005:17 [read 917504~4096 [fadvise_nocache]] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:30:59.124 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:43.854992+0000", 2024-09-14T06:30:59.124 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 75.258334437000002, 2024-09-14T06:30:59.124 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.063607792999999996, 2024-09-14T06:30:59.124 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:30:59.125 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:30:59.125 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:30:59.125 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:30:59.125 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:30:59.125 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:59.125 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 44427 2024-09-14T06:30:59.125 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:59.125 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:30:59.125 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:59.125 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:30:59.125 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.854992+0000", 2024-09-14T06:30:59.125 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:59.125 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:59.125 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:59.126 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:30:59.126 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.854992+0000", 2024-09-14T06:30:59.126 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:59.126 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:59.126 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:59.126 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:30:59.126 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.854993+0000", 2024-09-14T06:30:59.126 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.1400000000000001e-07 2024-09-14T06:30:59.126 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:59.126 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:59.126 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:30:59.126 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.854994+0000", 2024-09-14T06:30:59.126 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.091e-06 2024-09-14T06:30:59.126 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:59.127 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:59.127 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:30:59.127 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855001+0000", 2024-09-14T06:30:59.127 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.204e-06 2024-09-14T06:30:59.127 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:59.127 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:59.127 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:30:59.127 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855003+0000", 2024-09-14T06:30:59.127 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.514e-06 2024-09-14T06:30:59.127 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:59.127 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:59.127 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:30:59.127 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.918380+0000", 2024-09-14T06:30:59.127 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.063376738000000002 2024-09-14T06:30:59.128 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:59.128 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:59.128 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:30:59.128 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.918453+0000", 2024-09-14T06:30:59.128 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.3291e-05 2024-09-14T06:30:59.128 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:59.128 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:59.128 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:30:59.128 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.918600+0000", 2024-09-14T06:30:59.128 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00014674100000000001 2024-09-14T06:30:59.128 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:59.128 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:59.128 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:59.128 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:59.128 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:59.129 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:44428 3.2 3:4603ad7f:::rbd_data.112dca2f6ae1.0000000000000005:17 [read 942080~4096 [fadvise_nocache]] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:30:59.129 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:43.855006+0000", 2024-09-14T06:30:59.129 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 75.258320386999998, 2024-09-14T06:30:59.129 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.063846321999999997, 2024-09-14T06:30:59.129 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:30:59.129 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:30:59.129 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:30:59.129 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:30:59.129 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:30:59.129 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:59.129 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 44428 2024-09-14T06:30:59.129 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:59.129 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:30:59.129 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:59.130 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:30:59.130 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855006+0000", 2024-09-14T06:30:59.130 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:59.130 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:59.130 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:59.130 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:30:59.130 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855006+0000", 2024-09-14T06:30:59.130 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:59.130 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:59.130 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:59.130 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:30:59.130 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855007+0000", 2024-09-14T06:30:59.130 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.1199999999999999e-07 2024-09-14T06:30:59.130 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:59.130 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:59.131 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:30:59.131 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855011+0000", 2024-09-14T06:30:59.131 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.9539999999999998e-06 2024-09-14T06:30:59.131 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:59.131 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:59.131 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:30:59.131 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855014+0000", 2024-09-14T06:30:59.131 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.2930000000000001e-06 2024-09-14T06:30:59.131 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:59.131 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:59.131 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:30:59.131 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855016+0000", 2024-09-14T06:30:59.131 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.3039999999999999e-06 2024-09-14T06:30:59.131 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:59.132 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:59.132 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:30:59.132 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.918634+0000", 2024-09-14T06:30:59.132 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.063617928000000004 2024-09-14T06:30:59.132 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:59.132 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:59.132 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:30:59.132 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.918711+0000", 2024-09-14T06:30:59.132 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.6545000000000007e-05 2024-09-14T06:30:59.132 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:59.132 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:59.132 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:30:59.132 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.918853+0000", 2024-09-14T06:30:59.132 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00014208600000000001 2024-09-14T06:30:59.133 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:59.133 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:59.133 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:59.133 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:59.133 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:59.133 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:44429 3.2 3:4603ad7f:::rbd_data.112dca2f6ae1.0000000000000005:17 [read 966656~4096 [fadvise_nocache]] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:30:59.133 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:43.855019+0000", 2024-09-14T06:30:59.133 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 75.258307713999997, 2024-09-14T06:30:59.133 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.064059226999999996, 2024-09-14T06:30:59.133 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:30:59.133 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:30:59.133 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:30:59.133 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:30:59.133 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:30:59.134 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:59.134 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 44429 2024-09-14T06:30:59.134 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:59.134 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:30:59.134 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:59.134 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:30:59.134 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855019+0000", 2024-09-14T06:30:59.134 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:59.134 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:59.134 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:59.134 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:30:59.134 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855019+0000", 2024-09-14T06:30:59.134 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:59.134 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:59.135 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:59.135 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:30:59.135 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855019+0000", 2024-09-14T06:30:59.135 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.28e-07 2024-09-14T06:30:59.135 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:59.135 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:59.135 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:30:59.135 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855020+0000", 2024-09-14T06:30:59.135 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.0330000000000001e-06 2024-09-14T06:30:59.135 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:59.135 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:59.135 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:30:59.135 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855023+0000", 2024-09-14T06:30:59.135 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.79e-06 2024-09-14T06:30:59.136 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:59.136 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:59.136 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:30:59.136 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855025+0000", 2024-09-14T06:30:59.136 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.216e-06 2024-09-14T06:30:59.136 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:59.136 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:59.136 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:30:59.136 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.918886+0000", 2024-09-14T06:30:59.136 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.063861037999999995 2024-09-14T06:30:59.136 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:59.136 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:59.136 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:30:59.137 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.918952+0000", 2024-09-14T06:30:59.137 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.5993000000000003e-05 2024-09-14T06:30:59.137 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:59.137 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:59.137 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:30:59.137 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.919078+0000", 2024-09-14T06:30:59.137 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00012592899999999999 2024-09-14T06:30:59.137 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:59.137 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:59.137 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:59.137 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:59.137 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:59.137 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:44430 3.2 3:4603ad7f:::rbd_data.112dca2f6ae1.0000000000000005:17 [read 991232~4096 [fadvise_nocache]] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:30:59.137 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:43.855028+0000", 2024-09-14T06:30:59.138 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 75.258298703999998, 2024-09-14T06:30:59.138 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.064284346000000006, 2024-09-14T06:30:59.138 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:30:59.138 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:30:59.138 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:30:59.138 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:30:59.138 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:30:59.138 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:59.138 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 44430 2024-09-14T06:30:59.138 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:59.138 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:30:59.138 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:59.138 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:30:59.138 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855028+0000", 2024-09-14T06:30:59.138 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:59.139 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:59.139 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:59.139 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:30:59.139 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855028+0000", 2024-09-14T06:30:59.139 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:59.139 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:59.139 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:59.139 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:30:59.139 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855028+0000", 2024-09-14T06:30:59.139 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.1199999999999999e-07 2024-09-14T06:30:59.139 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:59.139 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:59.139 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:30:59.139 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855029+0000", 2024-09-14T06:30:59.140 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.1100000000000004e-07 2024-09-14T06:30:59.140 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:59.140 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:59.140 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:30:59.140 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855032+0000", 2024-09-14T06:30:59.140 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.7889999999999999e-06 2024-09-14T06:30:59.140 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:59.140 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:59.140 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:30:59.140 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855034+0000", 2024-09-14T06:30:59.140 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.125e-06 2024-09-14T06:30:59.140 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:59.140 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:59.140 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:30:59.141 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.919110+0000", 2024-09-14T06:30:59.141 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.064076413999999998 2024-09-14T06:30:59.141 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:59.141 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:59.141 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:30:59.141 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.919186+0000", 2024-09-14T06:30:59.141 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.5784999999999994e-05 2024-09-14T06:30:59.141 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:59.141 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:59.141 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:30:59.141 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.919312+0000", 2024-09-14T06:30:59.141 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00012611 2024-09-14T06:30:59.141 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:59.141 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:59.142 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:59.142 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:59.142 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:59.142 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:44431 3.2 3:4603ad7f:::rbd_data.112dca2f6ae1.0000000000000005:17 [read 1015808~4096 [fadvise_nocache]] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:30:59.142 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:43.855037+0000", 2024-09-14T06:30:59.142 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 75.258289836000003, 2024-09-14T06:30:59.142 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.064469755000000004, 2024-09-14T06:30:59.142 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:30:59.142 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:30:59.142 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:30:59.142 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:30:59.142 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:30:59.142 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:59.142 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 44431 2024-09-14T06:30:59.143 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:59.143 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:30:59.143 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:59.143 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:30:59.143 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855037+0000", 2024-09-14T06:30:59.143 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:59.143 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:59.143 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:59.143 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:30:59.143 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855037+0000", 2024-09-14T06:30:59.143 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:59.143 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:59.143 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:59.143 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:30:59.144 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855037+0000", 2024-09-14T06:30:59.144 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.1e-07 2024-09-14T06:30:59.144 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:59.144 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:59.144 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:30:59.144 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855038+0000", 2024-09-14T06:30:59.144 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.9400000000000004e-07 2024-09-14T06:30:59.144 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:59.144 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:59.144 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:30:59.144 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855041+0000", 2024-09-14T06:30:59.144 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.9639999999999999e-06 2024-09-14T06:30:59.144 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:59.145 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:59.145 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:30:59.145 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855043+0000", 2024-09-14T06:30:59.145 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.2259999999999999e-06 2024-09-14T06:30:59.145 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:59.145 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:59.145 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:30:59.145 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.919347+0000", 2024-09-14T06:30:59.145 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.064304091999999993 2024-09-14T06:30:59.145 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:59.145 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:59.145 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:30:59.145 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.919393+0000", 2024-09-14T06:30:59.145 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.5340999999999998e-05 2024-09-14T06:30:59.146 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:59.146 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:59.146 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:30:59.146 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.919507+0000", 2024-09-14T06:30:59.146 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000114028 2024-09-14T06:30:59.146 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:59.146 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:59.146 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:59.146 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:59.146 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:59.146 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:44432 3.2 3:4603ad7f:::rbd_data.112dca2f6ae1.0000000000000005:17 [read 1040384~4096 [fadvise_nocache]] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:30:59.146 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:43.855046+0000", 2024-09-14T06:30:59.146 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 75.258280768000006, 2024-09-14T06:30:59.146 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.064639253999999993, 2024-09-14T06:30:59.147 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:30:59.147 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:30:59.147 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:30:59.147 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:30:59.147 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:30:59.147 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:59.147 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 44432 2024-09-14T06:30:59.147 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:59.147 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:30:59.147 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:59.147 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:30:59.147 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855046+0000", 2024-09-14T06:30:59.147 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:59.148 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:59.148 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:59.148 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:30:59.148 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855046+0000", 2024-09-14T06:30:59.148 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:59.148 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:59.148 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:59.148 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:30:59.148 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855046+0000", 2024-09-14T06:30:59.148 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.05e-07 2024-09-14T06:30:59.148 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:59.148 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:59.148 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:30:59.148 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855047+0000", 2024-09-14T06:30:59.149 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.879999999999999e-07 2024-09-14T06:30:59.149 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:59.149 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:59.149 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:30:59.149 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855050+0000", 2024-09-14T06:30:59.149 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.8459999999999999e-06 2024-09-14T06:30:59.149 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:59.149 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:59.149 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:30:59.149 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855052+0000", 2024-09-14T06:30:59.149 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.136e-06 2024-09-14T06:30:59.149 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:59.149 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:59.149 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:30:59.149 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.919536+0000", 2024-09-14T06:30:59.150 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.064484132 2024-09-14T06:30:59.150 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:59.150 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:59.150 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:30:59.150 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.919611+0000", 2024-09-14T06:30:59.150 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.4407999999999996e-05 2024-09-14T06:30:59.150 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:59.150 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:59.150 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:30:59.150 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.919685+0000", 2024-09-14T06:30:59.150 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.4538999999999999e-05 2024-09-14T06:30:59.150 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:59.151 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:59.151 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:59.151 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:59.151 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:59.151 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:44518 3.2 3:4190d882:::rbd_data.112dca2f6ae1.0000000000000002:17 [read 12288~4096 [fadvise_nocache]] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:30:59.151 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:43.855055+0000", 2024-09-14T06:30:59.151 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 75.258271883999996, 2024-09-14T06:30:59.151 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.064894317000000007, 2024-09-14T06:30:59.151 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:30:59.151 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:30:59.151 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:30:59.151 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:30:59.151 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:30:59.152 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:59.152 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 44518 2024-09-14T06:30:59.152 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:59.152 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:30:59.152 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:59.152 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:30:59.152 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855055+0000", 2024-09-14T06:30:59.152 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:59.152 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:59.152 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:59.152 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:30:59.152 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855055+0000", 2024-09-14T06:30:59.152 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:59.152 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:59.153 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:59.153 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:30:59.153 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855055+0000", 2024-09-14T06:30:59.153 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.0699999999999999e-07 2024-09-14T06:30:59.153 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:59.153 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:59.153 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:30:59.153 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855056+0000", 2024-09-14T06:30:59.153 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.9400000000000004e-07 2024-09-14T06:30:59.153 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:59.153 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:59.153 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:30:59.153 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855059+0000", 2024-09-14T06:30:59.153 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.8870000000000001e-06 2024-09-14T06:30:59.154 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:59.154 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:59.154 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:30:59.154 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855061+0000", 2024-09-14T06:30:59.154 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.1919999999999999e-06 2024-09-14T06:30:59.154 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:59.154 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:59.154 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:30:59.154 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.919719+0000", 2024-09-14T06:30:59.154 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.064658013 2024-09-14T06:30:59.154 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:59.154 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:59.154 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:30:59.155 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.919788+0000", 2024-09-14T06:30:59.155 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.9282000000000005e-05 2024-09-14T06:30:59.155 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:59.155 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:59.155 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:30:59.155 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.919949+0000", 2024-09-14T06:30:59.155 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00016084199999999999 2024-09-14T06:30:59.155 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:59.155 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:59.155 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:59.155 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:59.155 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:59.155 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:44519 3.2 3:4190d882:::rbd_data.112dca2f6ae1.0000000000000002:17 [read 36864~4096 [fadvise_nocache]] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:30:59.155 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:43.855064+0000", 2024-09-14T06:30:59.155 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 75.258262907000002, 2024-09-14T06:30:59.156 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.065081865000000003, 2024-09-14T06:30:59.156 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:30:59.156 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:30:59.156 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:30:59.156 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:30:59.156 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:30:59.156 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:59.156 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 44519 2024-09-14T06:30:59.156 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:59.156 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:30:59.156 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:59.156 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:30:59.156 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855064+0000", 2024-09-14T06:30:59.157 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:59.157 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:59.157 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:59.157 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:30:59.157 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855064+0000", 2024-09-14T06:30:59.157 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:59.157 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:59.157 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:59.157 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:30:59.157 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855064+0000", 2024-09-14T06:30:59.157 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.1e-07 2024-09-14T06:30:59.157 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:59.157 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:59.158 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:30:59.158 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855065+0000", 2024-09-14T06:30:59.158 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.035e-06 2024-09-14T06:30:59.158 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:59.158 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:59.158 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:30:59.158 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855068+0000", 2024-09-14T06:30:59.158 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.8339999999999999e-06 2024-09-14T06:30:59.158 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:59.158 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:59.158 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:30:59.158 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855071+0000", 2024-09-14T06:30:59.158 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.3450000000000002e-06 2024-09-14T06:30:59.158 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:59.159 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:59.159 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:30:59.159 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.919979+0000", 2024-09-14T06:30:59.159 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.064907647999999998 2024-09-14T06:30:59.159 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:59.159 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:59.159 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:30:59.159 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.920026+0000", 2024-09-14T06:30:59.159 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.7595000000000001e-05 2024-09-14T06:30:59.159 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:59.159 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:59.159 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:30:59.159 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.920146+0000", 2024-09-14T06:30:59.159 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00011919800000000001 2024-09-14T06:30:59.159 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:59.160 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:59.160 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:59.160 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:59.160 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:59.160 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:44520 3.2 3:4190d882:::rbd_data.112dca2f6ae1.0000000000000002:17 [read 61440~4096 [fadvise_nocache]] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:30:59.160 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:43.855075+0000", 2024-09-14T06:30:59.160 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 75.258251856000001, 2024-09-14T06:30:59.160 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.065335078000000005, 2024-09-14T06:30:59.160 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:30:59.160 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:30:59.160 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:30:59.160 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:30:59.160 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:30:59.161 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:59.161 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 44520 2024-09-14T06:30:59.161 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:59.161 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:30:59.161 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:59.161 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:30:59.161 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855075+0000", 2024-09-14T06:30:59.161 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:59.161 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:59.161 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:59.161 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:30:59.161 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855075+0000", 2024-09-14T06:30:59.161 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:59.161 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:59.162 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:59.162 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:30:59.162 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855075+0000", 2024-09-14T06:30:59.162 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.1e-07 2024-09-14T06:30:59.162 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:59.162 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:59.162 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:30:59.162 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855076+0000", 2024-09-14T06:30:59.162 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.0839999999999999e-06 2024-09-14T06:30:59.162 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:59.162 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:59.162 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:30:59.162 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855079+0000", 2024-09-14T06:30:59.162 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.9359999999999999e-06 2024-09-14T06:30:59.163 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:59.163 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:59.163 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:30:59.163 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855084+0000", 2024-09-14T06:30:59.163 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.1730000000000003e-06 2024-09-14T06:30:59.163 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:59.163 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:59.163 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:30:59.163 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.920175+0000", 2024-09-14T06:30:59.163 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.065090234999999996 2024-09-14T06:30:59.163 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:59.163 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:59.163 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:30:59.163 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.920250+0000", 2024-09-14T06:30:59.164 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.5807000000000001e-05 2024-09-14T06:30:59.164 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:59.164 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:59.164 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:30:59.164 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.920410+0000", 2024-09-14T06:30:59.164 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00015963300000000001 2024-09-14T06:30:59.164 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:59.164 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:59.164 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:59.164 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:59.164 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:59.164 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:44521 3.2 3:4190d882:::rbd_data.112dca2f6ae1.0000000000000002:17 [read 86016~4096 [fadvise_nocache]] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:30:59.164 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:43.855087+0000", 2024-09-14T06:30:59.164 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 75.258239375000002, 2024-09-14T06:30:59.165 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.065563264999999996, 2024-09-14T06:30:59.165 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:30:59.165 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:30:59.165 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:30:59.165 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:30:59.165 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:30:59.165 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:59.165 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 44521 2024-09-14T06:30:59.165 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:59.165 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:30:59.165 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:59.165 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:30:59.165 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855087+0000", 2024-09-14T06:30:59.165 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:59.166 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:59.166 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:59.166 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:30:59.166 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855087+0000", 2024-09-14T06:30:59.166 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:59.166 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:59.166 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:59.166 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:30:59.166 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855088+0000", 2024-09-14T06:30:59.166 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.3699999999999999e-07 2024-09-14T06:30:59.166 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:59.166 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:59.166 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:30:59.167 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855089+0000", 2024-09-14T06:30:59.167 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.9900000000000009e-07 2024-09-14T06:30:59.167 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:59.167 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:59.167 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:30:59.167 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855091+0000", 2024-09-14T06:30:59.167 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.8940000000000002e-06 2024-09-14T06:30:59.167 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:59.167 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:59.167 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:30:59.167 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855094+0000", 2024-09-14T06:30:59.167 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.351e-06 2024-09-14T06:30:59.167 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:59.167 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:59.168 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:30:59.168 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.920437+0000", 2024-09-14T06:30:59.168 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.065343221000000007 2024-09-14T06:30:59.168 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:59.168 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:59.168 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:30:59.168 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.920481+0000", 2024-09-14T06:30:59.168 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.4431000000000002e-05 2024-09-14T06:30:59.168 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:59.168 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:59.168 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:30:59.168 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.920651+0000", 2024-09-14T06:30:59.168 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00016913200000000001 2024-09-14T06:30:59.168 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:59.169 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:59.169 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:59.169 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:59.169 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:59.169 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:44522 3.2 3:4190d882:::rbd_data.112dca2f6ae1.0000000000000002:17 [read 110592~4096 [fadvise_nocache]] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:30:59.169 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:43.855098+0000", 2024-09-14T06:30:59.169 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 75.258229147999998, 2024-09-14T06:30:59.169 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.065827628999999999, 2024-09-14T06:30:59.169 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:30:59.169 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:30:59.169 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:30:59.169 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:30:59.169 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:30:59.170 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:59.170 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 44522 2024-09-14T06:30:59.170 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:59.170 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:30:59.170 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:59.170 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:30:59.170 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855098+0000", 2024-09-14T06:30:59.170 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:59.170 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:59.170 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:59.170 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:30:59.170 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855098+0000", 2024-09-14T06:30:59.170 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:59.170 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:59.170 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:59.171 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:30:59.171 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855098+0000", 2024-09-14T06:30:59.171 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.1e-07 2024-09-14T06:30:59.171 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:59.171 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:59.171 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:30:59.171 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855099+0000", 2024-09-14T06:30:59.171 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.012e-06 2024-09-14T06:30:59.171 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:59.171 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:59.171 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:30:59.171 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855102+0000", 2024-09-14T06:30:59.171 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.9380000000000001e-06 2024-09-14T06:30:59.171 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:59.171 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:59.172 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:30:59.172 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855108+0000", 2024-09-14T06:30:59.172 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.4339999999999997e-06 2024-09-14T06:30:59.172 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:59.172 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:59.172 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:30:59.172 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.920681+0000", 2024-09-14T06:30:59.172 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.065572943999999994 2024-09-14T06:30:59.172 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:59.172 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:59.172 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:30:59.172 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.920724+0000", 2024-09-14T06:30:59.172 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.2877000000000003e-05 2024-09-14T06:30:59.173 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:59.173 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:59.173 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:30:59.173 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.920925+0000", 2024-09-14T06:30:59.173 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00020121399999999999 2024-09-14T06:30:59.173 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:59.173 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:59.173 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:59.173 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:59.173 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:59.173 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:44523 3.2 3:4190d882:::rbd_data.112dca2f6ae1.0000000000000002:17 [read 135168~4096 [fadvise_nocache]] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:30:59.173 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:43.855111+0000", 2024-09-14T06:30:59.173 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 75.258215426000007, 2024-09-14T06:30:59.173 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.066028198999999996, 2024-09-14T06:30:59.174 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:30:59.174 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:30:59.174 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:30:59.174 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:30:59.174 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:30:59.174 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:59.174 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 44523 2024-09-14T06:30:59.174 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:59.174 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:30:59.174 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:59.174 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:30:59.174 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855111+0000", 2024-09-14T06:30:59.174 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:59.174 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:59.174 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:59.175 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:30:59.175 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855111+0000", 2024-09-14T06:30:59.175 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:59.175 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:59.175 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:59.175 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:30:59.175 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855112+0000", 2024-09-14T06:30:59.175 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.1199999999999999e-07 2024-09-14T06:30:59.175 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:59.175 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:59.175 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:30:59.175 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855115+0000", 2024-09-14T06:30:59.175 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.3380000000000001e-06 2024-09-14T06:30:59.175 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:59.176 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:59.176 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:30:59.176 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855118+0000", 2024-09-14T06:30:59.176 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.366e-06 2024-09-14T06:30:59.176 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:59.176 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:59.176 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:30:59.176 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855121+0000", 2024-09-14T06:30:59.176 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.8329999999999998e-06 2024-09-14T06:30:59.176 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:59.176 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:59.176 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:30:59.176 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.920956+0000", 2024-09-14T06:30:59.177 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.065835251999999997 2024-09-14T06:30:59.177 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:59.177 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:59.177 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:30:59.177 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.921001+0000", 2024-09-14T06:30:59.177 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.4778999999999999e-05 2024-09-14T06:30:59.177 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:59.177 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:59.177 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:30:59.177 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.921139+0000", 2024-09-14T06:30:59.177 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000138419 2024-09-14T06:30:59.177 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:59.177 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:59.177 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:59.178 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:59.178 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:59.178 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:44524 3.2 3:4190d882:::rbd_data.112dca2f6ae1.0000000000000002:17 [read 159744~4096 [fadvise_nocache]] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:30:59.178 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:43.855124+0000", 2024-09-14T06:30:59.178 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 75.258202447000002, 2024-09-14T06:30:59.178 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.066249261000000004, 2024-09-14T06:30:59.178 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:30:59.178 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:30:59.178 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:30:59.178 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:30:59.178 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:30:59.178 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:59.178 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 44524 2024-09-14T06:30:59.178 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:59.178 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:30:59.179 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:59.179 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:30:59.179 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855124+0000", 2024-09-14T06:30:59.179 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:59.179 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:59.179 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:59.179 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:30:59.179 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855124+0000", 2024-09-14T06:30:59.179 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:59.179 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:59.179 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:59.179 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:30:59.179 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861086+0000", 2024-09-14T06:30:59.179 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0059613829999999998 2024-09-14T06:30:59.180 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:59.180 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:59.180 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:30:59.180 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861100+0000", 2024-09-14T06:30:59.180 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.4644000000000001e-05 2024-09-14T06:30:59.180 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:59.180 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:59.180 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:30:59.180 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861109+0000", 2024-09-14T06:30:59.180 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.541e-06 2024-09-14T06:30:59.180 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:59.180 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:59.180 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:30:59.180 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861117+0000", 2024-09-14T06:30:59.181 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.1049999999999995e-06 2024-09-14T06:30:59.181 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:59.181 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:59.181 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:30:59.181 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.921169+0000", 2024-09-14T06:30:59.181 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.060052405000000003 2024-09-14T06:30:59.181 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:59.181 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:59.181 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:30:59.181 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.921213+0000", 2024-09-14T06:30:59.181 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.3565e-05 2024-09-14T06:30:59.181 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:59.181 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:59.181 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:30:59.182 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.921374+0000", 2024-09-14T06:30:59.182 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00016061800000000001 2024-09-14T06:30:59.182 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:59.182 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:59.182 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:59.182 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:59.182 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:59.182 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:44536 3.2 3:4190d882:::rbd_data.112dca2f6ae1.0000000000000002:17 [read 454656~4096 [fadvise_nocache]] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:30:59.182 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:43.861384+0000", 2024-09-14T06:30:59.182 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 75.251942889999995, 2024-09-14T06:30:59.182 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.063058528000000003, 2024-09-14T06:30:59.182 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:30:59.182 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:30:59.182 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:30:59.182 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:30:59.183 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:30:59.183 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:59.183 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 44536 2024-09-14T06:30:59.183 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:59.183 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:30:59.183 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:59.183 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:30:59.183 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861384+0000", 2024-09-14T06:30:59.183 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:59.183 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:59.183 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:59.183 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:30:59.183 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861384+0000", 2024-09-14T06:30:59.183 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:59.183 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:59.184 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:59.184 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:30:59.184 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861384+0000", 2024-09-14T06:30:59.184 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.4000000000000002e-07 2024-09-14T06:30:59.184 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:59.184 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:59.184 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:30:59.184 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861386+0000", 2024-09-14T06:30:59.184 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.891e-06 2024-09-14T06:30:59.184 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:59.184 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:59.184 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:30:59.184 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861392+0000", 2024-09-14T06:30:59.185 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.9939999999999997e-06 2024-09-14T06:30:59.185 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:59.185 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:59.185 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:30:59.185 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861398+0000", 2024-09-14T06:30:59.185 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.507e-06 2024-09-14T06:30:59.185 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:59.185 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:59.185 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:30:59.185 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.924149+0000", 2024-09-14T06:30:59.185 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.062751135999999999 2024-09-14T06:30:59.185 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:59.185 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:59.185 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:30:59.185 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.924226+0000", 2024-09-14T06:30:59.186 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.7517999999999999e-05 2024-09-14T06:30:59.186 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:59.186 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:59.186 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:30:59.186 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.924442+0000", 2024-09-14T06:30:59.186 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000216042 2024-09-14T06:30:59.186 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:59.186 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:59.186 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:59.186 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:59.186 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:59.186 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:44537 3.2 3:4190d882:::rbd_data.112dca2f6ae1.0000000000000002:17 [read 479232~4096 [fadvise_nocache]] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:30:59.186 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:43.861405+0000", 2024-09-14T06:30:59.186 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 75.251921843999995, 2024-09-14T06:30:59.187 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.063351795000000002, 2024-09-14T06:30:59.187 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:30:59.187 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:30:59.187 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:30:59.187 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:30:59.187 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:30:59.187 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:59.187 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 44537 2024-09-14T06:30:59.187 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:59.187 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:30:59.187 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:59.187 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:30:59.187 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861405+0000", 2024-09-14T06:30:59.187 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:59.188 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:59.188 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:59.188 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:30:59.188 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861405+0000", 2024-09-14T06:30:59.188 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:59.188 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:59.188 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:59.188 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:30:59.188 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861405+0000", 2024-09-14T06:30:59.188 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.9099999999999999e-07 2024-09-14T06:30:59.188 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:59.188 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:59.188 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:30:59.189 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861407+0000", 2024-09-14T06:30:59.189 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.075e-06 2024-09-14T06:30:59.189 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:59.189 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:59.189 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:30:59.189 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861414+0000", 2024-09-14T06:30:59.189 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.4659999999999996e-06 2024-09-14T06:30:59.189 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:59.189 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:59.189 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:30:59.189 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861418+0000", 2024-09-14T06:30:59.189 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.5519999999999998e-06 2024-09-14T06:30:59.189 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:59.189 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:59.190 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:30:59.190 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.924474+0000", 2024-09-14T06:30:59.190 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.063056012999999994 2024-09-14T06:30:59.190 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:59.190 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:59.190 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:30:59.190 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.924556+0000", 2024-09-14T06:30:59.190 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.1181999999999996e-05 2024-09-14T06:30:59.190 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:59.190 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:59.190 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:30:59.190 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.924757+0000", 2024-09-14T06:30:59.190 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00020111599999999999 2024-09-14T06:30:59.190 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:59.190 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:59.191 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:59.191 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:59.191 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:59.191 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:44538 3.2 3:4190d882:::rbd_data.112dca2f6ae1.0000000000000002:17 [read 503808~4096 [fadvise_nocache]] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:30:59.191 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:43.861425+0000", 2024-09-14T06:30:59.191 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 75.251901802999996, 2024-09-14T06:30:59.191 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.063627125000000007, 2024-09-14T06:30:59.191 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:30:59.191 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:30:59.191 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:30:59.191 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:30:59.191 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:30:59.191 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:59.191 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 44538 2024-09-14T06:30:59.191 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:59.192 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:30:59.192 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:59.192 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:30:59.192 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861425+0000", 2024-09-14T06:30:59.192 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:59.192 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:59.192 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:59.192 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:30:59.192 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861425+0000", 2024-09-14T06:30:59.192 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:59.192 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:59.192 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:59.192 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:30:59.192 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861425+0000", 2024-09-14T06:30:59.193 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.9999999999999998e-07 2024-09-14T06:30:59.193 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:59.193 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:59.193 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:30:59.193 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861427+0000", 2024-09-14T06:30:59.193 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.9829999999999999e-06 2024-09-14T06:30:59.193 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:59.193 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:59.193 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:30:59.193 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861433+0000", 2024-09-14T06:30:59.193 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.2889999999999997e-06 2024-09-14T06:30:59.193 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:59.193 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:59.193 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:30:59.194 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861454+0000", 2024-09-14T06:30:59.194 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.1844999999999998e-05 2024-09-14T06:30:59.194 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:59.194 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:59.194 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:30:59.194 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.924786+0000", 2024-09-14T06:30:59.194 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.063331497 2024-09-14T06:30:59.194 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:59.194 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:59.194 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:30:59.194 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.924864+0000", 2024-09-14T06:30:59.194 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.7744000000000006e-05 2024-09-14T06:30:59.194 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:59.194 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:59.194 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:30:59.195 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.925052+0000", 2024-09-14T06:30:59.195 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00018836699999999999 2024-09-14T06:30:59.195 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:59.195 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:59.195 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:59.195 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:59.195 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:59.195 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:44539 3.2 3:4190d882:::rbd_data.112dca2f6ae1.0000000000000002:17 [read 528384~4096 [fadvise_nocache]] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:30:59.195 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:43.861462+0000", 2024-09-14T06:30:59.195 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 75.251864475999994, 2024-09-14T06:30:59.195 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.063855705999999998, 2024-09-14T06:30:59.195 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:30:59.195 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:30:59.195 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:30:59.196 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:30:59.196 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:30:59.196 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:59.196 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 44539 2024-09-14T06:30:59.196 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:59.196 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:30:59.196 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:59.196 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:30:59.196 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861462+0000", 2024-09-14T06:30:59.196 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:59.196 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:59.196 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:59.196 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:30:59.196 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861462+0000", 2024-09-14T06:30:59.197 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:59.197 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:59.197 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:59.197 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:30:59.197 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861463+0000", 2024-09-14T06:30:59.197 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.5200000000000002e-07 2024-09-14T06:30:59.197 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:59.197 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:59.197 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:30:59.197 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861465+0000", 2024-09-14T06:30:59.197 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.5189999999999999e-06 2024-09-14T06:30:59.197 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:59.197 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:59.197 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:30:59.198 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861471+0000", 2024-09-14T06:30:59.198 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.6899999999999997e-06 2024-09-14T06:30:59.198 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:59.198 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:59.198 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:30:59.198 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861475+0000", 2024-09-14T06:30:59.198 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.5730000000000001e-06 2024-09-14T06:30:59.198 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:59.198 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:59.198 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:30:59.198 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.925085+0000", 2024-09-14T06:30:59.198 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.063609902999999995 2024-09-14T06:30:59.198 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:59.198 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:59.199 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:30:59.199 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.925145+0000", 2024-09-14T06:30:59.199 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.9383999999999999e-05 2024-09-14T06:30:59.199 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:59.199 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:59.199 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:30:59.199 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.925318+0000", 2024-09-14T06:30:59.199 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00017318499999999999 2024-09-14T06:30:59.199 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:59.199 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:59.199 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:59.199 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:59.199 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:59.199 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:44540 3.2 3:4190d882:::rbd_data.112dca2f6ae1.0000000000000002:17 [read 552960~4096 [fadvise_nocache]] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:30:59.200 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:43.861483+0000", 2024-09-14T06:30:59.200 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 75.251844044999999, 2024-09-14T06:30:59.200 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.064065681999999999, 2024-09-14T06:30:59.200 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:30:59.200 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:30:59.200 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:30:59.200 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:30:59.200 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:30:59.200 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:59.200 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 44540 2024-09-14T06:30:59.200 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:59.200 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:30:59.200 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:59.200 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:30:59.200 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861483+0000", 2024-09-14T06:30:59.201 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:59.201 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:59.201 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:59.201 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:30:59.201 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861483+0000", 2024-09-14T06:30:59.201 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:30:59.201 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:59.201 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:59.201 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:30:59.201 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.867079+0000", 2024-09-14T06:30:59.201 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0055966480000000001 2024-09-14T06:30:59.201 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:59.201 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:59.201 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:30:59.202 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.867086+0000", 2024-09-14T06:30:59.202 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.387e-06 2024-09-14T06:30:59.202 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:59.202 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:59.202 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:30:59.202 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.867094+0000", 2024-09-14T06:30:59.202 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.9419999999999995e-06 2024-09-14T06:30:59.202 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:59.202 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:59.202 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:30:59.202 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.867109+0000", 2024-09-14T06:30:59.202 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.5818999999999999e-05 2024-09-14T06:30:59.202 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:59.202 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:59.203 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:30:59.203 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.925354+0000", 2024-09-14T06:30:59.203 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.058244047 2024-09-14T06:30:59.203 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:59.203 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:59.203 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:30:59.203 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.925406+0000", 2024-09-14T06:30:59.203 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.2661999999999998e-05 2024-09-14T06:30:59.203 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:30:59.203 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:30:59.203 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:30:59.203 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.925548+0000", 2024-09-14T06:30:59.203 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000142177 2024-09-14T06:30:59.203 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:59.203 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:59.204 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:59.204 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:30:59.204 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:30:59.204 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-14T06:30:59.205 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-14T06:30:59.206 INFO:tasks.ceph.osd.0.smithi094.stderr:2024-09-14T06:30:59.204+0000 7f07c9333640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33636) UID: 0 2024-09-14T06:30:59.306 INFO:tasks.ceph.osd.4.smithi138.stderr:2024-09-14T06:30:59.304+0000 7f6c653c8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33615) UID: 0 2024-09-14T06:30:59.322 INFO:teuthology.orchestra.run.smithi138.stdout:{ 2024-09-14T06:30:59.322 INFO:teuthology.orchestra.run.smithi138.stdout: "ops": [], 2024-09-14T06:30:59.322 INFO:teuthology.orchestra.run.smithi138.stdout: "num_ops": 0 2024-09-14T06:30:59.322 INFO:teuthology.orchestra.run.smithi138.stdout:} 2024-09-14T06:30:59.331 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-14T06:30:59.415 INFO:tasks.ceph.osd.4.smithi138.stderr:2024-09-14T06:30:59.414+0000 7f6c653c8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33615) UID: 0 2024-09-14T06:30:59.475 INFO:teuthology.orchestra.run.smithi138.stdout:{ 2024-09-14T06:30:59.475 INFO:teuthology.orchestra.run.smithi138.stdout: "ops": [], 2024-09-14T06:30:59.475 INFO:teuthology.orchestra.run.smithi138.stdout: "complaint_time": 30, 2024-09-14T06:30:59.475 INFO:teuthology.orchestra.run.smithi138.stdout: "num_blocked_ops": 0 2024-09-14T06:30:59.475 INFO:teuthology.orchestra.run.smithi138.stdout:} 2024-09-14T06:30:59.484 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-14T06:30:59.525 INFO:tasks.ceph.osd.5.smithi138.stderr:2024-09-14T06:30:59.524+0000 7fe13746f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33618) UID: 0 2024-09-14T06:30:59.608 INFO:tasks.ceph.osd.6.smithi138.stderr:2024-09-14T06:30:59.607+0000 7f8cd8849640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33620) UID: 0 2024-09-14T06:30:59.625 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-14T06:30:59.630 INFO:teuthology.orchestra.run.smithi138.stdout:{ 2024-09-14T06:30:59.631 INFO:teuthology.orchestra.run.smithi138.stdout: "size": 20, 2024-09-14T06:30:59.631 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 600, 2024-09-14T06:30:59.631 INFO:teuthology.orchestra.run.smithi138.stdout: "ops": [ 2024-09-14T06:30:59.631 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:59.631 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:19328 3.3 3:c34e5e51:::rbd_data.112d92027590.0000000000000024:f [read 1926144~1024 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:30:59.631 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:27.059768+0000", 2024-09-14T06:30:59.631 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 92.568377249999998, 2024-09-14T06:30:59.631 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046369147999999999, 2024-09-14T06:30:59.632 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:30:59.632 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:30:59.632 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:30:59.632 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:30:59.632 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:30:59.632 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:59.632 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 19328 2024-09-14T06:30:59.632 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:59.633 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:30:59.633 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:59.633 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:30:59.633 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.059768+0000", 2024-09-14T06:30:59.633 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:59.633 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:59.633 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:59.633 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:30:59.634 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.059768+0000", 2024-09-14T06:30:59.634 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:59.634 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:59.634 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:59.634 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:30:59.634 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.059874+0000", 2024-09-14T06:30:59.634 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00010551400000000001 2024-09-14T06:30:59.634 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:59.635 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:59.635 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:30:59.635 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.059877+0000", 2024-09-14T06:30:59.635 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.8279999999999999e-06 2024-09-14T06:30:59.635 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:59.635 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:59.635 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:30:59.635 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.059884+0000", 2024-09-14T06:30:59.636 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.6389999999999992e-06 2024-09-14T06:30:59.636 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:59.636 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:59.636 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:30:59.636 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.059890+0000", 2024-09-14T06:30:59.636 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.49e-06 2024-09-14T06:30:59.636 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:59.637 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:59.637 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:30:59.637 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.105804+0000", 2024-09-14T06:30:59.637 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.045914242000000001 2024-09-14T06:30:59.637 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:59.637 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:59.637 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:30:59.637 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.105886+0000", 2024-09-14T06:30:59.638 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.2305999999999994e-05 2024-09-14T06:30:59.638 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:59.638 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:59.638 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:30:59.638 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.106138+0000", 2024-09-14T06:30:59.638 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.000251129 2024-09-14T06:30:59.638 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:59.638 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:30:59.639 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:59.639 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:59.639 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:59.639 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:19329 3.3 3:c34e5e51:::rbd_data.112d92027590.0000000000000024:f [read 1936384~1024 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:30:59.639 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:27.059899+0000", 2024-09-14T06:30:59.639 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 92.568246727000002, 2024-09-14T06:30:59.639 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046572122000000001, 2024-09-14T06:30:59.639 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:30:59.639 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:30:59.640 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:30:59.640 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:30:59.640 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:30:59.640 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:59.640 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 19329 2024-09-14T06:30:59.640 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:59.641 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:30:59.641 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:59.641 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:30:59.641 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.059899+0000", 2024-09-14T06:30:59.641 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:59.641 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:59.641 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:59.641 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:30:59.642 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.059899+0000", 2024-09-14T06:30:59.642 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:59.642 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:59.642 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:59.642 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:30:59.642 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062501+0000", 2024-09-14T06:30:59.642 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.0026016419999999999 2024-09-14T06:30:59.642 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:59.642 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:59.642 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:30:59.642 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062514+0000", 2024-09-14T06:30:59.642 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.2947e-05 2024-09-14T06:30:59.642 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:59.642 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:59.642 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:30:59.643 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062523+0000", 2024-09-14T06:30:59.643 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.5519999999999993e-06 2024-09-14T06:30:59.643 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:59.643 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:59.643 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:30:59.643 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062534+0000", 2024-09-14T06:30:59.643 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.1416e-05 2024-09-14T06:30:59.643 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:59.643 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:59.643 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:30:59.643 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.106179+0000", 2024-09-14T06:30:59.643 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.043644049999999997 2024-09-14T06:30:59.643 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:59.643 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:59.644 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:30:59.644 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.106262+0000", 2024-09-14T06:30:59.644 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.3178999999999998e-05 2024-09-14T06:30:59.644 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:59.644 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:59.644 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:30:59.644 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.106471+0000", 2024-09-14T06:30:59.644 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00020933599999999999 2024-09-14T06:30:59.644 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:59.644 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:30:59.644 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:59.645 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:59.645 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:59.645 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:19351 3.3 3:c34e5e51:::rbd_data.112d92027590.0000000000000024:f [read 1159168~1024 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:30:59.645 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:27.062766+0000", 2024-09-14T06:30:59.645 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 92.565379501999999, 2024-09-14T06:30:59.645 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046099454999999998, 2024-09-14T06:30:59.645 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:30:59.645 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:30:59.645 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:30:59.645 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:30:59.645 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:30:59.645 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:59.645 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 19351 2024-09-14T06:30:59.645 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:59.646 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:30:59.646 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:59.646 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:30:59.646 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062766+0000", 2024-09-14T06:30:59.646 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:59.646 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:59.646 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:59.646 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:30:59.646 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062766+0000", 2024-09-14T06:30:59.646 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:59.646 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:59.647 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:59.647 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:30:59.647 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062766+0000", 2024-09-14T06:30:59.647 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.6e-07 2024-09-14T06:30:59.647 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:59.647 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:59.647 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:30:59.647 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062767+0000", 2024-09-14T06:30:59.647 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.0070000000000001e-06 2024-09-14T06:30:59.647 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:59.647 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:59.647 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:30:59.647 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062785+0000", 2024-09-14T06:30:59.647 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.7884999999999999e-05 2024-09-14T06:30:59.648 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:59.648 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:59.648 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:30:59.648 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062790+0000", 2024-09-14T06:30:59.648 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.0900000000000004e-06 2024-09-14T06:30:59.648 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:59.648 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:59.648 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:30:59.648 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.108735+0000", 2024-09-14T06:30:59.648 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.045944411999999997 2024-09-14T06:30:59.648 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:59.648 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:59.648 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:30:59.649 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.108809+0000", 2024-09-14T06:30:59.649 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.4157000000000002e-05 2024-09-14T06:30:59.649 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:59.649 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:59.649 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:30:59.649 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.108866+0000", 2024-09-14T06:30:59.649 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.6644000000000002e-05 2024-09-14T06:30:59.649 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:59.649 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:30:59.649 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:59.649 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:59.649 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:59.650 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:19352 3.3 3:c34e5e51:::rbd_data.112d92027590.0000000000000024:f [read 1169408~1024 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:30:59.650 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:27.062798+0000", 2024-09-14T06:30:59.650 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 92.565348091000004, 2024-09-14T06:30:59.650 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046255900000000003, 2024-09-14T06:30:59.650 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:30:59.650 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:30:59.650 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:30:59.650 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:30:59.650 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:30:59.650 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:59.650 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 19352 2024-09-14T06:30:59.650 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:59.651 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:30:59.651 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:59.651 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:30:59.651 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062798+0000", 2024-09-14T06:30:59.651 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:59.651 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:59.651 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:59.651 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:30:59.651 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062798+0000", 2024-09-14T06:30:59.651 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:59.651 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:59.651 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:59.651 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:30:59.651 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062798+0000", 2024-09-14T06:30:59.651 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.37e-07 2024-09-14T06:30:59.652 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:59.652 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:59.652 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:30:59.652 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062800+0000", 2024-09-14T06:30:59.652 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.0269999999999998e-06 2024-09-14T06:30:59.652 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:59.652 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:59.652 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:30:59.652 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062806+0000", 2024-09-14T06:30:59.652 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.6690000000000003e-06 2024-09-14T06:30:59.652 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:59.652 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:59.652 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:30:59.653 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062810+0000", 2024-09-14T06:30:59.653 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.3449999999999999e-06 2024-09-14T06:30:59.653 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:59.653 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:59.653 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:30:59.653 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.108892+0000", 2024-09-14T06:30:59.653 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046081715000000002 2024-09-14T06:30:59.653 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:59.653 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:59.653 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:30:59.653 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.108940+0000", 2024-09-14T06:30:59.653 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.7815999999999998e-05 2024-09-14T06:30:59.653 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:59.654 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:59.654 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:30:59.654 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.109053+0000", 2024-09-14T06:30:59.654 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.000113791 2024-09-14T06:30:59.654 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:59.654 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:30:59.654 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:59.654 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:59.654 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:59.654 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:19353 3.3 3:c34e5e51:::rbd_data.112d92027590.0000000000000024:f [read 1179648~1024 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:30:59.654 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:27.062816+0000", 2024-09-14T06:30:59.654 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 92.565329571999996, 2024-09-14T06:30:59.654 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046481256999999998, 2024-09-14T06:30:59.654 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:30:59.655 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:30:59.655 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:30:59.655 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:30:59.655 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:30:59.655 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:59.655 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 19353 2024-09-14T06:30:59.655 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:59.655 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:30:59.655 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:59.655 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:30:59.655 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062816+0000", 2024-09-14T06:30:59.655 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:59.655 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:59.656 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:59.656 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:30:59.656 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062816+0000", 2024-09-14T06:30:59.656 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:59.656 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:59.656 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:59.656 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:30:59.656 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062816+0000", 2024-09-14T06:30:59.656 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.2899999999999999e-07 2024-09-14T06:30:59.656 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:59.657 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:59.657 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:30:59.657 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062818+0000", 2024-09-14T06:30:59.657 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.699e-06 2024-09-14T06:30:59.657 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:59.657 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:59.657 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:30:59.657 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062825+0000", 2024-09-14T06:30:59.657 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.878e-06 2024-09-14T06:30:59.657 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:59.657 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:59.657 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:30:59.657 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062830+0000", 2024-09-14T06:30:59.657 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.5410000000000002e-06 2024-09-14T06:30:59.658 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:59.658 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:59.658 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:30:59.658 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.109096+0000", 2024-09-14T06:30:59.658 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046266517 2024-09-14T06:30:59.658 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:59.658 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:59.658 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:30:59.658 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.109190+0000", 2024-09-14T06:30:59.658 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.4000999999999997e-05 2024-09-14T06:30:59.658 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:59.658 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:59.658 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:30:59.658 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.109297+0000", 2024-09-14T06:30:59.659 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.000107292 2024-09-14T06:30:59.659 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:59.659 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:30:59.659 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:59.659 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:59.659 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:59.659 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:19354 3.3 3:c34e5e51:::rbd_data.112d92027590.0000000000000024:f [read 1189888~1024 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:30:59.659 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:27.062842+0000", 2024-09-14T06:30:59.659 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 92.565304116999997, 2024-09-14T06:30:59.659 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046594257, 2024-09-14T06:30:59.659 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:30:59.659 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:30:59.659 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:30:59.660 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:30:59.660 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:30:59.660 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:59.660 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 19354 2024-09-14T06:30:59.660 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:59.660 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:30:59.660 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:59.660 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:30:59.660 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062842+0000", 2024-09-14T06:30:59.660 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:59.660 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:59.660 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:59.660 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:30:59.660 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062842+0000", 2024-09-14T06:30:59.661 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:59.661 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:59.661 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:59.661 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:30:59.661 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062842+0000", 2024-09-14T06:30:59.661 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.7399999999999999e-07 2024-09-14T06:30:59.661 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:59.661 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:59.661 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:30:59.661 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062859+0000", 2024-09-14T06:30:59.661 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.7010000000000001e-05 2024-09-14T06:30:59.661 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:59.661 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:59.661 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:30:59.662 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062866+0000", 2024-09-14T06:30:59.662 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.9990000000000002e-06 2024-09-14T06:30:59.662 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:59.662 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:59.662 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:30:59.662 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062873+0000", 2024-09-14T06:30:59.662 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.8569999999999997e-06 2024-09-14T06:30:59.662 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:59.662 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:59.662 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:30:59.662 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.109335+0000", 2024-09-14T06:30:59.662 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046461943999999998 2024-09-14T06:30:59.662 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:59.662 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:59.663 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:30:59.663 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.109377+0000", 2024-09-14T06:30:59.663 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.2558999999999997e-05 2024-09-14T06:30:59.663 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:59.663 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:59.663 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:30:59.663 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.109436+0000", 2024-09-14T06:30:59.663 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.8514000000000003e-05 2024-09-14T06:30:59.663 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:59.663 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:30:59.663 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:59.663 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:59.663 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:59.663 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:19355 3.3 3:c34e5e51:::rbd_data.112d92027590.0000000000000024:f [read 1200128~1024 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:30:59.664 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:27.062884+0000", 2024-09-14T06:30:59.664 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 92.565261444000001, 2024-09-14T06:30:59.664 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046681650999999998, 2024-09-14T06:30:59.664 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:30:59.664 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:30:59.664 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:30:59.664 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:30:59.664 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:30:59.664 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:59.664 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 19355 2024-09-14T06:30:59.664 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:59.664 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:30:59.664 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:59.664 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:30:59.665 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062884+0000", 2024-09-14T06:30:59.665 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:59.665 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:59.665 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:59.665 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:30:59.665 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062884+0000", 2024-09-14T06:30:59.665 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:59.665 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:59.665 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:59.665 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:30:59.665 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062885+0000", 2024-09-14T06:30:59.665 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.8000000000000001e-07 2024-09-14T06:30:59.665 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:59.665 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:59.666 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:30:59.666 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062886+0000", 2024-09-14T06:30:59.666 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.8920000000000001e-06 2024-09-14T06:30:59.666 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:59.666 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:59.666 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:30:59.666 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062892+0000", 2024-09-14T06:30:59.666 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.84e-06 2024-09-14T06:30:59.666 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:59.666 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:59.666 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:30:59.667 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062899+0000", 2024-09-14T06:30:59.667 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.5339999999999996e-06 2024-09-14T06:30:59.667 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:59.667 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:59.667 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:30:59.667 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.109466+0000", 2024-09-14T06:30:59.667 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046566919999999998 2024-09-14T06:30:59.667 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:59.667 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:59.667 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:30:59.667 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.109510+0000", 2024-09-14T06:30:59.667 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.4153000000000002e-05 2024-09-14T06:30:59.667 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:59.668 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:59.668 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:30:59.668 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.109566+0000", 2024-09-14T06:30:59.668 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.5931999999999999e-05 2024-09-14T06:30:59.668 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:59.668 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:30:59.668 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:59.668 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:59.668 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:59.668 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:19356 3.3 3:c34e5e51:::rbd_data.112d92027590.0000000000000024:f [read 1210368~1024 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:30:59.668 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:27.062908+0000", 2024-09-14T06:30:59.668 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 92.565238007000005, 2024-09-14T06:30:59.668 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046807498000000003, 2024-09-14T06:30:59.669 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:30:59.669 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:30:59.669 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:30:59.669 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:30:59.669 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:30:59.669 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:59.669 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 19356 2024-09-14T06:30:59.669 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:59.669 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:30:59.669 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:59.670 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:30:59.670 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062908+0000", 2024-09-14T06:30:59.670 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:59.670 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:59.670 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:59.670 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:30:59.670 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062908+0000", 2024-09-14T06:30:59.670 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:59.670 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:59.670 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:59.670 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:30:59.670 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062908+0000", 2024-09-14T06:30:59.670 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.5699999999999998e-07 2024-09-14T06:30:59.670 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:59.671 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:59.671 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:30:59.671 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062910+0000", 2024-09-14T06:30:59.671 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.7710000000000001e-06 2024-09-14T06:30:59.671 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:59.671 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:59.671 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:30:59.671 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062920+0000", 2024-09-14T06:30:59.671 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.0654e-05 2024-09-14T06:30:59.671 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:59.671 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:59.671 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:30:59.671 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062931+0000", 2024-09-14T06:30:59.671 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.075e-05 2024-09-14T06:30:59.672 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:59.672 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:59.672 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:30:59.672 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.109595+0000", 2024-09-14T06:30:59.672 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046663713000000002 2024-09-14T06:30:59.672 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:59.672 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:59.672 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:30:59.672 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.109632+0000", 2024-09-14T06:30:59.672 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.7505000000000001e-05 2024-09-14T06:30:59.672 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:59.672 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:59.672 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:30:59.673 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.109715+0000", 2024-09-14T06:30:59.673 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.2748000000000001e-05 2024-09-14T06:30:59.673 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:59.673 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:30:59.673 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:59.673 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:59.673 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:59.673 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:19357 3.3 3:c34e5e51:::rbd_data.112d92027590.0000000000000024:f [read 1220608~1024 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:30:59.673 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:27.062939+0000", 2024-09-14T06:30:59.673 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 92.565206559000003, 2024-09-14T06:30:59.673 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046920508, 2024-09-14T06:30:59.673 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:30:59.673 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:30:59.674 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:30:59.674 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:30:59.674 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:30:59.674 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:59.674 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 19357 2024-09-14T06:30:59.674 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:59.674 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:30:59.674 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:59.674 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:30:59.674 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062939+0000", 2024-09-14T06:30:59.674 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:59.674 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:59.674 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:59.674 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:30:59.674 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062939+0000", 2024-09-14T06:30:59.675 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:59.675 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:59.675 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:59.675 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:30:59.675 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062940+0000", 2024-09-14T06:30:59.675 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.32e-07 2024-09-14T06:30:59.675 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:59.675 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:59.675 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:30:59.675 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062942+0000", 2024-09-14T06:30:59.675 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.1919999999999999e-06 2024-09-14T06:30:59.675 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:59.675 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:59.675 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:30:59.675 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062948+0000", 2024-09-14T06:30:59.675 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.1149999999999999e-06 2024-09-14T06:30:59.676 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:59.676 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:59.676 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:30:59.676 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062960+0000", 2024-09-14T06:30:59.676 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.1895e-05 2024-09-14T06:30:59.676 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:59.676 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:59.676 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:30:59.676 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.109704+0000", 2024-09-14T06:30:59.676 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046744584999999998 2024-09-14T06:30:59.677 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:59.677 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:59.677 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:30:59.677 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.109777+0000", 2024-09-14T06:30:59.677 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.2282999999999999e-05 2024-09-14T06:30:59.677 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:59.677 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:59.677 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:30:59.677 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.109860+0000", 2024-09-14T06:30:59.677 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.3005999999999998e-05 2024-09-14T06:30:59.678 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:59.678 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:30:59.678 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:59.678 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:59.678 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:59.678 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:19358 3.3 3:c34e5e51:::rbd_data.112d92027590.0000000000000024:f [read 1230848~1024 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:30:59.678 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:27.062967+0000", 2024-09-14T06:30:59.678 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 92.565178928999998, 2024-09-14T06:30:59.678 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047144471, 2024-09-14T06:30:59.678 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:30:59.678 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:30:59.678 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:30:59.678 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:30:59.679 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:30:59.679 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:59.679 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 19358 2024-09-14T06:30:59.679 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:59.679 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:30:59.679 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:59.679 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:30:59.679 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062967+0000", 2024-09-14T06:30:59.679 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:59.679 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:59.680 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:59.680 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:30:59.680 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062967+0000", 2024-09-14T06:30:59.680 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:59.680 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:59.680 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:59.680 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:30:59.680 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062967+0000", 2024-09-14T06:30:59.680 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.89e-07 2024-09-14T06:30:59.680 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:59.680 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:59.680 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:30:59.680 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062969+0000", 2024-09-14T06:30:59.680 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.9929999999999998e-06 2024-09-14T06:30:59.680 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:59.681 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:59.681 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:30:59.681 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062975+0000", 2024-09-14T06:30:59.681 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.9030000000000001e-06 2024-09-14T06:30:59.681 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:59.681 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:59.681 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:30:59.681 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062979+0000", 2024-09-14T06:30:59.681 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.2250000000000002e-06 2024-09-14T06:30:59.681 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:59.681 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:59.681 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:30:59.681 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.109934+0000", 2024-09-14T06:30:59.682 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046954410000000002 2024-09-14T06:30:59.682 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:59.682 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:59.682 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:30:59.682 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.110007+0000", 2024-09-14T06:30:59.682 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.3829000000000001e-05 2024-09-14T06:30:59.682 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:59.682 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:59.682 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:30:59.682 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.110111+0000", 2024-09-14T06:30:59.682 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00010372199999999999 2024-09-14T06:30:59.682 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:59.682 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:30:59.682 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:59.683 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:59.683 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:59.683 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:19359 3.3 3:c34e5e51:::rbd_data.112d92027590.0000000000000024:f [read 1241088~1024 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:30:59.683 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:27.062987+0000", 2024-09-14T06:30:59.683 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 92.565158839000006, 2024-09-14T06:30:59.683 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047297182, 2024-09-14T06:30:59.683 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:30:59.683 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:30:59.683 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:30:59.683 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:30:59.683 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:30:59.683 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:59.683 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 19359 2024-09-14T06:30:59.684 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:59.684 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:30:59.684 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:59.684 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:30:59.684 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062987+0000", 2024-09-14T06:30:59.684 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:59.684 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:59.684 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:59.684 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:30:59.684 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062987+0000", 2024-09-14T06:30:59.684 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:59.684 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:59.684 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:59.685 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:30:59.685 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062987+0000", 2024-09-14T06:30:59.685 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.2500000000000001e-07 2024-09-14T06:30:59.685 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:59.685 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:59.685 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:30:59.685 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062989+0000", 2024-09-14T06:30:59.685 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.6080000000000001e-06 2024-09-14T06:30:59.685 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:59.686 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:59.686 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:30:59.686 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062994+0000", 2024-09-14T06:30:59.686 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.5520000000000004e-06 2024-09-14T06:30:59.686 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:59.686 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:59.686 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:30:59.687 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062999+0000", 2024-09-14T06:30:59.687 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.3930000000000001e-06 2024-09-14T06:30:59.687 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:59.687 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:59.687 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:30:59.687 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.110142+0000", 2024-09-14T06:30:59.687 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047143154 2024-09-14T06:30:59.687 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:59.687 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:59.687 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:30:59.687 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.110219+0000", 2024-09-14T06:30:59.688 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.6737999999999996e-05 2024-09-14T06:30:59.688 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:59.688 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:59.688 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:30:59.688 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.110284+0000", 2024-09-14T06:30:59.688 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.5412000000000003e-05 2024-09-14T06:30:59.688 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:59.688 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:30:59.688 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:59.688 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:59.688 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:59.688 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:19360 3.3 3:c34e5e51:::rbd_data.112d92027590.0000000000000024:f [read 1251328~1024 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:30:59.688 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:27.063017+0000", 2024-09-14T06:30:59.688 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 92.565128377999997, 2024-09-14T06:30:59.689 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047453823999999999, 2024-09-14T06:30:59.689 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:30:59.689 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:30:59.689 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:30:59.689 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:30:59.689 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:30:59.689 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:59.689 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 19360 2024-09-14T06:30:59.689 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:59.689 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:30:59.689 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:59.689 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:30:59.689 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.063017+0000", 2024-09-14T06:30:59.689 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:59.690 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:59.690 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:59.690 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:30:59.690 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.063017+0000", 2024-09-14T06:30:59.690 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:59.690 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:59.690 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:59.690 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:30:59.690 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.064977+0000", 2024-09-14T06:30:59.690 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.0019597080000000001 2024-09-14T06:30:59.690 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:59.690 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:59.690 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:30:59.690 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.064987+0000", 2024-09-14T06:30:59.691 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.9620000000000005e-06 2024-09-14T06:30:59.691 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:59.691 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:59.691 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:30:59.691 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065000+0000", 2024-09-14T06:30:59.691 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.3173000000000001e-05 2024-09-14T06:30:59.691 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:59.691 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:59.691 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:30:59.691 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065005+0000", 2024-09-14T06:30:59.691 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.9960000000000001e-06 2024-09-14T06:30:59.691 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:59.691 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:59.691 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:30:59.691 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.110314+0000", 2024-09-14T06:30:59.691 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.04530899 2024-09-14T06:30:59.692 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:59.692 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:59.692 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:30:59.692 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.110398+0000", 2024-09-14T06:30:59.692 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.3539000000000001e-05 2024-09-14T06:30:59.692 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:59.692 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:59.692 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:30:59.692 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.110471+0000", 2024-09-14T06:30:59.692 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.3455999999999996e-05 2024-09-14T06:30:59.692 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:59.692 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:30:59.692 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:59.692 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:59.693 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:59.693 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:19365 3.3 3:c34e5e51:::rbd_data.112d92027590.0000000000000024:f [read 1302528~1024 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:30:59.693 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:27.065058+0000", 2024-09-14T06:30:59.693 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 92.563087707999998, 2024-09-14T06:30:59.693 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046105526000000001, 2024-09-14T06:30:59.693 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:30:59.693 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:30:59.693 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:30:59.693 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:30:59.693 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:30:59.693 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:59.693 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 19365 2024-09-14T06:30:59.693 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:59.693 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:30:59.694 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:59.694 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:30:59.694 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065058+0000", 2024-09-14T06:30:59.694 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:59.694 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:59.694 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:59.694 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:30:59.694 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065058+0000", 2024-09-14T06:30:59.694 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:59.694 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:59.694 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:59.694 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:30:59.694 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065058+0000", 2024-09-14T06:30:59.694 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.1899999999999999e-07 2024-09-14T06:30:59.694 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:59.695 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:59.695 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:30:59.695 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065059+0000", 2024-09-14T06:30:59.695 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.1299999999999998e-07 2024-09-14T06:30:59.695 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:59.695 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:59.695 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:30:59.695 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065063+0000", 2024-09-14T06:30:59.695 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.1099999999999996e-06 2024-09-14T06:30:59.695 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:59.695 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:59.695 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:30:59.695 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065070+0000", 2024-09-14T06:30:59.695 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.973e-06 2024-09-14T06:30:59.695 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:59.695 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:59.696 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:30:59.696 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.111014+0000", 2024-09-14T06:30:59.696 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.045943373000000003 2024-09-14T06:30:59.696 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:59.696 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:59.696 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:30:59.696 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.111073+0000", 2024-09-14T06:30:59.696 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.8987999999999998e-05 2024-09-14T06:30:59.696 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:59.696 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:59.696 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:30:59.696 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.111163+0000", 2024-09-14T06:30:59.696 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.0950000000000002e-05 2024-09-14T06:30:59.697 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:59.697 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:30:59.697 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:59.697 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:59.697 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:59.697 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:19366 3.3 3:c34e5e51:::rbd_data.112d92027590.0000000000000024:f [read 1312768~1024 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:30:59.697 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:27.065074+0000", 2024-09-14T06:30:59.697 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 92.563071858000001, 2024-09-14T06:30:59.697 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046289858000000003, 2024-09-14T06:30:59.697 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:30:59.697 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:30:59.698 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:30:59.698 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:30:59.698 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:30:59.698 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:59.698 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 19366 2024-09-14T06:30:59.698 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:59.698 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:30:59.698 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:59.698 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:30:59.698 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065074+0000", 2024-09-14T06:30:59.698 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:59.698 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:59.699 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:59.699 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:30:59.699 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065074+0000", 2024-09-14T06:30:59.699 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:59.699 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:59.699 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:59.699 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:30:59.699 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065074+0000", 2024-09-14T06:30:59.699 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.3900000000000001e-07 2024-09-14T06:30:59.699 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:59.699 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:59.700 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:30:59.700 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065078+0000", 2024-09-14T06:30:59.700 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.3229999999999999e-06 2024-09-14T06:30:59.700 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:59.700 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:59.700 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:30:59.701 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065082+0000", 2024-09-14T06:30:59.701 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.8299999999999998e-06 2024-09-14T06:30:59.701 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:59.701 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:59.701 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:30:59.701 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065085+0000", 2024-09-14T06:30:59.701 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.8660000000000002e-06 2024-09-14T06:30:59.701 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:59.701 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:59.701 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:30:59.701 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.111270+0000", 2024-09-14T06:30:59.701 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046185035999999999 2024-09-14T06:30:59.701 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:59.702 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:59.702 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:30:59.702 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.111311+0000", 2024-09-14T06:30:59.702 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.0935e-05 2024-09-14T06:30:59.702 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:59.702 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:59.702 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:30:59.702 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.111364+0000", 2024-09-14T06:30:59.702 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.2629e-05 2024-09-14T06:30:59.702 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:59.702 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:30:59.702 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:59.702 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:59.703 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:59.703 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:19367 3.3 3:c34e5e51:::rbd_data.112d92027590.0000000000000024:f [read 1323008~1024 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:30:59.703 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:27.065115+0000", 2024-09-14T06:30:59.703 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 92.563030296999997, 2024-09-14T06:30:59.703 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046407003000000002, 2024-09-14T06:30:59.703 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:30:59.703 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:30:59.703 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:30:59.703 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:30:59.703 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:30:59.703 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:59.703 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 19367 2024-09-14T06:30:59.703 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:59.703 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:30:59.704 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:59.704 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:30:59.704 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065115+0000", 2024-09-14T06:30:59.704 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:59.704 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:59.704 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:59.704 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:30:59.704 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065115+0000", 2024-09-14T06:30:59.704 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:59.704 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:59.704 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:59.704 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:30:59.704 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065116+0000", 2024-09-14T06:30:59.704 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.4999999999999998e-07 2024-09-14T06:30:59.705 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:59.705 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:59.705 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:30:59.705 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065117+0000", 2024-09-14T06:30:59.705 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.189e-06 2024-09-14T06:30:59.705 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:59.705 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:59.705 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:30:59.705 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065122+0000", 2024-09-14T06:30:59.705 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.3709999999999999e-06 2024-09-14T06:30:59.706 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:59.706 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:59.706 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:30:59.706 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065125+0000", 2024-09-14T06:30:59.706 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.8940000000000002e-06 2024-09-14T06:30:59.706 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:59.706 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:59.706 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:30:59.706 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.111402+0000", 2024-09-14T06:30:59.706 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046276645999999998 2024-09-14T06:30:59.706 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:59.706 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:59.707 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:30:59.707 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.111445+0000", 2024-09-14T06:30:59.707 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.3288000000000003e-05 2024-09-14T06:30:59.707 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:59.707 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:59.707 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:30:59.707 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.111522+0000", 2024-09-14T06:30:59.707 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.7164999999999997e-05 2024-09-14T06:30:59.707 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:59.707 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:30:59.707 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:59.707 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:59.708 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:59.708 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:19368 3.3 3:c34e5e51:::rbd_data.112d92027590.0000000000000024:f [read 1333248~1024 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:30:59.708 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:27.065130+0000", 2024-09-14T06:30:59.708 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 92.563016007000002, 2024-09-14T06:30:59.708 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046507275000000001, 2024-09-14T06:30:59.708 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:30:59.708 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:30:59.708 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:30:59.708 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:30:59.708 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:30:59.708 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:59.709 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 19368 2024-09-14T06:30:59.709 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:59.709 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:30:59.709 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:59.709 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:30:59.709 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065130+0000", 2024-09-14T06:30:59.709 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:59.709 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:59.709 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:59.709 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:30:59.709 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065130+0000", 2024-09-14T06:30:59.709 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:59.709 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:59.710 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:59.710 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:30:59.710 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065130+0000", 2024-09-14T06:30:59.710 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.0800000000000001e-07 2024-09-14T06:30:59.710 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:59.710 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:59.710 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:30:59.710 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065131+0000", 2024-09-14T06:30:59.710 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.8700000000000004e-07 2024-09-14T06:30:59.710 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:59.710 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:59.710 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:30:59.710 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065134+0000", 2024-09-14T06:30:59.710 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.1039999999999998e-06 2024-09-14T06:30:59.711 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:59.711 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:59.711 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:30:59.711 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065136+0000", 2024-09-14T06:30:59.711 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.3949999999999999e-06 2024-09-14T06:30:59.711 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:59.711 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:59.711 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:30:59.711 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.111541+0000", 2024-09-14T06:30:59.711 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046404206000000003 2024-09-14T06:30:59.711 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:59.711 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:59.711 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:30:59.711 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.111582+0000", 2024-09-14T06:30:59.712 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.1051999999999999e-05 2024-09-14T06:30:59.712 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:59.712 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:59.712 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:30:59.712 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.111637+0000", 2024-09-14T06:30:59.712 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.5322999999999999e-05 2024-09-14T06:30:59.712 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:59.712 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:30:59.712 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:59.712 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:59.712 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:59.712 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:19369 3.3 3:c34e5e51:::rbd_data.112d92027590.0000000000000024:f [read 1343488~1024 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:30:59.712 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:27.065139+0000", 2024-09-14T06:30:59.712 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 92.563006189999996, 2024-09-14T06:30:59.713 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046682169000000003, 2024-09-14T06:30:59.713 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:30:59.713 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:30:59.713 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:30:59.713 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:30:59.713 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:30:59.713 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:59.713 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 19369 2024-09-14T06:30:59.713 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:59.713 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:30:59.713 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:59.713 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:30:59.713 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065139+0000", 2024-09-14T06:30:59.713 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:59.714 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:59.714 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:59.714 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:30:59.714 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065139+0000", 2024-09-14T06:30:59.714 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:59.714 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:59.714 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:59.714 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:30:59.714 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065140+0000", 2024-09-14T06:30:59.714 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.8299999999999998e-07 2024-09-14T06:30:59.714 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:59.714 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:59.714 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:30:59.714 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065141+0000", 2024-09-14T06:30:59.715 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.3900000000000003e-07 2024-09-14T06:30:59.715 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:59.715 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:59.715 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:30:59.715 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065144+0000", 2024-09-14T06:30:59.715 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.7409999999999998e-06 2024-09-14T06:30:59.715 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:59.715 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:59.715 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:30:59.715 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065147+0000", 2024-09-14T06:30:59.715 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.4150000000000002e-06 2024-09-14T06:30:59.715 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:59.715 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:59.715 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:30:59.715 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.111673+0000", 2024-09-14T06:30:59.716 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046526004000000003 2024-09-14T06:30:59.716 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:59.716 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:59.716 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:30:59.716 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.111749+0000", 2024-09-14T06:30:59.716 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.5741999999999994e-05 2024-09-14T06:30:59.716 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:59.716 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:59.716 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:30:59.716 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.111822+0000", 2024-09-14T06:30:59.716 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.3045000000000003e-05 2024-09-14T06:30:59.716 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:59.717 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:30:59.717 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:59.717 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:59.717 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:59.717 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:19370 3.3 3:c34e5e51:::rbd_data.112d92027590.0000000000000024:f [read 1353728~1024 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:30:59.717 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:27.065150+0000", 2024-09-14T06:30:59.717 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 92.562995461, 2024-09-14T06:30:59.717 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046897556999999999, 2024-09-14T06:30:59.717 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:30:59.718 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:30:59.718 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:30:59.718 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:30:59.718 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:30:59.718 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:59.718 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 19370 2024-09-14T06:30:59.718 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:59.718 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:30:59.718 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:59.718 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:30:59.718 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065150+0000", 2024-09-14T06:30:59.718 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:59.719 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:59.719 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:59.719 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:30:59.719 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065150+0000", 2024-09-14T06:30:59.719 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:59.719 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:59.719 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:59.719 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:30:59.719 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065150+0000", 2024-09-14T06:30:59.719 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.0200000000000001e-07 2024-09-14T06:30:59.719 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:59.719 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:59.720 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:30:59.720 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065151+0000", 2024-09-14T06:30:59.720 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.1100000000000004e-07 2024-09-14T06:30:59.720 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:59.720 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:59.720 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:30:59.720 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065154+0000", 2024-09-14T06:30:59.720 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.9639999999999999e-06 2024-09-14T06:30:59.720 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:59.720 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:59.720 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:30:59.720 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065157+0000", 2024-09-14T06:30:59.720 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.4260000000000002e-06 2024-09-14T06:30:59.720 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:59.721 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:59.721 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:30:59.721 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.111861+0000", 2024-09-14T06:30:59.721 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046704268 2024-09-14T06:30:59.721 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:59.721 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:59.721 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:30:59.721 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.111939+0000", 2024-09-14T06:30:59.721 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.8231000000000004e-05 2024-09-14T06:30:59.721 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:59.721 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:59.721 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:30:59.721 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.112048+0000", 2024-09-14T06:30:59.721 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00010855500000000001 2024-09-14T06:30:59.721 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:59.722 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:30:59.722 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:59.722 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:59.722 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:59.722 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:19371 3.3 3:c34e5e51:::rbd_data.112d92027590.0000000000000024:f [read 1363968~1024 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:30:59.722 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:27.065161+0000", 2024-09-14T06:30:59.722 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 92.562984811000007, 2024-09-14T06:30:59.722 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047021944000000003, 2024-09-14T06:30:59.722 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:30:59.722 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:30:59.722 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:30:59.722 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:30:59.722 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:30:59.722 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:59.723 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 19371 2024-09-14T06:30:59.723 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:59.723 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:30:59.723 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:59.723 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:30:59.723 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065161+0000", 2024-09-14T06:30:59.723 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:59.723 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:59.723 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:59.723 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:30:59.723 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065161+0000", 2024-09-14T06:30:59.723 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:59.723 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:59.723 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:59.724 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:30:59.724 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065161+0000", 2024-09-14T06:30:59.724 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.0200000000000001e-07 2024-09-14T06:30:59.724 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:59.724 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:59.724 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:30:59.724 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065166+0000", 2024-09-14T06:30:59.724 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.9869999999999999e-06 2024-09-14T06:30:59.724 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:59.724 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:59.724 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:30:59.724 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065169+0000", 2024-09-14T06:30:59.724 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.3799999999999998e-06 2024-09-14T06:30:59.725 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:59.725 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:59.725 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:30:59.725 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065172+0000", 2024-09-14T06:30:59.725 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.4830000000000002e-06 2024-09-14T06:30:59.725 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:59.725 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:59.725 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:30:59.725 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.112071+0000", 2024-09-14T06:30:59.725 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046899529000000002 2024-09-14T06:30:59.725 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:59.725 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:59.725 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:30:59.726 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.112128+0000", 2024-09-14T06:30:59.726 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.6341999999999997e-05 2024-09-14T06:30:59.726 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:59.726 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:59.726 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:30:59.726 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.112183+0000", 2024-09-14T06:30:59.726 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.5021000000000001e-05 2024-09-14T06:30:59.726 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:59.726 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:30:59.726 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:59.726 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:59.726 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:59.726 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:19372 3.3 3:c34e5e51:::rbd_data.112d92027590.0000000000000024:f [read 1374208~1024 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:30:59.726 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:27.065176+0000", 2024-09-14T06:30:59.726 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 92.562969660999997, 2024-09-14T06:30:59.727 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047188889999999997, 2024-09-14T06:30:59.727 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:30:59.727 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:30:59.727 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:30:59.727 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:30:59.727 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:30:59.727 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:30:59.727 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 19372 2024-09-14T06:30:59.727 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:59.727 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:30:59.727 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:59.727 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:30:59.727 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065176+0000", 2024-09-14T06:30:59.728 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:59.728 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:59.728 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:59.728 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:30:59.728 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065176+0000", 2024-09-14T06:30:59.728 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:30:59.728 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:59.728 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:59.728 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:30:59.728 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.071117+0000", 2024-09-14T06:30:59.728 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.0059404990000000001 2024-09-14T06:30:59.728 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:59.728 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:59.729 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:30:59.729 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.071120+0000", 2024-09-14T06:30:59.729 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.4340000000000001e-06 2024-09-14T06:30:59.729 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:59.729 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:59.729 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:30:59.729 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.071129+0000", 2024-09-14T06:30:59.729 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.8389999999999996e-06 2024-09-14T06:30:59.729 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:59.730 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:59.730 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:30:59.730 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.071134+0000", 2024-09-14T06:30:59.730 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.412e-06 2024-09-14T06:30:59.730 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:59.730 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:59.730 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:30:59.730 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.112221+0000", 2024-09-14T06:30:59.730 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.041086473999999998 2024-09-14T06:30:59.731 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:59.731 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:59.731 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:30:59.731 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.112299+0000", 2024-09-14T06:30:59.731 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.8746999999999997e-05 2024-09-14T06:30:59.731 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:30:59.731 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:30:59.731 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:30:59.731 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.112365+0000", 2024-09-14T06:30:59.731 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.5485000000000001e-05 2024-09-14T06:30:59.731 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:59.731 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:30:59.731 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:59.732 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:30:59.732 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:30:59.732 INFO:teuthology.orchestra.run.smithi138.stdout:} 2024-09-14T06:30:59.733 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-14T06:30:59.734 INFO:tasks.ceph.osd.1.smithi094.stderr:2024-09-14T06:30:59.732+0000 7f328e47a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33641) UID: 0 2024-09-14T06:30:59.746 INFO:teuthology.orchestra.run.smithi094.stdout:osd.0: {} 2024-09-14T06:30:59.746 INFO:teuthology.orchestra.run.smithi094.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-14T06:30:59.762 INFO:teuthology.orchestra.run.smithi094.stdout:osd.1: {} 2024-09-14T06:30:59.763 INFO:teuthology.orchestra.run.smithi094.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-14T06:30:59.772 INFO:teuthology.orchestra.run.smithi094.stderr:reweighted osd.0 to 1 (10000) 2024-09-14T06:30:59.781 INFO:teuthology.orchestra.run.smithi094.stdout:osd.2: {} 2024-09-14T06:30:59.781 INFO:teuthology.orchestra.run.smithi094.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-14T06:30:59.787 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-14T06:30:59.788 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 1 1 2024-09-14T06:30:59.811 INFO:teuthology.orchestra.run.smithi094.stdout:osd.3: {} 2024-09-14T06:30:59.812 INFO:teuthology.orchestra.run.smithi094.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-14T06:30:59.834 INFO:tasks.ceph.osd.5.smithi138.stderr:2024-09-14T06:30:59.833+0000 7fe13746f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33618) UID: 0 2024-09-14T06:30:59.845 INFO:teuthology.orchestra.run.smithi094.stdout:osd.4: {} 2024-09-14T06:30:59.845 INFO:teuthology.orchestra.run.smithi094.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-14T06:30:59.874 INFO:teuthology.orchestra.run.smithi094.stdout:osd.5: {} 2024-09-14T06:30:59.875 INFO:teuthology.orchestra.run.smithi094.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-14T06:30:59.895 INFO:teuthology.orchestra.run.smithi094.stdout:osd.6: {} 2024-09-14T06:30:59.896 INFO:teuthology.orchestra.run.smithi094.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-14T06:30:59.905 INFO:teuthology.orchestra.run.smithi138.stdout:ERROR: (22) Invalid argument 2024-09-14T06:30:59.905 INFO:teuthology.orchestra.run.smithi138.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:30:59.917 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-14T06:30:59.926 INFO:teuthology.orchestra.run.smithi094.stdout:osd.7: {} 2024-09-14T06:30:59.926 INFO:teuthology.orchestra.run.smithi094.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-14T06:30:59.958 INFO:tasks.ceph.osd.4.smithi138.stderr:2024-09-14T06:30:59.957+0000 7f6c653c8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33615) UID: 0 2024-09-14T06:31:00.036 INFO:tasks.ceph.osd.6.smithi138.stderr:2024-09-14T06:31:00.034+0000 7f8cd8849640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33620) UID: 0 2024-09-14T06:31:00.087 INFO:teuthology.orchestra.run.smithi138.stdout:ERROR: (22) Invalid argument 2024-09-14T06:31:00.087 INFO:teuthology.orchestra.run.smithi138.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:31:00.098 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-14T06:31:00.137 INFO:tasks.ceph.osd.1.smithi094.stderr:2024-09-14T06:31:00.136+0000 7f328e47a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33641) UID: 0 2024-09-14T06:31:00.240 INFO:tasks.ceph.osd.0.smithi094.stderr:2024-09-14T06:31:00.239+0000 7f07c9333640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33636) UID: 0 2024-09-14T06:31:00.264 INFO:teuthology.orchestra.run.smithi138.stdout:ERROR: (22) Invalid argument 2024-09-14T06:31:00.264 INFO:teuthology.orchestra.run.smithi138.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:31:00.273 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-14T06:31:00.340 INFO:tasks.ceph.osd.0.smithi094.stderr:2024-09-14T06:31:00.339+0000 7f07c9333640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33636) UID: 0 2024-09-14T06:31:00.392 INFO:teuthology.orchestra.run.smithi138.stdout:ERROR: (22) Invalid argument 2024-09-14T06:31:00.392 INFO:teuthology.orchestra.run.smithi138.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:31:00.403 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-14T06:31:00.446 INFO:tasks.ceph.osd.4.smithi138.stderr:2024-09-14T06:31:00.445+0000 7f6c653c8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33615) UID: 0 2024-09-14T06:31:00.541 INFO:tasks.ceph.osd.3.smithi094.stderr:2024-09-14T06:31:00.540+0000 7feeeafe8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33655) UID: 0 2024-09-14T06:31:00.553 INFO:teuthology.orchestra.run.smithi138.stdout:ERROR: (22) Invalid argument 2024-09-14T06:31:00.553 INFO:teuthology.orchestra.run.smithi138.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:31:00.566 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-14T06:31:00.591 INFO:teuthology.orchestra.run.smithi094.stderr:reweighted osd.1 to 1 (10000) 2024-09-14T06:31:00.591 INFO:teuthology.orchestra.run.smithi094.stderr:noscrub is set 2024-09-14T06:31:00.610 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 2 1 2024-09-14T06:31:00.650 INFO:tasks.ceph.osd.5.smithi138.stderr:2024-09-14T06:31:00.649+0000 7fe13746f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33618) UID: 0 2024-09-14T06:31:00.743 INFO:tasks.ceph.osd.1.smithi094.stderr:2024-09-14T06:31:00.741+0000 7f328e47a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33641) UID: 0 2024-09-14T06:31:00.756 INFO:teuthology.orchestra.run.smithi138.stdout:ERROR: (22) Invalid argument 2024-09-14T06:31:00.756 INFO:teuthology.orchestra.run.smithi138.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:31:00.769 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-14T06:31:00.844 INFO:tasks.ceph.osd.2.smithi094.stderr:2024-09-14T06:31:00.843+0000 7faf3269b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33649) UID: 0 2024-09-14T06:31:00.913 INFO:teuthology.orchestra.run.smithi138.stdout:ERROR: (22) Invalid argument 2024-09-14T06:31:00.913 INFO:teuthology.orchestra.run.smithi138.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:31:00.922 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-14T06:31:00.965 INFO:tasks.ceph.osd.6.smithi138.stderr:2024-09-14T06:31:00.964+0000 7f8cd8849640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33620) UID: 0 2024-09-14T06:31:01.025 INFO:teuthology.orchestra.run.smithi138.stdout:ERROR: (22) Invalid argument 2024-09-14T06:31:01.025 INFO:teuthology.orchestra.run.smithi138.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:31:01.034 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-14T06:31:01.076 INFO:tasks.ceph.osd.6.smithi138.stderr:2024-09-14T06:31:01.074+0000 7f8cd8849640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33620) UID: 0 2024-09-14T06:31:01.147 INFO:tasks.ceph.osd.1.smithi094.stderr:2024-09-14T06:31:01.147+0000 7f328e47a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33641) UID: 0 2024-09-14T06:31:01.177 INFO:teuthology.orchestra.run.smithi138.stdout:ERROR: (22) Invalid argument 2024-09-14T06:31:01.178 INFO:teuthology.orchestra.run.smithi138.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:31:01.186 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-14T06:31:01.248 INFO:tasks.ceph.osd.0.smithi094.stderr:2024-09-14T06:31:01.247+0000 7f07c9333640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33636) UID: 0 2024-09-14T06:31:01.290 INFO:teuthology.orchestra.run.smithi094.stdout:ERROR: (22) Invalid argument 2024-09-14T06:31:01.290 INFO:teuthology.orchestra.run.smithi094.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:31:01.299 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-14T06:31:01.383 INFO:tasks.ceph.osd.3.smithi094.stderr:2024-09-14T06:31:01.382+0000 7feeeafe8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33655) UID: 0 2024-09-14T06:31:01.442 INFO:teuthology.orchestra.run.smithi094.stdout:ERROR: (22) Invalid argument 2024-09-14T06:31:01.442 INFO:teuthology.orchestra.run.smithi094.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:31:01.450 INFO:tasks.ceph.osd.4.smithi138.stderr:2024-09-14T06:31:01.449+0000 7f6c653c8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33615) UID: 0 2024-09-14T06:31:01.451 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-14T06:31:01.550 INFO:tasks.ceph.osd.3.smithi094.stderr:2024-09-14T06:31:01.549+0000 7feeeafe8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33655) UID: 0 2024-09-14T06:31:01.595 INFO:teuthology.orchestra.run.smithi094.stdout:ERROR: (22) Invalid argument 2024-09-14T06:31:01.595 INFO:teuthology.orchestra.run.smithi094.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:31:01.603 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-14T06:31:01.651 INFO:tasks.ceph.osd.6.smithi138.stderr:2024-09-14T06:31:01.650+0000 7f8cd8849640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33620) UID: 0 2024-09-14T06:31:01.748 INFO:teuthology.orchestra.run.smithi094.stdout:ERROR: (22) Invalid argument 2024-09-14T06:31:01.748 INFO:teuthology.orchestra.run.smithi094.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:31:01.751 INFO:tasks.ceph.osd.3.smithi094.stderr:2024-09-14T06:31:01.750+0000 7feeeafe8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33655) UID: 0 2024-09-14T06:31:01.757 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-14T06:31:01.779 INFO:teuthology.orchestra.run.smithi094.stderr:reweighted osd.2 to 1 (10000) 2024-09-14T06:31:01.795 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 3 1 2024-09-14T06:31:01.853 INFO:tasks.ceph.osd.6.smithi138.stderr:2024-09-14T06:31:01.852+0000 7f8cd8849640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33620) UID: 0 2024-09-14T06:31:01.890 INFO:teuthology.orchestra.run.smithi094.stdout:ERROR: (22) Invalid argument 2024-09-14T06:31:01.890 INFO:teuthology.orchestra.run.smithi094.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:31:01.900 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-14T06:31:01.945 INFO:teuthology.orchestra.run.smithi094.stdout:checking clone #29, image image_client.0-clone29 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent30 2024-09-14T06:31:01.947 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-14T06:31:01.954 INFO:tasks.ceph.osd.5.smithi138.stderr:2024-09-14T06:31:01.953+0000 7fe13746f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33618) UID: 0 2024-09-14T06:31:02.041 INFO:teuthology.orchestra.run.smithi094.stdout:ERROR: (22) Invalid argument 2024-09-14T06:31:02.041 INFO:teuthology.orchestra.run.smithi094.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:31:02.050 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-14T06:31:02.054 INFO:tasks.ceph.osd.6.smithi138.stderr:2024-09-14T06:31:02.054+0000 7f8cd8849640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33620) UID: 0 2024-09-14T06:31:02.104 INFO:teuthology.orchestra.run.smithi094.stdout:osd.0: {} 2024-09-14T06:31:02.104 INFO:teuthology.orchestra.run.smithi094.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-14T06:31:02.123 INFO:teuthology.orchestra.run.smithi094.stdout:osd.1: {} 2024-09-14T06:31:02.123 INFO:teuthology.orchestra.run.smithi094.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-14T06:31:02.139 INFO:teuthology.orchestra.run.smithi094.stdout:osd.2: {} 2024-09-14T06:31:02.139 INFO:teuthology.orchestra.run.smithi094.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-14T06:31:02.145 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-14T06:31:02.145 INFO:teuthology.orchestra.run.smithi094.stdout:3881 write 0x70862b0 thru 0x708d8b7 (0x7608 bytes) 2024-09-14T06:31:02.155 INFO:tasks.ceph.osd.1.smithi094.stderr:2024-09-14T06:31:02.154+0000 7f328e47a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33641) UID: 0 2024-09-14T06:31:02.158 INFO:teuthology.orchestra.run.smithi094.stdout:3883 write 0x7dd513b thru 0x7de4bb8 (0xfa7e bytes) 2024-09-14T06:31:02.161 INFO:teuthology.orchestra.run.smithi094.stdout:osd.3: {} 2024-09-14T06:31:02.161 INFO:teuthology.orchestra.run.smithi094.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-14T06:31:02.170 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-14T06:31:02.170 INFO:teuthology.orchestra.run.smithi094.stdout: "ops": [], 2024-09-14T06:31:02.170 INFO:teuthology.orchestra.run.smithi094.stdout: "num_ops": 0 2024-09-14T06:31:02.170 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-14T06:31:02.174 INFO:teuthology.orchestra.run.smithi094.stdout:3884 trunc from 0xe1917e7 to 0x5cbe832 2024-09-14T06:31:02.186 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-14T06:31:02.190 INFO:teuthology.orchestra.run.smithi094.stdout:osd.4: {} 2024-09-14T06:31:02.190 INFO:teuthology.orchestra.run.smithi094.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-14T06:31:02.216 INFO:teuthology.orchestra.run.smithi094.stdout:osd.5: {} 2024-09-14T06:31:02.216 INFO:teuthology.orchestra.run.smithi094.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-14T06:31:02.231 INFO:teuthology.orchestra.run.smithi094.stdout:3885 read 0x5a88fd6 thru 0x5a89f15 (0xf40 bytes) 2024-09-14T06:31:02.237 INFO:teuthology.orchestra.run.smithi094.stdout:osd.6: {} 2024-09-14T06:31:02.237 INFO:teuthology.orchestra.run.smithi094.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-14T06:31:02.246 INFO:teuthology.orchestra.run.smithi094.stdout:3886 write 0xd9ca7c7 thru 0xd9cc259 (0x1a93 bytes) 2024-09-14T06:31:02.250 INFO:teuthology.orchestra.run.smithi094.stdout:3888 write 0x9073f1 thru 0x9140c9 (0xccd9 bytes) 2024-09-14T06:31:02.254 INFO:teuthology.orchestra.run.smithi094.stdout:3889 punch from 0xd9be09a to 0xd9caa04, (0xc96a bytes) 2024-09-14T06:31:02.256 INFO:teuthology.orchestra.run.smithi094.stdout:osd.7: {} 2024-09-14T06:31:02.257 INFO:teuthology.orchestra.run.smithi094.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-14T06:31:02.257 INFO:tasks.ceph.osd.0.smithi094.stderr:2024-09-14T06:31:02.255+0000 7f07c9333640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33636) UID: 0 2024-09-14T06:31:02.270 INFO:teuthology.orchestra.run.smithi094.stdout:3891 trunc from 0xd9cc25a to 0x7733a8 2024-09-14T06:31:02.315 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-14T06:31:02.315 INFO:teuthology.orchestra.run.smithi094.stdout:3894 read 0x672ba2 thru 0x67a818 (0x7c77 bytes) 2024-09-14T06:31:02.317 INFO:teuthology.orchestra.run.smithi094.stdout:3895 trunc from 0x7733a8 to 0x1dd4dd5 2024-09-14T06:31:02.326 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-14T06:31:02.326 INFO:teuthology.orchestra.run.smithi094.stdout: "ops": [], 2024-09-14T06:31:02.326 INFO:teuthology.orchestra.run.smithi094.stdout: "complaint_time": 30, 2024-09-14T06:31:02.326 INFO:teuthology.orchestra.run.smithi094.stdout: "num_blocked_ops": 0 2024-09-14T06:31:02.326 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-14T06:31:02.329 INFO:teuthology.orchestra.run.smithi094.stdout:3897 writesame 0x30d8e06 thru 0x30e4e29 data_size 0x6(0xc024 bytes) 2024-09-14T06:31:02.330 INFO:teuthology.orchestra.run.smithi094.stdout:3898 read 0xe343aa thru 0xe40739 (0xc390 bytes) 2024-09-14T06:31:02.330 INFO:teuthology.orchestra.run.smithi094.stdout:3899 read 0x2058d2c thru 0x2068b24 (0xfdf9 bytes) 2024-09-14T06:31:02.330 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-14T06:31:02.337 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-14T06:31:02.339 INFO:teuthology.orchestra.run.smithi094.stdout:3901 writesame 0x6677667 thru 0x667a356 data_size 0x8(0x2cf0 bytes) 2024-09-14T06:31:02.339 INFO:teuthology.orchestra.run.smithi094.stdout:3902 punch from 0x5ac344c to 0x5ac4680, (0x1234 bytes) 2024-09-14T06:31:02.347 INFO:teuthology.orchestra.run.smithi094.stdout:3903 writesame 0x875688d thru 0x875b20f data_size 0x9(0x4983 bytes) 2024-09-14T06:31:02.357 INFO:teuthology.orchestra.run.smithi094.stdout:3904 write 0xcd3289f thru 0xcd36695 (0x3df7 bytes) 2024-09-14T06:31:02.357 INFO:tasks.ceph.osd.2.smithi094.stderr:2024-09-14T06:31:02.356+0000 7faf3269b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33649) UID: 0 2024-09-14T06:31:02.362 INFO:teuthology.orchestra.run.smithi094.stdout:3905 write 0x962f18e thru 0x963565c (0x64cf bytes) 2024-09-14T06:31:02.372 INFO:teuthology.orchestra.run.smithi094.stdout:3906 read 0xcae5da6 thru 0xcae6755 (0x9b0 bytes) 2024-09-14T06:31:02.372 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-14T06:31:02.372 INFO:teuthology.orchestra.run.smithi094.stdout:3911 writesame 0xa75f356 thru 0xa765e5b data_size 0x26(0x6b06 bytes) 2024-09-14T06:31:02.378 INFO:teuthology.orchestra.run.smithi094.stdout:3912 write 0x178aec5 thru 0x17954d7 (0xa613 bytes) 2024-09-14T06:31:02.387 INFO:teuthology.orchestra.run.smithi094.stdout:3913 punch from 0xc40274b to 0xc405f93, (0x3848 bytes) 2024-09-14T06:31:02.400 INFO:teuthology.orchestra.run.smithi094.stdout:3915 trunc from 0xcd36696 to 0x299015e 2024-09-14T06:31:02.426 INFO:teuthology.orchestra.run.smithi094.stdout:3916 write 0x72ae318 thru 0x72aece8 (0x9d1 bytes) 2024-09-14T06:31:02.443 INFO:teuthology.orchestra.run.smithi094.stdout:3917 write 0xe6518cd thru 0xe65c44b (0xab7f bytes) 2024-09-14T06:31:02.451 INFO:teuthology.orchestra.run.smithi094.stdout:3918 read 0xbe4a024 thru 0xbe4de7d (0x3e5a bytes) 2024-09-14T06:31:02.451 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-14T06:31:02.451 INFO:teuthology.orchestra.run.smithi094.stdout:3920 writesame 0x21186b5 thru 0x21262c8 data_size 0x5a(0xdc14 bytes) 2024-09-14T06:31:02.458 INFO:tasks.ceph.osd.7.smithi138.stderr:2024-09-14T06:31:02.457+0000 7f3c6a21d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33623) UID: 0 2024-09-14T06:31:02.465 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-14T06:31:02.465 INFO:teuthology.orchestra.run.smithi094.stdout: "size": 20, 2024-09-14T06:31:02.465 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 600, 2024-09-14T06:31:02.465 INFO:teuthology.orchestra.run.smithi094.stdout: "ops": [ 2024-09-14T06:31:02.466 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:02.466 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4397.0:26415 3.0 e53/23 3:05fd9be0:::rbd_data.112d38768680.0000000000000006:head v 53'21, mlcod=50'19)", 2024-09-14T06:31:02.466 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:30.721610+0000", 2024-09-14T06:31:02.466 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 91.741873502000004, 2024-09-14T06:31:02.466 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0021763160000000002, 2024-09-14T06:31:02.466 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:31:02.466 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:31:02.466 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:31:02.466 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:31:02.466 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:02.466 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:31:02.466 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:30.721610+0000", 2024-09-14T06:31:02.466 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:02.466 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:02.467 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:02.467 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:31:02.467 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:30.721610+0000", 2024-09-14T06:31:02.467 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:02.467 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:02.467 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:02.467 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:31:02.467 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:30.721611+0000", 2024-09-14T06:31:02.467 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.41e-07 2024-09-14T06:31:02.467 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:02.467 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:02.467 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:31:02.467 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:30.721905+0000", 2024-09-14T06:31:02.467 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00029394300000000001 2024-09-14T06:31:02.468 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:02.468 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:02.468 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:31:02.468 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:30.721921+0000", 2024-09-14T06:31:02.468 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.6609999999999999e-05 2024-09-14T06:31:02.468 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:02.468 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:02.468 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:31:02.468 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:30.721931+0000", 2024-09-14T06:31:02.468 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.6739999999999992e-06 2024-09-14T06:31:02.468 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:02.468 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:02.468 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:31:02.468 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:30.722001+0000", 2024-09-14T06:31:02.469 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.0549000000000005e-05 2024-09-14T06:31:02.469 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:02.469 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:02.469 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:31:02.469 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:30.722029+0000", 2024-09-14T06:31:02.469 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.7327000000000001e-05 2024-09-14T06:31:02.469 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:02.469 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:02.469 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-14T06:31:02.469 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:30.723727+0000", 2024-09-14T06:31:02.469 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0016977310000000001 2024-09-14T06:31:02.469 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:02.469 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:02.470 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:31:02.470 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:30.723787+0000", 2024-09-14T06:31:02.470 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.0140999999999999e-05 2024-09-14T06:31:02.470 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:02.470 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:31:02.470 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:02.470 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:02.470 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:02.470 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4397.0:36084 3.0 e56/23 3:05a48ef4:::rbd_data.112dc58639fa.0000000000000058:head v 56'29, mlcod=56'27)", 2024-09-14T06:31:02.470 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:34.508855+0000", 2024-09-14T06:31:02.470 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 87.954628748000005, 2024-09-14T06:31:02.470 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0020801140000000001, 2024-09-14T06:31:02.470 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:31:02.471 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:31:02.471 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:31:02.471 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:31:02.471 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:02.471 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:31:02.471 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:34.508855+0000", 2024-09-14T06:31:02.471 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:02.471 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:02.471 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:02.471 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:31:02.471 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:34.508855+0000", 2024-09-14T06:31:02.471 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:02.471 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:02.471 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:02.472 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:31:02.472 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:34.508856+0000", 2024-09-14T06:31:02.472 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.1909999999999999e-06 2024-09-14T06:31:02.472 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:02.472 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:02.472 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:31:02.472 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:34.509203+0000", 2024-09-14T06:31:02.472 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00034625900000000002 2024-09-14T06:31:02.472 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:02.472 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:02.472 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:31:02.472 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:34.509218+0000", 2024-09-14T06:31:02.472 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.5091e-05 2024-09-14T06:31:02.473 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:02.473 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:02.473 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:31:02.473 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:34.509227+0000", 2024-09-14T06:31:02.473 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.0850000000000003e-06 2024-09-14T06:31:02.473 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:02.473 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:02.473 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:31:02.473 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:34.509311+0000", 2024-09-14T06:31:02.473 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.4436000000000003e-05 2024-09-14T06:31:02.473 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:02.473 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:02.473 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:31:02.474 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:34.509335+0000", 2024-09-14T06:31:02.474 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.3577000000000001e-05 2024-09-14T06:31:02.474 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:02.474 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:02.474 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-14T06:31:02.474 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:34.510890+0000", 2024-09-14T06:31:02.474 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0015554259999999999 2024-09-14T06:31:02.474 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:02.474 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:02.474 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:31:02.474 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:34.510935+0000", 2024-09-14T06:31:02.474 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.5049000000000002e-05 2024-09-14T06:31:02.474 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:02.475 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:31:02.475 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:02.475 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:02.475 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:02.475 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4397.0:45722 3.9 e64/23 3:91fed098:::rbd_data.112d5a737e72.0000000000000001:head v 64'61, mlcod=62'59)", 2024-09-14T06:31:02.475 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:43.864269+0000", 2024-09-14T06:31:02.475 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 78.599214673000006, 2024-09-14T06:31:02.475 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.003055224, 2024-09-14T06:31:02.475 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:31:02.475 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:31:02.475 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:31:02.475 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:31:02.475 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:02.475 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:31:02.476 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.864269+0000", 2024-09-14T06:31:02.476 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:02.476 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:02.476 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:02.476 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:31:02.476 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.864269+0000", 2024-09-14T06:31:02.476 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:02.476 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:02.476 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:02.476 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:31:02.476 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.864269+0000", 2024-09-14T06:31:02.476 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.41e-07 2024-09-14T06:31:02.477 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:02.477 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:02.477 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:31:02.477 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.865490+0000", 2024-09-14T06:31:02.477 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.001220552 2024-09-14T06:31:02.477 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:02.477 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:02.477 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:31:02.477 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.865506+0000", 2024-09-14T06:31:02.477 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.6263999999999999e-05 2024-09-14T06:31:02.477 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:02.477 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:02.477 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:31:02.477 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.865516+0000", 2024-09-14T06:31:02.478 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.0091999999999999e-05 2024-09-14T06:31:02.478 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:02.478 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:02.478 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:31:02.478 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.865583+0000", 2024-09-14T06:31:02.478 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.6676999999999999e-05 2024-09-14T06:31:02.478 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:02.478 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:02.478 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:31:02.478 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.865610+0000", 2024-09-14T06:31:02.478 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.6529999999999998e-05 2024-09-14T06:31:02.478 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:02.478 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:02.479 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-14T06:31:02.479 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.867281+0000", 2024-09-14T06:31:02.479 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.001671203 2024-09-14T06:31:02.479 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:02.479 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:02.479 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:31:02.479 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.867324+0000", 2024-09-14T06:31:02.479 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.3665000000000003e-05 2024-09-14T06:31:02.479 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:02.479 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:31:02.479 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:02.479 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:02.479 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:02.479 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4397.0:50006 3.4 e64/23 3:20cfe300:::rbd_data.112d5a737e72.0000000000000019:head v 64'70, mlcod=61'68)", 2024-09-14T06:31:02.479 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:44.224218+0000", 2024-09-14T06:31:02.480 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 78.239265476, 2024-09-14T06:31:02.480 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0021104320000000002, 2024-09-14T06:31:02.480 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:31:02.480 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:31:02.480 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:31:02.480 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:31:02.480 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:02.480 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:31:02.480 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.224218+0000", 2024-09-14T06:31:02.480 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:02.480 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:02.480 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:02.480 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:31:02.480 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.224218+0000", 2024-09-14T06:31:02.481 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:02.481 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:02.481 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:02.481 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:31:02.481 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.224219+0000", 2024-09-14T06:31:02.481 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.0049999999999999e-06 2024-09-14T06:31:02.481 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:02.481 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:02.481 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:31:02.481 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.224326+0000", 2024-09-14T06:31:02.481 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000106848 2024-09-14T06:31:02.481 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:02.481 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:02.481 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:31:02.482 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.224345+0000", 2024-09-14T06:31:02.482 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.8587999999999999e-05 2024-09-14T06:31:02.482 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:02.482 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:02.482 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:31:02.482 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.224354+0000", 2024-09-14T06:31:02.482 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.7220000000000005e-06 2024-09-14T06:31:02.482 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:02.482 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:02.482 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:31:02.482 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.224416+0000", 2024-09-14T06:31:02.482 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.2738999999999997e-05 2024-09-14T06:31:02.482 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:02.482 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:02.482 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:31:02.482 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.224441+0000", 2024-09-14T06:31:02.483 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.4408000000000001e-05 2024-09-14T06:31:02.483 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:02.483 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:02.483 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-14T06:31:02.483 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.226295+0000", 2024-09-14T06:31:02.483 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00185428 2024-09-14T06:31:02.483 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:02.483 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:02.483 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:31:02.483 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.226329+0000", 2024-09-14T06:31:02.483 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.3841999999999999e-05 2024-09-14T06:31:02.483 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:02.483 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:31:02.484 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:02.484 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:02.484 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:02.484 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4397.0:74241 3.0 e73/23 3:06e93fb9:::rbd_data.112d80ca42af.0000000000000000:head v 73'102, mlcod=71'100)", 2024-09-14T06:31:02.484 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:53.876730+0000", 2024-09-14T06:31:02.484 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 68.586753572999996, 2024-09-14T06:31:02.484 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0079470329999999992, 2024-09-14T06:31:02.484 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:31:02.484 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:31:02.484 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:31:02.484 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:31:02.484 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:02.485 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:31:02.485 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:53.876730+0000", 2024-09-14T06:31:02.485 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:02.485 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:02.485 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:02.485 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:31:02.485 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:53.876730+0000", 2024-09-14T06:31:02.485 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:02.485 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:02.485 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:02.485 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:31:02.485 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:53.876731+0000", 2024-09-14T06:31:02.485 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.0500000000000004e-07 2024-09-14T06:31:02.486 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:02.486 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:02.486 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:31:02.486 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:53.879616+0000", 2024-09-14T06:31:02.486 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0028856680000000001 2024-09-14T06:31:02.486 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:02.486 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:02.486 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:31:02.486 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:53.879643+0000", 2024-09-14T06:31:02.486 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.6120999999999999e-05 2024-09-14T06:31:02.486 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:02.486 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:02.486 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:31:02.487 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:53.879654+0000", 2024-09-14T06:31:02.487 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.1586999999999999e-05 2024-09-14T06:31:02.487 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:02.487 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:02.487 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:31:02.487 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:53.879756+0000", 2024-09-14T06:31:02.487 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00010223899999999999 2024-09-14T06:31:02.487 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:02.487 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:02.487 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:31:02.487 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:53.879783+0000", 2024-09-14T06:31:02.487 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.6913000000000001e-05 2024-09-14T06:31:02.487 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:02.487 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:02.488 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-14T06:31:02.488 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:53.884594+0000", 2024-09-14T06:31:02.488 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0048110729999999999 2024-09-14T06:31:02.488 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:02.488 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:02.488 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:31:02.488 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:53.884677+0000", 2024-09-14T06:31:02.488 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.2927000000000001e-05 2024-09-14T06:31:02.488 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:02.488 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:31:02.488 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:02.488 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:02.488 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:02.488 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4397.0:74243 3.9 e73/23 3:94977f81:::rbd_data.112d80ca42af.0000000000000002:head v 73'94, mlcod=71'92)", 2024-09-14T06:31:02.489 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:53.885156+0000", 2024-09-14T06:31:02.489 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 68.578327654999995, 2024-09-14T06:31:02.489 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0096009860000000006, 2024-09-14T06:31:02.489 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:31:02.489 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:31:02.489 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:31:02.489 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:31:02.489 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:02.489 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:31:02.489 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:53.885156+0000", 2024-09-14T06:31:02.489 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:02.489 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:02.489 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:02.490 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:31:02.490 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:53.885156+0000", 2024-09-14T06:31:02.490 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:02.490 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:02.490 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:02.490 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:31:02.490 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:53.885156+0000", 2024-09-14T06:31:02.490 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.23e-07 2024-09-14T06:31:02.490 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:02.490 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:02.490 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:31:02.490 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:53.889694+0000", 2024-09-14T06:31:02.490 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0045378709999999997 2024-09-14T06:31:02.490 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:02.491 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:02.491 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:31:02.491 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:53.889720+0000", 2024-09-14T06:31:02.491 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.5259e-05 2024-09-14T06:31:02.491 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:02.491 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:02.491 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:31:02.491 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:53.889731+0000", 2024-09-14T06:31:02.491 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.1739000000000001e-05 2024-09-14T06:31:02.491 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:02.491 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:02.491 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:31:02.491 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:53.889822+0000", 2024-09-14T06:31:02.491 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.1200999999999996e-05 2024-09-14T06:31:02.492 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:02.492 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:02.492 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:31:02.492 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:53.889850+0000", 2024-09-14T06:31:02.492 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.7084999999999999e-05 2024-09-14T06:31:02.492 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:02.492 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:02.492 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-14T06:31:02.492 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:53.894703+0000", 2024-09-14T06:31:02.492 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0048532690000000003 2024-09-14T06:31:02.492 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:02.492 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:02.493 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:31:02.493 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:53.894757+0000", 2024-09-14T06:31:02.493 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.4339000000000003e-05 2024-09-14T06:31:02.493 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:02.493 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:31:02.493 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:02.493 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:02.493 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:02.493 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4397.0:74249 3.0 e73/23 3:009b0225:::rbd_data.112d80ca42af.0000000000000009:head v 73'103, mlcod=71'101)", 2024-09-14T06:31:02.493 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:53.893219+0000", 2024-09-14T06:31:02.493 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 68.570265176999996, 2024-09-14T06:31:02.493 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0065381980000000003, 2024-09-14T06:31:02.493 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:31:02.493 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:31:02.493 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:31:02.494 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:31:02.494 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:02.494 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:31:02.494 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:53.893219+0000", 2024-09-14T06:31:02.494 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:02.494 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:02.494 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:02.494 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:31:02.494 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:53.893219+0000", 2024-09-14T06:31:02.494 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:02.494 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:02.494 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:02.494 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:31:02.494 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:53.893221+0000", 2024-09-14T06:31:02.495 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.4329999999999998e-06 2024-09-14T06:31:02.495 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:02.495 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:02.495 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:31:02.495 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:53.895968+0000", 2024-09-14T06:31:02.495 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.002746746 2024-09-14T06:31:02.495 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:02.495 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:02.495 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:31:02.495 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:53.895988+0000", 2024-09-14T06:31:02.495 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.0227000000000001e-05 2024-09-14T06:31:02.495 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:02.495 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:02.495 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:31:02.496 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:53.896000+0000", 2024-09-14T06:31:02.496 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.2359e-05 2024-09-14T06:31:02.496 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:02.496 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:02.496 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:31:02.496 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:53.896084+0000", 2024-09-14T06:31:02.496 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.3659999999999995e-05 2024-09-14T06:31:02.496 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:02.496 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:02.496 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:31:02.496 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:53.896109+0000", 2024-09-14T06:31:02.496 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.5259999999999999e-05 2024-09-14T06:31:02.496 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:02.497 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:02.497 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-14T06:31:02.497 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:53.899704+0000", 2024-09-14T06:31:02.497 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0035942209999999999 2024-09-14T06:31:02.497 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:02.497 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:02.497 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:31:02.497 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:53.899757+0000", 2024-09-14T06:31:02.497 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.3291999999999997e-05 2024-09-14T06:31:02.497 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:02.497 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:31:02.497 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:02.497 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:02.497 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:02.497 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4397.0:78263 3.0 e77/23 3:00899b22:::rbd_data.112ddfbec471.0000000000000016:head v 77'104, mlcod=73'102)", 2024-09-14T06:31:02.498 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:57.634578+0000", 2024-09-14T06:31:02.498 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 64.828906125000003, 2024-09-14T06:31:02.498 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.002372454, 2024-09-14T06:31:02.498 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:31:02.498 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:31:02.498 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:31:02.498 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:31:02.498 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:02.498 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:31:02.498 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:57.634578+0000", 2024-09-14T06:31:02.498 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:02.498 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:02.498 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:02.498 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:31:02.498 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:57.634578+0000", 2024-09-14T06:31:02.498 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:02.499 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:02.499 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:02.499 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:31:02.499 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:57.634578+0000", 2024-09-14T06:31:02.499 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.5699999999999998e-07 2024-09-14T06:31:02.499 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:02.499 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:02.499 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:31:02.499 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:57.635347+0000", 2024-09-14T06:31:02.499 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00076848300000000003 2024-09-14T06:31:02.499 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:02.499 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:02.499 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:31:02.499 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:57.635361+0000", 2024-09-14T06:31:02.499 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.4238e-05 2024-09-14T06:31:02.500 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:02.500 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:02.500 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:31:02.500 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:57.635371+0000", 2024-09-14T06:31:02.500 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.0087e-05 2024-09-14T06:31:02.500 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:02.500 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:02.500 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:31:02.500 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:57.635439+0000", 2024-09-14T06:31:02.500 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.7953000000000005e-05 2024-09-14T06:31:02.500 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:02.500 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:02.500 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:31:02.500 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:57.635464+0000", 2024-09-14T06:31:02.500 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.5062000000000002e-05 2024-09-14T06:31:02.500 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:02.501 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:02.501 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-14T06:31:02.501 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:57.636914+0000", 2024-09-14T06:31:02.501 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.001449625 2024-09-14T06:31:02.501 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:02.501 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:02.501 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:31:02.501 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:57.636950+0000", 2024-09-14T06:31:02.501 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.6548999999999999e-05 2024-09-14T06:31:02.501 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:02.501 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:31:02.501 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:02.501 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:02.502 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:02.502 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4397.0:78604 3.9 e77/23 3:94c62a0e:::rbd_data.112ddfbec471.000000000000002c:head v 77'96, mlcod=73'94)", 2024-09-14T06:31:02.502 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:57.677568+0000", 2024-09-14T06:31:02.502 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 64.785915736999996, 2024-09-14T06:31:02.502 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.002564846, 2024-09-14T06:31:02.502 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:31:02.502 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:31:02.502 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:31:02.502 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:31:02.502 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:02.502 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:31:02.502 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:57.677568+0000", 2024-09-14T06:31:02.502 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:02.502 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:02.502 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:02.503 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:31:02.503 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:57.677568+0000", 2024-09-14T06:31:02.503 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:02.503 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:02.503 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:02.503 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:31:02.503 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:57.677568+0000", 2024-09-14T06:31:02.503 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.8900000000000001e-07 2024-09-14T06:31:02.503 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:02.503 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:02.503 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:31:02.503 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:57.678488+0000", 2024-09-14T06:31:02.503 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00091963899999999998 2024-09-14T06:31:02.504 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:02.504 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:02.504 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:31:02.504 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:57.678500+0000", 2024-09-14T06:31:02.504 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.1902000000000001e-05 2024-09-14T06:31:02.504 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:02.504 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:02.504 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:31:02.504 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:57.678507+0000", 2024-09-14T06:31:02.504 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.7739999999999999e-06 2024-09-14T06:31:02.504 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:02.504 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:02.504 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:31:02.504 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:57.678586+0000", 2024-09-14T06:31:02.504 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.9103000000000005e-05 2024-09-14T06:31:02.505 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:02.505 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:02.505 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:31:02.505 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:57.678614+0000", 2024-09-14T06:31:02.505 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.7912e-05 2024-09-14T06:31:02.505 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:02.505 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:02.505 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-14T06:31:02.505 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:57.680075+0000", 2024-09-14T06:31:02.505 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0014613370000000001 2024-09-14T06:31:02.505 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:02.505 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:02.505 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:31:02.505 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:57.680133+0000", 2024-09-14T06:31:02.506 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.7890000000000003e-05 2024-09-14T06:31:02.506 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:02.506 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:31:02.506 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:02.506 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:02.506 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:02.506 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4397.0:78641 3.9 e77/23 3:914b72e4:::rbd_data.112ddfbec471.0000000000000030:head v 77'98, mlcod=73'95)", 2024-09-14T06:31:02.506 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:57.682289+0000", 2024-09-14T06:31:02.506 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 64.781195056000001, 2024-09-14T06:31:02.506 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0020171400000000002, 2024-09-14T06:31:02.506 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:31:02.506 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:31:02.506 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:31:02.506 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:31:02.507 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:02.507 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:31:02.507 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:57.682289+0000", 2024-09-14T06:31:02.507 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:02.507 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:02.507 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:02.507 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:31:02.507 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:57.682289+0000", 2024-09-14T06:31:02.507 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:02.507 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:02.507 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:02.507 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:31:02.507 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:57.682290+0000", 2024-09-14T06:31:02.507 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.0100000000000003e-07 2024-09-14T06:31:02.508 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:02.508 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:02.508 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:31:02.508 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:57.683230+0000", 2024-09-14T06:31:02.508 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00094047599999999999 2024-09-14T06:31:02.508 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:02.508 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:02.508 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:31:02.508 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:57.683243+0000", 2024-09-14T06:31:02.508 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.2926e-05 2024-09-14T06:31:02.508 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:02.508 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:02.508 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:31:02.509 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:57.683250+0000", 2024-09-14T06:31:02.509 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.6109999999999999e-06 2024-09-14T06:31:02.509 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:02.509 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:02.509 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:31:02.509 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:57.683313+0000", 2024-09-14T06:31:02.509 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.2943999999999999e-05 2024-09-14T06:31:02.509 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:02.509 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:02.509 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:31:02.509 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:57.683337+0000", 2024-09-14T06:31:02.509 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.4377999999999999e-05 2024-09-14T06:31:02.510 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:02.510 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:02.510 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-14T06:31:02.510 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:57.684264+0000", 2024-09-14T06:31:02.510 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00092733099999999999 2024-09-14T06:31:02.510 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:02.510 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:02.510 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:31:02.510 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:57.684306+0000", 2024-09-14T06:31:02.510 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.1573000000000003e-05 2024-09-14T06:31:02.510 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:02.510 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:31:02.510 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:02.510 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:02.510 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:02.511 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4397.0:126010 3.9 e104/23 3:9120f7ff:::rbd_data.112d97471187.000000000000001b:head v 104'145, mlcod=99'143)", 2024-09-14T06:31:02.511 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:30:25.990254+0000", 2024-09-14T06:31:02.511 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 36.473230248, 2024-09-14T06:31:02.511 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0020629699999999999, 2024-09-14T06:31:02.511 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:31:02.511 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:31:02.511 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:31:02.511 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:31:02.511 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:02.511 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:31:02.511 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:25.990254+0000", 2024-09-14T06:31:02.511 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:02.511 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:02.511 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:02.512 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:31:02.512 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:25.990254+0000", 2024-09-14T06:31:02.512 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:02.512 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:02.512 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:02.512 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:31:02.512 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:25.990255+0000", 2024-09-14T06:31:02.512 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.3349999999999999e-06 2024-09-14T06:31:02.512 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:02.512 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:02.512 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:31:02.512 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:25.990966+0000", 2024-09-14T06:31:02.512 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00071085199999999999 2024-09-14T06:31:02.512 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:02.513 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:02.513 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:31:02.513 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:25.990986+0000", 2024-09-14T06:31:02.513 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.0054000000000001e-05 2024-09-14T06:31:02.513 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:02.513 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:02.513 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:31:02.513 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:25.990995+0000", 2024-09-14T06:31:02.513 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.166e-06 2024-09-14T06:31:02.513 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:02.513 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:02.513 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:31:02.513 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:25.991055+0000", 2024-09-14T06:31:02.513 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.0093000000000002e-05 2024-09-14T06:31:02.514 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:02.514 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:02.514 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:31:02.514 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:25.991082+0000", 2024-09-14T06:31:02.514 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.6505000000000001e-05 2024-09-14T06:31:02.514 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:02.514 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:02.514 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-14T06:31:02.514 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:25.992270+0000", 2024-09-14T06:31:02.514 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0011880479999999999 2024-09-14T06:31:02.514 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:02.514 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:02.514 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:31:02.514 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:25.992317+0000", 2024-09-14T06:31:02.515 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.6916999999999998e-05 2024-09-14T06:31:02.515 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:02.515 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:31:02.515 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:02.515 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:02.515 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:02.515 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4397.0:149885 3.9 e122/23 3:951acef3:::rbd_data.112d1bb7b269.0000000000000005:head v 122'176, mlcod=122'172)", 2024-09-14T06:31:02.515 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:30:44.739092+0000", 2024-09-14T06:31:02.515 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 17.724391427, 2024-09-14T06:31:02.515 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0028236659999999998, 2024-09-14T06:31:02.515 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:31:02.515 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:31:02.515 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:31:02.515 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:31:02.516 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:02.516 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:31:02.516 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.739092+0000", 2024-09-14T06:31:02.516 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:02.516 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:02.516 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:02.516 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:31:02.516 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.739092+0000", 2024-09-14T06:31:02.516 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:02.516 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:02.516 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:02.516 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:31:02.516 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.739093+0000", 2024-09-14T06:31:02.517 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.0109999999999999e-06 2024-09-14T06:31:02.517 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:02.517 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:02.517 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:31:02.517 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.740189+0000", 2024-09-14T06:31:02.517 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00109527 2024-09-14T06:31:02.517 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:02.517 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:02.517 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:31:02.517 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.740199+0000", 2024-09-14T06:31:02.517 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.0458999999999999e-05 2024-09-14T06:31:02.517 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:02.517 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:02.518 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:31:02.518 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.740207+0000", 2024-09-14T06:31:02.518 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.4499999999999998e-06 2024-09-14T06:31:02.518 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:02.518 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:02.518 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:31:02.518 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.740295+0000", 2024-09-14T06:31:02.518 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.8740999999999996e-05 2024-09-14T06:31:02.518 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:02.518 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:02.518 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:31:02.518 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.740317+0000", 2024-09-14T06:31:02.518 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.1523e-05 2024-09-14T06:31:02.518 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:02.518 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:02.519 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-14T06:31:02.519 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.741865+0000", 2024-09-14T06:31:02.519 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.001548315 2024-09-14T06:31:02.519 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:02.519 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:02.519 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:31:02.519 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.741916+0000", 2024-09-14T06:31:02.519 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.0896999999999997e-05 2024-09-14T06:31:02.519 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:02.519 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:31:02.519 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:02.519 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:02.519 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:02.519 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4397.0:149886 3.4 e122/23 3:2033c3d0:::rbd_data.112d1bb7b269.0000000000000006:head v 122'220, mlcod=120'217)", 2024-09-14T06:31:02.520 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:30:44.741736+0000", 2024-09-14T06:31:02.520 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 17.721748009999999, 2024-09-14T06:31:02.520 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0020268999999999999, 2024-09-14T06:31:02.520 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:31:02.520 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:31:02.520 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:31:02.520 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:31:02.520 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:02.520 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:31:02.520 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.741736+0000", 2024-09-14T06:31:02.520 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:02.520 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:02.520 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:02.521 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:31:02.521 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.741736+0000", 2024-09-14T06:31:02.521 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:02.521 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:02.521 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:02.521 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:31:02.521 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.741737+0000", 2024-09-14T06:31:02.521 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.9000000000000005e-07 2024-09-14T06:31:02.521 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:02.521 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:02.521 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:31:02.521 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.741918+0000", 2024-09-14T06:31:02.521 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00018120300000000001 2024-09-14T06:31:02.521 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:02.522 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:02.522 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:31:02.522 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.741930+0000", 2024-09-14T06:31:02.522 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.1875999999999999e-05 2024-09-14T06:31:02.522 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:02.522 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:02.522 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:31:02.522 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.741938+0000", 2024-09-14T06:31:02.522 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.8040000000000002e-06 2024-09-14T06:31:02.522 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:02.522 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:02.522 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:31:02.522 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.742185+0000", 2024-09-14T06:31:02.523 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00024747199999999999 2024-09-14T06:31:02.523 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:02.523 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:02.523 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:31:02.523 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.742207+0000", 2024-09-14T06:31:02.523 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.1416999999999998e-05 2024-09-14T06:31:02.523 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:02.523 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:02.523 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-14T06:31:02.523 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.743715+0000", 2024-09-14T06:31:02.523 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00150884 2024-09-14T06:31:02.523 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:02.523 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:02.523 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:31:02.524 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.743763+0000", 2024-09-14T06:31:02.524 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.7298e-05 2024-09-14T06:31:02.524 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:02.524 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:31:02.524 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:02.524 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:02.524 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:02.524 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4397.0:149933 3.4 e122/23 3:234f5229:::rbd_data.112d1bb7b269.0000000000000009:head v 122'221, mlcod=120'218)", 2024-09-14T06:31:02.524 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:30:44.743849+0000", 2024-09-14T06:31:02.524 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 17.719635010000001, 2024-09-14T06:31:02.524 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0031479519999999999, 2024-09-14T06:31:02.524 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:31:02.524 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:31:02.524 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:31:02.524 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:31:02.525 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:02.525 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:31:02.525 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.743849+0000", 2024-09-14T06:31:02.525 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:02.525 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:02.525 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:02.525 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:31:02.525 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.743849+0000", 2024-09-14T06:31:02.525 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:02.525 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:02.525 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:02.525 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:31:02.525 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.743850+0000", 2024-09-14T06:31:02.526 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.279e-06 2024-09-14T06:31:02.526 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:02.526 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:02.526 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:31:02.526 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.745151+0000", 2024-09-14T06:31:02.526 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.001300495 2024-09-14T06:31:02.526 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:02.526 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:02.526 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:31:02.526 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.745167+0000", 2024-09-14T06:31:02.526 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.6478e-05 2024-09-14T06:31:02.526 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:02.526 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:02.527 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:31:02.527 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.745175+0000", 2024-09-14T06:31:02.527 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.2679999999999994e-06 2024-09-14T06:31:02.527 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:02.527 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:02.527 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:31:02.527 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.745226+0000", 2024-09-14T06:31:02.527 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.0192999999999999e-05 2024-09-14T06:31:02.527 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:02.527 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:02.527 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:31:02.527 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.745247+0000", 2024-09-14T06:31:02.527 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.1767e-05 2024-09-14T06:31:02.528 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:02.528 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:02.528 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-14T06:31:02.528 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.746952+0000", 2024-09-14T06:31:02.528 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0017042769999999999 2024-09-14T06:31:02.528 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:02.528 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:02.528 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:31:02.528 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.746997+0000", 2024-09-14T06:31:02.528 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.5194999999999997e-05 2024-09-14T06:31:02.528 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:02.528 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:31:02.528 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:02.528 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:02.529 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:02.529 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4397.0:150494 3.9 e122/23 3:955152a4:::rbd_object_map.112d1bb7b269.0000000000000038:head v 122'194, mlcod=122'190)", 2024-09-14T06:31:02.529 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:30:44.781585+0000", 2024-09-14T06:31:02.529 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 17.681899118, 2024-09-14T06:31:02.529 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0026957499999999998, 2024-09-14T06:31:02.529 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:31:02.529 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:31:02.529 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:31:02.529 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:31:02.529 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:02.529 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:31:02.529 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.781585+0000", 2024-09-14T06:31:02.529 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:02.529 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:02.529 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:02.530 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:31:02.530 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.781585+0000", 2024-09-14T06:31:02.530 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:02.530 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:02.530 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:02.530 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:31:02.530 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.781585+0000", 2024-09-14T06:31:02.530 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.5000000000000002e-07 2024-09-14T06:31:02.530 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:02.530 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:02.530 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:31:02.530 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.781594+0000", 2024-09-14T06:31:02.531 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.4640000000000006e-06 2024-09-14T06:31:02.531 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:02.531 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:02.531 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:31:02.531 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.781604+0000", 2024-09-14T06:31:02.531 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.0071999999999999e-05 2024-09-14T06:31:02.531 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:02.531 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:02.531 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:31:02.531 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.781610+0000", 2024-09-14T06:31:02.531 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.3609999999999997e-06 2024-09-14T06:31:02.531 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:02.531 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:02.531 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:31:02.531 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.781676+0000", 2024-09-14T06:31:02.531 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.5963000000000005e-05 2024-09-14T06:31:02.532 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:02.532 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:02.532 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:31:02.532 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.781696+0000", 2024-09-14T06:31:02.532 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.9278000000000001e-05 2024-09-14T06:31:02.532 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:02.532 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:02.532 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-14T06:31:02.532 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.784231+0000", 2024-09-14T06:31:02.532 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.002535093 2024-09-14T06:31:02.532 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:02.532 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:02.532 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:31:02.533 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.784280+0000", 2024-09-14T06:31:02.533 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.9869e-05 2024-09-14T06:31:02.533 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:02.533 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:31:02.533 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:02.533 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:02.533 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:02.533 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4397.0:150519 3.9 e122/23 3:955152a4:::rbd_object_map.112d1bb7b269.0000000000000038:head v 122'195, mlcod=122'192)", 2024-09-14T06:31:02.533 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:30:44.782445+0000", 2024-09-14T06:31:02.533 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 17.681038726000001, 2024-09-14T06:31:02.533 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.002158227, 2024-09-14T06:31:02.533 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:31:02.533 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:31:02.533 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:31:02.533 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:31:02.533 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:02.534 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:31:02.534 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.782445+0000", 2024-09-14T06:31:02.534 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:02.534 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:02.534 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:02.534 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:31:02.534 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.782445+0000", 2024-09-14T06:31:02.534 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:02.534 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:02.534 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:02.534 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:31:02.534 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.782446+0000", 2024-09-14T06:31:02.534 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.6499999999999998e-07 2024-09-14T06:31:02.534 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:02.535 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:02.535 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:31:02.535 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.782455+0000", 2024-09-14T06:31:02.535 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.1940000000000004e-06 2024-09-14T06:31:02.535 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:02.535 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:02.535 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:31:02.535 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.782464+0000", 2024-09-14T06:31:02.535 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.7220000000000005e-06 2024-09-14T06:31:02.535 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:02.535 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:02.535 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:31:02.535 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.782470+0000", 2024-09-14T06:31:02.535 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.9580000000000004e-06 2024-09-14T06:31:02.535 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:02.536 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:02.536 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:31:02.536 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.782514+0000", 2024-09-14T06:31:02.536 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.3931999999999999e-05 2024-09-14T06:31:02.536 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:02.536 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:02.536 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:31:02.536 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.782534+0000", 2024-09-14T06:31:02.536 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.0237e-05 2024-09-14T06:31:02.536 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:02.536 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:02.536 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-14T06:31:02.536 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.784560+0000", 2024-09-14T06:31:02.537 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.002025813 2024-09-14T06:31:02.537 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:02.537 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:02.537 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:31:02.537 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.784603+0000", 2024-09-14T06:31:02.537 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.3606000000000002e-05 2024-09-14T06:31:02.537 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:02.537 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:31:02.537 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:02.537 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:02.537 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:02.537 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4397.0:150898 3.9 e122/23 3:97c98f90:::rbd_data.112d1bb7b269.0000000000000025:head v 122'206, mlcod=122'203)", 2024-09-14T06:31:02.538 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:30:44.807232+0000", 2024-09-14T06:31:02.538 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 17.656252201000001, 2024-09-14T06:31:02.538 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0022297889999999998, 2024-09-14T06:31:02.538 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:31:02.538 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:31:02.538 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:31:02.538 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:31:02.538 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:02.538 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:31:02.538 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.807232+0000", 2024-09-14T06:31:02.538 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:02.538 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:02.538 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:02.538 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:31:02.539 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.807232+0000", 2024-09-14T06:31:02.539 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:02.539 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:02.539 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:02.539 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:31:02.539 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.807233+0000", 2024-09-14T06:31:02.539 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.5710000000000001e-06 2024-09-14T06:31:02.539 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:02.539 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:02.539 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:31:02.539 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.807944+0000", 2024-09-14T06:31:02.539 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00071056299999999999 2024-09-14T06:31:02.539 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:02.539 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:02.540 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:31:02.540 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.807961+0000", 2024-09-14T06:31:02.540 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.7689e-05 2024-09-14T06:31:02.540 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:02.540 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:02.540 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:31:02.540 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.807971+0000", 2024-09-14T06:31:02.540 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.2e-06 2024-09-14T06:31:02.540 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:02.540 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:02.540 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:31:02.540 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.808050+0000", 2024-09-14T06:31:02.540 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.9308999999999996e-05 2024-09-14T06:31:02.540 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:02.541 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:02.541 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:31:02.541 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.808074+0000", 2024-09-14T06:31:02.541 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.4147000000000001e-05 2024-09-14T06:31:02.541 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:02.541 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:02.541 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-14T06:31:02.541 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.809415+0000", 2024-09-14T06:31:02.541 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.001341257 2024-09-14T06:31:02.541 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:02.541 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:02.541 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:31:02.541 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.809461+0000", 2024-09-14T06:31:02.541 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.6053000000000001e-05 2024-09-14T06:31:02.542 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:02.542 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:31:02.542 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:02.542 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:02.542 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:02.542 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4397.0:152325 3.9 e122/23 3:955152a4:::rbd_object_map.112d1bb7b269.0000000000000038:head v 122'268, mlcod=122'265)", 2024-09-14T06:31:02.542 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:30:44.914377+0000", 2024-09-14T06:31:02.542 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 17.549106896000001, 2024-09-14T06:31:02.542 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0020208840000000001, 2024-09-14T06:31:02.542 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:31:02.542 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:31:02.542 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:31:02.542 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:31:02.543 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:02.543 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:31:02.543 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.914377+0000", 2024-09-14T06:31:02.543 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:02.543 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:02.543 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:02.543 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:31:02.543 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.914377+0000", 2024-09-14T06:31:02.543 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:02.543 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:02.543 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:02.543 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:31:02.543 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.914378+0000", 2024-09-14T06:31:02.544 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.3600000000000003e-07 2024-09-14T06:31:02.544 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:02.544 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:02.544 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:31:02.544 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.914383+0000", 2024-09-14T06:31:02.544 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.7230000000000001e-06 2024-09-14T06:31:02.544 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:02.544 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:02.544 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:31:02.544 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.914393+0000", 2024-09-14T06:31:02.544 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.8179999999999999e-06 2024-09-14T06:31:02.544 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:02.544 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:02.544 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:31:02.545 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.914400+0000", 2024-09-14T06:31:02.545 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.968e-06 2024-09-14T06:31:02.545 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:02.545 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:02.545 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:31:02.545 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.914447+0000", 2024-09-14T06:31:02.545 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.7122e-05 2024-09-14T06:31:02.545 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:02.545 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:02.545 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:31:02.545 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.914466+0000", 2024-09-14T06:31:02.545 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.8958999999999999e-05 2024-09-14T06:31:02.545 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:02.546 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:02.546 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-14T06:31:02.546 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.916356+0000", 2024-09-14T06:31:02.546 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.001890121 2024-09-14T06:31:02.546 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:02.546 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:02.546 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:31:02.546 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.916398+0000", 2024-09-14T06:31:02.546 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.1437000000000003e-05 2024-09-14T06:31:02.546 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:02.546 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:31:02.546 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:02.546 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:02.546 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:02.547 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4397.0:152449 3.9 e122/23 3:955152a4:::rbd_object_map.112d1bb7b269.0000000000000038:head v 122'274, mlcod=122'268)", 2024-09-14T06:31:02.547 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:30:44.924063+0000", 2024-09-14T06:31:02.547 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 17.539420385, 2024-09-14T06:31:02.547 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.002131764, 2024-09-14T06:31:02.547 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:31:02.547 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:31:02.547 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:31:02.547 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:31:02.547 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:02.547 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:31:02.547 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.924063+0000", 2024-09-14T06:31:02.547 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:02.547 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:02.547 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:02.547 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:31:02.548 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.924063+0000", 2024-09-14T06:31:02.548 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:02.548 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:02.548 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:02.548 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:31:02.548 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.924064+0000", 2024-09-14T06:31:02.548 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.72e-07 2024-09-14T06:31:02.548 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:02.548 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:02.548 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:31:02.548 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.924071+0000", 2024-09-14T06:31:02.548 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.0999999999999998e-06 2024-09-14T06:31:02.548 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:02.549 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:02.549 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:31:02.549 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.924077+0000", 2024-09-14T06:31:02.549 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.268e-06 2024-09-14T06:31:02.549 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:02.549 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:02.549 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:31:02.549 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.924082+0000", 2024-09-14T06:31:02.549 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.7110000000000003e-06 2024-09-14T06:31:02.549 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:02.549 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:02.549 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:31:02.549 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.924397+0000", 2024-09-14T06:31:02.549 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00031503500000000002 2024-09-14T06:31:02.550 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:02.550 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:02.550 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:31:02.550 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.924418+0000", 2024-09-14T06:31:02.550 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.1044999999999999e-05 2024-09-14T06:31:02.550 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:02.550 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:02.550 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-14T06:31:02.550 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.926158+0000", 2024-09-14T06:31:02.550 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0017404860000000001 2024-09-14T06:31:02.550 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:02.550 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:02.550 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:31:02.550 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.926195+0000", 2024-09-14T06:31:02.551 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.6847000000000001e-05 2024-09-14T06:31:02.551 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:02.551 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:31:02.551 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:02.551 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:02.551 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:02.551 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4397.0:152452 3.9 e122/23 3:955152a4:::rbd_object_map.112d1bb7b269.0000000000000038:head v 122'275, mlcod=122'268)", 2024-09-14T06:31:02.551 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:30:44.924090+0000", 2024-09-14T06:31:02.551 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 17.539394065, 2024-09-14T06:31:02.551 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0021362400000000002, 2024-09-14T06:31:02.551 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:31:02.551 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:31:02.551 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:31:02.551 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:31:02.552 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:02.552 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:31:02.552 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.924090+0000", 2024-09-14T06:31:02.552 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:02.552 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:02.552 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:02.552 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:31:02.552 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.924090+0000", 2024-09-14T06:31:02.552 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:02.552 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:02.552 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:02.552 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:31:02.552 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.924090+0000", 2024-09-14T06:31:02.552 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.4699999999999998e-07 2024-09-14T06:31:02.552 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:02.553 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:02.553 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:31:02.553 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.924658+0000", 2024-09-14T06:31:02.553 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00056802700000000003 2024-09-14T06:31:02.553 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:02.553 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:02.553 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:31:02.553 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.924664+0000", 2024-09-14T06:31:02.553 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.3570000000000003e-06 2024-09-14T06:31:02.553 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:02.553 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:02.553 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:31:02.553 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.924669+0000", 2024-09-14T06:31:02.553 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.2300000000000002e-06 2024-09-14T06:31:02.554 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:02.554 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:02.554 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:31:02.554 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.924802+0000", 2024-09-14T06:31:02.554 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00013362500000000001 2024-09-14T06:31:02.554 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:02.554 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:02.554 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:31:02.554 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.924824+0000", 2024-09-14T06:31:02.554 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.1449000000000001e-05 2024-09-14T06:31:02.554 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:02.554 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:02.554 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-14T06:31:02.554 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.926197+0000", 2024-09-14T06:31:02.555 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.001373354 2024-09-14T06:31:02.555 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:02.555 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:02.555 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:31:02.555 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.926226+0000", 2024-09-14T06:31:02.555 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.8951000000000001e-05 2024-09-14T06:31:02.555 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:02.555 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:31:02.555 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:02.555 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:02.555 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:31:02.555 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-14T06:31:02.556 INFO:teuthology.orchestra.run.smithi094.stdout:3921 trunc from 0xe65c44c to 0x534b592 2024-09-14T06:31:02.557 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-14T06:31:02.558 INFO:teuthology.orchestra.run.smithi094.stdout:3922 read 0x51b3ca9 thru 0x51c34ef (0xf847 bytes) 2024-09-14T06:31:02.558 INFO:teuthology.orchestra.run.smithi094.stdout:3923 read 0x293cad4 thru 0x294b67e (0xebab bytes) 2024-09-14T06:31:02.558 INFO:teuthology.orchestra.run.smithi094.stdout:3924 punch from 0x501e975 to 0x502ae0e, (0xc499 bytes) 2024-09-14T06:31:02.559 INFO:teuthology.orchestra.run.smithi094.stdout:3925 punch from 0x4108466 to 0x411080e, (0x83a8 bytes) 2024-09-14T06:31:02.559 INFO:teuthology.orchestra.run.smithi094.stdout:3926 write 0xad9edd6 thru 0xadaebb0 (0xfddb bytes) 2024-09-14T06:31:02.559 INFO:teuthology.orchestra.run.smithi094.stdout:3927 write 0x7d616b9 thru 0x7d71014 (0xf95c bytes) 2024-09-14T06:31:02.559 INFO:teuthology.orchestra.run.smithi094.stdout:3928 read 0x834ba41 thru 0x835094a (0x4f0a bytes) 2024-09-14T06:31:02.559 INFO:teuthology.orchestra.run.smithi094.stdout:3931 read 0xab2da10 thru 0xab2e4b7 (0xaa8 bytes) 2024-09-14T06:31:02.559 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-14T06:31:02.559 INFO:teuthology.orchestra.run.smithi094.stdout:3933 write 0x8f83294 thru 0x8f8bdf8 (0x8b65 bytes) 2024-09-14T06:31:02.559 INFO:teuthology.orchestra.run.smithi094.stdout:3934 write 0x2a41b45 thru 0x2a50928 (0xede4 bytes) 2024-09-14T06:31:02.559 INFO:teuthology.orchestra.run.smithi094.stdout:3935 trunc from 0xadaebb1 to 0x56755d4 2024-09-14T06:31:02.559 INFO:teuthology.orchestra.run.smithi094.stdout:3936 write 0xdd7f1a6 thru 0xdd85c1a (0x6a75 bytes) 2024-09-14T06:31:02.562 INFO:teuthology.orchestra.run.smithi094.stdout:3937 punch from 0x25e2e8 to 0x269701, (0xb419 bytes) 2024-09-14T06:31:02.562 INFO:tasks.ceph.osd.2.smithi094.stderr:2024-09-14T06:31:02.561+0000 7faf3269b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33649) UID: 0 2024-09-14T06:31:02.566 INFO:teuthology.orchestra.run.smithi094.stdout:3939 trunc from 0xdd85c1b to 0xaa14114 2024-09-14T06:31:02.576 INFO:teuthology.orchestra.run.smithi094.stdout:3940 read 0xb6c603 thru 0xb7111e (0x4b1c bytes) 2024-09-14T06:31:02.576 INFO:teuthology.orchestra.run.smithi094.stdout:3941 read 0x3c07428 thru 0x3c0af9b (0x3b74 bytes) 2024-09-14T06:31:02.577 INFO:teuthology.orchestra.run.smithi094.stdout:3942 trunc from 0xaa14114 to 0x96159d8 2024-09-14T06:31:02.582 INFO:teuthology.orchestra.run.smithi094.stdout:3943 write 0x88844b8 thru 0x888b516 (0x705f bytes) 2024-09-14T06:31:02.588 INFO:teuthology.orchestra.run.smithi094.stdout:3944 write 0x13fa546 thru 0x14017c0 (0x727b bytes) 2024-09-14T06:31:02.596 INFO:teuthology.orchestra.run.smithi094.stdout:3946 read 0x805d443 thru 0x8063f36 (0x6af4 bytes) 2024-09-14T06:31:02.596 INFO:teuthology.orchestra.run.smithi094.stdout:3947 write 0x38d5569 thru 0x38e0c9c (0xb734 bytes) 2024-09-14T06:31:02.603 INFO:teuthology.orchestra.run.smithi094.stdout:3948 punch from 0x5b07792 to 0x5b11810, (0xa07e bytes) 2024-09-14T06:31:02.603 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-14T06:31:02.603 INFO:teuthology.orchestra.run.smithi094.stdout:3951 write 0x2656e1d thru 0x2664cdd (0xdec1 bytes) 2024-09-14T06:31:02.610 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-14T06:31:02.611 INFO:teuthology.orchestra.run.smithi094.stdout:3952 read 0xb84cda thru 0xb92052 (0xd379 bytes) 2024-09-14T06:31:02.611 INFO:teuthology.orchestra.run.smithi094.stdout:3953 read 0x366a4a1 thru 0x3676023 (0xbb83 bytes) 2024-09-14T06:31:02.611 INFO:teuthology.orchestra.run.smithi094.stdout:3954 read 0xf77e36 thru 0xf7b32c (0x34f7 bytes) 2024-09-14T06:31:02.611 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-14T06:31:02.611 INFO:teuthology.orchestra.run.smithi094.stdout:3956 read 0x8764ee2 thru 0x8771619 (0xc738 bytes) 2024-09-14T06:31:02.614 INFO:teuthology.orchestra.run.smithi094.stdout:3957 trunc from 0x96159d8 to 0xa3e8381 2024-09-14T06:31:02.618 INFO:teuthology.orchestra.run.smithi094.stdout:3958 read 0x457fea3 thru 0x4587789 (0x78e7 bytes) 2024-09-14T06:31:02.630 INFO:teuthology.orchestra.run.smithi094.stdout:3959 trunc from 0xa3e8381 to 0x6576df 2024-09-14T06:31:02.654 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-14T06:31:02.661 INFO:tasks.ceph.osd.3.smithi094.stderr:2024-09-14T06:31:02.660+0000 7feeeafe8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33655) UID: 0 2024-09-14T06:31:02.667 INFO:teuthology.orchestra.run.smithi094.stdout:3962 write 0x5596247 thru 0x559fe10 (0x9bca bytes) 2024-09-14T06:31:02.675 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-14T06:31:02.675 INFO:teuthology.orchestra.run.smithi094.stdout:3964 punch from 0x2f28d53 to 0x2f30757, (0x7a04 bytes) 2024-09-14T06:31:02.675 INFO:teuthology.orchestra.run.smithi094.stdout:3965 punch from 0x5096f04 to 0x50a6794, (0xf890 bytes) 2024-09-14T06:31:02.675 INFO:teuthology.orchestra.run.smithi094.stdout:3968 read 0x40caac thru 0x414a7a (0x7fcf bytes) 2024-09-14T06:31:02.676 INFO:teuthology.orchestra.run.smithi094.stdout:3969 read 0x1a41997 thru 0x1a4cd36 (0xb3a0 bytes) 2024-09-14T06:31:02.676 INFO:teuthology.orchestra.run.smithi094.stdout:3970 read 0x4413137 thru 0x4420081 (0xcf4b bytes) 2024-09-14T06:31:02.676 INFO:teuthology.orchestra.run.smithi094.stdout:3971 write 0x51e70b7 thru 0x51ef215 (0x815f bytes) 2024-09-14T06:31:02.683 INFO:teuthology.orchestra.run.smithi094.stdout:3972 punch from 0x274caa9 to 0x27556e3, (0x8c3a bytes) 2024-09-14T06:31:02.683 INFO:teuthology.orchestra.run.smithi094.stdout:3973 write 0x36e2019 thru 0x36e3c10 (0x1bf8 bytes) 2024-09-14T06:31:02.686 INFO:teuthology.orchestra.run.smithi094.stdout:3974 read 0x3cc285c thru 0x3cca1fd (0x79a2 bytes) 2024-09-14T06:31:02.686 INFO:teuthology.orchestra.run.smithi094.stdout:3977 read 0x83e099 thru 0x84aec3 (0xce2b bytes) 2024-09-14T06:31:02.694 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-14T06:31:02.694 INFO:teuthology.orchestra.run.smithi094.stdout: "ops": [], 2024-09-14T06:31:02.694 INFO:teuthology.orchestra.run.smithi094.stdout: "num_ops": 0 2024-09-14T06:31:02.694 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-14T06:31:02.697 INFO:teuthology.orchestra.run.smithi094.stdout:3979 write 0x96377bb thru 0x964600d (0xe853 bytes) 2024-09-14T06:31:02.704 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-14T06:31:02.708 INFO:teuthology.orchestra.run.smithi094.stdout:3981 read 0x6e5f1f8 thru 0x6e6a30d (0xb116 bytes) 2024-09-14T06:31:02.717 INFO:teuthology.orchestra.run.smithi094.stdout:3982 write 0xc456ec2 thru 0xc45bcb9 (0x4df8 bytes) 2024-09-14T06:31:02.725 INFO:teuthology.orchestra.run.smithi094.stdout:3985 punch from 0x4e880d to 0x4ea23d, (0x1a30 bytes) 2024-09-14T06:31:02.725 INFO:teuthology.orchestra.run.smithi094.stdout:3986 read 0x4db7117 thru 0x4dc50a6 (0xdf90 bytes) 2024-09-14T06:31:02.725 INFO:teuthology.orchestra.run.smithi094.stdout:3987 read 0x959a140 thru 0x959fff2 (0x5eb3 bytes) 2024-09-14T06:31:02.725 INFO:teuthology.orchestra.run.smithi094.stdout:3989 read 0xa965de1 thru 0xa96b39b (0x55bb bytes) 2024-09-14T06:31:02.725 INFO:teuthology.orchestra.run.smithi094.stdout:3991 write 0x8067ea1 thru 0x8073a7a (0xbbda bytes) 2024-09-14T06:31:02.732 INFO:teuthology.orchestra.run.smithi094.stdout:3992 read 0x9625df9 thru 0x962bd55 (0x5f5d bytes) 2024-09-14T06:31:02.733 INFO:teuthology.orchestra.run.smithi094.stdout:3993 write 0x17f24e9 thru 0x17fbb74 (0x968c bytes) 2024-09-14T06:31:02.742 INFO:teuthology.orchestra.run.smithi094.stdout:3994 write 0x836caac thru 0x837a70d (0xdc62 bytes) 2024-09-14T06:31:02.752 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-14T06:31:02.752 INFO:teuthology.orchestra.run.smithi094.stdout:3996 trunc from 0xc45bcba to 0xbec43c1 2024-09-14T06:31:02.761 INFO:tasks.ceph.osd.3.smithi094.stderr:2024-09-14T06:31:02.760+0000 7feeeafe8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33655) UID: 0 2024-09-14T06:31:02.765 INFO:teuthology.orchestra.run.smithi094.stdout:3997 read 0x2236247 thru 0x2237e8a (0x1c44 bytes) 2024-09-14T06:31:02.765 INFO:teuthology.orchestra.run.smithi094.stdout:3999 read 0xa70f597 thru 0xa717a20 (0x848a bytes) 2024-09-14T06:31:02.773 INFO:teuthology.orchestra.run.smithi094.stdout:4001 write 0xdef6497 thru 0xdf03c3e (0xd7a8 bytes) 2024-09-14T06:31:02.783 INFO:teuthology.orchestra.run.smithi094.stderr:reweighted osd.3 to 1 (10000) 2024-09-14T06:31:02.799 INFO:teuthology.orchestra.run.smithi094.stdout:4003 trunc from 0xdf03c3f to 0x336e680 2024-09-14T06:31:02.805 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 4 1 2024-09-14T06:31:02.825 INFO:teuthology.orchestra.run.smithi094.stdout:4004 punch from 0x12506af to 0x1255654, (0x4fa5 bytes) 2024-09-14T06:31:02.825 INFO:teuthology.orchestra.run.smithi094.stdout:4005 read 0x16c82c5 thru 0x16c9935 (0x1671 bytes) 2024-09-14T06:31:02.836 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-14T06:31:02.836 INFO:teuthology.orchestra.run.smithi094.stdout: "ops": [], 2024-09-14T06:31:02.836 INFO:teuthology.orchestra.run.smithi094.stdout: "complaint_time": 30, 2024-09-14T06:31:02.836 INFO:teuthology.orchestra.run.smithi094.stdout: "num_blocked_ops": 0 2024-09-14T06:31:02.836 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-14T06:31:02.837 INFO:teuthology.orchestra.run.smithi094.stdout:4006 write 0x90a43bf thru 0x90ab319 (0x6f5b bytes) 2024-09-14T06:31:02.847 INFO:teuthology.orchestra.run.smithi094.stdout:4008 read 0x4f9b972 thru 0x4fa8232 (0xc8c1 bytes) 2024-09-14T06:31:02.847 INFO:teuthology.orchestra.run.smithi094.stdout:4009 writesame 0x4ce42a8 thru 0x4cf3ad0 data_size 0x25(0xf829 bytes) 2024-09-14T06:31:02.848 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-14T06:31:02.856 INFO:teuthology.orchestra.run.smithi094.stdout:4012 punch from 0x6f5acd7 to 0x6f6042b, (0x5754 bytes) 2024-09-14T06:31:02.856 INFO:teuthology.orchestra.run.smithi094.stdout:4013 write 0x35c829b thru 0x35d3e87 (0xbbed bytes) 2024-09-14T06:31:02.863 INFO:teuthology.orchestra.run.smithi094.stdout:4015 read 0x339109d thru 0x339737f (0x62e3 bytes) 2024-09-14T06:31:02.864 INFO:tasks.ceph.osd.3.smithi094.stderr:2024-09-14T06:31:02.863+0000 7feeeafe8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33655) UID: 0 2024-09-14T06:31:02.864 INFO:teuthology.orchestra.run.smithi094.stdout:4016 write 0x6c9a461 thru 0x6ca4496 (0xa036 bytes) 2024-09-14T06:31:02.873 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-14T06:31:02.873 INFO:teuthology.orchestra.run.smithi094.stdout:4018 punch from 0x38e6b40 to 0x38ed5cc, (0x6a8c bytes) 2024-09-14T06:31:02.878 INFO:teuthology.orchestra.run.smithi094.stdout:4019 trunc from 0x90ab31a to 0x4c7aa72 2024-09-14T06:31:02.894 INFO:teuthology.orchestra.run.smithi094.stdout:4020 punch from 0x36c56f5 to 0x36d3a02, (0xe30d bytes) 2024-09-14T06:31:02.909 INFO:teuthology.orchestra.run.smithi094.stdout:4021 write 0xca39da3 thru 0xca3dcf6 (0x3f54 bytes) 2024-09-14T06:31:02.923 INFO:teuthology.orchestra.run.smithi094.stdout:4022 write 0xe7a2a21 thru 0xe7b17a3 (0xed83 bytes) 2024-09-14T06:31:02.932 INFO:teuthology.orchestra.run.smithi094.stdout:4023 write 0x737ecc3 thru 0x7387c66 (0x8fa4 bytes) 2024-09-14T06:31:02.940 INFO:teuthology.orchestra.run.smithi094.stdout:4025 write 0x5c954d1 thru 0x5c96154 (0xc84 bytes) 2024-09-14T06:31:02.944 INFO:teuthology.orchestra.run.smithi094.stdout:4026 writesame 0x334b53e thru 0x335230e data_size 0x1(0x6dd1 bytes) 2024-09-14T06:31:02.950 INFO:teuthology.orchestra.run.smithi094.stdout:4029 write 0x74ae781 thru 0x74b6f5a (0x87da bytes) 2024-09-14T06:31:02.953 INFO:teuthology.orchestra.run.smithi094.stdout:4030 punch from 0xc26867b to 0xc270c92, (0x8617 bytes) 2024-09-14T06:31:02.954 INFO:teuthology.orchestra.run.smithi094.stdout:4032 writesame 0xabcd840 thru 0xabd7a93 data_size 0x6(0xa254 bytes) 2024-09-14T06:31:02.963 INFO:teuthology.orchestra.run.smithi094.stdout:4034 punch from 0xb2edac7 to 0xb2f32c4, (0x57fd bytes) 2024-09-14T06:31:02.963 INFO:teuthology.orchestra.run.smithi094.stdout:4035 punch from 0x545d29b to 0x5461cc1, (0x4a26 bytes) 2024-09-14T06:31:02.963 INFO:tasks.ceph.osd.0.smithi094.stderr:2024-09-14T06:31:02.962+0000 7f07c9333640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33636) UID: 0 2024-09-14T06:31:02.970 INFO:teuthology.orchestra.run.smithi094.stdout:4036 trunc from 0xe7b17a4 to 0x84dbb32 2024-09-14T06:31:02.985 INFO:teuthology.orchestra.run.smithi094.stdout:4037 read 0x11836fc thru 0x118e85a (0xb15f bytes) 2024-09-14T06:31:02.985 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-14T06:31:02.989 INFO:teuthology.orchestra.run.smithi094.stdout:4039 trunc from 0x84dbb32 to 0x560ef5a 2024-09-14T06:31:02.997 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-14T06:31:02.997 INFO:teuthology.orchestra.run.smithi094.stdout: "size": 20, 2024-09-14T06:31:02.997 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 600, 2024-09-14T06:31:02.997 INFO:teuthology.orchestra.run.smithi094.stdout: "ops": [ 2024-09-14T06:31:02.997 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:02.997 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:44425 3.2 3:4603ad7f:::rbd_data.112dca2f6ae1.0000000000000005:17 [read 868352~4096 [fadvise_nocache]] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:31:02.997 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:43.854967+0000", 2024-09-14T06:31:02.997 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 79.140064469999999, 2024-09-14T06:31:02.997 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.063146025999999994, 2024-09-14T06:31:02.997 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:31:02.998 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:31:02.998 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:31:02.998 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:31:02.998 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:31:02.998 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:02.998 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 44425 2024-09-14T06:31:02.998 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:02.998 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:31:02.998 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:02.998 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:31:02.998 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.854967+0000", 2024-09-14T06:31:02.998 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:02.998 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:02.998 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:02.999 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:31:02.999 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.854967+0000", 2024-09-14T06:31:02.999 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:02.999 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:02.999 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:02.999 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:31:02.999 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.854967+0000", 2024-09-14T06:31:02.999 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.16e-07 2024-09-14T06:31:02.999 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:02.999 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:02.999 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:31:02.999 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.854968+0000", 2024-09-14T06:31:02.999 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.006e-06 2024-09-14T06:31:02.999 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:03.000 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:03.000 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:31:03.000 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.854971+0000", 2024-09-14T06:31:03.000 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.9359999999999999e-06 2024-09-14T06:31:03.000 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:03.000 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:03.000 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:31:03.000 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.854974+0000", 2024-09-14T06:31:03.000 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.1840000000000002e-06 2024-09-14T06:31:03.000 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:03.000 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:03.000 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:31:03.001 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.917945+0000", 2024-09-14T06:31:03.001 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.062970925999999997 2024-09-14T06:31:03.001 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:03.001 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:03.001 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:31:03.001 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.918030+0000", 2024-09-14T06:31:03.001 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.5863000000000001e-05 2024-09-14T06:31:03.001 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:03.001 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:03.001 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:31:03.001 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.918113+0000", 2024-09-14T06:31:03.001 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.2894999999999998e-05 2024-09-14T06:31:03.001 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:03.002 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:31:03.002 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:03.002 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:03.002 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:03.002 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:44426 3.2 3:4603ad7f:::rbd_data.112dca2f6ae1.0000000000000005:17 [read 892928~4096 [fadvise_nocache]] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:31:03.002 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:43.854977+0000", 2024-09-14T06:31:03.002 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 79.140055059999995, 2024-09-14T06:31:03.002 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.063369237999999994, 2024-09-14T06:31:03.002 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:31:03.002 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:31:03.002 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:31:03.002 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:31:03.002 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:31:03.002 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:03.003 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 44426 2024-09-14T06:31:03.003 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:03.003 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:31:03.003 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:03.003 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:31:03.003 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.854977+0000", 2024-09-14T06:31:03.003 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:03.003 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:03.003 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:03.003 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:31:03.003 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.854977+0000", 2024-09-14T06:31:03.003 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:03.003 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:03.003 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:03.004 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:31:03.004 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.854977+0000", 2024-09-14T06:31:03.004 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.1199999999999999e-07 2024-09-14T06:31:03.004 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:03.004 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:03.004 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:31:03.004 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.854980+0000", 2024-09-14T06:31:03.004 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.27e-06 2024-09-14T06:31:03.004 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:03.004 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:03.004 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:31:03.004 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.854983+0000", 2024-09-14T06:31:03.004 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.1650000000000002e-06 2024-09-14T06:31:03.004 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:03.004 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:03.005 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:31:03.005 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.854989+0000", 2024-09-14T06:31:03.005 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.6520000000000003e-06 2024-09-14T06:31:03.005 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:03.005 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:03.005 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:31:03.005 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.918143+0000", 2024-09-14T06:31:03.005 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.063154541999999994 2024-09-14T06:31:03.005 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:03.005 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:03.005 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:31:03.005 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.918209+0000", 2024-09-14T06:31:03.005 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.5529000000000002e-05 2024-09-14T06:31:03.006 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:03.006 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:03.006 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:31:03.006 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.918346+0000", 2024-09-14T06:31:03.006 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000136868 2024-09-14T06:31:03.006 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:03.006 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:31:03.006 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:03.006 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:03.006 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:03.006 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:44427 3.2 3:4603ad7f:::rbd_data.112dca2f6ae1.0000000000000005:17 [read 917504~4096 [fadvise_nocache]] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:31:03.006 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:43.854992+0000", 2024-09-14T06:31:03.006 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 79.140039414, 2024-09-14T06:31:03.006 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.063607792999999996, 2024-09-14T06:31:03.006 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:31:03.007 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:31:03.007 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:31:03.007 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:31:03.007 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:31:03.007 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:03.007 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 44427 2024-09-14T06:31:03.007 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:03.007 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:31:03.007 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:03.007 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:31:03.007 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.854992+0000", 2024-09-14T06:31:03.007 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:03.007 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:03.007 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:03.008 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:31:03.008 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.854992+0000", 2024-09-14T06:31:03.008 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:03.008 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:03.008 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:03.008 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:31:03.008 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.854993+0000", 2024-09-14T06:31:03.008 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.1400000000000001e-07 2024-09-14T06:31:03.008 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:03.008 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:03.008 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:31:03.008 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.854994+0000", 2024-09-14T06:31:03.008 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.091e-06 2024-09-14T06:31:03.008 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:03.009 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:03.009 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:31:03.009 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855001+0000", 2024-09-14T06:31:03.009 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.204e-06 2024-09-14T06:31:03.009 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:03.009 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:03.009 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:31:03.009 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855003+0000", 2024-09-14T06:31:03.009 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.514e-06 2024-09-14T06:31:03.009 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:03.009 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:03.009 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:31:03.009 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.918380+0000", 2024-09-14T06:31:03.009 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.063376738000000002 2024-09-14T06:31:03.009 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:03.010 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:03.010 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:31:03.010 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.918453+0000", 2024-09-14T06:31:03.010 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.3291e-05 2024-09-14T06:31:03.010 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:03.010 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:03.010 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:31:03.010 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.918600+0000", 2024-09-14T06:31:03.010 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00014674100000000001 2024-09-14T06:31:03.010 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:03.010 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:31:03.010 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:03.010 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:03.010 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:03.011 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:44428 3.2 3:4603ad7f:::rbd_data.112dca2f6ae1.0000000000000005:17 [read 942080~4096 [fadvise_nocache]] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:31:03.011 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:43.855006+0000", 2024-09-14T06:31:03.011 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 79.140025363999996, 2024-09-14T06:31:03.011 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.063846321999999997, 2024-09-14T06:31:03.011 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:31:03.011 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:31:03.011 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:31:03.011 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:31:03.011 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:31:03.011 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:03.011 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 44428 2024-09-14T06:31:03.011 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:03.011 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:31:03.012 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:03.012 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:31:03.012 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855006+0000", 2024-09-14T06:31:03.012 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:03.012 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:03.012 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:03.012 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:31:03.012 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855006+0000", 2024-09-14T06:31:03.012 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:03.012 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:03.012 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:03.012 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:31:03.012 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855007+0000", 2024-09-14T06:31:03.013 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.1199999999999999e-07 2024-09-14T06:31:03.013 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:03.013 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:03.013 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:31:03.013 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855011+0000", 2024-09-14T06:31:03.013 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.9539999999999998e-06 2024-09-14T06:31:03.013 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:03.013 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:03.013 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:31:03.013 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855014+0000", 2024-09-14T06:31:03.013 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.2930000000000001e-06 2024-09-14T06:31:03.013 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:03.013 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:03.014 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:31:03.014 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855016+0000", 2024-09-14T06:31:03.014 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.3039999999999999e-06 2024-09-14T06:31:03.014 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:03.014 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:03.014 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:31:03.014 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.918634+0000", 2024-09-14T06:31:03.014 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.063617928000000004 2024-09-14T06:31:03.014 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:03.014 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:03.014 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:31:03.014 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.918711+0000", 2024-09-14T06:31:03.014 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.6545000000000007e-05 2024-09-14T06:31:03.014 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:03.015 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:03.015 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:31:03.015 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.918853+0000", 2024-09-14T06:31:03.015 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00014208600000000001 2024-09-14T06:31:03.015 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:03.015 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:31:03.015 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:03.015 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:03.015 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:03.015 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:44429 3.2 3:4603ad7f:::rbd_data.112dca2f6ae1.0000000000000005:17 [read 966656~4096 [fadvise_nocache]] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:31:03.015 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:43.855019+0000", 2024-09-14T06:31:03.015 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 79.140012690999995, 2024-09-14T06:31:03.015 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.064059226999999996, 2024-09-14T06:31:03.015 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:31:03.015 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:31:03.016 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:31:03.016 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:31:03.016 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:31:03.016 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:03.016 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 44429 2024-09-14T06:31:03.016 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:03.016 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:31:03.016 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:03.016 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:31:03.016 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855019+0000", 2024-09-14T06:31:03.016 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:03.016 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:03.016 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:03.016 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:31:03.017 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855019+0000", 2024-09-14T06:31:03.017 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:03.017 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:03.017 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:03.017 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:31:03.017 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855019+0000", 2024-09-14T06:31:03.017 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.28e-07 2024-09-14T06:31:03.017 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:03.017 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:03.017 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:31:03.017 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855020+0000", 2024-09-14T06:31:03.017 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.0330000000000001e-06 2024-09-14T06:31:03.017 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:03.017 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:03.018 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:31:03.018 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855023+0000", 2024-09-14T06:31:03.018 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.79e-06 2024-09-14T06:31:03.018 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:03.018 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:03.018 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:31:03.018 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855025+0000", 2024-09-14T06:31:03.018 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.216e-06 2024-09-14T06:31:03.018 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:03.018 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:03.018 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:31:03.018 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.918886+0000", 2024-09-14T06:31:03.018 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.063861037999999995 2024-09-14T06:31:03.018 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:03.019 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:03.019 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:31:03.019 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.918952+0000", 2024-09-14T06:31:03.019 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.5993000000000003e-05 2024-09-14T06:31:03.019 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:03.019 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:03.019 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:31:03.019 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.919078+0000", 2024-09-14T06:31:03.019 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00012592899999999999 2024-09-14T06:31:03.019 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:03.019 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:31:03.019 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:03.019 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:03.019 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:03.020 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:44430 3.2 3:4603ad7f:::rbd_data.112dca2f6ae1.0000000000000005:17 [read 991232~4096 [fadvise_nocache]] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:31:03.020 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:43.855028+0000", 2024-09-14T06:31:03.020 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 79.140003680999996, 2024-09-14T06:31:03.020 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.064284346000000006, 2024-09-14T06:31:03.020 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:31:03.020 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:31:03.020 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:31:03.020 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:31:03.020 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:31:03.020 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:03.020 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 44430 2024-09-14T06:31:03.020 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:03.020 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:31:03.020 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:03.021 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:31:03.021 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855028+0000", 2024-09-14T06:31:03.021 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:03.021 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:03.021 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:03.021 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:31:03.021 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855028+0000", 2024-09-14T06:31:03.021 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:03.021 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:03.021 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:03.021 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:31:03.021 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855028+0000", 2024-09-14T06:31:03.021 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.1199999999999999e-07 2024-09-14T06:31:03.021 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:03.022 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:03.022 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:31:03.022 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855029+0000", 2024-09-14T06:31:03.022 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.1100000000000004e-07 2024-09-14T06:31:03.022 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:03.022 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:03.022 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:31:03.022 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855032+0000", 2024-09-14T06:31:03.022 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.7889999999999999e-06 2024-09-14T06:31:03.022 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:03.022 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:03.022 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:31:03.022 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855034+0000", 2024-09-14T06:31:03.022 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.125e-06 2024-09-14T06:31:03.023 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:03.023 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:03.023 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:31:03.023 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.919110+0000", 2024-09-14T06:31:03.023 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.064076413999999998 2024-09-14T06:31:03.023 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:03.023 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:03.023 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:31:03.023 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.919186+0000", 2024-09-14T06:31:03.023 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.5784999999999994e-05 2024-09-14T06:31:03.023 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:03.023 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:03.023 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:31:03.023 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.919312+0000", 2024-09-14T06:31:03.023 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00012611 2024-09-14T06:31:03.024 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:03.024 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:31:03.024 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:03.024 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:03.024 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:03.024 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:44431 3.2 3:4603ad7f:::rbd_data.112dca2f6ae1.0000000000000005:17 [read 1015808~4096 [fadvise_nocache]] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:31:03.024 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:43.855037+0000", 2024-09-14T06:31:03.024 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 79.139994813000001, 2024-09-14T06:31:03.024 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.064469755000000004, 2024-09-14T06:31:03.024 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:31:03.024 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:31:03.024 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:31:03.024 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:31:03.025 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:31:03.025 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:03.025 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 44431 2024-09-14T06:31:03.025 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:03.025 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:31:03.025 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:03.025 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:31:03.025 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855037+0000", 2024-09-14T06:31:03.025 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:03.025 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:03.025 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:03.025 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:31:03.025 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855037+0000", 2024-09-14T06:31:03.025 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:03.026 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:03.026 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:03.026 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:31:03.026 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855037+0000", 2024-09-14T06:31:03.026 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.1e-07 2024-09-14T06:31:03.026 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:03.026 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:03.026 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:31:03.026 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855038+0000", 2024-09-14T06:31:03.026 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.9400000000000004e-07 2024-09-14T06:31:03.026 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:03.026 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:03.026 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:31:03.026 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855041+0000", 2024-09-14T06:31:03.027 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.9639999999999999e-06 2024-09-14T06:31:03.027 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:03.027 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:03.027 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:31:03.027 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855043+0000", 2024-09-14T06:31:03.027 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.2259999999999999e-06 2024-09-14T06:31:03.027 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:03.027 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:03.027 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:31:03.027 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.919347+0000", 2024-09-14T06:31:03.027 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.064304091999999993 2024-09-14T06:31:03.027 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:03.027 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:03.027 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:31:03.027 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.919393+0000", 2024-09-14T06:31:03.028 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.5340999999999998e-05 2024-09-14T06:31:03.028 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:03.028 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:03.028 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:31:03.028 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.919507+0000", 2024-09-14T06:31:03.028 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000114028 2024-09-14T06:31:03.028 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:03.028 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:31:03.028 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:03.028 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:03.028 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:03.028 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:44432 3.2 3:4603ad7f:::rbd_data.112dca2f6ae1.0000000000000005:17 [read 1040384~4096 [fadvise_nocache]] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:31:03.028 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:43.855046+0000", 2024-09-14T06:31:03.028 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 79.139985745000004, 2024-09-14T06:31:03.029 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.064639253999999993, 2024-09-14T06:31:03.029 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:31:03.029 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:31:03.029 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:31:03.029 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:31:03.029 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:31:03.029 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:03.029 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 44432 2024-09-14T06:31:03.029 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:03.029 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:31:03.029 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:03.029 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:31:03.029 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855046+0000", 2024-09-14T06:31:03.029 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:03.030 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:03.030 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:03.030 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:31:03.030 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855046+0000", 2024-09-14T06:31:03.030 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:03.030 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:03.030 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:03.030 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:31:03.030 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855046+0000", 2024-09-14T06:31:03.030 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.05e-07 2024-09-14T06:31:03.030 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:03.030 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:03.030 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:31:03.030 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855047+0000", 2024-09-14T06:31:03.030 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.879999999999999e-07 2024-09-14T06:31:03.031 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:03.031 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:03.031 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:31:03.031 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855050+0000", 2024-09-14T06:31:03.031 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.8459999999999999e-06 2024-09-14T06:31:03.031 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:03.031 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:03.031 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:31:03.031 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855052+0000", 2024-09-14T06:31:03.031 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.136e-06 2024-09-14T06:31:03.031 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:03.031 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:03.031 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:31:03.031 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.919536+0000", 2024-09-14T06:31:03.031 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.064484132 2024-09-14T06:31:03.032 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:03.032 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:03.032 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:31:03.032 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.919611+0000", 2024-09-14T06:31:03.032 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.4407999999999996e-05 2024-09-14T06:31:03.032 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:03.032 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:03.032 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:31:03.032 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.919685+0000", 2024-09-14T06:31:03.032 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.4538999999999999e-05 2024-09-14T06:31:03.032 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:03.032 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:31:03.032 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:03.032 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:03.032 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:03.033 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:44518 3.2 3:4190d882:::rbd_data.112dca2f6ae1.0000000000000002:17 [read 12288~4096 [fadvise_nocache]] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:31:03.033 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:43.855055+0000", 2024-09-14T06:31:03.033 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 79.139976860999994, 2024-09-14T06:31:03.033 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.064894317000000007, 2024-09-14T06:31:03.033 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:31:03.033 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:31:03.033 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:31:03.033 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:31:03.033 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:31:03.033 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:03.033 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 44518 2024-09-14T06:31:03.033 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:03.033 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:31:03.033 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:03.033 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:31:03.034 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855055+0000", 2024-09-14T06:31:03.034 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:03.034 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:03.034 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:03.034 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:31:03.034 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855055+0000", 2024-09-14T06:31:03.034 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:03.034 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:03.034 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:03.034 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:31:03.034 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855055+0000", 2024-09-14T06:31:03.034 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.0699999999999999e-07 2024-09-14T06:31:03.034 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:03.034 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:03.034 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:31:03.034 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855056+0000", 2024-09-14T06:31:03.035 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.9400000000000004e-07 2024-09-14T06:31:03.035 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:03.035 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:03.035 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:31:03.035 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855059+0000", 2024-09-14T06:31:03.035 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.8870000000000001e-06 2024-09-14T06:31:03.035 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:03.035 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:03.035 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:31:03.035 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855061+0000", 2024-09-14T06:31:03.035 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.1919999999999999e-06 2024-09-14T06:31:03.035 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:03.035 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:03.035 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:31:03.036 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.919719+0000", 2024-09-14T06:31:03.036 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.064658013 2024-09-14T06:31:03.036 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:03.036 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:03.036 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:31:03.036 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.919788+0000", 2024-09-14T06:31:03.036 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.9282000000000005e-05 2024-09-14T06:31:03.036 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:03.036 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:03.036 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:31:03.036 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.919949+0000", 2024-09-14T06:31:03.036 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00016084199999999999 2024-09-14T06:31:03.036 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:03.036 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:31:03.036 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:03.037 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:03.037 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:03.037 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:44519 3.2 3:4190d882:::rbd_data.112dca2f6ae1.0000000000000002:17 [read 36864~4096 [fadvise_nocache]] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:31:03.037 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:43.855064+0000", 2024-09-14T06:31:03.037 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 79.139967884000001, 2024-09-14T06:31:03.037 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.065081865000000003, 2024-09-14T06:31:03.037 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:31:03.037 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:31:03.037 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:31:03.037 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:31:03.037 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:31:03.037 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:03.037 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 44519 2024-09-14T06:31:03.037 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:03.037 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:31:03.038 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:03.038 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:31:03.038 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855064+0000", 2024-09-14T06:31:03.038 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:03.038 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:03.038 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:03.038 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:31:03.038 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855064+0000", 2024-09-14T06:31:03.038 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:03.038 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:03.038 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:03.038 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:31:03.038 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855064+0000", 2024-09-14T06:31:03.038 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.1e-07 2024-09-14T06:31:03.038 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:03.039 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:03.039 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:31:03.039 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855065+0000", 2024-09-14T06:31:03.039 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.035e-06 2024-09-14T06:31:03.039 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:03.039 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:03.039 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:31:03.039 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855068+0000", 2024-09-14T06:31:03.039 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.8339999999999999e-06 2024-09-14T06:31:03.039 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:03.039 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:03.039 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:31:03.039 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855071+0000", 2024-09-14T06:31:03.039 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.3450000000000002e-06 2024-09-14T06:31:03.039 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:03.039 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:03.040 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:31:03.040 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.919979+0000", 2024-09-14T06:31:03.040 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.064907647999999998 2024-09-14T06:31:03.040 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:03.040 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:03.040 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:31:03.040 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.920026+0000", 2024-09-14T06:31:03.040 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.7595000000000001e-05 2024-09-14T06:31:03.040 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:03.040 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:03.040 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:31:03.040 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.920146+0000", 2024-09-14T06:31:03.040 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00011919800000000001 2024-09-14T06:31:03.041 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:03.041 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:31:03.041 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:03.041 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:03.041 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:03.041 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:44520 3.2 3:4190d882:::rbd_data.112dca2f6ae1.0000000000000002:17 [read 61440~4096 [fadvise_nocache]] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:31:03.041 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:43.855075+0000", 2024-09-14T06:31:03.041 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 79.139956832999999, 2024-09-14T06:31:03.041 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.065335078000000005, 2024-09-14T06:31:03.041 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:31:03.041 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:31:03.041 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:31:03.041 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:31:03.041 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:31:03.041 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:03.042 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 44520 2024-09-14T06:31:03.042 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:03.042 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:31:03.042 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:03.042 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:31:03.042 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855075+0000", 2024-09-14T06:31:03.042 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:03.042 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:03.042 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:03.042 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:31:03.042 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855075+0000", 2024-09-14T06:31:03.042 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:03.042 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:03.042 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:03.043 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:31:03.043 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855075+0000", 2024-09-14T06:31:03.043 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.1e-07 2024-09-14T06:31:03.043 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:03.043 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:03.043 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:31:03.043 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855076+0000", 2024-09-14T06:31:03.043 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.0839999999999999e-06 2024-09-14T06:31:03.043 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:03.043 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:03.043 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:31:03.043 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855079+0000", 2024-09-14T06:31:03.043 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.9359999999999999e-06 2024-09-14T06:31:03.043 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:03.043 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:03.043 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:31:03.044 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855084+0000", 2024-09-14T06:31:03.044 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.1730000000000003e-06 2024-09-14T06:31:03.044 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:03.044 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:03.044 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:31:03.044 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.920175+0000", 2024-09-14T06:31:03.044 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.065090234999999996 2024-09-14T06:31:03.044 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:03.044 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:03.044 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:31:03.044 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.920250+0000", 2024-09-14T06:31:03.044 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.5807000000000001e-05 2024-09-14T06:31:03.044 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:03.044 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:03.044 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:31:03.045 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.920410+0000", 2024-09-14T06:31:03.045 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00015963300000000001 2024-09-14T06:31:03.045 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:03.045 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:31:03.045 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:03.045 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:03.045 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:03.045 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:44521 3.2 3:4190d882:::rbd_data.112dca2f6ae1.0000000000000002:17 [read 86016~4096 [fadvise_nocache]] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:31:03.045 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:43.855087+0000", 2024-09-14T06:31:03.045 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 79.139944352000001, 2024-09-14T06:31:03.045 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.065563264999999996, 2024-09-14T06:31:03.045 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:31:03.045 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:31:03.045 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:31:03.045 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:31:03.046 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:31:03.046 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:03.046 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 44521 2024-09-14T06:31:03.046 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:03.046 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:31:03.046 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:03.046 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:31:03.046 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855087+0000", 2024-09-14T06:31:03.046 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:03.046 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:03.046 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:03.046 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:31:03.046 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855087+0000", 2024-09-14T06:31:03.046 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:03.047 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:03.047 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:03.047 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:31:03.047 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855088+0000", 2024-09-14T06:31:03.047 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.3699999999999999e-07 2024-09-14T06:31:03.047 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:03.047 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:03.047 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:31:03.047 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855089+0000", 2024-09-14T06:31:03.047 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.9900000000000009e-07 2024-09-14T06:31:03.047 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:03.047 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:03.047 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:31:03.047 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855091+0000", 2024-09-14T06:31:03.048 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.8940000000000002e-06 2024-09-14T06:31:03.048 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:03.048 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:03.048 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:31:03.048 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855094+0000", 2024-09-14T06:31:03.048 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.351e-06 2024-09-14T06:31:03.048 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:03.048 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:03.048 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:31:03.048 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.920437+0000", 2024-09-14T06:31:03.048 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.065343221000000007 2024-09-14T06:31:03.048 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:03.048 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:03.048 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:31:03.048 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.920481+0000", 2024-09-14T06:31:03.049 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.4431000000000002e-05 2024-09-14T06:31:03.049 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:03.049 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:03.049 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:31:03.049 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.920651+0000", 2024-09-14T06:31:03.049 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00016913200000000001 2024-09-14T06:31:03.049 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:03.049 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:31:03.049 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:03.049 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:03.049 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:03.049 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:44522 3.2 3:4190d882:::rbd_data.112dca2f6ae1.0000000000000002:17 [read 110592~4096 [fadvise_nocache]] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:31:03.049 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:43.855098+0000", 2024-09-14T06:31:03.050 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 79.139934124999996, 2024-09-14T06:31:03.050 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.065827628999999999, 2024-09-14T06:31:03.050 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:31:03.050 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:31:03.050 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:31:03.050 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:31:03.050 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:31:03.050 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:03.050 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 44522 2024-09-14T06:31:03.050 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:03.050 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:31:03.050 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:03.050 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:31:03.050 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855098+0000", 2024-09-14T06:31:03.050 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:03.050 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:03.051 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:03.051 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:31:03.051 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855098+0000", 2024-09-14T06:31:03.051 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:03.051 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:03.051 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:03.051 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:31:03.051 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855098+0000", 2024-09-14T06:31:03.051 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.1e-07 2024-09-14T06:31:03.051 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:03.051 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:03.051 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:31:03.051 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855099+0000", 2024-09-14T06:31:03.051 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.012e-06 2024-09-14T06:31:03.051 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:03.052 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:03.052 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:31:03.052 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855102+0000", 2024-09-14T06:31:03.052 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.9380000000000001e-06 2024-09-14T06:31:03.052 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:03.052 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:03.052 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:31:03.052 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855108+0000", 2024-09-14T06:31:03.052 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.4339999999999997e-06 2024-09-14T06:31:03.052 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:03.052 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:03.052 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:31:03.052 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.920681+0000", 2024-09-14T06:31:03.052 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.065572943999999994 2024-09-14T06:31:03.052 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:03.053 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:03.053 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:31:03.053 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.920724+0000", 2024-09-14T06:31:03.053 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.2877000000000003e-05 2024-09-14T06:31:03.053 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:03.053 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:03.053 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:31:03.053 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.920925+0000", 2024-09-14T06:31:03.053 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00020121399999999999 2024-09-14T06:31:03.053 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:03.053 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:31:03.053 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:03.053 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:03.053 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:03.053 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:44523 3.2 3:4190d882:::rbd_data.112dca2f6ae1.0000000000000002:17 [read 135168~4096 [fadvise_nocache]] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:31:03.054 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:43.855111+0000", 2024-09-14T06:31:03.054 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 79.139920403000005, 2024-09-14T06:31:03.054 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.066028198999999996, 2024-09-14T06:31:03.054 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:31:03.054 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:31:03.054 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:31:03.054 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:31:03.054 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:31:03.054 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:03.054 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 44523 2024-09-14T06:31:03.054 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:03.054 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:31:03.054 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:03.054 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:31:03.054 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855111+0000", 2024-09-14T06:31:03.055 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:03.055 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:03.055 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:03.055 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:31:03.055 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855111+0000", 2024-09-14T06:31:03.055 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:03.055 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:03.055 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:03.055 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:31:03.055 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855112+0000", 2024-09-14T06:31:03.055 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.1199999999999999e-07 2024-09-14T06:31:03.055 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:03.055 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:03.055 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:31:03.055 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855115+0000", 2024-09-14T06:31:03.055 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.3380000000000001e-06 2024-09-14T06:31:03.056 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:03.056 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:03.056 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:31:03.056 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855118+0000", 2024-09-14T06:31:03.056 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.366e-06 2024-09-14T06:31:03.056 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:03.056 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:03.056 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:31:03.056 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855121+0000", 2024-09-14T06:31:03.056 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.8329999999999998e-06 2024-09-14T06:31:03.056 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:03.057 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:03.057 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:31:03.057 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.920956+0000", 2024-09-14T06:31:03.057 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.065835251999999997 2024-09-14T06:31:03.057 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:03.057 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:03.057 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:31:03.057 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.921001+0000", 2024-09-14T06:31:03.057 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.4778999999999999e-05 2024-09-14T06:31:03.057 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:03.057 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:03.057 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:31:03.057 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.921139+0000", 2024-09-14T06:31:03.058 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000138419 2024-09-14T06:31:03.058 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:03.058 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:31:03.058 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:03.058 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:03.058 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:03.058 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:44524 3.2 3:4190d882:::rbd_data.112dca2f6ae1.0000000000000002:17 [read 159744~4096 [fadvise_nocache]] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:31:03.058 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:43.855124+0000", 2024-09-14T06:31:03.058 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 79.139907424, 2024-09-14T06:31:03.058 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.066249261000000004, 2024-09-14T06:31:03.058 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:31:03.058 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:31:03.058 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:31:03.058 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:31:03.058 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:31:03.059 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:03.059 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 44524 2024-09-14T06:31:03.059 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:03.059 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:31:03.059 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:03.059 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:31:03.059 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855124+0000", 2024-09-14T06:31:03.059 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:03.059 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:03.059 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:03.059 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:31:03.059 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855124+0000", 2024-09-14T06:31:03.059 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:03.059 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:03.060 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:03.060 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:31:03.060 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861086+0000", 2024-09-14T06:31:03.060 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0059613829999999998 2024-09-14T06:31:03.060 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:03.060 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:03.060 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:31:03.060 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861100+0000", 2024-09-14T06:31:03.060 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.4644000000000001e-05 2024-09-14T06:31:03.060 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:03.060 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:03.060 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:31:03.060 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861109+0000", 2024-09-14T06:31:03.060 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.541e-06 2024-09-14T06:31:03.060 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:03.060 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:03.061 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:31:03.061 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861117+0000", 2024-09-14T06:31:03.061 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.1049999999999995e-06 2024-09-14T06:31:03.061 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:03.061 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:03.061 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:31:03.061 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.921169+0000", 2024-09-14T06:31:03.061 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.060052405000000003 2024-09-14T06:31:03.061 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:03.061 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:03.061 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:31:03.061 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.921213+0000", 2024-09-14T06:31:03.061 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.3565e-05 2024-09-14T06:31:03.061 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:03.062 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:03.062 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:31:03.062 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.921374+0000", 2024-09-14T06:31:03.062 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00016061800000000001 2024-09-14T06:31:03.062 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:03.062 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:31:03.062 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:03.062 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:03.062 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:03.062 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:44536 3.2 3:4190d882:::rbd_data.112dca2f6ae1.0000000000000002:17 [read 454656~4096 [fadvise_nocache]] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:31:03.062 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:43.861384+0000", 2024-09-14T06:31:03.062 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 79.133647866999993, 2024-09-14T06:31:03.062 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.063058528000000003, 2024-09-14T06:31:03.062 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:31:03.062 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:31:03.062 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:31:03.063 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:31:03.063 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:31:03.063 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:03.063 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 44536 2024-09-14T06:31:03.063 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:03.063 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:31:03.063 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:03.063 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:31:03.063 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861384+0000", 2024-09-14T06:31:03.063 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:03.063 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:03.063 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:03.063 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:31:03.063 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861384+0000", 2024-09-14T06:31:03.063 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:03.063 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:03.064 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:03.064 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:31:03.064 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861384+0000", 2024-09-14T06:31:03.064 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.4000000000000002e-07 2024-09-14T06:31:03.064 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:03.064 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:03.064 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:31:03.064 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861386+0000", 2024-09-14T06:31:03.064 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.891e-06 2024-09-14T06:31:03.064 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:03.064 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:03.064 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:31:03.064 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861392+0000", 2024-09-14T06:31:03.064 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.9939999999999997e-06 2024-09-14T06:31:03.065 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:03.065 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:03.065 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:31:03.065 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861398+0000", 2024-09-14T06:31:03.065 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.507e-06 2024-09-14T06:31:03.065 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:03.065 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:03.065 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:31:03.065 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.924149+0000", 2024-09-14T06:31:03.065 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.062751135999999999 2024-09-14T06:31:03.065 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:03.065 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:03.065 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:31:03.065 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.924226+0000", 2024-09-14T06:31:03.066 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.7517999999999999e-05 2024-09-14T06:31:03.066 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:03.066 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:03.066 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:31:03.066 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.924442+0000", 2024-09-14T06:31:03.066 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000216042 2024-09-14T06:31:03.066 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:03.066 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:31:03.066 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:03.066 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:03.066 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:03.066 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:44537 3.2 3:4190d882:::rbd_data.112dca2f6ae1.0000000000000002:17 [read 479232~4096 [fadvise_nocache]] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:31:03.066 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:43.861405+0000", 2024-09-14T06:31:03.066 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 79.133626821000007, 2024-09-14T06:31:03.067 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.063351795000000002, 2024-09-14T06:31:03.067 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:31:03.067 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:31:03.067 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:31:03.067 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:31:03.067 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:31:03.067 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:03.067 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 44537 2024-09-14T06:31:03.067 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:03.067 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:31:03.067 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:03.067 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:31:03.067 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861405+0000", 2024-09-14T06:31:03.067 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:03.068 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:03.068 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:03.068 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:31:03.068 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861405+0000", 2024-09-14T06:31:03.068 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:03.068 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:03.068 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:03.068 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:31:03.068 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861405+0000", 2024-09-14T06:31:03.068 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.9099999999999999e-07 2024-09-14T06:31:03.068 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:03.068 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:03.068 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:31:03.068 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861407+0000", 2024-09-14T06:31:03.068 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.075e-06 2024-09-14T06:31:03.068 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:03.069 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:03.069 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:31:03.069 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861414+0000", 2024-09-14T06:31:03.069 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.4659999999999996e-06 2024-09-14T06:31:03.069 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:03.069 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:03.069 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:31:03.069 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861418+0000", 2024-09-14T06:31:03.069 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.5519999999999998e-06 2024-09-14T06:31:03.069 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:03.069 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:03.069 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:31:03.069 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.924474+0000", 2024-09-14T06:31:03.069 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.063056012999999994 2024-09-14T06:31:03.069 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:03.070 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:03.070 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:31:03.070 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.924556+0000", 2024-09-14T06:31:03.070 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.1181999999999996e-05 2024-09-14T06:31:03.070 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:03.070 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:03.070 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:31:03.070 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.924757+0000", 2024-09-14T06:31:03.070 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00020111599999999999 2024-09-14T06:31:03.070 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:03.070 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:31:03.070 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:03.070 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:03.070 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:03.070 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:44538 3.2 3:4190d882:::rbd_data.112dca2f6ae1.0000000000000002:17 [read 503808~4096 [fadvise_nocache]] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:31:03.070 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:43.861425+0000", 2024-09-14T06:31:03.070 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 79.133606779999994, 2024-09-14T06:31:03.071 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.063627125000000007, 2024-09-14T06:31:03.071 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:31:03.071 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:31:03.071 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:31:03.071 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:31:03.071 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:31:03.071 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:03.071 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 44538 2024-09-14T06:31:03.071 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:03.071 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:31:03.071 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:03.071 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:31:03.071 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861425+0000", 2024-09-14T06:31:03.071 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:03.071 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:03.071 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:03.072 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:31:03.072 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861425+0000", 2024-09-14T06:31:03.072 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:03.072 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:03.072 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:03.072 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:31:03.072 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861425+0000", 2024-09-14T06:31:03.072 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.9999999999999998e-07 2024-09-14T06:31:03.072 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:03.072 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:03.072 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:31:03.072 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861427+0000", 2024-09-14T06:31:03.072 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.9829999999999999e-06 2024-09-14T06:31:03.073 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:03.073 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:03.073 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:31:03.073 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861433+0000", 2024-09-14T06:31:03.073 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.2889999999999997e-06 2024-09-14T06:31:03.073 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:03.073 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:03.073 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:31:03.073 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861454+0000", 2024-09-14T06:31:03.073 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.1844999999999998e-05 2024-09-14T06:31:03.073 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:03.073 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:03.073 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:31:03.073 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.924786+0000", 2024-09-14T06:31:03.073 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.063331497 2024-09-14T06:31:03.074 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:03.074 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:03.074 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:31:03.074 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.924864+0000", 2024-09-14T06:31:03.074 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.7744000000000006e-05 2024-09-14T06:31:03.074 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:03.074 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:03.074 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:31:03.074 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.925052+0000", 2024-09-14T06:31:03.074 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00018836699999999999 2024-09-14T06:31:03.074 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:03.074 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:31:03.074 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:03.074 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:03.074 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:03.074 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:44539 3.2 3:4190d882:::rbd_data.112dca2f6ae1.0000000000000002:17 [read 528384~4096 [fadvise_nocache]] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:31:03.075 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:43.861462+0000", 2024-09-14T06:31:03.075 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 79.133569453000007, 2024-09-14T06:31:03.075 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.063855705999999998, 2024-09-14T06:31:03.075 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:31:03.075 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:31:03.075 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:31:03.075 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:31:03.075 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:31:03.075 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:03.075 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 44539 2024-09-14T06:31:03.075 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:03.075 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:31:03.075 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:03.075 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:31:03.076 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861462+0000", 2024-09-14T06:31:03.076 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:03.076 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:03.076 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:03.076 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:31:03.076 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861462+0000", 2024-09-14T06:31:03.076 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:03.076 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:03.076 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:03.076 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:31:03.076 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861463+0000", 2024-09-14T06:31:03.076 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.5200000000000002e-07 2024-09-14T06:31:03.076 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:03.076 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:03.077 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:31:03.077 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861465+0000", 2024-09-14T06:31:03.077 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.5189999999999999e-06 2024-09-14T06:31:03.077 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:03.077 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:03.077 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:31:03.077 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861471+0000", 2024-09-14T06:31:03.077 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.6899999999999997e-06 2024-09-14T06:31:03.077 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:03.077 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:03.077 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:31:03.077 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861475+0000", 2024-09-14T06:31:03.077 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.5730000000000001e-06 2024-09-14T06:31:03.077 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:03.078 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:03.078 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:31:03.078 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.925085+0000", 2024-09-14T06:31:03.078 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.063609902999999995 2024-09-14T06:31:03.078 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:03.078 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:03.078 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:31:03.078 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.925145+0000", 2024-09-14T06:31:03.078 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.9383999999999999e-05 2024-09-14T06:31:03.078 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:03.078 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:03.078 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:31:03.078 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.925318+0000", 2024-09-14T06:31:03.079 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00017318499999999999 2024-09-14T06:31:03.079 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:03.079 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:31:03.079 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:03.079 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:03.079 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:03.079 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:44540 3.2 3:4190d882:::rbd_data.112dca2f6ae1.0000000000000002:17 [read 552960~4096 [fadvise_nocache]] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:31:03.079 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:43.861483+0000", 2024-09-14T06:31:03.079 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 79.133549021999997, 2024-09-14T06:31:03.079 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.064065681999999999, 2024-09-14T06:31:03.079 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:31:03.079 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:31:03.079 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:31:03.079 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:31:03.080 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:31:03.080 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:03.080 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 44540 2024-09-14T06:31:03.080 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:03.080 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:31:03.080 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:03.080 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:31:03.080 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861483+0000", 2024-09-14T06:31:03.080 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:03.080 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:03.080 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:03.080 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:31:03.080 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861483+0000", 2024-09-14T06:31:03.081 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:03.081 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:03.081 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:03.081 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:31:03.081 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.867079+0000", 2024-09-14T06:31:03.081 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0055966480000000001 2024-09-14T06:31:03.081 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:03.081 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:03.081 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:31:03.081 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.867086+0000", 2024-09-14T06:31:03.081 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.387e-06 2024-09-14T06:31:03.081 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:03.081 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:03.081 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:31:03.081 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.867094+0000", 2024-09-14T06:31:03.082 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.9419999999999995e-06 2024-09-14T06:31:03.082 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:03.082 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:03.082 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:31:03.082 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.867109+0000", 2024-09-14T06:31:03.082 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.5818999999999999e-05 2024-09-14T06:31:03.082 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:03.082 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:03.082 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:31:03.082 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.925354+0000", 2024-09-14T06:31:03.082 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.058244047 2024-09-14T06:31:03.082 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:03.082 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:03.082 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:31:03.083 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.925406+0000", 2024-09-14T06:31:03.083 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.2661999999999998e-05 2024-09-14T06:31:03.083 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:03.083 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:03.083 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:31:03.083 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.925548+0000", 2024-09-14T06:31:03.083 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000142177 2024-09-14T06:31:03.083 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:03.083 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:31:03.083 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:03.083 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:03.083 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:31:03.083 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-14T06:31:03.084 INFO:teuthology.orchestra.run.smithi094.stdout:4045 writesame 0x3f04e68 thru 0x3f0d97b data_size 0x45(0x8b14 bytes) 2024-09-14T06:31:03.085 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-14T06:31:03.086 INFO:teuthology.orchestra.run.smithi094.stdout:4048 writesame 0xe9fc12c thru 0xea0aed0 data_size 0x7(0xeda5 bytes) 2024-09-14T06:31:03.087 INFO:teuthology.orchestra.run.smithi094.stdout:4049 punch from 0x2247f52 to 0x2252e79, (0xaf27 bytes) 2024-09-14T06:31:03.087 INFO:teuthology.orchestra.run.smithi094.stdout:4050 write 0x2da230a thru 0x2da7b89 (0x5880 bytes) 2024-09-14T06:31:03.087 INFO:teuthology.orchestra.run.smithi094.stdout:4051 punch from 0xe07b27a to 0xe080ddf, (0x5b65 bytes) 2024-09-14T06:31:03.087 INFO:teuthology.orchestra.run.smithi094.stdout:4052 read 0x12a350e thru 0x12aa0e9 (0x6bdc bytes) 2024-09-14T06:31:03.087 INFO:teuthology.orchestra.run.smithi094.stdout:4054 punch from 0xe6ace15 to 0xe6baed0, (0xe0bb bytes) 2024-09-14T06:31:03.087 INFO:teuthology.orchestra.run.smithi094.stdout:4055 punch from 0x7dcb1a to 0x7ea21f, (0xd705 bytes) 2024-09-14T06:31:03.087 INFO:teuthology.orchestra.run.smithi094.stdout:4056 read 0x50d71c thru 0x50f87b (0x2160 bytes) 2024-09-14T06:31:03.087 INFO:teuthology.orchestra.run.smithi094.stdout:4057 trunc from 0xea0aed1 to 0x186d37d 2024-09-14T06:31:03.087 INFO:tasks.ceph.osd.3.smithi094.stderr:2024-09-14T06:31:03.083+0000 7feeeafe8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33655) UID: 0 2024-09-14T06:31:03.088 INFO:teuthology.orchestra.run.smithi094.stdout:4060 writesame 0xc8767c6 thru 0xc88146e data_size 0x1(0xaca9 bytes) 2024-09-14T06:31:03.088 INFO:teuthology.orchestra.run.smithi094.stdout:4061 read 0x4bd1a55 thru 0x4be07b0 (0xed5c bytes) 2024-09-14T06:31:03.088 INFO:teuthology.orchestra.run.smithi094.stdout:4062 write 0x8e8ef53 thru 0x8e93839 (0x48e7 bytes) 2024-09-14T06:31:03.090 INFO:teuthology.orchestra.run.smithi094.stdout:4063 punch from 0x2e07429 to 0x2e137cd, (0xc3a4 bytes) 2024-09-14T06:31:03.090 INFO:teuthology.orchestra.run.smithi094.stdout:4064 write 0x9e66b5c thru 0x9e6e4e7 (0x798c bytes) 2024-09-14T06:31:03.107 INFO:teuthology.orchestra.run.smithi094.stdout:4065 trunc from 0xc88146f to 0x5ba1501 2024-09-14T06:31:03.119 INFO:teuthology.orchestra.run.smithi094.stdout:4066 read 0x3d642a1 thru 0x3d67b16 (0x3876 bytes) 2024-09-14T06:31:03.119 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-14T06:31:03.120 INFO:teuthology.orchestra.run.smithi094.stdout:4068 trunc from 0x5ba1501 to 0x492666a 2024-09-14T06:31:03.126 INFO:teuthology.orchestra.run.smithi094.stdout:4069 punch from 0x41641b4 to 0x416d739, (0x9585 bytes) 2024-09-14T06:31:03.126 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-14T06:31:03.126 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-14T06:31:03.126 INFO:teuthology.orchestra.run.smithi094.stdout:4075 read 0x248bb9e thru 0x249679f (0xac02 bytes) 2024-09-14T06:31:03.144 INFO:teuthology.orchestra.run.smithi094.stdout:4076 writesame 0xdd9473b thru 0xdd95c4c data_size 0x1f(0x1512 bytes) 2024-09-14T06:31:03.144 INFO:teuthology.orchestra.run.smithi094.stdout:4077 write 0x75394a4 thru 0x753f158 (0x5cb5 bytes) 2024-09-14T06:31:03.149 INFO:teuthology.orchestra.run.smithi094.stdout:4078 write 0x2f3460e thru 0x2f37863 (0x3256 bytes) 2024-09-14T06:31:03.153 INFO:teuthology.orchestra.run.smithi094.stdout:4079 write 0x4d6a061 thru 0x4d74126 (0xa0c6 bytes) 2024-09-14T06:31:03.160 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-14T06:31:03.160 INFO:teuthology.orchestra.run.smithi094.stdout:4081 read 0xcf2ed49 thru 0xcf39edd (0xb195 bytes) 2024-09-14T06:31:03.160 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-14T06:31:03.160 INFO:teuthology.orchestra.run.smithi094.stdout:4084 write 0x84ff80a thru 0x850ec6e (0xf465 bytes) 2024-09-14T06:31:03.167 INFO:teuthology.orchestra.run.smithi094.stdout:4085 writesame 0x18cfdfb thru 0x18dc757 data_size 0x1(0xc95d bytes) 2024-09-14T06:31:03.180 INFO:teuthology.orchestra.run.smithi094.stdout:4086 write 0x2d08ba1 thru 0x2d0e634 (0x5a94 bytes) 2024-09-14T06:31:03.185 INFO:teuthology.orchestra.run.smithi094.stdout:4087 writesame 0x994fa8c thru 0x9957ef6 data_size 0x1(0x846b bytes) 2024-09-14T06:31:03.185 INFO:tasks.ceph.osd.3.smithi094.stderr:2024-09-14T06:31:03.184+0000 7feeeafe8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33655) UID: 0 2024-09-14T06:31:03.199 INFO:teuthology.orchestra.run.smithi094.stdout:4089 writesame 0x39bcec7 thru 0x39c4878 data_size 0x25(0x79b2 bytes) 2024-09-14T06:31:03.203 INFO:teuthology.orchestra.run.smithi138.stdout:{ 2024-09-14T06:31:03.204 INFO:teuthology.orchestra.run.smithi138.stdout: "ops": [], 2024-09-14T06:31:03.204 INFO:teuthology.orchestra.run.smithi138.stdout: "num_ops": 0 2024-09-14T06:31:03.204 INFO:teuthology.orchestra.run.smithi138.stdout:} 2024-09-14T06:31:03.209 INFO:teuthology.orchestra.run.smithi094.stdout:4090 writesame 0xe1aec7 thru 0xe2acdf data_size 0x1(0xfe19 bytes) 2024-09-14T06:31:03.213 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-14T06:31:03.223 INFO:teuthology.orchestra.run.smithi094.stdout:4091 punch from 0x73849b to 0x73cc3d, (0x47a2 bytes) 2024-09-14T06:31:03.223 INFO:teuthology.orchestra.run.smithi094.stdout:4092 writesame 0x467bc7 thru 0x467c72 data_size 0x2(0xac bytes) 2024-09-14T06:31:03.224 INFO:teuthology.orchestra.run.smithi094.stdout:4093 read 0x75e5e85 thru 0x75ef8d1 (0x9a4d bytes) 2024-09-14T06:31:03.231 INFO:teuthology.orchestra.run.smithi094.stdout:4094 write 0xe05cf33 thru 0xe05e709 (0x17d7 bytes) 2024-09-14T06:31:03.236 INFO:teuthology.orchestra.run.smithi094.stdout:4095 trunc from 0xe05e70a to 0xdd46a56 2024-09-14T06:31:03.243 INFO:teuthology.orchestra.run.smithi094.stdout:4097 read 0x5bda6 thru 0x6ab69 (0xedc4 bytes) 2024-09-14T06:31:03.244 INFO:teuthology.orchestra.run.smithi094.stdout:4098 read 0xb2d3a67 thru 0xb2e2544 (0xeade bytes) 2024-09-14T06:31:03.245 INFO:teuthology.orchestra.run.smithi094.stdout:4099 writesame 0x2c62d91 thru 0x2c69861 data_size 0x3(0x6ad1 bytes) 2024-09-14T06:31:03.259 INFO:teuthology.orchestra.run.smithi094.stdout:4100 trunc from 0xdd46a56 to 0x6ba64ec 2024-09-14T06:31:03.274 INFO:teuthology.orchestra.run.smithi094.stdout:4103 write 0x455b328 thru 0x4563c0a (0x88e3 bytes) 2024-09-14T06:31:03.283 INFO:teuthology.orchestra.run.smithi094.stdout:4104 read 0xe5c5ec thru 0xe6ac38 (0xe64d bytes) 2024-09-14T06:31:03.286 INFO:tasks.ceph.osd.5.smithi138.stderr:2024-09-14T06:31:03.285+0000 7fe13746f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33618) UID: 0 2024-09-14T06:31:03.288 INFO:teuthology.orchestra.run.smithi094.stdout:4108 trunc from 0x6ba64ec to 0x3077006 2024-09-14T06:31:03.309 INFO:teuthology.orchestra.run.smithi094.stdout:4109 read 0x1672f66 thru 0x167b72f (0x87ca bytes) 2024-09-14T06:31:03.310 INFO:teuthology.orchestra.run.smithi094.stdout:4110 read 0x20aa4de thru 0x20b2aa7 (0x85ca bytes) 2024-09-14T06:31:03.310 INFO:teuthology.orchestra.run.smithi094.stdout:4112 writesame 0x1454e54 thru 0x145718b data_size 0x38(0x2338 bytes) 2024-09-14T06:31:03.311 INFO:teuthology.orchestra.run.smithi094.stdout:4113 write 0x23600df thru 0x236628a (0x61ac bytes) 2024-09-14T06:31:03.328 INFO:teuthology.orchestra.run.smithi138.stdout:{ 2024-09-14T06:31:03.328 INFO:teuthology.orchestra.run.smithi138.stdout: "ops": [], 2024-09-14T06:31:03.328 INFO:teuthology.orchestra.run.smithi138.stdout: "complaint_time": 30, 2024-09-14T06:31:03.328 INFO:teuthology.orchestra.run.smithi138.stdout: "num_blocked_ops": 0 2024-09-14T06:31:03.329 INFO:teuthology.orchestra.run.smithi138.stdout:} 2024-09-14T06:31:03.336 INFO:teuthology.orchestra.run.smithi094.stdout:4114 write 0xbb98243 thru 0xbba1874 (0x9632 bytes) 2024-09-14T06:31:03.338 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-14T06:31:03.343 INFO:teuthology.orchestra.run.smithi094.stdout:4115 read 0xa2d4646 thru 0xa2dc616 (0x7fd1 bytes) 2024-09-14T06:31:03.351 INFO:teuthology.orchestra.run.smithi094.stdout:4118 write 0xdba8bb1 thru 0xdbaf069 (0x64b9 bytes) 2024-09-14T06:31:03.358 INFO:teuthology.orchestra.run.smithi094.stdout:4119 read 0x4e6885b thru 0x4e6e96f (0x6115 bytes) 2024-09-14T06:31:03.358 INFO:teuthology.orchestra.run.smithi094.stdout:4120 writesame 0x5cfb42e thru 0x5d0020f data_size 0x3(0x4de2 bytes) 2024-09-14T06:31:03.371 INFO:teuthology.orchestra.run.smithi094.stdout:4121 writesame 0xe9ff619 thru 0xea0a048 data_size 0x10(0xaa30 bytes) 2024-09-14T06:31:03.371 INFO:teuthology.orchestra.run.smithi094.stdout:4123 punch from 0x519d9d5 to 0x51a2776, (0x4da1 bytes) 2024-09-14T06:31:03.381 INFO:teuthology.orchestra.run.smithi094.stdout:4125 trunc from 0xea0a049 to 0x56476a9 2024-09-14T06:31:03.386 INFO:tasks.ceph.osd.0.smithi094.stderr:2024-09-14T06:31:03.385+0000 7f07c9333640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33636) UID: 0 2024-09-14T06:31:03.393 INFO:teuthology.orchestra.run.smithi094.stdout:4126 read 0x53f5f6 thru 0x5408eb (0x12f6 bytes) 2024-09-14T06:31:03.402 INFO:teuthology.orchestra.run.smithi094.stdout:4127 write 0x793649b thru 0x793c7cc (0x6332 bytes) 2024-09-14T06:31:03.415 INFO:teuthology.orchestra.run.smithi094.stdout:4128 trunc from 0x793c7cd to 0xe45d39b 2024-09-14T06:31:03.424 INFO:teuthology.orchestra.run.smithi094.stdout:4129 trunc from 0xe45d39b to 0xb703490 2024-09-14T06:31:03.438 INFO:teuthology.orchestra.run.smithi094.stdout:4131 write 0xe2ca58f thru 0xe2d363b (0x90ad bytes) 2024-09-14T06:31:03.447 INFO:teuthology.orchestra.run.smithi094.stdout:4133 write 0x135246c thru 0x135e283 (0xbe18 bytes) 2024-09-14T06:31:03.455 INFO:teuthology.orchestra.run.smithi094.stdout:4135 read 0xad49f18 thru 0xad4e1f1 (0x42da bytes) 2024-09-14T06:31:03.455 INFO:teuthology.orchestra.run.smithi094.stdout:4137 writesame 0x3045cf9 thru 0x305534f data_size 0x3(0xf657 bytes) 2024-09-14T06:31:03.466 INFO:teuthology.orchestra.run.smithi094.stdout:4139 write 0x27a5a28 thru 0x27b2179 (0xc752 bytes) 2024-09-14T06:31:03.470 INFO:teuthology.orchestra.run.smithi094.stdout:4140 write 0x5577823 thru 0x557860c (0xdea bytes) 2024-09-14T06:31:03.474 INFO:teuthology.orchestra.run.smithi094.stdout:4142 read 0xbfe68e thru 0xc01ea4 (0x3817 bytes) 2024-09-14T06:31:03.474 INFO:teuthology.orchestra.run.smithi094.stdout:4143 writesame 0x52506af thru 0x525da1a data_size 0xe(0xd36c bytes) 2024-09-14T06:31:03.481 INFO:teuthology.orchestra.run.smithi094.stdout:4144 punch from 0x52384e3 to 0x5244f78, (0xca95 bytes) 2024-09-14T06:31:03.482 INFO:teuthology.orchestra.run.smithi094.stdout:4145 read 0x91dde00 thru 0x91deb6a (0xd6b bytes) 2024-09-14T06:31:03.482 INFO:teuthology.orchestra.run.smithi094.stdout:4146 punch from 0x1d0422b to 0x1d11372, (0xd147 bytes) 2024-09-14T06:31:03.482 INFO:teuthology.orchestra.run.smithi094.stdout:4147 write 0xc070cc7 thru 0xc071a53 (0xd8d bytes) 2024-09-14T06:31:03.484 INFO:teuthology.orchestra.run.smithi094.stdout:4148 write 0x961fc3c thru 0x96278cb (0x7c90 bytes) 2024-09-14T06:31:03.488 INFO:tasks.ceph.osd.6.smithi138.stderr:2024-09-14T06:31:03.487+0000 7f8cd8849640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33620) UID: 0 2024-09-14T06:31:03.490 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-14T06:31:03.490 INFO:teuthology.orchestra.run.smithi094.stdout:4150 read 0xd4f7eca thru 0xd4fed47 (0x6e7e bytes) 2024-09-14T06:31:03.490 INFO:teuthology.orchestra.run.smithi094.stdout:4151 punch from 0x29dffad to 0x29ecd38, (0xcd8b bytes) 2024-09-14T06:31:03.491 INFO:teuthology.orchestra.run.smithi094.stdout:4152 writesame 0x94cf5f1 thru 0x94de68e data_size 0x13(0xf09e bytes) 2024-09-14T06:31:03.492 INFO:teuthology.orchestra.run.smithi138.stdout:{ 2024-09-14T06:31:03.493 INFO:teuthology.orchestra.run.smithi138.stdout: "size": 20, 2024-09-14T06:31:03.493 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 600, 2024-09-14T06:31:03.493 INFO:teuthology.orchestra.run.smithi138.stdout: "ops": [ 2024-09-14T06:31:03.493 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:03.493 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:19328 3.3 3:c34e5e51:::rbd_data.112d92027590.0000000000000024:f [read 1926144~1024 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:31:03.493 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:27.059768+0000", 2024-09-14T06:31:03.493 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 96.431365881999994, 2024-09-14T06:31:03.493 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046369147999999999, 2024-09-14T06:31:03.493 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:31:03.493 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:31:03.493 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:31:03.493 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:31:03.494 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:31:03.494 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:03.494 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 19328 2024-09-14T06:31:03.494 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:03.494 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:31:03.494 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:03.494 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:31:03.494 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.059768+0000", 2024-09-14T06:31:03.494 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:03.494 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:03.494 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:03.494 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:31:03.494 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.059768+0000", 2024-09-14T06:31:03.495 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:03.495 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:03.495 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:03.495 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:31:03.495 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.059874+0000", 2024-09-14T06:31:03.495 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00010551400000000001 2024-09-14T06:31:03.495 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:03.495 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:03.495 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:31:03.495 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.059877+0000", 2024-09-14T06:31:03.495 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.8279999999999999e-06 2024-09-14T06:31:03.495 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:03.495 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:03.495 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:31:03.496 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.059884+0000", 2024-09-14T06:31:03.496 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.6389999999999992e-06 2024-09-14T06:31:03.496 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:03.496 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:03.496 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:31:03.496 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.059890+0000", 2024-09-14T06:31:03.496 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.49e-06 2024-09-14T06:31:03.496 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:03.496 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:03.496 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:31:03.496 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.105804+0000", 2024-09-14T06:31:03.496 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.045914242000000001 2024-09-14T06:31:03.496 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:03.496 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:03.497 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:31:03.497 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.105886+0000", 2024-09-14T06:31:03.497 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.2305999999999994e-05 2024-09-14T06:31:03.497 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:03.497 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:03.497 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:31:03.497 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.106138+0000", 2024-09-14T06:31:03.497 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.000251129 2024-09-14T06:31:03.497 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:03.497 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:31:03.497 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:03.497 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:03.497 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:03.497 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:19329 3.3 3:c34e5e51:::rbd_data.112d92027590.0000000000000024:f [read 1936384~1024 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:31:03.498 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:27.059899+0000", 2024-09-14T06:31:03.498 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 96.431235358999999, 2024-09-14T06:31:03.498 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046572122000000001, 2024-09-14T06:31:03.498 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:31:03.498 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:31:03.498 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:31:03.498 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:31:03.498 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:31:03.498 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:03.498 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 19329 2024-09-14T06:31:03.498 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:03.498 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:31:03.498 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:03.499 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:31:03.499 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.059899+0000", 2024-09-14T06:31:03.499 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:03.499 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:03.499 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:03.499 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:31:03.499 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.059899+0000", 2024-09-14T06:31:03.499 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:03.499 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:03.499 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:03.499 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:31:03.499 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062501+0000", 2024-09-14T06:31:03.499 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.0026016419999999999 2024-09-14T06:31:03.499 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:03.499 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:03.500 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:31:03.500 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062514+0000", 2024-09-14T06:31:03.500 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.2947e-05 2024-09-14T06:31:03.500 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:03.500 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:03.500 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:31:03.500 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062523+0000", 2024-09-14T06:31:03.500 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.5519999999999993e-06 2024-09-14T06:31:03.500 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:03.500 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:03.500 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:31:03.500 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062534+0000", 2024-09-14T06:31:03.500 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.1416e-05 2024-09-14T06:31:03.500 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:03.500 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:03.501 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:31:03.501 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.106179+0000", 2024-09-14T06:31:03.501 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.043644049999999997 2024-09-14T06:31:03.501 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:03.501 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:03.501 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:31:03.501 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.106262+0000", 2024-09-14T06:31:03.501 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.3178999999999998e-05 2024-09-14T06:31:03.501 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:03.501 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:03.501 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:31:03.501 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.106471+0000", 2024-09-14T06:31:03.501 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00020933599999999999 2024-09-14T06:31:03.502 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:03.502 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:31:03.502 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:03.502 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:03.502 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:03.502 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:19351 3.3 3:c34e5e51:::rbd_data.112d92027590.0000000000000024:f [read 1159168~1024 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:31:03.502 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:27.062766+0000", 2024-09-14T06:31:03.502 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 96.428368133999996, 2024-09-14T06:31:03.502 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046099454999999998, 2024-09-14T06:31:03.502 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:31:03.502 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:31:03.502 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:31:03.502 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:31:03.502 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:31:03.502 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:03.503 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 19351 2024-09-14T06:31:03.503 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:03.503 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:31:03.503 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:03.503 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:31:03.503 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062766+0000", 2024-09-14T06:31:03.503 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:03.503 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:03.503 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:03.503 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:31:03.504 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062766+0000", 2024-09-14T06:31:03.504 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:03.504 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:03.504 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:03.504 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:31:03.504 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062766+0000", 2024-09-14T06:31:03.504 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.6e-07 2024-09-14T06:31:03.504 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:03.504 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:03.504 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:31:03.504 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062767+0000", 2024-09-14T06:31:03.504 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.0070000000000001e-06 2024-09-14T06:31:03.504 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:03.504 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:03.505 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:31:03.505 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062785+0000", 2024-09-14T06:31:03.505 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.7884999999999999e-05 2024-09-14T06:31:03.505 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:03.505 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:03.505 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:31:03.505 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062790+0000", 2024-09-14T06:31:03.505 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.0900000000000004e-06 2024-09-14T06:31:03.505 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:03.505 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:03.505 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:31:03.505 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.108735+0000", 2024-09-14T06:31:03.505 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.045944411999999997 2024-09-14T06:31:03.505 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:03.505 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:03.506 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:31:03.506 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.108809+0000", 2024-09-14T06:31:03.506 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.4157000000000002e-05 2024-09-14T06:31:03.506 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:03.506 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:03.506 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:31:03.506 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.108866+0000", 2024-09-14T06:31:03.506 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.6644000000000002e-05 2024-09-14T06:31:03.506 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:03.506 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:31:03.506 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:03.506 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:03.506 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:03.506 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:19352 3.3 3:c34e5e51:::rbd_data.112d92027590.0000000000000024:f [read 1169408~1024 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:31:03.506 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:27.062798+0000", 2024-09-14T06:31:03.507 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 96.428336723000001, 2024-09-14T06:31:03.507 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046255900000000003, 2024-09-14T06:31:03.507 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:31:03.507 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:31:03.507 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:31:03.507 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:31:03.507 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:31:03.507 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:03.507 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 19352 2024-09-14T06:31:03.507 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:03.507 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:31:03.507 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:03.507 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:31:03.507 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062798+0000", 2024-09-14T06:31:03.507 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:03.507 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:03.508 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:03.508 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:31:03.508 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062798+0000", 2024-09-14T06:31:03.508 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:03.508 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:03.508 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:03.508 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:31:03.508 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062798+0000", 2024-09-14T06:31:03.508 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.37e-07 2024-09-14T06:31:03.508 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:03.508 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:03.508 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:31:03.508 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062800+0000", 2024-09-14T06:31:03.508 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.0269999999999998e-06 2024-09-14T06:31:03.509 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:03.509 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:03.509 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:31:03.509 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062806+0000", 2024-09-14T06:31:03.509 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.6690000000000003e-06 2024-09-14T06:31:03.509 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:03.509 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:03.509 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:31:03.509 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062810+0000", 2024-09-14T06:31:03.509 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.3449999999999999e-06 2024-09-14T06:31:03.509 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:03.509 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:03.509 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:31:03.509 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.108892+0000", 2024-09-14T06:31:03.509 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046081715000000002 2024-09-14T06:31:03.510 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:03.510 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:03.510 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:31:03.510 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.108940+0000", 2024-09-14T06:31:03.510 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.7815999999999998e-05 2024-09-14T06:31:03.510 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:03.510 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:03.510 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:31:03.510 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.109053+0000", 2024-09-14T06:31:03.510 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.000113791 2024-09-14T06:31:03.510 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:03.510 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:31:03.510 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:03.510 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:03.511 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:03.511 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:19353 3.3 3:c34e5e51:::rbd_data.112d92027590.0000000000000024:f [read 1179648~1024 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:31:03.511 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:27.062816+0000", 2024-09-14T06:31:03.511 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 96.428318203999993, 2024-09-14T06:31:03.511 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046481256999999998, 2024-09-14T06:31:03.511 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:31:03.511 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:31:03.511 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:31:03.511 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:31:03.511 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:31:03.511 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:03.511 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 19353 2024-09-14T06:31:03.511 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:03.511 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:31:03.512 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:03.512 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:31:03.512 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062816+0000", 2024-09-14T06:31:03.512 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:03.512 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:03.512 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:03.512 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:31:03.512 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062816+0000", 2024-09-14T06:31:03.512 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:03.512 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:03.512 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:03.512 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:31:03.512 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062816+0000", 2024-09-14T06:31:03.512 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.2899999999999999e-07 2024-09-14T06:31:03.512 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:03.513 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:03.513 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:31:03.513 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062818+0000", 2024-09-14T06:31:03.513 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.699e-06 2024-09-14T06:31:03.513 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:03.513 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:03.513 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:31:03.513 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062825+0000", 2024-09-14T06:31:03.513 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.878e-06 2024-09-14T06:31:03.513 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:03.513 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:03.513 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:31:03.513 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062830+0000", 2024-09-14T06:31:03.513 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.5410000000000002e-06 2024-09-14T06:31:03.514 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:03.514 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:03.514 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:31:03.514 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.109096+0000", 2024-09-14T06:31:03.514 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046266517 2024-09-14T06:31:03.514 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:03.514 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:03.514 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:31:03.514 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.109190+0000", 2024-09-14T06:31:03.514 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.4000999999999997e-05 2024-09-14T06:31:03.514 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:03.514 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:03.514 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:31:03.515 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.109297+0000", 2024-09-14T06:31:03.515 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.000107292 2024-09-14T06:31:03.515 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:03.515 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:31:03.515 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:03.515 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:03.515 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:03.515 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:19354 3.3 3:c34e5e51:::rbd_data.112d92027590.0000000000000024:f [read 1189888~1024 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:31:03.515 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:27.062842+0000", 2024-09-14T06:31:03.515 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 96.428292748999993, 2024-09-14T06:31:03.515 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046594257, 2024-09-14T06:31:03.515 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:31:03.515 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:31:03.515 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:31:03.515 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:31:03.516 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:31:03.516 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:03.516 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 19354 2024-09-14T06:31:03.516 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:03.516 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:31:03.516 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:03.516 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:31:03.516 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062842+0000", 2024-09-14T06:31:03.516 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:03.516 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:03.516 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:03.516 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:31:03.516 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062842+0000", 2024-09-14T06:31:03.516 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:03.517 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:03.517 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:03.517 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:31:03.517 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062842+0000", 2024-09-14T06:31:03.517 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.7399999999999999e-07 2024-09-14T06:31:03.517 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:03.517 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:03.517 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:31:03.517 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062859+0000", 2024-09-14T06:31:03.517 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.7010000000000001e-05 2024-09-14T06:31:03.517 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:03.517 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:03.517 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:31:03.517 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062866+0000", 2024-09-14T06:31:03.518 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.9990000000000002e-06 2024-09-14T06:31:03.518 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:03.518 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:03.518 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:31:03.518 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062873+0000", 2024-09-14T06:31:03.518 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.8569999999999997e-06 2024-09-14T06:31:03.518 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:03.518 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:03.518 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:31:03.518 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.109335+0000", 2024-09-14T06:31:03.518 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046461943999999998 2024-09-14T06:31:03.518 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:03.518 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:03.518 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:31:03.518 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.109377+0000", 2024-09-14T06:31:03.519 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.2558999999999997e-05 2024-09-14T06:31:03.519 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:03.519 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:03.519 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:31:03.519 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.109436+0000", 2024-09-14T06:31:03.519 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.8514000000000003e-05 2024-09-14T06:31:03.519 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:03.519 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:31:03.519 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:03.519 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:03.519 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:03.519 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:19355 3.3 3:c34e5e51:::rbd_data.112d92027590.0000000000000024:f [read 1200128~1024 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:31:03.519 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:27.062884+0000", 2024-09-14T06:31:03.519 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 96.428250075999998, 2024-09-14T06:31:03.520 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046681650999999998, 2024-09-14T06:31:03.520 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:31:03.520 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:31:03.520 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:31:03.520 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:31:03.520 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:31:03.520 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:03.520 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 19355 2024-09-14T06:31:03.520 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:03.520 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:31:03.520 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:03.520 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:31:03.520 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062884+0000", 2024-09-14T06:31:03.520 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:03.521 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:03.521 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:03.521 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:31:03.521 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062884+0000", 2024-09-14T06:31:03.521 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:03.521 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:03.521 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:03.521 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:31:03.521 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062885+0000", 2024-09-14T06:31:03.521 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.8000000000000001e-07 2024-09-14T06:31:03.521 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:03.521 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:03.521 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:31:03.521 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062886+0000", 2024-09-14T06:31:03.522 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.8920000000000001e-06 2024-09-14T06:31:03.522 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:03.522 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:03.522 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:31:03.522 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062892+0000", 2024-09-14T06:31:03.522 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.84e-06 2024-09-14T06:31:03.522 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:03.522 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:03.522 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:31:03.522 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062899+0000", 2024-09-14T06:31:03.522 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.5339999999999996e-06 2024-09-14T06:31:03.522 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:03.522 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:03.522 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:31:03.523 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.109466+0000", 2024-09-14T06:31:03.523 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046566919999999998 2024-09-14T06:31:03.523 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:03.523 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:03.523 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:31:03.523 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.109510+0000", 2024-09-14T06:31:03.523 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.4153000000000002e-05 2024-09-14T06:31:03.523 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:03.523 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:03.523 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:31:03.523 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.109566+0000", 2024-09-14T06:31:03.523 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.5931999999999999e-05 2024-09-14T06:31:03.523 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:03.523 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:31:03.523 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:03.523 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:03.524 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:03.524 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:19356 3.3 3:c34e5e51:::rbd_data.112d92027590.0000000000000024:f [read 1210368~1024 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:31:03.524 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:27.062908+0000", 2024-09-14T06:31:03.524 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 96.428226639000002, 2024-09-14T06:31:03.524 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046807498000000003, 2024-09-14T06:31:03.524 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:31:03.524 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:31:03.524 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:31:03.524 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:31:03.524 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:31:03.524 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:03.524 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 19356 2024-09-14T06:31:03.524 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:03.525 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:31:03.525 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:03.525 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:31:03.525 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062908+0000", 2024-09-14T06:31:03.525 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:03.525 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:03.525 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:03.525 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:31:03.525 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062908+0000", 2024-09-14T06:31:03.525 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:03.525 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:03.525 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:03.525 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:31:03.525 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062908+0000", 2024-09-14T06:31:03.526 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.5699999999999998e-07 2024-09-14T06:31:03.526 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:03.526 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:03.526 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:31:03.526 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062910+0000", 2024-09-14T06:31:03.526 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.7710000000000001e-06 2024-09-14T06:31:03.526 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:03.526 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:03.526 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:31:03.526 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062920+0000", 2024-09-14T06:31:03.526 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.0654e-05 2024-09-14T06:31:03.526 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:03.526 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:03.526 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:31:03.527 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062931+0000", 2024-09-14T06:31:03.527 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.075e-05 2024-09-14T06:31:03.527 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:03.527 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:03.527 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:31:03.527 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.109595+0000", 2024-09-14T06:31:03.527 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046663713000000002 2024-09-14T06:31:03.527 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:03.527 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:03.527 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:31:03.527 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.109632+0000", 2024-09-14T06:31:03.527 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.7505000000000001e-05 2024-09-14T06:31:03.527 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:03.528 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:03.528 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:31:03.528 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.109715+0000", 2024-09-14T06:31:03.528 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.2748000000000001e-05 2024-09-14T06:31:03.528 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:03.528 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:31:03.528 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:03.528 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:03.528 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:03.528 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:19357 3.3 3:c34e5e51:::rbd_data.112d92027590.0000000000000024:f [read 1220608~1024 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:31:03.528 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:27.062939+0000", 2024-09-14T06:31:03.528 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 96.428195191, 2024-09-14T06:31:03.528 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046920508, 2024-09-14T06:31:03.528 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:31:03.529 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:31:03.529 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:31:03.529 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:31:03.529 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:31:03.529 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:03.529 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 19357 2024-09-14T06:31:03.529 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:03.529 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:31:03.529 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:03.529 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:31:03.529 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062939+0000", 2024-09-14T06:31:03.529 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:03.529 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:03.529 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:03.530 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:31:03.530 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062939+0000", 2024-09-14T06:31:03.530 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:03.530 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:03.530 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:03.530 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:31:03.530 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062940+0000", 2024-09-14T06:31:03.530 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.32e-07 2024-09-14T06:31:03.530 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:03.530 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:03.530 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:31:03.530 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062942+0000", 2024-09-14T06:31:03.530 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.1919999999999999e-06 2024-09-14T06:31:03.530 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:03.530 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:03.531 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:31:03.531 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062948+0000", 2024-09-14T06:31:03.531 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.1149999999999999e-06 2024-09-14T06:31:03.531 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:03.531 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:03.531 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:31:03.531 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062960+0000", 2024-09-14T06:31:03.531 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.1895e-05 2024-09-14T06:31:03.531 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:03.531 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:03.531 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:31:03.531 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.109704+0000", 2024-09-14T06:31:03.531 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046744584999999998 2024-09-14T06:31:03.531 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:03.531 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:03.532 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:31:03.532 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.109777+0000", 2024-09-14T06:31:03.532 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.2282999999999999e-05 2024-09-14T06:31:03.532 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:03.532 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:03.532 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:31:03.532 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.109860+0000", 2024-09-14T06:31:03.532 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.3005999999999998e-05 2024-09-14T06:31:03.532 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:03.532 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:31:03.532 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:03.532 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:03.532 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:03.532 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:19358 3.3 3:c34e5e51:::rbd_data.112d92027590.0000000000000024:f [read 1230848~1024 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:31:03.532 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:27.062967+0000", 2024-09-14T06:31:03.533 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 96.428167560999995, 2024-09-14T06:31:03.533 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047144471, 2024-09-14T06:31:03.533 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:31:03.533 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:31:03.533 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:31:03.533 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:31:03.533 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:31:03.533 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:03.533 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 19358 2024-09-14T06:31:03.533 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:03.533 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:31:03.533 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:03.533 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:31:03.534 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062967+0000", 2024-09-14T06:31:03.534 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:03.534 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:03.534 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:03.534 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:31:03.534 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062967+0000", 2024-09-14T06:31:03.534 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:03.534 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:03.534 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:03.534 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:31:03.534 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062967+0000", 2024-09-14T06:31:03.534 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.89e-07 2024-09-14T06:31:03.534 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:03.534 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:03.535 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:31:03.535 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062969+0000", 2024-09-14T06:31:03.535 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.9929999999999998e-06 2024-09-14T06:31:03.535 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:03.535 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:03.535 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:31:03.535 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062975+0000", 2024-09-14T06:31:03.535 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.9030000000000001e-06 2024-09-14T06:31:03.535 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:03.535 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:03.535 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:31:03.535 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062979+0000", 2024-09-14T06:31:03.535 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.2250000000000002e-06 2024-09-14T06:31:03.535 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:03.536 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:03.536 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:31:03.536 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.109934+0000", 2024-09-14T06:31:03.536 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046954410000000002 2024-09-14T06:31:03.536 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:03.536 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:03.536 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:31:03.536 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.110007+0000", 2024-09-14T06:31:03.536 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.3829000000000001e-05 2024-09-14T06:31:03.536 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:03.536 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:03.536 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:31:03.536 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.110111+0000", 2024-09-14T06:31:03.536 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00010372199999999999 2024-09-14T06:31:03.537 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:03.537 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:31:03.537 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:03.537 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:03.537 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:03.537 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:19359 3.3 3:c34e5e51:::rbd_data.112d92027590.0000000000000024:f [read 1241088~1024 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:31:03.537 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:27.062987+0000", 2024-09-14T06:31:03.537 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 96.428147471000003, 2024-09-14T06:31:03.537 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047297182, 2024-09-14T06:31:03.537 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:31:03.537 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:31:03.537 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:31:03.537 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:31:03.537 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:31:03.538 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:03.538 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 19359 2024-09-14T06:31:03.538 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:03.538 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:31:03.538 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:03.538 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:31:03.538 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062987+0000", 2024-09-14T06:31:03.538 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:03.538 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:03.538 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:03.538 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:31:03.538 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062987+0000", 2024-09-14T06:31:03.538 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:03.538 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:03.538 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:03.538 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:31:03.539 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062987+0000", 2024-09-14T06:31:03.539 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.2500000000000001e-07 2024-09-14T06:31:03.539 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:03.539 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:03.539 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:31:03.539 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062989+0000", 2024-09-14T06:31:03.539 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.6080000000000001e-06 2024-09-14T06:31:03.539 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:03.539 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:03.539 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:31:03.539 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062994+0000", 2024-09-14T06:31:03.539 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.5520000000000004e-06 2024-09-14T06:31:03.539 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:03.539 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:03.540 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:31:03.540 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062999+0000", 2024-09-14T06:31:03.540 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.3930000000000001e-06 2024-09-14T06:31:03.540 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:03.540 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:03.540 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:31:03.540 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.110142+0000", 2024-09-14T06:31:03.540 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047143154 2024-09-14T06:31:03.540 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:03.540 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:03.540 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:31:03.540 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.110219+0000", 2024-09-14T06:31:03.540 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.6737999999999996e-05 2024-09-14T06:31:03.540 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:03.541 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:03.541 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:31:03.541 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.110284+0000", 2024-09-14T06:31:03.541 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.5412000000000003e-05 2024-09-14T06:31:03.541 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:03.541 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:31:03.541 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:03.541 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:03.541 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:03.541 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:19360 3.3 3:c34e5e51:::rbd_data.112d92027590.0000000000000024:f [read 1251328~1024 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:31:03.541 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:27.063017+0000", 2024-09-14T06:31:03.541 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 96.428117009999994, 2024-09-14T06:31:03.541 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047453823999999999, 2024-09-14T06:31:03.541 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:31:03.542 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:31:03.542 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:31:03.542 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:31:03.542 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:31:03.542 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:03.542 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 19360 2024-09-14T06:31:03.542 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:03.542 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:31:03.542 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:03.542 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:31:03.542 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.063017+0000", 2024-09-14T06:31:03.542 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:03.542 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:03.542 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:03.542 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:31:03.543 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.063017+0000", 2024-09-14T06:31:03.543 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:03.543 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:03.543 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:03.543 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:31:03.543 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.064977+0000", 2024-09-14T06:31:03.543 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.0019597080000000001 2024-09-14T06:31:03.543 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:03.543 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:03.543 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:31:03.543 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.064987+0000", 2024-09-14T06:31:03.543 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.9620000000000005e-06 2024-09-14T06:31:03.543 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:03.543 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:03.543 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:31:03.544 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065000+0000", 2024-09-14T06:31:03.544 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.3173000000000001e-05 2024-09-14T06:31:03.544 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:03.544 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:03.544 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:31:03.544 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065005+0000", 2024-09-14T06:31:03.544 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.9960000000000001e-06 2024-09-14T06:31:03.544 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:03.544 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:03.544 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:31:03.544 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.110314+0000", 2024-09-14T06:31:03.544 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.04530899 2024-09-14T06:31:03.544 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:03.544 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:03.545 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:31:03.545 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.110398+0000", 2024-09-14T06:31:03.545 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.3539000000000001e-05 2024-09-14T06:31:03.545 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:03.545 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:03.545 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:31:03.545 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.110471+0000", 2024-09-14T06:31:03.545 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.3455999999999996e-05 2024-09-14T06:31:03.545 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:03.545 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:31:03.545 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:03.545 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:03.545 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:03.545 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:19365 3.3 3:c34e5e51:::rbd_data.112d92027590.0000000000000024:f [read 1302528~1024 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:31:03.546 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:27.065058+0000", 2024-09-14T06:31:03.546 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 96.426076339999995, 2024-09-14T06:31:03.546 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046105526000000001, 2024-09-14T06:31:03.546 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:31:03.546 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:31:03.546 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:31:03.546 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:31:03.546 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:31:03.546 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:03.546 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 19365 2024-09-14T06:31:03.546 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:03.546 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:31:03.546 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:03.547 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:31:03.547 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065058+0000", 2024-09-14T06:31:03.547 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:03.547 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:03.547 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:03.547 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:31:03.547 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065058+0000", 2024-09-14T06:31:03.547 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:03.547 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:03.547 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:03.547 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:31:03.547 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065058+0000", 2024-09-14T06:31:03.547 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.1899999999999999e-07 2024-09-14T06:31:03.547 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:03.548 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:03.548 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:31:03.548 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065059+0000", 2024-09-14T06:31:03.548 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.1299999999999998e-07 2024-09-14T06:31:03.548 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:03.548 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:03.548 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:31:03.548 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065063+0000", 2024-09-14T06:31:03.548 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.1099999999999996e-06 2024-09-14T06:31:03.548 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:03.548 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:03.548 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:31:03.548 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065070+0000", 2024-09-14T06:31:03.548 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.973e-06 2024-09-14T06:31:03.549 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:03.549 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:03.549 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:31:03.549 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.111014+0000", 2024-09-14T06:31:03.549 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.045943373000000003 2024-09-14T06:31:03.549 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:03.549 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:03.549 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:31:03.549 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.111073+0000", 2024-09-14T06:31:03.549 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.8987999999999998e-05 2024-09-14T06:31:03.549 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:03.549 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:03.549 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:31:03.549 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.111163+0000", 2024-09-14T06:31:03.550 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.0950000000000002e-05 2024-09-14T06:31:03.550 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:03.550 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:31:03.550 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:03.550 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:03.550 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:03.550 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:19366 3.3 3:c34e5e51:::rbd_data.112d92027590.0000000000000024:f [read 1312768~1024 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:31:03.550 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:27.065074+0000", 2024-09-14T06:31:03.550 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 96.426060489999998, 2024-09-14T06:31:03.550 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046289858000000003, 2024-09-14T06:31:03.550 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:31:03.550 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:31:03.550 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:31:03.550 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:31:03.550 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:31:03.551 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:03.551 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 19366 2024-09-14T06:31:03.551 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:03.551 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:31:03.551 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:03.551 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:31:03.551 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065074+0000", 2024-09-14T06:31:03.551 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:03.551 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:03.551 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:03.551 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:31:03.551 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065074+0000", 2024-09-14T06:31:03.551 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:03.551 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:03.552 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:03.552 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:31:03.552 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065074+0000", 2024-09-14T06:31:03.552 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.3900000000000001e-07 2024-09-14T06:31:03.552 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:03.552 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:03.552 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:31:03.552 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065078+0000", 2024-09-14T06:31:03.552 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.3229999999999999e-06 2024-09-14T06:31:03.552 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:03.552 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:03.552 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:31:03.552 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065082+0000", 2024-09-14T06:31:03.552 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.8299999999999998e-06 2024-09-14T06:31:03.552 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:03.553 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:03.553 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:31:03.553 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065085+0000", 2024-09-14T06:31:03.553 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.8660000000000002e-06 2024-09-14T06:31:03.553 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:03.553 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:03.553 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:31:03.553 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.111270+0000", 2024-09-14T06:31:03.553 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046185035999999999 2024-09-14T06:31:03.553 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:03.553 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:03.553 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:31:03.553 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.111311+0000", 2024-09-14T06:31:03.554 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.0935e-05 2024-09-14T06:31:03.554 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:03.554 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:03.554 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:31:03.554 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.111364+0000", 2024-09-14T06:31:03.554 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.2629e-05 2024-09-14T06:31:03.554 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:03.554 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:31:03.554 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:03.554 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:03.554 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:03.554 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:19367 3.3 3:c34e5e51:::rbd_data.112d92027590.0000000000000024:f [read 1323008~1024 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:31:03.554 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:27.065115+0000", 2024-09-14T06:31:03.554 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 96.426018928999994, 2024-09-14T06:31:03.555 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046407003000000002, 2024-09-14T06:31:03.555 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:31:03.555 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:31:03.555 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:31:03.555 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:31:03.555 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:31:03.555 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:03.555 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 19367 2024-09-14T06:31:03.555 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:03.555 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:31:03.555 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:03.555 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:31:03.555 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065115+0000", 2024-09-14T06:31:03.555 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:03.556 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:03.556 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:03.556 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:31:03.556 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065115+0000", 2024-09-14T06:31:03.556 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:03.556 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:03.556 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:03.556 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:31:03.556 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065116+0000", 2024-09-14T06:31:03.556 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.4999999999999998e-07 2024-09-14T06:31:03.556 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:03.556 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:03.556 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:31:03.556 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065117+0000", 2024-09-14T06:31:03.557 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.189e-06 2024-09-14T06:31:03.557 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:03.557 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:03.557 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:31:03.557 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065122+0000", 2024-09-14T06:31:03.557 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.3709999999999999e-06 2024-09-14T06:31:03.557 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:03.557 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:03.557 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:31:03.557 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065125+0000", 2024-09-14T06:31:03.557 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.8940000000000002e-06 2024-09-14T06:31:03.557 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:03.557 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:03.557 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:31:03.557 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.111402+0000", 2024-09-14T06:31:03.558 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046276645999999998 2024-09-14T06:31:03.558 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:03.558 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:03.558 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:31:03.558 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.111445+0000", 2024-09-14T06:31:03.558 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.3288000000000003e-05 2024-09-14T06:31:03.558 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:03.558 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:03.558 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:31:03.558 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.111522+0000", 2024-09-14T06:31:03.559 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.7164999999999997e-05 2024-09-14T06:31:03.559 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:03.559 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:31:03.559 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:03.559 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:03.559 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:03.559 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:19368 3.3 3:c34e5e51:::rbd_data.112d92027590.0000000000000024:f [read 1333248~1024 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:31:03.559 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:27.065130+0000", 2024-09-14T06:31:03.559 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 96.426004638999999, 2024-09-14T06:31:03.559 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046507275000000001, 2024-09-14T06:31:03.559 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:31:03.559 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:31:03.559 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:31:03.559 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:31:03.560 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:31:03.560 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:03.560 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 19368 2024-09-14T06:31:03.560 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:03.560 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:31:03.560 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:03.560 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:31:03.560 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065130+0000", 2024-09-14T06:31:03.560 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:03.560 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:03.560 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:03.560 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:31:03.560 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065130+0000", 2024-09-14T06:31:03.560 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:03.561 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:03.561 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:03.561 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:31:03.561 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065130+0000", 2024-09-14T06:31:03.561 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.0800000000000001e-07 2024-09-14T06:31:03.561 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:03.561 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:03.561 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:31:03.561 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065131+0000", 2024-09-14T06:31:03.561 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.8700000000000004e-07 2024-09-14T06:31:03.561 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:03.561 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:03.561 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:31:03.561 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065134+0000", 2024-09-14T06:31:03.562 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.1039999999999998e-06 2024-09-14T06:31:03.562 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:03.562 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:03.562 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:31:03.562 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065136+0000", 2024-09-14T06:31:03.562 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.3949999999999999e-06 2024-09-14T06:31:03.562 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:03.562 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:03.562 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:31:03.562 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.111541+0000", 2024-09-14T06:31:03.562 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046404206000000003 2024-09-14T06:31:03.562 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:03.562 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:03.562 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:31:03.562 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.111582+0000", 2024-09-14T06:31:03.563 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.1051999999999999e-05 2024-09-14T06:31:03.563 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:03.563 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:03.563 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:31:03.563 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.111637+0000", 2024-09-14T06:31:03.563 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.5322999999999999e-05 2024-09-14T06:31:03.563 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:03.563 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:31:03.563 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:03.563 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:03.563 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:03.563 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:19369 3.3 3:c34e5e51:::rbd_data.112d92027590.0000000000000024:f [read 1343488~1024 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:31:03.563 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:27.065139+0000", 2024-09-14T06:31:03.563 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 96.425994822000007, 2024-09-14T06:31:03.564 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046682169000000003, 2024-09-14T06:31:03.564 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:31:03.564 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:31:03.564 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:31:03.564 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:31:03.564 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:31:03.564 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:03.564 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 19369 2024-09-14T06:31:03.564 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:03.564 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:31:03.564 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:03.564 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:31:03.564 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065139+0000", 2024-09-14T06:31:03.565 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:03.565 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:03.565 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:03.565 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:31:03.565 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065139+0000", 2024-09-14T06:31:03.565 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:03.565 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:03.565 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:03.565 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:31:03.565 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065140+0000", 2024-09-14T06:31:03.565 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.8299999999999998e-07 2024-09-14T06:31:03.565 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:03.565 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:03.565 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:31:03.566 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065141+0000", 2024-09-14T06:31:03.566 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.3900000000000003e-07 2024-09-14T06:31:03.566 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:03.566 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:03.566 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:31:03.566 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065144+0000", 2024-09-14T06:31:03.566 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.7409999999999998e-06 2024-09-14T06:31:03.566 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:03.566 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:03.566 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:31:03.566 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065147+0000", 2024-09-14T06:31:03.566 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.4150000000000002e-06 2024-09-14T06:31:03.566 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:03.566 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:03.567 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:31:03.567 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.111673+0000", 2024-09-14T06:31:03.567 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046526004000000003 2024-09-14T06:31:03.567 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:03.567 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:03.567 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:31:03.567 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.111749+0000", 2024-09-14T06:31:03.567 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.5741999999999994e-05 2024-09-14T06:31:03.567 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:03.567 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:03.567 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:31:03.567 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.111822+0000", 2024-09-14T06:31:03.567 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.3045000000000003e-05 2024-09-14T06:31:03.567 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:03.568 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:31:03.568 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:03.568 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:03.568 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:03.568 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:19370 3.3 3:c34e5e51:::rbd_data.112d92027590.0000000000000024:f [read 1353728~1024 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:31:03.568 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:27.065150+0000", 2024-09-14T06:31:03.568 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 96.425984092999997, 2024-09-14T06:31:03.568 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046897556999999999, 2024-09-14T06:31:03.568 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:31:03.568 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:31:03.568 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:31:03.568 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:31:03.568 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:31:03.568 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:03.569 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 19370 2024-09-14T06:31:03.569 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:03.569 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:31:03.569 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:03.569 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:31:03.569 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065150+0000", 2024-09-14T06:31:03.569 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:03.569 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:03.569 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:03.569 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:31:03.569 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065150+0000", 2024-09-14T06:31:03.569 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:03.569 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:03.569 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:03.570 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:31:03.570 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065150+0000", 2024-09-14T06:31:03.570 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.0200000000000001e-07 2024-09-14T06:31:03.570 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:03.570 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:03.570 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:31:03.570 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065151+0000", 2024-09-14T06:31:03.570 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.1100000000000004e-07 2024-09-14T06:31:03.570 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:03.570 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:03.570 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:31:03.570 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065154+0000", 2024-09-14T06:31:03.570 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.9639999999999999e-06 2024-09-14T06:31:03.570 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:03.570 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:03.571 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:31:03.571 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065157+0000", 2024-09-14T06:31:03.571 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.4260000000000002e-06 2024-09-14T06:31:03.571 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:03.571 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:03.571 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:31:03.571 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.111861+0000", 2024-09-14T06:31:03.571 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046704268 2024-09-14T06:31:03.571 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:03.571 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:03.571 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:31:03.571 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.111939+0000", 2024-09-14T06:31:03.571 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.8231000000000004e-05 2024-09-14T06:31:03.571 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:03.571 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:03.572 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:31:03.572 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.112048+0000", 2024-09-14T06:31:03.572 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00010855500000000001 2024-09-14T06:31:03.572 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:03.572 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:31:03.572 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:03.572 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:03.572 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:03.572 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:19371 3.3 3:c34e5e51:::rbd_data.112d92027590.0000000000000024:f [read 1363968~1024 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:31:03.572 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:27.065161+0000", 2024-09-14T06:31:03.572 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 96.425973443000004, 2024-09-14T06:31:03.572 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047021944000000003, 2024-09-14T06:31:03.572 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:31:03.572 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:31:03.573 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:31:03.573 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:31:03.573 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:31:03.573 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:03.573 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 19371 2024-09-14T06:31:03.573 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:03.573 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:31:03.573 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:03.573 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:31:03.573 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065161+0000", 2024-09-14T06:31:03.573 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:03.573 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:03.573 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:03.573 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:31:03.574 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065161+0000", 2024-09-14T06:31:03.574 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:03.574 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:03.574 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:03.574 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:31:03.574 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065161+0000", 2024-09-14T06:31:03.574 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.0200000000000001e-07 2024-09-14T06:31:03.574 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:03.574 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:03.574 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:31:03.574 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065166+0000", 2024-09-14T06:31:03.574 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.9869999999999999e-06 2024-09-14T06:31:03.574 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:03.574 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:03.574 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:31:03.575 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065169+0000", 2024-09-14T06:31:03.575 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.3799999999999998e-06 2024-09-14T06:31:03.575 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:03.575 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:03.575 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:31:03.575 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065172+0000", 2024-09-14T06:31:03.575 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.4830000000000002e-06 2024-09-14T06:31:03.575 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:03.575 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:03.575 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:31:03.575 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.112071+0000", 2024-09-14T06:31:03.575 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046899529000000002 2024-09-14T06:31:03.575 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:03.575 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:03.576 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:31:03.576 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.112128+0000", 2024-09-14T06:31:03.576 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.6341999999999997e-05 2024-09-14T06:31:03.576 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:03.576 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:03.576 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:31:03.576 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.112183+0000", 2024-09-14T06:31:03.576 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.5021000000000001e-05 2024-09-14T06:31:03.576 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:03.576 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:31:03.576 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:03.576 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:03.576 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:03.576 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:19372 3.3 3:c34e5e51:::rbd_data.112d92027590.0000000000000024:f [read 1374208~1024 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:31:03.577 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:27.065176+0000", 2024-09-14T06:31:03.577 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 96.425958292999994, 2024-09-14T06:31:03.577 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047188889999999997, 2024-09-14T06:31:03.577 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:31:03.577 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:31:03.577 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:31:03.577 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:31:03.577 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:31:03.577 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:03.577 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 19372 2024-09-14T06:31:03.577 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:03.577 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:31:03.577 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:03.577 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:31:03.577 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065176+0000", 2024-09-14T06:31:03.578 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:03.578 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:03.578 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:03.578 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:31:03.578 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065176+0000", 2024-09-14T06:31:03.578 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:03.578 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:03.578 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:03.578 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:31:03.578 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.071117+0000", 2024-09-14T06:31:03.578 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.0059404990000000001 2024-09-14T06:31:03.578 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:03.578 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:03.578 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:31:03.578 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.071120+0000", 2024-09-14T06:31:03.579 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.4340000000000001e-06 2024-09-14T06:31:03.579 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:03.579 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:03.579 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:31:03.579 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.071129+0000", 2024-09-14T06:31:03.579 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.8389999999999996e-06 2024-09-14T06:31:03.579 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:03.579 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:03.579 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:31:03.579 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.071134+0000", 2024-09-14T06:31:03.579 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.412e-06 2024-09-14T06:31:03.579 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:03.579 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:03.579 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:31:03.580 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.112221+0000", 2024-09-14T06:31:03.580 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.041086473999999998 2024-09-14T06:31:03.580 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:03.580 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:03.580 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:31:03.580 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.112299+0000", 2024-09-14T06:31:03.580 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.8746999999999997e-05 2024-09-14T06:31:03.580 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:03.580 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:03.580 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:31:03.580 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.112365+0000", 2024-09-14T06:31:03.580 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.5485000000000001e-05 2024-09-14T06:31:03.580 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:03.580 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:31:03.580 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:03.581 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:03.581 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:31:03.581 INFO:teuthology.orchestra.run.smithi138.stdout:} 2024-09-14T06:31:03.582 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-14T06:31:03.585 INFO:teuthology.orchestra.run.smithi094.stdout:4153 write 0x4193194 thru 0x419ed5c (0xbbc9 bytes) 2024-09-14T06:31:03.585 INFO:teuthology.orchestra.run.smithi094.stdout:4155 trunc from 0xe2d363c to 0xe25e5f7 2024-09-14T06:31:03.585 INFO:teuthology.orchestra.run.smithi094.stdout:4157 write 0xc44cea3 thru 0xc44fd30 (0x2e8e bytes) 2024-09-14T06:31:03.585 INFO:teuthology.orchestra.run.smithi094.stdout:4158 read 0xd8012fc thru 0xd810352 (0xf057 bytes) 2024-09-14T06:31:03.585 INFO:teuthology.orchestra.run.smithi094.stdout:4159 read 0xa9c7520 thru 0xa9d0e18 (0x98f9 bytes) 2024-09-14T06:31:03.585 INFO:teuthology.orchestra.run.smithi094.stdout:4160 read 0xd24c1e9 thru 0xd24e95a (0x2772 bytes) 2024-09-14T06:31:03.585 INFO:teuthology.orchestra.run.smithi094.stdout:4161 read 0xa61e34a thru 0xa629588 (0xb23f bytes) 2024-09-14T06:31:03.586 INFO:teuthology.orchestra.run.smithi094.stdout:4162 read 0xa199cae thru 0xa19ec80 (0x4fd3 bytes) 2024-09-14T06:31:03.586 INFO:teuthology.orchestra.run.smithi094.stdout:4163 read 0x170eab5 thru 0x171c608 (0xdb54 bytes) 2024-09-14T06:31:03.586 INFO:teuthology.orchestra.run.smithi094.stdout:4165 punch from 0x4baf629 to 0x4bbba2a, (0xc401 bytes) 2024-09-14T06:31:03.586 INFO:teuthology.orchestra.run.smithi094.stdout:4167 punch from 0x8c242aa to 0x8c31e08, (0xdb5e bytes) 2024-09-14T06:31:03.586 INFO:teuthology.orchestra.run.smithi094.stdout:4168 write 0xb32ef6 thru 0xb3c36d (0x9478 bytes) 2024-09-14T06:31:03.586 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-14T06:31:03.586 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-14T06:31:03.586 INFO:teuthology.orchestra.run.smithi094.stdout:4173 punch from 0x5f781b0 to 0x5f85c22, (0xda72 bytes) 2024-09-14T06:31:03.586 INFO:teuthology.orchestra.run.smithi094.stdout:4174 read 0x6ca0b67 thru 0x6cb050b (0xf9a5 bytes) 2024-09-14T06:31:03.586 INFO:teuthology.orchestra.run.smithi094.stdout:4175 punch from 0x69b25ec to 0x69bc860, (0xa274 bytes) 2024-09-14T06:31:03.586 INFO:teuthology.orchestra.run.smithi094.stdout:4176 writesame 0x656722b thru 0x6567f46 data_size 0x1(0xd1c bytes) 2024-09-14T06:31:03.586 INFO:teuthology.orchestra.run.smithi094.stdout:4177 read 0x6fa4230 thru 0x6fa6a33 (0x2804 bytes) 2024-09-14T06:31:03.586 INFO:teuthology.orchestra.run.smithi094.stdout:4179 read 0x2b27874 thru 0x2b31ed9 (0xa666 bytes) 2024-09-14T06:31:03.586 INFO:teuthology.orchestra.run.smithi094.stdout:4180 read 0x2f355ba thru 0x2f3e481 (0x8ec8 bytes) 2024-09-14T06:31:03.587 INFO:teuthology.orchestra.run.smithi094.stdout:4182 write 0x8870e15 thru 0x8874610 (0x37fc bytes) 2024-09-14T06:31:03.587 INFO:teuthology.orchestra.run.smithi094.stdout:4183 punch from 0xafabc97 to 0xafb2239, (0x65a2 bytes) 2024-09-14T06:31:03.587 INFO:teuthology.orchestra.run.smithi094.stdout:4184 read 0xacf62a2 thru 0xacfcf40 (0x6c9f bytes) 2024-09-14T06:31:03.587 INFO:teuthology.orchestra.run.smithi094.stdout:4185 punch from 0x9df9063 to 0x9dfe8ca, (0x5867 bytes) 2024-09-14T06:31:03.587 INFO:teuthology.orchestra.run.smithi094.stdout:4186 write 0xa02831 thru 0xa0d0a7 (0xa877 bytes) 2024-09-14T06:31:03.587 INFO:teuthology.orchestra.run.smithi094.stdout:4187 read 0x913e014 thru 0x913e92b (0x918 bytes) 2024-09-14T06:31:03.587 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-14T06:31:03.587 INFO:teuthology.orchestra.run.smithi094.stdout:4190 read 0xa3496f2 thru 0xa3580e1 (0xe9f0 bytes) 2024-09-14T06:31:03.587 INFO:teuthology.orchestra.run.smithi094.stdout:4191 writesame 0xa68840d thru 0xa6967b5 data_size 0x1(0xe3a9 bytes) 2024-09-14T06:31:03.587 INFO:teuthology.orchestra.run.smithi094.stdout:4193 trunc from 0xe25e5f7 to 0x61fb5cb 2024-09-14T06:31:03.587 INFO:teuthology.orchestra.run.smithi094.stdout:4194 write 0xfa34ad thru 0xfa717a (0x3cce bytes) 2024-09-14T06:31:03.587 INFO:teuthology.orchestra.run.smithi094.stdout:4195 trunc from 0x61fb5cb to 0x5fceddd 2024-09-14T06:31:03.587 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-14T06:31:03.587 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-14T06:31:03.588 INFO:teuthology.orchestra.run.smithi094.stdout:4201 read 0x4d71df thru 0x4d892d (0x174f bytes) 2024-09-14T06:31:03.588 INFO:teuthology.orchestra.run.smithi094.stdout:4202 trunc from 0x5fceddd to 0xb1f17ef 2024-09-14T06:31:03.590 INFO:teuthology.orchestra.run.smithi094.stdout:4203 read 0x935200a thru 0x9356dbb (0x4db2 bytes) 2024-09-14T06:31:03.591 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-14T06:31:03.591 INFO:teuthology.orchestra.run.smithi094.stdout:4205 read 0x9cd69a0 thru 0x9cd7c4e (0x12af bytes) 2024-09-14T06:31:03.591 INFO:teuthology.orchestra.run.smithi094.stdout:4206 punch from 0x35cd1ae to 0x35da8f7, (0xd749 bytes) 2024-09-14T06:31:03.592 INFO:teuthology.orchestra.run.smithi094.stdout:4207 trunc from 0xb1f17ef to 0x9d760b8 2024-09-14T06:31:03.599 INFO:teuthology.orchestra.run.smithi094.stdout:4210 read 0x6ca9419 thru 0x6cad8ca (0x44b2 bytes) 2024-09-14T06:31:03.599 INFO:teuthology.orchestra.run.smithi094.stdout:4212 read 0x7c3afa2 thru 0x7c438e2 (0x8941 bytes) 2024-09-14T06:31:03.599 INFO:teuthology.orchestra.run.smithi094.stdout:4214 read 0x888063b thru 0x88819fa (0x13c0 bytes) 2024-09-14T06:31:03.599 INFO:teuthology.orchestra.run.smithi094.stdout:4216 trunc from 0x9d760b8 to 0x9fc3ca3 2024-09-14T06:31:03.604 INFO:teuthology.orchestra.run.smithi094.stdout:4218 writesame 0x806df76 thru 0x807202f data_size 0x5(0x40ba bytes) 2024-09-14T06:31:03.611 INFO:teuthology.orchestra.run.smithi094.stdout:4219 write 0x2946642 thru 0x294ea83 (0x8442 bytes) 2024-09-14T06:31:03.616 INFO:teuthology.orchestra.run.smithi094.stdout:4221 write 0x5feaca2 thru 0x5ff2890 (0x7bef bytes) 2024-09-14T06:31:03.621 INFO:teuthology.orchestra.run.smithi094.stdout:4222 write 0xcca486 thru 0xccb547 (0x10c2 bytes) 2024-09-14T06:31:03.622 INFO:teuthology.orchestra.run.smithi094.stdout:4223 write 0x5036143 thru 0x5038c37 (0x2af5 bytes) 2024-09-14T06:31:03.623 INFO:tasks.ceph.osd.5.smithi138.stderr:2024-09-14T06:31:03.622+0000 7fe13746f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33618) UID: 0 2024-09-14T06:31:03.624 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-14T06:31:03.624 INFO:teuthology.orchestra.run.smithi094.stdout:4227 punch from 0x976d70a to 0x976fedb, (0x27d1 bytes) 2024-09-14T06:31:03.624 INFO:teuthology.orchestra.run.smithi094.stdout:4229 punch from 0x7283889 to 0x7284c6b, (0x13e2 bytes) 2024-09-14T06:31:03.624 INFO:teuthology.orchestra.run.smithi094.stdout:4230 write 0x673c371 thru 0x673c9da (0x66a bytes) 2024-09-14T06:31:03.630 INFO:teuthology.orchestra.run.smithi094.stdout:4231 trunc from 0x9fc3ca3 to 0xd293ce6 2024-09-14T06:31:03.637 INFO:teuthology.orchestra.run.smithi094.stdout:4232 punch from 0x9697dd1 to 0x969f09f, (0x72ce bytes) 2024-09-14T06:31:03.638 INFO:teuthology.orchestra.run.smithi094.stdout:4234 punch from 0x81f20f6 to 0x82003b5, (0xe2bf bytes) 2024-09-14T06:31:03.638 INFO:teuthology.orchestra.run.smithi094.stdout:4237 punch from 0x4de037 to 0x4deafd, (0xac6 bytes) 2024-09-14T06:31:03.638 INFO:teuthology.orchestra.run.smithi094.stdout:4239 write 0x5c99946 thru 0x5ca59dc (0xc097 bytes) 2024-09-14T06:31:03.645 INFO:teuthology.orchestra.run.smithi094.stdout:4240 write 0x441f189 thru 0x442a1c3 (0xb03b bytes) 2024-09-14T06:31:03.648 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-14T06:31:03.648 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-14T06:31:03.648 INFO:teuthology.orchestra.run.smithi094.stdout:4244 writesame 0x31d805 thru 0x322a6a data_size 0x35(0x5266 bytes) 2024-09-14T06:31:03.650 INFO:teuthology.orchestra.run.smithi094.stdout:4246 punch from 0x7d8b96f to 0x7d8c316, (0x9a7 bytes) 2024-09-14T06:31:03.650 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-14T06:31:03.651 INFO:teuthology.orchestra.run.smithi094.stdout:4249 trunc from 0xd293ce6 to 0xc3696ac 2024-09-14T06:31:03.662 INFO:teuthology.orchestra.run.smithi094.stdout:4250 write 0xcbced0a thru 0xcbd3a8b (0x4d82 bytes) 2024-09-14T06:31:03.667 INFO:teuthology.orchestra.run.smithi094.stdout:4251 read 0x950863e thru 0x950fff4 (0x79b7 bytes) 2024-09-14T06:31:03.667 INFO:teuthology.orchestra.run.smithi094.stdout:4252 punch from 0x340049d to 0x34047f8, (0x435b bytes) 2024-09-14T06:31:03.668 INFO:teuthology.orchestra.run.smithi094.stdout:4253 writesame 0x8c59811 thru 0x8c5c250 data_size 0x1a(0x2a40 bytes) 2024-09-14T06:31:03.674 INFO:teuthology.orchestra.run.smithi094.stdout:4254 read 0xb1e8c92 thru 0xb1ea5a8 (0x1917 bytes) 2024-09-14T06:31:03.680 INFO:teuthology.orchestra.run.smithi094.stdout:4255 trunc from 0xcbd3a8c to 0x80ea41c 2024-09-14T06:31:03.689 INFO:tasks.ceph.osd.3.smithi094.stderr:2024-09-14T06:31:03.688+0000 7feeeafe8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33655) UID: 0 2024-09-14T06:31:03.696 INFO:teuthology.orchestra.run.smithi094.stdout:4256 write 0x383aec9 thru 0x383cbb6 (0x1cee bytes) 2024-09-14T06:31:03.699 INFO:teuthology.orchestra.run.smithi094.stdout:4257 read 0x801a401 thru 0x80297a5 (0xf3a5 bytes) 2024-09-14T06:31:03.700 INFO:teuthology.orchestra.run.smithi094.stdout:4259 read 0x5b50569 thru 0x5b58a49 (0x84e1 bytes) 2024-09-14T06:31:03.700 INFO:teuthology.orchestra.run.smithi094.stdout:4260 read 0x60a414c thru 0x60aaeba (0x6d6f bytes) 2024-09-14T06:31:03.708 INFO:teuthology.orchestra.run.smithi094.stdout:4261 write 0x8ef27ec thru 0x8ef58b3 (0x30c8 bytes) 2024-09-14T06:31:03.712 INFO:teuthology.orchestra.run.smithi094.stdout:4262 read 0x2986d88 thru 0x298764a (0x8c3 bytes) 2024-09-14T06:31:03.713 INFO:teuthology.orchestra.run.smithi094.stdout:4263 writesame 0x51c9070 thru 0x51cb47b data_size 0x6(0x240c bytes) 2024-09-14T06:31:03.714 INFO:teuthology.orchestra.run.smithi094.stdout:4264 write 0x6bdcfaa thru 0x6be0846 (0x389d bytes) 2024-09-14T06:31:03.724 INFO:teuthology.orchestra.run.smithi094.stdout:4265 write 0x2b070fd thru 0x2b07cad (0xbb1 bytes) 2024-09-14T06:31:03.726 INFO:teuthology.orchestra.run.smithi094.stdout:4266 read 0x3801086 thru 0x380e2f6 (0xd271 bytes) 2024-09-14T06:31:03.726 INFO:teuthology.orchestra.run.smithi094.stdout:truncating to largest ever: 0xee34800 2024-09-14T06:31:03.732 INFO:teuthology.orchestra.run.smithi138.stdout:{ 2024-09-14T06:31:03.732 INFO:teuthology.orchestra.run.smithi138.stdout: "ops": [], 2024-09-14T06:31:03.733 INFO:teuthology.orchestra.run.smithi138.stdout: "num_ops": 0 2024-09-14T06:31:03.733 INFO:teuthology.orchestra.run.smithi138.stdout:} 2024-09-14T06:31:03.733 INFO:teuthology.orchestra.run.smithi094.stdout:4267 trunc from 0x8ef58b4 to 0xee34800 2024-09-14T06:31:03.739 INFO:teuthology.orchestra.run.smithi094.stdout:4268 punch from 0x2aecc6c to 0x2aeeec9, (0x225d bytes) 2024-09-14T06:31:03.739 INFO:teuthology.orchestra.run.smithi094.stdout:4269 writesame 0x41605dc thru 0x416f194 data_size 0x9(0xebb9 bytes) 2024-09-14T06:31:03.743 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-14T06:31:03.744 INFO:teuthology.orchestra.run.smithi094.stdout:4271 punch from 0x3c6f134 to 0x3c7ef61, (0xfe2d bytes) 2024-09-14T06:31:03.744 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-14T06:31:03.758 INFO:teuthology.orchestra.run.smithi094.stdout:4273 trunc from 0xee34800 to 0x34273f9 2024-09-14T06:31:03.788 INFO:teuthology.orchestra.run.smithi094.stdout:4274 trunc from 0x34273f9 to 0xd56857c 2024-09-14T06:31:03.790 INFO:teuthology.orchestra.run.smithi094.stderr:reweighted osd.4 to 1 (10000) 2024-09-14T06:31:03.790 INFO:teuthology.orchestra.run.smithi094.stderr:nodeep-scrub is set 2024-09-14T06:31:03.790 INFO:tasks.ceph.osd.2.smithi094.stderr:2024-09-14T06:31:03.789+0000 7faf3269b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33649) UID: 0 2024-09-14T06:31:03.806 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 5 1 2024-09-14T06:31:03.814 INFO:teuthology.orchestra.run.smithi094.stdout:4275 trunc from 0xd56857c to 0x21d314e 2024-09-14T06:31:03.828 INFO:teuthology.orchestra.run.smithi094.stdout:4277 punch from 0x1f3428f to 0x1f36050, (0x1dc1 bytes) 2024-09-14T06:31:03.828 INFO:teuthology.orchestra.run.smithi094.stdout:4278 read 0x1f02d1 thru 0x1fccc5 (0xc9f5 bytes) 2024-09-14T06:31:03.845 INFO:teuthology.orchestra.run.smithi094.stdout:4282 write 0xaa057ef thru 0xaa1207c (0xc88e bytes) 2024-09-14T06:31:03.857 INFO:teuthology.orchestra.run.smithi138.stdout:{ 2024-09-14T06:31:03.857 INFO:teuthology.orchestra.run.smithi138.stdout: "ops": [], 2024-09-14T06:31:03.857 INFO:teuthology.orchestra.run.smithi138.stdout: "complaint_time": 30, 2024-09-14T06:31:03.857 INFO:teuthology.orchestra.run.smithi138.stdout: "num_blocked_ops": 0 2024-09-14T06:31:03.857 INFO:teuthology.orchestra.run.smithi138.stdout:} 2024-09-14T06:31:03.859 INFO:teuthology.orchestra.run.smithi094.stdout:4283 trunc from 0xaa1207d to 0xe701d6a 2024-09-14T06:31:03.865 INFO:teuthology.orchestra.run.smithi094.stdout:4284 punch from 0xd17e7b4 to 0xd185c4a, (0x7496 bytes) 2024-09-14T06:31:03.865 INFO:teuthology.orchestra.run.smithi094.stdout:4287 punch from 0xa5343 to 0xb1bd0, (0xc88d bytes) 2024-09-14T06:31:03.865 INFO:teuthology.orchestra.run.smithi094.stdout:4291 writesame 0x83e4d4b thru 0x83eb37b data_size 0x1(0x6631 bytes) 2024-09-14T06:31:03.866 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-14T06:31:03.878 INFO:teuthology.orchestra.run.smithi094.stdout:4292 trunc from 0xe701d6a to 0xaa65258 2024-09-14T06:31:03.886 INFO:teuthology.orchestra.run.smithi094.stdout:4294 read 0xa703af2 thru 0xa708ed9 (0x53e8 bytes) 2024-09-14T06:31:03.886 INFO:teuthology.orchestra.run.smithi094.stdout:4295 read 0x494f10e thru 0x4952af9 (0x39ec bytes) 2024-09-14T06:31:03.889 INFO:teuthology.orchestra.run.smithi094.stdout:4296 trunc from 0xaa65258 to 0x757004e 2024-09-14T06:31:03.909 INFO:tasks.ceph.osd.7.smithi138.stderr:2024-09-14T06:31:03.908+0000 7f3c6a21d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33623) UID: 0 2024-09-14T06:31:03.911 INFO:teuthology.orchestra.run.smithi094.stdout:4297 writesame 0xa6ae75c thru 0xa6b074b data_size 0x7(0x1ff0 bytes) 2024-09-14T06:31:03.911 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-14T06:31:03.911 INFO:teuthology.orchestra.run.smithi094.stdout:4300 writesame 0x83378 thru 0x8e588 data_size 0xf(0xb211 bytes) 2024-09-14T06:31:03.914 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-14T06:31:03.914 INFO:teuthology.orchestra.run.smithi094.stdout:4302 read 0x7938256 thru 0x793d72e (0x54d9 bytes) 2024-09-14T06:31:03.915 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-14T06:31:03.915 INFO:teuthology.orchestra.run.smithi094.stdout:4306 write 0x8741c52 thru 0x8743e7f (0x222e bytes) 2024-09-14T06:31:03.918 INFO:teuthology.orchestra.run.smithi094.stdout:4308 read 0xd0b8d5 thru 0xd1244d (0x6b79 bytes) 2024-09-14T06:31:03.919 INFO:teuthology.orchestra.run.smithi094.stdout:4309 read 0x2d13c13 thru 0x2d1801e (0x440c bytes) 2024-09-14T06:31:03.919 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-14T06:31:03.930 INFO:teuthology.orchestra.run.smithi094.stdout:4311 write 0xebae93f thru 0xebbb039 (0xc6fb bytes) 2024-09-14T06:31:03.937 INFO:teuthology.orchestra.run.smithi094.stdout:4312 punch from 0xbf632ae to 0xbf6916a, (0x5ebc bytes) 2024-09-14T06:31:03.954 INFO:teuthology.orchestra.run.smithi094.stdout:4313 trunc from 0xebbb03a to 0x3e90fa 2024-09-14T06:31:03.982 INFO:teuthology.orchestra.run.smithi138.stdout:{ 2024-09-14T06:31:03.982 INFO:teuthology.orchestra.run.smithi138.stdout: "size": 20, 2024-09-14T06:31:03.982 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 600, 2024-09-14T06:31:03.982 INFO:teuthology.orchestra.run.smithi138.stdout: "ops": [ 2024-09-14T06:31:03.982 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:03.982 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:11453 3.1d 3:bcad7303:::rbd_data.112d92027590.0000000000000001:f [read 94208~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:31:03.983 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:25.654375+0000", 2024-09-14T06:31:03.983 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 98.325887562999995, 2024-09-14T06:31:03.983 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052449318000000002, 2024-09-14T06:31:03.983 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:31:03.983 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:31:03.983 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:31:03.983 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:31:03.983 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:31:03.983 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:03.984 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 11453 2024-09-14T06:31:03.984 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:03.984 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:31:03.984 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:03.984 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:31:03.984 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654375+0000", 2024-09-14T06:31:03.984 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:03.984 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:03.985 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:03.985 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:31:03.985 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654375+0000", 2024-09-14T06:31:03.985 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:03.985 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:03.985 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:03.985 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:31:03.985 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654496+0000", 2024-09-14T06:31:03.985 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00012081 2024-09-14T06:31:03.985 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:03.985 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:03.986 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:31:03.986 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654497+0000", 2024-09-14T06:31:03.986 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.8789999999999999e-06 2024-09-14T06:31:03.986 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:03.986 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:03.986 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:31:03.986 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654501+0000", 2024-09-14T06:31:03.986 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.8999999999999999e-06 2024-09-14T06:31:03.986 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:03.986 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:03.986 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:31:03.986 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654505+0000", 2024-09-14T06:31:03.986 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.2229999999999999e-06 2024-09-14T06:31:03.986 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:03.987 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:03.987 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:31:03.987 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.705917+0000", 2024-09-14T06:31:03.987 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.051412501999999999 2024-09-14T06:31:03.987 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:03.987 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:03.987 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:31:03.987 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.705999+0000", 2024-09-14T06:31:03.987 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.1551999999999994e-05 2024-09-14T06:31:03.987 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:03.988 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:03.988 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:31:03.988 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.706824+0000", 2024-09-14T06:31:03.988 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00082545199999999996 2024-09-14T06:31:03.988 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:03.988 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:31:03.988 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:03.988 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:03.988 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:03.988 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:11454 3.1d 3:bcad7303:::rbd_data.112d92027590.0000000000000001:f [read 104448~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:31:03.988 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:25.654510+0000", 2024-09-14T06:31:03.988 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 98.325752773000005, 2024-09-14T06:31:03.989 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052610713000000003, 2024-09-14T06:31:03.989 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:31:03.989 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:31:03.989 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:31:03.989 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:31:03.989 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:31:03.989 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:03.989 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 11454 2024-09-14T06:31:03.989 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:03.989 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:31:03.989 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:03.989 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:31:03.989 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654510+0000", 2024-09-14T06:31:03.990 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:03.990 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:03.990 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:03.990 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:31:03.990 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654510+0000", 2024-09-14T06:31:03.990 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:03.991 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:03.991 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:03.991 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:31:03.991 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654825+0000", 2024-09-14T06:31:03.991 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00031522400000000002 2024-09-14T06:31:03.991 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:03.991 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:03.992 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:31:03.992 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654827+0000", 2024-09-14T06:31:03.992 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.9709999999999998e-06 2024-09-14T06:31:03.992 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:03.992 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:03.992 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:31:03.992 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654832+0000", 2024-09-14T06:31:03.992 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.4099999999999999e-06 2024-09-14T06:31:03.992 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:03.992 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:03.992 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:31:03.993 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654835+0000", 2024-09-14T06:31:03.993 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.14e-06 2024-09-14T06:31:03.993 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:03.993 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:03.993 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:31:03.993 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.706851+0000", 2024-09-14T06:31:03.993 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052015761000000001 2024-09-14T06:31:03.993 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:03.993 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:03.993 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:31:03.993 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.706934+0000", 2024-09-14T06:31:03.993 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.2484999999999994e-05 2024-09-14T06:31:03.993 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:03.993 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:03.994 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:31:03.994 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.707120+0000", 2024-09-14T06:31:03.994 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00018672200000000001 2024-09-14T06:31:03.994 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:03.994 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:31:03.994 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:03.994 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:03.994 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:03.994 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:11455 3.1d 3:bcad7303:::rbd_data.112d92027590.0000000000000001:f [read 114688~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:31:03.994 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:25.654841+0000", 2024-09-14T06:31:03.994 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 98.325421444, 2024-09-14T06:31:03.994 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052579898, 2024-09-14T06:31:03.994 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:31:03.994 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:31:03.995 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:31:03.995 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:31:03.995 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:31:03.995 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:03.995 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 11455 2024-09-14T06:31:03.995 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:03.995 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:31:03.995 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:03.995 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:31:03.995 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654841+0000", 2024-09-14T06:31:03.995 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:03.995 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:03.996 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:03.996 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:31:03.996 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654841+0000", 2024-09-14T06:31:03.996 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:03.996 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:03.996 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:03.996 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:31:03.996 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654841+0000", 2024-09-14T06:31:03.996 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.7300000000000002e-07 2024-09-14T06:31:03.996 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:03.996 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:03.996 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:31:03.996 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654845+0000", 2024-09-14T06:31:03.996 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.0520000000000003e-06 2024-09-14T06:31:03.997 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:03.997 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:03.997 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:31:03.997 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654849+0000", 2024-09-14T06:31:03.997 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.0459999999999999e-06 2024-09-14T06:31:03.997 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:03.997 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:03.997 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:31:03.997 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654852+0000", 2024-09-14T06:31:03.998 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.9830000000000001e-06 2024-09-14T06:31:03.998 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:03.998 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:03.998 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:31:03.998 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.707159+0000", 2024-09-14T06:31:03.998 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052306826000000001 2024-09-14T06:31:03.998 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:03.998 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:03.998 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:31:03.998 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.707242+0000", 2024-09-14T06:31:03.998 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.2595000000000005e-05 2024-09-14T06:31:03.998 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:03.998 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:03.998 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:31:03.999 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.707421+0000", 2024-09-14T06:31:03.999 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.000179123 2024-09-14T06:31:03.999 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:03.999 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:31:03.999 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:03.999 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:03.999 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:03.999 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:11456 3.1d 3:bcad7303:::rbd_data.112d92027590.0000000000000001:f [read 124928~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:31:03.999 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:25.654856+0000", 2024-09-14T06:31:03.999 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 98.325406598000001, 2024-09-14T06:31:03.999 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052801580000000001, 2024-09-14T06:31:03.999 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:31:03.999 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:31:03.999 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:31:04.000 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:31:04.000 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:31:04.000 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:04.000 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 11456 2024-09-14T06:31:04.000 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:04.000 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:31:04.000 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:04.000 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:31:04.000 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654856+0000", 2024-09-14T06:31:04.000 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:04.000 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:04.000 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:04.000 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:31:04.001 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654856+0000", 2024-09-14T06:31:04.002 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:04.002 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:04.002 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:04.002 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:31:04.002 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654856+0000", 2024-09-14T06:31:04.002 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.29e-07 2024-09-14T06:31:04.002 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:04.002 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:04.002 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:31:04.003 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654857+0000", 2024-09-14T06:31:04.003 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.187e-06 2024-09-14T06:31:04.003 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:04.003 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:04.003 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:31:04.003 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654861+0000", 2024-09-14T06:31:04.003 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.3850000000000002e-06 2024-09-14T06:31:04.003 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:04.003 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:04.003 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:31:04.003 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654863+0000", 2024-09-14T06:31:04.003 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.9500000000000001e-06 2024-09-14T06:31:04.003 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:04.003 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:04.003 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:31:04.004 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.707456+0000", 2024-09-14T06:31:04.004 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052592434 2024-09-14T06:31:04.004 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:04.004 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:04.004 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:31:04.004 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.707516+0000", 2024-09-14T06:31:04.004 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.0167999999999997e-05 2024-09-14T06:31:04.004 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:04.004 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:04.004 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:31:04.004 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.707657+0000", 2024-09-14T06:31:04.004 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00014122700000000001 2024-09-14T06:31:04.004 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:04.004 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:31:04.005 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:04.005 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:04.005 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:04.005 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:11457 3.1d 3:bcad7303:::rbd_data.112d92027590.0000000000000001:f [read 135168~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:31:04.005 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:25.654867+0000", 2024-09-14T06:31:04.005 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 98.325395713000006, 2024-09-14T06:31:04.005 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052921902, 2024-09-14T06:31:04.005 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:31:04.005 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:31:04.005 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:31:04.005 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:31:04.005 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:31:04.005 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:04.006 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 11457 2024-09-14T06:31:04.006 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:04.006 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:31:04.006 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:04.006 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:31:04.006 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654867+0000", 2024-09-14T06:31:04.006 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:04.006 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:04.006 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:04.006 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:31:04.006 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654867+0000", 2024-09-14T06:31:04.006 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:04.006 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:04.006 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:04.007 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:31:04.007 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654972+0000", 2024-09-14T06:31:04.007 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00010584 2024-09-14T06:31:04.007 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:04.007 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:04.007 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:31:04.007 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654975+0000", 2024-09-14T06:31:04.007 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.1169999999999998e-06 2024-09-14T06:31:04.007 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:04.007 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:04.007 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:31:04.007 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654979+0000", 2024-09-14T06:31:04.007 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.1960000000000001e-06 2024-09-14T06:31:04.007 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:04.007 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:04.008 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:31:04.008 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654982+0000", 2024-09-14T06:31:04.008 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.2990000000000001e-06 2024-09-14T06:31:04.008 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:04.008 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:04.008 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:31:04.008 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.707684+0000", 2024-09-14T06:31:04.008 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052701979000000003 2024-09-14T06:31:04.008 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:04.008 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:04.008 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:31:04.008 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.707728+0000", 2024-09-14T06:31:04.008 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.3529999999999998e-05 2024-09-14T06:31:04.008 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:04.009 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:04.009 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:31:04.009 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.707789+0000", 2024-09-14T06:31:04.009 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.0940999999999998e-05 2024-09-14T06:31:04.009 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:04.009 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:31:04.009 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:04.009 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:04.009 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:04.009 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:11458 3.1d 3:bcad7303:::rbd_data.112d92027590.0000000000000001:f [read 145408~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:31:04.009 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:25.654987+0000", 2024-09-14T06:31:04.010 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 98.325275314999999, 2024-09-14T06:31:04.010 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.05291655, 2024-09-14T06:31:04.010 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:31:04.010 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:31:04.010 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:31:04.010 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:31:04.010 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:31:04.010 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:04.010 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 11458 2024-09-14T06:31:04.010 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:04.010 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:31:04.010 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:04.011 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:31:04.011 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654987+0000", 2024-09-14T06:31:04.011 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:04.011 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:04.011 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:04.011 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:31:04.011 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654987+0000", 2024-09-14T06:31:04.011 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:04.011 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:04.011 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:04.011 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:31:04.011 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655077+0000", 2024-09-14T06:31:04.011 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.0253000000000005e-05 2024-09-14T06:31:04.011 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:04.011 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:04.012 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:31:04.012 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655079+0000", 2024-09-14T06:31:04.012 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.9989999999999998e-06 2024-09-14T06:31:04.012 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:04.012 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:04.012 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:31:04.012 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655083+0000", 2024-09-14T06:31:04.012 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.9280000000000003e-06 2024-09-14T06:31:04.012 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:04.012 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:04.012 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:31:04.012 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655086+0000", 2024-09-14T06:31:04.012 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.056e-06 2024-09-14T06:31:04.012 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:04.013 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:04.013 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:31:04.013 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.707805+0000", 2024-09-14T06:31:04.013 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052718627999999997 2024-09-14T06:31:04.013 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:04.013 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:04.013 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:31:04.013 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.707848+0000", 2024-09-14T06:31:04.013 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.2728999999999997e-05 2024-09-14T06:31:04.013 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:04.013 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:04.013 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:31:04.013 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.707904+0000", 2024-09-14T06:31:04.013 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.5957e-05 2024-09-14T06:31:04.013 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:04.014 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:31:04.014 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:04.014 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:04.014 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:04.014 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:11459 3.1d 3:bcad7303:::rbd_data.112d92027590.0000000000000001:f [read 155648~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:31:04.014 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:25.655091+0000", 2024-09-14T06:31:04.014 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 98.325171307000005, 2024-09-14T06:31:04.014 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052944242000000002, 2024-09-14T06:31:04.014 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:31:04.014 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:31:04.014 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:31:04.014 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:31:04.014 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:31:04.014 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:04.015 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 11459 2024-09-14T06:31:04.015 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:04.015 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:31:04.015 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:04.015 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:31:04.015 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655091+0000", 2024-09-14T06:31:04.015 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:04.015 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:04.015 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:04.015 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:31:04.015 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655091+0000", 2024-09-14T06:31:04.015 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:04.015 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:04.015 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:04.016 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:31:04.016 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655237+0000", 2024-09-14T06:31:04.016 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00014570100000000001 2024-09-14T06:31:04.016 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:04.016 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:04.016 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:31:04.016 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655239+0000", 2024-09-14T06:31:04.016 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.8670000000000001e-06 2024-09-14T06:31:04.016 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:04.016 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:04.016 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:31:04.016 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655243+0000", 2024-09-14T06:31:04.017 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.9829999999999998e-06 2024-09-14T06:31:04.017 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:04.017 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:04.017 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:31:04.017 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655246+0000", 2024-09-14T06:31:04.017 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.1540000000000002e-06 2024-09-14T06:31:04.017 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:04.017 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:04.017 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:31:04.017 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.707934+0000", 2024-09-14T06:31:04.018 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052688286000000001 2024-09-14T06:31:04.018 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:04.018 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:04.018 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:31:04.018 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.707977+0000", 2024-09-14T06:31:04.018 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.3291999999999998e-05 2024-09-14T06:31:04.018 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:04.018 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:04.018 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:31:04.018 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.708035+0000", 2024-09-14T06:31:04.018 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.7958999999999999e-05 2024-09-14T06:31:04.018 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:04.018 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:31:04.018 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:04.019 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:04.019 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:04.019 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:11460 3.1d 3:bcad7303:::rbd_data.112d92027590.0000000000000001:f [read 165888~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:31:04.019 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:25.655250+0000", 2024-09-14T06:31:04.019 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 98.325012009999995, 2024-09-14T06:31:04.019 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052910698999999999, 2024-09-14T06:31:04.019 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:31:04.019 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:31:04.019 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:31:04.019 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:31:04.019 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:31:04.019 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:04.019 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 11460 2024-09-14T06:31:04.019 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:04.019 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:31:04.020 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:04.020 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:31:04.020 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655250+0000", 2024-09-14T06:31:04.020 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:04.020 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:04.020 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:04.020 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:31:04.020 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655250+0000", 2024-09-14T06:31:04.020 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:04.020 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:04.020 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:04.020 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:31:04.020 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655356+0000", 2024-09-14T06:31:04.020 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00010535900000000001 2024-09-14T06:31:04.021 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:04.021 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:04.021 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:31:04.021 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655357+0000", 2024-09-14T06:31:04.021 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.8080000000000001e-06 2024-09-14T06:31:04.021 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:04.021 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:04.021 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:31:04.021 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655361+0000", 2024-09-14T06:31:04.021 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.861e-06 2024-09-14T06:31:04.021 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:04.021 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:04.021 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:31:04.022 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655364+0000", 2024-09-14T06:31:04.022 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.1089999999999998e-06 2024-09-14T06:31:04.022 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:04.022 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:04.022 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:31:04.022 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.708065+0000", 2024-09-14T06:31:04.022 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052700595000000003 2024-09-14T06:31:04.022 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:04.022 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:04.022 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:31:04.022 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.708108+0000", 2024-09-14T06:31:04.022 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.2484999999999997e-05 2024-09-14T06:31:04.022 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:04.022 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:04.023 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:31:04.023 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.708161+0000", 2024-09-14T06:31:04.023 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.3482000000000001e-05 2024-09-14T06:31:04.023 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:04.023 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:31:04.023 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:04.023 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:04.023 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:04.023 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:11461 3.1d 3:bcad7303:::rbd_data.112d92027590.0000000000000001:f [read 176128~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:31:04.023 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:25.655370+0000", 2024-09-14T06:31:04.023 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 98.324892727999995, 2024-09-14T06:31:04.023 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052915141999999998, 2024-09-14T06:31:04.023 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:31:04.023 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:31:04.024 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:31:04.024 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:31:04.024 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:31:04.024 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:04.024 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 11461 2024-09-14T06:31:04.024 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:04.024 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:31:04.024 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:04.024 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:31:04.024 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655370+0000", 2024-09-14T06:31:04.024 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:04.024 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:04.024 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:04.024 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:31:04.025 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655370+0000", 2024-09-14T06:31:04.025 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:04.025 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:04.025 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:04.025 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:31:04.025 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655394+0000", 2024-09-14T06:31:04.025 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.4553999999999999e-05 2024-09-14T06:31:04.025 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:04.025 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:04.025 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:31:04.025 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655396+0000", 2024-09-14T06:31:04.025 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.8950000000000001e-06 2024-09-14T06:31:04.025 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:04.025 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:04.025 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:31:04.026 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655401+0000", 2024-09-14T06:31:04.026 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.0089999999999998e-06 2024-09-14T06:31:04.028 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:04.028 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:04.028 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:31:04.028 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655404+0000", 2024-09-14T06:31:04.028 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.3400000000000002e-06 2024-09-14T06:31:04.028 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:04.028 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:04.028 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:31:04.029 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.708186+0000", 2024-09-14T06:31:04.029 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052781726000000001 2024-09-14T06:31:04.029 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:04.029 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:04.029 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:31:04.029 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.708229+0000", 2024-09-14T06:31:04.029 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.2670999999999999e-05 2024-09-14T06:31:04.029 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:04.029 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:04.029 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:31:04.029 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.708285+0000", 2024-09-14T06:31:04.029 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.5946999999999998e-05 2024-09-14T06:31:04.029 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:04.029 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:31:04.030 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:04.030 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:04.030 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:04.030 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:11462 3.1d 3:bcad7303:::rbd_data.112d92027590.0000000000000001:f [read 186368~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:31:04.030 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:25.655410+0000", 2024-09-14T06:31:04.030 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 98.324852250000006, 2024-09-14T06:31:04.030 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052995258000000003, 2024-09-14T06:31:04.030 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:31:04.030 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:31:04.030 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:31:04.030 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:31:04.030 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:31:04.030 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:04.030 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 11462 2024-09-14T06:31:04.031 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:04.031 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:31:04.031 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:04.031 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:31:04.031 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655410+0000", 2024-09-14T06:31:04.031 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:04.031 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:04.031 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:04.031 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:31:04.031 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655410+0000", 2024-09-14T06:31:04.031 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:04.031 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:04.031 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:04.031 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:31:04.032 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655625+0000", 2024-09-14T06:31:04.032 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00021442600000000001 2024-09-14T06:31:04.032 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:04.032 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:04.032 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:31:04.032 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655626+0000", 2024-09-14T06:31:04.032 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.815e-06 2024-09-14T06:31:04.032 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:04.032 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:04.032 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:31:04.032 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655631+0000", 2024-09-14T06:31:04.032 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.651e-06 2024-09-14T06:31:04.032 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:04.032 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:04.033 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:31:04.033 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655634+0000", 2024-09-14T06:31:04.033 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.2449999999999998e-06 2024-09-14T06:31:04.033 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:04.033 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:04.033 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:31:04.033 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.708312+0000", 2024-09-14T06:31:04.033 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052677412 2024-09-14T06:31:04.033 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:04.033 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:04.033 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:31:04.033 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.708352+0000", 2024-09-14T06:31:04.033 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.0729999999999998e-05 2024-09-14T06:31:04.033 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:04.033 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:04.034 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:31:04.034 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.708405+0000", 2024-09-14T06:31:04.034 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.2979000000000002e-05 2024-09-14T06:31:04.034 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:04.034 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:31:04.034 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:04.034 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:04.034 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:04.034 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:11463 3.1d 3:bcad7303:::rbd_data.112d92027590.0000000000000001:f [read 196608~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:31:04.034 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:25.655639+0000", 2024-09-14T06:31:04.034 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 98.324623103999997, 2024-09-14T06:31:04.034 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052888859000000003, 2024-09-14T06:31:04.034 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:31:04.034 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:31:04.035 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:31:04.035 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:31:04.035 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:31:04.035 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:04.035 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 11463 2024-09-14T06:31:04.035 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:04.035 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:31:04.035 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:04.035 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:31:04.035 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655639+0000", 2024-09-14T06:31:04.035 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:04.035 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:04.035 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:04.035 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:31:04.036 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655639+0000", 2024-09-14T06:31:04.036 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:04.036 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:04.036 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:04.036 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:31:04.036 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655640+0000", 2024-09-14T06:31:04.036 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.08e-07 2024-09-14T06:31:04.036 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:04.036 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:04.036 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:31:04.036 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655641+0000", 2024-09-14T06:31:04.036 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.156e-06 2024-09-14T06:31:04.036 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:04.036 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:04.037 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:31:04.037 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655648+0000", 2024-09-14T06:31:04.037 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.8460000000000002e-06 2024-09-14T06:31:04.037 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:04.037 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:04.037 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:31:04.037 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655651+0000", 2024-09-14T06:31:04.037 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.0079999999999998e-06 2024-09-14T06:31:04.037 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:04.037 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:04.037 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:31:04.037 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.708429+0000", 2024-09-14T06:31:04.037 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052778459 2024-09-14T06:31:04.038 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:04.038 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:04.038 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:31:04.038 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.708473+0000", 2024-09-14T06:31:04.038 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.3782999999999997e-05 2024-09-14T06:31:04.038 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:04.038 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:04.038 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:31:04.038 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.708528+0000", 2024-09-14T06:31:04.038 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.5198999999999998e-05 2024-09-14T06:31:04.038 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:04.038 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:31:04.038 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:04.038 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:04.039 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:04.039 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:11464 3.1d 3:bcad7303:::rbd_data.112d92027590.0000000000000001:f [read 206848~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:31:04.039 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:25.655654+0000", 2024-09-14T06:31:04.039 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 98.324608234999999, 2024-09-14T06:31:04.039 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052981845999999999, 2024-09-14T06:31:04.039 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:31:04.039 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:31:04.039 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:31:04.039 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:31:04.039 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:31:04.039 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:04.039 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 11464 2024-09-14T06:31:04.039 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:04.039 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:31:04.040 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:04.040 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:31:04.040 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655654+0000", 2024-09-14T06:31:04.040 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:04.040 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:04.040 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:04.040 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:31:04.040 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655654+0000", 2024-09-14T06:31:04.040 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:04.040 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:04.040 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:04.040 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:31:04.040 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655946+0000", 2024-09-14T06:31:04.040 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00029148200000000003 2024-09-14T06:31:04.040 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:04.041 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:04.041 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:31:04.041 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655948+0000", 2024-09-14T06:31:04.041 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.973e-06 2024-09-14T06:31:04.041 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:04.041 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:04.041 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:31:04.041 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655953+0000", 2024-09-14T06:31:04.041 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.1240000000000004e-06 2024-09-14T06:31:04.041 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:04.041 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:04.041 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:31:04.041 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655956+0000", 2024-09-14T06:31:04.041 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.315e-06 2024-09-14T06:31:04.041 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:04.042 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:04.042 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:31:04.042 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.708543+0000", 2024-09-14T06:31:04.042 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052587416999999997 2024-09-14T06:31:04.042 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:04.042 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:04.042 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:31:04.042 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.708584+0000", 2024-09-14T06:31:04.042 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.1004000000000002e-05 2024-09-14T06:31:04.042 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:04.042 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:04.042 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:31:04.042 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.708636+0000", 2024-09-14T06:31:04.042 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.1530999999999998e-05 2024-09-14T06:31:04.043 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:04.043 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:31:04.043 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:04.043 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:04.043 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:04.043 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:11465 3.1d 3:bcad7303:::rbd_data.112d92027590.0000000000000001:f [read 217088~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:31:04.043 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:25.655961+0000", 2024-09-14T06:31:04.043 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 98.324301082999995, 2024-09-14T06:31:04.043 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052798682999999999, 2024-09-14T06:31:04.043 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:31:04.043 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:31:04.043 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:31:04.043 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:31:04.043 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:31:04.043 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:04.044 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 11465 2024-09-14T06:31:04.044 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:04.044 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:31:04.044 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:04.044 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:31:04.044 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655961+0000", 2024-09-14T06:31:04.044 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:04.044 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:04.044 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:04.044 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:31:04.044 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655961+0000", 2024-09-14T06:31:04.044 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:04.044 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:04.044 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:04.045 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:31:04.045 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.656114+0000", 2024-09-14T06:31:04.045 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.000152745 2024-09-14T06:31:04.045 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:04.045 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:04.045 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:31:04.045 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.656116+0000", 2024-09-14T06:31:04.045 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.804e-06 2024-09-14T06:31:04.045 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:04.045 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:04.045 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:31:04.045 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.656120+0000", 2024-09-14T06:31:04.045 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.4700000000000004e-06 2024-09-14T06:31:04.045 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:04.046 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:04.046 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:31:04.046 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.656123+0000", 2024-09-14T06:31:04.046 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.1269999999999999e-06 2024-09-14T06:31:04.046 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:04.046 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:04.046 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:31:04.046 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.708660+0000", 2024-09-14T06:31:04.046 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052536919000000001 2024-09-14T06:31:04.046 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:04.046 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:04.046 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:31:04.046 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.708703+0000", 2024-09-14T06:31:04.046 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.3077000000000001e-05 2024-09-14T06:31:04.046 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:04.047 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:04.047 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:31:04.047 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.708760+0000", 2024-09-14T06:31:04.047 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.6541e-05 2024-09-14T06:31:04.047 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:04.047 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:31:04.047 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:04.047 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:04.047 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:04.047 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:11466 3.1d 3:bcad7303:::rbd_data.112d92027590.0000000000000001:f [read 227328~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:31:04.047 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:25.656129+0000", 2024-09-14T06:31:04.047 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 98.324133695, 2024-09-14T06:31:04.047 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052750987999999999, 2024-09-14T06:31:04.047 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:31:04.048 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:31:04.048 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:31:04.048 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:31:04.048 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:31:04.048 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:04.048 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 11466 2024-09-14T06:31:04.048 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:04.048 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:31:04.048 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:04.048 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:31:04.048 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.656129+0000", 2024-09-14T06:31:04.048 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:04.048 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:04.048 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:04.049 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:31:04.049 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.656129+0000", 2024-09-14T06:31:04.049 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:04.049 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:04.049 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:04.049 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:31:04.049 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.656347+0000", 2024-09-14T06:31:04.049 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.000218054 2024-09-14T06:31:04.049 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:04.049 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:04.049 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:31:04.049 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.656349+0000", 2024-09-14T06:31:04.049 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.9199999999999998e-06 2024-09-14T06:31:04.049 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:04.050 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:04.050 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:31:04.050 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.656353+0000", 2024-09-14T06:31:04.050 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.9210000000000002e-06 2024-09-14T06:31:04.050 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:04.050 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:04.050 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:31:04.050 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.656356+0000", 2024-09-14T06:31:04.050 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.7670000000000001e-06 2024-09-14T06:31:04.050 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:04.050 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:04.050 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:31:04.050 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.708787+0000", 2024-09-14T06:31:04.050 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052430840999999999 2024-09-14T06:31:04.051 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:04.051 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:04.051 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:31:04.051 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.708827+0000", 2024-09-14T06:31:04.051 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.9538e-05 2024-09-14T06:31:04.051 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:04.051 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:04.051 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:31:04.051 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.708880+0000", 2024-09-14T06:31:04.051 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.2947e-05 2024-09-14T06:31:04.051 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:04.051 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:31:04.051 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:04.051 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:04.052 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:04.052 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:11467 3.1d 3:bcad7303:::rbd_data.112d92027590.0000000000000001:f [read 237568~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:31:04.052 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:25.656361+0000", 2024-09-14T06:31:04.052 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 98.323901047000007, 2024-09-14T06:31:04.052 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052643898000000001, 2024-09-14T06:31:04.052 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:31:04.052 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:31:04.052 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:31:04.052 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:31:04.052 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:31:04.052 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:04.052 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 11467 2024-09-14T06:31:04.052 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:04.052 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:31:04.053 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:04.053 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:31:04.053 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.656361+0000", 2024-09-14T06:31:04.053 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:04.053 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:04.053 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:04.053 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:31:04.053 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.656361+0000", 2024-09-14T06:31:04.053 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:04.053 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:04.053 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:04.053 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:31:04.053 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.656855+0000", 2024-09-14T06:31:04.053 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00049372000000000001 2024-09-14T06:31:04.054 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:04.054 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:04.054 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:31:04.054 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.656859+0000", 2024-09-14T06:31:04.054 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.1300000000000003e-06 2024-09-14T06:31:04.054 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:04.054 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:04.054 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:31:04.054 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.656863+0000", 2024-09-14T06:31:04.054 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.2629999999999997e-06 2024-09-14T06:31:04.054 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:04.054 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:04.054 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:31:04.054 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.656867+0000", 2024-09-14T06:31:04.054 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.3220000000000001e-06 2024-09-14T06:31:04.055 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:04.055 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:04.055 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:31:04.055 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.708903+0000", 2024-09-14T06:31:04.055 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052035855999999998 2024-09-14T06:31:04.055 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:04.055 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:04.055 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:31:04.055 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.708949+0000", 2024-09-14T06:31:04.055 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.6276999999999998e-05 2024-09-14T06:31:04.055 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:04.055 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:04.055 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:31:04.055 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.709005+0000", 2024-09-14T06:31:04.056 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.6329999999999998e-05 2024-09-14T06:31:04.056 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:04.056 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:31:04.056 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:04.056 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:04.056 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:04.056 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:11479 3.1d 3:bcad7303:::rbd_data.112d92027590.0000000000000001:f [read 360448~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:31:04.056 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:25.658388+0000", 2024-09-14T06:31:04.056 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 98.321874214000005, 2024-09-14T06:31:04.056 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052408920999999997, 2024-09-14T06:31:04.056 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:31:04.056 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:31:04.056 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:31:04.056 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:31:04.057 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:31:04.057 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:04.057 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 11479 2024-09-14T06:31:04.057 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:04.057 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:31:04.057 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:04.057 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:31:04.057 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.658388+0000", 2024-09-14T06:31:04.057 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:04.057 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:04.057 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:04.057 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:31:04.057 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.658388+0000", 2024-09-14T06:31:04.057 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:04.058 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:04.058 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:04.058 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:31:04.058 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.658534+0000", 2024-09-14T06:31:04.058 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.000145613 2024-09-14T06:31:04.058 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:04.058 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:04.058 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:31:04.058 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.658540+0000", 2024-09-14T06:31:04.058 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.6680000000000004e-06 2024-09-14T06:31:04.058 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:04.058 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:04.058 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:31:04.058 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.658548+0000", 2024-09-14T06:31:04.059 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.7980000000000006e-06 2024-09-14T06:31:04.059 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:04.059 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:04.059 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:31:04.059 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.658554+0000", 2024-09-14T06:31:04.059 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.6869999999999999e-06 2024-09-14T06:31:04.059 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:04.059 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:04.059 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:31:04.059 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.710624+0000", 2024-09-14T06:31:04.059 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052070567999999998 2024-09-14T06:31:04.059 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:04.059 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:04.059 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:31:04.060 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.710666+0000", 2024-09-14T06:31:04.060 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.1467000000000001e-05 2024-09-14T06:31:04.060 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:04.060 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:04.060 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:31:04.060 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.710797+0000", 2024-09-14T06:31:04.060 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00013112 2024-09-14T06:31:04.060 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:04.060 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:31:04.061 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:04.061 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:04.061 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:04.061 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:11480 3.1d 3:bcad7303:::rbd_data.112d92027590.0000000000000001:f [read 370688~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:31:04.061 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:25.658563+0000", 2024-09-14T06:31:04.061 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 98.321699283000001, 2024-09-14T06:31:04.061 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052429164, 2024-09-14T06:31:04.061 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:31:04.061 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:31:04.061 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:31:04.061 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:31:04.061 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:31:04.061 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:04.061 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 11480 2024-09-14T06:31:04.062 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:04.062 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:31:04.062 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:04.062 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:31:04.062 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.658563+0000", 2024-09-14T06:31:04.062 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:04.062 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:04.062 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:04.062 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:31:04.062 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.658563+0000", 2024-09-14T06:31:04.062 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:04.062 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:04.062 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:04.062 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:31:04.063 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.658658+0000", 2024-09-14T06:31:04.063 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.5346999999999994e-05 2024-09-14T06:31:04.063 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:04.063 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:04.063 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:31:04.063 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.658662+0000", 2024-09-14T06:31:04.063 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.14e-06 2024-09-14T06:31:04.063 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:04.063 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:04.063 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:31:04.063 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.658669+0000", 2024-09-14T06:31:04.063 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.7379999999999994e-06 2024-09-14T06:31:04.063 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:04.064 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:04.064 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:31:04.064 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.658675+0000", 2024-09-14T06:31:04.064 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.5430000000000001e-06 2024-09-14T06:31:04.064 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:04.064 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:04.064 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:31:04.064 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.710824+0000", 2024-09-14T06:31:04.064 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052148847999999998 2024-09-14T06:31:04.064 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:04.064 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:04.064 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:31:04.064 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.710864+0000", 2024-09-14T06:31:04.065 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.9925999999999997e-05 2024-09-14T06:31:04.065 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:04.065 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:04.065 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:31:04.065 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.710992+0000", 2024-09-14T06:31:04.065 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.000128622 2024-09-14T06:31:04.065 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:04.065 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:31:04.065 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:04.065 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:04.065 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:04.065 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:11481 3.1d 3:bcad7303:::rbd_data.112d92027590.0000000000000001:f [read 380928~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:31:04.065 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:25.658684+0000", 2024-09-14T06:31:04.065 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 98.321578212999995, 2024-09-14T06:31:04.066 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052508565, 2024-09-14T06:31:04.066 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:31:04.066 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:31:04.066 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:31:04.066 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:31:04.066 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:31:04.066 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:04.066 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 11481 2024-09-14T06:31:04.066 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:04.066 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:31:04.066 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:04.066 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:31:04.066 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.658684+0000", 2024-09-14T06:31:04.066 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:04.067 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:04.067 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:04.067 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:31:04.067 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.658684+0000", 2024-09-14T06:31:04.067 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:04.067 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:04.067 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:04.067 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:31:04.067 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.658715+0000", 2024-09-14T06:31:04.067 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.0703000000000001e-05 2024-09-14T06:31:04.067 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:04.067 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:04.067 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:31:04.067 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.658718+0000", 2024-09-14T06:31:04.067 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.0709999999999999e-06 2024-09-14T06:31:04.068 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:04.068 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:04.068 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:31:04.068 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.658725+0000", 2024-09-14T06:31:04.068 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.2339999999999997e-06 2024-09-14T06:31:04.068 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:04.068 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:04.068 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:31:04.068 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.658731+0000", 2024-09-14T06:31:04.068 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.6450000000000002e-06 2024-09-14T06:31:04.068 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:04.068 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:04.068 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:31:04.068 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.711020+0000", 2024-09-14T06:31:04.068 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052288886 2024-09-14T06:31:04.069 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:04.069 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:04.069 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:31:04.069 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.711061+0000", 2024-09-14T06:31:04.069 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.1084000000000002e-05 2024-09-14T06:31:04.069 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:04.069 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:04.069 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:31:04.069 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.711193+0000", 2024-09-14T06:31:04.069 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00013194199999999999 2024-09-14T06:31:04.069 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:04.069 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:31:04.069 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:04.069 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:04.070 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:04.070 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:11482 3.1d 3:bcad7303:::rbd_data.112d92027590.0000000000000001:f [read 391168~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:31:04.070 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:25.658740+0000", 2024-09-14T06:31:04.070 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 98.321522748000007, 2024-09-14T06:31:04.070 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052637015000000002, 2024-09-14T06:31:04.070 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:31:04.070 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:31:04.070 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:31:04.070 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:31:04.070 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:31:04.070 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:04.070 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 11482 2024-09-14T06:31:04.070 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:04.070 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:31:04.070 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:04.071 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:31:04.071 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.658740+0000", 2024-09-14T06:31:04.071 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:04.071 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:04.071 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:04.071 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:31:04.071 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.658740+0000", 2024-09-14T06:31:04.071 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:04.071 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:04.071 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:04.071 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:31:04.071 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.658880+0000", 2024-09-14T06:31:04.071 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00014036100000000001 2024-09-14T06:31:04.071 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:04.071 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:04.072 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:31:04.072 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.658883+0000", 2024-09-14T06:31:04.072 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.3969999999999998e-06 2024-09-14T06:31:04.072 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:04.072 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:04.072 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:31:04.072 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.658890+0000", 2024-09-14T06:31:04.072 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.1500000000000002e-06 2024-09-14T06:31:04.072 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:04.072 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:04.072 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:31:04.072 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.658897+0000", 2024-09-14T06:31:04.072 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.1260000000000003e-06 2024-09-14T06:31:04.073 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:04.073 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:04.073 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:31:04.073 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.711207+0000", 2024-09-14T06:31:04.073 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052310777000000003 2024-09-14T06:31:04.073 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:04.073 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:04.073 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:31:04.073 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.711249+0000", 2024-09-14T06:31:04.073 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.1786999999999998e-05 2024-09-14T06:31:04.073 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:04.073 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:04.073 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:31:04.073 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.711377+0000", 2024-09-14T06:31:04.074 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.000127417 2024-09-14T06:31:04.074 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:04.074 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:31:04.074 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:04.074 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:04.074 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:04.074 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:11483 3.1d 3:bcad7303:::rbd_data.112d92027590.0000000000000001:f [read 401408~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:31:04.074 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:25.658906+0000", 2024-09-14T06:31:04.074 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 98.321355964000006, 2024-09-14T06:31:04.074 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052595109000000001, 2024-09-14T06:31:04.074 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:31:04.074 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:31:04.074 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:31:04.074 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:31:04.074 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:31:04.075 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:04.075 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 11483 2024-09-14T06:31:04.075 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:04.075 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:31:04.075 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:04.075 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:31:04.075 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.658906+0000", 2024-09-14T06:31:04.075 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:04.075 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:04.075 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:04.075 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:31:04.075 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.658906+0000", 2024-09-14T06:31:04.075 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:04.075 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:04.075 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:04.076 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:31:04.076 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.664985+0000", 2024-09-14T06:31:04.076 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.0060787250000000001 2024-09-14T06:31:04.076 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:04.076 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:04.076 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:31:04.076 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.664991+0000", 2024-09-14T06:31:04.076 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.7030000000000003e-06 2024-09-14T06:31:04.076 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:04.076 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:04.076 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:31:04.076 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.664999+0000", 2024-09-14T06:31:04.076 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.4430000000000003e-06 2024-09-14T06:31:04.076 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:04.077 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:04.077 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:31:04.077 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.665006+0000", 2024-09-14T06:31:04.077 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.0840000000000003e-06 2024-09-14T06:31:04.077 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:04.077 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:04.077 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:31:04.077 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.711402+0000", 2024-09-14T06:31:04.077 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046395714999999997 2024-09-14T06:31:04.077 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:04.077 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:04.077 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:31:04.077 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.711443+0000", 2024-09-14T06:31:04.077 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.0911999999999997e-05 2024-09-14T06:31:04.077 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:04.078 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:04.078 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:31:04.078 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.711501+0000", 2024-09-14T06:31:04.078 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.8526999999999997e-05 2024-09-14T06:31:04.078 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:04.078 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:31:04.078 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:04.078 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:04.078 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:31:04.078 INFO:teuthology.orchestra.run.smithi138.stdout:} 2024-09-14T06:31:04.080 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-14T06:31:04.081 INFO:tasks.ceph.osd.6.smithi138.stderr:2024-09-14T06:31:04.078+0000 7f8cd8849640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33620) UID: 0 2024-09-14T06:31:04.081 INFO:teuthology.orchestra.run.smithi094.stdout:4314 write 0x111509e thru 0x1117989 (0x28ec bytes) 2024-09-14T06:31:04.081 INFO:teuthology.orchestra.run.smithi094.stdout:4316 writesame 0xb9b2f4d thru 0xb9bd6cc data_size 0x5(0xa780 bytes) 2024-09-14T06:31:04.081 INFO:teuthology.orchestra.run.smithi094.stdout:4317 read 0x3b948b5 thru 0x3b9dd88 (0x94d4 bytes) 2024-09-14T06:31:04.081 INFO:teuthology.orchestra.run.smithi094.stdout:4318 trunc from 0xb9bd6cd to 0x5fd8fc9 2024-09-14T06:31:04.081 INFO:teuthology.orchestra.run.smithi094.stdout:4319 write 0x6a6c4fe thru 0x6a705c6 (0x40c9 bytes) 2024-09-14T06:31:04.081 INFO:teuthology.orchestra.run.smithi094.stdout:4320 punch from 0x1b655e7 to 0x1b75390, (0xfda9 bytes) 2024-09-14T06:31:04.081 INFO:teuthology.orchestra.run.smithi094.stdout:4321 punch from 0x1d42136 to 0x1d441b4, (0x207e bytes) 2024-09-14T06:31:04.081 INFO:teuthology.orchestra.run.smithi094.stdout:4323 clone 32 order 25 su 262144 sc 9 2024-09-14T06:31:04.180 INFO:tasks.ceph.osd.2.smithi094.stderr:2024-09-14T06:31:04.179+0000 7faf3269b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33649) UID: 0 2024-09-14T06:31:04.225 INFO:teuthology.orchestra.run.smithi138.stdout:{ 2024-09-14T06:31:04.225 INFO:teuthology.orchestra.run.smithi138.stdout: "ops": [], 2024-09-14T06:31:04.225 INFO:teuthology.orchestra.run.smithi138.stdout: "num_ops": 0 2024-09-14T06:31:04.225 INFO:teuthology.orchestra.run.smithi138.stdout:} 2024-09-14T06:31:04.233 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-14T06:31:04.279 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-14T06:31:04.280 INFO:tasks.ceph.osd.5.smithi138.stderr:2024-09-14T06:31:04.279+0000 7fe13746f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33618) UID: 0 2024-09-14T06:31:04.336 INFO:teuthology.orchestra.run.smithi138.stdout:{ 2024-09-14T06:31:04.336 INFO:teuthology.orchestra.run.smithi138.stdout: "ops": [], 2024-09-14T06:31:04.336 INFO:teuthology.orchestra.run.smithi138.stdout: "complaint_time": 30, 2024-09-14T06:31:04.336 INFO:teuthology.orchestra.run.smithi138.stdout: "num_blocked_ops": 0 2024-09-14T06:31:04.337 INFO:teuthology.orchestra.run.smithi138.stdout:} 2024-09-14T06:31:04.345 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-14T06:31:04.381 INFO:tasks.ceph.osd.0.smithi094.stderr:2024-09-14T06:31:04.380+0000 7f07c9333640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33636) UID: 0 2024-09-14T06:31:04.399 INFO:teuthology.orchestra.run.smithi094.stdout:osd.0: {} 2024-09-14T06:31:04.400 INFO:teuthology.orchestra.run.smithi094.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-14T06:31:04.416 INFO:teuthology.orchestra.run.smithi094.stdout:osd.1: {} 2024-09-14T06:31:04.417 INFO:teuthology.orchestra.run.smithi094.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-14T06:31:04.432 INFO:teuthology.orchestra.run.smithi094.stdout:osd.2: {} 2024-09-14T06:31:04.432 INFO:teuthology.orchestra.run.smithi094.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-14T06:31:04.448 INFO:teuthology.orchestra.run.smithi094.stdout:osd.3: {} 2024-09-14T06:31:04.448 INFO:teuthology.orchestra.run.smithi094.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-14T06:31:04.464 INFO:teuthology.orchestra.run.smithi094.stdout:osd.4: {} 2024-09-14T06:31:04.464 INFO:teuthology.orchestra.run.smithi094.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-14T06:31:04.480 INFO:teuthology.orchestra.run.smithi094.stdout:osd.5: {} 2024-09-14T06:31:04.481 INFO:teuthology.orchestra.run.smithi094.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-14T06:31:04.482 INFO:tasks.ceph.osd.6.smithi138.stderr:2024-09-14T06:31:04.481+0000 7f8cd8849640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33620) UID: 0 2024-09-14T06:31:04.491 INFO:teuthology.orchestra.run.smithi138.stdout:{ 2024-09-14T06:31:04.491 INFO:teuthology.orchestra.run.smithi138.stdout: "size": 20, 2024-09-14T06:31:04.491 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 600, 2024-09-14T06:31:04.491 INFO:teuthology.orchestra.run.smithi138.stdout: "ops": [ 2024-09-14T06:31:04.491 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:04.491 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:98505 3.10 3.1bf56f90 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T06:31:04.491 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:30:07.045453+0000", 2024-09-14T06:31:04.491 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 57.444013403, 2024-09-14T06:31:04.491 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032144440000000003, 2024-09-14T06:31:04.491 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:31:04.492 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:31:04.492 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "reached pg", 2024-09-14T06:31:04.492 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:31:04.492 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:31:04.492 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:04.492 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 98505 2024-09-14T06:31:04.492 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:04.492 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:31:04.492 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:04.492 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:31:04.492 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045453+0000", 2024-09-14T06:31:04.492 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:04.492 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:04.493 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:04.493 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:31:04.493 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045453+0000", 2024-09-14T06:31:04.493 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:04.493 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:04.493 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:04.493 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:31:04.493 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045453+0000", 2024-09-14T06:31:04.493 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.36e-07 2024-09-14T06:31:04.493 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:04.493 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:04.493 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:31:04.493 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045454+0000", 2024-09-14T06:31:04.493 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.1299999999999998e-07 2024-09-14T06:31:04.494 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:04.494 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:04.494 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:31:04.494 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045457+0000", 2024-09-14T06:31:04.494 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.8559999999999998e-06 2024-09-14T06:31:04.494 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:04.494 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:04.494 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:31:04.494 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045460+0000", 2024-09-14T06:31:04.494 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.8750000000000002e-06 2024-09-14T06:31:04.494 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:04.494 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:04.494 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:31:04.494 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.077592+0000", 2024-09-14T06:31:04.495 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032131727999999998 2024-09-14T06:31:04.495 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:04.495 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:04.495 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:31:04.495 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.077597+0000", 2024-09-14T06:31:04.495 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.8319999999999996e-06 2024-09-14T06:31:04.495 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:04.495 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:31:04.495 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:04.495 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:04.495 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:04.495 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:98506 3.10 3.1bf56f90 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T06:31:04.495 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:30:07.045481+0000", 2024-09-14T06:31:04.495 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 57.443985148000003, 2024-09-14T06:31:04.496 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032142759999999999, 2024-09-14T06:31:04.496 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:31:04.496 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:31:04.496 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "reached pg", 2024-09-14T06:31:04.496 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:31:04.496 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:31:04.496 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:04.496 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 98506 2024-09-14T06:31:04.496 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:04.496 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:31:04.496 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:04.496 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:31:04.496 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045481+0000", 2024-09-14T06:31:04.496 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:04.497 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:04.497 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:04.497 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:31:04.497 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045481+0000", 2024-09-14T06:31:04.497 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:04.497 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:04.497 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:04.497 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:31:04.497 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045481+0000", 2024-09-14T06:31:04.497 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.8500000000000001e-07 2024-09-14T06:31:04.497 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:04.497 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:04.497 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:31:04.498 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045483+0000", 2024-09-14T06:31:04.498 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.8250000000000001e-06 2024-09-14T06:31:04.498 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:04.498 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:04.498 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:31:04.498 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045489+0000", 2024-09-14T06:31:04.498 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.5740000000000003e-06 2024-09-14T06:31:04.498 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:04.498 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:04.498 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:31:04.498 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045495+0000", 2024-09-14T06:31:04.498 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.7690000000000002e-06 2024-09-14T06:31:04.498 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:04.498 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:04.498 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:31:04.499 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.077619+0000", 2024-09-14T06:31:04.499 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032124621999999999 2024-09-14T06:31:04.499 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:04.499 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:04.499 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:31:04.499 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.077624+0000", 2024-09-14T06:31:04.499 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.3850000000000004e-06 2024-09-14T06:31:04.499 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:04.499 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:31:04.499 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:04.499 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:04.499 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:04.499 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:98507 3.10 3.1bf56f90 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T06:31:04.500 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:30:07.045502+0000", 2024-09-14T06:31:04.500 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 57.443963727000003, 2024-09-14T06:31:04.500 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032147219999999997, 2024-09-14T06:31:04.500 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:31:04.500 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:31:04.500 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "reached pg", 2024-09-14T06:31:04.500 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:31:04.500 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:31:04.500 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:04.500 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 98507 2024-09-14T06:31:04.500 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:04.500 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:31:04.500 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:04.500 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:31:04.501 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045502+0000", 2024-09-14T06:31:04.501 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:04.501 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:04.501 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:04.501 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:31:04.501 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045502+0000", 2024-09-14T06:31:04.501 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:04.501 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:04.501 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:04.501 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:31:04.501 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045503+0000", 2024-09-14T06:31:04.501 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.4700000000000002e-07 2024-09-14T06:31:04.501 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:04.502 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:04.502 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:31:04.502 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045509+0000", 2024-09-14T06:31:04.502 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.7329999999999997e-06 2024-09-14T06:31:04.502 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:04.502 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:04.502 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:31:04.502 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045515+0000", 2024-09-14T06:31:04.502 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.8030000000000002e-06 2024-09-14T06:31:04.502 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:04.502 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:04.502 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:31:04.502 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045522+0000", 2024-09-14T06:31:04.502 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.0369999999999997e-06 2024-09-14T06:31:04.502 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:04.503 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:04.503 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:31:04.503 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.077641+0000", 2024-09-14T06:31:04.503 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032119091000000002 2024-09-14T06:31:04.503 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:04.503 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:04.503 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:31:04.503 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.077650+0000", 2024-09-14T06:31:04.503 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.2090000000000005e-06 2024-09-14T06:31:04.503 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:04.503 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:31:04.503 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:04.503 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:04.503 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:04.504 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:98508 3.10 3.1bf56f90 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T06:31:04.504 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:30:07.045530+0000", 2024-09-14T06:31:04.504 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 57.443936110000003, 2024-09-14T06:31:04.504 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032198773999999999, 2024-09-14T06:31:04.504 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:31:04.504 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:31:04.504 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "reached pg", 2024-09-14T06:31:04.504 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:31:04.504 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:31:04.505 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:04.505 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 98508 2024-09-14T06:31:04.505 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:04.505 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:31:04.505 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:04.505 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:31:04.505 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045530+0000", 2024-09-14T06:31:04.505 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:04.505 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:04.505 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:04.505 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:31:04.505 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045530+0000", 2024-09-14T06:31:04.505 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:04.505 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:04.506 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:04.506 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:31:04.506 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045530+0000", 2024-09-14T06:31:04.506 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.3099999999999998e-07 2024-09-14T06:31:04.506 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:04.506 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:04.506 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:31:04.506 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045539+0000", 2024-09-14T06:31:04.506 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.4780000000000008e-06 2024-09-14T06:31:04.506 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:04.506 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:04.506 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:31:04.506 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045546+0000", 2024-09-14T06:31:04.507 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.3259999999999998e-06 2024-09-14T06:31:04.507 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:04.507 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:04.507 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:31:04.507 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045551+0000", 2024-09-14T06:31:04.507 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.9960000000000001e-06 2024-09-14T06:31:04.507 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:04.507 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:04.507 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:31:04.507 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.077716+0000", 2024-09-14T06:31:04.507 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032165081999999998 2024-09-14T06:31:04.507 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:04.507 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:04.507 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:31:04.508 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.077729+0000", 2024-09-14T06:31:04.508 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.2461e-05 2024-09-14T06:31:04.508 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:04.508 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:31:04.508 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:04.508 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:04.508 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:04.508 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:98509 3.10 3.1bf56f90 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T06:31:04.508 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:30:07.045558+0000", 2024-09-14T06:31:04.508 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 57.443907909000004, 2024-09-14T06:31:04.508 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032242198, 2024-09-14T06:31:04.508 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:31:04.508 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:31:04.508 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "reached pg", 2024-09-14T06:31:04.509 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:31:04.509 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:31:04.509 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:04.509 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 98509 2024-09-14T06:31:04.509 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:04.509 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:31:04.509 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:04.509 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:31:04.509 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045558+0000", 2024-09-14T06:31:04.509 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:04.509 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:04.509 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:04.509 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:31:04.509 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045558+0000", 2024-09-14T06:31:04.509 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:04.510 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:04.510 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:04.510 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:31:04.510 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045559+0000", 2024-09-14T06:31:04.510 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.27e-07 2024-09-14T06:31:04.510 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:04.510 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:04.510 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:31:04.510 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045560+0000", 2024-09-14T06:31:04.510 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.6959999999999999e-06 2024-09-14T06:31:04.510 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:04.510 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:04.510 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:31:04.510 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045567+0000", 2024-09-14T06:31:04.511 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.6900000000000003e-06 2024-09-14T06:31:04.511 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:04.511 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:04.511 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:31:04.511 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045571+0000", 2024-09-14T06:31:04.511 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.5750000000000002e-06 2024-09-14T06:31:04.511 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:04.511 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:04.511 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:31:04.511 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.077789+0000", 2024-09-14T06:31:04.511 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032217186000000002 2024-09-14T06:31:04.511 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:04.511 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:04.511 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:31:04.511 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.077800+0000", 2024-09-14T06:31:04.512 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.1623999999999999e-05 2024-09-14T06:31:04.512 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:04.512 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:31:04.512 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:04.512 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:04.512 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:04.512 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:98510 3.10 3.1bf56f90 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T06:31:04.512 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:30:07.045580+0000", 2024-09-14T06:31:04.512 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 57.443886095000003, 2024-09-14T06:31:04.512 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032299845000000001, 2024-09-14T06:31:04.512 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:31:04.512 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:31:04.512 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "reached pg", 2024-09-14T06:31:04.513 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:31:04.513 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:31:04.513 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:04.513 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 98510 2024-09-14T06:31:04.513 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:04.513 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:31:04.513 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:04.513 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:31:04.513 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045580+0000", 2024-09-14T06:31:04.513 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:04.513 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:04.513 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:04.513 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:31:04.513 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045580+0000", 2024-09-14T06:31:04.514 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:04.514 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:04.514 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:04.514 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:31:04.514 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045580+0000", 2024-09-14T06:31:04.514 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.0400000000000002e-07 2024-09-14T06:31:04.514 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:04.514 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:04.514 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:31:04.514 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045582+0000", 2024-09-14T06:31:04.514 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.7600000000000001e-06 2024-09-14T06:31:04.514 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:04.514 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:04.514 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:31:04.515 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045588+0000", 2024-09-14T06:31:04.515 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.5650000000000001e-06 2024-09-14T06:31:04.515 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:04.515 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:04.515 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:31:04.515 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045597+0000", 2024-09-14T06:31:04.515 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.465e-06 2024-09-14T06:31:04.515 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:04.515 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:04.515 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:31:04.515 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.077871+0000", 2024-09-14T06:31:04.515 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032273562999999998 2024-09-14T06:31:04.515 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:04.515 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:04.515 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:31:04.516 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.077880+0000", 2024-09-14T06:31:04.516 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.0880000000000001e-06 2024-09-14T06:31:04.516 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:04.516 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:31:04.516 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:04.516 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:04.516 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:04.516 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:98511 3.10 3.1bf56f90 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T06:31:04.516 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:30:07.045610+0000", 2024-09-14T06:31:04.516 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 57.443856248000003, 2024-09-14T06:31:04.516 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032338894, 2024-09-14T06:31:04.516 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:31:04.516 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:31:04.516 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "reached pg", 2024-09-14T06:31:04.517 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:31:04.517 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:31:04.517 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:04.517 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 98511 2024-09-14T06:31:04.517 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:04.517 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:31:04.517 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:04.517 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:31:04.517 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045610+0000", 2024-09-14T06:31:04.517 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:04.517 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:04.517 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:04.517 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:31:04.517 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045610+0000", 2024-09-14T06:31:04.518 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:04.518 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:04.518 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:04.518 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:31:04.518 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045610+0000", 2024-09-14T06:31:04.518 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.1600000000000002e-07 2024-09-14T06:31:04.518 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:04.518 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:04.518 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:31:04.518 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045612+0000", 2024-09-14T06:31:04.518 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.1150000000000001e-06 2024-09-14T06:31:04.518 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:04.518 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:04.518 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:31:04.518 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045618+0000", 2024-09-14T06:31:04.519 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.0499999999999997e-06 2024-09-14T06:31:04.519 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:04.519 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:04.519 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:31:04.519 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045623+0000", 2024-09-14T06:31:04.519 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.6770000000000003e-06 2024-09-14T06:31:04.519 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:04.519 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:04.519 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:31:04.519 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.077928+0000", 2024-09-14T06:31:04.519 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032305111999999997 2024-09-14T06:31:04.519 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:04.519 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:04.519 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:31:04.519 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.077949+0000", 2024-09-14T06:31:04.520 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.0523999999999999e-05 2024-09-14T06:31:04.520 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:04.520 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:31:04.520 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:04.520 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:04.520 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:04.520 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:98512 3.10 3.1bf56f90 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T06:31:04.520 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:30:07.045630+0000", 2024-09-14T06:31:04.520 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 57.443836243, 2024-09-14T06:31:04.520 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032346632, 2024-09-14T06:31:04.520 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:31:04.520 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:31:04.520 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "reached pg", 2024-09-14T06:31:04.521 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:31:04.521 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:31:04.521 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:04.521 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 98512 2024-09-14T06:31:04.521 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:04.521 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:31:04.521 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:04.521 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:31:04.521 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045630+0000", 2024-09-14T06:31:04.521 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:04.521 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:04.521 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:04.521 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:31:04.521 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045630+0000", 2024-09-14T06:31:04.522 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:04.522 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:04.522 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:04.522 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:31:04.522 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045630+0000", 2024-09-14T06:31:04.522 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.8200000000000001e-07 2024-09-14T06:31:04.522 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:04.522 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:04.522 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:31:04.522 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045632+0000", 2024-09-14T06:31:04.522 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.7430000000000001e-06 2024-09-14T06:31:04.522 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:04.522 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:04.522 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:31:04.522 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045637+0000", 2024-09-14T06:31:04.523 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.4550000000000003e-06 2024-09-14T06:31:04.523 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:04.523 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:04.523 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:31:04.523 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045644+0000", 2024-09-14T06:31:04.523 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.1330000000000001e-06 2024-09-14T06:31:04.523 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:04.523 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:04.523 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:31:04.523 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.077968+0000", 2024-09-14T06:31:04.523 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032323401000000002 2024-09-14T06:31:04.523 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:04.523 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:04.524 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:31:04.524 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.077976+0000", 2024-09-14T06:31:04.524 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.5180000000000004e-06 2024-09-14T06:31:04.524 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:04.524 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:31:04.524 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:04.524 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:04.524 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:04.524 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:98513 3.10 3.1bf56f90 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T06:31:04.524 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:30:07.045652+0000", 2024-09-14T06:31:04.524 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 57.443813679000002, 2024-09-14T06:31:04.524 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032353436999999999, 2024-09-14T06:31:04.524 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:31:04.524 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:31:04.525 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "reached pg", 2024-09-14T06:31:04.525 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:31:04.525 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:31:04.525 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:04.525 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 98513 2024-09-14T06:31:04.525 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:04.525 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:31:04.525 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:04.525 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:31:04.525 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045652+0000", 2024-09-14T06:31:04.525 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:04.525 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:04.525 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:04.525 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:31:04.526 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045652+0000", 2024-09-14T06:31:04.526 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:04.526 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:04.526 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:04.526 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:31:04.526 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045653+0000", 2024-09-14T06:31:04.526 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.3700000000000001e-07 2024-09-14T06:31:04.526 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:04.526 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:04.526 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:31:04.526 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045655+0000", 2024-09-14T06:31:04.526 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.854e-06 2024-09-14T06:31:04.526 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:04.526 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:04.526 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:31:04.527 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045661+0000", 2024-09-14T06:31:04.527 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.5509999999999996e-06 2024-09-14T06:31:04.527 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:04.527 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:04.527 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:31:04.527 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045668+0000", 2024-09-14T06:31:04.527 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.1980000000000004e-06 2024-09-14T06:31:04.527 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:04.527 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:04.527 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:31:04.527 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.077997+0000", 2024-09-14T06:31:04.527 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032328575999999998 2024-09-14T06:31:04.527 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:04.527 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:04.528 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:31:04.528 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.078006+0000", 2024-09-14T06:31:04.528 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.8210000000000008e-06 2024-09-14T06:31:04.528 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:04.528 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:31:04.528 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:04.528 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:04.528 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:04.528 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:98514 3.10 3.1bf56f90 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T06:31:04.528 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:30:07.045699+0000", 2024-09-14T06:31:04.528 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 57.443767344000001, 2024-09-14T06:31:04.528 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032330223999999998, 2024-09-14T06:31:04.528 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:31:04.528 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:31:04.529 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "reached pg", 2024-09-14T06:31:04.529 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:31:04.529 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:31:04.529 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:04.529 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 98514 2024-09-14T06:31:04.529 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:04.529 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:31:04.529 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:04.529 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:31:04.529 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045699+0000", 2024-09-14T06:31:04.529 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:04.529 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:04.529 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:04.529 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:31:04.529 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045699+0000", 2024-09-14T06:31:04.530 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:04.530 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:04.530 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:04.530 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:31:04.530 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045699+0000", 2024-09-14T06:31:04.530 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.3200000000000001e-07 2024-09-14T06:31:04.530 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:04.530 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:04.530 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:31:04.530 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045701+0000", 2024-09-14T06:31:04.530 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.762e-06 2024-09-14T06:31:04.530 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:04.530 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:04.530 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:31:04.531 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045705+0000", 2024-09-14T06:31:04.531 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.2359999999999998e-06 2024-09-14T06:31:04.531 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:04.531 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:04.531 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:31:04.531 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045708+0000", 2024-09-14T06:31:04.531 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.796e-06 2024-09-14T06:31:04.531 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:04.531 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:04.531 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:31:04.531 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.078023+0000", 2024-09-14T06:31:04.531 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032315213000000002 2024-09-14T06:31:04.531 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:04.531 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:04.532 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:31:04.532 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.078029+0000", 2024-09-14T06:31:04.532 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.8850000000000004e-06 2024-09-14T06:31:04.532 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:04.532 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:31:04.532 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:04.532 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:04.532 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:04.532 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:98515 3.10 3.1bf56f90 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T06:31:04.532 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:30:07.045712+0000", 2024-09-14T06:31:04.532 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 57.443754032999998, 2024-09-14T06:31:04.532 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032346871999999999, 2024-09-14T06:31:04.532 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:31:04.532 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:31:04.532 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "reached pg", 2024-09-14T06:31:04.533 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:31:04.533 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:31:04.533 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:04.533 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 98515 2024-09-14T06:31:04.533 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:04.533 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:31:04.533 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:04.533 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:31:04.533 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045712+0000", 2024-09-14T06:31:04.533 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:04.533 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:04.533 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:04.533 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:31:04.533 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045712+0000", 2024-09-14T06:31:04.534 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:04.534 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:04.534 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:04.534 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:31:04.534 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045712+0000", 2024-09-14T06:31:04.534 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.2100000000000001e-07 2024-09-14T06:31:04.534 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:04.534 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:04.534 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:31:04.534 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045713+0000", 2024-09-14T06:31:04.534 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.068e-06 2024-09-14T06:31:04.534 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:04.534 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:04.534 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:31:04.535 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045716+0000", 2024-09-14T06:31:04.535 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.971e-06 2024-09-14T06:31:04.535 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:04.535 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:04.535 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:31:04.535 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045719+0000", 2024-09-14T06:31:04.535 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.3879999999999998e-06 2024-09-14T06:31:04.535 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:04.535 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:04.535 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:31:04.535 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.078049+0000", 2024-09-14T06:31:04.535 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032330553999999997 2024-09-14T06:31:04.535 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:04.535 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:04.536 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:31:04.536 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.078059+0000", 2024-09-14T06:31:04.536 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.6700000000000006e-06 2024-09-14T06:31:04.536 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:04.536 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:31:04.536 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:04.536 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:04.536 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:04.536 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:98516 3.10 3.1bf56f90 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T06:31:04.536 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:30:07.045722+0000", 2024-09-14T06:31:04.536 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 57.443744279999997, 2024-09-14T06:31:04.536 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032374334999999997, 2024-09-14T06:31:04.536 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:31:04.536 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:31:04.537 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "reached pg", 2024-09-14T06:31:04.537 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:31:04.537 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:31:04.537 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:04.537 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 98516 2024-09-14T06:31:04.537 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:04.537 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:31:04.537 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:04.537 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:31:04.537 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045722+0000", 2024-09-14T06:31:04.537 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:04.537 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:04.537 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:04.537 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:31:04.538 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045722+0000", 2024-09-14T06:31:04.538 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:04.538 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:04.538 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:04.538 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:31:04.538 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045722+0000", 2024-09-14T06:31:04.538 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.8999999999999998e-07 2024-09-14T06:31:04.538 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:04.538 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:04.538 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:31:04.538 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045723+0000", 2024-09-14T06:31:04.538 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.0049999999999999e-06 2024-09-14T06:31:04.538 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:04.539 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:04.539 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:31:04.539 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045726+0000", 2024-09-14T06:31:04.539 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.9019999999999999e-06 2024-09-14T06:31:04.539 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:04.539 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:04.539 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:31:04.539 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045728+0000", 2024-09-14T06:31:04.539 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.3290000000000001e-06 2024-09-14T06:31:04.539 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:04.539 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:04.539 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:31:04.539 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.078090+0000", 2024-09-14T06:31:04.539 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032362123 2024-09-14T06:31:04.540 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:04.540 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:04.540 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:31:04.540 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.078096+0000", 2024-09-14T06:31:04.540 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.6860000000000003e-06 2024-09-14T06:31:04.540 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:04.540 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:31:04.540 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:04.540 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:04.540 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:04.540 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:98517 3.10 3.1bf56f90 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T06:31:04.540 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:30:07.045731+0000", 2024-09-14T06:31:04.540 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 57.443734548999998, 2024-09-14T06:31:04.540 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.03241227, 2024-09-14T06:31:04.541 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:31:04.541 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:31:04.541 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "reached pg", 2024-09-14T06:31:04.541 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:31:04.541 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:31:04.541 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:04.541 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 98517 2024-09-14T06:31:04.541 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:04.541 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:31:04.541 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:04.541 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:31:04.541 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045731+0000", 2024-09-14T06:31:04.541 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:04.541 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:04.542 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:04.542 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:31:04.542 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045731+0000", 2024-09-14T06:31:04.542 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:04.542 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:04.542 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:04.542 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:31:04.542 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045732+0000", 2024-09-14T06:31:04.542 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.0699999999999999e-07 2024-09-14T06:31:04.542 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:04.542 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:04.542 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:31:04.542 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045733+0000", 2024-09-14T06:31:04.542 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.9199999999999999e-07 2024-09-14T06:31:04.542 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:04.543 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:04.543 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:31:04.543 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045735+0000", 2024-09-14T06:31:04.543 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.8150000000000002e-06 2024-09-14T06:31:04.543 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:04.543 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:04.543 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:31:04.543 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045738+0000", 2024-09-14T06:31:04.543 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.3640000000000002e-06 2024-09-14T06:31:04.543 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:04.543 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:04.543 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:31:04.543 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.078126+0000", 2024-09-14T06:31:04.543 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032388206000000003 2024-09-14T06:31:04.544 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:04.544 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:04.544 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:31:04.544 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.078144+0000", 2024-09-14T06:31:04.544 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.7686e-05 2024-09-14T06:31:04.544 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:04.544 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:31:04.544 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:04.544 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:04.544 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:04.544 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:98518 3.10 3.1bf56f90 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T06:31:04.544 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:30:07.045741+0000", 2024-09-14T06:31:04.544 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 57.443725153999999, 2024-09-14T06:31:04.544 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032451793999999999, 2024-09-14T06:31:04.544 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:31:04.545 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:31:04.545 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "reached pg", 2024-09-14T06:31:04.545 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:31:04.545 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:31:04.545 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:04.545 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 98518 2024-09-14T06:31:04.545 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:04.545 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:31:04.545 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:04.545 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:31:04.545 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045741+0000", 2024-09-14T06:31:04.545 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:04.545 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:04.545 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:04.546 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:31:04.546 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045741+0000", 2024-09-14T06:31:04.546 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:04.546 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:04.546 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:04.546 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:31:04.546 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045741+0000", 2024-09-14T06:31:04.546 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.05e-07 2024-09-14T06:31:04.546 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:04.546 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:04.546 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:31:04.546 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045746+0000", 2024-09-14T06:31:04.546 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.2259999999999996e-06 2024-09-14T06:31:04.546 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:04.546 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:04.547 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:31:04.547 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045750+0000", 2024-09-14T06:31:04.547 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.2490000000000001e-06 2024-09-14T06:31:04.547 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:04.547 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:04.547 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:31:04.547 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045752+0000", 2024-09-14T06:31:04.547 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.3980000000000002e-06 2024-09-14T06:31:04.547 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:04.547 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:04.547 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:31:04.547 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.078184+0000", 2024-09-14T06:31:04.547 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032431593000000002 2024-09-14T06:31:04.547 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:04.547 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:04.548 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:31:04.548 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.078193+0000", 2024-09-14T06:31:04.548 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.1230000000000006e-06 2024-09-14T06:31:04.548 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:04.548 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:31:04.548 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:04.548 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:04.548 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:04.548 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:98519 3.10 3.1bf56f90 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T06:31:04.548 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:30:07.045755+0000", 2024-09-14T06:31:04.548 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 57.443710729000003, 2024-09-14T06:31:04.548 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032489431999999999, 2024-09-14T06:31:04.548 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:31:04.549 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:31:04.549 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "reached pg", 2024-09-14T06:31:04.549 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:31:04.549 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:31:04.549 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:04.549 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 98519 2024-09-14T06:31:04.549 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:04.549 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:31:04.549 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:04.549 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:31:04.549 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045755+0000", 2024-09-14T06:31:04.549 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:04.549 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:04.549 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:04.550 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:31:04.550 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045755+0000", 2024-09-14T06:31:04.550 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:04.550 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:04.550 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:04.550 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:31:04.550 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045756+0000", 2024-09-14T06:31:04.550 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.23e-07 2024-09-14T06:31:04.550 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:04.550 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:04.550 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:31:04.550 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045757+0000", 2024-09-14T06:31:04.550 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.1060000000000001e-06 2024-09-14T06:31:04.550 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:04.551 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:04.551 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:31:04.551 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045760+0000", 2024-09-14T06:31:04.551 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.988e-06 2024-09-14T06:31:04.551 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:04.551 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:04.551 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:31:04.551 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045765+0000", 2024-09-14T06:31:04.551 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.2689999999999999e-06 2024-09-14T06:31:04.551 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:04.551 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:04.551 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:31:04.551 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.078234+0000", 2024-09-14T06:31:04.551 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032468915000000001 2024-09-14T06:31:04.551 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:04.552 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:04.552 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:31:04.552 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.078245+0000", 2024-09-14T06:31:04.552 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.0930999999999999e-05 2024-09-14T06:31:04.552 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:04.552 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:31:04.552 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:04.552 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:04.552 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:04.552 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:98520 3.10 3.1bf56f90 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T06:31:04.552 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:30:07.045768+0000", 2024-09-14T06:31:04.552 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 57.443697851000003, 2024-09-14T06:31:04.552 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032531379999999999, 2024-09-14T06:31:04.552 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:31:04.553 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:31:04.553 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "reached pg", 2024-09-14T06:31:04.553 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:31:04.553 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:31:04.553 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:04.553 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 98520 2024-09-14T06:31:04.553 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:04.553 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:31:04.553 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:04.553 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:31:04.553 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045768+0000", 2024-09-14T06:31:04.553 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:04.553 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:04.553 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:04.554 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:31:04.554 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045768+0000", 2024-09-14T06:31:04.554 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:04.554 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:04.554 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:04.554 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:31:04.554 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045768+0000", 2024-09-14T06:31:04.554 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.22e-07 2024-09-14T06:31:04.554 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:04.554 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:04.554 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:31:04.554 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045773+0000", 2024-09-14T06:31:04.554 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.3470000000000001e-06 2024-09-14T06:31:04.554 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:04.555 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:04.555 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:31:04.555 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045776+0000", 2024-09-14T06:31:04.555 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.1319999999999998e-06 2024-09-14T06:31:04.555 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:04.555 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:04.555 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:31:04.555 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045779+0000", 2024-09-14T06:31:04.555 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.9749999999999999e-06 2024-09-14T06:31:04.555 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:04.555 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:04.555 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:31:04.555 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.078289+0000", 2024-09-14T06:31:04.555 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032509693999999999 2024-09-14T06:31:04.555 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:04.556 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:04.556 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:31:04.556 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.078300+0000", 2024-09-14T06:31:04.556 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.101e-05 2024-09-14T06:31:04.556 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:04.556 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:31:04.556 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:04.556 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:04.556 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:04.556 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:98521 3.10 3.1bf56f90 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T06:31:04.556 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:30:07.045782+0000", 2024-09-14T06:31:04.556 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 57.443683671000002, 2024-09-14T06:31:04.556 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032566109000000003, 2024-09-14T06:31:04.556 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:31:04.557 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:31:04.557 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "reached pg", 2024-09-14T06:31:04.557 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:31:04.557 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:31:04.557 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:04.557 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 98521 2024-09-14T06:31:04.557 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:04.557 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:31:04.557 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:04.557 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:31:04.557 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045782+0000", 2024-09-14T06:31:04.557 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:04.557 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:04.557 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:04.557 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:31:04.558 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045782+0000", 2024-09-14T06:31:04.558 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:04.558 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:04.558 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:04.558 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:31:04.558 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045783+0000", 2024-09-14T06:31:04.558 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.2499999999999999e-07 2024-09-14T06:31:04.558 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:04.558 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:04.558 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:31:04.558 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045784+0000", 2024-09-14T06:31:04.558 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.0109999999999999e-06 2024-09-14T06:31:04.558 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:04.558 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:04.559 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:31:04.559 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045786+0000", 2024-09-14T06:31:04.559 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.8289999999999999e-06 2024-09-14T06:31:04.559 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:04.559 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:04.559 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:31:04.559 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045793+0000", 2024-09-14T06:31:04.559 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.579e-06 2024-09-14T06:31:04.559 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:04.559 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:04.559 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:31:04.559 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.078338+0000", 2024-09-14T06:31:04.559 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032545389000000001 2024-09-14T06:31:04.559 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:04.559 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:04.560 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:31:04.560 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.078348+0000", 2024-09-14T06:31:04.560 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.0076e-05 2024-09-14T06:31:04.560 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:04.560 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:31:04.560 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:04.560 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:04.560 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:04.560 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:98522 3.10 3.1bf56f90 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T06:31:04.560 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:30:07.045796+0000", 2024-09-14T06:31:04.560 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 57.443669763000003, 2024-09-14T06:31:04.560 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032577846000000001, 2024-09-14T06:31:04.560 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:31:04.560 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:31:04.561 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "reached pg", 2024-09-14T06:31:04.561 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:31:04.561 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:31:04.561 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:04.561 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 98522 2024-09-14T06:31:04.561 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:04.561 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:31:04.561 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:04.561 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:31:04.561 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045796+0000", 2024-09-14T06:31:04.561 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:04.561 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:04.561 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:04.561 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:31:04.561 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045796+0000", 2024-09-14T06:31:04.562 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:04.562 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:04.562 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:04.562 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:31:04.562 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045796+0000", 2024-09-14T06:31:04.562 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.2100000000000001e-07 2024-09-14T06:31:04.562 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:04.562 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:04.562 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:31:04.562 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045797+0000", 2024-09-14T06:31:04.562 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.6899999999999996e-07 2024-09-14T06:31:04.562 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:04.562 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:04.562 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:31:04.563 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045800+0000", 2024-09-14T06:31:04.563 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.9979999999999999e-06 2024-09-14T06:31:04.563 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:04.563 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:04.563 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:31:04.563 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045803+0000", 2024-09-14T06:31:04.563 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.2850000000000001e-06 2024-09-14T06:31:04.563 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:04.563 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:04.563 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:31:04.563 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.078366+0000", 2024-09-14T06:31:04.563 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032562932000000003 2024-09-14T06:31:04.563 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:04.563 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:04.564 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:31:04.564 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.078374+0000", 2024-09-14T06:31:04.564 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.4409999999999993e-06 2024-09-14T06:31:04.564 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:04.564 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:31:04.564 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:04.564 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:04.564 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:04.564 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:98523 3.10 3.1bf56f90 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T06:31:04.564 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:30:07.045806+0000", 2024-09-14T06:31:04.564 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 57.443660219999998, 2024-09-14T06:31:04.564 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032611167000000003, 2024-09-14T06:31:04.564 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:31:04.564 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:31:04.565 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "reached pg", 2024-09-14T06:31:04.565 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:31:04.565 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:31:04.565 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:04.565 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 98523 2024-09-14T06:31:04.565 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:04.565 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:31:04.565 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:04.565 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:31:04.565 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045806+0000", 2024-09-14T06:31:04.565 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:04.565 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:04.565 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:04.565 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:31:04.566 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045806+0000", 2024-09-14T06:31:04.566 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:04.566 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:04.566 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:04.566 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:31:04.566 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045806+0000", 2024-09-14T06:31:04.566 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.2000000000000001e-07 2024-09-14T06:31:04.566 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:04.566 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:04.566 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:31:04.566 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045807+0000", 2024-09-14T06:31:04.566 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.4e-07 2024-09-14T06:31:04.566 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:04.566 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:04.567 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:31:04.567 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045810+0000", 2024-09-14T06:31:04.567 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.9229999999999998e-06 2024-09-14T06:31:04.567 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:04.567 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:04.567 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:31:04.567 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045812+0000", 2024-09-14T06:31:04.567 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.2220000000000001e-06 2024-09-14T06:31:04.567 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:04.567 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:04.567 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:31:04.567 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.078407+0000", 2024-09-14T06:31:04.567 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032594513999999998 2024-09-14T06:31:04.567 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:04.567 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:04.568 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:31:04.568 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.078417+0000", 2024-09-14T06:31:04.568 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.0348e-05 2024-09-14T06:31:04.568 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:04.568 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:31:04.568 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:04.568 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:04.568 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:04.568 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:98524 3.10 3.1bf56f90 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T06:31:04.568 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:30:07.045815+0000", 2024-09-14T06:31:04.568 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 57.443650851999998, 2024-09-14T06:31:04.568 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032643208999999999, 2024-09-14T06:31:04.568 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:31:04.568 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:31:04.569 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "reached pg", 2024-09-14T06:31:04.569 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:31:04.569 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:31:04.569 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:04.569 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 98524 2024-09-14T06:31:04.569 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:04.569 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:31:04.569 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:04.569 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:31:04.569 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045815+0000", 2024-09-14T06:31:04.569 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:04.569 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:04.569 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:04.569 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:31:04.570 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045815+0000", 2024-09-14T06:31:04.570 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:04.570 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:04.570 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:04.570 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:31:04.570 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045815+0000", 2024-09-14T06:31:04.570 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.11e-07 2024-09-14T06:31:04.570 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:04.570 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:04.570 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:31:04.570 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045816+0000", 2024-09-14T06:31:04.570 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.9099999999999991e-07 2024-09-14T06:31:04.570 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:04.570 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:04.570 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:31:04.571 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045819+0000", 2024-09-14T06:31:04.571 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.9399999999999998e-06 2024-09-14T06:31:04.571 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:04.571 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:04.571 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:31:04.571 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045822+0000", 2024-09-14T06:31:04.571 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.3470000000000001e-06 2024-09-14T06:31:04.571 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:04.571 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:04.571 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:31:04.571 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.078449+0000", 2024-09-14T06:31:04.571 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032627529000000002 2024-09-14T06:31:04.571 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:04.571 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:04.572 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:31:04.572 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.078458+0000", 2024-09-14T06:31:04.572 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.1910000000000006e-06 2024-09-14T06:31:04.572 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:04.572 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:31:04.572 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:04.572 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:04.572 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:31:04.572 INFO:teuthology.orchestra.run.smithi138.stdout:} 2024-09-14T06:31:04.573 INFO:teuthology.orchestra.run.smithi094.stdout:osd.6: {} 2024-09-14T06:31:04.573 INFO:teuthology.orchestra.run.smithi094.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-14T06:31:04.574 INFO:teuthology.orchestra.run.smithi094.stdout:osd.7: {} 2024-09-14T06:31:04.574 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-14T06:31:04.575 INFO:teuthology.orchestra.run.smithi094.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-14T06:31:04.616 INFO:tasks.ceph.osd.3.smithi094.stderr:2024-09-14T06:31:04.615+0000 7feeeafe8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33655) UID: 0 2024-09-14T06:31:04.683 INFO:tasks.ceph.osd.1.smithi094.stderr:2024-09-14T06:31:04.682+0000 7f328e47a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33641) UID: 0 2024-09-14T06:31:04.718 INFO:teuthology.orchestra.run.smithi138.stdout:ERROR: (22) Invalid argument 2024-09-14T06:31:04.718 INFO:teuthology.orchestra.run.smithi138.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:31:04.726 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-14T06:31:04.784 INFO:tasks.ceph.osd.5.smithi138.stderr:2024-09-14T06:31:04.783+0000 7fe13746f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33618) UID: 0 2024-09-14T06:31:04.986 INFO:teuthology.orchestra.run.smithi094.stderr:reweighted osd.5 to 1 (10000) 2024-09-14T06:31:04.987 INFO:teuthology.orchestra.run.smithi138.stdout:ERROR: (22) Invalid argument 2024-09-14T06:31:04.987 INFO:teuthology.orchestra.run.smithi138.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:31:04.987 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 6 1 2024-09-14T06:31:04.988 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-14T06:31:04.988 INFO:tasks.ceph.osd.5.smithi138.stderr:2024-09-14T06:31:04.985+0000 7fe13746f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33618) UID: 0 2024-09-14T06:31:05.086 INFO:tasks.ceph.osd.6.smithi138.stderr:2024-09-14T06:31:05.085+0000 7f8cd8849640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33620) UID: 0 2024-09-14T06:31:05.131 INFO:teuthology.orchestra.run.smithi138.stdout:ERROR: (22) Invalid argument 2024-09-14T06:31:05.131 INFO:teuthology.orchestra.run.smithi138.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:31:05.139 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-14T06:31:05.188 INFO:tasks.ceph.osd.5.smithi138.stderr:2024-09-14T06:31:05.187+0000 7fe13746f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33618) UID: 0 2024-09-14T06:31:05.244 INFO:teuthology.orchestra.run.smithi094.stdout:ERROR: (22) Invalid argument 2024-09-14T06:31:05.244 INFO:teuthology.orchestra.run.smithi094.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:31:05.253 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-14T06:31:05.296 INFO:tasks.ceph.osd.0.smithi094.stderr:2024-09-14T06:31:05.295+0000 7f07c9333640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33636) UID: 0 2024-09-14T06:31:05.356 INFO:teuthology.orchestra.run.smithi094.stdout:ERROR: (22) Invalid argument 2024-09-14T06:31:05.356 INFO:teuthology.orchestra.run.smithi094.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:31:05.364 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-14T06:31:05.390 INFO:tasks.ceph.osd.7.smithi138.stderr:2024-09-14T06:31:05.389+0000 7f3c6a21d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33623) UID: 0 2024-09-14T06:31:05.492 INFO:tasks.ceph.osd.1.smithi094.stderr:2024-09-14T06:31:05.491+0000 7f328e47a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33641) UID: 0 2024-09-14T06:31:05.507 INFO:teuthology.orchestra.run.smithi094.stdout:ERROR: (22) Invalid argument 2024-09-14T06:31:05.507 INFO:teuthology.orchestra.run.smithi094.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:31:05.515 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-14T06:31:05.594 INFO:teuthology.orchestra.run.smithi094.stderr:reweighted osd.6 to 1 (10000) 2024-09-14T06:31:05.595 INFO:tasks.ceph.osd.2.smithi094.stderr:2024-09-14T06:31:05.594+0000 7faf3269b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33649) UID: 0 2024-09-14T06:31:05.610 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 7 1 2024-09-14T06:31:05.695 INFO:tasks.ceph.osd.3.smithi094.stderr:2024-09-14T06:31:05.694+0000 7feeeafe8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33655) UID: 0 2024-09-14T06:31:05.700 INFO:teuthology.orchestra.run.smithi094.stdout:ERROR: (22) Invalid argument 2024-09-14T06:31:05.700 INFO:teuthology.orchestra.run.smithi094.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:31:05.710 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-14T06:31:05.795 INFO:tasks.ceph.osd.3.smithi094.stderr:2024-09-14T06:31:05.794+0000 7feeeafe8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33655) UID: 0 2024-09-14T06:31:05.809 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-14T06:31:05.860 INFO:teuthology.orchestra.run.smithi094.stdout:ERROR: (22) Invalid argument 2024-09-14T06:31:05.860 INFO:teuthology.orchestra.run.smithi094.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:31:05.869 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-14T06:31:05.895 INFO:tasks.ceph.osd.5.smithi138.stderr:2024-09-14T06:31:05.894+0000 7fe13746f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33618) UID: 0 2024-09-14T06:31:05.977 INFO:teuthology.orchestra.run.smithi094.stdout:ERROR: (22) Invalid argument 2024-09-14T06:31:05.977 INFO:teuthology.orchestra.run.smithi094.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:31:05.986 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-14T06:31:06.030 INFO:tasks.ceph.osd.2.smithi094.stderr:2024-09-14T06:31:06.029+0000 7faf3269b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33649) UID: 0 2024-09-14T06:31:06.090 INFO:teuthology.orchestra.run.smithi094.stdout:ERROR: (22) Invalid argument 2024-09-14T06:31:06.091 INFO:teuthology.orchestra.run.smithi094.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:31:06.096 INFO:tasks.ceph.osd.4.smithi138.stderr:2024-09-14T06:31:06.095+0000 7f6c653c8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33615) UID: 0 2024-09-14T06:31:06.099 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-14T06:31:06.197 INFO:tasks.ceph.osd.1.smithi094.stderr:2024-09-14T06:31:06.196+0000 7f328e47a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33641) UID: 0 2024-09-14T06:31:06.242 INFO:teuthology.orchestra.run.smithi094.stdout:ERROR: (22) Invalid argument 2024-09-14T06:31:06.242 INFO:teuthology.orchestra.run.smithi094.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:31:06.251 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-14T06:31:06.335 INFO:tasks.ceph.osd.0.smithi094.stderr:2024-09-14T06:31:06.334+0000 7f07c9333640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33636) UID: 0 2024-09-14T06:31:06.393 INFO:teuthology.orchestra.run.smithi094.stdout:ERROR: (22) Invalid argument 2024-09-14T06:31:06.393 INFO:teuthology.orchestra.run.smithi094.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:31:06.398 INFO:tasks.ceph.osd.0.smithi094.stderr:2024-09-14T06:31:06.397+0000 7f07c9333640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33636) UID: 0 2024-09-14T06:31:06.401 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-14T06:31:06.499 INFO:tasks.ceph.osd.1.smithi094.stderr:2024-09-14T06:31:06.498+0000 7f328e47a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33641) UID: 0 2024-09-14T06:31:06.545 INFO:teuthology.orchestra.run.smithi094.stdout:ERROR: (22) Invalid argument 2024-09-14T06:31:06.545 INFO:teuthology.orchestra.run.smithi094.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:31:06.554 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-14T06:31:06.576 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-14T06:31:06.593 INFO:teuthology.orchestra.run.smithi094.stderr:noscrub is unset 2024-09-14T06:31:06.597 INFO:teuthology.orchestra.run.smithi094.stderr:reweighted osd.7 to 1 (10000) 2024-09-14T06:31:06.618 INFO:tasks.ceph.osd.3.smithi094.stderr:2024-09-14T06:31:06.617+0000 7feeeafe8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33655) UID: 0 2024-09-14T06:31:06.619 INFO:tasks.ceph.ceph_manager.ceph:waiting for recovery to complete 2024-09-14T06:31:06.620 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-14T06:31:06.691 INFO:teuthology.orchestra.run.smithi094.stdout:checking clone #30, image image_client.0-clone30 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent31 2024-09-14T06:31:06.700 INFO:tasks.ceph.osd.3.smithi094.stderr:2024-09-14T06:31:06.699+0000 7feeeafe8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33655) UID: 0 2024-09-14T06:31:06.714 INFO:teuthology.orchestra.run.smithi094.stdout:ERROR: (22) Invalid argument 2024-09-14T06:31:06.714 INFO:teuthology.orchestra.run.smithi094.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:31:06.726 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-14T06:31:06.731 INFO:teuthology.orchestra.run.smithi094.stdout:osd.0: {} 2024-09-14T06:31:06.731 INFO:teuthology.orchestra.run.smithi094.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-14T06:31:06.761 INFO:teuthology.orchestra.run.smithi094.stdout:osd.1: {} 2024-09-14T06:31:06.761 INFO:teuthology.orchestra.run.smithi094.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-14T06:31:06.780 INFO:teuthology.orchestra.run.smithi094.stdout:osd.2: {} 2024-09-14T06:31:06.781 INFO:teuthology.orchestra.run.smithi094.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-14T06:31:06.801 INFO:tasks.ceph.osd.1.smithi094.stderr:2024-09-14T06:31:06.800+0000 7f328e47a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33641) UID: 0 2024-09-14T06:31:06.813 INFO:teuthology.orchestra.run.smithi094.stdout:osd.3: {} 2024-09-14T06:31:06.813 INFO:teuthology.orchestra.run.smithi094.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-14T06:31:06.844 INFO:teuthology.orchestra.run.smithi094.stdout:osd.4: {} 2024-09-14T06:31:06.845 INFO:teuthology.orchestra.run.smithi094.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-14T06:31:06.875 INFO:teuthology.orchestra.run.smithi094.stdout:osd.5: {} 2024-09-14T06:31:06.875 INFO:teuthology.orchestra.run.smithi094.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-14T06:31:06.895 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-14T06:31:06.896 INFO:teuthology.orchestra.run.smithi094.stdout: "size": 20, 2024-09-14T06:31:06.896 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 600, 2024-09-14T06:31:06.896 INFO:teuthology.orchestra.run.smithi094.stdout: "ops": [ 2024-09-14T06:31:06.896 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:06.896 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:44425 3.2 3:4603ad7f:::rbd_data.112dca2f6ae1.0000000000000005:17 [read 868352~4096 [fadvise_nocache]] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:31:06.896 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:43.854967+0000", 2024-09-14T06:31:06.896 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 83.038454551000001, 2024-09-14T06:31:06.896 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.063146025999999994, 2024-09-14T06:31:06.896 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:31:06.896 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:31:06.896 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:31:06.897 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:31:06.897 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:31:06.897 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:06.897 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 44425 2024-09-14T06:31:06.897 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:06.897 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:31:06.897 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:06.897 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:31:06.897 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.854967+0000", 2024-09-14T06:31:06.898 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:06.898 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:06.898 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:06.898 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:31:06.898 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.854967+0000", 2024-09-14T06:31:06.898 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:06.898 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:06.898 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:06.898 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:31:06.898 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.854967+0000", 2024-09-14T06:31:06.899 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.16e-07 2024-09-14T06:31:06.899 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:06.899 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:06.899 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:31:06.899 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.854968+0000", 2024-09-14T06:31:06.899 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.006e-06 2024-09-14T06:31:06.899 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:06.899 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:06.899 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:31:06.899 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.854971+0000", 2024-09-14T06:31:06.899 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.9359999999999999e-06 2024-09-14T06:31:06.899 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:06.899 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:06.899 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:31:06.899 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.854974+0000", 2024-09-14T06:31:06.900 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.1840000000000002e-06 2024-09-14T06:31:06.900 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:06.900 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:06.900 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:31:06.900 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.917945+0000", 2024-09-14T06:31:06.900 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.062970925999999997 2024-09-14T06:31:06.900 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:06.900 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:06.900 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:31:06.900 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.918030+0000", 2024-09-14T06:31:06.900 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.5863000000000001e-05 2024-09-14T06:31:06.900 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:06.900 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:06.901 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:31:06.901 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.918113+0000", 2024-09-14T06:31:06.901 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.2894999999999998e-05 2024-09-14T06:31:06.901 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:06.901 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:31:06.901 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:06.901 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:06.901 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:06.901 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:44426 3.2 3:4603ad7f:::rbd_data.112dca2f6ae1.0000000000000005:17 [read 892928~4096 [fadvise_nocache]] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:31:06.901 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:43.854977+0000", 2024-09-14T06:31:06.901 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 83.038445140999997, 2024-09-14T06:31:06.902 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.063369237999999994, 2024-09-14T06:31:06.902 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:31:06.902 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:31:06.902 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:31:06.902 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:31:06.902 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:31:06.902 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:06.902 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 44426 2024-09-14T06:31:06.902 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:06.902 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:31:06.902 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:06.902 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:31:06.902 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.854977+0000", 2024-09-14T06:31:06.903 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:06.903 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:06.903 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:06.903 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:31:06.903 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.854977+0000", 2024-09-14T06:31:06.903 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:06.903 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:06.903 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:06.903 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:31:06.903 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.854977+0000", 2024-09-14T06:31:06.903 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.1199999999999999e-07 2024-09-14T06:31:06.903 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:06.903 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:06.903 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:31:06.904 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.854980+0000", 2024-09-14T06:31:06.904 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.27e-06 2024-09-14T06:31:06.904 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:06.904 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:06.904 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:31:06.904 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.854983+0000", 2024-09-14T06:31:06.904 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.1650000000000002e-06 2024-09-14T06:31:06.904 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:06.904 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:06.904 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:31:06.904 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.854989+0000", 2024-09-14T06:31:06.904 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.6520000000000003e-06 2024-09-14T06:31:06.904 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:06.904 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:06.904 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:31:06.905 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.918143+0000", 2024-09-14T06:31:06.905 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.063154541999999994 2024-09-14T06:31:06.905 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:06.905 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:06.905 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:31:06.905 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.918209+0000", 2024-09-14T06:31:06.905 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.5529000000000002e-05 2024-09-14T06:31:06.905 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:06.905 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:06.905 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:31:06.906 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.918346+0000", 2024-09-14T06:31:06.906 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000136868 2024-09-14T06:31:06.906 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:06.906 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:31:06.906 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:06.906 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:06.906 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:06.906 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:44427 3.2 3:4603ad7f:::rbd_data.112dca2f6ae1.0000000000000005:17 [read 917504~4096 [fadvise_nocache]] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:31:06.906 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:43.854992+0000", 2024-09-14T06:31:06.906 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 83.038429495000003, 2024-09-14T06:31:06.906 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.063607792999999996, 2024-09-14T06:31:06.906 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:31:06.906 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:31:06.906 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:31:06.907 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:31:06.907 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:31:06.907 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:06.907 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 44427 2024-09-14T06:31:06.907 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:06.907 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:31:06.907 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:06.907 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:31:06.907 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.854992+0000", 2024-09-14T06:31:06.907 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:06.907 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:06.907 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:06.907 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:31:06.907 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.854992+0000", 2024-09-14T06:31:06.908 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:06.908 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:06.908 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:06.908 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:31:06.908 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.854993+0000", 2024-09-14T06:31:06.908 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.1400000000000001e-07 2024-09-14T06:31:06.908 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:06.908 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:06.908 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:31:06.908 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.854994+0000", 2024-09-14T06:31:06.908 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.091e-06 2024-09-14T06:31:06.908 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:06.908 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:06.908 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:31:06.909 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855001+0000", 2024-09-14T06:31:06.909 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.204e-06 2024-09-14T06:31:06.909 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:06.909 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:06.909 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:31:06.909 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855003+0000", 2024-09-14T06:31:06.909 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.514e-06 2024-09-14T06:31:06.909 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:06.909 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:06.909 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:31:06.909 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.918380+0000", 2024-09-14T06:31:06.909 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.063376738000000002 2024-09-14T06:31:06.910 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:06.910 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:06.910 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:31:06.910 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.918453+0000", 2024-09-14T06:31:06.910 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.3291e-05 2024-09-14T06:31:06.910 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:06.910 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:06.910 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:31:06.910 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.918600+0000", 2024-09-14T06:31:06.910 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00014674100000000001 2024-09-14T06:31:06.910 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:06.910 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:31:06.910 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:06.910 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:06.910 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:06.911 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:44428 3.2 3:4603ad7f:::rbd_data.112dca2f6ae1.0000000000000005:17 [read 942080~4096 [fadvise_nocache]] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:31:06.911 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:43.855006+0000", 2024-09-14T06:31:06.911 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 83.038415444999998, 2024-09-14T06:31:06.911 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.063846321999999997, 2024-09-14T06:31:06.911 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:31:06.911 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:31:06.911 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:31:06.912 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:31:06.912 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:31:06.912 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:06.912 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 44428 2024-09-14T06:31:06.912 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:06.912 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:31:06.912 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:06.912 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:31:06.912 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855006+0000", 2024-09-14T06:31:06.912 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:06.912 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:06.912 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:06.913 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:31:06.913 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855006+0000", 2024-09-14T06:31:06.913 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:06.913 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:06.913 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:06.913 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:31:06.913 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855007+0000", 2024-09-14T06:31:06.913 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.1199999999999999e-07 2024-09-14T06:31:06.913 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:06.913 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:06.913 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:31:06.913 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855011+0000", 2024-09-14T06:31:06.914 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.9539999999999998e-06 2024-09-14T06:31:06.914 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:06.914 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:06.914 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:31:06.914 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855014+0000", 2024-09-14T06:31:06.914 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.2930000000000001e-06 2024-09-14T06:31:06.914 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:06.914 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:06.914 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:31:06.914 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855016+0000", 2024-09-14T06:31:06.914 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.3039999999999999e-06 2024-09-14T06:31:06.914 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:06.914 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:06.914 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:31:06.915 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.918634+0000", 2024-09-14T06:31:06.915 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.063617928000000004 2024-09-14T06:31:06.915 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:06.915 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:06.915 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:31:06.915 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.918711+0000", 2024-09-14T06:31:06.915 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.6545000000000007e-05 2024-09-14T06:31:06.915 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:06.915 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:06.915 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:31:06.915 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.918853+0000", 2024-09-14T06:31:06.915 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00014208600000000001 2024-09-14T06:31:06.916 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:06.916 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:31:06.916 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:06.916 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:06.916 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:06.916 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:44429 3.2 3:4603ad7f:::rbd_data.112dca2f6ae1.0000000000000005:17 [read 966656~4096 [fadvise_nocache]] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:31:06.916 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:43.855019+0000", 2024-09-14T06:31:06.916 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 83.038402771999998, 2024-09-14T06:31:06.916 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.064059226999999996, 2024-09-14T06:31:06.916 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:31:06.916 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:31:06.916 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:31:06.916 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:31:06.917 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:31:06.917 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:06.917 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 44429 2024-09-14T06:31:06.917 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:06.917 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:31:06.917 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:06.917 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:31:06.917 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855019+0000", 2024-09-14T06:31:06.917 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:06.917 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:06.917 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:06.917 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:31:06.917 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855019+0000", 2024-09-14T06:31:06.918 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:06.918 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:06.918 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:06.918 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:31:06.918 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855019+0000", 2024-09-14T06:31:06.918 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.28e-07 2024-09-14T06:31:06.918 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:06.918 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:06.918 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:31:06.918 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855020+0000", 2024-09-14T06:31:06.918 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.0330000000000001e-06 2024-09-14T06:31:06.918 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:06.918 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:06.918 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:31:06.918 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855023+0000", 2024-09-14T06:31:06.919 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.79e-06 2024-09-14T06:31:06.919 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:06.919 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:06.919 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:31:06.919 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855025+0000", 2024-09-14T06:31:06.919 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.216e-06 2024-09-14T06:31:06.919 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:06.919 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:06.919 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:31:06.919 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.918886+0000", 2024-09-14T06:31:06.919 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.063861037999999995 2024-09-14T06:31:06.919 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:06.920 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:06.920 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:31:06.920 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.918952+0000", 2024-09-14T06:31:06.920 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.5993000000000003e-05 2024-09-14T06:31:06.920 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:06.920 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:06.920 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:31:06.920 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.919078+0000", 2024-09-14T06:31:06.920 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00012592899999999999 2024-09-14T06:31:06.920 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:06.920 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:31:06.920 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:06.920 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:06.920 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:06.921 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:44430 3.2 3:4603ad7f:::rbd_data.112dca2f6ae1.0000000000000005:17 [read 991232~4096 [fadvise_nocache]] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:31:06.921 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:43.855028+0000", 2024-09-14T06:31:06.921 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 83.038393761999998, 2024-09-14T06:31:06.921 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.064284346000000006, 2024-09-14T06:31:06.921 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:31:06.921 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:31:06.921 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:31:06.921 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:31:06.921 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:31:06.921 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:06.921 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 44430 2024-09-14T06:31:06.921 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:06.921 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:31:06.921 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:06.922 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:31:06.922 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855028+0000", 2024-09-14T06:31:06.922 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:06.922 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:06.922 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:06.922 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:31:06.922 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855028+0000", 2024-09-14T06:31:06.922 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:06.922 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:06.922 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:06.922 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:31:06.922 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855028+0000", 2024-09-14T06:31:06.922 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.1199999999999999e-07 2024-09-14T06:31:06.922 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:06.922 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:06.923 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:31:06.923 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855029+0000", 2024-09-14T06:31:06.923 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.1100000000000004e-07 2024-09-14T06:31:06.923 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:06.923 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:06.923 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:31:06.923 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855032+0000", 2024-09-14T06:31:06.923 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.7889999999999999e-06 2024-09-14T06:31:06.923 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:06.923 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:06.923 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:31:06.923 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855034+0000", 2024-09-14T06:31:06.923 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.125e-06 2024-09-14T06:31:06.923 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:06.924 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:06.924 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:31:06.924 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.919110+0000", 2024-09-14T06:31:06.924 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.064076413999999998 2024-09-14T06:31:06.924 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:06.924 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:06.924 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:31:06.924 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.919186+0000", 2024-09-14T06:31:06.924 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.5784999999999994e-05 2024-09-14T06:31:06.924 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:06.924 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:06.924 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:31:06.925 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.919312+0000", 2024-09-14T06:31:06.925 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00012611 2024-09-14T06:31:06.925 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:06.925 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:31:06.925 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:06.925 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:06.925 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:06.925 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:44431 3.2 3:4603ad7f:::rbd_data.112dca2f6ae1.0000000000000005:17 [read 1015808~4096 [fadvise_nocache]] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:31:06.925 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:43.855037+0000", 2024-09-14T06:31:06.925 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 83.038384894000004, 2024-09-14T06:31:06.925 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.064469755000000004, 2024-09-14T06:31:06.925 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:31:06.925 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:31:06.925 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:31:06.926 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:31:06.926 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:31:06.926 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:06.926 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 44431 2024-09-14T06:31:06.926 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:06.926 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:31:06.926 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:06.926 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:31:06.926 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855037+0000", 2024-09-14T06:31:06.926 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:06.926 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:06.926 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:06.926 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:31:06.926 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855037+0000", 2024-09-14T06:31:06.927 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:06.927 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:06.927 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:06.927 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:31:06.927 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855037+0000", 2024-09-14T06:31:06.927 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.1e-07 2024-09-14T06:31:06.927 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:06.927 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:06.927 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:31:06.927 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855038+0000", 2024-09-14T06:31:06.927 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.9400000000000004e-07 2024-09-14T06:31:06.927 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:06.927 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:06.927 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:31:06.928 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855041+0000", 2024-09-14T06:31:06.928 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.9639999999999999e-06 2024-09-14T06:31:06.928 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:06.928 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:06.928 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:31:06.928 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855043+0000", 2024-09-14T06:31:06.928 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.2259999999999999e-06 2024-09-14T06:31:06.928 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:06.928 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:06.928 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:31:06.928 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.919347+0000", 2024-09-14T06:31:06.928 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.064304091999999993 2024-09-14T06:31:06.928 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:06.929 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:06.929 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:31:06.929 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.919393+0000", 2024-09-14T06:31:06.929 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.5340999999999998e-05 2024-09-14T06:31:06.929 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:06.929 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:06.929 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:31:06.929 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.919507+0000", 2024-09-14T06:31:06.929 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000114028 2024-09-14T06:31:06.929 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:06.929 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:31:06.929 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:06.929 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:06.929 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:06.930 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:44432 3.2 3:4603ad7f:::rbd_data.112dca2f6ae1.0000000000000005:17 [read 1040384~4096 [fadvise_nocache]] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:31:06.930 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:43.855046+0000", 2024-09-14T06:31:06.930 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 83.038375826000006, 2024-09-14T06:31:06.930 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.064639253999999993, 2024-09-14T06:31:06.930 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:31:06.930 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:31:06.930 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:31:06.930 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:31:06.930 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:31:06.930 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:06.930 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 44432 2024-09-14T06:31:06.930 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:06.930 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:31:06.930 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:06.931 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:31:06.931 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855046+0000", 2024-09-14T06:31:06.931 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:06.931 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:06.931 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:06.931 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:31:06.931 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855046+0000", 2024-09-14T06:31:06.931 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:06.931 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:06.931 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:06.931 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:31:06.931 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855046+0000", 2024-09-14T06:31:06.931 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.05e-07 2024-09-14T06:31:06.931 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:06.931 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:06.932 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:31:06.932 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855047+0000", 2024-09-14T06:31:06.932 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.879999999999999e-07 2024-09-14T06:31:06.932 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:06.932 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:06.932 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:31:06.932 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855050+0000", 2024-09-14T06:31:06.932 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.8459999999999999e-06 2024-09-14T06:31:06.932 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:06.932 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:06.932 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:31:06.932 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855052+0000", 2024-09-14T06:31:06.932 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.136e-06 2024-09-14T06:31:06.933 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:06.933 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:06.933 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:31:06.933 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.919536+0000", 2024-09-14T06:31:06.933 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.064484132 2024-09-14T06:31:06.933 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:06.933 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:06.933 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:31:06.933 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.919611+0000", 2024-09-14T06:31:06.933 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.4407999999999996e-05 2024-09-14T06:31:06.933 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:06.933 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:06.933 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:31:06.933 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.919685+0000", 2024-09-14T06:31:06.934 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.4538999999999999e-05 2024-09-14T06:31:06.934 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:06.934 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:31:06.934 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:06.934 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:06.934 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:06.934 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:44518 3.2 3:4190d882:::rbd_data.112dca2f6ae1.0000000000000002:17 [read 12288~4096 [fadvise_nocache]] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:31:06.934 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:43.855055+0000", 2024-09-14T06:31:06.934 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 83.038366941999996, 2024-09-14T06:31:06.934 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.064894317000000007, 2024-09-14T06:31:06.934 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:31:06.934 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:31:06.934 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:31:06.935 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:31:06.935 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:31:06.935 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:06.935 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 44518 2024-09-14T06:31:06.935 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:06.935 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:31:06.935 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:06.935 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:31:06.935 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855055+0000", 2024-09-14T06:31:06.935 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:06.935 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:06.935 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:06.935 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:31:06.936 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855055+0000", 2024-09-14T06:31:06.936 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:06.936 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:06.936 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:06.936 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:31:06.936 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855055+0000", 2024-09-14T06:31:06.936 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.0699999999999999e-07 2024-09-14T06:31:06.936 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:06.936 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:06.936 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:31:06.936 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855056+0000", 2024-09-14T06:31:06.937 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.9400000000000004e-07 2024-09-14T06:31:06.937 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:06.937 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:06.937 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:31:06.937 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855059+0000", 2024-09-14T06:31:06.937 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.8870000000000001e-06 2024-09-14T06:31:06.937 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:06.937 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:06.937 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:31:06.937 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855061+0000", 2024-09-14T06:31:06.937 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.1919999999999999e-06 2024-09-14T06:31:06.937 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:06.937 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:06.937 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:31:06.938 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.919719+0000", 2024-09-14T06:31:06.938 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.064658013 2024-09-14T06:31:06.938 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:06.938 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:06.938 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:31:06.938 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.919788+0000", 2024-09-14T06:31:06.938 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.9282000000000005e-05 2024-09-14T06:31:06.938 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:06.938 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:06.938 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:31:06.938 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.919949+0000", 2024-09-14T06:31:06.938 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00016084199999999999 2024-09-14T06:31:06.938 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:06.938 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:31:06.938 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:06.939 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:06.939 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:06.939 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:44519 3.2 3:4190d882:::rbd_data.112dca2f6ae1.0000000000000002:17 [read 36864~4096 [fadvise_nocache]] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:31:06.939 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:43.855064+0000", 2024-09-14T06:31:06.939 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 83.038357965000003, 2024-09-14T06:31:06.939 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.065081865000000003, 2024-09-14T06:31:06.939 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:31:06.939 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:31:06.939 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:31:06.939 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:31:06.939 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:31:06.939 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:06.939 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 44519 2024-09-14T06:31:06.939 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:06.940 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:31:06.940 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:06.940 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:31:06.940 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855064+0000", 2024-09-14T06:31:06.940 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:06.940 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:06.940 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:06.940 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:31:06.940 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855064+0000", 2024-09-14T06:31:06.940 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:06.940 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:06.940 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:06.940 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:31:06.940 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855064+0000", 2024-09-14T06:31:06.941 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.1e-07 2024-09-14T06:31:06.941 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:06.941 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:06.941 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:31:06.941 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855065+0000", 2024-09-14T06:31:06.941 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.035e-06 2024-09-14T06:31:06.941 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:06.941 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:06.941 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:31:06.941 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855068+0000", 2024-09-14T06:31:06.941 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.8339999999999999e-06 2024-09-14T06:31:06.941 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:06.941 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:06.941 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:31:06.942 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855071+0000", 2024-09-14T06:31:06.942 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.3450000000000002e-06 2024-09-14T06:31:06.942 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:06.942 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:06.942 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:31:06.942 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.919979+0000", 2024-09-14T06:31:06.942 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.064907647999999998 2024-09-14T06:31:06.942 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:06.942 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:06.942 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:31:06.942 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.920026+0000", 2024-09-14T06:31:06.942 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.7595000000000001e-05 2024-09-14T06:31:06.942 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:06.942 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:06.942 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:31:06.943 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.920146+0000", 2024-09-14T06:31:06.943 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00011919800000000001 2024-09-14T06:31:06.943 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:06.943 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:31:06.943 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:06.943 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:06.943 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:06.943 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:44520 3.2 3:4190d882:::rbd_data.112dca2f6ae1.0000000000000002:17 [read 61440~4096 [fadvise_nocache]] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:31:06.943 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:43.855075+0000", 2024-09-14T06:31:06.943 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 83.038346914000002, 2024-09-14T06:31:06.943 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.065335078000000005, 2024-09-14T06:31:06.943 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:31:06.943 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:31:06.943 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:31:06.944 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:31:06.944 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:31:06.944 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:06.944 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 44520 2024-09-14T06:31:06.944 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:06.944 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:31:06.944 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:06.944 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:31:06.944 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855075+0000", 2024-09-14T06:31:06.944 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:06.945 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:06.945 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:06.945 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:31:06.945 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855075+0000", 2024-09-14T06:31:06.945 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:06.945 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:06.945 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:06.946 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:31:06.946 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855075+0000", 2024-09-14T06:31:06.946 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.1e-07 2024-09-14T06:31:06.946 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:06.946 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:06.946 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:31:06.946 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855076+0000", 2024-09-14T06:31:06.947 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.0839999999999999e-06 2024-09-14T06:31:06.947 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:06.947 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:06.947 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:31:06.947 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855079+0000", 2024-09-14T06:31:06.947 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.9359999999999999e-06 2024-09-14T06:31:06.947 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:06.947 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:06.947 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:31:06.948 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855084+0000", 2024-09-14T06:31:06.948 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.1730000000000003e-06 2024-09-14T06:31:06.948 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:06.948 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:06.948 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:31:06.948 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.920175+0000", 2024-09-14T06:31:06.949 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.065090234999999996 2024-09-14T06:31:06.949 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:06.949 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:06.949 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:31:06.949 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.920250+0000", 2024-09-14T06:31:06.949 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.5807000000000001e-05 2024-09-14T06:31:06.949 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:06.949 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:06.949 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:31:06.950 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.920410+0000", 2024-09-14T06:31:06.950 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00015963300000000001 2024-09-14T06:31:06.950 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:06.950 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:31:06.950 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:06.950 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:06.950 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:06.950 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:44521 3.2 3:4190d882:::rbd_data.112dca2f6ae1.0000000000000002:17 [read 86016~4096 [fadvise_nocache]] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:31:06.950 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:43.855087+0000", 2024-09-14T06:31:06.950 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 83.038334433000003, 2024-09-14T06:31:06.950 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.065563264999999996, 2024-09-14T06:31:06.951 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:31:06.951 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:31:06.951 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:31:06.951 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:31:06.951 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:31:06.951 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:06.952 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 44521 2024-09-14T06:31:06.952 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:06.952 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:31:06.952 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:06.952 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:31:06.952 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855087+0000", 2024-09-14T06:31:06.952 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:06.953 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:06.953 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:06.953 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:31:06.953 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855087+0000", 2024-09-14T06:31:06.953 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:06.953 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:06.954 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:06.954 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:31:06.954 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855088+0000", 2024-09-14T06:31:06.954 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.3699999999999999e-07 2024-09-14T06:31:06.954 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:06.954 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:06.954 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:31:06.955 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855089+0000", 2024-09-14T06:31:06.955 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.9900000000000009e-07 2024-09-14T06:31:06.955 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:06.955 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:06.955 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:31:06.955 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855091+0000", 2024-09-14T06:31:06.955 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.8940000000000002e-06 2024-09-14T06:31:06.956 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:06.956 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:06.956 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:31:06.956 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855094+0000", 2024-09-14T06:31:06.956 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.351e-06 2024-09-14T06:31:06.956 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:06.956 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:06.957 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:31:06.957 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.920437+0000", 2024-09-14T06:31:06.957 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.065343221000000007 2024-09-14T06:31:06.957 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:06.957 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:06.957 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:31:06.958 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.920481+0000", 2024-09-14T06:31:06.958 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.4431000000000002e-05 2024-09-14T06:31:06.958 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:06.958 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:06.958 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:31:06.958 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.920651+0000", 2024-09-14T06:31:06.958 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00016913200000000001 2024-09-14T06:31:06.959 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:06.959 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:31:06.959 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:06.959 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:06.959 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:06.959 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:44522 3.2 3:4190d882:::rbd_data.112dca2f6ae1.0000000000000002:17 [read 110592~4096 [fadvise_nocache]] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:31:06.959 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:43.855098+0000", 2024-09-14T06:31:06.959 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 83.038324205999999, 2024-09-14T06:31:06.959 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.065827628999999999, 2024-09-14T06:31:06.960 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:31:06.960 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:31:06.960 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:31:06.960 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:31:06.960 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:31:06.960 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:06.960 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 44522 2024-09-14T06:31:06.960 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:06.960 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:31:06.960 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:06.960 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:31:06.960 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855098+0000", 2024-09-14T06:31:06.961 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:06.961 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:06.961 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:06.961 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:31:06.961 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855098+0000", 2024-09-14T06:31:06.961 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:06.961 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:06.961 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:06.961 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:31:06.961 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855098+0000", 2024-09-14T06:31:06.961 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.1e-07 2024-09-14T06:31:06.961 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:06.962 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:06.962 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:31:06.962 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855099+0000", 2024-09-14T06:31:06.962 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.012e-06 2024-09-14T06:31:06.962 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:06.962 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:06.963 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:31:06.963 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855102+0000", 2024-09-14T06:31:06.963 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.9380000000000001e-06 2024-09-14T06:31:06.963 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:06.963 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:06.963 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:31:06.963 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855108+0000", 2024-09-14T06:31:06.963 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.4339999999999997e-06 2024-09-14T06:31:06.963 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:06.964 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:06.964 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:31:06.964 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.920681+0000", 2024-09-14T06:31:06.964 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.065572943999999994 2024-09-14T06:31:06.964 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:06.964 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:06.964 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:31:06.964 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.920724+0000", 2024-09-14T06:31:06.964 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.2877000000000003e-05 2024-09-14T06:31:06.964 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:06.964 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:06.964 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:31:06.964 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.920925+0000", 2024-09-14T06:31:06.964 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00020121399999999999 2024-09-14T06:31:06.965 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:06.965 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:31:06.965 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:06.965 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:06.965 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:06.965 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:44523 3.2 3:4190d882:::rbd_data.112dca2f6ae1.0000000000000002:17 [read 135168~4096 [fadvise_nocache]] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:31:06.965 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:43.855111+0000", 2024-09-14T06:31:06.966 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 83.038310483999993, 2024-09-14T06:31:06.966 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.066028198999999996, 2024-09-14T06:31:06.966 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:31:06.966 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:31:06.966 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:31:06.966 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:31:06.966 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:31:06.967 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:06.967 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 44523 2024-09-14T06:31:06.967 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:06.967 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:31:06.967 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:06.967 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:31:06.967 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855111+0000", 2024-09-14T06:31:06.967 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:06.967 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:06.967 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:06.967 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:31:06.967 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855111+0000", 2024-09-14T06:31:06.967 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:06.967 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:06.968 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:06.968 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:31:06.968 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855112+0000", 2024-09-14T06:31:06.968 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.1199999999999999e-07 2024-09-14T06:31:06.968 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:06.968 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:06.968 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:31:06.968 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855115+0000", 2024-09-14T06:31:06.968 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.3380000000000001e-06 2024-09-14T06:31:06.968 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:06.968 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:06.968 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:31:06.968 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855118+0000", 2024-09-14T06:31:06.968 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.366e-06 2024-09-14T06:31:06.969 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:06.969 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:06.969 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:31:06.969 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855121+0000", 2024-09-14T06:31:06.969 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.8329999999999998e-06 2024-09-14T06:31:06.969 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:06.969 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:06.969 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:31:06.969 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.920956+0000", 2024-09-14T06:31:06.969 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.065835251999999997 2024-09-14T06:31:06.969 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:06.969 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:06.969 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:31:06.970 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.921001+0000", 2024-09-14T06:31:06.970 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.4778999999999999e-05 2024-09-14T06:31:06.970 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:06.970 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:06.970 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:31:06.970 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.921139+0000", 2024-09-14T06:31:06.970 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000138419 2024-09-14T06:31:06.970 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:06.970 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:31:06.970 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:06.970 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:06.970 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:06.970 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:44524 3.2 3:4190d882:::rbd_data.112dca2f6ae1.0000000000000002:17 [read 159744~4096 [fadvise_nocache]] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:31:06.970 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:43.855124+0000", 2024-09-14T06:31:06.970 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 83.038297505000003, 2024-09-14T06:31:06.971 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.066249261000000004, 2024-09-14T06:31:06.971 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:31:06.971 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:31:06.971 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:31:06.971 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:31:06.971 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:31:06.971 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:06.971 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 44524 2024-09-14T06:31:06.971 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:06.971 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:31:06.971 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:06.971 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:31:06.971 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855124+0000", 2024-09-14T06:31:06.971 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:06.972 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:06.972 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:06.972 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:31:06.972 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855124+0000", 2024-09-14T06:31:06.972 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:06.972 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:06.972 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:06.972 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:31:06.972 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861086+0000", 2024-09-14T06:31:06.972 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0059613829999999998 2024-09-14T06:31:06.972 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:06.972 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:06.972 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:31:06.972 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861100+0000", 2024-09-14T06:31:06.973 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.4644000000000001e-05 2024-09-14T06:31:06.973 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:06.973 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:06.973 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:31:06.973 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861109+0000", 2024-09-14T06:31:06.973 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.541e-06 2024-09-14T06:31:06.973 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:06.973 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:06.973 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:31:06.973 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861117+0000", 2024-09-14T06:31:06.973 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.1049999999999995e-06 2024-09-14T06:31:06.973 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:06.973 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:06.973 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:31:06.974 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.921169+0000", 2024-09-14T06:31:06.974 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.060052405000000003 2024-09-14T06:31:06.974 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:06.974 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:06.974 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:31:06.974 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.921213+0000", 2024-09-14T06:31:06.974 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.3565e-05 2024-09-14T06:31:06.974 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:06.974 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:06.974 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:31:06.974 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.921374+0000", 2024-09-14T06:31:06.974 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00016061800000000001 2024-09-14T06:31:06.974 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:06.974 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:31:06.975 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:06.975 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:06.975 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:06.975 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:44536 3.2 3:4190d882:::rbd_data.112dca2f6ae1.0000000000000002:17 [read 454656~4096 [fadvise_nocache]] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:31:06.975 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:43.861384+0000", 2024-09-14T06:31:06.975 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 83.032037947999996, 2024-09-14T06:31:06.975 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.063058528000000003, 2024-09-14T06:31:06.975 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:31:06.975 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:31:06.975 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:31:06.975 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:31:06.975 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:31:06.975 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:06.975 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 44536 2024-09-14T06:31:06.975 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:06.976 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:31:06.976 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:06.976 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:31:06.976 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861384+0000", 2024-09-14T06:31:06.976 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:06.976 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:06.976 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:06.976 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:31:06.976 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861384+0000", 2024-09-14T06:31:06.976 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:06.976 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:06.976 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:06.976 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:31:06.976 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861384+0000", 2024-09-14T06:31:06.976 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.4000000000000002e-07 2024-09-14T06:31:06.977 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:06.977 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:06.977 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:31:06.977 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861386+0000", 2024-09-14T06:31:06.977 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.891e-06 2024-09-14T06:31:06.977 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:06.977 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:06.977 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:31:06.977 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861392+0000", 2024-09-14T06:31:06.977 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.9939999999999997e-06 2024-09-14T06:31:06.977 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:06.977 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:06.977 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:31:06.978 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861398+0000", 2024-09-14T06:31:06.978 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.507e-06 2024-09-14T06:31:06.978 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:06.978 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:06.978 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:31:06.978 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.924149+0000", 2024-09-14T06:31:06.978 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.062751135999999999 2024-09-14T06:31:06.978 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:06.978 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:06.978 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:31:06.978 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.924226+0000", 2024-09-14T06:31:06.978 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.7517999999999999e-05 2024-09-14T06:31:06.978 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:06.978 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:06.978 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:31:06.979 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.924442+0000", 2024-09-14T06:31:06.979 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000216042 2024-09-14T06:31:06.979 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:06.979 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:31:06.979 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:06.979 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:06.979 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:06.979 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:44537 3.2 3:4190d882:::rbd_data.112dca2f6ae1.0000000000000002:17 [read 479232~4096 [fadvise_nocache]] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:31:06.979 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:43.861405+0000", 2024-09-14T06:31:06.979 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 83.032016901999995, 2024-09-14T06:31:06.979 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.063351795000000002, 2024-09-14T06:31:06.979 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:31:06.979 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:31:06.979 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:31:06.980 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:31:06.980 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:31:06.980 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:06.980 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 44537 2024-09-14T06:31:06.980 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:06.980 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:31:06.980 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:06.980 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:31:06.980 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861405+0000", 2024-09-14T06:31:06.980 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:06.980 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:06.980 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:06.980 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:31:06.981 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861405+0000", 2024-09-14T06:31:06.981 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:06.981 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:06.981 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:06.981 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:31:06.981 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861405+0000", 2024-09-14T06:31:06.981 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.9099999999999999e-07 2024-09-14T06:31:06.981 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:06.981 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:06.981 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:31:06.981 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861407+0000", 2024-09-14T06:31:06.981 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.075e-06 2024-09-14T06:31:06.981 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:06.981 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:06.982 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:31:06.982 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861414+0000", 2024-09-14T06:31:06.982 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.4659999999999996e-06 2024-09-14T06:31:06.982 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:06.982 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:06.982 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:31:06.982 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861418+0000", 2024-09-14T06:31:06.982 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.5519999999999998e-06 2024-09-14T06:31:06.982 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:06.982 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:06.982 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:31:06.983 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.924474+0000", 2024-09-14T06:31:06.983 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.063056012999999994 2024-09-14T06:31:06.983 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:06.983 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:06.983 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:31:06.983 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.924556+0000", 2024-09-14T06:31:06.983 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.1181999999999996e-05 2024-09-14T06:31:06.983 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:06.983 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:06.983 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:31:06.983 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.924757+0000", 2024-09-14T06:31:06.983 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00020111599999999999 2024-09-14T06:31:06.983 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:06.983 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:31:06.984 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:06.984 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:06.984 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:06.984 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:44538 3.2 3:4190d882:::rbd_data.112dca2f6ae1.0000000000000002:17 [read 503808~4096 [fadvise_nocache]] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:31:06.984 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:43.861425+0000", 2024-09-14T06:31:06.984 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 83.031996860999996, 2024-09-14T06:31:06.984 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.063627125000000007, 2024-09-14T06:31:06.984 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:31:06.984 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:31:06.984 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:31:06.984 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:31:06.984 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:31:06.984 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:06.984 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 44538 2024-09-14T06:31:06.984 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:06.985 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:31:06.985 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:06.985 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:31:06.985 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861425+0000", 2024-09-14T06:31:06.985 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:06.985 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:06.985 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:06.985 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:31:06.985 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861425+0000", 2024-09-14T06:31:06.985 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:06.985 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:06.985 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:06.985 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:31:06.986 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861425+0000", 2024-09-14T06:31:06.986 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.9999999999999998e-07 2024-09-14T06:31:06.986 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:06.986 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:06.986 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:31:06.986 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861427+0000", 2024-09-14T06:31:06.986 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.9829999999999999e-06 2024-09-14T06:31:06.986 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:06.986 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:06.986 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:31:06.986 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861433+0000", 2024-09-14T06:31:06.986 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.2889999999999997e-06 2024-09-14T06:31:06.986 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:06.986 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:06.987 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:31:06.987 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861454+0000", 2024-09-14T06:31:06.987 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.1844999999999998e-05 2024-09-14T06:31:06.987 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:06.987 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:06.987 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:31:06.987 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.924786+0000", 2024-09-14T06:31:06.987 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.063331497 2024-09-14T06:31:06.987 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:06.987 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:06.987 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:31:06.987 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.924864+0000", 2024-09-14T06:31:06.988 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.7744000000000006e-05 2024-09-14T06:31:06.988 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:06.988 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:06.988 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:31:06.988 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.925052+0000", 2024-09-14T06:31:06.988 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00018836699999999999 2024-09-14T06:31:06.988 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:06.988 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:31:06.988 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:06.988 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:06.988 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:06.988 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:44539 3.2 3:4190d882:::rbd_data.112dca2f6ae1.0000000000000002:17 [read 528384~4096 [fadvise_nocache]] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:31:06.989 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:43.861462+0000", 2024-09-14T06:31:06.989 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 83.031959533999995, 2024-09-14T06:31:06.989 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.063855705999999998, 2024-09-14T06:31:06.989 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:31:06.989 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:31:06.989 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:31:06.989 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:31:06.989 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:31:06.989 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:06.989 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 44539 2024-09-14T06:31:06.989 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:06.989 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:31:06.989 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:06.990 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:31:06.990 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861462+0000", 2024-09-14T06:31:06.990 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:06.990 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:06.990 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:06.990 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:31:06.990 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861462+0000", 2024-09-14T06:31:06.990 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:06.990 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:06.990 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:06.990 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:31:06.990 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861463+0000", 2024-09-14T06:31:06.990 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.5200000000000002e-07 2024-09-14T06:31:06.990 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:06.991 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:06.991 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:31:06.991 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861465+0000", 2024-09-14T06:31:06.991 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.5189999999999999e-06 2024-09-14T06:31:06.991 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:06.991 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:06.991 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:31:06.991 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861471+0000", 2024-09-14T06:31:06.991 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.6899999999999997e-06 2024-09-14T06:31:06.991 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:06.991 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:06.991 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:31:06.991 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861475+0000", 2024-09-14T06:31:06.991 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.5730000000000001e-06 2024-09-14T06:31:06.992 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:06.992 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:06.992 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:31:06.992 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.925085+0000", 2024-09-14T06:31:06.992 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.063609902999999995 2024-09-14T06:31:06.992 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:06.992 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:06.992 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:31:06.992 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.925145+0000", 2024-09-14T06:31:06.992 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.9383999999999999e-05 2024-09-14T06:31:06.992 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:06.992 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:06.992 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:31:06.992 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.925318+0000", 2024-09-14T06:31:06.993 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00017318499999999999 2024-09-14T06:31:06.993 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:06.993 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:31:06.993 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:06.993 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:06.993 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:06.993 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:44540 3.2 3:4190d882:::rbd_data.112dca2f6ae1.0000000000000002:17 [read 552960~4096 [fadvise_nocache]] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:31:06.993 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:43.861483+0000", 2024-09-14T06:31:06.993 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 83.031939102999999, 2024-09-14T06:31:06.993 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.064065681999999999, 2024-09-14T06:31:06.993 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:31:06.993 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:31:06.993 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:31:06.994 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:31:06.994 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:31:06.994 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:06.994 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 44540 2024-09-14T06:31:06.994 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:06.994 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:31:06.994 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:06.994 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:31:06.994 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861483+0000", 2024-09-14T06:31:06.994 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:06.994 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:06.994 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:06.994 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:31:06.994 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861483+0000", 2024-09-14T06:31:06.995 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:06.995 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:06.995 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:06.995 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:31:06.995 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.867079+0000", 2024-09-14T06:31:06.995 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0055966480000000001 2024-09-14T06:31:06.995 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:06.995 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:06.995 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:31:06.995 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.867086+0000", 2024-09-14T06:31:06.995 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.387e-06 2024-09-14T06:31:06.995 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:06.995 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:06.995 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:31:06.995 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.867094+0000", 2024-09-14T06:31:06.995 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.9419999999999995e-06 2024-09-14T06:31:06.996 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:06.996 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:06.996 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:31:06.996 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.867109+0000", 2024-09-14T06:31:06.996 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.5818999999999999e-05 2024-09-14T06:31:06.996 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:06.996 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:06.996 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:31:06.996 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.925354+0000", 2024-09-14T06:31:06.996 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.058244047 2024-09-14T06:31:06.996 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:06.996 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:06.996 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:31:06.996 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.925406+0000", 2024-09-14T06:31:06.996 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.2661999999999998e-05 2024-09-14T06:31:06.997 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:06.997 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:06.997 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:31:06.997 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.925548+0000", 2024-09-14T06:31:06.997 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000142177 2024-09-14T06:31:06.997 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:06.997 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:31:06.997 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:06.997 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:06.997 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:31:06.997 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-14T06:31:06.998 INFO:teuthology.orchestra.run.smithi094.stdout:osd.6: {} 2024-09-14T06:31:06.998 INFO:teuthology.orchestra.run.smithi094.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-14T06:31:06.999 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-14T06:31:07.000 INFO:teuthology.orchestra.run.smithi094.stdout:osd.7: {} 2024-09-14T06:31:07.000 INFO:teuthology.orchestra.run.smithi094.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-14T06:31:07.001 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-14T06:31:07.001 INFO:teuthology.orchestra.run.smithi094.stdout:4326 read 0x17a7c0b thru 0x17b3cfc (0xc0f2 bytes) 2024-09-14T06:31:07.001 INFO:teuthology.orchestra.run.smithi094.stdout:4328 writesame 0x6589d8c thru 0x658fcc6 data_size 0x1(0x5f3b bytes) 2024-09-14T06:31:07.001 INFO:teuthology.orchestra.run.smithi094.stdout:4330 write 0x258f405 thru 0x259dd92 (0xe98e bytes) 2024-09-14T06:31:07.002 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-14T06:31:07.002 INFO:teuthology.orchestra.run.smithi094.stdout:4331 read 0x15db7a8 thru 0x15e7008 (0xb861 bytes) 2024-09-14T06:31:07.003 INFO:teuthology.orchestra.run.smithi094.stderr:dumped all 2024-09-14T06:31:07.005 INFO:teuthology.orchestra.run.smithi094.stdout:4332 trunc from 0x6a705c7 to 0xc8ea72b 2024-09-14T06:31:07.005 INFO:teuthology.orchestra.run.smithi094.stdout:4333 read 0x839ff92 thru 0x83a657e (0x65ed bytes) 2024-09-14T06:31:07.005 INFO:teuthology.orchestra.run.smithi094.stdout:4334 read 0x16e7dc4 thru 0x16ef111 (0x734e bytes) 2024-09-14T06:31:07.005 INFO:teuthology.orchestra.run.smithi094.stdout:4335 write 0x1507a3e thru 0x150ef9a (0x755d bytes) 2024-09-14T06:31:07.005 INFO:teuthology.orchestra.run.smithi094.stdout:{"pg_ready":true,"pg_map":{"version":209,"stamp":"2024-09-14T06:31:06.595591+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1618174966,"num_objects":1818,"num_object_clones":0,"num_object_copies":3636,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1818,"num_whiteouts":0,"num_read":180134,"num_read_kb":1836730,"num_write":23676,"num_write_kb":1245478,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":222,"num_bytes_recovered":31832958,"num_keys_recovered":33,"num_objects_omap":33,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":16863,"ondisk_log_size":16863,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":512728,"kb_used_data":125820,"kb_used_omap":12,"kb_used_meta":386739,"kb_avail":754461992,"statfs":{"total":773094113280,"available":772569079808,"internally_reserved":0,"allocated":128839680,"data_stored":1204157336,"data_compressed":73188306,"data_compressed_allocated":95780864,"data_compressed_original":1182188226,"omap_allocated":12751,"internal_metadata":396021297},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,3],"upper_bound":8},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-5196235,"num_objects":-8,"num_object_clones":0,"num_object_copies":-16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-8,"num_whiteouts":0,"num_read":13646,"num_read_kb":109161,"num_write":2806,"num_write_kb":62745,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"3.812525"},"pg_stats":[{"pgid":"3.1f","version":"139'570","reported_seq":5977,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-14T06:31:02.812307+0000","last_change":"2024-09-14T06:31:01.919564+0000","last_active":"2024-09-14T06:31:02.812307+0000","last_peered":"2024-09-14T06:31:02.812307+0000","last_clean":"2024-09-14T06:31:02.812307+0000","last_became_active":"2024-09-14T06:29:00.579883+0000","last_became_peered":"2024-09-14T06:29:00.579883+0000","last_unstale":"2024-09-14T06:31:02.812307+0000","last_undegraded":"2024-09-14T06:31:02.812307+0000","last_fullsized":"2024-09-14T06:31:02.812307+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":570,"log_dups_size":0,"ondisk_log_size":570,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T13:40:26.882814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057247699999999995,"stat_sum":{"num_bytes":58022812,"num_objects":58,"num_object_clones":0,"num_object_copies":116,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":58,"num_whiteouts":0,"num_read":6688,"num_read_kb":63306,"num_write":852,"num_write_kb":41563,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"3e","length":"1"},{"start":"40","length":"1"}]},{"pgid":"3.1e","version":"140'371","reported_seq":5310,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-14T06:31:05.680346+0000","last_change":"2024-09-14T06:31:01.917738+0000","last_active":"2024-09-14T06:31:05.680346+0000","last_peered":"2024-09-14T06:31:05.680346+0000","last_clean":"2024-09-14T06:31:05.680346+0000","last_became_active":"2024-09-14T06:29:00.580507+0000","last_became_peered":"2024-09-14T06:29:00.580507+0000","last_unstale":"2024-09-14T06:31:05.680346+0000","last_undegraded":"2024-09-14T06:31:05.680346+0000","last_fullsized":"2024-09-14T06:31:05.680346+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":371,"log_dups_size":0,"ondisk_log_size":371,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T06:52:19.272716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001385049,"stat_sum":{"num_bytes":34201836,"num_objects":48,"num_object_clones":0,"num_object_copies":96,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":48,"num_whiteouts":0,"num_read":4865,"num_read_kb":45167,"num_write":457,"num_write_kb":33834,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"40","length":"1"}]},{"pgid":"3.1d","version":"140'665","reported_seq":6003,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-09-14T06:31:03.818048+0000","last_change":"2024-09-14T06:31:01.938458+0000","last_active":"2024-09-14T06:31:03.818048+0000","last_peered":"2024-09-14T06:31:03.818048+0000","last_clean":"2024-09-14T06:31:03.818048+0000","last_became_active":"2024-09-14T06:29:00.579345+0000","last_became_peered":"2024-09-14T06:29:00.579345+0000","last_unstale":"2024-09-14T06:31:03.818048+0000","last_undegraded":"2024-09-14T06:31:03.818048+0000","last_fullsized":"2024-09-14T06:31:03.818048+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":665,"log_dups_size":0,"ondisk_log_size":665,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T09:07:51.379667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001030907,"stat_sum":{"num_bytes":62352084,"num_objects":64,"num_object_clones":0,"num_object_copies":128,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":6601,"num_read_kb":57568,"num_write":993,"num_write_kb":43462,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"3e","length":"1"},{"start":"40","length":"1"}]},{"pgid":"3.1c","version":"140'414","reported_seq":3494,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-09-14T06:31:03.817872+0000","last_change":"2024-09-14T06:31:01.938170+0000","last_active":"2024-09-14T06:31:03.817872+0000","last_peered":"2024-09-14T06:31:03.817872+0000","last_clean":"2024-09-14T06:31:03.817872+0000","last_became_active":"2024-09-14T06:29:00.579807+0000","last_became_peered":"2024-09-14T06:29:00.579807+0000","last_unstale":"2024-09-14T06:31:03.817872+0000","last_undegraded":"2024-09-14T06:31:03.817872+0000","last_fullsized":"2024-09-14T06:31:03.817872+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":414,"log_dups_size":0,"ondisk_log_size":414,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T08:13:11.154851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000790991,"stat_sum":{"num_bytes":31264888,"num_objects":61,"num_object_clones":0,"num_object_copies":122,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":61,"num_whiteouts":0,"num_read":3427,"num_read_kb":38458,"num_write":511,"num_write_kb":37590,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"3e","length":"1"},{"start":"40","length":"1"}]},{"pgid":"3.1b","version":"138'710","reported_seq":10191,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-14T06:31:02.803922+0000","last_change":"2024-09-14T06:31:01.919794+0000","last_active":"2024-09-14T06:31:02.803922+0000","last_peered":"2024-09-14T06:31:02.803922+0000","last_clean":"2024-09-14T06:31:02.803922+0000","last_became_active":"2024-09-14T06:29:00.586183+0000","last_became_peered":"2024-09-14T06:29:00.586183+0000","last_unstale":"2024-09-14T06:31:02.803922+0000","last_undegraded":"2024-09-14T06:31:02.803922+0000","last_fullsized":"2024-09-14T06:31:02.803922+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":710,"log_dups_size":0,"ondisk_log_size":710,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T08:39:00.851897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00056934900000000003,"stat_sum":{"num_bytes":76806229,"num_objects":73,"num_object_clones":0,"num_object_copies":146,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":73,"num_whiteouts":0,"num_read":11312,"num_read_kb":97647,"num_write":1189,"num_write_kb":47962,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"3e","length":"1"},{"start":"40","length":"1"}]},{"pgid":"3.1a","version":"140'1018","reported_seq":6779,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-09-14T06:31:03.827587+0000","last_change":"2024-09-14T06:31:01.786933+0000","last_active":"2024-09-14T06:31:03.827587+0000","last_peered":"2024-09-14T06:31:03.827587+0000","last_clean":"2024-09-14T06:31:03.827587+0000","last_became_active":"2024-09-14T06:29:00.576206+0000","last_became_peered":"2024-09-14T06:29:00.576206+0000","last_unstale":"2024-09-14T06:31:03.827587+0000","last_undegraded":"2024-09-14T06:31:03.827587+0000","last_fullsized":"2024-09-14T06:31:03.827587+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":1018,"log_dups_size":0,"ondisk_log_size":1018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T17:45:40.066062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015836159999999999,"stat_sum":{"num_bytes":58242654,"num_objects":64,"num_object_clones":0,"num_object_copies":128,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":8702,"num_read_kb":63625,"num_write":1442,"num_write_kb":47110,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"3e","length":"1"},{"start":"40","length":"1"}]},{"pgid":"3.19","version":"139'350","reported_seq":4930,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-09-14T06:31:03.794144+0000","last_change":"2024-09-14T06:31:01.781694+0000","last_active":"2024-09-14T06:31:03.794144+0000","last_peered":"2024-09-14T06:31:03.794144+0000","last_clean":"2024-09-14T06:31:03.794144+0000","last_became_active":"2024-09-14T06:29:00.575639+0000","last_became_peered":"2024-09-14T06:29:00.575639+0000","last_unstale":"2024-09-14T06:31:03.794144+0000","last_undegraded":"2024-09-14T06:31:03.794144+0000","last_fullsized":"2024-09-14T06:31:03.794144+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":350,"log_dups_size":0,"ondisk_log_size":350,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T13:28:04.931296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041605100000000002,"stat_sum":{"num_bytes":56176515,"num_objects":53,"num_object_clones":0,"num_object_copies":106,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":53,"num_whiteouts":0,"num_read":4693,"num_read_kb":61158,"num_write":486,"num_write_kb":38698,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"3e","length":"1"},{"start":"40","length":"1"}]},{"pgid":"3.18","version":"142'313","reported_seq":4760,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-14T06:31:05.611641+0000","last_change":"2024-09-14T06:31:01.916766+0000","last_active":"2024-09-14T06:31:05.611641+0000","last_peered":"2024-09-14T06:31:05.611641+0000","last_clean":"2024-09-14T06:31:05.611641+0000","last_became_active":"2024-09-14T06:29:00.578017+0000","last_became_peered":"2024-09-14T06:29:00.578017+0000","last_unstale":"2024-09-14T06:31:05.611641+0000","last_undegraded":"2024-09-14T06:31:05.611641+0000","last_fullsized":"2024-09-14T06:31:05.611641+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":313,"log_dups_size":0,"ondisk_log_size":313,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T14:59:35.197939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00079564200000000005,"stat_sum":{"num_bytes":59815675,"num_objects":59,"num_object_clones":0,"num_object_copies":118,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":59,"num_whiteouts":0,"num_read":4048,"num_read_kb":71687,"num_write":399,"num_write_kb":43500,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"40","length":"1"}]},{"pgid":"3.17","version":"139'534","reported_seq":7324,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-14T06:31:02.804331+0000","last_change":"2024-09-14T06:31:01.919719+0000","last_active":"2024-09-14T06:31:02.804331+0000","last_peered":"2024-09-14T06:31:02.804331+0000","last_clean":"2024-09-14T06:31:02.804331+0000","last_became_active":"2024-09-14T06:29:00.579714+0000","last_became_peered":"2024-09-14T06:29:00.579714+0000","last_unstale":"2024-09-14T06:31:02.804331+0000","last_undegraded":"2024-09-14T06:31:02.804331+0000","last_fullsized":"2024-09-14T06:31:02.804331+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":534,"log_dups_size":0,"ondisk_log_size":534,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T10:40:37.941470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053653399999999997,"stat_sum":{"num_bytes":65648946,"num_objects":70,"num_object_clones":0,"num_object_copies":140,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":70,"num_whiteouts":0,"num_read":7644,"num_read_kb":62728,"num_write":651,"num_write_kb":40481,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"3e","length":"1"},{"start":"40","length":"1"}]},{"pgid":"3.16","version":"139'643","reported_seq":2622,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-09-14T06:31:03.818021+0000","last_change":"2024-09-14T06:31:01.938564+0000","last_active":"2024-09-14T06:31:03.818021+0000","last_peered":"2024-09-14T06:31:03.818021+0000","last_clean":"2024-09-14T06:31:03.818021+0000","last_became_active":"2024-09-14T06:29:00.580070+0000","last_became_peered":"2024-09-14T06:29:00.580070+0000","last_unstale":"2024-09-14T06:31:03.818021+0000","last_undegraded":"2024-09-14T06:31:03.818021+0000","last_fullsized":"2024-09-14T06:31:03.818021+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":643,"log_dups_size":0,"ondisk_log_size":643,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T06:44:42.650168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011029950000000001,"stat_sum":{"num_bytes":27557728,"num_objects":53,"num_object_clones":0,"num_object_copies":106,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":53,"num_whiteouts":0,"num_read":3691,"num_read_kb":33448,"num_write":1006,"num_write_kb":29437,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"3e","length":"1"},{"start":"40","length":"1"}]},{"pgid":"3.15","version":"139'331","reported_seq":3566,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-09-14T06:31:03.796290+0000","last_change":"2024-09-14T06:31:01.789056+0000","last_active":"2024-09-14T06:31:03.796290+0000","last_peered":"2024-09-14T06:31:03.796290+0000","last_clean":"2024-09-14T06:31:03.796290+0000","last_became_active":"2024-09-14T06:29:00.578451+0000","last_became_peered":"2024-09-14T06:29:00.578451+0000","last_unstale":"2024-09-14T06:31:03.796290+0000","last_undegraded":"2024-09-14T06:31:03.796290+0000","last_fullsized":"2024-09-14T06:31:03.796290+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":331,"log_dups_size":0,"ondisk_log_size":331,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T18:21:37.497290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00079328700000000005,"stat_sum":{"num_bytes":46920598,"num_objects":51,"num_object_clones":0,"num_object_copies":102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":51,"num_whiteouts":0,"num_read":2950,"num_read_kb":61925,"num_write":362,"num_write_kb":44597,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"3e","length":"1"},{"start":"40","length":"1"}]},{"pgid":"3.14","version":"140'832","reported_seq":4798,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-09-14T06:31:03.827308+0000","last_change":"2024-09-14T06:31:01.786854+0000","last_active":"2024-09-14T06:31:03.827308+0000","last_peered":"2024-09-14T06:31:03.827308+0000","last_clean":"2024-09-14T06:31:03.827308+0000","last_became_active":"2024-09-14T06:29:00.582595+0000","last_became_peered":"2024-09-14T06:29:00.582595+0000","last_unstale":"2024-09-14T06:31:03.827308+0000","last_undegraded":"2024-09-14T06:31:03.827308+0000","last_fullsized":"2024-09-14T06:31:03.827308+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":832,"log_dups_size":0,"ondisk_log_size":832,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T10:30:33.822256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001517165,"stat_sum":{"num_bytes":37516262,"num_objects":42,"num_object_clones":0,"num_object_copies":84,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":42,"num_whiteouts":0,"num_read":7814,"num_read_kb":38155,"num_write":1228,"num_write_kb":19414,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"3e","length":"1"},{"start":"40","length":"1"}]},{"pgid":"2.7","version":"0'0","reported_seq":255,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-09-14T06:31:03.792007+0000","last_change":"2024-09-14T06:28:53.625193+0000","last_active":"2024-09-14T06:31:03.792007+0000","last_peered":"2024-09-14T06:31:03.792007+0000","last_clean":"2024-09-14T06:31:03.792007+0000","last_became_active":"2024-09-14T06:28:51.223535+0000","last_became_peered":"2024-09-14T06:28:51.223535+0000","last_unstale":"2024-09-14T06:31:03.792007+0000","last_undegraded":"2024-09-14T06:31:03.792007+0000","last_fullsized":"2024-09-14T06:31:03.792007+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:28:50.205350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:28:50.205350+0000","last_clean_scrub_stamp":"2024-09-14T06:28:50.205350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T18:14:22.822245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00052660300000000005,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"139'332","reported_seq":4025,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-14T06:31:02.805863+0000","last_change":"2024-09-14T06:31:01.918984+0000","last_active":"2024-09-14T06:31:02.805863+0000","last_peered":"2024-09-14T06:31:02.805863+0000","last_clean":"2024-09-14T06:31:02.805863+0000","last_became_active":"2024-09-14T06:29:00.578283+0000","last_became_peered":"2024-09-14T06:29:00.578283+0000","last_unstale":"2024-09-14T06:31:02.805863+0000","last_undegraded":"2024-09-14T06:31:02.805863+0000","last_fullsized":"2024-09-14T06:31:02.805863+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":332,"log_dups_size":0,"ondisk_log_size":332,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T16:11:06.928032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047921699999999999,"stat_sum":{"num_bytes":48379609,"num_objects":50,"num_object_clones":0,"num_object_copies":100,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":50,"num_whiteouts":0,"num_read":3493,"num_read_kb":53267,"num_write":402,"num_write_kb":38350,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"3e","length":"1"},{"start":"40","length":"1"}]},{"pgid":"2.6","version":"0'0","reported_seq":257,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-09-14T06:31:03.794405+0000","last_change":"2024-09-14T06:28:53.226122+0000","last_active":"2024-09-14T06:31:03.794405+0000","last_peered":"2024-09-14T06:31:03.794405+0000","last_clean":"2024-09-14T06:31:03.794405+0000","last_became_active":"2024-09-14T06:28:51.217205+0000","last_became_peered":"2024-09-14T06:28:51.217205+0000","last_unstale":"2024-09-14T06:31:03.794405+0000","last_undegraded":"2024-09-14T06:31:03.794405+0000","last_fullsized":"2024-09-14T06:31:03.794405+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:28:50.205350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:28:50.205350+0000","last_clean_scrub_stamp":"2024-09-14T06:28:50.205350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T15:29:19.563302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065018200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"139'430","reported_seq":3217,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-14T06:31:02.804212+0000","last_change":"2024-09-14T06:31:01.919513+0000","last_active":"2024-09-14T06:31:02.804212+0000","last_peered":"2024-09-14T06:31:02.804212+0000","last_clean":"2024-09-14T06:31:02.804212+0000","last_became_active":"2024-09-14T06:29:56.584268+0000","last_became_peered":"2024-09-14T06:29:56.584268+0000","last_unstale":"2024-09-14T06:31:02.804212+0000","last_undegraded":"2024-09-14T06:31:02.804212+0000","last_fullsized":"2024-09-14T06:31:02.804212+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":430,"log_dups_size":0,"ondisk_log_size":430,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T07:37:44.528082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059362200000000003,"stat_sum":{"num_bytes":37670663,"num_objects":46,"num_object_clones":0,"num_object_copies":92,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":46,"num_whiteouts":0,"num_read":3112,"num_read_kb":33627,"num_write":605,"num_write_kb":29492,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":39,"num_bytes_recovered":1400964,"num_keys_recovered":11,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"3e","length":"1"},{"start":"40","length":"1"}]},{"pgid":"2.5","version":"0'0","reported_seq":257,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-09-14T06:31:03.796442+0000","last_change":"2024-09-14T06:28:53.640035+0000","last_active":"2024-09-14T06:31:03.796442+0000","last_peered":"2024-09-14T06:31:03.796442+0000","last_clean":"2024-09-14T06:31:03.796442+0000","last_became_active":"2024-09-14T06:28:51.308058+0000","last_became_peered":"2024-09-14T06:28:51.308058+0000","last_unstale":"2024-09-14T06:31:03.796442+0000","last_undegraded":"2024-09-14T06:31:03.796442+0000","last_fullsized":"2024-09-14T06:31:03.796442+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:28:50.205350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:28:50.205350+0000","last_clean_scrub_stamp":"2024-09-14T06:28:50.205350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T13:35:48.544057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00037824000000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"139'293","reported_seq":1927,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-09-14T06:31:03.794277+0000","last_change":"2024-09-14T06:31:01.781062+0000","last_active":"2024-09-14T06:31:03.794277+0000","last_peered":"2024-09-14T06:31:03.794277+0000","last_clean":"2024-09-14T06:31:03.794277+0000","last_became_active":"2024-09-14T06:29:00.574895+0000","last_became_peered":"2024-09-14T06:29:00.574895+0000","last_unstale":"2024-09-14T06:31:03.794277+0000","last_undegraded":"2024-09-14T06:31:03.794277+0000","last_fullsized":"2024-09-14T06:31:03.794277+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":293,"log_dups_size":0,"ondisk_log_size":293,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T08:03:28.103577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000430507,"stat_sum":{"num_bytes":22244094,"num_objects":52,"num_object_clones":0,"num_object_copies":104,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":52,"num_whiteouts":0,"num_read":1332,"num_read_kb":30486,"num_write":379,"num_write_kb":31745,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"3e","length":"1"},{"start":"40","length":"1"}]},{"pgid":"2.4","version":"0'0","reported_seq":257,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-09-14T06:31:03.794330+0000","last_change":"2024-09-14T06:28:53.226072+0000","last_active":"2024-09-14T06:31:03.794330+0000","last_peered":"2024-09-14T06:31:03.794330+0000","last_clean":"2024-09-14T06:31:03.794330+0000","last_became_active":"2024-09-14T06:28:51.308072+0000","last_became_peered":"2024-09-14T06:28:51.308072+0000","last_unstale":"2024-09-14T06:31:03.794330+0000","last_undegraded":"2024-09-14T06:31:03.794330+0000","last_fullsized":"2024-09-14T06:31:03.794330+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:28:50.205350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:28:50.205350+0000","last_clean_scrub_stamp":"2024-09-14T06:28:50.205350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T13:15:09.052328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066197399999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"140'647","reported_seq":2751,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-09-14T06:31:03.818081+0000","last_change":"2024-09-14T06:31:01.938776+0000","last_active":"2024-09-14T06:31:03.818081+0000","last_peered":"2024-09-14T06:31:03.818081+0000","last_clean":"2024-09-14T06:31:03.818081+0000","last_became_active":"2024-09-14T06:29:00.581874+0000","last_became_peered":"2024-09-14T06:29:00.581874+0000","last_unstale":"2024-09-14T06:31:03.818081+0000","last_undegraded":"2024-09-14T06:31:03.818081+0000","last_fullsized":"2024-09-14T06:31:03.818081+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":647,"log_dups_size":0,"ondisk_log_size":647,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T17:26:41.671451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001192926,"stat_sum":{"num_bytes":34710717,"num_objects":60,"num_object_clones":0,"num_object_copies":120,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":3084,"num_read_kb":31551,"num_write":974,"num_write_kb":42864,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"3e","length":"1"},{"start":"40","length":"1"}]},{"pgid":"3.3","version":"140'326","reported_seq":9417,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-09-14T06:31:03.794693+0000","last_change":"2024-09-14T06:31:01.785860+0000","last_active":"2024-09-14T06:31:03.794693+0000","last_peered":"2024-09-14T06:31:03.794693+0000","last_clean":"2024-09-14T06:31:03.794693+0000","last_became_active":"2024-09-14T06:29:00.584678+0000","last_became_peered":"2024-09-14T06:29:00.584678+0000","last_unstale":"2024-09-14T06:31:03.794693+0000","last_undegraded":"2024-09-14T06:31:03.794693+0000","last_fullsized":"2024-09-14T06:31:03.794693+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":326,"log_dups_size":0,"ondisk_log_size":326,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T14:15:20.670976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00056711299999999995,"stat_sum":{"num_bytes":61494667,"num_objects":45,"num_object_clones":0,"num_object_copies":90,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":45,"num_whiteouts":0,"num_read":8870,"num_read_kb":80318,"num_write":369,"num_write_kb":44929,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"3e","length":"1"},{"start":"40","length":"1"}]},{"pgid":"2.2","version":"22'2","reported_seq":259,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-09-14T06:31:03.817774+0000","last_change":"2024-09-14T06:28:53.231428+0000","last_active":"2024-09-14T06:31:03.817774+0000","last_peered":"2024-09-14T06:31:03.817774+0000","last_clean":"2024-09-14T06:31:03.817774+0000","last_became_active":"2024-09-14T06:28:51.222937+0000","last_became_peered":"2024-09-14T06:28:51.222937+0000","last_unstale":"2024-09-14T06:31:03.817774+0000","last_undegraded":"2024-09-14T06:31:03.817774+0000","last_fullsized":"2024-09-14T06:31:03.817774+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:28:50.205350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:28:50.205350+0000","last_clean_scrub_stamp":"2024-09-14T06:28:50.205350+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T14:27:37.233976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010072970000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.0","version":"140'758","reported_seq":5892,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-09-14T06:31:03.825791+0000","last_change":"2024-09-14T06:31:01.781154+0000","last_active":"2024-09-14T06:31:03.825791+0000","last_peered":"2024-09-14T06:31:03.825791+0000","last_clean":"2024-09-14T06:31:03.825791+0000","last_became_active":"2024-09-14T06:29:00.574873+0000","last_became_peered":"2024-09-14T06:29:00.574873+0000","last_unstale":"2024-09-14T06:31:03.825791+0000","last_undegraded":"2024-09-14T06:31:03.825791+0000","last_fullsized":"2024-09-14T06:31:03.825791+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":758,"log_dups_size":0,"ondisk_log_size":758,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T14:13:04.398329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065928700000000005,"stat_sum":{"num_bytes":72480427,"num_objects":64,"num_object_clones":0,"num_object_copies":128,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":7228,"num_read_kb":80329,"num_write":1210,"num_write_kb":52343,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"3e","length":"1"},{"start":"40","length":"1"}]},{"pgid":"2.1","version":"0'0","reported_seq":254,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-14T06:31:02.801908+0000","last_change":"2024-09-14T06:28:53.226264+0000","last_active":"2024-09-14T06:31:02.801908+0000","last_peered":"2024-09-14T06:31:02.801908+0000","last_clean":"2024-09-14T06:31:02.801908+0000","last_became_active":"2024-09-14T06:28:51.223252+0000","last_became_peered":"2024-09-14T06:28:51.223252+0000","last_unstale":"2024-09-14T06:31:02.801908+0000","last_undegraded":"2024-09-14T06:31:02.801908+0000","last_fullsized":"2024-09-14T06:31:02.801908+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:28:50.205350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:28:50.205350+0000","last_clean_scrub_stamp":"2024-09-14T06:28:50.205350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T11:56:04.373414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053403100000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1","version":"139'546","reported_seq":8290,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-14T06:31:02.803859+0000","last_change":"2024-09-14T06:31:01.919251+0000","last_active":"2024-09-14T06:31:02.803859+0000","last_peered":"2024-09-14T06:31:02.803859+0000","last_clean":"2024-09-14T06:31:02.803859+0000","last_became_active":"2024-09-14T06:30:57.768238+0000","last_became_peered":"2024-09-14T06:30:57.768238+0000","last_unstale":"2024-09-14T06:31:02.803859+0000","last_undegraded":"2024-09-14T06:31:02.803859+0000","last_fullsized":"2024-09-14T06:31:02.803859+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":546,"log_dups_size":0,"ondisk_log_size":546,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T11:55:01.820297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00048719200000000001,"stat_sum":{"num_bytes":74816705,"num_objects":57,"num_object_clones":0,"num_object_copies":114,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":57,"num_whiteouts":0,"num_read":8355,"num_read_kb":88827,"num_write":685,"num_write_kb":41394,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":183,"num_bytes_recovered":30431994,"num_keys_recovered":22,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"3e","length":"1"},{"start":"40","length":"1"}]},{"pgid":"2.0","version":"0'0","reported_seq":257,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-09-14T06:31:03.796244+0000","last_change":"2024-09-14T06:28:53.640096+0000","last_active":"2024-09-14T06:31:03.796244+0000","last_peered":"2024-09-14T06:31:03.796244+0000","last_clean":"2024-09-14T06:31:03.796244+0000","last_became_active":"2024-09-14T06:28:51.223069+0000","last_became_peered":"2024-09-14T06:28:51.223069+0000","last_unstale":"2024-09-14T06:31:03.796244+0000","last_undegraded":"2024-09-14T06:31:03.796244+0000","last_fullsized":"2024-09-14T06:31:03.796244+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:28:50.205350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:28:50.205350+0000","last_clean_scrub_stamp":"2024-09-14T06:28:50.205350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T15:13:00.471509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00043714600000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"140'551","reported_seq":5428,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-14T06:31:05.611008+0000","last_change":"2024-09-14T06:31:01.917598+0000","last_active":"2024-09-14T06:31:05.611008+0000","last_peered":"2024-09-14T06:31:05.611008+0000","last_clean":"2024-09-14T06:31:05.611008+0000","last_became_active":"2024-09-14T06:29:00.578574+0000","last_became_peered":"2024-09-14T06:29:00.578574+0000","last_unstale":"2024-09-14T06:31:05.611008+0000","last_undegraded":"2024-09-14T06:31:05.611008+0000","last_fullsized":"2024-09-14T06:31:05.611008+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":551,"log_dups_size":0,"ondisk_log_size":551,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T09:15:42.092483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001559862,"stat_sum":{"num_bytes":54652618,"num_objects":73,"num_object_clones":0,"num_object_copies":146,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":73,"num_whiteouts":0,"num_read":5358,"num_read_kb":64175,"num_write":701,"num_write_kb":47159,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"40","length":"1"}]},{"pgid":"2.3","version":"20'1","reported_seq":258,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-09-14T06:31:03.817796+0000","last_change":"2024-09-14T06:28:53.230933+0000","last_active":"2024-09-14T06:31:03.817796+0000","last_peered":"2024-09-14T06:31:03.817796+0000","last_clean":"2024-09-14T06:31:03.817796+0000","last_became_active":"2024-09-14T06:28:51.222874+0000","last_became_peered":"2024-09-14T06:28:51.222874+0000","last_unstale":"2024-09-14T06:31:03.817796+0000","last_undegraded":"2024-09-14T06:31:03.817796+0000","last_fullsized":"2024-09-14T06:31:03.817796+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:28:50.205350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:28:50.205350+0000","last_clean_scrub_stamp":"2024-09-14T06:28:50.205350+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T12:04:25.190171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050808899999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'72","reported_seq":348,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-09-14T06:31:03.796255+0000","last_change":"2024-09-14T06:28:44.839885+0000","last_active":"2024-09-14T06:31:03.796255+0000","last_peered":"2024-09-14T06:31:03.796255+0000","last_clean":"2024-09-14T06:31:03.796255+0000","last_became_active":"2024-09-14T06:28:44.839573+0000","last_became_peered":"2024-09-14T06:28:44.839573+0000","last_unstale":"2024-09-14T06:31:03.796255+0000","last_undegraded":"2024-09-14T06:31:03.796255+0000","last_fullsized":"2024-09-14T06:31:03.796255+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:28:43.828494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:28:43.828494+0000","last_clean_scrub_stamp":"2024-09-14T06:28:43.828494+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T14:41:25.787293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.8","version":"140'442","reported_seq":4757,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-14T06:31:05.611798+0000","last_change":"2024-09-14T06:31:01.917881+0000","last_active":"2024-09-14T06:31:05.611798+0000","last_peered":"2024-09-14T06:31:05.611798+0000","last_clean":"2024-09-14T06:31:05.611798+0000","last_became_active":"2024-09-14T06:29:00.574458+0000","last_became_peered":"2024-09-14T06:29:00.574458+0000","last_unstale":"2024-09-14T06:31:05.611798+0000","last_undegraded":"2024-09-14T06:31:05.611798+0000","last_fullsized":"2024-09-14T06:31:05.611798+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":442,"log_dups_size":0,"ondisk_log_size":442,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T07:38:26.945535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001398652,"stat_sum":{"num_bytes":66464780,"num_objects":63,"num_object_clones":0,"num_object_copies":126,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":63,"num_whiteouts":0,"num_read":4953,"num_read_kb":81152,"num_write":649,"num_write_kb":49436,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"40","length":"1"}]},{"pgid":"3.9","version":"139'387","reported_seq":3354,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-09-14T06:31:03.794620+0000","last_change":"2024-09-14T06:31:01.785787+0000","last_active":"2024-09-14T06:31:03.794620+0000","last_peered":"2024-09-14T06:31:03.794620+0000","last_clean":"2024-09-14T06:31:03.794620+0000","last_became_active":"2024-09-14T06:29:00.573179+0000","last_became_peered":"2024-09-14T06:29:00.573179+0000","last_unstale":"2024-09-14T06:31:03.794620+0000","last_undegraded":"2024-09-14T06:31:03.794620+0000","last_fullsized":"2024-09-14T06:31:03.794620+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":387,"log_dups_size":0,"ondisk_log_size":387,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T10:32:51.378098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049863800000000003,"stat_sum":{"num_bytes":39282201,"num_objects":57,"num_object_clones":0,"num_object_copies":114,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":57,"num_whiteouts":0,"num_read":3301,"num_read_kb":48587,"num_write":561,"num_write_kb":37446,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"3e","length":"1"},{"start":"40","length":"1"}]},{"pgid":"3.a","version":"140'634","reported_seq":3575,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-09-14T06:31:03.791967+0000","last_change":"2024-09-14T06:31:01.783640+0000","last_active":"2024-09-14T06:31:03.791967+0000","last_peered":"2024-09-14T06:31:03.791967+0000","last_clean":"2024-09-14T06:31:03.791967+0000","last_became_active":"2024-09-14T06:29:00.582426+0000","last_became_peered":"2024-09-14T06:29:00.582426+0000","last_unstale":"2024-09-14T06:31:03.791967+0000","last_undegraded":"2024-09-14T06:31:03.791967+0000","last_fullsized":"2024-09-14T06:31:03.791967+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":634,"log_dups_size":0,"ondisk_log_size":634,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T06:36:25.903455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050981300000000002,"stat_sum":{"num_bytes":45710809,"num_objects":60,"num_object_clones":0,"num_object_copies":120,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":4570,"num_read_kb":55310,"num_write":979,"num_write_kb":42837,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"3e","length":"1"},{"start":"40","length":"1"}]},{"pgid":"3.b","version":"140'1292","reported_seq":7203,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-14T06:31:05.680573+0000","last_change":"2024-09-14T06:31:01.917524+0000","last_active":"2024-09-14T06:31:05.680573+0000","last_peered":"2024-09-14T06:31:05.680573+0000","last_clean":"2024-09-14T06:31:05.680573+0000","last_became_active":"2024-09-14T06:29:00.578483+0000","last_became_peered":"2024-09-14T06:29:00.578483+0000","last_unstale":"2024-09-14T06:31:05.680573+0000","last_undegraded":"2024-09-14T06:31:05.680573+0000","last_fullsized":"2024-09-14T06:31:05.680573+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":1292,"log_dups_size":0,"ondisk_log_size":1292,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T17:05:30.509426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015234949999999999,"stat_sum":{"num_bytes":51184528,"num_objects":65,"num_object_clones":0,"num_object_copies":130,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":65,"num_whiteouts":0,"num_read":11740,"num_read_kb":55342,"num_write":2075,"num_write_kb":38276,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"40","length":"1"}]},{"pgid":"3.c","version":"139'360","reported_seq":4250,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-09-14T06:31:03.817969+0000","last_change":"2024-09-14T06:31:01.938213+0000","last_active":"2024-09-14T06:31:03.817969+0000","last_peered":"2024-09-14T06:31:03.817969+0000","last_clean":"2024-09-14T06:31:03.817969+0000","last_became_active":"2024-09-14T06:29:00.583406+0000","last_became_peered":"2024-09-14T06:29:00.583406+0000","last_unstale":"2024-09-14T06:31:03.817969+0000","last_undegraded":"2024-09-14T06:31:03.817969+0000","last_fullsized":"2024-09-14T06:31:03.817969+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":360,"log_dups_size":0,"ondisk_log_size":360,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T09:30:33.072061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00079843100000000005,"stat_sum":{"num_bytes":33915796,"num_objects":47,"num_object_clones":0,"num_object_copies":94,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":47,"num_whiteouts":0,"num_read":3930,"num_read_kb":42748,"num_write":509,"num_write_kb":33232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"3e","length":"1"},{"start":"40","length":"1"}]},{"pgid":"3.d","version":"139'281","reported_seq":6811,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-09-14T06:31:03.796415+0000","last_change":"2024-09-14T06:31:01.789343+0000","last_active":"2024-09-14T06:31:03.796415+0000","last_peered":"2024-09-14T06:31:03.796415+0000","last_clean":"2024-09-14T06:31:03.796415+0000","last_became_active":"2024-09-14T06:29:00.580362+0000","last_became_peered":"2024-09-14T06:29:00.580362+0000","last_unstale":"2024-09-14T06:31:03.796415+0000","last_undegraded":"2024-09-14T06:31:03.796415+0000","last_fullsized":"2024-09-14T06:31:03.796415+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":281,"log_dups_size":0,"ondisk_log_size":281,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T16:37:17.438484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001047269,"stat_sum":{"num_bytes":74646840,"num_objects":56,"num_object_clones":0,"num_object_copies":112,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":56,"num_whiteouts":0,"num_read":6298,"num_read_kb":80413,"num_write":368,"num_write_kb":33200,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"3e","length":"1"},{"start":"40","length":"1"}]},{"pgid":"3.e","version":"139'371","reported_seq":2936,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-09-14T06:31:03.796412+0000","last_change":"2024-09-14T06:31:01.789145+0000","last_active":"2024-09-14T06:31:03.796412+0000","last_peered":"2024-09-14T06:31:03.796412+0000","last_clean":"2024-09-14T06:31:03.796412+0000","last_became_active":"2024-09-14T06:29:00.586374+0000","last_became_peered":"2024-09-14T06:29:00.586374+0000","last_unstale":"2024-09-14T06:31:03.796412+0000","last_undegraded":"2024-09-14T06:31:03.796412+0000","last_fullsized":"2024-09-14T06:31:03.796412+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":371,"log_dups_size":0,"ondisk_log_size":371,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T14:12:45.241905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00087468500000000005,"stat_sum":{"num_bytes":39633036,"num_objects":57,"num_object_clones":0,"num_object_copies":114,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":57,"num_whiteouts":0,"num_read":2526,"num_read_kb":49980,"num_write":530,"num_write_kb":38206,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"3e","length":"1"},{"start":"40","length":"1"}]},{"pgid":"3.f","version":"139'331","reported_seq":4015,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-09-14T06:31:03.796553+0000","last_change":"2024-09-14T06:31:01.789413+0000","last_active":"2024-09-14T06:31:03.796553+0000","last_peered":"2024-09-14T06:31:03.796553+0000","last_clean":"2024-09-14T06:31:03.796553+0000","last_became_active":"2024-09-14T06:29:00.586491+0000","last_became_peered":"2024-09-14T06:29:00.586491+0000","last_unstale":"2024-09-14T06:31:03.796553+0000","last_undegraded":"2024-09-14T06:31:03.796553+0000","last_fullsized":"2024-09-14T06:31:03.796553+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":331,"log_dups_size":0,"ondisk_log_size":331,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T08:00:59.989333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010042199999999999,"stat_sum":{"num_bytes":29838114,"num_objects":39,"num_object_clones":0,"num_object_copies":78,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":39,"num_whiteouts":0,"num_read":4093,"num_read_kb":28190,"num_write":419,"num_write_kb":22967,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"3e","length":"1"},{"start":"40","length":"1"}]},{"pgid":"3.10","version":"139'417","reported_seq":4403,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-09-14T06:31:03.791921+0000","last_change":"2024-09-14T06:31:01.783503+0000","last_active":"2024-09-14T06:31:03.791921+0000","last_peered":"2024-09-14T06:31:03.791921+0000","last_clean":"2024-09-14T06:31:03.791921+0000","last_became_active":"2024-09-14T06:29:00.579539+0000","last_became_peered":"2024-09-14T06:29:00.579539+0000","last_unstale":"2024-09-14T06:31:03.791921+0000","last_undegraded":"2024-09-14T06:31:03.791921+0000","last_fullsized":"2024-09-14T06:31:03.791921+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":417,"log_dups_size":0,"ondisk_log_size":417,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T07:36:51.932049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000393151,"stat_sum":{"num_bytes":35990704,"num_objects":49,"num_object_clones":0,"num_object_copies":98,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":49,"num_whiteouts":0,"num_read":4141,"num_read_kb":29547,"num_write":483,"num_write_kb":31881,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"3e","length":"1"},{"start":"40","length":"1"}]},{"pgid":"3.11","version":"139'480","reported_seq":8533,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-09-14T06:31:03.796259+0000","last_change":"2024-09-14T06:31:01.788774+0000","last_active":"2024-09-14T06:31:03.796259+0000","last_peered":"2024-09-14T06:31:03.796259+0000","last_clean":"2024-09-14T06:31:03.796259+0000","last_became_active":"2024-09-14T06:29:00.586480+0000","last_became_peered":"2024-09-14T06:29:00.586480+0000","last_unstale":"2024-09-14T06:31:03.796259+0000","last_undegraded":"2024-09-14T06:31:03.796259+0000","last_fullsized":"2024-09-14T06:31:03.796259+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":480,"log_dups_size":0,"ondisk_log_size":480,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T10:52:53.094514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070126599999999998,"stat_sum":{"num_bytes":80986002,"num_objects":66,"num_object_clones":0,"num_object_copies":132,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":66,"num_whiteouts":0,"num_read":8209,"num_read_kb":94978,"num_write":593,"num_write_kb":42173,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"3e","length":"1"},{"start":"40","length":"1"}]},{"pgid":"3.12","version":"138'555","reported_seq":6340,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-14T06:31:02.803817+0000","last_change":"2024-09-14T06:31:01.919119+0000","last_active":"2024-09-14T06:31:02.803817+0000","last_peered":"2024-09-14T06:31:02.803817+0000","last_clean":"2024-09-14T06:31:02.803817+0000","last_became_active":"2024-09-14T06:29:00.579826+0000","last_became_peered":"2024-09-14T06:29:00.579826+0000","last_unstale":"2024-09-14T06:31:02.803817+0000","last_undegraded":"2024-09-14T06:31:02.803817+0000","last_fullsized":"2024-09-14T06:31:02.803817+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":555,"log_dups_size":0,"ondisk_log_size":555,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T17:42:34.295152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059906900000000003,"stat_sum":{"num_bytes":40670147,"num_objects":59,"num_object_clones":0,"num_object_copies":118,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":59,"num_whiteouts":0,"num_read":6606,"num_read_kb":55492,"num_write":711,"num_write_kb":37640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"3e","length":"1"},{"start":"40","length":"1"}]},{"pgid":"3.13","version":"140'604","reported_seq":6098,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-09-14T06:31:03.796424+0000","last_change":"2024-09-14T06:31:01.788763+0000","last_active":"2024-09-14T06:31:03.796424+0000","last_peered":"2024-09-14T06:31:03.796424+0000","last_clean":"2024-09-14T06:31:03.796424+0000","last_became_active":"2024-09-14T06:29:00.586520+0000","last_became_peered":"2024-09-14T06:29:00.586520+0000","last_unstale":"2024-09-14T06:31:03.796424+0000","last_undegraded":"2024-09-14T06:31:03.796424+0000","last_fullsized":"2024-09-14T06:31:03.796424+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":604,"log_dups_size":0,"ondisk_log_size":604,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T08:13:00.310311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000653004,"stat_sum":{"num_bytes":58285895,"num_objects":53,"num_object_clones":0,"num_object_copies":106,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":53,"num_whiteouts":0,"num_read":6434,"num_read_kb":57484,"num_write":795,"num_write_kb":40630,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"3e","length":"1"},{"start":"40","length":"1"}]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":1617584579,"num_objects":1814,"num_object_clones":0,"num_object_copies":3628,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1814,"num_whiteouts":0,"num_read":180068,"num_read_kb":1836675,"num_write":23573,"num_write_kb":1243848,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":222,"num_bytes_recovered":31832958,"num_keys_recovered":33,"num_objects_omap":33,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":115662848,"data_stored":1197337077,"data_compressed":73217381,"data_compressed_allocated":95850496,"data_compressed_original":1181705871,"omap_allocated":0,"internal_metadata":0},"log_size":16788,"ondisk_log_size":16788,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":131072,"data_stored":1180736,"data_compressed":65166,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509471,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":65404,"kb_used_data":16168,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94306436,"statfs":{"total":96636764160,"available":96569790464,"internally_reserved":0,"allocated":16556032,"data_stored":151889280,"data_compressed":9086588,"data_compressed_allocated":11857920,"data_compressed_original":148666498,"omap_allocated":1590,"internal_metadata":50395594},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509470,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":41112,"kb_used_data":5760,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94330728,"statfs":{"total":96636764160,"available":96594665472,"internally_reserved":0,"allocated":5898240,"data_stored":45926974,"data_compressed":2607478,"data_compressed_allocated":3481600,"data_compressed_original":44520136,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509472,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":66716,"kb_used_data":17484,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94305124,"statfs":{"total":96636764160,"available":96568446976,"internally_reserved":0,"allocated":17903616,"data_stored":167020011,"data_compressed":10214698,"data_compressed_allocated":13459456,"data_compressed_original":164078015,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509471,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":100176,"kb_used_data":24572,"kb_used_omap":1,"kb_used_meta":75582,"kb_avail":94271664,"statfs":{"total":96636764160,"available":96534183936,"internally_reserved":0,"allocated":25161728,"data_stored":245720594,"data_compressed":15298769,"data_compressed_allocated":19836928,"data_compressed_original":242131945,"omap_allocated":1595,"internal_metadata":77396421},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509472,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":44244,"kb_used_data":13060,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94327596,"statfs":{"total":96636764160,"available":96591458304,"internally_reserved":0,"allocated":13373440,"data_stored":122416967,"data_compressed":7309432,"data_compressed_allocated":9641984,"data_compressed_original":119997180,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,3],"upper_bound":8},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":14,"seq":60129542175,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":42984,"kb_used_data":7636,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94328856,"statfs":{"total":96636764160,"available":96592748544,"internally_reserved":0,"allocated":7819264,"data_stored":68205503,"data_compressed":4215281,"data_compressed_allocated":5509120,"data_compressed_original":66940090,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542174,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":79768,"kb_used_data":22216,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94292072,"statfs":{"total":96636764160,"available":96555081728,"internally_reserved":0,"allocated":22749184,"data_stored":216267424,"data_compressed":13288707,"data_compressed_allocated":17256448,"data_compressed_original":212336133,"omap_allocated":1595,"internal_metadata":58915269},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574880,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":72324,"kb_used_data":18924,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94299516,"statfs":{"total":96636764160,"available":96562704384,"internally_reserved":0,"allocated":19378176,"data_stored":186710583,"data_compressed":11167353,"data_compressed_allocated":14737408,"data_compressed_original":183518229,"omap_allocated":1593,"internal_metadata":54655431},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32583,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32583,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":20926464,"data_stored":214363124,"data_compressed":13222067,"data_compressed_allocated":17162240,"data_compressed_original":211285220,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":11730944,"data_stored":121636225,"data_compressed":7309432,"data_compressed_allocated":9641984,"data_compressed_original":119997180,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":6172672,"data_stored":67415800,"data_compressed":4215281,"data_compressed_allocated":5509120,"data_compressed_original":66940090,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":17899520,"data_stored":187058080,"data_compressed":11295168,"data_compressed_allocated":14942208,"data_compressed_original":184673301,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":23552000,"data_stored":244961740,"data_compressed":15299605,"data_compressed_allocated":19841024,"data_compressed_original":242137922,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":16314368,"data_stored":166290490,"data_compressed":10215549,"data_compressed_allocated":13467648,"data_compressed_original":164086207,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4198400,"data_stored":45072775,"data_compressed":2606276,"data_compressed_allocated":3473408,"data_compressed_original":44492893,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":14868480,"data_stored":150538843,"data_compressed":9054003,"data_compressed_allocated":11812864,"data_compressed_original":148093058,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-14T06:31:07.008 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-14T06:31:07.008 INFO:tasks.ceph.osd.5.smithi138.stderr:2024-09-14T06:31:06.997+0000 7fe13746f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33618) UID: 0 2024-09-14T06:31:07.009 INFO:teuthology.orchestra.run.smithi094.stdout:4336 trunc from 0xc8ea72b to 0xa5ec19c 2024-09-14T06:31:07.016 INFO:teuthology.orchestra.run.smithi094.stdout:4338 punch from 0x5986286 to 0x599187f, (0xb5f9 bytes) 2024-09-14T06:31:07.017 INFO:teuthology.orchestra.run.smithi094.stdout:4339 read 0x4bf436a thru 0x4c00407 (0xc09e bytes) 2024-09-14T06:31:07.017 INFO:teuthology.orchestra.run.smithi094.stdout:4340 punch from 0x6c76d53 to 0x6c79431, (0x26de bytes) 2024-09-14T06:31:07.017 INFO:teuthology.orchestra.run.smithi094.stdout:4341 read 0x9e21d7f thru 0x9e2b456 (0x96d8 bytes) 2024-09-14T06:31:07.017 INFO:teuthology.orchestra.run.smithi094.stdout:4342 clone 33 order 25 su 4194304 sc 2 2024-09-14T06:31:07.100 INFO:tasks.ceph.osd.3.smithi094.stderr:2024-09-14T06:31:07.099+0000 7feeeafe8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33655) UID: 0 2024-09-14T06:31:07.113 INFO:teuthology.orchestra.run.smithi138.stdout:{ 2024-09-14T06:31:07.113 INFO:teuthology.orchestra.run.smithi138.stdout: "ops": [], 2024-09-14T06:31:07.113 INFO:teuthology.orchestra.run.smithi138.stdout: "num_ops": 0 2024-09-14T06:31:07.113 INFO:teuthology.orchestra.run.smithi138.stdout:} 2024-09-14T06:31:07.122 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-14T06:31:07.201 INFO:tasks.ceph.osd.4.smithi138.stderr:2024-09-14T06:31:07.200+0000 7f6c653c8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33615) UID: 0 2024-09-14T06:31:07.225 INFO:teuthology.orchestra.run.smithi138.stdout:{ 2024-09-14T06:31:07.225 INFO:teuthology.orchestra.run.smithi138.stdout: "ops": [], 2024-09-14T06:31:07.225 INFO:teuthology.orchestra.run.smithi138.stdout: "complaint_time": 30, 2024-09-14T06:31:07.225 INFO:teuthology.orchestra.run.smithi138.stdout: "num_blocked_ops": 0 2024-09-14T06:31:07.225 INFO:teuthology.orchestra.run.smithi138.stdout:} 2024-09-14T06:31:07.234 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-14T06:31:07.291 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-14T06:31:07.292 INFO:teuthology.orchestra.run.smithi094.stderr:dumped all 2024-09-14T06:31:07.302 INFO:tasks.ceph.osd.2.smithi094.stderr:2024-09-14T06:31:07.301+0000 7faf3269b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33649) UID: 0 2024-09-14T06:31:07.304 INFO:teuthology.orchestra.run.smithi094.stdout:{"pg_ready":true,"pg_map":{"version":209,"stamp":"2024-09-14T06:31:06.595591+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1618174966,"num_objects":1818,"num_object_clones":0,"num_object_copies":3636,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1818,"num_whiteouts":0,"num_read":180134,"num_read_kb":1836730,"num_write":23676,"num_write_kb":1245478,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":222,"num_bytes_recovered":31832958,"num_keys_recovered":33,"num_objects_omap":33,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":16863,"ondisk_log_size":16863,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":512728,"kb_used_data":125820,"kb_used_omap":12,"kb_used_meta":386739,"kb_avail":754461992,"statfs":{"total":773094113280,"available":772569079808,"internally_reserved":0,"allocated":128839680,"data_stored":1204157336,"data_compressed":73188306,"data_compressed_allocated":95780864,"data_compressed_original":1182188226,"omap_allocated":12751,"internal_metadata":396021297},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,3],"upper_bound":8},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-5196235,"num_objects":-8,"num_object_clones":0,"num_object_copies":-16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-8,"num_whiteouts":0,"num_read":13646,"num_read_kb":109161,"num_write":2806,"num_write_kb":62745,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"3.812525"},"pg_stats":[{"pgid":"3.1f","version":"139'570","reported_seq":5977,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-14T06:31:02.812307+0000","last_change":"2024-09-14T06:31:01.919564+0000","last_active":"2024-09-14T06:31:02.812307+0000","last_peered":"2024-09-14T06:31:02.812307+0000","last_clean":"2024-09-14T06:31:02.812307+0000","last_became_active":"2024-09-14T06:29:00.579883+0000","last_became_peered":"2024-09-14T06:29:00.579883+0000","last_unstale":"2024-09-14T06:31:02.812307+0000","last_undegraded":"2024-09-14T06:31:02.812307+0000","last_fullsized":"2024-09-14T06:31:02.812307+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":570,"log_dups_size":0,"ondisk_log_size":570,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T13:40:26.882814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057247699999999995,"stat_sum":{"num_bytes":58022812,"num_objects":58,"num_object_clones":0,"num_object_copies":116,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":58,"num_whiteouts":0,"num_read":6688,"num_read_kb":63306,"num_write":852,"num_write_kb":41563,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"3e","length":"1"},{"start":"40","length":"1"}]},{"pgid":"3.1e","version":"140'371","reported_seq":5310,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-14T06:31:05.680346+0000","last_change":"2024-09-14T06:31:01.917738+0000","last_active":"2024-09-14T06:31:05.680346+0000","last_peered":"2024-09-14T06:31:05.680346+0000","last_clean":"2024-09-14T06:31:05.680346+0000","last_became_active":"2024-09-14T06:29:00.580507+0000","last_became_peered":"2024-09-14T06:29:00.580507+0000","last_unstale":"2024-09-14T06:31:05.680346+0000","last_undegraded":"2024-09-14T06:31:05.680346+0000","last_fullsized":"2024-09-14T06:31:05.680346+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":371,"log_dups_size":0,"ondisk_log_size":371,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T06:52:19.272716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001385049,"stat_sum":{"num_bytes":34201836,"num_objects":48,"num_object_clones":0,"num_object_copies":96,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":48,"num_whiteouts":0,"num_read":4865,"num_read_kb":45167,"num_write":457,"num_write_kb":33834,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"40","length":"1"}]},{"pgid":"3.1d","version":"140'665","reported_seq":6003,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-09-14T06:31:03.818048+0000","last_change":"2024-09-14T06:31:01.938458+0000","last_active":"2024-09-14T06:31:03.818048+0000","last_peered":"2024-09-14T06:31:03.818048+0000","last_clean":"2024-09-14T06:31:03.818048+0000","last_became_active":"2024-09-14T06:29:00.579345+0000","last_became_peered":"2024-09-14T06:29:00.579345+0000","last_unstale":"2024-09-14T06:31:03.818048+0000","last_undegraded":"2024-09-14T06:31:03.818048+0000","last_fullsized":"2024-09-14T06:31:03.818048+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":665,"log_dups_size":0,"ondisk_log_size":665,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T09:07:51.379667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001030907,"stat_sum":{"num_bytes":62352084,"num_objects":64,"num_object_clones":0,"num_object_copies":128,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":6601,"num_read_kb":57568,"num_write":993,"num_write_kb":43462,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"3e","length":"1"},{"start":"40","length":"1"}]},{"pgid":"3.1c","version":"140'414","reported_seq":3494,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-09-14T06:31:03.817872+0000","last_change":"2024-09-14T06:31:01.938170+0000","last_active":"2024-09-14T06:31:03.817872+0000","last_peered":"2024-09-14T06:31:03.817872+0000","last_clean":"2024-09-14T06:31:03.817872+0000","last_became_active":"2024-09-14T06:29:00.579807+0000","last_became_peered":"2024-09-14T06:29:00.579807+0000","last_unstale":"2024-09-14T06:31:03.817872+0000","last_undegraded":"2024-09-14T06:31:03.817872+0000","last_fullsized":"2024-09-14T06:31:03.817872+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":414,"log_dups_size":0,"ondisk_log_size":414,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T08:13:11.154851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000790991,"stat_sum":{"num_bytes":31264888,"num_objects":61,"num_object_clones":0,"num_object_copies":122,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":61,"num_whiteouts":0,"num_read":3427,"num_read_kb":38458,"num_write":511,"num_write_kb":37590,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"3e","length":"1"},{"start":"40","length":"1"}]},{"pgid":"3.1b","version":"138'710","reported_seq":10191,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-14T06:31:02.803922+0000","last_change":"2024-09-14T06:31:01.919794+0000","last_active":"2024-09-14T06:31:02.803922+0000","last_peered":"2024-09-14T06:31:02.803922+0000","last_clean":"2024-09-14T06:31:02.803922+0000","last_became_active":"2024-09-14T06:29:00.586183+0000","last_became_peered":"2024-09-14T06:29:00.586183+0000","last_unstale":"2024-09-14T06:31:02.803922+0000","last_undegraded":"2024-09-14T06:31:02.803922+0000","last_fullsized":"2024-09-14T06:31:02.803922+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":710,"log_dups_size":0,"ondisk_log_size":710,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T08:39:00.851897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00056934900000000003,"stat_sum":{"num_bytes":76806229,"num_objects":73,"num_object_clones":0,"num_object_copies":146,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":73,"num_whiteouts":0,"num_read":11312,"num_read_kb":97647,"num_write":1189,"num_write_kb":47962,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"3e","length":"1"},{"start":"40","length":"1"}]},{"pgid":"3.1a","version":"140'1018","reported_seq":6779,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-09-14T06:31:03.827587+0000","last_change":"2024-09-14T06:31:01.786933+0000","last_active":"2024-09-14T06:31:03.827587+0000","last_peered":"2024-09-14T06:31:03.827587+0000","last_clean":"2024-09-14T06:31:03.827587+0000","last_became_active":"2024-09-14T06:29:00.576206+0000","last_became_peered":"2024-09-14T06:29:00.576206+0000","last_unstale":"2024-09-14T06:31:03.827587+0000","last_undegraded":"2024-09-14T06:31:03.827587+0000","last_fullsized":"2024-09-14T06:31:03.827587+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":1018,"log_dups_size":0,"ondisk_log_size":1018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T17:45:40.066062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015836159999999999,"stat_sum":{"num_bytes":58242654,"num_objects":64,"num_object_clones":0,"num_object_copies":128,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":8702,"num_read_kb":63625,"num_write":1442,"num_write_kb":47110,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"3e","length":"1"},{"start":"40","length":"1"}]},{"pgid":"3.19","version":"139'350","reported_seq":4930,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-09-14T06:31:03.794144+0000","last_change":"2024-09-14T06:31:01.781694+0000","last_active":"2024-09-14T06:31:03.794144+0000","last_peered":"2024-09-14T06:31:03.794144+0000","last_clean":"2024-09-14T06:31:03.794144+0000","last_became_active":"2024-09-14T06:29:00.575639+0000","last_became_peered":"2024-09-14T06:29:00.575639+0000","last_unstale":"2024-09-14T06:31:03.794144+0000","last_undegraded":"2024-09-14T06:31:03.794144+0000","last_fullsized":"2024-09-14T06:31:03.794144+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":350,"log_dups_size":0,"ondisk_log_size":350,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T13:28:04.931296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041605100000000002,"stat_sum":{"num_bytes":56176515,"num_objects":53,"num_object_clones":0,"num_object_copies":106,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":53,"num_whiteouts":0,"num_read":4693,"num_read_kb":61158,"num_write":486,"num_write_kb":38698,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"3e","length":"1"},{"start":"40","length":"1"}]},{"pgid":"3.18","version":"142'313","reported_seq":4760,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-14T06:31:05.611641+0000","last_change":"2024-09-14T06:31:01.916766+0000","last_active":"2024-09-14T06:31:05.611641+0000","last_peered":"2024-09-14T06:31:05.611641+0000","last_clean":"2024-09-14T06:31:05.611641+0000","last_became_active":"2024-09-14T06:29:00.578017+0000","last_became_peered":"2024-09-14T06:29:00.578017+0000","last_unstale":"2024-09-14T06:31:05.611641+0000","last_undegraded":"2024-09-14T06:31:05.611641+0000","last_fullsized":"2024-09-14T06:31:05.611641+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":313,"log_dups_size":0,"ondisk_log_size":313,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T14:59:35.197939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00079564200000000005,"stat_sum":{"num_bytes":59815675,"num_objects":59,"num_object_clones":0,"num_object_copies":118,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":59,"num_whiteouts":0,"num_read":4048,"num_read_kb":71687,"num_write":399,"num_write_kb":43500,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"40","length":"1"}]},{"pgid":"3.17","version":"139'534","reported_seq":7324,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-14T06:31:02.804331+0000","last_change":"2024-09-14T06:31:01.919719+0000","last_active":"2024-09-14T06:31:02.804331+0000","last_peered":"2024-09-14T06:31:02.804331+0000","last_clean":"2024-09-14T06:31:02.804331+0000","last_became_active":"2024-09-14T06:29:00.579714+0000","last_became_peered":"2024-09-14T06:29:00.579714+0000","last_unstale":"2024-09-14T06:31:02.804331+0000","last_undegraded":"2024-09-14T06:31:02.804331+0000","last_fullsized":"2024-09-14T06:31:02.804331+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":534,"log_dups_size":0,"ondisk_log_size":534,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T10:40:37.941470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053653399999999997,"stat_sum":{"num_bytes":65648946,"num_objects":70,"num_object_clones":0,"num_object_copies":140,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":70,"num_whiteouts":0,"num_read":7644,"num_read_kb":62728,"num_write":651,"num_write_kb":40481,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"3e","length":"1"},{"start":"40","length":"1"}]},{"pgid":"3.16","version":"139'643","reported_seq":2622,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-09-14T06:31:03.818021+0000","last_change":"2024-09-14T06:31:01.938564+0000","last_active":"2024-09-14T06:31:03.818021+0000","last_peered":"2024-09-14T06:31:03.818021+0000","last_clean":"2024-09-14T06:31:03.818021+0000","last_became_active":"2024-09-14T06:29:00.580070+0000","last_became_peered":"2024-09-14T06:29:00.580070+0000","last_unstale":"2024-09-14T06:31:03.818021+0000","last_undegraded":"2024-09-14T06:31:03.818021+0000","last_fullsized":"2024-09-14T06:31:03.818021+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":643,"log_dups_size":0,"ondisk_log_size":643,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T06:44:42.650168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011029950000000001,"stat_sum":{"num_bytes":27557728,"num_objects":53,"num_object_clones":0,"num_object_copies":106,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":53,"num_whiteouts":0,"num_read":3691,"num_read_kb":33448,"num_write":1006,"num_write_kb":29437,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"3e","length":"1"},{"start":"40","length":"1"}]},{"pgid":"3.15","version":"139'331","reported_seq":3566,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-09-14T06:31:03.796290+0000","last_change":"2024-09-14T06:31:01.789056+0000","last_active":"2024-09-14T06:31:03.796290+0000","last_peered":"2024-09-14T06:31:03.796290+0000","last_clean":"2024-09-14T06:31:03.796290+0000","last_became_active":"2024-09-14T06:29:00.578451+0000","last_became_peered":"2024-09-14T06:29:00.578451+0000","last_unstale":"2024-09-14T06:31:03.796290+0000","last_undegraded":"2024-09-14T06:31:03.796290+0000","last_fullsized":"2024-09-14T06:31:03.796290+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":331,"log_dups_size":0,"ondisk_log_size":331,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T18:21:37.497290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00079328700000000005,"stat_sum":{"num_bytes":46920598,"num_objects":51,"num_object_clones":0,"num_object_copies":102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":51,"num_whiteouts":0,"num_read":2950,"num_read_kb":61925,"num_write":362,"num_write_kb":44597,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"3e","length":"1"},{"start":"40","length":"1"}]},{"pgid":"3.14","version":"140'832","reported_seq":4798,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-09-14T06:31:03.827308+0000","last_change":"2024-09-14T06:31:01.786854+0000","last_active":"2024-09-14T06:31:03.827308+0000","last_peered":"2024-09-14T06:31:03.827308+0000","last_clean":"2024-09-14T06:31:03.827308+0000","last_became_active":"2024-09-14T06:29:00.582595+0000","last_became_peered":"2024-09-14T06:29:00.582595+0000","last_unstale":"2024-09-14T06:31:03.827308+0000","last_undegraded":"2024-09-14T06:31:03.827308+0000","last_fullsized":"2024-09-14T06:31:03.827308+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":832,"log_dups_size":0,"ondisk_log_size":832,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T10:30:33.822256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001517165,"stat_sum":{"num_bytes":37516262,"num_objects":42,"num_object_clones":0,"num_object_copies":84,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":42,"num_whiteouts":0,"num_read":7814,"num_read_kb":38155,"num_write":1228,"num_write_kb":19414,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"3e","length":"1"},{"start":"40","length":"1"}]},{"pgid":"2.7","version":"0'0","reported_seq":255,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-09-14T06:31:03.792007+0000","last_change":"2024-09-14T06:28:53.625193+0000","last_active":"2024-09-14T06:31:03.792007+0000","last_peered":"2024-09-14T06:31:03.792007+0000","last_clean":"2024-09-14T06:31:03.792007+0000","last_became_active":"2024-09-14T06:28:51.223535+0000","last_became_peered":"2024-09-14T06:28:51.223535+0000","last_unstale":"2024-09-14T06:31:03.792007+0000","last_undegraded":"2024-09-14T06:31:03.792007+0000","last_fullsized":"2024-09-14T06:31:03.792007+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:28:50.205350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:28:50.205350+0000","last_clean_scrub_stamp":"2024-09-14T06:28:50.205350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T18:14:22.822245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00052660300000000005,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"139'332","reported_seq":4025,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-14T06:31:02.805863+0000","last_change":"2024-09-14T06:31:01.918984+0000","last_active":"2024-09-14T06:31:02.805863+0000","last_peered":"2024-09-14T06:31:02.805863+0000","last_clean":"2024-09-14T06:31:02.805863+0000","last_became_active":"2024-09-14T06:29:00.578283+0000","last_became_peered":"2024-09-14T06:29:00.578283+0000","last_unstale":"2024-09-14T06:31:02.805863+0000","last_undegraded":"2024-09-14T06:31:02.805863+0000","last_fullsized":"2024-09-14T06:31:02.805863+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":332,"log_dups_size":0,"ondisk_log_size":332,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T16:11:06.928032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047921699999999999,"stat_sum":{"num_bytes":48379609,"num_objects":50,"num_object_clones":0,"num_object_copies":100,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":50,"num_whiteouts":0,"num_read":3493,"num_read_kb":53267,"num_write":402,"num_write_kb":38350,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"3e","length":"1"},{"start":"40","length":"1"}]},{"pgid":"2.6","version":"0'0","reported_seq":257,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-09-14T06:31:03.794405+0000","last_change":"2024-09-14T06:28:53.226122+0000","last_active":"2024-09-14T06:31:03.794405+0000","last_peered":"2024-09-14T06:31:03.794405+0000","last_clean":"2024-09-14T06:31:03.794405+0000","last_became_active":"2024-09-14T06:28:51.217205+0000","last_became_peered":"2024-09-14T06:28:51.217205+0000","last_unstale":"2024-09-14T06:31:03.794405+0000","last_undegraded":"2024-09-14T06:31:03.794405+0000","last_fullsized":"2024-09-14T06:31:03.794405+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:28:50.205350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:28:50.205350+0000","last_clean_scrub_stamp":"2024-09-14T06:28:50.205350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T15:29:19.563302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065018200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"139'430","reported_seq":3217,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-14T06:31:02.804212+0000","last_change":"2024-09-14T06:31:01.919513+0000","last_active":"2024-09-14T06:31:02.804212+0000","last_peered":"2024-09-14T06:31:02.804212+0000","last_clean":"2024-09-14T06:31:02.804212+0000","last_became_active":"2024-09-14T06:29:56.584268+0000","last_became_peered":"2024-09-14T06:29:56.584268+0000","last_unstale":"2024-09-14T06:31:02.804212+0000","last_undegraded":"2024-09-14T06:31:02.804212+0000","last_fullsized":"2024-09-14T06:31:02.804212+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":430,"log_dups_size":0,"ondisk_log_size":430,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T07:37:44.528082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059362200000000003,"stat_sum":{"num_bytes":37670663,"num_objects":46,"num_object_clones":0,"num_object_copies":92,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":46,"num_whiteouts":0,"num_read":3112,"num_read_kb":33627,"num_write":605,"num_write_kb":29492,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":39,"num_bytes_recovered":1400964,"num_keys_recovered":11,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"3e","length":"1"},{"start":"40","length":"1"}]},{"pgid":"2.5","version":"0'0","reported_seq":257,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-09-14T06:31:03.796442+0000","last_change":"2024-09-14T06:28:53.640035+0000","last_active":"2024-09-14T06:31:03.796442+0000","last_peered":"2024-09-14T06:31:03.796442+0000","last_clean":"2024-09-14T06:31:03.796442+0000","last_became_active":"2024-09-14T06:28:51.308058+0000","last_became_peered":"2024-09-14T06:28:51.308058+0000","last_unstale":"2024-09-14T06:31:03.796442+0000","last_undegraded":"2024-09-14T06:31:03.796442+0000","last_fullsized":"2024-09-14T06:31:03.796442+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:28:50.205350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:28:50.205350+0000","last_clean_scrub_stamp":"2024-09-14T06:28:50.205350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T13:35:48.544057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00037824000000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"139'293","reported_seq":1927,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-09-14T06:31:03.794277+0000","last_change":"2024-09-14T06:31:01.781062+0000","last_active":"2024-09-14T06:31:03.794277+0000","last_peered":"2024-09-14T06:31:03.794277+0000","last_clean":"2024-09-14T06:31:03.794277+0000","last_became_active":"2024-09-14T06:29:00.574895+0000","last_became_peered":"2024-09-14T06:29:00.574895+0000","last_unstale":"2024-09-14T06:31:03.794277+0000","last_undegraded":"2024-09-14T06:31:03.794277+0000","last_fullsized":"2024-09-14T06:31:03.794277+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":293,"log_dups_size":0,"ondisk_log_size":293,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T08:03:28.103577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000430507,"stat_sum":{"num_bytes":22244094,"num_objects":52,"num_object_clones":0,"num_object_copies":104,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":52,"num_whiteouts":0,"num_read":1332,"num_read_kb":30486,"num_write":379,"num_write_kb":31745,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"3e","length":"1"},{"start":"40","length":"1"}]},{"pgid":"2.4","version":"0'0","reported_seq":257,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-09-14T06:31:03.794330+0000","last_change":"2024-09-14T06:28:53.226072+0000","last_active":"2024-09-14T06:31:03.794330+0000","last_peered":"2024-09-14T06:31:03.794330+0000","last_clean":"2024-09-14T06:31:03.794330+0000","last_became_active":"2024-09-14T06:28:51.308072+0000","last_became_peered":"2024-09-14T06:28:51.308072+0000","last_unstale":"2024-09-14T06:31:03.794330+0000","last_undegraded":"2024-09-14T06:31:03.794330+0000","last_fullsized":"2024-09-14T06:31:03.794330+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:28:50.205350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:28:50.205350+0000","last_clean_scrub_stamp":"2024-09-14T06:28:50.205350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T13:15:09.052328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066197399999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"140'647","reported_seq":2751,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-09-14T06:31:03.818081+0000","last_change":"2024-09-14T06:31:01.938776+0000","last_active":"2024-09-14T06:31:03.818081+0000","last_peered":"2024-09-14T06:31:03.818081+0000","last_clean":"2024-09-14T06:31:03.818081+0000","last_became_active":"2024-09-14T06:29:00.581874+0000","last_became_peered":"2024-09-14T06:29:00.581874+0000","last_unstale":"2024-09-14T06:31:03.818081+0000","last_undegraded":"2024-09-14T06:31:03.818081+0000","last_fullsized":"2024-09-14T06:31:03.818081+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":647,"log_dups_size":0,"ondisk_log_size":647,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T17:26:41.671451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001192926,"stat_sum":{"num_bytes":34710717,"num_objects":60,"num_object_clones":0,"num_object_copies":120,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":3084,"num_read_kb":31551,"num_write":974,"num_write_kb":42864,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"3e","length":"1"},{"start":"40","length":"1"}]},{"pgid":"3.3","version":"140'326","reported_seq":9417,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-09-14T06:31:03.794693+0000","last_change":"2024-09-14T06:31:01.785860+0000","last_active":"2024-09-14T06:31:03.794693+0000","last_peered":"2024-09-14T06:31:03.794693+0000","last_clean":"2024-09-14T06:31:03.794693+0000","last_became_active":"2024-09-14T06:29:00.584678+0000","last_became_peered":"2024-09-14T06:29:00.584678+0000","last_unstale":"2024-09-14T06:31:03.794693+0000","last_undegraded":"2024-09-14T06:31:03.794693+0000","last_fullsized":"2024-09-14T06:31:03.794693+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":326,"log_dups_size":0,"ondisk_log_size":326,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T14:15:20.670976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00056711299999999995,"stat_sum":{"num_bytes":61494667,"num_objects":45,"num_object_clones":0,"num_object_copies":90,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":45,"num_whiteouts":0,"num_read":8870,"num_read_kb":80318,"num_write":369,"num_write_kb":44929,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"3e","length":"1"},{"start":"40","length":"1"}]},{"pgid":"2.2","version":"22'2","reported_seq":259,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-09-14T06:31:03.817774+0000","last_change":"2024-09-14T06:28:53.231428+0000","last_active":"2024-09-14T06:31:03.817774+0000","last_peered":"2024-09-14T06:31:03.817774+0000","last_clean":"2024-09-14T06:31:03.817774+0000","last_became_active":"2024-09-14T06:28:51.222937+0000","last_became_peered":"2024-09-14T06:28:51.222937+0000","last_unstale":"2024-09-14T06:31:03.817774+0000","last_undegraded":"2024-09-14T06:31:03.817774+0000","last_fullsized":"2024-09-14T06:31:03.817774+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:28:50.205350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:28:50.205350+0000","last_clean_scrub_stamp":"2024-09-14T06:28:50.205350+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T14:27:37.233976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010072970000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.0","version":"140'758","reported_seq":5892,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-09-14T06:31:03.825791+0000","last_change":"2024-09-14T06:31:01.781154+0000","last_active":"2024-09-14T06:31:03.825791+0000","last_peered":"2024-09-14T06:31:03.825791+0000","last_clean":"2024-09-14T06:31:03.825791+0000","last_became_active":"2024-09-14T06:29:00.574873+0000","last_became_peered":"2024-09-14T06:29:00.574873+0000","last_unstale":"2024-09-14T06:31:03.825791+0000","last_undegraded":"2024-09-14T06:31:03.825791+0000","last_fullsized":"2024-09-14T06:31:03.825791+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":758,"log_dups_size":0,"ondisk_log_size":758,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T14:13:04.398329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065928700000000005,"stat_sum":{"num_bytes":72480427,"num_objects":64,"num_object_clones":0,"num_object_copies":128,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":7228,"num_read_kb":80329,"num_write":1210,"num_write_kb":52343,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"3e","length":"1"},{"start":"40","length":"1"}]},{"pgid":"2.1","version":"0'0","reported_seq":254,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-14T06:31:02.801908+0000","last_change":"2024-09-14T06:28:53.226264+0000","last_active":"2024-09-14T06:31:02.801908+0000","last_peered":"2024-09-14T06:31:02.801908+0000","last_clean":"2024-09-14T06:31:02.801908+0000","last_became_active":"2024-09-14T06:28:51.223252+0000","last_became_peered":"2024-09-14T06:28:51.223252+0000","last_unstale":"2024-09-14T06:31:02.801908+0000","last_undegraded":"2024-09-14T06:31:02.801908+0000","last_fullsized":"2024-09-14T06:31:02.801908+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:28:50.205350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:28:50.205350+0000","last_clean_scrub_stamp":"2024-09-14T06:28:50.205350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T11:56:04.373414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053403100000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1","version":"139'546","reported_seq":8290,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-14T06:31:02.803859+0000","last_change":"2024-09-14T06:31:01.919251+0000","last_active":"2024-09-14T06:31:02.803859+0000","last_peered":"2024-09-14T06:31:02.803859+0000","last_clean":"2024-09-14T06:31:02.803859+0000","last_became_active":"2024-09-14T06:30:57.768238+0000","last_became_peered":"2024-09-14T06:30:57.768238+0000","last_unstale":"2024-09-14T06:31:02.803859+0000","last_undegraded":"2024-09-14T06:31:02.803859+0000","last_fullsized":"2024-09-14T06:31:02.803859+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":546,"log_dups_size":0,"ondisk_log_size":546,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T11:55:01.820297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00048719200000000001,"stat_sum":{"num_bytes":74816705,"num_objects":57,"num_object_clones":0,"num_object_copies":114,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":57,"num_whiteouts":0,"num_read":8355,"num_read_kb":88827,"num_write":685,"num_write_kb":41394,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":183,"num_bytes_recovered":30431994,"num_keys_recovered":22,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"3e","length":"1"},{"start":"40","length":"1"}]},{"pgid":"2.0","version":"0'0","reported_seq":257,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-09-14T06:31:03.796244+0000","last_change":"2024-09-14T06:28:53.640096+0000","last_active":"2024-09-14T06:31:03.796244+0000","last_peered":"2024-09-14T06:31:03.796244+0000","last_clean":"2024-09-14T06:31:03.796244+0000","last_became_active":"2024-09-14T06:28:51.223069+0000","last_became_peered":"2024-09-14T06:28:51.223069+0000","last_unstale":"2024-09-14T06:31:03.796244+0000","last_undegraded":"2024-09-14T06:31:03.796244+0000","last_fullsized":"2024-09-14T06:31:03.796244+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:28:50.205350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:28:50.205350+0000","last_clean_scrub_stamp":"2024-09-14T06:28:50.205350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T15:13:00.471509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00043714600000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"140'551","reported_seq":5428,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-14T06:31:05.611008+0000","last_change":"2024-09-14T06:31:01.917598+0000","last_active":"2024-09-14T06:31:05.611008+0000","last_peered":"2024-09-14T06:31:05.611008+0000","last_clean":"2024-09-14T06:31:05.611008+0000","last_became_active":"2024-09-14T06:29:00.578574+0000","last_became_peered":"2024-09-14T06:29:00.578574+0000","last_unstale":"2024-09-14T06:31:05.611008+0000","last_undegraded":"2024-09-14T06:31:05.611008+0000","last_fullsized":"2024-09-14T06:31:05.611008+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":551,"log_dups_size":0,"ondisk_log_size":551,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T09:15:42.092483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001559862,"stat_sum":{"num_bytes":54652618,"num_objects":73,"num_object_clones":0,"num_object_copies":146,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":73,"num_whiteouts":0,"num_read":5358,"num_read_kb":64175,"num_write":701,"num_write_kb":47159,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"40","length":"1"}]},{"pgid":"2.3","version":"20'1","reported_seq":258,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-09-14T06:31:03.817796+0000","last_change":"2024-09-14T06:28:53.230933+0000","last_active":"2024-09-14T06:31:03.817796+0000","last_peered":"2024-09-14T06:31:03.817796+0000","last_clean":"2024-09-14T06:31:03.817796+0000","last_became_active":"2024-09-14T06:28:51.222874+0000","last_became_peered":"2024-09-14T06:28:51.222874+0000","last_unstale":"2024-09-14T06:31:03.817796+0000","last_undegraded":"2024-09-14T06:31:03.817796+0000","last_fullsized":"2024-09-14T06:31:03.817796+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:28:50.205350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:28:50.205350+0000","last_clean_scrub_stamp":"2024-09-14T06:28:50.205350+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T12:04:25.190171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050808899999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'72","reported_seq":348,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-09-14T06:31:03.796255+0000","last_change":"2024-09-14T06:28:44.839885+0000","last_active":"2024-09-14T06:31:03.796255+0000","last_peered":"2024-09-14T06:31:03.796255+0000","last_clean":"2024-09-14T06:31:03.796255+0000","last_became_active":"2024-09-14T06:28:44.839573+0000","last_became_peered":"2024-09-14T06:28:44.839573+0000","last_unstale":"2024-09-14T06:31:03.796255+0000","last_undegraded":"2024-09-14T06:31:03.796255+0000","last_fullsized":"2024-09-14T06:31:03.796255+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:28:43.828494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:28:43.828494+0000","last_clean_scrub_stamp":"2024-09-14T06:28:43.828494+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T14:41:25.787293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.8","version":"140'442","reported_seq":4757,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-14T06:31:05.611798+0000","last_change":"2024-09-14T06:31:01.917881+0000","last_active":"2024-09-14T06:31:05.611798+0000","last_peered":"2024-09-14T06:31:05.611798+0000","last_clean":"2024-09-14T06:31:05.611798+0000","last_became_active":"2024-09-14T06:29:00.574458+0000","last_became_peered":"2024-09-14T06:29:00.574458+0000","last_unstale":"2024-09-14T06:31:05.611798+0000","last_undegraded":"2024-09-14T06:31:05.611798+0000","last_fullsized":"2024-09-14T06:31:05.611798+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":442,"log_dups_size":0,"ondisk_log_size":442,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T07:38:26.945535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001398652,"stat_sum":{"num_bytes":66464780,"num_objects":63,"num_object_clones":0,"num_object_copies":126,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":63,"num_whiteouts":0,"num_read":4953,"num_read_kb":81152,"num_write":649,"num_write_kb":49436,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"40","length":"1"}]},{"pgid":"3.9","version":"139'387","reported_seq":3354,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-09-14T06:31:03.794620+0000","last_change":"2024-09-14T06:31:01.785787+0000","last_active":"2024-09-14T06:31:03.794620+0000","last_peered":"2024-09-14T06:31:03.794620+0000","last_clean":"2024-09-14T06:31:03.794620+0000","last_became_active":"2024-09-14T06:29:00.573179+0000","last_became_peered":"2024-09-14T06:29:00.573179+0000","last_unstale":"2024-09-14T06:31:03.794620+0000","last_undegraded":"2024-09-14T06:31:03.794620+0000","last_fullsized":"2024-09-14T06:31:03.794620+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":387,"log_dups_size":0,"ondisk_log_size":387,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T10:32:51.378098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049863800000000003,"stat_sum":{"num_bytes":39282201,"num_objects":57,"num_object_clones":0,"num_object_copies":114,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":57,"num_whiteouts":0,"num_read":3301,"num_read_kb":48587,"num_write":561,"num_write_kb":37446,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"3e","length":"1"},{"start":"40","length":"1"}]},{"pgid":"3.a","version":"140'634","reported_seq":3575,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-09-14T06:31:03.791967+0000","last_change":"2024-09-14T06:31:01.783640+0000","last_active":"2024-09-14T06:31:03.791967+0000","last_peered":"2024-09-14T06:31:03.791967+0000","last_clean":"2024-09-14T06:31:03.791967+0000","last_became_active":"2024-09-14T06:29:00.582426+0000","last_became_peered":"2024-09-14T06:29:00.582426+0000","last_unstale":"2024-09-14T06:31:03.791967+0000","last_undegraded":"2024-09-14T06:31:03.791967+0000","last_fullsized":"2024-09-14T06:31:03.791967+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":634,"log_dups_size":0,"ondisk_log_size":634,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T06:36:25.903455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050981300000000002,"stat_sum":{"num_bytes":45710809,"num_objects":60,"num_object_clones":0,"num_object_copies":120,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":4570,"num_read_kb":55310,"num_write":979,"num_write_kb":42837,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"3e","length":"1"},{"start":"40","length":"1"}]},{"pgid":"3.b","version":"140'1292","reported_seq":7203,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-09-14T06:31:05.680573+0000","last_change":"2024-09-14T06:31:01.917524+0000","last_active":"2024-09-14T06:31:05.680573+0000","last_peered":"2024-09-14T06:31:05.680573+0000","last_clean":"2024-09-14T06:31:05.680573+0000","last_became_active":"2024-09-14T06:29:00.578483+0000","last_became_peered":"2024-09-14T06:29:00.578483+0000","last_unstale":"2024-09-14T06:31:05.680573+0000","last_undegraded":"2024-09-14T06:31:05.680573+0000","last_fullsized":"2024-09-14T06:31:05.680573+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":1292,"log_dups_size":0,"ondisk_log_size":1292,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T17:05:30.509426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015234949999999999,"stat_sum":{"num_bytes":51184528,"num_objects":65,"num_object_clones":0,"num_object_copies":130,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":65,"num_whiteouts":0,"num_read":11740,"num_read_kb":55342,"num_write":2075,"num_write_kb":38276,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"40","length":"1"}]},{"pgid":"3.c","version":"139'360","reported_seq":4250,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-09-14T06:31:03.817969+0000","last_change":"2024-09-14T06:31:01.938213+0000","last_active":"2024-09-14T06:31:03.817969+0000","last_peered":"2024-09-14T06:31:03.817969+0000","last_clean":"2024-09-14T06:31:03.817969+0000","last_became_active":"2024-09-14T06:29:00.583406+0000","last_became_peered":"2024-09-14T06:29:00.583406+0000","last_unstale":"2024-09-14T06:31:03.817969+0000","last_undegraded":"2024-09-14T06:31:03.817969+0000","last_fullsized":"2024-09-14T06:31:03.817969+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":360,"log_dups_size":0,"ondisk_log_size":360,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T09:30:33.072061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00079843100000000005,"stat_sum":{"num_bytes":33915796,"num_objects":47,"num_object_clones":0,"num_object_copies":94,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":47,"num_whiteouts":0,"num_read":3930,"num_read_kb":42748,"num_write":509,"num_write_kb":33232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"3e","length":"1"},{"start":"40","length":"1"}]},{"pgid":"3.d","version":"139'281","reported_seq":6811,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-09-14T06:31:03.796415+0000","last_change":"2024-09-14T06:31:01.789343+0000","last_active":"2024-09-14T06:31:03.796415+0000","last_peered":"2024-09-14T06:31:03.796415+0000","last_clean":"2024-09-14T06:31:03.796415+0000","last_became_active":"2024-09-14T06:29:00.580362+0000","last_became_peered":"2024-09-14T06:29:00.580362+0000","last_unstale":"2024-09-14T06:31:03.796415+0000","last_undegraded":"2024-09-14T06:31:03.796415+0000","last_fullsized":"2024-09-14T06:31:03.796415+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":281,"log_dups_size":0,"ondisk_log_size":281,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T16:37:17.438484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001047269,"stat_sum":{"num_bytes":74646840,"num_objects":56,"num_object_clones":0,"num_object_copies":112,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":56,"num_whiteouts":0,"num_read":6298,"num_read_kb":80413,"num_write":368,"num_write_kb":33200,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"3e","length":"1"},{"start":"40","length":"1"}]},{"pgid":"3.e","version":"139'371","reported_seq":2936,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-09-14T06:31:03.796412+0000","last_change":"2024-09-14T06:31:01.789145+0000","last_active":"2024-09-14T06:31:03.796412+0000","last_peered":"2024-09-14T06:31:03.796412+0000","last_clean":"2024-09-14T06:31:03.796412+0000","last_became_active":"2024-09-14T06:29:00.586374+0000","last_became_peered":"2024-09-14T06:29:00.586374+0000","last_unstale":"2024-09-14T06:31:03.796412+0000","last_undegraded":"2024-09-14T06:31:03.796412+0000","last_fullsized":"2024-09-14T06:31:03.796412+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":371,"log_dups_size":0,"ondisk_log_size":371,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T14:12:45.241905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00087468500000000005,"stat_sum":{"num_bytes":39633036,"num_objects":57,"num_object_clones":0,"num_object_copies":114,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":57,"num_whiteouts":0,"num_read":2526,"num_read_kb":49980,"num_write":530,"num_write_kb":38206,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"3e","length":"1"},{"start":"40","length":"1"}]},{"pgid":"3.f","version":"139'331","reported_seq":4015,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-09-14T06:31:03.796553+0000","last_change":"2024-09-14T06:31:01.789413+0000","last_active":"2024-09-14T06:31:03.796553+0000","last_peered":"2024-09-14T06:31:03.796553+0000","last_clean":"2024-09-14T06:31:03.796553+0000","last_became_active":"2024-09-14T06:29:00.586491+0000","last_became_peered":"2024-09-14T06:29:00.586491+0000","last_unstale":"2024-09-14T06:31:03.796553+0000","last_undegraded":"2024-09-14T06:31:03.796553+0000","last_fullsized":"2024-09-14T06:31:03.796553+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":331,"log_dups_size":0,"ondisk_log_size":331,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T08:00:59.989333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010042199999999999,"stat_sum":{"num_bytes":29838114,"num_objects":39,"num_object_clones":0,"num_object_copies":78,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":39,"num_whiteouts":0,"num_read":4093,"num_read_kb":28190,"num_write":419,"num_write_kb":22967,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"3e","length":"1"},{"start":"40","length":"1"}]},{"pgid":"3.10","version":"139'417","reported_seq":4403,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-09-14T06:31:03.791921+0000","last_change":"2024-09-14T06:31:01.783503+0000","last_active":"2024-09-14T06:31:03.791921+0000","last_peered":"2024-09-14T06:31:03.791921+0000","last_clean":"2024-09-14T06:31:03.791921+0000","last_became_active":"2024-09-14T06:29:00.579539+0000","last_became_peered":"2024-09-14T06:29:00.579539+0000","last_unstale":"2024-09-14T06:31:03.791921+0000","last_undegraded":"2024-09-14T06:31:03.791921+0000","last_fullsized":"2024-09-14T06:31:03.791921+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":417,"log_dups_size":0,"ondisk_log_size":417,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T07:36:51.932049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000393151,"stat_sum":{"num_bytes":35990704,"num_objects":49,"num_object_clones":0,"num_object_copies":98,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":49,"num_whiteouts":0,"num_read":4141,"num_read_kb":29547,"num_write":483,"num_write_kb":31881,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"3e","length":"1"},{"start":"40","length":"1"}]},{"pgid":"3.11","version":"139'480","reported_seq":8533,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-09-14T06:31:03.796259+0000","last_change":"2024-09-14T06:31:01.788774+0000","last_active":"2024-09-14T06:31:03.796259+0000","last_peered":"2024-09-14T06:31:03.796259+0000","last_clean":"2024-09-14T06:31:03.796259+0000","last_became_active":"2024-09-14T06:29:00.586480+0000","last_became_peered":"2024-09-14T06:29:00.586480+0000","last_unstale":"2024-09-14T06:31:03.796259+0000","last_undegraded":"2024-09-14T06:31:03.796259+0000","last_fullsized":"2024-09-14T06:31:03.796259+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":480,"log_dups_size":0,"ondisk_log_size":480,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T10:52:53.094514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070126599999999998,"stat_sum":{"num_bytes":80986002,"num_objects":66,"num_object_clones":0,"num_object_copies":132,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":66,"num_whiteouts":0,"num_read":8209,"num_read_kb":94978,"num_write":593,"num_write_kb":42173,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"3e","length":"1"},{"start":"40","length":"1"}]},{"pgid":"3.12","version":"138'555","reported_seq":6340,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-09-14T06:31:02.803817+0000","last_change":"2024-09-14T06:31:01.919119+0000","last_active":"2024-09-14T06:31:02.803817+0000","last_peered":"2024-09-14T06:31:02.803817+0000","last_clean":"2024-09-14T06:31:02.803817+0000","last_became_active":"2024-09-14T06:29:00.579826+0000","last_became_peered":"2024-09-14T06:29:00.579826+0000","last_unstale":"2024-09-14T06:31:02.803817+0000","last_undegraded":"2024-09-14T06:31:02.803817+0000","last_fullsized":"2024-09-14T06:31:02.803817+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":555,"log_dups_size":0,"ondisk_log_size":555,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T17:42:34.295152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059906900000000003,"stat_sum":{"num_bytes":40670147,"num_objects":59,"num_object_clones":0,"num_object_copies":118,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":59,"num_whiteouts":0,"num_read":6606,"num_read_kb":55492,"num_write":711,"num_write_kb":37640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"3e","length":"1"},{"start":"40","length":"1"}]},{"pgid":"3.13","version":"140'604","reported_seq":6098,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-09-14T06:31:03.796424+0000","last_change":"2024-09-14T06:31:01.788763+0000","last_active":"2024-09-14T06:31:03.796424+0000","last_peered":"2024-09-14T06:31:03.796424+0000","last_clean":"2024-09-14T06:31:03.796424+0000","last_became_active":"2024-09-14T06:29:00.586520+0000","last_became_peered":"2024-09-14T06:29:00.586520+0000","last_unstale":"2024-09-14T06:31:03.796424+0000","last_undegraded":"2024-09-14T06:31:03.796424+0000","last_fullsized":"2024-09-14T06:31:03.796424+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":604,"log_dups_size":0,"ondisk_log_size":604,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T08:13:00.310311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000653004,"stat_sum":{"num_bytes":58285895,"num_objects":53,"num_object_clones":0,"num_object_copies":106,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":53,"num_whiteouts":0,"num_read":6434,"num_read_kb":57484,"num_write":795,"num_write_kb":40630,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"3e","length":"1"},{"start":"40","length":"1"}]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":1617584579,"num_objects":1814,"num_object_clones":0,"num_object_copies":3628,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1814,"num_whiteouts":0,"num_read":180068,"num_read_kb":1836675,"num_write":23573,"num_write_kb":1243848,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":222,"num_bytes_recovered":31832958,"num_keys_recovered":33,"num_objects_omap":33,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":115662848,"data_stored":1197337077,"data_compressed":73217381,"data_compressed_allocated":95850496,"data_compressed_original":1181705871,"omap_allocated":0,"internal_metadata":0},"log_size":16788,"ondisk_log_size":16788,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":131072,"data_stored":1180736,"data_compressed":65166,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509471,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":65404,"kb_used_data":16168,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94306436,"statfs":{"total":96636764160,"available":96569790464,"internally_reserved":0,"allocated":16556032,"data_stored":151889280,"data_compressed":9086588,"data_compressed_allocated":11857920,"data_compressed_original":148666498,"omap_allocated":1590,"internal_metadata":50395594},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509470,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":41112,"kb_used_data":5760,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94330728,"statfs":{"total":96636764160,"available":96594665472,"internally_reserved":0,"allocated":5898240,"data_stored":45926974,"data_compressed":2607478,"data_compressed_allocated":3481600,"data_compressed_original":44520136,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509472,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":66716,"kb_used_data":17484,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94305124,"statfs":{"total":96636764160,"available":96568446976,"internally_reserved":0,"allocated":17903616,"data_stored":167020011,"data_compressed":10214698,"data_compressed_allocated":13459456,"data_compressed_original":164078015,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509471,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":100176,"kb_used_data":24572,"kb_used_omap":1,"kb_used_meta":75582,"kb_avail":94271664,"statfs":{"total":96636764160,"available":96534183936,"internally_reserved":0,"allocated":25161728,"data_stored":245720594,"data_compressed":15298769,"data_compressed_allocated":19836928,"data_compressed_original":242131945,"omap_allocated":1595,"internal_metadata":77396421},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509472,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":44244,"kb_used_data":13060,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94327596,"statfs":{"total":96636764160,"available":96591458304,"internally_reserved":0,"allocated":13373440,"data_stored":122416967,"data_compressed":7309432,"data_compressed_allocated":9641984,"data_compressed_original":119997180,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,3],"upper_bound":8},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":14,"seq":60129542175,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":42984,"kb_used_data":7636,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94328856,"statfs":{"total":96636764160,"available":96592748544,"internally_reserved":0,"allocated":7819264,"data_stored":68205503,"data_compressed":4215281,"data_compressed_allocated":5509120,"data_compressed_original":66940090,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542174,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":79768,"kb_used_data":22216,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94292072,"statfs":{"total":96636764160,"available":96555081728,"internally_reserved":0,"allocated":22749184,"data_stored":216267424,"data_compressed":13288707,"data_compressed_allocated":17256448,"data_compressed_original":212336133,"omap_allocated":1595,"internal_metadata":58915269},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574880,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":72324,"kb_used_data":18924,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94299516,"statfs":{"total":96636764160,"available":96562704384,"internally_reserved":0,"allocated":19378176,"data_stored":186710583,"data_compressed":11167353,"data_compressed_allocated":14737408,"data_compressed_original":183518229,"omap_allocated":1593,"internal_metadata":54655431},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32583,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32583,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":20926464,"data_stored":214363124,"data_compressed":13222067,"data_compressed_allocated":17162240,"data_compressed_original":211285220,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":11730944,"data_stored":121636225,"data_compressed":7309432,"data_compressed_allocated":9641984,"data_compressed_original":119997180,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":6172672,"data_stored":67415800,"data_compressed":4215281,"data_compressed_allocated":5509120,"data_compressed_original":66940090,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":17899520,"data_stored":187058080,"data_compressed":11295168,"data_compressed_allocated":14942208,"data_compressed_original":184673301,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":23552000,"data_stored":244961740,"data_compressed":15299605,"data_compressed_allocated":19841024,"data_compressed_original":242137922,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":16314368,"data_stored":166290490,"data_compressed":10215549,"data_compressed_allocated":13467648,"data_compressed_original":164086207,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4198400,"data_stored":45072775,"data_compressed":2606276,"data_compressed_allocated":3473408,"data_compressed_original":44492893,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":14868480,"data_stored":150538843,"data_compressed":9054003,"data_compressed_allocated":11812864,"data_compressed_original":148093058,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-14T06:31:07.306 INFO:tasks.ceph.ceph_manager.ceph:recovered! 2024-09-14T06:31:07.306 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-14T06:31:07.306 INFO:tasks.thrashosds.thrasher:inject_pause on osd.7 2024-09-14T06:31:07.306 INFO:tasks.thrashosds.thrasher:Testing filestore_inject_stall pause injection for duration 3 2024-09-14T06:31:07.306 INFO:tasks.thrashosds.thrasher:Checking after 0, should_be_down=False 2024-09-14T06:31:07.307 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok config set filestore_inject_stall 3 2024-09-14T06:31:07.381 INFO:teuthology.orchestra.run.smithi138.stdout:{ 2024-09-14T06:31:07.381 INFO:teuthology.orchestra.run.smithi138.stdout: "size": 20, 2024-09-14T06:31:07.381 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 600, 2024-09-14T06:31:07.381 INFO:teuthology.orchestra.run.smithi138.stdout: "ops": [ 2024-09-14T06:31:07.381 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:07.381 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:19328 3.3 3:c34e5e51:::rbd_data.112d92027590.0000000000000024:f [read 1926144~1024 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:31:07.382 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:27.059768+0000", 2024-09-14T06:31:07.382 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 100.319633532, 2024-09-14T06:31:07.382 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046369147999999999, 2024-09-14T06:31:07.382 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:31:07.382 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:31:07.382 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:31:07.382 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:31:07.382 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:31:07.382 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:07.382 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 19328 2024-09-14T06:31:07.382 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:07.383 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:31:07.383 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:07.383 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:31:07.383 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.059768+0000", 2024-09-14T06:31:07.383 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:07.383 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:07.383 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:07.383 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:31:07.383 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.059768+0000", 2024-09-14T06:31:07.383 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:07.383 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:07.384 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:07.384 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:31:07.384 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.059874+0000", 2024-09-14T06:31:07.384 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00010551400000000001 2024-09-14T06:31:07.384 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:07.384 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:07.384 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:31:07.384 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.059877+0000", 2024-09-14T06:31:07.384 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.8279999999999999e-06 2024-09-14T06:31:07.384 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:07.384 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:07.384 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:31:07.384 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.059884+0000", 2024-09-14T06:31:07.384 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.6389999999999992e-06 2024-09-14T06:31:07.385 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:07.385 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:07.385 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:31:07.385 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.059890+0000", 2024-09-14T06:31:07.385 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.49e-06 2024-09-14T06:31:07.385 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:07.385 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:07.385 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:31:07.385 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.105804+0000", 2024-09-14T06:31:07.385 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.045914242000000001 2024-09-14T06:31:07.385 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:07.385 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:07.385 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:31:07.386 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.105886+0000", 2024-09-14T06:31:07.386 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.2305999999999994e-05 2024-09-14T06:31:07.386 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:07.386 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:07.386 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:31:07.386 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.106138+0000", 2024-09-14T06:31:07.386 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.000251129 2024-09-14T06:31:07.386 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:07.386 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:31:07.386 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:07.386 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:07.386 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:07.386 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:19329 3.3 3:c34e5e51:::rbd_data.112d92027590.0000000000000024:f [read 1936384~1024 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:31:07.386 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:27.059899+0000", 2024-09-14T06:31:07.386 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 100.319503009, 2024-09-14T06:31:07.387 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046572122000000001, 2024-09-14T06:31:07.387 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:31:07.387 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:31:07.387 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:31:07.387 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:31:07.387 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:31:07.387 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:07.387 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 19329 2024-09-14T06:31:07.387 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:07.387 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:31:07.387 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:07.387 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:31:07.387 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.059899+0000", 2024-09-14T06:31:07.387 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:07.388 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:07.388 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:07.388 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:31:07.388 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.059899+0000", 2024-09-14T06:31:07.388 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:07.388 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:07.388 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:07.388 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:31:07.388 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062501+0000", 2024-09-14T06:31:07.388 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.0026016419999999999 2024-09-14T06:31:07.388 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:07.388 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:07.388 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:31:07.388 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062514+0000", 2024-09-14T06:31:07.388 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.2947e-05 2024-09-14T06:31:07.389 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:07.389 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:07.389 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:31:07.389 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062523+0000", 2024-09-14T06:31:07.389 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.5519999999999993e-06 2024-09-14T06:31:07.389 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:07.389 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:07.389 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:31:07.389 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062534+0000", 2024-09-14T06:31:07.389 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.1416e-05 2024-09-14T06:31:07.389 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:07.390 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:07.390 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:31:07.390 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.106179+0000", 2024-09-14T06:31:07.390 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.043644049999999997 2024-09-14T06:31:07.390 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:07.390 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:07.390 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:31:07.390 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.106262+0000", 2024-09-14T06:31:07.390 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.3178999999999998e-05 2024-09-14T06:31:07.390 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:07.390 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:07.390 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:31:07.390 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.106471+0000", 2024-09-14T06:31:07.390 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00020933599999999999 2024-09-14T06:31:07.390 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:07.390 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:31:07.391 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:07.391 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:07.391 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:07.391 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:19351 3.3 3:c34e5e51:::rbd_data.112d92027590.0000000000000024:f [read 1159168~1024 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:31:07.391 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:27.062766+0000", 2024-09-14T06:31:07.391 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 100.316635784, 2024-09-14T06:31:07.391 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046099454999999998, 2024-09-14T06:31:07.391 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:31:07.391 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:31:07.391 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:31:07.391 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:31:07.392 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:31:07.392 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:07.392 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 19351 2024-09-14T06:31:07.392 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:07.392 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:31:07.392 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:07.392 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:31:07.392 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062766+0000", 2024-09-14T06:31:07.392 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:07.392 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:07.392 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:07.392 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:31:07.392 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062766+0000", 2024-09-14T06:31:07.392 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:07.393 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:07.393 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:07.393 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:31:07.393 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062766+0000", 2024-09-14T06:31:07.393 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.6e-07 2024-09-14T06:31:07.393 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:07.393 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:07.393 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:31:07.393 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062767+0000", 2024-09-14T06:31:07.393 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.0070000000000001e-06 2024-09-14T06:31:07.393 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:07.393 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:07.393 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:31:07.393 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062785+0000", 2024-09-14T06:31:07.393 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.7884999999999999e-05 2024-09-14T06:31:07.394 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:07.394 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:07.394 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:31:07.394 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062790+0000", 2024-09-14T06:31:07.394 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.0900000000000004e-06 2024-09-14T06:31:07.394 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:07.394 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:07.394 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:31:07.394 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.108735+0000", 2024-09-14T06:31:07.394 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.045944411999999997 2024-09-14T06:31:07.394 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:07.394 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:07.394 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:31:07.394 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.108809+0000", 2024-09-14T06:31:07.394 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.4157000000000002e-05 2024-09-14T06:31:07.395 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:07.395 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:07.395 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:31:07.395 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.108866+0000", 2024-09-14T06:31:07.395 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.6644000000000002e-05 2024-09-14T06:31:07.395 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:07.395 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:31:07.395 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:07.395 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:07.395 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:07.395 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:19352 3.3 3:c34e5e51:::rbd_data.112d92027590.0000000000000024:f [read 1169408~1024 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:31:07.395 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:27.062798+0000", 2024-09-14T06:31:07.395 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 100.316604373, 2024-09-14T06:31:07.395 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046255900000000003, 2024-09-14T06:31:07.395 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:31:07.395 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:31:07.396 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:31:07.396 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:31:07.396 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:31:07.396 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:07.396 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 19352 2024-09-14T06:31:07.396 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:07.396 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:31:07.396 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:07.396 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:31:07.396 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062798+0000", 2024-09-14T06:31:07.396 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:07.396 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:07.396 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:07.396 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:31:07.397 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062798+0000", 2024-09-14T06:31:07.397 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:07.397 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:07.397 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:07.397 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:31:07.397 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062798+0000", 2024-09-14T06:31:07.397 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.37e-07 2024-09-14T06:31:07.397 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:07.397 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:07.397 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:31:07.397 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062800+0000", 2024-09-14T06:31:07.397 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.0269999999999998e-06 2024-09-14T06:31:07.397 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:07.397 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:07.397 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:31:07.398 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062806+0000", 2024-09-14T06:31:07.398 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.6690000000000003e-06 2024-09-14T06:31:07.398 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:07.398 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:07.398 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:31:07.398 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062810+0000", 2024-09-14T06:31:07.398 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.3449999999999999e-06 2024-09-14T06:31:07.398 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:07.398 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:07.398 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:31:07.398 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.108892+0000", 2024-09-14T06:31:07.398 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046081715000000002 2024-09-14T06:31:07.398 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:07.398 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:07.398 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:31:07.398 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.108940+0000", 2024-09-14T06:31:07.399 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.7815999999999998e-05 2024-09-14T06:31:07.399 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:07.399 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:07.399 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:31:07.399 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.109053+0000", 2024-09-14T06:31:07.399 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.000113791 2024-09-14T06:31:07.399 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:07.399 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:31:07.399 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:07.399 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:07.399 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:07.399 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:19353 3.3 3:c34e5e51:::rbd_data.112d92027590.0000000000000024:f [read 1179648~1024 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:31:07.399 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:27.062816+0000", 2024-09-14T06:31:07.399 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 100.316585854, 2024-09-14T06:31:07.400 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046481256999999998, 2024-09-14T06:31:07.400 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:31:07.400 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:31:07.400 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:31:07.400 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:31:07.400 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:31:07.400 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:07.400 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 19353 2024-09-14T06:31:07.400 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:07.400 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:31:07.400 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:07.400 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:31:07.400 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062816+0000", 2024-09-14T06:31:07.400 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:07.401 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:07.401 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:07.401 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:31:07.401 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062816+0000", 2024-09-14T06:31:07.401 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:07.401 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:07.401 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:07.401 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:31:07.401 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062816+0000", 2024-09-14T06:31:07.401 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.2899999999999999e-07 2024-09-14T06:31:07.401 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:07.401 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:07.401 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:31:07.402 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062818+0000", 2024-09-14T06:31:07.402 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.699e-06 2024-09-14T06:31:07.402 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:07.402 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:07.402 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:31:07.402 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062825+0000", 2024-09-14T06:31:07.402 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.878e-06 2024-09-14T06:31:07.402 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:07.402 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:07.402 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:31:07.402 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062830+0000", 2024-09-14T06:31:07.402 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.5410000000000002e-06 2024-09-14T06:31:07.402 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:07.402 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:07.402 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:31:07.403 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.109096+0000", 2024-09-14T06:31:07.403 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046266517 2024-09-14T06:31:07.403 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:07.403 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:07.403 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:31:07.403 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.109190+0000", 2024-09-14T06:31:07.403 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.4000999999999997e-05 2024-09-14T06:31:07.403 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:07.403 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:07.403 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:31:07.403 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.109297+0000", 2024-09-14T06:31:07.403 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.000107292 2024-09-14T06:31:07.403 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:07.403 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:31:07.404 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:07.404 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:07.404 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:07.404 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:19354 3.3 3:c34e5e51:::rbd_data.112d92027590.0000000000000024:f [read 1189888~1024 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:31:07.404 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:27.062842+0000", 2024-09-14T06:31:07.404 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 100.316560399, 2024-09-14T06:31:07.404 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046594257, 2024-09-14T06:31:07.404 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:31:07.404 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:31:07.404 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:31:07.404 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:31:07.404 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:31:07.404 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:07.404 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 19354 2024-09-14T06:31:07.404 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:07.405 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:31:07.405 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:07.405 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:31:07.405 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062842+0000", 2024-09-14T06:31:07.405 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:07.405 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:07.405 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:07.405 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:31:07.405 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062842+0000", 2024-09-14T06:31:07.405 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:07.406 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:07.406 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:07.406 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:31:07.406 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062842+0000", 2024-09-14T06:31:07.406 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.7399999999999999e-07 2024-09-14T06:31:07.406 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:07.406 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:07.406 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:31:07.406 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062859+0000", 2024-09-14T06:31:07.406 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.7010000000000001e-05 2024-09-14T06:31:07.406 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:07.406 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:07.406 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:31:07.406 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062866+0000", 2024-09-14T06:31:07.407 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.9990000000000002e-06 2024-09-14T06:31:07.407 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:07.407 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:07.407 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:31:07.407 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062873+0000", 2024-09-14T06:31:07.407 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.8569999999999997e-06 2024-09-14T06:31:07.407 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:07.407 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:07.407 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:31:07.407 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.109335+0000", 2024-09-14T06:31:07.407 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046461943999999998 2024-09-14T06:31:07.407 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:07.407 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:07.407 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:31:07.407 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.109377+0000", 2024-09-14T06:31:07.408 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.2558999999999997e-05 2024-09-14T06:31:07.408 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:07.408 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:07.408 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:31:07.408 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.109436+0000", 2024-09-14T06:31:07.408 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.8514000000000003e-05 2024-09-14T06:31:07.408 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:07.408 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:31:07.408 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:07.408 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:07.408 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:07.408 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:19355 3.3 3:c34e5e51:::rbd_data.112d92027590.0000000000000024:f [read 1200128~1024 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:31:07.408 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:27.062884+0000", 2024-09-14T06:31:07.408 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 100.316517726, 2024-09-14T06:31:07.408 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046681650999999998, 2024-09-14T06:31:07.409 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:31:07.409 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:31:07.409 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:31:07.409 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:31:07.409 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:31:07.409 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:07.409 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 19355 2024-09-14T06:31:07.409 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:07.409 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:31:07.409 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:07.409 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:31:07.409 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062884+0000", 2024-09-14T06:31:07.409 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:07.410 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:07.410 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:07.410 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:31:07.410 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062884+0000", 2024-09-14T06:31:07.410 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:07.410 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:07.410 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:07.410 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:31:07.410 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062885+0000", 2024-09-14T06:31:07.410 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.8000000000000001e-07 2024-09-14T06:31:07.410 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:07.410 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:07.410 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:31:07.410 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062886+0000", 2024-09-14T06:31:07.411 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.8920000000000001e-06 2024-09-14T06:31:07.411 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:07.411 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:07.411 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:31:07.411 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062892+0000", 2024-09-14T06:31:07.411 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.84e-06 2024-09-14T06:31:07.411 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:07.411 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:07.411 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:31:07.411 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062899+0000", 2024-09-14T06:31:07.411 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.5339999999999996e-06 2024-09-14T06:31:07.411 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:07.411 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:07.411 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:31:07.411 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.109466+0000", 2024-09-14T06:31:07.412 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046566919999999998 2024-09-14T06:31:07.412 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:07.412 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:07.412 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:31:07.412 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.109510+0000", 2024-09-14T06:31:07.412 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.4153000000000002e-05 2024-09-14T06:31:07.412 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:07.412 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:07.412 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:31:07.412 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.109566+0000", 2024-09-14T06:31:07.412 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.5931999999999999e-05 2024-09-14T06:31:07.412 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:07.412 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:31:07.412 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:07.413 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:07.413 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:07.413 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:19356 3.3 3:c34e5e51:::rbd_data.112d92027590.0000000000000024:f [read 1210368~1024 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:31:07.413 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:27.062908+0000", 2024-09-14T06:31:07.413 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 100.316494289, 2024-09-14T06:31:07.413 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046807498000000003, 2024-09-14T06:31:07.413 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:31:07.413 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:31:07.413 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:31:07.413 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:31:07.413 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:31:07.414 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:07.414 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 19356 2024-09-14T06:31:07.414 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:07.414 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:31:07.414 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:07.414 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:31:07.414 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062908+0000", 2024-09-14T06:31:07.414 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:07.414 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:07.414 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:07.414 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:31:07.414 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062908+0000", 2024-09-14T06:31:07.414 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:07.414 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:07.415 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:07.415 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:31:07.415 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062908+0000", 2024-09-14T06:31:07.415 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.5699999999999998e-07 2024-09-14T06:31:07.415 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:07.415 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:07.415 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:31:07.415 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062910+0000", 2024-09-14T06:31:07.415 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.7710000000000001e-06 2024-09-14T06:31:07.415 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:07.415 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:07.415 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:31:07.415 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062920+0000", 2024-09-14T06:31:07.415 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.0654e-05 2024-09-14T06:31:07.416 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:07.416 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:07.416 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:31:07.416 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062931+0000", 2024-09-14T06:31:07.416 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.075e-05 2024-09-14T06:31:07.416 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:07.416 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:07.416 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:31:07.416 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.109595+0000", 2024-09-14T06:31:07.416 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046663713000000002 2024-09-14T06:31:07.416 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:07.416 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:07.416 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:31:07.416 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.109632+0000", 2024-09-14T06:31:07.417 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.7505000000000001e-05 2024-09-14T06:31:07.417 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:07.417 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:07.417 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:31:07.417 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.109715+0000", 2024-09-14T06:31:07.417 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.2748000000000001e-05 2024-09-14T06:31:07.417 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:07.417 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:31:07.417 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:07.418 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:07.418 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:07.418 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:19357 3.3 3:c34e5e51:::rbd_data.112d92027590.0000000000000024:f [read 1220608~1024 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:31:07.418 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:27.062939+0000", 2024-09-14T06:31:07.418 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 100.316462841, 2024-09-14T06:31:07.418 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046920508, 2024-09-14T06:31:07.418 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:31:07.418 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:31:07.418 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:31:07.418 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:31:07.418 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:31:07.418 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:07.418 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 19357 2024-09-14T06:31:07.418 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:07.419 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:31:07.419 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:07.419 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:31:07.419 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062939+0000", 2024-09-14T06:31:07.419 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:07.419 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:07.419 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:07.419 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:31:07.419 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062939+0000", 2024-09-14T06:31:07.419 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:07.419 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:07.419 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:07.419 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:31:07.419 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062940+0000", 2024-09-14T06:31:07.419 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.32e-07 2024-09-14T06:31:07.420 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:07.420 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:07.420 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:31:07.420 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062942+0000", 2024-09-14T06:31:07.420 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.1919999999999999e-06 2024-09-14T06:31:07.420 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:07.420 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:07.420 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:31:07.420 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062948+0000", 2024-09-14T06:31:07.420 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.1149999999999999e-06 2024-09-14T06:31:07.420 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:07.420 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:07.420 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:31:07.421 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062960+0000", 2024-09-14T06:31:07.421 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.1895e-05 2024-09-14T06:31:07.421 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:07.421 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:07.421 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:31:07.421 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.109704+0000", 2024-09-14T06:31:07.421 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046744584999999998 2024-09-14T06:31:07.421 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:07.421 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:07.421 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:31:07.422 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.109777+0000", 2024-09-14T06:31:07.422 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.2282999999999999e-05 2024-09-14T06:31:07.422 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:07.422 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:07.422 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:31:07.422 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.109860+0000", 2024-09-14T06:31:07.422 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.3005999999999998e-05 2024-09-14T06:31:07.422 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:07.422 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:31:07.422 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:07.422 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:07.422 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:07.423 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:19358 3.3 3:c34e5e51:::rbd_data.112d92027590.0000000000000024:f [read 1230848~1024 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:31:07.423 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:27.062967+0000", 2024-09-14T06:31:07.423 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 100.316435211, 2024-09-14T06:31:07.423 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047144471, 2024-09-14T06:31:07.423 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:31:07.423 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:31:07.423 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:31:07.423 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:31:07.423 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:31:07.423 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:07.423 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 19358 2024-09-14T06:31:07.423 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:07.423 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:31:07.423 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:07.423 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:31:07.424 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062967+0000", 2024-09-14T06:31:07.424 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:07.424 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:07.424 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:07.424 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:31:07.424 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062967+0000", 2024-09-14T06:31:07.424 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:07.424 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:07.424 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:07.424 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:31:07.424 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062967+0000", 2024-09-14T06:31:07.424 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.89e-07 2024-09-14T06:31:07.424 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:07.425 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:07.425 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:31:07.425 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062969+0000", 2024-09-14T06:31:07.425 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.9929999999999998e-06 2024-09-14T06:31:07.425 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:07.425 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:07.425 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:31:07.425 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062975+0000", 2024-09-14T06:31:07.425 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.9030000000000001e-06 2024-09-14T06:31:07.425 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:07.425 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:07.425 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:31:07.425 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062979+0000", 2024-09-14T06:31:07.425 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.2250000000000002e-06 2024-09-14T06:31:07.426 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:07.426 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:07.426 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:31:07.426 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.109934+0000", 2024-09-14T06:31:07.426 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046954410000000002 2024-09-14T06:31:07.426 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:07.426 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:07.426 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:31:07.426 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.110007+0000", 2024-09-14T06:31:07.426 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.3829000000000001e-05 2024-09-14T06:31:07.426 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:07.426 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:07.426 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:31:07.426 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.110111+0000", 2024-09-14T06:31:07.427 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00010372199999999999 2024-09-14T06:31:07.427 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:07.427 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:31:07.427 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:07.427 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:07.427 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:07.427 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:19359 3.3 3:c34e5e51:::rbd_data.112d92027590.0000000000000024:f [read 1241088~1024 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:31:07.427 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:27.062987+0000", 2024-09-14T06:31:07.427 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 100.31641512100001, 2024-09-14T06:31:07.427 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047297182, 2024-09-14T06:31:07.427 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:31:07.427 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:31:07.427 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:31:07.427 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:31:07.427 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:31:07.427 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:07.428 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 19359 2024-09-14T06:31:07.428 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:07.428 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:31:07.428 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:07.428 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:31:07.428 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062987+0000", 2024-09-14T06:31:07.428 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:07.428 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:07.428 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:07.428 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:31:07.428 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062987+0000", 2024-09-14T06:31:07.428 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:07.429 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:07.429 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:07.429 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:31:07.429 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062987+0000", 2024-09-14T06:31:07.429 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.2500000000000001e-07 2024-09-14T06:31:07.429 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:07.429 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:07.429 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:31:07.429 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062989+0000", 2024-09-14T06:31:07.429 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.6080000000000001e-06 2024-09-14T06:31:07.429 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:07.429 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:07.429 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:31:07.430 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062994+0000", 2024-09-14T06:31:07.430 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.5520000000000004e-06 2024-09-14T06:31:07.430 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:07.430 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:07.430 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:31:07.430 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062999+0000", 2024-09-14T06:31:07.430 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.3930000000000001e-06 2024-09-14T06:31:07.430 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:07.430 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:07.430 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:31:07.430 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.110142+0000", 2024-09-14T06:31:07.430 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047143154 2024-09-14T06:31:07.430 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:07.430 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:07.430 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:31:07.431 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.110219+0000", 2024-09-14T06:31:07.431 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.6737999999999996e-05 2024-09-14T06:31:07.431 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:07.431 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:07.431 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:31:07.431 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.110284+0000", 2024-09-14T06:31:07.431 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.5412000000000003e-05 2024-09-14T06:31:07.431 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:07.431 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:31:07.431 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:07.431 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:07.431 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:07.431 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:19360 3.3 3:c34e5e51:::rbd_data.112d92027590.0000000000000024:f [read 1251328~1024 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:31:07.431 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:27.063017+0000", 2024-09-14T06:31:07.432 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 100.31638466, 2024-09-14T06:31:07.432 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047453823999999999, 2024-09-14T06:31:07.432 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:31:07.432 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:31:07.432 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:31:07.432 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:31:07.432 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:31:07.432 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:07.432 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 19360 2024-09-14T06:31:07.432 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:07.432 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:31:07.432 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:07.432 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:31:07.432 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.063017+0000", 2024-09-14T06:31:07.432 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:07.433 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:07.433 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:07.433 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:31:07.433 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.063017+0000", 2024-09-14T06:31:07.433 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:07.433 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:07.433 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:07.433 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:31:07.433 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.064977+0000", 2024-09-14T06:31:07.433 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.0019597080000000001 2024-09-14T06:31:07.433 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:07.433 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:07.433 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:31:07.434 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.064987+0000", 2024-09-14T06:31:07.434 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.9620000000000005e-06 2024-09-14T06:31:07.434 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:07.434 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:07.434 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:31:07.434 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065000+0000", 2024-09-14T06:31:07.434 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.3173000000000001e-05 2024-09-14T06:31:07.434 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:07.434 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:07.434 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:31:07.435 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065005+0000", 2024-09-14T06:31:07.435 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.9960000000000001e-06 2024-09-14T06:31:07.435 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:07.435 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:07.435 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:31:07.435 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.110314+0000", 2024-09-14T06:31:07.435 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.04530899 2024-09-14T06:31:07.435 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:07.435 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:07.435 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:31:07.435 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.110398+0000", 2024-09-14T06:31:07.435 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.3539000000000001e-05 2024-09-14T06:31:07.435 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:07.435 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:07.435 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:31:07.436 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.110471+0000", 2024-09-14T06:31:07.436 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.3455999999999996e-05 2024-09-14T06:31:07.436 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:07.436 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:31:07.436 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:07.436 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:07.436 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:07.436 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:19365 3.3 3:c34e5e51:::rbd_data.112d92027590.0000000000000024:f [read 1302528~1024 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:31:07.436 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:27.065058+0000", 2024-09-14T06:31:07.436 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 100.31434399, 2024-09-14T06:31:07.436 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046105526000000001, 2024-09-14T06:31:07.437 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:31:07.437 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:31:07.437 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:31:07.437 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:31:07.437 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:31:07.437 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:07.437 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 19365 2024-09-14T06:31:07.437 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:07.437 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:31:07.437 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:07.437 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:31:07.437 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065058+0000", 2024-09-14T06:31:07.438 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:07.438 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:07.438 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:07.438 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:31:07.438 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065058+0000", 2024-09-14T06:31:07.438 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:07.438 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:07.438 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:07.438 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:31:07.438 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065058+0000", 2024-09-14T06:31:07.438 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.1899999999999999e-07 2024-09-14T06:31:07.439 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:07.439 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:07.439 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:31:07.439 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065059+0000", 2024-09-14T06:31:07.439 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.1299999999999998e-07 2024-09-14T06:31:07.439 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:07.439 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:07.439 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:31:07.439 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065063+0000", 2024-09-14T06:31:07.439 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.1099999999999996e-06 2024-09-14T06:31:07.439 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:07.439 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:07.439 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:31:07.440 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065070+0000", 2024-09-14T06:31:07.440 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.973e-06 2024-09-14T06:31:07.440 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:07.440 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:07.440 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:31:07.440 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.111014+0000", 2024-09-14T06:31:07.440 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.045943373000000003 2024-09-14T06:31:07.440 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:07.440 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:07.441 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:31:07.441 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.111073+0000", 2024-09-14T06:31:07.441 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.8987999999999998e-05 2024-09-14T06:31:07.441 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:07.441 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:07.441 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:31:07.441 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.111163+0000", 2024-09-14T06:31:07.441 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.0950000000000002e-05 2024-09-14T06:31:07.442 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:07.442 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:31:07.442 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:07.442 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:07.442 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:07.442 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:19366 3.3 3:c34e5e51:::rbd_data.112d92027590.0000000000000024:f [read 1312768~1024 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:31:07.442 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:27.065074+0000", 2024-09-14T06:31:07.442 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 100.31432814, 2024-09-14T06:31:07.442 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046289858000000003, 2024-09-14T06:31:07.442 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:31:07.442 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:31:07.442 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:31:07.442 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:31:07.442 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:31:07.442 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:07.443 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 19366 2024-09-14T06:31:07.443 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:07.443 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:31:07.443 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:07.443 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:31:07.444 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065074+0000", 2024-09-14T06:31:07.444 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:07.444 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:07.444 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:07.444 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:31:07.444 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065074+0000", 2024-09-14T06:31:07.445 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:07.445 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:07.445 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:07.445 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:31:07.445 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065074+0000", 2024-09-14T06:31:07.445 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.3900000000000001e-07 2024-09-14T06:31:07.446 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:07.446 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:07.446 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:31:07.446 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065078+0000", 2024-09-14T06:31:07.446 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.3229999999999999e-06 2024-09-14T06:31:07.446 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:07.446 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:07.446 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:31:07.446 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065082+0000", 2024-09-14T06:31:07.446 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.8299999999999998e-06 2024-09-14T06:31:07.446 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:07.446 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:07.446 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:31:07.447 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065085+0000", 2024-09-14T06:31:07.447 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.8660000000000002e-06 2024-09-14T06:31:07.447 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:07.447 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:07.447 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:31:07.447 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.111270+0000", 2024-09-14T06:31:07.447 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046185035999999999 2024-09-14T06:31:07.448 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:07.448 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:07.448 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:31:07.448 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.111311+0000", 2024-09-14T06:31:07.448 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.0935e-05 2024-09-14T06:31:07.448 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:07.448 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:07.448 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:31:07.448 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.111364+0000", 2024-09-14T06:31:07.449 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.2629e-05 2024-09-14T06:31:07.449 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:07.449 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:31:07.449 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:07.449 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:07.449 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:07.449 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:19367 3.3 3:c34e5e51:::rbd_data.112d92027590.0000000000000024:f [read 1323008~1024 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:31:07.449 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:27.065115+0000", 2024-09-14T06:31:07.449 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 100.314286579, 2024-09-14T06:31:07.449 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046407003000000002, 2024-09-14T06:31:07.449 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:31:07.449 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:31:07.449 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:31:07.449 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:31:07.450 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:31:07.450 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:07.450 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 19367 2024-09-14T06:31:07.450 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:07.450 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:31:07.450 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:07.450 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:31:07.450 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065115+0000", 2024-09-14T06:31:07.450 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:07.450 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:07.450 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:07.451 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:31:07.451 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065115+0000", 2024-09-14T06:31:07.451 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:07.451 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:07.451 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:07.451 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:31:07.451 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065116+0000", 2024-09-14T06:31:07.451 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.4999999999999998e-07 2024-09-14T06:31:07.451 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:07.451 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:07.451 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:31:07.451 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065117+0000", 2024-09-14T06:31:07.451 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.189e-06 2024-09-14T06:31:07.451 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:07.452 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:07.452 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:31:07.452 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065122+0000", 2024-09-14T06:31:07.452 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.3709999999999999e-06 2024-09-14T06:31:07.452 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:07.452 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:07.452 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:31:07.452 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065125+0000", 2024-09-14T06:31:07.452 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.8940000000000002e-06 2024-09-14T06:31:07.452 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:07.452 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:07.452 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:31:07.452 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.111402+0000", 2024-09-14T06:31:07.452 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046276645999999998 2024-09-14T06:31:07.453 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:07.453 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:07.453 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:31:07.453 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.111445+0000", 2024-09-14T06:31:07.453 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.3288000000000003e-05 2024-09-14T06:31:07.453 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:07.453 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:07.453 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:31:07.453 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.111522+0000", 2024-09-14T06:31:07.453 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.7164999999999997e-05 2024-09-14T06:31:07.453 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:07.453 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:31:07.453 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:07.453 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:07.453 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:07.454 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:19368 3.3 3:c34e5e51:::rbd_data.112d92027590.0000000000000024:f [read 1333248~1024 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:31:07.454 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:27.065130+0000", 2024-09-14T06:31:07.454 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 100.314272289, 2024-09-14T06:31:07.454 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046507275000000001, 2024-09-14T06:31:07.454 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:31:07.454 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:31:07.454 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:31:07.454 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:31:07.454 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:31:07.454 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:07.454 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 19368 2024-09-14T06:31:07.454 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:07.454 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:31:07.454 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:07.455 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:31:07.455 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065130+0000", 2024-09-14T06:31:07.455 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:07.455 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:07.455 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:07.455 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:31:07.455 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065130+0000", 2024-09-14T06:31:07.455 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:07.455 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:07.455 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:07.455 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:31:07.455 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065130+0000", 2024-09-14T06:31:07.455 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.0800000000000001e-07 2024-09-14T06:31:07.455 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:07.456 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:07.456 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:31:07.456 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065131+0000", 2024-09-14T06:31:07.456 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.8700000000000004e-07 2024-09-14T06:31:07.456 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:07.456 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:07.456 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:31:07.456 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065134+0000", 2024-09-14T06:31:07.456 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.1039999999999998e-06 2024-09-14T06:31:07.456 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:07.456 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:07.456 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:31:07.456 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065136+0000", 2024-09-14T06:31:07.457 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.3949999999999999e-06 2024-09-14T06:31:07.457 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:07.457 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:07.457 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:31:07.457 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.111541+0000", 2024-09-14T06:31:07.457 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046404206000000003 2024-09-14T06:31:07.457 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:07.457 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:07.457 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:31:07.457 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.111582+0000", 2024-09-14T06:31:07.457 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.1051999999999999e-05 2024-09-14T06:31:07.457 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:07.457 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:07.458 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:31:07.458 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.111637+0000", 2024-09-14T06:31:07.458 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.5322999999999999e-05 2024-09-14T06:31:07.458 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:07.458 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:31:07.458 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:07.458 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:07.458 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:07.458 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:19369 3.3 3:c34e5e51:::rbd_data.112d92027590.0000000000000024:f [read 1343488~1024 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:31:07.458 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:27.065139+0000", 2024-09-14T06:31:07.458 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 100.314262472, 2024-09-14T06:31:07.458 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046682169000000003, 2024-09-14T06:31:07.458 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:31:07.458 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:31:07.459 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:31:07.459 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:31:07.459 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:31:07.459 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:07.459 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 19369 2024-09-14T06:31:07.459 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:07.459 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:31:07.459 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:07.459 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:31:07.459 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065139+0000", 2024-09-14T06:31:07.459 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:07.459 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:07.459 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:07.460 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:31:07.460 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065139+0000", 2024-09-14T06:31:07.460 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:07.460 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:07.460 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:07.460 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:31:07.460 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065140+0000", 2024-09-14T06:31:07.460 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.8299999999999998e-07 2024-09-14T06:31:07.460 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:07.460 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:07.460 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:31:07.460 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065141+0000", 2024-09-14T06:31:07.460 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.3900000000000003e-07 2024-09-14T06:31:07.460 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:07.461 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:07.461 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:31:07.461 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065144+0000", 2024-09-14T06:31:07.461 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.7409999999999998e-06 2024-09-14T06:31:07.461 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:07.461 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:07.461 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:31:07.461 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065147+0000", 2024-09-14T06:31:07.461 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.4150000000000002e-06 2024-09-14T06:31:07.461 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:07.461 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:07.461 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:31:07.461 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.111673+0000", 2024-09-14T06:31:07.461 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046526004000000003 2024-09-14T06:31:07.462 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:07.462 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:07.462 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:31:07.462 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.111749+0000", 2024-09-14T06:31:07.462 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.5741999999999994e-05 2024-09-14T06:31:07.462 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:07.462 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:07.462 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:31:07.462 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.111822+0000", 2024-09-14T06:31:07.462 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.3045000000000003e-05 2024-09-14T06:31:07.462 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:07.462 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:31:07.462 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:07.462 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:07.462 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:07.462 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:19370 3.3 3:c34e5e51:::rbd_data.112d92027590.0000000000000024:f [read 1353728~1024 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:31:07.463 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:27.065150+0000", 2024-09-14T06:31:07.463 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 100.314251743, 2024-09-14T06:31:07.463 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046897556999999999, 2024-09-14T06:31:07.463 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:31:07.463 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:31:07.463 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:31:07.463 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:31:07.463 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:31:07.463 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:07.463 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 19370 2024-09-14T06:31:07.463 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:07.463 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:31:07.463 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:07.463 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:31:07.463 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065150+0000", 2024-09-14T06:31:07.464 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:07.464 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:07.464 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:07.464 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:31:07.464 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065150+0000", 2024-09-14T06:31:07.464 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:07.464 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:07.464 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:07.464 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:31:07.464 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065150+0000", 2024-09-14T06:31:07.464 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.0200000000000001e-07 2024-09-14T06:31:07.464 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:07.464 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:07.464 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:31:07.464 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065151+0000", 2024-09-14T06:31:07.465 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.1100000000000004e-07 2024-09-14T06:31:07.465 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:07.465 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:07.465 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:31:07.465 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065154+0000", 2024-09-14T06:31:07.465 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.9639999999999999e-06 2024-09-14T06:31:07.465 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:07.465 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:07.465 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:31:07.465 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065157+0000", 2024-09-14T06:31:07.465 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.4260000000000002e-06 2024-09-14T06:31:07.465 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:07.465 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:07.466 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:31:07.466 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.111861+0000", 2024-09-14T06:31:07.466 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046704268 2024-09-14T06:31:07.466 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:07.466 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:07.466 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:31:07.466 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.111939+0000", 2024-09-14T06:31:07.466 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.8231000000000004e-05 2024-09-14T06:31:07.466 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:07.466 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:07.466 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:31:07.466 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.112048+0000", 2024-09-14T06:31:07.466 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00010855500000000001 2024-09-14T06:31:07.466 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:07.466 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:31:07.467 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:07.467 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:07.467 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:07.467 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:19371 3.3 3:c34e5e51:::rbd_data.112d92027590.0000000000000024:f [read 1363968~1024 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:31:07.467 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:27.065161+0000", 2024-09-14T06:31:07.467 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 100.31424109300001, 2024-09-14T06:31:07.467 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047021944000000003, 2024-09-14T06:31:07.467 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:31:07.467 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:31:07.467 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:31:07.468 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:31:07.468 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:31:07.468 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:07.468 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 19371 2024-09-14T06:31:07.468 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:07.468 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:31:07.468 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:07.468 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:31:07.468 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065161+0000", 2024-09-14T06:31:07.468 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:07.468 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:07.469 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:07.469 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:31:07.469 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065161+0000", 2024-09-14T06:31:07.469 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:07.469 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:07.469 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:07.469 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:31:07.469 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065161+0000", 2024-09-14T06:31:07.469 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.0200000000000001e-07 2024-09-14T06:31:07.469 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:07.469 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:07.469 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:31:07.469 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065166+0000", 2024-09-14T06:31:07.469 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.9869999999999999e-06 2024-09-14T06:31:07.469 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:07.470 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:07.470 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:31:07.470 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065169+0000", 2024-09-14T06:31:07.470 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.3799999999999998e-06 2024-09-14T06:31:07.470 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:07.470 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:07.470 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:31:07.470 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065172+0000", 2024-09-14T06:31:07.470 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.4830000000000002e-06 2024-09-14T06:31:07.470 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:07.470 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:07.470 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:31:07.470 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.112071+0000", 2024-09-14T06:31:07.470 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046899529000000002 2024-09-14T06:31:07.470 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:07.471 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:07.471 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:31:07.471 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.112128+0000", 2024-09-14T06:31:07.471 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.6341999999999997e-05 2024-09-14T06:31:07.471 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:07.471 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:07.471 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:31:07.471 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.112183+0000", 2024-09-14T06:31:07.471 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.5021000000000001e-05 2024-09-14T06:31:07.471 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:07.471 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:31:07.471 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:07.471 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:07.471 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:07.472 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:19372 3.3 3:c34e5e51:::rbd_data.112d92027590.0000000000000024:f [read 1374208~1024 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:31:07.472 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:27.065176+0000", 2024-09-14T06:31:07.472 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 100.314225943, 2024-09-14T06:31:07.472 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047188889999999997, 2024-09-14T06:31:07.472 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:31:07.472 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:31:07.472 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:31:07.472 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:31:07.472 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:31:07.472 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:07.472 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 19372 2024-09-14T06:31:07.472 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:07.472 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:31:07.473 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:07.473 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:31:07.473 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065176+0000", 2024-09-14T06:31:07.473 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:07.473 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:07.473 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:07.473 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:31:07.473 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065176+0000", 2024-09-14T06:31:07.473 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:07.473 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:07.473 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:07.473 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:31:07.474 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.071117+0000", 2024-09-14T06:31:07.474 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.0059404990000000001 2024-09-14T06:31:07.474 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:07.474 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:07.474 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:31:07.474 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.071120+0000", 2024-09-14T06:31:07.474 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.4340000000000001e-06 2024-09-14T06:31:07.474 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:07.474 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:07.474 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:31:07.474 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.071129+0000", 2024-09-14T06:31:07.474 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.8389999999999996e-06 2024-09-14T06:31:07.474 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:07.474 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:07.475 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:31:07.475 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.071134+0000", 2024-09-14T06:31:07.475 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.412e-06 2024-09-14T06:31:07.475 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:07.475 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:07.475 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:31:07.475 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.112221+0000", 2024-09-14T06:31:07.475 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.041086473999999998 2024-09-14T06:31:07.475 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:07.475 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:07.476 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:31:07.476 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.112299+0000", 2024-09-14T06:31:07.476 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.8746999999999997e-05 2024-09-14T06:31:07.476 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:07.476 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:07.476 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:31:07.476 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.112365+0000", 2024-09-14T06:31:07.476 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.5485000000000001e-05 2024-09-14T06:31:07.476 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:07.476 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:31:07.476 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:07.476 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:07.476 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:31:07.476 INFO:teuthology.orchestra.run.smithi138.stdout:} 2024-09-14T06:31:07.478 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-14T06:31:07.479 INFO:teuthology.orchestra.run.smithi138.stdout:{ 2024-09-14T06:31:07.479 INFO:teuthology.orchestra.run.smithi138.stdout: "success": "filestore_inject_stall = '' (not observed, change may require restart) " 2024-09-14T06:31:07.479 INFO:teuthology.orchestra.run.smithi138.stdout:} 2024-09-14T06:31:07.480 INFO:tasks.ceph.osd.2.smithi094.stderr:2024-09-14T06:31:07.476+0000 7faf3269b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33649) UID: 0 2024-09-14T06:31:07.578 INFO:tasks.ceph.osd.4.smithi138.stderr:2024-09-14T06:31:07.577+0000 7f6c653c8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33615) UID: 0 2024-09-14T06:31:07.623 INFO:teuthology.orchestra.run.smithi138.stdout:{ 2024-09-14T06:31:07.623 INFO:teuthology.orchestra.run.smithi138.stdout: "ops": [], 2024-09-14T06:31:07.623 INFO:teuthology.orchestra.run.smithi138.stdout: "num_ops": 0 2024-09-14T06:31:07.624 INFO:teuthology.orchestra.run.smithi138.stdout:} 2024-09-14T06:31:07.632 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-14T06:31:07.716 INFO:tasks.ceph.osd.6.smithi138.stderr:2024-09-14T06:31:07.715+0000 7f8cd8849640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33620) UID: 0 2024-09-14T06:31:07.776 INFO:teuthology.orchestra.run.smithi138.stdout:{ 2024-09-14T06:31:07.776 INFO:teuthology.orchestra.run.smithi138.stdout: "ops": [], 2024-09-14T06:31:07.776 INFO:teuthology.orchestra.run.smithi138.stdout: "complaint_time": 30, 2024-09-14T06:31:07.776 INFO:teuthology.orchestra.run.smithi138.stdout: "num_blocked_ops": 0 2024-09-14T06:31:07.776 INFO:teuthology.orchestra.run.smithi138.stdout:} 2024-09-14T06:31:07.779 INFO:tasks.ceph.osd.3.smithi094.stderr:2024-09-14T06:31:07.778+0000 7feeeafe8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33655) UID: 0 2024-09-14T06:31:07.784 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-14T06:31:07.880 INFO:tasks.ceph.osd.4.smithi138.stderr:2024-09-14T06:31:07.879+0000 7f6c653c8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33615) UID: 0 2024-09-14T06:31:07.930 INFO:teuthology.orchestra.run.smithi138.stdout:{ 2024-09-14T06:31:07.931 INFO:teuthology.orchestra.run.smithi138.stdout: "size": 20, 2024-09-14T06:31:07.931 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 600, 2024-09-14T06:31:07.931 INFO:teuthology.orchestra.run.smithi138.stdout: "ops": [ 2024-09-14T06:31:07.931 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:07.931 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:11453 3.1d 3:bcad7303:::rbd_data.112d92027590.0000000000000001:f [read 94208~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:31:07.931 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:25.654375+0000", 2024-09-14T06:31:07.931 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 102.27460850200001, 2024-09-14T06:31:07.931 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052449318000000002, 2024-09-14T06:31:07.931 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:31:07.931 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:31:07.931 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:31:07.931 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:31:07.932 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:31:07.932 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:07.932 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 11453 2024-09-14T06:31:07.932 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:07.932 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:31:07.932 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:07.932 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:31:07.932 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654375+0000", 2024-09-14T06:31:07.932 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:07.932 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:07.932 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:07.932 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:31:07.932 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654375+0000", 2024-09-14T06:31:07.932 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:07.933 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:07.933 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:07.933 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:31:07.933 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654496+0000", 2024-09-14T06:31:07.933 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00012081 2024-09-14T06:31:07.933 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:07.933 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:07.933 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:31:07.933 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654497+0000", 2024-09-14T06:31:07.933 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.8789999999999999e-06 2024-09-14T06:31:07.933 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:07.933 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:07.933 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:31:07.934 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654501+0000", 2024-09-14T06:31:07.934 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.8999999999999999e-06 2024-09-14T06:31:07.934 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:07.934 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:07.934 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:31:07.934 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654505+0000", 2024-09-14T06:31:07.934 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.2229999999999999e-06 2024-09-14T06:31:07.934 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:07.934 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:07.934 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:31:07.934 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.705917+0000", 2024-09-14T06:31:07.934 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.051412501999999999 2024-09-14T06:31:07.934 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:07.934 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:07.935 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:31:07.935 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.705999+0000", 2024-09-14T06:31:07.935 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.1551999999999994e-05 2024-09-14T06:31:07.935 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:07.935 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:07.935 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:31:07.935 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.706824+0000", 2024-09-14T06:31:07.935 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00082545199999999996 2024-09-14T06:31:07.935 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:07.935 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:31:07.935 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:07.935 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:07.935 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:07.935 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:11454 3.1d 3:bcad7303:::rbd_data.112d92027590.0000000000000001:f [read 104448~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:31:07.936 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:25.654510+0000", 2024-09-14T06:31:07.936 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 102.274473712, 2024-09-14T06:31:07.936 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052610713000000003, 2024-09-14T06:31:07.936 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:31:07.936 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:31:07.936 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:31:07.936 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:31:07.936 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:31:07.936 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:07.937 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 11454 2024-09-14T06:31:07.937 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:07.937 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:31:07.937 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:07.937 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:31:07.937 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654510+0000", 2024-09-14T06:31:07.937 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:07.937 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:07.937 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:07.937 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:31:07.937 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654510+0000", 2024-09-14T06:31:07.937 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:07.937 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:07.937 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:07.938 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:31:07.938 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654825+0000", 2024-09-14T06:31:07.938 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00031522400000000002 2024-09-14T06:31:07.938 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:07.938 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:07.938 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:31:07.938 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654827+0000", 2024-09-14T06:31:07.938 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.9709999999999998e-06 2024-09-14T06:31:07.938 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:07.938 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:07.938 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:31:07.938 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654832+0000", 2024-09-14T06:31:07.938 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.4099999999999999e-06 2024-09-14T06:31:07.938 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:07.939 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:07.939 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:31:07.939 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654835+0000", 2024-09-14T06:31:07.939 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.14e-06 2024-09-14T06:31:07.939 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:07.939 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:07.939 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:31:07.939 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.706851+0000", 2024-09-14T06:31:07.939 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052015761000000001 2024-09-14T06:31:07.939 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:07.939 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:07.939 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:31:07.939 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.706934+0000", 2024-09-14T06:31:07.939 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.2484999999999994e-05 2024-09-14T06:31:07.940 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:07.940 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:07.940 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:31:07.940 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.707120+0000", 2024-09-14T06:31:07.940 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00018672200000000001 2024-09-14T06:31:07.940 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:07.940 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:31:07.940 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:07.940 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:07.940 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:07.940 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:11455 3.1d 3:bcad7303:::rbd_data.112d92027590.0000000000000001:f [read 114688~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:31:07.940 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:25.654841+0000", 2024-09-14T06:31:07.940 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 102.274142383, 2024-09-14T06:31:07.940 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052579898, 2024-09-14T06:31:07.941 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:31:07.941 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:31:07.941 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:31:07.941 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:31:07.941 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:31:07.941 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:07.941 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 11455 2024-09-14T06:31:07.941 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:07.941 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:31:07.941 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:07.941 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:31:07.941 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654841+0000", 2024-09-14T06:31:07.941 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:07.941 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:07.942 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:07.942 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:31:07.942 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654841+0000", 2024-09-14T06:31:07.942 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:07.942 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:07.942 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:07.942 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:31:07.942 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654841+0000", 2024-09-14T06:31:07.942 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.7300000000000002e-07 2024-09-14T06:31:07.942 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:07.942 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:07.942 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:31:07.942 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654845+0000", 2024-09-14T06:31:07.942 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.0520000000000003e-06 2024-09-14T06:31:07.943 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:07.943 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:07.943 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:31:07.943 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654849+0000", 2024-09-14T06:31:07.943 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.0459999999999999e-06 2024-09-14T06:31:07.943 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:07.943 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:07.943 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:31:07.943 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654852+0000", 2024-09-14T06:31:07.943 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.9830000000000001e-06 2024-09-14T06:31:07.943 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:07.943 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:07.943 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:31:07.943 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.707159+0000", 2024-09-14T06:31:07.944 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052306826000000001 2024-09-14T06:31:07.944 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:07.944 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:07.944 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:31:07.944 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.707242+0000", 2024-09-14T06:31:07.944 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.2595000000000005e-05 2024-09-14T06:31:07.944 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:07.944 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:07.944 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:31:07.944 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.707421+0000", 2024-09-14T06:31:07.944 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.000179123 2024-09-14T06:31:07.944 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:07.944 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:31:07.944 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:07.944 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:07.945 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:07.945 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:11456 3.1d 3:bcad7303:::rbd_data.112d92027590.0000000000000001:f [read 124928~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:31:07.945 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:25.654856+0000", 2024-09-14T06:31:07.945 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 102.274127537, 2024-09-14T06:31:07.945 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052801580000000001, 2024-09-14T06:31:07.945 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:31:07.945 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:31:07.945 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:31:07.945 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:31:07.945 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:31:07.945 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:07.945 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 11456 2024-09-14T06:31:07.945 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:07.946 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:31:07.946 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:07.946 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:31:07.946 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654856+0000", 2024-09-14T06:31:07.946 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:07.946 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:07.946 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:07.946 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:31:07.946 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654856+0000", 2024-09-14T06:31:07.946 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:07.946 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:07.946 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:07.946 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:31:07.947 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654856+0000", 2024-09-14T06:31:07.947 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.29e-07 2024-09-14T06:31:07.947 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:07.947 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:07.947 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:31:07.947 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654857+0000", 2024-09-14T06:31:07.947 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.187e-06 2024-09-14T06:31:07.947 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:07.947 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:07.947 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:31:07.947 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654861+0000", 2024-09-14T06:31:07.947 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.3850000000000002e-06 2024-09-14T06:31:07.947 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:07.947 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:07.948 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:31:07.948 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654863+0000", 2024-09-14T06:31:07.948 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.9500000000000001e-06 2024-09-14T06:31:07.948 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:07.948 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:07.948 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:31:07.948 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.707456+0000", 2024-09-14T06:31:07.948 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052592434 2024-09-14T06:31:07.948 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:07.948 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:07.948 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:31:07.948 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.707516+0000", 2024-09-14T06:31:07.948 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.0167999999999997e-05 2024-09-14T06:31:07.949 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:07.949 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:07.949 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:31:07.949 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.707657+0000", 2024-09-14T06:31:07.949 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00014122700000000001 2024-09-14T06:31:07.949 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:07.949 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:31:07.949 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:07.949 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:07.949 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:07.949 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:11457 3.1d 3:bcad7303:::rbd_data.112d92027590.0000000000000001:f [read 135168~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:31:07.949 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:25.654867+0000", 2024-09-14T06:31:07.949 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 102.274116652, 2024-09-14T06:31:07.949 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052921902, 2024-09-14T06:31:07.950 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:31:07.950 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:31:07.950 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:31:07.950 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:31:07.950 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:31:07.950 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:07.950 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 11457 2024-09-14T06:31:07.950 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:07.950 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:31:07.950 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:07.950 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:31:07.950 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654867+0000", 2024-09-14T06:31:07.950 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:07.950 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:07.951 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:07.951 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:31:07.951 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654867+0000", 2024-09-14T06:31:07.951 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:07.951 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:07.951 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:07.951 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:31:07.951 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654972+0000", 2024-09-14T06:31:07.951 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00010584 2024-09-14T06:31:07.951 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:07.951 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:07.951 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:31:07.951 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654975+0000", 2024-09-14T06:31:07.951 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.1169999999999998e-06 2024-09-14T06:31:07.952 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:07.952 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:07.952 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:31:07.952 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654979+0000", 2024-09-14T06:31:07.952 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.1960000000000001e-06 2024-09-14T06:31:07.952 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:07.952 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:07.952 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:31:07.952 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654982+0000", 2024-09-14T06:31:07.952 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.2990000000000001e-06 2024-09-14T06:31:07.952 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:07.952 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:07.952 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:31:07.952 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.707684+0000", 2024-09-14T06:31:07.953 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052701979000000003 2024-09-14T06:31:07.953 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:07.953 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:07.953 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:31:07.953 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.707728+0000", 2024-09-14T06:31:07.953 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.3529999999999998e-05 2024-09-14T06:31:07.953 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:07.953 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:07.953 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:31:07.953 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.707789+0000", 2024-09-14T06:31:07.953 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.0940999999999998e-05 2024-09-14T06:31:07.953 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:07.953 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:31:07.953 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:07.954 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:07.954 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:07.954 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:11458 3.1d 3:bcad7303:::rbd_data.112d92027590.0000000000000001:f [read 145408~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:31:07.954 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:25.654987+0000", 2024-09-14T06:31:07.954 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 102.273996254, 2024-09-14T06:31:07.954 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.05291655, 2024-09-14T06:31:07.954 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:31:07.954 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:31:07.954 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:31:07.954 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:31:07.954 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:31:07.954 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:07.954 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 11458 2024-09-14T06:31:07.954 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:07.955 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:31:07.955 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:07.955 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:31:07.955 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654987+0000", 2024-09-14T06:31:07.955 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:07.955 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:07.955 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:07.955 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:31:07.955 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654987+0000", 2024-09-14T06:31:07.955 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:07.955 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:07.955 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:07.955 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:31:07.955 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655077+0000", 2024-09-14T06:31:07.956 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.0253000000000005e-05 2024-09-14T06:31:07.956 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:07.956 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:07.956 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:31:07.956 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655079+0000", 2024-09-14T06:31:07.956 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.9989999999999998e-06 2024-09-14T06:31:07.956 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:07.956 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:07.956 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:31:07.956 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655083+0000", 2024-09-14T06:31:07.956 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.9280000000000003e-06 2024-09-14T06:31:07.956 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:07.956 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:07.956 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:31:07.957 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655086+0000", 2024-09-14T06:31:07.957 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.056e-06 2024-09-14T06:31:07.957 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:07.957 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:07.957 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:31:07.957 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.707805+0000", 2024-09-14T06:31:07.957 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052718627999999997 2024-09-14T06:31:07.957 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:07.957 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:07.957 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:31:07.957 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.707848+0000", 2024-09-14T06:31:07.957 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.2728999999999997e-05 2024-09-14T06:31:07.957 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:07.957 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:07.958 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:31:07.958 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.707904+0000", 2024-09-14T06:31:07.958 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.5957e-05 2024-09-14T06:31:07.958 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:07.958 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:31:07.958 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:07.958 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:07.958 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:07.958 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:11459 3.1d 3:bcad7303:::rbd_data.112d92027590.0000000000000001:f [read 155648~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:31:07.958 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:25.655091+0000", 2024-09-14T06:31:07.958 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 102.273892246, 2024-09-14T06:31:07.958 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052944242000000002, 2024-09-14T06:31:07.958 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:31:07.958 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:31:07.959 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:31:07.959 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:31:07.959 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:31:07.959 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:07.959 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 11459 2024-09-14T06:31:07.959 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:07.959 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:31:07.959 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:07.959 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:31:07.959 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655091+0000", 2024-09-14T06:31:07.959 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:07.959 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:07.959 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:07.959 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:31:07.960 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655091+0000", 2024-09-14T06:31:07.960 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:07.960 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:07.960 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:07.960 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:31:07.960 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655237+0000", 2024-09-14T06:31:07.960 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00014570100000000001 2024-09-14T06:31:07.960 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:07.960 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:07.960 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:31:07.960 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655239+0000", 2024-09-14T06:31:07.960 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.8670000000000001e-06 2024-09-14T06:31:07.960 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:07.961 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:07.961 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:31:07.961 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655243+0000", 2024-09-14T06:31:07.961 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.9829999999999998e-06 2024-09-14T06:31:07.961 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:07.961 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:07.961 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:31:07.961 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655246+0000", 2024-09-14T06:31:07.961 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.1540000000000002e-06 2024-09-14T06:31:07.961 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:07.961 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:07.961 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:31:07.961 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.707934+0000", 2024-09-14T06:31:07.961 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052688286000000001 2024-09-14T06:31:07.962 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:07.962 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:07.962 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:31:07.962 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.707977+0000", 2024-09-14T06:31:07.962 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.3291999999999998e-05 2024-09-14T06:31:07.962 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:07.962 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:07.962 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:31:07.962 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.708035+0000", 2024-09-14T06:31:07.962 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.7958999999999999e-05 2024-09-14T06:31:07.962 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:07.962 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:31:07.962 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:07.962 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:07.963 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:07.963 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:11460 3.1d 3:bcad7303:::rbd_data.112d92027590.0000000000000001:f [read 165888~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:31:07.963 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:25.655250+0000", 2024-09-14T06:31:07.963 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 102.27373294900001, 2024-09-14T06:31:07.963 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052910698999999999, 2024-09-14T06:31:07.963 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:31:07.963 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:31:07.963 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:31:07.963 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:31:07.963 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:31:07.963 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:07.963 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 11460 2024-09-14T06:31:07.963 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:07.963 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:31:07.963 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:07.964 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:31:07.964 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655250+0000", 2024-09-14T06:31:07.964 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:07.964 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:07.964 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:07.964 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:31:07.964 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655250+0000", 2024-09-14T06:31:07.964 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:07.964 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:07.964 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:07.964 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:31:07.964 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655356+0000", 2024-09-14T06:31:07.964 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00010535900000000001 2024-09-14T06:31:07.965 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:07.965 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:07.965 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:31:07.965 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655357+0000", 2024-09-14T06:31:07.965 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.8080000000000001e-06 2024-09-14T06:31:07.965 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:07.965 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:07.965 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:31:07.965 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655361+0000", 2024-09-14T06:31:07.965 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.861e-06 2024-09-14T06:31:07.965 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:07.965 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:07.965 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:31:07.965 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655364+0000", 2024-09-14T06:31:07.965 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.1089999999999998e-06 2024-09-14T06:31:07.966 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:07.966 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:07.966 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:31:07.966 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.708065+0000", 2024-09-14T06:31:07.966 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052700595000000003 2024-09-14T06:31:07.966 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:07.966 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:07.966 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:31:07.966 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.708108+0000", 2024-09-14T06:31:07.966 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.2484999999999997e-05 2024-09-14T06:31:07.966 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:07.966 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:07.966 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:31:07.966 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.708161+0000", 2024-09-14T06:31:07.967 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.3482000000000001e-05 2024-09-14T06:31:07.967 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:07.967 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:31:07.967 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:07.967 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:07.967 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:07.967 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:11461 3.1d 3:bcad7303:::rbd_data.112d92027590.0000000000000001:f [read 176128~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:31:07.967 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:25.655370+0000", 2024-09-14T06:31:07.967 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 102.27361366700001, 2024-09-14T06:31:07.967 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052915141999999998, 2024-09-14T06:31:07.967 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:31:07.967 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:31:07.967 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:31:07.967 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:31:07.967 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:31:07.968 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:07.968 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 11461 2024-09-14T06:31:07.968 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:07.968 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:31:07.968 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:07.968 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:31:07.968 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655370+0000", 2024-09-14T06:31:07.968 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:07.968 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:07.968 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:07.968 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:31:07.968 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655370+0000", 2024-09-14T06:31:07.968 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:07.968 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:07.969 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:07.969 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:31:07.969 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655394+0000", 2024-09-14T06:31:07.969 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.4553999999999999e-05 2024-09-14T06:31:07.969 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:07.969 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:07.969 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:31:07.969 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655396+0000", 2024-09-14T06:31:07.969 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.8950000000000001e-06 2024-09-14T06:31:07.969 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:07.969 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:07.969 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:31:07.969 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655401+0000", 2024-09-14T06:31:07.969 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.0089999999999998e-06 2024-09-14T06:31:07.970 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:07.970 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:07.970 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:31:07.970 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655404+0000", 2024-09-14T06:31:07.970 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.3400000000000002e-06 2024-09-14T06:31:07.970 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:07.970 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:07.970 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:31:07.970 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.708186+0000", 2024-09-14T06:31:07.970 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052781726000000001 2024-09-14T06:31:07.970 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:07.970 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:07.970 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:31:07.970 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.708229+0000", 2024-09-14T06:31:07.971 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.2670999999999999e-05 2024-09-14T06:31:07.971 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:07.971 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:07.971 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:31:07.971 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.708285+0000", 2024-09-14T06:31:07.971 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.5946999999999998e-05 2024-09-14T06:31:07.971 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:07.971 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:31:07.971 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:07.971 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:07.971 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:07.971 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:11462 3.1d 3:bcad7303:::rbd_data.112d92027590.0000000000000001:f [read 186368~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:31:07.971 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:25.655410+0000", 2024-09-14T06:31:07.971 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 102.273573189, 2024-09-14T06:31:07.971 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052995258000000003, 2024-09-14T06:31:07.972 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:31:07.972 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:31:07.972 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:31:07.972 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:31:07.972 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:31:07.972 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:07.972 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 11462 2024-09-14T06:31:07.972 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:07.972 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:31:07.972 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:07.972 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:31:07.972 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655410+0000", 2024-09-14T06:31:07.972 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:07.972 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:07.973 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:07.973 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:31:07.973 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655410+0000", 2024-09-14T06:31:07.973 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:07.973 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:07.973 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:07.973 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:31:07.973 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655625+0000", 2024-09-14T06:31:07.973 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00021442600000000001 2024-09-14T06:31:07.973 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:07.973 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:07.973 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:31:07.973 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655626+0000", 2024-09-14T06:31:07.974 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.815e-06 2024-09-14T06:31:07.974 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:07.974 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:07.974 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:31:07.974 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655631+0000", 2024-09-14T06:31:07.974 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.651e-06 2024-09-14T06:31:07.974 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:07.974 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:07.974 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:31:07.974 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655634+0000", 2024-09-14T06:31:07.974 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.2449999999999998e-06 2024-09-14T06:31:07.974 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:07.974 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:07.974 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:31:07.975 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.708312+0000", 2024-09-14T06:31:07.975 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052677412 2024-09-14T06:31:07.975 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:07.975 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:07.975 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:31:07.975 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.708352+0000", 2024-09-14T06:31:07.975 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.0729999999999998e-05 2024-09-14T06:31:07.975 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:07.975 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:07.975 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:31:07.975 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.708405+0000", 2024-09-14T06:31:07.975 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.2979000000000002e-05 2024-09-14T06:31:07.975 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:07.975 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:31:07.976 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:07.976 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:07.976 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:07.976 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:11463 3.1d 3:bcad7303:::rbd_data.112d92027590.0000000000000001:f [read 196608~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:31:07.976 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:25.655639+0000", 2024-09-14T06:31:07.976 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 102.27334404299999, 2024-09-14T06:31:07.976 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052888859000000003, 2024-09-14T06:31:07.976 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:31:07.976 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:31:07.976 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:31:07.976 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:31:07.976 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:31:07.976 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:07.976 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 11463 2024-09-14T06:31:07.977 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:07.977 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:31:07.977 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:07.977 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:31:07.977 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655639+0000", 2024-09-14T06:31:07.977 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:07.977 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:07.977 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:07.977 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:31:07.977 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655639+0000", 2024-09-14T06:31:07.977 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:07.977 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:07.977 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:07.978 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:31:07.978 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655640+0000", 2024-09-14T06:31:07.978 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.08e-07 2024-09-14T06:31:07.978 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:07.978 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:07.978 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:31:07.978 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655641+0000", 2024-09-14T06:31:07.978 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.156e-06 2024-09-14T06:31:07.978 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:07.978 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:07.978 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:31:07.978 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655648+0000", 2024-09-14T06:31:07.978 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.8460000000000002e-06 2024-09-14T06:31:07.978 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:07.979 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:07.979 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:31:07.979 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655651+0000", 2024-09-14T06:31:07.979 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.0079999999999998e-06 2024-09-14T06:31:07.979 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:07.979 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:07.979 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:31:07.979 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.708429+0000", 2024-09-14T06:31:07.979 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052778459 2024-09-14T06:31:07.979 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:07.979 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:07.979 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:31:07.979 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.708473+0000", 2024-09-14T06:31:07.979 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.3782999999999997e-05 2024-09-14T06:31:07.980 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:07.980 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:07.980 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:31:07.980 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.708528+0000", 2024-09-14T06:31:07.980 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.5198999999999998e-05 2024-09-14T06:31:07.980 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:07.980 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:31:07.980 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:07.980 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:07.980 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:07.980 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:11464 3.1d 3:bcad7303:::rbd_data.112d92027590.0000000000000001:f [read 206848~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:31:07.980 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:25.655654+0000", 2024-09-14T06:31:07.980 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 102.273329174, 2024-09-14T06:31:07.981 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052981845999999999, 2024-09-14T06:31:07.981 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:31:07.981 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:31:07.981 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:31:07.981 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:31:07.981 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:31:07.981 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:07.981 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 11464 2024-09-14T06:31:07.981 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:07.981 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:31:07.981 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:07.981 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:31:07.981 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655654+0000", 2024-09-14T06:31:07.982 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:07.982 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:07.982 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:07.982 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:31:07.982 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655654+0000", 2024-09-14T06:31:07.982 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:07.982 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:07.982 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:07.982 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:31:07.982 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655946+0000", 2024-09-14T06:31:07.983 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00029148200000000003 2024-09-14T06:31:07.983 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:07.983 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:07.983 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:31:07.983 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655948+0000", 2024-09-14T06:31:07.983 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.973e-06 2024-09-14T06:31:07.983 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:07.983 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:07.983 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:31:07.983 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655953+0000", 2024-09-14T06:31:07.983 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.1240000000000004e-06 2024-09-14T06:31:07.983 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:07.983 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:07.983 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:31:07.983 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655956+0000", 2024-09-14T06:31:07.984 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.315e-06 2024-09-14T06:31:07.984 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:07.984 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:07.984 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:31:07.984 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.708543+0000", 2024-09-14T06:31:07.984 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052587416999999997 2024-09-14T06:31:07.984 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:07.984 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:07.984 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:31:07.984 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.708584+0000", 2024-09-14T06:31:07.984 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.1004000000000002e-05 2024-09-14T06:31:07.984 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:07.984 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:07.984 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:31:07.985 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.708636+0000", 2024-09-14T06:31:07.985 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.1530999999999998e-05 2024-09-14T06:31:07.985 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:07.985 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:31:07.985 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:07.985 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:07.985 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:07.985 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:11465 3.1d 3:bcad7303:::rbd_data.112d92027590.0000000000000001:f [read 217088~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:31:07.985 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:25.655961+0000", 2024-09-14T06:31:07.985 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 102.27302202200001, 2024-09-14T06:31:07.985 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052798682999999999, 2024-09-14T06:31:07.985 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:31:07.985 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:31:07.986 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:31:07.986 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:31:07.986 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:31:07.986 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:07.986 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 11465 2024-09-14T06:31:07.986 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:07.986 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:31:07.986 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:07.986 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:31:07.986 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655961+0000", 2024-09-14T06:31:07.986 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:07.986 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:07.986 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:07.986 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:31:07.987 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655961+0000", 2024-09-14T06:31:07.987 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:07.987 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:07.987 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:07.987 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:31:07.987 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.656114+0000", 2024-09-14T06:31:07.987 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.000152745 2024-09-14T06:31:07.987 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:07.987 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:07.987 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:31:07.987 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.656116+0000", 2024-09-14T06:31:07.987 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.804e-06 2024-09-14T06:31:07.987 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:07.987 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:07.987 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:31:07.988 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.656120+0000", 2024-09-14T06:31:07.988 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.4700000000000004e-06 2024-09-14T06:31:07.988 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:07.988 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:07.988 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:31:07.988 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.656123+0000", 2024-09-14T06:31:07.988 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.1269999999999999e-06 2024-09-14T06:31:07.988 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:07.988 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:07.988 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:31:07.988 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.708660+0000", 2024-09-14T06:31:07.988 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052536919000000001 2024-09-14T06:31:07.988 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:07.988 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:07.989 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:31:07.989 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.708703+0000", 2024-09-14T06:31:07.989 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.3077000000000001e-05 2024-09-14T06:31:07.989 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:07.989 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:07.989 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:31:07.989 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.708760+0000", 2024-09-14T06:31:07.989 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.6541e-05 2024-09-14T06:31:07.989 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:07.989 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:31:07.989 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:07.989 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:07.989 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:07.989 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:11466 3.1d 3:bcad7303:::rbd_data.112d92027590.0000000000000001:f [read 227328~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:31:07.990 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:25.656129+0000", 2024-09-14T06:31:07.990 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 102.272854634, 2024-09-14T06:31:07.990 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052750987999999999, 2024-09-14T06:31:07.990 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:31:07.990 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:31:07.990 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:31:07.990 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:31:07.990 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:31:07.990 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:07.990 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 11466 2024-09-14T06:31:07.990 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:07.990 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:31:07.990 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:07.990 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:31:07.991 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.656129+0000", 2024-09-14T06:31:07.991 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:07.991 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:07.991 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:07.991 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:31:07.991 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.656129+0000", 2024-09-14T06:31:07.991 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:07.991 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:07.991 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:07.991 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:31:07.991 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.656347+0000", 2024-09-14T06:31:07.991 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.000218054 2024-09-14T06:31:07.991 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:07.991 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:07.992 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:31:07.992 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.656349+0000", 2024-09-14T06:31:07.992 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.9199999999999998e-06 2024-09-14T06:31:07.992 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:07.992 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:07.992 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:31:07.992 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.656353+0000", 2024-09-14T06:31:07.992 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.9210000000000002e-06 2024-09-14T06:31:07.992 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:07.992 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:07.992 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:31:07.992 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.656356+0000", 2024-09-14T06:31:07.992 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.7670000000000001e-06 2024-09-14T06:31:07.992 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:07.993 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:07.993 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:31:07.993 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.708787+0000", 2024-09-14T06:31:07.993 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052430840999999999 2024-09-14T06:31:07.993 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:07.993 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:07.993 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:31:07.993 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.708827+0000", 2024-09-14T06:31:07.993 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.9538e-05 2024-09-14T06:31:07.993 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:07.993 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:07.993 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:31:07.993 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.708880+0000", 2024-09-14T06:31:07.994 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.2947e-05 2024-09-14T06:31:07.994 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:07.994 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:31:07.994 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:07.994 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:07.994 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:07.994 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:11467 3.1d 3:bcad7303:::rbd_data.112d92027590.0000000000000001:f [read 237568~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:31:07.994 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:25.656361+0000", 2024-09-14T06:31:07.994 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 102.272621986, 2024-09-14T06:31:07.994 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052643898000000001, 2024-09-14T06:31:07.994 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:31:07.994 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:31:07.994 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:31:07.994 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:31:07.994 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:31:07.995 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:07.995 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 11467 2024-09-14T06:31:07.995 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:07.995 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:31:07.995 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:07.995 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:31:07.995 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.656361+0000", 2024-09-14T06:31:07.995 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:07.995 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:07.995 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:07.995 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:31:07.995 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.656361+0000", 2024-09-14T06:31:07.995 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:07.996 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:07.996 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:07.996 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:31:07.996 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.656855+0000", 2024-09-14T06:31:07.996 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00049372000000000001 2024-09-14T06:31:07.996 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:07.996 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:07.996 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:31:07.996 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.656859+0000", 2024-09-14T06:31:07.996 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.1300000000000003e-06 2024-09-14T06:31:07.996 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:07.996 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:07.996 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:31:07.997 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.656863+0000", 2024-09-14T06:31:07.997 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.2629999999999997e-06 2024-09-14T06:31:07.997 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:07.997 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:07.997 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:31:07.997 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.656867+0000", 2024-09-14T06:31:07.997 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.3220000000000001e-06 2024-09-14T06:31:07.997 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:07.997 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:07.997 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:31:07.997 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.708903+0000", 2024-09-14T06:31:07.997 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052035855999999998 2024-09-14T06:31:07.997 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:07.998 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:07.998 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:31:07.998 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.708949+0000", 2024-09-14T06:31:07.998 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.6276999999999998e-05 2024-09-14T06:31:07.998 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:07.998 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:07.998 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:31:07.998 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.709005+0000", 2024-09-14T06:31:07.998 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.6329999999999998e-05 2024-09-14T06:31:07.998 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:07.998 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:31:07.998 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:07.998 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:07.998 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:07.999 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:11479 3.1d 3:bcad7303:::rbd_data.112d92027590.0000000000000001:f [read 360448~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:31:07.999 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:25.658388+0000", 2024-09-14T06:31:07.999 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 102.270595153, 2024-09-14T06:31:07.999 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052408920999999997, 2024-09-14T06:31:07.999 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:31:07.999 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:31:07.999 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:31:07.999 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:31:07.999 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:31:07.999 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:07.999 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 11479 2024-09-14T06:31:07.999 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:07.999 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:31:07.999 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:07.999 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:31:08.000 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.658388+0000", 2024-09-14T06:31:08.000 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:08.000 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:08.000 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:08.000 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:31:08.000 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.658388+0000", 2024-09-14T06:31:08.000 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:08.000 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:08.000 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:08.000 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:31:08.000 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.658534+0000", 2024-09-14T06:31:08.000 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.000145613 2024-09-14T06:31:08.000 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:08.000 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:08.001 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:31:08.001 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.658540+0000", 2024-09-14T06:31:08.001 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.6680000000000004e-06 2024-09-14T06:31:08.001 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:08.001 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:08.001 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:31:08.001 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.658548+0000", 2024-09-14T06:31:08.001 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.7980000000000006e-06 2024-09-14T06:31:08.001 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:08.001 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:08.001 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:31:08.001 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.658554+0000", 2024-09-14T06:31:08.001 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.6869999999999999e-06 2024-09-14T06:31:08.001 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:08.002 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:08.002 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:31:08.002 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.710624+0000", 2024-09-14T06:31:08.002 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052070567999999998 2024-09-14T06:31:08.002 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:08.002 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:08.002 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:31:08.002 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.710666+0000", 2024-09-14T06:31:08.002 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.1467000000000001e-05 2024-09-14T06:31:08.002 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:08.002 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:08.002 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:31:08.002 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.710797+0000", 2024-09-14T06:31:08.002 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00013112 2024-09-14T06:31:08.002 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:08.003 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:31:08.003 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:08.003 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:08.003 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:08.003 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:11480 3.1d 3:bcad7303:::rbd_data.112d92027590.0000000000000001:f [read 370688~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:31:08.003 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:25.658563+0000", 2024-09-14T06:31:08.003 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 102.270420222, 2024-09-14T06:31:08.003 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052429164, 2024-09-14T06:31:08.003 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:31:08.003 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:31:08.003 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:31:08.003 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:31:08.003 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:31:08.003 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:08.004 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 11480 2024-09-14T06:31:08.004 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:08.004 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:31:08.004 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:08.004 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:31:08.004 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.658563+0000", 2024-09-14T06:31:08.004 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:08.004 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:08.004 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:08.004 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:31:08.004 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.658563+0000", 2024-09-14T06:31:08.004 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:08.004 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:08.004 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:08.005 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:31:08.005 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.658658+0000", 2024-09-14T06:31:08.005 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.5346999999999994e-05 2024-09-14T06:31:08.005 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:08.005 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:08.005 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:31:08.005 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.658662+0000", 2024-09-14T06:31:08.005 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.14e-06 2024-09-14T06:31:08.005 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:08.005 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:08.005 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:31:08.005 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.658669+0000", 2024-09-14T06:31:08.005 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.7379999999999994e-06 2024-09-14T06:31:08.005 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:08.005 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:08.006 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:31:08.006 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.658675+0000", 2024-09-14T06:31:08.006 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.5430000000000001e-06 2024-09-14T06:31:08.006 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:08.006 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:08.006 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:31:08.006 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.710824+0000", 2024-09-14T06:31:08.006 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052148847999999998 2024-09-14T06:31:08.006 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:08.006 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:08.006 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:31:08.006 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.710864+0000", 2024-09-14T06:31:08.006 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.9925999999999997e-05 2024-09-14T06:31:08.006 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:08.007 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:08.007 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:31:08.007 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.710992+0000", 2024-09-14T06:31:08.007 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.000128622 2024-09-14T06:31:08.007 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:08.007 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:31:08.007 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:08.007 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:08.007 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:08.007 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:11481 3.1d 3:bcad7303:::rbd_data.112d92027590.0000000000000001:f [read 380928~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:31:08.007 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:25.658684+0000", 2024-09-14T06:31:08.007 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 102.27029915200001, 2024-09-14T06:31:08.007 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052508565, 2024-09-14T06:31:08.007 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:31:08.008 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:31:08.008 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:31:08.008 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:31:08.008 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:31:08.008 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:08.008 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 11481 2024-09-14T06:31:08.008 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:08.008 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:31:08.008 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:08.008 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:31:08.008 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.658684+0000", 2024-09-14T06:31:08.008 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:08.008 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:08.009 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:08.009 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:31:08.009 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.658684+0000", 2024-09-14T06:31:08.009 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:08.009 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:08.009 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:08.009 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:31:08.009 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.658715+0000", 2024-09-14T06:31:08.009 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.0703000000000001e-05 2024-09-14T06:31:08.009 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:08.009 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:08.009 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:31:08.009 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.658718+0000", 2024-09-14T06:31:08.009 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.0709999999999999e-06 2024-09-14T06:31:08.010 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:08.010 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:08.010 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:31:08.010 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.658725+0000", 2024-09-14T06:31:08.010 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.2339999999999997e-06 2024-09-14T06:31:08.010 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:08.010 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:08.010 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:31:08.010 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.658731+0000", 2024-09-14T06:31:08.010 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.6450000000000002e-06 2024-09-14T06:31:08.010 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:08.010 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:08.010 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:31:08.010 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.711020+0000", 2024-09-14T06:31:08.011 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052288886 2024-09-14T06:31:08.011 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:08.011 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:08.011 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:31:08.011 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.711061+0000", 2024-09-14T06:31:08.011 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.1084000000000002e-05 2024-09-14T06:31:08.011 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:08.011 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:08.011 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:31:08.011 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.711193+0000", 2024-09-14T06:31:08.011 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00013194199999999999 2024-09-14T06:31:08.011 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:08.011 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:31:08.011 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:08.012 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:08.012 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:08.012 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:11482 3.1d 3:bcad7303:::rbd_data.112d92027590.0000000000000001:f [read 391168~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:31:08.012 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:25.658740+0000", 2024-09-14T06:31:08.012 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 102.270243687, 2024-09-14T06:31:08.012 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052637015000000002, 2024-09-14T06:31:08.012 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:31:08.012 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:31:08.012 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:31:08.012 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:31:08.012 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:31:08.012 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:08.012 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 11482 2024-09-14T06:31:08.012 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:08.012 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:31:08.013 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:08.013 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:31:08.013 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.658740+0000", 2024-09-14T06:31:08.013 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:08.013 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:08.013 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:08.013 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:31:08.013 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.658740+0000", 2024-09-14T06:31:08.013 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:08.013 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:08.013 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:08.013 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:31:08.013 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.658880+0000", 2024-09-14T06:31:08.013 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00014036100000000001 2024-09-14T06:31:08.014 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:08.014 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:08.014 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:31:08.014 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.658883+0000", 2024-09-14T06:31:08.014 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.3969999999999998e-06 2024-09-14T06:31:08.014 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:08.014 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:08.014 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:31:08.014 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.658890+0000", 2024-09-14T06:31:08.014 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.1500000000000002e-06 2024-09-14T06:31:08.014 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:08.014 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:08.014 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:31:08.014 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.658897+0000", 2024-09-14T06:31:08.015 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.1260000000000003e-06 2024-09-14T06:31:08.015 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:08.015 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:08.015 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:31:08.015 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.711207+0000", 2024-09-14T06:31:08.015 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052310777000000003 2024-09-14T06:31:08.015 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:08.015 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:08.015 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:31:08.015 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.711249+0000", 2024-09-14T06:31:08.015 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.1786999999999998e-05 2024-09-14T06:31:08.015 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:08.015 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:08.015 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:31:08.016 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.711377+0000", 2024-09-14T06:31:08.016 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.000127417 2024-09-14T06:31:08.016 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:08.016 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:31:08.016 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:08.016 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:08.016 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:08.016 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:11483 3.1d 3:bcad7303:::rbd_data.112d92027590.0000000000000001:f [read 401408~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:31:08.016 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:25.658906+0000", 2024-09-14T06:31:08.016 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 102.270076903, 2024-09-14T06:31:08.016 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052595109000000001, 2024-09-14T06:31:08.016 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:31:08.016 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:31:08.016 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:31:08.017 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:31:08.017 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:31:08.017 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:08.017 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 11483 2024-09-14T06:31:08.017 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:08.017 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:31:08.017 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:08.017 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:31:08.017 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.658906+0000", 2024-09-14T06:31:08.017 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:08.017 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:08.017 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:08.017 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:31:08.017 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.658906+0000", 2024-09-14T06:31:08.018 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:08.018 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:08.018 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:08.018 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:31:08.018 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.664985+0000", 2024-09-14T06:31:08.018 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.0060787250000000001 2024-09-14T06:31:08.018 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:08.018 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:08.018 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:31:08.018 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.664991+0000", 2024-09-14T06:31:08.018 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.7030000000000003e-06 2024-09-14T06:31:08.018 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:08.018 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:08.018 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:31:08.019 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.664999+0000", 2024-09-14T06:31:08.019 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.4430000000000003e-06 2024-09-14T06:31:08.019 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:08.019 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:08.019 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:31:08.019 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.665006+0000", 2024-09-14T06:31:08.019 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.0840000000000003e-06 2024-09-14T06:31:08.019 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:08.019 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:08.019 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:31:08.019 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.711402+0000", 2024-09-14T06:31:08.019 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046395714999999997 2024-09-14T06:31:08.019 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:08.019 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:08.019 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:31:08.020 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.711443+0000", 2024-09-14T06:31:08.020 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.0911999999999997e-05 2024-09-14T06:31:08.020 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:08.020 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:08.020 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:31:08.020 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.711501+0000", 2024-09-14T06:31:08.020 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.8526999999999997e-05 2024-09-14T06:31:08.020 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:08.020 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:31:08.020 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:08.020 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:08.020 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:31:08.020 INFO:teuthology.orchestra.run.smithi138.stdout:} 2024-09-14T06:31:08.022 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-14T06:31:08.022 INFO:tasks.ceph.osd.7.smithi138.stderr:2024-09-14T06:31:08.021+0000 7f3c6a21d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33623) UID: 0 2024-09-14T06:31:08.123 INFO:tasks.ceph.osd.2.smithi094.stderr:2024-09-14T06:31:08.122+0000 7faf3269b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33649) UID: 0 2024-09-14T06:31:08.165 INFO:teuthology.orchestra.run.smithi138.stdout:{ 2024-09-14T06:31:08.165 INFO:teuthology.orchestra.run.smithi138.stdout: "ops": [], 2024-09-14T06:31:08.165 INFO:teuthology.orchestra.run.smithi138.stdout: "num_ops": 0 2024-09-14T06:31:08.165 INFO:teuthology.orchestra.run.smithi138.stdout:} 2024-09-14T06:31:08.173 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-14T06:31:08.224 INFO:tasks.ceph.osd.2.smithi094.stderr:2024-09-14T06:31:08.223+0000 7faf3269b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33649) UID: 0 2024-09-14T06:31:08.276 INFO:teuthology.orchestra.run.smithi138.stdout:{ 2024-09-14T06:31:08.276 INFO:teuthology.orchestra.run.smithi138.stdout: "ops": [], 2024-09-14T06:31:08.276 INFO:teuthology.orchestra.run.smithi138.stdout: "complaint_time": 30, 2024-09-14T06:31:08.276 INFO:teuthology.orchestra.run.smithi138.stdout: "num_blocked_ops": 0 2024-09-14T06:31:08.276 INFO:teuthology.orchestra.run.smithi138.stdout:} 2024-09-14T06:31:08.284 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-14T06:31:08.325 INFO:tasks.ceph.osd.3.smithi094.stderr:2024-09-14T06:31:08.324+0000 7feeeafe8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33655) UID: 0 2024-09-14T06:31:08.388 INFO:teuthology.orchestra.run.smithi138.stdout:{ 2024-09-14T06:31:08.388 INFO:teuthology.orchestra.run.smithi138.stdout: "size": 20, 2024-09-14T06:31:08.388 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 600, 2024-09-14T06:31:08.388 INFO:teuthology.orchestra.run.smithi138.stdout: "ops": [ 2024-09-14T06:31:08.388 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:08.389 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:98505 3.10 3.1bf56f90 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T06:31:08.389 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:30:07.045453+0000", 2024-09-14T06:31:08.389 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 61.341379205999999, 2024-09-14T06:31:08.389 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032144440000000003, 2024-09-14T06:31:08.389 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:31:08.389 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:31:08.389 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "reached pg", 2024-09-14T06:31:08.389 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:31:08.389 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:31:08.389 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:08.389 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 98505 2024-09-14T06:31:08.389 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:08.389 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:31:08.389 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:08.390 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:31:08.390 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045453+0000", 2024-09-14T06:31:08.390 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:08.390 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:08.390 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:08.390 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:31:08.390 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045453+0000", 2024-09-14T06:31:08.390 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:08.390 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:08.390 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:08.390 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:31:08.390 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045453+0000", 2024-09-14T06:31:08.390 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.36e-07 2024-09-14T06:31:08.390 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:08.391 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:08.391 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:31:08.391 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045454+0000", 2024-09-14T06:31:08.391 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.1299999999999998e-07 2024-09-14T06:31:08.391 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:08.391 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:08.391 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:31:08.391 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045457+0000", 2024-09-14T06:31:08.391 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.8559999999999998e-06 2024-09-14T06:31:08.391 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:08.391 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:08.391 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:31:08.391 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045460+0000", 2024-09-14T06:31:08.391 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.8750000000000002e-06 2024-09-14T06:31:08.391 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:08.392 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:08.392 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:31:08.392 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.077592+0000", 2024-09-14T06:31:08.392 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032131727999999998 2024-09-14T06:31:08.392 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:08.392 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:08.392 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:31:08.392 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.077597+0000", 2024-09-14T06:31:08.392 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.8319999999999996e-06 2024-09-14T06:31:08.392 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:08.392 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:31:08.392 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:08.392 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:08.393 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:08.393 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:98506 3.10 3.1bf56f90 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T06:31:08.393 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:30:07.045481+0000", 2024-09-14T06:31:08.393 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 61.341350951000003, 2024-09-14T06:31:08.393 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032142759999999999, 2024-09-14T06:31:08.393 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:31:08.393 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:31:08.393 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "reached pg", 2024-09-14T06:31:08.393 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:31:08.393 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:31:08.393 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:08.393 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 98506 2024-09-14T06:31:08.393 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:08.393 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:31:08.394 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:08.394 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:31:08.394 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045481+0000", 2024-09-14T06:31:08.394 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:08.394 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:08.394 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:08.394 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:31:08.394 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045481+0000", 2024-09-14T06:31:08.394 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:08.394 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:08.394 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:08.394 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:31:08.394 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045481+0000", 2024-09-14T06:31:08.394 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.8500000000000001e-07 2024-09-14T06:31:08.394 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:08.394 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:08.395 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:31:08.395 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045483+0000", 2024-09-14T06:31:08.395 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.8250000000000001e-06 2024-09-14T06:31:08.395 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:08.395 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:08.395 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:31:08.395 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045489+0000", 2024-09-14T06:31:08.395 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.5740000000000003e-06 2024-09-14T06:31:08.395 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:08.395 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:08.395 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:31:08.395 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045495+0000", 2024-09-14T06:31:08.395 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.7690000000000002e-06 2024-09-14T06:31:08.395 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:08.396 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:08.396 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:31:08.396 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.077619+0000", 2024-09-14T06:31:08.396 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032124621999999999 2024-09-14T06:31:08.396 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:08.396 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:08.396 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:31:08.396 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.077624+0000", 2024-09-14T06:31:08.396 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.3850000000000004e-06 2024-09-14T06:31:08.396 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:08.396 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:31:08.396 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:08.396 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:08.396 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:08.397 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:98507 3.10 3.1bf56f90 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T06:31:08.397 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:30:07.045502+0000", 2024-09-14T06:31:08.397 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 61.341329530000003, 2024-09-14T06:31:08.397 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032147219999999997, 2024-09-14T06:31:08.397 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:31:08.397 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:31:08.397 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "reached pg", 2024-09-14T06:31:08.397 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:31:08.397 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:31:08.397 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:08.397 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 98507 2024-09-14T06:31:08.397 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:08.397 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:31:08.397 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:08.398 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:31:08.398 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045502+0000", 2024-09-14T06:31:08.398 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:08.398 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:08.398 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:08.398 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:31:08.398 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045502+0000", 2024-09-14T06:31:08.398 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:08.398 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:08.398 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:08.398 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:31:08.398 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045503+0000", 2024-09-14T06:31:08.398 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.4700000000000002e-07 2024-09-14T06:31:08.398 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:08.398 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:08.399 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:31:08.399 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045509+0000", 2024-09-14T06:31:08.399 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.7329999999999997e-06 2024-09-14T06:31:08.399 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:08.399 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:08.399 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:31:08.399 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045515+0000", 2024-09-14T06:31:08.399 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.8030000000000002e-06 2024-09-14T06:31:08.399 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:08.399 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:08.399 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:31:08.399 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045522+0000", 2024-09-14T06:31:08.399 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.0369999999999997e-06 2024-09-14T06:31:08.399 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:08.399 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:08.400 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:31:08.400 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.077641+0000", 2024-09-14T06:31:08.400 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032119091000000002 2024-09-14T06:31:08.400 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:08.400 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:08.400 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:31:08.400 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.077650+0000", 2024-09-14T06:31:08.400 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.2090000000000005e-06 2024-09-14T06:31:08.400 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:08.400 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:31:08.400 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:08.400 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:08.400 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:08.401 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:98508 3.10 3.1bf56f90 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T06:31:08.401 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:30:07.045530+0000", 2024-09-14T06:31:08.401 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 61.341301913000002, 2024-09-14T06:31:08.401 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032198773999999999, 2024-09-14T06:31:08.401 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:31:08.401 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:31:08.401 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "reached pg", 2024-09-14T06:31:08.401 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:31:08.401 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:31:08.401 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:08.401 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 98508 2024-09-14T06:31:08.401 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:08.401 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:31:08.401 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:08.401 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:31:08.402 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045530+0000", 2024-09-14T06:31:08.402 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:08.402 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:08.402 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:08.402 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:31:08.402 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045530+0000", 2024-09-14T06:31:08.402 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:08.402 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:08.402 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:08.402 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:31:08.402 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045530+0000", 2024-09-14T06:31:08.402 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.3099999999999998e-07 2024-09-14T06:31:08.402 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:08.402 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:08.403 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:31:08.403 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045539+0000", 2024-09-14T06:31:08.403 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.4780000000000008e-06 2024-09-14T06:31:08.403 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:08.403 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:08.403 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:31:08.403 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045546+0000", 2024-09-14T06:31:08.403 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.3259999999999998e-06 2024-09-14T06:31:08.403 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:08.403 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:08.403 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:31:08.403 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045551+0000", 2024-09-14T06:31:08.403 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.9960000000000001e-06 2024-09-14T06:31:08.403 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:08.403 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:08.404 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:31:08.404 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.077716+0000", 2024-09-14T06:31:08.404 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032165081999999998 2024-09-14T06:31:08.404 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:08.404 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:08.404 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:31:08.404 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.077729+0000", 2024-09-14T06:31:08.404 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.2461e-05 2024-09-14T06:31:08.404 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:08.404 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:31:08.404 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:08.404 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:08.404 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:08.404 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:98509 3.10 3.1bf56f90 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T06:31:08.405 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:30:07.045558+0000", 2024-09-14T06:31:08.405 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 61.341273712000003, 2024-09-14T06:31:08.405 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032242198, 2024-09-14T06:31:08.405 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:31:08.405 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:31:08.405 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "reached pg", 2024-09-14T06:31:08.405 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:31:08.405 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:31:08.405 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:08.405 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 98509 2024-09-14T06:31:08.405 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:08.405 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:31:08.405 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:08.405 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:31:08.406 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045558+0000", 2024-09-14T06:31:08.406 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:08.406 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:08.406 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:08.406 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:31:08.406 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045558+0000", 2024-09-14T06:31:08.406 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:08.406 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:08.406 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:08.406 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:31:08.406 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045559+0000", 2024-09-14T06:31:08.406 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.27e-07 2024-09-14T06:31:08.406 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:08.406 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:08.407 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:31:08.407 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045560+0000", 2024-09-14T06:31:08.407 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.6959999999999999e-06 2024-09-14T06:31:08.407 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:08.407 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:08.407 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:31:08.407 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045567+0000", 2024-09-14T06:31:08.407 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.6900000000000003e-06 2024-09-14T06:31:08.407 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:08.407 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:08.407 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:31:08.407 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045571+0000", 2024-09-14T06:31:08.407 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.5750000000000002e-06 2024-09-14T06:31:08.408 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:08.408 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:08.408 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:31:08.408 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.077789+0000", 2024-09-14T06:31:08.408 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032217186000000002 2024-09-14T06:31:08.408 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:08.408 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:08.408 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:31:08.408 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.077800+0000", 2024-09-14T06:31:08.408 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.1623999999999999e-05 2024-09-14T06:31:08.408 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:08.408 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:31:08.408 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:08.408 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:08.409 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:08.409 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:98510 3.10 3.1bf56f90 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T06:31:08.409 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:30:07.045580+0000", 2024-09-14T06:31:08.409 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 61.341251898000003, 2024-09-14T06:31:08.409 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032299845000000001, 2024-09-14T06:31:08.409 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:31:08.409 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:31:08.409 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "reached pg", 2024-09-14T06:31:08.409 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:31:08.409 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:31:08.409 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:08.409 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 98510 2024-09-14T06:31:08.409 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:08.409 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:31:08.410 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:08.410 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:31:08.410 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045580+0000", 2024-09-14T06:31:08.410 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:08.410 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:08.410 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:08.410 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:31:08.410 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045580+0000", 2024-09-14T06:31:08.410 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:08.410 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:08.410 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:08.410 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:31:08.411 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045580+0000", 2024-09-14T06:31:08.411 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.0400000000000002e-07 2024-09-14T06:31:08.411 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:08.411 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:08.411 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:31:08.411 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045582+0000", 2024-09-14T06:31:08.411 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.7600000000000001e-06 2024-09-14T06:31:08.411 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:08.411 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:08.411 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:31:08.411 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045588+0000", 2024-09-14T06:31:08.411 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.5650000000000001e-06 2024-09-14T06:31:08.411 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:08.411 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:08.412 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:31:08.412 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045597+0000", 2024-09-14T06:31:08.412 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.465e-06 2024-09-14T06:31:08.412 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:08.412 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:08.412 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:31:08.412 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.077871+0000", 2024-09-14T06:31:08.412 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032273562999999998 2024-09-14T06:31:08.412 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:08.412 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:08.412 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:31:08.412 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.077880+0000", 2024-09-14T06:31:08.412 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.0880000000000001e-06 2024-09-14T06:31:08.412 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:08.412 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:31:08.413 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:08.413 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:08.413 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:08.413 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:98511 3.10 3.1bf56f90 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T06:31:08.413 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:30:07.045610+0000", 2024-09-14T06:31:08.413 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 61.341222051000003, 2024-09-14T06:31:08.413 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032338894, 2024-09-14T06:31:08.413 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:31:08.413 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:31:08.413 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "reached pg", 2024-09-14T06:31:08.413 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:31:08.413 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:31:08.413 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:08.413 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 98511 2024-09-14T06:31:08.414 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:08.414 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:31:08.414 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:08.414 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:31:08.414 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045610+0000", 2024-09-14T06:31:08.414 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:08.414 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:08.414 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:08.414 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:31:08.414 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045610+0000", 2024-09-14T06:31:08.414 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:08.414 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:08.414 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:08.414 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:31:08.415 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045610+0000", 2024-09-14T06:31:08.415 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.1600000000000002e-07 2024-09-14T06:31:08.415 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:08.415 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:08.415 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:31:08.415 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045612+0000", 2024-09-14T06:31:08.415 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.1150000000000001e-06 2024-09-14T06:31:08.415 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:08.415 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:08.415 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:31:08.415 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045618+0000", 2024-09-14T06:31:08.415 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.0499999999999997e-06 2024-09-14T06:31:08.415 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:08.416 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:08.416 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:31:08.416 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045623+0000", 2024-09-14T06:31:08.416 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.6770000000000003e-06 2024-09-14T06:31:08.416 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:08.416 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:08.416 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:31:08.416 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.077928+0000", 2024-09-14T06:31:08.416 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032305111999999997 2024-09-14T06:31:08.416 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:08.416 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:08.416 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:31:08.416 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.077949+0000", 2024-09-14T06:31:08.416 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.0523999999999999e-05 2024-09-14T06:31:08.417 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:08.417 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:31:08.417 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:08.417 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:08.417 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:08.417 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:98512 3.10 3.1bf56f90 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T06:31:08.417 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:30:07.045630+0000", 2024-09-14T06:31:08.417 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 61.341202045999999, 2024-09-14T06:31:08.417 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032346632, 2024-09-14T06:31:08.417 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:31:08.417 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:31:08.417 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "reached pg", 2024-09-14T06:31:08.417 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:31:08.417 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:31:08.418 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:08.418 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 98512 2024-09-14T06:31:08.418 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:08.418 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:31:08.418 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:08.418 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:31:08.418 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045630+0000", 2024-09-14T06:31:08.418 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:08.418 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:08.418 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:08.418 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:31:08.418 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045630+0000", 2024-09-14T06:31:08.418 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:08.418 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:08.419 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:08.419 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:31:08.419 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045630+0000", 2024-09-14T06:31:08.419 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.8200000000000001e-07 2024-09-14T06:31:08.419 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:08.419 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:08.419 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:31:08.419 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045632+0000", 2024-09-14T06:31:08.419 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.7430000000000001e-06 2024-09-14T06:31:08.419 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:08.419 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:08.419 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:31:08.419 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045637+0000", 2024-09-14T06:31:08.419 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.4550000000000003e-06 2024-09-14T06:31:08.419 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:08.420 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:08.420 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:31:08.420 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045644+0000", 2024-09-14T06:31:08.420 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.1330000000000001e-06 2024-09-14T06:31:08.420 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:08.420 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:08.420 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:31:08.420 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.077968+0000", 2024-09-14T06:31:08.420 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032323401000000002 2024-09-14T06:31:08.420 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:08.420 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:08.420 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:31:08.420 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.077976+0000", 2024-09-14T06:31:08.420 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.5180000000000004e-06 2024-09-14T06:31:08.420 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:08.421 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:31:08.421 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:08.421 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:08.421 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:08.421 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:98513 3.10 3.1bf56f90 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T06:31:08.421 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:30:07.045652+0000", 2024-09-14T06:31:08.421 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 61.341179482000001, 2024-09-14T06:31:08.421 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032353436999999999, 2024-09-14T06:31:08.421 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:31:08.421 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:31:08.421 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "reached pg", 2024-09-14T06:31:08.421 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:31:08.421 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:31:08.421 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:08.422 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 98513 2024-09-14T06:31:08.422 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:08.422 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:31:08.422 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:08.422 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:31:08.422 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045652+0000", 2024-09-14T06:31:08.422 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:08.422 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:08.422 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:08.422 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:31:08.422 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045652+0000", 2024-09-14T06:31:08.422 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:08.422 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:08.422 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:08.423 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:31:08.423 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045653+0000", 2024-09-14T06:31:08.423 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.3700000000000001e-07 2024-09-14T06:31:08.423 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:08.423 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:08.423 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:31:08.423 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045655+0000", 2024-09-14T06:31:08.423 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.854e-06 2024-09-14T06:31:08.423 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:08.423 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:08.423 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:31:08.423 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045661+0000", 2024-09-14T06:31:08.423 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.5509999999999996e-06 2024-09-14T06:31:08.423 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:08.424 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:08.424 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:31:08.424 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045668+0000", 2024-09-14T06:31:08.424 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.1980000000000004e-06 2024-09-14T06:31:08.424 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:08.424 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:08.424 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:31:08.424 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.077997+0000", 2024-09-14T06:31:08.424 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032328575999999998 2024-09-14T06:31:08.424 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:08.424 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:08.424 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:31:08.424 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.078006+0000", 2024-09-14T06:31:08.424 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.8210000000000008e-06 2024-09-14T06:31:08.425 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:08.425 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:31:08.425 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:08.425 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:08.425 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:08.425 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:98514 3.10 3.1bf56f90 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T06:31:08.425 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:30:07.045699+0000", 2024-09-14T06:31:08.425 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 61.341133147000001, 2024-09-14T06:31:08.425 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032330223999999998, 2024-09-14T06:31:08.425 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:31:08.425 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:31:08.425 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "reached pg", 2024-09-14T06:31:08.425 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:31:08.425 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:31:08.425 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:08.426 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 98514 2024-09-14T06:31:08.426 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:08.426 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:31:08.426 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:08.426 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:31:08.426 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045699+0000", 2024-09-14T06:31:08.426 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:08.426 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:08.426 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:08.426 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:31:08.426 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045699+0000", 2024-09-14T06:31:08.426 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:08.426 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:08.426 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:08.427 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:31:08.427 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045699+0000", 2024-09-14T06:31:08.427 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.3200000000000001e-07 2024-09-14T06:31:08.427 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:08.427 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:08.427 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:31:08.427 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045701+0000", 2024-09-14T06:31:08.427 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.762e-06 2024-09-14T06:31:08.427 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:08.427 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:08.427 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:31:08.427 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045705+0000", 2024-09-14T06:31:08.427 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.2359999999999998e-06 2024-09-14T06:31:08.427 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:08.427 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:08.428 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:31:08.428 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045708+0000", 2024-09-14T06:31:08.428 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.796e-06 2024-09-14T06:31:08.428 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:08.428 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:08.428 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:31:08.428 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.078023+0000", 2024-09-14T06:31:08.428 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032315213000000002 2024-09-14T06:31:08.428 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:08.428 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:08.428 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:31:08.428 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.078029+0000", 2024-09-14T06:31:08.428 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.8850000000000004e-06 2024-09-14T06:31:08.428 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:08.429 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:31:08.429 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:08.429 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:08.429 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:08.429 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:98515 3.10 3.1bf56f90 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T06:31:08.429 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:30:07.045712+0000", 2024-09-14T06:31:08.429 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 61.341119835999997, 2024-09-14T06:31:08.429 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032346871999999999, 2024-09-14T06:31:08.429 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:31:08.429 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:31:08.429 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "reached pg", 2024-09-14T06:31:08.429 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:31:08.429 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:31:08.429 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:08.430 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 98515 2024-09-14T06:31:08.430 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:08.430 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:31:08.430 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:08.430 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:31:08.430 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045712+0000", 2024-09-14T06:31:08.430 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:08.430 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:08.430 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:08.430 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:31:08.430 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045712+0000", 2024-09-14T06:31:08.430 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:08.430 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:08.430 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:08.430 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:31:08.431 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045712+0000", 2024-09-14T06:31:08.431 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.2100000000000001e-07 2024-09-14T06:31:08.431 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:08.431 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:08.431 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:31:08.431 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045713+0000", 2024-09-14T06:31:08.431 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.068e-06 2024-09-14T06:31:08.431 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:08.431 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:08.431 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:31:08.431 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045716+0000", 2024-09-14T06:31:08.431 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.971e-06 2024-09-14T06:31:08.431 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:08.431 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:08.432 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:31:08.432 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045719+0000", 2024-09-14T06:31:08.432 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.3879999999999998e-06 2024-09-14T06:31:08.432 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:08.432 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:08.432 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:31:08.432 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.078049+0000", 2024-09-14T06:31:08.432 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032330553999999997 2024-09-14T06:31:08.432 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:08.432 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:08.432 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:31:08.432 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.078059+0000", 2024-09-14T06:31:08.432 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.6700000000000006e-06 2024-09-14T06:31:08.432 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:08.433 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:31:08.433 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:08.433 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:08.433 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:08.433 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:98516 3.10 3.1bf56f90 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T06:31:08.433 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:30:07.045722+0000", 2024-09-14T06:31:08.433 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 61.341110082999997, 2024-09-14T06:31:08.433 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032374334999999997, 2024-09-14T06:31:08.433 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:31:08.433 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:31:08.433 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "reached pg", 2024-09-14T06:31:08.433 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:31:08.433 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:31:08.433 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:08.434 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 98516 2024-09-14T06:31:08.434 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:08.434 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:31:08.434 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:08.434 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:31:08.434 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045722+0000", 2024-09-14T06:31:08.434 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:08.434 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:08.434 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:08.434 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:31:08.434 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045722+0000", 2024-09-14T06:31:08.434 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:08.434 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:08.434 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:08.435 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:31:08.435 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045722+0000", 2024-09-14T06:31:08.435 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.8999999999999998e-07 2024-09-14T06:31:08.435 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:08.435 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:08.435 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:31:08.435 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045723+0000", 2024-09-14T06:31:08.435 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.0049999999999999e-06 2024-09-14T06:31:08.435 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:08.435 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:08.435 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:31:08.435 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045726+0000", 2024-09-14T06:31:08.435 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.9019999999999999e-06 2024-09-14T06:31:08.435 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:08.436 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:08.436 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:31:08.436 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045728+0000", 2024-09-14T06:31:08.436 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.3290000000000001e-06 2024-09-14T06:31:08.436 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:08.436 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:08.436 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:31:08.436 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.078090+0000", 2024-09-14T06:31:08.436 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032362123 2024-09-14T06:31:08.436 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:08.436 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:08.436 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:31:08.436 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.078096+0000", 2024-09-14T06:31:08.436 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.6860000000000003e-06 2024-09-14T06:31:08.437 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:08.437 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:31:08.437 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:08.437 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:08.437 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:08.437 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:98517 3.10 3.1bf56f90 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T06:31:08.437 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:30:07.045731+0000", 2024-09-14T06:31:08.437 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 61.341100351999998, 2024-09-14T06:31:08.437 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.03241227, 2024-09-14T06:31:08.437 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:31:08.437 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:31:08.437 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "reached pg", 2024-09-14T06:31:08.437 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:31:08.437 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:31:08.438 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:08.438 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 98517 2024-09-14T06:31:08.438 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:08.438 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:31:08.438 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:08.438 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:31:08.438 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045731+0000", 2024-09-14T06:31:08.438 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:08.438 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:08.438 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:08.438 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:31:08.438 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045731+0000", 2024-09-14T06:31:08.438 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:08.438 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:08.438 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:08.439 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:31:08.439 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045732+0000", 2024-09-14T06:31:08.439 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.0699999999999999e-07 2024-09-14T06:31:08.439 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:08.439 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:08.439 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:31:08.439 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045733+0000", 2024-09-14T06:31:08.439 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.9199999999999999e-07 2024-09-14T06:31:08.439 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:08.439 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:08.439 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:31:08.439 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045735+0000", 2024-09-14T06:31:08.439 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.8150000000000002e-06 2024-09-14T06:31:08.440 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:08.440 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:08.440 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:31:08.440 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045738+0000", 2024-09-14T06:31:08.440 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.3640000000000002e-06 2024-09-14T06:31:08.440 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:08.440 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:08.440 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:31:08.440 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.078126+0000", 2024-09-14T06:31:08.440 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032388206000000003 2024-09-14T06:31:08.440 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:08.440 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:08.440 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:31:08.440 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.078144+0000", 2024-09-14T06:31:08.441 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.7686e-05 2024-09-14T06:31:08.441 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:08.441 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:31:08.441 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:08.441 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:08.441 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:08.441 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:98518 3.10 3.1bf56f90 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T06:31:08.441 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:30:07.045741+0000", 2024-09-14T06:31:08.441 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 61.341090956999999, 2024-09-14T06:31:08.441 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032451793999999999, 2024-09-14T06:31:08.441 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:31:08.441 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:31:08.441 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "reached pg", 2024-09-14T06:31:08.441 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:31:08.442 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:31:08.442 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:08.442 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 98518 2024-09-14T06:31:08.442 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:08.442 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:31:08.442 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:08.442 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:31:08.442 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045741+0000", 2024-09-14T06:31:08.442 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:08.442 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:08.442 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:08.442 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:31:08.442 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045741+0000", 2024-09-14T06:31:08.442 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:08.443 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:08.443 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:08.443 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:31:08.443 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045741+0000", 2024-09-14T06:31:08.443 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.05e-07 2024-09-14T06:31:08.443 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:08.443 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:08.443 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:31:08.443 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045746+0000", 2024-09-14T06:31:08.443 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.2259999999999996e-06 2024-09-14T06:31:08.443 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:08.443 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:08.443 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:31:08.443 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045750+0000", 2024-09-14T06:31:08.444 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.2490000000000001e-06 2024-09-14T06:31:08.444 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:08.444 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:08.444 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:31:08.444 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045752+0000", 2024-09-14T06:31:08.444 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.3980000000000002e-06 2024-09-14T06:31:08.444 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:08.444 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:08.444 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:31:08.444 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.078184+0000", 2024-09-14T06:31:08.444 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032431593000000002 2024-09-14T06:31:08.444 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:08.444 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:08.444 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:31:08.445 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.078193+0000", 2024-09-14T06:31:08.445 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.1230000000000006e-06 2024-09-14T06:31:08.445 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:08.445 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:31:08.445 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:08.445 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:08.445 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:08.445 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:98519 3.10 3.1bf56f90 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T06:31:08.445 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:30:07.045755+0000", 2024-09-14T06:31:08.445 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 61.341076532000002, 2024-09-14T06:31:08.445 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032489431999999999, 2024-09-14T06:31:08.445 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:31:08.445 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:31:08.445 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "reached pg", 2024-09-14T06:31:08.446 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:31:08.446 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:31:08.446 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:08.446 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 98519 2024-09-14T06:31:08.446 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:08.446 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:31:08.446 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:08.446 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:31:08.446 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045755+0000", 2024-09-14T06:31:08.446 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:08.446 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:08.446 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:08.446 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:31:08.446 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045755+0000", 2024-09-14T06:31:08.446 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:08.447 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:08.447 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:08.447 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:31:08.447 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045756+0000", 2024-09-14T06:31:08.447 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.23e-07 2024-09-14T06:31:08.447 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:08.447 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:08.447 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:31:08.447 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045757+0000", 2024-09-14T06:31:08.447 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.1060000000000001e-06 2024-09-14T06:31:08.447 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:08.447 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:08.447 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:31:08.447 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045760+0000", 2024-09-14T06:31:08.448 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.988e-06 2024-09-14T06:31:08.448 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:08.448 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:08.448 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:31:08.448 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045765+0000", 2024-09-14T06:31:08.448 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.2689999999999999e-06 2024-09-14T06:31:08.448 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:08.448 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:08.448 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:31:08.448 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.078234+0000", 2024-09-14T06:31:08.448 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032468915000000001 2024-09-14T06:31:08.448 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:08.448 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:08.448 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:31:08.448 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.078245+0000", 2024-09-14T06:31:08.449 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.0930999999999999e-05 2024-09-14T06:31:08.449 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:08.449 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:31:08.449 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:08.449 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:08.449 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:08.449 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:98520 3.10 3.1bf56f90 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T06:31:08.449 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:30:07.045768+0000", 2024-09-14T06:31:08.449 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 61.341063654000003, 2024-09-14T06:31:08.449 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032531379999999999, 2024-09-14T06:31:08.449 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:31:08.449 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:31:08.449 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "reached pg", 2024-09-14T06:31:08.450 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:31:08.450 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:31:08.450 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:08.450 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 98520 2024-09-14T06:31:08.450 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:08.450 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:31:08.450 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:08.450 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:31:08.450 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045768+0000", 2024-09-14T06:31:08.450 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:08.450 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:08.450 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:08.450 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:31:08.450 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045768+0000", 2024-09-14T06:31:08.451 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:08.451 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:08.451 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:08.451 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:31:08.451 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045768+0000", 2024-09-14T06:31:08.451 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.22e-07 2024-09-14T06:31:08.451 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:08.451 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:08.451 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:31:08.451 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045773+0000", 2024-09-14T06:31:08.451 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.3470000000000001e-06 2024-09-14T06:31:08.451 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:08.451 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:08.451 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:31:08.451 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045776+0000", 2024-09-14T06:31:08.452 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.1319999999999998e-06 2024-09-14T06:31:08.452 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:08.452 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:08.452 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:31:08.452 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045779+0000", 2024-09-14T06:31:08.452 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.9749999999999999e-06 2024-09-14T06:31:08.452 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:08.452 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:08.452 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:31:08.452 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.078289+0000", 2024-09-14T06:31:08.452 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032509693999999999 2024-09-14T06:31:08.452 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:08.452 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:08.452 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:31:08.453 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.078300+0000", 2024-09-14T06:31:08.453 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.101e-05 2024-09-14T06:31:08.453 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:08.453 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:31:08.453 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:08.453 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:08.453 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:08.453 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:98521 3.10 3.1bf56f90 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T06:31:08.453 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:30:07.045782+0000", 2024-09-14T06:31:08.453 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 61.341049474000002, 2024-09-14T06:31:08.453 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032566109000000003, 2024-09-14T06:31:08.453 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:31:08.453 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:31:08.453 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "reached pg", 2024-09-14T06:31:08.454 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:31:08.454 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:31:08.454 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:08.454 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 98521 2024-09-14T06:31:08.454 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:08.454 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:31:08.454 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:08.454 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:31:08.454 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045782+0000", 2024-09-14T06:31:08.454 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:08.454 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:08.454 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:08.454 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:31:08.454 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045782+0000", 2024-09-14T06:31:08.454 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:08.455 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:08.455 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:08.455 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:31:08.455 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045783+0000", 2024-09-14T06:31:08.455 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.2499999999999999e-07 2024-09-14T06:31:08.455 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:08.455 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:08.455 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:31:08.455 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045784+0000", 2024-09-14T06:31:08.455 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.0109999999999999e-06 2024-09-14T06:31:08.455 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:08.455 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:08.455 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:31:08.455 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045786+0000", 2024-09-14T06:31:08.455 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.8289999999999999e-06 2024-09-14T06:31:08.456 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:08.456 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:08.456 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:31:08.456 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045793+0000", 2024-09-14T06:31:08.456 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.579e-06 2024-09-14T06:31:08.456 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:08.456 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:08.456 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:31:08.456 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.078338+0000", 2024-09-14T06:31:08.456 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032545389000000001 2024-09-14T06:31:08.456 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:08.456 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:08.456 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:31:08.456 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.078348+0000", 2024-09-14T06:31:08.457 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.0076e-05 2024-09-14T06:31:08.457 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:08.457 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:31:08.457 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:08.457 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:08.457 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:08.457 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:98522 3.10 3.1bf56f90 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T06:31:08.457 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:30:07.045796+0000", 2024-09-14T06:31:08.457 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 61.341035566000002, 2024-09-14T06:31:08.457 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032577846000000001, 2024-09-14T06:31:08.457 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:31:08.457 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:31:08.457 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "reached pg", 2024-09-14T06:31:08.457 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:31:08.458 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:31:08.458 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:08.458 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 98522 2024-09-14T06:31:08.458 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:08.458 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:31:08.458 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:08.458 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:31:08.458 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045796+0000", 2024-09-14T06:31:08.458 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:08.458 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:08.458 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:08.458 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:31:08.458 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045796+0000", 2024-09-14T06:31:08.458 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:08.458 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:08.459 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:08.459 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:31:08.459 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045796+0000", 2024-09-14T06:31:08.459 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.2100000000000001e-07 2024-09-14T06:31:08.459 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:08.459 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:08.459 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:31:08.459 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045797+0000", 2024-09-14T06:31:08.459 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.6899999999999996e-07 2024-09-14T06:31:08.459 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:08.459 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:08.459 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:31:08.459 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045800+0000", 2024-09-14T06:31:08.459 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.9979999999999999e-06 2024-09-14T06:31:08.460 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:08.460 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:08.460 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:31:08.460 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045803+0000", 2024-09-14T06:31:08.460 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.2850000000000001e-06 2024-09-14T06:31:08.460 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:08.460 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:08.460 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:31:08.460 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.078366+0000", 2024-09-14T06:31:08.460 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032562932000000003 2024-09-14T06:31:08.460 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:08.460 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:08.460 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:31:08.460 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.078374+0000", 2024-09-14T06:31:08.460 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.4409999999999993e-06 2024-09-14T06:31:08.461 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:08.461 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:31:08.461 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:08.461 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:08.461 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:08.461 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:98523 3.10 3.1bf56f90 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T06:31:08.461 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:30:07.045806+0000", 2024-09-14T06:31:08.461 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 61.341026022999998, 2024-09-14T06:31:08.461 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032611167000000003, 2024-09-14T06:31:08.461 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:31:08.461 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:31:08.461 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "reached pg", 2024-09-14T06:31:08.461 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:31:08.461 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:31:08.462 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:08.462 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 98523 2024-09-14T06:31:08.462 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:08.462 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:31:08.462 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:08.462 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:31:08.462 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045806+0000", 2024-09-14T06:31:08.462 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:08.462 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:08.462 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:08.462 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:31:08.462 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045806+0000", 2024-09-14T06:31:08.462 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:08.463 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:08.463 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:08.463 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:31:08.463 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045806+0000", 2024-09-14T06:31:08.463 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.2000000000000001e-07 2024-09-14T06:31:08.463 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:08.463 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:08.463 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:31:08.463 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045807+0000", 2024-09-14T06:31:08.463 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.4e-07 2024-09-14T06:31:08.463 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:08.463 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:08.463 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:31:08.463 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045810+0000", 2024-09-14T06:31:08.464 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.9229999999999998e-06 2024-09-14T06:31:08.464 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:08.464 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:08.464 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:31:08.464 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045812+0000", 2024-09-14T06:31:08.464 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.2220000000000001e-06 2024-09-14T06:31:08.464 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:08.464 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:08.464 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:31:08.464 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.078407+0000", 2024-09-14T06:31:08.464 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032594513999999998 2024-09-14T06:31:08.464 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:08.464 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:08.464 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:31:08.465 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.078417+0000", 2024-09-14T06:31:08.465 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.0348e-05 2024-09-14T06:31:08.465 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:08.465 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:31:08.465 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:08.465 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:08.465 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:08.465 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:98524 3.10 3.1bf56f90 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T06:31:08.465 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:30:07.045815+0000", 2024-09-14T06:31:08.465 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 61.341016654999997, 2024-09-14T06:31:08.465 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032643208999999999, 2024-09-14T06:31:08.465 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:31:08.465 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:31:08.466 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "reached pg", 2024-09-14T06:31:08.466 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:31:08.466 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:31:08.466 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:08.466 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 98524 2024-09-14T06:31:08.466 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:08.466 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:31:08.466 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:08.466 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:31:08.466 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045815+0000", 2024-09-14T06:31:08.466 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:08.466 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:08.466 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:08.466 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:31:08.467 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045815+0000", 2024-09-14T06:31:08.467 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:08.467 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:08.467 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:08.467 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:31:08.467 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045815+0000", 2024-09-14T06:31:08.467 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.11e-07 2024-09-14T06:31:08.467 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:08.467 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:08.467 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:31:08.467 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045816+0000", 2024-09-14T06:31:08.467 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.9099999999999991e-07 2024-09-14T06:31:08.467 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:08.467 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:08.468 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:31:08.468 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045819+0000", 2024-09-14T06:31:08.468 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.9399999999999998e-06 2024-09-14T06:31:08.468 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:08.468 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:08.468 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:31:08.468 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045822+0000", 2024-09-14T06:31:08.468 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.3470000000000001e-06 2024-09-14T06:31:08.468 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:08.468 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:08.468 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:31:08.468 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.078449+0000", 2024-09-14T06:31:08.468 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032627529000000002 2024-09-14T06:31:08.468 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:08.469 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:08.469 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:31:08.469 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.078458+0000", 2024-09-14T06:31:08.469 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.1910000000000006e-06 2024-09-14T06:31:08.469 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:08.469 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:31:08.469 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:08.469 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:08.469 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:31:08.469 INFO:teuthology.orchestra.run.smithi138.stdout:} 2024-09-14T06:31:08.471 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-14T06:31:08.471 INFO:tasks.ceph.osd.3.smithi094.stderr:2024-09-14T06:31:08.470+0000 7feeeafe8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33655) UID: 0 2024-09-14T06:31:08.572 INFO:tasks.ceph.osd.0.smithi094.stderr:2024-09-14T06:31:08.571+0000 7f07c9333640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33636) UID: 0 2024-09-14T06:31:08.574 INFO:teuthology.orchestra.run.smithi138.stdout:{ 2024-09-14T06:31:08.574 INFO:teuthology.orchestra.run.smithi138.stdout: "ops": [], 2024-09-14T06:31:08.574 INFO:teuthology.orchestra.run.smithi138.stdout: "num_ops": 0 2024-09-14T06:31:08.574 INFO:teuthology.orchestra.run.smithi138.stdout:} 2024-09-14T06:31:08.583 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-14T06:31:08.619 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-14T06:31:08.672 INFO:tasks.ceph.osd.6.smithi138.stderr:2024-09-14T06:31:08.671+0000 7f8cd8849640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33620) UID: 0 2024-09-14T06:31:08.685 INFO:teuthology.orchestra.run.smithi138.stdout:{ 2024-09-14T06:31:08.685 INFO:teuthology.orchestra.run.smithi138.stdout: "ops": [], 2024-09-14T06:31:08.685 INFO:teuthology.orchestra.run.smithi138.stdout: "complaint_time": 30, 2024-09-14T06:31:08.685 INFO:teuthology.orchestra.run.smithi138.stdout: "num_blocked_ops": 0 2024-09-14T06:31:08.685 INFO:teuthology.orchestra.run.smithi138.stdout:} 2024-09-14T06:31:08.694 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-14T06:31:08.778 INFO:tasks.ceph.osd.6.smithi138.stderr:2024-09-14T06:31:08.777+0000 7f8cd8849640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33620) UID: 0 2024-09-14T06:31:08.849 INFO:teuthology.orchestra.run.smithi138.stdout:{ 2024-09-14T06:31:08.850 INFO:teuthology.orchestra.run.smithi138.stdout: "size": 20, 2024-09-14T06:31:08.850 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 600, 2024-09-14T06:31:08.850 INFO:teuthology.orchestra.run.smithi138.stdout: "ops": [ 2024-09-14T06:31:08.850 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:08.850 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:53244 3.f 3:f71f51a7:::rbd_data.112dca2f6ae1.0000000000000015:17 [read 1863680~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:31:08.850 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:44.369399+0000", 2024-09-14T06:31:08.850 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 84.477885353999994, 2024-09-14T06:31:08.850 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047291189999999997, 2024-09-14T06:31:08.850 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:31:08.850 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:31:08.850 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:31:08.850 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:31:08.851 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:31:08.851 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:08.851 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 53244 2024-09-14T06:31:08.851 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:08.851 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:31:08.851 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:08.851 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:31:08.851 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369399+0000", 2024-09-14T06:31:08.851 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:08.851 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:08.851 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:08.851 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:31:08.851 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369399+0000", 2024-09-14T06:31:08.851 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:08.852 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:08.852 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:08.852 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:31:08.852 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369400+0000", 2024-09-14T06:31:08.852 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.0699999999999999e-07 2024-09-14T06:31:08.852 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:08.852 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:08.852 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:31:08.852 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369401+0000", 2024-09-14T06:31:08.852 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.4300000000000001e-06 2024-09-14T06:31:08.852 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:08.852 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:08.852 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:31:08.853 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369404+0000", 2024-09-14T06:31:08.853 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.061e-06 2024-09-14T06:31:08.853 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:08.853 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:08.853 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:31:08.853 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369407+0000", 2024-09-14T06:31:08.853 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.6139999999999999e-06 2024-09-14T06:31:08.853 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:08.853 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:08.853 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:31:08.853 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.416517+0000", 2024-09-14T06:31:08.853 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047110338000000002 2024-09-14T06:31:08.853 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:08.854 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:08.854 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:31:08.854 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.416616+0000", 2024-09-14T06:31:08.854 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.8651000000000001e-05 2024-09-14T06:31:08.854 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:08.854 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:08.854 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:31:08.854 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.416691+0000", 2024-09-14T06:31:08.854 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.4888999999999994e-05 2024-09-14T06:31:08.854 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:08.854 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:31:08.854 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:08.854 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:08.855 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:08.855 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:53245 3.f 3:f71f51a7:::rbd_data.112dca2f6ae1.0000000000000015:17 [read 1888256~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:31:08.855 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:44.369410+0000", 2024-09-14T06:31:08.855 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 84.477875065999996, 2024-09-14T06:31:08.855 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047399275999999997, 2024-09-14T06:31:08.855 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:31:08.855 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:31:08.855 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:31:08.855 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:31:08.855 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:31:08.855 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:08.855 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 53245 2024-09-14T06:31:08.855 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:08.855 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:31:08.856 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:08.856 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:31:08.856 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369410+0000", 2024-09-14T06:31:08.856 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:08.856 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:08.856 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:08.856 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:31:08.856 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369410+0000", 2024-09-14T06:31:08.856 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:08.856 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:08.856 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:08.856 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:31:08.856 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369410+0000", 2024-09-14T06:31:08.856 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.1899999999999999e-07 2024-09-14T06:31:08.857 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:08.857 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:08.857 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:31:08.857 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369411+0000", 2024-09-14T06:31:08.857 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.0300000000000001e-06 2024-09-14T06:31:08.857 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:08.857 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:08.857 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:31:08.857 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369414+0000", 2024-09-14T06:31:08.857 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.1099999999999999e-06 2024-09-14T06:31:08.857 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:08.857 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:08.857 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:31:08.858 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369417+0000", 2024-09-14T06:31:08.858 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.6189999999999998e-06 2024-09-14T06:31:08.858 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:08.858 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:08.858 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:31:08.858 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.416718+0000", 2024-09-14T06:31:08.858 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047301322999999999 2024-09-14T06:31:08.858 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:08.858 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:08.858 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:31:08.858 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.416759+0000", 2024-09-14T06:31:08.859 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.1009e-05 2024-09-14T06:31:08.859 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:08.859 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:08.859 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:31:08.859 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.416809+0000", 2024-09-14T06:31:08.859 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.9966000000000002e-05 2024-09-14T06:31:08.859 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:08.859 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:31:08.859 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:08.859 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:08.859 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:08.859 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:53246 3.f 3:f71f51a7:::rbd_data.112dca2f6ae1.0000000000000015:17 [read 1912832~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:31:08.859 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:44.369420+0000", 2024-09-14T06:31:08.860 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 84.477865121999997, 2024-09-14T06:31:08.860 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047529463000000001, 2024-09-14T06:31:08.860 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:31:08.860 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:31:08.860 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:31:08.860 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:31:08.860 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:31:08.860 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:08.860 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 53246 2024-09-14T06:31:08.860 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:08.860 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:31:08.860 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:08.860 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:31:08.860 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369420+0000", 2024-09-14T06:31:08.861 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:08.861 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:08.861 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:08.861 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:31:08.861 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369420+0000", 2024-09-14T06:31:08.861 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:08.861 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:08.861 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:08.861 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:31:08.861 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369420+0000", 2024-09-14T06:31:08.861 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.1299999999999999e-07 2024-09-14T06:31:08.861 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:08.861 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:08.862 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:31:08.862 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369425+0000", 2024-09-14T06:31:08.862 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.0130000000000001e-06 2024-09-14T06:31:08.862 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:08.862 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:08.862 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:31:08.862 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369428+0000", 2024-09-14T06:31:08.862 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.349e-06 2024-09-14T06:31:08.862 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:08.862 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:08.862 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:31:08.862 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369431+0000", 2024-09-14T06:31:08.862 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.7389999999999999e-06 2024-09-14T06:31:08.862 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:08.862 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:08.863 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:31:08.863 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.416834+0000", 2024-09-14T06:31:08.863 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047402669000000001 2024-09-14T06:31:08.863 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:08.863 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:08.863 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:31:08.863 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.416882+0000", 2024-09-14T06:31:08.863 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.8068999999999997e-05 2024-09-14T06:31:08.863 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:08.863 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:08.863 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:31:08.863 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.416949+0000", 2024-09-14T06:31:08.863 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.7410999999999995e-05 2024-09-14T06:31:08.864 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:08.864 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:31:08.864 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:08.864 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:08.864 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:08.864 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:53247 3.f 3:f71f51a7:::rbd_data.112dca2f6ae1.0000000000000015:17 [read 1937408~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:31:08.864 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:44.369434+0000", 2024-09-14T06:31:08.864 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 84.477850357999998, 2024-09-14T06:31:08.864 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047652264999999999, 2024-09-14T06:31:08.864 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:31:08.864 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:31:08.864 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:31:08.864 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:31:08.865 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:31:08.865 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:08.865 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 53247 2024-09-14T06:31:08.865 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:08.865 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:31:08.865 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:08.865 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:31:08.865 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369434+0000", 2024-09-14T06:31:08.865 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:08.865 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:08.865 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:08.865 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:31:08.865 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369434+0000", 2024-09-14T06:31:08.865 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:08.866 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:08.866 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:08.866 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:31:08.866 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369435+0000", 2024-09-14T06:31:08.866 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.2100000000000001e-07 2024-09-14T06:31:08.866 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:08.866 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:08.866 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:31:08.866 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369436+0000", 2024-09-14T06:31:08.866 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.122e-06 2024-09-14T06:31:08.866 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:08.866 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:08.866 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:31:08.866 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369439+0000", 2024-09-14T06:31:08.867 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.0400000000000001e-06 2024-09-14T06:31:08.867 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:08.867 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:08.867 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:31:08.867 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369441+0000", 2024-09-14T06:31:08.867 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.728e-06 2024-09-14T06:31:08.867 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:08.867 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:08.867 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:31:08.867 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.416977+0000", 2024-09-14T06:31:08.867 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047535344 2024-09-14T06:31:08.867 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:08.867 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:08.867 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:31:08.868 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.417029+0000", 2024-09-14T06:31:08.868 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.2037999999999999e-05 2024-09-14T06:31:08.868 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:08.868 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:08.868 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:31:08.868 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.417087+0000", 2024-09-14T06:31:08.868 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.7772000000000002e-05 2024-09-14T06:31:08.868 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:08.868 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:31:08.868 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:08.868 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:08.868 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:08.868 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:53248 3.f 3:f71f51a7:::rbd_data.112dca2f6ae1.0000000000000015:17 [read 1961984~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:31:08.868 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:44.369445+0000", 2024-09-14T06:31:08.869 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 84.477839806000006, 2024-09-14T06:31:08.869 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047798489, 2024-09-14T06:31:08.869 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:31:08.869 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:31:08.869 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:31:08.869 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:31:08.869 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:31:08.869 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:08.869 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 53248 2024-09-14T06:31:08.869 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:08.869 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:31:08.869 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:08.869 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:31:08.870 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369445+0000", 2024-09-14T06:31:08.870 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:08.870 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:08.870 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:08.870 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:31:08.870 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369445+0000", 2024-09-14T06:31:08.870 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:08.870 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:08.870 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:08.870 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:31:08.870 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369445+0000", 2024-09-14T06:31:08.870 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.0699999999999999e-07 2024-09-14T06:31:08.870 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:08.870 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:08.871 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:31:08.871 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369446+0000", 2024-09-14T06:31:08.871 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.1000000000000001e-06 2024-09-14T06:31:08.871 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:08.871 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:08.871 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:31:08.871 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369449+0000", 2024-09-14T06:31:08.871 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.1039999999999998e-06 2024-09-14T06:31:08.871 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:08.871 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:08.871 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:31:08.871 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369452+0000", 2024-09-14T06:31:08.871 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.6199999999999999e-06 2024-09-14T06:31:08.871 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:08.872 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:08.872 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:31:08.872 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.417114+0000", 2024-09-14T06:31:08.872 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047661901999999999 2024-09-14T06:31:08.872 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:08.872 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:08.872 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:31:08.872 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.417181+0000", 2024-09-14T06:31:08.872 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.7385999999999995e-05 2024-09-14T06:31:08.872 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:08.872 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:08.872 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:31:08.872 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.417243+0000", 2024-09-14T06:31:08.872 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.2169999999999996e-05 2024-09-14T06:31:08.873 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:08.873 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:31:08.873 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:08.873 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:08.873 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:08.873 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:53249 3.f 3:f71f51a7:::rbd_data.112dca2f6ae1.0000000000000015:17 [read 1986560~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:31:08.873 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:44.369455+0000", 2024-09-14T06:31:08.873 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 84.477829494999995, 2024-09-14T06:31:08.873 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047890493999999999, 2024-09-14T06:31:08.873 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:31:08.873 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:31:08.873 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:31:08.873 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:31:08.873 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:31:08.874 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:08.874 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 53249 2024-09-14T06:31:08.874 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:08.874 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:31:08.874 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:08.874 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:31:08.874 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369455+0000", 2024-09-14T06:31:08.874 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:08.874 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:08.874 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:08.874 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:31:08.874 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369455+0000", 2024-09-14T06:31:08.874 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:08.874 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:08.874 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:08.875 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:31:08.875 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369455+0000", 2024-09-14T06:31:08.875 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.0800000000000001e-07 2024-09-14T06:31:08.875 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:08.875 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:08.875 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:31:08.875 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369456+0000", 2024-09-14T06:31:08.875 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.0729999999999999e-06 2024-09-14T06:31:08.875 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:08.875 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:08.875 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:31:08.875 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369460+0000", 2024-09-14T06:31:08.875 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.027e-06 2024-09-14T06:31:08.875 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:08.876 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:08.876 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:31:08.876 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369462+0000", 2024-09-14T06:31:08.876 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.5320000000000001e-06 2024-09-14T06:31:08.876 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:08.876 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:08.876 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:31:08.876 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.417257+0000", 2024-09-14T06:31:08.876 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.04779489 2024-09-14T06:31:08.876 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:08.876 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:08.876 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:31:08.876 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.417296+0000", 2024-09-14T06:31:08.876 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.8658000000000002e-05 2024-09-14T06:31:08.877 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:08.877 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:08.877 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:31:08.877 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.417346+0000", 2024-09-14T06:31:08.877 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.0105999999999998e-05 2024-09-14T06:31:08.877 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:08.877 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:31:08.877 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:08.877 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:08.877 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:08.877 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:53250 3.f 3:f71f51a7:::rbd_data.112dca2f6ae1.0000000000000015:17 [read 2011136~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:31:08.877 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:44.369465+0000", 2024-09-14T06:31:08.877 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 84.477819525000001, 2024-09-14T06:31:08.877 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.048032230000000002, 2024-09-14T06:31:08.878 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:31:08.878 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:31:08.878 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:31:08.878 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:31:08.878 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:31:08.878 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:08.878 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 53250 2024-09-14T06:31:08.878 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:08.878 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:31:08.878 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:08.878 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:31:08.878 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369465+0000", 2024-09-14T06:31:08.878 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:08.878 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:08.879 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:08.879 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:31:08.879 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369465+0000", 2024-09-14T06:31:08.879 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:08.879 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:08.879 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:08.879 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:31:08.879 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369465+0000", 2024-09-14T06:31:08.879 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.2100000000000001e-07 2024-09-14T06:31:08.879 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:08.879 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:08.879 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:31:08.879 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369466+0000", 2024-09-14T06:31:08.879 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.0640000000000001e-06 2024-09-14T06:31:08.880 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:08.880 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:08.880 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:31:08.880 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369469+0000", 2024-09-14T06:31:08.880 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.0170000000000001e-06 2024-09-14T06:31:08.880 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:08.880 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:08.880 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:31:08.880 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369472+0000", 2024-09-14T06:31:08.880 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.6189999999999998e-06 2024-09-14T06:31:08.880 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:08.880 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:08.880 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:31:08.881 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.417383+0000", 2024-09-14T06:31:08.881 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047911092000000002 2024-09-14T06:31:08.881 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:08.881 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:08.881 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:31:08.881 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.417430+0000", 2024-09-14T06:31:08.881 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.6898999999999999e-05 2024-09-14T06:31:08.881 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:08.881 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:08.881 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:31:08.881 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.417497+0000", 2024-09-14T06:31:08.881 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.7317999999999995e-05 2024-09-14T06:31:08.881 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:08.881 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:31:08.882 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:08.882 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:08.882 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:08.882 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:53251 3.f 3:f71f51a7:::rbd_data.112dca2f6ae1.0000000000000015:17 [read 2035712~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:31:08.882 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:44.369475+0000", 2024-09-14T06:31:08.882 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 84.477809528999998, 2024-09-14T06:31:08.882 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.048154542000000002, 2024-09-14T06:31:08.882 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:31:08.882 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:31:08.882 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:31:08.882 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:31:08.882 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:31:08.882 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:08.882 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 53251 2024-09-14T06:31:08.883 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:08.883 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:31:08.883 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:08.883 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:31:08.883 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369475+0000", 2024-09-14T06:31:08.883 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:08.883 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:08.883 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:08.883 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:31:08.883 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369475+0000", 2024-09-14T06:31:08.883 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:08.883 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:08.883 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:08.883 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:31:08.884 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369475+0000", 2024-09-14T06:31:08.884 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.0100000000000001e-07 2024-09-14T06:31:08.884 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:08.884 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:08.884 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:31:08.884 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369476+0000", 2024-09-14T06:31:08.884 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.0580000000000001e-06 2024-09-14T06:31:08.884 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:08.884 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:08.884 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:31:08.884 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369479+0000", 2024-09-14T06:31:08.884 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.9950000000000001e-06 2024-09-14T06:31:08.884 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:08.884 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:08.885 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:31:08.885 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369482+0000", 2024-09-14T06:31:08.885 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.6340000000000001e-06 2024-09-14T06:31:08.885 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:08.885 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:08.885 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:31:08.885 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.417530+0000", 2024-09-14T06:31:08.885 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.048047535000000002 2024-09-14T06:31:08.885 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:08.885 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:08.885 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:31:08.885 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.417574+0000", 2024-09-14T06:31:08.885 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.4255999999999998e-05 2024-09-14T06:31:08.885 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:08.885 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:08.886 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:31:08.886 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.417630+0000", 2024-09-14T06:31:08.886 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.5862999999999997e-05 2024-09-14T06:31:08.886 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:08.886 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:31:08.886 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:08.886 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:08.886 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:08.886 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:53252 3.f 3:f71f51a7:::rbd_data.112dca2f6ae1.0000000000000015:17 [read 2060288~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:31:08.886 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:44.369485+0000", 2024-09-14T06:31:08.886 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 84.477799680999993, 2024-09-14T06:31:08.886 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.048254116, 2024-09-14T06:31:08.886 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:31:08.886 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:31:08.887 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:31:08.887 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:31:08.887 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:31:08.887 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:08.887 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 53252 2024-09-14T06:31:08.887 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:08.887 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:31:08.887 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:08.887 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:31:08.887 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369485+0000", 2024-09-14T06:31:08.887 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:08.887 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:08.888 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:08.888 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:31:08.888 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369485+0000", 2024-09-14T06:31:08.888 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:08.888 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:08.888 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:08.888 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:31:08.888 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369485+0000", 2024-09-14T06:31:08.888 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.1e-07 2024-09-14T06:31:08.888 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:08.888 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:08.888 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:31:08.889 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369487+0000", 2024-09-14T06:31:08.889 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.42e-06 2024-09-14T06:31:08.889 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:08.889 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:08.889 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:31:08.889 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369490+0000", 2024-09-14T06:31:08.889 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.9639999999999999e-06 2024-09-14T06:31:08.889 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:08.889 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:08.889 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:31:08.889 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369492+0000", 2024-09-14T06:31:08.889 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.5679999999999998e-06 2024-09-14T06:31:08.889 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:08.889 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:08.890 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:31:08.890 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.417645+0000", 2024-09-14T06:31:08.890 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.048152753 2024-09-14T06:31:08.890 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:08.890 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:08.890 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:31:08.890 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.417687+0000", 2024-09-14T06:31:08.890 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.1900999999999997e-05 2024-09-14T06:31:08.890 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:08.890 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:08.890 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:31:08.890 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.417739+0000", 2024-09-14T06:31:08.890 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.2299999999999997e-05 2024-09-14T06:31:08.890 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:08.890 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:31:08.891 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:08.891 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:08.891 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:08.891 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:53253 3.f 3:f71f51a7:::rbd_data.112dca2f6ae1.0000000000000015:17 [read 2084864~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:31:08.891 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:44.369495+0000", 2024-09-14T06:31:08.891 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 84.477789556999994, 2024-09-14T06:31:08.891 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.048366785000000002, 2024-09-14T06:31:08.891 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:31:08.891 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:31:08.891 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:31:08.891 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:31:08.891 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:31:08.891 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:08.891 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 53253 2024-09-14T06:31:08.892 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:08.892 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:31:08.892 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:08.892 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:31:08.892 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369495+0000", 2024-09-14T06:31:08.892 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:08.892 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:08.892 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:08.892 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:31:08.892 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369495+0000", 2024-09-14T06:31:08.892 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:08.892 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:08.892 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:08.893 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:31:08.893 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370388+0000", 2024-09-14T06:31:08.893 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00089287999999999998 2024-09-14T06:31:08.893 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:08.893 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:08.893 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:31:08.893 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370392+0000", 2024-09-14T06:31:08.893 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.2130000000000002e-06 2024-09-14T06:31:08.893 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:08.893 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:08.893 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:31:08.893 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370401+0000", 2024-09-14T06:31:08.893 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.7649999999999999e-06 2024-09-14T06:31:08.893 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:08.894 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:08.894 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:31:08.894 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370406+0000", 2024-09-14T06:31:08.894 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.0159999999999999e-06 2024-09-14T06:31:08.894 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:08.894 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:08.894 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:31:08.894 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.417764+0000", 2024-09-14T06:31:08.894 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047358348000000001 2024-09-14T06:31:08.894 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:08.894 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:08.894 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:31:08.894 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.417806+0000", 2024-09-14T06:31:08.894 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.1149999999999997e-05 2024-09-14T06:31:08.895 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:08.895 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:08.895 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:31:08.895 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.417862+0000", 2024-09-14T06:31:08.895 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.6412999999999997e-05 2024-09-14T06:31:08.895 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:08.895 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:31:08.895 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:08.895 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:08.895 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:08.895 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:53340 3.f 3:f71f51a7:::rbd_data.112dca2f6ae1.0000000000000015:17 [read 1056768~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:31:08.895 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:44.370421+0000", 2024-09-14T06:31:08.895 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 84.476863676999997, 2024-09-14T06:31:08.895 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047566313999999998, 2024-09-14T06:31:08.895 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:31:08.896 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:31:08.896 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:31:08.896 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:31:08.896 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:31:08.896 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:08.896 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 53340 2024-09-14T06:31:08.896 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:08.896 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:31:08.896 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:08.896 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:31:08.896 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370421+0000", 2024-09-14T06:31:08.896 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:08.896 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:08.897 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:08.897 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:31:08.897 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370421+0000", 2024-09-14T06:31:08.897 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:08.897 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:08.897 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:08.897 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:31:08.897 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370422+0000", 2024-09-14T06:31:08.897 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.3799999999999996e-07 2024-09-14T06:31:08.897 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:08.897 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:08.897 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:31:08.897 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370424+0000", 2024-09-14T06:31:08.897 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.6919999999999998e-06 2024-09-14T06:31:08.897 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:08.898 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:08.898 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:31:08.898 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370432+0000", 2024-09-14T06:31:08.898 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.2439999999999996e-06 2024-09-14T06:31:08.898 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:08.898 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:08.898 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:31:08.898 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370437+0000", 2024-09-14T06:31:08.898 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.5389999999999998e-06 2024-09-14T06:31:08.898 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:08.898 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:08.898 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:31:08.898 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.417888+0000", 2024-09-14T06:31:08.898 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047450432000000001 2024-09-14T06:31:08.899 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:08.899 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:08.899 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:31:08.899 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.417935+0000", 2024-09-14T06:31:08.899 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.7454999999999999e-05 2024-09-14T06:31:08.899 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:08.899 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:08.899 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:31:08.899 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.417987+0000", 2024-09-14T06:31:08.899 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.2213999999999998e-05 2024-09-14T06:31:08.899 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:08.899 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:31:08.899 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:08.899 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:08.900 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:08.900 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:53341 3.f 3:f71f51a7:::rbd_data.112dca2f6ae1.0000000000000015:17 [read 1081344~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:31:08.900 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:44.370446+0000", 2024-09-14T06:31:08.900 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 84.476838404000006, 2024-09-14T06:31:08.900 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047665723, 2024-09-14T06:31:08.900 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:31:08.900 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:31:08.900 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:31:08.900 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:31:08.900 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:31:08.900 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:08.900 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 53341 2024-09-14T06:31:08.900 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:08.900 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:31:08.901 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:08.901 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:31:08.901 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370446+0000", 2024-09-14T06:31:08.901 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:08.901 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:08.901 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:08.901 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:31:08.901 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370446+0000", 2024-09-14T06:31:08.901 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:08.901 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:08.901 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:08.901 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:31:08.901 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370447+0000", 2024-09-14T06:31:08.901 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.6100000000000002e-07 2024-09-14T06:31:08.902 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:08.902 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:08.902 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:31:08.902 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370448+0000", 2024-09-14T06:31:08.902 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.7370000000000001e-06 2024-09-14T06:31:08.902 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:08.902 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:08.902 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:31:08.902 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370462+0000", 2024-09-14T06:31:08.902 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.3227000000000001e-05 2024-09-14T06:31:08.902 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:08.902 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:08.902 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:31:08.902 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370467+0000", 2024-09-14T06:31:08.902 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.6540000000000004e-06 2024-09-14T06:31:08.903 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:08.903 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:08.903 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:31:08.903 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.418013+0000", 2024-09-14T06:31:08.903 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047545325999999999 2024-09-14T06:31:08.903 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:08.903 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:08.903 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:31:08.903 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.418054+0000", 2024-09-14T06:31:08.903 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.1696999999999997e-05 2024-09-14T06:31:08.903 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:08.903 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:08.903 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:31:08.903 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.418112+0000", 2024-09-14T06:31:08.904 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.7720999999999998e-05 2024-09-14T06:31:08.904 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:08.904 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:31:08.904 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:08.904 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:08.904 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:08.904 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:53342 3.f 3:f71f51a7:::rbd_data.112dca2f6ae1.0000000000000015:17 [read 1105920~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:31:08.904 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:44.370475+0000", 2024-09-14T06:31:08.904 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 84.476809342999999, 2024-09-14T06:31:08.904 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047748848000000003, 2024-09-14T06:31:08.904 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:31:08.904 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:31:08.904 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:31:08.904 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:31:08.905 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:31:08.905 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:08.905 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 53342 2024-09-14T06:31:08.905 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:08.905 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:31:08.905 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:08.905 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:31:08.905 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370475+0000", 2024-09-14T06:31:08.905 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:08.905 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:08.905 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:08.905 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:31:08.905 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370475+0000", 2024-09-14T06:31:08.905 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:08.906 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:08.906 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:08.906 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:31:08.906 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370476+0000", 2024-09-14T06:31:08.906 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.4700000000000002e-07 2024-09-14T06:31:08.906 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:08.906 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:08.906 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:31:08.906 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370478+0000", 2024-09-14T06:31:08.906 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.5890000000000001e-06 2024-09-14T06:31:08.906 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:08.906 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:08.906 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:31:08.906 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370486+0000", 2024-09-14T06:31:08.907 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.3470000000000001e-06 2024-09-14T06:31:08.907 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:08.907 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:08.907 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:31:08.907 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370491+0000", 2024-09-14T06:31:08.907 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.5720000000000002e-06 2024-09-14T06:31:08.907 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:08.907 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:08.907 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:31:08.907 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.418135+0000", 2024-09-14T06:31:08.907 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047644094999999997 2024-09-14T06:31:08.907 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:08.907 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:08.907 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:31:08.908 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.418174+0000", 2024-09-14T06:31:08.908 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.8204000000000002e-05 2024-09-14T06:31:08.908 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:08.908 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:08.908 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:31:08.908 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.418224+0000", 2024-09-14T06:31:08.908 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.0593999999999997e-05 2024-09-14T06:31:08.908 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:08.908 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:31:08.908 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:08.908 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:08.908 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:08.908 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:53343 3.f 3:f71f51a7:::rbd_data.112dca2f6ae1.0000000000000015:17 [read 1130496~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:31:08.908 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:44.370498+0000", 2024-09-14T06:31:08.909 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 84.476786466999997, 2024-09-14T06:31:08.909 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047841574999999997, 2024-09-14T06:31:08.909 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:31:08.909 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:31:08.909 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:31:08.909 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:31:08.909 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:31:08.909 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:08.909 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 53343 2024-09-14T06:31:08.909 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:08.909 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:31:08.909 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:08.909 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:31:08.909 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370498+0000", 2024-09-14T06:31:08.910 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:08.910 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:08.910 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:08.910 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:31:08.910 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370498+0000", 2024-09-14T06:31:08.910 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:08.910 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:08.910 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:08.910 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:31:08.910 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370499+0000", 2024-09-14T06:31:08.910 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.9999999999999998e-07 2024-09-14T06:31:08.910 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:08.910 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:08.910 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:31:08.911 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370501+0000", 2024-09-14T06:31:08.911 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.0310000000000001e-06 2024-09-14T06:31:08.911 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:08.911 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:08.911 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:31:08.911 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370507+0000", 2024-09-14T06:31:08.911 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.4169999999999997e-06 2024-09-14T06:31:08.911 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:08.911 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:08.911 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:31:08.911 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370516+0000", 2024-09-14T06:31:08.911 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.7190000000000007e-06 2024-09-14T06:31:08.911 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:08.911 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:08.912 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:31:08.912 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.418247+0000", 2024-09-14T06:31:08.912 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047731190999999999 2024-09-14T06:31:08.912 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:08.912 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:08.912 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:31:08.912 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.418286+0000", 2024-09-14T06:31:08.912 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.8988e-05 2024-09-14T06:31:08.912 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:08.912 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:08.912 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:31:08.912 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.418340+0000", 2024-09-14T06:31:08.912 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.3829000000000002e-05 2024-09-14T06:31:08.913 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:08.913 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:31:08.913 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:08.913 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:08.913 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:08.913 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:53344 3.f 3:f71f51a7:::rbd_data.112dca2f6ae1.0000000000000015:17 [read 1155072~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:31:08.913 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:44.370524+0000", 2024-09-14T06:31:08.913 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 84.476760697000003, 2024-09-14T06:31:08.913 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047972262000000002, 2024-09-14T06:31:08.913 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:31:08.913 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:31:08.913 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:31:08.913 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:31:08.913 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:31:08.914 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:08.914 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 53344 2024-09-14T06:31:08.914 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:08.914 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:31:08.914 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:08.914 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:31:08.914 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370524+0000", 2024-09-14T06:31:08.914 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:08.914 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:08.914 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:08.914 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:31:08.914 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370524+0000", 2024-09-14T06:31:08.914 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:08.915 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:08.915 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:08.915 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:31:08.915 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370618+0000", 2024-09-14T06:31:08.915 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.4425999999999993e-05 2024-09-14T06:31:08.915 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:08.915 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:08.915 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:31:08.915 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370622+0000", 2024-09-14T06:31:08.915 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.3519999999999998e-06 2024-09-14T06:31:08.915 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:08.915 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:08.915 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:31:08.915 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370631+0000", 2024-09-14T06:31:08.916 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.9299999999999992e-06 2024-09-14T06:31:08.916 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:08.916 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:08.916 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:31:08.916 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370637+0000", 2024-09-14T06:31:08.916 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.3269999999999997e-06 2024-09-14T06:31:08.916 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:08.916 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:08.916 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:31:08.916 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.418372+0000", 2024-09-14T06:31:08.916 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047734764999999998 2024-09-14T06:31:08.916 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:08.916 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:08.916 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:31:08.917 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.418431+0000", 2024-09-14T06:31:08.917 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.8696999999999997e-05 2024-09-14T06:31:08.917 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:08.917 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:08.917 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:31:08.917 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.418496+0000", 2024-09-14T06:31:08.917 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.5765000000000005e-05 2024-09-14T06:31:08.917 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:08.917 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:31:08.917 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:08.917 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:08.917 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:08.917 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:53345 3.f 3:f71f51a7:::rbd_data.112dca2f6ae1.0000000000000015:17 [read 1179648~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:31:08.918 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:44.370647+0000", 2024-09-14T06:31:08.918 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 84.476637229999994, 2024-09-14T06:31:08.918 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047953775999999997, 2024-09-14T06:31:08.918 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:31:08.918 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:31:08.918 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:31:08.918 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:31:08.918 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:31:08.918 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:08.918 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 53345 2024-09-14T06:31:08.918 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:08.918 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:31:08.918 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:08.918 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:31:08.919 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370647+0000", 2024-09-14T06:31:08.919 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:08.919 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:08.919 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:08.919 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:31:08.919 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370647+0000", 2024-09-14T06:31:08.919 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:08.919 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:08.919 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:08.919 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:31:08.919 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.372346+0000", 2024-09-14T06:31:08.919 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.001698284 2024-09-14T06:31:08.919 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:08.920 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:08.920 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:31:08.920 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.372351+0000", 2024-09-14T06:31:08.920 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.8890000000000001e-06 2024-09-14T06:31:08.920 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:08.920 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:08.920 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:31:08.920 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.372362+0000", 2024-09-14T06:31:08.920 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.0941000000000001e-05 2024-09-14T06:31:08.920 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:08.920 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:08.920 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:31:08.920 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.372369+0000", 2024-09-14T06:31:08.921 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.362e-06 2024-09-14T06:31:08.921 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:08.921 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:08.921 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:31:08.921 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.418510+0000", 2024-09-14T06:31:08.921 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046140914999999998 2024-09-14T06:31:08.921 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:08.921 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:08.921 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:31:08.921 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.418549+0000", 2024-09-14T06:31:08.921 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.8717999999999998e-05 2024-09-14T06:31:08.921 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:08.921 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:08.922 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:31:08.922 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.418601+0000", 2024-09-14T06:31:08.922 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.2667000000000002e-05 2024-09-14T06:31:08.922 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:08.922 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:31:08.922 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:08.922 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:08.922 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:08.922 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:53351 3.f 3:f71f51a7:::rbd_data.112dca2f6ae1.0000000000000015:17 [read 1327104~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:31:08.922 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:44.372499+0000", 2024-09-14T06:31:08.922 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 84.474786108999993, 2024-09-14T06:31:08.922 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047140596, 2024-09-14T06:31:08.922 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:31:08.922 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:31:08.923 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:31:08.923 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:31:08.923 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:31:08.923 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:08.923 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 53351 2024-09-14T06:31:08.923 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:08.923 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:31:08.923 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:08.923 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:31:08.923 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.372499+0000", 2024-09-14T06:31:08.923 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:08.923 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:08.923 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:08.923 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:31:08.924 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.372499+0000", 2024-09-14T06:31:08.924 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:08.924 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:08.924 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:08.924 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:31:08.924 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.372499+0000", 2024-09-14T06:31:08.924 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.3000000000000001e-07 2024-09-14T06:31:08.924 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:08.924 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:08.924 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:31:08.924 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.372501+0000", 2024-09-14T06:31:08.924 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.0820000000000001e-06 2024-09-14T06:31:08.924 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:08.925 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:08.925 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:31:08.925 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.372513+0000", 2024-09-14T06:31:08.925 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.1610999999999999e-05 2024-09-14T06:31:08.925 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:08.925 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:08.925 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:31:08.925 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.372518+0000", 2024-09-14T06:31:08.925 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.327e-06 2024-09-14T06:31:08.925 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:08.925 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:08.925 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:31:08.925 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.419530+0000", 2024-09-14T06:31:08.925 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047011798 2024-09-14T06:31:08.926 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:08.926 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:08.926 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:31:08.926 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.419578+0000", 2024-09-14T06:31:08.926 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.8446999999999999e-05 2024-09-14T06:31:08.926 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:08.926 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:08.926 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:31:08.926 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.419639+0000", 2024-09-14T06:31:08.926 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.0900999999999998e-05 2024-09-14T06:31:08.926 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:08.926 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:31:08.926 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:08.927 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:08.927 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:08.927 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:53352 3.f 3:f71f51a7:::rbd_data.112dca2f6ae1.0000000000000015:17 [read 1351680~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:31:08.927 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:44.372525+0000", 2024-09-14T06:31:08.927 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 84.474760195000002, 2024-09-14T06:31:08.927 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047259847000000001, 2024-09-14T06:31:08.927 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:31:08.927 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:31:08.927 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:31:08.927 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:31:08.927 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:31:08.927 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:08.927 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 53352 2024-09-14T06:31:08.927 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:08.928 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:31:08.928 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:08.928 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:31:08.928 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.372525+0000", 2024-09-14T06:31:08.928 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:08.928 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:08.928 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:08.928 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:31:08.928 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.372525+0000", 2024-09-14T06:31:08.928 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:08.928 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:08.928 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:08.928 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:31:08.929 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.372525+0000", 2024-09-14T06:31:08.929 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.0600000000000001e-07 2024-09-14T06:31:08.929 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:08.929 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:08.929 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:31:08.929 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.372527+0000", 2024-09-14T06:31:08.929 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.0499999999999999e-06 2024-09-14T06:31:08.929 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:08.929 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:08.929 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:31:08.929 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.372533+0000", 2024-09-14T06:31:08.929 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.4320000000000004e-06 2024-09-14T06:31:08.929 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:08.930 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:08.930 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:31:08.930 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.372538+0000", 2024-09-14T06:31:08.930 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.9130000000000002e-06 2024-09-14T06:31:08.930 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:08.930 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:08.930 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:31:08.930 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.419667+0000", 2024-09-14T06:31:08.930 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047128976000000003 2024-09-14T06:31:08.930 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:08.930 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:08.930 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:31:08.930 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.419715+0000", 2024-09-14T06:31:08.930 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.7868999999999999e-05 2024-09-14T06:31:08.931 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:08.931 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:08.931 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:31:08.931 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.419784+0000", 2024-09-14T06:31:08.931 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.9201000000000004e-05 2024-09-14T06:31:08.931 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:08.931 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:31:08.931 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:08.931 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:08.931 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:08.931 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:53353 3.f 3:f71f51a7:::rbd_data.112dca2f6ae1.0000000000000015:17 [read 1376256~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:31:08.931 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:44.372545+0000", 2024-09-14T06:31:08.931 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 84.474740155000006, 2024-09-14T06:31:08.931 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047364232999999999, 2024-09-14T06:31:08.932 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:31:08.932 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:31:08.932 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:31:08.932 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:31:08.932 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:31:08.932 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:08.932 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 53353 2024-09-14T06:31:08.932 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:08.932 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:31:08.932 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:08.932 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:31:08.932 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.372545+0000", 2024-09-14T06:31:08.932 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:08.932 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:08.933 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:08.933 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:31:08.933 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.372545+0000", 2024-09-14T06:31:08.933 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:08.933 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:08.933 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:08.933 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:31:08.933 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.372545+0000", 2024-09-14T06:31:08.933 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.27e-07 2024-09-14T06:31:08.933 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:08.933 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:08.933 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:31:08.933 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.372547+0000", 2024-09-14T06:31:08.934 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.0600000000000002e-06 2024-09-14T06:31:08.934 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:08.934 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:08.934 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:31:08.934 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.372553+0000", 2024-09-14T06:31:08.934 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.7980000000000002e-06 2024-09-14T06:31:08.934 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:08.934 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:08.934 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:31:08.934 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.372557+0000", 2024-09-14T06:31:08.934 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.7389999999999999e-06 2024-09-14T06:31:08.934 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:08.934 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:08.934 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:31:08.935 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.419811+0000", 2024-09-14T06:31:08.935 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047253310999999999 2024-09-14T06:31:08.935 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:08.935 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:08.935 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:31:08.935 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.419854+0000", 2024-09-14T06:31:08.935 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.3448000000000001e-05 2024-09-14T06:31:08.935 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:08.935 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:08.935 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:31:08.935 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.419909+0000", 2024-09-14T06:31:08.935 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.4549999999999998e-05 2024-09-14T06:31:08.935 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:08.935 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:31:08.936 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:08.936 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:08.936 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:08.936 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:53354 3.f 3:f71f51a7:::rbd_data.112dca2f6ae1.0000000000000015:17 [read 1400832~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:31:08.936 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:44.372563+0000", 2024-09-14T06:31:08.936 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 84.474721717999998, 2024-09-14T06:31:08.936 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047482233999999998, 2024-09-14T06:31:08.936 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:31:08.936 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:31:08.936 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:31:08.936 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:31:08.936 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:31:08.936 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:08.937 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 53354 2024-09-14T06:31:08.937 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:08.937 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:31:08.937 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:08.937 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:31:08.937 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.372563+0000", 2024-09-14T06:31:08.937 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:08.937 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:08.937 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:08.937 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:31:08.937 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.372563+0000", 2024-09-14T06:31:08.937 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:08.937 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:08.937 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:08.938 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:31:08.938 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.375009+0000", 2024-09-14T06:31:08.938 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.0024457680000000001 2024-09-14T06:31:08.938 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:08.938 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:08.938 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:31:08.938 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.375014+0000", 2024-09-14T06:31:08.938 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.0599999999999998e-06 2024-09-14T06:31:08.938 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:08.938 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:08.938 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:31:08.938 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.375026+0000", 2024-09-14T06:31:08.938 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.1783999999999999e-05 2024-09-14T06:31:08.939 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:08.939 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:08.939 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:31:08.939 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.375035+0000", 2024-09-14T06:31:08.939 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.7799999999999995e-06 2024-09-14T06:31:08.939 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:08.939 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:08.939 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:31:08.939 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.419942+0000", 2024-09-14T06:31:08.939 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.044906982999999998 2024-09-14T06:31:08.939 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:08.939 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:08.939 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:31:08.939 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.419989+0000", 2024-09-14T06:31:08.940 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.7104999999999997e-05 2024-09-14T06:31:08.940 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:08.940 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:08.940 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:31:08.940 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.420045+0000", 2024-09-14T06:31:08.940 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.5754000000000002e-05 2024-09-14T06:31:08.940 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:08.940 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:31:08.940 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:08.940 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:08.940 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:31:08.940 INFO:teuthology.orchestra.run.smithi138.stdout:} 2024-09-14T06:31:08.942 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-14T06:31:08.942 INFO:tasks.ceph.osd.1.smithi094.stderr:2024-09-14T06:31:08.941+0000 7f328e47a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33641) UID: 0 2024-09-14T06:31:09.002 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-14T06:31:09.043 INFO:tasks.ceph.osd.6.smithi138.stderr:2024-09-14T06:31:09.042+0000 7f8cd8849640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33620) UID: 0 2024-09-14T06:31:09.047 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-14T06:31:09.047 INFO:teuthology.orchestra.run.smithi094.stdout: "ops": [], 2024-09-14T06:31:09.048 INFO:teuthology.orchestra.run.smithi094.stdout: "num_ops": 0 2024-09-14T06:31:09.048 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-14T06:31:09.057 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-14T06:31:09.127 INFO:teuthology.orchestra.run.smithi094.stdout:osd.0: {} 2024-09-14T06:31:09.127 INFO:teuthology.orchestra.run.smithi094.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-14T06:31:09.143 INFO:tasks.ceph.osd.6.smithi138.stderr:2024-09-14T06:31:09.142+0000 7f8cd8849640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33620) UID: 0 2024-09-14T06:31:09.145 INFO:teuthology.orchestra.run.smithi094.stdout:osd.1: {} 2024-09-14T06:31:09.145 INFO:teuthology.orchestra.run.smithi094.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-14T06:31:09.162 INFO:teuthology.orchestra.run.smithi094.stdout:osd.2: {} 2024-09-14T06:31:09.162 INFO:teuthology.orchestra.run.smithi094.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-14T06:31:09.165 INFO:teuthology.orchestra.run.smithi094.stdout:ERROR: (22) Invalid argument 2024-09-14T06:31:09.166 INFO:teuthology.orchestra.run.smithi094.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:31:09.175 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-14T06:31:09.179 INFO:teuthology.orchestra.run.smithi094.stdout:osd.3: {} 2024-09-14T06:31:09.179 INFO:teuthology.orchestra.run.smithi094.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-14T06:31:09.197 INFO:teuthology.orchestra.run.smithi094.stdout:osd.4: {} 2024-09-14T06:31:09.198 INFO:teuthology.orchestra.run.smithi094.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-14T06:31:09.214 INFO:teuthology.orchestra.run.smithi094.stdout:osd.5: {} 2024-09-14T06:31:09.214 INFO:teuthology.orchestra.run.smithi094.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-14T06:31:09.231 INFO:teuthology.orchestra.run.smithi094.stdout:osd.6: {} 2024-09-14T06:31:09.231 INFO:teuthology.orchestra.run.smithi094.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-14T06:31:09.245 INFO:tasks.ceph.osd.5.smithi138.stderr:2024-09-14T06:31:09.243+0000 7fe13746f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33618) UID: 0 2024-09-14T06:31:09.248 INFO:teuthology.orchestra.run.smithi094.stdout:osd.7: {} 2024-09-14T06:31:09.248 INFO:teuthology.orchestra.run.smithi094.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-14T06:31:09.280 INFO:teuthology.orchestra.run.smithi094.stdout:ERROR: (22) Invalid argument 2024-09-14T06:31:09.280 INFO:teuthology.orchestra.run.smithi094.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:31:09.289 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-14T06:31:09.346 INFO:tasks.ceph.osd.7.smithi138.stderr:2024-09-14T06:31:09.345+0000 7f3c6a21d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33623) UID: 0 2024-09-14T06:31:09.432 INFO:teuthology.orchestra.run.smithi094.stdout:ERROR: (22) Invalid argument 2024-09-14T06:31:09.432 INFO:teuthology.orchestra.run.smithi094.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:31:09.441 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-14T06:31:09.447 INFO:tasks.ceph.osd.4.smithi138.stderr:2024-09-14T06:31:09.446+0000 7f6c653c8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33615) UID: 0 2024-09-14T06:31:09.542 INFO:teuthology.orchestra.run.smithi094.stdout:ERROR: (22) Invalid argument 2024-09-14T06:31:09.542 INFO:teuthology.orchestra.run.smithi094.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:31:09.548 INFO:tasks.ceph.osd.2.smithi094.stderr:2024-09-14T06:31:09.547+0000 7faf3269b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33649) UID: 0 2024-09-14T06:31:09.551 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-14T06:31:09.649 INFO:tasks.ceph.osd.5.smithi138.stderr:2024-09-14T06:31:09.648+0000 7fe13746f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33618) UID: 0 2024-09-14T06:31:09.694 INFO:teuthology.orchestra.run.smithi094.stdout:ERROR: (22) Invalid argument 2024-09-14T06:31:09.694 INFO:teuthology.orchestra.run.smithi094.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:31:09.703 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-14T06:31:09.750 INFO:tasks.ceph.osd.2.smithi094.stderr:2024-09-14T06:31:09.749+0000 7faf3269b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33649) UID: 0 2024-09-14T06:31:09.805 INFO:teuthology.orchestra.run.smithi094.stdout:ERROR: (22) Invalid argument 2024-09-14T06:31:09.806 INFO:teuthology.orchestra.run.smithi094.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:31:09.814 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-14T06:31:09.819 INFO:teuthology.orchestra.run.smithi094.stderr:nodeep-scrub is unset 2024-09-14T06:31:09.851 INFO:tasks.ceph.osd.6.smithi138.stderr:2024-09-14T06:31:09.850+0000 7f8cd8849640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33620) UID: 0 2024-09-14T06:31:09.930 INFO:teuthology.orchestra.run.smithi094.stdout:ERROR: (22) Invalid argument 2024-09-14T06:31:09.930 INFO:teuthology.orchestra.run.smithi094.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:31:09.939 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-14T06:31:09.946 INFO:teuthology.orchestra.run.smithi094.stdout:checking clone #31, image image_client.0-clone31 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent32 2024-09-14T06:31:09.952 INFO:tasks.ceph.osd.5.smithi138.stderr:2024-09-14T06:31:09.951+0000 7fe13746f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33618) UID: 0 2024-09-14T06:31:10.053 INFO:teuthology.orchestra.run.smithi094.stdout:ERROR: (22) Invalid argument 2024-09-14T06:31:10.053 INFO:teuthology.orchestra.run.smithi094.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:31:10.053 INFO:tasks.ceph.osd.0.smithi094.stderr:2024-09-14T06:31:10.052+0000 7f07c9333640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33636) UID: 0 2024-09-14T06:31:10.062 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-14T06:31:10.071 INFO:teuthology.orchestra.run.smithi094.stdout:4344 punch from 0x298443d to 0x29884f8, (0x40bb bytes) 2024-09-14T06:31:10.081 INFO:teuthology.orchestra.run.smithi094.stdout:4346 write 0xadc2aab thru 0xadcfc54 (0xd1aa bytes) 2024-09-14T06:31:10.093 INFO:teuthology.orchestra.run.smithi094.stdout:4347 writesame 0xd57d567 thru 0xd580fea data_size 0x23(0x3a84 bytes) 2024-09-14T06:31:10.093 INFO:teuthology.orchestra.run.smithi094.stdout:4349 punch from 0x476e6c to 0x47951c, (0x26b0 bytes) 2024-09-14T06:31:10.094 INFO:teuthology.orchestra.run.smithi094.stdout:4351 write 0x682eb18 thru 0x68368a7 (0x7d90 bytes) 2024-09-14T06:31:10.105 INFO:teuthology.orchestra.run.smithi094.stdout:4352 trunc from 0xd580feb to 0x4879d36 2024-09-14T06:31:10.120 INFO:teuthology.orchestra.run.smithi094.stdout:4353 clone 34 order 18 su 8192 sc 4 2024-09-14T06:31:10.154 INFO:tasks.ceph.osd.3.smithi094.stderr:2024-09-14T06:31:10.153+0000 7feeeafe8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33655) UID: 0 2024-09-14T06:31:10.177 INFO:teuthology.orchestra.run.smithi094.stdout:ERROR: (22) Invalid argument 2024-09-14T06:31:10.177 INFO:teuthology.orchestra.run.smithi094.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:31:10.186 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-14T06:31:10.255 INFO:tasks.ceph.osd.5.smithi138.stderr:2024-09-14T06:31:10.254+0000 7fe13746f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33618) UID: 0 2024-09-14T06:31:10.331 INFO:teuthology.orchestra.run.smithi094.stdout:ERROR: (22) Invalid argument 2024-09-14T06:31:10.331 INFO:teuthology.orchestra.run.smithi094.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:31:10.341 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-14T06:31:10.384 INFO:tasks.ceph.osd.1.smithi094.stderr:2024-09-14T06:31:10.383+0000 7f328e47a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33641) UID: 0 2024-09-14T06:31:10.443 INFO:teuthology.orchestra.run.smithi094.stdout:ERROR: (22) Invalid argument 2024-09-14T06:31:10.444 INFO:teuthology.orchestra.run.smithi094.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:31:10.452 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-14T06:31:10.495 INFO:tasks.ceph.osd.6.smithi138.stderr:2024-09-14T06:31:10.494+0000 7f8cd8849640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33620) UID: 0 2024-09-14T06:31:10.555 INFO:teuthology.orchestra.run.smithi138.stdout:ERROR: (22) Invalid argument 2024-09-14T06:31:10.556 INFO:teuthology.orchestra.run.smithi138.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:31:10.558 INFO:tasks.ceph.osd.3.smithi094.stderr:2024-09-14T06:31:10.557+0000 7feeeafe8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33655) UID: 0 2024-09-14T06:31:10.564 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-14T06:31:10.659 INFO:tasks.ceph.osd.1.smithi094.stderr:2024-09-14T06:31:10.658+0000 7f328e47a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33641) UID: 0 2024-09-14T06:31:10.709 INFO:teuthology.orchestra.run.smithi138.stdout:ERROR: (22) Invalid argument 2024-09-14T06:31:10.709 INFO:teuthology.orchestra.run.smithi138.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:31:10.717 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-14T06:31:10.759 INFO:tasks.ceph.osd.3.smithi094.stderr:2024-09-14T06:31:10.758+0000 7feeeafe8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33655) UID: 0 2024-09-14T06:31:10.820 INFO:teuthology.orchestra.run.smithi138.stdout:ERROR: (22) Invalid argument 2024-09-14T06:31:10.820 INFO:teuthology.orchestra.run.smithi138.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:31:10.829 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-14T06:31:10.872 INFO:tasks.ceph.osd.4.smithi138.stderr:2024-09-14T06:31:10.871+0000 7f6c653c8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33615) UID: 0 2024-09-14T06:31:10.931 INFO:teuthology.orchestra.run.smithi138.stdout:ERROR: (22) Invalid argument 2024-09-14T06:31:10.931 INFO:teuthology.orchestra.run.smithi138.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:31:10.939 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-14T06:31:10.980 INFO:tasks.ceph.osd.6.smithi138.stderr:2024-09-14T06:31:10.979+0000 7f8cd8849640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33620) UID: 0 2024-09-14T06:31:11.060 INFO:tasks.ceph.osd.7.smithi138.stderr:2024-09-14T06:31:11.059+0000 7f3c6a21d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33623) UID: 0 2024-09-14T06:31:11.081 INFO:teuthology.orchestra.run.smithi138.stdout:ERROR: (22) Invalid argument 2024-09-14T06:31:11.081 INFO:teuthology.orchestra.run.smithi138.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:31:11.089 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-14T06:31:11.162 INFO:tasks.ceph.osd.0.smithi094.stderr:2024-09-14T06:31:11.161+0000 7f07c9333640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33636) UID: 0 2024-09-14T06:31:11.232 INFO:teuthology.orchestra.run.smithi138.stdout:ERROR: (22) Invalid argument 2024-09-14T06:31:11.233 INFO:teuthology.orchestra.run.smithi138.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:31:11.241 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-14T06:31:11.259 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-14T06:31:11.284 INFO:tasks.ceph.osd.4.smithi138.stderr:2024-09-14T06:31:11.283+0000 7f6c653c8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33615) UID: 0 2024-09-14T06:31:11.344 INFO:teuthology.orchestra.run.smithi138.stdout:ERROR: (22) Invalid argument 2024-09-14T06:31:11.344 INFO:teuthology.orchestra.run.smithi138.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:31:11.354 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-14T06:31:11.381 INFO:teuthology.orchestra.run.smithi094.stdout:osd.0: {} 2024-09-14T06:31:11.381 INFO:teuthology.orchestra.run.smithi094.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-14T06:31:11.395 INFO:tasks.ceph.osd.4.smithi138.stderr:2024-09-14T06:31:11.394+0000 7f6c653c8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33615) UID: 0 2024-09-14T06:31:11.397 INFO:teuthology.orchestra.run.smithi094.stdout:osd.1: {} 2024-09-14T06:31:11.397 INFO:teuthology.orchestra.run.smithi094.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-14T06:31:11.413 INFO:teuthology.orchestra.run.smithi094.stdout:osd.2: {} 2024-09-14T06:31:11.414 INFO:teuthology.orchestra.run.smithi094.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-14T06:31:11.429 INFO:teuthology.orchestra.run.smithi094.stdout:osd.3: {} 2024-09-14T06:31:11.429 INFO:teuthology.orchestra.run.smithi094.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-14T06:31:11.446 INFO:teuthology.orchestra.run.smithi094.stdout:osd.4: {} 2024-09-14T06:31:11.446 INFO:teuthology.orchestra.run.smithi094.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-14T06:31:11.462 INFO:teuthology.orchestra.run.smithi094.stdout:osd.5: {} 2024-09-14T06:31:11.463 INFO:teuthology.orchestra.run.smithi094.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-14T06:31:11.464 INFO:tasks.ceph.osd.7.smithi138.stderr:2024-09-14T06:31:11.463+0000 7f3c6a21d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33623) UID: 0 2024-09-14T06:31:11.479 INFO:teuthology.orchestra.run.smithi094.stdout:osd.6: {} 2024-09-14T06:31:11.479 INFO:teuthology.orchestra.run.smithi094.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-14T06:31:11.495 INFO:teuthology.orchestra.run.smithi094.stdout:osd.7: {} 2024-09-14T06:31:11.496 INFO:teuthology.orchestra.run.smithi094.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-14T06:31:11.498 INFO:teuthology.orchestra.run.smithi138.stdout:{ 2024-09-14T06:31:11.498 INFO:teuthology.orchestra.run.smithi138.stdout: "ops": [], 2024-09-14T06:31:11.498 INFO:teuthology.orchestra.run.smithi138.stdout: "complaint_time": 30, 2024-09-14T06:31:11.498 INFO:teuthology.orchestra.run.smithi138.stdout: "num_blocked_ops": 0 2024-09-14T06:31:11.498 INFO:teuthology.orchestra.run.smithi138.stdout:} 2024-09-14T06:31:11.507 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-14T06:31:11.564 INFO:tasks.ceph.osd.2.smithi094.stderr:2024-09-14T06:31:11.564+0000 7faf3269b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33649) UID: 0 2024-09-14T06:31:11.657 INFO:teuthology.orchestra.run.smithi138.stdout:{ 2024-09-14T06:31:11.658 INFO:teuthology.orchestra.run.smithi138.stdout: "size": 20, 2024-09-14T06:31:11.658 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 600, 2024-09-14T06:31:11.658 INFO:teuthology.orchestra.run.smithi138.stdout: "ops": [ 2024-09-14T06:31:11.658 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:11.658 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:98505 3.10 3.1bf56f90 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T06:31:11.658 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:30:07.045453+0000", 2024-09-14T06:31:11.658 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 64.610575998000002, 2024-09-14T06:31:11.658 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032144440000000003, 2024-09-14T06:31:11.658 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:31:11.658 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:31:11.658 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "reached pg", 2024-09-14T06:31:11.658 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:31:11.659 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:31:11.659 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:11.659 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 98505 2024-09-14T06:31:11.659 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:11.659 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:31:11.659 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:11.659 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:31:11.659 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045453+0000", 2024-09-14T06:31:11.659 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:11.659 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:11.659 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:11.659 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:31:11.659 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045453+0000", 2024-09-14T06:31:11.659 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:11.660 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:11.660 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:11.660 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:31:11.660 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045453+0000", 2024-09-14T06:31:11.660 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.36e-07 2024-09-14T06:31:11.660 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:11.660 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:11.660 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:31:11.660 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045454+0000", 2024-09-14T06:31:11.660 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.1299999999999998e-07 2024-09-14T06:31:11.660 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:11.660 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:11.660 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:31:11.661 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045457+0000", 2024-09-14T06:31:11.661 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.8559999999999998e-06 2024-09-14T06:31:11.661 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:11.661 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:11.661 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:31:11.661 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045460+0000", 2024-09-14T06:31:11.661 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.8750000000000002e-06 2024-09-14T06:31:11.661 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:11.661 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:11.661 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:31:11.661 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.077592+0000", 2024-09-14T06:31:11.661 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032131727999999998 2024-09-14T06:31:11.661 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:11.661 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:11.662 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:31:11.662 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.077597+0000", 2024-09-14T06:31:11.662 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.8319999999999996e-06 2024-09-14T06:31:11.662 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:11.662 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:31:11.662 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:11.662 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:11.662 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:11.662 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:98506 3.10 3.1bf56f90 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T06:31:11.662 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:30:07.045481+0000", 2024-09-14T06:31:11.662 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 64.610547742999998, 2024-09-14T06:31:11.662 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032142759999999999, 2024-09-14T06:31:11.662 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:31:11.662 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:31:11.663 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "reached pg", 2024-09-14T06:31:11.663 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:31:11.663 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:31:11.663 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:11.663 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 98506 2024-09-14T06:31:11.663 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:11.663 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:31:11.663 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:11.663 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:31:11.663 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045481+0000", 2024-09-14T06:31:11.663 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:11.663 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:11.663 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:11.663 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:31:11.664 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045481+0000", 2024-09-14T06:31:11.664 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:11.664 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:11.664 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:11.664 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:31:11.664 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045481+0000", 2024-09-14T06:31:11.664 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.8500000000000001e-07 2024-09-14T06:31:11.664 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:11.664 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:11.664 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:31:11.664 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045483+0000", 2024-09-14T06:31:11.664 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.8250000000000001e-06 2024-09-14T06:31:11.664 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:11.665 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:11.665 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:31:11.665 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045489+0000", 2024-09-14T06:31:11.665 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.5740000000000003e-06 2024-09-14T06:31:11.665 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:11.665 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:11.665 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:31:11.665 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045495+0000", 2024-09-14T06:31:11.665 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.7690000000000002e-06 2024-09-14T06:31:11.665 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:11.665 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:11.665 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:31:11.665 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.077619+0000", 2024-09-14T06:31:11.665 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032124621999999999 2024-09-14T06:31:11.666 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:11.666 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:11.666 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:31:11.666 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.077624+0000", 2024-09-14T06:31:11.666 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.3850000000000004e-06 2024-09-14T06:31:11.666 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:11.666 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:31:11.666 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:11.666 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:11.666 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:11.666 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:98507 3.10 3.1bf56f90 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T06:31:11.666 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:30:07.045502+0000", 2024-09-14T06:31:11.666 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 64.610526321999998, 2024-09-14T06:31:11.667 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032147219999999997, 2024-09-14T06:31:11.667 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:31:11.667 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:31:11.667 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "reached pg", 2024-09-14T06:31:11.667 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:31:11.667 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:31:11.667 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:11.667 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 98507 2024-09-14T06:31:11.667 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:11.667 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:31:11.667 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:11.667 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:31:11.667 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045502+0000", 2024-09-14T06:31:11.667 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:11.668 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:11.668 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:11.668 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:31:11.668 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045502+0000", 2024-09-14T06:31:11.668 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:11.668 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:11.668 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:11.668 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:31:11.668 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045503+0000", 2024-09-14T06:31:11.668 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.4700000000000002e-07 2024-09-14T06:31:11.668 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:11.668 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:11.668 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:31:11.668 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045509+0000", 2024-09-14T06:31:11.668 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.7329999999999997e-06 2024-09-14T06:31:11.669 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:11.669 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:11.669 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:31:11.669 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045515+0000", 2024-09-14T06:31:11.669 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.8030000000000002e-06 2024-09-14T06:31:11.669 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:11.669 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:11.669 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:31:11.669 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045522+0000", 2024-09-14T06:31:11.669 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.0369999999999997e-06 2024-09-14T06:31:11.669 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:11.669 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:11.669 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:31:11.669 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.077641+0000", 2024-09-14T06:31:11.670 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032119091000000002 2024-09-14T06:31:11.670 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:11.670 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:11.670 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:31:11.670 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.077650+0000", 2024-09-14T06:31:11.670 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.2090000000000005e-06 2024-09-14T06:31:11.670 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:11.670 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:31:11.670 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:11.670 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:11.670 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:11.670 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:98508 3.10 3.1bf56f90 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T06:31:11.670 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:30:07.045530+0000", 2024-09-14T06:31:11.670 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 64.610498704999998, 2024-09-14T06:31:11.670 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032198773999999999, 2024-09-14T06:31:11.671 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:31:11.671 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:31:11.671 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "reached pg", 2024-09-14T06:31:11.671 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:31:11.671 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:31:11.671 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:11.671 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 98508 2024-09-14T06:31:11.671 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:11.671 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:31:11.671 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:11.671 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:31:11.671 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045530+0000", 2024-09-14T06:31:11.671 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:11.671 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:11.671 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:11.672 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:31:11.672 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045530+0000", 2024-09-14T06:31:11.672 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:11.672 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:11.672 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:11.672 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:31:11.672 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045530+0000", 2024-09-14T06:31:11.672 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.3099999999999998e-07 2024-09-14T06:31:11.672 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:11.672 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:11.672 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:31:11.672 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045539+0000", 2024-09-14T06:31:11.672 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.4780000000000008e-06 2024-09-14T06:31:11.673 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:11.673 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:11.673 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:31:11.673 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045546+0000", 2024-09-14T06:31:11.673 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.3259999999999998e-06 2024-09-14T06:31:11.673 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:11.673 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:11.673 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:31:11.673 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045551+0000", 2024-09-14T06:31:11.673 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.9960000000000001e-06 2024-09-14T06:31:11.673 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:11.673 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:11.673 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:31:11.673 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.077716+0000", 2024-09-14T06:31:11.674 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032165081999999998 2024-09-14T06:31:11.674 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:11.674 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:11.674 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:31:11.674 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.077729+0000", 2024-09-14T06:31:11.674 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.2461e-05 2024-09-14T06:31:11.674 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:11.674 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:31:11.674 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:11.674 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:11.674 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:11.674 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:98509 3.10 3.1bf56f90 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T06:31:11.674 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:30:07.045558+0000", 2024-09-14T06:31:11.674 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 64.610470504000006, 2024-09-14T06:31:11.675 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032242198, 2024-09-14T06:31:11.675 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:31:11.675 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:31:11.675 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "reached pg", 2024-09-14T06:31:11.675 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:31:11.675 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:31:11.675 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:11.675 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 98509 2024-09-14T06:31:11.675 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:11.675 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:31:11.675 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:11.675 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:31:11.675 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045558+0000", 2024-09-14T06:31:11.675 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:11.675 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:11.676 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:11.676 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:31:11.676 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045558+0000", 2024-09-14T06:31:11.676 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:11.676 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:11.676 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:11.676 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:31:11.676 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045559+0000", 2024-09-14T06:31:11.676 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.27e-07 2024-09-14T06:31:11.676 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:11.676 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:11.676 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:31:11.676 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045560+0000", 2024-09-14T06:31:11.677 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.6959999999999999e-06 2024-09-14T06:31:11.677 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:11.677 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:11.677 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:31:11.677 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045567+0000", 2024-09-14T06:31:11.677 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.6900000000000003e-06 2024-09-14T06:31:11.677 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:11.677 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:11.677 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:31:11.677 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045571+0000", 2024-09-14T06:31:11.677 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.5750000000000002e-06 2024-09-14T06:31:11.677 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:11.677 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:11.677 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:31:11.677 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.077789+0000", 2024-09-14T06:31:11.678 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032217186000000002 2024-09-14T06:31:11.678 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:11.678 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:11.678 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:31:11.678 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.077800+0000", 2024-09-14T06:31:11.678 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.1623999999999999e-05 2024-09-14T06:31:11.678 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:11.678 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:31:11.678 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:11.678 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:11.678 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:11.678 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:98510 3.10 3.1bf56f90 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T06:31:11.678 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:30:07.045580+0000", 2024-09-14T06:31:11.678 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 64.610448689999998, 2024-09-14T06:31:11.679 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032299845000000001, 2024-09-14T06:31:11.679 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:31:11.679 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:31:11.679 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "reached pg", 2024-09-14T06:31:11.679 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:31:11.679 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:31:11.679 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:11.679 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 98510 2024-09-14T06:31:11.679 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:11.679 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:31:11.679 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:11.679 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:31:11.679 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045580+0000", 2024-09-14T06:31:11.679 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:11.680 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:11.680 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:11.680 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:31:11.680 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045580+0000", 2024-09-14T06:31:11.680 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:11.680 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:11.680 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:11.680 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:31:11.680 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045580+0000", 2024-09-14T06:31:11.680 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.0400000000000002e-07 2024-09-14T06:31:11.680 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:11.680 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:11.680 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:31:11.680 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045582+0000", 2024-09-14T06:31:11.681 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.7600000000000001e-06 2024-09-14T06:31:11.681 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:11.681 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:11.681 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:31:11.681 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045588+0000", 2024-09-14T06:31:11.681 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.5650000000000001e-06 2024-09-14T06:31:11.681 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:11.681 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:11.681 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:31:11.681 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045597+0000", 2024-09-14T06:31:11.681 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.465e-06 2024-09-14T06:31:11.681 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:11.681 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:11.681 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:31:11.681 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.077871+0000", 2024-09-14T06:31:11.682 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032273562999999998 2024-09-14T06:31:11.682 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:11.682 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:11.682 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:31:11.682 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.077880+0000", 2024-09-14T06:31:11.682 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.0880000000000001e-06 2024-09-14T06:31:11.682 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:11.682 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:31:11.682 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:11.682 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:11.682 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:11.682 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:98511 3.10 3.1bf56f90 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T06:31:11.682 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:30:07.045610+0000", 2024-09-14T06:31:11.682 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 64.610418843000005, 2024-09-14T06:31:11.683 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032338894, 2024-09-14T06:31:11.683 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:31:11.683 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:31:11.683 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "reached pg", 2024-09-14T06:31:11.683 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:31:11.683 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:31:11.683 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:11.683 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 98511 2024-09-14T06:31:11.683 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:11.683 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:31:11.683 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:11.683 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:31:11.683 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045610+0000", 2024-09-14T06:31:11.683 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:11.684 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:11.684 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:11.684 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:31:11.684 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045610+0000", 2024-09-14T06:31:11.684 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:11.684 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:11.684 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:11.684 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:31:11.684 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045610+0000", 2024-09-14T06:31:11.684 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.1600000000000002e-07 2024-09-14T06:31:11.684 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:11.684 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:11.684 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:31:11.684 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045612+0000", 2024-09-14T06:31:11.685 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.1150000000000001e-06 2024-09-14T06:31:11.685 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:11.685 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:11.685 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:31:11.685 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045618+0000", 2024-09-14T06:31:11.685 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.0499999999999997e-06 2024-09-14T06:31:11.685 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:11.685 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:11.685 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:31:11.685 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045623+0000", 2024-09-14T06:31:11.685 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.6770000000000003e-06 2024-09-14T06:31:11.685 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:11.685 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:11.685 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:31:11.685 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.077928+0000", 2024-09-14T06:31:11.686 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032305111999999997 2024-09-14T06:31:11.686 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:11.686 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:11.686 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:31:11.686 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.077949+0000", 2024-09-14T06:31:11.686 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.0523999999999999e-05 2024-09-14T06:31:11.686 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:11.686 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:31:11.686 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:11.686 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:11.686 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:11.686 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:98512 3.10 3.1bf56f90 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T06:31:11.686 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:30:07.045630+0000", 2024-09-14T06:31:11.687 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 64.610398837999995, 2024-09-14T06:31:11.687 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032346632, 2024-09-14T06:31:11.687 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:31:11.687 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:31:11.687 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "reached pg", 2024-09-14T06:31:11.687 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:31:11.687 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:31:11.687 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:11.687 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 98512 2024-09-14T06:31:11.687 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:11.687 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:31:11.687 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:11.687 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:31:11.687 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045630+0000", 2024-09-14T06:31:11.688 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:11.688 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:11.688 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:11.688 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:31:11.688 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045630+0000", 2024-09-14T06:31:11.688 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:11.688 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:11.688 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:11.688 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:31:11.688 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045630+0000", 2024-09-14T06:31:11.688 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.8200000000000001e-07 2024-09-14T06:31:11.688 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:11.688 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:11.688 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:31:11.689 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045632+0000", 2024-09-14T06:31:11.689 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.7430000000000001e-06 2024-09-14T06:31:11.689 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:11.689 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:11.689 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:31:11.689 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045637+0000", 2024-09-14T06:31:11.689 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.4550000000000003e-06 2024-09-14T06:31:11.689 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:11.689 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:11.689 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:31:11.689 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045644+0000", 2024-09-14T06:31:11.689 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.1330000000000001e-06 2024-09-14T06:31:11.689 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:11.689 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:11.690 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:31:11.690 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.077968+0000", 2024-09-14T06:31:11.690 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032323401000000002 2024-09-14T06:31:11.690 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:11.690 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:11.690 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:31:11.690 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.077976+0000", 2024-09-14T06:31:11.690 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.5180000000000004e-06 2024-09-14T06:31:11.690 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:11.690 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:31:11.690 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:11.690 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:11.690 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:11.690 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:98513 3.10 3.1bf56f90 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T06:31:11.690 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:30:07.045652+0000", 2024-09-14T06:31:11.691 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 64.610376274000004, 2024-09-14T06:31:11.691 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032353436999999999, 2024-09-14T06:31:11.691 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:31:11.691 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:31:11.691 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "reached pg", 2024-09-14T06:31:11.691 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:31:11.691 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:31:11.691 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:11.691 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 98513 2024-09-14T06:31:11.691 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:11.691 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:31:11.691 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:11.691 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:31:11.691 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045652+0000", 2024-09-14T06:31:11.692 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:11.692 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:11.692 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:11.692 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:31:11.692 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045652+0000", 2024-09-14T06:31:11.692 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:11.692 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:11.692 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:11.692 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:31:11.692 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045653+0000", 2024-09-14T06:31:11.692 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.3700000000000001e-07 2024-09-14T06:31:11.692 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:11.692 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:11.692 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:31:11.693 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045655+0000", 2024-09-14T06:31:11.693 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.854e-06 2024-09-14T06:31:11.693 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:11.693 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:11.693 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:31:11.693 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045661+0000", 2024-09-14T06:31:11.693 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.5509999999999996e-06 2024-09-14T06:31:11.693 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:11.693 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:11.693 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:31:11.693 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045668+0000", 2024-09-14T06:31:11.693 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.1980000000000004e-06 2024-09-14T06:31:11.693 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:11.693 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:11.694 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:31:11.694 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.077997+0000", 2024-09-14T06:31:11.694 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032328575999999998 2024-09-14T06:31:11.694 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:11.694 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:11.694 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:31:11.694 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.078006+0000", 2024-09-14T06:31:11.694 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.8210000000000008e-06 2024-09-14T06:31:11.694 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:11.694 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:31:11.694 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:11.694 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:11.694 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:11.694 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:98514 3.10 3.1bf56f90 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T06:31:11.694 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:30:07.045699+0000", 2024-09-14T06:31:11.695 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 64.610329938999996, 2024-09-14T06:31:11.695 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032330223999999998, 2024-09-14T06:31:11.695 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:31:11.695 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:31:11.695 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "reached pg", 2024-09-14T06:31:11.695 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:31:11.695 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:31:11.695 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:11.695 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 98514 2024-09-14T06:31:11.695 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:11.695 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:31:11.695 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:11.695 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:31:11.695 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045699+0000", 2024-09-14T06:31:11.696 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:11.696 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:11.696 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:11.696 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:31:11.696 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045699+0000", 2024-09-14T06:31:11.696 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:11.696 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:11.696 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:11.696 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:31:11.696 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045699+0000", 2024-09-14T06:31:11.696 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.3200000000000001e-07 2024-09-14T06:31:11.696 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:11.696 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:11.696 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:31:11.697 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045701+0000", 2024-09-14T06:31:11.697 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.762e-06 2024-09-14T06:31:11.697 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:11.697 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:11.697 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:31:11.697 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045705+0000", 2024-09-14T06:31:11.697 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.2359999999999998e-06 2024-09-14T06:31:11.697 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:11.697 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:11.697 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:31:11.697 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045708+0000", 2024-09-14T06:31:11.697 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.796e-06 2024-09-14T06:31:11.697 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:11.697 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:11.698 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:31:11.698 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.078023+0000", 2024-09-14T06:31:11.698 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032315213000000002 2024-09-14T06:31:11.698 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:11.698 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:11.698 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:31:11.698 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.078029+0000", 2024-09-14T06:31:11.698 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.8850000000000004e-06 2024-09-14T06:31:11.698 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:11.698 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:31:11.698 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:11.698 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:11.698 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:11.698 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:98515 3.10 3.1bf56f90 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T06:31:11.698 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:30:07.045712+0000", 2024-09-14T06:31:11.699 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 64.610316628000007, 2024-09-14T06:31:11.699 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032346871999999999, 2024-09-14T06:31:11.699 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:31:11.699 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:31:11.699 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "reached pg", 2024-09-14T06:31:11.699 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:31:11.699 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:31:11.699 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:11.699 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 98515 2024-09-14T06:31:11.699 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:11.699 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:31:11.699 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:11.699 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:31:11.699 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045712+0000", 2024-09-14T06:31:11.699 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:11.700 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:11.700 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:11.700 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:31:11.700 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045712+0000", 2024-09-14T06:31:11.700 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:11.700 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:11.700 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:11.700 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:31:11.700 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045712+0000", 2024-09-14T06:31:11.700 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.2100000000000001e-07 2024-09-14T06:31:11.700 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:11.700 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:11.700 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:31:11.700 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045713+0000", 2024-09-14T06:31:11.701 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.068e-06 2024-09-14T06:31:11.701 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:11.701 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:11.701 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:31:11.701 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045716+0000", 2024-09-14T06:31:11.701 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.971e-06 2024-09-14T06:31:11.701 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:11.701 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:11.701 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:31:11.701 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045719+0000", 2024-09-14T06:31:11.701 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.3879999999999998e-06 2024-09-14T06:31:11.701 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:11.701 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:11.701 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:31:11.702 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.078049+0000", 2024-09-14T06:31:11.702 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032330553999999997 2024-09-14T06:31:11.702 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:11.702 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:11.702 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:31:11.702 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.078059+0000", 2024-09-14T06:31:11.702 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.6700000000000006e-06 2024-09-14T06:31:11.702 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:11.702 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:31:11.702 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:11.702 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:11.702 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:11.702 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:98516 3.10 3.1bf56f90 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T06:31:11.702 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:30:07.045722+0000", 2024-09-14T06:31:11.702 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 64.610306875000006, 2024-09-14T06:31:11.703 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032374334999999997, 2024-09-14T06:31:11.703 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:31:11.703 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:31:11.703 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "reached pg", 2024-09-14T06:31:11.703 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:31:11.703 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:31:11.703 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:11.703 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 98516 2024-09-14T06:31:11.703 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:11.703 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:31:11.703 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:11.703 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:31:11.703 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045722+0000", 2024-09-14T06:31:11.703 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:11.704 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:11.704 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:11.704 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:31:11.704 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045722+0000", 2024-09-14T06:31:11.704 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:11.704 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:11.704 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:11.704 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:31:11.704 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045722+0000", 2024-09-14T06:31:11.704 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.8999999999999998e-07 2024-09-14T06:31:11.704 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:11.704 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:11.704 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:31:11.704 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045723+0000", 2024-09-14T06:31:11.705 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.0049999999999999e-06 2024-09-14T06:31:11.705 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:11.705 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:11.705 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:31:11.705 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045726+0000", 2024-09-14T06:31:11.705 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.9019999999999999e-06 2024-09-14T06:31:11.705 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:11.705 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:11.705 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:31:11.705 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045728+0000", 2024-09-14T06:31:11.705 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.3290000000000001e-06 2024-09-14T06:31:11.705 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:11.705 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:11.705 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:31:11.706 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.078090+0000", 2024-09-14T06:31:11.706 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032362123 2024-09-14T06:31:11.706 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:11.706 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:11.706 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:31:11.706 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.078096+0000", 2024-09-14T06:31:11.706 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.6860000000000003e-06 2024-09-14T06:31:11.706 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:11.706 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:31:11.706 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:11.706 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:11.706 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:11.706 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:98517 3.10 3.1bf56f90 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T06:31:11.706 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:30:07.045731+0000", 2024-09-14T06:31:11.707 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 64.610297144, 2024-09-14T06:31:11.707 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.03241227, 2024-09-14T06:31:11.707 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:31:11.707 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:31:11.707 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "reached pg", 2024-09-14T06:31:11.707 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:31:11.707 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:31:11.707 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:11.707 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 98517 2024-09-14T06:31:11.707 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:11.707 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:31:11.707 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:11.707 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:31:11.707 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045731+0000", 2024-09-14T06:31:11.707 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:11.708 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:11.708 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:11.708 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:31:11.708 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045731+0000", 2024-09-14T06:31:11.708 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:11.708 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:11.708 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:11.708 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:31:11.708 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045732+0000", 2024-09-14T06:31:11.708 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.0699999999999999e-07 2024-09-14T06:31:11.708 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:11.708 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:11.708 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:31:11.708 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045733+0000", 2024-09-14T06:31:11.709 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.9199999999999999e-07 2024-09-14T06:31:11.709 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:11.709 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:11.709 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:31:11.709 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045735+0000", 2024-09-14T06:31:11.709 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.8150000000000002e-06 2024-09-14T06:31:11.709 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:11.709 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:11.709 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:31:11.709 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045738+0000", 2024-09-14T06:31:11.709 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.3640000000000002e-06 2024-09-14T06:31:11.709 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:11.709 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:11.709 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:31:11.709 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.078126+0000", 2024-09-14T06:31:11.710 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032388206000000003 2024-09-14T06:31:11.710 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:11.710 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:11.710 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:31:11.710 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.078144+0000", 2024-09-14T06:31:11.710 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.7686e-05 2024-09-14T06:31:11.710 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:11.710 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:31:11.710 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:11.710 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:11.710 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:11.710 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:98518 3.10 3.1bf56f90 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T06:31:11.710 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:30:07.045741+0000", 2024-09-14T06:31:11.710 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 64.610287748999994, 2024-09-14T06:31:11.710 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032451793999999999, 2024-09-14T06:31:11.711 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:31:11.711 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:31:11.711 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "reached pg", 2024-09-14T06:31:11.711 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:31:11.711 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:31:11.711 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:11.711 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 98518 2024-09-14T06:31:11.711 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:11.711 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:31:11.711 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:11.711 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:31:11.711 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045741+0000", 2024-09-14T06:31:11.711 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:11.711 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:11.712 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:11.712 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:31:11.712 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045741+0000", 2024-09-14T06:31:11.712 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:11.712 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:11.712 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:11.712 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:31:11.712 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045741+0000", 2024-09-14T06:31:11.712 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.05e-07 2024-09-14T06:31:11.712 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:11.712 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:11.712 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:31:11.712 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045746+0000", 2024-09-14T06:31:11.712 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.2259999999999996e-06 2024-09-14T06:31:11.713 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:11.713 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:11.713 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:31:11.713 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045750+0000", 2024-09-14T06:31:11.713 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.2490000000000001e-06 2024-09-14T06:31:11.713 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:11.713 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:11.713 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:31:11.713 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045752+0000", 2024-09-14T06:31:11.713 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.3980000000000002e-06 2024-09-14T06:31:11.713 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:11.713 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:11.713 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:31:11.713 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.078184+0000", 2024-09-14T06:31:11.714 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032431593000000002 2024-09-14T06:31:11.714 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:11.714 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:11.714 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:31:11.714 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.078193+0000", 2024-09-14T06:31:11.714 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.1230000000000006e-06 2024-09-14T06:31:11.714 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:11.714 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:31:11.714 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:11.714 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:11.714 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:11.714 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:98519 3.10 3.1bf56f90 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T06:31:11.714 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:30:07.045755+0000", 2024-09-14T06:31:11.714 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 64.610273324000005, 2024-09-14T06:31:11.715 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032489431999999999, 2024-09-14T06:31:11.715 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:31:11.715 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:31:11.715 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "reached pg", 2024-09-14T06:31:11.715 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:31:11.715 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:31:11.715 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:11.715 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 98519 2024-09-14T06:31:11.715 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:11.715 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:31:11.715 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:11.715 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:31:11.715 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045755+0000", 2024-09-14T06:31:11.715 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:11.716 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:11.716 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:11.716 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:31:11.716 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045755+0000", 2024-09-14T06:31:11.716 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:11.716 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:11.716 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:11.716 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:31:11.716 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045756+0000", 2024-09-14T06:31:11.716 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.23e-07 2024-09-14T06:31:11.716 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:11.716 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:11.716 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:31:11.716 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045757+0000", 2024-09-14T06:31:11.717 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.1060000000000001e-06 2024-09-14T06:31:11.717 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:11.717 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:11.717 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:31:11.717 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045760+0000", 2024-09-14T06:31:11.717 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.988e-06 2024-09-14T06:31:11.717 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:11.717 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:11.717 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:31:11.717 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045765+0000", 2024-09-14T06:31:11.717 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.2689999999999999e-06 2024-09-14T06:31:11.717 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:11.717 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:11.717 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:31:11.718 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.078234+0000", 2024-09-14T06:31:11.718 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032468915000000001 2024-09-14T06:31:11.718 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:11.718 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:11.718 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:31:11.718 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.078245+0000", 2024-09-14T06:31:11.718 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.0930999999999999e-05 2024-09-14T06:31:11.718 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:11.718 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:31:11.718 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:11.718 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:11.718 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:11.718 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:98520 3.10 3.1bf56f90 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T06:31:11.718 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:30:07.045768+0000", 2024-09-14T06:31:11.719 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 64.610260445999998, 2024-09-14T06:31:11.719 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032531379999999999, 2024-09-14T06:31:11.719 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:31:11.719 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:31:11.719 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "reached pg", 2024-09-14T06:31:11.719 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:31:11.719 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:31:11.719 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:11.719 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 98520 2024-09-14T06:31:11.719 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:11.719 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:31:11.719 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:11.719 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:31:11.719 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045768+0000", 2024-09-14T06:31:11.720 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:11.720 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:11.720 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:11.720 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:31:11.720 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045768+0000", 2024-09-14T06:31:11.720 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:11.720 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:11.720 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:11.720 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:31:11.720 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045768+0000", 2024-09-14T06:31:11.720 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.22e-07 2024-09-14T06:31:11.720 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:11.720 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:11.720 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:31:11.721 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045773+0000", 2024-09-14T06:31:11.721 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.3470000000000001e-06 2024-09-14T06:31:11.721 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:11.721 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:11.721 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:31:11.721 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045776+0000", 2024-09-14T06:31:11.721 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.1319999999999998e-06 2024-09-14T06:31:11.721 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:11.721 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:11.721 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:31:11.721 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045779+0000", 2024-09-14T06:31:11.721 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.9749999999999999e-06 2024-09-14T06:31:11.721 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:11.722 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:11.722 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:31:11.722 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.078289+0000", 2024-09-14T06:31:11.722 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032509693999999999 2024-09-14T06:31:11.722 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:11.722 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:11.722 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:31:11.722 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.078300+0000", 2024-09-14T06:31:11.722 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.101e-05 2024-09-14T06:31:11.722 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:11.722 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:31:11.722 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:11.722 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:11.722 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:11.722 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:98521 3.10 3.1bf56f90 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T06:31:11.723 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:30:07.045782+0000", 2024-09-14T06:31:11.723 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 64.610246266000004, 2024-09-14T06:31:11.723 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032566109000000003, 2024-09-14T06:31:11.723 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:31:11.723 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:31:11.723 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "reached pg", 2024-09-14T06:31:11.723 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:31:11.723 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:31:11.723 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:11.723 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 98521 2024-09-14T06:31:11.723 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:11.723 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:31:11.723 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:11.723 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:31:11.724 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045782+0000", 2024-09-14T06:31:11.724 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:11.724 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:11.724 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:11.724 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:31:11.724 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045782+0000", 2024-09-14T06:31:11.724 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:11.724 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:11.724 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:11.724 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:31:11.724 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045783+0000", 2024-09-14T06:31:11.724 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.2499999999999999e-07 2024-09-14T06:31:11.724 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:11.724 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:11.725 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:31:11.725 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045784+0000", 2024-09-14T06:31:11.725 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.0109999999999999e-06 2024-09-14T06:31:11.725 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:11.725 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:11.725 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:31:11.725 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045786+0000", 2024-09-14T06:31:11.725 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.8289999999999999e-06 2024-09-14T06:31:11.725 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:11.725 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:11.725 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:31:11.725 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045793+0000", 2024-09-14T06:31:11.725 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.579e-06 2024-09-14T06:31:11.725 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:11.726 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:11.726 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:31:11.726 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.078338+0000", 2024-09-14T06:31:11.726 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032545389000000001 2024-09-14T06:31:11.726 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:11.726 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:11.726 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:31:11.726 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.078348+0000", 2024-09-14T06:31:11.726 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.0076e-05 2024-09-14T06:31:11.726 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:11.726 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:31:11.726 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:11.726 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:11.726 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:11.727 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:98522 3.10 3.1bf56f90 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T06:31:11.727 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:30:07.045796+0000", 2024-09-14T06:31:11.727 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 64.610232358000005, 2024-09-14T06:31:11.727 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032577846000000001, 2024-09-14T06:31:11.727 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:31:11.727 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:31:11.727 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "reached pg", 2024-09-14T06:31:11.727 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:31:11.727 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:31:11.727 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:11.727 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 98522 2024-09-14T06:31:11.727 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:11.727 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:31:11.727 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:11.728 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:31:11.728 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045796+0000", 2024-09-14T06:31:11.728 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:11.728 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:11.728 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:11.728 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:31:11.728 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045796+0000", 2024-09-14T06:31:11.728 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:11.728 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:11.728 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:11.728 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:31:11.728 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045796+0000", 2024-09-14T06:31:11.728 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.2100000000000001e-07 2024-09-14T06:31:11.728 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:11.729 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:11.729 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:31:11.729 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045797+0000", 2024-09-14T06:31:11.729 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.6899999999999996e-07 2024-09-14T06:31:11.729 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:11.729 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:11.729 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:31:11.729 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045800+0000", 2024-09-14T06:31:11.729 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.9979999999999999e-06 2024-09-14T06:31:11.729 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:11.729 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:11.729 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:31:11.729 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045803+0000", 2024-09-14T06:31:11.729 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.2850000000000001e-06 2024-09-14T06:31:11.729 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:11.730 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:11.730 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:31:11.730 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.078366+0000", 2024-09-14T06:31:11.730 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032562932000000003 2024-09-14T06:31:11.730 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:11.730 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:11.730 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:31:11.730 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.078374+0000", 2024-09-14T06:31:11.730 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.4409999999999993e-06 2024-09-14T06:31:11.730 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:11.730 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:31:11.730 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:11.730 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:11.731 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:11.731 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:98523 3.10 3.1bf56f90 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T06:31:11.731 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:30:07.045806+0000", 2024-09-14T06:31:11.731 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 64.610222815, 2024-09-14T06:31:11.731 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032611167000000003, 2024-09-14T06:31:11.731 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:31:11.731 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:31:11.731 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "reached pg", 2024-09-14T06:31:11.731 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:31:11.731 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:31:11.731 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:11.731 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 98523 2024-09-14T06:31:11.731 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:11.732 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:31:11.732 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:11.732 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:31:11.732 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045806+0000", 2024-09-14T06:31:11.732 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:11.732 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:11.732 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:11.732 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:31:11.732 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045806+0000", 2024-09-14T06:31:11.732 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:11.732 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:11.732 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:11.732 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:31:11.732 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045806+0000", 2024-09-14T06:31:11.733 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.2000000000000001e-07 2024-09-14T06:31:11.733 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:11.733 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:11.733 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:31:11.733 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045807+0000", 2024-09-14T06:31:11.733 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.4e-07 2024-09-14T06:31:11.733 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:11.733 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:11.733 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:31:11.733 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045810+0000", 2024-09-14T06:31:11.733 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.9229999999999998e-06 2024-09-14T06:31:11.733 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:11.733 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:11.734 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:31:11.734 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045812+0000", 2024-09-14T06:31:11.734 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.2220000000000001e-06 2024-09-14T06:31:11.734 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:11.734 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:11.734 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:31:11.734 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.078407+0000", 2024-09-14T06:31:11.734 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032594513999999998 2024-09-14T06:31:11.734 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:11.734 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:11.734 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:31:11.734 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.078417+0000", 2024-09-14T06:31:11.734 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.0348e-05 2024-09-14T06:31:11.734 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:11.735 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:31:11.735 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:11.735 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:11.735 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:11.735 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:98524 3.10 3.1bf56f90 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T06:31:11.735 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:30:07.045815+0000", 2024-09-14T06:31:11.735 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 64.610213447000007, 2024-09-14T06:31:11.735 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032643208999999999, 2024-09-14T06:31:11.735 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:31:11.735 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:31:11.735 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "reached pg", 2024-09-14T06:31:11.735 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:31:11.735 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:31:11.736 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:11.736 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 98524 2024-09-14T06:31:11.736 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:11.736 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:31:11.736 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:11.736 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:31:11.736 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045815+0000", 2024-09-14T06:31:11.736 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:11.736 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:11.736 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:11.736 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:31:11.736 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045815+0000", 2024-09-14T06:31:11.736 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:11.736 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:11.736 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:11.737 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:31:11.737 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045815+0000", 2024-09-14T06:31:11.737 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.11e-07 2024-09-14T06:31:11.737 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:11.737 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:11.737 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:31:11.737 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045816+0000", 2024-09-14T06:31:11.737 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.9099999999999991e-07 2024-09-14T06:31:11.737 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:11.737 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:11.737 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:31:11.737 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045819+0000", 2024-09-14T06:31:11.737 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.9399999999999998e-06 2024-09-14T06:31:11.738 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:11.738 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:11.738 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:31:11.738 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045822+0000", 2024-09-14T06:31:11.738 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.3470000000000001e-06 2024-09-14T06:31:11.738 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:11.738 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:11.738 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:31:11.738 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.078449+0000", 2024-09-14T06:31:11.738 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032627529000000002 2024-09-14T06:31:11.738 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:11.738 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:11.738 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:31:11.738 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.078458+0000", 2024-09-14T06:31:11.739 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.1910000000000006e-06 2024-09-14T06:31:11.739 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:11.739 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:31:11.739 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:11.739 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:11.739 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:31:11.739 INFO:teuthology.orchestra.run.smithi138.stdout:} 2024-09-14T06:31:11.740 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-14T06:31:11.741 INFO:tasks.ceph.osd.1.smithi094.stderr:2024-09-14T06:31:11.739+0000 7f328e47a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33641) UID: 0 2024-09-14T06:31:11.835 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-14T06:31:11.841 INFO:tasks.ceph.osd.7.smithi138.stderr:2024-09-14T06:31:11.840+0000 7f3c6a21d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33623) UID: 0 2024-09-14T06:31:11.855 INFO:teuthology.orchestra.run.smithi138.stdout:{ 2024-09-14T06:31:11.855 INFO:teuthology.orchestra.run.smithi138.stdout: "ops": [], 2024-09-14T06:31:11.855 INFO:teuthology.orchestra.run.smithi138.stdout: "num_ops": 0 2024-09-14T06:31:11.855 INFO:teuthology.orchestra.run.smithi138.stdout:} 2024-09-14T06:31:11.865 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-14T06:31:11.949 INFO:tasks.ceph.osd.4.smithi138.stderr:2024-09-14T06:31:11.948+0000 7f6c653c8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33615) UID: 0 2024-09-14T06:31:12.016 INFO:teuthology.orchestra.run.smithi138.stdout:{ 2024-09-14T06:31:12.016 INFO:teuthology.orchestra.run.smithi138.stdout: "ops": [], 2024-09-14T06:31:12.016 INFO:teuthology.orchestra.run.smithi138.stdout: "complaint_time": 30, 2024-09-14T06:31:12.016 INFO:teuthology.orchestra.run.smithi138.stdout: "num_blocked_ops": 0 2024-09-14T06:31:12.016 INFO:teuthology.orchestra.run.smithi138.stdout:} 2024-09-14T06:31:12.026 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-14T06:31:12.067 INFO:tasks.ceph.osd.4.smithi138.stderr:2024-09-14T06:31:12.066+0000 7f6c653c8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33615) UID: 0 2024-09-14T06:31:12.144 INFO:tasks.ceph.osd.2.smithi094.stderr:2024-09-14T06:31:12.143+0000 7faf3269b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33649) UID: 0 2024-09-14T06:31:12.200 INFO:teuthology.orchestra.run.smithi138.stdout:{ 2024-09-14T06:31:12.200 INFO:teuthology.orchestra.run.smithi138.stdout: "size": 20, 2024-09-14T06:31:12.200 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 600, 2024-09-14T06:31:12.200 INFO:teuthology.orchestra.run.smithi138.stdout: "ops": [ 2024-09-14T06:31:12.201 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:12.201 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:53244 3.f 3:f71f51a7:::rbd_data.112dca2f6ae1.0000000000000015:17 [read 1863680~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:31:12.201 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:44.369399+0000", 2024-09-14T06:31:12.201 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 87.828711939000002, 2024-09-14T06:31:12.201 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047291189999999997, 2024-09-14T06:31:12.201 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:31:12.201 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:31:12.201 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:31:12.201 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:31:12.201 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:31:12.201 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:12.201 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 53244 2024-09-14T06:31:12.201 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:12.202 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:31:12.202 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:12.202 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:31:12.202 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369399+0000", 2024-09-14T06:31:12.202 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:12.202 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:12.202 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:12.202 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:31:12.202 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369399+0000", 2024-09-14T06:31:12.202 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:12.202 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:12.202 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:12.202 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:31:12.203 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369400+0000", 2024-09-14T06:31:12.203 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.0699999999999999e-07 2024-09-14T06:31:12.203 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:12.203 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:12.203 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:31:12.203 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369401+0000", 2024-09-14T06:31:12.203 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.4300000000000001e-06 2024-09-14T06:31:12.203 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:12.203 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:12.203 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:31:12.203 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369404+0000", 2024-09-14T06:31:12.203 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.061e-06 2024-09-14T06:31:12.203 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:12.203 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:12.204 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:31:12.204 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369407+0000", 2024-09-14T06:31:12.204 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.6139999999999999e-06 2024-09-14T06:31:12.204 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:12.204 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:12.204 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:31:12.204 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.416517+0000", 2024-09-14T06:31:12.204 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047110338000000002 2024-09-14T06:31:12.204 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:12.204 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:12.204 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:31:12.204 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.416616+0000", 2024-09-14T06:31:12.204 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.8651000000000001e-05 2024-09-14T06:31:12.204 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:12.205 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:12.205 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:31:12.205 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.416691+0000", 2024-09-14T06:31:12.205 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.4888999999999994e-05 2024-09-14T06:31:12.205 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:12.205 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:31:12.205 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:12.205 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:12.205 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:12.205 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:53245 3.f 3:f71f51a7:::rbd_data.112dca2f6ae1.0000000000000015:17 [read 1888256~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:31:12.205 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:44.369410+0000", 2024-09-14T06:31:12.205 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 87.828701651000003, 2024-09-14T06:31:12.205 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047399275999999997, 2024-09-14T06:31:12.206 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:31:12.206 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:31:12.206 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:31:12.206 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:31:12.206 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:31:12.206 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:12.206 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 53245 2024-09-14T06:31:12.206 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:12.206 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:31:12.206 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:12.206 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:31:12.206 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369410+0000", 2024-09-14T06:31:12.206 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:12.206 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:12.207 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:12.207 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:31:12.207 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369410+0000", 2024-09-14T06:31:12.207 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:12.207 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:12.207 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:12.207 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:31:12.207 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369410+0000", 2024-09-14T06:31:12.207 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.1899999999999999e-07 2024-09-14T06:31:12.207 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:12.207 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:12.207 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:31:12.207 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369411+0000", 2024-09-14T06:31:12.207 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.0300000000000001e-06 2024-09-14T06:31:12.208 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:12.208 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:12.208 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:31:12.208 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369414+0000", 2024-09-14T06:31:12.208 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.1099999999999999e-06 2024-09-14T06:31:12.208 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:12.208 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:12.208 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:31:12.208 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369417+0000", 2024-09-14T06:31:12.208 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.6189999999999998e-06 2024-09-14T06:31:12.208 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:12.208 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:12.208 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:31:12.208 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.416718+0000", 2024-09-14T06:31:12.209 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047301322999999999 2024-09-14T06:31:12.209 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:12.209 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:12.209 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:31:12.209 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.416759+0000", 2024-09-14T06:31:12.209 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.1009e-05 2024-09-14T06:31:12.209 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:12.209 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:12.209 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:31:12.209 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.416809+0000", 2024-09-14T06:31:12.209 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.9966000000000002e-05 2024-09-14T06:31:12.209 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:12.209 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:31:12.210 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:12.210 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:12.210 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:12.210 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:53246 3.f 3:f71f51a7:::rbd_data.112dca2f6ae1.0000000000000015:17 [read 1912832~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:31:12.210 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:44.369420+0000", 2024-09-14T06:31:12.210 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 87.828691707000004, 2024-09-14T06:31:12.210 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047529463000000001, 2024-09-14T06:31:12.210 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:31:12.210 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:31:12.210 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:31:12.210 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:31:12.210 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:31:12.210 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:12.211 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 53246 2024-09-14T06:31:12.211 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:12.211 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:31:12.211 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:12.211 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:31:12.211 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369420+0000", 2024-09-14T06:31:12.211 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:12.211 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:12.211 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:12.211 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:31:12.211 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369420+0000", 2024-09-14T06:31:12.211 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:12.211 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:12.211 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:12.212 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:31:12.212 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369420+0000", 2024-09-14T06:31:12.212 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.1299999999999999e-07 2024-09-14T06:31:12.212 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:12.212 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:12.212 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:31:12.212 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369425+0000", 2024-09-14T06:31:12.212 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.0130000000000001e-06 2024-09-14T06:31:12.212 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:12.212 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:12.212 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:31:12.212 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369428+0000", 2024-09-14T06:31:12.212 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.349e-06 2024-09-14T06:31:12.212 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:12.213 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:12.213 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:31:12.213 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369431+0000", 2024-09-14T06:31:12.213 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.7389999999999999e-06 2024-09-14T06:31:12.213 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:12.213 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:12.213 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:31:12.213 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.416834+0000", 2024-09-14T06:31:12.213 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047402669000000001 2024-09-14T06:31:12.213 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:12.213 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:12.213 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:31:12.213 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.416882+0000", 2024-09-14T06:31:12.213 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.8068999999999997e-05 2024-09-14T06:31:12.214 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:12.214 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:12.214 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:31:12.214 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.416949+0000", 2024-09-14T06:31:12.214 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.7410999999999995e-05 2024-09-14T06:31:12.214 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:12.214 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:31:12.214 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:12.214 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:12.214 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:12.214 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:53247 3.f 3:f71f51a7:::rbd_data.112dca2f6ae1.0000000000000015:17 [read 1937408~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:31:12.214 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:44.369434+0000", 2024-09-14T06:31:12.214 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 87.828676943000005, 2024-09-14T06:31:12.215 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047652264999999999, 2024-09-14T06:31:12.215 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:31:12.215 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:31:12.215 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:31:12.215 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:31:12.215 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:31:12.215 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:12.215 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 53247 2024-09-14T06:31:12.215 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:12.215 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:31:12.215 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:12.215 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:31:12.215 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369434+0000", 2024-09-14T06:31:12.216 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:12.216 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:12.216 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:12.216 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:31:12.216 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369434+0000", 2024-09-14T06:31:12.216 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:12.216 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:12.216 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:12.216 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:31:12.216 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369435+0000", 2024-09-14T06:31:12.216 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.2100000000000001e-07 2024-09-14T06:31:12.216 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:12.216 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:12.216 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:31:12.217 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369436+0000", 2024-09-14T06:31:12.217 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.122e-06 2024-09-14T06:31:12.217 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:12.217 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:12.217 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:31:12.217 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369439+0000", 2024-09-14T06:31:12.217 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.0400000000000001e-06 2024-09-14T06:31:12.217 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:12.217 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:12.217 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:31:12.217 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369441+0000", 2024-09-14T06:31:12.217 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.728e-06 2024-09-14T06:31:12.217 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:12.217 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:12.218 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:31:12.218 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.416977+0000", 2024-09-14T06:31:12.218 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047535344 2024-09-14T06:31:12.218 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:12.218 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:12.218 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:31:12.218 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.417029+0000", 2024-09-14T06:31:12.218 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.2037999999999999e-05 2024-09-14T06:31:12.218 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:12.218 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:12.218 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:31:12.218 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.417087+0000", 2024-09-14T06:31:12.218 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.7772000000000002e-05 2024-09-14T06:31:12.219 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:12.219 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:31:12.219 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:12.219 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:12.219 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:12.219 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:53248 3.f 3:f71f51a7:::rbd_data.112dca2f6ae1.0000000000000015:17 [read 1961984~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:31:12.219 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:44.369445+0000", 2024-09-14T06:31:12.219 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 87.828666390999999, 2024-09-14T06:31:12.219 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047798489, 2024-09-14T06:31:12.219 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:31:12.219 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:31:12.219 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:31:12.219 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:31:12.219 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:31:12.220 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:12.220 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 53248 2024-09-14T06:31:12.220 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:12.220 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:31:12.220 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:12.220 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:31:12.220 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369445+0000", 2024-09-14T06:31:12.220 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:12.220 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:12.220 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:12.220 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:31:12.220 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369445+0000", 2024-09-14T06:31:12.220 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:12.220 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:12.221 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:12.221 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:31:12.221 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369445+0000", 2024-09-14T06:31:12.221 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.0699999999999999e-07 2024-09-14T06:31:12.221 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:12.221 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:12.221 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:31:12.221 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369446+0000", 2024-09-14T06:31:12.221 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.1000000000000001e-06 2024-09-14T06:31:12.221 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:12.221 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:12.221 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:31:12.221 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369449+0000", 2024-09-14T06:31:12.221 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.1039999999999998e-06 2024-09-14T06:31:12.222 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:12.222 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:12.222 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:31:12.222 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369452+0000", 2024-09-14T06:31:12.222 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.6199999999999999e-06 2024-09-14T06:31:12.222 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:12.222 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:12.222 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:31:12.222 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.417114+0000", 2024-09-14T06:31:12.222 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047661901999999999 2024-09-14T06:31:12.222 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:12.222 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:12.222 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:31:12.222 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.417181+0000", 2024-09-14T06:31:12.223 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.7385999999999995e-05 2024-09-14T06:31:12.223 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:12.223 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:12.223 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:31:12.223 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.417243+0000", 2024-09-14T06:31:12.223 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.2169999999999996e-05 2024-09-14T06:31:12.223 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:12.223 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:31:12.223 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:12.223 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:12.223 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:12.223 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:53249 3.f 3:f71f51a7:::rbd_data.112dca2f6ae1.0000000000000015:17 [read 1986560~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:31:12.223 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:44.369455+0000", 2024-09-14T06:31:12.223 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 87.828656080000002, 2024-09-14T06:31:12.224 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047890493999999999, 2024-09-14T06:31:12.224 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:31:12.224 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:31:12.224 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:31:12.224 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:31:12.224 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:31:12.224 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:12.224 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 53249 2024-09-14T06:31:12.224 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:12.224 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:31:12.224 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:12.224 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:31:12.224 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369455+0000", 2024-09-14T06:31:12.224 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:12.224 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:12.225 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:12.225 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:31:12.225 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369455+0000", 2024-09-14T06:31:12.225 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:12.225 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:12.225 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:12.225 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:31:12.225 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369455+0000", 2024-09-14T06:31:12.225 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.0800000000000001e-07 2024-09-14T06:31:12.225 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:12.225 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:12.225 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:31:12.225 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369456+0000", 2024-09-14T06:31:12.226 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.0729999999999999e-06 2024-09-14T06:31:12.226 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:12.226 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:12.226 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:31:12.226 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369460+0000", 2024-09-14T06:31:12.226 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.027e-06 2024-09-14T06:31:12.226 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:12.226 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:12.226 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:31:12.226 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369462+0000", 2024-09-14T06:31:12.226 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.5320000000000001e-06 2024-09-14T06:31:12.226 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:12.226 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:12.226 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:31:12.226 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.417257+0000", 2024-09-14T06:31:12.227 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.04779489 2024-09-14T06:31:12.227 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:12.227 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:12.227 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:31:12.227 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.417296+0000", 2024-09-14T06:31:12.227 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.8658000000000002e-05 2024-09-14T06:31:12.227 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:12.227 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:12.228 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:31:12.228 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.417346+0000", 2024-09-14T06:31:12.228 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.0105999999999998e-05 2024-09-14T06:31:12.228 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:12.228 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:31:12.228 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:12.228 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:12.228 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:12.228 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:53250 3.f 3:f71f51a7:::rbd_data.112dca2f6ae1.0000000000000015:17 [read 2011136~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:31:12.228 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:44.369465+0000", 2024-09-14T06:31:12.228 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 87.828646109999994, 2024-09-14T06:31:12.229 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.048032230000000002, 2024-09-14T06:31:12.229 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:31:12.229 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:31:12.229 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:31:12.229 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:31:12.229 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:31:12.229 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:12.229 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 53250 2024-09-14T06:31:12.229 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:12.229 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:31:12.229 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:12.229 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:31:12.230 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369465+0000", 2024-09-14T06:31:12.230 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:12.230 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:12.230 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:12.230 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:31:12.230 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369465+0000", 2024-09-14T06:31:12.230 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:12.230 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:12.230 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:12.230 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:31:12.230 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369465+0000", 2024-09-14T06:31:12.230 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.2100000000000001e-07 2024-09-14T06:31:12.230 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:12.231 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:12.231 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:31:12.231 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369466+0000", 2024-09-14T06:31:12.231 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.0640000000000001e-06 2024-09-14T06:31:12.231 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:12.231 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:12.231 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:31:12.231 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369469+0000", 2024-09-14T06:31:12.231 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.0170000000000001e-06 2024-09-14T06:31:12.231 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:12.231 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:12.231 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:31:12.231 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369472+0000", 2024-09-14T06:31:12.231 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.6189999999999998e-06 2024-09-14T06:31:12.232 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:12.232 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:12.232 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:31:12.232 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.417383+0000", 2024-09-14T06:31:12.232 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047911092000000002 2024-09-14T06:31:12.232 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:12.232 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:12.232 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:31:12.232 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.417430+0000", 2024-09-14T06:31:12.232 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.6898999999999999e-05 2024-09-14T06:31:12.232 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:12.232 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:12.232 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:31:12.232 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.417497+0000", 2024-09-14T06:31:12.233 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.7317999999999995e-05 2024-09-14T06:31:12.233 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:12.233 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:31:12.233 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:12.233 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:12.233 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:12.233 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:53251 3.f 3:f71f51a7:::rbd_data.112dca2f6ae1.0000000000000015:17 [read 2035712~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:31:12.233 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:44.369475+0000", 2024-09-14T06:31:12.233 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 87.828636114000005, 2024-09-14T06:31:12.233 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.048154542000000002, 2024-09-14T06:31:12.233 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:31:12.233 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:31:12.233 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:31:12.233 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:31:12.234 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:31:12.234 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:12.234 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 53251 2024-09-14T06:31:12.234 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:12.234 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:31:12.234 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:12.234 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:31:12.234 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369475+0000", 2024-09-14T06:31:12.234 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:12.234 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:12.234 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:12.234 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:31:12.234 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369475+0000", 2024-09-14T06:31:12.234 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:12.235 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:12.235 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:12.235 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:31:12.235 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369475+0000", 2024-09-14T06:31:12.235 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.0100000000000001e-07 2024-09-14T06:31:12.235 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:12.235 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:12.235 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:31:12.235 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369476+0000", 2024-09-14T06:31:12.235 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.0580000000000001e-06 2024-09-14T06:31:12.235 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:12.235 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:12.235 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:31:12.236 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369479+0000", 2024-09-14T06:31:12.236 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.9950000000000001e-06 2024-09-14T06:31:12.236 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:12.236 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:12.236 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:31:12.236 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369482+0000", 2024-09-14T06:31:12.236 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.6340000000000001e-06 2024-09-14T06:31:12.236 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:12.236 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:12.236 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:31:12.236 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.417530+0000", 2024-09-14T06:31:12.236 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.048047535000000002 2024-09-14T06:31:12.236 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:12.236 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:12.236 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:31:12.237 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.417574+0000", 2024-09-14T06:31:12.237 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.4255999999999998e-05 2024-09-14T06:31:12.237 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:12.237 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:12.237 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:31:12.237 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.417630+0000", 2024-09-14T06:31:12.237 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.5862999999999997e-05 2024-09-14T06:31:12.237 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:12.237 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:31:12.237 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:12.237 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:12.237 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:12.237 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:53252 3.f 3:f71f51a7:::rbd_data.112dca2f6ae1.0000000000000015:17 [read 2060288~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:31:12.237 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:44.369485+0000", 2024-09-14T06:31:12.237 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 87.828626266000001, 2024-09-14T06:31:12.238 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.048254116, 2024-09-14T06:31:12.238 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:31:12.238 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:31:12.238 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:31:12.238 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:31:12.238 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:31:12.238 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:12.238 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 53252 2024-09-14T06:31:12.238 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:12.238 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:31:12.238 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:12.238 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:31:12.238 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369485+0000", 2024-09-14T06:31:12.238 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:12.238 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:12.239 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:12.239 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:31:12.239 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369485+0000", 2024-09-14T06:31:12.239 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:12.239 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:12.239 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:12.239 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:31:12.239 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369485+0000", 2024-09-14T06:31:12.239 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.1e-07 2024-09-14T06:31:12.239 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:12.239 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:12.239 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:31:12.239 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369487+0000", 2024-09-14T06:31:12.240 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.42e-06 2024-09-14T06:31:12.240 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:12.240 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:12.240 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:31:12.240 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369490+0000", 2024-09-14T06:31:12.240 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.9639999999999999e-06 2024-09-14T06:31:12.240 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:12.240 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:12.240 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:31:12.240 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369492+0000", 2024-09-14T06:31:12.240 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.5679999999999998e-06 2024-09-14T06:31:12.240 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:12.240 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:12.240 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:31:12.240 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.417645+0000", 2024-09-14T06:31:12.241 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.048152753 2024-09-14T06:31:12.241 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:12.241 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:12.241 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:31:12.241 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.417687+0000", 2024-09-14T06:31:12.241 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.1900999999999997e-05 2024-09-14T06:31:12.241 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:12.241 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:12.241 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:31:12.241 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.417739+0000", 2024-09-14T06:31:12.241 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.2299999999999997e-05 2024-09-14T06:31:12.241 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:12.241 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:31:12.242 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:12.242 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:12.242 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:12.242 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:53253 3.f 3:f71f51a7:::rbd_data.112dca2f6ae1.0000000000000015:17 [read 2084864~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:31:12.242 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:44.369495+0000", 2024-09-14T06:31:12.242 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 87.828616142000001, 2024-09-14T06:31:12.242 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.048366785000000002, 2024-09-14T06:31:12.242 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:31:12.242 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:31:12.242 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:31:12.242 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:31:12.242 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:31:12.242 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:12.242 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 53253 2024-09-14T06:31:12.243 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:12.243 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:31:12.243 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:12.243 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:31:12.243 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369495+0000", 2024-09-14T06:31:12.243 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:12.243 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:12.243 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:12.243 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:31:12.243 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369495+0000", 2024-09-14T06:31:12.243 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:12.243 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:12.243 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:12.244 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:31:12.244 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370388+0000", 2024-09-14T06:31:12.244 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00089287999999999998 2024-09-14T06:31:12.244 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:12.244 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:12.244 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:31:12.244 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370392+0000", 2024-09-14T06:31:12.244 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.2130000000000002e-06 2024-09-14T06:31:12.244 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:12.244 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:12.244 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:31:12.244 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370401+0000", 2024-09-14T06:31:12.244 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.7649999999999999e-06 2024-09-14T06:31:12.245 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:12.245 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:12.245 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:31:12.245 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370406+0000", 2024-09-14T06:31:12.245 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.0159999999999999e-06 2024-09-14T06:31:12.245 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:12.245 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:12.245 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:31:12.245 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.417764+0000", 2024-09-14T06:31:12.245 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047358348000000001 2024-09-14T06:31:12.245 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:12.245 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:12.245 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:31:12.245 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.417806+0000", 2024-09-14T06:31:12.246 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.1149999999999997e-05 2024-09-14T06:31:12.246 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:12.246 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:12.246 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:31:12.246 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.417862+0000", 2024-09-14T06:31:12.246 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.6412999999999997e-05 2024-09-14T06:31:12.246 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:12.246 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:31:12.246 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:12.246 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:12.246 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:12.246 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:53340 3.f 3:f71f51a7:::rbd_data.112dca2f6ae1.0000000000000015:17 [read 1056768~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:31:12.246 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:44.370421+0000", 2024-09-14T06:31:12.246 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 87.827690262000004, 2024-09-14T06:31:12.247 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047566313999999998, 2024-09-14T06:31:12.247 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:31:12.247 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:31:12.247 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:31:12.247 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:31:12.247 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:31:12.247 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:12.247 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 53340 2024-09-14T06:31:12.247 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:12.247 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:31:12.247 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:12.247 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:31:12.247 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370421+0000", 2024-09-14T06:31:12.247 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:12.247 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:12.248 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:12.248 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:31:12.248 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370421+0000", 2024-09-14T06:31:12.248 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:12.248 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:12.248 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:12.248 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:31:12.248 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370422+0000", 2024-09-14T06:31:12.248 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.3799999999999996e-07 2024-09-14T06:31:12.248 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:12.248 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:12.248 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:31:12.248 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370424+0000", 2024-09-14T06:31:12.249 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.6919999999999998e-06 2024-09-14T06:31:12.249 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:12.249 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:12.249 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:31:12.249 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370432+0000", 2024-09-14T06:31:12.249 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.2439999999999996e-06 2024-09-14T06:31:12.249 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:12.249 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:12.249 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:31:12.249 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370437+0000", 2024-09-14T06:31:12.249 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.5389999999999998e-06 2024-09-14T06:31:12.249 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:12.249 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:12.249 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:31:12.249 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.417888+0000", 2024-09-14T06:31:12.250 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047450432000000001 2024-09-14T06:31:12.250 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:12.250 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:12.250 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:31:12.250 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.417935+0000", 2024-09-14T06:31:12.250 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.7454999999999999e-05 2024-09-14T06:31:12.250 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:12.250 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:12.250 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:31:12.250 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.417987+0000", 2024-09-14T06:31:12.250 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.2213999999999998e-05 2024-09-14T06:31:12.250 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:12.250 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:31:12.250 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:12.251 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:12.251 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:12.251 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:53341 3.f 3:f71f51a7:::rbd_data.112dca2f6ae1.0000000000000015:17 [read 1081344~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:31:12.251 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:44.370446+0000", 2024-09-14T06:31:12.251 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 87.827664988999999, 2024-09-14T06:31:12.251 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047665723, 2024-09-14T06:31:12.251 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:31:12.251 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:31:12.251 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:31:12.251 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:31:12.251 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:31:12.251 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:12.251 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 53341 2024-09-14T06:31:12.251 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:12.252 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:31:12.252 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:12.252 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:31:12.252 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370446+0000", 2024-09-14T06:31:12.252 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:12.252 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:12.252 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:12.252 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:31:12.252 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370446+0000", 2024-09-14T06:31:12.252 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:12.252 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:12.252 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:12.252 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:31:12.252 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370447+0000", 2024-09-14T06:31:12.253 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.6100000000000002e-07 2024-09-14T06:31:12.253 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:12.253 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:12.253 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:31:12.253 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370448+0000", 2024-09-14T06:31:12.253 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.7370000000000001e-06 2024-09-14T06:31:12.253 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:12.253 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:12.253 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:31:12.253 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370462+0000", 2024-09-14T06:31:12.253 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.3227000000000001e-05 2024-09-14T06:31:12.253 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:12.253 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:12.253 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:31:12.254 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370467+0000", 2024-09-14T06:31:12.254 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.6540000000000004e-06 2024-09-14T06:31:12.254 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:12.254 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:12.254 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:31:12.254 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.418013+0000", 2024-09-14T06:31:12.254 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047545325999999999 2024-09-14T06:31:12.254 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:12.254 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:12.254 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:31:12.254 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.418054+0000", 2024-09-14T06:31:12.254 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.1696999999999997e-05 2024-09-14T06:31:12.254 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:12.254 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:12.255 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:31:12.255 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.418112+0000", 2024-09-14T06:31:12.255 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.7720999999999998e-05 2024-09-14T06:31:12.255 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:12.255 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:31:12.255 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:12.255 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:12.255 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:12.255 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:53342 3.f 3:f71f51a7:::rbd_data.112dca2f6ae1.0000000000000015:17 [read 1105920~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:31:12.255 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:44.370475+0000", 2024-09-14T06:31:12.255 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 87.827635928000007, 2024-09-14T06:31:12.255 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047748848000000003, 2024-09-14T06:31:12.255 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:31:12.255 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:31:12.256 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:31:12.256 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:31:12.256 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:31:12.256 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:12.256 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 53342 2024-09-14T06:31:12.256 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:12.256 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:31:12.256 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:12.256 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:31:12.256 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370475+0000", 2024-09-14T06:31:12.256 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:12.256 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:12.256 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:12.256 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:31:12.257 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370475+0000", 2024-09-14T06:31:12.257 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:12.257 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:12.257 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:12.257 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:31:12.257 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370476+0000", 2024-09-14T06:31:12.257 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.4700000000000002e-07 2024-09-14T06:31:12.257 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:12.257 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:12.257 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:31:12.257 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370478+0000", 2024-09-14T06:31:12.257 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.5890000000000001e-06 2024-09-14T06:31:12.257 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:12.258 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:12.258 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:31:12.258 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370486+0000", 2024-09-14T06:31:12.258 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.3470000000000001e-06 2024-09-14T06:31:12.258 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:12.258 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:12.258 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:31:12.258 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370491+0000", 2024-09-14T06:31:12.258 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.5720000000000002e-06 2024-09-14T06:31:12.258 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:12.258 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:12.258 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:31:12.258 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.418135+0000", 2024-09-14T06:31:12.258 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047644094999999997 2024-09-14T06:31:12.258 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:12.259 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:12.259 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:31:12.259 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.418174+0000", 2024-09-14T06:31:12.259 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.8204000000000002e-05 2024-09-14T06:31:12.259 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:12.259 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:12.259 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:31:12.259 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.418224+0000", 2024-09-14T06:31:12.259 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.0593999999999997e-05 2024-09-14T06:31:12.259 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:12.259 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:31:12.259 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:12.259 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:12.259 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:12.260 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:53343 3.f 3:f71f51a7:::rbd_data.112dca2f6ae1.0000000000000015:17 [read 1130496~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:31:12.260 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:44.370498+0000", 2024-09-14T06:31:12.260 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 87.827613052000004, 2024-09-14T06:31:12.260 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047841574999999997, 2024-09-14T06:31:12.260 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:31:12.260 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:31:12.260 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:31:12.260 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:31:12.260 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:31:12.260 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:12.260 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 53343 2024-09-14T06:31:12.260 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:12.260 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:31:12.261 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:12.261 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:31:12.261 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370498+0000", 2024-09-14T06:31:12.261 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:12.261 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:12.261 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:12.261 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:31:12.261 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370498+0000", 2024-09-14T06:31:12.261 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:12.261 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:12.261 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:12.261 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:31:12.261 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370499+0000", 2024-09-14T06:31:12.262 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.9999999999999998e-07 2024-09-14T06:31:12.262 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:12.262 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:12.262 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:31:12.262 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370501+0000", 2024-09-14T06:31:12.262 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.0310000000000001e-06 2024-09-14T06:31:12.262 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:12.262 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:12.262 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:31:12.262 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370507+0000", 2024-09-14T06:31:12.262 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.4169999999999997e-06 2024-09-14T06:31:12.262 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:12.262 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:12.262 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:31:12.263 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370516+0000", 2024-09-14T06:31:12.263 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.7190000000000007e-06 2024-09-14T06:31:12.263 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:12.263 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:12.263 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:31:12.263 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.418247+0000", 2024-09-14T06:31:12.263 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047731190999999999 2024-09-14T06:31:12.263 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:12.263 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:12.263 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:31:12.263 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.418286+0000", 2024-09-14T06:31:12.263 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.8988e-05 2024-09-14T06:31:12.263 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:12.263 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:12.264 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:31:12.264 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.418340+0000", 2024-09-14T06:31:12.264 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.3829000000000002e-05 2024-09-14T06:31:12.264 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:12.264 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:31:12.264 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:12.264 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:12.264 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:12.264 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:53344 3.f 3:f71f51a7:::rbd_data.112dca2f6ae1.0000000000000015:17 [read 1155072~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:31:12.264 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:44.370524+0000", 2024-09-14T06:31:12.264 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 87.827587281999996, 2024-09-14T06:31:12.264 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047972262000000002, 2024-09-14T06:31:12.264 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:31:12.264 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:31:12.265 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:31:12.265 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:31:12.265 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:31:12.265 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:12.265 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 53344 2024-09-14T06:31:12.265 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:12.265 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:31:12.265 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:12.265 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:31:12.265 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370524+0000", 2024-09-14T06:31:12.265 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:12.265 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:12.265 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:12.266 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:31:12.266 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370524+0000", 2024-09-14T06:31:12.266 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:12.266 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:12.266 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:12.266 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:31:12.266 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370618+0000", 2024-09-14T06:31:12.266 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.4425999999999993e-05 2024-09-14T06:31:12.266 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:12.266 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:12.266 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:31:12.266 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370622+0000", 2024-09-14T06:31:12.266 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.3519999999999998e-06 2024-09-14T06:31:12.266 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:12.267 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:12.267 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:31:12.267 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370631+0000", 2024-09-14T06:31:12.267 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.9299999999999992e-06 2024-09-14T06:31:12.267 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:12.267 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:12.267 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:31:12.267 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370637+0000", 2024-09-14T06:31:12.267 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.3269999999999997e-06 2024-09-14T06:31:12.267 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:12.267 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:12.267 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:31:12.267 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.418372+0000", 2024-09-14T06:31:12.268 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047734764999999998 2024-09-14T06:31:12.268 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:12.268 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:12.268 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:31:12.268 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.418431+0000", 2024-09-14T06:31:12.268 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.8696999999999997e-05 2024-09-14T06:31:12.268 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:12.268 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:12.268 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:31:12.268 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.418496+0000", 2024-09-14T06:31:12.268 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.5765000000000005e-05 2024-09-14T06:31:12.268 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:12.268 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:31:12.268 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:12.269 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:12.269 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:12.269 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:53345 3.f 3:f71f51a7:::rbd_data.112dca2f6ae1.0000000000000015:17 [read 1179648~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:31:12.269 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:44.370647+0000", 2024-09-14T06:31:12.269 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 87.827463815000002, 2024-09-14T06:31:12.269 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047953775999999997, 2024-09-14T06:31:12.269 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:31:12.269 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:31:12.269 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:31:12.269 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:31:12.269 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:31:12.269 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:12.269 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 53345 2024-09-14T06:31:12.269 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:12.269 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:31:12.270 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:12.270 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:31:12.270 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370647+0000", 2024-09-14T06:31:12.270 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:12.270 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:12.270 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:12.270 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:31:12.270 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370647+0000", 2024-09-14T06:31:12.270 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:12.270 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:12.270 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:12.270 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:31:12.270 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.372346+0000", 2024-09-14T06:31:12.270 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.001698284 2024-09-14T06:31:12.271 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:12.271 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:12.271 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:31:12.271 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.372351+0000", 2024-09-14T06:31:12.271 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.8890000000000001e-06 2024-09-14T06:31:12.271 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:12.271 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:12.271 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:31:12.271 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.372362+0000", 2024-09-14T06:31:12.271 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.0941000000000001e-05 2024-09-14T06:31:12.271 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:12.271 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:12.271 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:31:12.272 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.372369+0000", 2024-09-14T06:31:12.272 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.362e-06 2024-09-14T06:31:12.272 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:12.272 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:12.272 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:31:12.272 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.418510+0000", 2024-09-14T06:31:12.272 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046140914999999998 2024-09-14T06:31:12.272 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:12.272 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:12.272 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:31:12.272 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.418549+0000", 2024-09-14T06:31:12.272 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.8717999999999998e-05 2024-09-14T06:31:12.272 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:12.272 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:12.272 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:31:12.273 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.418601+0000", 2024-09-14T06:31:12.273 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.2667000000000002e-05 2024-09-14T06:31:12.273 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:12.273 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:31:12.273 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:12.273 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:12.273 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:12.273 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:53351 3.f 3:f71f51a7:::rbd_data.112dca2f6ae1.0000000000000015:17 [read 1327104~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:31:12.273 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:44.372499+0000", 2024-09-14T06:31:12.273 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 87.825612694, 2024-09-14T06:31:12.273 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047140596, 2024-09-14T06:31:12.273 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:31:12.273 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:31:12.274 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:31:12.274 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:31:12.274 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:31:12.274 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:12.274 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 53351 2024-09-14T06:31:12.274 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:12.274 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:31:12.274 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:12.274 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:31:12.274 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.372499+0000", 2024-09-14T06:31:12.274 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:12.274 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:12.274 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:12.274 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:31:12.274 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.372499+0000", 2024-09-14T06:31:12.275 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:12.275 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:12.275 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:12.275 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:31:12.275 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.372499+0000", 2024-09-14T06:31:12.275 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.3000000000000001e-07 2024-09-14T06:31:12.275 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:12.275 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:12.275 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:31:12.275 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.372501+0000", 2024-09-14T06:31:12.275 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.0820000000000001e-06 2024-09-14T06:31:12.275 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:12.275 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:12.275 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:31:12.275 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.372513+0000", 2024-09-14T06:31:12.276 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.1610999999999999e-05 2024-09-14T06:31:12.276 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:12.276 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:12.276 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:31:12.276 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.372518+0000", 2024-09-14T06:31:12.276 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.327e-06 2024-09-14T06:31:12.276 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:12.276 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:12.276 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:31:12.276 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.419530+0000", 2024-09-14T06:31:12.276 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047011798 2024-09-14T06:31:12.276 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:12.276 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:12.277 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:31:12.277 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.419578+0000", 2024-09-14T06:31:12.277 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.8446999999999999e-05 2024-09-14T06:31:12.277 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:12.277 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:12.277 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:31:12.277 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.419639+0000", 2024-09-14T06:31:12.277 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.0900999999999998e-05 2024-09-14T06:31:12.277 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:12.277 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:31:12.277 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:12.277 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:12.277 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:12.277 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:53352 3.f 3:f71f51a7:::rbd_data.112dca2f6ae1.0000000000000015:17 [read 1351680~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:31:12.277 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:44.372525+0000", 2024-09-14T06:31:12.278 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 87.825586779999995, 2024-09-14T06:31:12.278 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047259847000000001, 2024-09-14T06:31:12.278 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:31:12.278 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:31:12.278 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:31:12.278 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:31:12.278 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:31:12.278 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:12.278 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 53352 2024-09-14T06:31:12.278 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:12.278 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:31:12.278 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:12.278 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:31:12.278 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.372525+0000", 2024-09-14T06:31:12.279 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:12.279 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:12.279 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:12.279 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:31:12.279 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.372525+0000", 2024-09-14T06:31:12.279 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:12.279 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:12.279 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:12.279 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:31:12.279 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.372525+0000", 2024-09-14T06:31:12.279 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.0600000000000001e-07 2024-09-14T06:31:12.279 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:12.279 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:12.279 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:31:12.280 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.372527+0000", 2024-09-14T06:31:12.280 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.0499999999999999e-06 2024-09-14T06:31:12.280 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:12.280 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:12.280 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:31:12.280 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.372533+0000", 2024-09-14T06:31:12.280 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.4320000000000004e-06 2024-09-14T06:31:12.280 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:12.280 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:12.280 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:31:12.280 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.372538+0000", 2024-09-14T06:31:12.280 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.9130000000000002e-06 2024-09-14T06:31:12.280 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:12.281 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:12.281 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:31:12.281 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.419667+0000", 2024-09-14T06:31:12.281 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047128976000000003 2024-09-14T06:31:12.281 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:12.281 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:12.281 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:31:12.281 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.419715+0000", 2024-09-14T06:31:12.281 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.7868999999999999e-05 2024-09-14T06:31:12.281 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:12.281 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:12.281 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:31:12.281 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.419784+0000", 2024-09-14T06:31:12.281 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.9201000000000004e-05 2024-09-14T06:31:12.282 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:12.282 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:31:12.282 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:12.282 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:12.282 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:12.282 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:53353 3.f 3:f71f51a7:::rbd_data.112dca2f6ae1.0000000000000015:17 [read 1376256~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:31:12.282 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:44.372545+0000", 2024-09-14T06:31:12.282 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 87.825566739999999, 2024-09-14T06:31:12.282 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047364232999999999, 2024-09-14T06:31:12.282 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:31:12.282 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:31:12.282 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:31:12.282 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:31:12.282 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:31:12.283 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:12.283 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 53353 2024-09-14T06:31:12.283 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:12.283 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:31:12.283 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:12.283 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:31:12.283 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.372545+0000", 2024-09-14T06:31:12.283 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:12.283 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:12.283 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:12.283 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:31:12.283 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.372545+0000", 2024-09-14T06:31:12.283 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:12.283 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:12.283 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:12.283 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:31:12.284 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.372545+0000", 2024-09-14T06:31:12.284 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.27e-07 2024-09-14T06:31:12.284 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:12.284 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:12.284 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:31:12.284 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.372547+0000", 2024-09-14T06:31:12.284 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.0600000000000002e-06 2024-09-14T06:31:12.284 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:12.284 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:12.284 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:31:12.284 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.372553+0000", 2024-09-14T06:31:12.284 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.7980000000000002e-06 2024-09-14T06:31:12.284 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:12.284 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:12.285 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:31:12.285 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.372557+0000", 2024-09-14T06:31:12.285 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.7389999999999999e-06 2024-09-14T06:31:12.285 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:12.285 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:12.285 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:31:12.285 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.419811+0000", 2024-09-14T06:31:12.285 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047253310999999999 2024-09-14T06:31:12.285 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:12.285 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:12.285 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:31:12.285 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.419854+0000", 2024-09-14T06:31:12.285 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.3448000000000001e-05 2024-09-14T06:31:12.285 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:12.286 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:12.286 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:31:12.286 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.419909+0000", 2024-09-14T06:31:12.286 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.4549999999999998e-05 2024-09-14T06:31:12.286 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:12.286 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:31:12.286 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:12.286 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:12.286 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:12.286 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:53354 3.f 3:f71f51a7:::rbd_data.112dca2f6ae1.0000000000000015:17 [read 1400832~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:31:12.286 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:44.372563+0000", 2024-09-14T06:31:12.286 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 87.825548303000005, 2024-09-14T06:31:12.286 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047482233999999998, 2024-09-14T06:31:12.286 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:31:12.286 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:31:12.286 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:31:12.287 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:31:12.287 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:31:12.287 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:12.287 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 53354 2024-09-14T06:31:12.287 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:12.287 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:31:12.287 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:12.287 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:31:12.287 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.372563+0000", 2024-09-14T06:31:12.287 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:12.287 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:12.287 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:12.287 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:31:12.288 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.372563+0000", 2024-09-14T06:31:12.288 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:12.288 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:12.288 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:12.288 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:31:12.288 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.375009+0000", 2024-09-14T06:31:12.288 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.0024457680000000001 2024-09-14T06:31:12.288 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:12.288 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:12.288 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:31:12.288 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.375014+0000", 2024-09-14T06:31:12.288 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.0599999999999998e-06 2024-09-14T06:31:12.288 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:12.288 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:12.289 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:31:12.289 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.375026+0000", 2024-09-14T06:31:12.289 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.1783999999999999e-05 2024-09-14T06:31:12.289 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:12.289 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:12.289 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:31:12.289 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.375035+0000", 2024-09-14T06:31:12.289 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.7799999999999995e-06 2024-09-14T06:31:12.289 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:12.289 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:12.289 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:31:12.289 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.419942+0000", 2024-09-14T06:31:12.289 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.044906982999999998 2024-09-14T06:31:12.290 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:12.290 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:12.290 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:31:12.290 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.419989+0000", 2024-09-14T06:31:12.290 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.7104999999999997e-05 2024-09-14T06:31:12.290 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:12.290 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:12.290 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:31:12.290 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.420045+0000", 2024-09-14T06:31:12.290 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.5754000000000002e-05 2024-09-14T06:31:12.290 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:12.290 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:31:12.290 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:12.290 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:12.291 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:31:12.291 INFO:teuthology.orchestra.run.smithi138.stdout:} 2024-09-14T06:31:12.292 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-14T06:31:12.293 INFO:tasks.ceph.osd.0.smithi094.stderr:2024-09-14T06:31:12.291+0000 7f07c9333640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33636) UID: 0 2024-09-14T06:31:12.393 INFO:tasks.ceph.osd.6.smithi138.stderr:2024-09-14T06:31:12.392+0000 7f8cd8849640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33620) UID: 0 2024-09-14T06:31:12.395 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-14T06:31:12.395 INFO:teuthology.orchestra.run.smithi094.stdout: "ops": [], 2024-09-14T06:31:12.395 INFO:teuthology.orchestra.run.smithi094.stdout: "num_ops": 0 2024-09-14T06:31:12.395 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-14T06:31:12.403 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-14T06:31:12.480 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-14T06:31:12.480 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-14T06:31:12.480 INFO:tasks.thrashosds.thrasher:primary_affinity 2024-09-14T06:31:12.480 INFO:tasks.thrashosds.thrasher:Setting osd 3 primary_affinity to 1.000000 2024-09-14T06:31:12.481 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 3 1 2024-09-14T06:31:12.494 INFO:tasks.ceph.osd.7.smithi138.stderr:2024-09-14T06:31:12.492+0000 7f3c6a21d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33623) UID: 0 2024-09-14T06:31:12.507 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-14T06:31:12.507 INFO:teuthology.orchestra.run.smithi094.stdout: "ops": [], 2024-09-14T06:31:12.507 INFO:teuthology.orchestra.run.smithi094.stdout: "complaint_time": 30, 2024-09-14T06:31:12.507 INFO:teuthology.orchestra.run.smithi094.stdout: "num_blocked_ops": 0 2024-09-14T06:31:12.507 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-14T06:31:12.516 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-14T06:31:12.600 INFO:tasks.ceph.osd.0.smithi094.stderr:2024-09-14T06:31:12.599+0000 7f07c9333640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33636) UID: 0 2024-09-14T06:31:12.662 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-14T06:31:12.663 INFO:teuthology.orchestra.run.smithi094.stdout: "size": 20, 2024-09-14T06:31:12.663 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 600, 2024-09-14T06:31:12.663 INFO:teuthology.orchestra.run.smithi094.stdout: "ops": [ 2024-09-14T06:31:12.663 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:12.663 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:115193 3.1 3:856be886:::rbd_data.112d4d24c558.0000000000000004:29 [read 1523712~16384 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e96)", 2024-09-14T06:31:12.663 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:30:17.939854+0000", 2024-09-14T06:31:12.663 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 54.721148231999997, 2024-09-14T06:31:12.663 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.073897516999999996, 2024-09-14T06:31:12.663 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:31:12.663 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:31:12.663 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:31:12.664 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:31:12.664 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:31:12.664 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:12.664 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 115193 2024-09-14T06:31:12.664 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:12.664 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:31:12.664 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:12.664 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:31:12.664 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.939854+0000", 2024-09-14T06:31:12.664 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:12.664 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:12.664 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:12.664 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:31:12.664 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.939854+0000", 2024-09-14T06:31:12.665 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:12.665 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:12.665 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:12.665 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:31:12.665 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.939854+0000", 2024-09-14T06:31:12.665 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.9500000000000002e-07 2024-09-14T06:31:12.665 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:12.665 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:12.665 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:31:12.665 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.939856+0000", 2024-09-14T06:31:12.665 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.0959999999999999e-06 2024-09-14T06:31:12.665 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:12.665 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:12.665 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:31:12.666 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.939863+0000", 2024-09-14T06:31:12.666 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.1129999999999997e-06 2024-09-14T06:31:12.666 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:12.666 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:12.666 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:31:12.666 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.939867+0000", 2024-09-14T06:31:12.666 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.8350000000000003e-06 2024-09-14T06:31:12.666 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:12.666 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:12.666 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:31:12.666 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.013626+0000", 2024-09-14T06:31:12.666 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.073758799999999999 2024-09-14T06:31:12.666 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:12.667 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:12.667 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:31:12.667 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.013690+0000", 2024-09-14T06:31:12.667 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.3918999999999996e-05 2024-09-14T06:31:12.667 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:12.667 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:12.667 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:31:12.667 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.013751+0000", 2024-09-14T06:31:12.667 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.1159000000000002e-05 2024-09-14T06:31:12.667 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:12.667 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:31:12.667 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:12.667 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:12.667 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:12.668 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:115194 3.1 3:856be886:::rbd_data.112d4d24c558.0000000000000004:29 [read 1769472~16384 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e96)", 2024-09-14T06:31:12.668 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:30:17.939874+0000", 2024-09-14T06:31:12.668 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 54.721127920999997, 2024-09-14T06:31:12.668 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.074016988000000006, 2024-09-14T06:31:12.668 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:31:12.668 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:31:12.668 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:31:12.668 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:31:12.668 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:31:12.668 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:12.668 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 115194 2024-09-14T06:31:12.668 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:12.668 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:31:12.668 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:12.669 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:31:12.669 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.939874+0000", 2024-09-14T06:31:12.669 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:12.669 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:12.669 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:12.669 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:31:12.669 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.939874+0000", 2024-09-14T06:31:12.669 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:12.669 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:12.669 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:12.669 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:31:12.669 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.939875+0000", 2024-09-14T06:31:12.669 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.2600000000000002e-07 2024-09-14T06:31:12.669 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:12.670 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:12.670 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:31:12.670 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.939877+0000", 2024-09-14T06:31:12.670 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.9420000000000002e-06 2024-09-14T06:31:12.670 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:12.670 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:12.670 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:31:12.670 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.939882+0000", 2024-09-14T06:31:12.670 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.8749999999999997e-06 2024-09-14T06:31:12.670 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:12.670 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:12.670 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:31:12.670 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.939887+0000", 2024-09-14T06:31:12.670 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.741e-06 2024-09-14T06:31:12.671 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:12.671 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:12.671 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:31:12.671 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.013767+0000", 2024-09-14T06:31:12.671 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.073879720999999995 2024-09-14T06:31:12.671 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:12.671 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:12.671 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:31:12.671 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.013832+0000", 2024-09-14T06:31:12.671 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.5312000000000001e-05 2024-09-14T06:31:12.671 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:12.671 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:12.671 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:31:12.671 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.013891+0000", 2024-09-14T06:31:12.672 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.8870999999999999e-05 2024-09-14T06:31:12.672 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:12.672 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:31:12.672 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:12.672 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:12.672 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:12.672 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:115195 3.1 3:856be886:::rbd_data.112d4d24c558.0000000000000004:29 [read 2015232~16384 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e96)", 2024-09-14T06:31:12.672 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:30:17.939903+0000", 2024-09-14T06:31:12.672 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 54.721098736000002, 2024-09-14T06:31:12.672 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.074128199000000006, 2024-09-14T06:31:12.672 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:31:12.672 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:31:12.672 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:31:12.673 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:31:12.673 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:31:12.673 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:12.673 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 115195 2024-09-14T06:31:12.673 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:12.673 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:31:12.673 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:12.673 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:31:12.673 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.939903+0000", 2024-09-14T06:31:12.673 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:12.673 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:12.673 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:12.673 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:31:12.673 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.939903+0000", 2024-09-14T06:31:12.673 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:12.674 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:12.674 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:12.674 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:31:12.674 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.939904+0000", 2024-09-14T06:31:12.674 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.5799999999999999e-07 2024-09-14T06:31:12.674 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:12.674 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:12.674 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:31:12.674 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.939906+0000", 2024-09-14T06:31:12.674 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.2350000000000002e-06 2024-09-14T06:31:12.674 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:12.674 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:12.674 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:31:12.675 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.939913+0000", 2024-09-14T06:31:12.675 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.5649999999999998e-06 2024-09-14T06:31:12.675 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:12.675 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:12.675 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:31:12.675 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.939918+0000", 2024-09-14T06:31:12.675 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.784e-06 2024-09-14T06:31:12.675 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:12.675 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:12.675 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:31:12.675 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.013921+0000", 2024-09-14T06:31:12.675 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.074002911000000005 2024-09-14T06:31:12.675 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:12.676 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:12.676 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:31:12.676 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.013971+0000", 2024-09-14T06:31:12.676 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.9551e-05 2024-09-14T06:31:12.676 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:12.676 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:12.676 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:31:12.676 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.014031+0000", 2024-09-14T06:31:12.676 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.0594999999999998e-05 2024-09-14T06:31:12.676 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:12.676 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:31:12.676 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:12.676 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:12.677 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:12.677 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:115196 3.1 3:856be886:::rbd_data.112d4d24c558.0000000000000004:29 [read 2260992~16384 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e96)", 2024-09-14T06:31:12.677 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:30:17.939925+0000", 2024-09-14T06:31:12.677 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 54.721076543999999, 2024-09-14T06:31:12.677 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.074320137999999994, 2024-09-14T06:31:12.677 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:31:12.677 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:31:12.677 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:31:12.677 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:31:12.677 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:31:12.677 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:12.677 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 115196 2024-09-14T06:31:12.677 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:12.678 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:31:12.678 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:12.678 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:31:12.678 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.939925+0000", 2024-09-14T06:31:12.678 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:12.678 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:12.678 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:12.678 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:31:12.678 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.939925+0000", 2024-09-14T06:31:12.678 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:12.678 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:12.678 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:12.679 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:31:12.679 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.939926+0000", 2024-09-14T06:31:12.679 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.9999999999999998e-07 2024-09-14T06:31:12.679 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:12.679 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:12.679 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:31:12.679 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.939928+0000", 2024-09-14T06:31:12.679 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.9549999999999999e-06 2024-09-14T06:31:12.679 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:12.679 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:12.679 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:31:12.679 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.939934+0000", 2024-09-14T06:31:12.679 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.127e-06 2024-09-14T06:31:12.679 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:12.680 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:12.680 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:31:12.680 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.939939+0000", 2024-09-14T06:31:12.680 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.775e-06 2024-09-14T06:31:12.680 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:12.680 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:12.680 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:31:12.680 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.014075+0000", 2024-09-14T06:31:12.680 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.074136598999999997 2024-09-14T06:31:12.680 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:12.680 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:12.680 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:31:12.680 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.014162+0000", 2024-09-14T06:31:12.680 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.6477999999999994e-05 2024-09-14T06:31:12.681 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:12.681 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:12.681 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:31:12.681 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.014246+0000", 2024-09-14T06:31:12.681 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.3703999999999997e-05 2024-09-14T06:31:12.681 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:12.681 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:31:12.681 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:12.681 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:12.681 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:12.681 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:115197 3.1 3:856be886:::rbd_data.112d4d24c558.0000000000000004:29 [read 2506752~16384 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e96)", 2024-09-14T06:31:12.681 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:30:17.939969+0000", 2024-09-14T06:31:12.681 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 54.721033059, 2024-09-14T06:31:12.682 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.074528544000000002, 2024-09-14T06:31:12.682 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:31:12.682 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:31:12.682 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:31:12.682 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:31:12.682 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:31:12.682 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:12.682 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 115197 2024-09-14T06:31:12.682 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:12.682 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:31:12.682 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:12.682 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:31:12.682 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.939969+0000", 2024-09-14T06:31:12.683 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:12.683 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:12.683 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:12.683 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:31:12.683 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.939969+0000", 2024-09-14T06:31:12.683 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:12.683 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:12.683 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:12.683 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:31:12.683 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.939970+0000", 2024-09-14T06:31:12.683 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.3600000000000003e-07 2024-09-14T06:31:12.683 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:12.683 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:12.683 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:31:12.684 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.939972+0000", 2024-09-14T06:31:12.684 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.3180000000000001e-06 2024-09-14T06:31:12.684 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:12.684 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:12.684 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:31:12.684 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.939979+0000", 2024-09-14T06:31:12.684 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.8800000000000002e-06 2024-09-14T06:31:12.684 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:12.684 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:12.684 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:31:12.684 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.939984+0000", 2024-09-14T06:31:12.684 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.3510000000000001e-06 2024-09-14T06:31:12.684 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:12.684 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:12.685 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:31:12.685 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.014302+0000", 2024-09-14T06:31:12.685 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.074317416999999997 2024-09-14T06:31:12.685 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:12.685 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:12.685 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:31:12.685 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.014392+0000", 2024-09-14T06:31:12.685 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.0181999999999998e-05 2024-09-14T06:31:12.685 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:12.685 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:12.685 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:31:12.685 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.014497+0000", 2024-09-14T06:31:12.685 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00010576 2024-09-14T06:31:12.686 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:12.686 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:31:12.686 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:12.686 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:12.686 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:12.686 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:115198 3.1 3:856be886:::rbd_data.112d4d24c558.0000000000000004:29 [read 2752512~16384 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e96)", 2024-09-14T06:31:12.686 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:30:17.940004+0000", 2024-09-14T06:31:12.686 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 54.720997779000001, 2024-09-14T06:31:12.686 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.074736367999999997, 2024-09-14T06:31:12.686 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:31:12.686 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:31:12.686 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:31:12.686 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:31:12.687 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:31:12.687 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:12.687 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 115198 2024-09-14T06:31:12.687 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:12.687 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:31:12.687 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:12.687 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:31:12.687 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940004+0000", 2024-09-14T06:31:12.687 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:12.687 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:12.687 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:12.687 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:31:12.687 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940004+0000", 2024-09-14T06:31:12.687 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:12.688 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:12.688 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:12.688 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:31:12.688 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940005+0000", 2024-09-14T06:31:12.688 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.2200000000000004e-07 2024-09-14T06:31:12.688 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:12.688 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:12.688 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:31:12.688 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940007+0000", 2024-09-14T06:31:12.688 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.322e-06 2024-09-14T06:31:12.688 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:12.688 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:12.688 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:31:12.688 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940014+0000", 2024-09-14T06:31:12.689 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.6220000000000003e-06 2024-09-14T06:31:12.689 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:12.689 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:12.689 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:31:12.689 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940019+0000", 2024-09-14T06:31:12.689 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.6380000000000001e-06 2024-09-14T06:31:12.689 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:12.689 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:12.689 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:31:12.689 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.014562+0000", 2024-09-14T06:31:12.689 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.074542320999999995 2024-09-14T06:31:12.689 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:12.689 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:12.690 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:31:12.690 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.014646+0000", 2024-09-14T06:31:12.690 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.4079999999999995e-05 2024-09-14T06:31:12.690 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:12.690 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:12.690 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:31:12.690 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.014741+0000", 2024-09-14T06:31:12.690 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.4763000000000001e-05 2024-09-14T06:31:12.690 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:12.690 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:31:12.690 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:12.690 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:12.690 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:12.690 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:115199 3.1 3:856be886:::rbd_data.112d4d24c558.0000000000000004:29 [read 2998272~16384 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e96)", 2024-09-14T06:31:12.691 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:30:17.940040+0000", 2024-09-14T06:31:12.691 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 54.720962024000002, 2024-09-14T06:31:12.691 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.074957134999999994, 2024-09-14T06:31:12.691 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:31:12.691 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:31:12.691 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:31:12.691 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:31:12.691 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:31:12.691 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:12.691 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 115199 2024-09-14T06:31:12.691 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:12.691 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:31:12.691 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:12.692 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:31:12.692 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940040+0000", 2024-09-14T06:31:12.692 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:12.692 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:12.692 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:12.692 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:31:12.692 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940040+0000", 2024-09-14T06:31:12.692 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:12.692 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:12.692 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:12.692 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:31:12.692 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940041+0000", 2024-09-14T06:31:12.692 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.4000000000000001e-07 2024-09-14T06:31:12.692 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:12.693 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:12.693 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:31:12.693 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940046+0000", 2024-09-14T06:31:12.693 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.7250000000000002e-06 2024-09-14T06:31:12.693 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:12.693 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:12.693 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:31:12.693 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940053+0000", 2024-09-14T06:31:12.693 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.6109999999999999e-06 2024-09-14T06:31:12.693 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:12.693 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:12.693 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:31:12.693 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940058+0000", 2024-09-14T06:31:12.694 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.1590000000000001e-06 2024-09-14T06:31:12.694 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:12.694 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:12.694 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:31:12.694 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.014768+0000", 2024-09-14T06:31:12.694 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.074709624000000002 2024-09-14T06:31:12.694 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:12.694 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:12.694 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:31:12.694 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.014907+0000", 2024-09-14T06:31:12.694 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000139665 2024-09-14T06:31:12.694 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:12.694 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:12.694 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:31:12.695 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.014997+0000", 2024-09-14T06:31:12.695 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.9710999999999995e-05 2024-09-14T06:31:12.695 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:12.695 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:31:12.695 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:12.695 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:12.695 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:12.695 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:115200 3.1 3:856be886:::rbd_data.112d4d24c558.0000000000000004:29 [read 3244032~16384 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e96)", 2024-09-14T06:31:12.695 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:30:17.940080+0000", 2024-09-14T06:31:12.695 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 54.720921818999997, 2024-09-14T06:31:12.695 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.075137291999999994, 2024-09-14T06:31:12.695 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:31:12.695 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:31:12.695 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:31:12.695 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:31:12.696 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:31:12.696 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:12.696 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 115200 2024-09-14T06:31:12.696 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:12.696 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:31:12.696 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:12.696 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:31:12.696 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940080+0000", 2024-09-14T06:31:12.696 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:12.696 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:12.696 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:12.696 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:31:12.696 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940080+0000", 2024-09-14T06:31:12.697 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:12.697 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:12.697 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:12.697 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:31:12.697 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940081+0000", 2024-09-14T06:31:12.697 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.0800000000000004e-07 2024-09-14T06:31:12.697 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:12.697 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:12.697 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:31:12.697 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940083+0000", 2024-09-14T06:31:12.697 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.3920000000000001e-06 2024-09-14T06:31:12.697 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:12.697 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:12.697 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:31:12.698 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940091+0000", 2024-09-14T06:31:12.698 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.1389999999999995e-06 2024-09-14T06:31:12.698 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:12.698 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:12.698 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:31:12.698 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940097+0000", 2024-09-14T06:31:12.698 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.5550000000000002e-06 2024-09-14T06:31:12.698 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:12.698 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:12.698 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:31:12.698 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.015025+0000", 2024-09-14T06:31:12.698 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.074928579999999995 2024-09-14T06:31:12.698 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:12.698 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:12.699 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:31:12.699 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.015096+0000", 2024-09-14T06:31:12.699 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.0705999999999997e-05 2024-09-14T06:31:12.699 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:12.699 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:12.699 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:31:12.699 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.015217+0000", 2024-09-14T06:31:12.699 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00012131200000000001 2024-09-14T06:31:12.699 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:12.699 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:31:12.699 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:12.699 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:12.699 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:12.699 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:115201 3.1 3:856be886:::rbd_data.112d4d24c558.0000000000000004:29 [read 3489792~16384 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e96)", 2024-09-14T06:31:12.700 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:30:17.940113+0000", 2024-09-14T06:31:12.700 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 54.720889046000003, 2024-09-14T06:31:12.700 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.075246899000000006, 2024-09-14T06:31:12.700 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:31:12.700 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:31:12.700 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:31:12.700 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:31:12.700 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:31:12.700 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:12.700 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 115201 2024-09-14T06:31:12.700 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:12.700 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:31:12.700 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:12.701 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:31:12.701 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940113+0000", 2024-09-14T06:31:12.701 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:12.701 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:12.701 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:12.701 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:31:12.701 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940113+0000", 2024-09-14T06:31:12.701 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:12.701 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:12.701 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:12.701 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:31:12.701 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940114+0000", 2024-09-14T06:31:12.701 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.1500000000000004e-07 2024-09-14T06:31:12.701 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:12.702 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:12.702 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:31:12.702 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940116+0000", 2024-09-14T06:31:12.702 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.2809999999999998e-06 2024-09-14T06:31:12.702 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:12.702 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:12.702 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:31:12.702 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940123+0000", 2024-09-14T06:31:12.702 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.9229999999999996e-06 2024-09-14T06:31:12.702 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:12.702 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:12.702 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:31:12.702 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940128+0000", 2024-09-14T06:31:12.703 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.7790000000000003e-06 2024-09-14T06:31:12.703 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:12.703 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:12.703 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:31:12.703 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.015253+0000", 2024-09-14T06:31:12.703 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.075125792999999996 2024-09-14T06:31:12.703 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:12.703 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:12.703 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:31:12.703 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.015297+0000", 2024-09-14T06:31:12.703 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.3847000000000002e-05 2024-09-14T06:31:12.703 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:12.703 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:12.703 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:31:12.704 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.015360+0000", 2024-09-14T06:31:12.704 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.2661000000000002e-05 2024-09-14T06:31:12.704 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:12.704 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:31:12.704 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:12.704 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:12.704 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:12.704 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:115202 3.1 3:856be886:::rbd_data.112d4d24c558.0000000000000004:29 [read 3735552~16384 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e96)", 2024-09-14T06:31:12.704 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:30:17.940135+0000", 2024-09-14T06:31:12.704 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 54.720867267000003, 2024-09-14T06:31:12.704 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.075451105000000004, 2024-09-14T06:31:12.704 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:31:12.704 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:31:12.704 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:31:12.705 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:31:12.705 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:31:12.705 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:12.705 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 115202 2024-09-14T06:31:12.705 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:12.705 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:31:12.705 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:12.705 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:31:12.705 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940135+0000", 2024-09-14T06:31:12.705 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:12.705 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:12.705 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:12.705 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:31:12.705 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940135+0000", 2024-09-14T06:31:12.706 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:12.706 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:12.706 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:12.706 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:31:12.706 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940135+0000", 2024-09-14T06:31:12.706 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.0999999999999999e-07 2024-09-14T06:31:12.706 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:12.706 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:12.706 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:31:12.706 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940137+0000", 2024-09-14T06:31:12.706 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.0020000000000001e-06 2024-09-14T06:31:12.706 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:12.706 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:12.707 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:31:12.707 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940143+0000", 2024-09-14T06:31:12.707 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.9850000000000003e-06 2024-09-14T06:31:12.707 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:12.707 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:12.707 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:31:12.707 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940148+0000", 2024-09-14T06:31:12.707 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.9880000000000004e-06 2024-09-14T06:31:12.707 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:12.707 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:12.707 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:31:12.707 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.015400+0000", 2024-09-14T06:31:12.707 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.075252186999999998 2024-09-14T06:31:12.707 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:12.708 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:12.708 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:31:12.708 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.015481+0000", 2024-09-14T06:31:12.708 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.0698000000000006e-05 2024-09-14T06:31:12.708 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:12.708 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:12.708 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:31:12.708 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.015586+0000", 2024-09-14T06:31:12.708 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00010483499999999999 2024-09-14T06:31:12.708 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:12.708 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:31:12.708 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:12.708 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:12.709 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:12.709 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:115203 3.1 3:856be886:::rbd_data.112d4d24c558.0000000000000004:29 [read 3981312~16384 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e96)", 2024-09-14T06:31:12.709 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:30:17.940156+0000", 2024-09-14T06:31:12.709 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 54.720846041999998, 2024-09-14T06:31:12.709 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.075620860999999998, 2024-09-14T06:31:12.709 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:31:12.709 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:31:12.709 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:31:12.709 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:31:12.709 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:31:12.709 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:12.709 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 115203 2024-09-14T06:31:12.709 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:12.709 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:31:12.710 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:12.710 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:31:12.710 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940156+0000", 2024-09-14T06:31:12.710 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:12.710 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:12.710 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:12.710 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:31:12.710 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940156+0000", 2024-09-14T06:31:12.710 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:12.710 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:12.710 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:12.710 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:31:12.710 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940156+0000", 2024-09-14T06:31:12.710 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.1500000000000005e-07 2024-09-14T06:31:12.711 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:12.711 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:12.711 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:31:12.711 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940158+0000", 2024-09-14T06:31:12.711 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.967e-06 2024-09-14T06:31:12.711 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:12.711 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:12.711 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:31:12.711 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940165+0000", 2024-09-14T06:31:12.711 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.6939999999999998e-06 2024-09-14T06:31:12.711 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:12.711 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:12.711 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:31:12.711 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940170+0000", 2024-09-14T06:31:12.712 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.7539999999999997e-06 2024-09-14T06:31:12.712 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:12.712 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:12.712 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:31:12.712 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.015633+0000", 2024-09-14T06:31:12.712 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.075463334000000007 2024-09-14T06:31:12.712 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:12.712 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:12.712 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:31:12.712 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.015693+0000", 2024-09-14T06:31:12.712 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.9367000000000003e-05 2024-09-14T06:31:12.712 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:12.712 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:12.712 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:31:12.713 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.015777+0000", 2024-09-14T06:31:12.713 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.4229999999999998e-05 2024-09-14T06:31:12.713 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:12.713 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:31:12.713 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:12.713 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:12.713 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:12.713 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:115204 3.1 3:856be886:::rbd_data.112d4d24c558.0000000000000004:29 [read 4227072~16384 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e96)", 2024-09-14T06:31:12.713 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:30:17.940176+0000", 2024-09-14T06:31:12.713 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 54.720825509000001, 2024-09-14T06:31:12.713 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.075785505000000003, 2024-09-14T06:31:12.713 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:31:12.713 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:31:12.713 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:31:12.713 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:31:12.714 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:31:12.714 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:12.714 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 115204 2024-09-14T06:31:12.714 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:12.714 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:31:12.714 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:12.714 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:31:12.714 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940176+0000", 2024-09-14T06:31:12.714 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:12.714 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:12.714 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:12.714 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:31:12.714 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940176+0000", 2024-09-14T06:31:12.714 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:12.715 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:12.715 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:12.715 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:31:12.715 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940177+0000", 2024-09-14T06:31:12.715 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.7899999999999999e-07 2024-09-14T06:31:12.715 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:12.715 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:12.715 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:31:12.715 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940179+0000", 2024-09-14T06:31:12.715 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.103e-06 2024-09-14T06:31:12.715 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:12.715 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:12.715 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:31:12.715 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940185+0000", 2024-09-14T06:31:12.716 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.7239999999999997e-06 2024-09-14T06:31:12.716 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:12.716 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:12.716 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:31:12.716 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940190+0000", 2024-09-14T06:31:12.716 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.7269999999999998e-06 2024-09-14T06:31:12.716 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:12.716 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:12.716 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:31:12.716 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.015814+0000", 2024-09-14T06:31:12.716 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.075624114000000006 2024-09-14T06:31:12.716 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:12.716 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:12.716 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:31:12.717 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.015862+0000", 2024-09-14T06:31:12.717 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.8770999999999998e-05 2024-09-14T06:31:12.717 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:12.717 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:12.717 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:31:12.717 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.015962+0000", 2024-09-14T06:31:12.717 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.9586999999999994e-05 2024-09-14T06:31:12.717 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:12.717 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:31:12.717 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:12.717 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:12.717 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:12.717 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:115205 3.1 3:856be886:::rbd_data.112d4d24c558.0000000000000004:29 [read 4472832~16384 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e96)", 2024-09-14T06:31:12.717 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:30:17.940197+0000", 2024-09-14T06:31:12.718 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 54.720804800000003, 2024-09-14T06:31:12.718 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.076068920999999998, 2024-09-14T06:31:12.718 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:31:12.718 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:31:12.718 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:31:12.718 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:31:12.718 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:31:12.718 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:12.718 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 115205 2024-09-14T06:31:12.718 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:12.718 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:31:12.718 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:12.718 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:31:12.718 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940197+0000", 2024-09-14T06:31:12.718 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:12.719 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:12.719 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:12.719 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:31:12.719 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940197+0000", 2024-09-14T06:31:12.719 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:12.719 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:12.719 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:12.719 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:31:12.719 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940198+0000", 2024-09-14T06:31:12.719 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.5999999999999999e-07 2024-09-14T06:31:12.719 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:12.719 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:12.719 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:31:12.719 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940200+0000", 2024-09-14T06:31:12.720 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.0899999999999999e-06 2024-09-14T06:31:12.720 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:12.720 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:12.720 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:31:12.720 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940206+0000", 2024-09-14T06:31:12.720 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.8710000000000002e-06 2024-09-14T06:31:12.720 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:12.720 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:12.720 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:31:12.720 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940211+0000", 2024-09-14T06:31:12.720 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.9230000000000001e-06 2024-09-14T06:31:12.720 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:12.720 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:12.720 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:31:12.721 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.016059+0000", 2024-09-14T06:31:12.721 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.075848051999999999 2024-09-14T06:31:12.721 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:12.721 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:12.721 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:31:12.721 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.016151+0000", 2024-09-14T06:31:12.721 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.1982000000000001e-05 2024-09-14T06:31:12.721 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:12.721 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:12.721 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:31:12.721 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.016266+0000", 2024-09-14T06:31:12.721 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000115543 2024-09-14T06:31:12.721 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:12.722 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:31:12.722 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:12.722 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:12.722 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:12.722 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:115206 3.1 3:856be886:::rbd_data.112d4d24c558.0000000000000004:29 [read 4718592~16384 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e96)", 2024-09-14T06:31:12.722 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:30:17.940223+0000", 2024-09-14T06:31:12.722 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 54.720778709999998, 2024-09-14T06:31:12.722 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.076265115999999994, 2024-09-14T06:31:12.722 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:31:12.722 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:31:12.722 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:31:12.722 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:31:12.722 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:31:12.722 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:12.723 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 115206 2024-09-14T06:31:12.723 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:12.723 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:31:12.723 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:12.723 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:31:12.723 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940223+0000", 2024-09-14T06:31:12.723 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:12.723 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:12.723 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:12.723 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:31:12.723 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940223+0000", 2024-09-14T06:31:12.723 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:12.723 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:12.723 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:12.724 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:31:12.724 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940224+0000", 2024-09-14T06:31:12.724 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.0399999999999996e-07 2024-09-14T06:31:12.724 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:12.724 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:12.724 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:31:12.724 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940226+0000", 2024-09-14T06:31:12.724 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.1780000000000002e-06 2024-09-14T06:31:12.724 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:12.724 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:12.724 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:31:12.724 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940232+0000", 2024-09-14T06:31:12.724 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.5010000000000001e-06 2024-09-14T06:31:12.724 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:12.725 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:12.725 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:31:12.725 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940237+0000", 2024-09-14T06:31:12.725 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.0250000000000002e-06 2024-09-14T06:31:12.725 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:12.725 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:12.725 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:31:12.725 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.016305+0000", 2024-09-14T06:31:12.725 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.076067262999999996 2024-09-14T06:31:12.725 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:12.725 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:12.725 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:31:12.725 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.016383+0000", 2024-09-14T06:31:12.726 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.8578000000000006e-05 2024-09-14T06:31:12.726 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:12.726 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:12.726 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:31:12.726 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.016488+0000", 2024-09-14T06:31:12.726 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000105067 2024-09-14T06:31:12.726 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:12.726 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:31:12.726 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:12.726 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:12.726 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:12.726 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:115207 3.1 3:856be886:::rbd_data.112d4d24c558.0000000000000004:29 [read 4964352~16384 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e96)", 2024-09-14T06:31:12.726 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:30:17.940244+0000", 2024-09-14T06:31:12.727 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 54.720757593999998, 2024-09-14T06:31:12.727 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.076371154999999996, 2024-09-14T06:31:12.727 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:31:12.727 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:31:12.727 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:31:12.727 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:31:12.727 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:31:12.727 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:12.727 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 115207 2024-09-14T06:31:12.727 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:12.727 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:31:12.727 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:12.727 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:31:12.727 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940244+0000", 2024-09-14T06:31:12.727 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:12.728 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:12.728 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:12.728 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:31:12.728 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940244+0000", 2024-09-14T06:31:12.728 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:12.728 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:12.728 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:12.728 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:31:12.728 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940245+0000", 2024-09-14T06:31:12.728 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.1200000000000003e-07 2024-09-14T06:31:12.728 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:12.728 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:12.728 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:31:12.729 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940247+0000", 2024-09-14T06:31:12.729 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.0030000000000001e-06 2024-09-14T06:31:12.729 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:12.729 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:12.729 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:31:12.729 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940252+0000", 2024-09-14T06:31:12.729 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.5520000000000004e-06 2024-09-14T06:31:12.729 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:12.729 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:12.729 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:31:12.729 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940257+0000", 2024-09-14T06:31:12.729 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.6759999999999998e-06 2024-09-14T06:31:12.729 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:12.730 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:12.730 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:31:12.730 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.016521+0000", 2024-09-14T06:31:12.730 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.076263499999999998 2024-09-14T06:31:12.730 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:12.730 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:12.730 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:31:12.730 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.016562+0000", 2024-09-14T06:31:12.730 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.0967000000000003e-05 2024-09-14T06:31:12.730 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:12.730 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:12.730 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:31:12.730 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.016616+0000", 2024-09-14T06:31:12.730 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.3944999999999999e-05 2024-09-14T06:31:12.731 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:12.731 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:31:12.731 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:12.731 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:12.731 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:12.731 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:115208 3.1 3:856be886:::rbd_data.112d4d24c558.0000000000000004:29 [read 5210112~16384 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e96)", 2024-09-14T06:31:12.731 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:30:17.940264+0000", 2024-09-14T06:31:12.731 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 54.720737737, 2024-09-14T06:31:12.731 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.076558288000000002, 2024-09-14T06:31:12.731 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:31:12.731 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:31:12.731 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:31:12.731 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:31:12.731 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:31:12.732 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:12.732 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 115208 2024-09-14T06:31:12.732 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:12.732 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:31:12.732 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:12.732 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:31:12.732 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940264+0000", 2024-09-14T06:31:12.732 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:12.732 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:12.732 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:12.732 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:31:12.732 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940264+0000", 2024-09-14T06:31:12.732 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:12.733 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:12.733 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:12.733 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:31:12.733 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940265+0000", 2024-09-14T06:31:12.733 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.0500000000000004e-07 2024-09-14T06:31:12.733 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:12.733 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:12.733 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:31:12.733 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940267+0000", 2024-09-14T06:31:12.733 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.9810000000000002e-06 2024-09-14T06:31:12.733 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:12.733 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:12.733 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:31:12.733 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940272+0000", 2024-09-14T06:31:12.733 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.6999999999999996e-06 2024-09-14T06:31:12.734 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:12.734 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:12.734 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:31:12.734 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940277+0000", 2024-09-14T06:31:12.734 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.6480000000000002e-06 2024-09-14T06:31:12.734 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:12.734 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:12.734 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:31:12.734 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.016650+0000", 2024-09-14T06:31:12.734 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.076373089000000005 2024-09-14T06:31:12.734 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:12.734 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:12.734 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:31:12.734 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.016724+0000", 2024-09-14T06:31:12.735 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.4105999999999998e-05 2024-09-14T06:31:12.735 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:12.735 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:12.735 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:31:12.735 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.016823+0000", 2024-09-14T06:31:12.735 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.8258999999999996e-05 2024-09-14T06:31:12.735 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:12.735 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:31:12.735 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:12.735 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:12.735 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:12.735 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:115209 3.1 3:856be886:::rbd_data.112d4d24c558.0000000000000004:29 [read 5455872~16384 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e96)", 2024-09-14T06:31:12.735 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:30:17.940283+0000", 2024-09-14T06:31:12.735 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 54.720718711000004, 2024-09-14T06:31:12.736 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.076724029999999999, 2024-09-14T06:31:12.736 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:31:12.736 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:31:12.736 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:31:12.736 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:31:12.736 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:31:12.736 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:12.736 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 115209 2024-09-14T06:31:12.736 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:12.736 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:31:12.736 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:12.736 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:31:12.736 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940283+0000", 2024-09-14T06:31:12.736 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:12.737 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:12.737 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:12.737 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:31:12.737 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940283+0000", 2024-09-14T06:31:12.737 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:12.737 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:12.737 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:12.737 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:31:12.737 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940284+0000", 2024-09-14T06:31:12.737 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.6700000000000003e-07 2024-09-14T06:31:12.737 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:12.737 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:12.737 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:31:12.738 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940286+0000", 2024-09-14T06:31:12.738 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.9520000000000001e-06 2024-09-14T06:31:12.738 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:12.738 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:12.738 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:31:12.738 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940292+0000", 2024-09-14T06:31:12.738 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.7339999999999996e-06 2024-09-14T06:31:12.738 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:12.738 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:12.738 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:31:12.738 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940296+0000", 2024-09-14T06:31:12.738 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.7700000000000001e-06 2024-09-14T06:31:12.738 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:12.738 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:12.739 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:31:12.739 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.016859+0000", 2024-09-14T06:31:12.739 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.07656317 2024-09-14T06:31:12.739 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:12.739 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:12.739 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:31:12.739 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.016929+0000", 2024-09-14T06:31:12.739 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.9679999999999997e-05 2024-09-14T06:31:12.739 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:12.739 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:12.739 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:31:12.739 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.017007+0000", 2024-09-14T06:31:12.739 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.8157000000000004e-05 2024-09-14T06:31:12.739 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:12.740 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:31:12.740 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:12.740 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:12.740 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:12.740 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:115210 3.1 3:856be886:::rbd_data.112d4d24c558.0000000000000004:29 [read 5701632~16384 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e96)", 2024-09-14T06:31:12.740 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:30:17.940303+0000", 2024-09-14T06:31:12.740 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 54.720699193999998, 2024-09-14T06:31:12.740 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.076821358000000006, 2024-09-14T06:31:12.740 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:31:12.740 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:31:12.740 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:31:12.740 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:31:12.740 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:31:12.741 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:12.741 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 115210 2024-09-14T06:31:12.741 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:12.741 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:31:12.741 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:12.741 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:31:12.741 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940303+0000", 2024-09-14T06:31:12.741 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:12.741 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:12.741 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:12.741 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:31:12.741 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940303+0000", 2024-09-14T06:31:12.741 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:12.741 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:12.742 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:12.742 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:31:12.742 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940303+0000", 2024-09-14T06:31:12.742 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.3000000000000001e-07 2024-09-14T06:31:12.742 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:12.742 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:12.742 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:31:12.742 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940306+0000", 2024-09-14T06:31:12.742 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.255e-06 2024-09-14T06:31:12.742 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:12.742 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:12.742 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:31:12.742 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940312+0000", 2024-09-14T06:31:12.742 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.9710000000000001e-06 2024-09-14T06:31:12.743 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:12.743 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:12.743 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:31:12.743 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940316+0000", 2024-09-14T06:31:12.743 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.5340000000000001e-06 2024-09-14T06:31:12.743 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:12.743 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:12.743 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:31:12.743 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.017033+0000", 2024-09-14T06:31:12.743 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.076717169000000002 2024-09-14T06:31:12.743 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:12.743 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:12.743 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:31:12.743 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.017073+0000", 2024-09-14T06:31:12.744 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.9582000000000002e-05 2024-09-14T06:31:12.744 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:12.744 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:12.744 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:31:12.744 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.017124+0000", 2024-09-14T06:31:12.744 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.1316999999999997e-05 2024-09-14T06:31:12.744 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:12.744 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:31:12.744 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:12.744 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:12.744 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:12.744 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:115219 3.1 3:856be886:::rbd_data.112d4d24c558.0000000000000004:29 [read 65536~16384 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e96)", 2024-09-14T06:31:12.744 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:30:17.940323+0000", 2024-09-14T06:31:12.745 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 54.720679333, 2024-09-14T06:31:12.745 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.076915205, 2024-09-14T06:31:12.745 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:31:12.745 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:31:12.745 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:31:12.745 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:31:12.745 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:31:12.745 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:12.745 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 115219 2024-09-14T06:31:12.745 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:12.745 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:31:12.745 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:12.745 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:31:12.745 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940323+0000", 2024-09-14T06:31:12.746 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:12.746 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:12.746 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:12.746 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:31:12.746 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940323+0000", 2024-09-14T06:31:12.746 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:12.746 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:12.746 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:12.746 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:31:12.746 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940323+0000", 2024-09-14T06:31:12.746 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.0800000000000005e-07 2024-09-14T06:31:12.746 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:12.746 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:12.747 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:31:12.747 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940328+0000", 2024-09-14T06:31:12.747 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.887e-06 2024-09-14T06:31:12.747 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:12.747 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:12.747 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:31:12.747 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940334+0000", 2024-09-14T06:31:12.747 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.3269999999999997e-06 2024-09-14T06:31:12.747 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:12.747 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:12.747 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:31:12.747 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940339+0000", 2024-09-14T06:31:12.747 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.7960000000000003e-06 2024-09-14T06:31:12.748 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:12.748 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:12.748 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:31:12.748 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.017148+0000", 2024-09-14T06:31:12.748 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.076809125000000006 2024-09-14T06:31:12.748 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:12.748 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:12.748 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:31:12.748 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.017186+0000", 2024-09-14T06:31:12.748 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.7283000000000002e-05 2024-09-14T06:31:12.748 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:12.748 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:12.748 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:31:12.748 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.017238+0000", 2024-09-14T06:31:12.749 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.2278999999999999e-05 2024-09-14T06:31:12.749 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:12.749 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:31:12.749 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:12.749 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:12.749 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:12.749 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:115220 3.1 3:856be886:::rbd_data.112d4d24c558.0000000000000004:29 [read 311296~16384 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e96)", 2024-09-14T06:31:12.749 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:30:17.940346+0000", 2024-09-14T06:31:12.749 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 54.720656204000001, 2024-09-14T06:31:12.749 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.077033293000000003, 2024-09-14T06:31:12.749 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:31:12.749 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:31:12.749 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:31:12.749 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:31:12.750 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:31:12.750 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:12.750 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 115220 2024-09-14T06:31:12.750 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:12.750 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:31:12.750 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:12.750 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:31:12.750 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940346+0000", 2024-09-14T06:31:12.750 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:12.750 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:12.750 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:12.750 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:31:12.750 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940346+0000", 2024-09-14T06:31:12.751 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:12.751 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:12.751 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:12.751 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:31:12.751 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940346+0000", 2024-09-14T06:31:12.751 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.4300000000000003e-07 2024-09-14T06:31:12.751 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:12.751 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:12.751 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:31:12.751 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940348+0000", 2024-09-14T06:31:12.751 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.0030000000000001e-06 2024-09-14T06:31:12.751 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:12.751 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:12.751 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:31:12.752 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.004655+0000", 2024-09-14T06:31:12.752 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.064306323999999998 2024-09-14T06:31:12.752 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:12.752 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:12.752 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:31:12.752 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.004671+0000", 2024-09-14T06:31:12.752 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.6589999999999999e-05 2024-09-14T06:31:12.752 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:12.752 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:12.752 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:31:12.752 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.017262+0000", 2024-09-14T06:31:12.752 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.012590570000000001 2024-09-14T06:31:12.752 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:12.752 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:12.752 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:31:12.753 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.017321+0000", 2024-09-14T06:31:12.753 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.9210999999999999e-05 2024-09-14T06:31:12.753 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:12.753 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:12.753 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:31:12.753 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.017379+0000", 2024-09-14T06:31:12.753 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.8051999999999999e-05 2024-09-14T06:31:12.753 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:12.753 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:31:12.753 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:12.753 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:12.753 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:31:12.753 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-14T06:31:12.755 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-14T06:31:12.755 INFO:tasks.ceph.osd.3.smithi094.stderr:2024-09-14T06:31:12.753+0000 7feeeafe8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33655) UID: 0 2024-09-14T06:31:12.822 INFO:teuthology.orchestra.run.smithi094.stderr:noscrub is set 2024-09-14T06:31:12.826 INFO:teuthology.orchestra.run.smithi094.stderr:set osd.3 primary-affinity to 1 (8655362) 2024-09-14T06:31:12.882 INFO:tasks.ceph.osd.2.smithi094.stderr:2024-09-14T06:31:12.881+0000 7faf3269b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33649) UID: 0 2024-09-14T06:31:12.902 INFO:teuthology.orchestra.run.smithi094.stdout:checking clone #32, image image_client.0-clone32 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent33 2024-09-14T06:31:12.907 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-14T06:31:12.908 INFO:teuthology.orchestra.run.smithi094.stdout: "ops": [], 2024-09-14T06:31:12.908 INFO:teuthology.orchestra.run.smithi094.stdout: "num_ops": 0 2024-09-14T06:31:12.908 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-14T06:31:12.919 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-14T06:31:12.960 INFO:tasks.ceph.osd.3.smithi094.stderr:2024-09-14T06:31:12.959+0000 7feeeafe8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33655) UID: 0 2024-09-14T06:31:13.058 INFO:tasks.ceph.osd.3.smithi094.stderr:2024-09-14T06:31:13.057+0000 7feeeafe8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33655) UID: 0 2024-09-14T06:31:13.067 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-14T06:31:13.068 INFO:teuthology.orchestra.run.smithi094.stdout: "ops": [], 2024-09-14T06:31:13.068 INFO:teuthology.orchestra.run.smithi094.stdout: "complaint_time": 30, 2024-09-14T06:31:13.068 INFO:teuthology.orchestra.run.smithi094.stdout: "num_blocked_ops": 0 2024-09-14T06:31:13.068 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-14T06:31:13.077 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-14T06:31:13.093 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-14T06:31:13.110 INFO:teuthology.orchestra.run.smithi094.stdout:4355 write 0x9c9b434 thru 0x9ca05c2 (0x518f bytes) 2024-09-14T06:31:13.129 INFO:teuthology.orchestra.run.smithi094.stdout:4358 writesame 0xa95ec81 thru 0xa965f38 data_size 0x4(0x72b8 bytes) 2024-09-14T06:31:13.136 INFO:teuthology.orchestra.run.smithi094.stdout:4359 trunc from 0xa965f39 to 0x489dc55 2024-09-14T06:31:13.159 INFO:tasks.ceph.osd.2.smithi094.stderr:2024-09-14T06:31:13.158+0000 7faf3269b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33649) UID: 0 2024-09-14T06:31:13.162 INFO:teuthology.orchestra.run.smithi094.stdout:4360 writesame 0x7c5a6aa thru 0x7c6a2e6 data_size 0x1(0xfc3d bytes) 2024-09-14T06:31:13.162 INFO:teuthology.orchestra.run.smithi094.stdout:4362 read 0x1139826 thru 0x113d7d5 (0x3fb0 bytes) 2024-09-14T06:31:13.171 INFO:teuthology.orchestra.run.smithi094.stdout:4364 write 0xa4b8d7e thru 0xa4c1c51 (0x8ed4 bytes) 2024-09-14T06:31:13.187 INFO:teuthology.orchestra.run.smithi094.stdout:4365 writesame 0xb2eb3c7 thru 0xb2f456e data_size 0x4f(0x91a8 bytes) 2024-09-14T06:31:13.200 INFO:teuthology.orchestra.run.smithi094.stdout:4367 trunc from 0xb2f456f to 0x45d507 2024-09-14T06:31:13.212 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-14T06:31:13.212 INFO:teuthology.orchestra.run.smithi094.stdout: "size": 20, 2024-09-14T06:31:13.213 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 600, 2024-09-14T06:31:13.213 INFO:teuthology.orchestra.run.smithi094.stdout: "ops": [ 2024-09-14T06:31:13.213 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:13.213 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:14918 3.0 3:05c39535:::rbd_data.112d92027590.0000000000000009:f [read 795648~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:31:13.213 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:26.083834+0000", 2024-09-14T06:31:13.213 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 107.12652195299999, 2024-09-14T06:31:13.213 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.041776287000000002, 2024-09-14T06:31:13.213 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:31:13.213 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:31:13.213 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:31:13.213 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:31:13.214 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:31:13.214 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:13.214 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 14918 2024-09-14T06:31:13.214 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:13.214 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:31:13.214 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:13.214 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:31:13.214 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.083834+0000", 2024-09-14T06:31:13.214 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:13.214 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:13.214 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:13.214 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:31:13.214 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.083834+0000", 2024-09-14T06:31:13.215 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:13.215 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:13.215 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:13.215 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:31:13.215 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.083834+0000", 2024-09-14T06:31:13.215 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.0100000000000001e-07 2024-09-14T06:31:13.215 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:13.215 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:13.215 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:31:13.215 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.083835+0000", 2024-09-14T06:31:13.215 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.3900000000000003e-07 2024-09-14T06:31:13.215 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:13.215 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:13.215 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:31:13.216 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.083838+0000", 2024-09-14T06:31:13.216 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.9830000000000001e-06 2024-09-14T06:31:13.216 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:13.216 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:13.216 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:31:13.216 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.083840+0000", 2024-09-14T06:31:13.216 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.4679999999999999e-06 2024-09-14T06:31:13.216 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:13.216 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:13.216 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:31:13.216 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.125455+0000", 2024-09-14T06:31:13.216 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.041614578999999999 2024-09-14T06:31:13.216 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:13.216 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:13.217 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:31:13.217 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.125496+0000", 2024-09-14T06:31:13.217 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.1047000000000002e-05 2024-09-14T06:31:13.217 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:13.217 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:13.217 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:31:13.217 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.125610+0000", 2024-09-14T06:31:13.217 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00011407 2024-09-14T06:31:13.217 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:13.217 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:31:13.217 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:13.217 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:13.217 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:13.218 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:14919 3.0 3:05c39535:::rbd_data.112d92027590.0000000000000009:f [read 805888~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:31:13.218 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:26.083843+0000", 2024-09-14T06:31:13.218 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 107.126512205, 2024-09-14T06:31:13.218 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.041992356000000002, 2024-09-14T06:31:13.218 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:31:13.218 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:31:13.218 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:31:13.218 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:31:13.218 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:31:13.218 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:13.218 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 14919 2024-09-14T06:31:13.218 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:13.218 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:31:13.219 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:13.219 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:31:13.219 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.083843+0000", 2024-09-14T06:31:13.219 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:13.219 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:13.219 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:13.219 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:31:13.219 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.083843+0000", 2024-09-14T06:31:13.219 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:13.219 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:13.219 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:13.219 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:31:13.219 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.083844+0000", 2024-09-14T06:31:13.219 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.3300000000000001e-07 2024-09-14T06:31:13.220 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:13.220 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:13.220 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:31:13.220 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.083845+0000", 2024-09-14T06:31:13.220 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.4729999999999999e-06 2024-09-14T06:31:13.220 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:13.220 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:13.220 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:31:13.220 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.083850+0000", 2024-09-14T06:31:13.220 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.0130000000000001e-06 2024-09-14T06:31:13.220 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:13.220 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:13.220 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:31:13.221 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.083854+0000", 2024-09-14T06:31:13.221 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.4759999999999998e-06 2024-09-14T06:31:13.221 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:13.221 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:13.221 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:31:13.221 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.125636+0000", 2024-09-14T06:31:13.221 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.041782683000000001 2024-09-14T06:31:13.221 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:13.221 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:13.221 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:31:13.221 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.125680+0000", 2024-09-14T06:31:13.221 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.3614e-05 2024-09-14T06:31:13.221 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:13.221 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:13.221 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:31:13.222 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.125836+0000", 2024-09-14T06:31:13.222 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000155864 2024-09-14T06:31:13.222 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:13.222 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:31:13.222 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:13.222 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:13.222 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:13.222 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:14920 3.0 3:05c39535:::rbd_data.112d92027590.0000000000000009:f [read 816128~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:31:13.222 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:26.083868+0000", 2024-09-14T06:31:13.222 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 107.126487544, 2024-09-14T06:31:13.222 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.042166771999999998, 2024-09-14T06:31:13.222 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:31:13.222 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:31:13.222 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:31:13.223 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:31:13.223 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:31:13.223 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:13.223 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 14920 2024-09-14T06:31:13.223 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:13.223 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:31:13.223 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:13.223 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:31:13.223 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.083868+0000", 2024-09-14T06:31:13.223 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:13.223 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:13.223 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:13.223 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:31:13.223 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.083868+0000", 2024-09-14T06:31:13.224 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:13.224 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:13.224 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:13.224 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:31:13.224 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.083869+0000", 2024-09-14T06:31:13.224 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.99e-07 2024-09-14T06:31:13.224 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:13.224 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:13.224 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:31:13.224 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.083870+0000", 2024-09-14T06:31:13.224 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.782e-06 2024-09-14T06:31:13.224 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:13.224 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:13.224 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:31:13.225 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.083876+0000", 2024-09-14T06:31:13.225 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.3809999999999998e-06 2024-09-14T06:31:13.225 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:13.225 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:13.225 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:31:13.225 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.083882+0000", 2024-09-14T06:31:13.225 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.9479999999999996e-06 2024-09-14T06:31:13.225 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:13.225 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:13.225 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:31:13.225 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.125878+0000", 2024-09-14T06:31:13.225 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.041996264999999998 2024-09-14T06:31:13.225 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:13.225 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:13.226 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:31:13.226 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.125929+0000", 2024-09-14T06:31:13.226 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.0498999999999999e-05 2024-09-14T06:31:13.226 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:13.226 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:13.226 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:31:13.226 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.126035+0000", 2024-09-14T06:31:13.226 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000106298 2024-09-14T06:31:13.226 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:13.226 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:31:13.226 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:13.226 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:13.226 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:13.226 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:14921 3.0 3:05c39535:::rbd_data.112d92027590.0000000000000009:f [read 826368~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:31:13.227 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:26.083889+0000", 2024-09-14T06:31:13.227 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 107.126466534, 2024-09-14T06:31:13.227 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.042435309999999997, 2024-09-14T06:31:13.227 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:31:13.227 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:31:13.227 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:31:13.227 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:31:13.227 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:31:13.227 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:13.227 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 14921 2024-09-14T06:31:13.227 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:13.227 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:31:13.227 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:13.227 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:31:13.228 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.083889+0000", 2024-09-14T06:31:13.228 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:13.228 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:13.228 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:13.228 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:31:13.228 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.083889+0000", 2024-09-14T06:31:13.228 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:13.228 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:13.228 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:13.228 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:31:13.228 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.086033+0000", 2024-09-14T06:31:13.228 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0021444179999999999 2024-09-14T06:31:13.228 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:13.228 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:13.229 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:31:13.229 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.086038+0000", 2024-09-14T06:31:13.229 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.0969999999999999e-06 2024-09-14T06:31:13.229 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:13.229 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:13.229 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:31:13.229 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.086046+0000", 2024-09-14T06:31:13.229 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.9440000000000005e-06 2024-09-14T06:31:13.229 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:13.229 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:13.229 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:31:13.229 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.086050+0000", 2024-09-14T06:31:13.229 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.69e-06 2024-09-14T06:31:13.229 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:13.230 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:13.230 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:31:13.230 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.126080+0000", 2024-09-14T06:31:13.230 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.040030237000000003 2024-09-14T06:31:13.230 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:13.230 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:13.230 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:31:13.230 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.126163+0000", 2024-09-14T06:31:13.230 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.2071999999999996e-05 2024-09-14T06:31:13.230 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:13.230 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:13.230 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:31:13.230 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.126324+0000", 2024-09-14T06:31:13.230 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00016185200000000001 2024-09-14T06:31:13.231 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:13.231 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:31:13.231 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:13.231 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:13.231 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:13.231 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:55627 3.0 3:06490baf:::rbd_data.112dca2f6ae1.0000000000000030:17 [read 2002944~4096 [fadvise_nocache]] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:31:13.231 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:44.599966+0000", 2024-09-14T06:31:13.231 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 88.610389585999997, 2024-09-14T06:31:13.231 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.041868546999999999, 2024-09-14T06:31:13.231 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:31:13.231 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:31:13.231 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:31:13.231 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:31:13.231 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:31:13.231 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:13.232 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 55627 2024-09-14T06:31:13.232 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:13.232 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:31:13.232 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:13.232 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:31:13.232 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.599966+0000", 2024-09-14T06:31:13.232 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:13.232 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:13.232 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:13.232 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:31:13.232 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.599966+0000", 2024-09-14T06:31:13.232 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:13.232 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:13.232 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:13.233 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:31:13.233 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.599966+0000", 2024-09-14T06:31:13.233 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.0900000000000001e-07 2024-09-14T06:31:13.233 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:13.233 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:13.233 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:31:13.233 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.599967+0000", 2024-09-14T06:31:13.233 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.64e-07 2024-09-14T06:31:13.233 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:13.233 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:13.233 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:31:13.233 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.599970+0000", 2024-09-14T06:31:13.233 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.9670000000000001e-06 2024-09-14T06:31:13.233 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:13.234 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:13.234 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:31:13.234 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.599973+0000", 2024-09-14T06:31:13.234 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.632e-06 2024-09-14T06:31:13.234 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:13.234 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:13.234 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:31:13.234 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.641665+0000", 2024-09-14T06:31:13.234 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.041692547000000003 2024-09-14T06:31:13.234 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:13.234 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:13.234 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:31:13.234 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.641740+0000", 2024-09-14T06:31:13.234 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.4559000000000002e-05 2024-09-14T06:31:13.235 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:13.235 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:13.235 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:31:13.235 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.641835+0000", 2024-09-14T06:31:13.235 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.4668999999999998e-05 2024-09-14T06:31:13.235 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:13.235 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:31:13.235 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:13.235 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:13.235 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:13.235 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:55628 3.0 3:06490baf:::rbd_data.112dca2f6ae1.0000000000000030:17 [read 2027520~4096 [fadvise_nocache]] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:31:13.235 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:44.599976+0000", 2024-09-14T06:31:13.235 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 88.610379799, 2024-09-14T06:31:13.235 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.041991621, 2024-09-14T06:31:13.236 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:31:13.236 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:31:13.236 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:31:13.236 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:31:13.236 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:31:13.236 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:13.236 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 55628 2024-09-14T06:31:13.236 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:13.236 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:31:13.236 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:13.236 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:31:13.236 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.599976+0000", 2024-09-14T06:31:13.236 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:13.236 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:13.237 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:13.237 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:31:13.237 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.599976+0000", 2024-09-14T06:31:13.237 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:13.237 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:13.237 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:13.237 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:31:13.237 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.599976+0000", 2024-09-14T06:31:13.237 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.1299999999999999e-07 2024-09-14T06:31:13.237 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:13.237 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:13.237 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:31:13.237 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.599977+0000", 2024-09-14T06:31:13.237 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.7999999999999993e-07 2024-09-14T06:31:13.238 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:13.238 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:13.238 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:31:13.238 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.599980+0000", 2024-09-14T06:31:13.238 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.01e-06 2024-09-14T06:31:13.238 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:13.238 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:13.238 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:31:13.238 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.599983+0000", 2024-09-14T06:31:13.238 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.5409999999999999e-06 2024-09-14T06:31:13.238 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:13.238 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:13.238 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:31:13.238 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.641869+0000", 2024-09-14T06:31:13.239 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.041886235000000001 2024-09-14T06:31:13.239 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:13.239 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:13.239 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:31:13.239 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.641913+0000", 2024-09-14T06:31:13.239 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.4357000000000002e-05 2024-09-14T06:31:13.239 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:13.239 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:13.239 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:31:13.239 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.641967+0000", 2024-09-14T06:31:13.239 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.4284999999999999e-05 2024-09-14T06:31:13.239 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:13.239 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:31:13.239 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:13.240 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:13.240 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:13.240 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:55629 3.0 3:06490baf:::rbd_data.112dca2f6ae1.0000000000000030:17 [read 2052096~4096 [fadvise_nocache]] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:31:13.240 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:44.599985+0000", 2024-09-14T06:31:13.240 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 88.610370093, 2024-09-14T06:31:13.240 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.042183375000000002, 2024-09-14T06:31:13.240 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:31:13.240 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:31:13.240 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:31:13.240 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:31:13.240 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:31:13.240 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:13.240 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 55629 2024-09-14T06:31:13.241 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:13.241 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:31:13.241 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:13.241 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:31:13.241 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.599985+0000", 2024-09-14T06:31:13.241 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:13.241 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:13.241 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:13.241 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:31:13.241 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.599985+0000", 2024-09-14T06:31:13.241 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:13.241 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:13.241 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:13.241 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:31:13.242 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.599986+0000", 2024-09-14T06:31:13.242 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.05e-07 2024-09-14T06:31:13.242 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:13.242 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:13.242 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:31:13.242 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.599987+0000", 2024-09-14T06:31:13.242 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.8400000000000002e-07 2024-09-14T06:31:13.242 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:13.242 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:13.242 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:31:13.242 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.599990+0000", 2024-09-14T06:31:13.242 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.5640000000000001e-06 2024-09-14T06:31:13.242 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:13.243 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:13.243 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:31:13.243 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.599993+0000", 2024-09-14T06:31:13.243 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.615e-06 2024-09-14T06:31:13.243 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:13.243 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:13.243 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:31:13.243 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.642003+0000", 2024-09-14T06:31:13.243 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.042010345999999997 2024-09-14T06:31:13.243 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:13.243 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:13.243 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:31:13.243 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.642080+0000", 2024-09-14T06:31:13.243 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.6538999999999993e-05 2024-09-14T06:31:13.244 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:13.244 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:13.244 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:31:13.244 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.642169+0000", 2024-09-14T06:31:13.244 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.9122000000000005e-05 2024-09-14T06:31:13.244 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:13.244 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:31:13.244 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:13.244 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:13.244 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:13.244 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:55630 3.0 3:06490baf:::rbd_data.112dca2f6ae1.0000000000000030:17 [read 2076672~4096 [fadvise_nocache]] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:31:13.244 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:44.599996+0000", 2024-09-14T06:31:13.244 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 88.610359614999993, 2024-09-14T06:31:13.244 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.042299615999999998, 2024-09-14T06:31:13.244 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:31:13.245 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:31:13.245 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:31:13.245 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:31:13.245 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:31:13.245 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:13.245 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 55630 2024-09-14T06:31:13.245 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:13.245 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:31:13.245 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:13.245 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:31:13.245 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.599996+0000", 2024-09-14T06:31:13.245 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:13.245 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:13.246 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:13.246 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:31:13.246 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.599996+0000", 2024-09-14T06:31:13.246 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:13.246 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:13.246 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:13.246 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:31:13.246 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.599996+0000", 2024-09-14T06:31:13.246 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.1500000000000001e-07 2024-09-14T06:31:13.246 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:13.246 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:13.246 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:31:13.246 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.599997+0000", 2024-09-14T06:31:13.246 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.33e-07 2024-09-14T06:31:13.248 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:13.248 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:13.248 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:31:13.248 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600000+0000", 2024-09-14T06:31:13.248 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.0240000000000002e-06 2024-09-14T06:31:13.248 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:13.248 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:13.248 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:31:13.248 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600003+0000", 2024-09-14T06:31:13.249 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.3640000000000002e-06 2024-09-14T06:31:13.249 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:13.249 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:13.249 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:31:13.249 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.642200+0000", 2024-09-14T06:31:13.249 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.042197571000000003 2024-09-14T06:31:13.249 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:13.249 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:13.249 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:31:13.249 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.642243+0000", 2024-09-14T06:31:13.249 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.2765000000000001e-05 2024-09-14T06:31:13.249 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:13.249 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:13.250 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:31:13.250 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.642296+0000", 2024-09-14T06:31:13.250 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.2744000000000002e-05 2024-09-14T06:31:13.250 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:13.250 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:31:13.250 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:13.250 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:13.250 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:13.250 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:55631 3.0 3:06490baf:::rbd_data.112dca2f6ae1.0000000000000030:17 [read 1048576~4096 [fadvise_nocache]] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:31:13.250 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:44.600006+0000", 2024-09-14T06:31:13.250 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 88.610350026999996, 2024-09-14T06:31:13.250 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.042487905999999999, 2024-09-14T06:31:13.250 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:31:13.250 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:31:13.251 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:31:13.251 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:31:13.251 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:31:13.251 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:13.251 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 55631 2024-09-14T06:31:13.251 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:13.251 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:31:13.251 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:13.251 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:31:13.251 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600006+0000", 2024-09-14T06:31:13.251 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:13.251 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:13.251 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:13.251 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:31:13.252 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600006+0000", 2024-09-14T06:31:13.252 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:13.252 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:13.252 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:13.252 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:31:13.252 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600006+0000", 2024-09-14T06:31:13.252 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.0800000000000001e-07 2024-09-14T06:31:13.252 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:13.252 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:13.252 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:31:13.252 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600007+0000", 2024-09-14T06:31:13.252 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.7199999999999997e-07 2024-09-14T06:31:13.252 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:13.252 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:13.253 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:31:13.253 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600010+0000", 2024-09-14T06:31:13.253 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.0280000000000001e-06 2024-09-14T06:31:13.253 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:13.253 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:13.253 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:31:13.253 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600012+0000", 2024-09-14T06:31:13.253 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.475e-06 2024-09-14T06:31:13.253 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:13.253 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:13.253 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:31:13.253 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.642329+0000", 2024-09-14T06:31:13.253 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.042317157000000001 2024-09-14T06:31:13.254 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:13.254 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:13.254 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:31:13.254 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.642403+0000", 2024-09-14T06:31:13.254 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.3941000000000002e-05 2024-09-14T06:31:13.254 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:13.254 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:13.254 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:31:13.254 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.642493+0000", 2024-09-14T06:31:13.254 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.0124999999999995e-05 2024-09-14T06:31:13.254 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:13.254 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:31:13.254 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:13.254 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:13.255 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:13.255 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:55632 3.0 3:06490baf:::rbd_data.112dca2f6ae1.0000000000000030:17 [read 1073152~4096 [fadvise_nocache]] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:31:13.255 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:44.600015+0000", 2024-09-14T06:31:13.255 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 88.610340273999995, 2024-09-14T06:31:13.255 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.042604922000000003, 2024-09-14T06:31:13.255 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:31:13.255 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:31:13.255 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:31:13.255 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:31:13.255 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:31:13.255 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:13.255 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 55632 2024-09-14T06:31:13.255 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:13.255 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:31:13.255 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:13.256 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:31:13.256 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600015+0000", 2024-09-14T06:31:13.256 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:13.256 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:13.256 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:13.256 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:31:13.256 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600015+0000", 2024-09-14T06:31:13.256 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:13.256 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:13.256 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:13.256 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:31:13.256 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600016+0000", 2024-09-14T06:31:13.257 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.1e-07 2024-09-14T06:31:13.257 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:13.257 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:13.257 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:31:13.257 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600017+0000", 2024-09-14T06:31:13.257 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.9900000000000009e-07 2024-09-14T06:31:13.257 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:13.257 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:13.257 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:31:13.257 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600020+0000", 2024-09-14T06:31:13.257 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.4170000000000001e-06 2024-09-14T06:31:13.257 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:13.257 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:13.257 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:31:13.257 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600023+0000", 2024-09-14T06:31:13.258 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.588e-06 2024-09-14T06:31:13.258 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:13.258 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:13.258 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:31:13.258 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.642526+0000", 2024-09-14T06:31:13.258 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.042503236999999999 2024-09-14T06:31:13.258 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:13.258 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:13.258 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:31:13.258 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.642568+0000", 2024-09-14T06:31:13.258 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.1925000000000003e-05 2024-09-14T06:31:13.258 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:13.258 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:13.259 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:31:13.259 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.642620+0000", 2024-09-14T06:31:13.259 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.2546000000000001e-05 2024-09-14T06:31:13.259 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:13.259 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:31:13.259 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:13.259 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:13.259 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:13.259 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:55633 3.0 3:06490baf:::rbd_data.112dca2f6ae1.0000000000000030:17 [read 1097728~4096 [fadvise_nocache]] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:31:13.259 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:44.600026+0000", 2024-09-14T06:31:13.259 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 88.610330004999994, 2024-09-14T06:31:13.259 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.042797474000000002, 2024-09-14T06:31:13.259 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:31:13.259 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:31:13.260 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:31:13.260 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:31:13.260 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:31:13.260 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:13.260 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 55633 2024-09-14T06:31:13.260 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:13.260 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:31:13.260 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:13.260 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:31:13.260 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600026+0000", 2024-09-14T06:31:13.260 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:13.260 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:13.260 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:13.260 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:31:13.261 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600026+0000", 2024-09-14T06:31:13.261 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:13.261 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:13.261 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:13.261 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:31:13.261 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600026+0000", 2024-09-14T06:31:13.261 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.3099999999999999e-07 2024-09-14T06:31:13.261 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:13.261 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:13.261 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:31:13.261 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600029+0000", 2024-09-14T06:31:13.261 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.304e-06 2024-09-14T06:31:13.261 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:13.261 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:13.262 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:31:13.262 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600034+0000", 2024-09-14T06:31:13.262 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.1429999999999997e-06 2024-09-14T06:31:13.262 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:13.262 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:13.262 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:31:13.262 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600038+0000", 2024-09-14T06:31:13.262 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.692e-06 2024-09-14T06:31:13.262 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:13.262 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:13.262 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:31:13.262 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.642654+0000", 2024-09-14T06:31:13.262 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.042615564000000002 2024-09-14T06:31:13.262 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:13.263 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:13.263 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:31:13.263 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.642728+0000", 2024-09-14T06:31:13.263 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.4018999999999997e-05 2024-09-14T06:31:13.263 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:13.263 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:13.263 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:31:13.263 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.642823+0000", 2024-09-14T06:31:13.263 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.5520999999999996e-05 2024-09-14T06:31:13.263 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:13.263 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:31:13.263 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:13.263 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:13.264 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:13.264 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:55634 3.0 3:06490baf:::rbd_data.112dca2f6ae1.0000000000000030:17 [read 1122304~4096 [fadvise_nocache]] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:31:13.264 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:44.600042+0000", 2024-09-14T06:31:13.264 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 88.610313528000006, 2024-09-14T06:31:13.264 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.042903729000000002, 2024-09-14T06:31:13.264 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:31:13.264 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:31:13.264 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:31:13.264 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:31:13.264 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:31:13.264 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:13.264 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 55634 2024-09-14T06:31:13.264 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:13.265 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:31:13.265 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:13.265 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:31:13.265 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600042+0000", 2024-09-14T06:31:13.265 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:13.265 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:13.265 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:13.265 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:31:13.265 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600042+0000", 2024-09-14T06:31:13.265 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:13.265 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:13.265 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:13.265 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:31:13.265 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600043+0000", 2024-09-14T06:31:13.266 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.7000000000000005e-07 2024-09-14T06:31:13.266 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:13.266 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:13.266 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:31:13.266 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600044+0000", 2024-09-14T06:31:13.266 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.742e-06 2024-09-14T06:31:13.266 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:13.266 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:13.266 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:31:13.266 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600050+0000", 2024-09-14T06:31:13.266 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.0800000000000002e-06 2024-09-14T06:31:13.266 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:13.266 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:13.266 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:31:13.266 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600055+0000", 2024-09-14T06:31:13.267 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.3889999999999998e-06 2024-09-14T06:31:13.267 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:13.267 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:13.267 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:31:13.267 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.642845+0000", 2024-09-14T06:31:13.267 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.042790040000000001 2024-09-14T06:31:13.267 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:13.267 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:13.267 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:31:13.267 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.642888+0000", 2024-09-14T06:31:13.267 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.2790000000000002e-05 2024-09-14T06:31:13.267 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:13.267 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:13.267 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:31:13.268 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.642946+0000", 2024-09-14T06:31:13.268 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.8118000000000002e-05 2024-09-14T06:31:13.268 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:13.268 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:31:13.268 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:13.268 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:13.268 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:13.268 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:55635 3.0 3:06490baf:::rbd_data.112dca2f6ae1.0000000000000030:17 [read 1146880~4096 [fadvise_nocache]] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:31:13.268 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:44.600061+0000", 2024-09-14T06:31:13.268 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 88.610294956999994, 2024-09-14T06:31:13.268 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.043092695, 2024-09-14T06:31:13.268 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:31:13.268 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:31:13.268 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:31:13.269 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:31:13.269 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:31:13.269 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:13.269 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 55635 2024-09-14T06:31:13.269 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:13.269 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:31:13.269 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:13.269 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:31:13.269 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600061+0000", 2024-09-14T06:31:13.269 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:13.269 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:13.269 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:13.269 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:31:13.270 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600061+0000", 2024-09-14T06:31:13.270 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:13.270 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:13.270 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:13.270 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:31:13.270 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600061+0000", 2024-09-14T06:31:13.270 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.3099999999999999e-07 2024-09-14T06:31:13.270 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:13.270 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:13.270 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:31:13.270 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600063+0000", 2024-09-14T06:31:13.270 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.6220000000000001e-06 2024-09-14T06:31:13.270 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:13.270 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:13.271 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:31:13.271 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600068+0000", 2024-09-14T06:31:13.271 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.4790000000000004e-06 2024-09-14T06:31:13.271 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:13.271 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:13.271 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:31:13.271 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600072+0000", 2024-09-14T06:31:13.271 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.3100000000000002e-06 2024-09-14T06:31:13.271 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:13.271 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:13.271 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:31:13.271 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.642987+0000", 2024-09-14T06:31:13.271 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.042914489 2024-09-14T06:31:13.271 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:13.272 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:13.272 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:31:13.272 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.643062+0000", 2024-09-14T06:31:13.272 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.5011000000000004e-05 2024-09-14T06:31:13.272 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:13.272 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:13.272 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:31:13.272 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.643153+0000", 2024-09-14T06:31:13.272 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.1452999999999993e-05 2024-09-14T06:31:13.272 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:13.272 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:31:13.272 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:13.272 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:13.273 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:13.273 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:55636 3.0 3:06490baf:::rbd_data.112dca2f6ae1.0000000000000030:17 [read 1171456~4096 [fadvise_nocache]] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:31:13.273 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:44.600078+0000", 2024-09-14T06:31:13.273 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 88.610277545000002, 2024-09-14T06:31:13.273 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.043207966, 2024-09-14T06:31:13.273 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:31:13.273 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:31:13.273 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:31:13.273 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:31:13.273 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:31:13.273 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:13.273 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 55636 2024-09-14T06:31:13.273 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:13.273 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:31:13.274 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:13.274 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:31:13.274 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600078+0000", 2024-09-14T06:31:13.274 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:13.274 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:13.274 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:13.274 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:31:13.274 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600078+0000", 2024-09-14T06:31:13.274 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:13.274 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:13.274 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:13.274 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:31:13.274 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600078+0000", 2024-09-14T06:31:13.274 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.34e-07 2024-09-14T06:31:13.275 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:13.275 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:13.275 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:31:13.275 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600080+0000", 2024-09-14T06:31:13.275 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.641e-06 2024-09-14T06:31:13.275 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:13.275 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:13.275 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:31:13.275 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600086+0000", 2024-09-14T06:31:13.275 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.3860000000000004e-06 2024-09-14T06:31:13.275 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:13.275 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:13.275 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:31:13.275 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600091+0000", 2024-09-14T06:31:13.275 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.5970000000000002e-06 2024-09-14T06:31:13.276 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:13.276 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:13.276 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:31:13.276 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.643184+0000", 2024-09-14T06:31:13.276 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.043092893 2024-09-14T06:31:13.276 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:13.276 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:13.276 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:31:13.276 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.643228+0000", 2024-09-14T06:31:13.276 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.4536999999999997e-05 2024-09-14T06:31:13.276 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:13.276 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:13.276 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:31:13.276 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.643286+0000", 2024-09-14T06:31:13.277 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.7578000000000003e-05 2024-09-14T06:31:13.277 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:13.277 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:31:13.277 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:13.277 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:13.277 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:13.277 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:55637 3.0 3:06490baf:::rbd_data.112dca2f6ae1.0000000000000030:17 [read 1196032~4096 [fadvise_nocache]] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:31:13.277 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:44.600097+0000", 2024-09-14T06:31:13.277 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 88.610258981000001, 2024-09-14T06:31:13.277 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.043399124999999997, 2024-09-14T06:31:13.277 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:31:13.277 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:31:13.277 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:31:13.278 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:31:13.278 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:31:13.278 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:13.278 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 55637 2024-09-14T06:31:13.278 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:13.278 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:31:13.278 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:13.278 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:31:13.278 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600097+0000", 2024-09-14T06:31:13.278 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:13.278 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:13.278 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:13.278 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:31:13.278 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600097+0000", 2024-09-14T06:31:13.279 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:13.279 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:13.279 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:13.279 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:31:13.279 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.604372+0000", 2024-09-14T06:31:13.279 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0042751489999999998 2024-09-14T06:31:13.279 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:13.279 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:13.279 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:31:13.279 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.604377+0000", 2024-09-14T06:31:13.279 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.7899999999999999e-06 2024-09-14T06:31:13.279 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:13.279 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:13.280 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:31:13.280 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.604389+0000", 2024-09-14T06:31:13.280 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.2197e-05 2024-09-14T06:31:13.280 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:13.280 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:13.280 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:31:13.280 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.604397+0000", 2024-09-14T06:31:13.280 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.8730000000000007e-06 2024-09-14T06:31:13.280 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:13.280 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:13.280 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:31:13.280 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.643324+0000", 2024-09-14T06:31:13.280 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.038926897000000002 2024-09-14T06:31:13.281 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:13.281 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:13.281 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:31:13.281 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.643402+0000", 2024-09-14T06:31:13.281 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.8063000000000001e-05 2024-09-14T06:31:13.281 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:13.281 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:13.281 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:31:13.281 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.643496+0000", 2024-09-14T06:31:13.281 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.4155999999999998e-05 2024-09-14T06:31:13.281 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:13.281 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:31:13.281 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:13.281 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:13.281 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:13.282 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:55657 3.0 3:06490baf:::rbd_data.112dca2f6ae1.0000000000000030:17 [read 1687552~4096 [fadvise_nocache]] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:31:13.282 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:44.604841+0000", 2024-09-14T06:31:13.282 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 88.605514252000006, 2024-09-14T06:31:13.282 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.041890087999999999, 2024-09-14T06:31:13.282 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:31:13.282 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:31:13.282 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:31:13.282 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:31:13.282 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:31:13.282 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:13.282 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 55657 2024-09-14T06:31:13.282 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:13.282 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:31:13.282 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:13.282 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:31:13.283 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.604841+0000", 2024-09-14T06:31:13.283 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:13.283 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:13.283 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:13.283 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:31:13.283 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.604841+0000", 2024-09-14T06:31:13.283 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:13.283 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:13.283 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:13.283 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:31:13.283 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.604842+0000", 2024-09-14T06:31:13.283 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.0900000000000001e-07 2024-09-14T06:31:13.283 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:13.283 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:13.283 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:31:13.283 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.604846+0000", 2024-09-14T06:31:13.284 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.5759999999999999e-06 2024-09-14T06:31:13.284 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:13.284 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:13.284 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:31:13.284 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.604850+0000", 2024-09-14T06:31:13.284 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.405e-06 2024-09-14T06:31:13.284 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:13.284 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:13.284 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:31:13.284 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.604852+0000", 2024-09-14T06:31:13.284 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.8590000000000001e-06 2024-09-14T06:31:13.284 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:13.284 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:13.284 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:31:13.284 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.646633+0000", 2024-09-14T06:31:13.285 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.041780772000000001 2024-09-14T06:31:13.285 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:13.285 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:13.285 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:31:13.285 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.646677+0000", 2024-09-14T06:31:13.285 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.4115e-05 2024-09-14T06:31:13.285 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:13.285 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:13.285 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:31:13.285 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.646731+0000", 2024-09-14T06:31:13.285 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.4151999999999999e-05 2024-09-14T06:31:13.285 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:13.285 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:31:13.285 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:13.285 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:13.285 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:13.286 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:55658 3.0 3:06490baf:::rbd_data.112dca2f6ae1.0000000000000030:17 [read 1712128~4096 [fadvise_nocache]] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:31:13.286 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:44.604856+0000", 2024-09-14T06:31:13.286 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 88.605499652000006, 2024-09-14T06:31:13.286 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.041986227000000001, 2024-09-14T06:31:13.286 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:31:13.286 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:31:13.286 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:31:13.286 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:31:13.286 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:31:13.286 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:13.286 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 55658 2024-09-14T06:31:13.286 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:13.286 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:31:13.286 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:13.286 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:31:13.287 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.604856+0000", 2024-09-14T06:31:13.287 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:13.287 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:13.287 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:13.287 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:31:13.287 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.604856+0000", 2024-09-14T06:31:13.287 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:13.287 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:13.287 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:13.287 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:31:13.287 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.604856+0000", 2024-09-14T06:31:13.287 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.23e-07 2024-09-14T06:31:13.287 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:13.287 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:13.287 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:31:13.287 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.604857+0000", 2024-09-14T06:31:13.288 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.9999999999999995e-07 2024-09-14T06:31:13.288 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:13.288 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:13.288 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:31:13.288 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.604861+0000", 2024-09-14T06:31:13.288 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.9890000000000003e-06 2024-09-14T06:31:13.288 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:13.288 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:13.288 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:31:13.288 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.604864+0000", 2024-09-14T06:31:13.288 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.7429999999999998e-06 2024-09-14T06:31:13.288 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:13.288 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:13.288 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:31:13.288 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.646744+0000", 2024-09-14T06:31:13.289 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.041880325000000003 2024-09-14T06:31:13.289 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:13.289 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:13.289 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:31:13.289 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.646784+0000", 2024-09-14T06:31:13.289 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.9604000000000002e-05 2024-09-14T06:31:13.289 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:13.289 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:13.289 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:31:13.289 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.646842+0000", 2024-09-14T06:31:13.289 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.8343e-05 2024-09-14T06:31:13.289 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:13.289 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:31:13.289 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:13.289 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:13.290 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:13.290 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:55659 3.0 3:06490baf:::rbd_data.112dca2f6ae1.0000000000000030:17 [read 1736704~4096 [fadvise_nocache]] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:31:13.290 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:44.604867+0000", 2024-09-14T06:31:13.290 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 88.605488499000003, 2024-09-14T06:31:13.290 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.042097082000000001, 2024-09-14T06:31:13.290 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:31:13.290 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:31:13.290 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:31:13.290 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:31:13.290 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:31:13.290 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:13.290 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 55659 2024-09-14T06:31:13.290 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:13.291 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:31:13.291 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:13.291 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:31:13.291 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.604867+0000", 2024-09-14T06:31:13.291 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:13.291 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:13.291 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:13.291 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:31:13.291 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.604867+0000", 2024-09-14T06:31:13.291 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:13.291 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:13.291 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:13.291 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:31:13.291 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.604867+0000", 2024-09-14T06:31:13.291 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.2000000000000001e-07 2024-09-14T06:31:13.292 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:13.292 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:13.292 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:31:13.292 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.604868+0000", 2024-09-14T06:31:13.292 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.04e-06 2024-09-14T06:31:13.292 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:13.292 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:13.292 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:31:13.292 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.604871+0000", 2024-09-14T06:31:13.292 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.9689999999999999e-06 2024-09-14T06:31:13.292 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:13.292 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:13.292 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:31:13.292 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.604874+0000", 2024-09-14T06:31:13.293 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.5069999999999999e-06 2024-09-14T06:31:13.293 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:13.293 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:13.293 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:31:13.293 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.646874+0000", 2024-09-14T06:31:13.293 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.042000513000000003 2024-09-14T06:31:13.293 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:13.293 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:13.293 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:31:13.293 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.646914+0000", 2024-09-14T06:31:13.293 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.9443999999999997e-05 2024-09-14T06:31:13.293 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:13.293 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:13.293 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:31:13.294 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.646964+0000", 2024-09-14T06:31:13.294 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.0389000000000002e-05 2024-09-14T06:31:13.294 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:13.294 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:31:13.294 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:13.294 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:13.294 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:13.294 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:55660 3.0 3:06490baf:::rbd_data.112dca2f6ae1.0000000000000030:17 [read 1761280~4096 [fadvise_nocache]] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:31:13.294 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:44.604877+0000", 2024-09-14T06:31:13.294 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 88.605478750000003, 2024-09-14T06:31:13.294 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.042196349000000001, 2024-09-14T06:31:13.294 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:31:13.294 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:31:13.294 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:31:13.295 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:31:13.295 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:31:13.295 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:13.295 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 55660 2024-09-14T06:31:13.295 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:13.295 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:31:13.295 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:13.295 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:31:13.295 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.604877+0000", 2024-09-14T06:31:13.295 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:13.295 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:13.295 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:13.295 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:31:13.295 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.604877+0000", 2024-09-14T06:31:13.295 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:13.295 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:13.296 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:13.296 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:31:13.296 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.605810+0000", 2024-09-14T06:31:13.296 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00093336899999999997 2024-09-14T06:31:13.296 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:13.296 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:13.296 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:31:13.296 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.605815+0000", 2024-09-14T06:31:13.296 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.6140000000000002e-06 2024-09-14T06:31:13.296 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:13.296 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:13.296 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:31:13.296 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.605825+0000", 2024-09-14T06:31:13.296 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.8889999999999997e-06 2024-09-14T06:31:13.297 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:13.297 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:13.297 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:31:13.297 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.605831+0000", 2024-09-14T06:31:13.297 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.9800000000000003e-06 2024-09-14T06:31:13.297 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:13.297 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:13.297 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:31:13.297 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.646989+0000", 2024-09-14T06:31:13.297 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.041157878000000002 2024-09-14T06:31:13.297 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:13.297 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:13.297 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:31:13.297 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.647026+0000", 2024-09-14T06:31:13.297 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.7305000000000003e-05 2024-09-14T06:31:13.298 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:13.298 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:13.298 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:31:13.298 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.647073+0000", 2024-09-14T06:31:13.298 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.7314000000000001e-05 2024-09-14T06:31:13.298 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:13.298 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:31:13.298 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:13.298 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:13.298 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:13.298 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:55666 3.0 3:06490baf:::rbd_data.112dca2f6ae1.0000000000000030:17 [read 1908736~4096 [fadvise_nocache]] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:31:13.298 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:44.605898+0000", 2024-09-14T06:31:13.298 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 88.604458003999994, 2024-09-14T06:31:13.298 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.041862887000000001, 2024-09-14T06:31:13.299 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:31:13.299 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:31:13.299 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:31:13.299 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:31:13.299 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:31:13.299 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:13.299 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 55666 2024-09-14T06:31:13.299 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:13.299 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:31:13.299 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:13.299 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:31:13.299 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.605898+0000", 2024-09-14T06:31:13.299 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:13.299 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:13.299 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:13.299 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:31:13.300 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.605898+0000", 2024-09-14T06:31:13.300 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:13.300 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:13.300 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:13.300 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:31:13.300 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.611874+0000", 2024-09-14T06:31:13.300 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0059764049999999997 2024-09-14T06:31:13.300 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:13.300 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:13.300 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:31:13.300 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.611879+0000", 2024-09-14T06:31:13.300 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.8749999999999999e-06 2024-09-14T06:31:13.300 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:13.301 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:13.301 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:31:13.301 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.611887+0000", 2024-09-14T06:31:13.301 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.4109999999999996e-06 2024-09-14T06:31:13.301 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:13.301 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:13.301 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:31:13.301 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.611893+0000", 2024-09-14T06:31:13.301 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.2959999999999998e-06 2024-09-14T06:31:13.301 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:13.301 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:13.301 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:31:13.301 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.647674+0000", 2024-09-14T06:31:13.301 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.035781281999999998 2024-09-14T06:31:13.302 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:13.302 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:13.302 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:31:13.302 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.647711+0000", 2024-09-14T06:31:13.302 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.7434000000000001e-05 2024-09-14T06:31:13.302 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:13.302 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:13.302 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:31:13.302 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.647760+0000", 2024-09-14T06:31:13.302 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.9184000000000002e-05 2024-09-14T06:31:13.302 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:13.302 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:31:13.302 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:13.302 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:13.303 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:31:13.303 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-14T06:31:13.304 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-14T06:31:13.305 INFO:teuthology.orchestra.run.smithi094.stdout:4368 writesame 0x4b4cb4a thru 0x4b50df7 data_size 0xf(0x42ae bytes) 2024-09-14T06:31:13.305 INFO:teuthology.orchestra.run.smithi094.stdout:4369 trunc from 0x4b50df8 to 0x4303b84 2024-09-14T06:31:13.305 INFO:teuthology.orchestra.run.smithi094.stdout:4371 write 0xca6ec79 thru 0xca759fb (0x6d83 bytes) 2024-09-14T06:31:13.305 INFO:teuthology.orchestra.run.smithi094.stdout:4373 writesame 0x3ecb08c thru 0x3ed01ce data_size 0x1(0x5143 bytes) 2024-09-14T06:31:13.306 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-14T06:31:13.306 INFO:teuthology.orchestra.run.smithi094.stdout:4375 write 0x8d41776 thru 0x8d466a0 (0x4f2b bytes) 2024-09-14T06:31:13.306 INFO:tasks.ceph.osd.3.smithi094.stderr:2024-09-14T06:31:13.303+0000 7feeeafe8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33655) UID: 0 2024-09-14T06:31:13.307 INFO:teuthology.orchestra.run.smithi094.stdout:4376 write 0x7c20605 thru 0x7c27552 (0x6f4e bytes) 2024-09-14T06:31:13.312 INFO:teuthology.orchestra.run.smithi094.stdout:4378 punch from 0x379be06 to 0x37a6e48, (0xb042 bytes) 2024-09-14T06:31:13.312 INFO:teuthology.orchestra.run.smithi094.stdout:4380 read 0x7eaaa45 thru 0x7eaf335 (0x48f1 bytes) 2024-09-14T06:31:13.326 INFO:teuthology.orchestra.run.smithi094.stdout:4382 trunc from 0xca759fc to 0x112e2d8 2024-09-14T06:31:13.359 INFO:teuthology.orchestra.run.smithi094.stdout:4383 write 0x4f8d6e1 thru 0x4f92b7e (0x549e bytes) 2024-09-14T06:31:13.368 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-14T06:31:13.368 INFO:teuthology.orchestra.run.smithi094.stdout:4386 writesame 0x3859c4c thru 0x385a36a data_size 0x1(0x71f bytes) 2024-09-14T06:31:13.372 INFO:teuthology.orchestra.run.smithi094.stdout:4388 punch from 0x2ded44 to 0x2ed92d, (0xebe9 bytes) 2024-09-14T06:31:13.372 INFO:teuthology.orchestra.run.smithi094.stdout:4390 writesame 0x7a31 thru 0x168d3 data_size 0x1(0xeea3 bytes) 2024-09-14T06:31:13.385 INFO:teuthology.orchestra.run.smithi094.stdout:4392 punch from 0x146dcdd to 0x147c462, (0xe785 bytes) 2024-09-14T06:31:13.385 INFO:teuthology.orchestra.run.smithi094.stdout:4393 read 0x4ec555d thru 0x4ecc8a0 (0x7344 bytes) 2024-09-14T06:31:13.385 INFO:teuthology.orchestra.run.smithi094.stdout:4394 read 0x433fb61 thru 0x434f57a (0xfa1a bytes) 2024-09-14T06:31:13.401 INFO:teuthology.orchestra.run.smithi094.stdout:4395 writesame 0xc20c761 thru 0xc219fbd data_size 0x25(0xd85d bytes) 2024-09-14T06:31:13.401 INFO:teuthology.orchestra.run.smithi094.stdout:4397 write 0x99bad39 thru 0x99be3ce (0x3696 bytes) 2024-09-14T06:31:13.405 INFO:tasks.ceph.osd.2.smithi094.stderr:2024-09-14T06:31:13.404+0000 7faf3269b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33649) UID: 0 2024-09-14T06:31:13.405 INFO:teuthology.orchestra.run.smithi094.stdout:4399 punch from 0xa1af1e to 0xa274c5, (0xc5a7 bytes) 2024-09-14T06:31:13.408 INFO:teuthology.orchestra.run.smithi094.stdout:4400 trunc from 0xc219fbe to 0x986a2f4 2024-09-14T06:31:13.418 INFO:teuthology.orchestra.run.smithi094.stdout:4401 read 0x3bc3772 thru 0x3bd17b9 (0xe048 bytes) 2024-09-14T06:31:13.418 INFO:teuthology.orchestra.run.smithi094.stdout:4402 read 0xe5154e thru 0xe5c0a6 (0xab59 bytes) 2024-09-14T06:31:13.418 INFO:teuthology.orchestra.run.smithi094.stdout:4403 write 0x6efc789 thru 0x6f0683c (0xa0b4 bytes) 2024-09-14T06:31:13.426 INFO:teuthology.orchestra.run.smithi094.stdout:4404 write 0x83023e3 thru 0x831092f (0xe54d bytes) 2024-09-14T06:31:13.437 INFO:teuthology.orchestra.run.smithi094.stdout:4405 write 0xa6d38ba thru 0xa6e2eec (0xf633 bytes) 2024-09-14T06:31:13.440 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-14T06:31:13.440 INFO:teuthology.orchestra.run.smithi094.stdout: "ops": [], 2024-09-14T06:31:13.440 INFO:teuthology.orchestra.run.smithi094.stdout: "num_ops": 0 2024-09-14T06:31:13.440 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-14T06:31:13.443 INFO:teuthology.orchestra.run.smithi094.stdout:4406 punch from 0x9a46aff to 0x9a4780d, (0xd0e bytes) 2024-09-14T06:31:13.444 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-14T06:31:13.444 INFO:teuthology.orchestra.run.smithi094.stdout:4408 writesame 0x3a2d535 thru 0x3a2e49c data_size 0x22(0xf68 bytes) 2024-09-14T06:31:13.453 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-14T06:31:13.456 INFO:teuthology.orchestra.run.smithi094.stdout:4409 write 0xcb981f3 thru 0xcba29ab (0xa7b9 bytes) 2024-09-14T06:31:13.462 INFO:teuthology.orchestra.run.smithi094.stdout:4410 read 0xb99d7c1 thru 0xb99e45c (0xc9c bytes) 2024-09-14T06:31:13.462 INFO:teuthology.orchestra.run.smithi094.stdout:4411 read 0x600a1c8 thru 0x6017f24 (0xdd5d bytes) 2024-09-14T06:31:13.462 INFO:teuthology.orchestra.run.smithi094.stdout:4412 punch from 0xc8d73d0 to 0xc8db2e7, (0x3f17 bytes) 2024-09-14T06:31:13.462 INFO:teuthology.orchestra.run.smithi094.stdout:4413 write 0x85da31f thru 0x85e449e (0xa180 bytes) 2024-09-14T06:31:13.468 INFO:teuthology.orchestra.run.smithi094.stdout:4414 write 0x6e6303e thru 0x6e6a1b2 (0x7175 bytes) 2024-09-14T06:31:13.472 INFO:teuthology.orchestra.run.smithi094.stdout:4415 write 0x7d14512 thru 0x7d1fa21 (0xb510 bytes) 2024-09-14T06:31:13.477 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-14T06:31:13.477 INFO:teuthology.orchestra.run.smithi094.stdout:4417 read 0x6b15cca thru 0x6b167b8 (0xaef bytes) 2024-09-14T06:31:13.477 INFO:teuthology.orchestra.run.smithi094.stdout:4418 writesame 0x119124e thru 0x119911a data_size 0xd(0x7ecd bytes) 2024-09-14T06:31:13.483 INFO:teuthology.orchestra.run.smithi094.stdout:4421 punch from 0xa4a3a3 to 0xa57fd4, (0xdc31 bytes) 2024-09-14T06:31:13.483 INFO:teuthology.orchestra.run.smithi094.stdout:4424 punch from 0x5b6c2c7 to 0x5b6fab1, (0x37ea bytes) 2024-09-14T06:31:13.484 INFO:teuthology.orchestra.run.smithi094.stdout:4425 write 0x36e13d thru 0x372ddc (0x4ca0 bytes) 2024-09-14T06:31:13.485 INFO:teuthology.orchestra.run.smithi094.stdout:4427 write 0x717b90f thru 0x7188399 (0xca8b bytes) 2024-09-14T06:31:13.502 INFO:teuthology.orchestra.run.smithi094.stdout:4428 trunc from 0xcba29ac to 0x387e9f3 2024-09-14T06:31:13.506 INFO:tasks.ceph.osd.3.smithi094.stderr:2024-09-14T06:31:13.505+0000 7feeeafe8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33655) UID: 0 2024-09-14T06:31:13.507 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-14T06:31:13.523 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-14T06:31:13.523 INFO:teuthology.orchestra.run.smithi094.stdout:4430 writesame 0x2f1b3c6 thru 0x2f27303 data_size 0x1a(0xbf3e bytes) 2024-09-14T06:31:13.533 INFO:teuthology.orchestra.run.smithi094.stdout:4431 write 0x1afe261 thru 0x1b0b919 (0xd6b9 bytes) 2024-09-14T06:31:13.550 INFO:teuthology.orchestra.run.smithi094.stdout:4432 trunc from 0x387e9f3 to 0xc0f16d3 2024-09-14T06:31:13.556 INFO:teuthology.orchestra.run.smithi094.stdout:4434 read 0x49196e6 thru 0x491fbfc (0x6517 bytes) 2024-09-14T06:31:13.559 INFO:teuthology.orchestra.run.smithi094.stdout:4438 trunc from 0xc0f16d3 to 0xe4a741e 2024-09-14T06:31:13.564 INFO:teuthology.orchestra.run.smithi094.stdout:4439 clone 35 order 18 su 131072 sc 9 2024-09-14T06:31:13.579 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-14T06:31:13.579 INFO:teuthology.orchestra.run.smithi094.stdout: "ops": [], 2024-09-14T06:31:13.579 INFO:teuthology.orchestra.run.smithi094.stdout: "complaint_time": 30, 2024-09-14T06:31:13.579 INFO:teuthology.orchestra.run.smithi094.stdout: "num_blocked_ops": 0 2024-09-14T06:31:13.579 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-14T06:31:13.589 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-14T06:31:13.630 INFO:tasks.ceph.osd.0.smithi094.stderr:2024-09-14T06:31:13.629+0000 7f07c9333640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33636) UID: 0 2024-09-14T06:31:13.636 INFO:teuthology.orchestra.run.smithi094.stdout:osd.0: {} 2024-09-14T06:31:13.636 INFO:teuthology.orchestra.run.smithi094.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-14T06:31:13.654 INFO:teuthology.orchestra.run.smithi094.stdout:osd.1: {} 2024-09-14T06:31:13.654 INFO:teuthology.orchestra.run.smithi094.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-14T06:31:13.670 INFO:teuthology.orchestra.run.smithi094.stdout:osd.2: {} 2024-09-14T06:31:13.670 INFO:teuthology.orchestra.run.smithi094.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-14T06:31:13.687 INFO:teuthology.orchestra.run.smithi094.stdout:osd.3: {} 2024-09-14T06:31:13.687 INFO:teuthology.orchestra.run.smithi094.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-14T06:31:13.703 INFO:teuthology.orchestra.run.smithi094.stdout:osd.4: {} 2024-09-14T06:31:13.704 INFO:teuthology.orchestra.run.smithi094.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-14T06:31:13.708 INFO:tasks.ceph.osd.1.smithi094.stderr:2024-09-14T06:31:13.707+0000 7f328e47a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33641) UID: 0 2024-09-14T06:31:13.720 INFO:teuthology.orchestra.run.smithi094.stdout:osd.5: {} 2024-09-14T06:31:13.720 INFO:teuthology.orchestra.run.smithi094.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-14T06:31:13.734 INFO:teuthology.orchestra.run.smithi094.stdout:ERROR: (22) Invalid argument 2024-09-14T06:31:13.734 INFO:teuthology.orchestra.run.smithi094.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:31:13.737 INFO:teuthology.orchestra.run.smithi094.stdout:osd.6: {} 2024-09-14T06:31:13.737 INFO:teuthology.orchestra.run.smithi094.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-14T06:31:13.743 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-14T06:31:13.754 INFO:teuthology.orchestra.run.smithi094.stdout:osd.7: {} 2024-09-14T06:31:13.755 INFO:teuthology.orchestra.run.smithi094.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-14T06:31:13.808 INFO:tasks.ceph.osd.7.smithi138.stderr:2024-09-14T06:31:13.807+0000 7f3c6a21d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33623) UID: 0 2024-09-14T06:31:13.862 INFO:teuthology.orchestra.run.smithi094.stdout:ERROR: (22) Invalid argument 2024-09-14T06:31:13.863 INFO:teuthology.orchestra.run.smithi094.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:31:13.873 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-14T06:31:13.916 INFO:tasks.ceph.osd.0.smithi094.stderr:2024-09-14T06:31:13.915+0000 7f07c9333640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33636) UID: 0 2024-09-14T06:31:13.975 INFO:teuthology.orchestra.run.smithi094.stdout:ERROR: (22) Invalid argument 2024-09-14T06:31:13.975 INFO:teuthology.orchestra.run.smithi094.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:31:13.984 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-14T06:31:14.010 INFO:tasks.ceph.osd.7.smithi138.stderr:2024-09-14T06:31:14.010+0000 7f3c6a21d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33623) UID: 0 2024-09-14T06:31:14.112 INFO:tasks.ceph.osd.6.smithi138.stderr:2024-09-14T06:31:14.111+0000 7f8cd8849640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33620) UID: 0 2024-09-14T06:31:14.127 INFO:teuthology.orchestra.run.smithi094.stdout:ERROR: (22) Invalid argument 2024-09-14T06:31:14.127 INFO:teuthology.orchestra.run.smithi094.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:31:14.136 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-14T06:31:14.213 INFO:tasks.ceph.osd.3.smithi094.stderr:2024-09-14T06:31:14.212+0000 7feeeafe8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33655) UID: 0 2024-09-14T06:31:14.241 INFO:teuthology.orchestra.run.smithi138.stdout:ERROR: (22) Invalid argument 2024-09-14T06:31:14.241 INFO:teuthology.orchestra.run.smithi138.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:31:14.250 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-14T06:31:14.313 INFO:tasks.ceph.osd.6.smithi138.stderr:2024-09-14T06:31:14.312+0000 7f8cd8849640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33620) UID: 0 2024-09-14T06:31:14.352 INFO:teuthology.orchestra.run.smithi138.stdout:ERROR: (22) Invalid argument 2024-09-14T06:31:14.353 INFO:teuthology.orchestra.run.smithi138.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:31:14.361 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-14T06:31:14.445 INFO:tasks.ceph.osd.7.smithi138.stderr:2024-09-14T06:31:14.444+0000 7f3c6a21d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33623) UID: 0 2024-09-14T06:31:14.504 INFO:teuthology.orchestra.run.smithi138.stdout:ERROR: (22) Invalid argument 2024-09-14T06:31:14.504 INFO:teuthology.orchestra.run.smithi138.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:31:14.512 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-14T06:31:14.553 INFO:tasks.ceph.osd.4.smithi138.stderr:2024-09-14T06:31:14.552+0000 7f6c653c8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33615) UID: 0 2024-09-14T06:31:14.615 INFO:tasks.ceph.osd.4.smithi138.stderr:2024-09-14T06:31:14.614+0000 7f6c653c8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33615) UID: 0 2024-09-14T06:31:14.656 INFO:teuthology.orchestra.run.smithi138.stdout:ERROR: (22) Invalid argument 2024-09-14T06:31:14.657 INFO:teuthology.orchestra.run.smithi138.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:31:14.665 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-14T06:31:14.715 INFO:tasks.ceph.osd.2.smithi094.stderr:2024-09-14T06:31:14.715+0000 7faf3269b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33649) UID: 0 2024-09-14T06:31:14.807 INFO:teuthology.orchestra.run.smithi138.stdout:ERROR: (22) Invalid argument 2024-09-14T06:31:14.808 INFO:teuthology.orchestra.run.smithi138.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:31:14.816 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-14T06:31:14.817 INFO:tasks.ceph.osd.6.smithi138.stderr:2024-09-14T06:31:14.816+0000 7f8cd8849640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33620) UID: 0 2024-09-14T06:31:14.842 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-14T06:31:14.918 INFO:tasks.ceph.osd.2.smithi094.stderr:2024-09-14T06:31:14.917+0000 7faf3269b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33649) UID: 0 2024-09-14T06:31:14.953 INFO:teuthology.orchestra.run.smithi138.stdout:ERROR: (22) Invalid argument 2024-09-14T06:31:14.953 INFO:teuthology.orchestra.run.smithi138.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:31:14.963 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-14T06:31:15.019 INFO:tasks.ceph.osd.5.smithi138.stderr:2024-09-14T06:31:15.018+0000 7fe13746f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33618) UID: 0 2024-09-14T06:31:15.066 INFO:teuthology.orchestra.run.smithi138.stdout:ERROR: (22) Invalid argument 2024-09-14T06:31:15.066 INFO:teuthology.orchestra.run.smithi138.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:31:15.074 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-14T06:31:15.120 INFO:tasks.ceph.osd.0.smithi094.stderr:2024-09-14T06:31:15.119+0000 7f07c9333640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33636) UID: 0 2024-09-14T06:31:15.217 INFO:teuthology.orchestra.run.smithi138.stdout:ERROR: (22) Invalid argument 2024-09-14T06:31:15.217 INFO:teuthology.orchestra.run.smithi138.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:31:15.220 INFO:tasks.ceph.osd.4.smithi138.stderr:2024-09-14T06:31:15.219+0000 7f6c653c8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33615) UID: 0 2024-09-14T06:31:15.225 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-14T06:31:15.322 INFO:tasks.ceph.osd.4.smithi138.stderr:2024-09-14T06:31:15.321+0000 7f6c653c8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33615) UID: 0 2024-09-14T06:31:15.369 INFO:teuthology.orchestra.run.smithi138.stdout:ERROR: (22) Invalid argument 2024-09-14T06:31:15.369 INFO:teuthology.orchestra.run.smithi138.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:31:15.378 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-14T06:31:15.464 INFO:tasks.ceph.osd.5.smithi138.stderr:2024-09-14T06:31:15.462+0000 7fe13746f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33618) UID: 0 2024-09-14T06:31:15.522 INFO:teuthology.orchestra.run.smithi138.stdout:ERROR: (22) Invalid argument 2024-09-14T06:31:15.522 INFO:teuthology.orchestra.run.smithi138.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:31:15.524 INFO:tasks.ceph.osd.6.smithi138.stderr:2024-09-14T06:31:15.523+0000 7f8cd8849640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33620) UID: 0 2024-09-14T06:31:15.530 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-14T06:31:15.597 INFO:teuthology.orchestra.run.smithi094.stderr:nodeep-scrub is set 2024-09-14T06:31:15.654 INFO:tasks.ceph.osd.1.smithi094.stderr:2024-09-14T06:31:15.653+0000 7f328e47a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33641) UID: 0 2024-09-14T06:31:15.677 INFO:teuthology.orchestra.run.smithi138.stdout:ERROR: (22) Invalid argument 2024-09-14T06:31:15.677 INFO:teuthology.orchestra.run.smithi138.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:31:15.686 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-14T06:31:15.730 INFO:tasks.ceph.osd.6.smithi138.stderr:2024-09-14T06:31:15.729+0000 7f8cd8849640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33620) UID: 0 2024-09-14T06:31:15.753 INFO:teuthology.orchestra.run.smithi094.stdout:checking clone #33, image image_client.0-clone33 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent34 2024-09-14T06:31:15.766 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-14T06:31:15.793 INFO:teuthology.orchestra.run.smithi138.stdout:ERROR: (22) Invalid argument 2024-09-14T06:31:15.793 INFO:teuthology.orchestra.run.smithi138.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:31:15.802 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-14T06:31:15.843 INFO:tasks.ceph.osd.5.smithi138.stderr:2024-09-14T06:31:15.842+0000 7fe13746f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33618) UID: 0 2024-09-14T06:31:15.843 INFO:teuthology.orchestra.run.smithi094.stdout:4440 write 0x3306e49 thru 0x330e7a0 (0x7958 bytes) 2024-09-14T06:31:15.858 INFO:teuthology.orchestra.run.smithi094.stdout:4441 write 0xeab7b42 thru 0xeabcb35 (0x4ff4 bytes) 2024-09-14T06:31:15.863 INFO:teuthology.orchestra.run.smithi094.stdout:4442 punch from 0x7de8638 to 0x7dee5b6, (0x5f7e bytes) 2024-09-14T06:31:15.863 INFO:teuthology.orchestra.run.smithi094.stdout:4443 punch from 0x1be265a to 0x1befb73, (0xd519 bytes) 2024-09-14T06:31:15.863 INFO:teuthology.orchestra.run.smithi094.stdout:4444 read 0x61c0cd1 thru 0x61cc5bd (0xb8ed bytes) 2024-09-14T06:31:15.863 INFO:teuthology.orchestra.run.smithi094.stdout:4445 writesame 0xbea2155 thru 0xbea3d7e data_size 0x23(0x1c2a bytes) 2024-09-14T06:31:15.866 INFO:teuthology.orchestra.run.smithi094.stdout:4448 writesame 0x41e8e41 thru 0x41ecb19 data_size 0x1(0x3cd9 bytes) 2024-09-14T06:31:15.877 INFO:teuthology.orchestra.run.smithi094.stdout:4450 read 0x94b4e12 thru 0x94bb757 (0x6946 bytes) 2024-09-14T06:31:15.877 INFO:teuthology.orchestra.run.smithi094.stdout:4451 write 0x246d473 thru 0x247cc09 (0xf797 bytes) 2024-09-14T06:31:15.883 INFO:teuthology.orchestra.run.smithi094.stdout:4452 punch from 0x9bf5064 to 0x9bf84d2, (0x346e bytes) 2024-09-14T06:31:15.883 INFO:teuthology.orchestra.run.smithi094.stdout:4454 read 0x78fc32c thru 0x78fec55 (0x292a bytes) 2024-09-14T06:31:15.883 INFO:teuthology.orchestra.run.smithi094.stdout:4456 read 0x8c7ee21 thru 0x8c7fb66 (0xd46 bytes) 2024-09-14T06:31:15.883 INFO:teuthology.orchestra.run.smithi094.stdout:4458 punch from 0xb64521b to 0xb648a16, (0x37fb bytes) 2024-09-14T06:31:15.883 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-14T06:31:15.883 INFO:teuthology.orchestra.run.smithi094.stdout:4460 read 0x9d8197d thru 0x9d90d8c (0xf410 bytes) 2024-09-14T06:31:15.883 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-14T06:31:15.883 INFO:teuthology.orchestra.run.smithi094.stdout:4462 read 0xc4efcb5 thru 0xc4f292f (0x2c7b bytes) 2024-09-14T06:31:15.883 INFO:teuthology.orchestra.run.smithi094.stdout:4463 read 0x751b047 thru 0x7524110 (0x90ca bytes) 2024-09-14T06:31:15.884 INFO:teuthology.orchestra.run.smithi094.stdout:4465 writesame 0x45c7ccc thru 0x45d5bdb data_size 0x56(0xdf10 bytes) 2024-09-14T06:31:15.888 INFO:teuthology.orchestra.run.smithi094.stdout:4467 read 0xbb7d43e thru 0xbb88a3c (0xb5ff bytes) 2024-09-14T06:31:15.888 INFO:teuthology.orchestra.run.smithi094.stdout:4468 read 0xc5e116 thru 0xc65b6c (0x7a57 bytes) 2024-09-14T06:31:15.888 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-14T06:31:15.888 INFO:teuthology.orchestra.run.smithi094.stdout:4470 read 0x66e1453 thru 0x66eac30 (0x97de bytes) 2024-09-14T06:31:15.888 INFO:teuthology.orchestra.run.smithi094.stdout:4471 read 0x2f9ea00 thru 0x2f9f6ad (0xcae bytes) 2024-09-14T06:31:15.888 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-14T06:31:15.888 INFO:teuthology.orchestra.run.smithi094.stdout:4474 read 0x4214a95 thru 0x421b3d9 (0x6945 bytes) 2024-09-14T06:31:15.888 INFO:teuthology.orchestra.run.smithi094.stdout:4476 punch from 0x110f8b5 to 0x111ecee, (0xf439 bytes) 2024-09-14T06:31:15.889 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-14T06:31:15.889 INFO:teuthology.orchestra.run.smithi094.stdout:4479 punch from 0xc2fcc44 to 0xc30bc2a, (0xefe6 bytes) 2024-09-14T06:31:15.889 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-14T06:31:15.889 INFO:teuthology.orchestra.run.smithi094.stdout:4482 read 0x7a8ba4d thru 0x7a9459c (0x8b50 bytes) 2024-09-14T06:31:15.889 INFO:teuthology.orchestra.run.smithi094.stdout:4484 write 0xa30830 thru 0xa3998c (0x915d bytes) 2024-09-14T06:31:15.891 INFO:teuthology.orchestra.run.smithi094.stdout:4485 read 0x15f0636 thru 0x15f87e1 (0x81ac bytes) 2024-09-14T06:31:15.891 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-14T06:31:15.891 INFO:teuthology.orchestra.run.smithi094.stdout:4487 write 0x93b5866 thru 0x93b9115 (0x38b0 bytes) 2024-09-14T06:31:15.894 INFO:teuthology.orchestra.run.smithi094.stdout:4488 writesame 0xdb538d9 thru 0xdb5560a data_size 0x2(0x1d32 bytes) 2024-09-14T06:31:15.899 INFO:teuthology.orchestra.run.smithi094.stdout:4491 punch from 0x569c4ff to 0x56a5aaf, (0x95b0 bytes) 2024-09-14T06:31:15.899 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-14T06:31:15.899 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-14T06:31:15.899 INFO:teuthology.orchestra.run.smithi094.stdout:4495 write 0xdb4deb0 thru 0xdb51348 (0x3499 bytes) 2024-09-14T06:31:15.900 INFO:teuthology.orchestra.run.smithi094.stdout:osd.0: {} 2024-09-14T06:31:15.901 INFO:teuthology.orchestra.run.smithi094.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-14T06:31:15.901 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-14T06:31:15.901 INFO:teuthology.orchestra.run.smithi094.stdout:4500 punch from 0x15ac68a to 0x15b86df, (0xc055 bytes) 2024-09-14T06:31:15.902 INFO:teuthology.orchestra.run.smithi094.stdout:4501 write 0xba098cb thru 0xba15c01 (0xc337 bytes) 2024-09-14T06:31:15.906 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-14T06:31:15.912 INFO:teuthology.orchestra.run.smithi094.stdout:4506 trunc from 0xeabcb36 to 0x8e7c55d 2024-09-14T06:31:15.920 INFO:teuthology.orchestra.run.smithi094.stdout:osd.1: {} 2024-09-14T06:31:15.920 INFO:teuthology.orchestra.run.smithi094.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-14T06:31:15.927 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-14T06:31:15.927 INFO:teuthology.orchestra.run.smithi094.stdout: "ops": [], 2024-09-14T06:31:15.927 INFO:teuthology.orchestra.run.smithi094.stdout: "num_ops": 0 2024-09-14T06:31:15.927 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-14T06:31:15.928 INFO:teuthology.orchestra.run.smithi094.stdout:4507 read 0x36f1c46 thru 0x36ff14f (0xd50a bytes) 2024-09-14T06:31:15.928 INFO:teuthology.orchestra.run.smithi094.stdout:4509 write 0x80049f7 thru 0x8011997 (0xcfa1 bytes) 2024-09-14T06:31:15.928 INFO:tasks.ceph.osd.0.smithi094.stderr:2024-09-14T06:31:15.927+0000 7f07c9333640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33636) UID: 0 2024-09-14T06:31:15.932 INFO:teuthology.orchestra.run.smithi094.stdout:4511 punch from 0x75f04f2 to 0x75fb6f5, (0xb203 bytes) 2024-09-14T06:31:15.932 INFO:teuthology.orchestra.run.smithi094.stdout:4512 read 0x4f0b531 thru 0x4f153b7 (0x9e87 bytes) 2024-09-14T06:31:15.932 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-14T06:31:15.932 INFO:teuthology.orchestra.run.smithi094.stdout:4514 write 0x2e43ef0 thru 0x2e501a4 (0xc2b5 bytes) 2024-09-14T06:31:15.935 INFO:teuthology.orchestra.run.smithi094.stdout:4516 read 0x69089ab thru 0x6909dad (0x1403 bytes) 2024-09-14T06:31:15.935 INFO:teuthology.orchestra.run.smithi094.stdout:4517 read 0x6283b05 thru 0x62934ae (0xf9aa bytes) 2024-09-14T06:31:15.935 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-14T06:31:15.935 INFO:teuthology.orchestra.run.smithi094.stdout:4519 punch from 0x2dc1439 to 0x2dc9cdf, (0x88a6 bytes) 2024-09-14T06:31:15.936 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-14T06:31:15.937 INFO:teuthology.orchestra.run.smithi094.stdout:osd.2: {} 2024-09-14T06:31:15.937 INFO:teuthology.orchestra.run.smithi094.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-14T06:31:15.940 INFO:teuthology.orchestra.run.smithi094.stdout:4520 trunc from 0x8e7c55d to 0x5ed3d27 2024-09-14T06:31:15.955 INFO:teuthology.orchestra.run.smithi094.stdout:osd.3: {} 2024-09-14T06:31:15.955 INFO:teuthology.orchestra.run.smithi094.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-14T06:31:15.961 INFO:teuthology.orchestra.run.smithi094.stdout:4522 write 0xb5bc17e thru 0xb5c4e67 (0x8cea bytes) 2024-09-14T06:31:15.973 INFO:teuthology.orchestra.run.smithi094.stdout:4523 trunc from 0xb5c4e68 to 0x4872374 2024-09-14T06:31:15.974 INFO:teuthology.orchestra.run.smithi094.stdout:osd.4: {} 2024-09-14T06:31:15.975 INFO:teuthology.orchestra.run.smithi094.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-14T06:31:15.985 INFO:teuthology.orchestra.run.smithi094.stdout:4524 trunc from 0x4872374 to 0x2e6ec2d 2024-09-14T06:31:15.993 INFO:teuthology.orchestra.run.smithi094.stdout:osd.5: {} 2024-09-14T06:31:15.993 INFO:teuthology.orchestra.run.smithi094.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-14T06:31:16.010 INFO:teuthology.orchestra.run.smithi094.stdout:osd.6: {} 2024-09-14T06:31:16.011 INFO:teuthology.orchestra.run.smithi094.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-14T06:31:16.019 INFO:teuthology.orchestra.run.smithi094.stdout:4525 write 0xe67f658 thru 0xe688bf1 (0x959a bytes) 2024-09-14T06:31:16.024 INFO:teuthology.orchestra.run.smithi094.stdout:4527 clone 36 order 24 su 131072 sc 13 2024-09-14T06:31:16.029 INFO:teuthology.orchestra.run.smithi094.stdout:osd.7: {} 2024-09-14T06:31:16.029 INFO:teuthology.orchestra.run.smithi094.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-14T06:31:16.033 INFO:tasks.ceph.osd.3.smithi094.stderr:2024-09-14T06:31:16.028+0000 7feeeafe8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33655) UID: 0 2024-09-14T06:31:16.052 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-14T06:31:16.052 INFO:teuthology.orchestra.run.smithi094.stdout: "ops": [], 2024-09-14T06:31:16.052 INFO:teuthology.orchestra.run.smithi094.stdout: "complaint_time": 30, 2024-09-14T06:31:16.053 INFO:teuthology.orchestra.run.smithi094.stdout: "num_blocked_ops": 0 2024-09-14T06:31:16.053 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-14T06:31:16.061 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-14T06:31:16.145 INFO:tasks.ceph.osd.3.smithi094.stderr:2024-09-14T06:31:16.144+0000 7feeeafe8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33655) UID: 0 2024-09-14T06:31:16.206 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-14T06:31:16.206 INFO:teuthology.orchestra.run.smithi094.stdout: "size": 20, 2024-09-14T06:31:16.206 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 600, 2024-09-14T06:31:16.206 INFO:teuthology.orchestra.run.smithi094.stdout: "ops": [ 2024-09-14T06:31:16.207 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:16.207 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:115193 3.1 3:856be886:::rbd_data.112d4d24c558.0000000000000004:29 [read 1523712~16384 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e96)", 2024-09-14T06:31:16.207 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:30:17.939854+0000", 2024-09-14T06:31:16.207 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 58.264269444, 2024-09-14T06:31:16.207 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.073897516999999996, 2024-09-14T06:31:16.207 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:31:16.207 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:31:16.207 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:31:16.207 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:31:16.207 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:31:16.207 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:16.207 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 115193 2024-09-14T06:31:16.207 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:16.208 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:31:16.208 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:16.208 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:31:16.208 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.939854+0000", 2024-09-14T06:31:16.208 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:16.208 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:16.208 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:16.208 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:31:16.209 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.939854+0000", 2024-09-14T06:31:16.209 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:16.209 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:16.209 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:16.209 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:31:16.209 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.939854+0000", 2024-09-14T06:31:16.209 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.9500000000000002e-07 2024-09-14T06:31:16.209 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:16.209 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:16.209 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:31:16.209 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.939856+0000", 2024-09-14T06:31:16.210 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.0959999999999999e-06 2024-09-14T06:31:16.210 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:16.210 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:16.210 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:31:16.210 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.939863+0000", 2024-09-14T06:31:16.210 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.1129999999999997e-06 2024-09-14T06:31:16.210 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:16.210 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:16.210 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:31:16.211 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.939867+0000", 2024-09-14T06:31:16.211 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.8350000000000003e-06 2024-09-14T06:31:16.211 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:16.211 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:16.211 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:31:16.211 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.013626+0000", 2024-09-14T06:31:16.211 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.073758799999999999 2024-09-14T06:31:16.211 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:16.211 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:16.211 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:31:16.212 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.013690+0000", 2024-09-14T06:31:16.212 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.3918999999999996e-05 2024-09-14T06:31:16.212 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:16.212 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:16.212 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:31:16.212 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.013751+0000", 2024-09-14T06:31:16.212 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.1159000000000002e-05 2024-09-14T06:31:16.212 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:16.213 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:31:16.213 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:16.213 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:16.213 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:16.213 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:115194 3.1 3:856be886:::rbd_data.112d4d24c558.0000000000000004:29 [read 1769472~16384 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e96)", 2024-09-14T06:31:16.213 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:30:17.939874+0000", 2024-09-14T06:31:16.213 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 58.264249133, 2024-09-14T06:31:16.213 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.074016988000000006, 2024-09-14T06:31:16.213 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:31:16.213 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:31:16.213 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:31:16.213 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:31:16.214 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:31:16.214 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:16.214 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 115194 2024-09-14T06:31:16.214 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:16.214 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:31:16.214 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:16.214 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:31:16.214 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.939874+0000", 2024-09-14T06:31:16.214 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:16.214 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:16.214 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:16.214 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:31:16.214 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.939874+0000", 2024-09-14T06:31:16.214 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:16.215 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:16.215 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:16.215 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:31:16.215 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.939875+0000", 2024-09-14T06:31:16.215 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.2600000000000002e-07 2024-09-14T06:31:16.215 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:16.215 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:16.215 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:31:16.215 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.939877+0000", 2024-09-14T06:31:16.215 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.9420000000000002e-06 2024-09-14T06:31:16.215 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:16.215 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:16.215 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:31:16.215 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.939882+0000", 2024-09-14T06:31:16.216 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.8749999999999997e-06 2024-09-14T06:31:16.216 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:16.216 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:16.216 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:31:16.216 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.939887+0000", 2024-09-14T06:31:16.216 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.741e-06 2024-09-14T06:31:16.216 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:16.216 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:16.216 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:31:16.216 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.013767+0000", 2024-09-14T06:31:16.216 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.073879720999999995 2024-09-14T06:31:16.216 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:16.216 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:16.216 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:31:16.217 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.013832+0000", 2024-09-14T06:31:16.217 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.5312000000000001e-05 2024-09-14T06:31:16.217 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:16.217 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:16.217 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:31:16.217 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.013891+0000", 2024-09-14T06:31:16.217 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.8870999999999999e-05 2024-09-14T06:31:16.217 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:16.217 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:31:16.217 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:16.217 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:16.217 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:16.217 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:115195 3.1 3:856be886:::rbd_data.112d4d24c558.0000000000000004:29 [read 2015232~16384 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e96)", 2024-09-14T06:31:16.217 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:30:17.939903+0000", 2024-09-14T06:31:16.218 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 58.264219947999997, 2024-09-14T06:31:16.218 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.074128199000000006, 2024-09-14T06:31:16.218 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:31:16.218 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:31:16.218 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:31:16.218 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:31:16.218 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:31:16.218 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:16.218 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 115195 2024-09-14T06:31:16.218 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:16.218 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:31:16.218 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:16.218 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:31:16.218 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.939903+0000", 2024-09-14T06:31:16.218 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:16.219 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:16.219 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:16.219 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:31:16.219 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.939903+0000", 2024-09-14T06:31:16.219 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:16.219 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:16.219 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:16.219 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:31:16.219 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.939904+0000", 2024-09-14T06:31:16.219 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.5799999999999999e-07 2024-09-14T06:31:16.219 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:16.219 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:16.219 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:31:16.219 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.939906+0000", 2024-09-14T06:31:16.219 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.2350000000000002e-06 2024-09-14T06:31:16.220 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:16.220 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:16.220 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:31:16.220 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.939913+0000", 2024-09-14T06:31:16.220 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.5649999999999998e-06 2024-09-14T06:31:16.220 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:16.220 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:16.220 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:31:16.220 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.939918+0000", 2024-09-14T06:31:16.220 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.784e-06 2024-09-14T06:31:16.220 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:16.220 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:16.220 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:31:16.220 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.013921+0000", 2024-09-14T06:31:16.221 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.074002911000000005 2024-09-14T06:31:16.221 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:16.221 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:16.221 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:31:16.221 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.013971+0000", 2024-09-14T06:31:16.221 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.9551e-05 2024-09-14T06:31:16.221 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:16.221 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:16.221 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:31:16.221 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.014031+0000", 2024-09-14T06:31:16.221 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.0594999999999998e-05 2024-09-14T06:31:16.221 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:16.221 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:31:16.221 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:16.222 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:16.222 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:16.222 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:115196 3.1 3:856be886:::rbd_data.112d4d24c558.0000000000000004:29 [read 2260992~16384 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e96)", 2024-09-14T06:31:16.222 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:30:17.939925+0000", 2024-09-14T06:31:16.222 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 58.264197756000002, 2024-09-14T06:31:16.222 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.074320137999999994, 2024-09-14T06:31:16.222 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:31:16.222 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:31:16.222 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:31:16.222 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:31:16.222 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:31:16.222 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:16.222 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 115196 2024-09-14T06:31:16.222 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:16.223 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:31:16.223 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:16.223 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:31:16.223 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.939925+0000", 2024-09-14T06:31:16.223 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:16.223 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:16.223 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:16.223 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:31:16.223 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.939925+0000", 2024-09-14T06:31:16.223 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:16.223 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:16.223 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:16.223 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:31:16.223 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.939926+0000", 2024-09-14T06:31:16.224 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.9999999999999998e-07 2024-09-14T06:31:16.224 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:16.224 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:16.224 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:31:16.224 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.939928+0000", 2024-09-14T06:31:16.224 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.9549999999999999e-06 2024-09-14T06:31:16.224 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:16.224 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:16.224 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:31:16.224 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.939934+0000", 2024-09-14T06:31:16.224 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.127e-06 2024-09-14T06:31:16.224 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:16.224 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:16.224 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:31:16.224 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.939939+0000", 2024-09-14T06:31:16.225 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.775e-06 2024-09-14T06:31:16.225 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:16.225 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:16.225 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:31:16.225 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.014075+0000", 2024-09-14T06:31:16.225 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.074136598999999997 2024-09-14T06:31:16.225 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:16.225 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:16.225 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:31:16.225 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.014162+0000", 2024-09-14T06:31:16.225 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.6477999999999994e-05 2024-09-14T06:31:16.225 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:16.225 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:16.225 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:31:16.226 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.014246+0000", 2024-09-14T06:31:16.226 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.3703999999999997e-05 2024-09-14T06:31:16.226 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:16.226 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:31:16.226 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:16.226 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:16.226 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:16.226 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:115197 3.1 3:856be886:::rbd_data.112d4d24c558.0000000000000004:29 [read 2506752~16384 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e96)", 2024-09-14T06:31:16.226 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:30:17.939969+0000", 2024-09-14T06:31:16.226 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 58.264154271000002, 2024-09-14T06:31:16.226 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.074528544000000002, 2024-09-14T06:31:16.226 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:31:16.226 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:31:16.226 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:31:16.227 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:31:16.227 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:31:16.227 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:16.227 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 115197 2024-09-14T06:31:16.227 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:16.227 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:31:16.227 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:16.227 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:31:16.227 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.939969+0000", 2024-09-14T06:31:16.227 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:16.227 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:16.227 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:16.227 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:31:16.227 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.939969+0000", 2024-09-14T06:31:16.228 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:16.228 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:16.228 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:16.228 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:31:16.228 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.939970+0000", 2024-09-14T06:31:16.228 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.3600000000000003e-07 2024-09-14T06:31:16.228 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:16.228 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:16.228 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:31:16.228 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.939972+0000", 2024-09-14T06:31:16.228 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.3180000000000001e-06 2024-09-14T06:31:16.228 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:16.228 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:16.228 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:31:16.229 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.939979+0000", 2024-09-14T06:31:16.229 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.8800000000000002e-06 2024-09-14T06:31:16.229 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:16.229 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:16.229 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:31:16.229 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.939984+0000", 2024-09-14T06:31:16.229 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.3510000000000001e-06 2024-09-14T06:31:16.229 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:16.229 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:16.229 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:31:16.229 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.014302+0000", 2024-09-14T06:31:16.229 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.074317416999999997 2024-09-14T06:31:16.229 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:16.229 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:16.230 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:31:16.230 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.014392+0000", 2024-09-14T06:31:16.230 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.0181999999999998e-05 2024-09-14T06:31:16.230 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:16.230 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:16.230 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:31:16.230 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.014497+0000", 2024-09-14T06:31:16.230 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00010576 2024-09-14T06:31:16.230 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:16.230 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:31:16.230 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:16.230 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:16.230 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:16.230 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:115198 3.1 3:856be886:::rbd_data.112d4d24c558.0000000000000004:29 [read 2752512~16384 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e96)", 2024-09-14T06:31:16.231 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:30:17.940004+0000", 2024-09-14T06:31:16.231 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 58.264118990999997, 2024-09-14T06:31:16.231 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.074736367999999997, 2024-09-14T06:31:16.231 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:31:16.231 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:31:16.231 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:31:16.231 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:31:16.231 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:31:16.231 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:16.231 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 115198 2024-09-14T06:31:16.231 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:16.231 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:31:16.231 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:16.231 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:31:16.231 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940004+0000", 2024-09-14T06:31:16.231 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:16.232 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:16.232 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:16.232 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:31:16.232 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940004+0000", 2024-09-14T06:31:16.232 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:16.232 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:16.232 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:16.232 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:31:16.232 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940005+0000", 2024-09-14T06:31:16.232 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.2200000000000004e-07 2024-09-14T06:31:16.232 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:16.232 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:16.232 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:31:16.232 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940007+0000", 2024-09-14T06:31:16.233 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.322e-06 2024-09-14T06:31:16.233 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:16.233 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:16.233 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:31:16.233 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940014+0000", 2024-09-14T06:31:16.233 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.6220000000000003e-06 2024-09-14T06:31:16.233 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:16.233 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:16.233 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:31:16.233 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940019+0000", 2024-09-14T06:31:16.233 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.6380000000000001e-06 2024-09-14T06:31:16.233 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:16.233 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:16.233 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:31:16.234 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.014562+0000", 2024-09-14T06:31:16.234 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.074542320999999995 2024-09-14T06:31:16.234 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:16.234 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:16.234 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:31:16.234 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.014646+0000", 2024-09-14T06:31:16.234 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.4079999999999995e-05 2024-09-14T06:31:16.234 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:16.234 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:16.234 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:31:16.234 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.014741+0000", 2024-09-14T06:31:16.234 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.4763000000000001e-05 2024-09-14T06:31:16.234 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:16.235 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:31:16.235 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:16.235 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:16.235 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:16.235 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:115199 3.1 3:856be886:::rbd_data.112d4d24c558.0000000000000004:29 [read 2998272~16384 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e96)", 2024-09-14T06:31:16.235 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:30:17.940040+0000", 2024-09-14T06:31:16.235 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 58.264083235999998, 2024-09-14T06:31:16.235 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.074957134999999994, 2024-09-14T06:31:16.235 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:31:16.235 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:31:16.236 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:31:16.236 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:31:16.236 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:31:16.236 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:16.236 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 115199 2024-09-14T06:31:16.236 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:16.236 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:31:16.236 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:16.236 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:31:16.236 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940040+0000", 2024-09-14T06:31:16.236 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:16.236 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:16.237 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:16.237 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:31:16.237 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940040+0000", 2024-09-14T06:31:16.237 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:16.237 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:16.237 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:16.237 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:31:16.237 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940041+0000", 2024-09-14T06:31:16.237 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.4000000000000001e-07 2024-09-14T06:31:16.237 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:16.238 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:16.238 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:31:16.238 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940046+0000", 2024-09-14T06:31:16.238 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.7250000000000002e-06 2024-09-14T06:31:16.238 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:16.238 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:16.238 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:31:16.238 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940053+0000", 2024-09-14T06:31:16.238 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.6109999999999999e-06 2024-09-14T06:31:16.238 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:16.238 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:16.238 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:31:16.238 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940058+0000", 2024-09-14T06:31:16.238 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.1590000000000001e-06 2024-09-14T06:31:16.239 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:16.239 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:16.239 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:31:16.239 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.014768+0000", 2024-09-14T06:31:16.239 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.074709624000000002 2024-09-14T06:31:16.239 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:16.239 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:16.239 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:31:16.239 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.014907+0000", 2024-09-14T06:31:16.239 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000139665 2024-09-14T06:31:16.240 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:16.240 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:16.240 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:31:16.240 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.014997+0000", 2024-09-14T06:31:16.240 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.9710999999999995e-05 2024-09-14T06:31:16.240 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:16.240 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:31:16.240 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:16.240 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:16.240 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:16.240 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:115200 3.1 3:856be886:::rbd_data.112d4d24c558.0000000000000004:29 [read 3244032~16384 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e96)", 2024-09-14T06:31:16.240 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:30:17.940080+0000", 2024-09-14T06:31:16.240 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 58.264043031, 2024-09-14T06:31:16.241 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.075137291999999994, 2024-09-14T06:31:16.241 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:31:16.241 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:31:16.241 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:31:16.241 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:31:16.241 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:31:16.241 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:16.241 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 115200 2024-09-14T06:31:16.241 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:16.241 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:31:16.241 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:16.242 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:31:16.242 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940080+0000", 2024-09-14T06:31:16.242 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:16.242 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:16.242 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:16.242 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:31:16.242 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940080+0000", 2024-09-14T06:31:16.242 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:16.242 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:16.242 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:16.242 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:31:16.242 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940081+0000", 2024-09-14T06:31:16.243 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.0800000000000004e-07 2024-09-14T06:31:16.243 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:16.243 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:16.243 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:31:16.243 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940083+0000", 2024-09-14T06:31:16.243 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.3920000000000001e-06 2024-09-14T06:31:16.243 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:16.243 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:16.243 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:31:16.244 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940091+0000", 2024-09-14T06:31:16.244 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.1389999999999995e-06 2024-09-14T06:31:16.244 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:16.244 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:16.244 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:31:16.244 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940097+0000", 2024-09-14T06:31:16.244 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.5550000000000002e-06 2024-09-14T06:31:16.244 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:16.244 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:16.244 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:31:16.244 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.015025+0000", 2024-09-14T06:31:16.245 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.074928579999999995 2024-09-14T06:31:16.245 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:16.245 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:16.245 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:31:16.245 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.015096+0000", 2024-09-14T06:31:16.245 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.0705999999999997e-05 2024-09-14T06:31:16.245 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:16.245 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:16.245 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:31:16.246 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.015217+0000", 2024-09-14T06:31:16.246 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00012131200000000001 2024-09-14T06:31:16.246 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:16.246 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:31:16.246 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:16.246 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:16.246 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:16.246 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:115201 3.1 3:856be886:::rbd_data.112d4d24c558.0000000000000004:29 [read 3489792~16384 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e96)", 2024-09-14T06:31:16.247 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:30:17.940113+0000", 2024-09-14T06:31:16.247 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 58.264010257999999, 2024-09-14T06:31:16.247 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.075246899000000006, 2024-09-14T06:31:16.247 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:31:16.247 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:31:16.247 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:31:16.247 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:31:16.247 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:31:16.247 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:16.248 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 115201 2024-09-14T06:31:16.248 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:16.248 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:31:16.248 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:16.248 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:31:16.248 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940113+0000", 2024-09-14T06:31:16.248 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:16.248 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:16.248 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:16.249 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:31:16.249 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940113+0000", 2024-09-14T06:31:16.249 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:16.249 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:16.249 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:16.249 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:31:16.249 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940114+0000", 2024-09-14T06:31:16.249 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.1500000000000004e-07 2024-09-14T06:31:16.249 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:16.249 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:16.249 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:31:16.249 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940116+0000", 2024-09-14T06:31:16.250 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.2809999999999998e-06 2024-09-14T06:31:16.250 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:16.250 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:16.250 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:31:16.250 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940123+0000", 2024-09-14T06:31:16.250 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.9229999999999996e-06 2024-09-14T06:31:16.250 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:16.250 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:16.250 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:31:16.250 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940128+0000", 2024-09-14T06:31:16.250 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.7790000000000003e-06 2024-09-14T06:31:16.250 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:16.250 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:16.251 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:31:16.251 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.015253+0000", 2024-09-14T06:31:16.251 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.075125792999999996 2024-09-14T06:31:16.251 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:16.251 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:16.251 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:31:16.251 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.015297+0000", 2024-09-14T06:31:16.251 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.3847000000000002e-05 2024-09-14T06:31:16.251 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:16.251 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:16.251 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:31:16.251 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.015360+0000", 2024-09-14T06:31:16.251 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.2661000000000002e-05 2024-09-14T06:31:16.252 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:16.252 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:31:16.252 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:16.252 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:16.252 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:16.252 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:115202 3.1 3:856be886:::rbd_data.112d4d24c558.0000000000000004:29 [read 3735552~16384 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e96)", 2024-09-14T06:31:16.252 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:30:17.940135+0000", 2024-09-14T06:31:16.252 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 58.263988478999998, 2024-09-14T06:31:16.252 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.075451105000000004, 2024-09-14T06:31:16.252 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:31:16.252 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:31:16.252 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:31:16.252 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:31:16.252 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:31:16.253 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:16.253 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 115202 2024-09-14T06:31:16.253 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:16.253 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:31:16.253 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:16.253 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:31:16.253 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940135+0000", 2024-09-14T06:31:16.253 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:16.253 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:16.253 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:16.253 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:31:16.253 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940135+0000", 2024-09-14T06:31:16.253 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:16.253 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:16.254 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:16.254 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:31:16.254 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940135+0000", 2024-09-14T06:31:16.254 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.0999999999999999e-07 2024-09-14T06:31:16.254 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:16.254 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:16.254 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:31:16.254 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940137+0000", 2024-09-14T06:31:16.254 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.0020000000000001e-06 2024-09-14T06:31:16.254 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:16.254 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:16.254 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:31:16.254 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940143+0000", 2024-09-14T06:31:16.255 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.9850000000000003e-06 2024-09-14T06:31:16.255 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:16.255 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:16.255 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:31:16.255 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940148+0000", 2024-09-14T06:31:16.255 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.9880000000000004e-06 2024-09-14T06:31:16.255 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:16.255 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:16.255 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:31:16.255 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.015400+0000", 2024-09-14T06:31:16.255 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.075252186999999998 2024-09-14T06:31:16.255 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:16.255 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:16.256 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:31:16.256 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.015481+0000", 2024-09-14T06:31:16.256 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.0698000000000006e-05 2024-09-14T06:31:16.256 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:16.256 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:16.256 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:31:16.256 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.015586+0000", 2024-09-14T06:31:16.256 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00010483499999999999 2024-09-14T06:31:16.256 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:16.256 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:31:16.256 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:16.256 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:16.256 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:16.256 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:115203 3.1 3:856be886:::rbd_data.112d4d24c558.0000000000000004:29 [read 3981312~16384 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e96)", 2024-09-14T06:31:16.256 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:30:17.940156+0000", 2024-09-14T06:31:16.257 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 58.263967254000001, 2024-09-14T06:31:16.257 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.075620860999999998, 2024-09-14T06:31:16.257 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:31:16.257 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:31:16.257 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:31:16.257 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:31:16.257 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:31:16.257 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:16.257 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 115203 2024-09-14T06:31:16.257 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:16.257 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:31:16.257 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:16.257 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:31:16.258 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940156+0000", 2024-09-14T06:31:16.258 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:16.258 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:16.258 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:16.258 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:31:16.258 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940156+0000", 2024-09-14T06:31:16.258 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:16.258 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:16.258 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:16.258 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:31:16.258 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940156+0000", 2024-09-14T06:31:16.258 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.1500000000000005e-07 2024-09-14T06:31:16.258 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:16.259 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:16.259 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:31:16.259 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940158+0000", 2024-09-14T06:31:16.259 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.967e-06 2024-09-14T06:31:16.259 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:16.259 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:16.259 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:31:16.259 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940165+0000", 2024-09-14T06:31:16.259 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.6939999999999998e-06 2024-09-14T06:31:16.259 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:16.259 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:16.259 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:31:16.259 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940170+0000", 2024-09-14T06:31:16.260 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.7539999999999997e-06 2024-09-14T06:31:16.260 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:16.260 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:16.260 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:31:16.260 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.015633+0000", 2024-09-14T06:31:16.260 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.075463334000000007 2024-09-14T06:31:16.260 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:16.260 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:16.260 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:31:16.260 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.015693+0000", 2024-09-14T06:31:16.260 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.9367000000000003e-05 2024-09-14T06:31:16.261 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:16.261 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:16.261 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:31:16.261 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.015777+0000", 2024-09-14T06:31:16.261 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.4229999999999998e-05 2024-09-14T06:31:16.261 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:16.261 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:31:16.261 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:16.261 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:16.261 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:16.261 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:115204 3.1 3:856be886:::rbd_data.112d4d24c558.0000000000000004:29 [read 4227072~16384 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e96)", 2024-09-14T06:31:16.262 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:30:17.940176+0000", 2024-09-14T06:31:16.262 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 58.263946721000003, 2024-09-14T06:31:16.262 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.075785505000000003, 2024-09-14T06:31:16.262 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:31:16.262 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:31:16.262 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:31:16.262 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:31:16.262 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:31:16.262 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:16.262 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 115204 2024-09-14T06:31:16.262 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:16.262 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:31:16.262 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:16.262 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:31:16.263 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940176+0000", 2024-09-14T06:31:16.263 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:16.263 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:16.263 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:16.263 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:31:16.263 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940176+0000", 2024-09-14T06:31:16.263 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:16.263 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:16.263 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:16.263 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:31:16.263 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940177+0000", 2024-09-14T06:31:16.263 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.7899999999999999e-07 2024-09-14T06:31:16.263 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:16.264 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:16.264 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:31:16.264 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940179+0000", 2024-09-14T06:31:16.264 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.103e-06 2024-09-14T06:31:16.264 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:16.264 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:16.264 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:31:16.264 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940185+0000", 2024-09-14T06:31:16.264 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.7239999999999997e-06 2024-09-14T06:31:16.264 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:16.264 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:16.264 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:31:16.265 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940190+0000", 2024-09-14T06:31:16.265 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.7269999999999998e-06 2024-09-14T06:31:16.265 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:16.265 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:16.265 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:31:16.265 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.015814+0000", 2024-09-14T06:31:16.265 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.075624114000000006 2024-09-14T06:31:16.265 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:16.265 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:16.265 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:31:16.265 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.015862+0000", 2024-09-14T06:31:16.265 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.8770999999999998e-05 2024-09-14T06:31:16.265 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:16.266 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:16.266 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:31:16.266 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.015962+0000", 2024-09-14T06:31:16.266 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.9586999999999994e-05 2024-09-14T06:31:16.266 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:16.266 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:31:16.266 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:16.266 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:16.266 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:16.266 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:115205 3.1 3:856be886:::rbd_data.112d4d24c558.0000000000000004:29 [read 4472832~16384 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e96)", 2024-09-14T06:31:16.266 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:30:17.940197+0000", 2024-09-14T06:31:16.266 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 58.263926011999999, 2024-09-14T06:31:16.266 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.076068920999999998, 2024-09-14T06:31:16.266 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:31:16.266 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:31:16.267 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:31:16.267 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:31:16.267 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:31:16.267 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:16.267 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 115205 2024-09-14T06:31:16.267 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:16.267 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:31:16.267 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:16.267 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:31:16.267 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940197+0000", 2024-09-14T06:31:16.267 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:16.267 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:16.267 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:16.267 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:31:16.268 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940197+0000", 2024-09-14T06:31:16.268 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:16.268 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:16.268 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:16.268 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:31:16.268 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940198+0000", 2024-09-14T06:31:16.268 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.5999999999999999e-07 2024-09-14T06:31:16.268 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:16.268 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:16.268 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:31:16.268 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940200+0000", 2024-09-14T06:31:16.268 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.0899999999999999e-06 2024-09-14T06:31:16.269 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:16.269 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:16.269 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:31:16.269 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940206+0000", 2024-09-14T06:31:16.269 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.8710000000000002e-06 2024-09-14T06:31:16.269 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:16.269 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:16.269 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:31:16.269 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940211+0000", 2024-09-14T06:31:16.269 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.9230000000000001e-06 2024-09-14T06:31:16.269 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:16.269 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:16.270 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:31:16.270 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.016059+0000", 2024-09-14T06:31:16.270 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.075848051999999999 2024-09-14T06:31:16.270 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:16.270 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:16.270 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:31:16.270 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.016151+0000", 2024-09-14T06:31:16.270 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.1982000000000001e-05 2024-09-14T06:31:16.270 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:16.270 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:16.270 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:31:16.270 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.016266+0000", 2024-09-14T06:31:16.270 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000115543 2024-09-14T06:31:16.271 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:16.271 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:31:16.271 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:16.271 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:16.271 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:16.271 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:115206 3.1 3:856be886:::rbd_data.112d4d24c558.0000000000000004:29 [read 4718592~16384 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e96)", 2024-09-14T06:31:16.271 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:30:17.940223+0000", 2024-09-14T06:31:16.271 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 58.263899922, 2024-09-14T06:31:16.271 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.076265115999999994, 2024-09-14T06:31:16.271 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:31:16.271 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:31:16.271 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:31:16.271 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:31:16.271 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:31:16.271 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:16.272 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 115206 2024-09-14T06:31:16.272 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:16.272 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:31:16.272 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:16.272 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:31:16.272 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940223+0000", 2024-09-14T06:31:16.272 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:16.272 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:16.272 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:16.272 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:31:16.272 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940223+0000", 2024-09-14T06:31:16.272 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:16.272 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:16.272 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:16.273 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:31:16.273 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940224+0000", 2024-09-14T06:31:16.273 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.0399999999999996e-07 2024-09-14T06:31:16.273 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:16.273 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:16.273 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:31:16.273 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940226+0000", 2024-09-14T06:31:16.273 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.1780000000000002e-06 2024-09-14T06:31:16.273 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:16.273 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:16.273 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:31:16.273 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940232+0000", 2024-09-14T06:31:16.273 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.5010000000000001e-06 2024-09-14T06:31:16.274 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:16.274 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:16.274 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:31:16.274 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940237+0000", 2024-09-14T06:31:16.274 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.0250000000000002e-06 2024-09-14T06:31:16.274 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:16.274 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:16.274 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:31:16.274 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.016305+0000", 2024-09-14T06:31:16.274 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.076067262999999996 2024-09-14T06:31:16.274 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:16.274 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:16.275 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:31:16.275 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.016383+0000", 2024-09-14T06:31:16.275 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.8578000000000006e-05 2024-09-14T06:31:16.275 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:16.275 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:16.275 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:31:16.275 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.016488+0000", 2024-09-14T06:31:16.275 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000105067 2024-09-14T06:31:16.275 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:16.275 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:31:16.275 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:16.275 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:16.275 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:16.275 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:115207 3.1 3:856be886:::rbd_data.112d4d24c558.0000000000000004:29 [read 4964352~16384 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e96)", 2024-09-14T06:31:16.276 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:30:17.940244+0000", 2024-09-14T06:31:16.276 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 58.263878806000001, 2024-09-14T06:31:16.276 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.076371154999999996, 2024-09-14T06:31:16.276 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:31:16.276 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:31:16.276 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:31:16.276 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:31:16.276 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:31:16.276 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:16.276 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 115207 2024-09-14T06:31:16.276 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:16.277 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:31:16.277 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:16.277 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:31:16.277 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940244+0000", 2024-09-14T06:31:16.277 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:16.277 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:16.277 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:16.277 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:31:16.277 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940244+0000", 2024-09-14T06:31:16.277 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:16.277 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:16.278 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:16.278 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:31:16.278 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940245+0000", 2024-09-14T06:31:16.278 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.1200000000000003e-07 2024-09-14T06:31:16.278 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:16.278 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:16.278 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:31:16.278 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940247+0000", 2024-09-14T06:31:16.278 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.0030000000000001e-06 2024-09-14T06:31:16.278 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:16.278 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:16.278 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:31:16.278 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940252+0000", 2024-09-14T06:31:16.279 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.5520000000000004e-06 2024-09-14T06:31:16.279 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:16.279 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:16.279 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:31:16.279 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940257+0000", 2024-09-14T06:31:16.279 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.6759999999999998e-06 2024-09-14T06:31:16.279 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:16.279 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:16.279 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:31:16.280 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.016521+0000", 2024-09-14T06:31:16.280 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.076263499999999998 2024-09-14T06:31:16.280 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:16.280 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:16.280 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:31:16.280 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.016562+0000", 2024-09-14T06:31:16.280 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.0967000000000003e-05 2024-09-14T06:31:16.280 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:16.280 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:16.280 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:31:16.281 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.016616+0000", 2024-09-14T06:31:16.281 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.3944999999999999e-05 2024-09-14T06:31:16.281 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:16.281 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:31:16.281 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:16.281 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:16.281 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:16.281 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:115208 3.1 3:856be886:::rbd_data.112d4d24c558.0000000000000004:29 [read 5210112~16384 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e96)", 2024-09-14T06:31:16.281 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:30:17.940264+0000", 2024-09-14T06:31:16.281 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 58.263858949000003, 2024-09-14T06:31:16.281 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.076558288000000002, 2024-09-14T06:31:16.281 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:31:16.282 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:31:16.282 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:31:16.282 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:31:16.282 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:31:16.282 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:16.282 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 115208 2024-09-14T06:31:16.282 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:16.282 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:31:16.282 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:16.282 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:31:16.282 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940264+0000", 2024-09-14T06:31:16.283 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:16.283 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:16.283 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:16.283 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:31:16.283 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940264+0000", 2024-09-14T06:31:16.283 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:16.283 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:16.283 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:16.283 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:31:16.283 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940265+0000", 2024-09-14T06:31:16.283 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.0500000000000004e-07 2024-09-14T06:31:16.284 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:16.284 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:16.284 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:31:16.284 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940267+0000", 2024-09-14T06:31:16.284 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.9810000000000002e-06 2024-09-14T06:31:16.284 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:16.284 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:16.284 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:31:16.284 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940272+0000", 2024-09-14T06:31:16.284 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.6999999999999996e-06 2024-09-14T06:31:16.284 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:16.285 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:16.285 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:31:16.285 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940277+0000", 2024-09-14T06:31:16.285 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.6480000000000002e-06 2024-09-14T06:31:16.285 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:16.285 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:16.285 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:31:16.285 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.016650+0000", 2024-09-14T06:31:16.285 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.076373089000000005 2024-09-14T06:31:16.286 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:16.286 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:16.286 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:31:16.286 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.016724+0000", 2024-09-14T06:31:16.286 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.4105999999999998e-05 2024-09-14T06:31:16.286 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:16.286 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:16.286 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:31:16.286 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.016823+0000", 2024-09-14T06:31:16.286 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.8258999999999996e-05 2024-09-14T06:31:16.286 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:16.287 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:31:16.287 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:16.287 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:16.287 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:16.287 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:115209 3.1 3:856be886:::rbd_data.112d4d24c558.0000000000000004:29 [read 5455872~16384 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e96)", 2024-09-14T06:31:16.287 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:30:17.940283+0000", 2024-09-14T06:31:16.287 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 58.263839922999999, 2024-09-14T06:31:16.287 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.076724029999999999, 2024-09-14T06:31:16.287 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:31:16.287 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:31:16.287 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:31:16.287 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:31:16.288 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:31:16.288 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:16.288 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 115209 2024-09-14T06:31:16.288 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:16.288 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:31:16.288 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:16.288 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:31:16.288 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940283+0000", 2024-09-14T06:31:16.288 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:16.288 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:16.288 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:16.288 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:31:16.288 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940283+0000", 2024-09-14T06:31:16.288 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:16.289 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:16.289 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:16.289 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:31:16.289 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940284+0000", 2024-09-14T06:31:16.289 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.6700000000000003e-07 2024-09-14T06:31:16.289 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:16.289 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:16.289 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:31:16.289 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940286+0000", 2024-09-14T06:31:16.289 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.9520000000000001e-06 2024-09-14T06:31:16.289 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:16.289 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:16.289 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:31:16.290 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940292+0000", 2024-09-14T06:31:16.290 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.7339999999999996e-06 2024-09-14T06:31:16.290 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:16.290 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:16.290 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:31:16.290 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940296+0000", 2024-09-14T06:31:16.290 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.7700000000000001e-06 2024-09-14T06:31:16.290 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:16.290 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:16.290 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:31:16.290 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.016859+0000", 2024-09-14T06:31:16.290 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.07656317 2024-09-14T06:31:16.290 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:16.290 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:16.290 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:31:16.291 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.016929+0000", 2024-09-14T06:31:16.291 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.9679999999999997e-05 2024-09-14T06:31:16.291 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:16.291 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:16.291 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:31:16.291 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.017007+0000", 2024-09-14T06:31:16.291 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.8157000000000004e-05 2024-09-14T06:31:16.291 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:16.291 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:31:16.291 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:16.291 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:16.291 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:16.291 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:115210 3.1 3:856be886:::rbd_data.112d4d24c558.0000000000000004:29 [read 5701632~16384 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e96)", 2024-09-14T06:31:16.291 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:30:17.940303+0000", 2024-09-14T06:31:16.292 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 58.263820406000001, 2024-09-14T06:31:16.292 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.076821358000000006, 2024-09-14T06:31:16.292 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:31:16.292 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:31:16.292 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:31:16.292 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:31:16.292 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:31:16.292 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:16.292 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 115210 2024-09-14T06:31:16.292 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:16.292 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:31:16.292 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:16.292 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:31:16.293 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940303+0000", 2024-09-14T06:31:16.293 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:16.293 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:16.293 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:16.293 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:31:16.293 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940303+0000", 2024-09-14T06:31:16.293 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:16.293 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:16.293 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:16.293 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:31:16.293 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940303+0000", 2024-09-14T06:31:16.293 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.3000000000000001e-07 2024-09-14T06:31:16.293 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:16.293 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:16.294 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:31:16.294 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940306+0000", 2024-09-14T06:31:16.294 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.255e-06 2024-09-14T06:31:16.294 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:16.294 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:16.294 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:31:16.294 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940312+0000", 2024-09-14T06:31:16.294 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.9710000000000001e-06 2024-09-14T06:31:16.294 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:16.294 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:16.294 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:31:16.294 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940316+0000", 2024-09-14T06:31:16.294 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.5340000000000001e-06 2024-09-14T06:31:16.294 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:16.295 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:16.295 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:31:16.295 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.017033+0000", 2024-09-14T06:31:16.295 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.076717169000000002 2024-09-14T06:31:16.295 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:16.295 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:16.295 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:31:16.295 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.017073+0000", 2024-09-14T06:31:16.295 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.9582000000000002e-05 2024-09-14T06:31:16.295 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:16.295 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:16.295 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:31:16.295 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.017124+0000", 2024-09-14T06:31:16.295 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.1316999999999997e-05 2024-09-14T06:31:16.296 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:16.296 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:31:16.296 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:16.296 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:16.296 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:16.296 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:115219 3.1 3:856be886:::rbd_data.112d4d24c558.0000000000000004:29 [read 65536~16384 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e96)", 2024-09-14T06:31:16.296 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:30:17.940323+0000", 2024-09-14T06:31:16.296 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 58.263800545000002, 2024-09-14T06:31:16.296 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.076915205, 2024-09-14T06:31:16.296 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:31:16.296 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:31:16.296 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:31:16.296 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:31:16.297 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:31:16.297 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:16.297 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 115219 2024-09-14T06:31:16.297 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:16.297 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:31:16.297 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:16.297 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:31:16.297 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940323+0000", 2024-09-14T06:31:16.297 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:16.297 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:16.297 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:16.297 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:31:16.297 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940323+0000", 2024-09-14T06:31:16.297 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:16.298 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:16.298 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:16.298 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:31:16.298 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940323+0000", 2024-09-14T06:31:16.298 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.0800000000000005e-07 2024-09-14T06:31:16.298 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:16.298 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:16.298 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:31:16.298 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940328+0000", 2024-09-14T06:31:16.298 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.887e-06 2024-09-14T06:31:16.298 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:16.298 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:16.298 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:31:16.299 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940334+0000", 2024-09-14T06:31:16.299 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.3269999999999997e-06 2024-09-14T06:31:16.299 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:16.299 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:16.299 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:31:16.299 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940339+0000", 2024-09-14T06:31:16.299 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.7960000000000003e-06 2024-09-14T06:31:16.299 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:16.299 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:16.299 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:31:16.299 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.017148+0000", 2024-09-14T06:31:16.299 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.076809125000000006 2024-09-14T06:31:16.299 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:16.300 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:16.300 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:31:16.300 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.017186+0000", 2024-09-14T06:31:16.300 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.7283000000000002e-05 2024-09-14T06:31:16.300 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:16.300 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:16.300 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:31:16.300 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.017238+0000", 2024-09-14T06:31:16.300 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.2278999999999999e-05 2024-09-14T06:31:16.300 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:16.300 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:31:16.300 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:16.300 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:16.300 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:16.301 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:115220 3.1 3:856be886:::rbd_data.112d4d24c558.0000000000000004:29 [read 311296~16384 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e96)", 2024-09-14T06:31:16.301 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:30:17.940346+0000", 2024-09-14T06:31:16.301 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 58.263777416000003, 2024-09-14T06:31:16.301 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.077033293000000003, 2024-09-14T06:31:16.301 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:31:16.301 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:31:16.301 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:31:16.301 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:31:16.301 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:31:16.301 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:16.301 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 115220 2024-09-14T06:31:16.301 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:16.301 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:31:16.302 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:16.302 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:31:16.302 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940346+0000", 2024-09-14T06:31:16.302 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:16.302 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:16.302 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:16.302 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:31:16.302 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940346+0000", 2024-09-14T06:31:16.302 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:16.302 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:16.302 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:16.302 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:31:16.302 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940346+0000", 2024-09-14T06:31:16.302 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.4300000000000003e-07 2024-09-14T06:31:16.303 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:16.303 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:16.303 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:31:16.303 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940348+0000", 2024-09-14T06:31:16.303 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.0030000000000001e-06 2024-09-14T06:31:16.303 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:16.303 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:16.303 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:31:16.303 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.004655+0000", 2024-09-14T06:31:16.303 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.064306323999999998 2024-09-14T06:31:16.303 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:16.303 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:16.303 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:31:16.304 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.004671+0000", 2024-09-14T06:31:16.304 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.6589999999999999e-05 2024-09-14T06:31:16.304 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:16.304 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:16.304 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:31:16.304 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.017262+0000", 2024-09-14T06:31:16.304 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.012590570000000001 2024-09-14T06:31:16.304 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:16.304 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:16.304 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:31:16.304 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.017321+0000", 2024-09-14T06:31:16.304 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.9210999999999999e-05 2024-09-14T06:31:16.304 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:16.305 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:16.305 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:31:16.305 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.017379+0000", 2024-09-14T06:31:16.305 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.8051999999999999e-05 2024-09-14T06:31:16.305 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:16.305 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:31:16.305 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:16.305 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:16.305 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:31:16.305 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-14T06:31:16.307 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-14T06:31:16.307 INFO:tasks.ceph.osd.2.smithi094.stderr:2024-09-14T06:31:16.306+0000 7faf3269b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33649) UID: 0 2024-09-14T06:31:16.407 INFO:tasks.ceph.osd.4.smithi138.stderr:2024-09-14T06:31:16.406+0000 7f6c653c8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33615) UID: 0 2024-09-14T06:31:16.450 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-14T06:31:16.451 INFO:teuthology.orchestra.run.smithi094.stdout: "ops": [], 2024-09-14T06:31:16.451 INFO:teuthology.orchestra.run.smithi094.stdout: "num_ops": 0 2024-09-14T06:31:16.451 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-14T06:31:16.459 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-14T06:31:16.508 INFO:tasks.ceph.osd.2.smithi094.stderr:2024-09-14T06:31:16.507+0000 7faf3269b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33649) UID: 0 2024-09-14T06:31:16.562 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-14T06:31:16.562 INFO:teuthology.orchestra.run.smithi094.stdout: "ops": [], 2024-09-14T06:31:16.562 INFO:teuthology.orchestra.run.smithi094.stdout: "complaint_time": 30, 2024-09-14T06:31:16.562 INFO:teuthology.orchestra.run.smithi094.stdout: "num_blocked_ops": 0 2024-09-14T06:31:16.562 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-14T06:31:16.571 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-14T06:31:16.609 INFO:tasks.ceph.osd.7.smithi138.stderr:2024-09-14T06:31:16.608+0000 7f3c6a21d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33623) UID: 0 2024-09-14T06:31:16.709 INFO:tasks.ceph.osd.6.smithi138.stderr:2024-09-14T06:31:16.709+0000 7f8cd8849640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33620) UID: 0 2024-09-14T06:31:16.716 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-14T06:31:16.716 INFO:teuthology.orchestra.run.smithi094.stdout: "size": 20, 2024-09-14T06:31:16.716 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 600, 2024-09-14T06:31:16.716 INFO:teuthology.orchestra.run.smithi094.stdout: "ops": [ 2024-09-14T06:31:16.716 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:16.716 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:14918 3.0 3:05c39535:::rbd_data.112d92027590.0000000000000009:f [read 795648~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:31:16.716 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:26.083834+0000", 2024-09-14T06:31:16.716 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 110.630249241, 2024-09-14T06:31:16.716 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.041776287000000002, 2024-09-14T06:31:16.717 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:31:16.717 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:31:16.717 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:31:16.717 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:31:16.717 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:31:16.717 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:16.717 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 14918 2024-09-14T06:31:16.717 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:16.718 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:31:16.718 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:16.718 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:31:16.718 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.083834+0000", 2024-09-14T06:31:16.718 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:16.718 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:16.718 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:16.718 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:31:16.719 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.083834+0000", 2024-09-14T06:31:16.719 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:16.719 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:16.719 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:16.719 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:31:16.719 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.083834+0000", 2024-09-14T06:31:16.719 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.0100000000000001e-07 2024-09-14T06:31:16.719 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:16.719 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:16.719 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:31:16.719 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.083835+0000", 2024-09-14T06:31:16.719 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.3900000000000003e-07 2024-09-14T06:31:16.720 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:16.720 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:16.720 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:31:16.720 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.083838+0000", 2024-09-14T06:31:16.720 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.9830000000000001e-06 2024-09-14T06:31:16.720 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:16.720 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:16.720 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:31:16.720 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.083840+0000", 2024-09-14T06:31:16.720 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.4679999999999999e-06 2024-09-14T06:31:16.720 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:16.720 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:16.720 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:31:16.721 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.125455+0000", 2024-09-14T06:31:16.721 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.041614578999999999 2024-09-14T06:31:16.721 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:16.721 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:16.721 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:31:16.721 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.125496+0000", 2024-09-14T06:31:16.721 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.1047000000000002e-05 2024-09-14T06:31:16.721 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:16.721 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:16.721 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:31:16.721 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.125610+0000", 2024-09-14T06:31:16.721 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00011407 2024-09-14T06:31:16.721 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:16.721 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:31:16.722 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:16.722 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:16.722 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:16.722 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:14919 3.0 3:05c39535:::rbd_data.112d92027590.0000000000000009:f [read 805888~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:31:16.722 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:26.083843+0000", 2024-09-14T06:31:16.722 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 110.630239493, 2024-09-14T06:31:16.722 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.041992356000000002, 2024-09-14T06:31:16.722 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:31:16.722 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:31:16.722 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:31:16.722 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:31:16.722 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:31:16.722 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:16.722 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 14919 2024-09-14T06:31:16.723 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:16.723 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:31:16.723 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:16.723 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:31:16.723 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.083843+0000", 2024-09-14T06:31:16.723 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:16.723 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:16.723 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:16.723 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:31:16.723 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.083843+0000", 2024-09-14T06:31:16.723 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:16.723 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:16.723 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:16.723 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:31:16.724 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.083844+0000", 2024-09-14T06:31:16.724 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.3300000000000001e-07 2024-09-14T06:31:16.724 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:16.724 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:16.724 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:31:16.724 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.083845+0000", 2024-09-14T06:31:16.724 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.4729999999999999e-06 2024-09-14T06:31:16.724 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:16.724 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:16.724 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:31:16.724 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.083850+0000", 2024-09-14T06:31:16.724 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.0130000000000001e-06 2024-09-14T06:31:16.724 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:16.724 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:16.725 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:31:16.725 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.083854+0000", 2024-09-14T06:31:16.725 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.4759999999999998e-06 2024-09-14T06:31:16.725 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:16.725 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:16.725 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:31:16.725 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.125636+0000", 2024-09-14T06:31:16.725 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.041782683000000001 2024-09-14T06:31:16.725 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:16.725 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:16.725 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:31:16.725 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.125680+0000", 2024-09-14T06:31:16.725 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.3614e-05 2024-09-14T06:31:16.725 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:16.726 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:16.726 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:31:16.726 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.125836+0000", 2024-09-14T06:31:16.726 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000155864 2024-09-14T06:31:16.726 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:16.726 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:31:16.726 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:16.726 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:16.726 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:16.726 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:14920 3.0 3:05c39535:::rbd_data.112d92027590.0000000000000009:f [read 816128~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:31:16.726 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:26.083868+0000", 2024-09-14T06:31:16.726 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 110.63021483199999, 2024-09-14T06:31:16.727 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.042166771999999998, 2024-09-14T06:31:16.727 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:31:16.727 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:31:16.727 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:31:16.727 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:31:16.727 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:31:16.727 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:16.727 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 14920 2024-09-14T06:31:16.727 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:16.727 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:31:16.727 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:16.727 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:31:16.727 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.083868+0000", 2024-09-14T06:31:16.727 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:16.728 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:16.728 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:16.728 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:31:16.728 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.083868+0000", 2024-09-14T06:31:16.728 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:16.728 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:16.728 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:16.728 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:31:16.728 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.083869+0000", 2024-09-14T06:31:16.728 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.99e-07 2024-09-14T06:31:16.728 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:16.728 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:16.728 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:31:16.728 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.083870+0000", 2024-09-14T06:31:16.729 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.782e-06 2024-09-14T06:31:16.729 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:16.729 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:16.729 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:31:16.729 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.083876+0000", 2024-09-14T06:31:16.729 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.3809999999999998e-06 2024-09-14T06:31:16.729 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:16.729 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:16.729 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:31:16.729 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.083882+0000", 2024-09-14T06:31:16.729 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.9479999999999996e-06 2024-09-14T06:31:16.729 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:16.729 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:16.729 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:31:16.730 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.125878+0000", 2024-09-14T06:31:16.730 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.041996264999999998 2024-09-14T06:31:16.730 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:16.730 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:16.730 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:31:16.730 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.125929+0000", 2024-09-14T06:31:16.730 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.0498999999999999e-05 2024-09-14T06:31:16.730 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:16.730 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:16.730 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:31:16.730 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.126035+0000", 2024-09-14T06:31:16.730 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000106298 2024-09-14T06:31:16.730 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:16.730 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:31:16.731 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:16.731 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:16.731 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:16.731 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:14921 3.0 3:05c39535:::rbd_data.112d92027590.0000000000000009:f [read 826368~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:31:16.731 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:26.083889+0000", 2024-09-14T06:31:16.731 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 110.630193822, 2024-09-14T06:31:16.731 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.042435309999999997, 2024-09-14T06:31:16.731 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:31:16.731 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:31:16.731 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:31:16.731 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:31:16.731 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:31:16.731 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:16.732 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 14921 2024-09-14T06:31:16.732 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:16.732 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:31:16.732 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:16.732 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:31:16.732 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.083889+0000", 2024-09-14T06:31:16.732 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:16.732 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:16.732 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:16.732 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:31:16.732 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.083889+0000", 2024-09-14T06:31:16.732 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:16.732 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:16.732 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:16.733 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:31:16.733 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.086033+0000", 2024-09-14T06:31:16.733 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0021444179999999999 2024-09-14T06:31:16.733 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:16.733 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:16.733 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:31:16.733 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.086038+0000", 2024-09-14T06:31:16.733 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.0969999999999999e-06 2024-09-14T06:31:16.733 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:16.733 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:16.733 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:31:16.733 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.086046+0000", 2024-09-14T06:31:16.733 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.9440000000000005e-06 2024-09-14T06:31:16.733 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:16.734 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:16.734 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:31:16.734 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.086050+0000", 2024-09-14T06:31:16.734 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.69e-06 2024-09-14T06:31:16.734 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:16.734 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:16.734 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:31:16.734 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.126080+0000", 2024-09-14T06:31:16.734 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.040030237000000003 2024-09-14T06:31:16.734 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:16.734 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:16.734 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:31:16.734 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.126163+0000", 2024-09-14T06:31:16.734 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.2071999999999996e-05 2024-09-14T06:31:16.734 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:16.735 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:16.735 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:31:16.735 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.126324+0000", 2024-09-14T06:31:16.735 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00016185200000000001 2024-09-14T06:31:16.735 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:16.735 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:31:16.735 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:16.735 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:16.735 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:16.735 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:55627 3.0 3:06490baf:::rbd_data.112dca2f6ae1.0000000000000030:17 [read 2002944~4096 [fadvise_nocache]] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:31:16.735 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:44.599966+0000", 2024-09-14T06:31:16.735 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 92.114116874000004, 2024-09-14T06:31:16.735 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.041868546999999999, 2024-09-14T06:31:16.735 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:31:16.736 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:31:16.736 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:31:16.736 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:31:16.736 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:31:16.736 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:16.736 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 55627 2024-09-14T06:31:16.736 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:16.736 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:31:16.736 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:16.736 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:31:16.736 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.599966+0000", 2024-09-14T06:31:16.736 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:16.737 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:16.737 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:16.737 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:31:16.737 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.599966+0000", 2024-09-14T06:31:16.737 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:16.737 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:16.737 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:16.737 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:31:16.737 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.599966+0000", 2024-09-14T06:31:16.737 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.0900000000000001e-07 2024-09-14T06:31:16.737 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:16.737 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:16.737 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:31:16.737 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.599967+0000", 2024-09-14T06:31:16.738 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.64e-07 2024-09-14T06:31:16.738 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:16.738 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:16.738 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:31:16.738 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.599970+0000", 2024-09-14T06:31:16.738 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.9670000000000001e-06 2024-09-14T06:31:16.738 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:16.738 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:16.738 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:31:16.738 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.599973+0000", 2024-09-14T06:31:16.738 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.632e-06 2024-09-14T06:31:16.738 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:16.738 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:16.738 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:31:16.739 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.641665+0000", 2024-09-14T06:31:16.739 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.041692547000000003 2024-09-14T06:31:16.739 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:16.739 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:16.739 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:31:16.739 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.641740+0000", 2024-09-14T06:31:16.739 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.4559000000000002e-05 2024-09-14T06:31:16.739 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:16.739 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:16.739 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:31:16.739 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.641835+0000", 2024-09-14T06:31:16.739 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.4668999999999998e-05 2024-09-14T06:31:16.739 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:16.739 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:31:16.740 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:16.740 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:16.740 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:16.740 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:55628 3.0 3:06490baf:::rbd_data.112dca2f6ae1.0000000000000030:17 [read 2027520~4096 [fadvise_nocache]] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:31:16.740 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:44.599976+0000", 2024-09-14T06:31:16.740 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 92.114107086999994, 2024-09-14T06:31:16.740 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.041991621, 2024-09-14T06:31:16.740 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:31:16.740 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:31:16.740 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:31:16.740 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:31:16.740 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:31:16.740 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:16.740 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 55628 2024-09-14T06:31:16.741 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:16.741 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:31:16.741 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:16.741 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:31:16.741 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.599976+0000", 2024-09-14T06:31:16.741 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:16.741 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:16.741 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:16.741 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:31:16.741 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.599976+0000", 2024-09-14T06:31:16.741 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:16.741 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:16.741 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:16.742 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:31:16.742 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.599976+0000", 2024-09-14T06:31:16.742 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.1299999999999999e-07 2024-09-14T06:31:16.742 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:16.742 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:16.742 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:31:16.742 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.599977+0000", 2024-09-14T06:31:16.742 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.7999999999999993e-07 2024-09-14T06:31:16.742 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:16.742 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:16.742 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:31:16.742 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.599980+0000", 2024-09-14T06:31:16.742 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.01e-06 2024-09-14T06:31:16.742 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:16.743 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:16.743 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:31:16.743 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.599983+0000", 2024-09-14T06:31:16.743 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.5409999999999999e-06 2024-09-14T06:31:16.743 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:16.743 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:16.743 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:31:16.743 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.641869+0000", 2024-09-14T06:31:16.743 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.041886235000000001 2024-09-14T06:31:16.743 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:16.743 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:16.743 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:31:16.743 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.641913+0000", 2024-09-14T06:31:16.744 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.4357000000000002e-05 2024-09-14T06:31:16.744 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:16.744 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:16.744 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:31:16.744 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.641967+0000", 2024-09-14T06:31:16.744 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.4284999999999999e-05 2024-09-14T06:31:16.744 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:16.744 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:31:16.744 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:16.744 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:16.744 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:16.744 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:55629 3.0 3:06490baf:::rbd_data.112dca2f6ae1.0000000000000030:17 [read 2052096~4096 [fadvise_nocache]] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:31:16.744 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:44.599985+0000", 2024-09-14T06:31:16.744 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 92.114097380999993, 2024-09-14T06:31:16.745 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.042183375000000002, 2024-09-14T06:31:16.745 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:31:16.745 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:31:16.745 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:31:16.745 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:31:16.745 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:31:16.745 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:16.745 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 55629 2024-09-14T06:31:16.745 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:16.745 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:31:16.745 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:16.745 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:31:16.745 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.599985+0000", 2024-09-14T06:31:16.745 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:16.746 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:16.746 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:16.746 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:31:16.746 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.599985+0000", 2024-09-14T06:31:16.746 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:16.746 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:16.746 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:16.746 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:31:16.746 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.599986+0000", 2024-09-14T06:31:16.746 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.05e-07 2024-09-14T06:31:16.747 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:16.747 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:16.747 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:31:16.747 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.599987+0000", 2024-09-14T06:31:16.747 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.8400000000000002e-07 2024-09-14T06:31:16.747 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:16.747 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:16.747 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:31:16.747 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.599990+0000", 2024-09-14T06:31:16.747 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.5640000000000001e-06 2024-09-14T06:31:16.747 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:16.747 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:16.747 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:31:16.748 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.599993+0000", 2024-09-14T06:31:16.748 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.615e-06 2024-09-14T06:31:16.748 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:16.748 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:16.748 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:31:16.748 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.642003+0000", 2024-09-14T06:31:16.748 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.042010345999999997 2024-09-14T06:31:16.748 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:16.748 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:16.748 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:31:16.748 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.642080+0000", 2024-09-14T06:31:16.748 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.6538999999999993e-05 2024-09-14T06:31:16.748 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:16.749 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:16.749 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:31:16.749 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.642169+0000", 2024-09-14T06:31:16.749 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.9122000000000005e-05 2024-09-14T06:31:16.749 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:16.749 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:31:16.749 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:16.749 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:16.749 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:16.749 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:55630 3.0 3:06490baf:::rbd_data.112dca2f6ae1.0000000000000030:17 [read 2076672~4096 [fadvise_nocache]] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:31:16.750 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:44.599996+0000", 2024-09-14T06:31:16.750 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 92.114086903, 2024-09-14T06:31:16.750 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.042299615999999998, 2024-09-14T06:31:16.750 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:31:16.750 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:31:16.750 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:31:16.750 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:31:16.750 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:31:16.750 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:16.750 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 55630 2024-09-14T06:31:16.750 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:16.750 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:31:16.751 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:16.751 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:31:16.751 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.599996+0000", 2024-09-14T06:31:16.751 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:16.751 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:16.751 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:16.751 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:31:16.751 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.599996+0000", 2024-09-14T06:31:16.751 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:16.751 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:16.752 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:16.752 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:31:16.752 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.599996+0000", 2024-09-14T06:31:16.752 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.1500000000000001e-07 2024-09-14T06:31:16.752 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:16.752 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:16.752 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:31:16.752 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.599997+0000", 2024-09-14T06:31:16.752 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.33e-07 2024-09-14T06:31:16.752 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:16.752 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:16.752 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:31:16.753 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600000+0000", 2024-09-14T06:31:16.753 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.0240000000000002e-06 2024-09-14T06:31:16.753 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:16.753 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:16.753 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:31:16.753 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600003+0000", 2024-09-14T06:31:16.753 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.3640000000000002e-06 2024-09-14T06:31:16.753 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:16.753 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:16.753 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:31:16.753 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.642200+0000", 2024-09-14T06:31:16.754 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.042197571000000003 2024-09-14T06:31:16.754 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:16.754 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:16.754 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:31:16.754 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.642243+0000", 2024-09-14T06:31:16.754 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.2765000000000001e-05 2024-09-14T06:31:16.754 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:16.754 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:16.754 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:31:16.755 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.642296+0000", 2024-09-14T06:31:16.755 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.2744000000000002e-05 2024-09-14T06:31:16.755 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:16.755 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:31:16.755 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:16.755 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:16.755 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:16.755 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:55631 3.0 3:06490baf:::rbd_data.112dca2f6ae1.0000000000000030:17 [read 1048576~4096 [fadvise_nocache]] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:31:16.755 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:44.600006+0000", 2024-09-14T06:31:16.755 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 92.114077315000003, 2024-09-14T06:31:16.756 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.042487905999999999, 2024-09-14T06:31:16.756 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:31:16.756 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:31:16.756 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:31:16.756 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:31:16.756 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:31:16.756 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:16.756 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 55631 2024-09-14T06:31:16.756 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:16.756 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:31:16.757 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:16.757 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:31:16.757 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600006+0000", 2024-09-14T06:31:16.757 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:16.757 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:16.757 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:16.757 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:31:16.757 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600006+0000", 2024-09-14T06:31:16.757 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:16.758 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:16.758 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:16.758 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:31:16.758 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600006+0000", 2024-09-14T06:31:16.758 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.0800000000000001e-07 2024-09-14T06:31:16.758 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:16.758 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:16.758 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:31:16.758 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600007+0000", 2024-09-14T06:31:16.758 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.7199999999999997e-07 2024-09-14T06:31:16.759 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:16.759 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:16.759 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:31:16.759 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600010+0000", 2024-09-14T06:31:16.759 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.0280000000000001e-06 2024-09-14T06:31:16.759 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:16.759 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:16.759 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:31:16.759 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600012+0000", 2024-09-14T06:31:16.759 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.475e-06 2024-09-14T06:31:16.759 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:16.759 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:16.759 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:31:16.760 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.642329+0000", 2024-09-14T06:31:16.760 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.042317157000000001 2024-09-14T06:31:16.760 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:16.760 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:16.760 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:31:16.760 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.642403+0000", 2024-09-14T06:31:16.760 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.3941000000000002e-05 2024-09-14T06:31:16.760 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:16.760 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:16.760 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:31:16.760 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.642493+0000", 2024-09-14T06:31:16.760 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.0124999999999995e-05 2024-09-14T06:31:16.761 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:16.761 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:31:16.761 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:16.761 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:16.761 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:16.761 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:55632 3.0 3:06490baf:::rbd_data.112dca2f6ae1.0000000000000030:17 [read 1073152~4096 [fadvise_nocache]] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:31:16.761 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:44.600015+0000", 2024-09-14T06:31:16.761 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 92.114067562000002, 2024-09-14T06:31:16.761 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.042604922000000003, 2024-09-14T06:31:16.761 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:31:16.761 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:31:16.761 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:31:16.761 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:31:16.761 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:31:16.762 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:16.762 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 55632 2024-09-14T06:31:16.762 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:16.762 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:31:16.762 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:16.762 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:31:16.762 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600015+0000", 2024-09-14T06:31:16.762 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:16.762 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:16.762 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:16.762 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:31:16.762 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600015+0000", 2024-09-14T06:31:16.762 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:16.763 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:16.763 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:16.763 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:31:16.763 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600016+0000", 2024-09-14T06:31:16.763 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.1e-07 2024-09-14T06:31:16.763 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:16.763 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:16.763 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:31:16.763 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600017+0000", 2024-09-14T06:31:16.763 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.9900000000000009e-07 2024-09-14T06:31:16.763 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:16.763 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:16.763 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:31:16.763 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600020+0000", 2024-09-14T06:31:16.763 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.4170000000000001e-06 2024-09-14T06:31:16.764 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:16.764 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:16.764 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:31:16.764 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600023+0000", 2024-09-14T06:31:16.764 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.588e-06 2024-09-14T06:31:16.764 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:16.764 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:16.764 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:31:16.764 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.642526+0000", 2024-09-14T06:31:16.764 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.042503236999999999 2024-09-14T06:31:16.764 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:16.764 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:16.764 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:31:16.764 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.642568+0000", 2024-09-14T06:31:16.765 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.1925000000000003e-05 2024-09-14T06:31:16.765 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:16.765 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:16.765 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:31:16.765 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.642620+0000", 2024-09-14T06:31:16.765 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.2546000000000001e-05 2024-09-14T06:31:16.765 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:16.765 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:31:16.765 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:16.765 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:16.765 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:16.765 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:55633 3.0 3:06490baf:::rbd_data.112dca2f6ae1.0000000000000030:17 [read 1097728~4096 [fadvise_nocache]] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:31:16.765 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:44.600026+0000", 2024-09-14T06:31:16.765 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 92.114057293000002, 2024-09-14T06:31:16.766 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.042797474000000002, 2024-09-14T06:31:16.766 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:31:16.766 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:31:16.766 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:31:16.766 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:31:16.766 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:31:16.766 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:16.766 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 55633 2024-09-14T06:31:16.766 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:16.766 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:31:16.766 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:16.766 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:31:16.766 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600026+0000", 2024-09-14T06:31:16.766 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:16.767 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:16.767 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:16.767 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:31:16.767 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600026+0000", 2024-09-14T06:31:16.767 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:16.767 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:16.767 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:16.767 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:31:16.767 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600026+0000", 2024-09-14T06:31:16.767 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.3099999999999999e-07 2024-09-14T06:31:16.767 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:16.767 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:16.767 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:31:16.767 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600029+0000", 2024-09-14T06:31:16.768 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.304e-06 2024-09-14T06:31:16.768 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:16.768 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:16.768 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:31:16.768 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600034+0000", 2024-09-14T06:31:16.768 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.1429999999999997e-06 2024-09-14T06:31:16.768 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:16.768 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:16.768 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:31:16.768 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600038+0000", 2024-09-14T06:31:16.768 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.692e-06 2024-09-14T06:31:16.768 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:16.768 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:16.768 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:31:16.769 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.642654+0000", 2024-09-14T06:31:16.769 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.042615564000000002 2024-09-14T06:31:16.769 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:16.769 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:16.769 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:31:16.769 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.642728+0000", 2024-09-14T06:31:16.769 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.4018999999999997e-05 2024-09-14T06:31:16.769 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:16.769 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:16.769 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:31:16.769 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.642823+0000", 2024-09-14T06:31:16.769 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.5520999999999996e-05 2024-09-14T06:31:16.769 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:16.769 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:31:16.769 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:16.770 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:16.770 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:16.770 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:55634 3.0 3:06490baf:::rbd_data.112dca2f6ae1.0000000000000030:17 [read 1122304~4096 [fadvise_nocache]] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:31:16.770 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:44.600042+0000", 2024-09-14T06:31:16.770 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 92.114040815999999, 2024-09-14T06:31:16.770 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.042903729000000002, 2024-09-14T06:31:16.770 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:31:16.770 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:31:16.770 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:31:16.770 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:31:16.770 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:31:16.770 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:16.770 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 55634 2024-09-14T06:31:16.771 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:16.771 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:31:16.771 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:16.771 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:31:16.771 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600042+0000", 2024-09-14T06:31:16.771 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:16.771 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:16.771 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:16.771 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:31:16.771 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600042+0000", 2024-09-14T06:31:16.771 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:16.771 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:16.771 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:16.771 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:31:16.772 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600043+0000", 2024-09-14T06:31:16.772 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.7000000000000005e-07 2024-09-14T06:31:16.772 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:16.772 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:16.772 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:31:16.772 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600044+0000", 2024-09-14T06:31:16.772 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.742e-06 2024-09-14T06:31:16.772 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:16.772 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:16.772 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:31:16.772 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600050+0000", 2024-09-14T06:31:16.772 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.0800000000000002e-06 2024-09-14T06:31:16.772 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:16.773 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:16.773 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:31:16.773 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600055+0000", 2024-09-14T06:31:16.773 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.3889999999999998e-06 2024-09-14T06:31:16.773 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:16.773 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:16.773 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:31:16.773 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.642845+0000", 2024-09-14T06:31:16.773 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.042790040000000001 2024-09-14T06:31:16.773 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:16.773 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:16.773 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:31:16.773 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.642888+0000", 2024-09-14T06:31:16.773 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.2790000000000002e-05 2024-09-14T06:31:16.773 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:16.774 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:16.774 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:31:16.774 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.642946+0000", 2024-09-14T06:31:16.774 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.8118000000000002e-05 2024-09-14T06:31:16.774 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:16.774 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:31:16.774 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:16.774 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:16.774 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:16.774 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:55635 3.0 3:06490baf:::rbd_data.112dca2f6ae1.0000000000000030:17 [read 1146880~4096 [fadvise_nocache]] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:31:16.774 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:44.600061+0000", 2024-09-14T06:31:16.774 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 92.114022245000001, 2024-09-14T06:31:16.774 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.043092695, 2024-09-14T06:31:16.774 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:31:16.775 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:31:16.775 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:31:16.775 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:31:16.775 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:31:16.775 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:16.775 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 55635 2024-09-14T06:31:16.775 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:16.775 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:31:16.775 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:16.775 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:31:16.775 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600061+0000", 2024-09-14T06:31:16.775 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:16.775 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:16.775 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:16.775 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:31:16.775 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600061+0000", 2024-09-14T06:31:16.776 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:16.776 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:16.776 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:16.776 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:31:16.776 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600061+0000", 2024-09-14T06:31:16.776 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.3099999999999999e-07 2024-09-14T06:31:16.776 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:16.776 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:16.776 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:31:16.776 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600063+0000", 2024-09-14T06:31:16.776 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.6220000000000001e-06 2024-09-14T06:31:16.776 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:16.776 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:16.776 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:31:16.777 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600068+0000", 2024-09-14T06:31:16.777 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.4790000000000004e-06 2024-09-14T06:31:16.777 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:16.777 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:16.777 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:31:16.777 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600072+0000", 2024-09-14T06:31:16.777 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.3100000000000002e-06 2024-09-14T06:31:16.777 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:16.777 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:16.777 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:31:16.777 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.642987+0000", 2024-09-14T06:31:16.777 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.042914489 2024-09-14T06:31:16.777 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:16.777 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:16.778 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:31:16.778 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.643062+0000", 2024-09-14T06:31:16.778 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.5011000000000004e-05 2024-09-14T06:31:16.778 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:16.778 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:16.778 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:31:16.778 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.643153+0000", 2024-09-14T06:31:16.778 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.1452999999999993e-05 2024-09-14T06:31:16.778 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:16.778 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:31:16.778 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:16.778 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:16.778 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:16.778 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:55636 3.0 3:06490baf:::rbd_data.112dca2f6ae1.0000000000000030:17 [read 1171456~4096 [fadvise_nocache]] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:31:16.778 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:44.600078+0000", 2024-09-14T06:31:16.779 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 92.114004832999996, 2024-09-14T06:31:16.779 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.043207966, 2024-09-14T06:31:16.779 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:31:16.779 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:31:16.779 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:31:16.779 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:31:16.779 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:31:16.779 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:16.779 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 55636 2024-09-14T06:31:16.779 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:16.779 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:31:16.779 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:16.779 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:31:16.779 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600078+0000", 2024-09-14T06:31:16.780 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:16.780 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:16.780 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:16.780 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:31:16.780 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600078+0000", 2024-09-14T06:31:16.780 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:16.780 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:16.780 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:16.780 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:31:16.780 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600078+0000", 2024-09-14T06:31:16.780 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.34e-07 2024-09-14T06:31:16.781 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:16.781 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:16.781 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:31:16.781 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600080+0000", 2024-09-14T06:31:16.781 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.641e-06 2024-09-14T06:31:16.781 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:16.781 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:16.781 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:31:16.781 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600086+0000", 2024-09-14T06:31:16.781 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.3860000000000004e-06 2024-09-14T06:31:16.782 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:16.782 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:16.782 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:31:16.782 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600091+0000", 2024-09-14T06:31:16.782 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.5970000000000002e-06 2024-09-14T06:31:16.782 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:16.782 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:16.782 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:31:16.782 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.643184+0000", 2024-09-14T06:31:16.782 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.043092893 2024-09-14T06:31:16.782 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:16.782 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:16.782 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:31:16.782 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.643228+0000", 2024-09-14T06:31:16.783 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.4536999999999997e-05 2024-09-14T06:31:16.783 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:16.783 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:16.783 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:31:16.783 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.643286+0000", 2024-09-14T06:31:16.783 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.7578000000000003e-05 2024-09-14T06:31:16.783 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:16.783 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:31:16.783 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:16.783 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:16.784 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:16.784 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:55637 3.0 3:06490baf:::rbd_data.112dca2f6ae1.0000000000000030:17 [read 1196032~4096 [fadvise_nocache]] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:31:16.784 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:44.600097+0000", 2024-09-14T06:31:16.784 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 92.113986268999994, 2024-09-14T06:31:16.784 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.043399124999999997, 2024-09-14T06:31:16.784 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:31:16.784 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:31:16.784 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:31:16.784 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:31:16.784 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:31:16.784 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:16.784 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 55637 2024-09-14T06:31:16.784 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:16.784 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:31:16.785 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:16.785 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:31:16.785 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600097+0000", 2024-09-14T06:31:16.785 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:16.785 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:16.785 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:16.785 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:31:16.785 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600097+0000", 2024-09-14T06:31:16.785 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:16.786 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:16.786 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:16.786 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:31:16.786 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.604372+0000", 2024-09-14T06:31:16.786 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0042751489999999998 2024-09-14T06:31:16.786 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:16.786 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:16.786 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:31:16.786 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.604377+0000", 2024-09-14T06:31:16.786 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.7899999999999999e-06 2024-09-14T06:31:16.786 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:16.786 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:16.786 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:31:16.787 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.604389+0000", 2024-09-14T06:31:16.787 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.2197e-05 2024-09-14T06:31:16.787 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:16.787 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:16.787 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:31:16.787 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.604397+0000", 2024-09-14T06:31:16.787 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.8730000000000007e-06 2024-09-14T06:31:16.787 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:16.787 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:16.788 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:31:16.788 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.643324+0000", 2024-09-14T06:31:16.788 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.038926897000000002 2024-09-14T06:31:16.788 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:16.788 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:16.788 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:31:16.788 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.643402+0000", 2024-09-14T06:31:16.788 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.8063000000000001e-05 2024-09-14T06:31:16.788 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:16.788 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:16.788 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:31:16.788 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.643496+0000", 2024-09-14T06:31:16.789 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.4155999999999998e-05 2024-09-14T06:31:16.789 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:16.789 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:31:16.789 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:16.789 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:16.789 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:16.789 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:55657 3.0 3:06490baf:::rbd_data.112dca2f6ae1.0000000000000030:17 [read 1687552~4096 [fadvise_nocache]] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:31:16.789 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:44.604841+0000", 2024-09-14T06:31:16.789 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 92.109241539999999, 2024-09-14T06:31:16.789 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.041890087999999999, 2024-09-14T06:31:16.790 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:31:16.790 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:31:16.790 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:31:16.790 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:31:16.790 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:31:16.790 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:16.790 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 55657 2024-09-14T06:31:16.790 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:16.790 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:31:16.791 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:16.791 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:31:16.791 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.604841+0000", 2024-09-14T06:31:16.791 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:16.791 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:16.791 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:16.791 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:31:16.791 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.604841+0000", 2024-09-14T06:31:16.791 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:16.791 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:16.792 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:16.792 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:31:16.792 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.604842+0000", 2024-09-14T06:31:16.792 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.0900000000000001e-07 2024-09-14T06:31:16.792 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:16.792 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:16.792 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:31:16.792 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.604846+0000", 2024-09-14T06:31:16.793 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.5759999999999999e-06 2024-09-14T06:31:16.793 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:16.793 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:16.793 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:31:16.793 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.604850+0000", 2024-09-14T06:31:16.793 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.405e-06 2024-09-14T06:31:16.793 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:16.793 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:16.793 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:31:16.793 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.604852+0000", 2024-09-14T06:31:16.793 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.8590000000000001e-06 2024-09-14T06:31:16.793 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:16.793 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:16.794 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:31:16.794 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.646633+0000", 2024-09-14T06:31:16.794 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.041780772000000001 2024-09-14T06:31:16.794 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:16.794 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:16.794 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:31:16.794 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.646677+0000", 2024-09-14T06:31:16.794 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.4115e-05 2024-09-14T06:31:16.794 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:16.794 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:16.794 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:31:16.794 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.646731+0000", 2024-09-14T06:31:16.795 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.4151999999999999e-05 2024-09-14T06:31:16.795 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:16.795 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:31:16.795 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:16.795 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:16.795 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:16.795 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:55658 3.0 3:06490baf:::rbd_data.112dca2f6ae1.0000000000000030:17 [read 1712128~4096 [fadvise_nocache]] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:31:16.795 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:44.604856+0000", 2024-09-14T06:31:16.795 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 92.109226939999999, 2024-09-14T06:31:16.795 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.041986227000000001, 2024-09-14T06:31:16.795 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:31:16.795 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:31:16.795 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:31:16.795 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:31:16.796 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:31:16.796 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:16.796 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 55658 2024-09-14T06:31:16.796 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:16.796 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:31:16.796 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:16.796 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:31:16.796 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.604856+0000", 2024-09-14T06:31:16.796 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:16.796 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:16.796 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:16.796 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:31:16.796 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.604856+0000", 2024-09-14T06:31:16.797 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:16.797 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:16.797 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:16.797 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:31:16.797 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.604856+0000", 2024-09-14T06:31:16.797 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.23e-07 2024-09-14T06:31:16.797 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:16.797 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:16.797 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:31:16.797 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.604857+0000", 2024-09-14T06:31:16.797 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.9999999999999995e-07 2024-09-14T06:31:16.797 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:16.797 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:16.797 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:31:16.798 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.604861+0000", 2024-09-14T06:31:16.798 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.9890000000000003e-06 2024-09-14T06:31:16.798 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:16.798 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:16.798 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:31:16.798 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.604864+0000", 2024-09-14T06:31:16.798 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.7429999999999998e-06 2024-09-14T06:31:16.798 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:16.798 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:16.798 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:31:16.798 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.646744+0000", 2024-09-14T06:31:16.798 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.041880325000000003 2024-09-14T06:31:16.798 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:16.798 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:16.799 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:31:16.799 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.646784+0000", 2024-09-14T06:31:16.799 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.9604000000000002e-05 2024-09-14T06:31:16.799 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:16.799 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:16.799 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:31:16.799 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.646842+0000", 2024-09-14T06:31:16.799 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.8343e-05 2024-09-14T06:31:16.799 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:16.799 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:31:16.799 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:16.799 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:16.799 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:16.799 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:55659 3.0 3:06490baf:::rbd_data.112dca2f6ae1.0000000000000030:17 [read 1736704~4096 [fadvise_nocache]] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:31:16.800 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:44.604867+0000", 2024-09-14T06:31:16.800 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 92.109215786999997, 2024-09-14T06:31:16.800 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.042097082000000001, 2024-09-14T06:31:16.800 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:31:16.800 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:31:16.800 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:31:16.800 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:31:16.800 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:31:16.800 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:16.800 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 55659 2024-09-14T06:31:16.800 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:16.800 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:31:16.800 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:16.801 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:31:16.801 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.604867+0000", 2024-09-14T06:31:16.801 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:16.801 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:16.801 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:16.801 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:31:16.801 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.604867+0000", 2024-09-14T06:31:16.801 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:16.801 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:16.801 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:16.801 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:31:16.801 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.604867+0000", 2024-09-14T06:31:16.801 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.2000000000000001e-07 2024-09-14T06:31:16.802 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:16.802 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:16.802 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:31:16.802 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.604868+0000", 2024-09-14T06:31:16.802 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.04e-06 2024-09-14T06:31:16.802 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:16.802 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:16.802 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:31:16.802 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.604871+0000", 2024-09-14T06:31:16.802 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.9689999999999999e-06 2024-09-14T06:31:16.802 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:16.802 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:16.802 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:31:16.802 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.604874+0000", 2024-09-14T06:31:16.803 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.5069999999999999e-06 2024-09-14T06:31:16.803 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:16.803 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:16.803 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:31:16.803 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.646874+0000", 2024-09-14T06:31:16.803 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.042000513000000003 2024-09-14T06:31:16.803 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:16.803 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:16.803 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:31:16.803 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.646914+0000", 2024-09-14T06:31:16.803 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.9443999999999997e-05 2024-09-14T06:31:16.803 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:16.803 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:16.803 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:31:16.804 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.646964+0000", 2024-09-14T06:31:16.804 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.0389000000000002e-05 2024-09-14T06:31:16.804 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:16.804 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:31:16.804 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:16.804 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:16.804 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:16.804 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:55660 3.0 3:06490baf:::rbd_data.112dca2f6ae1.0000000000000030:17 [read 1761280~4096 [fadvise_nocache]] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:31:16.804 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:44.604877+0000", 2024-09-14T06:31:16.804 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 92.109206037999996, 2024-09-14T06:31:16.804 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.042196349000000001, 2024-09-14T06:31:16.804 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:31:16.804 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:31:16.804 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:31:16.805 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:31:16.805 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:31:16.805 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:16.805 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 55660 2024-09-14T06:31:16.805 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:16.805 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:31:16.805 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:16.805 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:31:16.805 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.604877+0000", 2024-09-14T06:31:16.805 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:16.805 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:16.805 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:16.805 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:31:16.805 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.604877+0000", 2024-09-14T06:31:16.806 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:16.806 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:16.806 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:16.806 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:31:16.806 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.605810+0000", 2024-09-14T06:31:16.806 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00093336899999999997 2024-09-14T06:31:16.806 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:16.806 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:16.806 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:31:16.806 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.605815+0000", 2024-09-14T06:31:16.806 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.6140000000000002e-06 2024-09-14T06:31:16.806 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:16.806 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:16.806 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:31:16.806 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.605825+0000", 2024-09-14T06:31:16.807 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.8889999999999997e-06 2024-09-14T06:31:16.807 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:16.807 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:16.807 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:31:16.807 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.605831+0000", 2024-09-14T06:31:16.807 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.9800000000000003e-06 2024-09-14T06:31:16.807 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:16.807 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:16.807 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:31:16.807 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.646989+0000", 2024-09-14T06:31:16.807 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.041157878000000002 2024-09-14T06:31:16.807 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:16.807 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:16.807 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:31:16.808 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.647026+0000", 2024-09-14T06:31:16.808 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.7305000000000003e-05 2024-09-14T06:31:16.808 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:16.808 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:16.808 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:31:16.808 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.647073+0000", 2024-09-14T06:31:16.808 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.7314000000000001e-05 2024-09-14T06:31:16.808 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:16.808 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:31:16.808 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:16.808 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:16.808 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:16.808 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:55666 3.0 3:06490baf:::rbd_data.112dca2f6ae1.0000000000000030:17 [read 1908736~4096 [fadvise_nocache]] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:31:16.808 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:44.605898+0000", 2024-09-14T06:31:16.809 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 92.108185292000002, 2024-09-14T06:31:16.809 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.041862887000000001, 2024-09-14T06:31:16.809 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:31:16.809 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:31:16.809 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:31:16.809 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:31:16.809 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:31:16.809 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:16.809 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 55666 2024-09-14T06:31:16.809 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:16.809 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:31:16.809 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:16.809 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:31:16.810 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.605898+0000", 2024-09-14T06:31:16.810 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:16.810 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:16.810 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:16.810 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:31:16.810 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.605898+0000", 2024-09-14T06:31:16.810 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:16.810 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:16.810 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:16.810 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:31:16.810 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.611874+0000", 2024-09-14T06:31:16.810 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0059764049999999997 2024-09-14T06:31:16.810 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:16.810 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:16.811 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:31:16.811 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.611879+0000", 2024-09-14T06:31:16.811 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.8749999999999999e-06 2024-09-14T06:31:16.811 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:16.811 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:16.811 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:31:16.811 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.611887+0000", 2024-09-14T06:31:16.811 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.4109999999999996e-06 2024-09-14T06:31:16.811 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:16.811 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:16.811 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:31:16.811 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.611893+0000", 2024-09-14T06:31:16.811 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.2959999999999998e-06 2024-09-14T06:31:16.811 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:16.812 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:16.812 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:31:16.812 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.647674+0000", 2024-09-14T06:31:16.812 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.035781281999999998 2024-09-14T06:31:16.812 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:16.812 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:16.812 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:31:16.812 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.647711+0000", 2024-09-14T06:31:16.812 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.7434000000000001e-05 2024-09-14T06:31:16.812 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:16.812 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:16.812 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:31:16.812 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.647760+0000", 2024-09-14T06:31:16.812 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.9184000000000002e-05 2024-09-14T06:31:16.813 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:16.813 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:31:16.813 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:16.813 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:16.813 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:31:16.813 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-14T06:31:16.814 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-14T06:31:16.815 INFO:tasks.ceph.osd.0.smithi094.stderr:2024-09-14T06:31:16.813+0000 7f07c9333640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33636) UID: 0 2024-09-14T06:31:16.915 INFO:tasks.ceph.osd.2.smithi094.stderr:2024-09-14T06:31:16.914+0000 7faf3269b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33649) UID: 0 2024-09-14T06:31:16.960 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-14T06:31:16.961 INFO:teuthology.orchestra.run.smithi094.stdout: "ops": [], 2024-09-14T06:31:16.961 INFO:teuthology.orchestra.run.smithi094.stdout: "num_ops": 0 2024-09-14T06:31:16.961 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-14T06:31:16.969 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-14T06:31:17.053 INFO:tasks.ceph.osd.0.smithi094.stderr:2024-09-14T06:31:17.052+0000 7f07c9333640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33636) UID: 0 2024-09-14T06:31:17.112 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-14T06:31:17.112 INFO:teuthology.orchestra.run.smithi094.stdout: "ops": [], 2024-09-14T06:31:17.113 INFO:teuthology.orchestra.run.smithi094.stdout: "complaint_time": 30, 2024-09-14T06:31:17.113 INFO:teuthology.orchestra.run.smithi094.stdout: "num_blocked_ops": 0 2024-09-14T06:31:17.113 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-14T06:31:17.116 INFO:tasks.ceph.osd.4.smithi138.stderr:2024-09-14T06:31:17.116+0000 7f6c653c8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33615) UID: 0 2024-09-14T06:31:17.121 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-14T06:31:17.217 INFO:tasks.ceph.osd.3.smithi094.stderr:2024-09-14T06:31:17.216+0000 7feeeafe8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33655) UID: 0 2024-09-14T06:31:17.265 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-14T06:31:17.265 INFO:teuthology.orchestra.run.smithi094.stdout: "size": 20, 2024-09-14T06:31:17.265 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 600, 2024-09-14T06:31:17.265 INFO:teuthology.orchestra.run.smithi094.stdout: "ops": [ 2024-09-14T06:31:17.265 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:17.265 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4397.0:26415 3.0 e53/23 3:05fd9be0:::rbd_data.112d38768680.0000000000000006:head v 53'21, mlcod=50'19)", 2024-09-14T06:31:17.266 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:30.721610+0000", 2024-09-14T06:31:17.266 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 106.54191691600001, 2024-09-14T06:31:17.266 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0021763160000000002, 2024-09-14T06:31:17.266 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:31:17.266 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:31:17.266 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:31:17.266 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:31:17.266 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:17.266 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:31:17.266 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:30.721610+0000", 2024-09-14T06:31:17.266 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:17.266 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:17.266 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:17.266 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:31:17.266 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:30.721610+0000", 2024-09-14T06:31:17.267 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:17.267 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:17.267 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:17.267 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:31:17.267 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:30.721611+0000", 2024-09-14T06:31:17.267 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.41e-07 2024-09-14T06:31:17.267 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:17.267 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:17.267 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:31:17.267 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:30.721905+0000", 2024-09-14T06:31:17.267 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00029394300000000001 2024-09-14T06:31:17.267 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:17.268 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:17.268 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:31:17.268 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:30.721921+0000", 2024-09-14T06:31:17.268 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.6609999999999999e-05 2024-09-14T06:31:17.268 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:17.268 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:17.268 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:31:17.268 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:30.721931+0000", 2024-09-14T06:31:17.268 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.6739999999999992e-06 2024-09-14T06:31:17.268 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:17.268 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:17.268 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:31:17.268 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:30.722001+0000", 2024-09-14T06:31:17.268 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.0549000000000005e-05 2024-09-14T06:31:17.269 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:17.269 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:17.269 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:31:17.269 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:30.722029+0000", 2024-09-14T06:31:17.269 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.7327000000000001e-05 2024-09-14T06:31:17.269 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:17.269 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:17.269 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-14T06:31:17.269 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:30.723727+0000", 2024-09-14T06:31:17.269 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0016977310000000001 2024-09-14T06:31:17.269 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:17.269 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:17.269 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:31:17.269 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:30.723787+0000", 2024-09-14T06:31:17.270 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.0140999999999999e-05 2024-09-14T06:31:17.270 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:17.270 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:31:17.270 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:17.270 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:17.270 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:17.270 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4397.0:36084 3.0 e56/23 3:05a48ef4:::rbd_data.112dc58639fa.0000000000000058:head v 56'29, mlcod=56'27)", 2024-09-14T06:31:17.270 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:34.508855+0000", 2024-09-14T06:31:17.270 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 102.75467216200001, 2024-09-14T06:31:17.271 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0020801140000000001, 2024-09-14T06:31:17.271 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:31:17.271 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:31:17.271 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:31:17.271 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:31:17.271 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:17.271 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:31:17.271 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:34.508855+0000", 2024-09-14T06:31:17.271 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:17.271 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:17.271 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:17.271 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:31:17.271 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:34.508855+0000", 2024-09-14T06:31:17.271 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:17.272 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:17.272 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:17.272 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:31:17.272 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:34.508856+0000", 2024-09-14T06:31:17.272 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.1909999999999999e-06 2024-09-14T06:31:17.272 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:17.272 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:17.272 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:31:17.272 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:34.509203+0000", 2024-09-14T06:31:17.272 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00034625900000000002 2024-09-14T06:31:17.272 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:17.272 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:17.272 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:31:17.273 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:34.509218+0000", 2024-09-14T06:31:17.273 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.5091e-05 2024-09-14T06:31:17.273 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:17.273 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:17.273 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:31:17.273 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:34.509227+0000", 2024-09-14T06:31:17.273 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.0850000000000003e-06 2024-09-14T06:31:17.273 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:17.273 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:17.273 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:31:17.273 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:34.509311+0000", 2024-09-14T06:31:17.273 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.4436000000000003e-05 2024-09-14T06:31:17.273 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:17.273 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:17.273 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:31:17.274 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:34.509335+0000", 2024-09-14T06:31:17.274 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.3577000000000001e-05 2024-09-14T06:31:17.274 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:17.274 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:17.274 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-14T06:31:17.274 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:34.510890+0000", 2024-09-14T06:31:17.274 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0015554259999999999 2024-09-14T06:31:17.274 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:17.274 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:17.274 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:31:17.274 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:34.510935+0000", 2024-09-14T06:31:17.274 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.5049000000000002e-05 2024-09-14T06:31:17.274 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:17.274 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:31:17.274 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:17.275 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:17.275 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:17.275 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4397.0:45722 3.9 e64/23 3:91fed098:::rbd_data.112d5a737e72.0000000000000001:head v 64'61, mlcod=62'59)", 2024-09-14T06:31:17.275 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:43.864269+0000", 2024-09-14T06:31:17.275 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 93.399258087000007, 2024-09-14T06:31:17.275 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.003055224, 2024-09-14T06:31:17.275 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:31:17.275 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:31:17.275 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:31:17.275 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:31:17.275 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:17.275 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:31:17.275 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.864269+0000", 2024-09-14T06:31:17.275 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:17.275 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:17.276 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:17.276 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:31:17.276 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.864269+0000", 2024-09-14T06:31:17.276 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:17.276 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:17.276 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:17.276 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:31:17.276 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.864269+0000", 2024-09-14T06:31:17.276 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.41e-07 2024-09-14T06:31:17.276 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:17.276 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:17.276 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:31:17.276 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.865490+0000", 2024-09-14T06:31:17.276 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.001220552 2024-09-14T06:31:17.276 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:17.277 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:17.277 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:31:17.277 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.865506+0000", 2024-09-14T06:31:17.277 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.6263999999999999e-05 2024-09-14T06:31:17.277 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:17.277 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:17.277 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:31:17.277 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.865516+0000", 2024-09-14T06:31:17.277 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.0091999999999999e-05 2024-09-14T06:31:17.277 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:17.277 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:17.277 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:31:17.277 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.865583+0000", 2024-09-14T06:31:17.278 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.6676999999999999e-05 2024-09-14T06:31:17.278 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:17.278 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:17.278 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:31:17.278 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.865610+0000", 2024-09-14T06:31:17.278 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.6529999999999998e-05 2024-09-14T06:31:17.278 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:17.278 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:17.278 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-14T06:31:17.278 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.867281+0000", 2024-09-14T06:31:17.278 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.001671203 2024-09-14T06:31:17.278 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:17.278 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:17.278 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:31:17.278 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.867324+0000", 2024-09-14T06:31:17.279 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.3665000000000003e-05 2024-09-14T06:31:17.279 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:17.279 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:31:17.279 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:17.279 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:17.279 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:17.279 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4397.0:50006 3.4 e64/23 3:20cfe300:::rbd_data.112d5a737e72.0000000000000019:head v 64'70, mlcod=61'68)", 2024-09-14T06:31:17.279 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:44.224218+0000", 2024-09-14T06:31:17.279 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 93.039308890000001, 2024-09-14T06:31:17.279 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0021104320000000002, 2024-09-14T06:31:17.279 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:31:17.279 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:31:17.279 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:31:17.279 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:31:17.279 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:17.279 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:31:17.280 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.224218+0000", 2024-09-14T06:31:17.280 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:17.280 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:17.280 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:17.280 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:31:17.280 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.224218+0000", 2024-09-14T06:31:17.280 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:17.280 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:17.280 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:17.280 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:31:17.280 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.224219+0000", 2024-09-14T06:31:17.280 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.0049999999999999e-06 2024-09-14T06:31:17.280 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:17.280 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:17.280 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:31:17.281 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.224326+0000", 2024-09-14T06:31:17.281 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000106848 2024-09-14T06:31:17.281 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:17.281 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:17.281 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:31:17.281 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.224345+0000", 2024-09-14T06:31:17.281 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.8587999999999999e-05 2024-09-14T06:31:17.281 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:17.281 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:17.281 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:31:17.281 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.224354+0000", 2024-09-14T06:31:17.281 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.7220000000000005e-06 2024-09-14T06:31:17.281 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:17.281 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:17.282 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:31:17.282 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.224416+0000", 2024-09-14T06:31:17.282 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.2738999999999997e-05 2024-09-14T06:31:17.282 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:17.282 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:17.282 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:31:17.282 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.224441+0000", 2024-09-14T06:31:17.282 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.4408000000000001e-05 2024-09-14T06:31:17.282 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:17.282 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:17.282 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-14T06:31:17.282 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.226295+0000", 2024-09-14T06:31:17.283 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00185428 2024-09-14T06:31:17.283 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:17.283 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:17.283 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:31:17.283 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.226329+0000", 2024-09-14T06:31:17.283 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.3841999999999999e-05 2024-09-14T06:31:17.283 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:17.283 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:31:17.283 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:17.284 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:17.284 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:17.284 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4397.0:74241 3.0 e73/23 3:06e93fb9:::rbd_data.112d80ca42af.0000000000000000:head v 73'102, mlcod=71'100)", 2024-09-14T06:31:17.284 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:53.876730+0000", 2024-09-14T06:31:17.284 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 83.386796986999997, 2024-09-14T06:31:17.284 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0079470329999999992, 2024-09-14T06:31:17.284 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:31:17.284 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:31:17.284 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:31:17.284 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:31:17.284 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:17.284 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:31:17.284 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:53.876730+0000", 2024-09-14T06:31:17.285 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:17.285 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:17.285 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:17.285 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:31:17.285 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:53.876730+0000", 2024-09-14T06:31:17.285 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:17.285 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:17.285 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:17.285 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:31:17.285 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:53.876731+0000", 2024-09-14T06:31:17.286 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.0500000000000004e-07 2024-09-14T06:31:17.286 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:17.286 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:17.286 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:31:17.286 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:53.879616+0000", 2024-09-14T06:31:17.286 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0028856680000000001 2024-09-14T06:31:17.286 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:17.286 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:17.286 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:31:17.286 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:53.879643+0000", 2024-09-14T06:31:17.286 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.6120999999999999e-05 2024-09-14T06:31:17.286 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:17.286 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:17.287 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:31:17.287 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:53.879654+0000", 2024-09-14T06:31:17.287 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.1586999999999999e-05 2024-09-14T06:31:17.287 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:17.287 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:17.287 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:31:17.287 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:53.879756+0000", 2024-09-14T06:31:17.287 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00010223899999999999 2024-09-14T06:31:17.287 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:17.287 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:17.287 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:31:17.287 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:53.879783+0000", 2024-09-14T06:31:17.287 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.6913000000000001e-05 2024-09-14T06:31:17.288 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:17.288 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:17.288 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-14T06:31:17.288 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:53.884594+0000", 2024-09-14T06:31:17.288 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0048110729999999999 2024-09-14T06:31:17.288 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:17.288 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:17.288 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:31:17.288 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:53.884677+0000", 2024-09-14T06:31:17.288 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.2927000000000001e-05 2024-09-14T06:31:17.288 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:17.288 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:31:17.288 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:17.289 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:17.289 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:17.289 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4397.0:74243 3.9 e73/23 3:94977f81:::rbd_data.112d80ca42af.0000000000000002:head v 73'94, mlcod=71'92)", 2024-09-14T06:31:17.289 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:53.885156+0000", 2024-09-14T06:31:17.289 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 83.378371068999996, 2024-09-14T06:31:17.289 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0096009860000000006, 2024-09-14T06:31:17.289 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:31:17.289 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:31:17.289 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:31:17.289 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:31:17.289 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:17.289 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:31:17.289 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:53.885156+0000", 2024-09-14T06:31:17.289 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:17.289 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:17.290 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:17.290 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:31:17.290 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:53.885156+0000", 2024-09-14T06:31:17.290 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:17.290 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:17.290 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:17.290 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:31:17.290 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:53.885156+0000", 2024-09-14T06:31:17.290 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.23e-07 2024-09-14T06:31:17.290 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:17.290 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:17.290 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:31:17.290 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:53.889694+0000", 2024-09-14T06:31:17.290 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0045378709999999997 2024-09-14T06:31:17.291 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:17.291 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:17.291 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:31:17.291 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:53.889720+0000", 2024-09-14T06:31:17.291 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.5259e-05 2024-09-14T06:31:17.291 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:17.291 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:17.291 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:31:17.291 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:53.889731+0000", 2024-09-14T06:31:17.291 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.1739000000000001e-05 2024-09-14T06:31:17.291 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:17.291 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:17.292 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:31:17.292 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:53.889822+0000", 2024-09-14T06:31:17.292 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.1200999999999996e-05 2024-09-14T06:31:17.292 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:17.292 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:17.292 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:31:17.292 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:53.889850+0000", 2024-09-14T06:31:17.292 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.7084999999999999e-05 2024-09-14T06:31:17.292 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:17.292 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:17.292 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-14T06:31:17.292 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:53.894703+0000", 2024-09-14T06:31:17.292 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0048532690000000003 2024-09-14T06:31:17.292 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:17.293 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:17.293 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:31:17.293 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:53.894757+0000", 2024-09-14T06:31:17.293 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.4339000000000003e-05 2024-09-14T06:31:17.293 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:17.293 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:31:17.293 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:17.293 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:17.293 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:17.293 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4397.0:74249 3.0 e73/23 3:009b0225:::rbd_data.112d80ca42af.0000000000000009:head v 73'103, mlcod=71'101)", 2024-09-14T06:31:17.294 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:53.893219+0000", 2024-09-14T06:31:17.294 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 83.370308590999997, 2024-09-14T06:31:17.294 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0065381980000000003, 2024-09-14T06:31:17.294 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:31:17.294 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:31:17.294 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:31:17.294 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:31:17.294 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:17.294 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:31:17.294 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:53.893219+0000", 2024-09-14T06:31:17.294 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:17.295 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:17.295 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:17.295 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:31:17.295 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:53.893219+0000", 2024-09-14T06:31:17.295 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:17.295 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:17.295 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:17.295 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:31:17.296 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:53.893221+0000", 2024-09-14T06:31:17.296 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.4329999999999998e-06 2024-09-14T06:31:17.296 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:17.296 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:17.296 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:31:17.296 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:53.895968+0000", 2024-09-14T06:31:17.296 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.002746746 2024-09-14T06:31:17.296 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:17.296 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:17.296 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:31:17.296 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:53.895988+0000", 2024-09-14T06:31:17.297 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.0227000000000001e-05 2024-09-14T06:31:17.297 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:17.297 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:17.297 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:31:17.297 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:53.896000+0000", 2024-09-14T06:31:17.297 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.2359e-05 2024-09-14T06:31:17.297 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:17.297 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:17.298 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:31:17.298 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:53.896084+0000", 2024-09-14T06:31:17.298 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.3659999999999995e-05 2024-09-14T06:31:17.298 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:17.298 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:17.298 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:31:17.298 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:53.896109+0000", 2024-09-14T06:31:17.298 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.5259999999999999e-05 2024-09-14T06:31:17.298 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:17.299 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:17.299 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-14T06:31:17.299 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:53.899704+0000", 2024-09-14T06:31:17.299 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0035942209999999999 2024-09-14T06:31:17.299 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:17.299 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:17.299 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:31:17.299 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:53.899757+0000", 2024-09-14T06:31:17.299 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.3291999999999997e-05 2024-09-14T06:31:17.299 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:17.300 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:31:17.300 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:17.300 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:17.300 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:17.300 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4397.0:78263 3.0 e77/23 3:00899b22:::rbd_data.112ddfbec471.0000000000000016:head v 77'104, mlcod=73'102)", 2024-09-14T06:31:17.300 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:57.634578+0000", 2024-09-14T06:31:17.300 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 79.628949539000004, 2024-09-14T06:31:17.300 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.002372454, 2024-09-14T06:31:17.300 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:31:17.300 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:31:17.301 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:31:17.301 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:31:17.301 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:17.301 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:31:17.301 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:57.634578+0000", 2024-09-14T06:31:17.301 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:17.301 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:17.301 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:17.301 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:31:17.301 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:57.634578+0000", 2024-09-14T06:31:17.301 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:17.301 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:17.302 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:17.302 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:31:17.302 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:57.634578+0000", 2024-09-14T06:31:17.302 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.5699999999999998e-07 2024-09-14T06:31:17.302 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:17.302 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:17.302 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:31:17.302 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:57.635347+0000", 2024-09-14T06:31:17.302 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00076848300000000003 2024-09-14T06:31:17.302 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:17.302 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:17.302 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:31:17.302 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:57.635361+0000", 2024-09-14T06:31:17.302 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.4238e-05 2024-09-14T06:31:17.303 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:17.303 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:17.303 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:31:17.303 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:57.635371+0000", 2024-09-14T06:31:17.303 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.0087e-05 2024-09-14T06:31:17.303 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:17.303 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:17.303 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:31:17.303 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:57.635439+0000", 2024-09-14T06:31:17.303 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.7953000000000005e-05 2024-09-14T06:31:17.303 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:17.303 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:17.303 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:31:17.303 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:57.635464+0000", 2024-09-14T06:31:17.304 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.5062000000000002e-05 2024-09-14T06:31:17.304 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:17.304 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:17.304 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-14T06:31:17.304 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:57.636914+0000", 2024-09-14T06:31:17.304 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.001449625 2024-09-14T06:31:17.304 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:17.304 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:17.304 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:31:17.304 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:57.636950+0000", 2024-09-14T06:31:17.304 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.6548999999999999e-05 2024-09-14T06:31:17.304 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:17.304 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:31:17.305 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:17.305 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:17.305 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:17.305 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4397.0:78604 3.9 e77/23 3:94c62a0e:::rbd_data.112ddfbec471.000000000000002c:head v 77'96, mlcod=73'94)", 2024-09-14T06:31:17.305 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:57.677568+0000", 2024-09-14T06:31:17.305 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 79.585959150999997, 2024-09-14T06:31:17.305 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.002564846, 2024-09-14T06:31:17.305 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:31:17.305 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:31:17.306 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:31:17.306 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:31:17.306 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:17.306 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:31:17.306 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:57.677568+0000", 2024-09-14T06:31:17.306 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:17.306 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:17.306 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:17.306 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:31:17.306 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:57.677568+0000", 2024-09-14T06:31:17.306 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:17.306 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:17.306 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:17.306 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:31:17.306 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:57.677568+0000", 2024-09-14T06:31:17.307 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.8900000000000001e-07 2024-09-14T06:31:17.307 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:17.307 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:17.307 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:31:17.307 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:57.678488+0000", 2024-09-14T06:31:17.307 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00091963899999999998 2024-09-14T06:31:17.307 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:17.307 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:17.307 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:31:17.307 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:57.678500+0000", 2024-09-14T06:31:17.307 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.1902000000000001e-05 2024-09-14T06:31:17.307 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:17.307 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:17.308 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:31:17.308 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:57.678507+0000", 2024-09-14T06:31:17.308 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.7739999999999999e-06 2024-09-14T06:31:17.308 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:17.308 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:17.308 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:31:17.308 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:57.678586+0000", 2024-09-14T06:31:17.308 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.9103000000000005e-05 2024-09-14T06:31:17.308 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:17.308 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:17.308 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:31:17.308 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:57.678614+0000", 2024-09-14T06:31:17.308 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.7912e-05 2024-09-14T06:31:17.309 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:17.309 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:17.309 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-14T06:31:17.309 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:57.680075+0000", 2024-09-14T06:31:17.309 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0014613370000000001 2024-09-14T06:31:17.309 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:17.309 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:17.309 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:31:17.309 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:57.680133+0000", 2024-09-14T06:31:17.309 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.7890000000000003e-05 2024-09-14T06:31:17.309 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:17.309 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:31:17.309 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:17.309 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:17.310 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:17.310 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4397.0:78641 3.9 e77/23 3:914b72e4:::rbd_data.112ddfbec471.0000000000000030:head v 77'98, mlcod=73'95)", 2024-09-14T06:31:17.310 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:57.682289+0000", 2024-09-14T06:31:17.310 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 79.581238470000002, 2024-09-14T06:31:17.310 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0020171400000000002, 2024-09-14T06:31:17.310 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:31:17.310 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:31:17.310 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:31:17.310 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:31:17.310 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:17.310 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:31:17.310 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:57.682289+0000", 2024-09-14T06:31:17.310 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:17.310 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:17.311 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:17.311 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:31:17.311 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:57.682289+0000", 2024-09-14T06:31:17.311 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:17.311 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:17.311 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:17.311 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:31:17.311 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:57.682290+0000", 2024-09-14T06:31:17.311 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.0100000000000003e-07 2024-09-14T06:31:17.311 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:17.311 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:17.311 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:31:17.311 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:57.683230+0000", 2024-09-14T06:31:17.312 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00094047599999999999 2024-09-14T06:31:17.312 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:17.312 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:17.312 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:31:17.312 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:57.683243+0000", 2024-09-14T06:31:17.312 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.2926e-05 2024-09-14T06:31:17.312 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:17.312 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:17.312 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:31:17.312 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:57.683250+0000", 2024-09-14T06:31:17.312 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.6109999999999999e-06 2024-09-14T06:31:17.312 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:17.312 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:17.313 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:31:17.313 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:57.683313+0000", 2024-09-14T06:31:17.313 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.2943999999999999e-05 2024-09-14T06:31:17.313 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:17.313 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:17.313 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:31:17.313 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:57.683337+0000", 2024-09-14T06:31:17.313 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.4377999999999999e-05 2024-09-14T06:31:17.313 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:17.313 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:17.313 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-14T06:31:17.313 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:57.684264+0000", 2024-09-14T06:31:17.313 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00092733099999999999 2024-09-14T06:31:17.314 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:17.314 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:17.314 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:31:17.314 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:57.684306+0000", 2024-09-14T06:31:17.314 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.1573000000000003e-05 2024-09-14T06:31:17.314 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:17.314 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:31:17.314 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:17.314 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:17.314 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:17.314 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4397.0:126010 3.9 e104/23 3:9120f7ff:::rbd_data.112d97471187.000000000000001b:head v 104'145, mlcod=99'143)", 2024-09-14T06:31:17.314 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:30:25.990254+0000", 2024-09-14T06:31:17.314 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 51.273273662000001, 2024-09-14T06:31:17.314 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0020629699999999999, 2024-09-14T06:31:17.315 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:31:17.315 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:31:17.315 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:31:17.315 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:31:17.315 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:17.315 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:31:17.315 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:25.990254+0000", 2024-09-14T06:31:17.315 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:17.315 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:17.315 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:17.315 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:31:17.315 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:25.990254+0000", 2024-09-14T06:31:17.315 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:17.315 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:17.315 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:17.316 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:31:17.316 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:25.990255+0000", 2024-09-14T06:31:17.316 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.3349999999999999e-06 2024-09-14T06:31:17.316 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:17.316 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:17.316 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:31:17.316 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:25.990966+0000", 2024-09-14T06:31:17.316 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00071085199999999999 2024-09-14T06:31:17.316 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:17.316 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:17.316 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:31:17.316 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:25.990986+0000", 2024-09-14T06:31:17.316 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.0054000000000001e-05 2024-09-14T06:31:17.317 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:17.317 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:17.317 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:31:17.317 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:25.990995+0000", 2024-09-14T06:31:17.317 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.166e-06 2024-09-14T06:31:17.317 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:17.317 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:17.317 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:31:17.317 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:25.991055+0000", 2024-09-14T06:31:17.317 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.0093000000000002e-05 2024-09-14T06:31:17.317 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:17.317 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:17.317 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:31:17.317 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:25.991082+0000", 2024-09-14T06:31:17.318 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.6505000000000001e-05 2024-09-14T06:31:17.318 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:17.318 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:17.318 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-14T06:31:17.318 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:25.992270+0000", 2024-09-14T06:31:17.318 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0011880479999999999 2024-09-14T06:31:17.318 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:17.318 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:17.318 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:31:17.318 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:25.992317+0000", 2024-09-14T06:31:17.318 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.6916999999999998e-05 2024-09-14T06:31:17.318 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:17.318 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:31:17.319 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:17.319 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:17.319 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:17.319 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4397.0:149885 3.9 e122/23 3:951acef3:::rbd_data.112d1bb7b269.0000000000000005:head v 122'176, mlcod=122'172)", 2024-09-14T06:31:17.319 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:30:44.739092+0000", 2024-09-14T06:31:17.319 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 32.524434841000001, 2024-09-14T06:31:17.319 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0028236659999999998, 2024-09-14T06:31:17.319 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:31:17.319 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:31:17.319 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:31:17.319 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:31:17.319 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:17.319 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:31:17.320 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.739092+0000", 2024-09-14T06:31:17.320 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:17.320 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:17.320 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:17.320 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:31:17.320 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.739092+0000", 2024-09-14T06:31:17.320 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:17.320 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:17.320 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:17.320 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:31:17.320 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.739093+0000", 2024-09-14T06:31:17.320 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.0109999999999999e-06 2024-09-14T06:31:17.320 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:17.320 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:17.320 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:31:17.321 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.740189+0000", 2024-09-14T06:31:17.321 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00109527 2024-09-14T06:31:17.321 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:17.321 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:17.321 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:31:17.321 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.740199+0000", 2024-09-14T06:31:17.321 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.0458999999999999e-05 2024-09-14T06:31:17.321 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:17.322 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:17.322 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:31:17.322 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.740207+0000", 2024-09-14T06:31:17.322 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.4499999999999998e-06 2024-09-14T06:31:17.322 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:17.322 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:17.322 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:31:17.322 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.740295+0000", 2024-09-14T06:31:17.322 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.8740999999999996e-05 2024-09-14T06:31:17.322 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:17.322 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:17.322 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:31:17.322 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.740317+0000", 2024-09-14T06:31:17.322 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.1523e-05 2024-09-14T06:31:17.322 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:17.323 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:17.323 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-14T06:31:17.323 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.741865+0000", 2024-09-14T06:31:17.323 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.001548315 2024-09-14T06:31:17.323 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:17.323 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:17.323 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:31:17.323 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.741916+0000", 2024-09-14T06:31:17.323 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.0896999999999997e-05 2024-09-14T06:31:17.324 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:17.324 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:31:17.324 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:17.324 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:17.324 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:17.324 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4397.0:149886 3.4 e122/23 3:2033c3d0:::rbd_data.112d1bb7b269.0000000000000006:head v 122'220, mlcod=120'217)", 2024-09-14T06:31:17.324 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:30:44.741736+0000", 2024-09-14T06:31:17.324 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 32.521791424, 2024-09-14T06:31:17.324 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0020268999999999999, 2024-09-14T06:31:17.324 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:31:17.324 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:31:17.324 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:31:17.324 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:31:17.325 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:17.325 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:31:17.325 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.741736+0000", 2024-09-14T06:31:17.325 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:17.325 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:17.325 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:17.325 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:31:17.325 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.741736+0000", 2024-09-14T06:31:17.325 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:17.326 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:17.326 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:17.326 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:31:17.326 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.741737+0000", 2024-09-14T06:31:17.326 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.9000000000000005e-07 2024-09-14T06:31:17.326 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:17.326 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:17.326 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:31:17.326 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.741918+0000", 2024-09-14T06:31:17.326 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00018120300000000001 2024-09-14T06:31:17.326 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:17.326 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:17.326 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:31:17.326 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.741930+0000", 2024-09-14T06:31:17.327 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.1875999999999999e-05 2024-09-14T06:31:17.327 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:17.327 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:17.327 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:31:17.327 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.741938+0000", 2024-09-14T06:31:17.327 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.8040000000000002e-06 2024-09-14T06:31:17.327 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:17.327 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:17.327 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:31:17.327 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.742185+0000", 2024-09-14T06:31:17.328 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00024747199999999999 2024-09-14T06:31:17.328 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:17.328 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:17.328 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:31:17.328 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.742207+0000", 2024-09-14T06:31:17.328 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.1416999999999998e-05 2024-09-14T06:31:17.328 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:17.328 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:17.328 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-14T06:31:17.328 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.743715+0000", 2024-09-14T06:31:17.328 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00150884 2024-09-14T06:31:17.329 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:17.329 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:17.329 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:31:17.329 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.743763+0000", 2024-09-14T06:31:17.329 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.7298e-05 2024-09-14T06:31:17.329 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:17.329 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:31:17.329 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:17.329 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:17.329 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:17.329 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4397.0:149933 3.4 e122/23 3:234f5229:::rbd_data.112d1bb7b269.0000000000000009:head v 122'221, mlcod=120'218)", 2024-09-14T06:31:17.330 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:30:44.743849+0000", 2024-09-14T06:31:17.330 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 32.519678423999999, 2024-09-14T06:31:17.330 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0031479519999999999, 2024-09-14T06:31:17.330 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:31:17.330 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:31:17.330 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:31:17.330 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:31:17.330 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:17.330 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:31:17.330 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.743849+0000", 2024-09-14T06:31:17.330 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:17.330 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:17.330 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:17.330 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:31:17.330 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.743849+0000", 2024-09-14T06:31:17.331 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:17.331 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:17.331 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:17.331 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:31:17.331 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.743850+0000", 2024-09-14T06:31:17.331 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.279e-06 2024-09-14T06:31:17.331 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:17.331 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:17.331 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:31:17.331 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.745151+0000", 2024-09-14T06:31:17.332 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.001300495 2024-09-14T06:31:17.332 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:17.332 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:17.332 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:31:17.332 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.745167+0000", 2024-09-14T06:31:17.332 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.6478e-05 2024-09-14T06:31:17.332 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:17.332 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:17.332 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:31:17.333 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.745175+0000", 2024-09-14T06:31:17.333 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.2679999999999994e-06 2024-09-14T06:31:17.333 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:17.333 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:17.333 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:31:17.333 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.745226+0000", 2024-09-14T06:31:17.333 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.0192999999999999e-05 2024-09-14T06:31:17.333 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:17.334 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:17.334 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:31:17.334 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.745247+0000", 2024-09-14T06:31:17.334 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.1767e-05 2024-09-14T06:31:17.334 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:17.334 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:17.334 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-14T06:31:17.334 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.746952+0000", 2024-09-14T06:31:17.334 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0017042769999999999 2024-09-14T06:31:17.334 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:17.334 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:17.334 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:31:17.334 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.746997+0000", 2024-09-14T06:31:17.334 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.5194999999999997e-05 2024-09-14T06:31:17.334 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:17.335 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:31:17.335 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:17.335 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:17.335 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:17.335 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4397.0:150494 3.9 e122/23 3:955152a4:::rbd_object_map.112d1bb7b269.0000000000000038:head v 122'194, mlcod=122'190)", 2024-09-14T06:31:17.335 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:30:44.781585+0000", 2024-09-14T06:31:17.335 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 32.481942531999998, 2024-09-14T06:31:17.335 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0026957499999999998, 2024-09-14T06:31:17.335 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:31:17.335 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:31:17.335 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:31:17.335 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:31:17.335 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:17.336 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:31:17.336 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.781585+0000", 2024-09-14T06:31:17.336 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:17.336 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:17.336 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:17.336 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:31:17.336 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.781585+0000", 2024-09-14T06:31:17.336 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:17.336 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:17.336 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:17.336 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:31:17.336 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.781585+0000", 2024-09-14T06:31:17.336 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.5000000000000002e-07 2024-09-14T06:31:17.336 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:17.336 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:17.337 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:31:17.337 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.781594+0000", 2024-09-14T06:31:17.337 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.4640000000000006e-06 2024-09-14T06:31:17.337 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:17.337 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:17.337 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:31:17.337 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.781604+0000", 2024-09-14T06:31:17.337 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.0071999999999999e-05 2024-09-14T06:31:17.337 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:17.337 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:17.337 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:31:17.337 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.781610+0000", 2024-09-14T06:31:17.338 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.3609999999999997e-06 2024-09-14T06:31:17.338 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:17.338 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:17.338 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:31:17.338 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.781676+0000", 2024-09-14T06:31:17.338 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.5963000000000005e-05 2024-09-14T06:31:17.338 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:17.338 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:17.338 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:31:17.338 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.781696+0000", 2024-09-14T06:31:17.338 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.9278000000000001e-05 2024-09-14T06:31:17.338 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:17.338 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:17.338 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-14T06:31:17.339 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.784231+0000", 2024-09-14T06:31:17.339 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.002535093 2024-09-14T06:31:17.339 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:17.339 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:17.339 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:31:17.339 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.784280+0000", 2024-09-14T06:31:17.339 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.9869e-05 2024-09-14T06:31:17.339 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:17.339 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:31:17.339 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:17.339 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:17.339 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:17.339 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4397.0:150519 3.9 e122/23 3:955152a4:::rbd_object_map.112d1bb7b269.0000000000000038:head v 122'195, mlcod=122'192)", 2024-09-14T06:31:17.339 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:30:44.782445+0000", 2024-09-14T06:31:17.340 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 32.481082139999998, 2024-09-14T06:31:17.340 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.002158227, 2024-09-14T06:31:17.340 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:31:17.340 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:31:17.340 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:31:17.340 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:31:17.340 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:17.340 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:31:17.340 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.782445+0000", 2024-09-14T06:31:17.340 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:17.340 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:17.340 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:17.341 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:31:17.341 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.782445+0000", 2024-09-14T06:31:17.341 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:17.341 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:17.341 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:17.341 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:31:17.341 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.782446+0000", 2024-09-14T06:31:17.341 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.6499999999999998e-07 2024-09-14T06:31:17.341 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:17.341 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:17.341 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:31:17.341 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.782455+0000", 2024-09-14T06:31:17.341 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.1940000000000004e-06 2024-09-14T06:31:17.341 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:17.341 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:17.342 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:31:17.342 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.782464+0000", 2024-09-14T06:31:17.342 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.7220000000000005e-06 2024-09-14T06:31:17.342 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:17.342 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:17.342 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:31:17.342 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.782470+0000", 2024-09-14T06:31:17.342 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.9580000000000004e-06 2024-09-14T06:31:17.342 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:17.342 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:17.342 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:31:17.342 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.782514+0000", 2024-09-14T06:31:17.343 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.3931999999999999e-05 2024-09-14T06:31:17.343 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:17.343 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:17.343 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:31:17.343 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.782534+0000", 2024-09-14T06:31:17.343 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.0237e-05 2024-09-14T06:31:17.343 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:17.343 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:17.343 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-14T06:31:17.343 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.784560+0000", 2024-09-14T06:31:17.343 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.002025813 2024-09-14T06:31:17.343 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:17.343 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:17.344 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:31:17.344 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.784603+0000", 2024-09-14T06:31:17.344 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.3606000000000002e-05 2024-09-14T06:31:17.344 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:17.344 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:31:17.344 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:17.344 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:17.344 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:17.344 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4397.0:150898 3.9 e122/23 3:97c98f90:::rbd_data.112d1bb7b269.0000000000000025:head v 122'206, mlcod=122'203)", 2024-09-14T06:31:17.344 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:30:44.807232+0000", 2024-09-14T06:31:17.344 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 32.456295615000002, 2024-09-14T06:31:17.344 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0022297889999999998, 2024-09-14T06:31:17.344 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:31:17.344 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:31:17.345 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:31:17.345 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:31:17.345 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:17.345 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:31:17.345 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.807232+0000", 2024-09-14T06:31:17.345 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:17.345 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:17.345 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:17.345 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:31:17.345 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.807232+0000", 2024-09-14T06:31:17.345 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:17.345 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:17.346 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:17.346 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:31:17.346 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.807233+0000", 2024-09-14T06:31:17.346 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.5710000000000001e-06 2024-09-14T06:31:17.346 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:17.346 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:17.346 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:31:17.346 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.807944+0000", 2024-09-14T06:31:17.346 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00071056299999999999 2024-09-14T06:31:17.346 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:17.346 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:17.346 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:31:17.346 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.807961+0000", 2024-09-14T06:31:17.346 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.7689e-05 2024-09-14T06:31:17.347 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:17.347 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:17.347 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:31:17.347 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.807971+0000", 2024-09-14T06:31:17.347 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.2e-06 2024-09-14T06:31:17.347 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:17.347 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:17.347 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:31:17.347 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.808050+0000", 2024-09-14T06:31:17.347 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.9308999999999996e-05 2024-09-14T06:31:17.347 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:17.347 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:17.347 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:31:17.347 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.808074+0000", 2024-09-14T06:31:17.348 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.4147000000000001e-05 2024-09-14T06:31:17.348 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:17.348 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:17.348 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-14T06:31:17.348 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.809415+0000", 2024-09-14T06:31:17.348 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.001341257 2024-09-14T06:31:17.348 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:17.348 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:17.348 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:31:17.348 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.809461+0000", 2024-09-14T06:31:17.348 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.6053000000000001e-05 2024-09-14T06:31:17.348 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:17.348 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:31:17.348 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:17.348 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:17.349 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:17.349 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4397.0:152325 3.9 e122/23 3:955152a4:::rbd_object_map.112d1bb7b269.0000000000000038:head v 122'268, mlcod=122'265)", 2024-09-14T06:31:17.349 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:30:44.914377+0000", 2024-09-14T06:31:17.349 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 32.349150309999999, 2024-09-14T06:31:17.349 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0020208840000000001, 2024-09-14T06:31:17.349 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:31:17.349 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:31:17.349 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:31:17.349 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:31:17.349 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:17.349 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:31:17.349 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.914377+0000", 2024-09-14T06:31:17.349 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:17.350 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:17.350 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:17.350 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:31:17.350 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.914377+0000", 2024-09-14T06:31:17.350 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:17.350 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:17.350 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:17.350 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:31:17.350 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.914378+0000", 2024-09-14T06:31:17.350 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.3600000000000003e-07 2024-09-14T06:31:17.350 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:17.350 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:17.350 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:31:17.350 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.914383+0000", 2024-09-14T06:31:17.351 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.7230000000000001e-06 2024-09-14T06:31:17.351 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:17.351 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:17.351 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:31:17.351 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.914393+0000", 2024-09-14T06:31:17.351 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.8179999999999999e-06 2024-09-14T06:31:17.351 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:17.351 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:17.351 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:31:17.351 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.914400+0000", 2024-09-14T06:31:17.351 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.968e-06 2024-09-14T06:31:17.351 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:17.351 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:17.351 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:31:17.352 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.914447+0000", 2024-09-14T06:31:17.352 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.7122e-05 2024-09-14T06:31:17.352 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:17.352 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:17.352 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:31:17.352 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.914466+0000", 2024-09-14T06:31:17.352 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.8958999999999999e-05 2024-09-14T06:31:17.352 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:17.352 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:17.352 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-14T06:31:17.352 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.916356+0000", 2024-09-14T06:31:17.352 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.001890121 2024-09-14T06:31:17.352 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:17.353 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:17.353 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:31:17.353 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.916398+0000", 2024-09-14T06:31:17.353 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.1437000000000003e-05 2024-09-14T06:31:17.353 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:17.353 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:31:17.353 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:17.353 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:17.353 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:17.353 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4397.0:152449 3.9 e122/23 3:955152a4:::rbd_object_map.112d1bb7b269.0000000000000038:head v 122'274, mlcod=122'268)", 2024-09-14T06:31:17.353 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:30:44.924063+0000", 2024-09-14T06:31:17.353 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 32.339463799000001, 2024-09-14T06:31:17.353 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.002131764, 2024-09-14T06:31:17.353 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:31:17.354 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:31:17.354 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:31:17.354 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:31:17.354 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:17.354 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:31:17.354 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.924063+0000", 2024-09-14T06:31:17.354 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:17.354 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:17.354 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:17.354 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:31:17.355 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.924063+0000", 2024-09-14T06:31:17.355 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:17.355 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:17.355 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:17.355 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:31:17.355 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.924064+0000", 2024-09-14T06:31:17.355 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.72e-07 2024-09-14T06:31:17.355 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:17.355 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:17.355 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:31:17.355 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.924071+0000", 2024-09-14T06:31:17.355 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.0999999999999998e-06 2024-09-14T06:31:17.356 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:17.356 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:17.356 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:31:17.356 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.924077+0000", 2024-09-14T06:31:17.356 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.268e-06 2024-09-14T06:31:17.356 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:17.356 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:17.356 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:31:17.356 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.924082+0000", 2024-09-14T06:31:17.356 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.7110000000000003e-06 2024-09-14T06:31:17.356 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:17.357 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:17.357 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:31:17.357 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.924397+0000", 2024-09-14T06:31:17.357 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00031503500000000002 2024-09-14T06:31:17.357 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:17.357 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:17.357 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:31:17.357 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.924418+0000", 2024-09-14T06:31:17.357 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.1044999999999999e-05 2024-09-14T06:31:17.357 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:17.358 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:17.358 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-14T06:31:17.358 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.926158+0000", 2024-09-14T06:31:17.358 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0017404860000000001 2024-09-14T06:31:17.358 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:17.358 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:17.358 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:31:17.358 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.926195+0000", 2024-09-14T06:31:17.358 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.6847000000000001e-05 2024-09-14T06:31:17.358 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:17.358 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:31:17.358 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:17.359 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:17.359 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:17.359 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4397.0:152452 3.9 e122/23 3:955152a4:::rbd_object_map.112d1bb7b269.0000000000000038:head v 122'275, mlcod=122'268)", 2024-09-14T06:31:17.359 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:30:44.924090+0000", 2024-09-14T06:31:17.359 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 32.339437478999997, 2024-09-14T06:31:17.359 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0021362400000000002, 2024-09-14T06:31:17.359 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:31:17.359 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:31:17.359 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:31:17.360 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:31:17.360 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:17.360 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:31:17.360 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.924090+0000", 2024-09-14T06:31:17.360 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:17.360 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:17.360 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:17.360 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:31:17.360 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.924090+0000", 2024-09-14T06:31:17.360 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:17.360 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:17.360 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:17.360 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:31:17.361 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.924090+0000", 2024-09-14T06:31:17.361 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.4699999999999998e-07 2024-09-14T06:31:17.361 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:17.361 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:17.361 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:31:17.361 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.924658+0000", 2024-09-14T06:31:17.361 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00056802700000000003 2024-09-14T06:31:17.361 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:17.361 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:17.362 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:31:17.362 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.924664+0000", 2024-09-14T06:31:17.362 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.3570000000000003e-06 2024-09-14T06:31:17.362 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:17.362 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:17.362 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:31:17.362 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.924669+0000", 2024-09-14T06:31:17.362 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.2300000000000002e-06 2024-09-14T06:31:17.362 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:17.362 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:17.362 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:31:17.362 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.924802+0000", 2024-09-14T06:31:17.363 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00013362500000000001 2024-09-14T06:31:17.363 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:17.363 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:17.363 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:31:17.363 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.924824+0000", 2024-09-14T06:31:17.363 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.1449000000000001e-05 2024-09-14T06:31:17.363 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:17.363 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:17.363 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-14T06:31:17.363 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.926197+0000", 2024-09-14T06:31:17.363 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.001373354 2024-09-14T06:31:17.363 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:17.363 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:17.363 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:31:17.364 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.926226+0000", 2024-09-14T06:31:17.364 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.8951000000000001e-05 2024-09-14T06:31:17.364 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:17.364 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:31:17.364 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:17.364 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:17.364 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:31:17.364 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-14T06:31:17.365 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-14T06:31:17.366 INFO:tasks.ceph.osd.4.smithi138.stderr:2024-09-14T06:31:17.365+0000 7f6c653c8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33615) UID: 0 2024-09-14T06:31:17.466 INFO:tasks.ceph.osd.0.smithi094.stderr:2024-09-14T06:31:17.465+0000 7f07c9333640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33636) UID: 0 2024-09-14T06:31:17.511 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-14T06:31:17.512 INFO:teuthology.orchestra.run.smithi094.stdout: "ops": [], 2024-09-14T06:31:17.512 INFO:teuthology.orchestra.run.smithi094.stdout: "num_ops": 0 2024-09-14T06:31:17.512 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-14T06:31:17.520 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-14T06:31:17.604 INFO:tasks.ceph.osd.3.smithi094.stderr:2024-09-14T06:31:17.603+0000 7feeeafe8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33655) UID: 0 2024-09-14T06:31:17.613 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-14T06:31:17.664 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-14T06:31:17.665 INFO:teuthology.orchestra.run.smithi094.stdout: "ops": [], 2024-09-14T06:31:17.665 INFO:teuthology.orchestra.run.smithi094.stdout: "complaint_time": 30, 2024-09-14T06:31:17.665 INFO:teuthology.orchestra.run.smithi094.stdout: "num_blocked_ops": 0 2024-09-14T06:31:17.665 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-14T06:31:17.668 INFO:tasks.ceph.osd.6.smithi138.stderr:2024-09-14T06:31:17.667+0000 7f8cd8849640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33620) UID: 0 2024-09-14T06:31:17.674 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-14T06:31:17.769 INFO:tasks.ceph.osd.2.smithi094.stderr:2024-09-14T06:31:17.768+0000 7faf3269b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33649) UID: 0 2024-09-14T06:31:17.784 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-14T06:31:17.784 INFO:teuthology.orchestra.run.smithi094.stdout: "size": 20, 2024-09-14T06:31:17.784 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 600, 2024-09-14T06:31:17.784 INFO:teuthology.orchestra.run.smithi094.stdout: "ops": [ 2024-09-14T06:31:17.784 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:17.785 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:44425 3.2 3:4603ad7f:::rbd_data.112dca2f6ae1.0000000000000005:17 [read 868352~4096 [fadvise_nocache]] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:31:17.785 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:43.854967+0000", 2024-09-14T06:31:17.785 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 93.927500366000004, 2024-09-14T06:31:17.785 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.063146025999999994, 2024-09-14T06:31:17.785 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:31:17.785 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:31:17.785 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:31:17.785 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:31:17.785 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:31:17.785 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:17.785 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 44425 2024-09-14T06:31:17.785 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:17.785 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:31:17.785 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:17.786 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:31:17.786 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.854967+0000", 2024-09-14T06:31:17.786 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:17.786 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:17.786 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:17.786 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:31:17.786 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.854967+0000", 2024-09-14T06:31:17.786 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:17.786 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:17.786 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:17.786 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:31:17.786 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.854967+0000", 2024-09-14T06:31:17.786 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.16e-07 2024-09-14T06:31:17.786 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:17.787 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:17.787 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:31:17.787 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.854968+0000", 2024-09-14T06:31:17.787 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.006e-06 2024-09-14T06:31:17.787 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:17.787 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:17.787 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:31:17.787 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.854971+0000", 2024-09-14T06:31:17.787 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.9359999999999999e-06 2024-09-14T06:31:17.787 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:17.787 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:17.787 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:31:17.787 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.854974+0000", 2024-09-14T06:31:17.788 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.1840000000000002e-06 2024-09-14T06:31:17.788 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:17.788 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:17.788 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:31:17.788 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.917945+0000", 2024-09-14T06:31:17.788 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.062970925999999997 2024-09-14T06:31:17.788 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:17.788 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:17.788 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:31:17.788 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.918030+0000", 2024-09-14T06:31:17.788 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.5863000000000001e-05 2024-09-14T06:31:17.788 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:17.788 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:17.788 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:31:17.789 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.918113+0000", 2024-09-14T06:31:17.789 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.2894999999999998e-05 2024-09-14T06:31:17.789 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:17.789 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:31:17.789 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:17.789 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:17.789 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:17.789 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:44426 3.2 3:4603ad7f:::rbd_data.112dca2f6ae1.0000000000000005:17 [read 892928~4096 [fadvise_nocache]] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:31:17.789 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:43.854977+0000", 2024-09-14T06:31:17.789 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 93.927490956, 2024-09-14T06:31:17.789 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.063369237999999994, 2024-09-14T06:31:17.789 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:31:17.789 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:31:17.790 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:31:17.790 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:31:17.790 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:31:17.790 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:17.790 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 44426 2024-09-14T06:31:17.790 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:17.790 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:31:17.790 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:17.790 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:31:17.790 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.854977+0000", 2024-09-14T06:31:17.790 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:17.790 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:17.790 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:17.790 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:31:17.791 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.854977+0000", 2024-09-14T06:31:17.791 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:17.791 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:17.791 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:17.791 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:31:17.791 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.854977+0000", 2024-09-14T06:31:17.791 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.1199999999999999e-07 2024-09-14T06:31:17.791 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:17.791 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:17.791 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:31:17.791 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.854980+0000", 2024-09-14T06:31:17.791 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.27e-06 2024-09-14T06:31:17.791 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:17.791 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:17.791 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:31:17.792 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.854983+0000", 2024-09-14T06:31:17.792 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.1650000000000002e-06 2024-09-14T06:31:17.792 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:17.792 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:17.792 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:31:17.792 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.854989+0000", 2024-09-14T06:31:17.792 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.6520000000000003e-06 2024-09-14T06:31:17.792 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:17.792 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:17.792 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:31:17.792 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.918143+0000", 2024-09-14T06:31:17.792 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.063154541999999994 2024-09-14T06:31:17.792 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:17.792 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:17.793 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:31:17.793 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.918209+0000", 2024-09-14T06:31:17.793 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.5529000000000002e-05 2024-09-14T06:31:17.793 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:17.793 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:17.793 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:31:17.793 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.918346+0000", 2024-09-14T06:31:17.793 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000136868 2024-09-14T06:31:17.793 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:17.793 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:31:17.793 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:17.793 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:17.793 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:17.794 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:44427 3.2 3:4603ad7f:::rbd_data.112dca2f6ae1.0000000000000005:17 [read 917504~4096 [fadvise_nocache]] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:31:17.794 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:43.854992+0000", 2024-09-14T06:31:17.794 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 93.927475310000005, 2024-09-14T06:31:17.794 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.063607792999999996, 2024-09-14T06:31:17.794 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:31:17.794 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:31:17.794 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:31:17.794 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:31:17.794 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:31:17.794 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:17.794 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 44427 2024-09-14T06:31:17.794 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:17.794 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:31:17.794 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:17.794 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:31:17.795 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.854992+0000", 2024-09-14T06:31:17.795 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:17.795 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:17.795 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:17.795 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:31:17.795 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.854992+0000", 2024-09-14T06:31:17.795 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:17.795 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:17.795 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:17.795 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:31:17.795 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.854993+0000", 2024-09-14T06:31:17.795 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.1400000000000001e-07 2024-09-14T06:31:17.795 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:17.795 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:17.796 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:31:17.796 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.854994+0000", 2024-09-14T06:31:17.796 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.091e-06 2024-09-14T06:31:17.796 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:17.796 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:17.796 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:31:17.796 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855001+0000", 2024-09-14T06:31:17.796 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.204e-06 2024-09-14T06:31:17.796 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:17.796 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:17.796 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:31:17.796 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855003+0000", 2024-09-14T06:31:17.796 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.514e-06 2024-09-14T06:31:17.796 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:17.797 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:17.797 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:31:17.797 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.918380+0000", 2024-09-14T06:31:17.797 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.063376738000000002 2024-09-14T06:31:17.797 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:17.797 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:17.797 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:31:17.797 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.918453+0000", 2024-09-14T06:31:17.797 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.3291e-05 2024-09-14T06:31:17.797 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:17.797 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:17.797 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:31:17.797 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.918600+0000", 2024-09-14T06:31:17.797 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00014674100000000001 2024-09-14T06:31:17.798 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:17.798 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:31:17.798 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:17.798 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:17.798 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:17.798 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:44428 3.2 3:4603ad7f:::rbd_data.112dca2f6ae1.0000000000000005:17 [read 942080~4096 [fadvise_nocache]] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:31:17.798 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:43.855006+0000", 2024-09-14T06:31:17.798 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 93.927461260000001, 2024-09-14T06:31:17.798 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.063846321999999997, 2024-09-14T06:31:17.798 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:31:17.798 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:31:17.798 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:31:17.798 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:31:17.798 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:31:17.799 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:17.799 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 44428 2024-09-14T06:31:17.799 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:17.799 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:31:17.799 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:17.799 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:31:17.799 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855006+0000", 2024-09-14T06:31:17.799 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:17.799 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:17.799 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:17.799 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:31:17.799 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855006+0000", 2024-09-14T06:31:17.800 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:17.800 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:17.800 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:17.800 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:31:17.800 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855007+0000", 2024-09-14T06:31:17.800 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.1199999999999999e-07 2024-09-14T06:31:17.800 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:17.800 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:17.800 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:31:17.800 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855011+0000", 2024-09-14T06:31:17.800 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.9539999999999998e-06 2024-09-14T06:31:17.800 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:17.800 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:17.801 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:31:17.801 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855014+0000", 2024-09-14T06:31:17.801 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.2930000000000001e-06 2024-09-14T06:31:17.801 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:17.801 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:17.801 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:31:17.801 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855016+0000", 2024-09-14T06:31:17.801 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.3039999999999999e-06 2024-09-14T06:31:17.801 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:17.801 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:17.801 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:31:17.801 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.918634+0000", 2024-09-14T06:31:17.801 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.063617928000000004 2024-09-14T06:31:17.801 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:17.802 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:17.802 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:31:17.802 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.918711+0000", 2024-09-14T06:31:17.802 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.6545000000000007e-05 2024-09-14T06:31:17.802 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:17.802 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:17.802 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:31:17.802 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.918853+0000", 2024-09-14T06:31:17.802 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00014208600000000001 2024-09-14T06:31:17.802 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:17.802 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:31:17.802 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:17.802 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:17.803 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:17.803 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:44429 3.2 3:4603ad7f:::rbd_data.112dca2f6ae1.0000000000000005:17 [read 966656~4096 [fadvise_nocache]] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:31:17.803 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:43.855019+0000", 2024-09-14T06:31:17.803 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 93.927448587000001, 2024-09-14T06:31:17.803 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.064059226999999996, 2024-09-14T06:31:17.803 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:31:17.803 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:31:17.803 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:31:17.803 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:31:17.803 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:31:17.803 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:17.803 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 44429 2024-09-14T06:31:17.803 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:17.803 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:31:17.804 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:17.804 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:31:17.804 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855019+0000", 2024-09-14T06:31:17.804 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:17.804 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:17.804 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:17.804 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:31:17.804 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855019+0000", 2024-09-14T06:31:17.804 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:17.804 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:17.804 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:17.804 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:31:17.804 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855019+0000", 2024-09-14T06:31:17.804 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.28e-07 2024-09-14T06:31:17.804 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:17.805 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:17.805 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:31:17.805 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855020+0000", 2024-09-14T06:31:17.805 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.0330000000000001e-06 2024-09-14T06:31:17.805 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:17.805 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:17.805 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:31:17.805 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855023+0000", 2024-09-14T06:31:17.805 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.79e-06 2024-09-14T06:31:17.805 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:17.805 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:17.805 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:31:17.805 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855025+0000", 2024-09-14T06:31:17.805 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.216e-06 2024-09-14T06:31:17.806 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:17.806 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:17.806 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:31:17.806 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.918886+0000", 2024-09-14T06:31:17.806 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.063861037999999995 2024-09-14T06:31:17.806 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:17.806 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:17.806 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:31:17.806 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.918952+0000", 2024-09-14T06:31:17.806 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.5993000000000003e-05 2024-09-14T06:31:17.806 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:17.807 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:17.807 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:31:17.807 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.919078+0000", 2024-09-14T06:31:17.807 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00012592899999999999 2024-09-14T06:31:17.807 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:17.807 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:31:17.807 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:17.807 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:17.807 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:17.807 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:44430 3.2 3:4603ad7f:::rbd_data.112dca2f6ae1.0000000000000005:17 [read 991232~4096 [fadvise_nocache]] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:31:17.807 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:43.855028+0000", 2024-09-14T06:31:17.807 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 93.927439577000001, 2024-09-14T06:31:17.807 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.064284346000000006, 2024-09-14T06:31:17.807 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:31:17.808 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:31:17.808 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:31:17.808 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:31:17.808 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:31:17.808 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:17.808 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 44430 2024-09-14T06:31:17.808 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:17.808 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:31:17.808 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:17.808 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:31:17.808 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855028+0000", 2024-09-14T06:31:17.808 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:17.808 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:17.809 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:17.809 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:31:17.809 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855028+0000", 2024-09-14T06:31:17.809 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:17.809 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:17.809 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:17.809 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:31:17.809 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855028+0000", 2024-09-14T06:31:17.809 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.1199999999999999e-07 2024-09-14T06:31:17.809 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:17.809 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:17.809 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:31:17.809 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855029+0000", 2024-09-14T06:31:17.809 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.1100000000000004e-07 2024-09-14T06:31:17.810 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:17.810 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:17.810 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:31:17.810 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855032+0000", 2024-09-14T06:31:17.810 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.7889999999999999e-06 2024-09-14T06:31:17.810 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:17.810 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:17.810 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:31:17.810 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855034+0000", 2024-09-14T06:31:17.810 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.125e-06 2024-09-14T06:31:17.810 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:17.810 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:17.810 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:31:17.810 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.919110+0000", 2024-09-14T06:31:17.811 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.064076413999999998 2024-09-14T06:31:17.811 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:17.811 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:17.811 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:31:17.811 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.919186+0000", 2024-09-14T06:31:17.811 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.5784999999999994e-05 2024-09-14T06:31:17.811 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:17.811 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:17.811 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:31:17.811 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.919312+0000", 2024-09-14T06:31:17.811 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00012611 2024-09-14T06:31:17.811 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:17.812 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:31:17.812 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:17.812 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:17.812 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:17.812 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:44431 3.2 3:4603ad7f:::rbd_data.112dca2f6ae1.0000000000000005:17 [read 1015808~4096 [fadvise_nocache]] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:31:17.812 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:43.855037+0000", 2024-09-14T06:31:17.812 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 93.927430709000006, 2024-09-14T06:31:17.812 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.064469755000000004, 2024-09-14T06:31:17.812 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:31:17.812 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:31:17.812 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:31:17.812 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:31:17.812 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:31:17.812 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:17.813 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 44431 2024-09-14T06:31:17.813 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:17.813 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:31:17.813 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:17.813 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:31:17.813 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855037+0000", 2024-09-14T06:31:17.813 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:17.813 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:17.813 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:17.814 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:31:17.814 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855037+0000", 2024-09-14T06:31:17.814 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:17.814 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:17.814 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:17.814 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:31:17.814 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855037+0000", 2024-09-14T06:31:17.814 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.1e-07 2024-09-14T06:31:17.814 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:17.814 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:17.814 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:31:17.814 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855038+0000", 2024-09-14T06:31:17.815 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.9400000000000004e-07 2024-09-14T06:31:17.815 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:17.815 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:17.815 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:31:17.815 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855041+0000", 2024-09-14T06:31:17.815 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.9639999999999999e-06 2024-09-14T06:31:17.815 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:17.815 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:17.815 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:31:17.815 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855043+0000", 2024-09-14T06:31:17.815 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.2259999999999999e-06 2024-09-14T06:31:17.815 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:17.815 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:17.816 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:31:17.816 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.919347+0000", 2024-09-14T06:31:17.816 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.064304091999999993 2024-09-14T06:31:17.816 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:17.816 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:17.816 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:31:17.816 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.919393+0000", 2024-09-14T06:31:17.816 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.5340999999999998e-05 2024-09-14T06:31:17.816 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:17.817 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:17.817 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:31:17.817 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.919507+0000", 2024-09-14T06:31:17.817 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000114028 2024-09-14T06:31:17.817 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:17.817 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:31:17.817 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:17.817 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:17.817 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:17.817 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:44432 3.2 3:4603ad7f:::rbd_data.112dca2f6ae1.0000000000000005:17 [read 1040384~4096 [fadvise_nocache]] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:31:17.817 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:43.855046+0000", 2024-09-14T06:31:17.817 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 93.927421640999995, 2024-09-14T06:31:17.817 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.064639253999999993, 2024-09-14T06:31:17.817 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:31:17.818 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:31:17.818 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:31:17.818 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:31:17.818 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:31:17.818 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:17.818 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 44432 2024-09-14T06:31:17.818 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:17.818 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:31:17.818 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:17.818 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:31:17.818 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855046+0000", 2024-09-14T06:31:17.818 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:17.818 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:17.818 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:17.819 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:31:17.819 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855046+0000", 2024-09-14T06:31:17.819 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:17.819 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:17.819 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:17.819 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:31:17.819 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855046+0000", 2024-09-14T06:31:17.819 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.05e-07 2024-09-14T06:31:17.819 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:17.819 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:17.819 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:31:17.819 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855047+0000", 2024-09-14T06:31:17.819 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.879999999999999e-07 2024-09-14T06:31:17.819 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:17.820 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:17.820 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:31:17.820 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855050+0000", 2024-09-14T06:31:17.820 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.8459999999999999e-06 2024-09-14T06:31:17.820 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:17.820 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:17.820 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:31:17.820 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855052+0000", 2024-09-14T06:31:17.820 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.136e-06 2024-09-14T06:31:17.820 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:17.821 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:17.821 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:31:17.821 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.919536+0000", 2024-09-14T06:31:17.821 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.064484132 2024-09-14T06:31:17.821 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:17.821 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:17.821 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:31:17.821 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.919611+0000", 2024-09-14T06:31:17.821 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.4407999999999996e-05 2024-09-14T06:31:17.821 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:17.821 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:17.821 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:31:17.821 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.919685+0000", 2024-09-14T06:31:17.821 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.4538999999999999e-05 2024-09-14T06:31:17.822 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:17.822 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:31:17.822 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:17.822 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:17.822 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:17.822 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:44518 3.2 3:4190d882:::rbd_data.112dca2f6ae1.0000000000000002:17 [read 12288~4096 [fadvise_nocache]] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:31:17.822 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:43.855055+0000", 2024-09-14T06:31:17.822 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 93.927412756999999, 2024-09-14T06:31:17.822 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.064894317000000007, 2024-09-14T06:31:17.822 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:31:17.822 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:31:17.822 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:31:17.822 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:31:17.823 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:31:17.823 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:17.823 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 44518 2024-09-14T06:31:17.823 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:17.823 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:31:17.823 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:17.823 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:31:17.823 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855055+0000", 2024-09-14T06:31:17.823 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:17.823 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:17.823 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:17.823 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:31:17.823 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855055+0000", 2024-09-14T06:31:17.823 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:17.823 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:17.824 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:17.824 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:31:17.824 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855055+0000", 2024-09-14T06:31:17.824 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.0699999999999999e-07 2024-09-14T06:31:17.824 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:17.824 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:17.824 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:31:17.824 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855056+0000", 2024-09-14T06:31:17.824 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.9400000000000004e-07 2024-09-14T06:31:17.824 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:17.824 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:17.824 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:31:17.825 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855059+0000", 2024-09-14T06:31:17.825 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.8870000000000001e-06 2024-09-14T06:31:17.825 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:17.825 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:17.825 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:31:17.825 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855061+0000", 2024-09-14T06:31:17.825 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.1919999999999999e-06 2024-09-14T06:31:17.825 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:17.825 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:17.825 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:31:17.825 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.919719+0000", 2024-09-14T06:31:17.825 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.064658013 2024-09-14T06:31:17.826 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:17.826 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:17.826 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:31:17.826 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.919788+0000", 2024-09-14T06:31:17.826 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.9282000000000005e-05 2024-09-14T06:31:17.826 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:17.826 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:17.826 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:31:17.826 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.919949+0000", 2024-09-14T06:31:17.826 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00016084199999999999 2024-09-14T06:31:17.826 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:17.826 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:31:17.826 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:17.827 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:17.827 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:17.827 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:44519 3.2 3:4190d882:::rbd_data.112dca2f6ae1.0000000000000002:17 [read 36864~4096 [fadvise_nocache]] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:31:17.827 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:43.855064+0000", 2024-09-14T06:31:17.827 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 93.927403780000006, 2024-09-14T06:31:17.827 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.065081865000000003, 2024-09-14T06:31:17.827 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:31:17.827 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:31:17.827 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:31:17.827 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:31:17.827 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:31:17.827 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:17.828 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 44519 2024-09-14T06:31:17.828 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:17.828 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:31:17.828 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:17.828 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:31:17.828 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855064+0000", 2024-09-14T06:31:17.828 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:17.828 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:17.828 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:17.828 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:31:17.828 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855064+0000", 2024-09-14T06:31:17.828 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:17.828 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:17.829 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:17.829 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:31:17.829 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855064+0000", 2024-09-14T06:31:17.829 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.1e-07 2024-09-14T06:31:17.829 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:17.829 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:17.829 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:31:17.829 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855065+0000", 2024-09-14T06:31:17.829 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.035e-06 2024-09-14T06:31:17.829 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:17.829 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:17.829 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:31:17.829 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855068+0000", 2024-09-14T06:31:17.830 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.8339999999999999e-06 2024-09-14T06:31:17.830 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:17.830 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:17.830 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:31:17.830 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855071+0000", 2024-09-14T06:31:17.830 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.3450000000000002e-06 2024-09-14T06:31:17.830 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:17.830 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:17.830 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:31:17.830 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.919979+0000", 2024-09-14T06:31:17.830 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.064907647999999998 2024-09-14T06:31:17.830 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:17.830 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:17.830 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:31:17.831 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.920026+0000", 2024-09-14T06:31:17.831 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.7595000000000001e-05 2024-09-14T06:31:17.831 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:17.831 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:17.831 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:31:17.831 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.920146+0000", 2024-09-14T06:31:17.831 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00011919800000000001 2024-09-14T06:31:17.831 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:17.831 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:31:17.831 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:17.831 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:17.831 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:17.831 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:44520 3.2 3:4190d882:::rbd_data.112dca2f6ae1.0000000000000002:17 [read 61440~4096 [fadvise_nocache]] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:31:17.831 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:43.855075+0000", 2024-09-14T06:31:17.831 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 93.927392729000005, 2024-09-14T06:31:17.832 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.065335078000000005, 2024-09-14T06:31:17.832 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:31:17.832 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:31:17.832 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:31:17.832 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:31:17.832 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:31:17.832 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:17.832 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 44520 2024-09-14T06:31:17.832 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:17.832 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:31:17.832 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:17.832 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:31:17.833 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855075+0000", 2024-09-14T06:31:17.833 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:17.833 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:17.833 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:17.833 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:31:17.833 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855075+0000", 2024-09-14T06:31:17.833 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:17.833 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:17.833 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:17.833 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:31:17.833 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855075+0000", 2024-09-14T06:31:17.833 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.1e-07 2024-09-14T06:31:17.833 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:17.834 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:17.834 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:31:17.834 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855076+0000", 2024-09-14T06:31:17.834 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.0839999999999999e-06 2024-09-14T06:31:17.834 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:17.834 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:17.834 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:31:17.834 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855079+0000", 2024-09-14T06:31:17.834 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.9359999999999999e-06 2024-09-14T06:31:17.834 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:17.834 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:17.834 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:31:17.834 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855084+0000", 2024-09-14T06:31:17.835 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.1730000000000003e-06 2024-09-14T06:31:17.835 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:17.835 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:17.835 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:31:17.835 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.920175+0000", 2024-09-14T06:31:17.835 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.065090234999999996 2024-09-14T06:31:17.835 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:17.835 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:17.835 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:31:17.835 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.920250+0000", 2024-09-14T06:31:17.835 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.5807000000000001e-05 2024-09-14T06:31:17.835 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:17.835 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:17.836 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:31:17.836 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.920410+0000", 2024-09-14T06:31:17.836 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00015963300000000001 2024-09-14T06:31:17.836 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:17.836 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:31:17.836 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:17.836 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:17.836 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:17.836 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:44521 3.2 3:4190d882:::rbd_data.112dca2f6ae1.0000000000000002:17 [read 86016~4096 [fadvise_nocache]] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:31:17.836 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:43.855087+0000", 2024-09-14T06:31:17.836 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 93.927380248000006, 2024-09-14T06:31:17.837 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.065563264999999996, 2024-09-14T06:31:17.837 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:31:17.837 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:31:17.837 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:31:17.837 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:31:17.837 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:31:17.837 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:17.837 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 44521 2024-09-14T06:31:17.837 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:17.837 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:31:17.837 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:17.837 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:31:17.837 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855087+0000", 2024-09-14T06:31:17.837 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:17.838 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:17.838 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:17.838 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:31:17.838 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855087+0000", 2024-09-14T06:31:17.838 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:17.838 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:17.838 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:17.838 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:31:17.838 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855088+0000", 2024-09-14T06:31:17.838 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.3699999999999999e-07 2024-09-14T06:31:17.838 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:17.838 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:17.838 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:31:17.839 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855089+0000", 2024-09-14T06:31:17.839 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.9900000000000009e-07 2024-09-14T06:31:17.839 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:17.839 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:17.839 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:31:17.839 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855091+0000", 2024-09-14T06:31:17.839 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.8940000000000002e-06 2024-09-14T06:31:17.839 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:17.839 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:17.839 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:31:17.839 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855094+0000", 2024-09-14T06:31:17.840 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.351e-06 2024-09-14T06:31:17.840 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:17.840 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:17.840 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:31:17.840 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.920437+0000", 2024-09-14T06:31:17.840 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.065343221000000007 2024-09-14T06:31:17.840 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:17.840 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:17.840 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:31:17.840 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.920481+0000", 2024-09-14T06:31:17.840 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.4431000000000002e-05 2024-09-14T06:31:17.840 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:17.841 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:17.841 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:31:17.841 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.920651+0000", 2024-09-14T06:31:17.841 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00016913200000000001 2024-09-14T06:31:17.841 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:17.841 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:31:17.841 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:17.841 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:17.841 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:17.841 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:44522 3.2 3:4190d882:::rbd_data.112dca2f6ae1.0000000000000002:17 [read 110592~4096 [fadvise_nocache]] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:31:17.841 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:43.855098+0000", 2024-09-14T06:31:17.841 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 93.927370021000002, 2024-09-14T06:31:17.841 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.065827628999999999, 2024-09-14T06:31:17.841 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:31:17.842 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:31:17.842 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:31:17.842 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:31:17.842 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:31:17.842 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:17.842 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 44522 2024-09-14T06:31:17.842 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:17.842 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:31:17.842 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:17.842 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:31:17.842 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855098+0000", 2024-09-14T06:31:17.842 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:17.842 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:17.842 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:17.843 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:31:17.843 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855098+0000", 2024-09-14T06:31:17.843 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:17.843 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:17.843 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:17.843 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:31:17.843 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855098+0000", 2024-09-14T06:31:17.843 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.1e-07 2024-09-14T06:31:17.843 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:17.843 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:17.843 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:31:17.843 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855099+0000", 2024-09-14T06:31:17.843 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.012e-06 2024-09-14T06:31:17.843 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:17.844 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:17.844 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:31:17.844 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855102+0000", 2024-09-14T06:31:17.844 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.9380000000000001e-06 2024-09-14T06:31:17.844 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:17.844 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:17.844 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:31:17.844 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855108+0000", 2024-09-14T06:31:17.844 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.4339999999999997e-06 2024-09-14T06:31:17.844 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:17.844 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:17.844 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:31:17.844 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.920681+0000", 2024-09-14T06:31:17.844 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.065572943999999994 2024-09-14T06:31:17.845 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:17.845 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:17.845 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:31:17.845 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.920724+0000", 2024-09-14T06:31:17.845 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.2877000000000003e-05 2024-09-14T06:31:17.845 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:17.845 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:17.845 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:31:17.845 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.920925+0000", 2024-09-14T06:31:17.845 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00020121399999999999 2024-09-14T06:31:17.845 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:17.845 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:31:17.845 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:17.845 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:17.846 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:17.846 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:44523 3.2 3:4190d882:::rbd_data.112dca2f6ae1.0000000000000002:17 [read 135168~4096 [fadvise_nocache]] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:31:17.846 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:43.855111+0000", 2024-09-14T06:31:17.846 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 93.927356298999996, 2024-09-14T06:31:17.846 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.066028198999999996, 2024-09-14T06:31:17.846 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:31:17.846 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:31:17.846 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:31:17.846 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:31:17.846 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:31:17.846 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:17.846 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 44523 2024-09-14T06:31:17.846 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:17.846 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:31:17.847 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:17.847 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:31:17.847 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855111+0000", 2024-09-14T06:31:17.847 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:17.847 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:17.847 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:17.847 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:31:17.847 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855111+0000", 2024-09-14T06:31:17.847 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:17.847 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:17.847 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:17.847 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:31:17.847 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855112+0000", 2024-09-14T06:31:17.847 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.1199999999999999e-07 2024-09-14T06:31:17.848 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:17.848 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:17.848 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:31:17.848 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855115+0000", 2024-09-14T06:31:17.848 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.3380000000000001e-06 2024-09-14T06:31:17.848 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:17.848 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:17.848 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:31:17.848 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855118+0000", 2024-09-14T06:31:17.848 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.366e-06 2024-09-14T06:31:17.848 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:17.848 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:17.848 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:31:17.848 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855121+0000", 2024-09-14T06:31:17.848 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.8329999999999998e-06 2024-09-14T06:31:17.849 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:17.849 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:17.849 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:31:17.849 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.920956+0000", 2024-09-14T06:31:17.849 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.065835251999999997 2024-09-14T06:31:17.849 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:17.849 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:17.849 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:31:17.849 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.921001+0000", 2024-09-14T06:31:17.849 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.4778999999999999e-05 2024-09-14T06:31:17.849 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:17.849 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:17.849 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:31:17.850 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.921139+0000", 2024-09-14T06:31:17.850 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000138419 2024-09-14T06:31:17.850 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:17.850 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:31:17.850 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:17.850 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:17.850 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:17.850 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:44524 3.2 3:4190d882:::rbd_data.112dca2f6ae1.0000000000000002:17 [read 159744~4096 [fadvise_nocache]] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:31:17.850 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:43.855124+0000", 2024-09-14T06:31:17.850 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 93.927343320000006, 2024-09-14T06:31:17.850 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.066249261000000004, 2024-09-14T06:31:17.850 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:31:17.850 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:31:17.851 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:31:17.851 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:31:17.851 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:31:17.851 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:17.851 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 44524 2024-09-14T06:31:17.851 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:17.851 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:31:17.851 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:17.851 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:31:17.851 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855124+0000", 2024-09-14T06:31:17.851 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:17.851 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:17.851 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:17.851 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:31:17.851 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855124+0000", 2024-09-14T06:31:17.852 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:17.852 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:17.852 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:17.852 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:31:17.852 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861086+0000", 2024-09-14T06:31:17.852 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0059613829999999998 2024-09-14T06:31:17.852 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:17.852 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:17.852 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:31:17.852 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861100+0000", 2024-09-14T06:31:17.852 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.4644000000000001e-05 2024-09-14T06:31:17.852 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:17.852 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:17.853 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:31:17.853 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861109+0000", 2024-09-14T06:31:17.853 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.541e-06 2024-09-14T06:31:17.853 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:17.853 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:17.853 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:31:17.853 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861117+0000", 2024-09-14T06:31:17.853 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.1049999999999995e-06 2024-09-14T06:31:17.853 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:17.853 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:17.853 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:31:17.853 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.921169+0000", 2024-09-14T06:31:17.853 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.060052405000000003 2024-09-14T06:31:17.854 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:17.854 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:17.854 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:31:17.854 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.921213+0000", 2024-09-14T06:31:17.854 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.3565e-05 2024-09-14T06:31:17.854 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:17.854 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:17.854 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:31:17.854 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.921374+0000", 2024-09-14T06:31:17.854 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00016061800000000001 2024-09-14T06:31:17.854 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:17.854 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:31:17.854 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:17.854 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:17.854 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:17.855 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:44536 3.2 3:4190d882:::rbd_data.112dca2f6ae1.0000000000000002:17 [read 454656~4096 [fadvise_nocache]] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:31:17.855 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:43.861384+0000", 2024-09-14T06:31:17.855 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 93.921083762999999, 2024-09-14T06:31:17.855 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.063058528000000003, 2024-09-14T06:31:17.855 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:31:17.855 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:31:17.855 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:31:17.855 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:31:17.855 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:31:17.855 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:17.855 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 44536 2024-09-14T06:31:17.855 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:17.855 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:31:17.855 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:17.856 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:31:17.856 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861384+0000", 2024-09-14T06:31:17.856 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:17.856 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:17.856 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:17.856 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:31:17.856 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861384+0000", 2024-09-14T06:31:17.856 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:17.856 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:17.856 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:17.856 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:31:17.856 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861384+0000", 2024-09-14T06:31:17.856 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.4000000000000002e-07 2024-09-14T06:31:17.856 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:17.857 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:17.857 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:31:17.857 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861386+0000", 2024-09-14T06:31:17.857 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.891e-06 2024-09-14T06:31:17.857 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:17.857 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:17.857 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:31:17.857 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861392+0000", 2024-09-14T06:31:17.857 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.9939999999999997e-06 2024-09-14T06:31:17.857 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:17.857 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:17.857 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:31:17.857 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861398+0000", 2024-09-14T06:31:17.857 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.507e-06 2024-09-14T06:31:17.858 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:17.858 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:17.858 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:31:17.858 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.924149+0000", 2024-09-14T06:31:17.858 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.062751135999999999 2024-09-14T06:31:17.858 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:17.858 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:17.858 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:31:17.858 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.924226+0000", 2024-09-14T06:31:17.858 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.7517999999999999e-05 2024-09-14T06:31:17.858 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:17.858 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:17.858 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:31:17.858 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.924442+0000", 2024-09-14T06:31:17.859 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000216042 2024-09-14T06:31:17.859 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:17.859 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:31:17.859 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:17.859 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:17.859 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:17.859 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:44537 3.2 3:4190d882:::rbd_data.112dca2f6ae1.0000000000000002:17 [read 479232~4096 [fadvise_nocache]] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:31:17.859 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:43.861405+0000", 2024-09-14T06:31:17.859 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 93.921062716999998, 2024-09-14T06:31:17.859 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.063351795000000002, 2024-09-14T06:31:17.859 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:31:17.859 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:31:17.859 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:31:17.860 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:31:17.860 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:31:17.860 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:17.860 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 44537 2024-09-14T06:31:17.860 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:17.860 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:31:17.860 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:17.860 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:31:17.860 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861405+0000", 2024-09-14T06:31:17.860 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:17.860 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:17.860 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:17.860 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:31:17.861 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861405+0000", 2024-09-14T06:31:17.861 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:17.861 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:17.861 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:17.861 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:31:17.861 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861405+0000", 2024-09-14T06:31:17.861 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.9099999999999999e-07 2024-09-14T06:31:17.861 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:17.861 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:17.861 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:31:17.861 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861407+0000", 2024-09-14T06:31:17.861 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.075e-06 2024-09-14T06:31:17.861 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:17.862 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:17.862 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:31:17.862 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861414+0000", 2024-09-14T06:31:17.862 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.4659999999999996e-06 2024-09-14T06:31:17.862 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:17.862 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:17.862 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:31:17.862 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861418+0000", 2024-09-14T06:31:17.862 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.5519999999999998e-06 2024-09-14T06:31:17.862 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:17.862 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:17.862 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:31:17.862 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.924474+0000", 2024-09-14T06:31:17.862 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.063056012999999994 2024-09-14T06:31:17.863 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:17.863 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:17.863 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:31:17.863 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.924556+0000", 2024-09-14T06:31:17.863 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.1181999999999996e-05 2024-09-14T06:31:17.863 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:17.863 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:17.863 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:31:17.863 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.924757+0000", 2024-09-14T06:31:17.863 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00020111599999999999 2024-09-14T06:31:17.863 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:17.863 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:31:17.863 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:17.863 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:17.864 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:17.864 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:44538 3.2 3:4190d882:::rbd_data.112dca2f6ae1.0000000000000002:17 [read 503808~4096 [fadvise_nocache]] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:31:17.864 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:43.861425+0000", 2024-09-14T06:31:17.864 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 93.921042675999999, 2024-09-14T06:31:17.864 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.063627125000000007, 2024-09-14T06:31:17.864 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:31:17.864 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:31:17.864 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:31:17.864 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:31:17.864 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:31:17.864 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:17.864 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 44538 2024-09-14T06:31:17.864 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:17.865 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:31:17.865 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:17.865 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:31:17.865 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861425+0000", 2024-09-14T06:31:17.865 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:17.865 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:17.865 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:17.865 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:31:17.865 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861425+0000", 2024-09-14T06:31:17.865 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:17.865 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:17.865 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:17.865 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:31:17.865 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861425+0000", 2024-09-14T06:31:17.866 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.9999999999999998e-07 2024-09-14T06:31:17.866 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:17.866 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:17.866 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:31:17.866 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861427+0000", 2024-09-14T06:31:17.866 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.9829999999999999e-06 2024-09-14T06:31:17.866 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:17.866 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:17.866 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:31:17.866 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861433+0000", 2024-09-14T06:31:17.866 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.2889999999999997e-06 2024-09-14T06:31:17.866 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:17.866 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:17.866 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:31:17.867 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861454+0000", 2024-09-14T06:31:17.867 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.1844999999999998e-05 2024-09-14T06:31:17.867 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:17.867 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:17.867 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:31:17.867 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.924786+0000", 2024-09-14T06:31:17.867 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.063331497 2024-09-14T06:31:17.867 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:17.867 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:17.867 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:31:17.867 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.924864+0000", 2024-09-14T06:31:17.867 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.7744000000000006e-05 2024-09-14T06:31:17.868 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:17.868 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:17.868 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:31:17.868 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.925052+0000", 2024-09-14T06:31:17.868 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00018836699999999999 2024-09-14T06:31:17.868 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:17.868 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:31:17.868 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:17.868 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:17.868 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:17.868 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:44539 3.2 3:4190d882:::rbd_data.112dca2f6ae1.0000000000000002:17 [read 528384~4096 [fadvise_nocache]] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:31:17.868 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:43.861462+0000", 2024-09-14T06:31:17.868 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 93.921005348999998, 2024-09-14T06:31:17.868 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.063855705999999998, 2024-09-14T06:31:17.869 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:31:17.869 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:31:17.869 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:31:17.869 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:31:17.869 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:31:17.869 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:17.869 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 44539 2024-09-14T06:31:17.869 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:17.869 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:31:17.869 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:17.869 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:31:17.869 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861462+0000", 2024-09-14T06:31:17.869 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:17.869 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:17.870 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:17.870 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:31:17.870 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861462+0000", 2024-09-14T06:31:17.870 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:17.870 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:17.870 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:17.870 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:31:17.870 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861463+0000", 2024-09-14T06:31:17.870 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.5200000000000002e-07 2024-09-14T06:31:17.870 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:17.870 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:17.870 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:31:17.871 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861465+0000", 2024-09-14T06:31:17.871 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.5189999999999999e-06 2024-09-14T06:31:17.871 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:17.871 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:17.871 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:31:17.871 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861471+0000", 2024-09-14T06:31:17.871 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.6899999999999997e-06 2024-09-14T06:31:17.871 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:17.871 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:17.871 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:31:17.871 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861475+0000", 2024-09-14T06:31:17.871 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.5730000000000001e-06 2024-09-14T06:31:17.871 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:17.872 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:17.872 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:31:17.872 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.925085+0000", 2024-09-14T06:31:17.872 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.063609902999999995 2024-09-14T06:31:17.872 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:17.872 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:17.872 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:31:17.872 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.925145+0000", 2024-09-14T06:31:17.872 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.9383999999999999e-05 2024-09-14T06:31:17.872 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:17.872 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:17.872 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:31:17.872 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.925318+0000", 2024-09-14T06:31:17.872 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00017318499999999999 2024-09-14T06:31:17.873 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:17.873 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:31:17.873 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:17.873 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:17.873 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:17.873 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:44540 3.2 3:4190d882:::rbd_data.112dca2f6ae1.0000000000000002:17 [read 552960~4096 [fadvise_nocache]] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:31:17.873 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:43.861483+0000", 2024-09-14T06:31:17.873 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 93.920984918000002, 2024-09-14T06:31:17.873 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.064065681999999999, 2024-09-14T06:31:17.873 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:31:17.873 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:31:17.873 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:31:17.873 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:31:17.874 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:31:17.874 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:17.874 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 44540 2024-09-14T06:31:17.874 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:17.874 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:31:17.874 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:17.874 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:31:17.874 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861483+0000", 2024-09-14T06:31:17.874 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:17.874 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:17.874 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:17.874 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:31:17.874 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861483+0000", 2024-09-14T06:31:17.874 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:17.875 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:17.875 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:17.875 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:31:17.875 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.867079+0000", 2024-09-14T06:31:17.875 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0055966480000000001 2024-09-14T06:31:17.875 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:17.875 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:17.875 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:31:17.875 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.867086+0000", 2024-09-14T06:31:17.875 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.387e-06 2024-09-14T06:31:17.875 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:17.875 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:17.875 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:31:17.875 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.867094+0000", 2024-09-14T06:31:17.875 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.9419999999999995e-06 2024-09-14T06:31:17.876 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:17.876 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:17.876 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:31:17.876 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.867109+0000", 2024-09-14T06:31:17.876 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.5818999999999999e-05 2024-09-14T06:31:17.876 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:17.876 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:17.876 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:31:17.876 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.925354+0000", 2024-09-14T06:31:17.876 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.058244047 2024-09-14T06:31:17.876 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:17.876 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:17.876 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:31:17.876 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.925406+0000", 2024-09-14T06:31:17.876 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.2661999999999998e-05 2024-09-14T06:31:17.877 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:17.877 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:17.877 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:31:17.877 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.925548+0000", 2024-09-14T06:31:17.877 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000142177 2024-09-14T06:31:17.877 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:17.877 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:31:17.877 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:17.877 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:17.877 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:31:17.877 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-14T06:31:17.877 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-14T06:31:17.878 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-14T06:31:17.878 INFO:tasks.thrashosds.thrasher:primary_affinity 2024-09-14T06:31:17.878 INFO:tasks.thrashosds.thrasher:Setting osd 1 primary_affinity to 0.841109 2024-09-14T06:31:17.878 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 1 0.8411094417114813 2024-09-14T06:31:17.880 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-14T06:31:17.880 INFO:tasks.ceph.osd.4.smithi138.stderr:2024-09-14T06:31:17.878+0000 7f6c653c8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33615) UID: 0 2024-09-14T06:31:17.980 INFO:tasks.ceph.osd.3.smithi094.stderr:2024-09-14T06:31:17.979+0000 7feeeafe8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33655) UID: 0 2024-09-14T06:31:18.030 INFO:teuthology.orchestra.run.smithi138.stdout:{ 2024-09-14T06:31:18.031 INFO:teuthology.orchestra.run.smithi138.stdout: "ops": [], 2024-09-14T06:31:18.031 INFO:teuthology.orchestra.run.smithi138.stdout: "num_ops": 0 2024-09-14T06:31:18.031 INFO:teuthology.orchestra.run.smithi138.stdout:} 2024-09-14T06:31:18.040 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-14T06:31:18.040 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-14T06:31:18.085 INFO:tasks.ceph.osd.5.smithi138.stderr:2024-09-14T06:31:18.084+0000 7fe13746f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33618) UID: 0 2024-09-14T06:31:18.142 INFO:teuthology.orchestra.run.smithi138.stdout:{ 2024-09-14T06:31:18.143 INFO:teuthology.orchestra.run.smithi138.stdout: "ops": [], 2024-09-14T06:31:18.143 INFO:teuthology.orchestra.run.smithi138.stdout: "complaint_time": 30, 2024-09-14T06:31:18.143 INFO:teuthology.orchestra.run.smithi138.stdout: "num_blocked_ops": 0 2024-09-14T06:31:18.143 INFO:teuthology.orchestra.run.smithi138.stdout:} 2024-09-14T06:31:18.152 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-14T06:31:18.179 INFO:teuthology.orchestra.run.smithi094.stdout:osd.0: {} 2024-09-14T06:31:18.179 INFO:teuthology.orchestra.run.smithi094.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-14T06:31:18.182 INFO:tasks.ceph.osd.1.smithi094.stderr:2024-09-14T06:31:18.181+0000 7f328e47a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33641) UID: 0 2024-09-14T06:31:18.195 INFO:teuthology.orchestra.run.smithi094.stdout:osd.1: {} 2024-09-14T06:31:18.195 INFO:teuthology.orchestra.run.smithi094.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-14T06:31:18.211 INFO:teuthology.orchestra.run.smithi094.stdout:osd.2: {} 2024-09-14T06:31:18.211 INFO:teuthology.orchestra.run.smithi094.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-14T06:31:18.227 INFO:teuthology.orchestra.run.smithi094.stdout:osd.3: {} 2024-09-14T06:31:18.227 INFO:teuthology.orchestra.run.smithi094.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-14T06:31:18.245 INFO:teuthology.orchestra.run.smithi094.stdout:osd.4: {} 2024-09-14T06:31:18.245 INFO:teuthology.orchestra.run.smithi094.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-14T06:31:18.261 INFO:teuthology.orchestra.run.smithi094.stdout:osd.5: {} 2024-09-14T06:31:18.261 INFO:teuthology.orchestra.run.smithi094.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-14T06:31:18.278 INFO:teuthology.orchestra.run.smithi094.stdout:osd.6: {} 2024-09-14T06:31:18.278 INFO:teuthology.orchestra.run.smithi094.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-14T06:31:18.283 INFO:tasks.ceph.osd.7.smithi138.stderr:2024-09-14T06:31:18.282+0000 7f3c6a21d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33623) UID: 0 2024-09-14T06:31:18.295 INFO:teuthology.orchestra.run.smithi094.stdout:osd.7: {} 2024-09-14T06:31:18.296 INFO:teuthology.orchestra.run.smithi094.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-14T06:31:18.296 INFO:teuthology.orchestra.run.smithi138.stdout:ERROR: (22) Invalid argument 2024-09-14T06:31:18.296 INFO:teuthology.orchestra.run.smithi138.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:31:18.305 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-14T06:31:18.389 INFO:tasks.ceph.osd.6.smithi138.stderr:2024-09-14T06:31:18.388+0000 7f8cd8849640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33620) UID: 0 2024-09-14T06:31:18.448 INFO:teuthology.orchestra.run.smithi138.stdout:ERROR: (22) Invalid argument 2024-09-14T06:31:18.448 INFO:teuthology.orchestra.run.smithi138.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:31:18.457 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-14T06:31:18.498 INFO:tasks.ceph.osd.7.smithi138.stderr:2024-09-14T06:31:18.497+0000 7f3c6a21d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33623) UID: 0 2024-09-14T06:31:18.585 INFO:tasks.ceph.osd.6.smithi138.stderr:2024-09-14T06:31:18.584+0000 7f8cd8849640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33620) UID: 0 2024-09-14T06:31:18.600 INFO:teuthology.orchestra.run.smithi138.stdout:ERROR: (22) Invalid argument 2024-09-14T06:31:18.600 INFO:teuthology.orchestra.run.smithi138.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:31:18.609 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-14T06:31:18.693 INFO:tasks.ceph.osd.4.smithi138.stderr:2024-09-14T06:31:18.692+0000 7f6c653c8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33615) UID: 0 2024-09-14T06:31:18.751 INFO:teuthology.orchestra.run.smithi138.stdout:ERROR: (22) Invalid argument 2024-09-14T06:31:18.751 INFO:teuthology.orchestra.run.smithi138.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:31:18.760 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-14T06:31:18.789 INFO:tasks.ceph.osd.2.smithi094.stderr:2024-09-14T06:31:18.788+0000 7faf3269b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33649) UID: 0 2024-09-14T06:31:18.858 INFO:teuthology.orchestra.run.smithi094.stderr:noscrub is unset 2024-09-14T06:31:18.862 INFO:teuthology.orchestra.run.smithi094.stderr:set osd.1 primary-affinity to 0.841109 (8551222) 2024-09-14T06:31:18.889 INFO:tasks.ceph.osd.7.smithi138.stderr:2024-09-14T06:31:18.888+0000 7f3c6a21d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33623) UID: 0 2024-09-14T06:31:18.906 INFO:teuthology.orchestra.run.smithi138.stdout:ERROR: (22) Invalid argument 2024-09-14T06:31:18.907 INFO:teuthology.orchestra.run.smithi138.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:31:18.915 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-14T06:31:18.999 INFO:tasks.ceph.osd.6.smithi138.stderr:2024-09-14T06:31:18.998+0000 7f8cd8849640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33620) UID: 0 2024-09-14T06:31:19.017 INFO:teuthology.orchestra.run.smithi094.stdout:checking clone #34, image image_client.0-clone34 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent35 2024-09-14T06:31:19.060 INFO:teuthology.orchestra.run.smithi138.stdout:ERROR: (22) Invalid argument 2024-09-14T06:31:19.060 INFO:teuthology.orchestra.run.smithi138.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:31:19.069 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-14T06:31:19.110 INFO:tasks.ceph.osd.5.smithi138.stderr:2024-09-14T06:31:19.109+0000 7fe13746f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33618) UID: 0 2024-09-14T06:31:19.192 INFO:tasks.ceph.osd.4.smithi138.stderr:2024-09-14T06:31:19.191+0000 7f6c653c8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33615) UID: 0 2024-09-14T06:31:19.212 INFO:teuthology.orchestra.run.smithi138.stdout:ERROR: (22) Invalid argument 2024-09-14T06:31:19.213 INFO:teuthology.orchestra.run.smithi138.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:31:19.221 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-14T06:31:19.284 INFO:teuthology.orchestra.run.smithi094.stdout:4528 writesame 0xb6eb321 thru 0xb6f9fd6 data_size 0x2(0xecb6 bytes) 2024-09-14T06:31:19.304 INFO:teuthology.orchestra.run.smithi094.stdout:4532 writesame 0x8030a6f thru 0x803dd0a data_size 0x6(0xd29c bytes) 2024-09-14T06:31:19.305 INFO:tasks.ceph.osd.5.smithi138.stderr:2024-09-14T06:31:19.304+0000 7fe13746f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33618) UID: 0 2024-09-14T06:31:19.309 INFO:teuthology.orchestra.run.smithi094.stdout:4534 read 0x2ce5321 thru 0x2cf0556 (0xb236 bytes) 2024-09-14T06:31:19.310 INFO:teuthology.orchestra.run.smithi094.stdout:4535 writesame 0xe8528d thru 0xe90a75 data_size 0x17(0xb7e9 bytes) 2024-09-14T06:31:19.312 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-14T06:31:19.312 INFO:teuthology.orchestra.run.smithi094.stdout:4538 read 0x4a462aa thru 0x4a52314 (0xc06b bytes) 2024-09-14T06:31:19.312 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-14T06:31:19.313 INFO:teuthology.orchestra.run.smithi094.stdout:4540 punch from 0xbc008a9 to 0xbc0828c, (0x79e3 bytes) 2024-09-14T06:31:19.313 INFO:teuthology.orchestra.run.smithi094.stdout:4541 write 0xb15e7cf thru 0xb1665db (0x7e0d bytes) 2024-09-14T06:31:19.318 INFO:teuthology.orchestra.run.smithi094.stdout:4542 trunc from 0xe688bf2 to 0xb802df1 2024-09-14T06:31:19.339 INFO:teuthology.orchestra.run.smithi094.stdout:4543 punch from 0x493aaa0 to 0x4944078, (0x95d8 bytes) 2024-09-14T06:31:19.350 INFO:teuthology.orchestra.run.smithi094.stdout:4544 writesame 0xeac8da8 thru 0xeacf422 data_size 0x37(0x667b bytes) 2024-09-14T06:31:19.350 INFO:teuthology.orchestra.run.smithi094.stdout:4545 read 0x8537332 thru 0x85472fa (0xffc9 bytes) 2024-09-14T06:31:19.351 INFO:teuthology.orchestra.run.smithi094.stdout:4546 write 0xe53211f thru 0xe53266f (0x551 bytes) 2024-09-14T06:31:19.354 INFO:teuthology.orchestra.run.smithi094.stdout:4547 writesame 0x814bc81 thru 0x8154be5 data_size 0x1(0x8f65 bytes) 2024-09-14T06:31:19.372 INFO:teuthology.orchestra.run.smithi094.stdout:4548 writesame 0xab4288 thru 0xac2a6d data_size 0x1(0xe7e6 bytes) 2024-09-14T06:31:19.376 INFO:teuthology.orchestra.run.smithi138.stdout:ERROR: (22) Invalid argument 2024-09-14T06:31:19.376 INFO:teuthology.orchestra.run.smithi138.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:31:19.386 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-14T06:31:19.390 INFO:teuthology.orchestra.run.smithi094.stdout:4549 read 0x357d5a5 thru 0x358c58c (0xefe8 bytes) 2024-09-14T06:31:19.391 INFO:teuthology.orchestra.run.smithi094.stdout:4551 read 0x784fd99 thru 0x7856bce (0x6e36 bytes) 2024-09-14T06:31:19.408 INFO:teuthology.orchestra.run.smithi094.stdout:4552 trunc from 0xeacf423 to 0x8747b 2024-09-14T06:31:19.428 INFO:tasks.ceph.osd.6.smithi138.stderr:2024-09-14T06:31:19.427+0000 7f8cd8849640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33620) UID: 0 2024-09-14T06:31:19.441 INFO:teuthology.orchestra.run.smithi094.stdout:4554 trunc from 0x8747b to 0x53d3622 2024-09-14T06:31:19.456 INFO:teuthology.orchestra.run.smithi094.stdout:4555 write 0x7144fc6 thru 0x7149f08 (0x4f43 bytes) 2024-09-14T06:31:19.459 INFO:teuthology.orchestra.run.smithi094.stdout:4556 read 0x6be53f3 thru 0x6bf2c45 (0xd853 bytes) 2024-09-14T06:31:19.459 INFO:teuthology.orchestra.run.smithi094.stdout:4557 punch from 0x5e842fd to 0x5e8b0be, (0x6dc1 bytes) 2024-09-14T06:31:19.459 INFO:teuthology.orchestra.run.smithi094.stdout:4558 read 0x5763815 thru 0x5770fe7 (0xd7d3 bytes) 2024-09-14T06:31:19.459 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-14T06:31:19.459 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-14T06:31:19.459 INFO:teuthology.orchestra.run.smithi094.stdout:4563 trunc from 0x7149f09 to 0x75a2058 2024-09-14T06:31:19.464 INFO:teuthology.orchestra.run.smithi094.stdout:4567 punch from 0x3e9336e to 0x3e95a95, (0x2727 bytes) 2024-09-14T06:31:19.473 INFO:teuthology.orchestra.run.smithi094.stdout:4570 writesame 0x8eb142e thru 0x8eb1e52 data_size 0x7(0xa25 bytes) 2024-09-14T06:31:19.484 INFO:teuthology.orchestra.run.smithi094.stdout:4575 write 0xcda216a thru 0xcdb04ba (0xe351 bytes) 2024-09-14T06:31:19.494 INFO:teuthology.orchestra.run.smithi094.stdout:4576 write 0xd007a39 thru 0xd015584 (0xdb4c bytes) 2024-09-14T06:31:19.494 INFO:tasks.ceph.osd.7.smithi138.stderr:2024-09-14T06:31:19.493+0000 7f3c6a21d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33623) UID: 0 2024-09-14T06:31:19.497 INFO:teuthology.orchestra.run.smithi094.stdout:4578 writesame 0xb61291b thru 0xb61dea3 data_size 0x7(0xb589 bytes) 2024-09-14T06:31:19.501 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-14T06:31:19.501 INFO:teuthology.orchestra.run.smithi094.stdout:4581 read 0x392c148 thru 0x392f52d (0x33e6 bytes) 2024-09-14T06:31:19.501 INFO:teuthology.orchestra.run.smithi094.stdout:4582 write 0x1500d13 thru 0x1503872 (0x2b60 bytes) 2024-09-14T06:31:19.516 INFO:teuthology.orchestra.run.smithi094.stdout:4583 trunc from 0xd015585 to 0x303069c 2024-09-14T06:31:19.529 INFO:teuthology.orchestra.run.smithi094.stdout:4585 read 0x961832 thru 0x967cb4 (0x6483 bytes) 2024-09-14T06:31:19.529 INFO:teuthology.orchestra.run.smithi094.stdout:4586 read 0x550d2d thru 0x55c4e6 (0xb7ba bytes) 2024-09-14T06:31:19.530 INFO:teuthology.orchestra.run.smithi094.stdout:4588 read 0x2ba9b45 thru 0x2bb7ef1 (0xe3ad bytes) 2024-09-14T06:31:19.530 INFO:teuthology.orchestra.run.smithi094.stdout:4589 read 0x1f8b517 thru 0x1f94ca9 (0x9793 bytes) 2024-09-14T06:31:19.533 INFO:teuthology.orchestra.run.smithi094.stdout:4590 trunc from 0x303069c to 0x81adc8 2024-09-14T06:31:19.537 INFO:teuthology.orchestra.run.smithi138.stdout:ERROR: (22) Invalid argument 2024-09-14T06:31:19.537 INFO:teuthology.orchestra.run.smithi138.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:31:19.542 INFO:teuthology.orchestra.run.smithi094.stdout:4591 read 0x7923e7 thru 0x79275a (0x374 bytes) 2024-09-14T06:31:19.546 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-14T06:31:19.548 INFO:teuthology.orchestra.run.smithi094.stdout:4593 write 0xb08e35 thru 0xb09bfe (0xdca bytes) 2024-09-14T06:31:19.568 INFO:teuthology.orchestra.run.smithi094.stdout:4595 write 0xa80cfd7 thru 0xa80fead (0x2ed7 bytes) 2024-09-14T06:31:19.572 INFO:teuthology.orchestra.run.smithi094.stdout:4597 read 0x5b5d131 thru 0x5b664f2 (0x93c2 bytes) 2024-09-14T06:31:19.572 INFO:teuthology.orchestra.run.smithi094.stdout:4598 punch from 0x72c4cc7 to 0x72cff67, (0xb2a0 bytes) 2024-09-14T06:31:19.572 INFO:teuthology.orchestra.run.smithi094.stdout:4600 punch from 0x7b7c465 to 0x7b7ee1c, (0x29b7 bytes) 2024-09-14T06:31:19.572 INFO:teuthology.orchestra.run.smithi094.stdout:4602 write 0x66e790f thru 0x66ef51f (0x7c11 bytes) 2024-09-14T06:31:19.583 INFO:teuthology.orchestra.run.smithi094.stdout:4603 write 0xd95ba8e thru 0xd969738 (0xdcab bytes) 2024-09-14T06:31:19.589 INFO:teuthology.orchestra.run.smithi094.stdout:4604 write 0x2371b thru 0x27d12 (0x45f8 bytes) 2024-09-14T06:31:19.592 INFO:teuthology.orchestra.run.smithi094.stdout:4605 read 0x7a43e31 thru 0x7a4eed1 (0xb0a1 bytes) 2024-09-14T06:31:19.592 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-14T06:31:19.593 INFO:teuthology.orchestra.run.smithi094.stdout:4607 trunc from 0xd969739 to 0xe8ceb01 2024-09-14T06:31:19.599 INFO:teuthology.orchestra.run.smithi094.stdout:4608 read 0x3cf576f thru 0x3cff65a (0x9eec bytes) 2024-09-14T06:31:19.599 INFO:teuthology.orchestra.run.smithi094.stdout:4609 read 0x6be1593 thru 0x6be4b06 (0x3574 bytes) 2024-09-14T06:31:19.600 INFO:teuthology.orchestra.run.smithi094.stdout:4610 write 0x427153b thru 0x427857b (0x7041 bytes) 2024-09-14T06:31:19.617 INFO:teuthology.orchestra.run.smithi094.stdout:4611 trunc from 0xe8ceb01 to 0x24831f6 2024-09-14T06:31:19.630 INFO:tasks.ceph.osd.5.smithi138.stderr:2024-09-14T06:31:19.629+0000 7fe13746f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33618) UID: 0 2024-09-14T06:31:19.632 INFO:teuthology.orchestra.run.smithi094.stdout:4615 punch from 0x1c8cab6 to 0x1c958a2, (0x8dec bytes) 2024-09-14T06:31:19.632 INFO:teuthology.orchestra.run.smithi094.stdout:4616 read 0x17d5d18 thru 0x17d9aea (0x3dd3 bytes) 2024-09-14T06:31:19.632 INFO:teuthology.orchestra.run.smithi094.stdout:4617 read 0x4f3126 thru 0x501165 (0xe040 bytes) 2024-09-14T06:31:19.640 INFO:teuthology.orchestra.run.smithi094.stdout:4619 trunc from 0x24831f6 to 0x86b48f8 2024-09-14T06:31:19.646 INFO:teuthology.orchestra.run.smithi094.stdout:4621 read 0x51af072 thru 0x51bdc5a (0xebe9 bytes) 2024-09-14T06:31:19.660 INFO:teuthology.orchestra.run.smithi094.stdout:4622 write 0xeb097fe thru 0xeb142aa (0xaaad bytes) 2024-09-14T06:31:19.663 INFO:teuthology.orchestra.run.smithi094.stdout:4623 read 0x2675aca thru 0x2679874 (0x3dab bytes) 2024-09-14T06:31:19.677 INFO:teuthology.orchestra.run.smithi094.stdout:4624 trunc from 0xeb142ab to 0x2c96e9b 2024-09-14T06:31:19.691 INFO:teuthology.orchestra.run.smithi094.stdout:4626 read 0x28f7c1 thru 0x299adb (0xa31b bytes) 2024-09-14T06:31:19.695 INFO:teuthology.orchestra.run.smithi138.stdout:ERROR: (22) Invalid argument 2024-09-14T06:31:19.695 INFO:teuthology.orchestra.run.smithi138.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:31:19.696 INFO:tasks.ceph.osd.3.smithi094.stderr:2024-09-14T06:31:19.695+0000 7feeeafe8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33655) UID: 0 2024-09-14T06:31:19.699 INFO:teuthology.orchestra.run.smithi094.stdout:4627 write 0x3de8863 thru 0x3defada (0x7278 bytes) 2024-09-14T06:31:19.704 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-14T06:31:19.711 INFO:teuthology.orchestra.run.smithi094.stdout:4628 write 0x7ab901e thru 0x7ac4a21 (0xba04 bytes) 2024-09-14T06:31:19.727 INFO:teuthology.orchestra.run.smithi094.stdout:4629 write 0xd93e67f thru 0xd941d18 (0x369a bytes) 2024-09-14T06:31:19.732 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-14T06:31:19.737 INFO:teuthology.orchestra.run.smithi094.stdout:4631 trunc from 0xd941d19 to 0x9169403 2024-09-14T06:31:19.751 INFO:teuthology.orchestra.run.smithi094.stdout:4632 write 0x52ab2f6 thru 0x52ab3cc (0xd7 bytes) 2024-09-14T06:31:19.753 INFO:teuthology.orchestra.run.smithi094.stdout:4633 read 0x51193ee thru 0x5119e8c (0xa9f bytes) 2024-09-14T06:31:19.753 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-14T06:31:19.753 INFO:teuthology.orchestra.run.smithi094.stdout:4636 read 0x874bdab thru 0x874ecab (0x2f01 bytes) 2024-09-14T06:31:19.753 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-14T06:31:19.756 INFO:teuthology.orchestra.run.smithi094.stdout:4638 trunc from 0x9169403 to 0xb380f2e 2024-09-14T06:31:19.763 INFO:teuthology.orchestra.run.smithi094.stdout:4639 writesame 0x9b8129b thru 0x9b8c196 data_size 0x6(0xaefc bytes) 2024-09-14T06:31:19.767 INFO:teuthology.orchestra.run.smithi094.stdout:4641 read 0x4e696e7 thru 0x4e6fb56 (0x6470 bytes) 2024-09-14T06:31:19.768 INFO:teuthology.orchestra.run.smithi094.stdout:4642 read 0xaa74a85 thru 0xaa7a9fe (0x5f7a bytes) 2024-09-14T06:31:19.768 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-14T06:31:19.768 INFO:teuthology.orchestra.run.smithi094.stdout:4646 writesame 0x7be3ffd thru 0x7be4edd data_size 0x1(0xee1 bytes) 2024-09-14T06:31:19.770 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-14T06:31:19.781 INFO:teuthology.orchestra.run.smithi094.stdout:4649 trunc from 0xb380f2e to 0x210b39e 2024-09-14T06:31:19.791 INFO:teuthology.orchestra.run.smithi094.stdout:4650 trunc from 0x210b39e to 0x2f0930a 2024-09-14T06:31:19.796 INFO:tasks.ceph.osd.5.smithi138.stderr:2024-09-14T06:31:19.795+0000 7fe13746f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33618) UID: 0 2024-09-14T06:31:19.807 INFO:teuthology.orchestra.run.smithi094.stdout:4651 write 0x647917a thru 0x6484f97 (0xbe1e bytes) 2024-09-14T06:31:19.818 INFO:teuthology.orchestra.run.smithi094.stdout:4652 trunc from 0x6484f98 to 0xe9abe48 2024-09-14T06:31:19.822 INFO:teuthology.orchestra.run.smithi094.stdout:ERROR: (22) Invalid argument 2024-09-14T06:31:19.822 INFO:teuthology.orchestra.run.smithi094.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:31:19.824 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-14T06:31:19.824 INFO:teuthology.orchestra.run.smithi094.stdout:4654 write 0x31a7b43 thru 0x31b1a6a (0x9f28 bytes) 2024-09-14T06:31:19.826 INFO:teuthology.orchestra.run.smithi094.stdout:4656 write 0x988eae thru 0x98fb7a (0x6ccd bytes) 2024-09-14T06:31:19.827 INFO:teuthology.orchestra.run.smithi094.stdout:4657 read 0xa8ce6a8 thru 0xa8db613 (0xcf6c bytes) 2024-09-14T06:31:19.832 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-14T06:31:19.832 INFO:teuthology.orchestra.run.smithi094.stdout:4658 trunc from 0xe9abe48 to 0xb4633cc 2024-09-14T06:31:19.851 INFO:teuthology.orchestra.run.smithi094.stdout:4660 write 0xdabf5e4 thru 0xdac7025 (0x7a42 bytes) 2024-09-14T06:31:19.854 INFO:teuthology.orchestra.run.smithi094.stdout:4661 read 0xab7e281 thru 0xab85c21 (0x79a1 bytes) 2024-09-14T06:31:19.855 INFO:teuthology.orchestra.run.smithi094.stdout:4662 writesame 0xb4df017 thru 0xb4e1f73 data_size 0x19(0x2f5d bytes) 2024-09-14T06:31:19.856 INFO:teuthology.orchestra.run.smithi094.stdout:4666 write 0x58201b2 thru 0x582aba4 (0xa9f3 bytes) 2024-09-14T06:31:19.859 INFO:teuthology.orchestra.run.smithi094.stdout:4669 read 0x334175c thru 0x3341e56 (0x6fb bytes) 2024-09-14T06:31:19.860 INFO:teuthology.orchestra.run.smithi094.stdout:4670 read 0x4efb104 thru 0x4efc473 (0x1370 bytes) 2024-09-14T06:31:19.860 INFO:teuthology.orchestra.run.smithi094.stdout:4671 writesame 0x5dee8f9 thru 0x5df4a17 data_size 0x17(0x611f bytes) 2024-09-14T06:31:19.869 INFO:teuthology.orchestra.run.smithi094.stdout:4672 trunc from 0xdac7026 to 0x78bc4e3 2024-09-14T06:31:19.887 INFO:teuthology.orchestra.run.smithi094.stdout:4673 writesame 0x67aea13 thru 0x67b1f82 data_size 0x3c(0x3570 bytes) 2024-09-14T06:31:19.888 INFO:teuthology.orchestra.run.smithi094.stdout:4674 writesame 0x4af4d1 thru 0x4b62cc data_size 0x2(0x6dfc bytes) 2024-09-14T06:31:19.896 INFO:teuthology.orchestra.run.smithi094.stdout:4675 writesame 0x7711f65 thru 0x771f0bd data_size 0x1(0xd159 bytes) 2024-09-14T06:31:19.897 INFO:tasks.ceph.osd.4.smithi138.stderr:2024-09-14T06:31:19.896+0000 7f6c653c8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33615) UID: 0 2024-09-14T06:31:19.936 INFO:teuthology.orchestra.run.smithi094.stdout:4676 write 0xe934db8 thru 0xe93acdb (0x5f24 bytes) 2024-09-14T06:31:19.946 INFO:teuthology.orchestra.run.smithi094.stdout:4677 trunc from 0xe93acdc to 0x8b0af9a 2024-09-14T06:31:19.966 INFO:teuthology.orchestra.run.smithi094.stdout:4680 read 0x63c494c thru 0x63cc98d (0x8042 bytes) 2024-09-14T06:31:19.966 INFO:teuthology.orchestra.run.smithi094.stdout:4681 clone 37 order 23 su 1048576 sc 8 2024-09-14T06:31:19.967 INFO:teuthology.orchestra.run.smithi094.stdout:ERROR: (22) Invalid argument 2024-09-14T06:31:19.967 INFO:teuthology.orchestra.run.smithi094.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:31:19.976 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-14T06:31:19.998 INFO:tasks.ceph.osd.4.smithi138.stderr:2024-09-14T06:31:19.996+0000 7f6c653c8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33615) UID: 0 2024-09-14T06:31:20.099 INFO:tasks.ceph.osd.6.smithi138.stderr:2024-09-14T06:31:20.098+0000 7f8cd8849640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33620) UID: 0 2024-09-14T06:31:20.119 INFO:teuthology.orchestra.run.smithi094.stdout:ERROR: (22) Invalid argument 2024-09-14T06:31:20.119 INFO:teuthology.orchestra.run.smithi094.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:31:20.128 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-14T06:31:20.200 INFO:tasks.ceph.osd.1.smithi094.stderr:2024-09-14T06:31:20.199+0000 7f328e47a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33641) UID: 0 2024-09-14T06:31:20.231 INFO:teuthology.orchestra.run.smithi094.stdout:ERROR: (22) Invalid argument 2024-09-14T06:31:20.231 INFO:teuthology.orchestra.run.smithi094.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:31:20.240 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-14T06:31:20.301 INFO:tasks.ceph.osd.7.smithi138.stderr:2024-09-14T06:31:20.300+0000 7f3c6a21d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33623) UID: 0 2024-09-14T06:31:20.307 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-14T06:31:20.385 INFO:teuthology.orchestra.run.smithi094.stdout:ERROR: (22) Invalid argument 2024-09-14T06:31:20.386 INFO:teuthology.orchestra.run.smithi094.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:31:20.394 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-14T06:31:20.440 INFO:tasks.ceph.osd.0.smithi094.stderr:2024-09-14T06:31:20.439+0000 7f07c9333640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33636) UID: 0 2024-09-14T06:31:20.447 INFO:teuthology.orchestra.run.smithi094.stdout:osd.0: {} 2024-09-14T06:31:20.447 INFO:teuthology.orchestra.run.smithi094.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-14T06:31:20.465 INFO:teuthology.orchestra.run.smithi094.stdout:osd.1: {} 2024-09-14T06:31:20.465 INFO:teuthology.orchestra.run.smithi094.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-14T06:31:20.481 INFO:teuthology.orchestra.run.smithi094.stdout:osd.2: {} 2024-09-14T06:31:20.481 INFO:teuthology.orchestra.run.smithi094.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-14T06:31:20.497 INFO:teuthology.orchestra.run.smithi094.stdout:osd.3: {} 2024-09-14T06:31:20.497 INFO:teuthology.orchestra.run.smithi094.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-14T06:31:20.503 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-14T06:31:20.503 INFO:teuthology.orchestra.run.smithi094.stdout: "size": 20, 2024-09-14T06:31:20.503 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 600, 2024-09-14T06:31:20.503 INFO:teuthology.orchestra.run.smithi094.stdout: "ops": [ 2024-09-14T06:31:20.503 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:20.503 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:14918 3.0 3:05c39535:::rbd_data.112d92027590.0000000000000009:f [read 795648~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:31:20.503 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:26.083834+0000", 2024-09-14T06:31:20.503 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 114.41723646200001, 2024-09-14T06:31:20.503 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.041776287000000002, 2024-09-14T06:31:20.503 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:31:20.503 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:31:20.504 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:31:20.504 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:31:20.504 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:31:20.504 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:20.504 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 14918 2024-09-14T06:31:20.504 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:20.504 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:31:20.504 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:20.504 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:31:20.504 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.083834+0000", 2024-09-14T06:31:20.504 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:20.504 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:20.504 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:20.505 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:31:20.505 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.083834+0000", 2024-09-14T06:31:20.505 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:20.505 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:20.505 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:20.505 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:31:20.505 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.083834+0000", 2024-09-14T06:31:20.505 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.0100000000000001e-07 2024-09-14T06:31:20.505 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:20.505 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:20.505 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:31:20.505 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.083835+0000", 2024-09-14T06:31:20.505 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.3900000000000003e-07 2024-09-14T06:31:20.505 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:20.506 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:20.506 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:31:20.506 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.083838+0000", 2024-09-14T06:31:20.506 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.9830000000000001e-06 2024-09-14T06:31:20.506 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:20.506 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:20.506 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:31:20.506 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.083840+0000", 2024-09-14T06:31:20.506 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.4679999999999999e-06 2024-09-14T06:31:20.506 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:20.506 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:20.506 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:31:20.506 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.125455+0000", 2024-09-14T06:31:20.506 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.041614578999999999 2024-09-14T06:31:20.507 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:20.507 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:20.507 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:31:20.507 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.125496+0000", 2024-09-14T06:31:20.507 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.1047000000000002e-05 2024-09-14T06:31:20.507 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:20.507 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:20.507 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:31:20.507 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.125610+0000", 2024-09-14T06:31:20.507 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00011407 2024-09-14T06:31:20.507 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:20.507 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:31:20.507 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:20.508 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:20.508 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:20.508 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:14919 3.0 3:05c39535:::rbd_data.112d92027590.0000000000000009:f [read 805888~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:31:20.508 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:26.083843+0000", 2024-09-14T06:31:20.508 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 114.41722671399999, 2024-09-14T06:31:20.508 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.041992356000000002, 2024-09-14T06:31:20.508 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:31:20.508 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:31:20.508 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:31:20.508 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:31:20.508 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:31:20.508 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:20.508 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 14919 2024-09-14T06:31:20.508 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:20.509 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:31:20.509 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:20.509 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:31:20.509 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.083843+0000", 2024-09-14T06:31:20.509 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:20.509 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:20.509 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:20.509 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:31:20.509 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.083843+0000", 2024-09-14T06:31:20.509 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:20.509 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:20.509 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:20.509 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:31:20.509 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.083844+0000", 2024-09-14T06:31:20.510 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.3300000000000001e-07 2024-09-14T06:31:20.510 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:20.510 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:20.510 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:31:20.510 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.083845+0000", 2024-09-14T06:31:20.510 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.4729999999999999e-06 2024-09-14T06:31:20.510 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:20.510 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:20.510 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:31:20.510 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.083850+0000", 2024-09-14T06:31:20.510 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.0130000000000001e-06 2024-09-14T06:31:20.510 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:20.510 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:20.510 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:31:20.510 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.083854+0000", 2024-09-14T06:31:20.511 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.4759999999999998e-06 2024-09-14T06:31:20.511 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:20.511 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:20.511 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:31:20.511 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.125636+0000", 2024-09-14T06:31:20.511 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.041782683000000001 2024-09-14T06:31:20.511 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:20.511 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:20.511 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:31:20.511 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.125680+0000", 2024-09-14T06:31:20.511 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.3614e-05 2024-09-14T06:31:20.511 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:20.511 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:20.511 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:31:20.512 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.125836+0000", 2024-09-14T06:31:20.512 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000155864 2024-09-14T06:31:20.512 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:20.512 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:31:20.512 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:20.512 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:20.512 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:20.512 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:14920 3.0 3:05c39535:::rbd_data.112d92027590.0000000000000009:f [read 816128~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:31:20.512 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:26.083868+0000", 2024-09-14T06:31:20.512 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 114.417202053, 2024-09-14T06:31:20.512 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.042166771999999998, 2024-09-14T06:31:20.512 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:31:20.512 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:31:20.512 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:31:20.512 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:31:20.513 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:31:20.513 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:20.513 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 14920 2024-09-14T06:31:20.513 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:20.513 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:31:20.513 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:20.513 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:31:20.513 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.083868+0000", 2024-09-14T06:31:20.513 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:20.513 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:20.513 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:20.513 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:31:20.513 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.083868+0000", 2024-09-14T06:31:20.513 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:20.514 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:20.514 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:20.514 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:31:20.514 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.083869+0000", 2024-09-14T06:31:20.514 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.99e-07 2024-09-14T06:31:20.514 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:20.514 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:20.514 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:31:20.514 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.083870+0000", 2024-09-14T06:31:20.514 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.782e-06 2024-09-14T06:31:20.514 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:20.514 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:20.514 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:31:20.514 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.083876+0000", 2024-09-14T06:31:20.515 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.3809999999999998e-06 2024-09-14T06:31:20.515 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:20.515 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:20.515 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:31:20.515 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.083882+0000", 2024-09-14T06:31:20.515 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.9479999999999996e-06 2024-09-14T06:31:20.515 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:20.515 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:20.515 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:31:20.515 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.125878+0000", 2024-09-14T06:31:20.515 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.041996264999999998 2024-09-14T06:31:20.515 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:20.515 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:20.516 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:31:20.516 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.125929+0000", 2024-09-14T06:31:20.516 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.0498999999999999e-05 2024-09-14T06:31:20.516 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:20.516 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:20.516 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:31:20.516 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.126035+0000", 2024-09-14T06:31:20.516 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000106298 2024-09-14T06:31:20.516 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:20.516 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:31:20.516 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:20.516 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:20.516 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:20.516 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:14921 3.0 3:05c39535:::rbd_data.112d92027590.0000000000000009:f [read 826368~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:31:20.517 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:26.083889+0000", 2024-09-14T06:31:20.517 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 114.417181043, 2024-09-14T06:31:20.517 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.042435309999999997, 2024-09-14T06:31:20.517 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:31:20.517 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:31:20.517 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:31:20.517 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:31:20.517 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:31:20.517 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:20.517 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 14921 2024-09-14T06:31:20.517 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:20.517 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:31:20.518 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:20.518 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:31:20.518 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.083889+0000", 2024-09-14T06:31:20.518 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:20.518 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:20.518 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:20.518 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:31:20.518 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.083889+0000", 2024-09-14T06:31:20.518 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:20.518 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:20.518 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:20.518 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:31:20.518 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.086033+0000", 2024-09-14T06:31:20.519 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0021444179999999999 2024-09-14T06:31:20.519 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:20.519 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:20.519 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:31:20.519 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.086038+0000", 2024-09-14T06:31:20.519 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.0969999999999999e-06 2024-09-14T06:31:20.519 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:20.519 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:20.519 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:31:20.519 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.086046+0000", 2024-09-14T06:31:20.519 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.9440000000000005e-06 2024-09-14T06:31:20.519 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:20.519 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:20.519 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:31:20.520 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.086050+0000", 2024-09-14T06:31:20.520 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.69e-06 2024-09-14T06:31:20.520 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:20.520 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:20.520 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:31:20.520 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.126080+0000", 2024-09-14T06:31:20.520 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.040030237000000003 2024-09-14T06:31:20.520 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:20.520 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:20.520 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:31:20.520 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.126163+0000", 2024-09-14T06:31:20.520 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.2071999999999996e-05 2024-09-14T06:31:20.521 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:20.521 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:20.521 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:31:20.521 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.126324+0000", 2024-09-14T06:31:20.521 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00016185200000000001 2024-09-14T06:31:20.521 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:20.521 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:31:20.521 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:20.521 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:20.521 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:20.521 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:55627 3.0 3:06490baf:::rbd_data.112dca2f6ae1.0000000000000030:17 [read 2002944~4096 [fadvise_nocache]] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:31:20.521 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:44.599966+0000", 2024-09-14T06:31:20.521 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 95.901104094999994, 2024-09-14T06:31:20.522 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.041868546999999999, 2024-09-14T06:31:20.522 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:31:20.522 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:31:20.522 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:31:20.522 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:31:20.522 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:31:20.522 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:20.522 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 55627 2024-09-14T06:31:20.522 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:20.522 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:31:20.522 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:20.522 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:31:20.522 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.599966+0000", 2024-09-14T06:31:20.522 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:20.523 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:20.523 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:20.523 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:31:20.523 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.599966+0000", 2024-09-14T06:31:20.523 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:20.523 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:20.523 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:20.523 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:31:20.523 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.599966+0000", 2024-09-14T06:31:20.523 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.0900000000000001e-07 2024-09-14T06:31:20.523 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:20.523 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:20.523 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:31:20.523 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.599967+0000", 2024-09-14T06:31:20.523 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.64e-07 2024-09-14T06:31:20.524 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:20.524 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:20.524 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:31:20.524 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.599970+0000", 2024-09-14T06:31:20.524 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.9670000000000001e-06 2024-09-14T06:31:20.524 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:20.524 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:20.524 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:31:20.524 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.599973+0000", 2024-09-14T06:31:20.524 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.632e-06 2024-09-14T06:31:20.524 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:20.524 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:20.524 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:31:20.524 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.641665+0000", 2024-09-14T06:31:20.525 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.041692547000000003 2024-09-14T06:31:20.525 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:20.525 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:20.525 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:31:20.525 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.641740+0000", 2024-09-14T06:31:20.525 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.4559000000000002e-05 2024-09-14T06:31:20.525 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:20.525 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:20.525 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:31:20.525 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.641835+0000", 2024-09-14T06:31:20.525 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.4668999999999998e-05 2024-09-14T06:31:20.525 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:20.525 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:31:20.525 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:20.526 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:20.526 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:20.526 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:55628 3.0 3:06490baf:::rbd_data.112dca2f6ae1.0000000000000030:17 [read 2027520~4096 [fadvise_nocache]] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:31:20.526 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:44.599976+0000", 2024-09-14T06:31:20.526 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 95.901094307999998, 2024-09-14T06:31:20.526 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.041991621, 2024-09-14T06:31:20.526 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:31:20.526 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:31:20.526 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:31:20.526 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:31:20.526 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:31:20.526 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:20.526 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 55628 2024-09-14T06:31:20.526 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:20.526 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:31:20.526 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:20.527 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:31:20.527 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.599976+0000", 2024-09-14T06:31:20.527 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:20.527 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:20.527 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:20.527 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:31:20.527 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.599976+0000", 2024-09-14T06:31:20.527 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:20.527 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:20.527 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:20.527 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:31:20.527 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.599976+0000", 2024-09-14T06:31:20.527 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.1299999999999999e-07 2024-09-14T06:31:20.528 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:20.528 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:20.528 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:31:20.528 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.599977+0000", 2024-09-14T06:31:20.528 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.7999999999999993e-07 2024-09-14T06:31:20.528 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:20.528 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:20.528 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:31:20.528 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.599980+0000", 2024-09-14T06:31:20.528 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.01e-06 2024-09-14T06:31:20.528 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:20.528 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:20.529 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:31:20.529 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.599983+0000", 2024-09-14T06:31:20.529 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.5409999999999999e-06 2024-09-14T06:31:20.529 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:20.529 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:20.529 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:31:20.529 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.641869+0000", 2024-09-14T06:31:20.529 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.041886235000000001 2024-09-14T06:31:20.529 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:20.529 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:20.529 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:31:20.529 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.641913+0000", 2024-09-14T06:31:20.530 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.4357000000000002e-05 2024-09-14T06:31:20.530 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:20.530 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:20.530 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:31:20.530 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.641967+0000", 2024-09-14T06:31:20.530 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.4284999999999999e-05 2024-09-14T06:31:20.530 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:20.530 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:31:20.530 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:20.530 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:20.530 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:20.530 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:55629 3.0 3:06490baf:::rbd_data.112dca2f6ae1.0000000000000030:17 [read 2052096~4096 [fadvise_nocache]] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:31:20.530 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:44.599985+0000", 2024-09-14T06:31:20.530 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 95.901084601999997, 2024-09-14T06:31:20.531 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.042183375000000002, 2024-09-14T06:31:20.531 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:31:20.531 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:31:20.531 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:31:20.531 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:31:20.531 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:31:20.531 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:20.531 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 55629 2024-09-14T06:31:20.531 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:20.531 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:31:20.531 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:20.531 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:31:20.532 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.599985+0000", 2024-09-14T06:31:20.532 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:20.532 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:20.532 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:20.532 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:31:20.532 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.599985+0000", 2024-09-14T06:31:20.532 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:20.532 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:20.532 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:20.532 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:31:20.532 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.599986+0000", 2024-09-14T06:31:20.532 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.05e-07 2024-09-14T06:31:20.532 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:20.532 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:20.533 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:31:20.533 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.599987+0000", 2024-09-14T06:31:20.533 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.8400000000000002e-07 2024-09-14T06:31:20.533 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:20.533 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:20.533 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:31:20.533 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.599990+0000", 2024-09-14T06:31:20.533 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.5640000000000001e-06 2024-09-14T06:31:20.533 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:20.533 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:20.533 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:31:20.533 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.599993+0000", 2024-09-14T06:31:20.533 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.615e-06 2024-09-14T06:31:20.534 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:20.534 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:20.534 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:31:20.534 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.642003+0000", 2024-09-14T06:31:20.534 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.042010345999999997 2024-09-14T06:31:20.534 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:20.534 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:20.534 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:31:20.534 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.642080+0000", 2024-09-14T06:31:20.534 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.6538999999999993e-05 2024-09-14T06:31:20.534 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:20.534 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:20.534 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:31:20.535 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.642169+0000", 2024-09-14T06:31:20.535 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.9122000000000005e-05 2024-09-14T06:31:20.535 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:20.535 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:31:20.535 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:20.535 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:20.535 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:20.535 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:55630 3.0 3:06490baf:::rbd_data.112dca2f6ae1.0000000000000030:17 [read 2076672~4096 [fadvise_nocache]] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:31:20.535 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:44.599996+0000", 2024-09-14T06:31:20.535 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 95.901074124000004, 2024-09-14T06:31:20.535 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.042299615999999998, 2024-09-14T06:31:20.535 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:31:20.535 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:31:20.536 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:31:20.536 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:31:20.536 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:31:20.536 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:20.536 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 55630 2024-09-14T06:31:20.536 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:20.536 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:31:20.536 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:20.536 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:31:20.536 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.599996+0000", 2024-09-14T06:31:20.536 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:20.536 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:20.536 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:20.537 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:31:20.537 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.599996+0000", 2024-09-14T06:31:20.537 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:20.537 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:20.537 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:20.537 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:31:20.537 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.599996+0000", 2024-09-14T06:31:20.537 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.1500000000000001e-07 2024-09-14T06:31:20.537 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:20.537 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:20.537 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:31:20.537 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.599997+0000", 2024-09-14T06:31:20.537 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.33e-07 2024-09-14T06:31:20.538 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:20.538 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:20.538 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:31:20.538 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600000+0000", 2024-09-14T06:31:20.538 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.0240000000000002e-06 2024-09-14T06:31:20.538 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:20.538 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:20.538 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:31:20.538 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600003+0000", 2024-09-14T06:31:20.538 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.3640000000000002e-06 2024-09-14T06:31:20.538 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:20.538 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:20.538 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:31:20.539 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.642200+0000", 2024-09-14T06:31:20.539 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.042197571000000003 2024-09-14T06:31:20.539 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:20.539 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:20.539 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:31:20.539 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.642243+0000", 2024-09-14T06:31:20.539 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.2765000000000001e-05 2024-09-14T06:31:20.539 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:20.539 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:20.539 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:31:20.539 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.642296+0000", 2024-09-14T06:31:20.539 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.2744000000000002e-05 2024-09-14T06:31:20.539 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:20.539 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:31:20.540 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:20.540 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:20.540 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:20.540 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:55631 3.0 3:06490baf:::rbd_data.112dca2f6ae1.0000000000000030:17 [read 1048576~4096 [fadvise_nocache]] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:31:20.540 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:44.600006+0000", 2024-09-14T06:31:20.540 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 95.901064536000007, 2024-09-14T06:31:20.540 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.042487905999999999, 2024-09-14T06:31:20.540 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:31:20.540 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:31:20.540 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:31:20.540 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:31:20.540 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:31:20.540 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:20.540 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 55631 2024-09-14T06:31:20.540 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:20.541 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:31:20.541 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:20.541 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:31:20.541 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600006+0000", 2024-09-14T06:31:20.541 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:20.541 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:20.541 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:20.541 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:31:20.541 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600006+0000", 2024-09-14T06:31:20.541 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:20.541 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:20.541 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:20.541 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:31:20.541 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600006+0000", 2024-09-14T06:31:20.542 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.0800000000000001e-07 2024-09-14T06:31:20.542 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:20.542 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:20.542 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:31:20.542 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600007+0000", 2024-09-14T06:31:20.542 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.7199999999999997e-07 2024-09-14T06:31:20.542 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:20.542 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:20.542 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:31:20.542 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600010+0000", 2024-09-14T06:31:20.542 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.0280000000000001e-06 2024-09-14T06:31:20.542 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:20.542 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:20.542 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:31:20.542 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600012+0000", 2024-09-14T06:31:20.543 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.475e-06 2024-09-14T06:31:20.543 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:20.543 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:20.543 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:31:20.543 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.642329+0000", 2024-09-14T06:31:20.543 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.042317157000000001 2024-09-14T06:31:20.543 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:20.543 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:20.543 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:31:20.543 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.642403+0000", 2024-09-14T06:31:20.543 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.3941000000000002e-05 2024-09-14T06:31:20.543 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:20.543 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:20.543 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:31:20.544 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.642493+0000", 2024-09-14T06:31:20.544 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.0124999999999995e-05 2024-09-14T06:31:20.544 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:20.544 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:31:20.544 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:20.544 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:20.544 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:20.544 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:55632 3.0 3:06490baf:::rbd_data.112dca2f6ae1.0000000000000030:17 [read 1073152~4096 [fadvise_nocache]] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:31:20.544 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:44.600015+0000", 2024-09-14T06:31:20.544 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 95.901054783000006, 2024-09-14T06:31:20.544 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.042604922000000003, 2024-09-14T06:31:20.544 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:31:20.544 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:31:20.544 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:31:20.545 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:31:20.545 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:31:20.545 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:20.545 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 55632 2024-09-14T06:31:20.545 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:20.545 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:31:20.545 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:20.545 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:31:20.545 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600015+0000", 2024-09-14T06:31:20.545 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:20.545 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:20.545 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:20.545 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:31:20.545 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600015+0000", 2024-09-14T06:31:20.545 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:20.546 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:20.546 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:20.546 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:31:20.546 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600016+0000", 2024-09-14T06:31:20.546 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.1e-07 2024-09-14T06:31:20.546 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:20.546 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:20.546 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:31:20.546 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600017+0000", 2024-09-14T06:31:20.546 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.9900000000000009e-07 2024-09-14T06:31:20.546 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:20.546 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:20.546 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:31:20.547 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600020+0000", 2024-09-14T06:31:20.547 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.4170000000000001e-06 2024-09-14T06:31:20.547 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:20.547 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:20.547 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:31:20.547 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600023+0000", 2024-09-14T06:31:20.547 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.588e-06 2024-09-14T06:31:20.547 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:20.547 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:20.547 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:31:20.547 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.642526+0000", 2024-09-14T06:31:20.547 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.042503236999999999 2024-09-14T06:31:20.547 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:20.547 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:20.547 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:31:20.548 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.642568+0000", 2024-09-14T06:31:20.548 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.1925000000000003e-05 2024-09-14T06:31:20.548 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:20.548 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:20.548 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:31:20.548 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.642620+0000", 2024-09-14T06:31:20.548 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.2546000000000001e-05 2024-09-14T06:31:20.548 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:20.548 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:31:20.548 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:20.548 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:20.548 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:20.548 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:55633 3.0 3:06490baf:::rbd_data.112dca2f6ae1.0000000000000030:17 [read 1097728~4096 [fadvise_nocache]] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:31:20.548 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:44.600026+0000", 2024-09-14T06:31:20.548 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 95.901044514000006, 2024-09-14T06:31:20.549 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.042797474000000002, 2024-09-14T06:31:20.549 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:31:20.549 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:31:20.549 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:31:20.549 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:31:20.549 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:31:20.549 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:20.549 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 55633 2024-09-14T06:31:20.549 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:20.549 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:31:20.549 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:20.549 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:31:20.549 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600026+0000", 2024-09-14T06:31:20.550 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:20.550 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:20.550 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:20.550 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:31:20.550 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600026+0000", 2024-09-14T06:31:20.550 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:20.550 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:20.550 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:20.550 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:31:20.550 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600026+0000", 2024-09-14T06:31:20.550 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.3099999999999999e-07 2024-09-14T06:31:20.550 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:20.550 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:20.550 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:31:20.550 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600029+0000", 2024-09-14T06:31:20.551 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.304e-06 2024-09-14T06:31:20.551 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:20.551 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:20.551 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:31:20.551 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600034+0000", 2024-09-14T06:31:20.551 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.1429999999999997e-06 2024-09-14T06:31:20.551 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:20.551 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:20.551 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:31:20.551 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600038+0000", 2024-09-14T06:31:20.551 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.692e-06 2024-09-14T06:31:20.551 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:20.551 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:20.551 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:31:20.552 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.642654+0000", 2024-09-14T06:31:20.552 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.042615564000000002 2024-09-14T06:31:20.552 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:20.552 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:20.552 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:31:20.552 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.642728+0000", 2024-09-14T06:31:20.552 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.4018999999999997e-05 2024-09-14T06:31:20.552 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:20.552 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:20.552 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:31:20.552 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.642823+0000", 2024-09-14T06:31:20.552 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.5520999999999996e-05 2024-09-14T06:31:20.552 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:20.552 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:31:20.553 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:20.553 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:20.553 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:20.553 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:55634 3.0 3:06490baf:::rbd_data.112dca2f6ae1.0000000000000030:17 [read 1122304~4096 [fadvise_nocache]] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:31:20.553 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:44.600042+0000", 2024-09-14T06:31:20.553 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 95.901028037000003, 2024-09-14T06:31:20.553 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.042903729000000002, 2024-09-14T06:31:20.553 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:31:20.553 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:31:20.553 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:31:20.553 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:31:20.553 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:31:20.553 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:20.553 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 55634 2024-09-14T06:31:20.554 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:20.554 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:31:20.554 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:20.554 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:31:20.554 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600042+0000", 2024-09-14T06:31:20.554 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:20.554 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:20.554 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:20.554 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:31:20.554 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600042+0000", 2024-09-14T06:31:20.554 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:20.554 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:20.554 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:20.555 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:31:20.555 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600043+0000", 2024-09-14T06:31:20.555 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.7000000000000005e-07 2024-09-14T06:31:20.555 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:20.555 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:20.555 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:31:20.555 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600044+0000", 2024-09-14T06:31:20.555 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.742e-06 2024-09-14T06:31:20.555 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:20.555 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:20.555 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:31:20.555 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600050+0000", 2024-09-14T06:31:20.555 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.0800000000000002e-06 2024-09-14T06:31:20.555 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:20.555 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:20.556 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:31:20.556 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600055+0000", 2024-09-14T06:31:20.556 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.3889999999999998e-06 2024-09-14T06:31:20.556 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:20.556 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:20.556 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:31:20.556 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.642845+0000", 2024-09-14T06:31:20.556 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.042790040000000001 2024-09-14T06:31:20.556 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:20.556 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:20.556 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:31:20.556 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.642888+0000", 2024-09-14T06:31:20.556 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.2790000000000002e-05 2024-09-14T06:31:20.557 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:20.557 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:20.557 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:31:20.557 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.642946+0000", 2024-09-14T06:31:20.557 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.8118000000000002e-05 2024-09-14T06:31:20.557 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:20.557 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:31:20.557 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:20.557 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:20.557 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:20.557 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:55635 3.0 3:06490baf:::rbd_data.112dca2f6ae1.0000000000000030:17 [read 1146880~4096 [fadvise_nocache]] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:31:20.557 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:44.600061+0000", 2024-09-14T06:31:20.557 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 95.901009466000005, 2024-09-14T06:31:20.558 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.043092695, 2024-09-14T06:31:20.558 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:31:20.558 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:31:20.558 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:31:20.558 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:31:20.558 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:31:20.558 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:20.558 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 55635 2024-09-14T06:31:20.558 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:20.558 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:31:20.558 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:20.558 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:31:20.558 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600061+0000", 2024-09-14T06:31:20.558 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:20.559 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:20.559 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:20.559 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:31:20.559 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600061+0000", 2024-09-14T06:31:20.559 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:20.559 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:20.559 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:20.559 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:31:20.559 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600061+0000", 2024-09-14T06:31:20.559 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.3099999999999999e-07 2024-09-14T06:31:20.559 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:20.559 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:20.559 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:31:20.560 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600063+0000", 2024-09-14T06:31:20.560 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.6220000000000001e-06 2024-09-14T06:31:20.560 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:20.560 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:20.560 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:31:20.560 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600068+0000", 2024-09-14T06:31:20.560 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.4790000000000004e-06 2024-09-14T06:31:20.560 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:20.560 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:20.560 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:31:20.560 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600072+0000", 2024-09-14T06:31:20.560 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.3100000000000002e-06 2024-09-14T06:31:20.560 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:20.561 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:20.561 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:31:20.561 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.642987+0000", 2024-09-14T06:31:20.561 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.042914489 2024-09-14T06:31:20.561 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:20.561 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:20.561 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:31:20.561 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.643062+0000", 2024-09-14T06:31:20.561 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.5011000000000004e-05 2024-09-14T06:31:20.561 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:20.561 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:20.561 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:31:20.561 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.643153+0000", 2024-09-14T06:31:20.562 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.1452999999999993e-05 2024-09-14T06:31:20.562 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:20.562 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:31:20.562 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:20.562 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:20.562 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:20.562 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:55636 3.0 3:06490baf:::rbd_data.112dca2f6ae1.0000000000000030:17 [read 1171456~4096 [fadvise_nocache]] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:31:20.562 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:44.600078+0000", 2024-09-14T06:31:20.562 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 95.900992054, 2024-09-14T06:31:20.562 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.043207966, 2024-09-14T06:31:20.562 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:31:20.562 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:31:20.562 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:31:20.562 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:31:20.563 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:31:20.563 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:20.563 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 55636 2024-09-14T06:31:20.563 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:20.563 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:31:20.563 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:20.563 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:31:20.563 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600078+0000", 2024-09-14T06:31:20.563 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:20.563 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:20.563 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:20.563 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:31:20.563 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600078+0000", 2024-09-14T06:31:20.563 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:20.564 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:20.564 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:20.564 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:31:20.564 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600078+0000", 2024-09-14T06:31:20.564 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.34e-07 2024-09-14T06:31:20.564 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:20.564 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:20.564 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:31:20.564 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600080+0000", 2024-09-14T06:31:20.564 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.641e-06 2024-09-14T06:31:20.564 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:20.564 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:20.564 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:31:20.565 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600086+0000", 2024-09-14T06:31:20.565 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.3860000000000004e-06 2024-09-14T06:31:20.565 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:20.565 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:20.565 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:31:20.565 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600091+0000", 2024-09-14T06:31:20.565 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.5970000000000002e-06 2024-09-14T06:31:20.565 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:20.565 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:20.565 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:31:20.565 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.643184+0000", 2024-09-14T06:31:20.565 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.043092893 2024-09-14T06:31:20.565 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:20.566 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:20.566 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:31:20.566 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.643228+0000", 2024-09-14T06:31:20.566 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.4536999999999997e-05 2024-09-14T06:31:20.566 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:20.566 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:20.566 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:31:20.566 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.643286+0000", 2024-09-14T06:31:20.566 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.7578000000000003e-05 2024-09-14T06:31:20.566 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:20.566 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:31:20.566 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:20.566 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:20.566 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:20.567 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:55637 3.0 3:06490baf:::rbd_data.112dca2f6ae1.0000000000000030:17 [read 1196032~4096 [fadvise_nocache]] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:31:20.567 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:44.600097+0000", 2024-09-14T06:31:20.567 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 95.900973489999998, 2024-09-14T06:31:20.567 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.043399124999999997, 2024-09-14T06:31:20.567 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:31:20.567 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:31:20.567 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:31:20.567 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:31:20.567 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:31:20.568 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:20.568 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 55637 2024-09-14T06:31:20.568 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:20.568 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:31:20.568 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:20.568 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:31:20.568 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600097+0000", 2024-09-14T06:31:20.568 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:20.568 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:20.568 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:20.568 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:31:20.568 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600097+0000", 2024-09-14T06:31:20.568 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:20.569 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:20.569 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:20.569 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:31:20.569 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.604372+0000", 2024-09-14T06:31:20.569 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0042751489999999998 2024-09-14T06:31:20.569 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:20.569 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:20.569 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:31:20.569 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.604377+0000", 2024-09-14T06:31:20.569 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.7899999999999999e-06 2024-09-14T06:31:20.569 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:20.569 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:20.570 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:31:20.570 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.604389+0000", 2024-09-14T06:31:20.570 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.2197e-05 2024-09-14T06:31:20.570 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:20.570 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:20.570 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:31:20.570 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.604397+0000", 2024-09-14T06:31:20.570 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.8730000000000007e-06 2024-09-14T06:31:20.570 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:20.570 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:20.570 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:31:20.570 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.643324+0000", 2024-09-14T06:31:20.570 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.038926897000000002 2024-09-14T06:31:20.570 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:20.571 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:20.571 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:31:20.571 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.643402+0000", 2024-09-14T06:31:20.571 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.8063000000000001e-05 2024-09-14T06:31:20.571 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:20.571 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:20.571 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:31:20.571 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.643496+0000", 2024-09-14T06:31:20.571 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.4155999999999998e-05 2024-09-14T06:31:20.571 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:20.571 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:31:20.571 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:20.571 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:20.571 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:20.572 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:55657 3.0 3:06490baf:::rbd_data.112dca2f6ae1.0000000000000030:17 [read 1687552~4096 [fadvise_nocache]] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:31:20.572 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:44.604841+0000", 2024-09-14T06:31:20.572 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 95.896228761000003, 2024-09-14T06:31:20.572 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.041890087999999999, 2024-09-14T06:31:20.572 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:31:20.572 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:31:20.572 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:31:20.572 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:31:20.572 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:31:20.572 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:20.572 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 55657 2024-09-14T06:31:20.572 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:20.572 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:31:20.572 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:20.573 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:31:20.573 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.604841+0000", 2024-09-14T06:31:20.573 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:20.573 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:20.573 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:20.573 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:31:20.573 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.604841+0000", 2024-09-14T06:31:20.573 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:20.573 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:20.573 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:20.573 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:31:20.573 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.604842+0000", 2024-09-14T06:31:20.574 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.0900000000000001e-07 2024-09-14T06:31:20.574 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:20.574 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:20.574 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:31:20.574 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.604846+0000", 2024-09-14T06:31:20.574 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.5759999999999999e-06 2024-09-14T06:31:20.574 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:20.574 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:20.574 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:31:20.574 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.604850+0000", 2024-09-14T06:31:20.574 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.405e-06 2024-09-14T06:31:20.574 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:20.574 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:20.575 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:31:20.575 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.604852+0000", 2024-09-14T06:31:20.575 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.8590000000000001e-06 2024-09-14T06:31:20.575 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:20.575 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:20.575 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:31:20.575 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.646633+0000", 2024-09-14T06:31:20.575 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.041780772000000001 2024-09-14T06:31:20.575 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:20.575 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:20.575 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:31:20.575 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.646677+0000", 2024-09-14T06:31:20.575 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.4115e-05 2024-09-14T06:31:20.576 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:20.576 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:20.576 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:31:20.576 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.646731+0000", 2024-09-14T06:31:20.576 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.4151999999999999e-05 2024-09-14T06:31:20.576 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:20.576 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:31:20.576 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:20.576 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:20.576 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:20.576 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:55658 3.0 3:06490baf:::rbd_data.112dca2f6ae1.0000000000000030:17 [read 1712128~4096 [fadvise_nocache]] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:31:20.576 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:44.604856+0000", 2024-09-14T06:31:20.576 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 95.896214161000003, 2024-09-14T06:31:20.577 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.041986227000000001, 2024-09-14T06:31:20.577 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:31:20.577 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:31:20.577 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:31:20.577 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:31:20.577 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:31:20.577 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:20.577 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 55658 2024-09-14T06:31:20.577 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:20.577 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:31:20.577 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:20.577 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:31:20.577 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.604856+0000", 2024-09-14T06:31:20.578 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:20.578 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:20.578 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:20.578 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:31:20.578 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.604856+0000", 2024-09-14T06:31:20.578 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:20.578 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:20.578 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:20.578 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:31:20.578 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.604856+0000", 2024-09-14T06:31:20.578 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.23e-07 2024-09-14T06:31:20.578 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:20.578 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:20.578 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:31:20.579 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.604857+0000", 2024-09-14T06:31:20.579 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.9999999999999995e-07 2024-09-14T06:31:20.579 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:20.579 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:20.579 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:31:20.579 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.604861+0000", 2024-09-14T06:31:20.579 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.9890000000000003e-06 2024-09-14T06:31:20.579 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:20.579 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:20.579 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:31:20.579 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.604864+0000", 2024-09-14T06:31:20.579 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.7429999999999998e-06 2024-09-14T06:31:20.579 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:20.579 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:20.580 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:31:20.580 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.646744+0000", 2024-09-14T06:31:20.580 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.041880325000000003 2024-09-14T06:31:20.580 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:20.580 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:20.580 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:31:20.580 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.646784+0000", 2024-09-14T06:31:20.580 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.9604000000000002e-05 2024-09-14T06:31:20.580 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:20.580 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:20.580 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:31:20.580 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.646842+0000", 2024-09-14T06:31:20.581 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.8343e-05 2024-09-14T06:31:20.581 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:20.581 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:31:20.581 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:20.581 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:20.581 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:20.581 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:55659 3.0 3:06490baf:::rbd_data.112dca2f6ae1.0000000000000030:17 [read 1736704~4096 [fadvise_nocache]] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:31:20.581 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:44.604867+0000", 2024-09-14T06:31:20.581 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 95.896203008000001, 2024-09-14T06:31:20.581 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.042097082000000001, 2024-09-14T06:31:20.581 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:31:20.581 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:31:20.581 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:31:20.582 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:31:20.582 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:31:20.582 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:20.582 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 55659 2024-09-14T06:31:20.582 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:20.582 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:31:20.582 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:20.582 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:31:20.582 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.604867+0000", 2024-09-14T06:31:20.582 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:20.582 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:20.582 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:20.582 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:31:20.582 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.604867+0000", 2024-09-14T06:31:20.583 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:20.583 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:20.583 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:20.583 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:31:20.583 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.604867+0000", 2024-09-14T06:31:20.583 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.2000000000000001e-07 2024-09-14T06:31:20.583 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:20.583 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:20.583 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:31:20.583 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.604868+0000", 2024-09-14T06:31:20.583 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.04e-06 2024-09-14T06:31:20.583 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:20.583 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:20.584 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:31:20.584 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.604871+0000", 2024-09-14T06:31:20.584 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.9689999999999999e-06 2024-09-14T06:31:20.584 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:20.584 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:20.584 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:31:20.584 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.604874+0000", 2024-09-14T06:31:20.584 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.5069999999999999e-06 2024-09-14T06:31:20.584 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:20.584 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:20.584 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:31:20.584 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.646874+0000", 2024-09-14T06:31:20.585 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.042000513000000003 2024-09-14T06:31:20.585 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:20.585 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:20.585 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:31:20.585 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.646914+0000", 2024-09-14T06:31:20.585 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.9443999999999997e-05 2024-09-14T06:31:20.585 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:20.585 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:20.585 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:31:20.585 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.646964+0000", 2024-09-14T06:31:20.585 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.0389000000000002e-05 2024-09-14T06:31:20.585 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:20.585 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:31:20.586 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:20.586 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:20.586 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:20.586 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:55660 3.0 3:06490baf:::rbd_data.112dca2f6ae1.0000000000000030:17 [read 1761280~4096 [fadvise_nocache]] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:31:20.586 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:44.604877+0000", 2024-09-14T06:31:20.586 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 95.896193259, 2024-09-14T06:31:20.586 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.042196349000000001, 2024-09-14T06:31:20.586 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:31:20.586 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:31:20.586 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:31:20.586 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:31:20.586 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:31:20.586 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:20.587 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 55660 2024-09-14T06:31:20.587 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:20.587 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:31:20.587 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:20.587 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:31:20.587 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.604877+0000", 2024-09-14T06:31:20.587 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:20.587 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:20.587 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:20.587 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:31:20.587 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.604877+0000", 2024-09-14T06:31:20.587 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:20.587 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:20.587 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:20.588 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:31:20.588 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.605810+0000", 2024-09-14T06:31:20.588 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00093336899999999997 2024-09-14T06:31:20.588 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:20.588 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:20.588 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:31:20.588 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.605815+0000", 2024-09-14T06:31:20.588 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.6140000000000002e-06 2024-09-14T06:31:20.588 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:20.588 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:20.588 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:31:20.588 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.605825+0000", 2024-09-14T06:31:20.588 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.8889999999999997e-06 2024-09-14T06:31:20.589 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:20.589 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:20.589 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:31:20.589 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.605831+0000", 2024-09-14T06:31:20.589 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.9800000000000003e-06 2024-09-14T06:31:20.589 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:20.589 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:20.589 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:31:20.589 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.646989+0000", 2024-09-14T06:31:20.589 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.041157878000000002 2024-09-14T06:31:20.589 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:20.589 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:20.589 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:31:20.590 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.647026+0000", 2024-09-14T06:31:20.590 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.7305000000000003e-05 2024-09-14T06:31:20.590 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:20.590 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:20.590 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:31:20.590 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.647073+0000", 2024-09-14T06:31:20.590 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.7314000000000001e-05 2024-09-14T06:31:20.590 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:20.590 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:31:20.590 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:20.590 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:20.590 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:20.590 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:55666 3.0 3:06490baf:::rbd_data.112dca2f6ae1.0000000000000030:17 [read 1908736~4096 [fadvise_nocache]] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:31:20.591 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:44.605898+0000", 2024-09-14T06:31:20.591 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 95.895172513000006, 2024-09-14T06:31:20.591 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.041862887000000001, 2024-09-14T06:31:20.591 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:31:20.591 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:31:20.591 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:31:20.591 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:31:20.591 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:31:20.591 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:20.591 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 55666 2024-09-14T06:31:20.591 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:20.591 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:31:20.591 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:20.591 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:31:20.592 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.605898+0000", 2024-09-14T06:31:20.592 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:20.592 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:20.592 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:20.592 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:31:20.592 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.605898+0000", 2024-09-14T06:31:20.592 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:20.592 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:20.592 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:20.592 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:31:20.592 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.611874+0000", 2024-09-14T06:31:20.592 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0059764049999999997 2024-09-14T06:31:20.592 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:20.593 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:20.593 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:31:20.593 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.611879+0000", 2024-09-14T06:31:20.593 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.8749999999999999e-06 2024-09-14T06:31:20.593 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:20.593 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:20.593 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:31:20.593 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.611887+0000", 2024-09-14T06:31:20.593 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.4109999999999996e-06 2024-09-14T06:31:20.593 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:20.593 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:20.593 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:31:20.593 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.611893+0000", 2024-09-14T06:31:20.593 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.2959999999999998e-06 2024-09-14T06:31:20.594 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:20.594 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:20.594 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:31:20.594 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.647674+0000", 2024-09-14T06:31:20.594 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.035781281999999998 2024-09-14T06:31:20.594 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:20.594 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:20.594 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:31:20.594 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.647711+0000", 2024-09-14T06:31:20.594 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.7434000000000001e-05 2024-09-14T06:31:20.594 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:20.594 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:20.594 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:31:20.594 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.647760+0000", 2024-09-14T06:31:20.594 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.9184000000000002e-05 2024-09-14T06:31:20.594 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:20.595 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:31:20.595 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:20.595 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:20.595 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:31:20.595 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-14T06:31:20.595 INFO:tasks.ceph.osd.0.smithi094.stderr:2024-09-14T06:31:20.501+0000 7f07c9333640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33636) UID: 0 2024-09-14T06:31:20.596 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-14T06:31:20.596 INFO:teuthology.orchestra.run.smithi094.stdout:osd.4: {} 2024-09-14T06:31:20.597 INFO:teuthology.orchestra.run.smithi094.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-14T06:31:20.597 INFO:teuthology.orchestra.run.smithi094.stdout:osd.5: {} 2024-09-14T06:31:20.597 INFO:teuthology.orchestra.run.smithi094.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-14T06:31:20.598 INFO:teuthology.orchestra.run.smithi094.stdout:osd.6: {} 2024-09-14T06:31:20.598 INFO:teuthology.orchestra.run.smithi094.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-14T06:31:20.598 INFO:teuthology.orchestra.run.smithi094.stdout:osd.7: {} 2024-09-14T06:31:20.599 INFO:teuthology.orchestra.run.smithi094.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-14T06:31:20.603 INFO:tasks.ceph.osd.4.smithi138.stderr:2024-09-14T06:31:20.602+0000 7f6c653c8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33615) UID: 0 2024-09-14T06:31:20.704 INFO:tasks.ceph.osd.1.smithi094.stderr:2024-09-14T06:31:20.703+0000 7f328e47a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33641) UID: 0 2024-09-14T06:31:20.739 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-14T06:31:20.739 INFO:teuthology.orchestra.run.smithi094.stdout: "ops": [], 2024-09-14T06:31:20.739 INFO:teuthology.orchestra.run.smithi094.stdout: "num_ops": 0 2024-09-14T06:31:20.740 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-14T06:31:20.748 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-14T06:31:20.832 INFO:tasks.ceph.osd.2.smithi094.stderr:2024-09-14T06:31:20.831+0000 7faf3269b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33649) UID: 0 2024-09-14T06:31:20.875 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-14T06:31:20.894 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-14T06:31:20.894 INFO:teuthology.orchestra.run.smithi094.stdout: "ops": [], 2024-09-14T06:31:20.894 INFO:teuthology.orchestra.run.smithi094.stdout: "complaint_time": 30, 2024-09-14T06:31:20.894 INFO:teuthology.orchestra.run.smithi094.stdout: "num_blocked_ops": 0 2024-09-14T06:31:20.894 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-14T06:31:20.903 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-14T06:31:20.944 INFO:tasks.ceph.osd.1.smithi094.stderr:2024-09-14T06:31:20.943+0000 7f328e47a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33641) UID: 0 2024-09-14T06:31:21.007 INFO:tasks.ceph.osd.7.smithi138.stderr:2024-09-14T06:31:21.006+0000 7f3c6a21d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33623) UID: 0 2024-09-14T06:31:21.049 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-14T06:31:21.050 INFO:teuthology.orchestra.run.smithi094.stdout: "size": 20, 2024-09-14T06:31:21.050 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 600, 2024-09-14T06:31:21.050 INFO:teuthology.orchestra.run.smithi094.stdout: "ops": [ 2024-09-14T06:31:21.050 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:21.050 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4397.0:26415 3.0 e53/23 3:05fd9be0:::rbd_data.112d38768680.0000000000000006:head v 53'21, mlcod=50'19)", 2024-09-14T06:31:21.050 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:30.721610+0000", 2024-09-14T06:31:21.050 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 110.3258027, 2024-09-14T06:31:21.050 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0021763160000000002, 2024-09-14T06:31:21.050 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:31:21.050 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:31:21.050 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:31:21.050 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:31:21.051 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:21.051 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:31:21.051 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:30.721610+0000", 2024-09-14T06:31:21.051 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:21.051 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:21.051 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:21.051 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:31:21.051 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:30.721610+0000", 2024-09-14T06:31:21.051 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:21.051 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:21.051 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:21.051 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:31:21.051 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:30.721611+0000", 2024-09-14T06:31:21.051 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.41e-07 2024-09-14T06:31:21.051 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:21.052 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:21.052 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:31:21.052 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:30.721905+0000", 2024-09-14T06:31:21.052 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00029394300000000001 2024-09-14T06:31:21.052 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:21.052 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:21.052 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:31:21.052 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:30.721921+0000", 2024-09-14T06:31:21.052 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.6609999999999999e-05 2024-09-14T06:31:21.052 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:21.052 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:21.052 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:31:21.052 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:30.721931+0000", 2024-09-14T06:31:21.053 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.6739999999999992e-06 2024-09-14T06:31:21.053 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:21.053 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:21.053 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:31:21.053 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:30.722001+0000", 2024-09-14T06:31:21.053 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.0549000000000005e-05 2024-09-14T06:31:21.053 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:21.053 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:21.053 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:31:21.053 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:30.722029+0000", 2024-09-14T06:31:21.053 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.7327000000000001e-05 2024-09-14T06:31:21.053 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:21.053 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:21.054 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-14T06:31:21.054 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:30.723727+0000", 2024-09-14T06:31:21.054 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0016977310000000001 2024-09-14T06:31:21.054 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:21.054 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:21.054 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:31:21.054 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:30.723787+0000", 2024-09-14T06:31:21.054 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.0140999999999999e-05 2024-09-14T06:31:21.054 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:21.054 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:31:21.054 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:21.054 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:21.054 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:21.055 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4397.0:36084 3.0 e56/23 3:05a48ef4:::rbd_data.112dc58639fa.0000000000000058:head v 56'29, mlcod=56'27)", 2024-09-14T06:31:21.055 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:34.508855+0000", 2024-09-14T06:31:21.055 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 106.538557946, 2024-09-14T06:31:21.055 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0020801140000000001, 2024-09-14T06:31:21.055 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:31:21.055 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:31:21.055 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:31:21.055 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:31:21.055 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:21.055 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:31:21.055 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:34.508855+0000", 2024-09-14T06:31:21.055 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:21.055 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:21.056 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:21.056 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:31:21.056 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:34.508855+0000", 2024-09-14T06:31:21.056 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:21.056 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:21.056 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:21.056 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:31:21.056 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:34.508856+0000", 2024-09-14T06:31:21.056 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.1909999999999999e-06 2024-09-14T06:31:21.056 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:21.056 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:21.056 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:31:21.056 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:34.509203+0000", 2024-09-14T06:31:21.056 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00034625900000000002 2024-09-14T06:31:21.057 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:21.057 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:21.057 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:31:21.057 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:34.509218+0000", 2024-09-14T06:31:21.057 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.5091e-05 2024-09-14T06:31:21.057 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:21.057 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:21.057 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:31:21.057 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:34.509227+0000", 2024-09-14T06:31:21.057 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.0850000000000003e-06 2024-09-14T06:31:21.057 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:21.057 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:21.057 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:31:21.058 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:34.509311+0000", 2024-09-14T06:31:21.058 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.4436000000000003e-05 2024-09-14T06:31:21.058 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:21.058 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:21.058 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:31:21.058 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:34.509335+0000", 2024-09-14T06:31:21.058 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.3577000000000001e-05 2024-09-14T06:31:21.058 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:21.058 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:21.058 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-14T06:31:21.058 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:34.510890+0000", 2024-09-14T06:31:21.058 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0015554259999999999 2024-09-14T06:31:21.058 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:21.058 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:21.058 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:31:21.059 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:34.510935+0000", 2024-09-14T06:31:21.059 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.5049000000000002e-05 2024-09-14T06:31:21.059 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:21.059 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:31:21.059 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:21.059 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:21.059 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:21.059 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4397.0:45722 3.9 e64/23 3:91fed098:::rbd_data.112d5a737e72.0000000000000001:head v 64'61, mlcod=62'59)", 2024-09-14T06:31:21.059 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:43.864269+0000", 2024-09-14T06:31:21.059 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 97.183143870999999, 2024-09-14T06:31:21.059 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.003055224, 2024-09-14T06:31:21.059 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:31:21.059 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:31:21.059 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:31:21.059 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:31:21.060 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:21.060 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:31:21.060 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.864269+0000", 2024-09-14T06:31:21.060 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:21.060 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:21.060 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:21.060 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:31:21.060 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.864269+0000", 2024-09-14T06:31:21.060 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:21.060 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:21.060 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:21.060 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:31:21.060 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.864269+0000", 2024-09-14T06:31:21.060 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.41e-07 2024-09-14T06:31:21.060 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:21.061 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:21.061 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:31:21.061 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.865490+0000", 2024-09-14T06:31:21.061 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.001220552 2024-09-14T06:31:21.061 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:21.061 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:21.061 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:31:21.061 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.865506+0000", 2024-09-14T06:31:21.061 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.6263999999999999e-05 2024-09-14T06:31:21.061 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:21.061 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:21.061 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:31:21.061 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.865516+0000", 2024-09-14T06:31:21.061 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.0091999999999999e-05 2024-09-14T06:31:21.062 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:21.062 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:21.062 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:31:21.062 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.865583+0000", 2024-09-14T06:31:21.062 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.6676999999999999e-05 2024-09-14T06:31:21.062 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:21.062 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:21.062 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:31:21.062 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.865610+0000", 2024-09-14T06:31:21.062 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.6529999999999998e-05 2024-09-14T06:31:21.062 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:21.062 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:21.062 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-14T06:31:21.063 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.867281+0000", 2024-09-14T06:31:21.063 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.001671203 2024-09-14T06:31:21.063 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:21.063 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:21.063 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:31:21.063 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.867324+0000", 2024-09-14T06:31:21.063 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.3665000000000003e-05 2024-09-14T06:31:21.063 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:21.063 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:31:21.063 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:21.063 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:21.063 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:21.063 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4397.0:50006 3.4 e64/23 3:20cfe300:::rbd_data.112d5a737e72.0000000000000019:head v 64'70, mlcod=61'68)", 2024-09-14T06:31:21.064 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:44.224218+0000", 2024-09-14T06:31:21.064 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 96.823194674000007, 2024-09-14T06:31:21.064 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0021104320000000002, 2024-09-14T06:31:21.064 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:31:21.064 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:31:21.064 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:31:21.064 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:31:21.064 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:21.064 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:31:21.064 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.224218+0000", 2024-09-14T06:31:21.064 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:21.064 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:21.064 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:21.064 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:31:21.065 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.224218+0000", 2024-09-14T06:31:21.065 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:21.065 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:21.065 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:21.065 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:31:21.065 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.224219+0000", 2024-09-14T06:31:21.065 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.0049999999999999e-06 2024-09-14T06:31:21.065 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:21.065 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:21.065 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:31:21.065 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.224326+0000", 2024-09-14T06:31:21.065 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000106848 2024-09-14T06:31:21.066 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:21.066 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:21.066 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:31:21.066 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.224345+0000", 2024-09-14T06:31:21.066 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.8587999999999999e-05 2024-09-14T06:31:21.066 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:21.066 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:21.066 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:31:21.066 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.224354+0000", 2024-09-14T06:31:21.066 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.7220000000000005e-06 2024-09-14T06:31:21.066 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:21.066 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:21.067 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:31:21.067 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.224416+0000", 2024-09-14T06:31:21.067 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.2738999999999997e-05 2024-09-14T06:31:21.067 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:21.067 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:21.067 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:31:21.067 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.224441+0000", 2024-09-14T06:31:21.067 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.4408000000000001e-05 2024-09-14T06:31:21.067 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:21.067 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:21.067 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-14T06:31:21.067 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.226295+0000", 2024-09-14T06:31:21.068 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00185428 2024-09-14T06:31:21.068 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:21.068 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:21.068 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:31:21.068 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.226329+0000", 2024-09-14T06:31:21.068 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.3841999999999999e-05 2024-09-14T06:31:21.068 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:21.068 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:31:21.068 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:21.068 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:21.068 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:21.068 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4397.0:74241 3.0 e73/23 3:06e93fb9:::rbd_data.112d80ca42af.0000000000000000:head v 73'102, mlcod=71'100)", 2024-09-14T06:31:21.068 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:53.876730+0000", 2024-09-14T06:31:21.069 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 87.170682771000003, 2024-09-14T06:31:21.069 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0079470329999999992, 2024-09-14T06:31:21.069 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:31:21.069 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:31:21.069 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:31:21.069 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:31:21.069 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:21.069 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:31:21.069 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:53.876730+0000", 2024-09-14T06:31:21.069 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:21.069 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:21.069 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:21.070 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:31:21.070 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:53.876730+0000", 2024-09-14T06:31:21.070 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:21.070 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:21.070 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:21.070 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:31:21.070 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:53.876731+0000", 2024-09-14T06:31:21.070 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.0500000000000004e-07 2024-09-14T06:31:21.070 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:21.070 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:21.070 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:31:21.070 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:53.879616+0000", 2024-09-14T06:31:21.071 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0028856680000000001 2024-09-14T06:31:21.071 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:21.071 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:21.071 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:31:21.071 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:53.879643+0000", 2024-09-14T06:31:21.071 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.6120999999999999e-05 2024-09-14T06:31:21.071 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:21.071 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:21.071 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:31:21.071 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:53.879654+0000", 2024-09-14T06:31:21.071 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.1586999999999999e-05 2024-09-14T06:31:21.071 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:21.071 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:21.072 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:31:21.072 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:53.879756+0000", 2024-09-14T06:31:21.072 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00010223899999999999 2024-09-14T06:31:21.072 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:21.072 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:21.072 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:31:21.072 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:53.879783+0000", 2024-09-14T06:31:21.072 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.6913000000000001e-05 2024-09-14T06:31:21.072 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:21.072 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:21.072 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-14T06:31:21.072 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:53.884594+0000", 2024-09-14T06:31:21.072 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0048110729999999999 2024-09-14T06:31:21.073 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:21.073 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:21.073 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:31:21.073 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:53.884677+0000", 2024-09-14T06:31:21.073 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.2927000000000001e-05 2024-09-14T06:31:21.073 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:21.073 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:31:21.073 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:21.073 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:21.073 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:21.073 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4397.0:74243 3.9 e73/23 3:94977f81:::rbd_data.112d80ca42af.0000000000000002:head v 73'94, mlcod=71'92)", 2024-09-14T06:31:21.073 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:53.885156+0000", 2024-09-14T06:31:21.073 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 87.162256853000002, 2024-09-14T06:31:21.074 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0096009860000000006, 2024-09-14T06:31:21.074 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:31:21.074 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:31:21.074 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:31:21.074 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:31:21.074 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:21.074 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:31:21.074 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:53.885156+0000", 2024-09-14T06:31:21.074 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:21.074 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:21.074 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:21.074 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:31:21.074 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:53.885156+0000", 2024-09-14T06:31:21.074 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:21.075 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:21.075 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:21.075 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:31:21.075 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:53.885156+0000", 2024-09-14T06:31:21.075 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.23e-07 2024-09-14T06:31:21.075 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:21.075 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:21.075 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:31:21.075 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:53.889694+0000", 2024-09-14T06:31:21.075 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0045378709999999997 2024-09-14T06:31:21.075 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:21.075 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:21.075 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:31:21.076 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:53.889720+0000", 2024-09-14T06:31:21.076 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.5259e-05 2024-09-14T06:31:21.076 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:21.076 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:21.076 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:31:21.076 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:53.889731+0000", 2024-09-14T06:31:21.076 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.1739000000000001e-05 2024-09-14T06:31:21.076 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:21.076 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:21.076 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:31:21.076 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:53.889822+0000", 2024-09-14T06:31:21.076 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.1200999999999996e-05 2024-09-14T06:31:21.076 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:21.077 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:21.077 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:31:21.077 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:53.889850+0000", 2024-09-14T06:31:21.077 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.7084999999999999e-05 2024-09-14T06:31:21.077 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:21.077 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:21.077 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-14T06:31:21.077 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:53.894703+0000", 2024-09-14T06:31:21.077 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0048532690000000003 2024-09-14T06:31:21.077 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:21.077 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:21.077 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:31:21.077 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:53.894757+0000", 2024-09-14T06:31:21.077 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.4339000000000003e-05 2024-09-14T06:31:21.078 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:21.078 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:31:21.078 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:21.078 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:21.078 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:21.078 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4397.0:74249 3.0 e73/23 3:009b0225:::rbd_data.112d80ca42af.0000000000000009:head v 73'103, mlcod=71'101)", 2024-09-14T06:31:21.078 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:53.893219+0000", 2024-09-14T06:31:21.078 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 87.154194375000003, 2024-09-14T06:31:21.078 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0065381980000000003, 2024-09-14T06:31:21.078 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:31:21.078 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:31:21.078 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:31:21.078 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:31:21.078 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:21.078 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:31:21.079 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:53.893219+0000", 2024-09-14T06:31:21.079 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:21.079 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:21.079 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:21.079 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:31:21.079 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:53.893219+0000", 2024-09-14T06:31:21.079 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:21.079 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:21.079 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:21.079 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:31:21.079 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:53.893221+0000", 2024-09-14T06:31:21.079 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.4329999999999998e-06 2024-09-14T06:31:21.079 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:21.080 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:21.080 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:31:21.080 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:53.895968+0000", 2024-09-14T06:31:21.080 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.002746746 2024-09-14T06:31:21.080 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:21.080 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:21.080 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:31:21.080 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:53.895988+0000", 2024-09-14T06:31:21.080 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.0227000000000001e-05 2024-09-14T06:31:21.080 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:21.080 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:21.080 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:31:21.080 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:53.896000+0000", 2024-09-14T06:31:21.081 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.2359e-05 2024-09-14T06:31:21.081 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:21.081 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:21.081 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:31:21.081 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:53.896084+0000", 2024-09-14T06:31:21.081 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.3659999999999995e-05 2024-09-14T06:31:21.081 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:21.081 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:21.081 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:31:21.081 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:53.896109+0000", 2024-09-14T06:31:21.081 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.5259999999999999e-05 2024-09-14T06:31:21.081 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:21.081 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:21.082 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-14T06:31:21.082 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:53.899704+0000", 2024-09-14T06:31:21.082 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0035942209999999999 2024-09-14T06:31:21.082 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:21.082 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:21.082 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:31:21.082 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:53.899757+0000", 2024-09-14T06:31:21.082 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.3291999999999997e-05 2024-09-14T06:31:21.082 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:21.083 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:31:21.083 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:21.083 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:21.083 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:21.083 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4397.0:78263 3.0 e77/23 3:00899b22:::rbd_data.112ddfbec471.0000000000000016:head v 77'104, mlcod=73'102)", 2024-09-14T06:31:21.083 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:57.634578+0000", 2024-09-14T06:31:21.083 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 83.412835322999996, 2024-09-14T06:31:21.083 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.002372454, 2024-09-14T06:31:21.083 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:31:21.083 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:31:21.083 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:31:21.083 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:31:21.083 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:21.084 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:31:21.084 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:57.634578+0000", 2024-09-14T06:31:21.084 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:21.084 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:21.084 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:21.084 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:31:21.084 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:57.634578+0000", 2024-09-14T06:31:21.084 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:21.084 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:21.084 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:21.084 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:31:21.084 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:57.634578+0000", 2024-09-14T06:31:21.084 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.5699999999999998e-07 2024-09-14T06:31:21.084 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:21.085 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:21.085 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:31:21.085 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:57.635347+0000", 2024-09-14T06:31:21.085 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00076848300000000003 2024-09-14T06:31:21.085 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:21.085 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:21.085 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:31:21.085 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:57.635361+0000", 2024-09-14T06:31:21.085 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.4238e-05 2024-09-14T06:31:21.085 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:21.085 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:21.085 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:31:21.085 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:57.635371+0000", 2024-09-14T06:31:21.086 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.0087e-05 2024-09-14T06:31:21.086 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:21.086 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:21.086 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:31:21.086 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:57.635439+0000", 2024-09-14T06:31:21.086 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.7953000000000005e-05 2024-09-14T06:31:21.086 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:21.086 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:21.086 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:31:21.086 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:57.635464+0000", 2024-09-14T06:31:21.086 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.5062000000000002e-05 2024-09-14T06:31:21.086 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:21.086 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:21.087 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-14T06:31:21.087 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:57.636914+0000", 2024-09-14T06:31:21.087 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.001449625 2024-09-14T06:31:21.087 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:21.087 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:21.087 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:31:21.087 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:57.636950+0000", 2024-09-14T06:31:21.087 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.6548999999999999e-05 2024-09-14T06:31:21.087 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:21.087 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:31:21.087 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:21.087 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:21.087 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:21.087 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4397.0:78604 3.9 e77/23 3:94c62a0e:::rbd_data.112ddfbec471.000000000000002c:head v 77'96, mlcod=73'94)", 2024-09-14T06:31:21.088 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:57.677568+0000", 2024-09-14T06:31:21.088 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 83.369844935000003, 2024-09-14T06:31:21.088 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.002564846, 2024-09-14T06:31:21.088 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:31:21.088 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:31:21.088 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:31:21.088 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:31:21.088 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:21.088 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:31:21.088 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:57.677568+0000", 2024-09-14T06:31:21.088 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:21.088 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:21.088 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:21.089 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:31:21.089 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:57.677568+0000", 2024-09-14T06:31:21.089 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:21.089 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:21.089 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:21.089 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:31:21.089 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:57.677568+0000", 2024-09-14T06:31:21.089 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.8900000000000001e-07 2024-09-14T06:31:21.089 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:21.089 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:21.089 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:31:21.089 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:57.678488+0000", 2024-09-14T06:31:21.089 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00091963899999999998 2024-09-14T06:31:21.089 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:21.090 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:21.090 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:31:21.090 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:57.678500+0000", 2024-09-14T06:31:21.090 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.1902000000000001e-05 2024-09-14T06:31:21.090 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:21.090 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:21.090 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:31:21.090 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:57.678507+0000", 2024-09-14T06:31:21.090 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.7739999999999999e-06 2024-09-14T06:31:21.090 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:21.090 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:21.090 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:31:21.090 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:57.678586+0000", 2024-09-14T06:31:21.090 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.9103000000000005e-05 2024-09-14T06:31:21.090 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:21.091 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:21.091 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:31:21.091 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:57.678614+0000", 2024-09-14T06:31:21.091 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.7912e-05 2024-09-14T06:31:21.091 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:21.091 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:21.091 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-14T06:31:21.091 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:57.680075+0000", 2024-09-14T06:31:21.091 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0014613370000000001 2024-09-14T06:31:21.091 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:21.091 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:21.091 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:31:21.091 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:57.680133+0000", 2024-09-14T06:31:21.091 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.7890000000000003e-05 2024-09-14T06:31:21.091 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:21.092 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:31:21.092 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:21.092 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:21.092 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:21.092 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4397.0:78641 3.9 e77/23 3:914b72e4:::rbd_data.112ddfbec471.0000000000000030:head v 77'98, mlcod=73'95)", 2024-09-14T06:31:21.092 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:57.682289+0000", 2024-09-14T06:31:21.092 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 83.365124253999994, 2024-09-14T06:31:21.092 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0020171400000000002, 2024-09-14T06:31:21.092 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:31:21.092 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:31:21.092 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:31:21.092 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:31:21.092 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:21.092 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:31:21.093 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:57.682289+0000", 2024-09-14T06:31:21.093 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:21.093 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:21.093 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:21.093 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:31:21.093 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:57.682289+0000", 2024-09-14T06:31:21.093 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:21.093 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:21.093 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:21.093 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:31:21.093 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:57.682290+0000", 2024-09-14T06:31:21.093 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.0100000000000003e-07 2024-09-14T06:31:21.093 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:21.093 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:21.094 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:31:21.094 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:57.683230+0000", 2024-09-14T06:31:21.094 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00094047599999999999 2024-09-14T06:31:21.094 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:21.094 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:21.094 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:31:21.094 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:57.683243+0000", 2024-09-14T06:31:21.094 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.2926e-05 2024-09-14T06:31:21.094 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:21.094 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:21.094 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:31:21.094 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:57.683250+0000", 2024-09-14T06:31:21.094 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.6109999999999999e-06 2024-09-14T06:31:21.094 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:21.095 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:21.095 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:31:21.095 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:57.683313+0000", 2024-09-14T06:31:21.095 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.2943999999999999e-05 2024-09-14T06:31:21.095 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:21.095 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:21.095 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:31:21.095 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:57.683337+0000", 2024-09-14T06:31:21.095 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.4377999999999999e-05 2024-09-14T06:31:21.095 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:21.095 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:21.095 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-14T06:31:21.095 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:57.684264+0000", 2024-09-14T06:31:21.096 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00092733099999999999 2024-09-14T06:31:21.096 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:21.096 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:21.096 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:31:21.096 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:57.684306+0000", 2024-09-14T06:31:21.096 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.1573000000000003e-05 2024-09-14T06:31:21.096 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:21.096 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:31:21.096 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:21.096 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:21.096 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:21.096 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4397.0:126010 3.9 e104/23 3:9120f7ff:::rbd_data.112d97471187.000000000000001b:head v 104'145, mlcod=99'143)", 2024-09-14T06:31:21.096 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:30:25.990254+0000", 2024-09-14T06:31:21.097 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 55.057159446, 2024-09-14T06:31:21.097 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0020629699999999999, 2024-09-14T06:31:21.097 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:31:21.097 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:31:21.097 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:31:21.097 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:31:21.097 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:21.097 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:31:21.097 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:25.990254+0000", 2024-09-14T06:31:21.097 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:21.097 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:21.097 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:21.098 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:31:21.098 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:25.990254+0000", 2024-09-14T06:31:21.098 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:21.098 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:21.098 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:21.098 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:31:21.098 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:25.990255+0000", 2024-09-14T06:31:21.098 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.3349999999999999e-06 2024-09-14T06:31:21.098 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:21.098 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:21.098 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:31:21.098 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:25.990966+0000", 2024-09-14T06:31:21.098 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00071085199999999999 2024-09-14T06:31:21.098 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:21.099 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:21.099 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:31:21.099 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:25.990986+0000", 2024-09-14T06:31:21.099 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.0054000000000001e-05 2024-09-14T06:31:21.099 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:21.099 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:21.099 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:31:21.099 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:25.990995+0000", 2024-09-14T06:31:21.099 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.166e-06 2024-09-14T06:31:21.099 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:21.099 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:21.099 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:31:21.099 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:25.991055+0000", 2024-09-14T06:31:21.099 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.0093000000000002e-05 2024-09-14T06:31:21.100 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:21.100 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:21.100 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:31:21.100 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:25.991082+0000", 2024-09-14T06:31:21.100 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.6505000000000001e-05 2024-09-14T06:31:21.100 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:21.100 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:21.100 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-14T06:31:21.100 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:25.992270+0000", 2024-09-14T06:31:21.100 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0011880479999999999 2024-09-14T06:31:21.100 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:21.100 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:21.100 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:31:21.100 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:25.992317+0000", 2024-09-14T06:31:21.101 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.6916999999999998e-05 2024-09-14T06:31:21.101 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:21.101 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:31:21.101 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:21.101 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:21.101 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:21.101 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4397.0:149885 3.9 e122/23 3:951acef3:::rbd_data.112d1bb7b269.0000000000000005:head v 122'176, mlcod=122'172)", 2024-09-14T06:31:21.101 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:30:44.739092+0000", 2024-09-14T06:31:21.101 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 36.308320625, 2024-09-14T06:31:21.101 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0028236659999999998, 2024-09-14T06:31:21.101 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:31:21.101 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:31:21.101 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:31:21.101 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:31:21.102 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:21.102 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:31:21.102 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.739092+0000", 2024-09-14T06:31:21.102 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:21.102 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:21.102 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:21.102 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:31:21.102 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.739092+0000", 2024-09-14T06:31:21.102 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:21.102 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:21.102 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:21.102 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:31:21.102 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.739093+0000", 2024-09-14T06:31:21.102 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.0109999999999999e-06 2024-09-14T06:31:21.103 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:21.103 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:21.103 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:31:21.103 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.740189+0000", 2024-09-14T06:31:21.103 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00109527 2024-09-14T06:31:21.103 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:21.103 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:21.103 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:31:21.103 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.740199+0000", 2024-09-14T06:31:21.103 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.0458999999999999e-05 2024-09-14T06:31:21.103 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:21.103 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:21.103 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:31:21.103 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.740207+0000", 2024-09-14T06:31:21.104 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.4499999999999998e-06 2024-09-14T06:31:21.104 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:21.104 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:21.104 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:31:21.104 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.740295+0000", 2024-09-14T06:31:21.104 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.8740999999999996e-05 2024-09-14T06:31:21.104 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:21.104 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:21.104 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:31:21.104 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.740317+0000", 2024-09-14T06:31:21.104 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.1523e-05 2024-09-14T06:31:21.104 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:21.104 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:21.104 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-14T06:31:21.105 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.741865+0000", 2024-09-14T06:31:21.105 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.001548315 2024-09-14T06:31:21.105 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:21.105 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:21.105 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:31:21.105 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.741916+0000", 2024-09-14T06:31:21.105 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.0896999999999997e-05 2024-09-14T06:31:21.105 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:21.105 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:31:21.105 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:21.105 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:21.105 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:21.105 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4397.0:149886 3.4 e122/23 3:2033c3d0:::rbd_data.112d1bb7b269.0000000000000006:head v 122'220, mlcod=120'217)", 2024-09-14T06:31:21.106 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:30:44.741736+0000", 2024-09-14T06:31:21.106 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 36.305677207999999, 2024-09-14T06:31:21.106 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0020268999999999999, 2024-09-14T06:31:21.106 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:31:21.106 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:31:21.106 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:31:21.106 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:31:21.106 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:21.106 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:31:21.106 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.741736+0000", 2024-09-14T06:31:21.106 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:21.106 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:21.106 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:21.106 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:31:21.107 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.741736+0000", 2024-09-14T06:31:21.107 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:21.107 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:21.107 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:21.107 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:31:21.107 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.741737+0000", 2024-09-14T06:31:21.107 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.9000000000000005e-07 2024-09-14T06:31:21.107 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:21.107 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:21.107 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:31:21.107 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.741918+0000", 2024-09-14T06:31:21.107 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00018120300000000001 2024-09-14T06:31:21.107 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:21.107 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:21.108 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:31:21.108 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.741930+0000", 2024-09-14T06:31:21.108 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.1875999999999999e-05 2024-09-14T06:31:21.108 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:21.108 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:21.108 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:31:21.108 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.741938+0000", 2024-09-14T06:31:21.108 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.8040000000000002e-06 2024-09-14T06:31:21.108 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:21.108 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:21.108 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:31:21.108 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.742185+0000", 2024-09-14T06:31:21.109 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00024747199999999999 2024-09-14T06:31:21.109 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:21.109 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:21.109 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:31:21.109 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.742207+0000", 2024-09-14T06:31:21.109 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.1416999999999998e-05 2024-09-14T06:31:21.109 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:21.109 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:21.109 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-14T06:31:21.109 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.743715+0000", 2024-09-14T06:31:21.109 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00150884 2024-09-14T06:31:21.109 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:21.109 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:21.109 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:31:21.110 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.743763+0000", 2024-09-14T06:31:21.110 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.7298e-05 2024-09-14T06:31:21.110 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:21.110 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:31:21.110 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:21.110 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:21.110 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:21.110 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4397.0:149933 3.4 e122/23 3:234f5229:::rbd_data.112d1bb7b269.0000000000000009:head v 122'221, mlcod=120'218)", 2024-09-14T06:31:21.110 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:30:44.743849+0000", 2024-09-14T06:31:21.110 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 36.303564207999997, 2024-09-14T06:31:21.110 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0031479519999999999, 2024-09-14T06:31:21.110 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:31:21.110 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:31:21.110 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:31:21.111 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:31:21.111 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:21.111 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:31:21.111 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.743849+0000", 2024-09-14T06:31:21.111 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:21.111 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:21.111 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:21.111 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:31:21.111 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.743849+0000", 2024-09-14T06:31:21.111 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:21.111 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:21.111 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:21.111 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:31:21.111 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.743850+0000", 2024-09-14T06:31:21.112 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.279e-06 2024-09-14T06:31:21.112 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:21.112 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:21.112 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:31:21.112 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.745151+0000", 2024-09-14T06:31:21.112 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.001300495 2024-09-14T06:31:21.112 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:21.112 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:21.112 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:31:21.112 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.745167+0000", 2024-09-14T06:31:21.112 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.6478e-05 2024-09-14T06:31:21.112 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:21.112 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:21.113 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:31:21.113 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.745175+0000", 2024-09-14T06:31:21.113 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.2679999999999994e-06 2024-09-14T06:31:21.113 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:21.113 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:21.113 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:31:21.113 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.745226+0000", 2024-09-14T06:31:21.113 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.0192999999999999e-05 2024-09-14T06:31:21.113 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:21.113 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:21.113 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:31:21.113 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.745247+0000", 2024-09-14T06:31:21.113 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.1767e-05 2024-09-14T06:31:21.114 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:21.114 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:21.114 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-14T06:31:21.114 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.746952+0000", 2024-09-14T06:31:21.114 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0017042769999999999 2024-09-14T06:31:21.114 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:21.114 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:21.114 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:31:21.114 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.746997+0000", 2024-09-14T06:31:21.114 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.5194999999999997e-05 2024-09-14T06:31:21.114 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:21.114 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:31:21.114 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:21.114 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:21.115 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:21.115 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4397.0:150494 3.9 e122/23 3:955152a4:::rbd_object_map.112d1bb7b269.0000000000000038:head v 122'194, mlcod=122'190)", 2024-09-14T06:31:21.115 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:30:44.781585+0000", 2024-09-14T06:31:21.115 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 36.265828315999997, 2024-09-14T06:31:21.115 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0026957499999999998, 2024-09-14T06:31:21.115 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:31:21.115 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:31:21.115 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:31:21.115 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:31:21.115 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:21.115 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:31:21.115 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.781585+0000", 2024-09-14T06:31:21.115 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:21.115 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:21.115 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:21.116 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:31:21.116 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.781585+0000", 2024-09-14T06:31:21.116 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:21.116 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:21.116 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:21.116 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:31:21.116 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.781585+0000", 2024-09-14T06:31:21.116 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.5000000000000002e-07 2024-09-14T06:31:21.116 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:21.116 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:21.116 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:31:21.116 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.781594+0000", 2024-09-14T06:31:21.116 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.4640000000000006e-06 2024-09-14T06:31:21.117 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:21.117 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:21.117 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:31:21.117 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.781604+0000", 2024-09-14T06:31:21.117 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.0071999999999999e-05 2024-09-14T06:31:21.117 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:21.117 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:21.117 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:31:21.117 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.781610+0000", 2024-09-14T06:31:21.117 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.3609999999999997e-06 2024-09-14T06:31:21.117 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:21.117 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:21.117 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:31:21.118 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.781676+0000", 2024-09-14T06:31:21.118 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.5963000000000005e-05 2024-09-14T06:31:21.118 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:21.118 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:21.118 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:31:21.118 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.781696+0000", 2024-09-14T06:31:21.118 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.9278000000000001e-05 2024-09-14T06:31:21.118 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:21.118 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:21.118 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-14T06:31:21.118 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.784231+0000", 2024-09-14T06:31:21.118 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.002535093 2024-09-14T06:31:21.118 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:21.118 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:21.119 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:31:21.119 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.784280+0000", 2024-09-14T06:31:21.119 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.9869e-05 2024-09-14T06:31:21.119 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:21.119 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:31:21.119 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:21.119 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:21.119 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:21.119 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4397.0:150519 3.9 e122/23 3:955152a4:::rbd_object_map.112d1bb7b269.0000000000000038:head v 122'195, mlcod=122'192)", 2024-09-14T06:31:21.119 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:30:44.782445+0000", 2024-09-14T06:31:21.119 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 36.264967923999997, 2024-09-14T06:31:21.119 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.002158227, 2024-09-14T06:31:21.119 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:31:21.119 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:31:21.120 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:31:21.120 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:31:21.120 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:21.120 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:31:21.120 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.782445+0000", 2024-09-14T06:31:21.120 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:21.120 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:21.120 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:21.120 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:31:21.120 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.782445+0000", 2024-09-14T06:31:21.120 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:21.120 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:21.120 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:21.120 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:31:21.120 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.782446+0000", 2024-09-14T06:31:21.121 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.6499999999999998e-07 2024-09-14T06:31:21.121 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:21.121 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:21.121 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:31:21.121 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.782455+0000", 2024-09-14T06:31:21.121 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.1940000000000004e-06 2024-09-14T06:31:21.121 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:21.121 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:21.121 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:31:21.121 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.782464+0000", 2024-09-14T06:31:21.121 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.7220000000000005e-06 2024-09-14T06:31:21.121 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:21.121 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:21.122 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:31:21.122 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.782470+0000", 2024-09-14T06:31:21.122 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.9580000000000004e-06 2024-09-14T06:31:21.122 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:21.122 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:21.122 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:31:21.122 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.782514+0000", 2024-09-14T06:31:21.122 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.3931999999999999e-05 2024-09-14T06:31:21.122 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:21.122 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:21.122 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:31:21.122 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.782534+0000", 2024-09-14T06:31:21.122 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.0237e-05 2024-09-14T06:31:21.122 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:21.123 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:21.123 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-14T06:31:21.123 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.784560+0000", 2024-09-14T06:31:21.123 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.002025813 2024-09-14T06:31:21.123 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:21.123 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:21.123 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:31:21.123 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.784603+0000", 2024-09-14T06:31:21.123 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.3606000000000002e-05 2024-09-14T06:31:21.123 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:21.123 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:31:21.123 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:21.123 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:21.124 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:21.124 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4397.0:150898 3.9 e122/23 3:97c98f90:::rbd_data.112d1bb7b269.0000000000000025:head v 122'206, mlcod=122'203)", 2024-09-14T06:31:21.124 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:30:44.807232+0000", 2024-09-14T06:31:21.124 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 36.240181399000001, 2024-09-14T06:31:21.124 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0022297889999999998, 2024-09-14T06:31:21.124 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:31:21.124 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:31:21.124 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:31:21.124 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:31:21.124 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:21.124 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:31:21.124 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.807232+0000", 2024-09-14T06:31:21.124 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:21.125 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:21.125 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:21.125 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:31:21.125 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.807232+0000", 2024-09-14T06:31:21.125 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:21.125 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:21.125 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:21.125 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:31:21.125 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.807233+0000", 2024-09-14T06:31:21.125 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.5710000000000001e-06 2024-09-14T06:31:21.125 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:21.125 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:21.125 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:31:21.126 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.807944+0000", 2024-09-14T06:31:21.126 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00071056299999999999 2024-09-14T06:31:21.126 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:21.126 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:21.126 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:31:21.126 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.807961+0000", 2024-09-14T06:31:21.126 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.7689e-05 2024-09-14T06:31:21.126 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:21.126 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:21.126 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:31:21.126 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.807971+0000", 2024-09-14T06:31:21.126 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.2e-06 2024-09-14T06:31:21.126 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:21.127 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:21.127 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:31:21.127 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.808050+0000", 2024-09-14T06:31:21.127 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.9308999999999996e-05 2024-09-14T06:31:21.127 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:21.127 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:21.127 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:31:21.127 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.808074+0000", 2024-09-14T06:31:21.127 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.4147000000000001e-05 2024-09-14T06:31:21.127 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:21.127 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:21.127 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-14T06:31:21.127 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.809415+0000", 2024-09-14T06:31:21.127 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.001341257 2024-09-14T06:31:21.128 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:21.128 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:21.128 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:31:21.128 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.809461+0000", 2024-09-14T06:31:21.128 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.6053000000000001e-05 2024-09-14T06:31:21.128 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:21.128 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:31:21.128 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:21.128 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:21.128 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:21.128 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4397.0:152325 3.9 e122/23 3:955152a4:::rbd_object_map.112d1bb7b269.0000000000000038:head v 122'268, mlcod=122'265)", 2024-09-14T06:31:21.128 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:30:44.914377+0000", 2024-09-14T06:31:21.128 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 36.133036093999998, 2024-09-14T06:31:21.129 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0020208840000000001, 2024-09-14T06:31:21.129 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:31:21.129 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:31:21.129 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:31:21.129 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:31:21.129 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:21.129 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:31:21.129 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.914377+0000", 2024-09-14T06:31:21.129 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:21.129 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:21.129 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:21.129 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:31:21.130 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.914377+0000", 2024-09-14T06:31:21.130 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:21.130 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:21.130 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:21.130 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:31:21.130 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.914378+0000", 2024-09-14T06:31:21.130 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.3600000000000003e-07 2024-09-14T06:31:21.130 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:21.130 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:21.130 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:31:21.130 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.914383+0000", 2024-09-14T06:31:21.130 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.7230000000000001e-06 2024-09-14T06:31:21.130 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:21.130 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:21.131 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:31:21.131 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.914393+0000", 2024-09-14T06:31:21.131 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.8179999999999999e-06 2024-09-14T06:31:21.131 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:21.131 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:21.131 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:31:21.131 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.914400+0000", 2024-09-14T06:31:21.131 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.968e-06 2024-09-14T06:31:21.131 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:21.131 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:21.131 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:31:21.131 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.914447+0000", 2024-09-14T06:31:21.131 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.7122e-05 2024-09-14T06:31:21.131 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:21.132 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:21.132 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:31:21.132 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.914466+0000", 2024-09-14T06:31:21.132 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.8958999999999999e-05 2024-09-14T06:31:21.132 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:21.132 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:21.132 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-14T06:31:21.132 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.916356+0000", 2024-09-14T06:31:21.132 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.001890121 2024-09-14T06:31:21.132 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:21.132 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:21.132 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:31:21.132 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.916398+0000", 2024-09-14T06:31:21.132 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.1437000000000003e-05 2024-09-14T06:31:21.132 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:21.133 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:31:21.133 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:21.133 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:21.133 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:21.133 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4397.0:152449 3.9 e122/23 3:955152a4:::rbd_object_map.112d1bb7b269.0000000000000038:head v 122'274, mlcod=122'268)", 2024-09-14T06:31:21.133 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:30:44.924063+0000", 2024-09-14T06:31:21.133 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 36.123349583, 2024-09-14T06:31:21.133 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.002131764, 2024-09-14T06:31:21.133 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:31:21.133 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:31:21.133 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:31:21.133 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:31:21.133 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:21.134 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:31:21.134 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.924063+0000", 2024-09-14T06:31:21.134 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:21.134 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:21.134 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:21.134 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:31:21.134 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.924063+0000", 2024-09-14T06:31:21.134 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:21.134 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:21.134 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:21.134 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:31:21.134 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.924064+0000", 2024-09-14T06:31:21.134 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.72e-07 2024-09-14T06:31:21.134 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:21.134 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:21.134 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:31:21.135 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.924071+0000", 2024-09-14T06:31:21.135 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.0999999999999998e-06 2024-09-14T06:31:21.135 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:21.135 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:21.135 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:31:21.135 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.924077+0000", 2024-09-14T06:31:21.135 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.268e-06 2024-09-14T06:31:21.135 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:21.135 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:21.135 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:31:21.135 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.924082+0000", 2024-09-14T06:31:21.135 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.7110000000000003e-06 2024-09-14T06:31:21.135 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:21.135 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:21.135 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:31:21.136 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.924397+0000", 2024-09-14T06:31:21.136 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00031503500000000002 2024-09-14T06:31:21.136 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:21.136 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:21.136 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:31:21.136 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.924418+0000", 2024-09-14T06:31:21.136 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.1044999999999999e-05 2024-09-14T06:31:21.136 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:21.136 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:21.136 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-14T06:31:21.136 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.926158+0000", 2024-09-14T06:31:21.136 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0017404860000000001 2024-09-14T06:31:21.136 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:21.136 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:21.137 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:31:21.137 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.926195+0000", 2024-09-14T06:31:21.137 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.6847000000000001e-05 2024-09-14T06:31:21.137 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:21.137 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:31:21.137 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:21.137 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:21.137 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:21.137 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4397.0:152452 3.9 e122/23 3:955152a4:::rbd_object_map.112d1bb7b269.0000000000000038:head v 122'275, mlcod=122'268)", 2024-09-14T06:31:21.137 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:30:44.924090+0000", 2024-09-14T06:31:21.137 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 36.123323263000003, 2024-09-14T06:31:21.137 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0021362400000000002, 2024-09-14T06:31:21.137 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:31:21.137 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:31:21.138 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:31:21.138 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:31:21.138 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:21.138 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:31:21.138 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.924090+0000", 2024-09-14T06:31:21.138 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:21.138 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:21.138 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:21.138 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:31:21.138 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.924090+0000", 2024-09-14T06:31:21.138 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:21.138 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:21.138 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:21.138 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:31:21.138 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.924090+0000", 2024-09-14T06:31:21.139 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.4699999999999998e-07 2024-09-14T06:31:21.139 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:21.139 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:21.139 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:31:21.139 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.924658+0000", 2024-09-14T06:31:21.139 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00056802700000000003 2024-09-14T06:31:21.139 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:21.139 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:21.139 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:31:21.139 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.924664+0000", 2024-09-14T06:31:21.139 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.3570000000000003e-06 2024-09-14T06:31:21.139 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:21.139 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:21.139 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:31:21.139 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.924669+0000", 2024-09-14T06:31:21.139 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.2300000000000002e-06 2024-09-14T06:31:21.140 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:21.140 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:21.140 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:31:21.140 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.924802+0000", 2024-09-14T06:31:21.140 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00013362500000000001 2024-09-14T06:31:21.140 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:21.140 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:21.140 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:31:21.140 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.924824+0000", 2024-09-14T06:31:21.140 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.1449000000000001e-05 2024-09-14T06:31:21.140 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:21.140 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:21.140 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-14T06:31:21.140 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.926197+0000", 2024-09-14T06:31:21.141 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.001373354 2024-09-14T06:31:21.141 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:21.141 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:21.141 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:31:21.141 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.926226+0000", 2024-09-14T06:31:21.141 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.8951000000000001e-05 2024-09-14T06:31:21.141 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:21.141 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:31:21.141 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:21.141 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:21.141 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:31:21.141 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-14T06:31:21.143 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-14T06:31:21.143 INFO:tasks.ceph.osd.4.smithi138.stderr:2024-09-14T06:31:21.142+0000 7f6c653c8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33615) UID: 0 2024-09-14T06:31:21.243 INFO:tasks.ceph.osd.7.smithi138.stderr:2024-09-14T06:31:21.242+0000 7f3c6a21d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33623) UID: 0 2024-09-14T06:31:21.245 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-14T06:31:21.245 INFO:teuthology.orchestra.run.smithi094.stdout: "ops": [], 2024-09-14T06:31:21.245 INFO:teuthology.orchestra.run.smithi094.stdout: "num_ops": 0 2024-09-14T06:31:21.245 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-14T06:31:21.254 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-14T06:31:21.344 INFO:tasks.ceph.osd.7.smithi138.stderr:2024-09-14T06:31:21.344+0000 7f3c6a21d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33623) UID: 0 2024-09-14T06:31:21.356 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-14T06:31:21.356 INFO:teuthology.orchestra.run.smithi094.stdout: "ops": [], 2024-09-14T06:31:21.356 INFO:teuthology.orchestra.run.smithi094.stdout: "complaint_time": 30, 2024-09-14T06:31:21.356 INFO:teuthology.orchestra.run.smithi094.stdout: "num_blocked_ops": 0 2024-09-14T06:31:21.356 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-14T06:31:21.364 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-14T06:31:21.445 INFO:tasks.ceph.osd.7.smithi138.stderr:2024-09-14T06:31:21.445+0000 7f3c6a21d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33623) UID: 0 2024-09-14T06:31:21.469 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-14T06:31:21.469 INFO:teuthology.orchestra.run.smithi094.stdout: "size": 20, 2024-09-14T06:31:21.469 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 600, 2024-09-14T06:31:21.469 INFO:teuthology.orchestra.run.smithi094.stdout: "ops": [ 2024-09-14T06:31:21.469 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:21.469 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:44425 3.2 3:4603ad7f:::rbd_data.112dca2f6ae1.0000000000000005:17 [read 868352~4096 [fadvise_nocache]] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:31:21.469 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:43.854967+0000", 2024-09-14T06:31:21.469 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 97.61223056, 2024-09-14T06:31:21.469 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.063146025999999994, 2024-09-14T06:31:21.470 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:31:21.470 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:31:21.470 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:31:21.470 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:31:21.470 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:31:21.470 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:21.470 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 44425 2024-09-14T06:31:21.470 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:21.470 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:31:21.470 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:21.470 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:31:21.470 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.854967+0000", 2024-09-14T06:31:21.470 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:21.471 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:21.471 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:21.471 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:31:21.471 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.854967+0000", 2024-09-14T06:31:21.471 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:21.471 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:21.471 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:21.471 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:31:21.471 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.854967+0000", 2024-09-14T06:31:21.471 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.16e-07 2024-09-14T06:31:21.472 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:21.472 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:21.472 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:31:21.472 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.854968+0000", 2024-09-14T06:31:21.472 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.006e-06 2024-09-14T06:31:21.472 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:21.472 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:21.472 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:31:21.472 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.854971+0000", 2024-09-14T06:31:21.472 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.9359999999999999e-06 2024-09-14T06:31:21.472 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:21.472 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:21.473 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:31:21.473 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.854974+0000", 2024-09-14T06:31:21.473 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.1840000000000002e-06 2024-09-14T06:31:21.473 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:21.473 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:21.473 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:31:21.473 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.917945+0000", 2024-09-14T06:31:21.473 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.062970925999999997 2024-09-14T06:31:21.473 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:21.473 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:21.473 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:31:21.473 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.918030+0000", 2024-09-14T06:31:21.473 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.5863000000000001e-05 2024-09-14T06:31:21.473 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:21.474 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:21.474 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:31:21.474 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.918113+0000", 2024-09-14T06:31:21.474 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.2894999999999998e-05 2024-09-14T06:31:21.474 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:21.474 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:31:21.474 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:21.474 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:21.474 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:21.474 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:44426 3.2 3:4603ad7f:::rbd_data.112dca2f6ae1.0000000000000005:17 [read 892928~4096 [fadvise_nocache]] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:31:21.474 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:43.854977+0000", 2024-09-14T06:31:21.474 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 97.612221149999996, 2024-09-14T06:31:21.474 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.063369237999999994, 2024-09-14T06:31:21.475 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:31:21.475 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:31:21.475 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:31:21.475 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:31:21.475 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:31:21.475 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:21.475 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 44426 2024-09-14T06:31:21.475 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:21.475 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:31:21.475 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:21.475 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:31:21.475 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.854977+0000", 2024-09-14T06:31:21.475 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:21.476 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:21.476 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:21.476 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:31:21.476 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.854977+0000", 2024-09-14T06:31:21.476 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:21.476 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:21.476 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:21.476 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:31:21.476 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.854977+0000", 2024-09-14T06:31:21.476 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.1199999999999999e-07 2024-09-14T06:31:21.476 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:21.476 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:21.476 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:31:21.476 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.854980+0000", 2024-09-14T06:31:21.477 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.27e-06 2024-09-14T06:31:21.477 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:21.477 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:21.477 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:31:21.477 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.854983+0000", 2024-09-14T06:31:21.477 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.1650000000000002e-06 2024-09-14T06:31:21.477 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:21.477 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:21.477 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:31:21.477 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.854989+0000", 2024-09-14T06:31:21.477 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.6520000000000003e-06 2024-09-14T06:31:21.477 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:21.477 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:21.477 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:31:21.478 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.918143+0000", 2024-09-14T06:31:21.478 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.063154541999999994 2024-09-14T06:31:21.478 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:21.478 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:21.478 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:31:21.478 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.918209+0000", 2024-09-14T06:31:21.478 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.5529000000000002e-05 2024-09-14T06:31:21.478 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:21.478 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:21.478 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:31:21.478 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.918346+0000", 2024-09-14T06:31:21.478 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000136868 2024-09-14T06:31:21.478 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:21.478 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:31:21.479 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:21.479 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:21.479 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:21.479 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:44427 3.2 3:4603ad7f:::rbd_data.112dca2f6ae1.0000000000000005:17 [read 917504~4096 [fadvise_nocache]] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:31:21.479 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:43.854992+0000", 2024-09-14T06:31:21.479 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 97.612205504000002, 2024-09-14T06:31:21.479 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.063607792999999996, 2024-09-14T06:31:21.479 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:31:21.479 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:31:21.479 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:31:21.479 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:31:21.479 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:31:21.479 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:21.479 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 44427 2024-09-14T06:31:21.480 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:21.480 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:31:21.480 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:21.480 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:31:21.480 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.854992+0000", 2024-09-14T06:31:21.480 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:21.480 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:21.480 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:21.480 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:31:21.480 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.854992+0000", 2024-09-14T06:31:21.480 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:21.480 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:21.480 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:21.481 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:31:21.481 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.854993+0000", 2024-09-14T06:31:21.481 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.1400000000000001e-07 2024-09-14T06:31:21.481 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:21.481 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:21.481 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:31:21.481 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.854994+0000", 2024-09-14T06:31:21.481 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.091e-06 2024-09-14T06:31:21.481 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:21.481 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:21.481 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:31:21.481 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855001+0000", 2024-09-14T06:31:21.481 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.204e-06 2024-09-14T06:31:21.481 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:21.481 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:21.482 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:31:21.482 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855003+0000", 2024-09-14T06:31:21.482 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.514e-06 2024-09-14T06:31:21.482 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:21.482 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:21.482 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:31:21.482 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.918380+0000", 2024-09-14T06:31:21.482 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.063376738000000002 2024-09-14T06:31:21.482 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:21.482 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:21.482 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:31:21.482 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.918453+0000", 2024-09-14T06:31:21.482 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.3291e-05 2024-09-14T06:31:21.483 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:21.483 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:21.483 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:31:21.483 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.918600+0000", 2024-09-14T06:31:21.483 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00014674100000000001 2024-09-14T06:31:21.483 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:21.483 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:31:21.483 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:21.483 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:21.483 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:21.483 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:44428 3.2 3:4603ad7f:::rbd_data.112dca2f6ae1.0000000000000005:17 [read 942080~4096 [fadvise_nocache]] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:31:21.483 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:43.855006+0000", 2024-09-14T06:31:21.483 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 97.612191453999998, 2024-09-14T06:31:21.484 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.063846321999999997, 2024-09-14T06:31:21.484 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:31:21.484 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:31:21.484 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:31:21.484 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:31:21.484 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:31:21.484 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:21.484 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 44428 2024-09-14T06:31:21.484 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:21.484 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:31:21.484 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:21.484 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:31:21.484 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855006+0000", 2024-09-14T06:31:21.484 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:21.485 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:21.485 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:21.485 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:31:21.485 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855006+0000", 2024-09-14T06:31:21.485 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:21.485 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:21.485 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:21.485 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:31:21.485 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855007+0000", 2024-09-14T06:31:21.485 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.1199999999999999e-07 2024-09-14T06:31:21.485 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:21.485 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:21.485 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:31:21.486 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855011+0000", 2024-09-14T06:31:21.486 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.9539999999999998e-06 2024-09-14T06:31:21.486 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:21.486 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:21.486 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:31:21.486 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855014+0000", 2024-09-14T06:31:21.486 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.2930000000000001e-06 2024-09-14T06:31:21.486 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:21.486 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:21.486 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:31:21.486 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855016+0000", 2024-09-14T06:31:21.486 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.3039999999999999e-06 2024-09-14T06:31:21.486 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:21.486 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:21.487 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:31:21.487 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.918634+0000", 2024-09-14T06:31:21.487 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.063617928000000004 2024-09-14T06:31:21.487 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:21.487 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:21.487 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:31:21.487 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.918711+0000", 2024-09-14T06:31:21.487 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.6545000000000007e-05 2024-09-14T06:31:21.487 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:21.487 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:21.487 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:31:21.487 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.918853+0000", 2024-09-14T06:31:21.487 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00014208600000000001 2024-09-14T06:31:21.487 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:21.488 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:31:21.488 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:21.488 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:21.488 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:21.488 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:44429 3.2 3:4603ad7f:::rbd_data.112dca2f6ae1.0000000000000005:17 [read 966656~4096 [fadvise_nocache]] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:31:21.488 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:43.855019+0000", 2024-09-14T06:31:21.488 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 97.612178780999997, 2024-09-14T06:31:21.488 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.064059226999999996, 2024-09-14T06:31:21.488 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:31:21.488 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:31:21.488 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:31:21.488 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:31:21.488 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:31:21.489 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:21.489 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 44429 2024-09-14T06:31:21.489 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:21.489 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:31:21.489 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:21.489 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:31:21.489 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855019+0000", 2024-09-14T06:31:21.489 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:21.489 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:21.489 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:21.489 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:31:21.489 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855019+0000", 2024-09-14T06:31:21.489 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:21.489 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:21.490 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:21.490 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:31:21.490 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855019+0000", 2024-09-14T06:31:21.490 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.28e-07 2024-09-14T06:31:21.490 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:21.490 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:21.490 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:31:21.490 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855020+0000", 2024-09-14T06:31:21.490 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.0330000000000001e-06 2024-09-14T06:31:21.490 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:21.490 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:21.490 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:31:21.490 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855023+0000", 2024-09-14T06:31:21.490 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.79e-06 2024-09-14T06:31:21.491 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:21.491 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:21.491 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:31:21.491 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855025+0000", 2024-09-14T06:31:21.491 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.216e-06 2024-09-14T06:31:21.491 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:21.491 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:21.491 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:31:21.491 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.918886+0000", 2024-09-14T06:31:21.491 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.063861037999999995 2024-09-14T06:31:21.491 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:21.491 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:21.491 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:31:21.491 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.918952+0000", 2024-09-14T06:31:21.492 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.5993000000000003e-05 2024-09-14T06:31:21.492 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:21.492 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:21.492 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:31:21.492 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.919078+0000", 2024-09-14T06:31:21.492 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00012592899999999999 2024-09-14T06:31:21.492 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:21.492 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:31:21.492 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:21.492 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:21.492 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:21.492 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:44430 3.2 3:4603ad7f:::rbd_data.112dca2f6ae1.0000000000000005:17 [read 991232~4096 [fadvise_nocache]] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:31:21.492 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:43.855028+0000", 2024-09-14T06:31:21.493 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 97.612169770999998, 2024-09-14T06:31:21.493 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.064284346000000006, 2024-09-14T06:31:21.493 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:31:21.493 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:31:21.493 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:31:21.493 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:31:21.493 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:31:21.493 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:21.493 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 44430 2024-09-14T06:31:21.493 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:21.493 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:31:21.493 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:21.493 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:31:21.493 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855028+0000", 2024-09-14T06:31:21.494 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:21.494 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:21.494 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:21.494 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:31:21.494 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855028+0000", 2024-09-14T06:31:21.494 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:21.494 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:21.494 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:21.494 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:31:21.494 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855028+0000", 2024-09-14T06:31:21.494 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.1199999999999999e-07 2024-09-14T06:31:21.494 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:21.494 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:21.495 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:31:21.495 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855029+0000", 2024-09-14T06:31:21.495 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.1100000000000004e-07 2024-09-14T06:31:21.495 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:21.495 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:21.495 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:31:21.495 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855032+0000", 2024-09-14T06:31:21.495 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.7889999999999999e-06 2024-09-14T06:31:21.495 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:21.495 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:21.495 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:31:21.495 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855034+0000", 2024-09-14T06:31:21.495 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.125e-06 2024-09-14T06:31:21.495 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:21.496 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:21.496 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:31:21.496 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.919110+0000", 2024-09-14T06:31:21.496 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.064076413999999998 2024-09-14T06:31:21.496 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:21.496 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:21.496 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:31:21.496 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.919186+0000", 2024-09-14T06:31:21.496 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.5784999999999994e-05 2024-09-14T06:31:21.496 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:21.496 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:21.496 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:31:21.496 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.919312+0000", 2024-09-14T06:31:21.497 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00012611 2024-09-14T06:31:21.497 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:21.497 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:31:21.497 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:21.497 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:21.497 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:21.497 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:44431 3.2 3:4603ad7f:::rbd_data.112dca2f6ae1.0000000000000005:17 [read 1015808~4096 [fadvise_nocache]] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:31:21.497 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:43.855037+0000", 2024-09-14T06:31:21.497 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 97.612160903000003, 2024-09-14T06:31:21.497 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.064469755000000004, 2024-09-14T06:31:21.497 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:31:21.497 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:31:21.497 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:31:21.498 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:31:21.498 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:31:21.498 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:21.498 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 44431 2024-09-14T06:31:21.498 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:21.498 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:31:21.498 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:21.498 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:31:21.498 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855037+0000", 2024-09-14T06:31:21.498 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:21.498 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:21.498 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:21.498 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:31:21.499 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855037+0000", 2024-09-14T06:31:21.499 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:21.499 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:21.499 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:21.499 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:31:21.499 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855037+0000", 2024-09-14T06:31:21.499 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.1e-07 2024-09-14T06:31:21.499 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:21.499 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:21.499 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:31:21.499 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855038+0000", 2024-09-14T06:31:21.499 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.9400000000000004e-07 2024-09-14T06:31:21.499 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:21.499 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:21.500 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:31:21.500 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855041+0000", 2024-09-14T06:31:21.500 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.9639999999999999e-06 2024-09-14T06:31:21.500 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:21.500 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:21.500 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:31:21.500 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855043+0000", 2024-09-14T06:31:21.500 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.2259999999999999e-06 2024-09-14T06:31:21.500 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:21.500 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:21.500 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:31:21.500 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.919347+0000", 2024-09-14T06:31:21.500 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.064304091999999993 2024-09-14T06:31:21.500 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:21.501 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:21.501 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:31:21.501 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.919393+0000", 2024-09-14T06:31:21.501 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.5340999999999998e-05 2024-09-14T06:31:21.501 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:21.501 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:21.501 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:31:21.501 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.919507+0000", 2024-09-14T06:31:21.501 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000114028 2024-09-14T06:31:21.501 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:21.501 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:31:21.501 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:21.501 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:21.502 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:21.502 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:44432 3.2 3:4603ad7f:::rbd_data.112dca2f6ae1.0000000000000005:17 [read 1040384~4096 [fadvise_nocache]] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:31:21.502 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:43.855046+0000", 2024-09-14T06:31:21.502 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 97.612151835000006, 2024-09-14T06:31:21.502 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.064639253999999993, 2024-09-14T06:31:21.502 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:31:21.502 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:31:21.502 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:31:21.502 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:31:21.502 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:31:21.502 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:21.502 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 44432 2024-09-14T06:31:21.502 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:21.502 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:31:21.503 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:21.503 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:31:21.503 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855046+0000", 2024-09-14T06:31:21.503 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:21.503 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:21.503 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:21.503 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:31:21.503 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855046+0000", 2024-09-14T06:31:21.503 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:21.503 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:21.503 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:21.503 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:31:21.503 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855046+0000", 2024-09-14T06:31:21.503 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.05e-07 2024-09-14T06:31:21.504 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:21.504 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:21.504 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:31:21.504 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855047+0000", 2024-09-14T06:31:21.504 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.879999999999999e-07 2024-09-14T06:31:21.504 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:21.504 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:21.504 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:31:21.504 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855050+0000", 2024-09-14T06:31:21.504 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.8459999999999999e-06 2024-09-14T06:31:21.504 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:21.504 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:21.504 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:31:21.504 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855052+0000", 2024-09-14T06:31:21.505 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.136e-06 2024-09-14T06:31:21.505 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:21.505 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:21.505 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:31:21.505 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.919536+0000", 2024-09-14T06:31:21.505 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.064484132 2024-09-14T06:31:21.505 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:21.505 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:21.505 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:31:21.505 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.919611+0000", 2024-09-14T06:31:21.505 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.4407999999999996e-05 2024-09-14T06:31:21.505 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:21.505 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:21.505 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:31:21.506 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.919685+0000", 2024-09-14T06:31:21.506 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.4538999999999999e-05 2024-09-14T06:31:21.506 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:21.506 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:31:21.506 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:21.506 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:21.506 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:21.506 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:44518 3.2 3:4190d882:::rbd_data.112dca2f6ae1.0000000000000002:17 [read 12288~4096 [fadvise_nocache]] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:31:21.506 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:43.855055+0000", 2024-09-14T06:31:21.506 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 97.612142950999996, 2024-09-14T06:31:21.506 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.064894317000000007, 2024-09-14T06:31:21.506 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:31:21.506 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:31:21.506 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:31:21.507 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:31:21.507 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:31:21.507 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:21.507 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 44518 2024-09-14T06:31:21.507 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:21.507 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:31:21.507 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:21.507 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:31:21.507 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855055+0000", 2024-09-14T06:31:21.507 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:21.507 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:21.507 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:21.507 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:31:21.507 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855055+0000", 2024-09-14T06:31:21.508 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:21.508 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:21.508 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:21.508 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:31:21.508 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855055+0000", 2024-09-14T06:31:21.508 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.0699999999999999e-07 2024-09-14T06:31:21.508 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:21.508 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:21.508 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:31:21.508 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855056+0000", 2024-09-14T06:31:21.508 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.9400000000000004e-07 2024-09-14T06:31:21.508 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:21.508 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:21.509 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:31:21.509 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855059+0000", 2024-09-14T06:31:21.509 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.8870000000000001e-06 2024-09-14T06:31:21.509 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:21.509 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:21.509 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:31:21.509 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855061+0000", 2024-09-14T06:31:21.509 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.1919999999999999e-06 2024-09-14T06:31:21.509 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:21.509 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:21.509 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:31:21.509 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.919719+0000", 2024-09-14T06:31:21.509 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.064658013 2024-09-14T06:31:21.509 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:21.510 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:21.510 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:31:21.510 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.919788+0000", 2024-09-14T06:31:21.510 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.9282000000000005e-05 2024-09-14T06:31:21.510 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:21.510 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:21.510 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:31:21.510 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.919949+0000", 2024-09-14T06:31:21.510 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00016084199999999999 2024-09-14T06:31:21.510 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:21.510 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:31:21.510 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:21.510 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:21.511 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:21.511 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:44519 3.2 3:4190d882:::rbd_data.112dca2f6ae1.0000000000000002:17 [read 36864~4096 [fadvise_nocache]] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:31:21.511 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:43.855064+0000", 2024-09-14T06:31:21.511 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 97.612133974000002, 2024-09-14T06:31:21.511 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.065081865000000003, 2024-09-14T06:31:21.511 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:31:21.511 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:31:21.511 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:31:21.511 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:31:21.511 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:31:21.511 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:21.511 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 44519 2024-09-14T06:31:21.511 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:21.511 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:31:21.511 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:21.512 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:31:21.512 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855064+0000", 2024-09-14T06:31:21.512 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:21.512 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:21.512 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:21.512 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:31:21.512 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855064+0000", 2024-09-14T06:31:21.512 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:21.512 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:21.512 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:21.512 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:31:21.512 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855064+0000", 2024-09-14T06:31:21.512 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.1e-07 2024-09-14T06:31:21.513 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:21.513 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:21.513 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:31:21.513 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855065+0000", 2024-09-14T06:31:21.513 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.035e-06 2024-09-14T06:31:21.513 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:21.513 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:21.513 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:31:21.513 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855068+0000", 2024-09-14T06:31:21.513 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.8339999999999999e-06 2024-09-14T06:31:21.513 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:21.513 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:21.513 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:31:21.513 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855071+0000", 2024-09-14T06:31:21.513 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.3450000000000002e-06 2024-09-14T06:31:21.514 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:21.514 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:21.514 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:31:21.514 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.919979+0000", 2024-09-14T06:31:21.514 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.064907647999999998 2024-09-14T06:31:21.514 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:21.514 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:21.514 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:31:21.514 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.920026+0000", 2024-09-14T06:31:21.514 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.7595000000000001e-05 2024-09-14T06:31:21.514 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:21.514 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:21.514 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:31:21.514 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.920146+0000", 2024-09-14T06:31:21.515 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00011919800000000001 2024-09-14T06:31:21.515 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:21.515 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:31:21.515 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:21.515 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:21.515 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:21.515 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:44520 3.2 3:4190d882:::rbd_data.112dca2f6ae1.0000000000000002:17 [read 61440~4096 [fadvise_nocache]] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:31:21.515 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:43.855075+0000", 2024-09-14T06:31:21.515 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 97.612122923000001, 2024-09-14T06:31:21.515 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.065335078000000005, 2024-09-14T06:31:21.515 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:31:21.515 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:31:21.515 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:31:21.515 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:31:21.516 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:31:21.516 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:21.516 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 44520 2024-09-14T06:31:21.516 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:21.516 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:31:21.516 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:21.516 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:31:21.516 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855075+0000", 2024-09-14T06:31:21.516 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:21.516 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:21.516 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:21.516 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:31:21.516 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855075+0000", 2024-09-14T06:31:21.517 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:21.517 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:21.517 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:21.517 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:31:21.517 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855075+0000", 2024-09-14T06:31:21.517 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.1e-07 2024-09-14T06:31:21.517 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:21.517 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:21.517 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:31:21.517 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855076+0000", 2024-09-14T06:31:21.517 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.0839999999999999e-06 2024-09-14T06:31:21.517 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:21.517 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:21.517 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:31:21.518 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855079+0000", 2024-09-14T06:31:21.518 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.9359999999999999e-06 2024-09-14T06:31:21.518 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:21.518 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:21.518 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:31:21.518 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855084+0000", 2024-09-14T06:31:21.518 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.1730000000000003e-06 2024-09-14T06:31:21.518 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:21.518 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:21.518 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:31:21.518 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.920175+0000", 2024-09-14T06:31:21.518 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.065090234999999996 2024-09-14T06:31:21.518 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:21.518 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:21.519 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:31:21.519 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.920250+0000", 2024-09-14T06:31:21.519 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.5807000000000001e-05 2024-09-14T06:31:21.519 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:21.519 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:21.519 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:31:21.519 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.920410+0000", 2024-09-14T06:31:21.519 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00015963300000000001 2024-09-14T06:31:21.519 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:21.519 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:31:21.519 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:21.519 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:21.519 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:21.519 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:44521 3.2 3:4190d882:::rbd_data.112dca2f6ae1.0000000000000002:17 [read 86016~4096 [fadvise_nocache]] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:31:21.520 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:43.855087+0000", 2024-09-14T06:31:21.520 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 97.612110442000002, 2024-09-14T06:31:21.520 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.065563264999999996, 2024-09-14T06:31:21.520 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:31:21.520 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:31:21.520 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:31:21.520 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:31:21.520 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:31:21.520 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:21.520 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 44521 2024-09-14T06:31:21.520 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:21.520 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:31:21.520 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:21.520 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:31:21.521 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855087+0000", 2024-09-14T06:31:21.521 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:21.521 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:21.521 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:21.521 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:31:21.521 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855087+0000", 2024-09-14T06:31:21.521 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:21.521 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:21.521 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:21.521 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:31:21.521 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855088+0000", 2024-09-14T06:31:21.521 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.3699999999999999e-07 2024-09-14T06:31:21.521 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:21.521 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:21.522 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:31:21.522 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855089+0000", 2024-09-14T06:31:21.522 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.9900000000000009e-07 2024-09-14T06:31:21.522 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:21.522 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:21.522 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:31:21.522 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855091+0000", 2024-09-14T06:31:21.522 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.8940000000000002e-06 2024-09-14T06:31:21.522 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:21.522 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:21.522 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:31:21.522 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855094+0000", 2024-09-14T06:31:21.522 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.351e-06 2024-09-14T06:31:21.523 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:21.523 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:21.523 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:31:21.523 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.920437+0000", 2024-09-14T06:31:21.523 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.065343221000000007 2024-09-14T06:31:21.523 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:21.523 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:21.523 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:31:21.523 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.920481+0000", 2024-09-14T06:31:21.523 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.4431000000000002e-05 2024-09-14T06:31:21.523 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:21.523 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:21.523 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:31:21.523 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.920651+0000", 2024-09-14T06:31:21.524 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00016913200000000001 2024-09-14T06:31:21.524 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:21.524 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:31:21.524 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:21.524 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:21.524 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:21.524 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:44522 3.2 3:4190d882:::rbd_data.112dca2f6ae1.0000000000000002:17 [read 110592~4096 [fadvise_nocache]] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:31:21.524 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:43.855098+0000", 2024-09-14T06:31:21.524 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 97.612100214999998, 2024-09-14T06:31:21.524 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.065827628999999999, 2024-09-14T06:31:21.524 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:31:21.524 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:31:21.524 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:31:21.524 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:31:21.525 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:31:21.525 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:21.525 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 44522 2024-09-14T06:31:21.525 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:21.525 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:31:21.525 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:21.525 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:31:21.525 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855098+0000", 2024-09-14T06:31:21.525 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:21.525 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:21.525 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:21.525 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:31:21.525 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855098+0000", 2024-09-14T06:31:21.525 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:21.526 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:21.526 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:21.526 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:31:21.526 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855098+0000", 2024-09-14T06:31:21.526 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.1e-07 2024-09-14T06:31:21.526 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:21.526 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:21.526 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:31:21.526 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855099+0000", 2024-09-14T06:31:21.526 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.012e-06 2024-09-14T06:31:21.526 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:21.526 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:21.526 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:31:21.526 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855102+0000", 2024-09-14T06:31:21.527 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.9380000000000001e-06 2024-09-14T06:31:21.527 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:21.527 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:21.527 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:31:21.527 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855108+0000", 2024-09-14T06:31:21.527 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.4339999999999997e-06 2024-09-14T06:31:21.527 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:21.527 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:21.527 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:31:21.527 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.920681+0000", 2024-09-14T06:31:21.527 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.065572943999999994 2024-09-14T06:31:21.527 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:21.527 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:21.527 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:31:21.527 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.920724+0000", 2024-09-14T06:31:21.528 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.2877000000000003e-05 2024-09-14T06:31:21.528 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:21.528 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:21.528 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:31:21.528 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.920925+0000", 2024-09-14T06:31:21.528 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00020121399999999999 2024-09-14T06:31:21.528 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:21.528 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:31:21.528 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:21.528 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:21.528 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:21.528 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:44523 3.2 3:4190d882:::rbd_data.112dca2f6ae1.0000000000000002:17 [read 135168~4096 [fadvise_nocache]] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:31:21.528 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:43.855111+0000", 2024-09-14T06:31:21.528 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 97.612086493000007, 2024-09-14T06:31:21.528 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.066028198999999996, 2024-09-14T06:31:21.528 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:31:21.529 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:31:21.529 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:31:21.529 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:31:21.529 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:31:21.529 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:21.529 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 44523 2024-09-14T06:31:21.529 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:21.529 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:31:21.529 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:21.529 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:31:21.529 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855111+0000", 2024-09-14T06:31:21.529 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:21.529 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:21.529 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:21.530 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:31:21.530 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855111+0000", 2024-09-14T06:31:21.530 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:21.530 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:21.530 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:21.530 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:31:21.530 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855112+0000", 2024-09-14T06:31:21.530 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.1199999999999999e-07 2024-09-14T06:31:21.530 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:21.530 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:21.530 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:31:21.530 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855115+0000", 2024-09-14T06:31:21.530 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.3380000000000001e-06 2024-09-14T06:31:21.531 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:21.531 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:21.531 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:31:21.531 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855118+0000", 2024-09-14T06:31:21.531 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.366e-06 2024-09-14T06:31:21.531 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:21.531 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:21.531 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:31:21.531 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855121+0000", 2024-09-14T06:31:21.531 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.8329999999999998e-06 2024-09-14T06:31:21.532 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:21.532 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:21.532 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:31:21.532 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.920956+0000", 2024-09-14T06:31:21.532 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.065835251999999997 2024-09-14T06:31:21.532 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:21.532 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:21.532 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:31:21.532 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.921001+0000", 2024-09-14T06:31:21.532 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.4778999999999999e-05 2024-09-14T06:31:21.532 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:21.532 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:21.532 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:31:21.532 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.921139+0000", 2024-09-14T06:31:21.533 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000138419 2024-09-14T06:31:21.533 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:21.533 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:31:21.533 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:21.533 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:21.533 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:21.533 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:44524 3.2 3:4190d882:::rbd_data.112dca2f6ae1.0000000000000002:17 [read 159744~4096 [fadvise_nocache]] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:31:21.533 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:43.855124+0000", 2024-09-14T06:31:21.533 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 97.612073514000002, 2024-09-14T06:31:21.533 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.066249261000000004, 2024-09-14T06:31:21.533 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:31:21.533 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:31:21.533 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:31:21.534 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:31:21.534 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:31:21.534 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:21.534 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 44524 2024-09-14T06:31:21.534 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:21.534 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:31:21.534 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:21.534 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:31:21.534 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855124+0000", 2024-09-14T06:31:21.534 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:21.534 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:21.534 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:21.535 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:31:21.535 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855124+0000", 2024-09-14T06:31:21.535 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:21.535 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:21.535 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:21.535 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:31:21.535 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861086+0000", 2024-09-14T06:31:21.535 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0059613829999999998 2024-09-14T06:31:21.535 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:21.535 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:21.535 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:31:21.535 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861100+0000", 2024-09-14T06:31:21.535 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.4644000000000001e-05 2024-09-14T06:31:21.535 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:21.536 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:21.536 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:31:21.536 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861109+0000", 2024-09-14T06:31:21.536 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.541e-06 2024-09-14T06:31:21.536 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:21.536 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:21.536 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:31:21.536 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861117+0000", 2024-09-14T06:31:21.536 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.1049999999999995e-06 2024-09-14T06:31:21.536 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:21.536 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:21.536 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:31:21.536 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.921169+0000", 2024-09-14T06:31:21.537 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.060052405000000003 2024-09-14T06:31:21.537 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:21.537 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:21.537 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:31:21.537 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.921213+0000", 2024-09-14T06:31:21.537 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.3565e-05 2024-09-14T06:31:21.537 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:21.537 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:21.537 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:31:21.537 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.921374+0000", 2024-09-14T06:31:21.537 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00016061800000000001 2024-09-14T06:31:21.537 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:21.537 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:31:21.537 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:21.537 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:21.538 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:21.538 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:44536 3.2 3:4190d882:::rbd_data.112dca2f6ae1.0000000000000002:17 [read 454656~4096 [fadvise_nocache]] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:31:21.538 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:43.861384+0000", 2024-09-14T06:31:21.538 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 97.605813956999995, 2024-09-14T06:31:21.538 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.063058528000000003, 2024-09-14T06:31:21.538 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:31:21.538 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:31:21.538 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:31:21.538 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:31:21.538 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:31:21.538 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:21.538 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 44536 2024-09-14T06:31:21.538 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:21.538 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:31:21.539 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:21.539 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:31:21.539 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861384+0000", 2024-09-14T06:31:21.539 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:21.539 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:21.539 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:21.539 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:31:21.539 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861384+0000", 2024-09-14T06:31:21.539 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:21.539 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:21.539 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:21.539 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:31:21.539 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861384+0000", 2024-09-14T06:31:21.539 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.4000000000000002e-07 2024-09-14T06:31:21.539 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:21.539 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:21.540 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:31:21.540 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861386+0000", 2024-09-14T06:31:21.540 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.891e-06 2024-09-14T06:31:21.540 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:21.540 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:21.540 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:31:21.540 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861392+0000", 2024-09-14T06:31:21.540 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.9939999999999997e-06 2024-09-14T06:31:21.540 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:21.540 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:21.540 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:31:21.540 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861398+0000", 2024-09-14T06:31:21.540 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.507e-06 2024-09-14T06:31:21.540 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:21.541 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:21.541 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:31:21.541 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.924149+0000", 2024-09-14T06:31:21.541 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.062751135999999999 2024-09-14T06:31:21.541 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:21.541 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:21.541 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:31:21.541 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.924226+0000", 2024-09-14T06:31:21.541 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.7517999999999999e-05 2024-09-14T06:31:21.541 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:21.541 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:21.541 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:31:21.541 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.924442+0000", 2024-09-14T06:31:21.542 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000216042 2024-09-14T06:31:21.542 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:21.542 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:31:21.542 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:21.542 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:21.542 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:21.542 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:44537 3.2 3:4190d882:::rbd_data.112dca2f6ae1.0000000000000002:17 [read 479232~4096 [fadvise_nocache]] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:31:21.542 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:43.861405+0000", 2024-09-14T06:31:21.542 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 97.605792910999995, 2024-09-14T06:31:21.542 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.063351795000000002, 2024-09-14T06:31:21.542 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:31:21.542 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:31:21.542 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:31:21.542 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:31:21.542 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:31:21.543 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:21.543 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 44537 2024-09-14T06:31:21.543 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:21.543 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:31:21.543 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:21.543 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:31:21.543 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861405+0000", 2024-09-14T06:31:21.543 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:21.543 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:21.543 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:21.543 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:31:21.543 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861405+0000", 2024-09-14T06:31:21.543 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:21.543 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:21.544 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:21.544 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:31:21.544 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861405+0000", 2024-09-14T06:31:21.544 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.9099999999999999e-07 2024-09-14T06:31:21.544 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:21.544 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:21.544 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:31:21.544 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861407+0000", 2024-09-14T06:31:21.544 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.075e-06 2024-09-14T06:31:21.544 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:21.544 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:21.544 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:31:21.544 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861414+0000", 2024-09-14T06:31:21.545 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.4659999999999996e-06 2024-09-14T06:31:21.545 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:21.545 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:21.545 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:31:21.545 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861418+0000", 2024-09-14T06:31:21.545 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.5519999999999998e-06 2024-09-14T06:31:21.545 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:21.545 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:21.545 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:31:21.545 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.924474+0000", 2024-09-14T06:31:21.545 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.063056012999999994 2024-09-14T06:31:21.545 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:21.545 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:21.545 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:31:21.546 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.924556+0000", 2024-09-14T06:31:21.546 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.1181999999999996e-05 2024-09-14T06:31:21.546 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:21.546 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:21.546 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:31:21.546 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.924757+0000", 2024-09-14T06:31:21.546 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00020111599999999999 2024-09-14T06:31:21.546 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:21.546 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:31:21.546 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:21.546 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:21.546 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:21.546 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:44538 3.2 3:4190d882:::rbd_data.112dca2f6ae1.0000000000000002:17 [read 503808~4096 [fadvise_nocache]] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:31:21.546 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:43.861425+0000", 2024-09-14T06:31:21.547 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 97.605772869999996, 2024-09-14T06:31:21.547 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.063627125000000007, 2024-09-14T06:31:21.547 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:31:21.547 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:31:21.547 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:31:21.547 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:31:21.547 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:31:21.547 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:21.547 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 44538 2024-09-14T06:31:21.547 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:21.547 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:31:21.547 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:21.547 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:31:21.547 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861425+0000", 2024-09-14T06:31:21.548 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:21.548 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:21.548 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:21.548 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:31:21.548 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861425+0000", 2024-09-14T06:31:21.548 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:21.548 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:21.548 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:21.548 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:31:21.548 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861425+0000", 2024-09-14T06:31:21.548 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.9999999999999998e-07 2024-09-14T06:31:21.548 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:21.548 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:21.549 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:31:21.549 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861427+0000", 2024-09-14T06:31:21.549 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.9829999999999999e-06 2024-09-14T06:31:21.549 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:21.549 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:21.549 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:31:21.549 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861433+0000", 2024-09-14T06:31:21.549 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.2889999999999997e-06 2024-09-14T06:31:21.549 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:21.549 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:21.549 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:31:21.549 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861454+0000", 2024-09-14T06:31:21.549 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.1844999999999998e-05 2024-09-14T06:31:21.549 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:21.550 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:21.550 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:31:21.550 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.924786+0000", 2024-09-14T06:31:21.550 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.063331497 2024-09-14T06:31:21.550 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:21.550 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:21.550 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:31:21.550 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.924864+0000", 2024-09-14T06:31:21.550 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.7744000000000006e-05 2024-09-14T06:31:21.550 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:21.550 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:21.550 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:31:21.550 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.925052+0000", 2024-09-14T06:31:21.550 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00018836699999999999 2024-09-14T06:31:21.550 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:21.551 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:31:21.551 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:21.551 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:21.551 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:21.551 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:44539 3.2 3:4190d882:::rbd_data.112dca2f6ae1.0000000000000002:17 [read 528384~4096 [fadvise_nocache]] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:31:21.551 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:43.861462+0000", 2024-09-14T06:31:21.551 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 97.605735542999994, 2024-09-14T06:31:21.551 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.063855705999999998, 2024-09-14T06:31:21.551 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:31:21.551 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:31:21.551 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:31:21.551 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:31:21.551 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:31:21.551 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:21.552 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 44539 2024-09-14T06:31:21.552 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:21.552 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:31:21.552 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:21.552 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:31:21.552 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861462+0000", 2024-09-14T06:31:21.552 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:21.552 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:21.552 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:21.552 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:31:21.552 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861462+0000", 2024-09-14T06:31:21.552 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:21.552 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:21.552 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:21.553 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:31:21.553 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861463+0000", 2024-09-14T06:31:21.553 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.5200000000000002e-07 2024-09-14T06:31:21.553 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:21.553 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:21.553 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:31:21.553 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861465+0000", 2024-09-14T06:31:21.553 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.5189999999999999e-06 2024-09-14T06:31:21.553 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:21.553 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:21.553 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:31:21.553 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861471+0000", 2024-09-14T06:31:21.553 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.6899999999999997e-06 2024-09-14T06:31:21.554 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:21.554 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:21.554 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:31:21.554 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861475+0000", 2024-09-14T06:31:21.554 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.5730000000000001e-06 2024-09-14T06:31:21.554 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:21.554 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:21.554 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:31:21.554 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.925085+0000", 2024-09-14T06:31:21.554 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.063609902999999995 2024-09-14T06:31:21.554 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:21.554 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:21.554 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:31:21.554 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.925145+0000", 2024-09-14T06:31:21.554 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.9383999999999999e-05 2024-09-14T06:31:21.555 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:21.555 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:21.555 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:31:21.555 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.925318+0000", 2024-09-14T06:31:21.555 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00017318499999999999 2024-09-14T06:31:21.555 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:21.555 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:31:21.555 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:21.555 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:21.555 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:21.555 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:44540 3.2 3:4190d882:::rbd_data.112dca2f6ae1.0000000000000002:17 [read 552960~4096 [fadvise_nocache]] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:31:21.555 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:43.861483+0000", 2024-09-14T06:31:21.555 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 97.605715111999999, 2024-09-14T06:31:21.555 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.064065681999999999, 2024-09-14T06:31:21.555 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:31:21.556 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:31:21.556 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:31:21.556 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:31:21.556 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:31:21.556 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:21.556 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 44540 2024-09-14T06:31:21.556 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:21.556 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:31:21.556 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:21.556 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:31:21.556 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861483+0000", 2024-09-14T06:31:21.556 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:21.556 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:21.556 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:21.557 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:31:21.557 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861483+0000", 2024-09-14T06:31:21.557 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:21.557 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:21.557 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:21.557 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:31:21.557 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.867079+0000", 2024-09-14T06:31:21.557 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0055966480000000001 2024-09-14T06:31:21.557 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:21.557 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:21.557 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:31:21.557 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.867086+0000", 2024-09-14T06:31:21.557 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.387e-06 2024-09-14T06:31:21.557 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:21.557 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:21.558 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:31:21.558 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.867094+0000", 2024-09-14T06:31:21.558 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.9419999999999995e-06 2024-09-14T06:31:21.558 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:21.558 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:21.558 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:31:21.558 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.867109+0000", 2024-09-14T06:31:21.558 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.5818999999999999e-05 2024-09-14T06:31:21.558 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:21.558 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:21.558 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:31:21.558 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.925354+0000", 2024-09-14T06:31:21.558 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.058244047 2024-09-14T06:31:21.558 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:21.558 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:21.559 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:31:21.559 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.925406+0000", 2024-09-14T06:31:21.559 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.2661999999999998e-05 2024-09-14T06:31:21.559 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:21.559 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:21.559 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:31:21.559 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.925548+0000", 2024-09-14T06:31:21.559 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000142177 2024-09-14T06:31:21.559 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:21.559 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:31:21.559 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:21.559 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:21.559 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:31:21.559 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-14T06:31:21.561 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-14T06:31:21.562 INFO:tasks.ceph.osd.6.smithi138.stderr:2024-09-14T06:31:21.560+0000 7f8cd8849640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33620) UID: 0 2024-09-14T06:31:21.596 INFO:teuthology.orchestra.run.smithi094.stderr:nodeep-scrub is unset 2024-09-14T06:31:21.662 INFO:tasks.ceph.osd.6.smithi138.stderr:2024-09-14T06:31:21.661+0000 7f8cd8849640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33620) UID: 0 2024-09-14T06:31:21.707 INFO:teuthology.orchestra.run.smithi138.stdout:{ 2024-09-14T06:31:21.708 INFO:teuthology.orchestra.run.smithi138.stdout: "ops": [], 2024-09-14T06:31:21.708 INFO:teuthology.orchestra.run.smithi138.stdout: "num_ops": 0 2024-09-14T06:31:21.708 INFO:teuthology.orchestra.run.smithi138.stdout:} 2024-09-14T06:31:21.717 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-14T06:31:21.763 INFO:tasks.ceph.osd.3.smithi094.stderr:2024-09-14T06:31:21.762+0000 7feeeafe8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33655) UID: 0 2024-09-14T06:31:21.861 INFO:teuthology.orchestra.run.smithi138.stdout:{ 2024-09-14T06:31:21.861 INFO:teuthology.orchestra.run.smithi138.stdout: "ops": [], 2024-09-14T06:31:21.861 INFO:teuthology.orchestra.run.smithi138.stdout: "complaint_time": 30, 2024-09-14T06:31:21.861 INFO:teuthology.orchestra.run.smithi138.stdout: "num_blocked_ops": 0 2024-09-14T06:31:21.861 INFO:teuthology.orchestra.run.smithi138.stdout:} 2024-09-14T06:31:21.863 INFO:tasks.ceph.osd.4.smithi138.stderr:2024-09-14T06:31:21.862+0000 7f6c653c8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33615) UID: 0 2024-09-14T06:31:21.872 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-14T06:31:21.965 INFO:tasks.ceph.osd.1.smithi094.stderr:2024-09-14T06:31:21.964+0000 7f328e47a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33641) UID: 0 2024-09-14T06:31:22.017 INFO:teuthology.orchestra.run.smithi138.stdout:{ 2024-09-14T06:31:22.017 INFO:teuthology.orchestra.run.smithi138.stdout: "size": 20, 2024-09-14T06:31:22.017 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 600, 2024-09-14T06:31:22.017 INFO:teuthology.orchestra.run.smithi138.stdout: "ops": [ 2024-09-14T06:31:22.017 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:22.017 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:19328 3.3 3:c34e5e51:::rbd_data.112d92027590.0000000000000024:f [read 1926144~1024 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:31:22.017 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:27.059768+0000", 2024-09-14T06:31:22.017 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 114.955278566, 2024-09-14T06:31:22.017 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046369147999999999, 2024-09-14T06:31:22.017 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:31:22.018 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:31:22.018 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:31:22.018 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:31:22.018 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:31:22.018 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:22.018 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 19328 2024-09-14T06:31:22.018 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:22.018 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:31:22.018 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:22.018 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:31:22.018 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.059768+0000", 2024-09-14T06:31:22.018 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:22.018 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:22.019 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:22.019 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:31:22.019 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.059768+0000", 2024-09-14T06:31:22.019 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:22.019 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:22.019 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:22.019 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:31:22.019 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.059874+0000", 2024-09-14T06:31:22.019 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00010551400000000001 2024-09-14T06:31:22.019 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:22.019 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:22.019 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:31:22.019 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.059877+0000", 2024-09-14T06:31:22.020 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.8279999999999999e-06 2024-09-14T06:31:22.020 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:22.020 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:22.020 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:31:22.020 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.059884+0000", 2024-09-14T06:31:22.020 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.6389999999999992e-06 2024-09-14T06:31:22.020 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:22.020 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:22.020 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:31:22.020 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.059890+0000", 2024-09-14T06:31:22.020 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.49e-06 2024-09-14T06:31:22.020 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:22.020 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:22.021 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:31:22.021 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.105804+0000", 2024-09-14T06:31:22.021 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.045914242000000001 2024-09-14T06:31:22.021 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:22.021 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:22.021 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:31:22.021 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.105886+0000", 2024-09-14T06:31:22.021 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.2305999999999994e-05 2024-09-14T06:31:22.021 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:22.021 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:22.021 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:31:22.021 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.106138+0000", 2024-09-14T06:31:22.021 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.000251129 2024-09-14T06:31:22.022 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:22.022 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:31:22.022 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:22.022 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:22.022 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:22.022 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:19329 3.3 3:c34e5e51:::rbd_data.112d92027590.0000000000000024:f [read 1936384~1024 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:31:22.022 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:27.059899+0000", 2024-09-14T06:31:22.022 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 114.95514804299999, 2024-09-14T06:31:22.022 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046572122000000001, 2024-09-14T06:31:22.022 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:31:22.022 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:31:22.022 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:31:22.022 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:31:22.023 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:31:22.023 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:22.023 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 19329 2024-09-14T06:31:22.023 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:22.023 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:31:22.023 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:22.023 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:31:22.023 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.059899+0000", 2024-09-14T06:31:22.023 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:22.023 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:22.023 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:22.023 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:31:22.023 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.059899+0000", 2024-09-14T06:31:22.024 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:22.024 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:22.024 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:22.024 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:31:22.024 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062501+0000", 2024-09-14T06:31:22.024 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.0026016419999999999 2024-09-14T06:31:22.024 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:22.024 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:22.024 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:31:22.024 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062514+0000", 2024-09-14T06:31:22.024 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.2947e-05 2024-09-14T06:31:22.024 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:22.024 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:22.025 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:31:22.025 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062523+0000", 2024-09-14T06:31:22.025 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.5519999999999993e-06 2024-09-14T06:31:22.025 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:22.025 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:22.025 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:31:22.025 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062534+0000", 2024-09-14T06:31:22.025 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.1416e-05 2024-09-14T06:31:22.025 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:22.025 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:22.025 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:31:22.025 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.106179+0000", 2024-09-14T06:31:22.025 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.043644049999999997 2024-09-14T06:31:22.026 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:22.026 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:22.026 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:31:22.026 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.106262+0000", 2024-09-14T06:31:22.026 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.3178999999999998e-05 2024-09-14T06:31:22.026 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:22.026 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:22.026 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:31:22.026 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.106471+0000", 2024-09-14T06:31:22.026 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00020933599999999999 2024-09-14T06:31:22.026 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:22.026 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:31:22.026 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:22.026 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:22.027 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:22.027 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:19351 3.3 3:c34e5e51:::rbd_data.112d92027590.0000000000000024:f [read 1159168~1024 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:31:22.027 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:27.062766+0000", 2024-09-14T06:31:22.027 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 114.95228081800001, 2024-09-14T06:31:22.027 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046099454999999998, 2024-09-14T06:31:22.027 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:31:22.027 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:31:22.027 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:31:22.027 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:31:22.027 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:31:22.027 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:22.027 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 19351 2024-09-14T06:31:22.027 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:22.028 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:31:22.028 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:22.028 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:31:22.028 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062766+0000", 2024-09-14T06:31:22.028 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:22.028 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:22.028 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:22.028 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:31:22.028 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062766+0000", 2024-09-14T06:31:22.028 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:22.028 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:22.028 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:22.028 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:31:22.028 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062766+0000", 2024-09-14T06:31:22.029 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.6e-07 2024-09-14T06:31:22.029 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:22.029 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:22.029 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:31:22.029 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062767+0000", 2024-09-14T06:31:22.029 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.0070000000000001e-06 2024-09-14T06:31:22.029 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:22.029 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:22.029 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:31:22.029 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062785+0000", 2024-09-14T06:31:22.029 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.7884999999999999e-05 2024-09-14T06:31:22.029 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:22.029 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:22.030 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:31:22.030 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062790+0000", 2024-09-14T06:31:22.030 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.0900000000000004e-06 2024-09-14T06:31:22.030 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:22.030 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:22.030 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:31:22.030 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.108735+0000", 2024-09-14T06:31:22.030 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.045944411999999997 2024-09-14T06:31:22.030 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:22.030 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:22.030 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:31:22.030 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.108809+0000", 2024-09-14T06:31:22.030 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.4157000000000002e-05 2024-09-14T06:31:22.030 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:22.031 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:22.031 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:31:22.031 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.108866+0000", 2024-09-14T06:31:22.031 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.6644000000000002e-05 2024-09-14T06:31:22.031 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:22.031 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:31:22.031 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:22.031 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:22.031 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:22.031 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:19352 3.3 3:c34e5e51:::rbd_data.112d92027590.0000000000000024:f [read 1169408~1024 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:31:22.031 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:27.062798+0000", 2024-09-14T06:31:22.031 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 114.952249407, 2024-09-14T06:31:22.031 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046255900000000003, 2024-09-14T06:31:22.032 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:31:22.032 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:31:22.032 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:31:22.032 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:31:22.032 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:31:22.032 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:22.032 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 19352 2024-09-14T06:31:22.032 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:22.032 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:31:22.032 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:22.032 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:31:22.032 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062798+0000", 2024-09-14T06:31:22.032 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:22.033 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:22.033 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:22.033 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:31:22.033 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062798+0000", 2024-09-14T06:31:22.033 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:22.033 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:22.033 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:22.033 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:31:22.033 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062798+0000", 2024-09-14T06:31:22.033 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.37e-07 2024-09-14T06:31:22.033 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:22.033 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:22.033 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:31:22.034 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062800+0000", 2024-09-14T06:31:22.034 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.0269999999999998e-06 2024-09-14T06:31:22.034 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:22.034 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:22.034 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:31:22.034 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062806+0000", 2024-09-14T06:31:22.034 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.6690000000000003e-06 2024-09-14T06:31:22.034 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:22.034 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:22.034 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:31:22.034 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062810+0000", 2024-09-14T06:31:22.034 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.3449999999999999e-06 2024-09-14T06:31:22.034 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:22.034 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:22.035 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:31:22.035 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.108892+0000", 2024-09-14T06:31:22.035 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046081715000000002 2024-09-14T06:31:22.035 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:22.035 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:22.035 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:31:22.035 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.108940+0000", 2024-09-14T06:31:22.035 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.7815999999999998e-05 2024-09-14T06:31:22.035 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:22.035 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:22.035 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:31:22.035 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.109053+0000", 2024-09-14T06:31:22.035 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.000113791 2024-09-14T06:31:22.036 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:22.036 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:31:22.036 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:22.036 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:22.036 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:22.036 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:19353 3.3 3:c34e5e51:::rbd_data.112d92027590.0000000000000024:f [read 1179648~1024 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:31:22.036 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:27.062816+0000", 2024-09-14T06:31:22.036 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 114.952230888, 2024-09-14T06:31:22.036 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046481256999999998, 2024-09-14T06:31:22.036 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:31:22.036 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:31:22.036 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:31:22.036 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:31:22.036 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:31:22.037 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:22.037 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 19353 2024-09-14T06:31:22.037 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:22.037 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:31:22.037 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:22.037 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:31:22.037 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062816+0000", 2024-09-14T06:31:22.037 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:22.037 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:22.037 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:22.037 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:31:22.037 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062816+0000", 2024-09-14T06:31:22.037 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:22.038 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:22.038 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:22.038 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:31:22.038 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062816+0000", 2024-09-14T06:31:22.038 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.2899999999999999e-07 2024-09-14T06:31:22.038 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:22.038 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:22.038 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:31:22.038 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062818+0000", 2024-09-14T06:31:22.038 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.699e-06 2024-09-14T06:31:22.038 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:22.038 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:22.038 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:31:22.038 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062825+0000", 2024-09-14T06:31:22.039 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.878e-06 2024-09-14T06:31:22.039 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:22.039 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:22.039 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:31:22.039 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062830+0000", 2024-09-14T06:31:22.039 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.5410000000000002e-06 2024-09-14T06:31:22.039 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:22.039 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:22.039 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:31:22.039 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.109096+0000", 2024-09-14T06:31:22.039 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046266517 2024-09-14T06:31:22.039 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:22.039 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:22.039 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:31:22.040 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.109190+0000", 2024-09-14T06:31:22.040 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.4000999999999997e-05 2024-09-14T06:31:22.040 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:22.040 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:22.040 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:31:22.040 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.109297+0000", 2024-09-14T06:31:22.040 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.000107292 2024-09-14T06:31:22.040 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:22.040 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:31:22.040 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:22.040 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:22.040 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:22.040 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:19354 3.3 3:c34e5e51:::rbd_data.112d92027590.0000000000000024:f [read 1189888~1024 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:31:22.040 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:27.062842+0000", 2024-09-14T06:31:22.041 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 114.952205433, 2024-09-14T06:31:22.041 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046594257, 2024-09-14T06:31:22.041 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:31:22.041 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:31:22.041 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:31:22.041 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:31:22.041 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:31:22.041 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:22.041 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 19354 2024-09-14T06:31:22.041 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:22.041 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:31:22.041 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:22.041 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:31:22.042 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062842+0000", 2024-09-14T06:31:22.042 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:22.042 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:22.042 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:22.042 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:31:22.042 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062842+0000", 2024-09-14T06:31:22.042 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:22.042 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:22.042 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:22.042 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:31:22.042 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062842+0000", 2024-09-14T06:31:22.042 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.7399999999999999e-07 2024-09-14T06:31:22.042 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:22.043 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:22.043 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:31:22.043 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062859+0000", 2024-09-14T06:31:22.043 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.7010000000000001e-05 2024-09-14T06:31:22.043 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:22.043 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:22.043 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:31:22.043 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062866+0000", 2024-09-14T06:31:22.043 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.9990000000000002e-06 2024-09-14T06:31:22.043 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:22.043 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:22.043 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:31:22.043 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062873+0000", 2024-09-14T06:31:22.043 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.8569999999999997e-06 2024-09-14T06:31:22.044 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:22.044 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:22.044 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:31:22.044 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.109335+0000", 2024-09-14T06:31:22.044 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046461943999999998 2024-09-14T06:31:22.044 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:22.044 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:22.044 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:31:22.044 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.109377+0000", 2024-09-14T06:31:22.044 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.2558999999999997e-05 2024-09-14T06:31:22.044 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:22.044 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:22.044 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:31:22.044 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.109436+0000", 2024-09-14T06:31:22.045 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.8514000000000003e-05 2024-09-14T06:31:22.045 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:22.045 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:31:22.045 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:22.045 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:22.045 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:22.045 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:19355 3.3 3:c34e5e51:::rbd_data.112d92027590.0000000000000024:f [read 1200128~1024 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:31:22.045 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:27.062884+0000", 2024-09-14T06:31:22.045 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 114.95216275999999, 2024-09-14T06:31:22.045 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046681650999999998, 2024-09-14T06:31:22.045 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:31:22.045 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:31:22.045 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:31:22.046 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:31:22.046 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:31:22.046 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:22.046 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 19355 2024-09-14T06:31:22.046 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:22.046 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:31:22.046 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:22.046 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:31:22.046 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062884+0000", 2024-09-14T06:31:22.046 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:22.046 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:22.046 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:22.046 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:31:22.046 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062884+0000", 2024-09-14T06:31:22.047 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:22.047 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:22.047 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:22.047 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:31:22.047 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062885+0000", 2024-09-14T06:31:22.047 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.8000000000000001e-07 2024-09-14T06:31:22.047 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:22.047 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:22.047 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:31:22.047 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062886+0000", 2024-09-14T06:31:22.047 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.8920000000000001e-06 2024-09-14T06:31:22.047 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:22.047 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:22.047 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:31:22.047 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062892+0000", 2024-09-14T06:31:22.048 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.84e-06 2024-09-14T06:31:22.048 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:22.048 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:22.048 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:31:22.048 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062899+0000", 2024-09-14T06:31:22.048 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.5339999999999996e-06 2024-09-14T06:31:22.048 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:22.048 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:22.048 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:31:22.048 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.109466+0000", 2024-09-14T06:31:22.048 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046566919999999998 2024-09-14T06:31:22.048 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:22.048 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:22.049 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:31:22.049 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.109510+0000", 2024-09-14T06:31:22.049 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.4153000000000002e-05 2024-09-14T06:31:22.049 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:22.049 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:22.049 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:31:22.049 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.109566+0000", 2024-09-14T06:31:22.049 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.5931999999999999e-05 2024-09-14T06:31:22.049 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:22.049 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:31:22.049 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:22.049 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:22.049 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:22.049 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:19356 3.3 3:c34e5e51:::rbd_data.112d92027590.0000000000000024:f [read 1210368~1024 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:31:22.049 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:27.062908+0000", 2024-09-14T06:31:22.050 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 114.952139323, 2024-09-14T06:31:22.050 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046807498000000003, 2024-09-14T06:31:22.050 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:31:22.050 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:31:22.050 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:31:22.050 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:31:22.050 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:31:22.050 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:22.050 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 19356 2024-09-14T06:31:22.050 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:22.050 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:31:22.050 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:22.050 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:31:22.050 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062908+0000", 2024-09-14T06:31:22.051 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:22.051 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:22.051 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:22.051 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:31:22.051 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062908+0000", 2024-09-14T06:31:22.051 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:22.051 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:22.051 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:22.051 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:31:22.051 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062908+0000", 2024-09-14T06:31:22.051 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.5699999999999998e-07 2024-09-14T06:31:22.051 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:22.051 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:22.051 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:31:22.052 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062910+0000", 2024-09-14T06:31:22.052 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.7710000000000001e-06 2024-09-14T06:31:22.052 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:22.052 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:22.052 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:31:22.052 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062920+0000", 2024-09-14T06:31:22.052 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.0654e-05 2024-09-14T06:31:22.052 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:22.052 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:22.052 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:31:22.052 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062931+0000", 2024-09-14T06:31:22.052 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.075e-05 2024-09-14T06:31:22.052 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:22.052 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:22.053 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:31:22.053 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.109595+0000", 2024-09-14T06:31:22.053 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046663713000000002 2024-09-14T06:31:22.053 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:22.053 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:22.053 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:31:22.053 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.109632+0000", 2024-09-14T06:31:22.053 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.7505000000000001e-05 2024-09-14T06:31:22.053 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:22.053 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:22.053 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:31:22.053 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.109715+0000", 2024-09-14T06:31:22.053 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.2748000000000001e-05 2024-09-14T06:31:22.054 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:22.054 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:31:22.054 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:22.054 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:22.054 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:22.054 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:19357 3.3 3:c34e5e51:::rbd_data.112d92027590.0000000000000024:f [read 1220608~1024 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:31:22.054 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:27.062939+0000", 2024-09-14T06:31:22.054 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 114.952107875, 2024-09-14T06:31:22.054 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046920508, 2024-09-14T06:31:22.054 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:31:22.054 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:31:22.054 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:31:22.054 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:31:22.055 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:31:22.055 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:22.055 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 19357 2024-09-14T06:31:22.055 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:22.055 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:31:22.055 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:22.055 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:31:22.055 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062939+0000", 2024-09-14T06:31:22.055 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:22.055 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:22.055 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:22.055 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:31:22.055 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062939+0000", 2024-09-14T06:31:22.055 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:22.056 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:22.056 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:22.056 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:31:22.056 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062940+0000", 2024-09-14T06:31:22.056 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.32e-07 2024-09-14T06:31:22.056 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:22.056 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:22.056 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:31:22.056 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062942+0000", 2024-09-14T06:31:22.056 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.1919999999999999e-06 2024-09-14T06:31:22.056 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:22.056 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:22.056 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:31:22.057 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062948+0000", 2024-09-14T06:31:22.057 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.1149999999999999e-06 2024-09-14T06:31:22.057 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:22.057 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:22.057 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:31:22.057 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062960+0000", 2024-09-14T06:31:22.057 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.1895e-05 2024-09-14T06:31:22.057 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:22.057 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:22.057 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:31:22.057 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.109704+0000", 2024-09-14T06:31:22.057 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046744584999999998 2024-09-14T06:31:22.057 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:22.057 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:22.058 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:31:22.058 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.109777+0000", 2024-09-14T06:31:22.058 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.2282999999999999e-05 2024-09-14T06:31:22.058 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:22.058 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:22.058 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:31:22.058 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.109860+0000", 2024-09-14T06:31:22.058 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.3005999999999998e-05 2024-09-14T06:31:22.058 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:22.058 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:31:22.058 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:22.058 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:22.058 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:22.058 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:19358 3.3 3:c34e5e51:::rbd_data.112d92027590.0000000000000024:f [read 1230848~1024 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:31:22.058 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:27.062967+0000", 2024-09-14T06:31:22.059 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 114.952080245, 2024-09-14T06:31:22.059 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047144471, 2024-09-14T06:31:22.059 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:31:22.059 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:31:22.059 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:31:22.059 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:31:22.059 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:31:22.059 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:22.059 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 19358 2024-09-14T06:31:22.059 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:22.059 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:31:22.059 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:22.059 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:31:22.060 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062967+0000", 2024-09-14T06:31:22.060 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:22.060 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:22.060 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:22.060 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:31:22.060 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062967+0000", 2024-09-14T06:31:22.060 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:22.060 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:22.060 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:22.060 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:31:22.060 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062967+0000", 2024-09-14T06:31:22.060 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.89e-07 2024-09-14T06:31:22.060 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:22.061 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:22.061 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:31:22.061 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062969+0000", 2024-09-14T06:31:22.061 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.9929999999999998e-06 2024-09-14T06:31:22.061 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:22.061 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:22.061 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:31:22.061 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062975+0000", 2024-09-14T06:31:22.061 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.9030000000000001e-06 2024-09-14T06:31:22.061 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:22.061 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:22.061 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:31:22.061 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062979+0000", 2024-09-14T06:31:22.061 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.2250000000000002e-06 2024-09-14T06:31:22.062 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:22.062 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:22.062 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:31:22.062 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.109934+0000", 2024-09-14T06:31:22.062 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046954410000000002 2024-09-14T06:31:22.062 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:22.062 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:22.062 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:31:22.062 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.110007+0000", 2024-09-14T06:31:22.062 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.3829000000000001e-05 2024-09-14T06:31:22.062 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:22.062 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:22.062 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:31:22.063 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.110111+0000", 2024-09-14T06:31:22.063 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00010372199999999999 2024-09-14T06:31:22.063 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:22.063 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:31:22.063 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:22.063 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:22.063 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:22.063 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:19359 3.3 3:c34e5e51:::rbd_data.112d92027590.0000000000000024:f [read 1241088~1024 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:31:22.063 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:27.062987+0000", 2024-09-14T06:31:22.063 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 114.952060155, 2024-09-14T06:31:22.063 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047297182, 2024-09-14T06:31:22.063 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:31:22.063 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:31:22.063 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:31:22.064 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:31:22.064 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:31:22.064 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:22.064 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 19359 2024-09-14T06:31:22.064 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:22.064 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:31:22.064 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:22.064 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:31:22.064 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062987+0000", 2024-09-14T06:31:22.064 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:22.064 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:22.064 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:22.064 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:31:22.064 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062987+0000", 2024-09-14T06:31:22.065 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:22.065 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:22.065 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:22.065 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:31:22.065 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062987+0000", 2024-09-14T06:31:22.065 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.2500000000000001e-07 2024-09-14T06:31:22.065 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:22.065 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:22.065 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:31:22.065 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062989+0000", 2024-09-14T06:31:22.065 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.6080000000000001e-06 2024-09-14T06:31:22.065 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:22.065 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:22.065 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:31:22.066 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062994+0000", 2024-09-14T06:31:22.066 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.5520000000000004e-06 2024-09-14T06:31:22.066 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:22.066 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:22.066 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:31:22.066 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062999+0000", 2024-09-14T06:31:22.066 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.3930000000000001e-06 2024-09-14T06:31:22.066 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:22.066 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:22.066 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:31:22.066 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.110142+0000", 2024-09-14T06:31:22.066 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047143154 2024-09-14T06:31:22.066 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:22.067 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:22.067 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:31:22.067 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.110219+0000", 2024-09-14T06:31:22.067 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.6737999999999996e-05 2024-09-14T06:31:22.067 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:22.067 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:22.067 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:31:22.067 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.110284+0000", 2024-09-14T06:31:22.067 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.5412000000000003e-05 2024-09-14T06:31:22.067 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:22.067 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:31:22.067 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:22.067 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:22.067 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:22.068 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:19360 3.3 3:c34e5e51:::rbd_data.112d92027590.0000000000000024:f [read 1251328~1024 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:31:22.068 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:27.063017+0000", 2024-09-14T06:31:22.068 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 114.952029694, 2024-09-14T06:31:22.068 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047453823999999999, 2024-09-14T06:31:22.068 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:31:22.068 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:31:22.068 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:31:22.068 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:31:22.068 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:31:22.068 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:22.068 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 19360 2024-09-14T06:31:22.068 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:22.068 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:31:22.068 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:22.069 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:31:22.069 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.063017+0000", 2024-09-14T06:31:22.069 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:22.069 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:22.069 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:22.069 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:31:22.069 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.063017+0000", 2024-09-14T06:31:22.069 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:22.069 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:22.069 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:22.069 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:31:22.069 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.064977+0000", 2024-09-14T06:31:22.069 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.0019597080000000001 2024-09-14T06:31:22.069 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:22.069 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:22.070 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:31:22.070 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.064987+0000", 2024-09-14T06:31:22.070 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.9620000000000005e-06 2024-09-14T06:31:22.070 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:22.070 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:22.070 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:31:22.070 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065000+0000", 2024-09-14T06:31:22.070 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.3173000000000001e-05 2024-09-14T06:31:22.070 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:22.071 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:22.071 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:31:22.071 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065005+0000", 2024-09-14T06:31:22.071 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.9960000000000001e-06 2024-09-14T06:31:22.071 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:22.071 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:22.071 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:31:22.071 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.110314+0000", 2024-09-14T06:31:22.071 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.04530899 2024-09-14T06:31:22.071 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:22.071 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:22.071 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:31:22.072 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.110398+0000", 2024-09-14T06:31:22.072 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.3539000000000001e-05 2024-09-14T06:31:22.072 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:22.072 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:22.072 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:31:22.072 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.110471+0000", 2024-09-14T06:31:22.072 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.3455999999999996e-05 2024-09-14T06:31:22.072 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:22.072 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:31:22.072 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:22.072 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:22.072 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:22.072 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:19365 3.3 3:c34e5e51:::rbd_data.112d92027590.0000000000000024:f [read 1302528~1024 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:31:22.072 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:27.065058+0000", 2024-09-14T06:31:22.073 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 114.949989024, 2024-09-14T06:31:22.073 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046105526000000001, 2024-09-14T06:31:22.073 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:31:22.073 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:31:22.073 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:31:22.073 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:31:22.073 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:31:22.073 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:22.073 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 19365 2024-09-14T06:31:22.073 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:22.073 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:31:22.073 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:22.073 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:31:22.073 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065058+0000", 2024-09-14T06:31:22.074 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:22.074 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:22.074 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:22.074 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:31:22.074 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065058+0000", 2024-09-14T06:31:22.074 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:22.074 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:22.074 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:22.074 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:31:22.074 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065058+0000", 2024-09-14T06:31:22.074 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.1899999999999999e-07 2024-09-14T06:31:22.074 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:22.074 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:22.075 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:31:22.075 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065059+0000", 2024-09-14T06:31:22.075 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.1299999999999998e-07 2024-09-14T06:31:22.075 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:22.075 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:22.075 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:31:22.075 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065063+0000", 2024-09-14T06:31:22.075 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.1099999999999996e-06 2024-09-14T06:31:22.075 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:22.075 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:22.075 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:31:22.075 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065070+0000", 2024-09-14T06:31:22.075 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.973e-06 2024-09-14T06:31:22.075 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:22.076 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:22.076 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:31:22.076 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.111014+0000", 2024-09-14T06:31:22.076 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.045943373000000003 2024-09-14T06:31:22.076 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:22.076 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:22.076 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:31:22.076 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.111073+0000", 2024-09-14T06:31:22.076 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.8987999999999998e-05 2024-09-14T06:31:22.076 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:22.076 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:22.076 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:31:22.076 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.111163+0000", 2024-09-14T06:31:22.076 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.0950000000000002e-05 2024-09-14T06:31:22.077 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:22.077 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:31:22.077 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:22.077 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:22.077 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:22.077 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:19366 3.3 3:c34e5e51:::rbd_data.112d92027590.0000000000000024:f [read 1312768~1024 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:31:22.077 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:27.065074+0000", 2024-09-14T06:31:22.077 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 114.94997317399999, 2024-09-14T06:31:22.077 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046289858000000003, 2024-09-14T06:31:22.077 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:31:22.077 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:31:22.077 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:31:22.077 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:31:22.077 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:31:22.078 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:22.078 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 19366 2024-09-14T06:31:22.078 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:22.078 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:31:22.078 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:22.078 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:31:22.078 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065074+0000", 2024-09-14T06:31:22.078 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:22.078 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:22.078 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:22.078 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:31:22.078 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065074+0000", 2024-09-14T06:31:22.078 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:22.079 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:22.079 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:22.079 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:31:22.079 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065074+0000", 2024-09-14T06:31:22.079 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.3900000000000001e-07 2024-09-14T06:31:22.079 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:22.079 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:22.079 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:31:22.079 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065078+0000", 2024-09-14T06:31:22.079 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.3229999999999999e-06 2024-09-14T06:31:22.079 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:22.079 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:22.079 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:31:22.079 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065082+0000", 2024-09-14T06:31:22.080 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.8299999999999998e-06 2024-09-14T06:31:22.080 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:22.080 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:22.080 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:31:22.080 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065085+0000", 2024-09-14T06:31:22.080 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.8660000000000002e-06 2024-09-14T06:31:22.080 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:22.080 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:22.080 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:31:22.080 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.111270+0000", 2024-09-14T06:31:22.080 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046185035999999999 2024-09-14T06:31:22.080 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:22.080 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:22.081 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:31:22.081 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.111311+0000", 2024-09-14T06:31:22.081 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.0935e-05 2024-09-14T06:31:22.081 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:22.081 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:22.081 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:31:22.081 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.111364+0000", 2024-09-14T06:31:22.081 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.2629e-05 2024-09-14T06:31:22.081 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:22.081 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:31:22.081 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:22.081 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:22.081 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:22.081 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:19367 3.3 3:c34e5e51:::rbd_data.112d92027590.0000000000000024:f [read 1323008~1024 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:31:22.082 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:27.065115+0000", 2024-09-14T06:31:22.082 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 114.949931613, 2024-09-14T06:31:22.082 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046407003000000002, 2024-09-14T06:31:22.082 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:31:22.082 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:31:22.082 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:31:22.082 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:31:22.082 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:31:22.082 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:22.082 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 19367 2024-09-14T06:31:22.082 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:22.082 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:31:22.082 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:22.082 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:31:22.082 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065115+0000", 2024-09-14T06:31:22.083 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:22.083 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:22.083 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:22.083 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:31:22.083 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065115+0000", 2024-09-14T06:31:22.083 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:22.083 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:22.083 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:22.083 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:31:22.083 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065116+0000", 2024-09-14T06:31:22.083 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.4999999999999998e-07 2024-09-14T06:31:22.083 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:22.083 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:22.083 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:31:22.084 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065117+0000", 2024-09-14T06:31:22.084 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.189e-06 2024-09-14T06:31:22.084 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:22.084 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:22.084 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:31:22.084 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065122+0000", 2024-09-14T06:31:22.084 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.3709999999999999e-06 2024-09-14T06:31:22.084 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:22.084 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:22.084 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:31:22.084 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065125+0000", 2024-09-14T06:31:22.084 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.8940000000000002e-06 2024-09-14T06:31:22.085 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:22.085 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:22.085 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:31:22.085 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.111402+0000", 2024-09-14T06:31:22.085 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046276645999999998 2024-09-14T06:31:22.085 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:22.085 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:22.085 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:31:22.085 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.111445+0000", 2024-09-14T06:31:22.085 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.3288000000000003e-05 2024-09-14T06:31:22.085 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:22.085 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:22.085 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:31:22.085 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.111522+0000", 2024-09-14T06:31:22.086 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.7164999999999997e-05 2024-09-14T06:31:22.086 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:22.086 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:31:22.086 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:22.086 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:22.086 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:22.086 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:19368 3.3 3:c34e5e51:::rbd_data.112d92027590.0000000000000024:f [read 1333248~1024 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:31:22.086 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:27.065130+0000", 2024-09-14T06:31:22.086 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 114.94991732299999, 2024-09-14T06:31:22.086 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046507275000000001, 2024-09-14T06:31:22.086 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:31:22.086 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:31:22.086 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:31:22.086 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:31:22.087 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:31:22.087 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:22.087 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 19368 2024-09-14T06:31:22.087 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:22.087 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:31:22.087 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:22.087 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:31:22.087 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065130+0000", 2024-09-14T06:31:22.087 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:22.087 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:22.087 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:22.087 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:31:22.087 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065130+0000", 2024-09-14T06:31:22.087 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:22.088 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:22.088 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:22.088 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:31:22.088 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065130+0000", 2024-09-14T06:31:22.088 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.0800000000000001e-07 2024-09-14T06:31:22.088 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:22.088 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:22.088 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:31:22.088 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065131+0000", 2024-09-14T06:31:22.088 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.8700000000000004e-07 2024-09-14T06:31:22.088 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:22.088 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:22.089 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:31:22.089 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065134+0000", 2024-09-14T06:31:22.089 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.1039999999999998e-06 2024-09-14T06:31:22.089 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:22.089 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:22.089 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:31:22.089 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065136+0000", 2024-09-14T06:31:22.089 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.3949999999999999e-06 2024-09-14T06:31:22.089 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:22.089 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:22.089 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:31:22.089 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.111541+0000", 2024-09-14T06:31:22.089 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046404206000000003 2024-09-14T06:31:22.090 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:22.090 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:22.090 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:31:22.090 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.111582+0000", 2024-09-14T06:31:22.090 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.1051999999999999e-05 2024-09-14T06:31:22.090 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:22.090 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:22.090 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:31:22.090 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.111637+0000", 2024-09-14T06:31:22.090 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.5322999999999999e-05 2024-09-14T06:31:22.090 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:22.090 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:31:22.090 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:22.090 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:22.091 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:22.091 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:19369 3.3 3:c34e5e51:::rbd_data.112d92027590.0000000000000024:f [read 1343488~1024 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:31:22.091 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:27.065139+0000", 2024-09-14T06:31:22.091 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 114.949907506, 2024-09-14T06:31:22.091 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046682169000000003, 2024-09-14T06:31:22.091 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:31:22.091 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:31:22.091 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:31:22.091 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:31:22.091 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:31:22.091 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:22.091 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 19369 2024-09-14T06:31:22.091 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:22.091 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:31:22.092 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:22.092 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:31:22.092 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065139+0000", 2024-09-14T06:31:22.092 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:22.092 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:22.092 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:22.092 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:31:22.092 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065139+0000", 2024-09-14T06:31:22.092 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:22.092 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:22.092 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:22.092 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:31:22.092 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065140+0000", 2024-09-14T06:31:22.093 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.8299999999999998e-07 2024-09-14T06:31:22.093 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:22.093 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:22.093 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:31:22.093 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065141+0000", 2024-09-14T06:31:22.093 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.3900000000000003e-07 2024-09-14T06:31:22.093 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:22.093 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:22.093 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:31:22.093 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065144+0000", 2024-09-14T06:31:22.093 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.7409999999999998e-06 2024-09-14T06:31:22.093 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:22.093 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:22.093 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:31:22.094 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065147+0000", 2024-09-14T06:31:22.094 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.4150000000000002e-06 2024-09-14T06:31:22.094 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:22.094 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:22.094 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:31:22.094 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.111673+0000", 2024-09-14T06:31:22.094 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046526004000000003 2024-09-14T06:31:22.094 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:22.094 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:22.094 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:31:22.094 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.111749+0000", 2024-09-14T06:31:22.094 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.5741999999999994e-05 2024-09-14T06:31:22.094 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:22.094 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:22.095 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:31:22.095 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.111822+0000", 2024-09-14T06:31:22.095 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.3045000000000003e-05 2024-09-14T06:31:22.095 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:22.095 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:31:22.095 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:22.095 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:22.095 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:22.095 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:19370 3.3 3:c34e5e51:::rbd_data.112d92027590.0000000000000024:f [read 1353728~1024 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:31:22.095 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:27.065150+0000", 2024-09-14T06:31:22.095 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 114.94989677700001, 2024-09-14T06:31:22.095 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046897556999999999, 2024-09-14T06:31:22.095 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:31:22.095 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:31:22.096 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:31:22.096 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:31:22.096 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:31:22.096 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:22.096 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 19370 2024-09-14T06:31:22.096 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:22.096 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:31:22.096 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:22.096 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:31:22.096 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065150+0000", 2024-09-14T06:31:22.096 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:22.096 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:22.096 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:22.096 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:31:22.097 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065150+0000", 2024-09-14T06:31:22.097 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:22.097 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:22.097 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:22.097 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:31:22.097 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065150+0000", 2024-09-14T06:31:22.097 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.0200000000000001e-07 2024-09-14T06:31:22.097 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:22.097 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:22.097 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:31:22.097 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065151+0000", 2024-09-14T06:31:22.097 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.1100000000000004e-07 2024-09-14T06:31:22.097 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:22.098 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:22.098 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:31:22.098 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065154+0000", 2024-09-14T06:31:22.098 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.9639999999999999e-06 2024-09-14T06:31:22.098 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:22.098 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:22.098 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:31:22.098 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065157+0000", 2024-09-14T06:31:22.098 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.4260000000000002e-06 2024-09-14T06:31:22.098 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:22.098 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:22.098 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:31:22.098 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.111861+0000", 2024-09-14T06:31:22.098 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046704268 2024-09-14T06:31:22.099 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:22.099 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:22.099 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:31:22.099 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.111939+0000", 2024-09-14T06:31:22.099 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.8231000000000004e-05 2024-09-14T06:31:22.099 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:22.099 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:22.099 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:31:22.099 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.112048+0000", 2024-09-14T06:31:22.099 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00010855500000000001 2024-09-14T06:31:22.099 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:22.099 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:31:22.099 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:22.099 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:22.100 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:22.100 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:19371 3.3 3:c34e5e51:::rbd_data.112d92027590.0000000000000024:f [read 1363968~1024 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:31:22.100 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:27.065161+0000", 2024-09-14T06:31:22.100 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 114.949886127, 2024-09-14T06:31:22.100 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047021944000000003, 2024-09-14T06:31:22.100 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:31:22.100 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:31:22.100 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:31:22.100 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:31:22.100 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:31:22.100 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:22.100 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 19371 2024-09-14T06:31:22.100 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:22.100 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:31:22.101 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:22.101 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:31:22.101 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065161+0000", 2024-09-14T06:31:22.101 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:22.101 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:22.101 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:22.101 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:31:22.101 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065161+0000", 2024-09-14T06:31:22.101 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:22.101 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:22.101 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:22.101 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:31:22.101 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065161+0000", 2024-09-14T06:31:22.101 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.0200000000000001e-07 2024-09-14T06:31:22.102 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:22.102 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:22.102 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:31:22.102 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065166+0000", 2024-09-14T06:31:22.102 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.9869999999999999e-06 2024-09-14T06:31:22.102 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:22.102 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:22.102 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:31:22.102 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065169+0000", 2024-09-14T06:31:22.102 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.3799999999999998e-06 2024-09-14T06:31:22.102 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:22.102 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:22.102 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:31:22.102 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065172+0000", 2024-09-14T06:31:22.103 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.4830000000000002e-06 2024-09-14T06:31:22.103 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:22.103 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:22.103 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:31:22.103 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.112071+0000", 2024-09-14T06:31:22.103 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046899529000000002 2024-09-14T06:31:22.103 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:22.103 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:22.103 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:31:22.103 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.112128+0000", 2024-09-14T06:31:22.103 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.6341999999999997e-05 2024-09-14T06:31:22.103 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:22.103 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:22.104 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:31:22.104 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.112183+0000", 2024-09-14T06:31:22.104 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.5021000000000001e-05 2024-09-14T06:31:22.104 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:22.104 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:31:22.104 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:22.104 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:22.104 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:22.104 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:19372 3.3 3:c34e5e51:::rbd_data.112d92027590.0000000000000024:f [read 1374208~1024 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:31:22.104 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:27.065176+0000", 2024-09-14T06:31:22.104 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 114.949870977, 2024-09-14T06:31:22.104 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047188889999999997, 2024-09-14T06:31:22.104 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:31:22.105 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:31:22.105 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:31:22.105 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:31:22.105 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:31:22.105 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:22.105 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 19372 2024-09-14T06:31:22.105 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:22.105 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:31:22.105 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:22.105 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:31:22.105 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065176+0000", 2024-09-14T06:31:22.105 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:22.105 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:22.105 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:22.106 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:31:22.106 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065176+0000", 2024-09-14T06:31:22.106 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:22.106 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:22.106 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:22.106 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:31:22.106 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.071117+0000", 2024-09-14T06:31:22.106 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.0059404990000000001 2024-09-14T06:31:22.106 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:22.106 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:22.106 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:31:22.106 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.071120+0000", 2024-09-14T06:31:22.106 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.4340000000000001e-06 2024-09-14T06:31:22.106 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:22.107 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:22.107 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:31:22.107 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.071129+0000", 2024-09-14T06:31:22.107 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.8389999999999996e-06 2024-09-14T06:31:22.107 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:22.107 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:22.107 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:31:22.107 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.071134+0000", 2024-09-14T06:31:22.107 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.412e-06 2024-09-14T06:31:22.107 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:22.107 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:22.107 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:31:22.107 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.112221+0000", 2024-09-14T06:31:22.107 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.041086473999999998 2024-09-14T06:31:22.108 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:22.108 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:22.108 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:31:22.108 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.112299+0000", 2024-09-14T06:31:22.108 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.8746999999999997e-05 2024-09-14T06:31:22.108 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:22.108 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:22.108 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:31:22.108 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.112365+0000", 2024-09-14T06:31:22.108 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.5485000000000001e-05 2024-09-14T06:31:22.108 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:22.108 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:31:22.108 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:22.108 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:22.109 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:31:22.109 INFO:teuthology.orchestra.run.smithi138.stdout:} 2024-09-14T06:31:22.110 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-14T06:31:22.111 INFO:tasks.ceph.osd.3.smithi094.stderr:2024-09-14T06:31:22.109+0000 7feeeafe8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33655) UID: 0 2024-09-14T06:31:22.211 INFO:tasks.ceph.osd.0.smithi094.stderr:2024-09-14T06:31:22.210+0000 7f07c9333640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33636) UID: 0 2024-09-14T06:31:22.214 INFO:teuthology.orchestra.run.smithi138.stdout:{ 2024-09-14T06:31:22.214 INFO:teuthology.orchestra.run.smithi138.stdout: "ops": [], 2024-09-14T06:31:22.214 INFO:teuthology.orchestra.run.smithi138.stdout: "num_ops": 0 2024-09-14T06:31:22.214 INFO:teuthology.orchestra.run.smithi138.stdout:} 2024-09-14T06:31:22.222 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-14T06:31:22.312 INFO:tasks.ceph.osd.4.smithi138.stderr:2024-09-14T06:31:22.311+0000 7f6c653c8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33615) UID: 0 2024-09-14T06:31:22.325 INFO:teuthology.orchestra.run.smithi138.stdout:{ 2024-09-14T06:31:22.325 INFO:teuthology.orchestra.run.smithi138.stdout: "ops": [], 2024-09-14T06:31:22.325 INFO:teuthology.orchestra.run.smithi138.stdout: "complaint_time": 30, 2024-09-14T06:31:22.325 INFO:teuthology.orchestra.run.smithi138.stdout: "num_blocked_ops": 0 2024-09-14T06:31:22.325 INFO:teuthology.orchestra.run.smithi138.stdout:} 2024-09-14T06:31:22.334 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-14T06:31:22.413 INFO:tasks.ceph.osd.1.smithi094.stderr:2024-09-14T06:31:22.412+0000 7f328e47a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33641) UID: 0 2024-09-14T06:31:22.478 INFO:teuthology.orchestra.run.smithi138.stdout:{ 2024-09-14T06:31:22.479 INFO:teuthology.orchestra.run.smithi138.stdout: "size": 20, 2024-09-14T06:31:22.479 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 600, 2024-09-14T06:31:22.479 INFO:teuthology.orchestra.run.smithi138.stdout: "ops": [ 2024-09-14T06:31:22.479 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:22.479 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:11453 3.1d 3:bcad7303:::rbd_data.112d92027590.0000000000000001:f [read 94208~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:31:22.479 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:25.654375+0000", 2024-09-14T06:31:22.479 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 116.822689572, 2024-09-14T06:31:22.479 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052449318000000002, 2024-09-14T06:31:22.479 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:31:22.479 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:31:22.479 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:31:22.479 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:31:22.479 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:31:22.480 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:22.480 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 11453 2024-09-14T06:31:22.480 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:22.480 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:31:22.480 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:22.480 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:31:22.480 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654375+0000", 2024-09-14T06:31:22.480 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:22.480 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:22.480 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:22.480 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:31:22.480 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654375+0000", 2024-09-14T06:31:22.480 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:22.481 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:22.481 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:22.481 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:31:22.481 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654496+0000", 2024-09-14T06:31:22.481 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00012081 2024-09-14T06:31:22.481 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:22.481 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:22.481 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:31:22.481 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654497+0000", 2024-09-14T06:31:22.481 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.8789999999999999e-06 2024-09-14T06:31:22.484 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:22.484 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:22.484 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:31:22.484 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654501+0000", 2024-09-14T06:31:22.484 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.8999999999999999e-06 2024-09-14T06:31:22.484 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:22.484 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:22.484 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:31:22.484 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654505+0000", 2024-09-14T06:31:22.484 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.2229999999999999e-06 2024-09-14T06:31:22.484 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:22.485 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:22.485 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:31:22.485 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.705917+0000", 2024-09-14T06:31:22.485 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.051412501999999999 2024-09-14T06:31:22.485 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:22.485 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:22.485 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:31:22.485 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.705999+0000", 2024-09-14T06:31:22.485 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.1551999999999994e-05 2024-09-14T06:31:22.485 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:22.485 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:22.485 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:31:22.485 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.706824+0000", 2024-09-14T06:31:22.485 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00082545199999999996 2024-09-14T06:31:22.486 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:22.486 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:31:22.486 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:22.486 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:22.486 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:22.486 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:11454 3.1d 3:bcad7303:::rbd_data.112d92027590.0000000000000001:f [read 104448~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:31:22.486 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:25.654510+0000", 2024-09-14T06:31:22.486 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 116.822554782, 2024-09-14T06:31:22.486 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052610713000000003, 2024-09-14T06:31:22.486 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:31:22.486 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:31:22.486 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:31:22.486 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:31:22.486 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:31:22.486 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:22.487 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 11454 2024-09-14T06:31:22.487 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:22.487 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:31:22.487 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:22.487 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:31:22.487 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654510+0000", 2024-09-14T06:31:22.487 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:22.487 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:22.487 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:22.487 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:31:22.487 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654510+0000", 2024-09-14T06:31:22.487 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:22.487 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:22.487 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:22.488 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:31:22.488 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654825+0000", 2024-09-14T06:31:22.488 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00031522400000000002 2024-09-14T06:31:22.488 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:22.488 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:22.488 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:31:22.488 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654827+0000", 2024-09-14T06:31:22.488 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.9709999999999998e-06 2024-09-14T06:31:22.488 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:22.488 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:22.488 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:31:22.488 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654832+0000", 2024-09-14T06:31:22.488 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.4099999999999999e-06 2024-09-14T06:31:22.489 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:22.489 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:22.489 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:31:22.489 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654835+0000", 2024-09-14T06:31:22.489 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.14e-06 2024-09-14T06:31:22.489 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:22.489 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:22.489 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:31:22.489 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.706851+0000", 2024-09-14T06:31:22.489 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052015761000000001 2024-09-14T06:31:22.489 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:22.489 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:22.489 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:31:22.489 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.706934+0000", 2024-09-14T06:31:22.490 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.2484999999999994e-05 2024-09-14T06:31:22.490 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:22.490 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:22.490 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:31:22.490 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.707120+0000", 2024-09-14T06:31:22.490 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00018672200000000001 2024-09-14T06:31:22.490 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:22.490 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:31:22.490 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:22.490 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:22.490 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:22.490 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:11455 3.1d 3:bcad7303:::rbd_data.112d92027590.0000000000000001:f [read 114688~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:31:22.490 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:25.654841+0000", 2024-09-14T06:31:22.491 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 116.82222345300001, 2024-09-14T06:31:22.491 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052579898, 2024-09-14T06:31:22.491 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:31:22.491 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:31:22.491 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:31:22.491 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:31:22.491 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:31:22.491 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:22.491 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 11455 2024-09-14T06:31:22.491 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:22.491 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:31:22.491 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:22.491 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:31:22.491 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654841+0000", 2024-09-14T06:31:22.492 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:22.492 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:22.492 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:22.492 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:31:22.492 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654841+0000", 2024-09-14T06:31:22.492 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:22.492 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:22.492 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:22.492 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:31:22.492 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654841+0000", 2024-09-14T06:31:22.492 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.7300000000000002e-07 2024-09-14T06:31:22.492 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:22.492 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:22.492 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:31:22.493 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654845+0000", 2024-09-14T06:31:22.493 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.0520000000000003e-06 2024-09-14T06:31:22.493 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:22.493 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:22.493 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:31:22.493 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654849+0000", 2024-09-14T06:31:22.493 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.0459999999999999e-06 2024-09-14T06:31:22.493 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:22.493 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:22.493 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:31:22.493 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654852+0000", 2024-09-14T06:31:22.493 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.9830000000000001e-06 2024-09-14T06:31:22.493 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:22.493 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:22.494 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:31:22.494 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.707159+0000", 2024-09-14T06:31:22.494 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052306826000000001 2024-09-14T06:31:22.494 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:22.494 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:22.494 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:31:22.494 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.707242+0000", 2024-09-14T06:31:22.494 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.2595000000000005e-05 2024-09-14T06:31:22.494 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:22.494 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:22.494 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:31:22.494 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.707421+0000", 2024-09-14T06:31:22.494 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.000179123 2024-09-14T06:31:22.494 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:22.494 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:31:22.495 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:22.495 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:22.495 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:22.495 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:11456 3.1d 3:bcad7303:::rbd_data.112d92027590.0000000000000001:f [read 124928~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:31:22.495 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:25.654856+0000", 2024-09-14T06:31:22.495 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 116.82220860699999, 2024-09-14T06:31:22.495 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052801580000000001, 2024-09-14T06:31:22.495 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:31:22.495 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:31:22.495 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:31:22.495 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:31:22.495 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:31:22.495 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:22.496 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 11456 2024-09-14T06:31:22.496 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:22.496 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:31:22.496 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:22.496 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:31:22.496 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654856+0000", 2024-09-14T06:31:22.496 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:22.496 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:22.496 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:22.496 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:31:22.496 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654856+0000", 2024-09-14T06:31:22.496 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:22.496 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:22.497 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:22.497 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:31:22.497 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654856+0000", 2024-09-14T06:31:22.497 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.29e-07 2024-09-14T06:31:22.497 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:22.497 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:22.497 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:31:22.497 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654857+0000", 2024-09-14T06:31:22.497 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.187e-06 2024-09-14T06:31:22.497 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:22.497 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:22.497 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:31:22.497 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654861+0000", 2024-09-14T06:31:22.497 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.3850000000000002e-06 2024-09-14T06:31:22.498 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:22.498 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:22.498 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:31:22.498 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654863+0000", 2024-09-14T06:31:22.498 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.9500000000000001e-06 2024-09-14T06:31:22.498 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:22.498 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:22.498 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:31:22.498 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.707456+0000", 2024-09-14T06:31:22.498 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052592434 2024-09-14T06:31:22.498 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:22.498 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:22.498 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:31:22.498 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.707516+0000", 2024-09-14T06:31:22.499 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.0167999999999997e-05 2024-09-14T06:31:22.499 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:22.499 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:22.499 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:31:22.499 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.707657+0000", 2024-09-14T06:31:22.499 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00014122700000000001 2024-09-14T06:31:22.499 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:22.499 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:31:22.499 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:22.499 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:22.499 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:22.499 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:11457 3.1d 3:bcad7303:::rbd_data.112d92027590.0000000000000001:f [read 135168~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:31:22.499 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:25.654867+0000", 2024-09-14T06:31:22.499 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 116.822197722, 2024-09-14T06:31:22.500 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052921902, 2024-09-14T06:31:22.500 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:31:22.500 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:31:22.500 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:31:22.500 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:31:22.500 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:31:22.500 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:22.500 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 11457 2024-09-14T06:31:22.500 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:22.500 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:31:22.500 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:22.500 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:31:22.500 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654867+0000", 2024-09-14T06:31:22.501 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:22.501 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:22.501 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:22.501 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:31:22.501 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654867+0000", 2024-09-14T06:31:22.501 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:22.501 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:22.501 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:22.501 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:31:22.501 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654972+0000", 2024-09-14T06:31:22.501 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00010584 2024-09-14T06:31:22.501 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:22.501 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:22.501 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:31:22.502 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654975+0000", 2024-09-14T06:31:22.502 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.1169999999999998e-06 2024-09-14T06:31:22.502 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:22.502 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:22.502 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:31:22.502 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654979+0000", 2024-09-14T06:31:22.502 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.1960000000000001e-06 2024-09-14T06:31:22.502 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:22.502 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:22.502 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:31:22.502 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654982+0000", 2024-09-14T06:31:22.502 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.2990000000000001e-06 2024-09-14T06:31:22.502 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:22.502 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:22.502 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:31:22.503 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.707684+0000", 2024-09-14T06:31:22.503 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052701979000000003 2024-09-14T06:31:22.503 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:22.503 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:22.503 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:31:22.503 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.707728+0000", 2024-09-14T06:31:22.503 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.3529999999999998e-05 2024-09-14T06:31:22.503 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:22.503 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:22.503 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:31:22.503 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.707789+0000", 2024-09-14T06:31:22.503 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.0940999999999998e-05 2024-09-14T06:31:22.503 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:22.503 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:31:22.504 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:22.504 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:22.504 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:22.504 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:11458 3.1d 3:bcad7303:::rbd_data.112d92027590.0000000000000001:f [read 145408~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:31:22.504 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:25.654987+0000", 2024-09-14T06:31:22.504 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 116.82207732400001, 2024-09-14T06:31:22.504 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.05291655, 2024-09-14T06:31:22.504 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:31:22.504 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:31:22.504 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:31:22.504 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:31:22.504 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:31:22.504 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:22.505 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 11458 2024-09-14T06:31:22.505 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:22.505 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:31:22.505 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:22.505 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:31:22.505 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654987+0000", 2024-09-14T06:31:22.505 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:22.505 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:22.505 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:22.505 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:31:22.505 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654987+0000", 2024-09-14T06:31:22.505 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:22.505 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:22.505 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:22.506 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:31:22.506 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655077+0000", 2024-09-14T06:31:22.506 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.0253000000000005e-05 2024-09-14T06:31:22.506 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:22.506 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:22.506 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:31:22.506 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655079+0000", 2024-09-14T06:31:22.506 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.9989999999999998e-06 2024-09-14T06:31:22.506 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:22.506 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:22.506 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:31:22.506 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655083+0000", 2024-09-14T06:31:22.506 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.9280000000000003e-06 2024-09-14T06:31:22.506 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:22.506 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:22.507 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:31:22.507 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655086+0000", 2024-09-14T06:31:22.507 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.056e-06 2024-09-14T06:31:22.507 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:22.507 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:22.507 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:31:22.507 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.707805+0000", 2024-09-14T06:31:22.507 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052718627999999997 2024-09-14T06:31:22.507 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:22.507 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:22.507 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:31:22.507 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.707848+0000", 2024-09-14T06:31:22.507 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.2728999999999997e-05 2024-09-14T06:31:22.507 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:22.508 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:22.508 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:31:22.508 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.707904+0000", 2024-09-14T06:31:22.508 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.5957e-05 2024-09-14T06:31:22.508 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:22.508 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:31:22.508 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:22.508 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:22.508 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:22.508 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:11459 3.1d 3:bcad7303:::rbd_data.112d92027590.0000000000000001:f [read 155648~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:31:22.508 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:25.655091+0000", 2024-09-14T06:31:22.508 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 116.821973316, 2024-09-14T06:31:22.508 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052944242000000002, 2024-09-14T06:31:22.508 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:31:22.509 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:31:22.509 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:31:22.509 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:31:22.509 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:31:22.509 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:22.509 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 11459 2024-09-14T06:31:22.509 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:22.509 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:31:22.509 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:22.509 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:31:22.509 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655091+0000", 2024-09-14T06:31:22.509 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:22.509 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:22.510 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:22.510 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:31:22.510 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655091+0000", 2024-09-14T06:31:22.510 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:22.510 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:22.510 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:22.510 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:31:22.510 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655237+0000", 2024-09-14T06:31:22.510 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00014570100000000001 2024-09-14T06:31:22.510 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:22.510 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:22.510 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:31:22.510 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655239+0000", 2024-09-14T06:31:22.510 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.8670000000000001e-06 2024-09-14T06:31:22.511 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:22.511 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:22.511 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:31:22.511 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655243+0000", 2024-09-14T06:31:22.511 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.9829999999999998e-06 2024-09-14T06:31:22.511 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:22.511 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:22.511 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:31:22.511 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655246+0000", 2024-09-14T06:31:22.511 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.1540000000000002e-06 2024-09-14T06:31:22.511 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:22.511 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:22.512 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:31:22.512 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.707934+0000", 2024-09-14T06:31:22.512 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052688286000000001 2024-09-14T06:31:22.512 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:22.512 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:22.512 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:31:22.512 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.707977+0000", 2024-09-14T06:31:22.512 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.3291999999999998e-05 2024-09-14T06:31:22.512 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:22.512 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:22.512 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:31:22.512 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.708035+0000", 2024-09-14T06:31:22.512 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.7958999999999999e-05 2024-09-14T06:31:22.512 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:22.513 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:31:22.513 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:22.513 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:22.513 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:22.513 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:11460 3.1d 3:bcad7303:::rbd_data.112d92027590.0000000000000001:f [read 165888~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:31:22.513 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:25.655250+0000", 2024-09-14T06:31:22.513 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 116.821814019, 2024-09-14T06:31:22.513 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052910698999999999, 2024-09-14T06:31:22.513 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:31:22.513 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:31:22.513 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:31:22.513 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:31:22.513 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:31:22.513 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:22.514 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 11460 2024-09-14T06:31:22.514 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:22.514 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:31:22.514 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:22.514 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:31:22.514 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655250+0000", 2024-09-14T06:31:22.514 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:22.514 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:22.514 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:22.514 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:31:22.514 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655250+0000", 2024-09-14T06:31:22.514 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:22.514 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:22.514 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:22.514 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:31:22.515 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655356+0000", 2024-09-14T06:31:22.515 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00010535900000000001 2024-09-14T06:31:22.515 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:22.515 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:22.515 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:31:22.515 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655357+0000", 2024-09-14T06:31:22.515 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.8080000000000001e-06 2024-09-14T06:31:22.515 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:22.515 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:22.515 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:31:22.515 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655361+0000", 2024-09-14T06:31:22.515 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.861e-06 2024-09-14T06:31:22.515 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:22.515 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:22.516 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:31:22.516 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655364+0000", 2024-09-14T06:31:22.516 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.1089999999999998e-06 2024-09-14T06:31:22.516 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:22.516 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:22.516 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:31:22.516 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.708065+0000", 2024-09-14T06:31:22.516 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052700595000000003 2024-09-14T06:31:22.516 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:22.516 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:22.516 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:31:22.516 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.708108+0000", 2024-09-14T06:31:22.516 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.2484999999999997e-05 2024-09-14T06:31:22.516 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:22.516 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:22.517 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:31:22.517 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.708161+0000", 2024-09-14T06:31:22.517 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.3482000000000001e-05 2024-09-14T06:31:22.517 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:22.517 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:31:22.517 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:22.517 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:22.517 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:22.517 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:11461 3.1d 3:bcad7303:::rbd_data.112d92027590.0000000000000001:f [read 176128~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:31:22.517 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:25.655370+0000", 2024-09-14T06:31:22.517 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 116.821694737, 2024-09-14T06:31:22.517 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052915141999999998, 2024-09-14T06:31:22.517 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:31:22.517 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:31:22.518 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:31:22.518 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:31:22.518 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:31:22.518 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:22.518 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 11461 2024-09-14T06:31:22.518 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:22.518 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:31:22.518 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:22.518 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:31:22.518 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655370+0000", 2024-09-14T06:31:22.518 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:22.518 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:22.518 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:22.518 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:31:22.519 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655370+0000", 2024-09-14T06:31:22.519 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:22.519 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:22.519 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:22.519 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:31:22.519 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655394+0000", 2024-09-14T06:31:22.519 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.4553999999999999e-05 2024-09-14T06:31:22.519 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:22.519 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:22.519 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:31:22.519 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655396+0000", 2024-09-14T06:31:22.519 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.8950000000000001e-06 2024-09-14T06:31:22.519 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:22.519 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:22.519 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:31:22.520 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655401+0000", 2024-09-14T06:31:22.520 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.0089999999999998e-06 2024-09-14T06:31:22.520 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:22.520 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:22.520 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:31:22.520 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655404+0000", 2024-09-14T06:31:22.520 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.3400000000000002e-06 2024-09-14T06:31:22.520 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:22.520 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:22.520 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:31:22.520 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.708186+0000", 2024-09-14T06:31:22.520 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052781726000000001 2024-09-14T06:31:22.520 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:22.520 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:22.520 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:31:22.521 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.708229+0000", 2024-09-14T06:31:22.521 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.2670999999999999e-05 2024-09-14T06:31:22.521 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:22.521 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:22.521 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:31:22.521 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.708285+0000", 2024-09-14T06:31:22.521 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.5946999999999998e-05 2024-09-14T06:31:22.521 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:22.521 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:31:22.521 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:22.521 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:22.521 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:22.521 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:11462 3.1d 3:bcad7303:::rbd_data.112d92027590.0000000000000001:f [read 186368~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:31:22.521 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:25.655410+0000", 2024-09-14T06:31:22.522 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 116.821654259, 2024-09-14T06:31:22.522 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052995258000000003, 2024-09-14T06:31:22.522 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:31:22.522 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:31:22.522 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:31:22.522 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:31:22.522 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:31:22.522 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:22.522 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 11462 2024-09-14T06:31:22.522 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:22.522 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:31:22.522 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:22.522 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:31:22.522 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655410+0000", 2024-09-14T06:31:22.523 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:22.523 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:22.523 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:22.523 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:31:22.523 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655410+0000", 2024-09-14T06:31:22.523 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:22.523 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:22.523 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:22.523 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:31:22.523 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655625+0000", 2024-09-14T06:31:22.523 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00021442600000000001 2024-09-14T06:31:22.523 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:22.523 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:22.523 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:31:22.524 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655626+0000", 2024-09-14T06:31:22.524 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.815e-06 2024-09-14T06:31:22.524 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:22.524 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:22.524 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:31:22.524 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655631+0000", 2024-09-14T06:31:22.524 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.651e-06 2024-09-14T06:31:22.524 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:22.524 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:22.524 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:31:22.524 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655634+0000", 2024-09-14T06:31:22.524 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.2449999999999998e-06 2024-09-14T06:31:22.524 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:22.524 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:22.525 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:31:22.525 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.708312+0000", 2024-09-14T06:31:22.525 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052677412 2024-09-14T06:31:22.525 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:22.525 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:22.525 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:31:22.525 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.708352+0000", 2024-09-14T06:31:22.525 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.0729999999999998e-05 2024-09-14T06:31:22.525 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:22.525 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:22.525 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:31:22.525 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.708405+0000", 2024-09-14T06:31:22.525 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.2979000000000002e-05 2024-09-14T06:31:22.526 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:22.526 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:31:22.526 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:22.526 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:22.526 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:22.526 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:11463 3.1d 3:bcad7303:::rbd_data.112d92027590.0000000000000001:f [read 196608~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:31:22.526 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:25.655639+0000", 2024-09-14T06:31:22.526 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 116.821425113, 2024-09-14T06:31:22.526 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052888859000000003, 2024-09-14T06:31:22.526 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:31:22.526 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:31:22.526 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:31:22.526 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:31:22.526 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:31:22.527 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:22.527 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 11463 2024-09-14T06:31:22.527 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:22.527 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:31:22.527 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:22.527 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:31:22.527 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655639+0000", 2024-09-14T06:31:22.527 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:22.527 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:22.527 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:22.527 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:31:22.527 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655639+0000", 2024-09-14T06:31:22.527 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:22.527 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:22.528 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:22.528 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:31:22.528 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655640+0000", 2024-09-14T06:31:22.528 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.08e-07 2024-09-14T06:31:22.528 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:22.528 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:22.528 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:31:22.528 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655641+0000", 2024-09-14T06:31:22.528 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.156e-06 2024-09-14T06:31:22.528 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:22.528 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:22.528 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:31:22.528 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655648+0000", 2024-09-14T06:31:22.528 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.8460000000000002e-06 2024-09-14T06:31:22.529 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:22.529 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:22.529 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:31:22.529 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655651+0000", 2024-09-14T06:31:22.529 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.0079999999999998e-06 2024-09-14T06:31:22.529 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:22.529 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:22.529 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:31:22.529 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.708429+0000", 2024-09-14T06:31:22.529 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052778459 2024-09-14T06:31:22.529 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:22.529 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:22.529 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:31:22.530 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.708473+0000", 2024-09-14T06:31:22.530 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.3782999999999997e-05 2024-09-14T06:31:22.530 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:22.530 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:22.530 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:31:22.530 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.708528+0000", 2024-09-14T06:31:22.530 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.5198999999999998e-05 2024-09-14T06:31:22.530 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:22.530 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:31:22.530 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:22.530 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:22.530 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:22.530 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:11464 3.1d 3:bcad7303:::rbd_data.112d92027590.0000000000000001:f [read 206848~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:31:22.530 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:25.655654+0000", 2024-09-14T06:31:22.531 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 116.82141024400001, 2024-09-14T06:31:22.531 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052981845999999999, 2024-09-14T06:31:22.531 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:31:22.531 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:31:22.531 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:31:22.531 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:31:22.531 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:31:22.531 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:22.531 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 11464 2024-09-14T06:31:22.531 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:22.531 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:31:22.531 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:22.531 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:31:22.531 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655654+0000", 2024-09-14T06:31:22.532 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:22.532 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:22.532 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:22.532 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:31:22.532 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655654+0000", 2024-09-14T06:31:22.532 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:22.532 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:22.532 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:22.532 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:31:22.532 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655946+0000", 2024-09-14T06:31:22.532 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00029148200000000003 2024-09-14T06:31:22.532 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:22.532 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:22.532 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:31:22.533 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655948+0000", 2024-09-14T06:31:22.533 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.973e-06 2024-09-14T06:31:22.533 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:22.533 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:22.533 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:31:22.533 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655953+0000", 2024-09-14T06:31:22.533 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.1240000000000004e-06 2024-09-14T06:31:22.533 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:22.533 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:22.533 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:31:22.533 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655956+0000", 2024-09-14T06:31:22.533 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.315e-06 2024-09-14T06:31:22.533 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:22.534 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:22.534 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:31:22.534 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.708543+0000", 2024-09-14T06:31:22.534 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052587416999999997 2024-09-14T06:31:22.534 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:22.534 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:22.534 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:31:22.534 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.708584+0000", 2024-09-14T06:31:22.534 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.1004000000000002e-05 2024-09-14T06:31:22.534 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:22.534 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:22.534 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:31:22.534 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.708636+0000", 2024-09-14T06:31:22.534 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.1530999999999998e-05 2024-09-14T06:31:22.534 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:22.535 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:31:22.535 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:22.535 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:22.535 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:22.535 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:11465 3.1d 3:bcad7303:::rbd_data.112d92027590.0000000000000001:f [read 217088~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:31:22.535 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:25.655961+0000", 2024-09-14T06:31:22.535 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 116.821103092, 2024-09-14T06:31:22.535 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052798682999999999, 2024-09-14T06:31:22.535 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:31:22.535 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:31:22.535 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:31:22.535 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:31:22.535 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:31:22.535 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:22.535 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 11465 2024-09-14T06:31:22.536 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:22.536 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:31:22.536 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:22.536 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:31:22.536 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655961+0000", 2024-09-14T06:31:22.536 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:22.536 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:22.536 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:22.536 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:31:22.536 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655961+0000", 2024-09-14T06:31:22.536 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:22.536 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:22.536 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:22.537 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:31:22.537 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.656114+0000", 2024-09-14T06:31:22.537 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.000152745 2024-09-14T06:31:22.537 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:22.537 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:22.537 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:31:22.537 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.656116+0000", 2024-09-14T06:31:22.537 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.804e-06 2024-09-14T06:31:22.537 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:22.537 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:22.537 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:31:22.537 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.656120+0000", 2024-09-14T06:31:22.538 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.4700000000000004e-06 2024-09-14T06:31:22.538 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:22.538 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:22.538 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:31:22.538 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.656123+0000", 2024-09-14T06:31:22.538 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.1269999999999999e-06 2024-09-14T06:31:22.538 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:22.538 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:22.538 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:31:22.538 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.708660+0000", 2024-09-14T06:31:22.538 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052536919000000001 2024-09-14T06:31:22.538 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:22.538 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:22.538 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:31:22.539 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.708703+0000", 2024-09-14T06:31:22.539 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.3077000000000001e-05 2024-09-14T06:31:22.539 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:22.539 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:22.539 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:31:22.539 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.708760+0000", 2024-09-14T06:31:22.539 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.6541e-05 2024-09-14T06:31:22.539 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:22.539 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:31:22.539 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:22.539 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:22.539 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:22.539 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:11466 3.1d 3:bcad7303:::rbd_data.112d92027590.0000000000000001:f [read 227328~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:31:22.539 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:25.656129+0000", 2024-09-14T06:31:22.540 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 116.82093570399999, 2024-09-14T06:31:22.540 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052750987999999999, 2024-09-14T06:31:22.540 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:31:22.540 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:31:22.540 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:31:22.540 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:31:22.540 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:31:22.540 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:22.540 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 11466 2024-09-14T06:31:22.540 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:22.540 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:31:22.540 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:22.540 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:31:22.541 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.656129+0000", 2024-09-14T06:31:22.541 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:22.541 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:22.541 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:22.541 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:31:22.541 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.656129+0000", 2024-09-14T06:31:22.541 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:22.541 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:22.541 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:22.541 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:31:22.541 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.656347+0000", 2024-09-14T06:31:22.541 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.000218054 2024-09-14T06:31:22.541 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:22.542 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:22.542 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:31:22.542 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.656349+0000", 2024-09-14T06:31:22.542 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.9199999999999998e-06 2024-09-14T06:31:22.542 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:22.542 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:22.542 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:31:22.542 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.656353+0000", 2024-09-14T06:31:22.542 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.9210000000000002e-06 2024-09-14T06:31:22.542 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:22.542 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:22.542 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:31:22.542 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.656356+0000", 2024-09-14T06:31:22.542 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.7670000000000001e-06 2024-09-14T06:31:22.542 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:22.543 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:22.543 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:31:22.543 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.708787+0000", 2024-09-14T06:31:22.543 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052430840999999999 2024-09-14T06:31:22.543 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:22.543 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:22.543 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:31:22.543 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.708827+0000", 2024-09-14T06:31:22.543 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.9538e-05 2024-09-14T06:31:22.543 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:22.543 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:22.543 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:31:22.543 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.708880+0000", 2024-09-14T06:31:22.543 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.2947e-05 2024-09-14T06:31:22.544 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:22.544 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:31:22.544 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:22.544 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:22.544 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:22.544 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:11467 3.1d 3:bcad7303:::rbd_data.112d92027590.0000000000000001:f [read 237568~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:31:22.544 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:25.656361+0000", 2024-09-14T06:31:22.544 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 116.820703056, 2024-09-14T06:31:22.544 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052643898000000001, 2024-09-14T06:31:22.544 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:31:22.544 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:31:22.544 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:31:22.544 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:31:22.544 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:31:22.545 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:22.545 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 11467 2024-09-14T06:31:22.545 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:22.545 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:31:22.545 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:22.545 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:31:22.545 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.656361+0000", 2024-09-14T06:31:22.545 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:22.545 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:22.545 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:22.545 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:31:22.545 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.656361+0000", 2024-09-14T06:31:22.546 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:22.546 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:22.546 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:22.546 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:31:22.546 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.656855+0000", 2024-09-14T06:31:22.546 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00049372000000000001 2024-09-14T06:31:22.546 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:22.546 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:22.546 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:31:22.546 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.656859+0000", 2024-09-14T06:31:22.546 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.1300000000000003e-06 2024-09-14T06:31:22.546 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:22.546 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:22.547 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:31:22.547 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.656863+0000", 2024-09-14T06:31:22.547 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.2629999999999997e-06 2024-09-14T06:31:22.547 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:22.547 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:22.547 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:31:22.547 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.656867+0000", 2024-09-14T06:31:22.547 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.3220000000000001e-06 2024-09-14T06:31:22.547 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:22.547 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:22.547 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:31:22.547 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.708903+0000", 2024-09-14T06:31:22.547 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052035855999999998 2024-09-14T06:31:22.547 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:22.548 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:22.548 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:31:22.548 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.708949+0000", 2024-09-14T06:31:22.548 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.6276999999999998e-05 2024-09-14T06:31:22.548 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:22.548 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:22.548 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:31:22.548 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.709005+0000", 2024-09-14T06:31:22.548 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.6329999999999998e-05 2024-09-14T06:31:22.548 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:22.548 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:31:22.548 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:22.548 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:22.548 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:22.549 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:11479 3.1d 3:bcad7303:::rbd_data.112d92027590.0000000000000001:f [read 360448~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:31:22.549 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:25.658388+0000", 2024-09-14T06:31:22.549 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 116.818676223, 2024-09-14T06:31:22.549 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052408920999999997, 2024-09-14T06:31:22.549 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:31:22.549 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:31:22.549 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:31:22.549 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:31:22.549 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:31:22.549 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:22.549 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 11479 2024-09-14T06:31:22.549 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:22.549 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:31:22.549 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:22.550 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:31:22.550 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.658388+0000", 2024-09-14T06:31:22.550 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:22.550 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:22.550 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:22.550 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:31:22.550 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.658388+0000", 2024-09-14T06:31:22.550 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:22.550 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:22.550 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:22.550 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:31:22.550 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.658534+0000", 2024-09-14T06:31:22.550 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.000145613 2024-09-14T06:31:22.550 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:22.551 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:22.551 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:31:22.551 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.658540+0000", 2024-09-14T06:31:22.551 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.6680000000000004e-06 2024-09-14T06:31:22.551 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:22.551 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:22.551 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:31:22.551 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.658548+0000", 2024-09-14T06:31:22.551 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.7980000000000006e-06 2024-09-14T06:31:22.551 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:22.551 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:22.551 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:31:22.551 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.658554+0000", 2024-09-14T06:31:22.551 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.6869999999999999e-06 2024-09-14T06:31:22.552 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:22.552 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:22.552 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:31:22.552 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.710624+0000", 2024-09-14T06:31:22.552 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052070567999999998 2024-09-14T06:31:22.552 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:22.552 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:22.552 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:31:22.552 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.710666+0000", 2024-09-14T06:31:22.552 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.1467000000000001e-05 2024-09-14T06:31:22.552 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:22.552 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:22.552 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:31:22.552 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.710797+0000", 2024-09-14T06:31:22.553 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00013112 2024-09-14T06:31:22.553 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:22.553 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:31:22.553 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:22.553 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:22.553 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:22.553 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:11480 3.1d 3:bcad7303:::rbd_data.112d92027590.0000000000000001:f [read 370688~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:31:22.553 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:25.658563+0000", 2024-09-14T06:31:22.553 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 116.81850129199999, 2024-09-14T06:31:22.553 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052429164, 2024-09-14T06:31:22.553 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:31:22.553 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:31:22.553 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:31:22.553 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:31:22.554 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:31:22.554 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:22.554 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 11480 2024-09-14T06:31:22.554 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:22.554 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:31:22.554 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:22.554 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:31:22.554 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.658563+0000", 2024-09-14T06:31:22.554 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:22.554 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:22.554 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:22.554 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:31:22.554 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.658563+0000", 2024-09-14T06:31:22.554 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:22.555 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:22.555 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:22.555 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:31:22.555 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.658658+0000", 2024-09-14T06:31:22.555 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.5346999999999994e-05 2024-09-14T06:31:22.555 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:22.555 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:22.555 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:31:22.555 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.658662+0000", 2024-09-14T06:31:22.555 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.14e-06 2024-09-14T06:31:22.555 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:22.555 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:22.555 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:31:22.555 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.658669+0000", 2024-09-14T06:31:22.556 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.7379999999999994e-06 2024-09-14T06:31:22.556 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:22.556 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:22.556 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:31:22.556 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.658675+0000", 2024-09-14T06:31:22.556 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.5430000000000001e-06 2024-09-14T06:31:22.556 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:22.556 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:22.556 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:31:22.556 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.710824+0000", 2024-09-14T06:31:22.556 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052148847999999998 2024-09-14T06:31:22.556 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:22.557 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:22.557 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:31:22.557 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.710864+0000", 2024-09-14T06:31:22.557 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.9925999999999997e-05 2024-09-14T06:31:22.557 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:22.557 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:22.557 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:31:22.557 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.710992+0000", 2024-09-14T06:31:22.557 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.000128622 2024-09-14T06:31:22.557 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:22.557 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:31:22.557 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:22.557 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:22.557 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:22.558 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:11481 3.1d 3:bcad7303:::rbd_data.112d92027590.0000000000000001:f [read 380928~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:31:22.558 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:25.658684+0000", 2024-09-14T06:31:22.558 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 116.818380222, 2024-09-14T06:31:22.558 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052508565, 2024-09-14T06:31:22.558 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:31:22.558 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:31:22.558 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:31:22.558 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:31:22.558 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:31:22.558 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:22.558 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 11481 2024-09-14T06:31:22.558 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:22.558 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:31:22.559 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:22.559 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:31:22.559 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.658684+0000", 2024-09-14T06:31:22.559 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:22.559 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:22.559 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:22.559 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:31:22.559 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.658684+0000", 2024-09-14T06:31:22.559 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:22.559 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:22.559 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:22.559 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:31:22.559 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.658715+0000", 2024-09-14T06:31:22.559 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.0703000000000001e-05 2024-09-14T06:31:22.560 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:22.560 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:22.560 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:31:22.560 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.658718+0000", 2024-09-14T06:31:22.560 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.0709999999999999e-06 2024-09-14T06:31:22.560 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:22.560 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:22.560 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:31:22.560 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.658725+0000", 2024-09-14T06:31:22.560 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.2339999999999997e-06 2024-09-14T06:31:22.560 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:22.560 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:22.560 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:31:22.561 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.658731+0000", 2024-09-14T06:31:22.561 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.6450000000000002e-06 2024-09-14T06:31:22.561 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:22.561 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:22.561 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:31:22.561 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.711020+0000", 2024-09-14T06:31:22.561 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052288886 2024-09-14T06:31:22.561 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:22.561 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:22.561 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:31:22.561 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.711061+0000", 2024-09-14T06:31:22.561 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.1084000000000002e-05 2024-09-14T06:31:22.561 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:22.562 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:22.562 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:31:22.562 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.711193+0000", 2024-09-14T06:31:22.562 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00013194199999999999 2024-09-14T06:31:22.562 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:22.562 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:31:22.562 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:22.562 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:22.562 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:22.562 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:11482 3.1d 3:bcad7303:::rbd_data.112d92027590.0000000000000001:f [read 391168~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:31:22.562 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:25.658740+0000", 2024-09-14T06:31:22.562 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 116.818324757, 2024-09-14T06:31:22.562 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052637015000000002, 2024-09-14T06:31:22.562 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:31:22.563 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:31:22.563 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:31:22.563 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:31:22.563 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:31:22.563 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:22.563 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 11482 2024-09-14T06:31:22.563 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:22.563 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:31:22.563 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:22.563 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:31:22.563 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.658740+0000", 2024-09-14T06:31:22.563 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:22.563 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:22.564 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:22.564 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:31:22.564 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.658740+0000", 2024-09-14T06:31:22.564 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:22.564 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:22.564 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:22.564 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:31:22.564 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.658880+0000", 2024-09-14T06:31:22.564 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00014036100000000001 2024-09-14T06:31:22.564 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:22.564 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:22.564 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:31:22.564 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.658883+0000", 2024-09-14T06:31:22.565 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.3969999999999998e-06 2024-09-14T06:31:22.565 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:22.565 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:22.565 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:31:22.565 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.658890+0000", 2024-09-14T06:31:22.565 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.1500000000000002e-06 2024-09-14T06:31:22.565 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:22.565 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:22.565 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:31:22.565 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.658897+0000", 2024-09-14T06:31:22.565 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.1260000000000003e-06 2024-09-14T06:31:22.565 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:22.565 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:22.565 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:31:22.566 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.711207+0000", 2024-09-14T06:31:22.566 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052310777000000003 2024-09-14T06:31:22.566 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:22.566 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:22.566 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:31:22.566 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.711249+0000", 2024-09-14T06:31:22.566 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.1786999999999998e-05 2024-09-14T06:31:22.566 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:22.566 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:22.566 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:31:22.566 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.711377+0000", 2024-09-14T06:31:22.566 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.000127417 2024-09-14T06:31:22.566 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:22.566 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:31:22.567 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:22.567 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:22.567 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:22.567 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:11483 3.1d 3:bcad7303:::rbd_data.112d92027590.0000000000000001:f [read 401408~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:31:22.567 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:25.658906+0000", 2024-09-14T06:31:22.567 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 116.818157973, 2024-09-14T06:31:22.567 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052595109000000001, 2024-09-14T06:31:22.567 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:31:22.567 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:31:22.567 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:31:22.567 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:31:22.567 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:31:22.567 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:22.567 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 11483 2024-09-14T06:31:22.568 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:22.568 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:31:22.568 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:22.568 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:31:22.568 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.658906+0000", 2024-09-14T06:31:22.568 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:22.568 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:22.568 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:22.568 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:31:22.568 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.658906+0000", 2024-09-14T06:31:22.568 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:22.568 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:22.568 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:22.569 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:31:22.569 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.664985+0000", 2024-09-14T06:31:22.569 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.0060787250000000001 2024-09-14T06:31:22.569 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:22.569 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:22.569 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:31:22.569 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.664991+0000", 2024-09-14T06:31:22.569 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.7030000000000003e-06 2024-09-14T06:31:22.569 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:22.569 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:22.569 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:31:22.569 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.664999+0000", 2024-09-14T06:31:22.569 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.4430000000000003e-06 2024-09-14T06:31:22.570 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:22.570 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:22.570 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:31:22.570 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.665006+0000", 2024-09-14T06:31:22.570 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.0840000000000003e-06 2024-09-14T06:31:22.570 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:22.570 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:22.570 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:31:22.570 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.711402+0000", 2024-09-14T06:31:22.570 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046395714999999997 2024-09-14T06:31:22.570 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:22.570 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:22.570 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:31:22.571 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.711443+0000", 2024-09-14T06:31:22.571 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.0911999999999997e-05 2024-09-14T06:31:22.571 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:22.571 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:22.571 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:31:22.571 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.711501+0000", 2024-09-14T06:31:22.571 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.8526999999999997e-05 2024-09-14T06:31:22.571 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:22.571 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:31:22.571 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:22.571 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:22.571 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:31:22.572 INFO:teuthology.orchestra.run.smithi138.stdout:} 2024-09-14T06:31:22.573 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-14T06:31:22.574 INFO:tasks.ceph.osd.2.smithi094.stderr:2024-09-14T06:31:22.572+0000 7faf3269b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33649) UID: 0 2024-09-14T06:31:22.600 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-14T06:31:22.674 INFO:tasks.ceph.osd.1.smithi094.stderr:2024-09-14T06:31:22.673+0000 7f328e47a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33641) UID: 0 2024-09-14T06:31:22.680 INFO:teuthology.orchestra.run.smithi138.stdout:{ 2024-09-14T06:31:22.680 INFO:teuthology.orchestra.run.smithi138.stdout: "ops": [], 2024-09-14T06:31:22.680 INFO:teuthology.orchestra.run.smithi138.stdout: "num_ops": 0 2024-09-14T06:31:22.680 INFO:teuthology.orchestra.run.smithi138.stdout:} 2024-09-14T06:31:22.689 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-14T06:31:22.713 INFO:teuthology.orchestra.run.smithi094.stdout:checking clone #35, image image_client.0-clone35 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent36 2024-09-14T06:31:22.736 INFO:teuthology.orchestra.run.smithi094.stdout:osd.0: {} 2024-09-14T06:31:22.737 INFO:teuthology.orchestra.run.smithi094.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-14T06:31:22.754 INFO:teuthology.orchestra.run.smithi094.stdout:osd.1: {} 2024-09-14T06:31:22.755 INFO:teuthology.orchestra.run.smithi094.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-14T06:31:22.775 INFO:tasks.ceph.osd.1.smithi094.stderr:2024-09-14T06:31:22.774+0000 7f328e47a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33641) UID: 0 2024-09-14T06:31:22.776 INFO:teuthology.orchestra.run.smithi094.stdout:osd.2: {} 2024-09-14T06:31:22.776 INFO:teuthology.orchestra.run.smithi094.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-14T06:31:22.799 INFO:teuthology.orchestra.run.smithi094.stdout:osd.3: {} 2024-09-14T06:31:22.800 INFO:teuthology.orchestra.run.smithi094.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-14T06:31:22.803 INFO:teuthology.orchestra.run.smithi138.stdout:{ 2024-09-14T06:31:22.803 INFO:teuthology.orchestra.run.smithi138.stdout: "ops": [], 2024-09-14T06:31:22.803 INFO:teuthology.orchestra.run.smithi138.stdout: "complaint_time": 30, 2024-09-14T06:31:22.803 INFO:teuthology.orchestra.run.smithi138.stdout: "num_blocked_ops": 0 2024-09-14T06:31:22.804 INFO:teuthology.orchestra.run.smithi138.stdout:} 2024-09-14T06:31:22.812 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-14T06:31:22.817 INFO:teuthology.orchestra.run.smithi094.stdout:osd.4: {} 2024-09-14T06:31:22.817 INFO:teuthology.orchestra.run.smithi094.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-14T06:31:22.834 INFO:teuthology.orchestra.run.smithi094.stdout:osd.5: {} 2024-09-14T06:31:22.835 INFO:teuthology.orchestra.run.smithi094.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-14T06:31:22.851 INFO:teuthology.orchestra.run.smithi094.stdout:osd.6: {} 2024-09-14T06:31:22.852 INFO:teuthology.orchestra.run.smithi094.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-14T06:31:22.868 INFO:teuthology.orchestra.run.smithi094.stdout:osd.7: {} 2024-09-14T06:31:22.869 INFO:teuthology.orchestra.run.smithi094.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-14T06:31:22.876 INFO:tasks.ceph.osd.7.smithi138.stderr:2024-09-14T06:31:22.875+0000 7f3c6a21d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33623) UID: 0 2024-09-14T06:31:22.915 INFO:teuthology.orchestra.run.smithi138.stdout:ERROR: (22) Invalid argument 2024-09-14T06:31:22.915 INFO:teuthology.orchestra.run.smithi138.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:31:22.924 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-14T06:31:22.987 INFO:teuthology.orchestra.run.smithi094.stdout:4683 clone 38 order 24 su 524288 sc 10 2024-09-14T06:31:23.008 INFO:tasks.ceph.osd.7.smithi138.stderr:2024-09-14T06:31:23.007+0000 7f3c6a21d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33623) UID: 0 2024-09-14T06:31:23.067 INFO:teuthology.orchestra.run.smithi138.stdout:ERROR: (22) Invalid argument 2024-09-14T06:31:23.068 INFO:teuthology.orchestra.run.smithi138.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:31:23.077 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-14T06:31:23.078 INFO:tasks.ceph.osd.0.smithi094.stderr:2024-09-14T06:31:23.077+0000 7f07c9333640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33636) UID: 0 2024-09-14T06:31:23.179 INFO:tasks.ceph.osd.5.smithi138.stderr:2024-09-14T06:31:23.178+0000 7fe13746f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33618) UID: 0 2024-09-14T06:31:23.221 INFO:teuthology.orchestra.run.smithi138.stdout:ERROR: (22) Invalid argument 2024-09-14T06:31:23.221 INFO:teuthology.orchestra.run.smithi138.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:31:23.230 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-14T06:31:23.279 INFO:tasks.ceph.osd.2.smithi094.stderr:2024-09-14T06:31:23.279+0000 7faf3269b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33649) UID: 0 2024-09-14T06:31:23.374 INFO:teuthology.orchestra.run.smithi138.stdout:ERROR: (22) Invalid argument 2024-09-14T06:31:23.374 INFO:teuthology.orchestra.run.smithi138.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:31:23.380 INFO:tasks.ceph.osd.0.smithi094.stderr:2024-09-14T06:31:23.379+0000 7f07c9333640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33636) UID: 0 2024-09-14T06:31:23.382 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-14T06:31:23.481 INFO:tasks.ceph.osd.4.smithi138.stderr:2024-09-14T06:31:23.480+0000 7f6c653c8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33615) UID: 0 2024-09-14T06:31:23.497 INFO:teuthology.orchestra.run.smithi094.stdout:ERROR: (22) Invalid argument 2024-09-14T06:31:23.497 INFO:teuthology.orchestra.run.smithi094.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:31:23.506 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-14T06:31:23.582 INFO:tasks.ceph.osd.4.smithi138.stderr:2024-09-14T06:31:23.582+0000 7f6c653c8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33615) UID: 0 2024-09-14T06:31:23.613 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-14T06:31:23.614 INFO:teuthology.orchestra.run.smithi094.stdout:ERROR: (22) Invalid argument 2024-09-14T06:31:23.614 INFO:teuthology.orchestra.run.smithi094.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:31:23.623 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-14T06:31:23.684 INFO:tasks.ceph.osd.5.smithi138.stderr:2024-09-14T06:31:23.683+0000 7fe13746f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33618) UID: 0 2024-09-14T06:31:23.772 INFO:teuthology.orchestra.run.smithi094.stdout:ERROR: (22) Invalid argument 2024-09-14T06:31:23.772 INFO:teuthology.orchestra.run.smithi094.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:31:23.781 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-14T06:31:23.784 INFO:tasks.ceph.osd.0.smithi094.stderr:2024-09-14T06:31:23.783+0000 7f07c9333640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33636) UID: 0 2024-09-14T06:31:23.877 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-14T06:31:23.877 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-14T06:31:23.877 INFO:tasks.thrashosds.thrasher:thrash_pg_upmap_items 2024-09-14T06:31:23.877 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump -f json-pretty 2024-09-14T06:31:23.886 INFO:teuthology.orchestra.run.smithi094.stdout:ERROR: (22) Invalid argument 2024-09-14T06:31:23.887 INFO:teuthology.orchestra.run.smithi094.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:31:23.887 INFO:tasks.ceph.osd.0.smithi094.stderr:2024-09-14T06:31:23.886+0000 7f07c9333640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33636) UID: 0 2024-09-14T06:31:23.896 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-14T06:31:23.986 INFO:tasks.ceph.osd.0.smithi094.stderr:2024-09-14T06:31:23.985+0000 7f07c9333640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33636) UID: 0 2024-09-14T06:31:24.041 INFO:teuthology.orchestra.run.smithi094.stdout:ERROR: (22) Invalid argument 2024-09-14T06:31:24.041 INFO:teuthology.orchestra.run.smithi094.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:31:24.050 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-14T06:31:24.088 INFO:tasks.ceph.osd.7.smithi138.stderr:2024-09-14T06:31:24.086+0000 7f3c6a21d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33623) UID: 0 2024-09-14T06:31:24.181 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-14T06:31:24.181 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-14T06:31:24.181 INFO:teuthology.orchestra.run.smithi094.stdout: "epoch": 159, 2024-09-14T06:31:24.181 INFO:teuthology.orchestra.run.smithi094.stdout: "fsid": "200ad3da-bd63-4e04-af74-efb5ca0af75e", 2024-09-14T06:31:24.181 INFO:teuthology.orchestra.run.smithi094.stdout: "created": "2024-09-14T06:28:30.782046+0000", 2024-09-14T06:31:24.181 INFO:teuthology.orchestra.run.smithi094.stdout: "modified": "2024-09-14T06:31:23.597633+0000", 2024-09-14T06:31:24.181 INFO:teuthology.orchestra.run.smithi094.stdout: "last_up_change": "2024-09-14T06:28:42.827239+0000", 2024-09-14T06:31:24.181 INFO:teuthology.orchestra.run.smithi094.stdout: "last_in_change": "2024-09-14T06:28:33.724176+0000", 2024-09-14T06:31:24.181 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2024-09-14T06:31:24.181 INFO:teuthology.orchestra.run.smithi094.stdout: "flags_num": 5799936, 2024-09-14T06:31:24.182 INFO:teuthology.orchestra.run.smithi094.stdout: "flags_set": [ 2024-09-14T06:31:24.182 INFO:teuthology.orchestra.run.smithi094.stdout: "pglog_hardlimit", 2024-09-14T06:31:24.182 INFO:teuthology.orchestra.run.smithi094.stdout: "purged_snapdirs", 2024-09-14T06:31:24.182 INFO:teuthology.orchestra.run.smithi094.stdout: "recovery_deletes", 2024-09-14T06:31:24.182 INFO:teuthology.orchestra.run.smithi094.stdout: "sortbitwise" 2024-09-14T06:31:24.182 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-09-14T06:31:24.182 INFO:teuthology.orchestra.run.smithi094.stdout: "crush_version": 5, 2024-09-14T06:31:24.182 INFO:teuthology.orchestra.run.smithi094.stdout: "full_ratio": 0.94999998807907104, 2024-09-14T06:31:24.182 INFO:teuthology.orchestra.run.smithi094.stdout: "backfillfull_ratio": 0.89999997615814209, 2024-09-14T06:31:24.182 INFO:teuthology.orchestra.run.smithi094.stdout: "nearfull_ratio": 0.85000002384185791, 2024-09-14T06:31:24.182 INFO:teuthology.orchestra.run.smithi094.stdout: "cluster_snapshot": "", 2024-09-14T06:31:24.182 INFO:teuthology.orchestra.run.smithi094.stdout: "pool_max": 3, 2024-09-14T06:31:24.182 INFO:teuthology.orchestra.run.smithi094.stdout: "max_osd": 8, 2024-09-14T06:31:24.182 INFO:teuthology.orchestra.run.smithi094.stdout: "require_min_compat_client": "luminous", 2024-09-14T06:31:24.183 INFO:teuthology.orchestra.run.smithi094.stdout: "min_compat_client": "luminous", 2024-09-14T06:31:24.183 INFO:teuthology.orchestra.run.smithi094.stdout: "require_osd_release": "squid", 2024-09-14T06:31:24.183 INFO:teuthology.orchestra.run.smithi094.stdout: "allow_crimson": false, 2024-09-14T06:31:24.183 INFO:teuthology.orchestra.run.smithi094.stdout: "pools": [ 2024-09-14T06:31:24.183 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:24.183 INFO:teuthology.orchestra.run.smithi094.stdout: "pool": 1, 2024-09-14T06:31:24.183 INFO:teuthology.orchestra.run.smithi094.stdout: "pool_name": ".mgr", 2024-09-14T06:31:24.183 INFO:teuthology.orchestra.run.smithi094.stdout: "create_time": "2024-09-14T06:28:43.295990+0000", 2024-09-14T06:31:24.183 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-14T06:31:24.183 INFO:teuthology.orchestra.run.smithi094.stdout: "flags_names": "hashpspool", 2024-09-14T06:31:24.183 INFO:teuthology.orchestra.run.smithi094.stdout: "type": 1, 2024-09-14T06:31:24.183 INFO:teuthology.orchestra.run.smithi094.stdout: "size": 2, 2024-09-14T06:31:24.183 INFO:teuthology.orchestra.run.smithi094.stdout: "min_size": 1, 2024-09-14T06:31:24.183 INFO:teuthology.orchestra.run.smithi094.stdout: "crush_rule": 0, 2024-09-14T06:31:24.184 INFO:teuthology.orchestra.run.smithi094.stdout: "peering_crush_bucket_count": 0, 2024-09-14T06:31:24.184 INFO:teuthology.orchestra.run.smithi094.stdout: "peering_crush_bucket_target": 0, 2024-09-14T06:31:24.184 INFO:teuthology.orchestra.run.smithi094.stdout: "peering_crush_bucket_barrier": 0, 2024-09-14T06:31:24.184 INFO:teuthology.orchestra.run.smithi094.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-14T06:31:24.184 INFO:teuthology.orchestra.run.smithi094.stdout: "is_stretch_pool": false, 2024-09-14T06:31:24.184 INFO:teuthology.orchestra.run.smithi094.stdout: "object_hash": 2, 2024-09-14T06:31:24.184 INFO:teuthology.orchestra.run.smithi094.stdout: "pg_autoscale_mode": "off", 2024-09-14T06:31:24.184 INFO:teuthology.orchestra.run.smithi094.stdout: "pg_num": 1, 2024-09-14T06:31:24.184 INFO:teuthology.orchestra.run.smithi094.stdout: "pg_placement_num": 1, 2024-09-14T06:31:24.184 INFO:teuthology.orchestra.run.smithi094.stdout: "pg_placement_num_target": 1, 2024-09-14T06:31:24.184 INFO:teuthology.orchestra.run.smithi094.stdout: "pg_num_target": 1, 2024-09-14T06:31:24.184 INFO:teuthology.orchestra.run.smithi094.stdout: "pg_num_pending": 1, 2024-09-14T06:31:24.184 INFO:teuthology.orchestra.run.smithi094.stdout: "last_pg_merge_meta": { 2024-09-14T06:31:24.185 INFO:teuthology.orchestra.run.smithi094.stdout: "source_pgid": "0.0", 2024-09-14T06:31:24.185 INFO:teuthology.orchestra.run.smithi094.stdout: "ready_epoch": 0, 2024-09-14T06:31:24.185 INFO:teuthology.orchestra.run.smithi094.stdout: "last_epoch_started": 0, 2024-09-14T06:31:24.185 INFO:teuthology.orchestra.run.smithi094.stdout: "last_epoch_clean": 0, 2024-09-14T06:31:24.185 INFO:teuthology.orchestra.run.smithi094.stdout: "source_version": "0'0", 2024-09-14T06:31:24.185 INFO:teuthology.orchestra.run.smithi094.stdout: "target_version": "0'0" 2024-09-14T06:31:24.185 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:24.185 INFO:teuthology.orchestra.run.smithi094.stdout: "last_change": "18", 2024-09-14T06:31:24.185 INFO:teuthology.orchestra.run.smithi094.stdout: "last_force_op_resend": "0", 2024-09-14T06:31:24.185 INFO:teuthology.orchestra.run.smithi094.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-14T06:31:24.185 INFO:teuthology.orchestra.run.smithi094.stdout: "last_force_op_resend_preluminous": "0", 2024-09-14T06:31:24.185 INFO:teuthology.orchestra.run.smithi094.stdout: "auid": 0, 2024-09-14T06:31:24.185 INFO:teuthology.orchestra.run.smithi094.stdout: "snap_mode": "selfmanaged", 2024-09-14T06:31:24.185 INFO:teuthology.orchestra.run.smithi094.stdout: "snap_seq": 0, 2024-09-14T06:31:24.186 INFO:teuthology.orchestra.run.smithi094.stdout: "snap_epoch": 0, 2024-09-14T06:31:24.186 INFO:teuthology.orchestra.run.smithi094.stdout: "pool_snaps": [], 2024-09-14T06:31:24.186 INFO:teuthology.orchestra.run.smithi094.stdout: "removed_snaps": "[]", 2024-09-14T06:31:24.186 INFO:teuthology.orchestra.run.smithi094.stdout: "quota_max_bytes": 0, 2024-09-14T06:31:24.186 INFO:teuthology.orchestra.run.smithi094.stdout: "quota_max_objects": 0, 2024-09-14T06:31:24.186 INFO:teuthology.orchestra.run.smithi094.stdout: "tiers": [], 2024-09-14T06:31:24.186 INFO:teuthology.orchestra.run.smithi094.stdout: "tier_of": -1, 2024-09-14T06:31:24.186 INFO:teuthology.orchestra.run.smithi094.stdout: "read_tier": -1, 2024-09-14T06:31:24.186 INFO:teuthology.orchestra.run.smithi094.stdout: "write_tier": -1, 2024-09-14T06:31:24.186 INFO:teuthology.orchestra.run.smithi094.stdout: "cache_mode": "none", 2024-09-14T06:31:24.186 INFO:teuthology.orchestra.run.smithi094.stdout: "target_max_bytes": 0, 2024-09-14T06:31:24.186 INFO:teuthology.orchestra.run.smithi094.stdout: "target_max_objects": 0, 2024-09-14T06:31:24.186 INFO:teuthology.orchestra.run.smithi094.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-14T06:31:24.186 INFO:teuthology.orchestra.run.smithi094.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-14T06:31:24.187 INFO:teuthology.orchestra.run.smithi094.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-14T06:31:24.187 INFO:teuthology.orchestra.run.smithi094.stdout: "cache_min_flush_age": 0, 2024-09-14T06:31:24.187 INFO:teuthology.orchestra.run.smithi094.stdout: "cache_min_evict_age": 0, 2024-09-14T06:31:24.187 INFO:teuthology.orchestra.run.smithi094.stdout: "erasure_code_profile": "", 2024-09-14T06:31:24.187 INFO:teuthology.orchestra.run.smithi094.stdout: "hit_set_params": { 2024-09-14T06:31:24.187 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "none" 2024-09-14T06:31:24.187 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:24.187 INFO:teuthology.orchestra.run.smithi094.stdout: "hit_set_period": 0, 2024-09-14T06:31:24.187 INFO:teuthology.orchestra.run.smithi094.stdout: "hit_set_count": 0, 2024-09-14T06:31:24.187 INFO:teuthology.orchestra.run.smithi094.stdout: "use_gmt_hitset": true, 2024-09-14T06:31:24.187 INFO:teuthology.orchestra.run.smithi094.stdout: "min_read_recency_for_promote": 0, 2024-09-14T06:31:24.187 INFO:teuthology.orchestra.run.smithi094.stdout: "min_write_recency_for_promote": 0, 2024-09-14T06:31:24.187 INFO:teuthology.orchestra.run.smithi094.stdout: "hit_set_grade_decay_rate": 0, 2024-09-14T06:31:24.187 INFO:teuthology.orchestra.run.smithi094.stdout: "hit_set_search_last_n": 0, 2024-09-14T06:31:24.188 INFO:teuthology.orchestra.run.smithi094.stdout: "grade_table": [], 2024-09-14T06:31:24.188 INFO:teuthology.orchestra.run.smithi094.stdout: "stripe_width": 0, 2024-09-14T06:31:24.188 INFO:teuthology.orchestra.run.smithi094.stdout: "expected_num_objects": 0, 2024-09-14T06:31:24.188 INFO:teuthology.orchestra.run.smithi094.stdout: "fast_read": false, 2024-09-14T06:31:24.188 INFO:teuthology.orchestra.run.smithi094.stdout: "options": { 2024-09-14T06:31:24.188 INFO:teuthology.orchestra.run.smithi094.stdout: "pg_num_max": 32, 2024-09-14T06:31:24.188 INFO:teuthology.orchestra.run.smithi094.stdout: "pg_num_min": 1 2024-09-14T06:31:24.188 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:24.188 INFO:teuthology.orchestra.run.smithi094.stdout: "application_metadata": { 2024-09-14T06:31:24.188 INFO:teuthology.orchestra.run.smithi094.stdout: "mgr": {} 2024-09-14T06:31:24.188 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:24.188 INFO:teuthology.orchestra.run.smithi094.stdout: "read_balance": { 2024-09-14T06:31:24.188 INFO:teuthology.orchestra.run.smithi094.stdout: "score_type": "Fair distribution", 2024-09-14T06:31:24.189 INFO:teuthology.orchestra.run.smithi094.stdout: "score_acting": 8, 2024-09-14T06:31:24.189 INFO:teuthology.orchestra.run.smithi094.stdout: "score_stable": 8, 2024-09-14T06:31:24.189 INFO:teuthology.orchestra.run.smithi094.stdout: "optimal_score": 0.25, 2024-09-14T06:31:24.189 INFO:teuthology.orchestra.run.smithi094.stdout: "raw_score_acting": 2, 2024-09-14T06:31:24.189 INFO:teuthology.orchestra.run.smithi094.stdout: "raw_score_stable": 2, 2024-09-14T06:31:24.189 INFO:teuthology.orchestra.run.smithi094.stdout: "primary_affinity_weighted": 0.98013687133789062, 2024-09-14T06:31:24.189 INFO:teuthology.orchestra.run.smithi094.stdout: "average_primary_affinity": 0.98000001907348633, 2024-09-14T06:31:24.189 INFO:teuthology.orchestra.run.smithi094.stdout: "average_primary_affinity_weighted": 0.98000001907348633 2024-09-14T06:31:24.189 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:24.189 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:24.189 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:24.189 INFO:teuthology.orchestra.run.smithi094.stdout: "pool": 2, 2024-09-14T06:31:24.189 INFO:teuthology.orchestra.run.smithi094.stdout: "pool_name": "rbd", 2024-09-14T06:31:24.190 INFO:teuthology.orchestra.run.smithi094.stdout: "create_time": "2024-09-14T06:28:49.451714+0000", 2024-09-14T06:31:24.190 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 8193, 2024-09-14T06:31:24.190 INFO:teuthology.orchestra.run.smithi094.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-09-14T06:31:24.190 INFO:teuthology.orchestra.run.smithi094.stdout: "type": 1, 2024-09-14T06:31:24.190 INFO:teuthology.orchestra.run.smithi094.stdout: "size": 2, 2024-09-14T06:31:24.190 INFO:teuthology.orchestra.run.smithi094.stdout: "min_size": 1, 2024-09-14T06:31:24.190 INFO:teuthology.orchestra.run.smithi094.stdout: "crush_rule": 0, 2024-09-14T06:31:24.190 INFO:teuthology.orchestra.run.smithi094.stdout: "peering_crush_bucket_count": 0, 2024-09-14T06:31:24.190 INFO:teuthology.orchestra.run.smithi094.stdout: "peering_crush_bucket_target": 0, 2024-09-14T06:31:24.190 INFO:teuthology.orchestra.run.smithi094.stdout: "peering_crush_bucket_barrier": 0, 2024-09-14T06:31:24.190 INFO:teuthology.orchestra.run.smithi094.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-14T06:31:24.190 INFO:teuthology.orchestra.run.smithi094.stdout: "is_stretch_pool": false, 2024-09-14T06:31:24.190 INFO:teuthology.orchestra.run.smithi094.stdout: "object_hash": 2, 2024-09-14T06:31:24.190 INFO:teuthology.orchestra.run.smithi094.stdout: "pg_autoscale_mode": "off", 2024-09-14T06:31:24.191 INFO:teuthology.orchestra.run.smithi094.stdout: "pg_num": 8, 2024-09-14T06:31:24.191 INFO:teuthology.orchestra.run.smithi094.stdout: "pg_placement_num": 8, 2024-09-14T06:31:24.191 INFO:teuthology.orchestra.run.smithi094.stdout: "pg_placement_num_target": 8, 2024-09-14T06:31:24.191 INFO:teuthology.orchestra.run.smithi094.stdout: "pg_num_target": 8, 2024-09-14T06:31:24.191 INFO:teuthology.orchestra.run.smithi094.stdout: "pg_num_pending": 8, 2024-09-14T06:31:24.191 INFO:teuthology.orchestra.run.smithi094.stdout: "last_pg_merge_meta": { 2024-09-14T06:31:24.191 INFO:teuthology.orchestra.run.smithi094.stdout: "source_pgid": "0.0", 2024-09-14T06:31:24.191 INFO:teuthology.orchestra.run.smithi094.stdout: "ready_epoch": 0, 2024-09-14T06:31:24.191 INFO:teuthology.orchestra.run.smithi094.stdout: "last_epoch_started": 0, 2024-09-14T06:31:24.191 INFO:teuthology.orchestra.run.smithi094.stdout: "last_epoch_clean": 0, 2024-09-14T06:31:24.191 INFO:teuthology.orchestra.run.smithi094.stdout: "source_version": "0'0", 2024-09-14T06:31:24.191 INFO:teuthology.orchestra.run.smithi094.stdout: "target_version": "0'0" 2024-09-14T06:31:24.191 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:24.191 INFO:teuthology.orchestra.run.smithi094.stdout: "last_change": "22", 2024-09-14T06:31:24.191 INFO:teuthology.orchestra.run.smithi094.stdout: "last_force_op_resend": "0", 2024-09-14T06:31:24.192 INFO:teuthology.orchestra.run.smithi094.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-14T06:31:24.192 INFO:teuthology.orchestra.run.smithi094.stdout: "last_force_op_resend_preluminous": "0", 2024-09-14T06:31:24.192 INFO:teuthology.orchestra.run.smithi094.stdout: "auid": 0, 2024-09-14T06:31:24.192 INFO:teuthology.orchestra.run.smithi094.stdout: "snap_mode": "selfmanaged", 2024-09-14T06:31:24.192 INFO:teuthology.orchestra.run.smithi094.stdout: "snap_seq": 2, 2024-09-14T06:31:24.192 INFO:teuthology.orchestra.run.smithi094.stdout: "snap_epoch": 22, 2024-09-14T06:31:24.192 INFO:teuthology.orchestra.run.smithi094.stdout: "pool_snaps": [], 2024-09-14T06:31:24.192 INFO:teuthology.orchestra.run.smithi094.stdout: "removed_snaps": "[]", 2024-09-14T06:31:24.192 INFO:teuthology.orchestra.run.smithi094.stdout: "quota_max_bytes": 0, 2024-09-14T06:31:24.192 INFO:teuthology.orchestra.run.smithi094.stdout: "quota_max_objects": 0, 2024-09-14T06:31:24.192 INFO:teuthology.orchestra.run.smithi094.stdout: "tiers": [], 2024-09-14T06:31:24.192 INFO:teuthology.orchestra.run.smithi094.stdout: "tier_of": -1, 2024-09-14T06:31:24.192 INFO:teuthology.orchestra.run.smithi094.stdout: "read_tier": -1, 2024-09-14T06:31:24.193 INFO:teuthology.orchestra.run.smithi094.stdout: "write_tier": -1, 2024-09-14T06:31:24.193 INFO:teuthology.orchestra.run.smithi094.stdout: "cache_mode": "none", 2024-09-14T06:31:24.193 INFO:teuthology.orchestra.run.smithi094.stdout: "target_max_bytes": 0, 2024-09-14T06:31:24.193 INFO:teuthology.orchestra.run.smithi094.stdout: "target_max_objects": 0, 2024-09-14T06:31:24.193 INFO:teuthology.orchestra.run.smithi094.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-14T06:31:24.193 INFO:teuthology.orchestra.run.smithi094.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-14T06:31:24.193 INFO:teuthology.orchestra.run.smithi094.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-14T06:31:24.193 INFO:teuthology.orchestra.run.smithi094.stdout: "cache_min_flush_age": 0, 2024-09-14T06:31:24.193 INFO:teuthology.orchestra.run.smithi094.stdout: "cache_min_evict_age": 0, 2024-09-14T06:31:24.193 INFO:teuthology.orchestra.run.smithi094.stdout: "erasure_code_profile": "", 2024-09-14T06:31:24.193 INFO:teuthology.orchestra.run.smithi094.stdout: "hit_set_params": { 2024-09-14T06:31:24.193 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "none" 2024-09-14T06:31:24.193 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:24.193 INFO:teuthology.orchestra.run.smithi094.stdout: "hit_set_period": 0, 2024-09-14T06:31:24.194 INFO:teuthology.orchestra.run.smithi094.stdout: "hit_set_count": 0, 2024-09-14T06:31:24.194 INFO:teuthology.orchestra.run.smithi094.stdout: "use_gmt_hitset": true, 2024-09-14T06:31:24.194 INFO:teuthology.orchestra.run.smithi094.stdout: "min_read_recency_for_promote": 0, 2024-09-14T06:31:24.194 INFO:teuthology.orchestra.run.smithi094.stdout: "min_write_recency_for_promote": 0, 2024-09-14T06:31:24.194 INFO:teuthology.orchestra.run.smithi094.stdout: "hit_set_grade_decay_rate": 0, 2024-09-14T06:31:24.194 INFO:teuthology.orchestra.run.smithi094.stdout: "hit_set_search_last_n": 0, 2024-09-14T06:31:24.194 INFO:teuthology.orchestra.run.smithi094.stdout: "grade_table": [], 2024-09-14T06:31:24.194 INFO:teuthology.orchestra.run.smithi094.stdout: "stripe_width": 0, 2024-09-14T06:31:24.194 INFO:teuthology.orchestra.run.smithi094.stdout: "expected_num_objects": 0, 2024-09-14T06:31:24.194 INFO:teuthology.orchestra.run.smithi094.stdout: "fast_read": false, 2024-09-14T06:31:24.194 INFO:teuthology.orchestra.run.smithi094.stdout: "options": {}, 2024-09-14T06:31:24.194 INFO:teuthology.orchestra.run.smithi094.stdout: "application_metadata": { 2024-09-14T06:31:24.194 INFO:teuthology.orchestra.run.smithi094.stdout: "rbd": {} 2024-09-14T06:31:24.194 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:24.195 INFO:teuthology.orchestra.run.smithi094.stdout: "read_balance": { 2024-09-14T06:31:24.195 INFO:teuthology.orchestra.run.smithi094.stdout: "score_type": "Fair distribution", 2024-09-14T06:31:24.195 INFO:teuthology.orchestra.run.smithi094.stdout: "score_acting": 1.9900000095367432, 2024-09-14T06:31:24.195 INFO:teuthology.orchestra.run.smithi094.stdout: "score_stable": 1.9900000095367432, 2024-09-14T06:31:24.195 INFO:teuthology.orchestra.run.smithi094.stdout: "optimal_score": 0.87999999523162842, 2024-09-14T06:31:24.195 INFO:teuthology.orchestra.run.smithi094.stdout: "raw_score_acting": 1.75, 2024-09-14T06:31:24.195 INFO:teuthology.orchestra.run.smithi094.stdout: "raw_score_stable": 1.75, 2024-09-14T06:31:24.195 INFO:teuthology.orchestra.run.smithi094.stdout: "primary_affinity_weighted": 0.98013687133789062, 2024-09-14T06:31:24.195 INFO:teuthology.orchestra.run.smithi094.stdout: "average_primary_affinity": 0.98000001907348633, 2024-09-14T06:31:24.195 INFO:teuthology.orchestra.run.smithi094.stdout: "average_primary_affinity_weighted": 0.98000001907348633 2024-09-14T06:31:24.195 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:24.195 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:24.195 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:24.195 INFO:teuthology.orchestra.run.smithi094.stdout: "pool": 3, 2024-09-14T06:31:24.196 INFO:teuthology.orchestra.run.smithi094.stdout: "pool_name": "pool_client.0", 2024-09-14T06:31:24.196 INFO:teuthology.orchestra.run.smithi094.stdout: "create_time": "2024-09-14T06:28:59.688535+0000", 2024-09-14T06:31:24.196 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 8193, 2024-09-14T06:31:24.196 INFO:teuthology.orchestra.run.smithi094.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-09-14T06:31:24.196 INFO:teuthology.orchestra.run.smithi094.stdout: "type": 1, 2024-09-14T06:31:24.196 INFO:teuthology.orchestra.run.smithi094.stdout: "size": 2, 2024-09-14T06:31:24.196 INFO:teuthology.orchestra.run.smithi094.stdout: "min_size": 1, 2024-09-14T06:31:24.196 INFO:teuthology.orchestra.run.smithi094.stdout: "crush_rule": 0, 2024-09-14T06:31:24.196 INFO:teuthology.orchestra.run.smithi094.stdout: "peering_crush_bucket_count": 0, 2024-09-14T06:31:24.196 INFO:teuthology.orchestra.run.smithi094.stdout: "peering_crush_bucket_target": 0, 2024-09-14T06:31:24.196 INFO:teuthology.orchestra.run.smithi094.stdout: "peering_crush_bucket_barrier": 0, 2024-09-14T06:31:24.196 INFO:teuthology.orchestra.run.smithi094.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-14T06:31:24.196 INFO:teuthology.orchestra.run.smithi094.stdout: "is_stretch_pool": false, 2024-09-14T06:31:24.196 INFO:teuthology.orchestra.run.smithi094.stdout: "object_hash": 2, 2024-09-14T06:31:24.197 INFO:teuthology.orchestra.run.smithi094.stdout: "pg_autoscale_mode": "off", 2024-09-14T06:31:24.197 INFO:teuthology.orchestra.run.smithi094.stdout: "pg_num": 32, 2024-09-14T06:31:24.197 INFO:teuthology.orchestra.run.smithi094.stdout: "pg_placement_num": 32, 2024-09-14T06:31:24.197 INFO:teuthology.orchestra.run.smithi094.stdout: "pg_placement_num_target": 32, 2024-09-14T06:31:24.197 INFO:teuthology.orchestra.run.smithi094.stdout: "pg_num_target": 32, 2024-09-14T06:31:24.197 INFO:teuthology.orchestra.run.smithi094.stdout: "pg_num_pending": 32, 2024-09-14T06:31:24.197 INFO:teuthology.orchestra.run.smithi094.stdout: "last_pg_merge_meta": { 2024-09-14T06:31:24.197 INFO:teuthology.orchestra.run.smithi094.stdout: "source_pgid": "0.0", 2024-09-14T06:31:24.197 INFO:teuthology.orchestra.run.smithi094.stdout: "ready_epoch": 0, 2024-09-14T06:31:24.197 INFO:teuthology.orchestra.run.smithi094.stdout: "last_epoch_started": 0, 2024-09-14T06:31:24.197 INFO:teuthology.orchestra.run.smithi094.stdout: "last_epoch_clean": 0, 2024-09-14T06:31:24.197 INFO:teuthology.orchestra.run.smithi094.stdout: "source_version": "0'0", 2024-09-14T06:31:24.197 INFO:teuthology.orchestra.run.smithi094.stdout: "target_version": "0'0" 2024-09-14T06:31:24.198 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:24.198 INFO:teuthology.orchestra.run.smithi094.stdout: "last_change": "159", 2024-09-14T06:31:24.198 INFO:teuthology.orchestra.run.smithi094.stdout: "last_force_op_resend": "0", 2024-09-14T06:31:24.198 INFO:teuthology.orchestra.run.smithi094.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-14T06:31:24.198 INFO:teuthology.orchestra.run.smithi094.stdout: "last_force_op_resend_preluminous": "0", 2024-09-14T06:31:24.198 INFO:teuthology.orchestra.run.smithi094.stdout: "auid": 0, 2024-09-14T06:31:24.198 INFO:teuthology.orchestra.run.smithi094.stdout: "snap_mode": "selfmanaged", 2024-09-14T06:31:24.198 INFO:teuthology.orchestra.run.smithi094.stdout: "snap_seq": 77, 2024-09-14T06:31:24.198 INFO:teuthology.orchestra.run.smithi094.stdout: "snap_epoch": 159, 2024-09-14T06:31:24.198 INFO:teuthology.orchestra.run.smithi094.stdout: "pool_snaps": [], 2024-09-14T06:31:24.198 INFO:teuthology.orchestra.run.smithi094.stdout: "removed_snaps": "[]", 2024-09-14T06:31:24.198 INFO:teuthology.orchestra.run.smithi094.stdout: "quota_max_bytes": 0, 2024-09-14T06:31:24.198 INFO:teuthology.orchestra.run.smithi094.stdout: "quota_max_objects": 0, 2024-09-14T06:31:24.198 INFO:teuthology.orchestra.run.smithi094.stdout: "tiers": [], 2024-09-14T06:31:24.199 INFO:teuthology.orchestra.run.smithi094.stdout: "tier_of": -1, 2024-09-14T06:31:24.199 INFO:teuthology.orchestra.run.smithi094.stdout: "read_tier": -1, 2024-09-14T06:31:24.199 INFO:teuthology.orchestra.run.smithi094.stdout: "write_tier": -1, 2024-09-14T06:31:24.199 INFO:teuthology.orchestra.run.smithi094.stdout: "cache_mode": "none", 2024-09-14T06:31:24.199 INFO:teuthology.orchestra.run.smithi094.stdout: "target_max_bytes": 0, 2024-09-14T06:31:24.199 INFO:teuthology.orchestra.run.smithi094.stdout: "target_max_objects": 0, 2024-09-14T06:31:24.199 INFO:teuthology.orchestra.run.smithi094.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-14T06:31:24.199 INFO:teuthology.orchestra.run.smithi094.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-14T06:31:24.199 INFO:teuthology.orchestra.run.smithi094.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-14T06:31:24.199 INFO:teuthology.orchestra.run.smithi094.stdout: "cache_min_flush_age": 0, 2024-09-14T06:31:24.199 INFO:teuthology.orchestra.run.smithi094.stdout: "cache_min_evict_age": 0, 2024-09-14T06:31:24.199 INFO:teuthology.orchestra.run.smithi094.stdout: "erasure_code_profile": "", 2024-09-14T06:31:24.199 INFO:teuthology.orchestra.run.smithi094.stdout: "hit_set_params": { 2024-09-14T06:31:24.199 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "none" 2024-09-14T06:31:24.199 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:24.200 INFO:teuthology.orchestra.run.smithi094.stdout: "hit_set_period": 0, 2024-09-14T06:31:24.200 INFO:teuthology.orchestra.run.smithi094.stdout: "hit_set_count": 0, 2024-09-14T06:31:24.200 INFO:teuthology.orchestra.run.smithi094.stdout: "use_gmt_hitset": true, 2024-09-14T06:31:24.200 INFO:teuthology.orchestra.run.smithi094.stdout: "min_read_recency_for_promote": 0, 2024-09-14T06:31:24.200 INFO:teuthology.orchestra.run.smithi094.stdout: "min_write_recency_for_promote": 0, 2024-09-14T06:31:24.200 INFO:teuthology.orchestra.run.smithi094.stdout: "hit_set_grade_decay_rate": 0, 2024-09-14T06:31:24.200 INFO:teuthology.orchestra.run.smithi094.stdout: "hit_set_search_last_n": 0, 2024-09-14T06:31:24.200 INFO:teuthology.orchestra.run.smithi094.stdout: "grade_table": [], 2024-09-14T06:31:24.200 INFO:teuthology.orchestra.run.smithi094.stdout: "stripe_width": 0, 2024-09-14T06:31:24.200 INFO:teuthology.orchestra.run.smithi094.stdout: "expected_num_objects": 0, 2024-09-14T06:31:24.200 INFO:teuthology.orchestra.run.smithi094.stdout: "fast_read": false, 2024-09-14T06:31:24.200 INFO:teuthology.orchestra.run.smithi094.stdout: "options": {}, 2024-09-14T06:31:24.200 INFO:teuthology.orchestra.run.smithi094.stdout: "application_metadata": { 2024-09-14T06:31:24.201 INFO:teuthology.orchestra.run.smithi094.stdout: "rbd": {} 2024-09-14T06:31:24.201 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:24.201 INFO:teuthology.orchestra.run.smithi094.stdout: "read_balance": { 2024-09-14T06:31:24.201 INFO:teuthology.orchestra.run.smithi094.stdout: "score_type": "Fair distribution", 2024-09-14T06:31:24.201 INFO:teuthology.orchestra.run.smithi094.stdout: "score_acting": 1.75, 2024-09-14T06:31:24.201 INFO:teuthology.orchestra.run.smithi094.stdout: "score_stable": 1.75, 2024-09-14T06:31:24.201 INFO:teuthology.orchestra.run.smithi094.stdout: "optimal_score": 1, 2024-09-14T06:31:24.201 INFO:teuthology.orchestra.run.smithi094.stdout: "raw_score_acting": 1.75, 2024-09-14T06:31:24.201 INFO:teuthology.orchestra.run.smithi094.stdout: "raw_score_stable": 1.75, 2024-09-14T06:31:24.201 INFO:teuthology.orchestra.run.smithi094.stdout: "primary_affinity_weighted": 0.98013687133789062, 2024-09-14T06:31:24.201 INFO:teuthology.orchestra.run.smithi094.stdout: "average_primary_affinity": 0.98000001907348633, 2024-09-14T06:31:24.201 INFO:teuthology.orchestra.run.smithi094.stdout: "average_primary_affinity_weighted": 0.98000001907348633 2024-09-14T06:31:24.201 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:24.201 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:24.202 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-09-14T06:31:24.202 INFO:teuthology.orchestra.run.smithi094.stdout: "osds": [ 2024-09-14T06:31:24.202 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:24.202 INFO:teuthology.orchestra.run.smithi094.stdout: "osd": 0, 2024-09-14T06:31:24.202 INFO:teuthology.orchestra.run.smithi094.stdout: "uuid": "8b22a249-b842-406a-a02c-c0f31f57b635", 2024-09-14T06:31:24.202 INFO:teuthology.orchestra.run.smithi094.stdout: "up": 1, 2024-09-14T06:31:24.202 INFO:teuthology.orchestra.run.smithi094.stdout: "in": 1, 2024-09-14T06:31:24.202 INFO:teuthology.orchestra.run.smithi094.stdout: "weight": 1, 2024-09-14T06:31:24.202 INFO:teuthology.orchestra.run.smithi094.stdout: "primary_affinity": 1, 2024-09-14T06:31:24.202 INFO:teuthology.orchestra.run.smithi094.stdout: "last_clean_begin": 0, 2024-09-14T06:31:24.202 INFO:teuthology.orchestra.run.smithi094.stdout: "last_clean_end": 0, 2024-09-14T06:31:24.202 INFO:teuthology.orchestra.run.smithi094.stdout: "up_from": 14, 2024-09-14T06:31:24.202 INFO:teuthology.orchestra.run.smithi094.stdout: "up_thru": 133, 2024-09-14T06:31:24.202 INFO:teuthology.orchestra.run.smithi094.stdout: "down_at": 0, 2024-09-14T06:31:24.203 INFO:teuthology.orchestra.run.smithi094.stdout: "lost_at": 0, 2024-09-14T06:31:24.203 INFO:teuthology.orchestra.run.smithi094.stdout: "public_addrs": { 2024-09-14T06:31:24.203 INFO:teuthology.orchestra.run.smithi094.stdout: "addrvec": [ 2024-09-14T06:31:24.203 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:24.203 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v2", 2024-09-14T06:31:24.203 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.94:6824", 2024-09-14T06:31:24.203 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 1809564923 2024-09-14T06:31:24.203 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:24.203 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:24.203 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v1", 2024-09-14T06:31:24.203 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.94:6825", 2024-09-14T06:31:24.203 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 1809564923 2024-09-14T06:31:24.203 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:24.203 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:31:24.204 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:24.204 INFO:teuthology.orchestra.run.smithi094.stdout: "cluster_addrs": { 2024-09-14T06:31:24.204 INFO:teuthology.orchestra.run.smithi094.stdout: "addrvec": [ 2024-09-14T06:31:24.204 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:24.204 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v2", 2024-09-14T06:31:24.204 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.94:6826", 2024-09-14T06:31:24.204 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 1809564923 2024-09-14T06:31:24.204 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:24.204 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:24.204 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v1", 2024-09-14T06:31:24.204 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.94:6827", 2024-09-14T06:31:24.204 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 1809564923 2024-09-14T06:31:24.204 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:24.205 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:31:24.205 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:24.205 INFO:teuthology.orchestra.run.smithi094.stdout: "heartbeat_back_addrs": { 2024-09-14T06:31:24.205 INFO:teuthology.orchestra.run.smithi094.stdout: "addrvec": [ 2024-09-14T06:31:24.205 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:24.205 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v2", 2024-09-14T06:31:24.205 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.94:6830", 2024-09-14T06:31:24.205 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 1809564923 2024-09-14T06:31:24.205 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:24.205 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:24.205 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v1", 2024-09-14T06:31:24.205 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.94:6831", 2024-09-14T06:31:24.205 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 1809564923 2024-09-14T06:31:24.205 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:24.206 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:31:24.206 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:24.206 INFO:teuthology.orchestra.run.smithi094.stdout: "heartbeat_front_addrs": { 2024-09-14T06:31:24.206 INFO:teuthology.orchestra.run.smithi094.stdout: "addrvec": [ 2024-09-14T06:31:24.206 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:24.206 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v2", 2024-09-14T06:31:24.206 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.94:6828", 2024-09-14T06:31:24.206 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 1809564923 2024-09-14T06:31:24.206 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:24.206 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:24.206 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v1", 2024-09-14T06:31:24.206 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.94:6829", 2024-09-14T06:31:24.206 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 1809564923 2024-09-14T06:31:24.206 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:24.207 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:31:24.207 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:24.207 INFO:teuthology.orchestra.run.smithi094.stdout: "public_addr": "172.21.15.94:6825/1809564923", 2024-09-14T06:31:24.207 INFO:teuthology.orchestra.run.smithi094.stdout: "cluster_addr": "172.21.15.94:6827/1809564923", 2024-09-14T06:31:24.207 INFO:teuthology.orchestra.run.smithi094.stdout: "heartbeat_back_addr": "172.21.15.94:6831/1809564923", 2024-09-14T06:31:24.207 INFO:teuthology.orchestra.run.smithi094.stdout: "heartbeat_front_addr": "172.21.15.94:6829/1809564923", 2024-09-14T06:31:24.207 INFO:teuthology.orchestra.run.smithi094.stdout: "state": [ 2024-09-14T06:31:24.207 INFO:teuthology.orchestra.run.smithi094.stdout: "exists", 2024-09-14T06:31:24.207 INFO:teuthology.orchestra.run.smithi094.stdout: "up" 2024-09-14T06:31:24.207 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:31:24.207 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:24.207 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:24.207 INFO:teuthology.orchestra.run.smithi094.stdout: "osd": 1, 2024-09-14T06:31:24.207 INFO:teuthology.orchestra.run.smithi094.stdout: "uuid": "02251a03-84a0-414d-a980-fac4fd043cb6", 2024-09-14T06:31:24.208 INFO:teuthology.orchestra.run.smithi094.stdout: "up": 1, 2024-09-14T06:31:24.208 INFO:teuthology.orchestra.run.smithi094.stdout: "in": 1, 2024-09-14T06:31:24.208 INFO:teuthology.orchestra.run.smithi094.stdout: "weight": 1, 2024-09-14T06:31:24.208 INFO:teuthology.orchestra.run.smithi094.stdout: "primary_affinity": 0.841094970703125, 2024-09-14T06:31:24.208 INFO:teuthology.orchestra.run.smithi094.stdout: "last_clean_begin": 0, 2024-09-14T06:31:24.208 INFO:teuthology.orchestra.run.smithi094.stdout: "last_clean_end": 0, 2024-09-14T06:31:24.208 INFO:teuthology.orchestra.run.smithi094.stdout: "up_from": 15, 2024-09-14T06:31:24.208 INFO:teuthology.orchestra.run.smithi094.stdout: "up_thru": 132, 2024-09-14T06:31:24.208 INFO:teuthology.orchestra.run.smithi094.stdout: "down_at": 0, 2024-09-14T06:31:24.208 INFO:teuthology.orchestra.run.smithi094.stdout: "lost_at": 0, 2024-09-14T06:31:24.208 INFO:teuthology.orchestra.run.smithi094.stdout: "public_addrs": { 2024-09-14T06:31:24.208 INFO:teuthology.orchestra.run.smithi094.stdout: "addrvec": [ 2024-09-14T06:31:24.208 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:24.208 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v2", 2024-09-14T06:31:24.209 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.94:6816", 2024-09-14T06:31:24.209 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 736402652 2024-09-14T06:31:24.209 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:24.209 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:24.209 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v1", 2024-09-14T06:31:24.209 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.94:6817", 2024-09-14T06:31:24.209 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 736402652 2024-09-14T06:31:24.209 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:24.209 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:31:24.209 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:24.209 INFO:teuthology.orchestra.run.smithi094.stdout: "cluster_addrs": { 2024-09-14T06:31:24.209 INFO:teuthology.orchestra.run.smithi094.stdout: "addrvec": [ 2024-09-14T06:31:24.209 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:24.209 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v2", 2024-09-14T06:31:24.210 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.94:6818", 2024-09-14T06:31:24.210 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 736402652 2024-09-14T06:31:24.210 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:24.210 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:24.210 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v1", 2024-09-14T06:31:24.210 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.94:6819", 2024-09-14T06:31:24.210 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 736402652 2024-09-14T06:31:24.210 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:24.210 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:31:24.210 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:24.210 INFO:teuthology.orchestra.run.smithi094.stdout: "heartbeat_back_addrs": { 2024-09-14T06:31:24.210 INFO:teuthology.orchestra.run.smithi094.stdout: "addrvec": [ 2024-09-14T06:31:24.210 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:24.210 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v2", 2024-09-14T06:31:24.211 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.94:6822", 2024-09-14T06:31:24.211 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 736402652 2024-09-14T06:31:24.211 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:24.211 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:24.211 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v1", 2024-09-14T06:31:24.211 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.94:6823", 2024-09-14T06:31:24.211 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 736402652 2024-09-14T06:31:24.211 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:24.211 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:31:24.211 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:24.211 INFO:teuthology.orchestra.run.smithi094.stdout: "heartbeat_front_addrs": { 2024-09-14T06:31:24.211 INFO:teuthology.orchestra.run.smithi094.stdout: "addrvec": [ 2024-09-14T06:31:24.211 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:24.211 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v2", 2024-09-14T06:31:24.212 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.94:6820", 2024-09-14T06:31:24.212 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 736402652 2024-09-14T06:31:24.212 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:24.212 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:24.212 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v1", 2024-09-14T06:31:24.212 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.94:6821", 2024-09-14T06:31:24.212 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 736402652 2024-09-14T06:31:24.212 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:24.212 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:31:24.212 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:24.212 INFO:teuthology.orchestra.run.smithi094.stdout: "public_addr": "172.21.15.94:6817/736402652", 2024-09-14T06:31:24.212 INFO:teuthology.orchestra.run.smithi094.stdout: "cluster_addr": "172.21.15.94:6819/736402652", 2024-09-14T06:31:24.212 INFO:teuthology.orchestra.run.smithi094.stdout: "heartbeat_back_addr": "172.21.15.94:6823/736402652", 2024-09-14T06:31:24.212 INFO:teuthology.orchestra.run.smithi094.stdout: "heartbeat_front_addr": "172.21.15.94:6821/736402652", 2024-09-14T06:31:24.213 INFO:teuthology.orchestra.run.smithi094.stdout: "state": [ 2024-09-14T06:31:24.213 INFO:teuthology.orchestra.run.smithi094.stdout: "exists", 2024-09-14T06:31:24.213 INFO:teuthology.orchestra.run.smithi094.stdout: "up" 2024-09-14T06:31:24.213 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:31:24.213 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:24.213 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:24.213 INFO:teuthology.orchestra.run.smithi094.stdout: "osd": 2, 2024-09-14T06:31:24.213 INFO:teuthology.orchestra.run.smithi094.stdout: "uuid": "d45bf470-bb29-4739-8294-cd122b2b4102", 2024-09-14T06:31:24.213 INFO:teuthology.orchestra.run.smithi094.stdout: "up": 1, 2024-09-14T06:31:24.213 INFO:teuthology.orchestra.run.smithi094.stdout: "in": 1, 2024-09-14T06:31:24.213 INFO:teuthology.orchestra.run.smithi094.stdout: "weight": 1, 2024-09-14T06:31:24.213 INFO:teuthology.orchestra.run.smithi094.stdout: "primary_affinity": 1, 2024-09-14T06:31:24.213 INFO:teuthology.orchestra.run.smithi094.stdout: "last_clean_begin": 0, 2024-09-14T06:31:24.214 INFO:teuthology.orchestra.run.smithi094.stdout: "last_clean_end": 0, 2024-09-14T06:31:24.214 INFO:teuthology.orchestra.run.smithi094.stdout: "up_from": 14, 2024-09-14T06:31:24.214 INFO:teuthology.orchestra.run.smithi094.stdout: "up_thru": 19, 2024-09-14T06:31:24.214 INFO:teuthology.orchestra.run.smithi094.stdout: "down_at": 0, 2024-09-14T06:31:24.214 INFO:teuthology.orchestra.run.smithi094.stdout: "lost_at": 0, 2024-09-14T06:31:24.214 INFO:teuthology.orchestra.run.smithi094.stdout: "public_addrs": { 2024-09-14T06:31:24.214 INFO:teuthology.orchestra.run.smithi094.stdout: "addrvec": [ 2024-09-14T06:31:24.214 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:24.214 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v2", 2024-09-14T06:31:24.214 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.94:6808", 2024-09-14T06:31:24.214 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 1063204782 2024-09-14T06:31:24.214 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:24.214 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:24.214 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v1", 2024-09-14T06:31:24.215 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.94:6809", 2024-09-14T06:31:24.215 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 1063204782 2024-09-14T06:31:24.215 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:24.215 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:31:24.215 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:24.215 INFO:teuthology.orchestra.run.smithi094.stdout: "cluster_addrs": { 2024-09-14T06:31:24.215 INFO:teuthology.orchestra.run.smithi094.stdout: "addrvec": [ 2024-09-14T06:31:24.215 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:24.215 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v2", 2024-09-14T06:31:24.215 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.94:6810", 2024-09-14T06:31:24.215 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 1063204782 2024-09-14T06:31:24.215 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:24.215 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:24.216 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v1", 2024-09-14T06:31:24.216 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.94:6811", 2024-09-14T06:31:24.216 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 1063204782 2024-09-14T06:31:24.216 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:24.216 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:31:24.216 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:24.216 INFO:teuthology.orchestra.run.smithi094.stdout: "heartbeat_back_addrs": { 2024-09-14T06:31:24.216 INFO:teuthology.orchestra.run.smithi094.stdout: "addrvec": [ 2024-09-14T06:31:24.216 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:24.216 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v2", 2024-09-14T06:31:24.216 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.94:6814", 2024-09-14T06:31:24.216 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 1063204782 2024-09-14T06:31:24.216 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:24.217 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:24.217 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v1", 2024-09-14T06:31:24.217 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.94:6815", 2024-09-14T06:31:24.217 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 1063204782 2024-09-14T06:31:24.217 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:24.217 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:31:24.217 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:24.217 INFO:teuthology.orchestra.run.smithi094.stdout: "heartbeat_front_addrs": { 2024-09-14T06:31:24.217 INFO:teuthology.orchestra.run.smithi094.stdout: "addrvec": [ 2024-09-14T06:31:24.217 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:24.217 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v2", 2024-09-14T06:31:24.217 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.94:6812", 2024-09-14T06:31:24.217 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 1063204782 2024-09-14T06:31:24.218 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:24.218 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:24.218 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v1", 2024-09-14T06:31:24.218 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.94:6813", 2024-09-14T06:31:24.218 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 1063204782 2024-09-14T06:31:24.218 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:24.218 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:31:24.218 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:24.218 INFO:teuthology.orchestra.run.smithi094.stdout: "public_addr": "172.21.15.94:6809/1063204782", 2024-09-14T06:31:24.218 INFO:teuthology.orchestra.run.smithi094.stdout: "cluster_addr": "172.21.15.94:6811/1063204782", 2024-09-14T06:31:24.218 INFO:teuthology.orchestra.run.smithi094.stdout: "heartbeat_back_addr": "172.21.15.94:6815/1063204782", 2024-09-14T06:31:24.218 INFO:teuthology.orchestra.run.smithi094.stdout: "heartbeat_front_addr": "172.21.15.94:6813/1063204782", 2024-09-14T06:31:24.218 INFO:teuthology.orchestra.run.smithi094.stdout: "state": [ 2024-09-14T06:31:24.218 INFO:teuthology.orchestra.run.smithi094.stdout: "exists", 2024-09-14T06:31:24.219 INFO:teuthology.orchestra.run.smithi094.stdout: "up" 2024-09-14T06:31:24.219 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:31:24.219 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:24.219 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:24.219 INFO:teuthology.orchestra.run.smithi094.stdout: "osd": 3, 2024-09-14T06:31:24.219 INFO:teuthology.orchestra.run.smithi094.stdout: "uuid": "456ec2d9-c438-4243-b931-335704f661eb", 2024-09-14T06:31:24.219 INFO:teuthology.orchestra.run.smithi094.stdout: "up": 1, 2024-09-14T06:31:24.219 INFO:teuthology.orchestra.run.smithi094.stdout: "in": 1, 2024-09-14T06:31:24.219 INFO:teuthology.orchestra.run.smithi094.stdout: "weight": 1, 2024-09-14T06:31:24.219 INFO:teuthology.orchestra.run.smithi094.stdout: "primary_affinity": 1, 2024-09-14T06:31:24.219 INFO:teuthology.orchestra.run.smithi094.stdout: "last_clean_begin": 0, 2024-09-14T06:31:24.219 INFO:teuthology.orchestra.run.smithi094.stdout: "last_clean_end": 0, 2024-09-14T06:31:24.219 INFO:teuthology.orchestra.run.smithi094.stdout: "up_from": 13, 2024-09-14T06:31:24.219 INFO:teuthology.orchestra.run.smithi094.stdout: "up_thru": 74, 2024-09-14T06:31:24.219 INFO:teuthology.orchestra.run.smithi094.stdout: "down_at": 0, 2024-09-14T06:31:24.220 INFO:teuthology.orchestra.run.smithi094.stdout: "lost_at": 0, 2024-09-14T06:31:24.220 INFO:teuthology.orchestra.run.smithi094.stdout: "public_addrs": { 2024-09-14T06:31:24.220 INFO:teuthology.orchestra.run.smithi094.stdout: "addrvec": [ 2024-09-14T06:31:24.220 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:24.220 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v2", 2024-09-14T06:31:24.220 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.94:6800", 2024-09-14T06:31:24.220 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 3382046754 2024-09-14T06:31:24.220 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:24.220 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:24.220 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v1", 2024-09-14T06:31:24.220 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.94:6801", 2024-09-14T06:31:24.220 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 3382046754 2024-09-14T06:31:24.220 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:24.221 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:31:24.221 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:24.221 INFO:teuthology.orchestra.run.smithi094.stdout: "cluster_addrs": { 2024-09-14T06:31:24.221 INFO:teuthology.orchestra.run.smithi094.stdout: "addrvec": [ 2024-09-14T06:31:24.221 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:24.221 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v2", 2024-09-14T06:31:24.221 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.94:6802", 2024-09-14T06:31:24.221 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 3382046754 2024-09-14T06:31:24.221 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:24.221 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:24.221 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v1", 2024-09-14T06:31:24.221 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.94:6803", 2024-09-14T06:31:24.221 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 3382046754 2024-09-14T06:31:24.222 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:24.222 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:31:24.222 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:24.222 INFO:teuthology.orchestra.run.smithi094.stdout: "heartbeat_back_addrs": { 2024-09-14T06:31:24.222 INFO:teuthology.orchestra.run.smithi094.stdout: "addrvec": [ 2024-09-14T06:31:24.222 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:24.222 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v2", 2024-09-14T06:31:24.222 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.94:6806", 2024-09-14T06:31:24.222 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 3382046754 2024-09-14T06:31:24.222 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:24.222 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:24.222 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v1", 2024-09-14T06:31:24.222 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.94:6807", 2024-09-14T06:31:24.222 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 3382046754 2024-09-14T06:31:24.223 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:24.223 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:31:24.223 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:24.223 INFO:teuthology.orchestra.run.smithi094.stdout: "heartbeat_front_addrs": { 2024-09-14T06:31:24.223 INFO:teuthology.orchestra.run.smithi094.stdout: "addrvec": [ 2024-09-14T06:31:24.223 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:24.223 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v2", 2024-09-14T06:31:24.223 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.94:6804", 2024-09-14T06:31:24.223 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 3382046754 2024-09-14T06:31:24.223 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:24.223 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:24.223 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v1", 2024-09-14T06:31:24.223 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.94:6805", 2024-09-14T06:31:24.224 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 3382046754 2024-09-14T06:31:24.224 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:24.224 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:31:24.224 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:24.224 INFO:teuthology.orchestra.run.smithi094.stdout: "public_addr": "172.21.15.94:6801/3382046754", 2024-09-14T06:31:24.224 INFO:teuthology.orchestra.run.smithi094.stdout: "cluster_addr": "172.21.15.94:6803/3382046754", 2024-09-14T06:31:24.224 INFO:teuthology.orchestra.run.smithi094.stdout: "heartbeat_back_addr": "172.21.15.94:6807/3382046754", 2024-09-14T06:31:24.224 INFO:teuthology.orchestra.run.smithi094.stdout: "heartbeat_front_addr": "172.21.15.94:6805/3382046754", 2024-09-14T06:31:24.224 INFO:teuthology.orchestra.run.smithi094.stdout: "state": [ 2024-09-14T06:31:24.224 INFO:teuthology.orchestra.run.smithi094.stdout: "exists", 2024-09-14T06:31:24.224 INFO:teuthology.orchestra.run.smithi094.stdout: "up" 2024-09-14T06:31:24.224 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:31:24.224 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:24.225 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:24.225 INFO:teuthology.orchestra.run.smithi094.stdout: "osd": 4, 2024-09-14T06:31:24.225 INFO:teuthology.orchestra.run.smithi094.stdout: "uuid": "2dab948e-c6e4-4437-bf17-017d1ac6821c", 2024-09-14T06:31:24.225 INFO:teuthology.orchestra.run.smithi094.stdout: "up": 1, 2024-09-14T06:31:24.225 INFO:teuthology.orchestra.run.smithi094.stdout: "in": 1, 2024-09-14T06:31:24.225 INFO:teuthology.orchestra.run.smithi094.stdout: "weight": 1, 2024-09-14T06:31:24.225 INFO:teuthology.orchestra.run.smithi094.stdout: "primary_affinity": 1, 2024-09-14T06:31:24.225 INFO:teuthology.orchestra.run.smithi094.stdout: "last_clean_begin": 0, 2024-09-14T06:31:24.225 INFO:teuthology.orchestra.run.smithi094.stdout: "last_clean_end": 0, 2024-09-14T06:31:24.225 INFO:teuthology.orchestra.run.smithi094.stdout: "up_from": 15, 2024-09-14T06:31:24.225 INFO:teuthology.orchestra.run.smithi094.stdout: "up_thru": 116, 2024-09-14T06:31:24.225 INFO:teuthology.orchestra.run.smithi094.stdout: "down_at": 0, 2024-09-14T06:31:24.225 INFO:teuthology.orchestra.run.smithi094.stdout: "lost_at": 0, 2024-09-14T06:31:24.225 INFO:teuthology.orchestra.run.smithi094.stdout: "public_addrs": { 2024-09-14T06:31:24.226 INFO:teuthology.orchestra.run.smithi094.stdout: "addrvec": [ 2024-09-14T06:31:24.226 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:24.226 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v2", 2024-09-14T06:31:24.226 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.138:6810", 2024-09-14T06:31:24.226 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 501726729 2024-09-14T06:31:24.226 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:24.226 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:24.226 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v1", 2024-09-14T06:31:24.226 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.138:6811", 2024-09-14T06:31:24.226 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 501726729 2024-09-14T06:31:24.226 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:24.226 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:31:24.226 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:24.226 INFO:teuthology.orchestra.run.smithi094.stdout: "cluster_addrs": { 2024-09-14T06:31:24.227 INFO:teuthology.orchestra.run.smithi094.stdout: "addrvec": [ 2024-09-14T06:31:24.227 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:24.227 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v2", 2024-09-14T06:31:24.227 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.138:6812", 2024-09-14T06:31:24.227 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 501726729 2024-09-14T06:31:24.227 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:24.227 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:24.227 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v1", 2024-09-14T06:31:24.227 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.138:6813", 2024-09-14T06:31:24.227 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 501726729 2024-09-14T06:31:24.227 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:24.227 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:31:24.228 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:24.228 INFO:teuthology.orchestra.run.smithi094.stdout: "heartbeat_back_addrs": { 2024-09-14T06:31:24.228 INFO:teuthology.orchestra.run.smithi094.stdout: "addrvec": [ 2024-09-14T06:31:24.228 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:24.228 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v2", 2024-09-14T06:31:24.228 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.138:6816", 2024-09-14T06:31:24.228 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 501726729 2024-09-14T06:31:24.228 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:24.228 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:24.228 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v1", 2024-09-14T06:31:24.228 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.138:6817", 2024-09-14T06:31:24.228 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 501726729 2024-09-14T06:31:24.228 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:24.228 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:31:24.229 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:24.229 INFO:teuthology.orchestra.run.smithi094.stdout: "heartbeat_front_addrs": { 2024-09-14T06:31:24.229 INFO:teuthology.orchestra.run.smithi094.stdout: "addrvec": [ 2024-09-14T06:31:24.229 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:24.229 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v2", 2024-09-14T06:31:24.229 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.138:6814", 2024-09-14T06:31:24.229 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 501726729 2024-09-14T06:31:24.229 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:24.229 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:24.229 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v1", 2024-09-14T06:31:24.229 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.138:6815", 2024-09-14T06:31:24.229 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 501726729 2024-09-14T06:31:24.229 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:24.230 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:31:24.230 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:24.230 INFO:teuthology.orchestra.run.smithi094.stdout: "public_addr": "172.21.15.138:6811/501726729", 2024-09-14T06:31:24.230 INFO:teuthology.orchestra.run.smithi094.stdout: "cluster_addr": "172.21.15.138:6813/501726729", 2024-09-14T06:31:24.230 INFO:teuthology.orchestra.run.smithi094.stdout: "heartbeat_back_addr": "172.21.15.138:6817/501726729", 2024-09-14T06:31:24.230 INFO:teuthology.orchestra.run.smithi094.stdout: "heartbeat_front_addr": "172.21.15.138:6815/501726729", 2024-09-14T06:31:24.230 INFO:teuthology.orchestra.run.smithi094.stdout: "state": [ 2024-09-14T06:31:24.230 INFO:teuthology.orchestra.run.smithi094.stdout: "exists", 2024-09-14T06:31:24.230 INFO:teuthology.orchestra.run.smithi094.stdout: "up" 2024-09-14T06:31:24.230 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:31:24.230 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:24.230 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:24.230 INFO:teuthology.orchestra.run.smithi094.stdout: "osd": 5, 2024-09-14T06:31:24.230 INFO:teuthology.orchestra.run.smithi094.stdout: "uuid": "8caf3aa3-ad87-4d33-b82e-4c15195176fd", 2024-09-14T06:31:24.230 INFO:teuthology.orchestra.run.smithi094.stdout: "up": 1, 2024-09-14T06:31:24.231 INFO:teuthology.orchestra.run.smithi094.stdout: "in": 1, 2024-09-14T06:31:24.231 INFO:teuthology.orchestra.run.smithi094.stdout: "weight": 1, 2024-09-14T06:31:24.231 INFO:teuthology.orchestra.run.smithi094.stdout: "primary_affinity": 1, 2024-09-14T06:31:24.231 INFO:teuthology.orchestra.run.smithi094.stdout: "last_clean_begin": 0, 2024-09-14T06:31:24.231 INFO:teuthology.orchestra.run.smithi094.stdout: "last_clean_end": 0, 2024-09-14T06:31:24.231 INFO:teuthology.orchestra.run.smithi094.stdout: "up_from": 15, 2024-09-14T06:31:24.231 INFO:teuthology.orchestra.run.smithi094.stdout: "up_thru": 23, 2024-09-14T06:31:24.231 INFO:teuthology.orchestra.run.smithi094.stdout: "down_at": 0, 2024-09-14T06:31:24.231 INFO:teuthology.orchestra.run.smithi094.stdout: "lost_at": 0, 2024-09-14T06:31:24.231 INFO:teuthology.orchestra.run.smithi094.stdout: "public_addrs": { 2024-09-14T06:31:24.231 INFO:teuthology.orchestra.run.smithi094.stdout: "addrvec": [ 2024-09-14T06:31:24.231 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:24.231 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v2", 2024-09-14T06:31:24.232 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.138:6826", 2024-09-14T06:31:24.232 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 1599292735 2024-09-14T06:31:24.232 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:24.232 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:24.232 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v1", 2024-09-14T06:31:24.232 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.138:6827", 2024-09-14T06:31:24.232 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 1599292735 2024-09-14T06:31:24.232 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:24.232 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:31:24.232 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:24.232 INFO:teuthology.orchestra.run.smithi094.stdout: "cluster_addrs": { 2024-09-14T06:31:24.232 INFO:teuthology.orchestra.run.smithi094.stdout: "addrvec": [ 2024-09-14T06:31:24.232 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:24.233 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v2", 2024-09-14T06:31:24.233 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.138:6828", 2024-09-14T06:31:24.233 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 1599292735 2024-09-14T06:31:24.233 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:24.233 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:24.233 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v1", 2024-09-14T06:31:24.233 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.138:6829", 2024-09-14T06:31:24.233 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 1599292735 2024-09-14T06:31:24.233 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:24.233 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:31:24.233 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:24.233 INFO:teuthology.orchestra.run.smithi094.stdout: "heartbeat_back_addrs": { 2024-09-14T06:31:24.233 INFO:teuthology.orchestra.run.smithi094.stdout: "addrvec": [ 2024-09-14T06:31:24.233 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:24.234 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v2", 2024-09-14T06:31:24.234 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.138:6832", 2024-09-14T06:31:24.234 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 1599292735 2024-09-14T06:31:24.234 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:24.234 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:24.234 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v1", 2024-09-14T06:31:24.234 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.138:6833", 2024-09-14T06:31:24.234 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 1599292735 2024-09-14T06:31:24.234 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:24.234 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:31:24.234 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:24.234 INFO:teuthology.orchestra.run.smithi094.stdout: "heartbeat_front_addrs": { 2024-09-14T06:31:24.234 INFO:teuthology.orchestra.run.smithi094.stdout: "addrvec": [ 2024-09-14T06:31:24.235 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:24.235 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v2", 2024-09-14T06:31:24.235 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.138:6830", 2024-09-14T06:31:24.235 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 1599292735 2024-09-14T06:31:24.235 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:24.235 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:24.235 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v1", 2024-09-14T06:31:24.235 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.138:6831", 2024-09-14T06:31:24.235 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 1599292735 2024-09-14T06:31:24.235 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:24.235 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:31:24.235 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:24.235 INFO:teuthology.orchestra.run.smithi094.stdout: "public_addr": "172.21.15.138:6827/1599292735", 2024-09-14T06:31:24.235 INFO:teuthology.orchestra.run.smithi094.stdout: "cluster_addr": "172.21.15.138:6829/1599292735", 2024-09-14T06:31:24.236 INFO:teuthology.orchestra.run.smithi094.stdout: "heartbeat_back_addr": "172.21.15.138:6833/1599292735", 2024-09-14T06:31:24.236 INFO:teuthology.orchestra.run.smithi094.stdout: "heartbeat_front_addr": "172.21.15.138:6831/1599292735", 2024-09-14T06:31:24.236 INFO:teuthology.orchestra.run.smithi094.stdout: "state": [ 2024-09-14T06:31:24.236 INFO:teuthology.orchestra.run.smithi094.stdout: "exists", 2024-09-14T06:31:24.236 INFO:teuthology.orchestra.run.smithi094.stdout: "up" 2024-09-14T06:31:24.236 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:31:24.236 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:24.236 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:24.236 INFO:teuthology.orchestra.run.smithi094.stdout: "osd": 6, 2024-09-14T06:31:24.236 INFO:teuthology.orchestra.run.smithi094.stdout: "uuid": "e55d11fd-433d-4462-93a7-d5e84560221b", 2024-09-14T06:31:24.236 INFO:teuthology.orchestra.run.smithi094.stdout: "up": 1, 2024-09-14T06:31:24.236 INFO:teuthology.orchestra.run.smithi094.stdout: "in": 1, 2024-09-14T06:31:24.236 INFO:teuthology.orchestra.run.smithi094.stdout: "weight": 1, 2024-09-14T06:31:24.237 INFO:teuthology.orchestra.run.smithi094.stdout: "primary_affinity": 1, 2024-09-14T06:31:24.237 INFO:teuthology.orchestra.run.smithi094.stdout: "last_clean_begin": 0, 2024-09-14T06:31:24.237 INFO:teuthology.orchestra.run.smithi094.stdout: "last_clean_end": 0, 2024-09-14T06:31:24.237 INFO:teuthology.orchestra.run.smithi094.stdout: "up_from": 15, 2024-09-14T06:31:24.237 INFO:teuthology.orchestra.run.smithi094.stdout: "up_thru": 23, 2024-09-14T06:31:24.237 INFO:teuthology.orchestra.run.smithi094.stdout: "down_at": 0, 2024-09-14T06:31:24.237 INFO:teuthology.orchestra.run.smithi094.stdout: "lost_at": 0, 2024-09-14T06:31:24.237 INFO:teuthology.orchestra.run.smithi094.stdout: "public_addrs": { 2024-09-14T06:31:24.237 INFO:teuthology.orchestra.run.smithi094.stdout: "addrvec": [ 2024-09-14T06:31:24.237 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:24.237 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v2", 2024-09-14T06:31:24.237 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.138:6818", 2024-09-14T06:31:24.237 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 163531479 2024-09-14T06:31:24.237 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:24.238 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:24.238 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v1", 2024-09-14T06:31:24.238 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.138:6819", 2024-09-14T06:31:24.238 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 163531479 2024-09-14T06:31:24.238 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:24.238 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:31:24.238 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:24.238 INFO:teuthology.orchestra.run.smithi094.stdout: "cluster_addrs": { 2024-09-14T06:31:24.238 INFO:teuthology.orchestra.run.smithi094.stdout: "addrvec": [ 2024-09-14T06:31:24.238 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:24.238 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v2", 2024-09-14T06:31:24.238 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.138:6820", 2024-09-14T06:31:24.238 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 163531479 2024-09-14T06:31:24.239 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:24.239 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:24.239 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v1", 2024-09-14T06:31:24.239 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.138:6821", 2024-09-14T06:31:24.239 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 163531479 2024-09-14T06:31:24.239 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:24.239 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:31:24.239 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:24.239 INFO:teuthology.orchestra.run.smithi094.stdout: "heartbeat_back_addrs": { 2024-09-14T06:31:24.239 INFO:teuthology.orchestra.run.smithi094.stdout: "addrvec": [ 2024-09-14T06:31:24.239 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:24.239 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v2", 2024-09-14T06:31:24.239 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.138:6824", 2024-09-14T06:31:24.239 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 163531479 2024-09-14T06:31:24.240 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:24.240 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:24.240 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v1", 2024-09-14T06:31:24.240 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.138:6825", 2024-09-14T06:31:24.240 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 163531479 2024-09-14T06:31:24.240 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:24.240 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:31:24.240 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:24.240 INFO:teuthology.orchestra.run.smithi094.stdout: "heartbeat_front_addrs": { 2024-09-14T06:31:24.240 INFO:teuthology.orchestra.run.smithi094.stdout: "addrvec": [ 2024-09-14T06:31:24.240 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:24.240 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v2", 2024-09-14T06:31:24.240 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.138:6822", 2024-09-14T06:31:24.240 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 163531479 2024-09-14T06:31:24.241 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:24.241 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:24.241 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v1", 2024-09-14T06:31:24.241 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.138:6823", 2024-09-14T06:31:24.241 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 163531479 2024-09-14T06:31:24.241 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:24.241 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:31:24.241 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:24.241 INFO:teuthology.orchestra.run.smithi094.stdout: "public_addr": "172.21.15.138:6819/163531479", 2024-09-14T06:31:24.241 INFO:teuthology.orchestra.run.smithi094.stdout: "cluster_addr": "172.21.15.138:6821/163531479", 2024-09-14T06:31:24.241 INFO:teuthology.orchestra.run.smithi094.stdout: "heartbeat_back_addr": "172.21.15.138:6825/163531479", 2024-09-14T06:31:24.241 INFO:teuthology.orchestra.run.smithi094.stdout: "heartbeat_front_addr": "172.21.15.138:6823/163531479", 2024-09-14T06:31:24.241 INFO:teuthology.orchestra.run.smithi094.stdout: "state": [ 2024-09-14T06:31:24.241 INFO:teuthology.orchestra.run.smithi094.stdout: "exists", 2024-09-14T06:31:24.242 INFO:teuthology.orchestra.run.smithi094.stdout: "up" 2024-09-14T06:31:24.242 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:31:24.242 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:24.242 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:24.242 INFO:teuthology.orchestra.run.smithi094.stdout: "osd": 7, 2024-09-14T06:31:24.242 INFO:teuthology.orchestra.run.smithi094.stdout: "uuid": "ba14ba69-2c6c-4d09-81e2-2593dc3a7ea9", 2024-09-14T06:31:24.242 INFO:teuthology.orchestra.run.smithi094.stdout: "up": 1, 2024-09-14T06:31:24.242 INFO:teuthology.orchestra.run.smithi094.stdout: "in": 1, 2024-09-14T06:31:24.242 INFO:teuthology.orchestra.run.smithi094.stdout: "weight": 1, 2024-09-14T06:31:24.242 INFO:teuthology.orchestra.run.smithi094.stdout: "primary_affinity": 1, 2024-09-14T06:31:24.242 INFO:teuthology.orchestra.run.smithi094.stdout: "last_clean_begin": 0, 2024-09-14T06:31:24.242 INFO:teuthology.orchestra.run.smithi094.stdout: "last_clean_end": 0, 2024-09-14T06:31:24.242 INFO:teuthology.orchestra.run.smithi094.stdout: "up_from": 15, 2024-09-14T06:31:24.242 INFO:teuthology.orchestra.run.smithi094.stdout: "up_thru": 23, 2024-09-14T06:31:24.243 INFO:teuthology.orchestra.run.smithi094.stdout: "down_at": 0, 2024-09-14T06:31:24.243 INFO:teuthology.orchestra.run.smithi094.stdout: "lost_at": 0, 2024-09-14T06:31:24.243 INFO:teuthology.orchestra.run.smithi094.stdout: "public_addrs": { 2024-09-14T06:31:24.243 INFO:teuthology.orchestra.run.smithi094.stdout: "addrvec": [ 2024-09-14T06:31:24.243 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:24.243 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v2", 2024-09-14T06:31:24.243 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.138:6800", 2024-09-14T06:31:24.243 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 218507235 2024-09-14T06:31:24.243 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:24.243 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:24.243 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v1", 2024-09-14T06:31:24.243 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.138:6801", 2024-09-14T06:31:24.243 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 218507235 2024-09-14T06:31:24.243 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:24.244 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:31:24.244 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:24.244 INFO:teuthology.orchestra.run.smithi094.stdout: "cluster_addrs": { 2024-09-14T06:31:24.244 INFO:teuthology.orchestra.run.smithi094.stdout: "addrvec": [ 2024-09-14T06:31:24.244 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:24.244 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v2", 2024-09-14T06:31:24.244 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.138:6802", 2024-09-14T06:31:24.244 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 218507235 2024-09-14T06:31:24.244 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:24.244 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:24.244 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v1", 2024-09-14T06:31:24.244 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.138:6803", 2024-09-14T06:31:24.244 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 218507235 2024-09-14T06:31:24.245 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:24.245 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:31:24.245 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:24.245 INFO:teuthology.orchestra.run.smithi094.stdout: "heartbeat_back_addrs": { 2024-09-14T06:31:24.245 INFO:teuthology.orchestra.run.smithi094.stdout: "addrvec": [ 2024-09-14T06:31:24.245 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:24.245 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v2", 2024-09-14T06:31:24.245 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.138:6806", 2024-09-14T06:31:24.245 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 218507235 2024-09-14T06:31:24.245 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:24.245 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:24.245 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v1", 2024-09-14T06:31:24.245 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.138:6807", 2024-09-14T06:31:24.246 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 218507235 2024-09-14T06:31:24.246 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:24.246 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:31:24.246 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:24.246 INFO:teuthology.orchestra.run.smithi094.stdout: "heartbeat_front_addrs": { 2024-09-14T06:31:24.246 INFO:teuthology.orchestra.run.smithi094.stdout: "addrvec": [ 2024-09-14T06:31:24.246 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:24.246 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v2", 2024-09-14T06:31:24.246 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.138:6804", 2024-09-14T06:31:24.246 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 218507235 2024-09-14T06:31:24.246 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:24.246 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:24.246 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v1", 2024-09-14T06:31:24.246 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.138:6805", 2024-09-14T06:31:24.247 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 218507235 2024-09-14T06:31:24.247 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:24.247 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:31:24.247 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:24.247 INFO:teuthology.orchestra.run.smithi094.stdout: "public_addr": "172.21.15.138:6801/218507235", 2024-09-14T06:31:24.247 INFO:teuthology.orchestra.run.smithi094.stdout: "cluster_addr": "172.21.15.138:6803/218507235", 2024-09-14T06:31:24.247 INFO:teuthology.orchestra.run.smithi094.stdout: "heartbeat_back_addr": "172.21.15.138:6807/218507235", 2024-09-14T06:31:24.247 INFO:teuthology.orchestra.run.smithi094.stdout: "heartbeat_front_addr": "172.21.15.138:6805/218507235", 2024-09-14T06:31:24.247 INFO:teuthology.orchestra.run.smithi094.stdout: "state": [ 2024-09-14T06:31:24.247 INFO:teuthology.orchestra.run.smithi094.stdout: "exists", 2024-09-14T06:31:24.247 INFO:teuthology.orchestra.run.smithi094.stdout: "up" 2024-09-14T06:31:24.247 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:31:24.247 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:24.248 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-09-14T06:31:24.248 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_xinfo": [ 2024-09-14T06:31:24.248 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:24.248 INFO:teuthology.orchestra.run.smithi094.stdout: "osd": 0, 2024-09-14T06:31:24.248 INFO:teuthology.orchestra.run.smithi094.stdout: "down_stamp": "0.000000", 2024-09-14T06:31:24.248 INFO:teuthology.orchestra.run.smithi094.stdout: "laggy_probability": 0, 2024-09-14T06:31:24.248 INFO:teuthology.orchestra.run.smithi094.stdout: "laggy_interval": 0, 2024-09-14T06:31:24.248 INFO:teuthology.orchestra.run.smithi094.stdout: "features": 4540719139924082687, 2024-09-14T06:31:24.248 INFO:teuthology.orchestra.run.smithi094.stdout: "old_weight": 0, 2024-09-14T06:31:24.248 INFO:teuthology.orchestra.run.smithi094.stdout: "last_purged_snaps_scrub": "2024-09-14T06:28:39.906639+0000", 2024-09-14T06:31:24.248 INFO:teuthology.orchestra.run.smithi094.stdout: "dead_epoch": 0 2024-09-14T06:31:24.248 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:24.248 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:24.249 INFO:teuthology.orchestra.run.smithi094.stdout: "osd": 1, 2024-09-14T06:31:24.249 INFO:teuthology.orchestra.run.smithi094.stdout: "down_stamp": "0.000000", 2024-09-14T06:31:24.249 INFO:teuthology.orchestra.run.smithi094.stdout: "laggy_probability": 0, 2024-09-14T06:31:24.249 INFO:teuthology.orchestra.run.smithi094.stdout: "laggy_interval": 0, 2024-09-14T06:31:24.249 INFO:teuthology.orchestra.run.smithi094.stdout: "features": 4540719139924082687, 2024-09-14T06:31:24.249 INFO:teuthology.orchestra.run.smithi094.stdout: "old_weight": 0, 2024-09-14T06:31:24.249 INFO:teuthology.orchestra.run.smithi094.stdout: "last_purged_snaps_scrub": "2024-09-14T06:28:40.541279+0000", 2024-09-14T06:31:24.249 INFO:teuthology.orchestra.run.smithi094.stdout: "dead_epoch": 0 2024-09-14T06:31:24.249 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:24.249 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:24.249 INFO:teuthology.orchestra.run.smithi094.stdout: "osd": 2, 2024-09-14T06:31:24.249 INFO:teuthology.orchestra.run.smithi094.stdout: "down_stamp": "0.000000", 2024-09-14T06:31:24.249 INFO:teuthology.orchestra.run.smithi094.stdout: "laggy_probability": 0, 2024-09-14T06:31:24.249 INFO:teuthology.orchestra.run.smithi094.stdout: "laggy_interval": 0, 2024-09-14T06:31:24.250 INFO:teuthology.orchestra.run.smithi094.stdout: "features": 4540719139924082687, 2024-09-14T06:31:24.250 INFO:teuthology.orchestra.run.smithi094.stdout: "old_weight": 0, 2024-09-14T06:31:24.250 INFO:teuthology.orchestra.run.smithi094.stdout: "last_purged_snaps_scrub": "2024-09-14T06:28:40.025350+0000", 2024-09-14T06:31:24.250 INFO:teuthology.orchestra.run.smithi094.stdout: "dead_epoch": 0 2024-09-14T06:31:24.250 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:24.250 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:24.250 INFO:teuthology.orchestra.run.smithi094.stdout: "osd": 3, 2024-09-14T06:31:24.250 INFO:teuthology.orchestra.run.smithi094.stdout: "down_stamp": "0.000000", 2024-09-14T06:31:24.250 INFO:teuthology.orchestra.run.smithi094.stdout: "laggy_probability": 0, 2024-09-14T06:31:24.250 INFO:teuthology.orchestra.run.smithi094.stdout: "laggy_interval": 0, 2024-09-14T06:31:24.250 INFO:teuthology.orchestra.run.smithi094.stdout: "features": 4540719139924082687, 2024-09-14T06:31:24.250 INFO:teuthology.orchestra.run.smithi094.stdout: "old_weight": 0, 2024-09-14T06:31:24.250 INFO:teuthology.orchestra.run.smithi094.stdout: "last_purged_snaps_scrub": "2024-09-14T06:28:39.217944+0000", 2024-09-14T06:31:24.250 INFO:teuthology.orchestra.run.smithi094.stdout: "dead_epoch": 0 2024-09-14T06:31:24.251 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:24.251 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:24.251 INFO:teuthology.orchestra.run.smithi094.stdout: "osd": 4, 2024-09-14T06:31:24.251 INFO:teuthology.orchestra.run.smithi094.stdout: "down_stamp": "0.000000", 2024-09-14T06:31:24.251 INFO:teuthology.orchestra.run.smithi094.stdout: "laggy_probability": 0, 2024-09-14T06:31:24.251 INFO:teuthology.orchestra.run.smithi094.stdout: "laggy_interval": 0, 2024-09-14T06:31:24.251 INFO:teuthology.orchestra.run.smithi094.stdout: "features": 4540719139924082687, 2024-09-14T06:31:24.251 INFO:teuthology.orchestra.run.smithi094.stdout: "old_weight": 0, 2024-09-14T06:31:24.251 INFO:teuthology.orchestra.run.smithi094.stdout: "last_purged_snaps_scrub": "2024-09-14T06:28:40.185898+0000", 2024-09-14T06:31:24.251 INFO:teuthology.orchestra.run.smithi094.stdout: "dead_epoch": 0 2024-09-14T06:31:24.251 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:24.251 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:24.251 INFO:teuthology.orchestra.run.smithi094.stdout: "osd": 5, 2024-09-14T06:31:24.252 INFO:teuthology.orchestra.run.smithi094.stdout: "down_stamp": "0.000000", 2024-09-14T06:31:24.252 INFO:teuthology.orchestra.run.smithi094.stdout: "laggy_probability": 0, 2024-09-14T06:31:24.252 INFO:teuthology.orchestra.run.smithi094.stdout: "laggy_interval": 0, 2024-09-14T06:31:24.252 INFO:teuthology.orchestra.run.smithi094.stdout: "features": 4540719139924082687, 2024-09-14T06:31:24.252 INFO:teuthology.orchestra.run.smithi094.stdout: "old_weight": 0, 2024-09-14T06:31:24.252 INFO:teuthology.orchestra.run.smithi094.stdout: "last_purged_snaps_scrub": "2024-09-14T06:28:40.783504+0000", 2024-09-14T06:31:24.252 INFO:teuthology.orchestra.run.smithi094.stdout: "dead_epoch": 0 2024-09-14T06:31:24.252 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:24.252 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:24.252 INFO:teuthology.orchestra.run.smithi094.stdout: "osd": 6, 2024-09-14T06:31:24.252 INFO:teuthology.orchestra.run.smithi094.stdout: "down_stamp": "0.000000", 2024-09-14T06:31:24.252 INFO:teuthology.orchestra.run.smithi094.stdout: "laggy_probability": 0, 2024-09-14T06:31:24.252 INFO:teuthology.orchestra.run.smithi094.stdout: "laggy_interval": 0, 2024-09-14T06:31:24.253 INFO:teuthology.orchestra.run.smithi094.stdout: "features": 4540719139924082687, 2024-09-14T06:31:24.253 INFO:teuthology.orchestra.run.smithi094.stdout: "old_weight": 0, 2024-09-14T06:31:24.253 INFO:teuthology.orchestra.run.smithi094.stdout: "last_purged_snaps_scrub": "2024-09-14T06:28:40.527305+0000", 2024-09-14T06:31:24.253 INFO:teuthology.orchestra.run.smithi094.stdout: "dead_epoch": 0 2024-09-14T06:31:24.253 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:24.253 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:24.253 INFO:teuthology.orchestra.run.smithi094.stdout: "osd": 7, 2024-09-14T06:31:24.253 INFO:teuthology.orchestra.run.smithi094.stdout: "down_stamp": "0.000000", 2024-09-14T06:31:24.253 INFO:teuthology.orchestra.run.smithi094.stdout: "laggy_probability": 0, 2024-09-14T06:31:24.253 INFO:teuthology.orchestra.run.smithi094.stdout: "laggy_interval": 0, 2024-09-14T06:31:24.253 INFO:teuthology.orchestra.run.smithi094.stdout: "features": 4540719139924082687, 2024-09-14T06:31:24.253 INFO:teuthology.orchestra.run.smithi094.stdout: "old_weight": 0, 2024-09-14T06:31:24.253 INFO:teuthology.orchestra.run.smithi094.stdout: "last_purged_snaps_scrub": "2024-09-14T06:28:40.349676+0000", 2024-09-14T06:31:24.253 INFO:teuthology.orchestra.run.smithi094.stdout: "dead_epoch": 0 2024-09-14T06:31:24.253 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:24.254 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-09-14T06:31:24.254 INFO:teuthology.orchestra.run.smithi094.stdout: "pg_upmap": [], 2024-09-14T06:31:24.254 INFO:teuthology.orchestra.run.smithi094.stdout: "pg_upmap_items": [ 2024-09-14T06:31:24.254 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:24.254 INFO:teuthology.orchestra.run.smithi094.stdout: "pgid": "3.7", 2024-09-14T06:31:24.254 INFO:teuthology.orchestra.run.smithi094.stdout: "mappings": [ 2024-09-14T06:31:24.254 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:24.254 INFO:teuthology.orchestra.run.smithi094.stdout: "from": 3, 2024-09-14T06:31:24.254 INFO:teuthology.orchestra.run.smithi094.stdout: "to": 0 2024-09-14T06:31:24.254 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:24.254 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:31:24.254 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:24.254 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-09-14T06:31:24.255 INFO:teuthology.orchestra.run.smithi094.stdout: "pg_upmap_primaries": [], 2024-09-14T06:31:24.255 INFO:teuthology.orchestra.run.smithi094.stdout: "pg_temp": [], 2024-09-14T06:31:24.255 INFO:teuthology.orchestra.run.smithi094.stdout: "primary_temp": [], 2024-09-14T06:31:24.255 INFO:teuthology.orchestra.run.smithi094.stdout: "blocklist": {}, 2024-09-14T06:31:24.255 INFO:teuthology.orchestra.run.smithi094.stdout: "range_blocklist": {}, 2024-09-14T06:31:24.255 INFO:teuthology.orchestra.run.smithi094.stdout: "erasure_code_profiles": { 2024-09-14T06:31:24.255 INFO:teuthology.orchestra.run.smithi094.stdout: "default": { 2024-09-14T06:31:24.255 INFO:teuthology.orchestra.run.smithi094.stdout: "crush-failure-domain": "osd", 2024-09-14T06:31:24.255 INFO:teuthology.orchestra.run.smithi094.stdout: "k": "2", 2024-09-14T06:31:24.255 INFO:teuthology.orchestra.run.smithi094.stdout: "m": "1", 2024-09-14T06:31:24.255 INFO:teuthology.orchestra.run.smithi094.stdout: "plugin": "jerasure", 2024-09-14T06:31:24.255 INFO:teuthology.orchestra.run.smithi094.stdout: "technique": "reed_sol_van" 2024-09-14T06:31:24.255 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:24.255 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:24.255 INFO:teuthology.orchestra.run.smithi094.stdout: "removed_snaps_queue": [ 2024-09-14T06:31:24.256 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:24.256 INFO:teuthology.orchestra.run.smithi094.stdout: "pool": 3, 2024-09-14T06:31:24.256 INFO:teuthology.orchestra.run.smithi094.stdout: "snaps": [ 2024-09-14T06:31:24.256 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:24.256 INFO:teuthology.orchestra.run.smithi094.stdout: "begin": 74, 2024-09-14T06:31:24.256 INFO:teuthology.orchestra.run.smithi094.stdout: "length": 1 2024-09-14T06:31:24.256 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:24.256 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:24.256 INFO:teuthology.orchestra.run.smithi094.stdout: "begin": 76, 2024-09-14T06:31:24.256 INFO:teuthology.orchestra.run.smithi094.stdout: "length": 1 2024-09-14T06:31:24.256 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:24.256 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:31:24.256 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:24.256 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-09-14T06:31:24.257 INFO:teuthology.orchestra.run.smithi094.stdout: "new_removed_snaps": [], 2024-09-14T06:31:24.257 INFO:teuthology.orchestra.run.smithi094.stdout: "new_purged_snaps": [], 2024-09-14T06:31:24.257 INFO:teuthology.orchestra.run.smithi094.stdout: "crush_node_flags": {}, 2024-09-14T06:31:24.257 INFO:teuthology.orchestra.run.smithi094.stdout: "device_class_flags": {}, 2024-09-14T06:31:24.257 INFO:teuthology.orchestra.run.smithi094.stdout: "stretch_mode": { 2024-09-14T06:31:24.257 INFO:teuthology.orchestra.run.smithi094.stdout: "stretch_mode_enabled": false, 2024-09-14T06:31:24.257 INFO:teuthology.orchestra.run.smithi094.stdout: "stretch_bucket_count": 0, 2024-09-14T06:31:24.257 INFO:teuthology.orchestra.run.smithi094.stdout: "degraded_stretch_mode": 0, 2024-09-14T06:31:24.257 INFO:teuthology.orchestra.run.smithi094.stdout: "recovering_stretch_mode": 0, 2024-09-14T06:31:24.257 INFO:teuthology.orchestra.run.smithi094.stdout: "stretch_mode_bucket": 0 2024-09-14T06:31:24.257 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:24.257 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-14T06:31:24.259 INFO:tasks.thrashosds.thrasher:j is {'epoch': 159, 'fsid': '200ad3da-bd63-4e04-af74-efb5ca0af75e', 'created': '2024-09-14T06:28:30.782046+0000', 'modified': '2024-09-14T06:31:23.597633+0000', 'last_up_change': '2024-09-14T06:28:42.827239+0000', 'last_in_change': '2024-09-14T06:28:33.724176+0000', 'flags': 'sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit', 'flags_num': 5799936, 'flags_set': ['pglog_hardlimit', 'purged_snapdirs', 'recovery_deletes', 'sortbitwise'], 'crush_version': 5, 'full_ratio': 0.949999988079071, 'backfillfull_ratio': 0.8999999761581421, 'nearfull_ratio': 0.8500000238418579, 'cluster_snapshot': '', 'pool_max': 3, 'max_osd': 8, 'require_min_compat_client': 'luminous', 'min_compat_client': 'luminous', 'require_osd_release': 'squid', 'allow_crimson': False, 'pools': [{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-14T06:28:43.295990+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 0.9801368713378906, 'average_primary_affinity': 0.9800000190734863, 'average_primary_affinity_weighted': 0.9800000190734863}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2024-09-14T06:28:49.451714+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 2, 'snap_epoch': 22, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.9900000095367432, 'score_stable': 1.9900000095367432, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 0.9801368713378906, 'average_primary_affinity': 0.9800000190734863, 'average_primary_affinity_weighted': 0.9800000190734863}}, {'pool': 3, 'pool_name': 'pool_client.0', 'create_time': '2024-09-14T06:28:59.688535+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '159', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 77, 'snap_epoch': 159, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 0.9801368713378906, 'average_primary_affinity': 0.9800000190734863, 'average_primary_affinity_weighted': 0.9800000190734863}}], 'osds': [{'osd': 0, 'uuid': '8b22a249-b842-406a-a02c-c0f31f57b635', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 133, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.94:6824', 'nonce': 1809564923}, {'type': 'v1', 'addr': '172.21.15.94:6825', 'nonce': 1809564923}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.94:6826', 'nonce': 1809564923}, {'type': 'v1', 'addr': '172.21.15.94:6827', 'nonce': 1809564923}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.94:6830', 'nonce': 1809564923}, {'type': 'v1', 'addr': '172.21.15.94:6831', 'nonce': 1809564923}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.94:6828', 'nonce': 1809564923}, {'type': 'v1', 'addr': '172.21.15.94:6829', 'nonce': 1809564923}]}, 'public_addr': '172.21.15.94:6825/1809564923', 'cluster_addr': '172.21.15.94:6827/1809564923', 'heartbeat_back_addr': '172.21.15.94:6831/1809564923', 'heartbeat_front_addr': '172.21.15.94:6829/1809564923', 'state': ['exists', 'up']}, {'osd': 1, 'uuid': '02251a03-84a0-414d-a980-fac4fd043cb6', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 0.841094970703125, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 132, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.94:6816', 'nonce': 736402652}, {'type': 'v1', 'addr': '172.21.15.94:6817', 'nonce': 736402652}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.94:6818', 'nonce': 736402652}, {'type': 'v1', 'addr': '172.21.15.94:6819', 'nonce': 736402652}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.94:6822', 'nonce': 736402652}, {'type': 'v1', 'addr': '172.21.15.94:6823', 'nonce': 736402652}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.94:6820', 'nonce': 736402652}, {'type': 'v1', 'addr': '172.21.15.94:6821', 'nonce': 736402652}]}, 'public_addr': '172.21.15.94:6817/736402652', 'cluster_addr': '172.21.15.94:6819/736402652', 'heartbeat_back_addr': '172.21.15.94:6823/736402652', 'heartbeat_front_addr': '172.21.15.94:6821/736402652', 'state': ['exists', 'up']}, {'osd': 2, 'uuid': 'd45bf470-bb29-4739-8294-cd122b2b4102', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 19, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.94:6808', 'nonce': 1063204782}, {'type': 'v1', 'addr': '172.21.15.94:6809', 'nonce': 1063204782}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.94:6810', 'nonce': 1063204782}, {'type': 'v1', 'addr': '172.21.15.94:6811', 'nonce': 1063204782}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.94:6814', 'nonce': 1063204782}, {'type': 'v1', 'addr': '172.21.15.94:6815', 'nonce': 1063204782}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.94:6812', 'nonce': 1063204782}, {'type': 'v1', 'addr': '172.21.15.94:6813', 'nonce': 1063204782}]}, 'public_addr': '172.21.15.94:6809/1063204782', 'cluster_addr': '172.21.15.94:6811/1063204782', 'heartbeat_back_addr': '172.21.15.94:6815/1063204782', 'heartbeat_front_addr': '172.21.15.94:6813/1063204782', 'state': ['exists', 'up']}, {'osd': 3, 'uuid': '456ec2d9-c438-4243-b931-335704f661eb', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 74, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.94:6800', 'nonce': 3382046754}, {'type': 'v1', 'addr': '172.21.15.94:6801', 'nonce': 3382046754}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.94:6802', 'nonce': 3382046754}, {'type': 'v1', 'addr': '172.21.15.94:6803', 'nonce': 3382046754}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.94:6806', 'nonce': 3382046754}, {'type': 'v1', 'addr': '172.21.15.94:6807', 'nonce': 3382046754}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.94:6804', 'nonce': 3382046754}, {'type': 'v1', 'addr': '172.21.15.94:6805', 'nonce': 3382046754}]}, 'public_addr': '172.21.15.94:6801/3382046754', 'cluster_addr': '172.21.15.94:6803/3382046754', 'heartbeat_back_addr': '172.21.15.94:6807/3382046754', 'heartbeat_front_addr': '172.21.15.94:6805/3382046754', 'state': ['exists', 'up']}, {'osd': 4, 'uuid': '2dab948e-c6e4-4437-bf17-017d1ac6821c', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 116, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.138:6810', 'nonce': 501726729}, {'type': 'v1', 'addr': '172.21.15.138:6811', 'nonce': 501726729}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.138:6812', 'nonce': 501726729}, {'type': 'v1', 'addr': '172.21.15.138:6813', 'nonce': 501726729}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.138:6816', 'nonce': 501726729}, {'type': 'v1', 'addr': '172.21.15.138:6817', 'nonce': 501726729}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.138:6814', 'nonce': 501726729}, {'type': 'v1', 'addr': '172.21.15.138:6815', 'nonce': 501726729}]}, 'public_addr': '172.21.15.138:6811/501726729', 'cluster_addr': '172.21.15.138:6813/501726729', 'heartbeat_back_addr': '172.21.15.138:6817/501726729', 'heartbeat_front_addr': '172.21.15.138:6815/501726729', 'state': ['exists', 'up']}, {'osd': 5, 'uuid': '8caf3aa3-ad87-4d33-b82e-4c15195176fd', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 23, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.138:6826', 'nonce': 1599292735}, {'type': 'v1', 'addr': '172.21.15.138:6827', 'nonce': 1599292735}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.138:6828', 'nonce': 1599292735}, {'type': 'v1', 'addr': '172.21.15.138:6829', 'nonce': 1599292735}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.138:6832', 'nonce': 1599292735}, {'type': 'v1', 'addr': '172.21.15.138:6833', 'nonce': 1599292735}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.138:6830', 'nonce': 1599292735}, {'type': 'v1', 'addr': '172.21.15.138:6831', 'nonce': 1599292735}]}, 'public_addr': '172.21.15.138:6827/1599292735', 'cluster_addr': '172.21.15.138:6829/1599292735', 'heartbeat_back_addr': '172.21.15.138:6833/1599292735', 'heartbeat_front_addr': '172.21.15.138:6831/1599292735', 'state': ['exists', 'up']}, {'osd': 6, 'uuid': 'e55d11fd-433d-4462-93a7-d5e84560221b', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 23, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.138:6818', 'nonce': 163531479}, {'type': 'v1', 'addr': '172.21.15.138:6819', 'nonce': 163531479}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.138:6820', 'nonce': 163531479}, {'type': 'v1', 'addr': '172.21.15.138:6821', 'nonce': 163531479}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.138:6824', 'nonce': 163531479}, {'type': 'v1', 'addr': '172.21.15.138:6825', 'nonce': 163531479}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.138:6822', 'nonce': 163531479}, {'type': 'v1', 'addr': '172.21.15.138:6823', 'nonce': 163531479}]}, 'public_addr': '172.21.15.138:6819/163531479', 'cluster_addr': '172.21.15.138:6821/163531479', 'heartbeat_back_addr': '172.21.15.138:6825/163531479', 'heartbeat_front_addr': '172.21.15.138:6823/163531479', 'state': ['exists', 'up']}, {'osd': 7, 'uuid': 'ba14ba69-2c6c-4d09-81e2-2593dc3a7ea9', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 23, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.138:6800', 'nonce': 218507235}, {'type': 'v1', 'addr': '172.21.15.138:6801', 'nonce': 218507235}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.138:6802', 'nonce': 218507235}, {'type': 'v1', 'addr': '172.21.15.138:6803', 'nonce': 218507235}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.138:6806', 'nonce': 218507235}, {'type': 'v1', 'addr': '172.21.15.138:6807', 'nonce': 218507235}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.138:6804', 'nonce': 218507235}, {'type': 'v1', 'addr': '172.21.15.138:6805', 'nonce': 218507235}]}, 'public_addr': '172.21.15.138:6801/218507235', 'cluster_addr': '172.21.15.138:6803/218507235', 'heartbeat_back_addr': '172.21.15.138:6807/218507235', 'heartbeat_front_addr': '172.21.15.138:6805/218507235', 'state': ['exists', 'up']}], 'osd_xinfo': [{'osd': 0, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-14T06:28:39.906639+0000', 'dead_epoch': 0}, {'osd': 1, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-14T06:28:40.541279+0000', 'dead_epoch': 0}, {'osd': 2, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-14T06:28:40.025350+0000', 'dead_epoch': 0}, {'osd': 3, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-14T06:28:39.217944+0000', 'dead_epoch': 0}, {'osd': 4, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-14T06:28:40.185898+0000', 'dead_epoch': 0}, {'osd': 5, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-14T06:28:40.783504+0000', 'dead_epoch': 0}, {'osd': 6, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-14T06:28:40.527305+0000', 'dead_epoch': 0}, {'osd': 7, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-14T06:28:40.349676+0000', 'dead_epoch': 0}], 'pg_upmap': [], 'pg_upmap_items': [{'pgid': '3.7', 'mappings': [{'from': 3, 'to': 0}]}], 'pg_upmap_primaries': [], 'pg_temp': [], 'primary_temp': [], 'blocklist': {}, 'range_blocklist': {}, 'erasure_code_profiles': {'default': {'crush-failure-domain': 'osd', 'k': '2', 'm': '1', 'plugin': 'jerasure', 'technique': 'reed_sol_van'}}, 'removed_snaps_queue': [{'pool': 3, 'snaps': [{'begin': 74, 'length': 1}, {'begin': 76, 'length': 1}]}], 'new_removed_snaps': [], 'new_purged_snaps': [], 'crush_node_flags': {}, 'device_class_flags': {}, 'stretch_mode': {'stretch_mode_enabled': False, 'stretch_bucket_count': 0, 'degraded_stretch_mode': 0, 'recovering_stretch_mode': 0, 'stretch_mode_bucket': 0}} 2024-09-14T06:31:24.260 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-14T06:31:24.261 INFO:teuthology.orchestra.run.smithi094.stdout:ERROR: (22) Invalid argument 2024-09-14T06:31:24.261 INFO:teuthology.orchestra.run.smithi094.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:31:24.261 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-14T06:31:24.262 INFO:tasks.ceph.osd.4.smithi138.stderr:2024-09-14T06:31:24.257+0000 7f6c653c8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33615) UID: 0 2024-09-14T06:31:24.359 INFO:tasks.ceph.osd.3.smithi094.stderr:2024-09-14T06:31:24.358+0000 7feeeafe8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33655) UID: 0 2024-09-14T06:31:24.408 INFO:teuthology.orchestra.run.smithi094.stdout:ERROR: (22) Invalid argument 2024-09-14T06:31:24.408 INFO:teuthology.orchestra.run.smithi094.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:31:24.419 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-14T06:31:24.460 INFO:tasks.ceph.osd.4.smithi138.stderr:2024-09-14T06:31:24.459+0000 7f6c653c8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33615) UID: 0 2024-09-14T06:31:24.561 INFO:tasks.ceph.osd.5.smithi138.stderr:2024-09-14T06:31:24.560+0000 7fe13746f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33618) UID: 0 2024-09-14T06:31:24.563 INFO:teuthology.orchestra.run.smithi094.stdout:ERROR: (22) Invalid argument 2024-09-14T06:31:24.563 INFO:teuthology.orchestra.run.smithi094.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:31:24.572 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-14T06:31:24.597 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-14T06:31:24.598 INFO:teuthology.orchestra.run.smithi094.stderr:dumped all 2024-09-14T06:31:24.606 INFO:teuthology.orchestra.run.smithi094.stderr:noscrub is set 2024-09-14T06:31:24.613 INFO:teuthology.orchestra.run.smithi094.stdout:{"pg_ready":true,"pg_map":{"version":234,"stamp":"2024-09-14T06:31:23.601974+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1793923041,"num_objects":1925,"num_object_clones":0,"num_object_copies":3850,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1925,"num_whiteouts":0,"num_read":190578,"num_read_kb":1949312,"num_write":25704,"num_write_kb":1307420,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":222,"num_bytes_recovered":31832958,"num_keys_recovered":33,"num_objects_omap":39,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":18305,"ondisk_log_size":18305,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":552372,"kb_used_data":134888,"kb_used_omap":12,"kb_used_meta":417267,"kb_avail":754422348,"statfs":{"total":773094113280,"available":772528484352,"internally_reserved":0,"allocated":138125312,"data_stored":1252034423,"data_compressed":77171514,"data_compressed_allocated":101916672,"data_compressed_original":1227988192,"omap_allocated":12751,"internal_metadata":427281969},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":33554698,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":311,"num_read_kb":3266,"num_write":88,"num_write_kb":1568,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.293516"},"pg_stats":[{"pgid":"3.1f","version":"149'609","reported_seq":6211,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-09-14T06:31:22.601847+0000","last_change":"2024-09-14T06:31:22.602153+0000","last_active":"2024-09-14T06:31:22.601847+0000","last_peered":"2024-09-14T06:31:22.601847+0000","last_clean":"2024-09-14T06:31:22.601847+0000","last_became_active":"2024-09-14T06:29:00.579883+0000","last_became_peered":"2024-09-14T06:29:00.579883+0000","last_unstale":"2024-09-14T06:31:22.601847+0000","last_undegraded":"2024-09-14T06:31:22.601847+0000","last_fullsized":"2024-09-14T06:31:22.601847+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":609,"log_dups_size":0,"ondisk_log_size":609,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T15:06:47.196622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001031077,"stat_sum":{"num_bytes":82504854,"num_objects":61,"num_object_clones":0,"num_object_copies":122,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":61,"num_whiteouts":0,"num_read":6802,"num_read_kb":75924,"num_write":915,"num_write_kb":44426,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"4a","length":"1"},{"start":"4c","length":"1"}]},{"pgid":"3.1e","version":"153'536","reported_seq":5542,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-09-14T06:31:20.609028+0000","last_change":"2024-09-14T06:31:19.072981+0000","last_active":"2024-09-14T06:31:20.609028+0000","last_peered":"2024-09-14T06:31:20.609028+0000","last_clean":"2024-09-14T06:31:20.609028+0000","last_became_active":"2024-09-14T06:29:00.580507+0000","last_became_peered":"2024-09-14T06:29:00.580507+0000","last_unstale":"2024-09-14T06:31:20.609028+0000","last_undegraded":"2024-09-14T06:31:20.609028+0000","last_fullsized":"2024-09-14T06:31:20.609028+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":536,"log_dups_size":0,"ondisk_log_size":536,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T17:42:14.798698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001092069,"stat_sum":{"num_bytes":34202102,"num_objects":49,"num_object_clones":0,"num_object_copies":98,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":49,"num_whiteouts":0,"num_read":5614,"num_read_kb":46563,"num_write":770,"num_write_kb":35203,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"4a","length":"1"}]},{"pgid":"3.1d","version":"154'693","reported_seq":6168,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-09-14T06:31:17.924284+0000","last_change":"2024-09-14T06:31:15.601274+0000","last_active":"2024-09-14T06:31:17.924284+0000","last_peered":"2024-09-14T06:31:17.924284+0000","last_clean":"2024-09-14T06:31:17.924284+0000","last_became_active":"2024-09-14T06:29:00.579345+0000","last_became_peered":"2024-09-14T06:29:00.579345+0000","last_unstale":"2024-09-14T06:31:17.924284+0000","last_undegraded":"2024-09-14T06:31:17.924284+0000","last_fullsized":"2024-09-14T06:31:17.924284+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":693,"log_dups_size":0,"ondisk_log_size":693,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T16:03:59.180241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010077560000000001,"stat_sum":{"num_bytes":77154097,"num_objects":67,"num_object_clones":0,"num_object_copies":134,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":67,"num_whiteouts":0,"num_read":6680,"num_read_kb":59800,"num_write":1041,"num_write_kb":46769,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"44","length":"1"},{"start":"46","length":"1"},{"start":"48","length":"1"}]},{"pgid":"3.1c","version":"154'450","reported_seq":3593,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-09-14T06:31:17.919274+0000","last_change":"2024-09-14T06:31:15.600905+0000","last_active":"2024-09-14T06:31:17.919274+0000","last_peered":"2024-09-14T06:31:17.919274+0000","last_clean":"2024-09-14T06:31:17.919274+0000","last_became_active":"2024-09-14T06:29:00.579807+0000","last_became_peered":"2024-09-14T06:29:00.579807+0000","last_unstale":"2024-09-14T06:31:17.919274+0000","last_undegraded":"2024-09-14T06:31:17.919274+0000","last_fullsized":"2024-09-14T06:31:17.919274+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":450,"log_dups_size":0,"ondisk_log_size":450,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T15:11:14.277370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00086512899999999996,"stat_sum":{"num_bytes":31846520,"num_objects":60,"num_object_clones":0,"num_object_copies":120,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":3442,"num_read_kb":38732,"num_write":552,"num_write_kb":40256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"44","length":"1"},{"start":"46","length":"1"},{"start":"48","length":"1"}]},{"pgid":"3.1b","version":"155'748","reported_seq":10319,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-09-14T06:31:22.601391+0000","last_change":"2024-09-14T06:31:22.601663+0000","last_active":"2024-09-14T06:31:22.601391+0000","last_peered":"2024-09-14T06:31:22.601391+0000","last_clean":"2024-09-14T06:31:22.601391+0000","last_became_active":"2024-09-14T06:29:00.586183+0000","last_became_peered":"2024-09-14T06:29:00.586183+0000","last_unstale":"2024-09-14T06:31:22.601391+0000","last_undegraded":"2024-09-14T06:31:22.601391+0000","last_fullsized":"2024-09-14T06:31:22.601391+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":748,"log_dups_size":0,"ondisk_log_size":748,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T10:31:52.456953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001070108,"stat_sum":{"num_bytes":76806245,"num_objects":74,"num_object_clones":0,"num_object_copies":148,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":74,"num_whiteouts":0,"num_read":11328,"num_read_kb":101803,"num_write":1227,"num_write_kb":49443,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"4a","length":"1"},{"start":"4c","length":"1"}]},{"pgid":"3.1a","version":"153'1072","reported_seq":7012,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-09-14T06:31:17.999523+0000","last_change":"2024-09-14T06:31:15.609610+0000","last_active":"2024-09-14T06:31:17.999523+0000","last_peered":"2024-09-14T06:31:17.999523+0000","last_clean":"2024-09-14T06:31:17.999523+0000","last_became_active":"2024-09-14T06:29:00.576206+0000","last_became_peered":"2024-09-14T06:29:00.576206+0000","last_unstale":"2024-09-14T06:31:17.999523+0000","last_undegraded":"2024-09-14T06:31:17.999523+0000","last_fullsized":"2024-09-14T06:31:17.999523+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":1072,"log_dups_size":0,"ondisk_log_size":1072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T09:56:08.827823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013136140000000001,"stat_sum":{"num_bytes":57404040,"num_objects":65,"num_object_clones":0,"num_object_copies":130,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":65,"num_whiteouts":0,"num_read":9006,"num_read_kb":65717,"num_write":1518,"num_write_kb":48456,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"44","length":"1"},{"start":"46","length":"1"},{"start":"48","length":"1"}]},{"pgid":"3.19","version":"151'370","reported_seq":5013,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-09-14T06:31:17.909043+0000","last_change":"2024-09-14T06:31:15.596418+0000","last_active":"2024-09-14T06:31:17.909043+0000","last_peered":"2024-09-14T06:31:17.909043+0000","last_clean":"2024-09-14T06:31:17.909043+0000","last_became_active":"2024-09-14T06:29:00.575639+0000","last_became_peered":"2024-09-14T06:29:00.575639+0000","last_unstale":"2024-09-14T06:31:17.909043+0000","last_undegraded":"2024-09-14T06:31:17.909043+0000","last_fullsized":"2024-09-14T06:31:17.909043+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":370,"log_dups_size":0,"ondisk_log_size":370,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T11:36:16.701495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062530799999999998,"stat_sum":{"num_bytes":56438659,"num_objects":54,"num_object_clones":0,"num_object_copies":108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":54,"num_whiteouts":0,"num_read":4698,"num_read_kb":65322,"num_write":509,"num_write_kb":40148,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"44","length":"1"},{"start":"46","length":"1"},{"start":"48","length":"1"}]},{"pgid":"3.18","version":"152'329","reported_seq":4855,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-09-14T06:31:20.608665+0000","last_change":"2024-09-14T06:31:19.080462+0000","last_active":"2024-09-14T06:31:20.608665+0000","last_peered":"2024-09-14T06:31:20.608665+0000","last_clean":"2024-09-14T06:31:20.608665+0000","last_became_active":"2024-09-14T06:29:00.578017+0000","last_became_peered":"2024-09-14T06:29:00.578017+0000","last_unstale":"2024-09-14T06:31:20.608665+0000","last_undegraded":"2024-09-14T06:31:20.608665+0000","last_fullsized":"2024-09-14T06:31:20.608665+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":329,"log_dups_size":0,"ondisk_log_size":329,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T12:20:24.394820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.003224401,"stat_sum":{"num_bytes":59946747,"num_objects":60,"num_object_clones":0,"num_object_copies":120,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":4065,"num_read_kb":73031,"num_write":414,"num_write_kb":44190,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"4a","length":"1"}]},{"pgid":"3.17","version":"155'595","reported_seq":7619,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-09-14T06:31:22.602208+0000","last_change":"2024-09-14T06:31:22.602431+0000","last_active":"2024-09-14T06:31:22.602208+0000","last_peered":"2024-09-14T06:31:22.602208+0000","last_clean":"2024-09-14T06:31:22.602208+0000","last_became_active":"2024-09-14T06:29:00.579714+0000","last_became_peered":"2024-09-14T06:29:00.579714+0000","last_unstale":"2024-09-14T06:31:22.602208+0000","last_undegraded":"2024-09-14T06:31:22.602208+0000","last_fullsized":"2024-09-14T06:31:22.602208+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":595,"log_dups_size":0,"ondisk_log_size":595,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T10:16:42.634710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010676209999999999,"stat_sum":{"num_bytes":76829434,"num_objects":73,"num_object_clones":0,"num_object_copies":146,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":73,"num_whiteouts":0,"num_read":7800,"num_read_kb":63533,"num_write":716,"num_write_kb":42748,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"4a","length":"1"},{"start":"4c","length":"1"}]},{"pgid":"3.16","version":"154'714","reported_seq":2969,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-09-14T06:31:17.976241+0000","last_change":"2024-09-14T06:31:15.601445+0000","last_active":"2024-09-14T06:31:17.976241+0000","last_peered":"2024-09-14T06:31:17.976241+0000","last_clean":"2024-09-14T06:31:17.976241+0000","last_became_active":"2024-09-14T06:29:00.580070+0000","last_became_peered":"2024-09-14T06:29:00.580070+0000","last_unstale":"2024-09-14T06:31:17.976241+0000","last_undegraded":"2024-09-14T06:31:17.976241+0000","last_fullsized":"2024-09-14T06:31:17.976241+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":714,"log_dups_size":0,"ondisk_log_size":714,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T18:17:51.006344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00158461,"stat_sum":{"num_bytes":41312662,"num_objects":62,"num_object_clones":0,"num_object_copies":124,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":62,"num_whiteouts":0,"num_read":4167,"num_read_kb":36270,"num_write":1101,"num_write_kb":31790,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"44","length":"1"},{"start":"46","length":"1"},{"start":"48","length":"1"}]},{"pgid":"3.15","version":"154'349","reported_seq":3728,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-09-14T06:31:17.944699+0000","last_change":"2024-09-14T06:31:15.603502+0000","last_active":"2024-09-14T06:31:17.944699+0000","last_peered":"2024-09-14T06:31:17.944699+0000","last_clean":"2024-09-14T06:31:17.944699+0000","last_became_active":"2024-09-14T06:29:00.578451+0000","last_became_peered":"2024-09-14T06:29:00.578451+0000","last_unstale":"2024-09-14T06:31:17.944699+0000","last_undegraded":"2024-09-14T06:31:17.944699+0000","last_fullsized":"2024-09-14T06:31:17.944699+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":349,"log_dups_size":0,"ondisk_log_size":349,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T09:42:46.072410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000662259,"stat_sum":{"num_bytes":62493863,"num_objects":58,"num_object_clones":0,"num_object_copies":116,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":58,"num_whiteouts":0,"num_read":3036,"num_read_kb":63915,"num_write":398,"num_write_kb":46791,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"44","length":"1"},{"start":"46","length":"1"},{"start":"48","length":"1"}]},{"pgid":"3.14","version":"152'876","reported_seq":5077,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-09-14T06:31:17.910157+0000","last_change":"2024-09-14T06:31:15.609790+0000","last_active":"2024-09-14T06:31:17.910157+0000","last_peered":"2024-09-14T06:31:17.910157+0000","last_clean":"2024-09-14T06:31:17.910157+0000","last_became_active":"2024-09-14T06:29:00.582595+0000","last_became_peered":"2024-09-14T06:29:00.582595+0000","last_unstale":"2024-09-14T06:31:17.910157+0000","last_undegraded":"2024-09-14T06:31:17.910157+0000","last_fullsized":"2024-09-14T06:31:17.910157+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":876,"log_dups_size":0,"ondisk_log_size":876,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T07:35:05.941636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001073772,"stat_sum":{"num_bytes":50793843,"num_objects":47,"num_object_clones":0,"num_object_copies":94,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":47,"num_whiteouts":0,"num_read":8176,"num_read_kb":40500,"num_write":1285,"num_write_kb":22321,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"44","length":"1"},{"start":"46","length":"1"},{"start":"48","length":"1"}]},{"pgid":"2.7","version":"0'0","reported_seq":283,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-09-14T06:31:17.868225+0000","last_change":"2024-09-14T06:28:53.625193+0000","last_active":"2024-09-14T06:31:17.868225+0000","last_peered":"2024-09-14T06:31:17.868225+0000","last_clean":"2024-09-14T06:31:17.868225+0000","last_became_active":"2024-09-14T06:28:51.223535+0000","last_became_peered":"2024-09-14T06:28:51.223535+0000","last_unstale":"2024-09-14T06:31:17.868225+0000","last_undegraded":"2024-09-14T06:31:17.868225+0000","last_fullsized":"2024-09-14T06:31:17.868225+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:28:50.205350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:28:50.205350+0000","last_clean_scrub_stamp":"2024-09-14T06:28:50.205350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T18:14:22.822245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00052660300000000005,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"158'404","reported_seq":5544,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-09-14T06:31:22.701167+0000","last_change":"2024-09-14T06:31:22.601648+0000","last_active":"2024-09-14T06:31:22.701167+0000","last_peered":"2024-09-14T06:31:22.701167+0000","last_clean":"2024-09-14T06:31:22.701167+0000","last_became_active":"2024-09-14T06:29:00.578283+0000","last_became_peered":"2024-09-14T06:29:00.578283+0000","last_unstale":"2024-09-14T06:31:22.701167+0000","last_undegraded":"2024-09-14T06:31:22.701167+0000","last_fullsized":"2024-09-14T06:31:22.701167+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":404,"log_dups_size":0,"ondisk_log_size":404,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T17:25:35.913208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011820310000000001,"stat_sum":{"num_bytes":57610392,"num_objects":53,"num_object_clones":0,"num_object_copies":106,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":53,"num_whiteouts":0,"num_read":4985,"num_read_kb":67214,"num_write":501,"num_write_kb":43730,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"4a","length":"1"},{"start":"4c","length":"1"}]},{"pgid":"2.6","version":"0'0","reported_seq":285,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-09-14T06:31:17.856921+0000","last_change":"2024-09-14T06:28:53.226122+0000","last_active":"2024-09-14T06:31:17.856921+0000","last_peered":"2024-09-14T06:31:17.856921+0000","last_clean":"2024-09-14T06:31:17.856921+0000","last_became_active":"2024-09-14T06:28:51.217205+0000","last_became_peered":"2024-09-14T06:28:51.217205+0000","last_unstale":"2024-09-14T06:31:17.856921+0000","last_undegraded":"2024-09-14T06:31:17.856921+0000","last_fullsized":"2024-09-14T06:31:17.856921+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:28:50.205350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:28:50.205350+0000","last_clean_scrub_stamp":"2024-09-14T06:28:50.205350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T15:29:19.563302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065018200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"155'593","reported_seq":3492,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-09-14T06:31:22.602081+0000","last_change":"2024-09-14T06:31:22.602081+0000","last_active":"2024-09-14T06:31:22.602081+0000","last_peered":"2024-09-14T06:31:22.602081+0000","last_clean":"2024-09-14T06:31:22.602081+0000","last_became_active":"2024-09-14T06:29:56.584268+0000","last_became_peered":"2024-09-14T06:29:56.584268+0000","last_unstale":"2024-09-14T06:31:22.602081+0000","last_undegraded":"2024-09-14T06:31:22.602081+0000","last_fullsized":"2024-09-14T06:31:22.602081+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":593,"log_dups_size":0,"ondisk_log_size":593,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T11:27:51.987194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013356119999999999,"stat_sum":{"num_bytes":37624111,"num_objects":47,"num_object_clones":0,"num_object_copies":94,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":47,"num_whiteouts":0,"num_read":3760,"num_read_kb":34721,"num_write":859,"num_write_kb":30984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":39,"num_bytes_recovered":1400964,"num_keys_recovered":11,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"4a","length":"1"},{"start":"4c","length":"1"}]},{"pgid":"2.5","version":"0'0","reported_seq":285,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-09-14T06:31:17.863822+0000","last_change":"2024-09-14T06:28:53.640035+0000","last_active":"2024-09-14T06:31:17.863822+0000","last_peered":"2024-09-14T06:31:17.863822+0000","last_clean":"2024-09-14T06:31:17.863822+0000","last_became_active":"2024-09-14T06:28:51.308058+0000","last_became_peered":"2024-09-14T06:28:51.308058+0000","last_unstale":"2024-09-14T06:31:17.863822+0000","last_undegraded":"2024-09-14T06:31:17.863822+0000","last_fullsized":"2024-09-14T06:31:17.863822+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:28:50.205350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:28:50.205350+0000","last_clean_scrub_stamp":"2024-09-14T06:28:50.205350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T13:35:48.544057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00037824000000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"154'312","reported_seq":2026,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-09-14T06:31:17.928995+0000","last_change":"2024-09-14T06:31:15.596414+0000","last_active":"2024-09-14T06:31:17.928995+0000","last_peered":"2024-09-14T06:31:17.928995+0000","last_clean":"2024-09-14T06:31:17.928995+0000","last_became_active":"2024-09-14T06:29:00.574895+0000","last_became_peered":"2024-09-14T06:29:00.574895+0000","last_unstale":"2024-09-14T06:31:17.928995+0000","last_undegraded":"2024-09-14T06:31:17.928995+0000","last_fullsized":"2024-09-14T06:31:17.928995+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":312,"log_dups_size":0,"ondisk_log_size":312,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T14:00:54.538920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00060380999999999998,"stat_sum":{"num_bytes":24020525,"num_objects":55,"num_object_clones":0,"num_object_copies":110,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":55,"num_whiteouts":0,"num_read":1353,"num_read_kb":31917,"num_write":400,"num_write_kb":33281,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"44","length":"1"},{"start":"46","length":"1"},{"start":"48","length":"1"}]},{"pgid":"2.4","version":"0'0","reported_seq":285,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-09-14T06:31:17.856581+0000","last_change":"2024-09-14T06:28:53.226072+0000","last_active":"2024-09-14T06:31:17.856581+0000","last_peered":"2024-09-14T06:31:17.856581+0000","last_clean":"2024-09-14T06:31:17.856581+0000","last_became_active":"2024-09-14T06:28:51.308072+0000","last_became_peered":"2024-09-14T06:28:51.308072+0000","last_unstale":"2024-09-14T06:31:17.856581+0000","last_undegraded":"2024-09-14T06:31:17.856581+0000","last_fullsized":"2024-09-14T06:31:17.856581+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:28:50.205350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:28:50.205350+0000","last_clean_scrub_stamp":"2024-09-14T06:28:50.205350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T13:15:09.052328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066197399999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"149'674","reported_seq":2902,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-09-14T06:31:17.878428+0000","last_change":"2024-09-14T06:31:15.600659+0000","last_active":"2024-09-14T06:31:17.878428+0000","last_peered":"2024-09-14T06:31:17.878428+0000","last_clean":"2024-09-14T06:31:17.878428+0000","last_became_active":"2024-09-14T06:29:00.581874+0000","last_became_peered":"2024-09-14T06:29:00.581874+0000","last_unstale":"2024-09-14T06:31:17.878428+0000","last_undegraded":"2024-09-14T06:31:17.878428+0000","last_fullsized":"2024-09-14T06:31:17.878428+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":674,"log_dups_size":0,"ondisk_log_size":674,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T14:38:35.992336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064949500000000004,"stat_sum":{"num_bytes":34515588,"num_objects":63,"num_object_clones":0,"num_object_copies":126,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":63,"num_whiteouts":0,"num_read":3149,"num_read_kb":32535,"num_write":1001,"num_write_kb":44224,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"44","length":"1"},{"start":"46","length":"1"},{"start":"48","length":"1"}]},{"pgid":"3.3","version":"151'347","reported_seq":9529,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-09-14T06:31:17.912022+0000","last_change":"2024-09-14T06:31:15.609707+0000","last_active":"2024-09-14T06:31:17.912022+0000","last_peered":"2024-09-14T06:31:17.912022+0000","last_clean":"2024-09-14T06:31:17.912022+0000","last_became_active":"2024-09-14T06:29:00.584678+0000","last_became_peered":"2024-09-14T06:29:00.584678+0000","last_unstale":"2024-09-14T06:31:17.912022+0000","last_undegraded":"2024-09-14T06:31:17.912022+0000","last_fullsized":"2024-09-14T06:31:17.912022+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":347,"log_dups_size":0,"ondisk_log_size":347,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T07:15:12.937143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001325379,"stat_sum":{"num_bytes":62215563,"num_objects":49,"num_object_clones":0,"num_object_copies":98,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":49,"num_whiteouts":0,"num_read":8902,"num_read_kb":85274,"num_write":390,"num_write_kb":46535,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"44","length":"1"},{"start":"46","length":"1"},{"start":"48","length":"1"}]},{"pgid":"2.2","version":"22'2","reported_seq":287,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-09-14T06:31:17.876908+0000","last_change":"2024-09-14T06:28:53.231428+0000","last_active":"2024-09-14T06:31:17.876908+0000","last_peered":"2024-09-14T06:31:17.876908+0000","last_clean":"2024-09-14T06:31:17.876908+0000","last_became_active":"2024-09-14T06:28:51.222937+0000","last_became_peered":"2024-09-14T06:28:51.222937+0000","last_unstale":"2024-09-14T06:31:17.876908+0000","last_undegraded":"2024-09-14T06:31:17.876908+0000","last_fullsized":"2024-09-14T06:31:17.876908+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:28:50.205350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:28:50.205350+0000","last_clean_scrub_stamp":"2024-09-14T06:28:50.205350+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T14:27:37.233976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010072970000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.0","version":"154'830","reported_seq":6072,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-09-14T06:31:18.013050+0000","last_change":"2024-09-14T06:31:15.596872+0000","last_active":"2024-09-14T06:31:18.013050+0000","last_peered":"2024-09-14T06:31:18.013050+0000","last_clean":"2024-09-14T06:31:18.013050+0000","last_became_active":"2024-09-14T06:29:00.574873+0000","last_became_peered":"2024-09-14T06:29:00.574873+0000","last_unstale":"2024-09-14T06:31:18.013050+0000","last_undegraded":"2024-09-14T06:31:18.013050+0000","last_fullsized":"2024-09-14T06:31:18.013050+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":830,"log_dups_size":0,"ondisk_log_size":830,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T08:51:34.995886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00083719000000000003,"stat_sum":{"num_bytes":73791470,"num_objects":71,"num_object_clones":0,"num_object_copies":142,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":71,"num_whiteouts":0,"num_read":7478,"num_read_kb":81770,"num_write":1305,"num_write_kb":55510,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"44","length":"1"},{"start":"46","length":"1"},{"start":"48","length":"1"}]},{"pgid":"2.1","version":"0'0","reported_seq":289,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-14T06:31:21.597779+0000","last_change":"2024-09-14T06:28:53.226264+0000","last_active":"2024-09-14T06:31:21.597779+0000","last_peered":"2024-09-14T06:31:21.597779+0000","last_clean":"2024-09-14T06:31:21.597779+0000","last_became_active":"2024-09-14T06:28:51.223252+0000","last_became_peered":"2024-09-14T06:28:51.223252+0000","last_unstale":"2024-09-14T06:31:21.597779+0000","last_undegraded":"2024-09-14T06:31:21.597779+0000","last_fullsized":"2024-09-14T06:31:21.597779+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:28:50.205350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:28:50.205350+0000","last_clean_scrub_stamp":"2024-09-14T06:28:50.205350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T11:56:04.373414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053403100000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1","version":"152'605","reported_seq":10936,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-09-14T06:31:22.600763+0000","last_change":"2024-09-14T06:31:22.601041+0000","last_active":"2024-09-14T06:31:22.600763+0000","last_peered":"2024-09-14T06:31:22.600763+0000","last_clean":"2024-09-14T06:31:22.600763+0000","last_became_active":"2024-09-14T06:30:57.768238+0000","last_became_peered":"2024-09-14T06:30:57.768238+0000","last_unstale":"2024-09-14T06:31:22.600763+0000","last_undegraded":"2024-09-14T06:31:22.600763+0000","last_fullsized":"2024-09-14T06:31:22.600763+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":605,"log_dups_size":0,"ondisk_log_size":605,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T17:42:16.917013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010181050000000001,"stat_sum":{"num_bytes":93187966,"num_objects":61,"num_object_clones":0,"num_object_copies":122,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":61,"num_whiteouts":0,"num_read":10907,"num_read_kb":111631,"num_write":775,"num_write_kb":44188,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":183,"num_bytes_recovered":30431994,"num_keys_recovered":22,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"4a","length":"1"},{"start":"4c","length":"1"}]},{"pgid":"2.0","version":"0'0","reported_seq":285,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-09-14T06:31:17.863420+0000","last_change":"2024-09-14T06:28:53.640096+0000","last_active":"2024-09-14T06:31:17.863420+0000","last_peered":"2024-09-14T06:31:17.863420+0000","last_clean":"2024-09-14T06:31:17.863420+0000","last_became_active":"2024-09-14T06:28:51.223069+0000","last_became_peered":"2024-09-14T06:28:51.223069+0000","last_unstale":"2024-09-14T06:31:17.863420+0000","last_undegraded":"2024-09-14T06:31:17.863420+0000","last_fullsized":"2024-09-14T06:31:17.863420+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:28:50.205350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:28:50.205350+0000","last_clean_scrub_stamp":"2024-09-14T06:28:50.205350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T15:13:00.471509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00043714600000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"155'576","reported_seq":5537,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-09-14T06:31:20.608713+0000","last_change":"2024-09-14T06:31:19.076395+0000","last_active":"2024-09-14T06:31:20.608713+0000","last_peered":"2024-09-14T06:31:20.608713+0000","last_clean":"2024-09-14T06:31:20.608713+0000","last_became_active":"2024-09-14T06:29:00.578574+0000","last_became_peered":"2024-09-14T06:29:00.578574+0000","last_unstale":"2024-09-14T06:31:20.608713+0000","last_undegraded":"2024-09-14T06:31:20.608713+0000","last_fullsized":"2024-09-14T06:31:20.608713+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":576,"log_dups_size":0,"ondisk_log_size":576,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T12:13:01.429670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0045060960000000002,"stat_sum":{"num_bytes":66318026,"num_objects":76,"num_object_clones":0,"num_object_copies":152,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":76,"num_whiteouts":0,"num_read":5380,"num_read_kb":64719,"num_write":730,"num_write_kb":48935,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"4a","length":"1"}]},{"pgid":"2.3","version":"20'1","reported_seq":286,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-09-14T06:31:17.876884+0000","last_change":"2024-09-14T06:28:53.230933+0000","last_active":"2024-09-14T06:31:17.876884+0000","last_peered":"2024-09-14T06:31:17.876884+0000","last_clean":"2024-09-14T06:31:17.876884+0000","last_became_active":"2024-09-14T06:28:51.222874+0000","last_became_peered":"2024-09-14T06:28:51.222874+0000","last_unstale":"2024-09-14T06:31:17.876884+0000","last_undegraded":"2024-09-14T06:31:17.876884+0000","last_fullsized":"2024-09-14T06:31:17.876884+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:28:50.205350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:28:50.205350+0000","last_clean_scrub_stamp":"2024-09-14T06:28:50.205350+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T12:04:25.190171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050808899999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'72","reported_seq":376,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-09-14T06:31:17.863433+0000","last_change":"2024-09-14T06:28:44.839885+0000","last_active":"2024-09-14T06:31:17.863433+0000","last_peered":"2024-09-14T06:31:17.863433+0000","last_clean":"2024-09-14T06:31:17.863433+0000","last_became_active":"2024-09-14T06:28:44.839573+0000","last_became_peered":"2024-09-14T06:28:44.839573+0000","last_unstale":"2024-09-14T06:31:17.863433+0000","last_undegraded":"2024-09-14T06:31:17.863433+0000","last_fullsized":"2024-09-14T06:31:17.863433+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:28:43.828494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:28:43.828494+0000","last_clean_scrub_stamp":"2024-09-14T06:28:43.828494+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T14:41:25.787293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.8","version":"155'472","reported_seq":4861,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-09-14T06:31:20.608904+0000","last_change":"2024-09-14T06:31:19.081012+0000","last_active":"2024-09-14T06:31:20.608904+0000","last_peered":"2024-09-14T06:31:20.608904+0000","last_clean":"2024-09-14T06:31:20.608904+0000","last_became_active":"2024-09-14T06:29:00.574458+0000","last_became_peered":"2024-09-14T06:29:00.574458+0000","last_unstale":"2024-09-14T06:31:20.608904+0000","last_undegraded":"2024-09-14T06:31:20.608904+0000","last_fullsized":"2024-09-14T06:31:20.608904+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":472,"log_dups_size":0,"ondisk_log_size":472,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T13:41:50.322454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00338033,"stat_sum":{"num_bytes":77868079,"num_objects":66,"num_object_clones":0,"num_object_copies":132,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":66,"num_whiteouts":0,"num_read":4966,"num_read_kb":81424,"num_write":679,"num_write_kb":51396,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"4a","length":"1"}]},{"pgid":"3.9","version":"153'409","reported_seq":3484,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-09-14T06:31:17.994596+0000","last_change":"2024-09-14T06:31:15.608777+0000","last_active":"2024-09-14T06:31:17.994596+0000","last_peered":"2024-09-14T06:31:17.994596+0000","last_clean":"2024-09-14T06:31:17.994596+0000","last_became_active":"2024-09-14T06:29:00.573179+0000","last_became_peered":"2024-09-14T06:29:00.573179+0000","last_unstale":"2024-09-14T06:31:17.994596+0000","last_undegraded":"2024-09-14T06:31:17.994596+0000","last_fullsized":"2024-09-14T06:31:17.994596+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":409,"log_dups_size":0,"ondisk_log_size":409,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T12:21:18.058117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050635299999999997,"stat_sum":{"num_bytes":42020611,"num_objects":63,"num_object_clones":0,"num_object_copies":126,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":63,"num_whiteouts":0,"num_read":3356,"num_read_kb":58575,"num_write":589,"num_write_kb":39949,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"44","length":"1"},{"start":"46","length":"1"},{"start":"48","length":"1"}]},{"pgid":"3.a","version":"154'669","reported_seq":3674,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-09-14T06:31:18.015606+0000","last_change":"2024-09-14T06:31:15.596553+0000","last_active":"2024-09-14T06:31:18.015606+0000","last_peered":"2024-09-14T06:31:18.015606+0000","last_clean":"2024-09-14T06:31:18.015606+0000","last_became_active":"2024-09-14T06:29:00.582426+0000","last_became_peered":"2024-09-14T06:29:00.582426+0000","last_unstale":"2024-09-14T06:31:18.015606+0000","last_undegraded":"2024-09-14T06:31:18.015606+0000","last_fullsized":"2024-09-14T06:31:18.015606+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":669,"log_dups_size":0,"ondisk_log_size":669,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T13:31:01.385338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00048087800000000001,"stat_sum":{"num_bytes":45746635,"num_objects":61,"num_object_clones":0,"num_object_copies":122,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":61,"num_whiteouts":0,"num_read":4652,"num_read_kb":55407,"num_write":1029,"num_write_kb":43974,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"44","length":"1"},{"start":"46","length":"1"},{"start":"48","length":"1"}]},{"pgid":"3.b","version":"151'1308","reported_seq":7319,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-09-14T06:31:20.608715+0000","last_change":"2024-09-14T06:31:19.077022+0000","last_active":"2024-09-14T06:31:20.608715+0000","last_peered":"2024-09-14T06:31:20.608715+0000","last_clean":"2024-09-14T06:31:20.608715+0000","last_became_active":"2024-09-14T06:29:00.578483+0000","last_became_peered":"2024-09-14T06:29:00.578483+0000","last_unstale":"2024-09-14T06:31:20.608715+0000","last_undegraded":"2024-09-14T06:31:20.608715+0000","last_fullsized":"2024-09-14T06:31:20.608715+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":1308,"log_dups_size":0,"ondisk_log_size":1308,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T11:50:58.221075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0043554509999999998,"stat_sum":{"num_bytes":51970960,"num_objects":68,"num_object_clones":0,"num_object_copies":136,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":68,"num_whiteouts":0,"num_read":11802,"num_read_kb":60397,"num_write":2089,"num_write_kb":39968,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"4a","length":"1"}]},{"pgid":"3.c","version":"154'369","reported_seq":4338,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-09-14T06:31:17.917055+0000","last_change":"2024-09-14T06:31:15.600458+0000","last_active":"2024-09-14T06:31:17.917055+0000","last_peered":"2024-09-14T06:31:17.917055+0000","last_clean":"2024-09-14T06:31:17.917055+0000","last_became_active":"2024-09-14T06:29:00.583406+0000","last_became_peered":"2024-09-14T06:29:00.583406+0000","last_unstale":"2024-09-14T06:31:17.917055+0000","last_undegraded":"2024-09-14T06:31:17.917055+0000","last_fullsized":"2024-09-14T06:31:17.917055+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":369,"log_dups_size":0,"ondisk_log_size":369,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T08:45:50.489265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059378299999999996,"stat_sum":{"num_bytes":36668308,"num_objects":50,"num_object_clones":0,"num_object_copies":100,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":50,"num_whiteouts":0,"num_read":3950,"num_read_kb":43260,"num_write":522,"num_write_kb":34429,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"44","length":"1"},{"start":"46","length":"1"},{"start":"48","length":"1"}]},{"pgid":"3.d","version":"154'337","reported_seq":6943,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-09-14T06:31:17.952011+0000","last_change":"2024-09-14T06:31:15.603342+0000","last_active":"2024-09-14T06:31:17.952011+0000","last_peered":"2024-09-14T06:31:17.952011+0000","last_clean":"2024-09-14T06:31:17.952011+0000","last_became_active":"2024-09-14T06:29:00.580362+0000","last_became_peered":"2024-09-14T06:29:00.580362+0000","last_unstale":"2024-09-14T06:31:17.952011+0000","last_undegraded":"2024-09-14T06:31:17.952011+0000","last_fullsized":"2024-09-14T06:31:17.952011+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":337,"log_dups_size":0,"ondisk_log_size":337,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T07:29:52.788818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051650399999999999,"stat_sum":{"num_bytes":78172498,"num_objects":60,"num_object_clones":0,"num_object_copies":120,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":6512,"num_read_kb":81017,"num_write":467,"num_write_kb":34687,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"44","length":"1"},{"start":"46","length":"1"},{"start":"48","length":"1"}]},{"pgid":"3.e","version":"154'379","reported_seq":3007,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-09-14T06:31:17.983977+0000","last_change":"2024-09-14T06:31:15.603189+0000","last_active":"2024-09-14T06:31:17.983977+0000","last_peered":"2024-09-14T06:31:17.983977+0000","last_clean":"2024-09-14T06:31:17.983977+0000","last_became_active":"2024-09-14T06:29:00.586374+0000","last_became_peered":"2024-09-14T06:29:00.586374+0000","last_unstale":"2024-09-14T06:31:17.983977+0000","last_undegraded":"2024-09-14T06:31:17.983977+0000","last_fullsized":"2024-09-14T06:31:17.983977+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":379,"log_dups_size":0,"ondisk_log_size":379,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T11:07:17.334928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067055100000000002,"stat_sum":{"num_bytes":41634942,"num_objects":59,"num_object_clones":0,"num_object_copies":118,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":59,"num_whiteouts":0,"num_read":2530,"num_read_kb":50108,"num_write":538,"num_write_kb":38654,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"44","length":"1"},{"start":"46","length":"1"},{"start":"48","length":"1"}]},{"pgid":"3.f","version":"152'399","reported_seq":4398,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-09-14T06:31:17.949405+0000","last_change":"2024-09-14T06:31:15.603181+0000","last_active":"2024-09-14T06:31:17.949405+0000","last_peered":"2024-09-14T06:31:17.949405+0000","last_clean":"2024-09-14T06:31:17.949405+0000","last_became_active":"2024-09-14T06:29:00.586491+0000","last_became_peered":"2024-09-14T06:29:00.586491+0000","last_unstale":"2024-09-14T06:31:17.949405+0000","last_undegraded":"2024-09-14T06:31:17.949405+0000","last_fullsized":"2024-09-14T06:31:17.949405+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":399,"log_dups_size":0,"ondisk_log_size":399,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T13:49:43.149522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00068408900000000003,"stat_sum":{"num_bytes":29256498,"num_objects":44,"num_object_clones":0,"num_object_copies":88,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":44,"num_whiteouts":0,"num_read":4721,"num_read_kb":29608,"num_write":496,"num_write_kb":24340,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"44","length":"1"},{"start":"46","length":"1"},{"start":"48","length":"1"}]},{"pgid":"3.10","version":"151'453","reported_seq":4632,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-09-14T06:31:17.914034+0000","last_change":"2024-09-14T06:31:15.596735+0000","last_active":"2024-09-14T06:31:17.914034+0000","last_peered":"2024-09-14T06:31:17.914034+0000","last_clean":"2024-09-14T06:31:17.914034+0000","last_became_active":"2024-09-14T06:29:00.579539+0000","last_became_peered":"2024-09-14T06:29:00.579539+0000","last_unstale":"2024-09-14T06:31:17.914034+0000","last_undegraded":"2024-09-14T06:31:17.914034+0000","last_fullsized":"2024-09-14T06:31:17.914034+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":453,"log_dups_size":0,"ondisk_log_size":453,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T07:22:10.234397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065979600000000004,"stat_sum":{"num_bytes":36814772,"num_objects":54,"num_object_clones":0,"num_object_copies":108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":54,"num_whiteouts":0,"num_read":4455,"num_read_kb":30142,"num_write":524,"num_write_kb":32511,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"44","length":"1"},{"start":"46","length":"1"},{"start":"48","length":"1"}]},{"pgid":"3.11","version":"154'499","reported_seq":9161,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-09-14T06:31:17.941798+0000","last_change":"2024-09-14T06:31:15.602890+0000","last_active":"2024-09-14T06:31:17.941798+0000","last_peered":"2024-09-14T06:31:17.941798+0000","last_clean":"2024-09-14T06:31:17.941798+0000","last_became_active":"2024-09-14T06:29:00.586480+0000","last_became_peered":"2024-09-14T06:29:00.586480+0000","last_unstale":"2024-09-14T06:31:17.941798+0000","last_undegraded":"2024-09-14T06:31:17.941798+0000","last_fullsized":"2024-09-14T06:31:17.941798+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":499,"log_dups_size":0,"ondisk_log_size":499,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T16:38:54.163326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000464823,"stat_sum":{"num_bytes":88437497,"num_objects":70,"num_object_clones":0,"num_object_copies":140,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":70,"num_whiteouts":0,"num_read":8760,"num_read_kb":99409,"num_write":621,"num_write_kb":43940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"44","length":"1"},{"start":"46","length":"1"},{"start":"48","length":"1"}]},{"pgid":"3.12","version":"152'586","reported_seq":6468,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-09-14T06:31:22.771601+0000","last_change":"2024-09-14T06:31:22.600459+0000","last_active":"2024-09-14T06:31:22.771601+0000","last_peered":"2024-09-14T06:31:22.771601+0000","last_clean":"2024-09-14T06:31:22.771601+0000","last_became_active":"2024-09-14T06:29:00.579826+0000","last_became_peered":"2024-09-14T06:29:00.579826+0000","last_unstale":"2024-09-14T06:31:22.771601+0000","last_undegraded":"2024-09-14T06:31:22.771601+0000","last_fullsized":"2024-09-14T06:31:22.771601+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":586,"log_dups_size":0,"ondisk_log_size":586,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T11:15:43.047957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00052818099999999996,"stat_sum":{"num_bytes":41325507,"num_objects":62,"num_object_clones":0,"num_object_copies":124,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":62,"num_whiteouts":0,"num_read":6622,"num_read_kb":56524,"num_write":742,"num_write_kb":38542,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"4a","length":"1"},{"start":"4c","length":"1"}]},{"pgid":"3.13","version":"154'668","reported_seq":6980,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-09-14T06:31:18.014747+0000","last_change":"2024-09-14T06:31:15.602782+0000","last_active":"2024-09-14T06:31:18.014747+0000","last_peered":"2024-09-14T06:31:18.014747+0000","last_clean":"2024-09-14T06:31:18.014747+0000","last_became_active":"2024-09-14T06:29:00.586520+0000","last_became_peered":"2024-09-14T06:29:00.586520+0000","last_unstale":"2024-09-14T06:31:18.014747+0000","last_undegraded":"2024-09-14T06:31:18.014747+0000","last_fullsized":"2024-09-14T06:31:18.014747+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":668,"log_dups_size":0,"ondisk_log_size":668,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T07:31:11.271092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00046521699999999997,"stat_sum":{"num_bytes":66399640,"num_objects":59,"num_object_clones":0,"num_object_copies":118,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":59,"num_whiteouts":0,"num_read":7458,"num_read_kb":62495,"num_write":898,"num_write_kb":43472,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"44","length":"1"},{"start":"46","length":"1"},{"start":"48","length":"1"}]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":1793332654,"num_objects":1921,"num_object_clones":0,"num_object_copies":3842,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1921,"num_whiteouts":0,"num_read":190512,"num_read_kb":1949257,"num_write":25601,"num_write_kb":1305790,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":222,"num_bytes_recovered":31832958,"num_keys_recovered":33,"num_objects_omap":39,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":123162624,"data_stored":1243700383,"data_compressed":77106348,"data_compressed_allocated":101826560,"data_compressed_original":1226841312,"omap_allocated":0,"internal_metadata":0},"log_size":18230,"ondisk_log_size":18230,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":131072,"data_stored":1180736,"data_compressed":65166,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509474,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":66936,"kb_used_data":17680,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94304904,"statfs":{"total":96636764160,"available":96568221696,"internally_reserved":0,"allocated":18104320,"data_stored":159955617,"data_compressed":9782849,"data_compressed_allocated":12910592,"data_compressed_original":156377507,"omap_allocated":1590,"internal_metadata":50395594},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509473,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":41328,"kb_used_data":5980,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94330512,"statfs":{"total":96636764160,"available":96594444288,"internally_reserved":0,"allocated":6123520,"data_stored":46108902,"data_compressed":2629129,"data_compressed_allocated":3538944,"data_compressed_original":44634836,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509475,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":68128,"kb_used_data":18872,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94303712,"statfs":{"total":96636764160,"available":96567001088,"internally_reserved":0,"allocated":19324928,"data_stored":174683329,"data_compressed":10836759,"data_compressed_allocated":14450688,"data_compressed_original":171431790,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509474,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":101696,"kb_used_data":26092,"kb_used_omap":1,"kb_used_meta":75582,"kb_avail":94270144,"statfs":{"total":96636764160,"available":96532627456,"internally_reserved":0,"allocated":26718208,"data_stored":253733593,"data_compressed":15891310,"data_compressed_allocated":20852736,"data_compressed_original":249749749,"omap_allocated":1595,"internal_metadata":77396421},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509475,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":45324,"kb_used_data":14132,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94326516,"statfs":{"total":96636764160,"available":96590352384,"internally_reserved":0,"allocated":14471168,"data_stored":130075594,"data_compressed":7919879,"data_compressed_allocated":10481664,"data_compressed_original":127522079,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":14,"seq":60129542179,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":47844,"kb_used_data":8340,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94323996,"statfs":{"total":96636764160,"available":96587771904,"internally_reserved":0,"allocated":8540160,"data_stored":71541671,"data_compressed":4442766,"data_compressed_allocated":5820416,"data_compressed_original":70012637,"omap_allocated":1594,"internal_metadata":40434118},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542178,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":103580,"kb_used_data":23812,"kb_used_omap":1,"kb_used_meta":79742,"kb_avail":94268260,"statfs":{"total":96636764160,"available":96530698240,"internally_reserved":0,"allocated":24383488,"data_stored":224716000,"data_compressed":13982179,"data_compressed_allocated":18427904,"data_compressed_original":220497243,"omap_allocated":1595,"internal_metadata":81656261},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574883,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":77536,"kb_used_data":19980,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94294304,"statfs":{"total":96636764160,"available":96557367296,"internally_reserved":0,"allocated":20459520,"data_stored":191219717,"data_compressed":11686643,"data_compressed_allocated":15433728,"data_compressed_original":187762351,"omap_allocated":1593,"internal_metadata":58915271},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32583,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32583,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":22437888,"data_stored":223218129,"data_compressed":13949596,"data_compressed_allocated":18382848,"data_compressed_original":219923803,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12623872,"data_stored":129186904,"data_compressed":7919879,"data_compressed_allocated":10481664,"data_compressed_original":127522079,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":6660096,"data_stored":70634168,"data_compressed":4442766,"data_compressed_allocated":5820416,"data_compressed_original":70012637,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":18612224,"data_stored":190324812,"data_compressed":11686643,"data_compressed_allocated":15433728,"data_compressed_original":187762351,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":24875008,"data_stored":252844922,"data_compressed":15891310,"data_compressed_allocated":20852736,"data_compressed_original":249749749,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":17477632,"data_stored":173794639,"data_compressed":10836759,"data_compressed_allocated":14450688,"data_compressed_original":171431790,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4280320,"data_stored":45220231,"data_compressed":2629129,"data_compressed_allocated":3538944,"data_compressed_original":44634836,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16195584,"data_stored":158476578,"data_compressed":9750266,"data_compressed_allocated":12865536,"data_compressed_original":155804067,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-14T06:31:24.617 INFO:tasks.thrashosds.thrasher:Setting 3.1c to [4, 6, 1, 0] 2024-09-14T06:31:24.617 INFO:tasks.thrashosds.thrasher:cmd ['osd', 'pg-upmap-items', '3.1c', '4', '6', '1', '0'] 2024-09-14T06:31:24.617 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pg-upmap-items 3.1c 4 6 1 0 2024-09-14T06:31:24.668 INFO:tasks.ceph.osd.2.smithi094.stderr:2024-09-14T06:31:24.667+0000 7faf3269b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33649) UID: 0 2024-09-14T06:31:24.697 INFO:teuthology.orchestra.run.smithi094.stdout:ERROR: (22) Invalid argument 2024-09-14T06:31:24.697 INFO:teuthology.orchestra.run.smithi094.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:31:24.706 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-14T06:31:24.790 INFO:tasks.ceph.osd.2.smithi094.stderr:2024-09-14T06:31:24.789+0000 7faf3269b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33649) UID: 0 2024-09-14T06:31:24.863 INFO:tasks.ceph.osd.1.smithi094.stderr:2024-09-14T06:31:24.862+0000 7f328e47a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33641) UID: 0 2024-09-14T06:31:24.867 INFO:teuthology.orchestra.run.smithi094.stdout:ERROR: (22) Invalid argument 2024-09-14T06:31:24.867 INFO:teuthology.orchestra.run.smithi094.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:31:24.876 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-14T06:31:24.881 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-14T06:31:24.965 INFO:tasks.ceph.osd.2.smithi094.stderr:2024-09-14T06:31:24.964+0000 7faf3269b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33649) UID: 0 2024-09-14T06:31:25.024 INFO:teuthology.orchestra.run.smithi094.stdout:ERROR: (22) Invalid argument 2024-09-14T06:31:25.025 INFO:teuthology.orchestra.run.smithi094.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:31:25.034 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-14T06:31:25.041 INFO:teuthology.orchestra.run.smithi094.stdout:osd.0: {} 2024-09-14T06:31:25.041 INFO:teuthology.orchestra.run.smithi094.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-14T06:31:25.059 INFO:teuthology.orchestra.run.smithi094.stdout:osd.1: {} 2024-09-14T06:31:25.059 INFO:teuthology.orchestra.run.smithi094.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-14T06:31:25.065 INFO:tasks.ceph.osd.6.smithi138.stderr:2024-09-14T06:31:25.063+0000 7f8cd8849640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33620) UID: 0 2024-09-14T06:31:25.075 INFO:teuthology.orchestra.run.smithi094.stdout:osd.2: {} 2024-09-14T06:31:25.075 INFO:teuthology.orchestra.run.smithi094.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-14T06:31:25.092 INFO:teuthology.orchestra.run.smithi094.stdout:osd.3: {} 2024-09-14T06:31:25.092 INFO:teuthology.orchestra.run.smithi094.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-14T06:31:25.109 INFO:teuthology.orchestra.run.smithi094.stdout:osd.4: {} 2024-09-14T06:31:25.109 INFO:teuthology.orchestra.run.smithi094.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-14T06:31:25.126 INFO:teuthology.orchestra.run.smithi094.stdout:osd.5: {} 2024-09-14T06:31:25.126 INFO:teuthology.orchestra.run.smithi094.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-14T06:31:25.144 INFO:teuthology.orchestra.run.smithi094.stdout:osd.6: {} 2024-09-14T06:31:25.144 INFO:teuthology.orchestra.run.smithi094.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-14T06:31:25.147 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-14T06:31:25.147 INFO:teuthology.orchestra.run.smithi094.stdout: "size": 20, 2024-09-14T06:31:25.147 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 600, 2024-09-14T06:31:25.147 INFO:teuthology.orchestra.run.smithi094.stdout: "ops": [ 2024-09-14T06:31:25.147 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:25.148 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:44425 3.2 3:4603ad7f:::rbd_data.112dca2f6ae1.0000000000000005:17 [read 868352~4096 [fadvise_nocache]] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:31:25.148 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:43.854967+0000", 2024-09-14T06:31:25.148 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 101.29059046099999, 2024-09-14T06:31:25.148 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.063146025999999994, 2024-09-14T06:31:25.148 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:31:25.148 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:31:25.148 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:31:25.148 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:31:25.148 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:31:25.148 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:25.148 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 44425 2024-09-14T06:31:25.148 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:25.148 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:31:25.149 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:25.149 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:31:25.149 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.854967+0000", 2024-09-14T06:31:25.149 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:25.149 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:25.149 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:25.149 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:31:25.149 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.854967+0000", 2024-09-14T06:31:25.149 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:25.149 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:25.149 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:25.149 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:31:25.149 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.854967+0000", 2024-09-14T06:31:25.150 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.16e-07 2024-09-14T06:31:25.150 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:25.150 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:25.150 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:31:25.150 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.854968+0000", 2024-09-14T06:31:25.150 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.006e-06 2024-09-14T06:31:25.150 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:25.150 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:25.150 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:31:25.150 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.854971+0000", 2024-09-14T06:31:25.150 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.9359999999999999e-06 2024-09-14T06:31:25.150 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:25.150 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:25.150 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:31:25.151 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.854974+0000", 2024-09-14T06:31:25.151 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.1840000000000002e-06 2024-09-14T06:31:25.151 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:25.151 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:25.151 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:31:25.151 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.917945+0000", 2024-09-14T06:31:25.151 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.062970925999999997 2024-09-14T06:31:25.151 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:25.151 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:25.151 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:31:25.151 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.918030+0000", 2024-09-14T06:31:25.151 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.5863000000000001e-05 2024-09-14T06:31:25.151 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:25.152 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:25.152 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:31:25.152 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.918113+0000", 2024-09-14T06:31:25.152 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.2894999999999998e-05 2024-09-14T06:31:25.152 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:25.152 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:31:25.152 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:25.152 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:25.152 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:25.152 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:44426 3.2 3:4603ad7f:::rbd_data.112dca2f6ae1.0000000000000005:17 [read 892928~4096 [fadvise_nocache]] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:31:25.152 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:43.854977+0000", 2024-09-14T06:31:25.152 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 101.290581051, 2024-09-14T06:31:25.152 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.063369237999999994, 2024-09-14T06:31:25.152 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:31:25.153 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:31:25.153 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:31:25.153 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:31:25.153 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:31:25.153 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:25.153 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 44426 2024-09-14T06:31:25.153 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:25.153 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:31:25.153 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:25.153 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:31:25.153 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.854977+0000", 2024-09-14T06:31:25.153 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:25.153 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:25.153 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:25.154 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:31:25.154 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.854977+0000", 2024-09-14T06:31:25.154 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:25.154 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:25.154 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:25.154 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:31:25.154 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.854977+0000", 2024-09-14T06:31:25.154 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.1199999999999999e-07 2024-09-14T06:31:25.154 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:25.154 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:25.154 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:31:25.154 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.854980+0000", 2024-09-14T06:31:25.155 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.27e-06 2024-09-14T06:31:25.155 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:25.155 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:25.155 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:31:25.155 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.854983+0000", 2024-09-14T06:31:25.155 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.1650000000000002e-06 2024-09-14T06:31:25.155 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:25.155 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:25.155 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:31:25.155 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.854989+0000", 2024-09-14T06:31:25.155 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.6520000000000003e-06 2024-09-14T06:31:25.155 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:25.155 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:25.156 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:31:25.156 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.918143+0000", 2024-09-14T06:31:25.156 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.063154541999999994 2024-09-14T06:31:25.156 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:25.156 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:25.156 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:31:25.156 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.918209+0000", 2024-09-14T06:31:25.156 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.5529000000000002e-05 2024-09-14T06:31:25.156 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:25.156 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:25.156 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:31:25.156 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.918346+0000", 2024-09-14T06:31:25.156 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000136868 2024-09-14T06:31:25.157 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:25.157 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:31:25.157 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:25.157 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:25.157 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:25.157 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:44427 3.2 3:4603ad7f:::rbd_data.112dca2f6ae1.0000000000000005:17 [read 917504~4096 [fadvise_nocache]] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:31:25.157 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:43.854992+0000", 2024-09-14T06:31:25.157 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 101.290565405, 2024-09-14T06:31:25.157 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.063607792999999996, 2024-09-14T06:31:25.157 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:31:25.157 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:31:25.157 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:31:25.157 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:31:25.157 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:31:25.158 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:25.158 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 44427 2024-09-14T06:31:25.158 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:25.158 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:31:25.158 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:25.158 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:31:25.158 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.854992+0000", 2024-09-14T06:31:25.158 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:25.158 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:25.158 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:25.158 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:31:25.158 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.854992+0000", 2024-09-14T06:31:25.158 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:25.159 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:25.159 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:25.159 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:31:25.159 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.854993+0000", 2024-09-14T06:31:25.159 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.1400000000000001e-07 2024-09-14T06:31:25.159 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:25.159 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:25.159 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:31:25.159 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.854994+0000", 2024-09-14T06:31:25.159 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.091e-06 2024-09-14T06:31:25.159 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:25.159 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:25.159 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:31:25.159 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855001+0000", 2024-09-14T06:31:25.160 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.204e-06 2024-09-14T06:31:25.160 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:25.160 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:25.160 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:31:25.160 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855003+0000", 2024-09-14T06:31:25.160 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.514e-06 2024-09-14T06:31:25.160 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:25.160 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:25.160 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:31:25.160 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.918380+0000", 2024-09-14T06:31:25.160 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.063376738000000002 2024-09-14T06:31:25.160 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:25.160 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:25.161 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:31:25.161 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.918453+0000", 2024-09-14T06:31:25.161 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.3291e-05 2024-09-14T06:31:25.161 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:25.161 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:25.161 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:31:25.161 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.918600+0000", 2024-09-14T06:31:25.161 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00014674100000000001 2024-09-14T06:31:25.161 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:25.161 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:31:25.161 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:25.161 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:25.161 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:25.162 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:44428 3.2 3:4603ad7f:::rbd_data.112dca2f6ae1.0000000000000005:17 [read 942080~4096 [fadvise_nocache]] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:31:25.162 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:43.855006+0000", 2024-09-14T06:31:25.162 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 101.29055135500001, 2024-09-14T06:31:25.162 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.063846321999999997, 2024-09-14T06:31:25.162 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:31:25.162 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:31:25.162 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:31:25.162 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:31:25.162 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:31:25.162 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:25.162 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 44428 2024-09-14T06:31:25.162 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:25.162 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:31:25.162 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:25.163 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:31:25.163 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855006+0000", 2024-09-14T06:31:25.163 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:25.163 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:25.163 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:25.163 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:31:25.163 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855006+0000", 2024-09-14T06:31:25.163 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:25.163 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:25.163 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:25.163 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:31:25.163 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855007+0000", 2024-09-14T06:31:25.163 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.1199999999999999e-07 2024-09-14T06:31:25.164 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:25.164 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:25.164 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:31:25.164 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855011+0000", 2024-09-14T06:31:25.164 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.9539999999999998e-06 2024-09-14T06:31:25.164 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:25.164 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:25.164 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:31:25.164 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855014+0000", 2024-09-14T06:31:25.164 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.2930000000000001e-06 2024-09-14T06:31:25.164 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:25.164 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:25.164 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:31:25.164 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855016+0000", 2024-09-14T06:31:25.165 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.3039999999999999e-06 2024-09-14T06:31:25.165 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:25.165 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:25.165 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:31:25.165 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.918634+0000", 2024-09-14T06:31:25.165 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.063617928000000004 2024-09-14T06:31:25.165 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:25.165 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:25.165 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:31:25.165 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.918711+0000", 2024-09-14T06:31:25.165 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.6545000000000007e-05 2024-09-14T06:31:25.165 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:25.165 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:25.165 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:31:25.166 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.918853+0000", 2024-09-14T06:31:25.166 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00014208600000000001 2024-09-14T06:31:25.166 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:25.166 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:31:25.166 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:25.166 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:25.166 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:25.166 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:44429 3.2 3:4603ad7f:::rbd_data.112dca2f6ae1.0000000000000005:17 [read 966656~4096 [fadvise_nocache]] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:31:25.166 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:43.855019+0000", 2024-09-14T06:31:25.166 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 101.290538682, 2024-09-14T06:31:25.166 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.064059226999999996, 2024-09-14T06:31:25.166 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:31:25.167 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:31:25.167 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:31:25.167 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:31:25.167 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:31:25.167 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:25.167 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 44429 2024-09-14T06:31:25.167 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:25.167 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:31:25.167 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:25.167 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:31:25.167 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855019+0000", 2024-09-14T06:31:25.167 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:25.167 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:25.167 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:25.168 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:31:25.168 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855019+0000", 2024-09-14T06:31:25.168 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:25.168 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:25.168 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:25.168 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:31:25.168 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855019+0000", 2024-09-14T06:31:25.168 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.28e-07 2024-09-14T06:31:25.168 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:25.168 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:25.168 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:31:25.169 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855020+0000", 2024-09-14T06:31:25.169 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.0330000000000001e-06 2024-09-14T06:31:25.169 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:25.169 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:25.169 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:31:25.169 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855023+0000", 2024-09-14T06:31:25.169 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.79e-06 2024-09-14T06:31:25.169 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:25.169 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:25.169 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:31:25.169 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855025+0000", 2024-09-14T06:31:25.169 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.216e-06 2024-09-14T06:31:25.169 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:25.170 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:25.170 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:31:25.170 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.918886+0000", 2024-09-14T06:31:25.170 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.063861037999999995 2024-09-14T06:31:25.170 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:25.170 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:25.170 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:31:25.170 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.918952+0000", 2024-09-14T06:31:25.170 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.5993000000000003e-05 2024-09-14T06:31:25.170 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:25.171 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:25.171 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:31:25.171 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.919078+0000", 2024-09-14T06:31:25.171 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00012592899999999999 2024-09-14T06:31:25.171 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:25.171 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:31:25.171 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:25.171 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:25.171 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:25.171 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:44430 3.2 3:4603ad7f:::rbd_data.112dca2f6ae1.0000000000000005:17 [read 991232~4096 [fadvise_nocache]] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:31:25.171 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:43.855028+0000", 2024-09-14T06:31:25.171 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 101.29052967200001, 2024-09-14T06:31:25.171 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.064284346000000006, 2024-09-14T06:31:25.172 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:31:25.172 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:31:25.172 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:31:25.172 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:31:25.172 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:31:25.172 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:25.172 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 44430 2024-09-14T06:31:25.172 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:25.172 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:31:25.172 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:25.172 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:31:25.172 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855028+0000", 2024-09-14T06:31:25.173 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:25.173 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:25.173 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:25.173 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:31:25.173 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855028+0000", 2024-09-14T06:31:25.173 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:25.173 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:25.173 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:25.173 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:31:25.173 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855028+0000", 2024-09-14T06:31:25.173 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.1199999999999999e-07 2024-09-14T06:31:25.173 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:25.173 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:25.173 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:31:25.174 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855029+0000", 2024-09-14T06:31:25.174 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.1100000000000004e-07 2024-09-14T06:31:25.174 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:25.174 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:25.174 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:31:25.174 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855032+0000", 2024-09-14T06:31:25.174 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.7889999999999999e-06 2024-09-14T06:31:25.174 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:25.174 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:25.174 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:31:25.174 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855034+0000", 2024-09-14T06:31:25.174 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.125e-06 2024-09-14T06:31:25.174 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:25.175 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:25.175 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:31:25.175 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.919110+0000", 2024-09-14T06:31:25.175 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.064076413999999998 2024-09-14T06:31:25.175 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:25.175 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:25.175 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:31:25.175 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.919186+0000", 2024-09-14T06:31:25.175 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.5784999999999994e-05 2024-09-14T06:31:25.175 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:25.175 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:25.175 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:31:25.175 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.919312+0000", 2024-09-14T06:31:25.175 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00012611 2024-09-14T06:31:25.176 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:25.176 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:31:25.176 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:25.176 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:25.176 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:25.176 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:44431 3.2 3:4603ad7f:::rbd_data.112dca2f6ae1.0000000000000005:17 [read 1015808~4096 [fadvise_nocache]] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:31:25.176 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:43.855037+0000", 2024-09-14T06:31:25.176 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 101.290520804, 2024-09-14T06:31:25.176 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.064469755000000004, 2024-09-14T06:31:25.176 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:31:25.176 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:31:25.176 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:31:25.176 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:31:25.177 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:31:25.177 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:25.177 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 44431 2024-09-14T06:31:25.177 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:25.177 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:31:25.177 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:25.177 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:31:25.177 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855037+0000", 2024-09-14T06:31:25.177 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:25.177 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:25.177 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:25.177 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:31:25.177 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855037+0000", 2024-09-14T06:31:25.177 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:25.178 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:25.178 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:25.178 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:31:25.178 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855037+0000", 2024-09-14T06:31:25.178 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.1e-07 2024-09-14T06:31:25.178 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:25.178 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:25.178 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:31:25.178 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855038+0000", 2024-09-14T06:31:25.178 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.9400000000000004e-07 2024-09-14T06:31:25.178 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:25.178 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:25.179 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:31:25.179 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855041+0000", 2024-09-14T06:31:25.179 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.9639999999999999e-06 2024-09-14T06:31:25.179 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:25.179 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:25.179 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:31:25.179 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855043+0000", 2024-09-14T06:31:25.179 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.2259999999999999e-06 2024-09-14T06:31:25.179 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:25.180 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:25.180 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:31:25.180 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.919347+0000", 2024-09-14T06:31:25.180 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.064304091999999993 2024-09-14T06:31:25.180 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:25.180 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:25.180 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:31:25.180 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.919393+0000", 2024-09-14T06:31:25.180 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.5340999999999998e-05 2024-09-14T06:31:25.180 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:25.180 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:25.180 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:31:25.180 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.919507+0000", 2024-09-14T06:31:25.181 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000114028 2024-09-14T06:31:25.181 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:25.181 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:31:25.181 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:25.181 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:25.181 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:25.181 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:44432 3.2 3:4603ad7f:::rbd_data.112dca2f6ae1.0000000000000005:17 [read 1040384~4096 [fadvise_nocache]] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:31:25.181 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:43.855046+0000", 2024-09-14T06:31:25.181 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 101.290511736, 2024-09-14T06:31:25.181 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.064639253999999993, 2024-09-14T06:31:25.181 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:31:25.181 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:31:25.181 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:31:25.182 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:31:25.182 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:31:25.182 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:25.182 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 44432 2024-09-14T06:31:25.182 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:25.182 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:31:25.182 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:25.182 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:31:25.182 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855046+0000", 2024-09-14T06:31:25.182 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:25.182 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:25.182 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:25.183 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:31:25.183 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855046+0000", 2024-09-14T06:31:25.183 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:25.183 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:25.183 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:25.183 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:31:25.183 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855046+0000", 2024-09-14T06:31:25.183 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.05e-07 2024-09-14T06:31:25.183 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:25.183 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:25.183 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:31:25.183 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855047+0000", 2024-09-14T06:31:25.183 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.879999999999999e-07 2024-09-14T06:31:25.184 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:25.184 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:25.184 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:31:25.184 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855050+0000", 2024-09-14T06:31:25.184 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.8459999999999999e-06 2024-09-14T06:31:25.184 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:25.184 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:25.184 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:31:25.184 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855052+0000", 2024-09-14T06:31:25.184 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.136e-06 2024-09-14T06:31:25.184 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:25.184 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:25.184 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:31:25.185 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.919536+0000", 2024-09-14T06:31:25.185 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.064484132 2024-09-14T06:31:25.185 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:25.185 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:25.185 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:31:25.185 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.919611+0000", 2024-09-14T06:31:25.185 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.4407999999999996e-05 2024-09-14T06:31:25.185 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:25.185 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:25.185 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:31:25.185 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.919685+0000", 2024-09-14T06:31:25.185 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.4538999999999999e-05 2024-09-14T06:31:25.185 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:25.186 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:31:25.186 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:25.186 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:25.186 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:25.186 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:44518 3.2 3:4190d882:::rbd_data.112dca2f6ae1.0000000000000002:17 [read 12288~4096 [fadvise_nocache]] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:31:25.186 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:43.855055+0000", 2024-09-14T06:31:25.186 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 101.290502852, 2024-09-14T06:31:25.186 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.064894317000000007, 2024-09-14T06:31:25.186 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:31:25.186 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:31:25.186 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:31:25.186 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:31:25.186 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:31:25.187 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:25.187 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 44518 2024-09-14T06:31:25.187 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:25.187 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:31:25.187 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:25.187 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:31:25.187 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855055+0000", 2024-09-14T06:31:25.187 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:25.187 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:25.187 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:25.187 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:31:25.187 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855055+0000", 2024-09-14T06:31:25.187 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:25.188 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:25.188 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:25.188 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:31:25.188 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855055+0000", 2024-09-14T06:31:25.188 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.0699999999999999e-07 2024-09-14T06:31:25.188 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:25.188 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:25.188 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:31:25.188 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855056+0000", 2024-09-14T06:31:25.188 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.9400000000000004e-07 2024-09-14T06:31:25.188 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:25.188 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:25.188 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:31:25.189 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855059+0000", 2024-09-14T06:31:25.189 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.8870000000000001e-06 2024-09-14T06:31:25.189 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:25.189 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:25.189 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:31:25.189 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855061+0000", 2024-09-14T06:31:25.189 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.1919999999999999e-06 2024-09-14T06:31:25.189 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:25.189 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:25.189 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:31:25.189 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.919719+0000", 2024-09-14T06:31:25.189 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.064658013 2024-09-14T06:31:25.189 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:25.190 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:25.190 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:31:25.190 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.919788+0000", 2024-09-14T06:31:25.190 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.9282000000000005e-05 2024-09-14T06:31:25.190 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:25.190 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:25.190 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:31:25.190 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.919949+0000", 2024-09-14T06:31:25.190 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00016084199999999999 2024-09-14T06:31:25.190 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:25.190 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:31:25.190 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:25.190 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:25.190 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:25.191 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:44519 3.2 3:4190d882:::rbd_data.112dca2f6ae1.0000000000000002:17 [read 36864~4096 [fadvise_nocache]] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:31:25.191 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:43.855064+0000", 2024-09-14T06:31:25.191 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 101.290493875, 2024-09-14T06:31:25.191 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.065081865000000003, 2024-09-14T06:31:25.191 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:31:25.191 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:31:25.191 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:31:25.191 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:31:25.191 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:31:25.191 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:25.191 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 44519 2024-09-14T06:31:25.191 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:25.191 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:31:25.192 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:25.192 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:31:25.192 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855064+0000", 2024-09-14T06:31:25.192 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:25.192 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:25.192 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:25.192 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:31:25.192 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855064+0000", 2024-09-14T06:31:25.192 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:25.192 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:25.192 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:25.192 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:31:25.192 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855064+0000", 2024-09-14T06:31:25.193 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.1e-07 2024-09-14T06:31:25.193 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:25.193 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:25.193 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:31:25.193 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855065+0000", 2024-09-14T06:31:25.193 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.035e-06 2024-09-14T06:31:25.193 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:25.193 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:25.193 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:31:25.193 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855068+0000", 2024-09-14T06:31:25.193 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.8339999999999999e-06 2024-09-14T06:31:25.193 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:25.193 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:25.194 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:31:25.194 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855071+0000", 2024-09-14T06:31:25.194 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.3450000000000002e-06 2024-09-14T06:31:25.194 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:25.194 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:25.194 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:31:25.194 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.919979+0000", 2024-09-14T06:31:25.194 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.064907647999999998 2024-09-14T06:31:25.194 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:25.194 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:25.194 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:31:25.194 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.920026+0000", 2024-09-14T06:31:25.194 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.7595000000000001e-05 2024-09-14T06:31:25.194 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:25.195 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:25.195 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:31:25.195 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.920146+0000", 2024-09-14T06:31:25.195 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00011919800000000001 2024-09-14T06:31:25.195 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:25.195 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:31:25.195 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:25.195 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:25.195 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:25.195 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:44520 3.2 3:4190d882:::rbd_data.112dca2f6ae1.0000000000000002:17 [read 61440~4096 [fadvise_nocache]] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:31:25.195 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:43.855075+0000", 2024-09-14T06:31:25.195 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 101.29048282399999, 2024-09-14T06:31:25.196 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.065335078000000005, 2024-09-14T06:31:25.196 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:31:25.196 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:31:25.196 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:31:25.196 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:31:25.196 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:31:25.196 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:25.196 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 44520 2024-09-14T06:31:25.196 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:25.196 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:31:25.196 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:25.196 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:31:25.196 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855075+0000", 2024-09-14T06:31:25.196 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:25.197 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:25.197 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:25.197 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:31:25.197 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855075+0000", 2024-09-14T06:31:25.197 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:25.197 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:25.197 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:25.197 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:31:25.197 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855075+0000", 2024-09-14T06:31:25.197 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.1e-07 2024-09-14T06:31:25.197 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:25.197 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:25.198 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:31:25.198 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855076+0000", 2024-09-14T06:31:25.198 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.0839999999999999e-06 2024-09-14T06:31:25.198 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:25.198 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:25.198 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:31:25.198 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855079+0000", 2024-09-14T06:31:25.198 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.9359999999999999e-06 2024-09-14T06:31:25.198 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:25.198 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:25.198 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:31:25.198 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855084+0000", 2024-09-14T06:31:25.198 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.1730000000000003e-06 2024-09-14T06:31:25.199 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:25.199 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:25.199 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:31:25.199 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.920175+0000", 2024-09-14T06:31:25.199 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.065090234999999996 2024-09-14T06:31:25.199 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:25.199 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:25.199 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:31:25.199 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.920250+0000", 2024-09-14T06:31:25.199 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.5807000000000001e-05 2024-09-14T06:31:25.199 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:25.199 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:25.200 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:31:25.200 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.920410+0000", 2024-09-14T06:31:25.200 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00015963300000000001 2024-09-14T06:31:25.200 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:25.200 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:31:25.200 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:25.200 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:25.200 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:25.200 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:44521 3.2 3:4190d882:::rbd_data.112dca2f6ae1.0000000000000002:17 [read 86016~4096 [fadvise_nocache]] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:31:25.200 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:43.855087+0000", 2024-09-14T06:31:25.200 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 101.290470343, 2024-09-14T06:31:25.200 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.065563264999999996, 2024-09-14T06:31:25.200 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:31:25.200 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:31:25.201 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:31:25.201 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:31:25.201 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:31:25.201 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:25.201 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 44521 2024-09-14T06:31:25.201 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:25.201 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:31:25.201 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:25.201 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:31:25.201 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855087+0000", 2024-09-14T06:31:25.201 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:25.201 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:25.202 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:25.202 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:31:25.202 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855087+0000", 2024-09-14T06:31:25.202 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:25.202 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:25.202 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:25.202 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:31:25.202 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855088+0000", 2024-09-14T06:31:25.202 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.3699999999999999e-07 2024-09-14T06:31:25.202 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:25.202 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:25.202 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:31:25.202 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855089+0000", 2024-09-14T06:31:25.202 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.9900000000000009e-07 2024-09-14T06:31:25.202 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:25.203 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:25.203 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:31:25.203 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855091+0000", 2024-09-14T06:31:25.203 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.8940000000000002e-06 2024-09-14T06:31:25.203 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:25.203 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:25.203 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:31:25.203 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855094+0000", 2024-09-14T06:31:25.203 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.351e-06 2024-09-14T06:31:25.203 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:25.203 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:25.203 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:31:25.203 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.920437+0000", 2024-09-14T06:31:25.203 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.065343221000000007 2024-09-14T06:31:25.204 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:25.204 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:25.204 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:31:25.204 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.920481+0000", 2024-09-14T06:31:25.204 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.4431000000000002e-05 2024-09-14T06:31:25.204 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:25.204 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:25.204 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:31:25.204 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.920651+0000", 2024-09-14T06:31:25.204 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00016913200000000001 2024-09-14T06:31:25.204 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:25.204 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:31:25.204 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:25.205 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:25.205 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:25.205 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:44522 3.2 3:4190d882:::rbd_data.112dca2f6ae1.0000000000000002:17 [read 110592~4096 [fadvise_nocache]] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:31:25.205 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:43.855098+0000", 2024-09-14T06:31:25.205 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 101.29046011600001, 2024-09-14T06:31:25.205 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.065827628999999999, 2024-09-14T06:31:25.205 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:31:25.205 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:31:25.205 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:31:25.205 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:31:25.205 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:31:25.205 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:25.205 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 44522 2024-09-14T06:31:25.206 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:25.206 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:31:25.206 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:25.206 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:31:25.206 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855098+0000", 2024-09-14T06:31:25.206 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:25.206 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:25.206 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:25.206 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:31:25.206 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855098+0000", 2024-09-14T06:31:25.206 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:25.206 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:25.206 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:25.207 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:31:25.207 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855098+0000", 2024-09-14T06:31:25.207 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.1e-07 2024-09-14T06:31:25.207 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:25.207 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:25.207 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:31:25.207 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855099+0000", 2024-09-14T06:31:25.207 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.012e-06 2024-09-14T06:31:25.207 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:25.207 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:25.207 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:31:25.207 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855102+0000", 2024-09-14T06:31:25.207 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.9380000000000001e-06 2024-09-14T06:31:25.207 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:25.208 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:25.208 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:31:25.208 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855108+0000", 2024-09-14T06:31:25.208 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.4339999999999997e-06 2024-09-14T06:31:25.208 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:25.208 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:25.208 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:31:25.208 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.920681+0000", 2024-09-14T06:31:25.208 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.065572943999999994 2024-09-14T06:31:25.208 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:25.208 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:25.208 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:31:25.208 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.920724+0000", 2024-09-14T06:31:25.208 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.2877000000000003e-05 2024-09-14T06:31:25.209 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:25.209 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:25.209 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:31:25.209 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.920925+0000", 2024-09-14T06:31:25.209 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00020121399999999999 2024-09-14T06:31:25.209 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:25.209 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:31:25.209 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:25.209 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:25.209 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:25.209 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:44523 3.2 3:4190d882:::rbd_data.112dca2f6ae1.0000000000000002:17 [read 135168~4096 [fadvise_nocache]] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:31:25.209 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:43.855111+0000", 2024-09-14T06:31:25.209 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 101.290446394, 2024-09-14T06:31:25.210 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.066028198999999996, 2024-09-14T06:31:25.210 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:31:25.210 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:31:25.210 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:31:25.210 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:31:25.210 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:31:25.210 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:25.210 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 44523 2024-09-14T06:31:25.210 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:25.210 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:31:25.210 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:25.210 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:31:25.210 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855111+0000", 2024-09-14T06:31:25.211 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:25.211 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:25.211 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:25.211 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:31:25.211 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855111+0000", 2024-09-14T06:31:25.211 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:25.211 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:25.211 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:25.211 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:31:25.211 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855112+0000", 2024-09-14T06:31:25.211 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.1199999999999999e-07 2024-09-14T06:31:25.211 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:25.211 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:25.211 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:31:25.212 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855115+0000", 2024-09-14T06:31:25.212 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.3380000000000001e-06 2024-09-14T06:31:25.212 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:25.212 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:25.212 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:31:25.212 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855118+0000", 2024-09-14T06:31:25.212 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.366e-06 2024-09-14T06:31:25.212 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:25.212 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:25.212 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:31:25.212 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855121+0000", 2024-09-14T06:31:25.212 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.8329999999999998e-06 2024-09-14T06:31:25.212 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:25.213 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:25.213 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:31:25.213 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.920956+0000", 2024-09-14T06:31:25.213 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.065835251999999997 2024-09-14T06:31:25.213 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:25.213 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:25.213 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:31:25.213 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.921001+0000", 2024-09-14T06:31:25.213 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.4778999999999999e-05 2024-09-14T06:31:25.213 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:25.213 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:25.213 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:31:25.213 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.921139+0000", 2024-09-14T06:31:25.213 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000138419 2024-09-14T06:31:25.214 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:25.214 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:31:25.214 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:25.214 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:25.214 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:25.214 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:44524 3.2 3:4190d882:::rbd_data.112dca2f6ae1.0000000000000002:17 [read 159744~4096 [fadvise_nocache]] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:31:25.214 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:43.855124+0000", 2024-09-14T06:31:25.214 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 101.290433415, 2024-09-14T06:31:25.214 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.066249261000000004, 2024-09-14T06:31:25.214 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:31:25.214 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:31:25.214 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:31:25.214 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:31:25.215 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:31:25.215 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:25.215 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 44524 2024-09-14T06:31:25.215 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:25.215 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:31:25.215 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:25.215 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:31:25.215 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855124+0000", 2024-09-14T06:31:25.215 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:25.215 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:25.215 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:25.215 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:31:25.215 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855124+0000", 2024-09-14T06:31:25.215 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:25.216 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:25.216 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:25.216 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:31:25.216 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861086+0000", 2024-09-14T06:31:25.216 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0059613829999999998 2024-09-14T06:31:25.216 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:25.216 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:25.216 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:31:25.216 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861100+0000", 2024-09-14T06:31:25.216 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.4644000000000001e-05 2024-09-14T06:31:25.216 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:25.216 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:25.216 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:31:25.216 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861109+0000", 2024-09-14T06:31:25.217 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.541e-06 2024-09-14T06:31:25.217 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:25.217 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:25.217 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:31:25.217 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861117+0000", 2024-09-14T06:31:25.217 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.1049999999999995e-06 2024-09-14T06:31:25.217 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:25.217 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:25.217 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:31:25.217 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.921169+0000", 2024-09-14T06:31:25.217 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.060052405000000003 2024-09-14T06:31:25.217 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:25.217 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:25.217 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:31:25.218 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.921213+0000", 2024-09-14T06:31:25.218 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.3565e-05 2024-09-14T06:31:25.218 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:25.218 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:25.218 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:31:25.218 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.921374+0000", 2024-09-14T06:31:25.218 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00016061800000000001 2024-09-14T06:31:25.218 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:25.218 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:31:25.218 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:25.218 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:25.218 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:25.218 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:44536 3.2 3:4190d882:::rbd_data.112dca2f6ae1.0000000000000002:17 [read 454656~4096 [fadvise_nocache]] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:31:25.218 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:43.861384+0000", 2024-09-14T06:31:25.219 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 101.284173858, 2024-09-14T06:31:25.219 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.063058528000000003, 2024-09-14T06:31:25.219 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:31:25.219 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:31:25.219 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:31:25.219 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:31:25.219 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:31:25.219 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:25.219 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 44536 2024-09-14T06:31:25.219 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:25.219 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:31:25.219 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:25.219 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:31:25.219 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861384+0000", 2024-09-14T06:31:25.220 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:25.220 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:25.220 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:25.220 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:31:25.220 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861384+0000", 2024-09-14T06:31:25.220 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:25.220 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:25.220 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:25.220 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:31:25.220 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861384+0000", 2024-09-14T06:31:25.220 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.4000000000000002e-07 2024-09-14T06:31:25.220 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:25.220 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:25.221 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:31:25.221 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861386+0000", 2024-09-14T06:31:25.221 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.891e-06 2024-09-14T06:31:25.221 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:25.221 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:25.221 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:31:25.221 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861392+0000", 2024-09-14T06:31:25.221 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.9939999999999997e-06 2024-09-14T06:31:25.221 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:25.221 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:25.221 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:31:25.222 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861398+0000", 2024-09-14T06:31:25.222 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.507e-06 2024-09-14T06:31:25.222 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:25.222 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:25.222 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:31:25.222 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.924149+0000", 2024-09-14T06:31:25.222 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.062751135999999999 2024-09-14T06:31:25.222 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:25.222 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:25.222 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:31:25.222 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.924226+0000", 2024-09-14T06:31:25.222 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.7517999999999999e-05 2024-09-14T06:31:25.222 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:25.222 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:25.223 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:31:25.223 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.924442+0000", 2024-09-14T06:31:25.223 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000216042 2024-09-14T06:31:25.223 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:25.223 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:31:25.223 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:25.223 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:25.223 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:25.223 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:44537 3.2 3:4190d882:::rbd_data.112dca2f6ae1.0000000000000002:17 [read 479232~4096 [fadvise_nocache]] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:31:25.223 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:43.861405+0000", 2024-09-14T06:31:25.223 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 101.284152812, 2024-09-14T06:31:25.223 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.063351795000000002, 2024-09-14T06:31:25.223 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:31:25.223 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:31:25.224 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:31:25.224 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:31:25.224 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:31:25.224 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:25.224 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 44537 2024-09-14T06:31:25.224 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:25.224 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:31:25.224 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:25.224 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:31:25.224 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861405+0000", 2024-09-14T06:31:25.224 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:25.224 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:25.224 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:25.225 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:31:25.225 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861405+0000", 2024-09-14T06:31:25.225 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:25.225 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:25.225 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:25.225 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:31:25.225 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861405+0000", 2024-09-14T06:31:25.225 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.9099999999999999e-07 2024-09-14T06:31:25.225 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:25.225 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:25.225 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:31:25.225 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861407+0000", 2024-09-14T06:31:25.225 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.075e-06 2024-09-14T06:31:25.225 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:25.226 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:25.226 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:31:25.226 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861414+0000", 2024-09-14T06:31:25.226 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.4659999999999996e-06 2024-09-14T06:31:25.226 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:25.226 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:25.226 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:31:25.226 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861418+0000", 2024-09-14T06:31:25.226 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.5519999999999998e-06 2024-09-14T06:31:25.226 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:25.226 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:25.226 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:31:25.226 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.924474+0000", 2024-09-14T06:31:25.227 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.063056012999999994 2024-09-14T06:31:25.227 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:25.227 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:25.227 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:31:25.227 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.924556+0000", 2024-09-14T06:31:25.227 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.1181999999999996e-05 2024-09-14T06:31:25.227 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:25.227 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:25.227 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:31:25.227 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.924757+0000", 2024-09-14T06:31:25.227 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00020111599999999999 2024-09-14T06:31:25.227 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:25.227 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:31:25.228 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:25.228 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:25.228 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:25.228 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:44538 3.2 3:4190d882:::rbd_data.112dca2f6ae1.0000000000000002:17 [read 503808~4096 [fadvise_nocache]] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:31:25.228 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:43.861425+0000", 2024-09-14T06:31:25.228 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 101.284132771, 2024-09-14T06:31:25.228 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.063627125000000007, 2024-09-14T06:31:25.228 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:31:25.228 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:31:25.228 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:31:25.228 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:31:25.228 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:31:25.228 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:25.228 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 44538 2024-09-14T06:31:25.229 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:25.229 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:31:25.229 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:25.229 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:31:25.229 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861425+0000", 2024-09-14T06:31:25.229 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:25.229 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:25.229 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:25.229 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:31:25.229 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861425+0000", 2024-09-14T06:31:25.229 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:25.229 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:25.229 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:25.229 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:31:25.230 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861425+0000", 2024-09-14T06:31:25.230 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.9999999999999998e-07 2024-09-14T06:31:25.230 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:25.230 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:25.230 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:31:25.230 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861427+0000", 2024-09-14T06:31:25.230 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.9829999999999999e-06 2024-09-14T06:31:25.230 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:25.230 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:25.230 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:31:25.230 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861433+0000", 2024-09-14T06:31:25.230 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.2889999999999997e-06 2024-09-14T06:31:25.230 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:25.230 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:25.231 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:31:25.231 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861454+0000", 2024-09-14T06:31:25.231 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.1844999999999998e-05 2024-09-14T06:31:25.231 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:25.231 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:25.231 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:31:25.231 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.924786+0000", 2024-09-14T06:31:25.231 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.063331497 2024-09-14T06:31:25.231 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:25.231 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:25.231 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:31:25.231 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.924864+0000", 2024-09-14T06:31:25.231 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.7744000000000006e-05 2024-09-14T06:31:25.231 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:25.232 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:25.232 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:31:25.232 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.925052+0000", 2024-09-14T06:31:25.232 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00018836699999999999 2024-09-14T06:31:25.232 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:25.232 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:31:25.232 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:25.232 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:25.232 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:25.232 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:44539 3.2 3:4190d882:::rbd_data.112dca2f6ae1.0000000000000002:17 [read 528384~4096 [fadvise_nocache]] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:31:25.232 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:43.861462+0000", 2024-09-14T06:31:25.232 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 101.284095444, 2024-09-14T06:31:25.232 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.063855705999999998, 2024-09-14T06:31:25.233 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:31:25.233 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:31:25.233 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:31:25.233 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:31:25.233 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:31:25.233 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:25.233 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 44539 2024-09-14T06:31:25.233 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:25.233 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:31:25.233 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:25.233 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:31:25.233 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861462+0000", 2024-09-14T06:31:25.233 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:25.234 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:25.234 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:25.234 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:31:25.234 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861462+0000", 2024-09-14T06:31:25.234 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:25.234 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:25.234 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:25.234 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:31:25.234 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861463+0000", 2024-09-14T06:31:25.234 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.5200000000000002e-07 2024-09-14T06:31:25.234 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:25.234 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:25.234 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:31:25.234 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861465+0000", 2024-09-14T06:31:25.235 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.5189999999999999e-06 2024-09-14T06:31:25.235 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:25.235 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:25.235 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:31:25.235 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861471+0000", 2024-09-14T06:31:25.235 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.6899999999999997e-06 2024-09-14T06:31:25.235 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:25.235 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:25.235 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:31:25.235 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861475+0000", 2024-09-14T06:31:25.235 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.5730000000000001e-06 2024-09-14T06:31:25.235 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:25.235 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:25.235 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:31:25.236 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.925085+0000", 2024-09-14T06:31:25.236 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.063609902999999995 2024-09-14T06:31:25.236 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:25.236 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:25.236 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:31:25.236 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.925145+0000", 2024-09-14T06:31:25.236 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.9383999999999999e-05 2024-09-14T06:31:25.236 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:25.236 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:25.236 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:31:25.236 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.925318+0000", 2024-09-14T06:31:25.236 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00017318499999999999 2024-09-14T06:31:25.236 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:25.237 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:31:25.237 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:25.237 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:25.237 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:25.237 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:44540 3.2 3:4190d882:::rbd_data.112dca2f6ae1.0000000000000002:17 [read 552960~4096 [fadvise_nocache]] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:31:25.237 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:43.861483+0000", 2024-09-14T06:31:25.237 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 101.28407501300001, 2024-09-14T06:31:25.237 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.064065681999999999, 2024-09-14T06:31:25.237 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:31:25.237 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:31:25.237 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:31:25.237 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:31:25.237 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:31:25.237 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:25.238 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 44540 2024-09-14T06:31:25.238 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:25.238 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:31:25.238 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:25.238 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:31:25.238 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861483+0000", 2024-09-14T06:31:25.238 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:25.238 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:25.238 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:25.238 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:31:25.238 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861483+0000", 2024-09-14T06:31:25.238 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:25.238 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:25.238 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:25.239 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:31:25.239 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.867079+0000", 2024-09-14T06:31:25.239 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0055966480000000001 2024-09-14T06:31:25.239 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:25.239 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:25.239 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:31:25.239 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.867086+0000", 2024-09-14T06:31:25.239 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.387e-06 2024-09-14T06:31:25.239 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:25.239 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:25.239 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:31:25.239 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.867094+0000", 2024-09-14T06:31:25.239 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.9419999999999995e-06 2024-09-14T06:31:25.240 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:25.240 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:25.240 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:31:25.240 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.867109+0000", 2024-09-14T06:31:25.240 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.5818999999999999e-05 2024-09-14T06:31:25.240 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:25.240 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:25.240 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:31:25.240 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.925354+0000", 2024-09-14T06:31:25.240 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.058244047 2024-09-14T06:31:25.240 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:25.240 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:25.240 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:31:25.241 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.925406+0000", 2024-09-14T06:31:25.241 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.2661999999999998e-05 2024-09-14T06:31:25.241 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:25.241 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:25.241 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:31:25.241 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.925548+0000", 2024-09-14T06:31:25.241 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000142177 2024-09-14T06:31:25.241 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:25.241 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:31:25.241 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:25.241 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:25.241 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:31:25.241 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-14T06:31:25.243 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-14T06:31:25.243 INFO:teuthology.orchestra.run.smithi094.stdout:osd.7: {} 2024-09-14T06:31:25.244 INFO:teuthology.orchestra.run.smithi094.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-14T06:31:25.245 INFO:tasks.ceph.osd.6.smithi138.stderr:2024-09-14T06:31:25.241+0000 7f8cd8849640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33620) UID: 0 2024-09-14T06:31:25.346 INFO:tasks.ceph.osd.4.smithi138.stderr:2024-09-14T06:31:25.343+0000 7f6c653c8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33615) UID: 0 2024-09-14T06:31:25.358 INFO:teuthology.orchestra.run.smithi138.stdout:{ 2024-09-14T06:31:25.358 INFO:teuthology.orchestra.run.smithi138.stdout: "ops": [], 2024-09-14T06:31:25.358 INFO:teuthology.orchestra.run.smithi138.stdout: "num_ops": 0 2024-09-14T06:31:25.358 INFO:teuthology.orchestra.run.smithi138.stdout:} 2024-09-14T06:31:25.367 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-14T06:31:25.447 INFO:tasks.ceph.osd.0.smithi094.stderr:2024-09-14T06:31:25.446+0000 7f07c9333640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33636) UID: 0 2024-09-14T06:31:25.511 INFO:teuthology.orchestra.run.smithi138.stdout:{ 2024-09-14T06:31:25.511 INFO:teuthology.orchestra.run.smithi138.stdout: "ops": [], 2024-09-14T06:31:25.511 INFO:teuthology.orchestra.run.smithi138.stdout: "complaint_time": 30, 2024-09-14T06:31:25.511 INFO:teuthology.orchestra.run.smithi138.stdout: "num_blocked_ops": 0 2024-09-14T06:31:25.511 INFO:teuthology.orchestra.run.smithi138.stdout:} 2024-09-14T06:31:25.520 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-14T06:31:25.547 INFO:tasks.ceph.osd.1.smithi094.stderr:2024-09-14T06:31:25.546+0000 7f328e47a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33641) UID: 0 2024-09-14T06:31:25.612 INFO:teuthology.orchestra.run.smithi094.stderr:set 3.1c pg_upmap_items mapping to [4->6,1->0] 2024-09-14T06:31:25.627 INFO:teuthology.orchestra.run.smithi138.stdout:{ 2024-09-14T06:31:25.627 INFO:teuthology.orchestra.run.smithi138.stdout: "size": 20, 2024-09-14T06:31:25.627 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 600, 2024-09-14T06:31:25.627 INFO:teuthology.orchestra.run.smithi138.stdout: "ops": [ 2024-09-14T06:31:25.628 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:25.628 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:19328 3.3 3:c34e5e51:::rbd_data.112d92027590.0000000000000024:f [read 1926144~1024 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:31:25.628 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:27.059768+0000", 2024-09-14T06:31:25.628 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 118.56577230800001, 2024-09-14T06:31:25.628 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046369147999999999, 2024-09-14T06:31:25.628 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:31:25.628 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:31:25.628 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:31:25.628 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:31:25.628 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:31:25.628 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:25.628 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 19328 2024-09-14T06:31:25.628 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:25.628 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:31:25.629 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:25.629 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:31:25.629 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.059768+0000", 2024-09-14T06:31:25.629 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:25.629 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:25.629 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:25.629 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:31:25.629 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.059768+0000", 2024-09-14T06:31:25.629 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:25.629 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:25.629 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:25.629 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:31:25.630 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.059874+0000", 2024-09-14T06:31:25.630 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00010551400000000001 2024-09-14T06:31:25.630 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:25.630 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:25.630 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:31:25.630 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.059877+0000", 2024-09-14T06:31:25.630 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.8279999999999999e-06 2024-09-14T06:31:25.630 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:25.630 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:25.630 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:31:25.630 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.059884+0000", 2024-09-14T06:31:25.630 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.6389999999999992e-06 2024-09-14T06:31:25.630 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:25.631 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:25.631 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:31:25.631 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.059890+0000", 2024-09-14T06:31:25.631 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.49e-06 2024-09-14T06:31:25.631 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:25.631 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:25.631 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:31:25.631 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.105804+0000", 2024-09-14T06:31:25.631 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.045914242000000001 2024-09-14T06:31:25.631 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:25.631 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:25.631 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:31:25.631 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.105886+0000", 2024-09-14T06:31:25.631 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.2305999999999994e-05 2024-09-14T06:31:25.632 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:25.632 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:25.632 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:31:25.632 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.106138+0000", 2024-09-14T06:31:25.632 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.000251129 2024-09-14T06:31:25.632 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:25.632 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:31:25.632 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:25.632 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:25.632 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:25.632 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:19329 3.3 3:c34e5e51:::rbd_data.112d92027590.0000000000000024:f [read 1936384~1024 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:31:25.633 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:27.059899+0000", 2024-09-14T06:31:25.633 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 118.565641785, 2024-09-14T06:31:25.633 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046572122000000001, 2024-09-14T06:31:25.633 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:31:25.633 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:31:25.633 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:31:25.633 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:31:25.633 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:31:25.633 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:25.633 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 19329 2024-09-14T06:31:25.633 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:25.633 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:31:25.633 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:25.633 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:31:25.634 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.059899+0000", 2024-09-14T06:31:25.634 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:25.634 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:25.634 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:25.634 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:31:25.634 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.059899+0000", 2024-09-14T06:31:25.634 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:25.634 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:25.634 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:25.634 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:31:25.634 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062501+0000", 2024-09-14T06:31:25.634 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.0026016419999999999 2024-09-14T06:31:25.634 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:25.634 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:25.635 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:31:25.635 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062514+0000", 2024-09-14T06:31:25.635 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.2947e-05 2024-09-14T06:31:25.635 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:25.635 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:25.635 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:31:25.635 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062523+0000", 2024-09-14T06:31:25.635 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.5519999999999993e-06 2024-09-14T06:31:25.635 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:25.635 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:25.635 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:31:25.635 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062534+0000", 2024-09-14T06:31:25.635 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.1416e-05 2024-09-14T06:31:25.635 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:25.636 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:25.636 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:31:25.636 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.106179+0000", 2024-09-14T06:31:25.636 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.043644049999999997 2024-09-14T06:31:25.636 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:25.636 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:25.636 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:31:25.636 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.106262+0000", 2024-09-14T06:31:25.636 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.3178999999999998e-05 2024-09-14T06:31:25.636 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:25.636 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:25.636 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:31:25.636 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.106471+0000", 2024-09-14T06:31:25.637 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00020933599999999999 2024-09-14T06:31:25.637 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:25.637 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:31:25.637 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:25.637 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:25.637 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:25.637 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:19351 3.3 3:c34e5e51:::rbd_data.112d92027590.0000000000000024:f [read 1159168~1024 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:31:25.637 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:27.062766+0000", 2024-09-14T06:31:25.637 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 118.56277455999999, 2024-09-14T06:31:25.637 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046099454999999998, 2024-09-14T06:31:25.637 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:31:25.637 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:31:25.637 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:31:25.637 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:31:25.638 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:31:25.638 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:25.638 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 19351 2024-09-14T06:31:25.638 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:25.638 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:31:25.638 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:25.638 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:31:25.638 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062766+0000", 2024-09-14T06:31:25.638 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:25.638 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:25.638 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:25.639 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:31:25.639 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062766+0000", 2024-09-14T06:31:25.639 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:25.639 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:25.639 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:25.639 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:31:25.639 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062766+0000", 2024-09-14T06:31:25.639 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.6e-07 2024-09-14T06:31:25.639 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:25.639 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:25.639 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:31:25.639 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062767+0000", 2024-09-14T06:31:25.639 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.0070000000000001e-06 2024-09-14T06:31:25.639 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:25.640 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:25.640 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:31:25.640 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062785+0000", 2024-09-14T06:31:25.640 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.7884999999999999e-05 2024-09-14T06:31:25.640 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:25.640 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:25.640 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:31:25.640 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062790+0000", 2024-09-14T06:31:25.640 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.0900000000000004e-06 2024-09-14T06:31:25.640 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:25.640 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:25.640 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:31:25.640 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.108735+0000", 2024-09-14T06:31:25.641 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.045944411999999997 2024-09-14T06:31:25.641 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:25.641 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:25.641 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:31:25.641 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.108809+0000", 2024-09-14T06:31:25.641 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.4157000000000002e-05 2024-09-14T06:31:25.641 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:25.641 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:25.641 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:31:25.641 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.108866+0000", 2024-09-14T06:31:25.641 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.6644000000000002e-05 2024-09-14T06:31:25.641 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:25.642 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:31:25.642 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:25.642 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:25.642 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:25.642 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:19352 3.3 3:c34e5e51:::rbd_data.112d92027590.0000000000000024:f [read 1169408~1024 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:31:25.642 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:27.062798+0000", 2024-09-14T06:31:25.642 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 118.562743149, 2024-09-14T06:31:25.642 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046255900000000003, 2024-09-14T06:31:25.642 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:31:25.642 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:31:25.642 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:31:25.642 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:31:25.643 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:31:25.643 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:25.643 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 19352 2024-09-14T06:31:25.643 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:25.643 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:31:25.643 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:25.643 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:31:25.643 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062798+0000", 2024-09-14T06:31:25.643 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:25.643 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:25.643 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:25.643 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:31:25.643 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062798+0000", 2024-09-14T06:31:25.643 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:25.644 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:25.644 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:25.644 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:31:25.644 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062798+0000", 2024-09-14T06:31:25.644 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.37e-07 2024-09-14T06:31:25.644 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:25.644 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:25.644 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:31:25.644 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062800+0000", 2024-09-14T06:31:25.644 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.0269999999999998e-06 2024-09-14T06:31:25.644 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:25.644 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:25.644 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:31:25.644 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062806+0000", 2024-09-14T06:31:25.644 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.6690000000000003e-06 2024-09-14T06:31:25.645 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:25.645 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:25.645 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:31:25.645 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062810+0000", 2024-09-14T06:31:25.645 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.3449999999999999e-06 2024-09-14T06:31:25.645 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:25.645 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:25.645 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:31:25.645 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.108892+0000", 2024-09-14T06:31:25.645 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046081715000000002 2024-09-14T06:31:25.645 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:25.645 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:25.645 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:31:25.646 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.108940+0000", 2024-09-14T06:31:25.646 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.7815999999999998e-05 2024-09-14T06:31:25.646 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:25.646 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:25.646 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:31:25.646 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.109053+0000", 2024-09-14T06:31:25.646 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.000113791 2024-09-14T06:31:25.646 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:25.646 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:31:25.646 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:25.646 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:25.646 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:25.646 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:19353 3.3 3:c34e5e51:::rbd_data.112d92027590.0000000000000024:f [read 1179648~1024 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:31:25.646 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:27.062816+0000", 2024-09-14T06:31:25.646 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 118.56272463000001, 2024-09-14T06:31:25.647 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046481256999999998, 2024-09-14T06:31:25.647 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:31:25.647 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:31:25.647 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:31:25.647 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:31:25.647 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:31:25.647 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:25.647 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 19353 2024-09-14T06:31:25.647 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:25.647 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:31:25.647 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:25.647 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:31:25.647 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062816+0000", 2024-09-14T06:31:25.647 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:25.648 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:25.648 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:25.648 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:31:25.648 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062816+0000", 2024-09-14T06:31:25.648 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:25.648 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:25.648 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:25.648 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:31:25.648 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062816+0000", 2024-09-14T06:31:25.648 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.2899999999999999e-07 2024-09-14T06:31:25.648 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:25.648 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:25.648 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:31:25.648 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062818+0000", 2024-09-14T06:31:25.649 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.699e-06 2024-09-14T06:31:25.649 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:25.649 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:25.649 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:31:25.649 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062825+0000", 2024-09-14T06:31:25.649 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.878e-06 2024-09-14T06:31:25.649 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:25.649 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:25.649 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:31:25.650 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062830+0000", 2024-09-14T06:31:25.650 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.5410000000000002e-06 2024-09-14T06:31:25.650 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:25.650 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:25.650 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:31:25.650 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.109096+0000", 2024-09-14T06:31:25.650 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046266517 2024-09-14T06:31:25.650 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:25.650 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:25.650 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:31:25.650 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.109190+0000", 2024-09-14T06:31:25.650 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.4000999999999997e-05 2024-09-14T06:31:25.650 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:25.650 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:25.651 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:31:25.651 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.109297+0000", 2024-09-14T06:31:25.651 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.000107292 2024-09-14T06:31:25.651 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:25.651 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:31:25.651 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:25.651 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:25.651 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:25.651 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:19354 3.3 3:c34e5e51:::rbd_data.112d92027590.0000000000000024:f [read 1189888~1024 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:31:25.651 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:27.062842+0000", 2024-09-14T06:31:25.651 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 118.56269917500001, 2024-09-14T06:31:25.651 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046594257, 2024-09-14T06:31:25.651 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:31:25.651 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:31:25.652 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:31:25.652 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:31:25.652 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:31:25.652 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:25.652 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 19354 2024-09-14T06:31:25.652 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:25.652 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:31:25.652 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:25.652 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:31:25.652 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062842+0000", 2024-09-14T06:31:25.652 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:25.652 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:25.653 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:25.653 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:31:25.653 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062842+0000", 2024-09-14T06:31:25.653 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:25.653 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:25.653 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:25.653 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:31:25.653 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062842+0000", 2024-09-14T06:31:25.653 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.7399999999999999e-07 2024-09-14T06:31:25.653 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:25.653 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:25.653 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:31:25.653 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062859+0000", 2024-09-14T06:31:25.653 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.7010000000000001e-05 2024-09-14T06:31:25.654 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:25.654 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:25.654 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:31:25.654 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062866+0000", 2024-09-14T06:31:25.654 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.9990000000000002e-06 2024-09-14T06:31:25.654 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:25.654 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:25.654 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:31:25.654 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062873+0000", 2024-09-14T06:31:25.654 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.8569999999999997e-06 2024-09-14T06:31:25.654 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:25.654 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:25.654 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:31:25.654 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.109335+0000", 2024-09-14T06:31:25.655 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046461943999999998 2024-09-14T06:31:25.655 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:25.655 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:25.655 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:31:25.655 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.109377+0000", 2024-09-14T06:31:25.655 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.2558999999999997e-05 2024-09-14T06:31:25.655 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:25.655 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:25.655 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:31:25.655 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.109436+0000", 2024-09-14T06:31:25.655 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.8514000000000003e-05 2024-09-14T06:31:25.655 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:25.655 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:31:25.656 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:25.656 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:25.656 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:25.656 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:19355 3.3 3:c34e5e51:::rbd_data.112d92027590.0000000000000024:f [read 1200128~1024 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:31:25.656 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:27.062884+0000", 2024-09-14T06:31:25.656 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 118.562656502, 2024-09-14T06:31:25.656 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046681650999999998, 2024-09-14T06:31:25.656 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:31:25.656 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:31:25.656 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:31:25.657 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:31:25.657 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:31:25.657 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:25.657 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 19355 2024-09-14T06:31:25.657 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:25.657 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:31:25.657 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:25.657 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:31:25.657 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062884+0000", 2024-09-14T06:31:25.657 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:25.657 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:25.657 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:25.657 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:31:25.658 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062884+0000", 2024-09-14T06:31:25.658 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:25.658 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:25.658 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:25.658 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:31:25.658 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062885+0000", 2024-09-14T06:31:25.658 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.8000000000000001e-07 2024-09-14T06:31:25.658 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:25.658 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:25.658 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:31:25.658 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062886+0000", 2024-09-14T06:31:25.658 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.8920000000000001e-06 2024-09-14T06:31:25.658 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:25.659 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:25.659 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:31:25.659 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062892+0000", 2024-09-14T06:31:25.659 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.84e-06 2024-09-14T06:31:25.659 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:25.659 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:25.659 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:31:25.659 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062899+0000", 2024-09-14T06:31:25.659 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.5339999999999996e-06 2024-09-14T06:31:25.659 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:25.659 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:25.659 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:31:25.659 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.109466+0000", 2024-09-14T06:31:25.659 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046566919999999998 2024-09-14T06:31:25.660 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:25.660 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:25.660 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:31:25.660 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.109510+0000", 2024-09-14T06:31:25.660 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.4153000000000002e-05 2024-09-14T06:31:25.660 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:25.660 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:25.660 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:31:25.660 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.109566+0000", 2024-09-14T06:31:25.660 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.5931999999999999e-05 2024-09-14T06:31:25.660 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:25.660 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:31:25.660 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:25.660 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:25.661 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:25.661 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:19356 3.3 3:c34e5e51:::rbd_data.112d92027590.0000000000000024:f [read 1210368~1024 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:31:25.661 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:27.062908+0000", 2024-09-14T06:31:25.661 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 118.562633065, 2024-09-14T06:31:25.661 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046807498000000003, 2024-09-14T06:31:25.661 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:31:25.661 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:31:25.661 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:31:25.661 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:31:25.661 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:31:25.661 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:25.661 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 19356 2024-09-14T06:31:25.661 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:25.662 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:31:25.662 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:25.662 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:31:25.662 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062908+0000", 2024-09-14T06:31:25.662 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:25.662 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:25.662 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:25.662 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:31:25.662 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062908+0000", 2024-09-14T06:31:25.662 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:25.662 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:25.662 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:25.662 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:31:25.662 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062908+0000", 2024-09-14T06:31:25.663 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.5699999999999998e-07 2024-09-14T06:31:25.663 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:25.663 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:25.663 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:31:25.663 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062910+0000", 2024-09-14T06:31:25.663 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.7710000000000001e-06 2024-09-14T06:31:25.663 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:25.663 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:25.663 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:31:25.663 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062920+0000", 2024-09-14T06:31:25.663 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.0654e-05 2024-09-14T06:31:25.663 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:25.663 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:25.664 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:31:25.664 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062931+0000", 2024-09-14T06:31:25.664 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.075e-05 2024-09-14T06:31:25.664 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:25.664 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:25.664 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:31:25.665 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.109595+0000", 2024-09-14T06:31:25.665 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046663713000000002 2024-09-14T06:31:25.665 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:25.665 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:25.665 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:31:25.665 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.109632+0000", 2024-09-14T06:31:25.665 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.7505000000000001e-05 2024-09-14T06:31:25.666 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:25.666 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:25.666 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:31:25.666 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.109715+0000", 2024-09-14T06:31:25.666 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.2748000000000001e-05 2024-09-14T06:31:25.666 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:25.666 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:31:25.667 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:25.667 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:25.667 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:25.667 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:19357 3.3 3:c34e5e51:::rbd_data.112d92027590.0000000000000024:f [read 1220608~1024 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:31:25.667 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:27.062939+0000", 2024-09-14T06:31:25.667 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 118.562601617, 2024-09-14T06:31:25.667 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046920508, 2024-09-14T06:31:25.668 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:31:25.668 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:31:25.668 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:31:25.668 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:31:25.668 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:31:25.668 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:25.668 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 19357 2024-09-14T06:31:25.668 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:25.669 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:31:25.669 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:25.669 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:31:25.669 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062939+0000", 2024-09-14T06:31:25.669 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:25.669 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:25.670 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:25.670 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:31:25.670 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062939+0000", 2024-09-14T06:31:25.670 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:25.670 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:25.670 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:25.671 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:31:25.671 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062940+0000", 2024-09-14T06:31:25.671 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.32e-07 2024-09-14T06:31:25.671 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:25.671 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:25.671 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:31:25.672 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062942+0000", 2024-09-14T06:31:25.672 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.1919999999999999e-06 2024-09-14T06:31:25.672 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:25.672 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:25.672 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:31:25.672 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062948+0000", 2024-09-14T06:31:25.672 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.1149999999999999e-06 2024-09-14T06:31:25.672 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:25.672 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:25.673 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:31:25.673 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062960+0000", 2024-09-14T06:31:25.673 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.1895e-05 2024-09-14T06:31:25.673 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:25.673 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:25.673 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:31:25.673 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.109704+0000", 2024-09-14T06:31:25.673 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046744584999999998 2024-09-14T06:31:25.673 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:25.673 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:25.673 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:31:25.673 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.109777+0000", 2024-09-14T06:31:25.674 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.2282999999999999e-05 2024-09-14T06:31:25.674 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:25.674 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:25.674 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:31:25.674 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.109860+0000", 2024-09-14T06:31:25.674 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.3005999999999998e-05 2024-09-14T06:31:25.674 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:25.674 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:31:25.674 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:25.674 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:25.674 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:25.674 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:19358 3.3 3:c34e5e51:::rbd_data.112d92027590.0000000000000024:f [read 1230848~1024 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:31:25.674 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:27.062967+0000", 2024-09-14T06:31:25.674 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 118.56257398699999, 2024-09-14T06:31:25.675 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047144471, 2024-09-14T06:31:25.675 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:31:25.675 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:31:25.675 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:31:25.675 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:31:25.675 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:31:25.675 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:25.675 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 19358 2024-09-14T06:31:25.675 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:25.675 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:31:25.675 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:25.675 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:31:25.675 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062967+0000", 2024-09-14T06:31:25.675 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:25.675 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:25.675 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:25.676 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:31:25.676 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062967+0000", 2024-09-14T06:31:25.676 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:25.676 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:25.676 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:25.676 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:31:25.676 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062967+0000", 2024-09-14T06:31:25.676 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.89e-07 2024-09-14T06:31:25.676 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:25.676 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:25.676 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:31:25.676 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062969+0000", 2024-09-14T06:31:25.676 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.9929999999999998e-06 2024-09-14T06:31:25.676 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:25.677 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:25.677 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:31:25.677 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062975+0000", 2024-09-14T06:31:25.677 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.9030000000000001e-06 2024-09-14T06:31:25.677 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:25.677 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:25.677 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:31:25.677 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062979+0000", 2024-09-14T06:31:25.677 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.2250000000000002e-06 2024-09-14T06:31:25.677 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:25.677 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:25.678 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:31:25.678 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.109934+0000", 2024-09-14T06:31:25.678 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046954410000000002 2024-09-14T06:31:25.678 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:25.678 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:25.678 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:31:25.678 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.110007+0000", 2024-09-14T06:31:25.678 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.3829000000000001e-05 2024-09-14T06:31:25.678 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:25.678 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:25.678 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:31:25.678 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.110111+0000", 2024-09-14T06:31:25.678 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00010372199999999999 2024-09-14T06:31:25.678 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:25.678 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:31:25.679 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:25.679 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:25.679 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:25.679 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:19359 3.3 3:c34e5e51:::rbd_data.112d92027590.0000000000000024:f [read 1241088~1024 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:31:25.679 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:27.062987+0000", 2024-09-14T06:31:25.679 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 118.562553897, 2024-09-14T06:31:25.679 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047297182, 2024-09-14T06:31:25.679 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:31:25.679 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:31:25.679 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:31:25.679 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:31:25.680 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:31:25.680 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:25.680 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 19359 2024-09-14T06:31:25.680 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:25.680 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:31:25.680 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:25.680 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:31:25.680 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062987+0000", 2024-09-14T06:31:25.680 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:25.680 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:25.681 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:25.681 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:31:25.681 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062987+0000", 2024-09-14T06:31:25.681 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:25.681 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:25.681 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:25.681 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:31:25.681 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062987+0000", 2024-09-14T06:31:25.681 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.2500000000000001e-07 2024-09-14T06:31:25.681 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:25.681 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:25.681 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:31:25.682 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062989+0000", 2024-09-14T06:31:25.682 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.6080000000000001e-06 2024-09-14T06:31:25.682 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:25.682 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:25.682 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:31:25.682 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062994+0000", 2024-09-14T06:31:25.682 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.5520000000000004e-06 2024-09-14T06:31:25.682 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:25.682 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:25.682 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:31:25.682 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062999+0000", 2024-09-14T06:31:25.682 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.3930000000000001e-06 2024-09-14T06:31:25.683 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:25.683 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:25.683 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:31:25.683 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.110142+0000", 2024-09-14T06:31:25.683 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047143154 2024-09-14T06:31:25.683 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:25.683 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:25.683 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:31:25.683 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.110219+0000", 2024-09-14T06:31:25.683 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.6737999999999996e-05 2024-09-14T06:31:25.683 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:25.683 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:25.683 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:31:25.683 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.110284+0000", 2024-09-14T06:31:25.684 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.5412000000000003e-05 2024-09-14T06:31:25.684 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:25.684 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:31:25.684 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:25.684 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:25.684 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:25.684 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:19360 3.3 3:c34e5e51:::rbd_data.112d92027590.0000000000000024:f [read 1251328~1024 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:31:25.684 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:27.063017+0000", 2024-09-14T06:31:25.684 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 118.56252343600001, 2024-09-14T06:31:25.684 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047453823999999999, 2024-09-14T06:31:25.684 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:31:25.684 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:31:25.685 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:31:25.685 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:31:25.685 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:31:25.685 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:25.685 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 19360 2024-09-14T06:31:25.685 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:25.685 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:31:25.685 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:25.685 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:31:25.685 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.063017+0000", 2024-09-14T06:31:25.685 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:25.685 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:25.685 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:25.685 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:31:25.686 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.063017+0000", 2024-09-14T06:31:25.686 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:25.686 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:25.686 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:25.686 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:31:25.686 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.064977+0000", 2024-09-14T06:31:25.686 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.0019597080000000001 2024-09-14T06:31:25.686 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:25.686 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:25.686 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:31:25.686 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.064987+0000", 2024-09-14T06:31:25.686 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.9620000000000005e-06 2024-09-14T06:31:25.686 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:25.686 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:25.686 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:31:25.687 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065000+0000", 2024-09-14T06:31:25.687 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.3173000000000001e-05 2024-09-14T06:31:25.687 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:25.687 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:25.687 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:31:25.687 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065005+0000", 2024-09-14T06:31:25.687 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.9960000000000001e-06 2024-09-14T06:31:25.687 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:25.687 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:25.687 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:31:25.687 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.110314+0000", 2024-09-14T06:31:25.687 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.04530899 2024-09-14T06:31:25.687 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:25.687 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:25.687 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:31:25.688 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.110398+0000", 2024-09-14T06:31:25.688 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.3539000000000001e-05 2024-09-14T06:31:25.688 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:25.688 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:25.688 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:31:25.688 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.110471+0000", 2024-09-14T06:31:25.688 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.3455999999999996e-05 2024-09-14T06:31:25.688 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:25.688 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:31:25.688 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:25.688 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:25.688 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:25.688 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:19365 3.3 3:c34e5e51:::rbd_data.112d92027590.0000000000000024:f [read 1302528~1024 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:31:25.688 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:27.065058+0000", 2024-09-14T06:31:25.689 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 118.56048276600001, 2024-09-14T06:31:25.689 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046105526000000001, 2024-09-14T06:31:25.689 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:31:25.689 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:31:25.689 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:31:25.689 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:31:25.689 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:31:25.689 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:25.689 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 19365 2024-09-14T06:31:25.689 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:25.689 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:31:25.689 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:25.689 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:31:25.689 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065058+0000", 2024-09-14T06:31:25.690 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:25.690 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:25.690 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:25.690 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:31:25.690 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065058+0000", 2024-09-14T06:31:25.690 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:25.690 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:25.690 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:25.690 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:31:25.690 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065058+0000", 2024-09-14T06:31:25.690 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.1899999999999999e-07 2024-09-14T06:31:25.690 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:25.690 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:25.690 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:31:25.690 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065059+0000", 2024-09-14T06:31:25.691 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.1299999999999998e-07 2024-09-14T06:31:25.691 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:25.691 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:25.691 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:31:25.691 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065063+0000", 2024-09-14T06:31:25.691 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.1099999999999996e-06 2024-09-14T06:31:25.691 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:25.691 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:25.691 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:31:25.691 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065070+0000", 2024-09-14T06:31:25.691 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.973e-06 2024-09-14T06:31:25.691 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:25.691 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:25.691 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:31:25.692 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.111014+0000", 2024-09-14T06:31:25.692 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.045943373000000003 2024-09-14T06:31:25.692 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:25.692 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:25.692 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:31:25.692 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.111073+0000", 2024-09-14T06:31:25.692 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.8987999999999998e-05 2024-09-14T06:31:25.692 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:25.692 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:25.692 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:31:25.692 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.111163+0000", 2024-09-14T06:31:25.692 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.0950000000000002e-05 2024-09-14T06:31:25.693 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:25.693 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:31:25.693 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:25.693 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:25.693 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:25.693 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:19366 3.3 3:c34e5e51:::rbd_data.112d92027590.0000000000000024:f [read 1312768~1024 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:31:25.693 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:27.065074+0000", 2024-09-14T06:31:25.693 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 118.560466916, 2024-09-14T06:31:25.693 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046289858000000003, 2024-09-14T06:31:25.693 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:31:25.693 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:31:25.693 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:31:25.694 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:31:25.694 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:31:25.694 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:25.694 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 19366 2024-09-14T06:31:25.694 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:25.694 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:31:25.694 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:25.694 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:31:25.694 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065074+0000", 2024-09-14T06:31:25.694 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:25.694 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:25.694 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:25.694 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:31:25.694 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065074+0000", 2024-09-14T06:31:25.694 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:25.695 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:25.695 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:25.695 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:31:25.695 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065074+0000", 2024-09-14T06:31:25.695 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.3900000000000001e-07 2024-09-14T06:31:25.695 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:25.695 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:25.695 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:31:25.695 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065078+0000", 2024-09-14T06:31:25.695 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.3229999999999999e-06 2024-09-14T06:31:25.695 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:25.695 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:25.695 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:31:25.696 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065082+0000", 2024-09-14T06:31:25.696 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.8299999999999998e-06 2024-09-14T06:31:25.696 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:25.696 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:25.696 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:31:25.696 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065085+0000", 2024-09-14T06:31:25.696 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.8660000000000002e-06 2024-09-14T06:31:25.696 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:25.696 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:25.696 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:31:25.696 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.111270+0000", 2024-09-14T06:31:25.696 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046185035999999999 2024-09-14T06:31:25.696 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:25.697 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:25.697 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:31:25.697 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.111311+0000", 2024-09-14T06:31:25.697 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.0935e-05 2024-09-14T06:31:25.697 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:25.697 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:25.697 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:31:25.697 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.111364+0000", 2024-09-14T06:31:25.697 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.2629e-05 2024-09-14T06:31:25.697 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:25.697 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:31:25.697 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:25.697 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:25.697 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:25.698 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:19367 3.3 3:c34e5e51:::rbd_data.112d92027590.0000000000000024:f [read 1323008~1024 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:31:25.698 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:27.065115+0000", 2024-09-14T06:31:25.698 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 118.56042535500001, 2024-09-14T06:31:25.698 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046407003000000002, 2024-09-14T06:31:25.698 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:31:25.698 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:31:25.698 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:31:25.698 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:31:25.698 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:31:25.698 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:25.698 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 19367 2024-09-14T06:31:25.698 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:25.698 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:31:25.698 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:25.699 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:31:25.699 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065115+0000", 2024-09-14T06:31:25.699 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:25.699 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:25.699 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:25.699 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:31:25.699 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065115+0000", 2024-09-14T06:31:25.699 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:25.699 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:25.699 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:25.699 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:31:25.700 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065116+0000", 2024-09-14T06:31:25.700 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.4999999999999998e-07 2024-09-14T06:31:25.700 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:25.700 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:25.700 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:31:25.700 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065117+0000", 2024-09-14T06:31:25.700 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.189e-06 2024-09-14T06:31:25.700 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:25.700 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:25.700 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:31:25.701 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065122+0000", 2024-09-14T06:31:25.701 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.3709999999999999e-06 2024-09-14T06:31:25.701 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:25.701 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:25.701 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:31:25.701 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065125+0000", 2024-09-14T06:31:25.701 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.8940000000000002e-06 2024-09-14T06:31:25.701 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:25.701 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:25.701 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:31:25.701 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.111402+0000", 2024-09-14T06:31:25.701 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046276645999999998 2024-09-14T06:31:25.701 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:25.702 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:25.702 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:31:25.702 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.111445+0000", 2024-09-14T06:31:25.702 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.3288000000000003e-05 2024-09-14T06:31:25.702 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:25.702 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:25.702 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:31:25.702 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.111522+0000", 2024-09-14T06:31:25.702 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.7164999999999997e-05 2024-09-14T06:31:25.702 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:25.702 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:31:25.702 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:25.702 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:25.703 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:25.703 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:19368 3.3 3:c34e5e51:::rbd_data.112d92027590.0000000000000024:f [read 1333248~1024 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:31:25.703 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:27.065130+0000", 2024-09-14T06:31:25.703 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 118.560411065, 2024-09-14T06:31:25.703 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046507275000000001, 2024-09-14T06:31:25.703 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:31:25.703 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:31:25.703 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:31:25.703 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:31:25.703 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:31:25.703 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:25.703 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 19368 2024-09-14T06:31:25.703 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:25.704 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:31:25.704 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:25.704 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:31:25.704 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065130+0000", 2024-09-14T06:31:25.704 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:25.704 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:25.704 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:25.704 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:31:25.704 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065130+0000", 2024-09-14T06:31:25.704 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:25.704 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:25.704 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:25.704 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:31:25.704 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065130+0000", 2024-09-14T06:31:25.705 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.0800000000000001e-07 2024-09-14T06:31:25.705 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:25.705 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:25.705 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:31:25.705 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065131+0000", 2024-09-14T06:31:25.705 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.8700000000000004e-07 2024-09-14T06:31:25.705 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:25.705 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:25.705 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:31:25.705 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065134+0000", 2024-09-14T06:31:25.705 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.1039999999999998e-06 2024-09-14T06:31:25.705 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:25.705 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:25.705 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:31:25.706 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065136+0000", 2024-09-14T06:31:25.706 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.3949999999999999e-06 2024-09-14T06:31:25.706 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:25.706 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:25.706 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:31:25.706 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.111541+0000", 2024-09-14T06:31:25.706 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046404206000000003 2024-09-14T06:31:25.706 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:25.706 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:25.706 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:31:25.706 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.111582+0000", 2024-09-14T06:31:25.706 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.1051999999999999e-05 2024-09-14T06:31:25.706 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:25.707 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:25.707 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:31:25.707 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.111637+0000", 2024-09-14T06:31:25.707 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.5322999999999999e-05 2024-09-14T06:31:25.707 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:25.707 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:31:25.707 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:25.707 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:25.707 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:25.707 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:19369 3.3 3:c34e5e51:::rbd_data.112d92027590.0000000000000024:f [read 1343488~1024 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:31:25.707 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:27.065139+0000", 2024-09-14T06:31:25.707 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 118.56040124800001, 2024-09-14T06:31:25.707 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046682169000000003, 2024-09-14T06:31:25.708 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:31:25.708 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:31:25.708 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:31:25.708 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:31:25.708 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:31:25.708 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:25.708 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 19369 2024-09-14T06:31:25.708 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:25.708 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:31:25.708 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:25.708 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:31:25.708 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065139+0000", 2024-09-14T06:31:25.708 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:25.709 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:25.709 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:25.709 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:31:25.709 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065139+0000", 2024-09-14T06:31:25.709 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:25.709 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:25.709 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:25.709 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:31:25.709 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065140+0000", 2024-09-14T06:31:25.709 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.8299999999999998e-07 2024-09-14T06:31:25.709 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:25.709 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:25.709 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:31:25.710 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065141+0000", 2024-09-14T06:31:25.710 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.3900000000000003e-07 2024-09-14T06:31:25.710 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:25.710 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:25.710 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:31:25.710 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065144+0000", 2024-09-14T06:31:25.710 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.7409999999999998e-06 2024-09-14T06:31:25.710 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:25.710 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:25.710 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:31:25.710 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065147+0000", 2024-09-14T06:31:25.710 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.4150000000000002e-06 2024-09-14T06:31:25.710 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:25.710 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:25.710 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:31:25.711 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.111673+0000", 2024-09-14T06:31:25.711 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046526004000000003 2024-09-14T06:31:25.711 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:25.711 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:25.711 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:31:25.711 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.111749+0000", 2024-09-14T06:31:25.711 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.5741999999999994e-05 2024-09-14T06:31:25.711 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:25.711 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:25.711 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:31:25.711 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.111822+0000", 2024-09-14T06:31:25.711 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.3045000000000003e-05 2024-09-14T06:31:25.711 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:25.711 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:31:25.711 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:25.712 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:25.712 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:25.712 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:19370 3.3 3:c34e5e51:::rbd_data.112d92027590.0000000000000024:f [read 1353728~1024 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:31:25.712 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:27.065150+0000", 2024-09-14T06:31:25.712 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 118.56039051899999, 2024-09-14T06:31:25.712 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046897556999999999, 2024-09-14T06:31:25.712 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:31:25.712 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:31:25.712 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:31:25.712 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:31:25.712 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:31:25.712 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:25.712 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 19370 2024-09-14T06:31:25.712 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:25.713 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:31:25.713 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:25.713 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:31:25.713 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065150+0000", 2024-09-14T06:31:25.713 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:25.713 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:25.713 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:25.713 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:31:25.713 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065150+0000", 2024-09-14T06:31:25.713 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:25.713 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:25.713 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:25.713 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:31:25.714 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065150+0000", 2024-09-14T06:31:25.714 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.0200000000000001e-07 2024-09-14T06:31:25.714 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:25.714 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:25.714 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:31:25.714 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065151+0000", 2024-09-14T06:31:25.714 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.1100000000000004e-07 2024-09-14T06:31:25.714 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:25.714 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:25.714 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:31:25.714 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065154+0000", 2024-09-14T06:31:25.714 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.9639999999999999e-06 2024-09-14T06:31:25.714 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:25.714 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:25.714 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:31:25.715 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065157+0000", 2024-09-14T06:31:25.715 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.4260000000000002e-06 2024-09-14T06:31:25.715 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:25.715 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:25.715 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:31:25.715 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.111861+0000", 2024-09-14T06:31:25.715 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046704268 2024-09-14T06:31:25.715 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:25.715 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:25.715 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:31:25.715 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.111939+0000", 2024-09-14T06:31:25.715 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.8231000000000004e-05 2024-09-14T06:31:25.715 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:25.715 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:25.715 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:31:25.716 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.112048+0000", 2024-09-14T06:31:25.716 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00010855500000000001 2024-09-14T06:31:25.716 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:25.716 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:31:25.716 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:25.716 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:25.716 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:25.716 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:19371 3.3 3:c34e5e51:::rbd_data.112d92027590.0000000000000024:f [read 1363968~1024 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:31:25.716 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:27.065161+0000", 2024-09-14T06:31:25.716 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 118.560379869, 2024-09-14T06:31:25.716 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047021944000000003, 2024-09-14T06:31:25.716 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:31:25.716 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:31:25.717 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:31:25.717 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:31:25.717 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:31:25.717 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:25.717 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 19371 2024-09-14T06:31:25.717 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:25.717 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:31:25.717 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:25.717 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:31:25.717 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065161+0000", 2024-09-14T06:31:25.717 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:25.717 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:25.718 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:25.718 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:31:25.718 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065161+0000", 2024-09-14T06:31:25.718 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:25.718 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:25.718 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:25.718 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:31:25.718 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065161+0000", 2024-09-14T06:31:25.718 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.0200000000000001e-07 2024-09-14T06:31:25.718 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:25.718 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:25.718 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:31:25.718 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065166+0000", 2024-09-14T06:31:25.718 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.9869999999999999e-06 2024-09-14T06:31:25.719 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:25.719 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:25.719 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:31:25.719 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065169+0000", 2024-09-14T06:31:25.719 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.3799999999999998e-06 2024-09-14T06:31:25.719 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:25.719 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:25.719 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:31:25.720 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065172+0000", 2024-09-14T06:31:25.720 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.4830000000000002e-06 2024-09-14T06:31:25.720 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:25.720 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:25.720 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:31:25.720 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.112071+0000", 2024-09-14T06:31:25.720 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046899529000000002 2024-09-14T06:31:25.720 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:25.720 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:25.720 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:31:25.720 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.112128+0000", 2024-09-14T06:31:25.720 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.6341999999999997e-05 2024-09-14T06:31:25.720 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:25.721 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:25.721 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:31:25.721 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.112183+0000", 2024-09-14T06:31:25.721 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.5021000000000001e-05 2024-09-14T06:31:25.721 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:25.721 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:31:25.721 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:25.721 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:25.721 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:25.721 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:19372 3.3 3:c34e5e51:::rbd_data.112d92027590.0000000000000024:f [read 1374208~1024 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:31:25.721 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:27.065176+0000", 2024-09-14T06:31:25.721 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 118.56036471900001, 2024-09-14T06:31:25.721 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047188889999999997, 2024-09-14T06:31:25.721 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:31:25.722 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:31:25.722 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:31:25.722 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:31:25.722 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:31:25.722 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:25.722 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 19372 2024-09-14T06:31:25.722 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:25.722 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:31:25.722 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:25.722 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:31:25.722 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065176+0000", 2024-09-14T06:31:25.722 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:25.723 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:25.723 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:25.723 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:31:25.723 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065176+0000", 2024-09-14T06:31:25.723 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:25.723 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:25.723 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:25.723 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:31:25.723 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.071117+0000", 2024-09-14T06:31:25.723 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.0059404990000000001 2024-09-14T06:31:25.723 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:25.723 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:25.723 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:31:25.723 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.071120+0000", 2024-09-14T06:31:25.724 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.4340000000000001e-06 2024-09-14T06:31:25.724 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:25.724 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:25.724 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:31:25.724 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.071129+0000", 2024-09-14T06:31:25.724 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.8389999999999996e-06 2024-09-14T06:31:25.724 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:25.724 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:25.724 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:31:25.724 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.071134+0000", 2024-09-14T06:31:25.724 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.412e-06 2024-09-14T06:31:25.724 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:25.724 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:25.725 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:31:25.725 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.112221+0000", 2024-09-14T06:31:25.725 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.041086473999999998 2024-09-14T06:31:25.725 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:25.725 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:25.725 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:31:25.725 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.112299+0000", 2024-09-14T06:31:25.725 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.8746999999999997e-05 2024-09-14T06:31:25.725 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:25.725 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:25.725 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:31:25.725 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.112365+0000", 2024-09-14T06:31:25.725 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.5485000000000001e-05 2024-09-14T06:31:25.726 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:25.726 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:31:25.726 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:25.726 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:25.726 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:31:25.726 INFO:teuthology.orchestra.run.smithi138.stdout:} 2024-09-14T06:31:25.728 INFO:teuthology.orchestra.run.smithi094.stdout:checking clone #36, image image_client.0-clone36 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent37 2024-09-14T06:31:25.728 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-14T06:31:25.729 INFO:tasks.ceph.osd.3.smithi094.stderr:2024-09-14T06:31:25.726+0000 7feeeafe8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33655) UID: 0 2024-09-14T06:31:25.828 INFO:tasks.ceph.osd.5.smithi138.stderr:2024-09-14T06:31:25.826+0000 7fe13746f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33618) UID: 0 2024-09-14T06:31:25.832 INFO:teuthology.orchestra.run.smithi138.stdout:{ 2024-09-14T06:31:25.833 INFO:teuthology.orchestra.run.smithi138.stdout: "ops": [ 2024-09-14T06:31:25.833 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:25.833 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:200597 3.1c 3.3c56629c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e161)", 2024-09-14T06:31:25.833 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:31:25.759501+0000", 2024-09-14T06:31:25.833 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 0.072962041000000005, 2024-09-14T06:31:25.833 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.072981001000000004, 2024-09-14T06:31:25.833 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:31:25.833 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:31:25.833 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "waiting for peered", 2024-09-14T06:31:25.833 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:31:25.833 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:31:25.834 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:25.834 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 200597 2024-09-14T06:31:25.834 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:25.834 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:31:25.834 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:25.834 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:31:25.834 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:31:25.759501+0000", 2024-09-14T06:31:25.834 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:25.834 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:25.834 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:25.834 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:31:25.834 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:31:25.759501+0000", 2024-09-14T06:31:25.834 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:25.834 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:25.834 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:25.834 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:31:25.835 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:31:25.759501+0000", 2024-09-14T06:31:25.835 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.58e-07 2024-09-14T06:31:25.835 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:25.835 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:25.835 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:31:25.835 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:31:25.759505+0000", 2024-09-14T06:31:25.835 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.4410000000000002e-06 2024-09-14T06:31:25.835 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:25.835 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:25.835 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:31:25.835 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:31:25.759511+0000", 2024-09-14T06:31:25.836 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.981e-06 2024-09-14T06:31:25.836 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:25.836 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:25.836 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:31:25.836 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:31:25.759517+0000", 2024-09-14T06:31:25.836 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.7810000000000002e-06 2024-09-14T06:31:25.836 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:25.836 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:25.836 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:31:25.836 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:31:25.759572+0000", 2024-09-14T06:31:25.836 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.5594999999999999e-05 2024-09-14T06:31:25.836 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:25.836 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:25.836 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "waiting for peered", 2024-09-14T06:31:25.837 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:31:25.759583+0000", 2024-09-14T06:31:25.837 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.1207e-05 2024-09-14T06:31:25.837 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:25.837 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:31:25.837 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:25.837 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:25.837 INFO:teuthology.orchestra.run.smithi138.stdout: ], 2024-09-14T06:31:25.837 INFO:teuthology.orchestra.run.smithi138.stdout: "num_ops": 1 2024-09-14T06:31:25.837 INFO:teuthology.orchestra.run.smithi138.stdout:} 2024-09-14T06:31:25.841 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-14T06:31:25.929 INFO:tasks.ceph.osd.5.smithi138.stderr:2024-09-14T06:31:25.927+0000 7fe13746f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33618) UID: 0 2024-09-14T06:31:25.984 INFO:teuthology.orchestra.run.smithi138.stdout:{ 2024-09-14T06:31:25.984 INFO:teuthology.orchestra.run.smithi138.stdout: "ops": [], 2024-09-14T06:31:25.984 INFO:teuthology.orchestra.run.smithi138.stdout: "complaint_time": 30, 2024-09-14T06:31:25.984 INFO:teuthology.orchestra.run.smithi138.stdout: "num_blocked_ops": 0 2024-09-14T06:31:25.984 INFO:teuthology.orchestra.run.smithi138.stdout:} 2024-09-14T06:31:25.992 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-14T06:31:26.030 INFO:tasks.ceph.osd.3.smithi094.stderr:2024-09-14T06:31:26.029+0000 7feeeafe8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33655) UID: 0 2024-09-14T06:31:26.131 INFO:tasks.ceph.osd.0.smithi094.stderr:2024-09-14T06:31:26.130+0000 7f07c9333640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33636) UID: 0 2024-09-14T06:31:26.137 INFO:teuthology.orchestra.run.smithi138.stdout:{ 2024-09-14T06:31:26.137 INFO:teuthology.orchestra.run.smithi138.stdout: "size": 20, 2024-09-14T06:31:26.137 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 600, 2024-09-14T06:31:26.137 INFO:teuthology.orchestra.run.smithi138.stdout: "ops": [ 2024-09-14T06:31:26.137 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:26.137 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:11453 3.1d 3:bcad7303:::rbd_data.112d92027590.0000000000000001:f [read 94208~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:31:26.138 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:25.654375+0000", 2024-09-14T06:31:26.138 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 120.48075969999999, 2024-09-14T06:31:26.138 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052449318000000002, 2024-09-14T06:31:26.138 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:31:26.138 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:31:26.138 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:31:26.138 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:31:26.138 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:31:26.139 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:26.139 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 11453 2024-09-14T06:31:26.139 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:26.139 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:31:26.139 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:26.139 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:31:26.139 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654375+0000", 2024-09-14T06:31:26.139 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:26.139 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:26.139 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:26.139 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:31:26.140 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654375+0000", 2024-09-14T06:31:26.140 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:26.140 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:26.140 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:26.140 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:31:26.140 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654496+0000", 2024-09-14T06:31:26.140 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00012081 2024-09-14T06:31:26.140 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:26.140 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:26.140 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:31:26.140 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654497+0000", 2024-09-14T06:31:26.141 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.8789999999999999e-06 2024-09-14T06:31:26.141 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:26.141 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:26.141 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:31:26.141 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654501+0000", 2024-09-14T06:31:26.141 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.8999999999999999e-06 2024-09-14T06:31:26.141 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:26.141 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:26.141 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:31:26.141 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654505+0000", 2024-09-14T06:31:26.141 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.2229999999999999e-06 2024-09-14T06:31:26.141 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:26.142 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:26.142 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:31:26.142 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.705917+0000", 2024-09-14T06:31:26.142 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.051412501999999999 2024-09-14T06:31:26.142 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:26.142 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:26.142 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:31:26.142 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.705999+0000", 2024-09-14T06:31:26.142 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.1551999999999994e-05 2024-09-14T06:31:26.142 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:26.142 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:26.143 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:31:26.143 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.706824+0000", 2024-09-14T06:31:26.143 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00082545199999999996 2024-09-14T06:31:26.143 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:26.143 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:31:26.143 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:26.143 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:26.143 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:26.143 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:11454 3.1d 3:bcad7303:::rbd_data.112d92027590.0000000000000001:f [read 104448~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:31:26.143 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:25.654510+0000", 2024-09-14T06:31:26.143 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 120.48062491, 2024-09-14T06:31:26.144 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052610713000000003, 2024-09-14T06:31:26.144 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:31:26.144 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:31:26.144 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:31:26.144 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:31:26.144 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:31:26.144 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:26.144 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 11454 2024-09-14T06:31:26.144 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:26.144 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:31:26.144 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:26.144 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:31:26.145 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654510+0000", 2024-09-14T06:31:26.145 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:26.145 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:26.145 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:26.145 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:31:26.145 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654510+0000", 2024-09-14T06:31:26.145 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:26.145 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:26.145 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:26.145 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:31:26.145 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654825+0000", 2024-09-14T06:31:26.145 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00031522400000000002 2024-09-14T06:31:26.145 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:26.146 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:26.146 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:31:26.146 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654827+0000", 2024-09-14T06:31:26.146 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.9709999999999998e-06 2024-09-14T06:31:26.146 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:26.146 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:26.146 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:31:26.146 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654832+0000", 2024-09-14T06:31:26.147 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.4099999999999999e-06 2024-09-14T06:31:26.147 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:26.147 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:26.147 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:31:26.147 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654835+0000", 2024-09-14T06:31:26.147 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.14e-06 2024-09-14T06:31:26.147 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:26.147 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:26.147 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:31:26.147 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.706851+0000", 2024-09-14T06:31:26.148 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052015761000000001 2024-09-14T06:31:26.148 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:26.148 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:26.148 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:31:26.148 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.706934+0000", 2024-09-14T06:31:26.148 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.2484999999999994e-05 2024-09-14T06:31:26.148 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:26.148 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:26.148 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:31:26.149 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.707120+0000", 2024-09-14T06:31:26.149 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00018672200000000001 2024-09-14T06:31:26.149 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:26.149 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:31:26.149 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:26.149 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:26.149 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:26.150 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:11455 3.1d 3:bcad7303:::rbd_data.112d92027590.0000000000000001:f [read 114688~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:31:26.150 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:25.654841+0000", 2024-09-14T06:31:26.150 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 120.480293581, 2024-09-14T06:31:26.150 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052579898, 2024-09-14T06:31:26.150 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:31:26.150 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:31:26.150 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:31:26.150 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:31:26.151 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:31:26.151 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:26.151 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 11455 2024-09-14T06:31:26.151 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:26.151 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:31:26.151 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:26.151 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:31:26.151 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654841+0000", 2024-09-14T06:31:26.151 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:26.151 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:26.151 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:26.151 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:31:26.151 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654841+0000", 2024-09-14T06:31:26.152 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:26.152 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:26.152 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:26.152 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:31:26.152 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654841+0000", 2024-09-14T06:31:26.152 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.7300000000000002e-07 2024-09-14T06:31:26.152 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:26.152 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:26.152 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:31:26.152 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654845+0000", 2024-09-14T06:31:26.152 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.0520000000000003e-06 2024-09-14T06:31:26.152 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:26.152 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:26.152 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:31:26.153 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654849+0000", 2024-09-14T06:31:26.153 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.0459999999999999e-06 2024-09-14T06:31:26.153 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:26.153 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:26.153 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:31:26.153 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654852+0000", 2024-09-14T06:31:26.153 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.9830000000000001e-06 2024-09-14T06:31:26.153 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:26.153 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:26.153 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:31:26.153 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.707159+0000", 2024-09-14T06:31:26.153 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052306826000000001 2024-09-14T06:31:26.153 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:26.153 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:26.153 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:31:26.154 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.707242+0000", 2024-09-14T06:31:26.154 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.2595000000000005e-05 2024-09-14T06:31:26.154 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:26.154 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:26.154 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:31:26.154 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.707421+0000", 2024-09-14T06:31:26.154 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.000179123 2024-09-14T06:31:26.154 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:26.154 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:31:26.154 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:26.154 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:26.154 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:26.154 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:11456 3.1d 3:bcad7303:::rbd_data.112d92027590.0000000000000001:f [read 124928~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:31:26.154 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:25.654856+0000", 2024-09-14T06:31:26.154 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 120.480278735, 2024-09-14T06:31:26.155 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052801580000000001, 2024-09-14T06:31:26.155 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:31:26.155 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:31:26.155 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:31:26.155 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:31:26.155 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:31:26.155 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:26.155 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 11456 2024-09-14T06:31:26.155 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:26.155 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:31:26.155 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:26.155 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:31:26.155 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654856+0000", 2024-09-14T06:31:26.155 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:26.155 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:26.156 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:26.156 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:31:26.156 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654856+0000", 2024-09-14T06:31:26.156 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:26.156 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:26.156 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:26.156 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:31:26.156 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654856+0000", 2024-09-14T06:31:26.156 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.29e-07 2024-09-14T06:31:26.156 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:26.156 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:26.156 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:31:26.156 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654857+0000", 2024-09-14T06:31:26.156 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.187e-06 2024-09-14T06:31:26.157 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:26.157 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:26.157 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:31:26.157 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654861+0000", 2024-09-14T06:31:26.157 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.3850000000000002e-06 2024-09-14T06:31:26.157 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:26.157 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:26.157 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:31:26.157 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654863+0000", 2024-09-14T06:31:26.157 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.9500000000000001e-06 2024-09-14T06:31:26.157 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:26.157 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:26.157 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:31:26.157 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.707456+0000", 2024-09-14T06:31:26.157 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052592434 2024-09-14T06:31:26.158 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:26.158 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:26.158 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:31:26.158 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.707516+0000", 2024-09-14T06:31:26.158 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.0167999999999997e-05 2024-09-14T06:31:26.158 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:26.158 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:26.158 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:31:26.158 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.707657+0000", 2024-09-14T06:31:26.158 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00014122700000000001 2024-09-14T06:31:26.158 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:26.158 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:31:26.158 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:26.159 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:26.159 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:26.159 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:11457 3.1d 3:bcad7303:::rbd_data.112d92027590.0000000000000001:f [read 135168~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:31:26.159 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:25.654867+0000", 2024-09-14T06:31:26.159 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 120.48026785, 2024-09-14T06:31:26.159 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052921902, 2024-09-14T06:31:26.159 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:31:26.159 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:31:26.159 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:31:26.159 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:31:26.159 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:31:26.159 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:26.159 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 11457 2024-09-14T06:31:26.159 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:26.160 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:31:26.160 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:26.160 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:31:26.160 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654867+0000", 2024-09-14T06:31:26.160 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:26.160 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:26.160 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:26.160 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:31:26.160 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654867+0000", 2024-09-14T06:31:26.160 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:26.160 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:26.160 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:26.160 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:31:26.160 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654972+0000", 2024-09-14T06:31:26.161 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00010584 2024-09-14T06:31:26.161 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:26.161 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:26.161 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:31:26.161 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654975+0000", 2024-09-14T06:31:26.161 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.1169999999999998e-06 2024-09-14T06:31:26.161 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:26.161 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:26.161 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:31:26.161 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654979+0000", 2024-09-14T06:31:26.161 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.1960000000000001e-06 2024-09-14T06:31:26.161 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:26.161 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:26.161 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:31:26.162 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654982+0000", 2024-09-14T06:31:26.162 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.2990000000000001e-06 2024-09-14T06:31:26.162 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:26.162 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:26.162 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:31:26.162 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.707684+0000", 2024-09-14T06:31:26.162 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052701979000000003 2024-09-14T06:31:26.162 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:26.162 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:26.162 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:31:26.162 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.707728+0000", 2024-09-14T06:31:26.162 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.3529999999999998e-05 2024-09-14T06:31:26.162 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:26.162 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:26.163 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:31:26.163 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.707789+0000", 2024-09-14T06:31:26.163 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.0940999999999998e-05 2024-09-14T06:31:26.163 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:26.163 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:31:26.163 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:26.163 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:26.163 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:26.163 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:11458 3.1d 3:bcad7303:::rbd_data.112d92027590.0000000000000001:f [read 145408~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:31:26.163 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:25.654987+0000", 2024-09-14T06:31:26.163 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 120.480147452, 2024-09-14T06:31:26.163 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.05291655, 2024-09-14T06:31:26.164 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:31:26.164 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:31:26.164 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:31:26.164 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:31:26.164 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:31:26.164 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:26.164 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 11458 2024-09-14T06:31:26.164 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:26.164 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:31:26.164 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:26.164 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:31:26.164 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654987+0000", 2024-09-14T06:31:26.165 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:26.165 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:26.165 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:26.165 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:31:26.169 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654987+0000", 2024-09-14T06:31:26.169 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:26.169 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:26.169 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:26.169 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:31:26.169 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655077+0000", 2024-09-14T06:31:26.169 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.0253000000000005e-05 2024-09-14T06:31:26.169 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:26.169 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:26.170 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:31:26.170 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655079+0000", 2024-09-14T06:31:26.170 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.9989999999999998e-06 2024-09-14T06:31:26.170 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:26.170 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:26.170 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:31:26.170 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655083+0000", 2024-09-14T06:31:26.170 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.9280000000000003e-06 2024-09-14T06:31:26.170 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:26.170 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:26.170 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:31:26.170 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655086+0000", 2024-09-14T06:31:26.170 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.056e-06 2024-09-14T06:31:26.171 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:26.171 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:26.171 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:31:26.171 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.707805+0000", 2024-09-14T06:31:26.171 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052718627999999997 2024-09-14T06:31:26.171 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:26.171 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:26.171 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:31:26.171 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.707848+0000", 2024-09-14T06:31:26.171 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.2728999999999997e-05 2024-09-14T06:31:26.171 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:26.171 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:26.171 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:31:26.171 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.707904+0000", 2024-09-14T06:31:26.171 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.5957e-05 2024-09-14T06:31:26.171 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:26.172 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:31:26.172 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:26.172 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:26.172 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:26.172 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:11459 3.1d 3:bcad7303:::rbd_data.112d92027590.0000000000000001:f [read 155648~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:31:26.172 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:25.655091+0000", 2024-09-14T06:31:26.172 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 120.480043444, 2024-09-14T06:31:26.172 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052944242000000002, 2024-09-14T06:31:26.172 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:31:26.172 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:31:26.172 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:31:26.172 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:31:26.172 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:31:26.173 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:26.173 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 11459 2024-09-14T06:31:26.173 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:26.173 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:31:26.173 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:26.173 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:31:26.173 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655091+0000", 2024-09-14T06:31:26.173 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:26.173 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:26.173 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:26.173 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:31:26.173 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655091+0000", 2024-09-14T06:31:26.173 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:26.173 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:26.174 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:26.174 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:31:26.174 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655237+0000", 2024-09-14T06:31:26.174 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00014570100000000001 2024-09-14T06:31:26.174 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:26.174 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:26.174 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:31:26.174 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655239+0000", 2024-09-14T06:31:26.174 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.8670000000000001e-06 2024-09-14T06:31:26.174 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:26.174 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:26.174 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:31:26.174 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655243+0000", 2024-09-14T06:31:26.174 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.9829999999999998e-06 2024-09-14T06:31:26.175 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:26.175 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:26.175 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:31:26.175 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655246+0000", 2024-09-14T06:31:26.175 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.1540000000000002e-06 2024-09-14T06:31:26.175 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:26.175 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:26.175 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:31:26.175 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.707934+0000", 2024-09-14T06:31:26.175 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052688286000000001 2024-09-14T06:31:26.176 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:26.176 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:26.176 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:31:26.176 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.707977+0000", 2024-09-14T06:31:26.176 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.3291999999999998e-05 2024-09-14T06:31:26.176 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:26.176 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:26.176 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:31:26.176 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.708035+0000", 2024-09-14T06:31:26.176 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.7958999999999999e-05 2024-09-14T06:31:26.176 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:26.176 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:31:26.177 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:26.177 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:26.177 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:26.177 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:11460 3.1d 3:bcad7303:::rbd_data.112d92027590.0000000000000001:f [read 165888~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:31:26.177 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:25.655250+0000", 2024-09-14T06:31:26.177 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 120.47988414700001, 2024-09-14T06:31:26.177 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052910698999999999, 2024-09-14T06:31:26.177 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:31:26.177 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:31:26.177 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:31:26.177 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:31:26.177 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:31:26.178 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:26.178 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 11460 2024-09-14T06:31:26.178 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:26.178 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:31:26.178 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:26.178 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:31:26.178 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655250+0000", 2024-09-14T06:31:26.178 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:26.179 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:26.179 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:26.179 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:31:26.179 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655250+0000", 2024-09-14T06:31:26.179 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:26.179 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:26.179 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:26.179 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:31:26.179 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655356+0000", 2024-09-14T06:31:26.179 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00010535900000000001 2024-09-14T06:31:26.179 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:26.180 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:26.180 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:31:26.180 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655357+0000", 2024-09-14T06:31:26.180 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.8080000000000001e-06 2024-09-14T06:31:26.180 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:26.180 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:26.180 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:31:26.180 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655361+0000", 2024-09-14T06:31:26.180 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.861e-06 2024-09-14T06:31:26.180 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:26.180 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:26.180 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:31:26.181 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655364+0000", 2024-09-14T06:31:26.181 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.1089999999999998e-06 2024-09-14T06:31:26.181 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:26.181 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:26.181 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:31:26.181 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.708065+0000", 2024-09-14T06:31:26.181 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052700595000000003 2024-09-14T06:31:26.181 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:26.181 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:26.181 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:31:26.182 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.708108+0000", 2024-09-14T06:31:26.182 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.2484999999999997e-05 2024-09-14T06:31:26.182 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:26.182 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:26.182 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:31:26.182 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.708161+0000", 2024-09-14T06:31:26.182 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.3482000000000001e-05 2024-09-14T06:31:26.182 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:26.182 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:31:26.183 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:26.183 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:26.183 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:26.183 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:11461 3.1d 3:bcad7303:::rbd_data.112d92027590.0000000000000001:f [read 176128~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:31:26.183 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:25.655370+0000", 2024-09-14T06:31:26.183 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 120.47976486500001, 2024-09-14T06:31:26.183 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052915141999999998, 2024-09-14T06:31:26.183 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:31:26.183 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:31:26.183 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:31:26.183 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:31:26.184 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:31:26.184 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:26.184 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 11461 2024-09-14T06:31:26.184 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:26.184 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:31:26.184 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:26.184 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:31:26.184 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655370+0000", 2024-09-14T06:31:26.184 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:26.184 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:26.184 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:26.185 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:31:26.185 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655370+0000", 2024-09-14T06:31:26.185 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:26.185 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:26.185 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:26.185 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:31:26.185 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655394+0000", 2024-09-14T06:31:26.185 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.4553999999999999e-05 2024-09-14T06:31:26.185 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:26.185 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:26.185 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:31:26.186 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655396+0000", 2024-09-14T06:31:26.186 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.8950000000000001e-06 2024-09-14T06:31:26.186 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:26.186 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:26.186 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:31:26.186 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655401+0000", 2024-09-14T06:31:26.186 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.0089999999999998e-06 2024-09-14T06:31:26.186 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:26.186 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:26.186 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:31:26.186 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655404+0000", 2024-09-14T06:31:26.187 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.3400000000000002e-06 2024-09-14T06:31:26.187 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:26.187 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:26.187 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:31:26.187 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.708186+0000", 2024-09-14T06:31:26.187 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052781726000000001 2024-09-14T06:31:26.187 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:26.187 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:26.187 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:31:26.187 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.708229+0000", 2024-09-14T06:31:26.188 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.2670999999999999e-05 2024-09-14T06:31:26.188 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:26.188 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:26.188 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:31:26.188 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.708285+0000", 2024-09-14T06:31:26.188 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.5946999999999998e-05 2024-09-14T06:31:26.188 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:26.188 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:31:26.188 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:26.188 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:26.189 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:26.189 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:11462 3.1d 3:bcad7303:::rbd_data.112d92027590.0000000000000001:f [read 186368~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:31:26.189 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:25.655410+0000", 2024-09-14T06:31:26.189 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 120.479724387, 2024-09-14T06:31:26.189 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052995258000000003, 2024-09-14T06:31:26.189 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:31:26.189 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:31:26.189 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:31:26.189 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:31:26.189 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:31:26.190 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:26.190 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 11462 2024-09-14T06:31:26.190 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:26.190 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:31:26.190 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:26.190 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:31:26.190 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655410+0000", 2024-09-14T06:31:26.190 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:26.190 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:26.190 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:26.191 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:31:26.191 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655410+0000", 2024-09-14T06:31:26.191 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:26.191 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:26.191 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:26.191 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:31:26.191 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655625+0000", 2024-09-14T06:31:26.191 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00021442600000000001 2024-09-14T06:31:26.191 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:26.191 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:26.191 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:31:26.191 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655626+0000", 2024-09-14T06:31:26.192 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.815e-06 2024-09-14T06:31:26.192 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:26.192 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:26.192 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:31:26.192 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655631+0000", 2024-09-14T06:31:26.192 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.651e-06 2024-09-14T06:31:26.192 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:26.192 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:26.192 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:31:26.192 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655634+0000", 2024-09-14T06:31:26.192 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.2449999999999998e-06 2024-09-14T06:31:26.193 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:26.193 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:26.193 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:31:26.193 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.708312+0000", 2024-09-14T06:31:26.193 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052677412 2024-09-14T06:31:26.193 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:26.193 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:26.193 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:31:26.193 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.708352+0000", 2024-09-14T06:31:26.193 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.0729999999999998e-05 2024-09-14T06:31:26.193 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:26.193 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:26.194 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:31:26.194 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.708405+0000", 2024-09-14T06:31:26.194 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.2979000000000002e-05 2024-09-14T06:31:26.194 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:26.194 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:31:26.194 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:26.194 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:26.194 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:26.194 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:11463 3.1d 3:bcad7303:::rbd_data.112d92027590.0000000000000001:f [read 196608~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:31:26.194 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:25.655639+0000", 2024-09-14T06:31:26.194 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 120.479495241, 2024-09-14T06:31:26.195 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052888859000000003, 2024-09-14T06:31:26.195 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:31:26.195 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:31:26.195 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:31:26.195 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:31:26.195 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:31:26.195 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:26.195 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 11463 2024-09-14T06:31:26.195 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:26.195 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:31:26.195 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:26.196 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:31:26.196 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655639+0000", 2024-09-14T06:31:26.196 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:26.196 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:26.196 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:26.196 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:31:26.196 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655639+0000", 2024-09-14T06:31:26.196 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:26.196 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:26.196 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:26.196 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:31:26.196 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655640+0000", 2024-09-14T06:31:26.197 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.08e-07 2024-09-14T06:31:26.197 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:26.197 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:26.197 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:31:26.197 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655641+0000", 2024-09-14T06:31:26.197 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.156e-06 2024-09-14T06:31:26.197 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:26.197 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:26.197 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:31:26.197 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655648+0000", 2024-09-14T06:31:26.198 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.8460000000000002e-06 2024-09-14T06:31:26.198 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:26.198 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:26.198 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:31:26.198 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655651+0000", 2024-09-14T06:31:26.198 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.0079999999999998e-06 2024-09-14T06:31:26.198 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:26.198 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:26.198 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:31:26.198 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.708429+0000", 2024-09-14T06:31:26.198 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052778459 2024-09-14T06:31:26.199 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:26.199 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:26.199 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:31:26.199 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.708473+0000", 2024-09-14T06:31:26.199 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.3782999999999997e-05 2024-09-14T06:31:26.199 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:26.199 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:26.199 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:31:26.199 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.708528+0000", 2024-09-14T06:31:26.199 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.5198999999999998e-05 2024-09-14T06:31:26.199 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:26.199 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:31:26.199 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:26.200 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:26.200 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:26.200 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:11464 3.1d 3:bcad7303:::rbd_data.112d92027590.0000000000000001:f [read 206848~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:31:26.200 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:25.655654+0000", 2024-09-14T06:31:26.200 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 120.479480372, 2024-09-14T06:31:26.200 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052981845999999999, 2024-09-14T06:31:26.200 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:31:26.201 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:31:26.201 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:31:26.201 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:31:26.201 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:31:26.201 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:26.201 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 11464 2024-09-14T06:31:26.201 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:26.201 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:31:26.201 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:26.201 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:31:26.201 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655654+0000", 2024-09-14T06:31:26.202 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:26.202 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:26.202 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:26.202 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:31:26.202 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655654+0000", 2024-09-14T06:31:26.202 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:26.202 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:26.202 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:26.202 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:31:26.202 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655946+0000", 2024-09-14T06:31:26.202 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00029148200000000003 2024-09-14T06:31:26.202 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:26.203 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:26.203 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:31:26.203 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655948+0000", 2024-09-14T06:31:26.203 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.973e-06 2024-09-14T06:31:26.203 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:26.203 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:26.203 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:31:26.203 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655953+0000", 2024-09-14T06:31:26.203 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.1240000000000004e-06 2024-09-14T06:31:26.203 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:26.204 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:26.204 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:31:26.204 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655956+0000", 2024-09-14T06:31:26.204 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.315e-06 2024-09-14T06:31:26.204 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:26.204 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:26.204 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:31:26.204 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.708543+0000", 2024-09-14T06:31:26.204 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052587416999999997 2024-09-14T06:31:26.204 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:26.205 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:26.205 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:31:26.205 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.708584+0000", 2024-09-14T06:31:26.205 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.1004000000000002e-05 2024-09-14T06:31:26.205 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:26.205 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:26.205 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:31:26.205 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.708636+0000", 2024-09-14T06:31:26.205 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.1530999999999998e-05 2024-09-14T06:31:26.205 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:26.206 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:31:26.206 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:26.206 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:26.206 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:26.206 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:11465 3.1d 3:bcad7303:::rbd_data.112d92027590.0000000000000001:f [read 217088~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:31:26.206 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:25.655961+0000", 2024-09-14T06:31:26.206 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 120.47917322000001, 2024-09-14T06:31:26.206 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052798682999999999, 2024-09-14T06:31:26.206 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:31:26.206 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:31:26.206 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:31:26.207 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:31:26.207 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:31:26.207 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:26.207 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 11465 2024-09-14T06:31:26.207 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:26.207 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:31:26.207 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:26.207 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:31:26.207 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655961+0000", 2024-09-14T06:31:26.207 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:26.207 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:26.208 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:26.208 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:31:26.208 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655961+0000", 2024-09-14T06:31:26.208 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:26.208 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:26.208 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:26.208 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:31:26.208 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.656114+0000", 2024-09-14T06:31:26.209 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.000152745 2024-09-14T06:31:26.209 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:26.209 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:26.209 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:31:26.209 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.656116+0000", 2024-09-14T06:31:26.209 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.804e-06 2024-09-14T06:31:26.209 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:26.209 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:26.209 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:31:26.210 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.656120+0000", 2024-09-14T06:31:26.210 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.4700000000000004e-06 2024-09-14T06:31:26.210 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:26.210 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:26.210 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:31:26.210 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.656123+0000", 2024-09-14T06:31:26.210 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.1269999999999999e-06 2024-09-14T06:31:26.210 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:26.210 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:26.210 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:31:26.211 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.708660+0000", 2024-09-14T06:31:26.211 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052536919000000001 2024-09-14T06:31:26.211 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:26.211 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:26.211 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:31:26.211 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.708703+0000", 2024-09-14T06:31:26.211 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.3077000000000001e-05 2024-09-14T06:31:26.211 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:26.212 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:26.212 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:31:26.212 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.708760+0000", 2024-09-14T06:31:26.212 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.6541e-05 2024-09-14T06:31:26.212 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:26.212 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:31:26.212 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:26.212 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:26.212 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:26.212 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:11466 3.1d 3:bcad7303:::rbd_data.112d92027590.0000000000000001:f [read 227328~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:31:26.213 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:25.656129+0000", 2024-09-14T06:31:26.213 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 120.479005832, 2024-09-14T06:31:26.213 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052750987999999999, 2024-09-14T06:31:26.213 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:31:26.213 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:31:26.213 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:31:26.214 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:31:26.214 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:31:26.214 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:26.214 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 11466 2024-09-14T06:31:26.214 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:26.214 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:31:26.214 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:26.214 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:31:26.214 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.656129+0000", 2024-09-14T06:31:26.214 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:26.214 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:26.214 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:26.215 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:31:26.215 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.656129+0000", 2024-09-14T06:31:26.215 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:26.215 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:26.215 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:26.215 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:31:26.215 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.656347+0000", 2024-09-14T06:31:26.215 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.000218054 2024-09-14T06:31:26.215 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:26.216 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:26.216 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:31:26.216 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.656349+0000", 2024-09-14T06:31:26.216 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.9199999999999998e-06 2024-09-14T06:31:26.216 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:26.216 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:26.216 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:31:26.216 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.656353+0000", 2024-09-14T06:31:26.216 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.9210000000000002e-06 2024-09-14T06:31:26.216 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:26.216 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:26.217 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:31:26.217 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.656356+0000", 2024-09-14T06:31:26.217 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.7670000000000001e-06 2024-09-14T06:31:26.217 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:26.217 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:26.217 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:31:26.217 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.708787+0000", 2024-09-14T06:31:26.217 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052430840999999999 2024-09-14T06:31:26.217 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:26.217 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:26.218 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:31:26.218 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.708827+0000", 2024-09-14T06:31:26.218 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.9538e-05 2024-09-14T06:31:26.218 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:26.218 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:26.218 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:31:26.218 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.708880+0000", 2024-09-14T06:31:26.218 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.2947e-05 2024-09-14T06:31:26.218 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:26.218 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:31:26.219 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:26.219 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:26.219 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:26.219 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:11467 3.1d 3:bcad7303:::rbd_data.112d92027590.0000000000000001:f [read 237568~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:31:26.219 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:25.656361+0000", 2024-09-14T06:31:26.219 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 120.478773184, 2024-09-14T06:31:26.219 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052643898000000001, 2024-09-14T06:31:26.219 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:31:26.219 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:31:26.219 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:31:26.220 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:31:26.220 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:31:26.220 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:26.220 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 11467 2024-09-14T06:31:26.220 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:26.220 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:31:26.220 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:26.220 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:31:26.221 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.656361+0000", 2024-09-14T06:31:26.221 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:26.221 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:26.221 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:26.221 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:31:26.221 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.656361+0000", 2024-09-14T06:31:26.221 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:26.221 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:26.221 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:26.221 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:31:26.222 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.656855+0000", 2024-09-14T06:31:26.222 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00049372000000000001 2024-09-14T06:31:26.222 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:26.222 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:26.222 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:31:26.222 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.656859+0000", 2024-09-14T06:31:26.222 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.1300000000000003e-06 2024-09-14T06:31:26.222 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:26.222 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:26.222 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:31:26.222 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.656863+0000", 2024-09-14T06:31:26.223 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.2629999999999997e-06 2024-09-14T06:31:26.223 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:26.223 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:26.223 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:31:26.223 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.656867+0000", 2024-09-14T06:31:26.223 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.3220000000000001e-06 2024-09-14T06:31:26.223 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:26.223 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:26.223 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:31:26.223 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.708903+0000", 2024-09-14T06:31:26.224 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052035855999999998 2024-09-14T06:31:26.224 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:26.224 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:26.224 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:31:26.224 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.708949+0000", 2024-09-14T06:31:26.224 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.6276999999999998e-05 2024-09-14T06:31:26.224 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:26.224 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:26.224 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:31:26.224 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.709005+0000", 2024-09-14T06:31:26.224 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.6329999999999998e-05 2024-09-14T06:31:26.225 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:26.225 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:31:26.225 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:26.225 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:26.225 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:26.225 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:11479 3.1d 3:bcad7303:::rbd_data.112d92027590.0000000000000001:f [read 360448~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:31:26.225 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:25.658388+0000", 2024-09-14T06:31:26.225 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 120.476746351, 2024-09-14T06:31:26.225 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052408920999999997, 2024-09-14T06:31:26.226 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:31:26.226 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:31:26.226 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:31:26.226 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:31:26.226 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:31:26.226 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:26.226 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 11479 2024-09-14T06:31:26.226 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:26.226 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:31:26.226 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:26.226 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:31:26.226 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.658388+0000", 2024-09-14T06:31:26.227 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:26.227 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:26.227 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:26.227 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:31:26.227 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.658388+0000", 2024-09-14T06:31:26.227 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:26.227 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:26.227 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:26.227 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:31:26.227 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.658534+0000", 2024-09-14T06:31:26.227 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.000145613 2024-09-14T06:31:26.228 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:26.228 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:26.228 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:31:26.228 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.658540+0000", 2024-09-14T06:31:26.228 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.6680000000000004e-06 2024-09-14T06:31:26.228 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:26.228 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:26.228 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:31:26.228 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.658548+0000", 2024-09-14T06:31:26.228 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.7980000000000006e-06 2024-09-14T06:31:26.229 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:26.229 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:26.229 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:31:26.229 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.658554+0000", 2024-09-14T06:31:26.229 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.6869999999999999e-06 2024-09-14T06:31:26.229 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:26.229 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:26.229 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:31:26.229 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.710624+0000", 2024-09-14T06:31:26.229 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052070567999999998 2024-09-14T06:31:26.230 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:26.230 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:26.230 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:31:26.230 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.710666+0000", 2024-09-14T06:31:26.230 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.1467000000000001e-05 2024-09-14T06:31:26.230 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:26.230 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:26.230 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:31:26.231 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.710797+0000", 2024-09-14T06:31:26.231 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00013112 2024-09-14T06:31:26.231 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:26.231 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:31:26.231 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:26.231 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:26.231 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:26.231 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:11480 3.1d 3:bcad7303:::rbd_data.112d92027590.0000000000000001:f [read 370688~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:31:26.231 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:25.658563+0000", 2024-09-14T06:31:26.231 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 120.47657142, 2024-09-14T06:31:26.232 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052429164, 2024-09-14T06:31:26.232 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:31:26.232 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:31:26.232 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:31:26.232 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:31:26.232 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:31:26.232 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:26.232 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 11480 2024-09-14T06:31:26.232 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:26.232 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:31:26.233 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:26.233 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:31:26.233 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.658563+0000", 2024-09-14T06:31:26.233 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:26.233 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:26.233 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:26.233 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:31:26.233 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.658563+0000", 2024-09-14T06:31:26.233 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:26.234 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:26.234 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:26.234 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:31:26.234 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.658658+0000", 2024-09-14T06:31:26.234 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.5346999999999994e-05 2024-09-14T06:31:26.234 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:26.234 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:26.234 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:31:26.234 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.658662+0000", 2024-09-14T06:31:26.235 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.14e-06 2024-09-14T06:31:26.235 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:26.235 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:26.235 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:31:26.235 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.658669+0000", 2024-09-14T06:31:26.235 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.7379999999999994e-06 2024-09-14T06:31:26.235 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:26.235 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:26.235 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:31:26.235 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.658675+0000", 2024-09-14T06:31:26.235 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.5430000000000001e-06 2024-09-14T06:31:26.236 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:26.236 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:26.236 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:31:26.236 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.710824+0000", 2024-09-14T06:31:26.236 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052148847999999998 2024-09-14T06:31:26.236 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:26.236 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:26.236 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:31:26.236 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.710864+0000", 2024-09-14T06:31:26.237 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.9925999999999997e-05 2024-09-14T06:31:26.237 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:26.237 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:26.237 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:31:26.237 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.710992+0000", 2024-09-14T06:31:26.237 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.000128622 2024-09-14T06:31:26.237 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:26.237 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:31:26.237 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:26.237 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:26.237 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:26.237 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:11481 3.1d 3:bcad7303:::rbd_data.112d92027590.0000000000000001:f [read 380928~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:31:26.237 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:25.658684+0000", 2024-09-14T06:31:26.238 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 120.47645034999999, 2024-09-14T06:31:26.238 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052508565, 2024-09-14T06:31:26.238 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:31:26.238 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:31:26.238 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:31:26.238 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:31:26.238 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:31:26.238 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:26.238 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 11481 2024-09-14T06:31:26.238 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:26.238 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:31:26.238 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:26.239 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:31:26.239 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.658684+0000", 2024-09-14T06:31:26.239 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:26.239 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:26.239 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:26.239 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:31:26.239 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.658684+0000", 2024-09-14T06:31:26.239 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:26.239 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:26.239 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:26.239 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:31:26.239 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.658715+0000", 2024-09-14T06:31:26.239 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.0703000000000001e-05 2024-09-14T06:31:26.239 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:26.240 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:26.240 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:31:26.240 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.658718+0000", 2024-09-14T06:31:26.240 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.0709999999999999e-06 2024-09-14T06:31:26.240 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:26.240 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:26.240 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:31:26.240 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.658725+0000", 2024-09-14T06:31:26.240 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.2339999999999997e-06 2024-09-14T06:31:26.240 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:26.240 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:26.240 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:31:26.240 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.658731+0000", 2024-09-14T06:31:26.240 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.6450000000000002e-06 2024-09-14T06:31:26.241 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:26.241 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:26.241 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:31:26.241 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.711020+0000", 2024-09-14T06:31:26.241 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052288886 2024-09-14T06:31:26.241 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:26.241 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:26.241 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:31:26.241 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.711061+0000", 2024-09-14T06:31:26.241 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.1084000000000002e-05 2024-09-14T06:31:26.241 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:26.241 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:26.241 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:31:26.241 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.711193+0000", 2024-09-14T06:31:26.242 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00013194199999999999 2024-09-14T06:31:26.242 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:26.242 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:31:26.242 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:26.242 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:26.242 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:26.242 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:11482 3.1d 3:bcad7303:::rbd_data.112d92027590.0000000000000001:f [read 391168~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:31:26.242 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:25.658740+0000", 2024-09-14T06:31:26.242 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 120.476394885, 2024-09-14T06:31:26.242 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052637015000000002, 2024-09-14T06:31:26.242 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:31:26.242 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:31:26.242 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:31:26.242 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:31:26.242 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:31:26.243 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:26.243 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 11482 2024-09-14T06:31:26.243 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:26.243 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:31:26.243 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:26.243 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:31:26.243 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.658740+0000", 2024-09-14T06:31:26.243 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:26.243 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:26.243 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:26.243 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:31:26.243 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.658740+0000", 2024-09-14T06:31:26.243 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:26.243 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:26.244 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:26.244 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:31:26.244 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.658880+0000", 2024-09-14T06:31:26.244 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00014036100000000001 2024-09-14T06:31:26.244 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:26.244 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:26.244 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:31:26.244 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.658883+0000", 2024-09-14T06:31:26.244 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.3969999999999998e-06 2024-09-14T06:31:26.244 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:26.244 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:26.244 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:31:26.244 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.658890+0000", 2024-09-14T06:31:26.244 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.1500000000000002e-06 2024-09-14T06:31:26.245 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:26.245 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:26.245 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:31:26.245 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.658897+0000", 2024-09-14T06:31:26.245 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.1260000000000003e-06 2024-09-14T06:31:26.245 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:26.245 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:26.245 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:31:26.245 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.711207+0000", 2024-09-14T06:31:26.245 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052310777000000003 2024-09-14T06:31:26.245 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:26.245 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:26.245 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:31:26.245 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.711249+0000", 2024-09-14T06:31:26.246 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.1786999999999998e-05 2024-09-14T06:31:26.246 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:26.246 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:26.246 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:31:26.246 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.711377+0000", 2024-09-14T06:31:26.246 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.000127417 2024-09-14T06:31:26.246 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:26.246 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:31:26.246 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:26.246 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:26.246 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:26.246 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:11483 3.1d 3:bcad7303:::rbd_data.112d92027590.0000000000000001:f [read 401408~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:31:26.246 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:25.658906+0000", 2024-09-14T06:31:26.246 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 120.476228101, 2024-09-14T06:31:26.246 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052595109000000001, 2024-09-14T06:31:26.247 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:31:26.247 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:31:26.247 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:31:26.247 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:31:26.247 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:31:26.247 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:26.247 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 11483 2024-09-14T06:31:26.247 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:26.247 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:31:26.247 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:26.247 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:31:26.247 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.658906+0000", 2024-09-14T06:31:26.247 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:26.247 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:26.248 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:26.248 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:31:26.248 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.658906+0000", 2024-09-14T06:31:26.248 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:26.248 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:26.248 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:26.248 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:31:26.248 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.664985+0000", 2024-09-14T06:31:26.248 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.0060787250000000001 2024-09-14T06:31:26.248 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:26.248 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:26.248 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:31:26.248 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.664991+0000", 2024-09-14T06:31:26.248 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.7030000000000003e-06 2024-09-14T06:31:26.249 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:26.249 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:26.249 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:31:26.249 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.664999+0000", 2024-09-14T06:31:26.249 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.4430000000000003e-06 2024-09-14T06:31:26.249 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:26.249 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:26.249 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:31:26.249 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.665006+0000", 2024-09-14T06:31:26.249 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.0840000000000003e-06 2024-09-14T06:31:26.249 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:26.249 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:26.249 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:31:26.250 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.711402+0000", 2024-09-14T06:31:26.250 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046395714999999997 2024-09-14T06:31:26.250 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:26.250 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:26.250 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:31:26.250 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.711443+0000", 2024-09-14T06:31:26.250 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.0911999999999997e-05 2024-09-14T06:31:26.250 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:26.250 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:26.250 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:31:26.250 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.711501+0000", 2024-09-14T06:31:26.250 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.8526999999999997e-05 2024-09-14T06:31:26.250 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:26.250 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:31:26.250 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:26.251 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:26.251 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:31:26.251 INFO:teuthology.orchestra.run.smithi138.stdout:} 2024-09-14T06:31:26.252 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-14T06:31:26.253 INFO:tasks.ceph.osd.4.smithi138.stderr:2024-09-14T06:31:26.250+0000 7f6c653c8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33615) UID: 0 2024-09-14T06:31:26.353 INFO:tasks.ceph.osd.1.smithi094.stderr:2024-09-14T06:31:26.352+0000 7f328e47a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33641) UID: 0 2024-09-14T06:31:26.394 INFO:teuthology.orchestra.run.smithi138.stdout:{ 2024-09-14T06:31:26.395 INFO:teuthology.orchestra.run.smithi138.stdout: "ops": [], 2024-09-14T06:31:26.395 INFO:teuthology.orchestra.run.smithi138.stdout: "num_ops": 0 2024-09-14T06:31:26.395 INFO:teuthology.orchestra.run.smithi138.stdout:} 2024-09-14T06:31:26.403 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-14T06:31:26.453 INFO:tasks.ceph.osd.5.smithi138.stderr:2024-09-14T06:31:26.451+0000 7fe13746f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33618) UID: 0 2024-09-14T06:31:26.505 INFO:teuthology.orchestra.run.smithi138.stdout:{ 2024-09-14T06:31:26.506 INFO:teuthology.orchestra.run.smithi138.stdout: "ops": [], 2024-09-14T06:31:26.506 INFO:teuthology.orchestra.run.smithi138.stdout: "complaint_time": 30, 2024-09-14T06:31:26.506 INFO:teuthology.orchestra.run.smithi138.stdout: "num_blocked_ops": 0 2024-09-14T06:31:26.506 INFO:teuthology.orchestra.run.smithi138.stdout:} 2024-09-14T06:31:26.514 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-14T06:31:26.555 INFO:tasks.ceph.osd.7.smithi138.stderr:2024-09-14T06:31:26.553+0000 7f3c6a21d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33623) UID: 0 2024-09-14T06:31:26.627 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-14T06:31:26.659 INFO:teuthology.orchestra.run.smithi138.stdout:{ 2024-09-14T06:31:26.659 INFO:teuthology.orchestra.run.smithi138.stdout: "size": 20, 2024-09-14T06:31:26.660 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 600, 2024-09-14T06:31:26.660 INFO:teuthology.orchestra.run.smithi138.stdout: "ops": [ 2024-09-14T06:31:26.660 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:26.660 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:98505 3.10 3.1bf56f90 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T06:31:26.660 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:30:07.045453+0000", 2024-09-14T06:31:26.660 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 79.612297651000006, 2024-09-14T06:31:26.660 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032144440000000003, 2024-09-14T06:31:26.660 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:31:26.660 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:31:26.660 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "reached pg", 2024-09-14T06:31:26.660 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:31:26.660 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:31:26.660 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:26.660 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 98505 2024-09-14T06:31:26.661 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:26.661 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:31:26.661 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:26.661 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:31:26.661 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045453+0000", 2024-09-14T06:31:26.662 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:26.662 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:26.662 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:26.662 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:31:26.662 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045453+0000", 2024-09-14T06:31:26.662 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:26.663 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:26.663 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:26.663 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:31:26.663 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045453+0000", 2024-09-14T06:31:26.663 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.36e-07 2024-09-14T06:31:26.664 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:26.664 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:26.664 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:31:26.664 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045454+0000", 2024-09-14T06:31:26.664 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.1299999999999998e-07 2024-09-14T06:31:26.665 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:26.665 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:26.665 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:31:26.665 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045457+0000", 2024-09-14T06:31:26.666 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.8559999999999998e-06 2024-09-14T06:31:26.666 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:26.666 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:26.666 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:31:26.666 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045460+0000", 2024-09-14T06:31:26.666 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.8750000000000002e-06 2024-09-14T06:31:26.666 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:26.666 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:26.666 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:31:26.666 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.077592+0000", 2024-09-14T06:31:26.666 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032131727999999998 2024-09-14T06:31:26.666 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:26.667 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:26.667 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:31:26.667 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.077597+0000", 2024-09-14T06:31:26.667 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.8319999999999996e-06 2024-09-14T06:31:26.667 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:26.667 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:31:26.667 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:26.667 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:26.667 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:26.667 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:98506 3.10 3.1bf56f90 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T06:31:26.667 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:30:07.045481+0000", 2024-09-14T06:31:26.667 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 79.612269396000002, 2024-09-14T06:31:26.667 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032142759999999999, 2024-09-14T06:31:26.667 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:31:26.668 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:31:26.668 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "reached pg", 2024-09-14T06:31:26.668 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:31:26.668 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:31:26.668 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:26.668 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 98506 2024-09-14T06:31:26.668 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:26.668 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:31:26.668 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:26.668 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:31:26.668 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045481+0000", 2024-09-14T06:31:26.668 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:26.668 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:26.668 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:26.669 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:31:26.669 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045481+0000", 2024-09-14T06:31:26.669 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:26.669 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:26.669 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:26.669 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:31:26.669 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045481+0000", 2024-09-14T06:31:26.669 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.8500000000000001e-07 2024-09-14T06:31:26.669 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:26.669 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:26.669 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:31:26.669 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045483+0000", 2024-09-14T06:31:26.669 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.8250000000000001e-06 2024-09-14T06:31:26.669 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:26.670 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:26.670 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:31:26.670 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045489+0000", 2024-09-14T06:31:26.670 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.5740000000000003e-06 2024-09-14T06:31:26.670 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:26.670 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:26.670 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:31:26.670 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045495+0000", 2024-09-14T06:31:26.670 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.7690000000000002e-06 2024-09-14T06:31:26.670 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:26.670 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:26.670 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:31:26.670 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.077619+0000", 2024-09-14T06:31:26.670 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032124621999999999 2024-09-14T06:31:26.670 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:26.671 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:26.671 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:31:26.671 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.077624+0000", 2024-09-14T06:31:26.671 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.3850000000000004e-06 2024-09-14T06:31:26.671 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:26.671 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:31:26.671 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:26.671 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:26.671 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:26.671 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:98507 3.10 3.1bf56f90 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T06:31:26.671 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:30:07.045502+0000", 2024-09-14T06:31:26.671 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 79.612247975000002, 2024-09-14T06:31:26.671 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032147219999999997, 2024-09-14T06:31:26.671 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:31:26.671 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:31:26.672 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "reached pg", 2024-09-14T06:31:26.672 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:31:26.672 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:31:26.672 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:26.672 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 98507 2024-09-14T06:31:26.672 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:26.672 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:31:26.672 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:26.672 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:31:26.672 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045502+0000", 2024-09-14T06:31:26.672 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:26.673 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:26.673 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:26.673 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:31:26.673 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045502+0000", 2024-09-14T06:31:26.673 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:26.673 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:26.673 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:26.673 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:31:26.673 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045503+0000", 2024-09-14T06:31:26.673 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.4700000000000002e-07 2024-09-14T06:31:26.673 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:26.673 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:26.673 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:31:26.673 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045509+0000", 2024-09-14T06:31:26.673 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.7329999999999997e-06 2024-09-14T06:31:26.674 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:26.674 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:26.674 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:31:26.674 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045515+0000", 2024-09-14T06:31:26.674 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.8030000000000002e-06 2024-09-14T06:31:26.674 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:26.674 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:26.674 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:31:26.674 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045522+0000", 2024-09-14T06:31:26.674 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.0369999999999997e-06 2024-09-14T06:31:26.674 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:26.674 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:26.674 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:31:26.674 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.077641+0000", 2024-09-14T06:31:26.675 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032119091000000002 2024-09-14T06:31:26.675 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:26.675 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:26.675 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:31:26.675 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.077650+0000", 2024-09-14T06:31:26.675 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.2090000000000005e-06 2024-09-14T06:31:26.675 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:26.675 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:31:26.675 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:26.675 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:26.676 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:26.676 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:98508 3.10 3.1bf56f90 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T06:31:26.676 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:30:07.045530+0000", 2024-09-14T06:31:26.676 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 79.612220358000002, 2024-09-14T06:31:26.676 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032198773999999999, 2024-09-14T06:31:26.676 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:31:26.676 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:31:26.676 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "reached pg", 2024-09-14T06:31:26.676 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:31:26.676 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:31:26.676 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:26.676 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 98508 2024-09-14T06:31:26.676 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:26.676 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:31:26.676 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:26.677 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:31:26.677 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045530+0000", 2024-09-14T06:31:26.677 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:26.677 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:26.677 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:26.677 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:31:26.677 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045530+0000", 2024-09-14T06:31:26.677 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:26.677 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:26.677 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:26.677 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:31:26.677 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045530+0000", 2024-09-14T06:31:26.677 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.3099999999999998e-07 2024-09-14T06:31:26.677 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:26.678 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:26.678 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:31:26.678 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045539+0000", 2024-09-14T06:31:26.678 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.4780000000000008e-06 2024-09-14T06:31:26.678 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:26.678 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:26.678 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:31:26.678 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045546+0000", 2024-09-14T06:31:26.678 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.3259999999999998e-06 2024-09-14T06:31:26.678 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:26.678 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:26.678 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:31:26.678 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045551+0000", 2024-09-14T06:31:26.678 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.9960000000000001e-06 2024-09-14T06:31:26.679 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:26.679 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:26.679 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:31:26.679 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.077716+0000", 2024-09-14T06:31:26.679 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032165081999999998 2024-09-14T06:31:26.679 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:26.679 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:26.679 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:31:26.679 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.077729+0000", 2024-09-14T06:31:26.679 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.2461e-05 2024-09-14T06:31:26.679 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:26.679 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:31:26.679 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:26.679 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:26.679 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:26.679 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:98509 3.10 3.1bf56f90 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T06:31:26.680 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:30:07.045558+0000", 2024-09-14T06:31:26.680 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 79.612192156999996, 2024-09-14T06:31:26.680 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032242198, 2024-09-14T06:31:26.680 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:31:26.680 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:31:26.680 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "reached pg", 2024-09-14T06:31:26.680 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:31:26.680 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:31:26.680 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:26.680 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 98509 2024-09-14T06:31:26.680 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:26.680 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:31:26.680 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:26.680 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:31:26.681 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045558+0000", 2024-09-14T06:31:26.681 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:26.681 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:26.681 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:26.681 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:31:26.681 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045558+0000", 2024-09-14T06:31:26.681 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:26.681 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:26.681 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:26.681 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:31:26.681 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045559+0000", 2024-09-14T06:31:26.681 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.27e-07 2024-09-14T06:31:26.681 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:26.682 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:26.682 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:31:26.682 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045560+0000", 2024-09-14T06:31:26.682 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.6959999999999999e-06 2024-09-14T06:31:26.682 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:26.682 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:26.682 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:31:26.682 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045567+0000", 2024-09-14T06:31:26.682 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.6900000000000003e-06 2024-09-14T06:31:26.682 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:26.682 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:26.682 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:31:26.683 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045571+0000", 2024-09-14T06:31:26.683 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.5750000000000002e-06 2024-09-14T06:31:26.683 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:26.683 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:26.683 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:31:26.683 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.077789+0000", 2024-09-14T06:31:26.683 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032217186000000002 2024-09-14T06:31:26.683 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:26.683 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:26.683 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:31:26.683 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.077800+0000", 2024-09-14T06:31:26.683 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.1623999999999999e-05 2024-09-14T06:31:26.683 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:26.683 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:31:26.683 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:26.684 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:26.684 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:26.684 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:98510 3.10 3.1bf56f90 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T06:31:26.684 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:30:07.045580+0000", 2024-09-14T06:31:26.684 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 79.612170343000003, 2024-09-14T06:31:26.684 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032299845000000001, 2024-09-14T06:31:26.684 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:31:26.684 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:31:26.684 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "reached pg", 2024-09-14T06:31:26.684 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:31:26.684 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:31:26.684 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:26.684 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 98510 2024-09-14T06:31:26.684 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:26.685 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:31:26.685 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:26.685 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:31:26.685 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045580+0000", 2024-09-14T06:31:26.685 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:26.685 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:26.685 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:26.685 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:31:26.685 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045580+0000", 2024-09-14T06:31:26.685 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:26.685 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:26.686 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:26.686 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:31:26.686 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045580+0000", 2024-09-14T06:31:26.686 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.0400000000000002e-07 2024-09-14T06:31:26.686 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:26.686 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:26.686 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:31:26.686 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045582+0000", 2024-09-14T06:31:26.686 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.7600000000000001e-06 2024-09-14T06:31:26.686 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:26.686 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:26.686 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:31:26.687 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045588+0000", 2024-09-14T06:31:26.687 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.5650000000000001e-06 2024-09-14T06:31:26.687 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:26.687 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:26.687 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:31:26.687 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045597+0000", 2024-09-14T06:31:26.687 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.465e-06 2024-09-14T06:31:26.687 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:26.687 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:26.687 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:31:26.687 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.077871+0000", 2024-09-14T06:31:26.687 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032273562999999998 2024-09-14T06:31:26.688 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:26.688 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:26.688 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:31:26.688 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.077880+0000", 2024-09-14T06:31:26.688 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.0880000000000001e-06 2024-09-14T06:31:26.688 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:26.688 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:31:26.688 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:26.688 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:26.688 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:26.688 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:98511 3.10 3.1bf56f90 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T06:31:26.688 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:30:07.045610+0000", 2024-09-14T06:31:26.689 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 79.612140495999995, 2024-09-14T06:31:26.689 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032338894, 2024-09-14T06:31:26.689 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:31:26.689 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:31:26.689 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "reached pg", 2024-09-14T06:31:26.689 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:31:26.689 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:31:26.689 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:26.689 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 98511 2024-09-14T06:31:26.689 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:26.689 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:31:26.690 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:26.690 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:31:26.690 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045610+0000", 2024-09-14T06:31:26.690 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:26.690 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:26.690 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:26.690 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:31:26.690 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045610+0000", 2024-09-14T06:31:26.690 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:26.690 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:26.690 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:26.690 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:31:26.690 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045610+0000", 2024-09-14T06:31:26.690 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.1600000000000002e-07 2024-09-14T06:31:26.691 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:26.691 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:26.691 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:31:26.691 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045612+0000", 2024-09-14T06:31:26.691 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.1150000000000001e-06 2024-09-14T06:31:26.691 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:26.691 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:26.691 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:31:26.691 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045618+0000", 2024-09-14T06:31:26.691 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.0499999999999997e-06 2024-09-14T06:31:26.691 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:26.691 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:26.691 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:31:26.692 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045623+0000", 2024-09-14T06:31:26.692 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.6770000000000003e-06 2024-09-14T06:31:26.692 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:26.692 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:26.692 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:31:26.692 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.077928+0000", 2024-09-14T06:31:26.692 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032305111999999997 2024-09-14T06:31:26.692 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:26.692 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:26.692 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:31:26.692 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.077949+0000", 2024-09-14T06:31:26.692 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.0523999999999999e-05 2024-09-14T06:31:26.692 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:26.692 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:31:26.693 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:26.693 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:26.693 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:26.693 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:98512 3.10 3.1bf56f90 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T06:31:26.693 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:30:07.045630+0000", 2024-09-14T06:31:26.693 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 79.612120490999999, 2024-09-14T06:31:26.693 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032346632, 2024-09-14T06:31:26.693 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:31:26.693 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:31:26.693 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "reached pg", 2024-09-14T06:31:26.693 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:31:26.693 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:31:26.693 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:26.693 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 98512 2024-09-14T06:31:26.694 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:26.694 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:31:26.694 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:26.694 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:31:26.694 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045630+0000", 2024-09-14T06:31:26.694 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:26.694 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:26.694 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:26.694 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:31:26.694 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045630+0000", 2024-09-14T06:31:26.694 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:26.694 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:26.694 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:26.694 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:31:26.694 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045630+0000", 2024-09-14T06:31:26.695 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.8200000000000001e-07 2024-09-14T06:31:26.695 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:26.695 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:26.695 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:31:26.695 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045632+0000", 2024-09-14T06:31:26.695 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.7430000000000001e-06 2024-09-14T06:31:26.695 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:26.695 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:26.695 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:31:26.695 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045637+0000", 2024-09-14T06:31:26.695 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.4550000000000003e-06 2024-09-14T06:31:26.695 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:26.695 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:26.695 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:31:26.696 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045644+0000", 2024-09-14T06:31:26.696 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.1330000000000001e-06 2024-09-14T06:31:26.696 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:26.696 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:26.696 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:31:26.696 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.077968+0000", 2024-09-14T06:31:26.696 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032323401000000002 2024-09-14T06:31:26.696 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:26.696 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:26.696 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:31:26.696 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.077976+0000", 2024-09-14T06:31:26.696 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.5180000000000004e-06 2024-09-14T06:31:26.697 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:26.697 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:31:26.697 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:26.697 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:26.697 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:26.697 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:98513 3.10 3.1bf56f90 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T06:31:26.697 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:30:07.045652+0000", 2024-09-14T06:31:26.697 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 79.612097926999994, 2024-09-14T06:31:26.697 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032353436999999999, 2024-09-14T06:31:26.697 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:31:26.697 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:31:26.697 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "reached pg", 2024-09-14T06:31:26.698 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:31:26.698 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:31:26.698 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:26.698 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 98513 2024-09-14T06:31:26.698 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:26.698 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:31:26.698 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:26.698 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:31:26.698 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045652+0000", 2024-09-14T06:31:26.698 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:26.698 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:26.698 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:26.698 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:31:26.698 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045652+0000", 2024-09-14T06:31:26.698 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:26.699 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:26.699 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:26.699 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:31:26.699 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045653+0000", 2024-09-14T06:31:26.699 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.3700000000000001e-07 2024-09-14T06:31:26.699 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:26.699 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:26.699 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:31:26.699 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045655+0000", 2024-09-14T06:31:26.699 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.854e-06 2024-09-14T06:31:26.700 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:26.700 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:26.700 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:31:26.700 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045661+0000", 2024-09-14T06:31:26.700 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.5509999999999996e-06 2024-09-14T06:31:26.700 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:26.700 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:26.700 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:31:26.700 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045668+0000", 2024-09-14T06:31:26.700 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.1980000000000004e-06 2024-09-14T06:31:26.701 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:26.701 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:26.701 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:31:26.701 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.077997+0000", 2024-09-14T06:31:26.701 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032328575999999998 2024-09-14T06:31:26.701 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:26.701 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:26.701 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:31:26.701 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.078006+0000", 2024-09-14T06:31:26.701 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.8210000000000008e-06 2024-09-14T06:31:26.701 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:26.701 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:31:26.701 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:26.701 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:26.702 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:26.702 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:98514 3.10 3.1bf56f90 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T06:31:26.702 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:30:07.045699+0000", 2024-09-14T06:31:26.702 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 79.612051592, 2024-09-14T06:31:26.702 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032330223999999998, 2024-09-14T06:31:26.702 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:31:26.702 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:31:26.702 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "reached pg", 2024-09-14T06:31:26.702 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:31:26.702 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:31:26.702 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:26.702 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 98514 2024-09-14T06:31:26.702 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:26.702 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:31:26.702 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:26.703 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:31:26.703 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045699+0000", 2024-09-14T06:31:26.703 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:26.703 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:26.703 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:26.703 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:31:26.703 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045699+0000", 2024-09-14T06:31:26.703 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:26.703 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:26.703 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:26.703 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:31:26.703 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045699+0000", 2024-09-14T06:31:26.703 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.3200000000000001e-07 2024-09-14T06:31:26.704 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:26.704 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:26.704 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:31:26.704 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045701+0000", 2024-09-14T06:31:26.704 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.762e-06 2024-09-14T06:31:26.704 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:26.704 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:26.704 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:31:26.704 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045705+0000", 2024-09-14T06:31:26.704 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.2359999999999998e-06 2024-09-14T06:31:26.704 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:26.704 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:26.704 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:31:26.704 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045708+0000", 2024-09-14T06:31:26.705 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.796e-06 2024-09-14T06:31:26.705 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:26.705 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:26.705 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:31:26.705 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.078023+0000", 2024-09-14T06:31:26.705 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032315213000000002 2024-09-14T06:31:26.705 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:26.705 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:26.705 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:31:26.705 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.078029+0000", 2024-09-14T06:31:26.705 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.8850000000000004e-06 2024-09-14T06:31:26.705 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:26.705 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:31:26.705 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:26.705 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:26.706 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:26.706 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:98515 3.10 3.1bf56f90 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T06:31:26.706 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:30:07.045712+0000", 2024-09-14T06:31:26.706 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 79.612038280999997, 2024-09-14T06:31:26.706 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032346871999999999, 2024-09-14T06:31:26.706 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:31:26.706 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:31:26.706 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "reached pg", 2024-09-14T06:31:26.706 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:31:26.706 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:31:26.706 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:26.706 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 98515 2024-09-14T06:31:26.706 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:26.706 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:31:26.707 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:26.707 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:31:26.707 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045712+0000", 2024-09-14T06:31:26.707 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:26.707 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:26.707 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:26.707 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:31:26.707 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045712+0000", 2024-09-14T06:31:26.707 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:26.707 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:26.707 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:26.708 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:31:26.708 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045712+0000", 2024-09-14T06:31:26.708 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.2100000000000001e-07 2024-09-14T06:31:26.708 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:26.708 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:26.708 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:31:26.708 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045713+0000", 2024-09-14T06:31:26.708 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.068e-06 2024-09-14T06:31:26.708 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:26.708 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:26.708 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:31:26.708 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045716+0000", 2024-09-14T06:31:26.709 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.971e-06 2024-09-14T06:31:26.709 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:26.709 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:26.709 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:31:26.709 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045719+0000", 2024-09-14T06:31:26.709 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.3879999999999998e-06 2024-09-14T06:31:26.709 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:26.709 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:26.709 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:31:26.709 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.078049+0000", 2024-09-14T06:31:26.709 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032330553999999997 2024-09-14T06:31:26.709 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:26.710 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:26.710 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:31:26.710 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.078059+0000", 2024-09-14T06:31:26.710 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.6700000000000006e-06 2024-09-14T06:31:26.710 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:26.710 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:31:26.710 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:26.710 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:26.710 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:26.710 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:98516 3.10 3.1bf56f90 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T06:31:26.710 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:30:07.045722+0000", 2024-09-14T06:31:26.710 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 79.612028527999996, 2024-09-14T06:31:26.711 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032374334999999997, 2024-09-14T06:31:26.711 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:31:26.711 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:31:26.711 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "reached pg", 2024-09-14T06:31:26.711 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:31:26.711 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:31:26.711 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:26.711 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 98516 2024-09-14T06:31:26.711 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:26.711 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:31:26.711 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:26.711 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:31:26.711 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045722+0000", 2024-09-14T06:31:26.711 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:26.711 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:26.712 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:26.712 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:31:26.712 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045722+0000", 2024-09-14T06:31:26.712 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:26.712 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:26.712 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:26.712 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:31:26.712 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045722+0000", 2024-09-14T06:31:26.712 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.8999999999999998e-07 2024-09-14T06:31:26.712 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:26.712 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:26.712 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:31:26.712 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045723+0000", 2024-09-14T06:31:26.712 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.0049999999999999e-06 2024-09-14T06:31:26.713 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:26.713 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:26.713 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:31:26.713 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045726+0000", 2024-09-14T06:31:26.713 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.9019999999999999e-06 2024-09-14T06:31:26.713 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:26.713 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:26.713 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:31:26.713 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045728+0000", 2024-09-14T06:31:26.713 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.3290000000000001e-06 2024-09-14T06:31:26.713 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:26.713 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:26.713 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:31:26.713 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.078090+0000", 2024-09-14T06:31:26.714 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032362123 2024-09-14T06:31:26.714 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:26.714 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:26.714 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:31:26.714 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.078096+0000", 2024-09-14T06:31:26.714 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.6860000000000003e-06 2024-09-14T06:31:26.714 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:26.714 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:31:26.714 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:26.714 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:26.714 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:26.714 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:98517 3.10 3.1bf56f90 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T06:31:26.714 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:30:07.045731+0000", 2024-09-14T06:31:26.715 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 79.612018797000005, 2024-09-14T06:31:26.715 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.03241227, 2024-09-14T06:31:26.715 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:31:26.715 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:31:26.715 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "reached pg", 2024-09-14T06:31:26.715 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:31:26.715 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:31:26.715 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:26.715 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 98517 2024-09-14T06:31:26.715 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:26.715 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:31:26.715 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:26.715 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:31:26.715 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045731+0000", 2024-09-14T06:31:26.716 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:26.716 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:26.716 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:26.716 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:31:26.716 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045731+0000", 2024-09-14T06:31:26.716 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:26.716 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:26.717 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:26.717 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:31:26.717 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045732+0000", 2024-09-14T06:31:26.717 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.0699999999999999e-07 2024-09-14T06:31:26.717 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:26.717 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:26.717 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:31:26.717 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045733+0000", 2024-09-14T06:31:26.717 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.9199999999999999e-07 2024-09-14T06:31:26.717 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:26.717 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:26.717 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:31:26.717 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045735+0000", 2024-09-14T06:31:26.717 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.8150000000000002e-06 2024-09-14T06:31:26.718 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:26.718 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:26.718 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:31:26.718 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045738+0000", 2024-09-14T06:31:26.718 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.3640000000000002e-06 2024-09-14T06:31:26.718 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:26.718 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:26.718 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:31:26.718 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.078126+0000", 2024-09-14T06:31:26.718 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032388206000000003 2024-09-14T06:31:26.718 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:26.718 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:26.718 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:31:26.718 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.078144+0000", 2024-09-14T06:31:26.718 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.7686e-05 2024-09-14T06:31:26.719 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:26.719 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:31:26.719 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:26.719 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:26.719 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:26.719 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:98518 3.10 3.1bf56f90 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T06:31:26.719 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:30:07.045741+0000", 2024-09-14T06:31:26.719 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 79.612009401999998, 2024-09-14T06:31:26.719 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032451793999999999, 2024-09-14T06:31:26.719 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:31:26.720 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:31:26.720 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "reached pg", 2024-09-14T06:31:26.720 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:31:26.720 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:31:26.720 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:26.720 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 98518 2024-09-14T06:31:26.720 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:26.720 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:31:26.720 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:26.720 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:31:26.720 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045741+0000", 2024-09-14T06:31:26.720 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:26.720 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:26.721 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:26.721 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:31:26.721 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045741+0000", 2024-09-14T06:31:26.721 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:26.721 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:26.721 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:26.721 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:31:26.721 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045741+0000", 2024-09-14T06:31:26.721 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.05e-07 2024-09-14T06:31:26.721 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:26.721 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:26.721 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:31:26.721 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045746+0000", 2024-09-14T06:31:26.722 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.2259999999999996e-06 2024-09-14T06:31:26.722 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:26.722 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:26.722 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:31:26.722 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045750+0000", 2024-09-14T06:31:26.722 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.2490000000000001e-06 2024-09-14T06:31:26.722 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:26.722 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:26.722 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:31:26.722 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045752+0000", 2024-09-14T06:31:26.722 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.3980000000000002e-06 2024-09-14T06:31:26.722 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:26.722 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:26.723 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:31:26.723 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.078184+0000", 2024-09-14T06:31:26.723 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032431593000000002 2024-09-14T06:31:26.723 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:26.723 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:26.723 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:31:26.723 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.078193+0000", 2024-09-14T06:31:26.723 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.1230000000000006e-06 2024-09-14T06:31:26.723 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:26.723 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:31:26.723 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:26.723 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:26.723 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:26.724 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:98519 3.10 3.1bf56f90 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T06:31:26.724 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:30:07.045755+0000", 2024-09-14T06:31:26.724 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 79.611994976999995, 2024-09-14T06:31:26.724 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032489431999999999, 2024-09-14T06:31:26.724 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:31:26.724 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:31:26.724 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "reached pg", 2024-09-14T06:31:26.724 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:31:26.724 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:31:26.724 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:26.724 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 98519 2024-09-14T06:31:26.724 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:26.724 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:31:26.724 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:26.725 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:31:26.725 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045755+0000", 2024-09-14T06:31:26.725 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:26.725 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:26.725 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:26.725 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:31:26.725 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045755+0000", 2024-09-14T06:31:26.725 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:26.725 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:26.725 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:26.725 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:31:26.725 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045756+0000", 2024-09-14T06:31:26.725 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.23e-07 2024-09-14T06:31:26.726 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:26.726 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:26.726 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:31:26.726 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045757+0000", 2024-09-14T06:31:26.726 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.1060000000000001e-06 2024-09-14T06:31:26.726 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:26.726 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:26.726 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:31:26.726 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045760+0000", 2024-09-14T06:31:26.726 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.988e-06 2024-09-14T06:31:26.726 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:26.726 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:26.726 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:31:26.726 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045765+0000", 2024-09-14T06:31:26.726 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.2689999999999999e-06 2024-09-14T06:31:26.727 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:26.727 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:26.727 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:31:26.727 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.078234+0000", 2024-09-14T06:31:26.727 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032468915000000001 2024-09-14T06:31:26.727 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:26.727 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:26.727 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:31:26.727 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.078245+0000", 2024-09-14T06:31:26.727 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.0930999999999999e-05 2024-09-14T06:31:26.727 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:26.727 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:31:26.727 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:26.727 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:26.728 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:26.728 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:98520 3.10 3.1bf56f90 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T06:31:26.728 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:30:07.045768+0000", 2024-09-14T06:31:26.728 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 79.611982099000002, 2024-09-14T06:31:26.728 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032531379999999999, 2024-09-14T06:31:26.728 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:31:26.728 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:31:26.728 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "reached pg", 2024-09-14T06:31:26.728 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:31:26.728 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:31:26.728 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:26.728 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 98520 2024-09-14T06:31:26.728 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:26.728 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:31:26.728 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:26.729 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:31:26.729 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045768+0000", 2024-09-14T06:31:26.729 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:26.729 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:26.729 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:26.729 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:31:26.729 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045768+0000", 2024-09-14T06:31:26.729 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:26.729 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:26.729 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:26.729 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:31:26.729 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045768+0000", 2024-09-14T06:31:26.729 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.22e-07 2024-09-14T06:31:26.730 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:26.730 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:26.730 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:31:26.730 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045773+0000", 2024-09-14T06:31:26.730 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.3470000000000001e-06 2024-09-14T06:31:26.730 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:26.730 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:26.730 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:31:26.730 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045776+0000", 2024-09-14T06:31:26.730 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.1319999999999998e-06 2024-09-14T06:31:26.730 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:26.730 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:26.730 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:31:26.730 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045779+0000", 2024-09-14T06:31:26.730 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.9749999999999999e-06 2024-09-14T06:31:26.731 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:26.731 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:26.731 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:31:26.731 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.078289+0000", 2024-09-14T06:31:26.731 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032509693999999999 2024-09-14T06:31:26.731 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:26.731 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:26.731 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:31:26.731 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.078300+0000", 2024-09-14T06:31:26.731 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.101e-05 2024-09-14T06:31:26.731 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:26.731 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:31:26.731 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:26.731 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:26.731 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:26.732 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:98521 3.10 3.1bf56f90 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T06:31:26.732 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:30:07.045782+0000", 2024-09-14T06:31:26.732 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 79.611967918999994, 2024-09-14T06:31:26.732 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032566109000000003, 2024-09-14T06:31:26.732 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:31:26.732 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:31:26.732 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "reached pg", 2024-09-14T06:31:26.732 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:31:26.732 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:31:26.732 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:26.732 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 98521 2024-09-14T06:31:26.732 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:26.732 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:31:26.732 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:26.733 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:31:26.733 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045782+0000", 2024-09-14T06:31:26.733 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:26.733 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:26.733 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:26.733 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:31:26.733 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045782+0000", 2024-09-14T06:31:26.733 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:26.734 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:26.734 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:26.734 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:31:26.734 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045783+0000", 2024-09-14T06:31:26.734 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.2499999999999999e-07 2024-09-14T06:31:26.734 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:26.734 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:26.734 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:31:26.734 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045784+0000", 2024-09-14T06:31:26.734 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.0109999999999999e-06 2024-09-14T06:31:26.734 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:26.734 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:26.734 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:31:26.734 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045786+0000", 2024-09-14T06:31:26.734 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.8289999999999999e-06 2024-09-14T06:31:26.735 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:26.735 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:26.735 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:31:26.735 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045793+0000", 2024-09-14T06:31:26.735 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.579e-06 2024-09-14T06:31:26.735 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:26.735 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:26.735 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:31:26.735 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.078338+0000", 2024-09-14T06:31:26.735 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032545389000000001 2024-09-14T06:31:26.735 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:26.736 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:26.736 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:31:26.736 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.078348+0000", 2024-09-14T06:31:26.736 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.0076e-05 2024-09-14T06:31:26.736 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:26.736 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:31:26.736 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:26.736 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:26.736 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:26.736 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:98522 3.10 3.1bf56f90 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T06:31:26.736 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:30:07.045796+0000", 2024-09-14T06:31:26.736 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 79.611954010999995, 2024-09-14T06:31:26.736 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032577846000000001, 2024-09-14T06:31:26.736 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:31:26.737 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:31:26.737 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "reached pg", 2024-09-14T06:31:26.737 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:31:26.737 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:31:26.737 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:26.737 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 98522 2024-09-14T06:31:26.737 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:26.737 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:31:26.737 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:26.737 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:31:26.737 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045796+0000", 2024-09-14T06:31:26.737 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:26.737 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:26.737 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:26.737 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:31:26.738 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045796+0000", 2024-09-14T06:31:26.738 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:26.738 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:26.738 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:26.738 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:31:26.738 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045796+0000", 2024-09-14T06:31:26.738 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.2100000000000001e-07 2024-09-14T06:31:26.738 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:26.738 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:26.738 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:31:26.738 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045797+0000", 2024-09-14T06:31:26.738 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.6899999999999996e-07 2024-09-14T06:31:26.738 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:26.738 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:26.738 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:31:26.739 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045800+0000", 2024-09-14T06:31:26.739 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.9979999999999999e-06 2024-09-14T06:31:26.739 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:26.739 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:26.739 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:31:26.739 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045803+0000", 2024-09-14T06:31:26.739 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.2850000000000001e-06 2024-09-14T06:31:26.739 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:26.739 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:26.739 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:31:26.740 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.078366+0000", 2024-09-14T06:31:26.740 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032562932000000003 2024-09-14T06:31:26.740 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:26.740 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:26.740 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:31:26.740 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.078374+0000", 2024-09-14T06:31:26.740 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.4409999999999993e-06 2024-09-14T06:31:26.740 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:26.740 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:31:26.740 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:26.740 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:26.740 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:26.740 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:98523 3.10 3.1bf56f90 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T06:31:26.741 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:30:07.045806+0000", 2024-09-14T06:31:26.741 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 79.611944468000004, 2024-09-14T06:31:26.741 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032611167000000003, 2024-09-14T06:31:26.741 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:31:26.741 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:31:26.741 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "reached pg", 2024-09-14T06:31:26.741 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:31:26.741 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:31:26.741 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:26.741 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 98523 2024-09-14T06:31:26.741 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:26.742 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:31:26.742 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:26.742 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:31:26.742 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045806+0000", 2024-09-14T06:31:26.742 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:26.742 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:26.742 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:26.742 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:31:26.742 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045806+0000", 2024-09-14T06:31:26.742 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:26.742 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:26.742 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:26.742 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:31:26.743 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045806+0000", 2024-09-14T06:31:26.743 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.2000000000000001e-07 2024-09-14T06:31:26.743 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:26.743 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:26.743 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:31:26.743 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045807+0000", 2024-09-14T06:31:26.743 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.4e-07 2024-09-14T06:31:26.743 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:26.743 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:26.743 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:31:26.744 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045810+0000", 2024-09-14T06:31:26.744 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.9229999999999998e-06 2024-09-14T06:31:26.744 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:26.744 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:26.744 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:31:26.744 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045812+0000", 2024-09-14T06:31:26.744 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.2220000000000001e-06 2024-09-14T06:31:26.744 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:26.744 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:26.744 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:31:26.744 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.078407+0000", 2024-09-14T06:31:26.744 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032594513999999998 2024-09-14T06:31:26.744 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:26.744 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:26.745 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:31:26.745 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.078417+0000", 2024-09-14T06:31:26.745 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.0348e-05 2024-09-14T06:31:26.745 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:26.745 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:31:26.745 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:26.745 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:26.745 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:26.745 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:98524 3.10 3.1bf56f90 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T06:31:26.745 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:30:07.045815+0000", 2024-09-14T06:31:26.745 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 79.611935099999997, 2024-09-14T06:31:26.746 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032643208999999999, 2024-09-14T06:31:26.746 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:31:26.746 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:31:26.746 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "reached pg", 2024-09-14T06:31:26.746 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:31:26.746 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:31:26.746 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:26.746 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 98524 2024-09-14T06:31:26.746 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:26.746 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:31:26.746 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:26.746 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:31:26.746 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045815+0000", 2024-09-14T06:31:26.746 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:26.747 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:26.747 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:26.747 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:31:26.747 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045815+0000", 2024-09-14T06:31:26.747 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:26.747 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:26.747 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:26.747 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:31:26.747 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045815+0000", 2024-09-14T06:31:26.747 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.11e-07 2024-09-14T06:31:26.747 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:26.747 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:26.747 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:31:26.747 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045816+0000", 2024-09-14T06:31:26.747 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.9099999999999991e-07 2024-09-14T06:31:26.748 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:26.748 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:26.748 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:31:26.748 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045819+0000", 2024-09-14T06:31:26.748 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.9399999999999998e-06 2024-09-14T06:31:26.748 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:26.748 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:26.748 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:31:26.748 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045822+0000", 2024-09-14T06:31:26.748 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.3470000000000001e-06 2024-09-14T06:31:26.748 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:26.748 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:26.748 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:31:26.749 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.078449+0000", 2024-09-14T06:31:26.749 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032627529000000002 2024-09-14T06:31:26.749 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:26.749 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:26.749 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:31:26.749 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.078458+0000", 2024-09-14T06:31:26.749 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.1910000000000006e-06 2024-09-14T06:31:26.749 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:26.749 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:31:26.749 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:26.749 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:26.749 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:31:26.749 INFO:teuthology.orchestra.run.smithi138.stdout:} 2024-09-14T06:31:26.750 INFO:tasks.ceph.osd.1.smithi094.stderr:2024-09-14T06:31:26.671+0000 7f328e47a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33641) UID: 0 2024-09-14T06:31:26.751 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-14T06:31:26.794 INFO:tasks.ceph.osd.4.smithi138.stderr:2024-09-14T06:31:26.792+0000 7f6c653c8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33615) UID: 0 2024-09-14T06:31:26.854 INFO:teuthology.orchestra.run.smithi138.stdout:{ 2024-09-14T06:31:26.854 INFO:teuthology.orchestra.run.smithi138.stdout: "ops": [], 2024-09-14T06:31:26.854 INFO:teuthology.orchestra.run.smithi138.stdout: "num_ops": 0 2024-09-14T06:31:26.854 INFO:teuthology.orchestra.run.smithi138.stdout:} 2024-09-14T06:31:26.858 INFO:tasks.ceph.osd.4.smithi138.stderr:2024-09-14T06:31:26.856+0000 7f6c653c8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33615) UID: 0 2024-09-14T06:31:26.863 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-14T06:31:26.960 INFO:tasks.ceph.osd.3.smithi094.stderr:2024-09-14T06:31:26.959+0000 7feeeafe8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33655) UID: 0 2024-09-14T06:31:27.005 INFO:teuthology.orchestra.run.smithi138.stdout:{ 2024-09-14T06:31:27.006 INFO:teuthology.orchestra.run.smithi138.stdout: "ops": [], 2024-09-14T06:31:27.006 INFO:teuthology.orchestra.run.smithi138.stdout: "complaint_time": 30, 2024-09-14T06:31:27.006 INFO:teuthology.orchestra.run.smithi138.stdout: "num_blocked_ops": 0 2024-09-14T06:31:27.006 INFO:teuthology.orchestra.run.smithi138.stdout:} 2024-09-14T06:31:27.014 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-14T06:31:27.061 INFO:tasks.ceph.osd.3.smithi094.stderr:2024-09-14T06:31:27.059+0000 7feeeafe8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33655) UID: 0 2024-09-14T06:31:27.118 INFO:teuthology.orchestra.run.smithi138.stdout:{ 2024-09-14T06:31:27.118 INFO:teuthology.orchestra.run.smithi138.stdout: "size": 20, 2024-09-14T06:31:27.118 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 600, 2024-09-14T06:31:27.118 INFO:teuthology.orchestra.run.smithi138.stdout: "ops": [ 2024-09-14T06:31:27.119 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:27.119 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:53244 3.f 3:f71f51a7:::rbd_data.112dca2f6ae1.0000000000000015:17 [read 1863680~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:31:27.119 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:44.369399+0000", 2024-09-14T06:31:27.119 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 102.747341256, 2024-09-14T06:31:27.119 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047291189999999997, 2024-09-14T06:31:27.119 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:31:27.119 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:31:27.119 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:31:27.119 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:31:27.119 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:31:27.119 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:27.119 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 53244 2024-09-14T06:31:27.119 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:27.120 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:31:27.120 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:27.120 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:31:27.120 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369399+0000", 2024-09-14T06:31:27.120 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:27.120 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:27.120 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:27.120 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:31:27.120 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369399+0000", 2024-09-14T06:31:27.120 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:27.120 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:27.120 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:27.120 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:31:27.121 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369400+0000", 2024-09-14T06:31:27.121 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.0699999999999999e-07 2024-09-14T06:31:27.121 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:27.121 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:27.121 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:31:27.121 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369401+0000", 2024-09-14T06:31:27.121 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.4300000000000001e-06 2024-09-14T06:31:27.121 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:27.121 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:27.121 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:31:27.121 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369404+0000", 2024-09-14T06:31:27.121 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.061e-06 2024-09-14T06:31:27.121 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:27.122 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:27.122 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:31:27.122 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369407+0000", 2024-09-14T06:31:27.122 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.6139999999999999e-06 2024-09-14T06:31:27.122 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:27.122 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:27.122 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:31:27.122 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.416517+0000", 2024-09-14T06:31:27.122 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047110338000000002 2024-09-14T06:31:27.122 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:27.122 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:27.122 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:31:27.122 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.416616+0000", 2024-09-14T06:31:27.123 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.8651000000000001e-05 2024-09-14T06:31:27.123 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:27.123 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:27.123 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:31:27.123 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.416691+0000", 2024-09-14T06:31:27.123 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.4888999999999994e-05 2024-09-14T06:31:27.123 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:27.123 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:31:27.123 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:27.123 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:27.123 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:27.123 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:53245 3.f 3:f71f51a7:::rbd_data.112dca2f6ae1.0000000000000015:17 [read 1888256~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:31:27.123 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:44.369410+0000", 2024-09-14T06:31:27.123 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 102.747330968, 2024-09-14T06:31:27.124 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047399275999999997, 2024-09-14T06:31:27.124 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:31:27.124 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:31:27.124 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:31:27.124 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:31:27.124 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:31:27.124 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:27.124 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 53245 2024-09-14T06:31:27.124 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:27.124 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:31:27.124 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:27.124 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:31:27.124 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369410+0000", 2024-09-14T06:31:27.124 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:27.125 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:27.125 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:27.125 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:31:27.125 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369410+0000", 2024-09-14T06:31:27.125 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:27.125 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:27.125 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:27.125 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:31:27.125 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369410+0000", 2024-09-14T06:31:27.125 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.1899999999999999e-07 2024-09-14T06:31:27.125 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:27.125 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:27.126 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:31:27.126 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369411+0000", 2024-09-14T06:31:27.126 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.0300000000000001e-06 2024-09-14T06:31:27.126 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:27.126 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:27.126 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:31:27.126 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369414+0000", 2024-09-14T06:31:27.126 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.1099999999999999e-06 2024-09-14T06:31:27.126 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:27.126 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:27.126 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:31:27.126 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369417+0000", 2024-09-14T06:31:27.126 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.6189999999999998e-06 2024-09-14T06:31:27.126 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:27.126 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:27.127 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:31:27.127 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.416718+0000", 2024-09-14T06:31:27.127 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047301322999999999 2024-09-14T06:31:27.127 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:27.127 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:27.127 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:31:27.127 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.416759+0000", 2024-09-14T06:31:27.127 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.1009e-05 2024-09-14T06:31:27.127 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:27.127 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:27.127 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:31:27.127 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.416809+0000", 2024-09-14T06:31:27.127 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.9966000000000002e-05 2024-09-14T06:31:27.127 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:27.128 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:31:27.128 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:27.128 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:27.128 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:27.128 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:53246 3.f 3:f71f51a7:::rbd_data.112dca2f6ae1.0000000000000015:17 [read 1912832~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:31:27.128 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:44.369420+0000", 2024-09-14T06:31:27.128 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 102.747321024, 2024-09-14T06:31:27.128 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047529463000000001, 2024-09-14T06:31:27.128 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:31:27.128 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:31:27.128 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:31:27.128 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:31:27.128 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:31:27.128 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:27.129 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 53246 2024-09-14T06:31:27.129 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:27.129 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:31:27.129 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:27.129 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:31:27.129 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369420+0000", 2024-09-14T06:31:27.129 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:27.129 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:27.129 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:27.129 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:31:27.129 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369420+0000", 2024-09-14T06:31:27.129 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:27.129 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:27.129 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:27.130 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:31:27.130 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369420+0000", 2024-09-14T06:31:27.130 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.1299999999999999e-07 2024-09-14T06:31:27.130 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:27.130 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:27.130 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:31:27.130 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369425+0000", 2024-09-14T06:31:27.130 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.0130000000000001e-06 2024-09-14T06:31:27.130 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:27.130 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:27.130 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:31:27.130 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369428+0000", 2024-09-14T06:31:27.130 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.349e-06 2024-09-14T06:31:27.130 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:27.130 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:27.131 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:31:27.131 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369431+0000", 2024-09-14T06:31:27.131 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.7389999999999999e-06 2024-09-14T06:31:27.131 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:27.131 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:27.131 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:31:27.131 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.416834+0000", 2024-09-14T06:31:27.131 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047402669000000001 2024-09-14T06:31:27.131 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:27.131 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:27.131 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:31:27.131 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.416882+0000", 2024-09-14T06:31:27.131 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.8068999999999997e-05 2024-09-14T06:31:27.131 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:27.131 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:27.132 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:31:27.132 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.416949+0000", 2024-09-14T06:31:27.132 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.7410999999999995e-05 2024-09-14T06:31:27.132 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:27.132 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:31:27.132 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:27.132 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:27.132 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:27.132 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:53247 3.f 3:f71f51a7:::rbd_data.112dca2f6ae1.0000000000000015:17 [read 1937408~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:31:27.132 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:44.369434+0000", 2024-09-14T06:31:27.132 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 102.74730626, 2024-09-14T06:31:27.132 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047652264999999999, 2024-09-14T06:31:27.132 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:31:27.132 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:31:27.133 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:31:27.133 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:31:27.133 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:31:27.133 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:27.133 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 53247 2024-09-14T06:31:27.133 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:27.133 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:31:27.133 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:27.133 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:31:27.133 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369434+0000", 2024-09-14T06:31:27.133 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:27.133 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:27.133 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:27.133 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:31:27.133 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369434+0000", 2024-09-14T06:31:27.134 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:27.134 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:27.134 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:27.134 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:31:27.134 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369435+0000", 2024-09-14T06:31:27.134 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.2100000000000001e-07 2024-09-14T06:31:27.134 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:27.134 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:27.134 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:31:27.134 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369436+0000", 2024-09-14T06:31:27.134 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.122e-06 2024-09-14T06:31:27.134 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:27.134 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:27.134 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:31:27.135 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369439+0000", 2024-09-14T06:31:27.135 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.0400000000000001e-06 2024-09-14T06:31:27.135 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:27.135 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:27.135 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:31:27.135 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369441+0000", 2024-09-14T06:31:27.135 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.728e-06 2024-09-14T06:31:27.135 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:27.135 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:27.135 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:31:27.135 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.416977+0000", 2024-09-14T06:31:27.135 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047535344 2024-09-14T06:31:27.135 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:27.136 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:27.136 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:31:27.136 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.417029+0000", 2024-09-14T06:31:27.136 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.2037999999999999e-05 2024-09-14T06:31:27.136 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:27.136 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:27.136 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:31:27.136 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.417087+0000", 2024-09-14T06:31:27.136 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.7772000000000002e-05 2024-09-14T06:31:27.136 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:27.136 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:31:27.137 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:27.137 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:27.137 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:27.137 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:53248 3.f 3:f71f51a7:::rbd_data.112dca2f6ae1.0000000000000015:17 [read 1961984~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:31:27.137 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:44.369445+0000", 2024-09-14T06:31:27.137 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 102.747295708, 2024-09-14T06:31:27.137 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047798489, 2024-09-14T06:31:27.137 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:31:27.137 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:31:27.137 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:31:27.137 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:31:27.137 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:31:27.137 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:27.137 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 53248 2024-09-14T06:31:27.138 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:27.138 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:31:27.138 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:27.138 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:31:27.138 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369445+0000", 2024-09-14T06:31:27.138 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:27.138 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:27.138 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:27.138 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:31:27.138 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369445+0000", 2024-09-14T06:31:27.138 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:27.138 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:27.138 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:27.138 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:31:27.139 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369445+0000", 2024-09-14T06:31:27.139 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.0699999999999999e-07 2024-09-14T06:31:27.139 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:27.139 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:27.139 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:31:27.139 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369446+0000", 2024-09-14T06:31:27.139 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.1000000000000001e-06 2024-09-14T06:31:27.139 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:27.139 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:27.139 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:31:27.139 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369449+0000", 2024-09-14T06:31:27.139 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.1039999999999998e-06 2024-09-14T06:31:27.139 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:27.139 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:27.139 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:31:27.140 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369452+0000", 2024-09-14T06:31:27.140 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.6199999999999999e-06 2024-09-14T06:31:27.140 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:27.140 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:27.140 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:31:27.140 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.417114+0000", 2024-09-14T06:31:27.140 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047661901999999999 2024-09-14T06:31:27.140 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:27.140 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:27.140 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:31:27.140 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.417181+0000", 2024-09-14T06:31:27.140 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.7385999999999995e-05 2024-09-14T06:31:27.140 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:27.140 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:27.141 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:31:27.141 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.417243+0000", 2024-09-14T06:31:27.141 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.2169999999999996e-05 2024-09-14T06:31:27.141 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:27.141 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:31:27.141 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:27.141 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:27.141 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:27.141 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:53249 3.f 3:f71f51a7:::rbd_data.112dca2f6ae1.0000000000000015:17 [read 1986560~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:31:27.141 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:44.369455+0000", 2024-09-14T06:31:27.141 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 102.747285397, 2024-09-14T06:31:27.141 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047890493999999999, 2024-09-14T06:31:27.141 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:31:27.141 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:31:27.142 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:31:27.142 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:31:27.142 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:31:27.142 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:27.142 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 53249 2024-09-14T06:31:27.142 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:27.142 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:31:27.142 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:27.142 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:31:27.142 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369455+0000", 2024-09-14T06:31:27.142 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:27.142 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:27.142 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:27.142 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:31:27.142 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369455+0000", 2024-09-14T06:31:27.143 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:27.143 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:27.143 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:27.143 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:31:27.143 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369455+0000", 2024-09-14T06:31:27.143 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.0800000000000001e-07 2024-09-14T06:31:27.143 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:27.143 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:27.143 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:31:27.143 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369456+0000", 2024-09-14T06:31:27.143 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.0729999999999999e-06 2024-09-14T06:31:27.143 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:27.143 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:27.143 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:31:27.144 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369460+0000", 2024-09-14T06:31:27.144 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.027e-06 2024-09-14T06:31:27.144 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:27.144 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:27.144 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:31:27.144 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369462+0000", 2024-09-14T06:31:27.144 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.5320000000000001e-06 2024-09-14T06:31:27.144 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:27.144 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:27.144 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:31:27.144 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.417257+0000", 2024-09-14T06:31:27.144 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.04779489 2024-09-14T06:31:27.144 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:27.145 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:27.145 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:31:27.145 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.417296+0000", 2024-09-14T06:31:27.145 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.8658000000000002e-05 2024-09-14T06:31:27.145 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:27.145 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:27.145 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:31:27.145 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.417346+0000", 2024-09-14T06:31:27.145 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.0105999999999998e-05 2024-09-14T06:31:27.145 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:27.145 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:31:27.145 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:27.145 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:27.145 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:27.145 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:53250 3.f 3:f71f51a7:::rbd_data.112dca2f6ae1.0000000000000015:17 [read 2011136~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:31:27.146 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:44.369465+0000", 2024-09-14T06:31:27.146 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 102.74727542700001, 2024-09-14T06:31:27.146 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.048032230000000002, 2024-09-14T06:31:27.146 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:31:27.146 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:31:27.146 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:31:27.146 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:31:27.146 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:31:27.146 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:27.146 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 53250 2024-09-14T06:31:27.146 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:27.146 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:31:27.146 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:27.146 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:31:27.146 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369465+0000", 2024-09-14T06:31:27.147 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:27.147 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:27.147 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:27.147 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:31:27.147 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369465+0000", 2024-09-14T06:31:27.147 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:27.147 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:27.147 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:27.147 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:31:27.147 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369465+0000", 2024-09-14T06:31:27.147 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.2100000000000001e-07 2024-09-14T06:31:27.147 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:27.147 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:27.148 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:31:27.148 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369466+0000", 2024-09-14T06:31:27.148 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.0640000000000001e-06 2024-09-14T06:31:27.148 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:27.148 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:27.148 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:31:27.148 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369469+0000", 2024-09-14T06:31:27.148 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.0170000000000001e-06 2024-09-14T06:31:27.148 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:27.148 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:27.148 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:31:27.148 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369472+0000", 2024-09-14T06:31:27.148 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.6189999999999998e-06 2024-09-14T06:31:27.148 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:27.149 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:27.149 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:31:27.149 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.417383+0000", 2024-09-14T06:31:27.149 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047911092000000002 2024-09-14T06:31:27.149 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:27.149 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:27.149 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:31:27.149 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.417430+0000", 2024-09-14T06:31:27.149 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.6898999999999999e-05 2024-09-14T06:31:27.149 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:27.149 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:27.149 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:31:27.149 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.417497+0000", 2024-09-14T06:31:27.149 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.7317999999999995e-05 2024-09-14T06:31:27.150 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:27.150 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:31:27.150 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:27.150 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:27.150 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:27.150 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:53251 3.f 3:f71f51a7:::rbd_data.112dca2f6ae1.0000000000000015:17 [read 2035712~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:31:27.150 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:44.369475+0000", 2024-09-14T06:31:27.150 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 102.747265431, 2024-09-14T06:31:27.150 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.048154542000000002, 2024-09-14T06:31:27.150 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:31:27.150 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:31:27.150 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:31:27.150 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:31:27.150 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:31:27.150 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:27.151 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 53251 2024-09-14T06:31:27.151 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:27.151 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:31:27.151 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:27.151 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:31:27.151 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369475+0000", 2024-09-14T06:31:27.151 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:27.151 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:27.151 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:27.151 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:31:27.151 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369475+0000", 2024-09-14T06:31:27.151 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:27.151 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:27.152 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:27.152 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:31:27.152 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369475+0000", 2024-09-14T06:31:27.152 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.0100000000000001e-07 2024-09-14T06:31:27.152 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:27.152 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:27.152 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:31:27.152 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369476+0000", 2024-09-14T06:31:27.152 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.0580000000000001e-06 2024-09-14T06:31:27.152 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:27.152 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:27.152 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:31:27.152 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369479+0000", 2024-09-14T06:31:27.152 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.9950000000000001e-06 2024-09-14T06:31:27.153 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:27.153 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:27.153 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:31:27.153 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369482+0000", 2024-09-14T06:31:27.153 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.6340000000000001e-06 2024-09-14T06:31:27.153 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:27.153 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:27.153 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:31:27.153 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.417530+0000", 2024-09-14T06:31:27.153 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.048047535000000002 2024-09-14T06:31:27.153 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:27.153 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:27.153 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:31:27.154 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.417574+0000", 2024-09-14T06:31:27.154 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.4255999999999998e-05 2024-09-14T06:31:27.154 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:27.154 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:27.154 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:31:27.154 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.417630+0000", 2024-09-14T06:31:27.154 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.5862999999999997e-05 2024-09-14T06:31:27.154 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:27.154 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:31:27.154 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:27.154 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:27.154 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:27.154 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:53252 3.f 3:f71f51a7:::rbd_data.112dca2f6ae1.0000000000000015:17 [read 2060288~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:31:27.154 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:44.369485+0000", 2024-09-14T06:31:27.155 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 102.747255583, 2024-09-14T06:31:27.155 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.048254116, 2024-09-14T06:31:27.155 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:31:27.155 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:31:27.155 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:31:27.155 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:31:27.155 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:31:27.155 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:27.155 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 53252 2024-09-14T06:31:27.155 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:27.155 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:31:27.155 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:27.155 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:31:27.156 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369485+0000", 2024-09-14T06:31:27.156 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:27.156 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:27.156 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:27.156 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:31:27.156 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369485+0000", 2024-09-14T06:31:27.156 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:27.156 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:27.156 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:27.156 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:31:27.156 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369485+0000", 2024-09-14T06:31:27.156 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.1e-07 2024-09-14T06:31:27.156 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:27.157 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:27.157 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:31:27.157 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369487+0000", 2024-09-14T06:31:27.157 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.42e-06 2024-09-14T06:31:27.157 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:27.157 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:27.157 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:31:27.157 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369490+0000", 2024-09-14T06:31:27.157 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.9639999999999999e-06 2024-09-14T06:31:27.157 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:27.157 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:27.157 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:31:27.157 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369492+0000", 2024-09-14T06:31:27.157 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.5679999999999998e-06 2024-09-14T06:31:27.158 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:27.158 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:27.158 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:31:27.158 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.417645+0000", 2024-09-14T06:31:27.158 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.048152753 2024-09-14T06:31:27.158 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:27.158 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:27.158 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:31:27.158 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.417687+0000", 2024-09-14T06:31:27.158 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.1900999999999997e-05 2024-09-14T06:31:27.158 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:27.158 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:27.158 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:31:27.158 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.417739+0000", 2024-09-14T06:31:27.159 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.2299999999999997e-05 2024-09-14T06:31:27.159 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:27.159 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:31:27.159 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:27.159 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:27.159 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:27.159 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:53253 3.f 3:f71f51a7:::rbd_data.112dca2f6ae1.0000000000000015:17 [read 2084864~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:31:27.159 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:44.369495+0000", 2024-09-14T06:31:27.159 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 102.747245459, 2024-09-14T06:31:27.159 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.048366785000000002, 2024-09-14T06:31:27.159 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:31:27.159 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:31:27.159 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:31:27.159 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:31:27.160 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:31:27.160 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:27.160 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 53253 2024-09-14T06:31:27.160 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:27.160 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:31:27.160 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:27.160 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:31:27.160 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369495+0000", 2024-09-14T06:31:27.160 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:27.160 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:27.160 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:27.160 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:31:27.160 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369495+0000", 2024-09-14T06:31:27.160 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:27.161 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:27.161 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:27.161 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:31:27.161 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370388+0000", 2024-09-14T06:31:27.161 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00089287999999999998 2024-09-14T06:31:27.161 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:27.161 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:27.161 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:31:27.161 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370392+0000", 2024-09-14T06:31:27.161 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.2130000000000002e-06 2024-09-14T06:31:27.161 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:27.161 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:27.161 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:31:27.161 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370401+0000", 2024-09-14T06:31:27.162 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.7649999999999999e-06 2024-09-14T06:31:27.162 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:27.162 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:27.162 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:31:27.162 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370406+0000", 2024-09-14T06:31:27.162 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.0159999999999999e-06 2024-09-14T06:31:27.162 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:27.162 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:27.162 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:31:27.162 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.417764+0000", 2024-09-14T06:31:27.162 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047358348000000001 2024-09-14T06:31:27.162 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:27.162 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:27.162 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:31:27.162 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.417806+0000", 2024-09-14T06:31:27.163 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.1149999999999997e-05 2024-09-14T06:31:27.163 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:27.163 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:27.163 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:31:27.163 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.417862+0000", 2024-09-14T06:31:27.163 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.6412999999999997e-05 2024-09-14T06:31:27.163 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:27.163 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:31:27.163 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:27.163 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:27.163 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:27.163 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:53340 3.f 3:f71f51a7:::rbd_data.112dca2f6ae1.0000000000000015:17 [read 1056768~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:31:27.163 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:44.370421+0000", 2024-09-14T06:31:27.163 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 102.746319579, 2024-09-14T06:31:27.163 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047566313999999998, 2024-09-14T06:31:27.164 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:31:27.164 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:31:27.164 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:31:27.164 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:31:27.164 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:31:27.164 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:27.164 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 53340 2024-09-14T06:31:27.164 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:27.164 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:31:27.164 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:27.164 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:31:27.164 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370421+0000", 2024-09-14T06:31:27.164 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:27.164 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:27.164 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:27.165 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:31:27.165 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370421+0000", 2024-09-14T06:31:27.165 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:27.165 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:27.165 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:27.165 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:31:27.165 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370422+0000", 2024-09-14T06:31:27.165 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.3799999999999996e-07 2024-09-14T06:31:27.165 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:27.165 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:27.165 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:31:27.165 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370424+0000", 2024-09-14T06:31:27.165 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.6919999999999998e-06 2024-09-14T06:31:27.165 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:27.166 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:27.166 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:31:27.166 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370432+0000", 2024-09-14T06:31:27.166 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.2439999999999996e-06 2024-09-14T06:31:27.166 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:27.166 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:27.166 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:31:27.166 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370437+0000", 2024-09-14T06:31:27.166 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.5389999999999998e-06 2024-09-14T06:31:27.166 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:27.166 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:27.167 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:31:27.167 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.417888+0000", 2024-09-14T06:31:27.167 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047450432000000001 2024-09-14T06:31:27.167 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:27.167 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:27.167 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:31:27.167 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.417935+0000", 2024-09-14T06:31:27.167 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.7454999999999999e-05 2024-09-14T06:31:27.167 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:27.167 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:27.167 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:31:27.167 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.417987+0000", 2024-09-14T06:31:27.167 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.2213999999999998e-05 2024-09-14T06:31:27.167 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:27.168 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:31:27.168 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:27.168 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:27.168 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:27.168 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:53341 3.f 3:f71f51a7:::rbd_data.112dca2f6ae1.0000000000000015:17 [read 1081344~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:31:27.168 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:44.370446+0000", 2024-09-14T06:31:27.168 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 102.746294306, 2024-09-14T06:31:27.168 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047665723, 2024-09-14T06:31:27.168 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:31:27.168 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:31:27.168 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:31:27.168 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:31:27.168 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:31:27.169 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:27.169 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 53341 2024-09-14T06:31:27.169 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:27.169 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:31:27.169 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:27.169 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:31:27.169 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370446+0000", 2024-09-14T06:31:27.169 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:27.169 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:27.169 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:27.169 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:31:27.169 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370446+0000", 2024-09-14T06:31:27.169 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:27.169 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:27.170 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:27.170 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:31:27.170 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370447+0000", 2024-09-14T06:31:27.170 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.6100000000000002e-07 2024-09-14T06:31:27.170 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:27.170 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:27.170 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:31:27.170 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370448+0000", 2024-09-14T06:31:27.170 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.7370000000000001e-06 2024-09-14T06:31:27.170 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:27.170 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:27.170 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:31:27.170 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370462+0000", 2024-09-14T06:31:27.170 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.3227000000000001e-05 2024-09-14T06:31:27.170 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:27.171 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:27.171 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:31:27.171 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370467+0000", 2024-09-14T06:31:27.171 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.6540000000000004e-06 2024-09-14T06:31:27.171 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:27.171 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:27.171 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:31:27.171 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.418013+0000", 2024-09-14T06:31:27.171 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047545325999999999 2024-09-14T06:31:27.171 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:27.171 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:27.171 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:31:27.171 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.418054+0000", 2024-09-14T06:31:27.171 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.1696999999999997e-05 2024-09-14T06:31:27.172 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:27.172 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:27.172 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:31:27.172 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.418112+0000", 2024-09-14T06:31:27.172 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.7720999999999998e-05 2024-09-14T06:31:27.172 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:27.172 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:31:27.172 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:27.172 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:27.172 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:27.172 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:53342 3.f 3:f71f51a7:::rbd_data.112dca2f6ae1.0000000000000015:17 [read 1105920~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:31:27.172 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:44.370475+0000", 2024-09-14T06:31:27.172 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 102.746265245, 2024-09-14T06:31:27.172 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047748848000000003, 2024-09-14T06:31:27.173 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:31:27.173 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:31:27.173 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:31:27.173 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:31:27.173 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:31:27.173 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:27.173 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 53342 2024-09-14T06:31:27.173 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:27.173 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:31:27.173 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:27.173 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:31:27.173 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370475+0000", 2024-09-14T06:31:27.173 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:27.173 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:27.173 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:27.174 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:31:27.174 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370475+0000", 2024-09-14T06:31:27.174 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:27.174 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:27.174 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:27.174 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:31:27.174 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370476+0000", 2024-09-14T06:31:27.174 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.4700000000000002e-07 2024-09-14T06:31:27.174 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:27.174 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:27.174 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:31:27.174 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370478+0000", 2024-09-14T06:31:27.174 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.5890000000000001e-06 2024-09-14T06:31:27.174 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:27.174 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:27.175 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:31:27.175 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370486+0000", 2024-09-14T06:31:27.175 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.3470000000000001e-06 2024-09-14T06:31:27.175 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:27.175 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:27.175 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:31:27.175 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370491+0000", 2024-09-14T06:31:27.175 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.5720000000000002e-06 2024-09-14T06:31:27.175 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:27.175 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:27.175 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:31:27.175 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.418135+0000", 2024-09-14T06:31:27.175 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047644094999999997 2024-09-14T06:31:27.175 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:27.175 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:27.175 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:31:27.176 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.418174+0000", 2024-09-14T06:31:27.176 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.8204000000000002e-05 2024-09-14T06:31:27.176 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:27.176 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:27.176 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:31:27.176 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.418224+0000", 2024-09-14T06:31:27.176 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.0593999999999997e-05 2024-09-14T06:31:27.176 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:27.176 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:31:27.176 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:27.176 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:27.176 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:27.176 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:53343 3.f 3:f71f51a7:::rbd_data.112dca2f6ae1.0000000000000015:17 [read 1130496~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:31:27.176 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:44.370498+0000", 2024-09-14T06:31:27.177 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 102.746242369, 2024-09-14T06:31:27.177 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047841574999999997, 2024-09-14T06:31:27.177 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:31:27.177 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:31:27.177 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:31:27.177 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:31:27.177 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:31:27.177 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:27.177 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 53343 2024-09-14T06:31:27.177 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:27.177 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:31:27.177 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:27.177 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:31:27.177 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370498+0000", 2024-09-14T06:31:27.178 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:27.178 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:27.178 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:27.178 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:31:27.178 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370498+0000", 2024-09-14T06:31:27.178 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:27.178 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:27.178 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:27.178 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:31:27.178 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370499+0000", 2024-09-14T06:31:27.178 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.9999999999999998e-07 2024-09-14T06:31:27.178 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:27.178 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:27.178 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:31:27.179 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370501+0000", 2024-09-14T06:31:27.179 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.0310000000000001e-06 2024-09-14T06:31:27.179 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:27.179 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:27.179 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:31:27.179 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370507+0000", 2024-09-14T06:31:27.179 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.4169999999999997e-06 2024-09-14T06:31:27.179 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:27.179 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:27.179 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:31:27.179 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370516+0000", 2024-09-14T06:31:27.179 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.7190000000000007e-06 2024-09-14T06:31:27.179 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:27.179 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:27.179 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:31:27.180 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.418247+0000", 2024-09-14T06:31:27.180 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047731190999999999 2024-09-14T06:31:27.180 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:27.180 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:27.180 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:31:27.180 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.418286+0000", 2024-09-14T06:31:27.180 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.8988e-05 2024-09-14T06:31:27.180 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:27.180 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:27.180 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:31:27.180 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.418340+0000", 2024-09-14T06:31:27.180 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.3829000000000002e-05 2024-09-14T06:31:27.180 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:27.180 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:31:27.180 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:27.181 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:27.181 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:27.181 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:53344 3.f 3:f71f51a7:::rbd_data.112dca2f6ae1.0000000000000015:17 [read 1155072~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:31:27.181 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:44.370524+0000", 2024-09-14T06:31:27.181 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 102.74621659899999, 2024-09-14T06:31:27.181 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047972262000000002, 2024-09-14T06:31:27.181 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:31:27.181 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:31:27.181 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:31:27.181 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:31:27.181 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:31:27.181 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:27.181 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 53344 2024-09-14T06:31:27.181 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:27.181 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:31:27.182 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:27.182 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:31:27.182 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370524+0000", 2024-09-14T06:31:27.182 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:27.182 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:27.182 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:27.182 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:31:27.182 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370524+0000", 2024-09-14T06:31:27.182 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:27.182 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:27.182 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:27.182 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:31:27.182 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370618+0000", 2024-09-14T06:31:27.182 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.4425999999999993e-05 2024-09-14T06:31:27.182 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:27.183 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:27.183 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:31:27.183 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370622+0000", 2024-09-14T06:31:27.183 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.3519999999999998e-06 2024-09-14T06:31:27.183 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:27.183 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:27.183 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:31:27.183 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370631+0000", 2024-09-14T06:31:27.183 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.9299999999999992e-06 2024-09-14T06:31:27.183 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:27.183 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:27.183 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:31:27.183 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370637+0000", 2024-09-14T06:31:27.183 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.3269999999999997e-06 2024-09-14T06:31:27.184 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:27.184 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:27.184 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:31:27.184 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.418372+0000", 2024-09-14T06:31:27.184 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047734764999999998 2024-09-14T06:31:27.184 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:27.184 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:27.184 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:31:27.184 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.418431+0000", 2024-09-14T06:31:27.184 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.8696999999999997e-05 2024-09-14T06:31:27.185 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:27.185 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:27.185 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:31:27.185 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.418496+0000", 2024-09-14T06:31:27.185 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.5765000000000005e-05 2024-09-14T06:31:27.185 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:27.185 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:31:27.185 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:27.185 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:27.185 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:27.185 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:53345 3.f 3:f71f51a7:::rbd_data.112dca2f6ae1.0000000000000015:17 [read 1179648~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:31:27.185 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:44.370647+0000", 2024-09-14T06:31:27.185 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 102.746093132, 2024-09-14T06:31:27.186 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047953775999999997, 2024-09-14T06:31:27.186 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:31:27.186 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:31:27.186 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:31:27.186 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:31:27.186 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:31:27.186 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:27.186 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 53345 2024-09-14T06:31:27.186 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:27.186 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:31:27.186 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:27.186 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:31:27.186 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370647+0000", 2024-09-14T06:31:27.186 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:27.187 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:27.187 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:27.187 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:31:27.187 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370647+0000", 2024-09-14T06:31:27.187 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:27.187 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:27.187 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:27.187 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:31:27.187 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.372346+0000", 2024-09-14T06:31:27.187 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.001698284 2024-09-14T06:31:27.187 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:27.187 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:27.187 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:31:27.187 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.372351+0000", 2024-09-14T06:31:27.188 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.8890000000000001e-06 2024-09-14T06:31:27.188 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:27.188 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:27.188 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:31:27.188 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.372362+0000", 2024-09-14T06:31:27.188 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.0941000000000001e-05 2024-09-14T06:31:27.188 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:27.188 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:27.188 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:31:27.188 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.372369+0000", 2024-09-14T06:31:27.188 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.362e-06 2024-09-14T06:31:27.188 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:27.188 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:27.188 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:31:27.189 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.418510+0000", 2024-09-14T06:31:27.189 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046140914999999998 2024-09-14T06:31:27.189 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:27.189 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:27.189 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:31:27.189 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.418549+0000", 2024-09-14T06:31:27.189 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.8717999999999998e-05 2024-09-14T06:31:27.189 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:27.189 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:27.189 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:31:27.190 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.418601+0000", 2024-09-14T06:31:27.190 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.2667000000000002e-05 2024-09-14T06:31:27.190 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:27.190 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:31:27.190 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:27.190 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:27.190 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:27.190 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:53351 3.f 3:f71f51a7:::rbd_data.112dca2f6ae1.0000000000000015:17 [read 1327104~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:31:27.190 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:44.372499+0000", 2024-09-14T06:31:27.190 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 102.744242011, 2024-09-14T06:31:27.190 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047140596, 2024-09-14T06:31:27.190 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:31:27.190 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:31:27.190 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:31:27.191 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:31:27.191 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:31:27.191 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:27.191 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 53351 2024-09-14T06:31:27.191 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:27.191 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:31:27.191 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:27.191 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:31:27.191 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.372499+0000", 2024-09-14T06:31:27.191 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:27.191 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:27.191 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:27.191 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:31:27.191 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.372499+0000", 2024-09-14T06:31:27.192 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:27.192 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:27.192 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:27.192 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:31:27.192 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.372499+0000", 2024-09-14T06:31:27.192 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.3000000000000001e-07 2024-09-14T06:31:27.192 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:27.192 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:27.192 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:31:27.192 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.372501+0000", 2024-09-14T06:31:27.192 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.0820000000000001e-06 2024-09-14T06:31:27.192 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:27.192 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:27.192 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:31:27.193 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.372513+0000", 2024-09-14T06:31:27.193 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.1610999999999999e-05 2024-09-14T06:31:27.193 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:27.193 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:27.193 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:31:27.193 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.372518+0000", 2024-09-14T06:31:27.193 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.327e-06 2024-09-14T06:31:27.193 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:27.193 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:27.193 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:31:27.193 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.419530+0000", 2024-09-14T06:31:27.193 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047011798 2024-09-14T06:31:27.193 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:27.194 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:27.194 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:31:27.194 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.419578+0000", 2024-09-14T06:31:27.194 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.8446999999999999e-05 2024-09-14T06:31:27.194 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:27.194 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:27.194 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:31:27.194 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.419639+0000", 2024-09-14T06:31:27.194 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.0900999999999998e-05 2024-09-14T06:31:27.194 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:27.194 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:31:27.194 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:27.194 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:27.194 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:27.195 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:53352 3.f 3:f71f51a7:::rbd_data.112dca2f6ae1.0000000000000015:17 [read 1351680~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:31:27.195 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:44.372525+0000", 2024-09-14T06:31:27.195 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 102.74421609700001, 2024-09-14T06:31:27.195 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047259847000000001, 2024-09-14T06:31:27.195 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:31:27.195 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:31:27.195 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:31:27.195 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:31:27.195 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:31:27.195 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:27.195 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 53352 2024-09-14T06:31:27.195 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:27.195 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:31:27.195 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:27.196 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:31:27.196 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.372525+0000", 2024-09-14T06:31:27.196 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:27.196 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:27.196 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:27.196 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:31:27.196 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.372525+0000", 2024-09-14T06:31:27.196 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:27.196 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:27.196 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:27.196 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:31:27.196 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.372525+0000", 2024-09-14T06:31:27.196 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.0600000000000001e-07 2024-09-14T06:31:27.196 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:27.197 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:27.197 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:31:27.197 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.372527+0000", 2024-09-14T06:31:27.197 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.0499999999999999e-06 2024-09-14T06:31:27.197 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:27.197 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:27.197 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:31:27.197 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.372533+0000", 2024-09-14T06:31:27.197 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.4320000000000004e-06 2024-09-14T06:31:27.197 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:27.197 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:27.197 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:31:27.197 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.372538+0000", 2024-09-14T06:31:27.198 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.9130000000000002e-06 2024-09-14T06:31:27.198 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:27.198 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:27.198 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:31:27.198 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.419667+0000", 2024-09-14T06:31:27.198 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047128976000000003 2024-09-14T06:31:27.198 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:27.198 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:27.198 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:31:27.198 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.419715+0000", 2024-09-14T06:31:27.198 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.7868999999999999e-05 2024-09-14T06:31:27.198 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:27.198 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:27.198 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:31:27.199 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.419784+0000", 2024-09-14T06:31:27.199 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.9201000000000004e-05 2024-09-14T06:31:27.199 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:27.199 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:31:27.199 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:27.199 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:27.199 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:27.199 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:53353 3.f 3:f71f51a7:::rbd_data.112dca2f6ae1.0000000000000015:17 [read 1376256~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:31:27.199 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:44.372545+0000", 2024-09-14T06:31:27.199 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 102.744196057, 2024-09-14T06:31:27.199 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047364232999999999, 2024-09-14T06:31:27.199 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:31:27.199 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:31:27.199 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:31:27.200 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:31:27.200 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:31:27.200 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:27.200 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 53353 2024-09-14T06:31:27.200 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:27.200 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:31:27.200 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:27.200 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:31:27.200 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.372545+0000", 2024-09-14T06:31:27.200 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:27.200 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:27.200 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:27.200 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:31:27.200 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.372545+0000", 2024-09-14T06:31:27.201 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:27.201 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:27.201 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:27.201 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:31:27.201 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.372545+0000", 2024-09-14T06:31:27.201 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.27e-07 2024-09-14T06:31:27.201 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:27.201 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:27.201 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:31:27.201 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.372547+0000", 2024-09-14T06:31:27.201 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.0600000000000002e-06 2024-09-14T06:31:27.201 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:27.201 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:27.201 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:31:27.202 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.372553+0000", 2024-09-14T06:31:27.202 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.7980000000000002e-06 2024-09-14T06:31:27.202 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:27.202 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:27.202 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:31:27.202 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.372557+0000", 2024-09-14T06:31:27.202 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.7389999999999999e-06 2024-09-14T06:31:27.202 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:27.202 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:27.202 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:31:27.202 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.419811+0000", 2024-09-14T06:31:27.202 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047253310999999999 2024-09-14T06:31:27.202 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:27.202 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:27.203 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:31:27.203 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.419854+0000", 2024-09-14T06:31:27.203 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.3448000000000001e-05 2024-09-14T06:31:27.203 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:27.203 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:27.203 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:31:27.203 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.419909+0000", 2024-09-14T06:31:27.203 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.4549999999999998e-05 2024-09-14T06:31:27.203 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:27.203 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:31:27.203 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:27.203 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:27.203 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:27.203 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:53354 3.f 3:f71f51a7:::rbd_data.112dca2f6ae1.0000000000000015:17 [read 1400832~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:31:27.203 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:44.372563+0000", 2024-09-14T06:31:27.204 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 102.74417762, 2024-09-14T06:31:27.204 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047482233999999998, 2024-09-14T06:31:27.204 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:31:27.204 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:31:27.204 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:31:27.204 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:31:27.204 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:31:27.204 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:27.204 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 53354 2024-09-14T06:31:27.204 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:27.204 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:31:27.204 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:27.204 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:31:27.204 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.372563+0000", 2024-09-14T06:31:27.204 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:27.205 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:27.205 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:27.205 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:31:27.205 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.372563+0000", 2024-09-14T06:31:27.205 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:27.205 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:27.205 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:27.205 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:31:27.205 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.375009+0000", 2024-09-14T06:31:27.205 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.0024457680000000001 2024-09-14T06:31:27.205 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:27.205 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:27.205 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:31:27.205 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.375014+0000", 2024-09-14T06:31:27.206 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.0599999999999998e-06 2024-09-14T06:31:27.206 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:27.206 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:27.206 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:31:27.206 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.375026+0000", 2024-09-14T06:31:27.206 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.1783999999999999e-05 2024-09-14T06:31:27.206 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:27.206 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:27.206 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:31:27.206 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.375035+0000", 2024-09-14T06:31:27.206 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.7799999999999995e-06 2024-09-14T06:31:27.206 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:27.206 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:27.206 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:31:27.207 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.419942+0000", 2024-09-14T06:31:27.207 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.044906982999999998 2024-09-14T06:31:27.207 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:27.207 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:27.207 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:31:27.207 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.419989+0000", 2024-09-14T06:31:27.207 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.7104999999999997e-05 2024-09-14T06:31:27.207 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:27.207 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:27.207 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:31:27.207 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.420045+0000", 2024-09-14T06:31:27.207 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.5754000000000002e-05 2024-09-14T06:31:27.207 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:27.208 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:31:27.208 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:27.208 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:27.208 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:31:27.208 INFO:teuthology.orchestra.run.smithi138.stdout:} 2024-09-14T06:31:27.209 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-14T06:31:27.210 INFO:tasks.ceph.osd.0.smithi094.stderr:2024-09-14T06:31:27.209+0000 7f07c9333640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33636) UID: 0 2024-09-14T06:31:27.245 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-14T06:31:27.310 INFO:tasks.ceph.osd.3.smithi094.stderr:2024-09-14T06:31:27.308+0000 7feeeafe8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33655) UID: 0 2024-09-14T06:31:27.315 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-14T06:31:27.315 INFO:teuthology.orchestra.run.smithi094.stdout: "ops": [], 2024-09-14T06:31:27.315 INFO:teuthology.orchestra.run.smithi094.stdout: "num_ops": 0 2024-09-14T06:31:27.315 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-14T06:31:27.324 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-14T06:31:27.374 INFO:teuthology.orchestra.run.smithi094.stdout:osd.0: {} 2024-09-14T06:31:27.374 INFO:teuthology.orchestra.run.smithi094.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-14T06:31:27.392 INFO:teuthology.orchestra.run.smithi094.stdout:osd.1: {} 2024-09-14T06:31:27.393 INFO:teuthology.orchestra.run.smithi094.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-14T06:31:27.408 INFO:teuthology.orchestra.run.smithi094.stdout:osd.2: {} 2024-09-14T06:31:27.409 INFO:teuthology.orchestra.run.smithi094.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-14T06:31:27.410 INFO:tasks.ceph.osd.5.smithi138.stderr:2024-09-14T06:31:27.409+0000 7fe13746f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33618) UID: 0 2024-09-14T06:31:27.426 INFO:teuthology.orchestra.run.smithi094.stdout:osd.3: {} 2024-09-14T06:31:27.426 INFO:teuthology.orchestra.run.smithi094.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-14T06:31:27.443 INFO:teuthology.orchestra.run.smithi094.stdout:osd.4: {} 2024-09-14T06:31:27.443 INFO:teuthology.orchestra.run.smithi094.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-14T06:31:27.459 INFO:teuthology.orchestra.run.smithi094.stdout:osd.5: {} 2024-09-14T06:31:27.460 INFO:teuthology.orchestra.run.smithi094.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-14T06:31:27.470 INFO:teuthology.orchestra.run.smithi094.stdout:ERROR: (22) Invalid argument 2024-09-14T06:31:27.470 INFO:teuthology.orchestra.run.smithi094.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:31:27.476 INFO:teuthology.orchestra.run.smithi094.stdout:osd.6: {} 2024-09-14T06:31:27.476 INFO:teuthology.orchestra.run.smithi094.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-14T06:31:27.479 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-14T06:31:27.493 INFO:teuthology.orchestra.run.smithi094.stdout:osd.7: {} 2024-09-14T06:31:27.493 INFO:teuthology.orchestra.run.smithi094.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-14T06:31:27.511 INFO:tasks.ceph.osd.6.smithi138.stderr:2024-09-14T06:31:27.510+0000 7f8cd8849640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33620) UID: 0 2024-09-14T06:31:27.595 INFO:teuthology.orchestra.run.smithi094.stdout:ERROR: (22) Invalid argument 2024-09-14T06:31:27.595 INFO:teuthology.orchestra.run.smithi094.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:31:27.604 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-14T06:31:27.612 INFO:tasks.ceph.osd.6.smithi138.stderr:2024-09-14T06:31:27.610+0000 7f8cd8849640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33620) UID: 0 2024-09-14T06:31:27.620 INFO:teuthology.orchestra.run.smithi094.stderr:nodeep-scrub is set 2024-09-14T06:31:27.710 INFO:teuthology.orchestra.run.smithi094.stdout:ERROR: (22) Invalid argument 2024-09-14T06:31:27.710 INFO:teuthology.orchestra.run.smithi094.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:31:27.713 INFO:tasks.ceph.osd.2.smithi094.stderr:2024-09-14T06:31:27.712+0000 7faf3269b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33649) UID: 0 2024-09-14T06:31:27.719 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-14T06:31:27.813 INFO:tasks.ceph.osd.1.smithi094.stderr:2024-09-14T06:31:27.812+0000 7f328e47a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33641) UID: 0 2024-09-14T06:31:27.868 INFO:teuthology.orchestra.run.smithi094.stdout:ERROR: (22) Invalid argument 2024-09-14T06:31:27.868 INFO:teuthology.orchestra.run.smithi094.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:31:27.877 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-14T06:31:27.889 INFO:teuthology.orchestra.run.smithi094.stdout:4684 trunc from 0x8b0af9a to 0xeb4c654 2024-09-14T06:31:27.900 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-14T06:31:27.900 INFO:teuthology.orchestra.run.smithi094.stdout:4686 writesame 0x48e329b thru 0x48ea7d8 data_size 0x2(0x753e bytes) 2024-09-14T06:31:27.914 INFO:tasks.ceph.osd.4.smithi138.stderr:2024-09-14T06:31:27.912+0000 7f6c653c8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33615) UID: 0 2024-09-14T06:31:27.914 INFO:teuthology.orchestra.run.smithi094.stdout:4688 trunc from 0xeb4c654 to 0xa703e3b 2024-09-14T06:31:27.920 INFO:teuthology.orchestra.run.smithi094.stdout:4689 write 0xa241bc7 thru 0xa248fc5 (0x73ff bytes) 2024-09-14T06:31:27.924 INFO:teuthology.orchestra.run.smithi094.stdout:4690 punch from 0x65bcdc9 to 0x65c2ee8, (0x611f bytes) 2024-09-14T06:31:27.924 INFO:teuthology.orchestra.run.smithi094.stdout:4691 write 0x6f353f9 thru 0x6f43d4f (0xe957 bytes) 2024-09-14T06:31:27.926 INFO:teuthology.orchestra.run.smithi094.stdout:4693 punch from 0x4984f50 to 0x4985371, (0x421 bytes) 2024-09-14T06:31:27.931 INFO:teuthology.orchestra.run.smithi094.stdout:4694 trunc from 0xa703e3b to 0x5cae095 2024-09-14T06:31:27.987 INFO:teuthology.orchestra.run.smithi094.stdout:4696 punch from 0x43edc39 to 0x43f5a82, (0x7e49 bytes) 2024-09-14T06:31:27.999 INFO:teuthology.orchestra.run.smithi094.stdout:4699 write 0xb334d4c thru 0xb33ac0d (0x5ec2 bytes) 2024-09-14T06:31:28.002 INFO:teuthology.orchestra.run.smithi094.stdout:4700 read 0x8cdced9 thru 0x8ce32b8 (0x63e0 bytes) 2024-09-14T06:31:28.004 INFO:teuthology.orchestra.run.smithi094.stdout:ERROR: (22) Invalid argument 2024-09-14T06:31:28.004 INFO:teuthology.orchestra.run.smithi094.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:31:28.011 INFO:teuthology.orchestra.run.smithi094.stdout:4701 trunc from 0xb33ac0e to 0x452f098 2024-09-14T06:31:28.015 INFO:tasks.ceph.osd.2.smithi094.stderr:2024-09-14T06:31:28.014+0000 7faf3269b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33649) UID: 0 2024-09-14T06:31:28.017 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-14T06:31:28.026 INFO:teuthology.orchestra.run.smithi094.stdout:4702 punch from 0xc21de1 to 0xc22e40, (0x105f bytes) 2024-09-14T06:31:28.026 INFO:teuthology.orchestra.run.smithi094.stdout:4704 punch from 0x1fc44a9 to 0x1fccd14, (0x886b bytes) 2024-09-14T06:31:28.026 INFO:teuthology.orchestra.run.smithi094.stdout:4706 write 0x1dd0e06 thru 0x1dd9d5d (0x8f58 bytes) 2024-09-14T06:31:28.028 INFO:teuthology.orchestra.run.smithi094.stdout:4707 read 0x7b8f18 thru 0x7c6760 (0xd849 bytes) 2024-09-14T06:31:28.028 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-14T06:31:28.034 INFO:teuthology.orchestra.run.smithi094.stdout:4709 writesame 0x5677904 thru 0x568404a data_size 0x5f(0xc747 bytes) 2024-09-14T06:31:28.036 INFO:teuthology.orchestra.run.smithi094.stdout:4710 writesame 0x2dccef8 thru 0x2ddc669 data_size 0x13(0xf772 bytes) 2024-09-14T06:31:28.038 INFO:teuthology.orchestra.run.smithi094.stdout:4711 punch from 0x23911d7 to 0x239cedb, (0xbd04 bytes) 2024-09-14T06:31:28.055 INFO:teuthology.orchestra.run.smithi094.stdout:4712 write 0xe5e0d86 thru 0xe5ed0b9 (0xc334 bytes) 2024-09-14T06:31:28.058 INFO:teuthology.orchestra.run.smithi094.stdout:4713 punch from 0x201f45 to 0x210b76, (0xec31 bytes) 2024-09-14T06:31:28.058 INFO:teuthology.orchestra.run.smithi094.stdout:4715 read 0x4af00b6 thru 0x4afe87e (0xe7c9 bytes) 2024-09-14T06:31:28.058 INFO:teuthology.orchestra.run.smithi094.stdout:4716 punch from 0x8e1952b to 0x8e27089, (0xdb5e bytes) 2024-09-14T06:31:28.058 INFO:teuthology.orchestra.run.smithi094.stdout:4717 write 0x8c6d4c thru 0x8d12d9 (0xa58e bytes) 2024-09-14T06:31:28.077 INFO:teuthology.orchestra.run.smithi094.stdout:4718 trunc from 0xe5ed0ba to 0x31d7d 2024-09-14T06:31:28.105 INFO:teuthology.orchestra.run.smithi094.stdout:4719 read 0x2c506 thru 0x31d7c (0x5877 bytes) 2024-09-14T06:31:28.116 INFO:tasks.ceph.osd.5.smithi138.stderr:2024-09-14T06:31:28.115+0000 7fe13746f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33618) UID: 0 2024-09-14T06:31:28.124 INFO:teuthology.orchestra.run.smithi094.stdout:4720 writesame 0xae9fa51 thru 0xaea07cb data_size 0x7(0xd7b bytes) 2024-09-14T06:31:28.132 INFO:teuthology.orchestra.run.smithi094.stdout:4721 write 0xd2419af thru 0xd2460a0 (0x46f2 bytes) 2024-09-14T06:31:28.135 INFO:teuthology.orchestra.run.smithi094.stdout:4722 read 0x68c5143 thru 0x68c86bf (0x357d bytes) 2024-09-14T06:31:28.135 INFO:teuthology.orchestra.run.smithi094.stdout:4723 read 0x5e69aa3 thru 0x5e755ff (0xbb5d bytes) 2024-09-14T06:31:28.135 INFO:teuthology.orchestra.run.smithi094.stdout:4724 read 0xb319b48 thru 0xb324c37 (0xb0f0 bytes) 2024-09-14T06:31:28.139 INFO:teuthology.orchestra.run.smithi094.stdout:4726 trunc from 0xd2460a1 to 0x9d0e596 2024-09-14T06:31:28.149 INFO:teuthology.orchestra.run.smithi094.stdout:4728 read 0x9b12a6f thru 0x9b20af0 (0xe082 bytes) 2024-09-14T06:31:28.150 INFO:teuthology.orchestra.run.smithi094.stdout:ERROR: (22) Invalid argument 2024-09-14T06:31:28.150 INFO:teuthology.orchestra.run.smithi094.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:31:28.158 INFO:teuthology.orchestra.run.smithi094.stdout:4729 write 0xbe8e85e thru 0xbe93311 (0x4ab4 bytes) 2024-09-14T06:31:28.162 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-14T06:31:28.162 INFO:teuthology.orchestra.run.smithi094.stdout:4731 writesame 0x8481701 thru 0x8487e0d data_size 0x1f(0x670d bytes) 2024-09-14T06:31:28.165 INFO:teuthology.orchestra.run.smithi094.stdout:4732 writesame 0x14be5ce thru 0x14c5c56 data_size 0x69(0x7689 bytes) 2024-09-14T06:31:28.167 INFO:teuthology.orchestra.run.smithi094.stdout:4733 write 0x776a8c9 thru 0x7772245 (0x797d bytes) 2024-09-14T06:31:28.171 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-14T06:31:28.171 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-14T06:31:28.171 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-14T06:31:28.177 INFO:teuthology.orchestra.run.smithi094.stdout:4737 trunc from 0xbe93312 to 0x6da5782 2024-09-14T06:31:28.189 INFO:teuthology.orchestra.run.smithi094.stdout:4739 read 0x5450d16 thru 0x5457d44 (0x702f bytes) 2024-09-14T06:31:28.200 INFO:teuthology.orchestra.run.smithi094.stdout:4741 writesame 0xaab19be thru 0xaab3c60 data_size 0x1(0x22a3 bytes) 2024-09-14T06:31:28.204 INFO:teuthology.orchestra.run.smithi094.stdout:4743 trunc from 0xaab3c61 to 0x75374db 2024-09-14T06:31:28.213 INFO:teuthology.orchestra.run.smithi094.stdout:4744 trunc from 0x75374db to 0x9688c04 2024-09-14T06:31:28.216 INFO:tasks.ceph.osd.7.smithi138.stderr:2024-09-14T06:31:28.215+0000 7f3c6a21d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33623) UID: 0 2024-09-14T06:31:28.221 INFO:teuthology.orchestra.run.smithi094.stdout:4745 punch from 0x1027a7 to 0x10a682, (0x7edb bytes) 2024-09-14T06:31:28.221 INFO:teuthology.orchestra.run.smithi094.stdout:4746 read 0x1bd197a thru 0x1be0364 (0xe9eb bytes) 2024-09-14T06:31:28.221 INFO:teuthology.orchestra.run.smithi094.stdout:4747 read 0x786988 thru 0x786ce9 (0x362 bytes) 2024-09-14T06:31:28.221 INFO:teuthology.orchestra.run.smithi094.stdout:4748 read 0x31d6bb4 thru 0x31e168c (0xaad9 bytes) 2024-09-14T06:31:28.221 INFO:teuthology.orchestra.run.smithi094.stdout:4749 write 0x3ea4dd2 thru 0x3eabfa3 (0x71d2 bytes) 2024-09-14T06:31:28.221 INFO:teuthology.orchestra.run.smithi094.stdout:4750 write 0x93de4b6 thru 0x93df777 (0x12c2 bytes) 2024-09-14T06:31:28.222 INFO:teuthology.orchestra.run.smithi094.stdout:4751 trunc from 0x9688c04 to 0x9425dbd 2024-09-14T06:31:28.237 INFO:teuthology.orchestra.run.smithi094.stdout:4753 trunc from 0x9425dbd to 0x2d366b1 2024-09-14T06:31:28.254 INFO:teuthology.orchestra.run.smithi094.stdout:4754 writesame 0x5ec41eb thru 0x5ecc7c9 data_size 0x1(0x85df bytes) 2024-09-14T06:31:28.254 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-14T06:31:28.263 INFO:teuthology.orchestra.run.smithi094.stdout:4757 write 0x7e15269 thru 0x7e1e88e (0x9626 bytes) 2024-09-14T06:31:28.266 INFO:teuthology.orchestra.run.smithi094.stdout:4758 read 0x32404c7 thru 0x324098b (0x4c5 bytes) 2024-09-14T06:31:28.267 INFO:teuthology.orchestra.run.smithi094.stdout:4759 punch from 0x5eb548b to 0x5ebedf0, (0x9965 bytes) 2024-09-14T06:31:28.275 INFO:teuthology.orchestra.run.smithi094.stdout:4761 write 0xa452f5f thru 0xa45aa62 (0x7b04 bytes) 2024-09-14T06:31:28.278 INFO:teuthology.orchestra.run.smithi094.stdout:4762 read 0x679792a thru 0x67981a4 (0x87b bytes) 2024-09-14T06:31:28.286 INFO:teuthology.orchestra.run.smithi094.stdout:ERROR: (22) Invalid argument 2024-09-14T06:31:28.286 INFO:teuthology.orchestra.run.smithi094.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:31:28.287 INFO:teuthology.orchestra.run.smithi094.stdout:4764 trunc from 0xa45aa63 to 0x34827f2 2024-09-14T06:31:28.296 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-14T06:31:28.314 INFO:teuthology.orchestra.run.smithi094.stdout:4765 write 0xb655ad5 thru 0xb65befb (0x6427 bytes) 2024-09-14T06:31:28.321 INFO:teuthology.orchestra.run.smithi094.stdout:4766 trunc from 0xb65befc to 0x74a3f05 2024-09-14T06:31:28.322 INFO:tasks.ceph.osd.1.smithi094.stderr:2024-09-14T06:31:28.321+0000 7f328e47a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33641) UID: 0 2024-09-14T06:31:28.333 INFO:teuthology.orchestra.run.smithi094.stdout:4767 read 0x607ecd0 thru 0x6087ead (0x91de bytes) 2024-09-14T06:31:28.333 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-14T06:31:28.334 INFO:teuthology.orchestra.run.smithi094.stdout:4770 read 0x6937e29 thru 0x693c625 (0x47fd bytes) 2024-09-14T06:31:28.334 INFO:teuthology.orchestra.run.smithi094.stdout:4771 read 0x269bf03 thru 0x26a303c (0x713a bytes) 2024-09-14T06:31:28.341 INFO:teuthology.orchestra.run.smithi094.stdout:4772 trunc from 0x74a3f05 to 0xcef8ecc 2024-09-14T06:31:28.347 INFO:teuthology.orchestra.run.smithi094.stdout:4773 trunc from 0xcef8ecc to 0xd976f01 2024-09-14T06:31:28.354 INFO:teuthology.orchestra.run.smithi094.stdout:4774 write 0xd37e8b7 thru 0xd3803e9 (0x1b33 bytes) 2024-09-14T06:31:28.358 INFO:teuthology.orchestra.run.smithi094.stdout:4776 read 0xc96ddb4 thru 0xc970eec (0x3139 bytes) 2024-09-14T06:31:28.358 INFO:teuthology.orchestra.run.smithi094.stdout:4777 write 0x291839b thru 0x29229f5 (0xa65b bytes) 2024-09-14T06:31:28.359 INFO:teuthology.orchestra.run.smithi094.stdout:4779 writesame 0x733d67d thru 0x733da13 data_size 0x1(0x397 bytes) 2024-09-14T06:31:28.362 INFO:teuthology.orchestra.run.smithi094.stdout:4782 trunc from 0xd976f01 to 0xd281f3e 2024-09-14T06:31:28.371 INFO:teuthology.orchestra.run.smithi094.stdout:4783 trunc from 0xd281f3e to 0xc4205ad 2024-09-14T06:31:28.378 INFO:teuthology.orchestra.run.smithi094.stdout:4785 read 0xb3abe0e thru 0xb3ae30a (0x24fd bytes) 2024-09-14T06:31:28.378 INFO:teuthology.orchestra.run.smithi094.stdout:4786 punch from 0x807e999 to 0x8083a31, (0x5098 bytes) 2024-09-14T06:31:28.379 INFO:teuthology.orchestra.run.smithi094.stdout:4787 write 0x5bd227b thru 0x5bdf030 (0xcdb6 bytes) 2024-09-14T06:31:28.382 INFO:teuthology.orchestra.run.smithi094.stdout:4788 punch from 0xb0e0875 to 0xb0e916c, (0x88f7 bytes) 2024-09-14T06:31:28.382 INFO:teuthology.orchestra.run.smithi094.stdout:4789 read 0x749403f thru 0x749775c (0x371e bytes) 2024-09-14T06:31:28.390 INFO:teuthology.orchestra.run.smithi094.stdout:4790 trunc from 0xc4205ad to 0x4efd043 2024-09-14T06:31:28.403 INFO:teuthology.orchestra.run.smithi094.stdout:4791 read 0x3a58135 thru 0x3a6459c (0xc468 bytes) 2024-09-14T06:31:28.408 INFO:teuthology.orchestra.run.smithi094.stdout:4792 trunc from 0x4efd043 to 0x6fcab7 2024-09-14T06:31:28.416 INFO:teuthology.orchestra.run.smithi094.stdout:4793 punch from 0x33216a to 0x33b0ce, (0x8f64 bytes) 2024-09-14T06:31:28.416 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-14T06:31:28.418 INFO:tasks.ceph.osd.7.smithi138.stderr:2024-09-14T06:31:28.417+0000 7f3c6a21d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33623) UID: 0 2024-09-14T06:31:28.423 INFO:teuthology.orchestra.run.smithi094.stdout:4795 writesame 0x1c09ffc thru 0x1c0ea53 data_size 0xd(0x4a58 bytes) 2024-09-14T06:31:28.423 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-14T06:31:28.423 INFO:teuthology.orchestra.run.smithi094.stdout:4801 trunc from 0x1c0ea54 to 0x1c0e9ef 2024-09-14T06:31:28.433 INFO:teuthology.orchestra.run.smithi094.stdout:ERROR: (22) Invalid argument 2024-09-14T06:31:28.433 INFO:teuthology.orchestra.run.smithi094.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:31:28.435 INFO:teuthology.orchestra.run.smithi094.stdout:4802 write 0x252acba thru 0x2534b3e (0x9e85 bytes) 2024-09-14T06:31:28.438 INFO:teuthology.orchestra.run.smithi094.stdout:4803 punch from 0x144e8fc to 0x145160e, (0x2d12 bytes) 2024-09-14T06:31:28.443 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-14T06:31:28.443 INFO:teuthology.orchestra.run.smithi094.stdout:4805 trunc from 0x2534b3f to 0x61d3474 2024-09-14T06:31:28.449 INFO:teuthology.orchestra.run.smithi094.stdout:4806 punch from 0xd1d09a to 0xd1d109, (0x6f bytes) 2024-09-14T06:31:28.449 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-14T06:31:28.449 INFO:teuthology.orchestra.run.smithi094.stdout:4809 punch from 0x2af27cc to 0x2b012b3, (0xeae7 bytes) 2024-09-14T06:31:28.449 INFO:teuthology.orchestra.run.smithi094.stdout:4811 punch from 0x50d14f2 to 0x50e087e, (0xf38c bytes) 2024-09-14T06:31:28.449 INFO:teuthology.orchestra.run.smithi094.stdout:4812 write 0x424bc1d thru 0x4252d0b (0x70ef bytes) 2024-09-14T06:31:28.450 INFO:teuthology.orchestra.run.smithi094.stdout:4813 punch from 0x5a9ad8b to 0x5a9ee70, (0x40e5 bytes) 2024-09-14T06:31:28.451 INFO:teuthology.orchestra.run.smithi094.stdout:4815 read 0x37ba5f8 thru 0x37bcbaa (0x25b3 bytes) 2024-09-14T06:31:28.460 INFO:teuthology.orchestra.run.smithi094.stdout:4816 writesame 0x9c28929 thru 0x9c2edd1 data_size 0x1(0x64a9 bytes) 2024-09-14T06:31:28.462 INFO:teuthology.orchestra.run.smithi094.stdout:4817 trunc from 0x9c2edd2 to 0x90d8fe6 2024-09-14T06:31:28.470 INFO:teuthology.orchestra.run.smithi094.stdout:4818 read 0x66f8637 thru 0x6702519 (0x9ee3 bytes) 2024-09-14T06:31:28.470 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-14T06:31:28.470 INFO:teuthology.orchestra.run.smithi094.stdout:4824 read 0x77d6cd9 thru 0x77dc7bb (0x5ae3 bytes) 2024-09-14T06:31:28.470 INFO:teuthology.orchestra.run.smithi094.stdout:4826 writesame 0x8f3df72 thru 0x8f422aa data_size 0x1(0x4339 bytes) 2024-09-14T06:31:28.481 INFO:teuthology.orchestra.run.smithi094.stdout:4827 write 0x24f579f thru 0x24f7d04 (0x2566 bytes) 2024-09-14T06:31:28.483 INFO:teuthology.orchestra.run.smithi094.stdout:4828 writesame 0x59a00b8 thru 0x59aa647 data_size 0x18(0xa590 bytes) 2024-09-14T06:31:28.487 INFO:teuthology.orchestra.run.smithi094.stdout:4829 read 0x7cc580c thru 0x7ccdf1f (0x8714 bytes) 2024-09-14T06:31:28.488 INFO:teuthology.orchestra.run.smithi094.stdout:4830 punch from 0x1398387 to 0x139e8f1, (0x656a bytes) 2024-09-14T06:31:28.497 INFO:teuthology.orchestra.run.smithi094.stdout:4832 trunc from 0x90d8fe6 to 0x1c8817d 2024-09-14T06:31:28.519 INFO:tasks.ceph.osd.5.smithi138.stderr:2024-09-14T06:31:28.518+0000 7fe13746f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33618) UID: 0 2024-09-14T06:31:28.520 INFO:teuthology.orchestra.run.smithi094.stdout:4833 write 0x6bf9597 thru 0x6c03c6e (0xa6d8 bytes) 2024-09-14T06:31:28.537 INFO:teuthology.orchestra.run.smithi094.stdout:4834 writesame 0xe6c7913 thru 0xe6d174d data_size 0x1(0x9e3b bytes) 2024-09-14T06:31:28.537 INFO:teuthology.orchestra.run.smithi094.stdout:4837 writesame 0xe0cdc2 thru 0xe13695 data_size 0x2(0x68d4 bytes) 2024-09-14T06:31:28.544 INFO:teuthology.orchestra.run.smithi094.stdout:4838 read 0x5dfdbb7 thru 0x5e08229 (0xa673 bytes) 2024-09-14T06:31:28.544 INFO:teuthology.orchestra.run.smithi094.stdout:4839 read 0x6e790b2 thru 0x6e88f1b (0xfe6a bytes) 2024-09-14T06:31:28.553 INFO:teuthology.orchestra.run.smithi094.stdout:4840 trunc from 0xe6d174e to 0x6d2b455 2024-09-14T06:31:28.561 INFO:teuthology.orchestra.run.smithi094.stdout:4841 read 0x17ff0f1 thru 0x1801b5d (0x2a6d bytes) 2024-09-14T06:31:28.561 INFO:teuthology.orchestra.run.smithi094.stdout:4843 writesame 0x5527b3e thru 0x553747f data_size 0x2d(0xf942 bytes) 2024-09-14T06:31:28.563 INFO:teuthology.orchestra.run.smithi094.stdout:4844 read 0x9b4c76 thru 0x9c1583 (0xc90e bytes) 2024-09-14T06:31:28.563 INFO:teuthology.orchestra.run.smithi094.stdout:4845 read 0x29ffa9b thru 0x2a0afff (0xb565 bytes) 2024-09-14T06:31:28.564 INFO:teuthology.orchestra.run.smithi094.stdout:4846 punch from 0x6645d01 to 0x664b200, (0x54ff bytes) 2024-09-14T06:31:28.567 INFO:teuthology.orchestra.run.smithi094.stdout:ERROR: (22) Invalid argument 2024-09-14T06:31:28.567 INFO:teuthology.orchestra.run.smithi094.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:31:28.578 INFO:teuthology.orchestra.run.smithi094.stdout:4847 write 0xd6bd203 thru 0xd6c7e9c (0xac9a bytes) 2024-09-14T06:31:28.579 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-14T06:31:28.581 INFO:teuthology.orchestra.run.smithi094.stdout:4850 punch from 0x1d30596 to 0x1d3e7ec, (0xe256 bytes) 2024-09-14T06:31:28.581 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-14T06:31:28.581 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-14T06:31:28.581 INFO:teuthology.orchestra.run.smithi094.stdout:4858 read 0xbddef2d thru 0xbde2f86 (0x405a bytes) 2024-09-14T06:31:28.582 INFO:teuthology.orchestra.run.smithi094.stdout:4860 read 0xbe7fa7c thru 0xbe87895 (0x7e1a bytes) 2024-09-14T06:31:28.582 INFO:teuthology.orchestra.run.smithi094.stdout:4861 writesame 0xc2092e7 thru 0xc218ba8 data_size 0x11(0xf8c2 bytes) 2024-09-14T06:31:28.585 INFO:teuthology.orchestra.run.smithi094.stdout:4862 write 0xd52bec1 thru 0xd5327ce (0x690e bytes) 2024-09-14T06:31:28.588 INFO:teuthology.orchestra.run.smithi094.stdout:4863 punch from 0xa4d067d to 0xa4d7b4c, (0x74cf bytes) 2024-09-14T06:31:28.602 INFO:teuthology.orchestra.run.smithi094.stdout:4864 trunc from 0xd6c7e9d to 0x1754b8d 2024-09-14T06:31:28.620 INFO:tasks.ceph.osd.2.smithi094.stderr:2024-09-14T06:31:28.619+0000 7faf3269b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33649) UID: 0 2024-09-14T06:31:28.635 INFO:teuthology.orchestra.run.smithi094.stdout:4868 write 0xa185df5 thru 0xa192016 (0xc222 bytes) 2024-09-14T06:31:28.638 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-14T06:31:28.647 INFO:teuthology.orchestra.run.smithi094.stdout:4870 writesame 0xcd9ff51 thru 0xcda6e06 data_size 0x25(0x6eb6 bytes) 2024-09-14T06:31:28.647 INFO:teuthology.orchestra.run.smithi094.stdout:4871 writesame 0x5aad352 thru 0x5ab7c29 data_size 0xc(0xa8d8 bytes) 2024-09-14T06:31:28.651 INFO:teuthology.orchestra.run.smithi094.stdout:4872 read 0xa308236 thru 0xa30d0bd (0x4e88 bytes) 2024-09-14T06:31:28.651 INFO:teuthology.orchestra.run.smithi094.stdout:4873 writesame 0x6204d04 thru 0x6206109 data_size 0xb(0x1406 bytes) 2024-09-14T06:31:28.654 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-14T06:31:28.654 INFO:teuthology.orchestra.run.smithi094.stdout:4876 read 0x1306448 thru 0x1315aac (0xf665 bytes) 2024-09-14T06:31:28.654 INFO:teuthology.orchestra.run.smithi094.stdout:4877 write 0xc90821f thru 0xc90c0be (0x3ea0 bytes) 2024-09-14T06:31:28.658 INFO:teuthology.orchestra.run.smithi094.stdout:4879 read 0x9ee6758 thru 0x9ef2e28 (0xc6d1 bytes) 2024-09-14T06:31:28.658 INFO:teuthology.orchestra.run.smithi094.stdout:4880 write 0x3538c75 thru 0x3541414 (0x87a0 bytes) 2024-09-14T06:31:28.659 INFO:teuthology.orchestra.run.smithi094.stdout:4881 read 0x3743853 thru 0x374dd2d (0xa4db bytes) 2024-09-14T06:31:28.659 INFO:teuthology.orchestra.run.smithi094.stdout:4884 read 0x25e75b7 thru 0x25e7e44 (0x88e bytes) 2024-09-14T06:31:28.660 INFO:teuthology.orchestra.run.smithi094.stdout:4885 read 0x3db4f09 thru 0x3dc1802 (0xc8fa bytes) 2024-09-14T06:31:28.660 INFO:teuthology.orchestra.run.smithi094.stdout:4886 punch from 0x5979f04 to 0x5988994, (0xea90 bytes) 2024-09-14T06:31:28.660 INFO:teuthology.orchestra.run.smithi094.stdout:4887 write 0x76b4270 thru 0x76c3595 (0xf326 bytes) 2024-09-14T06:31:28.663 INFO:teuthology.orchestra.run.smithi094.stdout:4888 read 0x3a4aee1 thru 0x3a4c2d8 (0x13f8 bytes) 2024-09-14T06:31:28.664 INFO:teuthology.orchestra.run.smithi094.stdout:4890 writesame 0xbb9c845 thru 0xbba7a88 data_size 0x6(0xb244 bytes) 2024-09-14T06:31:28.672 INFO:teuthology.orchestra.run.smithi094.stdout:4892 trunc from 0xcda6e07 to 0xaa3567a 2024-09-14T06:31:28.679 INFO:teuthology.orchestra.run.smithi094.stdout:4893 punch from 0x32ac73a to 0x32b4a16, (0x82dc bytes) 2024-09-14T06:31:28.679 INFO:teuthology.orchestra.run.smithi094.stdout:4895 read 0x1e0cefd thru 0x1e1b76b (0xe86f bytes) 2024-09-14T06:31:28.679 INFO:teuthology.orchestra.run.smithi094.stdout:4900 punch from 0x2208a8c to 0x220e245, (0x57b9 bytes) 2024-09-14T06:31:28.684 INFO:teuthology.orchestra.run.smithi094.stdout:4902 trunc from 0xaa3567a to 0x632f251 2024-09-14T06:31:28.697 INFO:teuthology.orchestra.run.smithi094.stdout:4903 write 0x3d8151d thru 0x3d87313 (0x5df7 bytes) 2024-09-14T06:31:28.705 INFO:teuthology.orchestra.run.smithi094.stdout:ERROR: (22) Invalid argument 2024-09-14T06:31:28.705 INFO:teuthology.orchestra.run.smithi094.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:31:28.706 INFO:teuthology.orchestra.run.smithi094.stdout:4904 writesame 0x8658a54 thru 0x8661a5f data_size 0x2a(0x900c bytes) 2024-09-14T06:31:28.708 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-14T06:31:28.717 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-14T06:31:28.720 INFO:teuthology.orchestra.run.smithi094.stdout:4906 write 0xdfa696f thru 0xdfa8cd3 (0x2365 bytes) 2024-09-14T06:31:28.721 INFO:tasks.ceph.osd.3.smithi094.stderr:2024-09-14T06:31:28.719+0000 7feeeafe8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33655) UID: 0 2024-09-14T06:31:28.723 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-14T06:31:28.724 INFO:teuthology.orchestra.run.smithi094.stdout:4908 writesame 0x5072327 thru 0x5074e6d data_size 0x3(0x2b47 bytes) 2024-09-14T06:31:28.726 INFO:teuthology.orchestra.run.smithi094.stdout:4909 read 0x98b1662 thru 0x98b240c (0xdab bytes) 2024-09-14T06:31:28.726 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-14T06:31:28.730 INFO:teuthology.orchestra.run.smithi094.stdout:4912 trunc from 0xdfa8cd4 to 0xa896e7b 2024-09-14T06:31:28.745 INFO:teuthology.orchestra.run.smithi094.stdout:4913 write 0xc43aa13 thru 0xc4491e8 (0xe7d6 bytes) 2024-09-14T06:31:28.748 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-14T06:31:28.748 INFO:teuthology.orchestra.run.smithi094.stdout:4919 read 0x7497e86 thru 0x749fa90 (0x7c0b bytes) 2024-09-14T06:31:28.750 INFO:teuthology.orchestra.run.smithi094.stdout:4920 trunc from 0xc4491e9 to 0xb5b0a79 2024-09-14T06:31:28.757 INFO:teuthology.orchestra.run.smithi094.stdout:4921 punch from 0x67af6ae to 0x67bd6bd, (0xe00f bytes) 2024-09-14T06:31:28.757 INFO:teuthology.orchestra.run.smithi094.stdout:4922 writesame 0x7e824a0 thru 0x7e8ebae data_size 0x1(0xc70f bytes) 2024-09-14T06:31:28.782 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-14T06:31:28.782 INFO:teuthology.orchestra.run.smithi094.stdout:4925 write 0x4287729 thru 0x4294aae (0xd386 bytes) 2024-09-14T06:31:28.783 INFO:teuthology.orchestra.run.smithi094.stdout:4926 read 0x2131649 thru 0x213e3bc (0xcd74 bytes) 2024-09-14T06:31:28.784 INFO:teuthology.orchestra.run.smithi094.stdout:4927 read 0x8b8644d thru 0x8b95494 (0xf048 bytes) 2024-09-14T06:31:28.785 INFO:teuthology.orchestra.run.smithi094.stdout:4929 read 0x1a5c9f8 thru 0x1a63386 (0x698f bytes) 2024-09-14T06:31:28.785 INFO:teuthology.orchestra.run.smithi094.stdout:4930 writesame 0x72700f7 thru 0x7270b33 data_size 0x1(0xa3d bytes) 2024-09-14T06:31:28.793 INFO:teuthology.orchestra.run.smithi094.stdout:4931 write 0xbdb6c31 thru 0xbdc5dca (0xf19a bytes) 2024-09-14T06:31:28.796 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-14T06:31:28.796 INFO:teuthology.orchestra.run.smithi094.stdout:4934 write 0x1ad0ed2 thru 0x1ad61ad (0x52dc bytes) 2024-09-14T06:31:28.801 INFO:teuthology.orchestra.run.smithi094.stdout:4936 trunc from 0xbdc5dcb to 0x8ed4fdf 2024-09-14T06:31:28.813 INFO:teuthology.orchestra.run.smithi094.stdout:4938 read 0x6766760 thru 0x6773b0e (0xd3af bytes) 2024-09-14T06:31:28.814 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-14T06:31:28.821 INFO:tasks.ceph.osd.3.smithi094.stderr:2024-09-14T06:31:28.820+0000 7feeeafe8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33655) UID: 0 2024-09-14T06:31:28.824 INFO:teuthology.orchestra.run.smithi094.stdout:4940 trunc from 0x8ed4fdf to 0xa16224 2024-09-14T06:31:28.836 INFO:teuthology.orchestra.run.smithi094.stdout:4942 read 0x44f777 thru 0x45eced (0xf577 bytes) 2024-09-14T06:31:28.837 INFO:teuthology.orchestra.run.smithi094.stdout:4945 punch from 0x4b4a7b to 0x4c2aef, (0xe074 bytes) 2024-09-14T06:31:28.837 INFO:teuthology.orchestra.run.smithi094.stdout:4946 read 0x81f390 thru 0x82c0e3 (0xcd54 bytes) 2024-09-14T06:31:28.840 INFO:teuthology.orchestra.run.smithi094.stdout:ERROR: (22) Invalid argument 2024-09-14T06:31:28.840 INFO:teuthology.orchestra.run.smithi094.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:31:28.846 INFO:teuthology.orchestra.run.smithi094.stdout:4947 write 0x6c0088c thru 0x6c02339 (0x1aae bytes) 2024-09-14T06:31:28.848 INFO:teuthology.orchestra.run.smithi094.stdout:4948 read 0x458964e thru 0x4594814 (0xb1c7 bytes) 2024-09-14T06:31:28.851 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-14T06:31:28.851 INFO:teuthology.orchestra.run.smithi094.stdout:4949 trunc from 0x6c0233a to 0x823ae5a 2024-09-14T06:31:28.859 INFO:teuthology.orchestra.run.smithi094.stdout:4951 trunc from 0x823ae5a to 0xba7d8f5 2024-09-14T06:31:28.864 INFO:teuthology.orchestra.run.smithi094.stdout:4952 punch from 0x3831ee4 to 0x3840e0e, (0xef2a bytes) 2024-09-14T06:31:28.864 INFO:teuthology.orchestra.run.smithi094.stdout:4953 read 0x4831072 thru 0x483bf58 (0xaee7 bytes) 2024-09-14T06:31:28.865 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-14T06:31:28.865 INFO:teuthology.orchestra.run.smithi094.stdout:4956 punch from 0x35f1fd7 to 0x35f732f, (0x5358 bytes) 2024-09-14T06:31:28.865 INFO:teuthology.orchestra.run.smithi094.stdout:4957 punch from 0xb57c74b to 0xb58b6f1, (0xefa6 bytes) 2024-09-14T06:31:28.868 INFO:teuthology.orchestra.run.smithi094.stdout:4958 trunc from 0xba7d8f5 to 0x872b9bd 2024-09-14T06:31:28.877 INFO:teuthology.orchestra.run.smithi094.stdout:4960 punch from 0x2ad79ca to 0x2ade9ab, (0x6fe1 bytes) 2024-09-14T06:31:28.877 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-14T06:31:28.886 INFO:teuthology.orchestra.run.smithi094.stdout:4962 trunc from 0x872b9bd to 0xf1dac1 2024-09-14T06:31:28.893 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-14T06:31:28.896 INFO:teuthology.orchestra.run.smithi094.stdout:4964 trunc from 0xf1dac1 to 0x340ace3 2024-09-14T06:31:28.913 INFO:teuthology.orchestra.run.smithi094.stdout:4965 write 0x77ea2ac thru 0x77f2b18 (0x886d bytes) 2024-09-14T06:31:28.914 INFO:teuthology.orchestra.run.smithi094.stdout:4967 write 0x2a5d675 thru 0x2a635b9 (0x5f45 bytes) 2024-09-14T06:31:28.921 INFO:teuthology.orchestra.run.smithi094.stdout:4968 trunc from 0x77f2b19 to 0x3482ffb 2024-09-14T06:31:28.922 INFO:tasks.ceph.osd.4.smithi138.stderr:2024-09-14T06:31:28.921+0000 7f6c653c8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33615) UID: 0 2024-09-14T06:31:28.934 INFO:teuthology.orchestra.run.smithi094.stdout:4969 trunc from 0x3482ffb to 0x7fabda6 2024-09-14T06:31:28.939 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-14T06:31:28.952 INFO:teuthology.orchestra.run.smithi094.stdout:4971 writesame 0xd59f2bd thru 0xd5aec51 data_size 0x1(0xf995 bytes) 2024-09-14T06:31:28.952 INFO:teuthology.orchestra.run.smithi094.stdout:4973 write 0x82a279 thru 0x82e2fe (0x4086 bytes) 2024-09-14T06:31:28.954 INFO:teuthology.orchestra.run.smithi094.stdout:4975 punch from 0x9ed14cb to 0x9ede9b2, (0xd4e7 bytes) 2024-09-14T06:31:28.954 INFO:teuthology.orchestra.run.smithi094.stdout:4976 read 0x870709b thru 0x870b98f (0x48f5 bytes) 2024-09-14T06:31:28.954 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-14T06:31:28.954 INFO:teuthology.orchestra.run.smithi094.stdout:4978 punch from 0x5b713ce to 0x5b7af0e, (0x9b40 bytes) 2024-09-14T06:31:28.959 INFO:teuthology.orchestra.run.smithi138.stdout:ERROR: (22) Invalid argument 2024-09-14T06:31:28.959 INFO:teuthology.orchestra.run.smithi138.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:31:28.968 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-14T06:31:28.969 INFO:teuthology.orchestra.run.smithi094.stdout:4979 trunc from 0xd5aec52 to 0x1a30086 2024-09-14T06:31:28.997 INFO:teuthology.orchestra.run.smithi094.stdout:4980 write 0xe2ffd94 thru 0xe309739 (0x99a6 bytes) 2024-09-14T06:31:29.000 INFO:teuthology.orchestra.run.smithi094.stdout:4981 write 0xa930560 thru 0xa939cef (0x9790 bytes) 2024-09-14T06:31:29.003 INFO:teuthology.orchestra.run.smithi094.stdout:4982 read 0x301c1a3 thru 0x30209f5 (0x4853 bytes) 2024-09-14T06:31:29.004 INFO:teuthology.orchestra.run.smithi094.stdout:4984 write 0x6c232de thru 0x6c26457 (0x317a bytes) 2024-09-14T06:31:29.005 INFO:teuthology.orchestra.run.smithi094.stdout:4985 punch from 0x889832d to 0x88a8183, (0xfe56 bytes) 2024-09-14T06:31:29.006 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-14T06:31:29.006 INFO:teuthology.orchestra.run.smithi094.stdout:4987 write 0xc42c416 thru 0xc43af26 (0xeb11 bytes) 2024-09-14T06:31:29.008 INFO:teuthology.orchestra.run.smithi094.stdout:4988 read 0x7697468 thru 0x76a5011 (0xdbaa bytes) 2024-09-14T06:31:29.009 INFO:teuthology.orchestra.run.smithi094.stdout:4989 read 0x9204818 thru 0x9209995 (0x517e bytes) 2024-09-14T06:31:29.009 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-14T06:31:29.010 INFO:teuthology.orchestra.run.smithi094.stdout:4992 read 0xadb2f80 thru 0xadc0a53 (0xdad4 bytes) 2024-09-14T06:31:29.010 INFO:teuthology.orchestra.run.smithi094.stdout:4993 read 0x9719e59 thru 0x9724093 (0xa23b bytes) 2024-09-14T06:31:29.010 INFO:teuthology.orchestra.run.smithi094.stdout:4995 read 0x4418f77 thru 0x44234b0 (0xa53a bytes) 2024-09-14T06:31:29.011 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-14T06:31:29.011 INFO:teuthology.orchestra.run.smithi094.stdout:4999 writesame 0x6a59b37 thru 0x6a65142 data_size 0x4(0xb60c bytes) 2024-09-14T06:31:29.017 INFO:teuthology.orchestra.run.smithi094.stdout:5000 read 0x6e19616 thru 0x6e23f79 (0xa964 bytes) 2024-09-14T06:31:29.017 INFO:teuthology.orchestra.run.smithi094.stdout:5002 read 0x3bbf36 thru 0x3be6da (0x27a5 bytes) 2024-09-14T06:31:29.018 INFO:teuthology.orchestra.run.smithi094.stdout:5004 punch from 0x1c6a89f to 0x1c6fa11, (0x5172 bytes) 2024-09-14T06:31:29.018 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-14T06:31:29.018 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-14T06:31:29.018 INFO:teuthology.orchestra.run.smithi094.stdout:5008 punch from 0x15b9ae to 0x163c60, (0x82b2 bytes) 2024-09-14T06:31:29.018 INFO:teuthology.orchestra.run.smithi094.stdout:5009 write 0x5483d47 thru 0x54905f8 (0xc8b2 bytes) 2024-09-14T06:31:29.019 INFO:teuthology.orchestra.run.smithi094.stdout:5011 writesame 0x89b4cbb thru 0x89b8b4b data_size 0x13(0x3e91 bytes) 2024-09-14T06:31:29.021 INFO:teuthology.orchestra.run.smithi094.stdout:5012 read 0x3fd1acc thru 0x3fdad88 (0x92bd bytes) 2024-09-14T06:31:29.021 INFO:teuthology.orchestra.run.smithi094.stdout:5014 write 0x1a35b35 thru 0x1a35e5a (0x326 bytes) 2024-09-14T06:31:29.023 INFO:teuthology.orchestra.run.smithi094.stdout:5015 read 0x8679c65 thru 0x867bd3b (0x20d7 bytes) 2024-09-14T06:31:29.023 INFO:teuthology.orchestra.run.smithi094.stdout:5016 write 0x9a26c55 thru 0x9a29c74 (0x3020 bytes) 2024-09-14T06:31:29.025 INFO:teuthology.orchestra.run.smithi094.stdout:5017 write 0x2465062 thru 0x246c48d (0x742c bytes) 2024-09-14T06:31:29.026 INFO:teuthology.orchestra.run.smithi094.stdout:5018 write 0xbd197a7 thru 0xbd1aac0 (0x131a bytes) 2024-09-14T06:31:29.028 INFO:teuthology.orchestra.run.smithi094.stdout:5019 read 0x6d59718 thru 0x6d66f06 (0xd7ef bytes) 2024-09-14T06:31:29.028 INFO:teuthology.orchestra.run.smithi094.stdout:5020 write 0x4644412 thru 0x464680a (0x23f9 bytes) 2024-09-14T06:31:29.030 INFO:teuthology.orchestra.run.smithi094.stdout:5021 writesame 0x80ba184 thru 0x80c517a data_size 0x1(0xaff7 bytes) 2024-09-14T06:31:29.051 INFO:teuthology.orchestra.run.smithi094.stdout:5022 read 0xc4b7351 thru 0xc4c5c84 (0xe934 bytes) 2024-09-14T06:31:29.052 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-14T06:31:29.052 INFO:teuthology.orchestra.run.smithi094.stdout:5024 punch from 0x11bd3f8 to 0x11c71b6, (0x9dbe bytes) 2024-09-14T06:31:29.052 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-14T06:31:29.052 INFO:teuthology.orchestra.run.smithi094.stdout:5026 punch from 0xe2aeb31 to 0xe2ba6bd, (0xbb8c bytes) 2024-09-14T06:31:29.053 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-14T06:31:29.053 INFO:teuthology.orchestra.run.smithi094.stdout:5029 write 0xca166e3 thru 0xca18242 (0x1b60 bytes) 2024-09-14T06:31:29.053 INFO:tasks.ceph.osd.4.smithi138.stderr:2024-09-14T06:31:29.052+0000 7f6c653c8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33615) UID: 0 2024-09-14T06:31:29.054 INFO:teuthology.orchestra.run.smithi094.stdout:5030 write 0x92189a4 thru 0x922009a (0x76f7 bytes) 2024-09-14T06:31:29.055 INFO:teuthology.orchestra.run.smithi094.stdout:5031 write 0xbc04bce thru 0xbc0fcbb (0xb0ee bytes) 2024-09-14T06:31:29.068 INFO:teuthology.orchestra.run.smithi094.stdout:5032 trunc from 0xe30973a to 0x6a30dce 2024-09-14T06:31:29.113 INFO:teuthology.orchestra.run.smithi138.stdout:ERROR: (22) Invalid argument 2024-09-14T06:31:29.113 INFO:teuthology.orchestra.run.smithi138.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:31:29.122 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-14T06:31:29.163 INFO:tasks.ceph.osd.6.smithi138.stderr:2024-09-14T06:31:29.162+0000 7f8cd8849640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33620) UID: 0 2024-09-14T06:31:29.224 INFO:tasks.ceph.osd.7.smithi138.stderr:2024-09-14T06:31:29.223+0000 7f3c6a21d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33623) UID: 0 2024-09-14T06:31:29.266 INFO:teuthology.orchestra.run.smithi138.stdout:ERROR: (22) Invalid argument 2024-09-14T06:31:29.266 INFO:teuthology.orchestra.run.smithi138.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:31:29.275 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-14T06:31:29.285 INFO:teuthology.orchestra.run.smithi094.stdout:5034 read 0x1380845 thru 0x138f81c (0xefd8 bytes) 2024-09-14T06:31:29.285 INFO:teuthology.orchestra.run.smithi094.stdout:5035 read 0x2f22003 thru 0x2f25bdb (0x3bd9 bytes) 2024-09-14T06:31:29.286 INFO:teuthology.orchestra.run.smithi094.stdout:5036 read 0x4c5170f thru 0x4c53e72 (0x2764 bytes) 2024-09-14T06:31:29.297 INFO:teuthology.orchestra.run.smithi094.stdout:5037 write 0xb5c7f9e thru 0xb5c97a4 (0x1807 bytes) 2024-09-14T06:31:29.301 INFO:teuthology.orchestra.run.smithi094.stdout:5040 punch from 0x8f1705e to 0x8f2028f, (0x9231 bytes) 2024-09-14T06:31:29.301 INFO:teuthology.orchestra.run.smithi094.stdout:5042 read 0x28e40c9 thru 0x28e5fd7 (0x1f0f bytes) 2024-09-14T06:31:29.301 INFO:teuthology.orchestra.run.smithi094.stdout:5044 write 0x93eae22 thru 0x93efaad (0x4c8c bytes) 2024-09-14T06:31:29.312 INFO:teuthology.orchestra.run.smithi094.stdout:5045 trunc from 0xb5c97a5 to 0x36cde45 2024-09-14T06:31:29.325 INFO:tasks.ceph.osd.2.smithi094.stderr:2024-09-14T06:31:29.324+0000 7faf3269b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33649) UID: 0 2024-09-14T06:31:29.325 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-14T06:31:29.334 INFO:teuthology.orchestra.run.smithi094.stdout:5047 write 0x5cbb306 thru 0x5cc6e2e (0xbb29 bytes) 2024-09-14T06:31:29.336 INFO:teuthology.orchestra.run.smithi094.stdout:5050 write 0x1f912cb thru 0x1f98451 (0x7187 bytes) 2024-09-14T06:31:29.353 INFO:teuthology.orchestra.run.smithi094.stdout:5051 writesame 0xe94ee71 thru 0xe9583c9 data_size 0x11(0x9559 bytes) 2024-09-14T06:31:29.353 INFO:teuthology.orchestra.run.smithi094.stdout:5053 read 0x23669d3 thru 0x2367594 (0xbc2 bytes) 2024-09-14T06:31:29.353 INFO:teuthology.orchestra.run.smithi094.stdout:5054 read 0x610edf6 thru 0x61135a6 (0x47b1 bytes) 2024-09-14T06:31:29.354 INFO:teuthology.orchestra.run.smithi094.stdout:5055 read 0x85d9e2c thru 0x85dac28 (0xdfd bytes) 2024-09-14T06:31:29.362 INFO:teuthology.orchestra.run.smithi094.stdout:5056 trunc from 0xe9583ca to 0x7b3e35f 2024-09-14T06:31:29.371 INFO:teuthology.orchestra.run.smithi094.stdout:5057 write 0x690e631 thru 0x691364c (0x501c bytes) 2024-09-14T06:31:29.373 INFO:teuthology.orchestra.run.smithi094.stdout:5058 write 0x9409ba thru 0x94b5d2 (0xac19 bytes) 2024-09-14T06:31:29.381 INFO:teuthology.orchestra.run.smithi094.stdout:5059 trunc from 0x7b3e35f to 0x250db6a 2024-09-14T06:31:29.391 INFO:teuthology.orchestra.run.smithi094.stdout:5061 read 0x24de4b1 thru 0x24e42f7 (0x5e47 bytes) 2024-09-14T06:31:29.391 INFO:teuthology.orchestra.run.smithi094.stdout:5062 read 0x1a8cb8d thru 0x1a93e3d (0x72b1 bytes) 2024-09-14T06:31:29.398 INFO:teuthology.orchestra.run.smithi094.stdout:5063 write 0x38c51e1 thru 0x38ce849 (0x9669 bytes) 2024-09-14T06:31:29.415 INFO:teuthology.orchestra.run.smithi094.stdout:5065 write 0xb7cd55d thru 0xb7dcdbb (0xf85f bytes) 2024-09-14T06:31:29.424 INFO:teuthology.orchestra.run.smithi138.stdout:ERROR: (22) Invalid argument 2024-09-14T06:31:29.425 INFO:teuthology.orchestra.run.smithi138.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:31:29.425 INFO:tasks.ceph.osd.7.smithi138.stderr:2024-09-14T06:31:29.424+0000 7f3c6a21d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33623) UID: 0 2024-09-14T06:31:29.426 INFO:teuthology.orchestra.run.smithi094.stdout:5066 trunc from 0xb7dcdbc to 0x4c15893 2024-09-14T06:31:29.434 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-14T06:31:29.442 INFO:teuthology.orchestra.run.smithi094.stdout:5067 write 0x13880d8 thru 0x13931b4 (0xb0dd bytes) 2024-09-14T06:31:29.444 INFO:teuthology.orchestra.run.smithi094.stdout:5068 writesame 0x4bd31b8 thru 0x4bdaff7 data_size 0x50(0x7e40 bytes) 2024-09-14T06:31:29.445 INFO:teuthology.orchestra.run.smithi094.stdout:5069 clone 39 order 18 su 8192 sc 12 2024-09-14T06:31:29.504 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-14T06:31:29.549 INFO:tasks.ceph.osd.1.smithi094.stderr:2024-09-14T06:31:29.548+0000 7f328e47a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33641) UID: 0 2024-09-14T06:31:29.578 INFO:teuthology.orchestra.run.smithi138.stdout:ERROR: (22) Invalid argument 2024-09-14T06:31:29.578 INFO:teuthology.orchestra.run.smithi138.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:31:29.587 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-14T06:31:29.624 INFO:teuthology.orchestra.run.smithi094.stdout:osd.0: {} 2024-09-14T06:31:29.625 INFO:teuthology.orchestra.run.smithi094.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-14T06:31:29.627 INFO:tasks.ceph.osd.1.smithi094.stderr:2024-09-14T06:31:29.626+0000 7f328e47a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33641) UID: 0 2024-09-14T06:31:29.636 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-14T06:31:29.646 INFO:teuthology.orchestra.run.smithi094.stdout:osd.1: {} 2024-09-14T06:31:29.646 INFO:teuthology.orchestra.run.smithi094.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-14T06:31:29.665 INFO:teuthology.orchestra.run.smithi094.stdout:osd.2: {} 2024-09-14T06:31:29.665 INFO:teuthology.orchestra.run.smithi094.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-14T06:31:29.683 INFO:teuthology.orchestra.run.smithi094.stdout:osd.3: {} 2024-09-14T06:31:29.684 INFO:teuthology.orchestra.run.smithi094.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-14T06:31:29.695 INFO:teuthology.orchestra.run.smithi138.stdout:ERROR: (22) Invalid argument 2024-09-14T06:31:29.695 INFO:teuthology.orchestra.run.smithi138.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:31:29.702 INFO:teuthology.orchestra.run.smithi094.stdout:osd.4: {} 2024-09-14T06:31:29.702 INFO:teuthology.orchestra.run.smithi094.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-14T06:31:29.704 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-14T06:31:29.718 INFO:teuthology.orchestra.run.smithi094.stdout:osd.5: {} 2024-09-14T06:31:29.719 INFO:teuthology.orchestra.run.smithi094.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-14T06:31:29.728 INFO:tasks.ceph.osd.3.smithi094.stderr:2024-09-14T06:31:29.727+0000 7feeeafe8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33655) UID: 0 2024-09-14T06:31:29.735 INFO:teuthology.orchestra.run.smithi094.stdout:osd.6: {} 2024-09-14T06:31:29.735 INFO:teuthology.orchestra.run.smithi094.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-14T06:31:29.756 INFO:teuthology.orchestra.run.smithi094.stdout:osd.7: {} 2024-09-14T06:31:29.756 INFO:teuthology.orchestra.run.smithi094.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-14T06:31:29.808 INFO:teuthology.orchestra.run.smithi138.stdout:{ 2024-09-14T06:31:29.808 INFO:teuthology.orchestra.run.smithi138.stdout: "ops": [], 2024-09-14T06:31:29.808 INFO:teuthology.orchestra.run.smithi138.stdout: "num_ops": 0 2024-09-14T06:31:29.808 INFO:teuthology.orchestra.run.smithi138.stdout:} 2024-09-14T06:31:29.816 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-14T06:31:29.860 INFO:tasks.ceph.osd.7.smithi138.stderr:2024-09-14T06:31:29.859+0000 7f3c6a21d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33623) UID: 0 2024-09-14T06:31:29.919 INFO:teuthology.orchestra.run.smithi138.stdout:{ 2024-09-14T06:31:29.919 INFO:teuthology.orchestra.run.smithi138.stdout: "ops": [], 2024-09-14T06:31:29.919 INFO:teuthology.orchestra.run.smithi138.stdout: "complaint_time": 30, 2024-09-14T06:31:29.919 INFO:teuthology.orchestra.run.smithi138.stdout: "num_blocked_ops": 0 2024-09-14T06:31:29.919 INFO:teuthology.orchestra.run.smithi138.stdout:} 2024-09-14T06:31:29.927 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-14T06:31:29.968 INFO:tasks.ceph.osd.7.smithi138.stderr:2024-09-14T06:31:29.967+0000 7f3c6a21d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33623) UID: 0 2024-09-14T06:31:30.030 INFO:tasks.ceph.osd.7.smithi138.stderr:2024-09-14T06:31:30.029+0000 7f3c6a21d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33623) UID: 0 2024-09-14T06:31:30.072 INFO:teuthology.orchestra.run.smithi138.stdout:{ 2024-09-14T06:31:30.072 INFO:teuthology.orchestra.run.smithi138.stdout: "size": 20, 2024-09-14T06:31:30.073 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 600, 2024-09-14T06:31:30.073 INFO:teuthology.orchestra.run.smithi138.stdout: "ops": [ 2024-09-14T06:31:30.073 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:30.073 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:98505 3.10 3.1bf56f90 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T06:31:30.073 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:30:07.045453+0000", 2024-09-14T06:31:30.073 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 83.025273122000002, 2024-09-14T06:31:30.073 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032144440000000003, 2024-09-14T06:31:30.073 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:31:30.073 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:31:30.073 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "reached pg", 2024-09-14T06:31:30.073 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:31:30.073 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:31:30.073 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:30.074 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 98505 2024-09-14T06:31:30.074 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:30.074 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:31:30.074 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:30.074 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:31:30.074 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045453+0000", 2024-09-14T06:31:30.074 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:30.074 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:30.074 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:30.074 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:31:30.074 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045453+0000", 2024-09-14T06:31:30.074 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:30.074 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:30.074 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:30.074 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:31:30.074 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045453+0000", 2024-09-14T06:31:30.075 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.36e-07 2024-09-14T06:31:30.075 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:30.075 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:30.075 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:31:30.075 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045454+0000", 2024-09-14T06:31:30.075 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.1299999999999998e-07 2024-09-14T06:31:30.075 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:30.075 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:30.075 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:31:30.075 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045457+0000", 2024-09-14T06:31:30.075 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.8559999999999998e-06 2024-09-14T06:31:30.075 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:30.075 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:30.075 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:31:30.075 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045460+0000", 2024-09-14T06:31:30.075 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.8750000000000002e-06 2024-09-14T06:31:30.076 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:30.076 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:30.076 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:31:30.076 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.077592+0000", 2024-09-14T06:31:30.076 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032131727999999998 2024-09-14T06:31:30.076 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:30.076 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:30.076 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:31:30.076 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.077597+0000", 2024-09-14T06:31:30.076 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.8319999999999996e-06 2024-09-14T06:31:30.076 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:30.076 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:31:30.076 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:30.077 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:30.077 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:30.077 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:98506 3.10 3.1bf56f90 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T06:31:30.077 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:30:07.045481+0000", 2024-09-14T06:31:30.077 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 83.025244866999998, 2024-09-14T06:31:30.077 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032142759999999999, 2024-09-14T06:31:30.077 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:31:30.077 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:31:30.077 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "reached pg", 2024-09-14T06:31:30.078 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:31:30.078 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:31:30.078 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:30.078 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 98506 2024-09-14T06:31:30.078 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:30.078 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:31:30.078 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:30.078 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:31:30.078 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045481+0000", 2024-09-14T06:31:30.078 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:30.078 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:30.078 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:30.078 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:31:30.078 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045481+0000", 2024-09-14T06:31:30.079 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:30.079 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:30.079 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:30.079 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:31:30.079 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045481+0000", 2024-09-14T06:31:30.079 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.8500000000000001e-07 2024-09-14T06:31:30.079 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:30.079 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:30.079 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:31:30.079 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045483+0000", 2024-09-14T06:31:30.079 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.8250000000000001e-06 2024-09-14T06:31:30.079 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:30.079 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:30.079 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:31:30.079 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045489+0000", 2024-09-14T06:31:30.080 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.5740000000000003e-06 2024-09-14T06:31:30.080 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:30.080 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:30.080 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:31:30.080 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045495+0000", 2024-09-14T06:31:30.080 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.7690000000000002e-06 2024-09-14T06:31:30.080 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:30.080 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:30.080 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:31:30.080 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.077619+0000", 2024-09-14T06:31:30.080 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032124621999999999 2024-09-14T06:31:30.080 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:30.080 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:30.081 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:31:30.081 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.077624+0000", 2024-09-14T06:31:30.081 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.3850000000000004e-06 2024-09-14T06:31:30.081 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:30.081 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:31:30.081 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:30.081 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:30.081 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:30.081 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:98507 3.10 3.1bf56f90 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T06:31:30.081 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:30:07.045502+0000", 2024-09-14T06:31:30.081 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 83.025223445999998, 2024-09-14T06:31:30.081 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032147219999999997, 2024-09-14T06:31:30.082 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:31:30.082 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:31:30.082 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "reached pg", 2024-09-14T06:31:30.082 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:31:30.082 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:31:30.082 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:30.082 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 98507 2024-09-14T06:31:30.082 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:30.082 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:31:30.082 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:30.082 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:31:30.082 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045502+0000", 2024-09-14T06:31:30.082 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:30.082 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:30.083 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:30.083 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:31:30.083 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045502+0000", 2024-09-14T06:31:30.083 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:30.083 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:30.083 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:30.083 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:31:30.083 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045503+0000", 2024-09-14T06:31:30.083 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.4700000000000002e-07 2024-09-14T06:31:30.083 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:30.083 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:30.083 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:31:30.083 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045509+0000", 2024-09-14T06:31:30.084 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.7329999999999997e-06 2024-09-14T06:31:30.084 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:30.084 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:30.084 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:31:30.084 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045515+0000", 2024-09-14T06:31:30.084 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.8030000000000002e-06 2024-09-14T06:31:30.084 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:30.084 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:30.084 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:31:30.084 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045522+0000", 2024-09-14T06:31:30.084 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.0369999999999997e-06 2024-09-14T06:31:30.084 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:30.084 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:30.085 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:31:30.085 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.077641+0000", 2024-09-14T06:31:30.085 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032119091000000002 2024-09-14T06:31:30.085 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:30.085 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:30.085 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:31:30.085 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.077650+0000", 2024-09-14T06:31:30.085 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.2090000000000005e-06 2024-09-14T06:31:30.085 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:30.085 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:31:30.085 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:30.085 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:30.085 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:30.086 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:98508 3.10 3.1bf56f90 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T06:31:30.086 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:30:07.045530+0000", 2024-09-14T06:31:30.086 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 83.025195828999998, 2024-09-14T06:31:30.086 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032198773999999999, 2024-09-14T06:31:30.086 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:31:30.086 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:31:30.086 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "reached pg", 2024-09-14T06:31:30.086 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:31:30.086 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:31:30.086 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:30.086 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 98508 2024-09-14T06:31:30.086 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:30.086 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:31:30.087 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:30.087 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:31:30.087 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045530+0000", 2024-09-14T06:31:30.087 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:30.087 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:30.087 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:30.087 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:31:30.087 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045530+0000", 2024-09-14T06:31:30.087 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:30.087 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:30.087 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:30.087 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:31:30.087 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045530+0000", 2024-09-14T06:31:30.088 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.3099999999999998e-07 2024-09-14T06:31:30.088 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:30.088 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:30.088 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:31:30.088 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045539+0000", 2024-09-14T06:31:30.088 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.4780000000000008e-06 2024-09-14T06:31:30.088 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:30.088 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:30.088 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:31:30.088 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045546+0000", 2024-09-14T06:31:30.088 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.3259999999999998e-06 2024-09-14T06:31:30.088 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:30.088 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:30.089 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:31:30.089 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045551+0000", 2024-09-14T06:31:30.089 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.9960000000000001e-06 2024-09-14T06:31:30.089 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:30.089 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:30.089 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:31:30.089 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.077716+0000", 2024-09-14T06:31:30.089 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032165081999999998 2024-09-14T06:31:30.089 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:30.089 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:30.089 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:31:30.089 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.077729+0000", 2024-09-14T06:31:30.090 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.2461e-05 2024-09-14T06:31:30.090 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:30.090 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:31:30.090 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:30.090 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:30.090 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:30.090 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:98509 3.10 3.1bf56f90 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T06:31:30.090 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:30:07.045558+0000", 2024-09-14T06:31:30.090 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 83.025167628000005, 2024-09-14T06:31:30.090 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032242198, 2024-09-14T06:31:30.090 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:31:30.090 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:31:30.090 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "reached pg", 2024-09-14T06:31:30.091 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:31:30.091 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:31:30.091 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:30.091 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 98509 2024-09-14T06:31:30.091 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:30.091 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:31:30.091 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:30.091 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:31:30.091 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045558+0000", 2024-09-14T06:31:30.091 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:30.091 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:30.091 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:30.092 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:31:30.092 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045558+0000", 2024-09-14T06:31:30.092 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:30.092 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:30.092 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:30.092 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:31:30.092 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045559+0000", 2024-09-14T06:31:30.092 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.27e-07 2024-09-14T06:31:30.092 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:30.092 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:30.092 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:31:30.092 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045560+0000", 2024-09-14T06:31:30.092 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.6959999999999999e-06 2024-09-14T06:31:30.093 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:30.093 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:30.093 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:31:30.093 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045567+0000", 2024-09-14T06:31:30.093 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.6900000000000003e-06 2024-09-14T06:31:30.093 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:30.093 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:30.093 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:31:30.093 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045571+0000", 2024-09-14T06:31:30.093 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.5750000000000002e-06 2024-09-14T06:31:30.093 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:30.093 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:30.093 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:31:30.093 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.077789+0000", 2024-09-14T06:31:30.094 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032217186000000002 2024-09-14T06:31:30.094 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:30.094 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:30.094 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:31:30.094 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.077800+0000", 2024-09-14T06:31:30.094 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.1623999999999999e-05 2024-09-14T06:31:30.094 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:30.094 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:31:30.094 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:30.094 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:30.094 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:30.094 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:98510 3.10 3.1bf56f90 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T06:31:30.094 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:30:07.045580+0000", 2024-09-14T06:31:30.094 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 83.025145813999998, 2024-09-14T06:31:30.094 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032299845000000001, 2024-09-14T06:31:30.095 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:31:30.095 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:31:30.095 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "reached pg", 2024-09-14T06:31:30.095 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:31:30.095 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:31:30.095 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:30.095 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 98510 2024-09-14T06:31:30.095 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:30.095 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:31:30.095 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:30.095 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:31:30.095 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045580+0000", 2024-09-14T06:31:30.095 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:30.096 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:30.096 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:30.096 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:31:30.096 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045580+0000", 2024-09-14T06:31:30.096 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:30.096 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:30.096 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:30.096 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:31:30.096 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045580+0000", 2024-09-14T06:31:30.096 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.0400000000000002e-07 2024-09-14T06:31:30.096 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:30.096 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:30.097 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:31:30.097 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045582+0000", 2024-09-14T06:31:30.097 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.7600000000000001e-06 2024-09-14T06:31:30.097 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:30.097 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:30.097 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:31:30.097 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045588+0000", 2024-09-14T06:31:30.097 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.5650000000000001e-06 2024-09-14T06:31:30.097 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:30.097 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:30.097 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:31:30.097 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045597+0000", 2024-09-14T06:31:30.097 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.465e-06 2024-09-14T06:31:30.097 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:30.097 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:30.098 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:31:30.098 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.077871+0000", 2024-09-14T06:31:30.098 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032273562999999998 2024-09-14T06:31:30.098 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:30.098 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:30.098 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:31:30.098 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.077880+0000", 2024-09-14T06:31:30.098 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.0880000000000001e-06 2024-09-14T06:31:30.098 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:30.098 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:31:30.098 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:30.098 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:30.098 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:30.099 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:98511 3.10 3.1bf56f90 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T06:31:30.099 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:30:07.045610+0000", 2024-09-14T06:31:30.099 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 83.025115967000005, 2024-09-14T06:31:30.099 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032338894, 2024-09-14T06:31:30.099 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:31:30.099 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:31:30.099 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "reached pg", 2024-09-14T06:31:30.099 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:31:30.099 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:31:30.099 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:30.099 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 98511 2024-09-14T06:31:30.099 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:30.099 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:31:30.099 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:30.099 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:31:30.100 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045610+0000", 2024-09-14T06:31:30.100 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:30.100 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:30.100 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:30.100 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:31:30.100 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045610+0000", 2024-09-14T06:31:30.100 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:30.100 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:30.100 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:30.100 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:31:30.100 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045610+0000", 2024-09-14T06:31:30.100 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.1600000000000002e-07 2024-09-14T06:31:30.100 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:30.100 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:30.101 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:31:30.101 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045612+0000", 2024-09-14T06:31:30.101 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.1150000000000001e-06 2024-09-14T06:31:30.101 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:30.101 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:30.101 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:31:30.101 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045618+0000", 2024-09-14T06:31:30.101 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.0499999999999997e-06 2024-09-14T06:31:30.101 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:30.101 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:30.101 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:31:30.101 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045623+0000", 2024-09-14T06:31:30.101 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.6770000000000003e-06 2024-09-14T06:31:30.102 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:30.102 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:30.102 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:31:30.102 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.077928+0000", 2024-09-14T06:31:30.102 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032305111999999997 2024-09-14T06:31:30.102 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:30.102 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:30.102 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:31:30.102 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.077949+0000", 2024-09-14T06:31:30.102 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.0523999999999999e-05 2024-09-14T06:31:30.102 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:30.102 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:31:30.102 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:30.102 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:30.102 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:30.103 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:98512 3.10 3.1bf56f90 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T06:31:30.103 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:30:07.045630+0000", 2024-09-14T06:31:30.103 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 83.025095961999995, 2024-09-14T06:31:30.103 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032346632, 2024-09-14T06:31:30.103 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:31:30.103 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:31:30.103 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "reached pg", 2024-09-14T06:31:30.103 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:31:30.103 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:31:30.103 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:30.103 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 98512 2024-09-14T06:31:30.103 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:30.103 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:31:30.103 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:30.104 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:31:30.104 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045630+0000", 2024-09-14T06:31:30.104 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:30.104 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:30.104 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:30.104 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:31:30.104 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045630+0000", 2024-09-14T06:31:30.104 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:30.104 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:30.104 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:30.104 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:31:30.104 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045630+0000", 2024-09-14T06:31:30.105 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.8200000000000001e-07 2024-09-14T06:31:30.105 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:30.105 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:30.105 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:31:30.105 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045632+0000", 2024-09-14T06:31:30.105 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.7430000000000001e-06 2024-09-14T06:31:30.105 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:30.105 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:30.105 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:31:30.105 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045637+0000", 2024-09-14T06:31:30.105 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.4550000000000003e-06 2024-09-14T06:31:30.105 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:30.105 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:30.106 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:31:30.106 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045644+0000", 2024-09-14T06:31:30.106 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.1330000000000001e-06 2024-09-14T06:31:30.106 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:30.106 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:30.106 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:31:30.106 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.077968+0000", 2024-09-14T06:31:30.106 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032323401000000002 2024-09-14T06:31:30.106 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:30.106 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:30.106 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:31:30.106 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.077976+0000", 2024-09-14T06:31:30.107 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.5180000000000004e-06 2024-09-14T06:31:30.107 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:30.107 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:31:30.107 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:30.107 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:30.107 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:30.107 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:98513 3.10 3.1bf56f90 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T06:31:30.107 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:30:07.045652+0000", 2024-09-14T06:31:30.107 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 83.025073398000004, 2024-09-14T06:31:30.107 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032353436999999999, 2024-09-14T06:31:30.107 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:31:30.107 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:31:30.107 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "reached pg", 2024-09-14T06:31:30.108 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:31:30.108 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:31:30.108 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:30.108 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 98513 2024-09-14T06:31:30.108 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:30.108 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:31:30.108 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:30.108 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:31:30.108 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045652+0000", 2024-09-14T06:31:30.108 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:30.108 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:30.108 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:30.108 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:31:30.108 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045652+0000", 2024-09-14T06:31:30.109 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:30.109 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:30.109 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:30.109 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:31:30.109 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045653+0000", 2024-09-14T06:31:30.109 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.3700000000000001e-07 2024-09-14T06:31:30.109 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:30.109 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:30.109 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:31:30.109 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045655+0000", 2024-09-14T06:31:30.109 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.854e-06 2024-09-14T06:31:30.109 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:30.109 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:30.109 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:31:30.110 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045661+0000", 2024-09-14T06:31:30.110 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.5509999999999996e-06 2024-09-14T06:31:30.110 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:30.110 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:30.110 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:31:30.110 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045668+0000", 2024-09-14T06:31:30.110 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.1980000000000004e-06 2024-09-14T06:31:30.110 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:30.110 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:30.110 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:31:30.110 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.077997+0000", 2024-09-14T06:31:30.110 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032328575999999998 2024-09-14T06:31:30.110 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:30.110 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:30.110 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:31:30.111 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.078006+0000", 2024-09-14T06:31:30.111 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.8210000000000008e-06 2024-09-14T06:31:30.111 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:30.111 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:31:30.111 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:30.111 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:30.111 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:30.111 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:98514 3.10 3.1bf56f90 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T06:31:30.111 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:30:07.045699+0000", 2024-09-14T06:31:30.111 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 83.025027062999996, 2024-09-14T06:31:30.111 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032330223999999998, 2024-09-14T06:31:30.111 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:31:30.111 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:31:30.112 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "reached pg", 2024-09-14T06:31:30.112 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:31:30.112 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:31:30.112 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:30.112 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 98514 2024-09-14T06:31:30.112 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:30.112 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:31:30.112 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:30.112 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:31:30.112 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045699+0000", 2024-09-14T06:31:30.112 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:30.112 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:30.112 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:30.112 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:31:30.112 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045699+0000", 2024-09-14T06:31:30.113 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:30.113 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:30.113 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:30.113 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:31:30.113 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045699+0000", 2024-09-14T06:31:30.113 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.3200000000000001e-07 2024-09-14T06:31:30.113 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:30.113 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:30.113 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:31:30.113 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045701+0000", 2024-09-14T06:31:30.113 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.762e-06 2024-09-14T06:31:30.113 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:30.113 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:30.114 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:31:30.114 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045705+0000", 2024-09-14T06:31:30.114 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.2359999999999998e-06 2024-09-14T06:31:30.114 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:30.114 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:30.114 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:31:30.114 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045708+0000", 2024-09-14T06:31:30.114 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.796e-06 2024-09-14T06:31:30.114 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:30.114 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:30.114 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:31:30.114 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.078023+0000", 2024-09-14T06:31:30.114 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032315213000000002 2024-09-14T06:31:30.114 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:30.114 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:30.114 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:31:30.115 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.078029+0000", 2024-09-14T06:31:30.115 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.8850000000000004e-06 2024-09-14T06:31:30.115 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:30.115 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:31:30.115 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:30.115 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:30.115 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:30.115 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:98515 3.10 3.1bf56f90 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T06:31:30.115 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:30:07.045712+0000", 2024-09-14T06:31:30.115 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 83.025013752000007, 2024-09-14T06:31:30.115 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032346871999999999, 2024-09-14T06:31:30.115 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:31:30.115 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:31:30.115 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "reached pg", 2024-09-14T06:31:30.115 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:31:30.116 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:31:30.116 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:30.116 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 98515 2024-09-14T06:31:30.116 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:30.116 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:31:30.116 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:30.116 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:31:30.116 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045712+0000", 2024-09-14T06:31:30.116 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:30.116 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:30.116 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:30.116 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:31:30.116 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045712+0000", 2024-09-14T06:31:30.116 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:30.116 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:30.117 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:30.117 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:31:30.117 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045712+0000", 2024-09-14T06:31:30.117 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.2100000000000001e-07 2024-09-14T06:31:30.117 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:30.117 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:30.117 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:31:30.117 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045713+0000", 2024-09-14T06:31:30.117 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.068e-06 2024-09-14T06:31:30.117 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:30.117 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:30.117 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:31:30.117 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045716+0000", 2024-09-14T06:31:30.117 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.971e-06 2024-09-14T06:31:30.118 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:30.118 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:30.118 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:31:30.118 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045719+0000", 2024-09-14T06:31:30.118 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.3879999999999998e-06 2024-09-14T06:31:30.118 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:30.118 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:30.118 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:31:30.118 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.078049+0000", 2024-09-14T06:31:30.118 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032330553999999997 2024-09-14T06:31:30.118 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:30.118 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:30.118 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:31:30.118 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.078059+0000", 2024-09-14T06:31:30.119 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.6700000000000006e-06 2024-09-14T06:31:30.119 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:30.119 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:31:30.119 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:30.119 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:30.119 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:30.119 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:98516 3.10 3.1bf56f90 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T06:31:30.119 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:30:07.045722+0000", 2024-09-14T06:31:30.119 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 83.025003999000006, 2024-09-14T06:31:30.119 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032374334999999997, 2024-09-14T06:31:30.119 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:31:30.119 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:31:30.119 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "reached pg", 2024-09-14T06:31:30.119 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:31:30.120 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:31:30.120 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:30.120 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 98516 2024-09-14T06:31:30.120 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:30.120 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:31:30.120 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:30.120 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:31:30.120 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045722+0000", 2024-09-14T06:31:30.120 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:30.120 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:30.120 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:30.120 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:31:30.120 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045722+0000", 2024-09-14T06:31:30.120 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:30.121 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:30.121 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:30.121 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:31:30.121 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045722+0000", 2024-09-14T06:31:30.121 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.8999999999999998e-07 2024-09-14T06:31:30.121 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:30.121 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:30.121 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:31:30.121 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045723+0000", 2024-09-14T06:31:30.121 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.0049999999999999e-06 2024-09-14T06:31:30.121 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:30.121 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:30.121 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:31:30.121 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045726+0000", 2024-09-14T06:31:30.121 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.9019999999999999e-06 2024-09-14T06:31:30.122 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:30.122 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:30.122 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:31:30.122 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045728+0000", 2024-09-14T06:31:30.122 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.3290000000000001e-06 2024-09-14T06:31:30.122 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:30.122 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:30.122 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:31:30.122 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.078090+0000", 2024-09-14T06:31:30.122 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032362123 2024-09-14T06:31:30.122 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:30.122 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:30.123 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:31:30.123 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.078096+0000", 2024-09-14T06:31:30.123 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.6860000000000003e-06 2024-09-14T06:31:30.123 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:30.123 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:31:30.123 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:30.123 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:30.123 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:30.123 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:98517 3.10 3.1bf56f90 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T06:31:30.123 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:30:07.045731+0000", 2024-09-14T06:31:30.123 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 83.024994268, 2024-09-14T06:31:30.123 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.03241227, 2024-09-14T06:31:30.123 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:31:30.123 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:31:30.124 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "reached pg", 2024-09-14T06:31:30.124 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:31:30.124 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:31:30.124 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:30.124 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 98517 2024-09-14T06:31:30.124 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:30.124 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:31:30.124 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:30.124 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:31:30.124 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045731+0000", 2024-09-14T06:31:30.124 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:30.124 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:30.124 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:30.125 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:31:30.125 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045731+0000", 2024-09-14T06:31:30.125 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:30.125 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:30.125 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:30.125 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:31:30.125 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045732+0000", 2024-09-14T06:31:30.125 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.0699999999999999e-07 2024-09-14T06:31:30.125 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:30.125 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:30.125 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:31:30.125 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045733+0000", 2024-09-14T06:31:30.125 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.9199999999999999e-07 2024-09-14T06:31:30.125 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:30.126 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:30.126 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:31:30.126 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045735+0000", 2024-09-14T06:31:30.126 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.8150000000000002e-06 2024-09-14T06:31:30.126 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:30.126 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:30.126 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:31:30.126 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045738+0000", 2024-09-14T06:31:30.126 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.3640000000000002e-06 2024-09-14T06:31:30.126 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:30.126 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:30.126 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:31:30.126 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.078126+0000", 2024-09-14T06:31:30.126 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032388206000000003 2024-09-14T06:31:30.126 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:30.127 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:30.127 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:31:30.127 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.078144+0000", 2024-09-14T06:31:30.127 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.7686e-05 2024-09-14T06:31:30.127 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:30.127 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:31:30.127 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:30.127 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:30.127 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:30.127 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:98518 3.10 3.1bf56f90 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T06:31:30.127 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:30:07.045741+0000", 2024-09-14T06:31:30.127 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 83.024984872999994, 2024-09-14T06:31:30.127 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032451793999999999, 2024-09-14T06:31:30.127 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:31:30.128 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:31:30.128 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "reached pg", 2024-09-14T06:31:30.128 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:31:30.128 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:31:30.128 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:30.128 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 98518 2024-09-14T06:31:30.128 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:30.128 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:31:30.128 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:30.128 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:31:30.128 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045741+0000", 2024-09-14T06:31:30.128 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:30.129 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:30.129 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:30.129 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:31:30.129 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045741+0000", 2024-09-14T06:31:30.129 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:30.129 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:30.129 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:30.129 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:31:30.129 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045741+0000", 2024-09-14T06:31:30.129 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.05e-07 2024-09-14T06:31:30.129 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:30.129 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:30.129 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:31:30.130 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045746+0000", 2024-09-14T06:31:30.130 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.2259999999999996e-06 2024-09-14T06:31:30.130 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:30.130 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:30.130 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:31:30.130 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045750+0000", 2024-09-14T06:31:30.130 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.2490000000000001e-06 2024-09-14T06:31:30.130 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:30.130 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:30.130 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:31:30.130 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045752+0000", 2024-09-14T06:31:30.130 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.3980000000000002e-06 2024-09-14T06:31:30.130 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:30.130 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:30.131 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:31:30.131 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.078184+0000", 2024-09-14T06:31:30.131 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032431593000000002 2024-09-14T06:31:30.131 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:30.131 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:30.131 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:31:30.131 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.078193+0000", 2024-09-14T06:31:30.131 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.1230000000000006e-06 2024-09-14T06:31:30.131 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:30.131 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:31:30.131 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:30.131 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:30.131 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:30.131 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:98519 3.10 3.1bf56f90 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T06:31:30.132 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:30:07.045755+0000", 2024-09-14T06:31:30.132 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 83.024970448000005, 2024-09-14T06:31:30.132 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032489431999999999, 2024-09-14T06:31:30.132 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:31:30.132 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:31:30.132 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "reached pg", 2024-09-14T06:31:30.132 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:31:30.132 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:31:30.132 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:30.132 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 98519 2024-09-14T06:31:30.132 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:30.132 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:31:30.132 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:30.132 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:31:30.133 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045755+0000", 2024-09-14T06:31:30.133 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:30.133 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:30.133 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:30.133 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:31:30.133 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045755+0000", 2024-09-14T06:31:30.133 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:30.133 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:30.133 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:30.133 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:31:30.133 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045756+0000", 2024-09-14T06:31:30.133 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.23e-07 2024-09-14T06:31:30.133 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:30.133 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:30.134 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:31:30.134 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045757+0000", 2024-09-14T06:31:30.134 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.1060000000000001e-06 2024-09-14T06:31:30.134 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:30.134 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:30.134 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:31:30.134 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045760+0000", 2024-09-14T06:31:30.134 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.988e-06 2024-09-14T06:31:30.134 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:30.134 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:30.134 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:31:30.134 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045765+0000", 2024-09-14T06:31:30.134 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.2689999999999999e-06 2024-09-14T06:31:30.134 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:30.134 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:30.134 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:31:30.135 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.078234+0000", 2024-09-14T06:31:30.135 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032468915000000001 2024-09-14T06:31:30.135 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:30.135 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:30.135 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:31:30.135 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.078245+0000", 2024-09-14T06:31:30.135 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.0930999999999999e-05 2024-09-14T06:31:30.135 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:30.135 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:31:30.135 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:30.135 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:30.135 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:30.135 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:98520 3.10 3.1bf56f90 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T06:31:30.135 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:30:07.045768+0000", 2024-09-14T06:31:30.135 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 83.024957569999998, 2024-09-14T06:31:30.135 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032531379999999999, 2024-09-14T06:31:30.136 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:31:30.136 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:31:30.136 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "reached pg", 2024-09-14T06:31:30.136 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:31:30.136 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:31:30.136 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:30.136 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 98520 2024-09-14T06:31:30.136 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:30.136 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:31:30.136 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:30.136 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:31:30.136 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045768+0000", 2024-09-14T06:31:30.136 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:30.136 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:30.136 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:30.136 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:31:30.137 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045768+0000", 2024-09-14T06:31:30.137 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:30.137 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:30.137 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:30.137 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:31:30.137 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045768+0000", 2024-09-14T06:31:30.137 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.22e-07 2024-09-14T06:31:30.137 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:30.137 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:30.137 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:31:30.137 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045773+0000", 2024-09-14T06:31:30.137 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.3470000000000001e-06 2024-09-14T06:31:30.137 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:30.137 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:30.138 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:31:30.138 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045776+0000", 2024-09-14T06:31:30.138 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.1319999999999998e-06 2024-09-14T06:31:30.138 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:30.138 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:30.138 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:31:30.138 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045779+0000", 2024-09-14T06:31:30.138 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.9749999999999999e-06 2024-09-14T06:31:30.138 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:30.138 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:30.138 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:31:30.138 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.078289+0000", 2024-09-14T06:31:30.138 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032509693999999999 2024-09-14T06:31:30.138 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:30.138 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:30.139 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:31:30.139 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.078300+0000", 2024-09-14T06:31:30.139 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.101e-05 2024-09-14T06:31:30.139 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:30.139 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:31:30.139 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:30.139 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:30.139 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:30.139 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:98521 3.10 3.1bf56f90 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T06:31:30.139 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:30:07.045782+0000", 2024-09-14T06:31:30.139 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 83.024943390000004, 2024-09-14T06:31:30.139 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032566109000000003, 2024-09-14T06:31:30.139 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:31:30.139 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:31:30.140 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "reached pg", 2024-09-14T06:31:30.140 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:31:30.140 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:31:30.140 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:30.140 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 98521 2024-09-14T06:31:30.140 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:30.140 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:31:30.140 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:30.140 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:31:30.140 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045782+0000", 2024-09-14T06:31:30.140 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:30.140 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:30.140 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:30.140 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:31:30.141 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045782+0000", 2024-09-14T06:31:30.141 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:30.141 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:30.141 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:30.141 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:31:30.141 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045783+0000", 2024-09-14T06:31:30.141 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.2499999999999999e-07 2024-09-14T06:31:30.141 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:30.141 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:30.141 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:31:30.141 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045784+0000", 2024-09-14T06:31:30.141 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.0109999999999999e-06 2024-09-14T06:31:30.141 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:30.141 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:30.141 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:31:30.142 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045786+0000", 2024-09-14T06:31:30.142 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.8289999999999999e-06 2024-09-14T06:31:30.142 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:30.142 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:30.142 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:31:30.142 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045793+0000", 2024-09-14T06:31:30.142 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.579e-06 2024-09-14T06:31:30.142 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:30.142 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:30.142 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:31:30.142 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.078338+0000", 2024-09-14T06:31:30.142 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032545389000000001 2024-09-14T06:31:30.142 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:30.142 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:30.142 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:31:30.143 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.078348+0000", 2024-09-14T06:31:30.143 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.0076e-05 2024-09-14T06:31:30.143 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:30.143 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:31:30.143 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:30.143 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:30.143 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:30.143 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:98522 3.10 3.1bf56f90 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T06:31:30.143 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:30:07.045796+0000", 2024-09-14T06:31:30.143 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 83.024929482000005, 2024-09-14T06:31:30.143 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032577846000000001, 2024-09-14T06:31:30.143 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:31:30.143 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:31:30.143 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "reached pg", 2024-09-14T06:31:30.143 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:31:30.143 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:31:30.143 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:30.144 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 98522 2024-09-14T06:31:30.144 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:30.144 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:31:30.144 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:30.144 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:31:30.144 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045796+0000", 2024-09-14T06:31:30.144 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:30.144 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:30.144 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:30.144 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:31:30.144 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045796+0000", 2024-09-14T06:31:30.144 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:30.144 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:30.144 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:30.145 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:31:30.145 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045796+0000", 2024-09-14T06:31:30.145 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.2100000000000001e-07 2024-09-14T06:31:30.145 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:30.145 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:30.145 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:31:30.145 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045797+0000", 2024-09-14T06:31:30.145 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.6899999999999996e-07 2024-09-14T06:31:30.145 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:30.145 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:30.145 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:31:30.145 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045800+0000", 2024-09-14T06:31:30.145 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.9979999999999999e-06 2024-09-14T06:31:30.145 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:30.145 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:30.146 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:31:30.146 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045803+0000", 2024-09-14T06:31:30.146 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.2850000000000001e-06 2024-09-14T06:31:30.146 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:30.146 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:30.146 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:31:30.146 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.078366+0000", 2024-09-14T06:31:30.146 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032562932000000003 2024-09-14T06:31:30.146 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:30.146 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:30.146 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:31:30.146 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.078374+0000", 2024-09-14T06:31:30.146 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.4409999999999993e-06 2024-09-14T06:31:30.146 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:30.146 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:31:30.146 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:30.147 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:30.147 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:30.147 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:98523 3.10 3.1bf56f90 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T06:31:30.147 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:30:07.045806+0000", 2024-09-14T06:31:30.147 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 83.024919939, 2024-09-14T06:31:30.147 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032611167000000003, 2024-09-14T06:31:30.147 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:31:30.147 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:31:30.147 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "reached pg", 2024-09-14T06:31:30.147 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:31:30.147 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:31:30.147 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:30.147 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 98523 2024-09-14T06:31:30.147 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:30.147 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:31:30.147 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:30.147 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:31:30.148 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045806+0000", 2024-09-14T06:31:30.148 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:30.148 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:30.148 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:30.148 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:31:30.148 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045806+0000", 2024-09-14T06:31:30.148 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:30.148 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:30.148 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:30.148 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:31:30.148 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045806+0000", 2024-09-14T06:31:30.148 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.2000000000000001e-07 2024-09-14T06:31:30.148 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:30.148 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:30.148 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:31:30.149 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045807+0000", 2024-09-14T06:31:30.149 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.4e-07 2024-09-14T06:31:30.149 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:30.149 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:30.149 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:31:30.149 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045810+0000", 2024-09-14T06:31:30.149 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.9229999999999998e-06 2024-09-14T06:31:30.149 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:30.149 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:30.149 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:31:30.149 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045812+0000", 2024-09-14T06:31:30.149 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.2220000000000001e-06 2024-09-14T06:31:30.149 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:30.149 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:30.149 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:31:30.149 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.078407+0000", 2024-09-14T06:31:30.150 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032594513999999998 2024-09-14T06:31:30.150 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:30.150 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:30.150 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:31:30.150 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.078417+0000", 2024-09-14T06:31:30.150 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.0348e-05 2024-09-14T06:31:30.150 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:30.150 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:31:30.150 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:30.150 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:30.150 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:30.150 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:98524 3.10 3.1bf56f90 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T06:31:30.150 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:30:07.045815+0000", 2024-09-14T06:31:30.150 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 83.024910571000007, 2024-09-14T06:31:30.151 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032643208999999999, 2024-09-14T06:31:30.151 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:31:30.151 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:31:30.151 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "reached pg", 2024-09-14T06:31:30.151 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:31:30.151 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:31:30.151 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:30.151 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 98524 2024-09-14T06:31:30.151 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:30.151 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:31:30.151 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:30.151 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:31:30.151 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045815+0000", 2024-09-14T06:31:30.151 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:30.151 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:30.151 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:30.152 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:31:30.152 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045815+0000", 2024-09-14T06:31:30.152 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:30.152 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:30.152 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:30.152 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:31:30.152 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045815+0000", 2024-09-14T06:31:30.152 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.11e-07 2024-09-14T06:31:30.152 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:30.152 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:30.152 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:31:30.152 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045816+0000", 2024-09-14T06:31:30.152 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.9099999999999991e-07 2024-09-14T06:31:30.152 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:30.152 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:30.153 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:31:30.153 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045819+0000", 2024-09-14T06:31:30.153 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.9399999999999998e-06 2024-09-14T06:31:30.153 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:30.153 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:30.153 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:31:30.153 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045822+0000", 2024-09-14T06:31:30.153 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.3470000000000001e-06 2024-09-14T06:31:30.153 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:30.153 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:30.153 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:31:30.153 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.078449+0000", 2024-09-14T06:31:30.153 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032627529000000002 2024-09-14T06:31:30.153 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:30.153 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:30.153 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:31:30.154 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.078458+0000", 2024-09-14T06:31:30.154 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.1910000000000006e-06 2024-09-14T06:31:30.154 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:30.154 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:31:30.154 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:30.154 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:30.154 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:31:30.154 INFO:teuthology.orchestra.run.smithi138.stdout:} 2024-09-14T06:31:30.155 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-14T06:31:30.156 INFO:tasks.ceph.osd.5.smithi138.stderr:2024-09-14T06:31:30.154+0000 7fe13746f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33618) UID: 0 2024-09-14T06:31:30.256 INFO:tasks.ceph.osd.3.smithi094.stderr:2024-09-14T06:31:30.254+0000 7feeeafe8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33655) UID: 0 2024-09-14T06:31:30.298 INFO:teuthology.orchestra.run.smithi138.stdout:{ 2024-09-14T06:31:30.298 INFO:teuthology.orchestra.run.smithi138.stdout: "ops": [], 2024-09-14T06:31:30.298 INFO:teuthology.orchestra.run.smithi138.stdout: "num_ops": 0 2024-09-14T06:31:30.298 INFO:teuthology.orchestra.run.smithi138.stdout:} 2024-09-14T06:31:30.307 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-14T06:31:30.356 INFO:tasks.ceph.osd.7.smithi138.stderr:2024-09-14T06:31:30.355+0000 7f3c6a21d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33623) UID: 0 2024-09-14T06:31:30.409 INFO:teuthology.orchestra.run.smithi138.stdout:{ 2024-09-14T06:31:30.409 INFO:teuthology.orchestra.run.smithi138.stdout: "ops": [], 2024-09-14T06:31:30.409 INFO:teuthology.orchestra.run.smithi138.stdout: "complaint_time": 30, 2024-09-14T06:31:30.409 INFO:teuthology.orchestra.run.smithi138.stdout: "num_blocked_ops": 0 2024-09-14T06:31:30.409 INFO:teuthology.orchestra.run.smithi138.stdout:} 2024-09-14T06:31:30.417 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-14T06:31:30.458 INFO:tasks.ceph.osd.7.smithi138.stderr:2024-09-14T06:31:30.457+0000 7f3c6a21d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33623) UID: 0 2024-09-14T06:31:30.558 INFO:tasks.ceph.osd.1.smithi094.stderr:2024-09-14T06:31:30.557+0000 7f328e47a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33641) UID: 0 2024-09-14T06:31:30.562 INFO:teuthology.orchestra.run.smithi138.stdout:{ 2024-09-14T06:31:30.562 INFO:teuthology.orchestra.run.smithi138.stdout: "size": 20, 2024-09-14T06:31:30.562 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 600, 2024-09-14T06:31:30.562 INFO:teuthology.orchestra.run.smithi138.stdout: "ops": [ 2024-09-14T06:31:30.562 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:30.563 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:53244 3.f 3:f71f51a7:::rbd_data.112dca2f6ae1.0000000000000015:17 [read 1863680~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:31:30.563 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:44.369399+0000", 2024-09-14T06:31:30.563 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 106.19144766700001, 2024-09-14T06:31:30.563 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047291189999999997, 2024-09-14T06:31:30.563 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:31:30.563 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:31:30.563 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:31:30.563 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:31:30.563 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:31:30.563 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:30.563 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 53244 2024-09-14T06:31:30.563 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:30.563 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:31:30.563 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:30.564 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:31:30.564 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369399+0000", 2024-09-14T06:31:30.564 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:30.564 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:30.564 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:30.564 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:31:30.564 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369399+0000", 2024-09-14T06:31:30.564 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:30.564 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:30.564 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:30.564 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:31:30.564 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369400+0000", 2024-09-14T06:31:30.564 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.0699999999999999e-07 2024-09-14T06:31:30.565 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:30.565 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:30.565 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:31:30.565 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369401+0000", 2024-09-14T06:31:30.565 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.4300000000000001e-06 2024-09-14T06:31:30.565 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:30.565 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:30.565 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:31:30.565 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369404+0000", 2024-09-14T06:31:30.565 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.061e-06 2024-09-14T06:31:30.565 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:30.565 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:30.565 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:31:30.565 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369407+0000", 2024-09-14T06:31:30.566 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.6139999999999999e-06 2024-09-14T06:31:30.566 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:30.566 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:30.566 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:31:30.566 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.416517+0000", 2024-09-14T06:31:30.566 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047110338000000002 2024-09-14T06:31:30.566 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:30.566 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:30.566 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:31:30.566 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.416616+0000", 2024-09-14T06:31:30.566 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.8651000000000001e-05 2024-09-14T06:31:30.566 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:30.566 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:30.566 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:31:30.566 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.416691+0000", 2024-09-14T06:31:30.567 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.4888999999999994e-05 2024-09-14T06:31:30.567 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:30.567 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:31:30.567 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:30.567 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:30.567 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:30.567 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:53245 3.f 3:f71f51a7:::rbd_data.112dca2f6ae1.0000000000000015:17 [read 1888256~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:31:30.567 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:44.369410+0000", 2024-09-14T06:31:30.567 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 106.19143737900001, 2024-09-14T06:31:30.567 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047399275999999997, 2024-09-14T06:31:30.567 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:31:30.567 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:31:30.567 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:31:30.567 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:31:30.568 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:31:30.568 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:30.568 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 53245 2024-09-14T06:31:30.568 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:30.568 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:31:30.568 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:30.568 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:31:30.568 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369410+0000", 2024-09-14T06:31:30.568 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:30.568 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:30.568 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:30.568 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:31:30.568 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369410+0000", 2024-09-14T06:31:30.568 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:30.569 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:30.569 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:30.569 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:31:30.569 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369410+0000", 2024-09-14T06:31:30.569 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.1899999999999999e-07 2024-09-14T06:31:30.569 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:30.569 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:30.569 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:31:30.569 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369411+0000", 2024-09-14T06:31:30.569 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.0300000000000001e-06 2024-09-14T06:31:30.569 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:30.569 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:30.569 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:31:30.569 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369414+0000", 2024-09-14T06:31:30.570 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.1099999999999999e-06 2024-09-14T06:31:30.570 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:30.570 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:30.570 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:31:30.570 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369417+0000", 2024-09-14T06:31:30.570 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.6189999999999998e-06 2024-09-14T06:31:30.570 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:30.570 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:30.570 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:31:30.570 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.416718+0000", 2024-09-14T06:31:30.570 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047301322999999999 2024-09-14T06:31:30.570 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:30.570 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:30.570 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:31:30.571 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.416759+0000", 2024-09-14T06:31:30.571 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.1009e-05 2024-09-14T06:31:30.571 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:30.571 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:30.571 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:31:30.571 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.416809+0000", 2024-09-14T06:31:30.571 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.9966000000000002e-05 2024-09-14T06:31:30.571 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:30.571 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:31:30.571 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:30.571 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:30.571 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:30.571 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:53246 3.f 3:f71f51a7:::rbd_data.112dca2f6ae1.0000000000000015:17 [read 1912832~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:31:30.571 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:44.369420+0000", 2024-09-14T06:31:30.571 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 106.19142743499999, 2024-09-14T06:31:30.572 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047529463000000001, 2024-09-14T06:31:30.572 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:31:30.572 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:31:30.572 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:31:30.572 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:31:30.572 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:31:30.572 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:30.572 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 53246 2024-09-14T06:31:30.572 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:30.572 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:31:30.572 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:30.572 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:31:30.572 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369420+0000", 2024-09-14T06:31:30.572 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:30.573 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:30.573 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:30.573 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:31:30.573 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369420+0000", 2024-09-14T06:31:30.573 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:30.573 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:30.573 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:30.573 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:31:30.573 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369420+0000", 2024-09-14T06:31:30.573 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.1299999999999999e-07 2024-09-14T06:31:30.573 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:30.573 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:30.573 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:31:30.573 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369425+0000", 2024-09-14T06:31:30.574 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.0130000000000001e-06 2024-09-14T06:31:30.574 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:30.574 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:30.574 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:31:30.574 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369428+0000", 2024-09-14T06:31:30.574 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.349e-06 2024-09-14T06:31:30.574 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:30.574 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:30.574 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:31:30.574 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369431+0000", 2024-09-14T06:31:30.574 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.7389999999999999e-06 2024-09-14T06:31:30.574 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:30.574 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:30.574 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:31:30.575 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.416834+0000", 2024-09-14T06:31:30.575 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047402669000000001 2024-09-14T06:31:30.575 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:30.575 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:30.575 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:31:30.575 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.416882+0000", 2024-09-14T06:31:30.575 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.8068999999999997e-05 2024-09-14T06:31:30.575 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:30.575 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:30.575 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:31:30.575 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.416949+0000", 2024-09-14T06:31:30.575 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.7410999999999995e-05 2024-09-14T06:31:30.575 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:30.575 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:31:30.575 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:30.576 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:30.576 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:30.576 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:53247 3.f 3:f71f51a7:::rbd_data.112dca2f6ae1.0000000000000015:17 [read 1937408~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:31:30.576 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:44.369434+0000", 2024-09-14T06:31:30.576 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 106.19141267099999, 2024-09-14T06:31:30.576 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047652264999999999, 2024-09-14T06:31:30.576 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:31:30.576 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:31:30.576 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:31:30.576 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:31:30.576 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:31:30.576 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:30.576 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 53247 2024-09-14T06:31:30.577 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:30.577 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:31:30.577 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:30.577 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:31:30.577 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369434+0000", 2024-09-14T06:31:30.577 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:30.577 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:30.577 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:30.577 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:31:30.577 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369434+0000", 2024-09-14T06:31:30.577 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:30.577 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:30.577 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:30.578 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:31:30.578 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369435+0000", 2024-09-14T06:31:30.578 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.2100000000000001e-07 2024-09-14T06:31:30.578 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:30.578 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:30.578 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:31:30.578 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369436+0000", 2024-09-14T06:31:30.578 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.122e-06 2024-09-14T06:31:30.578 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:30.578 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:30.578 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:31:30.578 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369439+0000", 2024-09-14T06:31:30.578 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.0400000000000001e-06 2024-09-14T06:31:30.578 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:30.579 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:30.579 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:31:30.579 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369441+0000", 2024-09-14T06:31:30.579 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.728e-06 2024-09-14T06:31:30.579 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:30.579 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:30.579 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:31:30.579 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.416977+0000", 2024-09-14T06:31:30.579 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047535344 2024-09-14T06:31:30.579 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:30.579 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:30.579 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:31:30.579 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.417029+0000", 2024-09-14T06:31:30.579 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.2037999999999999e-05 2024-09-14T06:31:30.579 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:30.580 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:30.580 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:31:30.580 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.417087+0000", 2024-09-14T06:31:30.580 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.7772000000000002e-05 2024-09-14T06:31:30.580 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:30.580 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:31:30.580 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:30.580 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:30.580 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:30.580 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:53248 3.f 3:f71f51a7:::rbd_data.112dca2f6ae1.0000000000000015:17 [read 1961984~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:31:30.580 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:44.369445+0000", 2024-09-14T06:31:30.580 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 106.191402119, 2024-09-14T06:31:30.580 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047798489, 2024-09-14T06:31:30.581 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:31:30.581 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:31:30.581 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:31:30.581 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:31:30.581 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:31:30.581 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:30.581 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 53248 2024-09-14T06:31:30.581 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:30.581 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:31:30.581 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:30.581 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:31:30.581 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369445+0000", 2024-09-14T06:31:30.581 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:30.582 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:30.582 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:30.582 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:31:30.582 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369445+0000", 2024-09-14T06:31:30.582 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:30.582 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:30.582 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:30.582 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:31:30.582 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369445+0000", 2024-09-14T06:31:30.582 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.0699999999999999e-07 2024-09-14T06:31:30.582 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:30.582 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:30.582 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:31:30.582 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369446+0000", 2024-09-14T06:31:30.583 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.1000000000000001e-06 2024-09-14T06:31:30.583 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:30.583 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:30.583 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:31:30.583 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369449+0000", 2024-09-14T06:31:30.583 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.1039999999999998e-06 2024-09-14T06:31:30.583 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:30.583 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:30.583 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:31:30.583 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369452+0000", 2024-09-14T06:31:30.583 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.6199999999999999e-06 2024-09-14T06:31:30.583 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:30.583 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:30.583 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:31:30.583 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.417114+0000", 2024-09-14T06:31:30.584 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047661901999999999 2024-09-14T06:31:30.584 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:30.584 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:30.584 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:31:30.584 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.417181+0000", 2024-09-14T06:31:30.584 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.7385999999999995e-05 2024-09-14T06:31:30.584 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:30.584 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:30.584 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:31:30.584 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.417243+0000", 2024-09-14T06:31:30.584 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.2169999999999996e-05 2024-09-14T06:31:30.584 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:30.584 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:31:30.584 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:30.585 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:30.585 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:30.585 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:53249 3.f 3:f71f51a7:::rbd_data.112dca2f6ae1.0000000000000015:17 [read 1986560~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:31:30.585 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:44.369455+0000", 2024-09-14T06:31:30.585 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 106.19139180800001, 2024-09-14T06:31:30.585 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047890493999999999, 2024-09-14T06:31:30.585 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:31:30.585 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:31:30.585 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:31:30.585 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:31:30.585 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:31:30.585 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:30.585 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 53249 2024-09-14T06:31:30.585 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:30.585 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:31:30.586 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:30.586 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:31:30.586 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369455+0000", 2024-09-14T06:31:30.586 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:30.586 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:30.586 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:30.586 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:31:30.586 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369455+0000", 2024-09-14T06:31:30.586 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:30.586 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:30.586 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:30.586 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:31:30.586 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369455+0000", 2024-09-14T06:31:30.586 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.0800000000000001e-07 2024-09-14T06:31:30.587 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:30.587 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:30.587 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:31:30.587 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369456+0000", 2024-09-14T06:31:30.587 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.0729999999999999e-06 2024-09-14T06:31:30.587 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:30.587 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:30.587 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:31:30.587 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369460+0000", 2024-09-14T06:31:30.587 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.027e-06 2024-09-14T06:31:30.587 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:30.587 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:30.587 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:31:30.587 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369462+0000", 2024-09-14T06:31:30.588 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.5320000000000001e-06 2024-09-14T06:31:30.588 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:30.588 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:30.588 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:31:30.588 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.417257+0000", 2024-09-14T06:31:30.588 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.04779489 2024-09-14T06:31:30.588 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:30.588 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:30.588 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:31:30.588 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.417296+0000", 2024-09-14T06:31:30.588 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.8658000000000002e-05 2024-09-14T06:31:30.588 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:30.588 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:30.588 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:31:30.588 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.417346+0000", 2024-09-14T06:31:30.589 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.0105999999999998e-05 2024-09-14T06:31:30.589 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:30.589 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:31:30.589 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:30.589 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:30.589 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:30.589 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:53250 3.f 3:f71f51a7:::rbd_data.112dca2f6ae1.0000000000000015:17 [read 2011136~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:31:30.589 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:44.369465+0000", 2024-09-14T06:31:30.589 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 106.191381838, 2024-09-14T06:31:30.589 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.048032230000000002, 2024-09-14T06:31:30.589 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:31:30.589 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:31:30.589 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:31:30.590 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:31:30.590 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:31:30.590 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:30.590 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 53250 2024-09-14T06:31:30.590 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:30.590 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:31:30.590 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:30.590 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:31:30.590 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369465+0000", 2024-09-14T06:31:30.590 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:30.590 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:30.590 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:30.590 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:31:30.591 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369465+0000", 2024-09-14T06:31:30.591 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:30.591 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:30.591 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:30.591 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:31:30.591 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369465+0000", 2024-09-14T06:31:30.591 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.2100000000000001e-07 2024-09-14T06:31:30.591 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:30.591 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:30.591 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:31:30.591 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369466+0000", 2024-09-14T06:31:30.591 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.0640000000000001e-06 2024-09-14T06:31:30.591 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:30.591 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:30.591 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:31:30.592 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369469+0000", 2024-09-14T06:31:30.592 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.0170000000000001e-06 2024-09-14T06:31:30.592 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:30.592 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:30.592 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:31:30.592 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369472+0000", 2024-09-14T06:31:30.592 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.6189999999999998e-06 2024-09-14T06:31:30.592 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:30.592 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:30.592 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:31:30.592 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.417383+0000", 2024-09-14T06:31:30.592 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047911092000000002 2024-09-14T06:31:30.592 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:30.592 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:30.592 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:31:30.593 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.417430+0000", 2024-09-14T06:31:30.593 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.6898999999999999e-05 2024-09-14T06:31:30.593 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:30.593 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:30.593 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:31:30.593 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.417497+0000", 2024-09-14T06:31:30.593 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.7317999999999995e-05 2024-09-14T06:31:30.593 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:30.593 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:31:30.593 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:30.593 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:30.593 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:30.593 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:53251 3.f 3:f71f51a7:::rbd_data.112dca2f6ae1.0000000000000015:17 [read 2035712~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:31:30.594 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:44.369475+0000", 2024-09-14T06:31:30.594 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 106.191371842, 2024-09-14T06:31:30.594 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.048154542000000002, 2024-09-14T06:31:30.594 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:31:30.594 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:31:30.594 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:31:30.594 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:31:30.594 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:31:30.594 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:30.594 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 53251 2024-09-14T06:31:30.594 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:30.594 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:31:30.594 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:30.594 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:31:30.594 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369475+0000", 2024-09-14T06:31:30.595 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:30.595 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:30.595 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:30.595 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:31:30.595 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369475+0000", 2024-09-14T06:31:30.595 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:30.595 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:30.595 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:30.595 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:31:30.595 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369475+0000", 2024-09-14T06:31:30.595 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.0100000000000001e-07 2024-09-14T06:31:30.595 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:30.595 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:30.596 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:31:30.596 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369476+0000", 2024-09-14T06:31:30.596 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.0580000000000001e-06 2024-09-14T06:31:30.596 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:30.596 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:30.596 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:31:30.596 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369479+0000", 2024-09-14T06:31:30.596 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.9950000000000001e-06 2024-09-14T06:31:30.596 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:30.596 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:30.596 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:31:30.596 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369482+0000", 2024-09-14T06:31:30.596 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.6340000000000001e-06 2024-09-14T06:31:30.597 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:30.597 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:30.597 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:31:30.597 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.417530+0000", 2024-09-14T06:31:30.597 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.048047535000000002 2024-09-14T06:31:30.597 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:30.597 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:30.597 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:31:30.597 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.417574+0000", 2024-09-14T06:31:30.597 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.4255999999999998e-05 2024-09-14T06:31:30.597 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:30.597 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:30.597 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:31:30.597 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.417630+0000", 2024-09-14T06:31:30.598 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.5862999999999997e-05 2024-09-14T06:31:30.598 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:30.598 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:31:30.598 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:30.598 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:30.598 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:30.598 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:53252 3.f 3:f71f51a7:::rbd_data.112dca2f6ae1.0000000000000015:17 [read 2060288~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:31:30.598 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:44.369485+0000", 2024-09-14T06:31:30.598 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 106.191361994, 2024-09-14T06:31:30.598 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.048254116, 2024-09-14T06:31:30.598 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:31:30.598 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:31:30.598 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:31:30.598 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:31:30.599 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:31:30.599 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:30.599 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 53252 2024-09-14T06:31:30.599 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:30.599 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:31:30.599 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:30.599 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:31:30.599 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369485+0000", 2024-09-14T06:31:30.599 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:30.599 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:30.599 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:30.599 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:31:30.599 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369485+0000", 2024-09-14T06:31:30.600 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:30.600 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:30.600 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:30.600 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:31:30.600 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369485+0000", 2024-09-14T06:31:30.600 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.1e-07 2024-09-14T06:31:30.600 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:30.600 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:30.600 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:31:30.600 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369487+0000", 2024-09-14T06:31:30.600 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.42e-06 2024-09-14T06:31:30.600 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:30.600 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:30.601 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:31:30.601 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369490+0000", 2024-09-14T06:31:30.601 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.9639999999999999e-06 2024-09-14T06:31:30.601 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:30.601 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:30.601 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:31:30.601 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369492+0000", 2024-09-14T06:31:30.601 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.5679999999999998e-06 2024-09-14T06:31:30.601 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:30.601 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:30.601 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:31:30.601 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.417645+0000", 2024-09-14T06:31:30.601 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.048152753 2024-09-14T06:31:30.601 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:30.602 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:30.602 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:31:30.602 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.417687+0000", 2024-09-14T06:31:30.602 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.1900999999999997e-05 2024-09-14T06:31:30.602 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:30.602 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:30.602 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:31:30.602 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.417739+0000", 2024-09-14T06:31:30.602 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.2299999999999997e-05 2024-09-14T06:31:30.602 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:30.602 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:31:30.602 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:30.602 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:30.603 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:30.603 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:53253 3.f 3:f71f51a7:::rbd_data.112dca2f6ae1.0000000000000015:17 [read 2084864~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:31:30.603 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:44.369495+0000", 2024-09-14T06:31:30.603 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 106.19135187000001, 2024-09-14T06:31:30.603 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.048366785000000002, 2024-09-14T06:31:30.603 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:31:30.603 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:31:30.603 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:31:30.603 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:31:30.603 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:31:30.603 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:30.603 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 53253 2024-09-14T06:31:30.603 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:30.603 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:31:30.604 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:30.604 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:31:30.604 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369495+0000", 2024-09-14T06:31:30.604 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:30.604 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:30.604 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:30.604 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:31:30.604 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369495+0000", 2024-09-14T06:31:30.604 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:30.604 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:30.604 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:30.604 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:31:30.604 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370388+0000", 2024-09-14T06:31:30.604 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00089287999999999998 2024-09-14T06:31:30.604 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:30.605 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:30.605 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:31:30.605 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370392+0000", 2024-09-14T06:31:30.605 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.2130000000000002e-06 2024-09-14T06:31:30.605 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:30.605 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:30.605 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:31:30.605 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370401+0000", 2024-09-14T06:31:30.605 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.7649999999999999e-06 2024-09-14T06:31:30.605 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:30.605 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:30.605 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:31:30.605 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370406+0000", 2024-09-14T06:31:30.605 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.0159999999999999e-06 2024-09-14T06:31:30.605 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:30.606 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:30.606 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:31:30.606 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.417764+0000", 2024-09-14T06:31:30.606 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047358348000000001 2024-09-14T06:31:30.606 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:30.606 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:30.606 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:31:30.606 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.417806+0000", 2024-09-14T06:31:30.606 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.1149999999999997e-05 2024-09-14T06:31:30.606 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:30.606 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:30.606 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:31:30.606 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.417862+0000", 2024-09-14T06:31:30.606 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.6412999999999997e-05 2024-09-14T06:31:30.606 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:30.606 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:31:30.607 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:30.607 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:30.607 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:30.607 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:53340 3.f 3:f71f51a7:::rbd_data.112dca2f6ae1.0000000000000015:17 [read 1056768~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:31:30.607 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:44.370421+0000", 2024-09-14T06:31:30.607 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 106.19042598999999, 2024-09-14T06:31:30.607 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047566313999999998, 2024-09-14T06:31:30.607 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:31:30.607 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:31:30.607 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:31:30.607 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:31:30.607 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:31:30.607 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:30.608 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 53340 2024-09-14T06:31:30.608 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:30.608 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:31:30.608 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:30.608 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:31:30.608 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370421+0000", 2024-09-14T06:31:30.608 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:30.608 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:30.608 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:30.608 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:31:30.608 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370421+0000", 2024-09-14T06:31:30.608 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:30.608 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:30.608 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:30.609 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:31:30.609 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370422+0000", 2024-09-14T06:31:30.609 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.3799999999999996e-07 2024-09-14T06:31:30.609 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:30.609 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:30.609 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:31:30.609 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370424+0000", 2024-09-14T06:31:30.609 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.6919999999999998e-06 2024-09-14T06:31:30.609 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:30.609 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:30.609 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:31:30.609 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370432+0000", 2024-09-14T06:31:30.609 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.2439999999999996e-06 2024-09-14T06:31:30.610 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:30.610 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:30.610 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:31:30.610 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370437+0000", 2024-09-14T06:31:30.610 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.5389999999999998e-06 2024-09-14T06:31:30.610 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:30.610 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:30.610 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:31:30.610 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.417888+0000", 2024-09-14T06:31:30.610 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047450432000000001 2024-09-14T06:31:30.610 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:30.610 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:30.610 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:31:30.610 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.417935+0000", 2024-09-14T06:31:30.611 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.7454999999999999e-05 2024-09-14T06:31:30.611 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:30.611 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:30.611 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:31:30.611 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.417987+0000", 2024-09-14T06:31:30.611 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.2213999999999998e-05 2024-09-14T06:31:30.611 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:30.611 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:31:30.611 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:30.611 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:30.611 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:30.611 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:53341 3.f 3:f71f51a7:::rbd_data.112dca2f6ae1.0000000000000015:17 [read 1081344~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:31:30.612 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:44.370446+0000", 2024-09-14T06:31:30.612 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 106.190400717, 2024-09-14T06:31:30.612 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047665723, 2024-09-14T06:31:30.612 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:31:30.612 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:31:30.612 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:31:30.612 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:31:30.612 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:31:30.612 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:30.612 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 53341 2024-09-14T06:31:30.612 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:30.612 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:31:30.612 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:30.612 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:31:30.613 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370446+0000", 2024-09-14T06:31:30.613 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:30.613 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:30.613 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:30.613 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:31:30.613 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370446+0000", 2024-09-14T06:31:30.613 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:30.613 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:30.613 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:30.613 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:31:30.613 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370447+0000", 2024-09-14T06:31:30.613 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.6100000000000002e-07 2024-09-14T06:31:30.613 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:30.613 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:30.614 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:31:30.614 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370448+0000", 2024-09-14T06:31:30.614 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.7370000000000001e-06 2024-09-14T06:31:30.614 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:30.614 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:30.614 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:31:30.614 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370462+0000", 2024-09-14T06:31:30.614 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.3227000000000001e-05 2024-09-14T06:31:30.614 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:30.614 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:30.614 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:31:30.614 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370467+0000", 2024-09-14T06:31:30.614 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.6540000000000004e-06 2024-09-14T06:31:30.615 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:30.615 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:30.615 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:31:30.615 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.418013+0000", 2024-09-14T06:31:30.615 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047545325999999999 2024-09-14T06:31:30.615 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:30.615 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:30.615 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:31:30.615 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.418054+0000", 2024-09-14T06:31:30.615 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.1696999999999997e-05 2024-09-14T06:31:30.615 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:30.615 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:30.615 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:31:30.615 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.418112+0000", 2024-09-14T06:31:30.615 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.7720999999999998e-05 2024-09-14T06:31:30.616 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:30.616 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:31:30.616 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:30.616 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:30.616 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:30.616 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:53342 3.f 3:f71f51a7:::rbd_data.112dca2f6ae1.0000000000000015:17 [read 1105920~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:31:30.616 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:44.370475+0000", 2024-09-14T06:31:30.616 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 106.190371656, 2024-09-14T06:31:30.616 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047748848000000003, 2024-09-14T06:31:30.616 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:31:30.616 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:31:30.616 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:31:30.616 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:31:30.616 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:31:30.616 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:30.617 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 53342 2024-09-14T06:31:30.617 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:30.617 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:31:30.617 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:30.617 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:31:30.617 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370475+0000", 2024-09-14T06:31:30.617 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:30.617 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:30.617 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:30.617 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:31:30.617 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370475+0000", 2024-09-14T06:31:30.617 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:30.617 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:30.617 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:30.618 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:31:30.618 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370476+0000", 2024-09-14T06:31:30.618 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.4700000000000002e-07 2024-09-14T06:31:30.618 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:30.618 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:30.618 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:31:30.618 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370478+0000", 2024-09-14T06:31:30.618 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.5890000000000001e-06 2024-09-14T06:31:30.618 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:30.618 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:30.618 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:31:30.618 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370486+0000", 2024-09-14T06:31:30.618 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.3470000000000001e-06 2024-09-14T06:31:30.619 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:30.619 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:30.619 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:31:30.619 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370491+0000", 2024-09-14T06:31:30.619 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.5720000000000002e-06 2024-09-14T06:31:30.619 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:30.619 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:30.619 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:31:30.619 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.418135+0000", 2024-09-14T06:31:30.619 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047644094999999997 2024-09-14T06:31:30.619 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:30.619 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:30.619 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:31:30.619 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.418174+0000", 2024-09-14T06:31:30.620 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.8204000000000002e-05 2024-09-14T06:31:30.620 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:30.620 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:30.620 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:31:30.620 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.418224+0000", 2024-09-14T06:31:30.620 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.0593999999999997e-05 2024-09-14T06:31:30.620 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:30.620 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:31:30.620 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:30.620 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:30.620 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:30.620 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:53343 3.f 3:f71f51a7:::rbd_data.112dca2f6ae1.0000000000000015:17 [read 1130496~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:31:30.620 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:44.370498+0000", 2024-09-14T06:31:30.621 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 106.19034877999999, 2024-09-14T06:31:30.621 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047841574999999997, 2024-09-14T06:31:30.621 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:31:30.621 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:31:30.621 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:31:30.621 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:31:30.621 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:31:30.621 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:30.621 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 53343 2024-09-14T06:31:30.621 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:30.621 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:31:30.621 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:30.621 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:31:30.621 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370498+0000", 2024-09-14T06:31:30.622 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:30.622 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:30.622 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:30.622 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:31:30.622 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370498+0000", 2024-09-14T06:31:30.622 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:30.622 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:30.622 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:30.622 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:31:30.622 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370499+0000", 2024-09-14T06:31:30.622 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.9999999999999998e-07 2024-09-14T06:31:30.622 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:30.622 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:30.622 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:31:30.623 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370501+0000", 2024-09-14T06:31:30.623 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.0310000000000001e-06 2024-09-14T06:31:30.623 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:30.623 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:30.623 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:31:30.623 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370507+0000", 2024-09-14T06:31:30.623 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.4169999999999997e-06 2024-09-14T06:31:30.623 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:30.623 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:30.623 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:31:30.623 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370516+0000", 2024-09-14T06:31:30.623 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.7190000000000007e-06 2024-09-14T06:31:30.623 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:30.623 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:30.624 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:31:30.624 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.418247+0000", 2024-09-14T06:31:30.624 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047731190999999999 2024-09-14T06:31:30.624 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:30.624 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:30.624 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:31:30.624 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.418286+0000", 2024-09-14T06:31:30.624 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.8988e-05 2024-09-14T06:31:30.624 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:30.624 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:30.624 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:31:30.624 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.418340+0000", 2024-09-14T06:31:30.624 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.3829000000000002e-05 2024-09-14T06:31:30.625 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:30.625 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:31:30.625 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:30.625 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:30.625 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:30.625 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:53344 3.f 3:f71f51a7:::rbd_data.112dca2f6ae1.0000000000000015:17 [read 1155072~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:31:30.625 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:44.370524+0000", 2024-09-14T06:31:30.625 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 106.19032301, 2024-09-14T06:31:30.625 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047972262000000002, 2024-09-14T06:31:30.625 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:31:30.625 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:31:30.625 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:31:30.625 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:31:30.625 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:31:30.626 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:30.626 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 53344 2024-09-14T06:31:30.626 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:30.626 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:31:30.626 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:30.626 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:31:30.626 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370524+0000", 2024-09-14T06:31:30.626 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:30.626 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:30.626 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:30.626 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:31:30.626 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370524+0000", 2024-09-14T06:31:30.626 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:30.626 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:30.626 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:30.626 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:31:30.627 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370618+0000", 2024-09-14T06:31:30.627 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.4425999999999993e-05 2024-09-14T06:31:30.627 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:30.627 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:30.627 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:31:30.627 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370622+0000", 2024-09-14T06:31:30.627 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.3519999999999998e-06 2024-09-14T06:31:30.627 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:30.627 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:30.627 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:31:30.627 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370631+0000", 2024-09-14T06:31:30.627 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.9299999999999992e-06 2024-09-14T06:31:30.627 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:30.627 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:30.628 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:31:30.628 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370637+0000", 2024-09-14T06:31:30.628 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.3269999999999997e-06 2024-09-14T06:31:30.628 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:30.628 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:30.628 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:31:30.628 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.418372+0000", 2024-09-14T06:31:30.628 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047734764999999998 2024-09-14T06:31:30.628 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:30.628 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:30.628 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:31:30.628 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.418431+0000", 2024-09-14T06:31:30.628 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.8696999999999997e-05 2024-09-14T06:31:30.628 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:30.629 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:30.629 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:31:30.629 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.418496+0000", 2024-09-14T06:31:30.629 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.5765000000000005e-05 2024-09-14T06:31:30.629 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:30.629 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:31:30.629 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:30.629 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:30.629 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:30.629 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:53345 3.f 3:f71f51a7:::rbd_data.112dca2f6ae1.0000000000000015:17 [read 1179648~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:31:30.629 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:44.370647+0000", 2024-09-14T06:31:30.629 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 106.19019954300001, 2024-09-14T06:31:30.629 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047953775999999997, 2024-09-14T06:31:30.629 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:31:30.629 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:31:30.630 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:31:30.630 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:31:30.630 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:31:30.630 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:30.630 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 53345 2024-09-14T06:31:30.630 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:30.630 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:31:30.630 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:30.630 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:31:30.630 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370647+0000", 2024-09-14T06:31:30.630 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:30.630 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:30.630 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:30.631 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:31:30.631 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370647+0000", 2024-09-14T06:31:30.631 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:30.631 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:30.631 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:30.631 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:31:30.631 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.372346+0000", 2024-09-14T06:31:30.631 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.001698284 2024-09-14T06:31:30.631 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:30.631 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:30.631 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:31:30.631 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.372351+0000", 2024-09-14T06:31:30.631 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.8890000000000001e-06 2024-09-14T06:31:30.631 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:30.631 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:30.632 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:31:30.632 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.372362+0000", 2024-09-14T06:31:30.632 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.0941000000000001e-05 2024-09-14T06:31:30.632 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:30.632 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:30.632 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:31:30.632 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.372369+0000", 2024-09-14T06:31:30.632 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.362e-06 2024-09-14T06:31:30.632 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:30.632 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:30.632 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:31:30.632 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.418510+0000", 2024-09-14T06:31:30.632 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046140914999999998 2024-09-14T06:31:30.632 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:30.633 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:30.633 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:31:30.633 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.418549+0000", 2024-09-14T06:31:30.633 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.8717999999999998e-05 2024-09-14T06:31:30.633 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:30.633 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:30.633 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:31:30.633 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.418601+0000", 2024-09-14T06:31:30.633 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.2667000000000002e-05 2024-09-14T06:31:30.633 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:30.633 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:31:30.633 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:30.633 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:30.634 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:30.634 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:53351 3.f 3:f71f51a7:::rbd_data.112dca2f6ae1.0000000000000015:17 [read 1327104~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:31:30.634 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:44.372499+0000", 2024-09-14T06:31:30.634 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 106.188348422, 2024-09-14T06:31:30.634 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047140596, 2024-09-14T06:31:30.634 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:31:30.634 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:31:30.634 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:31:30.634 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:31:30.634 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:31:30.634 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:30.634 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 53351 2024-09-14T06:31:30.634 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:30.634 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:31:30.635 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:30.635 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:31:30.635 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.372499+0000", 2024-09-14T06:31:30.635 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:30.635 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:30.635 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:30.635 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:31:30.635 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.372499+0000", 2024-09-14T06:31:30.635 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:30.635 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:30.635 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:30.635 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:31:30.635 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.372499+0000", 2024-09-14T06:31:30.635 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.3000000000000001e-07 2024-09-14T06:31:30.635 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:30.636 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:30.636 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:31:30.636 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.372501+0000", 2024-09-14T06:31:30.636 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.0820000000000001e-06 2024-09-14T06:31:30.636 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:30.636 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:30.636 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:31:30.636 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.372513+0000", 2024-09-14T06:31:30.636 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.1610999999999999e-05 2024-09-14T06:31:30.636 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:30.636 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:30.636 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:31:30.636 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.372518+0000", 2024-09-14T06:31:30.636 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.327e-06 2024-09-14T06:31:30.637 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:30.637 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:30.637 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:31:30.637 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.419530+0000", 2024-09-14T06:31:30.637 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047011798 2024-09-14T06:31:30.637 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:30.637 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:30.637 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:31:30.637 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.419578+0000", 2024-09-14T06:31:30.637 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.8446999999999999e-05 2024-09-14T06:31:30.637 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:30.637 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:30.637 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:31:30.638 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.419639+0000", 2024-09-14T06:31:30.638 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.0900999999999998e-05 2024-09-14T06:31:30.638 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:30.638 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:31:30.638 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:30.638 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:30.638 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:30.638 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:53352 3.f 3:f71f51a7:::rbd_data.112dca2f6ae1.0000000000000015:17 [read 1351680~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:31:30.638 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:44.372525+0000", 2024-09-14T06:31:30.638 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 106.188322508, 2024-09-14T06:31:30.638 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047259847000000001, 2024-09-14T06:31:30.638 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:31:30.638 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:31:30.638 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:31:30.638 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:31:30.639 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:31:30.639 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:30.639 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 53352 2024-09-14T06:31:30.639 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:30.639 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:31:30.639 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:30.639 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:31:30.639 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.372525+0000", 2024-09-14T06:31:30.639 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:30.639 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:30.639 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:30.639 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:31:30.639 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.372525+0000", 2024-09-14T06:31:30.639 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:30.640 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:30.640 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:30.640 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:31:30.640 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.372525+0000", 2024-09-14T06:31:30.640 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.0600000000000001e-07 2024-09-14T06:31:30.640 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:30.640 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:30.640 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:31:30.640 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.372527+0000", 2024-09-14T06:31:30.640 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.0499999999999999e-06 2024-09-14T06:31:30.640 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:30.640 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:30.640 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:31:30.640 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.372533+0000", 2024-09-14T06:31:30.641 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.4320000000000004e-06 2024-09-14T06:31:30.641 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:30.641 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:30.641 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:31:30.641 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.372538+0000", 2024-09-14T06:31:30.641 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.9130000000000002e-06 2024-09-14T06:31:30.641 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:30.641 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:30.641 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:31:30.641 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.419667+0000", 2024-09-14T06:31:30.641 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047128976000000003 2024-09-14T06:31:30.641 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:30.641 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:30.641 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:31:30.641 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.419715+0000", 2024-09-14T06:31:30.642 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.7868999999999999e-05 2024-09-14T06:31:30.642 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:30.642 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:30.642 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:31:30.642 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.419784+0000", 2024-09-14T06:31:30.642 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.9201000000000004e-05 2024-09-14T06:31:30.642 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:30.642 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:31:30.642 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:30.643 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:30.643 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:30.643 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:53353 3.f 3:f71f51a7:::rbd_data.112dca2f6ae1.0000000000000015:17 [read 1376256~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:31:30.643 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:44.372545+0000", 2024-09-14T06:31:30.643 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 106.188302468, 2024-09-14T06:31:30.643 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047364232999999999, 2024-09-14T06:31:30.643 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:31:30.643 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:31:30.643 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:31:30.643 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:31:30.643 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:31:30.643 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:30.643 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 53353 2024-09-14T06:31:30.644 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:30.644 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:31:30.644 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:30.644 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:31:30.644 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.372545+0000", 2024-09-14T06:31:30.644 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:30.644 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:30.644 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:30.644 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:31:30.644 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.372545+0000", 2024-09-14T06:31:30.644 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:30.644 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:30.644 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:30.644 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:31:30.644 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.372545+0000", 2024-09-14T06:31:30.645 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.27e-07 2024-09-14T06:31:30.645 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:30.645 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:30.645 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:31:30.645 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.372547+0000", 2024-09-14T06:31:30.645 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.0600000000000002e-06 2024-09-14T06:31:30.645 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:30.645 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:30.645 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:31:30.645 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.372553+0000", 2024-09-14T06:31:30.645 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.7980000000000002e-06 2024-09-14T06:31:30.645 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:30.645 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:30.646 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:31:30.646 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.372557+0000", 2024-09-14T06:31:30.646 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.7389999999999999e-06 2024-09-14T06:31:30.646 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:30.646 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:30.646 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:31:30.646 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.419811+0000", 2024-09-14T06:31:30.646 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047253310999999999 2024-09-14T06:31:30.646 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:30.646 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:30.646 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:31:30.647 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.419854+0000", 2024-09-14T06:31:30.647 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.3448000000000001e-05 2024-09-14T06:31:30.647 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:30.647 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:30.647 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:31:30.647 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.419909+0000", 2024-09-14T06:31:30.647 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.4549999999999998e-05 2024-09-14T06:31:30.647 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:30.647 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:31:30.647 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:30.647 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:30.647 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:30.647 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:53354 3.f 3:f71f51a7:::rbd_data.112dca2f6ae1.0000000000000015:17 [read 1400832~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:31:30.647 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:44.372563+0000", 2024-09-14T06:31:30.647 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 106.18828403099999, 2024-09-14T06:31:30.648 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047482233999999998, 2024-09-14T06:31:30.648 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:31:30.648 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:31:30.648 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:31:30.648 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:31:30.648 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:31:30.648 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:30.648 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 53354 2024-09-14T06:31:30.648 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:30.648 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:31:30.648 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:30.648 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:31:30.648 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.372563+0000", 2024-09-14T06:31:30.649 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:30.649 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:30.649 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:30.649 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:31:30.649 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.372563+0000", 2024-09-14T06:31:30.649 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:30.649 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:30.649 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:30.649 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:31:30.649 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.375009+0000", 2024-09-14T06:31:30.649 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.0024457680000000001 2024-09-14T06:31:30.649 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:30.649 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:30.650 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:31:30.650 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.375014+0000", 2024-09-14T06:31:30.650 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.0599999999999998e-06 2024-09-14T06:31:30.650 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:30.650 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:30.650 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:31:30.650 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.375026+0000", 2024-09-14T06:31:30.650 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.1783999999999999e-05 2024-09-14T06:31:30.650 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:30.650 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:30.650 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:31:30.650 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.375035+0000", 2024-09-14T06:31:30.650 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.7799999999999995e-06 2024-09-14T06:31:30.651 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:30.651 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:30.651 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:31:30.651 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.419942+0000", 2024-09-14T06:31:30.651 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.044906982999999998 2024-09-14T06:31:30.651 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:30.651 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:30.651 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:31:30.651 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.419989+0000", 2024-09-14T06:31:30.651 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.7104999999999997e-05 2024-09-14T06:31:30.651 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:30.651 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:30.651 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:31:30.651 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.420045+0000", 2024-09-14T06:31:30.651 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.5754000000000002e-05 2024-09-14T06:31:30.652 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:30.652 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:31:30.652 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:30.652 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:30.652 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:31:30.652 INFO:teuthology.orchestra.run.smithi138.stdout:} 2024-09-14T06:31:30.653 INFO:teuthology.orchestra.run.smithi094.stderr:noscrub is unset 2024-09-14T06:31:30.654 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-14T06:31:30.695 INFO:tasks.ceph.osd.3.smithi094.stderr:2024-09-14T06:31:30.693+0000 7feeeafe8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33655) UID: 0 2024-09-14T06:31:30.729 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-14T06:31:30.729 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-14T06:31:30.729 INFO:tasks.thrashosds.thrasher:Removing osd 3, in_osds are: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-14T06:31:30.730 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd out 3 2024-09-14T06:31:30.760 INFO:tasks.ceph.osd.5.smithi138.stderr:2024-09-14T06:31:30.759+0000 7fe13746f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33618) UID: 0 2024-09-14T06:31:30.861 INFO:tasks.ceph.osd.6.smithi138.stderr:2024-09-14T06:31:30.859+0000 7f8cd8849640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33620) UID: 0 2024-09-14T06:31:30.863 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-14T06:31:30.863 INFO:teuthology.orchestra.run.smithi094.stdout: "ops": [ 2024-09-14T06:31:30.863 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:30.863 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:206184 3.6 3:67bb8101:::rbd_data.112dcb5c6fa8.0000000000000003:4f [read 2646016~8192 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e166)", 2024-09-14T06:31:30.863 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:31:30.862214+0000", 2024-09-14T06:31:30.864 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 0.00019388200000000001, 2024-09-14T06:31:30.864 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00021952899999999999, 2024-09-14T06:31:30.864 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:31:30.864 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:31:30.864 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "reached pg", 2024-09-14T06:31:30.864 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:31:30.864 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:31:30.864 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:30.864 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 206184 2024-09-14T06:31:30.864 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:30.864 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:31:30.865 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:30.865 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:31:30.865 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:30.862214+0000", 2024-09-14T06:31:30.865 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:30.865 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:30.865 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:30.865 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:31:30.865 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:30.862214+0000", 2024-09-14T06:31:30.865 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:30.865 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:30.865 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:30.865 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:31:30.865 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:30.862215+0000", 2024-09-14T06:31:30.865 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.0109999999999999e-06 2024-09-14T06:31:30.866 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:30.866 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:30.866 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:31:30.866 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:30.862220+0000", 2024-09-14T06:31:30.866 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.0019999999999997e-06 2024-09-14T06:31:30.866 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:30.866 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:30.866 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:31:30.866 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:30.862236+0000", 2024-09-14T06:31:30.866 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.5671999999999998e-05 2024-09-14T06:31:30.866 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:30.866 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:30.866 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:31:30.866 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:30.862244+0000", 2024-09-14T06:31:30.867 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.9489999999999996e-06 2024-09-14T06:31:30.867 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:30.867 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:30.867 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:31:30.867 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:30.862403+0000", 2024-09-14T06:31:30.867 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00015964500000000001 2024-09-14T06:31:30.867 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:30.867 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:31:30.867 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:30.867 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:30.867 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:30.867 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:206185 3.6 3.8081dde6 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e166)", 2024-09-14T06:31:30.867 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:31:30.862260+0000", 2024-09-14T06:31:30.867 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 0.00014755200000000001, 2024-09-14T06:31:30.868 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000232816, 2024-09-14T06:31:30.868 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:31:30.868 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:31:30.868 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "queued for pg", 2024-09-14T06:31:30.868 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:31:30.868 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:31:30.868 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:30.868 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 206185 2024-09-14T06:31:30.868 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:30.868 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:31:30.868 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:30.868 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:31:30.868 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:30.862260+0000", 2024-09-14T06:31:30.868 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:30.869 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:30.869 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:30.869 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:31:30.869 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:30.862260+0000", 2024-09-14T06:31:30.869 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:30.869 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:30.869 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:30.869 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:31:30.869 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:30.862261+0000", 2024-09-14T06:31:30.869 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.7300000000000001e-07 2024-09-14T06:31:30.869 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:30.869 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:30.869 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:31:30.869 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:30.862263+0000", 2024-09-14T06:31:30.870 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.509e-06 2024-09-14T06:31:30.870 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:30.870 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:30.870 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:31:30.870 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:30.862272+0000", 2024-09-14T06:31:30.870 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.6279999999999994e-06 2024-09-14T06:31:30.870 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:30.870 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:30.870 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:31:30.870 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:30.862278+0000", 2024-09-14T06:31:30.870 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.8490000000000002e-06 2024-09-14T06:31:30.870 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:30.870 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:31:30.871 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:30.871 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:30.871 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:30.871 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:206186 3.6 3.8081dde6 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e166)", 2024-09-14T06:31:30.871 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:31:30.862285+0000", 2024-09-14T06:31:30.871 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 0.000122822, 2024-09-14T06:31:30.871 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00025516699999999999, 2024-09-14T06:31:30.871 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:31:30.871 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:31:30.871 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "queued for pg", 2024-09-14T06:31:30.871 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:31:30.871 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:31:30.871 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:30.872 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 206186 2024-09-14T06:31:30.872 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:30.872 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:31:30.872 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:30.872 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:31:30.872 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:30.862285+0000", 2024-09-14T06:31:30.872 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:30.872 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:30.872 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:30.872 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:31:30.872 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:30.862285+0000", 2024-09-14T06:31:30.872 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:30.872 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:30.872 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:30.873 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:31:30.873 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:30.862285+0000", 2024-09-14T06:31:30.873 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.4200000000000001e-07 2024-09-14T06:31:30.873 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:30.873 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:30.873 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:31:30.873 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:30.862288+0000", 2024-09-14T06:31:30.873 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.181e-06 2024-09-14T06:31:30.873 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:30.873 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:30.873 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:31:30.873 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:30.862293+0000", 2024-09-14T06:31:30.873 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.7899999999999996e-06 2024-09-14T06:31:30.873 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:30.873 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:30.874 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:31:30.874 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:30.862298+0000", 2024-09-14T06:31:30.874 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.8080000000000004e-06 2024-09-14T06:31:30.874 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:30.874 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:31:30.874 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:30.874 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:30.874 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:30.874 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:206187 3.6 3.8081dde6 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e166)", 2024-09-14T06:31:30.874 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:31:30.862304+0000", 2024-09-14T06:31:30.874 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 0.00010380200000000001, 2024-09-14T06:31:30.874 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000293126, 2024-09-14T06:31:30.874 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:31:30.874 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:31:30.875 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "queued for pg", 2024-09-14T06:31:30.875 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:31:30.875 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:31:30.875 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:30.875 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 206187 2024-09-14T06:31:30.875 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:30.875 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:31:30.875 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:30.875 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:31:30.875 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:30.862304+0000", 2024-09-14T06:31:30.875 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:30.875 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:30.875 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:30.875 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:31:30.876 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:30.862304+0000", 2024-09-14T06:31:30.876 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:30.876 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:30.876 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:30.876 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:31:30.876 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:30.862304+0000", 2024-09-14T06:31:30.876 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.41e-07 2024-09-14T06:31:30.876 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:30.876 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:30.876 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:31:30.876 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:30.862306+0000", 2024-09-14T06:31:30.876 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.72e-06 2024-09-14T06:31:30.876 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:30.876 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:30.877 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:31:30.877 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:30.862313+0000", 2024-09-14T06:31:30.877 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.5860000000000001e-06 2024-09-14T06:31:30.877 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:30.877 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:30.877 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:31:30.877 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:30.862318+0000", 2024-09-14T06:31:30.877 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.7929999999999997e-06 2024-09-14T06:31:30.877 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:30.877 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:31:30.877 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:30.880 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:30.880 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:30.880 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:206188 3.6 3.8081dde6 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e166)", 2024-09-14T06:31:30.880 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:31:30.862323+0000", 2024-09-14T06:31:30.880 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 8.4648000000000007e-05, 2024-09-14T06:31:30.880 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00032988199999999998, 2024-09-14T06:31:30.880 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:31:30.880 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:31:30.880 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "queued for pg", 2024-09-14T06:31:30.881 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:31:30.881 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:31:30.881 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:30.881 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 206188 2024-09-14T06:31:30.881 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:30.881 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:31:30.881 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:30.881 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:31:30.881 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:30.862323+0000", 2024-09-14T06:31:30.881 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:30.881 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:30.881 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:30.881 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:31:30.882 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:30.862323+0000", 2024-09-14T06:31:30.882 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:30.882 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:30.882 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:30.882 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:31:30.882 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:30.862324+0000", 2024-09-14T06:31:30.882 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.3500000000000002e-07 2024-09-14T06:31:30.882 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:30.882 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:30.882 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:31:30.882 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:30.862326+0000", 2024-09-14T06:31:30.882 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.012e-06 2024-09-14T06:31:30.882 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:30.882 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:30.883 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:31:30.883 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:30.862331+0000", 2024-09-14T06:31:30.883 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.5860000000000004e-06 2024-09-14T06:31:30.883 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:30.883 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:30.883 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:31:30.883 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:30.862336+0000", 2024-09-14T06:31:30.883 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.7520000000000004e-06 2024-09-14T06:31:30.883 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:30.883 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:31:30.883 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:30.883 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:30.883 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:30.883 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:206189 3.6 3.8081dde6 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e166)", 2024-09-14T06:31:30.883 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:31:30.862341+0000", 2024-09-14T06:31:30.884 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 6.6458000000000006e-05, 2024-09-14T06:31:30.884 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00036062799999999999, 2024-09-14T06:31:30.884 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:31:30.884 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:31:30.884 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "queued for pg", 2024-09-14T06:31:30.884 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:31:30.884 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:31:30.884 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:30.884 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 206189 2024-09-14T06:31:30.884 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:30.884 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:31:30.884 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:30.884 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:31:30.884 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:30.862341+0000", 2024-09-14T06:31:30.884 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:30.885 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:30.885 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:30.885 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:31:30.885 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:30.862341+0000", 2024-09-14T06:31:30.885 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:30.885 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:30.885 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:30.885 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:31:30.885 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:30.862342+0000", 2024-09-14T06:31:30.885 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.2000000000000001e-07 2024-09-14T06:31:30.885 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:30.885 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:30.885 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:31:30.885 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:30.862344+0000", 2024-09-14T06:31:30.885 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.9429999999999999e-06 2024-09-14T06:31:30.886 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:30.886 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:30.886 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:31:30.886 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:30.862349+0000", 2024-09-14T06:31:30.886 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.7819999999999999e-06 2024-09-14T06:31:30.886 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:30.886 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:30.886 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:31:30.886 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:30.862354+0000", 2024-09-14T06:31:30.886 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.7890000000000002e-06 2024-09-14T06:31:30.886 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:30.886 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:31:30.886 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:30.886 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:30.886 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-09-14T06:31:30.887 INFO:teuthology.orchestra.run.smithi094.stdout: "num_ops": 6 2024-09-14T06:31:30.887 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-14T06:31:30.888 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-14T06:31:30.962 INFO:tasks.ceph.osd.6.smithi138.stderr:2024-09-14T06:31:30.961+0000 7f8cd8849640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33620) UID: 0 2024-09-14T06:31:31.045 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-14T06:31:31.045 INFO:teuthology.orchestra.run.smithi094.stdout: "ops": [], 2024-09-14T06:31:31.045 INFO:teuthology.orchestra.run.smithi094.stdout: "complaint_time": 30, 2024-09-14T06:31:31.046 INFO:teuthology.orchestra.run.smithi094.stdout: "num_blocked_ops": 0 2024-09-14T06:31:31.046 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-14T06:31:31.055 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-14T06:31:31.098 INFO:tasks.ceph.osd.0.smithi094.stderr:2024-09-14T06:31:31.096+0000 7f07c9333640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33636) UID: 0 2024-09-14T06:31:31.160 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-14T06:31:31.160 INFO:teuthology.orchestra.run.smithi094.stdout: "size": 20, 2024-09-14T06:31:31.160 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 600, 2024-09-14T06:31:31.160 INFO:teuthology.orchestra.run.smithi094.stdout: "ops": [ 2024-09-14T06:31:31.160 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:31.160 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:115193 3.1 3:856be886:::rbd_data.112d4d24c558.0000000000000004:29 [read 1523712~16384 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e96)", 2024-09-14T06:31:31.160 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:30:17.939854+0000", 2024-09-14T06:31:31.160 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 73.218191180999995, 2024-09-14T06:31:31.161 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.073897516999999996, 2024-09-14T06:31:31.161 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:31:31.161 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:31:31.161 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:31:31.161 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:31:31.161 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:31:31.161 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:31.161 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 115193 2024-09-14T06:31:31.161 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:31.161 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:31:31.161 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:31.161 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:31:31.162 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.939854+0000", 2024-09-14T06:31:31.162 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:31.162 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:31.162 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:31.162 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:31:31.162 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.939854+0000", 2024-09-14T06:31:31.162 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:31.162 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:31.162 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:31.162 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:31:31.162 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.939854+0000", 2024-09-14T06:31:31.162 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.9500000000000002e-07 2024-09-14T06:31:31.163 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:31.163 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:31.163 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:31:31.163 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.939856+0000", 2024-09-14T06:31:31.163 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.0959999999999999e-06 2024-09-14T06:31:31.163 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:31.163 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:31.163 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:31:31.163 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.939863+0000", 2024-09-14T06:31:31.163 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.1129999999999997e-06 2024-09-14T06:31:31.163 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:31.164 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:31.164 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:31:31.164 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.939867+0000", 2024-09-14T06:31:31.164 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.8350000000000003e-06 2024-09-14T06:31:31.164 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:31.164 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:31.164 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:31:31.164 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.013626+0000", 2024-09-14T06:31:31.164 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.073758799999999999 2024-09-14T06:31:31.164 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:31.164 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:31.164 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:31:31.165 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.013690+0000", 2024-09-14T06:31:31.165 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.3918999999999996e-05 2024-09-14T06:31:31.165 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:31.165 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:31.165 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:31:31.165 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.013751+0000", 2024-09-14T06:31:31.165 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.1159000000000002e-05 2024-09-14T06:31:31.165 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:31.165 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:31:31.165 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:31.165 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:31.166 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:31.166 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:115194 3.1 3:856be886:::rbd_data.112d4d24c558.0000000000000004:29 [read 1769472~16384 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e96)", 2024-09-14T06:31:31.166 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:30:17.939874+0000", 2024-09-14T06:31:31.166 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 73.218170869999994, 2024-09-14T06:31:31.166 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.074016988000000006, 2024-09-14T06:31:31.166 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:31:31.166 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:31:31.166 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:31:31.166 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:31:31.166 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:31:31.166 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:31.166 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 115194 2024-09-14T06:31:31.167 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:31.167 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:31:31.167 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:31.167 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:31:31.167 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.939874+0000", 2024-09-14T06:31:31.167 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:31.167 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:31.167 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:31.167 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:31:31.167 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.939874+0000", 2024-09-14T06:31:31.167 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:31.168 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:31.168 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:31.168 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:31:31.168 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.939875+0000", 2024-09-14T06:31:31.168 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.2600000000000002e-07 2024-09-14T06:31:31.168 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:31.168 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:31.168 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:31:31.168 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.939877+0000", 2024-09-14T06:31:31.168 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.9420000000000002e-06 2024-09-14T06:31:31.168 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:31.169 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:31.169 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:31:31.169 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.939882+0000", 2024-09-14T06:31:31.169 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.8749999999999997e-06 2024-09-14T06:31:31.169 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:31.169 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:31.169 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:31:31.169 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.939887+0000", 2024-09-14T06:31:31.169 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.741e-06 2024-09-14T06:31:31.169 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:31.169 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:31.170 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:31:31.170 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.013767+0000", 2024-09-14T06:31:31.170 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.073879720999999995 2024-09-14T06:31:31.170 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:31.170 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:31.170 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:31:31.170 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.013832+0000", 2024-09-14T06:31:31.170 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.5312000000000001e-05 2024-09-14T06:31:31.170 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:31.170 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:31.170 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:31:31.171 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.013891+0000", 2024-09-14T06:31:31.171 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.8870999999999999e-05 2024-09-14T06:31:31.171 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:31.171 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:31:31.171 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:31.171 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:31.171 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:31.171 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:115195 3.1 3:856be886:::rbd_data.112d4d24c558.0000000000000004:29 [read 2015232~16384 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e96)", 2024-09-14T06:31:31.171 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:30:17.939903+0000", 2024-09-14T06:31:31.171 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 73.218141685000006, 2024-09-14T06:31:31.171 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.074128199000000006, 2024-09-14T06:31:31.172 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:31:31.172 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:31:31.172 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:31:31.172 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:31:31.172 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:31:31.172 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:31.172 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 115195 2024-09-14T06:31:31.172 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:31.172 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:31:31.172 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:31.172 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:31:31.172 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.939903+0000", 2024-09-14T06:31:31.173 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:31.173 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:31.173 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:31.173 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:31:31.173 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.939903+0000", 2024-09-14T06:31:31.173 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:31.173 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:31.173 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:31.173 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:31:31.173 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.939904+0000", 2024-09-14T06:31:31.173 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.5799999999999999e-07 2024-09-14T06:31:31.173 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:31.174 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:31.174 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:31:31.174 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.939906+0000", 2024-09-14T06:31:31.174 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.2350000000000002e-06 2024-09-14T06:31:31.174 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:31.174 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:31.174 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:31:31.174 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.939913+0000", 2024-09-14T06:31:31.174 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.5649999999999998e-06 2024-09-14T06:31:31.174 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:31.174 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:31.174 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:31:31.175 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.939918+0000", 2024-09-14T06:31:31.175 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.784e-06 2024-09-14T06:31:31.175 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:31.175 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:31.175 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:31:31.175 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.013921+0000", 2024-09-14T06:31:31.175 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.074002911000000005 2024-09-14T06:31:31.175 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:31.175 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:31.175 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:31:31.175 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.013971+0000", 2024-09-14T06:31:31.175 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.9551e-05 2024-09-14T06:31:31.176 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:31.176 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:31.176 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:31:31.176 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.014031+0000", 2024-09-14T06:31:31.176 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.0594999999999998e-05 2024-09-14T06:31:31.176 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:31.176 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:31:31.176 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:31.176 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:31.176 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:31.176 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:115196 3.1 3:856be886:::rbd_data.112d4d24c558.0000000000000004:29 [read 2260992~16384 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e96)", 2024-09-14T06:31:31.177 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:30:17.939925+0000", 2024-09-14T06:31:31.177 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 73.218119493000003, 2024-09-14T06:31:31.177 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.074320137999999994, 2024-09-14T06:31:31.177 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:31:31.177 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:31:31.177 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:31:31.177 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:31:31.177 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:31:31.177 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:31.177 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 115196 2024-09-14T06:31:31.177 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:31.178 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:31:31.178 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:31.178 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:31:31.178 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.939925+0000", 2024-09-14T06:31:31.178 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:31.178 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:31.178 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:31.178 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:31:31.178 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.939925+0000", 2024-09-14T06:31:31.178 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:31.178 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:31.179 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:31.179 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:31:31.179 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.939926+0000", 2024-09-14T06:31:31.179 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.9999999999999998e-07 2024-09-14T06:31:31.179 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:31.179 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:31.179 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:31:31.179 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.939928+0000", 2024-09-14T06:31:31.179 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.9549999999999999e-06 2024-09-14T06:31:31.179 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:31.179 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:31.179 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:31:31.180 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.939934+0000", 2024-09-14T06:31:31.180 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.127e-06 2024-09-14T06:31:31.180 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:31.180 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:31.180 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:31:31.180 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.939939+0000", 2024-09-14T06:31:31.181 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.775e-06 2024-09-14T06:31:31.181 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:31.181 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:31.181 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:31:31.181 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.014075+0000", 2024-09-14T06:31:31.181 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.074136598999999997 2024-09-14T06:31:31.181 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:31.182 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:31.182 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:31:31.182 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.014162+0000", 2024-09-14T06:31:31.182 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.6477999999999994e-05 2024-09-14T06:31:31.182 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:31.182 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:31.182 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:31:31.182 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.014246+0000", 2024-09-14T06:31:31.182 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.3703999999999997e-05 2024-09-14T06:31:31.182 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:31.182 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:31:31.182 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:31.183 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:31.183 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:31.183 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:115197 3.1 3:856be886:::rbd_data.112d4d24c558.0000000000000004:29 [read 2506752~16384 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e96)", 2024-09-14T06:31:31.183 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:30:17.939969+0000", 2024-09-14T06:31:31.183 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 73.218076007999997, 2024-09-14T06:31:31.183 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.074528544000000002, 2024-09-14T06:31:31.183 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:31:31.183 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:31:31.183 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:31:31.183 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:31:31.183 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:31:31.183 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:31.184 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 115197 2024-09-14T06:31:31.184 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:31.184 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:31:31.184 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:31.184 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:31:31.184 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.939969+0000", 2024-09-14T06:31:31.184 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:31.184 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:31.184 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:31.184 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:31:31.184 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.939969+0000", 2024-09-14T06:31:31.185 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:31.185 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:31.185 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:31.185 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:31:31.185 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.939970+0000", 2024-09-14T06:31:31.185 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.3600000000000003e-07 2024-09-14T06:31:31.185 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:31.185 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:31.185 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:31:31.185 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.939972+0000", 2024-09-14T06:31:31.185 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.3180000000000001e-06 2024-09-14T06:31:31.185 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:31.186 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:31.186 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:31:31.186 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.939979+0000", 2024-09-14T06:31:31.186 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.8800000000000002e-06 2024-09-14T06:31:31.186 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:31.186 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:31.186 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:31:31.186 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.939984+0000", 2024-09-14T06:31:31.186 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.3510000000000001e-06 2024-09-14T06:31:31.186 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:31.186 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:31.187 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:31:31.187 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.014302+0000", 2024-09-14T06:31:31.187 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.074317416999999997 2024-09-14T06:31:31.187 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:31.187 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:31.187 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:31:31.187 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.014392+0000", 2024-09-14T06:31:31.187 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.0181999999999998e-05 2024-09-14T06:31:31.187 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:31.187 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:31.187 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:31:31.187 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.014497+0000", 2024-09-14T06:31:31.188 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00010576 2024-09-14T06:31:31.188 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:31.188 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:31:31.188 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:31.188 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:31.188 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:31.188 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:115198 3.1 3:856be886:::rbd_data.112d4d24c558.0000000000000004:29 [read 2752512~16384 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e96)", 2024-09-14T06:31:31.188 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:30:17.940004+0000", 2024-09-14T06:31:31.188 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 73.218040728000005, 2024-09-14T06:31:31.188 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.074736367999999997, 2024-09-14T06:31:31.188 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:31:31.188 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:31:31.189 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:31:31.189 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:31:31.189 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:31:31.189 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:31.189 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 115198 2024-09-14T06:31:31.189 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:31.189 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:31:31.189 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:31.189 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:31:31.189 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940004+0000", 2024-09-14T06:31:31.189 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:31.189 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:31.190 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:31.190 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:31:31.190 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940004+0000", 2024-09-14T06:31:31.190 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:31.190 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:31.190 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:31.190 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:31:31.190 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940005+0000", 2024-09-14T06:31:31.190 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.2200000000000004e-07 2024-09-14T06:31:31.190 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:31.190 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:31.190 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:31:31.191 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940007+0000", 2024-09-14T06:31:31.191 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.322e-06 2024-09-14T06:31:31.191 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:31.191 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:31.191 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:31:31.191 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940014+0000", 2024-09-14T06:31:31.191 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.6220000000000003e-06 2024-09-14T06:31:31.191 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:31.191 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:31.191 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:31:31.191 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940019+0000", 2024-09-14T06:31:31.191 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.6380000000000001e-06 2024-09-14T06:31:31.192 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:31.192 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:31.192 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:31:31.192 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.014562+0000", 2024-09-14T06:31:31.192 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.074542320999999995 2024-09-14T06:31:31.192 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:31.192 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:31.192 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:31:31.192 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.014646+0000", 2024-09-14T06:31:31.192 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.4079999999999995e-05 2024-09-14T06:31:31.192 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:31.193 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:31.193 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:31:31.193 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.014741+0000", 2024-09-14T06:31:31.193 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.4763000000000001e-05 2024-09-14T06:31:31.193 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:31.193 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:31:31.193 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:31.193 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:31.193 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:31.193 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:115199 3.1 3:856be886:::rbd_data.112d4d24c558.0000000000000004:29 [read 2998272~16384 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e96)", 2024-09-14T06:31:31.193 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:30:17.940040+0000", 2024-09-14T06:31:31.194 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 73.218004973000006, 2024-09-14T06:31:31.194 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.074957134999999994, 2024-09-14T06:31:31.194 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:31:31.194 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:31:31.194 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:31:31.194 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:31:31.194 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:31:31.194 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:31.194 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 115199 2024-09-14T06:31:31.194 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:31.195 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:31:31.195 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:31.195 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:31:31.195 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940040+0000", 2024-09-14T06:31:31.195 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:31.195 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:31.195 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:31.195 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:31:31.195 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940040+0000", 2024-09-14T06:31:31.195 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:31.195 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:31.195 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:31.196 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:31:31.196 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940041+0000", 2024-09-14T06:31:31.196 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.4000000000000001e-07 2024-09-14T06:31:31.196 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:31.196 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:31.196 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:31:31.196 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940046+0000", 2024-09-14T06:31:31.196 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.7250000000000002e-06 2024-09-14T06:31:31.196 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:31.196 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:31.196 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:31:31.196 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940053+0000", 2024-09-14T06:31:31.197 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.6109999999999999e-06 2024-09-14T06:31:31.197 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:31.197 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:31.197 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:31:31.197 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940058+0000", 2024-09-14T06:31:31.197 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.1590000000000001e-06 2024-09-14T06:31:31.197 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:31.197 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:31.197 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:31:31.198 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.014768+0000", 2024-09-14T06:31:31.198 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.074709624000000002 2024-09-14T06:31:31.198 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:31.198 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:31.198 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:31:31.198 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.014907+0000", 2024-09-14T06:31:31.198 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000139665 2024-09-14T06:31:31.198 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:31.198 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:31.198 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:31:31.198 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.014997+0000", 2024-09-14T06:31:31.199 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.9710999999999995e-05 2024-09-14T06:31:31.199 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:31.199 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:31:31.199 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:31.199 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:31.199 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:31.199 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:115200 3.1 3:856be886:::rbd_data.112d4d24c558.0000000000000004:29 [read 3244032~16384 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e96)", 2024-09-14T06:31:31.199 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:30:17.940080+0000", 2024-09-14T06:31:31.199 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 73.217964768000002, 2024-09-14T06:31:31.199 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.075137291999999994, 2024-09-14T06:31:31.199 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:31:31.199 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:31:31.199 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:31:31.200 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:31:31.200 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:31:31.200 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:31.200 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 115200 2024-09-14T06:31:31.200 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:31.200 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:31:31.200 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:31.200 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:31:31.200 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940080+0000", 2024-09-14T06:31:31.200 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:31.200 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:31.201 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:31.201 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:31:31.201 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940080+0000", 2024-09-14T06:31:31.201 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:31.201 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:31.201 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:31.201 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:31:31.201 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940081+0000", 2024-09-14T06:31:31.201 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.0800000000000004e-07 2024-09-14T06:31:31.201 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:31.201 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:31.201 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:31:31.202 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940083+0000", 2024-09-14T06:31:31.202 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.3920000000000001e-06 2024-09-14T06:31:31.202 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:31.202 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:31.202 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:31:31.202 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940091+0000", 2024-09-14T06:31:31.202 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.1389999999999995e-06 2024-09-14T06:31:31.202 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:31.202 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:31.202 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:31:31.202 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940097+0000", 2024-09-14T06:31:31.202 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.5550000000000002e-06 2024-09-14T06:31:31.203 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:31.203 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:31.203 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:31:31.203 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.015025+0000", 2024-09-14T06:31:31.203 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.074928579999999995 2024-09-14T06:31:31.203 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:31.203 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:31.203 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:31:31.203 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.015096+0000", 2024-09-14T06:31:31.203 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.0705999999999997e-05 2024-09-14T06:31:31.203 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:31.203 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:31.204 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:31:31.204 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.015217+0000", 2024-09-14T06:31:31.204 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00012131200000000001 2024-09-14T06:31:31.204 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:31.204 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:31:31.204 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:31.204 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:31.204 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:31.204 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:115201 3.1 3:856be886:::rbd_data.112d4d24c558.0000000000000004:29 [read 3489792~16384 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e96)", 2024-09-14T06:31:31.204 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:30:17.940113+0000", 2024-09-14T06:31:31.204 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 73.217931995000001, 2024-09-14T06:31:31.205 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.075246899000000006, 2024-09-14T06:31:31.205 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:31:31.205 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:31:31.205 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:31:31.205 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:31:31.205 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:31:31.205 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:31.205 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 115201 2024-09-14T06:31:31.205 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:31.205 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:31:31.205 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:31.205 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:31:31.206 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940113+0000", 2024-09-14T06:31:31.206 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:31.206 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:31.206 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:31.206 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:31:31.206 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940113+0000", 2024-09-14T06:31:31.206 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:31.206 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:31.206 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:31.206 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:31:31.206 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940114+0000", 2024-09-14T06:31:31.206 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.1500000000000004e-07 2024-09-14T06:31:31.207 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:31.207 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:31.207 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:31:31.207 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940116+0000", 2024-09-14T06:31:31.207 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.2809999999999998e-06 2024-09-14T06:31:31.207 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:31.207 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:31.207 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:31:31.207 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940123+0000", 2024-09-14T06:31:31.207 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.9229999999999996e-06 2024-09-14T06:31:31.207 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:31.207 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:31.208 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:31:31.208 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940128+0000", 2024-09-14T06:31:31.208 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.7790000000000003e-06 2024-09-14T06:31:31.208 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:31.208 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:31.208 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:31:31.208 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.015253+0000", 2024-09-14T06:31:31.208 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.075125792999999996 2024-09-14T06:31:31.208 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:31.208 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:31.208 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:31:31.209 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.015297+0000", 2024-09-14T06:31:31.209 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.3847000000000002e-05 2024-09-14T06:31:31.209 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:31.209 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:31.209 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:31:31.209 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.015360+0000", 2024-09-14T06:31:31.209 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.2661000000000002e-05 2024-09-14T06:31:31.209 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:31.209 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:31:31.209 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:31.209 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:31.210 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:31.210 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:115202 3.1 3:856be886:::rbd_data.112d4d24c558.0000000000000004:29 [read 3735552~16384 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e96)", 2024-09-14T06:31:31.210 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:30:17.940135+0000", 2024-09-14T06:31:31.210 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 73.217910216000007, 2024-09-14T06:31:31.210 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.075451105000000004, 2024-09-14T06:31:31.210 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:31:31.210 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:31:31.210 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:31:31.210 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:31:31.210 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:31:31.210 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:31.210 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 115202 2024-09-14T06:31:31.211 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:31.211 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:31:31.211 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:31.211 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:31:31.211 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940135+0000", 2024-09-14T06:31:31.211 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:31.211 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:31.211 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:31.211 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:31:31.211 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940135+0000", 2024-09-14T06:31:31.211 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:31.211 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:31.212 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:31.212 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:31:31.212 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940135+0000", 2024-09-14T06:31:31.212 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.0999999999999999e-07 2024-09-14T06:31:31.212 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:31.212 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:31.212 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:31:31.212 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940137+0000", 2024-09-14T06:31:31.212 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.0020000000000001e-06 2024-09-14T06:31:31.212 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:31.212 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:31.213 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:31:31.213 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940143+0000", 2024-09-14T06:31:31.213 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.9850000000000003e-06 2024-09-14T06:31:31.213 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:31.213 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:31.213 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:31:31.213 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940148+0000", 2024-09-14T06:31:31.213 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.9880000000000004e-06 2024-09-14T06:31:31.213 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:31.213 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:31.213 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:31:31.214 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.015400+0000", 2024-09-14T06:31:31.214 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.075252186999999998 2024-09-14T06:31:31.214 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:31.214 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:31.214 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:31:31.214 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.015481+0000", 2024-09-14T06:31:31.214 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.0698000000000006e-05 2024-09-14T06:31:31.214 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:31.214 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:31.214 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:31:31.214 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.015586+0000", 2024-09-14T06:31:31.214 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00010483499999999999 2024-09-14T06:31:31.215 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:31.215 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:31:31.215 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:31.215 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:31.215 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:31.215 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:115203 3.1 3:856be886:::rbd_data.112d4d24c558.0000000000000004:29 [read 3981312~16384 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e96)", 2024-09-14T06:31:31.215 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:30:17.940156+0000", 2024-09-14T06:31:31.215 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 73.217888990999995, 2024-09-14T06:31:31.215 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.075620860999999998, 2024-09-14T06:31:31.215 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:31:31.215 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:31:31.215 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:31:31.216 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:31:31.216 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:31:31.216 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:31.216 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 115203 2024-09-14T06:31:31.216 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:31.216 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:31:31.216 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:31.216 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:31:31.216 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940156+0000", 2024-09-14T06:31:31.216 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:31.216 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:31.216 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:31.217 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:31:31.217 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940156+0000", 2024-09-14T06:31:31.217 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:31.217 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:31.217 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:31.217 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:31:31.217 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940156+0000", 2024-09-14T06:31:31.217 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.1500000000000005e-07 2024-09-14T06:31:31.217 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:31.217 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:31.217 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:31:31.218 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940158+0000", 2024-09-14T06:31:31.218 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.967e-06 2024-09-14T06:31:31.218 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:31.218 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:31.218 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:31:31.218 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940165+0000", 2024-09-14T06:31:31.218 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.6939999999999998e-06 2024-09-14T06:31:31.218 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:31.218 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:31.218 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:31:31.218 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940170+0000", 2024-09-14T06:31:31.218 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.7539999999999997e-06 2024-09-14T06:31:31.219 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:31.219 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:31.219 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:31:31.219 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.015633+0000", 2024-09-14T06:31:31.219 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.075463334000000007 2024-09-14T06:31:31.219 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:31.219 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:31.219 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:31:31.219 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.015693+0000", 2024-09-14T06:31:31.219 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.9367000000000003e-05 2024-09-14T06:31:31.219 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:31.219 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:31.220 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:31:31.220 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.015777+0000", 2024-09-14T06:31:31.220 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.4229999999999998e-05 2024-09-14T06:31:31.220 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:31.220 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:31:31.220 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:31.220 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:31.220 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:31.220 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:115204 3.1 3:856be886:::rbd_data.112d4d24c558.0000000000000004:29 [read 4227072~16384 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e96)", 2024-09-14T06:31:31.220 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:30:17.940176+0000", 2024-09-14T06:31:31.220 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 73.217868457999998, 2024-09-14T06:31:31.221 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.075785505000000003, 2024-09-14T06:31:31.221 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:31:31.221 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:31:31.221 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:31:31.221 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:31:31.221 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:31:31.221 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:31.221 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 115204 2024-09-14T06:31:31.221 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:31.221 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:31:31.221 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:31.221 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:31:31.222 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940176+0000", 2024-09-14T06:31:31.222 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:31.222 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:31.222 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:31.222 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:31:31.222 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940176+0000", 2024-09-14T06:31:31.222 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:31.222 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:31.222 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:31.222 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:31:31.222 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940177+0000", 2024-09-14T06:31:31.222 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.7899999999999999e-07 2024-09-14T06:31:31.223 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:31.223 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:31.223 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:31:31.223 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940179+0000", 2024-09-14T06:31:31.223 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.103e-06 2024-09-14T06:31:31.223 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:31.223 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:31.223 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:31:31.223 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940185+0000", 2024-09-14T06:31:31.223 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.7239999999999997e-06 2024-09-14T06:31:31.223 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:31.224 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:31.224 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:31:31.224 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940190+0000", 2024-09-14T06:31:31.224 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.7269999999999998e-06 2024-09-14T06:31:31.224 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:31.224 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:31.224 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:31:31.224 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.015814+0000", 2024-09-14T06:31:31.224 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.075624114000000006 2024-09-14T06:31:31.224 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:31.224 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:31.224 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:31:31.225 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.015862+0000", 2024-09-14T06:31:31.225 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.8770999999999998e-05 2024-09-14T06:31:31.225 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:31.225 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:31.225 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:31:31.225 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.015962+0000", 2024-09-14T06:31:31.225 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.9586999999999994e-05 2024-09-14T06:31:31.225 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:31.225 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:31:31.225 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:31.225 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:31.226 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:31.226 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:115205 3.1 3:856be886:::rbd_data.112d4d24c558.0000000000000004:29 [read 4472832~16384 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e96)", 2024-09-14T06:31:31.226 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:30:17.940197+0000", 2024-09-14T06:31:31.226 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 73.217847749000001, 2024-09-14T06:31:31.226 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.076068920999999998, 2024-09-14T06:31:31.226 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:31:31.226 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:31:31.226 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:31:31.226 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:31:31.226 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:31:31.226 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:31.226 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 115205 2024-09-14T06:31:31.227 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:31.227 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:31:31.227 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:31.227 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:31:31.227 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940197+0000", 2024-09-14T06:31:31.227 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:31.227 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:31.227 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:31.227 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:31:31.227 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940197+0000", 2024-09-14T06:31:31.227 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:31.228 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:31.228 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:31.228 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:31:31.228 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940198+0000", 2024-09-14T06:31:31.228 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.5999999999999999e-07 2024-09-14T06:31:31.228 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:31.228 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:31.228 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:31:31.228 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940200+0000", 2024-09-14T06:31:31.228 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.0899999999999999e-06 2024-09-14T06:31:31.229 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:31.229 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:31.229 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:31:31.229 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940206+0000", 2024-09-14T06:31:31.229 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.8710000000000002e-06 2024-09-14T06:31:31.229 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:31.229 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:31.229 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:31:31.229 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940211+0000", 2024-09-14T06:31:31.229 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.9230000000000001e-06 2024-09-14T06:31:31.229 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:31.229 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:31.230 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:31:31.230 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.016059+0000", 2024-09-14T06:31:31.230 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.075848051999999999 2024-09-14T06:31:31.230 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:31.230 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:31.230 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:31:31.230 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.016151+0000", 2024-09-14T06:31:31.230 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.1982000000000001e-05 2024-09-14T06:31:31.230 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:31.230 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:31.230 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:31:31.231 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.016266+0000", 2024-09-14T06:31:31.231 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000115543 2024-09-14T06:31:31.231 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:31.231 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:31:31.231 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:31.231 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:31.231 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:31.231 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:115206 3.1 3:856be886:::rbd_data.112d4d24c558.0000000000000004:29 [read 4718592~16384 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e96)", 2024-09-14T06:31:31.231 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:30:17.940223+0000", 2024-09-14T06:31:31.231 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 73.217821658999995, 2024-09-14T06:31:31.231 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.076265115999999994, 2024-09-14T06:31:31.231 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:31:31.231 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:31:31.232 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:31:31.232 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:31:31.232 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:31:31.232 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:31.232 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 115206 2024-09-14T06:31:31.232 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:31.232 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:31:31.232 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:31.232 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:31:31.232 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940223+0000", 2024-09-14T06:31:31.232 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:31.233 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:31.233 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:31.233 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:31:31.233 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940223+0000", 2024-09-14T06:31:31.233 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:31.233 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:31.233 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:31.233 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:31:31.233 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940224+0000", 2024-09-14T06:31:31.233 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.0399999999999996e-07 2024-09-14T06:31:31.233 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:31.233 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:31.234 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:31:31.234 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940226+0000", 2024-09-14T06:31:31.234 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.1780000000000002e-06 2024-09-14T06:31:31.234 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:31.234 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:31.234 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:31:31.234 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940232+0000", 2024-09-14T06:31:31.234 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.5010000000000001e-06 2024-09-14T06:31:31.234 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:31.234 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:31.234 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:31:31.235 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940237+0000", 2024-09-14T06:31:31.235 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.0250000000000002e-06 2024-09-14T06:31:31.235 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:31.235 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:31.235 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:31:31.235 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.016305+0000", 2024-09-14T06:31:31.235 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.076067262999999996 2024-09-14T06:31:31.235 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:31.235 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:31.235 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:31:31.235 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.016383+0000", 2024-09-14T06:31:31.235 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.8578000000000006e-05 2024-09-14T06:31:31.236 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:31.236 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:31.236 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:31:31.236 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.016488+0000", 2024-09-14T06:31:31.236 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000105067 2024-09-14T06:31:31.236 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:31.236 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:31:31.236 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:31.236 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:31.236 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:31.236 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:115207 3.1 3:856be886:::rbd_data.112d4d24c558.0000000000000004:29 [read 4964352~16384 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e96)", 2024-09-14T06:31:31.236 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:30:17.940244+0000", 2024-09-14T06:31:31.237 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 73.217800542999996, 2024-09-14T06:31:31.237 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.076371154999999996, 2024-09-14T06:31:31.237 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:31:31.237 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:31:31.237 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:31:31.237 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:31:31.237 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:31:31.237 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:31.237 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 115207 2024-09-14T06:31:31.237 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:31.237 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:31:31.238 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:31.238 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:31:31.238 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940244+0000", 2024-09-14T06:31:31.238 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:31.238 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:31.238 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:31.238 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:31:31.238 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940244+0000", 2024-09-14T06:31:31.238 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:31.238 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:31.238 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:31.238 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:31:31.239 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940245+0000", 2024-09-14T06:31:31.239 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.1200000000000003e-07 2024-09-14T06:31:31.239 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:31.239 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:31.239 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:31:31.239 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940247+0000", 2024-09-14T06:31:31.239 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.0030000000000001e-06 2024-09-14T06:31:31.239 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:31.239 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:31.239 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:31:31.239 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940252+0000", 2024-09-14T06:31:31.240 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.5520000000000004e-06 2024-09-14T06:31:31.240 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:31.240 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:31.240 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:31:31.240 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940257+0000", 2024-09-14T06:31:31.240 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.6759999999999998e-06 2024-09-14T06:31:31.240 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:31.240 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:31.240 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:31:31.240 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.016521+0000", 2024-09-14T06:31:31.240 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.076263499999999998 2024-09-14T06:31:31.240 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:31.241 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:31.241 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:31:31.241 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.016562+0000", 2024-09-14T06:31:31.241 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.0967000000000003e-05 2024-09-14T06:31:31.241 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:31.241 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:31.241 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:31:31.241 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.016616+0000", 2024-09-14T06:31:31.241 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.3944999999999999e-05 2024-09-14T06:31:31.241 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:31.241 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:31:31.242 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:31.242 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:31.242 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:31.242 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:115208 3.1 3:856be886:::rbd_data.112d4d24c558.0000000000000004:29 [read 5210112~16384 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e96)", 2024-09-14T06:31:31.242 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:30:17.940264+0000", 2024-09-14T06:31:31.242 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 73.217780685999998, 2024-09-14T06:31:31.242 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.076558288000000002, 2024-09-14T06:31:31.242 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:31:31.242 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:31:31.242 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:31:31.242 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:31:31.243 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:31:31.243 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:31.243 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 115208 2024-09-14T06:31:31.243 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:31.243 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:31:31.243 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:31.243 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:31:31.243 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940264+0000", 2024-09-14T06:31:31.243 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:31.243 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:31.243 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:31.243 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:31:31.244 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940264+0000", 2024-09-14T06:31:31.244 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:31.244 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:31.244 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:31.244 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:31:31.244 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940265+0000", 2024-09-14T06:31:31.244 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.0500000000000004e-07 2024-09-14T06:31:31.244 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:31.244 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:31.244 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:31:31.244 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940267+0000", 2024-09-14T06:31:31.245 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.9810000000000002e-06 2024-09-14T06:31:31.245 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:31.245 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:31.245 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:31:31.245 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940272+0000", 2024-09-14T06:31:31.245 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.6999999999999996e-06 2024-09-14T06:31:31.245 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:31.245 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:31.245 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:31:31.245 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940277+0000", 2024-09-14T06:31:31.245 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.6480000000000002e-06 2024-09-14T06:31:31.245 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:31.246 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:31.246 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:31:31.246 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.016650+0000", 2024-09-14T06:31:31.246 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.076373089000000005 2024-09-14T06:31:31.246 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:31.246 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:31.246 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:31:31.246 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.016724+0000", 2024-09-14T06:31:31.246 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.4105999999999998e-05 2024-09-14T06:31:31.246 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:31.246 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:31.246 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:31:31.247 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.016823+0000", 2024-09-14T06:31:31.247 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.8258999999999996e-05 2024-09-14T06:31:31.247 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:31.247 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:31:31.247 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:31.247 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:31.247 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:31.247 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:115209 3.1 3:856be886:::rbd_data.112d4d24c558.0000000000000004:29 [read 5455872~16384 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e96)", 2024-09-14T06:31:31.247 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:30:17.940283+0000", 2024-09-14T06:31:31.247 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 73.217761659999994, 2024-09-14T06:31:31.247 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.076724029999999999, 2024-09-14T06:31:31.247 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:31:31.247 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:31:31.248 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:31:31.248 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:31:31.248 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:31:31.248 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:31.248 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 115209 2024-09-14T06:31:31.248 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:31.248 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:31:31.248 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:31.248 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:31:31.248 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940283+0000", 2024-09-14T06:31:31.248 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:31.248 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:31.249 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:31.249 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:31:31.249 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940283+0000", 2024-09-14T06:31:31.249 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:31.249 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:31.249 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:31.249 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:31:31.249 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940284+0000", 2024-09-14T06:31:31.249 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.6700000000000003e-07 2024-09-14T06:31:31.249 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:31.249 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:31.249 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:31:31.250 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940286+0000", 2024-09-14T06:31:31.250 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.9520000000000001e-06 2024-09-14T06:31:31.250 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:31.250 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:31.250 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:31:31.250 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940292+0000", 2024-09-14T06:31:31.250 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.7339999999999996e-06 2024-09-14T06:31:31.250 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:31.250 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:31.250 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:31:31.251 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940296+0000", 2024-09-14T06:31:31.251 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.7700000000000001e-06 2024-09-14T06:31:31.251 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:31.251 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:31.251 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:31:31.251 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.016859+0000", 2024-09-14T06:31:31.251 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.07656317 2024-09-14T06:31:31.251 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:31.251 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:31.251 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:31:31.251 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.016929+0000", 2024-09-14T06:31:31.252 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.9679999999999997e-05 2024-09-14T06:31:31.252 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:31.252 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:31.252 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:31:31.252 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.017007+0000", 2024-09-14T06:31:31.252 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.8157000000000004e-05 2024-09-14T06:31:31.252 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:31.252 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:31:31.252 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:31.252 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:31.252 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:31.253 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:115210 3.1 3:856be886:::rbd_data.112d4d24c558.0000000000000004:29 [read 5701632~16384 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e96)", 2024-09-14T06:31:31.253 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:30:17.940303+0000", 2024-09-14T06:31:31.253 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 73.217742142999995, 2024-09-14T06:31:31.253 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.076821358000000006, 2024-09-14T06:31:31.253 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:31:31.253 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:31:31.253 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:31:31.253 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:31:31.253 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:31:31.253 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:31.253 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 115210 2024-09-14T06:31:31.254 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:31.254 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:31:31.254 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:31.254 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:31:31.254 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940303+0000", 2024-09-14T06:31:31.254 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:31.254 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:31.254 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:31.254 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:31:31.254 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940303+0000", 2024-09-14T06:31:31.254 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:31.254 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:31.255 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:31.255 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:31:31.255 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940303+0000", 2024-09-14T06:31:31.255 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.3000000000000001e-07 2024-09-14T06:31:31.255 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:31.255 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:31.255 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:31:31.255 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940306+0000", 2024-09-14T06:31:31.255 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.255e-06 2024-09-14T06:31:31.255 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:31.255 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:31.255 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:31:31.256 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940312+0000", 2024-09-14T06:31:31.256 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.9710000000000001e-06 2024-09-14T06:31:31.256 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:31.256 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:31.256 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:31:31.256 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940316+0000", 2024-09-14T06:31:31.256 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.5340000000000001e-06 2024-09-14T06:31:31.256 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:31.256 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:31.256 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:31:31.256 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.017033+0000", 2024-09-14T06:31:31.257 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.076717169000000002 2024-09-14T06:31:31.257 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:31.257 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:31.257 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:31:31.257 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.017073+0000", 2024-09-14T06:31:31.257 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.9582000000000002e-05 2024-09-14T06:31:31.257 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:31.257 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:31.257 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:31:31.257 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.017124+0000", 2024-09-14T06:31:31.257 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.1316999999999997e-05 2024-09-14T06:31:31.257 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:31.258 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:31:31.258 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:31.258 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:31.258 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:31.258 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:115219 3.1 3:856be886:::rbd_data.112d4d24c558.0000000000000004:29 [read 65536~16384 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e96)", 2024-09-14T06:31:31.258 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:30:17.940323+0000", 2024-09-14T06:31:31.258 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 73.217722281999997, 2024-09-14T06:31:31.258 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.076915205, 2024-09-14T06:31:31.258 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:31:31.258 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:31:31.258 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:31:31.259 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:31:31.259 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:31:31.259 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:31.259 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 115219 2024-09-14T06:31:31.259 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:31.259 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:31:31.259 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:31.259 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:31:31.259 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940323+0000", 2024-09-14T06:31:31.259 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:31.259 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:31.259 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:31.260 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:31:31.260 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940323+0000", 2024-09-14T06:31:31.260 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:31.260 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:31.260 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:31.260 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:31:31.260 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940323+0000", 2024-09-14T06:31:31.260 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.0800000000000005e-07 2024-09-14T06:31:31.260 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:31.260 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:31.260 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:31:31.260 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940328+0000", 2024-09-14T06:31:31.261 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.887e-06 2024-09-14T06:31:31.261 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:31.261 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:31.261 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:31:31.261 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940334+0000", 2024-09-14T06:31:31.261 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.3269999999999997e-06 2024-09-14T06:31:31.261 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:31.261 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:31.261 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:31:31.261 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940339+0000", 2024-09-14T06:31:31.261 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.7960000000000003e-06 2024-09-14T06:31:31.261 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:31.262 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:31.262 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:31:31.262 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.017148+0000", 2024-09-14T06:31:31.262 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.076809125000000006 2024-09-14T06:31:31.262 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:31.262 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:31.262 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:31:31.262 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.017186+0000", 2024-09-14T06:31:31.262 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.7283000000000002e-05 2024-09-14T06:31:31.262 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:31.262 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:31.262 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:31:31.263 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.017238+0000", 2024-09-14T06:31:31.263 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.2278999999999999e-05 2024-09-14T06:31:31.263 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:31.263 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:31:31.263 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:31.263 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:31.263 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:31.263 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:115220 3.1 3:856be886:::rbd_data.112d4d24c558.0000000000000004:29 [read 311296~16384 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e96)", 2024-09-14T06:31:31.263 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:30:17.940346+0000", 2024-09-14T06:31:31.263 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 73.217699152999998, 2024-09-14T06:31:31.263 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.077033293000000003, 2024-09-14T06:31:31.264 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:31:31.264 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:31:31.264 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:31:31.264 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:31:31.264 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:31:31.264 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:31.264 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 115220 2024-09-14T06:31:31.264 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:31.264 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:31:31.264 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:31.264 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:31:31.264 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940346+0000", 2024-09-14T06:31:31.265 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:31.265 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:31.265 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:31.265 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:31:31.265 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940346+0000", 2024-09-14T06:31:31.265 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:31.265 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:31.265 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:31.265 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:31:31.265 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940346+0000", 2024-09-14T06:31:31.265 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.4300000000000003e-07 2024-09-14T06:31:31.265 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:31.266 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:31.266 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:31:31.266 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940348+0000", 2024-09-14T06:31:31.266 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.0030000000000001e-06 2024-09-14T06:31:31.266 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:31.266 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:31.266 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:31:31.266 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.004655+0000", 2024-09-14T06:31:31.266 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.064306323999999998 2024-09-14T06:31:31.266 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:31.266 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:31.266 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:31:31.267 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.004671+0000", 2024-09-14T06:31:31.267 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.6589999999999999e-05 2024-09-14T06:31:31.267 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:31.267 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:31.267 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:31:31.267 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.017262+0000", 2024-09-14T06:31:31.267 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.012590570000000001 2024-09-14T06:31:31.267 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:31.267 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:31.267 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:31:31.267 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.017321+0000", 2024-09-14T06:31:31.268 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.9210999999999999e-05 2024-09-14T06:31:31.268 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:31.268 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:31.268 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:31:31.268 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.017379+0000", 2024-09-14T06:31:31.268 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.8051999999999999e-05 2024-09-14T06:31:31.268 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:31.268 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:31:31.268 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:31.268 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:31.268 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:31:31.268 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-14T06:31:31.270 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-14T06:31:31.271 INFO:tasks.ceph.mon.a.smithi094.stderr:2024-09-14T06:31:31.167+0000 7f3f97784640 -1 mon.a@0(leader).osd e166 definitely_dead 0 2024-09-14T06:31:31.271 INFO:tasks.ceph.osd.1.smithi094.stderr:2024-09-14T06:31:31.268+0000 7f328e47a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33641) UID: 0 2024-09-14T06:31:31.371 INFO:tasks.ceph.osd.2.smithi094.stderr:2024-09-14T06:31:31.369+0000 7faf3269b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33649) UID: 0 2024-09-14T06:31:31.472 INFO:tasks.ceph.osd.4.smithi138.stderr:2024-09-14T06:31:31.471+0000 7f6c653c8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33615) UID: 0 2024-09-14T06:31:31.475 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-14T06:31:31.476 INFO:teuthology.orchestra.run.smithi094.stdout: "ops": [], 2024-09-14T06:31:31.476 INFO:teuthology.orchestra.run.smithi094.stdout: "num_ops": 0 2024-09-14T06:31:31.476 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-14T06:31:31.488 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-14T06:31:31.573 INFO:tasks.ceph.osd.7.smithi138.stderr:2024-09-14T06:31:31.572+0000 7f3c6a21d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33623) UID: 0 2024-09-14T06:31:31.631 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-14T06:31:31.632 INFO:teuthology.orchestra.run.smithi094.stdout: "ops": [], 2024-09-14T06:31:31.632 INFO:teuthology.orchestra.run.smithi094.stdout: "complaint_time": 30, 2024-09-14T06:31:31.632 INFO:teuthology.orchestra.run.smithi094.stdout: "num_blocked_ops": 0 2024-09-14T06:31:31.632 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-14T06:31:31.642 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-14T06:31:31.646 INFO:teuthology.orchestra.run.smithi094.stderr:marked out osd.3. 2024-09-14T06:31:31.674 INFO:tasks.ceph.osd.4.smithi138.stderr:2024-09-14T06:31:31.673+0000 7f6c653c8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33615) UID: 0 2024-09-14T06:31:31.748 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-14T06:31:31.749 INFO:teuthology.orchestra.run.smithi094.stdout: "size": 20, 2024-09-14T06:31:31.749 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 600, 2024-09-14T06:31:31.749 INFO:teuthology.orchestra.run.smithi094.stdout: "ops": [ 2024-09-14T06:31:31.749 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:31.749 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:14918 3.0 3:05c39535:::rbd_data.112d92027590.0000000000000009:f [read 795648~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:31:31.749 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:26.083834+0000", 2024-09-14T06:31:31.749 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 125.663103632, 2024-09-14T06:31:31.749 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.041776287000000002, 2024-09-14T06:31:31.749 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:31:31.749 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:31:31.749 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:31:31.750 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:31:31.750 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:31:31.750 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:31.750 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 14918 2024-09-14T06:31:31.750 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:31.750 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:31:31.750 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:31.750 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:31:31.750 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.083834+0000", 2024-09-14T06:31:31.750 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:31.750 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:31.751 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:31.751 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:31:31.751 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.083834+0000", 2024-09-14T06:31:31.751 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:31.751 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:31.751 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:31.751 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:31:31.751 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.083834+0000", 2024-09-14T06:31:31.751 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.0100000000000001e-07 2024-09-14T06:31:31.751 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:31.751 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:31.751 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:31:31.752 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.083835+0000", 2024-09-14T06:31:31.752 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.3900000000000003e-07 2024-09-14T06:31:31.752 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:31.752 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:31.752 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:31:31.752 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.083838+0000", 2024-09-14T06:31:31.752 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.9830000000000001e-06 2024-09-14T06:31:31.752 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:31.752 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:31.752 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:31:31.752 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.083840+0000", 2024-09-14T06:31:31.753 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.4679999999999999e-06 2024-09-14T06:31:31.753 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:31.753 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:31.753 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:31:31.753 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.125455+0000", 2024-09-14T06:31:31.753 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.041614578999999999 2024-09-14T06:31:31.753 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:31.753 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:31.753 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:31:31.753 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.125496+0000", 2024-09-14T06:31:31.753 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.1047000000000002e-05 2024-09-14T06:31:31.753 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:31.754 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:31.754 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:31:31.754 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.125610+0000", 2024-09-14T06:31:31.754 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00011407 2024-09-14T06:31:31.754 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:31.754 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:31:31.754 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:31.754 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:31.754 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:31.754 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:14919 3.0 3:05c39535:::rbd_data.112d92027590.0000000000000009:f [read 805888~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:31:31.754 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:26.083843+0000", 2024-09-14T06:31:31.754 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 125.66309388400001, 2024-09-14T06:31:31.754 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.041992356000000002, 2024-09-14T06:31:31.755 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:31:31.755 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:31:31.755 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:31:31.755 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:31:31.755 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:31:31.755 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:31.755 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 14919 2024-09-14T06:31:31.755 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:31.755 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:31:31.755 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:31.755 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:31:31.755 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.083843+0000", 2024-09-14T06:31:31.756 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:31.756 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:31.756 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:31.756 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:31:31.756 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.083843+0000", 2024-09-14T06:31:31.756 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:31.756 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:31.756 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:31.756 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:31:31.756 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.083844+0000", 2024-09-14T06:31:31.756 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.3300000000000001e-07 2024-09-14T06:31:31.756 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:31.757 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:31.757 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:31:31.757 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.083845+0000", 2024-09-14T06:31:31.757 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.4729999999999999e-06 2024-09-14T06:31:31.757 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:31.757 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:31.757 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:31:31.757 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.083850+0000", 2024-09-14T06:31:31.757 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.0130000000000001e-06 2024-09-14T06:31:31.757 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:31.757 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:31.757 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:31:31.758 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.083854+0000", 2024-09-14T06:31:31.758 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.4759999999999998e-06 2024-09-14T06:31:31.758 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:31.758 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:31.758 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:31:31.758 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.125636+0000", 2024-09-14T06:31:31.758 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.041782683000000001 2024-09-14T06:31:31.758 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:31.758 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:31.758 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:31:31.758 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.125680+0000", 2024-09-14T06:31:31.758 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.3614e-05 2024-09-14T06:31:31.759 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:31.759 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:31.759 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:31:31.759 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.125836+0000", 2024-09-14T06:31:31.759 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000155864 2024-09-14T06:31:31.759 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:31.759 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:31:31.759 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:31.759 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:31.759 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:31.759 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:14920 3.0 3:05c39535:::rbd_data.112d92027590.0000000000000009:f [read 816128~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:31:31.759 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:26.083868+0000", 2024-09-14T06:31:31.759 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 125.66306922299999, 2024-09-14T06:31:31.760 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.042166771999999998, 2024-09-14T06:31:31.760 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:31:31.760 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:31:31.760 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:31:31.760 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:31:31.760 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:31:31.760 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:31.760 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 14920 2024-09-14T06:31:31.760 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:31.760 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:31:31.760 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:31.761 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:31:31.761 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.083868+0000", 2024-09-14T06:31:31.761 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:31.761 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:31.761 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:31.761 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:31:31.761 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.083868+0000", 2024-09-14T06:31:31.761 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:31.761 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:31.761 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:31.761 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:31:31.762 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.083869+0000", 2024-09-14T06:31:31.762 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.99e-07 2024-09-14T06:31:31.762 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:31.762 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:31.762 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:31:31.762 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.083870+0000", 2024-09-14T06:31:31.762 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.782e-06 2024-09-14T06:31:31.762 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:31.762 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:31.762 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:31:31.762 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.083876+0000", 2024-09-14T06:31:31.762 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.3809999999999998e-06 2024-09-14T06:31:31.762 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:31.763 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:31.763 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:31:31.763 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.083882+0000", 2024-09-14T06:31:31.763 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.9479999999999996e-06 2024-09-14T06:31:31.763 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:31.763 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:31.763 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:31:31.763 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.125878+0000", 2024-09-14T06:31:31.763 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.041996264999999998 2024-09-14T06:31:31.763 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:31.763 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:31.763 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:31:31.764 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.125929+0000", 2024-09-14T06:31:31.764 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.0498999999999999e-05 2024-09-14T06:31:31.764 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:31.764 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:31.764 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:31:31.764 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.126035+0000", 2024-09-14T06:31:31.764 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000106298 2024-09-14T06:31:31.764 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:31.764 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:31:31.764 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:31.764 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:31.764 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:31.764 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:14921 3.0 3:05c39535:::rbd_data.112d92027590.0000000000000009:f [read 826368~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:31:31.765 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:26.083889+0000", 2024-09-14T06:31:31.765 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 125.663048213, 2024-09-14T06:31:31.765 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.042435309999999997, 2024-09-14T06:31:31.765 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:31:31.765 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:31:31.765 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:31:31.765 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:31:31.765 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:31:31.765 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:31.765 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 14921 2024-09-14T06:31:31.765 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:31.766 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:31:31.766 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:31.766 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:31:31.766 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.083889+0000", 2024-09-14T06:31:31.766 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:31.766 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:31.766 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:31.766 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:31:31.766 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.083889+0000", 2024-09-14T06:31:31.766 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:31.766 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:31.766 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:31.766 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:31:31.767 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.086033+0000", 2024-09-14T06:31:31.767 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0021444179999999999 2024-09-14T06:31:31.767 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:31.767 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:31.767 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:31:31.767 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.086038+0000", 2024-09-14T06:31:31.767 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.0969999999999999e-06 2024-09-14T06:31:31.767 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:31.767 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:31.767 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:31:31.767 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.086046+0000", 2024-09-14T06:31:31.767 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.9440000000000005e-06 2024-09-14T06:31:31.767 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:31.768 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:31.768 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:31:31.768 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.086050+0000", 2024-09-14T06:31:31.768 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.69e-06 2024-09-14T06:31:31.768 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:31.768 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:31.768 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:31:31.768 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.126080+0000", 2024-09-14T06:31:31.768 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.040030237000000003 2024-09-14T06:31:31.768 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:31.768 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:31.769 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:31:31.769 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.126163+0000", 2024-09-14T06:31:31.769 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.2071999999999996e-05 2024-09-14T06:31:31.769 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:31.769 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:31.769 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:31:31.769 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.126324+0000", 2024-09-14T06:31:31.769 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00016185200000000001 2024-09-14T06:31:31.769 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:31.769 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:31:31.770 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:31.770 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:31.770 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:31.770 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:55627 3.0 3:06490baf:::rbd_data.112dca2f6ae1.0000000000000030:17 [read 2002944~4096 [fadvise_nocache]] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:31:31.770 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:44.599966+0000", 2024-09-14T06:31:31.770 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 107.146971265, 2024-09-14T06:31:31.770 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.041868546999999999, 2024-09-14T06:31:31.770 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:31:31.770 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:31:31.770 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:31:31.771 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:31:31.771 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:31:31.771 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:31.771 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 55627 2024-09-14T06:31:31.771 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:31.771 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:31:31.771 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:31.771 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:31:31.771 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.599966+0000", 2024-09-14T06:31:31.771 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:31.771 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:31.771 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:31.771 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:31:31.772 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.599966+0000", 2024-09-14T06:31:31.772 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:31.772 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:31.772 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:31.772 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:31:31.772 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.599966+0000", 2024-09-14T06:31:31.772 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.0900000000000001e-07 2024-09-14T06:31:31.772 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:31.772 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:31.772 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:31:31.772 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.599967+0000", 2024-09-14T06:31:31.772 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.64e-07 2024-09-14T06:31:31.772 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:31.773 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:31.773 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:31:31.773 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.599970+0000", 2024-09-14T06:31:31.773 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.9670000000000001e-06 2024-09-14T06:31:31.773 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:31.773 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:31.773 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:31:31.773 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.599973+0000", 2024-09-14T06:31:31.773 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.632e-06 2024-09-14T06:31:31.773 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:31.773 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:31.773 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:31:31.774 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.641665+0000", 2024-09-14T06:31:31.774 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.041692547000000003 2024-09-14T06:31:31.774 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:31.774 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:31.774 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:31:31.774 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.641740+0000", 2024-09-14T06:31:31.774 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.4559000000000002e-05 2024-09-14T06:31:31.774 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:31.774 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:31.774 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:31:31.774 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.641835+0000", 2024-09-14T06:31:31.774 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.4668999999999998e-05 2024-09-14T06:31:31.774 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:31.775 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:31:31.775 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:31.775 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:31.775 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:31.775 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:55628 3.0 3:06490baf:::rbd_data.112dca2f6ae1.0000000000000030:17 [read 2027520~4096 [fadvise_nocache]] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:31:31.775 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:44.599976+0000", 2024-09-14T06:31:31.775 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 107.14696147799999, 2024-09-14T06:31:31.775 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.041991621, 2024-09-14T06:31:31.775 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:31:31.775 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:31:31.775 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:31:31.775 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:31:31.775 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:31:31.775 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:31.776 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 55628 2024-09-14T06:31:31.776 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:31.776 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:31:31.776 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:31.776 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:31:31.776 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.599976+0000", 2024-09-14T06:31:31.776 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:31.776 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:31.776 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:31.776 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:31:31.776 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.599976+0000", 2024-09-14T06:31:31.777 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:31.777 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:31.777 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:31.777 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:31:31.777 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.599976+0000", 2024-09-14T06:31:31.777 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.1299999999999999e-07 2024-09-14T06:31:31.777 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:31.777 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:31.777 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:31:31.777 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.599977+0000", 2024-09-14T06:31:31.777 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.7999999999999993e-07 2024-09-14T06:31:31.777 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:31.778 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:31.778 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:31:31.778 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.599980+0000", 2024-09-14T06:31:31.778 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.01e-06 2024-09-14T06:31:31.778 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:31.778 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:31.778 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:31:31.778 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.599983+0000", 2024-09-14T06:31:31.778 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.5409999999999999e-06 2024-09-14T06:31:31.778 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:31.778 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:31.779 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:31:31.779 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.641869+0000", 2024-09-14T06:31:31.779 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.041886235000000001 2024-09-14T06:31:31.779 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:31.779 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:31.779 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:31:31.779 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.641913+0000", 2024-09-14T06:31:31.779 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.4357000000000002e-05 2024-09-14T06:31:31.779 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:31.779 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:31.779 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:31:31.779 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.641967+0000", 2024-09-14T06:31:31.779 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.4284999999999999e-05 2024-09-14T06:31:31.780 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:31.780 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:31:31.780 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:31.780 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:31.780 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:31.780 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:55629 3.0 3:06490baf:::rbd_data.112dca2f6ae1.0000000000000030:17 [read 2052096~4096 [fadvise_nocache]] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:31:31.780 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:44.599985+0000", 2024-09-14T06:31:31.780 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 107.14695177199999, 2024-09-14T06:31:31.780 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.042183375000000002, 2024-09-14T06:31:31.780 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:31:31.780 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:31:31.781 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:31:31.781 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:31:31.781 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:31:31.781 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:31.781 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 55629 2024-09-14T06:31:31.781 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:31.781 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:31:31.781 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:31.781 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:31:31.781 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.599985+0000", 2024-09-14T06:31:31.781 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:31.782 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:31.782 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:31.782 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:31:31.782 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.599985+0000", 2024-09-14T06:31:31.782 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:31.782 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:31.782 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:31.782 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:31:31.782 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.599986+0000", 2024-09-14T06:31:31.782 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.05e-07 2024-09-14T06:31:31.782 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:31.782 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:31.782 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:31:31.783 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.599987+0000", 2024-09-14T06:31:31.783 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.8400000000000002e-07 2024-09-14T06:31:31.783 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:31.783 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:31.783 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:31:31.783 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.599990+0000", 2024-09-14T06:31:31.783 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.5640000000000001e-06 2024-09-14T06:31:31.783 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:31.783 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:31.783 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:31:31.783 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.599993+0000", 2024-09-14T06:31:31.783 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.615e-06 2024-09-14T06:31:31.784 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:31.784 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:31.784 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:31:31.784 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.642003+0000", 2024-09-14T06:31:31.784 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.042010345999999997 2024-09-14T06:31:31.784 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:31.784 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:31.784 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:31:31.784 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.642080+0000", 2024-09-14T06:31:31.784 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.6538999999999993e-05 2024-09-14T06:31:31.784 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:31.785 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:31.785 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:31:31.785 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.642169+0000", 2024-09-14T06:31:31.785 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.9122000000000005e-05 2024-09-14T06:31:31.785 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:31.785 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:31:31.785 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:31.785 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:31.785 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:31.785 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:55630 3.0 3:06490baf:::rbd_data.112dca2f6ae1.0000000000000030:17 [read 2076672~4096 [fadvise_nocache]] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:31:31.785 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:44.599996+0000", 2024-09-14T06:31:31.785 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 107.146941294, 2024-09-14T06:31:31.786 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.042299615999999998, 2024-09-14T06:31:31.786 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:31:31.786 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:31:31.786 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:31:31.786 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:31:31.786 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:31:31.786 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:31.786 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 55630 2024-09-14T06:31:31.786 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:31.786 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:31:31.786 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:31.786 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:31:31.787 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.599996+0000", 2024-09-14T06:31:31.787 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:31.787 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:31.787 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:31.787 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:31:31.787 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.599996+0000", 2024-09-14T06:31:31.787 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:31.787 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:31.787 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:31.787 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:31:31.787 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.599996+0000", 2024-09-14T06:31:31.787 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.1500000000000001e-07 2024-09-14T06:31:31.787 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:31.788 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:31.788 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:31:31.788 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.599997+0000", 2024-09-14T06:31:31.788 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.33e-07 2024-09-14T06:31:31.788 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:31.788 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:31.788 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:31:31.788 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600000+0000", 2024-09-14T06:31:31.788 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.0240000000000002e-06 2024-09-14T06:31:31.788 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:31.788 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:31.788 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:31:31.789 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600003+0000", 2024-09-14T06:31:31.789 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.3640000000000002e-06 2024-09-14T06:31:31.789 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:31.789 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:31.789 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:31:31.789 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.642200+0000", 2024-09-14T06:31:31.789 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.042197571000000003 2024-09-14T06:31:31.789 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:31.789 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:31.789 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:31:31.789 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.642243+0000", 2024-09-14T06:31:31.789 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.2765000000000001e-05 2024-09-14T06:31:31.790 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:31.790 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:31.790 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:31:31.790 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.642296+0000", 2024-09-14T06:31:31.790 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.2744000000000002e-05 2024-09-14T06:31:31.790 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:31.790 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:31:31.790 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:31.790 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:31.790 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:31.790 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:55631 3.0 3:06490baf:::rbd_data.112dca2f6ae1.0000000000000030:17 [read 1048576~4096 [fadvise_nocache]] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:31:31.790 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:44.600006+0000", 2024-09-14T06:31:31.791 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 107.146931706, 2024-09-14T06:31:31.791 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.042487905999999999, 2024-09-14T06:31:31.791 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:31:31.791 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:31:31.791 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:31:31.791 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:31:31.791 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:31:31.791 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:31.791 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 55631 2024-09-14T06:31:31.791 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:31.791 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:31:31.791 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:31.792 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:31:31.792 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600006+0000", 2024-09-14T06:31:31.792 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:31.792 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:31.792 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:31.792 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:31:31.792 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600006+0000", 2024-09-14T06:31:31.792 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:31.792 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:31.792 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:31.792 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:31:31.792 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600006+0000", 2024-09-14T06:31:31.792 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.0800000000000001e-07 2024-09-14T06:31:31.793 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:31.793 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:31.793 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:31:31.793 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600007+0000", 2024-09-14T06:31:31.793 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.7199999999999997e-07 2024-09-14T06:31:31.793 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:31.793 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:31.793 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:31:31.793 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600010+0000", 2024-09-14T06:31:31.793 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.0280000000000001e-06 2024-09-14T06:31:31.793 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:31.793 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:31.793 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:31:31.794 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600012+0000", 2024-09-14T06:31:31.794 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.475e-06 2024-09-14T06:31:31.794 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:31.794 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:31.794 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:31:31.794 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.642329+0000", 2024-09-14T06:31:31.794 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.042317157000000001 2024-09-14T06:31:31.794 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:31.794 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:31.794 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:31:31.794 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.642403+0000", 2024-09-14T06:31:31.794 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.3941000000000002e-05 2024-09-14T06:31:31.794 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:31.795 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:31.795 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:31:31.795 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.642493+0000", 2024-09-14T06:31:31.795 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.0124999999999995e-05 2024-09-14T06:31:31.795 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:31.795 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:31:31.795 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:31.795 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:31.795 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:31.795 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:55632 3.0 3:06490baf:::rbd_data.112dca2f6ae1.0000000000000030:17 [read 1073152~4096 [fadvise_nocache]] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:31:31.795 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:44.600015+0000", 2024-09-14T06:31:31.795 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 107.146921953, 2024-09-14T06:31:31.795 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.042604922000000003, 2024-09-14T06:31:31.796 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:31:31.796 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:31:31.796 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:31:31.796 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:31:31.796 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:31:31.796 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:31.796 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 55632 2024-09-14T06:31:31.796 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:31.796 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:31:31.796 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:31.796 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:31:31.796 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600015+0000", 2024-09-14T06:31:31.797 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:31.797 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:31.797 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:31.797 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:31:31.797 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600015+0000", 2024-09-14T06:31:31.797 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:31.797 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:31.797 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:31.797 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:31:31.798 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600016+0000", 2024-09-14T06:31:31.798 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.1e-07 2024-09-14T06:31:31.798 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:31.798 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:31.798 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:31:31.798 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600017+0000", 2024-09-14T06:31:31.798 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.9900000000000009e-07 2024-09-14T06:31:31.798 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:31.798 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:31.798 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:31:31.798 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600020+0000", 2024-09-14T06:31:31.798 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.4170000000000001e-06 2024-09-14T06:31:31.798 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:31.799 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:31.799 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:31:31.799 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600023+0000", 2024-09-14T06:31:31.799 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.588e-06 2024-09-14T06:31:31.799 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:31.799 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:31.799 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:31:31.799 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.642526+0000", 2024-09-14T06:31:31.799 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.042503236999999999 2024-09-14T06:31:31.799 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:31.799 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:31.799 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:31:31.800 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.642568+0000", 2024-09-14T06:31:31.800 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.1925000000000003e-05 2024-09-14T06:31:31.800 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:31.800 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:31.800 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:31:31.800 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.642620+0000", 2024-09-14T06:31:31.800 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.2546000000000001e-05 2024-09-14T06:31:31.800 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:31.800 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:31:31.800 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:31.800 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:31.800 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:31.801 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:55633 3.0 3:06490baf:::rbd_data.112dca2f6ae1.0000000000000030:17 [read 1097728~4096 [fadvise_nocache]] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:31:31.801 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:44.600026+0000", 2024-09-14T06:31:31.801 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 107.146911684, 2024-09-14T06:31:31.801 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.042797474000000002, 2024-09-14T06:31:31.801 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:31:31.801 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:31:31.801 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:31:31.801 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:31:31.801 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:31:31.801 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:31.801 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 55633 2024-09-14T06:31:31.802 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:31.802 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:31:31.802 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:31.802 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:31:31.802 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600026+0000", 2024-09-14T06:31:31.802 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:31.802 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:31.802 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:31.802 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:31:31.802 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600026+0000", 2024-09-14T06:31:31.802 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:31.802 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:31.803 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:31.803 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:31:31.803 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600026+0000", 2024-09-14T06:31:31.803 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.3099999999999999e-07 2024-09-14T06:31:31.803 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:31.803 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:31.803 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:31:31.803 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600029+0000", 2024-09-14T06:31:31.803 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.304e-06 2024-09-14T06:31:31.803 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:31.804 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:31.804 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:31:31.804 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600034+0000", 2024-09-14T06:31:31.804 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.1429999999999997e-06 2024-09-14T06:31:31.804 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:31.804 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:31.804 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:31:31.804 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600038+0000", 2024-09-14T06:31:31.804 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.692e-06 2024-09-14T06:31:31.804 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:31.805 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:31.805 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:31:31.805 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.642654+0000", 2024-09-14T06:31:31.805 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.042615564000000002 2024-09-14T06:31:31.805 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:31.805 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:31.805 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:31:31.805 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.642728+0000", 2024-09-14T06:31:31.805 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.4018999999999997e-05 2024-09-14T06:31:31.805 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:31.805 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:31.806 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:31:31.806 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.642823+0000", 2024-09-14T06:31:31.806 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.5520999999999996e-05 2024-09-14T06:31:31.806 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:31.806 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:31:31.806 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:31.806 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:31.806 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:31.806 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:55634 3.0 3:06490baf:::rbd_data.112dca2f6ae1.0000000000000030:17 [read 1122304~4096 [fadvise_nocache]] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:31:31.806 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:44.600042+0000", 2024-09-14T06:31:31.806 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 107.146895207, 2024-09-14T06:31:31.807 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.042903729000000002, 2024-09-14T06:31:31.807 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:31:31.807 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:31:31.807 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:31:31.807 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:31:31.807 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:31:31.807 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:31.807 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 55634 2024-09-14T06:31:31.807 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:31.807 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:31:31.808 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:31.808 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:31:31.808 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600042+0000", 2024-09-14T06:31:31.808 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:31.808 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:31.808 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:31.808 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:31:31.808 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600042+0000", 2024-09-14T06:31:31.808 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:31.808 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:31.808 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:31.809 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:31:31.809 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600043+0000", 2024-09-14T06:31:31.809 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.7000000000000005e-07 2024-09-14T06:31:31.809 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:31.809 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:31.809 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:31:31.809 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600044+0000", 2024-09-14T06:31:31.809 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.742e-06 2024-09-14T06:31:31.809 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:31.809 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:31.809 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:31:31.810 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600050+0000", 2024-09-14T06:31:31.810 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.0800000000000002e-06 2024-09-14T06:31:31.810 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:31.810 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:31.810 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:31:31.810 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600055+0000", 2024-09-14T06:31:31.810 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.3889999999999998e-06 2024-09-14T06:31:31.810 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:31.810 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:31.810 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:31:31.810 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.642845+0000", 2024-09-14T06:31:31.810 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.042790040000000001 2024-09-14T06:31:31.811 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:31.811 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:31.811 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:31:31.811 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.642888+0000", 2024-09-14T06:31:31.811 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.2790000000000002e-05 2024-09-14T06:31:31.811 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:31.811 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:31.811 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:31:31.811 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.642946+0000", 2024-09-14T06:31:31.811 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.8118000000000002e-05 2024-09-14T06:31:31.811 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:31.812 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:31:31.812 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:31.812 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:31.812 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:31.812 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:55635 3.0 3:06490baf:::rbd_data.112dca2f6ae1.0000000000000030:17 [read 1146880~4096 [fadvise_nocache]] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:31:31.812 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:44.600061+0000", 2024-09-14T06:31:31.812 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 107.146876636, 2024-09-14T06:31:31.812 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.043092695, 2024-09-14T06:31:31.812 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:31:31.812 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:31:31.812 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:31:31.812 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:31:31.812 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:31:31.813 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:31.813 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 55635 2024-09-14T06:31:31.813 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:31.813 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:31:31.813 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:31.813 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:31:31.813 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600061+0000", 2024-09-14T06:31:31.813 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:31.813 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:31.813 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:31.813 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:31:31.813 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600061+0000", 2024-09-14T06:31:31.813 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:31.814 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:31.814 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:31.814 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:31:31.814 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600061+0000", 2024-09-14T06:31:31.814 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.3099999999999999e-07 2024-09-14T06:31:31.814 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:31.814 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:31.814 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:31:31.814 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600063+0000", 2024-09-14T06:31:31.814 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.6220000000000001e-06 2024-09-14T06:31:31.814 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:31.814 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:31.815 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:31:31.815 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600068+0000", 2024-09-14T06:31:31.815 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.4790000000000004e-06 2024-09-14T06:31:31.815 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:31.815 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:31.815 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:31:31.815 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600072+0000", 2024-09-14T06:31:31.815 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.3100000000000002e-06 2024-09-14T06:31:31.815 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:31.815 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:31.815 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:31:31.815 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.642987+0000", 2024-09-14T06:31:31.815 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.042914489 2024-09-14T06:31:31.816 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:31.816 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:31.816 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:31:31.816 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.643062+0000", 2024-09-14T06:31:31.816 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.5011000000000004e-05 2024-09-14T06:31:31.816 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:31.816 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:31.816 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:31:31.816 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.643153+0000", 2024-09-14T06:31:31.816 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.1452999999999993e-05 2024-09-14T06:31:31.816 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:31.816 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:31:31.817 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:31.817 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:31.817 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:31.817 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:55636 3.0 3:06490baf:::rbd_data.112dca2f6ae1.0000000000000030:17 [read 1171456~4096 [fadvise_nocache]] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:31:31.817 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:44.600078+0000", 2024-09-14T06:31:31.817 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 107.146859224, 2024-09-14T06:31:31.817 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.043207966, 2024-09-14T06:31:31.817 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:31:31.817 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:31:31.817 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:31:31.817 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:31:31.817 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:31:31.818 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:31.818 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 55636 2024-09-14T06:31:31.818 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:31.818 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:31:31.818 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:31.818 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:31:31.818 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600078+0000", 2024-09-14T06:31:31.818 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:31.818 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:31.818 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:31.818 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:31:31.818 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600078+0000", 2024-09-14T06:31:31.819 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:31.819 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:31.819 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:31.819 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:31:31.819 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600078+0000", 2024-09-14T06:31:31.819 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.34e-07 2024-09-14T06:31:31.819 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:31.819 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:31.819 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:31:31.819 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600080+0000", 2024-09-14T06:31:31.820 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.641e-06 2024-09-14T06:31:31.820 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:31.820 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:31.820 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:31:31.820 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600086+0000", 2024-09-14T06:31:31.820 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.3860000000000004e-06 2024-09-14T06:31:31.820 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:31.820 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:31.820 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:31:31.820 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600091+0000", 2024-09-14T06:31:31.820 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.5970000000000002e-06 2024-09-14T06:31:31.821 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:31.821 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:31.821 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:31:31.821 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.643184+0000", 2024-09-14T06:31:31.821 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.043092893 2024-09-14T06:31:31.821 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:31.821 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:31.821 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:31:31.821 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.643228+0000", 2024-09-14T06:31:31.821 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.4536999999999997e-05 2024-09-14T06:31:31.821 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:31.821 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:31.822 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:31:31.822 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.643286+0000", 2024-09-14T06:31:31.822 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.7578000000000003e-05 2024-09-14T06:31:31.822 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:31.822 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:31:31.822 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:31.822 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:31.822 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:31.822 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:55637 3.0 3:06490baf:::rbd_data.112dca2f6ae1.0000000000000030:17 [read 1196032~4096 [fadvise_nocache]] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:31:31.822 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:44.600097+0000", 2024-09-14T06:31:31.822 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 107.14684066, 2024-09-14T06:31:31.822 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.043399124999999997, 2024-09-14T06:31:31.822 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:31:31.823 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:31:31.823 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:31:31.823 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:31:31.823 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:31:31.823 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:31.823 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 55637 2024-09-14T06:31:31.823 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:31.823 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:31:31.823 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:31.823 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:31:31.823 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600097+0000", 2024-09-14T06:31:31.823 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:31.823 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:31.824 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:31.824 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:31:31.824 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600097+0000", 2024-09-14T06:31:31.824 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:31.824 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:31.824 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:31.824 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:31:31.824 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.604372+0000", 2024-09-14T06:31:31.824 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0042751489999999998 2024-09-14T06:31:31.824 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:31.824 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:31.824 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:31:31.825 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.604377+0000", 2024-09-14T06:31:31.825 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.7899999999999999e-06 2024-09-14T06:31:31.825 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:31.825 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:31.825 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:31:31.825 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.604389+0000", 2024-09-14T06:31:31.825 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.2197e-05 2024-09-14T06:31:31.825 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:31.825 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:31.826 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:31:31.826 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.604397+0000", 2024-09-14T06:31:31.826 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.8730000000000007e-06 2024-09-14T06:31:31.826 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:31.826 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:31.826 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:31:31.826 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.643324+0000", 2024-09-14T06:31:31.826 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.038926897000000002 2024-09-14T06:31:31.826 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:31.826 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:31.826 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:31:31.826 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.643402+0000", 2024-09-14T06:31:31.826 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.8063000000000001e-05 2024-09-14T06:31:31.827 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:31.827 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:31.827 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:31:31.827 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.643496+0000", 2024-09-14T06:31:31.827 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.4155999999999998e-05 2024-09-14T06:31:31.827 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:31.827 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:31:31.827 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:31.827 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:31.827 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:31.827 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:55657 3.0 3:06490baf:::rbd_data.112dca2f6ae1.0000000000000030:17 [read 1687552~4096 [fadvise_nocache]] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:31:31.827 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:44.604841+0000", 2024-09-14T06:31:31.827 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 107.142095931, 2024-09-14T06:31:31.827 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.041890087999999999, 2024-09-14T06:31:31.828 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:31:31.828 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:31:31.828 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:31:31.828 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:31:31.828 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:31:31.828 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:31.828 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 55657 2024-09-14T06:31:31.828 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:31.828 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:31:31.828 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:31.828 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:31:31.828 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.604841+0000", 2024-09-14T06:31:31.828 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:31.829 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:31.829 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:31.829 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:31:31.829 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.604841+0000", 2024-09-14T06:31:31.829 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:31.829 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:31.829 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:31.829 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:31:31.829 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.604842+0000", 2024-09-14T06:31:31.829 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.0900000000000001e-07 2024-09-14T06:31:31.829 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:31.829 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:31.830 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:31:31.830 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.604846+0000", 2024-09-14T06:31:31.830 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.5759999999999999e-06 2024-09-14T06:31:31.830 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:31.830 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:31.830 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:31:31.830 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.604850+0000", 2024-09-14T06:31:31.830 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.405e-06 2024-09-14T06:31:31.830 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:31.830 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:31.830 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:31:31.830 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.604852+0000", 2024-09-14T06:31:31.831 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.8590000000000001e-06 2024-09-14T06:31:31.831 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:31.831 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:31.831 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:31:31.831 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.646633+0000", 2024-09-14T06:31:31.831 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.041780772000000001 2024-09-14T06:31:31.831 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:31.831 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:31.831 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:31:31.831 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.646677+0000", 2024-09-14T06:31:31.831 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.4115e-05 2024-09-14T06:31:31.831 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:31.831 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:31.832 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:31:31.832 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.646731+0000", 2024-09-14T06:31:31.832 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.4151999999999999e-05 2024-09-14T06:31:31.832 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:31.832 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:31:31.832 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:31.832 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:31.832 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:31.832 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:55658 3.0 3:06490baf:::rbd_data.112dca2f6ae1.0000000000000030:17 [read 1712128~4096 [fadvise_nocache]] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:31:31.832 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:44.604856+0000", 2024-09-14T06:31:31.833 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 107.142081331, 2024-09-14T06:31:31.833 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.041986227000000001, 2024-09-14T06:31:31.833 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:31:31.833 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:31:31.833 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:31:31.833 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:31:31.833 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:31:31.833 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:31.833 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 55658 2024-09-14T06:31:31.833 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:31.833 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:31:31.833 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:31.833 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:31:31.834 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.604856+0000", 2024-09-14T06:31:31.834 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:31.834 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:31.834 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:31.834 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:31:31.834 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.604856+0000", 2024-09-14T06:31:31.834 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:31.834 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:31.834 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:31.834 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:31:31.834 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.604856+0000", 2024-09-14T06:31:31.835 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.23e-07 2024-09-14T06:31:31.835 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:31.835 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:31.835 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:31:31.835 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.604857+0000", 2024-09-14T06:31:31.835 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.9999999999999995e-07 2024-09-14T06:31:31.835 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:31.835 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:31.835 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:31:31.835 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.604861+0000", 2024-09-14T06:31:31.836 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.9890000000000003e-06 2024-09-14T06:31:31.836 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:31.836 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:31.836 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:31:31.836 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.604864+0000", 2024-09-14T06:31:31.836 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.7429999999999998e-06 2024-09-14T06:31:31.836 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:31.836 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:31.836 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:31:31.836 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.646744+0000", 2024-09-14T06:31:31.836 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.041880325000000003 2024-09-14T06:31:31.837 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:31.837 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:31.837 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:31:31.837 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.646784+0000", 2024-09-14T06:31:31.837 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.9604000000000002e-05 2024-09-14T06:31:31.837 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:31.837 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:31.837 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:31:31.837 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.646842+0000", 2024-09-14T06:31:31.837 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.8343e-05 2024-09-14T06:31:31.838 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:31.838 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:31:31.838 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:31.838 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:31.838 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:31.838 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:55659 3.0 3:06490baf:::rbd_data.112dca2f6ae1.0000000000000030:17 [read 1736704~4096 [fadvise_nocache]] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:31:31.838 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:44.604867+0000", 2024-09-14T06:31:31.838 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 107.142070178, 2024-09-14T06:31:31.838 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.042097082000000001, 2024-09-14T06:31:31.838 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:31:31.838 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:31:31.838 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:31:31.839 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:31:31.839 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:31:31.839 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:31.839 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 55659 2024-09-14T06:31:31.839 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:31.839 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:31:31.839 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:31.839 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:31:31.839 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.604867+0000", 2024-09-14T06:31:31.839 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:31.839 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:31.839 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:31.840 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:31:31.840 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.604867+0000", 2024-09-14T06:31:31.840 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:31.840 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:31.840 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:31.840 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:31:31.840 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.604867+0000", 2024-09-14T06:31:31.840 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.2000000000000001e-07 2024-09-14T06:31:31.840 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:31.840 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:31.840 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:31:31.840 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.604868+0000", 2024-09-14T06:31:31.841 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.04e-06 2024-09-14T06:31:31.841 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:31.841 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:31.841 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:31:31.841 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.604871+0000", 2024-09-14T06:31:31.841 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.9689999999999999e-06 2024-09-14T06:31:31.841 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:31.841 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:31.841 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:31:31.841 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.604874+0000", 2024-09-14T06:31:31.841 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.5069999999999999e-06 2024-09-14T06:31:31.841 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:31.841 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:31.841 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:31:31.842 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.646874+0000", 2024-09-14T06:31:31.842 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.042000513000000003 2024-09-14T06:31:31.842 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:31.842 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:31.842 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:31:31.842 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.646914+0000", 2024-09-14T06:31:31.842 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.9443999999999997e-05 2024-09-14T06:31:31.842 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:31.842 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:31.842 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:31:31.842 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.646964+0000", 2024-09-14T06:31:31.842 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.0389000000000002e-05 2024-09-14T06:31:31.843 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:31.843 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:31:31.843 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:31.843 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:31.843 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:31.843 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:55660 3.0 3:06490baf:::rbd_data.112dca2f6ae1.0000000000000030:17 [read 1761280~4096 [fadvise_nocache]] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:31:31.843 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:44.604877+0000", 2024-09-14T06:31:31.843 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 107.142060429, 2024-09-14T06:31:31.843 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.042196349000000001, 2024-09-14T06:31:31.843 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:31:31.843 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:31:31.843 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:31:31.844 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:31:31.844 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:31:31.844 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:31.844 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 55660 2024-09-14T06:31:31.844 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:31.844 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:31:31.844 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:31.844 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:31:31.844 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.604877+0000", 2024-09-14T06:31:31.844 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:31.844 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:31.844 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:31.844 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:31:31.845 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.604877+0000", 2024-09-14T06:31:31.845 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:31.845 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:31.845 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:31.845 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:31:31.845 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.605810+0000", 2024-09-14T06:31:31.845 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00093336899999999997 2024-09-14T06:31:31.845 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:31.845 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:31.845 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:31:31.845 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.605815+0000", 2024-09-14T06:31:31.845 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.6140000000000002e-06 2024-09-14T06:31:31.846 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:31.846 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:31.846 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:31:31.846 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.605825+0000", 2024-09-14T06:31:31.846 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.8889999999999997e-06 2024-09-14T06:31:31.846 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:31.846 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:31.846 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:31:31.846 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.605831+0000", 2024-09-14T06:31:31.846 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.9800000000000003e-06 2024-09-14T06:31:31.846 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:31.846 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:31.847 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:31:31.847 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.646989+0000", 2024-09-14T06:31:31.847 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.041157878000000002 2024-09-14T06:31:31.847 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:31.847 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:31.847 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:31:31.847 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.647026+0000", 2024-09-14T06:31:31.847 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.7305000000000003e-05 2024-09-14T06:31:31.847 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:31.847 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:31.847 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:31:31.847 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.647073+0000", 2024-09-14T06:31:31.848 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.7314000000000001e-05 2024-09-14T06:31:31.848 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:31.848 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:31:31.848 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:31.848 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:31.848 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:31.848 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:55666 3.0 3:06490baf:::rbd_data.112dca2f6ae1.0000000000000030:17 [read 1908736~4096 [fadvise_nocache]] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:31:31.848 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:44.605898+0000", 2024-09-14T06:31:31.848 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 107.141039683, 2024-09-14T06:31:31.848 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.041862887000000001, 2024-09-14T06:31:31.848 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:31:31.848 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:31:31.849 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:31:31.849 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:31:31.849 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:31:31.849 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:31.849 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 55666 2024-09-14T06:31:31.849 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:31.849 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:31:31.849 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:31.849 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:31:31.849 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.605898+0000", 2024-09-14T06:31:31.849 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:31.849 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:31.850 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:31.850 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:31:31.850 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.605898+0000", 2024-09-14T06:31:31.850 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:31.850 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:31.850 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:31.850 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:31:31.850 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.611874+0000", 2024-09-14T06:31:31.850 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0059764049999999997 2024-09-14T06:31:31.850 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:31.850 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:31.850 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:31:31.851 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.611879+0000", 2024-09-14T06:31:31.851 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.8749999999999999e-06 2024-09-14T06:31:31.851 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:31.851 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:31.851 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:31:31.851 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.611887+0000", 2024-09-14T06:31:31.851 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.4109999999999996e-06 2024-09-14T06:31:31.851 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:31.851 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:31.851 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:31:31.851 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.611893+0000", 2024-09-14T06:31:31.851 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.2959999999999998e-06 2024-09-14T06:31:31.852 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:31.852 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:31.852 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:31:31.852 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.647674+0000", 2024-09-14T06:31:31.852 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.035781281999999998 2024-09-14T06:31:31.852 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:31.852 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:31.852 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:31:31.852 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.647711+0000", 2024-09-14T06:31:31.852 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.7434000000000001e-05 2024-09-14T06:31:31.852 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:31.852 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:31.853 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:31:31.853 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.647760+0000", 2024-09-14T06:31:31.853 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.9184000000000002e-05 2024-09-14T06:31:31.853 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:31.853 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:31:31.853 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:31.853 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:31.853 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:31:31.853 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-14T06:31:31.854 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-14T06:31:31.855 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-14T06:31:31.856 INFO:tasks.ceph.osd.2.smithi094.stderr:2024-09-14T06:31:31.853+0000 7faf3269b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33649) UID: 0 2024-09-14T06:31:31.956 INFO:tasks.ceph.osd.4.smithi138.stderr:2024-09-14T06:31:31.955+0000 7f6c653c8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33615) UID: 0 2024-09-14T06:31:32.002 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-14T06:31:32.002 INFO:teuthology.orchestra.run.smithi094.stdout: "ops": [], 2024-09-14T06:31:32.002 INFO:teuthology.orchestra.run.smithi094.stdout: "num_ops": 0 2024-09-14T06:31:32.002 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-14T06:31:32.011 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-14T06:31:32.018 INFO:teuthology.orchestra.run.smithi094.stdout:osd.0: {} 2024-09-14T06:31:32.018 INFO:teuthology.orchestra.run.smithi094.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-14T06:31:32.035 INFO:teuthology.orchestra.run.smithi094.stdout:osd.1: {} 2024-09-14T06:31:32.036 INFO:teuthology.orchestra.run.smithi094.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-14T06:31:32.052 INFO:teuthology.orchestra.run.smithi094.stdout:osd.2: {} 2024-09-14T06:31:32.052 INFO:teuthology.orchestra.run.smithi094.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-14T06:31:32.057 INFO:tasks.ceph.osd.4.smithi138.stderr:2024-09-14T06:31:32.056+0000 7f6c653c8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33615) UID: 0 2024-09-14T06:31:32.069 INFO:teuthology.orchestra.run.smithi094.stdout:osd.3: {} 2024-09-14T06:31:32.069 INFO:teuthology.orchestra.run.smithi094.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-14T06:31:32.086 INFO:teuthology.orchestra.run.smithi094.stdout:osd.4: {} 2024-09-14T06:31:32.086 INFO:teuthology.orchestra.run.smithi094.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-14T06:31:32.104 INFO:teuthology.orchestra.run.smithi094.stdout:osd.5: {} 2024-09-14T06:31:32.104 INFO:teuthology.orchestra.run.smithi094.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-14T06:31:32.116 INFO:teuthology.orchestra.run.smithi094.stdout:ERROR: (22) Invalid argument 2024-09-14T06:31:32.116 INFO:teuthology.orchestra.run.smithi094.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:31:32.121 INFO:teuthology.orchestra.run.smithi094.stdout:osd.6: {} 2024-09-14T06:31:32.121 INFO:teuthology.orchestra.run.smithi094.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-14T06:31:32.126 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-14T06:31:32.138 INFO:teuthology.orchestra.run.smithi094.stdout:osd.7: {} 2024-09-14T06:31:32.139 INFO:teuthology.orchestra.run.smithi094.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-14T06:31:32.158 INFO:tasks.ceph.osd.7.smithi138.stderr:2024-09-14T06:31:32.157+0000 7f3c6a21d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33623) UID: 0 2024-09-14T06:31:32.240 INFO:teuthology.orchestra.run.smithi094.stdout:ERROR: (22) Invalid argument 2024-09-14T06:31:32.240 INFO:teuthology.orchestra.run.smithi094.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:31:32.248 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-14T06:31:32.259 INFO:tasks.ceph.osd.4.smithi138.stderr:2024-09-14T06:31:32.258+0000 7f6c653c8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33615) UID: 0 2024-09-14T06:31:32.351 INFO:teuthology.orchestra.run.smithi094.stdout:ERROR: (22) Invalid argument 2024-09-14T06:31:32.351 INFO:teuthology.orchestra.run.smithi094.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:31:32.359 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-14T06:31:32.360 INFO:tasks.ceph.osd.7.smithi138.stderr:2024-09-14T06:31:32.358+0000 7f3c6a21d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33623) UID: 0 2024-09-14T06:31:32.460 INFO:tasks.ceph.osd.3.smithi094.stderr:2024-09-14T06:31:32.458+0000 7feeeafe8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33655) UID: 0 2024-09-14T06:31:32.462 INFO:teuthology.orchestra.run.smithi094.stdout:ERROR: (22) Invalid argument 2024-09-14T06:31:32.462 INFO:teuthology.orchestra.run.smithi094.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:31:32.471 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-14T06:31:32.561 INFO:tasks.ceph.osd.5.smithi138.stderr:2024-09-14T06:31:32.560+0000 7fe13746f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33618) UID: 0 2024-09-14T06:31:32.615 INFO:teuthology.orchestra.run.smithi094.stdout:ERROR: (22) Invalid argument 2024-09-14T06:31:32.615 INFO:teuthology.orchestra.run.smithi094.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:31:32.624 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-14T06:31:32.654 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-14T06:31:32.698 INFO:tasks.ceph.osd.0.smithi094.stderr:2024-09-14T06:31:32.696+0000 7f07c9333640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33636) UID: 0 2024-09-14T06:31:32.739 INFO:teuthology.orchestra.run.smithi138.stdout:ERROR: (22) Invalid argument 2024-09-14T06:31:32.739 INFO:teuthology.orchestra.run.smithi138.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:31:32.748 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-14T06:31:32.793 INFO:tasks.ceph.osd.7.smithi138.stderr:2024-09-14T06:31:32.792+0000 7f3c6a21d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33623) UID: 0 2024-09-14T06:31:32.850 INFO:teuthology.orchestra.run.smithi138.stdout:ERROR: (22) Invalid argument 2024-09-14T06:31:32.850 INFO:teuthology.orchestra.run.smithi138.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:31:32.859 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-14T06:31:32.863 INFO:tasks.ceph.osd.0.smithi094.stderr:2024-09-14T06:31:32.861+0000 7f07c9333640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33636) UID: 0 2024-09-14T06:31:32.964 INFO:tasks.ceph.osd.4.smithi138.stderr:2024-09-14T06:31:32.963+0000 7f6c653c8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33615) UID: 0 2024-09-14T06:31:33.002 INFO:teuthology.orchestra.run.smithi138.stdout:ERROR: (22) Invalid argument 2024-09-14T06:31:33.002 INFO:teuthology.orchestra.run.smithi138.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:31:33.011 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-14T06:31:33.096 INFO:tasks.ceph.osd.7.smithi138.stderr:2024-09-14T06:31:33.095+0000 7f3c6a21d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33623) UID: 0 2024-09-14T06:31:33.155 INFO:teuthology.orchestra.run.smithi138.stdout:ERROR: (22) Invalid argument 2024-09-14T06:31:33.155 INFO:teuthology.orchestra.run.smithi138.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:31:33.163 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-14T06:31:33.204 INFO:tasks.ceph.osd.7.smithi138.stderr:2024-09-14T06:31:33.203+0000 7f3c6a21d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33623) UID: 0 2024-09-14T06:31:33.267 INFO:tasks.ceph.osd.5.smithi138.stderr:2024-09-14T06:31:33.265+0000 7fe13746f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33618) UID: 0 2024-09-14T06:31:33.306 INFO:teuthology.orchestra.run.smithi138.stdout:ERROR: (22) Invalid argument 2024-09-14T06:31:33.306 INFO:teuthology.orchestra.run.smithi138.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:31:33.314 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-14T06:31:33.367 INFO:tasks.ceph.osd.3.smithi094.stderr:2024-09-14T06:31:33.366+0000 7feeeafe8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33655) UID: 0 2024-09-14T06:31:33.457 INFO:teuthology.orchestra.run.smithi138.stdout:ERROR: (22) Invalid argument 2024-09-14T06:31:33.457 INFO:teuthology.orchestra.run.smithi138.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:31:33.466 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-14T06:31:33.468 INFO:tasks.ceph.osd.6.smithi138.stderr:2024-09-14T06:31:33.467+0000 7f8cd8849640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33620) UID: 0 2024-09-14T06:31:33.568 INFO:tasks.ceph.osd.3.smithi094.stderr:2024-09-14T06:31:33.566+0000 7feeeafe8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33655) UID: 0 2024-09-14T06:31:33.569 INFO:teuthology.orchestra.run.smithi138.stdout:ERROR: (22) Invalid argument 2024-09-14T06:31:33.569 INFO:teuthology.orchestra.run.smithi138.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:31:33.577 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-14T06:31:33.640 INFO:teuthology.orchestra.run.smithi094.stderr:nodeep-scrub is unset 2024-09-14T06:31:33.668 INFO:tasks.ceph.osd.2.smithi094.stderr:2024-09-14T06:31:33.666+0000 7faf3269b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33649) UID: 0 2024-09-14T06:31:33.700 INFO:teuthology.orchestra.run.smithi138.stdout:ERROR: (22) Invalid argument 2024-09-14T06:31:33.700 INFO:teuthology.orchestra.run.smithi138.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:31:33.711 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-14T06:31:33.763 INFO:teuthology.orchestra.run.smithi094.stdout:checking clone #37, image image_client.0-clone37 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent38 2024-09-14T06:31:33.769 INFO:tasks.ceph.osd.5.smithi138.stderr:2024-09-14T06:31:33.768+0000 7fe13746f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33618) UID: 0 2024-09-14T06:31:33.860 INFO:teuthology.orchestra.run.smithi138.stdout:ERROR: (22) Invalid argument 2024-09-14T06:31:33.861 INFO:teuthology.orchestra.run.smithi138.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:31:33.870 INFO:tasks.ceph.osd.3.smithi094.stderr:2024-09-14T06:31:33.868+0000 7feeeafe8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33655) UID: 0 2024-09-14T06:31:33.873 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-14T06:31:33.944 INFO:teuthology.orchestra.run.smithi094.stdout:5070 read 0x9fe811 thru 0xa00824 (0x2014 bytes) 2024-09-14T06:31:33.945 INFO:teuthology.orchestra.run.smithi094.stdout:5071 read 0x3974102 thru 0x397e7d1 (0xa6d0 bytes) 2024-09-14T06:31:33.946 INFO:teuthology.orchestra.run.smithi094.stdout:5072 read 0x2e3cd1a thru 0x2e47e7c (0xb163 bytes) 2024-09-14T06:31:33.958 INFO:teuthology.orchestra.run.smithi094.stdout:5074 trunc from 0x4c15893 to 0xdca929a 2024-09-14T06:31:33.971 INFO:tasks.ceph.osd.6.smithi138.stderr:2024-09-14T06:31:33.970+0000 7f8cd8849640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33620) UID: 0 2024-09-14T06:31:33.977 INFO:teuthology.orchestra.run.smithi094.stdout:5075 read 0x7bbfc9f thru 0x7bcf16d (0xf4cf bytes) 2024-09-14T06:31:33.977 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-14T06:31:33.977 INFO:teuthology.orchestra.run.smithi094.stdout:5077 writesame 0x87ec544 thru 0x87fb3d8 data_size 0x1(0xee95 bytes) 2024-09-14T06:31:34.000 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-14T06:31:34.000 INFO:teuthology.orchestra.run.smithi094.stdout:5081 write 0x5062d63 thru 0x506c438 (0x96d6 bytes) 2024-09-14T06:31:34.012 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-14T06:31:34.012 INFO:teuthology.orchestra.run.smithi094.stdout:5083 write 0x1c39ac3 thru 0x1c4146d (0x79ab bytes) 2024-09-14T06:31:34.016 INFO:teuthology.orchestra.run.smithi094.stdout:5084 write 0xab9a6c8 thru 0xab9ace7 (0x620 bytes) 2024-09-14T06:31:34.020 INFO:teuthology.orchestra.run.smithi094.stdout:5086 punch from 0xabb9f34 to 0xabc886e, (0xe93a bytes) 2024-09-14T06:31:34.036 INFO:teuthology.orchestra.run.smithi094.stdout:5087 trunc from 0xdca929a to 0x8f888e 2024-09-14T06:31:34.059 INFO:teuthology.orchestra.run.smithi138.stdout:ERROR: (22) Invalid argument 2024-09-14T06:31:34.059 INFO:teuthology.orchestra.run.smithi138.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:31:34.072 INFO:tasks.ceph.osd.0.smithi094.stderr:2024-09-14T06:31:34.070+0000 7f07c9333640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33636) UID: 0 2024-09-14T06:31:34.073 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-14T06:31:34.150 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-14T06:31:34.172 INFO:tasks.ceph.osd.4.smithi138.stderr:2024-09-14T06:31:34.171+0000 7f6c653c8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33615) UID: 0 2024-09-14T06:31:34.248 INFO:teuthology.orchestra.run.smithi138.stdout:ERROR: (22) Invalid argument 2024-09-14T06:31:34.248 INFO:teuthology.orchestra.run.smithi138.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:31:34.259 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-14T06:31:34.275 INFO:teuthology.orchestra.run.smithi094.stdout:osd.0: {} 2024-09-14T06:31:34.275 INFO:teuthology.orchestra.run.smithi094.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-14T06:31:34.293 INFO:teuthology.orchestra.run.smithi094.stdout:osd.1: {} 2024-09-14T06:31:34.293 INFO:teuthology.orchestra.run.smithi094.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-14T06:31:34.300 INFO:tasks.ceph.osd.4.smithi138.stderr:2024-09-14T06:31:34.299+0000 7f6c653c8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33615) UID: 0 2024-09-14T06:31:34.313 INFO:teuthology.orchestra.run.smithi094.stdout:osd.2: {} 2024-09-14T06:31:34.313 INFO:teuthology.orchestra.run.smithi094.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-14T06:31:34.329 INFO:teuthology.orchestra.run.smithi094.stdout:osd.3: {} 2024-09-14T06:31:34.330 INFO:teuthology.orchestra.run.smithi094.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-14T06:31:34.346 INFO:teuthology.orchestra.run.smithi094.stdout:osd.4: {} 2024-09-14T06:31:34.346 INFO:teuthology.orchestra.run.smithi094.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-14T06:31:34.363 INFO:teuthology.orchestra.run.smithi094.stdout:osd.5: {} 2024-09-14T06:31:34.363 INFO:teuthology.orchestra.run.smithi094.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-14T06:31:34.374 INFO:tasks.ceph.osd.3.smithi094.stderr:2024-09-14T06:31:34.372+0000 7feeeafe8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33655) UID: 0 2024-09-14T06:31:34.380 INFO:teuthology.orchestra.run.smithi094.stdout:osd.6: {} 2024-09-14T06:31:34.381 INFO:teuthology.orchestra.run.smithi094.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-14T06:31:34.397 INFO:teuthology.orchestra.run.smithi094.stdout:osd.7: {} 2024-09-14T06:31:34.397 INFO:teuthology.orchestra.run.smithi094.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-14T06:31:34.424 INFO:teuthology.orchestra.run.smithi138.stdout:{ 2024-09-14T06:31:34.425 INFO:teuthology.orchestra.run.smithi138.stdout: "size": 20, 2024-09-14T06:31:34.425 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 600, 2024-09-14T06:31:34.425 INFO:teuthology.orchestra.run.smithi138.stdout: "ops": [ 2024-09-14T06:31:34.425 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:34.425 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:53244 3.f 3:f71f51a7:::rbd_data.112dca2f6ae1.0000000000000015:17 [read 1863680~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:31:34.425 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:44.369399+0000", 2024-09-14T06:31:34.425 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 110.053174848, 2024-09-14T06:31:34.425 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047291189999999997, 2024-09-14T06:31:34.425 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:31:34.425 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:31:34.425 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:31:34.426 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:31:34.426 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:31:34.426 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:34.426 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 53244 2024-09-14T06:31:34.426 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:34.426 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:31:34.426 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:34.426 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:31:34.426 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369399+0000", 2024-09-14T06:31:34.426 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:34.426 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:34.426 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:34.427 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:31:34.427 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369399+0000", 2024-09-14T06:31:34.427 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:34.427 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:34.427 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:34.427 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:31:34.427 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369400+0000", 2024-09-14T06:31:34.427 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.0699999999999999e-07 2024-09-14T06:31:34.427 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:34.427 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:34.428 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:31:34.428 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369401+0000", 2024-09-14T06:31:34.428 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.4300000000000001e-06 2024-09-14T06:31:34.428 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:34.428 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:34.428 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:31:34.428 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369404+0000", 2024-09-14T06:31:34.428 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.061e-06 2024-09-14T06:31:34.428 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:34.428 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:34.428 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:31:34.429 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369407+0000", 2024-09-14T06:31:34.429 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.6139999999999999e-06 2024-09-14T06:31:34.429 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:34.429 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:34.429 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:31:34.429 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.416517+0000", 2024-09-14T06:31:34.429 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047110338000000002 2024-09-14T06:31:34.429 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:34.429 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:34.429 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:31:34.429 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.416616+0000", 2024-09-14T06:31:34.430 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.8651000000000001e-05 2024-09-14T06:31:34.430 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:34.430 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:34.430 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:31:34.430 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.416691+0000", 2024-09-14T06:31:34.430 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.4888999999999994e-05 2024-09-14T06:31:34.430 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:34.430 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:31:34.430 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:34.430 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:34.430 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:34.431 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:53245 3.f 3:f71f51a7:::rbd_data.112dca2f6ae1.0000000000000015:17 [read 1888256~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:31:34.431 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:44.369410+0000", 2024-09-14T06:31:34.431 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 110.05316456, 2024-09-14T06:31:34.431 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047399275999999997, 2024-09-14T06:31:34.431 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:31:34.431 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:31:34.431 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:31:34.431 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:31:34.431 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:31:34.431 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:34.432 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 53245 2024-09-14T06:31:34.432 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:34.432 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:31:34.432 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:34.432 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:31:34.432 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369410+0000", 2024-09-14T06:31:34.432 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:34.432 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:34.432 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:34.433 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:31:34.433 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369410+0000", 2024-09-14T06:31:34.433 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:34.433 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:34.433 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:34.433 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:31:34.433 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369410+0000", 2024-09-14T06:31:34.433 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.1899999999999999e-07 2024-09-14T06:31:34.433 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:34.433 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:34.434 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:31:34.434 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369411+0000", 2024-09-14T06:31:34.434 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.0300000000000001e-06 2024-09-14T06:31:34.434 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:34.434 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:34.434 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:31:34.434 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369414+0000", 2024-09-14T06:31:34.434 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.1099999999999999e-06 2024-09-14T06:31:34.434 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:34.434 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:34.434 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:31:34.435 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369417+0000", 2024-09-14T06:31:34.435 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.6189999999999998e-06 2024-09-14T06:31:34.435 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:34.435 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:34.435 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:31:34.435 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.416718+0000", 2024-09-14T06:31:34.435 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047301322999999999 2024-09-14T06:31:34.435 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:34.435 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:34.435 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:31:34.435 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.416759+0000", 2024-09-14T06:31:34.435 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.1009e-05 2024-09-14T06:31:34.436 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:34.436 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:34.436 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:31:34.436 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.416809+0000", 2024-09-14T06:31:34.436 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.9966000000000002e-05 2024-09-14T06:31:34.436 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:34.436 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:31:34.436 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:34.436 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:34.436 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:34.436 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:53246 3.f 3:f71f51a7:::rbd_data.112dca2f6ae1.0000000000000015:17 [read 1912832~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:31:34.437 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:44.369420+0000", 2024-09-14T06:31:34.437 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 110.053154616, 2024-09-14T06:31:34.437 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047529463000000001, 2024-09-14T06:31:34.437 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:31:34.437 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:31:34.437 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:31:34.437 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:31:34.437 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:31:34.437 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:34.437 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 53246 2024-09-14T06:31:34.437 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:34.437 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:31:34.438 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:34.438 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:31:34.438 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369420+0000", 2024-09-14T06:31:34.438 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:34.438 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:34.438 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:34.438 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:31:34.438 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369420+0000", 2024-09-14T06:31:34.438 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:34.438 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:34.438 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:34.438 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:31:34.439 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369420+0000", 2024-09-14T06:31:34.439 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.1299999999999999e-07 2024-09-14T06:31:34.439 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:34.439 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:34.439 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:31:34.439 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369425+0000", 2024-09-14T06:31:34.439 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.0130000000000001e-06 2024-09-14T06:31:34.439 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:34.439 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:34.439 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:31:34.439 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369428+0000", 2024-09-14T06:31:34.439 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.349e-06 2024-09-14T06:31:34.440 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:34.440 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:34.440 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:31:34.440 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369431+0000", 2024-09-14T06:31:34.440 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.7389999999999999e-06 2024-09-14T06:31:34.440 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:34.440 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:34.440 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:31:34.440 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.416834+0000", 2024-09-14T06:31:34.440 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047402669000000001 2024-09-14T06:31:34.440 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:34.441 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:34.441 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:31:34.441 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.416882+0000", 2024-09-14T06:31:34.441 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.8068999999999997e-05 2024-09-14T06:31:34.441 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:34.441 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:34.441 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:31:34.441 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.416949+0000", 2024-09-14T06:31:34.441 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.7410999999999995e-05 2024-09-14T06:31:34.441 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:34.441 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:31:34.441 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:34.442 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:34.442 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:34.442 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:53247 3.f 3:f71f51a7:::rbd_data.112dca2f6ae1.0000000000000015:17 [read 1937408~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:31:34.442 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:44.369434+0000", 2024-09-14T06:31:34.442 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 110.053139852, 2024-09-14T06:31:34.442 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047652264999999999, 2024-09-14T06:31:34.442 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:31:34.442 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:31:34.442 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:31:34.442 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:31:34.442 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:31:34.443 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:34.443 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 53247 2024-09-14T06:31:34.443 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:34.443 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:31:34.443 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:34.443 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:31:34.443 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369434+0000", 2024-09-14T06:31:34.443 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:34.443 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:34.443 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:34.443 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:31:34.443 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369434+0000", 2024-09-14T06:31:34.444 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:34.444 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:34.444 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:34.444 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:31:34.444 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369435+0000", 2024-09-14T06:31:34.444 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.2100000000000001e-07 2024-09-14T06:31:34.444 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:34.444 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:34.444 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:31:34.444 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369436+0000", 2024-09-14T06:31:34.444 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.122e-06 2024-09-14T06:31:34.445 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:34.445 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:34.445 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:31:34.445 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369439+0000", 2024-09-14T06:31:34.445 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.0400000000000001e-06 2024-09-14T06:31:34.445 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:34.445 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:34.445 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:31:34.445 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369441+0000", 2024-09-14T06:31:34.445 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.728e-06 2024-09-14T06:31:34.445 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:34.445 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:34.446 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:31:34.446 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.416977+0000", 2024-09-14T06:31:34.446 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047535344 2024-09-14T06:31:34.446 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:34.446 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:34.446 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:31:34.446 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.417029+0000", 2024-09-14T06:31:34.446 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.2037999999999999e-05 2024-09-14T06:31:34.446 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:34.446 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:34.446 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:31:34.446 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.417087+0000", 2024-09-14T06:31:34.447 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.7772000000000002e-05 2024-09-14T06:31:34.447 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:34.447 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:31:34.447 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:34.447 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:34.447 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:34.447 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:53248 3.f 3:f71f51a7:::rbd_data.112dca2f6ae1.0000000000000015:17 [read 1961984~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:31:34.447 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:44.369445+0000", 2024-09-14T06:31:34.447 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 110.05312929999999, 2024-09-14T06:31:34.447 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047798489, 2024-09-14T06:31:34.447 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:31:34.447 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:31:34.448 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:31:34.448 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:31:34.448 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:31:34.448 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:34.448 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 53248 2024-09-14T06:31:34.448 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:34.448 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:31:34.448 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:34.448 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:31:34.448 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369445+0000", 2024-09-14T06:31:34.448 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:34.448 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:34.449 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:34.449 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:31:34.449 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369445+0000", 2024-09-14T06:31:34.449 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:34.449 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:34.449 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:34.449 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:31:34.449 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369445+0000", 2024-09-14T06:31:34.449 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.0699999999999999e-07 2024-09-14T06:31:34.449 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:34.449 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:34.450 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:31:34.450 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369446+0000", 2024-09-14T06:31:34.450 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.1000000000000001e-06 2024-09-14T06:31:34.450 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:34.450 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:34.450 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:31:34.450 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369449+0000", 2024-09-14T06:31:34.450 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.1039999999999998e-06 2024-09-14T06:31:34.450 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:34.450 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:34.450 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:31:34.450 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369452+0000", 2024-09-14T06:31:34.451 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.6199999999999999e-06 2024-09-14T06:31:34.451 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:34.451 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:34.451 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:31:34.451 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.417114+0000", 2024-09-14T06:31:34.451 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047661901999999999 2024-09-14T06:31:34.451 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:34.451 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:34.451 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:31:34.451 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.417181+0000", 2024-09-14T06:31:34.451 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.7385999999999995e-05 2024-09-14T06:31:34.452 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:34.452 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:34.452 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:31:34.452 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.417243+0000", 2024-09-14T06:31:34.452 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.2169999999999996e-05 2024-09-14T06:31:34.452 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:34.452 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:31:34.452 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:34.452 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:34.452 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:34.452 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:53249 3.f 3:f71f51a7:::rbd_data.112dca2f6ae1.0000000000000015:17 [read 1986560~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:31:34.452 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:44.369455+0000", 2024-09-14T06:31:34.453 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 110.053118989, 2024-09-14T06:31:34.453 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047890493999999999, 2024-09-14T06:31:34.453 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:31:34.453 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:31:34.453 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:31:34.453 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:31:34.453 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:31:34.453 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:34.453 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 53249 2024-09-14T06:31:34.453 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:34.453 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:31:34.453 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:34.454 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:31:34.454 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369455+0000", 2024-09-14T06:31:34.454 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:34.454 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:34.454 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:34.454 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:31:34.454 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369455+0000", 2024-09-14T06:31:34.454 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:34.454 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:34.454 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:34.454 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:31:34.454 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369455+0000", 2024-09-14T06:31:34.454 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.0800000000000001e-07 2024-09-14T06:31:34.454 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:34.455 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:34.455 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:31:34.455 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369456+0000", 2024-09-14T06:31:34.455 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.0729999999999999e-06 2024-09-14T06:31:34.455 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:34.455 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:34.455 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:31:34.455 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369460+0000", 2024-09-14T06:31:34.455 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.027e-06 2024-09-14T06:31:34.455 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:34.455 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:34.456 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:31:34.456 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369462+0000", 2024-09-14T06:31:34.456 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.5320000000000001e-06 2024-09-14T06:31:34.456 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:34.456 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:34.456 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:31:34.456 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.417257+0000", 2024-09-14T06:31:34.456 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.04779489 2024-09-14T06:31:34.456 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:34.456 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:34.456 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:31:34.456 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.417296+0000", 2024-09-14T06:31:34.457 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.8658000000000002e-05 2024-09-14T06:31:34.457 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:34.457 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:34.457 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:31:34.457 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.417346+0000", 2024-09-14T06:31:34.457 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.0105999999999998e-05 2024-09-14T06:31:34.457 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:34.457 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:31:34.457 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:34.457 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:34.457 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:34.457 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:53250 3.f 3:f71f51a7:::rbd_data.112dca2f6ae1.0000000000000015:17 [read 2011136~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:31:34.457 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:44.369465+0000", 2024-09-14T06:31:34.458 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 110.053109019, 2024-09-14T06:31:34.458 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.048032230000000002, 2024-09-14T06:31:34.458 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:31:34.458 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:31:34.458 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:31:34.458 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:31:34.458 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:31:34.458 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:34.458 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 53250 2024-09-14T06:31:34.458 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:34.459 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:31:34.459 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:34.459 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:31:34.459 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369465+0000", 2024-09-14T06:31:34.459 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:34.459 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:34.459 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:34.459 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:31:34.459 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369465+0000", 2024-09-14T06:31:34.459 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:34.459 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:34.459 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:34.460 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:31:34.460 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369465+0000", 2024-09-14T06:31:34.460 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.2100000000000001e-07 2024-09-14T06:31:34.460 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:34.460 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:34.460 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:31:34.460 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369466+0000", 2024-09-14T06:31:34.460 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.0640000000000001e-06 2024-09-14T06:31:34.460 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:34.460 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:34.461 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:31:34.461 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369469+0000", 2024-09-14T06:31:34.461 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.0170000000000001e-06 2024-09-14T06:31:34.461 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:34.461 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:34.461 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:31:34.461 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369472+0000", 2024-09-14T06:31:34.461 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.6189999999999998e-06 2024-09-14T06:31:34.461 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:34.461 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:34.461 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:31:34.462 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.417383+0000", 2024-09-14T06:31:34.462 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047911092000000002 2024-09-14T06:31:34.462 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:34.462 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:34.462 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:31:34.462 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.417430+0000", 2024-09-14T06:31:34.462 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.6898999999999999e-05 2024-09-14T06:31:34.462 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:34.462 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:34.462 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:31:34.462 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.417497+0000", 2024-09-14T06:31:34.462 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.7317999999999995e-05 2024-09-14T06:31:34.463 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:34.463 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:31:34.463 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:34.463 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:34.463 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:34.463 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:53251 3.f 3:f71f51a7:::rbd_data.112dca2f6ae1.0000000000000015:17 [read 2035712~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:31:34.463 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:44.369475+0000", 2024-09-14T06:31:34.463 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 110.053099023, 2024-09-14T06:31:34.463 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.048154542000000002, 2024-09-14T06:31:34.463 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:31:34.463 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:31:34.463 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:31:34.463 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:31:34.464 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:31:34.464 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:34.464 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 53251 2024-09-14T06:31:34.464 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:34.464 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:31:34.464 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:34.464 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:31:34.464 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369475+0000", 2024-09-14T06:31:34.464 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:34.464 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:34.464 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:34.464 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:31:34.465 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369475+0000", 2024-09-14T06:31:34.465 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:34.465 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:34.465 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:34.465 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:31:34.465 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369475+0000", 2024-09-14T06:31:34.465 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.0100000000000001e-07 2024-09-14T06:31:34.465 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:34.465 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:34.465 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:31:34.465 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369476+0000", 2024-09-14T06:31:34.466 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.0580000000000001e-06 2024-09-14T06:31:34.466 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:34.466 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:34.466 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:31:34.466 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369479+0000", 2024-09-14T06:31:34.466 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.9950000000000001e-06 2024-09-14T06:31:34.466 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:34.466 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:34.466 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:31:34.466 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369482+0000", 2024-09-14T06:31:34.466 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.6340000000000001e-06 2024-09-14T06:31:34.467 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:34.467 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:34.467 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:31:34.467 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.417530+0000", 2024-09-14T06:31:34.467 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.048047535000000002 2024-09-14T06:31:34.467 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:34.467 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:34.467 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:31:34.467 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.417574+0000", 2024-09-14T06:31:34.467 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.4255999999999998e-05 2024-09-14T06:31:34.467 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:34.468 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:34.468 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:31:34.468 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.417630+0000", 2024-09-14T06:31:34.468 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.5862999999999997e-05 2024-09-14T06:31:34.468 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:34.468 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:31:34.468 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:34.468 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:34.468 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:34.468 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:53252 3.f 3:f71f51a7:::rbd_data.112dca2f6ae1.0000000000000015:17 [read 2060288~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:31:34.468 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:44.369485+0000", 2024-09-14T06:31:34.468 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 110.053089175, 2024-09-14T06:31:34.468 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.048254116, 2024-09-14T06:31:34.469 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:31:34.469 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:31:34.469 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:31:34.469 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:31:34.469 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:31:34.469 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:34.469 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 53252 2024-09-14T06:31:34.469 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:34.469 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:31:34.469 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:34.469 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:31:34.469 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369485+0000", 2024-09-14T06:31:34.470 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:34.470 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:34.470 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:34.470 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:31:34.470 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369485+0000", 2024-09-14T06:31:34.470 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:34.470 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:34.470 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:34.470 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:31:34.470 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369485+0000", 2024-09-14T06:31:34.470 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.1e-07 2024-09-14T06:31:34.471 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:34.471 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:34.471 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:31:34.471 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369487+0000", 2024-09-14T06:31:34.471 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.42e-06 2024-09-14T06:31:34.471 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:34.471 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:34.471 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:31:34.471 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369490+0000", 2024-09-14T06:31:34.471 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.9639999999999999e-06 2024-09-14T06:31:34.471 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:34.471 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:34.472 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:31:34.472 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369492+0000", 2024-09-14T06:31:34.472 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.5679999999999998e-06 2024-09-14T06:31:34.472 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:34.472 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:34.472 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:31:34.472 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.417645+0000", 2024-09-14T06:31:34.472 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.048152753 2024-09-14T06:31:34.472 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:34.472 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:34.472 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:31:34.472 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.417687+0000", 2024-09-14T06:31:34.473 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.1900999999999997e-05 2024-09-14T06:31:34.473 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:34.473 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:34.473 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:31:34.473 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.417739+0000", 2024-09-14T06:31:34.473 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.2299999999999997e-05 2024-09-14T06:31:34.473 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:34.473 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:31:34.473 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:34.473 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:34.473 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:34.473 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:53253 3.f 3:f71f51a7:::rbd_data.112dca2f6ae1.0000000000000015:17 [read 2084864~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:31:34.474 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:44.369495+0000", 2024-09-14T06:31:34.474 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 110.053079051, 2024-09-14T06:31:34.474 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.048366785000000002, 2024-09-14T06:31:34.474 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:31:34.474 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:31:34.474 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:31:34.474 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:31:34.474 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:31:34.474 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:34.474 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 53253 2024-09-14T06:31:34.474 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:34.474 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:31:34.474 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:34.475 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:31:34.475 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369495+0000", 2024-09-14T06:31:34.475 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:34.475 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:34.475 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:34.475 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:31:34.475 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369495+0000", 2024-09-14T06:31:34.475 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:34.475 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:34.475 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:34.475 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:31:34.476 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370388+0000", 2024-09-14T06:31:34.476 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00089287999999999998 2024-09-14T06:31:34.476 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:34.476 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:34.476 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:31:34.476 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370392+0000", 2024-09-14T06:31:34.476 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.2130000000000002e-06 2024-09-14T06:31:34.476 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:34.476 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:34.476 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:31:34.476 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370401+0000", 2024-09-14T06:31:34.476 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.7649999999999999e-06 2024-09-14T06:31:34.477 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:34.477 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:34.477 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:31:34.477 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370406+0000", 2024-09-14T06:31:34.477 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.0159999999999999e-06 2024-09-14T06:31:34.477 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:34.477 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:34.477 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:31:34.477 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.417764+0000", 2024-09-14T06:31:34.477 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047358348000000001 2024-09-14T06:31:34.477 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:34.478 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:34.478 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:31:34.478 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.417806+0000", 2024-09-14T06:31:34.478 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.1149999999999997e-05 2024-09-14T06:31:34.478 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:34.478 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:34.478 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:31:34.478 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.417862+0000", 2024-09-14T06:31:34.478 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.6412999999999997e-05 2024-09-14T06:31:34.478 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:34.478 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:31:34.478 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:34.478 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:34.479 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:34.479 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:53340 3.f 3:f71f51a7:::rbd_data.112dca2f6ae1.0000000000000015:17 [read 1056768~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:31:34.479 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:44.370421+0000", 2024-09-14T06:31:34.479 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 110.052153171, 2024-09-14T06:31:34.479 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047566313999999998, 2024-09-14T06:31:34.479 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:31:34.479 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:31:34.479 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:31:34.479 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:31:34.479 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:31:34.479 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:34.479 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 53340 2024-09-14T06:31:34.480 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:34.480 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:31:34.480 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:34.480 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:31:34.480 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370421+0000", 2024-09-14T06:31:34.480 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:34.480 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:34.480 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:34.480 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:31:34.480 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370421+0000", 2024-09-14T06:31:34.480 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:34.481 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:34.481 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:34.481 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:31:34.481 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370422+0000", 2024-09-14T06:31:34.481 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.3799999999999996e-07 2024-09-14T06:31:34.481 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:34.481 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:34.481 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:31:34.481 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370424+0000", 2024-09-14T06:31:34.481 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.6919999999999998e-06 2024-09-14T06:31:34.481 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:34.481 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:34.481 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:31:34.482 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370432+0000", 2024-09-14T06:31:34.482 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.2439999999999996e-06 2024-09-14T06:31:34.482 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:34.482 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:34.482 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:31:34.482 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370437+0000", 2024-09-14T06:31:34.482 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.5389999999999998e-06 2024-09-14T06:31:34.482 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:34.482 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:34.482 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:31:34.482 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.417888+0000", 2024-09-14T06:31:34.482 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047450432000000001 2024-09-14T06:31:34.483 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:34.483 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:34.483 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:31:34.483 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.417935+0000", 2024-09-14T06:31:34.483 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.7454999999999999e-05 2024-09-14T06:31:34.483 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:34.483 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:34.483 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:31:34.483 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.417987+0000", 2024-09-14T06:31:34.483 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.2213999999999998e-05 2024-09-14T06:31:34.483 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:34.483 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:31:34.483 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:34.484 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:34.484 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:34.484 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:53341 3.f 3:f71f51a7:::rbd_data.112dca2f6ae1.0000000000000015:17 [read 1081344~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:31:34.484 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:44.370446+0000", 2024-09-14T06:31:34.484 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 110.05212789799999, 2024-09-14T06:31:34.484 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047665723, 2024-09-14T06:31:34.484 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:31:34.484 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:31:34.484 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:31:34.484 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:31:34.484 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:31:34.484 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:34.485 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 53341 2024-09-14T06:31:34.485 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:34.485 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:31:34.485 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:34.485 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:31:34.485 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370446+0000", 2024-09-14T06:31:34.485 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:34.485 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:34.485 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:34.485 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:31:34.485 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370446+0000", 2024-09-14T06:31:34.485 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:34.486 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:34.486 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:34.486 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:31:34.486 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370447+0000", 2024-09-14T06:31:34.486 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.6100000000000002e-07 2024-09-14T06:31:34.486 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:34.486 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:34.486 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:31:34.486 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370448+0000", 2024-09-14T06:31:34.486 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.7370000000000001e-06 2024-09-14T06:31:34.486 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:34.486 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:34.487 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:31:34.487 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370462+0000", 2024-09-14T06:31:34.487 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.3227000000000001e-05 2024-09-14T06:31:34.487 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:34.487 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:34.487 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:31:34.487 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370467+0000", 2024-09-14T06:31:34.487 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.6540000000000004e-06 2024-09-14T06:31:34.487 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:34.487 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:34.487 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:31:34.487 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.418013+0000", 2024-09-14T06:31:34.488 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047545325999999999 2024-09-14T06:31:34.488 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:34.488 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:34.488 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:31:34.488 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.418054+0000", 2024-09-14T06:31:34.488 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.1696999999999997e-05 2024-09-14T06:31:34.488 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:34.488 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:34.488 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:31:34.488 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.418112+0000", 2024-09-14T06:31:34.488 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.7720999999999998e-05 2024-09-14T06:31:34.488 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:34.489 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:31:34.489 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:34.489 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:34.489 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:34.489 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:53342 3.f 3:f71f51a7:::rbd_data.112dca2f6ae1.0000000000000015:17 [read 1105920~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:31:34.489 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:44.370475+0000", 2024-09-14T06:31:34.489 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 110.052098837, 2024-09-14T06:31:34.489 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047748848000000003, 2024-09-14T06:31:34.489 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:31:34.489 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:31:34.489 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:31:34.489 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:31:34.489 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:31:34.490 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:34.490 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 53342 2024-09-14T06:31:34.490 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:34.490 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:31:34.490 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:34.490 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:31:34.490 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370475+0000", 2024-09-14T06:31:34.490 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:34.490 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:34.490 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:34.490 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:31:34.490 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370475+0000", 2024-09-14T06:31:34.491 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:34.491 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:34.491 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:34.491 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:31:34.491 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370476+0000", 2024-09-14T06:31:34.491 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.4700000000000002e-07 2024-09-14T06:31:34.491 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:34.491 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:34.491 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:31:34.491 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370478+0000", 2024-09-14T06:31:34.491 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.5890000000000001e-06 2024-09-14T06:31:34.492 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:34.492 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:34.492 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:31:34.492 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370486+0000", 2024-09-14T06:31:34.492 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.3470000000000001e-06 2024-09-14T06:31:34.492 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:34.492 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:34.492 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:31:34.492 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370491+0000", 2024-09-14T06:31:34.492 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.5720000000000002e-06 2024-09-14T06:31:34.492 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:34.492 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:34.493 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:31:34.493 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.418135+0000", 2024-09-14T06:31:34.493 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047644094999999997 2024-09-14T06:31:34.493 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:34.493 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:34.493 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:31:34.493 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.418174+0000", 2024-09-14T06:31:34.493 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.8204000000000002e-05 2024-09-14T06:31:34.493 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:34.493 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:34.493 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:31:34.494 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.418224+0000", 2024-09-14T06:31:34.494 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.0593999999999997e-05 2024-09-14T06:31:34.494 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:34.494 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:31:34.494 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:34.494 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:34.494 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:34.494 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:53343 3.f 3:f71f51a7:::rbd_data.112dca2f6ae1.0000000000000015:17 [read 1130496~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:31:34.494 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:44.370498+0000", 2024-09-14T06:31:34.494 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 110.052075961, 2024-09-14T06:31:34.494 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047841574999999997, 2024-09-14T06:31:34.494 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:31:34.494 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:31:34.495 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:31:34.495 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:31:34.495 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:31:34.495 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:34.495 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 53343 2024-09-14T06:31:34.495 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:34.495 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:31:34.495 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:34.495 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:31:34.495 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370498+0000", 2024-09-14T06:31:34.495 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:34.495 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:34.496 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:34.496 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:31:34.496 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370498+0000", 2024-09-14T06:31:34.496 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:34.496 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:34.496 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:34.496 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:31:34.496 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370499+0000", 2024-09-14T06:31:34.496 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.9999999999999998e-07 2024-09-14T06:31:34.496 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:34.496 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:34.497 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:31:34.497 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370501+0000", 2024-09-14T06:31:34.497 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.0310000000000001e-06 2024-09-14T06:31:34.497 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:34.497 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:34.497 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:31:34.497 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370507+0000", 2024-09-14T06:31:34.497 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.4169999999999997e-06 2024-09-14T06:31:34.497 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:34.497 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:34.497 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:31:34.497 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370516+0000", 2024-09-14T06:31:34.498 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.7190000000000007e-06 2024-09-14T06:31:34.498 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:34.498 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:34.498 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:31:34.498 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.418247+0000", 2024-09-14T06:31:34.498 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047731190999999999 2024-09-14T06:31:34.498 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:34.498 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:34.498 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:31:34.498 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.418286+0000", 2024-09-14T06:31:34.498 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.8988e-05 2024-09-14T06:31:34.498 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:34.498 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:34.499 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:31:34.499 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.418340+0000", 2024-09-14T06:31:34.499 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.3829000000000002e-05 2024-09-14T06:31:34.499 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:34.499 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:31:34.499 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:34.499 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:34.499 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:34.499 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:53344 3.f 3:f71f51a7:::rbd_data.112dca2f6ae1.0000000000000015:17 [read 1155072~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:31:34.499 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:44.370524+0000", 2024-09-14T06:31:34.499 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 110.05205019100001, 2024-09-14T06:31:34.499 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047972262000000002, 2024-09-14T06:31:34.500 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:31:34.500 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:31:34.500 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:31:34.500 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:31:34.500 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:31:34.500 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:34.500 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 53344 2024-09-14T06:31:34.500 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:34.500 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:31:34.500 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:34.500 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:31:34.501 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370524+0000", 2024-09-14T06:31:34.501 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:34.501 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:34.501 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:34.501 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:31:34.501 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370524+0000", 2024-09-14T06:31:34.501 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:34.501 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:34.501 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:34.501 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:31:34.501 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370618+0000", 2024-09-14T06:31:34.501 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.4425999999999993e-05 2024-09-14T06:31:34.501 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:34.502 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:34.502 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:31:34.502 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370622+0000", 2024-09-14T06:31:34.502 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.3519999999999998e-06 2024-09-14T06:31:34.502 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:34.502 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:34.502 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:31:34.502 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370631+0000", 2024-09-14T06:31:34.502 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.9299999999999992e-06 2024-09-14T06:31:34.502 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:34.502 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:34.502 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:31:34.503 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370637+0000", 2024-09-14T06:31:34.503 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.3269999999999997e-06 2024-09-14T06:31:34.503 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:34.503 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:34.503 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:31:34.503 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.418372+0000", 2024-09-14T06:31:34.503 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047734764999999998 2024-09-14T06:31:34.503 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:34.503 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:34.503 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:31:34.503 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.418431+0000", 2024-09-14T06:31:34.503 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.8696999999999997e-05 2024-09-14T06:31:34.503 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:34.504 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:34.504 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:31:34.504 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.418496+0000", 2024-09-14T06:31:34.504 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.5765000000000005e-05 2024-09-14T06:31:34.504 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:34.504 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:31:34.504 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:34.504 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:34.504 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:34.504 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:53345 3.f 3:f71f51a7:::rbd_data.112dca2f6ae1.0000000000000015:17 [read 1179648~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:31:34.504 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:44.370647+0000", 2024-09-14T06:31:34.504 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 110.051926724, 2024-09-14T06:31:34.505 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047953775999999997, 2024-09-14T06:31:34.505 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:31:34.505 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:31:34.505 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:31:34.505 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:31:34.505 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:31:34.505 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:34.505 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 53345 2024-09-14T06:31:34.505 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:34.505 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:31:34.505 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:34.505 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:31:34.505 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370647+0000", 2024-09-14T06:31:34.505 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:34.506 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:34.506 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:34.506 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:31:34.506 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370647+0000", 2024-09-14T06:31:34.506 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:34.506 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:34.506 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:34.506 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:31:34.506 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.372346+0000", 2024-09-14T06:31:34.506 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.001698284 2024-09-14T06:31:34.506 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:34.506 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:34.506 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:31:34.507 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.372351+0000", 2024-09-14T06:31:34.507 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.8890000000000001e-06 2024-09-14T06:31:34.507 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:34.507 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:34.507 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:31:34.507 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.372362+0000", 2024-09-14T06:31:34.507 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.0941000000000001e-05 2024-09-14T06:31:34.507 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:34.507 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:34.507 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:31:34.507 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.372369+0000", 2024-09-14T06:31:34.507 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.362e-06 2024-09-14T06:31:34.508 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:34.508 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:34.508 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:31:34.508 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.418510+0000", 2024-09-14T06:31:34.508 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046140914999999998 2024-09-14T06:31:34.508 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:34.508 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:34.508 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:31:34.508 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.418549+0000", 2024-09-14T06:31:34.508 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.8717999999999998e-05 2024-09-14T06:31:34.508 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:34.508 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:34.508 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:31:34.509 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.418601+0000", 2024-09-14T06:31:34.509 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.2667000000000002e-05 2024-09-14T06:31:34.509 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:34.509 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:31:34.509 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:34.509 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:34.509 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:34.509 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:53351 3.f 3:f71f51a7:::rbd_data.112dca2f6ae1.0000000000000015:17 [read 1327104~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:31:34.509 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:44.372499+0000", 2024-09-14T06:31:34.509 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 110.050075603, 2024-09-14T06:31:34.509 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047140596, 2024-09-14T06:31:34.510 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:31:34.510 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:31:34.510 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:31:34.510 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:31:34.510 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:31:34.510 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:34.510 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 53351 2024-09-14T06:31:34.510 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:34.510 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:31:34.510 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:34.510 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:31:34.510 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.372499+0000", 2024-09-14T06:31:34.511 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:34.511 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:34.511 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:34.511 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:31:34.511 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.372499+0000", 2024-09-14T06:31:34.511 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:34.511 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:34.511 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:34.511 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:31:34.511 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.372499+0000", 2024-09-14T06:31:34.511 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.3000000000000001e-07 2024-09-14T06:31:34.511 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:34.511 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:34.512 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:31:34.512 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.372501+0000", 2024-09-14T06:31:34.512 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.0820000000000001e-06 2024-09-14T06:31:34.512 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:34.512 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:34.512 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:31:34.512 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.372513+0000", 2024-09-14T06:31:34.512 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.1610999999999999e-05 2024-09-14T06:31:34.512 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:34.512 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:34.512 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:31:34.512 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.372518+0000", 2024-09-14T06:31:34.513 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.327e-06 2024-09-14T06:31:34.513 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:34.513 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:34.513 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:31:34.513 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.419530+0000", 2024-09-14T06:31:34.513 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047011798 2024-09-14T06:31:34.513 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:34.513 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:34.513 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:31:34.513 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.419578+0000", 2024-09-14T06:31:34.513 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.8446999999999999e-05 2024-09-14T06:31:34.513 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:34.514 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:34.514 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:31:34.514 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.419639+0000", 2024-09-14T06:31:34.514 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.0900999999999998e-05 2024-09-14T06:31:34.514 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:34.514 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:31:34.514 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:34.514 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:34.514 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:34.514 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:53352 3.f 3:f71f51a7:::rbd_data.112dca2f6ae1.0000000000000015:17 [read 1351680~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:31:34.514 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:44.372525+0000", 2024-09-14T06:31:34.515 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 110.05004968900001, 2024-09-14T06:31:34.515 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047259847000000001, 2024-09-14T06:31:34.515 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:31:34.515 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:31:34.515 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:31:34.515 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:31:34.515 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:31:34.515 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:34.515 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 53352 2024-09-14T06:31:34.515 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:34.515 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:31:34.515 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:34.516 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:31:34.516 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.372525+0000", 2024-09-14T06:31:34.516 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:34.516 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:34.516 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:34.516 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:31:34.516 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.372525+0000", 2024-09-14T06:31:34.516 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:34.516 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:34.516 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:34.516 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:31:34.516 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.372525+0000", 2024-09-14T06:31:34.517 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.0600000000000001e-07 2024-09-14T06:31:34.517 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:34.517 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:34.517 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:31:34.517 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.372527+0000", 2024-09-14T06:31:34.517 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.0499999999999999e-06 2024-09-14T06:31:34.517 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:34.517 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:34.517 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:31:34.517 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.372533+0000", 2024-09-14T06:31:34.517 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.4320000000000004e-06 2024-09-14T06:31:34.518 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:34.518 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:34.518 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:31:34.518 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.372538+0000", 2024-09-14T06:31:34.518 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.9130000000000002e-06 2024-09-14T06:31:34.518 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:34.518 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:34.518 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:31:34.518 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.419667+0000", 2024-09-14T06:31:34.518 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047128976000000003 2024-09-14T06:31:34.518 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:34.519 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:34.519 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:31:34.519 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.419715+0000", 2024-09-14T06:31:34.519 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.7868999999999999e-05 2024-09-14T06:31:34.519 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:34.519 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:34.519 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:31:34.519 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.419784+0000", 2024-09-14T06:31:34.519 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.9201000000000004e-05 2024-09-14T06:31:34.519 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:34.519 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:31:34.520 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:34.520 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:34.520 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:34.520 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:53353 3.f 3:f71f51a7:::rbd_data.112dca2f6ae1.0000000000000015:17 [read 1376256~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:31:34.520 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:44.372545+0000", 2024-09-14T06:31:34.520 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 110.050029649, 2024-09-14T06:31:34.520 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047364232999999999, 2024-09-14T06:31:34.520 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:31:34.520 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:31:34.520 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:31:34.520 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:31:34.521 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:31:34.521 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:34.521 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 53353 2024-09-14T06:31:34.521 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:34.521 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:31:34.521 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:34.521 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:31:34.521 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.372545+0000", 2024-09-14T06:31:34.521 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:34.521 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:34.521 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:34.522 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:31:34.522 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.372545+0000", 2024-09-14T06:31:34.522 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:34.522 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:34.522 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:34.522 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:31:34.522 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.372545+0000", 2024-09-14T06:31:34.522 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.27e-07 2024-09-14T06:31:34.522 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:34.522 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:34.522 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:31:34.522 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.372547+0000", 2024-09-14T06:31:34.522 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.0600000000000002e-06 2024-09-14T06:31:34.523 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:34.523 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:34.523 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:31:34.523 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.372553+0000", 2024-09-14T06:31:34.523 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.7980000000000002e-06 2024-09-14T06:31:34.523 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:34.523 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:34.523 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:31:34.523 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.372557+0000", 2024-09-14T06:31:34.523 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.7389999999999999e-06 2024-09-14T06:31:34.523 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:34.524 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:34.524 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:31:34.524 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.419811+0000", 2024-09-14T06:31:34.524 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047253310999999999 2024-09-14T06:31:34.524 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:34.524 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:34.524 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:31:34.524 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.419854+0000", 2024-09-14T06:31:34.524 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.3448000000000001e-05 2024-09-14T06:31:34.524 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:34.524 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:34.525 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:31:34.525 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.419909+0000", 2024-09-14T06:31:34.525 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.4549999999999998e-05 2024-09-14T06:31:34.525 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:34.525 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:31:34.525 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:34.525 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:34.525 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:34.525 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:53354 3.f 3:f71f51a7:::rbd_data.112dca2f6ae1.0000000000000015:17 [read 1400832~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:31:34.525 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:44.372563+0000", 2024-09-14T06:31:34.525 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 110.050011212, 2024-09-14T06:31:34.525 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047482233999999998, 2024-09-14T06:31:34.526 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:31:34.526 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:31:34.526 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:31:34.526 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:31:34.526 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:31:34.526 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:34.526 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 53354 2024-09-14T06:31:34.526 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:34.526 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:31:34.526 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:34.526 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:31:34.526 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.372563+0000", 2024-09-14T06:31:34.526 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:34.527 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:34.527 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:34.527 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:31:34.527 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.372563+0000", 2024-09-14T06:31:34.527 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:34.527 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:34.527 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:34.527 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:31:34.527 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.375009+0000", 2024-09-14T06:31:34.527 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.0024457680000000001 2024-09-14T06:31:34.527 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:34.527 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:34.528 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:31:34.528 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.375014+0000", 2024-09-14T06:31:34.528 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.0599999999999998e-06 2024-09-14T06:31:34.528 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:34.528 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:34.528 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:31:34.528 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.375026+0000", 2024-09-14T06:31:34.528 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.1783999999999999e-05 2024-09-14T06:31:34.528 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:34.528 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:34.528 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:31:34.528 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.375035+0000", 2024-09-14T06:31:34.529 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.7799999999999995e-06 2024-09-14T06:31:34.529 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:34.529 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:34.529 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:31:34.529 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.419942+0000", 2024-09-14T06:31:34.529 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.044906982999999998 2024-09-14T06:31:34.529 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:34.529 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:34.529 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:31:34.529 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.419989+0000", 2024-09-14T06:31:34.529 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.7104999999999997e-05 2024-09-14T06:31:34.530 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:34.530 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:34.530 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:31:34.530 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.420045+0000", 2024-09-14T06:31:34.530 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.5754000000000002e-05 2024-09-14T06:31:34.530 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:34.530 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:31:34.530 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:34.530 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:34.530 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:31:34.530 INFO:teuthology.orchestra.run.smithi138.stdout:} 2024-09-14T06:31:34.532 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-14T06:31:34.533 INFO:tasks.ceph.osd.0.smithi094.stderr:2024-09-14T06:31:34.529+0000 7f07c9333640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33636) UID: 0 2024-09-14T06:31:34.633 INFO:tasks.ceph.osd.7.smithi138.stderr:2024-09-14T06:31:34.631+0000 7f3c6a21d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33623) UID: 0 2024-09-14T06:31:34.645 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-14T06:31:34.645 INFO:teuthology.orchestra.run.smithi094.stdout: "ops": [], 2024-09-14T06:31:34.645 INFO:teuthology.orchestra.run.smithi094.stdout: "num_ops": 0 2024-09-14T06:31:34.645 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-14T06:31:34.655 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-14T06:31:34.733 INFO:tasks.ceph.osd.3.smithi094.stderr:2024-09-14T06:31:34.731+0000 7feeeafe8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33655) UID: 0 2024-09-14T06:31:34.773 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-14T06:31:34.773 INFO:teuthology.orchestra.run.smithi094.stdout: "ops": [], 2024-09-14T06:31:34.773 INFO:teuthology.orchestra.run.smithi094.stdout: "complaint_time": 30, 2024-09-14T06:31:34.773 INFO:teuthology.orchestra.run.smithi094.stdout: "num_blocked_ops": 0 2024-09-14T06:31:34.773 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-14T06:31:34.782 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-14T06:31:34.866 INFO:tasks.ceph.osd.0.smithi094.stderr:2024-09-14T06:31:34.864+0000 7f07c9333640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33636) UID: 0 2024-09-14T06:31:34.935 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-14T06:31:34.935 INFO:teuthology.orchestra.run.smithi094.stdout: "size": 20, 2024-09-14T06:31:34.935 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 600, 2024-09-14T06:31:34.935 INFO:teuthology.orchestra.run.smithi094.stdout: "ops": [ 2024-09-14T06:31:34.935 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:34.935 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:115193 3.1 3:856be886:::rbd_data.112d4d24c558.0000000000000004:29 [read 1523712~16384 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e96)", 2024-09-14T06:31:34.935 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:30:17.939854+0000", 2024-09-14T06:31:34.935 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 76.993141390000005, 2024-09-14T06:31:34.935 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.073897516999999996, 2024-09-14T06:31:34.935 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:31:34.935 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:31:34.936 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:31:34.936 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:31:34.936 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:31:34.936 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:34.936 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 115193 2024-09-14T06:31:34.936 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:34.936 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:31:34.936 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:34.936 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:31:34.936 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.939854+0000", 2024-09-14T06:31:34.936 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:34.936 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:34.937 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:34.937 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:31:34.937 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.939854+0000", 2024-09-14T06:31:34.937 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:34.937 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:34.937 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:34.937 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:31:34.937 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.939854+0000", 2024-09-14T06:31:34.937 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.9500000000000002e-07 2024-09-14T06:31:34.937 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:34.937 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:34.937 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:31:34.938 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.939856+0000", 2024-09-14T06:31:34.938 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.0959999999999999e-06 2024-09-14T06:31:34.938 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:34.938 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:34.938 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:31:34.938 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.939863+0000", 2024-09-14T06:31:34.938 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.1129999999999997e-06 2024-09-14T06:31:34.938 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:34.938 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:34.938 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:31:34.938 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.939867+0000", 2024-09-14T06:31:34.938 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.8350000000000003e-06 2024-09-14T06:31:34.939 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:34.939 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:34.939 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:31:34.939 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.013626+0000", 2024-09-14T06:31:34.939 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.073758799999999999 2024-09-14T06:31:34.939 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:34.939 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:34.939 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:31:34.939 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.013690+0000", 2024-09-14T06:31:34.939 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.3918999999999996e-05 2024-09-14T06:31:34.939 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:34.939 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:34.939 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:31:34.939 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.013751+0000", 2024-09-14T06:31:34.940 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.1159000000000002e-05 2024-09-14T06:31:34.940 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:34.940 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:31:34.940 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:34.940 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:34.940 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:34.940 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:115194 3.1 3:856be886:::rbd_data.112d4d24c558.0000000000000004:29 [read 1769472~16384 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e96)", 2024-09-14T06:31:34.940 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:30:17.939874+0000", 2024-09-14T06:31:34.940 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 76.993121079000005, 2024-09-14T06:31:34.940 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.074016988000000006, 2024-09-14T06:31:34.940 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:31:34.941 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:31:34.941 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:31:34.941 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:31:34.941 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:31:34.941 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:34.941 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 115194 2024-09-14T06:31:34.941 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:34.941 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:31:34.941 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:34.941 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:31:34.941 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.939874+0000", 2024-09-14T06:31:34.942 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:34.942 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:34.942 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:34.942 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:31:34.942 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.939874+0000", 2024-09-14T06:31:34.942 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:34.942 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:34.942 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:34.942 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:31:34.942 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.939875+0000", 2024-09-14T06:31:34.942 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.2600000000000002e-07 2024-09-14T06:31:34.942 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:34.942 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:34.943 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:31:34.943 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.939877+0000", 2024-09-14T06:31:34.943 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.9420000000000002e-06 2024-09-14T06:31:34.943 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:34.943 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:34.943 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:31:34.943 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.939882+0000", 2024-09-14T06:31:34.943 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.8749999999999997e-06 2024-09-14T06:31:34.943 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:34.943 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:34.943 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:31:34.943 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.939887+0000", 2024-09-14T06:31:34.943 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.741e-06 2024-09-14T06:31:34.944 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:34.944 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:34.944 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:31:34.944 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.013767+0000", 2024-09-14T06:31:34.944 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.073879720999999995 2024-09-14T06:31:34.944 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:34.944 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:34.944 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:31:34.944 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.013832+0000", 2024-09-14T06:31:34.944 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.5312000000000001e-05 2024-09-14T06:31:34.944 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:34.944 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:34.944 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:31:34.945 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.013891+0000", 2024-09-14T06:31:34.945 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.8870999999999999e-05 2024-09-14T06:31:34.945 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:34.945 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:31:34.945 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:34.945 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:34.945 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:34.945 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:115195 3.1 3:856be886:::rbd_data.112d4d24c558.0000000000000004:29 [read 2015232~16384 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e96)", 2024-09-14T06:31:34.945 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:30:17.939903+0000", 2024-09-14T06:31:34.945 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 76.993091894000003, 2024-09-14T06:31:34.945 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.074128199000000006, 2024-09-14T06:31:34.945 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:31:34.945 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:31:34.945 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:31:34.946 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:31:34.946 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:31:34.946 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:34.946 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 115195 2024-09-14T06:31:34.946 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:34.946 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:31:34.946 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:34.946 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:31:34.946 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.939903+0000", 2024-09-14T06:31:34.946 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:34.946 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:34.946 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:34.946 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:31:34.947 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.939903+0000", 2024-09-14T06:31:34.947 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:34.947 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:34.947 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:34.947 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:31:34.947 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.939904+0000", 2024-09-14T06:31:34.947 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.5799999999999999e-07 2024-09-14T06:31:34.947 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:34.947 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:34.947 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:31:34.947 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.939906+0000", 2024-09-14T06:31:34.947 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.2350000000000002e-06 2024-09-14T06:31:34.947 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:34.947 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:34.948 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:31:34.948 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.939913+0000", 2024-09-14T06:31:34.948 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.5649999999999998e-06 2024-09-14T06:31:34.948 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:34.948 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:34.948 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:31:34.948 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.939918+0000", 2024-09-14T06:31:34.948 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.784e-06 2024-09-14T06:31:34.948 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:34.948 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:34.948 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:31:34.948 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.013921+0000", 2024-09-14T06:31:34.948 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.074002911000000005 2024-09-14T06:31:34.949 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:34.949 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:34.949 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:31:34.949 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.013971+0000", 2024-09-14T06:31:34.949 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.9551e-05 2024-09-14T06:31:34.949 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:34.949 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:34.949 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:31:34.949 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.014031+0000", 2024-09-14T06:31:34.949 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.0594999999999998e-05 2024-09-14T06:31:34.949 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:34.949 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:31:34.950 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:34.950 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:34.950 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:34.950 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:115196 3.1 3:856be886:::rbd_data.112d4d24c558.0000000000000004:29 [read 2260992~16384 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e96)", 2024-09-14T06:31:34.950 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:30:17.939925+0000", 2024-09-14T06:31:34.950 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 76.993069702, 2024-09-14T06:31:34.950 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.074320137999999994, 2024-09-14T06:31:34.950 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:31:34.950 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:31:34.950 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:31:34.950 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:31:34.950 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:31:34.951 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:34.951 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 115196 2024-09-14T06:31:34.951 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:34.951 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:31:34.951 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:34.951 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:31:34.951 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.939925+0000", 2024-09-14T06:31:34.951 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:34.951 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:34.951 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:34.951 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:31:34.952 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.939925+0000", 2024-09-14T06:31:34.952 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:34.952 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:34.952 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:34.952 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:31:34.952 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.939926+0000", 2024-09-14T06:31:34.952 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.9999999999999998e-07 2024-09-14T06:31:34.952 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:34.952 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:34.952 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:31:34.952 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.939928+0000", 2024-09-14T06:31:34.952 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.9549999999999999e-06 2024-09-14T06:31:34.953 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:34.953 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:34.953 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:31:34.953 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.939934+0000", 2024-09-14T06:31:34.953 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.127e-06 2024-09-14T06:31:34.953 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:34.953 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:34.953 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:31:34.953 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.939939+0000", 2024-09-14T06:31:34.953 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.775e-06 2024-09-14T06:31:34.953 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:34.953 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:34.954 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:31:34.954 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.014075+0000", 2024-09-14T06:31:34.954 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.074136598999999997 2024-09-14T06:31:34.954 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:34.954 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:34.954 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:31:34.954 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.014162+0000", 2024-09-14T06:31:34.954 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.6477999999999994e-05 2024-09-14T06:31:34.954 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:34.954 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:34.954 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:31:34.954 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.014246+0000", 2024-09-14T06:31:34.955 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.3703999999999997e-05 2024-09-14T06:31:34.955 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:34.955 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:31:34.955 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:34.955 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:34.955 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:34.955 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:115197 3.1 3:856be886:::rbd_data.112d4d24c558.0000000000000004:29 [read 2506752~16384 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e96)", 2024-09-14T06:31:34.955 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:30:17.939969+0000", 2024-09-14T06:31:34.955 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 76.993026216999993, 2024-09-14T06:31:34.955 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.074528544000000002, 2024-09-14T06:31:34.955 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:31:34.955 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:31:34.956 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:31:34.956 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:31:34.956 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:31:34.956 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:34.956 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 115197 2024-09-14T06:31:34.956 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:34.956 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:31:34.956 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:34.956 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:31:34.956 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.939969+0000", 2024-09-14T06:31:34.956 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:34.957 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:34.957 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:34.957 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:31:34.957 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.939969+0000", 2024-09-14T06:31:34.957 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:34.957 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:34.957 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:34.957 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:31:34.957 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.939970+0000", 2024-09-14T06:31:34.957 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.3600000000000003e-07 2024-09-14T06:31:34.957 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:34.958 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:34.958 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:31:34.958 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.939972+0000", 2024-09-14T06:31:34.958 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.3180000000000001e-06 2024-09-14T06:31:34.958 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:34.958 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:34.958 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:31:34.958 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.939979+0000", 2024-09-14T06:31:34.958 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.8800000000000002e-06 2024-09-14T06:31:34.958 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:34.958 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:34.958 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:31:34.958 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.939984+0000", 2024-09-14T06:31:34.959 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.3510000000000001e-06 2024-09-14T06:31:34.959 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:34.959 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:34.959 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:31:34.959 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.014302+0000", 2024-09-14T06:31:34.959 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.074317416999999997 2024-09-14T06:31:34.959 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:34.959 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:34.959 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:31:34.959 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.014392+0000", 2024-09-14T06:31:34.959 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.0181999999999998e-05 2024-09-14T06:31:34.959 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:34.960 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:34.960 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:31:34.960 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.014497+0000", 2024-09-14T06:31:34.960 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00010576 2024-09-14T06:31:34.960 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:34.960 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:31:34.960 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:34.960 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:34.960 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:34.960 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:115198 3.1 3:856be886:::rbd_data.112d4d24c558.0000000000000004:29 [read 2752512~16384 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e96)", 2024-09-14T06:31:34.960 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:30:17.940004+0000", 2024-09-14T06:31:34.961 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 76.992990937000002, 2024-09-14T06:31:34.961 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.074736367999999997, 2024-09-14T06:31:34.961 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:31:34.961 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:31:34.961 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:31:34.961 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:31:34.961 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:31:34.961 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:34.961 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 115198 2024-09-14T06:31:34.961 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:34.961 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:31:34.962 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:34.962 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:31:34.962 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940004+0000", 2024-09-14T06:31:34.962 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:34.962 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:34.962 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:34.962 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:31:34.962 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940004+0000", 2024-09-14T06:31:34.962 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:34.962 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:34.962 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:34.962 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:31:34.962 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940005+0000", 2024-09-14T06:31:34.963 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.2200000000000004e-07 2024-09-14T06:31:34.963 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:34.963 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:34.963 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:31:34.963 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940007+0000", 2024-09-14T06:31:34.963 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.322e-06 2024-09-14T06:31:34.963 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:34.963 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:34.963 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:31:34.963 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940014+0000", 2024-09-14T06:31:34.963 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.6220000000000003e-06 2024-09-14T06:31:34.963 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:34.964 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:34.964 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:31:34.964 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940019+0000", 2024-09-14T06:31:34.964 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.6380000000000001e-06 2024-09-14T06:31:34.964 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:34.964 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:34.964 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:31:34.964 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.014562+0000", 2024-09-14T06:31:34.964 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.074542320999999995 2024-09-14T06:31:34.964 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:34.964 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:34.964 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:31:34.964 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.014646+0000", 2024-09-14T06:31:34.965 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.4079999999999995e-05 2024-09-14T06:31:34.965 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:34.965 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:34.965 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:31:34.965 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.014741+0000", 2024-09-14T06:31:34.965 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.4763000000000001e-05 2024-09-14T06:31:34.965 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:34.965 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:31:34.965 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:34.965 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:34.965 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:34.965 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:115199 3.1 3:856be886:::rbd_data.112d4d24c558.0000000000000004:29 [read 2998272~16384 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e96)", 2024-09-14T06:31:34.965 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:30:17.940040+0000", 2024-09-14T06:31:34.966 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 76.992955182000003, 2024-09-14T06:31:34.966 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.074957134999999994, 2024-09-14T06:31:34.966 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:31:34.966 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:31:34.966 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:31:34.966 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:31:34.966 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:31:34.966 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:34.966 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 115199 2024-09-14T06:31:34.966 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:34.966 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:31:34.966 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:34.966 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:31:34.967 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940040+0000", 2024-09-14T06:31:34.967 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:34.967 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:34.967 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:34.967 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:31:34.967 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940040+0000", 2024-09-14T06:31:34.967 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:34.967 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:34.967 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:34.967 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:31:34.967 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940041+0000", 2024-09-14T06:31:34.967 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.4000000000000001e-07 2024-09-14T06:31:34.968 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:34.968 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:34.968 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:31:34.968 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940046+0000", 2024-09-14T06:31:34.968 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.7250000000000002e-06 2024-09-14T06:31:34.968 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:34.968 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:34.968 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:31:34.968 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940053+0000", 2024-09-14T06:31:34.968 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.6109999999999999e-06 2024-09-14T06:31:34.968 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:34.968 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:34.968 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:31:34.969 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940058+0000", 2024-09-14T06:31:34.969 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.1590000000000001e-06 2024-09-14T06:31:34.969 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:34.969 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:34.969 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:31:34.969 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.014768+0000", 2024-09-14T06:31:34.969 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.074709624000000002 2024-09-14T06:31:34.972 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:34.972 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:34.972 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:31:34.972 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.014907+0000", 2024-09-14T06:31:34.972 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000139665 2024-09-14T06:31:34.972 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:34.972 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:34.973 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:31:34.973 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.014997+0000", 2024-09-14T06:31:34.973 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.9710999999999995e-05 2024-09-14T06:31:34.973 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:34.973 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:31:34.973 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:34.973 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:34.973 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:34.973 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:115200 3.1 3:856be886:::rbd_data.112d4d24c558.0000000000000004:29 [read 3244032~16384 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e96)", 2024-09-14T06:31:34.973 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:30:17.940080+0000", 2024-09-14T06:31:34.973 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 76.992914976999998, 2024-09-14T06:31:34.973 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.075137291999999994, 2024-09-14T06:31:34.974 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:31:34.974 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:31:34.974 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:31:34.974 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:31:34.974 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:31:34.974 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:34.974 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 115200 2024-09-14T06:31:34.974 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:34.974 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:31:34.974 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:34.974 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:31:34.974 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940080+0000", 2024-09-14T06:31:34.974 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:34.975 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:34.975 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:34.975 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:31:34.975 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940080+0000", 2024-09-14T06:31:34.975 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:34.975 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:34.975 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:34.975 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:31:34.975 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940081+0000", 2024-09-14T06:31:34.975 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.0800000000000004e-07 2024-09-14T06:31:34.975 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:34.975 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:34.975 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:31:34.976 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940083+0000", 2024-09-14T06:31:34.976 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.3920000000000001e-06 2024-09-14T06:31:34.976 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:34.976 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:34.976 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:31:34.976 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940091+0000", 2024-09-14T06:31:34.976 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.1389999999999995e-06 2024-09-14T06:31:34.976 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:34.976 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:34.976 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:31:34.976 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940097+0000", 2024-09-14T06:31:34.976 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.5550000000000002e-06 2024-09-14T06:31:34.977 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:34.977 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:34.977 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:31:34.977 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.015025+0000", 2024-09-14T06:31:34.977 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.074928579999999995 2024-09-14T06:31:34.977 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:34.977 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:34.977 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:31:34.977 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.015096+0000", 2024-09-14T06:31:34.977 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.0705999999999997e-05 2024-09-14T06:31:34.977 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:34.978 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:34.978 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:31:34.978 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.015217+0000", 2024-09-14T06:31:34.978 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00012131200000000001 2024-09-14T06:31:34.978 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:34.978 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:31:34.978 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:34.978 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:34.978 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:34.978 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:115201 3.1 3:856be886:::rbd_data.112d4d24c558.0000000000000004:29 [read 3489792~16384 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e96)", 2024-09-14T06:31:34.978 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:30:17.940113+0000", 2024-09-14T06:31:34.979 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 76.992882203999997, 2024-09-14T06:31:34.979 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.075246899000000006, 2024-09-14T06:31:34.979 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:31:34.979 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:31:34.979 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:31:34.979 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:31:34.979 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:31:34.979 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:34.979 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 115201 2024-09-14T06:31:34.979 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:34.979 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:31:34.980 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:34.980 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:31:34.980 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940113+0000", 2024-09-14T06:31:34.980 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:34.980 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:34.980 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:34.980 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:31:34.980 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940113+0000", 2024-09-14T06:31:34.980 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:34.980 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:34.980 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:34.980 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:31:34.981 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940114+0000", 2024-09-14T06:31:34.981 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.1500000000000004e-07 2024-09-14T06:31:34.981 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:34.981 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:34.981 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:31:34.981 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940116+0000", 2024-09-14T06:31:34.981 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.2809999999999998e-06 2024-09-14T06:31:34.981 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:34.981 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:34.981 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:31:34.981 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940123+0000", 2024-09-14T06:31:34.981 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.9229999999999996e-06 2024-09-14T06:31:34.982 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:34.982 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:34.982 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:31:34.982 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940128+0000", 2024-09-14T06:31:34.982 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.7790000000000003e-06 2024-09-14T06:31:34.982 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:34.982 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:34.982 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:31:34.982 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.015253+0000", 2024-09-14T06:31:34.982 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.075125792999999996 2024-09-14T06:31:34.982 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:34.982 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:34.983 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:31:34.983 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.015297+0000", 2024-09-14T06:31:34.983 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.3847000000000002e-05 2024-09-14T06:31:34.983 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:34.983 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:34.983 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:31:34.983 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.015360+0000", 2024-09-14T06:31:34.983 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.2661000000000002e-05 2024-09-14T06:31:34.983 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:34.983 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:31:34.983 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:34.983 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:34.983 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:34.984 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:115202 3.1 3:856be886:::rbd_data.112d4d24c558.0000000000000004:29 [read 3735552~16384 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e96)", 2024-09-14T06:31:34.984 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:30:17.940135+0000", 2024-09-14T06:31:34.984 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 76.992860425000003, 2024-09-14T06:31:34.984 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.075451105000000004, 2024-09-14T06:31:34.984 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:31:34.984 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:31:34.984 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:31:34.984 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:31:34.984 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:31:34.984 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:34.984 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 115202 2024-09-14T06:31:34.985 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:34.985 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:31:34.985 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:34.985 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:31:34.985 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940135+0000", 2024-09-14T06:31:34.985 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:34.985 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:34.985 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:34.985 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:31:34.985 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940135+0000", 2024-09-14T06:31:34.985 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:34.985 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:34.986 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:34.986 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:31:34.986 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940135+0000", 2024-09-14T06:31:34.986 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.0999999999999999e-07 2024-09-14T06:31:34.986 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:34.986 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:34.986 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:31:34.986 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940137+0000", 2024-09-14T06:31:34.986 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.0020000000000001e-06 2024-09-14T06:31:34.986 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:34.986 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:34.986 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:31:34.986 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940143+0000", 2024-09-14T06:31:34.987 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.9850000000000003e-06 2024-09-14T06:31:34.987 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:34.987 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:34.987 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:31:34.987 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940148+0000", 2024-09-14T06:31:34.987 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.9880000000000004e-06 2024-09-14T06:31:34.987 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:34.987 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:34.987 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:31:34.987 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.015400+0000", 2024-09-14T06:31:34.987 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.075252186999999998 2024-09-14T06:31:34.987 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:34.987 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:34.988 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:31:34.988 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.015481+0000", 2024-09-14T06:31:34.988 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.0698000000000006e-05 2024-09-14T06:31:34.988 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:34.988 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:34.988 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:31:34.988 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.015586+0000", 2024-09-14T06:31:34.988 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00010483499999999999 2024-09-14T06:31:34.988 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:34.988 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:31:34.988 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:34.988 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:34.989 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:34.989 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:115203 3.1 3:856be886:::rbd_data.112d4d24c558.0000000000000004:29 [read 3981312~16384 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e96)", 2024-09-14T06:31:34.989 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:30:17.940156+0000", 2024-09-14T06:31:34.989 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 76.992839200000006, 2024-09-14T06:31:34.989 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.075620860999999998, 2024-09-14T06:31:34.989 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:31:34.989 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:31:34.989 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:31:34.989 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:31:34.989 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:31:34.989 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:34.989 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 115203 2024-09-14T06:31:34.989 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:34.990 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:31:34.990 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:34.990 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:31:34.990 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940156+0000", 2024-09-14T06:31:34.990 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:34.990 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:34.990 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:34.990 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:31:34.990 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940156+0000", 2024-09-14T06:31:34.990 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:34.990 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:34.990 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:34.991 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:31:34.991 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940156+0000", 2024-09-14T06:31:34.991 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.1500000000000005e-07 2024-09-14T06:31:34.991 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:34.991 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:34.991 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:31:34.991 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940158+0000", 2024-09-14T06:31:34.991 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.967e-06 2024-09-14T06:31:34.991 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:34.991 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:34.991 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:31:34.992 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940165+0000", 2024-09-14T06:31:34.992 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.6939999999999998e-06 2024-09-14T06:31:34.992 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:34.992 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:34.992 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:31:34.992 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940170+0000", 2024-09-14T06:31:34.992 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.7539999999999997e-06 2024-09-14T06:31:34.992 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:34.992 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:34.992 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:31:34.992 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.015633+0000", 2024-09-14T06:31:34.992 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.075463334000000007 2024-09-14T06:31:34.992 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:34.993 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:34.993 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:31:34.993 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.015693+0000", 2024-09-14T06:31:34.993 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.9367000000000003e-05 2024-09-14T06:31:34.993 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:34.993 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:34.993 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:31:34.993 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.015777+0000", 2024-09-14T06:31:34.993 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.4229999999999998e-05 2024-09-14T06:31:34.993 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:34.993 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:31:34.993 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:34.993 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:34.993 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:34.994 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:115204 3.1 3:856be886:::rbd_data.112d4d24c558.0000000000000004:29 [read 4227072~16384 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e96)", 2024-09-14T06:31:34.994 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:30:17.940176+0000", 2024-09-14T06:31:34.994 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 76.992818666999995, 2024-09-14T06:31:34.994 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.075785505000000003, 2024-09-14T06:31:34.994 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:31:34.994 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:31:34.994 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:31:34.994 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:31:34.994 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:31:34.994 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:34.994 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 115204 2024-09-14T06:31:34.995 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:34.995 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:31:34.995 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:34.995 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:31:34.995 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940176+0000", 2024-09-14T06:31:34.995 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:34.995 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:34.995 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:34.995 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:31:34.995 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940176+0000", 2024-09-14T06:31:34.995 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:34.995 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:34.995 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:34.996 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:31:34.996 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940177+0000", 2024-09-14T06:31:34.996 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.7899999999999999e-07 2024-09-14T06:31:34.996 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:34.996 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:34.996 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:31:34.996 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940179+0000", 2024-09-14T06:31:34.996 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.103e-06 2024-09-14T06:31:34.996 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:34.996 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:34.996 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:31:34.996 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940185+0000", 2024-09-14T06:31:34.997 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.7239999999999997e-06 2024-09-14T06:31:34.997 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:34.997 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:34.997 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:31:34.997 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940190+0000", 2024-09-14T06:31:34.997 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.7269999999999998e-06 2024-09-14T06:31:34.997 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:34.997 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:34.997 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:31:34.997 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.015814+0000", 2024-09-14T06:31:34.997 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.075624114000000006 2024-09-14T06:31:34.997 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:34.997 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:34.998 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:31:34.998 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.015862+0000", 2024-09-14T06:31:34.998 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.8770999999999998e-05 2024-09-14T06:31:34.998 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:34.998 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:34.998 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:31:34.998 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.015962+0000", 2024-09-14T06:31:34.998 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.9586999999999994e-05 2024-09-14T06:31:34.998 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:34.998 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:31:34.998 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:34.998 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:34.998 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:34.998 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:115205 3.1 3:856be886:::rbd_data.112d4d24c558.0000000000000004:29 [read 4472832~16384 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e96)", 2024-09-14T06:31:34.999 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:30:17.940197+0000", 2024-09-14T06:31:34.999 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 76.992797957999997, 2024-09-14T06:31:34.999 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.076068920999999998, 2024-09-14T06:31:34.999 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:31:34.999 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:31:34.999 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:31:34.999 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:31:34.999 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:31:34.999 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:34.999 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 115205 2024-09-14T06:31:34.999 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:34.999 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:31:34.999 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:35.000 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:31:35.000 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940197+0000", 2024-09-14T06:31:35.000 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:35.000 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:35.000 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:35.000 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:31:35.000 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940197+0000", 2024-09-14T06:31:35.000 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:35.000 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:35.000 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:35.000 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:31:35.001 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940198+0000", 2024-09-14T06:31:35.001 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.5999999999999999e-07 2024-09-14T06:31:35.001 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:35.001 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:35.001 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:31:35.001 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940200+0000", 2024-09-14T06:31:35.001 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.0899999999999999e-06 2024-09-14T06:31:35.001 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:35.001 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:35.001 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:31:35.001 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940206+0000", 2024-09-14T06:31:35.001 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.8710000000000002e-06 2024-09-14T06:31:35.001 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:35.002 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:35.002 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:31:35.002 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940211+0000", 2024-09-14T06:31:35.002 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.9230000000000001e-06 2024-09-14T06:31:35.002 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:35.002 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:35.002 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:31:35.002 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.016059+0000", 2024-09-14T06:31:35.002 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.075848051999999999 2024-09-14T06:31:35.002 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:35.002 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:35.002 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:31:35.003 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.016151+0000", 2024-09-14T06:31:35.003 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.1982000000000001e-05 2024-09-14T06:31:35.003 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:35.003 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:35.003 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:31:35.003 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.016266+0000", 2024-09-14T06:31:35.003 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000115543 2024-09-14T06:31:35.003 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:35.003 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:31:35.003 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:35.003 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:35.003 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:35.004 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:115206 3.1 3:856be886:::rbd_data.112d4d24c558.0000000000000004:29 [read 4718592~16384 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e96)", 2024-09-14T06:31:35.004 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:30:17.940223+0000", 2024-09-14T06:31:35.004 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 76.992771868000005, 2024-09-14T06:31:35.004 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.076265115999999994, 2024-09-14T06:31:35.004 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:31:35.004 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:31:35.004 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:31:35.004 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:31:35.004 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:31:35.004 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:35.004 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 115206 2024-09-14T06:31:35.004 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:35.005 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:31:35.005 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:35.005 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:31:35.005 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940223+0000", 2024-09-14T06:31:35.005 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:35.005 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:35.005 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:35.005 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:31:35.005 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940223+0000", 2024-09-14T06:31:35.005 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:35.005 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:35.005 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:35.006 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:31:35.006 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940224+0000", 2024-09-14T06:31:35.006 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.0399999999999996e-07 2024-09-14T06:31:35.006 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:35.006 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:35.006 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:31:35.006 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940226+0000", 2024-09-14T06:31:35.006 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.1780000000000002e-06 2024-09-14T06:31:35.006 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:35.006 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:35.006 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:31:35.006 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940232+0000", 2024-09-14T06:31:35.006 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.5010000000000001e-06 2024-09-14T06:31:35.007 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:35.007 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:35.007 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:31:35.007 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940237+0000", 2024-09-14T06:31:35.007 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.0250000000000002e-06 2024-09-14T06:31:35.007 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:35.007 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:35.007 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:31:35.007 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.016305+0000", 2024-09-14T06:31:35.007 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.076067262999999996 2024-09-14T06:31:35.007 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:35.007 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:35.008 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:31:35.008 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.016383+0000", 2024-09-14T06:31:35.008 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.8578000000000006e-05 2024-09-14T06:31:35.008 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:35.008 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:35.008 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:31:35.008 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.016488+0000", 2024-09-14T06:31:35.008 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000105067 2024-09-14T06:31:35.008 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:35.008 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:31:35.008 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:35.008 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:35.009 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:35.009 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:115207 3.1 3:856be886:::rbd_data.112d4d24c558.0000000000000004:29 [read 4964352~16384 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e96)", 2024-09-14T06:31:35.009 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:30:17.940244+0000", 2024-09-14T06:31:35.009 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 76.992750752000006, 2024-09-14T06:31:35.009 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.076371154999999996, 2024-09-14T06:31:35.009 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:31:35.009 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:31:35.009 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:31:35.009 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:31:35.009 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:31:35.009 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:35.009 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 115207 2024-09-14T06:31:35.010 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:35.010 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:31:35.010 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:35.010 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:31:35.010 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940244+0000", 2024-09-14T06:31:35.010 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:35.010 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:35.010 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:35.010 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:31:35.010 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940244+0000", 2024-09-14T06:31:35.010 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:35.010 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:35.011 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:35.011 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:31:35.011 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940245+0000", 2024-09-14T06:31:35.011 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.1200000000000003e-07 2024-09-14T06:31:35.011 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:35.011 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:35.011 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:31:35.011 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940247+0000", 2024-09-14T06:31:35.011 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.0030000000000001e-06 2024-09-14T06:31:35.011 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:35.011 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:35.011 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:31:35.011 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940252+0000", 2024-09-14T06:31:35.012 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.5520000000000004e-06 2024-09-14T06:31:35.012 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:35.012 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:35.012 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:31:35.012 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940257+0000", 2024-09-14T06:31:35.012 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.6759999999999998e-06 2024-09-14T06:31:35.012 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:35.012 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:35.012 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:31:35.012 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.016521+0000", 2024-09-14T06:31:35.012 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.076263499999999998 2024-09-14T06:31:35.012 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:35.013 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:35.013 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:31:35.013 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.016562+0000", 2024-09-14T06:31:35.013 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.0967000000000003e-05 2024-09-14T06:31:35.013 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:35.013 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:35.013 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:31:35.013 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.016616+0000", 2024-09-14T06:31:35.013 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.3944999999999999e-05 2024-09-14T06:31:35.013 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:35.013 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:31:35.013 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:35.013 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:35.014 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:35.014 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:115208 3.1 3:856be886:::rbd_data.112d4d24c558.0000000000000004:29 [read 5210112~16384 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e96)", 2024-09-14T06:31:35.014 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:30:17.940264+0000", 2024-09-14T06:31:35.014 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 76.992730894999994, 2024-09-14T06:31:35.014 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.076558288000000002, 2024-09-14T06:31:35.014 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:31:35.014 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:31:35.014 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:31:35.014 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:31:35.014 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:31:35.014 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:35.014 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 115208 2024-09-14T06:31:35.014 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:35.015 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:31:35.015 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:35.015 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:31:35.015 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940264+0000", 2024-09-14T06:31:35.015 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:35.015 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:35.015 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:35.015 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:31:35.015 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940264+0000", 2024-09-14T06:31:35.015 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:35.015 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:35.015 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:35.015 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:31:35.016 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940265+0000", 2024-09-14T06:31:35.016 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.0500000000000004e-07 2024-09-14T06:31:35.016 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:35.016 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:35.016 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:31:35.016 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940267+0000", 2024-09-14T06:31:35.016 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.9810000000000002e-06 2024-09-14T06:31:35.016 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:35.016 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:35.016 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:31:35.016 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940272+0000", 2024-09-14T06:31:35.016 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.6999999999999996e-06 2024-09-14T06:31:35.016 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:35.017 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:35.017 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:31:35.017 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940277+0000", 2024-09-14T06:31:35.017 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.6480000000000002e-06 2024-09-14T06:31:35.017 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:35.017 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:35.017 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:31:35.017 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.016650+0000", 2024-09-14T06:31:35.017 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.076373089000000005 2024-09-14T06:31:35.017 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:35.017 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:35.017 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:31:35.017 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.016724+0000", 2024-09-14T06:31:35.018 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.4105999999999998e-05 2024-09-14T06:31:35.018 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:35.018 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:35.018 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:31:35.018 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.016823+0000", 2024-09-14T06:31:35.018 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.8258999999999996e-05 2024-09-14T06:31:35.018 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:35.018 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:31:35.018 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:35.018 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:35.018 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:35.018 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:115209 3.1 3:856be886:::rbd_data.112d4d24c558.0000000000000004:29 [read 5455872~16384 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e96)", 2024-09-14T06:31:35.018 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:30:17.940283+0000", 2024-09-14T06:31:35.019 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 76.992711869000004, 2024-09-14T06:31:35.019 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.076724029999999999, 2024-09-14T06:31:35.019 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:31:35.019 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:31:35.019 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:31:35.019 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:31:35.019 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:31:35.019 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:35.019 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 115209 2024-09-14T06:31:35.019 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:35.019 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:31:35.020 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:35.020 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:31:35.020 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940283+0000", 2024-09-14T06:31:35.020 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:35.020 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:35.020 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:35.020 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:31:35.020 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940283+0000", 2024-09-14T06:31:35.020 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:35.020 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:35.020 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:35.020 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:31:35.021 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940284+0000", 2024-09-14T06:31:35.021 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.6700000000000003e-07 2024-09-14T06:31:35.021 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:35.021 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:35.021 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:31:35.021 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940286+0000", 2024-09-14T06:31:35.021 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.9520000000000001e-06 2024-09-14T06:31:35.021 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:35.021 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:35.021 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:31:35.021 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940292+0000", 2024-09-14T06:31:35.021 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.7339999999999996e-06 2024-09-14T06:31:35.022 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:35.022 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:35.022 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:31:35.022 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940296+0000", 2024-09-14T06:31:35.022 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.7700000000000001e-06 2024-09-14T06:31:35.022 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:35.022 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:35.022 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:31:35.022 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.016859+0000", 2024-09-14T06:31:35.022 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.07656317 2024-09-14T06:31:35.022 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:35.023 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:35.023 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:31:35.023 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.016929+0000", 2024-09-14T06:31:35.023 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.9679999999999997e-05 2024-09-14T06:31:35.023 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:35.023 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:35.023 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:31:35.023 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.017007+0000", 2024-09-14T06:31:35.023 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.8157000000000004e-05 2024-09-14T06:31:35.023 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:35.023 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:31:35.023 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:35.023 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:35.024 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:35.024 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:115210 3.1 3:856be886:::rbd_data.112d4d24c558.0000000000000004:29 [read 5701632~16384 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e96)", 2024-09-14T06:31:35.024 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:30:17.940303+0000", 2024-09-14T06:31:35.024 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 76.992692352000006, 2024-09-14T06:31:35.024 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.076821358000000006, 2024-09-14T06:31:35.024 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:31:35.024 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:31:35.024 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:31:35.024 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:31:35.024 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:31:35.024 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:35.024 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 115210 2024-09-14T06:31:35.024 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:35.025 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:31:35.025 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:35.025 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:31:35.025 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940303+0000", 2024-09-14T06:31:35.025 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:35.025 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:35.025 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:35.025 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:31:35.025 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940303+0000", 2024-09-14T06:31:35.025 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:35.025 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:35.025 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:35.026 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:31:35.026 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940303+0000", 2024-09-14T06:31:35.026 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.3000000000000001e-07 2024-09-14T06:31:35.026 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:35.026 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:35.026 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:31:35.026 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940306+0000", 2024-09-14T06:31:35.026 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.255e-06 2024-09-14T06:31:35.026 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:35.026 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:35.026 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:31:35.026 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940312+0000", 2024-09-14T06:31:35.027 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.9710000000000001e-06 2024-09-14T06:31:35.027 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:35.027 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:35.027 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:31:35.027 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940316+0000", 2024-09-14T06:31:35.027 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.5340000000000001e-06 2024-09-14T06:31:35.027 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:35.027 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:35.027 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:31:35.027 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.017033+0000", 2024-09-14T06:31:35.027 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.076717169000000002 2024-09-14T06:31:35.027 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:35.028 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:35.028 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:31:35.028 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.017073+0000", 2024-09-14T06:31:35.028 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.9582000000000002e-05 2024-09-14T06:31:35.028 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:35.028 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:35.028 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:31:35.028 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.017124+0000", 2024-09-14T06:31:35.028 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.1316999999999997e-05 2024-09-14T06:31:35.028 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:35.028 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:31:35.028 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:35.029 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:35.029 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:35.029 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:115219 3.1 3:856be886:::rbd_data.112d4d24c558.0000000000000004:29 [read 65536~16384 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e96)", 2024-09-14T06:31:35.029 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:30:17.940323+0000", 2024-09-14T06:31:35.029 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 76.992672490999993, 2024-09-14T06:31:35.029 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.076915205, 2024-09-14T06:31:35.029 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:31:35.029 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:31:35.029 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:31:35.029 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:31:35.029 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:31:35.029 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:35.030 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 115219 2024-09-14T06:31:35.030 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:35.030 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:31:35.030 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:35.030 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:31:35.030 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940323+0000", 2024-09-14T06:31:35.030 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:35.030 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:35.030 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:35.030 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:31:35.030 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940323+0000", 2024-09-14T06:31:35.030 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:35.031 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:35.031 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:35.031 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:31:35.031 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940323+0000", 2024-09-14T06:31:35.031 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.0800000000000005e-07 2024-09-14T06:31:35.031 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:35.031 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:35.031 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:31:35.031 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940328+0000", 2024-09-14T06:31:35.031 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.887e-06 2024-09-14T06:31:35.031 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:35.031 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:35.032 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:31:35.032 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940334+0000", 2024-09-14T06:31:35.032 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.3269999999999997e-06 2024-09-14T06:31:35.032 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:35.032 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:35.032 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:31:35.032 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940339+0000", 2024-09-14T06:31:35.032 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.7960000000000003e-06 2024-09-14T06:31:35.032 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:35.032 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:35.032 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:31:35.032 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.017148+0000", 2024-09-14T06:31:35.032 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.076809125000000006 2024-09-14T06:31:35.033 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:35.033 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:35.033 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:31:35.033 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.017186+0000", 2024-09-14T06:31:35.033 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.7283000000000002e-05 2024-09-14T06:31:35.033 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:35.033 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:35.033 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:31:35.033 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.017238+0000", 2024-09-14T06:31:35.033 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.2278999999999999e-05 2024-09-14T06:31:35.033 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:35.033 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:31:35.034 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:35.034 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:35.034 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:35.034 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:115220 3.1 3:856be886:::rbd_data.112d4d24c558.0000000000000004:29 [read 311296~16384 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e96)", 2024-09-14T06:31:35.034 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:30:17.940346+0000", 2024-09-14T06:31:35.034 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 76.992649361999995, 2024-09-14T06:31:35.034 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.077033293000000003, 2024-09-14T06:31:35.034 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:31:35.034 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:31:35.034 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:31:35.034 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:31:35.034 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:31:35.034 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:35.035 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 115220 2024-09-14T06:31:35.035 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:35.035 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:31:35.035 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:35.035 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:31:35.035 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940346+0000", 2024-09-14T06:31:35.035 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:35.035 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:35.035 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:35.035 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:31:35.035 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940346+0000", 2024-09-14T06:31:35.035 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:35.036 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:35.036 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:35.036 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:31:35.036 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940346+0000", 2024-09-14T06:31:35.036 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.4300000000000003e-07 2024-09-14T06:31:35.036 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:35.036 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:35.036 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:31:35.036 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940348+0000", 2024-09-14T06:31:35.036 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.0030000000000001e-06 2024-09-14T06:31:35.036 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:35.037 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:35.037 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:31:35.037 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.004655+0000", 2024-09-14T06:31:35.037 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.064306323999999998 2024-09-14T06:31:35.037 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:35.037 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:35.037 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:31:35.037 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.004671+0000", 2024-09-14T06:31:35.037 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.6589999999999999e-05 2024-09-14T06:31:35.037 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:35.037 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:35.037 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:31:35.037 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.017262+0000", 2024-09-14T06:31:35.038 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.012590570000000001 2024-09-14T06:31:35.038 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:35.038 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:35.038 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:31:35.038 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.017321+0000", 2024-09-14T06:31:35.038 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.9210999999999999e-05 2024-09-14T06:31:35.038 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:35.038 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:35.038 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:31:35.038 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.017379+0000", 2024-09-14T06:31:35.038 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.8051999999999999e-05 2024-09-14T06:31:35.038 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:35.038 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:31:35.039 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:35.039 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:35.039 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:31:35.039 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-14T06:31:35.040 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-14T06:31:35.041 INFO:tasks.ceph.osd.6.smithi138.stderr:2024-09-14T06:31:35.039+0000 7f8cd8849640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33620) UID: 0 2024-09-14T06:31:35.141 INFO:tasks.ceph.osd.7.smithi138.stderr:2024-09-14T06:31:35.140+0000 7f3c6a21d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33623) UID: 0 2024-09-14T06:31:35.183 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-14T06:31:35.183 INFO:teuthology.orchestra.run.smithi094.stdout: "ops": [], 2024-09-14T06:31:35.183 INFO:teuthology.orchestra.run.smithi094.stdout: "num_ops": 0 2024-09-14T06:31:35.183 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-14T06:31:35.191 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-14T06:31:35.242 INFO:tasks.ceph.osd.3.smithi094.stderr:2024-09-14T06:31:35.240+0000 7feeeafe8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33655) UID: 0 2024-09-14T06:31:35.294 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-14T06:31:35.294 INFO:teuthology.orchestra.run.smithi094.stdout: "ops": [], 2024-09-14T06:31:35.294 INFO:teuthology.orchestra.run.smithi094.stdout: "complaint_time": 30, 2024-09-14T06:31:35.294 INFO:teuthology.orchestra.run.smithi094.stdout: "num_blocked_ops": 0 2024-09-14T06:31:35.294 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-14T06:31:35.303 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-14T06:31:35.343 INFO:tasks.ceph.osd.6.smithi138.stderr:2024-09-14T06:31:35.342+0000 7f8cd8849640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33620) UID: 0 2024-09-14T06:31:35.444 INFO:tasks.ceph.osd.7.smithi138.stderr:2024-09-14T06:31:35.443+0000 7f3c6a21d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33623) UID: 0 2024-09-14T06:31:35.448 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-14T06:31:35.448 INFO:teuthology.orchestra.run.smithi094.stdout: "size": 20, 2024-09-14T06:31:35.448 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 600, 2024-09-14T06:31:35.449 INFO:teuthology.orchestra.run.smithi094.stdout: "ops": [ 2024-09-14T06:31:35.449 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:35.449 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:14918 3.0 3:05c39535:::rbd_data.112d92027590.0000000000000009:f [read 795648~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:31:35.449 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:26.083834+0000", 2024-09-14T06:31:35.449 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 129.36282758499999, 2024-09-14T06:31:35.449 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.041776287000000002, 2024-09-14T06:31:35.449 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:31:35.449 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:31:35.449 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:31:35.449 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:31:35.449 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:31:35.449 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:35.449 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 14918 2024-09-14T06:31:35.450 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:35.450 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:31:35.450 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:35.450 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:31:35.450 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.083834+0000", 2024-09-14T06:31:35.450 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:35.450 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:35.450 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:35.450 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:31:35.450 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.083834+0000", 2024-09-14T06:31:35.450 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:35.450 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:35.451 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:35.451 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:31:35.451 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.083834+0000", 2024-09-14T06:31:35.451 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.0100000000000001e-07 2024-09-14T06:31:35.451 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:35.451 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:35.451 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:31:35.451 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.083835+0000", 2024-09-14T06:31:35.451 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.3900000000000003e-07 2024-09-14T06:31:35.451 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:35.451 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:35.451 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:31:35.451 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.083838+0000", 2024-09-14T06:31:35.452 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.9830000000000001e-06 2024-09-14T06:31:35.452 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:35.452 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:35.452 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:31:35.452 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.083840+0000", 2024-09-14T06:31:35.452 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.4679999999999999e-06 2024-09-14T06:31:35.452 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:35.452 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:35.452 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:31:35.452 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.125455+0000", 2024-09-14T06:31:35.452 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.041614578999999999 2024-09-14T06:31:35.452 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:35.452 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:35.453 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:31:35.453 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.125496+0000", 2024-09-14T06:31:35.453 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.1047000000000002e-05 2024-09-14T06:31:35.453 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:35.453 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:35.453 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:31:35.453 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.125610+0000", 2024-09-14T06:31:35.453 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00011407 2024-09-14T06:31:35.453 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:35.453 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:31:35.453 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:35.453 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:35.454 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:35.454 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:14919 3.0 3:05c39535:::rbd_data.112d92027590.0000000000000009:f [read 805888~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:31:35.454 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:26.083843+0000", 2024-09-14T06:31:35.454 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 129.36281783699999, 2024-09-14T06:31:35.454 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.041992356000000002, 2024-09-14T06:31:35.454 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:31:35.454 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:31:35.454 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:31:35.454 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:31:35.454 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:31:35.454 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:35.454 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 14919 2024-09-14T06:31:35.454 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:35.455 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:31:35.455 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:35.455 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:31:35.455 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.083843+0000", 2024-09-14T06:31:35.455 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:35.455 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:35.455 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:35.455 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:31:35.455 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.083843+0000", 2024-09-14T06:31:35.455 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:35.456 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:35.456 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:35.456 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:31:35.456 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.083844+0000", 2024-09-14T06:31:35.456 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.3300000000000001e-07 2024-09-14T06:31:35.456 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:35.456 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:35.456 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:31:35.456 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.083845+0000", 2024-09-14T06:31:35.456 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.4729999999999999e-06 2024-09-14T06:31:35.456 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:35.456 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:35.456 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:31:35.457 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.083850+0000", 2024-09-14T06:31:35.457 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.0130000000000001e-06 2024-09-14T06:31:35.457 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:35.457 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:35.457 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:31:35.457 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.083854+0000", 2024-09-14T06:31:35.457 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.4759999999999998e-06 2024-09-14T06:31:35.457 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:35.457 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:35.457 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:31:35.457 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.125636+0000", 2024-09-14T06:31:35.458 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.041782683000000001 2024-09-14T06:31:35.458 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:35.458 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:35.458 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:31:35.458 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.125680+0000", 2024-09-14T06:31:35.458 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.3614e-05 2024-09-14T06:31:35.458 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:35.458 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:35.458 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:31:35.458 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.125836+0000", 2024-09-14T06:31:35.458 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000155864 2024-09-14T06:31:35.458 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:35.459 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:31:35.459 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:35.459 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:35.459 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:35.459 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:14920 3.0 3:05c39535:::rbd_data.112d92027590.0000000000000009:f [read 816128~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:31:35.459 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:26.083868+0000", 2024-09-14T06:31:35.459 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 129.362793176, 2024-09-14T06:31:35.459 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.042166771999999998, 2024-09-14T06:31:35.459 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:31:35.459 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:31:35.459 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:31:35.459 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:31:35.460 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:31:35.460 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:35.460 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 14920 2024-09-14T06:31:35.460 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:35.460 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:31:35.460 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:35.460 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:31:35.460 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.083868+0000", 2024-09-14T06:31:35.460 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:35.460 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:35.460 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:35.460 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:31:35.461 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.083868+0000", 2024-09-14T06:31:35.461 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:35.461 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:35.461 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:35.461 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:31:35.461 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.083869+0000", 2024-09-14T06:31:35.461 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.99e-07 2024-09-14T06:31:35.461 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:35.461 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:35.461 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:31:35.461 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.083870+0000", 2024-09-14T06:31:35.462 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.782e-06 2024-09-14T06:31:35.462 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:35.462 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:35.462 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:31:35.462 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.083876+0000", 2024-09-14T06:31:35.462 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.3809999999999998e-06 2024-09-14T06:31:35.462 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:35.462 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:35.462 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:31:35.462 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.083882+0000", 2024-09-14T06:31:35.462 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.9479999999999996e-06 2024-09-14T06:31:35.462 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:35.463 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:35.463 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:31:35.463 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.125878+0000", 2024-09-14T06:31:35.463 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.041996264999999998 2024-09-14T06:31:35.463 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:35.463 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:35.463 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:31:35.463 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.125929+0000", 2024-09-14T06:31:35.463 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.0498999999999999e-05 2024-09-14T06:31:35.463 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:35.463 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:35.464 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:31:35.464 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.126035+0000", 2024-09-14T06:31:35.464 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000106298 2024-09-14T06:31:35.464 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:35.464 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:31:35.464 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:35.464 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:35.464 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:35.464 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:14921 3.0 3:05c39535:::rbd_data.112d92027590.0000000000000009:f [read 826368~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:31:35.464 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:26.083889+0000", 2024-09-14T06:31:35.464 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 129.36277216600001, 2024-09-14T06:31:35.464 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.042435309999999997, 2024-09-14T06:31:35.465 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:31:35.465 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:31:35.465 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:31:35.465 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:31:35.465 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:31:35.465 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:35.465 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 14921 2024-09-14T06:31:35.465 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:35.465 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:31:35.465 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:35.465 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:31:35.466 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.083889+0000", 2024-09-14T06:31:35.466 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:35.466 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:35.466 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:35.466 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:31:35.466 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.083889+0000", 2024-09-14T06:31:35.466 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:35.466 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:35.466 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:35.466 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:31:35.466 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.086033+0000", 2024-09-14T06:31:35.466 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0021444179999999999 2024-09-14T06:31:35.467 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:35.467 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:35.467 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:31:35.467 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.086038+0000", 2024-09-14T06:31:35.467 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.0969999999999999e-06 2024-09-14T06:31:35.467 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:35.467 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:35.467 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:31:35.467 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.086046+0000", 2024-09-14T06:31:35.467 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.9440000000000005e-06 2024-09-14T06:31:35.467 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:35.467 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:35.467 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:31:35.468 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.086050+0000", 2024-09-14T06:31:35.468 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.69e-06 2024-09-14T06:31:35.468 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:35.468 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:35.468 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:31:35.468 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.126080+0000", 2024-09-14T06:31:35.468 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.040030237000000003 2024-09-14T06:31:35.468 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:35.468 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:35.468 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:31:35.469 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.126163+0000", 2024-09-14T06:31:35.469 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.2071999999999996e-05 2024-09-14T06:31:35.469 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:35.469 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:35.469 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:31:35.469 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.126324+0000", 2024-09-14T06:31:35.469 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00016185200000000001 2024-09-14T06:31:35.469 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:35.469 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:31:35.469 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:35.469 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:35.470 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:35.470 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:55627 3.0 3:06490baf:::rbd_data.112dca2f6ae1.0000000000000030:17 [read 2002944~4096 [fadvise_nocache]] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:31:35.470 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:44.599966+0000", 2024-09-14T06:31:35.470 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 110.84669521799999, 2024-09-14T06:31:35.470 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.041868546999999999, 2024-09-14T06:31:35.470 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:31:35.470 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:31:35.470 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:31:35.470 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:31:35.470 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:31:35.470 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:35.471 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 55627 2024-09-14T06:31:35.471 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:35.471 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:31:35.471 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:35.471 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:31:35.471 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.599966+0000", 2024-09-14T06:31:35.471 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:35.471 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:35.471 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:35.471 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:31:35.471 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.599966+0000", 2024-09-14T06:31:35.471 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:35.472 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:35.472 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:35.472 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:31:35.472 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.599966+0000", 2024-09-14T06:31:35.472 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.0900000000000001e-07 2024-09-14T06:31:35.472 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:35.472 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:35.472 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:31:35.472 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.599967+0000", 2024-09-14T06:31:35.472 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.64e-07 2024-09-14T06:31:35.472 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:35.472 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:35.473 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:31:35.473 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.599970+0000", 2024-09-14T06:31:35.473 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.9670000000000001e-06 2024-09-14T06:31:35.473 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:35.473 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:35.473 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:31:35.473 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.599973+0000", 2024-09-14T06:31:35.473 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.632e-06 2024-09-14T06:31:35.473 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:35.473 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:35.473 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:31:35.473 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.641665+0000", 2024-09-14T06:31:35.473 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.041692547000000003 2024-09-14T06:31:35.473 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:35.474 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:35.474 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:31:35.474 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.641740+0000", 2024-09-14T06:31:35.474 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.4559000000000002e-05 2024-09-14T06:31:35.474 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:35.474 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:35.474 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:31:35.474 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.641835+0000", 2024-09-14T06:31:35.474 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.4668999999999998e-05 2024-09-14T06:31:35.474 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:35.474 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:31:35.474 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:35.475 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:35.475 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:35.475 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:55628 3.0 3:06490baf:::rbd_data.112dca2f6ae1.0000000000000030:17 [read 2027520~4096 [fadvise_nocache]] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:31:35.475 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:44.599976+0000", 2024-09-14T06:31:35.475 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 110.846685431, 2024-09-14T06:31:35.475 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.041991621, 2024-09-14T06:31:35.475 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:31:35.475 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:31:35.475 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:31:35.475 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:31:35.475 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:31:35.475 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:35.475 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 55628 2024-09-14T06:31:35.475 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:35.476 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:31:35.476 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:35.476 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:31:35.476 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.599976+0000", 2024-09-14T06:31:35.476 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:35.476 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:35.476 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:35.476 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:31:35.476 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.599976+0000", 2024-09-14T06:31:35.476 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:35.476 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:35.476 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:35.476 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:31:35.477 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.599976+0000", 2024-09-14T06:31:35.477 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.1299999999999999e-07 2024-09-14T06:31:35.477 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:35.477 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:35.477 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:31:35.477 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.599977+0000", 2024-09-14T06:31:35.477 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.7999999999999993e-07 2024-09-14T06:31:35.477 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:35.477 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:35.477 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:31:35.477 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.599980+0000", 2024-09-14T06:31:35.477 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.01e-06 2024-09-14T06:31:35.478 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:35.478 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:35.478 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:31:35.478 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.599983+0000", 2024-09-14T06:31:35.478 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.5409999999999999e-06 2024-09-14T06:31:35.478 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:35.478 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:35.478 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:31:35.478 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.641869+0000", 2024-09-14T06:31:35.478 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.041886235000000001 2024-09-14T06:31:35.478 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:35.478 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:35.478 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:31:35.478 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.641913+0000", 2024-09-14T06:31:35.479 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.4357000000000002e-05 2024-09-14T06:31:35.479 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:35.479 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:35.479 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:31:35.479 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.641967+0000", 2024-09-14T06:31:35.479 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.4284999999999999e-05 2024-09-14T06:31:35.479 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:35.479 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:31:35.479 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:35.479 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:35.479 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:35.479 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:55629 3.0 3:06490baf:::rbd_data.112dca2f6ae1.0000000000000030:17 [read 2052096~4096 [fadvise_nocache]] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:31:35.480 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:44.599985+0000", 2024-09-14T06:31:35.480 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 110.846675725, 2024-09-14T06:31:35.480 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.042183375000000002, 2024-09-14T06:31:35.480 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:31:35.480 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:31:35.480 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:31:35.480 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:31:35.480 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:31:35.480 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:35.480 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 55629 2024-09-14T06:31:35.481 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:35.481 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:31:35.481 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:35.481 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:31:35.481 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.599985+0000", 2024-09-14T06:31:35.481 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:35.481 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:35.481 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:35.481 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:31:35.481 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.599985+0000", 2024-09-14T06:31:35.481 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:35.482 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:35.482 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:35.482 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:31:35.482 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.599986+0000", 2024-09-14T06:31:35.482 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.05e-07 2024-09-14T06:31:35.482 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:35.482 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:35.482 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:31:35.482 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.599987+0000", 2024-09-14T06:31:35.482 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.8400000000000002e-07 2024-09-14T06:31:35.482 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:35.483 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:35.483 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:31:35.483 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.599990+0000", 2024-09-14T06:31:35.483 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.5640000000000001e-06 2024-09-14T06:31:35.483 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:35.483 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:35.483 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:31:35.483 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.599993+0000", 2024-09-14T06:31:35.483 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.615e-06 2024-09-14T06:31:35.483 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:35.483 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:35.483 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:31:35.483 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.642003+0000", 2024-09-14T06:31:35.484 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.042010345999999997 2024-09-14T06:31:35.484 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:35.484 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:35.484 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:31:35.484 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.642080+0000", 2024-09-14T06:31:35.484 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.6538999999999993e-05 2024-09-14T06:31:35.484 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:35.484 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:35.484 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:31:35.484 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.642169+0000", 2024-09-14T06:31:35.484 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.9122000000000005e-05 2024-09-14T06:31:35.484 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:35.485 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:31:35.485 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:35.485 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:35.485 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:35.485 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:55630 3.0 3:06490baf:::rbd_data.112dca2f6ae1.0000000000000030:17 [read 2076672~4096 [fadvise_nocache]] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:31:35.485 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:44.599996+0000", 2024-09-14T06:31:35.485 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 110.846665247, 2024-09-14T06:31:35.485 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.042299615999999998, 2024-09-14T06:31:35.485 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:31:35.485 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:31:35.485 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:31:35.485 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:31:35.486 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:31:35.486 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:35.486 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 55630 2024-09-14T06:31:35.486 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:35.486 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:31:35.486 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:35.486 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:31:35.486 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.599996+0000", 2024-09-14T06:31:35.486 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:35.486 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:35.486 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:35.486 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:31:35.487 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.599996+0000", 2024-09-14T06:31:35.487 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:35.487 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:35.487 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:35.487 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:31:35.487 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.599996+0000", 2024-09-14T06:31:35.487 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.1500000000000001e-07 2024-09-14T06:31:35.487 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:35.487 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:35.487 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:31:35.487 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.599997+0000", 2024-09-14T06:31:35.487 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.33e-07 2024-09-14T06:31:35.488 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:35.488 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:35.488 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:31:35.488 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600000+0000", 2024-09-14T06:31:35.488 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.0240000000000002e-06 2024-09-14T06:31:35.488 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:35.488 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:35.488 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:31:35.488 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600003+0000", 2024-09-14T06:31:35.488 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.3640000000000002e-06 2024-09-14T06:31:35.488 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:35.488 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:35.489 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:31:35.489 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.642200+0000", 2024-09-14T06:31:35.489 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.042197571000000003 2024-09-14T06:31:35.489 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:35.489 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:35.489 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:31:35.566 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.642243+0000", 2024-09-14T06:31:35.566 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.2765000000000001e-05 2024-09-14T06:31:35.567 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:35.567 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:35.567 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:31:35.567 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.642296+0000", 2024-09-14T06:31:35.567 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.2744000000000002e-05 2024-09-14T06:31:35.567 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:35.567 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:31:35.567 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:35.567 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:35.567 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:35.568 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:55631 3.0 3:06490baf:::rbd_data.112dca2f6ae1.0000000000000030:17 [read 1048576~4096 [fadvise_nocache]] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:31:35.568 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:44.600006+0000", 2024-09-14T06:31:35.568 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 110.84665565900001, 2024-09-14T06:31:35.568 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.042487905999999999, 2024-09-14T06:31:35.568 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:31:35.568 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:31:35.568 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:31:35.568 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:31:35.568 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:31:35.568 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:35.568 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 55631 2024-09-14T06:31:35.568 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:35.569 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:31:35.569 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:35.569 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:31:35.569 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600006+0000", 2024-09-14T06:31:35.569 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:35.569 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:35.569 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:35.569 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:31:35.569 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600006+0000", 2024-09-14T06:31:35.569 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:35.569 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:35.569 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:35.570 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:31:35.570 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600006+0000", 2024-09-14T06:31:35.570 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.0800000000000001e-07 2024-09-14T06:31:35.570 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:35.570 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:35.570 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:31:35.570 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600007+0000", 2024-09-14T06:31:35.570 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.7199999999999997e-07 2024-09-14T06:31:35.570 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:35.570 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:35.570 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:31:35.570 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600010+0000", 2024-09-14T06:31:35.571 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.0280000000000001e-06 2024-09-14T06:31:35.571 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:35.571 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:35.571 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:31:35.571 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600012+0000", 2024-09-14T06:31:35.571 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.475e-06 2024-09-14T06:31:35.571 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:35.571 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:35.571 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:31:35.571 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.642329+0000", 2024-09-14T06:31:35.571 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.042317157000000001 2024-09-14T06:31:35.571 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:35.572 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:35.572 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:31:35.572 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.642403+0000", 2024-09-14T06:31:35.572 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.3941000000000002e-05 2024-09-14T06:31:35.572 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:35.572 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:35.572 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:31:35.572 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.642493+0000", 2024-09-14T06:31:35.572 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.0124999999999995e-05 2024-09-14T06:31:35.572 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:35.572 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:31:35.573 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:35.573 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:35.573 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:35.573 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:55632 3.0 3:06490baf:::rbd_data.112dca2f6ae1.0000000000000030:17 [read 1073152~4096 [fadvise_nocache]] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:31:35.573 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:44.600015+0000", 2024-09-14T06:31:35.573 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 110.84664590600001, 2024-09-14T06:31:35.573 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.042604922000000003, 2024-09-14T06:31:35.573 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:31:35.573 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:31:35.573 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:31:35.573 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:31:35.574 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:31:35.574 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:35.574 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 55632 2024-09-14T06:31:35.574 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:35.574 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:31:35.574 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:35.574 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:31:35.574 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600015+0000", 2024-09-14T06:31:35.574 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:35.574 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:35.575 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:35.575 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:31:35.575 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600015+0000", 2024-09-14T06:31:35.575 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:35.575 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:35.575 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:35.575 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:31:35.575 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600016+0000", 2024-09-14T06:31:35.575 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.1e-07 2024-09-14T06:31:35.575 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:35.575 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:35.575 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:31:35.576 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600017+0000", 2024-09-14T06:31:35.576 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.9900000000000009e-07 2024-09-14T06:31:35.576 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:35.576 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:35.576 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:31:35.576 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600020+0000", 2024-09-14T06:31:35.576 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.4170000000000001e-06 2024-09-14T06:31:35.576 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:35.576 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:35.576 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:31:35.576 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600023+0000", 2024-09-14T06:31:35.577 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.588e-06 2024-09-14T06:31:35.577 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:35.577 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:35.577 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:31:35.577 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.642526+0000", 2024-09-14T06:31:35.577 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.042503236999999999 2024-09-14T06:31:35.577 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:35.577 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:35.577 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:31:35.577 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.642568+0000", 2024-09-14T06:31:35.577 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.1925000000000003e-05 2024-09-14T06:31:35.577 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:35.578 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:35.578 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:31:35.578 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.642620+0000", 2024-09-14T06:31:35.578 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.2546000000000001e-05 2024-09-14T06:31:35.578 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:35.578 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:31:35.578 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:35.578 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:35.578 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:35.578 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:55633 3.0 3:06490baf:::rbd_data.112dca2f6ae1.0000000000000030:17 [read 1097728~4096 [fadvise_nocache]] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:31:35.579 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:44.600026+0000", 2024-09-14T06:31:35.579 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 110.84663563700001, 2024-09-14T06:31:35.579 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.042797474000000002, 2024-09-14T06:31:35.579 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:31:35.579 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:31:35.579 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:31:35.579 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:31:35.579 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:31:35.579 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:35.579 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 55633 2024-09-14T06:31:35.579 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:35.579 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:31:35.580 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:35.580 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:31:35.580 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600026+0000", 2024-09-14T06:31:35.580 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:35.580 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:35.580 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:35.580 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:31:35.580 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600026+0000", 2024-09-14T06:31:35.580 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:35.580 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:35.580 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:35.581 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:31:35.581 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600026+0000", 2024-09-14T06:31:35.581 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.3099999999999999e-07 2024-09-14T06:31:35.581 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:35.581 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:35.581 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:31:35.581 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600029+0000", 2024-09-14T06:31:35.581 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.304e-06 2024-09-14T06:31:35.581 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:35.581 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:35.581 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:31:35.582 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600034+0000", 2024-09-14T06:31:35.582 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.1429999999999997e-06 2024-09-14T06:31:35.582 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:35.582 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:35.582 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:31:35.582 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600038+0000", 2024-09-14T06:31:35.582 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.692e-06 2024-09-14T06:31:35.582 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:35.582 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:35.582 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:31:35.582 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.642654+0000", 2024-09-14T06:31:35.582 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.042615564000000002 2024-09-14T06:31:35.583 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:35.583 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:35.583 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:31:35.583 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.642728+0000", 2024-09-14T06:31:35.583 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.4018999999999997e-05 2024-09-14T06:31:35.583 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:35.583 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:35.583 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:31:35.583 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.642823+0000", 2024-09-14T06:31:35.583 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.5520999999999996e-05 2024-09-14T06:31:35.583 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:35.583 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:31:35.584 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:35.584 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:35.584 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:35.584 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:55634 3.0 3:06490baf:::rbd_data.112dca2f6ae1.0000000000000030:17 [read 1122304~4096 [fadvise_nocache]] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:31:35.584 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:44.600042+0000", 2024-09-14T06:31:35.584 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 110.84661916, 2024-09-14T06:31:35.584 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.042903729000000002, 2024-09-14T06:31:35.584 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:31:35.584 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:31:35.584 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:31:35.584 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:31:35.584 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:31:35.584 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:35.585 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 55634 2024-09-14T06:31:35.585 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:35.585 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:31:35.585 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:35.585 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:31:35.585 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600042+0000", 2024-09-14T06:31:35.585 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:35.585 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:35.585 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:35.585 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:31:35.585 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600042+0000", 2024-09-14T06:31:35.585 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:35.586 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:35.586 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:35.586 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:31:35.586 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600043+0000", 2024-09-14T06:31:35.586 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.7000000000000005e-07 2024-09-14T06:31:35.586 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:35.586 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:35.586 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:31:35.586 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600044+0000", 2024-09-14T06:31:35.586 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.742e-06 2024-09-14T06:31:35.586 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:35.586 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:35.587 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:31:35.587 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600050+0000", 2024-09-14T06:31:35.587 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.0800000000000002e-06 2024-09-14T06:31:35.587 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:35.587 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:35.587 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:31:35.587 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600055+0000", 2024-09-14T06:31:35.587 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.3889999999999998e-06 2024-09-14T06:31:35.587 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:35.587 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:35.587 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:31:35.588 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.642845+0000", 2024-09-14T06:31:35.588 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.042790040000000001 2024-09-14T06:31:35.588 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:35.588 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:35.588 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:31:35.588 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.642888+0000", 2024-09-14T06:31:35.588 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.2790000000000002e-05 2024-09-14T06:31:35.588 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:35.588 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:35.588 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:31:35.588 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.642946+0000", 2024-09-14T06:31:35.588 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.8118000000000002e-05 2024-09-14T06:31:35.588 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:35.589 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:31:35.589 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:35.589 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:35.589 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:35.589 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:55635 3.0 3:06490baf:::rbd_data.112dca2f6ae1.0000000000000030:17 [read 1146880~4096 [fadvise_nocache]] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:31:35.589 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:44.600061+0000", 2024-09-14T06:31:35.589 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 110.846600589, 2024-09-14T06:31:35.589 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.043092695, 2024-09-14T06:31:35.589 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:31:35.589 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:31:35.589 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:31:35.589 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:31:35.590 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:31:35.590 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:35.590 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 55635 2024-09-14T06:31:35.590 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:35.590 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:31:35.590 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:35.590 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:31:35.590 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600061+0000", 2024-09-14T06:31:35.590 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:35.590 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:35.590 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:35.590 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:31:35.591 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600061+0000", 2024-09-14T06:31:35.591 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:35.591 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:35.591 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:35.591 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:31:35.591 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600061+0000", 2024-09-14T06:31:35.591 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.3099999999999999e-07 2024-09-14T06:31:35.591 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:35.591 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:35.591 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:31:35.591 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600063+0000", 2024-09-14T06:31:35.592 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.6220000000000001e-06 2024-09-14T06:31:35.592 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:35.592 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:35.592 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:31:35.592 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600068+0000", 2024-09-14T06:31:35.592 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.4790000000000004e-06 2024-09-14T06:31:35.592 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:35.592 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:35.592 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:31:35.592 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600072+0000", 2024-09-14T06:31:35.592 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.3100000000000002e-06 2024-09-14T06:31:35.592 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:35.592 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:35.593 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:31:35.593 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.642987+0000", 2024-09-14T06:31:35.593 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.042914489 2024-09-14T06:31:35.593 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:35.593 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:35.593 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:31:35.593 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.643062+0000", 2024-09-14T06:31:35.593 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.5011000000000004e-05 2024-09-14T06:31:35.593 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:35.593 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:35.593 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:31:35.594 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.643153+0000", 2024-09-14T06:31:35.594 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.1452999999999993e-05 2024-09-14T06:31:35.594 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:35.594 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:31:35.594 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:35.594 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:35.594 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:35.594 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:55636 3.0 3:06490baf:::rbd_data.112dca2f6ae1.0000000000000030:17 [read 1171456~4096 [fadvise_nocache]] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:31:35.594 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:44.600078+0000", 2024-09-14T06:31:35.594 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 110.846583177, 2024-09-14T06:31:35.594 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.043207966, 2024-09-14T06:31:35.594 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:31:35.595 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:31:35.595 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:31:35.595 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:31:35.595 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:31:35.595 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:35.595 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 55636 2024-09-14T06:31:35.595 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:35.595 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:31:35.595 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:35.595 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:31:35.595 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600078+0000", 2024-09-14T06:31:35.595 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:35.595 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:35.596 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:35.596 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:31:35.596 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600078+0000", 2024-09-14T06:31:35.596 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:35.596 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:35.596 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:35.596 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:31:35.596 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600078+0000", 2024-09-14T06:31:35.596 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.34e-07 2024-09-14T06:31:35.596 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:35.596 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:35.597 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:31:35.597 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600080+0000", 2024-09-14T06:31:35.597 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.641e-06 2024-09-14T06:31:35.597 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:35.597 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:35.597 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:31:35.597 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600086+0000", 2024-09-14T06:31:35.597 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.3860000000000004e-06 2024-09-14T06:31:35.597 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:35.597 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:35.597 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:31:35.597 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600091+0000", 2024-09-14T06:31:35.598 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.5970000000000002e-06 2024-09-14T06:31:35.598 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:35.598 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:35.598 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:31:35.598 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.643184+0000", 2024-09-14T06:31:35.598 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.043092893 2024-09-14T06:31:35.598 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:35.598 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:35.598 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:31:35.598 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.643228+0000", 2024-09-14T06:31:35.598 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.4536999999999997e-05 2024-09-14T06:31:35.599 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:35.599 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:35.599 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:31:35.599 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.643286+0000", 2024-09-14T06:31:35.599 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.7578000000000003e-05 2024-09-14T06:31:35.599 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:35.599 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:31:35.599 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:35.599 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:35.599 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:35.599 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:55637 3.0 3:06490baf:::rbd_data.112dca2f6ae1.0000000000000030:17 [read 1196032~4096 [fadvise_nocache]] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:31:35.599 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:44.600097+0000", 2024-09-14T06:31:35.600 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 110.846564613, 2024-09-14T06:31:35.600 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.043399124999999997, 2024-09-14T06:31:35.600 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:31:35.600 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:31:35.600 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:31:35.600 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:31:35.600 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:31:35.600 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:35.600 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 55637 2024-09-14T06:31:35.600 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:35.600 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:31:35.601 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:35.601 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:31:35.601 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600097+0000", 2024-09-14T06:31:35.601 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:35.601 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:35.601 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:35.601 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:31:35.601 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600097+0000", 2024-09-14T06:31:35.601 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:35.601 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:35.601 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:35.602 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:31:35.602 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.604372+0000", 2024-09-14T06:31:35.602 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0042751489999999998 2024-09-14T06:31:35.602 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:35.602 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:35.602 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:31:35.602 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.604377+0000", 2024-09-14T06:31:35.602 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.7899999999999999e-06 2024-09-14T06:31:35.602 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:35.602 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:35.602 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:31:35.602 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.604389+0000", 2024-09-14T06:31:35.603 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.2197e-05 2024-09-14T06:31:35.603 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:35.603 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:35.603 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:31:35.603 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.604397+0000", 2024-09-14T06:31:35.603 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.8730000000000007e-06 2024-09-14T06:31:35.603 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:35.603 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:35.603 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:31:35.603 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.643324+0000", 2024-09-14T06:31:35.603 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.038926897000000002 2024-09-14T06:31:35.603 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:35.604 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:35.604 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:31:35.604 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.643402+0000", 2024-09-14T06:31:35.604 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.8063000000000001e-05 2024-09-14T06:31:35.604 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:35.604 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:35.604 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:31:35.604 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.643496+0000", 2024-09-14T06:31:35.604 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.4155999999999998e-05 2024-09-14T06:31:35.604 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:35.604 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:31:35.605 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:35.605 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:35.605 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:35.605 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:55657 3.0 3:06490baf:::rbd_data.112dca2f6ae1.0000000000000030:17 [read 1687552~4096 [fadvise_nocache]] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:31:35.605 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:44.604841+0000", 2024-09-14T06:31:35.605 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 110.841819884, 2024-09-14T06:31:35.605 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.041890087999999999, 2024-09-14T06:31:35.605 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:31:35.605 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:31:35.605 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:31:35.605 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:31:35.605 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:31:35.606 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:35.606 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 55657 2024-09-14T06:31:35.606 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:35.606 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:31:35.606 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:35.606 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:31:35.606 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.604841+0000", 2024-09-14T06:31:35.606 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:35.606 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:35.606 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:35.607 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:31:35.607 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.604841+0000", 2024-09-14T06:31:35.607 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:35.607 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:35.607 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:35.607 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:31:35.607 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.604842+0000", 2024-09-14T06:31:35.607 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.0900000000000001e-07 2024-09-14T06:31:35.607 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:35.607 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:35.607 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:31:35.607 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.604846+0000", 2024-09-14T06:31:35.608 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.5759999999999999e-06 2024-09-14T06:31:35.608 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:35.608 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:35.608 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:31:35.608 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.604850+0000", 2024-09-14T06:31:35.608 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.405e-06 2024-09-14T06:31:35.608 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:35.608 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:35.608 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:31:35.608 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.604852+0000", 2024-09-14T06:31:35.608 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.8590000000000001e-06 2024-09-14T06:31:35.608 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:35.609 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:35.609 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:31:35.609 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.646633+0000", 2024-09-14T06:31:35.609 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.041780772000000001 2024-09-14T06:31:35.609 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:35.609 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:35.609 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:31:35.609 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.646677+0000", 2024-09-14T06:31:35.609 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.4115e-05 2024-09-14T06:31:35.609 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:35.609 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:35.609 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:31:35.610 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.646731+0000", 2024-09-14T06:31:35.610 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.4151999999999999e-05 2024-09-14T06:31:35.610 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:35.610 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:31:35.610 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:35.610 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:35.610 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:35.610 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:55658 3.0 3:06490baf:::rbd_data.112dca2f6ae1.0000000000000030:17 [read 1712128~4096 [fadvise_nocache]] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:31:35.610 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:44.604856+0000", 2024-09-14T06:31:35.610 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 110.841805284, 2024-09-14T06:31:35.610 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.041986227000000001, 2024-09-14T06:31:35.610 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:31:35.611 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:31:35.611 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:31:35.611 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:31:35.611 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:31:35.611 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:35.611 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 55658 2024-09-14T06:31:35.611 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:35.611 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:31:35.611 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:35.611 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:31:35.611 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.604856+0000", 2024-09-14T06:31:35.612 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:35.612 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:35.612 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:35.612 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:31:35.612 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.604856+0000", 2024-09-14T06:31:35.612 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:35.612 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:35.612 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:35.612 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:31:35.612 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.604856+0000", 2024-09-14T06:31:35.612 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.23e-07 2024-09-14T06:31:35.612 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:35.613 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:35.613 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:31:35.613 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.604857+0000", 2024-09-14T06:31:35.613 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.9999999999999995e-07 2024-09-14T06:31:35.613 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:35.613 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:35.613 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:31:35.613 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.604861+0000", 2024-09-14T06:31:35.613 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.9890000000000003e-06 2024-09-14T06:31:35.613 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:35.613 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:35.613 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:31:35.614 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.604864+0000", 2024-09-14T06:31:35.614 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.7429999999999998e-06 2024-09-14T06:31:35.614 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:35.614 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:35.614 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:31:35.614 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.646744+0000", 2024-09-14T06:31:35.614 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.041880325000000003 2024-09-14T06:31:35.614 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:35.614 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:35.614 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:31:35.614 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.646784+0000", 2024-09-14T06:31:35.615 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.9604000000000002e-05 2024-09-14T06:31:35.615 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:35.615 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:35.615 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:31:35.615 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.646842+0000", 2024-09-14T06:31:35.615 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.8343e-05 2024-09-14T06:31:35.615 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:35.615 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:31:35.615 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:35.615 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:35.615 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:35.615 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:55659 3.0 3:06490baf:::rbd_data.112dca2f6ae1.0000000000000030:17 [read 1736704~4096 [fadvise_nocache]] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:31:35.616 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:44.604867+0000", 2024-09-14T06:31:35.616 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 110.841794131, 2024-09-14T06:31:35.616 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.042097082000000001, 2024-09-14T06:31:35.616 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:31:35.616 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:31:35.616 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:31:35.616 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:31:35.616 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:31:35.616 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:35.616 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 55659 2024-09-14T06:31:35.616 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:35.616 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:31:35.617 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:35.617 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:31:35.617 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.604867+0000", 2024-09-14T06:31:35.617 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:35.617 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:35.617 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:35.617 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:31:35.617 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.604867+0000", 2024-09-14T06:31:35.617 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:35.617 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:35.617 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:35.617 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:31:35.617 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.604867+0000", 2024-09-14T06:31:35.618 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.2000000000000001e-07 2024-09-14T06:31:35.618 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:35.618 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:35.618 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:31:35.618 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.604868+0000", 2024-09-14T06:31:35.618 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.04e-06 2024-09-14T06:31:35.618 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:35.618 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:35.618 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:31:35.618 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.604871+0000", 2024-09-14T06:31:35.618 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.9689999999999999e-06 2024-09-14T06:31:35.619 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:35.619 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:35.619 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:31:35.619 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.604874+0000", 2024-09-14T06:31:35.619 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.5069999999999999e-06 2024-09-14T06:31:35.619 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:35.619 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:35.619 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:31:35.619 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.646874+0000", 2024-09-14T06:31:35.619 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.042000513000000003 2024-09-14T06:31:35.619 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:35.619 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:35.619 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:31:35.620 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.646914+0000", 2024-09-14T06:31:35.620 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.9443999999999997e-05 2024-09-14T06:31:35.620 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:35.620 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:35.620 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:31:35.620 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.646964+0000", 2024-09-14T06:31:35.620 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.0389000000000002e-05 2024-09-14T06:31:35.620 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:35.620 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:31:35.620 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:35.620 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:35.620 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:35.621 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:55660 3.0 3:06490baf:::rbd_data.112dca2f6ae1.0000000000000030:17 [read 1761280~4096 [fadvise_nocache]] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:31:35.621 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:44.604877+0000", 2024-09-14T06:31:35.621 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 110.841784382, 2024-09-14T06:31:35.621 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.042196349000000001, 2024-09-14T06:31:35.621 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:31:35.621 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:31:35.621 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:31:35.621 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:31:35.621 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:31:35.621 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:35.621 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 55660 2024-09-14T06:31:35.621 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:35.622 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:31:35.622 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:35.622 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:31:35.622 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.604877+0000", 2024-09-14T06:31:35.622 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:35.622 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:35.622 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:35.622 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:31:35.622 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.604877+0000", 2024-09-14T06:31:35.622 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:35.622 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:35.623 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:35.623 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:31:35.623 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.605810+0000", 2024-09-14T06:31:35.623 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00093336899999999997 2024-09-14T06:31:35.623 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:35.623 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:35.623 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:31:35.623 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.605815+0000", 2024-09-14T06:31:35.623 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.6140000000000002e-06 2024-09-14T06:31:35.623 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:35.623 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:35.623 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:31:35.623 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.605825+0000", 2024-09-14T06:31:35.624 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.8889999999999997e-06 2024-09-14T06:31:35.624 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:35.624 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:35.624 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:31:35.624 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.605831+0000", 2024-09-14T06:31:35.624 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.9800000000000003e-06 2024-09-14T06:31:35.624 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:35.624 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:35.624 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:31:35.624 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.646989+0000", 2024-09-14T06:31:35.624 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.041157878000000002 2024-09-14T06:31:35.624 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:35.625 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:35.625 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:31:35.625 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.647026+0000", 2024-09-14T06:31:35.625 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.7305000000000003e-05 2024-09-14T06:31:35.625 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:35.625 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:35.625 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:31:35.625 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.647073+0000", 2024-09-14T06:31:35.625 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.7314000000000001e-05 2024-09-14T06:31:35.625 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:35.625 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:31:35.626 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:35.626 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:35.626 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:35.626 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:55666 3.0 3:06490baf:::rbd_data.112dca2f6ae1.0000000000000030:17 [read 1908736~4096 [fadvise_nocache]] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:31:35.626 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:44.605898+0000", 2024-09-14T06:31:35.626 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 110.84076363600001, 2024-09-14T06:31:35.626 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.041862887000000001, 2024-09-14T06:31:35.626 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:31:35.626 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:31:35.626 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:31:35.626 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:31:35.626 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:31:35.626 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:35.627 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 55666 2024-09-14T06:31:35.627 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:35.627 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:31:35.627 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:35.627 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:31:35.627 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.605898+0000", 2024-09-14T06:31:35.627 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:35.627 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:35.627 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:35.627 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:31:35.627 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.605898+0000", 2024-09-14T06:31:35.627 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:35.628 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:35.628 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:35.628 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:31:35.628 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.611874+0000", 2024-09-14T06:31:35.628 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0059764049999999997 2024-09-14T06:31:35.628 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:35.628 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:35.628 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:31:35.628 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.611879+0000", 2024-09-14T06:31:35.628 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.8749999999999999e-06 2024-09-14T06:31:35.628 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:35.628 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:35.629 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:31:35.629 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.611887+0000", 2024-09-14T06:31:35.629 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.4109999999999996e-06 2024-09-14T06:31:35.629 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:35.629 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:35.629 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:31:35.629 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.611893+0000", 2024-09-14T06:31:35.629 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.2959999999999998e-06 2024-09-14T06:31:35.629 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:35.629 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:35.629 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:31:35.629 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.647674+0000", 2024-09-14T06:31:35.630 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.035781281999999998 2024-09-14T06:31:35.630 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:35.630 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:35.630 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:31:35.630 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.647711+0000", 2024-09-14T06:31:35.630 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.7434000000000001e-05 2024-09-14T06:31:35.630 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:35.630 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:35.630 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:31:35.630 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.647760+0000", 2024-09-14T06:31:35.630 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.9184000000000002e-05 2024-09-14T06:31:35.631 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:35.631 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:31:35.631 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:35.631 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:35.631 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:31:35.631 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-14T06:31:35.633 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-14T06:31:35.633 INFO:tasks.ceph.osd.1.smithi094.stderr:2024-09-14T06:31:35.630+0000 7f328e47a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33641) UID: 0 2024-09-14T06:31:35.643 INFO:teuthology.orchestra.run.smithi094.stdout:5088 writesame 0xab450e5 thru 0xab478e8 data_size 0x1a(0x2804 bytes) 2024-09-14T06:31:35.651 INFO:teuthology.orchestra.run.smithi094.stdout:5089 write 0xd323c9e thru 0xd3291b4 (0x5517 bytes) 2024-09-14T06:31:35.659 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-14T06:31:35.662 INFO:teuthology.orchestra.run.smithi094.stdout:5094 trunc from 0xd3291b5 to 0xacb644b 2024-09-14T06:31:35.671 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-14T06:31:35.675 INFO:teuthology.orchestra.run.smithi094.stdout:5096 read 0x37b0c05 thru 0x37bad2d (0xa129 bytes) 2024-09-14T06:31:35.675 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-14T06:31:35.675 INFO:teuthology.orchestra.run.smithi094.stdout:5099 writesame 0x685f4c5 thru 0x6861d93 data_size 0x1(0x28cf bytes) 2024-09-14T06:31:35.684 INFO:teuthology.orchestra.run.smithi094.stdout:5100 punch from 0x47f82dd to 0x48011c7, (0x8eea bytes) 2024-09-14T06:31:35.684 INFO:teuthology.orchestra.run.smithi094.stdout:5101 write 0x2e71071 thru 0x2e785c4 (0x7554 bytes) 2024-09-14T06:31:35.692 INFO:teuthology.orchestra.run.smithi094.stdout:5103 trunc from 0xacb644b to 0xac9d058 2024-09-14T06:31:35.698 INFO:teuthology.orchestra.run.smithi094.stdout:5105 writesame 0xac4ce16 thru 0xac520e3 data_size 0x3(0x52ce bytes) 2024-09-14T06:31:35.709 INFO:teuthology.orchestra.run.smithi094.stdout:5106 read 0x5f9db91 thru 0x5fac4bf (0xe92f bytes) 2024-09-14T06:31:35.709 INFO:teuthology.orchestra.run.smithi094.stdout:5107 write 0x61f86e thru 0x62952b (0x9cbe bytes) 2024-09-14T06:31:35.720 INFO:teuthology.orchestra.run.smithi094.stdout:5108 write 0xc377a47 thru 0xc37fffd (0x85b7 bytes) 2024-09-14T06:31:35.733 INFO:tasks.ceph.osd.4.smithi138.stderr:2024-09-14T06:31:35.732+0000 7f6c653c8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33615) UID: 0 2024-09-14T06:31:35.756 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-14T06:31:35.757 INFO:teuthology.orchestra.run.smithi094.stdout: "ops": [], 2024-09-14T06:31:35.757 INFO:teuthology.orchestra.run.smithi094.stdout: "num_ops": 0 2024-09-14T06:31:35.757 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-14T06:31:35.766 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-14T06:31:35.833 INFO:tasks.ceph.osd.6.smithi138.stderr:2024-09-14T06:31:35.832+0000 7f8cd8849640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33620) UID: 0 2024-09-14T06:31:35.909 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-14T06:31:35.909 INFO:teuthology.orchestra.run.smithi094.stdout: "ops": [], 2024-09-14T06:31:35.909 INFO:teuthology.orchestra.run.smithi094.stdout: "complaint_time": 30, 2024-09-14T06:31:35.909 INFO:teuthology.orchestra.run.smithi094.stdout: "num_blocked_ops": 0 2024-09-14T06:31:35.909 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-14T06:31:35.918 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-14T06:31:35.934 INFO:tasks.ceph.osd.7.smithi138.stderr:2024-09-14T06:31:35.933+0000 7f3c6a21d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33623) UID: 0 2024-09-14T06:31:36.022 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-14T06:31:36.022 INFO:teuthology.orchestra.run.smithi094.stdout: "size": 20, 2024-09-14T06:31:36.023 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 600, 2024-09-14T06:31:36.023 INFO:teuthology.orchestra.run.smithi094.stdout: "ops": [ 2024-09-14T06:31:36.023 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:36.023 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4397.0:45722 3.9 e64/23 3:91fed098:::rbd_data.112d5a737e72.0000000000000001:head v 64'61, mlcod=62'59)", 2024-09-14T06:31:36.023 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:43.864269+0000", 2024-09-14T06:31:36.023 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 112.15659625000001, 2024-09-14T06:31:36.023 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.003055224, 2024-09-14T06:31:36.023 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:31:36.023 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:31:36.023 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:31:36.023 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:31:36.024 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:36.024 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:31:36.024 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.864269+0000", 2024-09-14T06:31:36.024 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:36.024 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:36.024 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:36.024 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:31:36.024 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.864269+0000", 2024-09-14T06:31:36.024 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:36.024 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:36.024 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:36.025 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:31:36.025 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.864269+0000", 2024-09-14T06:31:36.025 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.41e-07 2024-09-14T06:31:36.025 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:36.025 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:36.025 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:31:36.025 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.865490+0000", 2024-09-14T06:31:36.025 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.001220552 2024-09-14T06:31:36.025 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:36.025 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:36.025 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:31:36.025 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.865506+0000", 2024-09-14T06:31:36.026 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.6263999999999999e-05 2024-09-14T06:31:36.026 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:36.026 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:36.026 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:31:36.026 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.865516+0000", 2024-09-14T06:31:36.026 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.0091999999999999e-05 2024-09-14T06:31:36.026 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:36.026 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:36.026 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:31:36.026 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.865583+0000", 2024-09-14T06:31:36.026 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.6676999999999999e-05 2024-09-14T06:31:36.026 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:36.027 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:36.027 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:31:36.027 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.865610+0000", 2024-09-14T06:31:36.027 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.6529999999999998e-05 2024-09-14T06:31:36.027 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:36.027 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:36.027 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-14T06:31:36.027 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.867281+0000", 2024-09-14T06:31:36.027 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.001671203 2024-09-14T06:31:36.027 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:36.027 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:36.027 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:31:36.027 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.867324+0000", 2024-09-14T06:31:36.028 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.3665000000000003e-05 2024-09-14T06:31:36.028 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:36.028 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:31:36.028 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:36.028 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:36.028 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:36.028 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4397.0:74241 3.0 e73/23 3:06e93fb9:::rbd_data.112d80ca42af.0000000000000000:head v 73'102, mlcod=71'100)", 2024-09-14T06:31:36.028 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:53.876730+0000", 2024-09-14T06:31:36.028 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 102.14413515, 2024-09-14T06:31:36.028 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0079470329999999992, 2024-09-14T06:31:36.028 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:31:36.028 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:31:36.029 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:31:36.029 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:31:36.029 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:36.029 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:31:36.029 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:53.876730+0000", 2024-09-14T06:31:36.029 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:36.029 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:36.029 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:36.029 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:31:36.029 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:53.876730+0000", 2024-09-14T06:31:36.029 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:36.030 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:36.030 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:36.030 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:31:36.030 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:53.876731+0000", 2024-09-14T06:31:36.030 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.0500000000000004e-07 2024-09-14T06:31:36.030 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:36.030 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:36.030 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:31:36.030 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:53.879616+0000", 2024-09-14T06:31:36.030 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0028856680000000001 2024-09-14T06:31:36.031 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:36.031 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:36.031 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:31:36.031 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:53.879643+0000", 2024-09-14T06:31:36.031 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.6120999999999999e-05 2024-09-14T06:31:36.031 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:36.031 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:36.031 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:31:36.031 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:53.879654+0000", 2024-09-14T06:31:36.031 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.1586999999999999e-05 2024-09-14T06:31:36.031 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:36.031 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:36.032 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:31:36.032 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:53.879756+0000", 2024-09-14T06:31:36.032 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00010223899999999999 2024-09-14T06:31:36.032 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:36.032 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:36.032 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:31:36.032 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:53.879783+0000", 2024-09-14T06:31:36.032 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.6913000000000001e-05 2024-09-14T06:31:36.032 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:36.032 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:36.033 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-14T06:31:36.033 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:53.884594+0000", 2024-09-14T06:31:36.033 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0048110729999999999 2024-09-14T06:31:36.033 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:36.033 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:36.033 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:31:36.033 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:53.884677+0000", 2024-09-14T06:31:36.033 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.2927000000000001e-05 2024-09-14T06:31:36.033 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:36.033 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:31:36.033 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:36.034 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:36.034 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:36.034 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4397.0:74243 3.9 e73/23 3:94977f81:::rbd_data.112d80ca42af.0000000000000002:head v 73'94, mlcod=71'92)", 2024-09-14T06:31:36.034 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:53.885156+0000", 2024-09-14T06:31:36.034 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 102.135709232, 2024-09-14T06:31:36.034 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0096009860000000006, 2024-09-14T06:31:36.034 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:31:36.034 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:31:36.034 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:31:36.034 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:31:36.034 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:36.035 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:31:36.035 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:53.885156+0000", 2024-09-14T06:31:36.035 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:36.035 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:36.035 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:36.035 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:31:36.035 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:53.885156+0000", 2024-09-14T06:31:36.035 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:36.035 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:36.035 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:36.035 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:31:36.035 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:53.885156+0000", 2024-09-14T06:31:36.036 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.23e-07 2024-09-14T06:31:36.036 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:36.036 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:36.036 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:31:36.036 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:53.889694+0000", 2024-09-14T06:31:36.036 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0045378709999999997 2024-09-14T06:31:36.036 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:36.036 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:36.036 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:31:36.036 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:53.889720+0000", 2024-09-14T06:31:36.036 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.5259e-05 2024-09-14T06:31:36.036 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:36.037 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:36.037 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:31:36.037 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:53.889731+0000", 2024-09-14T06:31:36.037 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.1739000000000001e-05 2024-09-14T06:31:36.037 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:36.037 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:36.037 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:31:36.037 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:53.889822+0000", 2024-09-14T06:31:36.037 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.1200999999999996e-05 2024-09-14T06:31:36.037 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:36.037 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:36.038 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:31:36.038 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:53.889850+0000", 2024-09-14T06:31:36.038 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.7084999999999999e-05 2024-09-14T06:31:36.038 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:36.038 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:36.038 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-14T06:31:36.038 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:53.894703+0000", 2024-09-14T06:31:36.038 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0048532690000000003 2024-09-14T06:31:36.038 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:36.038 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:36.038 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:31:36.038 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:53.894757+0000", 2024-09-14T06:31:36.039 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.4339000000000003e-05 2024-09-14T06:31:36.039 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:36.039 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:31:36.039 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:36.039 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:36.039 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:36.039 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4397.0:74249 3.0 e73/23 3:009b0225:::rbd_data.112d80ca42af.0000000000000009:head v 73'103, mlcod=71'101)", 2024-09-14T06:31:36.039 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:53.893219+0000", 2024-09-14T06:31:36.039 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 102.127646754, 2024-09-14T06:31:36.039 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0065381980000000003, 2024-09-14T06:31:36.039 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:31:36.040 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:31:36.040 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:31:36.040 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:31:36.040 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:36.040 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:31:36.040 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:53.893219+0000", 2024-09-14T06:31:36.040 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:36.040 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:36.040 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:36.040 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:31:36.040 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:53.893219+0000", 2024-09-14T06:31:36.041 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:36.041 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:36.041 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:36.041 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:31:36.041 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:53.893221+0000", 2024-09-14T06:31:36.041 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.4329999999999998e-06 2024-09-14T06:31:36.041 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:36.041 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:36.041 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:31:36.041 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:53.895968+0000", 2024-09-14T06:31:36.041 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.002746746 2024-09-14T06:31:36.041 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:36.042 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:36.042 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:31:36.042 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:53.895988+0000", 2024-09-14T06:31:36.042 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.0227000000000001e-05 2024-09-14T06:31:36.042 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:36.042 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:36.042 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:31:36.042 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:53.896000+0000", 2024-09-14T06:31:36.042 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.2359e-05 2024-09-14T06:31:36.042 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:36.042 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:36.043 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:31:36.043 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:53.896084+0000", 2024-09-14T06:31:36.043 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.3659999999999995e-05 2024-09-14T06:31:36.043 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:36.043 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:36.043 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:31:36.043 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:53.896109+0000", 2024-09-14T06:31:36.043 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.5259999999999999e-05 2024-09-14T06:31:36.043 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:36.043 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:36.043 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-14T06:31:36.043 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:53.899704+0000", 2024-09-14T06:31:36.044 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0035942209999999999 2024-09-14T06:31:36.044 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:36.044 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:36.044 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:31:36.044 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:53.899757+0000", 2024-09-14T06:31:36.044 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.3291999999999997e-05 2024-09-14T06:31:36.044 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:36.044 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:31:36.044 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:36.044 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:36.044 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:36.045 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4397.0:78263 3.0 e77/23 3:00899b22:::rbd_data.112ddfbec471.0000000000000016:head v 77'104, mlcod=73'102)", 2024-09-14T06:31:36.045 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:57.634578+0000", 2024-09-14T06:31:36.045 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 98.386287702000004, 2024-09-14T06:31:36.045 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.002372454, 2024-09-14T06:31:36.045 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:31:36.045 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:31:36.045 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:31:36.045 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:31:36.045 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:36.045 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:31:36.046 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:57.634578+0000", 2024-09-14T06:31:36.046 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:36.046 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:36.046 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:36.046 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:31:36.046 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:57.634578+0000", 2024-09-14T06:31:36.046 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:36.046 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:36.046 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:36.046 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:31:36.046 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:57.634578+0000", 2024-09-14T06:31:36.046 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.5699999999999998e-07 2024-09-14T06:31:36.047 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:36.047 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:36.047 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:31:36.047 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:57.635347+0000", 2024-09-14T06:31:36.047 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00076848300000000003 2024-09-14T06:31:36.047 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:36.047 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:36.047 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:31:36.047 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:57.635361+0000", 2024-09-14T06:31:36.047 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.4238e-05 2024-09-14T06:31:36.047 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:36.047 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:36.048 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:31:36.048 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:57.635371+0000", 2024-09-14T06:31:36.048 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.0087e-05 2024-09-14T06:31:36.048 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:36.048 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:36.048 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:31:36.048 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:57.635439+0000", 2024-09-14T06:31:36.048 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.7953000000000005e-05 2024-09-14T06:31:36.048 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:36.048 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:36.048 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:31:36.048 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:57.635464+0000", 2024-09-14T06:31:36.049 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.5062000000000002e-05 2024-09-14T06:31:36.049 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:36.049 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:36.049 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-14T06:31:36.049 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:57.636914+0000", 2024-09-14T06:31:36.049 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.001449625 2024-09-14T06:31:36.049 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:36.049 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:36.049 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:31:36.049 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:57.636950+0000", 2024-09-14T06:31:36.049 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.6548999999999999e-05 2024-09-14T06:31:36.049 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:36.050 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:31:36.050 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:36.050 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:36.050 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:36.050 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4397.0:78604 3.9 e77/23 3:94c62a0e:::rbd_data.112ddfbec471.000000000000002c:head v 77'96, mlcod=73'94)", 2024-09-14T06:31:36.050 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:57.677568+0000", 2024-09-14T06:31:36.050 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 98.343297313999997, 2024-09-14T06:31:36.050 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.002564846, 2024-09-14T06:31:36.050 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:31:36.050 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:31:36.050 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:31:36.050 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:31:36.051 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:36.051 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:31:36.051 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:57.677568+0000", 2024-09-14T06:31:36.051 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:36.051 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:36.051 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:36.051 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:31:36.051 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:57.677568+0000", 2024-09-14T06:31:36.051 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:36.051 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:36.051 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:36.051 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:31:36.052 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:57.677568+0000", 2024-09-14T06:31:36.052 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.8900000000000001e-07 2024-09-14T06:31:36.052 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:36.052 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:36.052 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:31:36.052 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:57.678488+0000", 2024-09-14T06:31:36.052 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00091963899999999998 2024-09-14T06:31:36.052 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:36.052 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:36.052 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:31:36.052 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:57.678500+0000", 2024-09-14T06:31:36.052 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.1902000000000001e-05 2024-09-14T06:31:36.053 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:36.053 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:36.053 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:31:36.053 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:57.678507+0000", 2024-09-14T06:31:36.053 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.7739999999999999e-06 2024-09-14T06:31:36.053 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:36.053 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:36.053 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:31:36.053 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:57.678586+0000", 2024-09-14T06:31:36.053 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.9103000000000005e-05 2024-09-14T06:31:36.053 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:36.054 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:36.054 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:31:36.054 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:57.678614+0000", 2024-09-14T06:31:36.054 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.7912e-05 2024-09-14T06:31:36.054 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:36.054 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:36.054 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-14T06:31:36.054 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:57.680075+0000", 2024-09-14T06:31:36.054 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0014613370000000001 2024-09-14T06:31:36.054 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:36.054 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:36.055 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:31:36.055 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:57.680133+0000", 2024-09-14T06:31:36.055 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.7890000000000003e-05 2024-09-14T06:31:36.055 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:36.055 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:31:36.055 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:36.055 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:36.055 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:36.055 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4397.0:149885 3.9 e122/23 3:951acef3:::rbd_data.112d1bb7b269.0000000000000005:head v 122'176, mlcod=122'172)", 2024-09-14T06:31:36.055 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:30:44.739092+0000", 2024-09-14T06:31:36.055 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 51.281773004000001, 2024-09-14T06:31:36.055 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0028236659999999998, 2024-09-14T06:31:36.056 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:31:36.056 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:31:36.056 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:31:36.056 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:31:36.056 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:36.056 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:31:36.056 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.739092+0000", 2024-09-14T06:31:36.056 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:36.056 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:36.056 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:36.056 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:31:36.056 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.739092+0000", 2024-09-14T06:31:36.057 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:36.057 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:36.057 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:36.057 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:31:36.057 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.739093+0000", 2024-09-14T06:31:36.057 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.0109999999999999e-06 2024-09-14T06:31:36.057 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:36.057 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:36.057 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:31:36.057 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.740189+0000", 2024-09-14T06:31:36.057 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00109527 2024-09-14T06:31:36.057 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:36.057 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:36.058 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:31:36.058 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.740199+0000", 2024-09-14T06:31:36.058 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.0458999999999999e-05 2024-09-14T06:31:36.058 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:36.058 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:36.058 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:31:36.058 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.740207+0000", 2024-09-14T06:31:36.058 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.4499999999999998e-06 2024-09-14T06:31:36.058 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:36.058 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:36.058 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:31:36.058 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.740295+0000", 2024-09-14T06:31:36.059 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.8740999999999996e-05 2024-09-14T06:31:36.059 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:36.059 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:36.059 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:31:36.059 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.740317+0000", 2024-09-14T06:31:36.059 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.1523e-05 2024-09-14T06:31:36.059 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:36.059 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:36.059 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-14T06:31:36.059 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.741865+0000", 2024-09-14T06:31:36.059 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.001548315 2024-09-14T06:31:36.059 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:36.060 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:36.060 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:31:36.060 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.741916+0000", 2024-09-14T06:31:36.060 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.0896999999999997e-05 2024-09-14T06:31:36.060 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:36.060 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:31:36.060 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:36.060 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:36.060 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:36.060 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4397.0:149933 3.4 e122/23 3:234f5229:::rbd_data.112d1bb7b269.0000000000000009:head v 122'221, mlcod=120'218)", 2024-09-14T06:31:36.060 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:30:44.743849+0000", 2024-09-14T06:31:36.061 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 51.277016586999999, 2024-09-14T06:31:36.061 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0031479519999999999, 2024-09-14T06:31:36.061 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:31:36.061 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:31:36.061 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:31:36.061 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:31:36.061 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:36.061 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:31:36.061 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.743849+0000", 2024-09-14T06:31:36.061 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:36.061 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:36.062 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:36.062 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:31:36.062 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.743849+0000", 2024-09-14T06:31:36.062 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:36.062 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:36.062 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:36.062 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:31:36.062 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.743850+0000", 2024-09-14T06:31:36.062 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.279e-06 2024-09-14T06:31:36.062 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:36.062 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:36.062 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:31:36.063 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.745151+0000", 2024-09-14T06:31:36.063 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.001300495 2024-09-14T06:31:36.063 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:36.063 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:36.063 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:31:36.063 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.745167+0000", 2024-09-14T06:31:36.063 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.6478e-05 2024-09-14T06:31:36.063 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:36.063 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:36.063 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:31:36.063 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.745175+0000", 2024-09-14T06:31:36.064 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.2679999999999994e-06 2024-09-14T06:31:36.064 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:36.064 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:36.064 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:31:36.064 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.745226+0000", 2024-09-14T06:31:36.064 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.0192999999999999e-05 2024-09-14T06:31:36.064 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:36.064 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:36.064 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:31:36.064 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.745247+0000", 2024-09-14T06:31:36.064 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.1767e-05 2024-09-14T06:31:36.064 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:36.065 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:36.065 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-14T06:31:36.065 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.746952+0000", 2024-09-14T06:31:36.065 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0017042769999999999 2024-09-14T06:31:36.065 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:36.065 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:36.065 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:31:36.065 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.746997+0000", 2024-09-14T06:31:36.065 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.5194999999999997e-05 2024-09-14T06:31:36.065 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:36.065 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:31:36.065 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:36.066 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:36.066 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:36.066 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4397.0:150494 3.9 e122/23 3:955152a4:::rbd_object_map.112d1bb7b269.0000000000000038:head v 122'194, mlcod=122'190)", 2024-09-14T06:31:36.066 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:30:44.781585+0000", 2024-09-14T06:31:36.066 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 51.239280694999998, 2024-09-14T06:31:36.066 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0026957499999999998, 2024-09-14T06:31:36.066 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:31:36.066 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:31:36.066 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:31:36.066 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:31:36.066 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:36.066 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:31:36.066 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.781585+0000", 2024-09-14T06:31:36.067 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:36.067 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:36.067 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:36.067 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:31:36.067 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.781585+0000", 2024-09-14T06:31:36.067 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:36.067 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:36.067 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:36.067 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:31:36.067 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.781585+0000", 2024-09-14T06:31:36.067 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.5000000000000002e-07 2024-09-14T06:31:36.067 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:36.067 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:36.068 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:31:36.068 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.781594+0000", 2024-09-14T06:31:36.068 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.4640000000000006e-06 2024-09-14T06:31:36.068 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:36.068 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:36.068 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:31:36.068 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.781604+0000", 2024-09-14T06:31:36.068 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.0071999999999999e-05 2024-09-14T06:31:36.068 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:36.068 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:36.068 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:31:36.068 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.781610+0000", 2024-09-14T06:31:36.068 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.3609999999999997e-06 2024-09-14T06:31:36.069 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:36.069 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:36.069 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:31:36.069 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.781676+0000", 2024-09-14T06:31:36.069 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.5963000000000005e-05 2024-09-14T06:31:36.069 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:36.069 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:36.069 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:31:36.069 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.781696+0000", 2024-09-14T06:31:36.069 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.9278000000000001e-05 2024-09-14T06:31:36.069 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:36.069 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:36.069 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-14T06:31:36.070 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.784231+0000", 2024-09-14T06:31:36.070 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.002535093 2024-09-14T06:31:36.070 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:36.070 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:36.070 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:31:36.070 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.784280+0000", 2024-09-14T06:31:36.070 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.9869e-05 2024-09-14T06:31:36.070 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:36.070 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:31:36.070 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:36.070 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:36.071 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:36.071 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4397.0:150898 3.9 e122/23 3:97c98f90:::rbd_data.112d1bb7b269.0000000000000025:head v 122'206, mlcod=122'203)", 2024-09-14T06:31:36.071 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:30:44.807232+0000", 2024-09-14T06:31:36.071 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 51.213633778000002, 2024-09-14T06:31:36.071 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0022297889999999998, 2024-09-14T06:31:36.071 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:31:36.071 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:31:36.071 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:31:36.071 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:31:36.071 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:36.071 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:31:36.071 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.807232+0000", 2024-09-14T06:31:36.072 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:36.072 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:36.072 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:36.072 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:31:36.072 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.807232+0000", 2024-09-14T06:31:36.072 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:36.072 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:36.072 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:36.072 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:31:36.072 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.807233+0000", 2024-09-14T06:31:36.072 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.5710000000000001e-06 2024-09-14T06:31:36.072 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:36.073 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:36.073 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:31:36.073 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.807944+0000", 2024-09-14T06:31:36.073 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00071056299999999999 2024-09-14T06:31:36.073 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:36.073 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:36.073 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:31:36.073 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.807961+0000", 2024-09-14T06:31:36.073 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.7689e-05 2024-09-14T06:31:36.073 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:36.074 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:36.074 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:31:36.074 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.807971+0000", 2024-09-14T06:31:36.074 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.2e-06 2024-09-14T06:31:36.074 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:36.074 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:36.074 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:31:36.074 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.808050+0000", 2024-09-14T06:31:36.074 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.9308999999999996e-05 2024-09-14T06:31:36.074 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:36.074 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:36.074 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:31:36.074 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.808074+0000", 2024-09-14T06:31:36.075 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.4147000000000001e-05 2024-09-14T06:31:36.075 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:36.075 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:36.075 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-14T06:31:36.075 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.809415+0000", 2024-09-14T06:31:36.075 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.001341257 2024-09-14T06:31:36.075 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:36.075 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:36.075 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:31:36.075 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.809461+0000", 2024-09-14T06:31:36.075 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.6053000000000001e-05 2024-09-14T06:31:36.075 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:36.076 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:31:36.076 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:36.076 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:36.076 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:36.076 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4397.0:202551 3.9 e166/23 3:965c09b6:::rbd_data.112d3bd84cb0.000000000000000a:head v 166'416, mlcod=164'414)", 2024-09-14T06:31:36.076 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:31:30.657011+0000", 2024-09-14T06:31:36.076 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 5.3638547450000003, 2024-09-14T06:31:36.076 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.002509262, 2024-09-14T06:31:36.076 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:31:36.076 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:31:36.076 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:31:36.077 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:31:36.077 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:36.077 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:31:36.077 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:30.657011+0000", 2024-09-14T06:31:36.077 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:36.077 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:36.077 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:36.077 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:31:36.077 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:30.657011+0000", 2024-09-14T06:31:36.077 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:36.077 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:36.078 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:36.078 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:31:36.078 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:30.657011+0000", 2024-09-14T06:31:36.078 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.08e-07 2024-09-14T06:31:36.078 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:36.078 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:36.078 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:31:36.078 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:30.658467+0000", 2024-09-14T06:31:36.078 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.001455708 2024-09-14T06:31:36.078 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:36.078 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:36.078 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:31:36.079 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:30.658484+0000", 2024-09-14T06:31:36.079 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.6844e-05 2024-09-14T06:31:36.079 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:36.079 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:36.079 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:31:36.079 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:30.658493+0000", 2024-09-14T06:31:36.079 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.2790000000000004e-06 2024-09-14T06:31:36.079 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:36.079 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:36.079 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:31:36.079 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:30.658559+0000", 2024-09-14T06:31:36.080 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.6459999999999997e-05 2024-09-14T06:31:36.080 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:36.080 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:36.080 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:31:36.080 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:30.658574+0000", 2024-09-14T06:31:36.080 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.5048000000000001e-05 2024-09-14T06:31:36.080 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:36.080 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:36.080 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-14T06:31:36.080 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:30.659473+0000", 2024-09-14T06:31:36.080 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00089842800000000005 2024-09-14T06:31:36.081 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:36.081 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:36.081 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:31:36.081 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:30.659520+0000", 2024-09-14T06:31:36.081 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.7086999999999998e-05 2024-09-14T06:31:36.081 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:36.081 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:31:36.081 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:36.081 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:36.081 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:36.082 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4397.0:202665 3.0 e166/23 3:00eeadfb:::rbd_data.112d3bd84cb0.000000000000000e:head v 166'920, mlcod=163'918)", 2024-09-14T06:31:36.082 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:31:30.662434+0000", 2024-09-14T06:31:36.082 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 5.3584309560000003, 2024-09-14T06:31:36.082 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.002522121, 2024-09-14T06:31:36.082 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:31:36.082 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:31:36.082 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:31:36.082 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:31:36.082 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:36.082 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:31:36.082 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:30.662434+0000", 2024-09-14T06:31:36.083 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:36.083 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:36.083 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:36.083 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:31:36.083 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:30.662434+0000", 2024-09-14T06:31:36.083 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:36.083 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:36.083 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:36.083 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:31:36.083 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:30.662436+0000", 2024-09-14T06:31:36.083 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.257e-06 2024-09-14T06:31:36.083 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:36.084 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:36.084 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:31:36.084 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:30.662527+0000", 2024-09-14T06:31:36.084 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.1644000000000006e-05 2024-09-14T06:31:36.084 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:36.084 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:36.084 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:31:36.084 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:30.662546+0000", 2024-09-14T06:31:36.084 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.8998e-05 2024-09-14T06:31:36.084 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:36.084 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:36.084 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:31:36.085 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:30.662556+0000", 2024-09-14T06:31:36.085 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.9809999999999998e-06 2024-09-14T06:31:36.085 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:36.085 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:36.085 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:31:36.085 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:30.662621+0000", 2024-09-14T06:31:36.085 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.4597000000000005e-05 2024-09-14T06:31:36.085 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:36.085 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:36.085 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:31:36.085 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:30.662643+0000", 2024-09-14T06:31:36.086 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.1608e-05 2024-09-14T06:31:36.086 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:36.086 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:36.086 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-14T06:31:36.086 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:30.664909+0000", 2024-09-14T06:31:36.086 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0022668689999999999 2024-09-14T06:31:36.086 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:36.086 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:36.086 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:31:36.086 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:30.664957+0000", 2024-09-14T06:31:36.086 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.7166999999999998e-05 2024-09-14T06:31:36.086 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:36.087 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:31:36.087 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:36.087 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:36.087 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:36.087 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4397.0:204589 3.4 e166/23 3:234552f0:::rbd_data.112d3bd84cb0.000000000000003f:head v 166'499, mlcod=165'497)", 2024-09-14T06:31:36.087 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:31:30.772894+0000", 2024-09-14T06:31:36.087 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 5.2479711, 2024-09-14T06:31:36.087 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0034564650000000001, 2024-09-14T06:31:36.087 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:31:36.087 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:31:36.087 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:31:36.087 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:31:36.088 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:36.088 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:31:36.088 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:30.772894+0000", 2024-09-14T06:31:36.088 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:36.088 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:36.088 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:36.088 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:31:36.088 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:30.772894+0000", 2024-09-14T06:31:36.088 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:36.088 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:36.089 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:36.089 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:31:36.089 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:30.772896+0000", 2024-09-14T06:31:36.089 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.5379999999999999e-06 2024-09-14T06:31:36.089 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:36.089 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:36.089 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:31:36.089 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:30.773042+0000", 2024-09-14T06:31:36.089 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000146107 2024-09-14T06:31:36.089 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:36.089 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:36.089 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:31:36.089 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:30.773061+0000", 2024-09-14T06:31:36.090 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.8862999999999999e-05 2024-09-14T06:31:36.090 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:36.090 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:36.090 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:31:36.090 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:30.773071+0000", 2024-09-14T06:31:36.090 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.0227000000000001e-05 2024-09-14T06:31:36.090 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:36.090 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:36.090 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:31:36.090 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:30.773149+0000", 2024-09-14T06:31:36.090 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.8071999999999994e-05 2024-09-14T06:31:36.090 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:36.091 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:36.091 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:31:36.091 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:30.773171+0000", 2024-09-14T06:31:36.091 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.1699999999999999e-05 2024-09-14T06:31:36.091 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:36.091 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:36.091 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-14T06:31:36.091 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:30.776285+0000", 2024-09-14T06:31:36.091 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0031142050000000001 2024-09-14T06:31:36.091 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:36.091 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:36.091 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:31:36.092 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:30.776351+0000", 2024-09-14T06:31:36.092 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.5753000000000006e-05 2024-09-14T06:31:36.092 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:36.092 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:31:36.092 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:36.092 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:36.092 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:36.092 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4397.0:204598 3.4 e166/23 3:245d069e:::rbd_data.112d3bd84cb0.0000000000000041:head v 166'500, mlcod=165'497)", 2024-09-14T06:31:36.092 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:31:30.775247+0000", 2024-09-14T06:31:36.092 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 5.2456179770000002, 2024-09-14T06:31:36.092 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00318235, 2024-09-14T06:31:36.092 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:31:36.092 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:31:36.093 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:31:36.093 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:31:36.093 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:36.093 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:31:36.093 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:30.775247+0000", 2024-09-14T06:31:36.093 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:36.093 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:36.093 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:36.093 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:31:36.093 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:30.775247+0000", 2024-09-14T06:31:36.094 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:36.094 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:36.094 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:36.094 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:31:36.094 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:30.775249+0000", 2024-09-14T06:31:36.094 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.265e-06 2024-09-14T06:31:36.094 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:36.094 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:36.094 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:31:36.094 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:30.775340+0000", 2024-09-14T06:31:36.094 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.1131999999999994e-05 2024-09-14T06:31:36.094 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:36.095 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:36.095 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:31:36.095 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:30.775352+0000", 2024-09-14T06:31:36.095 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.2294e-05 2024-09-14T06:31:36.095 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:36.095 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:36.095 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:31:36.095 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:30.775361+0000", 2024-09-14T06:31:36.095 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.6950000000000006e-06 2024-09-14T06:31:36.095 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:36.095 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:36.095 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:31:36.096 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:30.775439+0000", 2024-09-14T06:31:36.096 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.7910000000000005e-05 2024-09-14T06:31:36.096 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:36.096 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:36.096 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:31:36.096 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:30.775460+0000", 2024-09-14T06:31:36.096 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.0900999999999999e-05 2024-09-14T06:31:36.096 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:36.096 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:36.096 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-14T06:31:36.096 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:30.778385+0000", 2024-09-14T06:31:36.096 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.002925539 2024-09-14T06:31:36.097 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:36.097 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:36.097 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:31:36.097 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:30.778430+0000", 2024-09-14T06:31:36.097 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.4613999999999997e-05 2024-09-14T06:31:36.097 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:36.097 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:31:36.097 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:36.097 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:36.097 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:36.097 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4397.0:204975 3.4 e166/23 3:20972d18:::rbd_data.112d3bd84cb0.0000000000000046:head v 166'501, mlcod=166'499)", 2024-09-14T06:31:36.097 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:31:30.787253+0000", 2024-09-14T06:31:36.098 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 5.2336119480000001, 2024-09-14T06:31:36.098 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.002248318, 2024-09-14T06:31:36.098 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:31:36.098 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:31:36.098 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:31:36.098 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:31:36.098 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:36.098 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:31:36.099 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:30.787253+0000", 2024-09-14T06:31:36.099 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:36.099 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:36.099 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:36.099 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:31:36.099 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:30.787253+0000", 2024-09-14T06:31:36.099 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:36.099 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:36.099 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:36.099 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:31:36.099 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:30.787255+0000", 2024-09-14T06:31:36.099 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.271e-06 2024-09-14T06:31:36.099 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:36.100 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:36.100 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:31:36.100 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:30.787442+0000", 2024-09-14T06:31:36.100 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00018744000000000001 2024-09-14T06:31:36.100 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:36.100 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:36.100 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:31:36.100 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:30.787461+0000", 2024-09-14T06:31:36.100 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.9054000000000001e-05 2024-09-14T06:31:36.100 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:36.100 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:36.101 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:31:36.101 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:30.787470+0000", 2024-09-14T06:31:36.101 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.0929999999999992e-06 2024-09-14T06:31:36.101 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:36.101 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:36.101 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:31:36.101 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:30.788004+0000", 2024-09-14T06:31:36.101 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00053359399999999997 2024-09-14T06:31:36.101 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:36.101 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:36.101 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:31:36.102 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:30.788034+0000", 2024-09-14T06:31:36.102 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.9635e-05 2024-09-14T06:31:36.102 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:36.102 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:36.102 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-14T06:31:36.102 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:30.789457+0000", 2024-09-14T06:31:36.102 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.001423487 2024-09-14T06:31:36.102 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:36.102 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:36.102 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:31:36.102 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:30.789502+0000", 2024-09-14T06:31:36.102 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.4743999999999997e-05 2024-09-14T06:31:36.102 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:36.102 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:31:36.103 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:36.103 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:36.103 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:36.103 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4397.0:212188 3.4 e166/23 3:23871bc7:::rbd_data.112d3bd84cb0.00000000000000fa:head v 166'508, mlcod=166'506)", 2024-09-14T06:31:36.103 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:31:31.391415+0000", 2024-09-14T06:31:36.103 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 4.6294503660000004, 2024-09-14T06:31:36.103 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0022039049999999999, 2024-09-14T06:31:36.103 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:31:36.103 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:31:36.103 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:31:36.103 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:31:36.103 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:36.103 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:31:36.104 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:31.391415+0000", 2024-09-14T06:31:36.104 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:36.104 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:36.104 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:36.104 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:31:36.104 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:31.391415+0000", 2024-09-14T06:31:36.104 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:36.104 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:36.104 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:36.104 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:31:36.104 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:31.391416+0000", 2024-09-14T06:31:36.104 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.44e-06 2024-09-14T06:31:36.105 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:36.105 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:36.105 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:31:36.105 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:31.391554+0000", 2024-09-14T06:31:36.105 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000137119 2024-09-14T06:31:36.105 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:36.105 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:36.105 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:31:36.105 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:31.391570+0000", 2024-09-14T06:31:36.105 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.6827e-05 2024-09-14T06:31:36.105 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:36.105 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:36.106 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:31:36.106 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:31.391580+0000", 2024-09-14T06:31:36.106 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.3139999999999993e-06 2024-09-14T06:31:36.106 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:36.106 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:36.106 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:31:36.106 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:31.391664+0000", 2024-09-14T06:31:36.106 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.3757999999999993e-05 2024-09-14T06:31:36.106 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:36.106 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:36.106 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:31:36.106 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:31.391692+0000", 2024-09-14T06:31:36.106 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.8898e-05 2024-09-14T06:31:36.107 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:36.107 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:36.107 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-14T06:31:36.107 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:31.393584+0000", 2024-09-14T06:31:36.107 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.001891752 2024-09-14T06:31:36.107 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:36.107 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:36.107 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:31:36.107 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:31.393619+0000", 2024-09-14T06:31:36.107 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.4796999999999999e-05 2024-09-14T06:31:36.107 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:36.107 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:31:36.107 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:36.108 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:36.108 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:36.108 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "MOSDPGPush(3.5 169/168 [PushOp(3:a7c12b08:::rbd_data.112ddfbec471.000000000000000e:head, version: 77'100, data_included: [0~1048576], data_size: 1048576, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(3:a7c12b08:::rbd_data.112ddfbec471.000000000000000e:head@77'100, size: 1048576, copy_subset: [(0, 1048576)], clone_subset: {}, snapset: 0=[]:{}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 1048576, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2024-09-14T06:31:36.108 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:31:34.540707+0000", 2024-09-14T06:31:36.108 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 1.4801586010000001, 2024-09-14T06:31:36.108 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0023637369999999999, 2024-09-14T06:31:36.108 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:31:36.108 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:31:36.108 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:31:36.108 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:31:36.108 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:36.109 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:31:36.109 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:34.540707+0000", 2024-09-14T06:31:36.109 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:36.109 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:36.109 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:36.109 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:31:36.109 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:34.540707+0000", 2024-09-14T06:31:36.109 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:36.109 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:36.109 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:36.109 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:31:36.109 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:34.540708+0000", 2024-09-14T06:31:36.110 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.7199999999999998e-07 2024-09-14T06:31:36.110 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:36.110 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:36.110 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:31:36.110 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:34.542150+0000", 2024-09-14T06:31:36.110 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0014423229999999999 2024-09-14T06:31:36.110 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:36.110 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:36.110 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:31:36.110 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:34.542169+0000", 2024-09-14T06:31:36.110 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.8756000000000001e-05 2024-09-14T06:31:36.110 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:36.111 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:36.111 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:31:36.111 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:34.542180+0000", 2024-09-14T06:31:36.111 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.1051e-05 2024-09-14T06:31:36.111 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:36.111 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:36.111 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:31:36.111 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:34.542275+0000", 2024-09-14T06:31:36.111 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.5122000000000002e-05 2024-09-14T06:31:36.111 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:36.111 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:36.112 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:31:36.112 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:34.542284+0000", 2024-09-14T06:31:36.112 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.9190000000000005e-06 2024-09-14T06:31:36.112 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:36.112 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:36.112 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:31:36.112 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:34.543071+0000", 2024-09-14T06:31:36.112 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00078679399999999995 2024-09-14T06:31:36.112 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:36.112 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:31:36.112 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:36.113 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:36.113 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:36.113 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "MOSDPGPush(3.5 169/168 [PushOp(3:a5c8b3f2:::rbd_data.112ddfbec471.0000000000000018:head, version: 77'101, data_included: [0~1048576], data_size: 1048576, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(3:a5c8b3f2:::rbd_data.112ddfbec471.0000000000000018:head@77'101, size: 1048576, copy_subset: [(0, 1048576)], clone_subset: {}, snapset: 0=[]:{}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 1048576, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2024-09-14T06:31:36.113 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:31:34.545533+0000", 2024-09-14T06:31:36.113 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 1.4753327139999999, 2024-09-14T06:31:36.113 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0021867010000000001, 2024-09-14T06:31:36.113 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:31:36.113 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:31:36.113 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:31:36.113 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:31:36.113 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:36.113 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:31:36.114 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:34.545533+0000", 2024-09-14T06:31:36.114 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:36.114 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:36.114 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:36.114 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:31:36.114 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:34.545533+0000", 2024-09-14T06:31:36.114 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:36.114 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:36.114 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:36.114 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:31:36.114 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:34.545533+0000", 2024-09-14T06:31:36.114 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.8100000000000002e-07 2024-09-14T06:31:36.114 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:36.115 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:36.115 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:31:36.115 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:34.546784+0000", 2024-09-14T06:31:36.115 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.001250747 2024-09-14T06:31:36.115 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:36.115 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:36.115 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:31:36.115 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:34.546803+0000", 2024-09-14T06:31:36.115 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.8346e-05 2024-09-14T06:31:36.115 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:36.115 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:36.115 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:31:36.115 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:34.546823+0000", 2024-09-14T06:31:36.116 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.0699999999999998e-05 2024-09-14T06:31:36.116 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:36.116 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:36.116 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:31:36.116 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:34.546890+0000", 2024-09-14T06:31:36.116 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.6581000000000005e-05 2024-09-14T06:31:36.116 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:36.116 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:36.116 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:31:36.116 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:34.546899+0000", 2024-09-14T06:31:36.116 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.3889999999999994e-06 2024-09-14T06:31:36.116 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:36.116 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:36.117 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:31:36.117 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:34.547719+0000", 2024-09-14T06:31:36.117 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00082015699999999996 2024-09-14T06:31:36.117 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:36.117 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:31:36.117 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:36.117 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:36.117 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:36.117 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "MOSDPGPush(3.5 169/168 [PushOp(3:a7a54476:::rbd_data.112d1bb7b269.0000000000000038:head, version: 122'581, data_included: [0~524288], data_size: 524288, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(3:a7a54476:::rbd_data.112d1bb7b269.0000000000000038:head@122'581, size: 524288, copy_subset: [(0, 524288)], clone_subset: {}, snapset: 0=[]:{}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 524288, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2024-09-14T06:31:36.117 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:31:34.667757+0000", 2024-09-14T06:31:36.117 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 1.3531085549999999, 2024-09-14T06:31:36.117 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.002750382, 2024-09-14T06:31:36.118 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:31:36.118 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:31:36.118 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:31:36.118 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:31:36.118 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:36.118 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:31:36.118 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:34.667757+0000", 2024-09-14T06:31:36.118 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:36.118 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:36.118 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:36.118 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:31:36.118 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:34.667757+0000", 2024-09-14T06:31:36.118 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:36.118 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:36.119 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:36.119 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:31:36.119 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:34.667758+0000", 2024-09-14T06:31:36.119 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.3599999999999999e-06 2024-09-14T06:31:36.119 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:36.119 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:36.119 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:31:36.119 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:34.669375+0000", 2024-09-14T06:31:36.119 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.001616644 2024-09-14T06:31:36.119 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:36.119 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:36.119 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:31:36.119 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:34.669404+0000", 2024-09-14T06:31:36.120 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.9300999999999999e-05 2024-09-14T06:31:36.120 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:36.120 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:36.120 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:31:36.120 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:34.669428+0000", 2024-09-14T06:31:36.120 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.3788e-05 2024-09-14T06:31:36.120 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:36.120 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:36.120 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:31:36.120 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:34.669543+0000", 2024-09-14T06:31:36.120 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000115462 2024-09-14T06:31:36.120 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:36.121 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:36.121 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:31:36.121 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:34.669559+0000", 2024-09-14T06:31:36.121 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.5167999999999999e-05 2024-09-14T06:31:36.121 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:36.121 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:36.121 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:31:36.121 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:34.670507+0000", 2024-09-14T06:31:36.121 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00094865900000000003 2024-09-14T06:31:36.121 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:36.121 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:31:36.121 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:36.122 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:36.122 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:36.122 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "MOSDPGPush(3.5 170/168 [PushOp(3:a162138c:::rbd_data.112d1bb7b269.000000000000003c:head, version: 122'582, data_included: [0~524288], data_size: 524288, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(3:a162138c:::rbd_data.112d1bb7b269.000000000000003c:head@122'582, size: 524288, copy_subset: [(0, 524288)], clone_subset: {}, snapset: 0=[]:{}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 524288, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2024-09-14T06:31:36.122 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:31:34.670110+0000", 2024-09-14T06:31:36.122 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 1.3507556059999999, 2024-09-14T06:31:36.122 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0030066229999999999, 2024-09-14T06:31:36.122 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:31:36.122 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:31:36.122 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:31:36.122 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:31:36.122 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:36.122 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:31:36.122 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:34.670110+0000", 2024-09-14T06:31:36.123 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:36.123 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:36.123 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:36.123 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:31:36.123 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:34.670110+0000", 2024-09-14T06:31:36.123 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:36.123 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:36.123 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:36.123 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:31:36.123 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:34.670111+0000", 2024-09-14T06:31:36.123 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.6499999999999998e-07 2024-09-14T06:31:36.123 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:36.123 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:36.124 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:31:36.124 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:34.671507+0000", 2024-09-14T06:31:36.124 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.001396284 2024-09-14T06:31:36.124 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:36.124 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:36.124 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:31:36.124 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:34.671530+0000", 2024-09-14T06:31:36.124 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.3023e-05 2024-09-14T06:31:36.124 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:36.124 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:36.124 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:31:36.124 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:34.671548+0000", 2024-09-14T06:31:36.124 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.8116000000000001e-05 2024-09-14T06:31:36.125 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:36.125 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:36.125 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:31:36.125 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:34.671775+0000", 2024-09-14T06:31:36.125 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00022699600000000001 2024-09-14T06:31:36.125 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:36.125 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:36.125 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:31:36.125 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:34.671790+0000", 2024-09-14T06:31:36.125 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.539e-05 2024-09-14T06:31:36.125 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:36.125 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:36.126 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:31:36.126 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:34.673116+0000", 2024-09-14T06:31:36.126 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0013259490000000001 2024-09-14T06:31:36.126 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:36.126 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:31:36.126 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:36.126 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:36.126 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:31:36.126 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-14T06:31:36.128 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-14T06:31:36.128 INFO:tasks.ceph.osd.5.smithi138.stderr:2024-09-14T06:31:36.126+0000 7fe13746f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33618) UID: 0 2024-09-14T06:31:36.229 INFO:tasks.ceph.osd.1.smithi094.stderr:2024-09-14T06:31:36.227+0000 7f328e47a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33641) UID: 0 2024-09-14T06:31:36.232 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-14T06:31:36.232 INFO:teuthology.orchestra.run.smithi094.stdout: "ops": [], 2024-09-14T06:31:36.232 INFO:teuthology.orchestra.run.smithi094.stdout: "num_ops": 0 2024-09-14T06:31:36.232 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-14T06:31:36.241 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-14T06:31:36.330 INFO:tasks.ceph.osd.7.smithi138.stderr:2024-09-14T06:31:36.328+0000 7f3c6a21d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33623) UID: 0 2024-09-14T06:31:36.383 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-14T06:31:36.383 INFO:teuthology.orchestra.run.smithi094.stdout: "ops": [], 2024-09-14T06:31:36.383 INFO:teuthology.orchestra.run.smithi094.stdout: "complaint_time": 30, 2024-09-14T06:31:36.383 INFO:teuthology.orchestra.run.smithi094.stdout: "num_blocked_ops": 0 2024-09-14T06:31:36.383 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-14T06:31:36.392 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-14T06:31:36.410 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-14T06:31:36.430 INFO:tasks.ceph.osd.4.smithi138.stderr:2024-09-14T06:31:36.429+0000 7f6c653c8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33615) UID: 0 2024-09-14T06:31:36.499 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-14T06:31:36.499 INFO:teuthology.orchestra.run.smithi094.stdout: "size": 20, 2024-09-14T06:31:36.499 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 600, 2024-09-14T06:31:36.499 INFO:teuthology.orchestra.run.smithi094.stdout: "ops": [ 2024-09-14T06:31:36.499 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:36.499 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:44425 3.2 3:4603ad7f:::rbd_data.112dca2f6ae1.0000000000000005:17 [read 868352~4096 [fadvise_nocache]] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:31:36.499 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:43.854967+0000", 2024-09-14T06:31:36.499 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 112.64207650900001, 2024-09-14T06:31:36.499 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.063146025999999994, 2024-09-14T06:31:36.499 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:31:36.499 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:31:36.500 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:31:36.500 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:31:36.500 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:31:36.500 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:36.500 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 44425 2024-09-14T06:31:36.500 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:36.500 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:31:36.500 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:36.500 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:31:36.500 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.854967+0000", 2024-09-14T06:31:36.500 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:36.500 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:36.500 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:36.501 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:31:36.501 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.854967+0000", 2024-09-14T06:31:36.501 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:36.501 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:36.501 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:36.501 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:31:36.501 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.854967+0000", 2024-09-14T06:31:36.501 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.16e-07 2024-09-14T06:31:36.501 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:36.501 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:36.501 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:31:36.501 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.854968+0000", 2024-09-14T06:31:36.501 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.006e-06 2024-09-14T06:31:36.502 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:36.502 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:36.502 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:31:36.502 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.854971+0000", 2024-09-14T06:31:36.502 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.9359999999999999e-06 2024-09-14T06:31:36.502 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:36.502 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:36.502 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:31:36.502 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.854974+0000", 2024-09-14T06:31:36.502 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.1840000000000002e-06 2024-09-14T06:31:36.502 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:36.502 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:36.502 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:31:36.502 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.917945+0000", 2024-09-14T06:31:36.503 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.062970925999999997 2024-09-14T06:31:36.503 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:36.503 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:36.503 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:31:36.503 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.918030+0000", 2024-09-14T06:31:36.503 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.5863000000000001e-05 2024-09-14T06:31:36.503 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:36.503 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:36.503 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:31:36.503 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.918113+0000", 2024-09-14T06:31:36.503 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.2894999999999998e-05 2024-09-14T06:31:36.503 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:36.503 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:31:36.503 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:36.504 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:36.504 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:36.504 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:44426 3.2 3:4603ad7f:::rbd_data.112dca2f6ae1.0000000000000005:17 [read 892928~4096 [fadvise_nocache]] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:31:36.504 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:43.854977+0000", 2024-09-14T06:31:36.504 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 112.642067099, 2024-09-14T06:31:36.504 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.063369237999999994, 2024-09-14T06:31:36.504 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:31:36.504 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:31:36.504 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:31:36.504 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:31:36.504 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:31:36.504 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:36.504 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 44426 2024-09-14T06:31:36.505 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:36.505 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:31:36.505 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:36.505 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:31:36.505 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.854977+0000", 2024-09-14T06:31:36.505 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:36.505 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:36.505 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:36.505 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:31:36.505 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.854977+0000", 2024-09-14T06:31:36.505 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:36.505 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:36.505 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:36.505 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:31:36.505 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.854977+0000", 2024-09-14T06:31:36.506 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.1199999999999999e-07 2024-09-14T06:31:36.506 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:36.506 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:36.506 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:31:36.506 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.854980+0000", 2024-09-14T06:31:36.506 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.27e-06 2024-09-14T06:31:36.506 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:36.506 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:36.506 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:31:36.506 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.854983+0000", 2024-09-14T06:31:36.506 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.1650000000000002e-06 2024-09-14T06:31:36.506 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:36.506 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:36.506 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:31:36.507 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.854989+0000", 2024-09-14T06:31:36.507 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.6520000000000003e-06 2024-09-14T06:31:36.507 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:36.507 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:36.507 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:31:36.507 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.918143+0000", 2024-09-14T06:31:36.507 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.063154541999999994 2024-09-14T06:31:36.507 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:36.507 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:36.507 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:31:36.507 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.918209+0000", 2024-09-14T06:31:36.507 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.5529000000000002e-05 2024-09-14T06:31:36.507 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:36.508 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:36.508 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:31:36.508 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.918346+0000", 2024-09-14T06:31:36.508 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000136868 2024-09-14T06:31:36.508 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:36.508 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:31:36.508 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:36.508 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:36.508 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:36.508 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:44427 3.2 3:4603ad7f:::rbd_data.112dca2f6ae1.0000000000000005:17 [read 917504~4096 [fadvise_nocache]] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:31:36.508 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:43.854992+0000", 2024-09-14T06:31:36.508 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 112.64205145299999, 2024-09-14T06:31:36.508 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.063607792999999996, 2024-09-14T06:31:36.509 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:31:36.509 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:31:36.509 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:31:36.509 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:31:36.509 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:31:36.509 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:36.509 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 44427 2024-09-14T06:31:36.509 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:36.509 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:31:36.509 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:36.509 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:31:36.509 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.854992+0000", 2024-09-14T06:31:36.509 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:36.509 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:36.510 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:36.510 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:31:36.510 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.854992+0000", 2024-09-14T06:31:36.510 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:36.510 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:36.510 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:36.510 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:31:36.510 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.854993+0000", 2024-09-14T06:31:36.510 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.1400000000000001e-07 2024-09-14T06:31:36.510 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:36.510 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:36.510 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:31:36.510 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.854994+0000", 2024-09-14T06:31:36.511 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.091e-06 2024-09-14T06:31:36.511 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:36.511 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:36.511 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:31:36.511 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855001+0000", 2024-09-14T06:31:36.511 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.204e-06 2024-09-14T06:31:36.511 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:36.511 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:36.511 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:31:36.511 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855003+0000", 2024-09-14T06:31:36.511 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.514e-06 2024-09-14T06:31:36.511 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:36.511 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:36.511 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:31:36.511 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.918380+0000", 2024-09-14T06:31:36.512 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.063376738000000002 2024-09-14T06:31:36.512 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:36.512 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:36.512 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:31:36.512 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.918453+0000", 2024-09-14T06:31:36.512 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.3291e-05 2024-09-14T06:31:36.512 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:36.512 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:36.512 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:31:36.512 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.918600+0000", 2024-09-14T06:31:36.512 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00014674100000000001 2024-09-14T06:31:36.512 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:36.512 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:31:36.513 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:36.513 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:36.513 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:36.513 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:44428 3.2 3:4603ad7f:::rbd_data.112dca2f6ae1.0000000000000005:17 [read 942080~4096 [fadvise_nocache]] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:31:36.513 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:43.855006+0000", 2024-09-14T06:31:36.513 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 112.642037403, 2024-09-14T06:31:36.513 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.063846321999999997, 2024-09-14T06:31:36.513 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:31:36.513 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:31:36.513 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:31:36.513 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:31:36.513 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:31:36.514 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:36.514 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 44428 2024-09-14T06:31:36.514 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:36.514 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:31:36.514 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:36.514 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:31:36.514 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855006+0000", 2024-09-14T06:31:36.514 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:36.514 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:36.514 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:36.514 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:31:36.514 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855006+0000", 2024-09-14T06:31:36.514 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:36.514 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:36.514 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:36.515 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:31:36.515 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855007+0000", 2024-09-14T06:31:36.515 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.1199999999999999e-07 2024-09-14T06:31:36.515 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:36.515 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:36.515 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:31:36.515 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855011+0000", 2024-09-14T06:31:36.515 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.9539999999999998e-06 2024-09-14T06:31:36.515 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:36.515 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:36.515 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:31:36.515 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855014+0000", 2024-09-14T06:31:36.515 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.2930000000000001e-06 2024-09-14T06:31:36.515 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:36.515 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:36.516 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:31:36.516 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855016+0000", 2024-09-14T06:31:36.516 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.3039999999999999e-06 2024-09-14T06:31:36.516 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:36.516 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:36.516 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:31:36.516 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.918634+0000", 2024-09-14T06:31:36.516 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.063617928000000004 2024-09-14T06:31:36.516 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:36.516 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:36.516 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:31:36.516 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.918711+0000", 2024-09-14T06:31:36.516 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.6545000000000007e-05 2024-09-14T06:31:36.516 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:36.517 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:36.517 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:31:36.517 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.918853+0000", 2024-09-14T06:31:36.517 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00014208600000000001 2024-09-14T06:31:36.517 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:36.517 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:31:36.517 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:36.517 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:36.517 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:36.517 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:44429 3.2 3:4603ad7f:::rbd_data.112dca2f6ae1.0000000000000005:17 [read 966656~4096 [fadvise_nocache]] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:31:36.517 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:43.855019+0000", 2024-09-14T06:31:36.517 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 112.64202473, 2024-09-14T06:31:36.517 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.064059226999999996, 2024-09-14T06:31:36.517 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:31:36.517 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:31:36.518 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:31:36.518 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:31:36.518 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:31:36.518 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:36.518 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 44429 2024-09-14T06:31:36.518 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:36.518 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:31:36.518 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:36.518 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:31:36.518 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855019+0000", 2024-09-14T06:31:36.518 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:36.518 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:36.518 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:36.518 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:31:36.518 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855019+0000", 2024-09-14T06:31:36.518 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:36.519 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:36.519 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:36.519 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:31:36.519 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855019+0000", 2024-09-14T06:31:36.519 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.28e-07 2024-09-14T06:31:36.519 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:36.519 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:36.519 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:31:36.519 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855020+0000", 2024-09-14T06:31:36.519 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.0330000000000001e-06 2024-09-14T06:31:36.519 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:36.519 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:36.519 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:31:36.519 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855023+0000", 2024-09-14T06:31:36.519 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.79e-06 2024-09-14T06:31:36.519 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:36.520 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:36.520 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:31:36.520 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855025+0000", 2024-09-14T06:31:36.520 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.216e-06 2024-09-14T06:31:36.520 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:36.520 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:36.520 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:31:36.520 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.918886+0000", 2024-09-14T06:31:36.520 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.063861037999999995 2024-09-14T06:31:36.520 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:36.520 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:36.520 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:31:36.520 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.918952+0000", 2024-09-14T06:31:36.520 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.5993000000000003e-05 2024-09-14T06:31:36.520 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:36.521 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:36.521 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:31:36.521 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.919078+0000", 2024-09-14T06:31:36.521 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00012592899999999999 2024-09-14T06:31:36.521 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:36.521 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:31:36.521 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:36.521 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:36.521 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:36.521 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:44430 3.2 3:4603ad7f:::rbd_data.112dca2f6ae1.0000000000000005:17 [read 991232~4096 [fadvise_nocache]] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:31:36.521 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:43.855028+0000", 2024-09-14T06:31:36.521 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 112.64201572, 2024-09-14T06:31:36.521 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.064284346000000006, 2024-09-14T06:31:36.521 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:31:36.521 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:31:36.522 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:31:36.522 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:31:36.522 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:31:36.522 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:36.522 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 44430 2024-09-14T06:31:36.522 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:36.522 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:31:36.522 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:36.522 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:31:36.522 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855028+0000", 2024-09-14T06:31:36.522 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:36.522 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:36.522 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:36.522 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:31:36.522 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855028+0000", 2024-09-14T06:31:36.523 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:36.523 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:36.523 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:36.523 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:31:36.523 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855028+0000", 2024-09-14T06:31:36.523 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.1199999999999999e-07 2024-09-14T06:31:36.523 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:36.523 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:36.523 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:31:36.523 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855029+0000", 2024-09-14T06:31:36.523 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.1100000000000004e-07 2024-09-14T06:31:36.523 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:36.523 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:36.523 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:31:36.523 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855032+0000", 2024-09-14T06:31:36.523 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.7889999999999999e-06 2024-09-14T06:31:36.524 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:36.524 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:36.524 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:31:36.524 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855034+0000", 2024-09-14T06:31:36.524 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.125e-06 2024-09-14T06:31:36.524 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:36.524 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:36.524 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:31:36.524 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.919110+0000", 2024-09-14T06:31:36.524 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.064076413999999998 2024-09-14T06:31:36.524 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:36.524 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:36.524 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:31:36.524 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.919186+0000", 2024-09-14T06:31:36.524 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.5784999999999994e-05 2024-09-14T06:31:36.525 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:36.525 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:36.525 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:31:36.525 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.919312+0000", 2024-09-14T06:31:36.525 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00012611 2024-09-14T06:31:36.525 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:36.525 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:31:36.525 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:36.525 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:36.525 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:36.525 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:44431 3.2 3:4603ad7f:::rbd_data.112dca2f6ae1.0000000000000005:17 [read 1015808~4096 [fadvise_nocache]] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:31:36.525 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:43.855037+0000", 2024-09-14T06:31:36.525 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 112.64200685199999, 2024-09-14T06:31:36.525 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.064469755000000004, 2024-09-14T06:31:36.525 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:31:36.526 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:31:36.526 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:31:36.526 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:31:36.526 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:31:36.526 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:36.526 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 44431 2024-09-14T06:31:36.526 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:36.526 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:31:36.526 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:36.526 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:31:36.526 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855037+0000", 2024-09-14T06:31:36.526 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:36.526 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:36.526 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:36.526 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:31:36.526 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855037+0000", 2024-09-14T06:31:36.527 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:36.527 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:36.527 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:36.527 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:31:36.527 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855037+0000", 2024-09-14T06:31:36.527 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.1e-07 2024-09-14T06:31:36.527 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:36.527 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:36.527 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:31:36.527 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855038+0000", 2024-09-14T06:31:36.527 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.9400000000000004e-07 2024-09-14T06:31:36.527 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:36.527 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:36.527 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:31:36.527 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855041+0000", 2024-09-14T06:31:36.527 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.9639999999999999e-06 2024-09-14T06:31:36.528 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:36.528 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:36.528 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:31:36.528 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855043+0000", 2024-09-14T06:31:36.528 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.2259999999999999e-06 2024-09-14T06:31:36.528 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:36.528 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:36.528 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:31:36.528 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.919347+0000", 2024-09-14T06:31:36.528 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.064304091999999993 2024-09-14T06:31:36.528 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:36.528 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:36.528 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:31:36.528 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.919393+0000", 2024-09-14T06:31:36.528 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.5340999999999998e-05 2024-09-14T06:31:36.529 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:36.529 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:36.529 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:31:36.529 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.919507+0000", 2024-09-14T06:31:36.529 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000114028 2024-09-14T06:31:36.529 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:36.529 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:31:36.529 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:36.529 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:36.529 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:36.529 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:44432 3.2 3:4603ad7f:::rbd_data.112dca2f6ae1.0000000000000005:17 [read 1040384~4096 [fadvise_nocache]] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:31:36.529 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:43.855046+0000", 2024-09-14T06:31:36.529 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 112.641997784, 2024-09-14T06:31:36.529 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.064639253999999993, 2024-09-14T06:31:36.529 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:31:36.529 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:31:36.530 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:31:36.530 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:31:36.530 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:31:36.530 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:36.530 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 44432 2024-09-14T06:31:36.530 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:36.530 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:31:36.530 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:36.530 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:31:36.530 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855046+0000", 2024-09-14T06:31:36.530 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:36.530 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:36.530 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:36.531 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:31:36.531 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855046+0000", 2024-09-14T06:31:36.531 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:36.531 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:36.531 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:36.531 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:31:36.531 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855046+0000", 2024-09-14T06:31:36.531 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.05e-07 2024-09-14T06:31:36.531 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:36.531 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:36.531 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:31:36.531 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855047+0000", 2024-09-14T06:31:36.531 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.879999999999999e-07 2024-09-14T06:31:36.531 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:36.531 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:36.532 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:31:36.532 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855050+0000", 2024-09-14T06:31:36.532 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.8459999999999999e-06 2024-09-14T06:31:36.532 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:36.532 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:36.532 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:31:36.532 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855052+0000", 2024-09-14T06:31:36.532 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.136e-06 2024-09-14T06:31:36.532 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:36.532 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:36.532 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:31:36.532 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.919536+0000", 2024-09-14T06:31:36.532 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.064484132 2024-09-14T06:31:36.532 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:36.533 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:36.533 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:31:36.533 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.919611+0000", 2024-09-14T06:31:36.533 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.4407999999999996e-05 2024-09-14T06:31:36.533 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:36.533 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:36.533 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:31:36.533 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.919685+0000", 2024-09-14T06:31:36.533 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.4538999999999999e-05 2024-09-14T06:31:36.533 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:36.533 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:31:36.533 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:36.533 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:36.533 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:36.534 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:44518 3.2 3:4190d882:::rbd_data.112dca2f6ae1.0000000000000002:17 [read 12288~4096 [fadvise_nocache]] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:31:36.534 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:43.855055+0000", 2024-09-14T06:31:36.534 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 112.6419889, 2024-09-14T06:31:36.534 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.064894317000000007, 2024-09-14T06:31:36.534 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:31:36.534 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:31:36.534 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:31:36.534 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:31:36.534 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:31:36.534 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:36.534 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 44518 2024-09-14T06:31:36.534 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:36.534 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:31:36.534 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:36.535 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:31:36.535 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855055+0000", 2024-09-14T06:31:36.535 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:36.535 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:36.535 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:36.535 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:31:36.535 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855055+0000", 2024-09-14T06:31:36.535 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:36.535 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:36.535 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:36.535 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:31:36.535 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855055+0000", 2024-09-14T06:31:36.535 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.0699999999999999e-07 2024-09-14T06:31:36.535 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:36.536 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:36.536 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:31:36.536 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855056+0000", 2024-09-14T06:31:36.536 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.9400000000000004e-07 2024-09-14T06:31:36.536 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:36.536 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:36.536 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:31:36.536 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855059+0000", 2024-09-14T06:31:36.536 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.8870000000000001e-06 2024-09-14T06:31:36.536 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:36.536 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:36.536 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:31:36.536 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855061+0000", 2024-09-14T06:31:36.536 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.1919999999999999e-06 2024-09-14T06:31:36.537 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:36.537 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:36.537 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:31:36.537 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.919719+0000", 2024-09-14T06:31:36.537 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.064658013 2024-09-14T06:31:36.537 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:36.537 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:36.537 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:31:36.537 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.919788+0000", 2024-09-14T06:31:36.537 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.9282000000000005e-05 2024-09-14T06:31:36.537 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:36.537 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:36.537 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:31:36.537 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.919949+0000", 2024-09-14T06:31:36.538 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00016084199999999999 2024-09-14T06:31:36.538 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:36.538 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:31:36.538 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:36.538 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:36.538 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:36.538 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:44519 3.2 3:4190d882:::rbd_data.112dca2f6ae1.0000000000000002:17 [read 36864~4096 [fadvise_nocache]] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:31:36.538 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:43.855064+0000", 2024-09-14T06:31:36.538 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 112.64197992299999, 2024-09-14T06:31:36.538 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.065081865000000003, 2024-09-14T06:31:36.538 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:31:36.538 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:31:36.538 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:31:36.538 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:31:36.538 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:31:36.539 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:36.539 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 44519 2024-09-14T06:31:36.539 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:36.539 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:31:36.539 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:36.539 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:31:36.539 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855064+0000", 2024-09-14T06:31:36.539 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:36.539 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:36.539 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:36.539 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:31:36.539 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855064+0000", 2024-09-14T06:31:36.539 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:36.539 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:36.539 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:36.540 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:31:36.540 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855064+0000", 2024-09-14T06:31:36.540 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.1e-07 2024-09-14T06:31:36.540 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:36.540 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:36.540 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:31:36.540 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855065+0000", 2024-09-14T06:31:36.540 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.035e-06 2024-09-14T06:31:36.540 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:36.540 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:36.540 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:31:36.540 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855068+0000", 2024-09-14T06:31:36.540 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.8339999999999999e-06 2024-09-14T06:31:36.540 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:36.541 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:36.541 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:31:36.541 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855071+0000", 2024-09-14T06:31:36.541 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.3450000000000002e-06 2024-09-14T06:31:36.541 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:36.541 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:36.541 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:31:36.541 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.919979+0000", 2024-09-14T06:31:36.541 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.064907647999999998 2024-09-14T06:31:36.541 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:36.541 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:36.541 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:31:36.541 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.920026+0000", 2024-09-14T06:31:36.542 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.7595000000000001e-05 2024-09-14T06:31:36.542 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:36.542 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:36.542 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:31:36.542 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.920146+0000", 2024-09-14T06:31:36.542 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00011919800000000001 2024-09-14T06:31:36.542 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:36.542 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:31:36.542 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:36.542 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:36.542 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:36.542 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:44520 3.2 3:4190d882:::rbd_data.112dca2f6ae1.0000000000000002:17 [read 61440~4096 [fadvise_nocache]] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:31:36.542 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:43.855075+0000", 2024-09-14T06:31:36.542 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 112.64196887200001, 2024-09-14T06:31:36.542 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.065335078000000005, 2024-09-14T06:31:36.543 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:31:36.543 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:31:36.543 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:31:36.543 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:31:36.543 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:31:36.543 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:36.543 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 44520 2024-09-14T06:31:36.543 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:36.543 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:31:36.543 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:36.543 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:31:36.543 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855075+0000", 2024-09-14T06:31:36.543 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:36.543 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:36.543 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:36.544 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:31:36.544 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855075+0000", 2024-09-14T06:31:36.544 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:36.544 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:36.544 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:36.544 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:31:36.544 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855075+0000", 2024-09-14T06:31:36.544 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.1e-07 2024-09-14T06:31:36.544 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:36.544 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:36.544 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:31:36.544 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855076+0000", 2024-09-14T06:31:36.544 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.0839999999999999e-06 2024-09-14T06:31:36.544 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:36.544 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:36.545 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:31:36.545 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855079+0000", 2024-09-14T06:31:36.545 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.9359999999999999e-06 2024-09-14T06:31:36.545 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:36.545 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:36.545 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:31:36.545 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855084+0000", 2024-09-14T06:31:36.545 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.1730000000000003e-06 2024-09-14T06:31:36.545 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:36.545 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:36.545 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:31:36.545 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.920175+0000", 2024-09-14T06:31:36.545 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.065090234999999996 2024-09-14T06:31:36.545 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:36.546 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:36.546 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:31:36.546 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.920250+0000", 2024-09-14T06:31:36.546 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.5807000000000001e-05 2024-09-14T06:31:36.546 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:36.546 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:36.546 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:31:36.546 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.920410+0000", 2024-09-14T06:31:36.546 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00015963300000000001 2024-09-14T06:31:36.546 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:36.546 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:31:36.546 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:36.546 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:36.546 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:36.546 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:44521 3.2 3:4190d882:::rbd_data.112dca2f6ae1.0000000000000002:17 [read 86016~4096 [fadvise_nocache]] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:31:36.546 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:43.855087+0000", 2024-09-14T06:31:36.547 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 112.64195639099999, 2024-09-14T06:31:36.547 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.065563264999999996, 2024-09-14T06:31:36.547 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:31:36.547 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:31:36.547 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:31:36.547 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:31:36.547 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:31:36.547 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:36.547 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 44521 2024-09-14T06:31:36.547 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:36.547 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:31:36.547 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:36.547 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:31:36.547 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855087+0000", 2024-09-14T06:31:36.547 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:36.548 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:36.548 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:36.548 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:31:36.548 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855087+0000", 2024-09-14T06:31:36.548 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:36.548 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:36.548 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:36.548 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:31:36.548 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855088+0000", 2024-09-14T06:31:36.548 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.3699999999999999e-07 2024-09-14T06:31:36.548 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:36.548 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:36.548 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:31:36.548 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855089+0000", 2024-09-14T06:31:36.549 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.9900000000000009e-07 2024-09-14T06:31:36.549 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:36.549 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:36.549 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:31:36.549 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855091+0000", 2024-09-14T06:31:36.549 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.8940000000000002e-06 2024-09-14T06:31:36.549 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:36.549 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:36.549 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:31:36.549 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855094+0000", 2024-09-14T06:31:36.549 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.351e-06 2024-09-14T06:31:36.549 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:36.549 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:36.550 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:31:36.550 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.920437+0000", 2024-09-14T06:31:36.550 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.065343221000000007 2024-09-14T06:31:36.550 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:36.550 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:36.550 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:31:36.550 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.920481+0000", 2024-09-14T06:31:36.550 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.4431000000000002e-05 2024-09-14T06:31:36.550 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:36.550 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:36.550 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:31:36.550 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.920651+0000", 2024-09-14T06:31:36.550 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00016913200000000001 2024-09-14T06:31:36.550 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:36.550 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:31:36.551 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:36.551 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:36.551 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:36.551 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:44522 3.2 3:4190d882:::rbd_data.112dca2f6ae1.0000000000000002:17 [read 110592~4096 [fadvise_nocache]] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:31:36.551 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:43.855098+0000", 2024-09-14T06:31:36.551 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 112.641946164, 2024-09-14T06:31:36.551 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.065827628999999999, 2024-09-14T06:31:36.551 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:31:36.551 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:31:36.551 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:31:36.551 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:31:36.551 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:31:36.551 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:36.551 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 44522 2024-09-14T06:31:36.551 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:36.552 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:31:36.552 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:36.552 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:31:36.552 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855098+0000", 2024-09-14T06:31:36.552 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:36.552 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:36.552 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:36.552 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:31:36.552 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855098+0000", 2024-09-14T06:31:36.552 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:36.552 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:36.552 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:36.552 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:31:36.552 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855098+0000", 2024-09-14T06:31:36.552 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.1e-07 2024-09-14T06:31:36.553 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:36.553 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:36.553 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:31:36.553 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855099+0000", 2024-09-14T06:31:36.553 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.012e-06 2024-09-14T06:31:36.553 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:36.553 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:36.553 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:31:36.553 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855102+0000", 2024-09-14T06:31:36.553 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.9380000000000001e-06 2024-09-14T06:31:36.553 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:36.553 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:36.553 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:31:36.553 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855108+0000", 2024-09-14T06:31:36.553 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.4339999999999997e-06 2024-09-14T06:31:36.554 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:36.554 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:36.554 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:31:36.554 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.920681+0000", 2024-09-14T06:31:36.554 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.065572943999999994 2024-09-14T06:31:36.554 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:36.554 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:36.554 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:31:36.554 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.920724+0000", 2024-09-14T06:31:36.554 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.2877000000000003e-05 2024-09-14T06:31:36.554 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:36.554 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:36.554 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:31:36.554 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.920925+0000", 2024-09-14T06:31:36.554 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00020121399999999999 2024-09-14T06:31:36.554 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:36.555 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:31:36.555 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:36.555 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:36.555 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:36.555 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:44523 3.2 3:4190d882:::rbd_data.112dca2f6ae1.0000000000000002:17 [read 135168~4096 [fadvise_nocache]] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:31:36.555 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:43.855111+0000", 2024-09-14T06:31:36.555 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 112.641932442, 2024-09-14T06:31:36.555 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.066028198999999996, 2024-09-14T06:31:36.555 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:31:36.555 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:31:36.555 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:31:36.555 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:31:36.555 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:31:36.555 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:36.556 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 44523 2024-09-14T06:31:36.556 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:36.556 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:31:36.556 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:36.556 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:31:36.556 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855111+0000", 2024-09-14T06:31:36.556 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:36.556 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:36.556 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:36.556 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:31:36.556 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855111+0000", 2024-09-14T06:31:36.556 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:36.556 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:36.556 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:36.557 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:31:36.557 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855112+0000", 2024-09-14T06:31:36.557 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.1199999999999999e-07 2024-09-14T06:31:36.557 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:36.557 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:36.557 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:31:36.557 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855115+0000", 2024-09-14T06:31:36.557 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.3380000000000001e-06 2024-09-14T06:31:36.557 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:36.557 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:36.557 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:31:36.557 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855118+0000", 2024-09-14T06:31:36.557 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.366e-06 2024-09-14T06:31:36.557 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:36.558 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:36.558 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:31:36.558 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855121+0000", 2024-09-14T06:31:36.558 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.8329999999999998e-06 2024-09-14T06:31:36.558 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:36.558 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:36.558 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:31:36.558 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.920956+0000", 2024-09-14T06:31:36.558 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.065835251999999997 2024-09-14T06:31:36.558 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:36.558 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:36.558 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:31:36.558 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.921001+0000", 2024-09-14T06:31:36.558 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.4778999999999999e-05 2024-09-14T06:31:36.558 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:36.559 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:36.559 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:31:36.559 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.921139+0000", 2024-09-14T06:31:36.559 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000138419 2024-09-14T06:31:36.559 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:36.559 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:31:36.559 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:36.559 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:36.559 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:36.559 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:44524 3.2 3:4190d882:::rbd_data.112dca2f6ae1.0000000000000002:17 [read 159744~4096 [fadvise_nocache]] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:31:36.559 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:43.855124+0000", 2024-09-14T06:31:36.559 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 112.64191946299999, 2024-09-14T06:31:36.559 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.066249261000000004, 2024-09-14T06:31:36.559 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:31:36.559 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:31:36.560 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:31:36.560 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:31:36.560 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:31:36.560 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:36.560 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 44524 2024-09-14T06:31:36.560 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:36.560 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:31:36.560 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:36.560 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:31:36.560 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855124+0000", 2024-09-14T06:31:36.560 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:36.560 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:36.560 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:36.561 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:31:36.561 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855124+0000", 2024-09-14T06:31:36.561 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:36.561 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:36.561 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:36.561 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:31:36.561 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861086+0000", 2024-09-14T06:31:36.561 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0059613829999999998 2024-09-14T06:31:36.561 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:36.561 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:36.561 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:31:36.561 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861100+0000", 2024-09-14T06:31:36.561 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.4644000000000001e-05 2024-09-14T06:31:36.561 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:36.562 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:36.562 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:31:36.562 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861109+0000", 2024-09-14T06:31:36.562 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.541e-06 2024-09-14T06:31:36.562 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:36.562 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:36.562 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:31:36.562 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861117+0000", 2024-09-14T06:31:36.562 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.1049999999999995e-06 2024-09-14T06:31:36.562 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:36.562 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:36.562 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:31:36.562 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.921169+0000", 2024-09-14T06:31:36.562 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.060052405000000003 2024-09-14T06:31:36.562 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:36.562 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:36.563 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:31:36.563 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.921213+0000", 2024-09-14T06:31:36.563 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.3565e-05 2024-09-14T06:31:36.563 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:36.563 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:36.563 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:31:36.563 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.921374+0000", 2024-09-14T06:31:36.563 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00016061800000000001 2024-09-14T06:31:36.563 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:36.563 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:31:36.563 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:36.563 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:36.563 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:36.563 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:44536 3.2 3:4190d882:::rbd_data.112dca2f6ae1.0000000000000002:17 [read 454656~4096 [fadvise_nocache]] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:31:36.563 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:43.861384+0000", 2024-09-14T06:31:36.564 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 112.635659906, 2024-09-14T06:31:36.564 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.063058528000000003, 2024-09-14T06:31:36.564 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:31:36.564 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:31:36.564 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:31:36.564 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:31:36.564 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:31:36.564 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:36.564 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 44536 2024-09-14T06:31:36.564 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:36.564 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:31:36.564 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:36.564 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:31:36.565 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861384+0000", 2024-09-14T06:31:36.565 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:36.565 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:36.565 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:36.565 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:31:36.565 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861384+0000", 2024-09-14T06:31:36.565 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:36.565 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:36.565 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:36.565 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:31:36.565 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861384+0000", 2024-09-14T06:31:36.565 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.4000000000000002e-07 2024-09-14T06:31:36.565 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:36.565 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:36.566 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:31:36.566 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861386+0000", 2024-09-14T06:31:36.566 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.891e-06 2024-09-14T06:31:36.566 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:36.566 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:36.566 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:31:36.566 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861392+0000", 2024-09-14T06:31:36.566 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.9939999999999997e-06 2024-09-14T06:31:36.566 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:36.566 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:36.566 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:31:36.566 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861398+0000", 2024-09-14T06:31:36.566 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.507e-06 2024-09-14T06:31:36.566 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:36.566 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:36.567 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:31:36.567 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.924149+0000", 2024-09-14T06:31:36.567 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.062751135999999999 2024-09-14T06:31:36.567 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:36.567 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:36.567 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:31:36.567 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.924226+0000", 2024-09-14T06:31:36.567 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.7517999999999999e-05 2024-09-14T06:31:36.567 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:36.567 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:36.567 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:31:36.567 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.924442+0000", 2024-09-14T06:31:36.567 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000216042 2024-09-14T06:31:36.567 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:36.568 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:31:36.568 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:36.568 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:36.568 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:36.568 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:44537 3.2 3:4190d882:::rbd_data.112dca2f6ae1.0000000000000002:17 [read 479232~4096 [fadvise_nocache]] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:31:36.568 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:43.861405+0000", 2024-09-14T06:31:36.568 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 112.63563886, 2024-09-14T06:31:36.568 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.063351795000000002, 2024-09-14T06:31:36.568 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:31:36.568 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:31:36.568 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:31:36.568 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:31:36.568 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:31:36.568 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:36.569 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 44537 2024-09-14T06:31:36.569 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:36.569 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:31:36.569 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:36.569 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:31:36.569 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861405+0000", 2024-09-14T06:31:36.569 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:36.569 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:36.569 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:36.569 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:31:36.569 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861405+0000", 2024-09-14T06:31:36.569 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:36.569 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:36.569 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:36.569 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:31:36.570 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861405+0000", 2024-09-14T06:31:36.570 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.9099999999999999e-07 2024-09-14T06:31:36.570 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:36.570 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:36.570 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:31:36.570 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861407+0000", 2024-09-14T06:31:36.570 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.075e-06 2024-09-14T06:31:36.570 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:36.570 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:36.570 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:31:36.570 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861414+0000", 2024-09-14T06:31:36.570 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.4659999999999996e-06 2024-09-14T06:31:36.570 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:36.570 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:36.571 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:31:36.571 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861418+0000", 2024-09-14T06:31:36.571 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.5519999999999998e-06 2024-09-14T06:31:36.571 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:36.571 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:36.571 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:31:36.571 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.924474+0000", 2024-09-14T06:31:36.571 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.063056012999999994 2024-09-14T06:31:36.571 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:36.571 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:36.571 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:31:36.571 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.924556+0000", 2024-09-14T06:31:36.571 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.1181999999999996e-05 2024-09-14T06:31:36.571 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:36.572 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:36.572 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:31:36.572 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.924757+0000", 2024-09-14T06:31:36.572 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00020111599999999999 2024-09-14T06:31:36.572 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:36.572 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:31:36.572 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:36.572 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:36.572 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:36.572 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:44538 3.2 3:4190d882:::rbd_data.112dca2f6ae1.0000000000000002:17 [read 503808~4096 [fadvise_nocache]] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:31:36.572 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:43.861425+0000", 2024-09-14T06:31:36.572 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 112.635618819, 2024-09-14T06:31:36.572 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.063627125000000007, 2024-09-14T06:31:36.572 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:31:36.573 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:31:36.573 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:31:36.573 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:31:36.573 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:31:36.573 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:36.573 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 44538 2024-09-14T06:31:36.573 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:36.573 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:31:36.573 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:36.573 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:31:36.573 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861425+0000", 2024-09-14T06:31:36.573 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:36.573 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:36.573 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:36.574 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:31:36.574 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861425+0000", 2024-09-14T06:31:36.574 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:36.574 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:36.574 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:36.574 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:31:36.574 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861425+0000", 2024-09-14T06:31:36.574 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.9999999999999998e-07 2024-09-14T06:31:36.574 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:36.574 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:36.574 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:31:36.574 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861427+0000", 2024-09-14T06:31:36.574 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.9829999999999999e-06 2024-09-14T06:31:36.574 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:36.575 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:36.575 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:31:36.575 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861433+0000", 2024-09-14T06:31:36.575 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.2889999999999997e-06 2024-09-14T06:31:36.575 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:36.575 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:36.575 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:31:36.575 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861454+0000", 2024-09-14T06:31:36.575 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.1844999999999998e-05 2024-09-14T06:31:36.575 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:36.575 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:36.575 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:31:36.575 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.924786+0000", 2024-09-14T06:31:36.576 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.063331497 2024-09-14T06:31:36.576 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:36.576 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:36.576 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:31:36.576 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.924864+0000", 2024-09-14T06:31:36.576 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.7744000000000006e-05 2024-09-14T06:31:36.576 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:36.576 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:36.576 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:31:36.576 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.925052+0000", 2024-09-14T06:31:36.576 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00018836699999999999 2024-09-14T06:31:36.576 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:36.576 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:31:36.576 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:36.576 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:36.577 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:36.577 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:44539 3.2 3:4190d882:::rbd_data.112dca2f6ae1.0000000000000002:17 [read 528384~4096 [fadvise_nocache]] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:31:36.577 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:43.861462+0000", 2024-09-14T06:31:36.577 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 112.635581492, 2024-09-14T06:31:36.577 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.063855705999999998, 2024-09-14T06:31:36.577 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:31:36.577 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:31:36.577 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:31:36.577 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:31:36.577 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:31:36.577 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:36.577 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 44539 2024-09-14T06:31:36.577 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:36.578 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:31:36.578 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:36.578 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:31:36.578 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861462+0000", 2024-09-14T06:31:36.578 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:36.578 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:36.578 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:36.578 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:31:36.578 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861462+0000", 2024-09-14T06:31:36.578 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:36.578 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:36.578 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:36.578 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:31:36.578 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861463+0000", 2024-09-14T06:31:36.578 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.5200000000000002e-07 2024-09-14T06:31:36.578 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:36.579 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:36.579 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:31:36.579 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861465+0000", 2024-09-14T06:31:36.579 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.5189999999999999e-06 2024-09-14T06:31:36.579 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:36.579 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:36.579 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:31:36.579 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861471+0000", 2024-09-14T06:31:36.579 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.6899999999999997e-06 2024-09-14T06:31:36.579 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:36.579 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:36.579 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:31:36.579 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861475+0000", 2024-09-14T06:31:36.579 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.5730000000000001e-06 2024-09-14T06:31:36.579 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:36.580 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:36.580 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:31:36.580 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.925085+0000", 2024-09-14T06:31:36.580 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.063609902999999995 2024-09-14T06:31:36.580 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:36.580 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:36.580 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:31:36.580 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.925145+0000", 2024-09-14T06:31:36.580 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.9383999999999999e-05 2024-09-14T06:31:36.580 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:36.580 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:36.580 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:31:36.580 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.925318+0000", 2024-09-14T06:31:36.580 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00017318499999999999 2024-09-14T06:31:36.581 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:36.581 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:31:36.581 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:36.581 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:36.581 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:36.581 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:44540 3.2 3:4190d882:::rbd_data.112dca2f6ae1.0000000000000002:17 [read 552960~4096 [fadvise_nocache]] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:31:36.581 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:43.861483+0000", 2024-09-14T06:31:36.581 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 112.635561061, 2024-09-14T06:31:36.581 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.064065681999999999, 2024-09-14T06:31:36.581 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:31:36.581 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:31:36.581 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:31:36.581 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:31:36.581 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:31:36.581 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:36.581 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 44540 2024-09-14T06:31:36.582 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:36.582 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:31:36.582 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:36.582 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:31:36.582 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861483+0000", 2024-09-14T06:31:36.582 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:36.582 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:36.582 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:36.582 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:31:36.582 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861483+0000", 2024-09-14T06:31:36.582 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:36.582 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:36.582 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:36.582 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:31:36.582 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.867079+0000", 2024-09-14T06:31:36.582 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0055966480000000001 2024-09-14T06:31:36.583 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:36.583 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:36.583 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:31:36.583 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.867086+0000", 2024-09-14T06:31:36.583 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.387e-06 2024-09-14T06:31:36.583 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:36.583 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:36.583 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:31:36.583 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.867094+0000", 2024-09-14T06:31:36.583 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.9419999999999995e-06 2024-09-14T06:31:36.583 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:36.583 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:36.583 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:31:36.583 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.867109+0000", 2024-09-14T06:31:36.583 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.5818999999999999e-05 2024-09-14T06:31:36.583 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:36.583 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:36.584 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:31:36.584 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.925354+0000", 2024-09-14T06:31:36.584 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.058244047 2024-09-14T06:31:36.584 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:36.584 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:36.584 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:31:36.584 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.925406+0000", 2024-09-14T06:31:36.584 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.2661999999999998e-05 2024-09-14T06:31:36.584 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:36.584 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:36.584 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:31:36.584 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.925548+0000", 2024-09-14T06:31:36.584 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000142177 2024-09-14T06:31:36.584 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:36.584 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:31:36.585 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:36.585 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:36.585 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:31:36.585 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-14T06:31:36.586 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-14T06:31:36.586 INFO:teuthology.orchestra.run.smithi094.stdout:osd.0: {} 2024-09-14T06:31:36.587 INFO:teuthology.orchestra.run.smithi094.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-14T06:31:36.587 INFO:teuthology.orchestra.run.smithi094.stdout:osd.1: {} 2024-09-14T06:31:36.587 INFO:teuthology.orchestra.run.smithi094.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-14T06:31:36.588 INFO:tasks.ceph.osd.6.smithi138.stderr:2024-09-14T06:31:36.585+0000 7f8cd8849640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33620) UID: 0 2024-09-14T06:31:36.588 INFO:teuthology.orchestra.run.smithi094.stdout:osd.2: {} 2024-09-14T06:31:36.589 INFO:teuthology.orchestra.run.smithi094.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-14T06:31:36.599 INFO:teuthology.orchestra.run.smithi094.stderr:noscrub is set 2024-09-14T06:31:36.608 INFO:teuthology.orchestra.run.smithi094.stdout:5110 write 0xa501ec1 thru 0xa507918 (0x5a58 bytes) 2024-09-14T06:31:36.608 INFO:teuthology.orchestra.run.smithi094.stdout:osd.3: {} 2024-09-14T06:31:36.608 INFO:teuthology.orchestra.run.smithi094.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-14T06:31:36.625 INFO:teuthology.orchestra.run.smithi094.stdout:5111 trunc from 0xc37fffe to 0x67ea684 2024-09-14T06:31:36.634 INFO:teuthology.orchestra.run.smithi094.stdout:osd.4: {} 2024-09-14T06:31:36.634 INFO:teuthology.orchestra.run.smithi094.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-14T06:31:36.642 INFO:teuthology.orchestra.run.smithi094.stdout:5112 write 0x459aa19 thru 0x459b49f (0xa87 bytes) 2024-09-14T06:31:36.652 INFO:teuthology.orchestra.run.smithi094.stdout:osd.5: {} 2024-09-14T06:31:36.652 INFO:teuthology.orchestra.run.smithi094.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-14T06:31:36.652 INFO:teuthology.orchestra.run.smithi094.stdout:5113 trunc from 0x67ea684 to 0xbe8c60a 2024-09-14T06:31:36.659 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 4, 5, 6, 7] out_osds: [3] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-14T06:31:36.659 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-14T06:31:36.659 INFO:tasks.thrashosds.thrasher:Reweighting osd 2 to 0.48856605991993585 2024-09-14T06:31:36.659 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 2 0.48856605991993585 2024-09-14T06:31:36.660 INFO:teuthology.orchestra.run.smithi094.stdout:5117 write 0x2d0961b thru 0x2d0b8c9 (0x22af bytes) 2024-09-14T06:31:36.662 INFO:teuthology.orchestra.run.smithi094.stdout:5118 write 0x9d9b14a thru 0x9d9c26c (0x1123 bytes) 2024-09-14T06:31:36.668 INFO:teuthology.orchestra.run.smithi094.stdout:5119 write 0x1d437c6 thru 0x1d4bc2d (0x8468 bytes) 2024-09-14T06:31:36.670 INFO:teuthology.orchestra.run.smithi094.stdout:osd.6: {} 2024-09-14T06:31:36.670 INFO:teuthology.orchestra.run.smithi094.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-14T06:31:36.676 INFO:teuthology.orchestra.run.smithi094.stdout:5120 write 0x2a7ce00 thru 0x2a881be (0xb3bf bytes) 2024-09-14T06:31:36.686 INFO:tasks.ceph.osd.4.smithi138.stderr:2024-09-14T06:31:36.685+0000 7f6c653c8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33615) UID: 0 2024-09-14T06:31:36.691 INFO:teuthology.orchestra.run.smithi094.stdout:osd.7: {} 2024-09-14T06:31:36.692 INFO:teuthology.orchestra.run.smithi094.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-14T06:31:36.700 INFO:teuthology.orchestra.run.smithi094.stdout:5121 trunc from 0xbe8c60a to 0x30d74f1 2024-09-14T06:31:36.710 INFO:teuthology.orchestra.run.smithi094.stdout:5122 read 0x5178c0 thru 0x519f7b (0x26bc bytes) 2024-09-14T06:31:36.711 INFO:teuthology.orchestra.run.smithi094.stdout:5123 read 0x13c720f thru 0x13d58f0 (0xe6e2 bytes) 2024-09-14T06:31:36.718 INFO:teuthology.orchestra.run.smithi138.stdout:ERROR: (22) Invalid argument 2024-09-14T06:31:36.718 INFO:teuthology.orchestra.run.smithi138.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:31:36.719 INFO:teuthology.orchestra.run.smithi094.stdout:5125 write 0x47c71c2 thru 0x47d713d (0xff7c bytes) 2024-09-14T06:31:36.729 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-14T06:31:36.733 INFO:teuthology.orchestra.run.smithi094.stdout:5126 read 0x10fe6b3 thru 0x110302e (0x497c bytes) 2024-09-14T06:31:36.745 INFO:teuthology.orchestra.run.smithi094.stdout:5127 write 0xa096c66 thru 0xa09f843 (0x8bde bytes) 2024-09-14T06:31:36.753 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-14T06:31:36.753 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-14T06:31:36.764 INFO:teuthology.orchestra.run.smithi094.stdout:5130 write 0xe4b3b60 thru 0xe4b5f85 (0x2426 bytes) 2024-09-14T06:31:36.769 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-14T06:31:36.769 INFO:teuthology.orchestra.run.smithi094.stdout:5133 write 0xa1024ae thru 0xa10f9e7 (0xd53a bytes) 2024-09-14T06:31:36.781 INFO:teuthology.orchestra.run.smithi094.stdout:5134 trunc from 0xe4b5f86 to 0xbb4564b 2024-09-14T06:31:36.787 INFO:tasks.ceph.osd.1.smithi094.stderr:2024-09-14T06:31:36.785+0000 7f328e47a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33641) UID: 0 2024-09-14T06:31:36.792 INFO:teuthology.orchestra.run.smithi094.stdout:5136 write 0x6df01a2 thru 0x6df3caa (0x3b09 bytes) 2024-09-14T06:31:36.805 INFO:teuthology.orchestra.run.smithi094.stdout:5139 write 0xda4ed69 thru 0xda5ae67 (0xc0ff bytes) 2024-09-14T06:31:36.817 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-14T06:31:36.817 INFO:teuthology.orchestra.run.smithi094.stdout:5144 punch from 0x5cdde6d to 0x5ceb45e, (0xd5f1 bytes) 2024-09-14T06:31:36.817 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-14T06:31:36.817 INFO:teuthology.orchestra.run.smithi094.stdout:5147 write 0x39907dd thru 0x399b184 (0xa9a8 bytes) 2024-09-14T06:31:36.827 INFO:teuthology.orchestra.run.smithi094.stdout:5148 write 0xc856075 thru 0xc865bb6 (0xfb42 bytes) 2024-09-14T06:31:36.841 INFO:teuthology.orchestra.run.smithi094.stdout:5150 read 0x847be85 thru 0x847d371 (0x14ed bytes) 2024-09-14T06:31:36.841 INFO:teuthology.orchestra.run.smithi094.stdout:5151 read 0xb055de9 thru 0xb0606cd (0xa8e5 bytes) 2024-09-14T06:31:36.841 INFO:teuthology.orchestra.run.smithi094.stdout:5152 read 0xbd2ff34 thru 0xbd339aa (0x3a77 bytes) 2024-09-14T06:31:36.841 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-14T06:31:36.841 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-14T06:31:36.841 INFO:teuthology.orchestra.run.smithi094.stdout:5155 write 0x5c67852 thru 0x5c73f7d (0xc72c bytes) 2024-09-14T06:31:36.854 INFO:teuthology.orchestra.run.smithi094.stdout:5156 trunc from 0xda5ae68 to 0xc6d83c0 2024-09-14T06:31:36.872 INFO:teuthology.orchestra.run.smithi094.stdout:5157 trunc from 0xc6d83c0 to 0xee131be 2024-09-14T06:31:36.877 INFO:teuthology.orchestra.run.smithi094.stdout:5159 write 0x8262d09 thru 0x826a001 (0x72f9 bytes) 2024-09-14T06:31:36.885 INFO:teuthology.orchestra.run.smithi094.stdout:5160 punch from 0x30376a4 to 0x303c992, (0x52ee bytes) 2024-09-14T06:31:36.888 INFO:tasks.ceph.osd.0.smithi094.stderr:2024-09-14T06:31:36.886+0000 7f07c9333640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33636) UID: 0 2024-09-14T06:31:36.891 INFO:teuthology.orchestra.run.smithi138.stdout:ERROR: (22) Invalid argument 2024-09-14T06:31:36.891 INFO:teuthology.orchestra.run.smithi138.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:31:36.895 INFO:teuthology.orchestra.run.smithi094.stdout:5161 trunc from 0xee131be to 0x69e1081 2024-09-14T06:31:36.902 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-14T06:31:36.916 INFO:teuthology.orchestra.run.smithi094.stdout:5163 read 0x399e968 thru 0x39ac2bd (0xd956 bytes) 2024-09-14T06:31:36.916 INFO:teuthology.orchestra.run.smithi094.stdout:5164 read 0x49a4d71 thru 0x49ab8dc (0x6b6c bytes) 2024-09-14T06:31:36.917 INFO:teuthology.orchestra.run.smithi094.stdout:5166 read 0x390564f thru 0x390c16b (0x6b1d bytes) 2024-09-14T06:31:36.926 INFO:teuthology.orchestra.run.smithi094.stdout:5167 write 0x98b2ea1 thru 0x98b6450 (0x35b0 bytes) 2024-09-14T06:31:36.931 INFO:teuthology.orchestra.run.smithi094.stdout:5168 writesame 0xcd8695 thru 0xcdad52 data_size 0x1d(0x26be bytes) 2024-09-14T06:31:36.936 INFO:teuthology.orchestra.run.smithi094.stdout:5169 punch from 0x35e0da3 to 0x35e41b5, (0x3412 bytes) 2024-09-14T06:31:36.936 INFO:teuthology.orchestra.run.smithi094.stdout:5170 read 0x2121418 thru 0x2126ac6 (0x56af bytes) 2024-09-14T06:31:36.936 INFO:teuthology.orchestra.run.smithi094.stdout:5171 punch from 0x7e2d8e1 to 0x7e304e1, (0x2c00 bytes) 2024-09-14T06:31:36.936 INFO:teuthology.orchestra.run.smithi094.stdout:5175 read 0x92df646 thru 0x92e3dd2 (0x478d bytes) 2024-09-14T06:31:36.936 INFO:teuthology.orchestra.run.smithi094.stdout:5176 read 0x2d6a78a thru 0x2d7718d (0xca04 bytes) 2024-09-14T06:31:36.947 INFO:teuthology.orchestra.run.smithi094.stdout:5177 writesame 0xdcd3677 thru 0xdcde2b2 data_size 0x4(0xac3c bytes) 2024-09-14T06:31:36.947 INFO:teuthology.orchestra.run.smithi094.stdout:5178 read 0x6710d4a thru 0x671593c (0x4bf3 bytes) 2024-09-14T06:31:36.947 INFO:teuthology.orchestra.run.smithi094.stdout:5180 punch from 0x80dbd3a to 0x80dc187, (0x44d bytes) 2024-09-14T06:31:36.947 INFO:teuthology.orchestra.run.smithi094.stdout:5181 read 0x5953938 thru 0x5954cd6 (0x139f bytes) 2024-09-14T06:31:36.947 INFO:teuthology.orchestra.run.smithi094.stdout:5183 write 0x290667d thru 0x2908ef3 (0x2877 bytes) 2024-09-14T06:31:36.951 INFO:teuthology.orchestra.run.smithi094.stdout:5184 write 0xa1cce51 thru 0xa1d6a33 (0x9be3 bytes) 2024-09-14T06:31:36.961 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-14T06:31:36.975 INFO:teuthology.orchestra.run.smithi094.stdout:5188 trunc from 0xdcde2b3 to 0x1f0e95c 2024-09-14T06:31:36.989 INFO:tasks.ceph.osd.0.smithi094.stderr:2024-09-14T06:31:36.987+0000 7f07c9333640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33636) UID: 0 2024-09-14T06:31:37.003 INFO:teuthology.orchestra.run.smithi094.stdout:5190 punch from 0x51806 to 0x5ff3b, (0xe735 bytes) 2024-09-14T06:31:37.021 INFO:teuthology.orchestra.run.smithi138.stdout:ERROR: (22) Invalid argument 2024-09-14T06:31:37.021 INFO:teuthology.orchestra.run.smithi138.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:31:37.021 INFO:teuthology.orchestra.run.smithi094.stdout:5192 writesame 0xa38be45 thru 0xa391559 data_size 0x3(0x5715 bytes) 2024-09-14T06:31:37.021 INFO:teuthology.orchestra.run.smithi094.stdout:5194 write 0x5623791 thru 0x5630af2 (0xd362 bytes) 2024-09-14T06:31:37.029 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-14T06:31:37.035 INFO:teuthology.orchestra.run.smithi094.stdout:5197 write 0x63c3a5a thru 0x63c5736 (0x1cdd bytes) 2024-09-14T06:31:37.040 INFO:teuthology.orchestra.run.smithi094.stdout:5198 writesame 0x54f31a thru 0x55a49c data_size 0x1(0xb183 bytes) 2024-09-14T06:31:37.059 INFO:teuthology.orchestra.run.smithi094.stdout:5200 writesame 0xb1e3c55 thru 0xb1e6a54 data_size 0x20(0x2e00 bytes) 2024-09-14T06:31:37.059 INFO:teuthology.orchestra.run.smithi094.stdout:5201 writesame 0x7645b88 thru 0x76489fa data_size 0x17(0x2e73 bytes) 2024-09-14T06:31:37.067 INFO:teuthology.orchestra.run.smithi094.stdout:5202 punch from 0x933e55d to 0x934a297, (0xbd3a bytes) 2024-09-14T06:31:37.073 INFO:teuthology.orchestra.run.smithi094.stdout:5204 write 0xb6e77d4 thru 0xb6e8836 (0x1063 bytes) 2024-09-14T06:31:37.077 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-14T06:31:37.080 INFO:teuthology.orchestra.run.smithi094.stdout:5206 trunc from 0xb6e8837 to 0xdb54a70 2024-09-14T06:31:37.086 INFO:teuthology.orchestra.run.smithi094.stdout:5208 read 0x604ecfd thru 0x604fc82 (0xf86 bytes) 2024-09-14T06:31:37.086 INFO:teuthology.orchestra.run.smithi094.stdout:5209 write 0x8fde6e4 thru 0x8fe149c (0x2db9 bytes) 2024-09-14T06:31:37.090 INFO:tasks.ceph.osd.2.smithi094.stderr:2024-09-14T06:31:37.088+0000 7faf3269b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33649) UID: 0 2024-09-14T06:31:37.096 INFO:teuthology.orchestra.run.smithi094.stdout:5210 write 0xe8fef2d thru 0xe904b1d (0x5bf1 bytes) 2024-09-14T06:31:37.104 INFO:teuthology.orchestra.run.smithi094.stdout:5211 write 0x278f88c thru 0x279ee45 (0xf5ba bytes) 2024-09-14T06:31:37.135 INFO:teuthology.orchestra.run.smithi094.stdout:5212 trunc from 0xe904b1e to 0x210af36 2024-09-14T06:31:37.150 INFO:teuthology.orchestra.run.smithi138.stdout:ERROR: (22) Invalid argument 2024-09-14T06:31:37.150 INFO:teuthology.orchestra.run.smithi138.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:31:37.157 INFO:teuthology.orchestra.run.smithi094.stdout:5213 punch from 0xd68d9a to 0xd6fd61, (0x6fc7 bytes) 2024-09-14T06:31:37.157 INFO:teuthology.orchestra.run.smithi094.stdout:5214 read 0x55f1ce thru 0x564ce9 (0x5b1c bytes) 2024-09-14T06:31:37.158 INFO:teuthology.orchestra.run.smithi094.stdout:5215 trunc from 0x210af36 to 0x27de3da 2024-09-14T06:31:37.159 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-14T06:31:37.165 INFO:teuthology.orchestra.run.smithi094.stdout:5216 punch from 0x1ffa4d1 to 0x20086ab, (0xe1da bytes) 2024-09-14T06:31:37.180 INFO:teuthology.orchestra.run.smithi094.stdout:5218 write 0x9a70f97 thru 0x9a77746 (0x67b0 bytes) 2024-09-14T06:31:37.190 INFO:teuthology.orchestra.run.smithi094.stdout:5219 trunc from 0x9a77747 to 0xb52b3fa 2024-09-14T06:31:37.196 INFO:teuthology.orchestra.run.smithi094.stdout:5220 writesame 0xa4a45dd thru 0xa4b4307 data_size 0x1(0xfd2b bytes) 2024-09-14T06:31:37.203 INFO:tasks.ceph.osd.7.smithi138.stderr:2024-09-14T06:31:37.202+0000 7f3c6a21d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33623) UID: 0 2024-09-14T06:31:37.214 INFO:teuthology.orchestra.run.smithi094.stdout:5221 write 0x5823af2 thru 0x582a272 (0x6781 bytes) 2024-09-14T06:31:37.227 INFO:teuthology.orchestra.run.smithi094.stdout:5222 read 0x9cd2d62 thru 0x9cdae32 (0x80d1 bytes) 2024-09-14T06:31:37.233 INFO:teuthology.orchestra.run.smithi094.stdout:5223 write 0xb9b4aa4 thru 0xb9bca67 (0x7fc4 bytes) 2024-09-14T06:31:37.243 INFO:teuthology.orchestra.run.smithi094.stdout:5224 write 0x46af93b thru 0x46b0ba8 (0x126e bytes) 2024-09-14T06:31:37.256 INFO:teuthology.orchestra.run.smithi094.stdout:5225 trunc from 0xb9bca68 to 0x56fe143 2024-09-14T06:31:37.277 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-14T06:31:37.277 INFO:teuthology.orchestra.run.smithi094.stdout:5227 writesame 0x3ba3132 thru 0x3bb041b data_size 0x3(0xd2ea bytes) 2024-09-14T06:31:37.278 INFO:teuthology.orchestra.run.smithi138.stdout:ERROR: (22) Invalid argument 2024-09-14T06:31:37.278 INFO:teuthology.orchestra.run.smithi138.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:31:37.290 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-14T06:31:37.292 INFO:tasks.ceph.osd.1.smithi094.stderr:2024-09-14T06:31:37.290+0000 7f328e47a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33641) UID: 0 2024-09-14T06:31:37.296 INFO:teuthology.orchestra.run.smithi094.stdout:5229 trunc from 0x56fe143 to 0x110b5d4 2024-09-14T06:31:37.317 INFO:teuthology.orchestra.run.smithi094.stdout:5231 write 0x2eca1d0 thru 0x2eccae2 (0x2913 bytes) 2024-09-14T06:31:37.321 INFO:teuthology.orchestra.run.smithi094.stdout:5234 read 0x2498a20 thru 0x249a83b (0x1e1c bytes) 2024-09-14T06:31:37.324 INFO:teuthology.orchestra.run.smithi094.stdout:5235 trunc from 0x2eccae3 to 0x522bc2b 2024-09-14T06:31:37.329 INFO:teuthology.orchestra.run.smithi094.stdout:5236 read 0x1f4dff thru 0x1fa542 (0x5744 bytes) 2024-09-14T06:31:37.330 INFO:teuthology.orchestra.run.smithi094.stdout:5237 punch from 0x15800ad to 0x158cdb2, (0xcd05 bytes) 2024-09-14T06:31:37.332 INFO:teuthology.orchestra.run.smithi094.stdout:5238 trunc from 0x522bc2b to 0x4102f12 2024-09-14T06:31:37.338 INFO:teuthology.orchestra.run.smithi094.stdout:5239 read 0x335fe6d thru 0x3363635 (0x37c9 bytes) 2024-09-14T06:31:37.338 INFO:teuthology.orchestra.run.smithi094.stdout:5241 read 0x3273492 thru 0x327cc4b (0x97ba bytes) 2024-09-14T06:31:37.338 INFO:teuthology.orchestra.run.smithi094.stdout:5242 read 0x10d50b2 thru 0x10dd276 (0x81c5 bytes) 2024-09-14T06:31:37.338 INFO:teuthology.orchestra.run.smithi094.stdout:5244 punch from 0x5c5c2a to 0x5d1155, (0xb52b bytes) 2024-09-14T06:31:37.346 INFO:teuthology.orchestra.run.smithi094.stdout:5247 write 0x6566f20 thru 0x6570a91 (0x9b72 bytes) 2024-09-14T06:31:37.358 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-14T06:31:37.369 INFO:teuthology.orchestra.run.smithi094.stdout:5249 write 0xaf4881a thru 0xaf499f6 (0x11dd bytes) 2024-09-14T06:31:37.372 INFO:teuthology.orchestra.run.smithi094.stdout:5251 write 0x21c91d7 thru 0x21d7067 (0xde91 bytes) 2024-09-14T06:31:37.387 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-14T06:31:37.387 INFO:teuthology.orchestra.run.smithi094.stdout:5253 read 0x7cf1b5a thru 0x7d01949 (0xfdf0 bytes) 2024-09-14T06:31:37.387 INFO:teuthology.orchestra.run.smithi094.stdout:5254 read 0x3d326ea thru 0x3d34706 (0x201d bytes) 2024-09-14T06:31:37.392 INFO:tasks.ceph.osd.3.smithi094.stderr:2024-09-14T06:31:37.390+0000 7feeeafe8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33655) UID: 0 2024-09-14T06:31:37.399 INFO:teuthology.orchestra.run.smithi094.stdout:5255 trunc from 0xaf499f7 to 0x12ab95f 2024-09-14T06:31:37.416 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-14T06:31:37.424 INFO:teuthology.orchestra.run.smithi094.stdout:5257 write 0x2c8b7b3 thru 0x2c924d8 (0x6d26 bytes) 2024-09-14T06:31:37.433 INFO:teuthology.orchestra.run.smithi094.stdout:5258 read 0xa9cd29 thru 0xaa69fc (0x9cd4 bytes) 2024-09-14T06:31:37.441 INFO:teuthology.orchestra.run.smithi094.stdout:5259 write 0x4b96e8b thru 0x4ba56da (0xe850 bytes) 2024-09-14T06:31:37.452 INFO:teuthology.orchestra.run.smithi138.stdout:ERROR: (22) Invalid argument 2024-09-14T06:31:37.452 INFO:teuthology.orchestra.run.smithi138.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:31:37.454 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-14T06:31:37.461 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-14T06:31:37.462 INFO:teuthology.orchestra.run.smithi094.stdout:5261 trunc from 0x4ba56db to 0xa527a5c 2024-09-14T06:31:37.475 INFO:teuthology.orchestra.run.smithi094.stdout:5262 write 0xc5d2acf thru 0xc5da3f2 (0x7924 bytes) 2024-09-14T06:31:37.485 INFO:teuthology.orchestra.run.smithi094.stdout:5264 read 0x88e86df thru 0x88ea6d4 (0x1ff6 bytes) 2024-09-14T06:31:37.486 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-14T06:31:37.486 INFO:teuthology.orchestra.run.smithi094.stdout:5267 read 0x17f0c22 thru 0x17f7bce (0x6fad bytes) 2024-09-14T06:31:37.486 INFO:teuthology.orchestra.run.smithi094.stdout:5268 write 0x287faa1 thru 0x2889e87 (0xa3e7 bytes) 2024-09-14T06:31:37.500 INFO:teuthology.orchestra.run.smithi094.stdout:5269 read 0x87ee0e9 thru 0x87f88c8 (0xa7e0 bytes) 2024-09-14T06:31:37.500 INFO:teuthology.orchestra.run.smithi094.stdout:5271 punch from 0xb6c24e3 to 0xb6cef84, (0xcaa1 bytes) 2024-09-14T06:31:37.500 INFO:teuthology.orchestra.run.smithi094.stdout:5272 write 0x6a77032 thru 0x6a7d8df (0x68ae bytes) 2024-09-14T06:31:37.505 INFO:tasks.ceph.osd.6.smithi138.stderr:2024-09-14T06:31:37.504+0000 7f8cd8849640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33620) UID: 0 2024-09-14T06:31:37.508 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-14T06:31:37.508 INFO:teuthology.orchestra.run.smithi094.stdout:5276 writesame 0xa0c76bd thru 0xa0cb50f data_size 0x1(0x3e53 bytes) 2024-09-14T06:31:37.518 INFO:teuthology.orchestra.run.smithi094.stdout:5278 write 0x69aa453 thru 0x69b6ba4 (0xc752 bytes) 2024-09-14T06:31:37.527 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-14T06:31:37.527 INFO:teuthology.orchestra.run.smithi094.stdout:5281 read 0x3ef12d1 thru 0x3efb6ae (0xa3de bytes) 2024-09-14T06:31:37.527 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-14T06:31:37.532 INFO:teuthology.orchestra.run.smithi094.stdout:5283 trunc from 0xc5da3f3 to 0x7eeef81 2024-09-14T06:31:37.544 INFO:teuthology.orchestra.run.smithi094.stdout:5284 punch from 0x1ab278 to 0x1ad959, (0x26e1 bytes) 2024-09-14T06:31:37.544 INFO:teuthology.orchestra.run.smithi094.stdout:5285 writesame 0x4bc4a62 thru 0x4bcac90 data_size 0xb(0x622f bytes) 2024-09-14T06:31:37.561 INFO:teuthology.orchestra.run.smithi094.stdout:5286 write 0xe8ed787 thru 0xe8f675d (0x8fd7 bytes) 2024-09-14T06:31:37.570 INFO:teuthology.orchestra.run.smithi094.stdout:5288 write 0x4c34c5 thru 0x4cd655 (0xa191 bytes) 2024-09-14T06:31:37.576 INFO:teuthology.orchestra.run.smithi094.stdout:5290 trunc from 0xe8f675e to 0xd551c5c 2024-09-14T06:31:37.582 INFO:teuthology.orchestra.run.smithi138.stdout:ERROR: (22) Invalid argument 2024-09-14T06:31:37.582 INFO:teuthology.orchestra.run.smithi138.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:31:37.589 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-14T06:31:37.589 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-14T06:31:37.589 INFO:teuthology.orchestra.run.smithi094.stdout:5294 punch from 0x2a666be to 0x2a70650, (0x9f92 bytes) 2024-09-14T06:31:37.590 INFO:teuthology.orchestra.run.smithi094.stdout:5295 punch from 0x28d151b to 0x28d35d3, (0x20b8 bytes) 2024-09-14T06:31:37.590 INFO:teuthology.orchestra.run.smithi094.stdout:5296 read 0xb4b3bf0 thru 0xb4bce85 (0x9296 bytes) 2024-09-14T06:31:37.590 INFO:teuthology.orchestra.run.smithi094.stdout:5299 write 0x7c2e392 thru 0x7c352dd (0x6f4c bytes) 2024-09-14T06:31:37.593 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-14T06:31:37.594 INFO:tasks.ceph.osd.1.smithi094.stderr:2024-09-14T06:31:37.592+0000 7f328e47a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33641) UID: 0 2024-09-14T06:31:37.607 INFO:teuthology.orchestra.run.smithi094.stdout:5300 trunc from 0xd551c5c to 0x63be5a7 2024-09-14T06:31:37.625 INFO:teuthology.orchestra.run.smithi094.stdout:5301 write 0x3ed5052 thru 0x3ee0e52 (0xbe01 bytes) 2024-09-14T06:31:37.640 INFO:teuthology.orchestra.run.smithi094.stdout:5303 read 0xe589b thru 0xe8ec6 (0x362c bytes) 2024-09-14T06:31:37.647 INFO:teuthology.orchestra.run.smithi094.stdout:5304 trunc from 0x63be5a7 to 0xc5166f4 2024-09-14T06:31:37.653 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-14T06:31:37.653 INFO:teuthology.orchestra.run.smithi094.stdout:5306 write 0x2de79f5 thru 0x2dea1f7 (0x2803 bytes) 2024-09-14T06:31:37.661 INFO:teuthology.orchestra.run.smithi094.stdout:5307 read 0x564d9d2 thru 0x5653731 (0x5d60 bytes) 2024-09-14T06:31:37.672 INFO:teuthology.orchestra.run.smithi094.stderr:reweighted osd.2 to 0.488566 (7d12) 2024-09-14T06:31:37.673 INFO:teuthology.orchestra.run.smithi094.stdout:5309 trunc from 0xc5166f4 to 0x2bb483d 2024-09-14T06:31:37.694 INFO:tasks.ceph.osd.6.smithi138.stderr:2024-09-14T06:31:37.693+0000 7f8cd8849640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33620) UID: 0 2024-09-14T06:31:37.742 INFO:teuthology.orchestra.run.smithi138.stdout:ERROR: (22) Invalid argument 2024-09-14T06:31:37.742 INFO:teuthology.orchestra.run.smithi138.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:31:37.751 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-14T06:31:37.795 INFO:tasks.ceph.osd.0.smithi094.stderr:2024-09-14T06:31:37.794+0000 7f07c9333640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33636) UID: 0 2024-09-14T06:31:37.894 INFO:teuthology.orchestra.run.smithi138.stdout:ERROR: (22) Invalid argument 2024-09-14T06:31:37.894 INFO:teuthology.orchestra.run.smithi138.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:31:37.897 INFO:tasks.ceph.osd.3.smithi094.stderr:2024-09-14T06:31:37.895+0000 7feeeafe8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33655) UID: 0 2024-09-14T06:31:37.903 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-14T06:31:37.998 INFO:tasks.ceph.osd.0.smithi094.stderr:2024-09-14T06:31:37.996+0000 7f07c9333640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33636) UID: 0 2024-09-14T06:31:38.005 INFO:teuthology.orchestra.run.smithi138.stdout:ERROR: (22) Invalid argument 2024-09-14T06:31:38.005 INFO:teuthology.orchestra.run.smithi138.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:31:38.014 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-14T06:31:38.098 INFO:tasks.ceph.osd.4.smithi138.stderr:2024-09-14T06:31:38.097+0000 7f6c653c8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33615) UID: 0 2024-09-14T06:31:38.116 INFO:teuthology.orchestra.run.smithi138.stdout:ERROR: (22) Invalid argument 2024-09-14T06:31:38.117 INFO:teuthology.orchestra.run.smithi138.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:31:38.125 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-14T06:31:38.199 INFO:tasks.ceph.osd.2.smithi094.stderr:2024-09-14T06:31:38.197+0000 7faf3269b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33649) UID: 0 2024-09-14T06:31:38.267 INFO:teuthology.orchestra.run.smithi138.stdout:ERROR: (22) Invalid argument 2024-09-14T06:31:38.268 INFO:teuthology.orchestra.run.smithi138.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:31:38.276 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-14T06:31:38.300 INFO:tasks.ceph.osd.7.smithi138.stderr:2024-09-14T06:31:38.299+0000 7f3c6a21d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33623) UID: 0 2024-09-14T06:31:38.385 INFO:teuthology.orchestra.run.smithi094.stdout:ERROR: (22) Invalid argument 2024-09-14T06:31:38.386 INFO:teuthology.orchestra.run.smithi094.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:31:38.394 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-14T06:31:38.401 INFO:tasks.ceph.osd.5.smithi138.stderr:2024-09-14T06:31:38.400+0000 7fe13746f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33618) UID: 0 2024-09-14T06:31:38.496 INFO:teuthology.orchestra.run.smithi094.stdout:ERROR: (22) Invalid argument 2024-09-14T06:31:38.497 INFO:teuthology.orchestra.run.smithi094.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:31:38.502 INFO:tasks.ceph.osd.4.smithi138.stderr:2024-09-14T06:31:38.501+0000 7f6c653c8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33615) UID: 0 2024-09-14T06:31:38.505 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-14T06:31:38.603 INFO:tasks.ceph.osd.4.smithi138.stderr:2024-09-14T06:31:38.602+0000 7f6c653c8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33615) UID: 0 2024-09-14T06:31:38.607 INFO:teuthology.orchestra.run.smithi094.stdout:ERROR: (22) Invalid argument 2024-09-14T06:31:38.607 INFO:teuthology.orchestra.run.smithi094.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:31:38.616 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-14T06:31:38.619 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-14T06:31:38.705 INFO:tasks.ceph.osd.3.smithi094.stderr:2024-09-14T06:31:38.703+0000 7feeeafe8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33655) UID: 0 2024-09-14T06:31:38.707 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-14T06:31:38.725 INFO:teuthology.orchestra.run.smithi094.stdout:ERROR: (22) Invalid argument 2024-09-14T06:31:38.725 INFO:teuthology.orchestra.run.smithi094.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:31:38.735 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-14T06:31:38.820 INFO:tasks.ceph.osd.0.smithi094.stderr:2024-09-14T06:31:38.818+0000 7f07c9333640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33636) UID: 0 2024-09-14T06:31:38.833 INFO:teuthology.orchestra.run.smithi094.stdout:osd.0: {} 2024-09-14T06:31:38.833 INFO:teuthology.orchestra.run.smithi094.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-14T06:31:38.851 INFO:teuthology.orchestra.run.smithi094.stdout:osd.1: {} 2024-09-14T06:31:38.852 INFO:teuthology.orchestra.run.smithi094.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-14T06:31:38.869 INFO:teuthology.orchestra.run.smithi094.stdout:osd.2: {} 2024-09-14T06:31:38.870 INFO:teuthology.orchestra.run.smithi094.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-14T06:31:38.886 INFO:teuthology.orchestra.run.smithi094.stdout:osd.3: {} 2024-09-14T06:31:38.886 INFO:teuthology.orchestra.run.smithi094.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-14T06:31:38.886 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-14T06:31:38.886 INFO:teuthology.orchestra.run.smithi094.stdout: "ops": [], 2024-09-14T06:31:38.886 INFO:teuthology.orchestra.run.smithi094.stdout: "complaint_time": 30, 2024-09-14T06:31:38.886 INFO:teuthology.orchestra.run.smithi094.stdout: "num_blocked_ops": 0 2024-09-14T06:31:38.886 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-14T06:31:38.895 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-14T06:31:38.903 INFO:teuthology.orchestra.run.smithi094.stdout:osd.4: {} 2024-09-14T06:31:38.904 INFO:teuthology.orchestra.run.smithi094.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-14T06:31:38.906 INFO:tasks.ceph.osd.5.smithi138.stderr:2024-09-14T06:31:38.905+0000 7fe13746f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33618) UID: 0 2024-09-14T06:31:38.922 INFO:teuthology.orchestra.run.smithi094.stdout:osd.5: {} 2024-09-14T06:31:38.922 INFO:teuthology.orchestra.run.smithi094.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-14T06:31:38.939 INFO:teuthology.orchestra.run.smithi094.stdout:osd.6: {} 2024-09-14T06:31:38.939 INFO:teuthology.orchestra.run.smithi094.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-14T06:31:38.958 INFO:teuthology.orchestra.run.smithi094.stdout:osd.7: {} 2024-09-14T06:31:38.958 INFO:teuthology.orchestra.run.smithi094.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-14T06:31:39.009 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-14T06:31:39.010 INFO:teuthology.orchestra.run.smithi094.stdout: "size": 20, 2024-09-14T06:31:39.010 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 600, 2024-09-14T06:31:39.010 INFO:teuthology.orchestra.run.smithi094.stdout: "ops": [ 2024-09-14T06:31:39.010 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:39.010 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:14918 3.0 3:05c39535:::rbd_data.112d92027590.0000000000000009:f [read 795648~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:31:39.010 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:26.083834+0000", 2024-09-14T06:31:39.010 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 132.92402347399999, 2024-09-14T06:31:39.010 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.041776287000000002, 2024-09-14T06:31:39.010 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:31:39.010 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:31:39.010 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:31:39.011 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:31:39.011 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:31:39.011 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:39.011 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 14918 2024-09-14T06:31:39.011 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:39.011 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:31:39.011 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:39.011 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:31:39.011 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.083834+0000", 2024-09-14T06:31:39.011 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:39.011 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:39.011 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:39.011 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:31:39.012 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.083834+0000", 2024-09-14T06:31:39.012 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:39.012 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:39.012 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:39.012 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:31:39.012 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.083834+0000", 2024-09-14T06:31:39.012 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.0100000000000001e-07 2024-09-14T06:31:39.012 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:39.012 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:39.012 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:31:39.012 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.083835+0000", 2024-09-14T06:31:39.012 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.3900000000000003e-07 2024-09-14T06:31:39.012 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:39.013 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:39.013 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:31:39.013 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.083838+0000", 2024-09-14T06:31:39.013 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.9830000000000001e-06 2024-09-14T06:31:39.013 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:39.013 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:39.013 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:31:39.013 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.083840+0000", 2024-09-14T06:31:39.013 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.4679999999999999e-06 2024-09-14T06:31:39.013 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:39.013 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:39.013 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:31:39.014 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.125455+0000", 2024-09-14T06:31:39.014 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.041614578999999999 2024-09-14T06:31:39.014 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:39.014 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:39.014 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:31:39.014 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.125496+0000", 2024-09-14T06:31:39.014 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.1047000000000002e-05 2024-09-14T06:31:39.014 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:39.014 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:39.014 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:31:39.014 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.125610+0000", 2024-09-14T06:31:39.014 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00011407 2024-09-14T06:31:39.014 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:39.015 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:31:39.015 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:39.015 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:39.015 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:39.015 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:14919 3.0 3:05c39535:::rbd_data.112d92027590.0000000000000009:f [read 805888~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:31:39.015 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:26.083843+0000", 2024-09-14T06:31:39.015 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 132.924013726, 2024-09-14T06:31:39.015 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.041992356000000002, 2024-09-14T06:31:39.015 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:31:39.015 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:31:39.015 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:31:39.015 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:31:39.015 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:31:39.015 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:39.016 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 14919 2024-09-14T06:31:39.016 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:39.016 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:31:39.016 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:39.016 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:31:39.016 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.083843+0000", 2024-09-14T06:31:39.016 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:39.016 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:39.016 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:39.016 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:31:39.016 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.083843+0000", 2024-09-14T06:31:39.016 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:39.016 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:39.017 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:39.017 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:31:39.017 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.083844+0000", 2024-09-14T06:31:39.017 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.3300000000000001e-07 2024-09-14T06:31:39.017 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:39.017 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:39.017 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:31:39.017 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.083845+0000", 2024-09-14T06:31:39.017 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.4729999999999999e-06 2024-09-14T06:31:39.017 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:39.017 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:39.017 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:31:39.017 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.083850+0000", 2024-09-14T06:31:39.017 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.0130000000000001e-06 2024-09-14T06:31:39.018 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:39.018 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:39.018 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:31:39.018 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.083854+0000", 2024-09-14T06:31:39.018 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.4759999999999998e-06 2024-09-14T06:31:39.018 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:39.018 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:39.018 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:31:39.018 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.125636+0000", 2024-09-14T06:31:39.018 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.041782683000000001 2024-09-14T06:31:39.018 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:39.018 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:39.018 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:31:39.019 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.125680+0000", 2024-09-14T06:31:39.019 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.3614e-05 2024-09-14T06:31:39.019 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:39.019 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:39.019 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:31:39.019 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.125836+0000", 2024-09-14T06:31:39.019 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000155864 2024-09-14T06:31:39.019 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:39.019 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:31:39.019 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:39.019 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:39.019 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:39.019 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:14920 3.0 3:05c39535:::rbd_data.112d92027590.0000000000000009:f [read 816128~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:31:39.019 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:26.083868+0000", 2024-09-14T06:31:39.019 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 132.923989065, 2024-09-14T06:31:39.020 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.042166771999999998, 2024-09-14T06:31:39.020 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:31:39.020 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:31:39.020 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:31:39.020 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:31:39.020 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:31:39.020 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:39.020 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 14920 2024-09-14T06:31:39.020 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:39.020 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:31:39.020 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:39.020 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:31:39.020 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.083868+0000", 2024-09-14T06:31:39.020 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:39.021 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:39.021 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:39.021 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:31:39.021 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.083868+0000", 2024-09-14T06:31:39.021 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:39.021 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:39.021 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:39.021 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:31:39.021 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.083869+0000", 2024-09-14T06:31:39.021 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.99e-07 2024-09-14T06:31:39.021 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:39.021 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:39.021 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:31:39.022 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.083870+0000", 2024-09-14T06:31:39.022 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.782e-06 2024-09-14T06:31:39.022 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:39.022 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:39.022 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:31:39.022 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.083876+0000", 2024-09-14T06:31:39.022 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.3809999999999998e-06 2024-09-14T06:31:39.022 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:39.022 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:39.022 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:31:39.022 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.083882+0000", 2024-09-14T06:31:39.022 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.9479999999999996e-06 2024-09-14T06:31:39.022 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:39.022 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:39.023 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:31:39.023 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.125878+0000", 2024-09-14T06:31:39.023 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.041996264999999998 2024-09-14T06:31:39.023 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:39.023 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:39.023 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:31:39.023 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.125929+0000", 2024-09-14T06:31:39.023 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.0498999999999999e-05 2024-09-14T06:31:39.023 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:39.023 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:39.023 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:31:39.023 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.126035+0000", 2024-09-14T06:31:39.023 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000106298 2024-09-14T06:31:39.023 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:39.024 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:31:39.024 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:39.024 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:39.024 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:39.024 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:14921 3.0 3:05c39535:::rbd_data.112d92027590.0000000000000009:f [read 826368~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:31:39.024 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:26.083889+0000", 2024-09-14T06:31:39.024 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 132.92396805499999, 2024-09-14T06:31:39.024 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.042435309999999997, 2024-09-14T06:31:39.024 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:31:39.024 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:31:39.024 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:31:39.024 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:31:39.024 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:31:39.024 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:39.025 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 14921 2024-09-14T06:31:39.025 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:39.025 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:31:39.025 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:39.025 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:31:39.025 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.083889+0000", 2024-09-14T06:31:39.025 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:39.025 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:39.025 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:39.025 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:31:39.025 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.083889+0000", 2024-09-14T06:31:39.025 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:39.025 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:39.025 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:39.026 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:31:39.026 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.086033+0000", 2024-09-14T06:31:39.026 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0021444179999999999 2024-09-14T06:31:39.026 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:39.026 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:39.026 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:31:39.026 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.086038+0000", 2024-09-14T06:31:39.026 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.0969999999999999e-06 2024-09-14T06:31:39.026 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:39.026 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:39.026 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:31:39.026 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.086046+0000", 2024-09-14T06:31:39.026 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.9440000000000005e-06 2024-09-14T06:31:39.026 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:39.026 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:39.027 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:31:39.027 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.086050+0000", 2024-09-14T06:31:39.027 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.69e-06 2024-09-14T06:31:39.027 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:39.027 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:39.027 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:31:39.027 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.126080+0000", 2024-09-14T06:31:39.027 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.040030237000000003 2024-09-14T06:31:39.027 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:39.027 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:39.027 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:31:39.027 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.126163+0000", 2024-09-14T06:31:39.027 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.2071999999999996e-05 2024-09-14T06:31:39.027 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:39.027 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:39.028 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:31:39.028 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.126324+0000", 2024-09-14T06:31:39.028 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00016185200000000001 2024-09-14T06:31:39.028 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:39.028 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:31:39.028 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:39.028 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:39.028 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:39.028 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:55627 3.0 3:06490baf:::rbd_data.112dca2f6ae1.0000000000000030:17 [read 2002944~4096 [fadvise_nocache]] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:31:39.028 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:44.599966+0000", 2024-09-14T06:31:39.028 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 114.407891107, 2024-09-14T06:31:39.028 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.041868546999999999, 2024-09-14T06:31:39.028 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:31:39.028 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:31:39.029 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:31:39.029 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:31:39.029 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:31:39.029 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:39.029 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 55627 2024-09-14T06:31:39.029 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:39.029 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:31:39.029 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:39.029 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:31:39.029 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.599966+0000", 2024-09-14T06:31:39.029 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:39.029 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:39.029 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:39.029 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:31:39.029 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.599966+0000", 2024-09-14T06:31:39.030 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:39.030 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:39.030 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:39.030 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:31:39.030 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.599966+0000", 2024-09-14T06:31:39.030 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.0900000000000001e-07 2024-09-14T06:31:39.030 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:39.030 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:39.030 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:31:39.030 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.599967+0000", 2024-09-14T06:31:39.030 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.64e-07 2024-09-14T06:31:39.030 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:39.030 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:39.030 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:31:39.030 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.599970+0000", 2024-09-14T06:31:39.031 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.9670000000000001e-06 2024-09-14T06:31:39.031 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:39.031 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:39.031 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:31:39.031 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.599973+0000", 2024-09-14T06:31:39.031 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.632e-06 2024-09-14T06:31:39.031 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:39.031 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:39.031 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:31:39.031 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.641665+0000", 2024-09-14T06:31:39.031 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.041692547000000003 2024-09-14T06:31:39.031 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:39.031 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:39.031 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:31:39.031 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.641740+0000", 2024-09-14T06:31:39.031 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.4559000000000002e-05 2024-09-14T06:31:39.032 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:39.032 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:39.032 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:31:39.032 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.641835+0000", 2024-09-14T06:31:39.032 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.4668999999999998e-05 2024-09-14T06:31:39.032 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:39.032 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:31:39.032 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:39.032 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:39.032 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:39.032 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:55628 3.0 3:06490baf:::rbd_data.112dca2f6ae1.0000000000000030:17 [read 2027520~4096 [fadvise_nocache]] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:31:39.032 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:44.599976+0000", 2024-09-14T06:31:39.032 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 114.40788132, 2024-09-14T06:31:39.032 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.041991621, 2024-09-14T06:31:39.032 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:31:39.033 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:31:39.033 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:31:39.033 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:31:39.033 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:31:39.033 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:39.033 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 55628 2024-09-14T06:31:39.033 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:39.033 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:31:39.033 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:39.033 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:31:39.033 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.599976+0000", 2024-09-14T06:31:39.033 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:39.033 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:39.033 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:39.033 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:31:39.034 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.599976+0000", 2024-09-14T06:31:39.034 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:39.034 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:39.034 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:39.034 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:31:39.034 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.599976+0000", 2024-09-14T06:31:39.034 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.1299999999999999e-07 2024-09-14T06:31:39.034 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:39.034 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:39.034 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:31:39.034 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.599977+0000", 2024-09-14T06:31:39.035 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.7999999999999993e-07 2024-09-14T06:31:39.035 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:39.035 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:39.035 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:31:39.035 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.599980+0000", 2024-09-14T06:31:39.035 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.01e-06 2024-09-14T06:31:39.035 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:39.035 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:39.035 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:31:39.035 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.599983+0000", 2024-09-14T06:31:39.035 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.5409999999999999e-06 2024-09-14T06:31:39.035 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:39.035 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:39.035 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:31:39.036 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.641869+0000", 2024-09-14T06:31:39.036 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.041886235000000001 2024-09-14T06:31:39.036 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:39.036 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:39.036 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:31:39.036 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.641913+0000", 2024-09-14T06:31:39.036 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.4357000000000002e-05 2024-09-14T06:31:39.036 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:39.036 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:39.036 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:31:39.036 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.641967+0000", 2024-09-14T06:31:39.036 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.4284999999999999e-05 2024-09-14T06:31:39.036 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:39.036 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:31:39.037 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:39.037 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:39.037 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:39.037 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:55629 3.0 3:06490baf:::rbd_data.112dca2f6ae1.0000000000000030:17 [read 2052096~4096 [fadvise_nocache]] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:31:39.037 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:44.599985+0000", 2024-09-14T06:31:39.037 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 114.407871614, 2024-09-14T06:31:39.037 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.042183375000000002, 2024-09-14T06:31:39.037 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:31:39.037 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:31:39.037 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:31:39.037 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:31:39.037 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:31:39.037 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:39.038 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 55629 2024-09-14T06:31:39.038 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:39.038 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:31:39.038 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:39.038 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:31:39.038 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.599985+0000", 2024-09-14T06:31:39.038 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:39.038 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:39.038 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:39.038 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:31:39.038 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.599985+0000", 2024-09-14T06:31:39.038 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:39.038 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:39.038 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:39.038 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:31:39.039 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.599986+0000", 2024-09-14T06:31:39.039 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.05e-07 2024-09-14T06:31:39.039 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:39.039 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:39.039 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:31:39.039 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.599987+0000", 2024-09-14T06:31:39.039 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.8400000000000002e-07 2024-09-14T06:31:39.039 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:39.039 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:39.039 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:31:39.039 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.599990+0000", 2024-09-14T06:31:39.039 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.5640000000000001e-06 2024-09-14T06:31:39.039 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:39.039 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:39.040 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:31:39.040 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.599993+0000", 2024-09-14T06:31:39.040 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.615e-06 2024-09-14T06:31:39.040 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:39.040 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:39.040 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:31:39.040 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.642003+0000", 2024-09-14T06:31:39.040 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.042010345999999997 2024-09-14T06:31:39.040 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:39.040 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:39.040 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:31:39.040 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.642080+0000", 2024-09-14T06:31:39.040 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.6538999999999993e-05 2024-09-14T06:31:39.040 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:39.041 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:39.041 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:31:39.041 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.642169+0000", 2024-09-14T06:31:39.041 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.9122000000000005e-05 2024-09-14T06:31:39.041 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:39.041 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:31:39.041 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:39.041 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:39.041 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:39.041 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:55630 3.0 3:06490baf:::rbd_data.112dca2f6ae1.0000000000000030:17 [read 2076672~4096 [fadvise_nocache]] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:31:39.041 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:44.599996+0000", 2024-09-14T06:31:39.041 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 114.40786113599999, 2024-09-14T06:31:39.042 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.042299615999999998, 2024-09-14T06:31:39.042 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:31:39.042 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:31:39.042 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:31:39.042 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:31:39.042 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:31:39.042 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:39.042 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 55630 2024-09-14T06:31:39.042 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:39.042 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:31:39.042 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:39.042 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:31:39.042 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.599996+0000", 2024-09-14T06:31:39.042 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:39.042 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:39.043 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:39.043 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:31:39.043 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.599996+0000", 2024-09-14T06:31:39.043 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:39.043 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:39.043 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:39.043 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:31:39.043 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.599996+0000", 2024-09-14T06:31:39.043 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.1500000000000001e-07 2024-09-14T06:31:39.043 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:39.043 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:39.043 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:31:39.043 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.599997+0000", 2024-09-14T06:31:39.043 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.33e-07 2024-09-14T06:31:39.044 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:39.044 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:39.044 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:31:39.044 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600000+0000", 2024-09-14T06:31:39.044 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.0240000000000002e-06 2024-09-14T06:31:39.044 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:39.044 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:39.044 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:31:39.044 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600003+0000", 2024-09-14T06:31:39.044 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.3640000000000002e-06 2024-09-14T06:31:39.044 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:39.044 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:39.044 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:31:39.044 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.642200+0000", 2024-09-14T06:31:39.045 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.042197571000000003 2024-09-14T06:31:39.045 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:39.045 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:39.045 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:31:39.045 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.642243+0000", 2024-09-14T06:31:39.045 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.2765000000000001e-05 2024-09-14T06:31:39.045 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:39.045 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:39.045 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:31:39.045 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.642296+0000", 2024-09-14T06:31:39.045 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.2744000000000002e-05 2024-09-14T06:31:39.045 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:39.045 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:31:39.045 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:39.046 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:39.046 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:39.046 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:55631 3.0 3:06490baf:::rbd_data.112dca2f6ae1.0000000000000030:17 [read 1048576~4096 [fadvise_nocache]] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:31:39.046 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:44.600006+0000", 2024-09-14T06:31:39.046 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 114.407851548, 2024-09-14T06:31:39.046 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.042487905999999999, 2024-09-14T06:31:39.046 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:31:39.046 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:31:39.046 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:31:39.046 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:31:39.046 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:31:39.046 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:39.046 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 55631 2024-09-14T06:31:39.046 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:39.047 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:31:39.047 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:39.047 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:31:39.047 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600006+0000", 2024-09-14T06:31:39.047 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:39.047 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:39.047 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:39.047 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:31:39.047 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600006+0000", 2024-09-14T06:31:39.047 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:39.047 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:39.047 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:39.047 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:31:39.047 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600006+0000", 2024-09-14T06:31:39.048 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.0800000000000001e-07 2024-09-14T06:31:39.048 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:39.048 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:39.048 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:31:39.048 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600007+0000", 2024-09-14T06:31:39.048 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.7199999999999997e-07 2024-09-14T06:31:39.048 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:39.048 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:39.048 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:31:39.048 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600010+0000", 2024-09-14T06:31:39.048 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.0280000000000001e-06 2024-09-14T06:31:39.048 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:39.048 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:39.049 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:31:39.049 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600012+0000", 2024-09-14T06:31:39.049 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.475e-06 2024-09-14T06:31:39.049 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:39.049 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:39.049 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:31:39.049 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.642329+0000", 2024-09-14T06:31:39.049 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.042317157000000001 2024-09-14T06:31:39.049 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:39.049 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:39.049 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:31:39.049 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.642403+0000", 2024-09-14T06:31:39.049 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.3941000000000002e-05 2024-09-14T06:31:39.049 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:39.050 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:39.050 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:31:39.050 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.642493+0000", 2024-09-14T06:31:39.050 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.0124999999999995e-05 2024-09-14T06:31:39.050 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:39.050 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:31:39.050 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:39.050 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:39.050 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:39.050 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:55632 3.0 3:06490baf:::rbd_data.112dca2f6ae1.0000000000000030:17 [read 1073152~4096 [fadvise_nocache]] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:31:39.050 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:44.600015+0000", 2024-09-14T06:31:39.050 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 114.407841795, 2024-09-14T06:31:39.050 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.042604922000000003, 2024-09-14T06:31:39.051 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:31:39.051 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:31:39.051 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:31:39.051 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:31:39.051 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:31:39.051 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:39.051 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 55632 2024-09-14T06:31:39.051 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:39.051 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:31:39.051 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:39.051 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:31:39.051 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600015+0000", 2024-09-14T06:31:39.051 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:39.052 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:39.052 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:39.052 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:31:39.052 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600015+0000", 2024-09-14T06:31:39.052 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:39.052 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:39.052 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:39.052 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:31:39.052 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600016+0000", 2024-09-14T06:31:39.052 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.1e-07 2024-09-14T06:31:39.053 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:39.053 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:39.053 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:31:39.053 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600017+0000", 2024-09-14T06:31:39.053 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.9900000000000009e-07 2024-09-14T06:31:39.053 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:39.053 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:39.053 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:31:39.053 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600020+0000", 2024-09-14T06:31:39.053 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.4170000000000001e-06 2024-09-14T06:31:39.053 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:39.053 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:39.053 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:31:39.054 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600023+0000", 2024-09-14T06:31:39.054 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.588e-06 2024-09-14T06:31:39.054 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:39.054 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:39.054 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:31:39.054 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.642526+0000", 2024-09-14T06:31:39.054 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.042503236999999999 2024-09-14T06:31:39.054 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:39.054 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:39.054 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:31:39.054 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.642568+0000", 2024-09-14T06:31:39.054 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.1925000000000003e-05 2024-09-14T06:31:39.054 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:39.054 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:39.055 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:31:39.055 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.642620+0000", 2024-09-14T06:31:39.055 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.2546000000000001e-05 2024-09-14T06:31:39.055 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:39.055 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:31:39.055 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:39.055 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:39.055 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:39.055 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:55633 3.0 3:06490baf:::rbd_data.112dca2f6ae1.0000000000000030:17 [read 1097728~4096 [fadvise_nocache]] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:31:39.055 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:44.600026+0000", 2024-09-14T06:31:39.055 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 114.407831526, 2024-09-14T06:31:39.055 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.042797474000000002, 2024-09-14T06:31:39.055 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:31:39.055 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:31:39.056 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:31:39.056 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:31:39.056 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:31:39.056 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:39.056 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 55633 2024-09-14T06:31:39.056 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:39.056 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:31:39.056 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:39.056 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:31:39.056 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600026+0000", 2024-09-14T06:31:39.056 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:39.056 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:39.056 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:39.056 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:31:39.057 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600026+0000", 2024-09-14T06:31:39.057 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:39.057 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:39.057 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:39.057 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:31:39.057 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600026+0000", 2024-09-14T06:31:39.057 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.3099999999999999e-07 2024-09-14T06:31:39.057 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:39.057 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:39.057 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:31:39.057 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600029+0000", 2024-09-14T06:31:39.057 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.304e-06 2024-09-14T06:31:39.057 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:39.058 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:39.058 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:31:39.058 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600034+0000", 2024-09-14T06:31:39.060 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.1429999999999997e-06 2024-09-14T06:31:39.060 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:39.060 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:39.060 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:31:39.061 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600038+0000", 2024-09-14T06:31:39.061 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.692e-06 2024-09-14T06:31:39.061 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:39.061 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:39.061 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:31:39.061 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.642654+0000", 2024-09-14T06:31:39.061 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.042615564000000002 2024-09-14T06:31:39.061 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:39.061 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:39.061 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:31:39.061 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.642728+0000", 2024-09-14T06:31:39.061 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.4018999999999997e-05 2024-09-14T06:31:39.061 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:39.062 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:39.062 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:31:39.062 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.642823+0000", 2024-09-14T06:31:39.062 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.5520999999999996e-05 2024-09-14T06:31:39.062 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:39.062 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:31:39.062 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:39.062 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:39.062 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:39.062 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:55634 3.0 3:06490baf:::rbd_data.112dca2f6ae1.0000000000000030:17 [read 1122304~4096 [fadvise_nocache]] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:31:39.062 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:44.600042+0000", 2024-09-14T06:31:39.062 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 114.40781504900001, 2024-09-14T06:31:39.062 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.042903729000000002, 2024-09-14T06:31:39.063 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:31:39.063 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:31:39.063 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:31:39.063 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:31:39.063 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:31:39.063 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:39.063 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 55634 2024-09-14T06:31:39.063 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:39.063 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:31:39.063 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:39.063 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:31:39.063 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600042+0000", 2024-09-14T06:31:39.063 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:39.063 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:39.064 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:39.064 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:31:39.064 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600042+0000", 2024-09-14T06:31:39.064 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:39.064 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:39.064 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:39.064 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:31:39.064 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600043+0000", 2024-09-14T06:31:39.064 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.7000000000000005e-07 2024-09-14T06:31:39.064 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:39.064 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:39.064 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:31:39.064 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600044+0000", 2024-09-14T06:31:39.064 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.742e-06 2024-09-14T06:31:39.065 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:39.065 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:39.065 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:31:39.065 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600050+0000", 2024-09-14T06:31:39.065 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.0800000000000002e-06 2024-09-14T06:31:39.065 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:39.065 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:39.065 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:31:39.065 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600055+0000", 2024-09-14T06:31:39.065 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.3889999999999998e-06 2024-09-14T06:31:39.065 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:39.065 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:39.065 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:31:39.065 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.642845+0000", 2024-09-14T06:31:39.066 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.042790040000000001 2024-09-14T06:31:39.066 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:39.066 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:39.066 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:31:39.066 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.642888+0000", 2024-09-14T06:31:39.066 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.2790000000000002e-05 2024-09-14T06:31:39.066 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:39.066 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:39.066 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:31:39.066 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.642946+0000", 2024-09-14T06:31:39.066 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.8118000000000002e-05 2024-09-14T06:31:39.066 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:39.066 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:31:39.066 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:39.067 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:39.067 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:39.067 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:55635 3.0 3:06490baf:::rbd_data.112dca2f6ae1.0000000000000030:17 [read 1146880~4096 [fadvise_nocache]] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:31:39.067 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:44.600061+0000", 2024-09-14T06:31:39.067 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 114.40779647799999, 2024-09-14T06:31:39.067 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.043092695, 2024-09-14T06:31:39.067 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:31:39.067 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:31:39.067 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:31:39.067 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:31:39.067 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:31:39.067 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:39.067 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 55635 2024-09-14T06:31:39.068 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:39.068 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:31:39.068 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:39.068 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:31:39.068 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600061+0000", 2024-09-14T06:31:39.068 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:39.068 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:39.068 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:39.068 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:31:39.068 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600061+0000", 2024-09-14T06:31:39.068 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:39.068 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:39.069 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:39.069 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:31:39.069 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600061+0000", 2024-09-14T06:31:39.069 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.3099999999999999e-07 2024-09-14T06:31:39.069 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:39.069 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:39.069 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:31:39.069 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600063+0000", 2024-09-14T06:31:39.069 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.6220000000000001e-06 2024-09-14T06:31:39.069 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:39.069 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:39.069 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:31:39.069 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600068+0000", 2024-09-14T06:31:39.069 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.4790000000000004e-06 2024-09-14T06:31:39.070 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:39.070 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:39.070 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:31:39.070 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600072+0000", 2024-09-14T06:31:39.070 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.3100000000000002e-06 2024-09-14T06:31:39.070 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:39.070 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:39.070 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:31:39.070 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.642987+0000", 2024-09-14T06:31:39.070 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.042914489 2024-09-14T06:31:39.070 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:39.070 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:39.070 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:31:39.070 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.643062+0000", 2024-09-14T06:31:39.070 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.5011000000000004e-05 2024-09-14T06:31:39.071 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:39.071 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:39.071 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:31:39.071 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.643153+0000", 2024-09-14T06:31:39.071 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.1452999999999993e-05 2024-09-14T06:31:39.071 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:39.071 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:31:39.071 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:39.071 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:39.071 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:39.071 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:55636 3.0 3:06490baf:::rbd_data.112dca2f6ae1.0000000000000030:17 [read 1171456~4096 [fadvise_nocache]] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:31:39.071 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:44.600078+0000", 2024-09-14T06:31:39.071 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 114.407779066, 2024-09-14T06:31:39.071 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.043207966, 2024-09-14T06:31:39.072 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:31:39.072 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:31:39.072 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:31:39.072 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:31:39.072 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:31:39.072 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:39.072 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 55636 2024-09-14T06:31:39.072 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:39.072 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:31:39.072 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:39.072 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:31:39.072 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600078+0000", 2024-09-14T06:31:39.072 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:39.072 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:39.073 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:39.073 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:31:39.073 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600078+0000", 2024-09-14T06:31:39.073 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:39.073 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:39.073 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:39.073 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:31:39.073 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600078+0000", 2024-09-14T06:31:39.073 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.34e-07 2024-09-14T06:31:39.073 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:39.073 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:39.073 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:31:39.073 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600080+0000", 2024-09-14T06:31:39.073 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.641e-06 2024-09-14T06:31:39.074 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:39.074 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:39.074 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:31:39.074 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600086+0000", 2024-09-14T06:31:39.074 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.3860000000000004e-06 2024-09-14T06:31:39.074 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:39.074 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:39.074 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:31:39.074 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600091+0000", 2024-09-14T06:31:39.074 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.5970000000000002e-06 2024-09-14T06:31:39.074 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:39.074 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:39.074 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:31:39.074 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.643184+0000", 2024-09-14T06:31:39.075 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.043092893 2024-09-14T06:31:39.075 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:39.075 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:39.075 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:31:39.075 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.643228+0000", 2024-09-14T06:31:39.075 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.4536999999999997e-05 2024-09-14T06:31:39.075 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:39.075 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:39.075 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:31:39.075 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.643286+0000", 2024-09-14T06:31:39.075 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.7578000000000003e-05 2024-09-14T06:31:39.075 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:39.075 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:31:39.075 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:39.075 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:39.076 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:39.076 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:55637 3.0 3:06490baf:::rbd_data.112dca2f6ae1.0000000000000030:17 [read 1196032~4096 [fadvise_nocache]] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:31:39.076 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:44.600097+0000", 2024-09-14T06:31:39.076 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 114.407760502, 2024-09-14T06:31:39.076 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.043399124999999997, 2024-09-14T06:31:39.076 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:31:39.076 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:31:39.076 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:31:39.076 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:31:39.076 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:31:39.076 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:39.076 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 55637 2024-09-14T06:31:39.076 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:39.077 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:31:39.077 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:39.077 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:31:39.077 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600097+0000", 2024-09-14T06:31:39.077 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:39.077 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:39.077 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:39.077 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:31:39.077 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600097+0000", 2024-09-14T06:31:39.077 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:39.077 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:39.077 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:39.077 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:31:39.077 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.604372+0000", 2024-09-14T06:31:39.078 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0042751489999999998 2024-09-14T06:31:39.078 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:39.078 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:39.078 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:31:39.078 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.604377+0000", 2024-09-14T06:31:39.078 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.7899999999999999e-06 2024-09-14T06:31:39.078 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:39.078 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:39.078 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:31:39.078 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.604389+0000", 2024-09-14T06:31:39.078 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.2197e-05 2024-09-14T06:31:39.078 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:39.078 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:39.078 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:31:39.079 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.604397+0000", 2024-09-14T06:31:39.079 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.8730000000000007e-06 2024-09-14T06:31:39.079 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:39.079 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:39.079 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:31:39.079 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.643324+0000", 2024-09-14T06:31:39.079 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.038926897000000002 2024-09-14T06:31:39.079 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:39.079 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:39.079 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:31:39.079 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.643402+0000", 2024-09-14T06:31:39.079 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.8063000000000001e-05 2024-09-14T06:31:39.079 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:39.079 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:39.079 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:31:39.080 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.643496+0000", 2024-09-14T06:31:39.080 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.4155999999999998e-05 2024-09-14T06:31:39.080 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:39.080 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:31:39.080 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:39.080 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:39.080 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:39.080 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:55657 3.0 3:06490baf:::rbd_data.112dca2f6ae1.0000000000000030:17 [read 1687552~4096 [fadvise_nocache]] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:31:39.080 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:44.604841+0000", 2024-09-14T06:31:39.080 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 114.40301577300001, 2024-09-14T06:31:39.080 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.041890087999999999, 2024-09-14T06:31:39.080 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:31:39.080 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:31:39.080 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:31:39.081 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:31:39.081 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:31:39.081 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:39.081 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 55657 2024-09-14T06:31:39.081 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:39.081 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:31:39.081 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:39.081 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:31:39.081 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.604841+0000", 2024-09-14T06:31:39.081 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:39.081 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:39.081 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:39.081 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:31:39.082 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.604841+0000", 2024-09-14T06:31:39.082 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:39.082 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:39.082 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:39.082 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:31:39.082 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.604842+0000", 2024-09-14T06:31:39.082 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.0900000000000001e-07 2024-09-14T06:31:39.082 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:39.082 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:39.082 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:31:39.082 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.604846+0000", 2024-09-14T06:31:39.082 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.5759999999999999e-06 2024-09-14T06:31:39.082 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:39.082 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:39.083 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:31:39.083 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.604850+0000", 2024-09-14T06:31:39.083 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.405e-06 2024-09-14T06:31:39.083 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:39.083 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:39.083 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:31:39.083 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.604852+0000", 2024-09-14T06:31:39.083 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.8590000000000001e-06 2024-09-14T06:31:39.083 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:39.083 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:39.083 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:31:39.083 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.646633+0000", 2024-09-14T06:31:39.083 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.041780772000000001 2024-09-14T06:31:39.083 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:39.083 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:39.084 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:31:39.084 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.646677+0000", 2024-09-14T06:31:39.084 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.4115e-05 2024-09-14T06:31:39.084 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:39.084 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:39.084 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:31:39.084 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.646731+0000", 2024-09-14T06:31:39.084 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.4151999999999999e-05 2024-09-14T06:31:39.084 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:39.084 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:31:39.084 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:39.084 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:39.084 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:39.084 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:55658 3.0 3:06490baf:::rbd_data.112dca2f6ae1.0000000000000030:17 [read 1712128~4096 [fadvise_nocache]] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:31:39.084 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:44.604856+0000", 2024-09-14T06:31:39.085 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 114.40300117300001, 2024-09-14T06:31:39.085 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.041986227000000001, 2024-09-14T06:31:39.085 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:31:39.085 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:31:39.085 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:31:39.085 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:31:39.085 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:31:39.085 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:39.085 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 55658 2024-09-14T06:31:39.085 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:39.085 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:31:39.085 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:39.085 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:31:39.085 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.604856+0000", 2024-09-14T06:31:39.086 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:39.086 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:39.086 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:39.086 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:31:39.086 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.604856+0000", 2024-09-14T06:31:39.086 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:39.086 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:39.086 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:39.086 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:31:39.086 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.604856+0000", 2024-09-14T06:31:39.086 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.23e-07 2024-09-14T06:31:39.086 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:39.086 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:39.086 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:31:39.086 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.604857+0000", 2024-09-14T06:31:39.087 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.9999999999999995e-07 2024-09-14T06:31:39.087 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:39.087 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:39.087 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:31:39.087 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.604861+0000", 2024-09-14T06:31:39.087 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.9890000000000003e-06 2024-09-14T06:31:39.087 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:39.087 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:39.087 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:31:39.087 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.604864+0000", 2024-09-14T06:31:39.087 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.7429999999999998e-06 2024-09-14T06:31:39.087 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:39.087 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:39.087 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:31:39.087 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.646744+0000", 2024-09-14T06:31:39.088 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.041880325000000003 2024-09-14T06:31:39.088 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:39.088 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:39.088 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:31:39.088 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.646784+0000", 2024-09-14T06:31:39.088 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.9604000000000002e-05 2024-09-14T06:31:39.088 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:39.088 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:39.088 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:31:39.088 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.646842+0000", 2024-09-14T06:31:39.088 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.8343e-05 2024-09-14T06:31:39.088 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:39.088 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:31:39.089 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:39.089 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:39.089 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:39.089 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:55659 3.0 3:06490baf:::rbd_data.112dca2f6ae1.0000000000000030:17 [read 1736704~4096 [fadvise_nocache]] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:31:39.089 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:44.604867+0000", 2024-09-14T06:31:39.089 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 114.40299002, 2024-09-14T06:31:39.089 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.042097082000000001, 2024-09-14T06:31:39.089 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:31:39.089 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:31:39.089 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:31:39.089 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:31:39.089 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:31:39.089 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:39.090 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 55659 2024-09-14T06:31:39.090 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:39.090 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:31:39.090 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:39.090 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:31:39.090 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.604867+0000", 2024-09-14T06:31:39.090 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:39.090 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:39.090 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:39.090 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:31:39.090 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.604867+0000", 2024-09-14T06:31:39.090 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:39.090 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:39.091 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:39.091 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:31:39.091 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.604867+0000", 2024-09-14T06:31:39.091 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.2000000000000001e-07 2024-09-14T06:31:39.091 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:39.091 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:39.091 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:31:39.091 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.604868+0000", 2024-09-14T06:31:39.091 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.04e-06 2024-09-14T06:31:39.091 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:39.091 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:39.091 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:31:39.091 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.604871+0000", 2024-09-14T06:31:39.091 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.9689999999999999e-06 2024-09-14T06:31:39.092 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:39.092 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:39.092 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:31:39.092 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.604874+0000", 2024-09-14T06:31:39.092 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.5069999999999999e-06 2024-09-14T06:31:39.092 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:39.092 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:39.092 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:31:39.092 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.646874+0000", 2024-09-14T06:31:39.092 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.042000513000000003 2024-09-14T06:31:39.092 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:39.092 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:39.092 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:31:39.092 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.646914+0000", 2024-09-14T06:31:39.093 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.9443999999999997e-05 2024-09-14T06:31:39.093 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:39.093 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:39.093 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:31:39.093 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.646964+0000", 2024-09-14T06:31:39.093 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.0389000000000002e-05 2024-09-14T06:31:39.093 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:39.093 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:31:39.093 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:39.093 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:39.093 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:39.093 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:55660 3.0 3:06490baf:::rbd_data.112dca2f6ae1.0000000000000030:17 [read 1761280~4096 [fadvise_nocache]] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:31:39.093 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:44.604877+0000", 2024-09-14T06:31:39.093 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 114.402980271, 2024-09-14T06:31:39.094 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.042196349000000001, 2024-09-14T06:31:39.094 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:31:39.094 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:31:39.094 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:31:39.094 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:31:39.094 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:31:39.094 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:39.094 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 55660 2024-09-14T06:31:39.094 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:39.094 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:31:39.094 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:39.094 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:31:39.094 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.604877+0000", 2024-09-14T06:31:39.094 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:39.095 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:39.095 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:39.095 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:31:39.095 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.604877+0000", 2024-09-14T06:31:39.095 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:39.095 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:39.095 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:39.095 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:31:39.095 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.605810+0000", 2024-09-14T06:31:39.095 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00093336899999999997 2024-09-14T06:31:39.095 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:39.095 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:39.095 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:31:39.095 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.605815+0000", 2024-09-14T06:31:39.096 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.6140000000000002e-06 2024-09-14T06:31:39.096 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:39.096 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:39.096 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:31:39.096 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.605825+0000", 2024-09-14T06:31:39.096 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.8889999999999997e-06 2024-09-14T06:31:39.096 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:39.096 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:39.096 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:31:39.096 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.605831+0000", 2024-09-14T06:31:39.096 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.9800000000000003e-06 2024-09-14T06:31:39.096 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:39.097 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:39.097 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:31:39.097 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.646989+0000", 2024-09-14T06:31:39.097 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.041157878000000002 2024-09-14T06:31:39.097 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:39.097 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:39.097 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:31:39.097 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.647026+0000", 2024-09-14T06:31:39.097 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.7305000000000003e-05 2024-09-14T06:31:39.097 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:39.097 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:39.097 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:31:39.097 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.647073+0000", 2024-09-14T06:31:39.098 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.7314000000000001e-05 2024-09-14T06:31:39.098 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:39.098 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:31:39.098 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:39.098 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:39.098 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:39.098 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:55666 3.0 3:06490baf:::rbd_data.112dca2f6ae1.0000000000000030:17 [read 1908736~4096 [fadvise_nocache]] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:31:39.098 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:44.605898+0000", 2024-09-14T06:31:39.098 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 114.401959525, 2024-09-14T06:31:39.098 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.041862887000000001, 2024-09-14T06:31:39.098 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:31:39.098 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:31:39.098 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:31:39.098 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:31:39.099 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:31:39.099 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:39.099 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 55666 2024-09-14T06:31:39.099 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:39.099 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:31:39.099 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:39.099 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:31:39.099 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.605898+0000", 2024-09-14T06:31:39.099 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:39.099 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:39.099 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:39.099 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:31:39.099 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.605898+0000", 2024-09-14T06:31:39.099 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:39.100 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:39.100 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:39.100 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:31:39.100 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.611874+0000", 2024-09-14T06:31:39.100 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0059764049999999997 2024-09-14T06:31:39.100 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:39.100 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:39.100 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:31:39.100 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.611879+0000", 2024-09-14T06:31:39.100 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.8749999999999999e-06 2024-09-14T06:31:39.100 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:39.100 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:39.100 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:31:39.101 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.611887+0000", 2024-09-14T06:31:39.101 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.4109999999999996e-06 2024-09-14T06:31:39.101 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:39.101 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:39.101 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:31:39.101 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.611893+0000", 2024-09-14T06:31:39.101 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.2959999999999998e-06 2024-09-14T06:31:39.101 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:39.101 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:39.101 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:31:39.101 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.647674+0000", 2024-09-14T06:31:39.101 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.035781281999999998 2024-09-14T06:31:39.101 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:39.102 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:39.102 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:31:39.102 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.647711+0000", 2024-09-14T06:31:39.102 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.7434000000000001e-05 2024-09-14T06:31:39.102 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:39.102 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:39.102 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:31:39.102 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.647760+0000", 2024-09-14T06:31:39.102 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.9184000000000002e-05 2024-09-14T06:31:39.102 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:39.102 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:31:39.102 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:39.102 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:39.102 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:31:39.103 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-14T06:31:39.103 INFO:tasks.ceph.osd.0.smithi094.stderr:2024-09-14T06:31:39.007+0000 7f07c9333640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33636) UID: 0 2024-09-14T06:31:39.104 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-14T06:31:39.108 INFO:tasks.ceph.osd.6.smithi138.stderr:2024-09-14T06:31:39.107+0000 7f8cd8849640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33620) UID: 0 2024-09-14T06:31:39.209 INFO:tasks.ceph.osd.5.smithi138.stderr:2024-09-14T06:31:39.208+0000 7fe13746f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33618) UID: 0 2024-09-14T06:31:39.248 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-14T06:31:39.248 INFO:teuthology.orchestra.run.smithi094.stdout: "ops": [], 2024-09-14T06:31:39.248 INFO:teuthology.orchestra.run.smithi094.stdout: "num_ops": 0 2024-09-14T06:31:39.248 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-14T06:31:39.256 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-14T06:31:39.309 INFO:tasks.ceph.osd.6.smithi138.stderr:2024-09-14T06:31:39.308+0000 7f8cd8849640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33620) UID: 0 2024-09-14T06:31:39.360 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-14T06:31:39.360 INFO:teuthology.orchestra.run.smithi094.stdout: "ops": [], 2024-09-14T06:31:39.360 INFO:teuthology.orchestra.run.smithi094.stdout: "complaint_time": 30, 2024-09-14T06:31:39.360 INFO:teuthology.orchestra.run.smithi094.stdout: "num_blocked_ops": 0 2024-09-14T06:31:39.360 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-14T06:31:39.368 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-14T06:31:39.410 INFO:tasks.ceph.osd.7.smithi138.stderr:2024-09-14T06:31:39.409+0000 7f3c6a21d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33623) UID: 0 2024-09-14T06:31:39.472 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-14T06:31:39.472 INFO:teuthology.orchestra.run.smithi094.stdout: "size": 20, 2024-09-14T06:31:39.472 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 600, 2024-09-14T06:31:39.472 INFO:teuthology.orchestra.run.smithi094.stdout: "ops": [ 2024-09-14T06:31:39.472 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:39.472 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4397.0:45722 3.9 e64/23 3:91fed098:::rbd_data.112d5a737e72.0000000000000001:head v 64'61, mlcod=62'59)", 2024-09-14T06:31:39.472 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:43.864269+0000", 2024-09-14T06:31:39.472 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 115.606085951, 2024-09-14T06:31:39.472 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.003055224, 2024-09-14T06:31:39.473 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:31:39.473 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:31:39.473 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:31:39.473 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:31:39.473 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:39.473 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:31:39.473 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.864269+0000", 2024-09-14T06:31:39.473 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:39.473 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:39.473 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:39.473 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:31:39.473 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.864269+0000", 2024-09-14T06:31:39.473 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:39.474 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:39.474 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:39.474 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:31:39.474 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.864269+0000", 2024-09-14T06:31:39.474 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.41e-07 2024-09-14T06:31:39.474 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:39.474 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:39.474 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:31:39.474 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.865490+0000", 2024-09-14T06:31:39.474 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.001220552 2024-09-14T06:31:39.474 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:39.474 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:39.474 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:31:39.474 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.865506+0000", 2024-09-14T06:31:39.475 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.6263999999999999e-05 2024-09-14T06:31:39.475 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:39.475 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:39.475 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:31:39.475 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.865516+0000", 2024-09-14T06:31:39.475 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.0091999999999999e-05 2024-09-14T06:31:39.475 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:39.475 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:39.475 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:31:39.475 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.865583+0000", 2024-09-14T06:31:39.475 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.6676999999999999e-05 2024-09-14T06:31:39.475 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:39.475 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:39.476 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:31:39.476 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.865610+0000", 2024-09-14T06:31:39.476 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.6529999999999998e-05 2024-09-14T06:31:39.476 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:39.476 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:39.476 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-14T06:31:39.476 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.867281+0000", 2024-09-14T06:31:39.476 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.001671203 2024-09-14T06:31:39.476 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:39.476 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:39.476 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:31:39.476 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.867324+0000", 2024-09-14T06:31:39.476 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.3665000000000003e-05 2024-09-14T06:31:39.476 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:39.477 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:31:39.477 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:39.477 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:39.477 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:39.477 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4397.0:74241 3.0 e73/23 3:06e93fb9:::rbd_data.112d80ca42af.0000000000000000:head v 73'102, mlcod=71'100)", 2024-09-14T06:31:39.477 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:53.876730+0000", 2024-09-14T06:31:39.477 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 105.593624851, 2024-09-14T06:31:39.477 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0079470329999999992, 2024-09-14T06:31:39.477 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:31:39.477 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:31:39.477 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:31:39.477 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:31:39.477 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:39.478 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:31:39.478 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:53.876730+0000", 2024-09-14T06:31:39.478 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:39.478 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:39.478 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:39.478 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:31:39.478 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:53.876730+0000", 2024-09-14T06:31:39.478 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:39.478 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:39.478 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:39.478 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:31:39.478 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:53.876731+0000", 2024-09-14T06:31:39.478 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.0500000000000004e-07 2024-09-14T06:31:39.479 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:39.479 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:39.479 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:31:39.479 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:53.879616+0000", 2024-09-14T06:31:39.479 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0028856680000000001 2024-09-14T06:31:39.479 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:39.479 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:39.479 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:31:39.479 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:53.879643+0000", 2024-09-14T06:31:39.479 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.6120999999999999e-05 2024-09-14T06:31:39.479 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:39.479 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:39.479 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:31:39.479 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:53.879654+0000", 2024-09-14T06:31:39.479 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.1586999999999999e-05 2024-09-14T06:31:39.480 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:39.480 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:39.480 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:31:39.480 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:53.879756+0000", 2024-09-14T06:31:39.480 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00010223899999999999 2024-09-14T06:31:39.480 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:39.480 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:39.480 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:31:39.480 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:53.879783+0000", 2024-09-14T06:31:39.480 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.6913000000000001e-05 2024-09-14T06:31:39.480 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:39.480 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:39.480 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-14T06:31:39.480 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:53.884594+0000", 2024-09-14T06:31:39.481 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0048110729999999999 2024-09-14T06:31:39.481 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:39.481 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:39.481 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:31:39.481 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:53.884677+0000", 2024-09-14T06:31:39.481 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.2927000000000001e-05 2024-09-14T06:31:39.481 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:39.481 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:31:39.481 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:39.481 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:39.481 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:39.481 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4397.0:74243 3.9 e73/23 3:94977f81:::rbd_data.112d80ca42af.0000000000000002:head v 73'94, mlcod=71'92)", 2024-09-14T06:31:39.481 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:53.885156+0000", 2024-09-14T06:31:39.481 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 105.585198933, 2024-09-14T06:31:39.482 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0096009860000000006, 2024-09-14T06:31:39.482 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:31:39.482 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:31:39.482 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:31:39.482 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:31:39.482 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:39.482 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:31:39.482 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:53.885156+0000", 2024-09-14T06:31:39.482 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:39.482 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:39.482 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:39.482 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:31:39.482 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:53.885156+0000", 2024-09-14T06:31:39.482 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:39.483 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:39.483 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:39.483 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:31:39.483 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:53.885156+0000", 2024-09-14T06:31:39.483 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.23e-07 2024-09-14T06:31:39.483 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:39.483 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:39.483 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:31:39.483 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:53.889694+0000", 2024-09-14T06:31:39.483 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0045378709999999997 2024-09-14T06:31:39.483 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:39.483 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:39.483 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:31:39.483 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:53.889720+0000", 2024-09-14T06:31:39.483 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.5259e-05 2024-09-14T06:31:39.484 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:39.484 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:39.484 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:31:39.484 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:53.889731+0000", 2024-09-14T06:31:39.484 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.1739000000000001e-05 2024-09-14T06:31:39.484 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:39.484 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:39.484 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:31:39.484 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:53.889822+0000", 2024-09-14T06:31:39.484 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.1200999999999996e-05 2024-09-14T06:31:39.484 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:39.484 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:39.484 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:31:39.484 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:53.889850+0000", 2024-09-14T06:31:39.485 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.7084999999999999e-05 2024-09-14T06:31:39.485 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:39.485 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:39.485 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-14T06:31:39.485 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:53.894703+0000", 2024-09-14T06:31:39.485 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0048532690000000003 2024-09-14T06:31:39.485 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:39.485 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:39.485 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:31:39.485 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:53.894757+0000", 2024-09-14T06:31:39.485 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.4339000000000003e-05 2024-09-14T06:31:39.485 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:39.485 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:31:39.485 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:39.485 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:39.486 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:39.486 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4397.0:74249 3.0 e73/23 3:009b0225:::rbd_data.112d80ca42af.0000000000000009:head v 73'103, mlcod=71'101)", 2024-09-14T06:31:39.486 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:53.893219+0000", 2024-09-14T06:31:39.486 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 105.577136455, 2024-09-14T06:31:39.486 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0065381980000000003, 2024-09-14T06:31:39.486 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:31:39.486 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:31:39.486 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:31:39.486 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:31:39.486 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:39.486 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:31:39.486 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:53.893219+0000", 2024-09-14T06:31:39.486 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:39.486 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:39.487 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:39.487 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:31:39.487 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:53.893219+0000", 2024-09-14T06:31:39.487 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:39.487 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:39.487 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:39.487 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:31:39.487 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:53.893221+0000", 2024-09-14T06:31:39.487 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.4329999999999998e-06 2024-09-14T06:31:39.487 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:39.487 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:39.487 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:31:39.487 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:53.895968+0000", 2024-09-14T06:31:39.487 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.002746746 2024-09-14T06:31:39.488 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:39.488 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:39.488 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:31:39.488 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:53.895988+0000", 2024-09-14T06:31:39.488 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.0227000000000001e-05 2024-09-14T06:31:39.488 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:39.488 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:39.488 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:31:39.488 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:53.896000+0000", 2024-09-14T06:31:39.488 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.2359e-05 2024-09-14T06:31:39.488 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:39.488 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:39.488 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:31:39.488 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:53.896084+0000", 2024-09-14T06:31:39.488 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.3659999999999995e-05 2024-09-14T06:31:39.489 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:39.489 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:39.489 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:31:39.489 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:53.896109+0000", 2024-09-14T06:31:39.489 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.5259999999999999e-05 2024-09-14T06:31:39.489 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:39.489 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:39.489 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-14T06:31:39.489 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:53.899704+0000", 2024-09-14T06:31:39.489 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0035942209999999999 2024-09-14T06:31:39.489 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:39.489 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:39.489 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:31:39.489 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:53.899757+0000", 2024-09-14T06:31:39.490 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.3291999999999997e-05 2024-09-14T06:31:39.490 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:39.490 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:31:39.490 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:39.490 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:39.490 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:39.490 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4397.0:78263 3.0 e77/23 3:00899b22:::rbd_data.112ddfbec471.0000000000000016:head v 77'104, mlcod=73'102)", 2024-09-14T06:31:39.490 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:57.634578+0000", 2024-09-14T06:31:39.490 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 101.83577740299999, 2024-09-14T06:31:39.490 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.002372454, 2024-09-14T06:31:39.490 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:31:39.490 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:31:39.490 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:31:39.490 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:31:39.491 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:39.491 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:31:39.491 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:57.634578+0000", 2024-09-14T06:31:39.491 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:39.491 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:39.491 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:39.491 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:31:39.491 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:57.634578+0000", 2024-09-14T06:31:39.491 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:39.491 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:39.491 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:39.491 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:31:39.491 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:57.634578+0000", 2024-09-14T06:31:39.491 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.5699999999999998e-07 2024-09-14T06:31:39.492 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:39.492 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:39.492 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:31:39.492 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:57.635347+0000", 2024-09-14T06:31:39.492 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00076848300000000003 2024-09-14T06:31:39.492 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:39.492 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:39.492 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:31:39.492 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:57.635361+0000", 2024-09-14T06:31:39.492 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.4238e-05 2024-09-14T06:31:39.492 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:39.492 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:39.492 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:31:39.492 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:57.635371+0000", 2024-09-14T06:31:39.492 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.0087e-05 2024-09-14T06:31:39.493 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:39.493 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:39.493 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:31:39.493 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:57.635439+0000", 2024-09-14T06:31:39.493 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.7953000000000005e-05 2024-09-14T06:31:39.493 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:39.493 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:39.493 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:31:39.493 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:57.635464+0000", 2024-09-14T06:31:39.493 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.5062000000000002e-05 2024-09-14T06:31:39.493 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:39.493 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:39.493 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-14T06:31:39.493 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:57.636914+0000", 2024-09-14T06:31:39.494 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.001449625 2024-09-14T06:31:39.494 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:39.494 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:39.494 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:31:39.494 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:57.636950+0000", 2024-09-14T06:31:39.494 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.6548999999999999e-05 2024-09-14T06:31:39.494 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:39.494 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:31:39.494 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:39.494 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:39.494 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:39.494 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4397.0:78604 3.9 e77/23 3:94c62a0e:::rbd_data.112ddfbec471.000000000000002c:head v 77'96, mlcod=73'94)", 2024-09-14T06:31:39.494 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:57.677568+0000", 2024-09-14T06:31:39.494 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 101.792787015, 2024-09-14T06:31:39.495 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.002564846, 2024-09-14T06:31:39.495 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:31:39.495 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:31:39.495 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:31:39.495 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:31:39.495 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:39.495 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:31:39.495 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:57.677568+0000", 2024-09-14T06:31:39.495 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:39.495 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:39.495 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:39.495 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:31:39.495 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:57.677568+0000", 2024-09-14T06:31:39.495 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:39.495 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:39.496 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:39.496 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:31:39.496 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:57.677568+0000", 2024-09-14T06:31:39.496 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.8900000000000001e-07 2024-09-14T06:31:39.496 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:39.496 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:39.496 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:31:39.496 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:57.678488+0000", 2024-09-14T06:31:39.496 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00091963899999999998 2024-09-14T06:31:39.496 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:39.496 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:39.496 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:31:39.496 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:57.678500+0000", 2024-09-14T06:31:39.496 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.1902000000000001e-05 2024-09-14T06:31:39.497 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:39.497 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:39.497 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:31:39.497 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:57.678507+0000", 2024-09-14T06:31:39.497 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.7739999999999999e-06 2024-09-14T06:31:39.497 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:39.497 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:39.497 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:31:39.497 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:57.678586+0000", 2024-09-14T06:31:39.497 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.9103000000000005e-05 2024-09-14T06:31:39.497 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:39.497 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:39.497 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:31:39.498 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:57.678614+0000", 2024-09-14T06:31:39.498 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.7912e-05 2024-09-14T06:31:39.498 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:39.498 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:39.498 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-14T06:31:39.498 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:57.680075+0000", 2024-09-14T06:31:39.498 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0014613370000000001 2024-09-14T06:31:39.498 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:39.498 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:39.498 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:31:39.498 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:57.680133+0000", 2024-09-14T06:31:39.498 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.7890000000000003e-05 2024-09-14T06:31:39.498 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:39.498 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:31:39.498 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:39.499 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:39.499 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:39.499 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4397.0:149885 3.9 e122/23 3:951acef3:::rbd_data.112d1bb7b269.0000000000000005:head v 122'176, mlcod=122'172)", 2024-09-14T06:31:39.499 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:30:44.739092+0000", 2024-09-14T06:31:39.499 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 54.731262704999999, 2024-09-14T06:31:39.499 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0028236659999999998, 2024-09-14T06:31:39.499 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:31:39.499 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:31:39.499 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:31:39.499 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:31:39.499 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:39.499 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:31:39.499 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.739092+0000", 2024-09-14T06:31:39.499 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:39.499 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:39.500 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:39.500 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:31:39.500 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.739092+0000", 2024-09-14T06:31:39.500 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:39.500 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:39.500 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:39.500 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:31:39.500 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.739093+0000", 2024-09-14T06:31:39.500 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.0109999999999999e-06 2024-09-14T06:31:39.500 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:39.500 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:39.500 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:31:39.500 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.740189+0000", 2024-09-14T06:31:39.501 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00109527 2024-09-14T06:31:39.501 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:39.501 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:39.501 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:31:39.501 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.740199+0000", 2024-09-14T06:31:39.501 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.0458999999999999e-05 2024-09-14T06:31:39.501 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:39.501 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:39.501 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:31:39.501 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.740207+0000", 2024-09-14T06:31:39.501 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.4499999999999998e-06 2024-09-14T06:31:39.501 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:39.501 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:39.501 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:31:39.501 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.740295+0000", 2024-09-14T06:31:39.502 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.8740999999999996e-05 2024-09-14T06:31:39.502 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:39.502 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:39.502 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:31:39.502 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.740317+0000", 2024-09-14T06:31:39.502 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.1523e-05 2024-09-14T06:31:39.502 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:39.502 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:39.502 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-14T06:31:39.502 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.741865+0000", 2024-09-14T06:31:39.502 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.001548315 2024-09-14T06:31:39.502 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:39.502 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:39.502 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:31:39.502 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.741916+0000", 2024-09-14T06:31:39.502 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.0896999999999997e-05 2024-09-14T06:31:39.502 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:39.503 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:31:39.503 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:39.503 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:39.503 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:39.503 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4397.0:149933 3.4 e122/23 3:234f5229:::rbd_data.112d1bb7b269.0000000000000009:head v 122'221, mlcod=120'218)", 2024-09-14T06:31:39.503 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:30:44.743849+0000", 2024-09-14T06:31:39.503 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 54.726506288000003, 2024-09-14T06:31:39.503 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0031479519999999999, 2024-09-14T06:31:39.503 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:31:39.503 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:31:39.503 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:31:39.503 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:31:39.503 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:39.503 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:31:39.504 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.743849+0000", 2024-09-14T06:31:39.504 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:39.504 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:39.504 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:39.504 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:31:39.504 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.743849+0000", 2024-09-14T06:31:39.504 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:39.504 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:39.504 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:39.504 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:31:39.504 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.743850+0000", 2024-09-14T06:31:39.504 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.279e-06 2024-09-14T06:31:39.504 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:39.504 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:39.505 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:31:39.505 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.745151+0000", 2024-09-14T06:31:39.505 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.001300495 2024-09-14T06:31:39.505 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:39.505 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:39.505 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:31:39.505 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.745167+0000", 2024-09-14T06:31:39.505 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.6478e-05 2024-09-14T06:31:39.505 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:39.505 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:39.505 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:31:39.505 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.745175+0000", 2024-09-14T06:31:39.505 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.2679999999999994e-06 2024-09-14T06:31:39.505 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:39.506 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:39.506 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:31:39.506 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.745226+0000", 2024-09-14T06:31:39.506 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.0192999999999999e-05 2024-09-14T06:31:39.506 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:39.506 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:39.506 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:31:39.506 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.745247+0000", 2024-09-14T06:31:39.506 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.1767e-05 2024-09-14T06:31:39.506 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:39.506 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:39.506 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-14T06:31:39.506 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.746952+0000", 2024-09-14T06:31:39.506 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0017042769999999999 2024-09-14T06:31:39.506 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:39.506 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:39.507 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:31:39.507 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.746997+0000", 2024-09-14T06:31:39.507 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.5194999999999997e-05 2024-09-14T06:31:39.507 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:39.507 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:31:39.507 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:39.507 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:39.507 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:39.507 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4397.0:150494 3.9 e122/23 3:955152a4:::rbd_object_map.112d1bb7b269.0000000000000038:head v 122'194, mlcod=122'190)", 2024-09-14T06:31:39.507 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:30:44.781585+0000", 2024-09-14T06:31:39.507 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 54.688770396000002, 2024-09-14T06:31:39.507 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0026957499999999998, 2024-09-14T06:31:39.507 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:31:39.507 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:31:39.507 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:31:39.507 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:31:39.508 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:39.508 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:31:39.508 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.781585+0000", 2024-09-14T06:31:39.508 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:39.508 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:39.508 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:39.508 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:31:39.508 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.781585+0000", 2024-09-14T06:31:39.508 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:39.508 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:39.508 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:39.508 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:31:39.508 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.781585+0000", 2024-09-14T06:31:39.508 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.5000000000000002e-07 2024-09-14T06:31:39.508 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:39.509 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:39.509 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:31:39.509 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.781594+0000", 2024-09-14T06:31:39.509 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.4640000000000006e-06 2024-09-14T06:31:39.509 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:39.509 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:39.509 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:31:39.509 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.781604+0000", 2024-09-14T06:31:39.509 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.0071999999999999e-05 2024-09-14T06:31:39.509 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:39.509 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:39.509 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:31:39.509 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.781610+0000", 2024-09-14T06:31:39.509 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.3609999999999997e-06 2024-09-14T06:31:39.509 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:39.510 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:39.510 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:31:39.510 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.781676+0000", 2024-09-14T06:31:39.510 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.5963000000000005e-05 2024-09-14T06:31:39.510 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:39.510 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:39.510 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:31:39.510 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.781696+0000", 2024-09-14T06:31:39.510 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.9278000000000001e-05 2024-09-14T06:31:39.510 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:39.510 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:39.510 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-14T06:31:39.510 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.784231+0000", 2024-09-14T06:31:39.510 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.002535093 2024-09-14T06:31:39.510 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:39.510 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:39.511 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:31:39.511 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.784280+0000", 2024-09-14T06:31:39.511 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.9869e-05 2024-09-14T06:31:39.511 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:39.511 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:31:39.511 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:39.511 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:39.511 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:39.511 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4397.0:150898 3.9 e122/23 3:97c98f90:::rbd_data.112d1bb7b269.0000000000000025:head v 122'206, mlcod=122'203)", 2024-09-14T06:31:39.511 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:30:44.807232+0000", 2024-09-14T06:31:39.511 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 54.663123478999999, 2024-09-14T06:31:39.511 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0022297889999999998, 2024-09-14T06:31:39.511 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:31:39.511 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:31:39.511 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:31:39.511 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:31:39.512 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:39.512 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:31:39.512 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.807232+0000", 2024-09-14T06:31:39.512 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:39.512 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:39.512 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:39.512 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:31:39.512 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.807232+0000", 2024-09-14T06:31:39.512 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:39.512 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:39.512 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:39.512 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:31:39.512 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.807233+0000", 2024-09-14T06:31:39.512 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.5710000000000001e-06 2024-09-14T06:31:39.512 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:39.513 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:39.513 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:31:39.513 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.807944+0000", 2024-09-14T06:31:39.513 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00071056299999999999 2024-09-14T06:31:39.513 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:39.513 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:39.513 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:31:39.513 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.807961+0000", 2024-09-14T06:31:39.513 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.7689e-05 2024-09-14T06:31:39.513 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:39.513 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:39.513 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:31:39.513 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.807971+0000", 2024-09-14T06:31:39.514 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.2e-06 2024-09-14T06:31:39.514 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:39.514 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:39.514 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:31:39.514 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.808050+0000", 2024-09-14T06:31:39.514 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.9308999999999996e-05 2024-09-14T06:31:39.514 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:39.514 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:39.514 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:31:39.514 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.808074+0000", 2024-09-14T06:31:39.514 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.4147000000000001e-05 2024-09-14T06:31:39.514 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:39.514 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:39.514 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-14T06:31:39.515 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.809415+0000", 2024-09-14T06:31:39.515 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.001341257 2024-09-14T06:31:39.515 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:39.515 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:39.515 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:31:39.515 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.809461+0000", 2024-09-14T06:31:39.515 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.6053000000000001e-05 2024-09-14T06:31:39.515 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:39.515 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:31:39.515 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:39.515 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:39.515 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:39.515 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4397.0:202551 3.9 e166/23 3:965c09b6:::rbd_data.112d3bd84cb0.000000000000000a:head v 166'416, mlcod=164'414)", 2024-09-14T06:31:39.516 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:31:30.657011+0000", 2024-09-14T06:31:39.516 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 8.8133444460000003, 2024-09-14T06:31:39.516 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.002509262, 2024-09-14T06:31:39.516 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:31:39.516 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:31:39.516 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:31:39.516 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:31:39.516 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:39.516 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:31:39.516 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:30.657011+0000", 2024-09-14T06:31:39.516 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:39.516 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:39.517 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:39.517 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:31:39.517 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:30.657011+0000", 2024-09-14T06:31:39.517 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:39.517 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:39.517 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:39.517 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:31:39.517 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:30.657011+0000", 2024-09-14T06:31:39.517 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.08e-07 2024-09-14T06:31:39.517 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:39.517 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:39.517 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:31:39.517 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:30.658467+0000", 2024-09-14T06:31:39.517 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.001455708 2024-09-14T06:31:39.517 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:39.518 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:39.518 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:31:39.518 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:30.658484+0000", 2024-09-14T06:31:39.518 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.6844e-05 2024-09-14T06:31:39.518 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:39.518 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:39.518 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:31:39.518 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:30.658493+0000", 2024-09-14T06:31:39.518 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.2790000000000004e-06 2024-09-14T06:31:39.518 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:39.518 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:39.518 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:31:39.518 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:30.658559+0000", 2024-09-14T06:31:39.518 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.6459999999999997e-05 2024-09-14T06:31:39.518 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:39.519 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:39.519 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:31:39.519 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:30.658574+0000", 2024-09-14T06:31:39.519 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.5048000000000001e-05 2024-09-14T06:31:39.519 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:39.519 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:39.519 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-14T06:31:39.519 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:30.659473+0000", 2024-09-14T06:31:39.519 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00089842800000000005 2024-09-14T06:31:39.519 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:39.519 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:39.519 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:31:39.519 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:30.659520+0000", 2024-09-14T06:31:39.519 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.7086999999999998e-05 2024-09-14T06:31:39.519 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:39.520 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:31:39.520 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:39.520 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:39.520 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:39.520 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4397.0:202665 3.0 e166/23 3:00eeadfb:::rbd_data.112d3bd84cb0.000000000000000e:head v 166'920, mlcod=163'918)", 2024-09-14T06:31:39.520 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:31:30.662434+0000", 2024-09-14T06:31:39.520 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 8.8079206570000004, 2024-09-14T06:31:39.520 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.002522121, 2024-09-14T06:31:39.520 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:31:39.520 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:31:39.520 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:31:39.520 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:31:39.520 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:39.520 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:31:39.520 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:30.662434+0000", 2024-09-14T06:31:39.521 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:39.521 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:39.521 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:39.521 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:31:39.521 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:30.662434+0000", 2024-09-14T06:31:39.521 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:39.521 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:39.521 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:39.521 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:31:39.521 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:30.662436+0000", 2024-09-14T06:31:39.521 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.257e-06 2024-09-14T06:31:39.521 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:39.521 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:39.521 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:31:39.522 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:30.662527+0000", 2024-09-14T06:31:39.522 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.1644000000000006e-05 2024-09-14T06:31:39.522 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:39.522 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:39.522 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:31:39.522 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:30.662546+0000", 2024-09-14T06:31:39.522 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.8998e-05 2024-09-14T06:31:39.522 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:39.522 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:39.522 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:31:39.522 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:30.662556+0000", 2024-09-14T06:31:39.522 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.9809999999999998e-06 2024-09-14T06:31:39.522 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:39.522 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:39.522 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:31:39.522 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:30.662621+0000", 2024-09-14T06:31:39.522 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.4597000000000005e-05 2024-09-14T06:31:39.523 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:39.523 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:39.523 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:31:39.523 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:30.662643+0000", 2024-09-14T06:31:39.523 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.1608e-05 2024-09-14T06:31:39.523 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:39.523 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:39.523 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-14T06:31:39.523 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:30.664909+0000", 2024-09-14T06:31:39.523 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0022668689999999999 2024-09-14T06:31:39.523 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:39.523 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:39.523 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:31:39.524 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:30.664957+0000", 2024-09-14T06:31:39.524 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.7166999999999998e-05 2024-09-14T06:31:39.524 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:39.524 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:31:39.524 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:39.524 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:39.524 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:39.524 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4397.0:204589 3.4 e166/23 3:234552f0:::rbd_data.112d3bd84cb0.000000000000003f:head v 166'499, mlcod=165'497)", 2024-09-14T06:31:39.524 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:31:30.772894+0000", 2024-09-14T06:31:39.524 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 8.6974608010000001, 2024-09-14T06:31:39.524 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0034564650000000001, 2024-09-14T06:31:39.524 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:31:39.524 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:31:39.524 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:31:39.524 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:31:39.525 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:39.525 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:31:39.525 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:30.772894+0000", 2024-09-14T06:31:39.525 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:39.525 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:39.525 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:39.525 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:31:39.525 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:30.772894+0000", 2024-09-14T06:31:39.525 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:39.525 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:39.525 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:39.525 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:31:39.525 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:30.772896+0000", 2024-09-14T06:31:39.526 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.5379999999999999e-06 2024-09-14T06:31:39.526 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:39.526 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:39.526 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:31:39.526 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:30.773042+0000", 2024-09-14T06:31:39.526 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000146107 2024-09-14T06:31:39.526 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:39.526 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:39.526 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:31:39.526 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:30.773061+0000", 2024-09-14T06:31:39.526 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.8862999999999999e-05 2024-09-14T06:31:39.526 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:39.526 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:39.527 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:31:39.527 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:30.773071+0000", 2024-09-14T06:31:39.527 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.0227000000000001e-05 2024-09-14T06:31:39.527 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:39.527 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:39.527 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:31:39.527 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:30.773149+0000", 2024-09-14T06:31:39.527 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.8071999999999994e-05 2024-09-14T06:31:39.527 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:39.527 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:39.527 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:31:39.527 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:30.773171+0000", 2024-09-14T06:31:39.527 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.1699999999999999e-05 2024-09-14T06:31:39.527 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:39.528 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:39.528 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-14T06:31:39.528 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:30.776285+0000", 2024-09-14T06:31:39.528 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0031142050000000001 2024-09-14T06:31:39.528 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:39.528 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:39.528 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:31:39.528 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:30.776351+0000", 2024-09-14T06:31:39.528 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.5753000000000006e-05 2024-09-14T06:31:39.528 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:39.528 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:31:39.528 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:39.528 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:39.528 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:39.529 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4397.0:204598 3.4 e166/23 3:245d069e:::rbd_data.112d3bd84cb0.0000000000000041:head v 166'500, mlcod=165'497)", 2024-09-14T06:31:39.529 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:31:30.775247+0000", 2024-09-14T06:31:39.529 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 8.6951076779999994, 2024-09-14T06:31:39.529 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00318235, 2024-09-14T06:31:39.529 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:31:39.529 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:31:39.529 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:31:39.529 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:31:39.529 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:39.529 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:31:39.529 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:30.775247+0000", 2024-09-14T06:31:39.529 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:39.529 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:39.529 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:39.530 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:31:39.530 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:30.775247+0000", 2024-09-14T06:31:39.530 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:39.530 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:39.530 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:39.530 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:31:39.530 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:30.775249+0000", 2024-09-14T06:31:39.530 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.265e-06 2024-09-14T06:31:39.530 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:39.530 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:39.530 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:31:39.530 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:30.775340+0000", 2024-09-14T06:31:39.530 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.1131999999999994e-05 2024-09-14T06:31:39.531 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:39.531 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:39.531 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:31:39.531 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:30.775352+0000", 2024-09-14T06:31:39.531 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.2294e-05 2024-09-14T06:31:39.531 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:39.531 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:39.531 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:31:39.531 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:30.775361+0000", 2024-09-14T06:31:39.531 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.6950000000000006e-06 2024-09-14T06:31:39.531 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:39.531 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:39.531 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:31:39.531 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:30.775439+0000", 2024-09-14T06:31:39.532 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.7910000000000005e-05 2024-09-14T06:31:39.532 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:39.532 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:39.532 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:31:39.532 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:30.775460+0000", 2024-09-14T06:31:39.532 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.0900999999999999e-05 2024-09-14T06:31:39.532 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:39.532 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:39.532 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-14T06:31:39.532 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:30.778385+0000", 2024-09-14T06:31:39.532 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.002925539 2024-09-14T06:31:39.532 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:39.532 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:39.532 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:31:39.533 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:30.778430+0000", 2024-09-14T06:31:39.533 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.4613999999999997e-05 2024-09-14T06:31:39.533 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:39.533 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:31:39.533 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:39.533 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:39.533 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:39.533 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4397.0:204975 3.4 e166/23 3:20972d18:::rbd_data.112d3bd84cb0.0000000000000046:head v 166'501, mlcod=166'499)", 2024-09-14T06:31:39.533 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:31:30.787253+0000", 2024-09-14T06:31:39.533 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 8.6831016489999993, 2024-09-14T06:31:39.533 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.002248318, 2024-09-14T06:31:39.533 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:31:39.533 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:31:39.533 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:31:39.533 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:31:39.534 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:39.534 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:31:39.534 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:30.787253+0000", 2024-09-14T06:31:39.534 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:39.534 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:39.534 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:39.534 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:31:39.534 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:30.787253+0000", 2024-09-14T06:31:39.534 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:39.535 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:39.535 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:39.535 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:31:39.535 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:30.787255+0000", 2024-09-14T06:31:39.535 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.271e-06 2024-09-14T06:31:39.535 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:39.535 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:39.535 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:31:39.535 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:30.787442+0000", 2024-09-14T06:31:39.535 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00018744000000000001 2024-09-14T06:31:39.535 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:39.535 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:39.535 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:31:39.535 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:30.787461+0000", 2024-09-14T06:31:39.536 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.9054000000000001e-05 2024-09-14T06:31:39.536 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:39.536 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:39.536 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:31:39.536 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:30.787470+0000", 2024-09-14T06:31:39.536 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.0929999999999992e-06 2024-09-14T06:31:39.536 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:39.536 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:39.536 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:31:39.536 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:30.788004+0000", 2024-09-14T06:31:39.537 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00053359399999999997 2024-09-14T06:31:39.537 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:39.537 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:39.537 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:31:39.537 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:30.788034+0000", 2024-09-14T06:31:39.537 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.9635e-05 2024-09-14T06:31:39.537 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:39.537 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:39.537 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-14T06:31:39.537 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:30.789457+0000", 2024-09-14T06:31:39.537 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.001423487 2024-09-14T06:31:39.537 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:39.537 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:39.537 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:31:39.538 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:30.789502+0000", 2024-09-14T06:31:39.538 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.4743999999999997e-05 2024-09-14T06:31:39.538 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:39.538 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:31:39.538 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:39.538 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:39.538 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:39.538 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4397.0:212188 3.4 e166/23 3:23871bc7:::rbd_data.112d3bd84cb0.00000000000000fa:head v 166'508, mlcod=166'506)", 2024-09-14T06:31:39.538 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:31:31.391415+0000", 2024-09-14T06:31:39.538 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 8.0789400669999996, 2024-09-14T06:31:39.538 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0022039049999999999, 2024-09-14T06:31:39.538 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:31:39.538 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:31:39.538 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:31:39.539 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:31:39.539 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:39.539 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:31:39.539 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:31.391415+0000", 2024-09-14T06:31:39.539 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:39.539 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:39.539 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:39.539 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:31:39.539 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:31.391415+0000", 2024-09-14T06:31:39.539 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:39.539 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:39.539 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:39.539 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:31:39.539 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:31.391416+0000", 2024-09-14T06:31:39.539 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.44e-06 2024-09-14T06:31:39.540 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:39.540 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:39.540 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:31:39.540 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:31.391554+0000", 2024-09-14T06:31:39.540 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000137119 2024-09-14T06:31:39.540 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:39.540 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:39.540 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:31:39.540 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:31.391570+0000", 2024-09-14T06:31:39.540 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.6827e-05 2024-09-14T06:31:39.540 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:39.540 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:39.540 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:31:39.541 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:31.391580+0000", 2024-09-14T06:31:39.541 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.3139999999999993e-06 2024-09-14T06:31:39.541 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:39.541 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:39.541 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:31:39.541 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:31.391664+0000", 2024-09-14T06:31:39.541 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.3757999999999993e-05 2024-09-14T06:31:39.541 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:39.541 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:39.541 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:31:39.541 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:31.391692+0000", 2024-09-14T06:31:39.541 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.8898e-05 2024-09-14T06:31:39.541 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:39.541 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:39.542 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-14T06:31:39.542 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:31.393584+0000", 2024-09-14T06:31:39.542 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.001891752 2024-09-14T06:31:39.542 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:39.542 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:39.542 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:31:39.542 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:31.393619+0000", 2024-09-14T06:31:39.542 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.4796999999999999e-05 2024-09-14T06:31:39.542 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:39.542 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:31:39.542 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:39.542 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:39.542 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:39.542 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "MOSDPGPush(3.5 169/168 [PushOp(3:a7c12b08:::rbd_data.112ddfbec471.000000000000000e:head, version: 77'100, data_included: [0~1048576], data_size: 1048576, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(3:a7c12b08:::rbd_data.112ddfbec471.000000000000000e:head@77'100, size: 1048576, copy_subset: [(0, 1048576)], clone_subset: {}, snapset: 0=[]:{}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 1048576, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2024-09-14T06:31:39.543 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:31:34.540707+0000", 2024-09-14T06:31:39.543 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 4.9296483020000004, 2024-09-14T06:31:39.543 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0023637369999999999, 2024-09-14T06:31:39.543 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:31:39.543 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:31:39.543 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:31:39.543 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:31:39.543 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:39.543 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:31:39.543 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:34.540707+0000", 2024-09-14T06:31:39.543 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:39.543 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:39.543 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:39.543 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:31:39.543 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:34.540707+0000", 2024-09-14T06:31:39.543 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:39.544 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:39.544 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:39.544 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:31:39.544 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:34.540708+0000", 2024-09-14T06:31:39.544 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.7199999999999998e-07 2024-09-14T06:31:39.544 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:39.544 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:39.544 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:31:39.544 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:34.542150+0000", 2024-09-14T06:31:39.544 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0014423229999999999 2024-09-14T06:31:39.544 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:39.544 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:39.544 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:31:39.544 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:34.542169+0000", 2024-09-14T06:31:39.544 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.8756000000000001e-05 2024-09-14T06:31:39.545 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:39.545 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:39.545 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:31:39.545 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:34.542180+0000", 2024-09-14T06:31:39.545 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.1051e-05 2024-09-14T06:31:39.545 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:39.545 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:39.545 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:31:39.545 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:34.542275+0000", 2024-09-14T06:31:39.545 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.5122000000000002e-05 2024-09-14T06:31:39.545 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:39.545 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:39.545 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:31:39.545 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:34.542284+0000", 2024-09-14T06:31:39.546 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.9190000000000005e-06 2024-09-14T06:31:39.546 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:39.546 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:39.546 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:31:39.546 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:34.543071+0000", 2024-09-14T06:31:39.546 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00078679399999999995 2024-09-14T06:31:39.546 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:39.546 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:31:39.546 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:39.546 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:39.546 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:39.546 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "MOSDPGPush(3.5 169/168 [PushOp(3:a5c8b3f2:::rbd_data.112ddfbec471.0000000000000018:head, version: 77'101, data_included: [0~1048576], data_size: 1048576, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(3:a5c8b3f2:::rbd_data.112ddfbec471.0000000000000018:head@77'101, size: 1048576, copy_subset: [(0, 1048576)], clone_subset: {}, snapset: 0=[]:{}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 1048576, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2024-09-14T06:31:39.546 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:31:34.545533+0000", 2024-09-14T06:31:39.547 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 4.9248224149999995, 2024-09-14T06:31:39.547 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0021867010000000001, 2024-09-14T06:31:39.547 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:31:39.547 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:31:39.547 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:31:39.547 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:31:39.547 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:39.547 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:31:39.547 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:34.545533+0000", 2024-09-14T06:31:39.547 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:39.547 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:39.547 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:39.547 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:31:39.547 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:34.545533+0000", 2024-09-14T06:31:39.547 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:39.548 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:39.548 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:39.548 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:31:39.548 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:34.545533+0000", 2024-09-14T06:31:39.548 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.8100000000000002e-07 2024-09-14T06:31:39.548 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:39.548 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:39.548 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:31:39.548 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:34.546784+0000", 2024-09-14T06:31:39.548 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.001250747 2024-09-14T06:31:39.548 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:39.548 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:39.548 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:31:39.548 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:34.546803+0000", 2024-09-14T06:31:39.548 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.8346e-05 2024-09-14T06:31:39.549 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:39.549 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:39.549 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:31:39.549 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:34.546823+0000", 2024-09-14T06:31:39.549 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.0699999999999998e-05 2024-09-14T06:31:39.549 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:39.549 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:39.549 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:31:39.549 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:34.546890+0000", 2024-09-14T06:31:39.549 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.6581000000000005e-05 2024-09-14T06:31:39.549 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:39.549 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:39.549 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:31:39.549 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:34.546899+0000", 2024-09-14T06:31:39.549 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.3889999999999994e-06 2024-09-14T06:31:39.550 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:39.550 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:39.550 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:31:39.550 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:34.547719+0000", 2024-09-14T06:31:39.550 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00082015699999999996 2024-09-14T06:31:39.550 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:39.550 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:31:39.550 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:39.550 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:39.550 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:39.550 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "MOSDPGPush(3.5 169/168 [PushOp(3:a7a54476:::rbd_data.112d1bb7b269.0000000000000038:head, version: 122'581, data_included: [0~524288], data_size: 524288, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(3:a7a54476:::rbd_data.112d1bb7b269.0000000000000038:head@122'581, size: 524288, copy_subset: [(0, 524288)], clone_subset: {}, snapset: 0=[]:{}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 524288, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2024-09-14T06:31:39.550 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:31:34.667757+0000", 2024-09-14T06:31:39.550 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 4.8025982559999996, 2024-09-14T06:31:39.550 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.002750382, 2024-09-14T06:31:39.550 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:31:39.550 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:31:39.551 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:31:39.551 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:31:39.551 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:39.551 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:31:39.551 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:34.667757+0000", 2024-09-14T06:31:39.551 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:39.551 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:39.551 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:39.551 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:31:39.551 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:34.667757+0000", 2024-09-14T06:31:39.551 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:39.551 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:39.551 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:39.551 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:31:39.551 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:34.667758+0000", 2024-09-14T06:31:39.552 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.3599999999999999e-06 2024-09-14T06:31:39.552 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:39.552 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:39.552 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:31:39.552 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:34.669375+0000", 2024-09-14T06:31:39.552 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.001616644 2024-09-14T06:31:39.552 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:39.552 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:39.552 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:31:39.552 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:34.669404+0000", 2024-09-14T06:31:39.552 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.9300999999999999e-05 2024-09-14T06:31:39.552 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:39.552 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:39.552 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:31:39.553 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:34.669428+0000", 2024-09-14T06:31:39.553 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.3788e-05 2024-09-14T06:31:39.553 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:39.553 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:39.553 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:31:39.553 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:34.669543+0000", 2024-09-14T06:31:39.553 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000115462 2024-09-14T06:31:39.553 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:39.553 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:39.553 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:31:39.553 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:34.669559+0000", 2024-09-14T06:31:39.553 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.5167999999999999e-05 2024-09-14T06:31:39.553 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:39.553 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:39.553 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:31:39.554 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:34.670507+0000", 2024-09-14T06:31:39.554 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00094865900000000003 2024-09-14T06:31:39.554 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:39.554 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:31:39.554 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:39.554 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:39.554 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:39.554 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "MOSDPGPush(3.5 170/168 [PushOp(3:a162138c:::rbd_data.112d1bb7b269.000000000000003c:head, version: 122'582, data_included: [0~524288], data_size: 524288, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(3:a162138c:::rbd_data.112d1bb7b269.000000000000003c:head@122'582, size: 524288, copy_subset: [(0, 524288)], clone_subset: {}, snapset: 0=[]:{}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 524288, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2024-09-14T06:31:39.554 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:31:34.670110+0000", 2024-09-14T06:31:39.554 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 4.800245307, 2024-09-14T06:31:39.554 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0030066229999999999, 2024-09-14T06:31:39.554 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:31:39.554 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:31:39.554 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:31:39.555 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:31:39.555 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:39.555 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:31:39.555 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:34.670110+0000", 2024-09-14T06:31:39.555 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:39.555 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:39.555 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:39.555 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:31:39.555 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:34.670110+0000", 2024-09-14T06:31:39.555 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:39.555 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:39.555 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:39.555 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:31:39.555 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:34.670111+0000", 2024-09-14T06:31:39.555 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.6499999999999998e-07 2024-09-14T06:31:39.556 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:39.556 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:39.556 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:31:39.556 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:34.671507+0000", 2024-09-14T06:31:39.556 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.001396284 2024-09-14T06:31:39.556 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:39.556 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:39.556 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:31:39.556 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:34.671530+0000", 2024-09-14T06:31:39.556 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.3023e-05 2024-09-14T06:31:39.556 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:39.556 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:39.556 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:31:39.556 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:34.671548+0000", 2024-09-14T06:31:39.556 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.8116000000000001e-05 2024-09-14T06:31:39.556 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:39.557 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:39.557 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:31:39.557 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:34.671775+0000", 2024-09-14T06:31:39.557 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00022699600000000001 2024-09-14T06:31:39.557 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:39.557 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:39.557 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:31:39.557 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:34.671790+0000", 2024-09-14T06:31:39.557 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.539e-05 2024-09-14T06:31:39.557 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:39.557 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:39.557 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:31:39.557 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:34.673116+0000", 2024-09-14T06:31:39.557 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0013259490000000001 2024-09-14T06:31:39.557 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:39.557 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:31:39.558 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:39.558 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:39.558 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:31:39.558 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-14T06:31:39.559 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-14T06:31:39.560 INFO:tasks.ceph.osd.6.smithi138.stderr:2024-09-14T06:31:39.558+0000 7f8cd8849640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33620) UID: 0 2024-09-14T06:31:39.660 INFO:tasks.ceph.osd.4.smithi138.stderr:2024-09-14T06:31:39.659+0000 7f6c653c8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33615) UID: 0 2024-09-14T06:31:39.662 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-14T06:31:39.663 INFO:teuthology.orchestra.run.smithi094.stdout: "ops": [], 2024-09-14T06:31:39.663 INFO:teuthology.orchestra.run.smithi094.stdout: "num_ops": 0 2024-09-14T06:31:39.663 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-14T06:31:39.672 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-14T06:31:39.674 INFO:teuthology.orchestra.run.smithi094.stderr:nodeep-scrub is set 2024-09-14T06:31:39.750 INFO:teuthology.orchestra.run.smithi094.stdout:5310 write 0xbafaf4b thru 0xbafe5f3 (0x36a9 bytes) 2024-09-14T06:31:39.760 INFO:tasks.ceph.osd.0.smithi094.stderr:2024-09-14T06:31:39.758+0000 7f07c9333640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33636) UID: 0 2024-09-14T06:31:39.766 INFO:teuthology.orchestra.run.smithi094.stdout:5312 punch from 0x69f7740 to 0x69fd74a, (0x600a bytes) 2024-09-14T06:31:39.766 INFO:teuthology.orchestra.run.smithi094.stdout:5313 punch from 0x456a0b3 to 0x456de11, (0x3d5e bytes) 2024-09-14T06:31:39.766 INFO:teuthology.orchestra.run.smithi094.stdout:5317 writesame 0x9cfa1a4 thru 0x9d0106b data_size 0xa(0x6ec8 bytes) 2024-09-14T06:31:39.776 INFO:teuthology.orchestra.run.smithi094.stdout:5321 read 0x4f7b6de thru 0x4f83b87 (0x84aa bytes) 2024-09-14T06:31:39.776 INFO:teuthology.orchestra.run.smithi094.stdout:5322 write 0xa770caf thru 0xa77331d (0x266f bytes) 2024-09-14T06:31:39.784 INFO:teuthology.orchestra.run.smithi094.stdout:5323 punch from 0x3b4170c to 0x3b4c501, (0xadf5 bytes) 2024-09-14T06:31:39.784 INFO:teuthology.orchestra.run.smithi094.stdout:5324 punch from 0x1fe0599 to 0x1fe50fa, (0x4b61 bytes) 2024-09-14T06:31:39.784 INFO:teuthology.orchestra.run.smithi094.stdout:5325 write 0x929766a thru 0x9298560 (0xef7 bytes) 2024-09-14T06:31:39.792 INFO:teuthology.orchestra.run.smithi094.stdout:5326 trunc from 0xbafe5f4 to 0xe97be8d 2024-09-14T06:31:39.797 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-14T06:31:39.797 INFO:teuthology.orchestra.run.smithi094.stdout: "ops": [], 2024-09-14T06:31:39.797 INFO:teuthology.orchestra.run.smithi094.stdout: "complaint_time": 30, 2024-09-14T06:31:39.797 INFO:teuthology.orchestra.run.smithi094.stdout: "num_blocked_ops": 0 2024-09-14T06:31:39.797 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-14T06:31:39.813 INFO:teuthology.orchestra.run.smithi094.stdout:5329 trunc from 0xe97be8d to 0x27397fc 2024-09-14T06:31:39.814 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-14T06:31:39.860 INFO:teuthology.orchestra.run.smithi094.stdout:5330 writesame 0xa49fe61 thru 0xa4a94cd data_size 0x1(0x966d bytes) 2024-09-14T06:31:39.860 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-14T06:31:39.860 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-14T06:31:39.860 INFO:teuthology.orchestra.run.smithi094.stdout:5333 read 0x7f5dd58 thru 0x7f67929 (0x9bd2 bytes) 2024-09-14T06:31:39.860 INFO:teuthology.orchestra.run.smithi094.stdout:5334 read 0x65b0cd6 thru 0x65bcb27 (0xbe52 bytes) 2024-09-14T06:31:39.861 INFO:teuthology.orchestra.run.smithi094.stdout:5335 trunc from 0xa4a94ce to 0xb08172c 2024-09-14T06:31:39.861 INFO:tasks.ceph.osd.1.smithi094.stderr:2024-09-14T06:31:39.859+0000 7f328e47a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33641) UID: 0 2024-09-14T06:31:39.889 INFO:teuthology.orchestra.run.smithi094.stdout:5336 trunc from 0xb08172c to 0x84158 2024-09-14T06:31:39.927 INFO:teuthology.orchestra.run.smithi094.stdout:5338 read 0x1629a thru 0x16504 (0x26b bytes) 2024-09-14T06:31:39.946 INFO:teuthology.orchestra.run.smithi094.stdout:5341 write 0x3cee6e6 thru 0x3cf0fa2 (0x28bd bytes) 2024-09-14T06:31:39.954 INFO:teuthology.orchestra.run.smithi094.stdout:5343 read 0x33a88c4 thru 0x33af79e (0x6edb bytes) 2024-09-14T06:31:39.954 INFO:teuthology.orchestra.run.smithi094.stdout:5345 punch from 0x2bc97 to 0x398c3, (0xdc2c bytes) 2024-09-14T06:31:39.954 INFO:teuthology.orchestra.run.smithi094.stdout:5346 writesame 0x2df78bc thru 0x2dfdfaa data_size 0x1(0x66ef bytes) 2024-09-14T06:31:39.962 INFO:tasks.ceph.osd.5.smithi138.stderr:2024-09-14T06:31:39.961+0000 7fe13746f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33618) UID: 0 2024-09-14T06:31:39.965 INFO:teuthology.orchestra.run.smithi094.stdout:5347 read 0x11c3adf thru 0x11cb993 (0x7eb5 bytes) 2024-09-14T06:31:39.965 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-14T06:31:39.981 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-14T06:31:39.981 INFO:teuthology.orchestra.run.smithi094.stdout: "size": 20, 2024-09-14T06:31:39.981 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 600, 2024-09-14T06:31:39.981 INFO:teuthology.orchestra.run.smithi094.stdout: "ops": [ 2024-09-14T06:31:39.981 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:39.981 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:44425 3.2 3:4603ad7f:::rbd_data.112dca2f6ae1.0000000000000005:17 [read 868352~4096 [fadvise_nocache]] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:31:39.981 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:43.854967+0000", 2024-09-14T06:31:39.982 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 116.12466625499999, 2024-09-14T06:31:39.982 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.063146025999999994, 2024-09-14T06:31:39.982 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:31:39.982 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:31:39.982 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:31:39.982 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:31:39.982 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:31:39.982 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:39.982 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 44425 2024-09-14T06:31:39.982 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:39.982 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:31:39.982 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:39.982 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:31:39.982 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.854967+0000", 2024-09-14T06:31:39.983 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:39.983 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:39.983 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:39.983 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:31:39.983 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.854967+0000", 2024-09-14T06:31:39.983 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:39.983 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:39.983 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:39.983 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:31:39.983 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.854967+0000", 2024-09-14T06:31:39.983 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.16e-07 2024-09-14T06:31:39.983 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:39.983 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:39.983 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:31:39.983 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.854968+0000", 2024-09-14T06:31:39.984 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.006e-06 2024-09-14T06:31:39.984 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:39.984 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:39.984 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:31:39.984 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.854971+0000", 2024-09-14T06:31:39.984 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.9359999999999999e-06 2024-09-14T06:31:39.984 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:39.984 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:39.984 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:31:39.984 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.854974+0000", 2024-09-14T06:31:39.984 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.1840000000000002e-06 2024-09-14T06:31:39.984 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:39.984 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:39.984 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:31:39.984 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.917945+0000", 2024-09-14T06:31:39.985 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.062970925999999997 2024-09-14T06:31:39.985 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:39.985 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:39.985 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:31:39.985 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.918030+0000", 2024-09-14T06:31:39.985 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.5863000000000001e-05 2024-09-14T06:31:39.985 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:39.985 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:39.985 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:31:39.985 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.918113+0000", 2024-09-14T06:31:39.985 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.2894999999999998e-05 2024-09-14T06:31:39.985 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:39.985 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:31:39.985 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:39.986 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:39.986 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:39.986 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:44426 3.2 3:4603ad7f:::rbd_data.112dca2f6ae1.0000000000000005:17 [read 892928~4096 [fadvise_nocache]] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:31:39.986 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:43.854977+0000", 2024-09-14T06:31:39.986 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 116.124656845, 2024-09-14T06:31:39.986 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.063369237999999994, 2024-09-14T06:31:39.986 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:31:39.986 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:31:39.986 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:31:39.986 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:31:39.986 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:31:39.986 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:39.986 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 44426 2024-09-14T06:31:39.986 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:39.986 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:31:39.987 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:39.987 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:31:39.987 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.854977+0000", 2024-09-14T06:31:39.987 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:39.987 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:39.987 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:39.987 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:31:39.987 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.854977+0000", 2024-09-14T06:31:39.987 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:39.987 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:39.987 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:39.987 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:31:39.987 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.854977+0000", 2024-09-14T06:31:39.987 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.1199999999999999e-07 2024-09-14T06:31:39.987 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:39.988 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:39.988 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:31:39.988 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.854980+0000", 2024-09-14T06:31:39.988 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.27e-06 2024-09-14T06:31:39.988 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:39.988 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:39.988 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:31:39.988 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.854983+0000", 2024-09-14T06:31:39.988 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.1650000000000002e-06 2024-09-14T06:31:39.988 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:39.988 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:39.988 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:31:39.988 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.854989+0000", 2024-09-14T06:31:39.988 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.6520000000000003e-06 2024-09-14T06:31:39.988 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:39.989 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:39.989 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:31:39.989 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.918143+0000", 2024-09-14T06:31:39.989 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.063154541999999994 2024-09-14T06:31:39.989 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:39.989 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:39.989 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:31:39.989 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.918209+0000", 2024-09-14T06:31:39.989 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.5529000000000002e-05 2024-09-14T06:31:39.989 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:39.989 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:39.989 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:31:39.989 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.918346+0000", 2024-09-14T06:31:39.989 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000136868 2024-09-14T06:31:39.990 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:39.990 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:31:39.990 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:39.990 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:39.990 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:39.990 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:44427 3.2 3:4603ad7f:::rbd_data.112dca2f6ae1.0000000000000005:17 [read 917504~4096 [fadvise_nocache]] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:31:39.990 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:43.854992+0000", 2024-09-14T06:31:39.990 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 116.124641199, 2024-09-14T06:31:39.990 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.063607792999999996, 2024-09-14T06:31:39.990 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:31:39.990 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:31:39.990 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:31:39.990 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:31:39.990 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:31:39.991 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:39.991 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 44427 2024-09-14T06:31:39.991 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:39.991 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:31:39.991 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:39.991 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:31:39.991 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.854992+0000", 2024-09-14T06:31:39.991 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:39.991 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:39.991 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:39.991 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:31:39.991 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.854992+0000", 2024-09-14T06:31:39.991 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:39.991 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:39.992 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:39.992 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:31:39.992 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.854993+0000", 2024-09-14T06:31:39.992 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.1400000000000001e-07 2024-09-14T06:31:39.992 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:39.992 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:39.992 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:31:39.992 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.854994+0000", 2024-09-14T06:31:39.992 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.091e-06 2024-09-14T06:31:39.992 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:39.992 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:39.992 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:31:39.992 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855001+0000", 2024-09-14T06:31:39.992 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.204e-06 2024-09-14T06:31:39.992 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:39.993 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:39.993 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:31:39.993 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855003+0000", 2024-09-14T06:31:39.993 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.514e-06 2024-09-14T06:31:39.993 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:39.993 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:39.993 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:31:39.993 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.918380+0000", 2024-09-14T06:31:39.993 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.063376738000000002 2024-09-14T06:31:39.993 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:39.993 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:39.993 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:31:39.993 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.918453+0000", 2024-09-14T06:31:39.994 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.3291e-05 2024-09-14T06:31:39.994 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:39.994 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:39.994 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:31:39.994 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.918600+0000", 2024-09-14T06:31:39.994 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00014674100000000001 2024-09-14T06:31:39.994 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:39.994 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:31:39.994 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:39.994 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:39.994 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:39.994 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:44428 3.2 3:4603ad7f:::rbd_data.112dca2f6ae1.0000000000000005:17 [read 942080~4096 [fadvise_nocache]] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:31:39.994 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:43.855006+0000", 2024-09-14T06:31:39.994 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 116.12462714900001, 2024-09-14T06:31:39.995 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.063846321999999997, 2024-09-14T06:31:39.995 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:31:39.995 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:31:39.995 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:31:39.995 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:31:39.995 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:31:39.995 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:39.995 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 44428 2024-09-14T06:31:39.995 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:39.995 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:31:39.995 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:39.995 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:31:39.995 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855006+0000", 2024-09-14T06:31:39.995 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:39.995 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:39.996 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:39.996 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:31:39.996 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855006+0000", 2024-09-14T06:31:39.996 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:39.996 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:39.996 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:39.996 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:31:39.996 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855007+0000", 2024-09-14T06:31:39.996 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.1199999999999999e-07 2024-09-14T06:31:39.996 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:39.996 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:39.996 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:31:39.996 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855011+0000", 2024-09-14T06:31:39.997 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.9539999999999998e-06 2024-09-14T06:31:39.997 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:39.997 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:39.997 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:31:39.997 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855014+0000", 2024-09-14T06:31:39.997 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.2930000000000001e-06 2024-09-14T06:31:39.997 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:39.997 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:39.997 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:31:39.997 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855016+0000", 2024-09-14T06:31:39.997 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.3039999999999999e-06 2024-09-14T06:31:39.997 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:39.997 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:39.998 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:31:39.998 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.918634+0000", 2024-09-14T06:31:39.998 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.063617928000000004 2024-09-14T06:31:39.998 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:39.998 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:39.998 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:31:39.998 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.918711+0000", 2024-09-14T06:31:39.998 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.6545000000000007e-05 2024-09-14T06:31:39.998 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:39.998 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:39.998 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:31:39.998 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.918853+0000", 2024-09-14T06:31:39.998 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00014208600000000001 2024-09-14T06:31:39.998 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:39.999 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:31:39.999 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:39.999 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:39.999 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:39.999 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:44429 3.2 3:4603ad7f:::rbd_data.112dca2f6ae1.0000000000000005:17 [read 966656~4096 [fadvise_nocache]] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:31:39.999 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:43.855019+0000", 2024-09-14T06:31:39.999 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 116.124614476, 2024-09-14T06:31:39.999 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.064059226999999996, 2024-09-14T06:31:39.999 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:31:39.999 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:31:39.999 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:31:39.999 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:31:39.999 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:31:39.999 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:40.000 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 44429 2024-09-14T06:31:40.000 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:40.000 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:31:40.000 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:40.000 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:31:40.000 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855019+0000", 2024-09-14T06:31:40.000 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:40.000 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:40.000 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:40.000 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:31:40.000 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855019+0000", 2024-09-14T06:31:40.000 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:40.000 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:40.000 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:40.001 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:31:40.001 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855019+0000", 2024-09-14T06:31:40.001 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.28e-07 2024-09-14T06:31:40.001 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:40.001 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:40.001 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:31:40.001 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855020+0000", 2024-09-14T06:31:40.001 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.0330000000000001e-06 2024-09-14T06:31:40.001 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:40.001 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:40.001 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:31:40.001 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855023+0000", 2024-09-14T06:31:40.001 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.79e-06 2024-09-14T06:31:40.001 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:40.002 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:40.002 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:31:40.002 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855025+0000", 2024-09-14T06:31:40.002 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.216e-06 2024-09-14T06:31:40.002 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:40.002 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:40.002 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:31:40.002 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.918886+0000", 2024-09-14T06:31:40.002 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.063861037999999995 2024-09-14T06:31:40.002 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:40.002 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:40.002 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:31:40.002 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.918952+0000", 2024-09-14T06:31:40.002 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.5993000000000003e-05 2024-09-14T06:31:40.002 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:40.003 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:40.003 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:31:40.003 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.919078+0000", 2024-09-14T06:31:40.003 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00012592899999999999 2024-09-14T06:31:40.003 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:40.003 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:31:40.003 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:40.003 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:40.003 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:40.003 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:44430 3.2 3:4603ad7f:::rbd_data.112dca2f6ae1.0000000000000005:17 [read 991232~4096 [fadvise_nocache]] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:31:40.003 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:43.855028+0000", 2024-09-14T06:31:40.003 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 116.12460546600001, 2024-09-14T06:31:40.003 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.064284346000000006, 2024-09-14T06:31:40.003 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:31:40.004 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:31:40.004 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:31:40.004 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:31:40.004 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:31:40.004 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:40.004 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 44430 2024-09-14T06:31:40.004 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:40.004 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:31:40.004 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:40.004 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:31:40.004 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855028+0000", 2024-09-14T06:31:40.004 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:40.004 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:40.005 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:40.005 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:31:40.005 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855028+0000", 2024-09-14T06:31:40.005 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:40.005 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:40.005 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:40.005 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:31:40.005 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855028+0000", 2024-09-14T06:31:40.005 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.1199999999999999e-07 2024-09-14T06:31:40.005 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:40.005 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:40.005 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:31:40.005 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855029+0000", 2024-09-14T06:31:40.005 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.1100000000000004e-07 2024-09-14T06:31:40.006 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:40.006 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:40.006 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:31:40.006 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855032+0000", 2024-09-14T06:31:40.006 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.7889999999999999e-06 2024-09-14T06:31:40.006 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:40.006 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:40.006 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:31:40.006 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855034+0000", 2024-09-14T06:31:40.006 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.125e-06 2024-09-14T06:31:40.006 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:40.006 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:40.006 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:31:40.006 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.919110+0000", 2024-09-14T06:31:40.006 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.064076413999999998 2024-09-14T06:31:40.007 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:40.007 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:40.007 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:31:40.007 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.919186+0000", 2024-09-14T06:31:40.007 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.5784999999999994e-05 2024-09-14T06:31:40.007 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:40.007 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:40.007 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:31:40.007 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.919312+0000", 2024-09-14T06:31:40.007 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00012611 2024-09-14T06:31:40.007 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:40.007 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:31:40.007 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:40.007 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:40.007 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:40.008 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:44431 3.2 3:4603ad7f:::rbd_data.112dca2f6ae1.0000000000000005:17 [read 1015808~4096 [fadvise_nocache]] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:31:40.008 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:43.855037+0000", 2024-09-14T06:31:40.008 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 116.124596598, 2024-09-14T06:31:40.008 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.064469755000000004, 2024-09-14T06:31:40.008 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:31:40.008 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:31:40.008 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:31:40.008 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:31:40.008 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:31:40.008 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:40.008 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 44431 2024-09-14T06:31:40.008 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:40.008 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:31:40.009 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:40.009 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:31:40.009 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855037+0000", 2024-09-14T06:31:40.009 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:40.009 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:40.009 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:40.009 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:31:40.009 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855037+0000", 2024-09-14T06:31:40.009 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:40.009 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:40.009 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:40.009 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:31:40.009 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855037+0000", 2024-09-14T06:31:40.009 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.1e-07 2024-09-14T06:31:40.009 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:40.009 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:40.010 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:31:40.010 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855038+0000", 2024-09-14T06:31:40.010 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.9400000000000004e-07 2024-09-14T06:31:40.010 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:40.010 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:40.010 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:31:40.010 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855041+0000", 2024-09-14T06:31:40.010 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.9639999999999999e-06 2024-09-14T06:31:40.010 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:40.010 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:40.010 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:31:40.010 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855043+0000", 2024-09-14T06:31:40.010 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.2259999999999999e-06 2024-09-14T06:31:40.010 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:40.010 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:40.011 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:31:40.011 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.919347+0000", 2024-09-14T06:31:40.011 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.064304091999999993 2024-09-14T06:31:40.011 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:40.011 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:40.011 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:31:40.011 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.919393+0000", 2024-09-14T06:31:40.011 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.5340999999999998e-05 2024-09-14T06:31:40.011 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:40.011 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:40.011 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:31:40.011 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.919507+0000", 2024-09-14T06:31:40.011 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000114028 2024-09-14T06:31:40.012 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:40.012 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:31:40.012 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:40.012 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:40.012 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:40.012 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:44432 3.2 3:4603ad7f:::rbd_data.112dca2f6ae1.0000000000000005:17 [read 1040384~4096 [fadvise_nocache]] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:31:40.012 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:43.855046+0000", 2024-09-14T06:31:40.012 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 116.12458753, 2024-09-14T06:31:40.012 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.064639253999999993, 2024-09-14T06:31:40.012 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:31:40.012 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:31:40.012 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:31:40.012 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:31:40.012 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:31:40.013 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:40.013 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 44432 2024-09-14T06:31:40.013 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:40.013 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:31:40.013 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:40.013 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:31:40.013 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855046+0000", 2024-09-14T06:31:40.013 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:40.013 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:40.013 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:40.013 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:31:40.013 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855046+0000", 2024-09-14T06:31:40.013 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:40.013 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:40.014 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:40.014 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:31:40.014 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855046+0000", 2024-09-14T06:31:40.014 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.05e-07 2024-09-14T06:31:40.014 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:40.014 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:40.014 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:31:40.014 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855047+0000", 2024-09-14T06:31:40.014 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.879999999999999e-07 2024-09-14T06:31:40.014 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:40.014 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:40.014 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:31:40.014 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855050+0000", 2024-09-14T06:31:40.014 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.8459999999999999e-06 2024-09-14T06:31:40.015 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:40.015 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:40.015 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:31:40.015 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855052+0000", 2024-09-14T06:31:40.015 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.136e-06 2024-09-14T06:31:40.015 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:40.015 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:40.015 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:31:40.015 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.919536+0000", 2024-09-14T06:31:40.015 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.064484132 2024-09-14T06:31:40.015 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:40.015 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:40.015 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:31:40.015 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.919611+0000", 2024-09-14T06:31:40.016 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.4407999999999996e-05 2024-09-14T06:31:40.016 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:40.016 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:40.016 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:31:40.016 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.919685+0000", 2024-09-14T06:31:40.016 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.4538999999999999e-05 2024-09-14T06:31:40.016 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:40.016 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:31:40.016 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:40.016 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:40.016 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:40.016 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:44518 3.2 3:4190d882:::rbd_data.112dca2f6ae1.0000000000000002:17 [read 12288~4096 [fadvise_nocache]] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:31:40.016 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:43.855055+0000", 2024-09-14T06:31:40.016 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 116.124578646, 2024-09-14T06:31:40.017 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.064894317000000007, 2024-09-14T06:31:40.017 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:31:40.017 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:31:40.017 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:31:40.017 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:31:40.017 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:31:40.017 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:40.017 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 44518 2024-09-14T06:31:40.017 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:40.017 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:31:40.017 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:40.017 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:31:40.017 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855055+0000", 2024-09-14T06:31:40.017 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:40.018 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:40.018 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:40.018 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:31:40.018 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855055+0000", 2024-09-14T06:31:40.018 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:40.018 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:40.018 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:40.018 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:31:40.018 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855055+0000", 2024-09-14T06:31:40.018 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.0699999999999999e-07 2024-09-14T06:31:40.018 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:40.018 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:40.018 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:31:40.018 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855056+0000", 2024-09-14T06:31:40.019 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.9400000000000004e-07 2024-09-14T06:31:40.019 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:40.019 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:40.019 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:31:40.019 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855059+0000", 2024-09-14T06:31:40.019 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.8870000000000001e-06 2024-09-14T06:31:40.019 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:40.019 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:40.019 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:31:40.019 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855061+0000", 2024-09-14T06:31:40.019 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.1919999999999999e-06 2024-09-14T06:31:40.019 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:40.019 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:40.020 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:31:40.020 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.919719+0000", 2024-09-14T06:31:40.020 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.064658013 2024-09-14T06:31:40.020 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:40.020 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:40.020 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:31:40.020 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.919788+0000", 2024-09-14T06:31:40.020 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.9282000000000005e-05 2024-09-14T06:31:40.020 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:40.020 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:40.020 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:31:40.020 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.919949+0000", 2024-09-14T06:31:40.020 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00016084199999999999 2024-09-14T06:31:40.020 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:40.021 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:31:40.021 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:40.021 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:40.021 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:40.021 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:44519 3.2 3:4190d882:::rbd_data.112dca2f6ae1.0000000000000002:17 [read 36864~4096 [fadvise_nocache]] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:31:40.021 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:43.855064+0000", 2024-09-14T06:31:40.021 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 116.124569669, 2024-09-14T06:31:40.021 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.065081865000000003, 2024-09-14T06:31:40.021 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:31:40.021 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:31:40.021 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:31:40.021 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:31:40.022 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:31:40.022 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:40.022 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 44519 2024-09-14T06:31:40.022 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:40.022 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:31:40.022 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:40.022 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:31:40.022 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855064+0000", 2024-09-14T06:31:40.022 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:40.022 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:40.022 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:40.022 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:31:40.022 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855064+0000", 2024-09-14T06:31:40.022 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:40.023 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:40.023 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:40.023 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:31:40.023 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855064+0000", 2024-09-14T06:31:40.023 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.1e-07 2024-09-14T06:31:40.023 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:40.023 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:40.023 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:31:40.023 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855065+0000", 2024-09-14T06:31:40.023 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.035e-06 2024-09-14T06:31:40.023 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:40.023 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:40.023 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:31:40.023 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855068+0000", 2024-09-14T06:31:40.024 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.8339999999999999e-06 2024-09-14T06:31:40.024 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:40.024 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:40.024 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:31:40.024 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855071+0000", 2024-09-14T06:31:40.024 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.3450000000000002e-06 2024-09-14T06:31:40.024 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:40.024 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:40.024 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:31:40.024 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.919979+0000", 2024-09-14T06:31:40.024 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.064907647999999998 2024-09-14T06:31:40.024 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:40.024 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:40.024 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:31:40.025 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.920026+0000", 2024-09-14T06:31:40.025 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.7595000000000001e-05 2024-09-14T06:31:40.025 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:40.025 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:40.025 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:31:40.025 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.920146+0000", 2024-09-14T06:31:40.025 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00011919800000000001 2024-09-14T06:31:40.025 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:40.025 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:31:40.025 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:40.025 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:40.025 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:40.025 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:44520 3.2 3:4190d882:::rbd_data.112dca2f6ae1.0000000000000002:17 [read 61440~4096 [fadvise_nocache]] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:31:40.026 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:43.855075+0000", 2024-09-14T06:31:40.026 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 116.12455861799999, 2024-09-14T06:31:40.026 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.065335078000000005, 2024-09-14T06:31:40.026 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:31:40.026 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:31:40.026 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:31:40.026 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:31:40.026 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:31:40.026 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:40.026 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 44520 2024-09-14T06:31:40.026 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:40.026 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:31:40.026 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:40.026 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:31:40.026 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855075+0000", 2024-09-14T06:31:40.027 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:40.027 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:40.027 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:40.027 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:31:40.027 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855075+0000", 2024-09-14T06:31:40.027 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:40.027 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:40.027 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:40.027 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:31:40.027 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855075+0000", 2024-09-14T06:31:40.027 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.1e-07 2024-09-14T06:31:40.027 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:40.027 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:40.027 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:31:40.028 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855076+0000", 2024-09-14T06:31:40.028 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.0839999999999999e-06 2024-09-14T06:31:40.028 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:40.028 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:40.028 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:31:40.028 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855079+0000", 2024-09-14T06:31:40.028 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.9359999999999999e-06 2024-09-14T06:31:40.028 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:40.028 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:40.028 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:31:40.028 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855084+0000", 2024-09-14T06:31:40.028 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.1730000000000003e-06 2024-09-14T06:31:40.028 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:40.028 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:40.028 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:31:40.029 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.920175+0000", 2024-09-14T06:31:40.029 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.065090234999999996 2024-09-14T06:31:40.029 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:40.029 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:40.029 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:31:40.029 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.920250+0000", 2024-09-14T06:31:40.029 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.5807000000000001e-05 2024-09-14T06:31:40.029 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:40.029 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:40.029 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:31:40.029 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.920410+0000", 2024-09-14T06:31:40.029 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00015963300000000001 2024-09-14T06:31:40.029 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:40.029 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:31:40.029 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:40.030 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:40.030 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:40.030 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:44521 3.2 3:4190d882:::rbd_data.112dca2f6ae1.0000000000000002:17 [read 86016~4096 [fadvise_nocache]] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:31:40.030 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:43.855087+0000", 2024-09-14T06:31:40.030 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 116.124546137, 2024-09-14T06:31:40.030 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.065563264999999996, 2024-09-14T06:31:40.030 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:31:40.030 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:31:40.030 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:31:40.030 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:31:40.030 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:31:40.030 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:40.030 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 44521 2024-09-14T06:31:40.030 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:40.030 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:31:40.031 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:40.031 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:31:40.031 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855087+0000", 2024-09-14T06:31:40.031 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:40.031 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:40.031 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:40.031 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:31:40.031 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855087+0000", 2024-09-14T06:31:40.031 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:40.031 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:40.031 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:40.031 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:31:40.031 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855088+0000", 2024-09-14T06:31:40.031 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.3699999999999999e-07 2024-09-14T06:31:40.031 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:40.032 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:40.032 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:31:40.032 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855089+0000", 2024-09-14T06:31:40.032 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.9900000000000009e-07 2024-09-14T06:31:40.032 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:40.032 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:40.032 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:31:40.032 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855091+0000", 2024-09-14T06:31:40.032 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.8940000000000002e-06 2024-09-14T06:31:40.032 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:40.032 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:40.032 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:31:40.032 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855094+0000", 2024-09-14T06:31:40.033 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.351e-06 2024-09-14T06:31:40.033 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:40.033 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:40.033 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:31:40.033 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.920437+0000", 2024-09-14T06:31:40.033 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.065343221000000007 2024-09-14T06:31:40.033 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:40.033 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:40.033 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:31:40.033 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.920481+0000", 2024-09-14T06:31:40.033 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.4431000000000002e-05 2024-09-14T06:31:40.033 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:40.033 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:40.034 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:31:40.034 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.920651+0000", 2024-09-14T06:31:40.034 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00016913200000000001 2024-09-14T06:31:40.034 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:40.034 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:31:40.034 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:40.034 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:40.034 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:40.034 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:44522 3.2 3:4190d882:::rbd_data.112dca2f6ae1.0000000000000002:17 [read 110592~4096 [fadvise_nocache]] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:31:40.034 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:43.855098+0000", 2024-09-14T06:31:40.034 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 116.12453591000001, 2024-09-14T06:31:40.034 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.065827628999999999, 2024-09-14T06:31:40.034 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:31:40.035 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:31:40.035 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:31:40.035 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:31:40.035 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:31:40.035 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:40.035 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 44522 2024-09-14T06:31:40.035 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:40.035 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:31:40.035 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:40.035 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:31:40.035 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855098+0000", 2024-09-14T06:31:40.035 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:40.035 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:40.035 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:40.035 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:31:40.036 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855098+0000", 2024-09-14T06:31:40.036 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:40.036 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:40.036 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:40.036 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:31:40.036 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855098+0000", 2024-09-14T06:31:40.036 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.1e-07 2024-09-14T06:31:40.036 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:40.036 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:40.036 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:31:40.036 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855099+0000", 2024-09-14T06:31:40.036 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.012e-06 2024-09-14T06:31:40.036 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:40.037 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:40.037 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:31:40.037 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855102+0000", 2024-09-14T06:31:40.037 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.9380000000000001e-06 2024-09-14T06:31:40.037 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:40.037 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:40.037 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:31:40.037 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855108+0000", 2024-09-14T06:31:40.037 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.4339999999999997e-06 2024-09-14T06:31:40.037 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:40.037 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:40.037 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:31:40.037 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.920681+0000", 2024-09-14T06:31:40.038 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.065572943999999994 2024-09-14T06:31:40.038 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:40.038 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:40.038 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:31:40.038 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.920724+0000", 2024-09-14T06:31:40.038 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.2877000000000003e-05 2024-09-14T06:31:40.038 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:40.038 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:40.038 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:31:40.038 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.920925+0000", 2024-09-14T06:31:40.039 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00020121399999999999 2024-09-14T06:31:40.039 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:40.039 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:31:40.039 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:40.039 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:40.039 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:40.039 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:44523 3.2 3:4190d882:::rbd_data.112dca2f6ae1.0000000000000002:17 [read 135168~4096 [fadvise_nocache]] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:31:40.039 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:43.855111+0000", 2024-09-14T06:31:40.039 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 116.124522188, 2024-09-14T06:31:40.039 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.066028198999999996, 2024-09-14T06:31:40.039 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:31:40.039 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:31:40.039 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:31:40.039 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:31:40.040 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:31:40.040 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:40.040 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 44523 2024-09-14T06:31:40.040 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:40.040 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:31:40.040 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:40.040 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:31:40.040 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855111+0000", 2024-09-14T06:31:40.040 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:40.040 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:40.040 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:40.040 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:31:40.040 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855111+0000", 2024-09-14T06:31:40.040 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:40.041 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:40.041 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:40.041 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:31:40.041 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855112+0000", 2024-09-14T06:31:40.041 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.1199999999999999e-07 2024-09-14T06:31:40.041 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:40.041 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:40.041 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:31:40.041 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855115+0000", 2024-09-14T06:31:40.041 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.3380000000000001e-06 2024-09-14T06:31:40.041 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:40.041 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:40.041 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:31:40.041 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855118+0000", 2024-09-14T06:31:40.042 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.366e-06 2024-09-14T06:31:40.042 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:40.042 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:40.042 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:31:40.042 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855121+0000", 2024-09-14T06:31:40.042 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.8329999999999998e-06 2024-09-14T06:31:40.042 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:40.042 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:40.042 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:31:40.042 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.920956+0000", 2024-09-14T06:31:40.042 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.065835251999999997 2024-09-14T06:31:40.042 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:40.042 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:40.043 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:31:40.043 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.921001+0000", 2024-09-14T06:31:40.043 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.4778999999999999e-05 2024-09-14T06:31:40.043 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:40.043 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:40.043 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:31:40.043 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.921139+0000", 2024-09-14T06:31:40.043 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000138419 2024-09-14T06:31:40.043 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:40.043 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:31:40.043 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:40.043 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:40.043 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:40.043 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:44524 3.2 3:4190d882:::rbd_data.112dca2f6ae1.0000000000000002:17 [read 159744~4096 [fadvise_nocache]] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:31:40.044 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:43.855124+0000", 2024-09-14T06:31:40.044 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 116.124509209, 2024-09-14T06:31:40.044 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.066249261000000004, 2024-09-14T06:31:40.044 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:31:40.044 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:31:40.044 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:31:40.044 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:31:40.044 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:31:40.044 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:40.044 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 44524 2024-09-14T06:31:40.044 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:40.044 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:31:40.044 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:40.044 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:31:40.045 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855124+0000", 2024-09-14T06:31:40.045 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:40.045 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:40.045 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:40.045 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:31:40.045 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855124+0000", 2024-09-14T06:31:40.045 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:40.045 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:40.045 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:40.045 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:31:40.045 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861086+0000", 2024-09-14T06:31:40.045 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0059613829999999998 2024-09-14T06:31:40.045 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:40.045 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:40.046 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:31:40.046 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861100+0000", 2024-09-14T06:31:40.046 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.4644000000000001e-05 2024-09-14T06:31:40.046 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:40.046 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:40.046 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:31:40.046 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861109+0000", 2024-09-14T06:31:40.046 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.541e-06 2024-09-14T06:31:40.046 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:40.046 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:40.046 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:31:40.046 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861117+0000", 2024-09-14T06:31:40.046 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.1049999999999995e-06 2024-09-14T06:31:40.046 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:40.046 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:40.047 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:31:40.047 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.921169+0000", 2024-09-14T06:31:40.047 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.060052405000000003 2024-09-14T06:31:40.047 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:40.047 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:40.047 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:31:40.047 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.921213+0000", 2024-09-14T06:31:40.047 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.3565e-05 2024-09-14T06:31:40.047 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:40.047 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:40.047 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:31:40.047 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.921374+0000", 2024-09-14T06:31:40.047 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00016061800000000001 2024-09-14T06:31:40.047 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:40.047 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:31:40.048 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:40.048 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:40.048 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:40.048 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:44536 3.2 3:4190d882:::rbd_data.112dca2f6ae1.0000000000000002:17 [read 454656~4096 [fadvise_nocache]] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:31:40.048 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:43.861384+0000", 2024-09-14T06:31:40.048 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 116.118249652, 2024-09-14T06:31:40.048 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.063058528000000003, 2024-09-14T06:31:40.048 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:31:40.048 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:31:40.048 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:31:40.048 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:31:40.048 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:31:40.048 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:40.049 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 44536 2024-09-14T06:31:40.049 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:40.049 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:31:40.049 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:40.049 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:31:40.049 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861384+0000", 2024-09-14T06:31:40.049 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:40.049 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:40.049 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:40.049 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:31:40.049 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861384+0000", 2024-09-14T06:31:40.049 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:40.049 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:40.049 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:40.049 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:31:40.050 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861384+0000", 2024-09-14T06:31:40.050 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.4000000000000002e-07 2024-09-14T06:31:40.050 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:40.050 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:40.050 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:31:40.050 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861386+0000", 2024-09-14T06:31:40.050 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.891e-06 2024-09-14T06:31:40.050 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:40.050 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:40.050 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:31:40.050 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861392+0000", 2024-09-14T06:31:40.050 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.9939999999999997e-06 2024-09-14T06:31:40.050 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:40.050 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:40.050 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:31:40.051 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861398+0000", 2024-09-14T06:31:40.051 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.507e-06 2024-09-14T06:31:40.051 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:40.051 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:40.051 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:31:40.051 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.924149+0000", 2024-09-14T06:31:40.051 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.062751135999999999 2024-09-14T06:31:40.051 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:40.051 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:40.051 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:31:40.051 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.924226+0000", 2024-09-14T06:31:40.051 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.7517999999999999e-05 2024-09-14T06:31:40.051 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:40.051 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:40.051 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:31:40.052 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.924442+0000", 2024-09-14T06:31:40.052 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000216042 2024-09-14T06:31:40.052 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:40.052 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:31:40.052 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:40.052 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:40.052 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:40.052 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:44537 3.2 3:4190d882:::rbd_data.112dca2f6ae1.0000000000000002:17 [read 479232~4096 [fadvise_nocache]] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:31:40.052 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:43.861405+0000", 2024-09-14T06:31:40.052 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 116.118228606, 2024-09-14T06:31:40.052 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.063351795000000002, 2024-09-14T06:31:40.052 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:31:40.052 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:31:40.052 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:31:40.052 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:31:40.052 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:31:40.053 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:40.053 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 44537 2024-09-14T06:31:40.053 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:40.053 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:31:40.053 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:40.053 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:31:40.053 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861405+0000", 2024-09-14T06:31:40.053 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:40.053 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:40.053 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:40.053 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:31:40.053 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861405+0000", 2024-09-14T06:31:40.053 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:40.054 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:40.054 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:40.054 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:31:40.054 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861405+0000", 2024-09-14T06:31:40.054 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.9099999999999999e-07 2024-09-14T06:31:40.054 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:40.054 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:40.054 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:31:40.054 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861407+0000", 2024-09-14T06:31:40.054 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.075e-06 2024-09-14T06:31:40.054 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:40.054 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:40.054 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:31:40.055 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861414+0000", 2024-09-14T06:31:40.055 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.4659999999999996e-06 2024-09-14T06:31:40.055 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:40.055 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:40.055 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:31:40.055 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861418+0000", 2024-09-14T06:31:40.055 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.5519999999999998e-06 2024-09-14T06:31:40.055 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:40.055 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:40.055 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:31:40.055 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.924474+0000", 2024-09-14T06:31:40.055 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.063056012999999994 2024-09-14T06:31:40.055 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:40.055 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:40.056 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:31:40.056 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.924556+0000", 2024-09-14T06:31:40.056 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.1181999999999996e-05 2024-09-14T06:31:40.056 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:40.056 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:40.056 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:31:40.056 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.924757+0000", 2024-09-14T06:31:40.056 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00020111599999999999 2024-09-14T06:31:40.056 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:40.056 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:31:40.056 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:40.056 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:40.056 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:40.056 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:44538 3.2 3:4190d882:::rbd_data.112dca2f6ae1.0000000000000002:17 [read 503808~4096 [fadvise_nocache]] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:31:40.057 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:43.861425+0000", 2024-09-14T06:31:40.057 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 116.118208565, 2024-09-14T06:31:40.057 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.063627125000000007, 2024-09-14T06:31:40.057 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:31:40.057 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:31:40.057 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:31:40.057 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:31:40.057 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:31:40.057 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:40.057 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 44538 2024-09-14T06:31:40.057 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:40.057 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:31:40.058 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:40.058 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:31:40.058 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861425+0000", 2024-09-14T06:31:40.058 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:40.058 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:40.058 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:40.058 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:31:40.058 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861425+0000", 2024-09-14T06:31:40.058 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:40.058 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:40.058 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:40.058 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:31:40.058 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861425+0000", 2024-09-14T06:31:40.059 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.9999999999999998e-07 2024-09-14T06:31:40.059 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:40.059 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:40.059 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:31:40.059 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861427+0000", 2024-09-14T06:31:40.059 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.9829999999999999e-06 2024-09-14T06:31:40.059 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:40.059 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:40.059 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:31:40.059 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861433+0000", 2024-09-14T06:31:40.059 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.2889999999999997e-06 2024-09-14T06:31:40.059 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:40.059 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:40.059 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:31:40.059 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861454+0000", 2024-09-14T06:31:40.059 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.1844999999999998e-05 2024-09-14T06:31:40.060 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:40.060 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:40.060 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:31:40.060 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.924786+0000", 2024-09-14T06:31:40.060 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.063331497 2024-09-14T06:31:40.060 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:40.060 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:40.060 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:31:40.060 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.924864+0000", 2024-09-14T06:31:40.060 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.7744000000000006e-05 2024-09-14T06:31:40.060 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:40.060 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:40.060 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:31:40.060 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.925052+0000", 2024-09-14T06:31:40.060 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00018836699999999999 2024-09-14T06:31:40.061 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:40.061 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:31:40.061 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:40.061 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:40.061 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:40.061 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:44539 3.2 3:4190d882:::rbd_data.112dca2f6ae1.0000000000000002:17 [read 528384~4096 [fadvise_nocache]] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:31:40.061 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:43.861462+0000", 2024-09-14T06:31:40.061 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 116.118171238, 2024-09-14T06:31:40.061 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.063855705999999998, 2024-09-14T06:31:40.061 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:31:40.061 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:31:40.061 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:31:40.061 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:31:40.061 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:31:40.061 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:40.062 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 44539 2024-09-14T06:31:40.062 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:40.062 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:31:40.062 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:40.062 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:31:40.062 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861462+0000", 2024-09-14T06:31:40.062 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:40.062 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:40.062 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:40.062 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:31:40.062 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861462+0000", 2024-09-14T06:31:40.062 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:40.062 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:40.062 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:40.063 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:31:40.063 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861463+0000", 2024-09-14T06:31:40.063 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.5200000000000002e-07 2024-09-14T06:31:40.063 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:40.063 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:40.063 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:31:40.063 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861465+0000", 2024-09-14T06:31:40.063 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.5189999999999999e-06 2024-09-14T06:31:40.063 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:40.063 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:40.063 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:31:40.063 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861471+0000", 2024-09-14T06:31:40.063 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.6899999999999997e-06 2024-09-14T06:31:40.063 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:40.063 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:40.064 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:31:40.064 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861475+0000", 2024-09-14T06:31:40.064 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.5730000000000001e-06 2024-09-14T06:31:40.064 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:40.064 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:40.064 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:31:40.064 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.925085+0000", 2024-09-14T06:31:40.064 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.063609902999999995 2024-09-14T06:31:40.064 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:40.064 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:40.064 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:31:40.064 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.925145+0000", 2024-09-14T06:31:40.064 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.9383999999999999e-05 2024-09-14T06:31:40.064 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:40.065 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:40.065 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:31:40.065 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.925318+0000", 2024-09-14T06:31:40.065 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00017318499999999999 2024-09-14T06:31:40.065 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:40.065 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:31:40.065 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:40.065 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:40.065 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:40.065 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:44540 3.2 3:4190d882:::rbd_data.112dca2f6ae1.0000000000000002:17 [read 552960~4096 [fadvise_nocache]] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:31:40.065 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:43.861483+0000", 2024-09-14T06:31:40.065 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 116.11815080700001, 2024-09-14T06:31:40.065 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.064065681999999999, 2024-09-14T06:31:40.065 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:31:40.066 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:31:40.066 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:31:40.066 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:31:40.066 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:31:40.066 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:40.066 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 44540 2024-09-14T06:31:40.066 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:40.066 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:31:40.066 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:40.066 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:31:40.066 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861483+0000", 2024-09-14T06:31:40.066 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:40.066 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:40.066 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:40.067 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:31:40.067 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861483+0000", 2024-09-14T06:31:40.067 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:40.067 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:40.067 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:40.067 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:31:40.067 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.867079+0000", 2024-09-14T06:31:40.067 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0055966480000000001 2024-09-14T06:31:40.067 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:40.067 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:40.067 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:31:40.067 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.867086+0000", 2024-09-14T06:31:40.067 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.387e-06 2024-09-14T06:31:40.067 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:40.067 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:40.068 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:31:40.068 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.867094+0000", 2024-09-14T06:31:40.068 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.9419999999999995e-06 2024-09-14T06:31:40.068 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:40.068 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:40.068 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:31:40.068 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.867109+0000", 2024-09-14T06:31:40.068 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.5818999999999999e-05 2024-09-14T06:31:40.068 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:40.068 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:40.068 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:31:40.068 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.925354+0000", 2024-09-14T06:31:40.068 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.058244047 2024-09-14T06:31:40.068 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:40.069 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:40.069 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:31:40.069 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.925406+0000", 2024-09-14T06:31:40.069 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.2661999999999998e-05 2024-09-14T06:31:40.069 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:40.069 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:40.069 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:31:40.069 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.925548+0000", 2024-09-14T06:31:40.069 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000142177 2024-09-14T06:31:40.069 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:40.069 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:31:40.069 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:40.069 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:40.069 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:31:40.069 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-14T06:31:40.070 INFO:teuthology.orchestra.run.smithi094.stdout:5349 write 0xb19c604 thru 0xb1a0c8f (0x468c bytes) 2024-09-14T06:31:40.071 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-14T06:31:40.073 INFO:teuthology.orchestra.run.smithi094.stdout:5350 punch from 0x76e8f5c to 0x76f4221, (0xb2c5 bytes) 2024-09-14T06:31:40.073 INFO:teuthology.orchestra.run.smithi094.stdout:5351 read 0x1ea13e3 thru 0x1ea1aad (0x6cb bytes) 2024-09-14T06:31:40.073 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-14T06:31:40.073 INFO:teuthology.orchestra.run.smithi094.stdout:5354 write 0x99e5559 thru 0x99e9a42 (0x44ea bytes) 2024-09-14T06:31:40.073 INFO:teuthology.orchestra.run.smithi094.stdout:5355 writesame 0x6c8ea97 thru 0x6c92841 data_size 0x1(0x3dab bytes) 2024-09-14T06:31:40.073 INFO:teuthology.orchestra.run.smithi094.stdout:5356 read 0x7232cce thru 0x72333f3 (0x726 bytes) 2024-09-14T06:31:40.073 INFO:teuthology.orchestra.run.smithi094.stdout:5357 write 0xd8bcc1e thru 0xd8c959f (0xc982 bytes) 2024-09-14T06:31:40.073 INFO:teuthology.orchestra.run.smithi094.stdout:5359 writesame 0x8f9574f thru 0x8f95801 data_size 0x1(0xb3 bytes) 2024-09-14T06:31:40.073 INFO:teuthology.orchestra.run.smithi094.stdout:5360 write 0xd6345e6 thru 0xd63f0bb (0xaad6 bytes) 2024-09-14T06:31:40.073 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-14T06:31:40.074 INFO:teuthology.orchestra.run.smithi094.stdout:5362 read 0x58e3c2d thru 0x58eb2b5 (0x7689 bytes) 2024-09-14T06:31:40.074 INFO:teuthology.orchestra.run.smithi094.stdout:5363 writesame 0xba5f0fd thru 0xba6d8e2 data_size 0x1(0xe7e6 bytes) 2024-09-14T06:31:40.074 INFO:teuthology.orchestra.run.smithi094.stdout:5364 read 0x3cbf163 thru 0x3cbf456 (0x2f4 bytes) 2024-09-14T06:31:40.074 INFO:teuthology.orchestra.run.smithi094.stdout:5365 write 0x79e7a34 thru 0x79f48fa (0xcec7 bytes) 2024-09-14T06:31:40.074 INFO:tasks.ceph.osd.6.smithi138.stderr:2024-09-14T06:31:40.069+0000 7f8cd8849640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33620) UID: 0 2024-09-14T06:31:40.077 INFO:teuthology.orchestra.run.smithi094.stdout:5366 write 0x5a9363a thru 0x5aa2068 (0xea2f bytes) 2024-09-14T06:31:40.091 INFO:teuthology.orchestra.run.smithi094.stdout:5367 read 0xbe308e4 thru 0xbe3b616 (0xad33 bytes) 2024-09-14T06:31:40.100 INFO:teuthology.orchestra.run.smithi094.stdout:5368 write 0xeb53c27 thru 0xeb600bc (0xc496 bytes) 2024-09-14T06:31:40.115 INFO:teuthology.orchestra.run.smithi094.stdout:5370 read 0x2c5dc59 thru 0x2c6527c (0x7624 bytes) 2024-09-14T06:31:40.115 INFO:teuthology.orchestra.run.smithi094.stdout:5372 read 0xd92c64b thru 0xd92dd8e (0x1744 bytes) 2024-09-14T06:31:40.115 INFO:teuthology.orchestra.run.smithi094.stdout:5373 write 0x4a7f3b8 thru 0x4a80449 (0x1092 bytes) 2024-09-14T06:31:40.125 INFO:teuthology.orchestra.run.smithi094.stdout:5374 read 0x3933f2b thru 0x3935d2a (0x1e00 bytes) 2024-09-14T06:31:40.125 INFO:teuthology.orchestra.run.smithi094.stdout:5375 write 0x4522f0f thru 0x452f5a3 (0xc695 bytes) 2024-09-14T06:31:40.140 INFO:teuthology.orchestra.run.smithi094.stdout:5376 writesame 0x25dfb3f thru 0x25e0ac3 data_size 0x1(0xf85 bytes) 2024-09-14T06:31:40.157 INFO:teuthology.orchestra.run.smithi094.stdout:5378 trunc from 0xeb600bd to 0x805206e 2024-09-14T06:31:40.171 INFO:tasks.ceph.osd.0.smithi094.stderr:2024-09-14T06:31:40.169+0000 7f07c9333640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33636) UID: 0 2024-09-14T06:31:40.197 INFO:teuthology.orchestra.run.smithi094.stdout:5379 write 0xbf9fb99 thru 0xbfa3519 (0x3981 bytes) 2024-09-14T06:31:40.201 INFO:teuthology.orchestra.run.smithi138.stdout:{ 2024-09-14T06:31:40.201 INFO:teuthology.orchestra.run.smithi138.stdout: "ops": [], 2024-09-14T06:31:40.201 INFO:teuthology.orchestra.run.smithi138.stdout: "num_ops": 0 2024-09-14T06:31:40.202 INFO:teuthology.orchestra.run.smithi138.stdout:} 2024-09-14T06:31:40.204 INFO:teuthology.orchestra.run.smithi094.stdout:5380 read 0x426cd5e thru 0x427b5b8 (0xe85b bytes) 2024-09-14T06:31:40.211 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-14T06:31:40.212 INFO:teuthology.orchestra.run.smithi094.stdout:5381 trunc from 0xbfa351a to 0x5ea9027 2024-09-14T06:31:40.227 INFO:teuthology.orchestra.run.smithi094.stdout:5382 read 0x57dcd94 thru 0x57e3ea3 (0x7110 bytes) 2024-09-14T06:31:40.243 INFO:teuthology.orchestra.run.smithi094.stdout:5383 write 0xd5cc2cb thru 0xd5d48bd (0x85f3 bytes) 2024-09-14T06:31:40.251 INFO:teuthology.orchestra.run.smithi094.stdout:5385 write 0x149cfb0 thru 0x14aa5a8 (0xd5f9 bytes) 2024-09-14T06:31:40.265 INFO:teuthology.orchestra.run.smithi094.stdout:5387 write 0x5706899 thru 0x5713d15 (0xd47d bytes) 2024-09-14T06:31:40.272 INFO:tasks.ceph.osd.6.smithi138.stderr:2024-09-14T06:31:40.271+0000 7f8cd8849640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33620) UID: 0 2024-09-14T06:31:40.274 INFO:teuthology.orchestra.run.smithi094.stdout:5388 writesame 0x740b937 thru 0x74172db data_size 0x5(0xb9a5 bytes) 2024-09-14T06:31:40.291 INFO:teuthology.orchestra.run.smithi094.stdout:5389 write 0x581b780 thru 0x58278fb (0xc17c bytes) 2024-09-14T06:31:40.313 INFO:teuthology.orchestra.run.smithi094.stdout:5390 trunc from 0xd5d48be to 0x255206b 2024-09-14T06:31:40.336 INFO:teuthology.orchestra.run.smithi138.stdout:{ 2024-09-14T06:31:40.336 INFO:teuthology.orchestra.run.smithi138.stdout: "ops": [], 2024-09-14T06:31:40.336 INFO:teuthology.orchestra.run.smithi138.stdout: "complaint_time": 30, 2024-09-14T06:31:40.336 INFO:teuthology.orchestra.run.smithi138.stdout: "num_blocked_ops": 0 2024-09-14T06:31:40.336 INFO:teuthology.orchestra.run.smithi138.stdout:} 2024-09-14T06:31:40.345 INFO:teuthology.orchestra.run.smithi094.stdout:5391 read 0xe4f333 thru 0xe5193b (0x2609 bytes) 2024-09-14T06:31:40.345 INFO:teuthology.orchestra.run.smithi094.stdout:5392 read 0x1741eff thru 0x174932c (0x742e bytes) 2024-09-14T06:31:40.345 INFO:teuthology.orchestra.run.smithi094.stdout:5394 read 0x8e323c thru 0x8e5bec (0x29b1 bytes) 2024-09-14T06:31:40.345 INFO:teuthology.orchestra.run.smithi094.stdout:5395 read 0xfa18b9 thru 0xfb045c (0xeba4 bytes) 2024-09-14T06:31:40.345 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-14T06:31:40.345 INFO:teuthology.orchestra.run.smithi094.stdout:5399 punch from 0x1d624b9 to 0x1d6d614, (0xb15b bytes) 2024-09-14T06:31:40.345 INFO:teuthology.orchestra.run.smithi094.stdout:5400 read 0x13982b0 thru 0x13a53e5 (0xd136 bytes) 2024-09-14T06:31:40.346 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-14T06:31:40.363 INFO:teuthology.orchestra.run.smithi094.stdout:5401 writesame 0xc7503c0 thru 0xc75c9d3 data_size 0xe(0xc614 bytes) 2024-09-14T06:31:40.363 INFO:teuthology.orchestra.run.smithi094.stdout:5402 read 0xc2f554d thru 0xc2fb2d1 (0x5d85 bytes) 2024-09-14T06:31:40.363 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-14T06:31:40.363 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-14T06:31:40.363 INFO:teuthology.orchestra.run.smithi094.stdout:5405 punch from 0x32cedb3 to 0x32dd2bb, (0xe508 bytes) 2024-09-14T06:31:40.363 INFO:teuthology.orchestra.run.smithi094.stdout:5406 read 0x53271fc thru 0x532abdf (0x39e4 bytes) 2024-09-14T06:31:40.363 INFO:teuthology.orchestra.run.smithi094.stdout:5407 read 0xb92cecf thru 0xb92d277 (0x3a9 bytes) 2024-09-14T06:31:40.363 INFO:teuthology.orchestra.run.smithi094.stdout:5408 read 0x535a3c3 thru 0x535dce4 (0x3922 bytes) 2024-09-14T06:31:40.363 INFO:teuthology.orchestra.run.smithi094.stdout:5409 write 0x45452de thru 0x454d1b4 (0x7ed7 bytes) 2024-09-14T06:31:40.370 INFO:teuthology.orchestra.run.smithi094.stdout:5410 writesame 0x728fcad thru 0x729bab5 data_size 0x1(0xbe09 bytes) 2024-09-14T06:31:40.372 INFO:tasks.ceph.osd.3.smithi094.stderr:2024-09-14T06:31:40.370+0000 7feeeafe8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33655) UID: 0 2024-09-14T06:31:40.389 INFO:teuthology.orchestra.run.smithi094.stdout:5411 read 0x29e8188 thru 0x29f424e (0xc0c7 bytes) 2024-09-14T06:31:40.389 INFO:teuthology.orchestra.run.smithi094.stdout:5413 read 0x5a2701 thru 0x5ae868 (0xc168 bytes) 2024-09-14T06:31:40.396 INFO:teuthology.orchestra.run.smithi094.stdout:5416 write 0xd358c28 thru 0xd35be26 (0x31ff bytes) 2024-09-14T06:31:40.402 INFO:teuthology.orchestra.run.smithi094.stdout:5417 punch from 0x75c8298 to 0x75d1db7, (0x9b1f bytes) 2024-09-14T06:31:40.402 INFO:teuthology.orchestra.run.smithi094.stdout:5418 writesame 0xc4ae6ad thru 0xc4ba043 data_size 0x1(0xb997 bytes) 2024-09-14T06:31:40.421 INFO:teuthology.orchestra.run.smithi094.stdout:5420 read 0x6a6bb0 thru 0x6a9cf8 (0x3149 bytes) 2024-09-14T06:31:40.421 INFO:teuthology.orchestra.run.smithi094.stdout:5421 punch from 0xba709a8 to 0xba785be, (0x7c16 bytes) 2024-09-14T06:31:40.421 INFO:teuthology.orchestra.run.smithi094.stdout:5423 write 0xdcef30 thru 0xdda3c3 (0xb494 bytes) 2024-09-14T06:31:40.452 INFO:teuthology.orchestra.run.smithi094.stdout:5424 trunc from 0xd35be27 to 0x1a2f024 2024-09-14T06:31:40.468 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-14T06:31:40.469 INFO:teuthology.orchestra.run.smithi094.stdout:5426 read 0xed9a5b thru 0xed9fab (0x551 bytes) 2024-09-14T06:31:40.469 INFO:teuthology.orchestra.run.smithi094.stdout:5427 read 0x15b87ca thru 0x15c2522 (0x9d59 bytes) 2024-09-14T06:31:40.473 INFO:tasks.ceph.osd.6.smithi138.stderr:2024-09-14T06:31:40.472+0000 7f8cd8849640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33620) UID: 0 2024-09-14T06:31:40.478 INFO:teuthology.orchestra.run.smithi094.stdout:5428 trunc from 0x1a2f024 to 0x8f71f44 2024-09-14T06:31:40.492 INFO:teuthology.orchestra.run.smithi094.stdout:5429 trunc from 0x8f71f44 to 0xc64fd2b 2024-09-14T06:31:40.501 INFO:teuthology.orchestra.run.smithi094.stdout:5430 write 0xd13e2f7 thru 0xd146474 (0x817e bytes) 2024-09-14T06:31:40.505 INFO:teuthology.orchestra.run.smithi138.stdout:{ 2024-09-14T06:31:40.505 INFO:teuthology.orchestra.run.smithi138.stdout: "size": 20, 2024-09-14T06:31:40.505 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 600, 2024-09-14T06:31:40.505 INFO:teuthology.orchestra.run.smithi138.stdout: "ops": [ 2024-09-14T06:31:40.505 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:40.505 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:19328 3.3 3:c34e5e51:::rbd_data.112d92027590.0000000000000024:f [read 1926144~1024 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:31:40.505 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:27.059768+0000", 2024-09-14T06:31:40.505 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 133.442751689, 2024-09-14T06:31:40.505 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046369147999999999, 2024-09-14T06:31:40.505 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:31:40.505 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:31:40.505 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:31:40.505 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:31:40.506 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:31:40.506 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:40.506 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 19328 2024-09-14T06:31:40.506 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:40.506 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:31:40.506 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:40.506 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:31:40.506 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.059768+0000", 2024-09-14T06:31:40.506 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:40.506 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:40.506 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:40.506 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:31:40.506 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.059768+0000", 2024-09-14T06:31:40.506 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:40.506 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:40.507 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:40.507 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:31:40.507 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.059874+0000", 2024-09-14T06:31:40.507 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00010551400000000001 2024-09-14T06:31:40.507 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:40.507 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:40.507 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:31:40.507 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.059877+0000", 2024-09-14T06:31:40.507 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.8279999999999999e-06 2024-09-14T06:31:40.507 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:40.507 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:40.507 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:31:40.507 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.059884+0000", 2024-09-14T06:31:40.507 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.6389999999999992e-06 2024-09-14T06:31:40.507 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:40.508 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:40.508 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:31:40.508 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.059890+0000", 2024-09-14T06:31:40.508 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.49e-06 2024-09-14T06:31:40.508 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:40.508 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:40.508 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:31:40.508 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.105804+0000", 2024-09-14T06:31:40.508 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.045914242000000001 2024-09-14T06:31:40.508 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:40.508 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:40.508 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:31:40.508 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.105886+0000", 2024-09-14T06:31:40.508 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.2305999999999994e-05 2024-09-14T06:31:40.509 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:40.509 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:40.509 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:31:40.509 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.106138+0000", 2024-09-14T06:31:40.509 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.000251129 2024-09-14T06:31:40.509 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:40.509 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:31:40.509 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:40.509 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:40.509 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:40.509 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:19329 3.3 3:c34e5e51:::rbd_data.112d92027590.0000000000000024:f [read 1936384~1024 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:31:40.509 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:27.059899+0000", 2024-09-14T06:31:40.509 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 133.44262116600001, 2024-09-14T06:31:40.509 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046572122000000001, 2024-09-14T06:31:40.510 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:31:40.510 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:31:40.510 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:31:40.510 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:31:40.510 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:31:40.510 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:40.510 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 19329 2024-09-14T06:31:40.510 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:40.510 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:31:40.510 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:40.510 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:31:40.510 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.059899+0000", 2024-09-14T06:31:40.510 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:40.510 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:40.511 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:40.511 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:31:40.511 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.059899+0000", 2024-09-14T06:31:40.511 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:40.511 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:40.511 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:40.511 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:31:40.511 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062501+0000", 2024-09-14T06:31:40.511 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.0026016419999999999 2024-09-14T06:31:40.511 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:40.511 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:40.511 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:31:40.511 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062514+0000", 2024-09-14T06:31:40.512 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.2947e-05 2024-09-14T06:31:40.512 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:40.512 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:40.512 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:31:40.512 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062523+0000", 2024-09-14T06:31:40.512 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.5519999999999993e-06 2024-09-14T06:31:40.512 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:40.512 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:40.512 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:31:40.512 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062534+0000", 2024-09-14T06:31:40.512 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.1416e-05 2024-09-14T06:31:40.512 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:40.512 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:40.512 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:31:40.512 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.106179+0000", 2024-09-14T06:31:40.513 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.043644049999999997 2024-09-14T06:31:40.513 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:40.513 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:40.513 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:31:40.513 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.106262+0000", 2024-09-14T06:31:40.513 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.3178999999999998e-05 2024-09-14T06:31:40.513 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:40.513 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:40.513 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:31:40.513 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.106471+0000", 2024-09-14T06:31:40.513 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00020933599999999999 2024-09-14T06:31:40.513 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:40.513 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:31:40.513 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:40.513 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:40.514 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:40.514 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:19351 3.3 3:c34e5e51:::rbd_data.112d92027590.0000000000000024:f [read 1159168~1024 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:31:40.514 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:27.062766+0000", 2024-09-14T06:31:40.514 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 133.43975394099999, 2024-09-14T06:31:40.514 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046099454999999998, 2024-09-14T06:31:40.514 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:31:40.514 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:31:40.514 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:31:40.514 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:31:40.514 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:31:40.514 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:40.514 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 19351 2024-09-14T06:31:40.514 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:40.514 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:31:40.515 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:40.515 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:31:40.515 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062766+0000", 2024-09-14T06:31:40.515 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:40.515 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:40.515 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:40.515 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:31:40.515 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062766+0000", 2024-09-14T06:31:40.515 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:40.515 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:40.515 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:40.515 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:31:40.515 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062766+0000", 2024-09-14T06:31:40.515 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.6e-07 2024-09-14T06:31:40.516 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:40.516 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:40.516 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:31:40.516 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062767+0000", 2024-09-14T06:31:40.516 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.0070000000000001e-06 2024-09-14T06:31:40.516 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:40.516 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:40.516 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:31:40.516 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062785+0000", 2024-09-14T06:31:40.516 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.7884999999999999e-05 2024-09-14T06:31:40.516 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:40.516 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:40.516 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:31:40.516 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062790+0000", 2024-09-14T06:31:40.516 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.0900000000000004e-06 2024-09-14T06:31:40.517 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:40.517 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:40.517 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:31:40.517 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.108735+0000", 2024-09-14T06:31:40.517 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.045944411999999997 2024-09-14T06:31:40.517 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:40.517 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:40.517 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:31:40.517 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.108809+0000", 2024-09-14T06:31:40.517 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.4157000000000002e-05 2024-09-14T06:31:40.517 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:40.517 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:40.517 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:31:40.518 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.108866+0000", 2024-09-14T06:31:40.518 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.6644000000000002e-05 2024-09-14T06:31:40.518 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:40.518 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:31:40.518 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:40.518 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:40.518 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:40.518 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:19352 3.3 3:c34e5e51:::rbd_data.112d92027590.0000000000000024:f [read 1169408~1024 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:31:40.518 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:27.062798+0000", 2024-09-14T06:31:40.518 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 133.43972253000001, 2024-09-14T06:31:40.518 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046255900000000003, 2024-09-14T06:31:40.518 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:31:40.518 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:31:40.519 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:31:40.519 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:31:40.519 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:31:40.519 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:40.519 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 19352 2024-09-14T06:31:40.519 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:40.519 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:31:40.519 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:40.519 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:31:40.519 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062798+0000", 2024-09-14T06:31:40.519 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:40.519 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:40.520 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:40.520 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:31:40.520 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062798+0000", 2024-09-14T06:31:40.520 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:40.520 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:40.520 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:40.520 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:31:40.520 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062798+0000", 2024-09-14T06:31:40.520 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.37e-07 2024-09-14T06:31:40.520 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:40.521 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:40.521 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:31:40.521 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062800+0000", 2024-09-14T06:31:40.521 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.0269999999999998e-06 2024-09-14T06:31:40.521 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:40.521 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:40.521 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:31:40.521 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062806+0000", 2024-09-14T06:31:40.521 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.6690000000000003e-06 2024-09-14T06:31:40.521 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:40.521 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:40.521 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:31:40.521 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062810+0000", 2024-09-14T06:31:40.522 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.3449999999999999e-06 2024-09-14T06:31:40.522 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:40.522 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:40.522 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:31:40.522 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.108892+0000", 2024-09-14T06:31:40.522 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046081715000000002 2024-09-14T06:31:40.522 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:40.522 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:40.522 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:31:40.522 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.108940+0000", 2024-09-14T06:31:40.522 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.7815999999999998e-05 2024-09-14T06:31:40.523 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:40.523 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:40.523 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:31:40.523 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.109053+0000", 2024-09-14T06:31:40.523 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.000113791 2024-09-14T06:31:40.523 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:40.523 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:31:40.523 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:40.523 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:40.523 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:40.523 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:19353 3.3 3:c34e5e51:::rbd_data.112d92027590.0000000000000024:f [read 1179648~1024 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:31:40.523 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:27.062816+0000", 2024-09-14T06:31:40.523 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 133.439704011, 2024-09-14T06:31:40.524 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046481256999999998, 2024-09-14T06:31:40.524 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:31:40.524 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:31:40.524 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:31:40.524 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:31:40.524 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:31:40.524 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:40.524 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 19353 2024-09-14T06:31:40.524 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:40.524 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:31:40.524 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:40.524 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:31:40.524 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062816+0000", 2024-09-14T06:31:40.524 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:40.525 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:40.525 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:40.525 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:31:40.525 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062816+0000", 2024-09-14T06:31:40.525 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:40.525 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:40.525 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:40.525 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:31:40.525 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062816+0000", 2024-09-14T06:31:40.525 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.2899999999999999e-07 2024-09-14T06:31:40.525 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:40.525 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:40.525 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:31:40.525 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062818+0000", 2024-09-14T06:31:40.526 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.699e-06 2024-09-14T06:31:40.526 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:40.526 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:40.526 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:31:40.526 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062825+0000", 2024-09-14T06:31:40.526 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.878e-06 2024-09-14T06:31:40.526 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:40.526 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:40.526 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:31:40.526 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062830+0000", 2024-09-14T06:31:40.526 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.5410000000000002e-06 2024-09-14T06:31:40.526 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:40.526 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:40.526 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:31:40.526 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.109096+0000", 2024-09-14T06:31:40.527 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046266517 2024-09-14T06:31:40.527 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:40.527 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:40.527 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:31:40.527 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.109190+0000", 2024-09-14T06:31:40.527 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.4000999999999997e-05 2024-09-14T06:31:40.527 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:40.527 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:40.527 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:31:40.527 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.109297+0000", 2024-09-14T06:31:40.527 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.000107292 2024-09-14T06:31:40.527 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:40.527 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:31:40.527 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:40.527 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:40.528 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:40.528 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:19354 3.3 3:c34e5e51:::rbd_data.112d92027590.0000000000000024:f [read 1189888~1024 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:31:40.528 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:27.062842+0000", 2024-09-14T06:31:40.528 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 133.43967855599999, 2024-09-14T06:31:40.528 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046594257, 2024-09-14T06:31:40.528 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:31:40.528 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:31:40.528 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:31:40.528 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:31:40.528 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:31:40.528 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:40.528 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 19354 2024-09-14T06:31:40.528 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:40.528 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:31:40.528 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:40.529 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:31:40.529 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062842+0000", 2024-09-14T06:31:40.529 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:40.529 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:40.529 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:40.529 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:31:40.529 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062842+0000", 2024-09-14T06:31:40.529 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:40.529 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:40.529 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:40.529 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:31:40.529 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062842+0000", 2024-09-14T06:31:40.529 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.7399999999999999e-07 2024-09-14T06:31:40.529 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:40.530 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:40.530 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:31:40.530 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062859+0000", 2024-09-14T06:31:40.530 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.7010000000000001e-05 2024-09-14T06:31:40.530 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:40.530 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:40.530 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:31:40.530 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062866+0000", 2024-09-14T06:31:40.530 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.9990000000000002e-06 2024-09-14T06:31:40.530 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:40.530 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:40.530 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:31:40.530 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062873+0000", 2024-09-14T06:31:40.531 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.8569999999999997e-06 2024-09-14T06:31:40.531 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:40.531 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:40.531 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:31:40.531 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.109335+0000", 2024-09-14T06:31:40.531 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046461943999999998 2024-09-14T06:31:40.531 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:40.531 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:40.531 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:31:40.531 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.109377+0000", 2024-09-14T06:31:40.531 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.2558999999999997e-05 2024-09-14T06:31:40.531 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:40.531 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:40.532 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:31:40.532 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.109436+0000", 2024-09-14T06:31:40.532 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.8514000000000003e-05 2024-09-14T06:31:40.532 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:40.532 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:31:40.532 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:40.532 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:40.532 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:40.532 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:19355 3.3 3:c34e5e51:::rbd_data.112d92027590.0000000000000024:f [read 1200128~1024 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:31:40.532 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:27.062884+0000", 2024-09-14T06:31:40.532 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 133.43963588299999, 2024-09-14T06:31:40.532 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046681650999999998, 2024-09-14T06:31:40.533 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:31:40.533 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:31:40.533 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:31:40.533 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:31:40.533 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:31:40.533 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:40.533 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 19355 2024-09-14T06:31:40.533 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:40.533 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:31:40.533 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:40.533 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:31:40.533 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062884+0000", 2024-09-14T06:31:40.534 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:40.534 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:40.534 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:40.534 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:31:40.534 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062884+0000", 2024-09-14T06:31:40.534 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:40.534 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:40.534 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:40.534 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:31:40.534 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062885+0000", 2024-09-14T06:31:40.534 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.8000000000000001e-07 2024-09-14T06:31:40.535 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:40.535 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:40.535 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:31:40.535 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062886+0000", 2024-09-14T06:31:40.535 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.8920000000000001e-06 2024-09-14T06:31:40.535 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:40.535 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:40.535 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:31:40.535 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062892+0000", 2024-09-14T06:31:40.535 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.84e-06 2024-09-14T06:31:40.535 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:40.535 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:40.536 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:31:40.536 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062899+0000", 2024-09-14T06:31:40.536 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.5339999999999996e-06 2024-09-14T06:31:40.536 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:40.536 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:40.536 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:31:40.536 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.109466+0000", 2024-09-14T06:31:40.536 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046566919999999998 2024-09-14T06:31:40.536 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:40.536 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:40.537 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:31:40.537 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.109510+0000", 2024-09-14T06:31:40.537 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.4153000000000002e-05 2024-09-14T06:31:40.537 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:40.537 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:40.537 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:31:40.537 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.109566+0000", 2024-09-14T06:31:40.537 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.5931999999999999e-05 2024-09-14T06:31:40.537 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:40.537 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:31:40.537 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:40.537 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:40.537 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:40.537 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:19356 3.3 3:c34e5e51:::rbd_data.112d92027590.0000000000000024:f [read 1210368~1024 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:31:40.537 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:27.062908+0000", 2024-09-14T06:31:40.538 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 133.43961244600001, 2024-09-14T06:31:40.538 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046807498000000003, 2024-09-14T06:31:40.538 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:31:40.538 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:31:40.538 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:31:40.538 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:31:40.538 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:31:40.538 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:40.538 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 19356 2024-09-14T06:31:40.538 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:40.538 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:31:40.538 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:40.538 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:31:40.538 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062908+0000", 2024-09-14T06:31:40.538 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:40.539 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:40.539 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:40.539 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:31:40.539 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062908+0000", 2024-09-14T06:31:40.539 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:40.539 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:40.539 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:40.539 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:31:40.539 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062908+0000", 2024-09-14T06:31:40.539 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.5699999999999998e-07 2024-09-14T06:31:40.539 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:40.539 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:40.539 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:31:40.539 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062910+0000", 2024-09-14T06:31:40.540 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.7710000000000001e-06 2024-09-14T06:31:40.540 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:40.540 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:40.540 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:31:40.540 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062920+0000", 2024-09-14T06:31:40.540 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.0654e-05 2024-09-14T06:31:40.540 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:40.540 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:40.540 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:31:40.541 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062931+0000", 2024-09-14T06:31:40.541 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.075e-05 2024-09-14T06:31:40.541 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:40.541 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:40.541 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:31:40.541 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.109595+0000", 2024-09-14T06:31:40.541 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046663713000000002 2024-09-14T06:31:40.541 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:40.541 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:40.541 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:31:40.541 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.109632+0000", 2024-09-14T06:31:40.541 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.7505000000000001e-05 2024-09-14T06:31:40.541 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:40.542 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:40.542 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:31:40.542 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.109715+0000", 2024-09-14T06:31:40.542 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.2748000000000001e-05 2024-09-14T06:31:40.542 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:40.542 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:31:40.542 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:40.542 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:40.542 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:40.542 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:19357 3.3 3:c34e5e51:::rbd_data.112d92027590.0000000000000024:f [read 1220608~1024 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:31:40.542 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:27.062939+0000", 2024-09-14T06:31:40.542 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 133.439580998, 2024-09-14T06:31:40.542 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046920508, 2024-09-14T06:31:40.542 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:31:40.543 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:31:40.543 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:31:40.543 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:31:40.543 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:31:40.543 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:40.543 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 19357 2024-09-14T06:31:40.543 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:40.543 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:31:40.543 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:40.543 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:31:40.543 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062939+0000", 2024-09-14T06:31:40.543 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:40.543 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:40.543 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:40.544 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:31:40.544 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062939+0000", 2024-09-14T06:31:40.544 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:40.544 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:40.544 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:40.544 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:31:40.544 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062940+0000", 2024-09-14T06:31:40.544 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.32e-07 2024-09-14T06:31:40.544 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:40.544 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:40.544 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:31:40.544 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062942+0000", 2024-09-14T06:31:40.544 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.1919999999999999e-06 2024-09-14T06:31:40.544 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:40.545 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:40.545 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:31:40.545 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062948+0000", 2024-09-14T06:31:40.545 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.1149999999999999e-06 2024-09-14T06:31:40.545 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:40.545 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:40.545 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:31:40.545 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062960+0000", 2024-09-14T06:31:40.545 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.1895e-05 2024-09-14T06:31:40.545 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:40.545 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:40.545 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:31:40.545 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.109704+0000", 2024-09-14T06:31:40.546 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046744584999999998 2024-09-14T06:31:40.546 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:40.546 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:40.546 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:31:40.546 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.109777+0000", 2024-09-14T06:31:40.546 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.2282999999999999e-05 2024-09-14T06:31:40.546 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:40.546 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:40.546 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:31:40.546 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.109860+0000", 2024-09-14T06:31:40.546 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.3005999999999998e-05 2024-09-14T06:31:40.546 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:40.546 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:31:40.546 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:40.546 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:40.547 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:40.547 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:19358 3.3 3:c34e5e51:::rbd_data.112d92027590.0000000000000024:f [read 1230848~1024 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:31:40.547 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:27.062967+0000", 2024-09-14T06:31:40.547 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 133.43955336799999, 2024-09-14T06:31:40.547 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047144471, 2024-09-14T06:31:40.547 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:31:40.547 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:31:40.547 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:31:40.547 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:31:40.547 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:31:40.547 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:40.547 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 19358 2024-09-14T06:31:40.547 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:40.547 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:31:40.547 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:40.548 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:31:40.548 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062967+0000", 2024-09-14T06:31:40.548 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:40.548 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:40.548 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:40.548 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:31:40.548 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062967+0000", 2024-09-14T06:31:40.548 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:40.548 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:40.548 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:40.548 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:31:40.548 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062967+0000", 2024-09-14T06:31:40.548 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.89e-07 2024-09-14T06:31:40.548 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:40.548 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:40.549 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:31:40.549 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062969+0000", 2024-09-14T06:31:40.549 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.9929999999999998e-06 2024-09-14T06:31:40.549 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:40.549 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:40.549 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:31:40.549 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062975+0000", 2024-09-14T06:31:40.549 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.9030000000000001e-06 2024-09-14T06:31:40.549 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:40.549 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:40.549 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:31:40.549 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062979+0000", 2024-09-14T06:31:40.549 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.2250000000000002e-06 2024-09-14T06:31:40.549 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:40.550 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:40.550 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:31:40.550 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.109934+0000", 2024-09-14T06:31:40.550 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046954410000000002 2024-09-14T06:31:40.550 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:40.550 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:40.550 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:31:40.550 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.110007+0000", 2024-09-14T06:31:40.550 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.3829000000000001e-05 2024-09-14T06:31:40.550 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:40.550 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:40.550 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:31:40.550 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.110111+0000", 2024-09-14T06:31:40.550 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00010372199999999999 2024-09-14T06:31:40.550 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:40.551 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:31:40.551 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:40.551 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:40.551 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:40.551 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:19359 3.3 3:c34e5e51:::rbd_data.112d92027590.0000000000000024:f [read 1241088~1024 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:31:40.551 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:27.062987+0000", 2024-09-14T06:31:40.551 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 133.439533278, 2024-09-14T06:31:40.551 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047297182, 2024-09-14T06:31:40.551 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:31:40.551 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:31:40.551 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:31:40.551 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:31:40.551 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:31:40.552 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:40.552 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 19359 2024-09-14T06:31:40.552 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:40.552 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:31:40.552 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:40.552 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:31:40.552 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062987+0000", 2024-09-14T06:31:40.552 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:40.552 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:40.552 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:40.552 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:31:40.552 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062987+0000", 2024-09-14T06:31:40.552 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:40.552 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:40.552 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:40.553 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:31:40.553 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062987+0000", 2024-09-14T06:31:40.553 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.2500000000000001e-07 2024-09-14T06:31:40.553 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:40.553 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:40.553 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:31:40.553 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062989+0000", 2024-09-14T06:31:40.553 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.6080000000000001e-06 2024-09-14T06:31:40.553 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:40.553 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:40.553 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:31:40.553 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062994+0000", 2024-09-14T06:31:40.553 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.5520000000000004e-06 2024-09-14T06:31:40.553 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:40.553 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:40.554 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:31:40.554 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062999+0000", 2024-09-14T06:31:40.554 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.3930000000000001e-06 2024-09-14T06:31:40.554 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:40.554 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:40.554 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:31:40.554 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.110142+0000", 2024-09-14T06:31:40.554 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047143154 2024-09-14T06:31:40.554 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:40.554 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:40.554 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:31:40.554 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.110219+0000", 2024-09-14T06:31:40.554 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.6737999999999996e-05 2024-09-14T06:31:40.554 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:40.554 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:40.555 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:31:40.555 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.110284+0000", 2024-09-14T06:31:40.555 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.5412000000000003e-05 2024-09-14T06:31:40.555 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:40.555 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:31:40.555 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:40.555 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:40.555 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:40.555 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:19360 3.3 3:c34e5e51:::rbd_data.112d92027590.0000000000000024:f [read 1251328~1024 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:31:40.555 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:27.063017+0000", 2024-09-14T06:31:40.555 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 133.439502817, 2024-09-14T06:31:40.555 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047453823999999999, 2024-09-14T06:31:40.555 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:31:40.555 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:31:40.556 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:31:40.556 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:31:40.556 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:31:40.556 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:40.556 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 19360 2024-09-14T06:31:40.556 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:40.556 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:31:40.556 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:40.556 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:31:40.556 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.063017+0000", 2024-09-14T06:31:40.556 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:40.556 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:40.556 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:40.556 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:31:40.557 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.063017+0000", 2024-09-14T06:31:40.557 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:40.557 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:40.557 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:40.557 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:31:40.557 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.064977+0000", 2024-09-14T06:31:40.557 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.0019597080000000001 2024-09-14T06:31:40.557 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:40.557 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:40.557 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:31:40.557 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.064987+0000", 2024-09-14T06:31:40.557 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.9620000000000005e-06 2024-09-14T06:31:40.557 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:40.558 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:40.558 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:31:40.558 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065000+0000", 2024-09-14T06:31:40.558 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.3173000000000001e-05 2024-09-14T06:31:40.558 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:40.558 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:40.558 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:31:40.558 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065005+0000", 2024-09-14T06:31:40.558 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.9960000000000001e-06 2024-09-14T06:31:40.558 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:40.558 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:40.559 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:31:40.559 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.110314+0000", 2024-09-14T06:31:40.559 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.04530899 2024-09-14T06:31:40.559 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:40.559 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:40.559 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:31:40.559 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.110398+0000", 2024-09-14T06:31:40.559 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.3539000000000001e-05 2024-09-14T06:31:40.559 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:40.559 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:40.559 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:31:40.559 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.110471+0000", 2024-09-14T06:31:40.559 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.3455999999999996e-05 2024-09-14T06:31:40.559 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:40.560 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:31:40.560 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:40.560 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:40.560 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:40.560 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:19365 3.3 3:c34e5e51:::rbd_data.112d92027590.0000000000000024:f [read 1302528~1024 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:31:40.560 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:27.065058+0000", 2024-09-14T06:31:40.560 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 133.43746214699999, 2024-09-14T06:31:40.560 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046105526000000001, 2024-09-14T06:31:40.560 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:31:40.560 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:31:40.560 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:31:40.560 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:31:40.560 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:31:40.561 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:40.561 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 19365 2024-09-14T06:31:40.561 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:40.561 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:31:40.561 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:40.561 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:31:40.561 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065058+0000", 2024-09-14T06:31:40.561 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:40.561 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:40.561 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:40.561 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:31:40.561 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065058+0000", 2024-09-14T06:31:40.562 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:40.562 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:40.562 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:40.562 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:31:40.562 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065058+0000", 2024-09-14T06:31:40.562 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.1899999999999999e-07 2024-09-14T06:31:40.562 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:40.562 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:40.562 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:31:40.562 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065059+0000", 2024-09-14T06:31:40.562 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.1299999999999998e-07 2024-09-14T06:31:40.562 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:40.562 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:40.563 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:31:40.563 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065063+0000", 2024-09-14T06:31:40.563 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.1099999999999996e-06 2024-09-14T06:31:40.563 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:40.563 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:40.563 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:31:40.563 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065070+0000", 2024-09-14T06:31:40.563 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.973e-06 2024-09-14T06:31:40.563 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:40.563 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:40.563 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:31:40.564 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.111014+0000", 2024-09-14T06:31:40.564 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.045943373000000003 2024-09-14T06:31:40.564 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:40.564 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:40.564 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:31:40.564 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.111073+0000", 2024-09-14T06:31:40.564 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.8987999999999998e-05 2024-09-14T06:31:40.564 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:40.564 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:40.564 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:31:40.564 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.111163+0000", 2024-09-14T06:31:40.564 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.0950000000000002e-05 2024-09-14T06:31:40.564 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:40.565 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:31:40.565 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:40.565 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:40.565 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:40.565 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:19366 3.3 3:c34e5e51:::rbd_data.112d92027590.0000000000000024:f [read 1312768~1024 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:31:40.565 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:27.065074+0000", 2024-09-14T06:31:40.565 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 133.43744629700001, 2024-09-14T06:31:40.565 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046289858000000003, 2024-09-14T06:31:40.565 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:31:40.565 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:31:40.565 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:31:40.566 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:31:40.566 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:31:40.566 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:40.566 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 19366 2024-09-14T06:31:40.566 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:40.566 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:31:40.566 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:40.566 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:31:40.566 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065074+0000", 2024-09-14T06:31:40.566 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:40.566 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:40.566 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:40.567 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:31:40.567 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065074+0000", 2024-09-14T06:31:40.567 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:40.567 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:40.567 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:40.567 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:31:40.567 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065074+0000", 2024-09-14T06:31:40.567 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.3900000000000001e-07 2024-09-14T06:31:40.567 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:40.567 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:40.567 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:31:40.567 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065078+0000", 2024-09-14T06:31:40.568 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.3229999999999999e-06 2024-09-14T06:31:40.568 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:40.568 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:40.568 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:31:40.568 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065082+0000", 2024-09-14T06:31:40.568 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.8299999999999998e-06 2024-09-14T06:31:40.568 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:40.568 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:40.568 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:31:40.568 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065085+0000", 2024-09-14T06:31:40.568 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.8660000000000002e-06 2024-09-14T06:31:40.568 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:40.568 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:40.569 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:31:40.569 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.111270+0000", 2024-09-14T06:31:40.569 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046185035999999999 2024-09-14T06:31:40.569 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:40.569 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:40.569 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:31:40.569 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.111311+0000", 2024-09-14T06:31:40.569 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.0935e-05 2024-09-14T06:31:40.569 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:40.569 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:40.569 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:31:40.569 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.111364+0000", 2024-09-14T06:31:40.570 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.2629e-05 2024-09-14T06:31:40.570 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:40.570 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:31:40.570 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:40.570 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:40.570 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:40.570 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:19367 3.3 3:c34e5e51:::rbd_data.112d92027590.0000000000000024:f [read 1323008~1024 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:31:40.570 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:27.065115+0000", 2024-09-14T06:31:40.570 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 133.43740473599999, 2024-09-14T06:31:40.570 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046407003000000002, 2024-09-14T06:31:40.570 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:31:40.570 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:31:40.571 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:31:40.571 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:31:40.571 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:31:40.571 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:40.571 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 19367 2024-09-14T06:31:40.571 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:40.571 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:31:40.571 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:40.571 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:31:40.571 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065115+0000", 2024-09-14T06:31:40.571 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:40.571 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:40.572 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:40.572 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:31:40.572 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065115+0000", 2024-09-14T06:31:40.572 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:40.572 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:40.572 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:40.572 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:31:40.572 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065116+0000", 2024-09-14T06:31:40.572 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.4999999999999998e-07 2024-09-14T06:31:40.572 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:40.572 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:40.572 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:31:40.572 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065117+0000", 2024-09-14T06:31:40.573 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.189e-06 2024-09-14T06:31:40.573 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:40.573 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:40.573 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:31:40.573 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065122+0000", 2024-09-14T06:31:40.573 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.3709999999999999e-06 2024-09-14T06:31:40.573 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:40.573 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:40.573 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:31:40.573 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065125+0000", 2024-09-14T06:31:40.573 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.8940000000000002e-06 2024-09-14T06:31:40.573 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:40.573 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:40.573 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:31:40.574 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.111402+0000", 2024-09-14T06:31:40.574 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046276645999999998 2024-09-14T06:31:40.574 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:40.574 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:40.574 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:31:40.574 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.111445+0000", 2024-09-14T06:31:40.574 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.3288000000000003e-05 2024-09-14T06:31:40.574 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:40.574 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:40.574 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:31:40.574 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.111522+0000", 2024-09-14T06:31:40.574 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.7164999999999997e-05 2024-09-14T06:31:40.574 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:40.574 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:31:40.574 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:40.575 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:40.575 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:40.575 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:19368 3.3 3:c34e5e51:::rbd_data.112d92027590.0000000000000024:f [read 1333248~1024 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:31:40.575 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:27.065130+0000", 2024-09-14T06:31:40.575 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 133.43739044599999, 2024-09-14T06:31:40.575 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046507275000000001, 2024-09-14T06:31:40.575 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:31:40.575 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:31:40.575 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:31:40.575 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:31:40.575 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:31:40.575 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:40.575 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 19368 2024-09-14T06:31:40.575 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:40.575 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:31:40.575 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:40.576 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:31:40.576 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065130+0000", 2024-09-14T06:31:40.576 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:40.576 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:40.576 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:40.576 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:31:40.576 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065130+0000", 2024-09-14T06:31:40.576 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:40.576 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:40.576 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:40.576 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:31:40.576 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065130+0000", 2024-09-14T06:31:40.576 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.0800000000000001e-07 2024-09-14T06:31:40.576 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:40.577 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:40.577 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:31:40.577 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065131+0000", 2024-09-14T06:31:40.577 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.8700000000000004e-07 2024-09-14T06:31:40.577 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:40.577 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:40.577 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:31:40.577 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065134+0000", 2024-09-14T06:31:40.577 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.1039999999999998e-06 2024-09-14T06:31:40.577 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:40.577 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:40.577 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:31:40.577 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065136+0000", 2024-09-14T06:31:40.578 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.3949999999999999e-06 2024-09-14T06:31:40.578 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:40.578 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:40.578 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:31:40.578 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.111541+0000", 2024-09-14T06:31:40.578 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046404206000000003 2024-09-14T06:31:40.578 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:40.578 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:40.578 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:31:40.578 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.111582+0000", 2024-09-14T06:31:40.578 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.1051999999999999e-05 2024-09-14T06:31:40.578 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:40.578 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:40.578 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:31:40.579 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.111637+0000", 2024-09-14T06:31:40.579 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.5322999999999999e-05 2024-09-14T06:31:40.579 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:40.579 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:31:40.579 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:40.579 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:40.579 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:40.579 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:19369 3.3 3:c34e5e51:::rbd_data.112d92027590.0000000000000024:f [read 1343488~1024 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:31:40.579 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:27.065139+0000", 2024-09-14T06:31:40.579 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 133.43738062899999, 2024-09-14T06:31:40.579 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046682169000000003, 2024-09-14T06:31:40.579 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:31:40.579 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:31:40.580 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:31:40.580 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:31:40.580 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:31:40.580 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:40.580 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 19369 2024-09-14T06:31:40.580 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:40.580 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:31:40.580 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:40.580 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:31:40.580 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065139+0000", 2024-09-14T06:31:40.580 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:40.580 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:40.580 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:40.581 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:31:40.581 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065139+0000", 2024-09-14T06:31:40.581 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:40.581 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:40.581 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:40.581 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:31:40.581 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065140+0000", 2024-09-14T06:31:40.581 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.8299999999999998e-07 2024-09-14T06:31:40.581 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:40.581 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:40.581 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:31:40.581 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065141+0000", 2024-09-14T06:31:40.581 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.3900000000000003e-07 2024-09-14T06:31:40.581 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:40.582 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:40.582 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:31:40.582 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065144+0000", 2024-09-14T06:31:40.582 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.7409999999999998e-06 2024-09-14T06:31:40.582 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:40.582 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:40.582 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:31:40.582 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065147+0000", 2024-09-14T06:31:40.582 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.4150000000000002e-06 2024-09-14T06:31:40.582 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:40.582 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:40.582 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:31:40.582 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.111673+0000", 2024-09-14T06:31:40.583 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046526004000000003 2024-09-14T06:31:40.583 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:40.583 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:40.583 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:31:40.583 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.111749+0000", 2024-09-14T06:31:40.583 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.5741999999999994e-05 2024-09-14T06:31:40.583 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:40.583 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:40.583 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:31:40.583 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.111822+0000", 2024-09-14T06:31:40.583 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.3045000000000003e-05 2024-09-14T06:31:40.583 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:40.583 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:31:40.583 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:40.583 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:40.584 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:40.584 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:19370 3.3 3:c34e5e51:::rbd_data.112d92027590.0000000000000024:f [read 1353728~1024 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:31:40.584 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:27.065150+0000", 2024-09-14T06:31:40.584 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 133.43736989999999, 2024-09-14T06:31:40.584 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046897556999999999, 2024-09-14T06:31:40.584 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:31:40.584 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:31:40.584 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:31:40.584 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:31:40.584 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:31:40.584 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:40.584 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 19370 2024-09-14T06:31:40.584 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:40.585 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:31:40.585 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:40.585 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:31:40.585 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065150+0000", 2024-09-14T06:31:40.585 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:40.585 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:40.585 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:40.585 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:31:40.585 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065150+0000", 2024-09-14T06:31:40.585 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:40.585 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:40.585 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:40.585 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:31:40.585 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065150+0000", 2024-09-14T06:31:40.586 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.0200000000000001e-07 2024-09-14T06:31:40.586 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:40.586 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:40.586 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:31:40.586 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065151+0000", 2024-09-14T06:31:40.586 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.1100000000000004e-07 2024-09-14T06:31:40.586 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:40.586 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:40.586 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:31:40.586 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065154+0000", 2024-09-14T06:31:40.586 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.9639999999999999e-06 2024-09-14T06:31:40.586 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:40.586 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:40.587 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:31:40.587 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065157+0000", 2024-09-14T06:31:40.587 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.4260000000000002e-06 2024-09-14T06:31:40.587 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:40.587 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:40.587 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:31:40.587 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.111861+0000", 2024-09-14T06:31:40.587 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046704268 2024-09-14T06:31:40.587 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:40.587 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:40.587 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:31:40.587 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.111939+0000", 2024-09-14T06:31:40.587 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.8231000000000004e-05 2024-09-14T06:31:40.587 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:40.587 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:40.588 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:31:40.588 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.112048+0000", 2024-09-14T06:31:40.588 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00010855500000000001 2024-09-14T06:31:40.588 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:40.588 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:31:40.588 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:40.588 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:40.588 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:40.588 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:19371 3.3 3:c34e5e51:::rbd_data.112d92027590.0000000000000024:f [read 1363968~1024 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:31:40.588 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:27.065161+0000", 2024-09-14T06:31:40.588 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 133.43735924999999, 2024-09-14T06:31:40.588 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047021944000000003, 2024-09-14T06:31:40.588 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:31:40.588 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:31:40.589 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:31:40.589 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:31:40.589 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:31:40.589 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:40.589 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 19371 2024-09-14T06:31:40.589 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:40.589 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:31:40.589 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:40.589 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:31:40.589 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065161+0000", 2024-09-14T06:31:40.589 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:40.589 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:40.589 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:40.589 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:31:40.590 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065161+0000", 2024-09-14T06:31:40.590 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:40.590 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:40.590 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:40.590 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:31:40.590 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065161+0000", 2024-09-14T06:31:40.590 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.0200000000000001e-07 2024-09-14T06:31:40.590 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:40.590 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:40.590 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:31:40.590 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065166+0000", 2024-09-14T06:31:40.590 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.9869999999999999e-06 2024-09-14T06:31:40.590 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:40.590 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:40.591 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:31:40.591 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065169+0000", 2024-09-14T06:31:40.591 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.3799999999999998e-06 2024-09-14T06:31:40.591 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:40.591 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:40.591 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:31:40.591 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065172+0000", 2024-09-14T06:31:40.591 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.4830000000000002e-06 2024-09-14T06:31:40.591 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:40.591 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:40.591 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:31:40.591 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.112071+0000", 2024-09-14T06:31:40.591 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046899529000000002 2024-09-14T06:31:40.591 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:40.591 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:40.591 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:31:40.592 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.112128+0000", 2024-09-14T06:31:40.592 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.6341999999999997e-05 2024-09-14T06:31:40.592 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:40.592 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:40.592 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:31:40.592 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.112183+0000", 2024-09-14T06:31:40.592 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.5021000000000001e-05 2024-09-14T06:31:40.592 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:40.592 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:31:40.592 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:40.592 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:40.592 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:40.592 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:19372 3.3 3:c34e5e51:::rbd_data.112d92027590.0000000000000024:f [read 1374208~1024 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:31:40.592 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:27.065176+0000", 2024-09-14T06:31:40.593 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 133.43734409999999, 2024-09-14T06:31:40.593 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047188889999999997, 2024-09-14T06:31:40.593 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:31:40.593 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:31:40.593 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:31:40.593 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:31:40.593 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:31:40.593 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:40.593 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 19372 2024-09-14T06:31:40.593 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:40.593 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:31:40.593 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:40.593 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:31:40.593 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065176+0000", 2024-09-14T06:31:40.594 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:40.594 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:40.594 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:40.594 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:31:40.594 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065176+0000", 2024-09-14T06:31:40.594 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:40.594 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:40.594 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:40.594 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:31:40.594 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.071117+0000", 2024-09-14T06:31:40.594 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.0059404990000000001 2024-09-14T06:31:40.594 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:40.594 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:40.594 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:31:40.594 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.071120+0000", 2024-09-14T06:31:40.595 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.4340000000000001e-06 2024-09-14T06:31:40.595 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:40.595 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:40.595 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:31:40.595 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.071129+0000", 2024-09-14T06:31:40.595 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.8389999999999996e-06 2024-09-14T06:31:40.595 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:40.595 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:40.595 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:31:40.595 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.071134+0000", 2024-09-14T06:31:40.595 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.412e-06 2024-09-14T06:31:40.595 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:40.595 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:40.595 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:31:40.595 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.112221+0000", 2024-09-14T06:31:40.595 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.041086473999999998 2024-09-14T06:31:40.596 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:40.596 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:40.596 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:31:40.596 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.112299+0000", 2024-09-14T06:31:40.596 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.8746999999999997e-05 2024-09-14T06:31:40.596 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:40.596 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:40.596 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:31:40.596 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.112365+0000", 2024-09-14T06:31:40.596 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.5485000000000001e-05 2024-09-14T06:31:40.596 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:40.596 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:31:40.596 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:40.596 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:40.596 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:31:40.597 INFO:teuthology.orchestra.run.smithi138.stdout:} 2024-09-14T06:31:40.599 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-14T06:31:40.599 INFO:teuthology.orchestra.run.smithi094.stdout:5433 write 0x4c0fbae thru 0x4c1de3a (0xe28d bytes) 2024-09-14T06:31:40.599 INFO:teuthology.orchestra.run.smithi094.stdout:5434 writesame 0x9c4fdf8 thru 0x9c57171 data_size 0x27(0x737a bytes) 2024-09-14T06:31:40.599 INFO:teuthology.orchestra.run.smithi094.stdout:5435 writesame 0x436fdf1 thru 0x437a419 data_size 0xb(0xa629 bytes) 2024-09-14T06:31:40.599 INFO:teuthology.orchestra.run.smithi094.stdout:5438 writesame 0xd6a6967 thru 0xd6adc0d data_size 0x1(0x72a7 bytes) 2024-09-14T06:31:40.599 INFO:teuthology.orchestra.run.smithi094.stdout:5439 write 0x6cb6b70 thru 0x6cbc07e (0x550f bytes) 2024-09-14T06:31:40.599 INFO:teuthology.orchestra.run.smithi094.stdout:5440 write 0x1bbcd47 thru 0x1bc695d (0x9c17 bytes) 2024-09-14T06:31:40.599 INFO:teuthology.orchestra.run.smithi094.stdout:5441 trunc from 0xd6adc0e to 0x11f7395 2024-09-14T06:31:40.600 INFO:tasks.ceph.osd.2.smithi094.stderr:2024-09-14T06:31:40.596+0000 7faf3269b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33649) UID: 0 2024-09-14T06:31:40.638 INFO:teuthology.orchestra.run.smithi094.stdout:5442 write 0xddb1e78 thru 0xddb3a71 (0x1bfa bytes) 2024-09-14T06:31:40.647 INFO:teuthology.orchestra.run.smithi094.stdout:5443 trunc from 0xddb3a72 to 0xb1564fa 2024-09-14T06:31:40.664 INFO:teuthology.orchestra.run.smithi094.stdout:5444 write 0xbf787cf thru 0xbf81561 (0x8d93 bytes) 2024-09-14T06:31:40.672 INFO:teuthology.orchestra.run.smithi094.stdout:5446 writesame 0x83986c6 thru 0x839f8e5 data_size 0x16(0x7220 bytes) 2024-09-14T06:31:40.699 INFO:tasks.ceph.osd.0.smithi094.stderr:2024-09-14T06:31:40.697+0000 7f07c9333640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33636) UID: 0 2024-09-14T06:31:40.748 INFO:teuthology.orchestra.run.smithi138.stdout:{ 2024-09-14T06:31:40.749 INFO:teuthology.orchestra.run.smithi138.stdout: "ops": [], 2024-09-14T06:31:40.749 INFO:teuthology.orchestra.run.smithi138.stdout: "num_ops": 0 2024-09-14T06:31:40.749 INFO:teuthology.orchestra.run.smithi138.stdout:} 2024-09-14T06:31:40.757 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-14T06:31:40.800 INFO:tasks.ceph.osd.1.smithi094.stderr:2024-09-14T06:31:40.799+0000 7f328e47a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33641) UID: 0 2024-09-14T06:31:40.860 INFO:teuthology.orchestra.run.smithi138.stdout:{ 2024-09-14T06:31:40.860 INFO:teuthology.orchestra.run.smithi138.stdout: "ops": [], 2024-09-14T06:31:40.860 INFO:teuthology.orchestra.run.smithi138.stdout: "complaint_time": 30, 2024-09-14T06:31:40.861 INFO:teuthology.orchestra.run.smithi138.stdout: "num_blocked_ops": 0 2024-09-14T06:31:40.861 INFO:teuthology.orchestra.run.smithi138.stdout:} 2024-09-14T06:31:40.869 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-14T06:31:40.901 INFO:tasks.ceph.osd.0.smithi094.stderr:2024-09-14T06:31:40.899+0000 7f07c9333640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33636) UID: 0 2024-09-14T06:31:40.969 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-14T06:31:40.973 INFO:teuthology.orchestra.run.smithi138.stdout:{ 2024-09-14T06:31:40.973 INFO:teuthology.orchestra.run.smithi138.stdout: "size": 20, 2024-09-14T06:31:40.973 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 600, 2024-09-14T06:31:40.973 INFO:teuthology.orchestra.run.smithi138.stdout: "ops": [ 2024-09-14T06:31:40.973 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:40.973 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:11453 3.1d 3:bcad7303:::rbd_data.112d92027590.0000000000000001:f [read 94208~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:31:40.973 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:25.654375+0000", 2024-09-14T06:31:40.973 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 135.31687929899999, 2024-09-14T06:31:40.973 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052449318000000002, 2024-09-14T06:31:40.974 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:31:40.974 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:31:40.974 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:31:40.974 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:31:40.974 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:31:40.974 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:40.974 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 11453 2024-09-14T06:31:40.974 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:40.974 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:31:40.974 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:40.975 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:31:40.975 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654375+0000", 2024-09-14T06:31:40.975 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:40.975 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:40.975 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:40.975 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:31:40.975 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654375+0000", 2024-09-14T06:31:40.975 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:40.975 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:40.975 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:40.975 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:31:40.975 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654496+0000", 2024-09-14T06:31:40.975 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00012081 2024-09-14T06:31:40.976 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:40.976 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:40.976 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:31:40.976 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654497+0000", 2024-09-14T06:31:40.976 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.8789999999999999e-06 2024-09-14T06:31:40.976 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:40.976 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:40.976 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:31:40.976 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654501+0000", 2024-09-14T06:31:40.976 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.8999999999999999e-06 2024-09-14T06:31:40.976 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:40.976 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:40.976 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:31:40.977 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654505+0000", 2024-09-14T06:31:40.977 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.2229999999999999e-06 2024-09-14T06:31:40.977 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:40.977 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:40.977 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:31:40.977 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.705917+0000", 2024-09-14T06:31:40.977 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.051412501999999999 2024-09-14T06:31:40.977 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:40.977 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:40.978 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:31:40.978 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.705999+0000", 2024-09-14T06:31:40.978 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.1551999999999994e-05 2024-09-14T06:31:40.978 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:40.978 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:40.978 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:31:40.978 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.706824+0000", 2024-09-14T06:31:40.978 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00082545199999999996 2024-09-14T06:31:40.978 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:40.978 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:31:40.978 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:40.978 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:40.979 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:40.979 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:11454 3.1d 3:bcad7303:::rbd_data.112d92027590.0000000000000001:f [read 104448~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:31:40.979 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:25.654510+0000", 2024-09-14T06:31:40.979 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 135.31674450899999, 2024-09-14T06:31:40.979 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052610713000000003, 2024-09-14T06:31:40.979 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:31:40.979 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:31:40.979 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:31:40.979 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:31:40.979 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:31:40.979 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:40.979 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 11454 2024-09-14T06:31:40.979 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:40.980 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:31:40.980 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:40.980 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:31:40.980 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654510+0000", 2024-09-14T06:31:40.980 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:40.980 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:40.980 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:40.980 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:31:40.980 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654510+0000", 2024-09-14T06:31:40.980 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:40.981 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:40.981 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:40.981 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:31:40.981 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654825+0000", 2024-09-14T06:31:40.981 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00031522400000000002 2024-09-14T06:31:40.981 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:40.981 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:40.981 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:31:40.981 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654827+0000", 2024-09-14T06:31:40.981 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.9709999999999998e-06 2024-09-14T06:31:40.981 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:40.981 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:40.981 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:31:40.981 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654832+0000", 2024-09-14T06:31:40.982 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.4099999999999999e-06 2024-09-14T06:31:40.982 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:40.982 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:40.982 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:31:40.982 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654835+0000", 2024-09-14T06:31:40.982 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.14e-06 2024-09-14T06:31:40.982 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:40.982 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:40.982 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:31:40.982 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.706851+0000", 2024-09-14T06:31:40.982 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052015761000000001 2024-09-14T06:31:40.982 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:40.982 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:40.982 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:31:40.983 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.706934+0000", 2024-09-14T06:31:40.983 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.2484999999999994e-05 2024-09-14T06:31:40.983 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:40.983 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:40.983 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:31:40.983 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.707120+0000", 2024-09-14T06:31:40.983 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00018672200000000001 2024-09-14T06:31:40.983 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:40.983 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:31:40.983 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:40.983 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:40.983 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:40.983 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:11455 3.1d 3:bcad7303:::rbd_data.112d92027590.0000000000000001:f [read 114688~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:31:40.983 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:25.654841+0000", 2024-09-14T06:31:40.984 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 135.31641318000001, 2024-09-14T06:31:40.984 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052579898, 2024-09-14T06:31:40.984 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:31:40.984 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:31:40.984 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:31:40.984 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:31:40.984 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:31:40.984 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:40.984 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 11455 2024-09-14T06:31:40.984 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:40.984 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:31:40.984 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:40.984 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:31:40.985 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654841+0000", 2024-09-14T06:31:40.985 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:40.985 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:40.985 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:40.985 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:31:40.985 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654841+0000", 2024-09-14T06:31:40.985 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:40.985 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:40.985 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:40.986 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:31:40.986 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654841+0000", 2024-09-14T06:31:40.986 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.7300000000000002e-07 2024-09-14T06:31:40.986 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:40.986 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:40.986 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:31:40.986 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654845+0000", 2024-09-14T06:31:40.987 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.0520000000000003e-06 2024-09-14T06:31:40.987 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:40.987 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:40.987 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:31:40.987 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654849+0000", 2024-09-14T06:31:40.987 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.0459999999999999e-06 2024-09-14T06:31:40.987 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:40.988 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:40.988 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:31:40.988 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654852+0000", 2024-09-14T06:31:40.988 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.9830000000000001e-06 2024-09-14T06:31:40.988 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:40.988 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:40.988 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:31:40.989 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.707159+0000", 2024-09-14T06:31:40.989 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052306826000000001 2024-09-14T06:31:40.989 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:40.989 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:40.989 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:31:40.989 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.707242+0000", 2024-09-14T06:31:40.989 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.2595000000000005e-05 2024-09-14T06:31:40.989 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:40.989 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:40.989 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:31:40.990 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.707421+0000", 2024-09-14T06:31:40.990 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.000179123 2024-09-14T06:31:40.990 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:40.990 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:31:40.990 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:40.990 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:40.990 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:40.990 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:11456 3.1d 3:bcad7303:::rbd_data.112d92027590.0000000000000001:f [read 124928~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:31:40.990 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:25.654856+0000", 2024-09-14T06:31:40.990 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 135.31639833400001, 2024-09-14T06:31:40.990 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052801580000000001, 2024-09-14T06:31:40.991 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:31:40.991 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:31:40.991 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:31:40.991 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:31:40.991 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:31:40.991 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:40.991 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 11456 2024-09-14T06:31:40.991 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:40.991 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:31:40.991 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:40.991 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:31:40.991 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654856+0000", 2024-09-14T06:31:40.992 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:40.992 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:40.992 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:40.992 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:31:40.992 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654856+0000", 2024-09-14T06:31:40.992 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:40.992 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:40.992 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:40.992 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:31:40.993 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654856+0000", 2024-09-14T06:31:40.993 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.29e-07 2024-09-14T06:31:40.993 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:40.993 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:40.993 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:31:40.993 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654857+0000", 2024-09-14T06:31:40.993 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.187e-06 2024-09-14T06:31:40.993 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:40.993 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:40.993 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:31:40.994 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654861+0000", 2024-09-14T06:31:40.994 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.3850000000000002e-06 2024-09-14T06:31:40.994 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:40.994 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:40.994 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:31:40.994 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654863+0000", 2024-09-14T06:31:40.994 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.9500000000000001e-06 2024-09-14T06:31:40.994 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:40.994 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:40.994 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:31:40.994 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.707456+0000", 2024-09-14T06:31:40.994 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052592434 2024-09-14T06:31:40.994 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:40.995 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:40.995 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:31:40.995 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.707516+0000", 2024-09-14T06:31:40.995 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.0167999999999997e-05 2024-09-14T06:31:40.995 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:40.995 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:40.995 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:31:40.995 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.707657+0000", 2024-09-14T06:31:40.995 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00014122700000000001 2024-09-14T06:31:40.995 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:40.995 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:31:40.995 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:40.996 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:40.996 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:40.996 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:11457 3.1d 3:bcad7303:::rbd_data.112d92027590.0000000000000001:f [read 135168~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:31:40.996 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:25.654867+0000", 2024-09-14T06:31:40.996 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 135.31638744899999, 2024-09-14T06:31:40.996 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052921902, 2024-09-14T06:31:40.996 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:31:40.996 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:31:40.996 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:31:40.996 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:31:40.996 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:31:40.996 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:40.996 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 11457 2024-09-14T06:31:40.997 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:40.997 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:31:40.997 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:40.997 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:31:40.997 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654867+0000", 2024-09-14T06:31:40.997 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:40.997 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:40.997 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:40.997 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:31:40.997 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654867+0000", 2024-09-14T06:31:40.997 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:40.997 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:40.997 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:40.997 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:31:40.998 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654972+0000", 2024-09-14T06:31:40.998 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00010584 2024-09-14T06:31:40.998 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:40.998 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:40.998 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:31:40.998 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654975+0000", 2024-09-14T06:31:40.998 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.1169999999999998e-06 2024-09-14T06:31:40.998 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:40.998 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:40.998 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:31:40.998 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654979+0000", 2024-09-14T06:31:40.998 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.1960000000000001e-06 2024-09-14T06:31:40.998 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:40.998 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:40.999 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:31:40.999 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654982+0000", 2024-09-14T06:31:40.999 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.2990000000000001e-06 2024-09-14T06:31:40.999 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:40.999 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:40.999 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:31:40.999 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.707684+0000", 2024-09-14T06:31:40.999 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052701979000000003 2024-09-14T06:31:40.999 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:40.999 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:40.999 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:31:40.999 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.707728+0000", 2024-09-14T06:31:40.999 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.3529999999999998e-05 2024-09-14T06:31:41.000 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:41.000 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:41.000 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:31:41.000 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.707789+0000", 2024-09-14T06:31:41.000 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.0940999999999998e-05 2024-09-14T06:31:41.000 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:41.000 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:31:41.000 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:41.000 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:41.001 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:41.001 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:11458 3.1d 3:bcad7303:::rbd_data.112d92027590.0000000000000001:f [read 145408~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:31:41.001 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:25.654987+0000", 2024-09-14T06:31:41.001 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 135.31626705100001, 2024-09-14T06:31:41.001 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.05291655, 2024-09-14T06:31:41.001 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:31:41.001 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:31:41.001 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:31:41.001 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:31:41.002 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:31:41.002 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:41.002 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 11458 2024-09-14T06:31:41.002 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:41.002 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:31:41.002 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:41.002 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:31:41.002 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654987+0000", 2024-09-14T06:31:41.003 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:41.003 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:41.003 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:41.003 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:31:41.003 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654987+0000", 2024-09-14T06:31:41.003 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:41.003 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:41.003 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:41.003 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:31:41.004 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655077+0000", 2024-09-14T06:31:41.004 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.0253000000000005e-05 2024-09-14T06:31:41.004 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:41.004 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:41.004 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:31:41.004 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655079+0000", 2024-09-14T06:31:41.004 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.9989999999999998e-06 2024-09-14T06:31:41.005 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:41.005 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:41.005 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:31:41.005 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655083+0000", 2024-09-14T06:31:41.005 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.9280000000000003e-06 2024-09-14T06:31:41.005 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:41.005 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:41.006 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:31:41.006 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655086+0000", 2024-09-14T06:31:41.006 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.056e-06 2024-09-14T06:31:41.006 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:41.006 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:41.006 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:31:41.006 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.707805+0000", 2024-09-14T06:31:41.007 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052718627999999997 2024-09-14T06:31:41.007 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:41.007 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:41.007 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:31:41.007 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.707848+0000", 2024-09-14T06:31:41.007 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.2728999999999997e-05 2024-09-14T06:31:41.008 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:41.008 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:41.008 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:31:41.008 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.707904+0000", 2024-09-14T06:31:41.008 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.5957e-05 2024-09-14T06:31:41.008 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:41.008 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:31:41.009 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:41.009 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:41.009 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:41.009 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:11459 3.1d 3:bcad7303:::rbd_data.112d92027590.0000000000000001:f [read 155648~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:31:41.009 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:25.655091+0000", 2024-09-14T06:31:41.009 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 135.31616304299999, 2024-09-14T06:31:41.010 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052944242000000002, 2024-09-14T06:31:41.010 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:31:41.010 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:31:41.010 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:31:41.010 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:31:41.010 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:31:41.010 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:41.010 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 11459 2024-09-14T06:31:41.010 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:41.010 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:31:41.010 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:41.010 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:31:41.011 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655091+0000", 2024-09-14T06:31:41.011 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:41.011 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:41.011 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:41.011 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:31:41.011 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655091+0000", 2024-09-14T06:31:41.011 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:41.011 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:41.011 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:41.011 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:31:41.011 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655237+0000", 2024-09-14T06:31:41.011 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00014570100000000001 2024-09-14T06:31:41.011 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:41.011 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:41.012 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:31:41.012 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655239+0000", 2024-09-14T06:31:41.012 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.8670000000000001e-06 2024-09-14T06:31:41.012 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:41.012 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:41.012 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:31:41.012 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655243+0000", 2024-09-14T06:31:41.012 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.9829999999999998e-06 2024-09-14T06:31:41.012 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:41.012 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:41.012 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:31:41.012 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655246+0000", 2024-09-14T06:31:41.012 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.1540000000000002e-06 2024-09-14T06:31:41.013 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:41.013 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:41.013 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:31:41.013 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.707934+0000", 2024-09-14T06:31:41.013 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052688286000000001 2024-09-14T06:31:41.013 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:41.013 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:41.013 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:31:41.013 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.707977+0000", 2024-09-14T06:31:41.013 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.3291999999999998e-05 2024-09-14T06:31:41.013 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:41.013 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:41.013 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:31:41.014 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.708035+0000", 2024-09-14T06:31:41.014 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.7958999999999999e-05 2024-09-14T06:31:41.014 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:41.014 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:31:41.014 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:41.014 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:41.014 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:41.014 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:11460 3.1d 3:bcad7303:::rbd_data.112d92027590.0000000000000001:f [read 165888~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:31:41.014 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:25.655250+0000", 2024-09-14T06:31:41.014 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 135.31600374600001, 2024-09-14T06:31:41.014 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052910698999999999, 2024-09-14T06:31:41.014 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:31:41.014 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:31:41.014 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:31:41.015 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:31:41.015 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:31:41.015 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:41.015 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 11460 2024-09-14T06:31:41.015 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:41.015 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:31:41.015 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:41.015 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:31:41.015 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655250+0000", 2024-09-14T06:31:41.015 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:41.015 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:41.015 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:41.015 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:31:41.016 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655250+0000", 2024-09-14T06:31:41.016 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:41.016 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:41.016 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:41.016 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:31:41.016 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655356+0000", 2024-09-14T06:31:41.016 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00010535900000000001 2024-09-14T06:31:41.016 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:41.016 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:41.016 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:31:41.016 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655357+0000", 2024-09-14T06:31:41.016 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.8080000000000001e-06 2024-09-14T06:31:41.016 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:41.017 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:41.017 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:31:41.017 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655361+0000", 2024-09-14T06:31:41.017 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.861e-06 2024-09-14T06:31:41.017 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:41.017 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:41.017 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:31:41.017 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655364+0000", 2024-09-14T06:31:41.017 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.1089999999999998e-06 2024-09-14T06:31:41.017 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:41.017 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:41.017 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:31:41.017 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.708065+0000", 2024-09-14T06:31:41.018 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052700595000000003 2024-09-14T06:31:41.018 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:41.018 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:41.018 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:31:41.018 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.708108+0000", 2024-09-14T06:31:41.018 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.2484999999999997e-05 2024-09-14T06:31:41.018 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:41.018 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:41.018 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:31:41.018 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.708161+0000", 2024-09-14T06:31:41.018 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.3482000000000001e-05 2024-09-14T06:31:41.018 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:41.018 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:31:41.018 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:41.019 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:41.019 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:41.019 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:11461 3.1d 3:bcad7303:::rbd_data.112d92027590.0000000000000001:f [read 176128~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:31:41.019 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:25.655370+0000", 2024-09-14T06:31:41.019 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 135.31588446399999, 2024-09-14T06:31:41.019 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052915141999999998, 2024-09-14T06:31:41.019 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:31:41.019 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:31:41.019 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:31:41.019 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:31:41.019 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:31:41.019 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:41.019 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 11461 2024-09-14T06:31:41.019 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:41.020 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:31:41.020 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:41.020 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:31:41.020 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655370+0000", 2024-09-14T06:31:41.020 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:41.020 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:41.020 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:41.020 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:31:41.020 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655370+0000", 2024-09-14T06:31:41.020 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:41.020 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:41.020 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:41.021 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:31:41.021 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655394+0000", 2024-09-14T06:31:41.021 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.4553999999999999e-05 2024-09-14T06:31:41.021 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:41.021 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:41.021 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:31:41.021 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655396+0000", 2024-09-14T06:31:41.021 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.8950000000000001e-06 2024-09-14T06:31:41.021 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:41.021 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:41.021 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:31:41.021 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655401+0000", 2024-09-14T06:31:41.021 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.0089999999999998e-06 2024-09-14T06:31:41.022 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:41.022 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:41.022 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:31:41.022 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655404+0000", 2024-09-14T06:31:41.022 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.3400000000000002e-06 2024-09-14T06:31:41.022 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:41.022 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:41.022 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:31:41.022 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.708186+0000", 2024-09-14T06:31:41.022 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052781726000000001 2024-09-14T06:31:41.022 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:41.022 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:41.023 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:31:41.023 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.708229+0000", 2024-09-14T06:31:41.023 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.2670999999999999e-05 2024-09-14T06:31:41.023 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:41.023 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:41.023 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:31:41.023 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.708285+0000", 2024-09-14T06:31:41.023 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.5946999999999998e-05 2024-09-14T06:31:41.023 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:41.023 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:31:41.023 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:41.023 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:41.023 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:41.023 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:11462 3.1d 3:bcad7303:::rbd_data.112d92027590.0000000000000001:f [read 186368~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:31:41.024 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:25.655410+0000", 2024-09-14T06:31:41.024 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 135.315843986, 2024-09-14T06:31:41.024 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052995258000000003, 2024-09-14T06:31:41.024 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:31:41.024 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:31:41.024 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:31:41.024 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:31:41.024 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:31:41.024 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:41.024 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 11462 2024-09-14T06:31:41.024 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:41.024 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:31:41.024 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:41.024 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:31:41.025 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655410+0000", 2024-09-14T06:31:41.025 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:41.025 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:41.025 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:41.025 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:31:41.025 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655410+0000", 2024-09-14T06:31:41.025 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:41.025 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:41.025 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:41.025 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:31:41.025 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655625+0000", 2024-09-14T06:31:41.025 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00021442600000000001 2024-09-14T06:31:41.025 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:41.026 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:41.026 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:31:41.026 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655626+0000", 2024-09-14T06:31:41.026 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.815e-06 2024-09-14T06:31:41.026 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:41.026 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:41.026 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:31:41.026 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655631+0000", 2024-09-14T06:31:41.026 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.651e-06 2024-09-14T06:31:41.026 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:41.026 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:41.026 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:31:41.026 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655634+0000", 2024-09-14T06:31:41.027 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.2449999999999998e-06 2024-09-14T06:31:41.027 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:41.027 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:41.027 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:31:41.027 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.708312+0000", 2024-09-14T06:31:41.027 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052677412 2024-09-14T06:31:41.027 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:41.027 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:41.027 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:31:41.027 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.708352+0000", 2024-09-14T06:31:41.027 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.0729999999999998e-05 2024-09-14T06:31:41.027 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:41.027 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:41.028 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:31:41.028 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.708405+0000", 2024-09-14T06:31:41.028 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.2979000000000002e-05 2024-09-14T06:31:41.028 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:41.028 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:31:41.028 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:41.028 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:41.028 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:41.028 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:11463 3.1d 3:bcad7303:::rbd_data.112d92027590.0000000000000001:f [read 196608~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:31:41.028 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:25.655639+0000", 2024-09-14T06:31:41.028 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 135.31561483999999, 2024-09-14T06:31:41.028 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052888859000000003, 2024-09-14T06:31:41.028 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:31:41.028 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:31:41.029 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:31:41.029 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:31:41.029 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:31:41.029 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:41.029 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 11463 2024-09-14T06:31:41.029 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:41.029 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:31:41.029 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:41.029 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:31:41.029 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655639+0000", 2024-09-14T06:31:41.029 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:41.029 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:41.029 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:41.029 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:31:41.030 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655639+0000", 2024-09-14T06:31:41.030 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:41.030 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:41.030 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:41.030 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:31:41.030 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655640+0000", 2024-09-14T06:31:41.030 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.08e-07 2024-09-14T06:31:41.030 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:41.030 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:41.030 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:31:41.030 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655641+0000", 2024-09-14T06:31:41.030 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.156e-06 2024-09-14T06:31:41.030 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:41.031 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:41.031 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:31:41.031 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655648+0000", 2024-09-14T06:31:41.031 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.8460000000000002e-06 2024-09-14T06:31:41.031 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:41.031 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:41.031 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:31:41.031 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655651+0000", 2024-09-14T06:31:41.031 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.0079999999999998e-06 2024-09-14T06:31:41.031 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:41.031 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:41.031 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:31:41.031 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.708429+0000", 2024-09-14T06:31:41.032 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052778459 2024-09-14T06:31:41.032 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:41.032 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:41.032 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:31:41.032 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.708473+0000", 2024-09-14T06:31:41.032 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.3782999999999997e-05 2024-09-14T06:31:41.032 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:41.032 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:41.032 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:31:41.032 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.708528+0000", 2024-09-14T06:31:41.032 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.5198999999999998e-05 2024-09-14T06:31:41.032 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:41.033 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:31:41.033 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:41.033 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:41.033 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:41.033 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:11464 3.1d 3:bcad7303:::rbd_data.112d92027590.0000000000000001:f [read 206848~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:31:41.033 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:25.655654+0000", 2024-09-14T06:31:41.033 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 135.31559997100001, 2024-09-14T06:31:41.033 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052981845999999999, 2024-09-14T06:31:41.033 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:31:41.033 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:31:41.033 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:31:41.033 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:31:41.033 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:31:41.034 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:41.034 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 11464 2024-09-14T06:31:41.034 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:41.034 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:31:41.034 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:41.034 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:31:41.034 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655654+0000", 2024-09-14T06:31:41.034 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:41.034 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:41.034 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:41.034 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:31:41.034 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655654+0000", 2024-09-14T06:31:41.034 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:41.035 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:41.035 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:41.035 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:31:41.035 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655946+0000", 2024-09-14T06:31:41.035 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00029148200000000003 2024-09-14T06:31:41.035 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:41.035 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:41.035 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:31:41.035 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655948+0000", 2024-09-14T06:31:41.035 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.973e-06 2024-09-14T06:31:41.035 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:41.035 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:41.035 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:31:41.035 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655953+0000", 2024-09-14T06:31:41.036 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.1240000000000004e-06 2024-09-14T06:31:41.036 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:41.036 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:41.036 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:31:41.036 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655956+0000", 2024-09-14T06:31:41.036 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.315e-06 2024-09-14T06:31:41.036 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:41.036 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:41.036 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:31:41.036 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.708543+0000", 2024-09-14T06:31:41.036 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052587416999999997 2024-09-14T06:31:41.036 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:41.036 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:41.037 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:31:41.037 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.708584+0000", 2024-09-14T06:31:41.037 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.1004000000000002e-05 2024-09-14T06:31:41.037 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:41.037 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:41.037 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:31:41.037 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.708636+0000", 2024-09-14T06:31:41.037 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.1530999999999998e-05 2024-09-14T06:31:41.037 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:41.037 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:31:41.037 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:41.037 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:41.037 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:41.037 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:11465 3.1d 3:bcad7303:::rbd_data.112d92027590.0000000000000001:f [read 217088~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:31:41.038 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:25.655961+0000", 2024-09-14T06:31:41.038 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 135.31529281900001, 2024-09-14T06:31:41.038 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052798682999999999, 2024-09-14T06:31:41.038 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:31:41.038 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:31:41.038 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:31:41.038 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:31:41.038 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:31:41.038 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:41.038 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 11465 2024-09-14T06:31:41.038 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:41.038 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:31:41.038 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:41.039 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:31:41.039 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655961+0000", 2024-09-14T06:31:41.039 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:41.039 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:41.039 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:41.039 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:31:41.039 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655961+0000", 2024-09-14T06:31:41.039 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:41.039 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:41.039 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:41.039 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:31:41.039 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.656114+0000", 2024-09-14T06:31:41.039 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.000152745 2024-09-14T06:31:41.039 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:41.040 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:41.040 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:31:41.040 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.656116+0000", 2024-09-14T06:31:41.040 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.804e-06 2024-09-14T06:31:41.040 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:41.040 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:41.040 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:31:41.040 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.656120+0000", 2024-09-14T06:31:41.040 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.4700000000000004e-06 2024-09-14T06:31:41.040 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:41.040 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:41.040 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:31:41.040 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.656123+0000", 2024-09-14T06:31:41.041 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.1269999999999999e-06 2024-09-14T06:31:41.041 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:41.041 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:41.041 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:31:41.041 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.708660+0000", 2024-09-14T06:31:41.041 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052536919000000001 2024-09-14T06:31:41.041 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:41.041 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:41.041 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:31:41.041 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.708703+0000", 2024-09-14T06:31:41.041 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.3077000000000001e-05 2024-09-14T06:31:41.041 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:41.041 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:41.042 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:31:41.042 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.708760+0000", 2024-09-14T06:31:41.042 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.6541e-05 2024-09-14T06:31:41.042 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:41.042 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:31:41.042 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:41.042 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:41.042 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:41.042 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:11466 3.1d 3:bcad7303:::rbd_data.112d92027590.0000000000000001:f [read 227328~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:31:41.042 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:25.656129+0000", 2024-09-14T06:31:41.042 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 135.31512543100001, 2024-09-14T06:31:41.042 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052750987999999999, 2024-09-14T06:31:41.042 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:31:41.042 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:31:41.043 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:31:41.043 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:31:41.043 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:31:41.043 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:41.043 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 11466 2024-09-14T06:31:41.043 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:41.043 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:31:41.043 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:41.043 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:31:41.043 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.656129+0000", 2024-09-14T06:31:41.043 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:41.043 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:41.043 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:41.044 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:31:41.044 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.656129+0000", 2024-09-14T06:31:41.044 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:41.044 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:41.044 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:41.044 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:31:41.044 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.656347+0000", 2024-09-14T06:31:41.044 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.000218054 2024-09-14T06:31:41.044 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:41.044 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:41.044 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:31:41.044 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.656349+0000", 2024-09-14T06:31:41.045 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.9199999999999998e-06 2024-09-14T06:31:41.045 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:41.045 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:41.045 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:31:41.045 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.656353+0000", 2024-09-14T06:31:41.045 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.9210000000000002e-06 2024-09-14T06:31:41.045 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:41.045 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:41.045 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:31:41.045 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.656356+0000", 2024-09-14T06:31:41.045 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.7670000000000001e-06 2024-09-14T06:31:41.045 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:41.045 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:41.045 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:31:41.046 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.708787+0000", 2024-09-14T06:31:41.046 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052430840999999999 2024-09-14T06:31:41.046 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:41.046 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:41.046 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:31:41.046 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.708827+0000", 2024-09-14T06:31:41.046 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.9538e-05 2024-09-14T06:31:41.046 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:41.046 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:41.046 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:31:41.046 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.708880+0000", 2024-09-14T06:31:41.046 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.2947e-05 2024-09-14T06:31:41.046 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:41.047 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:31:41.047 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:41.047 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:41.047 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:41.047 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:11467 3.1d 3:bcad7303:::rbd_data.112d92027590.0000000000000001:f [read 237568~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:31:41.047 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:25.656361+0000", 2024-09-14T06:31:41.047 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 135.314892783, 2024-09-14T06:31:41.047 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052643898000000001, 2024-09-14T06:31:41.047 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:31:41.047 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:31:41.047 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:31:41.047 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:31:41.047 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:31:41.048 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:41.048 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 11467 2024-09-14T06:31:41.048 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:41.048 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:31:41.048 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:41.048 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:31:41.048 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.656361+0000", 2024-09-14T06:31:41.048 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:41.048 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:41.048 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:41.048 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:31:41.048 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.656361+0000", 2024-09-14T06:31:41.048 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:41.048 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:41.049 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:41.049 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:31:41.049 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.656855+0000", 2024-09-14T06:31:41.049 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00049372000000000001 2024-09-14T06:31:41.049 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:41.049 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:41.049 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:31:41.049 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.656859+0000", 2024-09-14T06:31:41.049 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.1300000000000003e-06 2024-09-14T06:31:41.049 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:41.049 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:41.049 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:31:41.049 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.656863+0000", 2024-09-14T06:31:41.050 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.2629999999999997e-06 2024-09-14T06:31:41.050 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:41.050 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:41.050 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:31:41.050 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.656867+0000", 2024-09-14T06:31:41.050 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.3220000000000001e-06 2024-09-14T06:31:41.050 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:41.050 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:41.050 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:31:41.050 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.708903+0000", 2024-09-14T06:31:41.050 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052035855999999998 2024-09-14T06:31:41.050 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:41.050 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:41.050 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:31:41.051 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.708949+0000", 2024-09-14T06:31:41.051 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.6276999999999998e-05 2024-09-14T06:31:41.051 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:41.051 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:41.051 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:31:41.051 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.709005+0000", 2024-09-14T06:31:41.051 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.6329999999999998e-05 2024-09-14T06:31:41.051 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:41.051 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:31:41.051 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:41.051 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:41.051 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:41.051 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:11480 3.1d 3:bcad7303:::rbd_data.112d92027590.0000000000000001:f [read 370688~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:31:41.051 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:25.658563+0000", 2024-09-14T06:31:41.052 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 135.312691019, 2024-09-14T06:31:41.052 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052429164, 2024-09-14T06:31:41.052 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:31:41.052 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:31:41.052 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:31:41.052 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:31:41.052 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:31:41.052 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:41.052 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 11480 2024-09-14T06:31:41.052 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:41.052 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:31:41.052 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:41.052 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:31:41.052 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.658563+0000", 2024-09-14T06:31:41.053 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:41.053 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:41.053 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:41.053 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:31:41.053 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.658563+0000", 2024-09-14T06:31:41.053 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:41.053 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:41.053 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:41.053 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:31:41.053 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.658658+0000", 2024-09-14T06:31:41.053 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.5346999999999994e-05 2024-09-14T06:31:41.053 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:41.054 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:41.054 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:31:41.054 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.658662+0000", 2024-09-14T06:31:41.054 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.14e-06 2024-09-14T06:31:41.054 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:41.054 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:41.054 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:31:41.054 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.658669+0000", 2024-09-14T06:31:41.054 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.7379999999999994e-06 2024-09-14T06:31:41.054 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:41.054 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:41.054 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:31:41.054 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.658675+0000", 2024-09-14T06:31:41.055 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.5430000000000001e-06 2024-09-14T06:31:41.055 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:41.055 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:41.055 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:31:41.055 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.710824+0000", 2024-09-14T06:31:41.055 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052148847999999998 2024-09-14T06:31:41.055 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:41.055 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:41.055 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:31:41.055 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.710864+0000", 2024-09-14T06:31:41.055 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.9925999999999997e-05 2024-09-14T06:31:41.055 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:41.055 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:41.056 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:31:41.056 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.710992+0000", 2024-09-14T06:31:41.056 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.000128622 2024-09-14T06:31:41.056 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:41.056 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:31:41.056 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:41.056 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:41.056 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:41.056 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:11481 3.1d 3:bcad7303:::rbd_data.112d92027590.0000000000000001:f [read 380928~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:31:41.056 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:25.658684+0000", 2024-09-14T06:31:41.056 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 135.31256994899999, 2024-09-14T06:31:41.056 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052508565, 2024-09-14T06:31:41.056 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:31:41.057 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:31:41.057 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:31:41.057 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:31:41.057 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:31:41.057 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:41.057 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 11481 2024-09-14T06:31:41.057 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:41.057 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:31:41.057 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:41.057 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:31:41.057 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.658684+0000", 2024-09-14T06:31:41.057 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:41.057 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:41.058 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:41.058 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:31:41.058 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.658684+0000", 2024-09-14T06:31:41.058 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:41.058 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:41.058 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:41.058 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:31:41.058 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.658715+0000", 2024-09-14T06:31:41.058 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.0703000000000001e-05 2024-09-14T06:31:41.058 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:41.058 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:41.058 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:31:41.058 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.658718+0000", 2024-09-14T06:31:41.058 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.0709999999999999e-06 2024-09-14T06:31:41.059 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:41.059 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:41.059 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:31:41.059 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.658725+0000", 2024-09-14T06:31:41.059 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.2339999999999997e-06 2024-09-14T06:31:41.059 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:41.059 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:41.059 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:31:41.059 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.658731+0000", 2024-09-14T06:31:41.059 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.6450000000000002e-06 2024-09-14T06:31:41.059 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:41.059 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:41.059 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:31:41.060 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.711020+0000", 2024-09-14T06:31:41.060 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052288886 2024-09-14T06:31:41.060 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:41.060 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:41.060 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:31:41.060 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.711061+0000", 2024-09-14T06:31:41.060 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.1084000000000002e-05 2024-09-14T06:31:41.060 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:41.060 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:41.060 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:31:41.060 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.711193+0000", 2024-09-14T06:31:41.060 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00013194199999999999 2024-09-14T06:31:41.060 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:41.061 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:31:41.061 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:41.061 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:41.061 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:41.061 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:11482 3.1d 3:bcad7303:::rbd_data.112d92027590.0000000000000001:f [read 391168~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:31:41.061 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:25.658740+0000", 2024-09-14T06:31:41.061 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 135.31251448399999, 2024-09-14T06:31:41.061 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052637015000000002, 2024-09-14T06:31:41.061 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:31:41.061 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:31:41.061 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:31:41.061 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:31:41.061 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:31:41.062 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:41.062 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 11482 2024-09-14T06:31:41.062 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:41.062 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:31:41.062 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:41.062 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:31:41.062 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.658740+0000", 2024-09-14T06:31:41.062 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:41.062 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:41.062 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:41.062 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:31:41.062 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.658740+0000", 2024-09-14T06:31:41.062 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:41.062 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:41.063 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:41.063 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:31:41.063 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.658880+0000", 2024-09-14T06:31:41.063 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00014036100000000001 2024-09-14T06:31:41.063 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:41.063 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:41.063 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:31:41.063 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.658883+0000", 2024-09-14T06:31:41.063 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.3969999999999998e-06 2024-09-14T06:31:41.063 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:41.063 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:41.063 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:31:41.063 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.658890+0000", 2024-09-14T06:31:41.063 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.1500000000000002e-06 2024-09-14T06:31:41.064 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:41.064 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:41.064 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:31:41.064 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.658897+0000", 2024-09-14T06:31:41.064 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.1260000000000003e-06 2024-09-14T06:31:41.064 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:41.064 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:41.064 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:31:41.064 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.711207+0000", 2024-09-14T06:31:41.064 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052310777000000003 2024-09-14T06:31:41.064 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:41.064 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:41.064 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:31:41.065 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.711249+0000", 2024-09-14T06:31:41.065 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.1786999999999998e-05 2024-09-14T06:31:41.065 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:41.065 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:41.065 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:31:41.065 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.711377+0000", 2024-09-14T06:31:41.065 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.000127417 2024-09-14T06:31:41.065 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:41.065 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:31:41.065 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:41.065 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:41.065 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:41.065 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:11483 3.1d 3:bcad7303:::rbd_data.112d92027590.0000000000000001:f [read 401408~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:31:41.066 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:25.658906+0000", 2024-09-14T06:31:41.066 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 135.3123477, 2024-09-14T06:31:41.066 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052595109000000001, 2024-09-14T06:31:41.066 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:31:41.066 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:31:41.066 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:31:41.066 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:31:41.066 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:31:41.066 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:41.066 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 11483 2024-09-14T06:31:41.066 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:41.066 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:31:41.066 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:41.066 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:31:41.067 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.658906+0000", 2024-09-14T06:31:41.067 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:41.067 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:41.067 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:41.067 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:31:41.067 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.658906+0000", 2024-09-14T06:31:41.067 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:41.067 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:41.067 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:41.067 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:31:41.067 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.664985+0000", 2024-09-14T06:31:41.067 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.0060787250000000001 2024-09-14T06:31:41.067 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:41.067 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:41.067 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:31:41.068 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.664991+0000", 2024-09-14T06:31:41.068 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.7030000000000003e-06 2024-09-14T06:31:41.068 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:41.068 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:41.068 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:31:41.068 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.664999+0000", 2024-09-14T06:31:41.068 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.4430000000000003e-06 2024-09-14T06:31:41.068 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:41.068 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:41.068 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:31:41.068 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.665006+0000", 2024-09-14T06:31:41.068 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.0840000000000003e-06 2024-09-14T06:31:41.069 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:41.069 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:41.069 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:31:41.069 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.711402+0000", 2024-09-14T06:31:41.069 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046395714999999997 2024-09-14T06:31:41.069 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:41.069 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:41.069 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:31:41.069 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.711443+0000", 2024-09-14T06:31:41.069 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.0911999999999997e-05 2024-09-14T06:31:41.069 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:41.069 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:41.070 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:31:41.070 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.711501+0000", 2024-09-14T06:31:41.070 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.8526999999999997e-05 2024-09-14T06:31:41.070 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:41.070 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:31:41.070 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:41.070 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:41.070 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:41.070 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:214569 3.2 3:479633bd:::rbd_object_map.112d3bd84cb0:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e170)", 2024-09-14T06:31:41.070 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:31:34.933901+0000", 2024-09-14T06:31:41.070 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 6.0373533159999999, 2024-09-14T06:31:41.070 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.68157511400000004, 2024-09-14T06:31:41.071 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:31:41.071 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:31:41.071 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:31:41.071 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:31:41.071 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:31:41.071 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:41.071 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 214569 2024-09-14T06:31:41.071 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:41.071 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:31:41.071 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:41.071 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:31:41.071 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:31:34.933901+0000", 2024-09-14T06:31:41.072 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:41.072 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:41.072 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:41.072 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:31:41.072 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:31:34.933901+0000", 2024-09-14T06:31:41.072 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:41.072 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:41.072 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:41.072 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:31:41.072 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:31:34.933902+0000", 2024-09-14T06:31:41.072 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.7439999999999999e-06 2024-09-14T06:31:41.072 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:41.072 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:41.073 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:31:41.073 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:31:34.933910+0000", 2024-09-14T06:31:41.073 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.8569999999999995e-06 2024-09-14T06:31:41.073 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:41.073 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:41.073 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:31:41.073 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:31:34.933932+0000", 2024-09-14T06:31:41.073 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.1997999999999998e-05 2024-09-14T06:31:41.073 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:41.073 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:41.073 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:31:41.073 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:31:34.933941+0000", 2024-09-14T06:31:41.073 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.7180000000000002e-06 2024-09-14T06:31:41.074 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:41.074 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:41.074 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:31:41.074 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:31:34.934013+0000", 2024-09-14T06:31:41.074 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.1761000000000006e-05 2024-09-14T06:31:41.074 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:41.074 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:41.074 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "waiting for peered", 2024-09-14T06:31:41.074 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:31:34.934022+0000", 2024-09-14T06:31:41.074 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.8740000000000001e-06 2024-09-14T06:31:41.074 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:41.074 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:41.074 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:31:41.074 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:31:35.608803+0000", 2024-09-14T06:31:41.074 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.67478164299999999 2024-09-14T06:31:41.075 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:41.075 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:41.075 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:31:41.075 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:31:35.608964+0000", 2024-09-14T06:31:41.075 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00016112900000000001 2024-09-14T06:31:41.075 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:41.075 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:41.075 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "waiting for subops from 6", 2024-09-14T06:31:41.075 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:31:35.609399+0000", 2024-09-14T06:31:41.075 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00043454000000000002 2024-09-14T06:31:41.075 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:41.075 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:41.075 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "op_commit", 2024-09-14T06:31:41.075 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:31:35.610495+0000", 2024-09-14T06:31:41.075 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.001096404 2024-09-14T06:31:41.076 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:41.076 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:41.076 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:31:41.076 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:31:35.615409+0000", 2024-09-14T06:31:41.076 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.0049133199999999997 2024-09-14T06:31:41.076 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:41.076 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:41.076 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "commit_sent", 2024-09-14T06:31:41.076 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:31:35.615444+0000", 2024-09-14T06:31:41.076 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.5457000000000003e-05 2024-09-14T06:31:41.076 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:41.076 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:41.076 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:31:41.076 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:31:35.615476+0000", 2024-09-14T06:31:41.076 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.1668999999999998e-05 2024-09-14T06:31:41.077 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:41.077 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:31:41.077 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:41.077 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:41.077 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:31:41.077 INFO:teuthology.orchestra.run.smithi138.stdout:} 2024-09-14T06:31:41.079 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-14T06:31:41.079 INFO:tasks.ceph.osd.5.smithi138.stderr:2024-09-14T06:31:41.077+0000 7fe13746f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33618) UID: 0 2024-09-14T06:31:41.090 INFO:teuthology.orchestra.run.smithi094.stdout:osd.0: {} 2024-09-14T06:31:41.091 INFO:teuthology.orchestra.run.smithi094.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-14T06:31:41.108 INFO:teuthology.orchestra.run.smithi094.stdout:osd.1: {} 2024-09-14T06:31:41.109 INFO:teuthology.orchestra.run.smithi094.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-14T06:31:41.123 INFO:teuthology.orchestra.run.smithi094.stdout:osd.2: {} 2024-09-14T06:31:41.123 INFO:teuthology.orchestra.run.smithi094.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-14T06:31:41.138 INFO:teuthology.orchestra.run.smithi094.stdout:osd.3: {} 2024-09-14T06:31:41.139 INFO:teuthology.orchestra.run.smithi094.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-14T06:31:41.155 INFO:teuthology.orchestra.run.smithi094.stdout:osd.4: {} 2024-09-14T06:31:41.155 INFO:teuthology.orchestra.run.smithi094.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-14T06:31:41.171 INFO:teuthology.orchestra.run.smithi094.stdout:osd.5: {} 2024-09-14T06:31:41.172 INFO:teuthology.orchestra.run.smithi094.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-14T06:31:41.179 INFO:tasks.ceph.osd.3.smithi094.stderr:2024-09-14T06:31:41.178+0000 7feeeafe8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33655) UID: 0 2024-09-14T06:31:41.188 INFO:teuthology.orchestra.run.smithi094.stdout:osd.6: {} 2024-09-14T06:31:41.188 INFO:teuthology.orchestra.run.smithi094.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-14T06:31:41.205 INFO:teuthology.orchestra.run.smithi094.stdout:osd.7: {} 2024-09-14T06:31:41.205 INFO:teuthology.orchestra.run.smithi094.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-14T06:31:41.223 INFO:teuthology.orchestra.run.smithi138.stdout:ERROR: (22) Invalid argument 2024-09-14T06:31:41.223 INFO:teuthology.orchestra.run.smithi138.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:31:41.231 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-14T06:31:41.280 INFO:tasks.ceph.osd.0.smithi094.stderr:2024-09-14T06:31:41.279+0000 7f07c9333640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33636) UID: 0 2024-09-14T06:31:41.335 INFO:teuthology.orchestra.run.smithi138.stdout:ERROR: (22) Invalid argument 2024-09-14T06:31:41.335 INFO:teuthology.orchestra.run.smithi138.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:31:41.344 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-14T06:31:41.380 INFO:tasks.ceph.osd.2.smithi094.stderr:2024-09-14T06:31:41.378+0000 7faf3269b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33649) UID: 0 2024-09-14T06:31:41.447 INFO:teuthology.orchestra.run.smithi138.stdout:ERROR: (22) Invalid argument 2024-09-14T06:31:41.447 INFO:teuthology.orchestra.run.smithi138.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:31:41.455 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-14T06:31:41.499 INFO:tasks.ceph.osd.4.smithi138.stderr:2024-09-14T06:31:41.498+0000 7f6c653c8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33615) UID: 0 2024-09-14T06:31:41.558 INFO:teuthology.orchestra.run.smithi138.stdout:ERROR: (22) Invalid argument 2024-09-14T06:31:41.559 INFO:teuthology.orchestra.run.smithi138.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:31:41.567 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-14T06:31:41.609 INFO:tasks.ceph.osd.5.smithi138.stderr:2024-09-14T06:31:41.608+0000 7fe13746f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33618) UID: 0 2024-09-14T06:31:41.622 INFO:teuthology.orchestra.run.smithi094.stdout:5449 trunc from 0xbf81562 to 0xd76fb6 2024-09-14T06:31:41.639 INFO:teuthology.orchestra.run.smithi094.stdout:5450 read 0x8b30c thru 0x9afcb (0xfcc0 bytes) 2024-09-14T06:31:41.654 INFO:teuthology.orchestra.run.smithi094.stdout:5451 write 0x7062042 thru 0x70659ad (0x396c bytes) 2024-09-14T06:31:41.659 INFO:teuthology.orchestra.run.smithi094.stdout:5452 read 0x2430137 thru 0x2434e9f (0x4d69 bytes) 2024-09-14T06:31:41.659 INFO:teuthology.orchestra.run.smithi094.stdout:5453 read 0x786a17 thru 0x793380 (0xc96a bytes) 2024-09-14T06:31:41.659 INFO:teuthology.orchestra.run.smithi094.stdout:5454 write 0x4867e19 thru 0x4871068 (0x9250 bytes) 2024-09-14T06:31:41.674 INFO:teuthology.orchestra.run.smithi094.stdout:5455 read 0x4449849 thru 0x445135f (0x7b17 bytes) 2024-09-14T06:31:41.674 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-14T06:31:41.674 INFO:teuthology.orchestra.run.smithi094.stdout:5457 read 0x1d362ad thru 0x1d40f51 (0xaca5 bytes) 2024-09-14T06:31:41.682 INFO:tasks.ceph.osd.7.smithi138.stderr:2024-09-14T06:31:41.681+0000 7f3c6a21d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33623) UID: 0 2024-09-14T06:31:41.690 INFO:teuthology.orchestra.run.smithi094.stdout:5458 writesame 0xe549933 thru 0xe55310a data_size 0x71(0x97d8 bytes) 2024-09-14T06:31:41.690 INFO:teuthology.orchestra.run.smithi094.stdout:5461 write 0x2509bed thru 0x251980b (0xfc1f bytes) 2024-09-14T06:31:41.695 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-14T06:31:41.706 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-14T06:31:41.707 INFO:teuthology.orchestra.run.smithi094.stdout:5463 read 0x27d4b11 thru 0x27d9d0f (0x51ff bytes) 2024-09-14T06:31:41.707 INFO:teuthology.orchestra.run.smithi094.stdout:5464 read 0x7f68046 thru 0x7f76077 (0xe032 bytes) 2024-09-14T06:31:41.720 INFO:teuthology.orchestra.run.smithi094.stdout:5466 trunc from 0xe55310b to 0x32ac5ea 2024-09-14T06:31:41.727 INFO:teuthology.orchestra.run.smithi138.stdout:ERROR: (22) Invalid argument 2024-09-14T06:31:41.727 INFO:teuthology.orchestra.run.smithi138.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:31:41.733 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-14T06:31:41.733 INFO:teuthology.orchestra.run.smithi094.stdout:5469 punch from 0x6ae26c to 0x6b3efe, (0x5c92 bytes) 2024-09-14T06:31:41.736 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-14T06:31:41.744 INFO:teuthology.orchestra.run.smithi094.stdout:5470 trunc from 0x32ac5ea to 0xc408daa 2024-09-14T06:31:41.749 INFO:teuthology.orchestra.run.smithi094.stdout:5471 punch from 0x91131c2 to 0x911d4d0, (0xa30e bytes) 2024-09-14T06:31:41.749 INFO:teuthology.orchestra.run.smithi094.stdout:5472 read 0x354d2c0 thru 0x3559032 (0xbd73 bytes) 2024-09-14T06:31:41.755 INFO:teuthology.orchestra.run.smithi094.stdout:5473 write 0xd1c4572 thru 0xd1cb7d3 (0x7262 bytes) 2024-09-14T06:31:41.768 INFO:teuthology.orchestra.run.smithi094.stdout:5474 trunc from 0xd1cb7d4 to 0x9295ed8 2024-09-14T06:31:41.779 INFO:teuthology.orchestra.run.smithi094.stdout:5475 writesame 0x1e9b2cf thru 0x1eaa239 data_size 0x1(0xef6b bytes) 2024-09-14T06:31:41.783 INFO:tasks.ceph.osd.3.smithi094.stderr:2024-09-14T06:31:41.781+0000 7feeeafe8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33655) UID: 0 2024-09-14T06:31:41.800 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-14T06:31:41.800 INFO:teuthology.orchestra.run.smithi094.stdout:5478 read 0x22eaace thru 0x22f896c (0xde9f bytes) 2024-09-14T06:31:41.800 INFO:teuthology.orchestra.run.smithi094.stdout:5479 read 0x29ad5c3 thru 0x29b15fc (0x403a bytes) 2024-09-14T06:31:41.800 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-14T06:31:41.804 INFO:teuthology.orchestra.run.smithi094.stdout:5481 trunc from 0x9295ed8 to 0x6100599 2024-09-14T06:31:41.812 INFO:teuthology.orchestra.run.smithi094.stdout:5482 trunc from 0x6100599 to 0x4284788 2024-09-14T06:31:41.818 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-14T06:31:41.818 INFO:teuthology.orchestra.run.smithi094.stdout:5489 read 0x5144a0 thru 0x51768c (0x31ed bytes) 2024-09-14T06:31:41.818 INFO:teuthology.orchestra.run.smithi094.stdout:5490 read 0xdd1dda thru 0xdd4516 (0x273d bytes) 2024-09-14T06:31:41.818 INFO:teuthology.orchestra.run.smithi094.stdout:5491 write 0x1b06f93 thru 0x1b168bc (0xf92a bytes) 2024-09-14T06:31:41.844 INFO:teuthology.orchestra.run.smithi094.stdout:5492 write 0x6651fac thru 0x665398f (0x19e4 bytes) 2024-09-14T06:31:41.851 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-14T06:31:41.856 INFO:teuthology.orchestra.run.smithi094.stdout:5494 trunc from 0x6653990 to 0xa8f9868 2024-09-14T06:31:41.864 INFO:teuthology.orchestra.run.smithi094.stdout:5497 trunc from 0xa8f9868 to 0xbeacbd9 2024-09-14T06:31:41.870 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-14T06:31:41.870 INFO:teuthology.orchestra.run.smithi094.stdout:5500 write 0x57f1f7f thru 0x57f6a00 (0x4a82 bytes) 2024-09-14T06:31:41.876 INFO:teuthology.orchestra.run.smithi094.stdout:5503 write 0x1265b98 thru 0x12717c5 (0xbc2e bytes) 2024-09-14T06:31:41.884 INFO:tasks.ceph.osd.7.smithi138.stderr:2024-09-14T06:31:41.883+0000 7f3c6a21d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33623) UID: 0 2024-09-14T06:31:41.891 INFO:teuthology.orchestra.run.smithi094.stdout:5504 read 0x76b164f thru 0x76b4bf3 (0x35a5 bytes) 2024-09-14T06:31:41.891 INFO:teuthology.orchestra.run.smithi094.stdout:5505 writesame 0x1567552 thru 0x156b50f data_size 0x29(0x3fbe bytes) 2024-09-14T06:31:41.898 INFO:teuthology.orchestra.run.smithi138.stdout:ERROR: (22) Invalid argument 2024-09-14T06:31:41.898 INFO:teuthology.orchestra.run.smithi138.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:31:41.900 INFO:teuthology.orchestra.run.smithi094.stdout:5507 read 0x9afd75f thru 0x9b03bca (0x646c bytes) 2024-09-14T06:31:41.907 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-14T06:31:41.908 INFO:teuthology.orchestra.run.smithi094.stdout:5508 trunc from 0xbeacbd9 to 0x5c7a417 2024-09-14T06:31:41.937 INFO:teuthology.orchestra.run.smithi094.stdout:5509 write 0xde1ed25 thru 0xde2cd33 (0xe00f bytes) 2024-09-14T06:31:41.955 INFO:teuthology.orchestra.run.smithi094.stdout:5510 trunc from 0xde2cd34 to 0x6f6e006 2024-09-14T06:31:41.967 INFO:teuthology.orchestra.run.smithi094.stdout:5511 punch from 0x60bbf2f to 0x60c1449, (0x551a bytes) 2024-09-14T06:31:41.979 INFO:teuthology.orchestra.run.smithi094.stdout:5514 write 0xc3648e0 thru 0xc368193 (0x38b4 bytes) 2024-09-14T06:31:41.984 INFO:teuthology.orchestra.run.smithi094.stdout:5516 write 0x475c72c thru 0x475e98a (0x225f bytes) 2024-09-14T06:31:41.984 INFO:tasks.ceph.osd.2.smithi094.stderr:2024-09-14T06:31:41.982+0000 7faf3269b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33649) UID: 0 2024-09-14T06:31:41.988 INFO:teuthology.orchestra.run.smithi094.stdout:5520 read 0x1797f52 thru 0x179816d (0x21c bytes) 2024-09-14T06:31:41.988 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-14T06:31:41.988 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-14T06:31:41.988 INFO:teuthology.orchestra.run.smithi094.stdout:5523 write 0x76464e4 thru 0x764af6e (0x4a8b bytes) 2024-09-14T06:31:41.994 INFO:teuthology.orchestra.run.smithi094.stdout:5525 write 0x68e108f thru 0x68ee64d (0xd5bf bytes) 2024-09-14T06:31:42.013 INFO:teuthology.orchestra.run.smithi094.stdout:5526 read 0xa36139e thru 0xa361510 (0x173 bytes) 2024-09-14T06:31:42.013 INFO:teuthology.orchestra.run.smithi094.stdout:5527 write 0x82c685a thru 0x82d64da (0xfc81 bytes) 2024-09-14T06:31:42.024 INFO:teuthology.orchestra.run.smithi094.stdout:ERROR: (22) Invalid argument 2024-09-14T06:31:42.024 INFO:teuthology.orchestra.run.smithi094.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:31:42.031 INFO:teuthology.orchestra.run.smithi094.stdout:5528 write 0xdacb014 thru 0xdad2e68 (0x7e55 bytes) 2024-09-14T06:31:42.035 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-14T06:31:42.041 INFO:teuthology.orchestra.run.smithi094.stdout:5529 read 0xa8f7c6f thru 0xa902422 (0xa7b4 bytes) 2024-09-14T06:31:42.042 INFO:teuthology.orchestra.run.smithi094.stdout:5530 read 0x4404341 thru 0x440c9f2 (0x86b2 bytes) 2024-09-14T06:31:42.042 INFO:teuthology.orchestra.run.smithi094.stdout:5531 read 0x694d150 thru 0x695665f (0x9510 bytes) 2024-09-14T06:31:42.048 INFO:teuthology.orchestra.run.smithi094.stdout:5532 write 0xdcc97b6 thru 0xdcca8db (0x1126 bytes) 2024-09-14T06:31:42.055 INFO:teuthology.orchestra.run.smithi094.stdout:5533 writesame 0x56caed0 thru 0x56d08df data_size 0x2c(0x5a10 bytes) 2024-09-14T06:31:42.063 INFO:teuthology.orchestra.run.smithi094.stdout:5534 read 0x31ab5dd thru 0x31bace0 (0xf704 bytes) 2024-09-14T06:31:42.063 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-14T06:31:42.075 INFO:teuthology.orchestra.run.smithi094.stdout:5538 trunc from 0xdcca8dc to 0x4350040 2024-09-14T06:31:42.085 INFO:tasks.ceph.osd.5.smithi138.stderr:2024-09-14T06:31:42.084+0000 7fe13746f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33618) UID: 0 2024-09-14T06:31:42.116 INFO:teuthology.orchestra.run.smithi094.stdout:5540 writesame 0xb56588f thru 0xb565ab5 data_size 0x1(0x227 bytes) 2024-09-14T06:31:42.117 INFO:teuthology.orchestra.run.smithi094.stdout:5541 writesame 0x183fef9 thru 0x184d8c2 data_size 0x3d(0xd9ca bytes) 2024-09-14T06:31:42.136 INFO:teuthology.orchestra.run.smithi094.stdout:5542 trunc from 0xb565ab6 to 0x3aa87c3 2024-09-14T06:31:42.142 INFO:teuthology.orchestra.run.smithi094.stdout:5544 read 0x3c5115 thru 0x3d4b6c (0xfa58 bytes) 2024-09-14T06:31:42.145 INFO:teuthology.orchestra.run.smithi094.stdout:5545 trunc from 0x3aa87c3 to 0x667cd58 2024-09-14T06:31:42.151 INFO:teuthology.orchestra.run.smithi094.stdout:5547 punch from 0x2d2ea7e to 0x2d31a7e, (0x3000 bytes) 2024-09-14T06:31:42.151 INFO:teuthology.orchestra.run.smithi094.stdout:5548 read 0x81ab25 thru 0x82065d (0x5b39 bytes) 2024-09-14T06:31:42.151 INFO:teuthology.orchestra.run.smithi094.stdout:5549 read 0x5ab54f9 thru 0x5ab8434 (0x2f3c bytes) 2024-09-14T06:31:42.158 INFO:teuthology.orchestra.run.smithi094.stdout:ERROR: (22) Invalid argument 2024-09-14T06:31:42.158 INFO:teuthology.orchestra.run.smithi094.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:31:42.166 INFO:teuthology.orchestra.run.smithi094.stdout:5551 writesame 0xe310d40 thru 0xe3123b6 data_size 0x3(0x1677 bytes) 2024-09-14T06:31:42.166 INFO:teuthology.orchestra.run.smithi094.stdout:5552 punch from 0x31c94a5 to 0x31cd1aa, (0x3d05 bytes) 2024-09-14T06:31:42.166 INFO:teuthology.orchestra.run.smithi094.stdout:5553 write 0xce7348a thru 0xce76b23 (0x369a bytes) 2024-09-14T06:31:42.168 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-14T06:31:42.171 INFO:teuthology.orchestra.run.smithi094.stdout:5554 read 0x906de1a thru 0x9072863 (0x4a4a bytes) 2024-09-14T06:31:42.183 INFO:teuthology.orchestra.run.smithi094.stdout:5555 trunc from 0xe3123b7 to 0x366d3a2 2024-09-14T06:31:42.185 INFO:tasks.ceph.osd.1.smithi094.stderr:2024-09-14T06:31:42.184+0000 7f328e47a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33641) UID: 0 2024-09-14T06:31:42.209 INFO:teuthology.orchestra.run.smithi094.stdout:5556 trunc from 0x366d3a2 to 0xd725d28 2024-09-14T06:31:42.215 INFO:teuthology.orchestra.run.smithi094.stdout:5557 read 0xa0e0c88 thru 0xa0e529d (0x4616 bytes) 2024-09-14T06:31:42.215 INFO:teuthology.orchestra.run.smithi094.stdout:5558 write 0xcb1d233 thru 0xcb23968 (0x6736 bytes) 2024-09-14T06:31:42.221 INFO:teuthology.orchestra.run.smithi094.stdout:5559 read 0x2ab2eeb thru 0x2abdecf (0xafe5 bytes) 2024-09-14T06:31:42.221 INFO:teuthology.orchestra.run.smithi094.stdout:5561 write 0xb55c821 thru 0xb568b15 (0xc2f5 bytes) 2024-09-14T06:31:42.233 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-14T06:31:42.233 INFO:teuthology.orchestra.run.smithi094.stdout:5565 writesame 0x848ca07 thru 0x849656d data_size 0x3(0x9b67 bytes) 2024-09-14T06:31:42.253 INFO:teuthology.orchestra.run.smithi094.stdout:5566 trunc from 0xd725d28 to 0xa4677e8 2024-09-14T06:31:42.269 INFO:teuthology.orchestra.run.smithi094.stdout:5567 read 0x50f7665 thru 0x50f9d70 (0x270c bytes) 2024-09-14T06:31:42.269 INFO:teuthology.orchestra.run.smithi094.stdout:5568 read 0x88d32b4 thru 0x88dc779 (0x94c6 bytes) 2024-09-14T06:31:42.269 INFO:teuthology.orchestra.run.smithi094.stdout:5570 read 0xa201b4 thru 0xa20e22 (0xc6f bytes) 2024-09-14T06:31:42.269 INFO:teuthology.orchestra.run.smithi094.stdout:5571 write 0x3f7653c thru 0x3f80daa (0xa86f bytes) 2024-09-14T06:31:42.277 INFO:teuthology.orchestra.run.smithi094.stdout:5572 read 0x1f24c83 thru 0x1f347d3 (0xfb51 bytes) 2024-09-14T06:31:42.280 INFO:teuthology.orchestra.run.smithi094.stdout:5574 trunc from 0xa4677e8 to 0xbb46198 2024-09-14T06:31:42.285 INFO:teuthology.orchestra.run.smithi094.stdout:5575 writesame 0x2381406 thru 0x2384952 data_size 0x1(0x354d bytes) 2024-09-14T06:31:42.286 INFO:teuthology.orchestra.run.smithi094.stdout:ERROR: (22) Invalid argument 2024-09-14T06:31:42.286 INFO:teuthology.orchestra.run.smithi094.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:31:42.286 INFO:tasks.ceph.osd.1.smithi094.stderr:2024-09-14T06:31:42.285+0000 7f328e47a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33641) UID: 0 2024-09-14T06:31:42.294 INFO:teuthology.orchestra.run.smithi094.stdout:5580 write 0x3ced859 thru 0x3cf115f (0x3907 bytes) 2024-09-14T06:31:42.295 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-14T06:31:42.303 INFO:teuthology.orchestra.run.smithi094.stdout:5581 write 0x8b4c44 thru 0x8b88b1 (0x3c6e bytes) 2024-09-14T06:31:42.311 INFO:teuthology.orchestra.run.smithi094.stdout:5582 writesame 0xae1a2d9 thru 0xae2479b data_size 0x1(0xa4c3 bytes) 2024-09-14T06:31:42.325 INFO:teuthology.orchestra.run.smithi094.stdout:5583 writesame 0x1834839 thru 0x1843cbd data_size 0x1(0xf485 bytes) 2024-09-14T06:31:42.341 INFO:teuthology.orchestra.run.smithi094.stdout:5584 read 0x299a6c6 thru 0x299e823 (0x415e bytes) 2024-09-14T06:31:42.341 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-14T06:31:42.341 INFO:teuthology.orchestra.run.smithi094.stdout:5587 read 0x1ee8cce thru 0x1eea5aa (0x18dd bytes) 2024-09-14T06:31:42.341 INFO:teuthology.orchestra.run.smithi094.stdout:5589 write 0x5ecfdf thru 0x5f269b (0x56bd bytes) 2024-09-14T06:31:42.351 INFO:teuthology.orchestra.run.smithi094.stdout:5590 read 0x955c09 thru 0x95a82a (0x4c22 bytes) 2024-09-14T06:31:42.351 INFO:teuthology.orchestra.run.smithi094.stdout:5591 write 0x34f6ca2 thru 0x350517d (0xe4dc bytes) 2024-09-14T06:31:42.376 INFO:teuthology.orchestra.run.smithi094.stdout:5592 trunc from 0xbb46198 to 0x1c036fd 2024-09-14T06:31:42.387 INFO:tasks.ceph.osd.5.smithi138.stderr:2024-09-14T06:31:42.386+0000 7fe13746f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33618) UID: 0 2024-09-14T06:31:42.412 INFO:teuthology.orchestra.run.smithi094.stdout:5593 writesame 0x3616a3c thru 0x3625a86 data_size 0xf(0xf04b bytes) 2024-09-14T06:31:42.421 INFO:teuthology.orchestra.run.smithi094.stdout:ERROR: (22) Invalid argument 2024-09-14T06:31:42.421 INFO:teuthology.orchestra.run.smithi094.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:31:42.422 INFO:teuthology.orchestra.run.smithi094.stdout:5594 writesame 0x6ef5805 thru 0x6efb110 data_size 0x29(0x590c bytes) 2024-09-14T06:31:42.422 INFO:teuthology.orchestra.run.smithi094.stdout:5597 writesame 0x60340ce thru 0x6035de3 data_size 0x2(0x1d16 bytes) 2024-09-14T06:31:42.426 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-14T06:31:42.426 INFO:teuthology.orchestra.run.smithi094.stdout:5599 writesame 0x6fa892 thru 0x70a26b data_size 0x1(0xf9da bytes) 2024-09-14T06:31:42.430 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-14T06:31:42.442 INFO:teuthology.orchestra.run.smithi094.stdout:5601 read 0x172f7f5 thru 0x17379b0 (0x81bc bytes) 2024-09-14T06:31:42.448 INFO:teuthology.orchestra.run.smithi094.stdout:5602 trunc from 0x6efb111 to 0xc39a8d5 2024-09-14T06:31:42.454 INFO:teuthology.orchestra.run.smithi094.stdout:5603 write 0x2b6a479 thru 0x2b6de0b (0x3993 bytes) 2024-09-14T06:31:42.469 INFO:teuthology.orchestra.run.smithi094.stdout:5604 writesame 0xe64f7a5 thru 0xe65527c data_size 0x39(0x5ad8 bytes) 2024-09-14T06:31:42.469 INFO:teuthology.orchestra.run.smithi094.stdout:5606 write 0xe4819e6 thru 0xe481dac (0x3c7 bytes) 2024-09-14T06:31:42.471 INFO:teuthology.orchestra.run.smithi094.stdout:5607 write 0x699518c thru 0x699edc4 (0x9c39 bytes) 2024-09-14T06:31:42.484 INFO:teuthology.orchestra.run.smithi094.stdout:5608 read 0xb0efe11 thru 0xb0f77c9 (0x79b9 bytes) 2024-09-14T06:31:42.485 INFO:teuthology.orchestra.run.smithi094.stdout:5609 writesame 0x753f5bf thru 0x75456ee data_size 0x28(0x6130 bytes) 2024-09-14T06:31:42.488 INFO:tasks.ceph.osd.3.smithi094.stderr:2024-09-14T06:31:42.487+0000 7feeeafe8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33655) UID: 0 2024-09-14T06:31:42.491 INFO:teuthology.orchestra.run.smithi094.stdout:5611 read 0x18d259b thru 0x18da822 (0x8288 bytes) 2024-09-14T06:31:42.491 INFO:teuthology.orchestra.run.smithi094.stdout:5612 writesame 0x5121f48 thru 0x5124f33 data_size 0x2(0x2fec bytes) 2024-09-14T06:31:42.499 INFO:teuthology.orchestra.run.smithi094.stdout:5613 read 0x3f5c9bc thru 0x3f5de68 (0x14ad bytes) 2024-09-14T06:31:42.500 INFO:teuthology.orchestra.run.smithi094.stdout:5614 write 0x56f0e0b thru 0x56f42d8 (0x34ce bytes) 2024-09-14T06:31:42.508 INFO:teuthology.orchestra.run.smithi094.stdout:5615 writesame 0x4cc4331 thru 0x4ccea3c data_size 0x2(0xa70c bytes) 2024-09-14T06:31:42.523 INFO:teuthology.orchestra.run.smithi094.stdout:5616 write 0x72448f2 thru 0x725313b (0xe84a bytes) 2024-09-14T06:31:42.537 INFO:teuthology.orchestra.run.smithi094.stdout:5617 writesame 0x3c985d2 thru 0x3c9f9f4 data_size 0x1(0x7423 bytes) 2024-09-14T06:31:42.550 INFO:teuthology.orchestra.run.smithi094.stdout:5620 writesame 0xb89a54f thru 0xb89dfd4 data_size 0x21(0x3a86 bytes) 2024-09-14T06:31:42.554 INFO:teuthology.orchestra.run.smithi094.stdout:ERROR: (22) Invalid argument 2024-09-14T06:31:42.554 INFO:teuthology.orchestra.run.smithi094.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:31:42.554 INFO:teuthology.orchestra.run.smithi094.stdout:5621 write 0xe06ba5a thru 0xe0737c2 (0x7d69 bytes) 2024-09-14T06:31:42.564 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-14T06:31:42.566 INFO:teuthology.orchestra.run.smithi094.stdout:5622 read 0xe8d081 thru 0xe8e73d (0x16bd bytes) 2024-09-14T06:31:42.566 INFO:teuthology.orchestra.run.smithi094.stdout:5623 read 0xa51ebfa thru 0xa51ec69 (0x70 bytes) 2024-09-14T06:31:42.567 INFO:teuthology.orchestra.run.smithi094.stdout:5624 write 0xcf61e56 thru 0xcf68734 (0x68df bytes) 2024-09-14T06:31:42.576 INFO:teuthology.orchestra.run.smithi094.stdout:5625 write 0x63e9c70 thru 0x63eb8f0 (0x1c81 bytes) 2024-09-14T06:31:42.583 INFO:teuthology.orchestra.run.smithi094.stdout:5627 read 0x5e1de9e thru 0x5e251e6 (0x7349 bytes) 2024-09-14T06:31:42.583 INFO:teuthology.orchestra.run.smithi094.stdout:5630 write 0x1005dc7 thru 0x1006524 (0x75e bytes) 2024-09-14T06:31:42.587 INFO:teuthology.orchestra.run.smithi094.stdout:5631 punch from 0x46b773d to 0x46b8d01, (0x15c4 bytes) 2024-09-14T06:31:42.587 INFO:teuthology.orchestra.run.smithi094.stdout:5633 read 0x3b9f244 thru 0x3ba3093 (0x3e50 bytes) 2024-09-14T06:31:42.587 INFO:teuthology.orchestra.run.smithi094.stdout:5634 punch from 0x98d82db to 0x98d8c5c, (0x981 bytes) 2024-09-14T06:31:42.587 INFO:teuthology.orchestra.run.smithi094.stdout:5635 read 0x37a05de thru 0x37aae68 (0xa88b bytes) 2024-09-14T06:31:42.587 INFO:teuthology.orchestra.run.smithi094.stdout:5636 clone 40 order 18 su 65536 sc 3 2024-09-14T06:31:42.588 INFO:tasks.ceph.osd.7.smithi138.stderr:2024-09-14T06:31:42.587+0000 7f3c6a21d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33623) UID: 0 2024-09-14T06:31:42.668 INFO:teuthology.orchestra.run.smithi094.stdout:ERROR: (22) Invalid argument 2024-09-14T06:31:42.668 INFO:teuthology.orchestra.run.smithi094.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:31:42.676 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-14T06:31:42.689 INFO:tasks.ceph.osd.4.smithi138.stderr:2024-09-14T06:31:42.688+0000 7f6c653c8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33615) UID: 0 2024-09-14T06:31:42.690 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 4, 5, 6, 7] out_osds: [3] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-14T06:31:42.690 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-14T06:31:42.690 INFO:tasks.thrashosds.thrasher:trim_stale_osdmaps 2024-09-14T06:31:42.690 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 'trim stale osdmaps' 2024-09-14T06:31:42.697 INFO:teuthology.orchestra.run.smithi094.stderr:noscrub is unset 2024-09-14T06:31:42.790 INFO:tasks.ceph.osd.4.smithi138.stderr:2024-09-14T06:31:42.789+0000 7f6c653c8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33615) UID: 0 2024-09-14T06:31:42.806 INFO:teuthology.orchestra.run.smithi094.stdout:ERROR: (22) Invalid argument 2024-09-14T06:31:42.806 INFO:teuthology.orchestra.run.smithi094.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:31:42.815 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-14T06:31:42.833 INFO:teuthology.orchestra.run.smithi094.stderr:no valid command found; 10 closest matches: 2024-09-14T06:31:42.834 INFO:teuthology.orchestra.run.smithi094.stderr:0 2024-09-14T06:31:42.834 INFO:teuthology.orchestra.run.smithi094.stderr:1 2024-09-14T06:31:42.834 INFO:teuthology.orchestra.run.smithi094.stderr:2 2024-09-14T06:31:42.834 INFO:teuthology.orchestra.run.smithi094.stderr:abort 2024-09-14T06:31:42.834 INFO:teuthology.orchestra.run.smithi094.stderr:assert 2024-09-14T06:31:42.834 INFO:teuthology.orchestra.run.smithi094.stderr:bench [] [] [] [] 2024-09-14T06:31:42.834 INFO:teuthology.orchestra.run.smithi094.stderr:bluefs debug_inject_read_zeros 2024-09-14T06:31:42.834 INFO:teuthology.orchestra.run.smithi094.stderr:bluefs files list 2024-09-14T06:31:42.834 INFO:teuthology.orchestra.run.smithi094.stderr:bluefs stats 2024-09-14T06:31:42.834 INFO:teuthology.orchestra.run.smithi094.stderr:bluestore allocator dump block 2024-09-14T06:31:42.834 INFO:teuthology.orchestra.run.smithi094.stderr:Error EINVAL: invalid command 2024-09-14T06:31:42.837 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-14T06:31:42.837 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-14T06:31:42.838 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 'trim stale osdmaps' 2024-09-14T06:31:42.892 INFO:tasks.ceph.osd.7.smithi138.stderr:2024-09-14T06:31:42.891+0000 7f3c6a21d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33623) UID: 0 2024-09-14T06:31:42.920 INFO:teuthology.orchestra.run.smithi094.stdout:ERROR: (22) Invalid argument 2024-09-14T06:31:42.920 INFO:teuthology.orchestra.run.smithi094.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:31:42.929 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-14T06:31:43.012 INFO:teuthology.orchestra.run.smithi094.stderr:no valid command found; 10 closest matches: 2024-09-14T06:31:43.012 INFO:teuthology.orchestra.run.smithi094.stderr:0 2024-09-14T06:31:43.012 INFO:teuthology.orchestra.run.smithi094.stderr:1 2024-09-14T06:31:43.012 INFO:teuthology.orchestra.run.smithi094.stderr:2 2024-09-14T06:31:43.012 INFO:teuthology.orchestra.run.smithi094.stderr:abort 2024-09-14T06:31:43.013 INFO:teuthology.orchestra.run.smithi094.stderr:assert 2024-09-14T06:31:43.013 INFO:teuthology.orchestra.run.smithi094.stderr:bench [] [] [] [] 2024-09-14T06:31:43.013 INFO:teuthology.orchestra.run.smithi094.stderr:bluefs debug_inject_read_zeros 2024-09-14T06:31:43.013 INFO:teuthology.orchestra.run.smithi094.stderr:bluefs files list 2024-09-14T06:31:43.013 INFO:teuthology.orchestra.run.smithi094.stderr:bluefs stats 2024-09-14T06:31:43.013 INFO:teuthology.orchestra.run.smithi094.stderr:bluestore allocator dump block 2024-09-14T06:31:43.013 INFO:teuthology.orchestra.run.smithi094.stderr:Error EINVAL: invalid command 2024-09-14T06:31:43.013 INFO:tasks.ceph.osd.3.smithi094.stderr:2024-09-14T06:31:43.011+0000 7feeeafe8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33655) UID: 0 2024-09-14T06:31:43.015 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-14T06:31:43.016 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-14T06:31:43.016 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 'trim stale osdmaps' 2024-09-14T06:31:43.075 INFO:teuthology.orchestra.run.smithi094.stdout:ERROR: (22) Invalid argument 2024-09-14T06:31:43.075 INFO:teuthology.orchestra.run.smithi094.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:31:43.085 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-14T06:31:43.093 INFO:tasks.ceph.osd.4.smithi138.stderr:2024-09-14T06:31:43.092+0000 7f6c653c8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33615) UID: 0 2024-09-14T06:31:43.189 INFO:teuthology.orchestra.run.smithi094.stderr:no valid command found; 10 closest matches: 2024-09-14T06:31:43.190 INFO:teuthology.orchestra.run.smithi094.stderr:0 2024-09-14T06:31:43.190 INFO:teuthology.orchestra.run.smithi094.stderr:1 2024-09-14T06:31:43.190 INFO:teuthology.orchestra.run.smithi094.stderr:2 2024-09-14T06:31:43.190 INFO:teuthology.orchestra.run.smithi094.stderr:abort 2024-09-14T06:31:43.190 INFO:teuthology.orchestra.run.smithi094.stderr:assert 2024-09-14T06:31:43.190 INFO:teuthology.orchestra.run.smithi094.stderr:bench [] [] [] [] 2024-09-14T06:31:43.190 INFO:teuthology.orchestra.run.smithi094.stderr:bluefs debug_inject_read_zeros 2024-09-14T06:31:43.190 INFO:teuthology.orchestra.run.smithi094.stderr:bluefs files list 2024-09-14T06:31:43.190 INFO:teuthology.orchestra.run.smithi094.stderr:bluefs stats 2024-09-14T06:31:43.190 INFO:teuthology.orchestra.run.smithi094.stderr:bluestore allocator dump block 2024-09-14T06:31:43.191 INFO:teuthology.orchestra.run.smithi094.stderr:Error EINVAL: invalid command 2024-09-14T06:31:43.192 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-14T06:31:43.193 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-14T06:31:43.193 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 'trim stale osdmaps' 2024-09-14T06:31:43.194 INFO:tasks.ceph.osd.1.smithi094.stderr:2024-09-14T06:31:43.193+0000 7f328e47a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33641) UID: 0 2024-09-14T06:31:43.217 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-14T06:31:43.232 INFO:teuthology.orchestra.run.smithi094.stdout:ERROR: (22) Invalid argument 2024-09-14T06:31:43.232 INFO:teuthology.orchestra.run.smithi094.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:31:43.241 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-14T06:31:43.327 INFO:tasks.ceph.osd.3.smithi094.stderr:2024-09-14T06:31:43.326+0000 7feeeafe8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33655) UID: 0 2024-09-14T06:31:43.329 INFO:teuthology.orchestra.run.smithi094.stderr:no valid command found; 10 closest matches: 2024-09-14T06:31:43.329 INFO:teuthology.orchestra.run.smithi094.stderr:0 2024-09-14T06:31:43.329 INFO:teuthology.orchestra.run.smithi094.stderr:1 2024-09-14T06:31:43.329 INFO:teuthology.orchestra.run.smithi094.stderr:2 2024-09-14T06:31:43.330 INFO:teuthology.orchestra.run.smithi094.stderr:abort 2024-09-14T06:31:43.330 INFO:teuthology.orchestra.run.smithi094.stderr:assert 2024-09-14T06:31:43.330 INFO:teuthology.orchestra.run.smithi094.stderr:bench [] [] [] [] 2024-09-14T06:31:43.330 INFO:teuthology.orchestra.run.smithi094.stderr:bluefs debug_inject_read_zeros 2024-09-14T06:31:43.330 INFO:teuthology.orchestra.run.smithi094.stderr:bluefs files list 2024-09-14T06:31:43.330 INFO:teuthology.orchestra.run.smithi094.stderr:bluefs stats 2024-09-14T06:31:43.330 INFO:teuthology.orchestra.run.smithi094.stderr:bluestore allocator dump block 2024-09-14T06:31:43.330 INFO:teuthology.orchestra.run.smithi094.stderr:Error EINVAL: invalid command 2024-09-14T06:31:43.332 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-14T06:31:43.333 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-14T06:31:43.333 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 'trim stale osdmaps' 2024-09-14T06:31:43.356 INFO:teuthology.orchestra.run.smithi094.stdout:osd.0: {} 2024-09-14T06:31:43.357 INFO:teuthology.orchestra.run.smithi094.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-14T06:31:43.373 INFO:teuthology.orchestra.run.smithi094.stdout:osd.1: {} 2024-09-14T06:31:43.374 INFO:teuthology.orchestra.run.smithi094.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-14T06:31:43.390 INFO:teuthology.orchestra.run.smithi094.stdout:osd.2: {} 2024-09-14T06:31:43.390 INFO:teuthology.orchestra.run.smithi094.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-14T06:31:43.392 INFO:teuthology.orchestra.run.smithi094.stdout:ERROR: (22) Invalid argument 2024-09-14T06:31:43.392 INFO:teuthology.orchestra.run.smithi094.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:31:43.396 INFO:tasks.ceph.osd.4.smithi138.stderr:2024-09-14T06:31:43.395+0000 7f6c653c8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33615) UID: 0 2024-09-14T06:31:43.401 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-14T06:31:43.407 INFO:teuthology.orchestra.run.smithi094.stdout:osd.3: {} 2024-09-14T06:31:43.407 INFO:teuthology.orchestra.run.smithi094.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-14T06:31:43.425 INFO:teuthology.orchestra.run.smithi094.stdout:osd.4: {} 2024-09-14T06:31:43.425 INFO:teuthology.orchestra.run.smithi094.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-14T06:31:43.442 INFO:teuthology.orchestra.run.smithi094.stdout:osd.5: {} 2024-09-14T06:31:43.442 INFO:teuthology.orchestra.run.smithi094.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-14T06:31:43.459 INFO:teuthology.orchestra.run.smithi094.stdout:osd.6: {} 2024-09-14T06:31:43.459 INFO:teuthology.orchestra.run.smithi094.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-14T06:31:43.490 INFO:teuthology.orchestra.run.smithi094.stdout:osd.7: {} 2024-09-14T06:31:43.490 INFO:teuthology.orchestra.run.smithi094.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-14T06:31:43.492 INFO:teuthology.orchestra.run.smithi094.stderr:no valid command found; 10 closest matches: 2024-09-14T06:31:43.492 INFO:teuthology.orchestra.run.smithi094.stderr:0 2024-09-14T06:31:43.492 INFO:teuthology.orchestra.run.smithi094.stderr:1 2024-09-14T06:31:43.492 INFO:teuthology.orchestra.run.smithi094.stderr:2 2024-09-14T06:31:43.492 INFO:teuthology.orchestra.run.smithi094.stderr:abort 2024-09-14T06:31:43.492 INFO:teuthology.orchestra.run.smithi094.stderr:assert 2024-09-14T06:31:43.492 INFO:teuthology.orchestra.run.smithi094.stderr:bench [] [] [] [] 2024-09-14T06:31:43.492 INFO:teuthology.orchestra.run.smithi094.stderr:bluefs debug_inject_read_zeros 2024-09-14T06:31:43.492 INFO:teuthology.orchestra.run.smithi094.stderr:bluefs files list 2024-09-14T06:31:43.492 INFO:teuthology.orchestra.run.smithi094.stderr:bluefs stats 2024-09-14T06:31:43.492 INFO:teuthology.orchestra.run.smithi094.stderr:bluestore allocator dump block 2024-09-14T06:31:43.492 INFO:teuthology.orchestra.run.smithi094.stderr:Error EINVAL: invalid command 2024-09-14T06:31:43.495 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-14T06:31:43.495 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-14T06:31:43.496 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 'trim stale osdmaps' 2024-09-14T06:31:43.497 INFO:tasks.ceph.osd.4.smithi138.stderr:2024-09-14T06:31:43.496+0000 7f6c653c8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33615) UID: 0 2024-09-14T06:31:43.529 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-14T06:31:43.529 INFO:teuthology.orchestra.run.smithi094.stdout: "size": 20, 2024-09-14T06:31:43.529 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 600, 2024-09-14T06:31:43.529 INFO:teuthology.orchestra.run.smithi094.stdout: "ops": [ 2024-09-14T06:31:43.530 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:43.530 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:44425 3.2 3:4603ad7f:::rbd_data.112dca2f6ae1.0000000000000005:17 [read 868352~4096 [fadvise_nocache]] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:31:43.530 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:43.854967+0000", 2024-09-14T06:31:43.530 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 119.672655811, 2024-09-14T06:31:43.530 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.063146025999999994, 2024-09-14T06:31:43.530 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:31:43.530 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:31:43.530 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:31:43.530 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:31:43.530 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:31:43.530 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:43.530 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 44425 2024-09-14T06:31:43.530 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:43.531 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:31:43.531 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:43.531 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:31:43.531 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.854967+0000", 2024-09-14T06:31:43.531 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:43.531 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:43.531 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:43.531 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:31:43.531 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.854967+0000", 2024-09-14T06:31:43.531 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:43.531 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:43.531 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:43.531 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:31:43.531 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.854967+0000", 2024-09-14T06:31:43.532 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.16e-07 2024-09-14T06:31:43.532 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:43.532 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:43.532 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:31:43.532 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.854968+0000", 2024-09-14T06:31:43.532 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.006e-06 2024-09-14T06:31:43.532 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:43.532 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:43.532 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:31:43.532 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.854971+0000", 2024-09-14T06:31:43.532 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.9359999999999999e-06 2024-09-14T06:31:43.532 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:43.532 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:43.532 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:31:43.533 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.854974+0000", 2024-09-14T06:31:43.533 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.1840000000000002e-06 2024-09-14T06:31:43.533 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:43.533 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:43.533 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:31:43.533 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.917945+0000", 2024-09-14T06:31:43.533 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.062970925999999997 2024-09-14T06:31:43.533 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:43.533 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:43.533 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:31:43.533 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.918030+0000", 2024-09-14T06:31:43.533 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.5863000000000001e-05 2024-09-14T06:31:43.533 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:43.534 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:43.534 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:31:43.534 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.918113+0000", 2024-09-14T06:31:43.534 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.2894999999999998e-05 2024-09-14T06:31:43.534 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:43.534 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:31:43.534 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:43.534 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:43.534 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:43.534 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:44426 3.2 3:4603ad7f:::rbd_data.112dca2f6ae1.0000000000000005:17 [read 892928~4096 [fadvise_nocache]] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:31:43.534 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:43.854977+0000", 2024-09-14T06:31:43.534 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 119.67264640099999, 2024-09-14T06:31:43.534 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.063369237999999994, 2024-09-14T06:31:43.534 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:31:43.534 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:31:43.535 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:31:43.535 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:31:43.535 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:31:43.535 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:43.535 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 44426 2024-09-14T06:31:43.535 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:43.535 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:31:43.535 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:43.535 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:31:43.535 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.854977+0000", 2024-09-14T06:31:43.535 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:43.535 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:43.535 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:43.535 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:31:43.536 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.854977+0000", 2024-09-14T06:31:43.536 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:43.536 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:43.536 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:43.536 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:31:43.536 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.854977+0000", 2024-09-14T06:31:43.536 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.1199999999999999e-07 2024-09-14T06:31:43.536 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:43.536 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:43.536 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:31:43.536 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.854980+0000", 2024-09-14T06:31:43.536 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.27e-06 2024-09-14T06:31:43.536 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:43.536 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:43.537 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:31:43.537 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.854983+0000", 2024-09-14T06:31:43.537 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.1650000000000002e-06 2024-09-14T06:31:43.537 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:43.537 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:43.537 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:31:43.537 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.854989+0000", 2024-09-14T06:31:43.537 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.6520000000000003e-06 2024-09-14T06:31:43.537 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:43.537 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:43.537 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:31:43.537 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.918143+0000", 2024-09-14T06:31:43.537 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.063154541999999994 2024-09-14T06:31:43.537 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:43.538 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:43.538 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:31:43.538 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.918209+0000", 2024-09-14T06:31:43.538 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.5529000000000002e-05 2024-09-14T06:31:43.538 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:43.538 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:43.538 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:31:43.538 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.918346+0000", 2024-09-14T06:31:43.538 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000136868 2024-09-14T06:31:43.538 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:43.538 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:31:43.538 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:43.538 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:43.538 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:43.539 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:44427 3.2 3:4603ad7f:::rbd_data.112dca2f6ae1.0000000000000005:17 [read 917504~4096 [fadvise_nocache]] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:31:43.539 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:43.854992+0000", 2024-09-14T06:31:43.539 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 119.672630755, 2024-09-14T06:31:43.539 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.063607792999999996, 2024-09-14T06:31:43.539 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:31:43.539 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:31:43.539 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:31:43.539 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:31:43.539 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:31:43.539 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:43.539 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 44427 2024-09-14T06:31:43.539 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:43.539 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:31:43.539 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:43.540 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:31:43.540 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.854992+0000", 2024-09-14T06:31:43.540 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:43.540 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:43.540 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:43.540 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:31:43.540 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.854992+0000", 2024-09-14T06:31:43.540 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:43.540 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:43.540 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:43.540 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:31:43.540 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.854993+0000", 2024-09-14T06:31:43.540 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.1400000000000001e-07 2024-09-14T06:31:43.541 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:43.541 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:43.541 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:31:43.541 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.854994+0000", 2024-09-14T06:31:43.541 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.091e-06 2024-09-14T06:31:43.541 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:43.541 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:43.541 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:31:43.541 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855001+0000", 2024-09-14T06:31:43.541 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.204e-06 2024-09-14T06:31:43.541 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:43.541 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:43.541 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:31:43.541 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855003+0000", 2024-09-14T06:31:43.542 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.514e-06 2024-09-14T06:31:43.542 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:43.542 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:43.542 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:31:43.542 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.918380+0000", 2024-09-14T06:31:43.542 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.063376738000000002 2024-09-14T06:31:43.542 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:43.542 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:43.542 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:31:43.542 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.918453+0000", 2024-09-14T06:31:43.542 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.3291e-05 2024-09-14T06:31:43.542 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:43.542 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:43.542 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:31:43.543 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.918600+0000", 2024-09-14T06:31:43.543 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00014674100000000001 2024-09-14T06:31:43.543 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:43.543 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:31:43.543 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:43.543 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:43.543 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:43.543 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:44428 3.2 3:4603ad7f:::rbd_data.112dca2f6ae1.0000000000000005:17 [read 942080~4096 [fadvise_nocache]] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:31:43.543 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:43.855006+0000", 2024-09-14T06:31:43.543 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 119.672616705, 2024-09-14T06:31:43.543 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.063846321999999997, 2024-09-14T06:31:43.543 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:31:43.543 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:31:43.543 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:31:43.543 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:31:43.544 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:31:43.544 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:43.544 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 44428 2024-09-14T06:31:43.544 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:43.544 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:31:43.544 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:43.544 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:31:43.544 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855006+0000", 2024-09-14T06:31:43.544 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:43.544 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:43.544 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:43.544 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:31:43.544 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855006+0000", 2024-09-14T06:31:43.545 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:43.545 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:43.545 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:43.545 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:31:43.545 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855007+0000", 2024-09-14T06:31:43.545 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.1199999999999999e-07 2024-09-14T06:31:43.545 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:43.545 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:43.545 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:31:43.545 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855011+0000", 2024-09-14T06:31:43.545 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.9539999999999998e-06 2024-09-14T06:31:43.545 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:43.545 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:43.545 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:31:43.546 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855014+0000", 2024-09-14T06:31:43.546 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.2930000000000001e-06 2024-09-14T06:31:43.546 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:43.546 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:43.546 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:31:43.546 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855016+0000", 2024-09-14T06:31:43.546 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.3039999999999999e-06 2024-09-14T06:31:43.546 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:43.546 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:43.546 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:31:43.546 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.918634+0000", 2024-09-14T06:31:43.546 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.063617928000000004 2024-09-14T06:31:43.546 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:43.546 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:43.546 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:31:43.547 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.918711+0000", 2024-09-14T06:31:43.547 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.6545000000000007e-05 2024-09-14T06:31:43.547 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:43.547 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:43.547 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:31:43.547 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.918853+0000", 2024-09-14T06:31:43.547 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00014208600000000001 2024-09-14T06:31:43.547 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:43.547 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:31:43.547 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:43.547 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:43.547 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:43.547 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:44429 3.2 3:4603ad7f:::rbd_data.112dca2f6ae1.0000000000000005:17 [read 966656~4096 [fadvise_nocache]] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:31:43.547 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:43.855019+0000", 2024-09-14T06:31:43.548 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 119.672604032, 2024-09-14T06:31:43.548 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.064059226999999996, 2024-09-14T06:31:43.548 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:31:43.548 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:31:43.548 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:31:43.548 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:31:43.548 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:31:43.548 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:43.548 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 44429 2024-09-14T06:31:43.548 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:43.548 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:31:43.548 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:43.548 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:31:43.548 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855019+0000", 2024-09-14T06:31:43.549 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:43.549 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:43.549 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:43.549 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:31:43.549 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855019+0000", 2024-09-14T06:31:43.549 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:43.549 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:43.549 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:43.549 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:31:43.549 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855019+0000", 2024-09-14T06:31:43.549 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.28e-07 2024-09-14T06:31:43.549 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:43.549 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:43.550 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:31:43.550 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855020+0000", 2024-09-14T06:31:43.550 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.0330000000000001e-06 2024-09-14T06:31:43.550 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:43.550 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:43.550 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:31:43.550 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855023+0000", 2024-09-14T06:31:43.550 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.79e-06 2024-09-14T06:31:43.550 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:43.550 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:43.550 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:31:43.550 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855025+0000", 2024-09-14T06:31:43.550 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.216e-06 2024-09-14T06:31:43.551 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:43.551 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:43.551 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:31:43.551 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.918886+0000", 2024-09-14T06:31:43.551 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.063861037999999995 2024-09-14T06:31:43.551 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:43.551 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:43.551 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:31:43.551 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.918952+0000", 2024-09-14T06:31:43.551 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.5993000000000003e-05 2024-09-14T06:31:43.551 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:43.551 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:43.551 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:31:43.551 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.919078+0000", 2024-09-14T06:31:43.552 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00012592899999999999 2024-09-14T06:31:43.552 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:43.552 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:31:43.552 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:43.552 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:43.552 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:43.552 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:44430 3.2 3:4603ad7f:::rbd_data.112dca2f6ae1.0000000000000005:17 [read 991232~4096 [fadvise_nocache]] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:31:43.552 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:43.855028+0000", 2024-09-14T06:31:43.552 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 119.672595022, 2024-09-14T06:31:43.552 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.064284346000000006, 2024-09-14T06:31:43.552 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:31:43.552 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:31:43.552 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:31:43.553 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:31:43.553 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:31:43.553 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:43.553 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 44430 2024-09-14T06:31:43.553 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:43.553 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:31:43.553 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:43.553 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:31:43.557 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855028+0000", 2024-09-14T06:31:43.557 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:43.557 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:43.557 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:43.557 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:31:43.557 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855028+0000", 2024-09-14T06:31:43.558 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:43.558 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:43.558 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:43.558 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:31:43.558 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855028+0000", 2024-09-14T06:31:43.558 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.1199999999999999e-07 2024-09-14T06:31:43.558 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:43.558 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:43.558 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:31:43.558 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855029+0000", 2024-09-14T06:31:43.558 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.1100000000000004e-07 2024-09-14T06:31:43.558 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:43.558 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:43.559 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:31:43.559 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855032+0000", 2024-09-14T06:31:43.559 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.7889999999999999e-06 2024-09-14T06:31:43.559 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:43.559 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:43.559 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:31:43.559 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855034+0000", 2024-09-14T06:31:43.559 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.125e-06 2024-09-14T06:31:43.559 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:43.559 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:43.559 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:31:43.559 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.919110+0000", 2024-09-14T06:31:43.559 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.064076413999999998 2024-09-14T06:31:43.560 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:43.560 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:43.560 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:31:43.560 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.919186+0000", 2024-09-14T06:31:43.560 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.5784999999999994e-05 2024-09-14T06:31:43.560 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:43.560 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:43.560 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:31:43.560 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.919312+0000", 2024-09-14T06:31:43.560 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00012611 2024-09-14T06:31:43.560 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:43.560 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:31:43.560 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:43.561 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:43.561 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:43.561 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:44431 3.2 3:4603ad7f:::rbd_data.112dca2f6ae1.0000000000000005:17 [read 1015808~4096 [fadvise_nocache]] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:31:43.561 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:43.855037+0000", 2024-09-14T06:31:43.561 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 119.672586154, 2024-09-14T06:31:43.561 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.064469755000000004, 2024-09-14T06:31:43.561 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:31:43.561 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:31:43.561 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:31:43.561 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:31:43.561 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:31:43.561 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:43.561 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 44431 2024-09-14T06:31:43.562 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:43.562 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:31:43.562 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:43.562 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:31:43.562 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855037+0000", 2024-09-14T06:31:43.562 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:43.562 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:43.562 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:43.562 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:31:43.562 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855037+0000", 2024-09-14T06:31:43.562 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:43.562 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:43.562 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:43.562 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:31:43.563 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855037+0000", 2024-09-14T06:31:43.563 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.1e-07 2024-09-14T06:31:43.563 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:43.563 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:43.563 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:31:43.563 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855038+0000", 2024-09-14T06:31:43.563 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.9400000000000004e-07 2024-09-14T06:31:43.563 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:43.563 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:43.563 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:31:43.563 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855041+0000", 2024-09-14T06:31:43.563 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.9639999999999999e-06 2024-09-14T06:31:43.563 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:43.564 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:43.564 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:31:43.564 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855043+0000", 2024-09-14T06:31:43.564 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.2259999999999999e-06 2024-09-14T06:31:43.564 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:43.564 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:43.564 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:31:43.564 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.919347+0000", 2024-09-14T06:31:43.564 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.064304091999999993 2024-09-14T06:31:43.564 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:43.564 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:43.564 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:31:43.564 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.919393+0000", 2024-09-14T06:31:43.564 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.5340999999999998e-05 2024-09-14T06:31:43.565 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:43.565 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:43.565 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:31:43.565 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.919507+0000", 2024-09-14T06:31:43.565 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000114028 2024-09-14T06:31:43.565 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:43.565 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:31:43.565 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:43.565 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:43.565 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:43.565 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:44432 3.2 3:4603ad7f:::rbd_data.112dca2f6ae1.0000000000000005:17 [read 1040384~4096 [fadvise_nocache]] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:31:43.565 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:43.855046+0000", 2024-09-14T06:31:43.565 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 119.672577086, 2024-09-14T06:31:43.566 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.064639253999999993, 2024-09-14T06:31:43.566 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:31:43.566 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:31:43.566 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:31:43.566 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:31:43.566 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:31:43.566 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:43.566 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 44432 2024-09-14T06:31:43.566 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:43.566 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:31:43.566 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:43.566 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:31:43.566 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855046+0000", 2024-09-14T06:31:43.566 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:43.567 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:43.567 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:43.567 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:31:43.567 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855046+0000", 2024-09-14T06:31:43.567 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:43.567 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:43.567 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:43.567 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:31:43.567 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855046+0000", 2024-09-14T06:31:43.567 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.05e-07 2024-09-14T06:31:43.567 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:43.567 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:43.567 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:31:43.568 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855047+0000", 2024-09-14T06:31:43.568 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.879999999999999e-07 2024-09-14T06:31:43.568 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:43.568 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:43.568 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:31:43.568 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855050+0000", 2024-09-14T06:31:43.568 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.8459999999999999e-06 2024-09-14T06:31:43.568 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:43.568 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:43.568 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:31:43.568 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855052+0000", 2024-09-14T06:31:43.568 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.136e-06 2024-09-14T06:31:43.568 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:43.569 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:43.569 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:31:43.569 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.919536+0000", 2024-09-14T06:31:43.569 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.064484132 2024-09-14T06:31:43.569 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:43.569 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:43.569 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:31:43.569 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.919611+0000", 2024-09-14T06:31:43.569 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.4407999999999996e-05 2024-09-14T06:31:43.569 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:43.569 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:43.569 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:31:43.569 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.919685+0000", 2024-09-14T06:31:43.569 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.4538999999999999e-05 2024-09-14T06:31:43.570 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:43.570 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:31:43.570 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:43.570 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:43.570 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:43.570 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:44518 3.2 3:4190d882:::rbd_data.112dca2f6ae1.0000000000000002:17 [read 12288~4096 [fadvise_nocache]] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:31:43.570 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:43.855055+0000", 2024-09-14T06:31:43.570 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 119.67256820199999, 2024-09-14T06:31:43.570 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.064894317000000007, 2024-09-14T06:31:43.570 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:31:43.570 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:31:43.570 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:31:43.570 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:31:43.571 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:31:43.571 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:43.571 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 44518 2024-09-14T06:31:43.571 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:43.571 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:31:43.571 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:43.571 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:31:43.571 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855055+0000", 2024-09-14T06:31:43.571 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:43.571 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:43.571 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:43.571 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:31:43.571 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855055+0000", 2024-09-14T06:31:43.571 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:43.572 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:43.572 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:43.572 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:31:43.572 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855055+0000", 2024-09-14T06:31:43.572 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.0699999999999999e-07 2024-09-14T06:31:43.572 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:43.572 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:43.572 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:31:43.572 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855056+0000", 2024-09-14T06:31:43.572 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.9400000000000004e-07 2024-09-14T06:31:43.572 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:43.572 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:43.572 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:31:43.572 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855059+0000", 2024-09-14T06:31:43.573 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.8870000000000001e-06 2024-09-14T06:31:43.573 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:43.573 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:43.573 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:31:43.573 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855061+0000", 2024-09-14T06:31:43.573 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.1919999999999999e-06 2024-09-14T06:31:43.573 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:43.573 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:43.573 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:31:43.573 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.919719+0000", 2024-09-14T06:31:43.573 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.064658013 2024-09-14T06:31:43.573 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:43.573 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:43.573 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:31:43.574 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.919788+0000", 2024-09-14T06:31:43.574 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.9282000000000005e-05 2024-09-14T06:31:43.574 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:43.574 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:43.574 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:31:43.574 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.919949+0000", 2024-09-14T06:31:43.574 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00016084199999999999 2024-09-14T06:31:43.574 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:43.574 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:31:43.574 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:43.574 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:43.574 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:43.574 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:44519 3.2 3:4190d882:::rbd_data.112dca2f6ae1.0000000000000002:17 [read 36864~4096 [fadvise_nocache]] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:31:43.575 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:43.855064+0000", 2024-09-14T06:31:43.575 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 119.672559225, 2024-09-14T06:31:43.575 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.065081865000000003, 2024-09-14T06:31:43.575 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:31:43.575 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:31:43.575 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:31:43.575 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:31:43.575 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:31:43.575 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:43.575 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 44519 2024-09-14T06:31:43.575 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:43.575 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:31:43.575 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:43.575 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:31:43.576 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855064+0000", 2024-09-14T06:31:43.576 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:43.576 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:43.576 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:43.576 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:31:43.576 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855064+0000", 2024-09-14T06:31:43.576 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:43.576 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:43.576 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:43.576 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:31:43.576 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855064+0000", 2024-09-14T06:31:43.576 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.1e-07 2024-09-14T06:31:43.576 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:43.577 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:43.577 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:31:43.577 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855065+0000", 2024-09-14T06:31:43.577 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.035e-06 2024-09-14T06:31:43.577 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:43.577 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:43.577 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:31:43.577 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855068+0000", 2024-09-14T06:31:43.577 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.8339999999999999e-06 2024-09-14T06:31:43.577 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:43.577 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:43.577 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:31:43.577 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855071+0000", 2024-09-14T06:31:43.578 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.3450000000000002e-06 2024-09-14T06:31:43.578 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:43.578 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:43.578 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:31:43.578 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.919979+0000", 2024-09-14T06:31:43.578 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.064907647999999998 2024-09-14T06:31:43.578 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:43.578 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:43.578 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:31:43.578 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.920026+0000", 2024-09-14T06:31:43.578 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.7595000000000001e-05 2024-09-14T06:31:43.578 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:43.578 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:43.578 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:31:43.579 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.920146+0000", 2024-09-14T06:31:43.579 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00011919800000000001 2024-09-14T06:31:43.579 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:43.579 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:31:43.579 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:43.579 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:43.579 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:43.579 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:44520 3.2 3:4190d882:::rbd_data.112dca2f6ae1.0000000000000002:17 [read 61440~4096 [fadvise_nocache]] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:31:43.579 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:43.855075+0000", 2024-09-14T06:31:43.579 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 119.672548174, 2024-09-14T06:31:43.579 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.065335078000000005, 2024-09-14T06:31:43.579 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:31:43.579 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:31:43.579 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:31:43.580 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:31:43.580 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:31:43.580 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:43.580 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 44520 2024-09-14T06:31:43.580 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:43.580 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:31:43.580 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:43.580 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:31:43.580 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855075+0000", 2024-09-14T06:31:43.580 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:43.580 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:43.580 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:43.581 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:31:43.581 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855075+0000", 2024-09-14T06:31:43.581 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:43.581 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:43.581 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:43.581 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:31:43.581 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855075+0000", 2024-09-14T06:31:43.581 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.1e-07 2024-09-14T06:31:43.581 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:43.581 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:43.581 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:31:43.581 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855076+0000", 2024-09-14T06:31:43.581 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.0839999999999999e-06 2024-09-14T06:31:43.581 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:43.582 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:43.582 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:31:43.582 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855079+0000", 2024-09-14T06:31:43.582 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.9359999999999999e-06 2024-09-14T06:31:43.582 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:43.582 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:43.582 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:31:43.582 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855084+0000", 2024-09-14T06:31:43.582 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.1730000000000003e-06 2024-09-14T06:31:43.582 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:43.582 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:43.582 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:31:43.582 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.920175+0000", 2024-09-14T06:31:43.582 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.065090234999999996 2024-09-14T06:31:43.582 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:43.583 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:43.583 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:31:43.583 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.920250+0000", 2024-09-14T06:31:43.583 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.5807000000000001e-05 2024-09-14T06:31:43.583 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:43.583 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:43.583 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:31:43.583 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.920410+0000", 2024-09-14T06:31:43.583 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00015963300000000001 2024-09-14T06:31:43.583 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:43.583 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:31:43.583 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:43.583 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:43.583 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:43.584 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:44521 3.2 3:4190d882:::rbd_data.112dca2f6ae1.0000000000000002:17 [read 86016~4096 [fadvise_nocache]] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:31:43.584 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:43.855087+0000", 2024-09-14T06:31:43.584 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 119.672535693, 2024-09-14T06:31:43.584 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.065563264999999996, 2024-09-14T06:31:43.584 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:31:43.584 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:31:43.584 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:31:43.584 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:31:43.584 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:31:43.584 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:43.584 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 44521 2024-09-14T06:31:43.584 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:43.584 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:31:43.585 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:43.585 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:31:43.585 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855087+0000", 2024-09-14T06:31:43.585 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:43.585 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:43.585 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:43.585 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:31:43.585 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855087+0000", 2024-09-14T06:31:43.585 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:43.585 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:43.585 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:43.585 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:31:43.585 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855088+0000", 2024-09-14T06:31:43.585 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.3699999999999999e-07 2024-09-14T06:31:43.586 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:43.586 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:43.586 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:31:43.586 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855089+0000", 2024-09-14T06:31:43.586 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.9900000000000009e-07 2024-09-14T06:31:43.586 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:43.586 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:43.586 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:31:43.586 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855091+0000", 2024-09-14T06:31:43.586 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.8940000000000002e-06 2024-09-14T06:31:43.586 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:43.586 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:43.586 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:31:43.586 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855094+0000", 2024-09-14T06:31:43.587 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.351e-06 2024-09-14T06:31:43.587 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:43.587 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:43.587 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:31:43.587 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.920437+0000", 2024-09-14T06:31:43.587 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.065343221000000007 2024-09-14T06:31:43.587 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:43.587 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:43.587 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:31:43.587 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.920481+0000", 2024-09-14T06:31:43.587 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.4431000000000002e-05 2024-09-14T06:31:43.587 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:43.587 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:43.587 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:31:43.588 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.920651+0000", 2024-09-14T06:31:43.588 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00016913200000000001 2024-09-14T06:31:43.588 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:43.588 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:31:43.588 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:43.588 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:43.588 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:43.588 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:44522 3.2 3:4190d882:::rbd_data.112dca2f6ae1.0000000000000002:17 [read 110592~4096 [fadvise_nocache]] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:31:43.588 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:43.855098+0000", 2024-09-14T06:31:43.588 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 119.672525466, 2024-09-14T06:31:43.588 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.065827628999999999, 2024-09-14T06:31:43.588 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:31:43.588 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:31:43.588 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:31:43.589 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:31:43.589 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:31:43.589 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:43.589 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 44522 2024-09-14T06:31:43.589 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:43.589 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:31:43.589 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:43.589 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:31:43.589 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855098+0000", 2024-09-14T06:31:43.589 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:43.589 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:43.589 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:43.589 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:31:43.590 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855098+0000", 2024-09-14T06:31:43.590 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:43.590 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:43.590 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:43.590 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:31:43.590 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855098+0000", 2024-09-14T06:31:43.590 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.1e-07 2024-09-14T06:31:43.590 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:43.590 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:43.590 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:31:43.590 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855099+0000", 2024-09-14T06:31:43.590 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.012e-06 2024-09-14T06:31:43.590 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:43.591 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:43.591 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:31:43.591 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855102+0000", 2024-09-14T06:31:43.591 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.9380000000000001e-06 2024-09-14T06:31:43.591 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:43.591 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:43.591 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:31:43.591 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855108+0000", 2024-09-14T06:31:43.591 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.4339999999999997e-06 2024-09-14T06:31:43.591 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:43.591 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:43.591 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:31:43.592 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.920681+0000", 2024-09-14T06:31:43.592 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.065572943999999994 2024-09-14T06:31:43.592 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:43.592 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:43.592 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:31:43.592 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.920724+0000", 2024-09-14T06:31:43.592 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.2877000000000003e-05 2024-09-14T06:31:43.592 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:43.592 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:43.592 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:31:43.592 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.920925+0000", 2024-09-14T06:31:43.592 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00020121399999999999 2024-09-14T06:31:43.592 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:43.592 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:31:43.593 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:43.593 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:43.593 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:43.593 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:44523 3.2 3:4190d882:::rbd_data.112dca2f6ae1.0000000000000002:17 [read 135168~4096 [fadvise_nocache]] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:31:43.593 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:43.855111+0000", 2024-09-14T06:31:43.593 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 119.672511744, 2024-09-14T06:31:43.593 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.066028198999999996, 2024-09-14T06:31:43.593 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:31:43.593 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:31:43.593 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:31:43.593 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:31:43.593 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:31:43.593 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:43.594 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 44523 2024-09-14T06:31:43.594 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:43.594 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:31:43.594 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:43.594 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:31:43.594 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855111+0000", 2024-09-14T06:31:43.594 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:43.594 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:43.594 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:43.594 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:31:43.594 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855111+0000", 2024-09-14T06:31:43.594 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:43.594 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:43.594 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:43.594 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:31:43.595 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855112+0000", 2024-09-14T06:31:43.595 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.1199999999999999e-07 2024-09-14T06:31:43.595 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:43.595 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:43.595 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:31:43.595 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855115+0000", 2024-09-14T06:31:43.595 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.3380000000000001e-06 2024-09-14T06:31:43.595 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:43.595 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:43.595 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:31:43.595 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855118+0000", 2024-09-14T06:31:43.595 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.366e-06 2024-09-14T06:31:43.595 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:43.595 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:43.596 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:31:43.596 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855121+0000", 2024-09-14T06:31:43.596 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.8329999999999998e-06 2024-09-14T06:31:43.596 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:43.596 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:43.596 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:31:43.596 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.920956+0000", 2024-09-14T06:31:43.596 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.065835251999999997 2024-09-14T06:31:43.596 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:43.596 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:43.596 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:31:43.596 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.921001+0000", 2024-09-14T06:31:43.596 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.4778999999999999e-05 2024-09-14T06:31:43.596 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:43.597 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:43.597 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:31:43.597 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.921139+0000", 2024-09-14T06:31:43.597 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000138419 2024-09-14T06:31:43.597 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:43.597 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:31:43.597 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:43.597 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:43.597 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:43.597 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:44524 3.2 3:4190d882:::rbd_data.112dca2f6ae1.0000000000000002:17 [read 159744~4096 [fadvise_nocache]] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:31:43.597 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:43.855124+0000", 2024-09-14T06:31:43.597 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 119.672498765, 2024-09-14T06:31:43.597 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.066249261000000004, 2024-09-14T06:31:43.597 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:31:43.598 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:31:43.598 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:31:43.598 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:31:43.598 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:31:43.598 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:43.598 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 44524 2024-09-14T06:31:43.598 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:43.598 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:31:43.598 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:43.598 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:31:43.598 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855124+0000", 2024-09-14T06:31:43.598 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:43.598 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:43.598 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:43.599 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:31:43.599 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855124+0000", 2024-09-14T06:31:43.599 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:43.599 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:43.599 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:43.599 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:31:43.599 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861086+0000", 2024-09-14T06:31:43.599 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0059613829999999998 2024-09-14T06:31:43.599 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:43.599 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:43.599 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:31:43.599 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861100+0000", 2024-09-14T06:31:43.599 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.4644000000000001e-05 2024-09-14T06:31:43.599 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:43.599 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:43.600 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:31:43.600 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861109+0000", 2024-09-14T06:31:43.600 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.541e-06 2024-09-14T06:31:43.600 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:43.600 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:43.600 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:31:43.600 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861117+0000", 2024-09-14T06:31:43.600 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.1049999999999995e-06 2024-09-14T06:31:43.600 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:43.600 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:43.600 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:31:43.600 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.921169+0000", 2024-09-14T06:31:43.600 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.060052405000000003 2024-09-14T06:31:43.600 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:43.600 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:43.601 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:31:43.601 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.921213+0000", 2024-09-14T06:31:43.601 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.3565e-05 2024-09-14T06:31:43.601 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:43.601 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:43.601 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:31:43.601 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.921374+0000", 2024-09-14T06:31:43.601 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00016061800000000001 2024-09-14T06:31:43.601 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:43.601 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:31:43.601 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:43.601 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:43.601 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:43.601 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:44536 3.2 3:4190d882:::rbd_data.112dca2f6ae1.0000000000000002:17 [read 454656~4096 [fadvise_nocache]] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:31:43.602 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:43.861384+0000", 2024-09-14T06:31:43.602 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 119.66623920799999, 2024-09-14T06:31:43.602 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.063058528000000003, 2024-09-14T06:31:43.602 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:31:43.602 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:31:43.602 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:31:43.602 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:31:43.602 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:31:43.602 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:43.602 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 44536 2024-09-14T06:31:43.602 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:43.602 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:31:43.602 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:43.602 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:31:43.602 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861384+0000", 2024-09-14T06:31:43.603 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:43.603 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:43.603 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:43.603 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:31:43.603 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861384+0000", 2024-09-14T06:31:43.603 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:43.603 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:43.603 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:43.603 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:31:43.603 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861384+0000", 2024-09-14T06:31:43.603 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.4000000000000002e-07 2024-09-14T06:31:43.603 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:43.603 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:43.603 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:31:43.603 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861386+0000", 2024-09-14T06:31:43.603 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.891e-06 2024-09-14T06:31:43.604 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:43.604 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:43.604 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:31:43.604 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861392+0000", 2024-09-14T06:31:43.604 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.9939999999999997e-06 2024-09-14T06:31:43.604 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:43.604 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:43.604 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:31:43.604 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861398+0000", 2024-09-14T06:31:43.604 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.507e-06 2024-09-14T06:31:43.604 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:43.604 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:43.604 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:31:43.604 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.924149+0000", 2024-09-14T06:31:43.604 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.062751135999999999 2024-09-14T06:31:43.605 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:43.605 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:43.605 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:31:43.605 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.924226+0000", 2024-09-14T06:31:43.605 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.7517999999999999e-05 2024-09-14T06:31:43.605 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:43.605 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:43.605 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:31:43.605 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.924442+0000", 2024-09-14T06:31:43.605 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000216042 2024-09-14T06:31:43.605 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:43.605 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:31:43.605 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:43.605 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:43.605 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:43.606 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:44537 3.2 3:4190d882:::rbd_data.112dca2f6ae1.0000000000000002:17 [read 479232~4096 [fadvise_nocache]] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:31:43.606 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:43.861405+0000", 2024-09-14T06:31:43.606 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 119.66621816200001, 2024-09-14T06:31:43.606 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.063351795000000002, 2024-09-14T06:31:43.606 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:31:43.606 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:31:43.606 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:31:43.606 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:31:43.606 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:31:43.606 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:43.606 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 44537 2024-09-14T06:31:43.606 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:43.606 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:31:43.606 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:43.607 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:31:43.607 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861405+0000", 2024-09-14T06:31:43.607 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:43.607 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:43.607 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:43.607 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:31:43.607 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861405+0000", 2024-09-14T06:31:43.607 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:43.607 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:43.607 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:43.607 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:31:43.607 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861405+0000", 2024-09-14T06:31:43.607 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.9099999999999999e-07 2024-09-14T06:31:43.607 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:43.608 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:43.608 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:31:43.608 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861407+0000", 2024-09-14T06:31:43.608 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.075e-06 2024-09-14T06:31:43.608 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:43.608 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:43.608 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:31:43.608 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861414+0000", 2024-09-14T06:31:43.608 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.4659999999999996e-06 2024-09-14T06:31:43.608 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:43.608 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:43.608 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:31:43.608 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861418+0000", 2024-09-14T06:31:43.608 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.5519999999999998e-06 2024-09-14T06:31:43.609 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:43.609 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:43.609 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:31:43.609 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.924474+0000", 2024-09-14T06:31:43.609 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.063056012999999994 2024-09-14T06:31:43.609 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:43.609 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:43.609 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:31:43.609 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.924556+0000", 2024-09-14T06:31:43.609 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.1181999999999996e-05 2024-09-14T06:31:43.609 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:43.609 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:43.609 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:31:43.609 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.924757+0000", 2024-09-14T06:31:43.609 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00020111599999999999 2024-09-14T06:31:43.610 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:43.610 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:31:43.610 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:43.610 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:43.610 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:43.610 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:44538 3.2 3:4190d882:::rbd_data.112dca2f6ae1.0000000000000002:17 [read 503808~4096 [fadvise_nocache]] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:31:43.610 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:43.861425+0000", 2024-09-14T06:31:43.610 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 119.66619812099999, 2024-09-14T06:31:43.610 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.063627125000000007, 2024-09-14T06:31:43.610 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:31:43.610 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:31:43.610 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:31:43.610 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:31:43.610 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:31:43.610 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:43.611 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 44538 2024-09-14T06:31:43.611 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:43.611 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:31:43.611 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:43.611 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:31:43.611 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861425+0000", 2024-09-14T06:31:43.611 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:43.611 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:43.611 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:43.611 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:31:43.611 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861425+0000", 2024-09-14T06:31:43.611 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:43.611 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:43.611 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:43.612 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:31:43.612 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861425+0000", 2024-09-14T06:31:43.612 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.9999999999999998e-07 2024-09-14T06:31:43.612 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:43.612 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:43.612 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:31:43.612 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861427+0000", 2024-09-14T06:31:43.612 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.9829999999999999e-06 2024-09-14T06:31:43.612 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:43.612 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:43.612 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:31:43.612 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861433+0000", 2024-09-14T06:31:43.612 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.2889999999999997e-06 2024-09-14T06:31:43.612 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:43.612 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:43.613 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:31:43.613 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861454+0000", 2024-09-14T06:31:43.613 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.1844999999999998e-05 2024-09-14T06:31:43.613 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:43.613 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:43.613 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:31:43.613 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.924786+0000", 2024-09-14T06:31:43.613 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.063331497 2024-09-14T06:31:43.613 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:43.613 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:43.613 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:31:43.613 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.924864+0000", 2024-09-14T06:31:43.613 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.7744000000000006e-05 2024-09-14T06:31:43.614 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:43.614 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:43.614 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:31:43.614 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.925052+0000", 2024-09-14T06:31:43.614 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00018836699999999999 2024-09-14T06:31:43.614 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:43.614 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:31:43.614 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:43.614 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:43.614 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:43.614 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:44539 3.2 3:4190d882:::rbd_data.112dca2f6ae1.0000000000000002:17 [read 528384~4096 [fadvise_nocache]] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:31:43.614 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:43.861462+0000", 2024-09-14T06:31:43.614 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 119.66616079400001, 2024-09-14T06:31:43.614 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.063855705999999998, 2024-09-14T06:31:43.615 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:31:43.615 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:31:43.615 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:31:43.615 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:31:43.615 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:31:43.615 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:43.615 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 44539 2024-09-14T06:31:43.615 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:43.615 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:31:43.615 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:43.615 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:31:43.615 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861462+0000", 2024-09-14T06:31:43.615 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:43.615 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:43.616 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:43.616 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:31:43.616 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861462+0000", 2024-09-14T06:31:43.616 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:43.616 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:43.616 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:43.616 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:31:43.616 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861463+0000", 2024-09-14T06:31:43.616 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.5200000000000002e-07 2024-09-14T06:31:43.616 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:43.616 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:43.616 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:31:43.616 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861465+0000", 2024-09-14T06:31:43.616 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.5189999999999999e-06 2024-09-14T06:31:43.616 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:43.617 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:43.617 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:31:43.617 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861471+0000", 2024-09-14T06:31:43.617 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.6899999999999997e-06 2024-09-14T06:31:43.617 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:43.617 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:43.617 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:31:43.617 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861475+0000", 2024-09-14T06:31:43.617 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.5730000000000001e-06 2024-09-14T06:31:43.617 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:43.617 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:43.617 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:31:43.617 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.925085+0000", 2024-09-14T06:31:43.617 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.063609902999999995 2024-09-14T06:31:43.617 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:43.617 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:43.618 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:31:43.618 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.925145+0000", 2024-09-14T06:31:43.618 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.9383999999999999e-05 2024-09-14T06:31:43.618 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:43.618 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:43.618 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:31:43.618 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.925318+0000", 2024-09-14T06:31:43.618 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00017318499999999999 2024-09-14T06:31:43.618 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:43.618 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:31:43.618 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:43.618 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:43.618 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:43.618 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:44540 3.2 3:4190d882:::rbd_data.112dca2f6ae1.0000000000000002:17 [read 552960~4096 [fadvise_nocache]] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:31:43.618 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:43.861483+0000", 2024-09-14T06:31:43.618 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 119.666140363, 2024-09-14T06:31:43.619 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.064065681999999999, 2024-09-14T06:31:43.619 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:31:43.619 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:31:43.619 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:31:43.619 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:31:43.619 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:31:43.619 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:43.619 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 44540 2024-09-14T06:31:43.619 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:43.619 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:31:43.619 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:43.619 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:31:43.619 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861483+0000", 2024-09-14T06:31:43.619 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:43.620 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:43.620 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:43.620 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:31:43.620 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861483+0000", 2024-09-14T06:31:43.620 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:43.620 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:43.620 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:43.620 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:31:43.620 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.867079+0000", 2024-09-14T06:31:43.620 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0055966480000000001 2024-09-14T06:31:43.620 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:43.620 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:43.620 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:31:43.621 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.867086+0000", 2024-09-14T06:31:43.621 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.387e-06 2024-09-14T06:31:43.621 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:43.621 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:43.621 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:31:43.621 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.867094+0000", 2024-09-14T06:31:43.621 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.9419999999999995e-06 2024-09-14T06:31:43.621 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:43.621 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:43.621 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:31:43.621 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.867109+0000", 2024-09-14T06:31:43.621 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.5818999999999999e-05 2024-09-14T06:31:43.621 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:43.621 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:43.622 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:31:43.622 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.925354+0000", 2024-09-14T06:31:43.622 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.058244047 2024-09-14T06:31:43.622 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:43.622 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:43.622 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:31:43.622 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.925406+0000", 2024-09-14T06:31:43.622 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.2661999999999998e-05 2024-09-14T06:31:43.622 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:43.622 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:43.622 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:31:43.622 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.925548+0000", 2024-09-14T06:31:43.622 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000142177 2024-09-14T06:31:43.622 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:43.622 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:31:43.623 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:43.623 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:43.623 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:31:43.623 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-14T06:31:43.624 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-14T06:31:43.659 INFO:teuthology.orchestra.run.smithi094.stderr:no valid command found; 10 closest matches: 2024-09-14T06:31:43.660 INFO:teuthology.orchestra.run.smithi094.stderr:0 2024-09-14T06:31:43.660 INFO:teuthology.orchestra.run.smithi094.stderr:1 2024-09-14T06:31:43.660 INFO:teuthology.orchestra.run.smithi094.stderr:2 2024-09-14T06:31:43.660 INFO:teuthology.orchestra.run.smithi094.stderr:abort 2024-09-14T06:31:43.660 INFO:teuthology.orchestra.run.smithi094.stderr:assert 2024-09-14T06:31:43.660 INFO:teuthology.orchestra.run.smithi094.stderr:bench [] [] [] [] 2024-09-14T06:31:43.660 INFO:teuthology.orchestra.run.smithi094.stderr:bluefs debug_inject_read_zeros 2024-09-14T06:31:43.660 INFO:teuthology.orchestra.run.smithi094.stderr:bluefs files list 2024-09-14T06:31:43.660 INFO:teuthology.orchestra.run.smithi094.stderr:bluefs stats 2024-09-14T06:31:43.660 INFO:teuthology.orchestra.run.smithi094.stderr:bluestore allocator dump block 2024-09-14T06:31:43.660 INFO:teuthology.orchestra.run.smithi094.stderr:Error EINVAL: invalid command 2024-09-14T06:31:43.661 INFO:tasks.ceph.osd.3.smithi094.stderr:2024-09-14T06:31:43.658+0000 7feeeafe8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33655) UID: 0 2024-09-14T06:31:43.662 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-14T06:31:43.662 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-14T06:31:43.663 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 'trim stale osdmaps' 2024-09-14T06:31:43.726 INFO:tasks.ceph.osd.5.smithi138.stderr:2024-09-14T06:31:43.725+0000 7fe13746f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33618) UID: 0 2024-09-14T06:31:43.752 INFO:teuthology.orchestra.run.smithi138.stdout:{ 2024-09-14T06:31:43.752 INFO:teuthology.orchestra.run.smithi138.stdout: "ops": [ 2024-09-14T06:31:43.752 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:43.752 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_repop(client.4397.0:218318 3.1d e179/23 3:bfa89004:::rbd_object_map.112df8d32353.0000000000000052:head v 179'804, mlcod=179'797)", 2024-09-14T06:31:43.752 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:31:43.746640+0000", 2024-09-14T06:31:43.753 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 0.0044049329999999998, 2024-09-14T06:31:43.753 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.0044271229999999998, 2024-09-14T06:31:43.753 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:31:43.753 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:31:43.753 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:31:43.753 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:31:43.753 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:43.753 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:31:43.753 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:31:43.746640+0000", 2024-09-14T06:31:43.753 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:43.753 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:43.753 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:43.753 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:31:43.753 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:31:43.746640+0000", 2024-09-14T06:31:43.754 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:43.754 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:43.754 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:43.754 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:31:43.754 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:31:43.746640+0000", 2024-09-14T06:31:43.754 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.3399999999999999e-07 2024-09-14T06:31:43.754 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:43.754 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:43.754 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:31:43.754 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:31:43.746645+0000", 2024-09-14T06:31:43.754 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.527e-06 2024-09-14T06:31:43.754 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:43.754 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:43.754 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:31:43.754 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:31:43.746651+0000", 2024-09-14T06:31:43.755 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.7429999999999996e-06 2024-09-14T06:31:43.755 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:43.755 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:43.755 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:31:43.755 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:31:43.746657+0000", 2024-09-14T06:31:43.755 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.6200000000000004e-06 2024-09-14T06:31:43.755 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:43.755 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:43.755 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:31:43.755 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:31:43.746710+0000", 2024-09-14T06:31:43.755 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.2457999999999998e-05 2024-09-14T06:31:43.755 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:43.755 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:43.755 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:31:43.755 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:31:43.746730+0000", 2024-09-14T06:31:43.756 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.0951e-05 2024-09-14T06:31:43.756 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:43.756 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:31:43.756 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:43.756 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:43.756 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:43.756 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:218338 3.14 3:2bf92b75:::rbd_data.112df8d32353.0000000000000017:head [write 0~16384 [fadvise_sequential+fadvise_nocache] in=16384b,write 65536~16384 [fadvise_sequential+fadvise_nocache] in=16384b,write 131072~16384 [fadvise_sequential+fadvise_nocache] in=16384b,write 196608~9884 [fadvise_sequential+fadvise_nocache] in=9884b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e179)", 2024-09-14T06:31:43.756 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:31:43.747276+0000", 2024-09-14T06:31:43.756 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 0.0037689590000000001, 2024-09-14T06:31:43.756 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.0038623960000000001, 2024-09-14T06:31:43.756 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:31:43.756 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:31:43.756 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "waiting for sub ops", 2024-09-14T06:31:43.756 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:31:43.756 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:31:43.757 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:43.757 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 218338 2024-09-14T06:31:43.757 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:43.757 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:31:43.757 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:43.757 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:31:43.757 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:31:43.747276+0000", 2024-09-14T06:31:43.757 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:43.757 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:43.757 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:43.757 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:31:43.757 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:31:43.747276+0000", 2024-09-14T06:31:43.758 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:43.758 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:43.758 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:43.758 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:31:43.758 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:31:43.747276+0000", 2024-09-14T06:31:43.758 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.8000000000000004e-07 2024-09-14T06:31:43.758 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:43.758 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:43.758 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:31:43.758 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:31:43.747321+0000", 2024-09-14T06:31:43.758 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.4885000000000001e-05 2024-09-14T06:31:43.759 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:43.759 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:43.759 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:31:43.759 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:31:43.747333+0000", 2024-09-14T06:31:43.759 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.1691999999999999e-05 2024-09-14T06:31:43.759 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:43.759 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:43.759 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:31:43.759 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:31:43.747341+0000", 2024-09-14T06:31:43.759 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.571e-06 2024-09-14T06:31:43.759 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:43.759 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:43.759 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:31:43.760 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:31:43.747406+0000", 2024-09-14T06:31:43.760 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.5618999999999996e-05 2024-09-14T06:31:43.760 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:43.760 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:43.760 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:31:43.760 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:31:43.747552+0000", 2024-09-14T06:31:43.760 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00014605600000000001 2024-09-14T06:31:43.760 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:43.760 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:43.760 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "waiting for subops from 7", 2024-09-14T06:31:43.760 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:31:43.747752+0000", 2024-09-14T06:31:43.760 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.000199314 2024-09-14T06:31:43.760 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:43.760 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:43.760 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:31:43.761 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:31:43.750720+0000", 2024-09-14T06:31:43.761 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.002968058 2024-09-14T06:31:43.761 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:43.761 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:31:43.761 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:43.764 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:43.764 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:43.765 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_repop(client.4397.0:218337 3.19 e179/171 3:98f8fa06:::rbd_data.112df8d32353.0000000000000019:head v 179'420, mlcod=177'418)", 2024-09-14T06:31:43.765 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:31:43.747643+0000", 2024-09-14T06:31:43.765 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 0.0034016509999999999, 2024-09-14T06:31:43.765 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.0035694329999999999, 2024-09-14T06:31:43.765 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:31:43.765 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:31:43.765 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:31:43.765 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:31:43.765 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:43.765 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:31:43.765 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:31:43.747643+0000", 2024-09-14T06:31:43.765 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:43.765 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:43.766 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:43.766 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:31:43.766 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:31:43.747643+0000", 2024-09-14T06:31:43.766 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:43.766 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:43.766 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:43.766 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:31:43.766 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:31:43.747644+0000", 2024-09-14T06:31:43.766 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.0200000000000001e-07 2024-09-14T06:31:43.766 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:43.766 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:43.766 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:31:43.766 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:31:43.747896+0000", 2024-09-14T06:31:43.766 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00025278200000000001 2024-09-14T06:31:43.767 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:43.767 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:43.767 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:31:43.767 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:31:43.747906+0000", 2024-09-14T06:31:43.767 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.9830000000000008e-06 2024-09-14T06:31:43.767 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:43.767 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:43.767 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:31:43.767 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:31:43.747920+0000", 2024-09-14T06:31:43.767 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.3939e-05 2024-09-14T06:31:43.767 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:43.767 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:43.767 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:31:43.768 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:31:43.747985+0000", 2024-09-14T06:31:43.768 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.4179999999999999e-05 2024-09-14T06:31:43.768 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:43.768 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:43.768 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:31:43.768 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:31:43.748004+0000", 2024-09-14T06:31:43.768 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.9346000000000001e-05 2024-09-14T06:31:43.768 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:43.768 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:31:43.769 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:43.769 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:43.769 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:43.769 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:218372 3.1a 3:5f18e465:::rbd_data.112d3bd84cb0.000000000000001a:snapdir [list-snaps] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e179)", 2024-09-14T06:31:43.769 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:31:43.749923+0000", 2024-09-14T06:31:43.769 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 0.0011214440000000001, 2024-09-14T06:31:43.769 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.0013449549999999999, 2024-09-14T06:31:43.769 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:31:43.769 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:31:43.769 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "reached pg", 2024-09-14T06:31:43.769 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:31:43.769 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:31:43.769 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:43.770 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 218372 2024-09-14T06:31:43.770 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:43.770 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:31:43.770 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:43.770 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:31:43.770 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:31:43.749923+0000", 2024-09-14T06:31:43.770 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:43.770 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:43.770 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:43.770 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:31:43.770 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:31:43.749923+0000", 2024-09-14T06:31:43.770 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:43.770 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:43.771 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:43.771 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:31:43.771 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:31:43.749924+0000", 2024-09-14T06:31:43.771 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.3099999999999996e-07 2024-09-14T06:31:43.771 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:43.771 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:43.771 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:31:43.771 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:31:43.749932+0000", 2024-09-14T06:31:43.771 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.3040000000000005e-06 2024-09-14T06:31:43.771 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:43.771 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:43.771 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:31:43.771 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:31:43.749943+0000", 2024-09-14T06:31:43.771 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.0507e-05 2024-09-14T06:31:43.772 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:43.772 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:43.772 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:31:43.772 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:31:43.749949+0000", 2024-09-14T06:31:43.772 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.5950000000000004e-06 2024-09-14T06:31:43.772 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:43.772 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:43.772 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:31:43.772 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:31:43.751217+0000", 2024-09-14T06:31:43.772 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.001267727 2024-09-14T06:31:43.772 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:43.772 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:31:43.773 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:43.773 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:43.773 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:43.773 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:218377 3.1a 3.316ac77a (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e179)", 2024-09-14T06:31:43.773 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:31:43.749973+0000", 2024-09-14T06:31:43.773 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 0.00107132, 2024-09-14T06:31:43.773 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00135407, 2024-09-14T06:31:43.773 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:31:43.773 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:31:43.773 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "queued for pg", 2024-09-14T06:31:43.773 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:31:43.773 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:31:43.773 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:43.774 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 218377 2024-09-14T06:31:43.774 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:43.774 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:31:43.774 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:43.774 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:31:43.774 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:31:43.749973+0000", 2024-09-14T06:31:43.774 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:43.774 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:43.774 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:43.774 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:31:43.774 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:31:43.749973+0000", 2024-09-14T06:31:43.774 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:43.774 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:43.775 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:43.775 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:31:43.775 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:31:43.749974+0000", 2024-09-14T06:31:43.775 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.1500000000000004e-07 2024-09-14T06:31:43.775 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:43.775 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:43.775 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:31:43.775 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:31:43.749977+0000", 2024-09-14T06:31:43.775 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.0299999999999998e-06 2024-09-14T06:31:43.775 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:43.775 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:43.775 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:31:43.775 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:31:43.749984+0000", 2024-09-14T06:31:43.775 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.4869999999999996e-06 2024-09-14T06:31:43.776 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:43.776 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:43.776 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:31:43.776 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:31:43.749989+0000", 2024-09-14T06:31:43.776 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.9609999999999996e-06 2024-09-14T06:31:43.776 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:43.776 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:31:43.776 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:43.776 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:43.776 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:43.776 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_repop(client.4397.0:218336 3.1d e179/23 3:bfa89004:::rbd_object_map.112df8d32353.0000000000000052:head v 179'805, mlcod=179'802)", 2024-09-14T06:31:43.776 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:31:43.750458+0000", 2024-09-14T06:31:43.776 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 0.00058611600000000002, 2024-09-14T06:31:43.776 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00093006499999999999, 2024-09-14T06:31:43.777 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:31:43.777 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:31:43.777 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:31:43.777 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:31:43.777 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:43.777 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:31:43.777 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:31:43.750458+0000", 2024-09-14T06:31:43.777 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:43.777 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:43.777 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:43.777 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:31:43.777 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:31:43.750458+0000", 2024-09-14T06:31:43.778 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:43.778 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:43.778 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:43.778 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:31:43.778 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:31:43.750459+0000", 2024-09-14T06:31:43.778 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.5700000000000001e-07 2024-09-14T06:31:43.778 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:43.778 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:43.778 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:31:43.778 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:31:43.750465+0000", 2024-09-14T06:31:43.778 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.9839999999999998e-06 2024-09-14T06:31:43.778 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:43.778 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:43.778 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:31:43.778 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:31:43.750477+0000", 2024-09-14T06:31:43.779 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.1987000000000001e-05 2024-09-14T06:31:43.779 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:43.779 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:43.779 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:31:43.779 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:31:43.750484+0000", 2024-09-14T06:31:43.779 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.8750000000000002e-06 2024-09-14T06:31:43.779 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:43.779 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:43.779 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:31:43.779 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:31:43.750545+0000", 2024-09-14T06:31:43.779 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.1023000000000002e-05 2024-09-14T06:31:43.779 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:43.779 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:43.779 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:31:43.780 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:31:43.750563+0000", 2024-09-14T06:31:43.780 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.7960000000000001e-05 2024-09-14T06:31:43.780 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:43.780 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:31:43.780 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:43.780 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:43.780 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:43.780 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:218389 3.1a 3.316ac77a (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e179)", 2024-09-14T06:31:43.780 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:31:43.751042+0000", 2024-09-14T06:31:43.780 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 3.061e-06, 2024-09-14T06:31:43.780 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.000396566, 2024-09-14T06:31:43.780 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:31:43.780 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:31:43.780 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "queued for pg", 2024-09-14T06:31:43.781 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:31:43.781 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:31:43.781 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:43.781 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 218389 2024-09-14T06:31:43.781 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:43.781 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:31:43.781 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:43.781 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:31:43.781 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:31:43.751042+0000", 2024-09-14T06:31:43.781 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:43.781 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:43.781 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:43.781 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:31:43.781 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:31:43.751042+0000", 2024-09-14T06:31:43.782 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:43.782 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:43.782 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:43.782 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:31:43.782 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:31:43.751042+0000", 2024-09-14T06:31:43.782 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.0699999999999996e-07 2024-09-14T06:31:43.782 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:43.782 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:43.782 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:31:43.782 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:31:43.751047+0000", 2024-09-14T06:31:43.782 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.6229999999999996e-06 2024-09-14T06:31:43.782 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:43.782 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:43.782 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:31:43.783 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:31:43.751057+0000", 2024-09-14T06:31:43.783 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.0390999999999999e-05 2024-09-14T06:31:43.783 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:43.783 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:43.783 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:31:43.783 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:31:43.751064+0000", 2024-09-14T06:31:43.783 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.759e-06 2024-09-14T06:31:43.783 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:43.783 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:31:43.783 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:43.783 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:43.783 INFO:teuthology.orchestra.run.smithi138.stdout: ], 2024-09-14T06:31:43.783 INFO:teuthology.orchestra.run.smithi138.stdout: "num_ops": 7 2024-09-14T06:31:43.783 INFO:teuthology.orchestra.run.smithi138.stdout:} 2024-09-14T06:31:43.785 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-14T06:31:43.827 INFO:tasks.ceph.osd.7.smithi138.stderr:2024-09-14T06:31:43.825+0000 7f3c6a21d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33623) UID: 0 2024-09-14T06:31:43.882 INFO:teuthology.orchestra.run.smithi094.stderr:no valid command found; 10 closest matches: 2024-09-14T06:31:43.883 INFO:teuthology.orchestra.run.smithi094.stderr:0 2024-09-14T06:31:43.883 INFO:teuthology.orchestra.run.smithi094.stderr:1 2024-09-14T06:31:43.883 INFO:teuthology.orchestra.run.smithi094.stderr:2 2024-09-14T06:31:43.883 INFO:teuthology.orchestra.run.smithi094.stderr:abort 2024-09-14T06:31:43.883 INFO:teuthology.orchestra.run.smithi094.stderr:assert 2024-09-14T06:31:43.883 INFO:teuthology.orchestra.run.smithi094.stderr:bench [] [] [] [] 2024-09-14T06:31:43.883 INFO:teuthology.orchestra.run.smithi094.stderr:bluefs debug_inject_read_zeros 2024-09-14T06:31:43.883 INFO:teuthology.orchestra.run.smithi094.stderr:bluefs files list 2024-09-14T06:31:43.883 INFO:teuthology.orchestra.run.smithi094.stderr:bluefs stats 2024-09-14T06:31:43.883 INFO:teuthology.orchestra.run.smithi094.stderr:bluestore allocator dump block 2024-09-14T06:31:43.883 INFO:teuthology.orchestra.run.smithi094.stderr:Error EINVAL: invalid command 2024-09-14T06:31:43.886 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-14T06:31:43.886 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-14T06:31:43.927 INFO:tasks.ceph.osd.3.smithi094.stderr:2024-09-14T06:31:43.926+0000 7feeeafe8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33655) UID: 0 2024-09-14T06:31:43.975 INFO:teuthology.orchestra.run.smithi138.stdout:{ 2024-09-14T06:31:43.975 INFO:teuthology.orchestra.run.smithi138.stdout: "ops": [], 2024-09-14T06:31:43.975 INFO:teuthology.orchestra.run.smithi138.stdout: "complaint_time": 30, 2024-09-14T06:31:43.975 INFO:teuthology.orchestra.run.smithi138.stdout: "num_blocked_ops": 0 2024-09-14T06:31:43.975 INFO:teuthology.orchestra.run.smithi138.stdout:} 2024-09-14T06:31:43.984 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-14T06:31:44.029 INFO:tasks.ceph.osd.1.smithi094.stderr:2024-09-14T06:31:44.028+0000 7f328e47a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33641) UID: 0 2024-09-14T06:31:44.088 INFO:teuthology.orchestra.run.smithi138.stdout:{ 2024-09-14T06:31:44.088 INFO:teuthology.orchestra.run.smithi138.stdout: "size": 20, 2024-09-14T06:31:44.088 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 600, 2024-09-14T06:31:44.088 INFO:teuthology.orchestra.run.smithi138.stdout: "ops": [ 2024-09-14T06:31:44.089 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:44.089 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:19328 3.3 3:c34e5e51:::rbd_data.112d92027590.0000000000000024:f [read 1926144~1024 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:31:44.089 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:27.059768+0000", 2024-09-14T06:31:44.089 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 137.02685775399999, 2024-09-14T06:31:44.089 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046369147999999999, 2024-09-14T06:31:44.089 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:31:44.089 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:31:44.089 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:31:44.089 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:31:44.089 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:31:44.089 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:44.089 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 19328 2024-09-14T06:31:44.089 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:44.090 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:31:44.090 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:44.090 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:31:44.090 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.059768+0000", 2024-09-14T06:31:44.090 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:44.090 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:44.090 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:44.090 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:31:44.090 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.059768+0000", 2024-09-14T06:31:44.090 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:44.090 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:44.090 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:44.090 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:31:44.090 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.059874+0000", 2024-09-14T06:31:44.090 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00010551400000000001 2024-09-14T06:31:44.091 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:44.091 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:44.091 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:31:44.091 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.059877+0000", 2024-09-14T06:31:44.091 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.8279999999999999e-06 2024-09-14T06:31:44.091 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:44.091 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:44.091 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:31:44.091 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.059884+0000", 2024-09-14T06:31:44.091 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.6389999999999992e-06 2024-09-14T06:31:44.091 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:44.091 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:44.091 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:31:44.091 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.059890+0000", 2024-09-14T06:31:44.092 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.49e-06 2024-09-14T06:31:44.092 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:44.092 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:44.092 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:31:44.092 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.105804+0000", 2024-09-14T06:31:44.092 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.045914242000000001 2024-09-14T06:31:44.092 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:44.092 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:44.092 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:31:44.092 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.105886+0000", 2024-09-14T06:31:44.092 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.2305999999999994e-05 2024-09-14T06:31:44.092 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:44.092 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:44.093 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:31:44.093 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.106138+0000", 2024-09-14T06:31:44.093 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.000251129 2024-09-14T06:31:44.093 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:44.093 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:31:44.093 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:44.093 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:44.093 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:44.093 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:19329 3.3 3:c34e5e51:::rbd_data.112d92027590.0000000000000024:f [read 1936384~1024 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:31:44.094 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:27.059899+0000", 2024-09-14T06:31:44.094 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 137.026727231, 2024-09-14T06:31:44.094 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046572122000000001, 2024-09-14T06:31:44.094 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:31:44.094 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:31:44.094 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:31:44.094 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:31:44.094 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:31:44.094 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:44.094 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 19329 2024-09-14T06:31:44.094 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:44.094 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:31:44.094 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:44.094 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:31:44.095 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.059899+0000", 2024-09-14T06:31:44.095 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:44.095 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:44.095 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:44.095 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:31:44.095 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.059899+0000", 2024-09-14T06:31:44.095 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:44.095 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:44.095 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:44.095 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:31:44.095 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062501+0000", 2024-09-14T06:31:44.096 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.0026016419999999999 2024-09-14T06:31:44.096 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:44.096 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:44.096 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:31:44.096 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062514+0000", 2024-09-14T06:31:44.096 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.2947e-05 2024-09-14T06:31:44.096 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:44.096 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:44.096 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:31:44.096 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062523+0000", 2024-09-14T06:31:44.097 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.5519999999999993e-06 2024-09-14T06:31:44.097 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:44.097 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:44.097 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:31:44.097 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062534+0000", 2024-09-14T06:31:44.097 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.1416e-05 2024-09-14T06:31:44.097 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:44.097 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:44.097 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:31:44.097 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.106179+0000", 2024-09-14T06:31:44.097 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.043644049999999997 2024-09-14T06:31:44.097 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:44.097 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:44.098 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:31:44.098 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.106262+0000", 2024-09-14T06:31:44.098 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.3178999999999998e-05 2024-09-14T06:31:44.098 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:44.098 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:44.098 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:31:44.098 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.106471+0000", 2024-09-14T06:31:44.098 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00020933599999999999 2024-09-14T06:31:44.098 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:44.098 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:31:44.098 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:44.098 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:44.098 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:44.099 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:19351 3.3 3:c34e5e51:::rbd_data.112d92027590.0000000000000024:f [read 1159168~1024 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:31:44.099 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:27.062766+0000", 2024-09-14T06:31:44.099 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 137.02386000600001, 2024-09-14T06:31:44.099 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046099454999999998, 2024-09-14T06:31:44.099 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:31:44.099 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:31:44.099 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:31:44.099 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:31:44.099 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:31:44.099 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:44.099 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 19351 2024-09-14T06:31:44.099 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:44.099 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:31:44.099 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:44.100 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:31:44.100 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062766+0000", 2024-09-14T06:31:44.100 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:44.100 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:44.100 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:44.100 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:31:44.100 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062766+0000", 2024-09-14T06:31:44.100 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:44.100 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:44.100 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:44.100 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:31:44.100 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062766+0000", 2024-09-14T06:31:44.100 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.6e-07 2024-09-14T06:31:44.100 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:44.101 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:44.101 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:31:44.101 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062767+0000", 2024-09-14T06:31:44.101 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.0070000000000001e-06 2024-09-14T06:31:44.101 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:44.101 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:44.101 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:31:44.101 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062785+0000", 2024-09-14T06:31:44.101 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.7884999999999999e-05 2024-09-14T06:31:44.101 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:44.101 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:44.101 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:31:44.101 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062790+0000", 2024-09-14T06:31:44.101 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.0900000000000004e-06 2024-09-14T06:31:44.101 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:44.102 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:44.102 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:31:44.102 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.108735+0000", 2024-09-14T06:31:44.102 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.045944411999999997 2024-09-14T06:31:44.102 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:44.102 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:44.102 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:31:44.102 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.108809+0000", 2024-09-14T06:31:44.102 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.4157000000000002e-05 2024-09-14T06:31:44.102 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:44.102 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:44.102 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:31:44.102 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.108866+0000", 2024-09-14T06:31:44.102 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.6644000000000002e-05 2024-09-14T06:31:44.102 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:44.103 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:31:44.103 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:44.103 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:44.103 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:44.103 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:19352 3.3 3:c34e5e51:::rbd_data.112d92027590.0000000000000024:f [read 1169408~1024 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:31:44.103 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:27.062798+0000", 2024-09-14T06:31:44.103 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 137.023828595, 2024-09-14T06:31:44.103 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046255900000000003, 2024-09-14T06:31:44.103 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:31:44.103 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:31:44.103 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:31:44.103 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:31:44.103 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:31:44.103 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:44.104 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 19352 2024-09-14T06:31:44.104 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:44.104 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:31:44.104 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:44.104 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:31:44.104 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062798+0000", 2024-09-14T06:31:44.104 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:44.104 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:44.104 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:44.104 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:31:44.104 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062798+0000", 2024-09-14T06:31:44.104 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:44.104 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:44.104 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:44.105 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:31:44.105 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062798+0000", 2024-09-14T06:31:44.105 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.37e-07 2024-09-14T06:31:44.105 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:44.105 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:44.105 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:31:44.105 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062800+0000", 2024-09-14T06:31:44.105 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.0269999999999998e-06 2024-09-14T06:31:44.105 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:44.105 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:44.105 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:31:44.105 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062806+0000", 2024-09-14T06:31:44.105 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.6690000000000003e-06 2024-09-14T06:31:44.105 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:44.106 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:44.106 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:31:44.106 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062810+0000", 2024-09-14T06:31:44.106 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.3449999999999999e-06 2024-09-14T06:31:44.106 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:44.106 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:44.106 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:31:44.106 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.108892+0000", 2024-09-14T06:31:44.106 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046081715000000002 2024-09-14T06:31:44.106 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:44.106 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:44.106 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:31:44.106 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.108940+0000", 2024-09-14T06:31:44.106 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.7815999999999998e-05 2024-09-14T06:31:44.107 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:44.107 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:44.107 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:31:44.107 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.109053+0000", 2024-09-14T06:31:44.107 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.000113791 2024-09-14T06:31:44.107 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:44.107 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:31:44.107 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:44.107 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:44.107 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:44.107 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:19353 3.3 3:c34e5e51:::rbd_data.112d92027590.0000000000000024:f [read 1179648~1024 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:31:44.107 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:27.062816+0000", 2024-09-14T06:31:44.107 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 137.02381007599999, 2024-09-14T06:31:44.107 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046481256999999998, 2024-09-14T06:31:44.108 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:31:44.108 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:31:44.108 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:31:44.108 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:31:44.108 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:31:44.108 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:44.108 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 19353 2024-09-14T06:31:44.108 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:44.108 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:31:44.108 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:44.108 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:31:44.108 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062816+0000", 2024-09-14T06:31:44.108 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:44.108 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:44.109 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:44.109 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:31:44.109 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062816+0000", 2024-09-14T06:31:44.109 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:44.109 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:44.109 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:44.109 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:31:44.109 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062816+0000", 2024-09-14T06:31:44.109 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.2899999999999999e-07 2024-09-14T06:31:44.109 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:44.109 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:44.109 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:31:44.109 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062818+0000", 2024-09-14T06:31:44.110 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.699e-06 2024-09-14T06:31:44.110 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:44.110 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:44.110 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:31:44.110 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062825+0000", 2024-09-14T06:31:44.110 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.878e-06 2024-09-14T06:31:44.110 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:44.110 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:44.110 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:31:44.110 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062830+0000", 2024-09-14T06:31:44.110 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.5410000000000002e-06 2024-09-14T06:31:44.110 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:44.110 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:44.110 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:31:44.111 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.109096+0000", 2024-09-14T06:31:44.111 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046266517 2024-09-14T06:31:44.111 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:44.111 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:44.111 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:31:44.111 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.109190+0000", 2024-09-14T06:31:44.111 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.4000999999999997e-05 2024-09-14T06:31:44.111 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:44.111 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:44.111 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:31:44.111 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.109297+0000", 2024-09-14T06:31:44.111 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.000107292 2024-09-14T06:31:44.111 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:44.111 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:31:44.112 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:44.112 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:44.112 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:44.112 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:19354 3.3 3:c34e5e51:::rbd_data.112d92027590.0000000000000024:f [read 1189888~1024 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:31:44.112 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:27.062842+0000", 2024-09-14T06:31:44.112 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 137.023784621, 2024-09-14T06:31:44.112 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046594257, 2024-09-14T06:31:44.112 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:31:44.112 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:31:44.112 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:31:44.112 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:31:44.112 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:31:44.112 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:44.113 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 19354 2024-09-14T06:31:44.113 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:44.113 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:31:44.113 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:44.113 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:31:44.113 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062842+0000", 2024-09-14T06:31:44.113 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:44.113 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:44.113 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:44.113 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:31:44.113 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062842+0000", 2024-09-14T06:31:44.113 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:44.113 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:44.113 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:44.114 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:31:44.114 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062842+0000", 2024-09-14T06:31:44.114 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.7399999999999999e-07 2024-09-14T06:31:44.114 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:44.114 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:44.114 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:31:44.114 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062859+0000", 2024-09-14T06:31:44.114 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.7010000000000001e-05 2024-09-14T06:31:44.114 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:44.114 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:44.114 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:31:44.114 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062866+0000", 2024-09-14T06:31:44.114 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.9990000000000002e-06 2024-09-14T06:31:44.114 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:44.115 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:44.115 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:31:44.115 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062873+0000", 2024-09-14T06:31:44.115 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.8569999999999997e-06 2024-09-14T06:31:44.115 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:44.115 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:44.115 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:31:44.115 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.109335+0000", 2024-09-14T06:31:44.115 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046461943999999998 2024-09-14T06:31:44.115 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:44.115 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:44.115 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:31:44.115 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.109377+0000", 2024-09-14T06:31:44.115 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.2558999999999997e-05 2024-09-14T06:31:44.116 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:44.116 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:44.116 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:31:44.116 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.109436+0000", 2024-09-14T06:31:44.116 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.8514000000000003e-05 2024-09-14T06:31:44.116 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:44.116 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:31:44.116 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:44.116 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:44.116 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:44.116 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:19355 3.3 3:c34e5e51:::rbd_data.112d92027590.0000000000000024:f [read 1200128~1024 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:31:44.116 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:27.062884+0000", 2024-09-14T06:31:44.116 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 137.02374194800001, 2024-09-14T06:31:44.116 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046681650999999998, 2024-09-14T06:31:44.116 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:31:44.117 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:31:44.117 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:31:44.117 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:31:44.117 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:31:44.117 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:44.117 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 19355 2024-09-14T06:31:44.117 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:44.117 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:31:44.117 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:44.117 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:31:44.117 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062884+0000", 2024-09-14T06:31:44.117 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:44.117 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:44.117 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:44.118 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:31:44.118 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062884+0000", 2024-09-14T06:31:44.118 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:44.118 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:44.118 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:44.118 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:31:44.118 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062885+0000", 2024-09-14T06:31:44.118 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.8000000000000001e-07 2024-09-14T06:31:44.118 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:44.118 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:44.118 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:31:44.118 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062886+0000", 2024-09-14T06:31:44.118 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.8920000000000001e-06 2024-09-14T06:31:44.118 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:44.118 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:44.119 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:31:44.119 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062892+0000", 2024-09-14T06:31:44.119 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.84e-06 2024-09-14T06:31:44.119 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:44.119 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:44.119 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:31:44.119 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062899+0000", 2024-09-14T06:31:44.119 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.5339999999999996e-06 2024-09-14T06:31:44.119 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:44.119 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:44.119 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:31:44.119 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.109466+0000", 2024-09-14T06:31:44.119 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046566919999999998 2024-09-14T06:31:44.119 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:44.119 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:44.119 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:31:44.120 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.109510+0000", 2024-09-14T06:31:44.120 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.4153000000000002e-05 2024-09-14T06:31:44.120 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:44.120 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:44.120 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:31:44.120 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.109566+0000", 2024-09-14T06:31:44.120 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.5931999999999999e-05 2024-09-14T06:31:44.120 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:44.120 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:31:44.120 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:44.120 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:44.120 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:44.120 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:19356 3.3 3:c34e5e51:::rbd_data.112d92027590.0000000000000024:f [read 1210368~1024 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:31:44.120 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:27.062908+0000", 2024-09-14T06:31:44.121 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 137.023718511, 2024-09-14T06:31:44.121 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046807498000000003, 2024-09-14T06:31:44.121 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:31:44.121 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:31:44.121 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:31:44.121 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:31:44.121 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:31:44.121 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:44.121 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 19356 2024-09-14T06:31:44.121 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:44.121 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:31:44.121 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:44.121 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:31:44.121 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062908+0000", 2024-09-14T06:31:44.121 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:44.122 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:44.122 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:44.122 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:31:44.122 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062908+0000", 2024-09-14T06:31:44.122 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:44.122 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:44.122 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:44.122 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:31:44.122 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062908+0000", 2024-09-14T06:31:44.122 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.5699999999999998e-07 2024-09-14T06:31:44.122 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:44.122 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:44.122 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:31:44.122 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062910+0000", 2024-09-14T06:31:44.122 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.7710000000000001e-06 2024-09-14T06:31:44.123 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:44.123 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:44.123 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:31:44.123 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062920+0000", 2024-09-14T06:31:44.123 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.0654e-05 2024-09-14T06:31:44.123 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:44.123 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:44.123 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:31:44.123 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062931+0000", 2024-09-14T06:31:44.123 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.075e-05 2024-09-14T06:31:44.123 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:44.123 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:44.123 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:31:44.123 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.109595+0000", 2024-09-14T06:31:44.124 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046663713000000002 2024-09-14T06:31:44.124 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:44.124 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:44.124 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:31:44.124 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.109632+0000", 2024-09-14T06:31:44.124 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.7505000000000001e-05 2024-09-14T06:31:44.124 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:44.124 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:44.124 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:31:44.124 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.109715+0000", 2024-09-14T06:31:44.124 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.2748000000000001e-05 2024-09-14T06:31:44.124 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:44.124 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:31:44.124 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:44.125 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:44.125 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:44.125 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:19357 3.3 3:c34e5e51:::rbd_data.112d92027590.0000000000000024:f [read 1220608~1024 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:31:44.125 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:27.062939+0000", 2024-09-14T06:31:44.125 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 137.02368706300001, 2024-09-14T06:31:44.125 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046920508, 2024-09-14T06:31:44.125 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:31:44.125 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:31:44.125 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:31:44.125 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:31:44.125 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:31:44.125 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:44.125 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 19357 2024-09-14T06:31:44.126 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:44.126 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:31:44.126 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:44.126 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:31:44.126 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062939+0000", 2024-09-14T06:31:44.126 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:44.126 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:44.126 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:44.126 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:31:44.126 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062939+0000", 2024-09-14T06:31:44.126 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:44.126 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:44.126 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:44.127 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:31:44.127 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062940+0000", 2024-09-14T06:31:44.127 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.32e-07 2024-09-14T06:31:44.127 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:44.127 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:44.127 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:31:44.127 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062942+0000", 2024-09-14T06:31:44.127 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.1919999999999999e-06 2024-09-14T06:31:44.127 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:44.127 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:44.127 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:31:44.127 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062948+0000", 2024-09-14T06:31:44.127 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.1149999999999999e-06 2024-09-14T06:31:44.127 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:44.128 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:44.128 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:31:44.128 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062960+0000", 2024-09-14T06:31:44.128 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.1895e-05 2024-09-14T06:31:44.128 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:44.128 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:44.128 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:31:44.128 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.109704+0000", 2024-09-14T06:31:44.128 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046744584999999998 2024-09-14T06:31:44.129 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:44.129 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:44.129 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:31:44.129 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.109777+0000", 2024-09-14T06:31:44.129 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.2282999999999999e-05 2024-09-14T06:31:44.129 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:44.129 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:44.129 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:31:44.129 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.109860+0000", 2024-09-14T06:31:44.129 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.3005999999999998e-05 2024-09-14T06:31:44.129 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:44.129 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:31:44.129 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:44.130 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:44.130 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:44.130 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:19358 3.3 3:c34e5e51:::rbd_data.112d92027590.0000000000000024:f [read 1230848~1024 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:31:44.130 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:27.062967+0000", 2024-09-14T06:31:44.130 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 137.02365943300001, 2024-09-14T06:31:44.130 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047144471, 2024-09-14T06:31:44.130 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:31:44.130 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:31:44.130 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:31:44.130 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:31:44.130 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:31:44.130 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:44.130 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 19358 2024-09-14T06:31:44.131 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:44.131 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:31:44.131 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:44.131 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:31:44.131 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062967+0000", 2024-09-14T06:31:44.131 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:44.131 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:44.131 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:44.131 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:31:44.131 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062967+0000", 2024-09-14T06:31:44.131 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:44.131 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:44.131 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:44.131 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:31:44.132 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062967+0000", 2024-09-14T06:31:44.132 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.89e-07 2024-09-14T06:31:44.132 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:44.132 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:44.132 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:31:44.132 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062969+0000", 2024-09-14T06:31:44.132 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.9929999999999998e-06 2024-09-14T06:31:44.132 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:44.132 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:44.132 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:31:44.132 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062975+0000", 2024-09-14T06:31:44.132 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.9030000000000001e-06 2024-09-14T06:31:44.132 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:44.132 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:44.133 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:31:44.133 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062979+0000", 2024-09-14T06:31:44.133 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.2250000000000002e-06 2024-09-14T06:31:44.133 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:44.133 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:44.133 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:31:44.133 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.109934+0000", 2024-09-14T06:31:44.133 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046954410000000002 2024-09-14T06:31:44.133 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:44.133 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:44.133 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:31:44.133 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.110007+0000", 2024-09-14T06:31:44.133 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.3829000000000001e-05 2024-09-14T06:31:44.133 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:44.134 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:44.134 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:31:44.134 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.110111+0000", 2024-09-14T06:31:44.134 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00010372199999999999 2024-09-14T06:31:44.134 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:44.134 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:31:44.134 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:44.134 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:44.134 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:44.134 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:19359 3.3 3:c34e5e51:::rbd_data.112d92027590.0000000000000024:f [read 1241088~1024 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:31:44.134 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:27.062987+0000", 2024-09-14T06:31:44.134 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 137.02363934300001, 2024-09-14T06:31:44.134 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047297182, 2024-09-14T06:31:44.134 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:31:44.135 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:31:44.135 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:31:44.135 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:31:44.135 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:31:44.135 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:44.135 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 19359 2024-09-14T06:31:44.135 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:44.135 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:31:44.135 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:44.135 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:31:44.135 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062987+0000", 2024-09-14T06:31:44.135 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:44.135 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:44.135 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:44.135 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:31:44.136 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062987+0000", 2024-09-14T06:31:44.136 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:44.136 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:44.136 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:44.136 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:31:44.136 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062987+0000", 2024-09-14T06:31:44.136 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.2500000000000001e-07 2024-09-14T06:31:44.136 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:44.136 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:44.136 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:31:44.136 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062989+0000", 2024-09-14T06:31:44.136 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.6080000000000001e-06 2024-09-14T06:31:44.136 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:44.137 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:44.137 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:31:44.137 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062994+0000", 2024-09-14T06:31:44.137 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.5520000000000004e-06 2024-09-14T06:31:44.137 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:44.137 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:44.137 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:31:44.137 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062999+0000", 2024-09-14T06:31:44.137 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.3930000000000001e-06 2024-09-14T06:31:44.137 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:44.137 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:44.137 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:31:44.137 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.110142+0000", 2024-09-14T06:31:44.137 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047143154 2024-09-14T06:31:44.137 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:44.138 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:44.138 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:31:44.138 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.110219+0000", 2024-09-14T06:31:44.138 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.6737999999999996e-05 2024-09-14T06:31:44.138 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:44.138 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:44.138 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:31:44.138 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.110284+0000", 2024-09-14T06:31:44.138 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.5412000000000003e-05 2024-09-14T06:31:44.138 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:44.138 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:31:44.138 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:44.138 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:44.139 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:44.139 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:19360 3.3 3:c34e5e51:::rbd_data.112d92027590.0000000000000024:f [read 1251328~1024 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:31:44.139 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:27.063017+0000", 2024-09-14T06:31:44.139 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 137.02360888199999, 2024-09-14T06:31:44.139 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047453823999999999, 2024-09-14T06:31:44.139 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:31:44.139 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:31:44.139 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:31:44.139 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:31:44.139 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:31:44.139 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:44.139 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 19360 2024-09-14T06:31:44.139 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:44.139 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:31:44.140 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:44.140 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:31:44.140 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.063017+0000", 2024-09-14T06:31:44.140 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:44.140 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:44.140 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:44.140 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:31:44.140 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.063017+0000", 2024-09-14T06:31:44.140 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:44.140 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:44.140 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:44.140 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:31:44.140 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.064977+0000", 2024-09-14T06:31:44.140 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.0019597080000000001 2024-09-14T06:31:44.141 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:44.141 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:44.141 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:31:44.141 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.064987+0000", 2024-09-14T06:31:44.141 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.9620000000000005e-06 2024-09-14T06:31:44.141 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:44.141 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:44.141 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:31:44.141 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065000+0000", 2024-09-14T06:31:44.141 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.3173000000000001e-05 2024-09-14T06:31:44.141 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:44.141 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:44.141 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:31:44.142 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065005+0000", 2024-09-14T06:31:44.142 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.9960000000000001e-06 2024-09-14T06:31:44.142 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:44.142 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:44.142 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:31:44.142 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.110314+0000", 2024-09-14T06:31:44.142 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.04530899 2024-09-14T06:31:44.142 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:44.142 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:44.142 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:31:44.142 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.110398+0000", 2024-09-14T06:31:44.142 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.3539000000000001e-05 2024-09-14T06:31:44.142 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:44.142 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:44.143 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:31:44.143 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.110471+0000", 2024-09-14T06:31:44.143 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.3455999999999996e-05 2024-09-14T06:31:44.143 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:44.143 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:31:44.143 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:44.143 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:44.143 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:44.143 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:19365 3.3 3:c34e5e51:::rbd_data.112d92027590.0000000000000024:f [read 1302528~1024 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:31:44.143 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:27.065058+0000", 2024-09-14T06:31:44.143 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 137.02156821200001, 2024-09-14T06:31:44.143 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046105526000000001, 2024-09-14T06:31:44.143 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:31:44.143 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:31:44.144 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:31:44.144 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:31:44.144 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:31:44.144 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:44.144 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 19365 2024-09-14T06:31:44.144 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:44.144 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:31:44.144 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:44.144 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:31:44.144 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065058+0000", 2024-09-14T06:31:44.144 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:44.144 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:44.144 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:44.144 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:31:44.145 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065058+0000", 2024-09-14T06:31:44.145 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:44.145 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:44.145 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:44.145 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:31:44.145 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065058+0000", 2024-09-14T06:31:44.145 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.1899999999999999e-07 2024-09-14T06:31:44.145 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:44.145 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:44.145 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:31:44.145 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065059+0000", 2024-09-14T06:31:44.145 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.1299999999999998e-07 2024-09-14T06:31:44.145 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:44.145 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:44.146 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:31:44.146 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065063+0000", 2024-09-14T06:31:44.146 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.1099999999999996e-06 2024-09-14T06:31:44.146 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:44.146 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:44.146 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:31:44.146 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065070+0000", 2024-09-14T06:31:44.146 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.973e-06 2024-09-14T06:31:44.146 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:44.146 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:44.146 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:31:44.146 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.111014+0000", 2024-09-14T06:31:44.146 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.045943373000000003 2024-09-14T06:31:44.146 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:44.146 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:44.147 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:31:44.147 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.111073+0000", 2024-09-14T06:31:44.147 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.8987999999999998e-05 2024-09-14T06:31:44.147 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:44.147 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:44.147 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:31:44.147 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.111163+0000", 2024-09-14T06:31:44.147 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.0950000000000002e-05 2024-09-14T06:31:44.147 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:44.147 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:31:44.147 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:44.147 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:44.147 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:44.147 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:19366 3.3 3:c34e5e51:::rbd_data.112d92027590.0000000000000024:f [read 1312768~1024 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:31:44.148 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:27.065074+0000", 2024-09-14T06:31:44.148 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 137.02155236199999, 2024-09-14T06:31:44.148 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046289858000000003, 2024-09-14T06:31:44.148 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:31:44.148 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:31:44.148 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:31:44.148 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:31:44.148 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:31:44.148 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:44.148 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 19366 2024-09-14T06:31:44.148 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:44.148 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:31:44.148 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:44.148 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:31:44.149 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065074+0000", 2024-09-14T06:31:44.149 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:44.149 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:44.149 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:44.149 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:31:44.149 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065074+0000", 2024-09-14T06:31:44.149 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:44.149 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:44.149 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:44.149 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:31:44.149 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065074+0000", 2024-09-14T06:31:44.149 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.3900000000000001e-07 2024-09-14T06:31:44.149 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:44.150 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:44.150 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:31:44.150 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065078+0000", 2024-09-14T06:31:44.150 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.3229999999999999e-06 2024-09-14T06:31:44.150 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:44.150 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:44.150 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:31:44.150 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065082+0000", 2024-09-14T06:31:44.150 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.8299999999999998e-06 2024-09-14T06:31:44.150 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:44.150 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:44.150 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:31:44.150 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065085+0000", 2024-09-14T06:31:44.151 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.8660000000000002e-06 2024-09-14T06:31:44.151 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:44.151 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:44.151 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:31:44.151 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.111270+0000", 2024-09-14T06:31:44.151 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046185035999999999 2024-09-14T06:31:44.151 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:44.151 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:44.151 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:31:44.151 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.111311+0000", 2024-09-14T06:31:44.151 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.0935e-05 2024-09-14T06:31:44.151 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:44.151 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:44.151 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:31:44.151 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.111364+0000", 2024-09-14T06:31:44.152 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.2629e-05 2024-09-14T06:31:44.152 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:44.152 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:31:44.152 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:44.152 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:44.152 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:44.152 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:19367 3.3 3:c34e5e51:::rbd_data.112d92027590.0000000000000024:f [read 1323008~1024 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:31:44.152 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:27.065115+0000", 2024-09-14T06:31:44.152 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 137.02151080100001, 2024-09-14T06:31:44.152 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046407003000000002, 2024-09-14T06:31:44.152 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:31:44.152 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:31:44.152 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:31:44.153 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:31:44.153 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:31:44.153 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:44.153 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 19367 2024-09-14T06:31:44.153 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:44.153 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:31:44.153 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:44.153 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:31:44.153 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065115+0000", 2024-09-14T06:31:44.153 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:44.153 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:44.153 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:44.153 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:31:44.153 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065115+0000", 2024-09-14T06:31:44.153 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:44.154 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:44.154 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:44.154 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:31:44.154 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065116+0000", 2024-09-14T06:31:44.154 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.4999999999999998e-07 2024-09-14T06:31:44.154 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:44.154 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:44.154 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:31:44.154 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065117+0000", 2024-09-14T06:31:44.154 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.189e-06 2024-09-14T06:31:44.154 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:44.154 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:44.154 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:31:44.154 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065122+0000", 2024-09-14T06:31:44.155 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.3709999999999999e-06 2024-09-14T06:31:44.155 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:44.155 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:44.155 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:31:44.155 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065125+0000", 2024-09-14T06:31:44.155 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.8940000000000002e-06 2024-09-14T06:31:44.155 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:44.155 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:44.155 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:31:44.155 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.111402+0000", 2024-09-14T06:31:44.155 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046276645999999998 2024-09-14T06:31:44.155 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:44.155 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:44.155 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:31:44.156 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.111445+0000", 2024-09-14T06:31:44.156 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.3288000000000003e-05 2024-09-14T06:31:44.156 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:44.156 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:44.156 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:31:44.156 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.111522+0000", 2024-09-14T06:31:44.156 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.7164999999999997e-05 2024-09-14T06:31:44.156 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:44.156 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:31:44.156 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:44.156 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:44.156 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:44.156 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:19368 3.3 3:c34e5e51:::rbd_data.112d92027590.0000000000000024:f [read 1333248~1024 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:31:44.156 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:27.065130+0000", 2024-09-14T06:31:44.157 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 137.02149651100001, 2024-09-14T06:31:44.157 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046507275000000001, 2024-09-14T06:31:44.157 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:31:44.157 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:31:44.157 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:31:44.157 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:31:44.157 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:31:44.157 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:44.157 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 19368 2024-09-14T06:31:44.157 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:44.157 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:31:44.157 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:44.157 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:31:44.157 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065130+0000", 2024-09-14T06:31:44.157 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:44.158 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:44.158 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:44.158 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:31:44.158 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065130+0000", 2024-09-14T06:31:44.158 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:44.158 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:44.158 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:44.158 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:31:44.158 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065130+0000", 2024-09-14T06:31:44.158 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.0800000000000001e-07 2024-09-14T06:31:44.158 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:44.158 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:44.158 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:31:44.158 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065131+0000", 2024-09-14T06:31:44.159 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.8700000000000004e-07 2024-09-14T06:31:44.159 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:44.159 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:44.159 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:31:44.159 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065134+0000", 2024-09-14T06:31:44.159 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.1039999999999998e-06 2024-09-14T06:31:44.159 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:44.159 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:44.159 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:31:44.160 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065136+0000", 2024-09-14T06:31:44.160 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.3949999999999999e-06 2024-09-14T06:31:44.160 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:44.160 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:44.160 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:31:44.160 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.111541+0000", 2024-09-14T06:31:44.160 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046404206000000003 2024-09-14T06:31:44.160 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:44.160 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:44.160 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:31:44.160 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.111582+0000", 2024-09-14T06:31:44.160 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.1051999999999999e-05 2024-09-14T06:31:44.160 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:44.161 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:44.161 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:31:44.161 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.111637+0000", 2024-09-14T06:31:44.161 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.5322999999999999e-05 2024-09-14T06:31:44.161 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:44.161 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:31:44.161 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:44.161 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:44.161 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:44.161 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:19369 3.3 3:c34e5e51:::rbd_data.112d92027590.0000000000000024:f [read 1343488~1024 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:31:44.161 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:27.065139+0000", 2024-09-14T06:31:44.161 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 137.021486694, 2024-09-14T06:31:44.161 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046682169000000003, 2024-09-14T06:31:44.161 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:31:44.162 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:31:44.162 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:31:44.162 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:31:44.162 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:31:44.162 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:44.162 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 19369 2024-09-14T06:31:44.162 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:44.162 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:31:44.162 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:44.162 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:31:44.163 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065139+0000", 2024-09-14T06:31:44.163 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:44.163 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:44.163 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:44.163 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:31:44.163 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065139+0000", 2024-09-14T06:31:44.163 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:44.163 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:44.163 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:44.163 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:31:44.163 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065140+0000", 2024-09-14T06:31:44.163 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.8299999999999998e-07 2024-09-14T06:31:44.163 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:44.163 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:44.164 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:31:44.164 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065141+0000", 2024-09-14T06:31:44.164 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.3900000000000003e-07 2024-09-14T06:31:44.164 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:44.164 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:44.164 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:31:44.164 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065144+0000", 2024-09-14T06:31:44.164 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.7409999999999998e-06 2024-09-14T06:31:44.164 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:44.164 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:44.164 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:31:44.164 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065147+0000", 2024-09-14T06:31:44.164 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.4150000000000002e-06 2024-09-14T06:31:44.164 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:44.165 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:44.165 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:31:44.165 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.111673+0000", 2024-09-14T06:31:44.165 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046526004000000003 2024-09-14T06:31:44.165 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:44.165 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:44.165 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:31:44.165 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.111749+0000", 2024-09-14T06:31:44.165 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.5741999999999994e-05 2024-09-14T06:31:44.165 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:44.165 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:44.165 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:31:44.165 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.111822+0000", 2024-09-14T06:31:44.166 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.3045000000000003e-05 2024-09-14T06:31:44.166 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:44.166 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:31:44.166 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:44.166 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:44.166 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:44.166 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:19370 3.3 3:c34e5e51:::rbd_data.112d92027590.0000000000000024:f [read 1353728~1024 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:31:44.166 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:27.065150+0000", 2024-09-14T06:31:44.166 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 137.02147596500001, 2024-09-14T06:31:44.166 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046897556999999999, 2024-09-14T06:31:44.167 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:31:44.167 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:31:44.168 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:31:44.168 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:31:44.168 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:31:44.168 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:44.168 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 19370 2024-09-14T06:31:44.168 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:44.168 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:31:44.168 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:44.168 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:31:44.168 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065150+0000", 2024-09-14T06:31:44.168 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:44.168 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:44.168 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:44.169 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:31:44.169 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065150+0000", 2024-09-14T06:31:44.169 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:44.169 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:44.169 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:44.169 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:31:44.169 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065150+0000", 2024-09-14T06:31:44.169 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.0200000000000001e-07 2024-09-14T06:31:44.169 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:44.169 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:44.169 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:31:44.169 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065151+0000", 2024-09-14T06:31:44.169 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.1100000000000004e-07 2024-09-14T06:31:44.169 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:44.170 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:44.170 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:31:44.170 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065154+0000", 2024-09-14T06:31:44.170 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.9639999999999999e-06 2024-09-14T06:31:44.170 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:44.170 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:44.170 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:31:44.170 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065157+0000", 2024-09-14T06:31:44.170 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.4260000000000002e-06 2024-09-14T06:31:44.171 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:44.171 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:44.171 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:31:44.171 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.111861+0000", 2024-09-14T06:31:44.171 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046704268 2024-09-14T06:31:44.171 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:44.171 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:44.171 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:31:44.171 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.111939+0000", 2024-09-14T06:31:44.171 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.8231000000000004e-05 2024-09-14T06:31:44.171 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:44.171 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:44.171 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:31:44.171 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.112048+0000", 2024-09-14T06:31:44.172 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00010855500000000001 2024-09-14T06:31:44.172 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:44.172 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:31:44.172 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:44.172 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:44.172 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:44.172 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:19371 3.3 3:c34e5e51:::rbd_data.112d92027590.0000000000000024:f [read 1363968~1024 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:31:44.172 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:27.065161+0000", 2024-09-14T06:31:44.172 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 137.021465315, 2024-09-14T06:31:44.172 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047021944000000003, 2024-09-14T06:31:44.172 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:31:44.172 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:31:44.172 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:31:44.173 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:31:44.173 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:31:44.173 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:44.173 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 19371 2024-09-14T06:31:44.173 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:44.173 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:31:44.173 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:44.173 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:31:44.173 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065161+0000", 2024-09-14T06:31:44.173 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:44.173 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:44.173 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:44.173 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:31:44.173 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065161+0000", 2024-09-14T06:31:44.174 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:44.174 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:44.174 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:44.174 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:31:44.174 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065161+0000", 2024-09-14T06:31:44.174 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.0200000000000001e-07 2024-09-14T06:31:44.174 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:44.174 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:44.174 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:31:44.174 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065166+0000", 2024-09-14T06:31:44.174 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.9869999999999999e-06 2024-09-14T06:31:44.174 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:44.174 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:44.174 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:31:44.175 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065169+0000", 2024-09-14T06:31:44.175 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.3799999999999998e-06 2024-09-14T06:31:44.175 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:44.175 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:44.175 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:31:44.175 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065172+0000", 2024-09-14T06:31:44.175 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.4830000000000002e-06 2024-09-14T06:31:44.175 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:44.175 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:44.175 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:31:44.175 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.112071+0000", 2024-09-14T06:31:44.175 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046899529000000002 2024-09-14T06:31:44.175 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:44.175 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:44.176 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:31:44.176 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.112128+0000", 2024-09-14T06:31:44.176 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.6341999999999997e-05 2024-09-14T06:31:44.176 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:44.176 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:44.176 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:31:44.176 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.112183+0000", 2024-09-14T06:31:44.176 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.5021000000000001e-05 2024-09-14T06:31:44.176 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:44.176 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:31:44.176 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:44.176 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:44.176 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:44.176 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:19372 3.3 3:c34e5e51:::rbd_data.112d92027590.0000000000000024:f [read 1374208~1024 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:31:44.177 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:27.065176+0000", 2024-09-14T06:31:44.177 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 137.021450165, 2024-09-14T06:31:44.177 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047188889999999997, 2024-09-14T06:31:44.177 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:31:44.177 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:31:44.177 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:31:44.177 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:31:44.177 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:31:44.177 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:44.177 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 19372 2024-09-14T06:31:44.177 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:44.177 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:31:44.177 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:44.178 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:31:44.178 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065176+0000", 2024-09-14T06:31:44.178 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:44.178 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:44.178 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:44.178 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:31:44.178 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065176+0000", 2024-09-14T06:31:44.178 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:44.178 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:44.178 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:44.178 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:31:44.178 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.071117+0000", 2024-09-14T06:31:44.178 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.0059404990000000001 2024-09-14T06:31:44.178 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:44.179 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:44.179 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:31:44.179 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.071120+0000", 2024-09-14T06:31:44.179 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.4340000000000001e-06 2024-09-14T06:31:44.179 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:44.179 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:44.179 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:31:44.179 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.071129+0000", 2024-09-14T06:31:44.179 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.8389999999999996e-06 2024-09-14T06:31:44.179 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:44.179 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:44.179 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:31:44.179 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.071134+0000", 2024-09-14T06:31:44.179 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.412e-06 2024-09-14T06:31:44.180 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:44.180 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:44.180 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:31:44.180 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.112221+0000", 2024-09-14T06:31:44.180 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.041086473999999998 2024-09-14T06:31:44.180 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:44.180 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:44.180 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:31:44.180 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.112299+0000", 2024-09-14T06:31:44.180 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.8746999999999997e-05 2024-09-14T06:31:44.180 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:44.180 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:44.180 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:31:44.181 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.112365+0000", 2024-09-14T06:31:44.181 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.5485000000000001e-05 2024-09-14T06:31:44.181 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:44.181 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:31:44.181 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:44.181 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:44.181 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:31:44.181 INFO:teuthology.orchestra.run.smithi138.stdout:} 2024-09-14T06:31:44.183 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-14T06:31:44.183 INFO:tasks.ceph.osd.7.smithi138.stderr:2024-09-14T06:31:44.181+0000 7f3c6a21d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33623) UID: 0 2024-09-14T06:31:44.283 INFO:tasks.ceph.osd.7.smithi138.stderr:2024-09-14T06:31:44.282+0000 7f3c6a21d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33623) UID: 0 2024-09-14T06:31:44.327 INFO:teuthology.orchestra.run.smithi138.stdout:{ 2024-09-14T06:31:44.328 INFO:teuthology.orchestra.run.smithi138.stdout: "ops": [], 2024-09-14T06:31:44.328 INFO:teuthology.orchestra.run.smithi138.stdout: "num_ops": 0 2024-09-14T06:31:44.328 INFO:teuthology.orchestra.run.smithi138.stdout:} 2024-09-14T06:31:44.336 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-14T06:31:44.384 INFO:tasks.ceph.osd.2.smithi094.stderr:2024-09-14T06:31:44.383+0000 7faf3269b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33649) UID: 0 2024-09-14T06:31:44.479 INFO:teuthology.orchestra.run.smithi138.stdout:{ 2024-09-14T06:31:44.479 INFO:teuthology.orchestra.run.smithi138.stdout: "ops": [], 2024-09-14T06:31:44.479 INFO:teuthology.orchestra.run.smithi138.stdout: "complaint_time": 30, 2024-09-14T06:31:44.479 INFO:teuthology.orchestra.run.smithi138.stdout: "num_blocked_ops": 0 2024-09-14T06:31:44.479 INFO:teuthology.orchestra.run.smithi138.stdout:} 2024-09-14T06:31:44.485 INFO:tasks.ceph.osd.6.smithi138.stderr:2024-09-14T06:31:44.483+0000 7f8cd8849640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33620) UID: 0 2024-09-14T06:31:44.487 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-14T06:31:44.585 INFO:tasks.ceph.osd.4.smithi138.stderr:2024-09-14T06:31:44.584+0000 7f6c653c8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33615) UID: 0 2024-09-14T06:31:44.632 INFO:teuthology.orchestra.run.smithi138.stdout:{ 2024-09-14T06:31:44.633 INFO:teuthology.orchestra.run.smithi138.stdout: "size": 20, 2024-09-14T06:31:44.633 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 600, 2024-09-14T06:31:44.633 INFO:teuthology.orchestra.run.smithi138.stdout: "ops": [ 2024-09-14T06:31:44.633 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:44.633 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:11453 3.1d 3:bcad7303:::rbd_data.112d92027590.0000000000000001:f [read 94208~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:31:44.633 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:25.654375+0000", 2024-09-14T06:31:44.633 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 138.97659514399999, 2024-09-14T06:31:44.633 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052449318000000002, 2024-09-14T06:31:44.633 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:31:44.633 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:31:44.633 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:31:44.634 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:31:44.634 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:31:44.634 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:44.634 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 11453 2024-09-14T06:31:44.634 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:44.634 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:31:44.634 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:44.634 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:31:44.634 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654375+0000", 2024-09-14T06:31:44.634 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:44.634 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:44.634 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:44.634 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:31:44.634 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654375+0000", 2024-09-14T06:31:44.635 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:44.635 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:44.635 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:44.635 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:31:44.635 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654496+0000", 2024-09-14T06:31:44.635 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00012081 2024-09-14T06:31:44.635 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:44.635 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:44.635 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:31:44.635 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654497+0000", 2024-09-14T06:31:44.635 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.8789999999999999e-06 2024-09-14T06:31:44.635 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:44.635 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:44.635 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:31:44.636 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654501+0000", 2024-09-14T06:31:44.636 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.8999999999999999e-06 2024-09-14T06:31:44.636 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:44.636 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:44.636 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:31:44.636 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654505+0000", 2024-09-14T06:31:44.636 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.2229999999999999e-06 2024-09-14T06:31:44.636 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:44.636 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:44.636 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:31:44.636 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.705917+0000", 2024-09-14T06:31:44.636 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.051412501999999999 2024-09-14T06:31:44.636 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:44.637 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:44.637 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:31:44.637 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.705999+0000", 2024-09-14T06:31:44.637 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.1551999999999994e-05 2024-09-14T06:31:44.637 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:44.637 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:44.637 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:31:44.637 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.706824+0000", 2024-09-14T06:31:44.637 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00082545199999999996 2024-09-14T06:31:44.637 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:44.637 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:31:44.637 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:44.637 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:44.638 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:44.638 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:11454 3.1d 3:bcad7303:::rbd_data.112d92027590.0000000000000001:f [read 104448~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:31:44.638 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:25.654510+0000", 2024-09-14T06:31:44.638 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 138.97646035400001, 2024-09-14T06:31:44.638 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052610713000000003, 2024-09-14T06:31:44.638 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:31:44.638 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:31:44.638 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:31:44.638 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:31:44.638 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:31:44.638 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:44.638 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 11454 2024-09-14T06:31:44.638 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:44.638 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:31:44.639 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:44.639 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:31:44.639 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654510+0000", 2024-09-14T06:31:44.639 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:44.639 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:44.639 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:44.639 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:31:44.639 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654510+0000", 2024-09-14T06:31:44.639 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:44.639 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:44.639 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:44.639 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:31:44.639 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654825+0000", 2024-09-14T06:31:44.640 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00031522400000000002 2024-09-14T06:31:44.640 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:44.640 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:44.640 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:31:44.640 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654827+0000", 2024-09-14T06:31:44.640 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.9709999999999998e-06 2024-09-14T06:31:44.640 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:44.640 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:44.640 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:31:44.640 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654832+0000", 2024-09-14T06:31:44.640 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.4099999999999999e-06 2024-09-14T06:31:44.640 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:44.640 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:44.641 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:31:44.641 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654835+0000", 2024-09-14T06:31:44.641 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.14e-06 2024-09-14T06:31:44.641 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:44.641 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:44.641 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:31:44.641 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.706851+0000", 2024-09-14T06:31:44.641 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052015761000000001 2024-09-14T06:31:44.641 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:44.641 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:44.641 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:31:44.641 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.706934+0000", 2024-09-14T06:31:44.641 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.2484999999999994e-05 2024-09-14T06:31:44.641 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:44.642 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:44.642 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:31:44.642 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.707120+0000", 2024-09-14T06:31:44.642 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00018672200000000001 2024-09-14T06:31:44.642 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:44.642 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:31:44.642 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:44.642 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:44.642 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:44.642 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:11455 3.1d 3:bcad7303:::rbd_data.112d92027590.0000000000000001:f [read 114688~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:31:44.642 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:25.654841+0000", 2024-09-14T06:31:44.642 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 138.97612902500001, 2024-09-14T06:31:44.642 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052579898, 2024-09-14T06:31:44.642 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:31:44.643 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:31:44.643 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:31:44.643 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:31:44.643 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:31:44.643 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:44.643 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 11455 2024-09-14T06:31:44.643 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:44.643 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:31:44.643 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:44.643 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:31:44.643 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654841+0000", 2024-09-14T06:31:44.643 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:44.643 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:44.644 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:44.644 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:31:44.644 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654841+0000", 2024-09-14T06:31:44.644 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:44.644 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:44.644 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:44.644 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:31:44.644 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654841+0000", 2024-09-14T06:31:44.644 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.7300000000000002e-07 2024-09-14T06:31:44.644 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:44.644 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:44.644 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:31:44.644 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654845+0000", 2024-09-14T06:31:44.644 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.0520000000000003e-06 2024-09-14T06:31:44.645 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:44.645 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:44.645 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:31:44.645 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654849+0000", 2024-09-14T06:31:44.645 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.0459999999999999e-06 2024-09-14T06:31:44.645 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:44.645 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:44.645 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:31:44.645 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654852+0000", 2024-09-14T06:31:44.645 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.9830000000000001e-06 2024-09-14T06:31:44.645 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:44.645 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:44.645 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:31:44.645 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.707159+0000", 2024-09-14T06:31:44.646 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052306826000000001 2024-09-14T06:31:44.646 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:44.646 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:44.646 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:31:44.646 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.707242+0000", 2024-09-14T06:31:44.646 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.2595000000000005e-05 2024-09-14T06:31:44.646 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:44.646 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:44.646 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:31:44.646 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.707421+0000", 2024-09-14T06:31:44.646 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.000179123 2024-09-14T06:31:44.646 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:44.646 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:31:44.646 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:44.647 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:44.647 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:44.647 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:11456 3.1d 3:bcad7303:::rbd_data.112d92027590.0000000000000001:f [read 124928~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:31:44.647 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:25.654856+0000", 2024-09-14T06:31:44.647 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 138.97611417900001, 2024-09-14T06:31:44.647 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052801580000000001, 2024-09-14T06:31:44.647 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:31:44.647 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:31:44.647 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:31:44.647 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:31:44.647 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:31:44.647 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:44.647 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 11456 2024-09-14T06:31:44.647 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:44.648 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:31:44.648 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:44.648 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:31:44.648 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654856+0000", 2024-09-14T06:31:44.648 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:44.648 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:44.648 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:44.648 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:31:44.648 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654856+0000", 2024-09-14T06:31:44.648 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:44.648 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:44.648 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:44.648 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:31:44.649 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654856+0000", 2024-09-14T06:31:44.649 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.29e-07 2024-09-14T06:31:44.649 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:44.649 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:44.649 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:31:44.649 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654857+0000", 2024-09-14T06:31:44.649 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.187e-06 2024-09-14T06:31:44.649 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:44.649 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:44.649 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:31:44.649 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654861+0000", 2024-09-14T06:31:44.649 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.3850000000000002e-06 2024-09-14T06:31:44.649 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:44.650 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:44.650 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:31:44.650 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654863+0000", 2024-09-14T06:31:44.650 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.9500000000000001e-06 2024-09-14T06:31:44.650 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:44.650 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:44.650 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:31:44.650 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.707456+0000", 2024-09-14T06:31:44.650 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052592434 2024-09-14T06:31:44.650 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:44.650 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:44.650 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:31:44.650 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.707516+0000", 2024-09-14T06:31:44.650 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.0167999999999997e-05 2024-09-14T06:31:44.651 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:44.651 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:44.651 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:31:44.651 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.707657+0000", 2024-09-14T06:31:44.651 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00014122700000000001 2024-09-14T06:31:44.651 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:44.651 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:31:44.651 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:44.651 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:44.651 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:44.651 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:11457 3.1d 3:bcad7303:::rbd_data.112d92027590.0000000000000001:f [read 135168~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:31:44.651 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:25.654867+0000", 2024-09-14T06:31:44.651 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 138.97610329400001, 2024-09-14T06:31:44.652 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052921902, 2024-09-14T06:31:44.652 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:31:44.652 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:31:44.652 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:31:44.652 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:31:44.652 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:31:44.652 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:44.652 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 11457 2024-09-14T06:31:44.652 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:44.652 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:31:44.652 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:44.652 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:31:44.652 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654867+0000", 2024-09-14T06:31:44.653 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:44.653 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:44.653 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:44.653 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:31:44.653 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654867+0000", 2024-09-14T06:31:44.653 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:44.653 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:44.653 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:44.653 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:31:44.653 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654972+0000", 2024-09-14T06:31:44.653 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00010584 2024-09-14T06:31:44.653 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:44.653 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:44.654 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:31:44.654 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654975+0000", 2024-09-14T06:31:44.654 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.1169999999999998e-06 2024-09-14T06:31:44.654 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:44.654 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:44.654 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:31:44.654 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654979+0000", 2024-09-14T06:31:44.654 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.1960000000000001e-06 2024-09-14T06:31:44.654 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:44.654 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:44.654 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:31:44.654 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654982+0000", 2024-09-14T06:31:44.654 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.2990000000000001e-06 2024-09-14T06:31:44.654 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:44.655 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:44.655 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:31:44.655 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.707684+0000", 2024-09-14T06:31:44.655 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052701979000000003 2024-09-14T06:31:44.655 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:44.655 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:44.655 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:31:44.655 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.707728+0000", 2024-09-14T06:31:44.655 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.3529999999999998e-05 2024-09-14T06:31:44.655 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:44.655 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:44.655 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:31:44.655 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.707789+0000", 2024-09-14T06:31:44.655 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.0940999999999998e-05 2024-09-14T06:31:44.656 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:44.656 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:31:44.656 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:44.656 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:44.656 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:44.656 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:11458 3.1d 3:bcad7303:::rbd_data.112d92027590.0000000000000001:f [read 145408~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:31:44.656 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:25.654987+0000", 2024-09-14T06:31:44.656 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 138.975982896, 2024-09-14T06:31:44.656 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.05291655, 2024-09-14T06:31:44.656 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:31:44.656 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:31:44.656 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:31:44.656 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:31:44.657 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:31:44.657 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:44.657 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 11458 2024-09-14T06:31:44.657 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:44.657 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:31:44.657 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:44.657 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:31:44.657 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654987+0000", 2024-09-14T06:31:44.657 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:44.657 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:44.657 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:44.657 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:31:44.657 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654987+0000", 2024-09-14T06:31:44.657 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:44.658 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:44.658 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:44.658 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:31:44.658 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655077+0000", 2024-09-14T06:31:44.658 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.0253000000000005e-05 2024-09-14T06:31:44.658 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:44.658 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:44.658 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:31:44.658 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655079+0000", 2024-09-14T06:31:44.658 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.9989999999999998e-06 2024-09-14T06:31:44.658 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:44.658 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:44.658 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:31:44.659 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655083+0000", 2024-09-14T06:31:44.659 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.9280000000000003e-06 2024-09-14T06:31:44.659 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:44.659 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:44.659 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:31:44.659 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655086+0000", 2024-09-14T06:31:44.659 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.056e-06 2024-09-14T06:31:44.659 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:44.659 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:44.659 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:31:44.659 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.707805+0000", 2024-09-14T06:31:44.659 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052718627999999997 2024-09-14T06:31:44.659 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:44.659 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:44.660 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:31:44.660 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.707848+0000", 2024-09-14T06:31:44.660 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.2728999999999997e-05 2024-09-14T06:31:44.660 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:44.660 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:44.660 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:31:44.660 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.707904+0000", 2024-09-14T06:31:44.660 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.5957e-05 2024-09-14T06:31:44.660 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:44.660 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:31:44.660 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:44.660 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:44.660 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:44.661 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:11459 3.1d 3:bcad7303:::rbd_data.112d92027590.0000000000000001:f [read 155648~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:31:44.661 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:25.655091+0000", 2024-09-14T06:31:44.661 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 138.97587888800001, 2024-09-14T06:31:44.661 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052944242000000002, 2024-09-14T06:31:44.661 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:31:44.661 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:31:44.661 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:31:44.661 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:31:44.661 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:31:44.661 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:44.661 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 11459 2024-09-14T06:31:44.661 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:44.661 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:31:44.662 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:44.662 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:31:44.662 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655091+0000", 2024-09-14T06:31:44.662 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:44.662 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:44.662 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:44.662 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:31:44.662 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655091+0000", 2024-09-14T06:31:44.662 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:44.662 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:44.662 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:44.662 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:31:44.662 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655237+0000", 2024-09-14T06:31:44.662 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00014570100000000001 2024-09-14T06:31:44.662 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:44.663 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:44.663 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:31:44.663 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655239+0000", 2024-09-14T06:31:44.663 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.8670000000000001e-06 2024-09-14T06:31:44.663 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:44.663 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:44.663 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:31:44.663 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655243+0000", 2024-09-14T06:31:44.663 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.9829999999999998e-06 2024-09-14T06:31:44.663 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:44.663 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:44.663 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:31:44.663 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655246+0000", 2024-09-14T06:31:44.663 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.1540000000000002e-06 2024-09-14T06:31:44.664 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:44.664 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:44.664 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:31:44.664 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.707934+0000", 2024-09-14T06:31:44.664 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052688286000000001 2024-09-14T06:31:44.664 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:44.664 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:44.664 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:31:44.664 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.707977+0000", 2024-09-14T06:31:44.664 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.3291999999999998e-05 2024-09-14T06:31:44.664 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:44.664 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:44.664 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:31:44.664 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.708035+0000", 2024-09-14T06:31:44.665 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.7958999999999999e-05 2024-09-14T06:31:44.665 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:44.665 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:31:44.665 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:44.665 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:44.665 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:44.665 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:11460 3.1d 3:bcad7303:::rbd_data.112d92027590.0000000000000001:f [read 165888~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:31:44.665 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:25.655250+0000", 2024-09-14T06:31:44.665 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 138.975719591, 2024-09-14T06:31:44.665 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052910698999999999, 2024-09-14T06:31:44.665 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:31:44.665 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:31:44.665 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:31:44.665 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:31:44.665 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:31:44.666 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:44.666 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 11460 2024-09-14T06:31:44.666 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:44.666 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:31:44.666 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:44.666 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:31:44.666 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655250+0000", 2024-09-14T06:31:44.666 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:44.666 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:44.666 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:44.666 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:31:44.666 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655250+0000", 2024-09-14T06:31:44.666 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:44.666 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:44.667 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:44.667 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:31:44.667 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655356+0000", 2024-09-14T06:31:44.667 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00010535900000000001 2024-09-14T06:31:44.667 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:44.667 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:44.667 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:31:44.667 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655357+0000", 2024-09-14T06:31:44.667 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.8080000000000001e-06 2024-09-14T06:31:44.667 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:44.667 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:44.667 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:31:44.667 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655361+0000", 2024-09-14T06:31:44.668 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.861e-06 2024-09-14T06:31:44.668 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:44.668 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:44.668 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:31:44.668 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655364+0000", 2024-09-14T06:31:44.668 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.1089999999999998e-06 2024-09-14T06:31:44.668 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:44.668 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:44.668 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:31:44.668 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.708065+0000", 2024-09-14T06:31:44.668 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052700595000000003 2024-09-14T06:31:44.668 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:44.668 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:44.668 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:31:44.669 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.708108+0000", 2024-09-14T06:31:44.669 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.2484999999999997e-05 2024-09-14T06:31:44.669 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:44.669 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:44.669 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:31:44.669 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.708161+0000", 2024-09-14T06:31:44.669 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.3482000000000001e-05 2024-09-14T06:31:44.669 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:44.669 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:31:44.669 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:44.669 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:44.669 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:44.669 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:11461 3.1d 3:bcad7303:::rbd_data.112d92027590.0000000000000001:f [read 176128~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:31:44.670 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:25.655370+0000", 2024-09-14T06:31:44.670 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 138.97560030899999, 2024-09-14T06:31:44.670 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052915141999999998, 2024-09-14T06:31:44.670 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:31:44.670 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:31:44.670 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:31:44.670 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:31:44.670 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:31:44.670 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:44.670 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 11461 2024-09-14T06:31:44.670 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:44.670 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:31:44.670 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:44.670 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:31:44.670 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655370+0000", 2024-09-14T06:31:44.671 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:44.671 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:44.671 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:44.671 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:31:44.671 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655370+0000", 2024-09-14T06:31:44.671 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:44.671 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:44.671 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:44.671 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:31:44.671 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655394+0000", 2024-09-14T06:31:44.671 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.4553999999999999e-05 2024-09-14T06:31:44.671 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:44.671 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:44.671 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:31:44.672 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655396+0000", 2024-09-14T06:31:44.672 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.8950000000000001e-06 2024-09-14T06:31:44.672 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:44.672 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:44.672 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:31:44.672 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655401+0000", 2024-09-14T06:31:44.672 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.0089999999999998e-06 2024-09-14T06:31:44.672 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:44.672 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:44.672 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:31:44.672 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655404+0000", 2024-09-14T06:31:44.672 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.3400000000000002e-06 2024-09-14T06:31:44.673 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:44.673 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:44.673 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:31:44.673 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.708186+0000", 2024-09-14T06:31:44.673 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052781726000000001 2024-09-14T06:31:44.673 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:44.673 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:44.673 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:31:44.673 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.708229+0000", 2024-09-14T06:31:44.673 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.2670999999999999e-05 2024-09-14T06:31:44.673 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:44.673 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:44.673 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:31:44.673 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.708285+0000", 2024-09-14T06:31:44.673 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.5946999999999998e-05 2024-09-14T06:31:44.674 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:44.674 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:31:44.674 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:44.674 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:44.674 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:44.674 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:11462 3.1d 3:bcad7303:::rbd_data.112d92027590.0000000000000001:f [read 186368~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:31:44.674 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:25.655410+0000", 2024-09-14T06:31:44.674 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 138.975559831, 2024-09-14T06:31:44.674 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052995258000000003, 2024-09-14T06:31:44.674 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:31:44.674 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:31:44.674 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:31:44.674 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:31:44.674 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:31:44.674 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:44.675 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 11462 2024-09-14T06:31:44.675 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:44.675 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:31:44.675 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:44.675 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:31:44.675 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655410+0000", 2024-09-14T06:31:44.675 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:44.675 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:44.675 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:44.675 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:31:44.675 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655410+0000", 2024-09-14T06:31:44.675 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:44.675 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:44.675 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:44.676 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:31:44.676 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655625+0000", 2024-09-14T06:31:44.676 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00021442600000000001 2024-09-14T06:31:44.676 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:44.676 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:44.676 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:31:44.676 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655626+0000", 2024-09-14T06:31:44.676 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.815e-06 2024-09-14T06:31:44.676 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:44.676 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:44.676 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:31:44.676 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655631+0000", 2024-09-14T06:31:44.676 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.651e-06 2024-09-14T06:31:44.677 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:44.677 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:44.677 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:31:44.677 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655634+0000", 2024-09-14T06:31:44.677 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.2449999999999998e-06 2024-09-14T06:31:44.677 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:44.677 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:44.677 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:31:44.677 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.708312+0000", 2024-09-14T06:31:44.677 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052677412 2024-09-14T06:31:44.677 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:44.677 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:44.677 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:31:44.678 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.708352+0000", 2024-09-14T06:31:44.678 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.0729999999999998e-05 2024-09-14T06:31:44.678 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:44.678 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:44.678 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:31:44.678 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.708405+0000", 2024-09-14T06:31:44.678 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.2979000000000002e-05 2024-09-14T06:31:44.678 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:44.678 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:31:44.678 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:44.678 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:44.678 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:44.678 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:11463 3.1d 3:bcad7303:::rbd_data.112d92027590.0000000000000001:f [read 196608~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:31:44.679 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:25.655639+0000", 2024-09-14T06:31:44.679 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 138.97533068499999, 2024-09-14T06:31:44.679 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052888859000000003, 2024-09-14T06:31:44.679 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:31:44.679 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:31:44.679 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:31:44.679 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:31:44.679 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:31:44.679 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:44.679 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 11463 2024-09-14T06:31:44.679 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:44.679 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:31:44.679 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:44.679 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:31:44.679 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655639+0000", 2024-09-14T06:31:44.680 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:44.680 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:44.680 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:44.680 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:31:44.680 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655639+0000", 2024-09-14T06:31:44.680 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:44.680 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:44.680 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:44.680 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:31:44.680 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655640+0000", 2024-09-14T06:31:44.680 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.08e-07 2024-09-14T06:31:44.680 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:44.680 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:44.680 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:31:44.681 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655641+0000", 2024-09-14T06:31:44.681 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.156e-06 2024-09-14T06:31:44.681 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:44.681 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:44.681 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:31:44.681 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655648+0000", 2024-09-14T06:31:44.681 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.8460000000000002e-06 2024-09-14T06:31:44.681 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:44.681 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:44.681 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:31:44.681 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655651+0000", 2024-09-14T06:31:44.681 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.0079999999999998e-06 2024-09-14T06:31:44.681 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:44.682 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:44.682 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:31:44.682 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.708429+0000", 2024-09-14T06:31:44.682 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052778459 2024-09-14T06:31:44.682 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:44.682 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:44.682 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:31:44.682 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.708473+0000", 2024-09-14T06:31:44.682 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.3782999999999997e-05 2024-09-14T06:31:44.682 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:44.682 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:44.682 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:31:44.682 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.708528+0000", 2024-09-14T06:31:44.683 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.5198999999999998e-05 2024-09-14T06:31:44.683 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:44.683 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:31:44.683 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:44.683 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:44.683 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:44.683 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:11464 3.1d 3:bcad7303:::rbd_data.112d92027590.0000000000000001:f [read 206848~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:31:44.683 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:25.655654+0000", 2024-09-14T06:31:44.683 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 138.97531581600001, 2024-09-14T06:31:44.683 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052981845999999999, 2024-09-14T06:31:44.683 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:31:44.683 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:31:44.683 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:31:44.683 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:31:44.684 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:31:44.684 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:44.684 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 11464 2024-09-14T06:31:44.684 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:44.684 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:31:44.684 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:44.684 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:31:44.684 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655654+0000", 2024-09-14T06:31:44.684 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:44.684 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:44.684 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:44.684 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:31:44.684 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655654+0000", 2024-09-14T06:31:44.685 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:44.685 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:44.685 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:44.685 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:31:44.685 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655946+0000", 2024-09-14T06:31:44.685 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00029148200000000003 2024-09-14T06:31:44.685 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:44.685 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:44.685 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:31:44.685 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655948+0000", 2024-09-14T06:31:44.685 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.973e-06 2024-09-14T06:31:44.685 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:44.685 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:44.685 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:31:44.685 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655953+0000", 2024-09-14T06:31:44.686 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.1240000000000004e-06 2024-09-14T06:31:44.686 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:44.686 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:44.686 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:31:44.686 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655956+0000", 2024-09-14T06:31:44.686 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.315e-06 2024-09-14T06:31:44.686 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:44.686 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:44.686 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:31:44.686 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.708543+0000", 2024-09-14T06:31:44.686 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052587416999999997 2024-09-14T06:31:44.686 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:44.686 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:44.687 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:31:44.687 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.708584+0000", 2024-09-14T06:31:44.687 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.1004000000000002e-05 2024-09-14T06:31:44.687 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:44.687 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:44.687 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:31:44.687 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.708636+0000", 2024-09-14T06:31:44.687 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.1530999999999998e-05 2024-09-14T06:31:44.687 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:44.687 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:31:44.687 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:44.687 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:44.687 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:44.687 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:11465 3.1d 3:bcad7303:::rbd_data.112d92027590.0000000000000001:f [read 217088~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:31:44.688 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:25.655961+0000", 2024-09-14T06:31:44.688 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 138.975008664, 2024-09-14T06:31:44.688 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052798682999999999, 2024-09-14T06:31:44.688 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:31:44.688 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:31:44.688 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:31:44.688 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:31:44.688 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:31:44.688 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:44.688 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 11465 2024-09-14T06:31:44.688 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:44.688 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:31:44.688 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:44.689 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:31:44.689 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655961+0000", 2024-09-14T06:31:44.689 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:44.689 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:44.689 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:44.689 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:31:44.689 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655961+0000", 2024-09-14T06:31:44.689 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:44.689 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:44.689 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:44.689 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:31:44.689 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.656114+0000", 2024-09-14T06:31:44.689 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.000152745 2024-09-14T06:31:44.689 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:44.690 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:44.690 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:31:44.690 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.656116+0000", 2024-09-14T06:31:44.690 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.804e-06 2024-09-14T06:31:44.690 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:44.690 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:44.690 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:31:44.690 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.656120+0000", 2024-09-14T06:31:44.690 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.4700000000000004e-06 2024-09-14T06:31:44.690 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:44.690 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:44.690 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:31:44.690 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.656123+0000", 2024-09-14T06:31:44.690 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.1269999999999999e-06 2024-09-14T06:31:44.691 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:44.691 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:44.691 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:31:44.691 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.708660+0000", 2024-09-14T06:31:44.691 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052536919000000001 2024-09-14T06:31:44.691 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:44.691 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:44.691 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:31:44.691 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.708703+0000", 2024-09-14T06:31:44.691 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.3077000000000001e-05 2024-09-14T06:31:44.691 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:44.691 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:44.691 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:31:44.691 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.708760+0000", 2024-09-14T06:31:44.692 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.6541e-05 2024-09-14T06:31:44.692 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:44.692 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:31:44.692 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:44.692 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:44.692 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:44.692 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:11466 3.1d 3:bcad7303:::rbd_data.112d92027590.0000000000000001:f [read 227328~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:31:44.692 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:25.656129+0000", 2024-09-14T06:31:44.692 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 138.97484127600001, 2024-09-14T06:31:44.692 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052750987999999999, 2024-09-14T06:31:44.692 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:31:44.692 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:31:44.693 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:31:44.693 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:31:44.693 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:31:44.693 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:44.693 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 11466 2024-09-14T06:31:44.693 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:44.693 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:31:44.693 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:44.693 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:31:44.693 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.656129+0000", 2024-09-14T06:31:44.693 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:44.693 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:44.693 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:44.694 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:31:44.694 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.656129+0000", 2024-09-14T06:31:44.694 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:44.694 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:44.694 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:44.694 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:31:44.694 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.656347+0000", 2024-09-14T06:31:44.694 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.000218054 2024-09-14T06:31:44.694 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:44.694 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:44.694 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:31:44.694 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.656349+0000", 2024-09-14T06:31:44.694 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.9199999999999998e-06 2024-09-14T06:31:44.695 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:44.695 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:44.695 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:31:44.695 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.656353+0000", 2024-09-14T06:31:44.695 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.9210000000000002e-06 2024-09-14T06:31:44.695 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:44.695 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:44.695 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:31:44.695 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.656356+0000", 2024-09-14T06:31:44.695 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.7670000000000001e-06 2024-09-14T06:31:44.695 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:44.695 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:44.695 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:31:44.696 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.708787+0000", 2024-09-14T06:31:44.696 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052430840999999999 2024-09-14T06:31:44.696 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:44.696 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:44.696 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:31:44.696 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.708827+0000", 2024-09-14T06:31:44.696 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.9538e-05 2024-09-14T06:31:44.696 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:44.696 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:44.696 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:31:44.696 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.708880+0000", 2024-09-14T06:31:44.696 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.2947e-05 2024-09-14T06:31:44.696 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:44.696 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:31:44.697 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:44.697 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:44.697 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:44.697 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:11467 3.1d 3:bcad7303:::rbd_data.112d92027590.0000000000000001:f [read 237568~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:31:44.697 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:25.656361+0000", 2024-09-14T06:31:44.697 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 138.974608628, 2024-09-14T06:31:44.697 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052643898000000001, 2024-09-14T06:31:44.697 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:31:44.697 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:31:44.697 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:31:44.697 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:31:44.697 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:31:44.697 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:44.697 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 11467 2024-09-14T06:31:44.698 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:44.698 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:31:44.698 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:44.698 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:31:44.698 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.656361+0000", 2024-09-14T06:31:44.698 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:44.698 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:44.698 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:44.698 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:31:44.698 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.656361+0000", 2024-09-14T06:31:44.698 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:44.698 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:44.698 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:44.698 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:31:44.699 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.656855+0000", 2024-09-14T06:31:44.699 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00049372000000000001 2024-09-14T06:31:44.699 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:44.699 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:44.699 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:31:44.699 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.656859+0000", 2024-09-14T06:31:44.699 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.1300000000000003e-06 2024-09-14T06:31:44.699 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:44.699 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:44.699 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:31:44.699 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.656863+0000", 2024-09-14T06:31:44.699 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.2629999999999997e-06 2024-09-14T06:31:44.699 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:44.699 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:44.700 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:31:44.700 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.656867+0000", 2024-09-14T06:31:44.700 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.3220000000000001e-06 2024-09-14T06:31:44.700 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:44.700 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:44.700 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:31:44.700 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.708903+0000", 2024-09-14T06:31:44.700 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052035855999999998 2024-09-14T06:31:44.700 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:44.700 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:44.700 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:31:44.700 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.708949+0000", 2024-09-14T06:31:44.700 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.6276999999999998e-05 2024-09-14T06:31:44.701 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:44.701 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:44.701 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:31:44.701 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.709005+0000", 2024-09-14T06:31:44.701 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.6329999999999998e-05 2024-09-14T06:31:44.701 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:44.701 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:31:44.701 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:44.701 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:44.701 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:44.701 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:11480 3.1d 3:bcad7303:::rbd_data.112d92027590.0000000000000001:f [read 370688~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:31:44.701 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:25.658563+0000", 2024-09-14T06:31:44.701 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 138.97240686399999, 2024-09-14T06:31:44.702 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052429164, 2024-09-14T06:31:44.702 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:31:44.702 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:31:44.702 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:31:44.702 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:31:44.702 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:31:44.702 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:44.702 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 11480 2024-09-14T06:31:44.702 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:44.702 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:31:44.702 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:44.702 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:31:44.702 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.658563+0000", 2024-09-14T06:31:44.702 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:44.703 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:44.703 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:44.703 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:31:44.703 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.658563+0000", 2024-09-14T06:31:44.703 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:44.703 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:44.703 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:44.703 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:31:44.703 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.658658+0000", 2024-09-14T06:31:44.703 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.5346999999999994e-05 2024-09-14T06:31:44.703 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:44.703 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:44.703 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:31:44.704 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.658662+0000", 2024-09-14T06:31:44.704 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.14e-06 2024-09-14T06:31:44.704 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:44.704 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:44.704 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:31:44.704 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.658669+0000", 2024-09-14T06:31:44.704 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.7379999999999994e-06 2024-09-14T06:31:44.704 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:44.704 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:44.704 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:31:44.704 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.658675+0000", 2024-09-14T06:31:44.704 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.5430000000000001e-06 2024-09-14T06:31:44.704 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:44.705 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:44.705 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:31:44.705 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.710824+0000", 2024-09-14T06:31:44.705 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052148847999999998 2024-09-14T06:31:44.705 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:44.705 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:44.705 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:31:44.705 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.710864+0000", 2024-09-14T06:31:44.705 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.9925999999999997e-05 2024-09-14T06:31:44.705 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:44.705 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:44.705 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:31:44.705 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.710992+0000", 2024-09-14T06:31:44.705 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.000128622 2024-09-14T06:31:44.706 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:44.706 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:31:44.706 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:44.706 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:44.706 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:44.706 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:11481 3.1d 3:bcad7303:::rbd_data.112d92027590.0000000000000001:f [read 380928~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:31:44.706 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:25.658684+0000", 2024-09-14T06:31:44.706 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 138.97228579399999, 2024-09-14T06:31:44.706 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052508565, 2024-09-14T06:31:44.706 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:31:44.706 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:31:44.706 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:31:44.706 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:31:44.707 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:31:44.707 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:44.707 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 11481 2024-09-14T06:31:44.707 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:44.707 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:31:44.707 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:44.707 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:31:44.707 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.658684+0000", 2024-09-14T06:31:44.707 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:44.707 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:44.707 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:44.707 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:31:44.707 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.658684+0000", 2024-09-14T06:31:44.707 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:44.708 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:44.708 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:44.708 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:31:44.708 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.658715+0000", 2024-09-14T06:31:44.708 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.0703000000000001e-05 2024-09-14T06:31:44.708 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:44.708 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:44.708 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:31:44.708 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.658718+0000", 2024-09-14T06:31:44.708 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.0709999999999999e-06 2024-09-14T06:31:44.708 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:44.708 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:44.708 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:31:44.708 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.658725+0000", 2024-09-14T06:31:44.709 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.2339999999999997e-06 2024-09-14T06:31:44.709 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:44.709 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:44.709 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:31:44.709 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.658731+0000", 2024-09-14T06:31:44.709 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.6450000000000002e-06 2024-09-14T06:31:44.709 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:44.709 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:44.709 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:31:44.709 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.711020+0000", 2024-09-14T06:31:44.709 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052288886 2024-09-14T06:31:44.709 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:44.709 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:44.710 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:31:44.710 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.711061+0000", 2024-09-14T06:31:44.710 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.1084000000000002e-05 2024-09-14T06:31:44.710 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:44.710 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:44.710 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:31:44.710 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.711193+0000", 2024-09-14T06:31:44.710 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00013194199999999999 2024-09-14T06:31:44.710 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:44.710 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:31:44.710 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:44.710 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:44.710 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:44.710 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:11482 3.1d 3:bcad7303:::rbd_data.112d92027590.0000000000000001:f [read 391168~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:31:44.711 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:25.658740+0000", 2024-09-14T06:31:44.711 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 138.97223032900001, 2024-09-14T06:31:44.711 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052637015000000002, 2024-09-14T06:31:44.711 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:31:44.711 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:31:44.711 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:31:44.711 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:31:44.711 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:31:44.711 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:44.711 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 11482 2024-09-14T06:31:44.711 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:44.711 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:31:44.711 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:44.711 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:31:44.712 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.658740+0000", 2024-09-14T06:31:44.712 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:44.712 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:44.712 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:44.712 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:31:44.712 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.658740+0000", 2024-09-14T06:31:44.712 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:44.712 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:44.712 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:44.712 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:31:44.712 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.658880+0000", 2024-09-14T06:31:44.712 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00014036100000000001 2024-09-14T06:31:44.712 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:44.713 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:44.713 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:31:44.713 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.658883+0000", 2024-09-14T06:31:44.713 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.3969999999999998e-06 2024-09-14T06:31:44.713 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:44.713 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:44.713 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:31:44.713 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.658890+0000", 2024-09-14T06:31:44.713 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.1500000000000002e-06 2024-09-14T06:31:44.713 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:44.713 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:44.713 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:31:44.713 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.658897+0000", 2024-09-14T06:31:44.714 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.1260000000000003e-06 2024-09-14T06:31:44.714 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:44.714 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:44.714 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:31:44.714 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.711207+0000", 2024-09-14T06:31:44.714 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052310777000000003 2024-09-14T06:31:44.714 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:44.714 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:44.714 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:31:44.714 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.711249+0000", 2024-09-14T06:31:44.714 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.1786999999999998e-05 2024-09-14T06:31:44.714 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:44.714 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:44.714 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:31:44.715 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.711377+0000", 2024-09-14T06:31:44.715 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.000127417 2024-09-14T06:31:44.715 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:44.715 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:31:44.715 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:44.715 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:44.715 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:44.715 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:11483 3.1d 3:bcad7303:::rbd_data.112d92027590.0000000000000001:f [read 401408~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:31:44.715 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:25.658906+0000", 2024-09-14T06:31:44.715 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 138.972063545, 2024-09-14T06:31:44.715 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052595109000000001, 2024-09-14T06:31:44.715 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:31:44.715 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:31:44.715 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:31:44.716 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:31:44.716 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:31:44.716 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:44.716 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 11483 2024-09-14T06:31:44.716 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:44.716 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:31:44.716 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:44.716 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:31:44.716 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.658906+0000", 2024-09-14T06:31:44.716 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:44.716 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:44.716 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:44.716 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:31:44.717 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.658906+0000", 2024-09-14T06:31:44.717 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:44.717 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:44.717 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:44.717 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:31:44.717 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.664985+0000", 2024-09-14T06:31:44.717 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.0060787250000000001 2024-09-14T06:31:44.717 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:44.717 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:44.717 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:31:44.717 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.664991+0000", 2024-09-14T06:31:44.717 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.7030000000000003e-06 2024-09-14T06:31:44.717 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:44.717 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:44.718 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:31:44.718 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.664999+0000", 2024-09-14T06:31:44.718 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.4430000000000003e-06 2024-09-14T06:31:44.718 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:44.718 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:44.718 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:31:44.718 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.665006+0000", 2024-09-14T06:31:44.718 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.0840000000000003e-06 2024-09-14T06:31:44.718 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:44.718 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:44.719 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:31:44.719 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.711402+0000", 2024-09-14T06:31:44.719 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046395714999999997 2024-09-14T06:31:44.719 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:44.719 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:44.719 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:31:44.719 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.711443+0000", 2024-09-14T06:31:44.719 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.0911999999999997e-05 2024-09-14T06:31:44.719 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:44.719 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:44.719 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:31:44.719 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.711501+0000", 2024-09-14T06:31:44.719 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.8526999999999997e-05 2024-09-14T06:31:44.719 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:44.720 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:31:44.720 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:44.720 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:44.720 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:44.720 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:214569 3.2 3:479633bd:::rbd_object_map.112d3bd84cb0:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e170)", 2024-09-14T06:31:44.720 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:31:34.933901+0000", 2024-09-14T06:31:44.720 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 9.6970691609999999, 2024-09-14T06:31:44.720 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.68157511400000004, 2024-09-14T06:31:44.720 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:31:44.720 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:31:44.720 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:31:44.720 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:31:44.720 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:31:44.720 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:44.721 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 214569 2024-09-14T06:31:44.721 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:44.721 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:31:44.721 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:44.721 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:31:44.721 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:31:34.933901+0000", 2024-09-14T06:31:44.721 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:44.721 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:44.721 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:44.721 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:31:44.721 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:31:34.933901+0000", 2024-09-14T06:31:44.721 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:44.721 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:44.722 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:44.722 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:31:44.722 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:31:34.933902+0000", 2024-09-14T06:31:44.722 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.7439999999999999e-06 2024-09-14T06:31:44.722 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:44.722 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:44.722 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:31:44.722 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:31:34.933910+0000", 2024-09-14T06:31:44.722 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.8569999999999995e-06 2024-09-14T06:31:44.722 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:44.722 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:44.722 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:31:44.722 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:31:34.933932+0000", 2024-09-14T06:31:44.723 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.1997999999999998e-05 2024-09-14T06:31:44.723 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:44.723 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:44.723 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:31:44.723 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:31:34.933941+0000", 2024-09-14T06:31:44.723 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.7180000000000002e-06 2024-09-14T06:31:44.723 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:44.723 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:44.723 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:31:44.723 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:31:34.934013+0000", 2024-09-14T06:31:44.723 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.1761000000000006e-05 2024-09-14T06:31:44.723 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:44.723 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:44.723 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "waiting for peered", 2024-09-14T06:31:44.724 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:31:34.934022+0000", 2024-09-14T06:31:44.724 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.8740000000000001e-06 2024-09-14T06:31:44.724 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:44.724 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:44.724 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:31:44.724 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:31:35.608803+0000", 2024-09-14T06:31:44.724 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.67478164299999999 2024-09-14T06:31:44.724 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:44.724 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:44.724 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:31:44.724 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:31:35.608964+0000", 2024-09-14T06:31:44.724 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00016112900000000001 2024-09-14T06:31:44.724 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:44.724 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:44.725 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "waiting for subops from 6", 2024-09-14T06:31:44.725 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:31:35.609399+0000", 2024-09-14T06:31:44.725 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00043454000000000002 2024-09-14T06:31:44.725 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:44.725 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:44.725 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "op_commit", 2024-09-14T06:31:44.725 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:31:35.610495+0000", 2024-09-14T06:31:44.725 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.001096404 2024-09-14T06:31:44.725 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:44.725 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:44.725 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:31:44.725 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:31:35.615409+0000", 2024-09-14T06:31:44.725 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.0049133199999999997 2024-09-14T06:31:44.725 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:44.726 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:44.726 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "commit_sent", 2024-09-14T06:31:44.726 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:31:35.615444+0000", 2024-09-14T06:31:44.726 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.5457000000000003e-05 2024-09-14T06:31:44.726 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:44.726 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:44.726 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:31:44.726 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:31:35.615476+0000", 2024-09-14T06:31:44.726 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.1668999999999998e-05 2024-09-14T06:31:44.726 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:44.726 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:31:44.726 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:44.726 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:44.727 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:31:44.727 INFO:teuthology.orchestra.run.smithi138.stdout:} 2024-09-14T06:31:44.727 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-14T06:31:44.728 INFO:tasks.ceph.osd.3.smithi094.stderr:2024-09-14T06:31:44.727+0000 7feeeafe8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33655) UID: 0 2024-09-14T06:31:44.729 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-14T06:31:44.829 INFO:tasks.ceph.osd.6.smithi138.stderr:2024-09-14T06:31:44.828+0000 7f8cd8849640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33620) UID: 0 2024-09-14T06:31:44.860 INFO:teuthology.orchestra.run.smithi094.stdout:checking clone #38, image image_client.0-clone38 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent39 2024-09-14T06:31:44.874 INFO:teuthology.orchestra.run.smithi138.stdout:{ 2024-09-14T06:31:44.874 INFO:teuthology.orchestra.run.smithi138.stdout: "ops": [], 2024-09-14T06:31:44.874 INFO:teuthology.orchestra.run.smithi138.stdout: "num_ops": 0 2024-09-14T06:31:44.875 INFO:teuthology.orchestra.run.smithi138.stdout:} 2024-09-14T06:31:44.883 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-14T06:31:44.930 INFO:tasks.ceph.osd.3.smithi094.stderr:2024-09-14T06:31:44.929+0000 7feeeafe8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33655) UID: 0 2024-09-14T06:31:44.959 INFO:teuthology.orchestra.run.smithi094.stdout:5638 trunc from 0xe65527d to 0x38961e1 2024-09-14T06:31:44.990 INFO:teuthology.orchestra.run.smithi094.stdout:5639 read 0x3eaba5 thru 0x3f4a1c (0x9e78 bytes) 2024-09-14T06:31:44.991 INFO:teuthology.orchestra.run.smithi094.stdout:5640 read 0x202afeb thru 0x202ff05 (0x4f1b bytes) 2024-09-14T06:31:44.991 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-14T06:31:44.991 INFO:teuthology.orchestra.run.smithi094.stdout:5642 trunc from 0x38961e1 to 0x3dd0c62 2024-09-14T06:31:44.997 INFO:teuthology.orchestra.run.smithi094.stdout:5643 write 0x7a00cb thru 0x7ab663 (0xb599 bytes) 2024-09-14T06:31:45.005 INFO:teuthology.orchestra.run.smithi094.stdout:5645 write 0x40dcffe thru 0x40e89b5 (0xb9b8 bytes) 2024-09-14T06:31:45.010 INFO:teuthology.orchestra.run.smithi094.stdout:5649 write 0x18c4f59 thru 0x18d180d (0xc8b5 bytes) 2024-09-14T06:31:45.017 INFO:teuthology.orchestra.run.smithi094.stdout:5650 read 0x78ac49 thru 0x7969d2 (0xbd8a bytes) 2024-09-14T06:31:45.017 INFO:teuthology.orchestra.run.smithi094.stdout:5651 read 0x3847e31 thru 0x384c6fa (0x48ca bytes) 2024-09-14T06:31:45.017 INFO:teuthology.orchestra.run.smithi094.stdout:5652 punch from 0x20f5bc to 0x218f9a, (0x99de bytes) 2024-09-14T06:31:45.018 INFO:teuthology.orchestra.run.smithi094.stdout:5653 read 0x37c0c8f thru 0x37c38d6 (0x2c48 bytes) 2024-09-14T06:31:45.026 INFO:teuthology.orchestra.run.smithi094.stdout:5654 write 0x6a84494 thru 0x6a9089a (0xc407 bytes) 2024-09-14T06:31:45.030 INFO:tasks.ceph.osd.7.smithi138.stderr:2024-09-14T06:31:45.029+0000 7f3c6a21d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33623) UID: 0 2024-09-14T06:31:45.031 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-14T06:31:45.031 INFO:teuthology.orchestra.run.smithi094.stdout:5658 read 0x67b619b thru 0x67b73d2 (0x1238 bytes) 2024-09-14T06:31:45.031 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-14T06:31:45.031 INFO:teuthology.orchestra.run.smithi094.stdout:5661 read 0x1743767 thru 0x17465e1 (0x2e7b bytes) 2024-09-14T06:31:45.031 INFO:teuthology.orchestra.run.smithi094.stdout:5663 read 0x5274ca3 thru 0x527ddd7 (0x9135 bytes) 2024-09-14T06:31:45.031 INFO:teuthology.orchestra.run.smithi094.stdout:5664 read 0x1fd226 thru 0x2007a2 (0x357d bytes) 2024-09-14T06:31:45.041 INFO:teuthology.orchestra.run.smithi094.stdout:5665 write 0xac28250 thru 0xac2b568 (0x3319 bytes) 2024-09-14T06:31:45.042 INFO:teuthology.orchestra.run.smithi138.stdout:{ 2024-09-14T06:31:45.042 INFO:teuthology.orchestra.run.smithi138.stdout: "ops": [], 2024-09-14T06:31:45.043 INFO:teuthology.orchestra.run.smithi138.stdout: "complaint_time": 30, 2024-09-14T06:31:45.043 INFO:teuthology.orchestra.run.smithi138.stdout: "num_blocked_ops": 0 2024-09-14T06:31:45.043 INFO:teuthology.orchestra.run.smithi138.stdout:} 2024-09-14T06:31:45.044 INFO:teuthology.orchestra.run.smithi094.stdout:5667 read 0x4b1a122 thru 0x4b1e116 (0x3ff5 bytes) 2024-09-14T06:31:45.044 INFO:teuthology.orchestra.run.smithi094.stdout:5668 write 0x302da6d thru 0x303ba9f (0xe033 bytes) 2024-09-14T06:31:45.048 INFO:teuthology.orchestra.run.smithi094.stdout:5669 read 0x2656df1 thru 0x2661b7a (0xad8a bytes) 2024-09-14T06:31:45.048 INFO:teuthology.orchestra.run.smithi094.stdout:5671 read 0x49c706b thru 0x49d4235 (0xd1cb bytes) 2024-09-14T06:31:45.049 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-14T06:31:45.049 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-14T06:31:45.049 INFO:teuthology.orchestra.run.smithi094.stdout:5675 read 0x50ed96 thru 0x51d6ce (0xe939 bytes) 2024-09-14T06:31:45.049 INFO:teuthology.orchestra.run.smithi094.stdout:5676 read 0x7a4389d thru 0x7a4ba22 (0x8186 bytes) 2024-09-14T06:31:45.049 INFO:teuthology.orchestra.run.smithi094.stdout:5677 read 0x5c12038 thru 0x5c206d1 (0xe69a bytes) 2024-09-14T06:31:45.050 INFO:teuthology.orchestra.run.smithi094.stdout:5678 read 0xa789227 thru 0xa79557c (0xc356 bytes) 2024-09-14T06:31:45.050 INFO:teuthology.orchestra.run.smithi094.stdout:5679 clone 41 order 19 su 65536 sc 6 2024-09-14T06:31:45.053 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-14T06:31:45.138 INFO:tasks.ceph.osd.7.smithi138.stderr:2024-09-14T06:31:45.137+0000 7f3c6a21d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33623) UID: 0 2024-09-14T06:31:45.198 INFO:teuthology.orchestra.run.smithi138.stdout:{ 2024-09-14T06:31:45.198 INFO:teuthology.orchestra.run.smithi138.stdout: "size": 20, 2024-09-14T06:31:45.198 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 600, 2024-09-14T06:31:45.199 INFO:teuthology.orchestra.run.smithi138.stdout: "ops": [ 2024-09-14T06:31:45.199 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:45.199 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:98505 3.10 3.1bf56f90 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T06:31:45.199 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:30:07.045453+0000", 2024-09-14T06:31:45.199 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 98.151485800000003, 2024-09-14T06:31:45.199 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032144440000000003, 2024-09-14T06:31:45.199 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:31:45.199 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:31:45.199 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "reached pg", 2024-09-14T06:31:45.199 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:31:45.199 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:31:45.199 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:45.199 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 98505 2024-09-14T06:31:45.199 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:45.200 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:31:45.200 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:45.200 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:31:45.200 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045453+0000", 2024-09-14T06:31:45.200 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:45.200 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:45.200 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:45.200 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:31:45.200 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045453+0000", 2024-09-14T06:31:45.200 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:45.200 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:45.200 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:45.200 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:31:45.200 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045453+0000", 2024-09-14T06:31:45.200 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.36e-07 2024-09-14T06:31:45.201 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:45.201 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:45.201 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:31:45.201 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045454+0000", 2024-09-14T06:31:45.201 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.1299999999999998e-07 2024-09-14T06:31:45.201 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:45.201 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:45.201 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:31:45.201 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045457+0000", 2024-09-14T06:31:45.201 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.8559999999999998e-06 2024-09-14T06:31:45.201 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:45.201 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:45.201 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:31:45.201 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045460+0000", 2024-09-14T06:31:45.202 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.8750000000000002e-06 2024-09-14T06:31:45.202 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:45.202 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:45.202 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:31:45.202 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.077592+0000", 2024-09-14T06:31:45.202 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032131727999999998 2024-09-14T06:31:45.202 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:45.202 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:45.202 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:31:45.202 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.077597+0000", 2024-09-14T06:31:45.202 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.8319999999999996e-06 2024-09-14T06:31:45.203 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:45.203 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:31:45.203 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:45.203 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:45.203 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:45.203 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:98506 3.10 3.1bf56f90 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T06:31:45.203 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:30:07.045481+0000", 2024-09-14T06:31:45.203 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 98.151457545, 2024-09-14T06:31:45.203 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032142759999999999, 2024-09-14T06:31:45.203 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:31:45.203 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:31:45.203 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "reached pg", 2024-09-14T06:31:45.203 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:31:45.203 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:31:45.203 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:45.204 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 98506 2024-09-14T06:31:45.204 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:45.204 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:31:45.204 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:45.204 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:31:45.204 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045481+0000", 2024-09-14T06:31:45.204 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:45.204 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:45.204 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:45.204 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:31:45.204 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045481+0000", 2024-09-14T06:31:45.204 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:45.204 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:45.204 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:45.204 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:31:45.205 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045481+0000", 2024-09-14T06:31:45.205 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.8500000000000001e-07 2024-09-14T06:31:45.205 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:45.205 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:45.205 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:31:45.205 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045483+0000", 2024-09-14T06:31:45.205 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.8250000000000001e-06 2024-09-14T06:31:45.205 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:45.205 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:45.205 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:31:45.205 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045489+0000", 2024-09-14T06:31:45.205 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.5740000000000003e-06 2024-09-14T06:31:45.205 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:45.205 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:45.205 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:31:45.206 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045495+0000", 2024-09-14T06:31:45.206 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.7690000000000002e-06 2024-09-14T06:31:45.206 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:45.206 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:45.206 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:31:45.206 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.077619+0000", 2024-09-14T06:31:45.206 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032124621999999999 2024-09-14T06:31:45.206 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:45.206 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:45.206 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:31:45.206 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.077624+0000", 2024-09-14T06:31:45.206 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.3850000000000004e-06 2024-09-14T06:31:45.206 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:45.207 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:31:45.207 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:45.207 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:45.207 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:45.207 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:98507 3.10 3.1bf56f90 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T06:31:45.207 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:30:07.045502+0000", 2024-09-14T06:31:45.207 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 98.151436124, 2024-09-14T06:31:45.207 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032147219999999997, 2024-09-14T06:31:45.207 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:31:45.207 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:31:45.207 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "reached pg", 2024-09-14T06:31:45.207 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:31:45.207 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:31:45.207 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:45.208 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 98507 2024-09-14T06:31:45.208 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:45.208 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:31:45.208 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:45.208 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:31:45.208 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045502+0000", 2024-09-14T06:31:45.208 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:45.208 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:45.208 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:45.208 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:31:45.208 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045502+0000", 2024-09-14T06:31:45.208 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:45.208 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:45.209 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:45.209 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:31:45.209 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045503+0000", 2024-09-14T06:31:45.209 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.4700000000000002e-07 2024-09-14T06:31:45.209 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:45.209 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:45.209 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:31:45.209 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045509+0000", 2024-09-14T06:31:45.209 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.7329999999999997e-06 2024-09-14T06:31:45.209 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:45.209 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:45.209 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:31:45.209 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045515+0000", 2024-09-14T06:31:45.209 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.8030000000000002e-06 2024-09-14T06:31:45.209 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:45.209 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:45.210 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:31:45.210 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045522+0000", 2024-09-14T06:31:45.210 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.0369999999999997e-06 2024-09-14T06:31:45.210 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:45.210 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:45.210 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:31:45.210 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.077641+0000", 2024-09-14T06:31:45.210 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032119091000000002 2024-09-14T06:31:45.210 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:45.210 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:45.210 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:31:45.210 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.077650+0000", 2024-09-14T06:31:45.210 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.2090000000000005e-06 2024-09-14T06:31:45.210 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:45.210 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:31:45.211 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:45.211 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:45.211 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:45.211 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:98508 3.10 3.1bf56f90 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T06:31:45.211 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:30:07.045530+0000", 2024-09-14T06:31:45.211 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 98.151408506999999, 2024-09-14T06:31:45.211 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032198773999999999, 2024-09-14T06:31:45.211 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:31:45.211 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:31:45.211 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "reached pg", 2024-09-14T06:31:45.211 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:31:45.211 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:31:45.211 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:45.211 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 98508 2024-09-14T06:31:45.211 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:45.212 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:31:45.212 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:45.212 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:31:45.212 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045530+0000", 2024-09-14T06:31:45.212 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:45.212 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:45.212 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:45.212 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:31:45.212 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045530+0000", 2024-09-14T06:31:45.212 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:45.212 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:45.212 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:45.212 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:31:45.212 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045530+0000", 2024-09-14T06:31:45.213 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.3099999999999998e-07 2024-09-14T06:31:45.213 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:45.213 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:45.213 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:31:45.213 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045539+0000", 2024-09-14T06:31:45.213 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.4780000000000008e-06 2024-09-14T06:31:45.213 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:45.213 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:45.213 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:31:45.214 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045546+0000", 2024-09-14T06:31:45.214 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.3259999999999998e-06 2024-09-14T06:31:45.214 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:45.214 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:45.214 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:31:45.214 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045551+0000", 2024-09-14T06:31:45.214 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.9960000000000001e-06 2024-09-14T06:31:45.214 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:45.214 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:45.214 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:31:45.214 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.077716+0000", 2024-09-14T06:31:45.214 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032165081999999998 2024-09-14T06:31:45.214 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:45.215 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:45.215 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:31:45.215 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.077729+0000", 2024-09-14T06:31:45.215 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.2461e-05 2024-09-14T06:31:45.215 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:45.215 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:31:45.215 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:45.215 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:45.215 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:45.215 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:98509 3.10 3.1bf56f90 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T06:31:45.215 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:30:07.045558+0000", 2024-09-14T06:31:45.215 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 98.151380305999993, 2024-09-14T06:31:45.215 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032242198, 2024-09-14T06:31:45.215 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:31:45.215 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:31:45.216 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "reached pg", 2024-09-14T06:31:45.216 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:31:45.216 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:31:45.216 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:45.216 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 98509 2024-09-14T06:31:45.216 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:45.216 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:31:45.216 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:45.216 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:31:45.216 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045558+0000", 2024-09-14T06:31:45.216 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:45.216 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:45.216 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:45.216 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:31:45.217 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045558+0000", 2024-09-14T06:31:45.217 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:45.217 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:45.217 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:45.217 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:31:45.217 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045559+0000", 2024-09-14T06:31:45.217 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.27e-07 2024-09-14T06:31:45.217 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:45.217 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:45.217 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:31:45.217 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045560+0000", 2024-09-14T06:31:45.217 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.6959999999999999e-06 2024-09-14T06:31:45.217 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:45.217 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:45.218 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:31:45.218 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045567+0000", 2024-09-14T06:31:45.218 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.6900000000000003e-06 2024-09-14T06:31:45.218 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:45.218 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:45.218 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:31:45.218 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045571+0000", 2024-09-14T06:31:45.218 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.5750000000000002e-06 2024-09-14T06:31:45.218 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:45.218 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:45.218 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:31:45.218 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.077789+0000", 2024-09-14T06:31:45.218 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032217186000000002 2024-09-14T06:31:45.218 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:45.219 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:45.219 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:31:45.219 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.077800+0000", 2024-09-14T06:31:45.219 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.1623999999999999e-05 2024-09-14T06:31:45.219 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:45.219 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:31:45.219 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:45.219 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:45.219 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:45.219 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:98510 3.10 3.1bf56f90 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T06:31:45.219 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:30:07.045580+0000", 2024-09-14T06:31:45.219 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 98.151358492, 2024-09-14T06:31:45.219 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032299845000000001, 2024-09-14T06:31:45.219 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:31:45.219 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:31:45.220 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "reached pg", 2024-09-14T06:31:45.220 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:31:45.220 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:31:45.220 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:45.220 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 98510 2024-09-14T06:31:45.220 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:45.220 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:31:45.220 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:45.220 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:31:45.220 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045580+0000", 2024-09-14T06:31:45.220 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:45.220 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:45.220 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:45.221 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:31:45.221 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045580+0000", 2024-09-14T06:31:45.221 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:45.221 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:45.221 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:45.221 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:31:45.221 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045580+0000", 2024-09-14T06:31:45.221 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.0400000000000002e-07 2024-09-14T06:31:45.221 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:45.221 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:45.221 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:31:45.221 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045582+0000", 2024-09-14T06:31:45.221 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.7600000000000001e-06 2024-09-14T06:31:45.221 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:45.222 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:45.222 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:31:45.222 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045588+0000", 2024-09-14T06:31:45.222 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.5650000000000001e-06 2024-09-14T06:31:45.222 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:45.222 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:45.222 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:31:45.222 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045597+0000", 2024-09-14T06:31:45.222 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.465e-06 2024-09-14T06:31:45.222 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:45.222 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:45.222 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:31:45.222 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.077871+0000", 2024-09-14T06:31:45.222 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032273562999999998 2024-09-14T06:31:45.222 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:45.223 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:45.223 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:31:45.223 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.077880+0000", 2024-09-14T06:31:45.223 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.0880000000000001e-06 2024-09-14T06:31:45.223 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:45.223 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:31:45.223 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:45.223 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:45.223 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:45.223 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:98511 3.10 3.1bf56f90 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T06:31:45.223 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:30:07.045610+0000", 2024-09-14T06:31:45.223 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 98.151328645000007, 2024-09-14T06:31:45.223 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032338894, 2024-09-14T06:31:45.223 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:31:45.223 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:31:45.224 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "reached pg", 2024-09-14T06:31:45.224 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:31:45.224 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:31:45.224 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:45.224 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 98511 2024-09-14T06:31:45.224 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:45.224 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:31:45.224 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:45.224 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:31:45.224 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045610+0000", 2024-09-14T06:31:45.224 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:45.224 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:45.224 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:45.224 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:31:45.224 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045610+0000", 2024-09-14T06:31:45.224 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:45.225 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:45.225 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:45.225 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:31:45.225 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045610+0000", 2024-09-14T06:31:45.225 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.1600000000000002e-07 2024-09-14T06:31:45.225 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:45.225 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:45.225 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:31:45.225 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045612+0000", 2024-09-14T06:31:45.225 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.1150000000000001e-06 2024-09-14T06:31:45.225 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:45.225 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:45.225 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:31:45.226 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045618+0000", 2024-09-14T06:31:45.226 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.0499999999999997e-06 2024-09-14T06:31:45.226 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:45.226 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:45.226 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:31:45.226 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045623+0000", 2024-09-14T06:31:45.226 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.6770000000000003e-06 2024-09-14T06:31:45.226 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:45.226 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:45.226 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:31:45.226 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.077928+0000", 2024-09-14T06:31:45.226 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032305111999999997 2024-09-14T06:31:45.226 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:45.226 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:45.226 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:31:45.226 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.077949+0000", 2024-09-14T06:31:45.227 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.0523999999999999e-05 2024-09-14T06:31:45.227 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:45.227 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:31:45.227 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:45.227 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:45.227 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:45.227 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:98512 3.10 3.1bf56f90 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T06:31:45.227 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:30:07.045630+0000", 2024-09-14T06:31:45.227 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 98.151308639999996, 2024-09-14T06:31:45.227 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032346632, 2024-09-14T06:31:45.227 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:31:45.227 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:31:45.227 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "reached pg", 2024-09-14T06:31:45.228 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:31:45.228 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:31:45.228 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:45.228 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 98512 2024-09-14T06:31:45.228 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:45.228 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:31:45.228 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:45.228 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:31:45.228 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045630+0000", 2024-09-14T06:31:45.228 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:45.228 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:45.228 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:45.228 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:31:45.228 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045630+0000", 2024-09-14T06:31:45.228 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:45.229 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:45.229 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:45.229 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:31:45.229 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045630+0000", 2024-09-14T06:31:45.229 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.8200000000000001e-07 2024-09-14T06:31:45.229 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:45.229 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:45.229 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:31:45.229 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045632+0000", 2024-09-14T06:31:45.229 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.7430000000000001e-06 2024-09-14T06:31:45.229 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:45.229 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:45.229 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:31:45.229 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045637+0000", 2024-09-14T06:31:45.230 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.4550000000000003e-06 2024-09-14T06:31:45.230 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:45.230 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:45.230 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:31:45.230 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045644+0000", 2024-09-14T06:31:45.230 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.1330000000000001e-06 2024-09-14T06:31:45.230 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:45.230 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:45.230 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:31:45.230 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.077968+0000", 2024-09-14T06:31:45.230 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032323401000000002 2024-09-14T06:31:45.230 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:45.230 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:45.230 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:31:45.230 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.077976+0000", 2024-09-14T06:31:45.231 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.5180000000000004e-06 2024-09-14T06:31:45.231 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:45.231 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:31:45.231 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:45.231 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:45.231 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:45.231 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:98513 3.10 3.1bf56f90 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T06:31:45.231 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:30:07.045652+0000", 2024-09-14T06:31:45.231 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 98.151286076000005, 2024-09-14T06:31:45.231 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032353436999999999, 2024-09-14T06:31:45.231 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:31:45.231 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:31:45.231 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "reached pg", 2024-09-14T06:31:45.232 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:31:45.232 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:31:45.232 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:45.232 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 98513 2024-09-14T06:31:45.232 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:45.232 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:31:45.232 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:45.232 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:31:45.232 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045652+0000", 2024-09-14T06:31:45.232 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:45.233 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:45.233 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:45.233 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:31:45.233 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045652+0000", 2024-09-14T06:31:45.233 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:45.233 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:45.233 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:45.233 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:31:45.233 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045653+0000", 2024-09-14T06:31:45.233 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.3700000000000001e-07 2024-09-14T06:31:45.233 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:45.233 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:45.233 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:31:45.233 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045655+0000", 2024-09-14T06:31:45.233 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.854e-06 2024-09-14T06:31:45.234 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:45.234 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:45.234 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:31:45.234 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045661+0000", 2024-09-14T06:31:45.234 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.5509999999999996e-06 2024-09-14T06:31:45.234 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:45.234 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:45.234 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:31:45.234 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045668+0000", 2024-09-14T06:31:45.234 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.1980000000000004e-06 2024-09-14T06:31:45.234 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:45.234 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:45.235 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:31:45.235 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.077997+0000", 2024-09-14T06:31:45.235 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032328575999999998 2024-09-14T06:31:45.235 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:45.235 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:45.235 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:31:45.235 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.078006+0000", 2024-09-14T06:31:45.235 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.8210000000000008e-06 2024-09-14T06:31:45.235 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:45.235 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:31:45.235 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:45.235 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:45.235 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:45.235 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:98514 3.10 3.1bf56f90 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T06:31:45.235 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:30:07.045699+0000", 2024-09-14T06:31:45.235 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 98.151239740999998, 2024-09-14T06:31:45.236 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032330223999999998, 2024-09-14T06:31:45.236 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:31:45.236 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:31:45.236 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "reached pg", 2024-09-14T06:31:45.236 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:31:45.236 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:31:45.236 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:45.236 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 98514 2024-09-14T06:31:45.236 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:45.236 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:31:45.236 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:45.236 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:31:45.236 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045699+0000", 2024-09-14T06:31:45.236 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:45.236 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:45.237 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:45.237 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:31:45.237 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045699+0000", 2024-09-14T06:31:45.237 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:45.237 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:45.237 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:45.237 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:31:45.237 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045699+0000", 2024-09-14T06:31:45.237 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.3200000000000001e-07 2024-09-14T06:31:45.237 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:45.237 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:45.237 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:31:45.237 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045701+0000", 2024-09-14T06:31:45.237 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.762e-06 2024-09-14T06:31:45.237 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:45.237 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:45.238 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:31:45.238 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045705+0000", 2024-09-14T06:31:45.238 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.2359999999999998e-06 2024-09-14T06:31:45.238 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:45.238 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:45.238 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:31:45.238 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045708+0000", 2024-09-14T06:31:45.238 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.796e-06 2024-09-14T06:31:45.238 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:45.238 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:45.238 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:31:45.238 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.078023+0000", 2024-09-14T06:31:45.238 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032315213000000002 2024-09-14T06:31:45.238 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:45.238 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:45.238 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:31:45.239 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.078029+0000", 2024-09-14T06:31:45.239 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.8850000000000004e-06 2024-09-14T06:31:45.239 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:45.239 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:31:45.239 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:45.239 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:45.239 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:45.239 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:98515 3.10 3.1bf56f90 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T06:31:45.239 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:30:07.045712+0000", 2024-09-14T06:31:45.239 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 98.151226429999994, 2024-09-14T06:31:45.239 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032346871999999999, 2024-09-14T06:31:45.239 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:31:45.239 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:31:45.239 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "reached pg", 2024-09-14T06:31:45.239 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:31:45.239 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:31:45.240 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:45.240 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 98515 2024-09-14T06:31:45.240 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:45.240 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:31:45.240 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:45.240 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:31:45.240 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045712+0000", 2024-09-14T06:31:45.240 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:45.240 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:45.240 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:45.240 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:31:45.240 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045712+0000", 2024-09-14T06:31:45.240 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:45.240 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:45.240 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:45.241 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:31:45.241 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045712+0000", 2024-09-14T06:31:45.241 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.2100000000000001e-07 2024-09-14T06:31:45.241 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:45.241 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:45.241 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:31:45.241 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045713+0000", 2024-09-14T06:31:45.241 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.068e-06 2024-09-14T06:31:45.241 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:45.241 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:45.241 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:31:45.241 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045716+0000", 2024-09-14T06:31:45.241 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.971e-06 2024-09-14T06:31:45.242 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:45.242 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:45.242 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:31:45.242 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045719+0000", 2024-09-14T06:31:45.242 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.3879999999999998e-06 2024-09-14T06:31:45.242 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:45.242 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:45.242 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:31:45.242 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.078049+0000", 2024-09-14T06:31:45.242 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032330553999999997 2024-09-14T06:31:45.242 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:45.242 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:45.242 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:31:45.242 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.078059+0000", 2024-09-14T06:31:45.243 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.6700000000000006e-06 2024-09-14T06:31:45.243 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:45.243 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:31:45.243 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:45.243 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:45.243 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:45.243 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:98516 3.10 3.1bf56f90 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T06:31:45.243 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:30:07.045722+0000", 2024-09-14T06:31:45.243 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 98.151216676999994, 2024-09-14T06:31:45.243 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032374334999999997, 2024-09-14T06:31:45.243 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:31:45.243 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:31:45.243 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "reached pg", 2024-09-14T06:31:45.243 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:31:45.243 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:31:45.244 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:45.244 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 98516 2024-09-14T06:31:45.244 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:45.244 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:31:45.244 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:45.244 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:31:45.244 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045722+0000", 2024-09-14T06:31:45.244 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:45.244 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:45.244 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:45.244 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:31:45.244 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045722+0000", 2024-09-14T06:31:45.244 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:45.244 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:45.245 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:45.245 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:31:45.245 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045722+0000", 2024-09-14T06:31:45.245 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.8999999999999998e-07 2024-09-14T06:31:45.245 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:45.245 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:45.245 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:31:45.245 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045723+0000", 2024-09-14T06:31:45.245 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.0049999999999999e-06 2024-09-14T06:31:45.245 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:45.245 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:45.245 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:31:45.245 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045726+0000", 2024-09-14T06:31:45.245 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.9019999999999999e-06 2024-09-14T06:31:45.246 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:45.246 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:45.246 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:31:45.246 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045728+0000", 2024-09-14T06:31:45.246 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.3290000000000001e-06 2024-09-14T06:31:45.246 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:45.246 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:45.246 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:31:45.246 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.078090+0000", 2024-09-14T06:31:45.246 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032362123 2024-09-14T06:31:45.246 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:45.246 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:45.246 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:31:45.246 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.078096+0000", 2024-09-14T06:31:45.247 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.6860000000000003e-06 2024-09-14T06:31:45.247 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:45.247 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:31:45.247 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:45.247 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:45.247 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:45.247 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:98517 3.10 3.1bf56f90 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T06:31:45.247 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:30:07.045731+0000", 2024-09-14T06:31:45.247 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 98.151206946000002, 2024-09-14T06:31:45.247 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.03241227, 2024-09-14T06:31:45.247 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:31:45.247 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:31:45.247 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "reached pg", 2024-09-14T06:31:45.247 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:31:45.247 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:31:45.248 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:45.248 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 98517 2024-09-14T06:31:45.248 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:45.248 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:31:45.248 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:45.248 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:31:45.248 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045731+0000", 2024-09-14T06:31:45.248 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:45.248 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:45.248 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:45.248 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:31:45.248 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045731+0000", 2024-09-14T06:31:45.248 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:45.248 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:45.249 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:45.249 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:31:45.249 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045732+0000", 2024-09-14T06:31:45.249 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.0699999999999999e-07 2024-09-14T06:31:45.249 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:45.249 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:45.249 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:31:45.249 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045733+0000", 2024-09-14T06:31:45.249 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.9199999999999999e-07 2024-09-14T06:31:45.249 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:45.249 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:45.249 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:31:45.249 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045735+0000", 2024-09-14T06:31:45.249 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.8150000000000002e-06 2024-09-14T06:31:45.249 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:45.250 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:45.250 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:31:45.250 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045738+0000", 2024-09-14T06:31:45.250 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.3640000000000002e-06 2024-09-14T06:31:45.250 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:45.250 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:45.250 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:31:45.250 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.078126+0000", 2024-09-14T06:31:45.250 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032388206000000003 2024-09-14T06:31:45.250 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:45.250 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:45.250 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:31:45.250 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.078144+0000", 2024-09-14T06:31:45.250 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.7686e-05 2024-09-14T06:31:45.251 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:45.251 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:31:45.251 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:45.251 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:45.251 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:45.251 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:98518 3.10 3.1bf56f90 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T06:31:45.251 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:30:07.045741+0000", 2024-09-14T06:31:45.251 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 98.151197550999996, 2024-09-14T06:31:45.251 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032451793999999999, 2024-09-14T06:31:45.251 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:31:45.251 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:31:45.251 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "reached pg", 2024-09-14T06:31:45.251 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:31:45.252 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:31:45.252 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:45.252 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 98518 2024-09-14T06:31:45.252 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:45.252 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:31:45.252 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:45.252 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:31:45.252 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045741+0000", 2024-09-14T06:31:45.252 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:45.252 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:45.252 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:45.252 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:31:45.252 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045741+0000", 2024-09-14T06:31:45.252 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:45.252 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:45.253 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:45.253 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:31:45.253 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045741+0000", 2024-09-14T06:31:45.253 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.05e-07 2024-09-14T06:31:45.253 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:45.253 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:45.253 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:31:45.253 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045746+0000", 2024-09-14T06:31:45.253 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.2259999999999996e-06 2024-09-14T06:31:45.253 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:45.253 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:45.253 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:31:45.253 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045750+0000", 2024-09-14T06:31:45.254 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.2490000000000001e-06 2024-09-14T06:31:45.254 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:45.254 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:45.254 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:31:45.254 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045752+0000", 2024-09-14T06:31:45.254 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.3980000000000002e-06 2024-09-14T06:31:45.254 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:45.254 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:45.254 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:31:45.254 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.078184+0000", 2024-09-14T06:31:45.254 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032431593000000002 2024-09-14T06:31:45.254 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:45.254 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:45.254 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:31:45.254 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.078193+0000", 2024-09-14T06:31:45.254 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.1230000000000006e-06 2024-09-14T06:31:45.255 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:45.255 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:31:45.255 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:45.255 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:45.255 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:45.255 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:98519 3.10 3.1bf56f90 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T06:31:45.255 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:30:07.045755+0000", 2024-09-14T06:31:45.255 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 98.151183126000006, 2024-09-14T06:31:45.255 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032489431999999999, 2024-09-14T06:31:45.255 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:31:45.255 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:31:45.255 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "reached pg", 2024-09-14T06:31:45.255 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:31:45.255 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:31:45.255 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:45.256 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 98519 2024-09-14T06:31:45.256 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:45.256 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:31:45.256 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:45.256 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:31:45.256 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045755+0000", 2024-09-14T06:31:45.256 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:45.256 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:45.256 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:45.256 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:31:45.256 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045755+0000", 2024-09-14T06:31:45.256 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:45.256 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:45.256 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:45.256 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:31:45.257 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045756+0000", 2024-09-14T06:31:45.257 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.23e-07 2024-09-14T06:31:45.257 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:45.257 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:45.257 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:31:45.257 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045757+0000", 2024-09-14T06:31:45.257 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.1060000000000001e-06 2024-09-14T06:31:45.257 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:45.257 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:45.257 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:31:45.257 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045760+0000", 2024-09-14T06:31:45.257 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.988e-06 2024-09-14T06:31:45.257 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:45.257 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:45.258 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:31:45.258 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045765+0000", 2024-09-14T06:31:45.258 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.2689999999999999e-06 2024-09-14T06:31:45.258 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:45.258 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:45.258 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:31:45.258 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.078234+0000", 2024-09-14T06:31:45.258 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032468915000000001 2024-09-14T06:31:45.258 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:45.258 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:45.258 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:31:45.258 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.078245+0000", 2024-09-14T06:31:45.258 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.0930999999999999e-05 2024-09-14T06:31:45.259 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:45.259 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:31:45.259 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:45.259 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:45.259 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:45.259 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:98520 3.10 3.1bf56f90 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T06:31:45.259 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:30:07.045768+0000", 2024-09-14T06:31:45.259 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 98.151170248, 2024-09-14T06:31:45.259 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032531379999999999, 2024-09-14T06:31:45.259 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:31:45.259 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:31:45.259 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "reached pg", 2024-09-14T06:31:45.259 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:31:45.260 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:31:45.260 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:45.260 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 98520 2024-09-14T06:31:45.260 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:45.260 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:31:45.260 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:45.260 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:31:45.260 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045768+0000", 2024-09-14T06:31:45.260 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:45.260 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:45.260 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:45.260 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:31:45.260 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045768+0000", 2024-09-14T06:31:45.260 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:45.261 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:45.261 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:45.261 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:31:45.261 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045768+0000", 2024-09-14T06:31:45.261 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.22e-07 2024-09-14T06:31:45.261 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:45.261 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:45.261 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:31:45.261 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045773+0000", 2024-09-14T06:31:45.261 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.3470000000000001e-06 2024-09-14T06:31:45.261 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:45.261 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:45.261 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:31:45.261 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045776+0000", 2024-09-14T06:31:45.262 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.1319999999999998e-06 2024-09-14T06:31:45.262 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:45.262 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:45.262 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:31:45.262 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045779+0000", 2024-09-14T06:31:45.262 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.9749999999999999e-06 2024-09-14T06:31:45.262 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:45.262 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:45.262 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:31:45.262 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.078289+0000", 2024-09-14T06:31:45.262 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032509693999999999 2024-09-14T06:31:45.262 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:45.262 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:45.262 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:31:45.263 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.078300+0000", 2024-09-14T06:31:45.263 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.101e-05 2024-09-14T06:31:45.263 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:45.263 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:31:45.263 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:45.263 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:45.263 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:45.263 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:98521 3.10 3.1bf56f90 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T06:31:45.263 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:30:07.045782+0000", 2024-09-14T06:31:45.263 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 98.151156068000006, 2024-09-14T06:31:45.263 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032566109000000003, 2024-09-14T06:31:45.263 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:31:45.263 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:31:45.263 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "reached pg", 2024-09-14T06:31:45.264 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:31:45.264 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:31:45.264 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:45.264 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 98521 2024-09-14T06:31:45.264 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:45.264 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:31:45.264 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:45.264 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:31:45.264 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045782+0000", 2024-09-14T06:31:45.264 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:45.264 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:45.264 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:45.264 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:31:45.264 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045782+0000", 2024-09-14T06:31:45.265 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:45.265 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:45.265 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:45.265 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:31:45.265 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045783+0000", 2024-09-14T06:31:45.265 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.2499999999999999e-07 2024-09-14T06:31:45.265 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:45.265 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:45.265 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:31:45.265 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045784+0000", 2024-09-14T06:31:45.265 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.0109999999999999e-06 2024-09-14T06:31:45.265 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:45.265 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:45.265 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:31:45.266 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045786+0000", 2024-09-14T06:31:45.266 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.8289999999999999e-06 2024-09-14T06:31:45.266 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:45.266 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:45.266 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:31:45.266 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045793+0000", 2024-09-14T06:31:45.266 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.579e-06 2024-09-14T06:31:45.266 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:45.266 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:45.266 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:31:45.266 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.078338+0000", 2024-09-14T06:31:45.266 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032545389000000001 2024-09-14T06:31:45.266 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:45.266 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:45.266 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:31:45.266 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.078348+0000", 2024-09-14T06:31:45.267 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.0076e-05 2024-09-14T06:31:45.267 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:45.267 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:31:45.267 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:45.267 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:45.267 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:45.267 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:98522 3.10 3.1bf56f90 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T06:31:45.267 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:30:07.045796+0000", 2024-09-14T06:31:45.267 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 98.151142160000006, 2024-09-14T06:31:45.267 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032577846000000001, 2024-09-14T06:31:45.267 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:31:45.267 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:31:45.267 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "reached pg", 2024-09-14T06:31:45.267 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:31:45.268 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:31:45.268 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:45.268 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 98522 2024-09-14T06:31:45.268 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:45.268 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:31:45.268 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:45.268 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:31:45.268 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045796+0000", 2024-09-14T06:31:45.268 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:45.268 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:45.268 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:45.268 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:31:45.268 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045796+0000", 2024-09-14T06:31:45.268 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:45.269 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:45.269 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:45.269 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:31:45.269 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045796+0000", 2024-09-14T06:31:45.269 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.2100000000000001e-07 2024-09-14T06:31:45.269 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:45.269 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:45.269 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:31:45.269 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045797+0000", 2024-09-14T06:31:45.269 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.6899999999999996e-07 2024-09-14T06:31:45.269 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:45.269 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:45.269 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:31:45.270 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045800+0000", 2024-09-14T06:31:45.270 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.9979999999999999e-06 2024-09-14T06:31:45.270 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:45.270 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:45.270 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:31:45.270 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045803+0000", 2024-09-14T06:31:45.270 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.2850000000000001e-06 2024-09-14T06:31:45.270 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:45.270 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:45.270 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:31:45.270 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.078366+0000", 2024-09-14T06:31:45.270 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032562932000000003 2024-09-14T06:31:45.270 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:45.270 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:45.271 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:31:45.271 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.078374+0000", 2024-09-14T06:31:45.271 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.4409999999999993e-06 2024-09-14T06:31:45.271 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:45.271 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:31:45.271 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:45.271 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:45.271 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:45.271 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:98523 3.10 3.1bf56f90 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T06:31:45.271 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:30:07.045806+0000", 2024-09-14T06:31:45.271 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 98.151132617000002, 2024-09-14T06:31:45.271 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032611167000000003, 2024-09-14T06:31:45.271 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:31:45.272 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:31:45.272 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "reached pg", 2024-09-14T06:31:45.272 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:31:45.272 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:31:45.272 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:45.272 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 98523 2024-09-14T06:31:45.272 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:45.272 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:31:45.272 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:45.272 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:31:45.272 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045806+0000", 2024-09-14T06:31:45.272 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:45.272 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:45.272 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:45.273 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:31:45.273 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045806+0000", 2024-09-14T06:31:45.273 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:45.273 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:45.273 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:45.273 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:31:45.273 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045806+0000", 2024-09-14T06:31:45.273 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.2000000000000001e-07 2024-09-14T06:31:45.273 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:45.273 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:45.273 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:31:45.273 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045807+0000", 2024-09-14T06:31:45.273 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.4e-07 2024-09-14T06:31:45.273 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:45.274 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:45.274 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:31:45.274 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045810+0000", 2024-09-14T06:31:45.274 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.9229999999999998e-06 2024-09-14T06:31:45.274 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:45.274 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:45.274 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:31:45.274 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045812+0000", 2024-09-14T06:31:45.274 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.2220000000000001e-06 2024-09-14T06:31:45.274 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:45.274 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:45.274 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:31:45.274 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.078407+0000", 2024-09-14T06:31:45.274 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032594513999999998 2024-09-14T06:31:45.274 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:45.274 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:45.275 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:31:45.275 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.078417+0000", 2024-09-14T06:31:45.275 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.0348e-05 2024-09-14T06:31:45.275 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:45.275 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:31:45.275 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:45.275 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:45.275 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:45.275 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:98524 3.10 3.1bf56f90 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T06:31:45.275 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:30:07.045815+0000", 2024-09-14T06:31:45.275 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 98.151123248999994, 2024-09-14T06:31:45.275 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032643208999999999, 2024-09-14T06:31:45.275 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:31:45.275 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:31:45.275 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "reached pg", 2024-09-14T06:31:45.276 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:31:45.276 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:31:45.276 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:45.276 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 98524 2024-09-14T06:31:45.276 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:45.276 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:31:45.276 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:45.276 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:31:45.276 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045815+0000", 2024-09-14T06:31:45.276 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:45.276 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:45.276 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:45.276 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:31:45.276 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045815+0000", 2024-09-14T06:31:45.276 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:45.277 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:45.277 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:45.277 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:31:45.277 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045815+0000", 2024-09-14T06:31:45.277 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.11e-07 2024-09-14T06:31:45.277 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:45.277 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:45.277 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:31:45.277 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045816+0000", 2024-09-14T06:31:45.277 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.9099999999999991e-07 2024-09-14T06:31:45.277 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:45.277 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:45.277 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:31:45.277 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045819+0000", 2024-09-14T06:31:45.278 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.9399999999999998e-06 2024-09-14T06:31:45.278 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:45.278 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:45.278 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:31:45.278 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045822+0000", 2024-09-14T06:31:45.278 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.3470000000000001e-06 2024-09-14T06:31:45.278 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:45.278 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:45.278 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:31:45.278 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.078449+0000", 2024-09-14T06:31:45.278 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032627529000000002 2024-09-14T06:31:45.278 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:45.278 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:45.278 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:31:45.278 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.078458+0000", 2024-09-14T06:31:45.278 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.1910000000000006e-06 2024-09-14T06:31:45.279 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:45.279 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:31:45.279 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:45.279 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:45.279 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:31:45.279 INFO:teuthology.orchestra.run.smithi138.stdout:} 2024-09-14T06:31:45.280 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-14T06:31:45.281 INFO:tasks.ceph.osd.6.smithi138.stderr:2024-09-14T06:31:45.279+0000 7f8cd8849640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33620) UID: 0 2024-09-14T06:31:45.381 INFO:tasks.ceph.osd.7.smithi138.stderr:2024-09-14T06:31:45.380+0000 7f3c6a21d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33623) UID: 0 2024-09-14T06:31:45.424 INFO:teuthology.orchestra.run.smithi138.stdout:{ 2024-09-14T06:31:45.424 INFO:teuthology.orchestra.run.smithi138.stdout: "ops": [], 2024-09-14T06:31:45.424 INFO:teuthology.orchestra.run.smithi138.stdout: "num_ops": 0 2024-09-14T06:31:45.424 INFO:teuthology.orchestra.run.smithi138.stdout:} 2024-09-14T06:31:45.432 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-14T06:31:45.505 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-14T06:31:45.516 INFO:tasks.ceph.osd.5.smithi138.stderr:2024-09-14T06:31:45.515+0000 7fe13746f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33618) UID: 0 2024-09-14T06:31:45.575 INFO:teuthology.orchestra.run.smithi138.stdout:{ 2024-09-14T06:31:45.575 INFO:teuthology.orchestra.run.smithi138.stdout: "ops": [], 2024-09-14T06:31:45.576 INFO:teuthology.orchestra.run.smithi138.stdout: "complaint_time": 30, 2024-09-14T06:31:45.576 INFO:teuthology.orchestra.run.smithi138.stdout: "num_blocked_ops": 0 2024-09-14T06:31:45.576 INFO:teuthology.orchestra.run.smithi138.stdout:} 2024-09-14T06:31:45.583 INFO:tasks.ceph.osd.0.smithi094.stderr:2024-09-14T06:31:45.582+0000 7f07c9333640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33636) UID: 0 2024-09-14T06:31:45.584 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-14T06:31:45.630 INFO:teuthology.orchestra.run.smithi094.stdout:osd.0: {} 2024-09-14T06:31:45.631 INFO:teuthology.orchestra.run.smithi094.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-14T06:31:45.647 INFO:teuthology.orchestra.run.smithi094.stdout:osd.1: {} 2024-09-14T06:31:45.647 INFO:teuthology.orchestra.run.smithi094.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-14T06:31:45.663 INFO:teuthology.orchestra.run.smithi094.stdout:osd.2: {} 2024-09-14T06:31:45.663 INFO:teuthology.orchestra.run.smithi094.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-14T06:31:45.679 INFO:teuthology.orchestra.run.smithi094.stdout:osd.3: {} 2024-09-14T06:31:45.679 INFO:teuthology.orchestra.run.smithi094.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-14T06:31:45.684 INFO:tasks.ceph.osd.1.smithi094.stderr:2024-09-14T06:31:45.683+0000 7f328e47a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33641) UID: 0 2024-09-14T06:31:45.696 INFO:teuthology.orchestra.run.smithi094.stdout:osd.4: {} 2024-09-14T06:31:45.696 INFO:teuthology.orchestra.run.smithi094.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-14T06:31:45.709 INFO:teuthology.orchestra.run.smithi094.stderr:nodeep-scrub is unset 2024-09-14T06:31:45.713 INFO:teuthology.orchestra.run.smithi094.stdout:osd.5: {} 2024-09-14T06:31:45.714 INFO:teuthology.orchestra.run.smithi094.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-14T06:31:45.732 INFO:teuthology.orchestra.run.smithi094.stdout:osd.6: {} 2024-09-14T06:31:45.733 INFO:teuthology.orchestra.run.smithi094.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-14T06:31:45.735 INFO:teuthology.orchestra.run.smithi138.stdout:{ 2024-09-14T06:31:45.735 INFO:teuthology.orchestra.run.smithi138.stdout: "size": 20, 2024-09-14T06:31:45.736 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 600, 2024-09-14T06:31:45.736 INFO:teuthology.orchestra.run.smithi138.stdout: "ops": [ 2024-09-14T06:31:45.736 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:45.736 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:53244 3.f 3:f71f51a7:::rbd_data.112dca2f6ae1.0000000000000015:17 [read 1863680~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:31:45.736 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:44.369399+0000", 2024-09-14T06:31:45.736 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 121.36460440800001, 2024-09-14T06:31:45.736 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047291189999999997, 2024-09-14T06:31:45.736 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:31:45.736 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:31:45.736 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:31:45.736 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:31:45.736 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:31:45.736 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:45.737 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 53244 2024-09-14T06:31:45.737 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:45.737 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:31:45.737 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:45.737 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:31:45.737 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369399+0000", 2024-09-14T06:31:45.737 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:45.737 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:45.737 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:45.737 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:31:45.737 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369399+0000", 2024-09-14T06:31:45.737 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:45.737 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:45.737 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:45.738 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:31:45.738 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369400+0000", 2024-09-14T06:31:45.738 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.0699999999999999e-07 2024-09-14T06:31:45.738 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:45.738 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:45.738 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:31:45.738 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369401+0000", 2024-09-14T06:31:45.738 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.4300000000000001e-06 2024-09-14T06:31:45.738 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:45.738 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:45.738 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:31:45.738 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369404+0000", 2024-09-14T06:31:45.738 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.061e-06 2024-09-14T06:31:45.738 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:45.739 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:45.739 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:31:45.739 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369407+0000", 2024-09-14T06:31:45.739 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.6139999999999999e-06 2024-09-14T06:31:45.739 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:45.739 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:45.739 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:31:45.739 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.416517+0000", 2024-09-14T06:31:45.739 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047110338000000002 2024-09-14T06:31:45.739 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:45.739 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:45.739 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:31:45.739 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.416616+0000", 2024-09-14T06:31:45.739 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.8651000000000001e-05 2024-09-14T06:31:45.739 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:45.740 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:45.740 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:31:45.740 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.416691+0000", 2024-09-14T06:31:45.740 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.4888999999999994e-05 2024-09-14T06:31:45.740 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:45.740 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:31:45.740 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:45.740 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:45.740 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:45.740 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:53245 3.f 3:f71f51a7:::rbd_data.112dca2f6ae1.0000000000000015:17 [read 1888256~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:31:45.740 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:44.369410+0000", 2024-09-14T06:31:45.740 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 121.36459412000001, 2024-09-14T06:31:45.740 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047399275999999997, 2024-09-14T06:31:45.740 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:31:45.741 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:31:45.741 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:31:45.741 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:31:45.741 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:31:45.741 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:45.741 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 53245 2024-09-14T06:31:45.741 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:45.741 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:31:45.741 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:45.741 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:31:45.741 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369410+0000", 2024-09-14T06:31:45.741 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:45.741 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:45.741 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:45.742 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:31:45.742 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369410+0000", 2024-09-14T06:31:45.742 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:45.742 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:45.742 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:45.742 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:31:45.742 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369410+0000", 2024-09-14T06:31:45.742 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.1899999999999999e-07 2024-09-14T06:31:45.742 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:45.742 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:45.742 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:31:45.742 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369411+0000", 2024-09-14T06:31:45.742 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.0300000000000001e-06 2024-09-14T06:31:45.742 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:45.743 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:45.743 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:31:45.743 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369414+0000", 2024-09-14T06:31:45.743 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.1099999999999999e-06 2024-09-14T06:31:45.743 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:45.743 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:45.743 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:31:45.743 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369417+0000", 2024-09-14T06:31:45.743 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.6189999999999998e-06 2024-09-14T06:31:45.743 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:45.743 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:45.743 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:31:45.743 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.416718+0000", 2024-09-14T06:31:45.743 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047301322999999999 2024-09-14T06:31:45.743 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:45.744 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:45.744 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:31:45.744 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.416759+0000", 2024-09-14T06:31:45.744 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.1009e-05 2024-09-14T06:31:45.744 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:45.744 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:45.744 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:31:45.744 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.416809+0000", 2024-09-14T06:31:45.744 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.9966000000000002e-05 2024-09-14T06:31:45.744 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:45.744 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:31:45.744 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:45.744 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:45.744 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:45.745 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:53246 3.f 3:f71f51a7:::rbd_data.112dca2f6ae1.0000000000000015:17 [read 1912832~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:31:45.745 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:44.369420+0000", 2024-09-14T06:31:45.745 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 121.36458417599999, 2024-09-14T06:31:45.745 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047529463000000001, 2024-09-14T06:31:45.745 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:31:45.745 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:31:45.745 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:31:45.745 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:31:45.745 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:31:45.745 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:45.745 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 53246 2024-09-14T06:31:45.745 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:45.745 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:31:45.745 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:45.745 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:31:45.746 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369420+0000", 2024-09-14T06:31:45.746 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:45.746 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:45.746 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:45.746 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:31:45.746 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369420+0000", 2024-09-14T06:31:45.746 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:45.746 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:45.746 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:45.746 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:31:45.746 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369420+0000", 2024-09-14T06:31:45.746 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.1299999999999999e-07 2024-09-14T06:31:45.746 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:45.746 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:45.747 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:31:45.747 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369425+0000", 2024-09-14T06:31:45.747 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.0130000000000001e-06 2024-09-14T06:31:45.747 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:45.747 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:45.747 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:31:45.747 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369428+0000", 2024-09-14T06:31:45.747 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.349e-06 2024-09-14T06:31:45.747 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:45.747 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:45.747 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:31:45.747 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369431+0000", 2024-09-14T06:31:45.747 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.7389999999999999e-06 2024-09-14T06:31:45.748 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:45.748 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:45.748 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:31:45.748 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.416834+0000", 2024-09-14T06:31:45.748 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047402669000000001 2024-09-14T06:31:45.748 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:45.748 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:45.748 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:31:45.748 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.416882+0000", 2024-09-14T06:31:45.748 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.8068999999999997e-05 2024-09-14T06:31:45.748 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:45.748 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:45.748 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:31:45.748 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.416949+0000", 2024-09-14T06:31:45.748 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.7410999999999995e-05 2024-09-14T06:31:45.749 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:45.749 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:31:45.749 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:45.749 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:45.749 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:45.749 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:53247 3.f 3:f71f51a7:::rbd_data.112dca2f6ae1.0000000000000015:17 [read 1937408~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:31:45.749 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:44.369434+0000", 2024-09-14T06:31:45.749 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 121.36456941199999, 2024-09-14T06:31:45.749 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047652264999999999, 2024-09-14T06:31:45.749 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:31:45.749 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:31:45.749 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:31:45.749 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:31:45.750 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:31:45.750 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:45.750 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 53247 2024-09-14T06:31:45.750 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:45.750 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:31:45.750 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:45.750 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:31:45.750 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369434+0000", 2024-09-14T06:31:45.750 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:45.750 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:45.750 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:45.750 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:31:45.750 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369434+0000", 2024-09-14T06:31:45.750 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:45.751 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:45.751 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:45.751 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:31:45.751 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369435+0000", 2024-09-14T06:31:45.751 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.2100000000000001e-07 2024-09-14T06:31:45.751 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:45.751 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:45.751 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:31:45.751 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369436+0000", 2024-09-14T06:31:45.751 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.122e-06 2024-09-14T06:31:45.751 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:45.751 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:45.751 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:31:45.751 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369439+0000", 2024-09-14T06:31:45.752 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.0400000000000001e-06 2024-09-14T06:31:45.752 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:45.752 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:45.752 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:31:45.752 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369441+0000", 2024-09-14T06:31:45.752 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.728e-06 2024-09-14T06:31:45.752 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:45.752 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:45.752 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:31:45.752 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.416977+0000", 2024-09-14T06:31:45.752 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047535344 2024-09-14T06:31:45.752 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:45.752 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:45.752 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:31:45.753 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.417029+0000", 2024-09-14T06:31:45.753 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.2037999999999999e-05 2024-09-14T06:31:45.753 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:45.753 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:45.753 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:31:45.753 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.417087+0000", 2024-09-14T06:31:45.753 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.7772000000000002e-05 2024-09-14T06:31:45.753 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:45.753 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:31:45.753 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:45.753 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:45.753 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:45.753 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:53248 3.f 3:f71f51a7:::rbd_data.112dca2f6ae1.0000000000000015:17 [read 1961984~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:31:45.753 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:44.369445+0000", 2024-09-14T06:31:45.753 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 121.36455886, 2024-09-14T06:31:45.754 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047798489, 2024-09-14T06:31:45.754 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:31:45.754 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:31:45.754 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:31:45.754 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:31:45.754 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:31:45.754 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:45.754 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 53248 2024-09-14T06:31:45.754 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:45.754 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:31:45.754 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:45.754 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:31:45.754 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369445+0000", 2024-09-14T06:31:45.755 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:45.755 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:45.755 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:45.755 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:31:45.755 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369445+0000", 2024-09-14T06:31:45.755 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:45.755 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:45.755 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:45.755 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:31:45.755 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369445+0000", 2024-09-14T06:31:45.755 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.0699999999999999e-07 2024-09-14T06:31:45.755 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:45.755 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:45.756 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:31:45.756 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369446+0000", 2024-09-14T06:31:45.756 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.1000000000000001e-06 2024-09-14T06:31:45.756 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:45.756 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:45.756 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:31:45.756 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369449+0000", 2024-09-14T06:31:45.756 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.1039999999999998e-06 2024-09-14T06:31:45.756 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:45.756 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:45.756 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:31:45.756 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369452+0000", 2024-09-14T06:31:45.756 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.6199999999999999e-06 2024-09-14T06:31:45.756 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:45.757 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:45.757 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:31:45.757 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.417114+0000", 2024-09-14T06:31:45.757 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047661901999999999 2024-09-14T06:31:45.757 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:45.757 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:45.757 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:31:45.757 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.417181+0000", 2024-09-14T06:31:45.757 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.7385999999999995e-05 2024-09-14T06:31:45.757 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:45.757 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:45.757 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:31:45.757 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.417243+0000", 2024-09-14T06:31:45.757 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.2169999999999996e-05 2024-09-14T06:31:45.758 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:45.758 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:31:45.758 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:45.758 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:45.758 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:45.758 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:53249 3.f 3:f71f51a7:::rbd_data.112dca2f6ae1.0000000000000015:17 [read 1986560~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:31:45.758 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:44.369455+0000", 2024-09-14T06:31:45.758 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 121.36454854900001, 2024-09-14T06:31:45.758 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047890493999999999, 2024-09-14T06:31:45.758 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:31:45.758 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:31:45.758 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:31:45.758 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:31:45.758 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:31:45.759 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:45.759 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 53249 2024-09-14T06:31:45.759 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:45.759 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:31:45.759 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:45.759 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:31:45.759 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369455+0000", 2024-09-14T06:31:45.759 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:45.759 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:45.759 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:45.759 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:31:45.759 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369455+0000", 2024-09-14T06:31:45.759 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:45.759 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:45.760 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:45.760 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:31:45.760 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369455+0000", 2024-09-14T06:31:45.760 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.0800000000000001e-07 2024-09-14T06:31:45.760 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:45.760 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:45.760 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:31:45.760 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369456+0000", 2024-09-14T06:31:45.760 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.0729999999999999e-06 2024-09-14T06:31:45.760 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:45.760 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:45.760 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:31:45.760 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369460+0000", 2024-09-14T06:31:45.760 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.027e-06 2024-09-14T06:31:45.761 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:45.761 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:45.761 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:31:45.761 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369462+0000", 2024-09-14T06:31:45.761 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.5320000000000001e-06 2024-09-14T06:31:45.761 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:45.761 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:45.761 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:31:45.761 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.417257+0000", 2024-09-14T06:31:45.761 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.04779489 2024-09-14T06:31:45.761 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:45.761 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:45.761 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:31:45.761 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.417296+0000", 2024-09-14T06:31:45.762 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.8658000000000002e-05 2024-09-14T06:31:45.762 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:45.762 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:45.762 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:31:45.762 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.417346+0000", 2024-09-14T06:31:45.762 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.0105999999999998e-05 2024-09-14T06:31:45.762 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:45.762 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:31:45.762 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:45.762 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:45.762 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:45.762 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:53250 3.f 3:f71f51a7:::rbd_data.112dca2f6ae1.0000000000000015:17 [read 2011136~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:31:45.762 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:44.369465+0000", 2024-09-14T06:31:45.762 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 121.364538579, 2024-09-14T06:31:45.762 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.048032230000000002, 2024-09-14T06:31:45.763 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:31:45.763 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:31:45.763 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:31:45.763 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:31:45.763 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:31:45.763 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:45.763 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 53250 2024-09-14T06:31:45.763 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:45.763 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:31:45.763 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:45.763 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:31:45.763 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369465+0000", 2024-09-14T06:31:45.763 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:45.763 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:45.764 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:45.764 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:31:45.764 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369465+0000", 2024-09-14T06:31:45.764 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:45.764 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:45.764 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:45.764 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:31:45.764 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369465+0000", 2024-09-14T06:31:45.764 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.2100000000000001e-07 2024-09-14T06:31:45.764 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:45.764 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:45.764 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:31:45.764 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369466+0000", 2024-09-14T06:31:45.765 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.0640000000000001e-06 2024-09-14T06:31:45.765 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:45.765 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:45.765 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:31:45.765 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369469+0000", 2024-09-14T06:31:45.765 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.0170000000000001e-06 2024-09-14T06:31:45.765 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:45.765 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:45.765 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:31:45.765 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369472+0000", 2024-09-14T06:31:45.766 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.6189999999999998e-06 2024-09-14T06:31:45.766 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:45.766 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:45.766 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:31:45.766 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.417383+0000", 2024-09-14T06:31:45.766 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047911092000000002 2024-09-14T06:31:45.766 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:45.766 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:45.766 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:31:45.766 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.417430+0000", 2024-09-14T06:31:45.766 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.6898999999999999e-05 2024-09-14T06:31:45.766 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:45.766 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:45.766 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:31:45.767 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.417497+0000", 2024-09-14T06:31:45.767 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.7317999999999995e-05 2024-09-14T06:31:45.767 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:45.767 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:31:45.767 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:45.767 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:45.767 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:45.767 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:53251 3.f 3:f71f51a7:::rbd_data.112dca2f6ae1.0000000000000015:17 [read 2035712~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:31:45.767 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:44.369475+0000", 2024-09-14T06:31:45.767 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 121.36452858299999, 2024-09-14T06:31:45.767 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.048154542000000002, 2024-09-14T06:31:45.767 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:31:45.767 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:31:45.767 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:31:45.767 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:31:45.768 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:31:45.768 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:45.768 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 53251 2024-09-14T06:31:45.768 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:45.768 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:31:45.768 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:45.768 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:31:45.768 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369475+0000", 2024-09-14T06:31:45.768 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:45.768 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:45.768 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:45.768 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:31:45.768 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369475+0000", 2024-09-14T06:31:45.768 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:45.769 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:45.769 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:45.769 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:31:45.769 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369475+0000", 2024-09-14T06:31:45.769 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.0100000000000001e-07 2024-09-14T06:31:45.769 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:45.769 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:45.769 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:31:45.769 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369476+0000", 2024-09-14T06:31:45.769 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.0580000000000001e-06 2024-09-14T06:31:45.769 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:45.769 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:45.769 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:31:45.769 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369479+0000", 2024-09-14T06:31:45.770 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.9950000000000001e-06 2024-09-14T06:31:45.770 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:45.770 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:45.770 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:31:45.770 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369482+0000", 2024-09-14T06:31:45.770 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.6340000000000001e-06 2024-09-14T06:31:45.770 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:45.770 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:45.770 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:31:45.770 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.417530+0000", 2024-09-14T06:31:45.770 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.048047535000000002 2024-09-14T06:31:45.770 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:45.770 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:45.770 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:31:45.771 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.417574+0000", 2024-09-14T06:31:45.771 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.4255999999999998e-05 2024-09-14T06:31:45.771 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:45.771 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:45.771 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:31:45.771 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.417630+0000", 2024-09-14T06:31:45.771 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.5862999999999997e-05 2024-09-14T06:31:45.771 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:45.771 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:31:45.771 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:45.771 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:45.771 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:45.771 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:53252 3.f 3:f71f51a7:::rbd_data.112dca2f6ae1.0000000000000015:17 [read 2060288~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:31:45.771 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:44.369485+0000", 2024-09-14T06:31:45.771 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 121.364518735, 2024-09-14T06:31:45.772 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.048254116, 2024-09-14T06:31:45.772 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:31:45.772 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:31:45.772 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:31:45.772 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:31:45.772 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:31:45.772 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:45.772 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 53252 2024-09-14T06:31:45.772 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:45.772 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:31:45.772 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:45.772 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:31:45.772 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369485+0000", 2024-09-14T06:31:45.772 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:45.773 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:45.773 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:45.773 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:31:45.773 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369485+0000", 2024-09-14T06:31:45.773 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:45.773 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:45.773 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:45.773 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:31:45.773 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369485+0000", 2024-09-14T06:31:45.773 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.1e-07 2024-09-14T06:31:45.773 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:45.773 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:45.773 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:31:45.773 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369487+0000", 2024-09-14T06:31:45.773 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.42e-06 2024-09-14T06:31:45.774 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:45.774 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:45.774 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:31:45.774 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369490+0000", 2024-09-14T06:31:45.774 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.9639999999999999e-06 2024-09-14T06:31:45.774 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:45.774 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:45.774 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:31:45.774 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369492+0000", 2024-09-14T06:31:45.774 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.5679999999999998e-06 2024-09-14T06:31:45.774 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:45.774 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:45.774 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:31:45.774 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.417645+0000", 2024-09-14T06:31:45.775 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.048152753 2024-09-14T06:31:45.775 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:45.775 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:45.775 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:31:45.775 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.417687+0000", 2024-09-14T06:31:45.775 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.1900999999999997e-05 2024-09-14T06:31:45.775 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:45.775 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:45.775 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:31:45.775 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.417739+0000", 2024-09-14T06:31:45.775 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.2299999999999997e-05 2024-09-14T06:31:45.775 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:45.775 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:31:45.775 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:45.776 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:45.776 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:45.776 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:53253 3.f 3:f71f51a7:::rbd_data.112dca2f6ae1.0000000000000015:17 [read 2084864~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:31:45.776 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:44.369495+0000", 2024-09-14T06:31:45.776 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 121.36450861100001, 2024-09-14T06:31:45.776 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.048366785000000002, 2024-09-14T06:31:45.776 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:31:45.776 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:31:45.776 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:31:45.776 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:31:45.776 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:31:45.776 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:45.776 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 53253 2024-09-14T06:31:45.776 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:45.777 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:31:45.777 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:45.777 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:31:45.777 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369495+0000", 2024-09-14T06:31:45.777 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:45.777 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:45.777 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:45.777 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:31:45.777 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369495+0000", 2024-09-14T06:31:45.777 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:45.777 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:45.777 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:45.777 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:31:45.778 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370388+0000", 2024-09-14T06:31:45.778 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00089287999999999998 2024-09-14T06:31:45.778 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:45.778 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:45.778 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:31:45.778 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370392+0000", 2024-09-14T06:31:45.778 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.2130000000000002e-06 2024-09-14T06:31:45.778 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:45.778 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:45.778 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:31:45.778 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370401+0000", 2024-09-14T06:31:45.778 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.7649999999999999e-06 2024-09-14T06:31:45.778 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:45.778 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:45.779 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:31:45.779 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370406+0000", 2024-09-14T06:31:45.779 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.0159999999999999e-06 2024-09-14T06:31:45.779 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:45.779 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:45.779 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:31:45.779 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.417764+0000", 2024-09-14T06:31:45.779 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047358348000000001 2024-09-14T06:31:45.779 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:45.779 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:45.779 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:31:45.779 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.417806+0000", 2024-09-14T06:31:45.779 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.1149999999999997e-05 2024-09-14T06:31:45.779 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:45.780 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:45.780 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:31:45.780 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.417862+0000", 2024-09-14T06:31:45.780 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.6412999999999997e-05 2024-09-14T06:31:45.780 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:45.780 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:31:45.780 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:45.780 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:45.780 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:45.780 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:53340 3.f 3:f71f51a7:::rbd_data.112dca2f6ae1.0000000000000015:17 [read 1056768~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:31:45.780 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:44.370421+0000", 2024-09-14T06:31:45.780 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 121.36358273099999, 2024-09-14T06:31:45.780 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047566313999999998, 2024-09-14T06:31:45.780 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:31:45.781 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:31:45.781 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:31:45.781 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:31:45.781 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:31:45.781 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:45.781 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 53340 2024-09-14T06:31:45.781 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:45.781 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:31:45.781 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:45.781 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:31:45.781 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370421+0000", 2024-09-14T06:31:45.781 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:45.781 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:45.781 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:45.782 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:31:45.782 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370421+0000", 2024-09-14T06:31:45.782 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:45.782 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:45.782 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:45.782 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:31:45.782 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370422+0000", 2024-09-14T06:31:45.782 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.3799999999999996e-07 2024-09-14T06:31:45.782 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:45.782 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:45.782 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:31:45.782 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370424+0000", 2024-09-14T06:31:45.782 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.6919999999999998e-06 2024-09-14T06:31:45.782 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:45.783 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:45.783 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:31:45.783 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370432+0000", 2024-09-14T06:31:45.783 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.2439999999999996e-06 2024-09-14T06:31:45.783 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:45.783 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:45.783 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:31:45.783 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370437+0000", 2024-09-14T06:31:45.783 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.5389999999999998e-06 2024-09-14T06:31:45.783 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:45.783 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:45.783 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:31:45.783 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.417888+0000", 2024-09-14T06:31:45.783 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047450432000000001 2024-09-14T06:31:45.784 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:45.784 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:45.784 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:31:45.784 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.417935+0000", 2024-09-14T06:31:45.784 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.7454999999999999e-05 2024-09-14T06:31:45.784 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:45.784 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:45.784 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:31:45.784 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.417987+0000", 2024-09-14T06:31:45.784 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.2213999999999998e-05 2024-09-14T06:31:45.784 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:45.784 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:31:45.784 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:45.785 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:45.785 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:45.785 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:53341 3.f 3:f71f51a7:::rbd_data.112dca2f6ae1.0000000000000015:17 [read 1081344~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:31:45.785 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:44.370446+0000", 2024-09-14T06:31:45.785 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 121.363557458, 2024-09-14T06:31:45.785 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047665723, 2024-09-14T06:31:45.785 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:31:45.785 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:31:45.785 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:31:45.785 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:31:45.785 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:31:45.785 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:45.785 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 53341 2024-09-14T06:31:45.785 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:45.786 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:31:45.786 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:45.786 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:31:45.786 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370446+0000", 2024-09-14T06:31:45.786 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:45.786 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:45.786 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:45.786 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:31:45.786 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370446+0000", 2024-09-14T06:31:45.786 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:45.786 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:45.786 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:45.786 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:31:45.786 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370447+0000", 2024-09-14T06:31:45.787 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.6100000000000002e-07 2024-09-14T06:31:45.787 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:45.787 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:45.787 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:31:45.787 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370448+0000", 2024-09-14T06:31:45.787 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.7370000000000001e-06 2024-09-14T06:31:45.787 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:45.787 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:45.787 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:31:45.787 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370462+0000", 2024-09-14T06:31:45.787 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.3227000000000001e-05 2024-09-14T06:31:45.787 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:45.787 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:45.787 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:31:45.788 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370467+0000", 2024-09-14T06:31:45.788 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.6540000000000004e-06 2024-09-14T06:31:45.788 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:45.788 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:45.788 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:31:45.788 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.418013+0000", 2024-09-14T06:31:45.788 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047545325999999999 2024-09-14T06:31:45.788 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:45.788 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:45.788 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:31:45.788 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.418054+0000", 2024-09-14T06:31:45.788 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.1696999999999997e-05 2024-09-14T06:31:45.788 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:45.788 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:45.789 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:31:45.789 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.418112+0000", 2024-09-14T06:31:45.789 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.7720999999999998e-05 2024-09-14T06:31:45.789 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:45.789 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:31:45.789 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:45.789 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:45.789 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:45.789 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:53342 3.f 3:f71f51a7:::rbd_data.112dca2f6ae1.0000000000000015:17 [read 1105920~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:31:45.789 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:44.370475+0000", 2024-09-14T06:31:45.789 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 121.363528397, 2024-09-14T06:31:45.789 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047748848000000003, 2024-09-14T06:31:45.789 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:31:45.789 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:31:45.790 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:31:45.790 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:31:45.790 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:31:45.790 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:45.790 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 53342 2024-09-14T06:31:45.790 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:45.790 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:31:45.790 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:45.790 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:31:45.790 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370475+0000", 2024-09-14T06:31:45.790 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:45.790 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:45.790 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:45.790 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:31:45.790 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370475+0000", 2024-09-14T06:31:45.791 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:45.791 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:45.791 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:45.791 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:31:45.791 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370476+0000", 2024-09-14T06:31:45.791 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.4700000000000002e-07 2024-09-14T06:31:45.791 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:45.791 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:45.791 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:31:45.791 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370478+0000", 2024-09-14T06:31:45.791 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.5890000000000001e-06 2024-09-14T06:31:45.791 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:45.791 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:45.792 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:31:45.792 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370486+0000", 2024-09-14T06:31:45.792 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.3470000000000001e-06 2024-09-14T06:31:45.792 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:45.792 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:45.792 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:31:45.792 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370491+0000", 2024-09-14T06:31:45.792 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.5720000000000002e-06 2024-09-14T06:31:45.792 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:45.792 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:45.792 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:31:45.792 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.418135+0000", 2024-09-14T06:31:45.792 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047644094999999997 2024-09-14T06:31:45.792 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:45.793 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:45.793 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:31:45.793 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.418174+0000", 2024-09-14T06:31:45.793 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.8204000000000002e-05 2024-09-14T06:31:45.793 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:45.793 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:45.793 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:31:45.793 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.418224+0000", 2024-09-14T06:31:45.793 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.0593999999999997e-05 2024-09-14T06:31:45.793 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:45.793 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:31:45.793 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:45.793 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:45.793 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:45.793 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:53343 3.f 3:f71f51a7:::rbd_data.112dca2f6ae1.0000000000000015:17 [read 1130496~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:31:45.794 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:44.370498+0000", 2024-09-14T06:31:45.794 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 121.36350552099999, 2024-09-14T06:31:45.794 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047841574999999997, 2024-09-14T06:31:45.794 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:31:45.794 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:31:45.794 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:31:45.794 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:31:45.794 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:31:45.794 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:45.794 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 53343 2024-09-14T06:31:45.794 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:45.794 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:31:45.794 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:45.794 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:31:45.795 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370498+0000", 2024-09-14T06:31:45.795 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:45.795 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:45.795 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:45.795 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:31:45.795 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370498+0000", 2024-09-14T06:31:45.795 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:45.795 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:45.795 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:45.795 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:31:45.795 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370499+0000", 2024-09-14T06:31:45.795 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.9999999999999998e-07 2024-09-14T06:31:45.795 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:45.795 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:45.796 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:31:45.796 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370501+0000", 2024-09-14T06:31:45.796 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.0310000000000001e-06 2024-09-14T06:31:45.796 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:45.796 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:45.796 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:31:45.796 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370507+0000", 2024-09-14T06:31:45.796 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.4169999999999997e-06 2024-09-14T06:31:45.796 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:45.796 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:45.796 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:31:45.796 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370516+0000", 2024-09-14T06:31:45.796 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.7190000000000007e-06 2024-09-14T06:31:45.796 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:45.797 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:45.797 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:31:45.797 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.418247+0000", 2024-09-14T06:31:45.797 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047731190999999999 2024-09-14T06:31:45.797 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:45.797 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:45.797 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:31:45.797 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.418286+0000", 2024-09-14T06:31:45.797 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.8988e-05 2024-09-14T06:31:45.797 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:45.797 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:45.797 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:31:45.797 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.418340+0000", 2024-09-14T06:31:45.797 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.3829000000000002e-05 2024-09-14T06:31:45.798 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:45.798 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:31:45.798 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:45.798 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:45.798 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:45.798 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:53344 3.f 3:f71f51a7:::rbd_data.112dca2f6ae1.0000000000000015:17 [read 1155072~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:31:45.798 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:44.370524+0000", 2024-09-14T06:31:45.798 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 121.363479751, 2024-09-14T06:31:45.798 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047972262000000002, 2024-09-14T06:31:45.798 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:31:45.798 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:31:45.798 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:31:45.798 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:31:45.798 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:31:45.798 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:45.799 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 53344 2024-09-14T06:31:45.799 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:45.799 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:31:45.799 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:45.799 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:31:45.799 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370524+0000", 2024-09-14T06:31:45.799 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:45.799 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:45.799 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:45.799 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:31:45.799 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370524+0000", 2024-09-14T06:31:45.799 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:45.799 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:45.800 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:45.800 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:31:45.800 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370618+0000", 2024-09-14T06:31:45.800 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.4425999999999993e-05 2024-09-14T06:31:45.800 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:45.800 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:45.800 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:31:45.800 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370622+0000", 2024-09-14T06:31:45.800 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.3519999999999998e-06 2024-09-14T06:31:45.800 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:45.800 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:45.800 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:31:45.800 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370631+0000", 2024-09-14T06:31:45.800 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.9299999999999992e-06 2024-09-14T06:31:45.800 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:45.801 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:45.801 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:31:45.801 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370637+0000", 2024-09-14T06:31:45.801 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.3269999999999997e-06 2024-09-14T06:31:45.801 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:45.801 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:45.801 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:31:45.801 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.418372+0000", 2024-09-14T06:31:45.801 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047734764999999998 2024-09-14T06:31:45.801 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:45.801 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:45.801 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:31:45.801 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.418431+0000", 2024-09-14T06:31:45.802 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.8696999999999997e-05 2024-09-14T06:31:45.802 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:45.802 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:45.802 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:31:45.802 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.418496+0000", 2024-09-14T06:31:45.802 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.5765000000000005e-05 2024-09-14T06:31:45.802 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:45.802 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:31:45.802 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:45.802 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:45.802 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:45.802 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:53345 3.f 3:f71f51a7:::rbd_data.112dca2f6ae1.0000000000000015:17 [read 1179648~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:31:45.802 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:44.370647+0000", 2024-09-14T06:31:45.802 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 121.36335628400001, 2024-09-14T06:31:45.802 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047953775999999997, 2024-09-14T06:31:45.803 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:31:45.803 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:31:45.803 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:31:45.803 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:31:45.803 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:31:45.803 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:45.803 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 53345 2024-09-14T06:31:45.803 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:45.803 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:31:45.803 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:45.803 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:31:45.803 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370647+0000", 2024-09-14T06:31:45.803 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:45.803 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:45.804 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:45.804 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:31:45.804 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370647+0000", 2024-09-14T06:31:45.804 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:45.804 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:45.804 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:45.804 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:31:45.804 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.372346+0000", 2024-09-14T06:31:45.804 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.001698284 2024-09-14T06:31:45.804 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:45.804 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:45.804 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:31:45.804 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.372351+0000", 2024-09-14T06:31:45.805 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.8890000000000001e-06 2024-09-14T06:31:45.805 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:45.805 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:45.805 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:31:45.805 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.372362+0000", 2024-09-14T06:31:45.805 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.0941000000000001e-05 2024-09-14T06:31:45.805 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:45.805 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:45.805 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:31:45.805 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.372369+0000", 2024-09-14T06:31:45.805 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.362e-06 2024-09-14T06:31:45.805 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:45.805 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:45.805 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:31:45.805 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.418510+0000", 2024-09-14T06:31:45.806 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046140914999999998 2024-09-14T06:31:45.806 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:45.806 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:45.806 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:31:45.806 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.418549+0000", 2024-09-14T06:31:45.806 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.8717999999999998e-05 2024-09-14T06:31:45.806 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:45.806 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:45.806 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:31:45.806 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.418601+0000", 2024-09-14T06:31:45.806 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.2667000000000002e-05 2024-09-14T06:31:45.806 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:45.806 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:31:45.806 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:45.807 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:45.807 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:45.807 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:53351 3.f 3:f71f51a7:::rbd_data.112dca2f6ae1.0000000000000015:17 [read 1327104~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:31:45.807 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:44.372499+0000", 2024-09-14T06:31:45.807 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 121.361505163, 2024-09-14T06:31:45.807 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047140596, 2024-09-14T06:31:45.807 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:31:45.807 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:31:45.807 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:31:45.807 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:31:45.807 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:31:45.807 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:45.807 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 53351 2024-09-14T06:31:45.807 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:45.808 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:31:45.808 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:45.808 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:31:45.808 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.372499+0000", 2024-09-14T06:31:45.808 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:45.808 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:45.808 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:45.808 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:31:45.808 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.372499+0000", 2024-09-14T06:31:45.808 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:45.808 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:45.808 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:45.808 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:31:45.809 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.372499+0000", 2024-09-14T06:31:45.809 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.3000000000000001e-07 2024-09-14T06:31:45.809 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:45.809 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:45.809 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:31:45.809 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.372501+0000", 2024-09-14T06:31:45.809 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.0820000000000001e-06 2024-09-14T06:31:45.809 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:45.809 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:45.809 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:31:45.809 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.372513+0000", 2024-09-14T06:31:45.809 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.1610999999999999e-05 2024-09-14T06:31:45.809 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:45.809 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:45.810 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:31:45.810 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.372518+0000", 2024-09-14T06:31:45.810 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.327e-06 2024-09-14T06:31:45.810 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:45.810 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:45.810 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:31:45.810 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.419530+0000", 2024-09-14T06:31:45.810 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047011798 2024-09-14T06:31:45.810 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:45.810 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:45.810 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:31:45.810 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.419578+0000", 2024-09-14T06:31:45.810 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.8446999999999999e-05 2024-09-14T06:31:45.810 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:45.811 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:45.811 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:31:45.811 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.419639+0000", 2024-09-14T06:31:45.811 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.0900999999999998e-05 2024-09-14T06:31:45.811 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:45.811 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:31:45.811 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:45.811 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:45.811 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:45.811 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:53352 3.f 3:f71f51a7:::rbd_data.112dca2f6ae1.0000000000000015:17 [read 1351680~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:31:45.811 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:44.372525+0000", 2024-09-14T06:31:45.811 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 121.361479249, 2024-09-14T06:31:45.811 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047259847000000001, 2024-09-14T06:31:45.811 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:31:45.812 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:31:45.812 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:31:45.812 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:31:45.812 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:31:45.812 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:45.812 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 53352 2024-09-14T06:31:45.812 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:45.812 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:31:45.812 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:45.812 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:31:45.812 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.372525+0000", 2024-09-14T06:31:45.812 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:45.812 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:45.812 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:45.813 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:31:45.813 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.372525+0000", 2024-09-14T06:31:45.813 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:45.813 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:45.813 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:45.813 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:31:45.813 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.372525+0000", 2024-09-14T06:31:45.813 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.0600000000000001e-07 2024-09-14T06:31:45.813 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:45.813 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:45.813 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:31:45.813 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.372527+0000", 2024-09-14T06:31:45.813 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.0499999999999999e-06 2024-09-14T06:31:45.813 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:45.813 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:45.814 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:31:45.814 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.372533+0000", 2024-09-14T06:31:45.814 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.4320000000000004e-06 2024-09-14T06:31:45.814 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:45.814 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:45.814 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:31:45.814 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.372538+0000", 2024-09-14T06:31:45.814 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.9130000000000002e-06 2024-09-14T06:31:45.814 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:45.814 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:45.814 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:31:45.814 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.419667+0000", 2024-09-14T06:31:45.814 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047128976000000003 2024-09-14T06:31:45.814 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:45.815 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:45.815 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:31:45.815 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.419715+0000", 2024-09-14T06:31:45.815 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.7868999999999999e-05 2024-09-14T06:31:45.815 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:45.815 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:45.815 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:31:45.815 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.419784+0000", 2024-09-14T06:31:45.815 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.9201000000000004e-05 2024-09-14T06:31:45.815 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:45.815 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:31:45.815 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:45.815 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:45.815 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:45.816 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:53353 3.f 3:f71f51a7:::rbd_data.112dca2f6ae1.0000000000000015:17 [read 1376256~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:31:45.816 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:44.372545+0000", 2024-09-14T06:31:45.816 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 121.361459209, 2024-09-14T06:31:45.816 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047364232999999999, 2024-09-14T06:31:45.816 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:31:45.816 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:31:45.816 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:31:45.816 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:31:45.816 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:31:45.816 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:45.816 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 53353 2024-09-14T06:31:45.816 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:45.816 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:31:45.816 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:45.817 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:31:45.817 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.372545+0000", 2024-09-14T06:31:45.817 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:45.817 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:45.817 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:45.817 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:31:45.817 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.372545+0000", 2024-09-14T06:31:45.817 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:45.817 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:45.817 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:45.817 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:31:45.817 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.372545+0000", 2024-09-14T06:31:45.817 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.27e-07 2024-09-14T06:31:45.817 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:45.818 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:45.818 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:31:45.818 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.372547+0000", 2024-09-14T06:31:45.818 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.0600000000000002e-06 2024-09-14T06:31:45.818 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:45.818 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:45.818 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:31:45.818 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.372553+0000", 2024-09-14T06:31:45.818 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.7980000000000002e-06 2024-09-14T06:31:45.818 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:45.818 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:45.818 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:31:45.818 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.372557+0000", 2024-09-14T06:31:45.818 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.7389999999999999e-06 2024-09-14T06:31:45.818 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:45.819 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:45.819 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:31:45.819 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.419811+0000", 2024-09-14T06:31:45.819 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047253310999999999 2024-09-14T06:31:45.819 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:45.819 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:45.819 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:31:45.819 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.419854+0000", 2024-09-14T06:31:45.819 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.3448000000000001e-05 2024-09-14T06:31:45.819 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:45.819 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:45.819 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:31:45.819 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.419909+0000", 2024-09-14T06:31:45.819 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.4549999999999998e-05 2024-09-14T06:31:45.820 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:45.820 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:31:45.820 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:45.820 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:45.820 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:45.820 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:53354 3.f 3:f71f51a7:::rbd_data.112dca2f6ae1.0000000000000015:17 [read 1400832~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:31:45.820 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:44.372563+0000", 2024-09-14T06:31:45.820 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 121.36144077199999, 2024-09-14T06:31:45.820 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047482233999999998, 2024-09-14T06:31:45.820 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:31:45.820 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:31:45.820 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:31:45.820 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:31:45.820 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:31:45.821 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:45.821 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 53354 2024-09-14T06:31:45.821 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:45.821 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:31:45.821 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:45.821 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:31:45.821 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.372563+0000", 2024-09-14T06:31:45.821 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:45.821 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:45.821 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:45.821 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:31:45.821 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.372563+0000", 2024-09-14T06:31:45.821 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:45.821 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:45.822 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:45.822 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:31:45.822 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.375009+0000", 2024-09-14T06:31:45.822 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.0024457680000000001 2024-09-14T06:31:45.822 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:45.822 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:45.822 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:31:45.822 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.375014+0000", 2024-09-14T06:31:45.822 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.0599999999999998e-06 2024-09-14T06:31:45.822 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:45.822 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:45.822 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:31:45.822 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.375026+0000", 2024-09-14T06:31:45.822 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.1783999999999999e-05 2024-09-14T06:31:45.823 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:45.823 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:45.823 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:31:45.823 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.375035+0000", 2024-09-14T06:31:45.823 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.7799999999999995e-06 2024-09-14T06:31:45.823 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:45.823 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:45.823 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:31:45.823 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.419942+0000", 2024-09-14T06:31:45.823 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.044906982999999998 2024-09-14T06:31:45.823 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:45.823 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:45.823 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:31:45.823 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.419989+0000", 2024-09-14T06:31:45.823 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.7104999999999997e-05 2024-09-14T06:31:45.824 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:45.824 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:45.824 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:31:45.824 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.420045+0000", 2024-09-14T06:31:45.824 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.5754000000000002e-05 2024-09-14T06:31:45.824 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:45.824 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:31:45.824 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:45.824 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:45.824 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:31:45.824 INFO:teuthology.orchestra.run.smithi138.stdout:} 2024-09-14T06:31:45.825 INFO:teuthology.orchestra.run.smithi094.stdout:osd.7: {} 2024-09-14T06:31:45.826 INFO:tasks.ceph.osd.5.smithi138.stderr:2024-09-14T06:31:45.824+0000 7fe13746f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33618) UID: 0 2024-09-14T06:31:45.827 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-14T06:31:45.827 INFO:teuthology.orchestra.run.smithi094.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-14T06:31:45.926 INFO:tasks.ceph.osd.5.smithi138.stderr:2024-09-14T06:31:45.925+0000 7fe13746f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33618) UID: 0 2024-09-14T06:31:45.972 INFO:teuthology.orchestra.run.smithi094.stdout:ERROR: (22) Invalid argument 2024-09-14T06:31:45.972 INFO:teuthology.orchestra.run.smithi094.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:31:45.981 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-14T06:31:46.026 INFO:tasks.ceph.osd.2.smithi094.stderr:2024-09-14T06:31:46.025+0000 7faf3269b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33649) UID: 0 2024-09-14T06:31:46.083 INFO:teuthology.orchestra.run.smithi094.stdout:ERROR: (22) Invalid argument 2024-09-14T06:31:46.083 INFO:teuthology.orchestra.run.smithi094.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:31:46.092 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-14T06:31:46.127 INFO:tasks.ceph.osd.5.smithi138.stderr:2024-09-14T06:31:46.126+0000 7fe13746f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33618) UID: 0 2024-09-14T06:31:46.228 INFO:tasks.ceph.osd.1.smithi094.stderr:2024-09-14T06:31:46.227+0000 7f328e47a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33641) UID: 0 2024-09-14T06:31:46.236 INFO:teuthology.orchestra.run.smithi094.stdout:ERROR: (22) Invalid argument 2024-09-14T06:31:46.236 INFO:teuthology.orchestra.run.smithi094.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:31:46.245 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-14T06:31:46.329 INFO:tasks.ceph.osd.2.smithi094.stderr:2024-09-14T06:31:46.328+0000 7faf3269b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33649) UID: 0 2024-09-14T06:31:46.389 INFO:teuthology.orchestra.run.smithi094.stdout:ERROR: (22) Invalid argument 2024-09-14T06:31:46.389 INFO:teuthology.orchestra.run.smithi094.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:31:46.397 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-14T06:31:46.438 INFO:tasks.ceph.osd.1.smithi094.stderr:2024-09-14T06:31:46.437+0000 7f328e47a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33641) UID: 0 2024-09-14T06:31:46.530 INFO:tasks.ceph.osd.7.smithi138.stderr:2024-09-14T06:31:46.529+0000 7f3c6a21d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33623) UID: 0 2024-09-14T06:31:46.541 INFO:teuthology.orchestra.run.smithi094.stdout:ERROR: (22) Invalid argument 2024-09-14T06:31:46.541 INFO:teuthology.orchestra.run.smithi094.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:31:46.550 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-14T06:31:46.631 INFO:tasks.ceph.osd.1.smithi094.stderr:2024-09-14T06:31:46.630+0000 7f328e47a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33641) UID: 0 2024-09-14T06:31:46.653 INFO:teuthology.orchestra.run.smithi094.stdout:ERROR: (22) Invalid argument 2024-09-14T06:31:46.653 INFO:teuthology.orchestra.run.smithi094.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:31:46.661 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-14T06:31:46.745 INFO:tasks.ceph.osd.2.smithi094.stderr:2024-09-14T06:31:46.744+0000 7faf3269b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33649) UID: 0 2024-09-14T06:31:46.833 INFO:teuthology.orchestra.run.smithi094.stdout:ERROR: (22) Invalid argument 2024-09-14T06:31:46.833 INFO:teuthology.orchestra.run.smithi094.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:31:46.833 INFO:tasks.ceph.osd.3.smithi094.stderr:2024-09-14T06:31:46.831+0000 7feeeafe8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33655) UID: 0 2024-09-14T06:31:46.842 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-14T06:31:46.934 INFO:tasks.ceph.osd.5.smithi138.stderr:2024-09-14T06:31:46.933+0000 7fe13746f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33618) UID: 0 2024-09-14T06:31:46.985 INFO:teuthology.orchestra.run.smithi094.stdout:ERROR: (22) Invalid argument 2024-09-14T06:31:46.985 INFO:teuthology.orchestra.run.smithi094.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:31:46.994 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-14T06:31:47.039 INFO:tasks.ceph.osd.1.smithi094.stderr:2024-09-14T06:31:47.038+0000 7f328e47a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33641) UID: 0 2024-09-14T06:31:47.096 INFO:teuthology.orchestra.run.smithi094.stdout:ERROR: (22) Invalid argument 2024-09-14T06:31:47.097 INFO:teuthology.orchestra.run.smithi094.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:31:47.105 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-14T06:31:47.146 INFO:tasks.ceph.osd.3.smithi094.stderr:2024-09-14T06:31:47.145+0000 7feeeafe8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33655) UID: 0 2024-09-14T06:31:47.237 INFO:tasks.ceph.osd.5.smithi138.stderr:2024-09-14T06:31:47.236+0000 7fe13746f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33618) UID: 0 2024-09-14T06:31:47.248 INFO:teuthology.orchestra.run.smithi094.stdout:ERROR: (22) Invalid argument 2024-09-14T06:31:47.249 INFO:teuthology.orchestra.run.smithi094.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:31:47.257 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-14T06:31:47.338 INFO:tasks.ceph.osd.5.smithi138.stderr:2024-09-14T06:31:47.337+0000 7fe13746f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33618) UID: 0 2024-09-14T06:31:47.359 INFO:teuthology.orchestra.run.smithi094.stdout:ERROR: (22) Invalid argument 2024-09-14T06:31:47.359 INFO:teuthology.orchestra.run.smithi094.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:31:47.368 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-14T06:31:47.439 INFO:tasks.ceph.osd.6.smithi138.stderr:2024-09-14T06:31:47.438+0000 7f8cd8849640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33620) UID: 0 2024-09-14T06:31:47.471 INFO:teuthology.orchestra.run.smithi094.stdout:ERROR: (22) Invalid argument 2024-09-14T06:31:47.471 INFO:teuthology.orchestra.run.smithi094.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:31:47.479 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-14T06:31:47.539 INFO:tasks.ceph.osd.2.smithi094.stderr:2024-09-14T06:31:47.538+0000 7faf3269b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33649) UID: 0 2024-09-14T06:31:47.593 INFO:teuthology.orchestra.run.smithi138.stdout:ERROR: (22) Invalid argument 2024-09-14T06:31:47.594 INFO:teuthology.orchestra.run.smithi138.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:31:47.602 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-14T06:31:47.640 INFO:tasks.ceph.osd.3.smithi094.stderr:2024-09-14T06:31:47.639+0000 7feeeafe8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33655) UID: 0 2024-09-14T06:31:47.706 INFO:teuthology.orchestra.run.smithi138.stdout:ERROR: (22) Invalid argument 2024-09-14T06:31:47.706 INFO:teuthology.orchestra.run.smithi138.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:31:47.715 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-14T06:31:47.723 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-14T06:31:47.768 INFO:tasks.ceph.osd.2.smithi094.stderr:2024-09-14T06:31:47.767+0000 7faf3269b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33649) UID: 0 2024-09-14T06:31:47.822 INFO:teuthology.orchestra.run.smithi138.stdout:ERROR: (22) Invalid argument 2024-09-14T06:31:47.822 INFO:teuthology.orchestra.run.smithi138.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:31:47.828 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-14T06:31:47.831 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-14T06:31:47.850 INFO:teuthology.orchestra.run.smithi094.stdout:checking clone #39, image image_client.0-clone39 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent40 2024-09-14T06:31:47.874 INFO:tasks.ceph.osd.6.smithi138.stderr:2024-09-14T06:31:47.873+0000 7f8cd8849640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33620) UID: 0 2024-09-14T06:31:47.942 INFO:tasks.ceph.osd.3.smithi094.stderr:2024-09-14T06:31:47.941+0000 7feeeafe8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33655) UID: 0 2024-09-14T06:31:47.944 INFO:teuthology.orchestra.run.smithi138.stdout:ERROR: (22) Invalid argument 2024-09-14T06:31:47.944 INFO:teuthology.orchestra.run.smithi138.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:31:47.953 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-14T06:31:47.978 INFO:teuthology.orchestra.run.smithi094.stdout:osd.0: {} 2024-09-14T06:31:47.978 INFO:teuthology.orchestra.run.smithi094.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-14T06:31:47.995 INFO:teuthology.orchestra.run.smithi094.stdout:osd.1: {} 2024-09-14T06:31:47.995 INFO:teuthology.orchestra.run.smithi094.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-14T06:31:48.011 INFO:teuthology.orchestra.run.smithi094.stdout:osd.2: {} 2024-09-14T06:31:48.011 INFO:teuthology.orchestra.run.smithi094.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-14T06:31:48.027 INFO:teuthology.orchestra.run.smithi094.stdout:osd.3: {} 2024-09-14T06:31:48.028 INFO:teuthology.orchestra.run.smithi094.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-14T06:31:48.043 INFO:tasks.ceph.osd.6.smithi138.stderr:2024-09-14T06:31:48.042+0000 7f8cd8849640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33620) UID: 0 2024-09-14T06:31:48.044 INFO:teuthology.orchestra.run.smithi094.stdout:osd.4: {} 2024-09-14T06:31:48.045 INFO:teuthology.orchestra.run.smithi094.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-14T06:31:48.061 INFO:teuthology.orchestra.run.smithi094.stdout:osd.5: {} 2024-09-14T06:31:48.062 INFO:teuthology.orchestra.run.smithi094.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-14T06:31:48.078 INFO:teuthology.orchestra.run.smithi094.stdout:osd.6: {} 2024-09-14T06:31:48.079 INFO:teuthology.orchestra.run.smithi094.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-14T06:31:48.095 INFO:teuthology.orchestra.run.smithi094.stdout:osd.7: {} 2024-09-14T06:31:48.095 INFO:teuthology.orchestra.run.smithi094.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-14T06:31:48.097 INFO:teuthology.orchestra.run.smithi138.stdout:{ 2024-09-14T06:31:48.097 INFO:teuthology.orchestra.run.smithi138.stdout: "ops": [], 2024-09-14T06:31:48.097 INFO:teuthology.orchestra.run.smithi138.stdout: "complaint_time": 30, 2024-09-14T06:31:48.098 INFO:teuthology.orchestra.run.smithi138.stdout: "num_blocked_ops": 0 2024-09-14T06:31:48.098 INFO:teuthology.orchestra.run.smithi138.stdout:} 2024-09-14T06:31:48.107 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-14T06:31:48.141 INFO:teuthology.orchestra.run.smithi094.stdout:5680 write 0x40d712c thru 0x40df414 (0x82e9 bytes) 2024-09-14T06:31:48.143 INFO:tasks.ceph.osd.0.smithi094.stderr:2024-09-14T06:31:48.142+0000 7f07c9333640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33636) UID: 0 2024-09-14T06:31:48.150 INFO:teuthology.orchestra.run.smithi094.stdout:5681 punch from 0x8e14bcd to 0x8e18b95, (0x3fc8 bytes) 2024-09-14T06:31:48.150 INFO:teuthology.orchestra.run.smithi094.stdout:5683 write 0x485ce54 thru 0x485fa87 (0x2c34 bytes) 2024-09-14T06:31:48.152 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-14T06:31:48.152 INFO:teuthology.orchestra.run.smithi094.stdout:5685 write 0x214bdc5 thru 0x215a64f (0xe88b bytes) 2024-09-14T06:31:48.158 INFO:teuthology.orchestra.run.smithi094.stdout:5686 writesame 0x2be5915 thru 0x2bf21d6 data_size 0x7(0xc8c2 bytes) 2024-09-14T06:31:48.162 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-14T06:31:48.162 INFO:teuthology.orchestra.run.smithi094.stdout:5690 read 0xafa54f thru 0xb01d2f (0x77e1 bytes) 2024-09-14T06:31:48.163 INFO:teuthology.orchestra.run.smithi094.stdout:5691 clone 42 order 19 su 8192 sc 3 2024-09-14T06:31:48.245 INFO:tasks.ceph.osd.1.smithi094.stderr:2024-09-14T06:31:48.244+0000 7f328e47a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33641) UID: 0 2024-09-14T06:31:48.252 INFO:teuthology.orchestra.run.smithi138.stdout:{ 2024-09-14T06:31:48.252 INFO:teuthology.orchestra.run.smithi138.stdout: "size": 20, 2024-09-14T06:31:48.252 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 600, 2024-09-14T06:31:48.252 INFO:teuthology.orchestra.run.smithi138.stdout: "ops": [ 2024-09-14T06:31:48.252 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:48.252 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:11453 3.1d 3:bcad7303:::rbd_data.112d92027590.0000000000000001:f [read 94208~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:31:48.252 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:25.654375+0000", 2024-09-14T06:31:48.252 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 142.59519659200001, 2024-09-14T06:31:48.252 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052449318000000002, 2024-09-14T06:31:48.253 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:31:48.253 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:31:48.253 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:31:48.253 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:31:48.253 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:31:48.253 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:48.254 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 11453 2024-09-14T06:31:48.254 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:48.254 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:31:48.254 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:48.254 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:31:48.254 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654375+0000", 2024-09-14T06:31:48.254 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:48.254 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:48.255 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:48.255 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:31:48.255 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654375+0000", 2024-09-14T06:31:48.255 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:48.255 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:48.255 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:48.255 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:31:48.255 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654496+0000", 2024-09-14T06:31:48.256 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00012081 2024-09-14T06:31:48.256 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:48.256 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:48.256 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:31:48.256 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654497+0000", 2024-09-14T06:31:48.256 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.8789999999999999e-06 2024-09-14T06:31:48.257 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:48.257 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:48.257 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:31:48.257 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654501+0000", 2024-09-14T06:31:48.257 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.8999999999999999e-06 2024-09-14T06:31:48.258 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:48.258 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:48.258 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:31:48.258 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654505+0000", 2024-09-14T06:31:48.258 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.2229999999999999e-06 2024-09-14T06:31:48.259 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:48.259 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:48.259 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:31:48.259 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.705917+0000", 2024-09-14T06:31:48.259 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.051412501999999999 2024-09-14T06:31:48.260 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:48.260 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:48.260 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:31:48.260 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.705999+0000", 2024-09-14T06:31:48.260 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.1551999999999994e-05 2024-09-14T06:31:48.260 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:48.260 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:48.260 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:31:48.261 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.706824+0000", 2024-09-14T06:31:48.261 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00082545199999999996 2024-09-14T06:31:48.261 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:48.261 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:31:48.261 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:48.261 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:48.261 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:48.261 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:11454 3.1d 3:bcad7303:::rbd_data.112d92027590.0000000000000001:f [read 104448~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:31:48.264 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:25.654510+0000", 2024-09-14T06:31:48.264 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 142.595061802, 2024-09-14T06:31:48.264 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052610713000000003, 2024-09-14T06:31:48.264 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:31:48.264 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:31:48.264 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:31:48.264 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:31:48.265 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:31:48.265 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:48.265 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 11454 2024-09-14T06:31:48.265 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:48.265 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:31:48.265 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:48.265 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:31:48.266 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654510+0000", 2024-09-14T06:31:48.266 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:48.266 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:48.266 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:48.266 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:31:48.266 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654510+0000", 2024-09-14T06:31:48.267 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:48.267 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:48.267 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:48.267 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:31:48.267 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654825+0000", 2024-09-14T06:31:48.267 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00031522400000000002 2024-09-14T06:31:48.268 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:48.268 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:48.268 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:31:48.268 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654827+0000", 2024-09-14T06:31:48.268 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.9709999999999998e-06 2024-09-14T06:31:48.268 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:48.269 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:48.269 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:31:48.269 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654832+0000", 2024-09-14T06:31:48.269 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.4099999999999999e-06 2024-09-14T06:31:48.269 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:48.269 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:48.270 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:31:48.270 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654835+0000", 2024-09-14T06:31:48.270 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.14e-06 2024-09-14T06:31:48.270 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:48.270 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:48.270 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:31:48.271 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.706851+0000", 2024-09-14T06:31:48.271 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052015761000000001 2024-09-14T06:31:48.271 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:48.271 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:48.271 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:31:48.271 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.706934+0000", 2024-09-14T06:31:48.271 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.2484999999999994e-05 2024-09-14T06:31:48.271 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:48.271 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:48.271 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:31:48.271 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.707120+0000", 2024-09-14T06:31:48.272 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00018672200000000001 2024-09-14T06:31:48.272 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:48.272 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:31:48.272 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:48.272 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:48.272 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:48.272 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:11455 3.1d 3:bcad7303:::rbd_data.112d92027590.0000000000000001:f [read 114688~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:31:48.272 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:25.654841+0000", 2024-09-14T06:31:48.272 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 142.594730473, 2024-09-14T06:31:48.272 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052579898, 2024-09-14T06:31:48.272 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:31:48.272 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:31:48.272 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:31:48.273 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:31:48.273 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:31:48.273 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:48.273 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 11455 2024-09-14T06:31:48.273 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:48.273 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:31:48.273 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:48.273 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:31:48.273 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654841+0000", 2024-09-14T06:31:48.273 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:48.273 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:48.273 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:48.273 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:31:48.273 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654841+0000", 2024-09-14T06:31:48.274 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:48.274 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:48.274 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:48.274 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:31:48.274 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654841+0000", 2024-09-14T06:31:48.274 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.7300000000000002e-07 2024-09-14T06:31:48.274 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:48.274 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:48.274 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:31:48.274 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654845+0000", 2024-09-14T06:31:48.274 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.0520000000000003e-06 2024-09-14T06:31:48.274 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:48.274 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:48.274 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:31:48.274 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654849+0000", 2024-09-14T06:31:48.275 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.0459999999999999e-06 2024-09-14T06:31:48.275 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:48.275 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:48.275 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:31:48.275 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654852+0000", 2024-09-14T06:31:48.275 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.9830000000000001e-06 2024-09-14T06:31:48.275 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:48.275 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:48.275 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:31:48.275 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.707159+0000", 2024-09-14T06:31:48.275 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052306826000000001 2024-09-14T06:31:48.275 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:48.275 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:48.275 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:31:48.276 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.707242+0000", 2024-09-14T06:31:48.276 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.2595000000000005e-05 2024-09-14T06:31:48.276 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:48.276 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:48.276 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:31:48.276 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.707421+0000", 2024-09-14T06:31:48.276 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.000179123 2024-09-14T06:31:48.276 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:48.276 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:31:48.276 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:48.276 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:48.277 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:48.277 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:11456 3.1d 3:bcad7303:::rbd_data.112d92027590.0000000000000001:f [read 124928~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:31:48.277 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:25.654856+0000", 2024-09-14T06:31:48.277 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 142.594715627, 2024-09-14T06:31:48.277 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052801580000000001, 2024-09-14T06:31:48.277 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:31:48.277 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:31:48.277 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:31:48.277 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:31:48.277 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:31:48.277 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:48.277 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 11456 2024-09-14T06:31:48.278 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:48.278 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:31:48.278 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:48.278 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:31:48.278 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654856+0000", 2024-09-14T06:31:48.278 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:48.278 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:48.278 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:48.278 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:31:48.278 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654856+0000", 2024-09-14T06:31:48.278 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:48.279 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:48.279 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:48.279 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:31:48.279 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654856+0000", 2024-09-14T06:31:48.279 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.29e-07 2024-09-14T06:31:48.279 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:48.279 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:48.279 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:31:48.279 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654857+0000", 2024-09-14T06:31:48.279 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.187e-06 2024-09-14T06:31:48.279 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:48.280 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:48.280 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:31:48.280 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654861+0000", 2024-09-14T06:31:48.280 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.3850000000000002e-06 2024-09-14T06:31:48.280 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:48.280 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:48.280 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:31:48.280 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654863+0000", 2024-09-14T06:31:48.280 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.9500000000000001e-06 2024-09-14T06:31:48.280 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:48.280 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:48.280 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:31:48.280 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.707456+0000", 2024-09-14T06:31:48.281 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052592434 2024-09-14T06:31:48.281 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:48.281 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:48.281 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:31:48.281 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.707516+0000", 2024-09-14T06:31:48.281 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.0167999999999997e-05 2024-09-14T06:31:48.281 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:48.282 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:48.282 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:31:48.282 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.707657+0000", 2024-09-14T06:31:48.282 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00014122700000000001 2024-09-14T06:31:48.283 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:48.283 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:31:48.283 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:48.283 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:48.283 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:48.283 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:11457 3.1d 3:bcad7303:::rbd_data.112d92027590.0000000000000001:f [read 135168~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:31:48.284 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:25.654867+0000", 2024-09-14T06:31:48.284 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 142.594704742, 2024-09-14T06:31:48.284 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052921902, 2024-09-14T06:31:48.284 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:31:48.284 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:31:48.284 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:31:48.285 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:31:48.285 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:31:48.285 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:48.285 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 11457 2024-09-14T06:31:48.285 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:48.285 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:31:48.285 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:48.286 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:31:48.286 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654867+0000", 2024-09-14T06:31:48.286 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:48.286 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:48.286 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:48.286 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:31:48.286 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654867+0000", 2024-09-14T06:31:48.286 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:48.286 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:48.287 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:48.287 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:31:48.287 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654972+0000", 2024-09-14T06:31:48.287 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00010584 2024-09-14T06:31:48.287 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:48.287 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:48.287 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:31:48.288 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654975+0000", 2024-09-14T06:31:48.288 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.1169999999999998e-06 2024-09-14T06:31:48.288 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:48.288 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:48.288 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:31:48.288 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654979+0000", 2024-09-14T06:31:48.289 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.1960000000000001e-06 2024-09-14T06:31:48.289 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:48.289 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:48.289 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:31:48.289 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654982+0000", 2024-09-14T06:31:48.289 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.2990000000000001e-06 2024-09-14T06:31:48.290 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:48.290 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:48.290 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:31:48.290 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.707684+0000", 2024-09-14T06:31:48.290 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052701979000000003 2024-09-14T06:31:48.290 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:48.291 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:48.291 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:31:48.291 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.707728+0000", 2024-09-14T06:31:48.291 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.3529999999999998e-05 2024-09-14T06:31:48.291 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:48.291 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:48.292 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:31:48.292 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.707789+0000", 2024-09-14T06:31:48.292 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.0940999999999998e-05 2024-09-14T06:31:48.292 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:48.292 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:31:48.292 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:48.292 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:48.293 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:48.293 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:11458 3.1d 3:bcad7303:::rbd_data.112d92027590.0000000000000001:f [read 145408~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:31:48.293 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:25.654987+0000", 2024-09-14T06:31:48.293 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 142.594584344, 2024-09-14T06:31:48.293 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.05291655, 2024-09-14T06:31:48.293 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:31:48.293 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:31:48.294 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:31:48.294 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:31:48.294 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:31:48.294 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:48.294 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 11458 2024-09-14T06:31:48.294 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:48.294 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:31:48.294 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:48.295 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:31:48.295 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654987+0000", 2024-09-14T06:31:48.295 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:48.295 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:48.295 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:48.295 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:31:48.296 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654987+0000", 2024-09-14T06:31:48.296 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:48.296 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:48.296 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:48.296 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:31:48.297 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655077+0000", 2024-09-14T06:31:48.297 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.0253000000000005e-05 2024-09-14T06:31:48.297 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:48.297 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:48.297 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:31:48.297 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655079+0000", 2024-09-14T06:31:48.298 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.9989999999999998e-06 2024-09-14T06:31:48.298 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:48.298 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:48.298 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:31:48.298 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655083+0000", 2024-09-14T06:31:48.299 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.9280000000000003e-06 2024-09-14T06:31:48.299 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:48.299 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:48.299 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:31:48.299 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655086+0000", 2024-09-14T06:31:48.299 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.056e-06 2024-09-14T06:31:48.300 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:48.300 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:48.300 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:31:48.300 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.707805+0000", 2024-09-14T06:31:48.300 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052718627999999997 2024-09-14T06:31:48.300 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:48.300 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:48.300 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:31:48.301 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.707848+0000", 2024-09-14T06:31:48.301 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.2728999999999997e-05 2024-09-14T06:31:48.301 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:48.301 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:48.301 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:31:48.301 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.707904+0000", 2024-09-14T06:31:48.302 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.5957e-05 2024-09-14T06:31:48.302 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:48.302 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:31:48.302 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:48.302 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:48.302 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:48.302 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:11459 3.1d 3:bcad7303:::rbd_data.112d92027590.0000000000000001:f [read 155648~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:31:48.303 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:25.655091+0000", 2024-09-14T06:31:48.303 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 142.594480336, 2024-09-14T06:31:48.303 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052944242000000002, 2024-09-14T06:31:48.303 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:31:48.303 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:31:48.303 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:31:48.304 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:31:48.304 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:31:48.304 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:48.304 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 11459 2024-09-14T06:31:48.304 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:48.304 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:31:48.305 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:48.305 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:31:48.305 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655091+0000", 2024-09-14T06:31:48.305 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:48.305 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:48.305 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:48.306 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:31:48.306 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655091+0000", 2024-09-14T06:31:48.306 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:48.306 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:48.306 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:48.306 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:31:48.307 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655237+0000", 2024-09-14T06:31:48.307 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00014570100000000001 2024-09-14T06:31:48.307 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:48.307 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:48.307 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:31:48.307 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655239+0000", 2024-09-14T06:31:48.308 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.8670000000000001e-06 2024-09-14T06:31:48.308 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:48.308 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:48.308 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:31:48.308 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655243+0000", 2024-09-14T06:31:48.308 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.9829999999999998e-06 2024-09-14T06:31:48.308 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:48.308 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:48.308 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:31:48.308 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655246+0000", 2024-09-14T06:31:48.308 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.1540000000000002e-06 2024-09-14T06:31:48.308 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:48.309 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:48.309 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:31:48.309 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.707934+0000", 2024-09-14T06:31:48.309 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052688286000000001 2024-09-14T06:31:48.309 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:48.309 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:48.309 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:31:48.309 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.707977+0000", 2024-09-14T06:31:48.309 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.3291999999999998e-05 2024-09-14T06:31:48.309 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:48.309 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:48.309 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:31:48.309 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.708035+0000", 2024-09-14T06:31:48.309 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.7958999999999999e-05 2024-09-14T06:31:48.310 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:48.310 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:31:48.310 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:48.310 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:48.310 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:48.310 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:11460 3.1d 3:bcad7303:::rbd_data.112d92027590.0000000000000001:f [read 165888~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:31:48.310 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:25.655250+0000", 2024-09-14T06:31:48.310 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 142.59432103899999, 2024-09-14T06:31:48.310 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052910698999999999, 2024-09-14T06:31:48.310 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:31:48.310 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:31:48.310 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:31:48.310 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:31:48.311 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:31:48.311 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:48.311 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 11460 2024-09-14T06:31:48.311 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:48.311 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:31:48.311 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:48.311 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:31:48.311 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655250+0000", 2024-09-14T06:31:48.311 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:48.311 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:48.311 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:48.311 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:31:48.312 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655250+0000", 2024-09-14T06:31:48.312 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:48.312 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:48.312 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:48.312 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:31:48.312 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655356+0000", 2024-09-14T06:31:48.312 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00010535900000000001 2024-09-14T06:31:48.312 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:48.312 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:48.312 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:31:48.312 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655357+0000", 2024-09-14T06:31:48.312 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.8080000000000001e-06 2024-09-14T06:31:48.312 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:48.312 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:48.313 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:31:48.313 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655361+0000", 2024-09-14T06:31:48.313 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.861e-06 2024-09-14T06:31:48.313 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:48.313 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:48.313 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:31:48.313 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655364+0000", 2024-09-14T06:31:48.313 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.1089999999999998e-06 2024-09-14T06:31:48.313 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:48.313 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:48.313 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:31:48.313 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.708065+0000", 2024-09-14T06:31:48.314 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052700595000000003 2024-09-14T06:31:48.314 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:48.314 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:48.314 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:31:48.314 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.708108+0000", 2024-09-14T06:31:48.314 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.2484999999999997e-05 2024-09-14T06:31:48.314 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:48.314 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:48.314 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:31:48.314 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.708161+0000", 2024-09-14T06:31:48.314 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.3482000000000001e-05 2024-09-14T06:31:48.314 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:48.314 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:31:48.314 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:48.315 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:48.315 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:48.315 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:11461 3.1d 3:bcad7303:::rbd_data.112d92027590.0000000000000001:f [read 176128~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:31:48.315 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:25.655370+0000", 2024-09-14T06:31:48.315 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 142.59420175700001, 2024-09-14T06:31:48.315 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052915141999999998, 2024-09-14T06:31:48.315 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:31:48.315 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:31:48.315 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:31:48.315 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:31:48.315 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:31:48.315 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:48.315 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 11461 2024-09-14T06:31:48.315 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:48.315 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:31:48.316 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:48.316 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:31:48.316 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655370+0000", 2024-09-14T06:31:48.316 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:48.316 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:48.316 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:48.316 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:31:48.316 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655370+0000", 2024-09-14T06:31:48.316 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:48.316 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:48.316 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:48.316 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:31:48.316 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655394+0000", 2024-09-14T06:31:48.317 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.4553999999999999e-05 2024-09-14T06:31:48.317 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:48.317 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:48.317 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:31:48.317 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655396+0000", 2024-09-14T06:31:48.317 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.8950000000000001e-06 2024-09-14T06:31:48.317 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:48.317 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:48.317 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:31:48.317 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655401+0000", 2024-09-14T06:31:48.317 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.0089999999999998e-06 2024-09-14T06:31:48.317 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:48.317 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:48.317 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:31:48.318 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655404+0000", 2024-09-14T06:31:48.318 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.3400000000000002e-06 2024-09-14T06:31:48.318 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:48.318 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:48.318 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:31:48.318 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.708186+0000", 2024-09-14T06:31:48.319 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052781726000000001 2024-09-14T06:31:48.319 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:48.319 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:48.319 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:31:48.319 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.708229+0000", 2024-09-14T06:31:48.319 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.2670999999999999e-05 2024-09-14T06:31:48.320 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:48.320 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:48.320 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:31:48.320 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.708285+0000", 2024-09-14T06:31:48.320 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.5946999999999998e-05 2024-09-14T06:31:48.320 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:48.321 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:31:48.321 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:48.321 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:48.321 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:48.321 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:11462 3.1d 3:bcad7303:::rbd_data.112d92027590.0000000000000001:f [read 186368~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:31:48.322 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:25.655410+0000", 2024-09-14T06:31:48.322 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 142.59416127899999, 2024-09-14T06:31:48.322 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052995258000000003, 2024-09-14T06:31:48.322 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:31:48.322 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:31:48.322 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:31:48.323 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:31:48.323 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:31:48.323 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:48.323 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 11462 2024-09-14T06:31:48.323 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:48.323 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:31:48.324 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:48.324 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:31:48.324 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655410+0000", 2024-09-14T06:31:48.324 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:48.324 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:48.324 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:48.325 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:31:48.325 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655410+0000", 2024-09-14T06:31:48.325 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:48.325 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:48.325 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:48.325 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:31:48.326 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655625+0000", 2024-09-14T06:31:48.326 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00021442600000000001 2024-09-14T06:31:48.326 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:48.326 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:48.326 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:31:48.326 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655626+0000", 2024-09-14T06:31:48.327 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.815e-06 2024-09-14T06:31:48.327 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:48.327 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:48.327 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:31:48.327 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655631+0000", 2024-09-14T06:31:48.327 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.651e-06 2024-09-14T06:31:48.327 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:48.327 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:48.327 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:31:48.327 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655634+0000", 2024-09-14T06:31:48.328 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.2449999999999998e-06 2024-09-14T06:31:48.328 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:48.328 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:48.328 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:31:48.328 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.708312+0000", 2024-09-14T06:31:48.328 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052677412 2024-09-14T06:31:48.328 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:48.328 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:48.328 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:31:48.328 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.708352+0000", 2024-09-14T06:31:48.328 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.0729999999999998e-05 2024-09-14T06:31:48.329 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:48.329 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:48.329 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:31:48.329 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.708405+0000", 2024-09-14T06:31:48.329 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.2979000000000002e-05 2024-09-14T06:31:48.329 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:48.329 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:31:48.329 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:48.329 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:48.329 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:48.330 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:11463 3.1d 3:bcad7303:::rbd_data.112d92027590.0000000000000001:f [read 196608~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:31:48.330 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:25.655639+0000", 2024-09-14T06:31:48.330 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 142.59393213300001, 2024-09-14T06:31:48.330 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052888859000000003, 2024-09-14T06:31:48.330 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:31:48.330 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:31:48.330 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:31:48.331 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:31:48.331 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:31:48.331 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:48.331 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 11463 2024-09-14T06:31:48.331 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:48.331 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:31:48.331 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:48.331 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:31:48.332 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655639+0000", 2024-09-14T06:31:48.332 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:48.332 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:48.332 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:48.332 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:31:48.332 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655639+0000", 2024-09-14T06:31:48.332 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:48.333 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:48.333 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:48.333 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:31:48.333 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655640+0000", 2024-09-14T06:31:48.333 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.08e-07 2024-09-14T06:31:48.333 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:48.333 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:48.334 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:31:48.334 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655641+0000", 2024-09-14T06:31:48.334 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.156e-06 2024-09-14T06:31:48.334 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:48.334 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:48.335 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:31:48.335 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655648+0000", 2024-09-14T06:31:48.335 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.8460000000000002e-06 2024-09-14T06:31:48.335 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:48.335 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:48.335 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:31:48.336 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655651+0000", 2024-09-14T06:31:48.336 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.0079999999999998e-06 2024-09-14T06:31:48.336 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:48.336 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:48.336 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:31:48.336 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.708429+0000", 2024-09-14T06:31:48.336 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052778459 2024-09-14T06:31:48.337 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:48.337 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:48.337 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:31:48.337 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.708473+0000", 2024-09-14T06:31:48.337 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.3782999999999997e-05 2024-09-14T06:31:48.337 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:48.337 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:48.337 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:31:48.338 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.708528+0000", 2024-09-14T06:31:48.338 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.5198999999999998e-05 2024-09-14T06:31:48.338 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:48.338 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:31:48.338 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:48.338 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:48.338 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:48.339 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:11464 3.1d 3:bcad7303:::rbd_data.112d92027590.0000000000000001:f [read 206848~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:31:48.339 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:25.655654+0000", 2024-09-14T06:31:48.339 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 142.593917264, 2024-09-14T06:31:48.339 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052981845999999999, 2024-09-14T06:31:48.339 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:31:48.339 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:31:48.339 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:31:48.339 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:31:48.339 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:31:48.339 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:48.339 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 11464 2024-09-14T06:31:48.339 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:48.340 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:31:48.340 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:48.340 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:31:48.340 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655654+0000", 2024-09-14T06:31:48.340 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:48.340 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:48.340 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:48.340 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:31:48.340 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655654+0000", 2024-09-14T06:31:48.340 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:48.340 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:48.340 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:48.340 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:31:48.341 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655946+0000", 2024-09-14T06:31:48.341 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00029148200000000003 2024-09-14T06:31:48.341 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:48.341 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:48.341 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:31:48.341 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655948+0000", 2024-09-14T06:31:48.341 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.973e-06 2024-09-14T06:31:48.341 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:48.341 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:48.341 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:31:48.341 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655953+0000", 2024-09-14T06:31:48.341 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.1240000000000004e-06 2024-09-14T06:31:48.341 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:48.341 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:48.342 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:31:48.342 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655956+0000", 2024-09-14T06:31:48.342 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.315e-06 2024-09-14T06:31:48.342 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:48.342 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:48.342 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:31:48.342 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.708543+0000", 2024-09-14T06:31:48.342 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052587416999999997 2024-09-14T06:31:48.342 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:48.342 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:48.342 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:31:48.342 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.708584+0000", 2024-09-14T06:31:48.342 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.1004000000000002e-05 2024-09-14T06:31:48.342 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:48.343 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:48.343 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:31:48.343 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.708636+0000", 2024-09-14T06:31:48.343 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.1530999999999998e-05 2024-09-14T06:31:48.343 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:48.343 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:31:48.343 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:48.343 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:48.343 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:48.343 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:11465 3.1d 3:bcad7303:::rbd_data.112d92027590.0000000000000001:f [read 217088~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:31:48.343 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:25.655961+0000", 2024-09-14T06:31:48.343 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 142.59361011199999, 2024-09-14T06:31:48.343 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052798682999999999, 2024-09-14T06:31:48.344 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:31:48.344 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:31:48.344 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:31:48.344 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:31:48.344 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:31:48.344 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:48.344 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 11465 2024-09-14T06:31:48.344 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:48.344 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:31:48.344 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:48.344 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:31:48.344 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655961+0000", 2024-09-14T06:31:48.344 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:48.344 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:48.345 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:48.345 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:31:48.345 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655961+0000", 2024-09-14T06:31:48.345 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:48.345 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:48.345 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:48.345 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:31:48.345 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.656114+0000", 2024-09-14T06:31:48.345 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.000152745 2024-09-14T06:31:48.345 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:48.345 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:48.345 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:31:48.345 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.656116+0000", 2024-09-14T06:31:48.345 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.804e-06 2024-09-14T06:31:48.346 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:48.346 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:48.346 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:31:48.346 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.656120+0000", 2024-09-14T06:31:48.346 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.4700000000000004e-06 2024-09-14T06:31:48.346 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:48.346 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:48.346 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:31:48.346 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.656123+0000", 2024-09-14T06:31:48.346 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.1269999999999999e-06 2024-09-14T06:31:48.346 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:48.346 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:48.346 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:31:48.346 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.708660+0000", 2024-09-14T06:31:48.347 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052536919000000001 2024-09-14T06:31:48.347 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:48.347 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:48.347 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:31:48.347 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.708703+0000", 2024-09-14T06:31:48.347 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.3077000000000001e-05 2024-09-14T06:31:48.347 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:48.347 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:48.347 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:31:48.347 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.708760+0000", 2024-09-14T06:31:48.347 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.6541e-05 2024-09-14T06:31:48.347 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:48.347 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:31:48.348 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:48.348 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:48.348 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:48.348 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:11466 3.1d 3:bcad7303:::rbd_data.112d92027590.0000000000000001:f [read 227328~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:31:48.348 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:25.656129+0000", 2024-09-14T06:31:48.348 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 142.593442724, 2024-09-14T06:31:48.348 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052750987999999999, 2024-09-14T06:31:48.348 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:31:48.348 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:31:48.348 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:31:48.348 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:31:48.348 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:31:48.348 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:48.348 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 11466 2024-09-14T06:31:48.349 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:48.349 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:31:48.349 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:48.349 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:31:48.349 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.656129+0000", 2024-09-14T06:31:48.349 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:48.349 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:48.349 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:48.349 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:31:48.349 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.656129+0000", 2024-09-14T06:31:48.349 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:48.349 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:48.349 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:48.349 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:31:48.350 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.656347+0000", 2024-09-14T06:31:48.350 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.000218054 2024-09-14T06:31:48.350 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:48.350 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:48.350 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:31:48.350 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.656349+0000", 2024-09-14T06:31:48.350 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.9199999999999998e-06 2024-09-14T06:31:48.350 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:48.350 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:48.350 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:31:48.350 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.656353+0000", 2024-09-14T06:31:48.350 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.9210000000000002e-06 2024-09-14T06:31:48.350 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:48.350 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:48.351 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:31:48.351 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.656356+0000", 2024-09-14T06:31:48.351 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.7670000000000001e-06 2024-09-14T06:31:48.351 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:48.351 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:48.351 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:31:48.351 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.708787+0000", 2024-09-14T06:31:48.351 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052430840999999999 2024-09-14T06:31:48.351 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:48.351 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:48.351 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:31:48.351 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.708827+0000", 2024-09-14T06:31:48.351 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.9538e-05 2024-09-14T06:31:48.351 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:48.352 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:48.352 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:31:48.352 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.708880+0000", 2024-09-14T06:31:48.352 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.2947e-05 2024-09-14T06:31:48.352 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:48.352 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:31:48.352 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:48.352 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:48.352 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:48.352 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:11467 3.1d 3:bcad7303:::rbd_data.112d92027590.0000000000000001:f [read 237568~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:31:48.352 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:25.656361+0000", 2024-09-14T06:31:48.352 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 142.59321007599999, 2024-09-14T06:31:48.352 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052643898000000001, 2024-09-14T06:31:48.352 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:31:48.352 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:31:48.353 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:31:48.353 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:31:48.353 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:31:48.353 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:48.353 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 11467 2024-09-14T06:31:48.353 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:48.353 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:31:48.353 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:48.353 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:31:48.353 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.656361+0000", 2024-09-14T06:31:48.353 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:48.353 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:48.353 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:48.354 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:31:48.354 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.656361+0000", 2024-09-14T06:31:48.354 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:48.354 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:48.354 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:48.354 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:31:48.354 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.656855+0000", 2024-09-14T06:31:48.354 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00049372000000000001 2024-09-14T06:31:48.354 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:48.354 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:48.354 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:31:48.354 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.656859+0000", 2024-09-14T06:31:48.354 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.1300000000000003e-06 2024-09-14T06:31:48.354 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:48.355 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:48.355 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:31:48.355 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.656863+0000", 2024-09-14T06:31:48.355 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.2629999999999997e-06 2024-09-14T06:31:48.355 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:48.355 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:48.355 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:31:48.355 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.656867+0000", 2024-09-14T06:31:48.355 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.3220000000000001e-06 2024-09-14T06:31:48.355 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:48.355 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:48.355 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:31:48.355 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.708903+0000", 2024-09-14T06:31:48.355 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052035855999999998 2024-09-14T06:31:48.356 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:48.356 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:48.356 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:31:48.356 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.708949+0000", 2024-09-14T06:31:48.356 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.6276999999999998e-05 2024-09-14T06:31:48.356 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:48.356 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:48.356 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:31:48.356 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.709005+0000", 2024-09-14T06:31:48.356 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.6329999999999998e-05 2024-09-14T06:31:48.356 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:48.356 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:31:48.356 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:48.356 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:48.357 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:48.357 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:11480 3.1d 3:bcad7303:::rbd_data.112d92027590.0000000000000001:f [read 370688~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:31:48.357 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:25.658563+0000", 2024-09-14T06:31:48.357 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 142.59100831200001, 2024-09-14T06:31:48.357 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052429164, 2024-09-14T06:31:48.357 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:31:48.357 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:31:48.357 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:31:48.357 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:31:48.357 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:31:48.357 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:48.357 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 11480 2024-09-14T06:31:48.357 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:48.357 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:31:48.358 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:48.358 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:31:48.358 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.658563+0000", 2024-09-14T06:31:48.358 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:48.358 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:48.358 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:48.358 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:31:48.358 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.658563+0000", 2024-09-14T06:31:48.358 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:48.358 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:48.358 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:48.358 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:31:48.358 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.658658+0000", 2024-09-14T06:31:48.358 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.5346999999999994e-05 2024-09-14T06:31:48.358 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:48.359 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:48.359 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:31:48.359 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.658662+0000", 2024-09-14T06:31:48.359 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.14e-06 2024-09-14T06:31:48.359 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:48.359 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:48.359 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:31:48.359 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.658669+0000", 2024-09-14T06:31:48.359 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.7379999999999994e-06 2024-09-14T06:31:48.359 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:48.359 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:48.359 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:31:48.359 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.658675+0000", 2024-09-14T06:31:48.360 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.5430000000000001e-06 2024-09-14T06:31:48.360 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:48.360 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:48.360 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:31:48.360 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.710824+0000", 2024-09-14T06:31:48.360 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052148847999999998 2024-09-14T06:31:48.360 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:48.360 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:48.360 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:31:48.360 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.710864+0000", 2024-09-14T06:31:48.360 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.9925999999999997e-05 2024-09-14T06:31:48.360 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:48.360 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:48.360 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:31:48.361 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.710992+0000", 2024-09-14T06:31:48.361 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.000128622 2024-09-14T06:31:48.361 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:48.361 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:31:48.361 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:48.361 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:48.361 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:48.361 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:11481 3.1d 3:bcad7303:::rbd_data.112d92027590.0000000000000001:f [read 380928~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:31:48.361 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:25.658684+0000", 2024-09-14T06:31:48.361 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 142.59088724200001, 2024-09-14T06:31:48.361 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052508565, 2024-09-14T06:31:48.361 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:31:48.361 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:31:48.362 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:31:48.362 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:31:48.362 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:31:48.362 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:48.362 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 11481 2024-09-14T06:31:48.362 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:48.362 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:31:48.362 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:48.362 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:31:48.362 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.658684+0000", 2024-09-14T06:31:48.362 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:48.362 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:48.362 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:48.362 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:31:48.363 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.658684+0000", 2024-09-14T06:31:48.363 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:48.363 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:48.363 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:48.363 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:31:48.363 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.658715+0000", 2024-09-14T06:31:48.363 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.0703000000000001e-05 2024-09-14T06:31:48.363 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:48.363 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:48.363 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:31:48.363 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.658718+0000", 2024-09-14T06:31:48.363 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.0709999999999999e-06 2024-09-14T06:31:48.363 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:48.363 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:48.364 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:31:48.364 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.658725+0000", 2024-09-14T06:31:48.364 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.2339999999999997e-06 2024-09-14T06:31:48.364 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:48.364 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:48.364 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:31:48.364 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.658731+0000", 2024-09-14T06:31:48.364 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.6450000000000002e-06 2024-09-14T06:31:48.364 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:48.364 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:48.364 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:31:48.364 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.711020+0000", 2024-09-14T06:31:48.364 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052288886 2024-09-14T06:31:48.364 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:48.365 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:48.365 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:31:48.365 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.711061+0000", 2024-09-14T06:31:48.365 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.1084000000000002e-05 2024-09-14T06:31:48.365 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:48.365 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:48.365 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:31:48.365 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.711193+0000", 2024-09-14T06:31:48.365 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00013194199999999999 2024-09-14T06:31:48.365 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:48.365 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:31:48.365 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:48.365 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:48.366 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:48.366 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:11482 3.1d 3:bcad7303:::rbd_data.112d92027590.0000000000000001:f [read 391168~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:31:48.366 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:25.658740+0000", 2024-09-14T06:31:48.366 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 142.59083177700001, 2024-09-14T06:31:48.366 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052637015000000002, 2024-09-14T06:31:48.366 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:31:48.366 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:31:48.366 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:31:48.366 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:31:48.366 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:31:48.366 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:48.366 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 11482 2024-09-14T06:31:48.366 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:48.366 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:31:48.367 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:48.367 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:31:48.367 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.658740+0000", 2024-09-14T06:31:48.367 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:48.367 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:48.367 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:48.367 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:31:48.367 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.658740+0000", 2024-09-14T06:31:48.367 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:48.367 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:48.367 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:48.367 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:31:48.367 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.658880+0000", 2024-09-14T06:31:48.367 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00014036100000000001 2024-09-14T06:31:48.368 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:48.368 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:48.368 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:31:48.368 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.658883+0000", 2024-09-14T06:31:48.368 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.3969999999999998e-06 2024-09-14T06:31:48.368 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:48.368 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:48.368 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:31:48.368 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.658890+0000", 2024-09-14T06:31:48.368 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.1500000000000002e-06 2024-09-14T06:31:48.368 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:48.368 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:48.368 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:31:48.368 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.658897+0000", 2024-09-14T06:31:48.369 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.1260000000000003e-06 2024-09-14T06:31:48.369 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:48.369 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:48.369 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:31:48.369 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.711207+0000", 2024-09-14T06:31:48.369 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052310777000000003 2024-09-14T06:31:48.369 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:48.369 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:48.369 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:31:48.369 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.711249+0000", 2024-09-14T06:31:48.369 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.1786999999999998e-05 2024-09-14T06:31:48.369 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:48.369 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:48.369 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:31:48.370 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.711377+0000", 2024-09-14T06:31:48.370 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.000127417 2024-09-14T06:31:48.370 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:48.370 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:31:48.370 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:48.370 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:48.370 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:48.370 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:11483 3.1d 3:bcad7303:::rbd_data.112d92027590.0000000000000001:f [read 401408~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:31:48.370 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:25.658906+0000", 2024-09-14T06:31:48.370 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 142.59066499299999, 2024-09-14T06:31:48.370 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052595109000000001, 2024-09-14T06:31:48.370 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:31:48.370 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:31:48.371 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:31:48.371 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:31:48.371 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:31:48.371 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:48.371 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 11483 2024-09-14T06:31:48.371 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:48.371 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:31:48.371 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:48.371 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:31:48.371 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.658906+0000", 2024-09-14T06:31:48.371 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:48.371 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:48.371 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:48.371 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:31:48.372 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.658906+0000", 2024-09-14T06:31:48.372 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:48.372 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:48.372 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:48.372 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:31:48.372 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.664985+0000", 2024-09-14T06:31:48.372 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.0060787250000000001 2024-09-14T06:31:48.372 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:48.372 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:48.372 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:31:48.372 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.664991+0000", 2024-09-14T06:31:48.372 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.7030000000000003e-06 2024-09-14T06:31:48.372 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:48.372 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:48.373 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:31:48.373 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.664999+0000", 2024-09-14T06:31:48.373 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.4430000000000003e-06 2024-09-14T06:31:48.373 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:48.373 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:48.373 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:31:48.373 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.665006+0000", 2024-09-14T06:31:48.373 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.0840000000000003e-06 2024-09-14T06:31:48.373 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:48.373 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:48.373 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:31:48.373 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.711402+0000", 2024-09-14T06:31:48.373 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046395714999999997 2024-09-14T06:31:48.374 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:48.374 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:48.374 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:31:48.374 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.711443+0000", 2024-09-14T06:31:48.374 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.0911999999999997e-05 2024-09-14T06:31:48.374 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:48.374 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:48.374 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:31:48.374 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.711501+0000", 2024-09-14T06:31:48.374 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.8526999999999997e-05 2024-09-14T06:31:48.374 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:48.374 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:31:48.374 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:48.375 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:48.375 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:48.375 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:214569 3.2 3:479633bd:::rbd_object_map.112d3bd84cb0:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e170)", 2024-09-14T06:31:48.375 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:31:34.933901+0000", 2024-09-14T06:31:48.375 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 13.315670609, 2024-09-14T06:31:48.375 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.68157511400000004, 2024-09-14T06:31:48.375 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:31:48.375 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:31:48.375 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:31:48.375 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:31:48.375 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:31:48.375 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:48.375 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 214569 2024-09-14T06:31:48.375 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:48.376 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:31:48.376 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:48.376 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:31:48.376 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:31:34.933901+0000", 2024-09-14T06:31:48.376 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:48.376 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:48.376 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:48.376 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:31:48.376 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:31:34.933901+0000", 2024-09-14T06:31:48.376 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:48.376 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:48.376 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:48.376 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:31:48.376 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:31:34.933902+0000", 2024-09-14T06:31:48.377 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.7439999999999999e-06 2024-09-14T06:31:48.377 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:48.377 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:48.377 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:31:48.377 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:31:34.933910+0000", 2024-09-14T06:31:48.377 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.8569999999999995e-06 2024-09-14T06:31:48.377 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:48.377 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:48.377 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:31:48.377 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:31:34.933932+0000", 2024-09-14T06:31:48.377 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.1997999999999998e-05 2024-09-14T06:31:48.377 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:48.377 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:48.377 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:31:48.378 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:31:34.933941+0000", 2024-09-14T06:31:48.378 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.7180000000000002e-06 2024-09-14T06:31:48.378 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:48.378 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:48.378 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:31:48.378 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:31:34.934013+0000", 2024-09-14T06:31:48.378 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.1761000000000006e-05 2024-09-14T06:31:48.378 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:48.378 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:48.378 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "waiting for peered", 2024-09-14T06:31:48.378 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:31:34.934022+0000", 2024-09-14T06:31:48.378 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.8740000000000001e-06 2024-09-14T06:31:48.378 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:48.378 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:48.379 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:31:48.379 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:31:35.608803+0000", 2024-09-14T06:31:48.379 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.67478164299999999 2024-09-14T06:31:48.379 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:48.379 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:48.379 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:31:48.379 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:31:35.608964+0000", 2024-09-14T06:31:48.379 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00016112900000000001 2024-09-14T06:31:48.379 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:48.379 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:48.379 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "waiting for subops from 6", 2024-09-14T06:31:48.379 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:31:35.609399+0000", 2024-09-14T06:31:48.379 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00043454000000000002 2024-09-14T06:31:48.379 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:48.379 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:48.380 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "op_commit", 2024-09-14T06:31:48.380 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:31:35.610495+0000", 2024-09-14T06:31:48.380 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.001096404 2024-09-14T06:31:48.380 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:48.380 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:48.380 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:31:48.380 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:31:35.615409+0000", 2024-09-14T06:31:48.380 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.0049133199999999997 2024-09-14T06:31:48.380 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:48.380 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:48.380 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "commit_sent", 2024-09-14T06:31:48.380 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:31:35.615444+0000", 2024-09-14T06:31:48.380 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.5457000000000003e-05 2024-09-14T06:31:48.380 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:48.381 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:48.381 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:31:48.381 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:31:35.615476+0000", 2024-09-14T06:31:48.381 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.1668999999999998e-05 2024-09-14T06:31:48.381 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:48.381 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:31:48.381 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:48.381 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:48.381 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:31:48.381 INFO:teuthology.orchestra.run.smithi138.stdout:} 2024-09-14T06:31:48.383 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-14T06:31:48.383 INFO:tasks.ceph.osd.7.smithi138.stderr:2024-09-14T06:31:48.381+0000 7f3c6a21d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33623) UID: 0 2024-09-14T06:31:48.484 INFO:tasks.ceph.osd.2.smithi094.stderr:2024-09-14T06:31:48.483+0000 7faf3269b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33649) UID: 0 2024-09-14T06:31:48.527 INFO:teuthology.orchestra.run.smithi138.stdout:{ 2024-09-14T06:31:48.527 INFO:teuthology.orchestra.run.smithi138.stdout: "ops": [], 2024-09-14T06:31:48.527 INFO:teuthology.orchestra.run.smithi138.stdout: "num_ops": 0 2024-09-14T06:31:48.527 INFO:teuthology.orchestra.run.smithi138.stdout:} 2024-09-14T06:31:48.536 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-14T06:31:48.585 INFO:tasks.ceph.osd.2.smithi094.stderr:2024-09-14T06:31:48.584+0000 7faf3269b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33649) UID: 0 2024-09-14T06:31:48.640 INFO:teuthology.orchestra.run.smithi138.stdout:{ 2024-09-14T06:31:48.641 INFO:teuthology.orchestra.run.smithi138.stdout: "ops": [], 2024-09-14T06:31:48.641 INFO:teuthology.orchestra.run.smithi138.stdout: "complaint_time": 30, 2024-09-14T06:31:48.641 INFO:teuthology.orchestra.run.smithi138.stdout: "num_blocked_ops": 0 2024-09-14T06:31:48.641 INFO:teuthology.orchestra.run.smithi138.stdout:} 2024-09-14T06:31:48.649 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-14T06:31:48.693 INFO:tasks.ceph.osd.4.smithi138.stderr:2024-09-14T06:31:48.692+0000 7f6c653c8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33615) UID: 0 2024-09-14T06:31:48.721 INFO:teuthology.orchestra.run.smithi094.stderr:noscrub is set 2024-09-14T06:31:48.759 INFO:teuthology.orchestra.run.smithi138.stdout:{ 2024-09-14T06:31:48.759 INFO:teuthology.orchestra.run.smithi138.stdout: "size": 20, 2024-09-14T06:31:48.759 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 600, 2024-09-14T06:31:48.759 INFO:teuthology.orchestra.run.smithi138.stdout: "ops": [ 2024-09-14T06:31:48.759 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:48.759 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:98505 3.10 3.1bf56f90 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T06:31:48.759 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:30:07.045453+0000", 2024-09-14T06:31:48.759 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 101.712194833, 2024-09-14T06:31:48.759 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032144440000000003, 2024-09-14T06:31:48.760 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:31:48.760 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:31:48.760 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "reached pg", 2024-09-14T06:31:48.760 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:31:48.760 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:31:48.760 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:48.760 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 98505 2024-09-14T06:31:48.760 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:48.760 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:31:48.760 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:48.760 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:31:48.760 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045453+0000", 2024-09-14T06:31:48.760 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:48.761 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:48.761 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:48.761 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:31:48.761 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045453+0000", 2024-09-14T06:31:48.761 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:48.761 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:48.761 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:48.761 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:31:48.761 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045453+0000", 2024-09-14T06:31:48.761 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.36e-07 2024-09-14T06:31:48.761 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:48.761 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:48.761 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:31:48.761 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045454+0000", 2024-09-14T06:31:48.762 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.1299999999999998e-07 2024-09-14T06:31:48.762 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:48.762 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:48.762 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:31:48.762 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045457+0000", 2024-09-14T06:31:48.762 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.8559999999999998e-06 2024-09-14T06:31:48.762 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:48.762 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:48.762 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:31:48.762 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045460+0000", 2024-09-14T06:31:48.762 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.8750000000000002e-06 2024-09-14T06:31:48.762 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:48.762 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:48.762 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:31:48.762 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.077592+0000", 2024-09-14T06:31:48.763 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032131727999999998 2024-09-14T06:31:48.763 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:48.763 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:48.763 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:31:48.763 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.077597+0000", 2024-09-14T06:31:48.763 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.8319999999999996e-06 2024-09-14T06:31:48.763 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:48.763 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:31:48.763 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:48.763 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:48.763 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:48.763 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:98506 3.10 3.1bf56f90 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T06:31:48.763 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:30:07.045481+0000", 2024-09-14T06:31:48.763 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 101.71216657799999, 2024-09-14T06:31:48.764 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032142759999999999, 2024-09-14T06:31:48.764 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:31:48.764 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:31:48.764 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "reached pg", 2024-09-14T06:31:48.764 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:31:48.764 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:31:48.764 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:48.764 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 98506 2024-09-14T06:31:48.764 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:48.764 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:31:48.764 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:48.764 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:31:48.764 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045481+0000", 2024-09-14T06:31:48.764 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:48.765 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:48.765 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:48.765 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:31:48.765 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045481+0000", 2024-09-14T06:31:48.765 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:48.765 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:48.765 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:48.765 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:31:48.765 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045481+0000", 2024-09-14T06:31:48.765 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.8500000000000001e-07 2024-09-14T06:31:48.765 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:48.765 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:48.765 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:31:48.765 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045483+0000", 2024-09-14T06:31:48.766 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.8250000000000001e-06 2024-09-14T06:31:48.766 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:48.766 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:48.766 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:31:48.766 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045489+0000", 2024-09-14T06:31:48.766 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.5740000000000003e-06 2024-09-14T06:31:48.766 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:48.766 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:48.766 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:31:48.766 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045495+0000", 2024-09-14T06:31:48.766 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.7690000000000002e-06 2024-09-14T06:31:48.766 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:48.766 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:48.766 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:31:48.766 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.077619+0000", 2024-09-14T06:31:48.767 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032124621999999999 2024-09-14T06:31:48.767 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:48.767 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:48.767 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:31:48.767 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.077624+0000", 2024-09-14T06:31:48.767 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.3850000000000004e-06 2024-09-14T06:31:48.767 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:48.767 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:31:48.767 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:48.767 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:48.767 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:48.767 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:98507 3.10 3.1bf56f90 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T06:31:48.767 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:30:07.045502+0000", 2024-09-14T06:31:48.767 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 101.71214515699999, 2024-09-14T06:31:48.768 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032147219999999997, 2024-09-14T06:31:48.768 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:31:48.768 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:31:48.768 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "reached pg", 2024-09-14T06:31:48.768 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:31:48.768 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:31:48.768 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:48.768 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 98507 2024-09-14T06:31:48.768 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:48.768 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:31:48.768 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:48.768 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:31:48.768 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045502+0000", 2024-09-14T06:31:48.768 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:48.768 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:48.769 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:48.769 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:31:48.769 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045502+0000", 2024-09-14T06:31:48.769 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:48.769 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:48.769 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:48.769 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:31:48.769 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045503+0000", 2024-09-14T06:31:48.769 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.4700000000000002e-07 2024-09-14T06:31:48.769 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:48.769 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:48.769 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:31:48.769 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045509+0000", 2024-09-14T06:31:48.769 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.7329999999999997e-06 2024-09-14T06:31:48.769 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:48.770 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:48.770 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:31:48.770 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045515+0000", 2024-09-14T06:31:48.770 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.8030000000000002e-06 2024-09-14T06:31:48.770 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:48.770 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:48.770 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:31:48.770 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045522+0000", 2024-09-14T06:31:48.770 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.0369999999999997e-06 2024-09-14T06:31:48.770 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:48.770 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:48.770 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:31:48.770 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.077641+0000", 2024-09-14T06:31:48.770 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032119091000000002 2024-09-14T06:31:48.770 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:48.771 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:48.771 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:31:48.771 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.077650+0000", 2024-09-14T06:31:48.771 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.2090000000000005e-06 2024-09-14T06:31:48.771 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:48.771 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:31:48.771 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:48.771 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:48.771 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:48.771 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:98508 3.10 3.1bf56f90 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T06:31:48.771 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:30:07.045530+0000", 2024-09-14T06:31:48.771 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 101.71211753999999, 2024-09-14T06:31:48.771 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032198773999999999, 2024-09-14T06:31:48.771 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:31:48.771 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:31:48.772 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "reached pg", 2024-09-14T06:31:48.772 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:31:48.772 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:31:48.772 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:48.772 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 98508 2024-09-14T06:31:48.772 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:48.772 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:31:48.772 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:48.772 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:31:48.772 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045530+0000", 2024-09-14T06:31:48.772 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:48.772 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:48.772 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:48.772 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:31:48.772 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045530+0000", 2024-09-14T06:31:48.773 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:48.773 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:48.773 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:48.773 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:31:48.773 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045530+0000", 2024-09-14T06:31:48.773 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.3099999999999998e-07 2024-09-14T06:31:48.773 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:48.773 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:48.773 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:31:48.773 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045539+0000", 2024-09-14T06:31:48.773 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.4780000000000008e-06 2024-09-14T06:31:48.773 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:48.773 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:48.773 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:31:48.774 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045546+0000", 2024-09-14T06:31:48.774 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.3259999999999998e-06 2024-09-14T06:31:48.774 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:48.774 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:48.774 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:31:48.774 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045551+0000", 2024-09-14T06:31:48.774 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.9960000000000001e-06 2024-09-14T06:31:48.774 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:48.774 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:48.774 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:31:48.774 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.077716+0000", 2024-09-14T06:31:48.774 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032165081999999998 2024-09-14T06:31:48.774 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:48.774 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:48.775 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:31:48.775 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.077729+0000", 2024-09-14T06:31:48.775 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.2461e-05 2024-09-14T06:31:48.775 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:48.775 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:31:48.775 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:48.775 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:48.775 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:48.775 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:98509 3.10 3.1bf56f90 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T06:31:48.775 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:30:07.045558+0000", 2024-09-14T06:31:48.775 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 101.712089339, 2024-09-14T06:31:48.775 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032242198, 2024-09-14T06:31:48.775 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:31:48.775 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:31:48.775 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "reached pg", 2024-09-14T06:31:48.776 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:31:48.776 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:31:48.776 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:48.776 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 98509 2024-09-14T06:31:48.776 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:48.776 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:31:48.776 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:48.776 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:31:48.776 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045558+0000", 2024-09-14T06:31:48.776 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:48.776 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:48.776 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:48.776 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:31:48.776 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045558+0000", 2024-09-14T06:31:48.777 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:48.777 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:48.777 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:48.777 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:31:48.777 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045559+0000", 2024-09-14T06:31:48.777 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.27e-07 2024-09-14T06:31:48.777 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:48.777 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:48.777 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:31:48.777 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045560+0000", 2024-09-14T06:31:48.777 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.6959999999999999e-06 2024-09-14T06:31:48.777 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:48.777 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:48.777 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:31:48.777 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045567+0000", 2024-09-14T06:31:48.778 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.6900000000000003e-06 2024-09-14T06:31:48.778 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:48.778 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:48.778 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:31:48.778 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045571+0000", 2024-09-14T06:31:48.778 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.5750000000000002e-06 2024-09-14T06:31:48.778 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:48.778 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:48.778 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:31:48.778 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.077789+0000", 2024-09-14T06:31:48.778 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032217186000000002 2024-09-14T06:31:48.778 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:48.778 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:48.778 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:31:48.779 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.077800+0000", 2024-09-14T06:31:48.779 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.1623999999999999e-05 2024-09-14T06:31:48.779 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:48.779 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:31:48.779 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:48.779 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:48.779 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:48.779 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:98510 3.10 3.1bf56f90 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T06:31:48.779 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:30:07.045580+0000", 2024-09-14T06:31:48.779 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 101.71206752499999, 2024-09-14T06:31:48.779 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032299845000000001, 2024-09-14T06:31:48.779 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:31:48.779 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:31:48.779 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "reached pg", 2024-09-14T06:31:48.780 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:31:48.780 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:31:48.780 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:48.780 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 98510 2024-09-14T06:31:48.780 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:48.780 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:31:48.780 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:48.780 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:31:48.780 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045580+0000", 2024-09-14T06:31:48.780 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:48.780 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:48.780 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:48.780 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:31:48.780 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045580+0000", 2024-09-14T06:31:48.780 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:48.781 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:48.781 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:48.781 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:31:48.781 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045580+0000", 2024-09-14T06:31:48.781 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.0400000000000002e-07 2024-09-14T06:31:48.781 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:48.781 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:48.781 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:31:48.781 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045582+0000", 2024-09-14T06:31:48.781 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.7600000000000001e-06 2024-09-14T06:31:48.781 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:48.781 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:48.781 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:31:48.781 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045588+0000", 2024-09-14T06:31:48.782 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.5650000000000001e-06 2024-09-14T06:31:48.782 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:48.782 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:48.782 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:31:48.782 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045597+0000", 2024-09-14T06:31:48.782 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.465e-06 2024-09-14T06:31:48.782 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:48.782 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:48.782 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:31:48.782 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.077871+0000", 2024-09-14T06:31:48.782 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032273562999999998 2024-09-14T06:31:48.782 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:48.782 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:48.782 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:31:48.783 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.077880+0000", 2024-09-14T06:31:48.783 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.0880000000000001e-06 2024-09-14T06:31:48.783 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:48.783 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:31:48.783 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:48.783 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:48.783 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:48.783 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:98511 3.10 3.1bf56f90 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T06:31:48.783 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:30:07.045610+0000", 2024-09-14T06:31:48.783 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 101.712037678, 2024-09-14T06:31:48.783 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032338894, 2024-09-14T06:31:48.783 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:31:48.783 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:31:48.783 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "reached pg", 2024-09-14T06:31:48.784 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:31:48.784 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:31:48.784 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:48.784 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 98511 2024-09-14T06:31:48.784 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:48.784 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:31:48.784 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:48.784 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:31:48.784 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045610+0000", 2024-09-14T06:31:48.784 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:48.784 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:48.784 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:48.784 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:31:48.784 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045610+0000", 2024-09-14T06:31:48.784 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:48.785 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:48.785 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:48.785 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:31:48.785 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045610+0000", 2024-09-14T06:31:48.785 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.1600000000000002e-07 2024-09-14T06:31:48.785 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:48.785 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:48.785 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:31:48.785 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045612+0000", 2024-09-14T06:31:48.785 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.1150000000000001e-06 2024-09-14T06:31:48.785 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:48.785 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:48.785 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:31:48.785 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045618+0000", 2024-09-14T06:31:48.786 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.0499999999999997e-06 2024-09-14T06:31:48.786 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:48.786 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:48.786 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:31:48.786 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045623+0000", 2024-09-14T06:31:48.786 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.6770000000000003e-06 2024-09-14T06:31:48.786 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:48.786 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:48.786 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:31:48.786 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.077928+0000", 2024-09-14T06:31:48.786 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032305111999999997 2024-09-14T06:31:48.786 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:48.786 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:48.786 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:31:48.787 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.077949+0000", 2024-09-14T06:31:48.787 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.0523999999999999e-05 2024-09-14T06:31:48.787 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:48.787 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:31:48.787 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:48.787 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:48.787 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:48.787 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:98512 3.10 3.1bf56f90 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T06:31:48.787 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:30:07.045630+0000", 2024-09-14T06:31:48.787 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 101.71201767300001, 2024-09-14T06:31:48.787 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032346632, 2024-09-14T06:31:48.787 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:31:48.787 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:31:48.788 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "reached pg", 2024-09-14T06:31:48.788 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:31:48.788 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:31:48.788 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:48.788 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 98512 2024-09-14T06:31:48.788 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:48.788 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:31:48.788 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:48.788 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:31:48.788 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045630+0000", 2024-09-14T06:31:48.788 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:48.788 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:48.788 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:48.789 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:31:48.789 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045630+0000", 2024-09-14T06:31:48.789 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:48.789 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:48.789 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:48.789 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:31:48.789 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045630+0000", 2024-09-14T06:31:48.789 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.8200000000000001e-07 2024-09-14T06:31:48.789 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:48.789 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:48.789 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:31:48.789 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045632+0000", 2024-09-14T06:31:48.789 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.7430000000000001e-06 2024-09-14T06:31:48.789 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:48.790 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:48.790 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:31:48.790 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045637+0000", 2024-09-14T06:31:48.790 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.4550000000000003e-06 2024-09-14T06:31:48.790 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:48.790 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:48.790 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:31:48.790 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045644+0000", 2024-09-14T06:31:48.790 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.1330000000000001e-06 2024-09-14T06:31:48.790 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:48.790 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:48.790 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:31:48.790 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.077968+0000", 2024-09-14T06:31:48.791 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032323401000000002 2024-09-14T06:31:48.791 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:48.791 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:48.791 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:31:48.791 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.077976+0000", 2024-09-14T06:31:48.791 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.5180000000000004e-06 2024-09-14T06:31:48.791 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:48.791 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:31:48.791 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:48.791 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:48.791 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:48.791 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:98513 3.10 3.1bf56f90 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T06:31:48.791 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:30:07.045652+0000", 2024-09-14T06:31:48.791 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 101.711995109, 2024-09-14T06:31:48.792 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032353436999999999, 2024-09-14T06:31:48.792 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:31:48.792 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:31:48.792 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "reached pg", 2024-09-14T06:31:48.792 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:31:48.792 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:31:48.792 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:48.792 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 98513 2024-09-14T06:31:48.792 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:48.792 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:31:48.792 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:48.792 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:31:48.792 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045652+0000", 2024-09-14T06:31:48.793 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:48.793 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:48.793 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:48.793 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:31:48.793 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045652+0000", 2024-09-14T06:31:48.793 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:48.793 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:48.793 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:48.793 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:31:48.793 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045653+0000", 2024-09-14T06:31:48.793 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.3700000000000001e-07 2024-09-14T06:31:48.793 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:48.793 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:48.793 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:31:48.794 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045655+0000", 2024-09-14T06:31:48.794 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.854e-06 2024-09-14T06:31:48.794 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:48.794 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:48.794 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:31:48.794 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045661+0000", 2024-09-14T06:31:48.794 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.5509999999999996e-06 2024-09-14T06:31:48.794 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:48.794 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:48.794 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:31:48.794 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045668+0000", 2024-09-14T06:31:48.794 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.1980000000000004e-06 2024-09-14T06:31:48.794 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:48.794 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:48.795 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:31:48.795 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.077997+0000", 2024-09-14T06:31:48.795 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032328575999999998 2024-09-14T06:31:48.795 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:48.795 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:48.795 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:31:48.795 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.078006+0000", 2024-09-14T06:31:48.795 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.8210000000000008e-06 2024-09-14T06:31:48.795 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:48.795 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:31:48.795 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:48.795 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:48.795 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:48.795 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:98514 3.10 3.1bf56f90 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T06:31:48.796 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:30:07.045699+0000", 2024-09-14T06:31:48.796 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 101.71194877400001, 2024-09-14T06:31:48.796 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032330223999999998, 2024-09-14T06:31:48.796 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:31:48.796 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:31:48.796 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "reached pg", 2024-09-14T06:31:48.796 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:31:48.796 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:31:48.796 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:48.796 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 98514 2024-09-14T06:31:48.796 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:48.796 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:31:48.796 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:48.797 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:31:48.797 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045699+0000", 2024-09-14T06:31:48.797 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:48.797 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:48.797 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:48.797 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:31:48.797 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045699+0000", 2024-09-14T06:31:48.797 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:48.797 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:48.797 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:48.797 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:31:48.797 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045699+0000", 2024-09-14T06:31:48.797 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.3200000000000001e-07 2024-09-14T06:31:48.797 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:48.797 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:48.798 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:31:48.798 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045701+0000", 2024-09-14T06:31:48.798 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.762e-06 2024-09-14T06:31:48.798 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:48.798 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:48.798 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:31:48.798 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045705+0000", 2024-09-14T06:31:48.798 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.2359999999999998e-06 2024-09-14T06:31:48.798 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:48.798 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:48.798 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:31:48.798 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045708+0000", 2024-09-14T06:31:48.798 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.796e-06 2024-09-14T06:31:48.798 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:48.798 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:48.799 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:31:48.799 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.078023+0000", 2024-09-14T06:31:48.799 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032315213000000002 2024-09-14T06:31:48.799 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:48.799 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:48.799 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:31:48.799 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.078029+0000", 2024-09-14T06:31:48.799 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.8850000000000004e-06 2024-09-14T06:31:48.799 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:48.799 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:31:48.799 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:48.799 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:48.799 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:48.799 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:98515 3.10 3.1bf56f90 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T06:31:48.799 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:30:07.045712+0000", 2024-09-14T06:31:48.800 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 101.711935463, 2024-09-14T06:31:48.800 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032346871999999999, 2024-09-14T06:31:48.800 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:31:48.800 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:31:48.800 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "reached pg", 2024-09-14T06:31:48.800 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:31:48.800 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:31:48.800 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:48.800 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 98515 2024-09-14T06:31:48.800 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:48.800 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:31:48.800 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:48.800 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:31:48.800 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045712+0000", 2024-09-14T06:31:48.800 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:48.800 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:48.801 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:48.801 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:31:48.801 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045712+0000", 2024-09-14T06:31:48.801 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:48.801 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:48.801 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:48.801 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:31:48.801 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045712+0000", 2024-09-14T06:31:48.801 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.2100000000000001e-07 2024-09-14T06:31:48.801 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:48.801 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:48.801 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:31:48.801 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045713+0000", 2024-09-14T06:31:48.802 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.068e-06 2024-09-14T06:31:48.802 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:48.802 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:48.802 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:31:48.802 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045716+0000", 2024-09-14T06:31:48.802 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.971e-06 2024-09-14T06:31:48.802 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:48.802 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:48.802 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:31:48.802 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045719+0000", 2024-09-14T06:31:48.802 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.3879999999999998e-06 2024-09-14T06:31:48.802 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:48.802 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:48.802 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:31:48.802 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.078049+0000", 2024-09-14T06:31:48.803 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032330553999999997 2024-09-14T06:31:48.803 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:48.803 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:48.803 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:31:48.803 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.078059+0000", 2024-09-14T06:31:48.803 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.6700000000000006e-06 2024-09-14T06:31:48.803 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:48.803 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:31:48.803 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:48.803 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:48.803 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:48.803 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:98516 3.10 3.1bf56f90 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T06:31:48.803 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:30:07.045722+0000", 2024-09-14T06:31:48.803 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 101.71192571, 2024-09-14T06:31:48.803 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032374334999999997, 2024-09-14T06:31:48.804 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:31:48.804 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:31:48.804 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "reached pg", 2024-09-14T06:31:48.804 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:31:48.804 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:31:48.804 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:48.804 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 98516 2024-09-14T06:31:48.804 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:48.804 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:31:48.804 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:48.804 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:31:48.804 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045722+0000", 2024-09-14T06:31:48.804 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:48.804 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:48.805 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:48.805 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:31:48.805 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045722+0000", 2024-09-14T06:31:48.805 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:48.805 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:48.805 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:48.805 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:31:48.805 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045722+0000", 2024-09-14T06:31:48.805 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.8999999999999998e-07 2024-09-14T06:31:48.805 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:48.805 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:48.805 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:31:48.805 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045723+0000", 2024-09-14T06:31:48.805 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.0049999999999999e-06 2024-09-14T06:31:48.805 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:48.806 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:48.806 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:31:48.806 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045726+0000", 2024-09-14T06:31:48.806 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.9019999999999999e-06 2024-09-14T06:31:48.806 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:48.806 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:48.806 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:31:48.806 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045728+0000", 2024-09-14T06:31:48.806 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.3290000000000001e-06 2024-09-14T06:31:48.806 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:48.806 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:48.806 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:31:48.806 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.078090+0000", 2024-09-14T06:31:48.806 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032362123 2024-09-14T06:31:48.806 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:48.807 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:48.807 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:31:48.807 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.078096+0000", 2024-09-14T06:31:48.807 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.6860000000000003e-06 2024-09-14T06:31:48.807 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:48.807 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:31:48.807 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:48.807 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:48.807 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:48.807 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:98517 3.10 3.1bf56f90 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T06:31:48.807 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:30:07.045731+0000", 2024-09-14T06:31:48.807 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 101.711915979, 2024-09-14T06:31:48.807 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.03241227, 2024-09-14T06:31:48.807 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:31:48.807 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:31:48.808 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "reached pg", 2024-09-14T06:31:48.808 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:31:48.808 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:31:48.808 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:48.808 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 98517 2024-09-14T06:31:48.808 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:48.808 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:31:48.808 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:48.808 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:31:48.808 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045731+0000", 2024-09-14T06:31:48.808 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:48.808 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:48.808 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:48.808 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:31:48.808 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045731+0000", 2024-09-14T06:31:48.809 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:48.809 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:48.809 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:48.809 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:31:48.809 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045732+0000", 2024-09-14T06:31:48.809 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.0699999999999999e-07 2024-09-14T06:31:48.809 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:48.809 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:48.809 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:31:48.809 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045733+0000", 2024-09-14T06:31:48.809 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.9199999999999999e-07 2024-09-14T06:31:48.809 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:48.809 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:48.809 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:31:48.809 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045735+0000", 2024-09-14T06:31:48.810 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.8150000000000002e-06 2024-09-14T06:31:48.810 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:48.810 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:48.810 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:31:48.810 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045738+0000", 2024-09-14T06:31:48.810 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.3640000000000002e-06 2024-09-14T06:31:48.810 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:48.810 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:48.810 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:31:48.810 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.078126+0000", 2024-09-14T06:31:48.810 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032388206000000003 2024-09-14T06:31:48.810 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:48.810 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:48.810 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:31:48.811 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.078144+0000", 2024-09-14T06:31:48.811 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.7686e-05 2024-09-14T06:31:48.811 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:48.811 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:31:48.811 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:48.811 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:48.811 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:48.811 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:98518 3.10 3.1bf56f90 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T06:31:48.811 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:30:07.045741+0000", 2024-09-14T06:31:48.811 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 101.711906584, 2024-09-14T06:31:48.811 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032451793999999999, 2024-09-14T06:31:48.811 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:31:48.811 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:31:48.811 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "reached pg", 2024-09-14T06:31:48.812 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:31:48.812 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:31:48.812 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:48.812 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 98518 2024-09-14T06:31:48.812 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:48.812 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:31:48.812 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:48.812 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:31:48.812 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045741+0000", 2024-09-14T06:31:48.812 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:48.812 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:48.812 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:48.812 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:31:48.812 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045741+0000", 2024-09-14T06:31:48.813 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:48.813 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:48.813 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:48.813 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:31:48.813 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045741+0000", 2024-09-14T06:31:48.813 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.05e-07 2024-09-14T06:31:48.813 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:48.813 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:48.813 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:31:48.813 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045746+0000", 2024-09-14T06:31:48.813 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.2259999999999996e-06 2024-09-14T06:31:48.813 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:48.813 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:48.813 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:31:48.813 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045750+0000", 2024-09-14T06:31:48.814 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.2490000000000001e-06 2024-09-14T06:31:48.814 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:48.814 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:48.814 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:31:48.814 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045752+0000", 2024-09-14T06:31:48.814 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.3980000000000002e-06 2024-09-14T06:31:48.814 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:48.814 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:48.814 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:31:48.814 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.078184+0000", 2024-09-14T06:31:48.814 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032431593000000002 2024-09-14T06:31:48.814 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:48.814 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:48.814 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:31:48.814 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.078193+0000", 2024-09-14T06:31:48.815 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.1230000000000006e-06 2024-09-14T06:31:48.815 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:48.815 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:31:48.815 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:48.815 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:48.815 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:48.815 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:98519 3.10 3.1bf56f90 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T06:31:48.815 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:30:07.045755+0000", 2024-09-14T06:31:48.815 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 101.711892159, 2024-09-14T06:31:48.815 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032489431999999999, 2024-09-14T06:31:48.815 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:31:48.815 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:31:48.815 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "reached pg", 2024-09-14T06:31:48.815 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:31:48.816 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:31:48.816 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:48.816 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 98519 2024-09-14T06:31:48.816 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:48.816 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:31:48.816 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:48.816 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:31:48.816 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045755+0000", 2024-09-14T06:31:48.816 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:48.816 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:48.816 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:48.816 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:31:48.816 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045755+0000", 2024-09-14T06:31:48.816 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:48.816 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:48.817 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:48.817 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:31:48.817 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045756+0000", 2024-09-14T06:31:48.817 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.23e-07 2024-09-14T06:31:48.817 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:48.817 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:48.817 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:31:48.817 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045757+0000", 2024-09-14T06:31:48.817 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.1060000000000001e-06 2024-09-14T06:31:48.817 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:48.817 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:48.817 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:31:48.817 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045760+0000", 2024-09-14T06:31:48.817 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.988e-06 2024-09-14T06:31:48.818 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:48.818 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:48.818 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:31:48.818 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045765+0000", 2024-09-14T06:31:48.818 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.2689999999999999e-06 2024-09-14T06:31:48.818 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:48.818 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:48.818 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:31:48.818 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.078234+0000", 2024-09-14T06:31:48.818 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032468915000000001 2024-09-14T06:31:48.818 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:48.818 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:48.818 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:31:48.818 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.078245+0000", 2024-09-14T06:31:48.818 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.0930999999999999e-05 2024-09-14T06:31:48.819 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:48.819 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:31:48.819 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:48.819 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:48.819 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:48.819 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:98520 3.10 3.1bf56f90 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T06:31:48.819 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:30:07.045768+0000", 2024-09-14T06:31:48.819 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 101.71187928099999, 2024-09-14T06:31:48.819 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032531379999999999, 2024-09-14T06:31:48.819 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:31:48.819 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:31:48.819 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "reached pg", 2024-09-14T06:31:48.819 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:31:48.819 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:31:48.819 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:48.820 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 98520 2024-09-14T06:31:48.820 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:48.820 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:31:48.820 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:48.820 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:31:48.820 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045768+0000", 2024-09-14T06:31:48.820 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:48.820 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:48.820 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:48.820 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:31:48.820 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045768+0000", 2024-09-14T06:31:48.820 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:48.820 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:48.820 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:48.821 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:31:48.821 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045768+0000", 2024-09-14T06:31:48.821 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.22e-07 2024-09-14T06:31:48.821 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:48.821 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:48.821 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:31:48.821 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045773+0000", 2024-09-14T06:31:48.821 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.3470000000000001e-06 2024-09-14T06:31:48.821 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:48.821 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:48.821 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:31:48.821 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045776+0000", 2024-09-14T06:31:48.821 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.1319999999999998e-06 2024-09-14T06:31:48.821 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:48.822 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:48.822 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:31:48.822 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045779+0000", 2024-09-14T06:31:48.822 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.9749999999999999e-06 2024-09-14T06:31:48.822 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:48.822 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:48.822 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:31:48.822 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.078289+0000", 2024-09-14T06:31:48.822 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032509693999999999 2024-09-14T06:31:48.822 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:48.822 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:48.822 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:31:48.822 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.078300+0000", 2024-09-14T06:31:48.823 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.101e-05 2024-09-14T06:31:48.823 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:48.823 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:31:48.823 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:48.823 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:48.823 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:48.823 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:98521 3.10 3.1bf56f90 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T06:31:48.823 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:30:07.045782+0000", 2024-09-14T06:31:48.823 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 101.711865101, 2024-09-14T06:31:48.823 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032566109000000003, 2024-09-14T06:31:48.823 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:31:48.823 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:31:48.823 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "reached pg", 2024-09-14T06:31:48.823 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:31:48.824 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:31:48.824 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:48.824 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 98521 2024-09-14T06:31:48.824 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:48.824 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:31:48.824 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:48.824 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:31:48.824 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045782+0000", 2024-09-14T06:31:48.824 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:48.824 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:48.824 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:48.824 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:31:48.824 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045782+0000", 2024-09-14T06:31:48.824 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:48.825 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:48.825 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:48.825 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:31:48.825 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045783+0000", 2024-09-14T06:31:48.825 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.2499999999999999e-07 2024-09-14T06:31:48.825 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:48.825 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:48.825 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:31:48.825 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045784+0000", 2024-09-14T06:31:48.825 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.0109999999999999e-06 2024-09-14T06:31:48.825 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:48.825 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:48.825 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:31:48.825 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045786+0000", 2024-09-14T06:31:48.826 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.8289999999999999e-06 2024-09-14T06:31:48.826 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:48.826 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:48.826 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:31:48.826 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045793+0000", 2024-09-14T06:31:48.826 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.579e-06 2024-09-14T06:31:48.826 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:48.826 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:48.826 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:31:48.826 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.078338+0000", 2024-09-14T06:31:48.826 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032545389000000001 2024-09-14T06:31:48.826 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:48.826 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:48.826 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:31:48.827 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.078348+0000", 2024-09-14T06:31:48.827 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.0076e-05 2024-09-14T06:31:48.827 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:48.827 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:31:48.827 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:48.827 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:48.827 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:48.827 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:98522 3.10 3.1bf56f90 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T06:31:48.827 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:30:07.045796+0000", 2024-09-14T06:31:48.827 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 101.711851193, 2024-09-14T06:31:48.827 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032577846000000001, 2024-09-14T06:31:48.827 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:31:48.827 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:31:48.827 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "reached pg", 2024-09-14T06:31:48.827 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:31:48.828 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:31:48.828 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:48.828 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 98522 2024-09-14T06:31:48.828 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:48.828 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:31:48.828 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:48.828 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:31:48.828 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045796+0000", 2024-09-14T06:31:48.828 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:48.828 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:48.828 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:48.828 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:31:48.828 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045796+0000", 2024-09-14T06:31:48.828 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:48.829 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:48.829 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:48.829 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:31:48.829 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045796+0000", 2024-09-14T06:31:48.829 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.2100000000000001e-07 2024-09-14T06:31:48.829 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:48.829 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:48.829 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:31:48.829 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045797+0000", 2024-09-14T06:31:48.829 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.6899999999999996e-07 2024-09-14T06:31:48.829 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:48.829 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:48.829 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:31:48.829 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045800+0000", 2024-09-14T06:31:48.829 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.9979999999999999e-06 2024-09-14T06:31:48.829 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:48.830 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:48.830 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:31:48.830 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045803+0000", 2024-09-14T06:31:48.830 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.2850000000000001e-06 2024-09-14T06:31:48.830 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:48.830 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:48.830 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:31:48.830 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.078366+0000", 2024-09-14T06:31:48.830 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032562932000000003 2024-09-14T06:31:48.830 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:48.830 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:48.830 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:31:48.830 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.078374+0000", 2024-09-14T06:31:48.830 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.4409999999999993e-06 2024-09-14T06:31:48.831 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:48.831 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:31:48.831 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:48.831 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:48.831 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:48.831 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:98523 3.10 3.1bf56f90 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T06:31:48.831 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:30:07.045806+0000", 2024-09-14T06:31:48.831 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 101.71184165, 2024-09-14T06:31:48.831 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032611167000000003, 2024-09-14T06:31:48.831 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:31:48.831 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:31:48.831 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "reached pg", 2024-09-14T06:31:48.832 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:31:48.832 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:31:48.832 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:48.832 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 98523 2024-09-14T06:31:48.832 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:48.832 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:31:48.832 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:48.832 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:31:48.832 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045806+0000", 2024-09-14T06:31:48.832 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:48.832 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:48.832 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:48.832 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:31:48.832 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045806+0000", 2024-09-14T06:31:48.832 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:48.833 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:48.833 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:48.833 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:31:48.833 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045806+0000", 2024-09-14T06:31:48.833 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.2000000000000001e-07 2024-09-14T06:31:48.833 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:48.833 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:48.833 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:31:48.833 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045807+0000", 2024-09-14T06:31:48.833 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.4e-07 2024-09-14T06:31:48.833 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:48.833 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:48.833 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:31:48.834 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045810+0000", 2024-09-14T06:31:48.834 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.9229999999999998e-06 2024-09-14T06:31:48.834 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:48.834 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:48.834 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:31:48.834 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045812+0000", 2024-09-14T06:31:48.834 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.2220000000000001e-06 2024-09-14T06:31:48.834 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:48.834 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:48.834 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:31:48.834 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.078407+0000", 2024-09-14T06:31:48.834 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032594513999999998 2024-09-14T06:31:48.834 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:48.834 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:48.834 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:31:48.834 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.078417+0000", 2024-09-14T06:31:48.835 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.0348e-05 2024-09-14T06:31:48.835 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:48.835 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:31:48.835 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:48.835 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:48.835 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:48.835 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:98524 3.10 3.1bf56f90 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T06:31:48.835 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:30:07.045815+0000", 2024-09-14T06:31:48.835 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 101.711832282, 2024-09-14T06:31:48.835 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032643208999999999, 2024-09-14T06:31:48.835 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:31:48.835 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:31:48.835 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "reached pg", 2024-09-14T06:31:48.835 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:31:48.836 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:31:48.836 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:48.836 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 98524 2024-09-14T06:31:48.836 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:48.836 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:31:48.836 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:48.836 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:31:48.836 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045815+0000", 2024-09-14T06:31:48.836 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:48.836 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:48.836 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:48.836 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:31:48.836 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045815+0000", 2024-09-14T06:31:48.836 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:48.836 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:48.837 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:48.837 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:31:48.837 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045815+0000", 2024-09-14T06:31:48.837 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.11e-07 2024-09-14T06:31:48.837 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:48.837 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:48.837 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:31:48.837 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045816+0000", 2024-09-14T06:31:48.837 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.9099999999999991e-07 2024-09-14T06:31:48.837 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:48.837 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:48.837 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:31:48.837 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045819+0000", 2024-09-14T06:31:48.837 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.9399999999999998e-06 2024-09-14T06:31:48.837 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:48.838 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:48.838 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:31:48.838 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045822+0000", 2024-09-14T06:31:48.838 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.3470000000000001e-06 2024-09-14T06:31:48.838 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:48.838 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:48.838 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:31:48.838 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.078449+0000", 2024-09-14T06:31:48.838 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032627529000000002 2024-09-14T06:31:48.838 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:48.838 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:48.838 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:31:48.838 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.078458+0000", 2024-09-14T06:31:48.838 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.1910000000000006e-06 2024-09-14T06:31:48.838 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:48.838 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:31:48.839 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:48.839 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:48.839 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:31:48.839 INFO:teuthology.orchestra.run.smithi138.stdout:} 2024-09-14T06:31:48.840 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-14T06:31:48.841 INFO:tasks.ceph.osd.3.smithi094.stderr:2024-09-14T06:31:48.839+0000 7feeeafe8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33655) UID: 0 2024-09-14T06:31:48.887 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 4, 5, 6, 7] out_osds: [3] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-14T06:31:48.887 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-14T06:31:48.887 INFO:tasks.thrashosds.thrasher:Adding osd 3 2024-09-14T06:31:48.888 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd in 3 2024-09-14T06:31:48.940 INFO:tasks.ceph.osd.4.smithi138.stderr:2024-09-14T06:31:48.939+0000 7f6c653c8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33615) UID: 0 2024-09-14T06:31:48.984 INFO:teuthology.orchestra.run.smithi138.stdout:{ 2024-09-14T06:31:48.984 INFO:teuthology.orchestra.run.smithi138.stdout: "ops": [], 2024-09-14T06:31:48.984 INFO:teuthology.orchestra.run.smithi138.stdout: "num_ops": 0 2024-09-14T06:31:48.984 INFO:teuthology.orchestra.run.smithi138.stdout:} 2024-09-14T06:31:48.992 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-14T06:31:49.076 INFO:tasks.ceph.osd.7.smithi138.stderr:2024-09-14T06:31:49.075+0000 7f3c6a21d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33623) UID: 0 2024-09-14T06:31:49.135 INFO:teuthology.orchestra.run.smithi138.stdout:{ 2024-09-14T06:31:49.135 INFO:teuthology.orchestra.run.smithi138.stdout: "ops": [], 2024-09-14T06:31:49.135 INFO:teuthology.orchestra.run.smithi138.stdout: "complaint_time": 30, 2024-09-14T06:31:49.135 INFO:teuthology.orchestra.run.smithi138.stdout: "num_blocked_ops": 0 2024-09-14T06:31:49.135 INFO:teuthology.orchestra.run.smithi138.stdout:} 2024-09-14T06:31:49.142 INFO:tasks.ceph.osd.2.smithi094.stderr:2024-09-14T06:31:49.141+0000 7faf3269b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33649) UID: 0 2024-09-14T06:31:49.143 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-14T06:31:49.182 INFO:tasks.ceph.mon.a.smithi094.stderr:2024-09-14T06:31:49.181+0000 7f3f97784640 -1 mon.a@0(leader).osd e184 definitely_dead 0 2024-09-14T06:31:49.244 INFO:tasks.ceph.osd.6.smithi138.stderr:2024-09-14T06:31:49.243+0000 7f8cd8849640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33620) UID: 0 2024-09-14T06:31:49.288 INFO:teuthology.orchestra.run.smithi138.stdout:{ 2024-09-14T06:31:49.289 INFO:teuthology.orchestra.run.smithi138.stdout: "size": 20, 2024-09-14T06:31:49.289 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 600, 2024-09-14T06:31:49.289 INFO:teuthology.orchestra.run.smithi138.stdout: "ops": [ 2024-09-14T06:31:49.289 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:49.289 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:53244 3.f 3:f71f51a7:::rbd_data.112dca2f6ae1.0000000000000015:17 [read 1863680~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:31:49.289 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:44.369399+0000", 2024-09-14T06:31:49.289 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 124.91766678, 2024-09-14T06:31:49.289 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047291189999999997, 2024-09-14T06:31:49.289 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:31:49.289 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:31:49.289 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:31:49.289 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:31:49.289 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:31:49.290 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:49.290 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 53244 2024-09-14T06:31:49.290 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:49.290 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:31:49.290 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:49.290 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:31:49.290 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369399+0000", 2024-09-14T06:31:49.290 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:49.290 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:49.290 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:49.290 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:31:49.290 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369399+0000", 2024-09-14T06:31:49.290 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:49.290 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:49.291 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:49.291 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:31:49.291 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369400+0000", 2024-09-14T06:31:49.291 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.0699999999999999e-07 2024-09-14T06:31:49.291 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:49.291 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:49.291 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:31:49.291 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369401+0000", 2024-09-14T06:31:49.291 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.4300000000000001e-06 2024-09-14T06:31:49.291 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:49.291 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:49.291 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:31:49.291 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369404+0000", 2024-09-14T06:31:49.291 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.061e-06 2024-09-14T06:31:49.292 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:49.292 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:49.292 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:31:49.292 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369407+0000", 2024-09-14T06:31:49.292 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.6139999999999999e-06 2024-09-14T06:31:49.292 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:49.292 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:49.292 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:31:49.292 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.416517+0000", 2024-09-14T06:31:49.292 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047110338000000002 2024-09-14T06:31:49.292 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:49.292 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:49.292 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:31:49.293 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.416616+0000", 2024-09-14T06:31:49.293 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.8651000000000001e-05 2024-09-14T06:31:49.293 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:49.293 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:49.293 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:31:49.293 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.416691+0000", 2024-09-14T06:31:49.293 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.4888999999999994e-05 2024-09-14T06:31:49.293 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:49.293 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:31:49.293 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:49.293 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:49.293 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:49.293 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:53245 3.f 3:f71f51a7:::rbd_data.112dca2f6ae1.0000000000000015:17 [read 1888256~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:31:49.293 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:44.369410+0000", 2024-09-14T06:31:49.294 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 124.91765649200001, 2024-09-14T06:31:49.294 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047399275999999997, 2024-09-14T06:31:49.294 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:31:49.294 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:31:49.294 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:31:49.294 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:31:49.294 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:31:49.294 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:49.294 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 53245 2024-09-14T06:31:49.294 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:49.294 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:31:49.294 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:49.294 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:31:49.294 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369410+0000", 2024-09-14T06:31:49.295 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:49.295 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:49.295 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:49.295 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:31:49.295 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369410+0000", 2024-09-14T06:31:49.295 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:49.295 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:49.295 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:49.295 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:31:49.295 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369410+0000", 2024-09-14T06:31:49.295 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.1899999999999999e-07 2024-09-14T06:31:49.295 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:49.295 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:49.295 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:31:49.296 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369411+0000", 2024-09-14T06:31:49.296 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.0300000000000001e-06 2024-09-14T06:31:49.296 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:49.296 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:49.296 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:31:49.296 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369414+0000", 2024-09-14T06:31:49.296 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.1099999999999999e-06 2024-09-14T06:31:49.296 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:49.296 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:49.296 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:31:49.296 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369417+0000", 2024-09-14T06:31:49.296 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.6189999999999998e-06 2024-09-14T06:31:49.296 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:49.296 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:49.297 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:31:49.297 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.416718+0000", 2024-09-14T06:31:49.297 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047301322999999999 2024-09-14T06:31:49.297 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:49.297 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:49.297 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:31:49.297 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.416759+0000", 2024-09-14T06:31:49.297 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.1009e-05 2024-09-14T06:31:49.297 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:49.297 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:49.297 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:31:49.297 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.416809+0000", 2024-09-14T06:31:49.297 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.9966000000000002e-05 2024-09-14T06:31:49.297 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:49.298 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:31:49.298 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:49.298 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:49.298 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:49.298 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:53246 3.f 3:f71f51a7:::rbd_data.112dca2f6ae1.0000000000000015:17 [read 1912832~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:31:49.298 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:44.369420+0000", 2024-09-14T06:31:49.298 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 124.91764654799999, 2024-09-14T06:31:49.298 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047529463000000001, 2024-09-14T06:31:49.298 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:31:49.298 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:31:49.298 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:31:49.298 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:31:49.298 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:31:49.298 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:49.299 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 53246 2024-09-14T06:31:49.299 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:49.299 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:31:49.299 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:49.299 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:31:49.299 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369420+0000", 2024-09-14T06:31:49.299 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:49.299 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:49.299 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:49.299 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:31:49.299 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369420+0000", 2024-09-14T06:31:49.299 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:49.299 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:49.299 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:49.300 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:31:49.300 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369420+0000", 2024-09-14T06:31:49.300 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.1299999999999999e-07 2024-09-14T06:31:49.300 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:49.300 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:49.300 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:31:49.300 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369425+0000", 2024-09-14T06:31:49.300 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.0130000000000001e-06 2024-09-14T06:31:49.300 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:49.300 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:49.300 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:31:49.300 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369428+0000", 2024-09-14T06:31:49.300 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.349e-06 2024-09-14T06:31:49.300 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:49.301 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:49.301 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:31:49.301 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369431+0000", 2024-09-14T06:31:49.301 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.7389999999999999e-06 2024-09-14T06:31:49.301 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:49.301 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:49.301 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:31:49.301 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.416834+0000", 2024-09-14T06:31:49.301 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047402669000000001 2024-09-14T06:31:49.301 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:49.301 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:49.301 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:31:49.301 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.416882+0000", 2024-09-14T06:31:49.302 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.8068999999999997e-05 2024-09-14T06:31:49.302 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:49.302 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:49.302 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:31:49.302 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.416949+0000", 2024-09-14T06:31:49.302 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.7410999999999995e-05 2024-09-14T06:31:49.302 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:49.302 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:31:49.302 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:49.302 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:49.302 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:49.302 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:53247 3.f 3:f71f51a7:::rbd_data.112dca2f6ae1.0000000000000015:17 [read 1937408~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:31:49.302 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:44.369434+0000", 2024-09-14T06:31:49.302 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 124.91763178399999, 2024-09-14T06:31:49.303 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047652264999999999, 2024-09-14T06:31:49.303 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:31:49.303 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:31:49.303 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:31:49.303 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:31:49.303 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:31:49.303 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:49.303 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 53247 2024-09-14T06:31:49.303 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:49.303 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:31:49.303 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:49.303 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:31:49.303 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369434+0000", 2024-09-14T06:31:49.303 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:49.304 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:49.304 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:49.304 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:31:49.304 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369434+0000", 2024-09-14T06:31:49.304 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:49.304 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:49.304 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:49.304 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:31:49.304 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369435+0000", 2024-09-14T06:31:49.304 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.2100000000000001e-07 2024-09-14T06:31:49.304 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:49.304 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:49.304 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:31:49.304 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369436+0000", 2024-09-14T06:31:49.304 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.122e-06 2024-09-14T06:31:49.305 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:49.305 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:49.305 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:31:49.305 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369439+0000", 2024-09-14T06:31:49.305 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.0400000000000001e-06 2024-09-14T06:31:49.305 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:49.305 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:49.305 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:31:49.305 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369441+0000", 2024-09-14T06:31:49.305 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.728e-06 2024-09-14T06:31:49.305 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:49.305 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:49.305 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:31:49.305 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.416977+0000", 2024-09-14T06:31:49.305 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047535344 2024-09-14T06:31:49.306 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:49.306 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:49.306 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:31:49.306 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.417029+0000", 2024-09-14T06:31:49.306 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.2037999999999999e-05 2024-09-14T06:31:49.306 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:49.306 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:49.306 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:31:49.306 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.417087+0000", 2024-09-14T06:31:49.306 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.7772000000000002e-05 2024-09-14T06:31:49.306 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:49.306 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:31:49.306 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:49.306 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:49.307 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:49.307 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:53248 3.f 3:f71f51a7:::rbd_data.112dca2f6ae1.0000000000000015:17 [read 1961984~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:31:49.307 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:44.369445+0000", 2024-09-14T06:31:49.307 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 124.917621232, 2024-09-14T06:31:49.307 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047798489, 2024-09-14T06:31:49.307 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:31:49.307 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:31:49.307 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:31:49.307 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:31:49.307 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:31:49.307 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:49.307 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 53248 2024-09-14T06:31:49.307 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:49.307 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:31:49.308 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:49.308 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:31:49.308 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369445+0000", 2024-09-14T06:31:49.308 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:49.308 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:49.308 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:49.308 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:31:49.308 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369445+0000", 2024-09-14T06:31:49.308 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:49.308 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:49.308 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:49.308 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:31:49.308 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369445+0000", 2024-09-14T06:31:49.308 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.0699999999999999e-07 2024-09-14T06:31:49.309 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:49.309 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:49.309 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:31:49.309 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369446+0000", 2024-09-14T06:31:49.309 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.1000000000000001e-06 2024-09-14T06:31:49.309 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:49.309 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:49.309 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:31:49.309 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369449+0000", 2024-09-14T06:31:49.309 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.1039999999999998e-06 2024-09-14T06:31:49.310 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:49.310 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:49.310 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:31:49.310 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369452+0000", 2024-09-14T06:31:49.310 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.6199999999999999e-06 2024-09-14T06:31:49.310 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:49.310 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:49.310 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:31:49.310 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.417114+0000", 2024-09-14T06:31:49.310 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047661901999999999 2024-09-14T06:31:49.311 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:49.311 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:49.311 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:31:49.311 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.417181+0000", 2024-09-14T06:31:49.311 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.7385999999999995e-05 2024-09-14T06:31:49.311 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:49.311 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:49.311 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:31:49.311 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.417243+0000", 2024-09-14T06:31:49.311 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.2169999999999996e-05 2024-09-14T06:31:49.311 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:49.312 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:31:49.312 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:49.312 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:49.312 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:49.312 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:53249 3.f 3:f71f51a7:::rbd_data.112dca2f6ae1.0000000000000015:17 [read 1986560~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:31:49.312 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:44.369455+0000", 2024-09-14T06:31:49.312 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 124.917610921, 2024-09-14T06:31:49.312 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047890493999999999, 2024-09-14T06:31:49.312 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:31:49.312 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:31:49.313 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:31:49.313 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:31:49.313 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:31:49.313 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:49.313 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 53249 2024-09-14T06:31:49.313 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:49.313 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:31:49.313 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:49.313 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:31:49.313 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369455+0000", 2024-09-14T06:31:49.313 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:49.313 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:49.314 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:49.314 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:31:49.314 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369455+0000", 2024-09-14T06:31:49.314 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:49.314 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:49.314 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:49.314 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:31:49.314 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369455+0000", 2024-09-14T06:31:49.314 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.0800000000000001e-07 2024-09-14T06:31:49.314 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:49.314 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:49.315 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:31:49.315 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369456+0000", 2024-09-14T06:31:49.315 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.0729999999999999e-06 2024-09-14T06:31:49.315 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:49.315 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:49.315 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:31:49.315 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369460+0000", 2024-09-14T06:31:49.315 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.027e-06 2024-09-14T06:31:49.315 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:49.315 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:49.315 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:31:49.315 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369462+0000", 2024-09-14T06:31:49.316 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.5320000000000001e-06 2024-09-14T06:31:49.316 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:49.316 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:49.316 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:31:49.316 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.417257+0000", 2024-09-14T06:31:49.316 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.04779489 2024-09-14T06:31:49.316 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:49.316 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:49.316 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:31:49.316 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.417296+0000", 2024-09-14T06:31:49.316 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.8658000000000002e-05 2024-09-14T06:31:49.316 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:49.317 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:49.317 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:31:49.317 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.417346+0000", 2024-09-14T06:31:49.317 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.0105999999999998e-05 2024-09-14T06:31:49.317 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:49.317 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:31:49.317 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:49.317 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:49.317 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:49.317 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:53250 3.f 3:f71f51a7:::rbd_data.112dca2f6ae1.0000000000000015:17 [read 2011136~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:31:49.317 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:44.369465+0000", 2024-09-14T06:31:49.317 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 124.917600951, 2024-09-14T06:31:49.317 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.048032230000000002, 2024-09-14T06:31:49.318 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:31:49.318 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:31:49.318 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:31:49.318 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:31:49.318 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:31:49.318 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:49.318 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 53250 2024-09-14T06:31:49.318 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:49.318 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:31:49.318 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:49.318 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:31:49.318 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369465+0000", 2024-09-14T06:31:49.318 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:49.319 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:49.319 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:49.319 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:31:49.319 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369465+0000", 2024-09-14T06:31:49.319 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:49.319 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:49.319 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:49.319 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:31:49.319 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369465+0000", 2024-09-14T06:31:49.319 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.2100000000000001e-07 2024-09-14T06:31:49.319 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:49.319 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:49.320 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:31:49.320 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369466+0000", 2024-09-14T06:31:49.320 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.0640000000000001e-06 2024-09-14T06:31:49.320 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:49.320 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:49.320 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:31:49.320 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369469+0000", 2024-09-14T06:31:49.320 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.0170000000000001e-06 2024-09-14T06:31:49.320 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:49.320 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:49.320 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:31:49.320 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369472+0000", 2024-09-14T06:31:49.320 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.6189999999999998e-06 2024-09-14T06:31:49.321 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:49.321 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:49.321 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:31:49.321 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.417383+0000", 2024-09-14T06:31:49.321 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047911092000000002 2024-09-14T06:31:49.321 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:49.321 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:49.321 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:31:49.321 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.417430+0000", 2024-09-14T06:31:49.321 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.6898999999999999e-05 2024-09-14T06:31:49.321 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:49.321 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:49.321 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:31:49.322 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.417497+0000", 2024-09-14T06:31:49.322 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.7317999999999995e-05 2024-09-14T06:31:49.322 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:49.322 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:31:49.322 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:49.322 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:49.322 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:49.322 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:53251 3.f 3:f71f51a7:::rbd_data.112dca2f6ae1.0000000000000015:17 [read 2035712~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:31:49.322 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:44.369475+0000", 2024-09-14T06:31:49.322 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 124.91759095499999, 2024-09-14T06:31:49.322 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.048154542000000002, 2024-09-14T06:31:49.322 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:31:49.323 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:31:49.323 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:31:49.323 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:31:49.323 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:31:49.323 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:49.323 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 53251 2024-09-14T06:31:49.323 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:49.323 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:31:49.323 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:49.323 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:31:49.323 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369475+0000", 2024-09-14T06:31:49.323 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:49.323 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:49.324 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:49.324 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:31:49.324 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369475+0000", 2024-09-14T06:31:49.324 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:49.324 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:49.324 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:49.324 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:31:49.324 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369475+0000", 2024-09-14T06:31:49.324 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.0100000000000001e-07 2024-09-14T06:31:49.324 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:49.324 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:49.324 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:31:49.325 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369476+0000", 2024-09-14T06:31:49.325 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.0580000000000001e-06 2024-09-14T06:31:49.325 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:49.325 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:49.325 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:31:49.325 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369479+0000", 2024-09-14T06:31:49.325 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.9950000000000001e-06 2024-09-14T06:31:49.325 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:49.325 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:49.325 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:31:49.325 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369482+0000", 2024-09-14T06:31:49.326 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.6340000000000001e-06 2024-09-14T06:31:49.326 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:49.326 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:49.326 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:31:49.326 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.417530+0000", 2024-09-14T06:31:49.326 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.048047535000000002 2024-09-14T06:31:49.326 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:49.326 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:49.326 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:31:49.326 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.417574+0000", 2024-09-14T06:31:49.326 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.4255999999999998e-05 2024-09-14T06:31:49.327 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:49.327 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:49.327 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:31:49.327 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.417630+0000", 2024-09-14T06:31:49.327 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.5862999999999997e-05 2024-09-14T06:31:49.327 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:49.327 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:31:49.327 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:49.327 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:49.327 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:49.327 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:53252 3.f 3:f71f51a7:::rbd_data.112dca2f6ae1.0000000000000015:17 [read 2060288~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:31:49.327 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:44.369485+0000", 2024-09-14T06:31:49.328 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 124.917581107, 2024-09-14T06:31:49.328 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.048254116, 2024-09-14T06:31:49.328 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:31:49.328 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:31:49.328 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:31:49.328 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:31:49.328 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:31:49.328 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:49.328 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 53252 2024-09-14T06:31:49.328 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:49.328 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:31:49.328 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:49.329 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:31:49.329 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369485+0000", 2024-09-14T06:31:49.329 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:49.329 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:49.329 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:49.329 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:31:49.329 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369485+0000", 2024-09-14T06:31:49.329 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:49.329 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:49.329 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:49.330 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:31:49.330 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369485+0000", 2024-09-14T06:31:49.330 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.1e-07 2024-09-14T06:31:49.330 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:49.330 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:49.330 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:31:49.330 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369487+0000", 2024-09-14T06:31:49.330 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.42e-06 2024-09-14T06:31:49.330 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:49.330 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:49.330 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:31:49.330 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369490+0000", 2024-09-14T06:31:49.331 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.9639999999999999e-06 2024-09-14T06:31:49.331 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:49.331 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:49.331 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:31:49.331 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369492+0000", 2024-09-14T06:31:49.331 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.5679999999999998e-06 2024-09-14T06:31:49.331 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:49.331 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:49.331 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:31:49.331 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.417645+0000", 2024-09-14T06:31:49.331 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.048152753 2024-09-14T06:31:49.332 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:49.332 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:49.332 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:31:49.332 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.417687+0000", 2024-09-14T06:31:49.332 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.1900999999999997e-05 2024-09-14T06:31:49.332 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:49.332 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:49.332 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:31:49.332 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.417739+0000", 2024-09-14T06:31:49.332 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.2299999999999997e-05 2024-09-14T06:31:49.332 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:49.333 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:31:49.333 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:49.333 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:49.333 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:49.333 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:53253 3.f 3:f71f51a7:::rbd_data.112dca2f6ae1.0000000000000015:17 [read 2084864~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:31:49.333 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:44.369495+0000", 2024-09-14T06:31:49.333 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 124.917570983, 2024-09-14T06:31:49.333 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.048366785000000002, 2024-09-14T06:31:49.333 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:31:49.333 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:31:49.333 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:31:49.334 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:31:49.334 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:31:49.334 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:49.334 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 53253 2024-09-14T06:31:49.334 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:49.334 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:31:49.334 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:49.334 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:31:49.334 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369495+0000", 2024-09-14T06:31:49.334 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:49.334 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:49.334 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:49.335 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:31:49.335 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369495+0000", 2024-09-14T06:31:49.335 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:49.335 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:49.335 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:49.335 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:31:49.335 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370388+0000", 2024-09-14T06:31:49.335 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00089287999999999998 2024-09-14T06:31:49.335 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:49.335 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:49.337 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:31:49.337 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370392+0000", 2024-09-14T06:31:49.337 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.2130000000000002e-06 2024-09-14T06:31:49.337 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:49.337 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:49.337 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:31:49.337 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370401+0000", 2024-09-14T06:31:49.337 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.7649999999999999e-06 2024-09-14T06:31:49.338 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:49.338 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:49.338 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:31:49.338 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370406+0000", 2024-09-14T06:31:49.338 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.0159999999999999e-06 2024-09-14T06:31:49.338 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:49.338 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:49.338 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:31:49.338 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.417764+0000", 2024-09-14T06:31:49.338 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047358348000000001 2024-09-14T06:31:49.338 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:49.338 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:49.339 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:31:49.339 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.417806+0000", 2024-09-14T06:31:49.339 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.1149999999999997e-05 2024-09-14T06:31:49.339 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:49.339 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:49.339 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:31:49.339 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.417862+0000", 2024-09-14T06:31:49.339 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.6412999999999997e-05 2024-09-14T06:31:49.339 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:49.339 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:31:49.339 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:49.339 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:49.339 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:49.340 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:53340 3.f 3:f71f51a7:::rbd_data.112dca2f6ae1.0000000000000015:17 [read 1056768~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:31:49.340 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:44.370421+0000", 2024-09-14T06:31:49.340 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 124.91664510299999, 2024-09-14T06:31:49.340 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047566313999999998, 2024-09-14T06:31:49.340 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:31:49.340 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:31:49.340 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:31:49.340 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:31:49.340 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:31:49.340 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:49.340 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 53340 2024-09-14T06:31:49.341 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:49.341 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:31:49.341 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:49.341 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:31:49.341 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370421+0000", 2024-09-14T06:31:49.341 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:49.341 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:49.341 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:49.341 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:31:49.341 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370421+0000", 2024-09-14T06:31:49.341 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:49.341 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:49.342 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:49.342 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:31:49.342 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370422+0000", 2024-09-14T06:31:49.342 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.3799999999999996e-07 2024-09-14T06:31:49.342 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:49.342 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:49.342 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:31:49.342 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370424+0000", 2024-09-14T06:31:49.342 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.6919999999999998e-06 2024-09-14T06:31:49.342 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:49.342 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:49.343 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:31:49.343 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370432+0000", 2024-09-14T06:31:49.343 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.2439999999999996e-06 2024-09-14T06:31:49.343 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:49.343 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:49.343 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:31:49.343 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370437+0000", 2024-09-14T06:31:49.343 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.5389999999999998e-06 2024-09-14T06:31:49.343 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:49.343 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:49.343 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:31:49.343 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.417888+0000", 2024-09-14T06:31:49.343 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047450432000000001 2024-09-14T06:31:49.344 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:49.344 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:49.344 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:31:49.344 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.417935+0000", 2024-09-14T06:31:49.344 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.7454999999999999e-05 2024-09-14T06:31:49.344 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:49.344 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:49.344 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:31:49.344 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.417987+0000", 2024-09-14T06:31:49.344 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.2213999999999998e-05 2024-09-14T06:31:49.344 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:49.344 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:31:49.345 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:49.345 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:49.345 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:49.345 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:53341 3.f 3:f71f51a7:::rbd_data.112dca2f6ae1.0000000000000015:17 [read 1081344~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:31:49.345 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:44.370446+0000", 2024-09-14T06:31:49.345 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 124.91661983, 2024-09-14T06:31:49.345 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047665723, 2024-09-14T06:31:49.345 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:31:49.345 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:31:49.345 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:31:49.345 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:31:49.345 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:31:49.346 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:49.346 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 53341 2024-09-14T06:31:49.346 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:49.346 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:31:49.346 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:49.346 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:31:49.346 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370446+0000", 2024-09-14T06:31:49.346 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:49.346 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:49.346 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:49.346 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:31:49.346 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370446+0000", 2024-09-14T06:31:49.347 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:49.347 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:49.347 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:49.347 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:31:49.347 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370447+0000", 2024-09-14T06:31:49.347 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.6100000000000002e-07 2024-09-14T06:31:49.347 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:49.347 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:49.347 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:31:49.347 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370448+0000", 2024-09-14T06:31:49.347 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.7370000000000001e-06 2024-09-14T06:31:49.347 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:49.348 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:49.348 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:31:49.348 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370462+0000", 2024-09-14T06:31:49.348 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.3227000000000001e-05 2024-09-14T06:31:49.348 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:49.348 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:49.348 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:31:49.348 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370467+0000", 2024-09-14T06:31:49.348 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.6540000000000004e-06 2024-09-14T06:31:49.348 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:49.348 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:49.348 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:31:49.349 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.418013+0000", 2024-09-14T06:31:49.349 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047545325999999999 2024-09-14T06:31:49.349 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:49.349 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:49.349 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:31:49.349 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.418054+0000", 2024-09-14T06:31:49.349 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.1696999999999997e-05 2024-09-14T06:31:49.349 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:49.349 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:49.349 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:31:49.349 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.418112+0000", 2024-09-14T06:31:49.349 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.7720999999999998e-05 2024-09-14T06:31:49.350 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:49.350 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:31:49.350 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:49.350 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:49.350 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:49.350 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:53342 3.f 3:f71f51a7:::rbd_data.112dca2f6ae1.0000000000000015:17 [read 1105920~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:31:49.350 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:44.370475+0000", 2024-09-14T06:31:49.350 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 124.916590769, 2024-09-14T06:31:49.350 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047748848000000003, 2024-09-14T06:31:49.350 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:31:49.350 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:31:49.351 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:31:49.351 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:31:49.351 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:31:49.351 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:49.351 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 53342 2024-09-14T06:31:49.351 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:49.351 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:31:49.351 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:49.351 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:31:49.351 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370475+0000", 2024-09-14T06:31:49.351 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:49.352 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:49.352 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:49.352 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:31:49.352 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370475+0000", 2024-09-14T06:31:49.352 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:49.352 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:49.352 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:49.352 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:31:49.352 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370476+0000", 2024-09-14T06:31:49.352 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.4700000000000002e-07 2024-09-14T06:31:49.352 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:49.353 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:49.353 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:31:49.353 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370478+0000", 2024-09-14T06:31:49.353 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.5890000000000001e-06 2024-09-14T06:31:49.353 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:49.353 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:49.353 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:31:49.353 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370486+0000", 2024-09-14T06:31:49.353 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.3470000000000001e-06 2024-09-14T06:31:49.353 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:49.353 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:49.353 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:31:49.354 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370491+0000", 2024-09-14T06:31:49.354 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.5720000000000002e-06 2024-09-14T06:31:49.354 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:49.354 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:49.354 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:31:49.354 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.418135+0000", 2024-09-14T06:31:49.354 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047644094999999997 2024-09-14T06:31:49.354 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:49.354 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:49.354 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:31:49.354 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.418174+0000", 2024-09-14T06:31:49.354 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.8204000000000002e-05 2024-09-14T06:31:49.355 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:49.355 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:49.355 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:31:49.355 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.418224+0000", 2024-09-14T06:31:49.355 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.0593999999999997e-05 2024-09-14T06:31:49.355 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:49.355 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:31:49.355 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:49.355 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:49.355 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:49.355 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:53343 3.f 3:f71f51a7:::rbd_data.112dca2f6ae1.0000000000000015:17 [read 1130496~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:31:49.356 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:44.370498+0000", 2024-09-14T06:31:49.356 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 124.91656789300001, 2024-09-14T06:31:49.356 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047841574999999997, 2024-09-14T06:31:49.356 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:31:49.356 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:31:49.356 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:31:49.356 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:31:49.356 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:31:49.356 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:49.356 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 53343 2024-09-14T06:31:49.356 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:49.356 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:31:49.357 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:49.357 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:31:49.357 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370498+0000", 2024-09-14T06:31:49.357 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:49.357 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:49.357 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:49.357 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:31:49.357 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370498+0000", 2024-09-14T06:31:49.357 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:49.357 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:49.357 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:49.357 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:31:49.358 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370499+0000", 2024-09-14T06:31:49.358 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.9999999999999998e-07 2024-09-14T06:31:49.358 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:49.358 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:49.358 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:31:49.358 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370501+0000", 2024-09-14T06:31:49.358 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.0310000000000001e-06 2024-09-14T06:31:49.358 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:49.358 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:49.358 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:31:49.358 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370507+0000", 2024-09-14T06:31:49.358 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.4169999999999997e-06 2024-09-14T06:31:49.358 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:49.358 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:49.359 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:31:49.359 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370516+0000", 2024-09-14T06:31:49.359 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.7190000000000007e-06 2024-09-14T06:31:49.359 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:49.359 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:49.359 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:31:49.359 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.418247+0000", 2024-09-14T06:31:49.359 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047731190999999999 2024-09-14T06:31:49.359 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:49.359 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:49.359 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:31:49.359 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.418286+0000", 2024-09-14T06:31:49.360 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.8988e-05 2024-09-14T06:31:49.360 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:49.360 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:49.360 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:31:49.360 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.418340+0000", 2024-09-14T06:31:49.360 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.3829000000000002e-05 2024-09-14T06:31:49.360 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:49.360 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:31:49.360 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:49.360 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:49.360 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:49.361 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:53344 3.f 3:f71f51a7:::rbd_data.112dca2f6ae1.0000000000000015:17 [read 1155072~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:31:49.361 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:44.370524+0000", 2024-09-14T06:31:49.361 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 124.916542123, 2024-09-14T06:31:49.361 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047972262000000002, 2024-09-14T06:31:49.361 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:31:49.361 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:31:49.361 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:31:49.361 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:31:49.361 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:31:49.361 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:49.361 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 53344 2024-09-14T06:31:49.362 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:49.362 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:31:49.362 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:49.362 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:31:49.362 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370524+0000", 2024-09-14T06:31:49.362 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:49.362 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:49.362 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:49.362 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:31:49.362 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370524+0000", 2024-09-14T06:31:49.362 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:49.362 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:49.362 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:49.363 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:31:49.363 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370618+0000", 2024-09-14T06:31:49.363 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.4425999999999993e-05 2024-09-14T06:31:49.363 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:49.363 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:49.363 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:31:49.363 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370622+0000", 2024-09-14T06:31:49.363 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.3519999999999998e-06 2024-09-14T06:31:49.363 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:49.363 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:49.363 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:31:49.363 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370631+0000", 2024-09-14T06:31:49.364 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.9299999999999992e-06 2024-09-14T06:31:49.364 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:49.364 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:49.364 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:31:49.364 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370637+0000", 2024-09-14T06:31:49.364 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.3269999999999997e-06 2024-09-14T06:31:49.364 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:49.364 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:49.364 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:31:49.364 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.418372+0000", 2024-09-14T06:31:49.364 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047734764999999998 2024-09-14T06:31:49.365 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:49.365 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:49.365 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:31:49.365 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.418431+0000", 2024-09-14T06:31:49.365 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.8696999999999997e-05 2024-09-14T06:31:49.365 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:49.365 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:49.365 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:31:49.365 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.418496+0000", 2024-09-14T06:31:49.365 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.5765000000000005e-05 2024-09-14T06:31:49.365 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:49.365 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:31:49.366 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:49.366 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:49.366 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:49.366 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:53345 3.f 3:f71f51a7:::rbd_data.112dca2f6ae1.0000000000000015:17 [read 1179648~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:31:49.366 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:44.370647+0000", 2024-09-14T06:31:49.366 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 124.916418656, 2024-09-14T06:31:49.366 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047953775999999997, 2024-09-14T06:31:49.366 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:31:49.366 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:31:49.366 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:31:49.366 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:31:49.366 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:31:49.367 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:49.367 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 53345 2024-09-14T06:31:49.367 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:49.367 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:31:49.367 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:49.367 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:31:49.367 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370647+0000", 2024-09-14T06:31:49.367 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:49.367 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:49.367 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:49.367 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:31:49.367 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370647+0000", 2024-09-14T06:31:49.367 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:49.368 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:49.368 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:49.368 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:31:49.368 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.372346+0000", 2024-09-14T06:31:49.368 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.001698284 2024-09-14T06:31:49.368 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:49.368 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:49.368 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:31:49.368 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.372351+0000", 2024-09-14T06:31:49.368 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.8890000000000001e-06 2024-09-14T06:31:49.368 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:49.369 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:49.369 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:31:49.369 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.372362+0000", 2024-09-14T06:31:49.369 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.0941000000000001e-05 2024-09-14T06:31:49.369 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:49.369 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:49.369 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:31:49.369 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.372369+0000", 2024-09-14T06:31:49.369 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.362e-06 2024-09-14T06:31:49.369 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:49.369 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:49.370 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:31:49.370 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.418510+0000", 2024-09-14T06:31:49.370 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046140914999999998 2024-09-14T06:31:49.370 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:49.370 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:49.370 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:31:49.370 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.418549+0000", 2024-09-14T06:31:49.370 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.8717999999999998e-05 2024-09-14T06:31:49.370 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:49.370 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:49.370 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:31:49.371 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.418601+0000", 2024-09-14T06:31:49.371 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.2667000000000002e-05 2024-09-14T06:31:49.371 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:49.371 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:31:49.371 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:49.371 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:49.371 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:49.371 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:53351 3.f 3:f71f51a7:::rbd_data.112dca2f6ae1.0000000000000015:17 [read 1327104~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:31:49.371 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:44.372499+0000", 2024-09-14T06:31:49.371 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 124.914567535, 2024-09-14T06:31:49.371 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047140596, 2024-09-14T06:31:49.372 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:31:49.372 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:31:49.372 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:31:49.372 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:31:49.372 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:31:49.372 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:49.372 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 53351 2024-09-14T06:31:49.372 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:49.372 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:31:49.372 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:49.372 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:31:49.372 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.372499+0000", 2024-09-14T06:31:49.373 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:49.373 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:49.373 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:49.373 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:31:49.373 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.372499+0000", 2024-09-14T06:31:49.373 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:49.373 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:49.373 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:49.373 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:31:49.373 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.372499+0000", 2024-09-14T06:31:49.373 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.3000000000000001e-07 2024-09-14T06:31:49.374 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:49.374 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:49.374 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:31:49.374 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.372501+0000", 2024-09-14T06:31:49.374 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.0820000000000001e-06 2024-09-14T06:31:49.374 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:49.374 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:49.374 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:31:49.374 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.372513+0000", 2024-09-14T06:31:49.374 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.1610999999999999e-05 2024-09-14T06:31:49.374 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:49.375 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:49.375 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:31:49.375 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.372518+0000", 2024-09-14T06:31:49.375 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.327e-06 2024-09-14T06:31:49.375 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:49.375 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:49.375 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:31:49.375 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.419530+0000", 2024-09-14T06:31:49.375 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047011798 2024-09-14T06:31:49.375 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:49.375 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:49.375 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:31:49.376 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.419578+0000", 2024-09-14T06:31:49.376 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.8446999999999999e-05 2024-09-14T06:31:49.376 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:49.376 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:49.376 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:31:49.376 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.419639+0000", 2024-09-14T06:31:49.376 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.0900999999999998e-05 2024-09-14T06:31:49.376 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:49.376 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:31:49.376 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:49.376 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:49.376 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:49.376 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:53352 3.f 3:f71f51a7:::rbd_data.112dca2f6ae1.0000000000000015:17 [read 1351680~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:31:49.377 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:44.372525+0000", 2024-09-14T06:31:49.377 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 124.914541621, 2024-09-14T06:31:49.377 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047259847000000001, 2024-09-14T06:31:49.377 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:31:49.377 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:31:49.377 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:31:49.377 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:31:49.377 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:31:49.377 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:49.377 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 53352 2024-09-14T06:31:49.377 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:49.377 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:31:49.378 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:49.378 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:31:49.378 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.372525+0000", 2024-09-14T06:31:49.378 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:49.378 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:49.378 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:49.378 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:31:49.378 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.372525+0000", 2024-09-14T06:31:49.378 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:49.378 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:49.378 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:49.378 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:31:49.379 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.372525+0000", 2024-09-14T06:31:49.379 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.0600000000000001e-07 2024-09-14T06:31:49.379 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:49.379 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:49.379 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:31:49.379 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.372527+0000", 2024-09-14T06:31:49.379 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.0499999999999999e-06 2024-09-14T06:31:49.379 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:49.379 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:49.379 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:31:49.379 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.372533+0000", 2024-09-14T06:31:49.379 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.4320000000000004e-06 2024-09-14T06:31:49.380 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:49.380 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:49.380 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:31:49.380 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.372538+0000", 2024-09-14T06:31:49.380 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.9130000000000002e-06 2024-09-14T06:31:49.380 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:49.380 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:49.380 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:31:49.380 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.419667+0000", 2024-09-14T06:31:49.380 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047128976000000003 2024-09-14T06:31:49.380 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:49.380 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:49.381 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:31:49.381 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.419715+0000", 2024-09-14T06:31:49.381 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.7868999999999999e-05 2024-09-14T06:31:49.381 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:49.381 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:49.381 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:31:49.381 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.419784+0000", 2024-09-14T06:31:49.381 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.9201000000000004e-05 2024-09-14T06:31:49.381 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:49.381 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:31:49.381 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:49.381 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:49.382 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:49.382 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:53353 3.f 3:f71f51a7:::rbd_data.112dca2f6ae1.0000000000000015:17 [read 1376256~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:31:49.382 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:44.372545+0000", 2024-09-14T06:31:49.382 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 124.914521581, 2024-09-14T06:31:49.382 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047364232999999999, 2024-09-14T06:31:49.382 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:31:49.382 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:31:49.382 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:31:49.382 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:31:49.382 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:31:49.382 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:49.383 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 53353 2024-09-14T06:31:49.383 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:49.383 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:31:49.383 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:49.383 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:31:49.383 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.372545+0000", 2024-09-14T06:31:49.383 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:49.383 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:49.383 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:49.383 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:31:49.383 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.372545+0000", 2024-09-14T06:31:49.383 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:49.383 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:49.384 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:49.384 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:31:49.384 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.372545+0000", 2024-09-14T06:31:49.384 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.27e-07 2024-09-14T06:31:49.384 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:49.384 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:49.384 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:31:49.384 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.372547+0000", 2024-09-14T06:31:49.384 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.0600000000000002e-06 2024-09-14T06:31:49.384 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:49.384 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:49.384 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:31:49.385 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.372553+0000", 2024-09-14T06:31:49.385 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.7980000000000002e-06 2024-09-14T06:31:49.385 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:49.385 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:49.385 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:31:49.385 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.372557+0000", 2024-09-14T06:31:49.385 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.7389999999999999e-06 2024-09-14T06:31:49.385 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:49.385 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:49.385 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:31:49.385 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.419811+0000", 2024-09-14T06:31:49.385 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047253310999999999 2024-09-14T06:31:49.386 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:49.386 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:49.386 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:31:49.386 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.419854+0000", 2024-09-14T06:31:49.386 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.3448000000000001e-05 2024-09-14T06:31:49.386 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:49.386 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:49.386 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:31:49.386 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.419909+0000", 2024-09-14T06:31:49.386 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.4549999999999998e-05 2024-09-14T06:31:49.386 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:49.386 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:31:49.387 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:49.387 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:49.387 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:49.387 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:53354 3.f 3:f71f51a7:::rbd_data.112dca2f6ae1.0000000000000015:17 [read 1400832~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:31:49.387 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:44.372563+0000", 2024-09-14T06:31:49.387 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 124.91450314399999, 2024-09-14T06:31:49.387 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047482233999999998, 2024-09-14T06:31:49.387 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:31:49.387 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:31:49.387 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:31:49.387 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:31:49.387 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:31:49.387 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:49.388 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 53354 2024-09-14T06:31:49.388 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:49.388 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:31:49.388 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:49.388 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:31:49.388 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.372563+0000", 2024-09-14T06:31:49.388 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:49.388 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:49.388 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:49.388 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:31:49.388 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.372563+0000", 2024-09-14T06:31:49.389 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:49.389 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:49.389 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:49.389 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:31:49.389 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.375009+0000", 2024-09-14T06:31:49.389 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.0024457680000000001 2024-09-14T06:31:49.389 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:49.389 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:49.389 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:31:49.389 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.375014+0000", 2024-09-14T06:31:49.389 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.0599999999999998e-06 2024-09-14T06:31:49.389 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:49.390 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:49.390 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:31:49.390 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.375026+0000", 2024-09-14T06:31:49.390 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.1783999999999999e-05 2024-09-14T06:31:49.390 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:49.390 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:49.390 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:31:49.390 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.375035+0000", 2024-09-14T06:31:49.390 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.7799999999999995e-06 2024-09-14T06:31:49.390 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:49.390 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:49.390 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:31:49.391 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.419942+0000", 2024-09-14T06:31:49.391 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.044906982999999998 2024-09-14T06:31:49.391 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:49.391 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:49.391 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:31:49.391 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.419989+0000", 2024-09-14T06:31:49.391 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.7104999999999997e-05 2024-09-14T06:31:49.391 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:49.391 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:49.391 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:31:49.391 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.420045+0000", 2024-09-14T06:31:49.391 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.5754000000000002e-05 2024-09-14T06:31:49.392 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:49.392 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:31:49.392 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:49.392 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:49.392 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:31:49.392 INFO:teuthology.orchestra.run.smithi138.stdout:} 2024-09-14T06:31:49.394 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-14T06:31:49.394 INFO:tasks.ceph.osd.0.smithi094.stderr:2024-09-14T06:31:49.392+0000 7f07c9333640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33636) UID: 0 2024-09-14T06:31:49.494 INFO:tasks.ceph.osd.5.smithi138.stderr:2024-09-14T06:31:49.493+0000 7fe13746f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33618) UID: 0 2024-09-14T06:31:49.497 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-14T06:31:49.497 INFO:teuthology.orchestra.run.smithi094.stdout: "ops": [], 2024-09-14T06:31:49.497 INFO:teuthology.orchestra.run.smithi094.stdout: "num_ops": 0 2024-09-14T06:31:49.497 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-14T06:31:49.505 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-14T06:31:49.595 INFO:tasks.ceph.osd.1.smithi094.stderr:2024-09-14T06:31:49.594+0000 7f328e47a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33641) UID: 0 2024-09-14T06:31:49.607 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-14T06:31:49.608 INFO:teuthology.orchestra.run.smithi094.stdout: "ops": [], 2024-09-14T06:31:49.608 INFO:teuthology.orchestra.run.smithi094.stdout: "complaint_time": 30, 2024-09-14T06:31:49.608 INFO:teuthology.orchestra.run.smithi094.stdout: "num_blocked_ops": 0 2024-09-14T06:31:49.608 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-14T06:31:49.616 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-14T06:31:49.696 INFO:tasks.ceph.osd.7.smithi138.stderr:2024-09-14T06:31:49.695+0000 7f3c6a21d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33623) UID: 0 2024-09-14T06:31:49.723 INFO:teuthology.orchestra.run.smithi094.stderr:marked in osd.3. 2024-09-14T06:31:49.746 INFO:tasks.thrashosds.thrasher:Added osd 3 2024-09-14T06:31:49.774 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-14T06:31:49.774 INFO:teuthology.orchestra.run.smithi094.stdout: "size": 20, 2024-09-14T06:31:49.774 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 600, 2024-09-14T06:31:49.774 INFO:teuthology.orchestra.run.smithi094.stdout: "ops": [ 2024-09-14T06:31:49.775 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:49.775 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:115193 3.1 3:856be886:::rbd_data.112d4d24c558.0000000000000004:29 [read 1523712~16384 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e96)", 2024-09-14T06:31:49.775 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:30:17.939854+0000", 2024-09-14T06:31:49.775 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 91.832671598999994, 2024-09-14T06:31:49.775 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.073897516999999996, 2024-09-14T06:31:49.775 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:31:49.775 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:31:49.775 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:31:49.775 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:31:49.775 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:31:49.775 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:49.775 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 115193 2024-09-14T06:31:49.775 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:49.775 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:31:49.776 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:49.776 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:31:49.776 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.939854+0000", 2024-09-14T06:31:49.776 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:49.776 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:49.776 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:49.776 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:31:49.776 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.939854+0000", 2024-09-14T06:31:49.776 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:49.776 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:49.776 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:49.776 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:31:49.776 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.939854+0000", 2024-09-14T06:31:49.776 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.9500000000000002e-07 2024-09-14T06:31:49.777 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:49.777 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:49.777 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:31:49.777 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.939856+0000", 2024-09-14T06:31:49.777 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.0959999999999999e-06 2024-09-14T06:31:49.777 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:49.777 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:49.777 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:31:49.777 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.939863+0000", 2024-09-14T06:31:49.777 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.1129999999999997e-06 2024-09-14T06:31:49.777 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:49.777 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:49.777 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:31:49.778 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.939867+0000", 2024-09-14T06:31:49.778 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.8350000000000003e-06 2024-09-14T06:31:49.778 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:49.778 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:49.778 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:31:49.778 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.013626+0000", 2024-09-14T06:31:49.778 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.073758799999999999 2024-09-14T06:31:49.778 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:49.778 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:49.778 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:31:49.778 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.013690+0000", 2024-09-14T06:31:49.778 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.3918999999999996e-05 2024-09-14T06:31:49.778 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:49.778 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:49.779 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:31:49.779 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.013751+0000", 2024-09-14T06:31:49.779 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.1159000000000002e-05 2024-09-14T06:31:49.779 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:49.779 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:31:49.779 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:49.779 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:49.779 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:49.779 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:115194 3.1 3:856be886:::rbd_data.112d4d24c558.0000000000000004:29 [read 1769472~16384 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e96)", 2024-09-14T06:31:49.779 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:30:17.939874+0000", 2024-09-14T06:31:49.779 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 91.832651287999994, 2024-09-14T06:31:49.779 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.074016988000000006, 2024-09-14T06:31:49.779 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:31:49.780 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:31:49.780 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:31:49.780 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:31:49.780 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:31:49.780 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:49.780 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 115194 2024-09-14T06:31:49.780 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:49.780 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:31:49.780 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:49.780 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:31:49.780 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.939874+0000", 2024-09-14T06:31:49.780 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:49.780 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:49.781 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:49.781 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:31:49.781 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.939874+0000", 2024-09-14T06:31:49.781 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:49.781 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:49.781 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:49.781 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:31:49.781 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.939875+0000", 2024-09-14T06:31:49.781 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.2600000000000002e-07 2024-09-14T06:31:49.781 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:49.781 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:49.781 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:31:49.782 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.939877+0000", 2024-09-14T06:31:49.782 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.9420000000000002e-06 2024-09-14T06:31:49.782 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:49.782 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:49.782 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:31:49.782 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.939882+0000", 2024-09-14T06:31:49.782 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.8749999999999997e-06 2024-09-14T06:31:49.782 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:49.782 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:49.782 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:31:49.782 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.939887+0000", 2024-09-14T06:31:49.782 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.741e-06 2024-09-14T06:31:49.782 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:49.782 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:49.783 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:31:49.783 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.013767+0000", 2024-09-14T06:31:49.783 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.073879720999999995 2024-09-14T06:31:49.783 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:49.783 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:49.783 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:31:49.783 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.013832+0000", 2024-09-14T06:31:49.783 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.5312000000000001e-05 2024-09-14T06:31:49.783 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:49.783 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:49.783 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:31:49.783 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.013891+0000", 2024-09-14T06:31:49.783 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.8870999999999999e-05 2024-09-14T06:31:49.783 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:49.784 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:31:49.784 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:49.784 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:49.784 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:49.784 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:115195 3.1 3:856be886:::rbd_data.112d4d24c558.0000000000000004:29 [read 2015232~16384 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e96)", 2024-09-14T06:31:49.784 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:30:17.939903+0000", 2024-09-14T06:31:49.784 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 91.832622103000006, 2024-09-14T06:31:49.784 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.074128199000000006, 2024-09-14T06:31:49.784 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:31:49.784 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:31:49.784 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:31:49.784 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:31:49.784 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:31:49.784 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:49.785 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 115195 2024-09-14T06:31:49.785 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:49.785 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:31:49.785 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:49.785 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:31:49.785 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.939903+0000", 2024-09-14T06:31:49.785 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:49.785 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:49.785 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:49.785 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:31:49.785 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.939903+0000", 2024-09-14T06:31:49.785 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:49.785 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:49.786 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:49.786 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:31:49.786 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.939904+0000", 2024-09-14T06:31:49.786 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.5799999999999999e-07 2024-09-14T06:31:49.786 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:49.786 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:49.786 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:31:49.786 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.939906+0000", 2024-09-14T06:31:49.786 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.2350000000000002e-06 2024-09-14T06:31:49.786 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:49.786 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:49.786 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:31:49.787 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.939913+0000", 2024-09-14T06:31:49.787 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.5649999999999998e-06 2024-09-14T06:31:49.787 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:49.787 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:49.787 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:31:49.787 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.939918+0000", 2024-09-14T06:31:49.787 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.784e-06 2024-09-14T06:31:49.787 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:49.787 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:49.787 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:31:49.787 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.013921+0000", 2024-09-14T06:31:49.787 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.074002911000000005 2024-09-14T06:31:49.787 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:49.788 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:49.788 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:31:49.788 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.013971+0000", 2024-09-14T06:31:49.788 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.9551e-05 2024-09-14T06:31:49.788 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:49.788 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:49.788 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:31:49.788 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.014031+0000", 2024-09-14T06:31:49.788 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.0594999999999998e-05 2024-09-14T06:31:49.788 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:49.788 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:31:49.788 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:49.788 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:49.788 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:49.789 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:115196 3.1 3:856be886:::rbd_data.112d4d24c558.0000000000000004:29 [read 2260992~16384 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e96)", 2024-09-14T06:31:49.789 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:30:17.939925+0000", 2024-09-14T06:31:49.789 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 91.832599911000003, 2024-09-14T06:31:49.789 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.074320137999999994, 2024-09-14T06:31:49.789 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:31:49.789 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:31:49.789 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:31:49.789 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:31:49.789 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:31:49.789 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:49.789 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 115196 2024-09-14T06:31:49.789 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:49.790 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:31:49.790 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:49.790 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:31:49.790 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.939925+0000", 2024-09-14T06:31:49.790 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:49.790 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:49.790 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:49.790 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:31:49.790 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.939925+0000", 2024-09-14T06:31:49.790 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:49.790 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:49.790 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:49.790 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:31:49.790 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.939926+0000", 2024-09-14T06:31:49.791 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.9999999999999998e-07 2024-09-14T06:31:49.791 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:49.791 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:49.791 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:31:49.791 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.939928+0000", 2024-09-14T06:31:49.791 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.9549999999999999e-06 2024-09-14T06:31:49.791 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:49.791 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:49.791 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:31:49.791 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.939934+0000", 2024-09-14T06:31:49.791 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.127e-06 2024-09-14T06:31:49.791 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:49.791 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:49.792 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:31:49.792 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.939939+0000", 2024-09-14T06:31:49.792 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.775e-06 2024-09-14T06:31:49.792 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:49.792 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:49.792 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:31:49.792 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.014075+0000", 2024-09-14T06:31:49.792 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.074136598999999997 2024-09-14T06:31:49.792 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:49.792 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:49.792 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:31:49.792 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.014162+0000", 2024-09-14T06:31:49.792 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.6477999999999994e-05 2024-09-14T06:31:49.793 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:49.793 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:49.793 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:31:49.793 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.014246+0000", 2024-09-14T06:31:49.793 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.3703999999999997e-05 2024-09-14T06:31:49.793 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:49.793 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:31:49.793 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:49.793 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:49.793 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:49.793 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:115197 3.1 3:856be886:::rbd_data.112d4d24c558.0000000000000004:29 [read 2506752~16384 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e96)", 2024-09-14T06:31:49.793 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:30:17.939969+0000", 2024-09-14T06:31:49.793 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 91.832556425999996, 2024-09-14T06:31:49.794 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.074528544000000002, 2024-09-14T06:31:49.794 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:31:49.794 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:31:49.794 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:31:49.794 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:31:49.794 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:31:49.794 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:49.794 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 115197 2024-09-14T06:31:49.794 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:49.794 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:31:49.794 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:49.794 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:31:49.794 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.939969+0000", 2024-09-14T06:31:49.795 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:49.795 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:49.795 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:49.795 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:31:49.795 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.939969+0000", 2024-09-14T06:31:49.795 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:49.795 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:49.795 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:49.795 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:31:49.795 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.939970+0000", 2024-09-14T06:31:49.795 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.3600000000000003e-07 2024-09-14T06:31:49.795 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:49.795 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:49.796 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:31:49.796 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.939972+0000", 2024-09-14T06:31:49.796 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.3180000000000001e-06 2024-09-14T06:31:49.796 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:49.796 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:49.796 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:31:49.796 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.939979+0000", 2024-09-14T06:31:49.796 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.8800000000000002e-06 2024-09-14T06:31:49.796 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:49.796 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:49.796 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:31:49.796 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.939984+0000", 2024-09-14T06:31:49.797 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.3510000000000001e-06 2024-09-14T06:31:49.797 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:49.797 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:49.797 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:31:49.797 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.014302+0000", 2024-09-14T06:31:49.797 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.074317416999999997 2024-09-14T06:31:49.797 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:49.797 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:49.797 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:31:49.797 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.014392+0000", 2024-09-14T06:31:49.797 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.0181999999999998e-05 2024-09-14T06:31:49.797 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:49.797 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:49.798 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:31:49.798 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.014497+0000", 2024-09-14T06:31:49.798 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00010576 2024-09-14T06:31:49.798 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:49.798 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:31:49.798 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:49.798 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:49.798 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:49.798 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:115198 3.1 3:856be886:::rbd_data.112d4d24c558.0000000000000004:29 [read 2752512~16384 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e96)", 2024-09-14T06:31:49.798 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:30:17.940004+0000", 2024-09-14T06:31:49.798 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 91.832521146000005, 2024-09-14T06:31:49.798 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.074736367999999997, 2024-09-14T06:31:49.798 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:31:49.798 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:31:49.798 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:31:49.799 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:31:49.799 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:31:49.799 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:49.799 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 115198 2024-09-14T06:31:49.799 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:49.799 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:31:49.799 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:49.799 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:31:49.799 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940004+0000", 2024-09-14T06:31:49.799 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:49.799 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:49.799 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:49.799 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:31:49.799 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940004+0000", 2024-09-14T06:31:49.800 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:49.800 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:49.800 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:49.800 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:31:49.800 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940005+0000", 2024-09-14T06:31:49.800 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.2200000000000004e-07 2024-09-14T06:31:49.800 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:49.800 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:49.800 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:31:49.800 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940007+0000", 2024-09-14T06:31:49.800 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.322e-06 2024-09-14T06:31:49.800 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:49.800 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:49.800 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:31:49.800 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940014+0000", 2024-09-14T06:31:49.801 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.6220000000000003e-06 2024-09-14T06:31:49.801 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:49.801 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:49.801 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:31:49.801 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940019+0000", 2024-09-14T06:31:49.801 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.6380000000000001e-06 2024-09-14T06:31:49.801 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:49.801 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:49.801 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:31:49.801 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.014562+0000", 2024-09-14T06:31:49.801 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.074542320999999995 2024-09-14T06:31:49.801 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:49.801 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:49.802 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:31:49.802 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.014646+0000", 2024-09-14T06:31:49.802 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.4079999999999995e-05 2024-09-14T06:31:49.802 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:49.802 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:49.802 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:31:49.802 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.014741+0000", 2024-09-14T06:31:49.802 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.4763000000000001e-05 2024-09-14T06:31:49.802 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:49.802 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:31:49.802 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:49.802 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:49.802 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:49.802 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:115199 3.1 3:856be886:::rbd_data.112d4d24c558.0000000000000004:29 [read 2998272~16384 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e96)", 2024-09-14T06:31:49.803 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:30:17.940040+0000", 2024-09-14T06:31:49.803 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 91.832485391000006, 2024-09-14T06:31:49.803 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.074957134999999994, 2024-09-14T06:31:49.803 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:31:49.803 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:31:49.803 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:31:49.803 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:31:49.803 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:31:49.803 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:49.803 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 115199 2024-09-14T06:31:49.803 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:49.803 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:31:49.803 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:49.803 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:31:49.804 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940040+0000", 2024-09-14T06:31:49.804 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:49.804 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:49.804 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:49.804 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:31:49.804 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940040+0000", 2024-09-14T06:31:49.804 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:49.804 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:49.804 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:49.804 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:31:49.805 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940041+0000", 2024-09-14T06:31:49.805 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.4000000000000001e-07 2024-09-14T06:31:49.805 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:49.805 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:49.805 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:31:49.805 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940046+0000", 2024-09-14T06:31:49.805 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.7250000000000002e-06 2024-09-14T06:31:49.805 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:49.805 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:49.805 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:31:49.805 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940053+0000", 2024-09-14T06:31:49.805 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.6109999999999999e-06 2024-09-14T06:31:49.805 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:49.805 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:49.806 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:31:49.806 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940058+0000", 2024-09-14T06:31:49.806 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.1590000000000001e-06 2024-09-14T06:31:49.806 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:49.806 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:49.806 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:31:49.806 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.014768+0000", 2024-09-14T06:31:49.806 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.074709624000000002 2024-09-14T06:31:49.806 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:49.806 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:49.806 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:31:49.806 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.014907+0000", 2024-09-14T06:31:49.806 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000139665 2024-09-14T06:31:49.806 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:49.806 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:49.807 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:31:49.807 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.014997+0000", 2024-09-14T06:31:49.807 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.9710999999999995e-05 2024-09-14T06:31:49.807 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:49.807 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:31:49.807 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:49.807 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:49.807 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:49.807 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:115200 3.1 3:856be886:::rbd_data.112d4d24c558.0000000000000004:29 [read 3244032~16384 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e96)", 2024-09-14T06:31:49.807 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:30:17.940080+0000", 2024-09-14T06:31:49.807 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 91.832445186000001, 2024-09-14T06:31:49.807 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.075137291999999994, 2024-09-14T06:31:49.807 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:31:49.808 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:31:49.808 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:31:49.808 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:31:49.808 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:31:49.808 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:49.808 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 115200 2024-09-14T06:31:49.808 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:49.808 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:31:49.808 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:49.808 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:31:49.808 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940080+0000", 2024-09-14T06:31:49.808 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:49.808 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:49.809 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:49.809 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:31:49.809 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940080+0000", 2024-09-14T06:31:49.809 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:49.809 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:49.809 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:49.809 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:31:49.809 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940081+0000", 2024-09-14T06:31:49.809 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.0800000000000004e-07 2024-09-14T06:31:49.809 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:49.809 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:49.809 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:31:49.809 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940083+0000", 2024-09-14T06:31:49.809 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.3920000000000001e-06 2024-09-14T06:31:49.810 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:49.810 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:49.810 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:31:49.810 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940091+0000", 2024-09-14T06:31:49.810 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.1389999999999995e-06 2024-09-14T06:31:49.810 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:49.810 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:49.810 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:31:49.810 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940097+0000", 2024-09-14T06:31:49.810 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.5550000000000002e-06 2024-09-14T06:31:49.810 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:49.810 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:49.810 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:31:49.810 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.015025+0000", 2024-09-14T06:31:49.810 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.074928579999999995 2024-09-14T06:31:49.810 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:49.811 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:49.811 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:31:49.811 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.015096+0000", 2024-09-14T06:31:49.811 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.0705999999999997e-05 2024-09-14T06:31:49.811 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:49.811 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:49.811 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:31:49.811 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.015217+0000", 2024-09-14T06:31:49.811 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00012131200000000001 2024-09-14T06:31:49.811 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:49.811 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:31:49.811 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:49.811 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:49.812 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:49.812 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:115201 3.1 3:856be886:::rbd_data.112d4d24c558.0000000000000004:29 [read 3489792~16384 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e96)", 2024-09-14T06:31:49.812 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:30:17.940113+0000", 2024-09-14T06:31:49.812 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 91.832412413, 2024-09-14T06:31:49.812 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.075246899000000006, 2024-09-14T06:31:49.812 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:31:49.812 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:31:49.812 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:31:49.812 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:31:49.812 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:31:49.812 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:49.812 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 115201 2024-09-14T06:31:49.812 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:49.812 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:31:49.813 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:49.813 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:31:49.813 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940113+0000", 2024-09-14T06:31:49.813 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:49.813 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:49.813 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:49.813 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:31:49.813 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940113+0000", 2024-09-14T06:31:49.813 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:49.813 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:49.813 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:49.813 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:31:49.813 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940114+0000", 2024-09-14T06:31:49.813 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.1500000000000004e-07 2024-09-14T06:31:49.814 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:49.814 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:49.814 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:31:49.814 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940116+0000", 2024-09-14T06:31:49.814 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.2809999999999998e-06 2024-09-14T06:31:49.814 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:49.814 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:49.814 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:31:49.814 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940123+0000", 2024-09-14T06:31:49.814 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.9229999999999996e-06 2024-09-14T06:31:49.814 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:49.814 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:49.814 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:31:49.814 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940128+0000", 2024-09-14T06:31:49.815 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.7790000000000003e-06 2024-09-14T06:31:49.815 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:49.815 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:49.815 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:31:49.815 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.015253+0000", 2024-09-14T06:31:49.815 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.075125792999999996 2024-09-14T06:31:49.815 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:49.815 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:49.815 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:31:49.815 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.015297+0000", 2024-09-14T06:31:49.815 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.3847000000000002e-05 2024-09-14T06:31:49.815 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:49.815 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:49.816 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:31:49.816 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.015360+0000", 2024-09-14T06:31:49.816 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.2661000000000002e-05 2024-09-14T06:31:49.816 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:49.816 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:31:49.816 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:49.816 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:49.816 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:49.816 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:115202 3.1 3:856be886:::rbd_data.112d4d24c558.0000000000000004:29 [read 3735552~16384 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e96)", 2024-09-14T06:31:49.816 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:30:17.940135+0000", 2024-09-14T06:31:49.816 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 91.832390634000006, 2024-09-14T06:31:49.816 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.075451105000000004, 2024-09-14T06:31:49.816 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:31:49.816 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:31:49.817 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:31:49.817 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:31:49.817 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:31:49.817 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:49.817 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 115202 2024-09-14T06:31:49.817 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:49.817 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:31:49.817 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:49.817 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:31:49.817 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940135+0000", 2024-09-14T06:31:49.817 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:49.817 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:49.817 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:49.818 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:31:49.818 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940135+0000", 2024-09-14T06:31:49.818 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:49.818 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:49.818 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:49.818 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:31:49.818 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940135+0000", 2024-09-14T06:31:49.818 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.0999999999999999e-07 2024-09-14T06:31:49.818 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:49.818 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:49.818 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:31:49.818 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940137+0000", 2024-09-14T06:31:49.818 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.0020000000000001e-06 2024-09-14T06:31:49.818 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:49.819 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:49.819 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:31:49.819 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940143+0000", 2024-09-14T06:31:49.819 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.9850000000000003e-06 2024-09-14T06:31:49.819 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:49.819 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:49.819 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:31:49.819 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940148+0000", 2024-09-14T06:31:49.819 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.9880000000000004e-06 2024-09-14T06:31:49.819 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:49.819 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:49.819 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:31:49.819 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.015400+0000", 2024-09-14T06:31:49.819 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.075252186999999998 2024-09-14T06:31:49.819 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:49.820 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:49.820 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:31:49.820 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.015481+0000", 2024-09-14T06:31:49.820 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.0698000000000006e-05 2024-09-14T06:31:49.820 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:49.820 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:49.820 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:31:49.820 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.015586+0000", 2024-09-14T06:31:49.820 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00010483499999999999 2024-09-14T06:31:49.820 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:49.820 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:31:49.820 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:49.820 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:49.820 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:49.821 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:115203 3.1 3:856be886:::rbd_data.112d4d24c558.0000000000000004:29 [read 3981312~16384 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e96)", 2024-09-14T06:31:49.821 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:30:17.940156+0000", 2024-09-14T06:31:49.821 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 91.832369408999995, 2024-09-14T06:31:49.821 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.075620860999999998, 2024-09-14T06:31:49.821 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:31:49.821 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:31:49.821 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:31:49.821 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:31:49.821 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:31:49.821 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:49.821 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 115203 2024-09-14T06:31:49.821 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:49.821 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:31:49.821 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:49.822 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:31:49.822 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940156+0000", 2024-09-14T06:31:49.822 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:49.822 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:49.822 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:49.822 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:31:49.822 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940156+0000", 2024-09-14T06:31:49.822 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:49.822 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:49.822 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:49.822 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:31:49.822 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940156+0000", 2024-09-14T06:31:49.822 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.1500000000000005e-07 2024-09-14T06:31:49.822 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:49.822 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:49.823 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:31:49.823 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940158+0000", 2024-09-14T06:31:49.823 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.967e-06 2024-09-14T06:31:49.823 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:49.823 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:49.823 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:31:49.823 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940165+0000", 2024-09-14T06:31:49.823 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.6939999999999998e-06 2024-09-14T06:31:49.823 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:49.823 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:49.823 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:31:49.823 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940170+0000", 2024-09-14T06:31:49.823 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.7539999999999997e-06 2024-09-14T06:31:49.823 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:49.823 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:49.824 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:31:49.824 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.015633+0000", 2024-09-14T06:31:49.824 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.075463334000000007 2024-09-14T06:31:49.824 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:49.824 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:49.824 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:31:49.824 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.015693+0000", 2024-09-14T06:31:49.824 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.9367000000000003e-05 2024-09-14T06:31:49.824 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:49.824 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:49.824 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:31:49.824 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.015777+0000", 2024-09-14T06:31:49.824 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.4229999999999998e-05 2024-09-14T06:31:49.824 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:49.824 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:31:49.825 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:49.825 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:49.825 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:49.825 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:115204 3.1 3:856be886:::rbd_data.112d4d24c558.0000000000000004:29 [read 4227072~16384 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e96)", 2024-09-14T06:31:49.825 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:30:17.940176+0000", 2024-09-14T06:31:49.825 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 91.832348875999998, 2024-09-14T06:31:49.825 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.075785505000000003, 2024-09-14T06:31:49.825 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:31:49.825 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:31:49.825 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:31:49.825 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:31:49.825 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:31:49.825 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:49.826 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 115204 2024-09-14T06:31:49.826 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:49.826 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:31:49.826 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:49.826 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:31:49.826 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940176+0000", 2024-09-14T06:31:49.826 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:49.826 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:49.826 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:49.826 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:31:49.826 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940176+0000", 2024-09-14T06:31:49.826 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:49.826 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:49.826 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:49.826 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:31:49.826 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940177+0000", 2024-09-14T06:31:49.827 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.7899999999999999e-07 2024-09-14T06:31:49.827 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:49.827 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:49.827 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:31:49.827 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940179+0000", 2024-09-14T06:31:49.827 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.103e-06 2024-09-14T06:31:49.827 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:49.827 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:49.827 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:31:49.827 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940185+0000", 2024-09-14T06:31:49.827 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.7239999999999997e-06 2024-09-14T06:31:49.827 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:49.827 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:49.827 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:31:49.827 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940190+0000", 2024-09-14T06:31:49.828 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.7269999999999998e-06 2024-09-14T06:31:49.828 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:49.828 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:49.828 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:31:49.828 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.015814+0000", 2024-09-14T06:31:49.828 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.075624114000000006 2024-09-14T06:31:49.828 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:49.828 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:49.828 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:31:49.828 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.015862+0000", 2024-09-14T06:31:49.828 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.8770999999999998e-05 2024-09-14T06:31:49.828 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:49.828 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:49.828 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:31:49.829 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.015962+0000", 2024-09-14T06:31:49.829 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.9586999999999994e-05 2024-09-14T06:31:49.829 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:49.829 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:31:49.829 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:49.829 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:49.829 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:49.829 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:115205 3.1 3:856be886:::rbd_data.112d4d24c558.0000000000000004:29 [read 4472832~16384 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e96)", 2024-09-14T06:31:49.829 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:30:17.940197+0000", 2024-09-14T06:31:49.829 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 91.832328167, 2024-09-14T06:31:49.829 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.076068920999999998, 2024-09-14T06:31:49.829 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:31:49.829 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:31:49.829 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:31:49.830 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:31:49.830 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:31:49.830 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:49.830 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 115205 2024-09-14T06:31:49.830 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:49.830 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:31:49.830 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:49.830 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:31:49.830 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940197+0000", 2024-09-14T06:31:49.830 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:49.830 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:49.830 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:49.830 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:31:49.831 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940197+0000", 2024-09-14T06:31:49.831 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:49.831 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:49.831 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:49.831 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:31:49.831 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940198+0000", 2024-09-14T06:31:49.831 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.5999999999999999e-07 2024-09-14T06:31:49.831 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:49.831 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:49.831 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:31:49.831 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940200+0000", 2024-09-14T06:31:49.831 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.0899999999999999e-06 2024-09-14T06:31:49.831 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:49.832 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:49.832 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:31:49.832 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940206+0000", 2024-09-14T06:31:49.832 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.8710000000000002e-06 2024-09-14T06:31:49.832 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:49.832 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:49.832 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:31:49.832 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940211+0000", 2024-09-14T06:31:49.832 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.9230000000000001e-06 2024-09-14T06:31:49.832 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:49.832 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:49.832 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:31:49.832 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.016059+0000", 2024-09-14T06:31:49.832 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.075848051999999999 2024-09-14T06:31:49.833 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:49.833 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:49.833 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:31:49.833 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.016151+0000", 2024-09-14T06:31:49.833 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.1982000000000001e-05 2024-09-14T06:31:49.833 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:49.833 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:49.833 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:31:49.833 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.016266+0000", 2024-09-14T06:31:49.833 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000115543 2024-09-14T06:31:49.833 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:49.833 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:31:49.833 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:49.834 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:49.834 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:49.834 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:115206 3.1 3:856be886:::rbd_data.112d4d24c558.0000000000000004:29 [read 4718592~16384 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e96)", 2024-09-14T06:31:49.834 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:30:17.940223+0000", 2024-09-14T06:31:49.834 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 91.832302076999994, 2024-09-14T06:31:49.834 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.076265115999999994, 2024-09-14T06:31:49.834 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:31:49.834 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:31:49.834 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:31:49.834 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:31:49.834 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:31:49.834 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:49.834 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 115206 2024-09-14T06:31:49.834 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:49.834 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:31:49.835 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:49.835 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:31:49.835 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940223+0000", 2024-09-14T06:31:49.835 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:49.835 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:49.835 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:49.835 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:31:49.835 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940223+0000", 2024-09-14T06:31:49.835 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:49.835 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:49.835 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:49.835 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:31:49.835 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940224+0000", 2024-09-14T06:31:49.835 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.0399999999999996e-07 2024-09-14T06:31:49.836 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:49.836 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:49.836 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:31:49.836 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940226+0000", 2024-09-14T06:31:49.836 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.1780000000000002e-06 2024-09-14T06:31:49.836 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:49.836 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:49.836 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:31:49.836 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940232+0000", 2024-09-14T06:31:49.836 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.5010000000000001e-06 2024-09-14T06:31:49.836 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:49.836 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:49.836 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:31:49.837 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940237+0000", 2024-09-14T06:31:49.837 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.0250000000000002e-06 2024-09-14T06:31:49.837 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:49.837 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:49.837 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:31:49.837 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.016305+0000", 2024-09-14T06:31:49.837 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.076067262999999996 2024-09-14T06:31:49.837 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:49.837 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:49.837 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:31:49.837 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.016383+0000", 2024-09-14T06:31:49.837 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.8578000000000006e-05 2024-09-14T06:31:49.837 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:49.838 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:49.838 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:31:49.838 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.016488+0000", 2024-09-14T06:31:49.838 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000105067 2024-09-14T06:31:49.838 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:49.838 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:31:49.838 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:49.838 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:49.838 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:49.838 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:115207 3.1 3:856be886:::rbd_data.112d4d24c558.0000000000000004:29 [read 4964352~16384 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e96)", 2024-09-14T06:31:49.838 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:30:17.940244+0000", 2024-09-14T06:31:49.838 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 91.832280960999995, 2024-09-14T06:31:49.839 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.076371154999999996, 2024-09-14T06:31:49.839 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:31:49.839 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:31:49.839 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:31:49.839 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:31:49.839 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:31:49.839 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:49.839 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 115207 2024-09-14T06:31:49.839 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:49.839 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:31:49.839 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:49.839 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:31:49.840 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940244+0000", 2024-09-14T06:31:49.840 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:49.840 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:49.840 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:49.840 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:31:49.840 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940244+0000", 2024-09-14T06:31:49.840 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:49.840 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:49.840 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:49.840 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:31:49.840 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940245+0000", 2024-09-14T06:31:49.840 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.1200000000000003e-07 2024-09-14T06:31:49.840 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:49.841 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:49.841 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:31:49.841 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940247+0000", 2024-09-14T06:31:49.841 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.0030000000000001e-06 2024-09-14T06:31:49.841 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:49.841 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:49.841 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:31:49.841 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940252+0000", 2024-09-14T06:31:49.841 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.5520000000000004e-06 2024-09-14T06:31:49.841 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:49.841 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:49.841 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:31:49.841 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940257+0000", 2024-09-14T06:31:49.842 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.6759999999999998e-06 2024-09-14T06:31:49.842 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:49.842 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:49.842 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:31:49.842 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.016521+0000", 2024-09-14T06:31:49.842 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.076263499999999998 2024-09-14T06:31:49.842 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:49.842 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:49.842 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:31:49.842 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.016562+0000", 2024-09-14T06:31:49.842 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.0967000000000003e-05 2024-09-14T06:31:49.842 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:49.842 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:49.842 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:31:49.842 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.016616+0000", 2024-09-14T06:31:49.843 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.3944999999999999e-05 2024-09-14T06:31:49.843 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:49.843 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:31:49.843 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:49.843 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:49.843 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:49.843 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:115208 3.1 3:856be886:::rbd_data.112d4d24c558.0000000000000004:29 [read 5210112~16384 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e96)", 2024-09-14T06:31:49.843 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:30:17.940264+0000", 2024-09-14T06:31:49.843 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 91.832261103999997, 2024-09-14T06:31:49.843 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.076558288000000002, 2024-09-14T06:31:49.843 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:31:49.843 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:31:49.843 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:31:49.843 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:31:49.843 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:31:49.844 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:49.844 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 115208 2024-09-14T06:31:49.844 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:49.844 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:31:49.844 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:49.844 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:31:49.844 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940264+0000", 2024-09-14T06:31:49.844 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:49.844 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:49.844 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:49.844 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:31:49.844 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940264+0000", 2024-09-14T06:31:49.844 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:49.844 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:49.845 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:49.845 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:31:49.845 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940265+0000", 2024-09-14T06:31:49.845 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.0500000000000004e-07 2024-09-14T06:31:49.845 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:49.845 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:49.845 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:31:49.845 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940267+0000", 2024-09-14T06:31:49.845 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.9810000000000002e-06 2024-09-14T06:31:49.845 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:49.845 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:49.845 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:31:49.845 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940272+0000", 2024-09-14T06:31:49.845 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.6999999999999996e-06 2024-09-14T06:31:49.846 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:49.846 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:49.846 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:31:49.846 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940277+0000", 2024-09-14T06:31:49.846 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.6480000000000002e-06 2024-09-14T06:31:49.846 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:49.846 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:49.846 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:31:49.846 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.016650+0000", 2024-09-14T06:31:49.846 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.076373089000000005 2024-09-14T06:31:49.846 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:49.846 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:49.846 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:31:49.846 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.016724+0000", 2024-09-14T06:31:49.846 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.4105999999999998e-05 2024-09-14T06:31:49.847 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:49.847 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:49.847 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:31:49.847 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.016823+0000", 2024-09-14T06:31:49.847 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.8258999999999996e-05 2024-09-14T06:31:49.847 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:49.847 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:31:49.847 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:49.847 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:49.847 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:49.847 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:115209 3.1 3:856be886:::rbd_data.112d4d24c558.0000000000000004:29 [read 5455872~16384 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e96)", 2024-09-14T06:31:49.847 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:30:17.940283+0000", 2024-09-14T06:31:49.847 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 91.832242077999993, 2024-09-14T06:31:49.847 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.076724029999999999, 2024-09-14T06:31:49.848 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:31:49.848 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:31:49.848 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:31:49.848 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:31:49.848 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:31:49.848 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:49.848 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 115209 2024-09-14T06:31:49.848 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:49.848 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:31:49.848 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:49.848 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:31:49.848 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940283+0000", 2024-09-14T06:31:49.848 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:49.848 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:49.848 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:49.849 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:31:49.849 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940283+0000", 2024-09-14T06:31:49.849 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:49.849 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:49.849 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:49.849 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:31:49.849 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940284+0000", 2024-09-14T06:31:49.849 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.6700000000000003e-07 2024-09-14T06:31:49.849 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:49.849 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:49.849 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:31:49.849 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940286+0000", 2024-09-14T06:31:49.849 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.9520000000000001e-06 2024-09-14T06:31:49.849 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:49.850 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:49.850 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:31:49.850 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940292+0000", 2024-09-14T06:31:49.850 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.7339999999999996e-06 2024-09-14T06:31:49.850 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:49.850 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:49.850 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:31:49.850 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940296+0000", 2024-09-14T06:31:49.850 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.7700000000000001e-06 2024-09-14T06:31:49.850 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:49.850 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:49.850 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:31:49.850 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.016859+0000", 2024-09-14T06:31:49.850 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.07656317 2024-09-14T06:31:49.850 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:49.850 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:49.851 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:31:49.851 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.016929+0000", 2024-09-14T06:31:49.851 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.9679999999999997e-05 2024-09-14T06:31:49.851 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:49.851 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:49.851 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:31:49.851 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.017007+0000", 2024-09-14T06:31:49.851 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.8157000000000004e-05 2024-09-14T06:31:49.851 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:49.851 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:31:49.851 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:49.851 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:49.851 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:49.851 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:115210 3.1 3:856be886:::rbd_data.112d4d24c558.0000000000000004:29 [read 5701632~16384 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e96)", 2024-09-14T06:31:49.852 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:30:17.940303+0000", 2024-09-14T06:31:49.852 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 91.832222560999995, 2024-09-14T06:31:49.852 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.076821358000000006, 2024-09-14T06:31:49.852 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:31:49.852 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:31:49.852 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:31:49.852 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:31:49.852 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:31:49.852 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:49.852 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 115210 2024-09-14T06:31:49.852 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:49.852 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:31:49.852 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:49.852 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:31:49.852 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940303+0000", 2024-09-14T06:31:49.853 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:49.853 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:49.853 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:49.853 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:31:49.853 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940303+0000", 2024-09-14T06:31:49.853 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:49.853 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:49.853 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:49.853 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:31:49.853 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940303+0000", 2024-09-14T06:31:49.853 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.3000000000000001e-07 2024-09-14T06:31:49.853 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:49.853 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:49.853 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:31:49.854 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940306+0000", 2024-09-14T06:31:49.854 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.255e-06 2024-09-14T06:31:49.854 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:49.854 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:49.854 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:31:49.854 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940312+0000", 2024-09-14T06:31:49.854 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.9710000000000001e-06 2024-09-14T06:31:49.854 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:49.854 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:49.854 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:31:49.854 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940316+0000", 2024-09-14T06:31:49.854 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.5340000000000001e-06 2024-09-14T06:31:49.854 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:49.854 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:49.854 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:31:49.855 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.017033+0000", 2024-09-14T06:31:49.855 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.076717169000000002 2024-09-14T06:31:49.855 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:49.855 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:49.855 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:31:49.855 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.017073+0000", 2024-09-14T06:31:49.855 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.9582000000000002e-05 2024-09-14T06:31:49.855 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:49.855 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:49.855 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:31:49.855 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.017124+0000", 2024-09-14T06:31:49.855 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.1316999999999997e-05 2024-09-14T06:31:49.855 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:49.855 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:31:49.855 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:49.856 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:49.856 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:49.856 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:115219 3.1 3:856be886:::rbd_data.112d4d24c558.0000000000000004:29 [read 65536~16384 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e96)", 2024-09-14T06:31:49.856 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:30:17.940323+0000", 2024-09-14T06:31:49.856 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 91.832202699999996, 2024-09-14T06:31:49.856 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.076915205, 2024-09-14T06:31:49.856 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:31:49.856 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:31:49.856 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:31:49.856 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:31:49.856 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:31:49.856 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:49.856 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 115219 2024-09-14T06:31:49.856 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:49.857 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:31:49.857 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:49.857 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:31:49.857 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940323+0000", 2024-09-14T06:31:49.857 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:49.857 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:49.857 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:49.857 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:31:49.857 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940323+0000", 2024-09-14T06:31:49.857 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:49.857 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:49.857 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:49.857 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:31:49.857 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940323+0000", 2024-09-14T06:31:49.858 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.0800000000000005e-07 2024-09-14T06:31:49.858 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:49.858 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:49.858 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:31:49.858 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940328+0000", 2024-09-14T06:31:49.858 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.887e-06 2024-09-14T06:31:49.858 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:49.858 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:49.858 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:31:49.858 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940334+0000", 2024-09-14T06:31:49.858 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.3269999999999997e-06 2024-09-14T06:31:49.858 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:49.858 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:49.858 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:31:49.858 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940339+0000", 2024-09-14T06:31:49.858 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.7960000000000003e-06 2024-09-14T06:31:49.859 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:49.859 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:49.859 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:31:49.859 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.017148+0000", 2024-09-14T06:31:49.859 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.076809125000000006 2024-09-14T06:31:49.859 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:49.859 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:49.859 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:31:49.859 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.017186+0000", 2024-09-14T06:31:49.859 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.7283000000000002e-05 2024-09-14T06:31:49.859 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:49.859 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:49.859 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:31:49.859 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.017238+0000", 2024-09-14T06:31:49.859 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.2278999999999999e-05 2024-09-14T06:31:49.859 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:49.860 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:31:49.860 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:49.860 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:49.860 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:49.860 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:115220 3.1 3:856be886:::rbd_data.112d4d24c558.0000000000000004:29 [read 311296~16384 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e96)", 2024-09-14T06:31:49.860 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:30:17.940346+0000", 2024-09-14T06:31:49.860 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 91.832179570999998, 2024-09-14T06:31:49.860 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.077033293000000003, 2024-09-14T06:31:49.860 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:31:49.860 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:31:49.860 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:31:49.860 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:31:49.860 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:31:49.860 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:49.860 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 115220 2024-09-14T06:31:49.861 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:49.861 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:31:49.861 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:49.861 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:31:49.861 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940346+0000", 2024-09-14T06:31:49.861 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:49.861 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:49.861 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:49.861 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:31:49.861 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940346+0000", 2024-09-14T06:31:49.861 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:49.861 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:49.861 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:49.861 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:31:49.861 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940346+0000", 2024-09-14T06:31:49.862 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.4300000000000003e-07 2024-09-14T06:31:49.862 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:49.862 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:49.862 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:31:49.862 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940348+0000", 2024-09-14T06:31:49.862 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.0030000000000001e-06 2024-09-14T06:31:49.862 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:49.862 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:49.862 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:31:49.862 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.004655+0000", 2024-09-14T06:31:49.862 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.064306323999999998 2024-09-14T06:31:49.862 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:49.862 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:49.862 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:31:49.862 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.004671+0000", 2024-09-14T06:31:49.862 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.6589999999999999e-05 2024-09-14T06:31:49.863 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:49.863 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:49.863 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:31:49.863 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.017262+0000", 2024-09-14T06:31:49.863 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.012590570000000001 2024-09-14T06:31:49.863 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:49.863 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:49.863 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:31:49.863 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.017321+0000", 2024-09-14T06:31:49.863 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.9210999999999999e-05 2024-09-14T06:31:49.863 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:49.863 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:49.863 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:31:49.863 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.017379+0000", 2024-09-14T06:31:49.863 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.8051999999999999e-05 2024-09-14T06:31:49.864 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:49.864 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:31:49.864 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:49.864 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:49.864 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:31:49.864 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-14T06:31:49.866 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-14T06:31:49.866 INFO:tasks.ceph.osd.2.smithi094.stderr:2024-09-14T06:31:49.864+0000 7faf3269b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33649) UID: 0 2024-09-14T06:31:49.966 INFO:tasks.ceph.osd.5.smithi138.stderr:2024-09-14T06:31:49.965+0000 7fe13746f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33618) UID: 0 2024-09-14T06:31:50.010 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-14T06:31:50.011 INFO:teuthology.orchestra.run.smithi094.stdout: "ops": [], 2024-09-14T06:31:50.011 INFO:teuthology.orchestra.run.smithi094.stdout: "num_ops": 0 2024-09-14T06:31:50.011 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-14T06:31:50.019 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-14T06:31:50.067 INFO:tasks.ceph.osd.0.smithi094.stderr:2024-09-14T06:31:50.066+0000 7f07c9333640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33636) UID: 0 2024-09-14T06:31:50.109 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-14T06:31:50.123 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-14T06:31:50.123 INFO:teuthology.orchestra.run.smithi094.stdout: "ops": [], 2024-09-14T06:31:50.124 INFO:teuthology.orchestra.run.smithi094.stdout: "complaint_time": 30, 2024-09-14T06:31:50.124 INFO:teuthology.orchestra.run.smithi094.stdout: "num_blocked_ops": 0 2024-09-14T06:31:50.124 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-14T06:31:50.133 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-14T06:31:50.174 INFO:tasks.ceph.osd.0.smithi094.stderr:2024-09-14T06:31:50.173+0000 7f07c9333640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33636) UID: 0 2024-09-14T06:31:50.233 INFO:teuthology.orchestra.run.smithi094.stdout:osd.0: {} 2024-09-14T06:31:50.233 INFO:teuthology.orchestra.run.smithi094.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-14T06:31:50.250 INFO:teuthology.orchestra.run.smithi094.stdout:osd.1: {} 2024-09-14T06:31:50.251 INFO:teuthology.orchestra.run.smithi094.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-14T06:31:50.266 INFO:teuthology.orchestra.run.smithi094.stdout:osd.2: {} 2024-09-14T06:31:50.267 INFO:teuthology.orchestra.run.smithi094.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-14T06:31:50.269 INFO:tasks.ceph.osd.5.smithi138.stderr:2024-09-14T06:31:50.268+0000 7fe13746f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33618) UID: 0 2024-09-14T06:31:50.280 INFO:teuthology.orchestra.run.smithi094.stdout:ERROR: (22) Invalid argument 2024-09-14T06:31:50.280 INFO:teuthology.orchestra.run.smithi094.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:31:50.282 INFO:teuthology.orchestra.run.smithi094.stdout:osd.3: {} 2024-09-14T06:31:50.283 INFO:teuthology.orchestra.run.smithi094.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-14T06:31:50.289 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-14T06:31:50.300 INFO:teuthology.orchestra.run.smithi094.stdout:osd.4: {} 2024-09-14T06:31:50.300 INFO:teuthology.orchestra.run.smithi094.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-14T06:31:50.317 INFO:teuthology.orchestra.run.smithi094.stdout:osd.5: {} 2024-09-14T06:31:50.317 INFO:teuthology.orchestra.run.smithi094.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-14T06:31:50.334 INFO:teuthology.orchestra.run.smithi094.stdout:osd.6: {} 2024-09-14T06:31:50.334 INFO:teuthology.orchestra.run.smithi094.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-14T06:31:50.351 INFO:teuthology.orchestra.run.smithi094.stdout:osd.7: {} 2024-09-14T06:31:50.351 INFO:teuthology.orchestra.run.smithi094.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-14T06:31:50.394 INFO:teuthology.orchestra.run.smithi094.stdout:ERROR: (22) Invalid argument 2024-09-14T06:31:50.394 INFO:teuthology.orchestra.run.smithi094.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:31:50.394 INFO:tasks.ceph.osd.3.smithi094.stderr:2024-09-14T06:31:50.393+0000 7feeeafe8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33655) UID: 0 2024-09-14T06:31:50.403 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-14T06:31:50.471 INFO:tasks.ceph.osd.4.smithi138.stderr:2024-09-14T06:31:50.470+0000 7f6c653c8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33615) UID: 0 2024-09-14T06:31:50.546 INFO:teuthology.orchestra.run.smithi094.stdout:ERROR: (22) Invalid argument 2024-09-14T06:31:50.546 INFO:teuthology.orchestra.run.smithi094.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:31:50.555 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-14T06:31:50.572 INFO:tasks.ceph.osd.7.smithi138.stderr:2024-09-14T06:31:50.571+0000 7f3c6a21d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33623) UID: 0 2024-09-14T06:31:50.661 INFO:teuthology.orchestra.run.smithi094.stdout:ERROR: (22) Invalid argument 2024-09-14T06:31:50.662 INFO:teuthology.orchestra.run.smithi094.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:31:50.670 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-14T06:31:50.673 INFO:tasks.ceph.osd.4.smithi138.stderr:2024-09-14T06:31:50.672+0000 7f6c653c8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33615) UID: 0 2024-09-14T06:31:50.736 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-14T06:31:50.774 INFO:tasks.ceph.osd.5.smithi138.stderr:2024-09-14T06:31:50.773+0000 7fe13746f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33618) UID: 0 2024-09-14T06:31:50.776 INFO:teuthology.orchestra.run.smithi094.stdout:ERROR: (22) Invalid argument 2024-09-14T06:31:50.776 INFO:teuthology.orchestra.run.smithi094.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:31:50.785 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-14T06:31:50.875 INFO:tasks.ceph.osd.3.smithi094.stderr:2024-09-14T06:31:50.874+0000 7feeeafe8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33655) UID: 0 2024-09-14T06:31:50.929 INFO:teuthology.orchestra.run.smithi094.stdout:ERROR: (22) Invalid argument 2024-09-14T06:31:50.930 INFO:teuthology.orchestra.run.smithi094.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:31:50.938 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-14T06:31:50.975 INFO:tasks.ceph.osd.5.smithi138.stderr:2024-09-14T06:31:50.974+0000 7fe13746f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33618) UID: 0 2024-09-14T06:31:51.076 INFO:tasks.ceph.osd.6.smithi138.stderr:2024-09-14T06:31:51.075+0000 7f8cd8849640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33620) UID: 0 2024-09-14T06:31:51.087 INFO:teuthology.orchestra.run.smithi094.stdout:ERROR: (22) Invalid argument 2024-09-14T06:31:51.087 INFO:teuthology.orchestra.run.smithi094.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:31:51.096 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-14T06:31:51.178 INFO:tasks.ceph.osd.1.smithi094.stderr:2024-09-14T06:31:51.176+0000 7f328e47a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33641) UID: 0 2024-09-14T06:31:51.212 INFO:teuthology.orchestra.run.smithi138.stdout:ERROR: (22) Invalid argument 2024-09-14T06:31:51.212 INFO:teuthology.orchestra.run.smithi138.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:31:51.221 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-14T06:31:51.279 INFO:tasks.ceph.osd.2.smithi094.stderr:2024-09-14T06:31:51.278+0000 7faf3269b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33649) UID: 0 2024-09-14T06:31:51.325 INFO:teuthology.orchestra.run.smithi138.stdout:ERROR: (22) Invalid argument 2024-09-14T06:31:51.325 INFO:teuthology.orchestra.run.smithi138.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:31:51.334 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-14T06:31:51.380 INFO:tasks.ceph.osd.7.smithi138.stderr:2024-09-14T06:31:51.378+0000 7f3c6a21d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33623) UID: 0 2024-09-14T06:31:51.436 INFO:teuthology.orchestra.run.smithi138.stdout:ERROR: (22) Invalid argument 2024-09-14T06:31:51.436 INFO:teuthology.orchestra.run.smithi138.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:31:51.445 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-14T06:31:51.486 INFO:tasks.ceph.osd.4.smithi138.stderr:2024-09-14T06:31:51.485+0000 7f6c653c8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33615) UID: 0 2024-09-14T06:31:51.582 INFO:tasks.ceph.osd.7.smithi138.stderr:2024-09-14T06:31:51.580+0000 7f3c6a21d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33623) UID: 0 2024-09-14T06:31:51.587 INFO:teuthology.orchestra.run.smithi138.stdout:ERROR: (22) Invalid argument 2024-09-14T06:31:51.587 INFO:teuthology.orchestra.run.smithi138.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:31:51.596 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-14T06:31:51.602 INFO:teuthology.orchestra.run.smithi094.stderr:nodeep-scrub is set 2024-09-14T06:31:51.683 INFO:tasks.ceph.osd.7.smithi138.stderr:2024-09-14T06:31:51.682+0000 7f3c6a21d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33623) UID: 0 2024-09-14T06:31:51.781 INFO:teuthology.orchestra.run.smithi138.stdout:ERROR: (22) Invalid argument 2024-09-14T06:31:51.781 INFO:teuthology.orchestra.run.smithi138.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:31:51.783 INFO:tasks.ceph.osd.2.smithi094.stderr:2024-09-14T06:31:51.782+0000 7faf3269b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33649) UID: 0 2024-09-14T06:31:51.794 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-14T06:31:51.885 INFO:tasks.ceph.osd.3.smithi094.stderr:2024-09-14T06:31:51.884+0000 7feeeafe8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33655) UID: 0 2024-09-14T06:31:51.949 INFO:teuthology.orchestra.run.smithi138.stdout:ERROR: (22) Invalid argument 2024-09-14T06:31:51.949 INFO:teuthology.orchestra.run.smithi138.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:31:51.958 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-14T06:31:52.003 INFO:tasks.ceph.osd.6.smithi138.stderr:2024-09-14T06:31:52.002+0000 7f8cd8849640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33620) UID: 0 2024-09-14T06:31:52.061 INFO:teuthology.orchestra.run.smithi138.stdout:ERROR: (22) Invalid argument 2024-09-14T06:31:52.062 INFO:teuthology.orchestra.run.smithi138.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:31:52.070 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-14T06:31:52.086 INFO:tasks.ceph.osd.0.smithi094.stderr:2024-09-14T06:31:52.085+0000 7f07c9333640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33636) UID: 0 2024-09-14T06:31:52.188 INFO:tasks.ceph.osd.3.smithi094.stderr:2024-09-14T06:31:52.187+0000 7feeeafe8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33655) UID: 0 2024-09-14T06:31:52.212 INFO:teuthology.orchestra.run.smithi138.stdout:ERROR: (22) Invalid argument 2024-09-14T06:31:52.213 INFO:teuthology.orchestra.run.smithi138.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:31:52.221 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-14T06:31:52.288 INFO:tasks.ceph.osd.0.smithi094.stderr:2024-09-14T06:31:52.287+0000 7f07c9333640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33636) UID: 0 2024-09-14T06:31:52.323 INFO:teuthology.orchestra.run.smithi138.stdout:ERROR: (22) Invalid argument 2024-09-14T06:31:52.323 INFO:teuthology.orchestra.run.smithi138.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:31:52.332 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-14T06:31:52.363 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-14T06:31:52.390 INFO:tasks.ceph.osd.7.smithi138.stderr:2024-09-14T06:31:52.389+0000 7f3c6a21d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33623) UID: 0 2024-09-14T06:31:52.434 INFO:teuthology.orchestra.run.smithi138.stdout:ERROR: (22) Invalid argument 2024-09-14T06:31:52.434 INFO:teuthology.orchestra.run.smithi138.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:31:52.443 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-14T06:31:52.491 INFO:tasks.ceph.osd.2.smithi094.stderr:2024-09-14T06:31:52.490+0000 7faf3269b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33649) UID: 0 2024-09-14T06:31:52.497 INFO:teuthology.orchestra.run.smithi094.stdout:osd.0: {} 2024-09-14T06:31:52.497 INFO:teuthology.orchestra.run.smithi094.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-14T06:31:52.514 INFO:teuthology.orchestra.run.smithi094.stdout:osd.1: {} 2024-09-14T06:31:52.514 INFO:teuthology.orchestra.run.smithi094.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-14T06:31:52.530 INFO:teuthology.orchestra.run.smithi094.stdout:osd.2: {} 2024-09-14T06:31:52.530 INFO:teuthology.orchestra.run.smithi094.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-14T06:31:52.546 INFO:teuthology.orchestra.run.smithi094.stdout:osd.3: {} 2024-09-14T06:31:52.547 INFO:teuthology.orchestra.run.smithi094.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-14T06:31:52.563 INFO:teuthology.orchestra.run.smithi094.stdout:osd.4: {} 2024-09-14T06:31:52.563 INFO:teuthology.orchestra.run.smithi094.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-14T06:31:52.581 INFO:teuthology.orchestra.run.smithi094.stdout:osd.5: {} 2024-09-14T06:31:52.581 INFO:teuthology.orchestra.run.smithi094.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-14T06:31:52.592 INFO:tasks.ceph.osd.2.smithi094.stderr:2024-09-14T06:31:52.591+0000 7faf3269b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33649) UID: 0 2024-09-14T06:31:52.592 INFO:teuthology.orchestra.run.smithi138.stdout:ERROR: (22) Invalid argument 2024-09-14T06:31:52.592 INFO:teuthology.orchestra.run.smithi138.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:31:52.601 INFO:teuthology.orchestra.run.smithi094.stdout:osd.6: {} 2024-09-14T06:31:52.601 INFO:teuthology.orchestra.run.smithi094.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-14T06:31:52.602 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-14T06:31:52.620 INFO:teuthology.orchestra.run.smithi094.stdout:osd.7: {} 2024-09-14T06:31:52.620 INFO:teuthology.orchestra.run.smithi094.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-14T06:31:52.692 INFO:tasks.ceph.osd.4.smithi138.stderr:2024-09-14T06:31:52.691+0000 7f6c653c8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33615) UID: 0 2024-09-14T06:31:52.710 INFO:teuthology.orchestra.run.smithi138.stdout:{ 2024-09-14T06:31:52.711 INFO:teuthology.orchestra.run.smithi138.stdout: "size": 20, 2024-09-14T06:31:52.711 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 600, 2024-09-14T06:31:52.711 INFO:teuthology.orchestra.run.smithi138.stdout: "ops": [ 2024-09-14T06:31:52.711 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:52.711 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:53244 3.f 3:f71f51a7:::rbd_data.112dca2f6ae1.0000000000000015:17 [read 1863680~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:31:52.711 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:44.369399+0000", 2024-09-14T06:31:52.711 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 128.33955984100001, 2024-09-14T06:31:52.711 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047291189999999997, 2024-09-14T06:31:52.711 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:31:52.711 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:31:52.711 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:31:52.711 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:31:52.712 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:31:52.712 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:52.712 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 53244 2024-09-14T06:31:52.712 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:52.712 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:31:52.712 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:52.712 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:31:52.712 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369399+0000", 2024-09-14T06:31:52.712 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:52.712 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:52.712 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:52.712 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:31:52.712 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369399+0000", 2024-09-14T06:31:52.712 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:52.713 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:52.713 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:52.713 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:31:52.713 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369400+0000", 2024-09-14T06:31:52.713 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.0699999999999999e-07 2024-09-14T06:31:52.713 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:52.713 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:52.713 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:31:52.713 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369401+0000", 2024-09-14T06:31:52.713 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.4300000000000001e-06 2024-09-14T06:31:52.713 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:52.713 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:52.713 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:31:52.713 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369404+0000", 2024-09-14T06:31:52.714 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.061e-06 2024-09-14T06:31:52.714 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:52.714 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:52.714 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:31:52.714 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369407+0000", 2024-09-14T06:31:52.714 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.6139999999999999e-06 2024-09-14T06:31:52.714 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:52.714 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:52.714 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:31:52.714 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.416517+0000", 2024-09-14T06:31:52.714 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047110338000000002 2024-09-14T06:31:52.715 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:52.715 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:52.715 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:31:52.715 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.416616+0000", 2024-09-14T06:31:52.715 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.8651000000000001e-05 2024-09-14T06:31:52.715 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:52.715 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:52.715 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:31:52.715 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.416691+0000", 2024-09-14T06:31:52.715 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.4888999999999994e-05 2024-09-14T06:31:52.715 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:52.715 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:31:52.715 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:52.716 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:52.716 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:52.716 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:53245 3.f 3:f71f51a7:::rbd_data.112dca2f6ae1.0000000000000015:17 [read 1888256~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:31:52.716 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:44.369410+0000", 2024-09-14T06:31:52.716 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 128.33954955300001, 2024-09-14T06:31:52.716 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047399275999999997, 2024-09-14T06:31:52.716 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:31:52.716 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:31:52.716 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:31:52.716 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:31:52.716 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:31:52.716 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:52.716 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 53245 2024-09-14T06:31:52.716 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:52.717 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:31:52.717 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:52.717 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:31:52.717 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369410+0000", 2024-09-14T06:31:52.717 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:52.717 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:52.717 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:52.717 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:31:52.717 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369410+0000", 2024-09-14T06:31:52.717 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:52.717 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:52.717 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:52.717 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:31:52.718 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369410+0000", 2024-09-14T06:31:52.718 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.1899999999999999e-07 2024-09-14T06:31:52.718 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:52.718 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:52.718 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:31:52.718 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369411+0000", 2024-09-14T06:31:52.718 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.0300000000000001e-06 2024-09-14T06:31:52.718 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:52.718 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:52.718 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:31:52.718 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369414+0000", 2024-09-14T06:31:52.718 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.1099999999999999e-06 2024-09-14T06:31:52.718 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:52.718 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:52.719 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:31:52.719 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369417+0000", 2024-09-14T06:31:52.719 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.6189999999999998e-06 2024-09-14T06:31:52.719 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:52.719 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:52.719 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:31:52.719 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.416718+0000", 2024-09-14T06:31:52.719 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047301322999999999 2024-09-14T06:31:52.719 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:52.719 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:52.719 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:31:52.719 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.416759+0000", 2024-09-14T06:31:52.719 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.1009e-05 2024-09-14T06:31:52.719 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:52.720 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:52.720 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:31:52.720 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.416809+0000", 2024-09-14T06:31:52.720 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.9966000000000002e-05 2024-09-14T06:31:52.720 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:52.720 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:31:52.720 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:52.720 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:52.720 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:52.720 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:53246 3.f 3:f71f51a7:::rbd_data.112dca2f6ae1.0000000000000015:17 [read 1912832~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:31:52.720 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:44.369420+0000", 2024-09-14T06:31:52.720 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 128.33953960900001, 2024-09-14T06:31:52.720 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047529463000000001, 2024-09-14T06:31:52.720 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:31:52.721 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:31:52.721 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:31:52.721 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:31:52.721 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:31:52.721 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:52.721 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 53246 2024-09-14T06:31:52.721 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:52.721 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:31:52.721 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:52.721 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:31:52.721 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369420+0000", 2024-09-14T06:31:52.721 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:52.721 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:52.721 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:52.722 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:31:52.722 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369420+0000", 2024-09-14T06:31:52.722 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:52.722 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:52.722 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:52.722 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:31:52.722 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369420+0000", 2024-09-14T06:31:52.722 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.1299999999999999e-07 2024-09-14T06:31:52.722 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:52.722 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:52.722 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:31:52.722 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369425+0000", 2024-09-14T06:31:52.722 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.0130000000000001e-06 2024-09-14T06:31:52.722 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:52.723 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:52.723 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:31:52.723 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369428+0000", 2024-09-14T06:31:52.723 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.349e-06 2024-09-14T06:31:52.723 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:52.723 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:52.723 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:31:52.723 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369431+0000", 2024-09-14T06:31:52.723 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.7389999999999999e-06 2024-09-14T06:31:52.723 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:52.723 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:52.723 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:31:52.723 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.416834+0000", 2024-09-14T06:31:52.723 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047402669000000001 2024-09-14T06:31:52.724 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:52.724 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:52.724 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:31:52.724 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.416882+0000", 2024-09-14T06:31:52.724 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.8068999999999997e-05 2024-09-14T06:31:52.724 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:52.724 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:52.724 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:31:52.724 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.416949+0000", 2024-09-14T06:31:52.724 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.7410999999999995e-05 2024-09-14T06:31:52.724 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:52.724 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:31:52.724 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:52.724 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:52.725 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:52.725 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:53247 3.f 3:f71f51a7:::rbd_data.112dca2f6ae1.0000000000000015:17 [read 1937408~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:31:52.725 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:44.369434+0000", 2024-09-14T06:31:52.725 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 128.339524845, 2024-09-14T06:31:52.725 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047652264999999999, 2024-09-14T06:31:52.725 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:31:52.725 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:31:52.725 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:31:52.725 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:31:52.725 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:31:52.725 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:52.725 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 53247 2024-09-14T06:31:52.725 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:52.726 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:31:52.726 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:52.726 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:31:52.726 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369434+0000", 2024-09-14T06:31:52.726 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:52.726 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:52.726 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:52.726 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:31:52.726 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369434+0000", 2024-09-14T06:31:52.726 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:52.726 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:52.726 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:52.726 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:31:52.726 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369435+0000", 2024-09-14T06:31:52.727 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.2100000000000001e-07 2024-09-14T06:31:52.727 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:52.727 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:52.727 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:31:52.727 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369436+0000", 2024-09-14T06:31:52.727 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.122e-06 2024-09-14T06:31:52.727 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:52.727 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:52.727 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:31:52.727 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369439+0000", 2024-09-14T06:31:52.727 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.0400000000000001e-06 2024-09-14T06:31:52.727 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:52.727 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:52.728 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:31:52.728 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369441+0000", 2024-09-14T06:31:52.728 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.728e-06 2024-09-14T06:31:52.728 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:52.728 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:52.728 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:31:52.728 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.416977+0000", 2024-09-14T06:31:52.728 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047535344 2024-09-14T06:31:52.728 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:52.728 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:52.728 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:31:52.728 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.417029+0000", 2024-09-14T06:31:52.728 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.2037999999999999e-05 2024-09-14T06:31:52.729 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:52.729 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:52.729 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:31:52.729 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.417087+0000", 2024-09-14T06:31:52.729 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.7772000000000002e-05 2024-09-14T06:31:52.729 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:52.729 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:31:52.729 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:52.729 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:52.729 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:52.729 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:53248 3.f 3:f71f51a7:::rbd_data.112dca2f6ae1.0000000000000015:17 [read 1961984~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:31:52.729 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:44.369445+0000", 2024-09-14T06:31:52.729 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 128.33951429300001, 2024-09-14T06:31:52.729 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047798489, 2024-09-14T06:31:52.730 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:31:52.730 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:31:52.730 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:31:52.730 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:31:52.730 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:31:52.730 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:52.730 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 53248 2024-09-14T06:31:52.730 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:52.730 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:31:52.730 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:52.730 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:31:52.730 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369445+0000", 2024-09-14T06:31:52.730 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:52.731 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:52.731 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:52.731 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:31:52.731 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369445+0000", 2024-09-14T06:31:52.731 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:52.731 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:52.731 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:52.731 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:31:52.731 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369445+0000", 2024-09-14T06:31:52.731 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.0699999999999999e-07 2024-09-14T06:31:52.731 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:52.731 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:52.731 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:31:52.731 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369446+0000", 2024-09-14T06:31:52.731 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.1000000000000001e-06 2024-09-14T06:31:52.732 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:52.732 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:52.732 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:31:52.732 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369449+0000", 2024-09-14T06:31:52.732 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.1039999999999998e-06 2024-09-14T06:31:52.732 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:52.732 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:52.732 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:31:52.732 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369452+0000", 2024-09-14T06:31:52.732 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.6199999999999999e-06 2024-09-14T06:31:52.732 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:52.732 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:52.732 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:31:52.732 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.417114+0000", 2024-09-14T06:31:52.733 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047661901999999999 2024-09-14T06:31:52.733 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:52.733 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:52.733 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:31:52.733 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.417181+0000", 2024-09-14T06:31:52.733 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.7385999999999995e-05 2024-09-14T06:31:52.733 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:52.733 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:52.733 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:31:52.733 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.417243+0000", 2024-09-14T06:31:52.733 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.2169999999999996e-05 2024-09-14T06:31:52.733 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:52.733 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:31:52.734 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:52.734 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:52.734 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:52.734 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:53249 3.f 3:f71f51a7:::rbd_data.112dca2f6ae1.0000000000000015:17 [read 1986560~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:31:52.734 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:44.369455+0000", 2024-09-14T06:31:52.734 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 128.339503982, 2024-09-14T06:31:52.734 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047890493999999999, 2024-09-14T06:31:52.734 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:31:52.734 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:31:52.734 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:31:52.734 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:31:52.734 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:31:52.734 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:52.734 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 53249 2024-09-14T06:31:52.735 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:52.735 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:31:52.735 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:52.735 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:31:52.735 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369455+0000", 2024-09-14T06:31:52.735 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:52.735 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:52.735 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:52.735 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:31:52.735 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369455+0000", 2024-09-14T06:31:52.735 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:52.735 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:52.735 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:52.735 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:31:52.735 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369455+0000", 2024-09-14T06:31:52.736 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.0800000000000001e-07 2024-09-14T06:31:52.736 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:52.736 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:52.736 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:31:52.736 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369456+0000", 2024-09-14T06:31:52.736 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.0729999999999999e-06 2024-09-14T06:31:52.736 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:52.736 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:52.736 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:31:52.736 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369460+0000", 2024-09-14T06:31:52.736 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.027e-06 2024-09-14T06:31:52.736 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:52.736 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:52.736 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:31:52.737 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369462+0000", 2024-09-14T06:31:52.737 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.5320000000000001e-06 2024-09-14T06:31:52.737 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:52.737 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:52.737 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:31:52.737 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.417257+0000", 2024-09-14T06:31:52.737 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.04779489 2024-09-14T06:31:52.737 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:52.737 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:52.737 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:31:52.737 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.417296+0000", 2024-09-14T06:31:52.737 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.8658000000000002e-05 2024-09-14T06:31:52.737 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:52.737 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:52.738 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:31:52.738 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.417346+0000", 2024-09-14T06:31:52.738 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.0105999999999998e-05 2024-09-14T06:31:52.738 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:52.738 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:31:52.738 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:52.738 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:52.738 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:52.738 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:53250 3.f 3:f71f51a7:::rbd_data.112dca2f6ae1.0000000000000015:17 [read 2011136~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:31:52.738 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:44.369465+0000", 2024-09-14T06:31:52.738 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 128.33949401199999, 2024-09-14T06:31:52.738 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.048032230000000002, 2024-09-14T06:31:52.738 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:31:52.738 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:31:52.739 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:31:52.739 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:31:52.739 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:31:52.739 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:52.739 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 53250 2024-09-14T06:31:52.739 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:52.739 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:31:52.739 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:52.739 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:31:52.739 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369465+0000", 2024-09-14T06:31:52.739 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:52.739 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:52.739 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:52.739 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:31:52.740 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369465+0000", 2024-09-14T06:31:52.740 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:52.740 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:52.740 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:52.740 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:31:52.740 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369465+0000", 2024-09-14T06:31:52.740 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.2100000000000001e-07 2024-09-14T06:31:52.740 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:52.740 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:52.740 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:31:52.740 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369466+0000", 2024-09-14T06:31:52.740 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.0640000000000001e-06 2024-09-14T06:31:52.740 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:52.741 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:52.741 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:31:52.741 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369469+0000", 2024-09-14T06:31:52.741 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.0170000000000001e-06 2024-09-14T06:31:52.741 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:52.741 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:52.741 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:31:52.741 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369472+0000", 2024-09-14T06:31:52.741 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.6189999999999998e-06 2024-09-14T06:31:52.741 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:52.741 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:52.741 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:31:52.741 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.417383+0000", 2024-09-14T06:31:52.741 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047911092000000002 2024-09-14T06:31:52.742 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:52.742 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:52.742 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:31:52.742 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.417430+0000", 2024-09-14T06:31:52.742 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.6898999999999999e-05 2024-09-14T06:31:52.742 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:52.742 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:52.742 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:31:52.742 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.417497+0000", 2024-09-14T06:31:52.742 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.7317999999999995e-05 2024-09-14T06:31:52.742 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:52.742 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:31:52.742 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:52.742 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:52.743 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:52.743 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:53251 3.f 3:f71f51a7:::rbd_data.112dca2f6ae1.0000000000000015:17 [read 2035712~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:31:52.743 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:44.369475+0000", 2024-09-14T06:31:52.743 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 128.339484016, 2024-09-14T06:31:52.743 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.048154542000000002, 2024-09-14T06:31:52.743 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:31:52.743 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:31:52.743 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:31:52.743 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:31:52.743 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:31:52.743 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:52.743 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 53251 2024-09-14T06:31:52.743 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:52.743 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:31:52.743 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:52.744 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:31:52.744 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369475+0000", 2024-09-14T06:31:52.744 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:52.744 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:52.744 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:52.744 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:31:52.744 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369475+0000", 2024-09-14T06:31:52.744 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:52.744 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:52.744 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:52.744 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:31:52.744 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369475+0000", 2024-09-14T06:31:52.744 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.0100000000000001e-07 2024-09-14T06:31:52.745 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:52.745 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:52.745 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:31:52.745 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369476+0000", 2024-09-14T06:31:52.745 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.0580000000000001e-06 2024-09-14T06:31:52.745 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:52.745 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:52.745 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:31:52.745 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369479+0000", 2024-09-14T06:31:52.745 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.9950000000000001e-06 2024-09-14T06:31:52.745 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:52.745 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:52.745 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:31:52.745 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369482+0000", 2024-09-14T06:31:52.746 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.6340000000000001e-06 2024-09-14T06:31:52.746 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:52.746 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:52.746 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:31:52.746 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.417530+0000", 2024-09-14T06:31:52.746 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.048047535000000002 2024-09-14T06:31:52.746 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:52.746 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:52.746 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:31:52.746 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.417574+0000", 2024-09-14T06:31:52.746 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.4255999999999998e-05 2024-09-14T06:31:52.746 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:52.746 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:52.746 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:31:52.747 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.417630+0000", 2024-09-14T06:31:52.747 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.5862999999999997e-05 2024-09-14T06:31:52.747 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:52.747 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:31:52.747 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:52.747 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:52.747 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:52.747 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:53252 3.f 3:f71f51a7:::rbd_data.112dca2f6ae1.0000000000000015:17 [read 2060288~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:31:52.747 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:44.369485+0000", 2024-09-14T06:31:52.747 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 128.33947416800001, 2024-09-14T06:31:52.747 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.048254116, 2024-09-14T06:31:52.747 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:31:52.747 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:31:52.747 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:31:52.748 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:31:52.748 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:31:52.748 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:52.748 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 53252 2024-09-14T06:31:52.748 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:52.748 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:31:52.748 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:52.748 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:31:52.748 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369485+0000", 2024-09-14T06:31:52.748 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:52.748 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:52.748 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:52.748 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:31:52.749 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369485+0000", 2024-09-14T06:31:52.749 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:52.749 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:52.749 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:52.749 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:31:52.749 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369485+0000", 2024-09-14T06:31:52.749 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.1e-07 2024-09-14T06:31:52.749 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:52.749 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:52.749 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:31:52.749 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369487+0000", 2024-09-14T06:31:52.749 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.42e-06 2024-09-14T06:31:52.749 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:52.750 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:52.750 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:31:52.750 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369490+0000", 2024-09-14T06:31:52.750 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.9639999999999999e-06 2024-09-14T06:31:52.750 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:52.750 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:52.750 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:31:52.750 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369492+0000", 2024-09-14T06:31:52.750 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.5679999999999998e-06 2024-09-14T06:31:52.750 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:52.750 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:52.750 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:31:52.750 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.417645+0000", 2024-09-14T06:31:52.750 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.048152753 2024-09-14T06:31:52.750 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:52.751 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:52.751 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:31:52.751 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.417687+0000", 2024-09-14T06:31:52.751 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.1900999999999997e-05 2024-09-14T06:31:52.751 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:52.751 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:52.751 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:31:52.751 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.417739+0000", 2024-09-14T06:31:52.751 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.2299999999999997e-05 2024-09-14T06:31:52.751 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:52.751 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:31:52.751 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:52.751 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:52.751 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:52.751 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:53253 3.f 3:f71f51a7:::rbd_data.112dca2f6ae1.0000000000000015:17 [read 2084864~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:31:52.752 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:44.369495+0000", 2024-09-14T06:31:52.752 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 128.33946404400001, 2024-09-14T06:31:52.752 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.048366785000000002, 2024-09-14T06:31:52.752 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:31:52.752 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:31:52.752 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:31:52.752 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:31:52.752 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:31:52.752 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:52.752 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 53253 2024-09-14T06:31:52.752 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:52.752 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:31:52.752 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:52.752 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:31:52.753 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369495+0000", 2024-09-14T06:31:52.753 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:52.753 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:52.753 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:52.753 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:31:52.753 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369495+0000", 2024-09-14T06:31:52.753 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:52.753 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:52.753 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:52.753 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:31:52.753 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370388+0000", 2024-09-14T06:31:52.753 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00089287999999999998 2024-09-14T06:31:52.753 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:52.754 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:52.754 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:31:52.754 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370392+0000", 2024-09-14T06:31:52.754 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.2130000000000002e-06 2024-09-14T06:31:52.754 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:52.754 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:52.754 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:31:52.754 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370401+0000", 2024-09-14T06:31:52.754 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.7649999999999999e-06 2024-09-14T06:31:52.754 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:52.754 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:52.754 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:31:52.754 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370406+0000", 2024-09-14T06:31:52.754 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.0159999999999999e-06 2024-09-14T06:31:52.755 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:52.755 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:52.755 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:31:52.755 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.417764+0000", 2024-09-14T06:31:52.755 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047358348000000001 2024-09-14T06:31:52.755 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:52.755 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:52.755 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:31:52.755 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.417806+0000", 2024-09-14T06:31:52.755 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.1149999999999997e-05 2024-09-14T06:31:52.755 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:52.755 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:52.755 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:31:52.755 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.417862+0000", 2024-09-14T06:31:52.756 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.6412999999999997e-05 2024-09-14T06:31:52.756 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:52.756 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:31:52.756 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:52.756 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:52.756 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:52.756 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:53340 3.f 3:f71f51a7:::rbd_data.112dca2f6ae1.0000000000000015:17 [read 1056768~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:31:52.756 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:44.370421+0000", 2024-09-14T06:31:52.756 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 128.338538164, 2024-09-14T06:31:52.756 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047566313999999998, 2024-09-14T06:31:52.756 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:31:52.756 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:31:52.756 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:31:52.756 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:31:52.757 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:31:52.757 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:52.757 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 53340 2024-09-14T06:31:52.757 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:52.757 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:31:52.757 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:52.757 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:31:52.757 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370421+0000", 2024-09-14T06:31:52.757 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:52.757 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:52.757 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:52.757 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:31:52.757 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370421+0000", 2024-09-14T06:31:52.757 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:52.758 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:52.758 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:52.758 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:31:52.758 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370422+0000", 2024-09-14T06:31:52.758 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.3799999999999996e-07 2024-09-14T06:31:52.758 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:52.758 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:52.758 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:31:52.758 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370424+0000", 2024-09-14T06:31:52.758 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.6919999999999998e-06 2024-09-14T06:31:52.758 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:52.758 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:52.758 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:31:52.758 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370432+0000", 2024-09-14T06:31:52.759 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.2439999999999996e-06 2024-09-14T06:31:52.759 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:52.759 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:52.759 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:31:52.759 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370437+0000", 2024-09-14T06:31:52.759 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.5389999999999998e-06 2024-09-14T06:31:52.759 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:52.759 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:52.759 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:31:52.759 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.417888+0000", 2024-09-14T06:31:52.759 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047450432000000001 2024-09-14T06:31:52.759 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:52.759 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:52.760 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:31:52.760 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.417935+0000", 2024-09-14T06:31:52.760 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.7454999999999999e-05 2024-09-14T06:31:52.760 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:52.760 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:52.760 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:31:52.760 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.417987+0000", 2024-09-14T06:31:52.760 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.2213999999999998e-05 2024-09-14T06:31:52.760 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:52.760 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:31:52.760 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:52.760 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:52.760 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:52.760 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:53341 3.f 3:f71f51a7:::rbd_data.112dca2f6ae1.0000000000000015:17 [read 1081344~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:31:52.761 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:44.370446+0000", 2024-09-14T06:31:52.761 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 128.33851289099999, 2024-09-14T06:31:52.761 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047665723, 2024-09-14T06:31:52.761 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:31:52.761 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:31:52.761 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:31:52.761 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:31:52.761 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:31:52.761 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:52.761 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 53341 2024-09-14T06:31:52.761 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:52.761 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:31:52.761 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:52.762 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:31:52.762 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370446+0000", 2024-09-14T06:31:52.762 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:52.762 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:52.762 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:52.762 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:31:52.762 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370446+0000", 2024-09-14T06:31:52.762 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:52.762 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:52.762 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:52.764 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:31:52.764 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370447+0000", 2024-09-14T06:31:52.764 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.6100000000000002e-07 2024-09-14T06:31:52.764 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:52.764 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:52.764 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:31:52.764 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370448+0000", 2024-09-14T06:31:52.764 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.7370000000000001e-06 2024-09-14T06:31:52.764 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:52.764 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:52.764 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:31:52.765 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370462+0000", 2024-09-14T06:31:52.765 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.3227000000000001e-05 2024-09-14T06:31:52.765 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:52.765 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:52.765 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:31:52.765 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370467+0000", 2024-09-14T06:31:52.765 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.6540000000000004e-06 2024-09-14T06:31:52.765 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:52.765 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:52.765 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:31:52.765 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.418013+0000", 2024-09-14T06:31:52.765 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047545325999999999 2024-09-14T06:31:52.765 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:52.765 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:52.766 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:31:52.766 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.418054+0000", 2024-09-14T06:31:52.766 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.1696999999999997e-05 2024-09-14T06:31:52.766 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:52.766 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:52.766 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:31:52.766 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.418112+0000", 2024-09-14T06:31:52.766 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.7720999999999998e-05 2024-09-14T06:31:52.766 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:52.766 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:31:52.766 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:52.766 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:52.766 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:52.767 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:53342 3.f 3:f71f51a7:::rbd_data.112dca2f6ae1.0000000000000015:17 [read 1105920~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:31:52.767 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:44.370475+0000", 2024-09-14T06:31:52.767 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 128.33848383, 2024-09-14T06:31:52.767 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047748848000000003, 2024-09-14T06:31:52.767 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:31:52.767 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:31:52.767 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:31:52.767 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:31:52.767 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:31:52.767 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:52.767 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 53342 2024-09-14T06:31:52.767 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:52.767 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:31:52.767 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:52.768 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:31:52.768 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370475+0000", 2024-09-14T06:31:52.768 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:52.768 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:52.768 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:52.768 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:31:52.768 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370475+0000", 2024-09-14T06:31:52.768 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:52.768 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:52.768 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:52.768 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:31:52.768 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370476+0000", 2024-09-14T06:31:52.768 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.4700000000000002e-07 2024-09-14T06:31:52.768 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:52.769 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:52.769 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:31:52.769 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370478+0000", 2024-09-14T06:31:52.769 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.5890000000000001e-06 2024-09-14T06:31:52.769 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:52.769 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:52.769 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:31:52.769 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370486+0000", 2024-09-14T06:31:52.769 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.3470000000000001e-06 2024-09-14T06:31:52.769 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:52.769 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:52.769 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:31:52.769 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370491+0000", 2024-09-14T06:31:52.770 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.5720000000000002e-06 2024-09-14T06:31:52.770 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:52.770 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:52.770 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:31:52.770 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.418135+0000", 2024-09-14T06:31:52.770 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047644094999999997 2024-09-14T06:31:52.770 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:52.770 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:52.770 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:31:52.770 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.418174+0000", 2024-09-14T06:31:52.770 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.8204000000000002e-05 2024-09-14T06:31:52.770 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:52.770 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:52.770 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:31:52.771 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.418224+0000", 2024-09-14T06:31:52.771 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.0593999999999997e-05 2024-09-14T06:31:52.771 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:52.771 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:31:52.771 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:52.771 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:52.771 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:52.771 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:53343 3.f 3:f71f51a7:::rbd_data.112dca2f6ae1.0000000000000015:17 [read 1130496~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:31:52.771 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:44.370498+0000", 2024-09-14T06:31:52.771 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 128.338460954, 2024-09-14T06:31:52.771 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047841574999999997, 2024-09-14T06:31:52.771 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:31:52.771 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:31:52.771 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:31:52.772 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:31:52.772 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:31:52.772 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:52.772 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 53343 2024-09-14T06:31:52.772 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:52.772 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:31:52.772 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:52.772 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:31:52.772 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370498+0000", 2024-09-14T06:31:52.772 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:52.772 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:52.772 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:52.772 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:31:52.773 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370498+0000", 2024-09-14T06:31:52.773 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:52.773 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:52.773 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:52.773 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:31:52.773 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370499+0000", 2024-09-14T06:31:52.773 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.9999999999999998e-07 2024-09-14T06:31:52.773 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:52.773 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:52.773 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:31:52.773 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370501+0000", 2024-09-14T06:31:52.773 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.0310000000000001e-06 2024-09-14T06:31:52.773 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:52.773 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:52.774 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:31:52.774 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370507+0000", 2024-09-14T06:31:52.774 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.4169999999999997e-06 2024-09-14T06:31:52.774 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:52.774 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:52.774 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:31:52.774 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370516+0000", 2024-09-14T06:31:52.774 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.7190000000000007e-06 2024-09-14T06:31:52.774 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:52.774 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:52.774 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:31:52.774 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.418247+0000", 2024-09-14T06:31:52.774 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047731190999999999 2024-09-14T06:31:52.774 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:52.775 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:52.775 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:31:52.775 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.418286+0000", 2024-09-14T06:31:52.775 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.8988e-05 2024-09-14T06:31:52.775 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:52.775 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:52.775 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:31:52.775 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.418340+0000", 2024-09-14T06:31:52.775 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.3829000000000002e-05 2024-09-14T06:31:52.775 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:52.775 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:31:52.775 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:52.775 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:52.775 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:52.776 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:53344 3.f 3:f71f51a7:::rbd_data.112dca2f6ae1.0000000000000015:17 [read 1155072~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:31:52.776 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:44.370524+0000", 2024-09-14T06:31:52.776 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 128.33843518399999, 2024-09-14T06:31:52.776 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047972262000000002, 2024-09-14T06:31:52.776 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:31:52.776 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:31:52.776 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:31:52.776 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:31:52.776 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:31:52.776 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:52.776 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 53344 2024-09-14T06:31:52.776 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:52.776 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:31:52.776 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:52.777 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:31:52.777 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370524+0000", 2024-09-14T06:31:52.777 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:52.777 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:52.777 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:52.777 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:31:52.777 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370524+0000", 2024-09-14T06:31:52.777 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:52.777 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:52.777 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:52.777 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:31:52.777 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370618+0000", 2024-09-14T06:31:52.777 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.4425999999999993e-05 2024-09-14T06:31:52.777 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:52.778 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:52.778 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:31:52.778 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370622+0000", 2024-09-14T06:31:52.778 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.3519999999999998e-06 2024-09-14T06:31:52.778 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:52.778 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:52.778 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:31:52.778 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370631+0000", 2024-09-14T06:31:52.778 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.9299999999999992e-06 2024-09-14T06:31:52.778 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:52.778 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:52.778 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:31:52.778 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370637+0000", 2024-09-14T06:31:52.778 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.3269999999999997e-06 2024-09-14T06:31:52.779 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:52.779 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:52.779 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:31:52.779 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.418372+0000", 2024-09-14T06:31:52.779 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047734764999999998 2024-09-14T06:31:52.779 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:52.779 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:52.779 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:31:52.779 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.418431+0000", 2024-09-14T06:31:52.779 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.8696999999999997e-05 2024-09-14T06:31:52.779 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:52.779 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:52.779 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:31:52.779 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.418496+0000", 2024-09-14T06:31:52.780 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.5765000000000005e-05 2024-09-14T06:31:52.780 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:52.780 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:31:52.780 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:52.780 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:52.780 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:52.780 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:53345 3.f 3:f71f51a7:::rbd_data.112dca2f6ae1.0000000000000015:17 [read 1179648~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:31:52.780 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:44.370647+0000", 2024-09-14T06:31:52.780 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 128.33831171700001, 2024-09-14T06:31:52.780 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047953775999999997, 2024-09-14T06:31:52.780 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:31:52.780 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:31:52.780 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:31:52.781 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:31:52.781 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:31:52.781 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:52.781 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 53345 2024-09-14T06:31:52.781 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:52.781 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:31:52.781 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:52.781 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:31:52.781 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370647+0000", 2024-09-14T06:31:52.781 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:52.781 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:52.781 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:52.781 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:31:52.781 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370647+0000", 2024-09-14T06:31:52.782 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:52.782 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:52.782 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:52.782 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:31:52.782 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.372346+0000", 2024-09-14T06:31:52.782 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.001698284 2024-09-14T06:31:52.782 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:52.782 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:52.782 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:31:52.782 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.372351+0000", 2024-09-14T06:31:52.782 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.8890000000000001e-06 2024-09-14T06:31:52.782 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:52.782 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:52.782 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:31:52.783 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.372362+0000", 2024-09-14T06:31:52.783 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.0941000000000001e-05 2024-09-14T06:31:52.783 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:52.783 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:52.783 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:31:52.783 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.372369+0000", 2024-09-14T06:31:52.783 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.362e-06 2024-09-14T06:31:52.783 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:52.783 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:52.783 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:31:52.783 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.418510+0000", 2024-09-14T06:31:52.783 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046140914999999998 2024-09-14T06:31:52.783 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:52.784 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:52.784 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:31:52.784 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.418549+0000", 2024-09-14T06:31:52.784 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.8717999999999998e-05 2024-09-14T06:31:52.784 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:52.784 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:52.784 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:31:52.784 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.418601+0000", 2024-09-14T06:31:52.784 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.2667000000000002e-05 2024-09-14T06:31:52.784 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:52.784 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:31:52.784 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:52.784 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:52.784 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:52.785 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:53351 3.f 3:f71f51a7:::rbd_data.112dca2f6ae1.0000000000000015:17 [read 1327104~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:31:52.785 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:44.372499+0000", 2024-09-14T06:31:52.785 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 128.33646059599999, 2024-09-14T06:31:52.785 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047140596, 2024-09-14T06:31:52.785 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:31:52.785 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:31:52.785 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:31:52.785 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:31:52.785 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:31:52.785 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:52.785 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 53351 2024-09-14T06:31:52.785 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:52.785 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:31:52.786 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:52.786 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:31:52.786 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.372499+0000", 2024-09-14T06:31:52.786 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:52.786 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:52.786 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:52.786 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:31:52.786 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.372499+0000", 2024-09-14T06:31:52.786 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:52.786 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:52.786 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:52.786 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:31:52.786 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.372499+0000", 2024-09-14T06:31:52.787 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.3000000000000001e-07 2024-09-14T06:31:52.787 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:52.787 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:52.787 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:31:52.787 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.372501+0000", 2024-09-14T06:31:52.787 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.0820000000000001e-06 2024-09-14T06:31:52.787 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:52.787 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:52.787 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:31:52.787 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.372513+0000", 2024-09-14T06:31:52.787 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.1610999999999999e-05 2024-09-14T06:31:52.787 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:52.787 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:52.787 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:31:52.788 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.372518+0000", 2024-09-14T06:31:52.788 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.327e-06 2024-09-14T06:31:52.788 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:52.788 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:52.788 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:31:52.788 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.419530+0000", 2024-09-14T06:31:52.788 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047011798 2024-09-14T06:31:52.788 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:52.788 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:52.788 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:31:52.788 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.419578+0000", 2024-09-14T06:31:52.788 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.8446999999999999e-05 2024-09-14T06:31:52.788 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:52.788 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:52.789 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:31:52.789 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.419639+0000", 2024-09-14T06:31:52.789 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.0900999999999998e-05 2024-09-14T06:31:52.789 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:52.789 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:31:52.789 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:52.789 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:52.789 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:52.789 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:53352 3.f 3:f71f51a7:::rbd_data.112dca2f6ae1.0000000000000015:17 [read 1351680~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:31:52.789 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:44.372525+0000", 2024-09-14T06:31:52.789 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 128.336434682, 2024-09-14T06:31:52.789 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047259847000000001, 2024-09-14T06:31:52.789 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:31:52.789 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:31:52.790 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:31:52.790 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:31:52.790 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:31:52.790 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:52.790 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 53352 2024-09-14T06:31:52.790 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:52.790 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:31:52.790 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:52.790 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:31:52.790 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.372525+0000", 2024-09-14T06:31:52.790 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:52.790 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:52.790 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:52.791 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:31:52.791 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.372525+0000", 2024-09-14T06:31:52.791 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:52.791 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:52.791 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:52.791 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:31:52.791 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.372525+0000", 2024-09-14T06:31:52.791 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.0600000000000001e-07 2024-09-14T06:31:52.791 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:52.791 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:52.791 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:31:52.791 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.372527+0000", 2024-09-14T06:31:52.791 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.0499999999999999e-06 2024-09-14T06:31:52.792 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:52.792 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:52.792 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:31:52.792 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.372533+0000", 2024-09-14T06:31:52.792 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.4320000000000004e-06 2024-09-14T06:31:52.792 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:52.792 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:52.792 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:31:52.792 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.372538+0000", 2024-09-14T06:31:52.792 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.9130000000000002e-06 2024-09-14T06:31:52.792 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:52.792 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:52.792 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:31:52.792 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.419667+0000", 2024-09-14T06:31:52.793 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047128976000000003 2024-09-14T06:31:52.793 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:52.793 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:52.793 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:31:52.793 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.419715+0000", 2024-09-14T06:31:52.793 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.7868999999999999e-05 2024-09-14T06:31:52.793 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:52.793 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:52.793 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:31:52.793 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.419784+0000", 2024-09-14T06:31:52.793 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.9201000000000004e-05 2024-09-14T06:31:52.793 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:52.793 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:31:52.793 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:52.794 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:52.794 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:52.794 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:53353 3.f 3:f71f51a7:::rbd_data.112dca2f6ae1.0000000000000015:17 [read 1376256~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:31:52.794 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:44.372545+0000", 2024-09-14T06:31:52.794 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 128.33641464199999, 2024-09-14T06:31:52.794 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047364232999999999, 2024-09-14T06:31:52.794 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:31:52.794 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:31:52.794 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:31:52.794 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:31:52.794 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:31:52.794 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:52.794 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 53353 2024-09-14T06:31:52.794 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:52.795 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:31:52.795 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:52.795 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:31:52.795 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.372545+0000", 2024-09-14T06:31:52.795 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:52.795 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:52.795 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:52.795 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:31:52.795 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.372545+0000", 2024-09-14T06:31:52.795 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:52.795 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:52.795 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:52.795 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:31:52.795 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.372545+0000", 2024-09-14T06:31:52.796 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.27e-07 2024-09-14T06:31:52.796 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:52.796 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:52.796 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:31:52.796 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.372547+0000", 2024-09-14T06:31:52.796 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.0600000000000002e-06 2024-09-14T06:31:52.796 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:52.796 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:52.796 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:31:52.796 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.372553+0000", 2024-09-14T06:31:52.796 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.7980000000000002e-06 2024-09-14T06:31:52.796 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:52.796 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:52.796 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:31:52.797 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.372557+0000", 2024-09-14T06:31:52.797 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.7389999999999999e-06 2024-09-14T06:31:52.797 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:52.797 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:52.797 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:31:52.797 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.419811+0000", 2024-09-14T06:31:52.797 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047253310999999999 2024-09-14T06:31:52.797 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:52.797 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:52.797 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:31:52.797 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.419854+0000", 2024-09-14T06:31:52.797 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.3448000000000001e-05 2024-09-14T06:31:52.797 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:52.798 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:52.798 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:31:52.798 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.419909+0000", 2024-09-14T06:31:52.798 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.4549999999999998e-05 2024-09-14T06:31:52.798 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:52.798 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:31:52.798 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:52.798 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:52.798 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:52.798 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:53354 3.f 3:f71f51a7:::rbd_data.112dca2f6ae1.0000000000000015:17 [read 1400832~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:31:52.798 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:44.372563+0000", 2024-09-14T06:31:52.798 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 128.336396205, 2024-09-14T06:31:52.798 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047482233999999998, 2024-09-14T06:31:52.798 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:31:52.798 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:31:52.799 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:31:52.799 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:31:52.799 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:31:52.799 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:52.799 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 53354 2024-09-14T06:31:52.799 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:52.799 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:31:52.799 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:52.799 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:31:52.799 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.372563+0000", 2024-09-14T06:31:52.799 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:52.799 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:52.799 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:52.799 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:31:52.800 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.372563+0000", 2024-09-14T06:31:52.800 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:52.800 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:52.800 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:52.800 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:31:52.800 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.375009+0000", 2024-09-14T06:31:52.800 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.0024457680000000001 2024-09-14T06:31:52.800 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:52.800 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:52.800 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:31:52.800 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.375014+0000", 2024-09-14T06:31:52.800 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.0599999999999998e-06 2024-09-14T06:31:52.800 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:52.800 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:52.801 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:31:52.801 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.375026+0000", 2024-09-14T06:31:52.801 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.1783999999999999e-05 2024-09-14T06:31:52.801 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:52.801 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:52.801 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:31:52.801 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.375035+0000", 2024-09-14T06:31:52.801 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.7799999999999995e-06 2024-09-14T06:31:52.801 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:52.801 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:52.801 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:31:52.801 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.419942+0000", 2024-09-14T06:31:52.801 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.044906982999999998 2024-09-14T06:31:52.802 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:52.802 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:52.802 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:31:52.802 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.419989+0000", 2024-09-14T06:31:52.802 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.7104999999999997e-05 2024-09-14T06:31:52.802 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:52.802 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:52.802 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:31:52.802 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.420045+0000", 2024-09-14T06:31:52.802 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.5754000000000002e-05 2024-09-14T06:31:52.802 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:52.802 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:31:52.802 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:52.802 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:52.802 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:31:52.803 INFO:teuthology.orchestra.run.smithi138.stdout:} 2024-09-14T06:31:52.804 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-14T06:31:52.805 INFO:tasks.ceph.osd.2.smithi094.stderr:2024-09-14T06:31:52.803+0000 7faf3269b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33649) UID: 0 2024-09-14T06:31:52.905 INFO:tasks.ceph.osd.4.smithi138.stderr:2024-09-14T06:31:52.903+0000 7f6c653c8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33615) UID: 0 2024-09-14T06:31:52.917 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-14T06:31:52.917 INFO:teuthology.orchestra.run.smithi094.stdout: "ops": [], 2024-09-14T06:31:52.918 INFO:teuthology.orchestra.run.smithi094.stdout: "num_ops": 0 2024-09-14T06:31:52.918 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-14T06:31:52.927 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-14T06:31:53.006 INFO:tasks.ceph.osd.7.smithi138.stderr:2024-09-14T06:31:53.005+0000 7f3c6a21d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33623) UID: 0 2024-09-14T06:31:53.041 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-14T06:31:53.042 INFO:teuthology.orchestra.run.smithi094.stdout: "ops": [], 2024-09-14T06:31:53.042 INFO:teuthology.orchestra.run.smithi094.stdout: "complaint_time": 30, 2024-09-14T06:31:53.042 INFO:teuthology.orchestra.run.smithi094.stdout: "num_blocked_ops": 0 2024-09-14T06:31:53.042 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-14T06:31:53.051 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-14T06:31:53.106 INFO:tasks.ceph.osd.4.smithi138.stderr:2024-09-14T06:31:53.105+0000 7f6c653c8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33615) UID: 0 2024-09-14T06:31:53.169 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-14T06:31:53.169 INFO:teuthology.orchestra.run.smithi094.stdout: "size": 20, 2024-09-14T06:31:53.169 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 600, 2024-09-14T06:31:53.169 INFO:teuthology.orchestra.run.smithi094.stdout: "ops": [ 2024-09-14T06:31:53.169 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:53.169 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:115193 3.1 3:856be886:::rbd_data.112d4d24c558.0000000000000004:29 [read 1523712~16384 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e96)", 2024-09-14T06:31:53.169 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:30:17.939854+0000", 2024-09-14T06:31:53.169 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 95.227091029999997, 2024-09-14T06:31:53.170 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.073897516999999996, 2024-09-14T06:31:53.170 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:31:53.170 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:31:53.170 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:31:53.170 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:31:53.170 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:31:53.170 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:53.170 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 115193 2024-09-14T06:31:53.170 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:53.170 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:31:53.170 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:53.170 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:31:53.170 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.939854+0000", 2024-09-14T06:31:53.170 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:53.171 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:53.171 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:53.171 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:31:53.171 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.939854+0000", 2024-09-14T06:31:53.171 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:53.171 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:53.171 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:53.171 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:31:53.171 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.939854+0000", 2024-09-14T06:31:53.171 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.9500000000000002e-07 2024-09-14T06:31:53.171 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:53.171 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:53.171 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:31:53.171 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.939856+0000", 2024-09-14T06:31:53.172 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.0959999999999999e-06 2024-09-14T06:31:53.172 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:53.172 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:53.172 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:31:53.172 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.939863+0000", 2024-09-14T06:31:53.172 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.1129999999999997e-06 2024-09-14T06:31:53.172 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:53.172 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:53.172 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:31:53.172 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.939867+0000", 2024-09-14T06:31:53.172 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.8350000000000003e-06 2024-09-14T06:31:53.172 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:53.172 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:53.172 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:31:53.173 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.013626+0000", 2024-09-14T06:31:53.173 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.073758799999999999 2024-09-14T06:31:53.173 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:53.173 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:53.173 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:31:53.173 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.013690+0000", 2024-09-14T06:31:53.173 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.3918999999999996e-05 2024-09-14T06:31:53.173 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:53.173 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:53.173 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:31:53.173 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.013751+0000", 2024-09-14T06:31:53.173 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.1159000000000002e-05 2024-09-14T06:31:53.173 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:53.173 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:31:53.174 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:53.174 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:53.174 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:53.174 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:115194 3.1 3:856be886:::rbd_data.112d4d24c558.0000000000000004:29 [read 1769472~16384 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e96)", 2024-09-14T06:31:53.174 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:30:17.939874+0000", 2024-09-14T06:31:53.174 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 95.227070718999997, 2024-09-14T06:31:53.175 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.074016988000000006, 2024-09-14T06:31:53.175 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:31:53.176 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:31:53.176 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:31:53.176 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:31:53.176 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:31:53.176 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:53.176 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 115194 2024-09-14T06:31:53.176 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:53.176 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:31:53.176 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:53.176 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:31:53.176 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.939874+0000", 2024-09-14T06:31:53.176 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:53.176 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:53.176 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:53.177 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:31:53.177 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.939874+0000", 2024-09-14T06:31:53.177 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:53.177 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:53.177 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:53.177 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:31:53.177 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.939875+0000", 2024-09-14T06:31:53.177 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.2600000000000002e-07 2024-09-14T06:31:53.177 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:53.177 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:53.177 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:31:53.177 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.939877+0000", 2024-09-14T06:31:53.177 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.9420000000000002e-06 2024-09-14T06:31:53.177 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:53.178 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:53.178 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:31:53.178 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.939882+0000", 2024-09-14T06:31:53.178 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.8749999999999997e-06 2024-09-14T06:31:53.178 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:53.178 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:53.178 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:31:53.178 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.939887+0000", 2024-09-14T06:31:53.178 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.741e-06 2024-09-14T06:31:53.178 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:53.178 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:53.178 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:31:53.178 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.013767+0000", 2024-09-14T06:31:53.178 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.073879720999999995 2024-09-14T06:31:53.179 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:53.179 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:53.179 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:31:53.179 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.013832+0000", 2024-09-14T06:31:53.179 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.5312000000000001e-05 2024-09-14T06:31:53.179 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:53.179 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:53.179 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:31:53.179 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.013891+0000", 2024-09-14T06:31:53.179 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.8870999999999999e-05 2024-09-14T06:31:53.179 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:53.179 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:31:53.179 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:53.179 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:53.180 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:53.180 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:115195 3.1 3:856be886:::rbd_data.112d4d24c558.0000000000000004:29 [read 2015232~16384 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e96)", 2024-09-14T06:31:53.180 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:30:17.939903+0000", 2024-09-14T06:31:53.180 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 95.227041533999994, 2024-09-14T06:31:53.180 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.074128199000000006, 2024-09-14T06:31:53.180 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:31:53.180 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:31:53.180 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:31:53.180 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:31:53.180 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:31:53.180 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:53.180 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 115195 2024-09-14T06:31:53.180 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:53.180 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:31:53.181 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:53.181 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:31:53.181 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.939903+0000", 2024-09-14T06:31:53.181 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:53.181 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:53.181 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:53.181 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:31:53.181 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.939903+0000", 2024-09-14T06:31:53.181 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:53.181 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:53.181 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:53.181 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:31:53.181 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.939904+0000", 2024-09-14T06:31:53.181 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.5799999999999999e-07 2024-09-14T06:31:53.182 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:53.182 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:53.182 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:31:53.182 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.939906+0000", 2024-09-14T06:31:53.182 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.2350000000000002e-06 2024-09-14T06:31:53.182 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:53.182 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:53.182 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:31:53.182 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.939913+0000", 2024-09-14T06:31:53.182 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.5649999999999998e-06 2024-09-14T06:31:53.182 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:53.182 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:53.182 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:31:53.182 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.939918+0000", 2024-09-14T06:31:53.183 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.784e-06 2024-09-14T06:31:53.183 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:53.183 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:53.183 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:31:53.183 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.013921+0000", 2024-09-14T06:31:53.183 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.074002911000000005 2024-09-14T06:31:53.183 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:53.183 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:53.183 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:31:53.183 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.013971+0000", 2024-09-14T06:31:53.183 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.9551e-05 2024-09-14T06:31:53.183 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:53.183 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:53.183 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:31:53.184 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.014031+0000", 2024-09-14T06:31:53.184 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.0594999999999998e-05 2024-09-14T06:31:53.184 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:53.184 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:31:53.184 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:53.184 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:53.184 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:53.184 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:115196 3.1 3:856be886:::rbd_data.112d4d24c558.0000000000000004:29 [read 2260992~16384 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e96)", 2024-09-14T06:31:53.184 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:30:17.939925+0000", 2024-09-14T06:31:53.184 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 95.227019342000006, 2024-09-14T06:31:53.184 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.074320137999999994, 2024-09-14T06:31:53.184 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:31:53.184 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:31:53.185 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:31:53.185 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:31:53.185 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:31:53.185 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:53.185 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 115196 2024-09-14T06:31:53.185 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:53.185 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:31:53.185 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:53.185 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:31:53.185 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.939925+0000", 2024-09-14T06:31:53.185 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:53.185 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:53.185 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:53.185 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:31:53.186 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.939925+0000", 2024-09-14T06:31:53.186 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:53.186 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:53.186 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:53.186 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:31:53.186 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.939926+0000", 2024-09-14T06:31:53.186 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.9999999999999998e-07 2024-09-14T06:31:53.186 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:53.186 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:53.186 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:31:53.186 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.939928+0000", 2024-09-14T06:31:53.186 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.9549999999999999e-06 2024-09-14T06:31:53.186 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:53.186 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:53.187 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:31:53.187 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.939934+0000", 2024-09-14T06:31:53.187 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.127e-06 2024-09-14T06:31:53.187 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:53.187 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:53.187 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:31:53.187 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.939939+0000", 2024-09-14T06:31:53.187 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.775e-06 2024-09-14T06:31:53.187 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:53.187 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:53.187 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:31:53.187 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.014075+0000", 2024-09-14T06:31:53.187 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.074136598999999997 2024-09-14T06:31:53.187 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:53.188 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:53.188 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:31:53.188 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.014162+0000", 2024-09-14T06:31:53.188 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.6477999999999994e-05 2024-09-14T06:31:53.188 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:53.188 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:53.188 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:31:53.188 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.014246+0000", 2024-09-14T06:31:53.188 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.3703999999999997e-05 2024-09-14T06:31:53.188 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:53.188 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:31:53.188 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:53.188 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:53.188 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:53.189 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:115197 3.1 3:856be886:::rbd_data.112d4d24c558.0000000000000004:29 [read 2506752~16384 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e96)", 2024-09-14T06:31:53.189 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:30:17.939969+0000", 2024-09-14T06:31:53.189 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 95.226975856999999, 2024-09-14T06:31:53.189 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.074528544000000002, 2024-09-14T06:31:53.189 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:31:53.189 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:31:53.189 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:31:53.189 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:31:53.189 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:31:53.189 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:53.189 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 115197 2024-09-14T06:31:53.189 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:53.189 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:31:53.190 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:53.190 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:31:53.190 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.939969+0000", 2024-09-14T06:31:53.190 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:53.190 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:53.190 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:53.190 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:31:53.190 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.939969+0000", 2024-09-14T06:31:53.190 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:53.190 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:53.190 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:53.190 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:31:53.190 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.939970+0000", 2024-09-14T06:31:53.190 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.3600000000000003e-07 2024-09-14T06:31:53.191 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:53.191 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:53.191 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:31:53.191 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.939972+0000", 2024-09-14T06:31:53.191 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.3180000000000001e-06 2024-09-14T06:31:53.191 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:53.191 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:53.191 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:31:53.191 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.939979+0000", 2024-09-14T06:31:53.191 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.8800000000000002e-06 2024-09-14T06:31:53.191 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:53.191 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:53.191 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:31:53.191 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.939984+0000", 2024-09-14T06:31:53.192 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.3510000000000001e-06 2024-09-14T06:31:53.192 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:53.192 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:53.192 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:31:53.192 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.014302+0000", 2024-09-14T06:31:53.192 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.074317416999999997 2024-09-14T06:31:53.192 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:53.192 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:53.192 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:31:53.192 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.014392+0000", 2024-09-14T06:31:53.192 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.0181999999999998e-05 2024-09-14T06:31:53.192 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:53.192 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:53.192 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:31:53.193 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.014497+0000", 2024-09-14T06:31:53.193 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00010576 2024-09-14T06:31:53.193 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:53.193 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:31:53.193 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:53.193 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:53.193 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:53.193 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:115198 3.1 3:856be886:::rbd_data.112d4d24c558.0000000000000004:29 [read 2752512~16384 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e96)", 2024-09-14T06:31:53.193 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:30:17.940004+0000", 2024-09-14T06:31:53.193 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 95.226940576999993, 2024-09-14T06:31:53.193 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.074736367999999997, 2024-09-14T06:31:53.193 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:31:53.193 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:31:53.193 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:31:53.194 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:31:53.194 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:31:53.194 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:53.194 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 115198 2024-09-14T06:31:53.194 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:53.194 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:31:53.194 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:53.194 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:31:53.194 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940004+0000", 2024-09-14T06:31:53.194 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:53.194 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:53.194 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:53.194 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:31:53.194 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940004+0000", 2024-09-14T06:31:53.194 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:53.195 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:53.195 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:53.195 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:31:53.195 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940005+0000", 2024-09-14T06:31:53.195 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.2200000000000004e-07 2024-09-14T06:31:53.195 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:53.195 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:53.195 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:31:53.195 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940007+0000", 2024-09-14T06:31:53.195 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.322e-06 2024-09-14T06:31:53.195 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:53.195 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:53.195 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:31:53.195 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940014+0000", 2024-09-14T06:31:53.196 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.6220000000000003e-06 2024-09-14T06:31:53.196 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:53.196 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:53.196 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:31:53.196 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940019+0000", 2024-09-14T06:31:53.196 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.6380000000000001e-06 2024-09-14T06:31:53.196 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:53.196 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:53.196 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:31:53.196 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.014562+0000", 2024-09-14T06:31:53.196 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.074542320999999995 2024-09-14T06:31:53.196 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:53.196 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:53.196 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:31:53.197 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.014646+0000", 2024-09-14T06:31:53.197 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.4079999999999995e-05 2024-09-14T06:31:53.197 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:53.197 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:53.197 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:31:53.197 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.014741+0000", 2024-09-14T06:31:53.197 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.4763000000000001e-05 2024-09-14T06:31:53.197 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:53.197 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:31:53.197 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:53.197 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:53.197 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:53.197 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:115199 3.1 3:856be886:::rbd_data.112d4d24c558.0000000000000004:29 [read 2998272~16384 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e96)", 2024-09-14T06:31:53.198 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:30:17.940040+0000", 2024-09-14T06:31:53.198 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 95.226904821999995, 2024-09-14T06:31:53.198 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.074957134999999994, 2024-09-14T06:31:53.198 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:31:53.198 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:31:53.198 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:31:53.198 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:31:53.198 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:31:53.198 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:53.198 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 115199 2024-09-14T06:31:53.198 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:53.198 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:31:53.198 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:53.198 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:31:53.198 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940040+0000", 2024-09-14T06:31:53.199 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:53.199 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:53.199 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:53.199 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:31:53.199 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940040+0000", 2024-09-14T06:31:53.199 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:53.199 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:53.199 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:53.199 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:31:53.199 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940041+0000", 2024-09-14T06:31:53.199 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.4000000000000001e-07 2024-09-14T06:31:53.199 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:53.199 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:53.199 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:31:53.199 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940046+0000", 2024-09-14T06:31:53.200 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.7250000000000002e-06 2024-09-14T06:31:53.200 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:53.200 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:53.200 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:31:53.200 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940053+0000", 2024-09-14T06:31:53.200 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.6109999999999999e-06 2024-09-14T06:31:53.200 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:53.200 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:53.200 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:31:53.200 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940058+0000", 2024-09-14T06:31:53.200 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.1590000000000001e-06 2024-09-14T06:31:53.200 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:53.200 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:53.200 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:31:53.200 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.014768+0000", 2024-09-14T06:31:53.201 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.074709624000000002 2024-09-14T06:31:53.201 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:53.201 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:53.201 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:31:53.201 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.014907+0000", 2024-09-14T06:31:53.201 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000139665 2024-09-14T06:31:53.201 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:53.201 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:53.201 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:31:53.201 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.014997+0000", 2024-09-14T06:31:53.201 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.9710999999999995e-05 2024-09-14T06:31:53.201 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:53.201 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:31:53.202 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:53.202 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:53.202 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:53.202 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:115200 3.1 3:856be886:::rbd_data.112d4d24c558.0000000000000004:29 [read 3244032~16384 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e96)", 2024-09-14T06:31:53.202 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:30:17.940080+0000", 2024-09-14T06:31:53.202 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 95.226864617000004, 2024-09-14T06:31:53.202 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.075137291999999994, 2024-09-14T06:31:53.202 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:31:53.202 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:31:53.202 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:31:53.202 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:31:53.202 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:31:53.202 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:53.202 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 115200 2024-09-14T06:31:53.203 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:53.203 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:31:53.203 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:53.203 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:31:53.203 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940080+0000", 2024-09-14T06:31:53.203 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:53.203 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:53.203 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:53.203 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:31:53.203 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940080+0000", 2024-09-14T06:31:53.203 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:53.203 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:53.203 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:53.204 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:31:53.204 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940081+0000", 2024-09-14T06:31:53.204 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.0800000000000004e-07 2024-09-14T06:31:53.204 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:53.204 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:53.204 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:31:53.204 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940083+0000", 2024-09-14T06:31:53.204 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.3920000000000001e-06 2024-09-14T06:31:53.204 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:53.204 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:53.204 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:31:53.204 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940091+0000", 2024-09-14T06:31:53.204 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.1389999999999995e-06 2024-09-14T06:31:53.205 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:53.205 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:53.205 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:31:53.205 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940097+0000", 2024-09-14T06:31:53.205 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.5550000000000002e-06 2024-09-14T06:31:53.205 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:53.205 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:53.205 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:31:53.205 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.015025+0000", 2024-09-14T06:31:53.205 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.074928579999999995 2024-09-14T06:31:53.205 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:53.205 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:53.205 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:31:53.205 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.015096+0000", 2024-09-14T06:31:53.206 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.0705999999999997e-05 2024-09-14T06:31:53.206 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:53.206 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:53.206 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:31:53.206 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.015217+0000", 2024-09-14T06:31:53.206 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00012131200000000001 2024-09-14T06:31:53.206 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:53.206 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:31:53.206 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:53.206 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:53.206 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:53.206 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:115201 3.1 3:856be886:::rbd_data.112d4d24c558.0000000000000004:29 [read 3489792~16384 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e96)", 2024-09-14T06:31:53.206 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:30:17.940113+0000", 2024-09-14T06:31:53.206 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 95.226831844000003, 2024-09-14T06:31:53.207 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.075246899000000006, 2024-09-14T06:31:53.207 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:31:53.207 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:31:53.207 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:31:53.207 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:31:53.207 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:31:53.207 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:53.207 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 115201 2024-09-14T06:31:53.207 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:53.207 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:31:53.207 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:53.207 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:31:53.207 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940113+0000", 2024-09-14T06:31:53.207 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:53.208 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:53.208 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:53.208 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:31:53.208 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940113+0000", 2024-09-14T06:31:53.208 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:53.208 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:53.208 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:53.208 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:31:53.208 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940114+0000", 2024-09-14T06:31:53.208 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.1500000000000004e-07 2024-09-14T06:31:53.208 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:53.208 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:53.208 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:31:53.208 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940116+0000", 2024-09-14T06:31:53.209 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.2809999999999998e-06 2024-09-14T06:31:53.209 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:53.209 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:53.209 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:31:53.209 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940123+0000", 2024-09-14T06:31:53.209 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.9229999999999996e-06 2024-09-14T06:31:53.209 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:53.209 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:53.209 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:31:53.209 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940128+0000", 2024-09-14T06:31:53.209 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.7790000000000003e-06 2024-09-14T06:31:53.209 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:53.209 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:53.209 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:31:53.210 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.015253+0000", 2024-09-14T06:31:53.210 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.075125792999999996 2024-09-14T06:31:53.210 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:53.210 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:53.210 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:31:53.210 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.015297+0000", 2024-09-14T06:31:53.210 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.3847000000000002e-05 2024-09-14T06:31:53.210 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:53.210 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:53.210 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:31:53.210 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.015360+0000", 2024-09-14T06:31:53.210 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.2661000000000002e-05 2024-09-14T06:31:53.210 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:53.210 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:31:53.211 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:53.211 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:53.211 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:53.211 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:115202 3.1 3:856be886:::rbd_data.112d4d24c558.0000000000000004:29 [read 3735552~16384 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e96)", 2024-09-14T06:31:53.211 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:30:17.940135+0000", 2024-09-14T06:31:53.211 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 95.226810064999995, 2024-09-14T06:31:53.211 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.075451105000000004, 2024-09-14T06:31:53.211 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:31:53.211 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:31:53.211 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:31:53.211 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:31:53.211 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:31:53.211 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:53.211 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 115202 2024-09-14T06:31:53.211 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:53.212 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:31:53.212 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:53.212 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:31:53.212 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940135+0000", 2024-09-14T06:31:53.212 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:53.212 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:53.212 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:53.212 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:31:53.212 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940135+0000", 2024-09-14T06:31:53.212 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:53.212 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:53.212 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:53.212 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:31:53.213 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940135+0000", 2024-09-14T06:31:53.213 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.0999999999999999e-07 2024-09-14T06:31:53.213 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:53.213 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:53.213 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:31:53.213 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940137+0000", 2024-09-14T06:31:53.213 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.0020000000000001e-06 2024-09-14T06:31:53.213 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:53.213 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:53.213 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:31:53.213 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940143+0000", 2024-09-14T06:31:53.213 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.9850000000000003e-06 2024-09-14T06:31:53.213 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:53.214 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:53.214 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:31:53.214 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940148+0000", 2024-09-14T06:31:53.214 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.9880000000000004e-06 2024-09-14T06:31:53.214 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:53.214 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:53.214 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:31:53.214 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.015400+0000", 2024-09-14T06:31:53.214 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.075252186999999998 2024-09-14T06:31:53.214 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:53.214 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:53.214 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:31:53.214 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.015481+0000", 2024-09-14T06:31:53.214 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.0698000000000006e-05 2024-09-14T06:31:53.214 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:53.215 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:53.215 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:31:53.215 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.015586+0000", 2024-09-14T06:31:53.215 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00010483499999999999 2024-09-14T06:31:53.215 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:53.215 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:31:53.215 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:53.215 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:53.215 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:53.215 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:115203 3.1 3:856be886:::rbd_data.112d4d24c558.0000000000000004:29 [read 3981312~16384 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e96)", 2024-09-14T06:31:53.215 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:30:17.940156+0000", 2024-09-14T06:31:53.215 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 95.226788839999998, 2024-09-14T06:31:53.215 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.075620860999999998, 2024-09-14T06:31:53.216 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:31:53.216 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:31:53.216 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:31:53.216 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:31:53.216 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:31:53.216 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:53.216 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 115203 2024-09-14T06:31:53.216 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:53.216 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:31:53.216 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:53.216 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:31:53.216 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940156+0000", 2024-09-14T06:31:53.216 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:53.216 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:53.217 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:53.217 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:31:53.217 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940156+0000", 2024-09-14T06:31:53.217 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:53.217 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:53.217 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:53.217 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:31:53.217 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940156+0000", 2024-09-14T06:31:53.217 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.1500000000000005e-07 2024-09-14T06:31:53.217 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:53.217 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:53.217 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:31:53.217 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940158+0000", 2024-09-14T06:31:53.218 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.967e-06 2024-09-14T06:31:53.218 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:53.218 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:53.218 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:31:53.218 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940165+0000", 2024-09-14T06:31:53.218 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.6939999999999998e-06 2024-09-14T06:31:53.218 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:53.218 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:53.218 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:31:53.218 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940170+0000", 2024-09-14T06:31:53.218 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.7539999999999997e-06 2024-09-14T06:31:53.218 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:53.218 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:53.218 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:31:53.218 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.015633+0000", 2024-09-14T06:31:53.219 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.075463334000000007 2024-09-14T06:31:53.219 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:53.219 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:53.219 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:31:53.219 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.015693+0000", 2024-09-14T06:31:53.219 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.9367000000000003e-05 2024-09-14T06:31:53.219 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:53.219 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:53.219 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:31:53.219 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.015777+0000", 2024-09-14T06:31:53.219 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.4229999999999998e-05 2024-09-14T06:31:53.219 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:53.219 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:31:53.219 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:53.219 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:53.220 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:53.220 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:115204 3.1 3:856be886:::rbd_data.112d4d24c558.0000000000000004:29 [read 4227072~16384 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e96)", 2024-09-14T06:31:53.220 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:30:17.940176+0000", 2024-09-14T06:31:53.220 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 95.226768307, 2024-09-14T06:31:53.220 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.075785505000000003, 2024-09-14T06:31:53.220 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:31:53.220 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:31:53.220 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:31:53.220 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:31:53.220 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:31:53.220 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:53.220 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 115204 2024-09-14T06:31:53.220 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:53.220 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:31:53.221 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:53.221 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:31:53.221 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940176+0000", 2024-09-14T06:31:53.221 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:53.221 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:53.221 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:53.221 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:31:53.221 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940176+0000", 2024-09-14T06:31:53.221 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:53.221 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:53.221 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:53.221 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:31:53.221 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940177+0000", 2024-09-14T06:31:53.222 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.7899999999999999e-07 2024-09-14T06:31:53.222 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:53.222 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:53.222 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:31:53.222 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940179+0000", 2024-09-14T06:31:53.222 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.103e-06 2024-09-14T06:31:53.222 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:53.222 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:53.222 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:31:53.222 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940185+0000", 2024-09-14T06:31:53.222 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.7239999999999997e-06 2024-09-14T06:31:53.222 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:53.222 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:53.222 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:31:53.222 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940190+0000", 2024-09-14T06:31:53.223 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.7269999999999998e-06 2024-09-14T06:31:53.223 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:53.223 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:53.223 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:31:53.223 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.015814+0000", 2024-09-14T06:31:53.223 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.075624114000000006 2024-09-14T06:31:53.223 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:53.223 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:53.223 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:31:53.223 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.015862+0000", 2024-09-14T06:31:53.223 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.8770999999999998e-05 2024-09-14T06:31:53.223 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:53.223 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:53.224 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:31:53.224 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.015962+0000", 2024-09-14T06:31:53.224 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.9586999999999994e-05 2024-09-14T06:31:53.224 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:53.224 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:31:53.224 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:53.224 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:53.224 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:53.224 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:115205 3.1 3:856be886:::rbd_data.112d4d24c558.0000000000000004:29 [read 4472832~16384 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e96)", 2024-09-14T06:31:53.224 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:30:17.940197+0000", 2024-09-14T06:31:53.224 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 95.226747598000003, 2024-09-14T06:31:53.224 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.076068920999999998, 2024-09-14T06:31:53.224 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:31:53.225 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:31:53.225 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:31:53.225 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:31:53.225 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:31:53.225 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:53.225 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 115205 2024-09-14T06:31:53.225 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:53.225 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:31:53.225 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:53.225 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:31:53.225 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940197+0000", 2024-09-14T06:31:53.225 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:53.225 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:53.226 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:53.226 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:31:53.226 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940197+0000", 2024-09-14T06:31:53.226 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:53.226 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:53.226 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:53.226 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:31:53.226 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940198+0000", 2024-09-14T06:31:53.226 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.5999999999999999e-07 2024-09-14T06:31:53.226 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:53.226 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:53.226 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:31:53.226 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940200+0000", 2024-09-14T06:31:53.226 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.0899999999999999e-06 2024-09-14T06:31:53.226 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:53.227 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:53.227 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:31:53.227 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940206+0000", 2024-09-14T06:31:53.227 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.8710000000000002e-06 2024-09-14T06:31:53.227 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:53.227 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:53.227 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:31:53.227 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940211+0000", 2024-09-14T06:31:53.227 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.9230000000000001e-06 2024-09-14T06:31:53.227 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:53.227 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:53.227 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:31:53.227 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.016059+0000", 2024-09-14T06:31:53.227 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.075848051999999999 2024-09-14T06:31:53.228 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:53.228 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:53.228 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:31:53.228 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.016151+0000", 2024-09-14T06:31:53.228 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.1982000000000001e-05 2024-09-14T06:31:53.228 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:53.228 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:53.228 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:31:53.228 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.016266+0000", 2024-09-14T06:31:53.228 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000115543 2024-09-14T06:31:53.228 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:53.228 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:31:53.228 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:53.228 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:53.229 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:53.229 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:115206 3.1 3:856be886:::rbd_data.112d4d24c558.0000000000000004:29 [read 4718592~16384 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e96)", 2024-09-14T06:31:53.229 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:30:17.940223+0000", 2024-09-14T06:31:53.229 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 95.226721507999997, 2024-09-14T06:31:53.229 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.076265115999999994, 2024-09-14T06:31:53.229 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:31:53.229 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:31:53.229 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:31:53.229 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:31:53.229 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:31:53.229 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:53.229 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 115206 2024-09-14T06:31:53.229 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:53.229 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:31:53.230 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:53.230 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:31:53.230 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940223+0000", 2024-09-14T06:31:53.230 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:53.230 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:53.230 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:53.230 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:31:53.230 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940223+0000", 2024-09-14T06:31:53.230 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:53.230 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:53.230 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:53.230 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:31:53.230 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940224+0000", 2024-09-14T06:31:53.230 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.0399999999999996e-07 2024-09-14T06:31:53.231 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:53.231 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:53.231 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:31:53.231 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940226+0000", 2024-09-14T06:31:53.231 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.1780000000000002e-06 2024-09-14T06:31:53.231 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:53.231 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:53.231 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:31:53.231 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940232+0000", 2024-09-14T06:31:53.231 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.5010000000000001e-06 2024-09-14T06:31:53.231 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:53.231 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:53.231 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:31:53.231 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940237+0000", 2024-09-14T06:31:53.231 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.0250000000000002e-06 2024-09-14T06:31:53.231 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:53.232 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:53.232 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:31:53.232 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.016305+0000", 2024-09-14T06:31:53.232 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.076067262999999996 2024-09-14T06:31:53.232 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:53.232 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:53.232 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:31:53.232 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.016383+0000", 2024-09-14T06:31:53.232 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.8578000000000006e-05 2024-09-14T06:31:53.232 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:53.232 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:53.232 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:31:53.232 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.016488+0000", 2024-09-14T06:31:53.232 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000105067 2024-09-14T06:31:53.233 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:53.233 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:31:53.233 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:53.233 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:53.233 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:53.233 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:115207 3.1 3:856be886:::rbd_data.112d4d24c558.0000000000000004:29 [read 4964352~16384 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e96)", 2024-09-14T06:31:53.233 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:30:17.940244+0000", 2024-09-14T06:31:53.233 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 95.226700391999998, 2024-09-14T06:31:53.233 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.076371154999999996, 2024-09-14T06:31:53.233 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:31:53.233 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:31:53.233 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:31:53.233 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:31:53.233 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:31:53.234 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:53.234 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 115207 2024-09-14T06:31:53.234 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:53.234 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:31:53.234 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:53.234 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:31:53.234 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940244+0000", 2024-09-14T06:31:53.234 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:53.234 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:53.234 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:53.234 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:31:53.234 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940244+0000", 2024-09-14T06:31:53.234 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:53.234 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:53.234 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:53.234 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:31:53.235 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940245+0000", 2024-09-14T06:31:53.235 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.1200000000000003e-07 2024-09-14T06:31:53.235 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:53.235 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:53.235 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:31:53.235 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940247+0000", 2024-09-14T06:31:53.235 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.0030000000000001e-06 2024-09-14T06:31:53.235 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:53.235 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:53.235 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:31:53.235 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940252+0000", 2024-09-14T06:31:53.235 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.5520000000000004e-06 2024-09-14T06:31:53.235 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:53.235 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:53.235 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:31:53.236 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940257+0000", 2024-09-14T06:31:53.236 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.6759999999999998e-06 2024-09-14T06:31:53.236 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:53.236 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:53.236 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:31:53.236 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.016521+0000", 2024-09-14T06:31:53.236 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.076263499999999998 2024-09-14T06:31:53.236 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:53.236 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:53.236 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:31:53.236 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.016562+0000", 2024-09-14T06:31:53.236 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.0967000000000003e-05 2024-09-14T06:31:53.236 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:53.236 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:53.237 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:31:53.237 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.016616+0000", 2024-09-14T06:31:53.237 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.3944999999999999e-05 2024-09-14T06:31:53.237 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:53.237 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:31:53.237 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:53.237 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:53.237 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:53.237 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:115208 3.1 3:856be886:::rbd_data.112d4d24c558.0000000000000004:29 [read 5210112~16384 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e96)", 2024-09-14T06:31:53.237 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:30:17.940264+0000", 2024-09-14T06:31:53.237 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 95.226680535, 2024-09-14T06:31:53.237 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.076558288000000002, 2024-09-14T06:31:53.237 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:31:53.237 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:31:53.238 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:31:53.238 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:31:53.238 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:31:53.238 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:53.238 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 115208 2024-09-14T06:31:53.238 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:53.238 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:31:53.238 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:53.238 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:31:53.238 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940264+0000", 2024-09-14T06:31:53.238 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:53.238 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:53.238 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:53.238 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:31:53.239 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940264+0000", 2024-09-14T06:31:53.239 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:53.239 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:53.239 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:53.239 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:31:53.239 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940265+0000", 2024-09-14T06:31:53.239 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.0500000000000004e-07 2024-09-14T06:31:53.239 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:53.239 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:53.239 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:31:53.239 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940267+0000", 2024-09-14T06:31:53.239 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.9810000000000002e-06 2024-09-14T06:31:53.239 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:53.239 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:53.240 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:31:53.240 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940272+0000", 2024-09-14T06:31:53.240 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.6999999999999996e-06 2024-09-14T06:31:53.240 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:53.240 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:53.240 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:31:53.240 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940277+0000", 2024-09-14T06:31:53.240 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.6480000000000002e-06 2024-09-14T06:31:53.240 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:53.240 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:53.240 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:31:53.240 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.016650+0000", 2024-09-14T06:31:53.240 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.076373089000000005 2024-09-14T06:31:53.241 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:53.241 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:53.241 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:31:53.241 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.016724+0000", 2024-09-14T06:31:53.241 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.4105999999999998e-05 2024-09-14T06:31:53.241 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:53.241 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:53.241 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:31:53.241 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.016823+0000", 2024-09-14T06:31:53.241 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.8258999999999996e-05 2024-09-14T06:31:53.241 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:53.241 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:31:53.241 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:53.242 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:53.242 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:53.242 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:115209 3.1 3:856be886:::rbd_data.112d4d24c558.0000000000000004:29 [read 5455872~16384 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e96)", 2024-09-14T06:31:53.242 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:30:17.940283+0000", 2024-09-14T06:31:53.242 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 95.226661508999996, 2024-09-14T06:31:53.242 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.076724029999999999, 2024-09-14T06:31:53.242 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:31:53.242 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:31:53.242 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:31:53.242 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:31:53.242 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:31:53.242 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:53.242 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 115209 2024-09-14T06:31:53.242 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:53.243 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:31:53.243 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:53.243 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:31:53.243 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940283+0000", 2024-09-14T06:31:53.243 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:53.243 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:53.243 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:53.243 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:31:53.243 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940283+0000", 2024-09-14T06:31:53.243 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:53.243 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:53.243 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:53.243 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:31:53.243 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940284+0000", 2024-09-14T06:31:53.244 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.6700000000000003e-07 2024-09-14T06:31:53.244 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:53.244 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:53.244 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:31:53.244 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940286+0000", 2024-09-14T06:31:53.244 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.9520000000000001e-06 2024-09-14T06:31:53.244 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:53.244 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:53.244 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:31:53.244 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940292+0000", 2024-09-14T06:31:53.244 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.7339999999999996e-06 2024-09-14T06:31:53.244 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:53.244 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:53.245 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:31:53.245 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940296+0000", 2024-09-14T06:31:53.245 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.7700000000000001e-06 2024-09-14T06:31:53.245 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:53.245 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:53.245 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:31:53.245 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.016859+0000", 2024-09-14T06:31:53.245 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.07656317 2024-09-14T06:31:53.245 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:53.245 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:53.245 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:31:53.245 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.016929+0000", 2024-09-14T06:31:53.245 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.9679999999999997e-05 2024-09-14T06:31:53.245 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:53.246 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:53.246 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:31:53.246 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.017007+0000", 2024-09-14T06:31:53.246 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.8157000000000004e-05 2024-09-14T06:31:53.246 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:53.246 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:31:53.246 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:53.246 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:53.246 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:53.246 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:115210 3.1 3:856be886:::rbd_data.112d4d24c558.0000000000000004:29 [read 5701632~16384 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e96)", 2024-09-14T06:31:53.246 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:30:17.940303+0000", 2024-09-14T06:31:53.246 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 95.226641991999998, 2024-09-14T06:31:53.246 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.076821358000000006, 2024-09-14T06:31:53.246 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:31:53.247 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:31:53.247 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:31:53.247 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:31:53.247 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:31:53.247 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:53.247 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 115210 2024-09-14T06:31:53.247 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:53.247 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:31:53.247 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:53.247 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:31:53.247 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940303+0000", 2024-09-14T06:31:53.247 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:53.247 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:53.247 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:53.248 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:31:53.248 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940303+0000", 2024-09-14T06:31:53.248 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:53.248 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:53.248 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:53.248 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:31:53.248 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940303+0000", 2024-09-14T06:31:53.248 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.3000000000000001e-07 2024-09-14T06:31:53.248 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:53.248 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:53.248 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:31:53.248 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940306+0000", 2024-09-14T06:31:53.248 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.255e-06 2024-09-14T06:31:53.249 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:53.249 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:53.249 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:31:53.249 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940312+0000", 2024-09-14T06:31:53.249 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.9710000000000001e-06 2024-09-14T06:31:53.249 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:53.249 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:53.249 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:31:53.249 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940316+0000", 2024-09-14T06:31:53.249 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.5340000000000001e-06 2024-09-14T06:31:53.249 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:53.249 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:53.249 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:31:53.249 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.017033+0000", 2024-09-14T06:31:53.250 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.076717169000000002 2024-09-14T06:31:53.250 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:53.250 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:53.250 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:31:53.250 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.017073+0000", 2024-09-14T06:31:53.250 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.9582000000000002e-05 2024-09-14T06:31:53.250 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:53.250 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:53.250 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:31:53.250 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.017124+0000", 2024-09-14T06:31:53.250 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.1316999999999997e-05 2024-09-14T06:31:53.250 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:53.250 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:31:53.250 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:53.251 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:53.251 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:53.251 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:115219 3.1 3:856be886:::rbd_data.112d4d24c558.0000000000000004:29 [read 65536~16384 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e96)", 2024-09-14T06:31:53.251 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:30:17.940323+0000", 2024-09-14T06:31:53.251 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 95.226622130999999, 2024-09-14T06:31:53.251 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.076915205, 2024-09-14T06:31:53.251 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:31:53.251 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:31:53.251 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:31:53.251 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:31:53.251 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:31:53.251 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:53.251 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 115219 2024-09-14T06:31:53.251 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:53.251 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:31:53.252 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:53.252 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:31:53.252 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940323+0000", 2024-09-14T06:31:53.252 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:53.252 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:53.252 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:53.252 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:31:53.252 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940323+0000", 2024-09-14T06:31:53.252 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:53.252 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:53.252 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:53.252 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:31:53.252 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940323+0000", 2024-09-14T06:31:53.253 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.0800000000000005e-07 2024-09-14T06:31:53.253 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:53.253 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:53.253 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:31:53.253 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940328+0000", 2024-09-14T06:31:53.253 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.887e-06 2024-09-14T06:31:53.253 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:53.253 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:53.253 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:31:53.253 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940334+0000", 2024-09-14T06:31:53.253 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.3269999999999997e-06 2024-09-14T06:31:53.253 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:53.253 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:53.254 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:31:53.254 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940339+0000", 2024-09-14T06:31:53.254 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.7960000000000003e-06 2024-09-14T06:31:53.254 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:53.254 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:53.254 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:31:53.254 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.017148+0000", 2024-09-14T06:31:53.254 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.076809125000000006 2024-09-14T06:31:53.254 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:53.254 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:53.254 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:31:53.254 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.017186+0000", 2024-09-14T06:31:53.254 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.7283000000000002e-05 2024-09-14T06:31:53.254 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:53.255 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:53.255 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:31:53.255 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.017238+0000", 2024-09-14T06:31:53.255 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.2278999999999999e-05 2024-09-14T06:31:53.255 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:53.255 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:31:53.255 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:53.255 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:53.255 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:53.255 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:115220 3.1 3:856be886:::rbd_data.112d4d24c558.0000000000000004:29 [read 311296~16384 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e96)", 2024-09-14T06:31:53.255 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:30:17.940346+0000", 2024-09-14T06:31:53.255 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 95.226599002, 2024-09-14T06:31:53.255 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.077033293000000003, 2024-09-14T06:31:53.255 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:31:53.255 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:31:53.256 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:31:53.256 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:31:53.256 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:31:53.256 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:53.256 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 115220 2024-09-14T06:31:53.256 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:53.256 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:31:53.256 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:53.256 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:31:53.256 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940346+0000", 2024-09-14T06:31:53.256 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:53.256 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:53.256 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:53.257 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:31:53.257 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940346+0000", 2024-09-14T06:31:53.257 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:53.257 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:53.257 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:53.257 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:31:53.257 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940346+0000", 2024-09-14T06:31:53.257 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.4300000000000003e-07 2024-09-14T06:31:53.257 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:53.257 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:53.257 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:31:53.257 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940348+0000", 2024-09-14T06:31:53.257 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.0030000000000001e-06 2024-09-14T06:31:53.257 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:53.258 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:53.258 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:31:53.258 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.004655+0000", 2024-09-14T06:31:53.258 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.064306323999999998 2024-09-14T06:31:53.258 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:53.258 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:53.258 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:31:53.258 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.004671+0000", 2024-09-14T06:31:53.258 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.6589999999999999e-05 2024-09-14T06:31:53.258 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:53.258 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:53.258 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:31:53.258 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.017262+0000", 2024-09-14T06:31:53.258 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.012590570000000001 2024-09-14T06:31:53.259 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:53.259 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:53.259 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:31:53.259 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.017321+0000", 2024-09-14T06:31:53.259 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.9210999999999999e-05 2024-09-14T06:31:53.259 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:53.259 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:53.259 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:31:53.259 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.017379+0000", 2024-09-14T06:31:53.259 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.8051999999999999e-05 2024-09-14T06:31:53.259 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:53.259 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:31:53.259 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:53.259 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:53.260 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:31:53.260 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-14T06:31:53.261 INFO:tasks.ceph.osd.7.smithi138.stderr:2024-09-14T06:31:53.260+0000 7f3c6a21d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33623) UID: 0 2024-09-14T06:31:53.262 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-14T06:31:53.362 INFO:tasks.ceph.osd.0.smithi094.stderr:2024-09-14T06:31:53.361+0000 7f07c9333640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33636) UID: 0 2024-09-14T06:31:53.374 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-14T06:31:53.375 INFO:teuthology.orchestra.run.smithi094.stdout: "ops": [], 2024-09-14T06:31:53.375 INFO:teuthology.orchestra.run.smithi094.stdout: "num_ops": 0 2024-09-14T06:31:53.375 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-14T06:31:53.384 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-14T06:31:53.469 INFO:tasks.ceph.osd.1.smithi094.stderr:2024-09-14T06:31:53.468+0000 7f328e47a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33641) UID: 0 2024-09-14T06:31:53.536 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-14T06:31:53.536 INFO:teuthology.orchestra.run.smithi094.stdout: "ops": [], 2024-09-14T06:31:53.536 INFO:teuthology.orchestra.run.smithi094.stdout: "complaint_time": 30, 2024-09-14T06:31:53.536 INFO:teuthology.orchestra.run.smithi094.stdout: "num_blocked_ops": 0 2024-09-14T06:31:53.536 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-14T06:31:53.545 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-14T06:31:53.586 INFO:tasks.ceph.osd.1.smithi094.stderr:2024-09-14T06:31:53.585+0000 7f328e47a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33641) UID: 0 2024-09-14T06:31:53.635 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-14T06:31:53.679 INFO:tasks.ceph.osd.2.smithi094.stderr:2024-09-14T06:31:53.678+0000 7faf3269b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33649) UID: 0 2024-09-14T06:31:53.703 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-14T06:31:53.703 INFO:teuthology.orchestra.run.smithi094.stdout: "size": 20, 2024-09-14T06:31:53.703 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 600, 2024-09-14T06:31:53.703 INFO:teuthology.orchestra.run.smithi094.stdout: "ops": [ 2024-09-14T06:31:53.703 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:53.703 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:14918 3.0 3:05c39535:::rbd_data.112d92027590.0000000000000009:f [read 795648~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:31:53.703 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:26.083834+0000", 2024-09-14T06:31:53.703 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 147.61733009599999, 2024-09-14T06:31:53.703 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.041776287000000002, 2024-09-14T06:31:53.703 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:31:53.704 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:31:53.704 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:31:53.704 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:31:53.704 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:31:53.704 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:53.704 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 14918 2024-09-14T06:31:53.704 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:53.704 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:31:53.704 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:53.704 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:31:53.704 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.083834+0000", 2024-09-14T06:31:53.704 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:53.704 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:53.705 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:53.705 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:31:53.705 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.083834+0000", 2024-09-14T06:31:53.705 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:53.705 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:53.705 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:53.705 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:31:53.705 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.083834+0000", 2024-09-14T06:31:53.705 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.0100000000000001e-07 2024-09-14T06:31:53.705 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:53.705 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:53.705 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:31:53.705 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.083835+0000", 2024-09-14T06:31:53.705 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.3900000000000003e-07 2024-09-14T06:31:53.706 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:53.706 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:53.706 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:31:53.706 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.083838+0000", 2024-09-14T06:31:53.706 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.9830000000000001e-06 2024-09-14T06:31:53.706 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:53.706 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:53.706 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:31:53.706 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.083840+0000", 2024-09-14T06:31:53.706 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.4679999999999999e-06 2024-09-14T06:31:53.706 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:53.706 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:53.706 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:31:53.707 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.125455+0000", 2024-09-14T06:31:53.707 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.041614578999999999 2024-09-14T06:31:53.707 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:53.707 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:53.707 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:31:53.707 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.125496+0000", 2024-09-14T06:31:53.707 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.1047000000000002e-05 2024-09-14T06:31:53.707 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:53.707 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:53.707 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:31:53.707 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.125610+0000", 2024-09-14T06:31:53.707 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00011407 2024-09-14T06:31:53.707 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:53.707 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:31:53.708 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:53.708 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:53.708 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:53.708 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:14919 3.0 3:05c39535:::rbd_data.112d92027590.0000000000000009:f [read 805888~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:31:53.708 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:26.083843+0000", 2024-09-14T06:31:53.708 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 147.61732034799999, 2024-09-14T06:31:53.708 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.041992356000000002, 2024-09-14T06:31:53.708 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:31:53.708 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:31:53.708 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:31:53.708 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:31:53.708 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:31:53.708 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:53.708 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 14919 2024-09-14T06:31:53.708 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:53.709 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:31:53.709 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:53.709 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:31:53.709 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.083843+0000", 2024-09-14T06:31:53.709 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:53.709 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:53.709 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:53.709 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:31:53.709 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.083843+0000", 2024-09-14T06:31:53.709 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:53.709 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:53.709 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:53.709 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:31:53.709 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.083844+0000", 2024-09-14T06:31:53.710 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.3300000000000001e-07 2024-09-14T06:31:53.710 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:53.710 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:53.710 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:31:53.710 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.083845+0000", 2024-09-14T06:31:53.710 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.4729999999999999e-06 2024-09-14T06:31:53.710 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:53.710 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:53.710 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:31:53.710 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.083850+0000", 2024-09-14T06:31:53.710 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.0130000000000001e-06 2024-09-14T06:31:53.710 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:53.710 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:53.710 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:31:53.710 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.083854+0000", 2024-09-14T06:31:53.711 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.4759999999999998e-06 2024-09-14T06:31:53.711 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:53.711 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:53.711 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:31:53.711 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.125636+0000", 2024-09-14T06:31:53.711 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.041782683000000001 2024-09-14T06:31:53.711 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:53.711 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:53.711 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:31:53.711 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.125680+0000", 2024-09-14T06:31:53.711 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.3614e-05 2024-09-14T06:31:53.711 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:53.711 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:53.712 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:31:53.712 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.125836+0000", 2024-09-14T06:31:53.712 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000155864 2024-09-14T06:31:53.712 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:53.712 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:31:53.712 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:53.712 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:53.712 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:53.712 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:14920 3.0 3:05c39535:::rbd_data.112d92027590.0000000000000009:f [read 816128~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:31:53.712 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:26.083868+0000", 2024-09-14T06:31:53.712 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 147.617295687, 2024-09-14T06:31:53.712 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.042166771999999998, 2024-09-14T06:31:53.712 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:31:53.713 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:31:53.713 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:31:53.713 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:31:53.713 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:31:53.713 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:53.713 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 14920 2024-09-14T06:31:53.713 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:53.713 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:31:53.713 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:53.713 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:31:53.713 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.083868+0000", 2024-09-14T06:31:53.713 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:53.713 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:53.713 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:53.714 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:31:53.714 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.083868+0000", 2024-09-14T06:31:53.714 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:53.714 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:53.714 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:53.714 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:31:53.714 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.083869+0000", 2024-09-14T06:31:53.714 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.99e-07 2024-09-14T06:31:53.714 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:53.714 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:53.714 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:31:53.714 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.083870+0000", 2024-09-14T06:31:53.714 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.782e-06 2024-09-14T06:31:53.714 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:53.715 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:53.715 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:31:53.715 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.083876+0000", 2024-09-14T06:31:53.715 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.3809999999999998e-06 2024-09-14T06:31:53.715 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:53.715 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:53.715 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:31:53.715 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.083882+0000", 2024-09-14T06:31:53.715 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.9479999999999996e-06 2024-09-14T06:31:53.715 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:53.715 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:53.715 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:31:53.715 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.125878+0000", 2024-09-14T06:31:53.715 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.041996264999999998 2024-09-14T06:31:53.716 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:53.716 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:53.716 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:31:53.716 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.125929+0000", 2024-09-14T06:31:53.716 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.0498999999999999e-05 2024-09-14T06:31:53.716 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:53.716 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:53.716 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:31:53.716 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.126035+0000", 2024-09-14T06:31:53.716 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000106298 2024-09-14T06:31:53.716 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:53.716 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:31:53.716 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:53.716 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:53.717 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:53.717 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:14921 3.0 3:05c39535:::rbd_data.112d92027590.0000000000000009:f [read 826368~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:31:53.717 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:26.083889+0000", 2024-09-14T06:31:53.717 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 147.61727467700001, 2024-09-14T06:31:53.717 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.042435309999999997, 2024-09-14T06:31:53.717 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:31:53.717 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:31:53.717 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:31:53.717 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:31:53.717 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:31:53.717 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:53.717 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 14921 2024-09-14T06:31:53.718 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:53.718 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:31:53.718 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:53.718 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:31:53.718 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.083889+0000", 2024-09-14T06:31:53.718 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:53.718 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:53.718 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:53.718 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:31:53.718 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.083889+0000", 2024-09-14T06:31:53.718 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:53.718 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:53.718 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:53.718 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:31:53.719 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.086033+0000", 2024-09-14T06:31:53.719 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0021444179999999999 2024-09-14T06:31:53.719 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:53.719 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:53.719 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:31:53.719 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.086038+0000", 2024-09-14T06:31:53.719 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.0969999999999999e-06 2024-09-14T06:31:53.719 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:53.719 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:53.719 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:31:53.719 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.086046+0000", 2024-09-14T06:31:53.719 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.9440000000000005e-06 2024-09-14T06:31:53.719 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:53.719 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:53.720 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:31:53.720 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.086050+0000", 2024-09-14T06:31:53.720 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.69e-06 2024-09-14T06:31:53.720 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:53.720 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:53.720 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:31:53.720 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.126080+0000", 2024-09-14T06:31:53.720 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.040030237000000003 2024-09-14T06:31:53.720 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:53.720 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:53.720 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:31:53.720 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.126163+0000", 2024-09-14T06:31:53.720 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.2071999999999996e-05 2024-09-14T06:31:53.721 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:53.721 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:53.721 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:31:53.721 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.126324+0000", 2024-09-14T06:31:53.721 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00016185200000000001 2024-09-14T06:31:53.721 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:53.721 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:31:53.721 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:53.721 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:53.721 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:53.721 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:55627 3.0 3:06490baf:::rbd_data.112dca2f6ae1.0000000000000030:17 [read 2002944~4096 [fadvise_nocache]] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:31:53.721 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:44.599966+0000", 2024-09-14T06:31:53.721 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 129.10119772900001, 2024-09-14T06:31:53.721 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.041868546999999999, 2024-09-14T06:31:53.722 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:31:53.722 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:31:53.722 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:31:53.722 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:31:53.722 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:31:53.722 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:53.722 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 55627 2024-09-14T06:31:53.722 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:53.722 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:31:53.722 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:53.722 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:31:53.722 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.599966+0000", 2024-09-14T06:31:53.722 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:53.722 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:53.723 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:53.723 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:31:53.723 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.599966+0000", 2024-09-14T06:31:53.723 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:53.723 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:53.723 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:53.723 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:31:53.723 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.599966+0000", 2024-09-14T06:31:53.723 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.0900000000000001e-07 2024-09-14T06:31:53.723 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:53.723 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:53.723 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:31:53.723 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.599967+0000", 2024-09-14T06:31:53.723 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.64e-07 2024-09-14T06:31:53.724 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:53.724 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:53.724 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:31:53.724 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.599970+0000", 2024-09-14T06:31:53.724 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.9670000000000001e-06 2024-09-14T06:31:53.724 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:53.724 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:53.724 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:31:53.724 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.599973+0000", 2024-09-14T06:31:53.724 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.632e-06 2024-09-14T06:31:53.724 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:53.724 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:53.724 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:31:53.724 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.641665+0000", 2024-09-14T06:31:53.725 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.041692547000000003 2024-09-14T06:31:53.725 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:53.725 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:53.725 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:31:53.725 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.641740+0000", 2024-09-14T06:31:53.725 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.4559000000000002e-05 2024-09-14T06:31:53.725 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:53.725 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:53.725 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:31:53.725 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.641835+0000", 2024-09-14T06:31:53.725 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.4668999999999998e-05 2024-09-14T06:31:53.725 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:53.725 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:31:53.725 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:53.726 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:53.726 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:53.726 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:55628 3.0 3:06490baf:::rbd_data.112dca2f6ae1.0000000000000030:17 [read 2027520~4096 [fadvise_nocache]] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:31:53.726 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:44.599976+0000", 2024-09-14T06:31:53.726 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 129.101187942, 2024-09-14T06:31:53.726 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.041991621, 2024-09-14T06:31:53.726 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:31:53.726 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:31:53.726 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:31:53.726 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:31:53.726 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:31:53.726 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:53.726 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 55628 2024-09-14T06:31:53.727 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:53.727 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:31:53.727 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:53.727 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:31:53.727 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.599976+0000", 2024-09-14T06:31:53.727 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:53.727 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:53.727 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:53.727 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:31:53.727 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.599976+0000", 2024-09-14T06:31:53.727 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:53.727 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:53.727 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:53.727 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:31:53.727 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.599976+0000", 2024-09-14T06:31:53.728 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.1299999999999999e-07 2024-09-14T06:31:53.728 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:53.728 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:53.728 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:31:53.728 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.599977+0000", 2024-09-14T06:31:53.728 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.7999999999999993e-07 2024-09-14T06:31:53.728 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:53.728 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:53.728 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:31:53.728 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.599980+0000", 2024-09-14T06:31:53.728 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.01e-06 2024-09-14T06:31:53.728 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:53.728 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:53.729 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:31:53.729 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.599983+0000", 2024-09-14T06:31:53.729 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.5409999999999999e-06 2024-09-14T06:31:53.729 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:53.729 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:53.729 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:31:53.729 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.641869+0000", 2024-09-14T06:31:53.729 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.041886235000000001 2024-09-14T06:31:53.729 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:53.729 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:53.729 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:31:53.729 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.641913+0000", 2024-09-14T06:31:53.729 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.4357000000000002e-05 2024-09-14T06:31:53.729 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:53.730 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:53.730 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:31:53.730 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.641967+0000", 2024-09-14T06:31:53.730 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.4284999999999999e-05 2024-09-14T06:31:53.730 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:53.730 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:31:53.730 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:53.730 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:53.730 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:53.730 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:55629 3.0 3:06490baf:::rbd_data.112dca2f6ae1.0000000000000030:17 [read 2052096~4096 [fadvise_nocache]] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:31:53.730 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:44.599985+0000", 2024-09-14T06:31:53.730 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 129.10117823600001, 2024-09-14T06:31:53.730 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.042183375000000002, 2024-09-14T06:31:53.731 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:31:53.731 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:31:53.731 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:31:53.731 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:31:53.731 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:31:53.731 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:53.731 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 55629 2024-09-14T06:31:53.731 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:53.731 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:31:53.731 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:53.731 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:31:53.731 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.599985+0000", 2024-09-14T06:31:53.731 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:53.731 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:53.732 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:53.732 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:31:53.732 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.599985+0000", 2024-09-14T06:31:53.732 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:53.732 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:53.732 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:53.732 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:31:53.732 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.599986+0000", 2024-09-14T06:31:53.732 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.05e-07 2024-09-14T06:31:53.732 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:53.732 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:53.732 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:31:53.732 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.599987+0000", 2024-09-14T06:31:53.732 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.8400000000000002e-07 2024-09-14T06:31:53.733 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:53.733 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:53.733 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:31:53.733 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.599990+0000", 2024-09-14T06:31:53.733 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.5640000000000001e-06 2024-09-14T06:31:53.733 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:53.733 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:53.733 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:31:53.733 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.599993+0000", 2024-09-14T06:31:53.733 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.615e-06 2024-09-14T06:31:53.733 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:53.733 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:53.733 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:31:53.734 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.642003+0000", 2024-09-14T06:31:53.734 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.042010345999999997 2024-09-14T06:31:53.734 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:53.734 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:53.734 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:31:53.734 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.642080+0000", 2024-09-14T06:31:53.734 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.6538999999999993e-05 2024-09-14T06:31:53.734 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:53.734 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:53.734 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:31:53.734 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.642169+0000", 2024-09-14T06:31:53.734 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.9122000000000005e-05 2024-09-14T06:31:53.734 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:53.734 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:31:53.735 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:53.735 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:53.735 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:53.735 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:55630 3.0 3:06490baf:::rbd_data.112dca2f6ae1.0000000000000030:17 [read 2076672~4096 [fadvise_nocache]] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:31:53.735 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:44.599996+0000", 2024-09-14T06:31:53.735 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 129.101167758, 2024-09-14T06:31:53.735 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.042299615999999998, 2024-09-14T06:31:53.735 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:31:53.735 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:31:53.735 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:31:53.735 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:31:53.735 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:31:53.735 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:53.735 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 55630 2024-09-14T06:31:53.736 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:53.736 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:31:53.736 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:53.736 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:31:53.736 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.599996+0000", 2024-09-14T06:31:53.736 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:53.736 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:53.736 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:53.736 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:31:53.736 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.599996+0000", 2024-09-14T06:31:53.736 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:53.736 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:53.736 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:53.736 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:31:53.737 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.599996+0000", 2024-09-14T06:31:53.737 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.1500000000000001e-07 2024-09-14T06:31:53.737 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:53.737 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:53.737 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:31:53.737 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.599997+0000", 2024-09-14T06:31:53.737 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.33e-07 2024-09-14T06:31:53.737 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:53.737 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:53.737 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:31:53.738 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600000+0000", 2024-09-14T06:31:53.738 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.0240000000000002e-06 2024-09-14T06:31:53.738 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:53.738 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:53.738 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:31:53.738 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600003+0000", 2024-09-14T06:31:53.738 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.3640000000000002e-06 2024-09-14T06:31:53.738 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:53.738 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:53.738 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:31:53.738 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.642200+0000", 2024-09-14T06:31:53.738 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.042197571000000003 2024-09-14T06:31:53.738 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:53.738 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:53.739 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:31:53.739 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.642243+0000", 2024-09-14T06:31:53.739 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.2765000000000001e-05 2024-09-14T06:31:53.739 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:53.739 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:53.739 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:31:53.739 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.642296+0000", 2024-09-14T06:31:53.739 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.2744000000000002e-05 2024-09-14T06:31:53.739 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:53.739 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:31:53.739 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:53.739 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:53.739 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:53.739 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:55631 3.0 3:06490baf:::rbd_data.112dca2f6ae1.0000000000000030:17 [read 1048576~4096 [fadvise_nocache]] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:31:53.740 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:44.600006+0000", 2024-09-14T06:31:53.740 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 129.10115816999999, 2024-09-14T06:31:53.740 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.042487905999999999, 2024-09-14T06:31:53.740 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:31:53.740 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:31:53.740 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:31:53.740 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:31:53.740 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:31:53.740 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:53.740 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 55631 2024-09-14T06:31:53.740 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:53.740 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:31:53.741 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:53.741 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:31:53.741 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600006+0000", 2024-09-14T06:31:53.741 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:53.741 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:53.741 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:53.741 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:31:53.741 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600006+0000", 2024-09-14T06:31:53.741 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:53.741 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:53.741 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:53.741 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:31:53.741 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600006+0000", 2024-09-14T06:31:53.742 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.0800000000000001e-07 2024-09-14T06:31:53.742 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:53.742 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:53.742 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:31:53.742 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600007+0000", 2024-09-14T06:31:53.742 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.7199999999999997e-07 2024-09-14T06:31:53.742 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:53.742 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:53.742 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:31:53.742 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600010+0000", 2024-09-14T06:31:53.742 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.0280000000000001e-06 2024-09-14T06:31:53.742 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:53.742 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:53.742 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:31:53.742 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600012+0000", 2024-09-14T06:31:53.743 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.475e-06 2024-09-14T06:31:53.743 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:53.743 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:53.743 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:31:53.743 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.642329+0000", 2024-09-14T06:31:53.743 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.042317157000000001 2024-09-14T06:31:53.743 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:53.743 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:53.743 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:31:53.743 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.642403+0000", 2024-09-14T06:31:53.743 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.3941000000000002e-05 2024-09-14T06:31:53.743 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:53.743 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:53.743 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:31:53.744 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.642493+0000", 2024-09-14T06:31:53.744 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.0124999999999995e-05 2024-09-14T06:31:53.744 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:53.744 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:31:53.744 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:53.744 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:53.744 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:53.744 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:55632 3.0 3:06490baf:::rbd_data.112dca2f6ae1.0000000000000030:17 [read 1073152~4096 [fadvise_nocache]] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:31:53.744 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:44.600015+0000", 2024-09-14T06:31:53.744 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 129.10114841699999, 2024-09-14T06:31:53.744 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.042604922000000003, 2024-09-14T06:31:53.744 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:31:53.744 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:31:53.744 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:31:53.745 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:31:53.745 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:31:53.745 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:53.745 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 55632 2024-09-14T06:31:53.745 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:53.745 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:31:53.745 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:53.745 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:31:53.745 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600015+0000", 2024-09-14T06:31:53.745 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:53.745 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:53.745 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:53.745 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:31:53.746 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600015+0000", 2024-09-14T06:31:53.746 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:53.746 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:53.746 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:53.746 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:31:53.746 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600016+0000", 2024-09-14T06:31:53.746 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.1e-07 2024-09-14T06:31:53.746 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:53.746 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:53.746 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:31:53.746 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600017+0000", 2024-09-14T06:31:53.746 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.9900000000000009e-07 2024-09-14T06:31:53.746 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:53.746 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:53.747 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:31:53.747 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600020+0000", 2024-09-14T06:31:53.747 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.4170000000000001e-06 2024-09-14T06:31:53.747 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:53.747 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:53.747 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:31:53.747 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600023+0000", 2024-09-14T06:31:53.747 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.588e-06 2024-09-14T06:31:53.747 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:53.747 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:53.747 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:31:53.747 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.642526+0000", 2024-09-14T06:31:53.747 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.042503236999999999 2024-09-14T06:31:53.748 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:53.748 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:53.748 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:31:53.748 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.642568+0000", 2024-09-14T06:31:53.748 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.1925000000000003e-05 2024-09-14T06:31:53.748 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:53.748 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:53.748 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:31:53.748 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.642620+0000", 2024-09-14T06:31:53.748 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.2546000000000001e-05 2024-09-14T06:31:53.748 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:53.748 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:31:53.748 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:53.748 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:53.749 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:53.749 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:55633 3.0 3:06490baf:::rbd_data.112dca2f6ae1.0000000000000030:17 [read 1097728~4096 [fadvise_nocache]] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:31:53.749 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:44.600026+0000", 2024-09-14T06:31:53.749 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 129.10113814799999, 2024-09-14T06:31:53.749 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.042797474000000002, 2024-09-14T06:31:53.749 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:31:53.749 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:31:53.749 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:31:53.749 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:31:53.749 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:31:53.749 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:53.749 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 55633 2024-09-14T06:31:53.749 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:53.749 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:31:53.750 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:53.750 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:31:53.750 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600026+0000", 2024-09-14T06:31:53.750 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:53.750 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:53.750 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:53.750 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:31:53.750 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600026+0000", 2024-09-14T06:31:53.750 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:53.750 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:53.750 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:53.750 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:31:53.750 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600026+0000", 2024-09-14T06:31:53.750 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.3099999999999999e-07 2024-09-14T06:31:53.751 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:53.751 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:53.751 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:31:53.751 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600029+0000", 2024-09-14T06:31:53.751 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.304e-06 2024-09-14T06:31:53.751 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:53.751 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:53.751 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:31:53.751 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600034+0000", 2024-09-14T06:31:53.751 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.1429999999999997e-06 2024-09-14T06:31:53.751 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:53.751 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:53.751 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:31:53.751 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600038+0000", 2024-09-14T06:31:53.752 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.692e-06 2024-09-14T06:31:53.752 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:53.752 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:53.752 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:31:53.752 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.642654+0000", 2024-09-14T06:31:53.752 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.042615564000000002 2024-09-14T06:31:53.752 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:53.752 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:53.752 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:31:53.752 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.642728+0000", 2024-09-14T06:31:53.752 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.4018999999999997e-05 2024-09-14T06:31:53.752 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:53.752 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:53.752 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:31:53.752 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.642823+0000", 2024-09-14T06:31:53.753 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.5520999999999996e-05 2024-09-14T06:31:53.753 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:53.753 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:31:53.753 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:53.753 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:53.753 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:53.753 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:55634 3.0 3:06490baf:::rbd_data.112dca2f6ae1.0000000000000030:17 [read 1122304~4096 [fadvise_nocache]] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:31:53.753 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:44.600042+0000", 2024-09-14T06:31:53.753 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 129.10112167099999, 2024-09-14T06:31:53.753 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.042903729000000002, 2024-09-14T06:31:53.753 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:31:53.753 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:31:53.753 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:31:53.753 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:31:53.754 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:31:53.754 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:53.754 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 55634 2024-09-14T06:31:53.754 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:53.754 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:31:53.754 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:53.754 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:31:53.754 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600042+0000", 2024-09-14T06:31:53.754 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:53.754 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:53.754 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:53.754 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:31:53.754 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600042+0000", 2024-09-14T06:31:53.754 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:53.755 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:53.755 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:53.755 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:31:53.755 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600043+0000", 2024-09-14T06:31:53.755 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.7000000000000005e-07 2024-09-14T06:31:53.755 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:53.755 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:53.755 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:31:53.755 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600044+0000", 2024-09-14T06:31:53.755 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.742e-06 2024-09-14T06:31:53.755 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:53.755 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:53.755 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:31:53.755 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600050+0000", 2024-09-14T06:31:53.755 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.0800000000000002e-06 2024-09-14T06:31:53.756 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:53.756 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:53.756 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:31:53.756 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600055+0000", 2024-09-14T06:31:53.756 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.3889999999999998e-06 2024-09-14T06:31:53.756 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:53.756 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:53.756 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:31:53.756 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.642845+0000", 2024-09-14T06:31:53.756 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.042790040000000001 2024-09-14T06:31:53.756 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:53.756 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:53.756 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:31:53.756 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.642888+0000", 2024-09-14T06:31:53.757 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.2790000000000002e-05 2024-09-14T06:31:53.757 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:53.757 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:53.757 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:31:53.757 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.642946+0000", 2024-09-14T06:31:53.757 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.8118000000000002e-05 2024-09-14T06:31:53.757 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:53.757 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:31:53.757 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:53.757 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:53.757 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:53.757 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:55635 3.0 3:06490baf:::rbd_data.112dca2f6ae1.0000000000000030:17 [read 1146880~4096 [fadvise_nocache]] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:31:53.757 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:44.600061+0000", 2024-09-14T06:31:53.757 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 129.10110309999999, 2024-09-14T06:31:53.758 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.043092695, 2024-09-14T06:31:53.758 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:31:53.758 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:31:53.758 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:31:53.758 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:31:53.758 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:31:53.758 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:53.758 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 55635 2024-09-14T06:31:53.758 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:53.758 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:31:53.758 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:53.758 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:31:53.758 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600061+0000", 2024-09-14T06:31:53.758 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:53.759 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:53.759 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:53.759 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:31:53.759 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600061+0000", 2024-09-14T06:31:53.759 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:53.759 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:53.759 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:53.759 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:31:53.759 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600061+0000", 2024-09-14T06:31:53.759 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.3099999999999999e-07 2024-09-14T06:31:53.759 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:53.759 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:53.759 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:31:53.759 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600063+0000", 2024-09-14T06:31:53.760 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.6220000000000001e-06 2024-09-14T06:31:53.760 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:53.760 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:53.760 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:31:53.760 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600068+0000", 2024-09-14T06:31:53.760 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.4790000000000004e-06 2024-09-14T06:31:53.760 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:53.760 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:53.760 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:31:53.760 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600072+0000", 2024-09-14T06:31:53.760 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.3100000000000002e-06 2024-09-14T06:31:53.760 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:53.760 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:53.760 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:31:53.761 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.642987+0000", 2024-09-14T06:31:53.761 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.042914489 2024-09-14T06:31:53.761 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:53.761 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:53.761 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:31:53.761 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.643062+0000", 2024-09-14T06:31:53.761 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.5011000000000004e-05 2024-09-14T06:31:53.761 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:53.761 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:53.761 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:31:53.761 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.643153+0000", 2024-09-14T06:31:53.761 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.1452999999999993e-05 2024-09-14T06:31:53.761 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:53.762 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:31:53.762 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:53.762 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:53.762 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:53.762 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:55636 3.0 3:06490baf:::rbd_data.112dca2f6ae1.0000000000000030:17 [read 1171456~4096 [fadvise_nocache]] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:31:53.762 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:44.600078+0000", 2024-09-14T06:31:53.762 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 129.10108568800001, 2024-09-14T06:31:53.762 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.043207966, 2024-09-14T06:31:53.762 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:31:53.762 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:31:53.762 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:31:53.762 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:31:53.762 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:31:53.762 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:53.763 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 55636 2024-09-14T06:31:53.763 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:53.763 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:31:53.763 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:53.763 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:31:53.763 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600078+0000", 2024-09-14T06:31:53.763 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:53.763 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:53.763 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:53.763 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:31:53.763 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600078+0000", 2024-09-14T06:31:53.763 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:53.763 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:53.763 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:53.764 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:31:53.764 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600078+0000", 2024-09-14T06:31:53.764 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.34e-07 2024-09-14T06:31:53.764 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:53.764 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:53.764 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:31:53.764 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600080+0000", 2024-09-14T06:31:53.764 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.641e-06 2024-09-14T06:31:53.764 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:53.764 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:53.764 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:31:53.765 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600086+0000", 2024-09-14T06:31:53.765 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.3860000000000004e-06 2024-09-14T06:31:53.765 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:53.765 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:53.765 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:31:53.765 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600091+0000", 2024-09-14T06:31:53.765 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.5970000000000002e-06 2024-09-14T06:31:53.765 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:53.765 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:53.765 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:31:53.765 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.643184+0000", 2024-09-14T06:31:53.765 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.043092893 2024-09-14T06:31:53.765 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:53.765 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:53.766 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:31:53.766 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.643228+0000", 2024-09-14T06:31:53.766 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.4536999999999997e-05 2024-09-14T06:31:53.766 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:53.766 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:53.766 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:31:53.766 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.643286+0000", 2024-09-14T06:31:53.766 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.7578000000000003e-05 2024-09-14T06:31:53.766 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:53.766 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:31:53.766 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:53.766 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:53.766 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:53.766 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:55637 3.0 3:06490baf:::rbd_data.112dca2f6ae1.0000000000000030:17 [read 1196032~4096 [fadvise_nocache]] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:31:53.767 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:44.600097+0000", 2024-09-14T06:31:53.767 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 129.101067124, 2024-09-14T06:31:53.767 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.043399124999999997, 2024-09-14T06:31:53.767 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:31:53.767 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:31:53.767 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:31:53.767 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:31:53.767 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:31:53.767 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:53.767 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 55637 2024-09-14T06:31:53.767 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:53.767 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:31:53.767 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:53.767 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:31:53.768 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600097+0000", 2024-09-14T06:31:53.768 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:53.768 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:53.768 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:53.768 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:31:53.768 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600097+0000", 2024-09-14T06:31:53.768 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:53.768 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:53.768 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:53.768 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:31:53.768 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.604372+0000", 2024-09-14T06:31:53.768 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0042751489999999998 2024-09-14T06:31:53.768 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:53.768 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:53.768 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:31:53.769 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.604377+0000", 2024-09-14T06:31:53.769 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.7899999999999999e-06 2024-09-14T06:31:53.769 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:53.769 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:53.769 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:31:53.769 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.604389+0000", 2024-09-14T06:31:53.769 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.2197e-05 2024-09-14T06:31:53.769 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:53.769 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:53.769 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:31:53.769 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.604397+0000", 2024-09-14T06:31:53.769 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.8730000000000007e-06 2024-09-14T06:31:53.769 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:53.769 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:53.769 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:31:53.770 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.643324+0000", 2024-09-14T06:31:53.770 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.038926897000000002 2024-09-14T06:31:53.770 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:53.770 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:53.770 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:31:53.770 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.643402+0000", 2024-09-14T06:31:53.770 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.8063000000000001e-05 2024-09-14T06:31:53.770 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:53.770 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:53.770 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:31:53.770 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.643496+0000", 2024-09-14T06:31:53.770 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.4155999999999998e-05 2024-09-14T06:31:53.770 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:53.771 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:31:53.771 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:53.771 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:53.771 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:53.771 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:55657 3.0 3:06490baf:::rbd_data.112dca2f6ae1.0000000000000030:17 [read 1687552~4096 [fadvise_nocache]] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:31:53.771 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:44.604841+0000", 2024-09-14T06:31:53.771 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 129.09632239499999, 2024-09-14T06:31:53.771 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.041890087999999999, 2024-09-14T06:31:53.771 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:31:53.771 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:31:53.771 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:31:53.771 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:31:53.771 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:31:53.771 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:53.772 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 55657 2024-09-14T06:31:53.772 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:53.772 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:31:53.772 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:53.772 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:31:53.772 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.604841+0000", 2024-09-14T06:31:53.772 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:53.772 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:53.772 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:53.772 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:31:53.772 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.604841+0000", 2024-09-14T06:31:53.772 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:53.772 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:53.772 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:53.773 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:31:53.773 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.604842+0000", 2024-09-14T06:31:53.773 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.0900000000000001e-07 2024-09-14T06:31:53.773 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:53.773 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:53.773 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:31:53.773 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.604846+0000", 2024-09-14T06:31:53.773 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.5759999999999999e-06 2024-09-14T06:31:53.773 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:53.773 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:53.773 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:31:53.773 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.604850+0000", 2024-09-14T06:31:53.773 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.405e-06 2024-09-14T06:31:53.774 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:53.774 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:53.774 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:31:53.774 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.604852+0000", 2024-09-14T06:31:53.774 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.8590000000000001e-06 2024-09-14T06:31:53.774 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:53.774 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:53.774 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:31:53.774 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.646633+0000", 2024-09-14T06:31:53.774 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.041780772000000001 2024-09-14T06:31:53.774 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:53.774 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:53.774 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:31:53.774 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.646677+0000", 2024-09-14T06:31:53.775 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.4115e-05 2024-09-14T06:31:53.775 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:53.775 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:53.775 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:31:53.775 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.646731+0000", 2024-09-14T06:31:53.775 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.4151999999999999e-05 2024-09-14T06:31:53.775 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:53.775 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:31:53.775 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:53.775 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:53.775 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:53.775 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:55658 3.0 3:06490baf:::rbd_data.112dca2f6ae1.0000000000000030:17 [read 1712128~4096 [fadvise_nocache]] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:31:53.775 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:44.604856+0000", 2024-09-14T06:31:53.775 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 129.096307795, 2024-09-14T06:31:53.776 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.041986227000000001, 2024-09-14T06:31:53.776 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:31:53.776 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:31:53.776 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:31:53.776 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:31:53.776 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:31:53.776 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:53.776 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 55658 2024-09-14T06:31:53.776 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:53.776 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:31:53.776 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:53.776 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:31:53.776 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.604856+0000", 2024-09-14T06:31:53.777 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:53.777 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:53.777 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:53.777 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:31:53.777 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.604856+0000", 2024-09-14T06:31:53.777 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:53.777 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:53.777 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:53.777 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:31:53.777 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.604856+0000", 2024-09-14T06:31:53.777 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.23e-07 2024-09-14T06:31:53.777 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:53.777 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:53.777 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:31:53.777 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.604857+0000", 2024-09-14T06:31:53.778 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.9999999999999995e-07 2024-09-14T06:31:53.778 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:53.778 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:53.778 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:31:53.778 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.604861+0000", 2024-09-14T06:31:53.778 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.9890000000000003e-06 2024-09-14T06:31:53.778 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:53.778 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:53.778 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:31:53.778 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.604864+0000", 2024-09-14T06:31:53.778 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.7429999999999998e-06 2024-09-14T06:31:53.778 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:53.778 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:53.779 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:31:53.779 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.646744+0000", 2024-09-14T06:31:53.779 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.041880325000000003 2024-09-14T06:31:53.779 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:53.779 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:53.779 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:31:53.779 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.646784+0000", 2024-09-14T06:31:53.779 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.9604000000000002e-05 2024-09-14T06:31:53.779 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:53.779 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:53.779 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:31:53.779 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.646842+0000", 2024-09-14T06:31:53.779 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.8343e-05 2024-09-14T06:31:53.779 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:53.779 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:31:53.780 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:53.780 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:53.780 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:53.780 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:55659 3.0 3:06490baf:::rbd_data.112dca2f6ae1.0000000000000030:17 [read 1736704~4096 [fadvise_nocache]] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:31:53.780 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:44.604867+0000", 2024-09-14T06:31:53.780 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 129.096296642, 2024-09-14T06:31:53.780 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.042097082000000001, 2024-09-14T06:31:53.780 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:31:53.780 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:31:53.780 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:31:53.780 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:31:53.780 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:31:53.780 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:53.780 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 55659 2024-09-14T06:31:53.781 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:53.781 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:31:53.781 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:53.781 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:31:53.781 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.604867+0000", 2024-09-14T06:31:53.781 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:53.781 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:53.781 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:53.781 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:31:53.781 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.604867+0000", 2024-09-14T06:31:53.781 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:53.781 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:53.781 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:53.781 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:31:53.782 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.604867+0000", 2024-09-14T06:31:53.782 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.2000000000000001e-07 2024-09-14T06:31:53.782 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:53.782 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:53.782 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:31:53.782 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.604868+0000", 2024-09-14T06:31:53.782 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.04e-06 2024-09-14T06:31:53.782 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:53.782 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:53.782 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:31:53.782 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.604871+0000", 2024-09-14T06:31:53.782 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.9689999999999999e-06 2024-09-14T06:31:53.782 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:53.782 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:53.783 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:31:53.783 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.604874+0000", 2024-09-14T06:31:53.783 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.5069999999999999e-06 2024-09-14T06:31:53.783 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:53.783 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:53.783 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:31:53.783 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.646874+0000", 2024-09-14T06:31:53.783 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.042000513000000003 2024-09-14T06:31:53.783 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:53.783 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:53.783 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:31:53.783 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.646914+0000", 2024-09-14T06:31:53.783 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.9443999999999997e-05 2024-09-14T06:31:53.783 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:53.783 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:53.784 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:31:53.784 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.646964+0000", 2024-09-14T06:31:53.784 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.0389000000000002e-05 2024-09-14T06:31:53.784 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:53.784 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:31:53.784 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:53.784 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:53.784 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:53.784 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:55660 3.0 3:06490baf:::rbd_data.112dca2f6ae1.0000000000000030:17 [read 1761280~4096 [fadvise_nocache]] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:31:53.784 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:44.604877+0000", 2024-09-14T06:31:53.784 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 129.09628689300001, 2024-09-14T06:31:53.784 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.042196349000000001, 2024-09-14T06:31:53.784 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:31:53.784 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:31:53.785 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:31:53.785 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:31:53.785 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:31:53.785 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:53.785 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 55660 2024-09-14T06:31:53.785 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:53.785 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:31:53.785 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:53.785 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:31:53.785 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.604877+0000", 2024-09-14T06:31:53.785 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:53.785 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:53.785 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:53.785 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:31:53.786 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.604877+0000", 2024-09-14T06:31:53.786 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:53.786 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:53.786 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:53.786 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:31:53.786 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.605810+0000", 2024-09-14T06:31:53.786 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00093336899999999997 2024-09-14T06:31:53.786 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:53.786 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:53.786 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:31:53.786 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.605815+0000", 2024-09-14T06:31:53.786 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.6140000000000002e-06 2024-09-14T06:31:53.786 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:53.786 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:53.786 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:31:53.787 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.605825+0000", 2024-09-14T06:31:53.787 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.8889999999999997e-06 2024-09-14T06:31:53.787 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:53.787 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:53.787 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:31:53.787 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.605831+0000", 2024-09-14T06:31:53.787 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.9800000000000003e-06 2024-09-14T06:31:53.787 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:53.787 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:53.787 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:31:53.787 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.646989+0000", 2024-09-14T06:31:53.787 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.041157878000000002 2024-09-14T06:31:53.787 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:53.788 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:53.788 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:31:53.788 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.647026+0000", 2024-09-14T06:31:53.788 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.7305000000000003e-05 2024-09-14T06:31:53.788 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:53.788 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:53.788 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:31:53.788 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.647073+0000", 2024-09-14T06:31:53.788 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.7314000000000001e-05 2024-09-14T06:31:53.788 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:53.788 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:31:53.788 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:53.788 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:53.788 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:53.788 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:55666 3.0 3:06490baf:::rbd_data.112dca2f6ae1.0000000000000030:17 [read 1908736~4096 [fadvise_nocache]] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:31:53.789 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:44.605898+0000", 2024-09-14T06:31:53.789 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 129.09526614699999, 2024-09-14T06:31:53.789 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.041862887000000001, 2024-09-14T06:31:53.789 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:31:53.789 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:31:53.789 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:31:53.789 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:31:53.789 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:31:53.789 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:53.789 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 55666 2024-09-14T06:31:53.789 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:53.789 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:31:53.789 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:53.790 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:31:53.790 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.605898+0000", 2024-09-14T06:31:53.790 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:53.790 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:53.790 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:53.790 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:31:53.790 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.605898+0000", 2024-09-14T06:31:53.790 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:53.790 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:53.790 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:53.790 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:31:53.790 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.611874+0000", 2024-09-14T06:31:53.790 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0059764049999999997 2024-09-14T06:31:53.790 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:53.790 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:53.791 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:31:53.791 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.611879+0000", 2024-09-14T06:31:53.791 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.8749999999999999e-06 2024-09-14T06:31:53.791 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:53.791 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:53.791 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:31:53.791 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.611887+0000", 2024-09-14T06:31:53.791 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.4109999999999996e-06 2024-09-14T06:31:53.791 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:53.791 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:53.791 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:31:53.791 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.611893+0000", 2024-09-14T06:31:53.791 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.2959999999999998e-06 2024-09-14T06:31:53.791 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:53.791 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:53.792 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:31:53.792 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.647674+0000", 2024-09-14T06:31:53.792 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.035781281999999998 2024-09-14T06:31:53.792 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:53.792 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:53.792 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:31:53.792 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.647711+0000", 2024-09-14T06:31:53.792 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.7434000000000001e-05 2024-09-14T06:31:53.792 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:53.792 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:53.792 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:31:53.792 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.647760+0000", 2024-09-14T06:31:53.792 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.9184000000000002e-05 2024-09-14T06:31:53.792 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:53.793 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:31:53.793 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:53.793 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:53.793 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:31:53.793 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-14T06:31:53.794 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-14T06:31:53.795 INFO:tasks.ceph.osd.3.smithi094.stderr:2024-09-14T06:31:53.793+0000 7feeeafe8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33655) UID: 0 2024-09-14T06:31:53.895 INFO:tasks.ceph.osd.3.smithi094.stderr:2024-09-14T06:31:53.894+0000 7feeeafe8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33655) UID: 0 2024-09-14T06:31:53.955 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-14T06:31:53.955 INFO:teuthology.orchestra.run.smithi094.stdout: "ops": [], 2024-09-14T06:31:53.955 INFO:teuthology.orchestra.run.smithi094.stdout: "num_ops": 0 2024-09-14T06:31:53.955 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-14T06:31:53.964 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-14T06:31:53.996 INFO:tasks.ceph.osd.7.smithi138.stderr:2024-09-14T06:31:53.995+0000 7f3c6a21d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33623) UID: 0 2024-09-14T06:31:54.098 INFO:tasks.ceph.osd.0.smithi094.stderr:2024-09-14T06:31:54.097+0000 7f07c9333640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33636) UID: 0 2024-09-14T06:31:54.120 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-14T06:31:54.120 INFO:teuthology.orchestra.run.smithi094.stdout: "ops": [], 2024-09-14T06:31:54.120 INFO:teuthology.orchestra.run.smithi094.stdout: "complaint_time": 30, 2024-09-14T06:31:54.120 INFO:teuthology.orchestra.run.smithi094.stdout: "num_blocked_ops": 0 2024-09-14T06:31:54.120 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-14T06:31:54.129 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-14T06:31:54.201 INFO:tasks.ceph.osd.4.smithi138.stderr:2024-09-14T06:31:54.200+0000 7f6c653c8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33615) UID: 0 2024-09-14T06:31:54.286 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-14T06:31:54.286 INFO:teuthology.orchestra.run.smithi094.stdout: "size": 20, 2024-09-14T06:31:54.287 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 600, 2024-09-14T06:31:54.287 INFO:teuthology.orchestra.run.smithi094.stdout: "ops": [ 2024-09-14T06:31:54.287 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:54.287 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4397.0:45722 3.9 e64/23 3:91fed098:::rbd_data.112d5a737e72.0000000000000001:head v 64'61, mlcod=62'59)", 2024-09-14T06:31:54.287 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:43.864269+0000", 2024-09-14T06:31:54.287 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 130.42046746, 2024-09-14T06:31:54.287 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.003055224, 2024-09-14T06:31:54.287 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:31:54.287 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:31:54.287 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:31:54.287 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:31:54.287 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:54.287 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:31:54.288 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.864269+0000", 2024-09-14T06:31:54.288 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:54.288 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:54.288 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:54.288 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:31:54.288 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.864269+0000", 2024-09-14T06:31:54.288 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:54.288 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:54.288 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:54.288 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:31:54.288 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.864269+0000", 2024-09-14T06:31:54.288 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.41e-07 2024-09-14T06:31:54.288 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:54.289 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:54.289 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:31:54.289 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.865490+0000", 2024-09-14T06:31:54.289 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.001220552 2024-09-14T06:31:54.289 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:54.289 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:54.289 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:31:54.289 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.865506+0000", 2024-09-14T06:31:54.289 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.6263999999999999e-05 2024-09-14T06:31:54.289 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:54.289 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:54.289 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:31:54.289 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.865516+0000", 2024-09-14T06:31:54.290 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.0091999999999999e-05 2024-09-14T06:31:54.290 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:54.290 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:54.290 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:31:54.290 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.865583+0000", 2024-09-14T06:31:54.290 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.6676999999999999e-05 2024-09-14T06:31:54.290 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:54.290 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:54.290 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:31:54.290 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.865610+0000", 2024-09-14T06:31:54.290 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.6529999999999998e-05 2024-09-14T06:31:54.290 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:54.290 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:54.291 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-14T06:31:54.291 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.867281+0000", 2024-09-14T06:31:54.291 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.001671203 2024-09-14T06:31:54.291 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:54.291 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:54.291 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:31:54.291 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.867324+0000", 2024-09-14T06:31:54.291 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.3665000000000003e-05 2024-09-14T06:31:54.291 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:54.291 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:31:54.291 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:54.291 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:54.291 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:54.291 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4397.0:74241 3.0 e73/23 3:06e93fb9:::rbd_data.112d80ca42af.0000000000000000:head v 73'102, mlcod=71'100)", 2024-09-14T06:31:54.292 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:53.876730+0000", 2024-09-14T06:31:54.292 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 120.40800636, 2024-09-14T06:31:54.292 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0079470329999999992, 2024-09-14T06:31:54.292 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:31:54.292 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:31:54.292 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:31:54.292 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:31:54.292 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:54.292 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:31:54.292 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:53.876730+0000", 2024-09-14T06:31:54.292 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:54.292 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:54.292 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:54.293 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:31:54.293 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:53.876730+0000", 2024-09-14T06:31:54.293 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:54.293 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:54.293 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:54.293 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:31:54.293 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:53.876731+0000", 2024-09-14T06:31:54.293 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.0500000000000004e-07 2024-09-14T06:31:54.293 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:54.293 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:54.293 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:31:54.293 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:53.879616+0000", 2024-09-14T06:31:54.293 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0028856680000000001 2024-09-14T06:31:54.293 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:54.294 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:54.294 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:31:54.294 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:53.879643+0000", 2024-09-14T06:31:54.294 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.6120999999999999e-05 2024-09-14T06:31:54.294 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:54.294 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:54.294 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:31:54.294 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:53.879654+0000", 2024-09-14T06:31:54.294 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.1586999999999999e-05 2024-09-14T06:31:54.294 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:54.294 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:54.294 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:31:54.294 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:53.879756+0000", 2024-09-14T06:31:54.295 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00010223899999999999 2024-09-14T06:31:54.295 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:54.295 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:54.295 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:31:54.295 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:53.879783+0000", 2024-09-14T06:31:54.295 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.6913000000000001e-05 2024-09-14T06:31:54.295 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:54.295 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:54.295 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-14T06:31:54.295 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:53.884594+0000", 2024-09-14T06:31:54.295 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0048110729999999999 2024-09-14T06:31:54.295 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:54.295 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:54.295 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:31:54.296 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:53.884677+0000", 2024-09-14T06:31:54.296 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.2927000000000001e-05 2024-09-14T06:31:54.296 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:54.296 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:31:54.296 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:54.296 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:54.296 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:54.296 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4397.0:74243 3.9 e73/23 3:94977f81:::rbd_data.112d80ca42af.0000000000000002:head v 73'94, mlcod=71'92)", 2024-09-14T06:31:54.296 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:53.885156+0000", 2024-09-14T06:31:54.296 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 120.399580442, 2024-09-14T06:31:54.297 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0096009860000000006, 2024-09-14T06:31:54.297 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:31:54.297 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:31:54.297 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:31:54.297 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:31:54.297 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:54.297 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:31:54.297 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:53.885156+0000", 2024-09-14T06:31:54.297 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:54.297 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:54.297 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:54.297 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:31:54.297 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:53.885156+0000", 2024-09-14T06:31:54.298 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:54.298 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:54.298 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:54.298 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:31:54.298 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:53.885156+0000", 2024-09-14T06:31:54.298 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.23e-07 2024-09-14T06:31:54.298 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:54.298 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:54.298 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:31:54.298 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:53.889694+0000", 2024-09-14T06:31:54.298 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0045378709999999997 2024-09-14T06:31:54.298 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:54.298 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:54.298 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:31:54.299 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:53.889720+0000", 2024-09-14T06:31:54.299 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.5259e-05 2024-09-14T06:31:54.299 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:54.299 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:54.299 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:31:54.299 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:53.889731+0000", 2024-09-14T06:31:54.299 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.1739000000000001e-05 2024-09-14T06:31:54.299 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:54.299 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:54.299 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:31:54.299 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:53.889822+0000", 2024-09-14T06:31:54.299 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.1200999999999996e-05 2024-09-14T06:31:54.299 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:54.299 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:54.300 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:31:54.300 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:53.889850+0000", 2024-09-14T06:31:54.300 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.7084999999999999e-05 2024-09-14T06:31:54.300 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:54.300 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:54.300 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-14T06:31:54.300 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:53.894703+0000", 2024-09-14T06:31:54.300 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0048532690000000003 2024-09-14T06:31:54.300 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:54.300 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:54.300 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:31:54.300 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:53.894757+0000", 2024-09-14T06:31:54.300 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.4339000000000003e-05 2024-09-14T06:31:54.301 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:54.301 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:31:54.301 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:54.301 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:54.301 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:54.301 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4397.0:74249 3.0 e73/23 3:009b0225:::rbd_data.112d80ca42af.0000000000000009:head v 73'103, mlcod=71'101)", 2024-09-14T06:31:54.301 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:53.893219+0000", 2024-09-14T06:31:54.301 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 120.391517964, 2024-09-14T06:31:54.301 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0065381980000000003, 2024-09-14T06:31:54.301 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:31:54.302 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:31:54.302 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:31:54.302 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:31:54.302 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:54.302 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:31:54.302 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:53.893219+0000", 2024-09-14T06:31:54.302 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:54.302 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:54.302 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:54.302 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:31:54.302 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:53.893219+0000", 2024-09-14T06:31:54.303 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:54.303 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:54.303 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:54.303 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:31:54.303 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:53.893221+0000", 2024-09-14T06:31:54.303 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.4329999999999998e-06 2024-09-14T06:31:54.303 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:54.303 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:54.303 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:31:54.303 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:53.895968+0000", 2024-09-14T06:31:54.303 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.002746746 2024-09-14T06:31:54.303 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:54.303 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:54.303 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:31:54.304 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:53.895988+0000", 2024-09-14T06:31:54.304 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.0227000000000001e-05 2024-09-14T06:31:54.304 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:54.304 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:54.304 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:31:54.304 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:53.896000+0000", 2024-09-14T06:31:54.304 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.2359e-05 2024-09-14T06:31:54.304 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:54.304 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:54.304 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:31:54.304 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:53.896084+0000", 2024-09-14T06:31:54.304 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.3659999999999995e-05 2024-09-14T06:31:54.304 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:54.305 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:54.305 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:31:54.305 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:53.896109+0000", 2024-09-14T06:31:54.305 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.5259999999999999e-05 2024-09-14T06:31:54.305 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:54.305 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:54.305 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-14T06:31:54.305 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:53.899704+0000", 2024-09-14T06:31:54.305 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0035942209999999999 2024-09-14T06:31:54.305 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:54.305 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:54.305 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:31:54.306 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:53.899757+0000", 2024-09-14T06:31:54.306 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.3291999999999997e-05 2024-09-14T06:31:54.306 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:54.306 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:31:54.306 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:54.306 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:54.306 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:54.306 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4397.0:78263 3.0 e77/23 3:00899b22:::rbd_data.112ddfbec471.0000000000000016:head v 77'104, mlcod=73'102)", 2024-09-14T06:31:54.306 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:57.634578+0000", 2024-09-14T06:31:54.306 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 116.65015891199999, 2024-09-14T06:31:54.306 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.002372454, 2024-09-14T06:31:54.306 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:31:54.306 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:31:54.306 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:31:54.307 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:31:54.307 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:54.307 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:31:54.307 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:57.634578+0000", 2024-09-14T06:31:54.307 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:54.307 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:54.307 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:54.307 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:31:54.307 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:57.634578+0000", 2024-09-14T06:31:54.307 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:54.308 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:54.308 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:54.308 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:31:54.308 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:57.634578+0000", 2024-09-14T06:31:54.308 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.5699999999999998e-07 2024-09-14T06:31:54.308 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:54.308 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:54.308 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:31:54.308 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:57.635347+0000", 2024-09-14T06:31:54.308 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00076848300000000003 2024-09-14T06:31:54.308 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:54.308 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:54.308 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:31:54.309 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:57.635361+0000", 2024-09-14T06:31:54.309 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.4238e-05 2024-09-14T06:31:54.309 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:54.309 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:54.309 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:31:54.309 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:57.635371+0000", 2024-09-14T06:31:54.309 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.0087e-05 2024-09-14T06:31:54.309 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:54.309 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:54.309 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:31:54.309 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:57.635439+0000", 2024-09-14T06:31:54.309 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.7953000000000005e-05 2024-09-14T06:31:54.309 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:54.310 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:54.310 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:31:54.310 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:57.635464+0000", 2024-09-14T06:31:54.310 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.5062000000000002e-05 2024-09-14T06:31:54.310 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:54.310 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:54.310 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-14T06:31:54.310 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:57.636914+0000", 2024-09-14T06:31:54.310 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.001449625 2024-09-14T06:31:54.310 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:54.310 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:54.310 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:31:54.310 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:57.636950+0000", 2024-09-14T06:31:54.310 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.6548999999999999e-05 2024-09-14T06:31:54.311 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:54.311 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:31:54.311 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:54.311 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:54.311 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:54.311 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4397.0:78604 3.9 e77/23 3:94c62a0e:::rbd_data.112ddfbec471.000000000000002c:head v 77'96, mlcod=73'94)", 2024-09-14T06:31:54.311 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:57.677568+0000", 2024-09-14T06:31:54.311 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 116.607168524, 2024-09-14T06:31:54.311 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.002564846, 2024-09-14T06:31:54.311 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:31:54.311 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:31:54.311 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:31:54.311 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:31:54.311 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:54.312 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:31:54.312 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:57.677568+0000", 2024-09-14T06:31:54.312 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:54.312 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:54.312 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:54.312 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:31:54.312 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:57.677568+0000", 2024-09-14T06:31:54.312 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:54.312 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:54.312 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:54.312 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:31:54.312 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:57.677568+0000", 2024-09-14T06:31:54.312 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.8900000000000001e-07 2024-09-14T06:31:54.312 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:54.312 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:54.313 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:31:54.313 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:57.678488+0000", 2024-09-14T06:31:54.313 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00091963899999999998 2024-09-14T06:31:54.313 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:54.313 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:54.313 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:31:54.313 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:57.678500+0000", 2024-09-14T06:31:54.313 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.1902000000000001e-05 2024-09-14T06:31:54.313 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:54.313 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:54.313 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:31:54.313 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:57.678507+0000", 2024-09-14T06:31:54.313 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.7739999999999999e-06 2024-09-14T06:31:54.314 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:54.314 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:54.314 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:31:54.314 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:57.678586+0000", 2024-09-14T06:31:54.314 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.9103000000000005e-05 2024-09-14T06:31:54.314 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:54.314 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:54.314 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:31:54.314 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:57.678614+0000", 2024-09-14T06:31:54.314 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.7912e-05 2024-09-14T06:31:54.314 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:54.314 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:54.314 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-14T06:31:54.314 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:57.680075+0000", 2024-09-14T06:31:54.315 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0014613370000000001 2024-09-14T06:31:54.315 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:54.315 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:54.315 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:31:54.315 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:57.680133+0000", 2024-09-14T06:31:54.315 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.7890000000000003e-05 2024-09-14T06:31:54.315 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:54.315 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:31:54.315 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:54.315 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:54.315 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:54.315 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4397.0:149885 3.9 e122/23 3:951acef3:::rbd_data.112d1bb7b269.0000000000000005:head v 122'176, mlcod=122'172)", 2024-09-14T06:31:54.315 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:30:44.739092+0000", 2024-09-14T06:31:54.316 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 69.545644214000006, 2024-09-14T06:31:54.316 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0028236659999999998, 2024-09-14T06:31:54.316 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:31:54.316 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:31:54.316 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:31:54.316 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:31:54.316 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:54.316 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:31:54.316 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.739092+0000", 2024-09-14T06:31:54.316 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:54.316 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:54.316 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:54.316 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:31:54.317 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.739092+0000", 2024-09-14T06:31:54.317 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:54.317 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:54.317 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:54.317 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:31:54.317 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.739093+0000", 2024-09-14T06:31:54.317 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.0109999999999999e-06 2024-09-14T06:31:54.317 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:54.317 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:54.317 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:31:54.317 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.740189+0000", 2024-09-14T06:31:54.317 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00109527 2024-09-14T06:31:54.317 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:54.318 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:54.318 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:31:54.318 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.740199+0000", 2024-09-14T06:31:54.318 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.0458999999999999e-05 2024-09-14T06:31:54.318 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:54.318 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:54.318 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:31:54.318 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.740207+0000", 2024-09-14T06:31:54.318 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.4499999999999998e-06 2024-09-14T06:31:54.318 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:54.318 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:54.318 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:31:54.318 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.740295+0000", 2024-09-14T06:31:54.318 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.8740999999999996e-05 2024-09-14T06:31:54.319 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:54.319 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:54.319 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:31:54.319 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.740317+0000", 2024-09-14T06:31:54.319 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.1523e-05 2024-09-14T06:31:54.319 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:54.319 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:54.319 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-14T06:31:54.319 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.741865+0000", 2024-09-14T06:31:54.319 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.001548315 2024-09-14T06:31:54.319 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:54.319 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:54.319 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:31:54.320 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.741916+0000", 2024-09-14T06:31:54.320 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.0896999999999997e-05 2024-09-14T06:31:54.320 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:54.320 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:31:54.320 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:54.320 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:54.320 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:54.320 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4397.0:149933 3.4 e122/23 3:234f5229:::rbd_data.112d1bb7b269.0000000000000009:head v 122'221, mlcod=120'218)", 2024-09-14T06:31:54.320 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:30:44.743849+0000", 2024-09-14T06:31:54.320 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 69.540887796999996, 2024-09-14T06:31:54.320 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0031479519999999999, 2024-09-14T06:31:54.320 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:31:54.320 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:31:54.321 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:31:54.321 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:31:54.321 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:54.321 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:31:54.321 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.743849+0000", 2024-09-14T06:31:54.321 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:54.321 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:54.321 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:54.322 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:31:54.322 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.743849+0000", 2024-09-14T06:31:54.322 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:54.322 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:54.322 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:54.322 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:31:54.322 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.743850+0000", 2024-09-14T06:31:54.322 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.279e-06 2024-09-14T06:31:54.322 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:54.322 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:54.322 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:31:54.322 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.745151+0000", 2024-09-14T06:31:54.323 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.001300495 2024-09-14T06:31:54.323 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:54.323 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:54.323 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:31:54.323 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.745167+0000", 2024-09-14T06:31:54.323 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.6478e-05 2024-09-14T06:31:54.323 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:54.323 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:54.323 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:31:54.323 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.745175+0000", 2024-09-14T06:31:54.323 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.2679999999999994e-06 2024-09-14T06:31:54.323 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:54.323 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:54.323 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:31:54.324 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.745226+0000", 2024-09-14T06:31:54.324 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.0192999999999999e-05 2024-09-14T06:31:54.324 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:54.324 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:54.324 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:31:54.324 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.745247+0000", 2024-09-14T06:31:54.324 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.1767e-05 2024-09-14T06:31:54.324 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:54.324 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:54.324 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-14T06:31:54.324 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.746952+0000", 2024-09-14T06:31:54.324 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0017042769999999999 2024-09-14T06:31:54.324 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:54.324 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:54.325 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:31:54.325 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.746997+0000", 2024-09-14T06:31:54.325 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.5194999999999997e-05 2024-09-14T06:31:54.325 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:54.325 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:31:54.325 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:54.325 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:54.325 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:54.325 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4397.0:150494 3.9 e122/23 3:955152a4:::rbd_object_map.112d1bb7b269.0000000000000038:head v 122'194, mlcod=122'190)", 2024-09-14T06:31:54.325 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:30:44.781585+0000", 2024-09-14T06:31:54.325 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 69.503151904999996, 2024-09-14T06:31:54.325 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0026957499999999998, 2024-09-14T06:31:54.325 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:31:54.326 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:31:54.326 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:31:54.326 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:31:54.326 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:54.326 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:31:54.326 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.781585+0000", 2024-09-14T06:31:54.326 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:54.326 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:54.326 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:54.326 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:31:54.326 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.781585+0000", 2024-09-14T06:31:54.326 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:54.326 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:54.326 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:54.327 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:31:54.327 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.781585+0000", 2024-09-14T06:31:54.327 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.5000000000000002e-07 2024-09-14T06:31:54.327 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:54.327 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:54.327 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:31:54.327 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.781594+0000", 2024-09-14T06:31:54.327 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.4640000000000006e-06 2024-09-14T06:31:54.327 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:54.327 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:54.327 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:31:54.327 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.781604+0000", 2024-09-14T06:31:54.327 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.0071999999999999e-05 2024-09-14T06:31:54.327 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:54.328 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:54.328 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:31:54.328 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.781610+0000", 2024-09-14T06:31:54.328 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.3609999999999997e-06 2024-09-14T06:31:54.328 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:54.328 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:54.328 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:31:54.328 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.781676+0000", 2024-09-14T06:31:54.328 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.5963000000000005e-05 2024-09-14T06:31:54.329 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:54.329 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:54.329 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:31:54.329 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.781696+0000", 2024-09-14T06:31:54.329 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.9278000000000001e-05 2024-09-14T06:31:54.329 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:54.329 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:54.329 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-14T06:31:54.329 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.784231+0000", 2024-09-14T06:31:54.329 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.002535093 2024-09-14T06:31:54.329 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:54.329 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:54.329 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:31:54.329 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.784280+0000", 2024-09-14T06:31:54.330 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.9869e-05 2024-09-14T06:31:54.330 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:54.330 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:31:54.330 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:54.330 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:54.330 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:54.330 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4397.0:150898 3.9 e122/23 3:97c98f90:::rbd_data.112d1bb7b269.0000000000000025:head v 122'206, mlcod=122'203)", 2024-09-14T06:31:54.330 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:30:44.807232+0000", 2024-09-14T06:31:54.330 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 69.477504988000007, 2024-09-14T06:31:54.330 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0022297889999999998, 2024-09-14T06:31:54.330 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:31:54.330 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:31:54.330 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:31:54.331 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:31:54.331 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:54.331 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:31:54.331 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.807232+0000", 2024-09-14T06:31:54.331 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:54.331 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:54.331 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:54.331 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:31:54.331 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.807232+0000", 2024-09-14T06:31:54.331 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:54.331 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:54.331 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:54.331 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:31:54.331 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.807233+0000", 2024-09-14T06:31:54.332 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.5710000000000001e-06 2024-09-14T06:31:54.332 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:54.332 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:54.332 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:31:54.332 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.807944+0000", 2024-09-14T06:31:54.332 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00071056299999999999 2024-09-14T06:31:54.332 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:54.332 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:54.332 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:31:54.332 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.807961+0000", 2024-09-14T06:31:54.332 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.7689e-05 2024-09-14T06:31:54.332 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:54.333 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:54.333 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:31:54.333 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.807971+0000", 2024-09-14T06:31:54.333 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.2e-06 2024-09-14T06:31:54.333 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:54.333 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:54.333 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:31:54.333 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.808050+0000", 2024-09-14T06:31:54.333 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.9308999999999996e-05 2024-09-14T06:31:54.333 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:54.333 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:54.333 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:31:54.333 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.808074+0000", 2024-09-14T06:31:54.334 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.4147000000000001e-05 2024-09-14T06:31:54.334 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:54.334 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:54.334 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-14T06:31:54.334 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.809415+0000", 2024-09-14T06:31:54.334 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.001341257 2024-09-14T06:31:54.334 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:54.334 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:54.334 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:31:54.334 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.809461+0000", 2024-09-14T06:31:54.334 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.6053000000000001e-05 2024-09-14T06:31:54.334 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:54.334 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:31:54.335 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:54.335 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:54.335 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:54.335 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4397.0:202551 3.9 e166/23 3:965c09b6:::rbd_data.112d3bd84cb0.000000000000000a:head v 166'416, mlcod=164'414)", 2024-09-14T06:31:54.335 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:31:30.657011+0000", 2024-09-14T06:31:54.335 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 23.627725954999999, 2024-09-14T06:31:54.335 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.002509262, 2024-09-14T06:31:54.335 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:31:54.335 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:31:54.335 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:31:54.335 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:31:54.335 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:54.335 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:31:54.335 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:30.657011+0000", 2024-09-14T06:31:54.336 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:54.336 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:54.336 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:54.336 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:31:54.336 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:30.657011+0000", 2024-09-14T06:31:54.336 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:54.336 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:54.336 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:54.336 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:31:54.336 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:30.657011+0000", 2024-09-14T06:31:54.336 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.08e-07 2024-09-14T06:31:54.336 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:54.336 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:54.336 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:31:54.337 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:30.658467+0000", 2024-09-14T06:31:54.337 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.001455708 2024-09-14T06:31:54.337 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:54.337 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:54.337 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:31:54.337 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:30.658484+0000", 2024-09-14T06:31:54.337 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.6844e-05 2024-09-14T06:31:54.337 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:54.337 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:54.337 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:31:54.337 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:30.658493+0000", 2024-09-14T06:31:54.337 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.2790000000000004e-06 2024-09-14T06:31:54.337 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:54.337 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:54.338 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:31:54.338 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:30.658559+0000", 2024-09-14T06:31:54.338 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.6459999999999997e-05 2024-09-14T06:31:54.338 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:54.338 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:54.338 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:31:54.338 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:30.658574+0000", 2024-09-14T06:31:54.338 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.5048000000000001e-05 2024-09-14T06:31:54.338 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:54.338 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:54.338 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-14T06:31:54.338 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:30.659473+0000", 2024-09-14T06:31:54.338 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00089842800000000005 2024-09-14T06:31:54.339 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:54.339 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:54.339 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:31:54.339 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:30.659520+0000", 2024-09-14T06:31:54.339 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.7086999999999998e-05 2024-09-14T06:31:54.339 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:54.339 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:31:54.339 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:54.339 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:54.339 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:54.339 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4397.0:202665 3.0 e166/23 3:00eeadfb:::rbd_data.112d3bd84cb0.000000000000000e:head v 166'920, mlcod=163'918)", 2024-09-14T06:31:54.339 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:31:30.662434+0000", 2024-09-14T06:31:54.339 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 23.622302166000001, 2024-09-14T06:31:54.339 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.002522121, 2024-09-14T06:31:54.339 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:31:54.340 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:31:54.340 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:31:54.340 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:31:54.340 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:54.340 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:31:54.340 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:30.662434+0000", 2024-09-14T06:31:54.340 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:54.340 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:54.340 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:54.340 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:31:54.340 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:30.662434+0000", 2024-09-14T06:31:54.340 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:54.340 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:54.341 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:54.341 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:31:54.341 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:30.662436+0000", 2024-09-14T06:31:54.341 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.257e-06 2024-09-14T06:31:54.341 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:54.341 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:54.341 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:31:54.341 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:30.662527+0000", 2024-09-14T06:31:54.341 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.1644000000000006e-05 2024-09-14T06:31:54.341 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:54.341 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:54.341 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:31:54.341 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:30.662546+0000", 2024-09-14T06:31:54.341 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.8998e-05 2024-09-14T06:31:54.342 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:54.342 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:54.342 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:31:54.342 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:30.662556+0000", 2024-09-14T06:31:54.342 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.9809999999999998e-06 2024-09-14T06:31:54.342 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:54.342 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:54.342 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:31:54.342 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:30.662621+0000", 2024-09-14T06:31:54.342 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.4597000000000005e-05 2024-09-14T06:31:54.342 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:54.342 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:54.342 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:31:54.343 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:30.662643+0000", 2024-09-14T06:31:54.343 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.1608e-05 2024-09-14T06:31:54.343 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:54.343 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:54.343 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-14T06:31:54.343 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:30.664909+0000", 2024-09-14T06:31:54.343 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0022668689999999999 2024-09-14T06:31:54.343 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:54.343 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:54.343 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:31:54.343 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:30.664957+0000", 2024-09-14T06:31:54.343 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.7166999999999998e-05 2024-09-14T06:31:54.343 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:54.343 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:31:54.343 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:54.344 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:54.344 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:54.344 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4397.0:204589 3.4 e166/23 3:234552f0:::rbd_data.112d3bd84cb0.000000000000003f:head v 166'499, mlcod=165'497)", 2024-09-14T06:31:54.344 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:31:30.772894+0000", 2024-09-14T06:31:54.344 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 23.511842309999999, 2024-09-14T06:31:54.344 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0034564650000000001, 2024-09-14T06:31:54.344 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:31:54.344 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:31:54.344 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:31:54.344 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:31:54.344 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:54.344 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:31:54.344 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:30.772894+0000", 2024-09-14T06:31:54.344 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:54.345 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:54.345 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:54.345 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:31:54.345 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:30.772894+0000", 2024-09-14T06:31:54.345 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:54.345 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:54.345 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:54.345 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:31:54.345 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:30.772896+0000", 2024-09-14T06:31:54.345 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.5379999999999999e-06 2024-09-14T06:31:54.345 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:54.345 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:54.345 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:31:54.346 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:30.773042+0000", 2024-09-14T06:31:54.346 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000146107 2024-09-14T06:31:54.346 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:54.346 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:54.346 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:31:54.346 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:30.773061+0000", 2024-09-14T06:31:54.346 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.8862999999999999e-05 2024-09-14T06:31:54.346 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:54.346 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:54.346 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:31:54.346 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:30.773071+0000", 2024-09-14T06:31:54.346 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.0227000000000001e-05 2024-09-14T06:31:54.346 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:54.347 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:54.347 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:31:54.347 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:30.773149+0000", 2024-09-14T06:31:54.347 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.8071999999999994e-05 2024-09-14T06:31:54.347 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:54.347 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:54.347 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:31:54.347 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:30.773171+0000", 2024-09-14T06:31:54.347 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.1699999999999999e-05 2024-09-14T06:31:54.347 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:54.347 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:54.347 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-14T06:31:54.347 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:30.776285+0000", 2024-09-14T06:31:54.347 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0031142050000000001 2024-09-14T06:31:54.347 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:54.348 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:54.348 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:31:54.348 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:30.776351+0000", 2024-09-14T06:31:54.348 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.5753000000000006e-05 2024-09-14T06:31:54.348 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:54.348 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:31:54.348 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:54.348 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:54.348 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:54.348 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4397.0:204598 3.4 e166/23 3:245d069e:::rbd_data.112d3bd84cb0.0000000000000041:head v 166'500, mlcod=165'497)", 2024-09-14T06:31:54.348 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:31:30.775247+0000", 2024-09-14T06:31:54.348 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 23.509489187, 2024-09-14T06:31:54.348 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00318235, 2024-09-14T06:31:54.349 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:31:54.349 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:31:54.349 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:31:54.349 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:31:54.349 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:54.349 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:31:54.349 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:30.775247+0000", 2024-09-14T06:31:54.349 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:54.349 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:54.349 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:54.349 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:31:54.349 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:30.775247+0000", 2024-09-14T06:31:54.350 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:54.350 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:54.350 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:54.350 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:31:54.350 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:30.775249+0000", 2024-09-14T06:31:54.350 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.265e-06 2024-09-14T06:31:54.350 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:54.350 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:54.350 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:31:54.350 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:30.775340+0000", 2024-09-14T06:31:54.350 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.1131999999999994e-05 2024-09-14T06:31:54.350 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:54.350 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:54.351 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:31:54.351 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:30.775352+0000", 2024-09-14T06:31:54.351 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.2294e-05 2024-09-14T06:31:54.351 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:54.351 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:54.351 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:31:54.351 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:30.775361+0000", 2024-09-14T06:31:54.351 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.6950000000000006e-06 2024-09-14T06:31:54.351 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:54.351 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:54.351 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:31:54.351 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:30.775439+0000", 2024-09-14T06:31:54.351 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.7910000000000005e-05 2024-09-14T06:31:54.351 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:54.352 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:54.352 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:31:54.352 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:30.775460+0000", 2024-09-14T06:31:54.352 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.0900999999999999e-05 2024-09-14T06:31:54.352 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:54.352 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:54.352 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-14T06:31:54.352 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:30.778385+0000", 2024-09-14T06:31:54.352 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.002925539 2024-09-14T06:31:54.352 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:54.352 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:54.352 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:31:54.352 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:30.778430+0000", 2024-09-14T06:31:54.353 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.4613999999999997e-05 2024-09-14T06:31:54.353 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:54.353 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:31:54.353 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:54.353 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:54.353 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:54.353 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4397.0:204975 3.4 e166/23 3:20972d18:::rbd_data.112d3bd84cb0.0000000000000046:head v 166'501, mlcod=166'499)", 2024-09-14T06:31:54.353 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:31:30.787253+0000", 2024-09-14T06:31:54.353 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 23.497483158000001, 2024-09-14T06:31:54.353 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.002248318, 2024-09-14T06:31:54.353 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:31:54.353 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:31:54.353 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:31:54.353 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:31:54.354 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:54.354 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:31:54.354 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:30.787253+0000", 2024-09-14T06:31:54.354 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:54.354 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:54.354 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:54.354 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:31:54.354 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:30.787253+0000", 2024-09-14T06:31:54.354 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:54.354 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:54.354 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:54.354 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:31:54.354 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:30.787255+0000", 2024-09-14T06:31:54.354 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.271e-06 2024-09-14T06:31:54.355 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:54.355 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:54.355 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:31:54.355 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:30.787442+0000", 2024-09-14T06:31:54.355 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00018744000000000001 2024-09-14T06:31:54.355 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:54.355 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:54.355 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:31:54.355 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:30.787461+0000", 2024-09-14T06:31:54.355 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.9054000000000001e-05 2024-09-14T06:31:54.355 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:54.355 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:54.355 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:31:54.356 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:30.787470+0000", 2024-09-14T06:31:54.356 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.0929999999999992e-06 2024-09-14T06:31:54.356 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:54.356 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:54.356 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:31:54.356 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:30.788004+0000", 2024-09-14T06:31:54.356 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00053359399999999997 2024-09-14T06:31:54.356 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:54.356 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:54.356 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:31:54.356 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:30.788034+0000", 2024-09-14T06:31:54.356 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.9635e-05 2024-09-14T06:31:54.356 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:54.356 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:54.357 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-14T06:31:54.357 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:30.789457+0000", 2024-09-14T06:31:54.357 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.001423487 2024-09-14T06:31:54.357 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:54.357 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:54.357 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:31:54.357 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:30.789502+0000", 2024-09-14T06:31:54.357 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.4743999999999997e-05 2024-09-14T06:31:54.357 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:54.357 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:31:54.357 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:54.357 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:54.357 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:54.357 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4397.0:212188 3.4 e166/23 3:23871bc7:::rbd_data.112d3bd84cb0.00000000000000fa:head v 166'508, mlcod=166'506)", 2024-09-14T06:31:54.358 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:31:31.391415+0000", 2024-09-14T06:31:54.358 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 22.893321575999998, 2024-09-14T06:31:54.358 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0022039049999999999, 2024-09-14T06:31:54.358 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:31:54.358 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:31:54.358 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:31:54.358 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:31:54.358 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:54.358 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:31:54.358 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:31.391415+0000", 2024-09-14T06:31:54.358 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:54.358 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:54.358 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:54.358 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:31:54.359 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:31.391415+0000", 2024-09-14T06:31:54.359 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:54.359 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:54.359 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:54.359 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:31:54.359 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:31.391416+0000", 2024-09-14T06:31:54.359 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.44e-06 2024-09-14T06:31:54.359 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:54.359 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:54.359 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:31:54.359 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:31.391554+0000", 2024-09-14T06:31:54.359 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000137119 2024-09-14T06:31:54.359 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:54.359 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:54.360 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:31:54.360 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:31.391570+0000", 2024-09-14T06:31:54.360 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.6827e-05 2024-09-14T06:31:54.360 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:54.360 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:54.360 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:31:54.360 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:31.391580+0000", 2024-09-14T06:31:54.360 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.3139999999999993e-06 2024-09-14T06:31:54.360 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:54.360 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:54.360 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:31:54.360 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:31.391664+0000", 2024-09-14T06:31:54.360 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.3757999999999993e-05 2024-09-14T06:31:54.361 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:54.361 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:54.361 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:31:54.361 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:31.391692+0000", 2024-09-14T06:31:54.361 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.8898e-05 2024-09-14T06:31:54.361 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:54.361 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:54.361 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-14T06:31:54.361 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:31.393584+0000", 2024-09-14T06:31:54.361 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.001891752 2024-09-14T06:31:54.361 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:54.361 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:54.361 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:31:54.362 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:31.393619+0000", 2024-09-14T06:31:54.362 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.4796999999999999e-05 2024-09-14T06:31:54.362 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:54.362 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:31:54.362 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:54.362 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:54.362 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:54.362 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "MOSDPGPush(3.5 169/168 [PushOp(3:a7c12b08:::rbd_data.112ddfbec471.000000000000000e:head, version: 77'100, data_included: [0~1048576], data_size: 1048576, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(3:a7c12b08:::rbd_data.112ddfbec471.000000000000000e:head@77'100, size: 1048576, copy_subset: [(0, 1048576)], clone_subset: {}, snapset: 0=[]:{}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 1048576, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2024-09-14T06:31:54.362 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:31:34.540707+0000", 2024-09-14T06:31:54.362 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 19.744029811000001, 2024-09-14T06:31:54.362 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0023637369999999999, 2024-09-14T06:31:54.362 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:31:54.362 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:31:54.362 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:31:54.363 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:31:54.363 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:54.363 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:31:54.363 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:34.540707+0000", 2024-09-14T06:31:54.363 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:54.363 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:54.363 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:54.363 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:31:54.363 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:34.540707+0000", 2024-09-14T06:31:54.363 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:54.363 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:54.363 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:54.363 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:31:54.363 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:34.540708+0000", 2024-09-14T06:31:54.364 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.7199999999999998e-07 2024-09-14T06:31:54.364 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:54.364 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:54.364 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:31:54.364 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:34.542150+0000", 2024-09-14T06:31:54.364 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0014423229999999999 2024-09-14T06:31:54.364 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:54.364 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:54.364 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:31:54.364 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:34.542169+0000", 2024-09-14T06:31:54.364 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.8756000000000001e-05 2024-09-14T06:31:54.364 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:54.365 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:54.365 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:31:54.365 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:34.542180+0000", 2024-09-14T06:31:54.365 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.1051e-05 2024-09-14T06:31:54.365 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:54.365 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:54.365 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:31:54.365 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:34.542275+0000", 2024-09-14T06:31:54.365 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.5122000000000002e-05 2024-09-14T06:31:54.365 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:54.365 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:54.365 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:31:54.365 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:34.542284+0000", 2024-09-14T06:31:54.365 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.9190000000000005e-06 2024-09-14T06:31:54.366 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:54.366 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:54.366 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:31:54.366 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:34.543071+0000", 2024-09-14T06:31:54.366 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00078679399999999995 2024-09-14T06:31:54.366 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:54.366 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:31:54.366 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:54.366 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:54.366 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:54.366 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "MOSDPGPush(3.5 169/168 [PushOp(3:a5c8b3f2:::rbd_data.112ddfbec471.0000000000000018:head, version: 77'101, data_included: [0~1048576], data_size: 1048576, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(3:a5c8b3f2:::rbd_data.112ddfbec471.0000000000000018:head@77'101, size: 1048576, copy_subset: [(0, 1048576)], clone_subset: {}, snapset: 0=[]:{}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 1048576, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2024-09-14T06:31:54.366 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:31:34.545533+0000", 2024-09-14T06:31:54.366 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 19.739203924000002, 2024-09-14T06:31:54.366 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0021867010000000001, 2024-09-14T06:31:54.367 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:31:54.367 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:31:54.367 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:31:54.367 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:31:54.367 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:54.367 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:31:54.367 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:34.545533+0000", 2024-09-14T06:31:54.367 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:54.367 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:54.367 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:54.367 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:31:54.367 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:34.545533+0000", 2024-09-14T06:31:54.367 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:54.368 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:54.368 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:54.368 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:31:54.368 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:34.545533+0000", 2024-09-14T06:31:54.368 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.8100000000000002e-07 2024-09-14T06:31:54.368 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:54.368 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:54.368 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:31:54.368 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:34.546784+0000", 2024-09-14T06:31:54.368 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.001250747 2024-09-14T06:31:54.368 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:54.368 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:54.368 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:31:54.368 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:34.546803+0000", 2024-09-14T06:31:54.369 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.8346e-05 2024-09-14T06:31:54.369 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:54.369 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:54.369 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:31:54.369 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:34.546823+0000", 2024-09-14T06:31:54.369 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.0699999999999998e-05 2024-09-14T06:31:54.369 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:54.369 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:54.369 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:31:54.369 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:34.546890+0000", 2024-09-14T06:31:54.369 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.6581000000000005e-05 2024-09-14T06:31:54.369 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:54.369 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:54.370 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:31:54.370 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:34.546899+0000", 2024-09-14T06:31:54.370 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.3889999999999994e-06 2024-09-14T06:31:54.370 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:54.370 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:54.370 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:31:54.370 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:34.547719+0000", 2024-09-14T06:31:54.370 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00082015699999999996 2024-09-14T06:31:54.370 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:54.370 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:31:54.370 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:54.370 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:54.370 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:54.370 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "MOSDPGPush(3.5 169/168 [PushOp(3:a7a54476:::rbd_data.112d1bb7b269.0000000000000038:head, version: 122'581, data_included: [0~524288], data_size: 524288, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(3:a7a54476:::rbd_data.112d1bb7b269.0000000000000038:head@122'581, size: 524288, copy_subset: [(0, 524288)], clone_subset: {}, snapset: 0=[]:{}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 524288, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2024-09-14T06:31:54.371 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:31:34.667757+0000", 2024-09-14T06:31:54.371 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 19.616979765, 2024-09-14T06:31:54.371 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.002750382, 2024-09-14T06:31:54.371 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:31:54.371 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:31:54.371 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:31:54.371 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:31:54.371 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:54.371 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:31:54.371 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:34.667757+0000", 2024-09-14T06:31:54.371 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:54.371 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:54.371 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:54.371 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:31:54.372 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:34.667757+0000", 2024-09-14T06:31:54.372 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:54.372 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:54.372 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:54.372 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:31:54.372 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:34.667758+0000", 2024-09-14T06:31:54.372 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.3599999999999999e-06 2024-09-14T06:31:54.372 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:54.372 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:54.372 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:31:54.372 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:34.669375+0000", 2024-09-14T06:31:54.372 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.001616644 2024-09-14T06:31:54.372 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:54.372 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:54.373 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:31:54.373 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:34.669404+0000", 2024-09-14T06:31:54.373 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.9300999999999999e-05 2024-09-14T06:31:54.373 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:54.373 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:54.373 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:31:54.373 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:34.669428+0000", 2024-09-14T06:31:54.373 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.3788e-05 2024-09-14T06:31:54.373 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:54.373 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:54.373 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:31:54.373 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:34.669543+0000", 2024-09-14T06:31:54.373 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000115462 2024-09-14T06:31:54.374 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:54.374 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:54.374 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:31:54.374 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:34.669559+0000", 2024-09-14T06:31:54.374 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.5167999999999999e-05 2024-09-14T06:31:54.374 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:54.374 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:54.374 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:31:54.374 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:34.670507+0000", 2024-09-14T06:31:54.374 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00094865900000000003 2024-09-14T06:31:54.374 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:54.374 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:31:54.374 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:54.374 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:54.375 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:54.375 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "MOSDPGPush(3.5 170/168 [PushOp(3:a162138c:::rbd_data.112d1bb7b269.000000000000003c:head, version: 122'582, data_included: [0~524288], data_size: 524288, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(3:a162138c:::rbd_data.112d1bb7b269.000000000000003c:head@122'582, size: 524288, copy_subset: [(0, 524288)], clone_subset: {}, snapset: 0=[]:{}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 524288, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2024-09-14T06:31:54.375 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:31:34.670110+0000", 2024-09-14T06:31:54.375 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 19.614626816000001, 2024-09-14T06:31:54.375 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0030066229999999999, 2024-09-14T06:31:54.375 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:31:54.375 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:31:54.375 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:31:54.375 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:31:54.375 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:54.375 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:31:54.375 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:34.670110+0000", 2024-09-14T06:31:54.375 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:54.375 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:54.376 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:54.376 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:31:54.376 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:34.670110+0000", 2024-09-14T06:31:54.376 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:54.376 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:54.376 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:54.376 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:31:54.376 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:34.670111+0000", 2024-09-14T06:31:54.376 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.6499999999999998e-07 2024-09-14T06:31:54.376 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:54.376 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:54.376 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:31:54.376 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:34.671507+0000", 2024-09-14T06:31:54.376 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.001396284 2024-09-14T06:31:54.377 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:54.377 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:54.377 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:31:54.377 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:34.671530+0000", 2024-09-14T06:31:54.377 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.3023e-05 2024-09-14T06:31:54.377 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:54.377 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:54.377 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:31:54.377 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:34.671548+0000", 2024-09-14T06:31:54.377 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.8116000000000001e-05 2024-09-14T06:31:54.377 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:54.377 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:54.377 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:31:54.377 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:34.671775+0000", 2024-09-14T06:31:54.378 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00022699600000000001 2024-09-14T06:31:54.378 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:54.378 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:54.378 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:31:54.378 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:34.671790+0000", 2024-09-14T06:31:54.378 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.539e-05 2024-09-14T06:31:54.378 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:54.378 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:54.378 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:31:54.378 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:34.673116+0000", 2024-09-14T06:31:54.378 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0013259490000000001 2024-09-14T06:31:54.378 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:54.378 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:31:54.378 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:54.378 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:54.379 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:31:54.379 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-14T06:31:54.380 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-14T06:31:54.381 INFO:tasks.ceph.osd.5.smithi138.stderr:2024-09-14T06:31:54.379+0000 7fe13746f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33618) UID: 0 2024-09-14T06:31:54.481 INFO:tasks.ceph.osd.1.smithi094.stderr:2024-09-14T06:31:54.480+0000 7f328e47a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33641) UID: 0 2024-09-14T06:31:54.482 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-14T06:31:54.482 INFO:teuthology.orchestra.run.smithi094.stdout: "ops": [ 2024-09-14T06:31:54.483 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:54.483 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:225587 3.7 3.cd474887 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e189)", 2024-09-14T06:31:54.483 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:31:53.874591+0000", 2024-09-14T06:31:54.483 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 0.60769262199999996, 2024-09-14T06:31:54.483 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.60771014199999995, 2024-09-14T06:31:54.483 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:31:54.483 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:31:54.483 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "waiting for peered", 2024-09-14T06:31:54.483 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:31:54.483 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:31:54.483 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:54.483 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 225587 2024-09-14T06:31:54.483 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:54.483 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:31:54.484 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:54.484 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:31:54.484 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:53.874591+0000", 2024-09-14T06:31:54.484 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:54.484 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:54.484 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:54.484 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:31:54.484 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:53.874591+0000", 2024-09-14T06:31:54.484 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:54.484 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:54.484 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:54.484 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:31:54.484 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:53.874592+0000", 2024-09-14T06:31:54.485 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.5209999999999999e-06 2024-09-14T06:31:54.485 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:54.485 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:54.485 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:31:54.485 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:53.874599+0000", 2024-09-14T06:31:54.485 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.6290000000000004e-06 2024-09-14T06:31:54.485 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:54.485 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:54.485 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:31:54.485 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:53.874618+0000", 2024-09-14T06:31:54.485 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.8950999999999999e-05 2024-09-14T06:31:54.485 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:54.485 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:54.486 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:31:54.486 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:53.874626+0000", 2024-09-14T06:31:54.486 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.3960000000000006e-06 2024-09-14T06:31:54.486 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:54.486 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:54.486 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:31:54.486 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:53.874699+0000", 2024-09-14T06:31:54.486 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.2329999999999994e-05 2024-09-14T06:31:54.486 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:54.486 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:54.486 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "waiting for peered", 2024-09-14T06:31:54.486 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:53.874711+0000", 2024-09-14T06:31:54.486 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.2473999999999999e-05 2024-09-14T06:31:54.487 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:54.487 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:31:54.487 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:54.487 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:54.487 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-09-14T06:31:54.487 INFO:teuthology.orchestra.run.smithi094.stdout: "num_ops": 1 2024-09-14T06:31:54.487 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-14T06:31:54.491 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-14T06:31:54.582 INFO:tasks.ceph.osd.3.smithi094.stderr:2024-09-14T06:31:54.581+0000 7feeeafe8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33655) UID: 0 2024-09-14T06:31:54.632 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-14T06:31:54.633 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-14T06:31:54.633 INFO:teuthology.orchestra.run.smithi094.stdout: "ops": [], 2024-09-14T06:31:54.634 INFO:teuthology.orchestra.run.smithi094.stdout: "complaint_time": 30, 2024-09-14T06:31:54.634 INFO:teuthology.orchestra.run.smithi094.stdout: "num_blocked_ops": 0 2024-09-14T06:31:54.634 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-14T06:31:54.642 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-14T06:31:54.683 INFO:tasks.ceph.osd.2.smithi094.stderr:2024-09-14T06:31:54.682+0000 7faf3269b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33649) UID: 0 2024-09-14T06:31:54.745 INFO:teuthology.orchestra.run.smithi094.stderr:noscrub is unset 2024-09-14T06:31:54.747 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 4, 5, 6, 7, 3] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-14T06:31:54.747 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-14T06:31:54.747 INFO:tasks.thrashosds.thrasher:inject_pause on osd.0 2024-09-14T06:31:54.747 INFO:tasks.thrashosds.thrasher:Testing filestore_inject_stall pause injection for duration 3 2024-09-14T06:31:54.747 INFO:tasks.thrashosds.thrasher:Checking after 0, should_be_down=False 2024-09-14T06:31:54.747 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok config set filestore_inject_stall 3 2024-09-14T06:31:54.772 INFO:teuthology.orchestra.run.smithi094.stdout:osd.0: {} 2024-09-14T06:31:54.772 INFO:teuthology.orchestra.run.smithi094.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-14T06:31:54.773 INFO:teuthology.orchestra.run.smithi094.stdout:checking clone #40, image image_client.0-clone40 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent41 2024-09-14T06:31:54.783 INFO:tasks.ceph.osd.7.smithi138.stderr:2024-09-14T06:31:54.782+0000 7f3c6a21d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33623) UID: 0 2024-09-14T06:31:54.791 INFO:teuthology.orchestra.run.smithi094.stdout:osd.1: {} 2024-09-14T06:31:54.791 INFO:teuthology.orchestra.run.smithi094.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-14T06:31:54.804 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-14T06:31:54.804 INFO:teuthology.orchestra.run.smithi094.stdout: "size": 20, 2024-09-14T06:31:54.805 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 600, 2024-09-14T06:31:54.805 INFO:teuthology.orchestra.run.smithi094.stdout: "ops": [ 2024-09-14T06:31:54.805 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:54.805 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:44425 3.2 3:4603ad7f:::rbd_data.112dca2f6ae1.0000000000000005:17 [read 868352~4096 [fadvise_nocache]] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:31:54.805 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:43.854967+0000", 2024-09-14T06:31:54.805 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 130.94771518900001, 2024-09-14T06:31:54.805 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.063146025999999994, 2024-09-14T06:31:54.805 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:31:54.805 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:31:54.805 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:31:54.805 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:31:54.805 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:31:54.805 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:54.806 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 44425 2024-09-14T06:31:54.806 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:54.806 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:31:54.806 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:54.806 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:31:54.806 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.854967+0000", 2024-09-14T06:31:54.806 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:54.806 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:54.806 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:54.806 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:31:54.806 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.854967+0000", 2024-09-14T06:31:54.806 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:54.806 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:54.806 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:54.806 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:31:54.807 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.854967+0000", 2024-09-14T06:31:54.807 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.16e-07 2024-09-14T06:31:54.807 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:54.807 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:54.807 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:31:54.807 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.854968+0000", 2024-09-14T06:31:54.807 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.006e-06 2024-09-14T06:31:54.807 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:54.807 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:54.807 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:31:54.807 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.854971+0000", 2024-09-14T06:31:54.807 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.9359999999999999e-06 2024-09-14T06:31:54.807 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:54.808 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:54.808 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:31:54.808 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.854974+0000", 2024-09-14T06:31:54.808 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.1840000000000002e-06 2024-09-14T06:31:54.808 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:54.808 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:54.808 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:31:54.808 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.917945+0000", 2024-09-14T06:31:54.808 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.062970925999999997 2024-09-14T06:31:54.808 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:54.808 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:54.808 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:31:54.808 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.918030+0000", 2024-09-14T06:31:54.808 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.5863000000000001e-05 2024-09-14T06:31:54.809 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:54.809 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:54.809 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:31:54.809 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.918113+0000", 2024-09-14T06:31:54.809 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.2894999999999998e-05 2024-09-14T06:31:54.809 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:54.809 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:31:54.809 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:54.809 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:54.809 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:54.809 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:44426 3.2 3:4603ad7f:::rbd_data.112dca2f6ae1.0000000000000005:17 [read 892928~4096 [fadvise_nocache]] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:31:54.809 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:43.854977+0000", 2024-09-14T06:31:54.809 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 130.94770577899999, 2024-09-14T06:31:54.809 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.063369237999999994, 2024-09-14T06:31:54.810 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:31:54.810 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:31:54.810 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:31:54.810 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:31:54.810 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:31:54.810 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:54.810 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 44426 2024-09-14T06:31:54.810 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:54.810 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:31:54.810 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:54.810 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:31:54.810 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.854977+0000", 2024-09-14T06:31:54.810 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:54.811 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:54.811 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:54.811 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:31:54.811 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.854977+0000", 2024-09-14T06:31:54.811 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:54.811 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:54.811 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:54.811 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:31:54.811 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.854977+0000", 2024-09-14T06:31:54.811 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.1199999999999999e-07 2024-09-14T06:31:54.811 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:54.812 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:54.812 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:31:54.812 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.854980+0000", 2024-09-14T06:31:54.812 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.27e-06 2024-09-14T06:31:54.812 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:54.812 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:54.812 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:31:54.812 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.854983+0000", 2024-09-14T06:31:54.812 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.1650000000000002e-06 2024-09-14T06:31:54.812 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:54.812 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:54.812 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:31:54.812 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.854989+0000", 2024-09-14T06:31:54.813 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.6520000000000003e-06 2024-09-14T06:31:54.813 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:54.813 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:54.813 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:31:54.813 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.918143+0000", 2024-09-14T06:31:54.813 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.063154541999999994 2024-09-14T06:31:54.813 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:54.813 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:54.813 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:31:54.813 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.918209+0000", 2024-09-14T06:31:54.813 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.5529000000000002e-05 2024-09-14T06:31:54.813 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:54.813 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:54.813 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:31:54.814 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.918346+0000", 2024-09-14T06:31:54.814 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000136868 2024-09-14T06:31:54.814 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:54.814 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:31:54.814 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:54.814 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:54.814 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:54.814 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:44427 3.2 3:4603ad7f:::rbd_data.112dca2f6ae1.0000000000000005:17 [read 917504~4096 [fadvise_nocache]] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:31:54.814 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:43.854992+0000", 2024-09-14T06:31:54.814 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 130.94769013300001, 2024-09-14T06:31:54.814 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.063607792999999996, 2024-09-14T06:31:54.814 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:31:54.814 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:31:54.814 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:31:54.815 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:31:54.815 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:31:54.815 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:54.815 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 44427 2024-09-14T06:31:54.815 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:54.815 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:31:54.815 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:54.815 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:31:54.815 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.854992+0000", 2024-09-14T06:31:54.815 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:54.815 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:54.815 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:54.815 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:31:54.815 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.854992+0000", 2024-09-14T06:31:54.815 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:54.816 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:54.816 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:54.816 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:31:54.816 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.854993+0000", 2024-09-14T06:31:54.816 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.1400000000000001e-07 2024-09-14T06:31:54.816 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:54.816 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:54.816 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:31:54.816 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.854994+0000", 2024-09-14T06:31:54.816 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.091e-06 2024-09-14T06:31:54.816 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:54.816 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:54.816 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:31:54.816 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855001+0000", 2024-09-14T06:31:54.817 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.204e-06 2024-09-14T06:31:54.817 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:54.817 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:54.817 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:31:54.817 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855003+0000", 2024-09-14T06:31:54.817 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.514e-06 2024-09-14T06:31:54.817 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:54.817 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:54.817 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:31:54.817 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.918380+0000", 2024-09-14T06:31:54.817 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.063376738000000002 2024-09-14T06:31:54.817 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:54.817 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:54.817 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:31:54.817 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.918453+0000", 2024-09-14T06:31:54.818 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.3291e-05 2024-09-14T06:31:54.818 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:54.818 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:54.818 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:31:54.818 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.918600+0000", 2024-09-14T06:31:54.818 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00014674100000000001 2024-09-14T06:31:54.818 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:54.818 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:31:54.818 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:54.818 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:54.818 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:54.818 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:44428 3.2 3:4603ad7f:::rbd_data.112dca2f6ae1.0000000000000005:17 [read 942080~4096 [fadvise_nocache]] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:31:54.818 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:43.855006+0000", 2024-09-14T06:31:54.818 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 130.947676083, 2024-09-14T06:31:54.818 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.063846321999999997, 2024-09-14T06:31:54.818 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:31:54.819 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:31:54.819 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:31:54.819 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:31:54.819 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:31:54.819 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:54.819 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 44428 2024-09-14T06:31:54.819 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:54.819 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:31:54.819 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:54.819 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:31:54.819 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855006+0000", 2024-09-14T06:31:54.819 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:54.819 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:54.819 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:54.820 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:31:54.820 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855006+0000", 2024-09-14T06:31:54.820 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:54.820 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:54.820 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:54.820 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:31:54.820 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855007+0000", 2024-09-14T06:31:54.820 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.1199999999999999e-07 2024-09-14T06:31:54.820 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:54.820 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:54.820 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:31:54.820 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855011+0000", 2024-09-14T06:31:54.820 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.9539999999999998e-06 2024-09-14T06:31:54.821 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:54.821 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:54.821 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:31:54.821 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855014+0000", 2024-09-14T06:31:54.821 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.2930000000000001e-06 2024-09-14T06:31:54.821 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:54.821 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:54.821 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:31:54.821 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855016+0000", 2024-09-14T06:31:54.821 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.3039999999999999e-06 2024-09-14T06:31:54.821 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:54.821 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:54.821 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:31:54.821 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.918634+0000", 2024-09-14T06:31:54.821 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.063617928000000004 2024-09-14T06:31:54.822 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:54.822 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:54.822 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:31:54.822 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.918711+0000", 2024-09-14T06:31:54.822 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.6545000000000007e-05 2024-09-14T06:31:54.822 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:54.822 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:54.822 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:31:54.822 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.918853+0000", 2024-09-14T06:31:54.822 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00014208600000000001 2024-09-14T06:31:54.822 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:54.823 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:31:54.823 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:54.823 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:54.823 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:54.823 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:44429 3.2 3:4603ad7f:::rbd_data.112dca2f6ae1.0000000000000005:17 [read 966656~4096 [fadvise_nocache]] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:31:54.823 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:43.855019+0000", 2024-09-14T06:31:54.823 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 130.94766340999999, 2024-09-14T06:31:54.823 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.064059226999999996, 2024-09-14T06:31:54.823 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:31:54.823 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:31:54.823 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:31:54.823 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:31:54.823 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:31:54.823 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:54.823 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 44429 2024-09-14T06:31:54.824 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:54.824 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:31:54.824 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:54.824 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:31:54.824 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855019+0000", 2024-09-14T06:31:54.824 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:54.824 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:54.824 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:54.824 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:31:54.824 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855019+0000", 2024-09-14T06:31:54.824 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:54.824 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:54.824 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:54.825 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:31:54.825 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855019+0000", 2024-09-14T06:31:54.825 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.28e-07 2024-09-14T06:31:54.825 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:54.825 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:54.825 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:31:54.825 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855020+0000", 2024-09-14T06:31:54.825 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.0330000000000001e-06 2024-09-14T06:31:54.825 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:54.825 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:54.825 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:31:54.825 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855023+0000", 2024-09-14T06:31:54.825 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.79e-06 2024-09-14T06:31:54.825 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:54.826 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:54.826 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:31:54.826 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855025+0000", 2024-09-14T06:31:54.826 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.216e-06 2024-09-14T06:31:54.826 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:54.826 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:54.826 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:31:54.826 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.918886+0000", 2024-09-14T06:31:54.826 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.063861037999999995 2024-09-14T06:31:54.826 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:54.826 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:54.826 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:31:54.826 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.918952+0000", 2024-09-14T06:31:54.826 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.5993000000000003e-05 2024-09-14T06:31:54.827 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:54.827 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:54.827 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:31:54.827 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.919078+0000", 2024-09-14T06:31:54.827 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00012592899999999999 2024-09-14T06:31:54.827 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:54.827 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:31:54.827 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:54.827 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:54.827 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:54.827 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:44430 3.2 3:4603ad7f:::rbd_data.112dca2f6ae1.0000000000000005:17 [read 991232~4096 [fadvise_nocache]] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:31:54.827 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:43.855028+0000", 2024-09-14T06:31:54.827 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 130.9476544, 2024-09-14T06:31:54.827 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.064284346000000006, 2024-09-14T06:31:54.828 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:31:54.828 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:31:54.828 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:31:54.828 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:31:54.828 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:31:54.828 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:54.828 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 44430 2024-09-14T06:31:54.828 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:54.828 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:31:54.828 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:54.828 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:31:54.828 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855028+0000", 2024-09-14T06:31:54.828 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:54.828 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:54.829 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:54.829 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:31:54.829 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855028+0000", 2024-09-14T06:31:54.829 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:54.829 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:54.829 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:54.829 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:31:54.829 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855028+0000", 2024-09-14T06:31:54.829 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.1199999999999999e-07 2024-09-14T06:31:54.829 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:54.829 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:54.829 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:31:54.829 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855029+0000", 2024-09-14T06:31:54.829 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.1100000000000004e-07 2024-09-14T06:31:54.830 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:54.830 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:54.830 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:31:54.830 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855032+0000", 2024-09-14T06:31:54.830 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.7889999999999999e-06 2024-09-14T06:31:54.830 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:54.830 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:54.830 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:31:54.830 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855034+0000", 2024-09-14T06:31:54.830 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.125e-06 2024-09-14T06:31:54.830 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:54.830 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:54.830 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:31:54.830 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.919110+0000", 2024-09-14T06:31:54.831 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.064076413999999998 2024-09-14T06:31:54.831 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:54.831 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:54.831 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:31:54.831 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.919186+0000", 2024-09-14T06:31:54.831 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.5784999999999994e-05 2024-09-14T06:31:54.831 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:54.831 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:54.831 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:31:54.831 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.919312+0000", 2024-09-14T06:31:54.831 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00012611 2024-09-14T06:31:54.831 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:54.831 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:31:54.831 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:54.831 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:54.832 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:54.832 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:44431 3.2 3:4603ad7f:::rbd_data.112dca2f6ae1.0000000000000005:17 [read 1015808~4096 [fadvise_nocache]] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:31:54.832 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:43.855037+0000", 2024-09-14T06:31:54.832 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 130.947645532, 2024-09-14T06:31:54.832 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.064469755000000004, 2024-09-14T06:31:54.832 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:31:54.832 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:31:54.832 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:31:54.832 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:31:54.832 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:31:54.832 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:54.832 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 44431 2024-09-14T06:31:54.832 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:54.832 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:31:54.833 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:54.833 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:31:54.833 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855037+0000", 2024-09-14T06:31:54.833 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:54.833 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:54.833 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:54.833 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:31:54.833 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855037+0000", 2024-09-14T06:31:54.833 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:54.833 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:54.833 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:54.833 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:31:54.833 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855037+0000", 2024-09-14T06:31:54.833 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.1e-07 2024-09-14T06:31:54.834 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:54.834 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:54.834 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:31:54.834 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855038+0000", 2024-09-14T06:31:54.834 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.9400000000000004e-07 2024-09-14T06:31:54.834 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:54.834 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:54.834 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:31:54.834 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855041+0000", 2024-09-14T06:31:54.834 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.9639999999999999e-06 2024-09-14T06:31:54.834 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:54.834 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:54.834 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:31:54.834 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855043+0000", 2024-09-14T06:31:54.835 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.2259999999999999e-06 2024-09-14T06:31:54.835 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:54.835 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:54.835 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:31:54.835 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.919347+0000", 2024-09-14T06:31:54.835 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.064304091999999993 2024-09-14T06:31:54.835 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:54.835 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:54.835 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:31:54.835 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.919393+0000", 2024-09-14T06:31:54.835 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.5340999999999998e-05 2024-09-14T06:31:54.835 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:54.835 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:54.835 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:31:54.835 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.919507+0000", 2024-09-14T06:31:54.836 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000114028 2024-09-14T06:31:54.836 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:54.836 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:31:54.836 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:54.836 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:54.836 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:54.836 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:44432 3.2 3:4603ad7f:::rbd_data.112dca2f6ae1.0000000000000005:17 [read 1040384~4096 [fadvise_nocache]] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:31:54.836 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:43.855046+0000", 2024-09-14T06:31:54.836 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 130.947636464, 2024-09-14T06:31:54.836 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.064639253999999993, 2024-09-14T06:31:54.836 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:31:54.836 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:31:54.836 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:31:54.837 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:31:54.837 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:31:54.837 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:54.837 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 44432 2024-09-14T06:31:54.837 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:54.837 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:31:54.837 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:54.837 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:31:54.837 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855046+0000", 2024-09-14T06:31:54.837 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:54.837 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:54.837 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:54.837 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:31:54.837 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855046+0000", 2024-09-14T06:31:54.838 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:54.838 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:54.838 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:54.838 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:31:54.838 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855046+0000", 2024-09-14T06:31:54.838 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.05e-07 2024-09-14T06:31:54.838 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:54.838 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:54.838 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:31:54.838 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855047+0000", 2024-09-14T06:31:54.838 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.879999999999999e-07 2024-09-14T06:31:54.838 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:54.838 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:54.838 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:31:54.838 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855050+0000", 2024-09-14T06:31:54.839 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.8459999999999999e-06 2024-09-14T06:31:54.839 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:54.839 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:54.839 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:31:54.839 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855052+0000", 2024-09-14T06:31:54.839 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.136e-06 2024-09-14T06:31:54.839 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:54.839 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:54.839 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:31:54.839 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.919536+0000", 2024-09-14T06:31:54.839 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.064484132 2024-09-14T06:31:54.839 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:54.839 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:54.840 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:31:54.840 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.919611+0000", 2024-09-14T06:31:54.840 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.4407999999999996e-05 2024-09-14T06:31:54.840 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:54.840 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:54.840 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:31:54.840 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.919685+0000", 2024-09-14T06:31:54.840 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.4538999999999999e-05 2024-09-14T06:31:54.840 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:54.840 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:31:54.840 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:54.840 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:54.840 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:54.840 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:44518 3.2 3:4190d882:::rbd_data.112dca2f6ae1.0000000000000002:17 [read 12288~4096 [fadvise_nocache]] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:31:54.841 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:43.855055+0000", 2024-09-14T06:31:54.841 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 130.94762757999999, 2024-09-14T06:31:54.841 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.064894317000000007, 2024-09-14T06:31:54.841 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:31:54.841 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:31:54.841 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:31:54.841 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:31:54.841 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:31:54.841 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:54.841 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 44518 2024-09-14T06:31:54.841 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:54.841 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:31:54.841 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:54.842 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:31:54.842 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855055+0000", 2024-09-14T06:31:54.842 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:54.842 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:54.842 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:54.842 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:31:54.842 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855055+0000", 2024-09-14T06:31:54.842 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:54.842 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:54.842 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:54.842 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:31:54.842 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855055+0000", 2024-09-14T06:31:54.842 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.0699999999999999e-07 2024-09-14T06:31:54.842 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:54.843 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:54.843 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:31:54.843 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855056+0000", 2024-09-14T06:31:54.843 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.9400000000000004e-07 2024-09-14T06:31:54.843 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:54.843 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:54.843 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:31:54.843 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855059+0000", 2024-09-14T06:31:54.843 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.8870000000000001e-06 2024-09-14T06:31:54.843 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:54.843 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:54.843 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:31:54.843 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855061+0000", 2024-09-14T06:31:54.843 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.1919999999999999e-06 2024-09-14T06:31:54.844 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:54.844 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:54.844 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:31:54.844 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.919719+0000", 2024-09-14T06:31:54.844 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.064658013 2024-09-14T06:31:54.844 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:54.844 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:54.844 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:31:54.844 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.919788+0000", 2024-09-14T06:31:54.844 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.9282000000000005e-05 2024-09-14T06:31:54.844 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:54.844 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:54.844 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:31:54.844 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.919949+0000", 2024-09-14T06:31:54.845 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00016084199999999999 2024-09-14T06:31:54.845 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:54.845 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:31:54.845 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:54.845 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:54.845 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:54.845 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:44519 3.2 3:4190d882:::rbd_data.112dca2f6ae1.0000000000000002:17 [read 36864~4096 [fadvise_nocache]] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:31:54.845 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:43.855064+0000", 2024-09-14T06:31:54.845 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 130.947618603, 2024-09-14T06:31:54.845 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.065081865000000003, 2024-09-14T06:31:54.845 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:31:54.845 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:31:54.845 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:31:54.846 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:31:54.846 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:31:54.846 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:54.846 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 44519 2024-09-14T06:31:54.846 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:54.846 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:31:54.846 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:54.846 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:31:54.846 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855064+0000", 2024-09-14T06:31:54.846 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:54.846 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:54.846 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:54.846 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:31:54.846 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855064+0000", 2024-09-14T06:31:54.847 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:54.847 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:54.847 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:54.847 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:31:54.847 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855064+0000", 2024-09-14T06:31:54.847 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.1e-07 2024-09-14T06:31:54.847 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:54.847 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:54.847 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:31:54.847 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855065+0000", 2024-09-14T06:31:54.847 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.035e-06 2024-09-14T06:31:54.847 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:54.847 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:54.847 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:31:54.847 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855068+0000", 2024-09-14T06:31:54.848 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.8339999999999999e-06 2024-09-14T06:31:54.848 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:54.848 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:54.848 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:31:54.848 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855071+0000", 2024-09-14T06:31:54.848 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.3450000000000002e-06 2024-09-14T06:31:54.848 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:54.848 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:54.848 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:31:54.848 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.919979+0000", 2024-09-14T06:31:54.848 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.064907647999999998 2024-09-14T06:31:54.848 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:54.848 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:54.849 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:31:54.849 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.920026+0000", 2024-09-14T06:31:54.849 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.7595000000000001e-05 2024-09-14T06:31:54.849 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:54.849 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:54.849 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:31:54.849 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.920146+0000", 2024-09-14T06:31:54.849 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00011919800000000001 2024-09-14T06:31:54.849 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:54.849 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:31:54.849 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:54.849 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:54.849 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:54.849 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:44520 3.2 3:4190d882:::rbd_data.112dca2f6ae1.0000000000000002:17 [read 61440~4096 [fadvise_nocache]] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:31:54.850 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:43.855075+0000", 2024-09-14T06:31:54.850 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 130.94760755199999, 2024-09-14T06:31:54.850 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.065335078000000005, 2024-09-14T06:31:54.850 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:31:54.850 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:31:54.850 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:31:54.850 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:31:54.850 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:31:54.850 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:54.850 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 44520 2024-09-14T06:31:54.850 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:54.850 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:31:54.850 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:54.851 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:31:54.851 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855075+0000", 2024-09-14T06:31:54.851 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:54.851 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:54.851 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:54.851 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:31:54.851 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855075+0000", 2024-09-14T06:31:54.851 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:54.851 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:54.851 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:54.851 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:31:54.851 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855075+0000", 2024-09-14T06:31:54.851 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.1e-07 2024-09-14T06:31:54.851 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:54.851 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:54.852 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:31:54.852 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855076+0000", 2024-09-14T06:31:54.852 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.0839999999999999e-06 2024-09-14T06:31:54.852 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:54.852 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:54.852 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:31:54.852 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855079+0000", 2024-09-14T06:31:54.852 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.9359999999999999e-06 2024-09-14T06:31:54.852 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:54.852 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:54.852 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:31:54.852 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855084+0000", 2024-09-14T06:31:54.852 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.1730000000000003e-06 2024-09-14T06:31:54.853 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:54.853 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:54.853 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:31:54.853 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.920175+0000", 2024-09-14T06:31:54.853 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.065090234999999996 2024-09-14T06:31:54.853 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:54.853 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:54.853 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:31:54.853 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.920250+0000", 2024-09-14T06:31:54.853 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.5807000000000001e-05 2024-09-14T06:31:54.853 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:54.853 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:54.853 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:31:54.853 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.920410+0000", 2024-09-14T06:31:54.854 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00015963300000000001 2024-09-14T06:31:54.854 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:54.854 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:31:54.854 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:54.854 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:54.854 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:54.854 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:44521 3.2 3:4190d882:::rbd_data.112dca2f6ae1.0000000000000002:17 [read 86016~4096 [fadvise_nocache]] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:31:54.854 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:43.855087+0000", 2024-09-14T06:31:54.854 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 130.94759507099999, 2024-09-14T06:31:54.854 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.065563264999999996, 2024-09-14T06:31:54.854 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:31:54.854 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:31:54.854 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:31:54.854 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:31:54.854 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:31:54.855 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:54.855 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 44521 2024-09-14T06:31:54.855 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:54.855 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:31:54.855 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:54.855 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:31:54.855 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855087+0000", 2024-09-14T06:31:54.855 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:54.855 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:54.855 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:54.855 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:31:54.855 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855087+0000", 2024-09-14T06:31:54.855 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:54.855 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:54.855 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:54.855 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:31:54.856 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855088+0000", 2024-09-14T06:31:54.856 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.3699999999999999e-07 2024-09-14T06:31:54.856 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:54.856 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:54.856 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:31:54.856 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855089+0000", 2024-09-14T06:31:54.856 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.9900000000000009e-07 2024-09-14T06:31:54.856 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:54.856 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:54.856 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:31:54.856 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855091+0000", 2024-09-14T06:31:54.856 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.8940000000000002e-06 2024-09-14T06:31:54.856 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:54.856 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:54.857 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:31:54.857 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855094+0000", 2024-09-14T06:31:54.857 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.351e-06 2024-09-14T06:31:54.857 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:54.857 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:54.857 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:31:54.857 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.920437+0000", 2024-09-14T06:31:54.857 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.065343221000000007 2024-09-14T06:31:54.857 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:54.857 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:54.857 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:31:54.857 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.920481+0000", 2024-09-14T06:31:54.857 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.4431000000000002e-05 2024-09-14T06:31:54.857 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:54.858 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:54.858 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:31:54.858 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.920651+0000", 2024-09-14T06:31:54.858 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00016913200000000001 2024-09-14T06:31:54.858 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:54.858 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:31:54.858 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:54.858 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:54.858 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:54.858 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:44522 3.2 3:4190d882:::rbd_data.112dca2f6ae1.0000000000000002:17 [read 110592~4096 [fadvise_nocache]] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:31:54.858 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:43.855098+0000", 2024-09-14T06:31:54.858 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 130.947584844, 2024-09-14T06:31:54.858 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.065827628999999999, 2024-09-14T06:31:54.858 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:31:54.859 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:31:54.859 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:31:54.859 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:31:54.859 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:31:54.859 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:54.859 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 44522 2024-09-14T06:31:54.859 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:54.859 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:31:54.859 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:54.859 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:31:54.859 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855098+0000", 2024-09-14T06:31:54.859 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:54.859 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:54.859 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:54.859 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:31:54.859 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855098+0000", 2024-09-14T06:31:54.860 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:54.860 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:54.860 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:54.860 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:31:54.860 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855098+0000", 2024-09-14T06:31:54.860 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.1e-07 2024-09-14T06:31:54.860 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:54.860 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:54.860 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:31:54.860 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855099+0000", 2024-09-14T06:31:54.860 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.012e-06 2024-09-14T06:31:54.860 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:54.860 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:54.861 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:31:54.861 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855102+0000", 2024-09-14T06:31:54.861 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.9380000000000001e-06 2024-09-14T06:31:54.861 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:54.861 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:54.861 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:31:54.861 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855108+0000", 2024-09-14T06:31:54.861 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.4339999999999997e-06 2024-09-14T06:31:54.861 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:54.861 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:54.861 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:31:54.861 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.920681+0000", 2024-09-14T06:31:54.861 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.065572943999999994 2024-09-14T06:31:54.861 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:54.862 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:54.862 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:31:54.862 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.920724+0000", 2024-09-14T06:31:54.862 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.2877000000000003e-05 2024-09-14T06:31:54.862 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:54.862 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:54.862 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:31:54.862 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.920925+0000", 2024-09-14T06:31:54.862 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00020121399999999999 2024-09-14T06:31:54.862 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:54.862 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:31:54.862 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:54.862 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:54.862 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:54.862 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:44523 3.2 3:4190d882:::rbd_data.112dca2f6ae1.0000000000000002:17 [read 135168~4096 [fadvise_nocache]] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:31:54.863 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:43.855111+0000", 2024-09-14T06:31:54.863 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 130.947571122, 2024-09-14T06:31:54.863 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.066028198999999996, 2024-09-14T06:31:54.863 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:31:54.863 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:31:54.863 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:31:54.863 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:31:54.863 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:31:54.863 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:54.863 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 44523 2024-09-14T06:31:54.863 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:54.863 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:31:54.863 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:54.863 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:31:54.863 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855111+0000", 2024-09-14T06:31:54.864 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:54.864 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:54.864 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:54.864 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:31:54.864 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855111+0000", 2024-09-14T06:31:54.864 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:54.864 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:54.864 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:54.864 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:31:54.864 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855112+0000", 2024-09-14T06:31:54.864 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.1199999999999999e-07 2024-09-14T06:31:54.864 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:54.864 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:54.864 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:31:54.864 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855115+0000", 2024-09-14T06:31:54.865 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.3380000000000001e-06 2024-09-14T06:31:54.865 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:54.865 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:54.865 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:31:54.865 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855118+0000", 2024-09-14T06:31:54.865 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.366e-06 2024-09-14T06:31:54.865 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:54.865 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:54.865 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:31:54.865 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855121+0000", 2024-09-14T06:31:54.865 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.8329999999999998e-06 2024-09-14T06:31:54.865 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:54.865 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:54.865 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:31:54.866 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.920956+0000", 2024-09-14T06:31:54.866 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.065835251999999997 2024-09-14T06:31:54.866 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:54.866 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:54.866 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:31:54.866 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.921001+0000", 2024-09-14T06:31:54.866 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.4778999999999999e-05 2024-09-14T06:31:54.866 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:54.866 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:54.866 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:31:54.866 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.921139+0000", 2024-09-14T06:31:54.866 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000138419 2024-09-14T06:31:54.866 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:54.866 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:31:54.866 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:54.867 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:54.867 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:54.867 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:44524 3.2 3:4190d882:::rbd_data.112dca2f6ae1.0000000000000002:17 [read 159744~4096 [fadvise_nocache]] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:31:54.867 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:43.855124+0000", 2024-09-14T06:31:54.867 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 130.94755814300001, 2024-09-14T06:31:54.867 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.066249261000000004, 2024-09-14T06:31:54.867 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:31:54.867 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:31:54.867 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:31:54.867 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:31:54.867 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:31:54.867 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:54.867 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 44524 2024-09-14T06:31:54.867 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:54.867 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:31:54.868 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:54.868 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:31:54.868 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855124+0000", 2024-09-14T06:31:54.868 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:54.868 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:54.868 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:54.868 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:31:54.868 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855124+0000", 2024-09-14T06:31:54.868 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:54.868 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:54.868 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:54.868 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:31:54.868 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861086+0000", 2024-09-14T06:31:54.868 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0059613829999999998 2024-09-14T06:31:54.868 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:54.869 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:54.869 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:31:54.869 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861100+0000", 2024-09-14T06:31:54.869 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.4644000000000001e-05 2024-09-14T06:31:54.869 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:54.869 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:54.869 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:31:54.869 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861109+0000", 2024-09-14T06:31:54.869 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.541e-06 2024-09-14T06:31:54.869 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:54.869 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:54.869 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:31:54.869 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861117+0000", 2024-09-14T06:31:54.869 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.1049999999999995e-06 2024-09-14T06:31:54.870 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:54.870 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:54.870 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:31:54.870 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.921169+0000", 2024-09-14T06:31:54.870 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.060052405000000003 2024-09-14T06:31:54.870 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:54.870 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:54.870 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:31:54.870 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.921213+0000", 2024-09-14T06:31:54.870 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.3565e-05 2024-09-14T06:31:54.870 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:54.870 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:54.870 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:31:54.870 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.921374+0000", 2024-09-14T06:31:54.871 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00016061800000000001 2024-09-14T06:31:54.871 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:54.871 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:31:54.871 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:54.871 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:54.871 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:54.871 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:44536 3.2 3:4190d882:::rbd_data.112dca2f6ae1.0000000000000002:17 [read 454656~4096 [fadvise_nocache]] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:31:54.871 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:43.861384+0000", 2024-09-14T06:31:54.871 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 130.94129858599999, 2024-09-14T06:31:54.871 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.063058528000000003, 2024-09-14T06:31:54.871 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:31:54.871 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:31:54.871 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:31:54.871 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:31:54.871 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:31:54.872 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:54.872 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 44536 2024-09-14T06:31:54.872 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:54.872 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:31:54.872 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:54.872 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:31:54.872 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861384+0000", 2024-09-14T06:31:54.872 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:54.872 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:54.872 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:54.872 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:31:54.872 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861384+0000", 2024-09-14T06:31:54.872 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:54.873 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:54.873 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:54.873 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:31:54.873 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861384+0000", 2024-09-14T06:31:54.873 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.4000000000000002e-07 2024-09-14T06:31:54.873 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:54.873 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:54.873 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:31:54.873 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861386+0000", 2024-09-14T06:31:54.873 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.891e-06 2024-09-14T06:31:54.873 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:54.873 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:54.873 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:31:54.873 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861392+0000", 2024-09-14T06:31:54.874 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.9939999999999997e-06 2024-09-14T06:31:54.874 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:54.874 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:54.874 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:31:54.874 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861398+0000", 2024-09-14T06:31:54.874 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.507e-06 2024-09-14T06:31:54.874 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:54.874 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:54.874 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:31:54.874 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.924149+0000", 2024-09-14T06:31:54.874 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.062751135999999999 2024-09-14T06:31:54.874 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:54.874 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:54.874 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:31:54.875 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.924226+0000", 2024-09-14T06:31:54.875 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.7517999999999999e-05 2024-09-14T06:31:54.875 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:54.875 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:54.875 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:31:54.875 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.924442+0000", 2024-09-14T06:31:54.875 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000216042 2024-09-14T06:31:54.875 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:54.875 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:31:54.875 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:54.875 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:54.875 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:54.875 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:44537 3.2 3:4190d882:::rbd_data.112dca2f6ae1.0000000000000002:17 [read 479232~4096 [fadvise_nocache]] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:31:54.875 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:43.861405+0000", 2024-09-14T06:31:54.875 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 130.94127753999999, 2024-09-14T06:31:54.876 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.063351795000000002, 2024-09-14T06:31:54.876 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:31:54.876 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:31:54.876 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:31:54.876 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:31:54.876 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:31:54.876 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:54.876 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 44537 2024-09-14T06:31:54.876 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:54.876 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:31:54.876 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:54.876 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:31:54.876 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861405+0000", 2024-09-14T06:31:54.876 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:54.877 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:54.877 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:54.877 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:31:54.877 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861405+0000", 2024-09-14T06:31:54.877 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:54.877 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:54.877 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:54.877 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:31:54.877 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861405+0000", 2024-09-14T06:31:54.877 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.9099999999999999e-07 2024-09-14T06:31:54.877 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:54.877 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:54.877 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:31:54.877 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861407+0000", 2024-09-14T06:31:54.878 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.075e-06 2024-09-14T06:31:54.878 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:54.878 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:54.878 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:31:54.878 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861414+0000", 2024-09-14T06:31:54.878 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.4659999999999996e-06 2024-09-14T06:31:54.878 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:54.878 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:54.878 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:31:54.878 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861418+0000", 2024-09-14T06:31:54.878 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.5519999999999998e-06 2024-09-14T06:31:54.878 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:54.878 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:54.878 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:31:54.879 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.924474+0000", 2024-09-14T06:31:54.879 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.063056012999999994 2024-09-14T06:31:54.879 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:54.879 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:54.879 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:31:54.879 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.924556+0000", 2024-09-14T06:31:54.879 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.1181999999999996e-05 2024-09-14T06:31:54.879 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:54.879 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:54.879 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:31:54.879 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.924757+0000", 2024-09-14T06:31:54.879 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00020111599999999999 2024-09-14T06:31:54.879 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:54.879 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:31:54.879 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:54.880 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:54.880 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:54.880 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:44538 3.2 3:4190d882:::rbd_data.112dca2f6ae1.0000000000000002:17 [read 503808~4096 [fadvise_nocache]] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:31:54.880 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:43.861425+0000", 2024-09-14T06:31:54.880 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 130.94125749899999, 2024-09-14T06:31:54.880 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.063627125000000007, 2024-09-14T06:31:54.880 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:31:54.880 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:31:54.880 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:31:54.880 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:31:54.880 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:31:54.880 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:54.880 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 44538 2024-09-14T06:31:54.880 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:54.880 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:31:54.881 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:54.881 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:31:54.881 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861425+0000", 2024-09-14T06:31:54.881 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:54.881 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:54.881 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:54.881 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:31:54.881 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861425+0000", 2024-09-14T06:31:54.881 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:54.881 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:54.881 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:54.881 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:31:54.881 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861425+0000", 2024-09-14T06:31:54.882 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.9999999999999998e-07 2024-09-14T06:31:54.882 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:54.882 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:54.882 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:31:54.882 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861427+0000", 2024-09-14T06:31:54.882 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.9829999999999999e-06 2024-09-14T06:31:54.882 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:54.882 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:54.882 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:31:54.882 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861433+0000", 2024-09-14T06:31:54.882 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.2889999999999997e-06 2024-09-14T06:31:54.882 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:54.882 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:54.882 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:31:54.882 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861454+0000", 2024-09-14T06:31:54.883 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.1844999999999998e-05 2024-09-14T06:31:54.883 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:54.883 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:54.883 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:31:54.883 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.924786+0000", 2024-09-14T06:31:54.883 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.063331497 2024-09-14T06:31:54.883 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:54.883 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:54.883 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:31:54.883 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.924864+0000", 2024-09-14T06:31:54.883 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.7744000000000006e-05 2024-09-14T06:31:54.883 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:54.883 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:54.883 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:31:54.883 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.925052+0000", 2024-09-14T06:31:54.883 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00018836699999999999 2024-09-14T06:31:54.884 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:54.884 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:31:54.884 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:54.884 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:54.884 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:54.884 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:44539 3.2 3:4190d882:::rbd_data.112dca2f6ae1.0000000000000002:17 [read 528384~4096 [fadvise_nocache]] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:31:54.884 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:43.861462+0000", 2024-09-14T06:31:54.884 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 130.94122017199999, 2024-09-14T06:31:54.884 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.063855705999999998, 2024-09-14T06:31:54.884 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:31:54.884 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:31:54.884 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:31:54.884 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:31:54.884 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:31:54.885 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:54.885 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 44539 2024-09-14T06:31:54.885 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:54.885 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:31:54.885 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:54.885 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:31:54.885 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861462+0000", 2024-09-14T06:31:54.885 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:54.885 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:54.885 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:54.885 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:31:54.885 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861462+0000", 2024-09-14T06:31:54.885 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:54.885 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:54.886 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:54.886 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:31:54.886 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861463+0000", 2024-09-14T06:31:54.886 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.5200000000000002e-07 2024-09-14T06:31:54.886 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:54.886 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:54.886 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:31:54.886 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861465+0000", 2024-09-14T06:31:54.886 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.5189999999999999e-06 2024-09-14T06:31:54.886 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:54.886 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:54.886 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:31:54.886 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861471+0000", 2024-09-14T06:31:54.886 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.6899999999999997e-06 2024-09-14T06:31:54.886 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:54.886 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:54.887 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:31:54.887 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861475+0000", 2024-09-14T06:31:54.887 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.5730000000000001e-06 2024-09-14T06:31:54.887 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:54.887 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:54.887 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:31:54.887 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.925085+0000", 2024-09-14T06:31:54.887 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.063609902999999995 2024-09-14T06:31:54.887 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:54.887 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:54.887 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:31:54.887 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.925145+0000", 2024-09-14T06:31:54.887 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.9383999999999999e-05 2024-09-14T06:31:54.887 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:54.887 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:54.888 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:31:54.888 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.925318+0000", 2024-09-14T06:31:54.888 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00017318499999999999 2024-09-14T06:31:54.888 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:54.888 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:31:54.888 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:54.888 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:54.888 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:54.888 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:44540 3.2 3:4190d882:::rbd_data.112dca2f6ae1.0000000000000002:17 [read 552960~4096 [fadvise_nocache]] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:31:54.888 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:43.861483+0000", 2024-09-14T06:31:54.888 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 130.94119974099999, 2024-09-14T06:31:54.888 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.064065681999999999, 2024-09-14T06:31:54.888 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:31:54.888 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:31:54.889 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:31:54.889 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:31:54.889 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:31:54.889 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:54.889 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 44540 2024-09-14T06:31:54.889 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:54.889 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:31:54.889 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:54.889 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:31:54.889 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861483+0000", 2024-09-14T06:31:54.889 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:54.889 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:54.889 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:54.889 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:31:54.889 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861483+0000", 2024-09-14T06:31:54.890 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:54.890 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:54.890 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:54.890 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:31:54.890 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.867079+0000", 2024-09-14T06:31:54.890 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0055966480000000001 2024-09-14T06:31:54.890 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:54.890 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:54.890 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:31:54.890 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.867086+0000", 2024-09-14T06:31:54.890 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.387e-06 2024-09-14T06:31:54.890 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:54.890 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:54.890 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:31:54.891 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.867094+0000", 2024-09-14T06:31:54.891 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.9419999999999995e-06 2024-09-14T06:31:54.891 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:54.891 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:54.891 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:31:54.891 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.867109+0000", 2024-09-14T06:31:54.891 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.5818999999999999e-05 2024-09-14T06:31:54.891 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:54.891 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:54.891 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:31:54.891 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.925354+0000", 2024-09-14T06:31:54.891 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.058244047 2024-09-14T06:31:54.891 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:54.891 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:54.891 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:31:54.892 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.925406+0000", 2024-09-14T06:31:54.892 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.2661999999999998e-05 2024-09-14T06:31:54.892 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:54.892 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:54.892 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:31:54.892 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.925548+0000", 2024-09-14T06:31:54.892 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000142177 2024-09-14T06:31:54.892 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:54.892 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:31:54.892 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:54.892 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:54.892 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:31:54.892 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-14T06:31:54.894 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-14T06:31:54.894 INFO:teuthology.orchestra.run.smithi094.stdout:osd.2: {} 2024-09-14T06:31:54.895 INFO:teuthology.orchestra.run.smithi094.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-14T06:31:54.895 INFO:teuthology.orchestra.run.smithi094.stdout:osd.3: {} 2024-09-14T06:31:54.895 INFO:teuthology.orchestra.run.smithi094.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-14T06:31:54.895 INFO:teuthology.orchestra.run.smithi094.stdout:osd.4: {} 2024-09-14T06:31:54.896 INFO:teuthology.orchestra.run.smithi094.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-14T06:31:54.896 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-14T06:31:54.897 INFO:teuthology.orchestra.run.smithi094.stdout: "success": "" 2024-09-14T06:31:54.897 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-14T06:31:54.897 INFO:teuthology.orchestra.run.smithi094.stdout:osd.5: {} 2024-09-14T06:31:54.897 INFO:teuthology.orchestra.run.smithi094.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-14T06:31:54.898 INFO:tasks.ceph.osd.6.smithi138.stderr:2024-09-14T06:31:54.893+0000 7f8cd8849640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33620) UID: 0 2024-09-14T06:31:54.900 INFO:teuthology.orchestra.run.smithi094.stdout:osd.6: {} 2024-09-14T06:31:54.900 INFO:teuthology.orchestra.run.smithi094.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-14T06:31:54.916 INFO:teuthology.orchestra.run.smithi094.stdout:osd.7: {} 2024-09-14T06:31:54.916 INFO:teuthology.orchestra.run.smithi094.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-14T06:31:54.995 INFO:tasks.ceph.osd.2.smithi094.stderr:2024-09-14T06:31:54.994+0000 7faf3269b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33649) UID: 0 2024-09-14T06:31:55.007 INFO:teuthology.orchestra.run.smithi138.stdout:ERROR: (22) Invalid argument 2024-09-14T06:31:55.008 INFO:teuthology.orchestra.run.smithi138.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:31:55.016 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-14T06:31:55.095 INFO:tasks.ceph.osd.3.smithi094.stderr:2024-09-14T06:31:55.094+0000 7feeeafe8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33655) UID: 0 2024-09-14T06:31:55.119 INFO:teuthology.orchestra.run.smithi138.stdout:ERROR: (22) Invalid argument 2024-09-14T06:31:55.119 INFO:teuthology.orchestra.run.smithi138.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:31:55.127 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-14T06:31:55.197 INFO:tasks.ceph.osd.4.smithi138.stderr:2024-09-14T06:31:55.196+0000 7f6c653c8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33615) UID: 0 2024-09-14T06:31:55.230 INFO:teuthology.orchestra.run.smithi138.stdout:ERROR: (22) Invalid argument 2024-09-14T06:31:55.230 INFO:teuthology.orchestra.run.smithi138.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:31:55.239 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-14T06:31:55.298 INFO:tasks.ceph.osd.3.smithi094.stderr:2024-09-14T06:31:55.297+0000 7feeeafe8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33655) UID: 0 2024-09-14T06:31:55.382 INFO:teuthology.orchestra.run.smithi138.stdout:ERROR: (22) Invalid argument 2024-09-14T06:31:55.382 INFO:teuthology.orchestra.run.smithi138.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:31:55.390 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-14T06:31:55.434 INFO:tasks.ceph.osd.6.smithi138.stderr:2024-09-14T06:31:55.433+0000 7f8cd8849640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33620) UID: 0 2024-09-14T06:31:55.493 INFO:teuthology.orchestra.run.smithi138.stdout:ERROR: (22) Invalid argument 2024-09-14T06:31:55.493 INFO:teuthology.orchestra.run.smithi138.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:31:55.499 INFO:tasks.ceph.osd.2.smithi094.stderr:2024-09-14T06:31:55.498+0000 7faf3269b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33649) UID: 0 2024-09-14T06:31:55.502 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-14T06:31:55.601 INFO:tasks.ceph.osd.5.smithi138.stderr:2024-09-14T06:31:55.599+0000 7fe13746f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33618) UID: 0 2024-09-14T06:31:55.646 INFO:teuthology.orchestra.run.smithi138.stdout:ERROR: (22) Invalid argument 2024-09-14T06:31:55.646 INFO:teuthology.orchestra.run.smithi138.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:31:55.654 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-14T06:31:55.738 INFO:tasks.ceph.osd.5.smithi138.stderr:2024-09-14T06:31:55.737+0000 7fe13746f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33618) UID: 0 2024-09-14T06:31:55.802 INFO:teuthology.orchestra.run.smithi138.stdout:ERROR: (22) Invalid argument 2024-09-14T06:31:55.802 INFO:teuthology.orchestra.run.smithi138.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:31:55.802 INFO:tasks.ceph.osd.4.smithi138.stderr:2024-09-14T06:31:55.801+0000 7f6c653c8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33615) UID: 0 2024-09-14T06:31:55.813 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-14T06:31:55.904 INFO:tasks.ceph.osd.2.smithi094.stderr:2024-09-14T06:31:55.903+0000 7faf3269b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33649) UID: 0 2024-09-14T06:31:55.913 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-14T06:31:55.913 INFO:teuthology.orchestra.run.smithi094.stdout:5694 writesame 0x948e8b5 thru 0x94942b9 data_size 0xb(0x5a05 bytes) 2024-09-14T06:31:55.922 INFO:teuthology.orchestra.run.smithi094.stdout:5695 punch from 0x4593daa to 0x459d730, (0x9986 bytes) 2024-09-14T06:31:55.922 INFO:teuthology.orchestra.run.smithi094.stdout:5696 read 0x3965804 thru 0x3968c92 (0x348f bytes) 2024-09-14T06:31:55.922 INFO:teuthology.orchestra.run.smithi094.stdout:5698 clone 43 order 23 su 4194304 sc 5 2024-09-14T06:31:55.958 INFO:teuthology.orchestra.run.smithi138.stdout:ERROR: (22) Invalid argument 2024-09-14T06:31:55.958 INFO:teuthology.orchestra.run.smithi138.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:31:55.966 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-14T06:31:56.010 INFO:tasks.ceph.osd.7.smithi138.stderr:2024-09-14T06:31:56.009+0000 7f3c6a21d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33623) UID: 0 2024-09-14T06:31:56.068 INFO:teuthology.orchestra.run.smithi138.stdout:ERROR: (22) Invalid argument 2024-09-14T06:31:56.069 INFO:teuthology.orchestra.run.smithi138.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:31:56.077 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-14T06:31:56.119 INFO:tasks.ceph.osd.5.smithi138.stderr:2024-09-14T06:31:56.117+0000 7fe13746f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33618) UID: 0 2024-09-14T06:31:56.206 INFO:tasks.ceph.osd.5.smithi138.stderr:2024-09-14T06:31:56.205+0000 7fe13746f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33618) UID: 0 2024-09-14T06:31:56.220 INFO:teuthology.orchestra.run.smithi138.stdout:ERROR: (22) Invalid argument 2024-09-14T06:31:56.221 INFO:teuthology.orchestra.run.smithi138.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:31:56.229 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-14T06:31:56.308 INFO:tasks.ceph.osd.2.smithi094.stderr:2024-09-14T06:31:56.307+0000 7faf3269b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33649) UID: 0 2024-09-14T06:31:56.372 INFO:teuthology.orchestra.run.smithi138.stdout:ERROR: (22) Invalid argument 2024-09-14T06:31:56.372 INFO:teuthology.orchestra.run.smithi138.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:31:56.380 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-14T06:31:56.408 INFO:tasks.ceph.osd.0.smithi094.stderr:2024-09-14T06:31:56.407+0000 7f07c9333640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33636) UID: 0 2024-09-14T06:31:56.483 INFO:teuthology.orchestra.run.smithi138.stdout:ERROR: (22) Invalid argument 2024-09-14T06:31:56.483 INFO:teuthology.orchestra.run.smithi138.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:31:56.491 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-14T06:31:56.535 INFO:tasks.ceph.osd.2.smithi094.stderr:2024-09-14T06:31:56.534+0000 7faf3269b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33649) UID: 0 2024-09-14T06:31:56.601 INFO:teuthology.orchestra.run.smithi094.stdout:ERROR: (22) Invalid argument 2024-09-14T06:31:56.601 INFO:teuthology.orchestra.run.smithi094.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:31:56.610 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-14T06:31:56.610 INFO:tasks.ceph.osd.4.smithi138.stderr:2024-09-14T06:31:56.609+0000 7f6c653c8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33615) UID: 0 2024-09-14T06:31:56.711 INFO:tasks.ceph.osd.5.smithi138.stderr:2024-09-14T06:31:56.710+0000 7fe13746f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33618) UID: 0 2024-09-14T06:31:56.755 INFO:teuthology.orchestra.run.smithi094.stdout:ERROR: (22) Invalid argument 2024-09-14T06:31:56.755 INFO:teuthology.orchestra.run.smithi094.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:31:56.765 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-14T06:31:56.768 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-14T06:31:56.811 INFO:tasks.ceph.osd.6.smithi138.stderr:2024-09-14T06:31:56.810+0000 7f8cd8849640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33620) UID: 0 2024-09-14T06:31:56.871 INFO:teuthology.orchestra.run.smithi094.stdout:ERROR: (22) Invalid argument 2024-09-14T06:31:56.871 INFO:teuthology.orchestra.run.smithi094.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:31:56.880 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-14T06:31:56.923 INFO:tasks.ceph.osd.2.smithi094.stderr:2024-09-14T06:31:56.922+0000 7faf3269b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33649) UID: 0 2024-09-14T06:31:56.927 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-14T06:31:56.986 INFO:teuthology.orchestra.run.smithi094.stdout:ERROR: (22) Invalid argument 2024-09-14T06:31:56.986 INFO:teuthology.orchestra.run.smithi094.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:31:56.996 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-14T06:31:57.014 INFO:tasks.ceph.osd.7.smithi138.stderr:2024-09-14T06:31:57.013+0000 7f3c6a21d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33623) UID: 0 2024-09-14T06:31:57.050 INFO:teuthology.orchestra.run.smithi094.stdout:osd.0: {} 2024-09-14T06:31:57.050 INFO:teuthology.orchestra.run.smithi094.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-14T06:31:57.069 INFO:teuthology.orchestra.run.smithi094.stdout:osd.1: {} 2024-09-14T06:31:57.069 INFO:teuthology.orchestra.run.smithi094.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-14T06:31:57.086 INFO:teuthology.orchestra.run.smithi094.stdout:osd.2: {} 2024-09-14T06:31:57.086 INFO:teuthology.orchestra.run.smithi094.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-14T06:31:57.102 INFO:teuthology.orchestra.run.smithi094.stdout:osd.3: {} 2024-09-14T06:31:57.102 INFO:teuthology.orchestra.run.smithi094.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-14T06:31:57.105 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-14T06:31:57.105 INFO:teuthology.orchestra.run.smithi094.stdout: "ops": [], 2024-09-14T06:31:57.105 INFO:teuthology.orchestra.run.smithi094.stdout: "complaint_time": 30, 2024-09-14T06:31:57.105 INFO:teuthology.orchestra.run.smithi094.stdout: "num_blocked_ops": 0 2024-09-14T06:31:57.105 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-14T06:31:57.114 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-14T06:31:57.115 INFO:tasks.ceph.osd.5.smithi138.stderr:2024-09-14T06:31:57.115+0000 7fe13746f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33618) UID: 0 2024-09-14T06:31:57.119 INFO:teuthology.orchestra.run.smithi094.stdout:osd.4: {} 2024-09-14T06:31:57.119 INFO:teuthology.orchestra.run.smithi094.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-14T06:31:57.136 INFO:teuthology.orchestra.run.smithi094.stdout:osd.5: {} 2024-09-14T06:31:57.136 INFO:teuthology.orchestra.run.smithi094.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-14T06:31:57.153 INFO:teuthology.orchestra.run.smithi094.stdout:osd.6: {} 2024-09-14T06:31:57.153 INFO:teuthology.orchestra.run.smithi094.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-14T06:31:57.170 INFO:teuthology.orchestra.run.smithi094.stdout:osd.7: {} 2024-09-14T06:31:57.170 INFO:teuthology.orchestra.run.smithi094.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-14T06:31:57.216 INFO:tasks.ceph.osd.4.smithi138.stderr:2024-09-14T06:31:57.216+0000 7f6c653c8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33615) UID: 0 2024-09-14T06:31:57.220 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-14T06:31:57.220 INFO:teuthology.orchestra.run.smithi094.stdout: "size": 20, 2024-09-14T06:31:57.220 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 600, 2024-09-14T06:31:57.221 INFO:teuthology.orchestra.run.smithi094.stdout: "ops": [ 2024-09-14T06:31:57.221 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:57.221 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:14918 3.0 3:05c39535:::rbd_data.112d92027590.0000000000000009:f [read 795648~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:31:57.221 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:26.083834+0000", 2024-09-14T06:31:57.221 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 151.13500710100001, 2024-09-14T06:31:57.221 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.041776287000000002, 2024-09-14T06:31:57.221 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:31:57.221 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:31:57.221 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:31:57.221 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:31:57.221 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:31:57.221 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:57.221 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 14918 2024-09-14T06:31:57.221 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:57.222 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:31:57.222 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:57.222 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:31:57.222 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.083834+0000", 2024-09-14T06:31:57.222 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:57.222 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:57.222 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:57.222 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:31:57.222 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.083834+0000", 2024-09-14T06:31:57.222 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:57.222 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:57.222 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:57.222 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:31:57.222 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.083834+0000", 2024-09-14T06:31:57.222 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.0100000000000001e-07 2024-09-14T06:31:57.223 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:57.223 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:57.223 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:31:57.223 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.083835+0000", 2024-09-14T06:31:57.223 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.3900000000000003e-07 2024-09-14T06:31:57.223 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:57.223 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:57.223 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:31:57.223 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.083838+0000", 2024-09-14T06:31:57.223 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.9830000000000001e-06 2024-09-14T06:31:57.223 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:57.223 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:57.223 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:31:57.223 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.083840+0000", 2024-09-14T06:31:57.224 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.4679999999999999e-06 2024-09-14T06:31:57.224 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:57.224 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:57.224 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:31:57.224 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.125455+0000", 2024-09-14T06:31:57.224 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.041614578999999999 2024-09-14T06:31:57.224 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:57.224 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:57.224 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:31:57.224 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.125496+0000", 2024-09-14T06:31:57.224 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.1047000000000002e-05 2024-09-14T06:31:57.224 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:57.224 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:57.224 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:31:57.225 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.125610+0000", 2024-09-14T06:31:57.225 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00011407 2024-09-14T06:31:57.225 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:57.225 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:31:57.225 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:57.225 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:57.225 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:57.225 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:14919 3.0 3:05c39535:::rbd_data.112d92027590.0000000000000009:f [read 805888~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:31:57.225 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:26.083843+0000", 2024-09-14T06:31:57.225 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 151.13499735299999, 2024-09-14T06:31:57.225 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.041992356000000002, 2024-09-14T06:31:57.225 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:31:57.225 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:31:57.225 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:31:57.226 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:31:57.226 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:31:57.226 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:57.226 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 14919 2024-09-14T06:31:57.226 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:57.226 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:31:57.226 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:57.226 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:31:57.226 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.083843+0000", 2024-09-14T06:31:57.226 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:57.226 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:57.226 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:57.226 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:31:57.226 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.083843+0000", 2024-09-14T06:31:57.227 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:57.227 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:57.227 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:57.227 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:31:57.227 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.083844+0000", 2024-09-14T06:31:57.227 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.3300000000000001e-07 2024-09-14T06:31:57.227 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:57.227 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:57.227 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:31:57.227 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.083845+0000", 2024-09-14T06:31:57.227 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.4729999999999999e-06 2024-09-14T06:31:57.227 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:57.227 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:57.227 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:31:57.228 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.083850+0000", 2024-09-14T06:31:57.228 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.0130000000000001e-06 2024-09-14T06:31:57.228 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:57.228 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:57.228 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:31:57.228 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.083854+0000", 2024-09-14T06:31:57.228 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.4759999999999998e-06 2024-09-14T06:31:57.228 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:57.228 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:57.228 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:31:57.228 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.125636+0000", 2024-09-14T06:31:57.228 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.041782683000000001 2024-09-14T06:31:57.228 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:57.228 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:57.229 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:31:57.229 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.125680+0000", 2024-09-14T06:31:57.229 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.3614e-05 2024-09-14T06:31:57.229 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:57.229 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:57.229 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:31:57.229 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.125836+0000", 2024-09-14T06:31:57.229 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000155864 2024-09-14T06:31:57.229 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:57.229 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:31:57.229 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:57.229 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:57.229 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:57.229 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:14920 3.0 3:05c39535:::rbd_data.112d92027590.0000000000000009:f [read 816128~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:31:57.229 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:26.083868+0000", 2024-09-14T06:31:57.229 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 151.13497269199999, 2024-09-14T06:31:57.230 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.042166771999999998, 2024-09-14T06:31:57.230 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:31:57.230 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:31:57.230 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:31:57.230 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:31:57.230 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:31:57.230 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:57.230 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 14920 2024-09-14T06:31:57.230 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:57.230 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:31:57.230 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:57.230 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:31:57.230 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.083868+0000", 2024-09-14T06:31:57.230 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:57.231 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:57.231 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:57.231 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:31:57.231 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.083868+0000", 2024-09-14T06:31:57.231 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:57.231 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:57.231 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:57.231 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:31:57.231 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.083869+0000", 2024-09-14T06:31:57.231 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.99e-07 2024-09-14T06:31:57.231 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:57.231 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:57.231 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:31:57.231 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.083870+0000", 2024-09-14T06:31:57.232 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.782e-06 2024-09-14T06:31:57.232 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:57.232 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:57.232 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:31:57.232 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.083876+0000", 2024-09-14T06:31:57.232 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.3809999999999998e-06 2024-09-14T06:31:57.232 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:57.232 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:57.232 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:31:57.232 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.083882+0000", 2024-09-14T06:31:57.232 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.9479999999999996e-06 2024-09-14T06:31:57.232 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:57.232 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:57.232 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:31:57.233 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.125878+0000", 2024-09-14T06:31:57.233 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.041996264999999998 2024-09-14T06:31:57.233 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:57.233 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:57.233 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:31:57.233 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.125929+0000", 2024-09-14T06:31:57.233 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.0498999999999999e-05 2024-09-14T06:31:57.233 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:57.233 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:57.233 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:31:57.233 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.126035+0000", 2024-09-14T06:31:57.233 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000106298 2024-09-14T06:31:57.233 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:57.233 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:31:57.234 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:57.234 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:57.234 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:57.234 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:14921 3.0 3:05c39535:::rbd_data.112d92027590.0000000000000009:f [read 826368~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:31:57.234 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:26.083889+0000", 2024-09-14T06:31:57.234 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 151.13495168200001, 2024-09-14T06:31:57.234 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.042435309999999997, 2024-09-14T06:31:57.234 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:31:57.234 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:31:57.234 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:31:57.234 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:31:57.234 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:31:57.234 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:57.234 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 14921 2024-09-14T06:31:57.235 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:57.235 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:31:57.235 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:57.235 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:31:57.235 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.083889+0000", 2024-09-14T06:31:57.235 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:57.235 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:57.235 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:57.235 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:31:57.235 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.083889+0000", 2024-09-14T06:31:57.235 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:57.235 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:57.235 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:57.235 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:31:57.236 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.086033+0000", 2024-09-14T06:31:57.236 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0021444179999999999 2024-09-14T06:31:57.236 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:57.236 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:57.236 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:31:57.236 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.086038+0000", 2024-09-14T06:31:57.236 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.0969999999999999e-06 2024-09-14T06:31:57.236 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:57.236 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:57.236 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:31:57.236 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.086046+0000", 2024-09-14T06:31:57.236 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.9440000000000005e-06 2024-09-14T06:31:57.236 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:57.236 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:57.237 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:31:57.237 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.086050+0000", 2024-09-14T06:31:57.237 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.69e-06 2024-09-14T06:31:57.237 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:57.237 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:57.237 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:31:57.237 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.126080+0000", 2024-09-14T06:31:57.237 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.040030237000000003 2024-09-14T06:31:57.237 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:57.237 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:57.237 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:31:57.237 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.126163+0000", 2024-09-14T06:31:57.237 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.2071999999999996e-05 2024-09-14T06:31:57.237 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:57.237 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:57.238 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:31:57.238 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.126324+0000", 2024-09-14T06:31:57.238 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00016185200000000001 2024-09-14T06:31:57.238 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:57.238 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:31:57.238 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:57.238 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:57.238 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:57.238 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:55627 3.0 3:06490baf:::rbd_data.112dca2f6ae1.0000000000000030:17 [read 2002944~4096 [fadvise_nocache]] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:31:57.238 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:44.599966+0000", 2024-09-14T06:31:57.238 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 132.618874734, 2024-09-14T06:31:57.238 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.041868546999999999, 2024-09-14T06:31:57.238 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:31:57.238 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:31:57.239 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:31:57.239 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:31:57.239 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:31:57.239 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:57.239 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 55627 2024-09-14T06:31:57.239 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:57.239 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:31:57.239 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:57.239 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:31:57.239 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.599966+0000", 2024-09-14T06:31:57.239 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:57.239 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:57.239 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:57.239 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:31:57.240 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.599966+0000", 2024-09-14T06:31:57.240 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:57.240 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:57.240 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:57.240 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:31:57.240 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.599966+0000", 2024-09-14T06:31:57.240 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.0900000000000001e-07 2024-09-14T06:31:57.240 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:57.240 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:57.240 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:31:57.240 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.599967+0000", 2024-09-14T06:31:57.240 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.64e-07 2024-09-14T06:31:57.240 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:57.240 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:57.241 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:31:57.241 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.599970+0000", 2024-09-14T06:31:57.241 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.9670000000000001e-06 2024-09-14T06:31:57.241 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:57.241 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:57.241 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:31:57.241 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.599973+0000", 2024-09-14T06:31:57.241 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.632e-06 2024-09-14T06:31:57.241 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:57.241 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:57.241 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:31:57.241 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.641665+0000", 2024-09-14T06:31:57.241 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.041692547000000003 2024-09-14T06:31:57.242 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:57.242 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:57.242 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:31:57.242 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.641740+0000", 2024-09-14T06:31:57.242 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.4559000000000002e-05 2024-09-14T06:31:57.242 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:57.242 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:57.242 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:31:57.242 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.641835+0000", 2024-09-14T06:31:57.242 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.4668999999999998e-05 2024-09-14T06:31:57.242 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:57.242 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:31:57.242 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:57.242 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:57.242 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:57.243 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:55628 3.0 3:06490baf:::rbd_data.112dca2f6ae1.0000000000000030:17 [read 2027520~4096 [fadvise_nocache]] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:31:57.243 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:44.599976+0000", 2024-09-14T06:31:57.243 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 132.61886494699999, 2024-09-14T06:31:57.243 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.041991621, 2024-09-14T06:31:57.243 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:31:57.243 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:31:57.243 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:31:57.243 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:31:57.243 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:31:57.243 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:57.243 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 55628 2024-09-14T06:31:57.243 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:57.243 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:31:57.243 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:57.243 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:31:57.244 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.599976+0000", 2024-09-14T06:31:57.244 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:57.244 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:57.244 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:57.244 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:31:57.244 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.599976+0000", 2024-09-14T06:31:57.244 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:57.244 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:57.244 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:57.244 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:31:57.244 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.599976+0000", 2024-09-14T06:31:57.244 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.1299999999999999e-07 2024-09-14T06:31:57.244 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:57.244 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:57.245 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:31:57.245 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.599977+0000", 2024-09-14T06:31:57.245 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.7999999999999993e-07 2024-09-14T06:31:57.245 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:57.245 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:57.245 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:31:57.245 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.599980+0000", 2024-09-14T06:31:57.245 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.01e-06 2024-09-14T06:31:57.245 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:57.245 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:57.245 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:31:57.245 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.599983+0000", 2024-09-14T06:31:57.245 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.5409999999999999e-06 2024-09-14T06:31:57.245 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:57.245 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:57.246 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:31:57.246 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.641869+0000", 2024-09-14T06:31:57.246 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.041886235000000001 2024-09-14T06:31:57.246 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:57.246 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:57.246 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:31:57.246 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.641913+0000", 2024-09-14T06:31:57.246 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.4357000000000002e-05 2024-09-14T06:31:57.246 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:57.246 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:57.246 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:31:57.246 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.641967+0000", 2024-09-14T06:31:57.246 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.4284999999999999e-05 2024-09-14T06:31:57.246 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:57.246 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:31:57.247 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:57.247 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:57.247 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:57.247 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:55629 3.0 3:06490baf:::rbd_data.112dca2f6ae1.0000000000000030:17 [read 2052096~4096 [fadvise_nocache]] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:31:57.247 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:44.599985+0000", 2024-09-14T06:31:57.247 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 132.61885524100001, 2024-09-14T06:31:57.247 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.042183375000000002, 2024-09-14T06:31:57.247 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:31:57.247 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:31:57.247 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:31:57.247 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:31:57.247 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:31:57.247 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:57.247 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 55629 2024-09-14T06:31:57.248 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:57.248 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:31:57.248 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:57.248 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:31:57.248 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.599985+0000", 2024-09-14T06:31:57.248 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:57.248 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:57.248 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:57.248 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:31:57.248 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.599985+0000", 2024-09-14T06:31:57.248 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:57.248 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:57.248 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:57.248 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:31:57.249 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.599986+0000", 2024-09-14T06:31:57.249 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.05e-07 2024-09-14T06:31:57.249 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:57.249 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:57.249 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:31:57.249 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.599987+0000", 2024-09-14T06:31:57.249 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.8400000000000002e-07 2024-09-14T06:31:57.249 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:57.249 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:57.249 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:31:57.249 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.599990+0000", 2024-09-14T06:31:57.249 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.5640000000000001e-06 2024-09-14T06:31:57.249 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:57.249 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:57.249 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:31:57.250 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.599993+0000", 2024-09-14T06:31:57.250 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.615e-06 2024-09-14T06:31:57.250 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:57.250 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:57.250 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:31:57.250 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.642003+0000", 2024-09-14T06:31:57.250 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.042010345999999997 2024-09-14T06:31:57.250 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:57.250 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:57.250 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:31:57.250 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.642080+0000", 2024-09-14T06:31:57.250 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.6538999999999993e-05 2024-09-14T06:31:57.250 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:57.250 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:57.251 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:31:57.251 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.642169+0000", 2024-09-14T06:31:57.251 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.9122000000000005e-05 2024-09-14T06:31:57.251 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:57.251 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:31:57.251 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:57.251 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:57.251 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:57.251 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:55630 3.0 3:06490baf:::rbd_data.112dca2f6ae1.0000000000000030:17 [read 2076672~4096 [fadvise_nocache]] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:31:57.251 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:44.599996+0000", 2024-09-14T06:31:57.251 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 132.618844763, 2024-09-14T06:31:57.251 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.042299615999999998, 2024-09-14T06:31:57.251 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:31:57.251 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:31:57.251 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:31:57.252 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:31:57.252 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:31:57.252 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:57.252 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 55630 2024-09-14T06:31:57.252 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:57.252 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:31:57.252 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:57.252 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:31:57.252 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.599996+0000", 2024-09-14T06:31:57.252 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:57.252 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:57.252 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:57.252 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:31:57.252 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.599996+0000", 2024-09-14T06:31:57.253 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:57.253 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:57.253 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:57.253 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:31:57.253 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.599996+0000", 2024-09-14T06:31:57.253 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.1500000000000001e-07 2024-09-14T06:31:57.253 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:57.253 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:57.253 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:31:57.253 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.599997+0000", 2024-09-14T06:31:57.253 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.33e-07 2024-09-14T06:31:57.253 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:57.253 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:57.253 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:31:57.254 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600000+0000", 2024-09-14T06:31:57.254 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.0240000000000002e-06 2024-09-14T06:31:57.254 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:57.254 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:57.254 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:31:57.254 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600003+0000", 2024-09-14T06:31:57.254 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.3640000000000002e-06 2024-09-14T06:31:57.254 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:57.254 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:57.254 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:31:57.254 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.642200+0000", 2024-09-14T06:31:57.254 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.042197571000000003 2024-09-14T06:31:57.254 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:57.254 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:57.255 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:31:57.255 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.642243+0000", 2024-09-14T06:31:57.255 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.2765000000000001e-05 2024-09-14T06:31:57.255 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:57.255 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:57.255 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:31:57.255 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.642296+0000", 2024-09-14T06:31:57.255 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.2744000000000002e-05 2024-09-14T06:31:57.255 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:57.255 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:31:57.255 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:57.255 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:57.255 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:57.255 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:55631 3.0 3:06490baf:::rbd_data.112dca2f6ae1.0000000000000030:17 [read 1048576~4096 [fadvise_nocache]] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:31:57.256 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:44.600006+0000", 2024-09-14T06:31:57.256 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 132.61883517499999, 2024-09-14T06:31:57.256 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.042487905999999999, 2024-09-14T06:31:57.256 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:31:57.256 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:31:57.256 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:31:57.256 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:31:57.256 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:31:57.256 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:57.256 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 55631 2024-09-14T06:31:57.256 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:57.256 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:31:57.256 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:57.256 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:31:57.256 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600006+0000", 2024-09-14T06:31:57.257 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:57.257 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:57.257 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:57.257 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:31:57.257 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600006+0000", 2024-09-14T06:31:57.257 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:57.257 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:57.257 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:57.257 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:31:57.257 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600006+0000", 2024-09-14T06:31:57.257 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.0800000000000001e-07 2024-09-14T06:31:57.257 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:57.257 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:57.258 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:31:57.258 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600007+0000", 2024-09-14T06:31:57.258 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.7199999999999997e-07 2024-09-14T06:31:57.258 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:57.258 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:57.258 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:31:57.258 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600010+0000", 2024-09-14T06:31:57.258 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.0280000000000001e-06 2024-09-14T06:31:57.258 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:57.258 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:57.258 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:31:57.258 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600012+0000", 2024-09-14T06:31:57.258 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.475e-06 2024-09-14T06:31:57.258 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:57.258 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:57.259 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:31:57.259 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.642329+0000", 2024-09-14T06:31:57.259 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.042317157000000001 2024-09-14T06:31:57.259 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:57.259 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:57.259 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:31:57.259 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.642403+0000", 2024-09-14T06:31:57.259 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.3941000000000002e-05 2024-09-14T06:31:57.259 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:57.259 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:57.259 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:31:57.259 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.642493+0000", 2024-09-14T06:31:57.259 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.0124999999999995e-05 2024-09-14T06:31:57.259 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:57.260 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:31:57.260 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:57.260 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:57.260 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:57.260 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:55632 3.0 3:06490baf:::rbd_data.112dca2f6ae1.0000000000000030:17 [read 1073152~4096 [fadvise_nocache]] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:31:57.260 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:44.600015+0000", 2024-09-14T06:31:57.260 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 132.61882542199999, 2024-09-14T06:31:57.260 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.042604922000000003, 2024-09-14T06:31:57.260 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:31:57.260 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:31:57.260 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:31:57.260 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:31:57.260 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:31:57.260 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:57.260 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 55632 2024-09-14T06:31:57.261 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:57.261 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:31:57.261 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:57.261 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:31:57.261 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600015+0000", 2024-09-14T06:31:57.261 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:57.261 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:57.261 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:57.261 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:31:57.261 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600015+0000", 2024-09-14T06:31:57.261 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:57.261 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:57.261 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:57.262 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:31:57.262 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600016+0000", 2024-09-14T06:31:57.262 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.1e-07 2024-09-14T06:31:57.262 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:57.262 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:57.262 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:31:57.262 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600017+0000", 2024-09-14T06:31:57.262 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.9900000000000009e-07 2024-09-14T06:31:57.262 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:57.262 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:57.262 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:31:57.262 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600020+0000", 2024-09-14T06:31:57.262 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.4170000000000001e-06 2024-09-14T06:31:57.262 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:57.262 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:57.263 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:31:57.263 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600023+0000", 2024-09-14T06:31:57.263 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.588e-06 2024-09-14T06:31:57.263 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:57.263 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:57.263 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:31:57.263 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.642526+0000", 2024-09-14T06:31:57.263 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.042503236999999999 2024-09-14T06:31:57.263 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:57.263 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:57.263 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:31:57.263 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.642568+0000", 2024-09-14T06:31:57.263 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.1925000000000003e-05 2024-09-14T06:31:57.263 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:57.264 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:57.264 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:31:57.264 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.642620+0000", 2024-09-14T06:31:57.264 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.2546000000000001e-05 2024-09-14T06:31:57.264 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:57.264 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:31:57.264 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:57.264 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:57.264 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:57.264 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:55633 3.0 3:06490baf:::rbd_data.112dca2f6ae1.0000000000000030:17 [read 1097728~4096 [fadvise_nocache]] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:31:57.264 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:44.600026+0000", 2024-09-14T06:31:57.264 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 132.61881515300001, 2024-09-14T06:31:57.264 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.042797474000000002, 2024-09-14T06:31:57.264 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:31:57.264 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:31:57.265 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:31:57.265 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:31:57.265 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:31:57.265 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:57.265 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 55633 2024-09-14T06:31:57.265 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:57.265 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:31:57.265 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:57.265 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:31:57.265 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600026+0000", 2024-09-14T06:31:57.265 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:57.265 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:57.265 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:57.265 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:31:57.266 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600026+0000", 2024-09-14T06:31:57.266 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:57.266 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:57.266 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:57.266 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:31:57.266 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600026+0000", 2024-09-14T06:31:57.266 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.3099999999999999e-07 2024-09-14T06:31:57.266 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:57.266 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:57.266 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:31:57.266 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600029+0000", 2024-09-14T06:31:57.266 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.304e-06 2024-09-14T06:31:57.266 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:57.266 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:57.267 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:31:57.267 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600034+0000", 2024-09-14T06:31:57.267 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.1429999999999997e-06 2024-09-14T06:31:57.267 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:57.267 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:57.267 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:31:57.267 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600038+0000", 2024-09-14T06:31:57.267 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.692e-06 2024-09-14T06:31:57.267 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:57.267 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:57.267 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:31:57.267 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.642654+0000", 2024-09-14T06:31:57.267 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.042615564000000002 2024-09-14T06:31:57.267 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:57.267 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:57.268 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:31:57.268 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.642728+0000", 2024-09-14T06:31:57.268 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.4018999999999997e-05 2024-09-14T06:31:57.268 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:57.268 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:57.268 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:31:57.268 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.642823+0000", 2024-09-14T06:31:57.268 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.5520999999999996e-05 2024-09-14T06:31:57.268 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:57.268 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:31:57.268 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:57.268 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:57.268 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:57.268 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:55634 3.0 3:06490baf:::rbd_data.112dca2f6ae1.0000000000000030:17 [read 1122304~4096 [fadvise_nocache]] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:31:57.269 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:44.600042+0000", 2024-09-14T06:31:57.269 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 132.61879867600001, 2024-09-14T06:31:57.269 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.042903729000000002, 2024-09-14T06:31:57.269 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:31:57.269 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:31:57.269 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:31:57.269 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:31:57.269 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:31:57.269 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:57.269 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 55634 2024-09-14T06:31:57.269 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:57.269 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:31:57.269 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:57.269 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:31:57.269 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600042+0000", 2024-09-14T06:31:57.270 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:57.270 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:57.270 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:57.270 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:31:57.270 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600042+0000", 2024-09-14T06:31:57.270 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:57.270 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:57.270 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:57.270 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:31:57.270 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600043+0000", 2024-09-14T06:31:57.270 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.7000000000000005e-07 2024-09-14T06:31:57.270 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:57.270 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:57.270 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:31:57.270 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600044+0000", 2024-09-14T06:31:57.271 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.742e-06 2024-09-14T06:31:57.271 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:57.271 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:57.271 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:31:57.271 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600050+0000", 2024-09-14T06:31:57.271 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.0800000000000002e-06 2024-09-14T06:31:57.271 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:57.271 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:57.271 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:31:57.271 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600055+0000", 2024-09-14T06:31:57.271 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.3889999999999998e-06 2024-09-14T06:31:57.271 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:57.271 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:57.271 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:31:57.272 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.642845+0000", 2024-09-14T06:31:57.272 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.042790040000000001 2024-09-14T06:31:57.272 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:57.272 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:57.272 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:31:57.272 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.642888+0000", 2024-09-14T06:31:57.272 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.2790000000000002e-05 2024-09-14T06:31:57.272 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:57.272 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:57.272 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:31:57.272 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.642946+0000", 2024-09-14T06:31:57.272 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.8118000000000002e-05 2024-09-14T06:31:57.272 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:57.272 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:31:57.272 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:57.273 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:57.273 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:57.273 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:55635 3.0 3:06490baf:::rbd_data.112dca2f6ae1.0000000000000030:17 [read 1146880~4096 [fadvise_nocache]] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:31:57.273 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:44.600061+0000", 2024-09-14T06:31:57.273 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 132.61878010500001, 2024-09-14T06:31:57.273 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.043092695, 2024-09-14T06:31:57.273 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:31:57.273 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:31:57.273 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:31:57.273 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:31:57.273 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:31:57.273 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:57.273 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 55635 2024-09-14T06:31:57.273 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:57.274 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:31:57.274 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:57.274 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:31:57.274 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600061+0000", 2024-09-14T06:31:57.274 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:57.274 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:57.274 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:57.274 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:31:57.274 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600061+0000", 2024-09-14T06:31:57.274 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:57.274 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:57.274 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:57.274 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:31:57.274 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600061+0000", 2024-09-14T06:31:57.275 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.3099999999999999e-07 2024-09-14T06:31:57.275 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:57.275 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:57.275 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:31:57.275 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600063+0000", 2024-09-14T06:31:57.275 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.6220000000000001e-06 2024-09-14T06:31:57.275 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:57.275 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:57.275 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:31:57.275 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600068+0000", 2024-09-14T06:31:57.275 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.4790000000000004e-06 2024-09-14T06:31:57.275 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:57.275 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:57.275 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:31:57.275 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600072+0000", 2024-09-14T06:31:57.276 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.3100000000000002e-06 2024-09-14T06:31:57.276 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:57.276 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:57.276 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:31:57.276 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.642987+0000", 2024-09-14T06:31:57.276 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.042914489 2024-09-14T06:31:57.276 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:57.276 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:57.276 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:31:57.276 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.643062+0000", 2024-09-14T06:31:57.276 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.5011000000000004e-05 2024-09-14T06:31:57.276 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:57.276 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:57.276 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:31:57.277 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.643153+0000", 2024-09-14T06:31:57.277 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.1452999999999993e-05 2024-09-14T06:31:57.277 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:57.277 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:31:57.277 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:57.277 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:57.277 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:57.277 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:55636 3.0 3:06490baf:::rbd_data.112dca2f6ae1.0000000000000030:17 [read 1171456~4096 [fadvise_nocache]] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:31:57.277 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:44.600078+0000", 2024-09-14T06:31:57.277 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 132.61876269300001, 2024-09-14T06:31:57.277 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.043207966, 2024-09-14T06:31:57.277 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:31:57.277 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:31:57.277 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:31:57.278 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:31:57.278 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:31:57.278 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:57.278 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 55636 2024-09-14T06:31:57.278 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:57.278 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:31:57.278 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:57.278 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:31:57.278 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600078+0000", 2024-09-14T06:31:57.278 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:57.278 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:57.278 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:57.278 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:31:57.278 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600078+0000", 2024-09-14T06:31:57.278 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:57.279 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:57.279 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:57.279 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:31:57.279 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600078+0000", 2024-09-14T06:31:57.279 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.34e-07 2024-09-14T06:31:57.279 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:57.279 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:57.279 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:31:57.279 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600080+0000", 2024-09-14T06:31:57.279 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.641e-06 2024-09-14T06:31:57.279 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:57.279 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:57.279 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:31:57.280 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600086+0000", 2024-09-14T06:31:57.280 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.3860000000000004e-06 2024-09-14T06:31:57.280 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:57.280 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:57.280 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:31:57.280 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600091+0000", 2024-09-14T06:31:57.280 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.5970000000000002e-06 2024-09-14T06:31:57.280 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:57.280 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:57.280 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:31:57.280 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.643184+0000", 2024-09-14T06:31:57.280 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.043092893 2024-09-14T06:31:57.280 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:57.280 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:57.281 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:31:57.281 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.643228+0000", 2024-09-14T06:31:57.281 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.4536999999999997e-05 2024-09-14T06:31:57.281 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:57.281 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:57.281 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:31:57.281 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.643286+0000", 2024-09-14T06:31:57.281 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.7578000000000003e-05 2024-09-14T06:31:57.281 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:57.281 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:31:57.281 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:57.281 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:57.281 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:57.281 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:55637 3.0 3:06490baf:::rbd_data.112dca2f6ae1.0000000000000030:17 [read 1196032~4096 [fadvise_nocache]] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:31:57.282 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:44.600097+0000", 2024-09-14T06:31:57.282 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 132.61874412899999, 2024-09-14T06:31:57.282 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.043399124999999997, 2024-09-14T06:31:57.282 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:31:57.282 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:31:57.282 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:31:57.282 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:31:57.282 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:31:57.282 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:57.282 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 55637 2024-09-14T06:31:57.282 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:57.282 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:31:57.282 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:57.282 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:31:57.283 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600097+0000", 2024-09-14T06:31:57.283 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:57.283 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:57.283 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:57.283 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:31:57.283 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600097+0000", 2024-09-14T06:31:57.283 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:57.283 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:57.283 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:57.283 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:31:57.283 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.604372+0000", 2024-09-14T06:31:57.283 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0042751489999999998 2024-09-14T06:31:57.283 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:57.283 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:57.283 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:31:57.284 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.604377+0000", 2024-09-14T06:31:57.284 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.7899999999999999e-06 2024-09-14T06:31:57.284 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:57.284 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:57.284 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:31:57.284 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.604389+0000", 2024-09-14T06:31:57.284 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.2197e-05 2024-09-14T06:31:57.284 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:57.284 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:57.284 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:31:57.284 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.604397+0000", 2024-09-14T06:31:57.284 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.8730000000000007e-06 2024-09-14T06:31:57.284 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:57.284 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:57.285 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:31:57.285 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.643324+0000", 2024-09-14T06:31:57.285 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.038926897000000002 2024-09-14T06:31:57.285 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:57.285 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:57.285 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:31:57.285 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.643402+0000", 2024-09-14T06:31:57.285 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.8063000000000001e-05 2024-09-14T06:31:57.285 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:57.285 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:57.285 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:31:57.285 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.643496+0000", 2024-09-14T06:31:57.285 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.4155999999999998e-05 2024-09-14T06:31:57.285 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:57.286 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:31:57.286 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:57.286 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:57.286 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:57.286 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:55657 3.0 3:06490baf:::rbd_data.112dca2f6ae1.0000000000000030:17 [read 1687552~4096 [fadvise_nocache]] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:31:57.286 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:44.604841+0000", 2024-09-14T06:31:57.286 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 132.61399940000001, 2024-09-14T06:31:57.286 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.041890087999999999, 2024-09-14T06:31:57.286 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:31:57.286 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:31:57.286 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:31:57.286 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:31:57.286 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:31:57.286 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:57.286 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 55657 2024-09-14T06:31:57.287 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:57.287 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:31:57.287 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:57.287 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:31:57.287 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.604841+0000", 2024-09-14T06:31:57.287 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:57.287 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:57.287 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:57.287 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:31:57.287 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.604841+0000", 2024-09-14T06:31:57.287 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:57.287 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:57.287 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:57.287 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:31:57.288 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.604842+0000", 2024-09-14T06:31:57.288 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.0900000000000001e-07 2024-09-14T06:31:57.288 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:57.288 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:57.288 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:31:57.288 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.604846+0000", 2024-09-14T06:31:57.288 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.5759999999999999e-06 2024-09-14T06:31:57.288 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:57.288 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:57.288 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:31:57.288 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.604850+0000", 2024-09-14T06:31:57.288 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.405e-06 2024-09-14T06:31:57.288 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:57.288 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:57.289 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:31:57.289 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.604852+0000", 2024-09-14T06:31:57.289 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.8590000000000001e-06 2024-09-14T06:31:57.289 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:57.289 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:57.289 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:31:57.289 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.646633+0000", 2024-09-14T06:31:57.289 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.041780772000000001 2024-09-14T06:31:57.289 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:57.289 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:57.289 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:31:57.289 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.646677+0000", 2024-09-14T06:31:57.289 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.4115e-05 2024-09-14T06:31:57.290 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:57.290 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:57.290 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:31:57.290 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.646731+0000", 2024-09-14T06:31:57.290 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.4151999999999999e-05 2024-09-14T06:31:57.290 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:57.290 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:31:57.290 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:57.290 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:57.290 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:57.290 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:55658 3.0 3:06490baf:::rbd_data.112dca2f6ae1.0000000000000030:17 [read 1712128~4096 [fadvise_nocache]] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:31:57.290 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:44.604856+0000", 2024-09-14T06:31:57.290 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 132.6139848, 2024-09-14T06:31:57.290 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.041986227000000001, 2024-09-14T06:31:57.290 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:31:57.291 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:31:57.291 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:31:57.291 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:31:57.291 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:31:57.291 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:57.291 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 55658 2024-09-14T06:31:57.291 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:57.291 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:31:57.291 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:57.291 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:31:57.291 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.604856+0000", 2024-09-14T06:31:57.291 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:57.291 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:57.291 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:57.291 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:31:57.292 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.604856+0000", 2024-09-14T06:31:57.292 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:57.292 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:57.292 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:57.292 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:31:57.292 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.604856+0000", 2024-09-14T06:31:57.292 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.23e-07 2024-09-14T06:31:57.292 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:57.292 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:57.292 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:31:57.292 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.604857+0000", 2024-09-14T06:31:57.292 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.9999999999999995e-07 2024-09-14T06:31:57.292 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:57.292 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:57.293 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:31:57.293 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.604861+0000", 2024-09-14T06:31:57.293 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.9890000000000003e-06 2024-09-14T06:31:57.293 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:57.293 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:57.293 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:31:57.293 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.604864+0000", 2024-09-14T06:31:57.293 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.7429999999999998e-06 2024-09-14T06:31:57.293 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:57.293 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:57.293 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:31:57.293 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.646744+0000", 2024-09-14T06:31:57.293 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.041880325000000003 2024-09-14T06:31:57.293 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:57.294 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:57.294 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:31:57.294 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.646784+0000", 2024-09-14T06:31:57.294 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.9604000000000002e-05 2024-09-14T06:31:57.294 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:57.294 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:57.294 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:31:57.294 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.646842+0000", 2024-09-14T06:31:57.294 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.8343e-05 2024-09-14T06:31:57.294 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:57.294 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:31:57.294 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:57.294 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:57.294 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:57.294 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:55659 3.0 3:06490baf:::rbd_data.112dca2f6ae1.0000000000000030:17 [read 1736704~4096 [fadvise_nocache]] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:31:57.295 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:44.604867+0000", 2024-09-14T06:31:57.295 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 132.61397364699999, 2024-09-14T06:31:57.295 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.042097082000000001, 2024-09-14T06:31:57.295 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:31:57.295 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:31:57.295 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:31:57.295 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:31:57.295 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:31:57.295 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:57.295 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 55659 2024-09-14T06:31:57.295 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:57.295 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:31:57.295 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:57.295 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:31:57.296 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.604867+0000", 2024-09-14T06:31:57.296 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:57.296 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:57.296 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:57.296 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:31:57.296 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.604867+0000", 2024-09-14T06:31:57.296 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:57.296 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:57.296 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:57.296 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:31:57.296 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.604867+0000", 2024-09-14T06:31:57.296 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.2000000000000001e-07 2024-09-14T06:31:57.296 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:57.297 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:57.297 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:31:57.297 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.604868+0000", 2024-09-14T06:31:57.297 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.04e-06 2024-09-14T06:31:57.297 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:57.297 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:57.297 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:31:57.297 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.604871+0000", 2024-09-14T06:31:57.297 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.9689999999999999e-06 2024-09-14T06:31:57.297 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:57.297 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:57.297 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:31:57.297 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.604874+0000", 2024-09-14T06:31:57.297 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.5069999999999999e-06 2024-09-14T06:31:57.297 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:57.298 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:57.298 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:31:57.298 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.646874+0000", 2024-09-14T06:31:57.298 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.042000513000000003 2024-09-14T06:31:57.298 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:57.298 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:57.298 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:31:57.298 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.646914+0000", 2024-09-14T06:31:57.298 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.9443999999999997e-05 2024-09-14T06:31:57.298 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:57.298 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:57.298 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:31:57.298 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.646964+0000", 2024-09-14T06:31:57.298 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.0389000000000002e-05 2024-09-14T06:31:57.298 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:57.299 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:31:57.299 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:57.299 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:57.299 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:57.299 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:55660 3.0 3:06490baf:::rbd_data.112dca2f6ae1.0000000000000030:17 [read 1761280~4096 [fadvise_nocache]] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:31:57.299 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:44.604877+0000", 2024-09-14T06:31:57.299 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 132.61396389800001, 2024-09-14T06:31:57.299 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.042196349000000001, 2024-09-14T06:31:57.299 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:31:57.299 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:31:57.299 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:31:57.299 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:31:57.299 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:31:57.299 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:57.300 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 55660 2024-09-14T06:31:57.300 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:57.300 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:31:57.300 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:57.300 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:31:57.300 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.604877+0000", 2024-09-14T06:31:57.300 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:57.300 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:57.300 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:57.300 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:31:57.300 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.604877+0000", 2024-09-14T06:31:57.300 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:57.300 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:57.300 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:57.301 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:31:57.301 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.605810+0000", 2024-09-14T06:31:57.301 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00093336899999999997 2024-09-14T06:31:57.301 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:57.301 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:57.301 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:31:57.301 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.605815+0000", 2024-09-14T06:31:57.301 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.6140000000000002e-06 2024-09-14T06:31:57.301 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:57.301 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:57.301 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:31:57.301 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.605825+0000", 2024-09-14T06:31:57.301 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.8889999999999997e-06 2024-09-14T06:31:57.301 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:57.302 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:57.302 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:31:57.302 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.605831+0000", 2024-09-14T06:31:57.302 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.9800000000000003e-06 2024-09-14T06:31:57.302 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:57.302 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:57.302 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:31:57.302 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.646989+0000", 2024-09-14T06:31:57.302 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.041157878000000002 2024-09-14T06:31:57.302 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:57.302 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:57.302 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:31:57.302 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.647026+0000", 2024-09-14T06:31:57.302 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.7305000000000003e-05 2024-09-14T06:31:57.302 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:57.303 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:57.303 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:31:57.303 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.647073+0000", 2024-09-14T06:31:57.303 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.7314000000000001e-05 2024-09-14T06:31:57.303 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:57.303 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:31:57.303 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:57.303 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:57.303 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:57.303 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:55666 3.0 3:06490baf:::rbd_data.112dca2f6ae1.0000000000000030:17 [read 1908736~4096 [fadvise_nocache]] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:31:57.303 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:44.605898+0000", 2024-09-14T06:31:57.303 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 132.61294315200001, 2024-09-14T06:31:57.303 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.041862887000000001, 2024-09-14T06:31:57.303 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:31:57.304 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:31:57.304 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:31:57.304 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:31:57.304 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:31:57.304 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:57.304 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 55666 2024-09-14T06:31:57.304 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:57.304 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:31:57.304 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:57.304 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:31:57.304 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.605898+0000", 2024-09-14T06:31:57.304 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:57.304 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:57.304 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:57.305 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:31:57.305 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.605898+0000", 2024-09-14T06:31:57.305 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:57.305 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:57.305 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:57.305 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:31:57.305 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.611874+0000", 2024-09-14T06:31:57.305 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0059764049999999997 2024-09-14T06:31:57.305 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:57.305 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:57.305 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:31:57.305 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.611879+0000", 2024-09-14T06:31:57.305 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.8749999999999999e-06 2024-09-14T06:31:57.305 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:57.306 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:57.306 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:31:57.306 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.611887+0000", 2024-09-14T06:31:57.306 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.4109999999999996e-06 2024-09-14T06:31:57.306 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:57.306 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:57.306 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:31:57.306 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.611893+0000", 2024-09-14T06:31:57.306 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.2959999999999998e-06 2024-09-14T06:31:57.306 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:57.306 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:57.306 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:31:57.306 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.647674+0000", 2024-09-14T06:31:57.306 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.035781281999999998 2024-09-14T06:31:57.306 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:57.307 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:57.307 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:31:57.307 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.647711+0000", 2024-09-14T06:31:57.307 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.7434000000000001e-05 2024-09-14T06:31:57.307 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:57.307 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:57.307 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:31:57.307 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.647760+0000", 2024-09-14T06:31:57.307 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.9184000000000002e-05 2024-09-14T06:31:57.307 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:57.307 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:31:57.307 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:57.307 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:57.307 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:31:57.307 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-14T06:31:57.309 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-14T06:31:57.350 INFO:tasks.ceph.osd.0.smithi094.stderr:2024-09-14T06:31:57.349+0000 7f07c9333640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33636) UID: 0 2024-09-14T06:31:57.418 INFO:tasks.ceph.osd.6.smithi138.stderr:2024-09-14T06:31:57.417+0000 7f8cd8849640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33620) UID: 0 2024-09-14T06:31:57.454 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-14T06:31:57.454 INFO:teuthology.orchestra.run.smithi094.stdout: "ops": [], 2024-09-14T06:31:57.454 INFO:teuthology.orchestra.run.smithi094.stdout: "num_ops": 0 2024-09-14T06:31:57.454 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-14T06:31:57.462 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-14T06:31:57.519 INFO:tasks.ceph.osd.0.smithi094.stderr:2024-09-14T06:31:57.518+0000 7f07c9333640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33636) UID: 0 2024-09-14T06:31:57.565 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-14T06:31:57.565 INFO:teuthology.orchestra.run.smithi094.stdout: "ops": [], 2024-09-14T06:31:57.565 INFO:teuthology.orchestra.run.smithi094.stdout: "complaint_time": 30, 2024-09-14T06:31:57.565 INFO:teuthology.orchestra.run.smithi094.stdout: "num_blocked_ops": 0 2024-09-14T06:31:57.565 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-14T06:31:57.574 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-14T06:31:57.620 INFO:tasks.ceph.osd.5.smithi138.stderr:2024-09-14T06:31:57.619+0000 7fe13746f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33618) UID: 0 2024-09-14T06:31:57.718 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-14T06:31:57.718 INFO:teuthology.orchestra.run.smithi094.stdout: "size": 20, 2024-09-14T06:31:57.718 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 600, 2024-09-14T06:31:57.718 INFO:teuthology.orchestra.run.smithi094.stdout: "ops": [ 2024-09-14T06:31:57.718 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:57.719 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4397.0:45722 3.9 e64/23 3:91fed098:::rbd_data.112d5a737e72.0000000000000001:head v 64'61, mlcod=62'59)", 2024-09-14T06:31:57.719 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:43.864269+0000", 2024-09-14T06:31:57.719 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 133.85212154000001, 2024-09-14T06:31:57.719 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.003055224, 2024-09-14T06:31:57.719 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:31:57.719 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:31:57.719 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:31:57.719 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:31:57.719 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:57.719 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:31:57.720 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.864269+0000", 2024-09-14T06:31:57.720 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:57.720 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:57.720 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:57.720 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:31:57.720 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.864269+0000", 2024-09-14T06:31:57.720 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:57.720 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:57.720 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:57.720 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:31:57.720 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.864269+0000", 2024-09-14T06:31:57.720 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.41e-07 2024-09-14T06:31:57.721 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:57.721 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:57.721 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:31:57.721 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.865490+0000", 2024-09-14T06:31:57.721 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.001220552 2024-09-14T06:31:57.721 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:57.721 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:57.721 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:31:57.721 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.865506+0000", 2024-09-14T06:31:57.721 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.6263999999999999e-05 2024-09-14T06:31:57.721 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:57.721 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:57.721 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:31:57.721 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.865516+0000", 2024-09-14T06:31:57.722 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.0091999999999999e-05 2024-09-14T06:31:57.722 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:57.722 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:57.722 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:31:57.722 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.865583+0000", 2024-09-14T06:31:57.722 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.6676999999999999e-05 2024-09-14T06:31:57.722 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:57.722 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:57.722 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:31:57.722 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.865610+0000", 2024-09-14T06:31:57.722 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.6529999999999998e-05 2024-09-14T06:31:57.722 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:57.722 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:57.722 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-14T06:31:57.723 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.867281+0000", 2024-09-14T06:31:57.723 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.001671203 2024-09-14T06:31:57.723 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:57.723 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:57.723 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:31:57.723 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.867324+0000", 2024-09-14T06:31:57.723 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.3665000000000003e-05 2024-09-14T06:31:57.723 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:57.723 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:31:57.723 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:57.723 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:57.723 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:57.723 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4397.0:74241 3.0 e73/23 3:06e93fb9:::rbd_data.112d80ca42af.0000000000000000:head v 73'102, mlcod=71'100)", 2024-09-14T06:31:57.723 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:53.876730+0000", 2024-09-14T06:31:57.724 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 123.83966044, 2024-09-14T06:31:57.724 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0079470329999999992, 2024-09-14T06:31:57.724 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:31:57.724 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:31:57.724 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:31:57.724 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:31:57.724 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:57.724 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:31:57.724 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:53.876730+0000", 2024-09-14T06:31:57.724 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:57.724 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:57.724 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:57.724 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:31:57.725 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:53.876730+0000", 2024-09-14T06:31:57.725 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:57.725 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:57.725 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:57.725 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:31:57.725 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:53.876731+0000", 2024-09-14T06:31:57.725 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.0500000000000004e-07 2024-09-14T06:31:57.725 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:57.725 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:57.725 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:31:57.725 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:53.879616+0000", 2024-09-14T06:31:57.725 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0028856680000000001 2024-09-14T06:31:57.725 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:57.725 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:57.726 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:31:57.726 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:53.879643+0000", 2024-09-14T06:31:57.726 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.6120999999999999e-05 2024-09-14T06:31:57.728 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:57.728 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:57.728 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:31:57.728 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:53.879654+0000", 2024-09-14T06:31:57.728 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.1586999999999999e-05 2024-09-14T06:31:57.728 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:57.728 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:57.728 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:31:57.728 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:53.879756+0000", 2024-09-14T06:31:57.728 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00010223899999999999 2024-09-14T06:31:57.728 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:57.728 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:57.729 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:31:57.729 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:53.879783+0000", 2024-09-14T06:31:57.729 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.6913000000000001e-05 2024-09-14T06:31:57.729 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:57.729 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:57.729 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-14T06:31:57.729 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:53.884594+0000", 2024-09-14T06:31:57.729 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0048110729999999999 2024-09-14T06:31:57.729 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:57.729 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:57.729 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:31:57.729 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:53.884677+0000", 2024-09-14T06:31:57.729 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.2927000000000001e-05 2024-09-14T06:31:57.730 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:57.730 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:31:57.730 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:57.730 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:57.730 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:57.730 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4397.0:74243 3.9 e73/23 3:94977f81:::rbd_data.112d80ca42af.0000000000000002:head v 73'94, mlcod=71'92)", 2024-09-14T06:31:57.730 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:53.885156+0000", 2024-09-14T06:31:57.730 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 123.831234522, 2024-09-14T06:31:57.730 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0096009860000000006, 2024-09-14T06:31:57.730 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:31:57.730 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:31:57.730 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:31:57.730 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:31:57.730 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:57.730 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:31:57.731 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:53.885156+0000", 2024-09-14T06:31:57.731 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:57.731 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:57.731 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:57.731 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:31:57.731 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:53.885156+0000", 2024-09-14T06:31:57.731 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:57.731 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:57.731 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:57.731 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:31:57.731 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:53.885156+0000", 2024-09-14T06:31:57.731 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.23e-07 2024-09-14T06:31:57.731 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:57.732 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:57.732 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:31:57.732 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:53.889694+0000", 2024-09-14T06:31:57.732 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0045378709999999997 2024-09-14T06:31:57.732 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:57.732 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:57.732 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:31:57.732 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:53.889720+0000", 2024-09-14T06:31:57.732 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.5259e-05 2024-09-14T06:31:57.732 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:57.732 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:57.732 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:31:57.732 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:53.889731+0000", 2024-09-14T06:31:57.732 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.1739000000000001e-05 2024-09-14T06:31:57.733 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:57.733 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:57.733 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:31:57.733 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:53.889822+0000", 2024-09-14T06:31:57.733 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.1200999999999996e-05 2024-09-14T06:31:57.733 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:57.733 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:57.733 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:31:57.733 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:53.889850+0000", 2024-09-14T06:31:57.733 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.7084999999999999e-05 2024-09-14T06:31:57.733 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:57.733 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:57.733 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-14T06:31:57.733 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:53.894703+0000", 2024-09-14T06:31:57.734 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0048532690000000003 2024-09-14T06:31:57.734 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:57.734 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:57.734 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:31:57.734 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:53.894757+0000", 2024-09-14T06:31:57.734 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.4339000000000003e-05 2024-09-14T06:31:57.734 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:57.734 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:31:57.734 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:57.734 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:57.734 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:57.734 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4397.0:74249 3.0 e73/23 3:009b0225:::rbd_data.112d80ca42af.0000000000000009:head v 73'103, mlcod=71'101)", 2024-09-14T06:31:57.734 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:53.893219+0000", 2024-09-14T06:31:57.734 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 123.823172044, 2024-09-14T06:31:57.734 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0065381980000000003, 2024-09-14T06:31:57.735 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:31:57.735 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:31:57.735 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:31:57.735 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:31:57.735 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:57.735 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:31:57.735 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:53.893219+0000", 2024-09-14T06:31:57.735 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:57.735 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:57.735 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:57.735 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:31:57.735 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:53.893219+0000", 2024-09-14T06:31:57.735 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:57.736 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:57.736 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:57.736 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:31:57.736 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:53.893221+0000", 2024-09-14T06:31:57.736 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.4329999999999998e-06 2024-09-14T06:31:57.736 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:57.736 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:57.736 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:31:57.736 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:53.895968+0000", 2024-09-14T06:31:57.736 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.002746746 2024-09-14T06:31:57.736 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:57.736 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:57.736 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:31:57.737 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:53.895988+0000", 2024-09-14T06:31:57.737 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.0227000000000001e-05 2024-09-14T06:31:57.737 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:57.737 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:57.737 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:31:57.737 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:53.896000+0000", 2024-09-14T06:31:57.737 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.2359e-05 2024-09-14T06:31:57.737 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:57.737 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:57.737 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:31:57.737 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:53.896084+0000", 2024-09-14T06:31:57.737 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.3659999999999995e-05 2024-09-14T06:31:57.737 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:57.737 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:57.738 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:31:57.738 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:53.896109+0000", 2024-09-14T06:31:57.738 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.5259999999999999e-05 2024-09-14T06:31:57.738 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:57.738 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:57.738 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-14T06:31:57.738 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:53.899704+0000", 2024-09-14T06:31:57.738 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0035942209999999999 2024-09-14T06:31:57.738 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:57.738 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:57.738 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:31:57.738 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:53.899757+0000", 2024-09-14T06:31:57.738 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.3291999999999997e-05 2024-09-14T06:31:57.739 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:57.739 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:31:57.739 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:57.739 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:57.739 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:57.739 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4397.0:78263 3.0 e77/23 3:00899b22:::rbd_data.112ddfbec471.0000000000000016:head v 77'104, mlcod=73'102)", 2024-09-14T06:31:57.739 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:57.634578+0000", 2024-09-14T06:31:57.739 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 120.081812992, 2024-09-14T06:31:57.739 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.002372454, 2024-09-14T06:31:57.739 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:31:57.739 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:31:57.739 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:31:57.739 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:31:57.739 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:57.739 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:31:57.740 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:57.634578+0000", 2024-09-14T06:31:57.740 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:57.740 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:57.740 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:57.740 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:31:57.740 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:57.634578+0000", 2024-09-14T06:31:57.740 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:57.740 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:57.740 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:57.740 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:31:57.740 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:57.634578+0000", 2024-09-14T06:31:57.740 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.5699999999999998e-07 2024-09-14T06:31:57.740 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:57.741 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:57.741 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:31:57.741 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:57.635347+0000", 2024-09-14T06:31:57.741 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00076848300000000003 2024-09-14T06:31:57.741 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:57.741 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:57.741 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:31:57.741 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:57.635361+0000", 2024-09-14T06:31:57.741 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.4238e-05 2024-09-14T06:31:57.741 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:57.741 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:57.741 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:31:57.741 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:57.635371+0000", 2024-09-14T06:31:57.741 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.0087e-05 2024-09-14T06:31:57.742 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:57.742 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:57.742 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:31:57.742 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:57.635439+0000", 2024-09-14T06:31:57.742 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.7953000000000005e-05 2024-09-14T06:31:57.742 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:57.742 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:57.742 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:31:57.742 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:57.635464+0000", 2024-09-14T06:31:57.742 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.5062000000000002e-05 2024-09-14T06:31:57.742 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:57.742 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:57.742 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-14T06:31:57.743 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:57.636914+0000", 2024-09-14T06:31:57.743 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.001449625 2024-09-14T06:31:57.743 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:57.743 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:57.743 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:31:57.743 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:57.636950+0000", 2024-09-14T06:31:57.743 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.6548999999999999e-05 2024-09-14T06:31:57.743 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:57.743 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:31:57.743 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:57.743 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:57.743 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:57.743 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4397.0:78604 3.9 e77/23 3:94c62a0e:::rbd_data.112ddfbec471.000000000000002c:head v 77'96, mlcod=73'94)", 2024-09-14T06:31:57.743 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:57.677568+0000", 2024-09-14T06:31:57.744 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 120.038822604, 2024-09-14T06:31:57.744 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.002564846, 2024-09-14T06:31:57.744 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:31:57.744 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:31:57.744 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:31:57.744 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:31:57.744 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:57.744 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:31:57.744 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:57.677568+0000", 2024-09-14T06:31:57.744 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:57.744 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:57.744 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:57.744 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:31:57.744 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:57.677568+0000", 2024-09-14T06:31:57.745 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:57.745 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:57.745 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:57.745 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:31:57.745 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:57.677568+0000", 2024-09-14T06:31:57.745 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.8900000000000001e-07 2024-09-14T06:31:57.745 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:57.745 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:57.745 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:31:57.745 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:57.678488+0000", 2024-09-14T06:31:57.745 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00091963899999999998 2024-09-14T06:31:57.745 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:57.745 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:57.745 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:31:57.746 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:57.678500+0000", 2024-09-14T06:31:57.746 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.1902000000000001e-05 2024-09-14T06:31:57.746 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:57.746 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:57.746 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:31:57.746 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:57.678507+0000", 2024-09-14T06:31:57.746 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.7739999999999999e-06 2024-09-14T06:31:57.746 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:57.746 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:57.746 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:31:57.746 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:57.678586+0000", 2024-09-14T06:31:57.746 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.9103000000000005e-05 2024-09-14T06:31:57.746 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:57.746 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:57.747 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:31:57.747 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:57.678614+0000", 2024-09-14T06:31:57.747 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.7912e-05 2024-09-14T06:31:57.747 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:57.747 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:57.747 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-14T06:31:57.747 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:57.680075+0000", 2024-09-14T06:31:57.747 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0014613370000000001 2024-09-14T06:31:57.747 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:57.747 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:57.747 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:31:57.747 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:57.680133+0000", 2024-09-14T06:31:57.747 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.7890000000000003e-05 2024-09-14T06:31:57.747 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:57.747 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:31:57.748 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:57.748 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:57.748 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:57.748 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4397.0:149885 3.9 e122/23 3:951acef3:::rbd_data.112d1bb7b269.0000000000000005:head v 122'176, mlcod=122'172)", 2024-09-14T06:31:57.748 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:30:44.739092+0000", 2024-09-14T06:31:57.748 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 72.977298293999993, 2024-09-14T06:31:57.748 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0028236659999999998, 2024-09-14T06:31:57.748 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:31:57.748 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:31:57.748 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:31:57.748 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:31:57.748 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:57.748 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:31:57.748 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.739092+0000", 2024-09-14T06:31:57.749 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:57.749 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:57.749 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:57.749 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:31:57.749 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.739092+0000", 2024-09-14T06:31:57.749 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:57.749 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:57.749 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:57.749 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:31:57.749 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.739093+0000", 2024-09-14T06:31:57.749 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.0109999999999999e-06 2024-09-14T06:31:57.749 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:57.749 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:57.749 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:31:57.750 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.740189+0000", 2024-09-14T06:31:57.750 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00109527 2024-09-14T06:31:57.750 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:57.750 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:57.750 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:31:57.750 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.740199+0000", 2024-09-14T06:31:57.750 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.0458999999999999e-05 2024-09-14T06:31:57.750 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:57.750 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:57.750 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:31:57.750 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.740207+0000", 2024-09-14T06:31:57.750 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.4499999999999998e-06 2024-09-14T06:31:57.750 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:57.751 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:57.751 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:31:57.751 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.740295+0000", 2024-09-14T06:31:57.751 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.8740999999999996e-05 2024-09-14T06:31:57.751 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:57.751 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:57.751 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:31:57.751 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.740317+0000", 2024-09-14T06:31:57.751 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.1523e-05 2024-09-14T06:31:57.751 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:57.751 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:57.751 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-14T06:31:57.751 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.741865+0000", 2024-09-14T06:31:57.751 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.001548315 2024-09-14T06:31:57.751 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:57.752 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:57.752 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:31:57.752 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.741916+0000", 2024-09-14T06:31:57.752 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.0896999999999997e-05 2024-09-14T06:31:57.752 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:57.752 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:31:57.752 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:57.752 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:57.752 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:57.752 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4397.0:149933 3.4 e122/23 3:234f5229:::rbd_data.112d1bb7b269.0000000000000009:head v 122'221, mlcod=120'218)", 2024-09-14T06:31:57.752 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:30:44.743849+0000", 2024-09-14T06:31:57.752 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 72.972541876999998, 2024-09-14T06:31:57.752 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0031479519999999999, 2024-09-14T06:31:57.752 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:31:57.753 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:31:57.753 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:31:57.753 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:31:57.753 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:57.753 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:31:57.753 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.743849+0000", 2024-09-14T06:31:57.753 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:57.753 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:57.753 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:57.753 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:31:57.753 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.743849+0000", 2024-09-14T06:31:57.753 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:57.753 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:57.753 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:57.754 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:31:57.754 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.743850+0000", 2024-09-14T06:31:57.754 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.279e-06 2024-09-14T06:31:57.754 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:57.754 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:57.754 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:31:57.754 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.745151+0000", 2024-09-14T06:31:57.754 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.001300495 2024-09-14T06:31:57.754 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:57.754 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:57.754 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:31:57.754 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.745167+0000", 2024-09-14T06:31:57.754 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.6478e-05 2024-09-14T06:31:57.754 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:57.755 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:57.755 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:31:57.755 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.745175+0000", 2024-09-14T06:31:57.755 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.2679999999999994e-06 2024-09-14T06:31:57.755 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:57.755 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:57.755 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:31:57.755 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.745226+0000", 2024-09-14T06:31:57.755 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.0192999999999999e-05 2024-09-14T06:31:57.755 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:57.755 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:57.755 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:31:57.755 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.745247+0000", 2024-09-14T06:31:57.755 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.1767e-05 2024-09-14T06:31:57.756 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:57.756 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:57.756 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-14T06:31:57.756 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.746952+0000", 2024-09-14T06:31:57.756 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0017042769999999999 2024-09-14T06:31:57.756 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:57.756 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:57.756 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:31:57.756 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.746997+0000", 2024-09-14T06:31:57.756 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.5194999999999997e-05 2024-09-14T06:31:57.756 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:57.756 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:31:57.756 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:57.757 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:57.757 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:57.757 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4397.0:150494 3.9 e122/23 3:955152a4:::rbd_object_map.112d1bb7b269.0000000000000038:head v 122'194, mlcod=122'190)", 2024-09-14T06:31:57.757 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:30:44.781585+0000", 2024-09-14T06:31:57.757 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 72.934805984999997, 2024-09-14T06:31:57.757 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0026957499999999998, 2024-09-14T06:31:57.757 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:31:57.757 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:31:57.757 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:31:57.757 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:31:57.757 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:57.757 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:31:57.757 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.781585+0000", 2024-09-14T06:31:57.757 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:57.758 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:57.758 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:57.758 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:31:57.758 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.781585+0000", 2024-09-14T06:31:57.758 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:57.758 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:57.758 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:57.758 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:31:57.758 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.781585+0000", 2024-09-14T06:31:57.758 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.5000000000000002e-07 2024-09-14T06:31:57.758 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:57.758 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:57.758 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:31:57.758 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.781594+0000", 2024-09-14T06:31:57.759 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.4640000000000006e-06 2024-09-14T06:31:57.759 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:57.759 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:57.759 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:31:57.759 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.781604+0000", 2024-09-14T06:31:57.759 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.0071999999999999e-05 2024-09-14T06:31:57.759 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:57.759 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:57.759 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:31:57.759 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.781610+0000", 2024-09-14T06:31:57.759 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.3609999999999997e-06 2024-09-14T06:31:57.759 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:57.759 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:57.759 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:31:57.760 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.781676+0000", 2024-09-14T06:31:57.760 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.5963000000000005e-05 2024-09-14T06:31:57.760 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:57.760 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:57.760 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:31:57.760 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.781696+0000", 2024-09-14T06:31:57.760 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.9278000000000001e-05 2024-09-14T06:31:57.760 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:57.760 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:57.760 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-14T06:31:57.760 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.784231+0000", 2024-09-14T06:31:57.760 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.002535093 2024-09-14T06:31:57.760 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:57.760 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:57.761 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:31:57.761 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.784280+0000", 2024-09-14T06:31:57.761 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.9869e-05 2024-09-14T06:31:57.761 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:57.761 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:31:57.761 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:57.761 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:57.761 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:57.761 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4397.0:150898 3.9 e122/23 3:97c98f90:::rbd_data.112d1bb7b269.0000000000000025:head v 122'206, mlcod=122'203)", 2024-09-14T06:31:57.761 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:30:44.807232+0000", 2024-09-14T06:31:57.761 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 72.909159067999994, 2024-09-14T06:31:57.761 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0022297889999999998, 2024-09-14T06:31:57.761 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:31:57.761 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:31:57.761 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:31:57.762 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:31:57.762 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:57.762 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:31:57.762 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.807232+0000", 2024-09-14T06:31:57.762 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:57.762 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:57.762 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:57.762 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:31:57.762 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.807232+0000", 2024-09-14T06:31:57.762 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:57.762 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:57.762 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:57.762 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:31:57.762 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.807233+0000", 2024-09-14T06:31:57.763 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.5710000000000001e-06 2024-09-14T06:31:57.763 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:57.763 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:57.763 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:31:57.763 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.807944+0000", 2024-09-14T06:31:57.763 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00071056299999999999 2024-09-14T06:31:57.763 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:57.763 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:57.763 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:31:57.763 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.807961+0000", 2024-09-14T06:31:57.763 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.7689e-05 2024-09-14T06:31:57.763 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:57.763 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:57.763 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:31:57.764 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.807971+0000", 2024-09-14T06:31:57.764 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.2e-06 2024-09-14T06:31:57.764 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:57.764 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:57.764 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:31:57.764 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.808050+0000", 2024-09-14T06:31:57.764 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.9308999999999996e-05 2024-09-14T06:31:57.764 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:57.764 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:57.764 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:31:57.764 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.808074+0000", 2024-09-14T06:31:57.764 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.4147000000000001e-05 2024-09-14T06:31:57.764 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:57.765 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:57.765 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-14T06:31:57.765 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.809415+0000", 2024-09-14T06:31:57.765 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.001341257 2024-09-14T06:31:57.765 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:57.765 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:57.765 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:31:57.765 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.809461+0000", 2024-09-14T06:31:57.765 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.6053000000000001e-05 2024-09-14T06:31:57.765 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:57.765 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:31:57.765 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:57.765 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:57.765 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:57.766 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4397.0:202551 3.9 e166/23 3:965c09b6:::rbd_data.112d3bd84cb0.000000000000000a:head v 166'416, mlcod=164'414)", 2024-09-14T06:31:57.766 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:31:30.657011+0000", 2024-09-14T06:31:57.766 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 27.059380035, 2024-09-14T06:31:57.766 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.002509262, 2024-09-14T06:31:57.766 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:31:57.766 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:31:57.766 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:31:57.766 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:31:57.766 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:57.766 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:31:57.766 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:30.657011+0000", 2024-09-14T06:31:57.766 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:57.766 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:57.766 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:57.767 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:31:57.767 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:30.657011+0000", 2024-09-14T06:31:57.767 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:57.767 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:57.767 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:57.767 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:31:57.767 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:30.657011+0000", 2024-09-14T06:31:57.767 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.08e-07 2024-09-14T06:31:57.767 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:57.767 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:57.767 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:31:57.767 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:30.658467+0000", 2024-09-14T06:31:57.767 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.001455708 2024-09-14T06:31:57.767 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:57.768 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:57.768 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:31:57.768 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:30.658484+0000", 2024-09-14T06:31:57.768 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.6844e-05 2024-09-14T06:31:57.768 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:57.768 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:57.768 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:31:57.768 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:30.658493+0000", 2024-09-14T06:31:57.768 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.2790000000000004e-06 2024-09-14T06:31:57.768 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:57.768 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:57.768 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:31:57.768 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:30.658559+0000", 2024-09-14T06:31:57.768 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.6459999999999997e-05 2024-09-14T06:31:57.769 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:57.769 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:57.769 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:31:57.769 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:30.658574+0000", 2024-09-14T06:31:57.769 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.5048000000000001e-05 2024-09-14T06:31:57.769 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:57.769 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:57.769 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-14T06:31:57.769 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:30.659473+0000", 2024-09-14T06:31:57.769 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00089842800000000005 2024-09-14T06:31:57.769 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:57.769 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:57.769 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:31:57.770 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:30.659520+0000", 2024-09-14T06:31:57.770 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.7086999999999998e-05 2024-09-14T06:31:57.770 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:57.770 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:31:57.770 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:57.770 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:57.770 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:57.770 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4397.0:202665 3.0 e166/23 3:00eeadfb:::rbd_data.112d3bd84cb0.000000000000000e:head v 166'920, mlcod=163'918)", 2024-09-14T06:31:57.770 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:31:30.662434+0000", 2024-09-14T06:31:57.770 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 27.053956245999998, 2024-09-14T06:31:57.770 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.002522121, 2024-09-14T06:31:57.770 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:31:57.770 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:31:57.770 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:31:57.771 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:31:57.771 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:57.771 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:31:57.771 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:30.662434+0000", 2024-09-14T06:31:57.771 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:57.771 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:57.771 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:57.771 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:31:57.771 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:30.662434+0000", 2024-09-14T06:31:57.771 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:57.771 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:57.771 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:57.771 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:31:57.771 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:30.662436+0000", 2024-09-14T06:31:57.771 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.257e-06 2024-09-14T06:31:57.772 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:57.772 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:57.772 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:31:57.772 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:30.662527+0000", 2024-09-14T06:31:57.772 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.1644000000000006e-05 2024-09-14T06:31:57.772 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:57.772 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:57.772 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:31:57.772 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:30.662546+0000", 2024-09-14T06:31:57.772 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.8998e-05 2024-09-14T06:31:57.772 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:57.772 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:57.772 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:31:57.772 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:30.662556+0000", 2024-09-14T06:31:57.773 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.9809999999999998e-06 2024-09-14T06:31:57.773 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:57.773 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:57.773 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:31:57.773 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:30.662621+0000", 2024-09-14T06:31:57.773 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.4597000000000005e-05 2024-09-14T06:31:57.773 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:57.773 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:57.773 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:31:57.773 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:30.662643+0000", 2024-09-14T06:31:57.773 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.1608e-05 2024-09-14T06:31:57.773 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:57.773 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:57.774 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-14T06:31:57.774 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:30.664909+0000", 2024-09-14T06:31:57.774 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0022668689999999999 2024-09-14T06:31:57.774 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:57.774 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:57.774 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:31:57.774 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:30.664957+0000", 2024-09-14T06:31:57.774 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.7166999999999998e-05 2024-09-14T06:31:57.774 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:57.774 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:31:57.774 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:57.775 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:57.775 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:57.775 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4397.0:204589 3.4 e166/23 3:234552f0:::rbd_data.112d3bd84cb0.000000000000003f:head v 166'499, mlcod=165'497)", 2024-09-14T06:31:57.775 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:31:30.772894+0000", 2024-09-14T06:31:57.775 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 26.94349639, 2024-09-14T06:31:57.775 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0034564650000000001, 2024-09-14T06:31:57.775 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:31:57.775 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:31:57.775 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:31:57.775 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:31:57.775 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:57.775 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:31:57.775 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:30.772894+0000", 2024-09-14T06:31:57.775 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:57.776 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:57.776 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:57.776 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:31:57.776 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:30.772894+0000", 2024-09-14T06:31:57.776 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:57.776 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:57.776 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:57.776 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:31:57.776 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:30.772896+0000", 2024-09-14T06:31:57.776 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.5379999999999999e-06 2024-09-14T06:31:57.776 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:57.776 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:57.776 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:31:57.776 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:30.773042+0000", 2024-09-14T06:31:57.777 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000146107 2024-09-14T06:31:57.777 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:57.777 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:57.777 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:31:57.777 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:30.773061+0000", 2024-09-14T06:31:57.777 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.8862999999999999e-05 2024-09-14T06:31:57.777 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:57.777 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:57.777 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:31:57.777 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:30.773071+0000", 2024-09-14T06:31:57.777 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.0227000000000001e-05 2024-09-14T06:31:57.777 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:57.777 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:57.778 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:31:57.778 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:30.773149+0000", 2024-09-14T06:31:57.778 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.8071999999999994e-05 2024-09-14T06:31:57.778 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:57.778 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:57.778 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:31:57.778 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:30.773171+0000", 2024-09-14T06:31:57.778 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.1699999999999999e-05 2024-09-14T06:31:57.778 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:57.778 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:57.778 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-14T06:31:57.778 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:30.776285+0000", 2024-09-14T06:31:57.778 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0031142050000000001 2024-09-14T06:31:57.778 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:57.779 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:57.779 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:31:57.779 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:30.776351+0000", 2024-09-14T06:31:57.779 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.5753000000000006e-05 2024-09-14T06:31:57.779 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:57.779 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:31:57.779 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:57.779 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:57.779 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:57.779 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4397.0:204598 3.4 e166/23 3:245d069e:::rbd_data.112d3bd84cb0.0000000000000041:head v 166'500, mlcod=165'497)", 2024-09-14T06:31:57.779 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:31:30.775247+0000", 2024-09-14T06:31:57.779 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 26.941143267000001, 2024-09-14T06:31:57.779 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00318235, 2024-09-14T06:31:57.779 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:31:57.779 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:31:57.780 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:31:57.780 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:31:57.780 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:57.780 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:31:57.780 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:30.775247+0000", 2024-09-14T06:31:57.780 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:57.780 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:57.780 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:57.780 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:31:57.780 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:30.775247+0000", 2024-09-14T06:31:57.780 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:57.780 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:57.780 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:57.780 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:31:57.781 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:30.775249+0000", 2024-09-14T06:31:57.781 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.265e-06 2024-09-14T06:31:57.781 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:57.781 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:57.781 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:31:57.781 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:30.775340+0000", 2024-09-14T06:31:57.781 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.1131999999999994e-05 2024-09-14T06:31:57.781 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:57.781 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:57.781 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:31:57.781 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:30.775352+0000", 2024-09-14T06:31:57.781 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.2294e-05 2024-09-14T06:31:57.781 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:57.781 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:57.782 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:31:57.782 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:30.775361+0000", 2024-09-14T06:31:57.782 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.6950000000000006e-06 2024-09-14T06:31:57.782 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:57.782 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:57.782 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:31:57.782 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:30.775439+0000", 2024-09-14T06:31:57.782 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.7910000000000005e-05 2024-09-14T06:31:57.782 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:57.782 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:57.782 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:31:57.782 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:30.775460+0000", 2024-09-14T06:31:57.782 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.0900999999999999e-05 2024-09-14T06:31:57.782 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:57.783 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:57.783 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-14T06:31:57.783 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:30.778385+0000", 2024-09-14T06:31:57.783 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.002925539 2024-09-14T06:31:57.783 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:57.783 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:57.783 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:31:57.783 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:30.778430+0000", 2024-09-14T06:31:57.783 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.4613999999999997e-05 2024-09-14T06:31:57.783 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:57.783 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:31:57.783 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:57.783 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:57.783 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:57.784 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4397.0:204975 3.4 e166/23 3:20972d18:::rbd_data.112d3bd84cb0.0000000000000046:head v 166'501, mlcod=166'499)", 2024-09-14T06:31:57.784 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:31:30.787253+0000", 2024-09-14T06:31:57.784 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 26.929137237999999, 2024-09-14T06:31:57.784 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.002248318, 2024-09-14T06:31:57.784 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:31:57.784 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:31:57.784 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:31:57.784 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:31:57.784 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:57.784 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:31:57.784 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:30.787253+0000", 2024-09-14T06:31:57.784 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:57.784 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:57.785 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:57.785 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:31:57.785 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:30.787253+0000", 2024-09-14T06:31:57.785 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:57.785 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:57.785 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:57.785 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:31:57.785 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:30.787255+0000", 2024-09-14T06:31:57.785 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.271e-06 2024-09-14T06:31:57.785 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:57.785 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:57.785 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:31:57.785 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:30.787442+0000", 2024-09-14T06:31:57.785 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00018744000000000001 2024-09-14T06:31:57.786 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:57.786 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:57.786 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:31:57.786 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:30.787461+0000", 2024-09-14T06:31:57.786 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.9054000000000001e-05 2024-09-14T06:31:57.786 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:57.786 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:57.786 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:31:57.786 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:30.787470+0000", 2024-09-14T06:31:57.786 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.0929999999999992e-06 2024-09-14T06:31:57.786 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:57.786 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:57.786 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:31:57.786 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:30.788004+0000", 2024-09-14T06:31:57.787 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00053359399999999997 2024-09-14T06:31:57.787 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:57.787 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:57.787 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:31:57.787 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:30.788034+0000", 2024-09-14T06:31:57.787 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.9635e-05 2024-09-14T06:31:57.787 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:57.787 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:57.787 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-14T06:31:57.787 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:30.789457+0000", 2024-09-14T06:31:57.787 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.001423487 2024-09-14T06:31:57.787 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:57.787 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:57.787 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:31:57.787 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:30.789502+0000", 2024-09-14T06:31:57.788 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.4743999999999997e-05 2024-09-14T06:31:57.788 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:57.788 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:31:57.788 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:57.788 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:57.788 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:57.788 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4397.0:212188 3.4 e166/23 3:23871bc7:::rbd_data.112d3bd84cb0.00000000000000fa:head v 166'508, mlcod=166'506)", 2024-09-14T06:31:57.788 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:31:31.391415+0000", 2024-09-14T06:31:57.788 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 26.324975655999999, 2024-09-14T06:31:57.788 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0022039049999999999, 2024-09-14T06:31:57.788 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:31:57.788 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:31:57.788 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:31:57.788 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:31:57.789 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:57.789 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:31:57.789 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:31.391415+0000", 2024-09-14T06:31:57.789 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:57.789 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:57.789 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:57.789 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:31:57.789 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:31.391415+0000", 2024-09-14T06:31:57.789 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:57.789 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:57.789 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:57.789 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:31:57.789 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:31.391416+0000", 2024-09-14T06:31:57.790 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.44e-06 2024-09-14T06:31:57.790 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:57.790 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:57.790 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:31:57.790 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:31.391554+0000", 2024-09-14T06:31:57.790 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000137119 2024-09-14T06:31:57.790 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:57.790 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:57.790 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:31:57.790 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:31.391570+0000", 2024-09-14T06:31:57.790 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.6827e-05 2024-09-14T06:31:57.790 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:57.790 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:57.790 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:31:57.791 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:31.391580+0000", 2024-09-14T06:31:57.791 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.3139999999999993e-06 2024-09-14T06:31:57.791 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:57.791 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:57.791 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:31:57.791 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:31.391664+0000", 2024-09-14T06:31:57.791 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.3757999999999993e-05 2024-09-14T06:31:57.791 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:57.791 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:57.791 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:31:57.791 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:31.391692+0000", 2024-09-14T06:31:57.791 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.8898e-05 2024-09-14T06:31:57.791 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:57.791 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:57.792 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-14T06:31:57.792 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:31.393584+0000", 2024-09-14T06:31:57.792 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.001891752 2024-09-14T06:31:57.792 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:57.792 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:57.792 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:31:57.792 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:31.393619+0000", 2024-09-14T06:31:57.792 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.4796999999999999e-05 2024-09-14T06:31:57.792 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:57.792 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:31:57.792 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:57.792 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:57.792 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:57.792 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "MOSDPGPush(3.5 169/168 [PushOp(3:a7c12b08:::rbd_data.112ddfbec471.000000000000000e:head, version: 77'100, data_included: [0~1048576], data_size: 1048576, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(3:a7c12b08:::rbd_data.112ddfbec471.000000000000000e:head@77'100, size: 1048576, copy_subset: [(0, 1048576)], clone_subset: {}, snapset: 0=[]:{}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 1048576, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2024-09-14T06:31:57.793 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:31:34.540707+0000", 2024-09-14T06:31:57.793 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 23.175683890999998, 2024-09-14T06:31:57.793 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0023637369999999999, 2024-09-14T06:31:57.793 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:31:57.793 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:31:57.793 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:31:57.793 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:31:57.793 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:57.793 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:31:57.793 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:34.540707+0000", 2024-09-14T06:31:57.793 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:57.793 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:57.793 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:57.793 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:31:57.794 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:34.540707+0000", 2024-09-14T06:31:57.794 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:57.794 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:57.794 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:57.794 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:31:57.794 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:34.540708+0000", 2024-09-14T06:31:57.794 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.7199999999999998e-07 2024-09-14T06:31:57.794 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:57.794 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:57.794 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:31:57.794 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:34.542150+0000", 2024-09-14T06:31:57.794 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0014423229999999999 2024-09-14T06:31:57.794 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:57.794 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:57.795 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:31:57.795 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:34.542169+0000", 2024-09-14T06:31:57.795 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.8756000000000001e-05 2024-09-14T06:31:57.795 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:57.795 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:57.795 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:31:57.795 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:34.542180+0000", 2024-09-14T06:31:57.795 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.1051e-05 2024-09-14T06:31:57.795 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:57.795 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:57.795 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:31:57.795 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:34.542275+0000", 2024-09-14T06:31:57.795 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.5122000000000002e-05 2024-09-14T06:31:57.795 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:57.796 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:57.796 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:31:57.796 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:34.542284+0000", 2024-09-14T06:31:57.796 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.9190000000000005e-06 2024-09-14T06:31:57.796 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:57.796 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:57.796 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:31:57.796 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:34.543071+0000", 2024-09-14T06:31:57.796 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00078679399999999995 2024-09-14T06:31:57.796 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:57.796 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:31:57.796 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:57.796 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:57.796 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:57.797 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "MOSDPGPush(3.5 169/168 [PushOp(3:a5c8b3f2:::rbd_data.112ddfbec471.0000000000000018:head, version: 77'101, data_included: [0~1048576], data_size: 1048576, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(3:a5c8b3f2:::rbd_data.112ddfbec471.0000000000000018:head@77'101, size: 1048576, copy_subset: [(0, 1048576)], clone_subset: {}, snapset: 0=[]:{}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 1048576, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2024-09-14T06:31:57.797 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:31:34.545533+0000", 2024-09-14T06:31:57.797 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 23.170858003999999, 2024-09-14T06:31:57.797 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0021867010000000001, 2024-09-14T06:31:57.797 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:31:57.797 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:31:57.797 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:31:57.797 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:31:57.797 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:57.797 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:31:57.797 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:34.545533+0000", 2024-09-14T06:31:57.797 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:57.797 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:57.797 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:57.798 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:31:57.798 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:34.545533+0000", 2024-09-14T06:31:57.798 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:57.798 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:57.798 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:57.798 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:31:57.798 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:34.545533+0000", 2024-09-14T06:31:57.798 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.8100000000000002e-07 2024-09-14T06:31:57.798 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:57.798 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:57.798 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:31:57.798 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:34.546784+0000", 2024-09-14T06:31:57.798 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.001250747 2024-09-14T06:31:57.798 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:57.799 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:57.799 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:31:57.799 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:34.546803+0000", 2024-09-14T06:31:57.799 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.8346e-05 2024-09-14T06:31:57.799 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:57.799 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:57.799 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:31:57.799 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:34.546823+0000", 2024-09-14T06:31:57.799 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.0699999999999998e-05 2024-09-14T06:31:57.799 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:57.799 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:57.799 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:31:57.799 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:34.546890+0000", 2024-09-14T06:31:57.799 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.6581000000000005e-05 2024-09-14T06:31:57.800 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:57.800 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:57.800 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:31:57.800 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:34.546899+0000", 2024-09-14T06:31:57.800 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.3889999999999994e-06 2024-09-14T06:31:57.800 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:57.800 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:57.800 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:31:57.800 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:34.547719+0000", 2024-09-14T06:31:57.800 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00082015699999999996 2024-09-14T06:31:57.800 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:57.800 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:31:57.800 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:57.800 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:57.801 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:57.801 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "MOSDPGPush(3.5 169/168 [PushOp(3:a7a54476:::rbd_data.112d1bb7b269.0000000000000038:head, version: 122'581, data_included: [0~524288], data_size: 524288, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(3:a7a54476:::rbd_data.112d1bb7b269.0000000000000038:head@122'581, size: 524288, copy_subset: [(0, 524288)], clone_subset: {}, snapset: 0=[]:{}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 524288, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2024-09-14T06:31:57.801 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:31:34.667757+0000", 2024-09-14T06:31:57.801 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 23.048633845000001, 2024-09-14T06:31:57.801 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.002750382, 2024-09-14T06:31:57.801 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:31:57.801 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:31:57.801 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:31:57.801 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:31:57.801 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:57.801 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:31:57.801 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:34.667757+0000", 2024-09-14T06:31:57.801 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:57.801 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:57.802 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:57.802 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:31:57.802 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:34.667757+0000", 2024-09-14T06:31:57.802 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:57.802 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:57.802 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:57.802 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:31:57.802 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:34.667758+0000", 2024-09-14T06:31:57.802 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.3599999999999999e-06 2024-09-14T06:31:57.802 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:57.802 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:57.802 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:31:57.802 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:34.669375+0000", 2024-09-14T06:31:57.803 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.001616644 2024-09-14T06:31:57.803 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:57.803 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:57.803 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:31:57.803 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:34.669404+0000", 2024-09-14T06:31:57.803 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.9300999999999999e-05 2024-09-14T06:31:57.803 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:57.803 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:57.803 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:31:57.803 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:34.669428+0000", 2024-09-14T06:31:57.803 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.3788e-05 2024-09-14T06:31:57.803 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:57.803 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:57.803 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:31:57.804 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:34.669543+0000", 2024-09-14T06:31:57.804 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000115462 2024-09-14T06:31:57.804 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:57.804 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:57.804 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:31:57.804 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:34.669559+0000", 2024-09-14T06:31:57.804 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.5167999999999999e-05 2024-09-14T06:31:57.804 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:57.804 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:57.804 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:31:57.804 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:34.670507+0000", 2024-09-14T06:31:57.804 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00094865900000000003 2024-09-14T06:31:57.804 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:57.805 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:31:57.805 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:57.805 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:57.805 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:57.805 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "MOSDPGPush(3.5 170/168 [PushOp(3:a162138c:::rbd_data.112d1bb7b269.000000000000003c:head, version: 122'582, data_included: [0~524288], data_size: 524288, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(3:a162138c:::rbd_data.112d1bb7b269.000000000000003c:head@122'582, size: 524288, copy_subset: [(0, 524288)], clone_subset: {}, snapset: 0=[]:{}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 524288, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2024-09-14T06:31:57.805 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:31:34.670110+0000", 2024-09-14T06:31:57.805 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 23.046280895999999, 2024-09-14T06:31:57.805 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0030066229999999999, 2024-09-14T06:31:57.805 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:31:57.805 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:31:57.805 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:31:57.805 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:31:57.805 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:57.805 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:31:57.806 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:34.670110+0000", 2024-09-14T06:31:57.806 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:57.806 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:57.806 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:57.806 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:31:57.806 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:34.670110+0000", 2024-09-14T06:31:57.806 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:57.806 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:57.806 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:57.806 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:31:57.806 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:34.670111+0000", 2024-09-14T06:31:57.806 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.6499999999999998e-07 2024-09-14T06:31:57.806 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:57.806 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:57.806 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:31:57.807 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:34.671507+0000", 2024-09-14T06:31:57.807 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.001396284 2024-09-14T06:31:57.807 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:57.807 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:57.807 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:31:57.807 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:34.671530+0000", 2024-09-14T06:31:57.807 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.3023e-05 2024-09-14T06:31:57.807 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:57.807 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:57.807 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:31:57.807 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:34.671548+0000", 2024-09-14T06:31:57.807 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.8116000000000001e-05 2024-09-14T06:31:57.807 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:57.808 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:57.808 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:31:57.808 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:34.671775+0000", 2024-09-14T06:31:57.808 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00022699600000000001 2024-09-14T06:31:57.808 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:57.808 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:57.808 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:31:57.808 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:34.671790+0000", 2024-09-14T06:31:57.808 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.539e-05 2024-09-14T06:31:57.808 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:57.808 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:57.808 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:31:57.808 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:34.673116+0000", 2024-09-14T06:31:57.808 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0013259490000000001 2024-09-14T06:31:57.809 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:57.809 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:31:57.809 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:57.809 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:57.809 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:31:57.809 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-14T06:31:57.810 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-14T06:31:57.812 INFO:teuthology.orchestra.run.smithi094.stderr:nodeep-scrub is unset 2024-09-14T06:31:57.812 INFO:tasks.ceph.osd.7.smithi138.stderr:2024-09-14T06:31:57.809+0000 7f3c6a21d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33623) UID: 0 2024-09-14T06:31:57.911 INFO:tasks.ceph.osd.4.smithi138.stderr:2024-09-14T06:31:57.911+0000 7f6c653c8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33615) UID: 0 2024-09-14T06:31:57.955 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-14T06:31:57.956 INFO:teuthology.orchestra.run.smithi094.stdout: "ops": [], 2024-09-14T06:31:57.956 INFO:teuthology.orchestra.run.smithi094.stdout: "num_ops": 0 2024-09-14T06:31:57.956 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-14T06:31:57.964 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-14T06:31:58.012 INFO:tasks.ceph.osd.2.smithi094.stderr:2024-09-14T06:31:58.011+0000 7faf3269b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33649) UID: 0 2024-09-14T06:31:58.066 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-14T06:31:58.066 INFO:teuthology.orchestra.run.smithi094.stdout: "ops": [], 2024-09-14T06:31:58.066 INFO:teuthology.orchestra.run.smithi094.stdout: "complaint_time": 30, 2024-09-14T06:31:58.066 INFO:teuthology.orchestra.run.smithi094.stdout: "num_blocked_ops": 0 2024-09-14T06:31:58.066 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-14T06:31:58.075 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-14T06:31:58.114 INFO:tasks.ceph.osd.6.smithi138.stderr:2024-09-14T06:31:58.113+0000 7f8cd8849640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33620) UID: 0 2024-09-14T06:31:58.215 INFO:tasks.ceph.osd.4.smithi138.stderr:2024-09-14T06:31:58.214+0000 7f6c653c8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33615) UID: 0 2024-09-14T06:31:58.220 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-14T06:31:58.220 INFO:teuthology.orchestra.run.smithi094.stdout: "size": 20, 2024-09-14T06:31:58.221 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 600, 2024-09-14T06:31:58.221 INFO:teuthology.orchestra.run.smithi094.stdout: "ops": [ 2024-09-14T06:31:58.221 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:58.221 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:44425 3.2 3:4603ad7f:::rbd_data.112dca2f6ae1.0000000000000005:17 [read 868352~4096 [fadvise_nocache]] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:31:58.221 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:43.854967+0000", 2024-09-14T06:31:58.221 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 134.36392884099999, 2024-09-14T06:31:58.221 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.063146025999999994, 2024-09-14T06:31:58.221 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:31:58.221 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:31:58.221 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:31:58.221 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:31:58.221 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:31:58.222 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:58.222 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 44425 2024-09-14T06:31:58.222 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:58.222 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:31:58.222 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:58.222 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:31:58.222 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.854967+0000", 2024-09-14T06:31:58.222 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:58.222 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:58.222 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:58.222 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:31:58.222 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.854967+0000", 2024-09-14T06:31:58.222 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:58.222 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:58.222 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:58.223 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:31:58.223 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.854967+0000", 2024-09-14T06:31:58.223 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.16e-07 2024-09-14T06:31:58.223 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:58.223 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:58.223 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:31:58.223 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.854968+0000", 2024-09-14T06:31:58.223 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.006e-06 2024-09-14T06:31:58.223 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:58.223 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:58.223 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:31:58.223 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.854971+0000", 2024-09-14T06:31:58.223 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.9359999999999999e-06 2024-09-14T06:31:58.223 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:58.223 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:58.224 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:31:58.224 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.854974+0000", 2024-09-14T06:31:58.224 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.1840000000000002e-06 2024-09-14T06:31:58.224 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:58.224 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:58.224 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:31:58.224 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.917945+0000", 2024-09-14T06:31:58.224 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.062970925999999997 2024-09-14T06:31:58.224 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:58.224 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:58.224 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:31:58.224 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.918030+0000", 2024-09-14T06:31:58.224 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.5863000000000001e-05 2024-09-14T06:31:58.225 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:58.225 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:58.225 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:31:58.225 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.918113+0000", 2024-09-14T06:31:58.225 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.2894999999999998e-05 2024-09-14T06:31:58.225 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:58.225 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:31:58.225 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:58.225 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:58.225 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:58.225 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:44426 3.2 3:4603ad7f:::rbd_data.112dca2f6ae1.0000000000000005:17 [read 892928~4096 [fadvise_nocache]] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:31:58.225 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:43.854977+0000", 2024-09-14T06:31:58.225 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 134.363919431, 2024-09-14T06:31:58.225 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.063369237999999994, 2024-09-14T06:31:58.226 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:31:58.226 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:31:58.226 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:31:58.226 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:31:58.226 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:31:58.226 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:58.226 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 44426 2024-09-14T06:31:58.226 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:58.226 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:31:58.226 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:58.226 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:31:58.226 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.854977+0000", 2024-09-14T06:31:58.226 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:58.226 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:58.227 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:58.227 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:31:58.227 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.854977+0000", 2024-09-14T06:31:58.227 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:58.227 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:58.227 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:58.227 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:31:58.227 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.854977+0000", 2024-09-14T06:31:58.227 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.1199999999999999e-07 2024-09-14T06:31:58.227 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:58.227 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:58.227 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:31:58.227 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.854980+0000", 2024-09-14T06:31:58.228 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.27e-06 2024-09-14T06:31:58.228 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:58.228 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:58.228 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:31:58.228 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.854983+0000", 2024-09-14T06:31:58.228 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.1650000000000002e-06 2024-09-14T06:31:58.228 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:58.228 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:58.228 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:31:58.228 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.854989+0000", 2024-09-14T06:31:58.228 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.6520000000000003e-06 2024-09-14T06:31:58.228 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:58.228 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:58.229 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:31:58.229 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.918143+0000", 2024-09-14T06:31:58.229 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.063154541999999994 2024-09-14T06:31:58.229 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:58.229 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:58.229 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:31:58.229 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.918209+0000", 2024-09-14T06:31:58.229 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.5529000000000002e-05 2024-09-14T06:31:58.229 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:58.229 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:58.229 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:31:58.229 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.918346+0000", 2024-09-14T06:31:58.229 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000136868 2024-09-14T06:31:58.229 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:58.230 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:31:58.230 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:58.230 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:58.230 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:58.230 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:44427 3.2 3:4603ad7f:::rbd_data.112dca2f6ae1.0000000000000005:17 [read 917504~4096 [fadvise_nocache]] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:31:58.230 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:43.854992+0000", 2024-09-14T06:31:58.230 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 134.36390378499999, 2024-09-14T06:31:58.230 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.063607792999999996, 2024-09-14T06:31:58.230 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:31:58.230 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:31:58.230 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:31:58.230 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:31:58.230 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:31:58.230 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:58.231 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 44427 2024-09-14T06:31:58.231 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:58.231 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:31:58.231 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:58.231 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:31:58.231 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.854992+0000", 2024-09-14T06:31:58.231 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:58.231 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:58.231 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:58.231 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:31:58.231 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.854992+0000", 2024-09-14T06:31:58.231 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:58.231 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:58.231 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:58.232 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:31:58.232 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.854993+0000", 2024-09-14T06:31:58.232 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.1400000000000001e-07 2024-09-14T06:31:58.232 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:58.232 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:58.232 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:31:58.232 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.854994+0000", 2024-09-14T06:31:58.232 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.091e-06 2024-09-14T06:31:58.232 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:58.232 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:58.232 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:31:58.232 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855001+0000", 2024-09-14T06:31:58.232 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.204e-06 2024-09-14T06:31:58.232 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:58.233 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:58.233 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:31:58.233 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855003+0000", 2024-09-14T06:31:58.233 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.514e-06 2024-09-14T06:31:58.233 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:58.233 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:58.233 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:31:58.233 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.918380+0000", 2024-09-14T06:31:58.233 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.063376738000000002 2024-09-14T06:31:58.233 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:58.233 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:58.233 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:31:58.233 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.918453+0000", 2024-09-14T06:31:58.233 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.3291e-05 2024-09-14T06:31:58.234 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:58.234 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:58.234 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:31:58.234 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.918600+0000", 2024-09-14T06:31:58.234 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00014674100000000001 2024-09-14T06:31:58.234 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:58.234 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:31:58.234 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:58.234 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:58.234 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:58.234 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:44428 3.2 3:4603ad7f:::rbd_data.112dca2f6ae1.0000000000000005:17 [read 942080~4096 [fadvise_nocache]] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:31:58.234 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:43.855006+0000", 2024-09-14T06:31:58.234 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 134.36388973499999, 2024-09-14T06:31:58.234 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.063846321999999997, 2024-09-14T06:31:58.235 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:31:58.235 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:31:58.235 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:31:58.235 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:31:58.235 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:31:58.235 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:58.235 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 44428 2024-09-14T06:31:58.235 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:58.235 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:31:58.235 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:58.235 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:31:58.235 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855006+0000", 2024-09-14T06:31:58.235 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:58.235 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:58.236 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:58.236 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:31:58.236 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855006+0000", 2024-09-14T06:31:58.236 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:58.236 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:58.236 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:58.236 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:31:58.236 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855007+0000", 2024-09-14T06:31:58.236 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.1199999999999999e-07 2024-09-14T06:31:58.236 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:58.236 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:58.236 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:31:58.236 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855011+0000", 2024-09-14T06:31:58.237 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.9539999999999998e-06 2024-09-14T06:31:58.237 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:58.237 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:58.237 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:31:58.237 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855014+0000", 2024-09-14T06:31:58.237 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.2930000000000001e-06 2024-09-14T06:31:58.237 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:58.237 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:58.237 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:31:58.237 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855016+0000", 2024-09-14T06:31:58.237 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.3039999999999999e-06 2024-09-14T06:31:58.237 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:58.237 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:58.238 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:31:58.238 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.918634+0000", 2024-09-14T06:31:58.238 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.063617928000000004 2024-09-14T06:31:58.238 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:58.238 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:58.238 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:31:58.238 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.918711+0000", 2024-09-14T06:31:58.238 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.6545000000000007e-05 2024-09-14T06:31:58.238 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:58.238 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:58.238 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:31:58.238 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.918853+0000", 2024-09-14T06:31:58.238 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00014208600000000001 2024-09-14T06:31:58.238 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:58.238 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:31:58.239 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:58.239 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:58.239 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:58.239 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:44429 3.2 3:4603ad7f:::rbd_data.112dca2f6ae1.0000000000000005:17 [read 966656~4096 [fadvise_nocache]] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:31:58.239 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:43.855019+0000", 2024-09-14T06:31:58.239 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 134.363877062, 2024-09-14T06:31:58.239 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.064059226999999996, 2024-09-14T06:31:58.239 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:31:58.239 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:31:58.239 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:31:58.239 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:31:58.239 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:31:58.239 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:58.239 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 44429 2024-09-14T06:31:58.240 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:58.240 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:31:58.240 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:58.240 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:31:58.240 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855019+0000", 2024-09-14T06:31:58.240 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:58.240 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:58.240 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:58.240 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:31:58.240 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855019+0000", 2024-09-14T06:31:58.240 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:58.240 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:58.240 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:58.240 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:31:58.241 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855019+0000", 2024-09-14T06:31:58.241 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.28e-07 2024-09-14T06:31:58.241 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:58.241 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:58.241 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:31:58.241 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855020+0000", 2024-09-14T06:31:58.241 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.0330000000000001e-06 2024-09-14T06:31:58.241 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:58.241 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:58.241 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:31:58.241 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855023+0000", 2024-09-14T06:31:58.241 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.79e-06 2024-09-14T06:31:58.241 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:58.242 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:58.242 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:31:58.242 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855025+0000", 2024-09-14T06:31:58.242 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.216e-06 2024-09-14T06:31:58.242 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:58.242 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:58.242 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:31:58.242 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.918886+0000", 2024-09-14T06:31:58.242 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.063861037999999995 2024-09-14T06:31:58.242 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:58.242 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:58.242 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:31:58.242 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.918952+0000", 2024-09-14T06:31:58.242 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.5993000000000003e-05 2024-09-14T06:31:58.243 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:58.243 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:58.243 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:31:58.243 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.919078+0000", 2024-09-14T06:31:58.243 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00012592899999999999 2024-09-14T06:31:58.243 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:58.243 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:31:58.243 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:58.243 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:58.243 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:58.243 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:44430 3.2 3:4603ad7f:::rbd_data.112dca2f6ae1.0000000000000005:17 [read 991232~4096 [fadvise_nocache]] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:31:58.243 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:43.855028+0000", 2024-09-14T06:31:58.243 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 134.36386805199999, 2024-09-14T06:31:58.243 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.064284346000000006, 2024-09-14T06:31:58.243 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:31:58.244 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:31:58.244 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:31:58.244 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:31:58.244 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:31:58.244 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:58.244 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 44430 2024-09-14T06:31:58.244 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:58.244 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:31:58.244 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:58.244 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:31:58.244 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855028+0000", 2024-09-14T06:31:58.244 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:58.244 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:58.244 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:58.244 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:31:58.245 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855028+0000", 2024-09-14T06:31:58.245 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:58.245 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:58.245 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:58.245 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:31:58.245 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855028+0000", 2024-09-14T06:31:58.245 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.1199999999999999e-07 2024-09-14T06:31:58.245 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:58.245 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:58.245 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:31:58.245 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855029+0000", 2024-09-14T06:31:58.245 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.1100000000000004e-07 2024-09-14T06:31:58.245 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:58.245 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:58.246 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:31:58.246 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855032+0000", 2024-09-14T06:31:58.246 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.7889999999999999e-06 2024-09-14T06:31:58.246 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:58.246 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:58.246 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:31:58.246 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855034+0000", 2024-09-14T06:31:58.246 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.125e-06 2024-09-14T06:31:58.246 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:58.246 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:58.246 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:31:58.246 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.919110+0000", 2024-09-14T06:31:58.246 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.064076413999999998 2024-09-14T06:31:58.246 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:58.247 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:58.247 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:31:58.247 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.919186+0000", 2024-09-14T06:31:58.247 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.5784999999999994e-05 2024-09-14T06:31:58.247 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:58.247 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:58.247 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:31:58.247 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.919312+0000", 2024-09-14T06:31:58.247 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00012611 2024-09-14T06:31:58.247 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:58.247 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:31:58.247 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:58.247 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:58.247 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:58.248 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:44431 3.2 3:4603ad7f:::rbd_data.112dca2f6ae1.0000000000000005:17 [read 1015808~4096 [fadvise_nocache]] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:31:58.248 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:43.855037+0000", 2024-09-14T06:31:58.248 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 134.36385918400001, 2024-09-14T06:31:58.248 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.064469755000000004, 2024-09-14T06:31:58.248 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:31:58.248 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:31:58.248 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:31:58.248 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:31:58.248 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:31:58.248 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:58.248 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 44431 2024-09-14T06:31:58.248 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:58.248 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:31:58.249 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:58.249 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:31:58.249 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855037+0000", 2024-09-14T06:31:58.249 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:58.249 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:58.249 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:58.249 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:31:58.249 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855037+0000", 2024-09-14T06:31:58.249 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:58.249 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:58.249 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:58.249 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:31:58.249 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855037+0000", 2024-09-14T06:31:58.250 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.1e-07 2024-09-14T06:31:58.250 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:58.250 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:58.250 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:31:58.250 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855038+0000", 2024-09-14T06:31:58.250 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.9400000000000004e-07 2024-09-14T06:31:58.250 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:58.250 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:58.250 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:31:58.250 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855041+0000", 2024-09-14T06:31:58.250 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.9639999999999999e-06 2024-09-14T06:31:58.250 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:58.250 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:58.250 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:31:58.250 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855043+0000", 2024-09-14T06:31:58.251 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.2259999999999999e-06 2024-09-14T06:31:58.251 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:58.251 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:58.251 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:31:58.251 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.919347+0000", 2024-09-14T06:31:58.251 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.064304091999999993 2024-09-14T06:31:58.251 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:58.251 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:58.251 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:31:58.251 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.919393+0000", 2024-09-14T06:31:58.251 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.5340999999999998e-05 2024-09-14T06:31:58.251 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:58.251 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:58.251 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:31:58.251 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.919507+0000", 2024-09-14T06:31:58.252 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000114028 2024-09-14T06:31:58.252 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:58.252 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:31:58.252 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:58.252 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:58.252 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:58.252 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:44432 3.2 3:4603ad7f:::rbd_data.112dca2f6ae1.0000000000000005:17 [read 1040384~4096 [fadvise_nocache]] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:31:58.252 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:43.855046+0000", 2024-09-14T06:31:58.252 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 134.36385011600001, 2024-09-14T06:31:58.252 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.064639253999999993, 2024-09-14T06:31:58.252 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:31:58.252 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:31:58.252 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:31:58.252 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:31:58.252 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:31:58.253 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:58.253 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 44432 2024-09-14T06:31:58.253 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:58.253 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:31:58.253 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:58.253 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:31:58.253 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855046+0000", 2024-09-14T06:31:58.253 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:58.253 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:58.253 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:58.253 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:31:58.253 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855046+0000", 2024-09-14T06:31:58.253 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:58.253 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:58.254 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:58.254 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:31:58.254 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855046+0000", 2024-09-14T06:31:58.254 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.05e-07 2024-09-14T06:31:58.254 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:58.254 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:58.254 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:31:58.254 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855047+0000", 2024-09-14T06:31:58.254 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.879999999999999e-07 2024-09-14T06:31:58.254 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:58.254 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:58.254 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:31:58.254 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855050+0000", 2024-09-14T06:31:58.254 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.8459999999999999e-06 2024-09-14T06:31:58.254 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:58.254 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:58.255 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:31:58.255 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855052+0000", 2024-09-14T06:31:58.255 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.136e-06 2024-09-14T06:31:58.255 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:58.255 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:58.255 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:31:58.255 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.919536+0000", 2024-09-14T06:31:58.255 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.064484132 2024-09-14T06:31:58.255 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:58.255 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:58.255 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:31:58.255 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.919611+0000", 2024-09-14T06:31:58.255 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.4407999999999996e-05 2024-09-14T06:31:58.256 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:58.256 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:58.256 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:31:58.256 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.919685+0000", 2024-09-14T06:31:58.256 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.4538999999999999e-05 2024-09-14T06:31:58.256 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:58.256 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:31:58.256 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:58.256 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:58.256 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:58.256 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:44518 3.2 3:4190d882:::rbd_data.112dca2f6ae1.0000000000000002:17 [read 12288~4096 [fadvise_nocache]] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:31:58.256 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:43.855055+0000", 2024-09-14T06:31:58.256 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 134.363841232, 2024-09-14T06:31:58.256 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.064894317000000007, 2024-09-14T06:31:58.257 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:31:58.257 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:31:58.257 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:31:58.257 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:31:58.257 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:31:58.257 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:58.257 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 44518 2024-09-14T06:31:58.257 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:58.257 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:31:58.257 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:58.257 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:31:58.257 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855055+0000", 2024-09-14T06:31:58.257 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:58.258 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:58.258 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:58.258 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:31:58.258 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855055+0000", 2024-09-14T06:31:58.258 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:58.258 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:58.258 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:58.258 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:31:58.258 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855055+0000", 2024-09-14T06:31:58.258 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.0699999999999999e-07 2024-09-14T06:31:58.258 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:58.258 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:58.258 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:31:58.259 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855056+0000", 2024-09-14T06:31:58.259 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.9400000000000004e-07 2024-09-14T06:31:58.259 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:58.259 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:58.259 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:31:58.259 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855059+0000", 2024-09-14T06:31:58.259 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.8870000000000001e-06 2024-09-14T06:31:58.259 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:58.259 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:58.259 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:31:58.259 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855061+0000", 2024-09-14T06:31:58.259 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.1919999999999999e-06 2024-09-14T06:31:58.259 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:58.259 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:58.260 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:31:58.260 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.919719+0000", 2024-09-14T06:31:58.260 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.064658013 2024-09-14T06:31:58.260 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:58.260 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:58.260 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:31:58.260 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.919788+0000", 2024-09-14T06:31:58.260 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.9282000000000005e-05 2024-09-14T06:31:58.260 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:58.260 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:58.260 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:31:58.260 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.919949+0000", 2024-09-14T06:31:58.260 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00016084199999999999 2024-09-14T06:31:58.260 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:58.260 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:31:58.261 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:58.261 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:58.261 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:58.261 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:44519 3.2 3:4190d882:::rbd_data.112dca2f6ae1.0000000000000002:17 [read 36864~4096 [fadvise_nocache]] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:31:58.261 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:43.855064+0000", 2024-09-14T06:31:58.261 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 134.36383225500001, 2024-09-14T06:31:58.261 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.065081865000000003, 2024-09-14T06:31:58.261 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:31:58.261 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:31:58.261 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:31:58.261 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:31:58.261 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:31:58.261 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:58.261 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 44519 2024-09-14T06:31:58.262 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:58.262 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:31:58.262 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:58.262 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:31:58.262 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855064+0000", 2024-09-14T06:31:58.262 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:58.262 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:58.262 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:58.262 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:31:58.262 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855064+0000", 2024-09-14T06:31:58.262 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:58.262 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:58.262 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:58.263 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:31:58.263 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855064+0000", 2024-09-14T06:31:58.263 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.1e-07 2024-09-14T06:31:58.263 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:58.263 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:58.263 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:31:58.263 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855065+0000", 2024-09-14T06:31:58.263 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.035e-06 2024-09-14T06:31:58.263 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:58.263 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:58.263 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:31:58.263 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855068+0000", 2024-09-14T06:31:58.263 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.8339999999999999e-06 2024-09-14T06:31:58.263 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:58.263 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:58.264 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:31:58.264 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855071+0000", 2024-09-14T06:31:58.264 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.3450000000000002e-06 2024-09-14T06:31:58.264 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:58.264 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:58.264 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:31:58.264 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.919979+0000", 2024-09-14T06:31:58.264 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.064907647999999998 2024-09-14T06:31:58.264 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:58.264 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:58.264 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:31:58.264 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.920026+0000", 2024-09-14T06:31:58.264 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.7595000000000001e-05 2024-09-14T06:31:58.265 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:58.265 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:58.265 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:31:58.265 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.920146+0000", 2024-09-14T06:31:58.265 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00011919800000000001 2024-09-14T06:31:58.265 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:58.265 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:31:58.265 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:58.265 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:58.265 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:58.265 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:44520 3.2 3:4190d882:::rbd_data.112dca2f6ae1.0000000000000002:17 [read 61440~4096 [fadvise_nocache]] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:31:58.265 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:43.855075+0000", 2024-09-14T06:31:58.265 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 134.363821204, 2024-09-14T06:31:58.265 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.065335078000000005, 2024-09-14T06:31:58.266 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:31:58.266 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:31:58.266 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:31:58.266 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:31:58.266 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:31:58.266 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:58.266 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 44520 2024-09-14T06:31:58.266 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:58.266 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:31:58.266 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:58.266 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:31:58.266 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855075+0000", 2024-09-14T06:31:58.266 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:58.266 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:58.266 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:58.267 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:31:58.267 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855075+0000", 2024-09-14T06:31:58.267 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:58.267 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:58.267 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:58.267 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:31:58.267 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855075+0000", 2024-09-14T06:31:58.267 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.1e-07 2024-09-14T06:31:58.267 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:58.267 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:58.267 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:31:58.267 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855076+0000", 2024-09-14T06:31:58.267 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.0839999999999999e-06 2024-09-14T06:31:58.267 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:58.268 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:58.268 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:31:58.268 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855079+0000", 2024-09-14T06:31:58.268 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.9359999999999999e-06 2024-09-14T06:31:58.268 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:58.268 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:58.268 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:31:58.268 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855084+0000", 2024-09-14T06:31:58.268 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.1730000000000003e-06 2024-09-14T06:31:58.268 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:58.268 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:58.268 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:31:58.268 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.920175+0000", 2024-09-14T06:31:58.268 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.065090234999999996 2024-09-14T06:31:58.269 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:58.269 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:58.269 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:31:58.269 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.920250+0000", 2024-09-14T06:31:58.269 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.5807000000000001e-05 2024-09-14T06:31:58.269 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:58.269 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:58.269 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:31:58.269 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.920410+0000", 2024-09-14T06:31:58.269 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00015963300000000001 2024-09-14T06:31:58.269 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:58.269 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:31:58.269 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:58.269 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:58.270 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:58.270 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:44521 3.2 3:4190d882:::rbd_data.112dca2f6ae1.0000000000000002:17 [read 86016~4096 [fadvise_nocache]] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:31:58.270 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:43.855087+0000", 2024-09-14T06:31:58.270 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 134.36380872300001, 2024-09-14T06:31:58.270 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.065563264999999996, 2024-09-14T06:31:58.270 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:31:58.270 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:31:58.270 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:31:58.270 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:31:58.270 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:31:58.270 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:58.270 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 44521 2024-09-14T06:31:58.270 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:58.270 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:31:58.271 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:58.271 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:31:58.271 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855087+0000", 2024-09-14T06:31:58.271 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:58.271 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:58.271 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:58.271 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:31:58.271 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855087+0000", 2024-09-14T06:31:58.271 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:58.271 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:58.271 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:58.271 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:31:58.271 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855088+0000", 2024-09-14T06:31:58.271 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.3699999999999999e-07 2024-09-14T06:31:58.272 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:58.272 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:58.272 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:31:58.272 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855089+0000", 2024-09-14T06:31:58.272 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.9900000000000009e-07 2024-09-14T06:31:58.272 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:58.272 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:58.272 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:31:58.272 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855091+0000", 2024-09-14T06:31:58.272 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.8940000000000002e-06 2024-09-14T06:31:58.272 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:58.272 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:58.272 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:31:58.272 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855094+0000", 2024-09-14T06:31:58.272 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.351e-06 2024-09-14T06:31:58.273 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:58.273 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:58.273 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:31:58.273 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.920437+0000", 2024-09-14T06:31:58.273 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.065343221000000007 2024-09-14T06:31:58.273 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:58.273 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:58.273 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:31:58.273 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.920481+0000", 2024-09-14T06:31:58.273 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.4431000000000002e-05 2024-09-14T06:31:58.273 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:58.273 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:58.273 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:31:58.273 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.920651+0000", 2024-09-14T06:31:58.273 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00016913200000000001 2024-09-14T06:31:58.274 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:58.274 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:31:58.274 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:58.274 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:58.274 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:58.274 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:44522 3.2 3:4190d882:::rbd_data.112dca2f6ae1.0000000000000002:17 [read 110592~4096 [fadvise_nocache]] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:31:58.274 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:43.855098+0000", 2024-09-14T06:31:58.274 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 134.36379849599999, 2024-09-14T06:31:58.274 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.065827628999999999, 2024-09-14T06:31:58.274 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:31:58.274 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:31:58.274 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:31:58.274 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:31:58.274 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:31:58.274 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:58.275 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 44522 2024-09-14T06:31:58.275 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:58.275 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:31:58.275 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:58.275 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:31:58.275 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855098+0000", 2024-09-14T06:31:58.275 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:58.275 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:58.275 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:58.275 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:31:58.275 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855098+0000", 2024-09-14T06:31:58.275 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:58.275 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:58.275 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:58.276 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:31:58.276 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855098+0000", 2024-09-14T06:31:58.276 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.1e-07 2024-09-14T06:31:58.276 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:58.276 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:58.276 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:31:58.276 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855099+0000", 2024-09-14T06:31:58.276 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.012e-06 2024-09-14T06:31:58.276 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:58.276 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:58.276 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:31:58.276 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855102+0000", 2024-09-14T06:31:58.276 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.9380000000000001e-06 2024-09-14T06:31:58.276 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:58.276 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:58.277 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:31:58.277 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855108+0000", 2024-09-14T06:31:58.277 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.4339999999999997e-06 2024-09-14T06:31:58.277 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:58.277 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:58.277 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:31:58.277 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.920681+0000", 2024-09-14T06:31:58.277 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.065572943999999994 2024-09-14T06:31:58.277 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:58.277 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:58.277 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:31:58.277 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.920724+0000", 2024-09-14T06:31:58.277 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.2877000000000003e-05 2024-09-14T06:31:58.278 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:58.278 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:58.278 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:31:58.278 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.920925+0000", 2024-09-14T06:31:58.278 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00020121399999999999 2024-09-14T06:31:58.278 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:58.278 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:31:58.278 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:58.278 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:58.278 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:58.278 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:44523 3.2 3:4190d882:::rbd_data.112dca2f6ae1.0000000000000002:17 [read 135168~4096 [fadvise_nocache]] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:31:58.278 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:43.855111+0000", 2024-09-14T06:31:58.278 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 134.36378477400001, 2024-09-14T06:31:58.278 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.066028198999999996, 2024-09-14T06:31:58.279 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:31:58.279 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:31:58.279 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:31:58.279 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:31:58.279 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:31:58.279 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:58.279 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 44523 2024-09-14T06:31:58.279 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:58.279 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:31:58.279 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:58.279 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:31:58.279 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855111+0000", 2024-09-14T06:31:58.279 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:58.279 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:58.280 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:58.280 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:31:58.280 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855111+0000", 2024-09-14T06:31:58.280 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:58.280 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:58.280 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:58.280 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:31:58.280 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855112+0000", 2024-09-14T06:31:58.280 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.1199999999999999e-07 2024-09-14T06:31:58.280 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:58.280 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:58.280 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:31:58.280 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855115+0000", 2024-09-14T06:31:58.280 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.3380000000000001e-06 2024-09-14T06:31:58.281 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:58.281 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:58.281 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:31:58.281 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855118+0000", 2024-09-14T06:31:58.281 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.366e-06 2024-09-14T06:31:58.281 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:58.281 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:58.281 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:31:58.281 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855121+0000", 2024-09-14T06:31:58.281 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.8329999999999998e-06 2024-09-14T06:31:58.281 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:58.281 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:58.281 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:31:58.282 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.920956+0000", 2024-09-14T06:31:58.282 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.065835251999999997 2024-09-14T06:31:58.282 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:58.282 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:58.282 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:31:58.282 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.921001+0000", 2024-09-14T06:31:58.282 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.4778999999999999e-05 2024-09-14T06:31:58.282 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:58.282 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:58.282 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:31:58.282 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.921139+0000", 2024-09-14T06:31:58.282 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000138419 2024-09-14T06:31:58.282 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:58.283 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:31:58.283 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:58.283 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:58.283 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:58.283 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:44524 3.2 3:4190d882:::rbd_data.112dca2f6ae1.0000000000000002:17 [read 159744~4096 [fadvise_nocache]] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:31:58.283 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:43.855124+0000", 2024-09-14T06:31:58.283 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 134.36377179499999, 2024-09-14T06:31:58.283 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.066249261000000004, 2024-09-14T06:31:58.283 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:31:58.283 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:31:58.283 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:31:58.283 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:31:58.283 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:31:58.284 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:58.284 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 44524 2024-09-14T06:31:58.284 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:58.284 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:31:58.284 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:58.284 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:31:58.284 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855124+0000", 2024-09-14T06:31:58.284 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:58.284 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:58.284 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:58.284 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:31:58.284 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855124+0000", 2024-09-14T06:31:58.284 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:58.284 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:58.285 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:58.285 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:31:58.285 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861086+0000", 2024-09-14T06:31:58.285 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0059613829999999998 2024-09-14T06:31:58.285 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:58.285 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:58.285 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:31:58.285 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861100+0000", 2024-09-14T06:31:58.285 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.4644000000000001e-05 2024-09-14T06:31:58.285 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:58.285 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:58.285 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:31:58.285 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861109+0000", 2024-09-14T06:31:58.285 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.541e-06 2024-09-14T06:31:58.286 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:58.286 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:58.286 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:31:58.286 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861117+0000", 2024-09-14T06:31:58.286 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.1049999999999995e-06 2024-09-14T06:31:58.286 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:58.286 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:58.286 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:31:58.286 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.921169+0000", 2024-09-14T06:31:58.286 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.060052405000000003 2024-09-14T06:31:58.286 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:58.286 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:58.286 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:31:58.286 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.921213+0000", 2024-09-14T06:31:58.286 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.3565e-05 2024-09-14T06:31:58.287 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:58.287 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:58.287 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:31:58.287 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.921374+0000", 2024-09-14T06:31:58.287 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00016061800000000001 2024-09-14T06:31:58.287 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:58.287 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:31:58.287 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:58.287 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:58.287 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:58.287 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:44537 3.2 3:4190d882:::rbd_data.112dca2f6ae1.0000000000000002:17 [read 479232~4096 [fadvise_nocache]] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:31:58.287 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:43.861405+0000", 2024-09-14T06:31:58.287 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 134.357491192, 2024-09-14T06:31:58.287 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.063351795000000002, 2024-09-14T06:31:58.287 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:31:58.288 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:31:58.288 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:31:58.288 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:31:58.288 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:31:58.288 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:58.288 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 44537 2024-09-14T06:31:58.288 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:58.288 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:31:58.288 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:58.288 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:31:58.288 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861405+0000", 2024-09-14T06:31:58.288 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:58.288 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:58.288 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:58.288 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:31:58.289 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861405+0000", 2024-09-14T06:31:58.289 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:58.289 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:58.289 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:58.289 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:31:58.289 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861405+0000", 2024-09-14T06:31:58.289 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.9099999999999999e-07 2024-09-14T06:31:58.289 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:58.289 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:58.289 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:31:58.289 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861407+0000", 2024-09-14T06:31:58.289 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.075e-06 2024-09-14T06:31:58.289 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:58.290 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:58.290 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:31:58.290 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861414+0000", 2024-09-14T06:31:58.290 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.4659999999999996e-06 2024-09-14T06:31:58.290 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:58.290 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:58.290 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:31:58.290 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861418+0000", 2024-09-14T06:31:58.290 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.5519999999999998e-06 2024-09-14T06:31:58.290 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:58.290 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:58.290 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:31:58.290 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.924474+0000", 2024-09-14T06:31:58.290 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.063056012999999994 2024-09-14T06:31:58.291 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:58.291 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:58.291 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:31:58.291 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.924556+0000", 2024-09-14T06:31:58.291 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.1181999999999996e-05 2024-09-14T06:31:58.291 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:58.291 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:58.291 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:31:58.291 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.924757+0000", 2024-09-14T06:31:58.291 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00020111599999999999 2024-09-14T06:31:58.291 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:58.291 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:31:58.291 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:58.292 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:58.292 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:58.292 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:44538 3.2 3:4190d882:::rbd_data.112dca2f6ae1.0000000000000002:17 [read 503808~4096 [fadvise_nocache]] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:31:58.292 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:43.861425+0000", 2024-09-14T06:31:58.292 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 134.357471151, 2024-09-14T06:31:58.292 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.063627125000000007, 2024-09-14T06:31:58.292 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:31:58.292 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:31:58.292 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:31:58.292 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:31:58.292 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:31:58.292 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:58.292 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 44538 2024-09-14T06:31:58.292 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:58.292 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:31:58.293 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:58.293 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:31:58.293 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861425+0000", 2024-09-14T06:31:58.293 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:58.293 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:58.293 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:58.293 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:31:58.293 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861425+0000", 2024-09-14T06:31:58.293 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:58.293 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:58.293 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:58.293 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:31:58.293 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861425+0000", 2024-09-14T06:31:58.293 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.9999999999999998e-07 2024-09-14T06:31:58.294 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:58.294 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:58.294 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:31:58.294 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861427+0000", 2024-09-14T06:31:58.294 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.9829999999999999e-06 2024-09-14T06:31:58.294 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:58.294 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:58.294 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:31:58.294 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861433+0000", 2024-09-14T06:31:58.294 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.2889999999999997e-06 2024-09-14T06:31:58.294 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:58.294 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:58.294 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:31:58.294 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861454+0000", 2024-09-14T06:31:58.295 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.1844999999999998e-05 2024-09-14T06:31:58.295 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:58.295 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:58.295 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:31:58.295 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.924786+0000", 2024-09-14T06:31:58.295 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.063331497 2024-09-14T06:31:58.295 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:58.295 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:58.295 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:31:58.295 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.924864+0000", 2024-09-14T06:31:58.295 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.7744000000000006e-05 2024-09-14T06:31:58.295 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:58.295 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:58.295 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:31:58.295 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.925052+0000", 2024-09-14T06:31:58.295 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00018836699999999999 2024-09-14T06:31:58.296 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:58.296 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:31:58.296 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:58.296 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:58.296 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:58.296 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:44539 3.2 3:4190d882:::rbd_data.112dca2f6ae1.0000000000000002:17 [read 528384~4096 [fadvise_nocache]] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:31:58.296 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:43.861462+0000", 2024-09-14T06:31:58.296 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 134.357433824, 2024-09-14T06:31:58.296 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.063855705999999998, 2024-09-14T06:31:58.296 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:31:58.296 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:31:58.296 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:31:58.296 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:31:58.297 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:31:58.297 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:58.297 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 44539 2024-09-14T06:31:58.297 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:58.297 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:31:58.297 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:58.297 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:31:58.297 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861462+0000", 2024-09-14T06:31:58.297 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:58.297 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:58.297 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:58.297 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:31:58.297 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861462+0000", 2024-09-14T06:31:58.298 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:58.298 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:58.298 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:58.298 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:31:58.298 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861463+0000", 2024-09-14T06:31:58.298 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.5200000000000002e-07 2024-09-14T06:31:58.298 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:58.298 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:58.298 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:31:58.298 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861465+0000", 2024-09-14T06:31:58.298 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.5189999999999999e-06 2024-09-14T06:31:58.298 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:58.298 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:58.298 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:31:58.298 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861471+0000", 2024-09-14T06:31:58.299 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.6899999999999997e-06 2024-09-14T06:31:58.299 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:58.299 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:58.299 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:31:58.299 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861475+0000", 2024-09-14T06:31:58.299 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.5730000000000001e-06 2024-09-14T06:31:58.299 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:58.299 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:58.299 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:31:58.299 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.925085+0000", 2024-09-14T06:31:58.299 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.063609902999999995 2024-09-14T06:31:58.299 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:58.299 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:58.300 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:31:58.300 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.925145+0000", 2024-09-14T06:31:58.300 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.9383999999999999e-05 2024-09-14T06:31:58.300 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:58.300 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:58.300 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:31:58.300 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.925318+0000", 2024-09-14T06:31:58.300 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00017318499999999999 2024-09-14T06:31:58.300 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:58.300 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:31:58.300 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:58.300 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:58.300 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:58.301 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:44540 3.2 3:4190d882:::rbd_data.112dca2f6ae1.0000000000000002:17 [read 552960~4096 [fadvise_nocache]] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:31:58.301 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:43.861483+0000", 2024-09-14T06:31:58.301 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 134.357413393, 2024-09-14T06:31:58.301 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.064065681999999999, 2024-09-14T06:31:58.301 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:31:58.301 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:31:58.301 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:31:58.301 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:31:58.301 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:31:58.301 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:58.301 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 44540 2024-09-14T06:31:58.301 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:58.302 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:31:58.302 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:58.302 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:31:58.302 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861483+0000", 2024-09-14T06:31:58.302 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:58.302 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:58.302 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:58.302 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:31:58.302 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861483+0000", 2024-09-14T06:31:58.302 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:58.302 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:58.302 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:58.302 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:31:58.302 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.867079+0000", 2024-09-14T06:31:58.303 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0055966480000000001 2024-09-14T06:31:58.303 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:58.303 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:58.303 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:31:58.303 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.867086+0000", 2024-09-14T06:31:58.303 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.387e-06 2024-09-14T06:31:58.303 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:58.303 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:58.303 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:31:58.303 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.867094+0000", 2024-09-14T06:31:58.303 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.9419999999999995e-06 2024-09-14T06:31:58.303 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:58.303 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:58.304 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:31:58.304 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.867109+0000", 2024-09-14T06:31:58.304 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.5818999999999999e-05 2024-09-14T06:31:58.304 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:58.304 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:58.304 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:31:58.304 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.925354+0000", 2024-09-14T06:31:58.304 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.058244047 2024-09-14T06:31:58.304 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:58.304 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:58.304 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:31:58.304 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.925406+0000", 2024-09-14T06:31:58.304 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.2661999999999998e-05 2024-09-14T06:31:58.304 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:58.304 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:58.305 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:31:58.305 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.925548+0000", 2024-09-14T06:31:58.305 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000142177 2024-09-14T06:31:58.305 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:58.305 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:31:58.305 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:58.305 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:58.305 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:58.305 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:225587 3.7 3:e112e2b3:::rbd_id.image_client.0-clone42:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e189)", 2024-09-14T06:31:58.305 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:31:53.874591+0000", 2024-09-14T06:31:58.305 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 4.3443051759999998, 2024-09-14T06:31:58.305 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.875541507, 2024-09-14T06:31:58.305 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:31:58.305 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:31:58.306 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:31:58.306 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:31:58.306 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:31:58.306 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:58.306 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 225587 2024-09-14T06:31:58.306 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:58.306 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:31:58.306 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:58.306 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:31:58.306 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:53.874591+0000", 2024-09-14T06:31:58.306 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:58.306 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:58.306 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:58.306 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:31:58.306 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:53.874591+0000", 2024-09-14T06:31:58.307 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:31:58.307 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:58.307 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:58.307 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:31:58.307 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:53.874592+0000", 2024-09-14T06:31:58.307 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.5209999999999999e-06 2024-09-14T06:31:58.307 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:58.307 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:58.307 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:31:58.307 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:53.874599+0000", 2024-09-14T06:31:58.307 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.6290000000000004e-06 2024-09-14T06:31:58.307 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:58.307 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:58.307 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:31:58.307 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:53.874618+0000", 2024-09-14T06:31:58.307 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.8950999999999999e-05 2024-09-14T06:31:58.308 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:58.308 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:58.308 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:31:58.308 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:53.874626+0000", 2024-09-14T06:31:58.308 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.3960000000000006e-06 2024-09-14T06:31:58.308 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:58.308 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:58.308 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:31:58.308 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:53.874699+0000", 2024-09-14T06:31:58.308 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.2329999999999994e-05 2024-09-14T06:31:58.308 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:58.308 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:58.308 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "waiting for peered", 2024-09-14T06:31:58.308 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:53.874711+0000", 2024-09-14T06:31:58.308 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.2473999999999999e-05 2024-09-14T06:31:58.309 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:58.309 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:58.309 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:31:58.309 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:54.749578+0000", 2024-09-14T06:31:58.309 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.87486642100000001 2024-09-14T06:31:58.309 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:58.309 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:58.309 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:31:58.309 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:54.749706+0000", 2024-09-14T06:31:58.309 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00012856200000000001 2024-09-14T06:31:58.309 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:31:58.309 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:31:58.309 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:31:58.310 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:54.750132+0000", 2024-09-14T06:31:58.310 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00042622299999999999 2024-09-14T06:31:58.310 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:58.310 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:31:58.310 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:58.310 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:31:58.310 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:31:58.310 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-14T06:31:58.311 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-14T06:31:58.316 INFO:tasks.ceph.osd.1.smithi094.stderr:2024-09-14T06:31:58.315+0000 7f328e47a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33641) UID: 0 2024-09-14T06:31:58.417 INFO:tasks.ceph.osd.5.smithi138.stderr:2024-09-14T06:31:58.415+0000 7fe13746f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33618) UID: 0 2024-09-14T06:31:58.417 INFO:teuthology.orchestra.run.smithi138.stdout:{ 2024-09-14T06:31:58.417 INFO:teuthology.orchestra.run.smithi138.stdout: "ops": [], 2024-09-14T06:31:58.417 INFO:teuthology.orchestra.run.smithi138.stdout: "num_ops": 0 2024-09-14T06:31:58.418 INFO:teuthology.orchestra.run.smithi138.stdout:} 2024-09-14T06:31:58.426 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-14T06:31:58.518 INFO:tasks.ceph.osd.7.smithi138.stderr:2024-09-14T06:31:58.517+0000 7f3c6a21d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33623) UID: 0 2024-09-14T06:31:58.568 INFO:teuthology.orchestra.run.smithi138.stdout:{ 2024-09-14T06:31:58.568 INFO:teuthology.orchestra.run.smithi138.stdout: "ops": [], 2024-09-14T06:31:58.569 INFO:teuthology.orchestra.run.smithi138.stdout: "complaint_time": 30, 2024-09-14T06:31:58.569 INFO:teuthology.orchestra.run.smithi138.stdout: "num_blocked_ops": 0 2024-09-14T06:31:58.569 INFO:teuthology.orchestra.run.smithi138.stdout:} 2024-09-14T06:31:58.577 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-14T06:31:58.618 INFO:tasks.ceph.osd.7.smithi138.stderr:2024-09-14T06:31:58.617+0000 7f3c6a21d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33623) UID: 0 2024-09-14T06:31:58.720 INFO:teuthology.orchestra.run.smithi138.stdout:{ 2024-09-14T06:31:58.721 INFO:teuthology.orchestra.run.smithi138.stdout: "size": 20, 2024-09-14T06:31:58.721 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 600, 2024-09-14T06:31:58.721 INFO:teuthology.orchestra.run.smithi138.stdout: "ops": [ 2024-09-14T06:31:58.721 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:58.721 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:19328 3.3 3:c34e5e51:::rbd_data.112d92027590.0000000000000024:f [read 1926144~1024 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:31:58.721 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:27.059768+0000", 2024-09-14T06:31:58.721 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 151.659081714, 2024-09-14T06:31:58.721 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046369147999999999, 2024-09-14T06:31:58.721 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:31:58.721 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:31:58.721 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:31:58.721 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:31:58.721 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:31:58.722 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:58.722 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 19328 2024-09-14T06:31:58.722 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:58.722 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:31:58.722 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:58.722 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:31:58.722 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.059768+0000", 2024-09-14T06:31:58.722 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:58.722 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:58.722 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:58.722 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:31:58.722 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.059768+0000", 2024-09-14T06:31:58.722 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:58.723 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:58.723 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:58.723 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:31:58.723 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.059874+0000", 2024-09-14T06:31:58.723 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00010551400000000001 2024-09-14T06:31:58.723 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:58.723 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:58.723 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:31:58.723 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.059877+0000", 2024-09-14T06:31:58.723 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.8279999999999999e-06 2024-09-14T06:31:58.723 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:58.723 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:58.723 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:31:58.723 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.059884+0000", 2024-09-14T06:31:58.724 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.6389999999999992e-06 2024-09-14T06:31:58.724 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:58.724 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:58.724 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:31:58.724 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.059890+0000", 2024-09-14T06:31:58.724 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.49e-06 2024-09-14T06:31:58.724 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:58.724 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:58.724 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:31:58.724 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.105804+0000", 2024-09-14T06:31:58.724 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.045914242000000001 2024-09-14T06:31:58.724 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:58.724 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:58.724 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:31:58.725 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.105886+0000", 2024-09-14T06:31:58.725 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.2305999999999994e-05 2024-09-14T06:31:58.725 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:58.725 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:58.725 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:31:58.725 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.106138+0000", 2024-09-14T06:31:58.725 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.000251129 2024-09-14T06:31:58.725 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:58.725 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:31:58.725 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:58.725 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:58.725 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:58.725 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:19329 3.3 3:c34e5e51:::rbd_data.112d92027590.0000000000000024:f [read 1936384~1024 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:31:58.725 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:27.059899+0000", 2024-09-14T06:31:58.726 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 151.658951191, 2024-09-14T06:31:58.726 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046572122000000001, 2024-09-14T06:31:58.726 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:31:58.726 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:31:58.726 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:31:58.726 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:31:58.726 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:31:58.726 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:58.726 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 19329 2024-09-14T06:31:58.726 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:58.726 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:31:58.726 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:58.726 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:31:58.726 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.059899+0000", 2024-09-14T06:31:58.726 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:58.727 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:58.727 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:58.727 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:31:58.727 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.059899+0000", 2024-09-14T06:31:58.727 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:58.727 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:58.727 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:58.727 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:31:58.727 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062501+0000", 2024-09-14T06:31:58.727 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.0026016419999999999 2024-09-14T06:31:58.727 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:58.727 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:58.728 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:31:58.728 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062514+0000", 2024-09-14T06:31:58.728 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.2947e-05 2024-09-14T06:31:58.728 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:58.728 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:58.728 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:31:58.728 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062523+0000", 2024-09-14T06:31:58.728 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.5519999999999993e-06 2024-09-14T06:31:58.728 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:58.728 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:58.728 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:31:58.728 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062534+0000", 2024-09-14T06:31:58.728 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.1416e-05 2024-09-14T06:31:58.729 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:58.729 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:58.729 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:31:58.729 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.106179+0000", 2024-09-14T06:31:58.729 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.043644049999999997 2024-09-14T06:31:58.729 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:58.729 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:58.729 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:31:58.729 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.106262+0000", 2024-09-14T06:31:58.729 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.3178999999999998e-05 2024-09-14T06:31:58.729 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:58.729 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:58.729 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:31:58.729 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.106471+0000", 2024-09-14T06:31:58.730 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00020933599999999999 2024-09-14T06:31:58.730 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:58.730 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:31:58.730 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:58.730 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:58.730 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:58.730 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:19351 3.3 3:c34e5e51:::rbd_data.112d92027590.0000000000000024:f [read 1159168~1024 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:31:58.730 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:27.062766+0000", 2024-09-14T06:31:58.730 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 151.65608396600001, 2024-09-14T06:31:58.730 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046099454999999998, 2024-09-14T06:31:58.730 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:31:58.730 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:31:58.730 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:31:58.730 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:31:58.730 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:31:58.731 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:58.731 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 19351 2024-09-14T06:31:58.731 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:58.731 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:31:58.731 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:58.731 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:31:58.731 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062766+0000", 2024-09-14T06:31:58.731 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:58.731 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:58.731 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:58.731 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:31:58.731 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062766+0000", 2024-09-14T06:31:58.731 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:58.731 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:58.732 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:58.732 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:31:58.732 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062766+0000", 2024-09-14T06:31:58.732 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.6e-07 2024-09-14T06:31:58.732 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:58.732 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:58.732 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:31:58.732 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062767+0000", 2024-09-14T06:31:58.732 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.0070000000000001e-06 2024-09-14T06:31:58.732 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:58.732 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:58.732 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:31:58.732 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062785+0000", 2024-09-14T06:31:58.732 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.7884999999999999e-05 2024-09-14T06:31:58.733 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:58.733 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:58.733 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:31:58.733 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062790+0000", 2024-09-14T06:31:58.733 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.0900000000000004e-06 2024-09-14T06:31:58.733 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:58.733 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:58.733 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:31:58.733 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.108735+0000", 2024-09-14T06:31:58.733 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.045944411999999997 2024-09-14T06:31:58.733 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:58.733 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:58.733 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:31:58.733 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.108809+0000", 2024-09-14T06:31:58.734 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.4157000000000002e-05 2024-09-14T06:31:58.734 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:58.734 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:58.734 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:31:58.734 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.108866+0000", 2024-09-14T06:31:58.734 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.6644000000000002e-05 2024-09-14T06:31:58.734 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:58.734 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:31:58.734 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:58.734 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:58.734 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:58.734 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:19352 3.3 3:c34e5e51:::rbd_data.112d92027590.0000000000000024:f [read 1169408~1024 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:31:58.734 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:27.062798+0000", 2024-09-14T06:31:58.734 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 151.656052555, 2024-09-14T06:31:58.735 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046255900000000003, 2024-09-14T06:31:58.735 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:31:58.735 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:31:58.735 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:31:58.735 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:31:58.735 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:31:58.735 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:58.735 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 19352 2024-09-14T06:31:58.735 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:58.735 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:31:58.735 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:58.735 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:31:58.735 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062798+0000", 2024-09-14T06:31:58.735 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:58.736 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:58.736 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:58.736 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:31:58.736 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062798+0000", 2024-09-14T06:31:58.736 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:58.736 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:58.736 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:58.736 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:31:58.736 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062798+0000", 2024-09-14T06:31:58.736 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.37e-07 2024-09-14T06:31:58.736 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:58.736 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:58.736 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:31:58.736 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062800+0000", 2024-09-14T06:31:58.737 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.0269999999999998e-06 2024-09-14T06:31:58.737 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:58.737 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:58.737 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:31:58.737 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062806+0000", 2024-09-14T06:31:58.737 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.6690000000000003e-06 2024-09-14T06:31:58.737 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:58.737 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:58.737 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:31:58.737 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062810+0000", 2024-09-14T06:31:58.737 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.3449999999999999e-06 2024-09-14T06:31:58.737 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:58.737 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:58.737 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:31:58.738 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.108892+0000", 2024-09-14T06:31:58.738 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046081715000000002 2024-09-14T06:31:58.738 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:58.738 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:58.738 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:31:58.738 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.108940+0000", 2024-09-14T06:31:58.738 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.7815999999999998e-05 2024-09-14T06:31:58.738 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:58.738 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:58.738 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:31:58.738 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.109053+0000", 2024-09-14T06:31:58.738 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.000113791 2024-09-14T06:31:58.738 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:58.738 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:31:58.739 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:58.739 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:58.739 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:58.739 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:19353 3.3 3:c34e5e51:::rbd_data.112d92027590.0000000000000024:f [read 1179648~1024 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:31:58.739 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:27.062816+0000", 2024-09-14T06:31:58.739 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 151.65603403599999, 2024-09-14T06:31:58.739 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046481256999999998, 2024-09-14T06:31:58.739 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:31:58.739 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:31:58.739 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:31:58.739 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:31:58.739 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:31:58.739 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:58.739 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 19353 2024-09-14T06:31:58.740 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:58.740 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:31:58.740 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:58.740 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:31:58.740 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062816+0000", 2024-09-14T06:31:58.740 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:58.740 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:58.740 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:58.740 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:31:58.740 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062816+0000", 2024-09-14T06:31:58.740 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:58.740 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:58.740 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:58.740 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:31:58.741 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062816+0000", 2024-09-14T06:31:58.741 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.2899999999999999e-07 2024-09-14T06:31:58.741 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:58.741 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:58.741 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:31:58.741 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062818+0000", 2024-09-14T06:31:58.741 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.699e-06 2024-09-14T06:31:58.741 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:58.741 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:58.741 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:31:58.741 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062825+0000", 2024-09-14T06:31:58.741 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.878e-06 2024-09-14T06:31:58.741 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:58.741 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:58.742 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:31:58.742 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062830+0000", 2024-09-14T06:31:58.742 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.5410000000000002e-06 2024-09-14T06:31:58.742 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:58.742 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:58.742 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:31:58.742 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.109096+0000", 2024-09-14T06:31:58.742 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046266517 2024-09-14T06:31:58.742 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:58.742 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:58.742 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:31:58.742 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.109190+0000", 2024-09-14T06:31:58.742 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.4000999999999997e-05 2024-09-14T06:31:58.742 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:58.742 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:58.743 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:31:58.743 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.109297+0000", 2024-09-14T06:31:58.743 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.000107292 2024-09-14T06:31:58.743 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:58.743 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:31:58.743 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:58.743 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:58.743 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:58.743 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:19354 3.3 3:c34e5e51:::rbd_data.112d92027590.0000000000000024:f [read 1189888~1024 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:31:58.743 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:27.062842+0000", 2024-09-14T06:31:58.743 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 151.65600858100001, 2024-09-14T06:31:58.743 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046594257, 2024-09-14T06:31:58.743 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:31:58.743 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:31:58.744 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:31:58.744 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:31:58.744 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:31:58.744 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:58.744 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 19354 2024-09-14T06:31:58.744 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:58.744 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:31:58.744 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:58.744 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:31:58.744 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062842+0000", 2024-09-14T06:31:58.744 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:58.744 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:58.744 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:58.745 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:31:58.745 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062842+0000", 2024-09-14T06:31:58.745 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:58.745 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:58.745 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:58.745 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:31:58.745 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062842+0000", 2024-09-14T06:31:58.745 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.7399999999999999e-07 2024-09-14T06:31:58.745 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:58.745 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:58.745 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:31:58.745 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062859+0000", 2024-09-14T06:31:58.745 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.7010000000000001e-05 2024-09-14T06:31:58.745 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:58.746 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:58.746 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:31:58.746 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062866+0000", 2024-09-14T06:31:58.746 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.9990000000000002e-06 2024-09-14T06:31:58.746 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:58.746 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:58.746 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:31:58.746 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062873+0000", 2024-09-14T06:31:58.746 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.8569999999999997e-06 2024-09-14T06:31:58.746 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:58.746 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:58.746 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:31:58.746 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.109335+0000", 2024-09-14T06:31:58.746 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046461943999999998 2024-09-14T06:31:58.746 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:58.747 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:58.747 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:31:58.747 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.109377+0000", 2024-09-14T06:31:58.747 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.2558999999999997e-05 2024-09-14T06:31:58.747 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:58.747 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:58.747 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:31:58.747 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.109436+0000", 2024-09-14T06:31:58.747 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.8514000000000003e-05 2024-09-14T06:31:58.747 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:58.747 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:31:58.747 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:58.747 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:58.747 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:58.748 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:19355 3.3 3:c34e5e51:::rbd_data.112d92027590.0000000000000024:f [read 1200128~1024 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:31:58.748 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:27.062884+0000", 2024-09-14T06:31:58.748 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 151.65596590800001, 2024-09-14T06:31:58.748 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046681650999999998, 2024-09-14T06:31:58.748 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:31:58.748 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:31:58.748 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:31:58.748 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:31:58.748 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:31:58.748 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:58.748 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 19355 2024-09-14T06:31:58.748 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:58.748 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:31:58.748 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:58.749 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:31:58.749 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062884+0000", 2024-09-14T06:31:58.749 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:58.749 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:58.749 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:58.749 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:31:58.749 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062884+0000", 2024-09-14T06:31:58.749 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:58.749 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:58.749 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:58.749 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:31:58.749 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062885+0000", 2024-09-14T06:31:58.749 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.8000000000000001e-07 2024-09-14T06:31:58.749 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:58.750 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:58.750 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:31:58.750 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062886+0000", 2024-09-14T06:31:58.750 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.8920000000000001e-06 2024-09-14T06:31:58.750 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:58.750 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:58.750 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:31:58.750 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062892+0000", 2024-09-14T06:31:58.750 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.84e-06 2024-09-14T06:31:58.750 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:58.750 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:58.750 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:31:58.750 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062899+0000", 2024-09-14T06:31:58.750 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.5339999999999996e-06 2024-09-14T06:31:58.751 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:58.751 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:58.751 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:31:58.751 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.109466+0000", 2024-09-14T06:31:58.751 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046566919999999998 2024-09-14T06:31:58.751 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:58.751 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:58.751 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:31:58.751 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.109510+0000", 2024-09-14T06:31:58.751 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.4153000000000002e-05 2024-09-14T06:31:58.751 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:58.751 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:58.751 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:31:58.751 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.109566+0000", 2024-09-14T06:31:58.751 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.5931999999999999e-05 2024-09-14T06:31:58.752 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:58.752 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:31:58.752 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:58.752 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:58.752 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:58.752 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:19356 3.3 3:c34e5e51:::rbd_data.112d92027590.0000000000000024:f [read 1210368~1024 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:31:58.752 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:27.062908+0000", 2024-09-14T06:31:58.752 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 151.655942471, 2024-09-14T06:31:58.752 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046807498000000003, 2024-09-14T06:31:58.752 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:31:58.752 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:31:58.752 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:31:58.752 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:31:58.753 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:31:58.753 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:58.753 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 19356 2024-09-14T06:31:58.753 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:58.753 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:31:58.753 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:58.753 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:31:58.753 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062908+0000", 2024-09-14T06:31:58.753 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:58.753 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:58.753 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:58.753 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:31:58.753 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062908+0000", 2024-09-14T06:31:58.753 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:58.753 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:58.754 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:58.754 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:31:58.754 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062908+0000", 2024-09-14T06:31:58.754 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.5699999999999998e-07 2024-09-14T06:31:58.754 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:58.754 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:58.754 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:31:58.754 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062910+0000", 2024-09-14T06:31:58.754 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.7710000000000001e-06 2024-09-14T06:31:58.754 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:58.754 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:58.754 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:31:58.754 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062920+0000", 2024-09-14T06:31:58.754 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.0654e-05 2024-09-14T06:31:58.755 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:58.755 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:58.755 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:31:58.755 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062931+0000", 2024-09-14T06:31:58.755 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.075e-05 2024-09-14T06:31:58.755 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:58.755 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:58.755 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:31:58.755 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.109595+0000", 2024-09-14T06:31:58.755 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046663713000000002 2024-09-14T06:31:58.755 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:58.755 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:58.755 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:31:58.755 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.109632+0000", 2024-09-14T06:31:58.755 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.7505000000000001e-05 2024-09-14T06:31:58.756 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:58.756 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:58.756 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:31:58.756 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.109715+0000", 2024-09-14T06:31:58.756 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.2748000000000001e-05 2024-09-14T06:31:58.756 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:58.756 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:31:58.756 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:58.756 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:58.756 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:58.756 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:19357 3.3 3:c34e5e51:::rbd_data.112d92027590.0000000000000024:f [read 1220608~1024 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:31:58.756 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:27.062939+0000", 2024-09-14T06:31:58.756 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 151.65591102299999, 2024-09-14T06:31:58.756 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046920508, 2024-09-14T06:31:58.757 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:31:58.757 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:31:58.757 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:31:58.757 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:31:58.757 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:31:58.757 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:58.757 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 19357 2024-09-14T06:31:58.757 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:58.757 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:31:58.757 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:58.757 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:31:58.757 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062939+0000", 2024-09-14T06:31:58.757 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:58.757 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:58.757 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:58.758 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:31:58.758 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062939+0000", 2024-09-14T06:31:58.758 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:58.758 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:58.758 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:58.758 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:31:58.758 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062940+0000", 2024-09-14T06:31:58.758 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.32e-07 2024-09-14T06:31:58.758 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:58.758 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:58.758 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:31:58.758 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062942+0000", 2024-09-14T06:31:58.758 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.1919999999999999e-06 2024-09-14T06:31:58.758 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:58.759 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:58.759 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:31:58.759 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062948+0000", 2024-09-14T06:31:58.759 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.1149999999999999e-06 2024-09-14T06:31:58.759 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:58.759 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:58.759 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:31:58.759 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062960+0000", 2024-09-14T06:31:58.759 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.1895e-05 2024-09-14T06:31:58.759 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:58.759 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:58.759 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:31:58.759 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.109704+0000", 2024-09-14T06:31:58.759 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046744584999999998 2024-09-14T06:31:58.760 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:58.760 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:58.760 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:31:58.760 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.109777+0000", 2024-09-14T06:31:58.760 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.2282999999999999e-05 2024-09-14T06:31:58.760 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:58.760 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:58.760 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:31:58.760 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.109860+0000", 2024-09-14T06:31:58.760 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.3005999999999998e-05 2024-09-14T06:31:58.760 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:58.760 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:31:58.760 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:58.761 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:58.761 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:58.761 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:19358 3.3 3:c34e5e51:::rbd_data.112d92027590.0000000000000024:f [read 1230848~1024 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:31:58.761 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:27.062967+0000", 2024-09-14T06:31:58.761 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 151.65588339300001, 2024-09-14T06:31:58.761 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047144471, 2024-09-14T06:31:58.761 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:31:58.761 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:31:58.761 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:31:58.761 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:31:58.761 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:31:58.761 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:58.762 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 19358 2024-09-14T06:31:58.762 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:58.762 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:31:58.762 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:58.762 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:31:58.762 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062967+0000", 2024-09-14T06:31:58.762 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:58.762 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:58.762 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:58.762 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:31:58.762 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062967+0000", 2024-09-14T06:31:58.762 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:58.762 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:58.762 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:58.763 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:31:58.763 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062967+0000", 2024-09-14T06:31:58.763 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.89e-07 2024-09-14T06:31:58.763 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:58.763 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:58.763 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:31:58.763 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062969+0000", 2024-09-14T06:31:58.763 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.9929999999999998e-06 2024-09-14T06:31:58.763 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:58.763 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:58.763 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:31:58.763 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062975+0000", 2024-09-14T06:31:58.763 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.9030000000000001e-06 2024-09-14T06:31:58.764 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:58.764 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:58.764 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:31:58.764 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062979+0000", 2024-09-14T06:31:58.764 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.2250000000000002e-06 2024-09-14T06:31:58.764 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:58.764 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:58.764 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:31:58.764 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.109934+0000", 2024-09-14T06:31:58.764 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046954410000000002 2024-09-14T06:31:58.764 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:58.764 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:58.764 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:31:58.765 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.110007+0000", 2024-09-14T06:31:58.765 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.3829000000000001e-05 2024-09-14T06:31:58.765 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:58.765 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:58.765 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:31:58.765 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.110111+0000", 2024-09-14T06:31:58.765 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00010372199999999999 2024-09-14T06:31:58.765 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:58.765 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:31:58.765 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:58.765 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:58.765 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:58.765 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:19359 3.3 3:c34e5e51:::rbd_data.112d92027590.0000000000000024:f [read 1241088~1024 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:31:58.765 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:27.062987+0000", 2024-09-14T06:31:58.766 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 151.65586330299999, 2024-09-14T06:31:58.766 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047297182, 2024-09-14T06:31:58.766 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:31:58.766 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:31:58.766 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:31:58.766 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:31:58.766 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:31:58.766 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:58.766 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 19359 2024-09-14T06:31:58.766 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:58.766 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:31:58.766 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:58.766 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:31:58.766 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062987+0000", 2024-09-14T06:31:58.767 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:58.767 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:58.767 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:58.767 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:31:58.767 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062987+0000", 2024-09-14T06:31:58.767 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:58.767 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:58.767 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:58.767 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:31:58.767 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062987+0000", 2024-09-14T06:31:58.767 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.2500000000000001e-07 2024-09-14T06:31:58.767 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:58.767 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:58.767 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:31:58.768 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062989+0000", 2024-09-14T06:31:58.768 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.6080000000000001e-06 2024-09-14T06:31:58.768 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:58.768 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:58.768 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:31:58.768 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062994+0000", 2024-09-14T06:31:58.768 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.5520000000000004e-06 2024-09-14T06:31:58.768 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:58.768 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:58.768 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:31:58.768 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062999+0000", 2024-09-14T06:31:58.768 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.3930000000000001e-06 2024-09-14T06:31:58.768 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:58.769 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:58.769 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:31:58.769 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.110142+0000", 2024-09-14T06:31:58.769 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047143154 2024-09-14T06:31:58.769 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:58.769 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:58.769 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:31:58.769 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.110219+0000", 2024-09-14T06:31:58.769 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.6737999999999996e-05 2024-09-14T06:31:58.769 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:58.769 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:58.769 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:31:58.769 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.110284+0000", 2024-09-14T06:31:58.769 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.5412000000000003e-05 2024-09-14T06:31:58.770 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:58.770 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:31:58.770 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:58.770 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:58.770 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:58.770 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:19360 3.3 3:c34e5e51:::rbd_data.112d92027590.0000000000000024:f [read 1251328~1024 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:31:58.770 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:27.063017+0000", 2024-09-14T06:31:58.770 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 151.655832842, 2024-09-14T06:31:58.770 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047453823999999999, 2024-09-14T06:31:58.770 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:31:58.770 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:31:58.770 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:31:58.770 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:31:58.770 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:31:58.771 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:58.771 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 19360 2024-09-14T06:31:58.771 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:58.771 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:31:58.771 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:58.771 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:31:58.771 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.063017+0000", 2024-09-14T06:31:58.771 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:58.771 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:58.771 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:58.771 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:31:58.771 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.063017+0000", 2024-09-14T06:31:58.771 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:58.771 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:58.772 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:58.772 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:31:58.772 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.064977+0000", 2024-09-14T06:31:58.772 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.0019597080000000001 2024-09-14T06:31:58.772 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:58.772 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:58.772 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:31:58.772 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.064987+0000", 2024-09-14T06:31:58.772 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.9620000000000005e-06 2024-09-14T06:31:58.772 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:58.772 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:58.772 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:31:58.772 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065000+0000", 2024-09-14T06:31:58.772 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.3173000000000001e-05 2024-09-14T06:31:58.773 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:58.773 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:58.773 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:31:58.773 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065005+0000", 2024-09-14T06:31:58.773 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.9960000000000001e-06 2024-09-14T06:31:58.773 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:58.773 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:58.773 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:31:58.773 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.110314+0000", 2024-09-14T06:31:58.773 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.04530899 2024-09-14T06:31:58.773 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:58.773 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:58.773 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:31:58.774 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.110398+0000", 2024-09-14T06:31:58.774 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.3539000000000001e-05 2024-09-14T06:31:58.774 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:58.774 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:58.774 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:31:58.774 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.110471+0000", 2024-09-14T06:31:58.774 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.3455999999999996e-05 2024-09-14T06:31:58.774 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:58.774 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:31:58.774 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:58.774 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:58.774 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:58.774 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:19365 3.3 3:c34e5e51:::rbd_data.112d92027590.0000000000000024:f [read 1302528~1024 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:31:58.774 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:27.065058+0000", 2024-09-14T06:31:58.775 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 151.65379217200001, 2024-09-14T06:31:58.775 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046105526000000001, 2024-09-14T06:31:58.775 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:31:58.775 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:31:58.775 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:31:58.775 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:31:58.775 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:31:58.775 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:58.775 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 19365 2024-09-14T06:31:58.775 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:58.775 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:31:58.775 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:58.775 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:31:58.775 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065058+0000", 2024-09-14T06:31:58.775 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:58.776 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:58.776 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:58.776 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:31:58.776 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065058+0000", 2024-09-14T06:31:58.776 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:58.776 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:58.776 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:58.776 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:31:58.776 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065058+0000", 2024-09-14T06:31:58.776 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.1899999999999999e-07 2024-09-14T06:31:58.776 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:58.776 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:58.776 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:31:58.777 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065059+0000", 2024-09-14T06:31:58.777 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.1299999999999998e-07 2024-09-14T06:31:58.777 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:58.777 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:58.777 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:31:58.777 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065063+0000", 2024-09-14T06:31:58.777 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.1099999999999996e-06 2024-09-14T06:31:58.777 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:58.777 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:58.777 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:31:58.777 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065070+0000", 2024-09-14T06:31:58.777 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.973e-06 2024-09-14T06:31:58.777 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:58.777 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:58.778 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:31:58.778 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.111014+0000", 2024-09-14T06:31:58.778 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.045943373000000003 2024-09-14T06:31:58.778 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:58.778 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:58.778 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:31:58.778 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.111073+0000", 2024-09-14T06:31:58.778 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.8987999999999998e-05 2024-09-14T06:31:58.778 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:58.778 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:58.778 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:31:58.778 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.111163+0000", 2024-09-14T06:31:58.778 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.0950000000000002e-05 2024-09-14T06:31:58.779 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:58.779 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:31:58.779 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:58.779 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:58.779 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:58.779 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:19366 3.3 3:c34e5e51:::rbd_data.112d92027590.0000000000000024:f [read 1312768~1024 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:31:58.779 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:27.065074+0000", 2024-09-14T06:31:58.779 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 151.653776322, 2024-09-14T06:31:58.779 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046289858000000003, 2024-09-14T06:31:58.779 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:31:58.779 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:31:58.779 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:31:58.779 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:31:58.780 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:31:58.780 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:58.780 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 19366 2024-09-14T06:31:58.780 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:58.780 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:31:58.780 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:58.780 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:31:58.780 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065074+0000", 2024-09-14T06:31:58.780 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:58.780 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:58.780 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:58.780 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:31:58.780 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065074+0000", 2024-09-14T06:31:58.780 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:58.781 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:58.781 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:58.781 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:31:58.781 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065074+0000", 2024-09-14T06:31:58.781 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.3900000000000001e-07 2024-09-14T06:31:58.781 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:58.781 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:58.781 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:31:58.781 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065078+0000", 2024-09-14T06:31:58.781 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.3229999999999999e-06 2024-09-14T06:31:58.781 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:58.781 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:58.781 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:31:58.781 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065082+0000", 2024-09-14T06:31:58.782 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.8299999999999998e-06 2024-09-14T06:31:58.782 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:58.782 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:58.782 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:31:58.782 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065085+0000", 2024-09-14T06:31:58.782 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.8660000000000002e-06 2024-09-14T06:31:58.782 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:58.782 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:58.782 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:31:58.782 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.111270+0000", 2024-09-14T06:31:58.782 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046185035999999999 2024-09-14T06:31:58.782 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:58.782 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:58.782 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:31:58.783 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.111311+0000", 2024-09-14T06:31:58.783 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.0935e-05 2024-09-14T06:31:58.783 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:58.783 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:58.783 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:31:58.783 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.111364+0000", 2024-09-14T06:31:58.783 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.2629e-05 2024-09-14T06:31:58.783 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:58.783 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:31:58.783 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:58.783 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:58.783 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:58.783 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:19367 3.3 3:c34e5e51:::rbd_data.112d92027590.0000000000000024:f [read 1323008~1024 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:31:58.783 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:27.065115+0000", 2024-09-14T06:31:58.784 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 151.65373476100001, 2024-09-14T06:31:58.784 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046407003000000002, 2024-09-14T06:31:58.784 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:31:58.784 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:31:58.784 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:31:58.784 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:31:58.784 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:31:58.784 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:58.784 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 19367 2024-09-14T06:31:58.784 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:58.784 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:31:58.784 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:58.784 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:31:58.784 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065115+0000", 2024-09-14T06:31:58.785 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:58.785 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:58.785 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:58.785 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:31:58.785 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065115+0000", 2024-09-14T06:31:58.785 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:58.785 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:58.785 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:58.785 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:31:58.785 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065116+0000", 2024-09-14T06:31:58.786 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.4999999999999998e-07 2024-09-14T06:31:58.786 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:58.786 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:58.786 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:31:58.786 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065117+0000", 2024-09-14T06:31:58.786 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.189e-06 2024-09-14T06:31:58.786 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:58.786 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:58.786 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:31:58.786 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065122+0000", 2024-09-14T06:31:58.786 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.3709999999999999e-06 2024-09-14T06:31:58.786 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:58.786 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:58.787 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:31:58.787 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065125+0000", 2024-09-14T06:31:58.787 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.8940000000000002e-06 2024-09-14T06:31:58.787 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:58.787 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:58.787 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:31:58.787 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.111402+0000", 2024-09-14T06:31:58.787 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046276645999999998 2024-09-14T06:31:58.787 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:58.787 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:58.787 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:31:58.787 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.111445+0000", 2024-09-14T06:31:58.787 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.3288000000000003e-05 2024-09-14T06:31:58.787 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:58.788 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:58.788 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:31:58.788 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.111522+0000", 2024-09-14T06:31:58.788 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.7164999999999997e-05 2024-09-14T06:31:58.788 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:58.788 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:31:58.788 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:58.788 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:58.788 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:58.788 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:19368 3.3 3:c34e5e51:::rbd_data.112d92027590.0000000000000024:f [read 1333248~1024 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:31:58.788 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:27.065130+0000", 2024-09-14T06:31:58.788 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 151.65372047100001, 2024-09-14T06:31:58.788 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046507275000000001, 2024-09-14T06:31:58.789 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:31:58.789 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:31:58.789 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:31:58.789 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:31:58.789 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:31:58.789 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:58.789 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 19368 2024-09-14T06:31:58.789 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:58.789 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:31:58.789 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:58.789 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:31:58.789 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065130+0000", 2024-09-14T06:31:58.789 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:58.789 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:58.789 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:58.790 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:31:58.790 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065130+0000", 2024-09-14T06:31:58.790 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:58.790 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:58.790 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:58.790 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:31:58.790 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065130+0000", 2024-09-14T06:31:58.790 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.0800000000000001e-07 2024-09-14T06:31:58.790 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:58.790 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:58.790 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:31:58.790 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065131+0000", 2024-09-14T06:31:58.790 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.8700000000000004e-07 2024-09-14T06:31:58.790 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:58.791 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:58.791 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:31:58.791 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065134+0000", 2024-09-14T06:31:58.791 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.1039999999999998e-06 2024-09-14T06:31:58.791 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:58.791 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:58.791 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:31:58.791 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065136+0000", 2024-09-14T06:31:58.791 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.3949999999999999e-06 2024-09-14T06:31:58.791 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:58.791 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:58.791 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:31:58.791 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.111541+0000", 2024-09-14T06:31:58.791 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046404206000000003 2024-09-14T06:31:58.792 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:58.792 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:58.792 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:31:58.792 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.111582+0000", 2024-09-14T06:31:58.792 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.1051999999999999e-05 2024-09-14T06:31:58.792 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:58.792 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:58.792 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:31:58.792 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.111637+0000", 2024-09-14T06:31:58.792 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.5322999999999999e-05 2024-09-14T06:31:58.792 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:58.792 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:31:58.792 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:58.792 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:58.793 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:58.793 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:19369 3.3 3:c34e5e51:::rbd_data.112d92027590.0000000000000024:f [read 1343488~1024 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:31:58.793 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:27.065139+0000", 2024-09-14T06:31:58.793 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 151.65371065400001, 2024-09-14T06:31:58.793 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046682169000000003, 2024-09-14T06:31:58.793 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:31:58.793 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:31:58.793 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:31:58.793 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:31:58.793 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:31:58.793 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:58.793 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 19369 2024-09-14T06:31:58.793 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:58.793 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:31:58.794 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:58.794 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:31:58.794 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065139+0000", 2024-09-14T06:31:58.794 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:58.794 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:58.794 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:58.794 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:31:58.794 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065139+0000", 2024-09-14T06:31:58.794 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:58.794 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:58.794 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:58.794 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:31:58.794 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065140+0000", 2024-09-14T06:31:58.794 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.8299999999999998e-07 2024-09-14T06:31:58.795 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:58.795 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:58.795 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:31:58.795 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065141+0000", 2024-09-14T06:31:58.795 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.3900000000000003e-07 2024-09-14T06:31:58.795 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:58.795 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:58.795 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:31:58.795 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065144+0000", 2024-09-14T06:31:58.795 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.7409999999999998e-06 2024-09-14T06:31:58.795 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:58.795 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:58.795 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:31:58.795 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065147+0000", 2024-09-14T06:31:58.796 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.4150000000000002e-06 2024-09-14T06:31:58.796 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:58.796 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:58.796 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:31:58.796 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.111673+0000", 2024-09-14T06:31:58.796 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046526004000000003 2024-09-14T06:31:58.796 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:58.796 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:58.796 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:31:58.796 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.111749+0000", 2024-09-14T06:31:58.796 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.5741999999999994e-05 2024-09-14T06:31:58.796 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:58.796 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:58.796 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:31:58.796 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.111822+0000", 2024-09-14T06:31:58.797 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.3045000000000003e-05 2024-09-14T06:31:58.797 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:58.797 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:31:58.797 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:58.797 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:58.797 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:58.797 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:19370 3.3 3:c34e5e51:::rbd_data.112d92027590.0000000000000024:f [read 1353728~1024 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:31:58.797 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:27.065150+0000", 2024-09-14T06:31:58.797 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 151.65369992500001, 2024-09-14T06:31:58.797 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046897556999999999, 2024-09-14T06:31:58.797 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:31:58.797 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:31:58.797 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:31:58.797 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:31:58.798 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:31:58.798 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:58.798 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 19370 2024-09-14T06:31:58.798 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:58.798 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:31:58.798 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:58.798 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:31:58.798 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065150+0000", 2024-09-14T06:31:58.798 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:58.798 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:58.798 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:58.798 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:31:58.798 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065150+0000", 2024-09-14T06:31:58.798 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:58.798 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:58.798 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:58.799 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:31:58.799 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065150+0000", 2024-09-14T06:31:58.799 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.0200000000000001e-07 2024-09-14T06:31:58.799 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:58.799 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:58.799 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:31:58.799 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065151+0000", 2024-09-14T06:31:58.799 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.1100000000000004e-07 2024-09-14T06:31:58.799 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:58.799 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:58.799 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:31:58.799 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065154+0000", 2024-09-14T06:31:58.799 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.9639999999999999e-06 2024-09-14T06:31:58.799 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:58.799 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:58.799 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:31:58.800 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065157+0000", 2024-09-14T06:31:58.800 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.4260000000000002e-06 2024-09-14T06:31:58.800 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:58.800 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:58.800 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:31:58.800 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.111861+0000", 2024-09-14T06:31:58.800 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046704268 2024-09-14T06:31:58.800 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:58.800 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:58.800 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:31:58.800 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.111939+0000", 2024-09-14T06:31:58.800 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.8231000000000004e-05 2024-09-14T06:31:58.800 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:58.800 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:58.801 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:31:58.801 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.112048+0000", 2024-09-14T06:31:58.801 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00010855500000000001 2024-09-14T06:31:58.801 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:58.801 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:31:58.801 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:58.801 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:58.801 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:58.801 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:19371 3.3 3:c34e5e51:::rbd_data.112d92027590.0000000000000024:f [read 1363968~1024 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:31:58.801 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:27.065161+0000", 2024-09-14T06:31:58.801 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 151.653689275, 2024-09-14T06:31:58.801 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047021944000000003, 2024-09-14T06:31:58.801 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:31:58.801 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:31:58.801 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:31:58.802 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:31:58.802 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:31:58.802 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:58.802 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 19371 2024-09-14T06:31:58.802 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:58.802 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:31:58.802 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:58.802 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:31:58.802 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065161+0000", 2024-09-14T06:31:58.802 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:58.802 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:58.802 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:58.802 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:31:58.802 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065161+0000", 2024-09-14T06:31:58.802 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:58.803 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:58.803 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:58.803 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:31:58.803 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065161+0000", 2024-09-14T06:31:58.803 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.0200000000000001e-07 2024-09-14T06:31:58.803 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:58.803 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:58.803 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:31:58.803 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065166+0000", 2024-09-14T06:31:58.803 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.9869999999999999e-06 2024-09-14T06:31:58.803 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:58.803 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:58.803 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:31:58.803 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065169+0000", 2024-09-14T06:31:58.804 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.3799999999999998e-06 2024-09-14T06:31:58.804 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:58.804 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:58.804 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:31:58.804 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065172+0000", 2024-09-14T06:31:58.804 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.4830000000000002e-06 2024-09-14T06:31:58.804 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:58.804 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:58.804 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:31:58.804 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.112071+0000", 2024-09-14T06:31:58.804 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046899529000000002 2024-09-14T06:31:58.804 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:58.804 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:58.804 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:31:58.805 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.112128+0000", 2024-09-14T06:31:58.805 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.6341999999999997e-05 2024-09-14T06:31:58.805 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:58.805 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:58.805 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:31:58.805 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.112183+0000", 2024-09-14T06:31:58.805 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.5021000000000001e-05 2024-09-14T06:31:58.805 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:58.805 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:31:58.805 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:58.805 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:58.805 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:58.805 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:19372 3.3 3:c34e5e51:::rbd_data.112d92027590.0000000000000024:f [read 1374208~1024 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:31:58.806 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:27.065176+0000", 2024-09-14T06:31:58.806 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 151.65367412500001, 2024-09-14T06:31:58.806 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047188889999999997, 2024-09-14T06:31:58.806 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:31:58.806 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:31:58.806 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:31:58.806 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:31:58.806 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:31:58.806 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:58.806 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 19372 2024-09-14T06:31:58.806 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:58.806 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:31:58.806 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:58.806 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:31:58.806 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065176+0000", 2024-09-14T06:31:58.807 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:58.807 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:58.807 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:58.807 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:31:58.807 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065176+0000", 2024-09-14T06:31:58.807 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:58.807 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:58.807 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:58.807 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:31:58.807 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.071117+0000", 2024-09-14T06:31:58.807 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.0059404990000000001 2024-09-14T06:31:58.807 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:58.807 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:58.807 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:31:58.808 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.071120+0000", 2024-09-14T06:31:58.808 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.4340000000000001e-06 2024-09-14T06:31:58.808 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:58.808 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:58.808 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:31:58.808 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.071129+0000", 2024-09-14T06:31:58.808 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.8389999999999996e-06 2024-09-14T06:31:58.808 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:58.808 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:58.808 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:31:58.808 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.071134+0000", 2024-09-14T06:31:58.808 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.412e-06 2024-09-14T06:31:58.808 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:58.808 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:58.809 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:31:58.809 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.112221+0000", 2024-09-14T06:31:58.809 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.041086473999999998 2024-09-14T06:31:58.809 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:58.809 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:58.809 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:31:58.809 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.112299+0000", 2024-09-14T06:31:58.809 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.8746999999999997e-05 2024-09-14T06:31:58.809 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:58.809 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:58.809 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:31:58.809 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.112365+0000", 2024-09-14T06:31:58.809 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.5485000000000001e-05 2024-09-14T06:31:58.809 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:58.809 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:31:58.810 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:58.810 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:58.810 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:31:58.810 INFO:teuthology.orchestra.run.smithi138.stdout:} 2024-09-14T06:31:58.810 INFO:tasks.ceph.osd.1.smithi094.stderr:2024-09-14T06:31:58.719+0000 7f328e47a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33641) UID: 0 2024-09-14T06:31:58.811 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-14T06:31:58.855 INFO:tasks.ceph.osd.7.smithi138.stderr:2024-09-14T06:31:58.854+0000 7f3c6a21d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33623) UID: 0 2024-09-14T06:31:58.889 INFO:teuthology.orchestra.run.smithi094.stdout:checking clone #41, image image_client.0-clone41 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent42 2024-09-14T06:31:58.916 INFO:teuthology.orchestra.run.smithi138.stdout:{ 2024-09-14T06:31:58.916 INFO:teuthology.orchestra.run.smithi138.stdout: "ops": [], 2024-09-14T06:31:58.916 INFO:teuthology.orchestra.run.smithi138.stdout: "num_ops": 0 2024-09-14T06:31:58.916 INFO:teuthology.orchestra.run.smithi138.stdout:} 2024-09-14T06:31:58.921 INFO:tasks.ceph.osd.5.smithi138.stderr:2024-09-14T06:31:58.920+0000 7fe13746f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33618) UID: 0 2024-09-14T06:31:58.925 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-14T06:31:59.023 INFO:tasks.ceph.osd.0.smithi094.stderr:2024-09-14T06:31:59.022+0000 7f07c9333640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33636) UID: 0 2024-09-14T06:31:59.068 INFO:teuthology.orchestra.run.smithi138.stdout:{ 2024-09-14T06:31:59.068 INFO:teuthology.orchestra.run.smithi138.stdout: "ops": [], 2024-09-14T06:31:59.068 INFO:teuthology.orchestra.run.smithi138.stdout: "complaint_time": 30, 2024-09-14T06:31:59.068 INFO:teuthology.orchestra.run.smithi138.stdout: "num_blocked_ops": 0 2024-09-14T06:31:59.068 INFO:teuthology.orchestra.run.smithi138.stdout:} 2024-09-14T06:31:59.076 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-14T06:31:59.084 INFO:teuthology.orchestra.run.smithi094.stdout:5700 read 0x365c00 thru 0x369268 (0x3669 bytes) 2024-09-14T06:31:59.086 INFO:teuthology.orchestra.run.smithi094.stdout:5701 write 0x94fd717 thru 0x950465a (0x6f44 bytes) 2024-09-14T06:31:59.092 INFO:teuthology.orchestra.run.smithi094.stdout:5702 read 0x2248e34 thru 0x224cb60 (0x3d2d bytes) 2024-09-14T06:31:59.092 INFO:teuthology.orchestra.run.smithi094.stdout:5703 punch from 0x2e58bc9 to 0x2e5b946, (0x2d7d bytes) 2024-09-14T06:31:59.092 INFO:teuthology.orchestra.run.smithi094.stdout:5704 clone 44 order 22 su 131072 sc 5 2024-09-14T06:31:59.124 INFO:tasks.ceph.osd.0.smithi094.stderr:2024-09-14T06:31:59.123+0000 7f07c9333640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33636) UID: 0 2024-09-14T06:31:59.181 INFO:teuthology.orchestra.run.smithi138.stdout:{ 2024-09-14T06:31:59.181 INFO:teuthology.orchestra.run.smithi138.stdout: "size": 20, 2024-09-14T06:31:59.181 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 600, 2024-09-14T06:31:59.181 INFO:teuthology.orchestra.run.smithi138.stdout: "ops": [ 2024-09-14T06:31:59.181 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:59.181 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:11453 3.1d 3:bcad7303:::rbd_data.112d92027590.0000000000000001:f [read 94208~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:31:59.181 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:25.654375+0000", 2024-09-14T06:31:59.181 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 153.524680265, 2024-09-14T06:31:59.181 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052449318000000002, 2024-09-14T06:31:59.182 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:31:59.182 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:31:59.182 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:31:59.182 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:31:59.182 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:31:59.182 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:59.182 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 11453 2024-09-14T06:31:59.182 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:59.182 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:31:59.182 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:59.182 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:31:59.182 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654375+0000", 2024-09-14T06:31:59.182 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:59.183 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:59.183 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:59.183 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:31:59.183 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654375+0000", 2024-09-14T06:31:59.183 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:59.183 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:59.183 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:59.183 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:31:59.183 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654496+0000", 2024-09-14T06:31:59.183 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00012081 2024-09-14T06:31:59.183 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:59.183 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:59.183 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:31:59.183 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654497+0000", 2024-09-14T06:31:59.183 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.8789999999999999e-06 2024-09-14T06:31:59.184 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:59.184 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:59.184 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:31:59.184 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654501+0000", 2024-09-14T06:31:59.184 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.8999999999999999e-06 2024-09-14T06:31:59.184 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:59.184 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:59.184 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:31:59.184 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654505+0000", 2024-09-14T06:31:59.184 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.2229999999999999e-06 2024-09-14T06:31:59.184 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:59.184 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:59.184 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:31:59.184 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.705917+0000", 2024-09-14T06:31:59.185 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.051412501999999999 2024-09-14T06:31:59.185 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:59.185 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:59.185 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:31:59.185 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.705999+0000", 2024-09-14T06:31:59.185 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.1551999999999994e-05 2024-09-14T06:31:59.185 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:59.185 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:59.185 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:31:59.185 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.706824+0000", 2024-09-14T06:31:59.185 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00082545199999999996 2024-09-14T06:31:59.185 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:59.185 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:31:59.185 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:59.186 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:59.186 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:59.186 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:11454 3.1d 3:bcad7303:::rbd_data.112d92027590.0000000000000001:f [read 104448~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:31:59.186 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:25.654510+0000", 2024-09-14T06:31:59.186 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 153.524545475, 2024-09-14T06:31:59.186 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052610713000000003, 2024-09-14T06:31:59.186 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:31:59.186 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:31:59.186 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:31:59.186 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:31:59.186 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:31:59.186 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:59.186 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 11454 2024-09-14T06:31:59.187 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:59.187 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:31:59.187 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:59.187 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:31:59.187 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654510+0000", 2024-09-14T06:31:59.187 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:59.187 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:59.187 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:59.187 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:31:59.187 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654510+0000", 2024-09-14T06:31:59.187 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:59.187 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:59.187 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:59.187 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:31:59.188 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654825+0000", 2024-09-14T06:31:59.188 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00031522400000000002 2024-09-14T06:31:59.188 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:59.188 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:59.188 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:31:59.188 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654827+0000", 2024-09-14T06:31:59.188 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.9709999999999998e-06 2024-09-14T06:31:59.188 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:59.188 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:59.188 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:31:59.188 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654832+0000", 2024-09-14T06:31:59.188 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.4099999999999999e-06 2024-09-14T06:31:59.188 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:59.189 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:59.189 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:31:59.189 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654835+0000", 2024-09-14T06:31:59.189 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.14e-06 2024-09-14T06:31:59.189 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:59.189 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:59.189 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:31:59.189 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.706851+0000", 2024-09-14T06:31:59.189 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052015761000000001 2024-09-14T06:31:59.189 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:59.189 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:59.189 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:31:59.189 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.706934+0000", 2024-09-14T06:31:59.189 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.2484999999999994e-05 2024-09-14T06:31:59.190 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:59.190 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:59.190 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:31:59.190 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.707120+0000", 2024-09-14T06:31:59.190 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00018672200000000001 2024-09-14T06:31:59.190 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:59.190 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:31:59.190 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:59.190 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:59.190 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:59.190 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:11455 3.1d 3:bcad7303:::rbd_data.112d92027590.0000000000000001:f [read 114688~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:31:59.190 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:25.654841+0000", 2024-09-14T06:31:59.190 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 153.52421414599999, 2024-09-14T06:31:59.190 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052579898, 2024-09-14T06:31:59.191 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:31:59.191 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:31:59.191 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:31:59.191 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:31:59.191 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:31:59.191 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:59.191 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 11455 2024-09-14T06:31:59.191 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:59.191 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:31:59.191 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:59.191 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:31:59.191 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654841+0000", 2024-09-14T06:31:59.191 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:59.191 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:59.191 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:59.192 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:31:59.192 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654841+0000", 2024-09-14T06:31:59.192 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:59.192 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:59.192 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:59.192 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:31:59.192 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654841+0000", 2024-09-14T06:31:59.192 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.7300000000000002e-07 2024-09-14T06:31:59.192 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:59.192 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:59.192 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:31:59.192 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654845+0000", 2024-09-14T06:31:59.192 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.0520000000000003e-06 2024-09-14T06:31:59.192 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:59.193 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:59.193 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:31:59.193 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654849+0000", 2024-09-14T06:31:59.193 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.0459999999999999e-06 2024-09-14T06:31:59.193 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:59.193 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:59.193 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:31:59.193 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654852+0000", 2024-09-14T06:31:59.193 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.9830000000000001e-06 2024-09-14T06:31:59.193 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:59.193 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:59.193 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:31:59.193 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.707159+0000", 2024-09-14T06:31:59.193 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052306826000000001 2024-09-14T06:31:59.194 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:59.194 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:59.194 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:31:59.194 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.707242+0000", 2024-09-14T06:31:59.194 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.2595000000000005e-05 2024-09-14T06:31:59.194 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:59.194 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:59.194 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:31:59.194 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.707421+0000", 2024-09-14T06:31:59.194 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.000179123 2024-09-14T06:31:59.194 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:59.194 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:31:59.194 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:59.194 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:59.194 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:59.195 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:11456 3.1d 3:bcad7303:::rbd_data.112d92027590.0000000000000001:f [read 124928~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:31:59.195 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:25.654856+0000", 2024-09-14T06:31:59.195 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 153.52419929999999, 2024-09-14T06:31:59.195 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052801580000000001, 2024-09-14T06:31:59.195 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:31:59.195 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:31:59.195 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:31:59.195 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:31:59.195 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:31:59.195 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:59.195 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 11456 2024-09-14T06:31:59.195 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:59.195 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:31:59.195 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:59.196 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:31:59.196 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654856+0000", 2024-09-14T06:31:59.196 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:59.196 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:59.196 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:59.196 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:31:59.196 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654856+0000", 2024-09-14T06:31:59.196 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:59.196 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:59.196 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:59.196 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:31:59.196 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654856+0000", 2024-09-14T06:31:59.197 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.29e-07 2024-09-14T06:31:59.197 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:59.197 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:59.197 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:31:59.197 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654857+0000", 2024-09-14T06:31:59.197 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.187e-06 2024-09-14T06:31:59.197 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:59.197 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:59.197 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:31:59.197 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654861+0000", 2024-09-14T06:31:59.197 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.3850000000000002e-06 2024-09-14T06:31:59.197 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:59.197 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:59.198 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:31:59.198 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654863+0000", 2024-09-14T06:31:59.198 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.9500000000000001e-06 2024-09-14T06:31:59.198 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:59.198 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:59.198 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:31:59.198 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.707456+0000", 2024-09-14T06:31:59.198 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052592434 2024-09-14T06:31:59.198 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:59.198 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:59.198 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:31:59.198 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.707516+0000", 2024-09-14T06:31:59.198 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.0167999999999997e-05 2024-09-14T06:31:59.198 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:59.198 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:59.199 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:31:59.199 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.707657+0000", 2024-09-14T06:31:59.199 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00014122700000000001 2024-09-14T06:31:59.199 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:59.199 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:31:59.199 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:59.199 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:59.199 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:59.199 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:11457 3.1d 3:bcad7303:::rbd_data.112d92027590.0000000000000001:f [read 135168~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:31:59.199 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:25.654867+0000", 2024-09-14T06:31:59.199 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 153.524188415, 2024-09-14T06:31:59.199 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052921902, 2024-09-14T06:31:59.199 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:31:59.199 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:31:59.200 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:31:59.200 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:31:59.200 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:31:59.200 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:59.200 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 11457 2024-09-14T06:31:59.200 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:59.200 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:31:59.200 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:59.200 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:31:59.200 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654867+0000", 2024-09-14T06:31:59.200 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:59.200 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:59.200 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:59.200 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:31:59.201 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654867+0000", 2024-09-14T06:31:59.201 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:59.201 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:59.201 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:59.201 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:31:59.201 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654972+0000", 2024-09-14T06:31:59.201 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00010584 2024-09-14T06:31:59.201 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:59.201 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:59.201 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:31:59.201 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654975+0000", 2024-09-14T06:31:59.201 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.1169999999999998e-06 2024-09-14T06:31:59.201 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:59.202 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:59.202 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:31:59.202 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654979+0000", 2024-09-14T06:31:59.202 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.1960000000000001e-06 2024-09-14T06:31:59.202 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:59.202 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:59.202 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:31:59.202 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654982+0000", 2024-09-14T06:31:59.202 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.2990000000000001e-06 2024-09-14T06:31:59.202 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:59.202 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:59.202 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:31:59.202 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.707684+0000", 2024-09-14T06:31:59.202 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052701979000000003 2024-09-14T06:31:59.203 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:59.203 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:59.203 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:31:59.203 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.707728+0000", 2024-09-14T06:31:59.203 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.3529999999999998e-05 2024-09-14T06:31:59.203 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:59.203 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:59.203 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:31:59.203 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.707789+0000", 2024-09-14T06:31:59.203 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.0940999999999998e-05 2024-09-14T06:31:59.203 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:59.203 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:31:59.203 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:59.203 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:59.203 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:59.204 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:11458 3.1d 3:bcad7303:::rbd_data.112d92027590.0000000000000001:f [read 145408~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:31:59.204 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:25.654987+0000", 2024-09-14T06:31:59.204 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 153.52406801699999, 2024-09-14T06:31:59.204 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.05291655, 2024-09-14T06:31:59.204 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:31:59.204 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:31:59.204 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:31:59.204 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:31:59.204 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:31:59.204 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:59.204 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 11458 2024-09-14T06:31:59.204 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:59.204 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:31:59.204 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:59.205 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:31:59.205 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654987+0000", 2024-09-14T06:31:59.205 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:59.205 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:59.205 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:59.205 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:31:59.205 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654987+0000", 2024-09-14T06:31:59.205 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:59.205 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:59.205 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:59.205 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:31:59.205 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655077+0000", 2024-09-14T06:31:59.205 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.0253000000000005e-05 2024-09-14T06:31:59.206 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:59.206 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:59.206 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:31:59.206 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655079+0000", 2024-09-14T06:31:59.206 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.9989999999999998e-06 2024-09-14T06:31:59.206 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:59.206 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:59.206 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:31:59.206 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655083+0000", 2024-09-14T06:31:59.206 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.9280000000000003e-06 2024-09-14T06:31:59.206 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:59.206 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:59.206 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:31:59.206 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655086+0000", 2024-09-14T06:31:59.207 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.056e-06 2024-09-14T06:31:59.207 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:59.207 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:59.207 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:31:59.207 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.707805+0000", 2024-09-14T06:31:59.207 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052718627999999997 2024-09-14T06:31:59.207 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:59.207 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:59.207 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:31:59.207 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.707848+0000", 2024-09-14T06:31:59.207 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.2728999999999997e-05 2024-09-14T06:31:59.207 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:59.207 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:59.207 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:31:59.207 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.707904+0000", 2024-09-14T06:31:59.208 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.5957e-05 2024-09-14T06:31:59.208 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:59.208 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:31:59.208 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:59.208 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:59.208 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:59.208 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:11459 3.1d 3:bcad7303:::rbd_data.112d92027590.0000000000000001:f [read 155648~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:31:59.208 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:25.655091+0000", 2024-09-14T06:31:59.208 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 153.523964009, 2024-09-14T06:31:59.208 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052944242000000002, 2024-09-14T06:31:59.208 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:31:59.208 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:31:59.208 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:31:59.208 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:31:59.208 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:31:59.209 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:59.209 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 11459 2024-09-14T06:31:59.209 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:59.209 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:31:59.209 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:59.209 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:31:59.209 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655091+0000", 2024-09-14T06:31:59.209 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:59.209 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:59.209 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:59.209 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:31:59.209 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655091+0000", 2024-09-14T06:31:59.209 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:59.210 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:59.210 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:59.210 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:31:59.210 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655237+0000", 2024-09-14T06:31:59.210 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00014570100000000001 2024-09-14T06:31:59.210 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:59.210 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:59.210 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:31:59.210 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655239+0000", 2024-09-14T06:31:59.210 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.8670000000000001e-06 2024-09-14T06:31:59.210 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:59.210 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:59.210 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:31:59.211 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655243+0000", 2024-09-14T06:31:59.211 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.9829999999999998e-06 2024-09-14T06:31:59.211 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:59.211 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:59.211 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:31:59.211 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655246+0000", 2024-09-14T06:31:59.211 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.1540000000000002e-06 2024-09-14T06:31:59.211 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:59.211 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:59.211 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:31:59.211 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.707934+0000", 2024-09-14T06:31:59.211 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052688286000000001 2024-09-14T06:31:59.211 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:59.211 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:59.211 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:31:59.212 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.707977+0000", 2024-09-14T06:31:59.212 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.3291999999999998e-05 2024-09-14T06:31:59.212 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:59.212 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:59.212 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:31:59.212 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.708035+0000", 2024-09-14T06:31:59.212 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.7958999999999999e-05 2024-09-14T06:31:59.212 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:59.212 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:31:59.212 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:59.212 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:59.212 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:59.212 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:11460 3.1d 3:bcad7303:::rbd_data.112d92027590.0000000000000001:f [read 165888~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:31:59.213 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:25.655250+0000", 2024-09-14T06:31:59.213 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 153.52380471199999, 2024-09-14T06:31:59.213 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052910698999999999, 2024-09-14T06:31:59.213 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:31:59.213 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:31:59.213 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:31:59.213 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:31:59.213 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:31:59.213 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:59.213 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 11460 2024-09-14T06:31:59.213 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:59.214 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:31:59.214 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:59.214 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:31:59.214 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655250+0000", 2024-09-14T06:31:59.214 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:59.214 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:59.214 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:59.214 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:31:59.214 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655250+0000", 2024-09-14T06:31:59.214 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:59.214 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:59.214 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:59.214 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:31:59.215 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655356+0000", 2024-09-14T06:31:59.215 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00010535900000000001 2024-09-14T06:31:59.215 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:59.215 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:59.215 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:31:59.215 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655357+0000", 2024-09-14T06:31:59.215 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.8080000000000001e-06 2024-09-14T06:31:59.215 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:59.215 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:59.215 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:31:59.215 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655361+0000", 2024-09-14T06:31:59.215 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.861e-06 2024-09-14T06:31:59.215 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:59.216 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:59.216 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:31:59.216 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655364+0000", 2024-09-14T06:31:59.216 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.1089999999999998e-06 2024-09-14T06:31:59.216 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:59.216 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:59.216 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:31:59.216 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.708065+0000", 2024-09-14T06:31:59.216 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052700595000000003 2024-09-14T06:31:59.216 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:59.216 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:59.216 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:31:59.216 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.708108+0000", 2024-09-14T06:31:59.217 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.2484999999999997e-05 2024-09-14T06:31:59.217 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:59.217 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:59.217 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:31:59.217 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.708161+0000", 2024-09-14T06:31:59.217 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.3482000000000001e-05 2024-09-14T06:31:59.217 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:59.217 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:31:59.217 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:59.217 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:59.217 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:59.217 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:11461 3.1d 3:bcad7303:::rbd_data.112d92027590.0000000000000001:f [read 176128~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:31:59.217 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:25.655370+0000", 2024-09-14T06:31:59.217 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 153.52368543, 2024-09-14T06:31:59.217 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052915141999999998, 2024-09-14T06:31:59.218 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:31:59.218 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:31:59.218 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:31:59.218 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:31:59.218 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:31:59.218 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:59.218 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 11461 2024-09-14T06:31:59.218 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:59.218 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:31:59.218 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:59.218 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:31:59.218 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655370+0000", 2024-09-14T06:31:59.218 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:59.218 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:59.218 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:59.219 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:31:59.219 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655370+0000", 2024-09-14T06:31:59.219 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:59.219 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:59.219 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:59.219 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:31:59.219 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655394+0000", 2024-09-14T06:31:59.219 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.4553999999999999e-05 2024-09-14T06:31:59.219 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:59.219 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:59.219 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:31:59.219 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655396+0000", 2024-09-14T06:31:59.219 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.8950000000000001e-06 2024-09-14T06:31:59.219 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:59.219 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:59.220 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:31:59.220 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655401+0000", 2024-09-14T06:31:59.220 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.0089999999999998e-06 2024-09-14T06:31:59.220 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:59.220 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:59.220 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:31:59.220 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655404+0000", 2024-09-14T06:31:59.220 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.3400000000000002e-06 2024-09-14T06:31:59.220 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:59.220 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:59.220 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:31:59.220 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.708186+0000", 2024-09-14T06:31:59.220 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052781726000000001 2024-09-14T06:31:59.220 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:59.221 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:59.221 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:31:59.221 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.708229+0000", 2024-09-14T06:31:59.221 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.2670999999999999e-05 2024-09-14T06:31:59.221 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:59.221 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:59.221 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:31:59.221 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.708285+0000", 2024-09-14T06:31:59.221 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.5946999999999998e-05 2024-09-14T06:31:59.221 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:59.221 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:31:59.221 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:59.221 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:59.221 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:59.222 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:11462 3.1d 3:bcad7303:::rbd_data.112d92027590.0000000000000001:f [read 186368~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:31:59.222 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:25.655410+0000", 2024-09-14T06:31:59.222 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 153.52364495200001, 2024-09-14T06:31:59.222 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052995258000000003, 2024-09-14T06:31:59.222 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:31:59.222 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:31:59.222 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:31:59.222 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:31:59.222 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:31:59.222 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:59.222 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 11462 2024-09-14T06:31:59.222 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:59.222 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:31:59.222 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:59.223 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:31:59.223 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655410+0000", 2024-09-14T06:31:59.223 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:59.223 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:59.223 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:59.223 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:31:59.223 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655410+0000", 2024-09-14T06:31:59.223 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:59.223 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:59.223 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:59.223 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:31:59.223 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655625+0000", 2024-09-14T06:31:59.223 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00021442600000000001 2024-09-14T06:31:59.223 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:59.224 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:59.224 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:31:59.224 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655626+0000", 2024-09-14T06:31:59.224 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.815e-06 2024-09-14T06:31:59.224 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:59.224 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:59.224 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:31:59.224 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655631+0000", 2024-09-14T06:31:59.224 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.651e-06 2024-09-14T06:31:59.224 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:59.224 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:59.224 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:31:59.224 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655634+0000", 2024-09-14T06:31:59.224 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.2449999999999998e-06 2024-09-14T06:31:59.225 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:59.225 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:59.225 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:31:59.225 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.708312+0000", 2024-09-14T06:31:59.225 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052677412 2024-09-14T06:31:59.225 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:59.225 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:59.225 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:31:59.225 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.708352+0000", 2024-09-14T06:31:59.225 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.0729999999999998e-05 2024-09-14T06:31:59.225 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:59.225 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:59.225 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:31:59.226 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.708405+0000", 2024-09-14T06:31:59.226 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.2979000000000002e-05 2024-09-14T06:31:59.226 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:59.226 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:31:59.226 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:59.226 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:59.226 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:59.226 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:11463 3.1d 3:bcad7303:::rbd_data.112d92027590.0000000000000001:f [read 196608~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:31:59.226 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:25.655639+0000", 2024-09-14T06:31:59.226 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 153.523415806, 2024-09-14T06:31:59.226 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052888859000000003, 2024-09-14T06:31:59.226 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:31:59.226 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:31:59.226 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:31:59.226 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:31:59.227 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:31:59.227 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:59.227 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 11463 2024-09-14T06:31:59.227 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:59.227 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:31:59.227 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:59.227 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:31:59.227 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655639+0000", 2024-09-14T06:31:59.227 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:59.227 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:59.227 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:59.227 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:31:59.227 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655639+0000", 2024-09-14T06:31:59.227 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:59.228 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:59.228 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:59.228 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:31:59.228 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655640+0000", 2024-09-14T06:31:59.228 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.08e-07 2024-09-14T06:31:59.228 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:59.228 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:59.228 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:31:59.228 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655641+0000", 2024-09-14T06:31:59.228 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.156e-06 2024-09-14T06:31:59.228 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:59.228 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:59.228 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:31:59.229 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655648+0000", 2024-09-14T06:31:59.229 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.8460000000000002e-06 2024-09-14T06:31:59.229 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:59.229 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:59.229 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:31:59.229 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655651+0000", 2024-09-14T06:31:59.229 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.0079999999999998e-06 2024-09-14T06:31:59.229 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:59.229 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:59.229 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:31:59.229 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.708429+0000", 2024-09-14T06:31:59.229 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052778459 2024-09-14T06:31:59.229 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:59.230 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:59.230 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:31:59.230 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.708473+0000", 2024-09-14T06:31:59.230 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.3782999999999997e-05 2024-09-14T06:31:59.230 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:59.230 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:59.230 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:31:59.230 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.708528+0000", 2024-09-14T06:31:59.230 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.5198999999999998e-05 2024-09-14T06:31:59.230 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:59.230 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:31:59.230 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:59.230 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:59.231 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:59.231 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:11464 3.1d 3:bcad7303:::rbd_data.112d92027590.0000000000000001:f [read 206848~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:31:59.231 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:25.655654+0000", 2024-09-14T06:31:59.231 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 153.52340093699999, 2024-09-14T06:31:59.231 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052981845999999999, 2024-09-14T06:31:59.231 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:31:59.231 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:31:59.231 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:31:59.231 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:31:59.231 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:31:59.231 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:59.231 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 11464 2024-09-14T06:31:59.231 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:59.231 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:31:59.231 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:59.232 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:31:59.232 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655654+0000", 2024-09-14T06:31:59.232 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:59.232 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:59.232 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:59.232 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:31:59.232 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655654+0000", 2024-09-14T06:31:59.232 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:59.232 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:59.232 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:59.232 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:31:59.232 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655946+0000", 2024-09-14T06:31:59.232 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00029148200000000003 2024-09-14T06:31:59.232 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:59.232 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:59.233 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:31:59.233 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655948+0000", 2024-09-14T06:31:59.233 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.973e-06 2024-09-14T06:31:59.233 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:59.233 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:59.233 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:31:59.233 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655953+0000", 2024-09-14T06:31:59.233 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.1240000000000004e-06 2024-09-14T06:31:59.233 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:59.233 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:59.233 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:31:59.233 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655956+0000", 2024-09-14T06:31:59.233 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.315e-06 2024-09-14T06:31:59.233 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:59.234 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:59.234 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:31:59.234 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.708543+0000", 2024-09-14T06:31:59.234 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052587416999999997 2024-09-14T06:31:59.234 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:59.234 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:59.234 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:31:59.234 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.708584+0000", 2024-09-14T06:31:59.234 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.1004000000000002e-05 2024-09-14T06:31:59.234 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:59.234 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:59.234 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:31:59.234 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.708636+0000", 2024-09-14T06:31:59.234 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.1530999999999998e-05 2024-09-14T06:31:59.235 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:59.235 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:31:59.235 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:59.235 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:59.235 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:59.235 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:11465 3.1d 3:bcad7303:::rbd_data.112d92027590.0000000000000001:f [read 217088~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:31:59.235 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:25.655961+0000", 2024-09-14T06:31:59.235 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 153.52309378499999, 2024-09-14T06:31:59.235 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052798682999999999, 2024-09-14T06:31:59.235 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:31:59.235 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:31:59.235 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:31:59.235 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:31:59.235 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:31:59.236 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:59.236 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 11465 2024-09-14T06:31:59.236 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:59.236 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:31:59.236 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:59.236 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:31:59.236 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655961+0000", 2024-09-14T06:31:59.236 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:59.236 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:59.236 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:59.236 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:31:59.236 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655961+0000", 2024-09-14T06:31:59.236 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:59.236 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:59.237 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:59.237 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:31:59.237 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.656114+0000", 2024-09-14T06:31:59.237 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.000152745 2024-09-14T06:31:59.237 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:59.237 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:59.237 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:31:59.237 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.656116+0000", 2024-09-14T06:31:59.237 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.804e-06 2024-09-14T06:31:59.237 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:59.237 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:59.237 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:31:59.238 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.656120+0000", 2024-09-14T06:31:59.238 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.4700000000000004e-06 2024-09-14T06:31:59.238 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:59.238 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:59.238 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:31:59.238 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.656123+0000", 2024-09-14T06:31:59.238 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.1269999999999999e-06 2024-09-14T06:31:59.238 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:59.238 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:59.238 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:31:59.238 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.708660+0000", 2024-09-14T06:31:59.238 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052536919000000001 2024-09-14T06:31:59.238 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:59.239 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:59.239 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:31:59.239 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.708703+0000", 2024-09-14T06:31:59.239 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.3077000000000001e-05 2024-09-14T06:31:59.239 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:59.239 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:59.239 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:31:59.239 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.708760+0000", 2024-09-14T06:31:59.239 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.6541e-05 2024-09-14T06:31:59.239 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:59.239 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:31:59.239 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:59.239 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:59.240 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:59.240 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:11466 3.1d 3:bcad7303:::rbd_data.112d92027590.0000000000000001:f [read 227328~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:31:59.240 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:25.656129+0000", 2024-09-14T06:31:59.240 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 153.52292639699999, 2024-09-14T06:31:59.240 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052750987999999999, 2024-09-14T06:31:59.240 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:31:59.240 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:31:59.240 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:31:59.240 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:31:59.240 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:31:59.240 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:59.240 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 11466 2024-09-14T06:31:59.240 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:59.240 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:31:59.241 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:59.241 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:31:59.241 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.656129+0000", 2024-09-14T06:31:59.241 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:59.241 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:59.241 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:59.241 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:31:59.241 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.656129+0000", 2024-09-14T06:31:59.241 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:59.241 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:59.241 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:59.241 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:31:59.241 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.656347+0000", 2024-09-14T06:31:59.241 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.000218054 2024-09-14T06:31:59.242 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:59.242 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:59.242 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:31:59.242 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.656349+0000", 2024-09-14T06:31:59.242 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.9199999999999998e-06 2024-09-14T06:31:59.242 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:59.242 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:59.242 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:31:59.242 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.656353+0000", 2024-09-14T06:31:59.242 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.9210000000000002e-06 2024-09-14T06:31:59.242 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:59.242 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:59.242 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:31:59.242 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.656356+0000", 2024-09-14T06:31:59.243 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.7670000000000001e-06 2024-09-14T06:31:59.243 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:59.243 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:59.243 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:31:59.243 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.708787+0000", 2024-09-14T06:31:59.243 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052430840999999999 2024-09-14T06:31:59.243 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:59.243 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:59.243 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:31:59.243 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.708827+0000", 2024-09-14T06:31:59.243 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.9538e-05 2024-09-14T06:31:59.243 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:59.243 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:59.243 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:31:59.243 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.708880+0000", 2024-09-14T06:31:59.244 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.2947e-05 2024-09-14T06:31:59.244 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:59.244 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:31:59.244 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:59.244 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:59.244 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:59.244 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:11467 3.1d 3:bcad7303:::rbd_data.112d92027590.0000000000000001:f [read 237568~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:31:59.244 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:25.656361+0000", 2024-09-14T06:31:59.244 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 153.52269374900001, 2024-09-14T06:31:59.244 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052643898000000001, 2024-09-14T06:31:59.244 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:31:59.244 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:31:59.244 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:31:59.245 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:31:59.245 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:31:59.245 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:59.245 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 11467 2024-09-14T06:31:59.245 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:59.245 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:31:59.245 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:59.245 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:31:59.245 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.656361+0000", 2024-09-14T06:31:59.245 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:59.245 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:59.245 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:59.245 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:31:59.245 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.656361+0000", 2024-09-14T06:31:59.246 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:59.246 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:59.246 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:59.246 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:31:59.246 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.656855+0000", 2024-09-14T06:31:59.246 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00049372000000000001 2024-09-14T06:31:59.246 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:59.246 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:59.246 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:31:59.246 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.656859+0000", 2024-09-14T06:31:59.246 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.1300000000000003e-06 2024-09-14T06:31:59.246 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:59.246 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:59.246 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:31:59.247 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.656863+0000", 2024-09-14T06:31:59.247 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.2629999999999997e-06 2024-09-14T06:31:59.247 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:59.247 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:59.247 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:31:59.247 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.656867+0000", 2024-09-14T06:31:59.247 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.3220000000000001e-06 2024-09-14T06:31:59.247 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:59.247 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:59.247 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:31:59.247 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.708903+0000", 2024-09-14T06:31:59.247 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052035855999999998 2024-09-14T06:31:59.248 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:59.248 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:59.248 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:31:59.248 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.708949+0000", 2024-09-14T06:31:59.248 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.6276999999999998e-05 2024-09-14T06:31:59.248 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:59.248 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:59.248 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:31:59.248 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.709005+0000", 2024-09-14T06:31:59.248 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.6329999999999998e-05 2024-09-14T06:31:59.248 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:59.248 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:31:59.248 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:59.249 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:59.249 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:59.249 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:11480 3.1d 3:bcad7303:::rbd_data.112d92027590.0000000000000001:f [read 370688~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:31:59.249 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:25.658563+0000", 2024-09-14T06:31:59.249 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 153.52049198500001, 2024-09-14T06:31:59.249 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052429164, 2024-09-14T06:31:59.249 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:31:59.249 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:31:59.249 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:31:59.249 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:31:59.249 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:31:59.249 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:59.249 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 11480 2024-09-14T06:31:59.249 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:59.250 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:31:59.250 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:59.250 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:31:59.250 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.658563+0000", 2024-09-14T06:31:59.250 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:59.250 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:59.250 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:59.250 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:31:59.250 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.658563+0000", 2024-09-14T06:31:59.250 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:59.250 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:59.250 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:59.250 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:31:59.250 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.658658+0000", 2024-09-14T06:31:59.251 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.5346999999999994e-05 2024-09-14T06:31:59.251 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:59.251 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:59.251 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:31:59.251 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.658662+0000", 2024-09-14T06:31:59.251 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.14e-06 2024-09-14T06:31:59.251 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:59.251 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:59.251 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:31:59.251 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.658669+0000", 2024-09-14T06:31:59.251 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.7379999999999994e-06 2024-09-14T06:31:59.251 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:59.251 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:59.251 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:31:59.252 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.658675+0000", 2024-09-14T06:31:59.252 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.5430000000000001e-06 2024-09-14T06:31:59.252 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:59.252 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:59.252 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:31:59.252 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.710824+0000", 2024-09-14T06:31:59.252 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052148847999999998 2024-09-14T06:31:59.252 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:59.252 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:59.252 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:31:59.252 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.710864+0000", 2024-09-14T06:31:59.252 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.9925999999999997e-05 2024-09-14T06:31:59.252 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:59.253 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:59.253 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:31:59.253 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.710992+0000", 2024-09-14T06:31:59.253 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.000128622 2024-09-14T06:31:59.253 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:59.253 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:31:59.253 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:59.253 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:59.253 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:59.253 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:11481 3.1d 3:bcad7303:::rbd_data.112d92027590.0000000000000001:f [read 380928~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:31:59.253 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:25.658684+0000", 2024-09-14T06:31:59.253 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 153.520370915, 2024-09-14T06:31:59.253 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052508565, 2024-09-14T06:31:59.253 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:31:59.254 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:31:59.254 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:31:59.254 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:31:59.254 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:31:59.254 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:59.254 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 11481 2024-09-14T06:31:59.254 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:59.254 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:31:59.254 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:59.254 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:31:59.254 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.658684+0000", 2024-09-14T06:31:59.254 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:59.254 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:59.254 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:59.254 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:31:59.254 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.658684+0000", 2024-09-14T06:31:59.255 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:59.255 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:59.255 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:59.255 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:31:59.255 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.658715+0000", 2024-09-14T06:31:59.255 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.0703000000000001e-05 2024-09-14T06:31:59.255 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:59.255 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:59.255 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:31:59.255 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.658718+0000", 2024-09-14T06:31:59.255 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.0709999999999999e-06 2024-09-14T06:31:59.255 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:59.255 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:59.255 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:31:59.256 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.658725+0000", 2024-09-14T06:31:59.256 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.2339999999999997e-06 2024-09-14T06:31:59.256 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:59.256 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:59.256 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:31:59.256 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.658731+0000", 2024-09-14T06:31:59.256 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.6450000000000002e-06 2024-09-14T06:31:59.256 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:59.256 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:59.256 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:31:59.256 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.711020+0000", 2024-09-14T06:31:59.256 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052288886 2024-09-14T06:31:59.256 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:59.257 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:59.257 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:31:59.257 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.711061+0000", 2024-09-14T06:31:59.257 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.1084000000000002e-05 2024-09-14T06:31:59.257 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:59.257 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:59.257 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:31:59.257 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.711193+0000", 2024-09-14T06:31:59.257 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00013194199999999999 2024-09-14T06:31:59.257 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:59.257 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:31:59.257 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:59.257 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:59.258 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:59.258 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:11482 3.1d 3:bcad7303:::rbd_data.112d92027590.0000000000000001:f [read 391168~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:31:59.258 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:25.658740+0000", 2024-09-14T06:31:59.258 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 153.52031545, 2024-09-14T06:31:59.258 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052637015000000002, 2024-09-14T06:31:59.258 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:31:59.258 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:31:59.258 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:31:59.258 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:31:59.258 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:31:59.258 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:59.258 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 11482 2024-09-14T06:31:59.258 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:59.258 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:31:59.259 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:59.259 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:31:59.259 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.658740+0000", 2024-09-14T06:31:59.259 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:59.259 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:59.259 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:59.259 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:31:59.259 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.658740+0000", 2024-09-14T06:31:59.259 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:59.259 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:59.259 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:59.259 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:31:59.259 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.658880+0000", 2024-09-14T06:31:59.259 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00014036100000000001 2024-09-14T06:31:59.259 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:59.259 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:59.260 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:31:59.260 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.658883+0000", 2024-09-14T06:31:59.260 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.3969999999999998e-06 2024-09-14T06:31:59.260 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:59.260 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:59.260 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:31:59.260 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.658890+0000", 2024-09-14T06:31:59.260 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.1500000000000002e-06 2024-09-14T06:31:59.260 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:59.260 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:59.260 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:31:59.260 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.658897+0000", 2024-09-14T06:31:59.260 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.1260000000000003e-06 2024-09-14T06:31:59.261 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:59.261 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:59.261 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:31:59.261 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.711207+0000", 2024-09-14T06:31:59.261 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052310777000000003 2024-09-14T06:31:59.261 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:59.261 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:59.261 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:31:59.261 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.711249+0000", 2024-09-14T06:31:59.261 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.1786999999999998e-05 2024-09-14T06:31:59.261 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:59.261 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:59.262 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:31:59.262 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.711377+0000", 2024-09-14T06:31:59.262 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.000127417 2024-09-14T06:31:59.262 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:59.262 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:31:59.262 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:59.262 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:59.262 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:59.262 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:11483 3.1d 3:bcad7303:::rbd_data.112d92027590.0000000000000001:f [read 401408~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:31:59.262 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:25.658906+0000", 2024-09-14T06:31:59.262 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 153.52014866600001, 2024-09-14T06:31:59.262 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052595109000000001, 2024-09-14T06:31:59.262 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:31:59.262 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:31:59.263 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:31:59.263 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:31:59.263 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:31:59.263 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:59.263 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 11483 2024-09-14T06:31:59.263 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:59.263 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:31:59.263 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:59.263 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:31:59.263 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.658906+0000", 2024-09-14T06:31:59.263 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:59.263 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:59.263 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:59.263 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:31:59.264 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.658906+0000", 2024-09-14T06:31:59.264 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:59.264 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:59.264 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:59.264 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:31:59.264 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.664985+0000", 2024-09-14T06:31:59.264 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.0060787250000000001 2024-09-14T06:31:59.264 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:59.264 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:59.264 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:31:59.264 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.664991+0000", 2024-09-14T06:31:59.264 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.7030000000000003e-06 2024-09-14T06:31:59.264 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:59.264 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:59.265 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:31:59.265 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.664999+0000", 2024-09-14T06:31:59.265 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.4430000000000003e-06 2024-09-14T06:31:59.265 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:59.265 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:59.265 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:31:59.265 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.665006+0000", 2024-09-14T06:31:59.265 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.0840000000000003e-06 2024-09-14T06:31:59.265 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:59.265 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:59.265 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:31:59.265 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.711402+0000", 2024-09-14T06:31:59.265 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046395714999999997 2024-09-14T06:31:59.265 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:59.266 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:59.266 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:31:59.266 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.711443+0000", 2024-09-14T06:31:59.266 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.0911999999999997e-05 2024-09-14T06:31:59.266 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:59.266 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:59.266 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:31:59.266 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.711501+0000", 2024-09-14T06:31:59.266 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.8526999999999997e-05 2024-09-14T06:31:59.266 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:59.266 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:31:59.266 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:59.266 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:59.266 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:59.267 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:214569 3.2 3:479633bd:::rbd_object_map.112d3bd84cb0:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e170)", 2024-09-14T06:31:59.267 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:31:34.933901+0000", 2024-09-14T06:31:59.267 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 24.245154282000001, 2024-09-14T06:31:59.267 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.68157511400000004, 2024-09-14T06:31:59.267 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:31:59.267 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:31:59.267 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:31:59.267 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:31:59.267 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:31:59.267 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:31:59.267 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 214569 2024-09-14T06:31:59.267 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:59.267 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:31:59.267 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:59.268 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:31:59.268 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:31:34.933901+0000", 2024-09-14T06:31:59.268 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:59.268 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:59.268 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:59.268 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:31:59.268 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:31:34.933901+0000", 2024-09-14T06:31:59.268 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:31:59.268 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:59.268 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:59.268 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:31:59.268 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:31:34.933902+0000", 2024-09-14T06:31:59.268 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.7439999999999999e-06 2024-09-14T06:31:59.268 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:59.269 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:59.269 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:31:59.269 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:31:34.933910+0000", 2024-09-14T06:31:59.269 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.8569999999999995e-06 2024-09-14T06:31:59.269 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:59.269 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:59.269 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:31:59.269 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:31:34.933932+0000", 2024-09-14T06:31:59.269 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.1997999999999998e-05 2024-09-14T06:31:59.269 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:59.269 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:59.269 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:31:59.269 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:31:34.933941+0000", 2024-09-14T06:31:59.269 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.7180000000000002e-06 2024-09-14T06:31:59.269 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:59.270 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:59.270 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:31:59.270 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:31:34.934013+0000", 2024-09-14T06:31:59.270 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.1761000000000006e-05 2024-09-14T06:31:59.270 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:59.270 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:59.270 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "waiting for peered", 2024-09-14T06:31:59.270 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:31:34.934022+0000", 2024-09-14T06:31:59.270 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.8740000000000001e-06 2024-09-14T06:31:59.270 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:59.270 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:59.270 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:31:59.270 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:31:35.608803+0000", 2024-09-14T06:31:59.271 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.67478164299999999 2024-09-14T06:31:59.271 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:59.271 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:59.271 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:31:59.271 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:31:35.608964+0000", 2024-09-14T06:31:59.271 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00016112900000000001 2024-09-14T06:31:59.271 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:59.271 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:59.271 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "waiting for subops from 6", 2024-09-14T06:31:59.271 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:31:35.609399+0000", 2024-09-14T06:31:59.271 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00043454000000000002 2024-09-14T06:31:59.271 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:59.271 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:59.271 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "op_commit", 2024-09-14T06:31:59.271 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:31:35.610495+0000", 2024-09-14T06:31:59.272 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.001096404 2024-09-14T06:31:59.272 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:59.272 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:59.272 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:31:59.272 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:31:35.615409+0000", 2024-09-14T06:31:59.272 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.0049133199999999997 2024-09-14T06:31:59.272 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:59.272 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:59.272 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "commit_sent", 2024-09-14T06:31:59.272 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:31:35.615444+0000", 2024-09-14T06:31:59.272 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.5457000000000003e-05 2024-09-14T06:31:59.272 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:31:59.272 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:31:59.272 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:31:59.272 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:31:35.615476+0000", 2024-09-14T06:31:59.273 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.1668999999999998e-05 2024-09-14T06:31:59.273 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:59.273 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:31:59.273 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:59.273 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:31:59.273 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:31:59.273 INFO:teuthology.orchestra.run.smithi138.stdout:} 2024-09-14T06:31:59.273 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-14T06:31:59.275 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-14T06:31:59.276 INFO:tasks.ceph.osd.4.smithi138.stderr:2024-09-14T06:31:59.274+0000 7f6c653c8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33615) UID: 0 2024-09-14T06:31:59.376 INFO:tasks.ceph.osd.2.smithi094.stderr:2024-09-14T06:31:59.375+0000 7faf3269b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33649) UID: 0 2024-09-14T06:31:59.394 INFO:teuthology.orchestra.run.smithi094.stdout:osd.0: {} 2024-09-14T06:31:59.394 INFO:teuthology.orchestra.run.smithi094.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-14T06:31:59.410 INFO:teuthology.orchestra.run.smithi094.stdout:osd.1: {} 2024-09-14T06:31:59.410 INFO:teuthology.orchestra.run.smithi094.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-14T06:31:59.418 INFO:teuthology.orchestra.run.smithi138.stdout:{ 2024-09-14T06:31:59.419 INFO:teuthology.orchestra.run.smithi138.stdout: "ops": [], 2024-09-14T06:31:59.419 INFO:teuthology.orchestra.run.smithi138.stdout: "num_ops": 0 2024-09-14T06:31:59.419 INFO:teuthology.orchestra.run.smithi138.stdout:} 2024-09-14T06:31:59.426 INFO:teuthology.orchestra.run.smithi094.stdout:osd.2: {} 2024-09-14T06:31:59.426 INFO:teuthology.orchestra.run.smithi094.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-14T06:31:59.427 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-14T06:31:59.442 INFO:teuthology.orchestra.run.smithi094.stdout:osd.3: {} 2024-09-14T06:31:59.442 INFO:teuthology.orchestra.run.smithi094.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-14T06:31:59.458 INFO:teuthology.orchestra.run.smithi094.stdout:osd.4: {} 2024-09-14T06:31:59.459 INFO:teuthology.orchestra.run.smithi094.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-14T06:31:59.476 INFO:teuthology.orchestra.run.smithi094.stdout:osd.5: {} 2024-09-14T06:31:59.477 INFO:teuthology.orchestra.run.smithi094.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-14T06:31:59.477 INFO:tasks.ceph.osd.4.smithi138.stderr:2024-09-14T06:31:59.475+0000 7f6c653c8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33615) UID: 0 2024-09-14T06:31:59.493 INFO:teuthology.orchestra.run.smithi094.stdout:osd.6: {} 2024-09-14T06:31:59.493 INFO:teuthology.orchestra.run.smithi094.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-14T06:31:59.509 INFO:teuthology.orchestra.run.smithi094.stdout:osd.7: {} 2024-09-14T06:31:59.509 INFO:teuthology.orchestra.run.smithi094.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-14T06:31:59.533 INFO:teuthology.orchestra.run.smithi138.stdout:ERROR: (22) Invalid argument 2024-09-14T06:31:59.533 INFO:teuthology.orchestra.run.smithi138.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:31:59.542 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-14T06:31:59.583 INFO:tasks.ceph.osd.6.smithi138.stderr:2024-09-14T06:31:59.582+0000 7f8cd8849640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33620) UID: 0 2024-09-14T06:31:59.678 INFO:tasks.ceph.osd.4.smithi138.stderr:2024-09-14T06:31:59.677+0000 7f6c653c8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33615) UID: 0 2024-09-14T06:31:59.684 INFO:teuthology.orchestra.run.smithi138.stdout:ERROR: (22) Invalid argument 2024-09-14T06:31:59.684 INFO:teuthology.orchestra.run.smithi138.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:31:59.693 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-14T06:31:59.779 INFO:tasks.ceph.osd.2.smithi094.stderr:2024-09-14T06:31:59.778+0000 7faf3269b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33649) UID: 0 2024-09-14T06:31:59.813 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-14T06:31:59.845 INFO:teuthology.orchestra.run.smithi138.stdout:ERROR: (22) Invalid argument 2024-09-14T06:31:59.846 INFO:teuthology.orchestra.run.smithi138.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:31:59.854 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-14T06:31:59.880 INFO:tasks.ceph.osd.0.smithi094.stderr:2024-09-14T06:31:59.879+0000 7f07c9333640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33636) UID: 0 2024-09-14T06:31:59.898 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 4, 5, 6, 7, 3] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-14T06:31:59.898 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-14T06:31:59.898 INFO:tasks.thrashosds.thrasher:Removing osd 6, in_osds are: [0, 1, 2, 4, 5, 6, 7, 3] 2024-09-14T06:31:59.898 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd out 6 2024-09-14T06:31:59.958 INFO:teuthology.orchestra.run.smithi138.stdout:ERROR: (22) Invalid argument 2024-09-14T06:31:59.958 INFO:teuthology.orchestra.run.smithi138.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:31:59.967 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-14T06:31:59.981 INFO:tasks.ceph.osd.0.smithi094.stderr:2024-09-14T06:31:59.980+0000 7f07c9333640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33636) UID: 0 2024-09-14T06:32:00.070 INFO:teuthology.orchestra.run.smithi138.stdout:ERROR: (22) Invalid argument 2024-09-14T06:32:00.070 INFO:teuthology.orchestra.run.smithi138.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:32:00.078 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-14T06:32:00.081 INFO:tasks.ceph.osd.5.smithi138.stderr:2024-09-14T06:32:00.080+0000 7fe13746f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33618) UID: 0 2024-09-14T06:32:00.183 INFO:tasks.ceph.osd.5.smithi138.stderr:2024-09-14T06:32:00.182+0000 7fe13746f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33618) UID: 0 2024-09-14T06:32:00.184 INFO:teuthology.orchestra.run.smithi094.stdout:ERROR: (22) Invalid argument 2024-09-14T06:32:00.184 INFO:teuthology.orchestra.run.smithi094.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:32:00.192 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-14T06:32:00.203 INFO:tasks.ceph.mon.a.smithi094.stderr:2024-09-14T06:32:00.202+0000 7f3f97784640 -1 mon.a@0(leader).osd e195 definitely_dead 0 2024-09-14T06:32:00.284 INFO:tasks.ceph.osd.2.smithi094.stderr:2024-09-14T06:32:00.283+0000 7faf3269b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33649) UID: 0 2024-09-14T06:32:00.294 INFO:teuthology.orchestra.run.smithi094.stdout:ERROR: (22) Invalid argument 2024-09-14T06:32:00.294 INFO:teuthology.orchestra.run.smithi094.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:32:00.302 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-14T06:32:00.385 INFO:tasks.ceph.osd.5.smithi138.stderr:2024-09-14T06:32:00.384+0000 7fe13746f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33618) UID: 0 2024-09-14T06:32:00.445 INFO:teuthology.orchestra.run.smithi094.stdout:ERROR: (22) Invalid argument 2024-09-14T06:32:00.445 INFO:teuthology.orchestra.run.smithi094.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:32:00.454 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-14T06:32:00.497 INFO:tasks.ceph.osd.1.smithi094.stderr:2024-09-14T06:32:00.496+0000 7f328e47a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33641) UID: 0 2024-09-14T06:32:00.556 INFO:teuthology.orchestra.run.smithi094.stdout:ERROR: (22) Invalid argument 2024-09-14T06:32:00.556 INFO:teuthology.orchestra.run.smithi094.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:32:00.565 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-14T06:32:00.587 INFO:tasks.ceph.osd.5.smithi138.stderr:2024-09-14T06:32:00.586+0000 7fe13746f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33618) UID: 0 2024-09-14T06:32:00.602 INFO:teuthology.orchestra.run.smithi094.stderr:noscrub is set 2024-09-14T06:32:00.603 INFO:teuthology.orchestra.run.smithi094.stderr:marked out osd.6. 2024-09-14T06:32:00.688 INFO:tasks.ceph.osd.4.smithi138.stderr:2024-09-14T06:32:00.688+0000 7f6c653c8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33615) UID: 0 2024-09-14T06:32:00.710 INFO:teuthology.orchestra.run.smithi094.stdout:ERROR: (22) Invalid argument 2024-09-14T06:32:00.710 INFO:teuthology.orchestra.run.smithi094.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:32:00.718 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-14T06:32:00.790 INFO:tasks.ceph.osd.4.smithi138.stderr:2024-09-14T06:32:00.788+0000 7f6c653c8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33615) UID: 0 2024-09-14T06:32:00.822 INFO:teuthology.orchestra.run.smithi094.stdout:ERROR: (22) Invalid argument 2024-09-14T06:32:00.822 INFO:teuthology.orchestra.run.smithi094.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:32:00.831 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-14T06:32:00.891 INFO:tasks.ceph.osd.1.smithi094.stderr:2024-09-14T06:32:00.889+0000 7f328e47a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33641) UID: 0 2024-09-14T06:32:00.933 INFO:teuthology.orchestra.run.smithi094.stdout:ERROR: (22) Invalid argument 2024-09-14T06:32:00.934 INFO:teuthology.orchestra.run.smithi094.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:32:00.942 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-14T06:32:01.026 INFO:tasks.ceph.osd.0.smithi094.stderr:2024-09-14T06:32:01.025+0000 7f07c9333640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33636) UID: 0 2024-09-14T06:32:01.084 INFO:teuthology.orchestra.run.smithi094.stdout:ERROR: (22) Invalid argument 2024-09-14T06:32:01.084 INFO:teuthology.orchestra.run.smithi094.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:32:01.093 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-14T06:32:01.093 INFO:tasks.ceph.osd.1.smithi094.stderr:2024-09-14T06:32:01.091+0000 7f328e47a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33641) UID: 0 2024-09-14T06:32:01.193 INFO:tasks.ceph.osd.4.smithi138.stderr:2024-09-14T06:32:01.192+0000 7f6c653c8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33615) UID: 0 2024-09-14T06:32:01.236 INFO:teuthology.orchestra.run.smithi094.stdout:ERROR: (22) Invalid argument 2024-09-14T06:32:01.236 INFO:teuthology.orchestra.run.smithi094.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:32:01.244 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-14T06:32:01.294 INFO:tasks.ceph.osd.0.smithi094.stderr:2024-09-14T06:32:01.293+0000 7f07c9333640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33636) UID: 0 2024-09-14T06:32:01.347 INFO:teuthology.orchestra.run.smithi094.stdout:ERROR: (22) Invalid argument 2024-09-14T06:32:01.348 INFO:teuthology.orchestra.run.smithi094.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:32:01.356 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-14T06:32:01.395 INFO:tasks.ceph.osd.7.smithi138.stderr:2024-09-14T06:32:01.394+0000 7f3c6a21d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33623) UID: 0 2024-09-14T06:32:01.496 INFO:tasks.ceph.osd.0.smithi094.stderr:2024-09-14T06:32:01.494+0000 7f07c9333640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33636) UID: 0 2024-09-14T06:32:01.499 INFO:teuthology.orchestra.run.smithi094.stdout:ERROR: (22) Invalid argument 2024-09-14T06:32:01.499 INFO:teuthology.orchestra.run.smithi094.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:32:01.508 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-14T06:32:01.521 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-14T06:32:01.596 INFO:tasks.ceph.osd.5.smithi138.stderr:2024-09-14T06:32:01.596+0000 7fe13746f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33618) UID: 0 2024-09-14T06:32:01.659 INFO:teuthology.orchestra.run.smithi094.stdout:ERROR: (22) Invalid argument 2024-09-14T06:32:01.659 INFO:teuthology.orchestra.run.smithi094.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:32:01.668 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-14T06:32:01.674 INFO:teuthology.orchestra.run.smithi094.stdout:osd.0: {} 2024-09-14T06:32:01.674 INFO:teuthology.orchestra.run.smithi094.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-14T06:32:01.691 INFO:teuthology.orchestra.run.smithi094.stdout:osd.1: {} 2024-09-14T06:32:01.691 INFO:teuthology.orchestra.run.smithi094.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-14T06:32:01.697 INFO:tasks.ceph.osd.0.smithi094.stderr:2024-09-14T06:32:01.696+0000 7f07c9333640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33636) UID: 0 2024-09-14T06:32:01.706 INFO:teuthology.orchestra.run.smithi094.stdout:osd.2: {} 2024-09-14T06:32:01.707 INFO:teuthology.orchestra.run.smithi094.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-14T06:32:01.722 INFO:teuthology.orchestra.run.smithi094.stdout:osd.3: {} 2024-09-14T06:32:01.723 INFO:teuthology.orchestra.run.smithi094.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-14T06:32:01.739 INFO:teuthology.orchestra.run.smithi094.stdout:osd.4: {} 2024-09-14T06:32:01.739 INFO:teuthology.orchestra.run.smithi094.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-14T06:32:01.760 INFO:teuthology.orchestra.run.smithi094.stdout:osd.5: {} 2024-09-14T06:32:01.761 INFO:teuthology.orchestra.run.smithi094.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-14T06:32:01.775 INFO:teuthology.orchestra.run.smithi094.stdout:osd.6: {} 2024-09-14T06:32:01.775 INFO:teuthology.orchestra.run.smithi094.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-14T06:32:01.785 INFO:teuthology.orchestra.run.smithi138.stdout:{ 2024-09-14T06:32:01.785 INFO:teuthology.orchestra.run.smithi138.stdout: "ops": [], 2024-09-14T06:32:01.785 INFO:teuthology.orchestra.run.smithi138.stdout: "num_ops": 0 2024-09-14T06:32:01.785 INFO:teuthology.orchestra.run.smithi138.stdout:} 2024-09-14T06:32:01.791 INFO:teuthology.orchestra.run.smithi094.stdout:osd.7: {} 2024-09-14T06:32:01.791 INFO:teuthology.orchestra.run.smithi094.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-14T06:32:01.795 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-14T06:32:01.798 INFO:tasks.ceph.osd.0.smithi094.stderr:2024-09-14T06:32:01.797+0000 7f07c9333640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33636) UID: 0 2024-09-14T06:32:01.899 INFO:teuthology.orchestra.run.smithi138.stdout:{ 2024-09-14T06:32:01.899 INFO:teuthology.orchestra.run.smithi138.stdout: "ops": [], 2024-09-14T06:32:01.899 INFO:teuthology.orchestra.run.smithi138.stdout: "complaint_time": 30, 2024-09-14T06:32:01.899 INFO:teuthology.orchestra.run.smithi138.stdout: "num_blocked_ops": 0 2024-09-14T06:32:01.899 INFO:teuthology.orchestra.run.smithi138.stdout:} 2024-09-14T06:32:01.900 INFO:tasks.ceph.osd.3.smithi094.stderr:2024-09-14T06:32:01.898+0000 7feeeafe8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33655) UID: 0 2024-09-14T06:32:01.907 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-14T06:32:02.000 INFO:tasks.ceph.osd.4.smithi138.stderr:2024-09-14T06:32:01.999+0000 7f6c653c8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33615) UID: 0 2024-09-14T06:32:02.011 INFO:teuthology.orchestra.run.smithi138.stdout:{ 2024-09-14T06:32:02.011 INFO:teuthology.orchestra.run.smithi138.stdout: "size": 20, 2024-09-14T06:32:02.011 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 600, 2024-09-14T06:32:02.011 INFO:teuthology.orchestra.run.smithi138.stdout: "ops": [ 2024-09-14T06:32:02.012 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:02.012 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:19328 3.3 3:c34e5e51:::rbd_data.112d92027590.0000000000000024:f [read 1926144~1024 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:32:02.012 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:27.059768+0000", 2024-09-14T06:32:02.012 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 154.95007356599999, 2024-09-14T06:32:02.012 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046369147999999999, 2024-09-14T06:32:02.012 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:32:02.012 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:32:02.012 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:32:02.012 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:32:02.012 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:32:02.012 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:02.012 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 19328 2024-09-14T06:32:02.012 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:02.013 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:32:02.013 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:02.013 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:32:02.013 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.059768+0000", 2024-09-14T06:32:02.013 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:02.013 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:02.013 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:02.013 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:32:02.013 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.059768+0000", 2024-09-14T06:32:02.013 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:02.013 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:02.013 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:02.013 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:32:02.013 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.059874+0000", 2024-09-14T06:32:02.014 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00010551400000000001 2024-09-14T06:32:02.014 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:02.014 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:02.014 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:32:02.014 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.059877+0000", 2024-09-14T06:32:02.014 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.8279999999999999e-06 2024-09-14T06:32:02.014 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:02.014 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:02.014 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:32:02.014 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.059884+0000", 2024-09-14T06:32:02.014 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.6389999999999992e-06 2024-09-14T06:32:02.014 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:02.014 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:02.014 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:32:02.014 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.059890+0000", 2024-09-14T06:32:02.015 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.49e-06 2024-09-14T06:32:02.015 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:02.015 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:02.015 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:32:02.015 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.105804+0000", 2024-09-14T06:32:02.015 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.045914242000000001 2024-09-14T06:32:02.015 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:02.015 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:02.015 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:32:02.015 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.105886+0000", 2024-09-14T06:32:02.015 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.2305999999999994e-05 2024-09-14T06:32:02.015 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:02.015 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:02.015 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:32:02.015 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.106138+0000", 2024-09-14T06:32:02.016 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.000251129 2024-09-14T06:32:02.016 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:02.016 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:32:02.016 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:02.016 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:02.016 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:02.016 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:19329 3.3 3:c34e5e51:::rbd_data.112d92027590.0000000000000024:f [read 1936384~1024 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:32:02.016 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:27.059899+0000", 2024-09-14T06:32:02.016 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 154.94994304299999, 2024-09-14T06:32:02.016 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046572122000000001, 2024-09-14T06:32:02.016 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:32:02.016 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:32:02.016 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:32:02.016 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:32:02.017 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:32:02.017 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:02.017 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 19329 2024-09-14T06:32:02.017 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:02.017 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:32:02.017 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:02.017 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:32:02.017 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.059899+0000", 2024-09-14T06:32:02.017 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:02.017 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:02.017 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:02.017 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:32:02.017 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.059899+0000", 2024-09-14T06:32:02.017 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:02.018 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:02.018 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:02.018 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:32:02.018 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062501+0000", 2024-09-14T06:32:02.018 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.0026016419999999999 2024-09-14T06:32:02.018 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:02.018 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:02.018 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:32:02.018 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062514+0000", 2024-09-14T06:32:02.018 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.2947e-05 2024-09-14T06:32:02.018 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:02.018 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:02.018 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:32:02.018 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062523+0000", 2024-09-14T06:32:02.018 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.5519999999999993e-06 2024-09-14T06:32:02.019 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:02.019 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:02.019 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:32:02.019 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062534+0000", 2024-09-14T06:32:02.019 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.1416e-05 2024-09-14T06:32:02.019 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:02.019 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:02.019 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:32:02.019 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.106179+0000", 2024-09-14T06:32:02.019 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.043644049999999997 2024-09-14T06:32:02.019 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:02.019 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:02.019 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:32:02.019 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.106262+0000", 2024-09-14T06:32:02.020 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.3178999999999998e-05 2024-09-14T06:32:02.020 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:02.020 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:02.020 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:32:02.020 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.106471+0000", 2024-09-14T06:32:02.020 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00020933599999999999 2024-09-14T06:32:02.020 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:02.020 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:32:02.020 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:02.020 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:02.020 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:02.020 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:19351 3.3 3:c34e5e51:::rbd_data.112d92027590.0000000000000024:f [read 1159168~1024 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:32:02.020 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:27.062766+0000", 2024-09-14T06:32:02.020 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 154.947075818, 2024-09-14T06:32:02.021 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046099454999999998, 2024-09-14T06:32:02.021 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:32:02.021 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:32:02.021 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:32:02.021 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:32:02.021 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:32:02.021 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:02.021 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 19351 2024-09-14T06:32:02.021 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:02.021 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:32:02.021 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:02.021 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:32:02.021 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062766+0000", 2024-09-14T06:32:02.021 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:02.022 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:02.022 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:02.022 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:32:02.022 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062766+0000", 2024-09-14T06:32:02.022 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:02.022 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:02.022 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:02.022 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:32:02.022 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062766+0000", 2024-09-14T06:32:02.022 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.6e-07 2024-09-14T06:32:02.022 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:02.022 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:02.022 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:32:02.022 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062767+0000", 2024-09-14T06:32:02.023 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.0070000000000001e-06 2024-09-14T06:32:02.023 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:02.023 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:02.023 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:32:02.023 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062785+0000", 2024-09-14T06:32:02.023 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.7884999999999999e-05 2024-09-14T06:32:02.023 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:02.023 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:02.023 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:32:02.023 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062790+0000", 2024-09-14T06:32:02.023 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.0900000000000004e-06 2024-09-14T06:32:02.023 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:02.023 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:02.024 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:32:02.024 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.108735+0000", 2024-09-14T06:32:02.024 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.045944411999999997 2024-09-14T06:32:02.024 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:02.024 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:02.024 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:32:02.024 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.108809+0000", 2024-09-14T06:32:02.024 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.4157000000000002e-05 2024-09-14T06:32:02.024 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:02.024 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:02.024 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:32:02.024 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.108866+0000", 2024-09-14T06:32:02.024 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.6644000000000002e-05 2024-09-14T06:32:02.024 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:02.025 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:32:02.025 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:02.025 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:02.025 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:02.025 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:19352 3.3 3:c34e5e51:::rbd_data.112d92027590.0000000000000024:f [read 1169408~1024 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:32:02.025 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:27.062798+0000", 2024-09-14T06:32:02.025 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 154.94704440699999, 2024-09-14T06:32:02.025 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046255900000000003, 2024-09-14T06:32:02.025 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:32:02.025 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:32:02.025 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:32:02.025 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:32:02.025 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:32:02.026 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:02.026 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 19352 2024-09-14T06:32:02.026 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:02.026 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:32:02.026 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:02.026 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:32:02.026 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062798+0000", 2024-09-14T06:32:02.026 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:02.026 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:02.026 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:02.026 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:32:02.026 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062798+0000", 2024-09-14T06:32:02.026 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:02.026 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:02.026 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:02.027 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:32:02.027 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062798+0000", 2024-09-14T06:32:02.027 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.37e-07 2024-09-14T06:32:02.027 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:02.027 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:02.027 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:32:02.027 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062800+0000", 2024-09-14T06:32:02.027 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.0269999999999998e-06 2024-09-14T06:32:02.027 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:02.027 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:02.027 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:32:02.027 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062806+0000", 2024-09-14T06:32:02.027 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.6690000000000003e-06 2024-09-14T06:32:02.027 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:02.028 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:02.028 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:32:02.028 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062810+0000", 2024-09-14T06:32:02.028 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.3449999999999999e-06 2024-09-14T06:32:02.028 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:02.028 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:02.028 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:32:02.028 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.108892+0000", 2024-09-14T06:32:02.028 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046081715000000002 2024-09-14T06:32:02.028 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:02.028 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:02.028 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:32:02.028 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.108940+0000", 2024-09-14T06:32:02.029 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.7815999999999998e-05 2024-09-14T06:32:02.029 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:02.029 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:02.029 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:32:02.029 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.109053+0000", 2024-09-14T06:32:02.029 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.000113791 2024-09-14T06:32:02.029 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:02.029 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:32:02.030 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:02.030 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:02.030 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:02.030 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:19353 3.3 3:c34e5e51:::rbd_data.112d92027590.0000000000000024:f [read 1179648~1024 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:32:02.030 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:27.062816+0000", 2024-09-14T06:32:02.030 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 154.94702588800001, 2024-09-14T06:32:02.030 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046481256999999998, 2024-09-14T06:32:02.030 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:32:02.030 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:32:02.030 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:32:02.030 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:32:02.030 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:32:02.030 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:02.030 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 19353 2024-09-14T06:32:02.031 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:02.031 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:32:02.031 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:02.031 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:32:02.031 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062816+0000", 2024-09-14T06:32:02.031 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:02.031 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:02.031 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:02.031 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:32:02.031 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062816+0000", 2024-09-14T06:32:02.031 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:02.031 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:02.031 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:02.031 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:32:02.031 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062816+0000", 2024-09-14T06:32:02.032 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.2899999999999999e-07 2024-09-14T06:32:02.032 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:02.032 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:02.032 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:32:02.032 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062818+0000", 2024-09-14T06:32:02.032 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.699e-06 2024-09-14T06:32:02.032 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:02.032 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:02.032 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:32:02.032 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062825+0000", 2024-09-14T06:32:02.032 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.878e-06 2024-09-14T06:32:02.032 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:02.032 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:02.032 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:32:02.032 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062830+0000", 2024-09-14T06:32:02.033 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.5410000000000002e-06 2024-09-14T06:32:02.033 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:02.033 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:02.033 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:32:02.033 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.109096+0000", 2024-09-14T06:32:02.033 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046266517 2024-09-14T06:32:02.033 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:02.033 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:02.033 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:32:02.033 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.109190+0000", 2024-09-14T06:32:02.033 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.4000999999999997e-05 2024-09-14T06:32:02.033 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:02.033 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:02.034 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:32:02.034 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.109297+0000", 2024-09-14T06:32:02.034 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.000107292 2024-09-14T06:32:02.034 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:02.034 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:32:02.034 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:02.034 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:02.034 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:02.034 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:19354 3.3 3:c34e5e51:::rbd_data.112d92027590.0000000000000024:f [read 1189888~1024 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:32:02.034 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:27.062842+0000", 2024-09-14T06:32:02.034 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 154.947000433, 2024-09-14T06:32:02.034 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046594257, 2024-09-14T06:32:02.034 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:32:02.034 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:32:02.035 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:32:02.035 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:32:02.035 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:32:02.035 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:02.035 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 19354 2024-09-14T06:32:02.035 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:02.035 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:32:02.035 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:02.035 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:32:02.035 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062842+0000", 2024-09-14T06:32:02.035 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:02.035 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:02.035 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:02.035 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:32:02.035 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062842+0000", 2024-09-14T06:32:02.035 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:02.036 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:02.036 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:02.036 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:32:02.036 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062842+0000", 2024-09-14T06:32:02.036 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.7399999999999999e-07 2024-09-14T06:32:02.036 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:02.036 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:02.036 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:32:02.036 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062859+0000", 2024-09-14T06:32:02.036 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.7010000000000001e-05 2024-09-14T06:32:02.036 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:02.036 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:02.036 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:32:02.036 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062866+0000", 2024-09-14T06:32:02.037 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.9990000000000002e-06 2024-09-14T06:32:02.037 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:02.037 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:02.037 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:32:02.037 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062873+0000", 2024-09-14T06:32:02.037 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.8569999999999997e-06 2024-09-14T06:32:02.037 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:02.037 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:02.037 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:32:02.037 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.109335+0000", 2024-09-14T06:32:02.037 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046461943999999998 2024-09-14T06:32:02.037 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:02.037 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:02.038 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:32:02.038 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.109377+0000", 2024-09-14T06:32:02.038 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.2558999999999997e-05 2024-09-14T06:32:02.038 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:02.038 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:02.038 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:32:02.038 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.109436+0000", 2024-09-14T06:32:02.038 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.8514000000000003e-05 2024-09-14T06:32:02.038 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:02.038 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:32:02.038 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:02.038 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:02.038 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:02.038 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:19355 3.3 3:c34e5e51:::rbd_data.112d92027590.0000000000000024:f [read 1200128~1024 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:32:02.038 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:27.062884+0000", 2024-09-14T06:32:02.039 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 154.94695776, 2024-09-14T06:32:02.039 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046681650999999998, 2024-09-14T06:32:02.039 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:32:02.039 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:32:02.039 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:32:02.039 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:32:02.039 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:32:02.039 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:02.039 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 19355 2024-09-14T06:32:02.039 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:02.039 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:32:02.039 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:02.039 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:32:02.039 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062884+0000", 2024-09-14T06:32:02.039 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:02.040 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:02.040 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:02.040 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:32:02.040 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062884+0000", 2024-09-14T06:32:02.040 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:02.040 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:02.040 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:02.040 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:32:02.040 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062885+0000", 2024-09-14T06:32:02.040 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.8000000000000001e-07 2024-09-14T06:32:02.040 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:02.040 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:02.040 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:32:02.040 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062886+0000", 2024-09-14T06:32:02.041 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.8920000000000001e-06 2024-09-14T06:32:02.041 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:02.041 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:02.041 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:32:02.041 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062892+0000", 2024-09-14T06:32:02.041 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.84e-06 2024-09-14T06:32:02.041 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:02.041 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:02.041 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:32:02.041 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062899+0000", 2024-09-14T06:32:02.041 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.5339999999999996e-06 2024-09-14T06:32:02.041 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:02.041 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:02.041 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:32:02.041 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.109466+0000", 2024-09-14T06:32:02.042 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046566919999999998 2024-09-14T06:32:02.042 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:02.042 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:02.042 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:32:02.042 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.109510+0000", 2024-09-14T06:32:02.042 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.4153000000000002e-05 2024-09-14T06:32:02.042 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:02.042 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:02.042 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:32:02.042 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.109566+0000", 2024-09-14T06:32:02.042 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.5931999999999999e-05 2024-09-14T06:32:02.042 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:02.042 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:32:02.042 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:02.043 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:02.043 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:02.043 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:19356 3.3 3:c34e5e51:::rbd_data.112d92027590.0000000000000024:f [read 1210368~1024 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:32:02.043 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:27.062908+0000", 2024-09-14T06:32:02.043 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 154.94693432299999, 2024-09-14T06:32:02.043 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046807498000000003, 2024-09-14T06:32:02.043 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:32:02.043 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:32:02.043 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:32:02.043 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:32:02.043 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:32:02.043 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:02.043 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 19356 2024-09-14T06:32:02.044 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:02.044 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:32:02.044 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:02.044 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:32:02.044 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062908+0000", 2024-09-14T06:32:02.044 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:02.044 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:02.044 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:02.044 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:32:02.044 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062908+0000", 2024-09-14T06:32:02.044 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:02.044 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:02.044 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:02.044 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:32:02.045 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062908+0000", 2024-09-14T06:32:02.045 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.5699999999999998e-07 2024-09-14T06:32:02.045 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:02.045 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:02.045 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:32:02.045 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062910+0000", 2024-09-14T06:32:02.045 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.7710000000000001e-06 2024-09-14T06:32:02.045 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:02.045 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:02.045 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:32:02.045 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062920+0000", 2024-09-14T06:32:02.045 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.0654e-05 2024-09-14T06:32:02.045 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:02.045 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:02.046 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:32:02.046 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062931+0000", 2024-09-14T06:32:02.046 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.075e-05 2024-09-14T06:32:02.046 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:02.046 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:02.046 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:32:02.046 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.109595+0000", 2024-09-14T06:32:02.046 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046663713000000002 2024-09-14T06:32:02.046 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:02.046 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:02.046 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:32:02.046 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.109632+0000", 2024-09-14T06:32:02.046 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.7505000000000001e-05 2024-09-14T06:32:02.046 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:02.047 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:02.047 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:32:02.047 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.109715+0000", 2024-09-14T06:32:02.047 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.2748000000000001e-05 2024-09-14T06:32:02.047 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:02.047 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:32:02.047 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:02.047 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:02.047 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:02.047 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:19357 3.3 3:c34e5e51:::rbd_data.112d92027590.0000000000000024:f [read 1220608~1024 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:32:02.047 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:27.062939+0000", 2024-09-14T06:32:02.047 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 154.94690287500001, 2024-09-14T06:32:02.047 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046920508, 2024-09-14T06:32:02.047 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:32:02.048 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:32:02.048 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:32:02.048 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:32:02.048 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:32:02.048 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:02.048 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 19357 2024-09-14T06:32:02.048 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:02.048 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:32:02.048 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:02.048 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:32:02.048 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062939+0000", 2024-09-14T06:32:02.048 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:02.048 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:02.048 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:02.049 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:32:02.049 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062939+0000", 2024-09-14T06:32:02.049 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:02.049 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:02.049 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:02.049 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:32:02.049 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062940+0000", 2024-09-14T06:32:02.049 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.32e-07 2024-09-14T06:32:02.049 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:02.049 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:02.049 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:32:02.049 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062942+0000", 2024-09-14T06:32:02.049 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.1919999999999999e-06 2024-09-14T06:32:02.049 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:02.050 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:02.050 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:32:02.050 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062948+0000", 2024-09-14T06:32:02.050 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.1149999999999999e-06 2024-09-14T06:32:02.050 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:02.050 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:02.050 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:32:02.050 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062960+0000", 2024-09-14T06:32:02.050 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.1895e-05 2024-09-14T06:32:02.050 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:02.050 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:02.050 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:32:02.050 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.109704+0000", 2024-09-14T06:32:02.050 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046744584999999998 2024-09-14T06:32:02.050 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:02.051 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:02.051 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:32:02.051 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.109777+0000", 2024-09-14T06:32:02.051 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.2282999999999999e-05 2024-09-14T06:32:02.051 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:02.051 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:02.051 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:32:02.051 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.109860+0000", 2024-09-14T06:32:02.051 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.3005999999999998e-05 2024-09-14T06:32:02.051 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:02.051 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:32:02.051 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:02.051 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:02.051 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:02.052 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:19358 3.3 3:c34e5e51:::rbd_data.112d92027590.0000000000000024:f [read 1230848~1024 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:32:02.052 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:27.062967+0000", 2024-09-14T06:32:02.052 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 154.946875245, 2024-09-14T06:32:02.052 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047144471, 2024-09-14T06:32:02.052 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:32:02.052 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:32:02.052 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:32:02.052 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:32:02.052 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:32:02.052 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:02.053 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 19358 2024-09-14T06:32:02.053 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:02.053 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:32:02.053 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:02.053 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:32:02.053 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062967+0000", 2024-09-14T06:32:02.053 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:02.053 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:02.053 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:02.053 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:32:02.053 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062967+0000", 2024-09-14T06:32:02.053 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:02.053 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:02.053 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:02.054 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:32:02.054 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062967+0000", 2024-09-14T06:32:02.054 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.89e-07 2024-09-14T06:32:02.054 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:02.054 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:02.054 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:32:02.054 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062969+0000", 2024-09-14T06:32:02.054 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.9929999999999998e-06 2024-09-14T06:32:02.054 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:02.054 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:02.054 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:32:02.054 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062975+0000", 2024-09-14T06:32:02.054 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.9030000000000001e-06 2024-09-14T06:32:02.054 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:02.054 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:02.055 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:32:02.055 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062979+0000", 2024-09-14T06:32:02.055 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.2250000000000002e-06 2024-09-14T06:32:02.055 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:02.055 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:02.055 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:32:02.055 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.109934+0000", 2024-09-14T06:32:02.055 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046954410000000002 2024-09-14T06:32:02.055 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:02.055 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:02.055 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:32:02.055 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.110007+0000", 2024-09-14T06:32:02.055 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.3829000000000001e-05 2024-09-14T06:32:02.055 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:02.056 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:02.056 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:32:02.056 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.110111+0000", 2024-09-14T06:32:02.056 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00010372199999999999 2024-09-14T06:32:02.056 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:02.056 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:32:02.056 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:02.056 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:02.056 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:02.056 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:19359 3.3 3:c34e5e51:::rbd_data.112d92027590.0000000000000024:f [read 1241088~1024 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:32:02.056 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:27.062987+0000", 2024-09-14T06:32:02.056 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 154.94685515500001, 2024-09-14T06:32:02.056 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047297182, 2024-09-14T06:32:02.056 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:32:02.057 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:32:02.057 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:32:02.057 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:32:02.057 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:32:02.057 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:02.057 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 19359 2024-09-14T06:32:02.057 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:02.057 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:32:02.057 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:02.057 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:32:02.057 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062987+0000", 2024-09-14T06:32:02.057 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:02.057 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:02.057 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:02.058 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:32:02.058 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062987+0000", 2024-09-14T06:32:02.058 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:02.058 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:02.058 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:02.058 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:32:02.058 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062987+0000", 2024-09-14T06:32:02.058 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.2500000000000001e-07 2024-09-14T06:32:02.058 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:02.058 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:02.058 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:32:02.058 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062989+0000", 2024-09-14T06:32:02.058 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.6080000000000001e-06 2024-09-14T06:32:02.058 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:02.059 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:02.059 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:32:02.059 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062994+0000", 2024-09-14T06:32:02.059 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.5520000000000004e-06 2024-09-14T06:32:02.059 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:02.059 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:02.059 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:32:02.059 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062999+0000", 2024-09-14T06:32:02.059 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.3930000000000001e-06 2024-09-14T06:32:02.059 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:02.059 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:02.059 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:32:02.059 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.110142+0000", 2024-09-14T06:32:02.059 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047143154 2024-09-14T06:32:02.059 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:02.060 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:02.060 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:32:02.060 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.110219+0000", 2024-09-14T06:32:02.060 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.6737999999999996e-05 2024-09-14T06:32:02.060 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:02.060 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:02.060 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:32:02.060 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.110284+0000", 2024-09-14T06:32:02.060 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.5412000000000003e-05 2024-09-14T06:32:02.060 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:02.060 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:32:02.060 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:02.060 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:02.061 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:02.061 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:19360 3.3 3:c34e5e51:::rbd_data.112d92027590.0000000000000024:f [read 1251328~1024 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:32:02.061 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:27.063017+0000", 2024-09-14T06:32:02.061 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 154.94682469399999, 2024-09-14T06:32:02.061 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047453823999999999, 2024-09-14T06:32:02.061 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:32:02.061 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:32:02.061 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:32:02.061 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:32:02.061 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:32:02.061 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:02.061 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 19360 2024-09-14T06:32:02.061 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:02.061 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:32:02.062 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:02.062 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:32:02.062 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.063017+0000", 2024-09-14T06:32:02.062 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:02.062 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:02.062 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:02.062 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:32:02.062 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.063017+0000", 2024-09-14T06:32:02.062 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:02.062 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:02.062 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:02.062 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:32:02.062 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.064977+0000", 2024-09-14T06:32:02.062 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.0019597080000000001 2024-09-14T06:32:02.062 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:02.063 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:02.063 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:32:02.063 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.064987+0000", 2024-09-14T06:32:02.063 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.9620000000000005e-06 2024-09-14T06:32:02.063 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:02.063 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:02.063 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:32:02.063 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065000+0000", 2024-09-14T06:32:02.063 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.3173000000000001e-05 2024-09-14T06:32:02.063 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:02.063 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:02.063 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:32:02.063 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065005+0000", 2024-09-14T06:32:02.063 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.9960000000000001e-06 2024-09-14T06:32:02.064 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:02.064 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:02.064 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:32:02.064 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.110314+0000", 2024-09-14T06:32:02.064 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.04530899 2024-09-14T06:32:02.064 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:02.064 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:02.064 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:32:02.064 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.110398+0000", 2024-09-14T06:32:02.064 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.3539000000000001e-05 2024-09-14T06:32:02.064 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:02.064 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:02.064 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:32:02.064 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.110471+0000", 2024-09-14T06:32:02.065 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.3455999999999996e-05 2024-09-14T06:32:02.065 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:02.065 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:32:02.065 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:02.065 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:02.065 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:02.065 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:19365 3.3 3:c34e5e51:::rbd_data.112d92027590.0000000000000024:f [read 1302528~1024 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:32:02.065 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:27.065058+0000", 2024-09-14T06:32:02.065 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 154.944784024, 2024-09-14T06:32:02.065 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046105526000000001, 2024-09-14T06:32:02.065 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:32:02.065 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:32:02.065 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:32:02.065 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:32:02.066 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:32:02.066 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:02.066 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 19365 2024-09-14T06:32:02.066 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:02.066 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:32:02.066 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:02.066 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:32:02.066 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065058+0000", 2024-09-14T06:32:02.066 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:02.066 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:02.066 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:02.066 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:32:02.066 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065058+0000", 2024-09-14T06:32:02.066 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:02.067 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:02.067 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:02.067 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:32:02.067 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065058+0000", 2024-09-14T06:32:02.067 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.1899999999999999e-07 2024-09-14T06:32:02.067 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:02.067 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:02.067 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:32:02.067 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065059+0000", 2024-09-14T06:32:02.067 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.1299999999999998e-07 2024-09-14T06:32:02.067 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:02.067 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:02.067 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:32:02.067 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065063+0000", 2024-09-14T06:32:02.067 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.1099999999999996e-06 2024-09-14T06:32:02.068 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:02.068 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:02.068 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:32:02.068 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065070+0000", 2024-09-14T06:32:02.068 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.973e-06 2024-09-14T06:32:02.068 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:02.068 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:02.068 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:32:02.068 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.111014+0000", 2024-09-14T06:32:02.068 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.045943373000000003 2024-09-14T06:32:02.068 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:02.068 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:02.068 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:32:02.068 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.111073+0000", 2024-09-14T06:32:02.068 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.8987999999999998e-05 2024-09-14T06:32:02.069 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:02.069 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:02.069 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:32:02.069 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.111163+0000", 2024-09-14T06:32:02.069 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.0950000000000002e-05 2024-09-14T06:32:02.069 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:02.069 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:32:02.069 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:02.069 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:02.069 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:02.069 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:19366 3.3 3:c34e5e51:::rbd_data.112d92027590.0000000000000024:f [read 1312768~1024 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:32:02.069 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:27.065074+0000", 2024-09-14T06:32:02.069 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 154.94476817399999, 2024-09-14T06:32:02.069 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046289858000000003, 2024-09-14T06:32:02.070 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:32:02.070 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:32:02.070 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:32:02.070 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:32:02.070 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:32:02.070 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:02.070 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 19366 2024-09-14T06:32:02.070 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:02.070 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:32:02.070 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:02.070 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:32:02.070 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065074+0000", 2024-09-14T06:32:02.070 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:02.071 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:02.071 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:02.071 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:32:02.071 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065074+0000", 2024-09-14T06:32:02.071 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:02.071 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:02.071 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:02.071 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:32:02.071 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065074+0000", 2024-09-14T06:32:02.071 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.3900000000000001e-07 2024-09-14T06:32:02.071 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:02.071 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:02.071 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:32:02.071 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065078+0000", 2024-09-14T06:32:02.072 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.3229999999999999e-06 2024-09-14T06:32:02.072 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:02.072 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:02.072 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:32:02.072 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065082+0000", 2024-09-14T06:32:02.072 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.8299999999999998e-06 2024-09-14T06:32:02.072 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:02.072 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:02.072 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:32:02.072 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065085+0000", 2024-09-14T06:32:02.072 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.8660000000000002e-06 2024-09-14T06:32:02.072 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:02.072 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:02.072 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:32:02.073 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.111270+0000", 2024-09-14T06:32:02.073 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046185035999999999 2024-09-14T06:32:02.073 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:02.073 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:02.073 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:32:02.073 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.111311+0000", 2024-09-14T06:32:02.073 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.0935e-05 2024-09-14T06:32:02.073 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:02.073 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:02.073 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:32:02.073 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.111364+0000", 2024-09-14T06:32:02.073 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.2629e-05 2024-09-14T06:32:02.073 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:02.073 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:32:02.074 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:02.074 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:02.074 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:02.074 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:19367 3.3 3:c34e5e51:::rbd_data.112d92027590.0000000000000024:f [read 1323008~1024 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:32:02.074 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:27.065115+0000", 2024-09-14T06:32:02.074 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 154.944726613, 2024-09-14T06:32:02.074 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046407003000000002, 2024-09-14T06:32:02.074 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:32:02.074 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:32:02.074 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:32:02.074 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:32:02.074 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:32:02.074 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:02.075 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 19367 2024-09-14T06:32:02.075 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:02.075 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:32:02.075 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:02.075 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:32:02.075 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065115+0000", 2024-09-14T06:32:02.075 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:02.075 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:02.075 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:02.075 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:32:02.075 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065115+0000", 2024-09-14T06:32:02.075 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:02.075 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:02.075 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:02.075 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:32:02.076 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065116+0000", 2024-09-14T06:32:02.076 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.4999999999999998e-07 2024-09-14T06:32:02.076 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:02.076 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:02.076 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:32:02.076 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065117+0000", 2024-09-14T06:32:02.076 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.189e-06 2024-09-14T06:32:02.076 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:02.076 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:02.076 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:32:02.076 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065122+0000", 2024-09-14T06:32:02.076 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.3709999999999999e-06 2024-09-14T06:32:02.076 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:02.077 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:02.077 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:32:02.077 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065125+0000", 2024-09-14T06:32:02.077 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.8940000000000002e-06 2024-09-14T06:32:02.077 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:02.077 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:02.077 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:32:02.077 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.111402+0000", 2024-09-14T06:32:02.077 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046276645999999998 2024-09-14T06:32:02.077 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:02.077 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:02.077 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:32:02.077 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.111445+0000", 2024-09-14T06:32:02.077 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.3288000000000003e-05 2024-09-14T06:32:02.078 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:02.078 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:02.078 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:32:02.078 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.111522+0000", 2024-09-14T06:32:02.078 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.7164999999999997e-05 2024-09-14T06:32:02.078 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:02.078 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:32:02.078 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:02.078 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:02.078 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:02.078 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:19368 3.3 3:c34e5e51:::rbd_data.112d92027590.0000000000000024:f [read 1333248~1024 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:32:02.078 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:27.065130+0000", 2024-09-14T06:32:02.078 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 154.944712323, 2024-09-14T06:32:02.078 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046507275000000001, 2024-09-14T06:32:02.079 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:32:02.079 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:32:02.079 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:32:02.079 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:32:02.079 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:32:02.079 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:02.079 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 19368 2024-09-14T06:32:02.079 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:02.079 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:32:02.079 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:02.079 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:32:02.079 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065130+0000", 2024-09-14T06:32:02.079 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:02.079 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:02.080 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:02.080 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:32:02.080 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065130+0000", 2024-09-14T06:32:02.080 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:02.080 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:02.080 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:02.080 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:32:02.080 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065130+0000", 2024-09-14T06:32:02.080 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.0800000000000001e-07 2024-09-14T06:32:02.080 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:02.080 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:02.080 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:32:02.080 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065131+0000", 2024-09-14T06:32:02.081 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.8700000000000004e-07 2024-09-14T06:32:02.081 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:02.081 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:02.081 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:32:02.081 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065134+0000", 2024-09-14T06:32:02.081 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.1039999999999998e-06 2024-09-14T06:32:02.081 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:02.081 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:02.081 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:32:02.081 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065136+0000", 2024-09-14T06:32:02.081 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.3949999999999999e-06 2024-09-14T06:32:02.081 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:02.081 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:02.081 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:32:02.082 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.111541+0000", 2024-09-14T06:32:02.082 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046404206000000003 2024-09-14T06:32:02.082 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:02.082 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:02.082 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:32:02.082 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.111582+0000", 2024-09-14T06:32:02.082 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.1051999999999999e-05 2024-09-14T06:32:02.082 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:02.082 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:02.082 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:32:02.082 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.111637+0000", 2024-09-14T06:32:02.082 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.5322999999999999e-05 2024-09-14T06:32:02.082 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:02.082 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:32:02.083 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:02.083 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:02.083 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:02.083 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:19369 3.3 3:c34e5e51:::rbd_data.112d92027590.0000000000000024:f [read 1343488~1024 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:32:02.083 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:27.065139+0000", 2024-09-14T06:32:02.083 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 154.944702506, 2024-09-14T06:32:02.083 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046682169000000003, 2024-09-14T06:32:02.083 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:32:02.083 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:32:02.083 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:32:02.083 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:32:02.083 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:32:02.083 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:02.083 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 19369 2024-09-14T06:32:02.084 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:02.084 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:32:02.084 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:02.084 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:32:02.084 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065139+0000", 2024-09-14T06:32:02.084 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:02.084 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:02.084 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:02.084 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:32:02.084 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065139+0000", 2024-09-14T06:32:02.084 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:02.084 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:02.084 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:02.084 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:32:02.085 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065140+0000", 2024-09-14T06:32:02.085 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.8299999999999998e-07 2024-09-14T06:32:02.085 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:02.085 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:02.085 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:32:02.085 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065141+0000", 2024-09-14T06:32:02.085 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.3900000000000003e-07 2024-09-14T06:32:02.085 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:02.085 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:02.085 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:32:02.085 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065144+0000", 2024-09-14T06:32:02.085 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.7409999999999998e-06 2024-09-14T06:32:02.085 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:02.085 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:02.086 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:32:02.086 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065147+0000", 2024-09-14T06:32:02.086 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.4150000000000002e-06 2024-09-14T06:32:02.086 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:02.086 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:02.086 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:32:02.086 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.111673+0000", 2024-09-14T06:32:02.086 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046526004000000003 2024-09-14T06:32:02.086 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:02.086 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:02.086 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:32:02.086 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.111749+0000", 2024-09-14T06:32:02.086 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.5741999999999994e-05 2024-09-14T06:32:02.086 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:02.086 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:02.087 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:32:02.087 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.111822+0000", 2024-09-14T06:32:02.087 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.3045000000000003e-05 2024-09-14T06:32:02.087 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:02.087 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:32:02.087 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:02.087 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:02.087 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:02.087 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:19370 3.3 3:c34e5e51:::rbd_data.112d92027590.0000000000000024:f [read 1353728~1024 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:32:02.087 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:27.065150+0000", 2024-09-14T06:32:02.087 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 154.944691777, 2024-09-14T06:32:02.087 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046897556999999999, 2024-09-14T06:32:02.087 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:32:02.087 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:32:02.088 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:32:02.088 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:32:02.088 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:32:02.088 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:02.088 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 19370 2024-09-14T06:32:02.088 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:02.088 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:32:02.088 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:02.088 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:32:02.088 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065150+0000", 2024-09-14T06:32:02.088 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:02.088 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:02.088 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:02.089 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:32:02.089 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065150+0000", 2024-09-14T06:32:02.089 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:02.089 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:02.089 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:02.089 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:32:02.089 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065150+0000", 2024-09-14T06:32:02.089 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.0200000000000001e-07 2024-09-14T06:32:02.089 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:02.089 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:02.089 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:32:02.089 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065151+0000", 2024-09-14T06:32:02.089 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.1100000000000004e-07 2024-09-14T06:32:02.089 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:02.089 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:02.090 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:32:02.090 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065154+0000", 2024-09-14T06:32:02.090 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.9639999999999999e-06 2024-09-14T06:32:02.090 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:02.090 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:02.090 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:32:02.090 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065157+0000", 2024-09-14T06:32:02.090 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.4260000000000002e-06 2024-09-14T06:32:02.090 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:02.090 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:02.090 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:32:02.090 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.111861+0000", 2024-09-14T06:32:02.090 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046704268 2024-09-14T06:32:02.090 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:02.091 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:02.091 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:32:02.091 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.111939+0000", 2024-09-14T06:32:02.091 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.8231000000000004e-05 2024-09-14T06:32:02.091 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:02.091 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:02.091 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:32:02.091 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.112048+0000", 2024-09-14T06:32:02.091 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00010855500000000001 2024-09-14T06:32:02.091 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:02.091 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:32:02.091 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:02.091 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:02.091 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:02.092 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:19371 3.3 3:c34e5e51:::rbd_data.112d92027590.0000000000000024:f [read 1363968~1024 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:32:02.092 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:27.065161+0000", 2024-09-14T06:32:02.092 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 154.944681127, 2024-09-14T06:32:02.092 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047021944000000003, 2024-09-14T06:32:02.092 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:32:02.092 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:32:02.092 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:32:02.092 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:32:02.092 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:32:02.092 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:02.092 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 19371 2024-09-14T06:32:02.092 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:02.092 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:32:02.092 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:02.093 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:32:02.093 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065161+0000", 2024-09-14T06:32:02.093 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:02.093 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:02.093 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:02.093 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:32:02.093 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065161+0000", 2024-09-14T06:32:02.093 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:02.093 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:02.093 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:02.093 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:32:02.093 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065161+0000", 2024-09-14T06:32:02.093 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.0200000000000001e-07 2024-09-14T06:32:02.094 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:02.094 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:02.094 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:32:02.094 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065166+0000", 2024-09-14T06:32:02.094 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.9869999999999999e-06 2024-09-14T06:32:02.094 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:02.094 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:02.094 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:32:02.094 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065169+0000", 2024-09-14T06:32:02.094 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.3799999999999998e-06 2024-09-14T06:32:02.094 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:02.094 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:02.094 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:32:02.094 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065172+0000", 2024-09-14T06:32:02.095 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.4830000000000002e-06 2024-09-14T06:32:02.095 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:02.095 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:02.095 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:32:02.095 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.112071+0000", 2024-09-14T06:32:02.095 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046899529000000002 2024-09-14T06:32:02.095 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:02.095 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:02.095 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:32:02.095 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.112128+0000", 2024-09-14T06:32:02.095 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.6341999999999997e-05 2024-09-14T06:32:02.095 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:02.095 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:02.095 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:32:02.096 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.112183+0000", 2024-09-14T06:32:02.096 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.5021000000000001e-05 2024-09-14T06:32:02.096 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:02.096 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:32:02.096 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:02.096 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:02.096 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:02.096 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:19372 3.3 3:c34e5e51:::rbd_data.112d92027590.0000000000000024:f [read 1374208~1024 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:32:02.096 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:27.065176+0000", 2024-09-14T06:32:02.096 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 154.944665977, 2024-09-14T06:32:02.096 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047188889999999997, 2024-09-14T06:32:02.096 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:32:02.096 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:32:02.096 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:32:02.097 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:32:02.097 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:32:02.097 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:02.097 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 19372 2024-09-14T06:32:02.097 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:02.097 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:32:02.097 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:02.097 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:32:02.097 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065176+0000", 2024-09-14T06:32:02.097 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:02.097 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:02.097 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:02.097 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:32:02.097 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065176+0000", 2024-09-14T06:32:02.098 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:02.098 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:02.098 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:02.098 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:32:02.098 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.071117+0000", 2024-09-14T06:32:02.098 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.0059404990000000001 2024-09-14T06:32:02.098 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:02.098 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:02.098 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:32:02.098 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.071120+0000", 2024-09-14T06:32:02.098 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.4340000000000001e-06 2024-09-14T06:32:02.098 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:02.098 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:02.099 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:32:02.099 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.071129+0000", 2024-09-14T06:32:02.099 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.8389999999999996e-06 2024-09-14T06:32:02.099 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:02.099 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:02.099 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:32:02.099 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.071134+0000", 2024-09-14T06:32:02.099 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.412e-06 2024-09-14T06:32:02.099 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:02.099 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:02.099 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:32:02.099 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.112221+0000", 2024-09-14T06:32:02.099 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.041086473999999998 2024-09-14T06:32:02.099 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:02.100 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:02.100 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:32:02.100 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.112299+0000", 2024-09-14T06:32:02.100 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.8746999999999997e-05 2024-09-14T06:32:02.100 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:02.100 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:02.100 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:32:02.100 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.112365+0000", 2024-09-14T06:32:02.100 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.5485000000000001e-05 2024-09-14T06:32:02.100 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:02.100 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:32:02.100 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:02.100 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:02.100 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:32:02.101 INFO:teuthology.orchestra.run.smithi138.stdout:} 2024-09-14T06:32:02.102 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-14T06:32:02.102 INFO:tasks.ceph.osd.6.smithi138.stderr:2024-09-14T06:32:02.101+0000 7f8cd8849640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33620) UID: 0 2024-09-14T06:32:02.203 INFO:tasks.ceph.osd.3.smithi094.stderr:2024-09-14T06:32:02.202+0000 7feeeafe8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33655) UID: 0 2024-09-14T06:32:02.244 INFO:teuthology.orchestra.run.smithi138.stdout:{ 2024-09-14T06:32:02.244 INFO:teuthology.orchestra.run.smithi138.stdout: "ops": [], 2024-09-14T06:32:02.244 INFO:teuthology.orchestra.run.smithi138.stdout: "num_ops": 0 2024-09-14T06:32:02.244 INFO:teuthology.orchestra.run.smithi138.stdout:} 2024-09-14T06:32:02.252 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-14T06:32:02.304 INFO:tasks.ceph.osd.0.smithi094.stderr:2024-09-14T06:32:02.303+0000 7f07c9333640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33636) UID: 0 2024-09-14T06:32:02.354 INFO:teuthology.orchestra.run.smithi138.stdout:{ 2024-09-14T06:32:02.355 INFO:teuthology.orchestra.run.smithi138.stdout: "ops": [], 2024-09-14T06:32:02.355 INFO:teuthology.orchestra.run.smithi138.stdout: "complaint_time": 30, 2024-09-14T06:32:02.355 INFO:teuthology.orchestra.run.smithi138.stdout: "num_blocked_ops": 0 2024-09-14T06:32:02.355 INFO:teuthology.orchestra.run.smithi138.stdout:} 2024-09-14T06:32:02.363 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-14T06:32:02.407 INFO:tasks.ceph.osd.5.smithi138.stderr:2024-09-14T06:32:02.406+0000 7fe13746f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33618) UID: 0 2024-09-14T06:32:02.467 INFO:teuthology.orchestra.run.smithi138.stdout:{ 2024-09-14T06:32:02.467 INFO:teuthology.orchestra.run.smithi138.stdout: "size": 20, 2024-09-14T06:32:02.467 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 600, 2024-09-14T06:32:02.467 INFO:teuthology.orchestra.run.smithi138.stdout: "ops": [ 2024-09-14T06:32:02.467 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:02.467 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:11453 3.1d 3:bcad7303:::rbd_data.112d92027590.0000000000000001:f [read 94208~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:32:02.467 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:25.654375+0000", 2024-09-14T06:32:02.468 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 156.81094092999999, 2024-09-14T06:32:02.468 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052449318000000002, 2024-09-14T06:32:02.468 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:32:02.468 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:32:02.468 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:32:02.468 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:32:02.468 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:32:02.468 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:02.468 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 11453 2024-09-14T06:32:02.468 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:02.468 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:32:02.468 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:02.469 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:32:02.469 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654375+0000", 2024-09-14T06:32:02.469 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:02.469 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:02.469 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:02.469 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:32:02.469 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654375+0000", 2024-09-14T06:32:02.469 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:02.469 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:02.469 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:02.469 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:32:02.469 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654496+0000", 2024-09-14T06:32:02.469 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00012081 2024-09-14T06:32:02.469 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:02.470 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:02.470 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:32:02.470 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654497+0000", 2024-09-14T06:32:02.470 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.8789999999999999e-06 2024-09-14T06:32:02.470 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:02.470 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:02.470 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:32:02.470 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654501+0000", 2024-09-14T06:32:02.470 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.8999999999999999e-06 2024-09-14T06:32:02.470 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:02.470 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:02.470 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:32:02.470 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654505+0000", 2024-09-14T06:32:02.470 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.2229999999999999e-06 2024-09-14T06:32:02.470 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:02.471 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:02.471 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:32:02.471 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.705917+0000", 2024-09-14T06:32:02.471 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.051412501999999999 2024-09-14T06:32:02.471 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:02.471 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:02.471 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:32:02.471 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.705999+0000", 2024-09-14T06:32:02.471 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.1551999999999994e-05 2024-09-14T06:32:02.471 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:02.471 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:02.471 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:32:02.471 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.706824+0000", 2024-09-14T06:32:02.472 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00082545199999999996 2024-09-14T06:32:02.472 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:02.472 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:32:02.472 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:02.472 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:02.472 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:02.472 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:11454 3.1d 3:bcad7303:::rbd_data.112d92027590.0000000000000001:f [read 104448~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:32:02.472 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:25.654510+0000", 2024-09-14T06:32:02.472 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 156.81080614000001, 2024-09-14T06:32:02.472 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052610713000000003, 2024-09-14T06:32:02.472 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:32:02.472 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:32:02.472 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:32:02.472 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:32:02.472 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:32:02.473 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:02.473 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 11454 2024-09-14T06:32:02.473 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:02.473 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:32:02.473 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:02.473 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:32:02.473 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654510+0000", 2024-09-14T06:32:02.473 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:02.473 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:02.473 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:02.473 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:32:02.473 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654510+0000", 2024-09-14T06:32:02.473 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:02.474 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:02.474 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:02.474 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:32:02.474 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654825+0000", 2024-09-14T06:32:02.474 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00031522400000000002 2024-09-14T06:32:02.474 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:02.474 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:02.474 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:32:02.474 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654827+0000", 2024-09-14T06:32:02.474 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.9709999999999998e-06 2024-09-14T06:32:02.474 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:02.474 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:02.474 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:32:02.474 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654832+0000", 2024-09-14T06:32:02.475 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.4099999999999999e-06 2024-09-14T06:32:02.475 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:02.475 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:02.475 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:32:02.475 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654835+0000", 2024-09-14T06:32:02.475 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.14e-06 2024-09-14T06:32:02.475 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:02.475 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:02.475 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:32:02.475 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.706851+0000", 2024-09-14T06:32:02.475 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052015761000000001 2024-09-14T06:32:02.475 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:02.475 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:02.475 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:32:02.476 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.706934+0000", 2024-09-14T06:32:02.476 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.2484999999999994e-05 2024-09-14T06:32:02.476 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:02.476 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:02.476 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:32:02.476 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.707120+0000", 2024-09-14T06:32:02.476 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00018672200000000001 2024-09-14T06:32:02.476 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:02.476 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:32:02.476 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:02.476 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:02.476 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:02.476 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:11455 3.1d 3:bcad7303:::rbd_data.112d92027590.0000000000000001:f [read 114688~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:32:02.477 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:25.654841+0000", 2024-09-14T06:32:02.477 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 156.81047481100001, 2024-09-14T06:32:02.477 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052579898, 2024-09-14T06:32:02.477 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:32:02.477 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:32:02.477 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:32:02.477 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:32:02.477 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:32:02.477 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:02.477 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 11455 2024-09-14T06:32:02.477 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:02.477 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:32:02.477 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:02.477 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:32:02.478 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654841+0000", 2024-09-14T06:32:02.478 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:02.478 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:02.478 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:02.478 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:32:02.478 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654841+0000", 2024-09-14T06:32:02.478 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:02.478 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:02.478 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:02.478 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:32:02.478 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654841+0000", 2024-09-14T06:32:02.478 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.7300000000000002e-07 2024-09-14T06:32:02.478 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:02.478 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:02.478 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:32:02.479 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654845+0000", 2024-09-14T06:32:02.479 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.0520000000000003e-06 2024-09-14T06:32:02.479 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:02.479 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:02.479 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:32:02.479 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654849+0000", 2024-09-14T06:32:02.479 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.0459999999999999e-06 2024-09-14T06:32:02.479 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:02.479 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:02.479 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:32:02.479 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654852+0000", 2024-09-14T06:32:02.479 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.9830000000000001e-06 2024-09-14T06:32:02.479 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:02.479 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:02.480 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:32:02.480 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.707159+0000", 2024-09-14T06:32:02.480 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052306826000000001 2024-09-14T06:32:02.480 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:02.480 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:02.480 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:32:02.480 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.707242+0000", 2024-09-14T06:32:02.480 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.2595000000000005e-05 2024-09-14T06:32:02.480 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:02.480 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:02.480 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:32:02.480 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.707421+0000", 2024-09-14T06:32:02.480 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.000179123 2024-09-14T06:32:02.480 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:02.480 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:32:02.481 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:02.481 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:02.481 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:02.481 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:11456 3.1d 3:bcad7303:::rbd_data.112d92027590.0000000000000001:f [read 124928~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:32:02.481 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:25.654856+0000", 2024-09-14T06:32:02.481 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 156.81045996500001, 2024-09-14T06:32:02.481 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052801580000000001, 2024-09-14T06:32:02.481 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:32:02.481 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:32:02.481 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:32:02.481 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:32:02.481 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:32:02.481 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:02.482 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 11456 2024-09-14T06:32:02.482 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:02.482 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:32:02.482 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:02.482 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:32:02.482 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654856+0000", 2024-09-14T06:32:02.482 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:02.482 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:02.482 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:02.482 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:32:02.482 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654856+0000", 2024-09-14T06:32:02.482 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:02.482 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:02.482 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:02.482 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:32:02.483 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654856+0000", 2024-09-14T06:32:02.483 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.29e-07 2024-09-14T06:32:02.483 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:02.483 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:02.483 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:32:02.483 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654857+0000", 2024-09-14T06:32:02.483 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.187e-06 2024-09-14T06:32:02.483 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:02.483 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:02.483 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:32:02.483 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654861+0000", 2024-09-14T06:32:02.483 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.3850000000000002e-06 2024-09-14T06:32:02.483 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:02.484 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:02.484 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:32:02.484 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654863+0000", 2024-09-14T06:32:02.484 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.9500000000000001e-06 2024-09-14T06:32:02.484 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:02.484 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:02.484 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:32:02.484 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.707456+0000", 2024-09-14T06:32:02.484 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052592434 2024-09-14T06:32:02.484 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:02.484 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:02.484 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:32:02.484 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.707516+0000", 2024-09-14T06:32:02.484 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.0167999999999997e-05 2024-09-14T06:32:02.484 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:02.485 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:02.485 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:32:02.485 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.707657+0000", 2024-09-14T06:32:02.485 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00014122700000000001 2024-09-14T06:32:02.485 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:02.485 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:32:02.485 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:02.485 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:02.485 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:02.485 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:11457 3.1d 3:bcad7303:::rbd_data.112d92027590.0000000000000001:f [read 135168~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:32:02.485 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:25.654867+0000", 2024-09-14T06:32:02.485 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 156.81044908000001, 2024-09-14T06:32:02.485 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052921902, 2024-09-14T06:32:02.485 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:32:02.486 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:32:02.486 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:32:02.486 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:32:02.486 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:32:02.486 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:02.486 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 11457 2024-09-14T06:32:02.486 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:02.486 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:32:02.486 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:02.486 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:32:02.486 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654867+0000", 2024-09-14T06:32:02.486 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:02.486 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:02.486 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:02.486 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:32:02.487 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654867+0000", 2024-09-14T06:32:02.487 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:02.487 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:02.487 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:02.487 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:32:02.487 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654972+0000", 2024-09-14T06:32:02.487 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00010584 2024-09-14T06:32:02.487 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:02.487 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:02.487 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:32:02.487 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654975+0000", 2024-09-14T06:32:02.487 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.1169999999999998e-06 2024-09-14T06:32:02.487 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:02.487 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:02.488 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:32:02.488 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654979+0000", 2024-09-14T06:32:02.488 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.1960000000000001e-06 2024-09-14T06:32:02.488 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:02.488 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:02.488 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:32:02.488 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654982+0000", 2024-09-14T06:32:02.488 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.2990000000000001e-06 2024-09-14T06:32:02.488 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:02.488 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:02.488 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:32:02.488 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.707684+0000", 2024-09-14T06:32:02.488 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052701979000000003 2024-09-14T06:32:02.488 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:02.489 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:02.489 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:32:02.489 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.707728+0000", 2024-09-14T06:32:02.489 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.3529999999999998e-05 2024-09-14T06:32:02.489 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:02.489 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:02.489 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:32:02.489 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.707789+0000", 2024-09-14T06:32:02.489 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.0940999999999998e-05 2024-09-14T06:32:02.489 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:02.489 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:32:02.489 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:02.489 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:02.490 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:02.490 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:11458 3.1d 3:bcad7303:::rbd_data.112d92027590.0000000000000001:f [read 145408~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:32:02.490 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:25.654987+0000", 2024-09-14T06:32:02.490 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 156.81032868200001, 2024-09-14T06:32:02.490 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.05291655, 2024-09-14T06:32:02.490 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:32:02.490 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:32:02.490 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:32:02.490 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:32:02.490 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:32:02.490 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:02.490 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 11458 2024-09-14T06:32:02.490 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:02.490 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:32:02.490 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:02.491 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:32:02.491 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654987+0000", 2024-09-14T06:32:02.491 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:02.491 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:02.491 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:02.491 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:32:02.491 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654987+0000", 2024-09-14T06:32:02.491 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:02.491 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:02.491 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:02.491 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:32:02.491 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655077+0000", 2024-09-14T06:32:02.491 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.0253000000000005e-05 2024-09-14T06:32:02.491 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:02.491 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:02.492 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:32:02.492 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655079+0000", 2024-09-14T06:32:02.492 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.9989999999999998e-06 2024-09-14T06:32:02.492 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:02.492 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:02.492 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:32:02.492 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655083+0000", 2024-09-14T06:32:02.492 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.9280000000000003e-06 2024-09-14T06:32:02.492 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:02.492 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:02.492 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:32:02.492 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655086+0000", 2024-09-14T06:32:02.492 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.056e-06 2024-09-14T06:32:02.493 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:02.493 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:02.493 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:32:02.493 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.707805+0000", 2024-09-14T06:32:02.493 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052718627999999997 2024-09-14T06:32:02.493 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:02.493 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:02.493 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:32:02.493 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.707848+0000", 2024-09-14T06:32:02.493 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.2728999999999997e-05 2024-09-14T06:32:02.493 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:02.493 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:02.493 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:32:02.493 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.707904+0000", 2024-09-14T06:32:02.493 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.5957e-05 2024-09-14T06:32:02.494 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:02.494 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:32:02.494 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:02.494 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:02.494 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:02.494 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:11459 3.1d 3:bcad7303:::rbd_data.112d92027590.0000000000000001:f [read 155648~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:32:02.494 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:25.655091+0000", 2024-09-14T06:32:02.494 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 156.81022467400001, 2024-09-14T06:32:02.494 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052944242000000002, 2024-09-14T06:32:02.494 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:32:02.494 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:32:02.494 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:32:02.494 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:32:02.495 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:32:02.495 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:02.495 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 11459 2024-09-14T06:32:02.495 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:02.495 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:32:02.495 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:02.495 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:32:02.495 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655091+0000", 2024-09-14T06:32:02.495 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:02.495 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:02.495 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:02.495 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:32:02.495 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655091+0000", 2024-09-14T06:32:02.495 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:02.496 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:02.496 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:02.496 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:32:02.496 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655237+0000", 2024-09-14T06:32:02.496 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00014570100000000001 2024-09-14T06:32:02.496 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:02.496 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:02.496 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:32:02.496 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655239+0000", 2024-09-14T06:32:02.496 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.8670000000000001e-06 2024-09-14T06:32:02.496 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:02.496 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:02.496 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:32:02.496 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655243+0000", 2024-09-14T06:32:02.497 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.9829999999999998e-06 2024-09-14T06:32:02.497 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:02.497 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:02.497 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:32:02.497 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655246+0000", 2024-09-14T06:32:02.497 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.1540000000000002e-06 2024-09-14T06:32:02.497 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:02.497 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:02.497 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:32:02.497 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.707934+0000", 2024-09-14T06:32:02.497 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052688286000000001 2024-09-14T06:32:02.497 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:02.497 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:02.497 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:32:02.497 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.707977+0000", 2024-09-14T06:32:02.498 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.3291999999999998e-05 2024-09-14T06:32:02.498 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:02.498 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:02.498 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:32:02.498 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.708035+0000", 2024-09-14T06:32:02.498 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.7958999999999999e-05 2024-09-14T06:32:02.498 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:02.498 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:32:02.498 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:02.498 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:02.498 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:02.498 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:11460 3.1d 3:bcad7303:::rbd_data.112d92027590.0000000000000001:f [read 165888~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:32:02.498 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:25.655250+0000", 2024-09-14T06:32:02.499 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 156.810065377, 2024-09-14T06:32:02.499 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052910698999999999, 2024-09-14T06:32:02.499 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:32:02.499 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:32:02.499 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:32:02.499 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:32:02.499 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:32:02.499 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:02.499 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 11460 2024-09-14T06:32:02.499 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:02.499 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:32:02.499 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:02.499 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:32:02.499 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655250+0000", 2024-09-14T06:32:02.500 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:02.500 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:02.500 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:02.500 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:32:02.500 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655250+0000", 2024-09-14T06:32:02.500 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:02.500 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:02.500 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:02.500 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:32:02.500 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655356+0000", 2024-09-14T06:32:02.500 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00010535900000000001 2024-09-14T06:32:02.500 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:02.500 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:02.500 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:32:02.501 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655357+0000", 2024-09-14T06:32:02.501 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.8080000000000001e-06 2024-09-14T06:32:02.501 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:02.501 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:02.501 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:32:02.501 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655361+0000", 2024-09-14T06:32:02.501 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.861e-06 2024-09-14T06:32:02.501 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:02.501 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:02.501 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:32:02.501 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655364+0000", 2024-09-14T06:32:02.501 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.1089999999999998e-06 2024-09-14T06:32:02.501 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:02.501 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:02.502 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:32:02.502 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.708065+0000", 2024-09-14T06:32:02.502 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052700595000000003 2024-09-14T06:32:02.502 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:02.502 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:02.502 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:32:02.502 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.708108+0000", 2024-09-14T06:32:02.502 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.2484999999999997e-05 2024-09-14T06:32:02.502 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:02.502 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:02.502 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:32:02.502 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.708161+0000", 2024-09-14T06:32:02.502 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.3482000000000001e-05 2024-09-14T06:32:02.502 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:02.503 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:32:02.503 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:02.503 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:02.503 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:02.503 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:11461 3.1d 3:bcad7303:::rbd_data.112d92027590.0000000000000001:f [read 176128~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:32:02.503 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:25.655370+0000", 2024-09-14T06:32:02.503 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 156.80994609499999, 2024-09-14T06:32:02.503 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052915141999999998, 2024-09-14T06:32:02.503 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:32:02.503 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:32:02.503 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:32:02.503 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:32:02.503 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:32:02.503 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:02.504 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 11461 2024-09-14T06:32:02.504 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:02.504 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:32:02.504 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:02.504 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:32:02.504 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655370+0000", 2024-09-14T06:32:02.504 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:02.504 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:02.504 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:02.504 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:32:02.504 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655370+0000", 2024-09-14T06:32:02.504 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:02.504 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:02.504 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:02.505 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:32:02.505 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655394+0000", 2024-09-14T06:32:02.505 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.4553999999999999e-05 2024-09-14T06:32:02.505 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:02.505 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:02.505 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:32:02.505 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655396+0000", 2024-09-14T06:32:02.505 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.8950000000000001e-06 2024-09-14T06:32:02.505 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:02.505 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:02.505 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:32:02.505 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655401+0000", 2024-09-14T06:32:02.505 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.0089999999999998e-06 2024-09-14T06:32:02.505 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:02.506 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:02.506 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:32:02.506 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655404+0000", 2024-09-14T06:32:02.506 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.3400000000000002e-06 2024-09-14T06:32:02.506 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:02.506 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:02.506 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:32:02.506 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.708186+0000", 2024-09-14T06:32:02.506 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052781726000000001 2024-09-14T06:32:02.506 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:02.506 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:02.506 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:32:02.506 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.708229+0000", 2024-09-14T06:32:02.506 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.2670999999999999e-05 2024-09-14T06:32:02.507 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:02.507 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:02.507 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:32:02.507 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.708285+0000", 2024-09-14T06:32:02.507 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.5946999999999998e-05 2024-09-14T06:32:02.507 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:02.507 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:32:02.507 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:02.507 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:02.507 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:02.507 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:11462 3.1d 3:bcad7303:::rbd_data.112d92027590.0000000000000001:f [read 186368~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:32:02.507 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:25.655410+0000", 2024-09-14T06:32:02.507 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 156.809905617, 2024-09-14T06:32:02.507 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052995258000000003, 2024-09-14T06:32:02.507 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:32:02.508 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:32:02.508 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:32:02.508 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:32:02.508 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:32:02.508 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:02.508 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 11462 2024-09-14T06:32:02.508 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:02.508 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:32:02.508 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:02.508 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:32:02.508 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655410+0000", 2024-09-14T06:32:02.508 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:02.508 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:02.508 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:02.509 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:32:02.509 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655410+0000", 2024-09-14T06:32:02.509 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:02.509 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:02.509 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:02.509 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:32:02.509 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655625+0000", 2024-09-14T06:32:02.509 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00021442600000000001 2024-09-14T06:32:02.509 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:02.509 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:02.509 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:32:02.509 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655626+0000", 2024-09-14T06:32:02.509 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.815e-06 2024-09-14T06:32:02.509 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:02.510 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:02.510 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:32:02.510 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655631+0000", 2024-09-14T06:32:02.510 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.651e-06 2024-09-14T06:32:02.510 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:02.510 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:02.510 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:32:02.510 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655634+0000", 2024-09-14T06:32:02.510 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.2449999999999998e-06 2024-09-14T06:32:02.510 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:02.510 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:02.510 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:32:02.510 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.708312+0000", 2024-09-14T06:32:02.510 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052677412 2024-09-14T06:32:02.511 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:02.511 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:02.511 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:32:02.511 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.708352+0000", 2024-09-14T06:32:02.511 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.0729999999999998e-05 2024-09-14T06:32:02.511 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:02.511 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:02.511 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:32:02.511 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.708405+0000", 2024-09-14T06:32:02.511 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.2979000000000002e-05 2024-09-14T06:32:02.511 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:02.511 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:32:02.511 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:02.512 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:02.512 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:02.512 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:11463 3.1d 3:bcad7303:::rbd_data.112d92027590.0000000000000001:f [read 196608~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:32:02.512 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:25.655639+0000", 2024-09-14T06:32:02.512 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 156.80967647099999, 2024-09-14T06:32:02.512 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052888859000000003, 2024-09-14T06:32:02.512 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:32:02.512 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:32:02.512 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:32:02.512 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:32:02.512 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:32:02.512 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:02.512 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 11463 2024-09-14T06:32:02.512 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:02.513 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:32:02.513 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:02.513 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:32:02.513 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655639+0000", 2024-09-14T06:32:02.513 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:02.513 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:02.513 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:02.513 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:32:02.513 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655639+0000", 2024-09-14T06:32:02.513 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:02.513 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:02.513 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:02.513 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:32:02.513 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655640+0000", 2024-09-14T06:32:02.514 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.08e-07 2024-09-14T06:32:02.514 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:02.514 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:02.514 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:32:02.514 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655641+0000", 2024-09-14T06:32:02.514 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.156e-06 2024-09-14T06:32:02.514 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:02.514 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:02.514 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:32:02.514 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655648+0000", 2024-09-14T06:32:02.514 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.8460000000000002e-06 2024-09-14T06:32:02.514 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:02.514 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:02.514 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:32:02.515 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655651+0000", 2024-09-14T06:32:02.515 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.0079999999999998e-06 2024-09-14T06:32:02.515 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:02.515 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:02.515 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:32:02.515 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.708429+0000", 2024-09-14T06:32:02.515 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052778459 2024-09-14T06:32:02.515 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:02.515 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:02.515 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:32:02.515 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.708473+0000", 2024-09-14T06:32:02.515 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.3782999999999997e-05 2024-09-14T06:32:02.515 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:02.515 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:02.516 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:32:02.516 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.708528+0000", 2024-09-14T06:32:02.516 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.5198999999999998e-05 2024-09-14T06:32:02.516 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:02.516 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:32:02.516 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:02.516 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:02.516 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:02.516 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:11464 3.1d 3:bcad7303:::rbd_data.112d92027590.0000000000000001:f [read 206848~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:32:02.516 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:25.655654+0000", 2024-09-14T06:32:02.516 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 156.80966160200001, 2024-09-14T06:32:02.516 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052981845999999999, 2024-09-14T06:32:02.516 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:32:02.516 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:32:02.517 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:32:02.517 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:32:02.517 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:32:02.517 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:02.517 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 11464 2024-09-14T06:32:02.517 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:02.517 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:32:02.517 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:02.517 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:32:02.517 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655654+0000", 2024-09-14T06:32:02.517 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:02.517 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:02.517 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:02.517 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:32:02.518 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655654+0000", 2024-09-14T06:32:02.518 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:02.518 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:02.518 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:02.518 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:32:02.518 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655946+0000", 2024-09-14T06:32:02.518 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00029148200000000003 2024-09-14T06:32:02.518 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:02.518 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:02.518 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:32:02.518 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655948+0000", 2024-09-14T06:32:02.518 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.973e-06 2024-09-14T06:32:02.518 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:02.518 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:02.519 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:32:02.519 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655953+0000", 2024-09-14T06:32:02.519 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.1240000000000004e-06 2024-09-14T06:32:02.519 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:02.519 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:02.519 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:32:02.519 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655956+0000", 2024-09-14T06:32:02.519 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.315e-06 2024-09-14T06:32:02.519 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:02.519 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:02.519 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:32:02.519 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.708543+0000", 2024-09-14T06:32:02.519 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052587416999999997 2024-09-14T06:32:02.519 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:02.520 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:02.520 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:32:02.520 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.708584+0000", 2024-09-14T06:32:02.520 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.1004000000000002e-05 2024-09-14T06:32:02.520 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:02.520 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:02.520 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:32:02.520 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.708636+0000", 2024-09-14T06:32:02.520 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.1530999999999998e-05 2024-09-14T06:32:02.520 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:02.520 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:32:02.520 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:02.520 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:02.520 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:02.521 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:11465 3.1d 3:bcad7303:::rbd_data.112d92027590.0000000000000001:f [read 217088~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:32:02.521 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:25.655961+0000", 2024-09-14T06:32:02.521 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 156.80935445, 2024-09-14T06:32:02.521 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052798682999999999, 2024-09-14T06:32:02.521 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:32:02.521 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:32:02.521 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:32:02.521 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:32:02.521 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:32:02.521 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:02.521 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 11465 2024-09-14T06:32:02.521 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:02.521 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:32:02.522 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:02.522 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:32:02.522 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655961+0000", 2024-09-14T06:32:02.522 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:02.522 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:02.522 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:02.522 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:32:02.522 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655961+0000", 2024-09-14T06:32:02.522 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:02.522 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:02.522 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:02.522 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:32:02.522 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.656114+0000", 2024-09-14T06:32:02.522 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.000152745 2024-09-14T06:32:02.522 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:02.523 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:02.523 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:32:02.523 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.656116+0000", 2024-09-14T06:32:02.523 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.804e-06 2024-09-14T06:32:02.523 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:02.523 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:02.523 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:32:02.523 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.656120+0000", 2024-09-14T06:32:02.523 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.4700000000000004e-06 2024-09-14T06:32:02.523 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:02.523 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:02.523 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:32:02.523 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.656123+0000", 2024-09-14T06:32:02.524 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.1269999999999999e-06 2024-09-14T06:32:02.524 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:02.524 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:02.524 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:32:02.524 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.708660+0000", 2024-09-14T06:32:02.524 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052536919000000001 2024-09-14T06:32:02.524 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:02.524 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:02.524 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:32:02.524 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.708703+0000", 2024-09-14T06:32:02.524 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.3077000000000001e-05 2024-09-14T06:32:02.524 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:02.524 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:02.524 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:32:02.525 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.708760+0000", 2024-09-14T06:32:02.525 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.6541e-05 2024-09-14T06:32:02.525 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:02.525 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:32:02.525 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:02.525 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:02.525 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:02.525 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:11466 3.1d 3:bcad7303:::rbd_data.112d92027590.0000000000000001:f [read 227328~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:32:02.525 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:25.656129+0000", 2024-09-14T06:32:02.525 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 156.80918706200001, 2024-09-14T06:32:02.525 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052750987999999999, 2024-09-14T06:32:02.525 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:32:02.525 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:32:02.526 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:32:02.526 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:32:02.526 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:32:02.526 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:02.526 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 11466 2024-09-14T06:32:02.526 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:02.526 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:32:02.526 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:02.526 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:32:02.526 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.656129+0000", 2024-09-14T06:32:02.526 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:02.526 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:02.526 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:02.526 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:32:02.526 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.656129+0000", 2024-09-14T06:32:02.527 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:02.527 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:02.527 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:02.527 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:32:02.527 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.656347+0000", 2024-09-14T06:32:02.527 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.000218054 2024-09-14T06:32:02.527 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:02.527 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:02.527 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:32:02.527 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.656349+0000", 2024-09-14T06:32:02.527 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.9199999999999998e-06 2024-09-14T06:32:02.527 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:02.527 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:02.527 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:32:02.528 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.656353+0000", 2024-09-14T06:32:02.528 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.9210000000000002e-06 2024-09-14T06:32:02.528 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:02.528 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:02.528 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:32:02.528 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.656356+0000", 2024-09-14T06:32:02.528 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.7670000000000001e-06 2024-09-14T06:32:02.528 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:02.528 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:02.528 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:32:02.528 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.708787+0000", 2024-09-14T06:32:02.528 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052430840999999999 2024-09-14T06:32:02.528 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:02.528 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:02.529 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:32:02.529 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.708827+0000", 2024-09-14T06:32:02.529 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.9538e-05 2024-09-14T06:32:02.529 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:02.529 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:02.529 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:32:02.529 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.708880+0000", 2024-09-14T06:32:02.529 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.2947e-05 2024-09-14T06:32:02.529 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:02.529 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:32:02.529 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:02.529 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:02.529 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:02.529 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:11467 3.1d 3:bcad7303:::rbd_data.112d92027590.0000000000000001:f [read 237568~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:32:02.530 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:25.656361+0000", 2024-09-14T06:32:02.530 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 156.808954414, 2024-09-14T06:32:02.530 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052643898000000001, 2024-09-14T06:32:02.530 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:32:02.530 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:32:02.530 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:32:02.530 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:32:02.530 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:32:02.530 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:02.530 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 11467 2024-09-14T06:32:02.530 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:02.530 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:32:02.530 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:02.530 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:32:02.531 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.656361+0000", 2024-09-14T06:32:02.531 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:02.531 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:02.531 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:02.531 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:32:02.531 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.656361+0000", 2024-09-14T06:32:02.531 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:02.531 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:02.531 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:02.531 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:32:02.532 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.656855+0000", 2024-09-14T06:32:02.532 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00049372000000000001 2024-09-14T06:32:02.532 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:02.532 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:02.532 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:32:02.532 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.656859+0000", 2024-09-14T06:32:02.532 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.1300000000000003e-06 2024-09-14T06:32:02.532 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:02.532 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:02.532 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:32:02.532 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.656863+0000", 2024-09-14T06:32:02.532 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.2629999999999997e-06 2024-09-14T06:32:02.532 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:02.532 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:02.533 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:32:02.533 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.656867+0000", 2024-09-14T06:32:02.533 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.3220000000000001e-06 2024-09-14T06:32:02.533 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:02.533 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:02.533 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:32:02.533 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.708903+0000", 2024-09-14T06:32:02.533 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052035855999999998 2024-09-14T06:32:02.533 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:02.533 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:02.533 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:32:02.533 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.708949+0000", 2024-09-14T06:32:02.533 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.6276999999999998e-05 2024-09-14T06:32:02.534 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:02.534 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:02.534 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:32:02.534 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.709005+0000", 2024-09-14T06:32:02.534 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.6329999999999998e-05 2024-09-14T06:32:02.534 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:02.534 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:32:02.534 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:02.534 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:02.534 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:02.534 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:11480 3.1d 3:bcad7303:::rbd_data.112d92027590.0000000000000001:f [read 370688~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:32:02.534 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:25.658563+0000", 2024-09-14T06:32:02.534 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 156.80675264999999, 2024-09-14T06:32:02.535 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052429164, 2024-09-14T06:32:02.535 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:32:02.535 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:32:02.535 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:32:02.535 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:32:02.535 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:32:02.535 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:02.535 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 11480 2024-09-14T06:32:02.535 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:02.535 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:32:02.535 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:02.535 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:32:02.535 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.658563+0000", 2024-09-14T06:32:02.535 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:02.536 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:02.536 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:02.536 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:32:02.536 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.658563+0000", 2024-09-14T06:32:02.536 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:02.536 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:02.536 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:02.536 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:32:02.536 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.658658+0000", 2024-09-14T06:32:02.536 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.5346999999999994e-05 2024-09-14T06:32:02.536 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:02.536 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:02.536 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:32:02.536 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.658662+0000", 2024-09-14T06:32:02.537 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.14e-06 2024-09-14T06:32:02.537 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:02.537 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:02.537 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:32:02.537 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.658669+0000", 2024-09-14T06:32:02.537 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.7379999999999994e-06 2024-09-14T06:32:02.537 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:02.537 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:02.537 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:32:02.537 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.658675+0000", 2024-09-14T06:32:02.537 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.5430000000000001e-06 2024-09-14T06:32:02.537 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:02.537 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:02.537 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:32:02.538 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.710824+0000", 2024-09-14T06:32:02.538 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052148847999999998 2024-09-14T06:32:02.538 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:02.538 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:02.538 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:32:02.538 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.710864+0000", 2024-09-14T06:32:02.538 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.9925999999999997e-05 2024-09-14T06:32:02.538 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:02.538 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:02.538 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:32:02.538 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.710992+0000", 2024-09-14T06:32:02.538 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.000128622 2024-09-14T06:32:02.538 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:02.538 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:32:02.538 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:02.539 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:02.539 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:02.539 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:11481 3.1d 3:bcad7303:::rbd_data.112d92027590.0000000000000001:f [read 380928~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:32:02.539 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:25.658684+0000", 2024-09-14T06:32:02.539 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 156.80663157999999, 2024-09-14T06:32:02.539 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052508565, 2024-09-14T06:32:02.539 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:32:02.539 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:32:02.539 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:32:02.539 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:32:02.539 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:32:02.539 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:02.539 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 11481 2024-09-14T06:32:02.539 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:02.540 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:32:02.540 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:02.540 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:32:02.540 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.658684+0000", 2024-09-14T06:32:02.540 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:02.540 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:02.540 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:02.540 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:32:02.540 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.658684+0000", 2024-09-14T06:32:02.540 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:02.540 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:02.540 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:02.540 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:32:02.540 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.658715+0000", 2024-09-14T06:32:02.541 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.0703000000000001e-05 2024-09-14T06:32:02.541 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:02.541 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:02.541 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:32:02.541 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.658718+0000", 2024-09-14T06:32:02.541 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.0709999999999999e-06 2024-09-14T06:32:02.541 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:02.541 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:02.541 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:32:02.541 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.658725+0000", 2024-09-14T06:32:02.541 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.2339999999999997e-06 2024-09-14T06:32:02.541 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:02.541 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:02.541 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:32:02.542 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.658731+0000", 2024-09-14T06:32:02.542 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.6450000000000002e-06 2024-09-14T06:32:02.542 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:02.542 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:02.542 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:32:02.542 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.711020+0000", 2024-09-14T06:32:02.542 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052288886 2024-09-14T06:32:02.542 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:02.542 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:02.542 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:32:02.542 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.711061+0000", 2024-09-14T06:32:02.542 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.1084000000000002e-05 2024-09-14T06:32:02.542 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:02.542 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:02.542 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:32:02.543 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.711193+0000", 2024-09-14T06:32:02.543 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00013194199999999999 2024-09-14T06:32:02.543 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:02.543 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:32:02.543 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:02.543 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:02.543 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:02.543 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:11482 3.1d 3:bcad7303:::rbd_data.112d92027590.0000000000000001:f [read 391168~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:32:02.543 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:25.658740+0000", 2024-09-14T06:32:02.543 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 156.80657611500001, 2024-09-14T06:32:02.543 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052637015000000002, 2024-09-14T06:32:02.543 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:32:02.543 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:32:02.543 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:32:02.544 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:32:02.544 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:32:02.544 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:02.544 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 11482 2024-09-14T06:32:02.544 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:02.544 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:32:02.544 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:02.544 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:32:02.544 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.658740+0000", 2024-09-14T06:32:02.544 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:02.544 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:02.544 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:02.544 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:32:02.544 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.658740+0000", 2024-09-14T06:32:02.545 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:02.545 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:02.545 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:02.545 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:32:02.545 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.658880+0000", 2024-09-14T06:32:02.545 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00014036100000000001 2024-09-14T06:32:02.545 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:02.545 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:02.545 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:32:02.545 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.658883+0000", 2024-09-14T06:32:02.545 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.3969999999999998e-06 2024-09-14T06:32:02.545 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:02.545 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:02.546 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:32:02.546 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.658890+0000", 2024-09-14T06:32:02.546 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.1500000000000002e-06 2024-09-14T06:32:02.546 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:02.546 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:02.546 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:32:02.546 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.658897+0000", 2024-09-14T06:32:02.546 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.1260000000000003e-06 2024-09-14T06:32:02.546 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:02.546 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:02.546 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:32:02.546 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.711207+0000", 2024-09-14T06:32:02.546 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052310777000000003 2024-09-14T06:32:02.546 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:02.547 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:02.547 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:32:02.547 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.711249+0000", 2024-09-14T06:32:02.547 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.1786999999999998e-05 2024-09-14T06:32:02.547 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:02.547 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:02.547 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:32:02.547 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.711377+0000", 2024-09-14T06:32:02.547 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.000127417 2024-09-14T06:32:02.547 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:02.547 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:32:02.547 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:02.547 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:02.547 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:02.548 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:11483 3.1d 3:bcad7303:::rbd_data.112d92027590.0000000000000001:f [read 401408~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:32:02.548 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:25.658906+0000", 2024-09-14T06:32:02.548 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 156.806409331, 2024-09-14T06:32:02.548 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052595109000000001, 2024-09-14T06:32:02.548 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:32:02.548 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:32:02.548 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:32:02.548 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:32:02.548 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:32:02.548 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:02.548 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 11483 2024-09-14T06:32:02.548 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:02.548 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:32:02.548 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:02.549 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:32:02.549 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.658906+0000", 2024-09-14T06:32:02.549 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:02.549 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:02.549 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:02.549 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:32:02.549 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.658906+0000", 2024-09-14T06:32:02.549 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:02.549 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:02.549 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:02.549 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:32:02.549 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.664985+0000", 2024-09-14T06:32:02.549 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.0060787250000000001 2024-09-14T06:32:02.549 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:02.549 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:02.550 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:32:02.550 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.664991+0000", 2024-09-14T06:32:02.550 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.7030000000000003e-06 2024-09-14T06:32:02.550 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:02.550 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:02.550 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:32:02.550 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.664999+0000", 2024-09-14T06:32:02.550 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.4430000000000003e-06 2024-09-14T06:32:02.550 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:02.550 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:02.550 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:32:02.550 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.665006+0000", 2024-09-14T06:32:02.550 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.0840000000000003e-06 2024-09-14T06:32:02.551 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:02.551 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:02.551 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:32:02.551 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.711402+0000", 2024-09-14T06:32:02.551 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046395714999999997 2024-09-14T06:32:02.551 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:02.551 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:02.551 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:32:02.551 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.711443+0000", 2024-09-14T06:32:02.551 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.0911999999999997e-05 2024-09-14T06:32:02.551 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:02.551 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:02.551 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:32:02.551 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.711501+0000", 2024-09-14T06:32:02.552 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.8526999999999997e-05 2024-09-14T06:32:02.552 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:02.552 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:32:02.552 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:02.552 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:02.552 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:02.552 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:214569 3.2 3:479633bd:::rbd_object_map.112d3bd84cb0:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e170)", 2024-09-14T06:32:02.552 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:31:34.933901+0000", 2024-09-14T06:32:02.552 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 27.531414947000002, 2024-09-14T06:32:02.552 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.68157511400000004, 2024-09-14T06:32:02.552 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:32:02.552 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:32:02.552 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:32:02.552 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:32:02.552 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:32:02.553 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:02.553 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 214569 2024-09-14T06:32:02.553 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:02.553 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:32:02.553 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:02.553 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:32:02.553 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:31:34.933901+0000", 2024-09-14T06:32:02.553 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:02.553 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:02.553 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:02.553 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:32:02.553 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:31:34.933901+0000", 2024-09-14T06:32:02.553 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:02.554 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:02.554 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:02.554 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:32:02.554 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:31:34.933902+0000", 2024-09-14T06:32:02.554 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.7439999999999999e-06 2024-09-14T06:32:02.554 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:02.554 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:02.554 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:32:02.554 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:31:34.933910+0000", 2024-09-14T06:32:02.554 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.8569999999999995e-06 2024-09-14T06:32:02.554 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:02.554 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:02.554 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:32:02.554 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:31:34.933932+0000", 2024-09-14T06:32:02.554 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.1997999999999998e-05 2024-09-14T06:32:02.555 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:02.555 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:02.555 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:32:02.555 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:31:34.933941+0000", 2024-09-14T06:32:02.555 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.7180000000000002e-06 2024-09-14T06:32:02.555 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:02.555 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:02.555 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:32:02.555 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:31:34.934013+0000", 2024-09-14T06:32:02.555 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.1761000000000006e-05 2024-09-14T06:32:02.555 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:02.555 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:02.555 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "waiting for peered", 2024-09-14T06:32:02.555 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:31:34.934022+0000", 2024-09-14T06:32:02.556 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.8740000000000001e-06 2024-09-14T06:32:02.556 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:02.556 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:02.556 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:32:02.556 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:31:35.608803+0000", 2024-09-14T06:32:02.556 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.67478164299999999 2024-09-14T06:32:02.556 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:02.556 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:02.556 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:32:02.556 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:31:35.608964+0000", 2024-09-14T06:32:02.556 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00016112900000000001 2024-09-14T06:32:02.556 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:02.556 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:02.557 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "waiting for subops from 6", 2024-09-14T06:32:02.557 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:31:35.609399+0000", 2024-09-14T06:32:02.557 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00043454000000000002 2024-09-14T06:32:02.557 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:02.557 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:02.557 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "op_commit", 2024-09-14T06:32:02.557 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:31:35.610495+0000", 2024-09-14T06:32:02.557 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.001096404 2024-09-14T06:32:02.557 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:02.557 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:02.557 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:32:02.557 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:31:35.615409+0000", 2024-09-14T06:32:02.557 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.0049133199999999997 2024-09-14T06:32:02.557 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:02.558 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:02.558 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "commit_sent", 2024-09-14T06:32:02.558 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:31:35.615444+0000", 2024-09-14T06:32:02.558 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.5457000000000003e-05 2024-09-14T06:32:02.558 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:02.558 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:02.558 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:32:02.558 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:31:35.615476+0000", 2024-09-14T06:32:02.558 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.1668999999999998e-05 2024-09-14T06:32:02.558 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:02.558 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:32:02.558 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:02.558 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:02.558 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:32:02.558 INFO:teuthology.orchestra.run.smithi138.stdout:} 2024-09-14T06:32:02.560 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-14T06:32:02.560 INFO:tasks.ceph.osd.1.smithi094.stderr:2024-09-14T06:32:02.559+0000 7f328e47a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33641) UID: 0 2024-09-14T06:32:02.623 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-14T06:32:02.668 INFO:tasks.ceph.osd.0.smithi094.stderr:2024-09-14T06:32:02.666+0000 7f07c9333640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33636) UID: 0 2024-09-14T06:32:02.740 INFO:teuthology.orchestra.run.smithi138.stdout:{ 2024-09-14T06:32:02.741 INFO:teuthology.orchestra.run.smithi138.stdout: "ops": [], 2024-09-14T06:32:02.741 INFO:teuthology.orchestra.run.smithi138.stdout: "num_ops": 0 2024-09-14T06:32:02.741 INFO:teuthology.orchestra.run.smithi138.stdout:} 2024-09-14T06:32:02.751 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-14T06:32:02.761 INFO:tasks.ceph.osd.3.smithi094.stderr:2024-09-14T06:32:02.760+0000 7feeeafe8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33655) UID: 0 2024-09-14T06:32:02.862 INFO:tasks.ceph.osd.3.smithi094.stderr:2024-09-14T06:32:02.861+0000 7feeeafe8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33655) UID: 0 2024-09-14T06:32:02.870 INFO:teuthology.orchestra.run.smithi138.stdout:{ 2024-09-14T06:32:02.870 INFO:teuthology.orchestra.run.smithi138.stdout: "ops": [], 2024-09-14T06:32:02.870 INFO:teuthology.orchestra.run.smithi138.stdout: "complaint_time": 30, 2024-09-14T06:32:02.871 INFO:teuthology.orchestra.run.smithi138.stdout: "num_blocked_ops": 0 2024-09-14T06:32:02.871 INFO:teuthology.orchestra.run.smithi138.stdout:} 2024-09-14T06:32:02.880 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-14T06:32:02.963 INFO:tasks.ceph.osd.7.smithi138.stderr:2024-09-14T06:32:02.962+0000 7f3c6a21d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33623) UID: 0 2024-09-14T06:32:03.012 INFO:teuthology.orchestra.run.smithi138.stdout:{ 2024-09-14T06:32:03.012 INFO:teuthology.orchestra.run.smithi138.stdout: "size": 20, 2024-09-14T06:32:03.012 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 600, 2024-09-14T06:32:03.012 INFO:teuthology.orchestra.run.smithi138.stdout: "ops": [ 2024-09-14T06:32:03.012 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:03.012 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:98505 3.10 3.1bf56f90 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T06:32:03.012 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:30:07.045453+0000", 2024-09-14T06:32:03.012 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 115.964482577, 2024-09-14T06:32:03.012 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032144440000000003, 2024-09-14T06:32:03.013 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:32:03.013 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:32:03.013 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "reached pg", 2024-09-14T06:32:03.013 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:32:03.013 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:32:03.013 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:03.013 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 98505 2024-09-14T06:32:03.014 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:03.014 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:32:03.014 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:03.014 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:32:03.014 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045453+0000", 2024-09-14T06:32:03.014 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:03.014 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:03.014 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:03.014 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:32:03.015 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045453+0000", 2024-09-14T06:32:03.015 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:03.015 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:03.015 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:03.015 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:32:03.015 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045453+0000", 2024-09-14T06:32:03.015 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.36e-07 2024-09-14T06:32:03.015 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:03.015 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:03.015 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:32:03.015 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045454+0000", 2024-09-14T06:32:03.015 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.1299999999999998e-07 2024-09-14T06:32:03.016 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:03.016 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:03.016 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:32:03.016 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045457+0000", 2024-09-14T06:32:03.016 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.8559999999999998e-06 2024-09-14T06:32:03.017 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:03.017 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:03.017 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:32:03.017 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045460+0000", 2024-09-14T06:32:03.017 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.8750000000000002e-06 2024-09-14T06:32:03.018 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:03.018 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:03.018 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:32:03.018 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.077592+0000", 2024-09-14T06:32:03.018 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032131727999999998 2024-09-14T06:32:03.018 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:03.019 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:03.019 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:32:03.019 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.077597+0000", 2024-09-14T06:32:03.019 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.8319999999999996e-06 2024-09-14T06:32:03.019 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:03.019 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:32:03.019 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:03.019 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:03.019 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:03.019 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:98506 3.10 3.1bf56f90 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T06:32:03.019 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:30:07.045481+0000", 2024-09-14T06:32:03.020 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 115.96445432199999, 2024-09-14T06:32:03.020 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032142759999999999, 2024-09-14T06:32:03.020 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:32:03.020 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:32:03.020 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "reached pg", 2024-09-14T06:32:03.020 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:32:03.021 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:32:03.021 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:03.021 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 98506 2024-09-14T06:32:03.021 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:03.021 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:32:03.021 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:03.021 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:32:03.021 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045481+0000", 2024-09-14T06:32:03.021 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:03.021 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:03.021 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:03.021 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:32:03.022 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045481+0000", 2024-09-14T06:32:03.022 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:03.022 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:03.022 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:03.022 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:32:03.022 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045481+0000", 2024-09-14T06:32:03.022 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.8500000000000001e-07 2024-09-14T06:32:03.022 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:03.022 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:03.022 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:32:03.022 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045483+0000", 2024-09-14T06:32:03.022 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.8250000000000001e-06 2024-09-14T06:32:03.022 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:03.022 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:03.022 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:32:03.023 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045489+0000", 2024-09-14T06:32:03.023 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.5740000000000003e-06 2024-09-14T06:32:03.023 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:03.023 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:03.023 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:32:03.023 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045495+0000", 2024-09-14T06:32:03.023 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.7690000000000002e-06 2024-09-14T06:32:03.023 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:03.023 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:03.023 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:32:03.023 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.077619+0000", 2024-09-14T06:32:03.023 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032124621999999999 2024-09-14T06:32:03.024 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:03.024 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:03.024 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:32:03.024 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.077624+0000", 2024-09-14T06:32:03.024 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.3850000000000004e-06 2024-09-14T06:32:03.024 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:03.024 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:32:03.024 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:03.024 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:03.024 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:03.024 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:98507 3.10 3.1bf56f90 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T06:32:03.024 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:30:07.045502+0000", 2024-09-14T06:32:03.025 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 115.964432901, 2024-09-14T06:32:03.025 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032147219999999997, 2024-09-14T06:32:03.025 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:32:03.025 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:32:03.025 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "reached pg", 2024-09-14T06:32:03.025 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:32:03.025 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:32:03.025 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:03.025 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 98507 2024-09-14T06:32:03.025 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:03.025 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:32:03.025 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:03.025 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:32:03.026 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045502+0000", 2024-09-14T06:32:03.026 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:03.026 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:03.026 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:03.026 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:32:03.026 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045502+0000", 2024-09-14T06:32:03.026 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:03.026 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:03.026 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:03.026 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:32:03.026 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045503+0000", 2024-09-14T06:32:03.027 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.4700000000000002e-07 2024-09-14T06:32:03.027 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:03.027 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:03.027 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:32:03.027 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045509+0000", 2024-09-14T06:32:03.027 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.7329999999999997e-06 2024-09-14T06:32:03.027 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:03.027 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:03.027 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:32:03.027 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045515+0000", 2024-09-14T06:32:03.027 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.8030000000000002e-06 2024-09-14T06:32:03.027 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:03.027 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:03.027 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:32:03.028 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045522+0000", 2024-09-14T06:32:03.028 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.0369999999999997e-06 2024-09-14T06:32:03.028 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:03.028 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:03.028 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:32:03.028 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.077641+0000", 2024-09-14T06:32:03.028 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032119091000000002 2024-09-14T06:32:03.028 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:03.028 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:03.028 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:32:03.028 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.077650+0000", 2024-09-14T06:32:03.028 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.2090000000000005e-06 2024-09-14T06:32:03.029 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:03.029 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:32:03.029 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:03.029 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:03.029 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:03.029 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:98508 3.10 3.1bf56f90 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T06:32:03.029 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:30:07.045530+0000", 2024-09-14T06:32:03.029 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 115.96440528399999, 2024-09-14T06:32:03.029 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032198773999999999, 2024-09-14T06:32:03.029 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:32:03.029 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:32:03.029 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "reached pg", 2024-09-14T06:32:03.029 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:32:03.030 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:32:03.030 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:03.030 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 98508 2024-09-14T06:32:03.030 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:03.030 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:32:03.030 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:03.030 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:32:03.030 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045530+0000", 2024-09-14T06:32:03.030 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:03.030 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:03.030 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:03.030 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:32:03.030 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045530+0000", 2024-09-14T06:32:03.030 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:03.031 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:03.031 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:03.031 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:32:03.031 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045530+0000", 2024-09-14T06:32:03.031 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.3099999999999998e-07 2024-09-14T06:32:03.031 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:03.031 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:03.031 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:32:03.031 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045539+0000", 2024-09-14T06:32:03.031 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.4780000000000008e-06 2024-09-14T06:32:03.031 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:03.031 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:03.032 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:32:03.032 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045546+0000", 2024-09-14T06:32:03.032 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.3259999999999998e-06 2024-09-14T06:32:03.032 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:03.032 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:03.032 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:32:03.032 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045551+0000", 2024-09-14T06:32:03.032 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.9960000000000001e-06 2024-09-14T06:32:03.032 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:03.032 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:03.032 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:32:03.032 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.077716+0000", 2024-09-14T06:32:03.032 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032165081999999998 2024-09-14T06:32:03.033 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:03.033 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:03.033 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:32:03.033 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.077729+0000", 2024-09-14T06:32:03.033 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.2461e-05 2024-09-14T06:32:03.033 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:03.033 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:32:03.033 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:03.033 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:03.033 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:03.034 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:98509 3.10 3.1bf56f90 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T06:32:03.034 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:30:07.045558+0000", 2024-09-14T06:32:03.034 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 115.964377083, 2024-09-14T06:32:03.034 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032242198, 2024-09-14T06:32:03.034 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:32:03.034 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:32:03.034 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "reached pg", 2024-09-14T06:32:03.034 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:32:03.034 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:32:03.034 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:03.034 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 98509 2024-09-14T06:32:03.034 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:03.034 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:32:03.034 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:03.035 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:32:03.035 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045558+0000", 2024-09-14T06:32:03.035 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:03.035 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:03.035 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:03.035 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:32:03.035 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045558+0000", 2024-09-14T06:32:03.035 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:03.035 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:03.035 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:03.035 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:32:03.035 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045559+0000", 2024-09-14T06:32:03.035 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.27e-07 2024-09-14T06:32:03.036 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:03.036 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:03.036 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:32:03.036 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045560+0000", 2024-09-14T06:32:03.036 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.6959999999999999e-06 2024-09-14T06:32:03.036 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:03.036 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:03.036 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:32:03.036 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045567+0000", 2024-09-14T06:32:03.036 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.6900000000000003e-06 2024-09-14T06:32:03.036 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:03.036 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:03.036 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:32:03.037 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045571+0000", 2024-09-14T06:32:03.037 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.5750000000000002e-06 2024-09-14T06:32:03.037 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:03.037 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:03.037 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:32:03.037 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.077789+0000", 2024-09-14T06:32:03.037 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032217186000000002 2024-09-14T06:32:03.037 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:03.037 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:03.037 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:32:03.037 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.077800+0000", 2024-09-14T06:32:03.037 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.1623999999999999e-05 2024-09-14T06:32:03.038 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:03.038 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:32:03.038 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:03.038 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:03.038 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:03.038 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:98510 3.10 3.1bf56f90 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T06:32:03.038 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:30:07.045580+0000", 2024-09-14T06:32:03.038 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 115.964355269, 2024-09-14T06:32:03.038 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032299845000000001, 2024-09-14T06:32:03.038 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:32:03.038 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:32:03.038 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "reached pg", 2024-09-14T06:32:03.038 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:32:03.038 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:32:03.039 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:03.039 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 98510 2024-09-14T06:32:03.039 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:03.039 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:32:03.039 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:03.039 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:32:03.039 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045580+0000", 2024-09-14T06:32:03.039 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:03.039 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:03.039 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:03.039 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:32:03.039 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045580+0000", 2024-09-14T06:32:03.039 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:03.039 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:03.040 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:03.040 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:32:03.040 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045580+0000", 2024-09-14T06:32:03.040 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.0400000000000002e-07 2024-09-14T06:32:03.040 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:03.040 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:03.040 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:32:03.040 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045582+0000", 2024-09-14T06:32:03.040 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.7600000000000001e-06 2024-09-14T06:32:03.040 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:03.040 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:03.040 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:32:03.040 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045588+0000", 2024-09-14T06:32:03.040 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.5650000000000001e-06 2024-09-14T06:32:03.041 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:03.041 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:03.041 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:32:03.041 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045597+0000", 2024-09-14T06:32:03.041 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.465e-06 2024-09-14T06:32:03.041 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:03.041 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:03.041 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:32:03.041 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.077871+0000", 2024-09-14T06:32:03.041 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032273562999999998 2024-09-14T06:32:03.041 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:03.041 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:03.041 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:32:03.041 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.077880+0000", 2024-09-14T06:32:03.042 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.0880000000000001e-06 2024-09-14T06:32:03.042 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:03.042 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:32:03.042 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:03.042 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:03.042 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:03.042 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:98511 3.10 3.1bf56f90 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T06:32:03.042 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:30:07.045610+0000", 2024-09-14T06:32:03.042 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 115.964325422, 2024-09-14T06:32:03.042 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032338894, 2024-09-14T06:32:03.042 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:32:03.042 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:32:03.042 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "reached pg", 2024-09-14T06:32:03.042 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:32:03.043 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:32:03.043 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:03.043 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 98511 2024-09-14T06:32:03.043 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:03.043 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:32:03.043 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:03.043 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:32:03.043 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045610+0000", 2024-09-14T06:32:03.043 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:03.043 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:03.043 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:03.043 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:32:03.043 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045610+0000", 2024-09-14T06:32:03.043 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:03.044 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:03.044 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:03.044 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:32:03.044 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045610+0000", 2024-09-14T06:32:03.044 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.1600000000000002e-07 2024-09-14T06:32:03.044 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:03.044 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:03.044 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:32:03.044 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045612+0000", 2024-09-14T06:32:03.044 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.1150000000000001e-06 2024-09-14T06:32:03.044 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:03.044 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:03.044 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:32:03.044 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045618+0000", 2024-09-14T06:32:03.045 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.0499999999999997e-06 2024-09-14T06:32:03.045 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:03.045 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:03.045 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:32:03.045 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045623+0000", 2024-09-14T06:32:03.045 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.6770000000000003e-06 2024-09-14T06:32:03.045 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:03.045 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:03.045 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:32:03.045 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.077928+0000", 2024-09-14T06:32:03.045 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032305111999999997 2024-09-14T06:32:03.045 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:03.045 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:03.046 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:32:03.046 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.077949+0000", 2024-09-14T06:32:03.046 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.0523999999999999e-05 2024-09-14T06:32:03.046 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:03.046 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:32:03.046 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:03.046 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:03.046 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:03.046 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:98512 3.10 3.1bf56f90 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T06:32:03.046 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:30:07.045630+0000", 2024-09-14T06:32:03.046 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 115.96430541700001, 2024-09-14T06:32:03.046 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032346632, 2024-09-14T06:32:03.046 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:32:03.046 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:32:03.047 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "reached pg", 2024-09-14T06:32:03.047 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:32:03.047 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:32:03.047 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:03.047 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 98512 2024-09-14T06:32:03.047 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:03.047 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:32:03.047 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:03.047 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:32:03.047 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045630+0000", 2024-09-14T06:32:03.047 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:03.047 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:03.047 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:03.047 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:32:03.048 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045630+0000", 2024-09-14T06:32:03.048 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:03.048 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:03.048 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:03.048 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:32:03.048 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045630+0000", 2024-09-14T06:32:03.048 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.8200000000000001e-07 2024-09-14T06:32:03.048 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:03.048 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:03.048 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:32:03.048 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045632+0000", 2024-09-14T06:32:03.048 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.7430000000000001e-06 2024-09-14T06:32:03.048 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:03.049 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:03.049 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:32:03.049 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045637+0000", 2024-09-14T06:32:03.049 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.4550000000000003e-06 2024-09-14T06:32:03.049 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:03.049 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:03.049 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:32:03.049 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045644+0000", 2024-09-14T06:32:03.049 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.1330000000000001e-06 2024-09-14T06:32:03.049 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:03.049 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:03.049 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:32:03.049 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.077968+0000", 2024-09-14T06:32:03.050 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032323401000000002 2024-09-14T06:32:03.050 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:03.050 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:03.050 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:32:03.050 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.077976+0000", 2024-09-14T06:32:03.050 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.5180000000000004e-06 2024-09-14T06:32:03.050 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:03.050 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:32:03.050 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:03.050 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:03.050 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:03.050 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:98513 3.10 3.1bf56f90 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T06:32:03.050 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:30:07.045652+0000", 2024-09-14T06:32:03.050 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 115.964282853, 2024-09-14T06:32:03.050 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032353436999999999, 2024-09-14T06:32:03.051 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:32:03.051 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:32:03.051 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "reached pg", 2024-09-14T06:32:03.051 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:32:03.051 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:32:03.051 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:03.051 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 98513 2024-09-14T06:32:03.051 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:03.051 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:32:03.051 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:03.051 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:32:03.051 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045652+0000", 2024-09-14T06:32:03.051 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:03.052 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:03.052 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:03.052 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:32:03.052 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045652+0000", 2024-09-14T06:32:03.052 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:03.052 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:03.052 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:03.052 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:32:03.052 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045653+0000", 2024-09-14T06:32:03.052 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.3700000000000001e-07 2024-09-14T06:32:03.052 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:03.052 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:03.052 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:32:03.053 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045655+0000", 2024-09-14T06:32:03.053 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.854e-06 2024-09-14T06:32:03.053 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:03.053 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:03.053 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:32:03.053 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045661+0000", 2024-09-14T06:32:03.054 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.5509999999999996e-06 2024-09-14T06:32:03.054 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:03.054 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:03.054 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:32:03.054 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045668+0000", 2024-09-14T06:32:03.054 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.1980000000000004e-06 2024-09-14T06:32:03.054 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:03.054 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:03.054 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:32:03.054 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.077997+0000", 2024-09-14T06:32:03.054 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032328575999999998 2024-09-14T06:32:03.054 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:03.054 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:03.054 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:32:03.055 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.078006+0000", 2024-09-14T06:32:03.055 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.8210000000000008e-06 2024-09-14T06:32:03.055 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:03.055 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:32:03.055 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:03.055 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:03.055 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:03.055 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:98514 3.10 3.1bf56f90 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T06:32:03.055 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:30:07.045699+0000", 2024-09-14T06:32:03.055 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 115.96423651800001, 2024-09-14T06:32:03.055 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032330223999999998, 2024-09-14T06:32:03.055 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:32:03.055 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:32:03.056 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "reached pg", 2024-09-14T06:32:03.056 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:32:03.056 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:32:03.056 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:03.056 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 98514 2024-09-14T06:32:03.056 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:03.056 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:32:03.056 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:03.056 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:32:03.056 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045699+0000", 2024-09-14T06:32:03.056 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:03.056 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:03.057 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:03.057 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:32:03.057 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045699+0000", 2024-09-14T06:32:03.057 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:03.057 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:03.057 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:03.057 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:32:03.057 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045699+0000", 2024-09-14T06:32:03.057 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.3200000000000001e-07 2024-09-14T06:32:03.057 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:03.057 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:03.058 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:32:03.058 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045701+0000", 2024-09-14T06:32:03.058 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.762e-06 2024-09-14T06:32:03.058 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:03.058 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:03.058 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:32:03.058 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045705+0000", 2024-09-14T06:32:03.058 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.2359999999999998e-06 2024-09-14T06:32:03.058 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:03.058 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:03.058 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:32:03.058 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045708+0000", 2024-09-14T06:32:03.058 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.796e-06 2024-09-14T06:32:03.059 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:03.059 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:03.059 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:32:03.059 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.078023+0000", 2024-09-14T06:32:03.059 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032315213000000002 2024-09-14T06:32:03.059 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:03.059 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:03.059 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:32:03.059 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.078029+0000", 2024-09-14T06:32:03.059 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.8850000000000004e-06 2024-09-14T06:32:03.059 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:03.059 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:32:03.059 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:03.059 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:03.059 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:03.060 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:98515 3.10 3.1bf56f90 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T06:32:03.060 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:30:07.045712+0000", 2024-09-14T06:32:03.060 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 115.964223207, 2024-09-14T06:32:03.060 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032346871999999999, 2024-09-14T06:32:03.060 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:32:03.060 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:32:03.060 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "reached pg", 2024-09-14T06:32:03.060 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:32:03.060 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:32:03.060 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:03.060 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 98515 2024-09-14T06:32:03.060 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:03.060 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:32:03.061 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:03.061 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:32:03.061 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045712+0000", 2024-09-14T06:32:03.061 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:03.061 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:03.061 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:03.061 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:32:03.061 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045712+0000", 2024-09-14T06:32:03.061 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:03.061 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:03.061 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:03.061 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:32:03.061 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045712+0000", 2024-09-14T06:32:03.062 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.2100000000000001e-07 2024-09-14T06:32:03.062 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:03.062 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:03.062 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:32:03.062 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045713+0000", 2024-09-14T06:32:03.062 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.068e-06 2024-09-14T06:32:03.062 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:03.062 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:03.062 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:32:03.062 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045716+0000", 2024-09-14T06:32:03.062 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.971e-06 2024-09-14T06:32:03.062 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:03.062 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:03.063 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:32:03.063 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045719+0000", 2024-09-14T06:32:03.063 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.3879999999999998e-06 2024-09-14T06:32:03.063 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:03.063 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:03.063 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:32:03.063 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.078049+0000", 2024-09-14T06:32:03.063 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032330553999999997 2024-09-14T06:32:03.063 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:03.063 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:03.063 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:32:03.063 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.078059+0000", 2024-09-14T06:32:03.063 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.6700000000000006e-06 2024-09-14T06:32:03.063 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:03.063 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:32:03.064 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:03.064 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:03.064 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:03.064 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:98516 3.10 3.1bf56f90 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T06:32:03.064 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:30:07.045722+0000", 2024-09-14T06:32:03.064 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 115.964213454, 2024-09-14T06:32:03.064 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032374334999999997, 2024-09-14T06:32:03.064 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:32:03.064 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:32:03.064 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "reached pg", 2024-09-14T06:32:03.064 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:32:03.064 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:32:03.064 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:03.064 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 98516 2024-09-14T06:32:03.065 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:03.065 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:32:03.065 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:03.065 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:32:03.065 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045722+0000", 2024-09-14T06:32:03.065 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:03.065 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:03.065 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:03.065 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:32:03.065 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045722+0000", 2024-09-14T06:32:03.065 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:03.065 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:03.065 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:03.065 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:32:03.066 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045722+0000", 2024-09-14T06:32:03.066 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.8999999999999998e-07 2024-09-14T06:32:03.066 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:03.066 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:03.066 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:32:03.066 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045723+0000", 2024-09-14T06:32:03.066 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.0049999999999999e-06 2024-09-14T06:32:03.066 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:03.066 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:03.066 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:32:03.066 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045726+0000", 2024-09-14T06:32:03.066 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.9019999999999999e-06 2024-09-14T06:32:03.066 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:03.067 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:03.067 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:32:03.067 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045728+0000", 2024-09-14T06:32:03.067 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.3290000000000001e-06 2024-09-14T06:32:03.067 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:03.067 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:03.067 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:32:03.067 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.078090+0000", 2024-09-14T06:32:03.067 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032362123 2024-09-14T06:32:03.067 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:03.067 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:03.067 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:32:03.068 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.078096+0000", 2024-09-14T06:32:03.068 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.6860000000000003e-06 2024-09-14T06:32:03.068 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:03.068 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:32:03.068 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:03.068 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:03.068 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:03.068 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:98517 3.10 3.1bf56f90 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T06:32:03.068 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:30:07.045731+0000", 2024-09-14T06:32:03.068 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 115.964203723, 2024-09-14T06:32:03.068 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.03241227, 2024-09-14T06:32:03.068 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:32:03.068 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:32:03.069 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "reached pg", 2024-09-14T06:32:03.069 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:32:03.069 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:32:03.069 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:03.069 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 98517 2024-09-14T06:32:03.069 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:03.069 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:32:03.069 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:03.069 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:32:03.069 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045731+0000", 2024-09-14T06:32:03.069 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:03.069 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:03.069 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:03.070 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:32:03.070 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045731+0000", 2024-09-14T06:32:03.070 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:03.070 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:03.070 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:03.070 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:32:03.070 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045732+0000", 2024-09-14T06:32:03.070 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.0699999999999999e-07 2024-09-14T06:32:03.070 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:03.070 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:03.070 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:32:03.070 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045733+0000", 2024-09-14T06:32:03.070 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.9199999999999999e-07 2024-09-14T06:32:03.070 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:03.070 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:03.071 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:32:03.071 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045735+0000", 2024-09-14T06:32:03.071 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.8150000000000002e-06 2024-09-14T06:32:03.071 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:03.071 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:03.071 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:32:03.071 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045738+0000", 2024-09-14T06:32:03.071 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.3640000000000002e-06 2024-09-14T06:32:03.071 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:03.071 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:03.071 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:32:03.071 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.078126+0000", 2024-09-14T06:32:03.071 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032388206000000003 2024-09-14T06:32:03.071 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:03.072 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:03.072 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:32:03.072 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.078144+0000", 2024-09-14T06:32:03.072 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.7686e-05 2024-09-14T06:32:03.072 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:03.072 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:32:03.072 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:03.072 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:03.072 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:03.072 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:98518 3.10 3.1bf56f90 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T06:32:03.072 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:30:07.045741+0000", 2024-09-14T06:32:03.073 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 115.964194328, 2024-09-14T06:32:03.073 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032451793999999999, 2024-09-14T06:32:03.073 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:32:03.073 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:32:03.073 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "reached pg", 2024-09-14T06:32:03.073 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:32:03.073 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:32:03.073 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:03.073 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 98518 2024-09-14T06:32:03.073 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:03.073 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:32:03.073 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:03.074 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:32:03.074 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045741+0000", 2024-09-14T06:32:03.074 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:03.074 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:03.074 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:03.074 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:32:03.074 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045741+0000", 2024-09-14T06:32:03.074 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:03.074 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:03.074 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:03.074 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:32:03.074 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045741+0000", 2024-09-14T06:32:03.074 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.05e-07 2024-09-14T06:32:03.074 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:03.075 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:03.075 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:32:03.075 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045746+0000", 2024-09-14T06:32:03.075 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.2259999999999996e-06 2024-09-14T06:32:03.075 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:03.075 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:03.075 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:32:03.075 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045750+0000", 2024-09-14T06:32:03.075 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.2490000000000001e-06 2024-09-14T06:32:03.075 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:03.075 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:03.075 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:32:03.075 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045752+0000", 2024-09-14T06:32:03.076 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.3980000000000002e-06 2024-09-14T06:32:03.076 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:03.076 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:03.076 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:32:03.076 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.078184+0000", 2024-09-14T06:32:03.076 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032431593000000002 2024-09-14T06:32:03.076 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:03.076 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:03.076 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:32:03.076 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.078193+0000", 2024-09-14T06:32:03.076 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.1230000000000006e-06 2024-09-14T06:32:03.076 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:03.076 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:32:03.077 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:03.077 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:03.077 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:03.077 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:98519 3.10 3.1bf56f90 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T06:32:03.077 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:30:07.045755+0000", 2024-09-14T06:32:03.077 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 115.964179903, 2024-09-14T06:32:03.077 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032489431999999999, 2024-09-14T06:32:03.077 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:32:03.077 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:32:03.077 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "reached pg", 2024-09-14T06:32:03.077 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:32:03.077 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:32:03.077 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:03.078 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 98519 2024-09-14T06:32:03.078 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:03.078 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:32:03.078 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:03.078 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:32:03.078 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045755+0000", 2024-09-14T06:32:03.078 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:03.078 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:03.078 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:03.078 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:32:03.078 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045755+0000", 2024-09-14T06:32:03.078 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:03.078 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:03.078 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:03.079 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:32:03.079 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045756+0000", 2024-09-14T06:32:03.079 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.23e-07 2024-09-14T06:32:03.079 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:03.079 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:03.079 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:32:03.079 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045757+0000", 2024-09-14T06:32:03.079 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.1060000000000001e-06 2024-09-14T06:32:03.079 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:03.079 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:03.079 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:32:03.079 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045760+0000", 2024-09-14T06:32:03.079 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.988e-06 2024-09-14T06:32:03.079 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:03.080 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:03.080 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:32:03.080 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045765+0000", 2024-09-14T06:32:03.080 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.2689999999999999e-06 2024-09-14T06:32:03.080 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:03.080 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:03.080 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:32:03.080 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.078234+0000", 2024-09-14T06:32:03.080 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032468915000000001 2024-09-14T06:32:03.080 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:03.080 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:03.080 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:32:03.081 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.078245+0000", 2024-09-14T06:32:03.081 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.0930999999999999e-05 2024-09-14T06:32:03.081 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:03.081 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:32:03.081 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:03.081 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:03.081 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:03.081 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:98520 3.10 3.1bf56f90 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T06:32:03.081 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:30:07.045768+0000", 2024-09-14T06:32:03.081 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 115.96416702499999, 2024-09-14T06:32:03.081 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032531379999999999, 2024-09-14T06:32:03.081 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:32:03.081 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:32:03.082 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "reached pg", 2024-09-14T06:32:03.082 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:32:03.082 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:32:03.082 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:03.082 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 98520 2024-09-14T06:32:03.082 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:03.082 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:32:03.082 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:03.082 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:32:03.082 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045768+0000", 2024-09-14T06:32:03.082 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:03.082 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:03.082 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:03.082 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:32:03.083 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045768+0000", 2024-09-14T06:32:03.083 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:03.083 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:03.083 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:03.083 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:32:03.083 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045768+0000", 2024-09-14T06:32:03.083 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.22e-07 2024-09-14T06:32:03.083 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:03.083 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:03.083 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:32:03.083 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045773+0000", 2024-09-14T06:32:03.083 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.3470000000000001e-06 2024-09-14T06:32:03.083 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:03.083 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:03.084 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:32:03.084 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045776+0000", 2024-09-14T06:32:03.084 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.1319999999999998e-06 2024-09-14T06:32:03.084 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:03.084 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:03.084 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:32:03.084 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045779+0000", 2024-09-14T06:32:03.084 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.9749999999999999e-06 2024-09-14T06:32:03.084 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:03.084 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:03.084 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:32:03.084 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.078289+0000", 2024-09-14T06:32:03.084 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032509693999999999 2024-09-14T06:32:03.084 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:03.085 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:03.085 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:32:03.085 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.078300+0000", 2024-09-14T06:32:03.085 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.101e-05 2024-09-14T06:32:03.085 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:03.085 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:32:03.085 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:03.085 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:03.085 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:03.085 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:98521 3.10 3.1bf56f90 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T06:32:03.085 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:30:07.045782+0000", 2024-09-14T06:32:03.085 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 115.964152845, 2024-09-14T06:32:03.085 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032566109000000003, 2024-09-14T06:32:03.086 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:32:03.086 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:32:03.086 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "reached pg", 2024-09-14T06:32:03.086 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:32:03.086 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:32:03.086 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:03.086 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 98521 2024-09-14T06:32:03.086 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:03.086 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:32:03.086 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:03.086 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:32:03.086 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045782+0000", 2024-09-14T06:32:03.086 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:03.086 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:03.087 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:03.087 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:32:03.087 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045782+0000", 2024-09-14T06:32:03.087 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:03.087 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:03.087 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:03.087 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:32:03.087 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045783+0000", 2024-09-14T06:32:03.087 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.2499999999999999e-07 2024-09-14T06:32:03.087 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:03.087 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:03.087 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:32:03.087 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045784+0000", 2024-09-14T06:32:03.087 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.0109999999999999e-06 2024-09-14T06:32:03.087 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:03.088 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:03.088 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:32:03.088 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045786+0000", 2024-09-14T06:32:03.088 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.8289999999999999e-06 2024-09-14T06:32:03.088 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:03.088 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:03.088 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:32:03.088 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045793+0000", 2024-09-14T06:32:03.088 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.579e-06 2024-09-14T06:32:03.088 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:03.088 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:03.088 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:32:03.088 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.078338+0000", 2024-09-14T06:32:03.089 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032545389000000001 2024-09-14T06:32:03.089 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:03.089 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:03.089 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:32:03.089 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.078348+0000", 2024-09-14T06:32:03.089 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.0076e-05 2024-09-14T06:32:03.089 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:03.089 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:32:03.089 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:03.089 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:03.089 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:03.089 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:98522 3.10 3.1bf56f90 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T06:32:03.089 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:30:07.045796+0000", 2024-09-14T06:32:03.089 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 115.964138937, 2024-09-14T06:32:03.090 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032577846000000001, 2024-09-14T06:32:03.090 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:32:03.090 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:32:03.090 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "reached pg", 2024-09-14T06:32:03.090 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:32:03.090 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:32:03.090 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:03.090 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 98522 2024-09-14T06:32:03.090 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:03.090 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:32:03.090 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:03.090 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:32:03.090 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045796+0000", 2024-09-14T06:32:03.090 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:03.091 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:03.091 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:03.091 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:32:03.091 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045796+0000", 2024-09-14T06:32:03.091 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:03.091 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:03.091 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:03.091 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:32:03.091 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045796+0000", 2024-09-14T06:32:03.091 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.2100000000000001e-07 2024-09-14T06:32:03.091 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:03.091 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:03.091 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:32:03.092 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045797+0000", 2024-09-14T06:32:03.092 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.6899999999999996e-07 2024-09-14T06:32:03.092 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:03.092 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:03.092 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:32:03.092 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045800+0000", 2024-09-14T06:32:03.092 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.9979999999999999e-06 2024-09-14T06:32:03.092 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:03.092 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:03.092 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:32:03.092 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045803+0000", 2024-09-14T06:32:03.092 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.2850000000000001e-06 2024-09-14T06:32:03.092 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:03.093 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:03.093 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:32:03.093 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.078366+0000", 2024-09-14T06:32:03.093 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032562932000000003 2024-09-14T06:32:03.093 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:03.093 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:03.093 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:32:03.093 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.078374+0000", 2024-09-14T06:32:03.093 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.4409999999999993e-06 2024-09-14T06:32:03.093 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:03.093 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:32:03.093 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:03.093 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:03.093 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:03.094 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:98523 3.10 3.1bf56f90 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T06:32:03.094 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:30:07.045806+0000", 2024-09-14T06:32:03.094 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 115.964129394, 2024-09-14T06:32:03.094 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032611167000000003, 2024-09-14T06:32:03.094 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:32:03.094 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:32:03.094 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "reached pg", 2024-09-14T06:32:03.094 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:32:03.094 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:32:03.094 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:03.094 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 98523 2024-09-14T06:32:03.094 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:03.094 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:32:03.094 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:03.095 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:32:03.095 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045806+0000", 2024-09-14T06:32:03.095 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:03.095 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:03.095 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:03.095 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:32:03.095 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045806+0000", 2024-09-14T06:32:03.095 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:03.095 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:03.095 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:03.095 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:32:03.095 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045806+0000", 2024-09-14T06:32:03.095 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.2000000000000001e-07 2024-09-14T06:32:03.095 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:03.096 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:03.096 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:32:03.096 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045807+0000", 2024-09-14T06:32:03.096 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.4e-07 2024-09-14T06:32:03.096 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:03.096 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:03.096 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:32:03.096 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045810+0000", 2024-09-14T06:32:03.096 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.9229999999999998e-06 2024-09-14T06:32:03.096 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:03.096 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:03.096 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:32:03.096 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045812+0000", 2024-09-14T06:32:03.097 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.2220000000000001e-06 2024-09-14T06:32:03.097 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:03.097 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:03.097 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:32:03.097 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.078407+0000", 2024-09-14T06:32:03.097 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032594513999999998 2024-09-14T06:32:03.097 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:03.097 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:03.097 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:32:03.097 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.078417+0000", 2024-09-14T06:32:03.097 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.0348e-05 2024-09-14T06:32:03.097 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:03.097 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:32:03.098 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:03.098 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:03.098 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:03.098 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:98524 3.10 3.1bf56f90 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T06:32:03.098 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:30:07.045815+0000", 2024-09-14T06:32:03.098 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 115.964120026, 2024-09-14T06:32:03.098 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032643208999999999, 2024-09-14T06:32:03.098 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:32:03.098 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:32:03.098 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "reached pg", 2024-09-14T06:32:03.098 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:32:03.098 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:32:03.098 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:03.098 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 98524 2024-09-14T06:32:03.099 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:03.099 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:32:03.099 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:03.099 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:32:03.099 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045815+0000", 2024-09-14T06:32:03.099 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:03.099 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:03.099 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:03.099 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:32:03.099 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045815+0000", 2024-09-14T06:32:03.099 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:03.099 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:03.099 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:03.099 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:32:03.100 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045815+0000", 2024-09-14T06:32:03.100 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.11e-07 2024-09-14T06:32:03.100 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:03.100 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:03.100 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:32:03.100 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045816+0000", 2024-09-14T06:32:03.100 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.9099999999999991e-07 2024-09-14T06:32:03.100 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:03.100 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:03.100 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:32:03.100 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045819+0000", 2024-09-14T06:32:03.100 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.9399999999999998e-06 2024-09-14T06:32:03.100 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:03.100 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:03.101 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:32:03.101 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045822+0000", 2024-09-14T06:32:03.101 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.3470000000000001e-06 2024-09-14T06:32:03.101 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:03.101 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:03.101 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:32:03.101 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.078449+0000", 2024-09-14T06:32:03.101 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032627529000000002 2024-09-14T06:32:03.101 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:03.101 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:03.101 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:32:03.101 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.078458+0000", 2024-09-14T06:32:03.101 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.1910000000000006e-06 2024-09-14T06:32:03.102 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:03.102 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:32:03.102 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:03.102 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:03.102 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:32:03.102 INFO:teuthology.orchestra.run.smithi138.stdout:} 2024-09-14T06:32:03.104 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-14T06:32:03.104 INFO:tasks.ceph.osd.1.smithi094.stderr:2024-09-14T06:32:03.102+0000 7f328e47a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33641) UID: 0 2024-09-14T06:32:03.204 INFO:tasks.ceph.osd.5.smithi138.stderr:2024-09-14T06:32:03.203+0000 7fe13746f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33618) UID: 0 2024-09-14T06:32:03.255 INFO:teuthology.orchestra.run.smithi138.stdout:{ 2024-09-14T06:32:03.255 INFO:teuthology.orchestra.run.smithi138.stdout: "ops": [], 2024-09-14T06:32:03.255 INFO:teuthology.orchestra.run.smithi138.stdout: "num_ops": 0 2024-09-14T06:32:03.255 INFO:teuthology.orchestra.run.smithi138.stdout:} 2024-09-14T06:32:03.264 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-14T06:32:03.305 INFO:tasks.ceph.osd.2.smithi094.stderr:2024-09-14T06:32:03.304+0000 7faf3269b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33649) UID: 0 2024-09-14T06:32:03.406 INFO:tasks.ceph.osd.7.smithi138.stderr:2024-09-14T06:32:03.405+0000 7f3c6a21d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33623) UID: 0 2024-09-14T06:32:03.407 INFO:teuthology.orchestra.run.smithi138.stdout:{ 2024-09-14T06:32:03.407 INFO:teuthology.orchestra.run.smithi138.stdout: "ops": [], 2024-09-14T06:32:03.407 INFO:teuthology.orchestra.run.smithi138.stdout: "complaint_time": 30, 2024-09-14T06:32:03.407 INFO:teuthology.orchestra.run.smithi138.stdout: "num_blocked_ops": 0 2024-09-14T06:32:03.408 INFO:teuthology.orchestra.run.smithi138.stdout:} 2024-09-14T06:32:03.416 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-14T06:32:03.507 INFO:tasks.ceph.osd.4.smithi138.stderr:2024-09-14T06:32:03.506+0000 7f6c653c8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33615) UID: 0 2024-09-14T06:32:03.560 INFO:teuthology.orchestra.run.smithi138.stdout:{ 2024-09-14T06:32:03.561 INFO:teuthology.orchestra.run.smithi138.stdout: "size": 20, 2024-09-14T06:32:03.561 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 600, 2024-09-14T06:32:03.561 INFO:teuthology.orchestra.run.smithi138.stdout: "ops": [ 2024-09-14T06:32:03.561 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:03.561 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:53244 3.f 3:f71f51a7:::rbd_data.112dca2f6ae1.0000000000000015:17 [read 1863680~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:32:03.561 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:44.369399+0000", 2024-09-14T06:32:03.561 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 139.18954000599999, 2024-09-14T06:32:03.561 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047291189999999997, 2024-09-14T06:32:03.561 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:32:03.561 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:32:03.561 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:32:03.561 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:32:03.561 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:32:03.562 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:03.562 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 53244 2024-09-14T06:32:03.562 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:03.562 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:32:03.562 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:03.562 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:32:03.562 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369399+0000", 2024-09-14T06:32:03.562 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:03.562 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:03.562 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:03.562 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:32:03.562 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369399+0000", 2024-09-14T06:32:03.562 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:03.562 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:03.563 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:03.563 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:32:03.563 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369400+0000", 2024-09-14T06:32:03.563 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.0699999999999999e-07 2024-09-14T06:32:03.563 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:03.563 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:03.563 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:32:03.563 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369401+0000", 2024-09-14T06:32:03.563 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.4300000000000001e-06 2024-09-14T06:32:03.563 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:03.563 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:03.563 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:32:03.563 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369404+0000", 2024-09-14T06:32:03.563 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.061e-06 2024-09-14T06:32:03.564 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:03.564 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:03.564 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:32:03.564 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369407+0000", 2024-09-14T06:32:03.564 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.6139999999999999e-06 2024-09-14T06:32:03.564 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:03.564 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:03.564 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:32:03.564 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.416517+0000", 2024-09-14T06:32:03.564 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047110338000000002 2024-09-14T06:32:03.564 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:03.564 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:03.564 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:32:03.564 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.416616+0000", 2024-09-14T06:32:03.565 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.8651000000000001e-05 2024-09-14T06:32:03.565 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:03.565 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:03.565 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:32:03.565 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.416691+0000", 2024-09-14T06:32:03.565 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.4888999999999994e-05 2024-09-14T06:32:03.565 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:03.565 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:32:03.565 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:03.565 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:03.565 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:03.565 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:53245 3.f 3:f71f51a7:::rbd_data.112dca2f6ae1.0000000000000015:17 [read 1888256~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:32:03.565 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:44.369410+0000", 2024-09-14T06:32:03.566 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 139.18952971799999, 2024-09-14T06:32:03.566 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047399275999999997, 2024-09-14T06:32:03.566 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:32:03.566 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:32:03.566 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:32:03.566 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:32:03.566 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:32:03.566 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:03.566 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 53245 2024-09-14T06:32:03.566 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:03.566 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:32:03.566 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:03.566 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:32:03.566 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369410+0000", 2024-09-14T06:32:03.567 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:03.567 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:03.567 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:03.567 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:32:03.567 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369410+0000", 2024-09-14T06:32:03.567 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:03.567 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:03.567 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:03.567 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:32:03.567 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369410+0000", 2024-09-14T06:32:03.567 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.1899999999999999e-07 2024-09-14T06:32:03.567 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:03.567 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:03.568 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:32:03.568 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369411+0000", 2024-09-14T06:32:03.568 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.0300000000000001e-06 2024-09-14T06:32:03.568 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:03.568 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:03.568 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:32:03.568 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369414+0000", 2024-09-14T06:32:03.568 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.1099999999999999e-06 2024-09-14T06:32:03.568 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:03.568 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:03.568 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:32:03.568 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369417+0000", 2024-09-14T06:32:03.568 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.6189999999999998e-06 2024-09-14T06:32:03.568 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:03.569 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:03.569 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:32:03.569 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.416718+0000", 2024-09-14T06:32:03.569 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047301322999999999 2024-09-14T06:32:03.569 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:03.569 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:03.569 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:32:03.569 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.416759+0000", 2024-09-14T06:32:03.569 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.1009e-05 2024-09-14T06:32:03.569 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:03.569 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:03.569 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:32:03.569 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.416809+0000", 2024-09-14T06:32:03.569 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.9966000000000002e-05 2024-09-14T06:32:03.570 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:03.570 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:32:03.570 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:03.570 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:03.570 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:03.570 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:53246 3.f 3:f71f51a7:::rbd_data.112dca2f6ae1.0000000000000015:17 [read 1912832~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:32:03.570 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:44.369420+0000", 2024-09-14T06:32:03.570 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 139.18951977399999, 2024-09-14T06:32:03.570 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047529463000000001, 2024-09-14T06:32:03.570 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:32:03.570 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:32:03.570 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:32:03.570 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:32:03.570 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:32:03.571 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:03.571 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 53246 2024-09-14T06:32:03.571 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:03.571 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:32:03.571 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:03.571 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:32:03.571 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369420+0000", 2024-09-14T06:32:03.571 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:03.571 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:03.571 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:03.571 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:32:03.571 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369420+0000", 2024-09-14T06:32:03.571 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:03.571 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:03.572 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:03.572 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:32:03.572 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369420+0000", 2024-09-14T06:32:03.572 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.1299999999999999e-07 2024-09-14T06:32:03.572 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:03.572 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:03.572 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:32:03.572 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369425+0000", 2024-09-14T06:32:03.572 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.0130000000000001e-06 2024-09-14T06:32:03.572 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:03.572 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:03.572 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:32:03.572 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369428+0000", 2024-09-14T06:32:03.573 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.349e-06 2024-09-14T06:32:03.573 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:03.573 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:03.573 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:32:03.573 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369431+0000", 2024-09-14T06:32:03.573 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.7389999999999999e-06 2024-09-14T06:32:03.573 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:03.573 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:03.573 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:32:03.573 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.416834+0000", 2024-09-14T06:32:03.573 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047402669000000001 2024-09-14T06:32:03.573 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:03.573 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:03.573 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:32:03.574 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.416882+0000", 2024-09-14T06:32:03.574 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.8068999999999997e-05 2024-09-14T06:32:03.574 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:03.574 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:03.574 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:32:03.574 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.416949+0000", 2024-09-14T06:32:03.574 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.7410999999999995e-05 2024-09-14T06:32:03.574 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:03.574 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:32:03.574 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:03.574 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:03.574 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:03.574 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:53247 3.f 3:f71f51a7:::rbd_data.112dca2f6ae1.0000000000000015:17 [read 1937408~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:32:03.575 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:44.369434+0000", 2024-09-14T06:32:03.575 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 139.18950501, 2024-09-14T06:32:03.575 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047652264999999999, 2024-09-14T06:32:03.575 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:32:03.575 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:32:03.575 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:32:03.575 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:32:03.575 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:32:03.575 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:03.575 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 53247 2024-09-14T06:32:03.575 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:03.575 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:32:03.575 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:03.575 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:32:03.576 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369434+0000", 2024-09-14T06:32:03.576 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:03.576 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:03.576 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:03.576 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:32:03.576 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369434+0000", 2024-09-14T06:32:03.576 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:03.576 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:03.576 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:03.576 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:32:03.576 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369435+0000", 2024-09-14T06:32:03.576 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.2100000000000001e-07 2024-09-14T06:32:03.576 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:03.577 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:03.577 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:32:03.577 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369436+0000", 2024-09-14T06:32:03.577 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.122e-06 2024-09-14T06:32:03.577 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:03.577 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:03.577 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:32:03.577 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369439+0000", 2024-09-14T06:32:03.577 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.0400000000000001e-06 2024-09-14T06:32:03.577 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:03.577 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:03.577 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:32:03.577 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369441+0000", 2024-09-14T06:32:03.577 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.728e-06 2024-09-14T06:32:03.578 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:03.578 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:03.578 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:32:03.578 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.416977+0000", 2024-09-14T06:32:03.578 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047535344 2024-09-14T06:32:03.578 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:03.578 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:03.578 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:32:03.578 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.417029+0000", 2024-09-14T06:32:03.578 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.2037999999999999e-05 2024-09-14T06:32:03.578 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:03.578 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:03.578 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:32:03.578 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.417087+0000", 2024-09-14T06:32:03.579 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.7772000000000002e-05 2024-09-14T06:32:03.579 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:03.579 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:32:03.579 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:03.579 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:03.579 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:03.579 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:53248 3.f 3:f71f51a7:::rbd_data.112dca2f6ae1.0000000000000015:17 [read 1961984~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:32:03.579 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:44.369445+0000", 2024-09-14T06:32:03.579 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 139.18949445800001, 2024-09-14T06:32:03.579 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047798489, 2024-09-14T06:32:03.579 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:32:03.579 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:32:03.579 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:32:03.579 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:32:03.580 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:32:03.580 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:03.580 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 53248 2024-09-14T06:32:03.580 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:03.580 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:32:03.580 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:03.580 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:32:03.580 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369445+0000", 2024-09-14T06:32:03.580 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:03.580 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:03.580 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:03.580 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:32:03.580 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369445+0000", 2024-09-14T06:32:03.581 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:03.581 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:03.581 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:03.581 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:32:03.581 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369445+0000", 2024-09-14T06:32:03.581 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.0699999999999999e-07 2024-09-14T06:32:03.581 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:03.581 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:03.581 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:32:03.581 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369446+0000", 2024-09-14T06:32:03.581 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.1000000000000001e-06 2024-09-14T06:32:03.581 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:03.581 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:03.581 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:32:03.582 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369449+0000", 2024-09-14T06:32:03.582 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.1039999999999998e-06 2024-09-14T06:32:03.582 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:03.582 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:03.582 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:32:03.582 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369452+0000", 2024-09-14T06:32:03.582 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.6199999999999999e-06 2024-09-14T06:32:03.582 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:03.582 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:03.582 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:32:03.582 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.417114+0000", 2024-09-14T06:32:03.582 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047661901999999999 2024-09-14T06:32:03.582 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:03.582 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:03.582 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:32:03.583 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.417181+0000", 2024-09-14T06:32:03.583 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.7385999999999995e-05 2024-09-14T06:32:03.583 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:03.583 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:03.583 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:32:03.583 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.417243+0000", 2024-09-14T06:32:03.583 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.2169999999999996e-05 2024-09-14T06:32:03.583 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:03.583 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:32:03.583 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:03.583 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:03.583 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:03.583 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:53249 3.f 3:f71f51a7:::rbd_data.112dca2f6ae1.0000000000000015:17 [read 1986560~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:32:03.583 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:44.369455+0000", 2024-09-14T06:32:03.584 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 139.189484147, 2024-09-14T06:32:03.584 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047890493999999999, 2024-09-14T06:32:03.584 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:32:03.584 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:32:03.584 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:32:03.584 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:32:03.584 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:32:03.584 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:03.584 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 53249 2024-09-14T06:32:03.584 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:03.584 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:32:03.584 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:03.584 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:32:03.584 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369455+0000", 2024-09-14T06:32:03.585 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:03.585 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:03.585 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:03.585 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:32:03.585 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369455+0000", 2024-09-14T06:32:03.585 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:03.585 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:03.585 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:03.585 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:32:03.585 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369455+0000", 2024-09-14T06:32:03.585 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.0800000000000001e-07 2024-09-14T06:32:03.585 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:03.585 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:03.585 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:32:03.585 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369456+0000", 2024-09-14T06:32:03.586 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.0729999999999999e-06 2024-09-14T06:32:03.586 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:03.586 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:03.586 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:32:03.586 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369460+0000", 2024-09-14T06:32:03.586 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.027e-06 2024-09-14T06:32:03.586 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:03.586 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:03.586 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:32:03.586 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369462+0000", 2024-09-14T06:32:03.586 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.5320000000000001e-06 2024-09-14T06:32:03.586 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:03.586 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:03.586 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:32:03.586 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.417257+0000", 2024-09-14T06:32:03.587 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.04779489 2024-09-14T06:32:03.587 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:03.587 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:03.587 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:32:03.587 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.417296+0000", 2024-09-14T06:32:03.587 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.8658000000000002e-05 2024-09-14T06:32:03.587 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:03.587 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:03.587 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:32:03.587 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.417346+0000", 2024-09-14T06:32:03.587 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.0105999999999998e-05 2024-09-14T06:32:03.587 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:03.587 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:32:03.587 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:03.588 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:03.588 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:03.588 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:53250 3.f 3:f71f51a7:::rbd_data.112dca2f6ae1.0000000000000015:17 [read 2011136~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:32:03.588 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:44.369465+0000", 2024-09-14T06:32:03.588 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 139.18947417699999, 2024-09-14T06:32:03.588 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.048032230000000002, 2024-09-14T06:32:03.588 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:32:03.588 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:32:03.588 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:32:03.588 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:32:03.588 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:32:03.588 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:03.588 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 53250 2024-09-14T06:32:03.589 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:03.589 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:32:03.589 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:03.589 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:32:03.589 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369465+0000", 2024-09-14T06:32:03.589 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:03.589 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:03.589 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:03.589 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:32:03.589 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369465+0000", 2024-09-14T06:32:03.589 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:03.589 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:03.589 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:03.589 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:32:03.590 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369465+0000", 2024-09-14T06:32:03.590 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.2100000000000001e-07 2024-09-14T06:32:03.590 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:03.590 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:03.590 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:32:03.590 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369466+0000", 2024-09-14T06:32:03.590 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.0640000000000001e-06 2024-09-14T06:32:03.590 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:03.590 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:03.590 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:32:03.590 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369469+0000", 2024-09-14T06:32:03.590 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.0170000000000001e-06 2024-09-14T06:32:03.590 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:03.591 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:03.591 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:32:03.591 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369472+0000", 2024-09-14T06:32:03.591 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.6189999999999998e-06 2024-09-14T06:32:03.591 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:03.591 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:03.591 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:32:03.591 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.417383+0000", 2024-09-14T06:32:03.591 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047911092000000002 2024-09-14T06:32:03.591 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:03.591 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:03.591 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:32:03.591 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.417430+0000", 2024-09-14T06:32:03.591 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.6898999999999999e-05 2024-09-14T06:32:03.592 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:03.592 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:03.592 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:32:03.592 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.417497+0000", 2024-09-14T06:32:03.592 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.7317999999999995e-05 2024-09-14T06:32:03.592 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:03.592 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:32:03.592 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:03.592 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:03.592 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:03.592 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:53251 3.f 3:f71f51a7:::rbd_data.112dca2f6ae1.0000000000000015:17 [read 2035712~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:32:03.592 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:44.369475+0000", 2024-09-14T06:32:03.592 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 139.18946418100001, 2024-09-14T06:32:03.592 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.048154542000000002, 2024-09-14T06:32:03.593 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:32:03.593 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:32:03.593 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:32:03.593 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:32:03.593 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:32:03.593 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:03.593 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 53251 2024-09-14T06:32:03.593 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:03.593 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:32:03.593 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:03.593 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:32:03.593 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369475+0000", 2024-09-14T06:32:03.593 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:03.593 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:03.594 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:03.594 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:32:03.594 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369475+0000", 2024-09-14T06:32:03.594 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:03.594 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:03.594 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:03.594 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:32:03.594 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369475+0000", 2024-09-14T06:32:03.594 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.0100000000000001e-07 2024-09-14T06:32:03.594 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:03.594 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:03.594 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:32:03.594 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369476+0000", 2024-09-14T06:32:03.594 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.0580000000000001e-06 2024-09-14T06:32:03.595 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:03.595 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:03.595 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:32:03.595 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369479+0000", 2024-09-14T06:32:03.595 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.9950000000000001e-06 2024-09-14T06:32:03.595 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:03.595 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:03.595 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:32:03.595 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369482+0000", 2024-09-14T06:32:03.595 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.6340000000000001e-06 2024-09-14T06:32:03.595 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:03.595 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:03.595 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:32:03.596 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.417530+0000", 2024-09-14T06:32:03.596 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.048047535000000002 2024-09-14T06:32:03.596 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:03.596 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:03.596 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:32:03.596 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.417574+0000", 2024-09-14T06:32:03.596 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.4255999999999998e-05 2024-09-14T06:32:03.596 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:03.596 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:03.596 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:32:03.596 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.417630+0000", 2024-09-14T06:32:03.596 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.5862999999999997e-05 2024-09-14T06:32:03.596 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:03.597 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:32:03.597 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:03.597 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:03.597 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:03.597 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:53252 3.f 3:f71f51a7:::rbd_data.112dca2f6ae1.0000000000000015:17 [read 2060288~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:32:03.597 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:44.369485+0000", 2024-09-14T06:32:03.597 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 139.18945433299999, 2024-09-14T06:32:03.597 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.048254116, 2024-09-14T06:32:03.597 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:32:03.597 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:32:03.597 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:32:03.597 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:32:03.597 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:32:03.597 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:03.598 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 53252 2024-09-14T06:32:03.598 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:03.598 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:32:03.598 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:03.598 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:32:03.598 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369485+0000", 2024-09-14T06:32:03.598 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:03.598 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:03.598 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:03.598 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:32:03.598 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369485+0000", 2024-09-14T06:32:03.598 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:03.598 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:03.598 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:03.599 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:32:03.599 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369485+0000", 2024-09-14T06:32:03.599 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.1e-07 2024-09-14T06:32:03.599 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:03.599 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:03.599 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:32:03.599 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369487+0000", 2024-09-14T06:32:03.599 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.42e-06 2024-09-14T06:32:03.599 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:03.599 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:03.599 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:32:03.599 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369490+0000", 2024-09-14T06:32:03.599 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.9639999999999999e-06 2024-09-14T06:32:03.599 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:03.600 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:03.600 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:32:03.600 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369492+0000", 2024-09-14T06:32:03.600 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.5679999999999998e-06 2024-09-14T06:32:03.600 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:03.600 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:03.600 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:32:03.600 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.417645+0000", 2024-09-14T06:32:03.600 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.048152753 2024-09-14T06:32:03.600 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:03.600 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:03.600 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:32:03.600 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.417687+0000", 2024-09-14T06:32:03.600 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.1900999999999997e-05 2024-09-14T06:32:03.601 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:03.601 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:03.601 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:32:03.601 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.417739+0000", 2024-09-14T06:32:03.601 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.2299999999999997e-05 2024-09-14T06:32:03.601 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:03.601 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:32:03.601 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:03.601 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:03.601 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:03.601 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:53253 3.f 3:f71f51a7:::rbd_data.112dca2f6ae1.0000000000000015:17 [read 2084864~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:32:03.601 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:44.369495+0000", 2024-09-14T06:32:03.601 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 139.18944420899999, 2024-09-14T06:32:03.601 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.048366785000000002, 2024-09-14T06:32:03.601 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:32:03.602 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:32:03.602 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:32:03.602 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:32:03.602 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:32:03.602 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:03.602 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 53253 2024-09-14T06:32:03.602 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:03.602 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:32:03.602 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:03.602 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:32:03.602 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369495+0000", 2024-09-14T06:32:03.602 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:03.602 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:03.602 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:03.603 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:32:03.603 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369495+0000", 2024-09-14T06:32:03.603 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:03.603 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:03.603 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:03.603 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:32:03.603 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370388+0000", 2024-09-14T06:32:03.603 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00089287999999999998 2024-09-14T06:32:03.603 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:03.603 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:03.603 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:32:03.603 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370392+0000", 2024-09-14T06:32:03.603 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.2130000000000002e-06 2024-09-14T06:32:03.603 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:03.603 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:03.604 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:32:03.604 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370401+0000", 2024-09-14T06:32:03.604 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.7649999999999999e-06 2024-09-14T06:32:03.604 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:03.604 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:03.604 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:32:03.604 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370406+0000", 2024-09-14T06:32:03.604 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.0159999999999999e-06 2024-09-14T06:32:03.604 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:03.604 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:03.604 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:32:03.604 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.417764+0000", 2024-09-14T06:32:03.604 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047358348000000001 2024-09-14T06:32:03.604 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:03.605 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:03.605 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:32:03.605 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.417806+0000", 2024-09-14T06:32:03.605 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.1149999999999997e-05 2024-09-14T06:32:03.605 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:03.605 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:03.605 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:32:03.605 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.417862+0000", 2024-09-14T06:32:03.605 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.6412999999999997e-05 2024-09-14T06:32:03.605 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:03.605 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:32:03.605 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:03.605 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:03.605 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:03.606 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:53340 3.f 3:f71f51a7:::rbd_data.112dca2f6ae1.0000000000000015:17 [read 1056768~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:32:03.606 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:44.370421+0000", 2024-09-14T06:32:03.606 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 139.188518329, 2024-09-14T06:32:03.606 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047566313999999998, 2024-09-14T06:32:03.606 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:32:03.606 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:32:03.606 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:32:03.606 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:32:03.606 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:32:03.606 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:03.606 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 53340 2024-09-14T06:32:03.606 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:03.606 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:32:03.606 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:03.607 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:32:03.607 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370421+0000", 2024-09-14T06:32:03.607 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:03.607 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:03.607 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:03.607 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:32:03.607 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370421+0000", 2024-09-14T06:32:03.607 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:03.607 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:03.607 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:03.607 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:32:03.607 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370422+0000", 2024-09-14T06:32:03.607 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.3799999999999996e-07 2024-09-14T06:32:03.607 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:03.608 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:03.608 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:32:03.608 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370424+0000", 2024-09-14T06:32:03.608 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.6919999999999998e-06 2024-09-14T06:32:03.608 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:03.608 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:03.608 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:32:03.608 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370432+0000", 2024-09-14T06:32:03.608 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.2439999999999996e-06 2024-09-14T06:32:03.608 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:03.608 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:03.608 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:32:03.608 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370437+0000", 2024-09-14T06:32:03.608 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.5389999999999998e-06 2024-09-14T06:32:03.609 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:03.609 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:03.609 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:32:03.609 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.417888+0000", 2024-09-14T06:32:03.609 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047450432000000001 2024-09-14T06:32:03.609 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:03.609 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:03.609 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:32:03.609 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.417935+0000", 2024-09-14T06:32:03.609 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.7454999999999999e-05 2024-09-14T06:32:03.609 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:03.609 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:03.609 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:32:03.610 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.417987+0000", 2024-09-14T06:32:03.610 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.2213999999999998e-05 2024-09-14T06:32:03.610 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:03.610 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:32:03.610 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:03.610 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:03.610 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:03.610 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:53341 3.f 3:f71f51a7:::rbd_data.112dca2f6ae1.0000000000000015:17 [read 1081344~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:32:03.610 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:44.370446+0000", 2024-09-14T06:32:03.610 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 139.188493056, 2024-09-14T06:32:03.610 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047665723, 2024-09-14T06:32:03.610 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:32:03.610 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:32:03.610 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:32:03.610 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:32:03.611 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:32:03.611 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:03.611 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 53341 2024-09-14T06:32:03.611 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:03.611 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:32:03.611 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:03.611 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:32:03.611 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370446+0000", 2024-09-14T06:32:03.611 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:03.611 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:03.611 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:03.611 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:32:03.611 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370446+0000", 2024-09-14T06:32:03.611 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:03.612 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:03.612 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:03.612 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:32:03.612 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370447+0000", 2024-09-14T06:32:03.612 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.6100000000000002e-07 2024-09-14T06:32:03.612 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:03.612 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:03.612 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:32:03.612 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370448+0000", 2024-09-14T06:32:03.612 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.7370000000000001e-06 2024-09-14T06:32:03.612 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:03.612 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:03.612 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:32:03.612 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370462+0000", 2024-09-14T06:32:03.612 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.3227000000000001e-05 2024-09-14T06:32:03.613 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:03.613 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:03.613 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:32:03.613 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370467+0000", 2024-09-14T06:32:03.613 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.6540000000000004e-06 2024-09-14T06:32:03.613 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:03.613 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:03.613 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:32:03.613 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.418013+0000", 2024-09-14T06:32:03.613 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047545325999999999 2024-09-14T06:32:03.613 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:03.613 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:03.613 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:32:03.613 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.418054+0000", 2024-09-14T06:32:03.614 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.1696999999999997e-05 2024-09-14T06:32:03.614 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:03.614 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:03.614 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:32:03.614 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.418112+0000", 2024-09-14T06:32:03.614 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.7720999999999998e-05 2024-09-14T06:32:03.614 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:03.614 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:32:03.614 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:03.614 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:03.614 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:03.614 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:53342 3.f 3:f71f51a7:::rbd_data.112dca2f6ae1.0000000000000015:17 [read 1105920~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:32:03.614 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:44.370475+0000", 2024-09-14T06:32:03.614 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 139.18846399500001, 2024-09-14T06:32:03.615 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047748848000000003, 2024-09-14T06:32:03.615 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:32:03.615 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:32:03.615 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:32:03.615 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:32:03.615 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:32:03.615 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:03.615 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 53342 2024-09-14T06:32:03.615 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:03.615 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:32:03.615 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:03.615 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:32:03.615 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370475+0000", 2024-09-14T06:32:03.615 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:03.616 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:03.616 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:03.616 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:32:03.616 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370475+0000", 2024-09-14T06:32:03.616 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:03.616 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:03.616 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:03.616 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:32:03.616 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370476+0000", 2024-09-14T06:32:03.616 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.4700000000000002e-07 2024-09-14T06:32:03.616 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:03.616 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:03.616 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:32:03.617 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370478+0000", 2024-09-14T06:32:03.617 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.5890000000000001e-06 2024-09-14T06:32:03.617 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:03.617 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:03.617 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:32:03.617 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370486+0000", 2024-09-14T06:32:03.617 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.3470000000000001e-06 2024-09-14T06:32:03.617 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:03.617 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:03.617 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:32:03.617 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370491+0000", 2024-09-14T06:32:03.617 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.5720000000000002e-06 2024-09-14T06:32:03.617 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:03.617 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:03.618 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:32:03.618 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.418135+0000", 2024-09-14T06:32:03.618 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047644094999999997 2024-09-14T06:32:03.618 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:03.618 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:03.618 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:32:03.618 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.418174+0000", 2024-09-14T06:32:03.618 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.8204000000000002e-05 2024-09-14T06:32:03.618 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:03.618 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:03.618 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:32:03.618 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.418224+0000", 2024-09-14T06:32:03.618 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.0593999999999997e-05 2024-09-14T06:32:03.618 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:03.619 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:32:03.619 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:03.619 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:03.619 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:03.619 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:53343 3.f 3:f71f51a7:::rbd_data.112dca2f6ae1.0000000000000015:17 [read 1130496~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:32:03.619 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:44.370498+0000", 2024-09-14T06:32:03.619 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 139.188441119, 2024-09-14T06:32:03.619 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047841574999999997, 2024-09-14T06:32:03.619 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:32:03.619 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:32:03.619 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:32:03.619 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:32:03.619 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:32:03.619 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:03.619 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 53343 2024-09-14T06:32:03.620 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:03.620 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:32:03.620 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:03.620 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:32:03.620 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370498+0000", 2024-09-14T06:32:03.620 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:03.620 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:03.620 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:03.620 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:32:03.620 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370498+0000", 2024-09-14T06:32:03.620 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:03.620 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:03.620 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:03.621 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:32:03.621 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370499+0000", 2024-09-14T06:32:03.621 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.9999999999999998e-07 2024-09-14T06:32:03.621 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:03.621 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:03.621 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:32:03.621 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370501+0000", 2024-09-14T06:32:03.621 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.0310000000000001e-06 2024-09-14T06:32:03.621 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:03.621 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:03.621 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:32:03.621 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370507+0000", 2024-09-14T06:32:03.621 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.4169999999999997e-06 2024-09-14T06:32:03.621 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:03.622 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:03.622 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:32:03.622 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370516+0000", 2024-09-14T06:32:03.622 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.7190000000000007e-06 2024-09-14T06:32:03.622 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:03.622 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:03.622 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:32:03.622 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.418247+0000", 2024-09-14T06:32:03.622 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047731190999999999 2024-09-14T06:32:03.622 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:03.622 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:03.622 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:32:03.622 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.418286+0000", 2024-09-14T06:32:03.622 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.8988e-05 2024-09-14T06:32:03.623 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:03.623 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:03.623 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:32:03.623 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.418340+0000", 2024-09-14T06:32:03.623 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.3829000000000002e-05 2024-09-14T06:32:03.623 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:03.623 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:32:03.623 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:03.623 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:03.623 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:03.623 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:53344 3.f 3:f71f51a7:::rbd_data.112dca2f6ae1.0000000000000015:17 [read 1155072~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:32:03.623 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:44.370524+0000", 2024-09-14T06:32:03.623 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 139.188415349, 2024-09-14T06:32:03.624 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047972262000000002, 2024-09-14T06:32:03.624 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:32:03.624 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:32:03.624 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:32:03.624 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:32:03.624 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:32:03.624 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:03.624 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 53344 2024-09-14T06:32:03.624 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:03.624 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:32:03.624 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:03.624 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:32:03.624 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370524+0000", 2024-09-14T06:32:03.625 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:03.625 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:03.625 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:03.625 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:32:03.625 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370524+0000", 2024-09-14T06:32:03.625 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:03.625 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:03.625 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:03.625 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:32:03.625 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370618+0000", 2024-09-14T06:32:03.625 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.4425999999999993e-05 2024-09-14T06:32:03.625 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:03.625 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:03.626 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:32:03.626 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370622+0000", 2024-09-14T06:32:03.626 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.3519999999999998e-06 2024-09-14T06:32:03.626 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:03.626 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:03.626 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:32:03.626 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370631+0000", 2024-09-14T06:32:03.626 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.9299999999999992e-06 2024-09-14T06:32:03.626 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:03.626 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:03.626 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:32:03.626 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370637+0000", 2024-09-14T06:32:03.626 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.3269999999999997e-06 2024-09-14T06:32:03.626 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:03.626 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:03.627 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:32:03.627 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.418372+0000", 2024-09-14T06:32:03.627 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047734764999999998 2024-09-14T06:32:03.627 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:03.627 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:03.627 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:32:03.627 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.418431+0000", 2024-09-14T06:32:03.627 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.8696999999999997e-05 2024-09-14T06:32:03.627 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:03.627 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:03.627 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:32:03.627 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.418496+0000", 2024-09-14T06:32:03.627 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.5765000000000005e-05 2024-09-14T06:32:03.627 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:03.628 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:32:03.628 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:03.628 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:03.628 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:03.628 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:53345 3.f 3:f71f51a7:::rbd_data.112dca2f6ae1.0000000000000015:17 [read 1179648~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:32:03.628 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:44.370647+0000", 2024-09-14T06:32:03.628 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 139.18829188199999, 2024-09-14T06:32:03.628 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047953775999999997, 2024-09-14T06:32:03.628 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:32:03.628 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:32:03.628 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:32:03.628 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:32:03.628 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:32:03.629 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:03.629 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 53345 2024-09-14T06:32:03.629 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:03.629 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:32:03.629 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:03.629 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:32:03.629 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370647+0000", 2024-09-14T06:32:03.629 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:03.629 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:03.629 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:03.629 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:32:03.629 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370647+0000", 2024-09-14T06:32:03.629 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:03.629 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:03.630 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:03.630 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:32:03.630 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.372346+0000", 2024-09-14T06:32:03.630 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.001698284 2024-09-14T06:32:03.630 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:03.630 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:03.630 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:32:03.630 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.372351+0000", 2024-09-14T06:32:03.630 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.8890000000000001e-06 2024-09-14T06:32:03.630 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:03.630 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:03.630 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:32:03.630 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.372362+0000", 2024-09-14T06:32:03.630 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.0941000000000001e-05 2024-09-14T06:32:03.631 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:03.631 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:03.631 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:32:03.631 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.372369+0000", 2024-09-14T06:32:03.631 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.362e-06 2024-09-14T06:32:03.631 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:03.631 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:03.631 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:32:03.631 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.418510+0000", 2024-09-14T06:32:03.631 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046140914999999998 2024-09-14T06:32:03.631 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:03.631 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:03.631 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:32:03.632 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.418549+0000", 2024-09-14T06:32:03.632 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.8717999999999998e-05 2024-09-14T06:32:03.632 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:03.632 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:03.632 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:32:03.632 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.418601+0000", 2024-09-14T06:32:03.632 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.2667000000000002e-05 2024-09-14T06:32:03.632 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:03.632 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:32:03.632 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:03.632 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:03.632 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:03.632 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:53353 3.f 3:f71f51a7:::rbd_data.112dca2f6ae1.0000000000000015:17 [read 1376256~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:32:03.632 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:44.372545+0000", 2024-09-14T06:32:03.633 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 139.186394807, 2024-09-14T06:32:03.633 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047364232999999999, 2024-09-14T06:32:03.633 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:32:03.633 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:32:03.633 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:32:03.633 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:32:03.633 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:32:03.633 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:03.633 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 53353 2024-09-14T06:32:03.633 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:03.633 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:32:03.633 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:03.633 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:32:03.633 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.372545+0000", 2024-09-14T06:32:03.634 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:03.634 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:03.634 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:03.634 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:32:03.634 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.372545+0000", 2024-09-14T06:32:03.634 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:03.634 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:03.634 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:03.634 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:32:03.634 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.372545+0000", 2024-09-14T06:32:03.634 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.27e-07 2024-09-14T06:32:03.634 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:03.634 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:03.634 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:32:03.635 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.372547+0000", 2024-09-14T06:32:03.635 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.0600000000000002e-06 2024-09-14T06:32:03.635 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:03.635 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:03.635 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:32:03.635 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.372553+0000", 2024-09-14T06:32:03.635 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.7980000000000002e-06 2024-09-14T06:32:03.635 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:03.635 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:03.635 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:32:03.635 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.372557+0000", 2024-09-14T06:32:03.635 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.7389999999999999e-06 2024-09-14T06:32:03.636 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:03.636 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:03.636 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:32:03.636 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.419811+0000", 2024-09-14T06:32:03.636 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047253310999999999 2024-09-14T06:32:03.636 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:03.636 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:03.636 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:32:03.636 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.419854+0000", 2024-09-14T06:32:03.636 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.3448000000000001e-05 2024-09-14T06:32:03.636 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:03.636 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:03.636 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:32:03.636 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.419909+0000", 2024-09-14T06:32:03.637 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.4549999999999998e-05 2024-09-14T06:32:03.637 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:03.637 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:32:03.637 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:03.637 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:03.637 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:03.637 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:53354 3.f 3:f71f51a7:::rbd_data.112dca2f6ae1.0000000000000015:17 [read 1400832~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:32:03.637 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:44.372563+0000", 2024-09-14T06:32:03.637 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 139.18637637, 2024-09-14T06:32:03.637 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047482233999999998, 2024-09-14T06:32:03.637 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:32:03.637 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:32:03.637 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:32:03.637 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:32:03.638 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:32:03.638 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:03.638 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 53354 2024-09-14T06:32:03.638 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:03.638 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:32:03.638 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:03.638 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:32:03.638 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.372563+0000", 2024-09-14T06:32:03.638 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:03.638 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:03.638 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:03.638 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:32:03.638 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.372563+0000", 2024-09-14T06:32:03.638 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:03.638 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:03.639 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:03.639 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:32:03.639 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.375009+0000", 2024-09-14T06:32:03.639 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.0024457680000000001 2024-09-14T06:32:03.639 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:03.639 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:03.639 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:32:03.639 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.375014+0000", 2024-09-14T06:32:03.639 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.0599999999999998e-06 2024-09-14T06:32:03.639 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:03.639 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:03.639 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:32:03.639 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.375026+0000", 2024-09-14T06:32:03.639 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.1783999999999999e-05 2024-09-14T06:32:03.640 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:03.640 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:03.640 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:32:03.640 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.375035+0000", 2024-09-14T06:32:03.640 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.7799999999999995e-06 2024-09-14T06:32:03.640 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:03.640 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:03.640 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:32:03.640 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.419942+0000", 2024-09-14T06:32:03.640 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.044906982999999998 2024-09-14T06:32:03.640 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:03.640 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:03.640 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:32:03.641 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.419989+0000", 2024-09-14T06:32:03.641 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.7104999999999997e-05 2024-09-14T06:32:03.641 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:03.641 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:03.641 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:32:03.641 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.420045+0000", 2024-09-14T06:32:03.641 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.5754000000000002e-05 2024-09-14T06:32:03.641 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:03.641 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:32:03.641 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:03.641 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:03.641 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:03.641 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:225577 3.15 3:ac159a70:::rbd_header.112daa9ec306:head [watch reconnect cookie 140616827151664 gen 4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e188)", 2024-09-14T06:32:03.641 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:31:52.733493+0000", 2024-09-14T06:32:03.641 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 10.825445906000001, 2024-09-14T06:32:03.642 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.0156593789999999, 2024-09-14T06:32:03.642 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:32:03.642 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:32:03.642 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:32:03.642 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:32:03.642 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:32:03.642 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:03.642 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 225577 2024-09-14T06:32:03.642 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:03.642 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:32:03.642 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:03.642 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:32:03.642 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:31:52.733493+0000", 2024-09-14T06:32:03.642 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:03.643 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:03.643 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:03.643 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:32:03.643 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:31:52.733493+0000", 2024-09-14T06:32:03.643 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:03.643 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:03.643 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:03.643 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:32:03.643 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:31:52.733495+0000", 2024-09-14T06:32:03.643 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.0059999999999999e-06 2024-09-14T06:32:03.643 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:03.643 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:03.643 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:32:03.643 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:31:52.733504+0000", 2024-09-14T06:32:03.643 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.5490000000000006e-06 2024-09-14T06:32:03.643 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:03.644 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:03.644 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:32:03.644 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:31:52.733522+0000", 2024-09-14T06:32:03.644 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.7660999999999999e-05 2024-09-14T06:32:03.644 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:03.644 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:03.644 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:32:03.644 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:31:52.733531+0000", 2024-09-14T06:32:03.644 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.7180000000000008e-06 2024-09-14T06:32:03.644 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:03.644 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:03.644 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:32:03.644 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:31:52.733598+0000", 2024-09-14T06:32:03.644 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.6490999999999997e-05 2024-09-14T06:32:03.645 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:03.645 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:03.645 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "op must wait for map", 2024-09-14T06:32:03.645 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:31:52.733605+0000", 2024-09-14T06:32:03.645 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.6109999999999996e-06 2024-09-14T06:32:03.645 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:03.645 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:03.645 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:32:03.645 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:31:52.737908+0000", 2024-09-14T06:32:03.645 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.0043023050000000002 2024-09-14T06:32:03.645 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:03.645 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:03.645 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "waiting for peered", 2024-09-14T06:32:03.645 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:31:52.737923+0000", 2024-09-14T06:32:03.645 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.488e-05 2024-09-14T06:32:03.646 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:03.646 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:03.646 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:32:03.646 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:31:53.746776+0000", 2024-09-14T06:32:03.646 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.0088535359999999 2024-09-14T06:32:03.646 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:03.646 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:03.646 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:32:03.646 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:31:53.746878+0000", 2024-09-14T06:32:03.646 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.000101788 2024-09-14T06:32:03.646 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:03.646 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:03.646 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:32:03.646 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:31:53.749153+0000", 2024-09-14T06:32:03.646 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.0022748339999999999 2024-09-14T06:32:03.647 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:03.647 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:32:03.647 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:03.647 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:03.647 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:03.647 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:225581 3.15 3:ab407115:::rbd_object_map.112d838b9898:head [call lock.unlock in=10b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e188)", 2024-09-14T06:32:03.647 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:31:52.743472+0000", 2024-09-14T06:32:03.647 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 10.815466867, 2024-09-14T06:32:03.647 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.007270501, 2024-09-14T06:32:03.647 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:32:03.647 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:32:03.647 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:32:03.647 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:32:03.647 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:32:03.648 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:03.648 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 225581 2024-09-14T06:32:03.648 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:03.648 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:32:03.648 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:03.648 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:32:03.648 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:31:52.743472+0000", 2024-09-14T06:32:03.648 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:03.648 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:03.648 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:03.648 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:32:03.648 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:31:52.743472+0000", 2024-09-14T06:32:03.648 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:03.648 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:03.649 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:03.649 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:32:03.649 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:31:52.743473+0000", 2024-09-14T06:32:03.649 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.1999999999999998e-07 2024-09-14T06:32:03.649 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:03.649 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:03.649 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:32:03.649 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:31:52.743478+0000", 2024-09-14T06:32:03.649 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.515e-06 2024-09-14T06:32:03.649 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:03.649 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:03.649 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:32:03.649 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:31:52.743489+0000", 2024-09-14T06:32:03.649 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.1411e-05 2024-09-14T06:32:03.650 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:03.650 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:03.650 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:32:03.650 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:31:52.743494+0000", 2024-09-14T06:32:03.650 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.1649999999999997e-06 2024-09-14T06:32:03.650 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:03.650 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:03.650 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:32:03.650 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:31:52.743548+0000", 2024-09-14T06:32:03.650 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.3758999999999998e-05 2024-09-14T06:32:03.650 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:03.650 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:03.650 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "waiting for peered", 2024-09-14T06:32:03.650 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:31:52.743558+0000", 2024-09-14T06:32:03.651 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.0336999999999999e-05 2024-09-14T06:32:03.651 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:03.651 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:03.651 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:32:03.651 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:31:53.747716+0000", 2024-09-14T06:32:03.651 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.004157035 2024-09-14T06:32:03.651 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:03.651 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:03.651 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:32:03.651 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:31:53.747831+0000", 2024-09-14T06:32:03.651 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.000115394 2024-09-14T06:32:03.651 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:03.651 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:03.651 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "waiting for subops from 3", 2024-09-14T06:32:03.651 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:31:53.748002+0000", 2024-09-14T06:32:03.652 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00017085099999999999 2024-09-14T06:32:03.652 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:03.652 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:03.652 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:32:03.652 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:31:53.749536+0000", 2024-09-14T06:32:03.652 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.0015339990000000001 2024-09-14T06:32:03.652 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:03.652 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:03.652 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "op_commit", 2024-09-14T06:32:03.652 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:31:53.750672+0000", 2024-09-14T06:32:03.652 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.001135973 2024-09-14T06:32:03.652 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:03.652 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:03.652 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "commit_sent", 2024-09-14T06:32:03.653 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:31:53.750704+0000", 2024-09-14T06:32:03.653 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.1791000000000001e-05 2024-09-14T06:32:03.653 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:03.653 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:03.653 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:32:03.653 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:31:53.750743+0000", 2024-09-14T06:32:03.653 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.9450999999999999e-05 2024-09-14T06:32:03.653 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:03.653 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:32:03.653 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:03.653 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:03.653 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:32:03.653 INFO:teuthology.orchestra.run.smithi138.stdout:} 2024-09-14T06:32:03.655 INFO:teuthology.orchestra.run.smithi094.stderr:nodeep-scrub is set 2024-09-14T06:32:03.655 INFO:tasks.ceph.osd.3.smithi094.stderr:2024-09-14T06:32:03.654+0000 7feeeafe8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33655) UID: 0 2024-09-14T06:32:03.656 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-14T06:32:03.755 INFO:tasks.ceph.osd.1.smithi094.stderr:2024-09-14T06:32:03.754+0000 7f328e47a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33641) UID: 0 2024-09-14T06:32:03.799 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-14T06:32:03.799 INFO:teuthology.orchestra.run.smithi094.stdout: "ops": [ 2024-09-14T06:32:03.799 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:03.799 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:228130 3.10 3.5fa61c10 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e199)", 2024-09-14T06:32:03.799 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:32:03.613036+0000", 2024-09-14T06:32:03.799 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 0.18571585700000001, 2024-09-14T06:32:03.799 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.185733706, 2024-09-14T06:32:03.799 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:32:03.800 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:32:03.800 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "waiting for peered", 2024-09-14T06:32:03.800 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:32:03.800 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:32:03.800 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:03.800 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 228130 2024-09-14T06:32:03.800 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:03.800 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:32:03.800 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:03.800 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:32:03.800 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:03.613036+0000", 2024-09-14T06:32:03.800 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:03.800 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:03.801 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:03.801 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:32:03.801 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:03.613036+0000", 2024-09-14T06:32:03.801 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:03.801 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:03.801 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:03.801 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:32:03.801 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:03.613039+0000", 2024-09-14T06:32:03.801 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.3590000000000002e-06 2024-09-14T06:32:03.801 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:03.801 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:03.801 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:32:03.801 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:03.613047+0000", 2024-09-14T06:32:03.801 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.9549999999999992e-06 2024-09-14T06:32:03.802 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:03.802 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:03.802 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:32:03.802 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:03.613068+0000", 2024-09-14T06:32:03.802 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.1212e-05 2024-09-14T06:32:03.802 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:03.802 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:03.802 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:32:03.802 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:03.613081+0000", 2024-09-14T06:32:03.802 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.2527999999999999e-05 2024-09-14T06:32:03.802 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:03.802 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:03.802 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:32:03.802 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:03.613164+0000", 2024-09-14T06:32:03.802 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.3743000000000001e-05 2024-09-14T06:32:03.803 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:03.803 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:03.803 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "op must wait for map", 2024-09-14T06:32:03.803 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:03.613175+0000", 2024-09-14T06:32:03.803 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.0789e-05 2024-09-14T06:32:03.803 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:03.803 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:03.803 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:32:03.803 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:03.619396+0000", 2024-09-14T06:32:03.803 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0062206730000000003 2024-09-14T06:32:03.803 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:03.803 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:03.803 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "waiting for peered", 2024-09-14T06:32:03.803 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:03.619407+0000", 2024-09-14T06:32:03.803 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.1282e-05 2024-09-14T06:32:03.804 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:03.804 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:32:03.804 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:03.804 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:03.804 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:03.804 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:228131 3.10 3.5fa61c10 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e199)", 2024-09-14T06:32:03.804 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:32:03.616234+0000", 2024-09-14T06:32:03.804 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 0.18251835599999999, 2024-09-14T06:32:03.804 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.1826007, 2024-09-14T06:32:03.804 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:32:03.804 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:32:03.804 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "waiting for peered", 2024-09-14T06:32:03.804 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:32:03.804 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:32:03.805 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:03.805 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 228131 2024-09-14T06:32:03.805 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:03.805 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:32:03.805 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:03.805 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:32:03.805 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:03.616234+0000", 2024-09-14T06:32:03.805 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:03.805 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:03.805 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:03.805 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:32:03.805 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:03.616234+0000", 2024-09-14T06:32:03.805 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:03.806 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:03.806 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:03.806 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:32:03.806 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:03.616235+0000", 2024-09-14T06:32:03.806 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.1739999999999999e-06 2024-09-14T06:32:03.806 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:03.806 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:03.806 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:32:03.806 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:03.616242+0000", 2024-09-14T06:32:03.806 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.0639999999999996e-06 2024-09-14T06:32:03.806 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:03.806 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:03.806 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:32:03.806 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:03.616257+0000", 2024-09-14T06:32:03.806 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.4839e-05 2024-09-14T06:32:03.807 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:03.807 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:03.807 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:32:03.807 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:03.616266+0000", 2024-09-14T06:32:03.807 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.5210000000000002e-06 2024-09-14T06:32:03.807 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:03.807 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:03.807 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:32:03.807 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:03.616331+0000", 2024-09-14T06:32:03.807 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.5247999999999996e-05 2024-09-14T06:32:03.807 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:03.807 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:03.807 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "waiting_for_map not empty", 2024-09-14T06:32:03.807 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:03.616340+0000", 2024-09-14T06:32:03.808 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.8100000000000004e-06 2024-09-14T06:32:03.808 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:03.808 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:03.808 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:32:03.808 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:03.619828+0000", 2024-09-14T06:32:03.808 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.003487923 2024-09-14T06:32:03.808 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:03.808 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:03.808 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "waiting for peered", 2024-09-14T06:32:03.808 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:03.619836+0000", 2024-09-14T06:32:03.808 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.5620000000000004e-06 2024-09-14T06:32:03.808 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:03.808 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:32:03.808 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:03.808 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:03.808 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-09-14T06:32:03.809 INFO:teuthology.orchestra.run.smithi094.stdout: "num_ops": 2 2024-09-14T06:32:03.809 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-14T06:32:03.809 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-14T06:32:03.810 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-14T06:32:03.894 INFO:tasks.ceph.osd.3.smithi094.stderr:2024-09-14T06:32:03.893+0000 7feeeafe8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33655) UID: 0 2024-09-14T06:32:03.956 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-14T06:32:03.956 INFO:teuthology.orchestra.run.smithi094.stdout: "ops": [], 2024-09-14T06:32:03.956 INFO:teuthology.orchestra.run.smithi094.stdout: "complaint_time": 30, 2024-09-14T06:32:03.956 INFO:teuthology.orchestra.run.smithi094.stdout: "num_blocked_ops": 0 2024-09-14T06:32:03.956 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-14T06:32:03.957 INFO:tasks.ceph.osd.0.smithi094.stderr:2024-09-14T06:32:03.955+0000 7f07c9333640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33636) UID: 0 2024-09-14T06:32:03.965 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-14T06:32:03.971 INFO:teuthology.orchestra.run.smithi094.stdout:osd.0: {} 2024-09-14T06:32:03.972 INFO:teuthology.orchestra.run.smithi094.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-14T06:32:03.989 INFO:teuthology.orchestra.run.smithi094.stdout:osd.1: {} 2024-09-14T06:32:03.989 INFO:teuthology.orchestra.run.smithi094.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-14T06:32:04.005 INFO:teuthology.orchestra.run.smithi094.stdout:osd.2: {} 2024-09-14T06:32:04.005 INFO:teuthology.orchestra.run.smithi094.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-14T06:32:04.021 INFO:teuthology.orchestra.run.smithi094.stdout:osd.3: {} 2024-09-14T06:32:04.021 INFO:teuthology.orchestra.run.smithi094.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-14T06:32:04.038 INFO:teuthology.orchestra.run.smithi094.stdout:osd.4: {} 2024-09-14T06:32:04.039 INFO:teuthology.orchestra.run.smithi094.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-14T06:32:04.055 INFO:teuthology.orchestra.run.smithi094.stdout:osd.5: {} 2024-09-14T06:32:04.055 INFO:teuthology.orchestra.run.smithi094.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-14T06:32:04.057 INFO:tasks.ceph.osd.0.smithi094.stderr:2024-09-14T06:32:04.056+0000 7f07c9333640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33636) UID: 0 2024-09-14T06:32:04.072 INFO:teuthology.orchestra.run.smithi094.stdout:osd.6: {} 2024-09-14T06:32:04.072 INFO:teuthology.orchestra.run.smithi094.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-14T06:32:04.076 INFO:teuthology.orchestra.run.smithi094.stdout:ERROR: (22) Invalid argument 2024-09-14T06:32:04.076 INFO:teuthology.orchestra.run.smithi094.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:32:04.085 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-14T06:32:04.089 INFO:teuthology.orchestra.run.smithi094.stdout:osd.7: {} 2024-09-14T06:32:04.090 INFO:teuthology.orchestra.run.smithi094.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-14T06:32:04.158 INFO:tasks.ceph.osd.4.smithi138.stderr:2024-09-14T06:32:04.157+0000 7f6c653c8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33615) UID: 0 2024-09-14T06:32:04.193 INFO:teuthology.orchestra.run.smithi094.stdout:ERROR: (22) Invalid argument 2024-09-14T06:32:04.193 INFO:teuthology.orchestra.run.smithi094.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:32:04.202 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-14T06:32:04.259 INFO:tasks.ceph.osd.0.smithi094.stderr:2024-09-14T06:32:04.258+0000 7f07c9333640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33636) UID: 0 2024-09-14T06:32:04.304 INFO:teuthology.orchestra.run.smithi094.stdout:ERROR: (22) Invalid argument 2024-09-14T06:32:04.304 INFO:teuthology.orchestra.run.smithi094.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:32:04.314 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-14T06:32:04.398 INFO:tasks.ceph.osd.1.smithi094.stderr:2024-09-14T06:32:04.397+0000 7f328e47a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33641) UID: 0 2024-09-14T06:32:04.456 INFO:teuthology.orchestra.run.smithi094.stdout:ERROR: (22) Invalid argument 2024-09-14T06:32:04.456 INFO:teuthology.orchestra.run.smithi094.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:32:04.460 INFO:tasks.ceph.osd.6.smithi138.stderr:2024-09-14T06:32:04.460+0000 7f8cd8849640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33620) UID: 0 2024-09-14T06:32:04.464 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-14T06:32:04.562 INFO:tasks.ceph.osd.7.smithi138.stderr:2024-09-14T06:32:04.561+0000 7f3c6a21d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33623) UID: 0 2024-09-14T06:32:04.609 INFO:teuthology.orchestra.run.smithi094.stdout:ERROR: (22) Invalid argument 2024-09-14T06:32:04.609 INFO:teuthology.orchestra.run.smithi094.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:32:04.618 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-14T06:32:04.663 INFO:tasks.ceph.osd.2.smithi094.stderr:2024-09-14T06:32:04.662+0000 7faf3269b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33649) UID: 0 2024-09-14T06:32:04.728 INFO:teuthology.orchestra.run.smithi094.stdout:ERROR: (22) Invalid argument 2024-09-14T06:32:04.728 INFO:teuthology.orchestra.run.smithi094.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:32:04.737 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-14T06:32:04.739 INFO:teuthology.orchestra.run.smithi094.stdout:checking clone #42, image image_client.0-clone42 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent43 2024-09-14T06:32:04.763 INFO:tasks.ceph.osd.4.smithi138.stderr:2024-09-14T06:32:04.763+0000 7f6c653c8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33615) UID: 0 2024-09-14T06:32:04.858 INFO:teuthology.orchestra.run.smithi094.stdout:ERROR: (22) Invalid argument 2024-09-14T06:32:04.858 INFO:teuthology.orchestra.run.smithi094.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:32:04.865 INFO:tasks.ceph.osd.6.smithi138.stderr:2024-09-14T06:32:04.864+0000 7f8cd8849640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33620) UID: 0 2024-09-14T06:32:04.868 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-14T06:32:04.945 INFO:teuthology.orchestra.run.smithi094.stdout:5705 writesame 0x3066c67 thru 0x3071c3e data_size 0x44(0xafd8 bytes) 2024-09-14T06:32:04.952 INFO:teuthology.orchestra.run.smithi094.stdout:5707 read 0x25af45a thru 0x25b838f (0x8f36 bytes) 2024-09-14T06:32:04.952 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-14T06:32:04.958 INFO:teuthology.orchestra.run.smithi094.stdout:5711 write 0xadb5bf0 thru 0xadba3b8 (0x47c9 bytes) 2024-09-14T06:32:04.965 INFO:tasks.ceph.osd.5.smithi138.stderr:2024-09-14T06:32:04.964+0000 7fe13746f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33618) UID: 0 2024-09-14T06:32:04.967 INFO:teuthology.orchestra.run.smithi094.stdout:5714 writesame 0xbf8b6ac thru 0xbf98b64 data_size 0x1(0xd4b9 bytes) 2024-09-14T06:32:04.967 INFO:teuthology.orchestra.run.smithi094.stdout:5716 read 0x9a49531 thru 0x9a55a51 (0xc521 bytes) 2024-09-14T06:32:04.967 INFO:teuthology.orchestra.run.smithi094.stdout:5717 writesame 0x7e63fef thru 0x7e6a780 data_size 0x1b(0x6792 bytes) 2024-09-14T06:32:04.971 INFO:teuthology.orchestra.run.smithi094.stdout:5718 write 0x3d78c01 thru 0x3d88352 (0xf752 bytes) 2024-09-14T06:32:04.973 INFO:teuthology.orchestra.run.smithi094.stdout:5719 punch from 0x6e7c481 to 0x6e81f4d, (0x5acc bytes) 2024-09-14T06:32:04.973 INFO:teuthology.orchestra.run.smithi094.stdout:5721 read 0x55e6586 thru 0x55f5799 (0xf214 bytes) 2024-09-14T06:32:04.973 INFO:teuthology.orchestra.run.smithi094.stdout:5722 read 0x864123a thru 0x86470e6 (0x5ead bytes) 2024-09-14T06:32:04.973 INFO:teuthology.orchestra.run.smithi094.stdout:5723 read 0x3f61d20 thru 0x3f63c6a (0x1f4b bytes) 2024-09-14T06:32:04.974 INFO:teuthology.orchestra.run.smithi094.stdout:5724 write 0x914c1a0 thru 0x9155b8a (0x99eb bytes) 2024-09-14T06:32:04.976 INFO:teuthology.orchestra.run.smithi094.stdout:5726 read 0x949fa3d thru 0x94a8a3b (0x8fff bytes) 2024-09-14T06:32:04.976 INFO:teuthology.orchestra.run.smithi094.stdout:5729 read 0x84e6394 thru 0x84f131a (0xaf87 bytes) 2024-09-14T06:32:04.976 INFO:teuthology.orchestra.run.smithi094.stdout:5730 writesame 0x76feddb thru 0x770e688 data_size 0x2(0xf8ae bytes) 2024-09-14T06:32:04.977 INFO:teuthology.orchestra.run.smithi094.stdout:ERROR: (22) Invalid argument 2024-09-14T06:32:04.977 INFO:teuthology.orchestra.run.smithi094.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:32:04.986 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-14T06:32:04.989 INFO:teuthology.orchestra.run.smithi094.stdout:5731 writesame 0xb4e5bb6 thru 0xb4eb04d data_size 0x4(0x5498 bytes) 2024-09-14T06:32:04.995 INFO:teuthology.orchestra.run.smithi094.stdout:5734 write 0x59efcc8 thru 0x59f65ef (0x6928 bytes) 2024-09-14T06:32:04.999 INFO:teuthology.orchestra.run.smithi094.stdout:5735 write 0x21d3957 thru 0x21e1f7d (0xe627 bytes) 2024-09-14T06:32:05.001 INFO:teuthology.orchestra.run.smithi094.stdout:5736 punch from 0x985d82e to 0x9863a92, (0x6264 bytes) 2024-09-14T06:32:05.001 INFO:teuthology.orchestra.run.smithi094.stdout:5737 read 0x3afe8d8 thru 0x3b072d5 (0x89fe bytes) 2024-09-14T06:32:05.001 INFO:teuthology.orchestra.run.smithi094.stdout:5739 write 0x448e975 thru 0x449a5d5 (0xbc61 bytes) 2024-09-14T06:32:05.014 INFO:teuthology.orchestra.run.smithi094.stdout:5740 trunc from 0xbf98b65 to 0x2b72b79 2024-09-14T06:32:05.033 INFO:teuthology.orchestra.run.smithi094.stdout:5741 punch from 0x4e3e0e to 0x4e6d1b, (0x2f0d bytes) 2024-09-14T06:32:05.043 INFO:teuthology.orchestra.run.smithi094.stdout:5742 writesame 0x63f5c7e thru 0x6401cd4 data_size 0x3(0xc057 bytes) 2024-09-14T06:32:05.043 INFO:teuthology.orchestra.run.smithi094.stdout:5743 write 0x634034 thru 0x635b7b (0x1b48 bytes) 2024-09-14T06:32:05.045 INFO:teuthology.orchestra.run.smithi094.stdout:5744 read 0x34a0485 thru 0x34a5938 (0x54b4 bytes) 2024-09-14T06:32:05.045 INFO:teuthology.orchestra.run.smithi094.stdout:5745 write 0x299918c thru 0x29a3318 (0xa18d bytes) 2024-09-14T06:32:05.048 INFO:teuthology.orchestra.run.smithi094.stdout:5746 read 0x25217bb thru 0x252df51 (0xc797 bytes) 2024-09-14T06:32:05.048 INFO:teuthology.orchestra.run.smithi094.stdout:5747 read 0x63f37a7 thru 0x63f8b29 (0x5383 bytes) 2024-09-14T06:32:05.055 INFO:teuthology.orchestra.run.smithi094.stdout:5749 write 0x8b01062 thru 0x8b02a16 (0x19b5 bytes) 2024-09-14T06:32:05.059 INFO:teuthology.orchestra.run.smithi094.stdout:5751 punch from 0x1f3d0a8 to 0x1f48e0d, (0xbd65 bytes) 2024-09-14T06:32:05.059 INFO:teuthology.orchestra.run.smithi094.stdout:5752 write 0x2f83ca4 thru 0x2f85c10 (0x1f6d bytes) 2024-09-14T06:32:05.061 INFO:teuthology.orchestra.run.smithi094.stdout:5753 read 0x6d6f4c4 thru 0x6d7108b (0x1bc8 bytes) 2024-09-14T06:32:05.061 INFO:teuthology.orchestra.run.smithi094.stdout:5754 read 0x7afc834 thru 0x7b08331 (0xbafe bytes) 2024-09-14T06:32:05.066 INFO:tasks.ceph.osd.2.smithi094.stderr:2024-09-14T06:32:05.065+0000 7faf3269b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33649) UID: 0 2024-09-14T06:32:05.067 INFO:teuthology.orchestra.run.smithi094.stdout:5755 trunc from 0x8b02a17 to 0x381f873 2024-09-14T06:32:05.079 INFO:teuthology.orchestra.run.smithi094.stdout:5756 read 0xf25b10 thru 0xf295a7 (0x3a98 bytes) 2024-09-14T06:32:05.079 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-14T06:32:05.095 INFO:teuthology.orchestra.run.smithi094.stdout:5758 write 0xcc24882 thru 0xcc2549d (0xc1c bytes) 2024-09-14T06:32:05.106 INFO:teuthology.orchestra.run.smithi094.stdout:5760 write 0xe66f31f thru 0xe67b60e (0xc2f0 bytes) 2024-09-14T06:32:05.106 INFO:teuthology.orchestra.run.smithi094.stdout:ERROR: (22) Invalid argument 2024-09-14T06:32:05.107 INFO:teuthology.orchestra.run.smithi094.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:32:05.109 INFO:teuthology.orchestra.run.smithi094.stdout:5761 punch from 0x9db9e77 to 0x9dc42f6, (0xa47f bytes) 2024-09-14T06:32:05.110 INFO:teuthology.orchestra.run.smithi094.stdout:5762 read 0xd557bb6 thru 0xd56104f (0x949a bytes) 2024-09-14T06:32:05.110 INFO:teuthology.orchestra.run.smithi094.stdout:5763 read 0xbd6876a thru 0xbd6f7a6 (0x703d bytes) 2024-09-14T06:32:05.110 INFO:teuthology.orchestra.run.smithi094.stdout:5764 punch from 0xcf4ced8 to 0xcf4d93c, (0xa64 bytes) 2024-09-14T06:32:05.110 INFO:teuthology.orchestra.run.smithi094.stdout:5765 read 0x3733071 thru 0x373906e (0x5ffe bytes) 2024-09-14T06:32:05.110 INFO:teuthology.orchestra.run.smithi094.stdout:5766 read 0x103d793 thru 0x1047c79 (0xa4e7 bytes) 2024-09-14T06:32:05.110 INFO:teuthology.orchestra.run.smithi094.stdout:5769 writesame 0xac6e3d3 thru 0xac76577 data_size 0x45(0x81a5 bytes) 2024-09-14T06:32:05.113 INFO:teuthology.orchestra.run.smithi094.stdout:5770 writesame 0x12f5bed thru 0x13056bb data_size 0xd(0xfacf bytes) 2024-09-14T06:32:05.117 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-14T06:32:05.133 INFO:teuthology.orchestra.run.smithi094.stdout:5771 trunc from 0xe67b60f to 0x3b068a 2024-09-14T06:32:05.150 INFO:teuthology.orchestra.run.smithi094.stdout:5773 read 0x1c59f7 thru 0x1ccfed (0x75f7 bytes) 2024-09-14T06:32:05.162 INFO:teuthology.orchestra.run.smithi094.stdout:5774 writesame 0x4f522ed thru 0x4f5812f data_size 0x1(0x5e43 bytes) 2024-09-14T06:32:05.162 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-14T06:32:05.162 INFO:teuthology.orchestra.run.smithi094.stdout:5778 read 0x472bf8e thru 0x4736651 (0xa6c4 bytes) 2024-09-14T06:32:05.166 INFO:tasks.ceph.osd.0.smithi094.stderr:2024-09-14T06:32:05.165+0000 7f07c9333640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33636) UID: 0 2024-09-14T06:32:05.174 INFO:teuthology.orchestra.run.smithi094.stdout:5779 write 0xb35e3c3 thru 0xb366a51 (0x868f bytes) 2024-09-14T06:32:05.178 INFO:teuthology.orchestra.run.smithi094.stdout:5780 writesame 0x2839587 thru 0x283ee90 data_size 0x6(0x590a bytes) 2024-09-14T06:32:05.183 INFO:teuthology.orchestra.run.smithi094.stdout:5781 punch from 0x9748ea4 to 0x9757365, (0xe4c1 bytes) 2024-09-14T06:32:05.183 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-14T06:32:05.183 INFO:teuthology.orchestra.run.smithi094.stdout:5783 read 0x23eddf5 thru 0x23f867b (0xa887 bytes) 2024-09-14T06:32:05.187 INFO:teuthology.orchestra.run.smithi094.stdout:5786 trunc from 0xb366a52 to 0xed88495 2024-09-14T06:32:05.192 INFO:teuthology.orchestra.run.smithi094.stdout:5789 writesame 0x535b021 thru 0x536531c data_size 0x6(0xa2fc bytes) 2024-09-14T06:32:05.198 INFO:teuthology.orchestra.run.smithi094.stdout:5790 read 0xcf57817 thru 0xcf5b11c (0x3906 bytes) 2024-09-14T06:32:05.198 INFO:teuthology.orchestra.run.smithi094.stdout:5792 writesame 0xada4b2 thru 0xae82fd data_size 0x29(0xde4c bytes) 2024-09-14T06:32:05.200 INFO:teuthology.orchestra.run.smithi094.stdout:5793 punch from 0x9037dd4 to 0x903b0c9, (0x32f5 bytes) 2024-09-14T06:32:05.201 INFO:teuthology.orchestra.run.smithi094.stdout:5795 writesame 0x1ca3282 thru 0x1cad47f data_size 0xd(0xa1fe bytes) 2024-09-14T06:32:05.205 INFO:teuthology.orchestra.run.smithi094.stdout:5796 punch from 0x2e02933 to 0x2e05a82, (0x314f bytes) 2024-09-14T06:32:05.205 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-14T06:32:05.205 INFO:teuthology.orchestra.run.smithi094.stdout:5799 writesame 0xe3dc526 thru 0xe3deb7a data_size 0x1(0x2655 bytes) 2024-09-14T06:32:05.212 INFO:teuthology.orchestra.run.smithi094.stdout:5800 punch from 0x64ee2f3 to 0x64f35ff, (0x530c bytes) 2024-09-14T06:32:05.212 INFO:teuthology.orchestra.run.smithi094.stdout:5801 writesame 0x580b714 thru 0x580ef8d data_size 0x1(0x387a bytes) 2024-09-14T06:32:05.221 INFO:teuthology.orchestra.run.smithi094.stdout:5804 write 0x18c118e thru 0x18c656f (0x53e2 bytes) 2024-09-14T06:32:05.224 INFO:teuthology.orchestra.run.smithi094.stdout:5805 read 0x568f15 thru 0x571416 (0x8502 bytes) 2024-09-14T06:32:05.228 INFO:teuthology.orchestra.run.smithi094.stdout:5806 trunc from 0xed88495 to 0xc04e236 2024-09-14T06:32:05.238 INFO:teuthology.orchestra.run.smithi094.stdout:5808 write 0x758a479 thru 0x758a8ad (0x435 bytes) 2024-09-14T06:32:05.241 INFO:teuthology.orchestra.run.smithi094.stdout:5809 writesame 0x79d9525 thru 0x79e3c48 data_size 0x26(0xa724 bytes) 2024-09-14T06:32:05.245 INFO:teuthology.orchestra.run.smithi094.stdout:5813 punch from 0x9dfffd9 to 0x9e0b13b, (0xb162 bytes) 2024-09-14T06:32:05.245 INFO:teuthology.orchestra.run.smithi094.stdout:5814 read 0x9482a8f thru 0x948ceed (0xa45f bytes) 2024-09-14T06:32:05.246 INFO:teuthology.orchestra.run.smithi094.stdout:5815 read 0x1d61538 thru 0x1d68093 (0x6b5c bytes) 2024-09-14T06:32:05.249 INFO:teuthology.orchestra.run.smithi094.stdout:ERROR: (22) Invalid argument 2024-09-14T06:32:05.249 INFO:teuthology.orchestra.run.smithi094.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:32:05.254 INFO:teuthology.orchestra.run.smithi094.stdout:5816 trunc from 0xc04e236 to 0x4cb9532 2024-09-14T06:32:05.262 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-14T06:32:05.273 INFO:teuthology.orchestra.run.smithi094.stdout:5817 writesame 0x6a020a1 thru 0x6a0737f data_size 0x1(0x52df bytes) 2024-09-14T06:32:05.273 INFO:tasks.ceph.osd.2.smithi094.stderr:2024-09-14T06:32:05.272+0000 7faf3269b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33649) UID: 0 2024-09-14T06:32:05.283 INFO:teuthology.orchestra.run.smithi094.stdout:5818 writesame 0xa2264a8 thru 0xa23420c data_size 0x13(0xdd65 bytes) 2024-09-14T06:32:05.283 INFO:teuthology.orchestra.run.smithi094.stdout:5819 punch from 0x9b9213c to 0x9b9eb00, (0xc9c4 bytes) 2024-09-14T06:32:05.292 INFO:teuthology.orchestra.run.smithi094.stdout:5820 writesame 0xd9955cc thru 0xd998753 data_size 0x14(0x3188 bytes) 2024-09-14T06:32:05.293 INFO:teuthology.orchestra.run.smithi094.stdout:5822 trunc from 0xd998754 to 0xed2dce1 2024-09-14T06:32:05.298 INFO:teuthology.orchestra.run.smithi094.stdout:5824 punch from 0x4edca56 to 0x4ee4802, (0x7dac bytes) 2024-09-14T06:32:05.299 INFO:teuthology.orchestra.run.smithi094.stdout:5825 read 0x2fcb7f5 thru 0x2fd249c (0x6ca8 bytes) 2024-09-14T06:32:05.299 INFO:teuthology.orchestra.run.smithi094.stdout:5826 write 0xaa1e478 thru 0xaa299e6 (0xb56f bytes) 2024-09-14T06:32:05.304 INFO:teuthology.orchestra.run.smithi094.stdout:5827 trunc from 0xed2dce1 to 0xd31bd14 2024-09-14T06:32:05.309 INFO:teuthology.orchestra.run.smithi094.stdout:5828 write 0xce73703 thru 0xce7dae5 (0xa3e3 bytes) 2024-09-14T06:32:05.315 INFO:teuthology.orchestra.run.smithi094.stdout:5829 trunc from 0xd31bd14 to 0xe62d3a5 2024-09-14T06:32:05.320 INFO:teuthology.orchestra.run.smithi094.stdout:5830 write 0x3202e56 thru 0x3211dd8 (0xef83 bytes) 2024-09-14T06:32:05.323 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-14T06:32:05.324 INFO:teuthology.orchestra.run.smithi094.stdout:5833 write 0xd2b16fd thru 0xd2bf7ce (0xe0d2 bytes) 2024-09-14T06:32:05.327 INFO:teuthology.orchestra.run.smithi094.stdout:5834 write 0xb2aa25a thru 0xb2ac3a5 (0x214c bytes) 2024-09-14T06:32:05.335 INFO:teuthology.orchestra.run.smithi094.stdout:5835 trunc from 0xe62d3a5 to 0xa30b37a 2024-09-14T06:32:05.346 INFO:teuthology.orchestra.run.smithi094.stdout:5836 write 0x7b6acc5 thru 0x7b7354c (0x8888 bytes) 2024-09-14T06:32:05.349 INFO:teuthology.orchestra.run.smithi094.stdout:5838 write 0x6ce259 thru 0x6d4d23 (0x6acb bytes) 2024-09-14T06:32:05.359 INFO:teuthology.orchestra.run.smithi094.stdout:5839 writesame 0xccb93c0 thru 0xccc5a82 data_size 0x7(0xc6c3 bytes) 2024-09-14T06:32:05.359 INFO:teuthology.orchestra.run.smithi094.stdout:5841 write 0x48b3e8e thru 0x48b4e16 (0xf89 bytes) 2024-09-14T06:32:05.368 INFO:tasks.ceph.osd.7.smithi138.stderr:2024-09-14T06:32:05.367+0000 7f3c6a21d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33623) UID: 0 2024-09-14T06:32:05.369 INFO:teuthology.orchestra.run.smithi094.stdout:5842 trunc from 0xccc5a83 to 0x66e4b5d 2024-09-14T06:32:05.372 INFO:teuthology.orchestra.run.smithi138.stdout:ERROR: (22) Invalid argument 2024-09-14T06:32:05.373 INFO:teuthology.orchestra.run.smithi138.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:32:05.382 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-14T06:32:05.387 INFO:teuthology.orchestra.run.smithi094.stdout:5845 write 0x86317dd thru 0x8635b32 (0x4356 bytes) 2024-09-14T06:32:05.398 INFO:teuthology.orchestra.run.smithi094.stdout:5847 write 0xb70e0c9 thru 0xb717c5b (0x9b93 bytes) 2024-09-14T06:32:05.402 INFO:teuthology.orchestra.run.smithi094.stdout:5849 read 0x736f710 thru 0x7379d21 (0xa612 bytes) 2024-09-14T06:32:05.405 INFO:teuthology.orchestra.run.smithi094.stdout:5850 trunc from 0xb717c5c to 0xe95f8b8 2024-09-14T06:32:05.412 INFO:teuthology.orchestra.run.smithi094.stdout:5851 write 0xdbcef02 thru 0xdbd8ccc (0x9dcb bytes) 2024-09-14T06:32:05.417 INFO:teuthology.orchestra.run.smithi094.stdout:5852 read 0x63ff0be thru 0x6407aae (0x89f1 bytes) 2024-09-14T06:32:05.417 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-14T06:32:05.417 INFO:teuthology.orchestra.run.smithi094.stdout:5854 read 0x9132d19 thru 0x914079d (0xda85 bytes) 2024-09-14T06:32:05.417 INFO:teuthology.orchestra.run.smithi094.stdout:5855 write 0x8f83a34 thru 0x8f911f7 (0xd7c4 bytes) 2024-09-14T06:32:05.420 INFO:teuthology.orchestra.run.smithi094.stdout:5858 read 0x1df3588 thru 0x1dfb30d (0x7d86 bytes) 2024-09-14T06:32:05.425 INFO:teuthology.orchestra.run.smithi094.stdout:5859 write 0xe992d28 thru 0xe9a0ceb (0xdfc4 bytes) 2024-09-14T06:32:05.429 INFO:teuthology.orchestra.run.smithi094.stdout:5861 read 0x7367256 thru 0x736b878 (0x4623 bytes) 2024-09-14T06:32:05.430 INFO:teuthology.orchestra.run.smithi094.stdout:5862 writesame 0xc5d403a thru 0xc5e0eeb data_size 0x3(0xceb2 bytes) 2024-09-14T06:32:05.448 INFO:teuthology.orchestra.run.smithi094.stdout:5865 trunc from 0xe9a0cec to 0x57fde35 2024-09-14T06:32:05.461 INFO:teuthology.orchestra.run.smithi094.stdout:5867 write 0x317f969 thru 0x31854c9 (0x5b61 bytes) 2024-09-14T06:32:05.470 INFO:tasks.ceph.osd.5.smithi138.stderr:2024-09-14T06:32:05.469+0000 7fe13746f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33618) UID: 0 2024-09-14T06:32:05.470 INFO:teuthology.orchestra.run.smithi094.stdout:5868 write 0x7e0d9ec thru 0x7e14d91 (0x73a6 bytes) 2024-09-14T06:32:05.486 INFO:teuthology.orchestra.run.smithi094.stdout:5869 write 0xd6f190b thru 0xd6ff793 (0xde89 bytes) 2024-09-14T06:32:05.489 INFO:teuthology.orchestra.run.smithi094.stdout:5870 write 0x5459fac thru 0x545c25f (0x22b4 bytes) 2024-09-14T06:32:05.493 INFO:teuthology.orchestra.run.smithi094.stdout:5871 punch from 0x387610b to 0x387d939, (0x782e bytes) 2024-09-14T06:32:05.493 INFO:teuthology.orchestra.run.smithi094.stdout:5873 write 0x20002f0 thru 0x200a713 (0xa424 bytes) 2024-09-14T06:32:05.507 INFO:teuthology.orchestra.run.smithi094.stdout:5874 trunc from 0xd6ff794 to 0x3b18b01 2024-09-14T06:32:05.519 INFO:teuthology.orchestra.run.smithi094.stdout:5875 writesame 0x24917f1 thru 0x2493b67 data_size 0x1(0x2377 bytes) 2024-09-14T06:32:05.525 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-14T06:32:05.531 INFO:teuthology.orchestra.run.smithi138.stdout:ERROR: (22) Invalid argument 2024-09-14T06:32:05.531 INFO:teuthology.orchestra.run.smithi138.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:32:05.533 INFO:teuthology.orchestra.run.smithi094.stdout:5877 write 0x5f27f3d thru 0x5f36a9d (0xeb61 bytes) 2024-09-14T06:32:05.536 INFO:teuthology.orchestra.run.smithi094.stdout:5878 read 0x541cb8f thru 0x542496c (0x7dde bytes) 2024-09-14T06:32:05.540 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-14T06:32:05.547 INFO:teuthology.orchestra.run.smithi094.stdout:5879 write 0xa46b37c thru 0xa475249 (0x9ece bytes) 2024-09-14T06:32:05.556 INFO:teuthology.orchestra.run.smithi094.stdout:5881 trunc from 0xa47524a to 0xe918ef6 2024-09-14T06:32:05.561 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-14T06:32:05.562 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-14T06:32:05.562 INFO:teuthology.orchestra.run.smithi094.stdout:5885 write 0x1ce8b13 thru 0x1ce9c0b (0x10f9 bytes) 2024-09-14T06:32:05.563 INFO:teuthology.orchestra.run.smithi094.stdout:5887 write 0x9e572e1 thru 0x9e5da6e (0x678e bytes) 2024-09-14T06:32:05.566 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-14T06:32:05.571 INFO:tasks.ceph.osd.0.smithi094.stderr:2024-09-14T06:32:05.570+0000 7f07c9333640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33636) UID: 0 2024-09-14T06:32:05.580 INFO:teuthology.orchestra.run.smithi094.stdout:5890 trunc from 0xe918ef6 to 0x2e0648c 2024-09-14T06:32:05.607 INFO:teuthology.orchestra.run.smithi094.stdout:5892 write 0x982e83e thru 0x982eadb (0x29e bytes) 2024-09-14T06:32:05.610 INFO:teuthology.orchestra.run.smithi094.stdout:5894 write 0x2822206 thru 0x28266b9 (0x44b4 bytes) 2024-09-14T06:32:05.612 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-14T06:32:05.612 INFO:teuthology.orchestra.run.smithi094.stdout:5897 read 0x2adf25 thru 0x2b4755 (0x6831 bytes) 2024-09-14T06:32:05.613 INFO:teuthology.orchestra.run.smithi094.stdout:5899 read 0x918ebe1 thru 0x919314b (0x456b bytes) 2024-09-14T06:32:05.613 INFO:teuthology.orchestra.run.smithi094.stdout:5900 punch from 0x25f909d to 0x2608002, (0xef65 bytes) 2024-09-14T06:32:05.613 INFO:teuthology.orchestra.run.smithi094.stdout:5901 read 0x8f47eb9 thru 0x8f52e89 (0xafd1 bytes) 2024-09-14T06:32:05.613 INFO:teuthology.orchestra.run.smithi094.stdout:5902 read 0x1cdfbf6 thru 0x1ceced0 (0xd2db bytes) 2024-09-14T06:32:05.618 INFO:teuthology.orchestra.run.smithi094.stdout:5905 trunc from 0x982eadc to 0xdff669f 2024-09-14T06:32:05.623 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 4, 5, 7, 3] out_osds: [6] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-14T06:32:05.623 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-14T06:32:05.623 INFO:tasks.thrashosds.thrasher:inject_pause on osd.0 2024-09-14T06:32:05.623 INFO:tasks.thrashosds.thrasher:Testing filestore_inject_stall pause injection for duration 3 2024-09-14T06:32:05.623 INFO:tasks.thrashosds.thrasher:Checking after 0, should_be_down=False 2024-09-14T06:32:05.623 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok config set filestore_inject_stall 3 2024-09-14T06:32:05.624 INFO:teuthology.orchestra.run.smithi094.stdout:5906 write 0x4d1ba3f thru 0x4d1d7f7 (0x1db9 bytes) 2024-09-14T06:32:05.636 INFO:teuthology.orchestra.run.smithi094.stdout:5907 trunc from 0xdff669f to 0x696e3e6 2024-09-14T06:32:05.646 INFO:teuthology.orchestra.run.smithi094.stdout:5908 writesame 0x41cb7c3 thru 0x41d71e3 data_size 0x7(0xba21 bytes) 2024-09-14T06:32:05.651 INFO:teuthology.orchestra.run.smithi094.stdout:5909 writesame 0x5d5b237 thru 0x5d68884 data_size 0x1(0xd64e bytes) 2024-09-14T06:32:05.656 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-14T06:32:05.672 INFO:tasks.ceph.osd.7.smithi138.stderr:2024-09-14T06:32:05.671+0000 7f3c6a21d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33623) UID: 0 2024-09-14T06:32:05.681 INFO:teuthology.orchestra.run.smithi094.stdout:5912 write 0x9c4aad7 thru 0x9c597b1 (0xecdb bytes) 2024-09-14T06:32:05.684 INFO:teuthology.orchestra.run.smithi094.stdout:5913 punch from 0x58c6153 to 0x58c877c, (0x2629 bytes) 2024-09-14T06:32:05.684 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-14T06:32:05.684 INFO:teuthology.orchestra.run.smithi094.stdout:5915 writesame 0x76028e2 thru 0x76087d9 data_size 0xc(0x5ef8 bytes) 2024-09-14T06:32:05.688 INFO:teuthology.orchestra.run.smithi138.stdout:ERROR: (22) Invalid argument 2024-09-14T06:32:05.688 INFO:teuthology.orchestra.run.smithi138.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:32:05.694 INFO:teuthology.orchestra.run.smithi094.stdout:5916 write 0xaa3a37a thru 0xaa3defd (0x3b84 bytes) 2024-09-14T06:32:05.698 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-14T06:32:05.698 INFO:teuthology.orchestra.run.smithi094.stdout:5917 read 0x3c5a04d thru 0x3c5ac46 (0xbfa bytes) 2024-09-14T06:32:05.699 INFO:teuthology.orchestra.run.smithi094.stdout:5918 trunc from 0xaa3defe to 0xbd43c55 2024-09-14T06:32:05.704 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-14T06:32:05.707 INFO:teuthology.orchestra.run.smithi094.stdout:5920 trunc from 0xbd43c55 to 0xee1c0dc 2024-09-14T06:32:05.712 INFO:teuthology.orchestra.run.smithi094.stdout:5922 write 0x70ee554 thru 0x70f6f1f (0x89cc bytes) 2024-09-14T06:32:05.716 INFO:teuthology.orchestra.run.smithi094.stdout:5923 writesame 0x424df11 thru 0x425a5ec data_size 0x59(0xc6dc bytes) 2024-09-14T06:32:05.720 INFO:teuthology.orchestra.run.smithi094.stdout:5924 write 0x57afa04 thru 0x57b7c6c (0x8269 bytes) 2024-09-14T06:32:05.729 INFO:teuthology.orchestra.run.smithi094.stdout:5925 trunc from 0xee1c0dc to 0x865ff4c 2024-09-14T06:32:05.746 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-14T06:32:05.747 INFO:teuthology.orchestra.run.smithi094.stdout: "success": "" 2024-09-14T06:32:05.747 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-14T06:32:05.751 INFO:teuthology.orchestra.run.smithi094.stdout:5926 writesame 0xdd91595 thru 0xdd95dfe data_size 0x2e(0x486a bytes) 2024-09-14T06:32:05.752 INFO:teuthology.orchestra.run.smithi094.stdout:5927 writesame 0xdc3999b thru 0xdc41e27 data_size 0x1(0x848d bytes) 2024-09-14T06:32:05.767 INFO:teuthology.orchestra.run.smithi094.stdout:5928 read 0x4576fd5 thru 0x4580faa (0x9fd6 bytes) 2024-09-14T06:32:05.768 INFO:teuthology.orchestra.run.smithi094.stdout:5929 punch from 0x6778c60 to 0x677b8b1, (0x2c51 bytes) 2024-09-14T06:32:05.768 INFO:teuthology.orchestra.run.smithi094.stdout:5930 punch from 0x86ec5b0 to 0x86ede0e, (0x185e bytes) 2024-09-14T06:32:05.768 INFO:teuthology.orchestra.run.smithi094.stdout:5932 punch from 0xb4a9bb to 0xb4dced, (0x3332 bytes) 2024-09-14T06:32:05.768 INFO:teuthology.orchestra.run.smithi094.stdout:5933 write 0x679ca0c thru 0x67a4805 (0x7dfa bytes) 2024-09-14T06:32:05.769 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-14T06:32:05.769 INFO:teuthology.orchestra.run.smithi094.stdout:5935 read 0x50d916f thru 0x50db936 (0x27c8 bytes) 2024-09-14T06:32:05.770 INFO:teuthology.orchestra.run.smithi094.stdout:5936 writesame 0x97380be thru 0x974365f data_size 0x2(0xb5a2 bytes) 2024-09-14T06:32:05.783 INFO:tasks.ceph.osd.6.smithi138.stderr:2024-09-14T06:32:05.782+0000 7f8cd8849640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33620) UID: 0 2024-09-14T06:32:05.797 INFO:teuthology.orchestra.run.smithi094.stdout:5937 trunc from 0xdd95dff to 0x615682 2024-09-14T06:32:05.813 INFO:teuthology.orchestra.run.smithi094.stdout:5938 punch from 0x39fbec to 0x3ae7af, (0xebc3 bytes) 2024-09-14T06:32:05.827 INFO:teuthology.orchestra.run.smithi094.stdout:5939 write 0x76833ab thru 0x768e083 (0xacd9 bytes) 2024-09-14T06:32:05.830 INFO:teuthology.orchestra.run.smithi094.stdout:5940 punch from 0xf553f0 to 0xf64215, (0xee25 bytes) 2024-09-14T06:32:05.830 INFO:teuthology.orchestra.run.smithi094.stdout:5941 read 0x35341f2 thru 0x353f8f2 (0xb701 bytes) 2024-09-14T06:32:05.838 INFO:teuthology.orchestra.run.smithi094.stdout:5945 writesame 0xa5c7c30 thru 0xa5d2e6c data_size 0x1(0xb23d bytes) 2024-09-14T06:32:05.838 INFO:teuthology.orchestra.run.smithi094.stdout:5946 writesame 0x851bca3 thru 0x8527be4 data_size 0x1(0xbf42 bytes) 2024-09-14T06:32:05.847 INFO:teuthology.orchestra.run.smithi138.stdout:ERROR: (22) Invalid argument 2024-09-14T06:32:05.847 INFO:teuthology.orchestra.run.smithi138.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:32:05.856 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-14T06:32:05.858 INFO:teuthology.orchestra.run.smithi094.stdout:5948 write 0x10fae1a thru 0x1105920 (0xab07 bytes) 2024-09-14T06:32:05.863 INFO:teuthology.orchestra.run.smithi094.stdout:5949 read 0x2dc12ef thru 0x2dc6272 (0x4f84 bytes) 2024-09-14T06:32:05.863 INFO:teuthology.orchestra.run.smithi094.stdout:5950 read 0x139847c thru 0x139f28c (0x6e11 bytes) 2024-09-14T06:32:05.873 INFO:tasks.ceph.osd.2.smithi094.stderr:2024-09-14T06:32:05.872+0000 7faf3269b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33649) UID: 0 2024-09-14T06:32:05.873 INFO:teuthology.orchestra.run.smithi094.stdout:5951 writesame 0xd83dba0 thru 0xd841eda data_size 0x1(0x433b bytes) 2024-09-14T06:32:05.873 INFO:teuthology.orchestra.run.smithi094.stdout:5953 clone 45 order 18 su 32768 sc 3 2024-09-14T06:32:05.974 INFO:tasks.ceph.osd.2.smithi094.stderr:2024-09-14T06:32:05.973+0000 7faf3269b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33649) UID: 0 2024-09-14T06:32:05.998 INFO:teuthology.orchestra.run.smithi138.stdout:ERROR: (22) Invalid argument 2024-09-14T06:32:05.998 INFO:teuthology.orchestra.run.smithi138.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:32:06.007 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-14T06:32:06.075 INFO:tasks.ceph.osd.4.smithi138.stderr:2024-09-14T06:32:06.074+0000 7f6c653c8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33615) UID: 0 2024-09-14T06:32:06.101 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-14T06:32:06.109 INFO:teuthology.orchestra.run.smithi138.stdout:ERROR: (22) Invalid argument 2024-09-14T06:32:06.109 INFO:teuthology.orchestra.run.smithi138.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:32:06.117 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-14T06:32:06.201 INFO:tasks.ceph.osd.4.smithi138.stderr:2024-09-14T06:32:06.200+0000 7f6c653c8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33615) UID: 0 2024-09-14T06:32:06.220 INFO:teuthology.orchestra.run.smithi094.stdout:osd.0: {} 2024-09-14T06:32:06.292 INFO:teuthology.orchestra.run.smithi094.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-14T06:32:06.293 INFO:teuthology.orchestra.run.smithi094.stdout:osd.1: {} 2024-09-14T06:32:06.293 INFO:teuthology.orchestra.run.smithi094.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-14T06:32:06.294 INFO:teuthology.orchestra.run.smithi094.stdout:osd.2: {} 2024-09-14T06:32:06.294 INFO:teuthology.orchestra.run.smithi094.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-14T06:32:06.294 INFO:teuthology.orchestra.run.smithi094.stdout:osd.3: {} 2024-09-14T06:32:06.294 INFO:teuthology.orchestra.run.smithi094.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-14T06:32:06.295 INFO:teuthology.orchestra.run.smithi094.stdout:osd.4: {} 2024-09-14T06:32:06.295 INFO:teuthology.orchestra.run.smithi094.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-14T06:32:06.296 INFO:teuthology.orchestra.run.smithi138.stdout:ERROR: (22) Invalid argument 2024-09-14T06:32:06.296 INFO:teuthology.orchestra.run.smithi138.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:32:06.297 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-14T06:32:06.297 INFO:tasks.ceph.osd.6.smithi138.stderr:2024-09-14T06:32:06.295+0000 7f8cd8849640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33620) UID: 0 2024-09-14T06:32:06.301 INFO:teuthology.orchestra.run.smithi094.stdout:osd.5: {} 2024-09-14T06:32:06.302 INFO:teuthology.orchestra.run.smithi094.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-14T06:32:06.318 INFO:teuthology.orchestra.run.smithi094.stdout:osd.6: {} 2024-09-14T06:32:06.318 INFO:teuthology.orchestra.run.smithi094.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-14T06:32:06.334 INFO:teuthology.orchestra.run.smithi094.stdout:osd.7: {} 2024-09-14T06:32:06.334 INFO:teuthology.orchestra.run.smithi094.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-14T06:32:06.397 INFO:tasks.ceph.osd.0.smithi094.stderr:2024-09-14T06:32:06.396+0000 7f07c9333640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33636) UID: 0 2024-09-14T06:32:06.438 INFO:teuthology.orchestra.run.smithi138.stdout:{ 2024-09-14T06:32:06.438 INFO:teuthology.orchestra.run.smithi138.stdout: "ops": [], 2024-09-14T06:32:06.438 INFO:teuthology.orchestra.run.smithi138.stdout: "complaint_time": 30, 2024-09-14T06:32:06.438 INFO:teuthology.orchestra.run.smithi138.stdout: "num_blocked_ops": 0 2024-09-14T06:32:06.438 INFO:teuthology.orchestra.run.smithi138.stdout:} 2024-09-14T06:32:06.447 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-14T06:32:06.497 INFO:tasks.ceph.osd.7.smithi138.stderr:2024-09-14T06:32:06.496+0000 7f3c6a21d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33623) UID: 0 2024-09-14T06:32:06.551 INFO:teuthology.orchestra.run.smithi138.stdout:{ 2024-09-14T06:32:06.551 INFO:teuthology.orchestra.run.smithi138.stdout: "size": 20, 2024-09-14T06:32:06.551 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 600, 2024-09-14T06:32:06.551 INFO:teuthology.orchestra.run.smithi138.stdout: "ops": [ 2024-09-14T06:32:06.551 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:06.552 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:98505 3.10 3.1bf56f90 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T06:32:06.552 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:30:07.045453+0000", 2024-09-14T06:32:06.552 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 119.504109484, 2024-09-14T06:32:06.552 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032144440000000003, 2024-09-14T06:32:06.552 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:32:06.552 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:32:06.552 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "reached pg", 2024-09-14T06:32:06.552 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:32:06.552 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:32:06.552 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:06.552 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 98505 2024-09-14T06:32:06.552 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:06.552 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:32:06.552 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:06.552 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:32:06.553 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045453+0000", 2024-09-14T06:32:06.553 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:06.553 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:06.553 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:06.553 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:32:06.553 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045453+0000", 2024-09-14T06:32:06.553 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:06.553 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:06.553 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:06.553 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:32:06.553 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045453+0000", 2024-09-14T06:32:06.553 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.36e-07 2024-09-14T06:32:06.553 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:06.553 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:06.554 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:32:06.554 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045454+0000", 2024-09-14T06:32:06.554 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.1299999999999998e-07 2024-09-14T06:32:06.554 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:06.554 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:06.554 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:32:06.554 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045457+0000", 2024-09-14T06:32:06.554 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.8559999999999998e-06 2024-09-14T06:32:06.554 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:06.554 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:06.554 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:32:06.554 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045460+0000", 2024-09-14T06:32:06.554 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.8750000000000002e-06 2024-09-14T06:32:06.554 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:06.554 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:06.555 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:32:06.555 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.077592+0000", 2024-09-14T06:32:06.555 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032131727999999998 2024-09-14T06:32:06.555 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:06.555 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:06.555 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:32:06.555 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.077597+0000", 2024-09-14T06:32:06.555 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.8319999999999996e-06 2024-09-14T06:32:06.555 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:06.555 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:32:06.555 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:06.555 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:06.555 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:06.555 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:98506 3.10 3.1bf56f90 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T06:32:06.556 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:30:07.045481+0000", 2024-09-14T06:32:06.556 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 119.50408122899999, 2024-09-14T06:32:06.556 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032142759999999999, 2024-09-14T06:32:06.556 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:32:06.556 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:32:06.556 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "reached pg", 2024-09-14T06:32:06.556 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:32:06.556 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:32:06.556 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:06.556 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 98506 2024-09-14T06:32:06.556 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:06.556 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:32:06.556 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:06.556 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:32:06.557 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045481+0000", 2024-09-14T06:32:06.557 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:06.557 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:06.557 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:06.557 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:32:06.557 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045481+0000", 2024-09-14T06:32:06.557 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:06.557 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:06.557 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:06.557 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:32:06.557 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045481+0000", 2024-09-14T06:32:06.557 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.8500000000000001e-07 2024-09-14T06:32:06.557 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:06.558 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:06.558 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:32:06.558 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045483+0000", 2024-09-14T06:32:06.558 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.8250000000000001e-06 2024-09-14T06:32:06.558 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:06.558 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:06.558 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:32:06.558 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045489+0000", 2024-09-14T06:32:06.558 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.5740000000000003e-06 2024-09-14T06:32:06.558 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:06.558 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:06.558 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:32:06.558 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045495+0000", 2024-09-14T06:32:06.558 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.7690000000000002e-06 2024-09-14T06:32:06.558 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:06.559 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:06.559 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:32:06.559 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.077619+0000", 2024-09-14T06:32:06.559 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032124621999999999 2024-09-14T06:32:06.559 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:06.559 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:06.559 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:32:06.559 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.077624+0000", 2024-09-14T06:32:06.559 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.3850000000000004e-06 2024-09-14T06:32:06.559 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:06.559 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:32:06.559 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:06.559 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:06.560 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:06.560 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:98507 3.10 3.1bf56f90 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T06:32:06.560 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:30:07.045502+0000", 2024-09-14T06:32:06.560 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 119.50405980799999, 2024-09-14T06:32:06.560 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032147219999999997, 2024-09-14T06:32:06.560 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:32:06.560 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:32:06.560 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "reached pg", 2024-09-14T06:32:06.560 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:32:06.560 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:32:06.560 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:06.560 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 98507 2024-09-14T06:32:06.560 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:06.560 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:32:06.560 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:06.561 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:32:06.561 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045502+0000", 2024-09-14T06:32:06.561 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:06.561 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:06.561 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:06.561 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:32:06.561 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045502+0000", 2024-09-14T06:32:06.561 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:06.561 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:06.561 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:06.561 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:32:06.561 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045503+0000", 2024-09-14T06:32:06.561 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.4700000000000002e-07 2024-09-14T06:32:06.562 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:06.562 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:06.562 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:32:06.562 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045509+0000", 2024-09-14T06:32:06.562 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.7329999999999997e-06 2024-09-14T06:32:06.562 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:06.562 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:06.562 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:32:06.562 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045515+0000", 2024-09-14T06:32:06.562 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.8030000000000002e-06 2024-09-14T06:32:06.562 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:06.562 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:06.562 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:32:06.562 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045522+0000", 2024-09-14T06:32:06.562 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.0369999999999997e-06 2024-09-14T06:32:06.563 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:06.563 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:06.563 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:32:06.563 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.077641+0000", 2024-09-14T06:32:06.563 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032119091000000002 2024-09-14T06:32:06.563 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:06.563 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:06.563 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:32:06.563 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.077650+0000", 2024-09-14T06:32:06.563 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.2090000000000005e-06 2024-09-14T06:32:06.563 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:06.563 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:32:06.563 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:06.563 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:06.564 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:06.564 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:98508 3.10 3.1bf56f90 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T06:32:06.564 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:30:07.045530+0000", 2024-09-14T06:32:06.564 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 119.50403219099999, 2024-09-14T06:32:06.564 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032198773999999999, 2024-09-14T06:32:06.564 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:32:06.564 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:32:06.564 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "reached pg", 2024-09-14T06:32:06.564 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:32:06.564 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:32:06.564 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:06.564 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 98508 2024-09-14T06:32:06.564 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:06.564 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:32:06.565 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:06.565 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:32:06.565 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045530+0000", 2024-09-14T06:32:06.565 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:06.565 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:06.565 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:06.565 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:32:06.565 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045530+0000", 2024-09-14T06:32:06.565 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:06.565 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:06.565 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:06.565 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:32:06.565 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045530+0000", 2024-09-14T06:32:06.565 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.3099999999999998e-07 2024-09-14T06:32:06.566 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:06.566 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:06.566 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:32:06.566 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045539+0000", 2024-09-14T06:32:06.566 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.4780000000000008e-06 2024-09-14T06:32:06.566 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:06.566 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:06.566 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:32:06.566 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045546+0000", 2024-09-14T06:32:06.566 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.3259999999999998e-06 2024-09-14T06:32:06.566 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:06.566 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:06.566 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:32:06.567 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045551+0000", 2024-09-14T06:32:06.567 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.9960000000000001e-06 2024-09-14T06:32:06.567 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:06.567 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:06.567 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:32:06.567 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.077716+0000", 2024-09-14T06:32:06.567 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032165081999999998 2024-09-14T06:32:06.567 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:06.567 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:06.567 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:32:06.567 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.077729+0000", 2024-09-14T06:32:06.567 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.2461e-05 2024-09-14T06:32:06.567 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:06.568 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:32:06.568 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:06.568 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:06.568 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:06.568 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:98509 3.10 3.1bf56f90 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T06:32:06.568 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:30:07.045558+0000", 2024-09-14T06:32:06.568 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 119.50400399, 2024-09-14T06:32:06.568 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032242198, 2024-09-14T06:32:06.568 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:32:06.568 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:32:06.568 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "reached pg", 2024-09-14T06:32:06.568 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:32:06.568 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:32:06.568 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:06.568 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 98509 2024-09-14T06:32:06.569 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:06.569 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:32:06.569 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:06.569 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:32:06.569 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045558+0000", 2024-09-14T06:32:06.569 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:06.569 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:06.569 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:06.569 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:32:06.569 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045558+0000", 2024-09-14T06:32:06.569 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:06.569 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:06.569 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:06.570 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:32:06.570 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045559+0000", 2024-09-14T06:32:06.570 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.27e-07 2024-09-14T06:32:06.570 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:06.570 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:06.570 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:32:06.570 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045560+0000", 2024-09-14T06:32:06.570 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.6959999999999999e-06 2024-09-14T06:32:06.570 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:06.570 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:06.570 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:32:06.570 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045567+0000", 2024-09-14T06:32:06.570 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.6900000000000003e-06 2024-09-14T06:32:06.570 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:06.570 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:06.571 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:32:06.571 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045571+0000", 2024-09-14T06:32:06.571 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.5750000000000002e-06 2024-09-14T06:32:06.571 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:06.571 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:06.571 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:32:06.571 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.077789+0000", 2024-09-14T06:32:06.571 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032217186000000002 2024-09-14T06:32:06.571 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:06.571 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:06.571 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:32:06.571 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.077800+0000", 2024-09-14T06:32:06.571 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.1623999999999999e-05 2024-09-14T06:32:06.571 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:06.572 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:32:06.572 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:06.572 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:06.572 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:06.572 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:98510 3.10 3.1bf56f90 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T06:32:06.572 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:30:07.045580+0000", 2024-09-14T06:32:06.572 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 119.50398217599999, 2024-09-14T06:32:06.572 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032299845000000001, 2024-09-14T06:32:06.572 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:32:06.572 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:32:06.572 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "reached pg", 2024-09-14T06:32:06.572 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:32:06.572 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:32:06.572 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:06.572 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 98510 2024-09-14T06:32:06.573 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:06.573 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:32:06.573 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:06.573 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:32:06.573 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045580+0000", 2024-09-14T06:32:06.573 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:06.573 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:06.573 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:06.573 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:32:06.573 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045580+0000", 2024-09-14T06:32:06.573 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:06.573 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:06.573 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:06.573 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:32:06.574 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045580+0000", 2024-09-14T06:32:06.574 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.0400000000000002e-07 2024-09-14T06:32:06.574 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:06.574 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:06.574 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:32:06.574 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045582+0000", 2024-09-14T06:32:06.574 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.7600000000000001e-06 2024-09-14T06:32:06.574 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:06.574 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:06.574 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:32:06.574 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045588+0000", 2024-09-14T06:32:06.574 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.5650000000000001e-06 2024-09-14T06:32:06.574 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:06.574 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:06.574 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:32:06.574 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045597+0000", 2024-09-14T06:32:06.575 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.465e-06 2024-09-14T06:32:06.575 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:06.575 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:06.575 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:32:06.575 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.077871+0000", 2024-09-14T06:32:06.575 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032273562999999998 2024-09-14T06:32:06.575 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:06.575 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:06.575 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:32:06.575 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.077880+0000", 2024-09-14T06:32:06.575 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.0880000000000001e-06 2024-09-14T06:32:06.575 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:06.575 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:32:06.575 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:06.575 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:06.576 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:06.576 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:98511 3.10 3.1bf56f90 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T06:32:06.576 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:30:07.045610+0000", 2024-09-14T06:32:06.576 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 119.503952329, 2024-09-14T06:32:06.576 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032338894, 2024-09-14T06:32:06.576 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:32:06.576 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:32:06.576 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "reached pg", 2024-09-14T06:32:06.576 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:32:06.576 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:32:06.576 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:06.576 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 98511 2024-09-14T06:32:06.576 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:06.576 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:32:06.576 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:06.577 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:32:06.577 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045610+0000", 2024-09-14T06:32:06.577 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:06.577 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:06.577 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:06.577 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:32:06.577 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045610+0000", 2024-09-14T06:32:06.577 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:06.577 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:06.577 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:06.577 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:32:06.577 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045610+0000", 2024-09-14T06:32:06.577 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.1600000000000002e-07 2024-09-14T06:32:06.577 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:06.578 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:06.578 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:32:06.578 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045612+0000", 2024-09-14T06:32:06.578 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.1150000000000001e-06 2024-09-14T06:32:06.578 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:06.578 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:06.578 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:32:06.578 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045618+0000", 2024-09-14T06:32:06.578 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.0499999999999997e-06 2024-09-14T06:32:06.578 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:06.578 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:06.578 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:32:06.578 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045623+0000", 2024-09-14T06:32:06.578 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.6770000000000003e-06 2024-09-14T06:32:06.579 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:06.579 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:06.579 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:32:06.579 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.077928+0000", 2024-09-14T06:32:06.579 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032305111999999997 2024-09-14T06:32:06.579 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:06.579 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:06.579 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:32:06.579 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.077949+0000", 2024-09-14T06:32:06.579 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.0523999999999999e-05 2024-09-14T06:32:06.579 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:06.579 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:32:06.579 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:06.579 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:06.580 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:06.580 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:98512 3.10 3.1bf56f90 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T06:32:06.580 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:30:07.045630+0000", 2024-09-14T06:32:06.580 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 119.503932324, 2024-09-14T06:32:06.580 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032346632, 2024-09-14T06:32:06.580 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:32:06.580 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:32:06.580 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "reached pg", 2024-09-14T06:32:06.580 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:32:06.580 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:32:06.580 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:06.580 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 98512 2024-09-14T06:32:06.580 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:06.581 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:32:06.581 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:06.581 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:32:06.581 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045630+0000", 2024-09-14T06:32:06.581 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:06.581 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:06.581 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:06.581 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:32:06.581 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045630+0000", 2024-09-14T06:32:06.581 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:06.581 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:06.581 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:06.581 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:32:06.582 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045630+0000", 2024-09-14T06:32:06.582 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.8200000000000001e-07 2024-09-14T06:32:06.582 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:06.582 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:06.582 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:32:06.582 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045632+0000", 2024-09-14T06:32:06.582 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.7430000000000001e-06 2024-09-14T06:32:06.582 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:06.582 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:06.582 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:32:06.582 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045637+0000", 2024-09-14T06:32:06.582 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.4550000000000003e-06 2024-09-14T06:32:06.582 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:06.582 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:06.582 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:32:06.583 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045644+0000", 2024-09-14T06:32:06.583 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.1330000000000001e-06 2024-09-14T06:32:06.583 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:06.583 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:06.583 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:32:06.583 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.077968+0000", 2024-09-14T06:32:06.583 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032323401000000002 2024-09-14T06:32:06.583 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:06.583 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:06.583 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:32:06.583 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.077976+0000", 2024-09-14T06:32:06.583 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.5180000000000004e-06 2024-09-14T06:32:06.583 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:06.584 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:32:06.584 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:06.584 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:06.584 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:06.584 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:98513 3.10 3.1bf56f90 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T06:32:06.584 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:30:07.045652+0000", 2024-09-14T06:32:06.584 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 119.50390976, 2024-09-14T06:32:06.584 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032353436999999999, 2024-09-14T06:32:06.584 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:32:06.584 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:32:06.584 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "reached pg", 2024-09-14T06:32:06.584 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:32:06.584 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:32:06.585 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:06.585 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 98513 2024-09-14T06:32:06.585 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:06.585 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:32:06.585 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:06.585 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:32:06.585 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045652+0000", 2024-09-14T06:32:06.585 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:06.585 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:06.585 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:06.585 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:32:06.585 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045652+0000", 2024-09-14T06:32:06.585 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:06.586 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:06.586 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:06.586 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:32:06.586 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045653+0000", 2024-09-14T06:32:06.586 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.3700000000000001e-07 2024-09-14T06:32:06.586 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:06.586 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:06.586 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:32:06.586 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045655+0000", 2024-09-14T06:32:06.586 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.854e-06 2024-09-14T06:32:06.586 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:06.586 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:06.586 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:32:06.586 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045661+0000", 2024-09-14T06:32:06.586 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.5509999999999996e-06 2024-09-14T06:32:06.586 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:06.587 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:06.587 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:32:06.587 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045668+0000", 2024-09-14T06:32:06.587 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.1980000000000004e-06 2024-09-14T06:32:06.587 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:06.587 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:06.587 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:32:06.587 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.077997+0000", 2024-09-14T06:32:06.587 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032328575999999998 2024-09-14T06:32:06.587 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:06.587 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:06.587 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:32:06.587 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.078006+0000", 2024-09-14T06:32:06.587 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.8210000000000008e-06 2024-09-14T06:32:06.588 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:06.588 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:32:06.588 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:06.588 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:06.588 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:06.588 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:98514 3.10 3.1bf56f90 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T06:32:06.588 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:30:07.045699+0000", 2024-09-14T06:32:06.588 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 119.50386342500001, 2024-09-14T06:32:06.588 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032330223999999998, 2024-09-14T06:32:06.588 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:32:06.588 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:32:06.588 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "reached pg", 2024-09-14T06:32:06.588 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:32:06.588 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:32:06.589 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:06.589 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 98514 2024-09-14T06:32:06.589 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:06.589 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:32:06.589 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:06.589 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:32:06.589 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045699+0000", 2024-09-14T06:32:06.589 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:06.589 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:06.589 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:06.589 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:32:06.589 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045699+0000", 2024-09-14T06:32:06.589 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:06.590 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:06.590 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:06.590 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:32:06.590 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045699+0000", 2024-09-14T06:32:06.590 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.3200000000000001e-07 2024-09-14T06:32:06.590 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:06.590 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:06.590 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:32:06.590 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045701+0000", 2024-09-14T06:32:06.590 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.762e-06 2024-09-14T06:32:06.590 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:06.590 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:06.590 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:32:06.590 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045705+0000", 2024-09-14T06:32:06.591 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.2359999999999998e-06 2024-09-14T06:32:06.591 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:06.591 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:06.591 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:32:06.591 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045708+0000", 2024-09-14T06:32:06.591 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.796e-06 2024-09-14T06:32:06.591 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:06.591 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:06.591 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:32:06.591 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.078023+0000", 2024-09-14T06:32:06.591 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032315213000000002 2024-09-14T06:32:06.591 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:06.591 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:06.591 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:32:06.592 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.078029+0000", 2024-09-14T06:32:06.592 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.8850000000000004e-06 2024-09-14T06:32:06.592 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:06.592 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:32:06.592 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:06.592 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:06.592 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:06.592 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:98515 3.10 3.1bf56f90 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T06:32:06.592 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:30:07.045712+0000", 2024-09-14T06:32:06.592 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 119.503850114, 2024-09-14T06:32:06.592 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032346871999999999, 2024-09-14T06:32:06.592 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:32:06.592 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:32:06.592 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "reached pg", 2024-09-14T06:32:06.593 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:32:06.593 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:32:06.593 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:06.593 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 98515 2024-09-14T06:32:06.593 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:06.593 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:32:06.593 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:06.593 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:32:06.593 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045712+0000", 2024-09-14T06:32:06.593 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:06.593 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:06.593 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:06.593 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:32:06.593 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045712+0000", 2024-09-14T06:32:06.594 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:06.594 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:06.594 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:06.594 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:32:06.594 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045712+0000", 2024-09-14T06:32:06.594 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.2100000000000001e-07 2024-09-14T06:32:06.594 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:06.594 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:06.594 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:32:06.594 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045713+0000", 2024-09-14T06:32:06.594 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.068e-06 2024-09-14T06:32:06.594 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:06.594 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:06.594 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:32:06.595 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045716+0000", 2024-09-14T06:32:06.595 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.971e-06 2024-09-14T06:32:06.595 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:06.595 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:06.595 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:32:06.595 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045719+0000", 2024-09-14T06:32:06.595 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.3879999999999998e-06 2024-09-14T06:32:06.595 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:06.595 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:06.595 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:32:06.595 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.078049+0000", 2024-09-14T06:32:06.595 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032330553999999997 2024-09-14T06:32:06.595 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:06.595 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:06.595 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:32:06.596 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.078059+0000", 2024-09-14T06:32:06.596 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.6700000000000006e-06 2024-09-14T06:32:06.596 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:06.596 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:32:06.596 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:06.596 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:06.596 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:06.596 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:98516 3.10 3.1bf56f90 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T06:32:06.596 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:30:07.045722+0000", 2024-09-14T06:32:06.596 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 119.503840361, 2024-09-14T06:32:06.596 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032374334999999997, 2024-09-14T06:32:06.596 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:32:06.596 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:32:06.596 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "reached pg", 2024-09-14T06:32:06.597 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:32:06.597 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:32:06.597 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:06.597 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 98516 2024-09-14T06:32:06.597 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:06.597 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:32:06.597 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:06.597 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:32:06.597 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045722+0000", 2024-09-14T06:32:06.597 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:06.597 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:06.597 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:06.597 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:32:06.597 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045722+0000", 2024-09-14T06:32:06.597 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:06.598 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:06.598 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:06.598 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:32:06.598 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045722+0000", 2024-09-14T06:32:06.598 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.8999999999999998e-07 2024-09-14T06:32:06.598 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:06.598 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:06.598 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:32:06.598 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045723+0000", 2024-09-14T06:32:06.598 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.0049999999999999e-06 2024-09-14T06:32:06.598 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:06.598 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:06.598 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:32:06.598 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045726+0000", 2024-09-14T06:32:06.599 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.9019999999999999e-06 2024-09-14T06:32:06.599 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:06.599 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:06.599 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:32:06.599 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045728+0000", 2024-09-14T06:32:06.599 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.3290000000000001e-06 2024-09-14T06:32:06.599 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:06.599 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:06.599 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:32:06.599 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.078090+0000", 2024-09-14T06:32:06.599 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032362123 2024-09-14T06:32:06.599 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:06.599 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:06.599 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:32:06.600 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.078096+0000", 2024-09-14T06:32:06.600 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.6860000000000003e-06 2024-09-14T06:32:06.600 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:06.600 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:32:06.600 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:06.600 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:06.600 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:06.600 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:98517 3.10 3.1bf56f90 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T06:32:06.600 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:30:07.045731+0000", 2024-09-14T06:32:06.600 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 119.50383063, 2024-09-14T06:32:06.600 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.03241227, 2024-09-14T06:32:06.600 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:32:06.600 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:32:06.600 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "reached pg", 2024-09-14T06:32:06.601 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:32:06.601 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:32:06.601 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:06.601 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 98517 2024-09-14T06:32:06.601 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:06.601 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:32:06.601 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:06.601 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:32:06.601 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045731+0000", 2024-09-14T06:32:06.601 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:06.601 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:06.601 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:06.601 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:32:06.601 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045731+0000", 2024-09-14T06:32:06.601 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:06.602 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:06.602 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:06.602 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:32:06.602 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045732+0000", 2024-09-14T06:32:06.602 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.0699999999999999e-07 2024-09-14T06:32:06.602 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:06.602 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:06.602 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:32:06.602 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045733+0000", 2024-09-14T06:32:06.602 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.9199999999999999e-07 2024-09-14T06:32:06.602 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:06.602 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:06.602 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:32:06.602 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045735+0000", 2024-09-14T06:32:06.603 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.8150000000000002e-06 2024-09-14T06:32:06.603 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:06.603 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:06.603 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:32:06.603 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045738+0000", 2024-09-14T06:32:06.603 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.3640000000000002e-06 2024-09-14T06:32:06.603 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:06.603 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:06.603 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:32:06.603 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.078126+0000", 2024-09-14T06:32:06.603 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032388206000000003 2024-09-14T06:32:06.603 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:06.603 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:06.603 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:32:06.603 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.078144+0000", 2024-09-14T06:32:06.604 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.7686e-05 2024-09-14T06:32:06.604 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:06.604 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:32:06.604 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:06.604 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:06.604 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:06.604 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:98518 3.10 3.1bf56f90 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T06:32:06.604 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:30:07.045741+0000", 2024-09-14T06:32:06.604 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 119.503821235, 2024-09-14T06:32:06.604 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032451793999999999, 2024-09-14T06:32:06.604 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:32:06.604 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:32:06.604 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "reached pg", 2024-09-14T06:32:06.604 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:32:06.605 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:32:06.605 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:06.605 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 98518 2024-09-14T06:32:06.605 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:06.605 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:32:06.605 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:06.605 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:32:06.605 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045741+0000", 2024-09-14T06:32:06.605 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:06.605 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:06.605 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:06.605 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:32:06.605 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045741+0000", 2024-09-14T06:32:06.606 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:06.606 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:06.606 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:06.606 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:32:06.606 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045741+0000", 2024-09-14T06:32:06.606 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.05e-07 2024-09-14T06:32:06.606 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:06.606 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:06.606 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:32:06.606 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045746+0000", 2024-09-14T06:32:06.606 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.2259999999999996e-06 2024-09-14T06:32:06.606 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:06.606 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:06.606 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:32:06.606 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045750+0000", 2024-09-14T06:32:06.607 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.2490000000000001e-06 2024-09-14T06:32:06.607 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:06.607 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:06.607 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:32:06.607 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045752+0000", 2024-09-14T06:32:06.607 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.3980000000000002e-06 2024-09-14T06:32:06.607 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:06.607 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:06.607 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:32:06.607 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.078184+0000", 2024-09-14T06:32:06.607 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032431593000000002 2024-09-14T06:32:06.607 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:06.607 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:06.607 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:32:06.608 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.078193+0000", 2024-09-14T06:32:06.608 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.1230000000000006e-06 2024-09-14T06:32:06.608 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:06.608 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:32:06.608 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:06.608 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:06.608 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:06.608 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:98519 3.10 3.1bf56f90 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T06:32:06.608 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:30:07.045755+0000", 2024-09-14T06:32:06.608 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 119.50380681, 2024-09-14T06:32:06.608 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032489431999999999, 2024-09-14T06:32:06.608 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:32:06.608 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:32:06.608 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "reached pg", 2024-09-14T06:32:06.608 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:32:06.609 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:32:06.609 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:06.609 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 98519 2024-09-14T06:32:06.609 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:06.609 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:32:06.609 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:06.609 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:32:06.609 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045755+0000", 2024-09-14T06:32:06.609 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:06.609 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:06.609 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:06.609 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:32:06.609 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045755+0000", 2024-09-14T06:32:06.609 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:06.610 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:06.610 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:06.610 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:32:06.610 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045756+0000", 2024-09-14T06:32:06.610 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.23e-07 2024-09-14T06:32:06.610 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:06.610 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:06.610 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:32:06.610 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045757+0000", 2024-09-14T06:32:06.610 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.1060000000000001e-06 2024-09-14T06:32:06.610 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:06.610 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:06.610 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:32:06.610 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045760+0000", 2024-09-14T06:32:06.610 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.988e-06 2024-09-14T06:32:06.611 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:06.611 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:06.611 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:32:06.611 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045765+0000", 2024-09-14T06:32:06.611 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.2689999999999999e-06 2024-09-14T06:32:06.611 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:06.611 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:06.611 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:32:06.611 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.078234+0000", 2024-09-14T06:32:06.611 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032468915000000001 2024-09-14T06:32:06.611 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:06.611 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:06.611 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:32:06.611 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.078245+0000", 2024-09-14T06:32:06.612 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.0930999999999999e-05 2024-09-14T06:32:06.612 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:06.612 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:32:06.612 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:06.612 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:06.612 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:06.612 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:98520 3.10 3.1bf56f90 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T06:32:06.612 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:30:07.045768+0000", 2024-09-14T06:32:06.612 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 119.50379393199999, 2024-09-14T06:32:06.612 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032531379999999999, 2024-09-14T06:32:06.612 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:32:06.612 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:32:06.612 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "reached pg", 2024-09-14T06:32:06.612 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:32:06.613 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:32:06.613 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:06.613 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 98520 2024-09-14T06:32:06.613 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:06.613 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:32:06.613 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:06.613 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:32:06.613 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045768+0000", 2024-09-14T06:32:06.613 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:06.613 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:06.613 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:06.613 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:32:06.613 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045768+0000", 2024-09-14T06:32:06.613 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:06.613 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:06.614 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:06.614 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:32:06.614 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045768+0000", 2024-09-14T06:32:06.614 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.22e-07 2024-09-14T06:32:06.614 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:06.614 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:06.614 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:32:06.614 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045773+0000", 2024-09-14T06:32:06.614 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.3470000000000001e-06 2024-09-14T06:32:06.614 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:06.614 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:06.614 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:32:06.614 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045776+0000", 2024-09-14T06:32:06.614 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.1319999999999998e-06 2024-09-14T06:32:06.614 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:06.614 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:06.615 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:32:06.615 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045779+0000", 2024-09-14T06:32:06.615 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.9749999999999999e-06 2024-09-14T06:32:06.615 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:06.615 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:06.615 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:32:06.615 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.078289+0000", 2024-09-14T06:32:06.615 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032509693999999999 2024-09-14T06:32:06.615 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:06.615 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:06.615 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:32:06.615 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.078300+0000", 2024-09-14T06:32:06.615 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.101e-05 2024-09-14T06:32:06.615 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:06.616 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:32:06.616 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:06.616 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:06.616 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:06.616 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:98521 3.10 3.1bf56f90 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T06:32:06.616 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:30:07.045782+0000", 2024-09-14T06:32:06.616 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 119.503779752, 2024-09-14T06:32:06.616 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032566109000000003, 2024-09-14T06:32:06.616 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:32:06.616 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:32:06.616 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "reached pg", 2024-09-14T06:32:06.616 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:32:06.616 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:32:06.616 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:06.616 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 98521 2024-09-14T06:32:06.617 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:06.617 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:32:06.617 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:06.617 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:32:06.617 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045782+0000", 2024-09-14T06:32:06.617 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:06.617 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:06.617 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:06.617 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:32:06.617 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045782+0000", 2024-09-14T06:32:06.617 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:06.617 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:06.617 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:06.617 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:32:06.618 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045783+0000", 2024-09-14T06:32:06.618 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.2499999999999999e-07 2024-09-14T06:32:06.618 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:06.618 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:06.618 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:32:06.618 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045784+0000", 2024-09-14T06:32:06.618 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.0109999999999999e-06 2024-09-14T06:32:06.618 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:06.618 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:06.618 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:32:06.618 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045786+0000", 2024-09-14T06:32:06.618 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.8289999999999999e-06 2024-09-14T06:32:06.618 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:06.618 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:06.618 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:32:06.619 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045793+0000", 2024-09-14T06:32:06.619 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.579e-06 2024-09-14T06:32:06.619 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:06.619 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:06.619 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:32:06.619 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.078338+0000", 2024-09-14T06:32:06.619 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032545389000000001 2024-09-14T06:32:06.619 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:06.619 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:06.619 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:32:06.619 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.078348+0000", 2024-09-14T06:32:06.619 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.0076e-05 2024-09-14T06:32:06.619 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:06.619 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:32:06.619 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:06.620 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:06.620 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:06.620 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:98522 3.10 3.1bf56f90 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T06:32:06.620 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:30:07.045796+0000", 2024-09-14T06:32:06.620 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 119.503765844, 2024-09-14T06:32:06.620 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032577846000000001, 2024-09-14T06:32:06.620 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:32:06.620 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:32:06.620 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "reached pg", 2024-09-14T06:32:06.620 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:32:06.620 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:32:06.620 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:06.620 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 98522 2024-09-14T06:32:06.620 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:06.621 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:32:06.621 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:06.621 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:32:06.621 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045796+0000", 2024-09-14T06:32:06.621 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:06.621 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:06.621 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:06.621 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:32:06.621 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045796+0000", 2024-09-14T06:32:06.621 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:06.621 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:06.621 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:06.621 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:32:06.621 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045796+0000", 2024-09-14T06:32:06.621 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.2100000000000001e-07 2024-09-14T06:32:06.622 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:06.622 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:06.622 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:32:06.622 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045797+0000", 2024-09-14T06:32:06.622 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.6899999999999996e-07 2024-09-14T06:32:06.622 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:06.622 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:06.622 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:32:06.622 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045800+0000", 2024-09-14T06:32:06.622 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.9979999999999999e-06 2024-09-14T06:32:06.622 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:06.622 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:06.622 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:32:06.622 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045803+0000", 2024-09-14T06:32:06.622 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.2850000000000001e-06 2024-09-14T06:32:06.623 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:06.623 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:06.623 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:32:06.623 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.078366+0000", 2024-09-14T06:32:06.623 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032562932000000003 2024-09-14T06:32:06.623 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:06.623 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:06.623 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:32:06.623 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.078374+0000", 2024-09-14T06:32:06.623 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.4409999999999993e-06 2024-09-14T06:32:06.623 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:06.623 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:32:06.623 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:06.623 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:06.623 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:06.623 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:98523 3.10 3.1bf56f90 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T06:32:06.624 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:30:07.045806+0000", 2024-09-14T06:32:06.624 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 119.503756301, 2024-09-14T06:32:06.624 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032611167000000003, 2024-09-14T06:32:06.624 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:32:06.624 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:32:06.624 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "reached pg", 2024-09-14T06:32:06.624 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:32:06.624 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:32:06.624 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:06.624 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 98523 2024-09-14T06:32:06.624 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:06.624 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:32:06.624 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:06.624 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:32:06.625 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045806+0000", 2024-09-14T06:32:06.625 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:06.625 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:06.625 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:06.625 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:32:06.625 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045806+0000", 2024-09-14T06:32:06.625 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:06.625 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:06.625 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:06.625 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:32:06.625 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045806+0000", 2024-09-14T06:32:06.625 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.2000000000000001e-07 2024-09-14T06:32:06.625 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:06.626 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:06.626 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:32:06.626 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045807+0000", 2024-09-14T06:32:06.626 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.4e-07 2024-09-14T06:32:06.626 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:06.626 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:06.626 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:32:06.626 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045810+0000", 2024-09-14T06:32:06.626 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.9229999999999998e-06 2024-09-14T06:32:06.626 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:06.626 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:06.626 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:32:06.626 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045812+0000", 2024-09-14T06:32:06.626 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.2220000000000001e-06 2024-09-14T06:32:06.626 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:06.627 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:06.627 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:32:06.627 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.078407+0000", 2024-09-14T06:32:06.627 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032594513999999998 2024-09-14T06:32:06.627 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:06.627 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:06.627 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:32:06.627 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.078417+0000", 2024-09-14T06:32:06.627 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.0348e-05 2024-09-14T06:32:06.627 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:06.627 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:32:06.627 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:06.627 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:06.627 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:06.628 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:98524 3.10 3.1bf56f90 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T06:32:06.628 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:30:07.045815+0000", 2024-09-14T06:32:06.628 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 119.503746933, 2024-09-14T06:32:06.628 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032643208999999999, 2024-09-14T06:32:06.628 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:32:06.628 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:32:06.628 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "reached pg", 2024-09-14T06:32:06.628 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:32:06.628 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:32:06.628 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:06.628 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 98524 2024-09-14T06:32:06.628 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:06.628 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:32:06.628 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:06.628 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:32:06.629 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045815+0000", 2024-09-14T06:32:06.629 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:06.629 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:06.629 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:06.629 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:32:06.629 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045815+0000", 2024-09-14T06:32:06.629 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:06.629 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:06.629 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:06.629 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:32:06.629 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045815+0000", 2024-09-14T06:32:06.629 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.11e-07 2024-09-14T06:32:06.629 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:06.629 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:06.629 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:32:06.630 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045816+0000", 2024-09-14T06:32:06.630 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.9099999999999991e-07 2024-09-14T06:32:06.630 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:06.630 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:06.630 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:32:06.630 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045819+0000", 2024-09-14T06:32:06.630 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.9399999999999998e-06 2024-09-14T06:32:06.630 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:06.630 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:06.630 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:32:06.630 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045822+0000", 2024-09-14T06:32:06.630 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.3470000000000001e-06 2024-09-14T06:32:06.630 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:06.630 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:06.631 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:32:06.631 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.078449+0000", 2024-09-14T06:32:06.631 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032627529000000002 2024-09-14T06:32:06.631 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:06.631 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:06.631 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:32:06.631 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.078458+0000", 2024-09-14T06:32:06.631 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.1910000000000006e-06 2024-09-14T06:32:06.631 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:06.631 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:32:06.631 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:06.631 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:06.631 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:32:06.631 INFO:teuthology.orchestra.run.smithi138.stdout:} 2024-09-14T06:32:06.632 INFO:teuthology.orchestra.run.smithi094.stderr:noscrub is unset 2024-09-14T06:32:06.633 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-14T06:32:06.633 INFO:tasks.ceph.osd.7.smithi138.stderr:2024-09-14T06:32:06.631+0000 7f3c6a21d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33623) UID: 0 2024-09-14T06:32:06.734 INFO:tasks.ceph.osd.2.smithi094.stderr:2024-09-14T06:32:06.733+0000 7faf3269b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33649) UID: 0 2024-09-14T06:32:06.777 INFO:teuthology.orchestra.run.smithi138.stdout:{ 2024-09-14T06:32:06.778 INFO:teuthology.orchestra.run.smithi138.stdout: "ops": [], 2024-09-14T06:32:06.778 INFO:teuthology.orchestra.run.smithi138.stdout: "num_ops": 0 2024-09-14T06:32:06.778 INFO:teuthology.orchestra.run.smithi138.stdout:} 2024-09-14T06:32:06.786 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-14T06:32:06.834 INFO:tasks.ceph.osd.6.smithi138.stderr:2024-09-14T06:32:06.833+0000 7f8cd8849640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33620) UID: 0 2024-09-14T06:32:06.889 INFO:teuthology.orchestra.run.smithi138.stdout:{ 2024-09-14T06:32:06.889 INFO:teuthology.orchestra.run.smithi138.stdout: "ops": [], 2024-09-14T06:32:06.889 INFO:teuthology.orchestra.run.smithi138.stdout: "complaint_time": 30, 2024-09-14T06:32:06.889 INFO:teuthology.orchestra.run.smithi138.stdout: "num_blocked_ops": 0 2024-09-14T06:32:06.889 INFO:teuthology.orchestra.run.smithi138.stdout:} 2024-09-14T06:32:06.898 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-14T06:32:06.935 INFO:tasks.ceph.osd.1.smithi094.stderr:2024-09-14T06:32:06.934+0000 7f328e47a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33641) UID: 0 2024-09-14T06:32:07.036 INFO:tasks.ceph.osd.6.smithi138.stderr:2024-09-14T06:32:07.035+0000 7f8cd8849640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33620) UID: 0 2024-09-14T06:32:07.043 INFO:teuthology.orchestra.run.smithi138.stdout:{ 2024-09-14T06:32:07.043 INFO:teuthology.orchestra.run.smithi138.stdout: "size": 20, 2024-09-14T06:32:07.043 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 600, 2024-09-14T06:32:07.043 INFO:teuthology.orchestra.run.smithi138.stdout: "ops": [ 2024-09-14T06:32:07.043 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:07.044 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:53244 3.f 3:f71f51a7:::rbd_data.112dca2f6ae1.0000000000000015:17 [read 1863680~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:32:07.044 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:44.369399+0000", 2024-09-14T06:32:07.044 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 142.67229840600001, 2024-09-14T06:32:07.044 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047291189999999997, 2024-09-14T06:32:07.044 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:32:07.044 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:32:07.044 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:32:07.044 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:32:07.044 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:32:07.044 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:07.044 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 53244 2024-09-14T06:32:07.044 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:07.044 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:32:07.045 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:07.045 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:32:07.045 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369399+0000", 2024-09-14T06:32:07.045 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:07.045 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:07.045 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:07.045 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:32:07.045 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369399+0000", 2024-09-14T06:32:07.045 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:07.045 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:07.045 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:07.045 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:32:07.045 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369400+0000", 2024-09-14T06:32:07.045 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.0699999999999999e-07 2024-09-14T06:32:07.046 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:07.046 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:07.046 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:32:07.046 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369401+0000", 2024-09-14T06:32:07.046 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.4300000000000001e-06 2024-09-14T06:32:07.046 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:07.046 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:07.046 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:32:07.046 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369404+0000", 2024-09-14T06:32:07.046 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.061e-06 2024-09-14T06:32:07.046 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:07.046 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:07.046 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:32:07.046 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369407+0000", 2024-09-14T06:32:07.046 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.6139999999999999e-06 2024-09-14T06:32:07.047 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:07.047 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:07.047 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:32:07.047 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.416517+0000", 2024-09-14T06:32:07.047 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047110338000000002 2024-09-14T06:32:07.047 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:07.047 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:07.047 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:32:07.047 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.416616+0000", 2024-09-14T06:32:07.047 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.8651000000000001e-05 2024-09-14T06:32:07.047 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:07.047 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:07.047 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:32:07.047 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.416691+0000", 2024-09-14T06:32:07.048 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.4888999999999994e-05 2024-09-14T06:32:07.048 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:07.048 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:32:07.048 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:07.048 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:07.048 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:07.048 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:53245 3.f 3:f71f51a7:::rbd_data.112dca2f6ae1.0000000000000015:17 [read 1888256~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:32:07.048 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:44.369410+0000", 2024-09-14T06:32:07.048 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 142.67228811800001, 2024-09-14T06:32:07.048 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047399275999999997, 2024-09-14T06:32:07.048 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:32:07.048 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:32:07.048 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:32:07.048 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:32:07.048 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:32:07.049 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:07.049 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 53245 2024-09-14T06:32:07.049 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:07.049 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:32:07.049 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:07.049 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:32:07.049 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369410+0000", 2024-09-14T06:32:07.049 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:07.049 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:07.049 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:07.049 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:32:07.049 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369410+0000", 2024-09-14T06:32:07.049 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:07.050 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:07.050 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:07.050 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:32:07.050 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369410+0000", 2024-09-14T06:32:07.050 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.1899999999999999e-07 2024-09-14T06:32:07.050 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:07.050 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:07.050 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:32:07.050 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369411+0000", 2024-09-14T06:32:07.050 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.0300000000000001e-06 2024-09-14T06:32:07.050 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:07.050 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:07.050 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:32:07.050 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369414+0000", 2024-09-14T06:32:07.050 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.1099999999999999e-06 2024-09-14T06:32:07.051 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:07.051 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:07.051 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:32:07.051 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369417+0000", 2024-09-14T06:32:07.051 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.6189999999999998e-06 2024-09-14T06:32:07.051 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:07.051 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:07.051 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:32:07.051 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.416718+0000", 2024-09-14T06:32:07.051 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047301322999999999 2024-09-14T06:32:07.051 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:07.051 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:07.051 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:32:07.051 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.416759+0000", 2024-09-14T06:32:07.052 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.1009e-05 2024-09-14T06:32:07.052 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:07.052 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:07.052 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:32:07.052 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.416809+0000", 2024-09-14T06:32:07.052 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.9966000000000002e-05 2024-09-14T06:32:07.052 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:07.052 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:32:07.052 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:07.053 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:07.053 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:07.053 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:53246 3.f 3:f71f51a7:::rbd_data.112dca2f6ae1.0000000000000015:17 [read 1912832~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:32:07.053 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:44.369420+0000", 2024-09-14T06:32:07.053 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 142.67227817400001, 2024-09-14T06:32:07.053 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047529463000000001, 2024-09-14T06:32:07.053 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:32:07.053 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:32:07.053 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:32:07.053 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:32:07.053 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:32:07.053 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:07.053 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 53246 2024-09-14T06:32:07.053 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:07.053 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:32:07.054 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:07.054 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:32:07.054 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369420+0000", 2024-09-14T06:32:07.054 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:07.054 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:07.054 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:07.054 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:32:07.054 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369420+0000", 2024-09-14T06:32:07.054 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:07.054 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:07.054 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:07.054 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:32:07.054 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369420+0000", 2024-09-14T06:32:07.054 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.1299999999999999e-07 2024-09-14T06:32:07.055 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:07.055 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:07.055 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:32:07.055 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369425+0000", 2024-09-14T06:32:07.055 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.0130000000000001e-06 2024-09-14T06:32:07.055 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:07.055 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:07.055 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:32:07.055 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369428+0000", 2024-09-14T06:32:07.055 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.349e-06 2024-09-14T06:32:07.055 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:07.055 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:07.055 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:32:07.055 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369431+0000", 2024-09-14T06:32:07.055 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.7389999999999999e-06 2024-09-14T06:32:07.056 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:07.056 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:07.056 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:32:07.056 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.416834+0000", 2024-09-14T06:32:07.056 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047402669000000001 2024-09-14T06:32:07.056 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:07.056 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:07.056 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:32:07.056 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.416882+0000", 2024-09-14T06:32:07.056 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.8068999999999997e-05 2024-09-14T06:32:07.056 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:07.056 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:07.056 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:32:07.057 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.416949+0000", 2024-09-14T06:32:07.057 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.7410999999999995e-05 2024-09-14T06:32:07.057 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:07.057 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:32:07.057 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:07.057 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:07.057 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:07.057 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:53247 3.f 3:f71f51a7:::rbd_data.112dca2f6ae1.0000000000000015:17 [read 1937408~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:32:07.057 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:44.369434+0000", 2024-09-14T06:32:07.057 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 142.67226341, 2024-09-14T06:32:07.057 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047652264999999999, 2024-09-14T06:32:07.057 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:32:07.057 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:32:07.058 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:32:07.058 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:32:07.058 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:32:07.058 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:07.058 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 53247 2024-09-14T06:32:07.058 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:07.058 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:32:07.058 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:07.058 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:32:07.058 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369434+0000", 2024-09-14T06:32:07.058 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:07.058 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:07.058 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:07.058 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:32:07.058 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369434+0000", 2024-09-14T06:32:07.059 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:07.059 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:07.059 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:07.059 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:32:07.059 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369435+0000", 2024-09-14T06:32:07.059 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.2100000000000001e-07 2024-09-14T06:32:07.059 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:07.059 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:07.059 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:32:07.059 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369436+0000", 2024-09-14T06:32:07.059 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.122e-06 2024-09-14T06:32:07.059 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:07.059 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:07.059 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:32:07.060 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369439+0000", 2024-09-14T06:32:07.060 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.0400000000000001e-06 2024-09-14T06:32:07.060 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:07.060 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:07.060 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:32:07.060 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369441+0000", 2024-09-14T06:32:07.060 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.728e-06 2024-09-14T06:32:07.060 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:07.060 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:07.060 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:32:07.060 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.416977+0000", 2024-09-14T06:32:07.060 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047535344 2024-09-14T06:32:07.060 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:07.060 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:07.061 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:32:07.061 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.417029+0000", 2024-09-14T06:32:07.061 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.2037999999999999e-05 2024-09-14T06:32:07.061 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:07.061 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:07.061 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:32:07.061 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.417087+0000", 2024-09-14T06:32:07.061 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.7772000000000002e-05 2024-09-14T06:32:07.061 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:07.061 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:32:07.061 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:07.061 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:07.061 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:07.062 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:53248 3.f 3:f71f51a7:::rbd_data.112dca2f6ae1.0000000000000015:17 [read 1961984~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:32:07.062 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:44.369445+0000", 2024-09-14T06:32:07.062 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 142.67225285800001, 2024-09-14T06:32:07.062 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047798489, 2024-09-14T06:32:07.062 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:32:07.062 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:32:07.062 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:32:07.062 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:32:07.062 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:32:07.062 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:07.062 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 53248 2024-09-14T06:32:07.062 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:07.062 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:32:07.062 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:07.063 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:32:07.063 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369445+0000", 2024-09-14T06:32:07.063 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:07.063 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:07.063 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:07.063 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:32:07.063 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369445+0000", 2024-09-14T06:32:07.063 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:07.063 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:07.063 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:07.063 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:32:07.063 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369445+0000", 2024-09-14T06:32:07.063 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.0699999999999999e-07 2024-09-14T06:32:07.063 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:07.064 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:07.064 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:32:07.064 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369446+0000", 2024-09-14T06:32:07.064 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.1000000000000001e-06 2024-09-14T06:32:07.064 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:07.064 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:07.064 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:32:07.064 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369449+0000", 2024-09-14T06:32:07.064 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.1039999999999998e-06 2024-09-14T06:32:07.064 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:07.064 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:07.064 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:32:07.064 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369452+0000", 2024-09-14T06:32:07.064 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.6199999999999999e-06 2024-09-14T06:32:07.065 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:07.065 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:07.065 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:32:07.065 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.417114+0000", 2024-09-14T06:32:07.065 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047661901999999999 2024-09-14T06:32:07.065 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:07.065 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:07.065 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:32:07.065 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.417181+0000", 2024-09-14T06:32:07.065 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.7385999999999995e-05 2024-09-14T06:32:07.065 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:07.065 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:07.065 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:32:07.066 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.417243+0000", 2024-09-14T06:32:07.066 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.2169999999999996e-05 2024-09-14T06:32:07.066 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:07.066 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:32:07.066 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:07.066 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:07.066 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:07.066 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:53249 3.f 3:f71f51a7:::rbd_data.112dca2f6ae1.0000000000000015:17 [read 1986560~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:32:07.066 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:44.369455+0000", 2024-09-14T06:32:07.066 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 142.672242547, 2024-09-14T06:32:07.066 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047890493999999999, 2024-09-14T06:32:07.066 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:32:07.066 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:32:07.066 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:32:07.066 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:32:07.067 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:32:07.067 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:07.067 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 53249 2024-09-14T06:32:07.067 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:07.067 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:32:07.067 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:07.067 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:32:07.067 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369455+0000", 2024-09-14T06:32:07.067 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:07.067 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:07.067 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:07.067 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:32:07.067 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369455+0000", 2024-09-14T06:32:07.067 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:07.067 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:07.068 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:07.068 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:32:07.068 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369455+0000", 2024-09-14T06:32:07.068 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.0800000000000001e-07 2024-09-14T06:32:07.068 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:07.068 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:07.068 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:32:07.068 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369456+0000", 2024-09-14T06:32:07.068 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.0729999999999999e-06 2024-09-14T06:32:07.068 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:07.068 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:07.068 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:32:07.068 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369460+0000", 2024-09-14T06:32:07.068 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.027e-06 2024-09-14T06:32:07.069 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:07.069 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:07.069 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:32:07.069 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369462+0000", 2024-09-14T06:32:07.069 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.5320000000000001e-06 2024-09-14T06:32:07.069 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:07.069 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:07.069 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:32:07.069 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.417257+0000", 2024-09-14T06:32:07.069 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.04779489 2024-09-14T06:32:07.069 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:07.069 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:07.069 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:32:07.069 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.417296+0000", 2024-09-14T06:32:07.069 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.8658000000000002e-05 2024-09-14T06:32:07.070 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:07.070 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:07.070 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:32:07.070 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.417346+0000", 2024-09-14T06:32:07.070 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.0105999999999998e-05 2024-09-14T06:32:07.070 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:07.070 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:32:07.070 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:07.070 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:07.070 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:07.070 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:53250 3.f 3:f71f51a7:::rbd_data.112dca2f6ae1.0000000000000015:17 [read 2011136~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:32:07.070 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:44.369465+0000", 2024-09-14T06:32:07.070 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 142.67223257699999, 2024-09-14T06:32:07.070 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.048032230000000002, 2024-09-14T06:32:07.071 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:32:07.071 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:32:07.071 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:32:07.071 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:32:07.071 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:32:07.071 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:07.071 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 53250 2024-09-14T06:32:07.071 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:07.071 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:32:07.071 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:07.071 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:32:07.071 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369465+0000", 2024-09-14T06:32:07.071 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:07.071 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:07.071 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:07.072 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:32:07.072 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369465+0000", 2024-09-14T06:32:07.072 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:07.072 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:07.072 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:07.072 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:32:07.072 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369465+0000", 2024-09-14T06:32:07.072 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.2100000000000001e-07 2024-09-14T06:32:07.072 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:07.072 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:07.072 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:32:07.072 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369466+0000", 2024-09-14T06:32:07.072 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.0640000000000001e-06 2024-09-14T06:32:07.072 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:07.073 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:07.073 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:32:07.073 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369469+0000", 2024-09-14T06:32:07.073 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.0170000000000001e-06 2024-09-14T06:32:07.073 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:07.073 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:07.073 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:32:07.073 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369472+0000", 2024-09-14T06:32:07.073 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.6189999999999998e-06 2024-09-14T06:32:07.073 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:07.073 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:07.073 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:32:07.073 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.417383+0000", 2024-09-14T06:32:07.073 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047911092000000002 2024-09-14T06:32:07.073 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:07.074 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:07.074 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:32:07.074 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.417430+0000", 2024-09-14T06:32:07.074 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.6898999999999999e-05 2024-09-14T06:32:07.074 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:07.074 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:07.074 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:32:07.074 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.417497+0000", 2024-09-14T06:32:07.074 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.7317999999999995e-05 2024-09-14T06:32:07.074 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:07.074 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:32:07.074 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:07.074 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:07.074 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:07.075 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:53251 3.f 3:f71f51a7:::rbd_data.112dca2f6ae1.0000000000000015:17 [read 2035712~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:32:07.075 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:44.369475+0000", 2024-09-14T06:32:07.075 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 142.672222581, 2024-09-14T06:32:07.075 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.048154542000000002, 2024-09-14T06:32:07.075 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:32:07.075 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:32:07.075 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:32:07.075 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:32:07.075 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:32:07.075 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:07.075 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 53251 2024-09-14T06:32:07.075 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:07.075 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:32:07.076 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:07.076 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:32:07.076 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369475+0000", 2024-09-14T06:32:07.076 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:07.076 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:07.076 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:07.076 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:32:07.076 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369475+0000", 2024-09-14T06:32:07.076 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:07.076 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:07.076 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:07.076 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:32:07.076 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369475+0000", 2024-09-14T06:32:07.076 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.0100000000000001e-07 2024-09-14T06:32:07.076 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:07.077 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:07.077 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:32:07.077 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369476+0000", 2024-09-14T06:32:07.077 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.0580000000000001e-06 2024-09-14T06:32:07.077 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:07.077 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:07.077 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:32:07.077 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369479+0000", 2024-09-14T06:32:07.077 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.9950000000000001e-06 2024-09-14T06:32:07.077 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:07.077 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:07.077 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:32:07.077 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369482+0000", 2024-09-14T06:32:07.077 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.6340000000000001e-06 2024-09-14T06:32:07.077 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:07.078 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:07.078 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:32:07.078 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.417530+0000", 2024-09-14T06:32:07.078 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.048047535000000002 2024-09-14T06:32:07.078 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:07.078 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:07.078 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:32:07.078 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.417574+0000", 2024-09-14T06:32:07.078 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.4255999999999998e-05 2024-09-14T06:32:07.078 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:07.078 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:07.078 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:32:07.078 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.417630+0000", 2024-09-14T06:32:07.078 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.5862999999999997e-05 2024-09-14T06:32:07.079 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:07.079 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:32:07.079 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:07.079 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:07.079 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:07.079 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:53252 3.f 3:f71f51a7:::rbd_data.112dca2f6ae1.0000000000000015:17 [read 2060288~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:32:07.079 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:44.369485+0000", 2024-09-14T06:32:07.079 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 142.67221273300001, 2024-09-14T06:32:07.079 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.048254116, 2024-09-14T06:32:07.079 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:32:07.079 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:32:07.079 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:32:07.079 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:32:07.079 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:32:07.080 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:07.080 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 53252 2024-09-14T06:32:07.080 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:07.080 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:32:07.080 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:07.080 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:32:07.080 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369485+0000", 2024-09-14T06:32:07.080 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:07.080 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:07.080 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:07.080 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:32:07.080 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369485+0000", 2024-09-14T06:32:07.080 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:07.080 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:07.081 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:07.081 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:32:07.081 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369485+0000", 2024-09-14T06:32:07.081 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.1e-07 2024-09-14T06:32:07.081 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:07.081 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:07.081 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:32:07.081 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369487+0000", 2024-09-14T06:32:07.081 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.42e-06 2024-09-14T06:32:07.081 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:07.081 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:07.081 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:32:07.081 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369490+0000", 2024-09-14T06:32:07.081 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.9639999999999999e-06 2024-09-14T06:32:07.081 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:07.082 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:07.082 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:32:07.082 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369492+0000", 2024-09-14T06:32:07.082 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.5679999999999998e-06 2024-09-14T06:32:07.082 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:07.082 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:07.082 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:32:07.082 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.417645+0000", 2024-09-14T06:32:07.082 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.048152753 2024-09-14T06:32:07.082 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:07.082 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:07.082 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:32:07.082 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.417687+0000", 2024-09-14T06:32:07.082 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.1900999999999997e-05 2024-09-14T06:32:07.083 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:07.083 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:07.083 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:32:07.083 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.417739+0000", 2024-09-14T06:32:07.083 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.2299999999999997e-05 2024-09-14T06:32:07.083 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:07.083 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:32:07.083 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:07.083 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:07.083 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:07.083 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:53253 3.f 3:f71f51a7:::rbd_data.112dca2f6ae1.0000000000000015:17 [read 2084864~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:32:07.083 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:44.369495+0000", 2024-09-14T06:32:07.083 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 142.67220260900001, 2024-09-14T06:32:07.083 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.048366785000000002, 2024-09-14T06:32:07.084 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:32:07.084 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:32:07.084 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:32:07.084 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:32:07.084 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:32:07.084 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:07.084 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 53253 2024-09-14T06:32:07.084 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:07.084 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:32:07.084 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:07.084 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:32:07.084 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369495+0000", 2024-09-14T06:32:07.084 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:07.085 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:07.085 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:07.085 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:32:07.085 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369495+0000", 2024-09-14T06:32:07.085 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:07.085 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:07.085 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:07.085 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:32:07.085 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370388+0000", 2024-09-14T06:32:07.085 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00089287999999999998 2024-09-14T06:32:07.085 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:07.085 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:07.085 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:32:07.085 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370392+0000", 2024-09-14T06:32:07.086 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.2130000000000002e-06 2024-09-14T06:32:07.086 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:07.086 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:07.086 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:32:07.086 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370401+0000", 2024-09-14T06:32:07.086 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.7649999999999999e-06 2024-09-14T06:32:07.086 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:07.086 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:07.086 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:32:07.086 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370406+0000", 2024-09-14T06:32:07.086 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.0159999999999999e-06 2024-09-14T06:32:07.086 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:07.086 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:07.086 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:32:07.087 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.417764+0000", 2024-09-14T06:32:07.087 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047358348000000001 2024-09-14T06:32:07.087 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:07.087 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:07.087 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:32:07.087 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.417806+0000", 2024-09-14T06:32:07.087 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.1149999999999997e-05 2024-09-14T06:32:07.087 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:07.087 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:07.087 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:32:07.087 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.417862+0000", 2024-09-14T06:32:07.087 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.6412999999999997e-05 2024-09-14T06:32:07.087 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:07.087 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:32:07.088 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:07.088 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:07.088 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:07.088 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:53340 3.f 3:f71f51a7:::rbd_data.112dca2f6ae1.0000000000000015:17 [read 1056768~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:32:07.088 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:44.370421+0000", 2024-09-14T06:32:07.088 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 142.671276729, 2024-09-14T06:32:07.088 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047566313999999998, 2024-09-14T06:32:07.088 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:32:07.088 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:32:07.088 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:32:07.088 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:32:07.088 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:32:07.088 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:07.088 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 53340 2024-09-14T06:32:07.089 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:07.089 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:32:07.089 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:07.089 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:32:07.089 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370421+0000", 2024-09-14T06:32:07.089 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:07.089 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:07.089 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:07.089 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:32:07.089 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370421+0000", 2024-09-14T06:32:07.089 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:07.089 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:07.089 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:07.090 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:32:07.090 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370422+0000", 2024-09-14T06:32:07.090 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.3799999999999996e-07 2024-09-14T06:32:07.090 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:07.090 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:07.090 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:32:07.090 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370424+0000", 2024-09-14T06:32:07.090 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.6919999999999998e-06 2024-09-14T06:32:07.090 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:07.090 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:07.090 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:32:07.090 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370432+0000", 2024-09-14T06:32:07.090 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.2439999999999996e-06 2024-09-14T06:32:07.090 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:07.091 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:07.091 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:32:07.091 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370437+0000", 2024-09-14T06:32:07.091 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.5389999999999998e-06 2024-09-14T06:32:07.091 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:07.091 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:07.091 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:32:07.091 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.417888+0000", 2024-09-14T06:32:07.091 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047450432000000001 2024-09-14T06:32:07.091 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:07.091 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:07.091 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:32:07.091 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.417935+0000", 2024-09-14T06:32:07.091 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.7454999999999999e-05 2024-09-14T06:32:07.092 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:07.092 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:07.092 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:32:07.092 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.417987+0000", 2024-09-14T06:32:07.092 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.2213999999999998e-05 2024-09-14T06:32:07.092 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:07.092 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:32:07.092 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:07.092 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:07.092 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:07.092 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:53341 3.f 3:f71f51a7:::rbd_data.112dca2f6ae1.0000000000000015:17 [read 1081344~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:32:07.092 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:44.370446+0000", 2024-09-14T06:32:07.092 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 142.67125145599999, 2024-09-14T06:32:07.092 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047665723, 2024-09-14T06:32:07.093 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:32:07.093 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:32:07.093 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:32:07.093 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:32:07.093 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:32:07.093 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:07.093 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 53341 2024-09-14T06:32:07.093 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:07.093 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:32:07.093 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:07.093 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:32:07.093 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370446+0000", 2024-09-14T06:32:07.094 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:07.094 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:07.094 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:07.094 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:32:07.094 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370446+0000", 2024-09-14T06:32:07.094 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:07.094 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:07.094 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:07.094 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:32:07.094 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370447+0000", 2024-09-14T06:32:07.094 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.6100000000000002e-07 2024-09-14T06:32:07.094 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:07.094 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:07.094 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:32:07.094 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370448+0000", 2024-09-14T06:32:07.095 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.7370000000000001e-06 2024-09-14T06:32:07.095 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:07.095 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:07.095 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:32:07.095 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370462+0000", 2024-09-14T06:32:07.095 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.3227000000000001e-05 2024-09-14T06:32:07.095 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:07.095 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:07.095 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:32:07.095 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370467+0000", 2024-09-14T06:32:07.095 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.6540000000000004e-06 2024-09-14T06:32:07.095 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:07.095 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:07.095 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:32:07.096 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.418013+0000", 2024-09-14T06:32:07.096 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047545325999999999 2024-09-14T06:32:07.096 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:07.096 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:07.096 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:32:07.096 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.418054+0000", 2024-09-14T06:32:07.096 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.1696999999999997e-05 2024-09-14T06:32:07.096 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:07.096 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:07.096 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:32:07.096 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.418112+0000", 2024-09-14T06:32:07.096 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.7720999999999998e-05 2024-09-14T06:32:07.096 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:07.096 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:32:07.097 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:07.097 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:07.097 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:07.097 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:53342 3.f 3:f71f51a7:::rbd_data.112dca2f6ae1.0000000000000015:17 [read 1105920~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:32:07.097 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:44.370475+0000", 2024-09-14T06:32:07.097 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 142.671222395, 2024-09-14T06:32:07.097 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047748848000000003, 2024-09-14T06:32:07.097 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:32:07.097 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:32:07.097 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:32:07.097 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:32:07.097 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:32:07.097 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:07.097 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 53342 2024-09-14T06:32:07.097 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:07.098 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:32:07.098 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:07.098 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:32:07.098 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370475+0000", 2024-09-14T06:32:07.098 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:07.098 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:07.098 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:07.098 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:32:07.098 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370475+0000", 2024-09-14T06:32:07.098 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:07.098 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:07.098 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:07.098 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:32:07.098 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370476+0000", 2024-09-14T06:32:07.098 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.4700000000000002e-07 2024-09-14T06:32:07.099 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:07.099 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:07.099 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:32:07.099 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370478+0000", 2024-09-14T06:32:07.099 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.5890000000000001e-06 2024-09-14T06:32:07.099 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:07.099 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:07.099 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:32:07.099 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370486+0000", 2024-09-14T06:32:07.099 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.3470000000000001e-06 2024-09-14T06:32:07.099 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:07.099 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:07.099 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:32:07.099 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370491+0000", 2024-09-14T06:32:07.100 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.5720000000000002e-06 2024-09-14T06:32:07.100 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:07.100 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:07.100 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:32:07.100 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.418135+0000", 2024-09-14T06:32:07.100 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047644094999999997 2024-09-14T06:32:07.100 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:07.100 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:07.100 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:32:07.100 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.418174+0000", 2024-09-14T06:32:07.100 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.8204000000000002e-05 2024-09-14T06:32:07.100 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:07.100 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:07.100 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:32:07.100 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.418224+0000", 2024-09-14T06:32:07.101 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.0593999999999997e-05 2024-09-14T06:32:07.101 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:07.101 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:32:07.101 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:07.101 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:07.101 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:07.101 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:53343 3.f 3:f71f51a7:::rbd_data.112dca2f6ae1.0000000000000015:17 [read 1130496~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:32:07.101 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:44.370498+0000", 2024-09-14T06:32:07.101 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 142.671199519, 2024-09-14T06:32:07.101 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047841574999999997, 2024-09-14T06:32:07.101 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:32:07.101 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:32:07.101 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:32:07.102 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:32:07.102 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:32:07.102 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:07.102 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 53343 2024-09-14T06:32:07.102 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:07.102 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:32:07.102 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:07.102 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:32:07.102 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370498+0000", 2024-09-14T06:32:07.102 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:07.102 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:07.102 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:07.102 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:32:07.102 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370498+0000", 2024-09-14T06:32:07.102 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:07.103 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:07.103 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:07.103 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:32:07.103 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370499+0000", 2024-09-14T06:32:07.103 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.9999999999999998e-07 2024-09-14T06:32:07.103 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:07.103 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:07.103 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:32:07.103 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370501+0000", 2024-09-14T06:32:07.103 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.0310000000000001e-06 2024-09-14T06:32:07.103 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:07.103 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:07.103 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:32:07.103 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370507+0000", 2024-09-14T06:32:07.103 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.4169999999999997e-06 2024-09-14T06:32:07.104 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:07.104 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:07.104 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:32:07.104 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370516+0000", 2024-09-14T06:32:07.104 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.7190000000000007e-06 2024-09-14T06:32:07.104 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:07.104 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:07.104 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:32:07.104 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.418247+0000", 2024-09-14T06:32:07.104 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047731190999999999 2024-09-14T06:32:07.104 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:07.104 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:07.104 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:32:07.105 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.418286+0000", 2024-09-14T06:32:07.105 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.8988e-05 2024-09-14T06:32:07.105 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:07.105 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:07.105 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:32:07.105 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.418340+0000", 2024-09-14T06:32:07.105 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.3829000000000002e-05 2024-09-14T06:32:07.105 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:07.105 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:32:07.105 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:07.105 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:07.105 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:07.105 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:53344 3.f 3:f71f51a7:::rbd_data.112dca2f6ae1.0000000000000015:17 [read 1155072~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:32:07.105 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:44.370524+0000", 2024-09-14T06:32:07.106 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 142.67117374899999, 2024-09-14T06:32:07.106 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047972262000000002, 2024-09-14T06:32:07.106 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:32:07.106 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:32:07.106 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:32:07.106 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:32:07.106 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:32:07.106 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:07.106 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 53344 2024-09-14T06:32:07.106 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:07.106 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:32:07.106 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:07.106 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:32:07.106 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370524+0000", 2024-09-14T06:32:07.107 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:07.107 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:07.107 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:07.107 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:32:07.107 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370524+0000", 2024-09-14T06:32:07.107 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:07.107 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:07.107 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:07.107 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:32:07.107 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370618+0000", 2024-09-14T06:32:07.107 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.4425999999999993e-05 2024-09-14T06:32:07.107 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:07.107 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:07.108 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:32:07.108 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370622+0000", 2024-09-14T06:32:07.108 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.3519999999999998e-06 2024-09-14T06:32:07.108 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:07.108 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:07.108 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:32:07.108 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370631+0000", 2024-09-14T06:32:07.108 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.9299999999999992e-06 2024-09-14T06:32:07.108 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:07.108 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:07.108 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:32:07.108 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370637+0000", 2024-09-14T06:32:07.108 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.3269999999999997e-06 2024-09-14T06:32:07.108 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:07.109 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:07.109 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:32:07.109 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.418372+0000", 2024-09-14T06:32:07.109 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047734764999999998 2024-09-14T06:32:07.109 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:07.109 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:07.109 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:32:07.109 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.418431+0000", 2024-09-14T06:32:07.109 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.8696999999999997e-05 2024-09-14T06:32:07.109 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:07.109 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:07.109 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:32:07.109 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.418496+0000", 2024-09-14T06:32:07.109 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.5765000000000005e-05 2024-09-14T06:32:07.110 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:07.110 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:32:07.110 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:07.110 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:07.110 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:07.110 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:53345 3.f 3:f71f51a7:::rbd_data.112dca2f6ae1.0000000000000015:17 [read 1179648~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:32:07.110 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:44.370647+0000", 2024-09-14T06:32:07.110 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 142.67105028200001, 2024-09-14T06:32:07.110 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047953775999999997, 2024-09-14T06:32:07.110 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:32:07.110 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:32:07.110 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:32:07.110 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:32:07.110 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:32:07.111 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:07.111 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 53345 2024-09-14T06:32:07.111 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:07.111 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:32:07.111 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:07.111 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:32:07.111 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370647+0000", 2024-09-14T06:32:07.111 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:07.111 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:07.111 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:07.111 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:32:07.111 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370647+0000", 2024-09-14T06:32:07.111 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:07.111 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:07.112 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:07.112 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:32:07.112 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.372346+0000", 2024-09-14T06:32:07.112 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.001698284 2024-09-14T06:32:07.112 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:07.112 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:07.112 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:32:07.112 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.372351+0000", 2024-09-14T06:32:07.112 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.8890000000000001e-06 2024-09-14T06:32:07.112 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:07.112 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:07.112 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:32:07.112 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.372362+0000", 2024-09-14T06:32:07.113 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.0941000000000001e-05 2024-09-14T06:32:07.113 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:07.113 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:07.113 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:32:07.113 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.372369+0000", 2024-09-14T06:32:07.113 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.362e-06 2024-09-14T06:32:07.113 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:07.113 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:07.113 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:32:07.113 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.418510+0000", 2024-09-14T06:32:07.113 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046140914999999998 2024-09-14T06:32:07.113 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:07.113 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:07.113 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:32:07.114 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.418549+0000", 2024-09-14T06:32:07.114 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.8717999999999998e-05 2024-09-14T06:32:07.114 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:07.114 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:07.114 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:32:07.114 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.418601+0000", 2024-09-14T06:32:07.114 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.2667000000000002e-05 2024-09-14T06:32:07.114 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:07.114 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:32:07.114 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:07.114 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:07.114 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:07.114 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:53353 3.f 3:f71f51a7:::rbd_data.112dca2f6ae1.0000000000000015:17 [read 1376256~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:32:07.115 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:44.372545+0000", 2024-09-14T06:32:07.115 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 142.66915320699999, 2024-09-14T06:32:07.115 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047364232999999999, 2024-09-14T06:32:07.115 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:32:07.115 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:32:07.115 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:32:07.115 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:32:07.115 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:32:07.115 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:07.115 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 53353 2024-09-14T06:32:07.115 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:07.115 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:32:07.115 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:07.116 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:32:07.116 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.372545+0000", 2024-09-14T06:32:07.116 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:07.116 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:07.116 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:07.116 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:32:07.116 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.372545+0000", 2024-09-14T06:32:07.116 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:07.116 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:07.116 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:07.116 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:32:07.116 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.372545+0000", 2024-09-14T06:32:07.116 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.27e-07 2024-09-14T06:32:07.116 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:07.117 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:07.117 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:32:07.117 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.372547+0000", 2024-09-14T06:32:07.117 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.0600000000000002e-06 2024-09-14T06:32:07.117 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:07.117 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:07.117 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:32:07.117 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.372553+0000", 2024-09-14T06:32:07.117 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.7980000000000002e-06 2024-09-14T06:32:07.117 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:07.117 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:07.117 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:32:07.117 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.372557+0000", 2024-09-14T06:32:07.118 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.7389999999999999e-06 2024-09-14T06:32:07.118 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:07.118 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:07.118 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:32:07.118 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.419811+0000", 2024-09-14T06:32:07.118 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047253310999999999 2024-09-14T06:32:07.118 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:07.118 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:07.118 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:32:07.118 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.419854+0000", 2024-09-14T06:32:07.118 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.3448000000000001e-05 2024-09-14T06:32:07.118 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:07.118 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:07.118 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:32:07.119 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.419909+0000", 2024-09-14T06:32:07.119 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.4549999999999998e-05 2024-09-14T06:32:07.119 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:07.119 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:32:07.119 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:07.119 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:07.119 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:07.119 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:53354 3.f 3:f71f51a7:::rbd_data.112dca2f6ae1.0000000000000015:17 [read 1400832~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:32:07.119 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:44.372563+0000", 2024-09-14T06:32:07.119 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 142.66913477, 2024-09-14T06:32:07.119 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047482233999999998, 2024-09-14T06:32:07.119 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:32:07.119 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:32:07.119 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:32:07.120 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:32:07.120 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:32:07.120 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:07.120 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 53354 2024-09-14T06:32:07.120 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:07.120 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:32:07.120 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:07.120 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:32:07.120 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.372563+0000", 2024-09-14T06:32:07.120 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:07.120 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:07.120 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:07.120 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:32:07.120 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.372563+0000", 2024-09-14T06:32:07.121 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:07.121 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:07.121 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:07.121 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:32:07.121 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.375009+0000", 2024-09-14T06:32:07.121 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.0024457680000000001 2024-09-14T06:32:07.121 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:07.121 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:07.121 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:32:07.121 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.375014+0000", 2024-09-14T06:32:07.121 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.0599999999999998e-06 2024-09-14T06:32:07.121 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:07.121 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:07.122 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:32:07.122 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.375026+0000", 2024-09-14T06:32:07.122 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.1783999999999999e-05 2024-09-14T06:32:07.122 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:07.122 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:07.122 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:32:07.122 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.375035+0000", 2024-09-14T06:32:07.122 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.7799999999999995e-06 2024-09-14T06:32:07.122 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:07.122 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:07.122 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:32:07.122 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.419942+0000", 2024-09-14T06:32:07.122 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.044906982999999998 2024-09-14T06:32:07.122 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:07.122 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:07.122 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:32:07.123 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.419989+0000", 2024-09-14T06:32:07.123 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.7104999999999997e-05 2024-09-14T06:32:07.123 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:07.123 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:07.123 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:32:07.123 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.420045+0000", 2024-09-14T06:32:07.123 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.5754000000000002e-05 2024-09-14T06:32:07.123 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:07.123 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:32:07.123 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:07.123 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:07.123 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:07.123 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:225577 3.15 3:ac159a70:::rbd_header.112daa9ec306:head [watch reconnect cookie 140616827151664 gen 4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e188)", 2024-09-14T06:32:07.123 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:31:52.733493+0000", 2024-09-14T06:32:07.123 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 14.308204306, 2024-09-14T06:32:07.124 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.0156593789999999, 2024-09-14T06:32:07.124 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:32:07.124 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:32:07.124 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:32:07.124 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:32:07.124 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:32:07.124 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:07.124 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 225577 2024-09-14T06:32:07.124 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:07.124 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:32:07.124 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:07.124 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:32:07.124 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:31:52.733493+0000", 2024-09-14T06:32:07.124 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:07.125 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:07.125 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:07.125 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:32:07.125 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:31:52.733493+0000", 2024-09-14T06:32:07.125 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:07.125 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:07.125 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:07.125 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:32:07.125 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:31:52.733495+0000", 2024-09-14T06:32:07.125 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.0059999999999999e-06 2024-09-14T06:32:07.125 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:07.125 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:07.125 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:32:07.125 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:31:52.733504+0000", 2024-09-14T06:32:07.126 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.5490000000000006e-06 2024-09-14T06:32:07.126 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:07.126 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:07.126 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:32:07.126 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:31:52.733522+0000", 2024-09-14T06:32:07.126 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.7660999999999999e-05 2024-09-14T06:32:07.126 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:07.126 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:07.126 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:32:07.126 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:31:52.733531+0000", 2024-09-14T06:32:07.126 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.7180000000000008e-06 2024-09-14T06:32:07.126 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:07.126 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:07.126 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:32:07.127 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:31:52.733598+0000", 2024-09-14T06:32:07.127 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.6490999999999997e-05 2024-09-14T06:32:07.127 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:07.127 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:07.127 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "op must wait for map", 2024-09-14T06:32:07.127 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:31:52.733605+0000", 2024-09-14T06:32:07.127 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.6109999999999996e-06 2024-09-14T06:32:07.127 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:07.127 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:07.127 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:32:07.127 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:31:52.737908+0000", 2024-09-14T06:32:07.127 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.0043023050000000002 2024-09-14T06:32:07.127 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:07.127 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:07.127 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "waiting for peered", 2024-09-14T06:32:07.127 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:31:52.737923+0000", 2024-09-14T06:32:07.128 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.488e-05 2024-09-14T06:32:07.128 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:07.128 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:07.128 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:32:07.128 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:31:53.746776+0000", 2024-09-14T06:32:07.128 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.0088535359999999 2024-09-14T06:32:07.128 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:07.128 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:07.128 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:32:07.128 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:31:53.746878+0000", 2024-09-14T06:32:07.128 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.000101788 2024-09-14T06:32:07.128 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:07.128 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:07.128 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:32:07.129 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:31:53.749153+0000", 2024-09-14T06:32:07.129 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.0022748339999999999 2024-09-14T06:32:07.129 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:07.129 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:32:07.129 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:07.129 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:07.129 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:07.129 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:225581 3.15 3:ab407115:::rbd_object_map.112d838b9898:head [call lock.unlock in=10b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e188)", 2024-09-14T06:32:07.129 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:31:52.743472+0000", 2024-09-14T06:32:07.129 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 14.298225266999999, 2024-09-14T06:32:07.129 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.007270501, 2024-09-14T06:32:07.129 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:32:07.129 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:32:07.130 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:32:07.130 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:32:07.130 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:32:07.130 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:07.130 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 225581 2024-09-14T06:32:07.130 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:07.130 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:32:07.130 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:07.130 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:32:07.130 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:31:52.743472+0000", 2024-09-14T06:32:07.130 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:07.130 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:07.130 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:07.130 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:32:07.130 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:31:52.743472+0000", 2024-09-14T06:32:07.130 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:07.131 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:07.131 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:07.131 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:32:07.131 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:31:52.743473+0000", 2024-09-14T06:32:07.131 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.1999999999999998e-07 2024-09-14T06:32:07.131 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:07.131 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:07.131 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:32:07.131 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:31:52.743478+0000", 2024-09-14T06:32:07.131 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.515e-06 2024-09-14T06:32:07.131 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:07.131 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:07.131 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:32:07.131 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:31:52.743489+0000", 2024-09-14T06:32:07.131 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.1411e-05 2024-09-14T06:32:07.132 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:07.132 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:07.132 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:32:07.132 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:31:52.743494+0000", 2024-09-14T06:32:07.132 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.1649999999999997e-06 2024-09-14T06:32:07.132 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:07.132 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:07.132 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:32:07.132 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:31:52.743548+0000", 2024-09-14T06:32:07.132 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.3758999999999998e-05 2024-09-14T06:32:07.132 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:07.132 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:07.132 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "waiting for peered", 2024-09-14T06:32:07.132 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:31:52.743558+0000", 2024-09-14T06:32:07.133 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.0336999999999999e-05 2024-09-14T06:32:07.133 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:07.133 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:07.133 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:32:07.133 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:31:53.747716+0000", 2024-09-14T06:32:07.133 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.004157035 2024-09-14T06:32:07.133 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:07.133 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:07.133 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:32:07.133 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:31:53.747831+0000", 2024-09-14T06:32:07.133 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.000115394 2024-09-14T06:32:07.133 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:07.133 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:07.133 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "waiting for subops from 3", 2024-09-14T06:32:07.133 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:31:53.748002+0000", 2024-09-14T06:32:07.134 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00017085099999999999 2024-09-14T06:32:07.134 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:07.134 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:07.134 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:32:07.134 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:31:53.749536+0000", 2024-09-14T06:32:07.134 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.0015339990000000001 2024-09-14T06:32:07.134 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:07.134 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:07.134 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "op_commit", 2024-09-14T06:32:07.134 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:31:53.750672+0000", 2024-09-14T06:32:07.134 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.001135973 2024-09-14T06:32:07.134 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:07.134 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:07.134 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "commit_sent", 2024-09-14T06:32:07.135 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:31:53.750704+0000", 2024-09-14T06:32:07.135 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.1791000000000001e-05 2024-09-14T06:32:07.135 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:07.135 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:07.135 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:32:07.135 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:31:53.750743+0000", 2024-09-14T06:32:07.135 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.9450999999999999e-05 2024-09-14T06:32:07.135 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:07.135 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:32:07.135 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:07.135 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:07.135 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:32:07.135 INFO:teuthology.orchestra.run.smithi138.stdout:} 2024-09-14T06:32:07.137 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-14T06:32:07.137 INFO:tasks.ceph.osd.0.smithi094.stderr:2024-09-14T06:32:07.136+0000 7f07c9333640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33636) UID: 0 2024-09-14T06:32:07.238 INFO:tasks.ceph.osd.5.smithi138.stderr:2024-09-14T06:32:07.237+0000 7fe13746f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33618) UID: 0 2024-09-14T06:32:07.241 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-14T06:32:07.241 INFO:teuthology.orchestra.run.smithi094.stdout: "ops": [], 2024-09-14T06:32:07.241 INFO:teuthology.orchestra.run.smithi094.stdout: "num_ops": 0 2024-09-14T06:32:07.242 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-14T06:32:07.250 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-14T06:32:07.338 INFO:tasks.ceph.osd.0.smithi094.stderr:2024-09-14T06:32:07.337+0000 7f07c9333640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33636) UID: 0 2024-09-14T06:32:07.352 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-14T06:32:07.353 INFO:teuthology.orchestra.run.smithi094.stdout: "ops": [], 2024-09-14T06:32:07.353 INFO:teuthology.orchestra.run.smithi094.stdout: "complaint_time": 30, 2024-09-14T06:32:07.353 INFO:teuthology.orchestra.run.smithi094.stdout: "num_blocked_ops": 0 2024-09-14T06:32:07.353 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-14T06:32:07.361 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-14T06:32:07.445 INFO:tasks.ceph.osd.0.smithi094.stderr:2024-09-14T06:32:07.444+0000 7f07c9333640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33636) UID: 0 2024-09-14T06:32:07.506 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-14T06:32:07.506 INFO:teuthology.orchestra.run.smithi094.stdout: "size": 20, 2024-09-14T06:32:07.506 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 600, 2024-09-14T06:32:07.506 INFO:teuthology.orchestra.run.smithi094.stdout: "ops": [ 2024-09-14T06:32:07.506 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:07.506 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:115193 3.1 3:856be886:::rbd_data.112d4d24c558.0000000000000004:29 [read 1523712~16384 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e96)", 2024-09-14T06:32:07.506 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:30:17.939854+0000", 2024-09-14T06:32:07.507 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 109.564497041, 2024-09-14T06:32:07.507 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.073897516999999996, 2024-09-14T06:32:07.507 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:32:07.507 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:32:07.507 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:32:07.507 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:32:07.507 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:32:07.507 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:07.507 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 115193 2024-09-14T06:32:07.507 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:07.507 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:32:07.507 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:07.507 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:32:07.507 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.939854+0000", 2024-09-14T06:32:07.508 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:07.508 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:07.508 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:07.508 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:32:07.508 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.939854+0000", 2024-09-14T06:32:07.508 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:07.508 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:07.508 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:07.508 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:32:07.508 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.939854+0000", 2024-09-14T06:32:07.508 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.9500000000000002e-07 2024-09-14T06:32:07.508 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:07.508 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:07.508 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:32:07.509 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.939856+0000", 2024-09-14T06:32:07.509 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.0959999999999999e-06 2024-09-14T06:32:07.509 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:07.509 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:07.509 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:32:07.509 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.939863+0000", 2024-09-14T06:32:07.509 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.1129999999999997e-06 2024-09-14T06:32:07.509 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:07.509 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:07.509 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:32:07.509 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.939867+0000", 2024-09-14T06:32:07.509 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.8350000000000003e-06 2024-09-14T06:32:07.509 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:07.509 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:07.509 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:32:07.510 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.013626+0000", 2024-09-14T06:32:07.510 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.073758799999999999 2024-09-14T06:32:07.510 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:07.510 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:07.510 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:32:07.510 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.013690+0000", 2024-09-14T06:32:07.510 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.3918999999999996e-05 2024-09-14T06:32:07.510 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:07.510 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:07.510 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:32:07.510 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.013751+0000", 2024-09-14T06:32:07.510 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.1159000000000002e-05 2024-09-14T06:32:07.510 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:07.511 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:32:07.511 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:07.511 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:07.511 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:07.511 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:115194 3.1 3:856be886:::rbd_data.112d4d24c558.0000000000000004:29 [read 1769472~16384 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e96)", 2024-09-14T06:32:07.511 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:30:17.939874+0000", 2024-09-14T06:32:07.511 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 109.56447673, 2024-09-14T06:32:07.511 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.074016988000000006, 2024-09-14T06:32:07.511 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:32:07.511 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:32:07.511 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:32:07.511 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:32:07.511 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:32:07.511 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:07.512 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 115194 2024-09-14T06:32:07.512 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:07.512 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:32:07.512 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:07.512 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:32:07.512 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.939874+0000", 2024-09-14T06:32:07.512 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:07.512 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:07.512 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:07.512 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:32:07.512 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.939874+0000", 2024-09-14T06:32:07.512 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:07.512 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:07.512 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:07.513 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:32:07.513 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.939875+0000", 2024-09-14T06:32:07.513 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.2600000000000002e-07 2024-09-14T06:32:07.513 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:07.513 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:07.513 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:32:07.513 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.939877+0000", 2024-09-14T06:32:07.513 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.9420000000000002e-06 2024-09-14T06:32:07.513 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:07.513 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:07.513 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:32:07.513 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.939882+0000", 2024-09-14T06:32:07.513 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.8749999999999997e-06 2024-09-14T06:32:07.513 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:07.514 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:07.514 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:32:07.514 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.939887+0000", 2024-09-14T06:32:07.514 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.741e-06 2024-09-14T06:32:07.514 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:07.514 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:07.514 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:32:07.514 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.013767+0000", 2024-09-14T06:32:07.514 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.073879720999999995 2024-09-14T06:32:07.514 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:07.514 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:07.514 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:32:07.514 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.013832+0000", 2024-09-14T06:32:07.514 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.5312000000000001e-05 2024-09-14T06:32:07.515 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:07.515 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:07.515 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:32:07.515 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.013891+0000", 2024-09-14T06:32:07.515 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.8870999999999999e-05 2024-09-14T06:32:07.515 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:07.515 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:32:07.515 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:07.515 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:07.515 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:07.515 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:115195 3.1 3:856be886:::rbd_data.112d4d24c558.0000000000000004:29 [read 2015232~16384 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e96)", 2024-09-14T06:32:07.515 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:30:17.939903+0000", 2024-09-14T06:32:07.515 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 109.56444754499999, 2024-09-14T06:32:07.515 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.074128199000000006, 2024-09-14T06:32:07.515 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:32:07.516 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:32:07.516 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:32:07.516 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:32:07.516 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:32:07.516 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:07.516 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 115195 2024-09-14T06:32:07.516 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:07.516 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:32:07.516 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:07.516 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:32:07.516 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.939903+0000", 2024-09-14T06:32:07.516 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:07.516 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:07.516 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:07.516 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:32:07.517 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.939903+0000", 2024-09-14T06:32:07.517 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:07.517 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:07.517 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:07.517 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:32:07.517 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.939904+0000", 2024-09-14T06:32:07.517 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.5799999999999999e-07 2024-09-14T06:32:07.517 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:07.517 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:07.517 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:32:07.517 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.939906+0000", 2024-09-14T06:32:07.517 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.2350000000000002e-06 2024-09-14T06:32:07.517 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:07.518 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:07.518 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:32:07.518 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.939913+0000", 2024-09-14T06:32:07.518 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.5649999999999998e-06 2024-09-14T06:32:07.518 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:07.518 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:07.518 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:32:07.518 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.939918+0000", 2024-09-14T06:32:07.518 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.784e-06 2024-09-14T06:32:07.518 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:07.518 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:07.518 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:32:07.518 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.013921+0000", 2024-09-14T06:32:07.518 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.074002911000000005 2024-09-14T06:32:07.518 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:07.519 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:07.519 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:32:07.519 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.013971+0000", 2024-09-14T06:32:07.519 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.9551e-05 2024-09-14T06:32:07.519 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:07.519 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:07.519 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:32:07.519 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.014031+0000", 2024-09-14T06:32:07.519 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.0594999999999998e-05 2024-09-14T06:32:07.519 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:07.519 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:32:07.519 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:07.519 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:07.519 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:07.520 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:115196 3.1 3:856be886:::rbd_data.112d4d24c558.0000000000000004:29 [read 2260992~16384 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e96)", 2024-09-14T06:32:07.520 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:30:17.939925+0000", 2024-09-14T06:32:07.520 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 109.564425353, 2024-09-14T06:32:07.520 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.074320137999999994, 2024-09-14T06:32:07.520 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:32:07.520 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:32:07.520 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:32:07.520 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:32:07.520 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:32:07.520 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:07.520 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 115196 2024-09-14T06:32:07.520 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:07.520 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:32:07.520 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:07.521 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:32:07.521 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.939925+0000", 2024-09-14T06:32:07.521 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:07.521 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:07.521 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:07.521 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:32:07.521 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.939925+0000", 2024-09-14T06:32:07.521 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:07.521 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:07.521 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:07.521 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:32:07.521 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.939926+0000", 2024-09-14T06:32:07.521 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.9999999999999998e-07 2024-09-14T06:32:07.521 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:07.522 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:07.522 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:32:07.522 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.939928+0000", 2024-09-14T06:32:07.522 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.9549999999999999e-06 2024-09-14T06:32:07.522 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:07.522 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:07.522 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:32:07.522 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.939934+0000", 2024-09-14T06:32:07.522 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.127e-06 2024-09-14T06:32:07.522 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:07.522 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:07.522 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:32:07.522 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.939939+0000", 2024-09-14T06:32:07.522 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.775e-06 2024-09-14T06:32:07.523 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:07.523 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:07.523 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:32:07.523 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.014075+0000", 2024-09-14T06:32:07.523 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.074136598999999997 2024-09-14T06:32:07.523 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:07.523 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:07.523 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:32:07.523 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.014162+0000", 2024-09-14T06:32:07.523 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.6477999999999994e-05 2024-09-14T06:32:07.523 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:07.523 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:07.523 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:32:07.523 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.014246+0000", 2024-09-14T06:32:07.523 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.3703999999999997e-05 2024-09-14T06:32:07.524 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:07.524 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:32:07.524 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:07.524 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:07.524 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:07.524 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:115197 3.1 3:856be886:::rbd_data.112d4d24c558.0000000000000004:29 [read 2506752~16384 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e96)", 2024-09-14T06:32:07.524 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:30:17.939969+0000", 2024-09-14T06:32:07.524 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 109.564381868, 2024-09-14T06:32:07.524 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.074528544000000002, 2024-09-14T06:32:07.524 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:32:07.524 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:32:07.524 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:32:07.524 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:32:07.524 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:32:07.524 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:07.525 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 115197 2024-09-14T06:32:07.525 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:07.525 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:32:07.525 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:07.525 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:32:07.525 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.939969+0000", 2024-09-14T06:32:07.525 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:07.525 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:07.525 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:07.525 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:32:07.525 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.939969+0000", 2024-09-14T06:32:07.525 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:07.525 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:07.525 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:07.525 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:32:07.526 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.939970+0000", 2024-09-14T06:32:07.526 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.3600000000000003e-07 2024-09-14T06:32:07.526 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:07.526 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:07.526 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:32:07.526 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.939972+0000", 2024-09-14T06:32:07.526 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.3180000000000001e-06 2024-09-14T06:32:07.526 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:07.526 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:07.526 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:32:07.526 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.939979+0000", 2024-09-14T06:32:07.526 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.8800000000000002e-06 2024-09-14T06:32:07.526 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:07.526 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:07.527 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:32:07.527 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.939984+0000", 2024-09-14T06:32:07.527 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.3510000000000001e-06 2024-09-14T06:32:07.527 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:07.527 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:07.527 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:32:07.527 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.014302+0000", 2024-09-14T06:32:07.527 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.074317416999999997 2024-09-14T06:32:07.527 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:07.527 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:07.527 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:32:07.527 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.014392+0000", 2024-09-14T06:32:07.527 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.0181999999999998e-05 2024-09-14T06:32:07.527 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:07.527 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:07.528 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:32:07.528 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.014497+0000", 2024-09-14T06:32:07.528 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00010576 2024-09-14T06:32:07.528 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:07.528 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:32:07.528 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:07.528 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:07.528 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:07.528 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:115198 3.1 3:856be886:::rbd_data.112d4d24c558.0000000000000004:29 [read 2752512~16384 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e96)", 2024-09-14T06:32:07.528 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:30:17.940004+0000", 2024-09-14T06:32:07.528 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 109.56434658800001, 2024-09-14T06:32:07.528 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.074736367999999997, 2024-09-14T06:32:07.528 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:32:07.528 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:32:07.529 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:32:07.529 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:32:07.529 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:32:07.529 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:07.529 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 115198 2024-09-14T06:32:07.529 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:07.529 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:32:07.529 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:07.529 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:32:07.529 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940004+0000", 2024-09-14T06:32:07.529 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:07.529 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:07.529 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:07.529 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:32:07.529 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940004+0000", 2024-09-14T06:32:07.529 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:07.530 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:07.530 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:07.530 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:32:07.530 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940005+0000", 2024-09-14T06:32:07.530 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.2200000000000004e-07 2024-09-14T06:32:07.530 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:07.530 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:07.530 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:32:07.530 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940007+0000", 2024-09-14T06:32:07.530 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.322e-06 2024-09-14T06:32:07.530 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:07.530 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:07.530 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:32:07.530 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940014+0000", 2024-09-14T06:32:07.531 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.6220000000000003e-06 2024-09-14T06:32:07.531 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:07.531 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:07.531 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:32:07.531 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940019+0000", 2024-09-14T06:32:07.531 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.6380000000000001e-06 2024-09-14T06:32:07.531 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:07.531 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:07.531 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:32:07.531 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.014562+0000", 2024-09-14T06:32:07.531 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.074542320999999995 2024-09-14T06:32:07.531 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:07.531 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:07.531 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:32:07.531 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.014646+0000", 2024-09-14T06:32:07.532 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.4079999999999995e-05 2024-09-14T06:32:07.532 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:07.532 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:07.532 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:32:07.532 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.014741+0000", 2024-09-14T06:32:07.532 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.4763000000000001e-05 2024-09-14T06:32:07.532 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:07.532 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:32:07.532 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:07.532 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:07.532 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:07.532 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:115199 3.1 3:856be886:::rbd_data.112d4d24c558.0000000000000004:29 [read 2998272~16384 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e96)", 2024-09-14T06:32:07.532 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:30:17.940040+0000", 2024-09-14T06:32:07.532 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 109.56431083299999, 2024-09-14T06:32:07.533 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.074957134999999994, 2024-09-14T06:32:07.533 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:32:07.533 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:32:07.533 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:32:07.533 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:32:07.533 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:32:07.533 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:07.533 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 115199 2024-09-14T06:32:07.533 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:07.533 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:32:07.533 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:07.533 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:32:07.533 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940040+0000", 2024-09-14T06:32:07.534 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:07.534 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:07.534 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:07.534 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:32:07.534 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940040+0000", 2024-09-14T06:32:07.534 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:07.534 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:07.534 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:07.534 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:32:07.534 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940041+0000", 2024-09-14T06:32:07.534 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.4000000000000001e-07 2024-09-14T06:32:07.534 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:07.534 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:07.534 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:32:07.535 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940046+0000", 2024-09-14T06:32:07.535 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.7250000000000002e-06 2024-09-14T06:32:07.535 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:07.535 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:07.535 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:32:07.535 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940053+0000", 2024-09-14T06:32:07.535 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.6109999999999999e-06 2024-09-14T06:32:07.535 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:07.535 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:07.535 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:32:07.535 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940058+0000", 2024-09-14T06:32:07.535 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.1590000000000001e-06 2024-09-14T06:32:07.535 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:07.535 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:07.536 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:32:07.536 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.014768+0000", 2024-09-14T06:32:07.536 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.074709624000000002 2024-09-14T06:32:07.536 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:07.536 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:07.536 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:32:07.536 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.014907+0000", 2024-09-14T06:32:07.536 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000139665 2024-09-14T06:32:07.536 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:07.536 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:07.536 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:32:07.536 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.014997+0000", 2024-09-14T06:32:07.536 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.9710999999999995e-05 2024-09-14T06:32:07.536 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:07.537 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:32:07.537 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:07.537 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:07.537 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:07.537 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:115200 3.1 3:856be886:::rbd_data.112d4d24c558.0000000000000004:29 [read 3244032~16384 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e96)", 2024-09-14T06:32:07.537 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:30:17.940080+0000", 2024-09-14T06:32:07.537 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 109.564270628, 2024-09-14T06:32:07.537 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.075137291999999994, 2024-09-14T06:32:07.537 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:32:07.537 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:32:07.537 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:32:07.537 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:32:07.537 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:32:07.537 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:07.537 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 115200 2024-09-14T06:32:07.538 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:07.538 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:32:07.538 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:07.538 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:32:07.538 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940080+0000", 2024-09-14T06:32:07.538 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:07.538 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:07.538 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:07.538 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:32:07.538 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940080+0000", 2024-09-14T06:32:07.538 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:07.538 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:07.538 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:07.539 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:32:07.539 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940081+0000", 2024-09-14T06:32:07.539 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.0800000000000004e-07 2024-09-14T06:32:07.539 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:07.539 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:07.539 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:32:07.539 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940083+0000", 2024-09-14T06:32:07.539 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.3920000000000001e-06 2024-09-14T06:32:07.539 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:07.539 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:07.539 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:32:07.539 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940091+0000", 2024-09-14T06:32:07.539 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.1389999999999995e-06 2024-09-14T06:32:07.539 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:07.540 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:07.540 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:32:07.540 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940097+0000", 2024-09-14T06:32:07.540 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.5550000000000002e-06 2024-09-14T06:32:07.540 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:07.540 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:07.540 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:32:07.540 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.015025+0000", 2024-09-14T06:32:07.540 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.074928579999999995 2024-09-14T06:32:07.540 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:07.540 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:07.540 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:32:07.540 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.015096+0000", 2024-09-14T06:32:07.540 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.0705999999999997e-05 2024-09-14T06:32:07.540 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:07.540 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:07.541 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:32:07.541 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.015217+0000", 2024-09-14T06:32:07.541 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00012131200000000001 2024-09-14T06:32:07.541 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:07.541 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:32:07.541 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:07.541 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:07.541 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:07.541 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:115201 3.1 3:856be886:::rbd_data.112d4d24c558.0000000000000004:29 [read 3489792~16384 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e96)", 2024-09-14T06:32:07.541 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:30:17.940113+0000", 2024-09-14T06:32:07.541 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 109.564237855, 2024-09-14T06:32:07.541 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.075246899000000006, 2024-09-14T06:32:07.541 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:32:07.541 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:32:07.541 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:32:07.542 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:32:07.542 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:32:07.542 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:07.542 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 115201 2024-09-14T06:32:07.542 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:07.542 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:32:07.542 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:07.542 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:32:07.542 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940113+0000", 2024-09-14T06:32:07.542 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:07.542 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:07.542 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:07.542 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:32:07.542 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940113+0000", 2024-09-14T06:32:07.542 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:07.542 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:07.543 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:07.543 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:32:07.543 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940114+0000", 2024-09-14T06:32:07.543 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.1500000000000004e-07 2024-09-14T06:32:07.543 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:07.543 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:07.543 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:32:07.543 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940116+0000", 2024-09-14T06:32:07.543 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.2809999999999998e-06 2024-09-14T06:32:07.543 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:07.543 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:07.543 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:32:07.543 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940123+0000", 2024-09-14T06:32:07.543 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.9229999999999996e-06 2024-09-14T06:32:07.544 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:07.544 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:07.544 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:32:07.544 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940128+0000", 2024-09-14T06:32:07.544 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.7790000000000003e-06 2024-09-14T06:32:07.544 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:07.544 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:07.544 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:32:07.544 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.015253+0000", 2024-09-14T06:32:07.544 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.075125792999999996 2024-09-14T06:32:07.544 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:07.544 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:07.544 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:32:07.544 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.015297+0000", 2024-09-14T06:32:07.545 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.3847000000000002e-05 2024-09-14T06:32:07.545 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:07.545 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:07.545 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:32:07.545 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.015360+0000", 2024-09-14T06:32:07.545 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.2661000000000002e-05 2024-09-14T06:32:07.545 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:07.545 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:32:07.545 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:07.545 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:07.545 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:07.545 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:115202 3.1 3:856be886:::rbd_data.112d4d24c558.0000000000000004:29 [read 3735552~16384 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e96)", 2024-09-14T06:32:07.545 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:30:17.940135+0000", 2024-09-14T06:32:07.546 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 109.56421607599999, 2024-09-14T06:32:07.546 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.075451105000000004, 2024-09-14T06:32:07.546 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:32:07.546 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:32:07.546 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:32:07.546 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:32:07.546 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:32:07.546 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:07.546 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 115202 2024-09-14T06:32:07.546 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:07.546 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:32:07.546 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:07.546 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:32:07.546 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940135+0000", 2024-09-14T06:32:07.547 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:07.547 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:07.547 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:07.547 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:32:07.547 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940135+0000", 2024-09-14T06:32:07.547 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:07.547 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:07.547 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:07.547 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:32:07.547 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940135+0000", 2024-09-14T06:32:07.547 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.0999999999999999e-07 2024-09-14T06:32:07.547 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:07.547 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:07.548 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:32:07.548 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940137+0000", 2024-09-14T06:32:07.548 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.0020000000000001e-06 2024-09-14T06:32:07.548 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:07.548 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:07.548 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:32:07.548 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940143+0000", 2024-09-14T06:32:07.548 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.9850000000000003e-06 2024-09-14T06:32:07.548 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:07.548 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:07.548 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:32:07.548 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940148+0000", 2024-09-14T06:32:07.548 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.9880000000000004e-06 2024-09-14T06:32:07.549 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:07.549 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:07.549 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:32:07.549 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.015400+0000", 2024-09-14T06:32:07.549 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.075252186999999998 2024-09-14T06:32:07.549 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:07.549 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:07.549 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:32:07.549 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.015481+0000", 2024-09-14T06:32:07.549 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.0698000000000006e-05 2024-09-14T06:32:07.549 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:07.549 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:07.549 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:32:07.549 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.015586+0000", 2024-09-14T06:32:07.550 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00010483499999999999 2024-09-14T06:32:07.550 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:07.550 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:32:07.550 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:07.550 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:07.550 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:07.550 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:115203 3.1 3:856be886:::rbd_data.112d4d24c558.0000000000000004:29 [read 3981312~16384 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e96)", 2024-09-14T06:32:07.550 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:30:17.940156+0000", 2024-09-14T06:32:07.550 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 109.564194851, 2024-09-14T06:32:07.550 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.075620860999999998, 2024-09-14T06:32:07.550 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:32:07.550 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:32:07.550 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:32:07.550 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:32:07.551 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:32:07.551 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:07.551 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 115203 2024-09-14T06:32:07.551 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:07.551 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:32:07.551 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:07.551 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:32:07.551 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940156+0000", 2024-09-14T06:32:07.551 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:07.551 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:07.551 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:07.551 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:32:07.551 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940156+0000", 2024-09-14T06:32:07.552 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:07.552 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:07.552 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:07.552 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:32:07.552 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940156+0000", 2024-09-14T06:32:07.552 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.1500000000000005e-07 2024-09-14T06:32:07.552 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:07.552 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:07.552 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:32:07.552 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940158+0000", 2024-09-14T06:32:07.552 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.967e-06 2024-09-14T06:32:07.552 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:07.552 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:07.552 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:32:07.553 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940165+0000", 2024-09-14T06:32:07.553 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.6939999999999998e-06 2024-09-14T06:32:07.553 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:07.553 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:07.553 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:32:07.553 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940170+0000", 2024-09-14T06:32:07.553 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.7539999999999997e-06 2024-09-14T06:32:07.553 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:07.553 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:07.553 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:32:07.553 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.015633+0000", 2024-09-14T06:32:07.553 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.075463334000000007 2024-09-14T06:32:07.553 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:07.554 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:07.554 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:32:07.554 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.015693+0000", 2024-09-14T06:32:07.554 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.9367000000000003e-05 2024-09-14T06:32:07.554 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:07.554 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:07.554 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:32:07.554 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.015777+0000", 2024-09-14T06:32:07.554 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.4229999999999998e-05 2024-09-14T06:32:07.554 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:07.554 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:32:07.554 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:07.554 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:07.554 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:07.555 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:115204 3.1 3:856be886:::rbd_data.112d4d24c558.0000000000000004:29 [read 4227072~16384 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e96)", 2024-09-14T06:32:07.555 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:30:17.940176+0000", 2024-09-14T06:32:07.555 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 109.564174318, 2024-09-14T06:32:07.555 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.075785505000000003, 2024-09-14T06:32:07.555 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:32:07.555 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:32:07.555 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:32:07.555 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:32:07.555 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:32:07.555 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:07.555 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 115204 2024-09-14T06:32:07.555 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:07.555 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:32:07.555 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:07.556 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:32:07.556 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940176+0000", 2024-09-14T06:32:07.556 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:07.556 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:07.556 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:07.556 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:32:07.556 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940176+0000", 2024-09-14T06:32:07.556 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:07.556 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:07.556 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:07.556 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:32:07.556 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940177+0000", 2024-09-14T06:32:07.556 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.7899999999999999e-07 2024-09-14T06:32:07.556 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:07.557 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:07.557 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:32:07.557 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940179+0000", 2024-09-14T06:32:07.557 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.103e-06 2024-09-14T06:32:07.557 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:07.557 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:07.557 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:32:07.557 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940185+0000", 2024-09-14T06:32:07.557 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.7239999999999997e-06 2024-09-14T06:32:07.557 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:07.557 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:07.557 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:32:07.558 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940190+0000", 2024-09-14T06:32:07.558 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.7269999999999998e-06 2024-09-14T06:32:07.558 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:07.558 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:07.558 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:32:07.558 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.015814+0000", 2024-09-14T06:32:07.558 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.075624114000000006 2024-09-14T06:32:07.558 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:07.558 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:07.558 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:32:07.558 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.015862+0000", 2024-09-14T06:32:07.558 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.8770999999999998e-05 2024-09-14T06:32:07.558 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:07.558 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:07.559 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:32:07.559 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.015962+0000", 2024-09-14T06:32:07.559 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.9586999999999994e-05 2024-09-14T06:32:07.559 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:07.559 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:32:07.559 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:07.559 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:07.559 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:07.559 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:115205 3.1 3:856be886:::rbd_data.112d4d24c558.0000000000000004:29 [read 4472832~16384 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e96)", 2024-09-14T06:32:07.559 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:30:17.940197+0000", 2024-09-14T06:32:07.559 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 109.564153609, 2024-09-14T06:32:07.559 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.076068920999999998, 2024-09-14T06:32:07.559 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:32:07.559 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:32:07.559 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:32:07.560 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:32:07.560 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:32:07.560 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:07.560 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 115205 2024-09-14T06:32:07.560 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:07.560 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:32:07.560 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:07.560 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:32:07.560 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940197+0000", 2024-09-14T06:32:07.560 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:07.560 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:07.560 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:07.560 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:32:07.561 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940197+0000", 2024-09-14T06:32:07.561 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:07.561 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:07.561 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:07.561 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:32:07.561 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940198+0000", 2024-09-14T06:32:07.561 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.5999999999999999e-07 2024-09-14T06:32:07.561 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:07.561 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:07.561 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:32:07.561 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940200+0000", 2024-09-14T06:32:07.561 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.0899999999999999e-06 2024-09-14T06:32:07.561 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:07.561 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:07.561 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:32:07.562 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940206+0000", 2024-09-14T06:32:07.562 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.8710000000000002e-06 2024-09-14T06:32:07.562 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:07.562 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:07.562 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:32:07.562 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940211+0000", 2024-09-14T06:32:07.562 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.9230000000000001e-06 2024-09-14T06:32:07.562 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:07.562 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:07.562 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:32:07.562 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.016059+0000", 2024-09-14T06:32:07.562 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.075848051999999999 2024-09-14T06:32:07.562 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:07.562 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:07.563 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:32:07.563 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.016151+0000", 2024-09-14T06:32:07.563 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.1982000000000001e-05 2024-09-14T06:32:07.563 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:07.563 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:07.563 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:32:07.563 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.016266+0000", 2024-09-14T06:32:07.563 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000115543 2024-09-14T06:32:07.563 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:07.563 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:32:07.563 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:07.563 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:07.563 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:07.563 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:115206 3.1 3:856be886:::rbd_data.112d4d24c558.0000000000000004:29 [read 4718592~16384 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e96)", 2024-09-14T06:32:07.564 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:30:17.940223+0000", 2024-09-14T06:32:07.564 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 109.564127519, 2024-09-14T06:32:07.564 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.076265115999999994, 2024-09-14T06:32:07.564 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:32:07.564 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:32:07.564 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:32:07.564 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:32:07.564 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:32:07.564 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:07.564 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 115206 2024-09-14T06:32:07.564 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:07.564 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:32:07.564 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:07.564 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:32:07.565 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940223+0000", 2024-09-14T06:32:07.565 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:07.565 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:07.565 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:07.565 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:32:07.565 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940223+0000", 2024-09-14T06:32:07.565 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:07.565 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:07.565 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:07.565 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:32:07.565 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940224+0000", 2024-09-14T06:32:07.565 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.0399999999999996e-07 2024-09-14T06:32:07.565 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:07.565 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:07.566 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:32:07.566 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940226+0000", 2024-09-14T06:32:07.566 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.1780000000000002e-06 2024-09-14T06:32:07.566 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:07.566 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:07.566 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:32:07.566 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940232+0000", 2024-09-14T06:32:07.566 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.5010000000000001e-06 2024-09-14T06:32:07.566 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:07.566 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:07.566 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:32:07.566 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940237+0000", 2024-09-14T06:32:07.566 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.0250000000000002e-06 2024-09-14T06:32:07.567 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:07.567 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:07.567 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:32:07.567 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.016305+0000", 2024-09-14T06:32:07.567 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.076067262999999996 2024-09-14T06:32:07.567 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:07.567 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:07.567 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:32:07.567 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.016383+0000", 2024-09-14T06:32:07.567 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.8578000000000006e-05 2024-09-14T06:32:07.567 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:07.567 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:07.567 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:32:07.567 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.016488+0000", 2024-09-14T06:32:07.568 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000105067 2024-09-14T06:32:07.568 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:07.568 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:32:07.568 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:07.568 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:07.568 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:07.568 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:115207 3.1 3:856be886:::rbd_data.112d4d24c558.0000000000000004:29 [read 4964352~16384 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e96)", 2024-09-14T06:32:07.568 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:30:17.940244+0000", 2024-09-14T06:32:07.568 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 109.564106403, 2024-09-14T06:32:07.568 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.076371154999999996, 2024-09-14T06:32:07.568 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:32:07.568 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:32:07.568 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:32:07.569 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:32:07.569 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:32:07.569 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:07.569 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 115207 2024-09-14T06:32:07.569 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:07.569 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:32:07.569 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:07.569 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:32:07.569 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940244+0000", 2024-09-14T06:32:07.569 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:07.569 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:07.569 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:07.569 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:32:07.569 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940244+0000", 2024-09-14T06:32:07.570 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:07.570 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:07.570 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:07.570 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:32:07.570 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940245+0000", 2024-09-14T06:32:07.570 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.1200000000000003e-07 2024-09-14T06:32:07.570 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:07.570 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:07.570 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:32:07.570 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940247+0000", 2024-09-14T06:32:07.570 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.0030000000000001e-06 2024-09-14T06:32:07.570 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:07.570 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:07.571 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:32:07.571 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940252+0000", 2024-09-14T06:32:07.571 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.5520000000000004e-06 2024-09-14T06:32:07.571 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:07.571 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:07.571 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:32:07.571 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940257+0000", 2024-09-14T06:32:07.571 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.6759999999999998e-06 2024-09-14T06:32:07.571 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:07.571 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:07.571 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:32:07.571 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.016521+0000", 2024-09-14T06:32:07.571 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.076263499999999998 2024-09-14T06:32:07.571 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:07.572 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:07.572 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:32:07.572 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.016562+0000", 2024-09-14T06:32:07.572 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.0967000000000003e-05 2024-09-14T06:32:07.572 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:07.572 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:07.572 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:32:07.572 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.016616+0000", 2024-09-14T06:32:07.572 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.3944999999999999e-05 2024-09-14T06:32:07.572 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:07.572 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:32:07.572 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:07.572 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:07.572 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:07.573 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:115208 3.1 3:856be886:::rbd_data.112d4d24c558.0000000000000004:29 [read 5210112~16384 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e96)", 2024-09-14T06:32:07.573 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:30:17.940264+0000", 2024-09-14T06:32:07.573 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 109.564086546, 2024-09-14T06:32:07.573 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.076558288000000002, 2024-09-14T06:32:07.573 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:32:07.573 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:32:07.573 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:32:07.573 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:32:07.573 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:32:07.573 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:07.573 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 115208 2024-09-14T06:32:07.573 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:07.573 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:32:07.573 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:07.573 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:32:07.574 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940264+0000", 2024-09-14T06:32:07.574 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:07.574 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:07.574 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:07.574 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:32:07.574 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940264+0000", 2024-09-14T06:32:07.574 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:07.574 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:07.574 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:07.574 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:32:07.574 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940265+0000", 2024-09-14T06:32:07.574 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.0500000000000004e-07 2024-09-14T06:32:07.574 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:07.575 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:07.575 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:32:07.575 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940267+0000", 2024-09-14T06:32:07.575 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.9810000000000002e-06 2024-09-14T06:32:07.575 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:07.575 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:07.575 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:32:07.575 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940272+0000", 2024-09-14T06:32:07.575 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.6999999999999996e-06 2024-09-14T06:32:07.575 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:07.575 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:07.575 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:32:07.575 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940277+0000", 2024-09-14T06:32:07.576 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.6480000000000002e-06 2024-09-14T06:32:07.576 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:07.576 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:07.576 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:32:07.576 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.016650+0000", 2024-09-14T06:32:07.576 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.076373089000000005 2024-09-14T06:32:07.576 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:07.576 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:07.576 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:32:07.576 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.016724+0000", 2024-09-14T06:32:07.576 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.4105999999999998e-05 2024-09-14T06:32:07.576 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:07.576 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:07.576 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:32:07.576 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.016823+0000", 2024-09-14T06:32:07.577 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.8258999999999996e-05 2024-09-14T06:32:07.577 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:07.577 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:32:07.577 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:07.577 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:07.577 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:07.577 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:115209 3.1 3:856be886:::rbd_data.112d4d24c558.0000000000000004:29 [read 5455872~16384 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e96)", 2024-09-14T06:32:07.577 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:30:17.940283+0000", 2024-09-14T06:32:07.577 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 109.56406751999999, 2024-09-14T06:32:07.577 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.076724029999999999, 2024-09-14T06:32:07.577 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:32:07.577 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:32:07.577 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:32:07.577 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:32:07.578 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:32:07.578 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:07.578 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 115209 2024-09-14T06:32:07.578 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:07.578 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:32:07.578 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:07.578 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:32:07.578 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940283+0000", 2024-09-14T06:32:07.578 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:07.578 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:07.578 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:07.578 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:32:07.578 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940283+0000", 2024-09-14T06:32:07.578 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:07.578 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:07.579 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:07.579 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:32:07.579 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940284+0000", 2024-09-14T06:32:07.579 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.6700000000000003e-07 2024-09-14T06:32:07.579 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:07.579 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:07.579 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:32:07.579 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940286+0000", 2024-09-14T06:32:07.579 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.9520000000000001e-06 2024-09-14T06:32:07.579 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:07.579 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:07.579 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:32:07.579 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940292+0000", 2024-09-14T06:32:07.579 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.7339999999999996e-06 2024-09-14T06:32:07.579 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:07.580 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:07.580 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:32:07.580 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940296+0000", 2024-09-14T06:32:07.580 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.7700000000000001e-06 2024-09-14T06:32:07.580 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:07.580 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:07.580 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:32:07.580 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.016859+0000", 2024-09-14T06:32:07.580 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.07656317 2024-09-14T06:32:07.580 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:07.580 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:07.580 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:32:07.580 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.016929+0000", 2024-09-14T06:32:07.580 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.9679999999999997e-05 2024-09-14T06:32:07.581 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:07.581 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:07.581 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:32:07.581 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.017007+0000", 2024-09-14T06:32:07.581 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.8157000000000004e-05 2024-09-14T06:32:07.581 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:07.581 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:32:07.581 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:07.581 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:07.581 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:07.581 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:115210 3.1 3:856be886:::rbd_data.112d4d24c558.0000000000000004:29 [read 5701632~16384 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e96)", 2024-09-14T06:32:07.581 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:30:17.940303+0000", 2024-09-14T06:32:07.581 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 109.564048003, 2024-09-14T06:32:07.581 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.076821358000000006, 2024-09-14T06:32:07.582 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:32:07.582 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:32:07.582 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:32:07.582 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:32:07.582 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:32:07.582 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:07.582 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 115210 2024-09-14T06:32:07.582 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:07.582 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:32:07.582 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:07.582 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:32:07.582 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940303+0000", 2024-09-14T06:32:07.582 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:07.582 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:07.582 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:07.583 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:32:07.583 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940303+0000", 2024-09-14T06:32:07.583 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:07.583 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:07.583 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:07.583 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:32:07.583 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940303+0000", 2024-09-14T06:32:07.583 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.3000000000000001e-07 2024-09-14T06:32:07.583 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:07.583 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:07.583 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:32:07.583 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940306+0000", 2024-09-14T06:32:07.583 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.255e-06 2024-09-14T06:32:07.584 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:07.584 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:07.584 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:32:07.584 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940312+0000", 2024-09-14T06:32:07.584 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.9710000000000001e-06 2024-09-14T06:32:07.584 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:07.584 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:07.584 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:32:07.584 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940316+0000", 2024-09-14T06:32:07.584 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.5340000000000001e-06 2024-09-14T06:32:07.584 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:07.584 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:07.584 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:32:07.584 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.017033+0000", 2024-09-14T06:32:07.585 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.076717169000000002 2024-09-14T06:32:07.585 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:07.585 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:07.585 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:32:07.585 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.017073+0000", 2024-09-14T06:32:07.585 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.9582000000000002e-05 2024-09-14T06:32:07.585 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:07.585 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:07.585 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:32:07.585 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.017124+0000", 2024-09-14T06:32:07.585 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.1316999999999997e-05 2024-09-14T06:32:07.585 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:07.585 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:32:07.585 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:07.586 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:07.586 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:07.586 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:115219 3.1 3:856be886:::rbd_data.112d4d24c558.0000000000000004:29 [read 65536~16384 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e96)", 2024-09-14T06:32:07.586 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:30:17.940323+0000", 2024-09-14T06:32:07.586 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 109.564028142, 2024-09-14T06:32:07.586 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.076915205, 2024-09-14T06:32:07.586 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:32:07.586 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:32:07.586 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:32:07.586 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:32:07.586 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:32:07.586 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:07.587 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 115219 2024-09-14T06:32:07.587 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:07.587 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:32:07.587 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:07.587 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:32:07.587 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940323+0000", 2024-09-14T06:32:07.587 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:07.587 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:07.587 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:07.587 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:32:07.587 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940323+0000", 2024-09-14T06:32:07.587 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:07.587 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:07.587 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:07.588 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:32:07.588 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940323+0000", 2024-09-14T06:32:07.588 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.0800000000000005e-07 2024-09-14T06:32:07.588 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:07.588 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:07.588 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:32:07.588 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940328+0000", 2024-09-14T06:32:07.588 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.887e-06 2024-09-14T06:32:07.588 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:07.588 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:07.588 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:32:07.588 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940334+0000", 2024-09-14T06:32:07.588 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.3269999999999997e-06 2024-09-14T06:32:07.588 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:07.589 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:07.589 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:32:07.589 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940339+0000", 2024-09-14T06:32:07.589 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.7960000000000003e-06 2024-09-14T06:32:07.589 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:07.589 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:07.589 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:32:07.589 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.017148+0000", 2024-09-14T06:32:07.589 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.076809125000000006 2024-09-14T06:32:07.589 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:07.589 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:07.589 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:32:07.589 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.017186+0000", 2024-09-14T06:32:07.589 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.7283000000000002e-05 2024-09-14T06:32:07.590 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:07.590 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:07.590 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:32:07.590 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.017238+0000", 2024-09-14T06:32:07.590 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.2278999999999999e-05 2024-09-14T06:32:07.590 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:07.590 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:32:07.590 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:07.590 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:07.590 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:07.590 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:115220 3.1 3:856be886:::rbd_data.112d4d24c558.0000000000000004:29 [read 311296~16384 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e96)", 2024-09-14T06:32:07.590 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:30:17.940346+0000", 2024-09-14T06:32:07.590 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 109.564005013, 2024-09-14T06:32:07.591 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.077033293000000003, 2024-09-14T06:32:07.591 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:32:07.591 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:32:07.591 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:32:07.591 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:32:07.591 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:32:07.591 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:07.591 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 115220 2024-09-14T06:32:07.591 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:07.591 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:32:07.591 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:07.591 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:32:07.591 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940346+0000", 2024-09-14T06:32:07.591 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:07.591 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:07.592 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:07.592 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:32:07.592 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940346+0000", 2024-09-14T06:32:07.592 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:07.592 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:07.592 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:07.592 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:32:07.592 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940346+0000", 2024-09-14T06:32:07.592 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.4300000000000003e-07 2024-09-14T06:32:07.592 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:07.592 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:07.592 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:32:07.592 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940348+0000", 2024-09-14T06:32:07.592 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.0030000000000001e-06 2024-09-14T06:32:07.593 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:07.593 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:07.593 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:32:07.593 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.004655+0000", 2024-09-14T06:32:07.593 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.064306323999999998 2024-09-14T06:32:07.593 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:07.593 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:07.593 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:32:07.593 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.004671+0000", 2024-09-14T06:32:07.593 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.6589999999999999e-05 2024-09-14T06:32:07.593 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:07.593 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:07.593 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:32:07.593 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.017262+0000", 2024-09-14T06:32:07.594 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.012590570000000001 2024-09-14T06:32:07.594 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:07.594 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:07.594 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:32:07.594 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.017321+0000", 2024-09-14T06:32:07.594 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.9210999999999999e-05 2024-09-14T06:32:07.594 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:07.594 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:07.594 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:32:07.594 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.017379+0000", 2024-09-14T06:32:07.594 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.8051999999999999e-05 2024-09-14T06:32:07.594 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:07.594 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:32:07.594 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:07.595 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:07.595 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:32:07.595 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-14T06:32:07.596 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-14T06:32:07.597 INFO:tasks.ceph.osd.5.smithi138.stderr:2024-09-14T06:32:07.595+0000 7fe13746f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33618) UID: 0 2024-09-14T06:32:07.697 INFO:tasks.ceph.osd.5.smithi138.stderr:2024-09-14T06:32:07.696+0000 7fe13746f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33618) UID: 0 2024-09-14T06:32:07.771 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-14T06:32:07.771 INFO:teuthology.orchestra.run.smithi094.stdout: "ops": [], 2024-09-14T06:32:07.771 INFO:teuthology.orchestra.run.smithi094.stdout: "num_ops": 0 2024-09-14T06:32:07.771 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-14T06:32:07.782 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-14T06:32:07.826 INFO:tasks.ceph.osd.3.smithi094.stderr:2024-09-14T06:32:07.825+0000 7feeeafe8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33655) UID: 0 2024-09-14T06:32:07.900 INFO:tasks.ceph.osd.6.smithi138.stderr:2024-09-14T06:32:07.899+0000 7f8cd8849640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33620) UID: 0 2024-09-14T06:32:07.914 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-14T06:32:07.914 INFO:teuthology.orchestra.run.smithi094.stdout: "ops": [], 2024-09-14T06:32:07.914 INFO:teuthology.orchestra.run.smithi094.stdout: "complaint_time": 30, 2024-09-14T06:32:07.914 INFO:teuthology.orchestra.run.smithi094.stdout: "num_blocked_ops": 0 2024-09-14T06:32:07.915 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-14T06:32:07.927 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-14T06:32:08.001 INFO:tasks.ceph.osd.6.smithi138.stderr:2024-09-14T06:32:08.000+0000 7f8cd8849640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33620) UID: 0 2024-09-14T06:32:08.073 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-14T06:32:08.073 INFO:teuthology.orchestra.run.smithi094.stdout: "size": 20, 2024-09-14T06:32:08.073 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 600, 2024-09-14T06:32:08.074 INFO:teuthology.orchestra.run.smithi094.stdout: "ops": [ 2024-09-14T06:32:08.074 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:08.074 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:14918 3.0 3:05c39535:::rbd_data.112d92027590.0000000000000009:f [read 795648~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:32:08.074 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:26.083834+0000", 2024-09-14T06:32:08.074 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 161.987798833, 2024-09-14T06:32:08.074 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.041776287000000002, 2024-09-14T06:32:08.074 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:32:08.074 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:32:08.074 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:32:08.074 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:32:08.074 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:32:08.074 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:08.074 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 14918 2024-09-14T06:32:08.075 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:08.075 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:32:08.075 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:08.075 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:32:08.075 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.083834+0000", 2024-09-14T06:32:08.075 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:08.075 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:08.075 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:08.075 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:32:08.075 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.083834+0000", 2024-09-14T06:32:08.075 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:08.075 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:08.075 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:08.076 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:32:08.076 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.083834+0000", 2024-09-14T06:32:08.076 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.0100000000000001e-07 2024-09-14T06:32:08.076 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:08.076 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:08.076 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:32:08.076 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.083835+0000", 2024-09-14T06:32:08.076 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.3900000000000003e-07 2024-09-14T06:32:08.076 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:08.076 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:08.076 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:32:08.076 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.083838+0000", 2024-09-14T06:32:08.076 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.9830000000000001e-06 2024-09-14T06:32:08.076 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:08.077 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:08.077 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:32:08.077 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.083840+0000", 2024-09-14T06:32:08.077 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.4679999999999999e-06 2024-09-14T06:32:08.077 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:08.077 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:08.077 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:32:08.077 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.125455+0000", 2024-09-14T06:32:08.077 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.041614578999999999 2024-09-14T06:32:08.077 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:08.077 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:08.077 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:32:08.077 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.125496+0000", 2024-09-14T06:32:08.077 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.1047000000000002e-05 2024-09-14T06:32:08.077 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:08.078 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:08.078 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:32:08.078 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.125610+0000", 2024-09-14T06:32:08.078 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00011407 2024-09-14T06:32:08.078 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:08.078 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:32:08.078 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:08.078 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:08.078 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:08.078 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:14919 3.0 3:05c39535:::rbd_data.112d92027590.0000000000000009:f [read 805888~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:32:08.078 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:26.083843+0000", 2024-09-14T06:32:08.078 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 161.987789085, 2024-09-14T06:32:08.078 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.041992356000000002, 2024-09-14T06:32:08.078 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:32:08.078 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:32:08.079 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:32:08.079 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:32:08.079 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:32:08.079 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:08.079 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 14919 2024-09-14T06:32:08.079 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:08.079 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:32:08.079 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:08.079 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:32:08.079 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.083843+0000", 2024-09-14T06:32:08.079 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:08.079 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:08.079 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:08.079 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:32:08.079 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.083843+0000", 2024-09-14T06:32:08.080 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:08.080 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:08.080 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:08.080 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:32:08.080 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.083844+0000", 2024-09-14T06:32:08.080 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.3300000000000001e-07 2024-09-14T06:32:08.080 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:08.080 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:08.080 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:32:08.080 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.083845+0000", 2024-09-14T06:32:08.080 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.4729999999999999e-06 2024-09-14T06:32:08.080 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:08.080 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:08.080 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:32:08.081 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.083850+0000", 2024-09-14T06:32:08.081 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.0130000000000001e-06 2024-09-14T06:32:08.081 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:08.081 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:08.081 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:32:08.081 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.083854+0000", 2024-09-14T06:32:08.081 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.4759999999999998e-06 2024-09-14T06:32:08.081 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:08.081 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:08.081 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:32:08.081 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.125636+0000", 2024-09-14T06:32:08.081 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.041782683000000001 2024-09-14T06:32:08.081 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:08.081 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:08.081 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:32:08.082 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.125680+0000", 2024-09-14T06:32:08.082 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.3614e-05 2024-09-14T06:32:08.082 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:08.082 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:08.082 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:32:08.082 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.125836+0000", 2024-09-14T06:32:08.082 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000155864 2024-09-14T06:32:08.082 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:08.082 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:32:08.082 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:08.082 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:08.082 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:08.082 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:14920 3.0 3:05c39535:::rbd_data.112d92027590.0000000000000009:f [read 816128~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:32:08.082 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:26.083868+0000", 2024-09-14T06:32:08.082 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 161.98776442400001, 2024-09-14T06:32:08.082 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.042166771999999998, 2024-09-14T06:32:08.083 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:32:08.083 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:32:08.083 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:32:08.083 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:32:08.083 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:32:08.083 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:08.083 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 14920 2024-09-14T06:32:08.083 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:08.083 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:32:08.083 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:08.083 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:32:08.083 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.083868+0000", 2024-09-14T06:32:08.083 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:08.083 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:08.084 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:08.084 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:32:08.084 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.083868+0000", 2024-09-14T06:32:08.084 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:08.084 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:08.084 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:08.084 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:32:08.084 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.083869+0000", 2024-09-14T06:32:08.084 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.99e-07 2024-09-14T06:32:08.084 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:08.084 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:08.084 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:32:08.084 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.083870+0000", 2024-09-14T06:32:08.084 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.782e-06 2024-09-14T06:32:08.085 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:08.085 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:08.085 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:32:08.085 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.083876+0000", 2024-09-14T06:32:08.085 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.3809999999999998e-06 2024-09-14T06:32:08.085 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:08.085 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:08.085 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:32:08.085 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.083882+0000", 2024-09-14T06:32:08.085 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.9479999999999996e-06 2024-09-14T06:32:08.085 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:08.085 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:08.085 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:32:08.085 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.125878+0000", 2024-09-14T06:32:08.086 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.041996264999999998 2024-09-14T06:32:08.086 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:08.086 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:08.086 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:32:08.086 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.125929+0000", 2024-09-14T06:32:08.086 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.0498999999999999e-05 2024-09-14T06:32:08.086 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:08.086 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:08.086 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:32:08.086 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.126035+0000", 2024-09-14T06:32:08.086 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000106298 2024-09-14T06:32:08.086 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:08.086 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:32:08.086 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:08.087 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:08.087 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:08.087 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:14921 3.0 3:05c39535:::rbd_data.112d92027590.0000000000000009:f [read 826368~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:32:08.087 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:26.083889+0000", 2024-09-14T06:32:08.087 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 161.98774341399999, 2024-09-14T06:32:08.087 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.042435309999999997, 2024-09-14T06:32:08.087 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:32:08.087 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:32:08.087 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:32:08.087 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:32:08.087 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:32:08.087 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:08.087 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 14921 2024-09-14T06:32:08.088 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:08.088 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:32:08.088 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:08.088 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:32:08.088 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.083889+0000", 2024-09-14T06:32:08.088 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:08.088 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:08.088 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:08.088 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:32:08.088 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.083889+0000", 2024-09-14T06:32:08.088 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:08.088 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:08.089 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:08.089 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:32:08.089 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.086033+0000", 2024-09-14T06:32:08.089 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0021444179999999999 2024-09-14T06:32:08.089 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:08.089 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:08.089 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:32:08.089 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.086038+0000", 2024-09-14T06:32:08.089 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.0969999999999999e-06 2024-09-14T06:32:08.089 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:08.089 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:08.089 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:32:08.089 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.086046+0000", 2024-09-14T06:32:08.089 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.9440000000000005e-06 2024-09-14T06:32:08.090 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:08.090 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:08.090 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:32:08.090 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.086050+0000", 2024-09-14T06:32:08.090 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.69e-06 2024-09-14T06:32:08.090 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:08.090 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:08.090 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:32:08.090 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.126080+0000", 2024-09-14T06:32:08.090 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.040030237000000003 2024-09-14T06:32:08.090 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:08.090 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:08.090 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:32:08.090 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.126163+0000", 2024-09-14T06:32:08.090 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.2071999999999996e-05 2024-09-14T06:32:08.091 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:08.091 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:08.091 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:32:08.091 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.126324+0000", 2024-09-14T06:32:08.091 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00016185200000000001 2024-09-14T06:32:08.091 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:08.091 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:32:08.091 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:08.091 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:08.091 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:08.091 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:55627 3.0 3:06490baf:::rbd_data.112dca2f6ae1.0000000000000030:17 [read 2002944~4096 [fadvise_nocache]] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:32:08.091 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:44.599966+0000", 2024-09-14T06:32:08.091 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 143.47166646599999, 2024-09-14T06:32:08.092 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.041868546999999999, 2024-09-14T06:32:08.092 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:32:08.092 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:32:08.092 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:32:08.092 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:32:08.092 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:32:08.092 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:08.092 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 55627 2024-09-14T06:32:08.092 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:08.092 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:32:08.092 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:08.092 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:32:08.092 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.599966+0000", 2024-09-14T06:32:08.092 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:08.092 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:08.093 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:08.093 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:32:08.093 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.599966+0000", 2024-09-14T06:32:08.093 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:08.093 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:08.093 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:08.093 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:32:08.093 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.599966+0000", 2024-09-14T06:32:08.093 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.0900000000000001e-07 2024-09-14T06:32:08.093 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:08.093 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:08.093 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:32:08.093 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.599967+0000", 2024-09-14T06:32:08.093 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.64e-07 2024-09-14T06:32:08.094 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:08.094 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:08.094 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:32:08.094 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.599970+0000", 2024-09-14T06:32:08.094 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.9670000000000001e-06 2024-09-14T06:32:08.094 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:08.094 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:08.094 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:32:08.094 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.599973+0000", 2024-09-14T06:32:08.094 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.632e-06 2024-09-14T06:32:08.094 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:08.094 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:08.094 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:32:08.094 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.641665+0000", 2024-09-14T06:32:08.095 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.041692547000000003 2024-09-14T06:32:08.095 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:08.095 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:08.095 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:32:08.095 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.641740+0000", 2024-09-14T06:32:08.095 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.4559000000000002e-05 2024-09-14T06:32:08.095 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:08.095 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:08.095 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:32:08.095 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.641835+0000", 2024-09-14T06:32:08.095 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.4668999999999998e-05 2024-09-14T06:32:08.095 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:08.095 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:32:08.095 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:08.095 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:08.096 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:08.096 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:55628 3.0 3:06490baf:::rbd_data.112dca2f6ae1.0000000000000030:17 [read 2027520~4096 [fadvise_nocache]] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:32:08.096 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:44.599976+0000", 2024-09-14T06:32:08.096 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 143.47165667900001, 2024-09-14T06:32:08.096 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.041991621, 2024-09-14T06:32:08.096 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:32:08.096 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:32:08.096 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:32:08.096 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:32:08.096 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:32:08.096 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:08.096 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 55628 2024-09-14T06:32:08.096 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:08.096 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:32:08.096 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:08.097 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:32:08.097 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.599976+0000", 2024-09-14T06:32:08.097 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:08.097 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:08.097 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:08.097 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:32:08.097 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.599976+0000", 2024-09-14T06:32:08.097 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:08.097 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:08.097 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:08.097 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:32:08.097 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.599976+0000", 2024-09-14T06:32:08.097 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.1299999999999999e-07 2024-09-14T06:32:08.097 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:08.098 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:08.098 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:32:08.098 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.599977+0000", 2024-09-14T06:32:08.098 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.7999999999999993e-07 2024-09-14T06:32:08.098 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:08.098 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:08.098 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:32:08.098 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.599980+0000", 2024-09-14T06:32:08.098 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.01e-06 2024-09-14T06:32:08.098 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:08.098 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:08.098 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:32:08.098 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.599983+0000", 2024-09-14T06:32:08.098 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.5409999999999999e-06 2024-09-14T06:32:08.099 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:08.099 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:08.099 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:32:08.099 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.641869+0000", 2024-09-14T06:32:08.099 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.041886235000000001 2024-09-14T06:32:08.099 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:08.099 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:08.099 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:32:08.099 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.641913+0000", 2024-09-14T06:32:08.099 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.4357000000000002e-05 2024-09-14T06:32:08.099 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:08.099 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:08.099 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:32:08.099 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.641967+0000", 2024-09-14T06:32:08.100 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.4284999999999999e-05 2024-09-14T06:32:08.100 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:08.100 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:32:08.100 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:08.100 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:08.100 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:08.100 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:55629 3.0 3:06490baf:::rbd_data.112dca2f6ae1.0000000000000030:17 [read 2052096~4096 [fadvise_nocache]] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:32:08.100 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:44.599985+0000", 2024-09-14T06:32:08.100 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 143.47164697299999, 2024-09-14T06:32:08.100 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.042183375000000002, 2024-09-14T06:32:08.100 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:32:08.100 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:32:08.100 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:32:08.100 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:32:08.101 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:32:08.101 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:08.101 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 55629 2024-09-14T06:32:08.101 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:08.101 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:32:08.101 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:08.101 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:32:08.101 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.599985+0000", 2024-09-14T06:32:08.101 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:08.101 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:08.101 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:08.101 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:32:08.101 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.599985+0000", 2024-09-14T06:32:08.102 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:08.102 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:08.102 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:08.102 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:32:08.102 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.599986+0000", 2024-09-14T06:32:08.102 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.05e-07 2024-09-14T06:32:08.102 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:08.102 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:08.102 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:32:08.102 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.599987+0000", 2024-09-14T06:32:08.102 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.8400000000000002e-07 2024-09-14T06:32:08.102 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:08.102 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:08.102 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:32:08.102 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.599990+0000", 2024-09-14T06:32:08.102 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.5640000000000001e-06 2024-09-14T06:32:08.103 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:08.103 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:08.103 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:32:08.103 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.599993+0000", 2024-09-14T06:32:08.103 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.615e-06 2024-09-14T06:32:08.103 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:08.103 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:08.103 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:32:08.103 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.642003+0000", 2024-09-14T06:32:08.103 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.042010345999999997 2024-09-14T06:32:08.103 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:08.103 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:08.103 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:32:08.103 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.642080+0000", 2024-09-14T06:32:08.103 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.6538999999999993e-05 2024-09-14T06:32:08.104 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:08.104 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:08.104 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:32:08.104 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.642169+0000", 2024-09-14T06:32:08.104 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.9122000000000005e-05 2024-09-14T06:32:08.104 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:08.104 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:32:08.104 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:08.104 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:08.104 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:08.104 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:55630 3.0 3:06490baf:::rbd_data.112dca2f6ae1.0000000000000030:17 [read 2076672~4096 [fadvise_nocache]] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:32:08.104 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:44.599996+0000", 2024-09-14T06:32:08.104 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 143.47163649500001, 2024-09-14T06:32:08.104 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.042299615999999998, 2024-09-14T06:32:08.105 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:32:08.105 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:32:08.105 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:32:08.105 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:32:08.105 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:32:08.105 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:08.105 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 55630 2024-09-14T06:32:08.105 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:08.105 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:32:08.105 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:08.105 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:32:08.105 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.599996+0000", 2024-09-14T06:32:08.105 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:08.105 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:08.106 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:08.106 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:32:08.106 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.599996+0000", 2024-09-14T06:32:08.106 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:08.106 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:08.106 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:08.106 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:32:08.106 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.599996+0000", 2024-09-14T06:32:08.106 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.1500000000000001e-07 2024-09-14T06:32:08.106 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:08.106 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:08.106 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:32:08.106 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.599997+0000", 2024-09-14T06:32:08.106 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.33e-07 2024-09-14T06:32:08.106 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:08.106 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:08.107 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:32:08.107 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600000+0000", 2024-09-14T06:32:08.107 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.0240000000000002e-06 2024-09-14T06:32:08.107 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:08.107 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:08.107 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:32:08.107 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600003+0000", 2024-09-14T06:32:08.107 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.3640000000000002e-06 2024-09-14T06:32:08.107 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:08.107 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:08.107 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:32:08.107 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.642200+0000", 2024-09-14T06:32:08.107 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.042197571000000003 2024-09-14T06:32:08.107 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:08.108 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:08.108 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:32:08.108 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.642243+0000", 2024-09-14T06:32:08.108 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.2765000000000001e-05 2024-09-14T06:32:08.108 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:08.108 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:08.108 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:32:08.108 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.642296+0000", 2024-09-14T06:32:08.108 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.2744000000000002e-05 2024-09-14T06:32:08.108 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:08.108 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:32:08.108 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:08.108 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:08.108 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:08.108 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:55631 3.0 3:06490baf:::rbd_data.112dca2f6ae1.0000000000000030:17 [read 1048576~4096 [fadvise_nocache]] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:32:08.109 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:44.600006+0000", 2024-09-14T06:32:08.109 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 143.471626907, 2024-09-14T06:32:08.109 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.042487905999999999, 2024-09-14T06:32:08.109 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:32:08.109 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:32:08.109 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:32:08.109 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:32:08.109 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:32:08.109 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:08.109 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 55631 2024-09-14T06:32:08.109 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:08.109 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:32:08.109 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:08.109 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:32:08.110 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600006+0000", 2024-09-14T06:32:08.110 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:08.110 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:08.110 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:08.110 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:32:08.110 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600006+0000", 2024-09-14T06:32:08.110 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:08.110 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:08.110 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:08.110 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:32:08.110 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600006+0000", 2024-09-14T06:32:08.110 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.0800000000000001e-07 2024-09-14T06:32:08.110 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:08.110 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:08.110 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:32:08.111 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600007+0000", 2024-09-14T06:32:08.111 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.7199999999999997e-07 2024-09-14T06:32:08.111 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:08.111 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:08.111 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:32:08.111 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600010+0000", 2024-09-14T06:32:08.111 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.0280000000000001e-06 2024-09-14T06:32:08.111 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:08.111 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:08.111 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:32:08.111 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600012+0000", 2024-09-14T06:32:08.111 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.475e-06 2024-09-14T06:32:08.111 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:08.111 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:08.112 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:32:08.112 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.642329+0000", 2024-09-14T06:32:08.112 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.042317157000000001 2024-09-14T06:32:08.112 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:08.112 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:08.112 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:32:08.112 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.642403+0000", 2024-09-14T06:32:08.112 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.3941000000000002e-05 2024-09-14T06:32:08.112 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:08.112 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:08.112 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:32:08.112 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.642493+0000", 2024-09-14T06:32:08.112 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.0124999999999995e-05 2024-09-14T06:32:08.112 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:08.113 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:32:08.113 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:08.113 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:08.113 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:08.113 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:55632 3.0 3:06490baf:::rbd_data.112dca2f6ae1.0000000000000030:17 [read 1073152~4096 [fadvise_nocache]] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:32:08.113 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:44.600015+0000", 2024-09-14T06:32:08.113 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 143.471617154, 2024-09-14T06:32:08.113 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.042604922000000003, 2024-09-14T06:32:08.113 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:32:08.113 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:32:08.113 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:32:08.113 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:32:08.113 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:32:08.113 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:08.114 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 55632 2024-09-14T06:32:08.114 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:08.114 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:32:08.114 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:08.114 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:32:08.114 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600015+0000", 2024-09-14T06:32:08.114 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:08.114 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:08.114 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:08.114 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:32:08.114 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600015+0000", 2024-09-14T06:32:08.114 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:08.114 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:08.114 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:08.115 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:32:08.115 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600016+0000", 2024-09-14T06:32:08.115 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.1e-07 2024-09-14T06:32:08.115 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:08.115 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:08.115 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:32:08.115 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600017+0000", 2024-09-14T06:32:08.115 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.9900000000000009e-07 2024-09-14T06:32:08.115 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:08.115 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:08.115 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:32:08.115 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600020+0000", 2024-09-14T06:32:08.115 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.4170000000000001e-06 2024-09-14T06:32:08.115 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:08.115 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:08.116 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:32:08.116 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600023+0000", 2024-09-14T06:32:08.116 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.588e-06 2024-09-14T06:32:08.116 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:08.116 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:08.116 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:32:08.116 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.642526+0000", 2024-09-14T06:32:08.116 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.042503236999999999 2024-09-14T06:32:08.116 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:08.116 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:08.116 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:32:08.116 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.642568+0000", 2024-09-14T06:32:08.116 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.1925000000000003e-05 2024-09-14T06:32:08.116 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:08.117 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:08.117 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:32:08.117 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.642620+0000", 2024-09-14T06:32:08.117 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.2546000000000001e-05 2024-09-14T06:32:08.117 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:08.117 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:32:08.117 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:08.117 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:08.117 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:08.117 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:55633 3.0 3:06490baf:::rbd_data.112dca2f6ae1.0000000000000030:17 [read 1097728~4096 [fadvise_nocache]] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:32:08.117 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:44.600026+0000", 2024-09-14T06:32:08.117 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 143.471606885, 2024-09-14T06:32:08.117 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.042797474000000002, 2024-09-14T06:32:08.117 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:32:08.118 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:32:08.118 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:32:08.118 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:32:08.118 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:32:08.118 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:08.118 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 55633 2024-09-14T06:32:08.118 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:08.118 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:32:08.118 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:08.118 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:32:08.118 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600026+0000", 2024-09-14T06:32:08.118 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:08.118 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:08.118 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:08.118 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:32:08.118 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600026+0000", 2024-09-14T06:32:08.119 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:08.119 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:08.119 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:08.119 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:32:08.119 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600026+0000", 2024-09-14T06:32:08.119 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.3099999999999999e-07 2024-09-14T06:32:08.119 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:08.119 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:08.119 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:32:08.119 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600029+0000", 2024-09-14T06:32:08.119 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.304e-06 2024-09-14T06:32:08.119 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:08.119 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:08.119 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:32:08.119 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600034+0000", 2024-09-14T06:32:08.119 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.1429999999999997e-06 2024-09-14T06:32:08.120 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:08.120 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:08.120 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:32:08.120 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600038+0000", 2024-09-14T06:32:08.120 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.692e-06 2024-09-14T06:32:08.120 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:08.120 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:08.120 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:32:08.120 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.642654+0000", 2024-09-14T06:32:08.120 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.042615564000000002 2024-09-14T06:32:08.120 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:08.120 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:08.120 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:32:08.120 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.642728+0000", 2024-09-14T06:32:08.121 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.4018999999999997e-05 2024-09-14T06:32:08.121 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:08.121 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:08.121 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:32:08.121 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.642823+0000", 2024-09-14T06:32:08.121 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.5520999999999996e-05 2024-09-14T06:32:08.121 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:08.121 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:32:08.121 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:08.121 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:08.121 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:08.121 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:55634 3.0 3:06490baf:::rbd_data.112dca2f6ae1.0000000000000030:17 [read 1122304~4096 [fadvise_nocache]] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:32:08.121 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:44.600042+0000", 2024-09-14T06:32:08.121 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 143.471590408, 2024-09-14T06:32:08.122 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.042903729000000002, 2024-09-14T06:32:08.122 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:32:08.122 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:32:08.122 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:32:08.122 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:32:08.122 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:32:08.122 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:08.122 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 55634 2024-09-14T06:32:08.122 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:08.122 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:32:08.122 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:08.122 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:32:08.122 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600042+0000", 2024-09-14T06:32:08.122 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:08.122 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:08.123 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:08.123 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:32:08.123 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600042+0000", 2024-09-14T06:32:08.123 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:08.123 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:08.123 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:08.123 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:32:08.123 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600043+0000", 2024-09-14T06:32:08.123 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.7000000000000005e-07 2024-09-14T06:32:08.123 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:08.123 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:08.123 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:32:08.123 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600044+0000", 2024-09-14T06:32:08.123 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.742e-06 2024-09-14T06:32:08.124 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:08.124 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:08.124 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:32:08.124 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600050+0000", 2024-09-14T06:32:08.124 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.0800000000000002e-06 2024-09-14T06:32:08.124 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:08.124 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:08.124 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:32:08.124 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600055+0000", 2024-09-14T06:32:08.124 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.3889999999999998e-06 2024-09-14T06:32:08.124 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:08.124 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:08.124 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:32:08.124 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.642845+0000", 2024-09-14T06:32:08.125 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.042790040000000001 2024-09-14T06:32:08.125 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:08.125 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:08.125 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:32:08.125 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.642888+0000", 2024-09-14T06:32:08.125 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.2790000000000002e-05 2024-09-14T06:32:08.125 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:08.125 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:08.125 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:32:08.125 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.642946+0000", 2024-09-14T06:32:08.125 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.8118000000000002e-05 2024-09-14T06:32:08.125 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:08.125 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:32:08.125 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:08.126 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:08.126 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:08.126 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:55635 3.0 3:06490baf:::rbd_data.112dca2f6ae1.0000000000000030:17 [read 1146880~4096 [fadvise_nocache]] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:32:08.126 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:44.600061+0000", 2024-09-14T06:32:08.126 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 143.471571837, 2024-09-14T06:32:08.126 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.043092695, 2024-09-14T06:32:08.126 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:32:08.126 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:32:08.126 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:32:08.126 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:32:08.126 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:32:08.126 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:08.126 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 55635 2024-09-14T06:32:08.126 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:08.126 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:32:08.127 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:08.127 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:32:08.127 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600061+0000", 2024-09-14T06:32:08.127 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:08.127 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:08.127 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:08.127 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:32:08.127 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600061+0000", 2024-09-14T06:32:08.127 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:08.127 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:08.127 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:08.127 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:32:08.127 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600061+0000", 2024-09-14T06:32:08.127 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.3099999999999999e-07 2024-09-14T06:32:08.128 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:08.128 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:08.128 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:32:08.128 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600063+0000", 2024-09-14T06:32:08.128 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.6220000000000001e-06 2024-09-14T06:32:08.128 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:08.128 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:08.128 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:32:08.128 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600068+0000", 2024-09-14T06:32:08.128 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.4790000000000004e-06 2024-09-14T06:32:08.128 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:08.128 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:08.128 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:32:08.128 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600072+0000", 2024-09-14T06:32:08.129 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.3100000000000002e-06 2024-09-14T06:32:08.129 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:08.129 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:08.129 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:32:08.129 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.642987+0000", 2024-09-14T06:32:08.129 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.042914489 2024-09-14T06:32:08.129 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:08.129 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:08.129 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:32:08.129 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.643062+0000", 2024-09-14T06:32:08.129 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.5011000000000004e-05 2024-09-14T06:32:08.129 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:08.129 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:08.129 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:32:08.129 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.643153+0000", 2024-09-14T06:32:08.130 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.1452999999999993e-05 2024-09-14T06:32:08.130 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:08.130 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:32:08.130 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:08.130 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:08.130 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:08.130 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:55636 3.0 3:06490baf:::rbd_data.112dca2f6ae1.0000000000000030:17 [read 1171456~4096 [fadvise_nocache]] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:32:08.130 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:44.600078+0000", 2024-09-14T06:32:08.130 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 143.47155442499999, 2024-09-14T06:32:08.130 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.043207966, 2024-09-14T06:32:08.130 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:32:08.130 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:32:08.130 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:32:08.130 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:32:08.131 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:32:08.131 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:08.131 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 55636 2024-09-14T06:32:08.131 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:08.131 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:32:08.131 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:08.131 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:32:08.131 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600078+0000", 2024-09-14T06:32:08.131 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:08.131 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:08.131 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:08.131 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:32:08.131 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600078+0000", 2024-09-14T06:32:08.131 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:08.131 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:08.132 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:08.132 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:32:08.132 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600078+0000", 2024-09-14T06:32:08.132 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.34e-07 2024-09-14T06:32:08.132 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:08.132 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:08.132 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:32:08.132 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600080+0000", 2024-09-14T06:32:08.132 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.641e-06 2024-09-14T06:32:08.132 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:08.132 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:08.132 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:32:08.132 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600086+0000", 2024-09-14T06:32:08.132 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.3860000000000004e-06 2024-09-14T06:32:08.133 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:08.133 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:08.133 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:32:08.133 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600091+0000", 2024-09-14T06:32:08.133 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.5970000000000002e-06 2024-09-14T06:32:08.133 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:08.133 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:08.133 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:32:08.133 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.643184+0000", 2024-09-14T06:32:08.133 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.043092893 2024-09-14T06:32:08.133 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:08.133 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:08.133 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:32:08.133 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.643228+0000", 2024-09-14T06:32:08.134 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.4536999999999997e-05 2024-09-14T06:32:08.134 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:08.134 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:08.134 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:32:08.134 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.643286+0000", 2024-09-14T06:32:08.134 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.7578000000000003e-05 2024-09-14T06:32:08.134 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:08.134 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:32:08.134 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:08.134 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:08.134 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:08.134 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:55637 3.0 3:06490baf:::rbd_data.112dca2f6ae1.0000000000000030:17 [read 1196032~4096 [fadvise_nocache]] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:32:08.134 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:44.600097+0000", 2024-09-14T06:32:08.135 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 143.47153586100001, 2024-09-14T06:32:08.135 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.043399124999999997, 2024-09-14T06:32:08.135 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:32:08.135 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:32:08.135 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:32:08.135 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:32:08.135 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:32:08.135 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:08.135 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 55637 2024-09-14T06:32:08.135 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:08.135 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:32:08.135 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:08.135 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:32:08.135 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600097+0000", 2024-09-14T06:32:08.136 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:08.136 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:08.136 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:08.136 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:32:08.136 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600097+0000", 2024-09-14T06:32:08.136 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:08.136 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:08.136 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:08.136 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:32:08.136 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.604372+0000", 2024-09-14T06:32:08.136 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0042751489999999998 2024-09-14T06:32:08.136 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:08.136 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:08.137 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:32:08.137 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.604377+0000", 2024-09-14T06:32:08.137 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.7899999999999999e-06 2024-09-14T06:32:08.137 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:08.137 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:08.137 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:32:08.137 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.604389+0000", 2024-09-14T06:32:08.137 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.2197e-05 2024-09-14T06:32:08.137 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:08.137 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:08.137 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:32:08.137 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.604397+0000", 2024-09-14T06:32:08.137 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.8730000000000007e-06 2024-09-14T06:32:08.137 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:08.138 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:08.138 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:32:08.138 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.643324+0000", 2024-09-14T06:32:08.138 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.038926897000000002 2024-09-14T06:32:08.138 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:08.138 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:08.138 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:32:08.138 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.643402+0000", 2024-09-14T06:32:08.138 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.8063000000000001e-05 2024-09-14T06:32:08.138 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:08.138 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:08.138 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:32:08.138 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.643496+0000", 2024-09-14T06:32:08.138 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.4155999999999998e-05 2024-09-14T06:32:08.139 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:08.139 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:32:08.139 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:08.139 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:08.139 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:08.139 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:55657 3.0 3:06490baf:::rbd_data.112dca2f6ae1.0000000000000030:17 [read 1687552~4096 [fadvise_nocache]] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:32:08.139 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:44.604841+0000", 2024-09-14T06:32:08.139 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 143.466791132, 2024-09-14T06:32:08.139 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.041890087999999999, 2024-09-14T06:32:08.139 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:32:08.139 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:32:08.139 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:32:08.139 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:32:08.139 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:32:08.139 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:08.139 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 55657 2024-09-14T06:32:08.140 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:08.140 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:32:08.140 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:08.140 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:32:08.140 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.604841+0000", 2024-09-14T06:32:08.140 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:08.140 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:08.140 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:08.140 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:32:08.140 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.604841+0000", 2024-09-14T06:32:08.140 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:08.140 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:08.140 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:08.140 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:32:08.140 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.604842+0000", 2024-09-14T06:32:08.141 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.0900000000000001e-07 2024-09-14T06:32:08.141 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:08.141 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:08.141 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:32:08.141 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.604846+0000", 2024-09-14T06:32:08.141 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.5759999999999999e-06 2024-09-14T06:32:08.141 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:08.141 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:08.141 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:32:08.141 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.604850+0000", 2024-09-14T06:32:08.141 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.405e-06 2024-09-14T06:32:08.141 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:08.141 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:08.141 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:32:08.142 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.604852+0000", 2024-09-14T06:32:08.142 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.8590000000000001e-06 2024-09-14T06:32:08.142 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:08.142 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:08.142 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:32:08.142 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.646633+0000", 2024-09-14T06:32:08.142 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.041780772000000001 2024-09-14T06:32:08.142 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:08.142 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:08.142 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:32:08.142 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.646677+0000", 2024-09-14T06:32:08.142 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.4115e-05 2024-09-14T06:32:08.142 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:08.142 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:08.142 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:32:08.143 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.646731+0000", 2024-09-14T06:32:08.143 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.4151999999999999e-05 2024-09-14T06:32:08.143 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:08.143 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:32:08.143 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:08.143 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:08.143 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:08.143 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:55658 3.0 3:06490baf:::rbd_data.112dca2f6ae1.0000000000000030:17 [read 1712128~4096 [fadvise_nocache]] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:32:08.143 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:44.604856+0000", 2024-09-14T06:32:08.143 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 143.46677653200001, 2024-09-14T06:32:08.143 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.041986227000000001, 2024-09-14T06:32:08.143 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:32:08.143 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:32:08.143 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:32:08.143 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:32:08.144 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:32:08.144 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:08.144 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 55658 2024-09-14T06:32:08.144 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:08.144 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:32:08.144 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:08.144 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:32:08.144 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.604856+0000", 2024-09-14T06:32:08.144 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:08.144 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:08.144 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:08.144 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:32:08.144 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.604856+0000", 2024-09-14T06:32:08.144 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:08.145 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:08.145 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:08.145 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:32:08.145 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.604856+0000", 2024-09-14T06:32:08.145 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.23e-07 2024-09-14T06:32:08.145 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:08.145 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:08.145 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:32:08.145 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.604857+0000", 2024-09-14T06:32:08.145 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.9999999999999995e-07 2024-09-14T06:32:08.145 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:08.145 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:08.145 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:32:08.145 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.604861+0000", 2024-09-14T06:32:08.146 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.9890000000000003e-06 2024-09-14T06:32:08.146 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:08.146 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:08.146 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:32:08.146 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.604864+0000", 2024-09-14T06:32:08.146 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.7429999999999998e-06 2024-09-14T06:32:08.146 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:08.146 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:08.146 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:32:08.146 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.646744+0000", 2024-09-14T06:32:08.146 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.041880325000000003 2024-09-14T06:32:08.146 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:08.146 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:08.146 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:32:08.146 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.646784+0000", 2024-09-14T06:32:08.147 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.9604000000000002e-05 2024-09-14T06:32:08.147 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:08.147 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:08.147 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:32:08.147 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.646842+0000", 2024-09-14T06:32:08.147 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.8343e-05 2024-09-14T06:32:08.147 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:08.147 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:32:08.147 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:08.147 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:08.147 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:08.147 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:55659 3.0 3:06490baf:::rbd_data.112dca2f6ae1.0000000000000030:17 [read 1736704~4096 [fadvise_nocache]] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:32:08.147 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:44.604867+0000", 2024-09-14T06:32:08.147 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 143.46676537900001, 2024-09-14T06:32:08.148 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.042097082000000001, 2024-09-14T06:32:08.148 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:32:08.148 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:32:08.148 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:32:08.148 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:32:08.148 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:32:08.148 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:08.148 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 55659 2024-09-14T06:32:08.148 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:08.148 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:32:08.148 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:08.148 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:32:08.148 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.604867+0000", 2024-09-14T06:32:08.148 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:08.149 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:08.149 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:08.149 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:32:08.149 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.604867+0000", 2024-09-14T06:32:08.149 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:08.149 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:08.149 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:08.149 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:32:08.149 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.604867+0000", 2024-09-14T06:32:08.149 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.2000000000000001e-07 2024-09-14T06:32:08.149 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:08.149 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:08.149 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:32:08.149 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.604868+0000", 2024-09-14T06:32:08.149 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.04e-06 2024-09-14T06:32:08.150 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:08.150 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:08.150 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:32:08.150 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.604871+0000", 2024-09-14T06:32:08.150 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.9689999999999999e-06 2024-09-14T06:32:08.150 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:08.150 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:08.150 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:32:08.150 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.604874+0000", 2024-09-14T06:32:08.150 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.5069999999999999e-06 2024-09-14T06:32:08.150 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:08.150 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:08.150 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:32:08.150 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.646874+0000", 2024-09-14T06:32:08.151 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.042000513000000003 2024-09-14T06:32:08.151 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:08.151 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:08.151 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:32:08.151 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.646914+0000", 2024-09-14T06:32:08.151 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.9443999999999997e-05 2024-09-14T06:32:08.151 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:08.151 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:08.151 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:32:08.151 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.646964+0000", 2024-09-14T06:32:08.151 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.0389000000000002e-05 2024-09-14T06:32:08.151 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:08.151 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:32:08.151 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:08.151 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:08.152 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:08.152 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:55660 3.0 3:06490baf:::rbd_data.112dca2f6ae1.0000000000000030:17 [read 1761280~4096 [fadvise_nocache]] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:32:08.152 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:44.604877+0000", 2024-09-14T06:32:08.152 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 143.46675562999999, 2024-09-14T06:32:08.152 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.042196349000000001, 2024-09-14T06:32:08.152 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:32:08.152 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:32:08.152 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:32:08.152 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:32:08.152 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:32:08.152 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:08.152 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 55660 2024-09-14T06:32:08.152 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:08.152 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:32:08.152 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:08.152 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:32:08.153 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.604877+0000", 2024-09-14T06:32:08.153 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:08.153 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:08.153 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:08.153 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:32:08.153 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.604877+0000", 2024-09-14T06:32:08.153 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:08.153 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:08.153 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:08.153 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:32:08.153 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.605810+0000", 2024-09-14T06:32:08.153 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00093336899999999997 2024-09-14T06:32:08.153 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:08.153 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:08.153 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:32:08.154 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.605815+0000", 2024-09-14T06:32:08.154 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.6140000000000002e-06 2024-09-14T06:32:08.154 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:08.154 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:08.154 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:32:08.154 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.605825+0000", 2024-09-14T06:32:08.154 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.8889999999999997e-06 2024-09-14T06:32:08.154 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:08.154 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:08.154 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:32:08.154 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.605831+0000", 2024-09-14T06:32:08.154 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.9800000000000003e-06 2024-09-14T06:32:08.154 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:08.154 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:08.155 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:32:08.155 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.646989+0000", 2024-09-14T06:32:08.155 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.041157878000000002 2024-09-14T06:32:08.155 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:08.155 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:08.155 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:32:08.155 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.647026+0000", 2024-09-14T06:32:08.155 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.7305000000000003e-05 2024-09-14T06:32:08.155 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:08.155 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:08.155 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:32:08.155 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.647073+0000", 2024-09-14T06:32:08.155 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.7314000000000001e-05 2024-09-14T06:32:08.155 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:08.156 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:32:08.156 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:08.156 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:08.156 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:08.156 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:55666 3.0 3:06490baf:::rbd_data.112dca2f6ae1.0000000000000030:17 [read 1908736~4096 [fadvise_nocache]] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:32:08.156 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:44.605898+0000", 2024-09-14T06:32:08.156 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 143.465734884, 2024-09-14T06:32:08.156 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.041862887000000001, 2024-09-14T06:32:08.156 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:32:08.156 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:32:08.156 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:32:08.156 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:32:08.156 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:32:08.157 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:08.157 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 55666 2024-09-14T06:32:08.157 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:08.157 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:32:08.157 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:08.157 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:32:08.157 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.605898+0000", 2024-09-14T06:32:08.157 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:08.157 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:08.157 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:08.157 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:32:08.157 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.605898+0000", 2024-09-14T06:32:08.157 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:08.157 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:08.158 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:08.158 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:32:08.158 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.611874+0000", 2024-09-14T06:32:08.158 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0059764049999999997 2024-09-14T06:32:08.158 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:08.158 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:08.158 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:32:08.158 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.611879+0000", 2024-09-14T06:32:08.158 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.8749999999999999e-06 2024-09-14T06:32:08.158 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:08.158 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:08.158 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:32:08.158 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.611887+0000", 2024-09-14T06:32:08.158 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.4109999999999996e-06 2024-09-14T06:32:08.159 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:08.159 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:08.159 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:32:08.159 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.611893+0000", 2024-09-14T06:32:08.159 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.2959999999999998e-06 2024-09-14T06:32:08.159 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:08.159 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:08.159 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:32:08.159 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.647674+0000", 2024-09-14T06:32:08.159 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.035781281999999998 2024-09-14T06:32:08.159 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:08.159 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:08.159 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:32:08.159 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.647711+0000", 2024-09-14T06:32:08.160 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.7434000000000001e-05 2024-09-14T06:32:08.160 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:08.160 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:08.160 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:32:08.160 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.647760+0000", 2024-09-14T06:32:08.160 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.9184000000000002e-05 2024-09-14T06:32:08.160 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:08.160 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:32:08.160 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:08.160 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:08.160 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:32:08.160 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-14T06:32:08.162 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-14T06:32:08.162 INFO:tasks.ceph.osd.7.smithi138.stderr:2024-09-14T06:32:08.161+0000 7f3c6a21d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33623) UID: 0 2024-09-14T06:32:08.263 INFO:tasks.ceph.osd.2.smithi094.stderr:2024-09-14T06:32:08.262+0000 7faf3269b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33649) UID: 0 2024-09-14T06:32:08.265 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-14T06:32:08.265 INFO:teuthology.orchestra.run.smithi094.stdout: "ops": [], 2024-09-14T06:32:08.265 INFO:teuthology.orchestra.run.smithi094.stdout: "num_ops": 0 2024-09-14T06:32:08.265 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-14T06:32:08.273 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-14T06:32:08.346 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-14T06:32:08.401 INFO:tasks.ceph.osd.3.smithi094.stderr:2024-09-14T06:32:08.400+0000 7feeeafe8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33655) UID: 0 2024-09-14T06:32:08.418 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-14T06:32:08.418 INFO:teuthology.orchestra.run.smithi094.stdout: "ops": [], 2024-09-14T06:32:08.418 INFO:teuthology.orchestra.run.smithi094.stdout: "complaint_time": 30, 2024-09-14T06:32:08.418 INFO:teuthology.orchestra.run.smithi094.stdout: "num_blocked_ops": 0 2024-09-14T06:32:08.419 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-14T06:32:08.427 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-14T06:32:08.469 INFO:tasks.ceph.osd.2.smithi094.stderr:2024-09-14T06:32:08.468+0000 7faf3269b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33649) UID: 0 2024-09-14T06:32:08.477 INFO:teuthology.orchestra.run.smithi094.stdout:osd.0: {} 2024-09-14T06:32:08.477 INFO:teuthology.orchestra.run.smithi094.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-14T06:32:08.495 INFO:teuthology.orchestra.run.smithi094.stdout:osd.1: {} 2024-09-14T06:32:08.495 INFO:teuthology.orchestra.run.smithi094.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-14T06:32:08.511 INFO:teuthology.orchestra.run.smithi094.stdout:osd.2: {} 2024-09-14T06:32:08.511 INFO:teuthology.orchestra.run.smithi094.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-14T06:32:08.528 INFO:teuthology.orchestra.run.smithi094.stdout:osd.3: {} 2024-09-14T06:32:08.528 INFO:teuthology.orchestra.run.smithi094.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-14T06:32:08.545 INFO:teuthology.orchestra.run.smithi094.stdout:osd.4: {} 2024-09-14T06:32:08.545 INFO:teuthology.orchestra.run.smithi094.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-14T06:32:08.562 INFO:teuthology.orchestra.run.smithi094.stdout:osd.5: {} 2024-09-14T06:32:08.562 INFO:teuthology.orchestra.run.smithi094.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-14T06:32:08.564 INFO:tasks.ceph.osd.1.smithi094.stderr:2024-09-14T06:32:08.563+0000 7f328e47a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33641) UID: 0 2024-09-14T06:32:08.574 INFO:teuthology.orchestra.run.smithi094.stdout:ERROR: (22) Invalid argument 2024-09-14T06:32:08.574 INFO:teuthology.orchestra.run.smithi094.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:32:08.579 INFO:teuthology.orchestra.run.smithi094.stdout:osd.6: {} 2024-09-14T06:32:08.579 INFO:teuthology.orchestra.run.smithi094.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-14T06:32:08.583 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-14T06:32:08.596 INFO:teuthology.orchestra.run.smithi094.stdout:osd.7: {} 2024-09-14T06:32:08.596 INFO:teuthology.orchestra.run.smithi094.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-14T06:32:08.642 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-14T06:32:08.693 INFO:tasks.ceph.osd.1.smithi094.stderr:2024-09-14T06:32:08.692+0000 7f328e47a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33641) UID: 0 2024-09-14T06:32:08.708 INFO:teuthology.orchestra.run.smithi094.stdout:ERROR: (22) Invalid argument 2024-09-14T06:32:08.708 INFO:teuthology.orchestra.run.smithi094.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:32:08.718 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-14T06:32:08.794 INFO:teuthology.orchestra.run.smithi094.stdout:checking clone #43, image image_client.0-clone43 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent44 2024-09-14T06:32:08.794 INFO:tasks.ceph.osd.1.smithi094.stderr:2024-09-14T06:32:08.793+0000 7f328e47a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33641) UID: 0 2024-09-14T06:32:08.866 INFO:tasks.ceph.osd.1.smithi094.stderr:2024-09-14T06:32:08.865+0000 7f328e47a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33641) UID: 0 2024-09-14T06:32:08.878 INFO:teuthology.orchestra.run.smithi094.stdout:ERROR: (22) Invalid argument 2024-09-14T06:32:08.878 INFO:teuthology.orchestra.run.smithi094.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:32:08.887 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-14T06:32:08.971 INFO:tasks.ceph.osd.3.smithi094.stderr:2024-09-14T06:32:08.970+0000 7feeeafe8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33655) UID: 0 2024-09-14T06:32:09.032 INFO:teuthology.orchestra.run.smithi094.stdout:5954 read 0xd1d60f6 thru 0xd1e16fd (0xb608 bytes) 2024-09-14T06:32:09.032 INFO:teuthology.orchestra.run.smithi094.stdout:5955 writesame 0xc31d8ad thru 0xc32a05c data_size 0x5a(0xc7b0 bytes) 2024-09-14T06:32:09.034 INFO:teuthology.orchestra.run.smithi094.stdout:ERROR: (22) Invalid argument 2024-09-14T06:32:09.034 INFO:teuthology.orchestra.run.smithi094.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:32:09.043 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-14T06:32:09.054 INFO:teuthology.orchestra.run.smithi094.stdout:5956 trunc from 0xd841edb to 0x7a6ca1b 2024-09-14T06:32:09.084 INFO:teuthology.orchestra.run.smithi094.stdout:5959 read 0x42a047a thru 0x42aedb1 (0xe938 bytes) 2024-09-14T06:32:09.084 INFO:teuthology.orchestra.run.smithi094.stdout:5960 write 0x1232a9b thru 0x1233ae9 (0x104f bytes) 2024-09-14T06:32:09.087 INFO:tasks.ceph.osd.4.smithi138.stderr:2024-09-14T06:32:09.086+0000 7f6c653c8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33615) UID: 0 2024-09-14T06:32:09.095 INFO:teuthology.orchestra.run.smithi094.stdout:5963 trunc from 0x7a6ca1b to 0xe11e7e2 2024-09-14T06:32:09.102 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-14T06:32:09.102 INFO:teuthology.orchestra.run.smithi094.stdout:5966 write 0x5b142eb thru 0x5b1d8f1 (0x9607 bytes) 2024-09-14T06:32:09.106 INFO:teuthology.orchestra.run.smithi094.stdout:5967 write 0x6fe55db thru 0x6fec775 (0x719b bytes) 2024-09-14T06:32:09.109 INFO:teuthology.orchestra.run.smithi094.stdout:5968 read 0x4c3d1ac thru 0x4c49f99 (0xcdee bytes) 2024-09-14T06:32:09.110 INFO:teuthology.orchestra.run.smithi094.stdout:5969 trunc from 0xe11e7e2 to 0xd0d0dbb 2024-09-14T06:32:09.120 INFO:teuthology.orchestra.run.smithi094.stdout:5971 write 0x7cbe72a thru 0x7cc5f68 (0x783f bytes) 2024-09-14T06:32:09.125 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-14T06:32:09.125 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-14T06:32:09.127 INFO:teuthology.orchestra.run.smithi094.stdout:5974 trunc from 0xd0d0dbb to 0xea9899d 2024-09-14T06:32:09.136 INFO:teuthology.orchestra.run.smithi094.stdout:5975 writesame 0xd445484 thru 0xd450494 data_size 0x2f(0xb011 bytes) 2024-09-14T06:32:09.141 INFO:teuthology.orchestra.run.smithi094.stdout:5976 read 0x1883c thru 0x2388e (0xb053 bytes) 2024-09-14T06:32:09.158 INFO:teuthology.orchestra.run.smithi094.stdout:5977 trunc from 0xea9899d to 0xcb0f22 2024-09-14T06:32:09.169 INFO:tasks.ceph.osd.0.smithi094.stderr:2024-09-14T06:32:09.168+0000 7f07c9333640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33636) UID: 0 2024-09-14T06:32:09.170 INFO:teuthology.orchestra.run.smithi138.stdout:ERROR: (22) Invalid argument 2024-09-14T06:32:09.170 INFO:teuthology.orchestra.run.smithi138.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:32:09.182 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-14T06:32:09.223 INFO:teuthology.orchestra.run.smithi094.stdout:5978 write 0x3bee0f2 thru 0x3befb27 (0x1a36 bytes) 2024-09-14T06:32:09.226 INFO:teuthology.orchestra.run.smithi094.stdout:5980 read 0xb32720 thru 0xb41c0e (0xf4ef bytes) 2024-09-14T06:32:09.226 INFO:teuthology.orchestra.run.smithi094.stdout:5981 writesame 0x2d02c3 thru 0x2ddcd0 data_size 0x2(0xda0e bytes) 2024-09-14T06:32:09.241 INFO:teuthology.orchestra.run.smithi094.stdout:5985 punch from 0x37df052 to 0x37ec28b, (0xd239 bytes) 2024-09-14T06:32:09.241 INFO:teuthology.orchestra.run.smithi094.stdout:5988 read 0x11a6e27 thru 0x11b43c0 (0xd59a bytes) 2024-09-14T06:32:09.241 INFO:teuthology.orchestra.run.smithi094.stdout:5989 read 0x2863466 thru 0x286b58a (0x8125 bytes) 2024-09-14T06:32:09.257 INFO:teuthology.orchestra.run.smithi094.stdout:5990 write 0xabc9d55 thru 0xabcff9d (0x6249 bytes) 2024-09-14T06:32:09.260 INFO:teuthology.orchestra.run.smithi094.stdout:5991 read 0x90e4ebe thru 0x90e7346 (0x2489 bytes) 2024-09-14T06:32:09.270 INFO:tasks.ceph.osd.0.smithi094.stderr:2024-09-14T06:32:09.269+0000 7f07c9333640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33636) UID: 0 2024-09-14T06:32:09.272 INFO:teuthology.orchestra.run.smithi094.stdout:5993 trunc from 0xabcff9e to 0x9acae3 2024-09-14T06:32:09.304 INFO:teuthology.orchestra.run.smithi094.stdout:5994 write 0x4ef315c thru 0x4ef879b (0x5640 bytes) 2024-09-14T06:32:09.314 INFO:teuthology.orchestra.run.smithi094.stdout:5997 trunc from 0x4ef879c to 0xa5d6151 2024-09-14T06:32:09.321 INFO:teuthology.orchestra.run.smithi094.stdout:compare and write disabled 2024-09-14T06:32:09.334 INFO:teuthology.orchestra.run.smithi094.stdout:checking clone #44, image image_client.0-clone44 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent45 2024-09-14T06:32:09.336 INFO:teuthology.orchestra.run.smithi138.stdout:ERROR: (22) Invalid argument 2024-09-14T06:32:09.336 INFO:teuthology.orchestra.run.smithi138.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:32:09.345 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-14T06:32:09.389 INFO:tasks.ceph.osd.4.smithi138.stderr:2024-09-14T06:32:09.388+0000 7f6c653c8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33615) UID: 0 2024-09-14T06:32:09.449 INFO:teuthology.orchestra.run.smithi138.stdout:ERROR: (22) Invalid argument 2024-09-14T06:32:09.449 INFO:teuthology.orchestra.run.smithi138.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:32:09.458 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-14T06:32:09.500 INFO:tasks.ceph.osd.5.smithi138.stderr:2024-09-14T06:32:09.499+0000 7fe13746f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33618) UID: 0 2024-09-14T06:32:09.573 INFO:tasks.ceph.osd.2.smithi094.stderr:2024-09-14T06:32:09.572+0000 7faf3269b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33649) UID: 0 2024-09-14T06:32:09.604 INFO:teuthology.orchestra.run.smithi138.stdout:ERROR: (22) Invalid argument 2024-09-14T06:32:09.605 INFO:teuthology.orchestra.run.smithi138.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:32:09.614 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-14T06:32:09.644 INFO:teuthology.orchestra.run.smithi094.stderr:nodeep-scrub is unset 2024-09-14T06:32:09.701 INFO:tasks.ceph.osd.1.smithi094.stderr:2024-09-14T06:32:09.700+0000 7f328e47a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33641) UID: 0 2024-09-14T06:32:09.726 INFO:teuthology.orchestra.run.smithi138.stdout:ERROR: (22) Invalid argument 2024-09-14T06:32:09.726 INFO:teuthology.orchestra.run.smithi138.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:32:09.734 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-14T06:32:09.775 INFO:tasks.ceph.osd.2.smithi094.stderr:2024-09-14T06:32:09.774+0000 7faf3269b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33649) UID: 0 2024-09-14T06:32:09.837 INFO:teuthology.orchestra.run.smithi138.stdout:ERROR: (22) Invalid argument 2024-09-14T06:32:09.837 INFO:teuthology.orchestra.run.smithi138.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:32:09.846 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-14T06:32:09.889 INFO:tasks.ceph.osd.4.smithi138.stderr:2024-09-14T06:32:09.888+0000 7f6c653c8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33615) UID: 0 2024-09-14T06:32:09.949 INFO:teuthology.orchestra.run.smithi138.stdout:ERROR: (22) Invalid argument 2024-09-14T06:32:09.949 INFO:teuthology.orchestra.run.smithi138.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:32:09.957 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-14T06:32:09.976 INFO:tasks.ceph.osd.1.smithi094.stderr:2024-09-14T06:32:09.975+0000 7f328e47a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33641) UID: 0 2024-09-14T06:32:10.077 INFO:tasks.ceph.osd.5.smithi138.stderr:2024-09-14T06:32:10.076+0000 7fe13746f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33618) UID: 0 2024-09-14T06:32:10.101 INFO:teuthology.orchestra.run.smithi138.stdout:ERROR: (22) Invalid argument 2024-09-14T06:32:10.102 INFO:teuthology.orchestra.run.smithi138.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:32:10.110 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-14T06:32:10.195 INFO:tasks.ceph.osd.5.smithi138.stderr:2024-09-14T06:32:10.194+0000 7fe13746f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33618) UID: 0 2024-09-14T06:32:10.253 INFO:teuthology.orchestra.run.smithi138.stdout:ERROR: (22) Invalid argument 2024-09-14T06:32:10.253 INFO:teuthology.orchestra.run.smithi138.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:32:10.261 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-14T06:32:10.303 INFO:tasks.ceph.osd.6.smithi138.stderr:2024-09-14T06:32:10.302+0000 7f8cd8849640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33620) UID: 0 2024-09-14T06:32:10.378 INFO:tasks.ceph.osd.6.smithi138.stderr:2024-09-14T06:32:10.377+0000 7f8cd8849640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33620) UID: 0 2024-09-14T06:32:10.405 INFO:teuthology.orchestra.run.smithi138.stdout:ERROR: (22) Invalid argument 2024-09-14T06:32:10.405 INFO:teuthology.orchestra.run.smithi138.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:32:10.414 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-14T06:32:10.480 INFO:tasks.ceph.osd.2.smithi094.stderr:2024-09-14T06:32:10.479+0000 7faf3269b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33649) UID: 0 2024-09-14T06:32:10.557 INFO:teuthology.orchestra.run.smithi138.stdout:ERROR: (22) Invalid argument 2024-09-14T06:32:10.557 INFO:teuthology.orchestra.run.smithi138.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:32:10.566 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-14T06:32:10.608 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-14T06:32:10.610 INFO:tasks.ceph.osd.7.smithi138.stderr:2024-09-14T06:32:10.609+0000 7f3c6a21d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33623) UID: 0 2024-09-14T06:32:10.678 INFO:teuthology.orchestra.run.smithi138.stdout:ERROR: (22) Invalid argument 2024-09-14T06:32:10.678 INFO:teuthology.orchestra.run.smithi138.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:32:10.682 INFO:tasks.ceph.osd.1.smithi094.stderr:2024-09-14T06:32:10.681+0000 7f328e47a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33641) UID: 0 2024-09-14T06:32:10.688 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-14T06:32:10.748 INFO:teuthology.orchestra.run.smithi094.stdout:osd.0: {} 2024-09-14T06:32:10.749 INFO:teuthology.orchestra.run.smithi094.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-14T06:32:10.759 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 4, 5, 7, 3] out_osds: [6] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-14T06:32:10.759 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 0 1 2024-09-14T06:32:10.766 INFO:teuthology.orchestra.run.smithi094.stdout:osd.1: {} 2024-09-14T06:32:10.766 INFO:teuthology.orchestra.run.smithi094.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-14T06:32:10.782 INFO:tasks.ceph.osd.3.smithi094.stderr:2024-09-14T06:32:10.781+0000 7feeeafe8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33655) UID: 0 2024-09-14T06:32:10.782 INFO:teuthology.orchestra.run.smithi094.stdout:osd.2: {} 2024-09-14T06:32:10.783 INFO:teuthology.orchestra.run.smithi094.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-14T06:32:10.797 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-14T06:32:10.797 INFO:teuthology.orchestra.run.smithi094.stdout: "ops": [], 2024-09-14T06:32:10.797 INFO:teuthology.orchestra.run.smithi094.stdout: "num_ops": 0 2024-09-14T06:32:10.797 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-14T06:32:10.799 INFO:teuthology.orchestra.run.smithi094.stdout:osd.3: {} 2024-09-14T06:32:10.799 INFO:teuthology.orchestra.run.smithi094.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-14T06:32:10.806 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-14T06:32:10.890 INFO:tasks.ceph.osd.0.smithi094.stderr:2024-09-14T06:32:10.889+0000 7f07c9333640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33636) UID: 0 2024-09-14T06:32:10.951 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-14T06:32:10.951 INFO:teuthology.orchestra.run.smithi094.stdout: "ops": [], 2024-09-14T06:32:10.951 INFO:teuthology.orchestra.run.smithi094.stdout: "complaint_time": 30, 2024-09-14T06:32:10.952 INFO:teuthology.orchestra.run.smithi094.stdout: "num_blocked_ops": 0 2024-09-14T06:32:10.952 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-14T06:32:10.960 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-14T06:32:11.001 INFO:tasks.ceph.osd.1.smithi094.stderr:2024-09-14T06:32:11.000+0000 7f328e47a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33641) UID: 0 2024-09-14T06:32:11.017 INFO:teuthology.orchestra.run.smithi094.stdout:osd.4: {} 2024-09-14T06:32:11.017 INFO:teuthology.orchestra.run.smithi094.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-14T06:32:11.034 INFO:teuthology.orchestra.run.smithi094.stdout:osd.5: {} 2024-09-14T06:32:11.034 INFO:teuthology.orchestra.run.smithi094.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-14T06:32:11.050 INFO:teuthology.orchestra.run.smithi094.stdout:osd.6: {} 2024-09-14T06:32:11.051 INFO:teuthology.orchestra.run.smithi094.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-14T06:32:11.067 INFO:teuthology.orchestra.run.smithi094.stdout:osd.7: {} 2024-09-14T06:32:11.068 INFO:teuthology.orchestra.run.smithi094.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-14T06:32:11.084 INFO:tasks.ceph.osd.4.smithi138.stderr:2024-09-14T06:32:11.084+0000 7f6c653c8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33615) UID: 0 2024-09-14T06:32:11.107 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-14T06:32:11.107 INFO:teuthology.orchestra.run.smithi094.stdout: "size": 20, 2024-09-14T06:32:11.107 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 600, 2024-09-14T06:32:11.107 INFO:teuthology.orchestra.run.smithi094.stdout: "ops": [ 2024-09-14T06:32:11.107 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:11.107 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:115193 3.1 3:856be886:::rbd_data.112d4d24c558.0000000000000004:29 [read 1523712~16384 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e96)", 2024-09-14T06:32:11.107 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:30:17.939854+0000", 2024-09-14T06:32:11.107 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 113.165092547, 2024-09-14T06:32:11.107 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.073897516999999996, 2024-09-14T06:32:11.107 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:32:11.107 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:32:11.107 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:32:11.107 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:32:11.108 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:32:11.108 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:11.108 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 115193 2024-09-14T06:32:11.108 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:11.108 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:32:11.108 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:11.108 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:32:11.108 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.939854+0000", 2024-09-14T06:32:11.108 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:11.108 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:11.108 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:11.108 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:32:11.108 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.939854+0000", 2024-09-14T06:32:11.108 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:11.109 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:11.109 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:11.109 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:32:11.109 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.939854+0000", 2024-09-14T06:32:11.109 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.9500000000000002e-07 2024-09-14T06:32:11.109 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:11.109 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:11.109 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:32:11.109 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.939856+0000", 2024-09-14T06:32:11.109 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.0959999999999999e-06 2024-09-14T06:32:11.109 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:11.109 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:11.109 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:32:11.109 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.939863+0000", 2024-09-14T06:32:11.110 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.1129999999999997e-06 2024-09-14T06:32:11.110 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:11.110 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:11.110 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:32:11.110 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.939867+0000", 2024-09-14T06:32:11.110 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.8350000000000003e-06 2024-09-14T06:32:11.110 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:11.110 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:11.110 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:32:11.110 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.013626+0000", 2024-09-14T06:32:11.110 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.073758799999999999 2024-09-14T06:32:11.110 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:11.110 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:11.110 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:32:11.111 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.013690+0000", 2024-09-14T06:32:11.111 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.3918999999999996e-05 2024-09-14T06:32:11.111 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:11.111 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:11.111 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:32:11.111 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.013751+0000", 2024-09-14T06:32:11.111 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.1159000000000002e-05 2024-09-14T06:32:11.111 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:11.111 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:32:11.111 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:11.111 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:11.112 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:11.112 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:115194 3.1 3:856be886:::rbd_data.112d4d24c558.0000000000000004:29 [read 1769472~16384 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e96)", 2024-09-14T06:32:11.112 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:30:17.939874+0000", 2024-09-14T06:32:11.112 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 113.165072236, 2024-09-14T06:32:11.112 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.074016988000000006, 2024-09-14T06:32:11.112 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:32:11.112 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:32:11.112 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:32:11.112 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:32:11.112 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:32:11.112 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:11.112 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 115194 2024-09-14T06:32:11.112 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:11.113 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:32:11.113 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:11.113 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:32:11.113 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.939874+0000", 2024-09-14T06:32:11.113 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:11.113 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:11.113 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:11.113 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:32:11.113 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.939874+0000", 2024-09-14T06:32:11.113 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:11.113 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:11.113 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:11.113 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:32:11.114 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.939875+0000", 2024-09-14T06:32:11.114 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.2600000000000002e-07 2024-09-14T06:32:11.114 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:11.114 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:11.114 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:32:11.114 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.939877+0000", 2024-09-14T06:32:11.114 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.9420000000000002e-06 2024-09-14T06:32:11.114 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:11.114 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:11.114 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:32:11.114 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.939882+0000", 2024-09-14T06:32:11.114 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.8749999999999997e-06 2024-09-14T06:32:11.114 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:11.114 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:11.115 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:32:11.115 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.939887+0000", 2024-09-14T06:32:11.115 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.741e-06 2024-09-14T06:32:11.115 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:11.115 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:11.115 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:32:11.115 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.013767+0000", 2024-09-14T06:32:11.115 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.073879720999999995 2024-09-14T06:32:11.115 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:11.115 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:11.115 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:32:11.115 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.013832+0000", 2024-09-14T06:32:11.115 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.5312000000000001e-05 2024-09-14T06:32:11.116 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:11.116 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:11.116 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:32:11.116 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.013891+0000", 2024-09-14T06:32:11.116 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.8870999999999999e-05 2024-09-14T06:32:11.116 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:11.116 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:32:11.116 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:11.116 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:11.116 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:11.116 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:115195 3.1 3:856be886:::rbd_data.112d4d24c558.0000000000000004:29 [read 2015232~16384 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e96)", 2024-09-14T06:32:11.116 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:30:17.939903+0000", 2024-09-14T06:32:11.116 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 113.165043051, 2024-09-14T06:32:11.116 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.074128199000000006, 2024-09-14T06:32:11.116 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:32:11.117 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:32:11.117 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:32:11.117 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:32:11.117 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:32:11.117 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:11.117 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 115195 2024-09-14T06:32:11.117 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:11.117 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:32:11.117 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:11.117 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:32:11.117 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.939903+0000", 2024-09-14T06:32:11.117 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:11.117 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:11.118 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:11.118 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:32:11.118 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.939903+0000", 2024-09-14T06:32:11.118 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:11.118 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:11.118 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:11.118 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:32:11.118 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.939904+0000", 2024-09-14T06:32:11.118 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.5799999999999999e-07 2024-09-14T06:32:11.118 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:11.118 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:11.118 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:32:11.118 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.939906+0000", 2024-09-14T06:32:11.118 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.2350000000000002e-06 2024-09-14T06:32:11.119 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:11.119 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:11.119 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:32:11.119 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.939913+0000", 2024-09-14T06:32:11.119 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.5649999999999998e-06 2024-09-14T06:32:11.119 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:11.119 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:11.119 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:32:11.119 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.939918+0000", 2024-09-14T06:32:11.119 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.784e-06 2024-09-14T06:32:11.119 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:11.119 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:11.119 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:32:11.119 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.013921+0000", 2024-09-14T06:32:11.119 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.074002911000000005 2024-09-14T06:32:11.119 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:11.120 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:11.120 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:32:11.120 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.013971+0000", 2024-09-14T06:32:11.120 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.9551e-05 2024-09-14T06:32:11.120 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:11.120 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:11.120 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:32:11.120 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.014031+0000", 2024-09-14T06:32:11.120 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.0594999999999998e-05 2024-09-14T06:32:11.120 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:11.120 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:32:11.120 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:11.120 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:11.120 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:11.121 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:115196 3.1 3:856be886:::rbd_data.112d4d24c558.0000000000000004:29 [read 2260992~16384 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e96)", 2024-09-14T06:32:11.121 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:30:17.939925+0000", 2024-09-14T06:32:11.121 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 113.16502085899999, 2024-09-14T06:32:11.121 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.074320137999999994, 2024-09-14T06:32:11.121 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:32:11.121 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:32:11.121 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:32:11.121 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:32:11.121 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:32:11.121 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:11.121 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 115196 2024-09-14T06:32:11.121 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:11.121 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:32:11.121 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:11.122 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:32:11.122 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.939925+0000", 2024-09-14T06:32:11.122 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:11.122 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:11.122 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:11.122 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:32:11.122 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.939925+0000", 2024-09-14T06:32:11.122 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:11.122 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:11.122 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:11.122 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:32:11.122 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.939926+0000", 2024-09-14T06:32:11.122 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.9999999999999998e-07 2024-09-14T06:32:11.122 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:11.122 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:11.122 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:32:11.123 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.939928+0000", 2024-09-14T06:32:11.123 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.9549999999999999e-06 2024-09-14T06:32:11.123 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:11.123 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:11.123 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:32:11.123 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.939934+0000", 2024-09-14T06:32:11.123 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.127e-06 2024-09-14T06:32:11.123 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:11.123 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:11.123 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:32:11.123 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.939939+0000", 2024-09-14T06:32:11.123 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.775e-06 2024-09-14T06:32:11.123 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:11.124 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:11.124 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:32:11.124 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.014075+0000", 2024-09-14T06:32:11.124 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.074136598999999997 2024-09-14T06:32:11.124 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:11.124 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:11.124 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:32:11.124 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.014162+0000", 2024-09-14T06:32:11.124 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.6477999999999994e-05 2024-09-14T06:32:11.124 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:11.124 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:11.124 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:32:11.124 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.014246+0000", 2024-09-14T06:32:11.124 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.3703999999999997e-05 2024-09-14T06:32:11.124 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:11.125 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:32:11.125 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:11.125 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:11.125 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:11.125 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:115197 3.1 3:856be886:::rbd_data.112d4d24c558.0000000000000004:29 [read 2506752~16384 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e96)", 2024-09-14T06:32:11.125 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:30:17.939969+0000", 2024-09-14T06:32:11.125 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 113.164977374, 2024-09-14T06:32:11.125 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.074528544000000002, 2024-09-14T06:32:11.125 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:32:11.125 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:32:11.125 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:32:11.125 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:32:11.125 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:32:11.125 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:11.125 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 115197 2024-09-14T06:32:11.126 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:11.126 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:32:11.126 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:11.126 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:32:11.126 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.939969+0000", 2024-09-14T06:32:11.126 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:11.126 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:11.126 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:11.126 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:32:11.126 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.939969+0000", 2024-09-14T06:32:11.126 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:11.126 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:11.126 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:11.126 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:32:11.126 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.939970+0000", 2024-09-14T06:32:11.126 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.3600000000000003e-07 2024-09-14T06:32:11.127 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:11.127 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:11.127 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:32:11.127 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.939972+0000", 2024-09-14T06:32:11.127 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.3180000000000001e-06 2024-09-14T06:32:11.127 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:11.127 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:11.127 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:32:11.127 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.939979+0000", 2024-09-14T06:32:11.127 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.8800000000000002e-06 2024-09-14T06:32:11.127 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:11.127 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:11.127 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:32:11.127 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.939984+0000", 2024-09-14T06:32:11.127 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.3510000000000001e-06 2024-09-14T06:32:11.128 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:11.128 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:11.128 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:32:11.128 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.014302+0000", 2024-09-14T06:32:11.128 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.074317416999999997 2024-09-14T06:32:11.128 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:11.128 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:11.128 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:32:11.128 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.014392+0000", 2024-09-14T06:32:11.128 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.0181999999999998e-05 2024-09-14T06:32:11.128 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:11.128 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:11.128 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:32:11.128 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.014497+0000", 2024-09-14T06:32:11.128 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00010576 2024-09-14T06:32:11.129 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:11.129 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:32:11.129 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:11.129 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:11.129 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:11.129 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:115198 3.1 3:856be886:::rbd_data.112d4d24c558.0000000000000004:29 [read 2752512~16384 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e96)", 2024-09-14T06:32:11.129 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:30:17.940004+0000", 2024-09-14T06:32:11.129 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 113.164942094, 2024-09-14T06:32:11.129 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.074736367999999997, 2024-09-14T06:32:11.129 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:32:11.129 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:32:11.129 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:32:11.129 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:32:11.130 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:32:11.130 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:11.130 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 115198 2024-09-14T06:32:11.130 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:11.130 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:32:11.130 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:11.130 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:32:11.130 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940004+0000", 2024-09-14T06:32:11.130 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:11.130 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:11.130 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:11.130 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:32:11.130 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940004+0000", 2024-09-14T06:32:11.130 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:11.130 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:11.130 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:11.131 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:32:11.131 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940005+0000", 2024-09-14T06:32:11.131 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.2200000000000004e-07 2024-09-14T06:32:11.131 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:11.131 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:11.131 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:32:11.131 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940007+0000", 2024-09-14T06:32:11.131 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.322e-06 2024-09-14T06:32:11.131 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:11.131 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:11.131 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:32:11.131 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940014+0000", 2024-09-14T06:32:11.131 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.6220000000000003e-06 2024-09-14T06:32:11.131 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:11.131 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:11.132 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:32:11.132 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940019+0000", 2024-09-14T06:32:11.132 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.6380000000000001e-06 2024-09-14T06:32:11.132 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:11.132 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:11.132 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:32:11.132 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.014562+0000", 2024-09-14T06:32:11.132 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.074542320999999995 2024-09-14T06:32:11.132 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:11.132 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:11.132 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:32:11.132 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.014646+0000", 2024-09-14T06:32:11.132 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.4079999999999995e-05 2024-09-14T06:32:11.132 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:11.133 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:11.133 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:32:11.133 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.014741+0000", 2024-09-14T06:32:11.133 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.4763000000000001e-05 2024-09-14T06:32:11.133 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:11.133 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:32:11.133 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:11.133 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:11.133 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:11.133 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:115199 3.1 3:856be886:::rbd_data.112d4d24c558.0000000000000004:29 [read 2998272~16384 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e96)", 2024-09-14T06:32:11.133 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:30:17.940040+0000", 2024-09-14T06:32:11.134 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 113.164906339, 2024-09-14T06:32:11.134 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.074957134999999994, 2024-09-14T06:32:11.134 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:32:11.134 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:32:11.134 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:32:11.134 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:32:11.134 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:32:11.134 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:11.134 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 115199 2024-09-14T06:32:11.134 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:11.134 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:32:11.134 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:11.134 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:32:11.135 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940040+0000", 2024-09-14T06:32:11.135 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:11.135 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:11.135 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:11.135 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:32:11.135 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940040+0000", 2024-09-14T06:32:11.135 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:11.135 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:11.135 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:11.135 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:32:11.135 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940041+0000", 2024-09-14T06:32:11.135 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.4000000000000001e-07 2024-09-14T06:32:11.135 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:11.135 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:11.136 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:32:11.136 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940046+0000", 2024-09-14T06:32:11.136 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.7250000000000002e-06 2024-09-14T06:32:11.136 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:11.136 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:11.136 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:32:11.136 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940053+0000", 2024-09-14T06:32:11.136 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.6109999999999999e-06 2024-09-14T06:32:11.136 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:11.136 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:11.136 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:32:11.136 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940058+0000", 2024-09-14T06:32:11.136 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.1590000000000001e-06 2024-09-14T06:32:11.137 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:11.137 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:11.137 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:32:11.137 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.014768+0000", 2024-09-14T06:32:11.137 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.074709624000000002 2024-09-14T06:32:11.137 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:11.137 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:11.137 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:32:11.137 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.014907+0000", 2024-09-14T06:32:11.137 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000139665 2024-09-14T06:32:11.137 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:11.137 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:11.137 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:32:11.138 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.014997+0000", 2024-09-14T06:32:11.138 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.9710999999999995e-05 2024-09-14T06:32:11.138 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:11.138 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:32:11.138 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:11.138 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:11.138 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:11.138 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:115200 3.1 3:856be886:::rbd_data.112d4d24c558.0000000000000004:29 [read 3244032~16384 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e96)", 2024-09-14T06:32:11.138 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:30:17.940080+0000", 2024-09-14T06:32:11.138 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 113.16486613399999, 2024-09-14T06:32:11.138 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.075137291999999994, 2024-09-14T06:32:11.138 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:32:11.138 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:32:11.138 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:32:11.138 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:32:11.139 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:32:11.139 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:11.139 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 115200 2024-09-14T06:32:11.139 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:11.139 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:32:11.139 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:11.139 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:32:11.139 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940080+0000", 2024-09-14T06:32:11.139 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:11.139 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:11.139 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:11.139 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:32:11.139 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940080+0000", 2024-09-14T06:32:11.139 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:11.140 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:11.140 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:11.140 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:32:11.140 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940081+0000", 2024-09-14T06:32:11.140 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.0800000000000004e-07 2024-09-14T06:32:11.140 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:11.140 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:11.140 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:32:11.140 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940083+0000", 2024-09-14T06:32:11.140 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.3920000000000001e-06 2024-09-14T06:32:11.140 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:11.140 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:11.140 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:32:11.140 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940091+0000", 2024-09-14T06:32:11.141 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.1389999999999995e-06 2024-09-14T06:32:11.141 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:11.141 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:11.141 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:32:11.141 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940097+0000", 2024-09-14T06:32:11.141 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.5550000000000002e-06 2024-09-14T06:32:11.141 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:11.141 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:11.141 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:32:11.141 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.015025+0000", 2024-09-14T06:32:11.141 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.074928579999999995 2024-09-14T06:32:11.141 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:11.141 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:11.141 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:32:11.141 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.015096+0000", 2024-09-14T06:32:11.142 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.0705999999999997e-05 2024-09-14T06:32:11.142 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:11.142 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:11.142 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:32:11.142 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.015217+0000", 2024-09-14T06:32:11.142 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00012131200000000001 2024-09-14T06:32:11.142 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:11.142 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:32:11.142 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:11.142 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:11.142 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:11.142 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:115201 3.1 3:856be886:::rbd_data.112d4d24c558.0000000000000004:29 [read 3489792~16384 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e96)", 2024-09-14T06:32:11.142 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:30:17.940113+0000", 2024-09-14T06:32:11.142 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 113.16483336100001, 2024-09-14T06:32:11.142 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.075246899000000006, 2024-09-14T06:32:11.143 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:32:11.143 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:32:11.143 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:32:11.143 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:32:11.143 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:32:11.143 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:11.143 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 115201 2024-09-14T06:32:11.143 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:11.143 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:32:11.143 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:11.143 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:32:11.143 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940113+0000", 2024-09-14T06:32:11.143 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:11.143 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:11.143 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:11.143 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:32:11.144 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940113+0000", 2024-09-14T06:32:11.144 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:11.144 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:11.144 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:11.144 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:32:11.144 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940114+0000", 2024-09-14T06:32:11.144 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.1500000000000004e-07 2024-09-14T06:32:11.144 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:11.144 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:11.144 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:32:11.144 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940116+0000", 2024-09-14T06:32:11.144 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.2809999999999998e-06 2024-09-14T06:32:11.144 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:11.144 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:11.144 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:32:11.144 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940123+0000", 2024-09-14T06:32:11.145 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.9229999999999996e-06 2024-09-14T06:32:11.145 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:11.145 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:11.145 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:32:11.145 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940128+0000", 2024-09-14T06:32:11.145 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.7790000000000003e-06 2024-09-14T06:32:11.145 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:11.145 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:11.145 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:32:11.145 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.015253+0000", 2024-09-14T06:32:11.145 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.075125792999999996 2024-09-14T06:32:11.145 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:11.145 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:11.145 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:32:11.145 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.015297+0000", 2024-09-14T06:32:11.146 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.3847000000000002e-05 2024-09-14T06:32:11.146 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:11.146 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:11.146 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:32:11.146 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.015360+0000", 2024-09-14T06:32:11.146 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.2661000000000002e-05 2024-09-14T06:32:11.146 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:11.146 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:32:11.146 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:11.146 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:11.146 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:11.146 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:115202 3.1 3:856be886:::rbd_data.112d4d24c558.0000000000000004:29 [read 3735552~16384 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e96)", 2024-09-14T06:32:11.146 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:30:17.940135+0000", 2024-09-14T06:32:11.146 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 113.164811582, 2024-09-14T06:32:11.146 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.075451105000000004, 2024-09-14T06:32:11.146 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:32:11.147 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:32:11.147 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:32:11.147 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:32:11.147 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:32:11.147 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:11.147 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 115202 2024-09-14T06:32:11.147 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:11.147 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:32:11.147 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:11.147 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:32:11.147 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940135+0000", 2024-09-14T06:32:11.147 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:11.147 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:11.147 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:11.147 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:32:11.148 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940135+0000", 2024-09-14T06:32:11.148 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:11.148 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:11.148 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:11.148 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:32:11.148 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940135+0000", 2024-09-14T06:32:11.148 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.0999999999999999e-07 2024-09-14T06:32:11.148 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:11.148 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:11.148 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:32:11.148 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940137+0000", 2024-09-14T06:32:11.148 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.0020000000000001e-06 2024-09-14T06:32:11.149 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:11.149 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:11.149 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:32:11.149 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940143+0000", 2024-09-14T06:32:11.149 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.9850000000000003e-06 2024-09-14T06:32:11.149 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:11.149 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:11.149 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:32:11.149 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940148+0000", 2024-09-14T06:32:11.149 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.9880000000000004e-06 2024-09-14T06:32:11.149 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:11.149 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:11.149 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:32:11.149 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.015400+0000", 2024-09-14T06:32:11.149 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.075252186999999998 2024-09-14T06:32:11.149 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:11.150 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:11.150 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:32:11.150 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.015481+0000", 2024-09-14T06:32:11.150 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.0698000000000006e-05 2024-09-14T06:32:11.150 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:11.150 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:11.150 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:32:11.150 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.015586+0000", 2024-09-14T06:32:11.150 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00010483499999999999 2024-09-14T06:32:11.150 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:11.150 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:32:11.150 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:11.150 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:11.150 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:11.150 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:115203 3.1 3:856be886:::rbd_data.112d4d24c558.0000000000000004:29 [read 3981312~16384 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e96)", 2024-09-14T06:32:11.151 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:30:17.940156+0000", 2024-09-14T06:32:11.151 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 113.164790357, 2024-09-14T06:32:11.151 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.075620860999999998, 2024-09-14T06:32:11.151 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:32:11.151 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:32:11.151 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:32:11.151 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:32:11.151 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:32:11.151 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:11.151 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 115203 2024-09-14T06:32:11.151 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:11.151 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:32:11.151 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:11.151 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:32:11.151 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940156+0000", 2024-09-14T06:32:11.151 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:11.152 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:11.152 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:11.152 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:32:11.152 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940156+0000", 2024-09-14T06:32:11.152 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:11.152 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:11.152 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:11.152 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:32:11.152 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940156+0000", 2024-09-14T06:32:11.152 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.1500000000000005e-07 2024-09-14T06:32:11.152 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:11.152 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:11.152 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:32:11.152 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940158+0000", 2024-09-14T06:32:11.152 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.967e-06 2024-09-14T06:32:11.152 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:11.153 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:11.153 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:32:11.153 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940165+0000", 2024-09-14T06:32:11.153 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.6939999999999998e-06 2024-09-14T06:32:11.153 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:11.153 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:11.153 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:32:11.153 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940170+0000", 2024-09-14T06:32:11.153 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.7539999999999997e-06 2024-09-14T06:32:11.153 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:11.153 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:11.153 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:32:11.153 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.015633+0000", 2024-09-14T06:32:11.153 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.075463334000000007 2024-09-14T06:32:11.153 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:11.154 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:11.154 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:32:11.154 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.015693+0000", 2024-09-14T06:32:11.154 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.9367000000000003e-05 2024-09-14T06:32:11.154 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:11.154 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:11.154 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:32:11.154 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.015777+0000", 2024-09-14T06:32:11.154 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.4229999999999998e-05 2024-09-14T06:32:11.154 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:11.154 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:32:11.154 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:11.154 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:11.155 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:11.155 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:115204 3.1 3:856be886:::rbd_data.112d4d24c558.0000000000000004:29 [read 4227072~16384 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e96)", 2024-09-14T06:32:11.155 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:30:17.940176+0000", 2024-09-14T06:32:11.155 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 113.164769824, 2024-09-14T06:32:11.155 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.075785505000000003, 2024-09-14T06:32:11.155 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:32:11.155 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:32:11.155 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:32:11.155 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:32:11.155 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:32:11.155 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:11.155 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 115204 2024-09-14T06:32:11.155 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:11.155 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:32:11.155 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:11.156 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:32:11.156 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940176+0000", 2024-09-14T06:32:11.156 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:11.156 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:11.156 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:11.156 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:32:11.156 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940176+0000", 2024-09-14T06:32:11.156 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:11.156 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:11.156 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:11.156 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:32:11.156 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940177+0000", 2024-09-14T06:32:11.157 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.7899999999999999e-07 2024-09-14T06:32:11.157 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:11.157 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:11.157 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:32:11.157 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940179+0000", 2024-09-14T06:32:11.157 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.103e-06 2024-09-14T06:32:11.157 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:11.157 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:11.157 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:32:11.157 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940185+0000", 2024-09-14T06:32:11.157 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.7239999999999997e-06 2024-09-14T06:32:11.157 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:11.158 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:11.158 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:32:11.158 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940190+0000", 2024-09-14T06:32:11.158 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.7269999999999998e-06 2024-09-14T06:32:11.158 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:11.158 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:11.158 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:32:11.158 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.015814+0000", 2024-09-14T06:32:11.158 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.075624114000000006 2024-09-14T06:32:11.158 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:11.158 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:11.158 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:32:11.159 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.015862+0000", 2024-09-14T06:32:11.159 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.8770999999999998e-05 2024-09-14T06:32:11.159 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:11.159 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:11.159 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:32:11.159 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.015962+0000", 2024-09-14T06:32:11.159 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.9586999999999994e-05 2024-09-14T06:32:11.159 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:11.159 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:32:11.159 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:11.159 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:11.159 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:11.160 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:115205 3.1 3:856be886:::rbd_data.112d4d24c558.0000000000000004:29 [read 4472832~16384 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e96)", 2024-09-14T06:32:11.160 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:30:17.940197+0000", 2024-09-14T06:32:11.160 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 113.16474911500001, 2024-09-14T06:32:11.160 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.076068920999999998, 2024-09-14T06:32:11.160 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:32:11.160 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:32:11.160 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:32:11.160 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:32:11.160 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:32:11.160 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:11.160 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 115205 2024-09-14T06:32:11.160 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:11.161 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:32:11.161 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:11.161 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:32:11.161 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940197+0000", 2024-09-14T06:32:11.161 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:11.161 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:11.161 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:11.161 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:32:11.161 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940197+0000", 2024-09-14T06:32:11.161 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:11.161 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:11.161 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:11.162 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:32:11.162 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940198+0000", 2024-09-14T06:32:11.162 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.5999999999999999e-07 2024-09-14T06:32:11.162 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:11.162 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:11.162 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:32:11.162 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940200+0000", 2024-09-14T06:32:11.162 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.0899999999999999e-06 2024-09-14T06:32:11.162 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:11.162 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:11.162 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:32:11.162 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940206+0000", 2024-09-14T06:32:11.162 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.8710000000000002e-06 2024-09-14T06:32:11.162 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:11.162 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:11.163 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:32:11.163 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940211+0000", 2024-09-14T06:32:11.163 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.9230000000000001e-06 2024-09-14T06:32:11.163 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:11.163 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:11.163 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:32:11.163 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.016059+0000", 2024-09-14T06:32:11.163 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.075848051999999999 2024-09-14T06:32:11.163 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:11.163 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:11.163 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:32:11.163 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.016151+0000", 2024-09-14T06:32:11.163 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.1982000000000001e-05 2024-09-14T06:32:11.163 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:11.164 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:11.164 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:32:11.164 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.016266+0000", 2024-09-14T06:32:11.164 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000115543 2024-09-14T06:32:11.164 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:11.164 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:32:11.164 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:11.164 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:11.164 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:11.164 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:115206 3.1 3:856be886:::rbd_data.112d4d24c558.0000000000000004:29 [read 4718592~16384 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e96)", 2024-09-14T06:32:11.164 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:30:17.940223+0000", 2024-09-14T06:32:11.164 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 113.164723025, 2024-09-14T06:32:11.164 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.076265115999999994, 2024-09-14T06:32:11.164 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:32:11.165 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:32:11.165 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:32:11.165 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:32:11.165 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:32:11.165 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:11.165 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 115206 2024-09-14T06:32:11.165 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:11.165 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:32:11.165 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:11.165 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:32:11.165 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940223+0000", 2024-09-14T06:32:11.165 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:11.165 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:11.166 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:11.166 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:32:11.166 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940223+0000", 2024-09-14T06:32:11.166 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:11.166 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:11.166 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:11.166 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:32:11.166 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940224+0000", 2024-09-14T06:32:11.166 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.0399999999999996e-07 2024-09-14T06:32:11.166 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:11.166 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:11.166 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:32:11.166 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940226+0000", 2024-09-14T06:32:11.167 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.1780000000000002e-06 2024-09-14T06:32:11.167 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:11.167 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:11.167 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:32:11.167 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940232+0000", 2024-09-14T06:32:11.167 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.5010000000000001e-06 2024-09-14T06:32:11.167 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:11.167 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:11.167 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:32:11.167 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940237+0000", 2024-09-14T06:32:11.168 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.0250000000000002e-06 2024-09-14T06:32:11.168 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:11.168 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:11.168 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:32:11.168 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.016305+0000", 2024-09-14T06:32:11.168 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.076067262999999996 2024-09-14T06:32:11.168 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:11.168 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:11.168 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:32:11.168 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.016383+0000", 2024-09-14T06:32:11.168 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.8578000000000006e-05 2024-09-14T06:32:11.168 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:11.168 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:11.168 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:32:11.169 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.016488+0000", 2024-09-14T06:32:11.169 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000105067 2024-09-14T06:32:11.169 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:11.169 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:32:11.169 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:11.169 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:11.169 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:11.169 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:115207 3.1 3:856be886:::rbd_data.112d4d24c558.0000000000000004:29 [read 4964352~16384 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e96)", 2024-09-14T06:32:11.169 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:30:17.940244+0000", 2024-09-14T06:32:11.169 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 113.164701909, 2024-09-14T06:32:11.169 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.076371154999999996, 2024-09-14T06:32:11.169 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:32:11.169 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:32:11.169 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:32:11.169 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:32:11.169 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:32:11.170 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:11.170 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 115207 2024-09-14T06:32:11.170 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:11.170 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:32:11.170 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:11.170 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:32:11.170 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940244+0000", 2024-09-14T06:32:11.170 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:11.170 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:11.170 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:11.170 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:32:11.170 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940244+0000", 2024-09-14T06:32:11.170 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:11.170 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:11.170 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:11.171 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:32:11.171 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940245+0000", 2024-09-14T06:32:11.171 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.1200000000000003e-07 2024-09-14T06:32:11.171 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:11.171 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:11.171 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:32:11.171 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940247+0000", 2024-09-14T06:32:11.171 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.0030000000000001e-06 2024-09-14T06:32:11.171 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:11.171 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:11.171 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:32:11.171 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940252+0000", 2024-09-14T06:32:11.171 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.5520000000000004e-06 2024-09-14T06:32:11.171 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:11.171 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:11.171 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:32:11.172 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940257+0000", 2024-09-14T06:32:11.172 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.6759999999999998e-06 2024-09-14T06:32:11.172 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:11.172 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:11.172 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:32:11.172 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.016521+0000", 2024-09-14T06:32:11.172 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.076263499999999998 2024-09-14T06:32:11.172 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:11.172 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:11.172 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:32:11.172 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.016562+0000", 2024-09-14T06:32:11.172 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.0967000000000003e-05 2024-09-14T06:32:11.172 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:11.172 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:11.172 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:32:11.172 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.016616+0000", 2024-09-14T06:32:11.173 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.3944999999999999e-05 2024-09-14T06:32:11.173 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:11.173 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:32:11.173 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:11.173 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:11.173 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:11.173 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:115208 3.1 3:856be886:::rbd_data.112d4d24c558.0000000000000004:29 [read 5210112~16384 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e96)", 2024-09-14T06:32:11.173 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:30:17.940264+0000", 2024-09-14T06:32:11.173 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 113.164682052, 2024-09-14T06:32:11.173 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.076558288000000002, 2024-09-14T06:32:11.173 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:32:11.173 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:32:11.173 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:32:11.174 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:32:11.174 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:32:11.174 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:11.174 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 115208 2024-09-14T06:32:11.174 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:11.174 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:32:11.174 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:11.174 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:32:11.174 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940264+0000", 2024-09-14T06:32:11.174 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:11.174 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:11.174 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:11.174 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:32:11.174 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940264+0000", 2024-09-14T06:32:11.174 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:11.175 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:11.175 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:11.175 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:32:11.175 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940265+0000", 2024-09-14T06:32:11.175 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.0500000000000004e-07 2024-09-14T06:32:11.175 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:11.175 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:11.175 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:32:11.175 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940267+0000", 2024-09-14T06:32:11.175 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.9810000000000002e-06 2024-09-14T06:32:11.175 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:11.175 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:11.176 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:32:11.176 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940272+0000", 2024-09-14T06:32:11.176 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.6999999999999996e-06 2024-09-14T06:32:11.176 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:11.176 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:11.176 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:32:11.176 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940277+0000", 2024-09-14T06:32:11.176 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.6480000000000002e-06 2024-09-14T06:32:11.176 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:11.176 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:11.176 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:32:11.176 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.016650+0000", 2024-09-14T06:32:11.176 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.076373089000000005 2024-09-14T06:32:11.176 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:11.176 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:11.177 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:32:11.177 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.016724+0000", 2024-09-14T06:32:11.177 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.4105999999999998e-05 2024-09-14T06:32:11.177 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:11.177 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:11.177 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:32:11.177 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.016823+0000", 2024-09-14T06:32:11.177 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.8258999999999996e-05 2024-09-14T06:32:11.177 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:11.177 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:32:11.177 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:11.177 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:11.177 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:11.178 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:115209 3.1 3:856be886:::rbd_data.112d4d24c558.0000000000000004:29 [read 5455872~16384 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e96)", 2024-09-14T06:32:11.178 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:30:17.940283+0000", 2024-09-14T06:32:11.178 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 113.164663026, 2024-09-14T06:32:11.178 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.076724029999999999, 2024-09-14T06:32:11.178 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:32:11.178 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:32:11.178 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:32:11.178 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:32:11.178 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:32:11.178 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:11.178 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 115209 2024-09-14T06:32:11.178 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:11.178 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:32:11.178 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:11.178 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:32:11.179 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940283+0000", 2024-09-14T06:32:11.179 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:11.179 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:11.179 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:11.179 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:32:11.179 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940283+0000", 2024-09-14T06:32:11.179 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:11.179 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:11.179 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:11.179 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:32:11.179 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940284+0000", 2024-09-14T06:32:11.179 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.6700000000000003e-07 2024-09-14T06:32:11.179 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:11.180 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:11.180 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:32:11.180 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940286+0000", 2024-09-14T06:32:11.180 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.9520000000000001e-06 2024-09-14T06:32:11.180 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:11.180 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:11.180 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:32:11.180 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940292+0000", 2024-09-14T06:32:11.180 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.7339999999999996e-06 2024-09-14T06:32:11.180 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:11.180 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:11.180 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:32:11.180 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940296+0000", 2024-09-14T06:32:11.180 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.7700000000000001e-06 2024-09-14T06:32:11.181 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:11.181 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:11.181 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:32:11.181 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.016859+0000", 2024-09-14T06:32:11.181 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.07656317 2024-09-14T06:32:11.181 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:11.181 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:11.181 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:32:11.181 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.016929+0000", 2024-09-14T06:32:11.181 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.9679999999999997e-05 2024-09-14T06:32:11.181 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:11.181 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:11.181 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:32:11.181 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.017007+0000", 2024-09-14T06:32:11.181 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.8157000000000004e-05 2024-09-14T06:32:11.182 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:11.182 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:32:11.182 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:11.182 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:11.182 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:11.182 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:115210 3.1 3:856be886:::rbd_data.112d4d24c558.0000000000000004:29 [read 5701632~16384 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e96)", 2024-09-14T06:32:11.182 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:30:17.940303+0000", 2024-09-14T06:32:11.182 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 113.164643509, 2024-09-14T06:32:11.182 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.076821358000000006, 2024-09-14T06:32:11.182 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:32:11.182 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:32:11.182 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:32:11.182 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:32:11.182 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:32:11.182 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:11.183 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 115210 2024-09-14T06:32:11.183 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:11.183 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:32:11.183 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:11.183 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:32:11.183 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940303+0000", 2024-09-14T06:32:11.183 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:11.183 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:11.183 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:11.183 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:32:11.183 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940303+0000", 2024-09-14T06:32:11.183 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:11.183 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:11.183 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:11.183 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:32:11.183 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940303+0000", 2024-09-14T06:32:11.184 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.3000000000000001e-07 2024-09-14T06:32:11.184 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:11.184 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:11.184 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:32:11.184 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940306+0000", 2024-09-14T06:32:11.184 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.255e-06 2024-09-14T06:32:11.184 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:11.184 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:11.184 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:32:11.184 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940312+0000", 2024-09-14T06:32:11.184 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.9710000000000001e-06 2024-09-14T06:32:11.184 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:11.184 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:11.184 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:32:11.184 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940316+0000", 2024-09-14T06:32:11.184 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.5340000000000001e-06 2024-09-14T06:32:11.185 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:11.185 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:11.185 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:32:11.185 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.017033+0000", 2024-09-14T06:32:11.185 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.076717169000000002 2024-09-14T06:32:11.185 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:11.185 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:11.185 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:32:11.185 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.017073+0000", 2024-09-14T06:32:11.185 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.9582000000000002e-05 2024-09-14T06:32:11.185 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:11.185 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:11.185 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:32:11.185 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.017124+0000", 2024-09-14T06:32:11.185 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.1316999999999997e-05 2024-09-14T06:32:11.186 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:11.186 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:32:11.186 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:11.186 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:11.186 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:11.186 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:115219 3.1 3:856be886:::rbd_data.112d4d24c558.0000000000000004:29 [read 65536~16384 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e96)", 2024-09-14T06:32:11.186 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:30:17.940323+0000", 2024-09-14T06:32:11.186 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 113.164623648, 2024-09-14T06:32:11.186 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.076915205, 2024-09-14T06:32:11.186 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:32:11.186 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:32:11.186 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:32:11.186 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:32:11.186 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:32:11.186 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:11.186 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 115219 2024-09-14T06:32:11.187 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:11.187 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:32:11.187 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:11.187 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:32:11.187 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940323+0000", 2024-09-14T06:32:11.187 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:11.187 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:11.187 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:11.187 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:32:11.187 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940323+0000", 2024-09-14T06:32:11.187 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:11.187 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:11.187 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:11.187 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:32:11.187 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940323+0000", 2024-09-14T06:32:11.187 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.0800000000000005e-07 2024-09-14T06:32:11.188 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:11.188 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:11.188 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:32:11.188 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940328+0000", 2024-09-14T06:32:11.188 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.887e-06 2024-09-14T06:32:11.188 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:11.188 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:11.188 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:32:11.188 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940334+0000", 2024-09-14T06:32:11.188 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.3269999999999997e-06 2024-09-14T06:32:11.188 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:11.188 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:11.188 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:32:11.188 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940339+0000", 2024-09-14T06:32:11.188 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.7960000000000003e-06 2024-09-14T06:32:11.188 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:11.189 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:11.189 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:32:11.189 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.017148+0000", 2024-09-14T06:32:11.189 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.076809125000000006 2024-09-14T06:32:11.189 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:11.189 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:11.189 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:32:11.189 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.017186+0000", 2024-09-14T06:32:11.189 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.7283000000000002e-05 2024-09-14T06:32:11.189 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:11.189 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:11.189 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:32:11.189 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.017238+0000", 2024-09-14T06:32:11.189 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.2278999999999999e-05 2024-09-14T06:32:11.189 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:11.190 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:32:11.190 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:11.190 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:11.190 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:11.190 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:115220 3.1 3:856be886:::rbd_data.112d4d24c558.0000000000000004:29 [read 311296~16384 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e96)", 2024-09-14T06:32:11.190 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:30:17.940346+0000", 2024-09-14T06:32:11.190 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 113.164600519, 2024-09-14T06:32:11.190 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.077033293000000003, 2024-09-14T06:32:11.190 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:32:11.190 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:32:11.190 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:32:11.190 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:32:11.190 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:32:11.190 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:11.190 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 115220 2024-09-14T06:32:11.191 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:11.191 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:32:11.191 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:11.191 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:32:11.191 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940346+0000", 2024-09-14T06:32:11.191 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:11.191 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:11.191 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:11.191 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:32:11.191 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940346+0000", 2024-09-14T06:32:11.191 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:11.191 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:11.191 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:11.191 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:32:11.191 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940346+0000", 2024-09-14T06:32:11.192 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.4300000000000003e-07 2024-09-14T06:32:11.192 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:11.192 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:11.192 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:32:11.192 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940348+0000", 2024-09-14T06:32:11.192 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.0030000000000001e-06 2024-09-14T06:32:11.192 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:11.192 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:11.192 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:32:11.192 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.004655+0000", 2024-09-14T06:32:11.192 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.064306323999999998 2024-09-14T06:32:11.193 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:11.193 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:11.193 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:32:11.193 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.004671+0000", 2024-09-14T06:32:11.193 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.6589999999999999e-05 2024-09-14T06:32:11.194 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:11.194 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:11.194 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:32:11.194 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.017262+0000", 2024-09-14T06:32:11.194 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.012590570000000001 2024-09-14T06:32:11.195 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:11.195 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:11.195 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:32:11.195 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.017321+0000", 2024-09-14T06:32:11.195 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.9210999999999999e-05 2024-09-14T06:32:11.196 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:11.196 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:11.196 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:32:11.196 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.017379+0000", 2024-09-14T06:32:11.196 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.8051999999999999e-05 2024-09-14T06:32:11.196 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:11.197 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:32:11.197 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:11.197 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:11.197 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:32:11.197 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-14T06:32:11.199 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-14T06:32:11.200 INFO:tasks.ceph.osd.6.smithi138.stderr:2024-09-14T06:32:11.197+0000 7f8cd8849640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33620) UID: 0 2024-09-14T06:32:11.299 INFO:tasks.ceph.osd.3.smithi094.stderr:2024-09-14T06:32:11.298+0000 7feeeafe8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33655) UID: 0 2024-09-14T06:32:11.343 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-14T06:32:11.344 INFO:teuthology.orchestra.run.smithi094.stdout: "ops": [], 2024-09-14T06:32:11.344 INFO:teuthology.orchestra.run.smithi094.stdout: "num_ops": 0 2024-09-14T06:32:11.344 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-14T06:32:11.352 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-14T06:32:11.437 INFO:tasks.ceph.osd.3.smithi094.stderr:2024-09-14T06:32:11.436+0000 7feeeafe8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33655) UID: 0 2024-09-14T06:32:11.496 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-14T06:32:11.496 INFO:teuthology.orchestra.run.smithi094.stdout: "ops": [], 2024-09-14T06:32:11.496 INFO:teuthology.orchestra.run.smithi094.stdout: "complaint_time": 30, 2024-09-14T06:32:11.496 INFO:teuthology.orchestra.run.smithi094.stdout: "num_blocked_ops": 0 2024-09-14T06:32:11.496 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-14T06:32:11.501 INFO:tasks.ceph.osd.7.smithi138.stderr:2024-09-14T06:32:11.501+0000 7f3c6a21d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33623) UID: 0 2024-09-14T06:32:11.504 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-14T06:32:11.602 INFO:tasks.ceph.osd.4.smithi138.stderr:2024-09-14T06:32:11.600+0000 7f6c653c8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33615) UID: 0 2024-09-14T06:32:11.649 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-14T06:32:11.649 INFO:teuthology.orchestra.run.smithi094.stdout: "size": 20, 2024-09-14T06:32:11.649 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 600, 2024-09-14T06:32:11.649 INFO:teuthology.orchestra.run.smithi094.stdout: "ops": [ 2024-09-14T06:32:11.649 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:11.649 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:14918 3.0 3:05c39535:::rbd_data.112d92027590.0000000000000009:f [read 795648~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:32:11.649 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:26.083834+0000", 2024-09-14T06:32:11.649 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 165.56339142600001, 2024-09-14T06:32:11.650 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.041776287000000002, 2024-09-14T06:32:11.650 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:32:11.650 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:32:11.650 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:32:11.650 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:32:11.650 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:32:11.650 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:11.650 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 14918 2024-09-14T06:32:11.650 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:11.650 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:32:11.650 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:11.650 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:32:11.650 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.083834+0000", 2024-09-14T06:32:11.650 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:11.651 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:11.651 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:11.651 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:32:11.651 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.083834+0000", 2024-09-14T06:32:11.651 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:11.651 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:11.651 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:11.651 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:32:11.651 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.083834+0000", 2024-09-14T06:32:11.651 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.0100000000000001e-07 2024-09-14T06:32:11.651 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:11.651 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:11.651 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:32:11.652 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.083835+0000", 2024-09-14T06:32:11.652 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.3900000000000003e-07 2024-09-14T06:32:11.652 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:11.652 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:11.652 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:32:11.652 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.083838+0000", 2024-09-14T06:32:11.652 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.9830000000000001e-06 2024-09-14T06:32:11.652 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:11.652 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:11.652 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:32:11.652 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.083840+0000", 2024-09-14T06:32:11.652 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.4679999999999999e-06 2024-09-14T06:32:11.653 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:11.653 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:11.653 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:32:11.653 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.125455+0000", 2024-09-14T06:32:11.653 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.041614578999999999 2024-09-14T06:32:11.653 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:11.653 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:11.653 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:32:11.653 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.125496+0000", 2024-09-14T06:32:11.653 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.1047000000000002e-05 2024-09-14T06:32:11.653 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:11.653 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:11.653 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:32:11.653 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.125610+0000", 2024-09-14T06:32:11.654 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00011407 2024-09-14T06:32:11.654 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:11.654 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:32:11.654 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:11.654 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:11.654 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:11.654 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:14919 3.0 3:05c39535:::rbd_data.112d92027590.0000000000000009:f [read 805888~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:32:11.654 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:26.083843+0000", 2024-09-14T06:32:11.654 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 165.56338167800001, 2024-09-14T06:32:11.654 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.041992356000000002, 2024-09-14T06:32:11.654 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:32:11.654 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:32:11.654 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:32:11.654 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:32:11.655 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:32:11.655 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:11.655 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 14919 2024-09-14T06:32:11.655 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:11.655 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:32:11.655 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:11.655 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:32:11.655 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.083843+0000", 2024-09-14T06:32:11.655 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:11.655 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:11.655 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:11.655 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:32:11.655 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.083843+0000", 2024-09-14T06:32:11.656 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:11.656 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:11.656 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:11.656 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:32:11.656 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.083844+0000", 2024-09-14T06:32:11.656 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.3300000000000001e-07 2024-09-14T06:32:11.656 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:11.656 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:11.656 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:32:11.656 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.083845+0000", 2024-09-14T06:32:11.656 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.4729999999999999e-06 2024-09-14T06:32:11.656 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:11.656 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:11.656 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:32:11.656 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.083850+0000", 2024-09-14T06:32:11.657 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.0130000000000001e-06 2024-09-14T06:32:11.657 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:11.657 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:11.657 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:32:11.657 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.083854+0000", 2024-09-14T06:32:11.657 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.4759999999999998e-06 2024-09-14T06:32:11.657 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:11.657 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:11.657 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:32:11.657 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.125636+0000", 2024-09-14T06:32:11.657 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.041782683000000001 2024-09-14T06:32:11.657 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:11.657 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:11.657 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:32:11.658 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.125680+0000", 2024-09-14T06:32:11.658 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.3614e-05 2024-09-14T06:32:11.658 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:11.658 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:11.658 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:32:11.658 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.125836+0000", 2024-09-14T06:32:11.658 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000155864 2024-09-14T06:32:11.658 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:11.658 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:32:11.658 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:11.658 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:11.658 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:11.658 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:14920 3.0 3:05c39535:::rbd_data.112d92027590.0000000000000009:f [read 816128~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:32:11.658 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:26.083868+0000", 2024-09-14T06:32:11.658 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 165.56335701699999, 2024-09-14T06:32:11.659 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.042166771999999998, 2024-09-14T06:32:11.659 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:32:11.659 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:32:11.659 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:32:11.659 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:32:11.659 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:32:11.659 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:11.659 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 14920 2024-09-14T06:32:11.659 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:11.659 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:32:11.659 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:11.659 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:32:11.659 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.083868+0000", 2024-09-14T06:32:11.659 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:11.660 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:11.660 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:11.660 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:32:11.660 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.083868+0000", 2024-09-14T06:32:11.660 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:11.660 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:11.660 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:11.660 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:32:11.660 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.083869+0000", 2024-09-14T06:32:11.660 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.99e-07 2024-09-14T06:32:11.660 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:11.660 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:11.661 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:32:11.661 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.083870+0000", 2024-09-14T06:32:11.661 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.782e-06 2024-09-14T06:32:11.661 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:11.661 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:11.661 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:32:11.661 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.083876+0000", 2024-09-14T06:32:11.662 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.3809999999999998e-06 2024-09-14T06:32:11.662 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:11.662 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:11.662 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:32:11.662 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.083882+0000", 2024-09-14T06:32:11.662 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.9479999999999996e-06 2024-09-14T06:32:11.662 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:11.663 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:11.663 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:32:11.663 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.125878+0000", 2024-09-14T06:32:11.663 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.041996264999999998 2024-09-14T06:32:11.663 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:11.663 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:11.663 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:32:11.663 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.125929+0000", 2024-09-14T06:32:11.663 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.0498999999999999e-05 2024-09-14T06:32:11.663 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:11.663 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:11.663 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:32:11.663 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.126035+0000", 2024-09-14T06:32:11.664 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000106298 2024-09-14T06:32:11.664 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:11.664 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:32:11.664 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:11.664 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:11.665 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:11.665 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:14921 3.0 3:05c39535:::rbd_data.112d92027590.0000000000000009:f [read 826368~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:32:11.665 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:26.083889+0000", 2024-09-14T06:32:11.665 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 165.563336007, 2024-09-14T06:32:11.665 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.042435309999999997, 2024-09-14T06:32:11.666 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:32:11.666 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:32:11.666 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:32:11.666 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:32:11.666 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:32:11.666 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:11.667 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 14921 2024-09-14T06:32:11.667 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:11.667 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:32:11.667 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:11.667 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:32:11.667 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.083889+0000", 2024-09-14T06:32:11.667 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:11.667 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:11.667 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:11.667 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:32:11.667 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.083889+0000", 2024-09-14T06:32:11.667 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:11.668 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:11.668 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:11.668 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:32:11.668 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.086033+0000", 2024-09-14T06:32:11.668 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0021444179999999999 2024-09-14T06:32:11.669 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:11.669 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:11.669 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:32:11.669 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.086038+0000", 2024-09-14T06:32:11.669 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.0969999999999999e-06 2024-09-14T06:32:11.669 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:11.669 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:11.669 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:32:11.669 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.086046+0000", 2024-09-14T06:32:11.669 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.9440000000000005e-06 2024-09-14T06:32:11.670 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:11.670 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:11.670 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:32:11.670 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.086050+0000", 2024-09-14T06:32:11.670 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.69e-06 2024-09-14T06:32:11.670 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:11.670 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:11.670 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:32:11.670 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.126080+0000", 2024-09-14T06:32:11.670 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.040030237000000003 2024-09-14T06:32:11.670 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:11.670 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:11.670 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:32:11.670 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.126163+0000", 2024-09-14T06:32:11.670 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.2071999999999996e-05 2024-09-14T06:32:11.671 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:11.671 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:11.671 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:32:11.671 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.126324+0000", 2024-09-14T06:32:11.671 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00016185200000000001 2024-09-14T06:32:11.671 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:11.671 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:32:11.671 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:11.671 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:11.671 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:11.671 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:55627 3.0 3:06490baf:::rbd_data.112dca2f6ae1.0000000000000030:17 [read 2002944~4096 [fadvise_nocache]] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:32:11.671 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:44.599966+0000", 2024-09-14T06:32:11.671 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 147.047259059, 2024-09-14T06:32:11.671 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.041868546999999999, 2024-09-14T06:32:11.671 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:32:11.672 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:32:11.672 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:32:11.672 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:32:11.672 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:32:11.672 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:11.672 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 55627 2024-09-14T06:32:11.672 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:11.672 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:32:11.672 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:11.672 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:32:11.672 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.599966+0000", 2024-09-14T06:32:11.672 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:11.672 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:11.672 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:11.672 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:32:11.673 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.599966+0000", 2024-09-14T06:32:11.673 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:11.673 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:11.673 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:11.673 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:32:11.673 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.599966+0000", 2024-09-14T06:32:11.673 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.0900000000000001e-07 2024-09-14T06:32:11.673 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:11.673 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:11.673 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:32:11.673 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.599967+0000", 2024-09-14T06:32:11.673 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.64e-07 2024-09-14T06:32:11.673 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:11.673 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:11.673 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:32:11.674 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.599970+0000", 2024-09-14T06:32:11.674 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.9670000000000001e-06 2024-09-14T06:32:11.674 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:11.674 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:11.674 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:32:11.674 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.599973+0000", 2024-09-14T06:32:11.674 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.632e-06 2024-09-14T06:32:11.674 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:11.674 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:11.674 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:32:11.674 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.641665+0000", 2024-09-14T06:32:11.674 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.041692547000000003 2024-09-14T06:32:11.674 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:11.674 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:11.674 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:32:11.674 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.641740+0000", 2024-09-14T06:32:11.675 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.4559000000000002e-05 2024-09-14T06:32:11.675 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:11.675 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:11.675 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:32:11.675 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.641835+0000", 2024-09-14T06:32:11.675 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.4668999999999998e-05 2024-09-14T06:32:11.675 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:11.675 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:32:11.675 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:11.675 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:11.675 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:11.676 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:55628 3.0 3:06490baf:::rbd_data.112dca2f6ae1.0000000000000030:17 [read 2027520~4096 [fadvise_nocache]] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:32:11.676 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:44.599976+0000", 2024-09-14T06:32:11.676 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 147.04724927199999, 2024-09-14T06:32:11.676 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.041991621, 2024-09-14T06:32:11.676 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:32:11.676 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:32:11.676 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:32:11.676 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:32:11.676 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:32:11.676 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:11.676 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 55628 2024-09-14T06:32:11.676 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:11.676 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:32:11.676 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:11.676 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:32:11.677 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.599976+0000", 2024-09-14T06:32:11.677 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:11.677 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:11.677 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:11.677 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:32:11.677 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.599976+0000", 2024-09-14T06:32:11.677 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:11.677 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:11.677 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:11.677 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:32:11.677 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.599976+0000", 2024-09-14T06:32:11.677 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.1299999999999999e-07 2024-09-14T06:32:11.677 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:11.677 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:11.678 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:32:11.678 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.599977+0000", 2024-09-14T06:32:11.678 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.7999999999999993e-07 2024-09-14T06:32:11.678 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:11.678 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:11.678 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:32:11.678 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.599980+0000", 2024-09-14T06:32:11.678 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.01e-06 2024-09-14T06:32:11.678 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:11.678 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:11.678 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:32:11.678 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.599983+0000", 2024-09-14T06:32:11.678 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.5409999999999999e-06 2024-09-14T06:32:11.678 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:11.679 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:11.679 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:32:11.679 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.641869+0000", 2024-09-14T06:32:11.679 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.041886235000000001 2024-09-14T06:32:11.679 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:11.679 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:11.679 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:32:11.679 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.641913+0000", 2024-09-14T06:32:11.679 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.4357000000000002e-05 2024-09-14T06:32:11.679 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:11.679 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:11.679 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:32:11.679 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.641967+0000", 2024-09-14T06:32:11.679 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.4284999999999999e-05 2024-09-14T06:32:11.679 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:11.679 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:32:11.680 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:11.680 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:11.680 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:11.680 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:55629 3.0 3:06490baf:::rbd_data.112dca2f6ae1.0000000000000030:17 [read 2052096~4096 [fadvise_nocache]] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:32:11.680 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:44.599985+0000", 2024-09-14T06:32:11.680 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 147.047239566, 2024-09-14T06:32:11.680 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.042183375000000002, 2024-09-14T06:32:11.680 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:32:11.680 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:32:11.680 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:32:11.680 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:32:11.680 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:32:11.680 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:11.680 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 55629 2024-09-14T06:32:11.681 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:11.681 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:32:11.681 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:11.681 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:32:11.681 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.599985+0000", 2024-09-14T06:32:11.681 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:11.681 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:11.681 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:11.681 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:32:11.681 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.599985+0000", 2024-09-14T06:32:11.681 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:11.681 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:11.681 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:11.681 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:32:11.681 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.599986+0000", 2024-09-14T06:32:11.682 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.05e-07 2024-09-14T06:32:11.682 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:11.682 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:11.682 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:32:11.682 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.599987+0000", 2024-09-14T06:32:11.682 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.8400000000000002e-07 2024-09-14T06:32:11.682 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:11.682 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:11.682 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:32:11.682 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.599990+0000", 2024-09-14T06:32:11.682 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.5640000000000001e-06 2024-09-14T06:32:11.682 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:11.682 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:11.682 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:32:11.682 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.599993+0000", 2024-09-14T06:32:11.683 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.615e-06 2024-09-14T06:32:11.683 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:11.683 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:11.683 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:32:11.683 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.642003+0000", 2024-09-14T06:32:11.683 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.042010345999999997 2024-09-14T06:32:11.683 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:11.683 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:11.683 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:32:11.683 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.642080+0000", 2024-09-14T06:32:11.683 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.6538999999999993e-05 2024-09-14T06:32:11.683 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:11.683 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:11.683 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:32:11.683 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.642169+0000", 2024-09-14T06:32:11.683 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.9122000000000005e-05 2024-09-14T06:32:11.684 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:11.684 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:32:11.684 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:11.684 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:11.684 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:11.684 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:55630 3.0 3:06490baf:::rbd_data.112dca2f6ae1.0000000000000030:17 [read 2076672~4096 [fadvise_nocache]] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:32:11.684 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:44.599996+0000", 2024-09-14T06:32:11.684 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 147.04722908799999, 2024-09-14T06:32:11.684 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.042299615999999998, 2024-09-14T06:32:11.684 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:32:11.684 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:32:11.684 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:32:11.684 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:32:11.684 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:32:11.684 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:11.684 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 55630 2024-09-14T06:32:11.685 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:11.685 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:32:11.685 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:11.685 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:32:11.685 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.599996+0000", 2024-09-14T06:32:11.685 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:11.685 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:11.685 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:11.685 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:32:11.685 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.599996+0000", 2024-09-14T06:32:11.685 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:11.685 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:11.685 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:11.685 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:32:11.685 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.599996+0000", 2024-09-14T06:32:11.686 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.1500000000000001e-07 2024-09-14T06:32:11.686 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:11.686 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:11.686 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:32:11.686 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.599997+0000", 2024-09-14T06:32:11.686 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.33e-07 2024-09-14T06:32:11.686 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:11.686 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:11.686 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:32:11.686 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600000+0000", 2024-09-14T06:32:11.686 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.0240000000000002e-06 2024-09-14T06:32:11.686 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:11.686 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:11.686 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:32:11.686 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600003+0000", 2024-09-14T06:32:11.686 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.3640000000000002e-06 2024-09-14T06:32:11.687 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:11.687 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:11.687 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:32:11.687 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.642200+0000", 2024-09-14T06:32:11.687 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.042197571000000003 2024-09-14T06:32:11.687 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:11.687 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:11.687 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:32:11.687 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.642243+0000", 2024-09-14T06:32:11.687 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.2765000000000001e-05 2024-09-14T06:32:11.687 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:11.687 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:11.687 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:32:11.687 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.642296+0000", 2024-09-14T06:32:11.687 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.2744000000000002e-05 2024-09-14T06:32:11.688 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:11.688 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:32:11.688 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:11.688 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:11.688 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:11.688 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:55631 3.0 3:06490baf:::rbd_data.112dca2f6ae1.0000000000000030:17 [read 1048576~4096 [fadvise_nocache]] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:32:11.688 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:44.600006+0000", 2024-09-14T06:32:11.688 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 147.04721950000001, 2024-09-14T06:32:11.688 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.042487905999999999, 2024-09-14T06:32:11.688 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:32:11.688 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:32:11.688 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:32:11.688 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:32:11.688 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:32:11.689 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:11.689 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 55631 2024-09-14T06:32:11.689 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:11.689 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:32:11.689 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:11.689 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:32:11.689 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600006+0000", 2024-09-14T06:32:11.689 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:11.689 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:11.689 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:11.689 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:32:11.689 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600006+0000", 2024-09-14T06:32:11.689 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:11.689 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:11.689 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:11.690 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:32:11.690 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600006+0000", 2024-09-14T06:32:11.690 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.0800000000000001e-07 2024-09-14T06:32:11.690 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:11.690 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:11.690 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:32:11.690 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600007+0000", 2024-09-14T06:32:11.690 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.7199999999999997e-07 2024-09-14T06:32:11.690 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:11.690 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:11.690 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:32:11.690 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600010+0000", 2024-09-14T06:32:11.690 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.0280000000000001e-06 2024-09-14T06:32:11.690 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:11.690 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:11.690 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:32:11.691 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600012+0000", 2024-09-14T06:32:11.691 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.475e-06 2024-09-14T06:32:11.691 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:11.691 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:11.691 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:32:11.691 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.642329+0000", 2024-09-14T06:32:11.691 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.042317157000000001 2024-09-14T06:32:11.691 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:11.691 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:11.691 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:32:11.691 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.642403+0000", 2024-09-14T06:32:11.691 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.3941000000000002e-05 2024-09-14T06:32:11.691 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:11.691 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:11.691 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:32:11.692 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.642493+0000", 2024-09-14T06:32:11.692 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.0124999999999995e-05 2024-09-14T06:32:11.692 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:11.692 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:32:11.692 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:11.692 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:11.692 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:11.692 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:55632 3.0 3:06490baf:::rbd_data.112dca2f6ae1.0000000000000030:17 [read 1073152~4096 [fadvise_nocache]] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:32:11.692 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:44.600015+0000", 2024-09-14T06:32:11.692 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 147.04720974700001, 2024-09-14T06:32:11.692 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.042604922000000003, 2024-09-14T06:32:11.692 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:32:11.692 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:32:11.692 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:32:11.693 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:32:11.693 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:32:11.693 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:11.693 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 55632 2024-09-14T06:32:11.693 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:11.693 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:32:11.693 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:11.693 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:32:11.693 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600015+0000", 2024-09-14T06:32:11.693 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:11.693 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:11.693 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:11.693 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:32:11.693 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600015+0000", 2024-09-14T06:32:11.694 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:11.694 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:11.694 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:11.694 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:32:11.694 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600016+0000", 2024-09-14T06:32:11.694 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.1e-07 2024-09-14T06:32:11.694 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:11.694 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:11.694 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:32:11.694 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600017+0000", 2024-09-14T06:32:11.694 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.9900000000000009e-07 2024-09-14T06:32:11.694 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:11.694 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:11.695 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:32:11.695 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600020+0000", 2024-09-14T06:32:11.695 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.4170000000000001e-06 2024-09-14T06:32:11.695 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:11.695 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:11.695 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:32:11.695 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600023+0000", 2024-09-14T06:32:11.695 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.588e-06 2024-09-14T06:32:11.695 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:11.695 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:11.695 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:32:11.695 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.642526+0000", 2024-09-14T06:32:11.695 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.042503236999999999 2024-09-14T06:32:11.695 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:11.696 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:11.696 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:32:11.696 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.642568+0000", 2024-09-14T06:32:11.696 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.1925000000000003e-05 2024-09-14T06:32:11.696 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:11.696 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:11.696 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:32:11.696 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.642620+0000", 2024-09-14T06:32:11.696 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.2546000000000001e-05 2024-09-14T06:32:11.696 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:11.696 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:32:11.696 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:11.696 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:11.697 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:11.697 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:55633 3.0 3:06490baf:::rbd_data.112dca2f6ae1.0000000000000030:17 [read 1097728~4096 [fadvise_nocache]] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:32:11.697 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:44.600026+0000", 2024-09-14T06:32:11.697 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 147.04719947800001, 2024-09-14T06:32:11.697 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.042797474000000002, 2024-09-14T06:32:11.697 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:32:11.697 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:32:11.697 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:32:11.697 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:32:11.698 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:32:11.698 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:11.698 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 55633 2024-09-14T06:32:11.698 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:11.698 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:32:11.699 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:11.699 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:32:11.699 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600026+0000", 2024-09-14T06:32:11.699 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:11.699 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:11.699 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:11.699 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:32:11.699 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600026+0000", 2024-09-14T06:32:11.699 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:11.699 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:11.699 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:11.700 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:32:11.700 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600026+0000", 2024-09-14T06:32:11.700 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.3099999999999999e-07 2024-09-14T06:32:11.700 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:11.700 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:11.700 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:32:11.700 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600029+0000", 2024-09-14T06:32:11.700 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.304e-06 2024-09-14T06:32:11.700 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:11.700 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:11.700 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:32:11.700 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600034+0000", 2024-09-14T06:32:11.700 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.1429999999999997e-06 2024-09-14T06:32:11.701 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:11.701 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:11.701 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:32:11.701 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600038+0000", 2024-09-14T06:32:11.701 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.692e-06 2024-09-14T06:32:11.701 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:11.701 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:11.701 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:32:11.701 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.642654+0000", 2024-09-14T06:32:11.701 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.042615564000000002 2024-09-14T06:32:11.701 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:11.701 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:11.701 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:32:11.702 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.642728+0000", 2024-09-14T06:32:11.702 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.4018999999999997e-05 2024-09-14T06:32:11.702 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:11.702 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:11.702 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:32:11.702 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.642823+0000", 2024-09-14T06:32:11.702 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.5520999999999996e-05 2024-09-14T06:32:11.702 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:11.702 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:32:11.702 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:11.702 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:11.702 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:11.702 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:55634 3.0 3:06490baf:::rbd_data.112dca2f6ae1.0000000000000030:17 [read 1122304~4096 [fadvise_nocache]] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:32:11.702 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:44.600042+0000", 2024-09-14T06:32:11.703 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 147.04718300100001, 2024-09-14T06:32:11.703 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.042903729000000002, 2024-09-14T06:32:11.703 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:32:11.703 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:32:11.703 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:32:11.703 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:32:11.703 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:32:11.703 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:11.703 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 55634 2024-09-14T06:32:11.703 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:11.703 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:32:11.703 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:11.704 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:32:11.704 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600042+0000", 2024-09-14T06:32:11.704 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:11.704 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:11.704 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:11.704 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:32:11.704 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600042+0000", 2024-09-14T06:32:11.704 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:11.704 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:11.704 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:11.704 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:32:11.704 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600043+0000", 2024-09-14T06:32:11.704 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.7000000000000005e-07 2024-09-14T06:32:11.704 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:11.704 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:11.705 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:32:11.705 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600044+0000", 2024-09-14T06:32:11.705 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.742e-06 2024-09-14T06:32:11.705 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:11.705 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:11.705 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:32:11.705 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600050+0000", 2024-09-14T06:32:11.705 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.0800000000000002e-06 2024-09-14T06:32:11.705 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:11.705 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:11.705 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:32:11.705 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600055+0000", 2024-09-14T06:32:11.705 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.3889999999999998e-06 2024-09-14T06:32:11.706 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:11.706 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:11.706 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:32:11.706 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.642845+0000", 2024-09-14T06:32:11.706 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.042790040000000001 2024-09-14T06:32:11.706 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:11.706 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:11.706 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:32:11.706 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.642888+0000", 2024-09-14T06:32:11.706 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.2790000000000002e-05 2024-09-14T06:32:11.706 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:11.706 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:11.706 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:32:11.707 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.642946+0000", 2024-09-14T06:32:11.707 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.8118000000000002e-05 2024-09-14T06:32:11.707 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:11.707 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:32:11.707 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:11.707 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:11.707 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:11.707 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:55635 3.0 3:06490baf:::rbd_data.112dca2f6ae1.0000000000000030:17 [read 1146880~4096 [fadvise_nocache]] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:32:11.707 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:44.600061+0000", 2024-09-14T06:32:11.707 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 147.04716443000001, 2024-09-14T06:32:11.707 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.043092695, 2024-09-14T06:32:11.707 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:32:11.707 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:32:11.707 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:32:11.708 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:32:11.708 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:32:11.708 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:11.708 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 55635 2024-09-14T06:32:11.708 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:11.708 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:32:11.708 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:11.708 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:32:11.708 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600061+0000", 2024-09-14T06:32:11.708 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:11.708 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:11.709 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:11.709 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:32:11.709 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600061+0000", 2024-09-14T06:32:11.709 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:11.709 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:11.709 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:11.709 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:32:11.709 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600061+0000", 2024-09-14T06:32:11.709 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.3099999999999999e-07 2024-09-14T06:32:11.709 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:11.709 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:11.709 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:32:11.709 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600063+0000", 2024-09-14T06:32:11.709 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.6220000000000001e-06 2024-09-14T06:32:11.709 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:11.710 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:11.710 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:32:11.710 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600068+0000", 2024-09-14T06:32:11.710 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.4790000000000004e-06 2024-09-14T06:32:11.710 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:11.710 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:11.710 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:32:11.710 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600072+0000", 2024-09-14T06:32:11.710 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.3100000000000002e-06 2024-09-14T06:32:11.710 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:11.710 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:11.710 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:32:11.710 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.642987+0000", 2024-09-14T06:32:11.710 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.042914489 2024-09-14T06:32:11.710 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:11.711 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:11.711 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:32:11.711 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.643062+0000", 2024-09-14T06:32:11.711 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.5011000000000004e-05 2024-09-14T06:32:11.711 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:11.711 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:11.711 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:32:11.711 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.643153+0000", 2024-09-14T06:32:11.711 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.1452999999999993e-05 2024-09-14T06:32:11.711 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:11.711 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:32:11.711 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:11.711 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:11.711 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:11.711 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:55636 3.0 3:06490baf:::rbd_data.112dca2f6ae1.0000000000000030:17 [read 1171456~4096 [fadvise_nocache]] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:32:11.712 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:44.600078+0000", 2024-09-14T06:32:11.712 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 147.047147018, 2024-09-14T06:32:11.712 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.043207966, 2024-09-14T06:32:11.712 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:32:11.712 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:32:11.712 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:32:11.712 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:32:11.712 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:32:11.712 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:11.712 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 55636 2024-09-14T06:32:11.712 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:11.712 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:32:11.712 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:11.712 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:32:11.712 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600078+0000", 2024-09-14T06:32:11.713 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:11.713 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:11.713 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:11.713 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:32:11.713 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600078+0000", 2024-09-14T06:32:11.713 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:11.713 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:11.713 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:11.713 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:32:11.713 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600078+0000", 2024-09-14T06:32:11.713 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.34e-07 2024-09-14T06:32:11.713 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:11.713 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:11.713 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:32:11.713 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600080+0000", 2024-09-14T06:32:11.713 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.641e-06 2024-09-14T06:32:11.714 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:11.714 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:11.714 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:32:11.714 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600086+0000", 2024-09-14T06:32:11.714 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.3860000000000004e-06 2024-09-14T06:32:11.714 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:11.714 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:11.714 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:32:11.714 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600091+0000", 2024-09-14T06:32:11.714 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.5970000000000002e-06 2024-09-14T06:32:11.714 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:11.714 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:11.714 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:32:11.715 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.643184+0000", 2024-09-14T06:32:11.715 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.043092893 2024-09-14T06:32:11.715 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:11.715 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:11.715 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:32:11.715 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.643228+0000", 2024-09-14T06:32:11.715 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.4536999999999997e-05 2024-09-14T06:32:11.715 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:11.715 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:11.715 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:32:11.715 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.643286+0000", 2024-09-14T06:32:11.715 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.7578000000000003e-05 2024-09-14T06:32:11.715 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:11.715 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:32:11.715 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:11.716 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:11.716 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:11.716 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:55637 3.0 3:06490baf:::rbd_data.112dca2f6ae1.0000000000000030:17 [read 1196032~4096 [fadvise_nocache]] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:32:11.716 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:44.600097+0000", 2024-09-14T06:32:11.716 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 147.04712845399999, 2024-09-14T06:32:11.716 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.043399124999999997, 2024-09-14T06:32:11.716 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:32:11.716 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:32:11.716 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:32:11.716 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:32:11.716 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:32:11.716 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:11.716 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 55637 2024-09-14T06:32:11.716 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:11.717 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:32:11.717 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:11.717 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:32:11.717 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600097+0000", 2024-09-14T06:32:11.717 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:11.717 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:11.717 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:11.717 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:32:11.717 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600097+0000", 2024-09-14T06:32:11.717 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:11.717 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:11.717 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:11.717 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:32:11.718 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.604372+0000", 2024-09-14T06:32:11.718 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0042751489999999998 2024-09-14T06:32:11.718 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:11.718 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:11.718 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:32:11.718 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.604377+0000", 2024-09-14T06:32:11.718 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.7899999999999999e-06 2024-09-14T06:32:11.718 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:11.718 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:11.718 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:32:11.718 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.604389+0000", 2024-09-14T06:32:11.718 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.2197e-05 2024-09-14T06:32:11.718 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:11.718 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:11.719 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:32:11.719 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.604397+0000", 2024-09-14T06:32:11.719 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.8730000000000007e-06 2024-09-14T06:32:11.719 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:11.719 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:11.719 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:32:11.719 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.643324+0000", 2024-09-14T06:32:11.719 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.038926897000000002 2024-09-14T06:32:11.719 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:11.719 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:11.719 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:32:11.719 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.643402+0000", 2024-09-14T06:32:11.719 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.8063000000000001e-05 2024-09-14T06:32:11.719 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:11.720 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:11.720 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:32:11.720 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.643496+0000", 2024-09-14T06:32:11.720 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.4155999999999998e-05 2024-09-14T06:32:11.720 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:11.720 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:32:11.720 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:11.720 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:11.720 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:11.720 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:55657 3.0 3:06490baf:::rbd_data.112dca2f6ae1.0000000000000030:17 [read 1687552~4096 [fadvise_nocache]] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:32:11.720 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:44.604841+0000", 2024-09-14T06:32:11.720 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 147.04238372500001, 2024-09-14T06:32:11.720 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.041890087999999999, 2024-09-14T06:32:11.720 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:32:11.720 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:32:11.720 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:32:11.721 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:32:11.721 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:32:11.721 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:11.721 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 55657 2024-09-14T06:32:11.721 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:11.721 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:32:11.721 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:11.721 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:32:11.721 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.604841+0000", 2024-09-14T06:32:11.721 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:11.721 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:11.721 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:11.721 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:32:11.722 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.604841+0000", 2024-09-14T06:32:11.722 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:11.722 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:11.722 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:11.722 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:32:11.722 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.604842+0000", 2024-09-14T06:32:11.722 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.0900000000000001e-07 2024-09-14T06:32:11.722 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:11.722 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:11.722 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:32:11.722 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.604846+0000", 2024-09-14T06:32:11.722 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.5759999999999999e-06 2024-09-14T06:32:11.722 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:11.722 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:11.722 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:32:11.723 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.604850+0000", 2024-09-14T06:32:11.723 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.405e-06 2024-09-14T06:32:11.723 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:11.723 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:11.723 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:32:11.723 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.604852+0000", 2024-09-14T06:32:11.723 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.8590000000000001e-06 2024-09-14T06:32:11.723 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:11.723 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:11.723 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:32:11.723 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.646633+0000", 2024-09-14T06:32:11.723 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.041780772000000001 2024-09-14T06:32:11.723 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:11.723 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:11.724 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:32:11.724 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.646677+0000", 2024-09-14T06:32:11.724 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.4115e-05 2024-09-14T06:32:11.724 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:11.724 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:11.724 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:32:11.724 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.646731+0000", 2024-09-14T06:32:11.724 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.4151999999999999e-05 2024-09-14T06:32:11.724 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:11.724 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:32:11.724 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:11.724 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:11.724 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:11.724 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:55658 3.0 3:06490baf:::rbd_data.112dca2f6ae1.0000000000000030:17 [read 1712128~4096 [fadvise_nocache]] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:32:11.724 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:44.604856+0000", 2024-09-14T06:32:11.725 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 147.04236912499999, 2024-09-14T06:32:11.725 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.041986227000000001, 2024-09-14T06:32:11.725 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:32:11.725 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:32:11.725 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:32:11.725 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:32:11.725 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:32:11.725 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:11.725 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 55658 2024-09-14T06:32:11.725 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:11.725 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:32:11.725 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:11.725 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:32:11.725 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.604856+0000", 2024-09-14T06:32:11.726 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:11.726 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:11.726 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:11.726 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:32:11.726 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.604856+0000", 2024-09-14T06:32:11.726 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:11.726 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:11.726 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:11.726 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:32:11.726 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.604856+0000", 2024-09-14T06:32:11.726 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.23e-07 2024-09-14T06:32:11.726 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:11.726 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:11.726 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:32:11.726 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.604857+0000", 2024-09-14T06:32:11.727 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.9999999999999995e-07 2024-09-14T06:32:11.727 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:11.727 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:11.727 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:32:11.727 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.604861+0000", 2024-09-14T06:32:11.727 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.9890000000000003e-06 2024-09-14T06:32:11.727 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:11.727 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:11.727 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:32:11.727 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.604864+0000", 2024-09-14T06:32:11.727 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.7429999999999998e-06 2024-09-14T06:32:11.727 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:11.727 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:11.727 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:32:11.728 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.646744+0000", 2024-09-14T06:32:11.728 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.041880325000000003 2024-09-14T06:32:11.728 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:11.728 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:11.728 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:32:11.728 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.646784+0000", 2024-09-14T06:32:11.728 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.9604000000000002e-05 2024-09-14T06:32:11.728 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:11.728 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:11.728 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:32:11.728 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.646842+0000", 2024-09-14T06:32:11.728 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.8343e-05 2024-09-14T06:32:11.728 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:11.728 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:32:11.728 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:11.729 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:11.729 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:11.729 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:55659 3.0 3:06490baf:::rbd_data.112dca2f6ae1.0000000000000030:17 [read 1736704~4096 [fadvise_nocache]] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:32:11.729 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:44.604867+0000", 2024-09-14T06:32:11.729 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 147.04235797199999, 2024-09-14T06:32:11.729 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.042097082000000001, 2024-09-14T06:32:11.729 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:32:11.729 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:32:11.729 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:32:11.729 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:32:11.729 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:32:11.729 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:11.729 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 55659 2024-09-14T06:32:11.729 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:11.729 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:32:11.730 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:11.730 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:32:11.730 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.604867+0000", 2024-09-14T06:32:11.730 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:11.730 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:11.730 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:11.730 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:32:11.730 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.604867+0000", 2024-09-14T06:32:11.730 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:11.730 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:11.730 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:11.730 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:32:11.730 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.604867+0000", 2024-09-14T06:32:11.730 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.2000000000000001e-07 2024-09-14T06:32:11.730 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:11.731 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:11.731 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:32:11.731 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.604868+0000", 2024-09-14T06:32:11.731 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.04e-06 2024-09-14T06:32:11.731 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:11.731 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:11.731 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:32:11.731 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.604871+0000", 2024-09-14T06:32:11.731 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.9689999999999999e-06 2024-09-14T06:32:11.731 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:11.731 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:11.731 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:32:11.731 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.604874+0000", 2024-09-14T06:32:11.731 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.5069999999999999e-06 2024-09-14T06:32:11.732 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:11.732 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:11.732 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:32:11.732 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.646874+0000", 2024-09-14T06:32:11.732 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.042000513000000003 2024-09-14T06:32:11.732 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:11.732 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:11.732 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:32:11.732 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.646914+0000", 2024-09-14T06:32:11.732 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.9443999999999997e-05 2024-09-14T06:32:11.732 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:11.732 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:11.732 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:32:11.732 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.646964+0000", 2024-09-14T06:32:11.733 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.0389000000000002e-05 2024-09-14T06:32:11.733 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:11.733 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:32:11.733 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:11.733 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:11.733 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:11.733 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:55660 3.0 3:06490baf:::rbd_data.112dca2f6ae1.0000000000000030:17 [read 1761280~4096 [fadvise_nocache]] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:32:11.733 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:44.604877+0000", 2024-09-14T06:32:11.733 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 147.042348223, 2024-09-14T06:32:11.733 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.042196349000000001, 2024-09-14T06:32:11.733 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:32:11.733 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:32:11.733 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:32:11.733 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:32:11.733 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:32:11.734 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:11.734 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 55660 2024-09-14T06:32:11.734 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:11.734 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:32:11.734 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:11.734 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:32:11.734 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.604877+0000", 2024-09-14T06:32:11.734 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:11.734 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:11.734 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:11.734 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:32:11.734 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.604877+0000", 2024-09-14T06:32:11.734 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:11.734 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:11.734 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:11.735 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:32:11.735 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.605810+0000", 2024-09-14T06:32:11.735 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00093336899999999997 2024-09-14T06:32:11.735 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:11.735 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:11.735 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:32:11.735 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.605815+0000", 2024-09-14T06:32:11.735 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.6140000000000002e-06 2024-09-14T06:32:11.735 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:11.735 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:11.735 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:32:11.735 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.605825+0000", 2024-09-14T06:32:11.735 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.8889999999999997e-06 2024-09-14T06:32:11.735 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:11.735 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:11.736 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:32:11.736 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.605831+0000", 2024-09-14T06:32:11.736 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.9800000000000003e-06 2024-09-14T06:32:11.736 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:11.736 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:11.736 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:32:11.736 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.646989+0000", 2024-09-14T06:32:11.736 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.041157878000000002 2024-09-14T06:32:11.736 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:11.736 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:11.736 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:32:11.736 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.647026+0000", 2024-09-14T06:32:11.736 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.7305000000000003e-05 2024-09-14T06:32:11.736 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:11.736 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:11.737 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:32:11.737 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.647073+0000", 2024-09-14T06:32:11.737 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.7314000000000001e-05 2024-09-14T06:32:11.737 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:11.737 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:32:11.737 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:11.737 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:11.737 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:11.737 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:55666 3.0 3:06490baf:::rbd_data.112dca2f6ae1.0000000000000030:17 [read 1908736~4096 [fadvise_nocache]] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:32:11.737 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:44.605898+0000", 2024-09-14T06:32:11.737 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 147.04132747700001, 2024-09-14T06:32:11.738 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.041862887000000001, 2024-09-14T06:32:11.738 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:32:11.738 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:32:11.738 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:32:11.738 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:32:11.738 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:32:11.738 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:11.738 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 55666 2024-09-14T06:32:11.738 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:11.738 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:32:11.738 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:11.738 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:32:11.738 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.605898+0000", 2024-09-14T06:32:11.738 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:11.738 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:11.739 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:11.739 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:32:11.739 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.605898+0000", 2024-09-14T06:32:11.739 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:11.739 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:11.739 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:11.739 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:32:11.739 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.611874+0000", 2024-09-14T06:32:11.739 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0059764049999999997 2024-09-14T06:32:11.739 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:11.739 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:11.739 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:32:11.739 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.611879+0000", 2024-09-14T06:32:11.739 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.8749999999999999e-06 2024-09-14T06:32:11.739 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:11.739 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:11.740 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:32:11.740 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.611887+0000", 2024-09-14T06:32:11.740 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.4109999999999996e-06 2024-09-14T06:32:11.740 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:11.740 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:11.740 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:32:11.740 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.611893+0000", 2024-09-14T06:32:11.740 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.2959999999999998e-06 2024-09-14T06:32:11.740 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:11.740 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:11.740 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:32:11.740 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.647674+0000", 2024-09-14T06:32:11.741 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.035781281999999998 2024-09-14T06:32:11.741 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:11.741 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:11.741 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:32:11.741 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.647711+0000", 2024-09-14T06:32:11.741 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.7434000000000001e-05 2024-09-14T06:32:11.741 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:11.741 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:11.741 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:32:11.741 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.647760+0000", 2024-09-14T06:32:11.741 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.9184000000000002e-05 2024-09-14T06:32:11.741 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:11.741 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:32:11.741 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:11.741 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:11.742 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:32:11.742 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-14T06:32:11.742 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-14T06:32:11.743 INFO:teuthology.orchestra.run.smithi094.stderr:reweighted osd.0 to 1 (10000) 2024-09-14T06:32:11.744 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-14T06:32:11.745 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 1 1 2024-09-14T06:32:11.745 INFO:tasks.ceph.osd.3.smithi094.stderr:2024-09-14T06:32:11.742+0000 7feeeafe8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33655) UID: 0 2024-09-14T06:32:11.844 INFO:tasks.ceph.osd.0.smithi094.stderr:2024-09-14T06:32:11.843+0000 7f07c9333640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33636) UID: 0 2024-09-14T06:32:11.893 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-14T06:32:11.893 INFO:teuthology.orchestra.run.smithi094.stdout: "ops": [], 2024-09-14T06:32:11.893 INFO:teuthology.orchestra.run.smithi094.stdout: "num_ops": 0 2024-09-14T06:32:11.893 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-14T06:32:11.904 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-14T06:32:11.945 INFO:tasks.ceph.osd.2.smithi094.stderr:2024-09-14T06:32:11.944+0000 7faf3269b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33649) UID: 0 2024-09-14T06:32:12.046 INFO:tasks.ceph.osd.3.smithi094.stderr:2024-09-14T06:32:12.045+0000 7feeeafe8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33655) UID: 0 2024-09-14T06:32:12.048 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-14T06:32:12.048 INFO:teuthology.orchestra.run.smithi094.stdout: "ops": [], 2024-09-14T06:32:12.048 INFO:teuthology.orchestra.run.smithi094.stdout: "complaint_time": 30, 2024-09-14T06:32:12.048 INFO:teuthology.orchestra.run.smithi094.stdout: "num_blocked_ops": 0 2024-09-14T06:32:12.048 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-14T06:32:12.057 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-14T06:32:12.147 INFO:tasks.ceph.osd.0.smithi094.stderr:2024-09-14T06:32:12.146+0000 7f07c9333640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33636) UID: 0 2024-09-14T06:32:12.202 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-14T06:32:12.203 INFO:teuthology.orchestra.run.smithi094.stdout: "size": 20, 2024-09-14T06:32:12.203 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 600, 2024-09-14T06:32:12.203 INFO:teuthology.orchestra.run.smithi094.stdout: "ops": [ 2024-09-14T06:32:12.203 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:12.203 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4397.0:45722 3.9 e64/23 3:91fed098:::rbd_data.112d5a737e72.0000000000000001:head v 64'61, mlcod=62'59)", 2024-09-14T06:32:12.203 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:43.864269+0000", 2024-09-14T06:32:12.203 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 148.33664555199999, 2024-09-14T06:32:12.203 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.003055224, 2024-09-14T06:32:12.203 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:32:12.203 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:32:12.203 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:32:12.204 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:32:12.204 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:12.204 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:32:12.204 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.864269+0000", 2024-09-14T06:32:12.204 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:12.204 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:12.204 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:12.204 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:32:12.204 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.864269+0000", 2024-09-14T06:32:12.204 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:12.204 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:12.204 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:12.205 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:32:12.205 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.864269+0000", 2024-09-14T06:32:12.205 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.41e-07 2024-09-14T06:32:12.205 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:12.205 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:12.205 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:32:12.205 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.865490+0000", 2024-09-14T06:32:12.205 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.001220552 2024-09-14T06:32:12.205 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:12.205 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:12.205 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:32:12.205 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.865506+0000", 2024-09-14T06:32:12.205 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.6263999999999999e-05 2024-09-14T06:32:12.205 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:12.206 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:12.206 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:32:12.206 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.865516+0000", 2024-09-14T06:32:12.206 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.0091999999999999e-05 2024-09-14T06:32:12.206 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:12.206 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:12.206 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:32:12.206 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.865583+0000", 2024-09-14T06:32:12.206 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.6676999999999999e-05 2024-09-14T06:32:12.206 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:12.206 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:12.206 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:32:12.206 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.865610+0000", 2024-09-14T06:32:12.206 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.6529999999999998e-05 2024-09-14T06:32:12.207 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:12.207 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:12.207 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-14T06:32:12.207 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.867281+0000", 2024-09-14T06:32:12.207 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.001671203 2024-09-14T06:32:12.207 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:12.207 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:12.207 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:32:12.207 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.867324+0000", 2024-09-14T06:32:12.207 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.3665000000000003e-05 2024-09-14T06:32:12.207 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:12.207 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:32:12.207 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:12.208 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:12.208 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:12.208 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4397.0:74241 3.0 e73/23 3:06e93fb9:::rbd_data.112d80ca42af.0000000000000000:head v 73'102, mlcod=71'100)", 2024-09-14T06:32:12.208 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:53.876730+0000", 2024-09-14T06:32:12.208 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 138.324184452, 2024-09-14T06:32:12.208 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0079470329999999992, 2024-09-14T06:32:12.208 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:32:12.208 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:32:12.208 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:32:12.208 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:32:12.208 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:12.208 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:32:12.208 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:53.876730+0000", 2024-09-14T06:32:12.208 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:12.209 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:12.209 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:12.209 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:32:12.209 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:53.876730+0000", 2024-09-14T06:32:12.209 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:12.209 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:12.209 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:12.209 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:32:12.209 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:53.876731+0000", 2024-09-14T06:32:12.209 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.0500000000000004e-07 2024-09-14T06:32:12.209 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:12.209 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:12.209 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:32:12.209 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:53.879616+0000", 2024-09-14T06:32:12.210 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0028856680000000001 2024-09-14T06:32:12.210 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:12.210 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:12.210 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:32:12.210 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:53.879643+0000", 2024-09-14T06:32:12.210 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.6120999999999999e-05 2024-09-14T06:32:12.210 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:12.210 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:12.210 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:32:12.210 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:53.879654+0000", 2024-09-14T06:32:12.210 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.1586999999999999e-05 2024-09-14T06:32:12.210 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:12.210 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:12.211 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:32:12.211 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:53.879756+0000", 2024-09-14T06:32:12.211 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00010223899999999999 2024-09-14T06:32:12.211 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:12.211 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:12.211 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:32:12.211 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:53.879783+0000", 2024-09-14T06:32:12.211 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.6913000000000001e-05 2024-09-14T06:32:12.211 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:12.211 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:12.212 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-14T06:32:12.212 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:53.884594+0000", 2024-09-14T06:32:12.212 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0048110729999999999 2024-09-14T06:32:12.212 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:12.212 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:12.212 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:32:12.212 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:53.884677+0000", 2024-09-14T06:32:12.212 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.2927000000000001e-05 2024-09-14T06:32:12.212 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:12.212 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:32:12.212 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:12.212 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:12.212 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:12.213 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4397.0:74243 3.9 e73/23 3:94977f81:::rbd_data.112d80ca42af.0000000000000002:head v 73'94, mlcod=71'92)", 2024-09-14T06:32:12.213 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:53.885156+0000", 2024-09-14T06:32:12.213 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 138.315758534, 2024-09-14T06:32:12.213 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0096009860000000006, 2024-09-14T06:32:12.213 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:32:12.213 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:32:12.213 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:32:12.213 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:32:12.213 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:12.213 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:32:12.213 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:53.885156+0000", 2024-09-14T06:32:12.213 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:12.213 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:12.213 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:12.214 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:32:12.214 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:53.885156+0000", 2024-09-14T06:32:12.214 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:12.214 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:12.214 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:12.214 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:32:12.214 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:53.885156+0000", 2024-09-14T06:32:12.214 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.23e-07 2024-09-14T06:32:12.214 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:12.214 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:12.214 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:32:12.214 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:53.889694+0000", 2024-09-14T06:32:12.214 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0045378709999999997 2024-09-14T06:32:12.215 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:12.215 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:12.215 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:32:12.215 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:53.889720+0000", 2024-09-14T06:32:12.215 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.5259e-05 2024-09-14T06:32:12.215 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:12.215 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:12.215 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:32:12.215 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:53.889731+0000", 2024-09-14T06:32:12.215 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.1739000000000001e-05 2024-09-14T06:32:12.215 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:12.215 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:12.216 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:32:12.216 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:53.889822+0000", 2024-09-14T06:32:12.216 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.1200999999999996e-05 2024-09-14T06:32:12.216 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:12.216 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:12.216 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:32:12.216 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:53.889850+0000", 2024-09-14T06:32:12.216 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.7084999999999999e-05 2024-09-14T06:32:12.216 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:12.216 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:12.217 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-14T06:32:12.217 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:53.894703+0000", 2024-09-14T06:32:12.217 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0048532690000000003 2024-09-14T06:32:12.217 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:12.217 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:12.217 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:32:12.217 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:53.894757+0000", 2024-09-14T06:32:12.217 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.4339000000000003e-05 2024-09-14T06:32:12.217 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:12.217 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:32:12.217 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:12.218 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:12.218 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:12.218 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4397.0:74249 3.0 e73/23 3:009b0225:::rbd_data.112d80ca42af.0000000000000009:head v 73'103, mlcod=71'101)", 2024-09-14T06:32:12.218 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:53.893219+0000", 2024-09-14T06:32:12.218 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 138.307696056, 2024-09-14T06:32:12.218 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0065381980000000003, 2024-09-14T06:32:12.218 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:32:12.218 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:32:12.218 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:32:12.218 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:32:12.218 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:12.218 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:32:12.218 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:53.893219+0000", 2024-09-14T06:32:12.219 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:12.219 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:12.219 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:12.219 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:32:12.219 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:53.893219+0000", 2024-09-14T06:32:12.219 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:12.219 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:12.219 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:12.219 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:32:12.219 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:53.893221+0000", 2024-09-14T06:32:12.219 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.4329999999999998e-06 2024-09-14T06:32:12.219 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:12.220 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:12.220 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:32:12.220 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:53.895968+0000", 2024-09-14T06:32:12.220 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.002746746 2024-09-14T06:32:12.220 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:12.220 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:12.220 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:32:12.220 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:53.895988+0000", 2024-09-14T06:32:12.220 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.0227000000000001e-05 2024-09-14T06:32:12.220 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:12.220 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:12.220 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:32:12.220 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:53.896000+0000", 2024-09-14T06:32:12.220 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.2359e-05 2024-09-14T06:32:12.221 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:12.221 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:12.221 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:32:12.221 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:53.896084+0000", 2024-09-14T06:32:12.221 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.3659999999999995e-05 2024-09-14T06:32:12.221 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:12.221 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:12.221 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:32:12.221 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:53.896109+0000", 2024-09-14T06:32:12.221 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.5259999999999999e-05 2024-09-14T06:32:12.221 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:12.222 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:12.222 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-14T06:32:12.222 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:53.899704+0000", 2024-09-14T06:32:12.222 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0035942209999999999 2024-09-14T06:32:12.222 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:12.222 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:12.222 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:32:12.222 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:53.899757+0000", 2024-09-14T06:32:12.222 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.3291999999999997e-05 2024-09-14T06:32:12.222 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:12.222 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:32:12.222 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:12.222 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:12.222 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:12.222 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4397.0:78263 3.0 e77/23 3:00899b22:::rbd_data.112ddfbec471.0000000000000016:head v 77'104, mlcod=73'102)", 2024-09-14T06:32:12.223 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:57.634578+0000", 2024-09-14T06:32:12.223 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 134.56633700399999, 2024-09-14T06:32:12.223 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.002372454, 2024-09-14T06:32:12.223 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:32:12.223 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:32:12.223 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:32:12.223 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:32:12.223 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:12.223 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:32:12.223 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:57.634578+0000", 2024-09-14T06:32:12.223 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:12.223 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:12.223 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:12.223 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:32:12.224 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:57.634578+0000", 2024-09-14T06:32:12.224 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:12.224 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:12.224 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:12.224 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:32:12.224 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:57.634578+0000", 2024-09-14T06:32:12.224 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.5699999999999998e-07 2024-09-14T06:32:12.224 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:12.224 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:12.224 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:32:12.224 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:57.635347+0000", 2024-09-14T06:32:12.224 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00076848300000000003 2024-09-14T06:32:12.225 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:12.225 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:12.225 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:32:12.225 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:57.635361+0000", 2024-09-14T06:32:12.225 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.4238e-05 2024-09-14T06:32:12.225 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:12.225 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:12.225 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:32:12.225 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:57.635371+0000", 2024-09-14T06:32:12.225 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.0087e-05 2024-09-14T06:32:12.225 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:12.225 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:12.225 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:32:12.225 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:57.635439+0000", 2024-09-14T06:32:12.225 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.7953000000000005e-05 2024-09-14T06:32:12.226 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:12.226 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:12.226 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:32:12.226 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:57.635464+0000", 2024-09-14T06:32:12.226 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.5062000000000002e-05 2024-09-14T06:32:12.226 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:12.226 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:12.226 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-14T06:32:12.226 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:57.636914+0000", 2024-09-14T06:32:12.226 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.001449625 2024-09-14T06:32:12.226 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:12.226 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:12.226 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:32:12.226 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:57.636950+0000", 2024-09-14T06:32:12.226 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.6548999999999999e-05 2024-09-14T06:32:12.226 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:12.227 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:32:12.227 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:12.227 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:12.227 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:12.227 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4397.0:78604 3.9 e77/23 3:94c62a0e:::rbd_data.112ddfbec471.000000000000002c:head v 77'96, mlcod=73'94)", 2024-09-14T06:32:12.227 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:57.677568+0000", 2024-09-14T06:32:12.227 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 134.523346616, 2024-09-14T06:32:12.227 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.002564846, 2024-09-14T06:32:12.227 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:32:12.227 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:32:12.227 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:32:12.227 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:32:12.227 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:12.227 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:32:12.227 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:57.677568+0000", 2024-09-14T06:32:12.227 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:12.228 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:12.228 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:12.228 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:32:12.228 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:57.677568+0000", 2024-09-14T06:32:12.228 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:12.228 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:12.228 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:12.228 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:32:12.228 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:57.677568+0000", 2024-09-14T06:32:12.228 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.8900000000000001e-07 2024-09-14T06:32:12.228 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:12.228 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:12.228 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:32:12.229 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:57.678488+0000", 2024-09-14T06:32:12.229 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00091963899999999998 2024-09-14T06:32:12.229 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:12.229 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:12.229 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:32:12.229 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:57.678500+0000", 2024-09-14T06:32:12.229 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.1902000000000001e-05 2024-09-14T06:32:12.229 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:12.229 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:12.229 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:32:12.229 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:57.678507+0000", 2024-09-14T06:32:12.229 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.7739999999999999e-06 2024-09-14T06:32:12.229 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:12.230 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:12.230 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:32:12.230 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:57.678586+0000", 2024-09-14T06:32:12.230 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.9103000000000005e-05 2024-09-14T06:32:12.230 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:12.230 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:12.230 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:32:12.230 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:57.678614+0000", 2024-09-14T06:32:12.230 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.7912e-05 2024-09-14T06:32:12.230 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:12.230 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:12.231 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-14T06:32:12.231 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:57.680075+0000", 2024-09-14T06:32:12.231 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0014613370000000001 2024-09-14T06:32:12.231 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:12.231 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:12.231 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:32:12.231 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:57.680133+0000", 2024-09-14T06:32:12.231 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.7890000000000003e-05 2024-09-14T06:32:12.231 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:12.231 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:32:12.231 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:12.231 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:12.232 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:12.232 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4397.0:149885 3.9 e122/23 3:951acef3:::rbd_data.112d1bb7b269.0000000000000005:head v 122'176, mlcod=122'172)", 2024-09-14T06:32:12.232 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:30:44.739092+0000", 2024-09-14T06:32:12.232 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 87.461822306000002, 2024-09-14T06:32:12.232 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0028236659999999998, 2024-09-14T06:32:12.232 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:32:12.232 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:32:12.232 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:32:12.232 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:32:12.232 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:12.232 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:32:12.232 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.739092+0000", 2024-09-14T06:32:12.232 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:12.232 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:12.233 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:12.233 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:32:12.233 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.739092+0000", 2024-09-14T06:32:12.233 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:12.233 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:12.233 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:12.233 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:32:12.233 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.739093+0000", 2024-09-14T06:32:12.233 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.0109999999999999e-06 2024-09-14T06:32:12.233 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:12.233 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:12.233 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:32:12.234 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.740189+0000", 2024-09-14T06:32:12.234 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00109527 2024-09-14T06:32:12.234 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:12.234 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:12.234 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:32:12.234 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.740199+0000", 2024-09-14T06:32:12.234 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.0458999999999999e-05 2024-09-14T06:32:12.234 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:12.234 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:12.234 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:32:12.234 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.740207+0000", 2024-09-14T06:32:12.234 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.4499999999999998e-06 2024-09-14T06:32:12.234 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:12.234 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:12.234 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:32:12.235 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.740295+0000", 2024-09-14T06:32:12.235 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.8740999999999996e-05 2024-09-14T06:32:12.235 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:12.235 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:12.235 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:32:12.235 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.740317+0000", 2024-09-14T06:32:12.235 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.1523e-05 2024-09-14T06:32:12.235 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:12.235 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:12.235 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-14T06:32:12.235 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.741865+0000", 2024-09-14T06:32:12.235 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.001548315 2024-09-14T06:32:12.235 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:12.235 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:12.235 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:32:12.236 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.741916+0000", 2024-09-14T06:32:12.236 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.0896999999999997e-05 2024-09-14T06:32:12.236 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:12.236 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:32:12.236 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:12.236 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:12.236 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:12.236 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4397.0:149933 3.4 e122/23 3:234f5229:::rbd_data.112d1bb7b269.0000000000000009:head v 122'221, mlcod=120'218)", 2024-09-14T06:32:12.236 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:30:44.743849+0000", 2024-09-14T06:32:12.236 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 87.457065889000006, 2024-09-14T06:32:12.236 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0031479519999999999, 2024-09-14T06:32:12.236 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:32:12.237 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:32:12.237 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:32:12.237 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:32:12.237 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:12.237 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:32:12.237 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.743849+0000", 2024-09-14T06:32:12.237 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:12.237 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:12.237 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:12.237 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:32:12.237 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.743849+0000", 2024-09-14T06:32:12.237 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:12.237 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:12.237 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:12.238 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:32:12.238 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.743850+0000", 2024-09-14T06:32:12.238 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.279e-06 2024-09-14T06:32:12.238 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:12.238 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:12.238 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:32:12.238 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.745151+0000", 2024-09-14T06:32:12.238 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.001300495 2024-09-14T06:32:12.238 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:12.238 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:12.238 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:32:12.238 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.745167+0000", 2024-09-14T06:32:12.238 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.6478e-05 2024-09-14T06:32:12.238 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:12.238 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:12.239 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:32:12.239 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.745175+0000", 2024-09-14T06:32:12.239 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.2679999999999994e-06 2024-09-14T06:32:12.239 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:12.239 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:12.239 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:32:12.239 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.745226+0000", 2024-09-14T06:32:12.239 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.0192999999999999e-05 2024-09-14T06:32:12.239 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:12.239 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:12.239 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:32:12.239 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.745247+0000", 2024-09-14T06:32:12.239 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.1767e-05 2024-09-14T06:32:12.239 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:12.239 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:12.240 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-14T06:32:12.240 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.746952+0000", 2024-09-14T06:32:12.240 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0017042769999999999 2024-09-14T06:32:12.240 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:12.240 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:12.240 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:32:12.240 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.746997+0000", 2024-09-14T06:32:12.240 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.5194999999999997e-05 2024-09-14T06:32:12.240 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:12.240 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:32:12.240 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:12.240 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:12.240 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:12.240 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4397.0:150494 3.9 e122/23 3:955152a4:::rbd_object_map.112d1bb7b269.0000000000000038:head v 122'194, mlcod=122'190)", 2024-09-14T06:32:12.240 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:30:44.781585+0000", 2024-09-14T06:32:12.241 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 87.419329997000006, 2024-09-14T06:32:12.241 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0026957499999999998, 2024-09-14T06:32:12.241 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:32:12.241 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:32:12.241 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:32:12.241 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:32:12.241 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:12.241 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:32:12.241 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.781585+0000", 2024-09-14T06:32:12.241 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:12.241 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:12.241 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:12.241 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:32:12.242 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.781585+0000", 2024-09-14T06:32:12.242 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:12.242 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:12.242 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:12.242 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:32:12.242 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.781585+0000", 2024-09-14T06:32:12.242 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.5000000000000002e-07 2024-09-14T06:32:12.242 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:12.242 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:12.242 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:32:12.242 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.781594+0000", 2024-09-14T06:32:12.242 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.4640000000000006e-06 2024-09-14T06:32:12.243 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:12.243 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:12.243 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:32:12.243 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.781604+0000", 2024-09-14T06:32:12.243 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.0071999999999999e-05 2024-09-14T06:32:12.243 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:12.243 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:12.243 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:32:12.243 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.781610+0000", 2024-09-14T06:32:12.243 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.3609999999999997e-06 2024-09-14T06:32:12.243 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:12.243 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:12.243 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:32:12.243 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.781676+0000", 2024-09-14T06:32:12.243 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.5963000000000005e-05 2024-09-14T06:32:12.244 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:12.244 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:12.244 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:32:12.244 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.781696+0000", 2024-09-14T06:32:12.244 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.9278000000000001e-05 2024-09-14T06:32:12.244 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:12.244 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:12.244 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-14T06:32:12.244 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.784231+0000", 2024-09-14T06:32:12.244 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.002535093 2024-09-14T06:32:12.244 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:12.245 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:12.245 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:32:12.245 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.784280+0000", 2024-09-14T06:32:12.245 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.9869e-05 2024-09-14T06:32:12.245 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:12.245 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:32:12.245 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:12.245 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:12.245 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:12.245 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4397.0:150898 3.9 e122/23 3:97c98f90:::rbd_data.112d1bb7b269.0000000000000025:head v 122'206, mlcod=122'203)", 2024-09-14T06:32:12.245 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:30:44.807232+0000", 2024-09-14T06:32:12.245 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 87.393683080000002, 2024-09-14T06:32:12.245 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0022297889999999998, 2024-09-14T06:32:12.246 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:32:12.246 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:32:12.246 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:32:12.246 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:32:12.246 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:12.246 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:32:12.246 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.807232+0000", 2024-09-14T06:32:12.246 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:12.246 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:12.246 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:12.246 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:32:12.246 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.807232+0000", 2024-09-14T06:32:12.246 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:12.246 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:12.246 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:12.247 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:32:12.247 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.807233+0000", 2024-09-14T06:32:12.247 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.5710000000000001e-06 2024-09-14T06:32:12.247 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:12.247 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:12.247 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:32:12.247 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.807944+0000", 2024-09-14T06:32:12.247 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00071056299999999999 2024-09-14T06:32:12.247 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:12.247 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:12.247 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:32:12.247 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.807961+0000", 2024-09-14T06:32:12.247 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.7689e-05 2024-09-14T06:32:12.247 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:12.247 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:12.247 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:32:12.248 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.807971+0000", 2024-09-14T06:32:12.248 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.2e-06 2024-09-14T06:32:12.248 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:12.248 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:12.248 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:32:12.248 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.808050+0000", 2024-09-14T06:32:12.248 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.9308999999999996e-05 2024-09-14T06:32:12.248 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:12.248 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:12.248 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:32:12.248 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.808074+0000", 2024-09-14T06:32:12.248 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.4147000000000001e-05 2024-09-14T06:32:12.248 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:12.248 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:12.248 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-14T06:32:12.249 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.809415+0000", 2024-09-14T06:32:12.249 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.001341257 2024-09-14T06:32:12.249 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:12.249 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:12.249 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:32:12.249 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.809461+0000", 2024-09-14T06:32:12.249 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.6053000000000001e-05 2024-09-14T06:32:12.249 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:12.249 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:32:12.249 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:12.249 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:12.249 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:12.249 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4397.0:202551 3.9 e166/23 3:965c09b6:::rbd_data.112d3bd84cb0.000000000000000a:head v 166'416, mlcod=164'414)", 2024-09-14T06:32:12.249 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:31:30.657011+0000", 2024-09-14T06:32:12.250 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 41.543904046999998, 2024-09-14T06:32:12.250 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.002509262, 2024-09-14T06:32:12.250 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:32:12.250 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:32:12.250 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:32:12.250 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:32:12.250 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:12.250 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:32:12.250 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:30.657011+0000", 2024-09-14T06:32:12.250 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:12.250 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:12.250 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:12.251 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:32:12.251 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:30.657011+0000", 2024-09-14T06:32:12.251 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:12.251 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:12.251 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:12.251 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:32:12.251 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:30.657011+0000", 2024-09-14T06:32:12.251 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.08e-07 2024-09-14T06:32:12.251 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:12.251 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:12.251 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:32:12.251 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:30.658467+0000", 2024-09-14T06:32:12.251 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.001455708 2024-09-14T06:32:12.252 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:12.252 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:12.252 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:32:12.252 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:30.658484+0000", 2024-09-14T06:32:12.252 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.6844e-05 2024-09-14T06:32:12.252 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:12.252 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:12.252 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:32:12.252 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:30.658493+0000", 2024-09-14T06:32:12.252 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.2790000000000004e-06 2024-09-14T06:32:12.252 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:12.252 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:12.253 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:32:12.253 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:30.658559+0000", 2024-09-14T06:32:12.253 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.6459999999999997e-05 2024-09-14T06:32:12.253 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:12.253 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:12.253 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:32:12.253 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:30.658574+0000", 2024-09-14T06:32:12.253 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.5048000000000001e-05 2024-09-14T06:32:12.253 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:12.253 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:12.254 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-14T06:32:12.254 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:30.659473+0000", 2024-09-14T06:32:12.254 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00089842800000000005 2024-09-14T06:32:12.254 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:12.254 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:12.254 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:32:12.254 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:30.659520+0000", 2024-09-14T06:32:12.254 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.7086999999999998e-05 2024-09-14T06:32:12.254 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:12.254 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:32:12.254 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:12.254 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:12.254 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:12.254 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4397.0:202665 3.0 e166/23 3:00eeadfb:::rbd_data.112d3bd84cb0.000000000000000e:head v 166'920, mlcod=163'918)", 2024-09-14T06:32:12.255 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:31:30.662434+0000", 2024-09-14T06:32:12.255 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 41.538480258, 2024-09-14T06:32:12.255 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.002522121, 2024-09-14T06:32:12.255 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:32:12.255 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:32:12.255 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:32:12.255 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:32:12.255 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:12.255 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:32:12.255 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:30.662434+0000", 2024-09-14T06:32:12.255 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:12.255 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:12.255 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:12.255 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:32:12.256 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:30.662434+0000", 2024-09-14T06:32:12.256 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:12.256 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:12.256 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:12.256 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:32:12.256 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:30.662436+0000", 2024-09-14T06:32:12.256 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.257e-06 2024-09-14T06:32:12.256 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:12.256 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:12.256 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:32:12.256 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:30.662527+0000", 2024-09-14T06:32:12.256 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.1644000000000006e-05 2024-09-14T06:32:12.256 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:12.256 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:12.256 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:32:12.257 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:30.662546+0000", 2024-09-14T06:32:12.257 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.8998e-05 2024-09-14T06:32:12.257 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:12.257 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:12.257 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:32:12.257 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:30.662556+0000", 2024-09-14T06:32:12.257 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.9809999999999998e-06 2024-09-14T06:32:12.257 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:12.257 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:12.257 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:32:12.257 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:30.662621+0000", 2024-09-14T06:32:12.257 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.4597000000000005e-05 2024-09-14T06:32:12.257 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:12.258 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:12.258 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:32:12.258 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:30.662643+0000", 2024-09-14T06:32:12.258 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.1608e-05 2024-09-14T06:32:12.258 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:12.258 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:12.258 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-14T06:32:12.258 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:30.664909+0000", 2024-09-14T06:32:12.258 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0022668689999999999 2024-09-14T06:32:12.258 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:12.258 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:12.258 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:32:12.259 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:30.664957+0000", 2024-09-14T06:32:12.259 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.7166999999999998e-05 2024-09-14T06:32:12.259 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:12.259 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:32:12.259 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:12.259 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:12.259 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:12.259 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4397.0:204589 3.4 e166/23 3:234552f0:::rbd_data.112d3bd84cb0.000000000000003f:head v 166'499, mlcod=165'497)", 2024-09-14T06:32:12.259 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:31:30.772894+0000", 2024-09-14T06:32:12.259 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 41.428020402000001, 2024-09-14T06:32:12.259 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0034564650000000001, 2024-09-14T06:32:12.259 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:32:12.259 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:32:12.259 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:32:12.260 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:32:12.260 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:12.260 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:32:12.260 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:30.772894+0000", 2024-09-14T06:32:12.260 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:12.260 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:12.260 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:12.260 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:32:12.260 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:30.772894+0000", 2024-09-14T06:32:12.260 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:12.260 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:12.261 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:12.261 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:32:12.261 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:30.772896+0000", 2024-09-14T06:32:12.261 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.5379999999999999e-06 2024-09-14T06:32:12.261 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:12.261 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:12.261 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:32:12.261 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:30.773042+0000", 2024-09-14T06:32:12.261 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000146107 2024-09-14T06:32:12.261 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:12.261 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:12.261 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:32:12.261 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:30.773061+0000", 2024-09-14T06:32:12.261 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.8862999999999999e-05 2024-09-14T06:32:12.262 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:12.262 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:12.262 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:32:12.262 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:30.773071+0000", 2024-09-14T06:32:12.262 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.0227000000000001e-05 2024-09-14T06:32:12.262 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:12.262 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:12.262 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:32:12.262 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:30.773149+0000", 2024-09-14T06:32:12.262 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.8071999999999994e-05 2024-09-14T06:32:12.262 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:12.262 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:12.263 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:32:12.263 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:30.773171+0000", 2024-09-14T06:32:12.263 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.1699999999999999e-05 2024-09-14T06:32:12.263 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:12.263 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:12.263 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-14T06:32:12.263 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:30.776285+0000", 2024-09-14T06:32:12.263 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0031142050000000001 2024-09-14T06:32:12.263 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:12.263 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:12.263 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:32:12.263 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:30.776351+0000", 2024-09-14T06:32:12.263 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.5753000000000006e-05 2024-09-14T06:32:12.264 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:12.264 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:32:12.264 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:12.264 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:12.264 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:12.264 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4397.0:204598 3.4 e166/23 3:245d069e:::rbd_data.112d3bd84cb0.0000000000000041:head v 166'500, mlcod=165'497)", 2024-09-14T06:32:12.264 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:31:30.775247+0000", 2024-09-14T06:32:12.264 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 41.425667279000002, 2024-09-14T06:32:12.264 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00318235, 2024-09-14T06:32:12.264 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:32:12.264 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:32:12.264 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:32:12.264 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:32:12.265 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:12.265 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:32:12.265 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:30.775247+0000", 2024-09-14T06:32:12.265 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:12.265 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:12.265 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:12.265 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:32:12.265 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:30.775247+0000", 2024-09-14T06:32:12.265 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:12.265 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:12.265 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:12.265 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:32:12.265 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:30.775249+0000", 2024-09-14T06:32:12.265 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.265e-06 2024-09-14T06:32:12.266 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:12.266 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:12.266 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:32:12.266 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:30.775340+0000", 2024-09-14T06:32:12.266 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.1131999999999994e-05 2024-09-14T06:32:12.266 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:12.266 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:12.266 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:32:12.266 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:30.775352+0000", 2024-09-14T06:32:12.266 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.2294e-05 2024-09-14T06:32:12.266 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:12.266 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:12.266 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:32:12.266 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:30.775361+0000", 2024-09-14T06:32:12.266 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.6950000000000006e-06 2024-09-14T06:32:12.266 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:12.267 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:12.267 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:32:12.267 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:30.775439+0000", 2024-09-14T06:32:12.267 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.7910000000000005e-05 2024-09-14T06:32:12.267 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:12.267 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:12.267 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:32:12.267 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:30.775460+0000", 2024-09-14T06:32:12.267 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.0900999999999999e-05 2024-09-14T06:32:12.267 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:12.267 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:12.267 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-14T06:32:12.267 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:30.778385+0000", 2024-09-14T06:32:12.267 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.002925539 2024-09-14T06:32:12.267 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:12.267 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:12.268 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:32:12.268 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:30.778430+0000", 2024-09-14T06:32:12.268 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.4613999999999997e-05 2024-09-14T06:32:12.268 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:12.268 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:32:12.268 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:12.268 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:12.268 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:12.268 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4397.0:204975 3.4 e166/23 3:20972d18:::rbd_data.112d3bd84cb0.0000000000000046:head v 166'501, mlcod=166'499)", 2024-09-14T06:32:12.268 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:31:30.787253+0000", 2024-09-14T06:32:12.268 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 41.413661249999997, 2024-09-14T06:32:12.268 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.002248318, 2024-09-14T06:32:12.268 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:32:12.268 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:32:12.269 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:32:12.269 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:32:12.269 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:12.269 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:32:12.269 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:30.787253+0000", 2024-09-14T06:32:12.269 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:12.269 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:12.269 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:12.269 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:32:12.269 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:30.787253+0000", 2024-09-14T06:32:12.269 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:12.269 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:12.269 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:12.269 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:32:12.270 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:30.787255+0000", 2024-09-14T06:32:12.270 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.271e-06 2024-09-14T06:32:12.270 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:12.270 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:12.270 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:32:12.270 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:30.787442+0000", 2024-09-14T06:32:12.270 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00018744000000000001 2024-09-14T06:32:12.270 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:12.270 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:12.270 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:32:12.270 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:30.787461+0000", 2024-09-14T06:32:12.270 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.9054000000000001e-05 2024-09-14T06:32:12.270 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:12.271 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:12.271 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:32:12.271 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:30.787470+0000", 2024-09-14T06:32:12.271 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.0929999999999992e-06 2024-09-14T06:32:12.271 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:12.271 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:12.271 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:32:12.271 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:30.788004+0000", 2024-09-14T06:32:12.271 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00053359399999999997 2024-09-14T06:32:12.271 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:12.271 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:12.271 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:32:12.271 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:30.788034+0000", 2024-09-14T06:32:12.271 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.9635e-05 2024-09-14T06:32:12.272 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:12.272 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:12.272 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-14T06:32:12.272 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:30.789457+0000", 2024-09-14T06:32:12.272 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.001423487 2024-09-14T06:32:12.272 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:12.272 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:12.272 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:32:12.272 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:30.789502+0000", 2024-09-14T06:32:12.272 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.4743999999999997e-05 2024-09-14T06:32:12.272 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:12.272 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:32:12.272 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:12.272 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:12.272 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:12.273 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4397.0:212188 3.4 e166/23 3:23871bc7:::rbd_data.112d3bd84cb0.00000000000000fa:head v 166'508, mlcod=166'506)", 2024-09-14T06:32:12.273 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:31:31.391415+0000", 2024-09-14T06:32:12.273 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 40.809499668000001, 2024-09-14T06:32:12.273 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0022039049999999999, 2024-09-14T06:32:12.273 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:32:12.273 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:32:12.273 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:32:12.273 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:32:12.273 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:12.273 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:32:12.273 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:31.391415+0000", 2024-09-14T06:32:12.273 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:12.273 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:12.273 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:12.273 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:32:12.274 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:31.391415+0000", 2024-09-14T06:32:12.274 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:12.274 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:12.274 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:12.274 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:32:12.274 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:31.391416+0000", 2024-09-14T06:32:12.274 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.44e-06 2024-09-14T06:32:12.274 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:12.274 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:12.274 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:32:12.274 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:31.391554+0000", 2024-09-14T06:32:12.274 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000137119 2024-09-14T06:32:12.274 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:12.274 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:12.274 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:32:12.275 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:31.391570+0000", 2024-09-14T06:32:12.275 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.6827e-05 2024-09-14T06:32:12.275 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:12.275 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:12.275 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:32:12.275 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:31.391580+0000", 2024-09-14T06:32:12.275 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.3139999999999993e-06 2024-09-14T06:32:12.275 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:12.275 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:12.275 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:32:12.275 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:31.391664+0000", 2024-09-14T06:32:12.276 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.3757999999999993e-05 2024-09-14T06:32:12.276 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:12.276 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:12.276 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:32:12.276 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:31.391692+0000", 2024-09-14T06:32:12.276 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.8898e-05 2024-09-14T06:32:12.276 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:12.276 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:12.276 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-14T06:32:12.276 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:31.393584+0000", 2024-09-14T06:32:12.276 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.001891752 2024-09-14T06:32:12.276 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:12.276 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:12.276 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:32:12.277 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:31.393619+0000", 2024-09-14T06:32:12.277 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.4796999999999999e-05 2024-09-14T06:32:12.277 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:12.277 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:32:12.277 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:12.277 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:12.277 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:12.277 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "MOSDPGPush(3.5 169/168 [PushOp(3:a7c12b08:::rbd_data.112ddfbec471.000000000000000e:head, version: 77'100, data_included: [0~1048576], data_size: 1048576, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(3:a7c12b08:::rbd_data.112ddfbec471.000000000000000e:head@77'100, size: 1048576, copy_subset: [(0, 1048576)], clone_subset: {}, snapset: 0=[]:{}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 1048576, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2024-09-14T06:32:12.277 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:31:34.540707+0000", 2024-09-14T06:32:12.277 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 37.660207903, 2024-09-14T06:32:12.277 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0023637369999999999, 2024-09-14T06:32:12.278 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:32:12.278 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:32:12.278 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:32:12.278 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:32:12.278 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:12.278 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:32:12.278 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:34.540707+0000", 2024-09-14T06:32:12.278 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:12.278 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:12.278 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:12.279 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:32:12.279 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:34.540707+0000", 2024-09-14T06:32:12.279 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:12.279 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:12.279 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:12.279 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:32:12.279 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:34.540708+0000", 2024-09-14T06:32:12.279 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.7199999999999998e-07 2024-09-14T06:32:12.279 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:12.279 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:12.279 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:32:12.279 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:34.542150+0000", 2024-09-14T06:32:12.279 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0014423229999999999 2024-09-14T06:32:12.280 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:12.280 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:12.280 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:32:12.280 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:34.542169+0000", 2024-09-14T06:32:12.280 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.8756000000000001e-05 2024-09-14T06:32:12.280 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:12.280 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:12.280 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:32:12.280 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:34.542180+0000", 2024-09-14T06:32:12.280 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.1051e-05 2024-09-14T06:32:12.280 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:12.280 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:12.280 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:32:12.281 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:34.542275+0000", 2024-09-14T06:32:12.281 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.5122000000000002e-05 2024-09-14T06:32:12.281 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:12.281 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:12.281 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:32:12.281 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:34.542284+0000", 2024-09-14T06:32:12.281 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.9190000000000005e-06 2024-09-14T06:32:12.281 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:12.281 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:12.281 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:32:12.281 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:34.543071+0000", 2024-09-14T06:32:12.281 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00078679399999999995 2024-09-14T06:32:12.281 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:12.282 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:32:12.282 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:12.282 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:12.282 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:12.282 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "MOSDPGPush(3.5 169/168 [PushOp(3:a5c8b3f2:::rbd_data.112ddfbec471.0000000000000018:head, version: 77'101, data_included: [0~1048576], data_size: 1048576, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(3:a5c8b3f2:::rbd_data.112ddfbec471.0000000000000018:head@77'101, size: 1048576, copy_subset: [(0, 1048576)], clone_subset: {}, snapset: 0=[]:{}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 1048576, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2024-09-14T06:32:12.282 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:31:34.545533+0000", 2024-09-14T06:32:12.282 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 37.655382015999997, 2024-09-14T06:32:12.282 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0021867010000000001, 2024-09-14T06:32:12.282 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:32:12.282 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:32:12.282 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:32:12.282 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:32:12.282 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:12.282 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:32:12.283 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:34.545533+0000", 2024-09-14T06:32:12.283 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:12.283 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:12.283 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:12.283 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:32:12.283 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:34.545533+0000", 2024-09-14T06:32:12.283 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:12.283 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:12.283 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:12.283 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:32:12.283 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:34.545533+0000", 2024-09-14T06:32:12.283 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.8100000000000002e-07 2024-09-14T06:32:12.283 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:12.283 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:12.283 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:32:12.284 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:34.546784+0000", 2024-09-14T06:32:12.284 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.001250747 2024-09-14T06:32:12.284 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:12.284 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:12.284 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:32:12.284 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:34.546803+0000", 2024-09-14T06:32:12.284 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.8346e-05 2024-09-14T06:32:12.284 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:12.284 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:12.285 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:32:12.285 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:34.546823+0000", 2024-09-14T06:32:12.285 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.0699999999999998e-05 2024-09-14T06:32:12.285 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:12.285 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:12.285 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:32:12.285 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:34.546890+0000", 2024-09-14T06:32:12.285 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.6581000000000005e-05 2024-09-14T06:32:12.285 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:12.285 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:12.285 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:32:12.285 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:34.546899+0000", 2024-09-14T06:32:12.285 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.3889999999999994e-06 2024-09-14T06:32:12.286 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:12.286 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:12.286 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:32:12.286 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:34.547719+0000", 2024-09-14T06:32:12.286 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00082015699999999996 2024-09-14T06:32:12.286 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:12.286 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:32:12.286 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:12.286 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:12.286 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:12.286 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "MOSDPGPush(3.5 169/168 [PushOp(3:a7a54476:::rbd_data.112d1bb7b269.0000000000000038:head, version: 122'581, data_included: [0~524288], data_size: 524288, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(3:a7a54476:::rbd_data.112d1bb7b269.0000000000000038:head@122'581, size: 524288, copy_subset: [(0, 524288)], clone_subset: {}, snapset: 0=[]:{}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 524288, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2024-09-14T06:32:12.286 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:31:34.667757+0000", 2024-09-14T06:32:12.286 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 37.533157856999999, 2024-09-14T06:32:12.286 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.002750382, 2024-09-14T06:32:12.287 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:32:12.287 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:32:12.287 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:32:12.287 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:32:12.287 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:12.287 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:32:12.287 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:34.667757+0000", 2024-09-14T06:32:12.287 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:12.287 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:12.287 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:12.287 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:32:12.287 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:34.667757+0000", 2024-09-14T06:32:12.287 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:12.287 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:12.288 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:12.288 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:32:12.288 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:34.667758+0000", 2024-09-14T06:32:12.288 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.3599999999999999e-06 2024-09-14T06:32:12.288 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:12.288 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:12.288 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:32:12.288 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:34.669375+0000", 2024-09-14T06:32:12.288 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.001616644 2024-09-14T06:32:12.288 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:12.288 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:12.288 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:32:12.288 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:34.669404+0000", 2024-09-14T06:32:12.289 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.9300999999999999e-05 2024-09-14T06:32:12.289 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:12.289 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:12.289 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:32:12.289 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:34.669428+0000", 2024-09-14T06:32:12.289 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.3788e-05 2024-09-14T06:32:12.289 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:12.289 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:12.289 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:32:12.289 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:34.669543+0000", 2024-09-14T06:32:12.289 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000115462 2024-09-14T06:32:12.290 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:12.290 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:12.290 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:32:12.290 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:34.669559+0000", 2024-09-14T06:32:12.290 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.5167999999999999e-05 2024-09-14T06:32:12.290 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:12.290 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:12.290 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:32:12.290 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:34.670507+0000", 2024-09-14T06:32:12.290 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00094865900000000003 2024-09-14T06:32:12.290 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:12.290 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:32:12.290 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:12.291 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:12.291 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:12.291 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "MOSDPGPush(3.5 170/168 [PushOp(3:a162138c:::rbd_data.112d1bb7b269.000000000000003c:head, version: 122'582, data_included: [0~524288], data_size: 524288, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(3:a162138c:::rbd_data.112d1bb7b269.000000000000003c:head@122'582, size: 524288, copy_subset: [(0, 524288)], clone_subset: {}, snapset: 0=[]:{}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 524288, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2024-09-14T06:32:12.291 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:31:34.670110+0000", 2024-09-14T06:32:12.291 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 37.530804908, 2024-09-14T06:32:12.291 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0030066229999999999, 2024-09-14T06:32:12.291 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:32:12.291 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:32:12.291 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:32:12.292 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:32:12.292 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:12.292 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:32:12.292 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:34.670110+0000", 2024-09-14T06:32:12.292 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:12.292 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:12.292 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:12.292 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:32:12.292 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:34.670110+0000", 2024-09-14T06:32:12.292 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:12.293 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:12.293 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:12.293 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:32:12.293 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:34.670111+0000", 2024-09-14T06:32:12.293 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.6499999999999998e-07 2024-09-14T06:32:12.293 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:12.293 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:12.293 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:32:12.293 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:34.671507+0000", 2024-09-14T06:32:12.293 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.001396284 2024-09-14T06:32:12.293 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:12.293 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:12.294 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:32:12.294 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:34.671530+0000", 2024-09-14T06:32:12.294 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.3023e-05 2024-09-14T06:32:12.294 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:12.294 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:12.294 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:32:12.294 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:34.671548+0000", 2024-09-14T06:32:12.294 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.8116000000000001e-05 2024-09-14T06:32:12.294 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:12.294 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:12.294 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:32:12.294 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:34.671775+0000", 2024-09-14T06:32:12.295 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00022699600000000001 2024-09-14T06:32:12.295 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:12.295 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:12.295 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:32:12.295 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:34.671790+0000", 2024-09-14T06:32:12.295 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.539e-05 2024-09-14T06:32:12.295 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:12.295 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:12.295 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:32:12.295 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:34.673116+0000", 2024-09-14T06:32:12.295 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0013259490000000001 2024-09-14T06:32:12.295 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:12.295 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:32:12.296 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:12.296 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:12.296 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:32:12.296 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-14T06:32:12.297 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-14T06:32:12.298 INFO:tasks.ceph.osd.6.smithi138.stderr:2024-09-14T06:32:12.296+0000 7f8cd8849640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33620) UID: 0 2024-09-14T06:32:12.398 INFO:tasks.ceph.osd.4.smithi138.stderr:2024-09-14T06:32:12.397+0000 7f6c653c8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33615) UID: 0 2024-09-14T06:32:12.441 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-14T06:32:12.441 INFO:teuthology.orchestra.run.smithi094.stdout: "ops": [], 2024-09-14T06:32:12.441 INFO:teuthology.orchestra.run.smithi094.stdout: "num_ops": 0 2024-09-14T06:32:12.441 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-14T06:32:12.449 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-14T06:32:12.499 INFO:tasks.ceph.osd.6.smithi138.stderr:2024-09-14T06:32:12.498+0000 7f8cd8849640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33620) UID: 0 2024-09-14T06:32:12.551 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-14T06:32:12.551 INFO:teuthology.orchestra.run.smithi094.stdout: "ops": [], 2024-09-14T06:32:12.551 INFO:teuthology.orchestra.run.smithi094.stdout: "complaint_time": 30, 2024-09-14T06:32:12.551 INFO:teuthology.orchestra.run.smithi094.stdout: "num_blocked_ops": 0 2024-09-14T06:32:12.552 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-14T06:32:12.560 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-14T06:32:12.600 INFO:tasks.ceph.osd.4.smithi138.stderr:2024-09-14T06:32:12.599+0000 7f6c653c8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33615) UID: 0 2024-09-14T06:32:12.655 INFO:teuthology.orchestra.run.smithi094.stderr:reweighted osd.1 to 1 (10000) 2024-09-14T06:32:12.659 INFO:teuthology.orchestra.run.smithi094.stderr:noscrub is set 2024-09-14T06:32:12.670 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-14T06:32:12.670 INFO:teuthology.orchestra.run.smithi094.stdout: "size": 20, 2024-09-14T06:32:12.670 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 600, 2024-09-14T06:32:12.670 INFO:teuthology.orchestra.run.smithi094.stdout: "ops": [ 2024-09-14T06:32:12.670 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:12.670 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:44425 3.2 3:4603ad7f:::rbd_data.112dca2f6ae1.0000000000000005:17 [read 868352~4096 [fadvise_nocache]] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:32:12.671 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:43.854967+0000", 2024-09-14T06:32:12.671 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 148.81359403499999, 2024-09-14T06:32:12.671 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.063146025999999994, 2024-09-14T06:32:12.671 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:32:12.671 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:32:12.671 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:32:12.671 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:32:12.671 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:32:12.671 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:12.671 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 44425 2024-09-14T06:32:12.671 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:12.671 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:32:12.672 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:12.672 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:32:12.672 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.854967+0000", 2024-09-14T06:32:12.672 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:12.672 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:12.672 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:12.672 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:32:12.672 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.854967+0000", 2024-09-14T06:32:12.672 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:12.672 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:12.672 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:12.672 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:32:12.672 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.854967+0000", 2024-09-14T06:32:12.673 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.16e-07 2024-09-14T06:32:12.673 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:12.673 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:12.673 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:32:12.673 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.854968+0000", 2024-09-14T06:32:12.673 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.006e-06 2024-09-14T06:32:12.673 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:12.673 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:12.673 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:32:12.673 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.854971+0000", 2024-09-14T06:32:12.673 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.9359999999999999e-06 2024-09-14T06:32:12.673 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:12.674 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:12.674 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:32:12.674 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.854974+0000", 2024-09-14T06:32:12.674 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.1840000000000002e-06 2024-09-14T06:32:12.674 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:12.674 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:12.674 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:32:12.674 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.917945+0000", 2024-09-14T06:32:12.674 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.062970925999999997 2024-09-14T06:32:12.674 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:12.674 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:12.674 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:32:12.674 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.918030+0000", 2024-09-14T06:32:12.674 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.5863000000000001e-05 2024-09-14T06:32:12.674 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:12.674 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:12.675 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:32:12.675 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.918113+0000", 2024-09-14T06:32:12.675 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.2894999999999998e-05 2024-09-14T06:32:12.675 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:12.675 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:32:12.675 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:12.675 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:12.675 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:12.675 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:44426 3.2 3:4603ad7f:::rbd_data.112dca2f6ae1.0000000000000005:17 [read 892928~4096 [fadvise_nocache]] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:32:12.676 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:43.854977+0000", 2024-09-14T06:32:12.676 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 148.813584625, 2024-09-14T06:32:12.676 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.063369237999999994, 2024-09-14T06:32:12.676 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:32:12.676 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:32:12.676 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:32:12.676 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:32:12.676 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:32:12.676 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:12.677 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 44426 2024-09-14T06:32:12.677 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:12.677 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:32:12.677 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:12.677 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:32:12.677 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.854977+0000", 2024-09-14T06:32:12.677 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:12.677 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:12.677 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:12.677 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:32:12.677 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.854977+0000", 2024-09-14T06:32:12.677 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:12.678 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:12.678 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:12.678 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:32:12.678 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.854977+0000", 2024-09-14T06:32:12.678 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.1199999999999999e-07 2024-09-14T06:32:12.678 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:12.678 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:12.678 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:32:12.678 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.854980+0000", 2024-09-14T06:32:12.678 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.27e-06 2024-09-14T06:32:12.678 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:12.678 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:12.678 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:32:12.678 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.854983+0000", 2024-09-14T06:32:12.678 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.1650000000000002e-06 2024-09-14T06:32:12.679 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:12.679 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:12.679 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:32:12.679 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.854989+0000", 2024-09-14T06:32:12.679 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.6520000000000003e-06 2024-09-14T06:32:12.679 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:12.679 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:12.679 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:32:12.679 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.918143+0000", 2024-09-14T06:32:12.679 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.063154541999999994 2024-09-14T06:32:12.679 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:12.679 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:12.680 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:32:12.680 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.918209+0000", 2024-09-14T06:32:12.680 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.5529000000000002e-05 2024-09-14T06:32:12.680 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:12.680 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:12.680 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:32:12.680 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.918346+0000", 2024-09-14T06:32:12.680 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000136868 2024-09-14T06:32:12.680 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:12.680 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:32:12.680 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:12.680 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:12.680 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:12.681 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:44427 3.2 3:4603ad7f:::rbd_data.112dca2f6ae1.0000000000000005:17 [read 917504~4096 [fadvise_nocache]] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:32:12.681 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:43.854992+0000", 2024-09-14T06:32:12.681 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 148.813568979, 2024-09-14T06:32:12.681 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.063607792999999996, 2024-09-14T06:32:12.681 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:32:12.681 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:32:12.681 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:32:12.681 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:32:12.681 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:32:12.681 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:12.681 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 44427 2024-09-14T06:32:12.681 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:12.681 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:32:12.681 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:12.682 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:32:12.682 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.854992+0000", 2024-09-14T06:32:12.682 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:12.682 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:12.682 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:12.682 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:32:12.682 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.854992+0000", 2024-09-14T06:32:12.682 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:12.682 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:12.682 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:12.682 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:32:12.682 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.854993+0000", 2024-09-14T06:32:12.682 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.1400000000000001e-07 2024-09-14T06:32:12.682 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:12.682 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:12.682 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:32:12.683 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.854994+0000", 2024-09-14T06:32:12.683 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.091e-06 2024-09-14T06:32:12.683 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:12.683 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:12.683 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:32:12.683 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855001+0000", 2024-09-14T06:32:12.683 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.204e-06 2024-09-14T06:32:12.683 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:12.683 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:12.683 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:32:12.683 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855003+0000", 2024-09-14T06:32:12.683 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.514e-06 2024-09-14T06:32:12.683 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:12.683 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:12.683 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:32:12.684 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.918380+0000", 2024-09-14T06:32:12.684 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.063376738000000002 2024-09-14T06:32:12.684 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:12.684 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:12.684 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:32:12.684 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.918453+0000", 2024-09-14T06:32:12.684 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.3291e-05 2024-09-14T06:32:12.684 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:12.684 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:12.685 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:32:12.685 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.918600+0000", 2024-09-14T06:32:12.685 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00014674100000000001 2024-09-14T06:32:12.685 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:12.685 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:32:12.685 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:12.685 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:12.685 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:12.685 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:44428 3.2 3:4603ad7f:::rbd_data.112dca2f6ae1.0000000000000005:17 [read 942080~4096 [fadvise_nocache]] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:32:12.685 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:43.855006+0000", 2024-09-14T06:32:12.685 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 148.81355492899999, 2024-09-14T06:32:12.685 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.063846321999999997, 2024-09-14T06:32:12.685 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:32:12.686 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:32:12.686 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:32:12.686 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:32:12.686 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:32:12.686 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:12.686 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 44428 2024-09-14T06:32:12.686 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:12.686 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:32:12.686 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:12.686 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:32:12.686 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855006+0000", 2024-09-14T06:32:12.686 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:12.686 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:12.686 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:12.687 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:32:12.687 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855006+0000", 2024-09-14T06:32:12.687 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:12.687 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:12.687 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:12.687 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:32:12.687 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855007+0000", 2024-09-14T06:32:12.687 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.1199999999999999e-07 2024-09-14T06:32:12.687 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:12.687 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:12.687 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:32:12.688 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855011+0000", 2024-09-14T06:32:12.688 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.9539999999999998e-06 2024-09-14T06:32:12.688 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:12.688 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:12.688 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:32:12.688 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855014+0000", 2024-09-14T06:32:12.688 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.2930000000000001e-06 2024-09-14T06:32:12.688 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:12.688 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:12.688 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:32:12.688 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855016+0000", 2024-09-14T06:32:12.688 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.3039999999999999e-06 2024-09-14T06:32:12.688 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:12.688 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:12.689 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:32:12.689 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.918634+0000", 2024-09-14T06:32:12.689 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.063617928000000004 2024-09-14T06:32:12.689 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:12.689 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:12.689 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:32:12.689 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.918711+0000", 2024-09-14T06:32:12.689 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.6545000000000007e-05 2024-09-14T06:32:12.689 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:12.689 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:12.689 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:32:12.689 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.918853+0000", 2024-09-14T06:32:12.689 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00014208600000000001 2024-09-14T06:32:12.689 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:12.689 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:32:12.690 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:12.690 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:12.690 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:12.690 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:44429 3.2 3:4603ad7f:::rbd_data.112dca2f6ae1.0000000000000005:17 [read 966656~4096 [fadvise_nocache]] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:32:12.690 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:43.855019+0000", 2024-09-14T06:32:12.690 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 148.81354225600001, 2024-09-14T06:32:12.690 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.064059226999999996, 2024-09-14T06:32:12.690 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:32:12.690 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:32:12.690 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:32:12.690 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:32:12.690 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:32:12.690 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:12.690 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 44429 2024-09-14T06:32:12.690 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:12.690 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:32:12.691 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:12.691 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:32:12.691 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855019+0000", 2024-09-14T06:32:12.691 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:12.691 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:12.691 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:12.691 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:32:12.691 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855019+0000", 2024-09-14T06:32:12.691 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:12.691 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:12.691 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:12.691 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:32:12.691 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855019+0000", 2024-09-14T06:32:12.691 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.28e-07 2024-09-14T06:32:12.691 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:12.692 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:12.692 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:32:12.692 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855020+0000", 2024-09-14T06:32:12.692 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.0330000000000001e-06 2024-09-14T06:32:12.692 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:12.692 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:12.692 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:32:12.692 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855023+0000", 2024-09-14T06:32:12.692 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.79e-06 2024-09-14T06:32:12.692 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:12.692 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:12.693 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:32:12.693 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855025+0000", 2024-09-14T06:32:12.693 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.216e-06 2024-09-14T06:32:12.693 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:12.693 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:12.693 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:32:12.693 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.918886+0000", 2024-09-14T06:32:12.693 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.063861037999999995 2024-09-14T06:32:12.693 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:12.693 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:12.693 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:32:12.693 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.918952+0000", 2024-09-14T06:32:12.693 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.5993000000000003e-05 2024-09-14T06:32:12.694 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:12.694 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:12.694 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:32:12.694 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.919078+0000", 2024-09-14T06:32:12.694 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00012592899999999999 2024-09-14T06:32:12.694 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:12.694 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:32:12.694 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:12.694 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:12.694 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:12.694 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:44430 3.2 3:4603ad7f:::rbd_data.112dca2f6ae1.0000000000000005:17 [read 991232~4096 [fadvise_nocache]] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:32:12.694 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:43.855028+0000", 2024-09-14T06:32:12.695 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 148.81353324599999, 2024-09-14T06:32:12.695 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.064284346000000006, 2024-09-14T06:32:12.695 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:32:12.695 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:32:12.695 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:32:12.695 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:32:12.695 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:32:12.695 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:12.695 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 44430 2024-09-14T06:32:12.695 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:12.695 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:32:12.695 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:12.695 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:32:12.696 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855028+0000", 2024-09-14T06:32:12.696 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:12.696 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:12.696 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:12.696 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:32:12.696 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855028+0000", 2024-09-14T06:32:12.696 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:12.696 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:12.696 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:12.696 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:32:12.696 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855028+0000", 2024-09-14T06:32:12.696 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.1199999999999999e-07 2024-09-14T06:32:12.696 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:12.696 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:12.697 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:32:12.697 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855029+0000", 2024-09-14T06:32:12.697 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.1100000000000004e-07 2024-09-14T06:32:12.697 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:12.697 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:12.697 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:32:12.697 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855032+0000", 2024-09-14T06:32:12.697 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.7889999999999999e-06 2024-09-14T06:32:12.697 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:12.697 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:12.697 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:32:12.697 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855034+0000", 2024-09-14T06:32:12.697 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.125e-06 2024-09-14T06:32:12.698 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:12.698 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:12.698 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:32:12.698 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.919110+0000", 2024-09-14T06:32:12.698 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.064076413999999998 2024-09-14T06:32:12.698 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:12.698 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:12.698 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:32:12.698 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.919186+0000", 2024-09-14T06:32:12.698 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.5784999999999994e-05 2024-09-14T06:32:12.698 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:12.699 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:12.699 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:32:12.699 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.919312+0000", 2024-09-14T06:32:12.699 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00012611 2024-09-14T06:32:12.699 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:12.699 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:32:12.699 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:12.699 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:12.699 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:12.699 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:44431 3.2 3:4603ad7f:::rbd_data.112dca2f6ae1.0000000000000005:17 [read 1015808~4096 [fadvise_nocache]] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:32:12.699 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:43.855037+0000", 2024-09-14T06:32:12.699 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 148.81352437800001, 2024-09-14T06:32:12.699 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.064469755000000004, 2024-09-14T06:32:12.700 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:32:12.700 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:32:12.700 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:32:12.700 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:32:12.700 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:32:12.700 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:12.700 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 44431 2024-09-14T06:32:12.700 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:12.700 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:32:12.700 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:12.700 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:32:12.700 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855037+0000", 2024-09-14T06:32:12.700 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:12.701 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:12.701 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:12.701 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:32:12.701 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855037+0000", 2024-09-14T06:32:12.701 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:12.701 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:12.701 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:12.701 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:32:12.701 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855037+0000", 2024-09-14T06:32:12.701 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.1e-07 2024-09-14T06:32:12.701 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:12.701 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:12.701 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:32:12.701 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855038+0000", 2024-09-14T06:32:12.701 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.9400000000000004e-07 2024-09-14T06:32:12.702 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:12.702 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:12.702 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:32:12.702 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855041+0000", 2024-09-14T06:32:12.702 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.9639999999999999e-06 2024-09-14T06:32:12.702 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:12.702 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:12.702 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:32:12.702 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855043+0000", 2024-09-14T06:32:12.702 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.2259999999999999e-06 2024-09-14T06:32:12.702 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:12.702 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:12.702 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:32:12.703 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.919347+0000", 2024-09-14T06:32:12.703 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.064304091999999993 2024-09-14T06:32:12.703 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:12.703 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:12.703 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:32:12.703 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.919393+0000", 2024-09-14T06:32:12.703 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.5340999999999998e-05 2024-09-14T06:32:12.703 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:12.703 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:12.703 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:32:12.703 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.919507+0000", 2024-09-14T06:32:12.703 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000114028 2024-09-14T06:32:12.703 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:12.703 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:32:12.703 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:12.704 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:12.704 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:12.704 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:44432 3.2 3:4603ad7f:::rbd_data.112dca2f6ae1.0000000000000005:17 [read 1040384~4096 [fadvise_nocache]] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:32:12.704 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:43.855046+0000", 2024-09-14T06:32:12.704 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 148.81351531000001, 2024-09-14T06:32:12.704 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.064639253999999993, 2024-09-14T06:32:12.704 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:32:12.704 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:32:12.704 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:32:12.704 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:32:12.705 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:32:12.705 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:12.705 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 44432 2024-09-14T06:32:12.705 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:12.705 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:32:12.705 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:12.705 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:32:12.705 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855046+0000", 2024-09-14T06:32:12.705 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:12.705 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:12.705 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:12.705 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:32:12.705 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855046+0000", 2024-09-14T06:32:12.705 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:12.706 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:12.706 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:12.706 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:32:12.706 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855046+0000", 2024-09-14T06:32:12.706 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.05e-07 2024-09-14T06:32:12.706 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:12.706 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:12.706 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:32:12.706 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855047+0000", 2024-09-14T06:32:12.706 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.879999999999999e-07 2024-09-14T06:32:12.706 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:12.706 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:12.706 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:32:12.706 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855050+0000", 2024-09-14T06:32:12.706 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.8459999999999999e-06 2024-09-14T06:32:12.706 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:12.707 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:12.707 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:32:12.707 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855052+0000", 2024-09-14T06:32:12.707 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.136e-06 2024-09-14T06:32:12.707 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:12.707 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:12.707 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:32:12.707 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.919536+0000", 2024-09-14T06:32:12.707 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.064484132 2024-09-14T06:32:12.707 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:12.707 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:12.707 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:32:12.707 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.919611+0000", 2024-09-14T06:32:12.707 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.4407999999999996e-05 2024-09-14T06:32:12.707 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:12.707 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:12.708 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:32:12.708 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.919685+0000", 2024-09-14T06:32:12.708 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.4538999999999999e-05 2024-09-14T06:32:12.708 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:12.708 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:32:12.708 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:12.708 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:12.708 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:12.708 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:44518 3.2 3:4190d882:::rbd_data.112dca2f6ae1.0000000000000002:17 [read 12288~4096 [fadvise_nocache]] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:32:12.708 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:43.855055+0000", 2024-09-14T06:32:12.708 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 148.813506426, 2024-09-14T06:32:12.708 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.064894317000000007, 2024-09-14T06:32:12.708 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:32:12.708 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:32:12.709 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:32:12.709 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:32:12.709 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:32:12.709 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:12.709 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 44518 2024-09-14T06:32:12.709 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:12.709 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:32:12.709 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:12.709 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:32:12.709 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855055+0000", 2024-09-14T06:32:12.709 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:12.709 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:12.709 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:12.709 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:32:12.710 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855055+0000", 2024-09-14T06:32:12.710 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:12.710 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:12.710 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:12.710 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:32:12.710 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855055+0000", 2024-09-14T06:32:12.710 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.0699999999999999e-07 2024-09-14T06:32:12.710 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:12.710 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:12.710 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:32:12.710 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855056+0000", 2024-09-14T06:32:12.710 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.9400000000000004e-07 2024-09-14T06:32:12.710 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:12.711 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:12.711 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:32:12.711 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855059+0000", 2024-09-14T06:32:12.711 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.8870000000000001e-06 2024-09-14T06:32:12.711 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:12.711 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:12.711 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:32:12.711 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855061+0000", 2024-09-14T06:32:12.711 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.1919999999999999e-06 2024-09-14T06:32:12.711 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:12.711 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:12.711 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:32:12.711 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.919719+0000", 2024-09-14T06:32:12.712 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.064658013 2024-09-14T06:32:12.712 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:12.712 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:12.712 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:32:12.712 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.919788+0000", 2024-09-14T06:32:12.712 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.9282000000000005e-05 2024-09-14T06:32:12.712 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:12.712 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:12.712 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:32:12.712 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.919949+0000", 2024-09-14T06:32:12.712 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00016084199999999999 2024-09-14T06:32:12.713 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:12.713 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:32:12.713 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:12.713 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:12.713 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:12.713 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:44519 3.2 3:4190d882:::rbd_data.112dca2f6ae1.0000000000000002:17 [read 36864~4096 [fadvise_nocache]] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:32:12.713 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:43.855064+0000", 2024-09-14T06:32:12.713 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 148.81349744900001, 2024-09-14T06:32:12.713 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.065081865000000003, 2024-09-14T06:32:12.713 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:32:12.713 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:32:12.713 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:32:12.714 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:32:12.714 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:32:12.714 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:12.714 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 44519 2024-09-14T06:32:12.714 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:12.714 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:32:12.714 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:12.714 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:32:12.714 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855064+0000", 2024-09-14T06:32:12.714 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:12.714 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:12.714 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:12.714 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:32:12.714 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855064+0000", 2024-09-14T06:32:12.715 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:12.715 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:12.715 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:12.715 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:32:12.715 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855064+0000", 2024-09-14T06:32:12.715 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.1e-07 2024-09-14T06:32:12.715 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:12.715 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:12.715 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:32:12.716 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855065+0000", 2024-09-14T06:32:12.716 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.035e-06 2024-09-14T06:32:12.716 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:12.716 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:12.716 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:32:12.716 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855068+0000", 2024-09-14T06:32:12.716 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.8339999999999999e-06 2024-09-14T06:32:12.716 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:12.716 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:12.716 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:32:12.716 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855071+0000", 2024-09-14T06:32:12.717 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.3450000000000002e-06 2024-09-14T06:32:12.717 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:12.717 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:12.717 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:32:12.717 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.919979+0000", 2024-09-14T06:32:12.717 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.064907647999999998 2024-09-14T06:32:12.717 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:12.717 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:12.717 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:32:12.717 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.920026+0000", 2024-09-14T06:32:12.717 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.7595000000000001e-05 2024-09-14T06:32:12.717 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:12.718 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:12.718 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:32:12.718 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.920146+0000", 2024-09-14T06:32:12.718 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00011919800000000001 2024-09-14T06:32:12.718 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:12.718 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:32:12.718 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:12.718 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:12.718 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:12.718 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:44520 3.2 3:4190d882:::rbd_data.112dca2f6ae1.0000000000000002:17 [read 61440~4096 [fadvise_nocache]] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:32:12.718 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:43.855075+0000", 2024-09-14T06:32:12.719 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 148.81348639800001, 2024-09-14T06:32:12.719 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.065335078000000005, 2024-09-14T06:32:12.719 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:32:12.719 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:32:12.719 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:32:12.719 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:32:12.719 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:32:12.719 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:12.719 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 44520 2024-09-14T06:32:12.719 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:12.719 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:32:12.719 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:12.719 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:32:12.719 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855075+0000", 2024-09-14T06:32:12.720 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:12.720 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:12.720 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:12.720 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:32:12.720 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855075+0000", 2024-09-14T06:32:12.720 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:12.720 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:12.720 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:12.720 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:32:12.720 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855075+0000", 2024-09-14T06:32:12.720 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.1e-07 2024-09-14T06:32:12.720 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:12.720 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:12.720 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:32:12.720 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855076+0000", 2024-09-14T06:32:12.720 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.0839999999999999e-06 2024-09-14T06:32:12.721 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:12.721 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:12.721 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:32:12.721 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855079+0000", 2024-09-14T06:32:12.721 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.9359999999999999e-06 2024-09-14T06:32:12.721 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:12.721 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:12.721 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:32:12.721 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855084+0000", 2024-09-14T06:32:12.721 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.1730000000000003e-06 2024-09-14T06:32:12.721 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:12.721 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:12.721 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:32:12.721 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.920175+0000", 2024-09-14T06:32:12.721 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.065090234999999996 2024-09-14T06:32:12.722 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:12.722 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:12.722 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:32:12.722 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.920250+0000", 2024-09-14T06:32:12.722 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.5807000000000001e-05 2024-09-14T06:32:12.722 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:12.722 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:12.722 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:32:12.722 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.920410+0000", 2024-09-14T06:32:12.722 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00015963300000000001 2024-09-14T06:32:12.723 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:12.723 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:32:12.723 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:12.723 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:12.723 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:12.723 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:44521 3.2 3:4190d882:::rbd_data.112dca2f6ae1.0000000000000002:17 [read 86016~4096 [fadvise_nocache]] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:32:12.723 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:43.855087+0000", 2024-09-14T06:32:12.723 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 148.81347391700001, 2024-09-14T06:32:12.723 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.065563264999999996, 2024-09-14T06:32:12.723 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:32:12.723 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:32:12.723 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:32:12.723 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:32:12.723 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:32:12.723 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:12.724 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 44521 2024-09-14T06:32:12.724 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:12.724 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:32:12.724 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:12.724 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:32:12.724 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855087+0000", 2024-09-14T06:32:12.724 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:12.724 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:12.724 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:12.724 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:32:12.724 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855087+0000", 2024-09-14T06:32:12.724 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:12.724 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:12.724 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:12.724 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:32:12.725 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855088+0000", 2024-09-14T06:32:12.725 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.3699999999999999e-07 2024-09-14T06:32:12.725 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:12.725 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:12.725 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:32:12.725 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855089+0000", 2024-09-14T06:32:12.725 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.9900000000000009e-07 2024-09-14T06:32:12.725 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:12.725 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:12.725 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:32:12.725 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855091+0000", 2024-09-14T06:32:12.725 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.8940000000000002e-06 2024-09-14T06:32:12.725 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:12.725 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:12.725 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:32:12.726 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855094+0000", 2024-09-14T06:32:12.726 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.351e-06 2024-09-14T06:32:12.726 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:12.726 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:12.726 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:32:12.726 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.920437+0000", 2024-09-14T06:32:12.726 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.065343221000000007 2024-09-14T06:32:12.726 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:12.726 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:12.726 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:32:12.726 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.920481+0000", 2024-09-14T06:32:12.726 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.4431000000000002e-05 2024-09-14T06:32:12.726 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:12.726 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:12.727 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:32:12.727 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.920651+0000", 2024-09-14T06:32:12.727 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00016913200000000001 2024-09-14T06:32:12.727 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:12.727 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:32:12.727 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:12.727 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:12.727 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:12.727 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:44522 3.2 3:4190d882:::rbd_data.112dca2f6ae1.0000000000000002:17 [read 110592~4096 [fadvise_nocache]] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:32:12.727 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:43.855098+0000", 2024-09-14T06:32:12.727 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 148.81346368999999, 2024-09-14T06:32:12.727 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.065827628999999999, 2024-09-14T06:32:12.727 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:32:12.727 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:32:12.727 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:32:12.728 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:32:12.728 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:32:12.728 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:12.728 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 44522 2024-09-14T06:32:12.728 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:12.728 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:32:12.728 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:12.728 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:32:12.728 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855098+0000", 2024-09-14T06:32:12.728 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:12.728 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:12.728 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:12.728 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:32:12.728 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855098+0000", 2024-09-14T06:32:12.729 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:12.729 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:12.729 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:12.729 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:32:12.729 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855098+0000", 2024-09-14T06:32:12.729 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.1e-07 2024-09-14T06:32:12.729 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:12.729 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:12.729 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:32:12.729 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855099+0000", 2024-09-14T06:32:12.729 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.012e-06 2024-09-14T06:32:12.730 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:12.730 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:12.730 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:32:12.730 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855102+0000", 2024-09-14T06:32:12.730 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.9380000000000001e-06 2024-09-14T06:32:12.730 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:12.730 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:12.730 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:32:12.730 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855108+0000", 2024-09-14T06:32:12.730 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.4339999999999997e-06 2024-09-14T06:32:12.730 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:12.730 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:12.730 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:32:12.730 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.920681+0000", 2024-09-14T06:32:12.731 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.065572943999999994 2024-09-14T06:32:12.731 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:12.731 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:12.731 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:32:12.731 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.920724+0000", 2024-09-14T06:32:12.731 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.2877000000000003e-05 2024-09-14T06:32:12.731 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:12.731 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:12.731 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:32:12.731 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.920925+0000", 2024-09-14T06:32:12.731 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00020121399999999999 2024-09-14T06:32:12.731 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:12.731 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:32:12.731 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:12.732 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:12.732 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:12.732 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:44523 3.2 3:4190d882:::rbd_data.112dca2f6ae1.0000000000000002:17 [read 135168~4096 [fadvise_nocache]] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:32:12.732 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:43.855111+0000", 2024-09-14T06:32:12.732 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 148.81344996799999, 2024-09-14T06:32:12.732 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.066028198999999996, 2024-09-14T06:32:12.732 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:32:12.732 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:32:12.732 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:32:12.733 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:32:12.733 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:32:12.733 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:12.733 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 44523 2024-09-14T06:32:12.733 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:12.733 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:32:12.733 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:12.733 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:32:12.733 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855111+0000", 2024-09-14T06:32:12.733 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:12.733 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:12.733 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:12.734 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:32:12.734 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855111+0000", 2024-09-14T06:32:12.734 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:12.734 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:12.734 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:12.734 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:32:12.734 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855112+0000", 2024-09-14T06:32:12.734 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.1199999999999999e-07 2024-09-14T06:32:12.734 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:12.734 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:12.734 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:32:12.734 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855115+0000", 2024-09-14T06:32:12.734 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.3380000000000001e-06 2024-09-14T06:32:12.735 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:12.735 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:12.735 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:32:12.735 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855118+0000", 2024-09-14T06:32:12.735 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.366e-06 2024-09-14T06:32:12.735 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:12.735 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:12.735 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:32:12.735 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855121+0000", 2024-09-14T06:32:12.735 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.8329999999999998e-06 2024-09-14T06:32:12.735 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:12.735 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:12.736 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:32:12.736 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.920956+0000", 2024-09-14T06:32:12.736 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.065835251999999997 2024-09-14T06:32:12.736 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:12.736 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:12.736 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:32:12.736 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.921001+0000", 2024-09-14T06:32:12.736 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.4778999999999999e-05 2024-09-14T06:32:12.736 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:12.736 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:12.736 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:32:12.736 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.921139+0000", 2024-09-14T06:32:12.736 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000138419 2024-09-14T06:32:12.736 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:12.736 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:32:12.737 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:12.737 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:12.737 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:12.737 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:44524 3.2 3:4190d882:::rbd_data.112dca2f6ae1.0000000000000002:17 [read 159744~4096 [fadvise_nocache]] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:32:12.737 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:43.855124+0000", 2024-09-14T06:32:12.737 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 148.813436989, 2024-09-14T06:32:12.737 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.066249261000000004, 2024-09-14T06:32:12.737 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:32:12.737 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:32:12.737 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:32:12.737 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:32:12.737 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:32:12.737 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:12.737 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 44524 2024-09-14T06:32:12.738 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:12.738 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:32:12.738 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:12.738 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:32:12.738 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855124+0000", 2024-09-14T06:32:12.738 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:12.738 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:12.738 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:12.738 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:32:12.738 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855124+0000", 2024-09-14T06:32:12.738 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:12.738 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:12.739 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:12.739 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:32:12.739 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861086+0000", 2024-09-14T06:32:12.739 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0059613829999999998 2024-09-14T06:32:12.739 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:12.739 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:12.739 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:32:12.739 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861100+0000", 2024-09-14T06:32:12.739 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.4644000000000001e-05 2024-09-14T06:32:12.739 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:12.739 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:12.739 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:32:12.739 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861109+0000", 2024-09-14T06:32:12.739 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.541e-06 2024-09-14T06:32:12.739 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:12.740 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:12.740 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:32:12.740 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861117+0000", 2024-09-14T06:32:12.740 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.1049999999999995e-06 2024-09-14T06:32:12.740 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:12.740 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:12.740 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:32:12.740 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.921169+0000", 2024-09-14T06:32:12.740 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.060052405000000003 2024-09-14T06:32:12.740 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:12.740 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:12.740 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:32:12.740 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.921213+0000", 2024-09-14T06:32:12.740 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.3565e-05 2024-09-14T06:32:12.740 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:12.740 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:12.741 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:32:12.741 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.921374+0000", 2024-09-14T06:32:12.741 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00016061800000000001 2024-09-14T06:32:12.741 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:12.741 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:32:12.741 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:12.741 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:12.741 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:12.741 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:44537 3.2 3:4190d882:::rbd_data.112dca2f6ae1.0000000000000002:17 [read 479232~4096 [fadvise_nocache]] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:32:12.741 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:43.861405+0000", 2024-09-14T06:32:12.741 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 148.807156386, 2024-09-14T06:32:12.741 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.063351795000000002, 2024-09-14T06:32:12.741 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:32:12.741 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:32:12.742 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:32:12.742 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:32:12.742 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:32:12.742 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:12.742 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 44537 2024-09-14T06:32:12.742 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:12.742 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:32:12.742 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:12.742 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:32:12.742 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861405+0000", 2024-09-14T06:32:12.742 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:12.742 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:12.742 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:12.742 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:32:12.742 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861405+0000", 2024-09-14T06:32:12.742 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:12.742 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:12.743 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:12.743 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:32:12.743 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861405+0000", 2024-09-14T06:32:12.743 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.9099999999999999e-07 2024-09-14T06:32:12.743 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:12.743 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:12.743 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:32:12.743 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861407+0000", 2024-09-14T06:32:12.743 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.075e-06 2024-09-14T06:32:12.743 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:12.743 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:12.743 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:32:12.743 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861414+0000", 2024-09-14T06:32:12.743 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.4659999999999996e-06 2024-09-14T06:32:12.743 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:12.744 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:12.744 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:32:12.744 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861418+0000", 2024-09-14T06:32:12.744 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.5519999999999998e-06 2024-09-14T06:32:12.744 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:12.744 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:12.744 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:32:12.744 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.924474+0000", 2024-09-14T06:32:12.744 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.063056012999999994 2024-09-14T06:32:12.744 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:12.744 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:12.744 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:32:12.745 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.924556+0000", 2024-09-14T06:32:12.745 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.1181999999999996e-05 2024-09-14T06:32:12.745 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:12.745 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:12.745 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:32:12.745 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.924757+0000", 2024-09-14T06:32:12.745 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00020111599999999999 2024-09-14T06:32:12.745 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:12.745 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:32:12.745 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:12.745 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:12.745 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:12.745 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:44538 3.2 3:4190d882:::rbd_data.112dca2f6ae1.0000000000000002:17 [read 503808~4096 [fadvise_nocache]] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:32:12.745 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:43.861425+0000", 2024-09-14T06:32:12.746 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 148.807136345, 2024-09-14T06:32:12.746 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.063627125000000007, 2024-09-14T06:32:12.746 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:32:12.746 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:32:12.746 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:32:12.746 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:32:12.746 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:32:12.746 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:12.746 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 44538 2024-09-14T06:32:12.746 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:12.746 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:32:12.746 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:12.746 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:32:12.746 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861425+0000", 2024-09-14T06:32:12.747 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:12.747 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:12.747 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:12.747 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:32:12.747 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861425+0000", 2024-09-14T06:32:12.747 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:12.747 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:12.747 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:12.747 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:32:12.747 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861425+0000", 2024-09-14T06:32:12.747 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.9999999999999998e-07 2024-09-14T06:32:12.747 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:12.747 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:12.747 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:32:12.748 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861427+0000", 2024-09-14T06:32:12.748 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.9829999999999999e-06 2024-09-14T06:32:12.748 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:12.748 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:12.748 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:32:12.748 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861433+0000", 2024-09-14T06:32:12.748 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.2889999999999997e-06 2024-09-14T06:32:12.748 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:12.748 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:12.748 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:32:12.748 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861454+0000", 2024-09-14T06:32:12.749 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.1844999999999998e-05 2024-09-14T06:32:12.749 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:12.749 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:12.749 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:32:12.749 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.924786+0000", 2024-09-14T06:32:12.749 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.063331497 2024-09-14T06:32:12.749 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:12.749 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:12.749 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:32:12.749 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.924864+0000", 2024-09-14T06:32:12.749 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.7744000000000006e-05 2024-09-14T06:32:12.749 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:12.749 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:12.749 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:32:12.750 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.925052+0000", 2024-09-14T06:32:12.750 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00018836699999999999 2024-09-14T06:32:12.750 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:12.750 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:32:12.750 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:12.750 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:12.750 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:12.750 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:44539 3.2 3:4190d882:::rbd_data.112dca2f6ae1.0000000000000002:17 [read 528384~4096 [fadvise_nocache]] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:32:12.750 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:43.861462+0000", 2024-09-14T06:32:12.750 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 148.807099018, 2024-09-14T06:32:12.750 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.063855705999999998, 2024-09-14T06:32:12.750 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:32:12.750 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:32:12.750 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:32:12.750 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:32:12.750 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:32:12.751 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:12.751 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 44539 2024-09-14T06:32:12.751 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:12.751 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:32:12.751 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:12.751 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:32:12.751 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861462+0000", 2024-09-14T06:32:12.751 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:12.751 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:12.751 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:12.751 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:32:12.751 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861462+0000", 2024-09-14T06:32:12.751 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:12.752 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:12.752 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:12.752 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:32:12.752 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861463+0000", 2024-09-14T06:32:12.752 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.5200000000000002e-07 2024-09-14T06:32:12.752 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:12.752 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:12.752 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:32:12.752 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861465+0000", 2024-09-14T06:32:12.752 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.5189999999999999e-06 2024-09-14T06:32:12.752 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:12.752 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:12.752 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:32:12.753 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861471+0000", 2024-09-14T06:32:12.753 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.6899999999999997e-06 2024-09-14T06:32:12.753 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:12.753 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:12.753 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:32:12.753 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861475+0000", 2024-09-14T06:32:12.753 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.5730000000000001e-06 2024-09-14T06:32:12.753 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:12.753 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:12.753 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:32:12.754 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.925085+0000", 2024-09-14T06:32:12.754 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.063609902999999995 2024-09-14T06:32:12.754 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:12.754 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:12.754 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:32:12.754 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.925145+0000", 2024-09-14T06:32:12.754 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.9383999999999999e-05 2024-09-14T06:32:12.754 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:12.754 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:12.754 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:32:12.754 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.925318+0000", 2024-09-14T06:32:12.754 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00017318499999999999 2024-09-14T06:32:12.754 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:12.754 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:32:12.754 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:12.755 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:12.755 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:12.755 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:44540 3.2 3:4190d882:::rbd_data.112dca2f6ae1.0000000000000002:17 [read 552960~4096 [fadvise_nocache]] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:32:12.755 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:43.861483+0000", 2024-09-14T06:32:12.755 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 148.80707858700001, 2024-09-14T06:32:12.755 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.064065681999999999, 2024-09-14T06:32:12.755 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:32:12.755 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:32:12.755 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:32:12.755 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:32:12.755 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:32:12.756 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:12.756 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 44540 2024-09-14T06:32:12.756 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:12.756 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:32:12.756 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:12.756 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:32:12.756 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861483+0000", 2024-09-14T06:32:12.756 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:12.756 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:12.756 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:12.756 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:32:12.756 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861483+0000", 2024-09-14T06:32:12.756 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:12.756 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:12.757 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:12.757 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:32:12.757 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.867079+0000", 2024-09-14T06:32:12.757 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0055966480000000001 2024-09-14T06:32:12.757 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:12.757 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:12.757 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:32:12.757 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.867086+0000", 2024-09-14T06:32:12.757 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.387e-06 2024-09-14T06:32:12.757 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:12.757 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:12.757 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:32:12.757 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.867094+0000", 2024-09-14T06:32:12.758 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.9419999999999995e-06 2024-09-14T06:32:12.758 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:12.758 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:12.758 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:32:12.758 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.867109+0000", 2024-09-14T06:32:12.758 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.5818999999999999e-05 2024-09-14T06:32:12.758 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:12.758 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:12.758 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:32:12.758 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.925354+0000", 2024-09-14T06:32:12.758 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.058244047 2024-09-14T06:32:12.758 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:12.758 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:12.758 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:32:12.759 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.925406+0000", 2024-09-14T06:32:12.759 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.2661999999999998e-05 2024-09-14T06:32:12.759 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:12.759 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:12.759 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:32:12.759 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.925548+0000", 2024-09-14T06:32:12.759 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000142177 2024-09-14T06:32:12.759 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:12.759 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:32:12.759 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:12.759 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:12.759 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:12.759 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:225587 3.7 3:e112e2b3:::rbd_id.image_client.0-clone42:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e189)", 2024-09-14T06:32:12.759 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:31:53.874591+0000", 2024-09-14T06:32:12.759 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 18.79397037, 2024-09-14T06:32:12.760 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.875541507, 2024-09-14T06:32:12.760 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:32:12.760 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:32:12.760 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:32:12.760 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:32:12.760 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:32:12.760 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:12.760 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 225587 2024-09-14T06:32:12.760 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:12.760 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:32:12.760 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:12.760 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:32:12.760 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:53.874591+0000", 2024-09-14T06:32:12.760 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:12.761 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:12.761 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:12.761 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:32:12.761 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:53.874591+0000", 2024-09-14T06:32:12.761 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:12.761 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:12.761 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:12.761 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:32:12.761 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:53.874592+0000", 2024-09-14T06:32:12.761 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.5209999999999999e-06 2024-09-14T06:32:12.761 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:12.761 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:12.761 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:32:12.761 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:53.874599+0000", 2024-09-14T06:32:12.761 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.6290000000000004e-06 2024-09-14T06:32:12.762 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:12.762 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:12.762 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:32:12.762 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:53.874618+0000", 2024-09-14T06:32:12.762 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.8950999999999999e-05 2024-09-14T06:32:12.762 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:12.762 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:12.762 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:32:12.762 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:53.874626+0000", 2024-09-14T06:32:12.762 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.3960000000000006e-06 2024-09-14T06:32:12.762 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:12.762 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:12.762 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:32:12.763 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:53.874699+0000", 2024-09-14T06:32:12.763 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.2329999999999994e-05 2024-09-14T06:32:12.763 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:12.763 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:12.763 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "waiting for peered", 2024-09-14T06:32:12.763 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:53.874711+0000", 2024-09-14T06:32:12.763 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.2473999999999999e-05 2024-09-14T06:32:12.763 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:12.763 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:12.763 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:32:12.763 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:54.749578+0000", 2024-09-14T06:32:12.763 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.87486642100000001 2024-09-14T06:32:12.763 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:12.764 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:12.764 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:32:12.764 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:54.749706+0000", 2024-09-14T06:32:12.764 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00012856200000000001 2024-09-14T06:32:12.764 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:12.764 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:12.764 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:32:12.764 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:54.750132+0000", 2024-09-14T06:32:12.764 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00042622299999999999 2024-09-14T06:32:12.764 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:12.765 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:32:12.765 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:12.765 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:12.765 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:32:12.765 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-14T06:32:12.766 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 2 1 2024-09-14T06:32:12.768 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-14T06:32:12.768 INFO:tasks.ceph.osd.6.smithi138.stderr:2024-09-14T06:32:12.764+0000 7f8cd8849640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33620) UID: 0 2024-09-14T06:32:12.867 INFO:tasks.ceph.osd.3.smithi094.stderr:2024-09-14T06:32:12.866+0000 7feeeafe8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33655) UID: 0 2024-09-14T06:32:12.872 INFO:teuthology.orchestra.run.smithi138.stdout:{ 2024-09-14T06:32:12.873 INFO:teuthology.orchestra.run.smithi138.stdout: "ops": [], 2024-09-14T06:32:12.873 INFO:teuthology.orchestra.run.smithi138.stdout: "num_ops": 0 2024-09-14T06:32:12.873 INFO:teuthology.orchestra.run.smithi138.stdout:} 2024-09-14T06:32:12.881 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-14T06:32:12.968 INFO:tasks.ceph.osd.3.smithi094.stderr:2024-09-14T06:32:12.967+0000 7feeeafe8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33655) UID: 0 2024-09-14T06:32:12.984 INFO:teuthology.orchestra.run.smithi138.stdout:{ 2024-09-14T06:32:12.984 INFO:teuthology.orchestra.run.smithi138.stdout: "ops": [], 2024-09-14T06:32:12.984 INFO:teuthology.orchestra.run.smithi138.stdout: "complaint_time": 30, 2024-09-14T06:32:12.984 INFO:teuthology.orchestra.run.smithi138.stdout: "num_blocked_ops": 0 2024-09-14T06:32:12.985 INFO:teuthology.orchestra.run.smithi138.stdout:} 2024-09-14T06:32:12.994 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-14T06:32:13.069 INFO:tasks.ceph.osd.0.smithi094.stderr:2024-09-14T06:32:13.068+0000 7f07c9333640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33636) UID: 0 2024-09-14T06:32:13.079 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-14T06:32:13.097 INFO:teuthology.orchestra.run.smithi138.stdout:{ 2024-09-14T06:32:13.097 INFO:teuthology.orchestra.run.smithi138.stdout: "size": 20, 2024-09-14T06:32:13.097 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 600, 2024-09-14T06:32:13.097 INFO:teuthology.orchestra.run.smithi138.stdout: "ops": [ 2024-09-14T06:32:13.098 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:13.098 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:19328 3.3 3:c34e5e51:::rbd_data.112d92027590.0000000000000024:f [read 1926144~1024 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:32:13.098 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:27.059768+0000", 2024-09-14T06:32:13.098 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 166.03606971599999, 2024-09-14T06:32:13.098 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046369147999999999, 2024-09-14T06:32:13.098 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:32:13.098 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:32:13.098 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:32:13.098 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:32:13.098 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:32:13.098 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:13.098 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 19328 2024-09-14T06:32:13.098 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:13.098 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:32:13.099 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:13.099 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:32:13.099 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.059768+0000", 2024-09-14T06:32:13.099 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:13.099 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:13.099 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:13.099 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:32:13.099 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.059768+0000", 2024-09-14T06:32:13.099 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:13.099 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:13.099 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:13.099 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:32:13.099 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.059874+0000", 2024-09-14T06:32:13.099 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00010551400000000001 2024-09-14T06:32:13.100 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:13.100 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:13.100 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:32:13.100 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.059877+0000", 2024-09-14T06:32:13.100 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.8279999999999999e-06 2024-09-14T06:32:13.100 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:13.100 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:13.100 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:32:13.100 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.059884+0000", 2024-09-14T06:32:13.100 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.6389999999999992e-06 2024-09-14T06:32:13.100 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:13.100 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:13.100 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:32:13.101 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.059890+0000", 2024-09-14T06:32:13.101 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.49e-06 2024-09-14T06:32:13.101 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:13.101 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:13.101 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:32:13.101 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.105804+0000", 2024-09-14T06:32:13.101 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.045914242000000001 2024-09-14T06:32:13.101 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:13.101 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:13.101 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:32:13.101 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.105886+0000", 2024-09-14T06:32:13.101 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.2305999999999994e-05 2024-09-14T06:32:13.101 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:13.101 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:13.101 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:32:13.102 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.106138+0000", 2024-09-14T06:32:13.102 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.000251129 2024-09-14T06:32:13.102 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:13.102 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:32:13.102 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:13.102 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:13.102 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:13.102 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:19329 3.3 3:c34e5e51:::rbd_data.112d92027590.0000000000000024:f [read 1936384~1024 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:32:13.102 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:27.059899+0000", 2024-09-14T06:32:13.102 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 166.03593919299999, 2024-09-14T06:32:13.102 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046572122000000001, 2024-09-14T06:32:13.102 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:32:13.102 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:32:13.102 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:32:13.102 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:32:13.103 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:32:13.103 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:13.103 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 19329 2024-09-14T06:32:13.103 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:13.103 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:32:13.103 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:13.103 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:32:13.103 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.059899+0000", 2024-09-14T06:32:13.103 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:13.103 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:13.103 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:13.103 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:32:13.103 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.059899+0000", 2024-09-14T06:32:13.103 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:13.103 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:13.103 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:13.104 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:32:13.104 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062501+0000", 2024-09-14T06:32:13.104 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.0026016419999999999 2024-09-14T06:32:13.104 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:13.104 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:13.104 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:32:13.104 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062514+0000", 2024-09-14T06:32:13.104 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.2947e-05 2024-09-14T06:32:13.104 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:13.104 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:13.104 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:32:13.104 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062523+0000", 2024-09-14T06:32:13.104 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.5519999999999993e-06 2024-09-14T06:32:13.104 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:13.104 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:13.105 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:32:13.105 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062534+0000", 2024-09-14T06:32:13.105 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.1416e-05 2024-09-14T06:32:13.105 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:13.105 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:13.105 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:32:13.105 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.106179+0000", 2024-09-14T06:32:13.105 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.043644049999999997 2024-09-14T06:32:13.105 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:13.105 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:13.105 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:32:13.105 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.106262+0000", 2024-09-14T06:32:13.105 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.3178999999999998e-05 2024-09-14T06:32:13.105 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:13.105 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:13.105 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:32:13.106 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.106471+0000", 2024-09-14T06:32:13.106 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00020933599999999999 2024-09-14T06:32:13.106 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:13.106 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:32:13.106 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:13.106 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:13.106 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:13.106 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:19351 3.3 3:c34e5e51:::rbd_data.112d92027590.0000000000000024:f [read 1159168~1024 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:32:13.106 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:27.062766+0000", 2024-09-14T06:32:13.106 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 166.033071968, 2024-09-14T06:32:13.106 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046099454999999998, 2024-09-14T06:32:13.106 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:32:13.106 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:32:13.106 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:32:13.106 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:32:13.106 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:32:13.107 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:13.107 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 19351 2024-09-14T06:32:13.107 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:13.107 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:32:13.107 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:13.107 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:32:13.107 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062766+0000", 2024-09-14T06:32:13.107 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:13.107 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:13.107 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:13.107 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:32:13.107 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062766+0000", 2024-09-14T06:32:13.107 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:13.107 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:13.107 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:13.107 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:32:13.108 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062766+0000", 2024-09-14T06:32:13.108 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.6e-07 2024-09-14T06:32:13.108 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:13.108 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:13.108 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:32:13.108 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062767+0000", 2024-09-14T06:32:13.108 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.0070000000000001e-06 2024-09-14T06:32:13.108 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:13.108 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:13.108 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:32:13.108 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062785+0000", 2024-09-14T06:32:13.108 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.7884999999999999e-05 2024-09-14T06:32:13.108 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:13.108 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:13.108 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:32:13.109 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062790+0000", 2024-09-14T06:32:13.109 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.0900000000000004e-06 2024-09-14T06:32:13.109 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:13.109 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:13.109 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:32:13.109 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.108735+0000", 2024-09-14T06:32:13.109 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.045944411999999997 2024-09-14T06:32:13.109 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:13.109 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:13.109 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:32:13.109 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.108809+0000", 2024-09-14T06:32:13.109 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.4157000000000002e-05 2024-09-14T06:32:13.109 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:13.109 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:13.109 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:32:13.109 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.108866+0000", 2024-09-14T06:32:13.110 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.6644000000000002e-05 2024-09-14T06:32:13.110 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:13.110 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:32:13.110 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:13.110 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:13.110 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:13.110 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:19352 3.3 3:c34e5e51:::rbd_data.112d92027590.0000000000000024:f [read 1169408~1024 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:32:13.110 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:27.062798+0000", 2024-09-14T06:32:13.110 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 166.03304055699999, 2024-09-14T06:32:13.110 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046255900000000003, 2024-09-14T06:32:13.110 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:32:13.110 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:32:13.110 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:32:13.110 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:32:13.110 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:32:13.111 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:13.111 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 19352 2024-09-14T06:32:13.111 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:13.111 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:32:13.111 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:13.111 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:32:13.111 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062798+0000", 2024-09-14T06:32:13.111 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:13.111 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:13.111 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:13.111 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:32:13.111 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062798+0000", 2024-09-14T06:32:13.111 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:13.111 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:13.111 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:13.112 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:32:13.112 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062798+0000", 2024-09-14T06:32:13.112 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.37e-07 2024-09-14T06:32:13.112 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:13.112 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:13.112 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:32:13.112 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062800+0000", 2024-09-14T06:32:13.112 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.0269999999999998e-06 2024-09-14T06:32:13.112 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:13.112 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:13.112 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:32:13.112 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062806+0000", 2024-09-14T06:32:13.112 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.6690000000000003e-06 2024-09-14T06:32:13.112 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:13.112 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:13.112 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:32:13.113 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062810+0000", 2024-09-14T06:32:13.113 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.3449999999999999e-06 2024-09-14T06:32:13.113 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:13.113 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:13.113 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:32:13.113 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.108892+0000", 2024-09-14T06:32:13.113 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046081715000000002 2024-09-14T06:32:13.113 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:13.113 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:13.113 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:32:13.113 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.108940+0000", 2024-09-14T06:32:13.113 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.7815999999999998e-05 2024-09-14T06:32:13.113 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:13.113 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:13.113 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:32:13.114 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.109053+0000", 2024-09-14T06:32:13.114 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.000113791 2024-09-14T06:32:13.114 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:13.114 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:32:13.114 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:13.114 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:13.114 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:13.114 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:19353 3.3 3:c34e5e51:::rbd_data.112d92027590.0000000000000024:f [read 1179648~1024 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:32:13.114 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:27.062816+0000", 2024-09-14T06:32:13.114 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 166.03302203800001, 2024-09-14T06:32:13.114 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046481256999999998, 2024-09-14T06:32:13.114 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:32:13.114 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:32:13.114 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:32:13.114 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:32:13.115 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:32:13.115 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:13.115 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 19353 2024-09-14T06:32:13.115 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:13.115 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:32:13.115 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:13.115 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:32:13.115 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062816+0000", 2024-09-14T06:32:13.115 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:13.115 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:13.115 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:13.115 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:32:13.115 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062816+0000", 2024-09-14T06:32:13.115 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:13.115 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:13.115 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:13.116 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:32:13.116 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062816+0000", 2024-09-14T06:32:13.116 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.2899999999999999e-07 2024-09-14T06:32:13.116 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:13.116 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:13.116 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:32:13.116 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062818+0000", 2024-09-14T06:32:13.116 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.699e-06 2024-09-14T06:32:13.116 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:13.116 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:13.116 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:32:13.116 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062825+0000", 2024-09-14T06:32:13.116 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.878e-06 2024-09-14T06:32:13.116 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:13.116 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:13.117 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:32:13.117 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062830+0000", 2024-09-14T06:32:13.117 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.5410000000000002e-06 2024-09-14T06:32:13.117 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:13.117 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:13.117 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:32:13.117 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.109096+0000", 2024-09-14T06:32:13.117 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046266517 2024-09-14T06:32:13.117 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:13.117 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:13.117 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:32:13.117 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.109190+0000", 2024-09-14T06:32:13.117 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.4000999999999997e-05 2024-09-14T06:32:13.117 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:13.117 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:13.118 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:32:13.118 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.109297+0000", 2024-09-14T06:32:13.118 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.000107292 2024-09-14T06:32:13.118 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:13.118 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:32:13.118 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:13.118 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:13.118 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:13.118 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:19354 3.3 3:c34e5e51:::rbd_data.112d92027590.0000000000000024:f [read 1189888~1024 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:32:13.118 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:27.062842+0000", 2024-09-14T06:32:13.118 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 166.032996583, 2024-09-14T06:32:13.118 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046594257, 2024-09-14T06:32:13.118 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:32:13.118 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:32:13.118 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:32:13.118 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:32:13.119 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:32:13.119 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:13.119 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 19354 2024-09-14T06:32:13.119 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:13.119 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:32:13.119 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:13.119 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:32:13.119 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062842+0000", 2024-09-14T06:32:13.119 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:13.119 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:13.119 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:13.119 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:32:13.119 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062842+0000", 2024-09-14T06:32:13.119 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:13.119 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:13.119 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:13.120 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:32:13.120 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062842+0000", 2024-09-14T06:32:13.120 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.7399999999999999e-07 2024-09-14T06:32:13.120 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:13.120 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:13.120 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:32:13.120 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062859+0000", 2024-09-14T06:32:13.120 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.7010000000000001e-05 2024-09-14T06:32:13.120 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:13.120 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:13.120 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:32:13.120 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062866+0000", 2024-09-14T06:32:13.120 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.9990000000000002e-06 2024-09-14T06:32:13.120 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:13.120 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:13.121 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:32:13.121 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062873+0000", 2024-09-14T06:32:13.121 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.8569999999999997e-06 2024-09-14T06:32:13.121 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:13.121 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:13.121 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:32:13.121 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.109335+0000", 2024-09-14T06:32:13.121 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046461943999999998 2024-09-14T06:32:13.121 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:13.121 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:13.121 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:32:13.121 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.109377+0000", 2024-09-14T06:32:13.121 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.2558999999999997e-05 2024-09-14T06:32:13.121 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:13.121 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:13.121 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:32:13.122 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.109436+0000", 2024-09-14T06:32:13.122 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.8514000000000003e-05 2024-09-14T06:32:13.122 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:13.122 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:32:13.122 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:13.122 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:13.122 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:13.122 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:19355 3.3 3:c34e5e51:::rbd_data.112d92027590.0000000000000024:f [read 1200128~1024 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:32:13.122 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:27.062884+0000", 2024-09-14T06:32:13.122 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 166.03295391, 2024-09-14T06:32:13.122 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046681650999999998, 2024-09-14T06:32:13.122 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:32:13.122 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:32:13.122 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:32:13.122 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:32:13.123 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:32:13.123 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:13.123 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 19355 2024-09-14T06:32:13.123 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:13.123 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:32:13.123 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:13.123 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:32:13.123 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062884+0000", 2024-09-14T06:32:13.123 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:13.123 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:13.123 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:13.123 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:32:13.123 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062884+0000", 2024-09-14T06:32:13.123 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:13.123 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:13.124 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:13.124 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:32:13.124 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062885+0000", 2024-09-14T06:32:13.124 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.8000000000000001e-07 2024-09-14T06:32:13.124 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:13.124 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:13.124 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:32:13.124 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062886+0000", 2024-09-14T06:32:13.124 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.8920000000000001e-06 2024-09-14T06:32:13.124 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:13.124 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:13.124 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:32:13.124 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062892+0000", 2024-09-14T06:32:13.124 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.84e-06 2024-09-14T06:32:13.124 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:13.124 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:13.125 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:32:13.125 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062899+0000", 2024-09-14T06:32:13.125 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.5339999999999996e-06 2024-09-14T06:32:13.125 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:13.125 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:13.125 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:32:13.125 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.109466+0000", 2024-09-14T06:32:13.125 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046566919999999998 2024-09-14T06:32:13.125 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:13.125 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:13.125 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:32:13.125 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.109510+0000", 2024-09-14T06:32:13.125 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.4153000000000002e-05 2024-09-14T06:32:13.125 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:13.125 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:13.125 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:32:13.126 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.109566+0000", 2024-09-14T06:32:13.126 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.5931999999999999e-05 2024-09-14T06:32:13.126 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:13.126 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:32:13.126 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:13.126 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:13.126 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:13.126 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:19356 3.3 3:c34e5e51:::rbd_data.112d92027590.0000000000000024:f [read 1210368~1024 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:32:13.126 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:27.062908+0000", 2024-09-14T06:32:13.126 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 166.03293047299999, 2024-09-14T06:32:13.126 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046807498000000003, 2024-09-14T06:32:13.126 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:32:13.126 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:32:13.126 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:32:13.127 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:32:13.127 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:32:13.127 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:13.127 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 19356 2024-09-14T06:32:13.127 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:13.127 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:32:13.127 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:13.127 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:32:13.127 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062908+0000", 2024-09-14T06:32:13.127 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:13.127 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:13.127 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:13.127 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:32:13.127 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062908+0000", 2024-09-14T06:32:13.127 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:13.127 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:13.128 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:13.128 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:32:13.128 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062908+0000", 2024-09-14T06:32:13.128 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.5699999999999998e-07 2024-09-14T06:32:13.128 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:13.128 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:13.128 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:32:13.128 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062910+0000", 2024-09-14T06:32:13.128 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.7710000000000001e-06 2024-09-14T06:32:13.128 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:13.128 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:13.128 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:32:13.128 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062920+0000", 2024-09-14T06:32:13.128 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.0654e-05 2024-09-14T06:32:13.128 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:13.129 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:13.129 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:32:13.129 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062931+0000", 2024-09-14T06:32:13.129 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.075e-05 2024-09-14T06:32:13.129 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:13.129 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:13.129 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:32:13.129 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.109595+0000", 2024-09-14T06:32:13.129 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046663713000000002 2024-09-14T06:32:13.129 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:13.129 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:13.129 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:32:13.129 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.109632+0000", 2024-09-14T06:32:13.129 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.7505000000000001e-05 2024-09-14T06:32:13.129 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:13.129 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:13.130 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:32:13.130 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.109715+0000", 2024-09-14T06:32:13.130 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.2748000000000001e-05 2024-09-14T06:32:13.130 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:13.130 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:32:13.130 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:13.130 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:13.130 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:13.130 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:19357 3.3 3:c34e5e51:::rbd_data.112d92027590.0000000000000024:f [read 1220608~1024 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:32:13.130 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:27.062939+0000", 2024-09-14T06:32:13.130 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 166.03289902500001, 2024-09-14T06:32:13.130 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046920508, 2024-09-14T06:32:13.130 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:32:13.131 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:32:13.131 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:32:13.131 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:32:13.131 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:32:13.131 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:13.131 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 19357 2024-09-14T06:32:13.131 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:13.131 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:32:13.131 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:13.131 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:32:13.131 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062939+0000", 2024-09-14T06:32:13.131 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:13.131 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:13.131 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:13.131 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:32:13.132 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062939+0000", 2024-09-14T06:32:13.132 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:13.132 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:13.132 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:13.132 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:32:13.132 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062940+0000", 2024-09-14T06:32:13.132 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.32e-07 2024-09-14T06:32:13.132 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:13.132 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:13.132 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:32:13.132 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062942+0000", 2024-09-14T06:32:13.132 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.1919999999999999e-06 2024-09-14T06:32:13.132 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:13.132 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:13.132 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:32:13.133 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062948+0000", 2024-09-14T06:32:13.133 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.1149999999999999e-06 2024-09-14T06:32:13.133 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:13.133 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:13.133 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:32:13.133 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062960+0000", 2024-09-14T06:32:13.133 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.1895e-05 2024-09-14T06:32:13.133 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:13.133 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:13.133 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:32:13.133 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.109704+0000", 2024-09-14T06:32:13.133 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046744584999999998 2024-09-14T06:32:13.133 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:13.133 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:13.133 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:32:13.133 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.109777+0000", 2024-09-14T06:32:13.134 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.2282999999999999e-05 2024-09-14T06:32:13.134 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:13.134 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:13.134 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:32:13.134 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.109860+0000", 2024-09-14T06:32:13.134 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.3005999999999998e-05 2024-09-14T06:32:13.134 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:13.134 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:32:13.134 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:13.134 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:13.134 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:13.134 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:19358 3.3 3:c34e5e51:::rbd_data.112d92027590.0000000000000024:f [read 1230848~1024 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:32:13.134 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:27.062967+0000", 2024-09-14T06:32:13.134 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 166.032871395, 2024-09-14T06:32:13.134 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047144471, 2024-09-14T06:32:13.134 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:32:13.135 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:32:13.135 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:32:13.135 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:32:13.135 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:32:13.135 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:13.135 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 19358 2024-09-14T06:32:13.135 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:13.135 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:32:13.135 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:13.135 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:32:13.135 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062967+0000", 2024-09-14T06:32:13.135 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:13.135 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:13.136 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:13.136 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:32:13.136 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062967+0000", 2024-09-14T06:32:13.136 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:13.136 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:13.136 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:13.136 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:32:13.136 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062967+0000", 2024-09-14T06:32:13.136 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.89e-07 2024-09-14T06:32:13.136 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:13.136 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:13.136 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:32:13.136 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062969+0000", 2024-09-14T06:32:13.136 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.9929999999999998e-06 2024-09-14T06:32:13.136 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:13.136 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:13.137 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:32:13.137 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062975+0000", 2024-09-14T06:32:13.137 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.9030000000000001e-06 2024-09-14T06:32:13.137 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:13.137 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:13.137 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:32:13.137 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062979+0000", 2024-09-14T06:32:13.137 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.2250000000000002e-06 2024-09-14T06:32:13.137 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:13.137 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:13.137 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:32:13.137 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.109934+0000", 2024-09-14T06:32:13.137 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046954410000000002 2024-09-14T06:32:13.138 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:13.138 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:13.138 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:32:13.138 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.110007+0000", 2024-09-14T06:32:13.138 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.3829000000000001e-05 2024-09-14T06:32:13.138 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:13.138 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:13.138 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:32:13.138 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.110111+0000", 2024-09-14T06:32:13.138 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00010372199999999999 2024-09-14T06:32:13.138 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:13.138 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:32:13.138 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:13.138 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:13.138 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:13.139 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:19359 3.3 3:c34e5e51:::rbd_data.112d92027590.0000000000000024:f [read 1241088~1024 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:32:13.139 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:27.062987+0000", 2024-09-14T06:32:13.139 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 166.03285130500001, 2024-09-14T06:32:13.139 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047297182, 2024-09-14T06:32:13.139 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:32:13.139 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:32:13.139 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:32:13.139 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:32:13.139 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:32:13.139 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:13.139 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 19359 2024-09-14T06:32:13.139 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:13.139 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:32:13.139 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:13.139 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:32:13.140 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062987+0000", 2024-09-14T06:32:13.140 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:13.140 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:13.140 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:13.140 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:32:13.140 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062987+0000", 2024-09-14T06:32:13.140 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:13.140 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:13.140 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:13.140 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:32:13.140 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062987+0000", 2024-09-14T06:32:13.140 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.2500000000000001e-07 2024-09-14T06:32:13.140 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:13.140 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:13.141 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:32:13.141 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062989+0000", 2024-09-14T06:32:13.141 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.6080000000000001e-06 2024-09-14T06:32:13.141 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:13.141 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:13.141 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:32:13.141 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062994+0000", 2024-09-14T06:32:13.141 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.5520000000000004e-06 2024-09-14T06:32:13.141 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:13.141 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:13.141 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:32:13.141 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062999+0000", 2024-09-14T06:32:13.141 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.3930000000000001e-06 2024-09-14T06:32:13.141 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:13.141 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:13.142 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:32:13.142 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.110142+0000", 2024-09-14T06:32:13.142 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047143154 2024-09-14T06:32:13.142 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:13.142 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:13.142 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:32:13.142 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.110219+0000", 2024-09-14T06:32:13.142 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.6737999999999996e-05 2024-09-14T06:32:13.142 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:13.142 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:13.142 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:32:13.142 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.110284+0000", 2024-09-14T06:32:13.142 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.5412000000000003e-05 2024-09-14T06:32:13.142 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:13.142 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:32:13.142 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:13.143 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:13.143 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:13.143 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:19360 3.3 3:c34e5e51:::rbd_data.112d92027590.0000000000000024:f [read 1251328~1024 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:32:13.143 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:27.063017+0000", 2024-09-14T06:32:13.143 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 166.03282084400001, 2024-09-14T06:32:13.143 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047453823999999999, 2024-09-14T06:32:13.143 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:32:13.143 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:32:13.143 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:32:13.143 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:32:13.143 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:32:13.143 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:13.143 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 19360 2024-09-14T06:32:13.143 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:13.143 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:32:13.144 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:13.144 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:32:13.144 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.063017+0000", 2024-09-14T06:32:13.144 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:13.144 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:13.144 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:13.144 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:32:13.144 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.063017+0000", 2024-09-14T06:32:13.144 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:13.144 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:13.144 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:13.144 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:32:13.144 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.064977+0000", 2024-09-14T06:32:13.144 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.0019597080000000001 2024-09-14T06:32:13.144 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:13.144 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:13.145 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:32:13.145 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.064987+0000", 2024-09-14T06:32:13.145 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.9620000000000005e-06 2024-09-14T06:32:13.145 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:13.145 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:13.145 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:32:13.145 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065000+0000", 2024-09-14T06:32:13.145 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.3173000000000001e-05 2024-09-14T06:32:13.145 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:13.145 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:13.145 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:32:13.145 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065005+0000", 2024-09-14T06:32:13.146 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.9960000000000001e-06 2024-09-14T06:32:13.146 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:13.146 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:13.146 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:32:13.146 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.110314+0000", 2024-09-14T06:32:13.146 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.04530899 2024-09-14T06:32:13.146 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:13.146 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:13.146 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:32:13.146 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.110398+0000", 2024-09-14T06:32:13.146 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.3539000000000001e-05 2024-09-14T06:32:13.146 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:13.146 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:13.146 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:32:13.146 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.110471+0000", 2024-09-14T06:32:13.147 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.3455999999999996e-05 2024-09-14T06:32:13.147 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:13.147 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:32:13.147 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:13.147 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:13.147 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:13.147 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:19365 3.3 3:c34e5e51:::rbd_data.112d92027590.0000000000000024:f [read 1302528~1024 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:32:13.147 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:27.065058+0000", 2024-09-14T06:32:13.147 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 166.030780174, 2024-09-14T06:32:13.147 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046105526000000001, 2024-09-14T06:32:13.147 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:32:13.147 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:32:13.147 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:32:13.147 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:32:13.147 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:32:13.148 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:13.148 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 19365 2024-09-14T06:32:13.148 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:13.148 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:32:13.148 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:13.148 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:32:13.148 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065058+0000", 2024-09-14T06:32:13.148 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:13.148 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:13.148 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:13.148 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:32:13.148 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065058+0000", 2024-09-14T06:32:13.148 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:13.148 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:13.148 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:13.149 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:32:13.149 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065058+0000", 2024-09-14T06:32:13.149 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.1899999999999999e-07 2024-09-14T06:32:13.149 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:13.149 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:13.149 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:32:13.149 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065059+0000", 2024-09-14T06:32:13.149 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.1299999999999998e-07 2024-09-14T06:32:13.149 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:13.149 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:13.149 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:32:13.149 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065063+0000", 2024-09-14T06:32:13.149 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.1099999999999996e-06 2024-09-14T06:32:13.149 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:13.150 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:13.150 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:32:13.150 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065070+0000", 2024-09-14T06:32:13.150 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.973e-06 2024-09-14T06:32:13.150 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:13.150 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:13.150 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:32:13.150 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.111014+0000", 2024-09-14T06:32:13.150 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.045943373000000003 2024-09-14T06:32:13.150 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:13.150 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:13.150 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:32:13.150 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.111073+0000", 2024-09-14T06:32:13.151 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.8987999999999998e-05 2024-09-14T06:32:13.151 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:13.151 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:13.151 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:32:13.151 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.111163+0000", 2024-09-14T06:32:13.151 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.0950000000000002e-05 2024-09-14T06:32:13.151 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:13.151 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:32:13.151 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:13.151 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:13.151 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:13.151 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:19366 3.3 3:c34e5e51:::rbd_data.112d92027590.0000000000000024:f [read 1312768~1024 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:32:13.151 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:27.065074+0000", 2024-09-14T06:32:13.151 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 166.03076432399999, 2024-09-14T06:32:13.152 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046289858000000003, 2024-09-14T06:32:13.152 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:32:13.152 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:32:13.152 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:32:13.152 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:32:13.152 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:32:13.152 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:13.152 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 19366 2024-09-14T06:32:13.152 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:13.152 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:32:13.152 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:13.153 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:32:13.153 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065074+0000", 2024-09-14T06:32:13.153 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:13.153 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:13.153 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:13.153 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:32:13.153 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065074+0000", 2024-09-14T06:32:13.153 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:13.153 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:13.153 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:13.153 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:32:13.153 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065074+0000", 2024-09-14T06:32:13.153 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.3900000000000001e-07 2024-09-14T06:32:13.153 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:13.153 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:13.154 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:32:13.154 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065078+0000", 2024-09-14T06:32:13.154 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.3229999999999999e-06 2024-09-14T06:32:13.154 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:13.154 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:13.154 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:32:13.154 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065082+0000", 2024-09-14T06:32:13.154 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.8299999999999998e-06 2024-09-14T06:32:13.154 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:13.154 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:13.154 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:32:13.154 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065085+0000", 2024-09-14T06:32:13.154 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.8660000000000002e-06 2024-09-14T06:32:13.154 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:13.154 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:13.155 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:32:13.155 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.111270+0000", 2024-09-14T06:32:13.155 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046185035999999999 2024-09-14T06:32:13.155 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:13.155 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:13.155 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:32:13.155 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.111311+0000", 2024-09-14T06:32:13.155 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.0935e-05 2024-09-14T06:32:13.155 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:13.155 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:13.155 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:32:13.155 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.111364+0000", 2024-09-14T06:32:13.155 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.2629e-05 2024-09-14T06:32:13.155 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:13.155 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:32:13.155 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:13.156 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:13.156 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:13.156 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:19367 3.3 3:c34e5e51:::rbd_data.112d92027590.0000000000000024:f [read 1323008~1024 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:32:13.156 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:27.065115+0000", 2024-09-14T06:32:13.156 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 166.030722763, 2024-09-14T06:32:13.156 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046407003000000002, 2024-09-14T06:32:13.156 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:32:13.156 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:32:13.156 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:32:13.156 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:32:13.156 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:32:13.156 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:13.156 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 19367 2024-09-14T06:32:13.156 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:13.156 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:32:13.157 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:13.157 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:32:13.157 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065115+0000", 2024-09-14T06:32:13.157 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:13.157 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:13.157 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:13.157 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:32:13.157 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065115+0000", 2024-09-14T06:32:13.157 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:13.157 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:13.157 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:13.157 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:32:13.157 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065116+0000", 2024-09-14T06:32:13.157 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.4999999999999998e-07 2024-09-14T06:32:13.157 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:13.158 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:13.158 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:32:13.158 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065117+0000", 2024-09-14T06:32:13.158 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.189e-06 2024-09-14T06:32:13.158 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:13.158 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:13.158 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:32:13.158 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065122+0000", 2024-09-14T06:32:13.158 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.3709999999999999e-06 2024-09-14T06:32:13.158 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:13.158 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:13.158 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:32:13.158 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065125+0000", 2024-09-14T06:32:13.158 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.8940000000000002e-06 2024-09-14T06:32:13.159 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:13.159 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:13.159 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:32:13.159 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.111402+0000", 2024-09-14T06:32:13.159 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046276645999999998 2024-09-14T06:32:13.159 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:13.159 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:13.159 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:32:13.159 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.111445+0000", 2024-09-14T06:32:13.159 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.3288000000000003e-05 2024-09-14T06:32:13.159 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:13.159 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:13.159 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:32:13.159 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.111522+0000", 2024-09-14T06:32:13.159 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.7164999999999997e-05 2024-09-14T06:32:13.160 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:13.160 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:32:13.160 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:13.160 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:13.160 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:13.160 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:19368 3.3 3:c34e5e51:::rbd_data.112d92027590.0000000000000024:f [read 1333248~1024 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:32:13.160 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:27.065130+0000", 2024-09-14T06:32:13.160 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 166.030708473, 2024-09-14T06:32:13.160 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046507275000000001, 2024-09-14T06:32:13.160 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:32:13.160 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:32:13.160 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:32:13.160 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:32:13.160 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:32:13.161 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:13.161 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 19368 2024-09-14T06:32:13.161 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:13.161 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:32:13.161 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:13.161 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:32:13.161 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065130+0000", 2024-09-14T06:32:13.161 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:13.161 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:13.161 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:13.161 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:32:13.161 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065130+0000", 2024-09-14T06:32:13.161 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:13.161 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:13.161 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:13.162 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:32:13.162 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065130+0000", 2024-09-14T06:32:13.162 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.0800000000000001e-07 2024-09-14T06:32:13.162 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:13.162 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:13.162 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:32:13.162 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065131+0000", 2024-09-14T06:32:13.162 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.8700000000000004e-07 2024-09-14T06:32:13.162 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:13.162 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:13.162 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:32:13.162 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065134+0000", 2024-09-14T06:32:13.162 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.1039999999999998e-06 2024-09-14T06:32:13.162 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:13.162 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:13.163 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:32:13.163 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065136+0000", 2024-09-14T06:32:13.163 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.3949999999999999e-06 2024-09-14T06:32:13.163 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:13.163 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:13.163 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:32:13.163 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.111541+0000", 2024-09-14T06:32:13.163 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046404206000000003 2024-09-14T06:32:13.163 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:13.163 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:13.163 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:32:13.163 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.111582+0000", 2024-09-14T06:32:13.163 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.1051999999999999e-05 2024-09-14T06:32:13.163 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:13.163 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:13.164 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:32:13.164 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.111637+0000", 2024-09-14T06:32:13.164 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.5322999999999999e-05 2024-09-14T06:32:13.164 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:13.164 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:32:13.164 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:13.164 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:13.164 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:13.164 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:19369 3.3 3:c34e5e51:::rbd_data.112d92027590.0000000000000024:f [read 1343488~1024 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:32:13.164 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:27.065139+0000", 2024-09-14T06:32:13.164 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 166.030698656, 2024-09-14T06:32:13.164 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046682169000000003, 2024-09-14T06:32:13.164 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:32:13.164 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:32:13.164 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:32:13.165 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:32:13.165 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:32:13.165 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:13.165 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 19369 2024-09-14T06:32:13.165 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:13.165 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:32:13.165 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:13.165 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:32:13.165 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065139+0000", 2024-09-14T06:32:13.165 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:13.165 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:13.165 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:13.165 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:32:13.166 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065139+0000", 2024-09-14T06:32:13.166 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:13.166 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:13.166 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:13.166 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:32:13.166 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065140+0000", 2024-09-14T06:32:13.166 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.8299999999999998e-07 2024-09-14T06:32:13.166 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:13.166 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:13.166 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:32:13.166 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065141+0000", 2024-09-14T06:32:13.166 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.3900000000000003e-07 2024-09-14T06:32:13.166 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:13.166 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:13.166 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:32:13.167 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065144+0000", 2024-09-14T06:32:13.167 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.7409999999999998e-06 2024-09-14T06:32:13.167 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:13.167 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:13.167 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:32:13.167 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065147+0000", 2024-09-14T06:32:13.167 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.4150000000000002e-06 2024-09-14T06:32:13.167 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:13.167 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:13.167 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:32:13.167 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.111673+0000", 2024-09-14T06:32:13.167 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046526004000000003 2024-09-14T06:32:13.167 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:13.167 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:13.168 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:32:13.168 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.111749+0000", 2024-09-14T06:32:13.168 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.5741999999999994e-05 2024-09-14T06:32:13.168 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:13.168 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:13.168 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:32:13.168 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.111822+0000", 2024-09-14T06:32:13.168 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.3045000000000003e-05 2024-09-14T06:32:13.168 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:13.168 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:32:13.168 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:13.168 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:13.168 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:13.168 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:19370 3.3 3:c34e5e51:::rbd_data.112d92027590.0000000000000024:f [read 1353728~1024 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:32:13.168 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:27.065150+0000", 2024-09-14T06:32:13.169 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 166.030687927, 2024-09-14T06:32:13.169 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046897556999999999, 2024-09-14T06:32:13.169 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:32:13.169 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:32:13.169 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:32:13.169 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:32:13.169 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:32:13.169 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:13.169 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 19370 2024-09-14T06:32:13.169 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:13.169 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:32:13.169 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:13.169 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:32:13.169 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065150+0000", 2024-09-14T06:32:13.169 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:13.170 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:13.170 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:13.170 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:32:13.170 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065150+0000", 2024-09-14T06:32:13.170 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:13.170 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:13.170 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:13.170 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:32:13.170 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065150+0000", 2024-09-14T06:32:13.170 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.0200000000000001e-07 2024-09-14T06:32:13.170 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:13.170 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:13.170 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:32:13.170 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065151+0000", 2024-09-14T06:32:13.170 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.1100000000000004e-07 2024-09-14T06:32:13.171 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:13.171 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:13.171 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:32:13.171 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065154+0000", 2024-09-14T06:32:13.171 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.9639999999999999e-06 2024-09-14T06:32:13.171 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:13.171 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:13.171 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:32:13.171 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065157+0000", 2024-09-14T06:32:13.171 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.4260000000000002e-06 2024-09-14T06:32:13.171 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:13.171 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:13.171 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:32:13.171 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.111861+0000", 2024-09-14T06:32:13.171 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046704268 2024-09-14T06:32:13.171 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:13.172 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:13.172 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:32:13.172 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.111939+0000", 2024-09-14T06:32:13.172 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.8231000000000004e-05 2024-09-14T06:32:13.172 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:13.172 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:13.172 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:32:13.172 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.112048+0000", 2024-09-14T06:32:13.172 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00010855500000000001 2024-09-14T06:32:13.172 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:13.172 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:32:13.172 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:13.172 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:13.172 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:13.172 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:19371 3.3 3:c34e5e51:::rbd_data.112d92027590.0000000000000024:f [read 1363968~1024 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:32:13.173 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:27.065161+0000", 2024-09-14T06:32:13.173 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 166.030677277, 2024-09-14T06:32:13.173 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047021944000000003, 2024-09-14T06:32:13.173 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:32:13.173 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:32:13.173 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:32:13.173 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:32:13.173 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:32:13.173 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:13.173 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 19371 2024-09-14T06:32:13.173 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:13.173 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:32:13.173 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:13.173 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:32:13.173 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065161+0000", 2024-09-14T06:32:13.174 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:13.174 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:13.174 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:13.174 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:32:13.174 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065161+0000", 2024-09-14T06:32:13.174 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:13.174 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:13.174 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:13.174 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:32:13.174 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065161+0000", 2024-09-14T06:32:13.174 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.0200000000000001e-07 2024-09-14T06:32:13.174 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:13.174 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:13.174 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:32:13.174 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065166+0000", 2024-09-14T06:32:13.175 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.9869999999999999e-06 2024-09-14T06:32:13.175 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:13.175 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:13.175 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:32:13.175 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065169+0000", 2024-09-14T06:32:13.175 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.3799999999999998e-06 2024-09-14T06:32:13.175 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:13.175 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:13.175 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:32:13.175 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065172+0000", 2024-09-14T06:32:13.175 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.4830000000000002e-06 2024-09-14T06:32:13.175 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:13.175 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:13.175 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:32:13.176 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.112071+0000", 2024-09-14T06:32:13.176 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046899529000000002 2024-09-14T06:32:13.176 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:13.176 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:13.176 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:32:13.176 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.112128+0000", 2024-09-14T06:32:13.176 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.6341999999999997e-05 2024-09-14T06:32:13.176 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:13.176 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:13.176 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:32:13.176 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.112183+0000", 2024-09-14T06:32:13.176 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.5021000000000001e-05 2024-09-14T06:32:13.176 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:13.176 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:32:13.176 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:13.177 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:13.177 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:13.177 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:19372 3.3 3:c34e5e51:::rbd_data.112d92027590.0000000000000024:f [read 1374208~1024 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:32:13.177 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:27.065176+0000", 2024-09-14T06:32:13.177 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 166.030662127, 2024-09-14T06:32:13.177 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047188889999999997, 2024-09-14T06:32:13.177 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:32:13.177 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:32:13.177 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:32:13.177 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:32:13.177 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:32:13.177 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:13.177 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 19372 2024-09-14T06:32:13.177 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:13.178 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:32:13.178 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:13.178 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:32:13.178 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065176+0000", 2024-09-14T06:32:13.178 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:13.178 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:13.178 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:13.178 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:32:13.178 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065176+0000", 2024-09-14T06:32:13.178 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:13.178 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:13.178 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:13.178 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:32:13.178 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.071117+0000", 2024-09-14T06:32:13.178 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.0059404990000000001 2024-09-14T06:32:13.178 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:13.179 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:13.179 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:32:13.179 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.071120+0000", 2024-09-14T06:32:13.179 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.4340000000000001e-06 2024-09-14T06:32:13.179 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:13.179 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:13.179 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:32:13.179 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.071129+0000", 2024-09-14T06:32:13.179 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.8389999999999996e-06 2024-09-14T06:32:13.179 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:13.179 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:13.179 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:32:13.179 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.071134+0000", 2024-09-14T06:32:13.179 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.412e-06 2024-09-14T06:32:13.179 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:13.180 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:13.180 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:32:13.180 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.112221+0000", 2024-09-14T06:32:13.180 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.041086473999999998 2024-09-14T06:32:13.180 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:13.180 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:13.180 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:32:13.180 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.112299+0000", 2024-09-14T06:32:13.180 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.8746999999999997e-05 2024-09-14T06:32:13.180 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:13.180 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:13.180 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:32:13.180 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.112365+0000", 2024-09-14T06:32:13.180 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.5485000000000001e-05 2024-09-14T06:32:13.180 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:13.180 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:32:13.181 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:13.181 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:13.181 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:32:13.181 INFO:teuthology.orchestra.run.smithi138.stdout:} 2024-09-14T06:32:13.182 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-14T06:32:13.183 INFO:tasks.ceph.osd.0.smithi094.stderr:2024-09-14T06:32:13.181+0000 7f07c9333640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33636) UID: 0 2024-09-14T06:32:13.199 INFO:teuthology.orchestra.run.smithi094.stdout:osd.0: {} 2024-09-14T06:32:13.200 INFO:teuthology.orchestra.run.smithi094.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-14T06:32:13.216 INFO:teuthology.orchestra.run.smithi094.stdout:osd.1: {} 2024-09-14T06:32:13.216 INFO:teuthology.orchestra.run.smithi094.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-14T06:32:13.231 INFO:teuthology.orchestra.run.smithi094.stdout:osd.2: {} 2024-09-14T06:32:13.232 INFO:teuthology.orchestra.run.smithi094.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-14T06:32:13.247 INFO:teuthology.orchestra.run.smithi094.stdout:osd.3: {} 2024-09-14T06:32:13.247 INFO:teuthology.orchestra.run.smithi094.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-14T06:32:13.265 INFO:teuthology.orchestra.run.smithi094.stdout:osd.4: {} 2024-09-14T06:32:13.265 INFO:teuthology.orchestra.run.smithi094.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-14T06:32:13.282 INFO:teuthology.orchestra.run.smithi094.stdout:osd.5: {} 2024-09-14T06:32:13.282 INFO:teuthology.orchestra.run.smithi094.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-14T06:32:13.282 INFO:tasks.ceph.osd.1.smithi094.stderr:2024-09-14T06:32:13.281+0000 7f328e47a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33641) UID: 0 2024-09-14T06:32:13.286 INFO:teuthology.orchestra.run.smithi138.stdout:ERROR: (22) Invalid argument 2024-09-14T06:32:13.286 INFO:teuthology.orchestra.run.smithi138.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:32:13.295 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-14T06:32:13.298 INFO:teuthology.orchestra.run.smithi094.stdout:osd.6: {} 2024-09-14T06:32:13.298 INFO:teuthology.orchestra.run.smithi094.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-14T06:32:13.314 INFO:teuthology.orchestra.run.smithi094.stdout:osd.7: {} 2024-09-14T06:32:13.315 INFO:teuthology.orchestra.run.smithi094.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-14T06:32:13.384 INFO:tasks.ceph.osd.0.smithi094.stderr:2024-09-14T06:32:13.382+0000 7f07c9333640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33636) UID: 0 2024-09-14T06:32:13.397 INFO:teuthology.orchestra.run.smithi138.stdout:ERROR: (22) Invalid argument 2024-09-14T06:32:13.397 INFO:teuthology.orchestra.run.smithi138.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:32:13.406 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-14T06:32:13.484 INFO:tasks.ceph.osd.4.smithi138.stderr:2024-09-14T06:32:13.482+0000 7f6c653c8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33615) UID: 0 2024-09-14T06:32:13.508 INFO:teuthology.orchestra.run.smithi138.stdout:ERROR: (22) Invalid argument 2024-09-14T06:32:13.508 INFO:teuthology.orchestra.run.smithi138.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:32:13.517 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-14T06:32:13.601 INFO:tasks.ceph.osd.7.smithi138.stderr:2024-09-14T06:32:13.599+0000 7f3c6a21d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33623) UID: 0 2024-09-14T06:32:13.660 INFO:teuthology.orchestra.run.smithi094.stderr:reweighted osd.2 to 1 (10000) 2024-09-14T06:32:13.663 INFO:teuthology.orchestra.run.smithi138.stdout:ERROR: (22) Invalid argument 2024-09-14T06:32:13.663 INFO:teuthology.orchestra.run.smithi138.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:32:13.673 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-14T06:32:13.677 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 4 1 2024-09-14T06:32:13.718 INFO:tasks.ceph.osd.3.smithi094.stderr:2024-09-14T06:32:13.717+0000 7feeeafe8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33655) UID: 0 2024-09-14T06:32:13.786 INFO:tasks.ceph.osd.0.smithi094.stderr:2024-09-14T06:32:13.785+0000 7f07c9333640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33636) UID: 0 2024-09-14T06:32:13.822 INFO:teuthology.orchestra.run.smithi138.stdout:ERROR: (22) Invalid argument 2024-09-14T06:32:13.822 INFO:teuthology.orchestra.run.smithi138.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:32:13.831 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-14T06:32:13.887 INFO:tasks.ceph.osd.7.smithi138.stderr:2024-09-14T06:32:13.885+0000 7f3c6a21d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33623) UID: 0 2024-09-14T06:32:13.933 INFO:teuthology.orchestra.run.smithi138.stdout:ERROR: (22) Invalid argument 2024-09-14T06:32:13.933 INFO:teuthology.orchestra.run.smithi138.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:32:13.942 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-14T06:32:13.988 INFO:tasks.ceph.osd.2.smithi094.stderr:2024-09-14T06:32:13.987+0000 7faf3269b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33649) UID: 0 2024-09-14T06:32:14.086 INFO:teuthology.orchestra.run.smithi138.stdout:ERROR: (22) Invalid argument 2024-09-14T06:32:14.086 INFO:teuthology.orchestra.run.smithi138.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:32:14.089 INFO:tasks.ceph.osd.7.smithi138.stderr:2024-09-14T06:32:14.087+0000 7f3c6a21d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33623) UID: 0 2024-09-14T06:32:14.095 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-14T06:32:14.190 INFO:tasks.ceph.osd.4.smithi138.stderr:2024-09-14T06:32:14.187+0000 7f6c653c8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33615) UID: 0 2024-09-14T06:32:14.237 INFO:teuthology.orchestra.run.smithi138.stdout:ERROR: (22) Invalid argument 2024-09-14T06:32:14.237 INFO:teuthology.orchestra.run.smithi138.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:32:14.248 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-14T06:32:14.291 INFO:tasks.ceph.osd.2.smithi094.stderr:2024-09-14T06:32:14.290+0000 7faf3269b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33649) UID: 0 2024-09-14T06:32:14.389 INFO:teuthology.orchestra.run.smithi138.stdout:ERROR: (22) Invalid argument 2024-09-14T06:32:14.390 INFO:teuthology.orchestra.run.smithi138.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:32:14.392 INFO:tasks.ceph.osd.4.smithi138.stderr:2024-09-14T06:32:14.390+0000 7f6c653c8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33615) UID: 0 2024-09-14T06:32:14.398 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-14T06:32:14.493 INFO:tasks.ceph.osd.3.smithi094.stderr:2024-09-14T06:32:14.492+0000 7feeeafe8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33655) UID: 0 2024-09-14T06:32:14.504 INFO:teuthology.orchestra.run.smithi094.stdout:ERROR: (22) Invalid argument 2024-09-14T06:32:14.504 INFO:teuthology.orchestra.run.smithi094.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:32:14.512 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-14T06:32:14.595 INFO:tasks.ceph.osd.7.smithi138.stderr:2024-09-14T06:32:14.593+0000 7f3c6a21d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33623) UID: 0 2024-09-14T06:32:14.657 INFO:teuthology.orchestra.run.smithi094.stdout:ERROR: (22) Invalid argument 2024-09-14T06:32:14.657 INFO:teuthology.orchestra.run.smithi094.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:32:14.663 INFO:teuthology.orchestra.run.smithi094.stderr:reweighted osd.4 to 1 (10000) 2024-09-14T06:32:14.666 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-14T06:32:14.677 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 5 1 2024-09-14T06:32:14.720 INFO:tasks.ceph.osd.0.smithi094.stderr:2024-09-14T06:32:14.719+0000 7f07c9333640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33636) UID: 0 2024-09-14T06:32:14.768 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-14T06:32:14.771 INFO:teuthology.orchestra.run.smithi094.stdout:ERROR: (22) Invalid argument 2024-09-14T06:32:14.772 INFO:teuthology.orchestra.run.smithi094.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:32:14.781 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-14T06:32:14.822 INFO:tasks.ceph.osd.2.smithi094.stderr:2024-09-14T06:32:14.821+0000 7faf3269b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33649) UID: 0 2024-09-14T06:32:14.898 INFO:tasks.ceph.osd.4.smithi138.stderr:2024-09-14T06:32:14.896+0000 7f6c653c8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33615) UID: 0 2024-09-14T06:32:14.929 INFO:teuthology.orchestra.run.smithi094.stdout:ERROR: (22) Invalid argument 2024-09-14T06:32:14.929 INFO:teuthology.orchestra.run.smithi094.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:32:14.938 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-14T06:32:14.999 INFO:tasks.ceph.osd.2.smithi094.stderr:2024-09-14T06:32:14.998+0000 7faf3269b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33649) UID: 0 2024-09-14T06:32:15.042 INFO:teuthology.orchestra.run.smithi094.stdout:ERROR: (22) Invalid argument 2024-09-14T06:32:15.042 INFO:teuthology.orchestra.run.smithi094.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:32:15.051 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-14T06:32:15.100 INFO:tasks.ceph.osd.6.smithi138.stderr:2024-09-14T06:32:15.099+0000 7f8cd8849640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33620) UID: 0 2024-09-14T06:32:15.194 INFO:teuthology.orchestra.run.smithi094.stdout:ERROR: (22) Invalid argument 2024-09-14T06:32:15.195 INFO:teuthology.orchestra.run.smithi094.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:32:15.201 INFO:tasks.ceph.osd.0.smithi094.stderr:2024-09-14T06:32:15.200+0000 7f07c9333640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33636) UID: 0 2024-09-14T06:32:15.204 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-14T06:32:15.302 INFO:tasks.ceph.osd.5.smithi138.stderr:2024-09-14T06:32:15.301+0000 7fe13746f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33618) UID: 0 2024-09-14T06:32:15.327 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-14T06:32:15.349 INFO:teuthology.orchestra.run.smithi094.stdout:ERROR: (22) Invalid argument 2024-09-14T06:32:15.349 INFO:teuthology.orchestra.run.smithi094.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:32:15.358 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-14T06:32:15.403 INFO:tasks.ceph.osd.7.smithi138.stderr:2024-09-14T06:32:15.401+0000 7f3c6a21d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33623) UID: 0 2024-09-14T06:32:15.452 INFO:teuthology.orchestra.run.smithi094.stdout:osd.0: {} 2024-09-14T06:32:15.452 INFO:teuthology.orchestra.run.smithi094.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-14T06:32:15.469 INFO:teuthology.orchestra.run.smithi094.stdout:osd.1: {} 2024-09-14T06:32:15.469 INFO:teuthology.orchestra.run.smithi094.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-14T06:32:15.486 INFO:teuthology.orchestra.run.smithi094.stdout:osd.2: {} 2024-09-14T06:32:15.487 INFO:teuthology.orchestra.run.smithi094.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-14T06:32:15.503 INFO:teuthology.orchestra.run.smithi094.stdout:osd.3: {} 2024-09-14T06:32:15.503 INFO:teuthology.orchestra.run.smithi094.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-14T06:32:15.503 INFO:tasks.ceph.osd.3.smithi094.stderr:2024-09-14T06:32:15.502+0000 7feeeafe8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33655) UID: 0 2024-09-14T06:32:15.509 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-14T06:32:15.510 INFO:teuthology.orchestra.run.smithi094.stdout: "ops": [], 2024-09-14T06:32:15.510 INFO:teuthology.orchestra.run.smithi094.stdout: "complaint_time": 30, 2024-09-14T06:32:15.510 INFO:teuthology.orchestra.run.smithi094.stdout: "num_blocked_ops": 0 2024-09-14T06:32:15.510 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-14T06:32:15.518 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-14T06:32:15.520 INFO:teuthology.orchestra.run.smithi094.stdout:osd.4: {} 2024-09-14T06:32:15.521 INFO:teuthology.orchestra.run.smithi094.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-14T06:32:15.538 INFO:teuthology.orchestra.run.smithi094.stdout:osd.5: {} 2024-09-14T06:32:15.538 INFO:teuthology.orchestra.run.smithi094.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-14T06:32:15.556 INFO:teuthology.orchestra.run.smithi094.stdout:osd.6: {} 2024-09-14T06:32:15.556 INFO:teuthology.orchestra.run.smithi094.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-14T06:32:15.573 INFO:teuthology.orchestra.run.smithi094.stdout:osd.7: {} 2024-09-14T06:32:15.573 INFO:teuthology.orchestra.run.smithi094.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-14T06:32:15.605 INFO:tasks.ceph.osd.5.smithi138.stderr:2024-09-14T06:32:15.603+0000 7fe13746f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33618) UID: 0 2024-09-14T06:32:15.605 INFO:teuthology.orchestra.run.smithi094.stderr:nodeep-scrub is set 2024-09-14T06:32:15.609 INFO:teuthology.orchestra.run.smithi094.stderr:reweighted osd.5 to 1 (10000) 2024-09-14T06:32:15.635 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 7 1 2024-09-14T06:32:15.642 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-14T06:32:15.642 INFO:teuthology.orchestra.run.smithi094.stdout: "size": 20, 2024-09-14T06:32:15.642 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 600, 2024-09-14T06:32:15.642 INFO:teuthology.orchestra.run.smithi094.stdout: "ops": [ 2024-09-14T06:32:15.642 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:15.642 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4397.0:45722 3.9 e64/23 3:91fed098:::rbd_data.112d5a737e72.0000000000000001:head v 64'61, mlcod=62'59)", 2024-09-14T06:32:15.642 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:43.864269+0000", 2024-09-14T06:32:15.642 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 151.77488889099999, 2024-09-14T06:32:15.642 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.003055224, 2024-09-14T06:32:15.642 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:32:15.643 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:32:15.643 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:32:15.643 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:32:15.643 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:15.643 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:32:15.643 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.864269+0000", 2024-09-14T06:32:15.643 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:15.643 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:15.643 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:15.643 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:32:15.643 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.864269+0000", 2024-09-14T06:32:15.643 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:15.643 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:15.643 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:15.643 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:32:15.644 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.864269+0000", 2024-09-14T06:32:15.644 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.41e-07 2024-09-14T06:32:15.644 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:15.644 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:15.644 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:32:15.644 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.865490+0000", 2024-09-14T06:32:15.644 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.001220552 2024-09-14T06:32:15.644 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:15.645 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:15.645 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:32:15.645 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.865506+0000", 2024-09-14T06:32:15.645 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.6263999999999999e-05 2024-09-14T06:32:15.645 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:15.645 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:15.645 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:32:15.645 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.865516+0000", 2024-09-14T06:32:15.645 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.0091999999999999e-05 2024-09-14T06:32:15.645 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:15.645 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:15.645 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:32:15.645 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.865583+0000", 2024-09-14T06:32:15.646 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.6676999999999999e-05 2024-09-14T06:32:15.646 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:15.646 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:15.646 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:32:15.646 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.865610+0000", 2024-09-14T06:32:15.646 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.6529999999999998e-05 2024-09-14T06:32:15.646 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:15.646 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:15.646 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-14T06:32:15.646 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.867281+0000", 2024-09-14T06:32:15.646 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.001671203 2024-09-14T06:32:15.646 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:15.646 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:15.647 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:32:15.647 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.867324+0000", 2024-09-14T06:32:15.647 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.3665000000000003e-05 2024-09-14T06:32:15.647 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:15.647 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:32:15.647 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:15.647 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:15.647 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:15.647 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4397.0:74241 3.0 e73/23 3:06e93fb9:::rbd_data.112d80ca42af.0000000000000000:head v 73'102, mlcod=71'100)", 2024-09-14T06:32:15.647 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:53.876730+0000", 2024-09-14T06:32:15.647 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 141.76242779099999, 2024-09-14T06:32:15.647 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0079470329999999992, 2024-09-14T06:32:15.647 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:32:15.647 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:32:15.648 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:32:15.648 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:32:15.648 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:15.648 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:32:15.648 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:53.876730+0000", 2024-09-14T06:32:15.648 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:15.648 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:15.648 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:15.648 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:32:15.648 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:53.876730+0000", 2024-09-14T06:32:15.648 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:15.648 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:15.648 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:15.648 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:32:15.649 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:53.876731+0000", 2024-09-14T06:32:15.649 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.0500000000000004e-07 2024-09-14T06:32:15.649 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:15.649 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:15.649 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:32:15.649 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:53.879616+0000", 2024-09-14T06:32:15.649 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0028856680000000001 2024-09-14T06:32:15.649 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:15.649 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:15.649 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:32:15.649 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:53.879643+0000", 2024-09-14T06:32:15.649 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.6120999999999999e-05 2024-09-14T06:32:15.649 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:15.650 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:15.650 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:32:15.650 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:53.879654+0000", 2024-09-14T06:32:15.650 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.1586999999999999e-05 2024-09-14T06:32:15.650 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:15.650 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:15.650 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:32:15.650 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:53.879756+0000", 2024-09-14T06:32:15.650 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00010223899999999999 2024-09-14T06:32:15.650 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:15.650 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:15.651 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:32:15.651 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:53.879783+0000", 2024-09-14T06:32:15.651 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.6913000000000001e-05 2024-09-14T06:32:15.651 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:15.651 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:15.651 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-14T06:32:15.651 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:53.884594+0000", 2024-09-14T06:32:15.651 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0048110729999999999 2024-09-14T06:32:15.651 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:15.652 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:15.652 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:32:15.652 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:53.884677+0000", 2024-09-14T06:32:15.652 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.2927000000000001e-05 2024-09-14T06:32:15.652 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:15.653 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:32:15.653 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:15.653 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:15.653 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:15.653 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4397.0:74243 3.9 e73/23 3:94977f81:::rbd_data.112d80ca42af.0000000000000002:head v 73'94, mlcod=71'92)", 2024-09-14T06:32:15.653 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:53.885156+0000", 2024-09-14T06:32:15.653 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 141.75400187299999, 2024-09-14T06:32:15.653 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0096009860000000006, 2024-09-14T06:32:15.653 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:32:15.653 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:32:15.654 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:32:15.654 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:32:15.654 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:15.654 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:32:15.654 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:53.885156+0000", 2024-09-14T06:32:15.654 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:15.654 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:15.654 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:15.654 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:32:15.654 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:53.885156+0000", 2024-09-14T06:32:15.654 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:15.654 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:15.654 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:15.655 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:32:15.655 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:53.885156+0000", 2024-09-14T06:32:15.655 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.23e-07 2024-09-14T06:32:15.655 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:15.655 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:15.655 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:32:15.655 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:53.889694+0000", 2024-09-14T06:32:15.655 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0045378709999999997 2024-09-14T06:32:15.655 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:15.655 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:15.655 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:32:15.655 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:53.889720+0000", 2024-09-14T06:32:15.655 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.5259e-05 2024-09-14T06:32:15.655 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:15.655 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:15.656 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:32:15.656 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:53.889731+0000", 2024-09-14T06:32:15.656 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.1739000000000001e-05 2024-09-14T06:32:15.656 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:15.656 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:15.656 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:32:15.656 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:53.889822+0000", 2024-09-14T06:32:15.656 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.1200999999999996e-05 2024-09-14T06:32:15.656 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:15.656 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:15.656 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:32:15.656 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:53.889850+0000", 2024-09-14T06:32:15.656 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.7084999999999999e-05 2024-09-14T06:32:15.656 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:15.657 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:15.657 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-14T06:32:15.657 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:53.894703+0000", 2024-09-14T06:32:15.657 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0048532690000000003 2024-09-14T06:32:15.657 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:15.657 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:15.657 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:32:15.657 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:53.894757+0000", 2024-09-14T06:32:15.657 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.4339000000000003e-05 2024-09-14T06:32:15.657 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:15.657 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:32:15.657 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:15.657 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:15.657 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:15.658 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4397.0:74249 3.0 e73/23 3:009b0225:::rbd_data.112d80ca42af.0000000000000009:head v 73'103, mlcod=71'101)", 2024-09-14T06:32:15.658 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:53.893219+0000", 2024-09-14T06:32:15.658 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 141.74593939499999, 2024-09-14T06:32:15.658 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0065381980000000003, 2024-09-14T06:32:15.658 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:32:15.658 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:32:15.658 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:32:15.658 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:32:15.658 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:15.658 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:32:15.659 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:53.893219+0000", 2024-09-14T06:32:15.659 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:15.659 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:15.659 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:15.659 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:32:15.659 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:53.893219+0000", 2024-09-14T06:32:15.659 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:15.659 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:15.659 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:15.659 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:32:15.659 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:53.893221+0000", 2024-09-14T06:32:15.660 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.4329999999999998e-06 2024-09-14T06:32:15.660 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:15.660 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:15.660 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:32:15.660 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:53.895968+0000", 2024-09-14T06:32:15.660 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.002746746 2024-09-14T06:32:15.660 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:15.660 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:15.660 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:32:15.660 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:53.895988+0000", 2024-09-14T06:32:15.660 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.0227000000000001e-05 2024-09-14T06:32:15.660 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:15.661 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:15.661 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:32:15.661 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:53.896000+0000", 2024-09-14T06:32:15.661 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.2359e-05 2024-09-14T06:32:15.661 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:15.661 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:15.661 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:32:15.661 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:53.896084+0000", 2024-09-14T06:32:15.661 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.3659999999999995e-05 2024-09-14T06:32:15.661 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:15.661 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:15.661 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:32:15.661 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:53.896109+0000", 2024-09-14T06:32:15.661 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.5259999999999999e-05 2024-09-14T06:32:15.661 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:15.662 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:15.662 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-14T06:32:15.662 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:53.899704+0000", 2024-09-14T06:32:15.662 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0035942209999999999 2024-09-14T06:32:15.662 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:15.662 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:15.662 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:32:15.662 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:53.899757+0000", 2024-09-14T06:32:15.662 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.3291999999999997e-05 2024-09-14T06:32:15.662 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:15.662 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:32:15.662 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:15.662 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:15.663 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:15.663 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4397.0:78263 3.0 e77/23 3:00899b22:::rbd_data.112ddfbec471.0000000000000016:head v 77'104, mlcod=73'102)", 2024-09-14T06:32:15.663 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:57.634578+0000", 2024-09-14T06:32:15.663 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 138.00458034299999, 2024-09-14T06:32:15.663 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.002372454, 2024-09-14T06:32:15.663 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:32:15.663 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:32:15.663 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:32:15.663 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:32:15.663 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:15.663 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:32:15.663 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:57.634578+0000", 2024-09-14T06:32:15.663 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:15.664 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:15.664 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:15.664 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:32:15.664 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:57.634578+0000", 2024-09-14T06:32:15.664 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:15.664 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:15.664 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:15.664 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:32:15.664 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:57.634578+0000", 2024-09-14T06:32:15.664 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.5699999999999998e-07 2024-09-14T06:32:15.664 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:15.664 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:15.664 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:32:15.665 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:57.635347+0000", 2024-09-14T06:32:15.665 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00076848300000000003 2024-09-14T06:32:15.665 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:15.665 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:15.665 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:32:15.665 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:57.635361+0000", 2024-09-14T06:32:15.665 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.4238e-05 2024-09-14T06:32:15.665 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:15.665 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:15.665 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:32:15.665 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:57.635371+0000", 2024-09-14T06:32:15.666 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.0087e-05 2024-09-14T06:32:15.666 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:15.666 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:15.666 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:32:15.666 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:57.635439+0000", 2024-09-14T06:32:15.666 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.7953000000000005e-05 2024-09-14T06:32:15.666 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:15.666 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:15.666 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:32:15.666 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:57.635464+0000", 2024-09-14T06:32:15.666 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.5062000000000002e-05 2024-09-14T06:32:15.667 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:15.667 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:15.667 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-14T06:32:15.667 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:57.636914+0000", 2024-09-14T06:32:15.667 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.001449625 2024-09-14T06:32:15.667 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:15.667 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:15.667 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:32:15.667 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:57.636950+0000", 2024-09-14T06:32:15.667 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.6548999999999999e-05 2024-09-14T06:32:15.667 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:15.667 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:32:15.667 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:15.667 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:15.667 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:15.668 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4397.0:78604 3.9 e77/23 3:94c62a0e:::rbd_data.112ddfbec471.000000000000002c:head v 77'96, mlcod=73'94)", 2024-09-14T06:32:15.668 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:57.677568+0000", 2024-09-14T06:32:15.668 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 137.96158995499999, 2024-09-14T06:32:15.668 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.002564846, 2024-09-14T06:32:15.668 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:32:15.668 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:32:15.668 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:32:15.668 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:32:15.668 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:15.668 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:32:15.668 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:57.677568+0000", 2024-09-14T06:32:15.668 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:15.668 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:15.669 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:15.669 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:32:15.669 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:57.677568+0000", 2024-09-14T06:32:15.669 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:15.669 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:15.669 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:15.669 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:32:15.669 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:57.677568+0000", 2024-09-14T06:32:15.669 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.8900000000000001e-07 2024-09-14T06:32:15.669 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:15.669 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:15.669 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:32:15.669 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:57.678488+0000", 2024-09-14T06:32:15.670 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00091963899999999998 2024-09-14T06:32:15.670 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:15.670 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:15.670 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:32:15.670 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:57.678500+0000", 2024-09-14T06:32:15.670 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.1902000000000001e-05 2024-09-14T06:32:15.670 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:15.670 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:15.670 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:32:15.670 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:57.678507+0000", 2024-09-14T06:32:15.670 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.7739999999999999e-06 2024-09-14T06:32:15.670 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:15.670 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:15.670 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:32:15.671 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:57.678586+0000", 2024-09-14T06:32:15.671 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.9103000000000005e-05 2024-09-14T06:32:15.671 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:15.671 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:15.671 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:32:15.671 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:57.678614+0000", 2024-09-14T06:32:15.671 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.7912e-05 2024-09-14T06:32:15.671 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:15.671 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:15.671 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-14T06:32:15.671 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:57.680075+0000", 2024-09-14T06:32:15.671 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0014613370000000001 2024-09-14T06:32:15.671 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:15.671 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:15.671 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:32:15.672 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:57.680133+0000", 2024-09-14T06:32:15.672 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.7890000000000003e-05 2024-09-14T06:32:15.672 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:15.672 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:32:15.672 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:15.672 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:15.672 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:15.672 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4397.0:149885 3.9 e122/23 3:951acef3:::rbd_data.112d1bb7b269.0000000000000005:head v 122'176, mlcod=122'172)", 2024-09-14T06:32:15.672 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:30:44.739092+0000", 2024-09-14T06:32:15.672 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 90.900065644999998, 2024-09-14T06:32:15.672 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0028236659999999998, 2024-09-14T06:32:15.672 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:32:15.672 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:32:15.672 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:32:15.673 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:32:15.673 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:15.673 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:32:15.673 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.739092+0000", 2024-09-14T06:32:15.673 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:15.673 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:15.673 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:15.673 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:32:15.673 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.739092+0000", 2024-09-14T06:32:15.673 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:15.673 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:15.673 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:15.673 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:32:15.674 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.739093+0000", 2024-09-14T06:32:15.674 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.0109999999999999e-06 2024-09-14T06:32:15.674 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:15.674 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:15.674 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:32:15.674 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.740189+0000", 2024-09-14T06:32:15.674 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00109527 2024-09-14T06:32:15.675 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:15.675 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:15.675 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:32:15.675 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.740199+0000", 2024-09-14T06:32:15.675 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.0458999999999999e-05 2024-09-14T06:32:15.675 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:15.675 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:15.675 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:32:15.675 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.740207+0000", 2024-09-14T06:32:15.675 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.4499999999999998e-06 2024-09-14T06:32:15.675 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:15.676 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:15.676 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:32:15.676 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.740295+0000", 2024-09-14T06:32:15.676 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.8740999999999996e-05 2024-09-14T06:32:15.676 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:15.676 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:15.676 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:32:15.676 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.740317+0000", 2024-09-14T06:32:15.676 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.1523e-05 2024-09-14T06:32:15.676 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:15.677 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:15.677 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-14T06:32:15.677 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.741865+0000", 2024-09-14T06:32:15.677 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.001548315 2024-09-14T06:32:15.677 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:15.677 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:15.677 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:32:15.677 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.741916+0000", 2024-09-14T06:32:15.677 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.0896999999999997e-05 2024-09-14T06:32:15.677 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:15.677 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:32:15.677 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:15.678 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:15.678 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:15.678 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4397.0:149933 3.4 e122/23 3:234f5229:::rbd_data.112d1bb7b269.0000000000000009:head v 122'221, mlcod=120'218)", 2024-09-14T06:32:15.678 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:30:44.743849+0000", 2024-09-14T06:32:15.678 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 90.895309228000002, 2024-09-14T06:32:15.678 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0031479519999999999, 2024-09-14T06:32:15.678 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:32:15.678 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:32:15.678 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:32:15.678 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:32:15.678 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:15.678 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:32:15.678 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.743849+0000", 2024-09-14T06:32:15.678 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:15.679 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:15.679 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:15.679 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:32:15.679 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.743849+0000", 2024-09-14T06:32:15.679 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:15.679 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:15.679 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:15.679 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:32:15.679 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.743850+0000", 2024-09-14T06:32:15.679 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.279e-06 2024-09-14T06:32:15.679 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:15.679 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:15.679 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:32:15.679 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.745151+0000", 2024-09-14T06:32:15.680 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.001300495 2024-09-14T06:32:15.680 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:15.680 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:15.680 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:32:15.680 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.745167+0000", 2024-09-14T06:32:15.680 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.6478e-05 2024-09-14T06:32:15.680 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:15.680 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:15.680 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:32:15.680 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.745175+0000", 2024-09-14T06:32:15.680 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.2679999999999994e-06 2024-09-14T06:32:15.680 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:15.680 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:15.681 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:32:15.681 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.745226+0000", 2024-09-14T06:32:15.681 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.0192999999999999e-05 2024-09-14T06:32:15.681 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:15.681 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:15.681 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:32:15.681 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.745247+0000", 2024-09-14T06:32:15.681 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.1767e-05 2024-09-14T06:32:15.681 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:15.681 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:15.681 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-14T06:32:15.681 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.746952+0000", 2024-09-14T06:32:15.681 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0017042769999999999 2024-09-14T06:32:15.681 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:15.681 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:15.682 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:32:15.682 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.746997+0000", 2024-09-14T06:32:15.682 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.5194999999999997e-05 2024-09-14T06:32:15.682 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:15.682 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:32:15.682 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:15.682 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:15.682 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:15.682 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4397.0:150494 3.9 e122/23 3:955152a4:::rbd_object_map.112d1bb7b269.0000000000000038:head v 122'194, mlcod=122'190)", 2024-09-14T06:32:15.682 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:30:44.781585+0000", 2024-09-14T06:32:15.683 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 90.857573336000002, 2024-09-14T06:32:15.683 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0026957499999999998, 2024-09-14T06:32:15.683 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:32:15.683 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:32:15.683 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:32:15.683 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:32:15.683 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:15.683 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:32:15.683 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.781585+0000", 2024-09-14T06:32:15.683 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:15.683 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:15.684 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:15.684 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:32:15.684 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.781585+0000", 2024-09-14T06:32:15.684 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:15.684 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:15.684 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:15.684 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:32:15.684 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.781585+0000", 2024-09-14T06:32:15.684 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.5000000000000002e-07 2024-09-14T06:32:15.684 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:15.684 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:15.685 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:32:15.685 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.781594+0000", 2024-09-14T06:32:15.685 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.4640000000000006e-06 2024-09-14T06:32:15.685 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:15.685 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:15.685 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:32:15.685 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.781604+0000", 2024-09-14T06:32:15.685 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.0071999999999999e-05 2024-09-14T06:32:15.685 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:15.685 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:15.685 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:32:15.685 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.781610+0000", 2024-09-14T06:32:15.685 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.3609999999999997e-06 2024-09-14T06:32:15.685 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:15.685 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:15.686 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:32:15.686 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.781676+0000", 2024-09-14T06:32:15.686 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.5963000000000005e-05 2024-09-14T06:32:15.686 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:15.686 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:15.686 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:32:15.686 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.781696+0000", 2024-09-14T06:32:15.686 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.9278000000000001e-05 2024-09-14T06:32:15.686 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:15.686 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:15.686 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-14T06:32:15.686 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.784231+0000", 2024-09-14T06:32:15.686 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.002535093 2024-09-14T06:32:15.686 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:15.686 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:15.687 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:32:15.687 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.784280+0000", 2024-09-14T06:32:15.687 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.9869e-05 2024-09-14T06:32:15.687 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:15.687 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:32:15.687 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:15.687 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:15.687 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:15.687 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4397.0:150898 3.9 e122/23 3:97c98f90:::rbd_data.112d1bb7b269.0000000000000025:head v 122'206, mlcod=122'203)", 2024-09-14T06:32:15.687 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:30:44.807232+0000", 2024-09-14T06:32:15.687 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 90.831926418999998, 2024-09-14T06:32:15.687 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0022297889999999998, 2024-09-14T06:32:15.687 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:32:15.688 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:32:15.688 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:32:15.688 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:32:15.688 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:15.688 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:32:15.688 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.807232+0000", 2024-09-14T06:32:15.688 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:15.688 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:15.688 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:15.688 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:32:15.688 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.807232+0000", 2024-09-14T06:32:15.688 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:15.688 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:15.688 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:15.689 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:32:15.689 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.807233+0000", 2024-09-14T06:32:15.689 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.5710000000000001e-06 2024-09-14T06:32:15.689 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:15.689 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:15.689 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:32:15.689 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.807944+0000", 2024-09-14T06:32:15.689 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00071056299999999999 2024-09-14T06:32:15.689 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:15.689 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:15.689 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:32:15.689 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.807961+0000", 2024-09-14T06:32:15.689 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.7689e-05 2024-09-14T06:32:15.689 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:15.690 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:15.690 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:32:15.690 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.807971+0000", 2024-09-14T06:32:15.690 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.2e-06 2024-09-14T06:32:15.690 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:15.690 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:15.690 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:32:15.690 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.808050+0000", 2024-09-14T06:32:15.690 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.9308999999999996e-05 2024-09-14T06:32:15.690 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:15.690 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:15.690 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:32:15.691 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.808074+0000", 2024-09-14T06:32:15.691 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.4147000000000001e-05 2024-09-14T06:32:15.691 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:15.691 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:15.691 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-14T06:32:15.691 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.809415+0000", 2024-09-14T06:32:15.691 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.001341257 2024-09-14T06:32:15.691 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:15.691 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:15.691 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:32:15.691 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.809461+0000", 2024-09-14T06:32:15.691 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.6053000000000001e-05 2024-09-14T06:32:15.691 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:15.691 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:32:15.692 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:15.692 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:15.692 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:15.692 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4397.0:202551 3.9 e166/23 3:965c09b6:::rbd_data.112d3bd84cb0.000000000000000a:head v 166'416, mlcod=164'414)", 2024-09-14T06:32:15.692 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:31:30.657011+0000", 2024-09-14T06:32:15.692 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 44.982147386000001, 2024-09-14T06:32:15.692 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.002509262, 2024-09-14T06:32:15.692 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:32:15.692 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:32:15.692 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:32:15.693 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:32:15.693 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:15.693 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:32:15.693 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:30.657011+0000", 2024-09-14T06:32:15.693 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:15.693 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:15.694 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:15.694 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:32:15.694 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:30.657011+0000", 2024-09-14T06:32:15.694 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:15.694 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:15.694 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:15.694 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:32:15.694 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:30.657011+0000", 2024-09-14T06:32:15.694 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.08e-07 2024-09-14T06:32:15.695 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:15.695 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:15.695 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:32:15.695 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:30.658467+0000", 2024-09-14T06:32:15.695 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.001455708 2024-09-14T06:32:15.696 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:15.696 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:15.696 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:32:15.696 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:30.658484+0000", 2024-09-14T06:32:15.696 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.6844e-05 2024-09-14T06:32:15.696 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:15.696 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:15.696 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:32:15.696 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:30.658493+0000", 2024-09-14T06:32:15.696 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.2790000000000004e-06 2024-09-14T06:32:15.696 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:15.696 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:15.697 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:32:15.697 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:30.658559+0000", 2024-09-14T06:32:15.697 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.6459999999999997e-05 2024-09-14T06:32:15.697 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:15.697 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:15.697 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:32:15.697 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:30.658574+0000", 2024-09-14T06:32:15.697 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.5048000000000001e-05 2024-09-14T06:32:15.697 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:15.697 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:15.697 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-14T06:32:15.697 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:30.659473+0000", 2024-09-14T06:32:15.697 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00089842800000000005 2024-09-14T06:32:15.697 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:15.698 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:15.698 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:32:15.698 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:30.659520+0000", 2024-09-14T06:32:15.698 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.7086999999999998e-05 2024-09-14T06:32:15.698 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:15.698 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:32:15.698 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:15.698 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:15.698 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:15.698 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4397.0:202665 3.0 e166/23 3:00eeadfb:::rbd_data.112d3bd84cb0.000000000000000e:head v 166'920, mlcod=163'918)", 2024-09-14T06:32:15.698 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:31:30.662434+0000", 2024-09-14T06:32:15.698 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 44.976723597000003, 2024-09-14T06:32:15.698 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.002522121, 2024-09-14T06:32:15.698 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:32:15.698 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:32:15.699 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:32:15.699 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:32:15.699 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:15.699 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:32:15.699 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:30.662434+0000", 2024-09-14T06:32:15.699 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:15.699 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:15.699 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:15.699 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:32:15.699 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:30.662434+0000", 2024-09-14T06:32:15.699 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:15.699 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:15.699 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:15.700 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:32:15.700 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:30.662436+0000", 2024-09-14T06:32:15.700 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.257e-06 2024-09-14T06:32:15.700 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:15.700 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:15.700 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:32:15.700 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:30.662527+0000", 2024-09-14T06:32:15.700 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.1644000000000006e-05 2024-09-14T06:32:15.700 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:15.700 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:15.700 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:32:15.700 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:30.662546+0000", 2024-09-14T06:32:15.701 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.8998e-05 2024-09-14T06:32:15.701 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:15.701 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:15.701 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:32:15.701 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:30.662556+0000", 2024-09-14T06:32:15.701 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.9809999999999998e-06 2024-09-14T06:32:15.701 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:15.701 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:15.701 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:32:15.701 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:30.662621+0000", 2024-09-14T06:32:15.701 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.4597000000000005e-05 2024-09-14T06:32:15.701 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:15.701 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:15.701 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:32:15.702 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:30.662643+0000", 2024-09-14T06:32:15.702 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.1608e-05 2024-09-14T06:32:15.702 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:15.702 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:15.702 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-14T06:32:15.702 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:30.664909+0000", 2024-09-14T06:32:15.702 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0022668689999999999 2024-09-14T06:32:15.702 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:15.702 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:15.702 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:32:15.702 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:30.664957+0000", 2024-09-14T06:32:15.702 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.7166999999999998e-05 2024-09-14T06:32:15.702 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:15.702 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:32:15.703 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:15.703 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:15.703 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:15.703 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4397.0:204589 3.4 e166/23 3:234552f0:::rbd_data.112d3bd84cb0.000000000000003f:head v 166'499, mlcod=165'497)", 2024-09-14T06:32:15.703 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:31:30.772894+0000", 2024-09-14T06:32:15.703 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 44.866263740999997, 2024-09-14T06:32:15.703 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0034564650000000001, 2024-09-14T06:32:15.703 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:32:15.703 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:32:15.703 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:32:15.703 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:32:15.704 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:15.704 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:32:15.704 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:30.772894+0000", 2024-09-14T06:32:15.704 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:15.704 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:15.704 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:15.704 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:32:15.704 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:30.772894+0000", 2024-09-14T06:32:15.704 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:15.704 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:15.704 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:15.704 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:32:15.705 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:30.772896+0000", 2024-09-14T06:32:15.705 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.5379999999999999e-06 2024-09-14T06:32:15.705 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:15.705 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:15.705 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:32:15.705 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:30.773042+0000", 2024-09-14T06:32:15.705 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000146107 2024-09-14T06:32:15.705 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:15.705 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:15.705 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:32:15.705 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:30.773061+0000", 2024-09-14T06:32:15.705 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.8862999999999999e-05 2024-09-14T06:32:15.705 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:15.706 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:15.706 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:32:15.706 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:30.773071+0000", 2024-09-14T06:32:15.706 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.0227000000000001e-05 2024-09-14T06:32:15.706 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:15.706 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:15.706 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:32:15.706 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:30.773149+0000", 2024-09-14T06:32:15.706 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.8071999999999994e-05 2024-09-14T06:32:15.706 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:15.707 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:15.707 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:32:15.707 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:30.773171+0000", 2024-09-14T06:32:15.707 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.1699999999999999e-05 2024-09-14T06:32:15.707 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:15.707 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:15.707 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-14T06:32:15.707 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:30.776285+0000", 2024-09-14T06:32:15.707 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0031142050000000001 2024-09-14T06:32:15.707 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:15.707 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:15.707 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:32:15.707 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:30.776351+0000", 2024-09-14T06:32:15.707 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.5753000000000006e-05 2024-09-14T06:32:15.707 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:15.708 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:32:15.708 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:15.708 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:15.708 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:15.708 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4397.0:204598 3.4 e166/23 3:245d069e:::rbd_data.112d3bd84cb0.0000000000000041:head v 166'500, mlcod=165'497)", 2024-09-14T06:32:15.708 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:31:30.775247+0000", 2024-09-14T06:32:15.708 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 44.863910617999998, 2024-09-14T06:32:15.708 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00318235, 2024-09-14T06:32:15.708 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:32:15.708 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:32:15.708 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:32:15.708 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:32:15.708 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:15.708 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:32:15.708 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:30.775247+0000", 2024-09-14T06:32:15.709 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:15.709 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:15.709 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:15.709 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:32:15.709 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:30.775247+0000", 2024-09-14T06:32:15.709 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:15.709 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:15.709 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:15.709 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:32:15.709 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:30.775249+0000", 2024-09-14T06:32:15.709 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.265e-06 2024-09-14T06:32:15.709 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:15.709 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:15.709 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:32:15.709 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:30.775340+0000", 2024-09-14T06:32:15.710 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.1131999999999994e-05 2024-09-14T06:32:15.710 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:15.710 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:15.710 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:32:15.710 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:30.775352+0000", 2024-09-14T06:32:15.710 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.2294e-05 2024-09-14T06:32:15.710 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:15.710 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:15.710 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:32:15.710 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:30.775361+0000", 2024-09-14T06:32:15.710 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.6950000000000006e-06 2024-09-14T06:32:15.710 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:15.711 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:15.711 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:32:15.711 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:30.775439+0000", 2024-09-14T06:32:15.711 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.7910000000000005e-05 2024-09-14T06:32:15.711 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:15.711 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:15.711 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:32:15.711 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:30.775460+0000", 2024-09-14T06:32:15.711 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.0900999999999999e-05 2024-09-14T06:32:15.711 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:15.711 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:15.711 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-14T06:32:15.711 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:30.778385+0000", 2024-09-14T06:32:15.712 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.002925539 2024-09-14T06:32:15.712 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:15.712 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:15.712 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:32:15.712 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:30.778430+0000", 2024-09-14T06:32:15.712 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.4613999999999997e-05 2024-09-14T06:32:15.712 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:15.712 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:32:15.712 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:15.712 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:15.712 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:15.713 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4397.0:204975 3.4 e166/23 3:20972d18:::rbd_data.112d3bd84cb0.0000000000000046:head v 166'501, mlcod=166'499)", 2024-09-14T06:32:15.713 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:31:30.787253+0000", 2024-09-14T06:32:15.713 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 44.851904589, 2024-09-14T06:32:15.713 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.002248318, 2024-09-14T06:32:15.713 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:32:15.713 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:32:15.713 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:32:15.713 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:32:15.713 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:15.713 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:32:15.713 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:30.787253+0000", 2024-09-14T06:32:15.713 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:15.713 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:15.713 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:15.713 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:32:15.714 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:30.787253+0000", 2024-09-14T06:32:15.714 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:15.714 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:15.714 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:15.714 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:32:15.714 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:30.787255+0000", 2024-09-14T06:32:15.714 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.271e-06 2024-09-14T06:32:15.714 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:15.714 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:15.714 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:32:15.714 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:30.787442+0000", 2024-09-14T06:32:15.714 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00018744000000000001 2024-09-14T06:32:15.714 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:15.714 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:15.714 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:32:15.714 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:30.787461+0000", 2024-09-14T06:32:15.715 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.9054000000000001e-05 2024-09-14T06:32:15.715 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:15.715 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:15.715 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:32:15.715 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:30.787470+0000", 2024-09-14T06:32:15.715 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.0929999999999992e-06 2024-09-14T06:32:15.715 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:15.715 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:15.715 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:32:15.715 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:30.788004+0000", 2024-09-14T06:32:15.715 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00053359399999999997 2024-09-14T06:32:15.715 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:15.715 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:15.715 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:32:15.715 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:30.788034+0000", 2024-09-14T06:32:15.716 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.9635e-05 2024-09-14T06:32:15.716 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:15.716 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:15.716 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-14T06:32:15.716 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:30.789457+0000", 2024-09-14T06:32:15.716 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.001423487 2024-09-14T06:32:15.716 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:15.716 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:15.716 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:32:15.716 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:30.789502+0000", 2024-09-14T06:32:15.716 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.4743999999999997e-05 2024-09-14T06:32:15.716 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:15.716 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:32:15.716 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:15.716 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:15.717 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:15.717 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4397.0:212188 3.4 e166/23 3:23871bc7:::rbd_data.112d3bd84cb0.00000000000000fa:head v 166'508, mlcod=166'506)", 2024-09-14T06:32:15.717 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:31:31.391415+0000", 2024-09-14T06:32:15.717 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 44.247743006999997, 2024-09-14T06:32:15.717 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0022039049999999999, 2024-09-14T06:32:15.717 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:32:15.717 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:32:15.717 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:32:15.717 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:32:15.717 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:15.717 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:32:15.717 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:31.391415+0000", 2024-09-14T06:32:15.717 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:15.717 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:15.718 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:15.718 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:32:15.718 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:31.391415+0000", 2024-09-14T06:32:15.718 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:15.718 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:15.718 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:15.718 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:32:15.718 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:31.391416+0000", 2024-09-14T06:32:15.718 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.44e-06 2024-09-14T06:32:15.718 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:15.718 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:15.718 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:32:15.718 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:31.391554+0000", 2024-09-14T06:32:15.718 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000137119 2024-09-14T06:32:15.718 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:15.719 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:15.719 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:32:15.719 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:31.391570+0000", 2024-09-14T06:32:15.719 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.6827e-05 2024-09-14T06:32:15.719 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:15.719 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:15.719 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:32:15.719 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:31.391580+0000", 2024-09-14T06:32:15.719 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.3139999999999993e-06 2024-09-14T06:32:15.719 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:15.719 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:15.719 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:32:15.719 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:31.391664+0000", 2024-09-14T06:32:15.719 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.3757999999999993e-05 2024-09-14T06:32:15.719 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:15.720 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:15.720 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:32:15.720 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:31.391692+0000", 2024-09-14T06:32:15.720 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.8898e-05 2024-09-14T06:32:15.720 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:15.720 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:15.720 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-14T06:32:15.720 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:31.393584+0000", 2024-09-14T06:32:15.720 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.001891752 2024-09-14T06:32:15.720 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:15.720 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:15.720 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:32:15.720 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:31.393619+0000", 2024-09-14T06:32:15.721 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.4796999999999999e-05 2024-09-14T06:32:15.721 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:15.721 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:32:15.721 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:15.721 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:15.721 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:15.721 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "MOSDPGPush(3.5 169/168 [PushOp(3:a7c12b08:::rbd_data.112ddfbec471.000000000000000e:head, version: 77'100, data_included: [0~1048576], data_size: 1048576, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(3:a7c12b08:::rbd_data.112ddfbec471.000000000000000e:head@77'100, size: 1048576, copy_subset: [(0, 1048576)], clone_subset: {}, snapset: 0=[]:{}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 1048576, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2024-09-14T06:32:15.721 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:31:34.540707+0000", 2024-09-14T06:32:15.721 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 41.098451242000003, 2024-09-14T06:32:15.721 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0023637369999999999, 2024-09-14T06:32:15.721 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:32:15.721 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:32:15.721 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:32:15.721 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:32:15.722 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:15.722 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:32:15.722 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:34.540707+0000", 2024-09-14T06:32:15.722 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:15.722 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:15.722 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:15.722 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:32:15.722 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:34.540707+0000", 2024-09-14T06:32:15.722 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:15.722 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:15.722 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:15.722 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:32:15.722 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:34.540708+0000", 2024-09-14T06:32:15.722 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.7199999999999998e-07 2024-09-14T06:32:15.723 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:15.723 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:15.723 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:32:15.723 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:34.542150+0000", 2024-09-14T06:32:15.723 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0014423229999999999 2024-09-14T06:32:15.723 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:15.723 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:15.723 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:32:15.723 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:34.542169+0000", 2024-09-14T06:32:15.723 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.8756000000000001e-05 2024-09-14T06:32:15.723 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:15.723 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:15.723 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:32:15.723 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:34.542180+0000", 2024-09-14T06:32:15.724 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.1051e-05 2024-09-14T06:32:15.724 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:15.724 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:15.724 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:32:15.724 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:34.542275+0000", 2024-09-14T06:32:15.724 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.5122000000000002e-05 2024-09-14T06:32:15.724 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:15.724 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:15.724 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:32:15.724 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:34.542284+0000", 2024-09-14T06:32:15.724 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.9190000000000005e-06 2024-09-14T06:32:15.724 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:15.724 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:15.725 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:32:15.725 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:34.543071+0000", 2024-09-14T06:32:15.725 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00078679399999999995 2024-09-14T06:32:15.725 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:15.725 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:32:15.725 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:15.725 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:15.725 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:15.725 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "MOSDPGPush(3.5 169/168 [PushOp(3:a7a54476:::rbd_data.112d1bb7b269.0000000000000038:head, version: 122'581, data_included: [0~524288], data_size: 524288, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(3:a7a54476:::rbd_data.112d1bb7b269.0000000000000038:head@122'581, size: 524288, copy_subset: [(0, 524288)], clone_subset: {}, snapset: 0=[]:{}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 524288, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2024-09-14T06:32:15.725 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:31:34.667757+0000", 2024-09-14T06:32:15.725 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 40.971401196000002, 2024-09-14T06:32:15.725 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.002750382, 2024-09-14T06:32:15.726 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:32:15.726 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:32:15.726 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:32:15.726 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:32:15.726 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:15.726 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:32:15.726 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:34.667757+0000", 2024-09-14T06:32:15.726 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:15.726 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:15.726 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:15.726 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:32:15.726 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:34.667757+0000", 2024-09-14T06:32:15.726 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:15.727 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:15.727 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:15.727 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:32:15.727 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:34.667758+0000", 2024-09-14T06:32:15.727 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.3599999999999999e-06 2024-09-14T06:32:15.727 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:15.727 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:15.727 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:32:15.727 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:34.669375+0000", 2024-09-14T06:32:15.727 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.001616644 2024-09-14T06:32:15.727 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:15.727 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:15.727 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:32:15.727 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:34.669404+0000", 2024-09-14T06:32:15.727 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.9300999999999999e-05 2024-09-14T06:32:15.728 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:15.728 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:15.728 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:32:15.728 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:34.669428+0000", 2024-09-14T06:32:15.728 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.3788e-05 2024-09-14T06:32:15.728 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:15.728 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:15.728 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:32:15.728 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:34.669543+0000", 2024-09-14T06:32:15.728 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000115462 2024-09-14T06:32:15.728 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:15.728 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:15.728 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:32:15.728 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:34.669559+0000", 2024-09-14T06:32:15.729 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.5167999999999999e-05 2024-09-14T06:32:15.729 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:15.729 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:15.729 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:32:15.729 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:34.670507+0000", 2024-09-14T06:32:15.729 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00094865900000000003 2024-09-14T06:32:15.729 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:15.729 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:32:15.729 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:15.729 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:15.729 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:15.729 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "MOSDPGPush(3.5 170/168 [PushOp(3:a162138c:::rbd_data.112d1bb7b269.000000000000003c:head, version: 122'582, data_included: [0~524288], data_size: 524288, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(3:a162138c:::rbd_data.112d1bb7b269.000000000000003c:head@122'582, size: 524288, copy_subset: [(0, 524288)], clone_subset: {}, snapset: 0=[]:{}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 524288, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2024-09-14T06:32:15.729 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:31:34.670110+0000", 2024-09-14T06:32:15.729 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 40.969048247000003, 2024-09-14T06:32:15.730 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0030066229999999999, 2024-09-14T06:32:15.730 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:32:15.730 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:32:15.730 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:32:15.730 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:32:15.730 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:15.730 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:32:15.730 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:34.670110+0000", 2024-09-14T06:32:15.730 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:15.730 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:15.730 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:15.731 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:32:15.731 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:34.670110+0000", 2024-09-14T06:32:15.731 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:15.731 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:15.731 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:15.731 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:32:15.731 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:34.670111+0000", 2024-09-14T06:32:15.731 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.6499999999999998e-07 2024-09-14T06:32:15.731 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:15.731 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:15.731 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:32:15.731 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:34.671507+0000", 2024-09-14T06:32:15.731 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.001396284 2024-09-14T06:32:15.731 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:15.732 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:15.732 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:32:15.732 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:34.671530+0000", 2024-09-14T06:32:15.732 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.3023e-05 2024-09-14T06:32:15.732 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:15.732 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:15.732 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:32:15.732 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:34.671548+0000", 2024-09-14T06:32:15.732 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.8116000000000001e-05 2024-09-14T06:32:15.732 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:15.732 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:15.732 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:32:15.732 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:34.671775+0000", 2024-09-14T06:32:15.732 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00022699600000000001 2024-09-14T06:32:15.733 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:15.733 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:15.733 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:32:15.733 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:34.671790+0000", 2024-09-14T06:32:15.733 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.539e-05 2024-09-14T06:32:15.733 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:15.733 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:15.733 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:32:15.733 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:34.673116+0000", 2024-09-14T06:32:15.733 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0013259490000000001 2024-09-14T06:32:15.733 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:15.733 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:32:15.733 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:15.733 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:15.734 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:15.734 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4397.0:232694 3.9 e207/23 3:95763440:::rbd_data.112d838b9898.0000000000000006:head v 207'513, mlcod=203'511)", 2024-09-14T06:32:15.734 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:32:12.742327+0000", 2024-09-14T06:32:15.734 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 2.8968312869999999, 2024-09-14T06:32:15.734 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.002504952, 2024-09-14T06:32:15.734 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:32:15.734 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:32:15.734 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:32:15.734 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:32:15.734 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:15.734 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:32:15.734 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:12.742327+0000", 2024-09-14T06:32:15.735 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:15.735 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:15.735 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:15.735 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:32:15.735 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:12.742327+0000", 2024-09-14T06:32:15.735 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:15.735 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:15.735 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:15.735 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:32:15.735 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:12.742327+0000", 2024-09-14T06:32:15.735 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.3999999999999997e-07 2024-09-14T06:32:15.735 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:15.735 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:15.735 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:32:15.735 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:12.742332+0000", 2024-09-14T06:32:15.736 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.3569999999999997e-06 2024-09-14T06:32:15.736 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:15.736 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:15.736 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:32:15.736 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:12.742360+0000", 2024-09-14T06:32:15.736 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.7650000000000001e-05 2024-09-14T06:32:15.736 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:15.736 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:15.736 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:32:15.736 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:12.742370+0000", 2024-09-14T06:32:15.736 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.0115000000000001e-05 2024-09-14T06:32:15.736 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:15.737 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:15.737 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:32:15.737 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:12.742433+0000", 2024-09-14T06:32:15.737 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.2364000000000001e-05 2024-09-14T06:32:15.737 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:15.737 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:15.737 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:32:15.737 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:12.742451+0000", 2024-09-14T06:32:15.737 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.8250999999999999e-05 2024-09-14T06:32:15.737 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:15.737 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:15.737 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-14T06:32:15.737 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:12.744776+0000", 2024-09-14T06:32:15.737 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0023253839999999998 2024-09-14T06:32:15.737 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:15.737 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:15.738 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:32:15.738 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:12.744832+0000", 2024-09-14T06:32:15.738 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.5491000000000001e-05 2024-09-14T06:32:15.738 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:15.738 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:32:15.738 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:15.738 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:15.738 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:32:15.738 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-14T06:32:15.740 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-14T06:32:15.741 INFO:tasks.ceph.osd.5.smithi138.stderr:2024-09-14T06:32:15.738+0000 7fe13746f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33618) UID: 0 2024-09-14T06:32:15.841 INFO:tasks.ceph.osd.4.smithi138.stderr:2024-09-14T06:32:15.840+0000 7f6c653c8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33615) UID: 0 2024-09-14T06:32:15.903 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-14T06:32:15.903 INFO:teuthology.orchestra.run.smithi094.stdout: "ops": [], 2024-09-14T06:32:15.903 INFO:teuthology.orchestra.run.smithi094.stdout: "num_ops": 0 2024-09-14T06:32:15.904 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-14T06:32:15.914 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-14T06:32:15.957 INFO:tasks.ceph.osd.1.smithi094.stderr:2024-09-14T06:32:15.956+0000 7f328e47a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33641) UID: 0 2024-09-14T06:32:16.032 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-14T06:32:16.033 INFO:teuthology.orchestra.run.smithi094.stdout: "ops": [], 2024-09-14T06:32:16.033 INFO:teuthology.orchestra.run.smithi094.stdout: "complaint_time": 30, 2024-09-14T06:32:16.033 INFO:teuthology.orchestra.run.smithi094.stdout: "num_blocked_ops": 0 2024-09-14T06:32:16.033 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-14T06:32:16.043 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-14T06:32:16.044 INFO:tasks.ceph.osd.2.smithi094.stderr:2024-09-14T06:32:16.042+0000 7faf3269b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33649) UID: 0 2024-09-14T06:32:16.144 INFO:tasks.ceph.osd.6.smithi138.stderr:2024-09-14T06:32:16.143+0000 7f8cd8849640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33620) UID: 0 2024-09-14T06:32:16.190 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-14T06:32:16.190 INFO:teuthology.orchestra.run.smithi094.stdout: "size": 20, 2024-09-14T06:32:16.190 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 600, 2024-09-14T06:32:16.190 INFO:teuthology.orchestra.run.smithi094.stdout: "ops": [ 2024-09-14T06:32:16.190 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:16.190 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:44425 3.2 3:4603ad7f:::rbd_data.112dca2f6ae1.0000000000000005:17 [read 868352~4096 [fadvise_nocache]] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:32:16.190 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:43.854967+0000", 2024-09-14T06:32:16.191 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 152.333674922, 2024-09-14T06:32:16.191 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.063146025999999994, 2024-09-14T06:32:16.191 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:32:16.191 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:32:16.191 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:32:16.191 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:32:16.191 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:32:16.191 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:16.191 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 44425 2024-09-14T06:32:16.191 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:16.191 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:32:16.191 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:16.191 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:32:16.191 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.854967+0000", 2024-09-14T06:32:16.192 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:16.192 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:16.192 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:16.192 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:32:16.192 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.854967+0000", 2024-09-14T06:32:16.192 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:16.192 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:16.192 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:16.192 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:32:16.192 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.854967+0000", 2024-09-14T06:32:16.192 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.16e-07 2024-09-14T06:32:16.192 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:16.192 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:16.193 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:32:16.193 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.854968+0000", 2024-09-14T06:32:16.193 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.006e-06 2024-09-14T06:32:16.193 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:16.193 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:16.193 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:32:16.193 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.854971+0000", 2024-09-14T06:32:16.193 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.9359999999999999e-06 2024-09-14T06:32:16.193 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:16.193 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:16.193 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:32:16.193 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.854974+0000", 2024-09-14T06:32:16.193 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.1840000000000002e-06 2024-09-14T06:32:16.193 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:16.194 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:16.194 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:32:16.194 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.917945+0000", 2024-09-14T06:32:16.194 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.062970925999999997 2024-09-14T06:32:16.194 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:16.194 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:16.194 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:32:16.194 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.918030+0000", 2024-09-14T06:32:16.194 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.5863000000000001e-05 2024-09-14T06:32:16.194 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:16.194 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:16.194 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:32:16.194 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.918113+0000", 2024-09-14T06:32:16.194 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.2894999999999998e-05 2024-09-14T06:32:16.195 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:16.195 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:32:16.195 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:16.195 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:16.195 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:16.195 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:44426 3.2 3:4603ad7f:::rbd_data.112dca2f6ae1.0000000000000005:17 [read 892928~4096 [fadvise_nocache]] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:32:16.195 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:43.854977+0000", 2024-09-14T06:32:16.195 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 152.33366551200001, 2024-09-14T06:32:16.195 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.063369237999999994, 2024-09-14T06:32:16.195 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:32:16.195 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:32:16.195 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:32:16.195 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:32:16.195 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:32:16.196 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:16.196 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 44426 2024-09-14T06:32:16.196 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:16.196 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:32:16.196 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:16.196 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:32:16.196 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.854977+0000", 2024-09-14T06:32:16.196 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:16.196 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:16.196 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:16.196 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:32:16.196 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.854977+0000", 2024-09-14T06:32:16.196 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:16.197 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:16.197 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:16.197 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:32:16.197 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.854977+0000", 2024-09-14T06:32:16.197 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.1199999999999999e-07 2024-09-14T06:32:16.197 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:16.197 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:16.197 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:32:16.197 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.854980+0000", 2024-09-14T06:32:16.197 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.27e-06 2024-09-14T06:32:16.197 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:16.197 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:16.197 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:32:16.197 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.854983+0000", 2024-09-14T06:32:16.198 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.1650000000000002e-06 2024-09-14T06:32:16.198 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:16.198 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:16.198 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:32:16.198 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.854989+0000", 2024-09-14T06:32:16.198 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.6520000000000003e-06 2024-09-14T06:32:16.198 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:16.198 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:16.198 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:32:16.198 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.918143+0000", 2024-09-14T06:32:16.198 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.063154541999999994 2024-09-14T06:32:16.198 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:16.198 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:16.198 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:32:16.198 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.918209+0000", 2024-09-14T06:32:16.199 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.5529000000000002e-05 2024-09-14T06:32:16.199 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:16.199 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:16.199 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:32:16.199 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.918346+0000", 2024-09-14T06:32:16.199 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000136868 2024-09-14T06:32:16.199 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:16.199 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:32:16.199 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:16.199 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:16.199 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:16.199 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:44427 3.2 3:4603ad7f:::rbd_data.112dca2f6ae1.0000000000000005:17 [read 917504~4096 [fadvise_nocache]] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:32:16.199 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:43.854992+0000", 2024-09-14T06:32:16.199 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 152.333649866, 2024-09-14T06:32:16.199 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.063607792999999996, 2024-09-14T06:32:16.200 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:32:16.200 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:32:16.200 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:32:16.200 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:32:16.200 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:32:16.200 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:16.200 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 44427 2024-09-14T06:32:16.200 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:16.200 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:32:16.200 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:16.200 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:32:16.200 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.854992+0000", 2024-09-14T06:32:16.200 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:16.200 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:16.200 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:16.201 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:32:16.201 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.854992+0000", 2024-09-14T06:32:16.201 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:16.201 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:16.201 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:16.201 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:32:16.201 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.854993+0000", 2024-09-14T06:32:16.201 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.1400000000000001e-07 2024-09-14T06:32:16.201 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:16.201 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:16.201 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:32:16.201 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.854994+0000", 2024-09-14T06:32:16.201 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.091e-06 2024-09-14T06:32:16.201 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:16.202 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:16.202 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:32:16.202 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855001+0000", 2024-09-14T06:32:16.202 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.204e-06 2024-09-14T06:32:16.202 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:16.202 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:16.202 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:32:16.202 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855003+0000", 2024-09-14T06:32:16.202 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.514e-06 2024-09-14T06:32:16.202 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:16.202 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:16.202 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:32:16.202 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.918380+0000", 2024-09-14T06:32:16.202 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.063376738000000002 2024-09-14T06:32:16.202 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:16.202 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:16.203 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:32:16.203 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.918453+0000", 2024-09-14T06:32:16.203 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.3291e-05 2024-09-14T06:32:16.203 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:16.203 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:16.203 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:32:16.203 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.918600+0000", 2024-09-14T06:32:16.203 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00014674100000000001 2024-09-14T06:32:16.203 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:16.203 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:32:16.203 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:16.203 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:16.203 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:16.203 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:44428 3.2 3:4603ad7f:::rbd_data.112dca2f6ae1.0000000000000005:17 [read 942080~4096 [fadvise_nocache]] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:32:16.204 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:43.855006+0000", 2024-09-14T06:32:16.204 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 152.333635816, 2024-09-14T06:32:16.204 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.063846321999999997, 2024-09-14T06:32:16.204 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:32:16.204 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:32:16.204 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:32:16.204 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:32:16.204 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:32:16.204 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:16.204 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 44428 2024-09-14T06:32:16.204 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:16.204 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:32:16.204 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:16.204 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:32:16.205 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855006+0000", 2024-09-14T06:32:16.205 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:16.205 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:16.205 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:16.205 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:32:16.205 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855006+0000", 2024-09-14T06:32:16.205 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:16.205 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:16.205 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:16.205 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:32:16.205 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855007+0000", 2024-09-14T06:32:16.205 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.1199999999999999e-07 2024-09-14T06:32:16.205 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:16.205 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:16.206 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:32:16.206 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855011+0000", 2024-09-14T06:32:16.206 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.9539999999999998e-06 2024-09-14T06:32:16.206 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:16.206 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:16.206 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:32:16.206 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855014+0000", 2024-09-14T06:32:16.206 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.2930000000000001e-06 2024-09-14T06:32:16.206 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:16.206 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:16.206 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:32:16.206 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855016+0000", 2024-09-14T06:32:16.206 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.3039999999999999e-06 2024-09-14T06:32:16.206 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:16.207 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:16.207 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:32:16.207 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.918634+0000", 2024-09-14T06:32:16.207 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.063617928000000004 2024-09-14T06:32:16.207 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:16.207 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:16.207 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:32:16.207 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.918711+0000", 2024-09-14T06:32:16.207 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.6545000000000007e-05 2024-09-14T06:32:16.207 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:16.207 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:16.207 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:32:16.208 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.918853+0000", 2024-09-14T06:32:16.208 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00014208600000000001 2024-09-14T06:32:16.208 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:16.208 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:32:16.208 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:16.208 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:16.208 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:16.208 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:44429 3.2 3:4603ad7f:::rbd_data.112dca2f6ae1.0000000000000005:17 [read 966656~4096 [fadvise_nocache]] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:32:16.208 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:43.855019+0000", 2024-09-14T06:32:16.208 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 152.33362314300001, 2024-09-14T06:32:16.208 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.064059226999999996, 2024-09-14T06:32:16.208 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:32:16.208 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:32:16.208 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:32:16.209 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:32:16.209 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:32:16.209 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:16.209 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 44429 2024-09-14T06:32:16.209 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:16.209 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:32:16.209 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:16.209 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:32:16.209 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855019+0000", 2024-09-14T06:32:16.209 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:16.209 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:16.209 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:16.209 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:32:16.209 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855019+0000", 2024-09-14T06:32:16.209 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:16.210 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:16.210 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:16.210 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:32:16.210 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855019+0000", 2024-09-14T06:32:16.210 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.28e-07 2024-09-14T06:32:16.210 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:16.210 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:16.210 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:32:16.210 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855020+0000", 2024-09-14T06:32:16.210 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.0330000000000001e-06 2024-09-14T06:32:16.210 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:16.210 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:16.210 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:32:16.210 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855023+0000", 2024-09-14T06:32:16.211 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.79e-06 2024-09-14T06:32:16.211 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:16.211 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:16.211 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:32:16.211 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855025+0000", 2024-09-14T06:32:16.211 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.216e-06 2024-09-14T06:32:16.211 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:16.211 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:16.211 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:32:16.211 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.918886+0000", 2024-09-14T06:32:16.211 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.063861037999999995 2024-09-14T06:32:16.211 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:16.211 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:16.211 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:32:16.212 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.918952+0000", 2024-09-14T06:32:16.212 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.5993000000000003e-05 2024-09-14T06:32:16.212 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:16.212 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:16.212 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:32:16.212 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.919078+0000", 2024-09-14T06:32:16.212 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00012592899999999999 2024-09-14T06:32:16.212 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:16.212 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:32:16.212 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:16.213 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:16.213 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:16.213 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:44430 3.2 3:4603ad7f:::rbd_data.112dca2f6ae1.0000000000000005:17 [read 991232~4096 [fadvise_nocache]] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:32:16.213 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:43.855028+0000", 2024-09-14T06:32:16.213 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 152.333614133, 2024-09-14T06:32:16.213 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.064284346000000006, 2024-09-14T06:32:16.213 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:32:16.213 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:32:16.213 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:32:16.213 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:32:16.213 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:32:16.213 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:16.213 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 44430 2024-09-14T06:32:16.213 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:16.214 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:32:16.214 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:16.214 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:32:16.214 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855028+0000", 2024-09-14T06:32:16.214 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:16.214 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:16.214 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:16.214 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:32:16.214 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855028+0000", 2024-09-14T06:32:16.214 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:16.214 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:16.214 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:16.214 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:32:16.214 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855028+0000", 2024-09-14T06:32:16.215 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.1199999999999999e-07 2024-09-14T06:32:16.215 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:16.215 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:16.215 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:32:16.215 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855029+0000", 2024-09-14T06:32:16.215 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.1100000000000004e-07 2024-09-14T06:32:16.215 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:16.215 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:16.215 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:32:16.215 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855032+0000", 2024-09-14T06:32:16.215 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.7889999999999999e-06 2024-09-14T06:32:16.215 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:16.215 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:16.215 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:32:16.215 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855034+0000", 2024-09-14T06:32:16.216 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.125e-06 2024-09-14T06:32:16.216 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:16.216 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:16.216 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:32:16.216 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.919110+0000", 2024-09-14T06:32:16.216 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.064076413999999998 2024-09-14T06:32:16.216 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:16.216 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:16.216 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:32:16.216 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.919186+0000", 2024-09-14T06:32:16.216 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.5784999999999994e-05 2024-09-14T06:32:16.216 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:16.216 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:16.216 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:32:16.217 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.919312+0000", 2024-09-14T06:32:16.217 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00012611 2024-09-14T06:32:16.217 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:16.217 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:32:16.217 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:16.217 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:16.217 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:16.217 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:44431 3.2 3:4603ad7f:::rbd_data.112dca2f6ae1.0000000000000005:17 [read 1015808~4096 [fadvise_nocache]] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:32:16.217 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:43.855037+0000", 2024-09-14T06:32:16.217 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 152.33360526499999, 2024-09-14T06:32:16.217 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.064469755000000004, 2024-09-14T06:32:16.217 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:32:16.217 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:32:16.217 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:32:16.218 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:32:16.218 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:32:16.218 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:16.218 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 44431 2024-09-14T06:32:16.218 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:16.218 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:32:16.218 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:16.218 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:32:16.218 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855037+0000", 2024-09-14T06:32:16.218 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:16.218 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:16.218 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:16.218 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:32:16.218 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855037+0000", 2024-09-14T06:32:16.219 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:16.219 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:16.219 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:16.219 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:32:16.219 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855037+0000", 2024-09-14T06:32:16.219 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.1e-07 2024-09-14T06:32:16.219 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:16.219 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:16.219 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:32:16.219 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855038+0000", 2024-09-14T06:32:16.219 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.9400000000000004e-07 2024-09-14T06:32:16.219 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:16.219 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:16.219 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:32:16.220 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855041+0000", 2024-09-14T06:32:16.220 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.9639999999999999e-06 2024-09-14T06:32:16.220 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:16.220 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:16.220 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:32:16.220 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855043+0000", 2024-09-14T06:32:16.220 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.2259999999999999e-06 2024-09-14T06:32:16.220 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:16.220 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:16.220 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:32:16.220 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.919347+0000", 2024-09-14T06:32:16.220 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.064304091999999993 2024-09-14T06:32:16.220 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:16.220 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:16.221 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:32:16.221 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.919393+0000", 2024-09-14T06:32:16.221 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.5340999999999998e-05 2024-09-14T06:32:16.221 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:16.221 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:16.221 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:32:16.221 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.919507+0000", 2024-09-14T06:32:16.221 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000114028 2024-09-14T06:32:16.221 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:16.221 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:32:16.221 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:16.221 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:16.221 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:16.222 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:44432 3.2 3:4603ad7f:::rbd_data.112dca2f6ae1.0000000000000005:17 [read 1040384~4096 [fadvise_nocache]] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:32:16.222 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:43.855046+0000", 2024-09-14T06:32:16.222 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 152.33359619699999, 2024-09-14T06:32:16.222 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.064639253999999993, 2024-09-14T06:32:16.222 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:32:16.222 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:32:16.222 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:32:16.222 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:32:16.222 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:32:16.222 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:16.222 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 44432 2024-09-14T06:32:16.222 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:16.222 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:32:16.222 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:16.223 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:32:16.223 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855046+0000", 2024-09-14T06:32:16.223 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:16.223 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:16.223 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:16.223 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:32:16.223 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855046+0000", 2024-09-14T06:32:16.223 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:16.223 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:16.223 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:16.223 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:32:16.223 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855046+0000", 2024-09-14T06:32:16.223 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.05e-07 2024-09-14T06:32:16.223 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:16.224 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:16.224 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:32:16.224 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855047+0000", 2024-09-14T06:32:16.224 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.879999999999999e-07 2024-09-14T06:32:16.224 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:16.224 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:16.224 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:32:16.224 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855050+0000", 2024-09-14T06:32:16.224 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.8459999999999999e-06 2024-09-14T06:32:16.224 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:16.224 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:16.224 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:32:16.224 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855052+0000", 2024-09-14T06:32:16.224 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.136e-06 2024-09-14T06:32:16.225 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:16.225 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:16.225 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:32:16.225 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.919536+0000", 2024-09-14T06:32:16.225 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.064484132 2024-09-14T06:32:16.225 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:16.225 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:16.225 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:32:16.225 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.919611+0000", 2024-09-14T06:32:16.225 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.4407999999999996e-05 2024-09-14T06:32:16.225 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:16.225 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:16.225 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:32:16.226 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.919685+0000", 2024-09-14T06:32:16.226 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.4538999999999999e-05 2024-09-14T06:32:16.226 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:16.226 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:32:16.226 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:16.226 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:16.226 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:16.226 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:44518 3.2 3:4190d882:::rbd_data.112dca2f6ae1.0000000000000002:17 [read 12288~4096 [fadvise_nocache]] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:32:16.226 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:43.855055+0000", 2024-09-14T06:32:16.226 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 152.33358731300001, 2024-09-14T06:32:16.226 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.064894317000000007, 2024-09-14T06:32:16.226 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:32:16.226 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:32:16.226 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:32:16.226 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:32:16.227 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:32:16.227 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:16.227 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 44518 2024-09-14T06:32:16.227 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:16.227 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:32:16.227 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:16.227 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:32:16.227 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855055+0000", 2024-09-14T06:32:16.227 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:16.227 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:16.227 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:16.227 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:32:16.227 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855055+0000", 2024-09-14T06:32:16.227 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:16.228 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:16.228 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:16.228 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:32:16.228 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855055+0000", 2024-09-14T06:32:16.228 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.0699999999999999e-07 2024-09-14T06:32:16.228 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:16.228 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:16.228 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:32:16.228 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855056+0000", 2024-09-14T06:32:16.228 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.9400000000000004e-07 2024-09-14T06:32:16.228 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:16.228 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:16.228 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:32:16.228 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855059+0000", 2024-09-14T06:32:16.229 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.8870000000000001e-06 2024-09-14T06:32:16.229 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:16.229 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:16.229 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:32:16.229 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855061+0000", 2024-09-14T06:32:16.229 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.1919999999999999e-06 2024-09-14T06:32:16.229 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:16.229 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:16.229 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:32:16.229 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.919719+0000", 2024-09-14T06:32:16.229 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.064658013 2024-09-14T06:32:16.229 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:16.229 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:16.229 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:32:16.229 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.919788+0000", 2024-09-14T06:32:16.230 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.9282000000000005e-05 2024-09-14T06:32:16.230 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:16.230 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:16.230 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:32:16.230 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.919949+0000", 2024-09-14T06:32:16.230 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00016084199999999999 2024-09-14T06:32:16.230 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:16.230 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:32:16.230 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:16.230 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:16.230 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:16.230 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:44519 3.2 3:4190d882:::rbd_data.112dca2f6ae1.0000000000000002:17 [read 36864~4096 [fadvise_nocache]] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:32:16.230 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:43.855064+0000", 2024-09-14T06:32:16.230 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 152.33357833599999, 2024-09-14T06:32:16.231 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.065081865000000003, 2024-09-14T06:32:16.231 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:32:16.231 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:32:16.231 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:32:16.231 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:32:16.231 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:32:16.231 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:16.231 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 44519 2024-09-14T06:32:16.231 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:16.231 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:32:16.231 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:16.231 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:32:16.231 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855064+0000", 2024-09-14T06:32:16.231 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:16.232 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:16.232 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:16.232 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:32:16.232 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855064+0000", 2024-09-14T06:32:16.232 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:16.232 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:16.232 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:16.232 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:32:16.232 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855064+0000", 2024-09-14T06:32:16.232 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.1e-07 2024-09-14T06:32:16.232 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:16.232 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:16.232 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:32:16.233 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855065+0000", 2024-09-14T06:32:16.233 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.035e-06 2024-09-14T06:32:16.233 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:16.233 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:16.233 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:32:16.233 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855068+0000", 2024-09-14T06:32:16.233 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.8339999999999999e-06 2024-09-14T06:32:16.233 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:16.233 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:16.233 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:32:16.233 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855071+0000", 2024-09-14T06:32:16.233 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.3450000000000002e-06 2024-09-14T06:32:16.233 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:16.234 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:16.234 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:32:16.234 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.919979+0000", 2024-09-14T06:32:16.234 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.064907647999999998 2024-09-14T06:32:16.234 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:16.234 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:16.234 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:32:16.234 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.920026+0000", 2024-09-14T06:32:16.234 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.7595000000000001e-05 2024-09-14T06:32:16.234 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:16.234 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:16.234 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:32:16.234 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.920146+0000", 2024-09-14T06:32:16.234 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00011919800000000001 2024-09-14T06:32:16.234 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:16.235 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:32:16.235 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:16.235 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:16.235 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:16.235 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:44520 3.2 3:4190d882:::rbd_data.112dca2f6ae1.0000000000000002:17 [read 61440~4096 [fadvise_nocache]] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:32:16.235 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:43.855075+0000", 2024-09-14T06:32:16.235 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 152.33356728499999, 2024-09-14T06:32:16.235 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.065335078000000005, 2024-09-14T06:32:16.235 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:32:16.235 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:32:16.235 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:32:16.235 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:32:16.235 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:32:16.235 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:16.236 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 44520 2024-09-14T06:32:16.236 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:16.236 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:32:16.236 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:16.236 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:32:16.236 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855075+0000", 2024-09-14T06:32:16.236 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:16.236 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:16.236 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:16.236 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:32:16.236 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855075+0000", 2024-09-14T06:32:16.236 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:16.236 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:16.236 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:16.236 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:32:16.237 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855075+0000", 2024-09-14T06:32:16.237 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.1e-07 2024-09-14T06:32:16.237 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:16.237 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:16.237 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:32:16.237 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855076+0000", 2024-09-14T06:32:16.237 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.0839999999999999e-06 2024-09-14T06:32:16.237 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:16.237 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:16.237 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:32:16.237 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855079+0000", 2024-09-14T06:32:16.237 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.9359999999999999e-06 2024-09-14T06:32:16.237 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:16.237 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:16.237 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:32:16.238 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855084+0000", 2024-09-14T06:32:16.238 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.1730000000000003e-06 2024-09-14T06:32:16.238 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:16.238 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:16.238 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:32:16.238 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.920175+0000", 2024-09-14T06:32:16.238 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.065090234999999996 2024-09-14T06:32:16.238 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:16.238 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:16.238 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:32:16.238 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.920250+0000", 2024-09-14T06:32:16.238 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.5807000000000001e-05 2024-09-14T06:32:16.238 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:16.238 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:16.238 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:32:16.239 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.920410+0000", 2024-09-14T06:32:16.239 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00015963300000000001 2024-09-14T06:32:16.239 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:16.239 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:32:16.239 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:16.239 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:16.239 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:16.239 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:44521 3.2 3:4190d882:::rbd_data.112dca2f6ae1.0000000000000002:17 [read 86016~4096 [fadvise_nocache]] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:32:16.239 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:43.855087+0000", 2024-09-14T06:32:16.239 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 152.33355480399999, 2024-09-14T06:32:16.239 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.065563264999999996, 2024-09-14T06:32:16.239 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:32:16.239 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:32:16.239 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:32:16.239 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:32:16.240 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:32:16.240 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:16.240 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 44521 2024-09-14T06:32:16.240 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:16.240 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:32:16.240 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:16.240 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:32:16.240 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855087+0000", 2024-09-14T06:32:16.240 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:16.240 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:16.240 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:16.240 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:32:16.240 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855087+0000", 2024-09-14T06:32:16.241 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:16.241 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:16.241 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:16.241 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:32:16.241 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855088+0000", 2024-09-14T06:32:16.241 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.3699999999999999e-07 2024-09-14T06:32:16.241 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:16.241 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:16.241 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:32:16.241 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855089+0000", 2024-09-14T06:32:16.241 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.9900000000000009e-07 2024-09-14T06:32:16.241 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:16.241 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:16.241 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:32:16.242 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855091+0000", 2024-09-14T06:32:16.242 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.8940000000000002e-06 2024-09-14T06:32:16.242 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:16.242 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:16.242 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:32:16.242 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855094+0000", 2024-09-14T06:32:16.242 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.351e-06 2024-09-14T06:32:16.242 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:16.242 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:16.242 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:32:16.242 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.920437+0000", 2024-09-14T06:32:16.242 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.065343221000000007 2024-09-14T06:32:16.242 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:16.242 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:16.243 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:32:16.243 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.920481+0000", 2024-09-14T06:32:16.243 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.4431000000000002e-05 2024-09-14T06:32:16.243 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:16.243 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:16.243 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:32:16.243 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.920651+0000", 2024-09-14T06:32:16.243 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00016913200000000001 2024-09-14T06:32:16.243 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:16.243 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:32:16.243 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:16.243 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:16.243 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:16.243 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:44522 3.2 3:4190d882:::rbd_data.112dca2f6ae1.0000000000000002:17 [read 110592~4096 [fadvise_nocache]] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:32:16.244 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:43.855098+0000", 2024-09-14T06:32:16.244 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 152.333544577, 2024-09-14T06:32:16.244 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.065827628999999999, 2024-09-14T06:32:16.244 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:32:16.244 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:32:16.244 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:32:16.244 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:32:16.244 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:32:16.244 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:16.244 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 44522 2024-09-14T06:32:16.244 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:16.244 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:32:16.244 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:16.245 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:32:16.245 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855098+0000", 2024-09-14T06:32:16.245 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:16.245 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:16.245 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:16.245 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:32:16.245 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855098+0000", 2024-09-14T06:32:16.245 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:16.245 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:16.245 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:16.245 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:32:16.245 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855098+0000", 2024-09-14T06:32:16.245 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.1e-07 2024-09-14T06:32:16.245 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:16.246 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:16.246 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:32:16.246 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855099+0000", 2024-09-14T06:32:16.246 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.012e-06 2024-09-14T06:32:16.246 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:16.246 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:16.246 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:32:16.246 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855102+0000", 2024-09-14T06:32:16.246 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.9380000000000001e-06 2024-09-14T06:32:16.246 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:16.246 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:16.246 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:32:16.246 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855108+0000", 2024-09-14T06:32:16.246 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.4339999999999997e-06 2024-09-14T06:32:16.247 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:16.247 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:16.247 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:32:16.247 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.920681+0000", 2024-09-14T06:32:16.247 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.065572943999999994 2024-09-14T06:32:16.247 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:16.247 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:16.247 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:32:16.247 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.920724+0000", 2024-09-14T06:32:16.247 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.2877000000000003e-05 2024-09-14T06:32:16.247 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:16.247 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:16.247 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:32:16.247 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.920925+0000", 2024-09-14T06:32:16.248 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00020121399999999999 2024-09-14T06:32:16.248 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:16.248 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:32:16.248 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:16.248 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:16.248 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:16.248 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:44523 3.2 3:4190d882:::rbd_data.112dca2f6ae1.0000000000000002:17 [read 135168~4096 [fadvise_nocache]] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:32:16.248 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:43.855111+0000", 2024-09-14T06:32:16.248 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 152.33353085499999, 2024-09-14T06:32:16.248 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.066028198999999996, 2024-09-14T06:32:16.248 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:32:16.248 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:32:16.248 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:32:16.248 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:32:16.248 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:32:16.248 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:16.249 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 44523 2024-09-14T06:32:16.249 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:16.249 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:32:16.249 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:16.249 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:32:16.249 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855111+0000", 2024-09-14T06:32:16.249 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:16.249 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:16.249 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:16.249 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:32:16.249 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855111+0000", 2024-09-14T06:32:16.249 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:16.249 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:16.249 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:16.250 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:32:16.250 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855112+0000", 2024-09-14T06:32:16.250 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.1199999999999999e-07 2024-09-14T06:32:16.250 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:16.250 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:16.250 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:32:16.250 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855115+0000", 2024-09-14T06:32:16.250 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.3380000000000001e-06 2024-09-14T06:32:16.250 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:16.250 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:16.250 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:32:16.250 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855118+0000", 2024-09-14T06:32:16.250 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.366e-06 2024-09-14T06:32:16.250 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:16.251 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:16.251 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:32:16.251 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855121+0000", 2024-09-14T06:32:16.251 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.8329999999999998e-06 2024-09-14T06:32:16.251 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:16.251 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:16.251 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:32:16.251 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.920956+0000", 2024-09-14T06:32:16.251 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.065835251999999997 2024-09-14T06:32:16.251 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:16.251 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:16.251 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:32:16.251 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.921001+0000", 2024-09-14T06:32:16.251 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.4778999999999999e-05 2024-09-14T06:32:16.251 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:16.251 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:16.252 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:32:16.252 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.921139+0000", 2024-09-14T06:32:16.252 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000138419 2024-09-14T06:32:16.252 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:16.252 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:32:16.252 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:16.252 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:16.252 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:16.252 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:44524 3.2 3:4190d882:::rbd_data.112dca2f6ae1.0000000000000002:17 [read 159744~4096 [fadvise_nocache]] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:32:16.252 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:43.855124+0000", 2024-09-14T06:32:16.252 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 152.333517876, 2024-09-14T06:32:16.252 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.066249261000000004, 2024-09-14T06:32:16.252 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:32:16.252 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:32:16.253 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:32:16.253 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:32:16.253 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:32:16.253 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:16.253 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 44524 2024-09-14T06:32:16.253 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:16.253 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:32:16.253 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:16.253 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:32:16.253 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855124+0000", 2024-09-14T06:32:16.253 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:16.253 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:16.253 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:16.253 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:32:16.254 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855124+0000", 2024-09-14T06:32:16.254 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:16.254 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:16.254 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:16.254 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:32:16.254 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861086+0000", 2024-09-14T06:32:16.254 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0059613829999999998 2024-09-14T06:32:16.254 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:16.254 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:16.254 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:32:16.254 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861100+0000", 2024-09-14T06:32:16.254 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.4644000000000001e-05 2024-09-14T06:32:16.254 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:16.255 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:16.255 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:32:16.255 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861109+0000", 2024-09-14T06:32:16.255 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.541e-06 2024-09-14T06:32:16.255 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:16.255 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:16.255 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:32:16.255 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861117+0000", 2024-09-14T06:32:16.255 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.1049999999999995e-06 2024-09-14T06:32:16.255 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:16.255 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:16.255 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:32:16.255 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.921169+0000", 2024-09-14T06:32:16.255 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.060052405000000003 2024-09-14T06:32:16.255 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:16.256 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:16.256 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:32:16.256 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.921213+0000", 2024-09-14T06:32:16.256 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.3565e-05 2024-09-14T06:32:16.256 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:16.256 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:16.256 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:32:16.256 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.921374+0000", 2024-09-14T06:32:16.256 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00016061800000000001 2024-09-14T06:32:16.256 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:16.256 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:32:16.256 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:16.256 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:16.256 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:16.256 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:44537 3.2 3:4190d882:::rbd_data.112dca2f6ae1.0000000000000002:17 [read 479232~4096 [fadvise_nocache]] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:32:16.257 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:43.861405+0000", 2024-09-14T06:32:16.257 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 152.32723727300001, 2024-09-14T06:32:16.257 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.063351795000000002, 2024-09-14T06:32:16.257 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:32:16.257 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:32:16.257 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:32:16.257 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:32:16.257 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:32:16.257 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:16.257 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 44537 2024-09-14T06:32:16.257 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:16.257 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:32:16.257 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:16.257 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:32:16.257 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861405+0000", 2024-09-14T06:32:16.258 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:16.258 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:16.258 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:16.258 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:32:16.258 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861405+0000", 2024-09-14T06:32:16.258 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:16.258 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:16.258 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:16.258 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:32:16.258 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861405+0000", 2024-09-14T06:32:16.258 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.9099999999999999e-07 2024-09-14T06:32:16.258 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:16.258 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:16.258 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:32:16.259 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861407+0000", 2024-09-14T06:32:16.259 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.075e-06 2024-09-14T06:32:16.259 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:16.259 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:16.259 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:32:16.259 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861414+0000", 2024-09-14T06:32:16.259 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.4659999999999996e-06 2024-09-14T06:32:16.259 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:16.259 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:16.259 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:32:16.259 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861418+0000", 2024-09-14T06:32:16.259 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.5519999999999998e-06 2024-09-14T06:32:16.259 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:16.259 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:16.260 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:32:16.260 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.924474+0000", 2024-09-14T06:32:16.260 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.063056012999999994 2024-09-14T06:32:16.260 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:16.260 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:16.260 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:32:16.260 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.924556+0000", 2024-09-14T06:32:16.260 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.1181999999999996e-05 2024-09-14T06:32:16.260 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:16.260 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:16.260 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:32:16.260 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.924757+0000", 2024-09-14T06:32:16.260 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00020111599999999999 2024-09-14T06:32:16.261 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:16.261 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:32:16.261 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:16.261 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:16.261 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:16.261 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:44538 3.2 3:4190d882:::rbd_data.112dca2f6ae1.0000000000000002:17 [read 503808~4096 [fadvise_nocache]] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:32:16.261 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:43.861425+0000", 2024-09-14T06:32:16.261 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 152.32721723200001, 2024-09-14T06:32:16.261 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.063627125000000007, 2024-09-14T06:32:16.261 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:32:16.261 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:32:16.261 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:32:16.261 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:32:16.261 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:32:16.261 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:16.262 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 44538 2024-09-14T06:32:16.262 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:16.262 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:32:16.262 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:16.262 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:32:16.262 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861425+0000", 2024-09-14T06:32:16.262 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:16.262 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:16.262 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:16.262 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:32:16.262 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861425+0000", 2024-09-14T06:32:16.262 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:16.262 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:16.262 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:16.262 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:32:16.262 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861425+0000", 2024-09-14T06:32:16.263 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.9999999999999998e-07 2024-09-14T06:32:16.263 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:16.263 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:16.263 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:32:16.263 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861427+0000", 2024-09-14T06:32:16.263 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.9829999999999999e-06 2024-09-14T06:32:16.263 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:16.263 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:16.263 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:32:16.263 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861433+0000", 2024-09-14T06:32:16.263 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.2889999999999997e-06 2024-09-14T06:32:16.263 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:16.263 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:16.263 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:32:16.263 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861454+0000", 2024-09-14T06:32:16.263 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.1844999999999998e-05 2024-09-14T06:32:16.264 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:16.264 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:16.264 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:32:16.264 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.924786+0000", 2024-09-14T06:32:16.264 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.063331497 2024-09-14T06:32:16.264 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:16.264 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:16.264 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:32:16.264 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.924864+0000", 2024-09-14T06:32:16.264 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.7744000000000006e-05 2024-09-14T06:32:16.264 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:16.264 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:16.264 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:32:16.264 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.925052+0000", 2024-09-14T06:32:16.264 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00018836699999999999 2024-09-14T06:32:16.265 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:16.265 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:32:16.265 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:16.265 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:16.265 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:16.265 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:44539 3.2 3:4190d882:::rbd_data.112dca2f6ae1.0000000000000002:17 [read 528384~4096 [fadvise_nocache]] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:32:16.265 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:43.861462+0000", 2024-09-14T06:32:16.265 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 152.32717990500001, 2024-09-14T06:32:16.265 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.063855705999999998, 2024-09-14T06:32:16.266 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:32:16.266 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:32:16.266 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:32:16.266 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:32:16.266 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:32:16.266 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:16.266 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 44539 2024-09-14T06:32:16.266 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:16.266 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:32:16.266 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:16.266 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:32:16.266 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861462+0000", 2024-09-14T06:32:16.267 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:16.267 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:16.267 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:16.267 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:32:16.267 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861462+0000", 2024-09-14T06:32:16.267 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:16.267 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:16.267 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:16.267 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:32:16.267 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861463+0000", 2024-09-14T06:32:16.267 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.5200000000000002e-07 2024-09-14T06:32:16.267 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:16.267 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:16.267 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:32:16.268 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861465+0000", 2024-09-14T06:32:16.268 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.5189999999999999e-06 2024-09-14T06:32:16.268 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:16.268 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:16.268 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:32:16.268 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861471+0000", 2024-09-14T06:32:16.268 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.6899999999999997e-06 2024-09-14T06:32:16.268 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:16.268 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:16.268 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:32:16.268 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861475+0000", 2024-09-14T06:32:16.268 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.5730000000000001e-06 2024-09-14T06:32:16.268 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:16.268 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:16.269 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:32:16.269 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.925085+0000", 2024-09-14T06:32:16.269 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.063609902999999995 2024-09-14T06:32:16.269 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:16.269 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:16.269 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:32:16.269 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.925145+0000", 2024-09-14T06:32:16.269 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.9383999999999999e-05 2024-09-14T06:32:16.269 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:16.269 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:16.269 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:32:16.269 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.925318+0000", 2024-09-14T06:32:16.269 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00017318499999999999 2024-09-14T06:32:16.269 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:16.270 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:32:16.270 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:16.270 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:16.270 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:16.270 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:44540 3.2 3:4190d882:::rbd_data.112dca2f6ae1.0000000000000002:17 [read 552960~4096 [fadvise_nocache]] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:32:16.270 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:43.861483+0000", 2024-09-14T06:32:16.270 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 152.32715947400001, 2024-09-14T06:32:16.270 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.064065681999999999, 2024-09-14T06:32:16.270 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:32:16.270 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:32:16.270 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:32:16.270 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:32:16.270 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:32:16.270 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:16.270 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 44540 2024-09-14T06:32:16.271 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:16.271 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:32:16.271 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:16.271 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:32:16.271 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861483+0000", 2024-09-14T06:32:16.271 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:16.271 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:16.271 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:16.271 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:32:16.271 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861483+0000", 2024-09-14T06:32:16.271 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:16.271 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:16.271 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:16.271 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:32:16.271 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.867079+0000", 2024-09-14T06:32:16.272 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0055966480000000001 2024-09-14T06:32:16.272 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:16.272 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:16.272 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:32:16.272 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.867086+0000", 2024-09-14T06:32:16.272 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.387e-06 2024-09-14T06:32:16.272 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:16.272 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:16.272 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:32:16.272 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.867094+0000", 2024-09-14T06:32:16.272 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.9419999999999995e-06 2024-09-14T06:32:16.272 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:16.272 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:16.273 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:32:16.273 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.867109+0000", 2024-09-14T06:32:16.273 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.5818999999999999e-05 2024-09-14T06:32:16.273 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:16.273 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:16.273 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:32:16.273 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.925354+0000", 2024-09-14T06:32:16.273 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.058244047 2024-09-14T06:32:16.273 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:16.273 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:16.273 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:32:16.273 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.925406+0000", 2024-09-14T06:32:16.273 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.2661999999999998e-05 2024-09-14T06:32:16.273 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:16.273 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:16.274 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:32:16.274 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.925548+0000", 2024-09-14T06:32:16.274 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000142177 2024-09-14T06:32:16.274 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:16.274 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:32:16.274 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:16.274 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:16.274 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:16.274 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:225587 3.7 3:e112e2b3:::rbd_id.image_client.0-clone42:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e189)", 2024-09-14T06:32:16.274 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:31:53.874591+0000", 2024-09-14T06:32:16.274 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 22.314051256999999, 2024-09-14T06:32:16.274 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.875541507, 2024-09-14T06:32:16.274 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:32:16.274 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:32:16.275 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:32:16.275 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:32:16.275 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:32:16.275 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:16.275 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 225587 2024-09-14T06:32:16.275 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:16.275 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:32:16.275 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:16.275 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:32:16.275 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:53.874591+0000", 2024-09-14T06:32:16.275 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:16.275 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:16.276 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:16.276 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:32:16.276 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:53.874591+0000", 2024-09-14T06:32:16.276 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:16.276 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:16.276 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:16.276 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:32:16.276 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:53.874592+0000", 2024-09-14T06:32:16.276 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.5209999999999999e-06 2024-09-14T06:32:16.276 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:16.276 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:16.276 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:32:16.276 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:53.874599+0000", 2024-09-14T06:32:16.277 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.6290000000000004e-06 2024-09-14T06:32:16.277 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:16.277 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:16.277 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:32:16.277 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:53.874618+0000", 2024-09-14T06:32:16.277 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.8950999999999999e-05 2024-09-14T06:32:16.277 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:16.277 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:16.277 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:32:16.277 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:53.874626+0000", 2024-09-14T06:32:16.277 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.3960000000000006e-06 2024-09-14T06:32:16.277 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:16.277 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:16.277 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:32:16.278 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:53.874699+0000", 2024-09-14T06:32:16.278 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.2329999999999994e-05 2024-09-14T06:32:16.278 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:16.278 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:16.278 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "waiting for peered", 2024-09-14T06:32:16.278 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:53.874711+0000", 2024-09-14T06:32:16.278 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.2473999999999999e-05 2024-09-14T06:32:16.278 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:16.278 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:16.278 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:32:16.278 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:54.749578+0000", 2024-09-14T06:32:16.278 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.87486642100000001 2024-09-14T06:32:16.278 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:16.278 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:16.278 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:32:16.279 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:54.749706+0000", 2024-09-14T06:32:16.279 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00012856200000000001 2024-09-14T06:32:16.279 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:16.279 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:16.279 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:32:16.279 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:54.750132+0000", 2024-09-14T06:32:16.279 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00042622299999999999 2024-09-14T06:32:16.279 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:16.279 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:32:16.279 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:16.279 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:16.279 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:32:16.279 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-14T06:32:16.281 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-14T06:32:16.281 INFO:tasks.ceph.osd.6.smithi138.stderr:2024-09-14T06:32:16.279+0000 7f8cd8849640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33620) UID: 0 2024-09-14T06:32:16.381 INFO:tasks.ceph.osd.5.smithi138.stderr:2024-09-14T06:32:16.380+0000 7fe13746f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33618) UID: 0 2024-09-14T06:32:16.395 INFO:teuthology.orchestra.run.smithi138.stdout:{ 2024-09-14T06:32:16.395 INFO:teuthology.orchestra.run.smithi138.stdout: "ops": [], 2024-09-14T06:32:16.395 INFO:teuthology.orchestra.run.smithi138.stdout: "num_ops": 0 2024-09-14T06:32:16.396 INFO:teuthology.orchestra.run.smithi138.stdout:} 2024-09-14T06:32:16.405 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-14T06:32:16.482 INFO:tasks.ceph.osd.2.smithi094.stderr:2024-09-14T06:32:16.481+0000 7faf3269b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33649) UID: 0 2024-09-14T06:32:16.550 INFO:teuthology.orchestra.run.smithi138.stdout:{ 2024-09-14T06:32:16.550 INFO:teuthology.orchestra.run.smithi138.stdout: "ops": [], 2024-09-14T06:32:16.550 INFO:teuthology.orchestra.run.smithi138.stdout: "complaint_time": 30, 2024-09-14T06:32:16.550 INFO:teuthology.orchestra.run.smithi138.stdout: "num_blocked_ops": 0 2024-09-14T06:32:16.550 INFO:teuthology.orchestra.run.smithi138.stdout:} 2024-09-14T06:32:16.558 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-14T06:32:16.583 INFO:tasks.ceph.osd.2.smithi094.stderr:2024-09-14T06:32:16.582+0000 7faf3269b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33649) UID: 0 2024-09-14T06:32:16.664 INFO:teuthology.orchestra.run.smithi138.stdout:{ 2024-09-14T06:32:16.664 INFO:teuthology.orchestra.run.smithi138.stdout: "size": 20, 2024-09-14T06:32:16.664 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 600, 2024-09-14T06:32:16.664 INFO:teuthology.orchestra.run.smithi138.stdout: "ops": [ 2024-09-14T06:32:16.664 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:16.664 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:19328 3.3 3:c34e5e51:::rbd_data.112d92027590.0000000000000024:f [read 1926144~1024 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:32:16.664 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:27.059768+0000", 2024-09-14T06:32:16.664 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 169.60235086200001, 2024-09-14T06:32:16.664 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046369147999999999, 2024-09-14T06:32:16.664 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:32:16.664 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:32:16.665 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:32:16.665 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:32:16.665 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:32:16.665 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:16.665 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 19328 2024-09-14T06:32:16.665 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:16.665 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:32:16.665 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:16.665 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:32:16.665 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.059768+0000", 2024-09-14T06:32:16.665 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:16.665 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:16.665 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:16.665 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:32:16.666 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.059768+0000", 2024-09-14T06:32:16.666 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:16.666 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:16.666 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:16.666 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:32:16.666 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.059874+0000", 2024-09-14T06:32:16.666 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00010551400000000001 2024-09-14T06:32:16.666 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:16.666 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:16.666 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:32:16.666 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.059877+0000", 2024-09-14T06:32:16.666 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.8279999999999999e-06 2024-09-14T06:32:16.666 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:16.666 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:16.667 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:32:16.667 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.059884+0000", 2024-09-14T06:32:16.667 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.6389999999999992e-06 2024-09-14T06:32:16.667 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:16.667 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:16.667 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:32:16.667 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.059890+0000", 2024-09-14T06:32:16.667 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.49e-06 2024-09-14T06:32:16.667 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:16.667 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:16.667 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:32:16.667 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.105804+0000", 2024-09-14T06:32:16.667 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.045914242000000001 2024-09-14T06:32:16.668 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:16.668 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:16.668 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:32:16.668 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.105886+0000", 2024-09-14T06:32:16.668 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.2305999999999994e-05 2024-09-14T06:32:16.668 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:16.668 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:16.668 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:32:16.668 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.106138+0000", 2024-09-14T06:32:16.668 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.000251129 2024-09-14T06:32:16.668 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:16.668 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:32:16.668 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:16.668 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:16.668 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:16.669 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:19329 3.3 3:c34e5e51:::rbd_data.112d92027590.0000000000000024:f [read 1936384~1024 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:32:16.669 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:27.059899+0000", 2024-09-14T06:32:16.669 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 169.60222033900001, 2024-09-14T06:32:16.669 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046572122000000001, 2024-09-14T06:32:16.669 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:32:16.669 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:32:16.669 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:32:16.669 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:32:16.669 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:32:16.669 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:16.669 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 19329 2024-09-14T06:32:16.669 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:16.669 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:32:16.669 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:16.670 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:32:16.670 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.059899+0000", 2024-09-14T06:32:16.670 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:16.670 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:16.670 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:16.670 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:32:16.670 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.059899+0000", 2024-09-14T06:32:16.670 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:16.670 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:16.670 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:16.670 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:32:16.670 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062501+0000", 2024-09-14T06:32:16.670 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.0026016419999999999 2024-09-14T06:32:16.670 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:16.671 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:16.671 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:32:16.671 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062514+0000", 2024-09-14T06:32:16.671 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.2947e-05 2024-09-14T06:32:16.671 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:16.671 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:16.671 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:32:16.671 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062523+0000", 2024-09-14T06:32:16.671 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.5519999999999993e-06 2024-09-14T06:32:16.671 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:16.671 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:16.671 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:32:16.671 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062534+0000", 2024-09-14T06:32:16.671 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.1416e-05 2024-09-14T06:32:16.672 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:16.672 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:16.672 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:32:16.672 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.106179+0000", 2024-09-14T06:32:16.672 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.043644049999999997 2024-09-14T06:32:16.672 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:16.672 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:16.672 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:32:16.672 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.106262+0000", 2024-09-14T06:32:16.672 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.3178999999999998e-05 2024-09-14T06:32:16.672 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:16.672 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:16.672 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:32:16.672 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.106471+0000", 2024-09-14T06:32:16.673 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00020933599999999999 2024-09-14T06:32:16.673 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:16.673 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:32:16.673 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:16.673 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:16.673 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:16.673 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:19351 3.3 3:c34e5e51:::rbd_data.112d92027590.0000000000000024:f [read 1159168~1024 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:32:16.673 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:27.062766+0000", 2024-09-14T06:32:16.673 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 169.599353114, 2024-09-14T06:32:16.673 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046099454999999998, 2024-09-14T06:32:16.673 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:32:16.673 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:32:16.673 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:32:16.673 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:32:16.674 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:32:16.674 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:16.674 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 19351 2024-09-14T06:32:16.674 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:16.674 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:32:16.674 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:16.674 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:32:16.674 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062766+0000", 2024-09-14T06:32:16.674 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:16.674 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:16.674 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:16.674 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:32:16.674 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062766+0000", 2024-09-14T06:32:16.675 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:16.675 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:16.675 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:16.675 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:32:16.675 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062766+0000", 2024-09-14T06:32:16.675 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.6e-07 2024-09-14T06:32:16.675 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:16.675 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:16.675 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:32:16.675 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062767+0000", 2024-09-14T06:32:16.675 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.0070000000000001e-06 2024-09-14T06:32:16.675 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:16.675 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:16.675 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:32:16.676 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062785+0000", 2024-09-14T06:32:16.676 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.7884999999999999e-05 2024-09-14T06:32:16.676 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:16.676 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:16.676 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:32:16.676 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062790+0000", 2024-09-14T06:32:16.676 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.0900000000000004e-06 2024-09-14T06:32:16.676 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:16.676 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:16.676 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:32:16.676 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.108735+0000", 2024-09-14T06:32:16.676 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.045944411999999997 2024-09-14T06:32:16.676 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:16.676 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:16.677 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:32:16.677 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.108809+0000", 2024-09-14T06:32:16.677 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.4157000000000002e-05 2024-09-14T06:32:16.677 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:16.677 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:16.677 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:32:16.677 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.108866+0000", 2024-09-14T06:32:16.677 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.6644000000000002e-05 2024-09-14T06:32:16.677 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:16.677 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:32:16.677 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:16.677 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:16.677 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:16.678 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:19352 3.3 3:c34e5e51:::rbd_data.112d92027590.0000000000000024:f [read 1169408~1024 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:32:16.678 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:27.062798+0000", 2024-09-14T06:32:16.678 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 169.59932170299999, 2024-09-14T06:32:16.678 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046255900000000003, 2024-09-14T06:32:16.678 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:32:16.678 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:32:16.678 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:32:16.678 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:32:16.678 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:32:16.678 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:16.678 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 19352 2024-09-14T06:32:16.678 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:16.678 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:32:16.679 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:16.679 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:32:16.679 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062798+0000", 2024-09-14T06:32:16.679 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:16.679 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:16.679 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:16.679 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:32:16.679 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062798+0000", 2024-09-14T06:32:16.679 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:16.679 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:16.679 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:16.679 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:32:16.679 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062798+0000", 2024-09-14T06:32:16.680 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.37e-07 2024-09-14T06:32:16.680 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:16.680 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:16.680 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:32:16.680 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062800+0000", 2024-09-14T06:32:16.680 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.0269999999999998e-06 2024-09-14T06:32:16.680 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:16.680 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:16.680 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:32:16.680 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062806+0000", 2024-09-14T06:32:16.680 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.6690000000000003e-06 2024-09-14T06:32:16.680 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:16.680 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:16.680 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:32:16.680 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062810+0000", 2024-09-14T06:32:16.681 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.3449999999999999e-06 2024-09-14T06:32:16.681 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:16.681 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:16.681 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:32:16.681 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.108892+0000", 2024-09-14T06:32:16.681 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046081715000000002 2024-09-14T06:32:16.681 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:16.681 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:16.681 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:32:16.681 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.108940+0000", 2024-09-14T06:32:16.681 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.7815999999999998e-05 2024-09-14T06:32:16.681 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:16.681 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:16.682 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:32:16.682 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.109053+0000", 2024-09-14T06:32:16.682 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.000113791 2024-09-14T06:32:16.682 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:16.682 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:32:16.682 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:16.682 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:16.682 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:16.682 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:19353 3.3 3:c34e5e51:::rbd_data.112d92027590.0000000000000024:f [read 1179648~1024 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:32:16.682 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:27.062816+0000", 2024-09-14T06:32:16.682 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 169.59930318400001, 2024-09-14T06:32:16.682 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046481256999999998, 2024-09-14T06:32:16.682 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:32:16.682 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:32:16.682 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:32:16.682 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:32:16.683 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:32:16.683 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:16.683 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 19353 2024-09-14T06:32:16.683 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:16.683 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:32:16.683 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:16.683 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:32:16.683 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062816+0000", 2024-09-14T06:32:16.683 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:16.683 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:16.683 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:16.683 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:32:16.683 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062816+0000", 2024-09-14T06:32:16.683 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:16.684 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:16.684 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:16.684 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:32:16.684 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062816+0000", 2024-09-14T06:32:16.684 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.2899999999999999e-07 2024-09-14T06:32:16.684 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:16.684 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:16.684 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:32:16.684 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062818+0000", 2024-09-14T06:32:16.684 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.699e-06 2024-09-14T06:32:16.684 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:16.684 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:16.684 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:32:16.684 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062825+0000", 2024-09-14T06:32:16.685 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.878e-06 2024-09-14T06:32:16.685 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:16.685 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:16.685 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:32:16.685 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062830+0000", 2024-09-14T06:32:16.685 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.5410000000000002e-06 2024-09-14T06:32:16.685 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:16.685 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:16.685 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:32:16.685 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.109096+0000", 2024-09-14T06:32:16.685 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046266517 2024-09-14T06:32:16.685 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:16.685 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:16.685 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:32:16.686 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.109190+0000", 2024-09-14T06:32:16.686 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.4000999999999997e-05 2024-09-14T06:32:16.686 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:16.686 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:16.686 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:32:16.686 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.109297+0000", 2024-09-14T06:32:16.686 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.000107292 2024-09-14T06:32:16.686 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:16.686 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:32:16.686 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:16.686 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:16.686 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:16.686 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:19354 3.3 3:c34e5e51:::rbd_data.112d92027590.0000000000000024:f [read 1189888~1024 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:32:16.686 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:27.062842+0000", 2024-09-14T06:32:16.687 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 169.59927772899999, 2024-09-14T06:32:16.687 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046594257, 2024-09-14T06:32:16.687 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:32:16.687 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:32:16.687 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:32:16.687 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:32:16.687 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:32:16.687 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:16.687 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 19354 2024-09-14T06:32:16.687 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:16.687 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:32:16.687 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:16.687 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:32:16.687 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062842+0000", 2024-09-14T06:32:16.688 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:16.688 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:16.688 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:16.688 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:32:16.688 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062842+0000", 2024-09-14T06:32:16.688 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:16.688 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:16.688 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:16.688 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:32:16.688 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062842+0000", 2024-09-14T06:32:16.688 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.7399999999999999e-07 2024-09-14T06:32:16.688 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:16.688 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:16.689 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:32:16.689 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062859+0000", 2024-09-14T06:32:16.689 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.7010000000000001e-05 2024-09-14T06:32:16.689 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:16.689 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:16.689 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:32:16.689 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062866+0000", 2024-09-14T06:32:16.689 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.9990000000000002e-06 2024-09-14T06:32:16.689 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:16.689 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:16.689 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:32:16.689 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062873+0000", 2024-09-14T06:32:16.689 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.8569999999999997e-06 2024-09-14T06:32:16.689 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:16.690 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:16.690 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:32:16.690 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.109335+0000", 2024-09-14T06:32:16.690 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046461943999999998 2024-09-14T06:32:16.690 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:16.690 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:16.690 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:32:16.690 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.109377+0000", 2024-09-14T06:32:16.690 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.2558999999999997e-05 2024-09-14T06:32:16.690 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:16.690 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:16.690 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:32:16.690 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.109436+0000", 2024-09-14T06:32:16.690 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.8514000000000003e-05 2024-09-14T06:32:16.690 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:16.690 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:32:16.691 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:16.691 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:16.691 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:16.691 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:19355 3.3 3:c34e5e51:::rbd_data.112d92027590.0000000000000024:f [read 1200128~1024 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:32:16.691 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:27.062884+0000", 2024-09-14T06:32:16.691 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 169.599235056, 2024-09-14T06:32:16.691 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046681650999999998, 2024-09-14T06:32:16.691 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:32:16.691 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:32:16.691 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:32:16.691 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:32:16.691 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:32:16.691 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:16.691 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 19355 2024-09-14T06:32:16.691 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:16.691 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:32:16.692 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:16.692 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:32:16.692 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062884+0000", 2024-09-14T06:32:16.692 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:16.692 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:16.692 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:16.692 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:32:16.692 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062884+0000", 2024-09-14T06:32:16.692 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:16.692 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:16.692 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:16.692 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:32:16.692 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062885+0000", 2024-09-14T06:32:16.692 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.8000000000000001e-07 2024-09-14T06:32:16.692 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:16.693 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:16.693 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:32:16.693 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062886+0000", 2024-09-14T06:32:16.693 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.8920000000000001e-06 2024-09-14T06:32:16.693 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:16.693 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:16.693 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:32:16.693 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062892+0000", 2024-09-14T06:32:16.693 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.84e-06 2024-09-14T06:32:16.693 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:16.693 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:16.693 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:32:16.693 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062899+0000", 2024-09-14T06:32:16.693 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.5339999999999996e-06 2024-09-14T06:32:16.694 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:16.694 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:16.694 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:32:16.694 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.109466+0000", 2024-09-14T06:32:16.694 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046566919999999998 2024-09-14T06:32:16.694 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:16.694 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:16.694 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:32:16.694 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.109510+0000", 2024-09-14T06:32:16.694 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.4153000000000002e-05 2024-09-14T06:32:16.694 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:16.694 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:16.694 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:32:16.694 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.109566+0000", 2024-09-14T06:32:16.695 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.5931999999999999e-05 2024-09-14T06:32:16.695 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:16.695 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:32:16.695 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:16.695 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:16.695 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:16.695 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:19356 3.3 3:c34e5e51:::rbd_data.112d92027590.0000000000000024:f [read 1210368~1024 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:32:16.695 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:27.062908+0000", 2024-09-14T06:32:16.695 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 169.59921161899999, 2024-09-14T06:32:16.695 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046807498000000003, 2024-09-14T06:32:16.695 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:32:16.695 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:32:16.695 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:32:16.695 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:32:16.695 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:32:16.695 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:16.696 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 19356 2024-09-14T06:32:16.696 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:16.696 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:32:16.696 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:16.696 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:32:16.696 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062908+0000", 2024-09-14T06:32:16.696 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:16.696 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:16.696 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:16.696 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:32:16.696 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062908+0000", 2024-09-14T06:32:16.696 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:16.696 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:16.696 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:16.697 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:32:16.697 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062908+0000", 2024-09-14T06:32:16.697 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.5699999999999998e-07 2024-09-14T06:32:16.697 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:16.697 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:16.697 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:32:16.697 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062910+0000", 2024-09-14T06:32:16.697 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.7710000000000001e-06 2024-09-14T06:32:16.697 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:16.697 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:16.697 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:32:16.697 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062920+0000", 2024-09-14T06:32:16.697 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.0654e-05 2024-09-14T06:32:16.697 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:16.698 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:16.698 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:32:16.698 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062931+0000", 2024-09-14T06:32:16.698 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.075e-05 2024-09-14T06:32:16.698 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:16.698 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:16.698 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:32:16.698 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.109595+0000", 2024-09-14T06:32:16.698 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046663713000000002 2024-09-14T06:32:16.698 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:16.698 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:16.698 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:32:16.698 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.109632+0000", 2024-09-14T06:32:16.699 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.7505000000000001e-05 2024-09-14T06:32:16.699 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:16.699 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:16.699 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:32:16.699 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.109715+0000", 2024-09-14T06:32:16.699 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.2748000000000001e-05 2024-09-14T06:32:16.699 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:16.699 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:32:16.699 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:16.699 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:16.699 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:16.699 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:19357 3.3 3:c34e5e51:::rbd_data.112d92027590.0000000000000024:f [read 1220608~1024 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:32:16.699 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:27.062939+0000", 2024-09-14T06:32:16.699 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 169.599180171, 2024-09-14T06:32:16.699 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046920508, 2024-09-14T06:32:16.700 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:32:16.700 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:32:16.700 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:32:16.700 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:32:16.700 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:32:16.700 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:16.700 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 19357 2024-09-14T06:32:16.700 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:16.700 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:32:16.700 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:16.700 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:32:16.700 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062939+0000", 2024-09-14T06:32:16.700 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:16.700 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:16.700 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:16.701 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:32:16.701 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062939+0000", 2024-09-14T06:32:16.701 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:16.701 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:16.701 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:16.701 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:32:16.701 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062940+0000", 2024-09-14T06:32:16.701 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.32e-07 2024-09-14T06:32:16.701 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:16.701 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:16.701 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:32:16.701 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062942+0000", 2024-09-14T06:32:16.701 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.1919999999999999e-06 2024-09-14T06:32:16.702 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:16.702 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:16.702 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:32:16.702 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062948+0000", 2024-09-14T06:32:16.702 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.1149999999999999e-06 2024-09-14T06:32:16.702 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:16.702 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:16.702 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:32:16.702 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062960+0000", 2024-09-14T06:32:16.702 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.1895e-05 2024-09-14T06:32:16.702 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:16.702 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:16.702 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:32:16.702 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.109704+0000", 2024-09-14T06:32:16.703 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046744584999999998 2024-09-14T06:32:16.703 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:16.703 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:16.703 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:32:16.703 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.109777+0000", 2024-09-14T06:32:16.703 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.2282999999999999e-05 2024-09-14T06:32:16.703 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:16.703 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:16.703 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:32:16.703 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.109860+0000", 2024-09-14T06:32:16.703 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.3005999999999998e-05 2024-09-14T06:32:16.703 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:16.703 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:32:16.703 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:16.704 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:16.704 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:16.704 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:19358 3.3 3:c34e5e51:::rbd_data.112d92027590.0000000000000024:f [read 1230848~1024 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:32:16.704 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:27.062967+0000", 2024-09-14T06:32:16.704 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 169.599152541, 2024-09-14T06:32:16.704 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047144471, 2024-09-14T06:32:16.704 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:32:16.704 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:32:16.704 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:32:16.704 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:32:16.704 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:32:16.704 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:16.704 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 19358 2024-09-14T06:32:16.704 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:16.705 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:32:16.705 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:16.705 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:32:16.705 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062967+0000", 2024-09-14T06:32:16.705 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:16.705 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:16.705 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:16.705 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:32:16.705 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062967+0000", 2024-09-14T06:32:16.705 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:16.705 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:16.705 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:16.705 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:32:16.705 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062967+0000", 2024-09-14T06:32:16.706 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.89e-07 2024-09-14T06:32:16.706 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:16.706 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:16.706 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:32:16.706 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062969+0000", 2024-09-14T06:32:16.706 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.9929999999999998e-06 2024-09-14T06:32:16.706 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:16.706 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:16.706 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:32:16.706 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062975+0000", 2024-09-14T06:32:16.706 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.9030000000000001e-06 2024-09-14T06:32:16.706 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:16.706 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:16.707 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:32:16.707 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062979+0000", 2024-09-14T06:32:16.707 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.2250000000000002e-06 2024-09-14T06:32:16.707 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:16.707 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:16.707 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:32:16.707 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.109934+0000", 2024-09-14T06:32:16.707 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046954410000000002 2024-09-14T06:32:16.707 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:16.707 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:16.707 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:32:16.707 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.110007+0000", 2024-09-14T06:32:16.707 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.3829000000000001e-05 2024-09-14T06:32:16.707 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:16.708 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:16.708 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:32:16.708 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.110111+0000", 2024-09-14T06:32:16.708 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00010372199999999999 2024-09-14T06:32:16.708 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:16.708 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:32:16.708 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:16.708 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:16.708 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:16.708 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:19359 3.3 3:c34e5e51:::rbd_data.112d92027590.0000000000000024:f [read 1241088~1024 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:32:16.708 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:27.062987+0000", 2024-09-14T06:32:16.708 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 169.599132451, 2024-09-14T06:32:16.708 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047297182, 2024-09-14T06:32:16.709 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:32:16.709 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:32:16.709 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:32:16.709 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:32:16.709 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:32:16.709 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:16.709 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 19359 2024-09-14T06:32:16.709 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:16.709 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:32:16.709 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:16.709 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:32:16.709 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062987+0000", 2024-09-14T06:32:16.709 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:16.709 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:16.709 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:16.710 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:32:16.710 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062987+0000", 2024-09-14T06:32:16.710 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:16.710 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:16.710 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:16.710 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:32:16.710 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062987+0000", 2024-09-14T06:32:16.710 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.2500000000000001e-07 2024-09-14T06:32:16.710 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:16.710 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:16.710 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:32:16.710 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062989+0000", 2024-09-14T06:32:16.711 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.6080000000000001e-06 2024-09-14T06:32:16.711 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:16.711 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:16.711 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:32:16.711 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062994+0000", 2024-09-14T06:32:16.711 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.5520000000000004e-06 2024-09-14T06:32:16.711 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:16.711 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:16.711 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:32:16.711 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062999+0000", 2024-09-14T06:32:16.711 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.3930000000000001e-06 2024-09-14T06:32:16.711 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:16.711 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:16.711 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:32:16.712 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.110142+0000", 2024-09-14T06:32:16.712 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047143154 2024-09-14T06:32:16.712 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:16.712 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:16.712 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:32:16.712 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.110219+0000", 2024-09-14T06:32:16.712 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.6737999999999996e-05 2024-09-14T06:32:16.712 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:16.712 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:16.712 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:32:16.712 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.110284+0000", 2024-09-14T06:32:16.712 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.5412000000000003e-05 2024-09-14T06:32:16.712 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:16.712 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:32:16.713 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:16.713 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:16.713 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:16.713 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:19360 3.3 3:c34e5e51:::rbd_data.112d92027590.0000000000000024:f [read 1251328~1024 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:32:16.713 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:27.063017+0000", 2024-09-14T06:32:16.713 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 169.59910199000001, 2024-09-14T06:32:16.713 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047453823999999999, 2024-09-14T06:32:16.713 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:32:16.713 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:32:16.713 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:32:16.713 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:32:16.713 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:32:16.713 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:16.713 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 19360 2024-09-14T06:32:16.713 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:16.713 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:32:16.714 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:16.714 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:32:16.714 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.063017+0000", 2024-09-14T06:32:16.714 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:16.714 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:16.714 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:16.714 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:32:16.714 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.063017+0000", 2024-09-14T06:32:16.714 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:16.714 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:16.714 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:16.714 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:32:16.714 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.064977+0000", 2024-09-14T06:32:16.714 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.0019597080000000001 2024-09-14T06:32:16.715 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:16.715 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:16.715 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:32:16.715 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.064987+0000", 2024-09-14T06:32:16.715 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.9620000000000005e-06 2024-09-14T06:32:16.715 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:16.715 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:16.715 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:32:16.715 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065000+0000", 2024-09-14T06:32:16.715 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.3173000000000001e-05 2024-09-14T06:32:16.715 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:16.715 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:16.715 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:32:16.715 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065005+0000", 2024-09-14T06:32:16.716 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.9960000000000001e-06 2024-09-14T06:32:16.716 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:16.716 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:16.716 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:32:16.716 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.110314+0000", 2024-09-14T06:32:16.716 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.04530899 2024-09-14T06:32:16.716 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:16.716 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:16.716 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:32:16.716 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.110398+0000", 2024-09-14T06:32:16.716 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.3539000000000001e-05 2024-09-14T06:32:16.716 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:16.716 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:16.716 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:32:16.716 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.110471+0000", 2024-09-14T06:32:16.717 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.3455999999999996e-05 2024-09-14T06:32:16.717 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:16.717 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:32:16.717 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:16.717 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:16.717 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:16.717 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:19365 3.3 3:c34e5e51:::rbd_data.112d92027590.0000000000000024:f [read 1302528~1024 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:32:16.717 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:27.065058+0000", 2024-09-14T06:32:16.717 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 169.59706131999999, 2024-09-14T06:32:16.717 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046105526000000001, 2024-09-14T06:32:16.717 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:32:16.717 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:32:16.717 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:32:16.717 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:32:16.718 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:32:16.718 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:16.718 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 19365 2024-09-14T06:32:16.718 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:16.718 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:32:16.718 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:16.718 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:32:16.718 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065058+0000", 2024-09-14T06:32:16.718 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:16.718 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:16.718 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:16.718 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:32:16.718 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065058+0000", 2024-09-14T06:32:16.718 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:16.718 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:16.719 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:16.719 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:32:16.719 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065058+0000", 2024-09-14T06:32:16.719 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.1899999999999999e-07 2024-09-14T06:32:16.719 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:16.719 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:16.719 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:32:16.719 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065059+0000", 2024-09-14T06:32:16.719 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.1299999999999998e-07 2024-09-14T06:32:16.719 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:16.719 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:16.719 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:32:16.719 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065063+0000", 2024-09-14T06:32:16.719 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.1099999999999996e-06 2024-09-14T06:32:16.720 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:16.720 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:16.720 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:32:16.720 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065070+0000", 2024-09-14T06:32:16.720 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.973e-06 2024-09-14T06:32:16.720 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:16.720 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:16.720 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:32:16.720 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.111014+0000", 2024-09-14T06:32:16.720 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.045943373000000003 2024-09-14T06:32:16.720 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:16.720 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:16.720 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:32:16.720 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.111073+0000", 2024-09-14T06:32:16.720 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.8987999999999998e-05 2024-09-14T06:32:16.720 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:16.721 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:16.721 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:32:16.721 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.111163+0000", 2024-09-14T06:32:16.721 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.0950000000000002e-05 2024-09-14T06:32:16.721 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:16.721 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:32:16.721 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:16.721 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:16.721 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:16.721 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:19366 3.3 3:c34e5e51:::rbd_data.112d92027590.0000000000000024:f [read 1312768~1024 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:32:16.721 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:27.065074+0000", 2024-09-14T06:32:16.721 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 169.59704547000001, 2024-09-14T06:32:16.722 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046289858000000003, 2024-09-14T06:32:16.722 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:32:16.722 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:32:16.722 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:32:16.722 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:32:16.722 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:32:16.722 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:16.722 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 19366 2024-09-14T06:32:16.722 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:16.722 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:32:16.722 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:16.722 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:32:16.722 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065074+0000", 2024-09-14T06:32:16.722 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:16.722 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:16.723 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:16.723 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:32:16.723 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065074+0000", 2024-09-14T06:32:16.723 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:16.723 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:16.723 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:16.723 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:32:16.723 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065074+0000", 2024-09-14T06:32:16.723 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.3900000000000001e-07 2024-09-14T06:32:16.723 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:16.723 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:16.723 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:32:16.723 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065078+0000", 2024-09-14T06:32:16.723 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.3229999999999999e-06 2024-09-14T06:32:16.724 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:16.724 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:16.724 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:32:16.724 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065082+0000", 2024-09-14T06:32:16.724 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.8299999999999998e-06 2024-09-14T06:32:16.724 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:16.724 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:16.724 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:32:16.724 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065085+0000", 2024-09-14T06:32:16.724 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.8660000000000002e-06 2024-09-14T06:32:16.725 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:16.725 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:16.725 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:32:16.725 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.111270+0000", 2024-09-14T06:32:16.725 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046185035999999999 2024-09-14T06:32:16.725 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:16.725 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:16.725 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:32:16.725 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.111311+0000", 2024-09-14T06:32:16.725 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.0935e-05 2024-09-14T06:32:16.725 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:16.725 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:16.725 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:32:16.726 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.111364+0000", 2024-09-14T06:32:16.726 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.2629e-05 2024-09-14T06:32:16.726 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:16.726 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:32:16.726 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:16.726 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:16.726 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:16.726 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:19367 3.3 3:c34e5e51:::rbd_data.112d92027590.0000000000000024:f [read 1323008~1024 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:32:16.726 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:27.065115+0000", 2024-09-14T06:32:16.726 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 169.59700390899999, 2024-09-14T06:32:16.726 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046407003000000002, 2024-09-14T06:32:16.726 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:32:16.726 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:32:16.726 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:32:16.727 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:32:16.727 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:32:16.727 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:16.727 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 19367 2024-09-14T06:32:16.727 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:16.727 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:32:16.727 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:16.727 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:32:16.727 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065115+0000", 2024-09-14T06:32:16.727 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:16.727 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:16.727 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:16.727 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:32:16.728 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065115+0000", 2024-09-14T06:32:16.728 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:16.728 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:16.728 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:16.728 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:32:16.728 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065116+0000", 2024-09-14T06:32:16.728 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.4999999999999998e-07 2024-09-14T06:32:16.728 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:16.728 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:16.728 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:32:16.728 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065117+0000", 2024-09-14T06:32:16.728 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.189e-06 2024-09-14T06:32:16.728 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:16.728 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:16.729 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:32:16.729 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065122+0000", 2024-09-14T06:32:16.729 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.3709999999999999e-06 2024-09-14T06:32:16.729 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:16.729 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:16.729 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:32:16.729 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065125+0000", 2024-09-14T06:32:16.729 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.8940000000000002e-06 2024-09-14T06:32:16.729 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:16.729 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:16.729 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:32:16.729 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.111402+0000", 2024-09-14T06:32:16.729 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046276645999999998 2024-09-14T06:32:16.730 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:16.730 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:16.730 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:32:16.730 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.111445+0000", 2024-09-14T06:32:16.730 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.3288000000000003e-05 2024-09-14T06:32:16.730 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:16.730 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:16.730 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:32:16.730 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.111522+0000", 2024-09-14T06:32:16.730 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.7164999999999997e-05 2024-09-14T06:32:16.730 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:16.730 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:32:16.730 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:16.730 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:16.731 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:16.731 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:19368 3.3 3:c34e5e51:::rbd_data.112d92027590.0000000000000024:f [read 1333248~1024 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:32:16.731 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:27.065130+0000", 2024-09-14T06:32:16.731 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 169.596989619, 2024-09-14T06:32:16.731 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046507275000000001, 2024-09-14T06:32:16.731 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:32:16.731 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:32:16.731 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:32:16.731 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:32:16.731 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:32:16.731 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:16.731 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 19368 2024-09-14T06:32:16.731 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:16.731 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:32:16.732 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:16.732 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:32:16.732 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065130+0000", 2024-09-14T06:32:16.732 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:16.732 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:16.732 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:16.732 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:32:16.732 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065130+0000", 2024-09-14T06:32:16.732 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:16.732 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:16.732 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:16.732 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:32:16.732 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065130+0000", 2024-09-14T06:32:16.732 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.0800000000000001e-07 2024-09-14T06:32:16.732 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:16.733 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:16.733 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:32:16.733 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065131+0000", 2024-09-14T06:32:16.733 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.8700000000000004e-07 2024-09-14T06:32:16.733 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:16.733 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:16.733 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:32:16.733 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065134+0000", 2024-09-14T06:32:16.733 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.1039999999999998e-06 2024-09-14T06:32:16.733 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:16.733 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:16.733 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:32:16.733 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065136+0000", 2024-09-14T06:32:16.734 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.3949999999999999e-06 2024-09-14T06:32:16.734 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:16.734 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:16.734 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:32:16.734 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.111541+0000", 2024-09-14T06:32:16.734 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046404206000000003 2024-09-14T06:32:16.734 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:16.734 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:16.734 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:32:16.734 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.111582+0000", 2024-09-14T06:32:16.734 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.1051999999999999e-05 2024-09-14T06:32:16.734 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:16.734 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:16.734 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:32:16.734 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.111637+0000", 2024-09-14T06:32:16.735 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.5322999999999999e-05 2024-09-14T06:32:16.735 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:16.735 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:32:16.735 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:16.735 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:16.735 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:16.735 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:19369 3.3 3:c34e5e51:::rbd_data.112d92027590.0000000000000024:f [read 1343488~1024 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:32:16.735 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:27.065139+0000", 2024-09-14T06:32:16.735 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 169.59697980199999, 2024-09-14T06:32:16.735 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046682169000000003, 2024-09-14T06:32:16.735 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:32:16.735 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:32:16.735 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:32:16.735 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:32:16.736 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:32:16.736 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:16.736 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 19369 2024-09-14T06:32:16.736 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:16.736 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:32:16.736 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:16.736 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:32:16.736 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065139+0000", 2024-09-14T06:32:16.736 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:16.736 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:16.736 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:16.736 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:32:16.736 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065139+0000", 2024-09-14T06:32:16.736 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:16.736 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:16.737 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:16.737 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:32:16.737 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065140+0000", 2024-09-14T06:32:16.737 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.8299999999999998e-07 2024-09-14T06:32:16.737 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:16.737 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:16.737 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:32:16.737 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065141+0000", 2024-09-14T06:32:16.737 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.3900000000000003e-07 2024-09-14T06:32:16.737 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:16.737 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:16.737 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:32:16.737 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065144+0000", 2024-09-14T06:32:16.737 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.7409999999999998e-06 2024-09-14T06:32:16.738 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:16.738 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:16.738 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:32:16.738 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065147+0000", 2024-09-14T06:32:16.738 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.4150000000000002e-06 2024-09-14T06:32:16.738 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:16.738 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:16.738 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:32:16.738 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.111673+0000", 2024-09-14T06:32:16.738 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046526004000000003 2024-09-14T06:32:16.738 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:16.738 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:16.738 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:32:16.738 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.111749+0000", 2024-09-14T06:32:16.739 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.5741999999999994e-05 2024-09-14T06:32:16.739 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:16.739 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:16.739 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:32:16.739 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.111822+0000", 2024-09-14T06:32:16.739 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.3045000000000003e-05 2024-09-14T06:32:16.739 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:16.739 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:32:16.739 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:16.739 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:16.739 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:16.739 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:19370 3.3 3:c34e5e51:::rbd_data.112d92027590.0000000000000024:f [read 1353728~1024 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:32:16.739 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:27.065150+0000", 2024-09-14T06:32:16.739 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 169.596969073, 2024-09-14T06:32:16.740 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046897556999999999, 2024-09-14T06:32:16.740 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:32:16.740 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:32:16.740 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:32:16.740 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:32:16.740 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:32:16.740 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:16.740 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 19370 2024-09-14T06:32:16.740 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:16.740 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:32:16.740 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:16.740 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:32:16.740 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065150+0000", 2024-09-14T06:32:16.740 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:16.741 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:16.741 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:16.741 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:32:16.741 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065150+0000", 2024-09-14T06:32:16.741 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:16.741 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:16.741 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:16.741 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:32:16.741 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065150+0000", 2024-09-14T06:32:16.741 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.0200000000000001e-07 2024-09-14T06:32:16.741 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:16.741 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:16.741 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:32:16.741 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065151+0000", 2024-09-14T06:32:16.742 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.1100000000000004e-07 2024-09-14T06:32:16.742 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:16.742 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:16.742 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:32:16.742 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065154+0000", 2024-09-14T06:32:16.742 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.9639999999999999e-06 2024-09-14T06:32:16.742 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:16.742 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:16.742 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:32:16.742 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065157+0000", 2024-09-14T06:32:16.742 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.4260000000000002e-06 2024-09-14T06:32:16.742 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:16.742 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:16.742 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:32:16.742 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.111861+0000", 2024-09-14T06:32:16.743 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046704268 2024-09-14T06:32:16.743 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:16.743 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:16.743 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:32:16.743 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.111939+0000", 2024-09-14T06:32:16.743 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.8231000000000004e-05 2024-09-14T06:32:16.743 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:16.743 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:16.743 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:32:16.743 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.112048+0000", 2024-09-14T06:32:16.743 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00010855500000000001 2024-09-14T06:32:16.743 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:16.743 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:32:16.743 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:16.744 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:16.744 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:16.744 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:19371 3.3 3:c34e5e51:::rbd_data.112d92027590.0000000000000024:f [read 1363968~1024 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:32:16.744 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:27.065161+0000", 2024-09-14T06:32:16.744 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 169.59695842299999, 2024-09-14T06:32:16.744 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047021944000000003, 2024-09-14T06:32:16.744 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:32:16.744 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:32:16.744 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:32:16.744 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:32:16.744 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:32:16.744 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:16.744 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 19371 2024-09-14T06:32:16.744 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:16.744 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:32:16.745 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:16.745 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:32:16.745 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065161+0000", 2024-09-14T06:32:16.745 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:16.745 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:16.745 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:16.745 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:32:16.745 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065161+0000", 2024-09-14T06:32:16.745 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:16.745 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:16.745 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:16.745 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:32:16.745 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065161+0000", 2024-09-14T06:32:16.745 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.0200000000000001e-07 2024-09-14T06:32:16.746 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:16.746 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:16.746 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:32:16.746 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065166+0000", 2024-09-14T06:32:16.746 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.9869999999999999e-06 2024-09-14T06:32:16.746 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:16.746 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:16.746 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:32:16.746 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065169+0000", 2024-09-14T06:32:16.746 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.3799999999999998e-06 2024-09-14T06:32:16.746 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:16.746 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:16.746 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:32:16.746 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065172+0000", 2024-09-14T06:32:16.746 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.4830000000000002e-06 2024-09-14T06:32:16.747 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:16.747 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:16.747 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:32:16.747 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.112071+0000", 2024-09-14T06:32:16.747 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046899529000000002 2024-09-14T06:32:16.747 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:16.747 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:16.747 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:32:16.747 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.112128+0000", 2024-09-14T06:32:16.747 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.6341999999999997e-05 2024-09-14T06:32:16.747 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:16.747 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:16.747 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:32:16.747 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.112183+0000", 2024-09-14T06:32:16.748 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.5021000000000001e-05 2024-09-14T06:32:16.748 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:16.748 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:32:16.748 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:16.748 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:16.748 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:16.748 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:19372 3.3 3:c34e5e51:::rbd_data.112d92027590.0000000000000024:f [read 1374208~1024 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:32:16.748 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:27.065176+0000", 2024-09-14T06:32:16.748 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 169.59694327299999, 2024-09-14T06:32:16.748 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047188889999999997, 2024-09-14T06:32:16.748 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:32:16.748 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:32:16.748 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:32:16.748 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:32:16.749 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:32:16.749 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:16.749 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 19372 2024-09-14T06:32:16.749 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:16.749 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:32:16.749 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:16.749 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:32:16.749 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065176+0000", 2024-09-14T06:32:16.749 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:16.749 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:16.749 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:16.749 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:32:16.749 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065176+0000", 2024-09-14T06:32:16.750 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:16.750 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:16.750 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:16.750 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:32:16.750 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.071117+0000", 2024-09-14T06:32:16.750 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.0059404990000000001 2024-09-14T06:32:16.750 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:16.750 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:16.750 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:32:16.750 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.071120+0000", 2024-09-14T06:32:16.750 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.4340000000000001e-06 2024-09-14T06:32:16.750 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:16.750 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:16.750 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:32:16.750 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.071129+0000", 2024-09-14T06:32:16.751 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.8389999999999996e-06 2024-09-14T06:32:16.751 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:16.751 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:16.751 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:32:16.751 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.071134+0000", 2024-09-14T06:32:16.751 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.412e-06 2024-09-14T06:32:16.751 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:16.751 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:16.751 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:32:16.751 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.112221+0000", 2024-09-14T06:32:16.751 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.041086473999999998 2024-09-14T06:32:16.751 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:16.751 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:16.751 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:32:16.751 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.112299+0000", 2024-09-14T06:32:16.752 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.8746999999999997e-05 2024-09-14T06:32:16.752 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:16.752 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:16.752 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:32:16.752 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.112365+0000", 2024-09-14T06:32:16.752 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.5485000000000001e-05 2024-09-14T06:32:16.752 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:16.752 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:32:16.752 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:16.752 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:16.752 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:32:16.752 INFO:teuthology.orchestra.run.smithi138.stdout:} 2024-09-14T06:32:16.754 INFO:teuthology.orchestra.run.smithi094.stderr:reweighted osd.7 to 1 (10000) 2024-09-14T06:32:16.754 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-14T06:32:16.755 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 3 1 2024-09-14T06:32:16.755 INFO:tasks.ceph.osd.3.smithi094.stderr:2024-09-14T06:32:16.752+0000 7feeeafe8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33655) UID: 0 2024-09-14T06:32:16.854 INFO:tasks.ceph.osd.3.smithi094.stderr:2024-09-14T06:32:16.853+0000 7feeeafe8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33655) UID: 0 2024-09-14T06:32:16.863 INFO:teuthology.orchestra.run.smithi138.stdout:{ 2024-09-14T06:32:16.863 INFO:teuthology.orchestra.run.smithi138.stdout: "ops": [], 2024-09-14T06:32:16.864 INFO:teuthology.orchestra.run.smithi138.stdout: "num_ops": 0 2024-09-14T06:32:16.864 INFO:teuthology.orchestra.run.smithi138.stdout:} 2024-09-14T06:32:16.872 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-14T06:32:16.954 INFO:tasks.ceph.osd.7.smithi138.stderr:2024-09-14T06:32:16.953+0000 7f3c6a21d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33623) UID: 0 2024-09-14T06:32:16.974 INFO:teuthology.orchestra.run.smithi138.stdout:{ 2024-09-14T06:32:16.974 INFO:teuthology.orchestra.run.smithi138.stdout: "ops": [], 2024-09-14T06:32:16.974 INFO:teuthology.orchestra.run.smithi138.stdout: "complaint_time": 30, 2024-09-14T06:32:16.974 INFO:teuthology.orchestra.run.smithi138.stdout: "num_blocked_ops": 0 2024-09-14T06:32:16.974 INFO:teuthology.orchestra.run.smithi138.stdout:} 2024-09-14T06:32:16.983 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-14T06:32:17.067 INFO:tasks.ceph.osd.4.smithi138.stderr:2024-09-14T06:32:17.066+0000 7f6c653c8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33615) UID: 0 2024-09-14T06:32:17.127 INFO:teuthology.orchestra.run.smithi138.stdout:{ 2024-09-14T06:32:17.127 INFO:teuthology.orchestra.run.smithi138.stdout: "size": 20, 2024-09-14T06:32:17.127 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 600, 2024-09-14T06:32:17.127 INFO:teuthology.orchestra.run.smithi138.stdout: "ops": [ 2024-09-14T06:32:17.127 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:17.127 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:11453 3.1d 3:bcad7303:::rbd_data.112d92027590.0000000000000001:f [read 94208~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:32:17.127 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:25.654375+0000", 2024-09-14T06:32:17.127 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 171.470737902, 2024-09-14T06:32:17.127 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052449318000000002, 2024-09-14T06:32:17.127 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:32:17.127 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:32:17.127 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:32:17.128 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:32:17.128 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:32:17.128 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:17.128 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 11453 2024-09-14T06:32:17.128 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:17.128 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:32:17.128 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:17.128 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:32:17.128 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654375+0000", 2024-09-14T06:32:17.128 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:17.128 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:17.129 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:17.129 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:32:17.129 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654375+0000", 2024-09-14T06:32:17.129 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:17.129 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:17.129 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:17.129 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:32:17.129 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654496+0000", 2024-09-14T06:32:17.129 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00012081 2024-09-14T06:32:17.129 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:17.129 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:17.129 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:32:17.129 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654497+0000", 2024-09-14T06:32:17.130 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.8789999999999999e-06 2024-09-14T06:32:17.130 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:17.130 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:17.130 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:32:17.130 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654501+0000", 2024-09-14T06:32:17.130 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.8999999999999999e-06 2024-09-14T06:32:17.130 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:17.130 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:17.130 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:32:17.130 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654505+0000", 2024-09-14T06:32:17.130 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.2229999999999999e-06 2024-09-14T06:32:17.130 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:17.130 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:17.130 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:32:17.130 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.705917+0000", 2024-09-14T06:32:17.131 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.051412501999999999 2024-09-14T06:32:17.131 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:17.131 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:17.131 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:32:17.131 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.705999+0000", 2024-09-14T06:32:17.131 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.1551999999999994e-05 2024-09-14T06:32:17.131 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:17.131 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:17.131 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:32:17.131 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.706824+0000", 2024-09-14T06:32:17.131 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00082545199999999996 2024-09-14T06:32:17.131 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:17.131 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:32:17.131 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:17.132 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:17.132 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:17.132 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:11454 3.1d 3:bcad7303:::rbd_data.112d92027590.0000000000000001:f [read 104448~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:32:17.132 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:25.654510+0000", 2024-09-14T06:32:17.132 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 171.47060311199999, 2024-09-14T06:32:17.132 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052610713000000003, 2024-09-14T06:32:17.132 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:32:17.132 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:32:17.132 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:32:17.132 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:32:17.132 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:32:17.132 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:17.133 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 11454 2024-09-14T06:32:17.133 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:17.133 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:32:17.133 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:17.133 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:32:17.133 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654510+0000", 2024-09-14T06:32:17.133 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:17.133 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:17.133 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:17.133 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:32:17.134 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654510+0000", 2024-09-14T06:32:17.134 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:17.134 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:17.134 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:17.134 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:32:17.134 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654825+0000", 2024-09-14T06:32:17.134 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00031522400000000002 2024-09-14T06:32:17.134 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:17.134 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:17.134 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:32:17.134 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654827+0000", 2024-09-14T06:32:17.134 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.9709999999999998e-06 2024-09-14T06:32:17.134 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:17.134 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:17.135 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:32:17.135 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654832+0000", 2024-09-14T06:32:17.135 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.4099999999999999e-06 2024-09-14T06:32:17.135 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:17.135 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:17.135 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:32:17.135 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654835+0000", 2024-09-14T06:32:17.135 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.14e-06 2024-09-14T06:32:17.135 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:17.135 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:17.135 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:32:17.135 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.706851+0000", 2024-09-14T06:32:17.135 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052015761000000001 2024-09-14T06:32:17.135 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:17.136 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:17.136 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:32:17.136 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.706934+0000", 2024-09-14T06:32:17.136 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.2484999999999994e-05 2024-09-14T06:32:17.136 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:17.136 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:17.136 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:32:17.136 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.707120+0000", 2024-09-14T06:32:17.136 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00018672200000000001 2024-09-14T06:32:17.136 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:17.136 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:32:17.136 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:17.136 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:17.137 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:17.137 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:11455 3.1d 3:bcad7303:::rbd_data.112d92027590.0000000000000001:f [read 114688~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:32:17.137 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:25.654841+0000", 2024-09-14T06:32:17.137 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 171.47027178299999, 2024-09-14T06:32:17.137 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052579898, 2024-09-14T06:32:17.137 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:32:17.137 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:32:17.137 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:32:17.137 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:32:17.137 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:32:17.137 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:17.137 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 11455 2024-09-14T06:32:17.137 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:17.137 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:32:17.138 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:17.138 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:32:17.138 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654841+0000", 2024-09-14T06:32:17.138 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:17.138 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:17.138 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:17.138 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:32:17.138 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654841+0000", 2024-09-14T06:32:17.138 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:17.138 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:17.138 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:17.138 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:32:17.138 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654841+0000", 2024-09-14T06:32:17.138 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.7300000000000002e-07 2024-09-14T06:32:17.138 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:17.139 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:17.139 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:32:17.139 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654845+0000", 2024-09-14T06:32:17.139 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.0520000000000003e-06 2024-09-14T06:32:17.139 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:17.139 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:17.139 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:32:17.139 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654849+0000", 2024-09-14T06:32:17.139 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.0459999999999999e-06 2024-09-14T06:32:17.139 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:17.139 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:17.139 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:32:17.139 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654852+0000", 2024-09-14T06:32:17.139 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.9830000000000001e-06 2024-09-14T06:32:17.139 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:17.140 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:17.140 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:32:17.140 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.707159+0000", 2024-09-14T06:32:17.140 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052306826000000001 2024-09-14T06:32:17.140 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:17.140 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:17.140 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:32:17.140 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.707242+0000", 2024-09-14T06:32:17.140 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.2595000000000005e-05 2024-09-14T06:32:17.140 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:17.140 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:17.140 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:32:17.140 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.707421+0000", 2024-09-14T06:32:17.140 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.000179123 2024-09-14T06:32:17.140 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:17.141 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:32:17.141 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:17.141 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:17.141 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:17.141 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:11456 3.1d 3:bcad7303:::rbd_data.112d92027590.0000000000000001:f [read 124928~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:32:17.141 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:25.654856+0000", 2024-09-14T06:32:17.141 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 171.47025693699999, 2024-09-14T06:32:17.141 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052801580000000001, 2024-09-14T06:32:17.141 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:32:17.141 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:32:17.141 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:32:17.141 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:32:17.141 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:32:17.141 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:17.142 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 11456 2024-09-14T06:32:17.142 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:17.142 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:32:17.142 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:17.142 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:32:17.142 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654856+0000", 2024-09-14T06:32:17.142 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:17.142 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:17.142 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:17.142 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:32:17.142 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654856+0000", 2024-09-14T06:32:17.142 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:17.143 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:17.143 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:17.143 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:32:17.143 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654856+0000", 2024-09-14T06:32:17.143 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.29e-07 2024-09-14T06:32:17.143 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:17.143 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:17.143 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:32:17.143 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654857+0000", 2024-09-14T06:32:17.143 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.187e-06 2024-09-14T06:32:17.143 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:17.143 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:17.144 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:32:17.144 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654861+0000", 2024-09-14T06:32:17.144 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.3850000000000002e-06 2024-09-14T06:32:17.144 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:17.144 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:17.144 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:32:17.144 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654863+0000", 2024-09-14T06:32:17.144 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.9500000000000001e-06 2024-09-14T06:32:17.144 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:17.144 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:17.144 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:32:17.144 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.707456+0000", 2024-09-14T06:32:17.144 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052592434 2024-09-14T06:32:17.145 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:17.145 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:17.145 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:32:17.145 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.707516+0000", 2024-09-14T06:32:17.145 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.0167999999999997e-05 2024-09-14T06:32:17.145 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:17.145 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:17.145 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:32:17.145 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.707657+0000", 2024-09-14T06:32:17.145 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00014122700000000001 2024-09-14T06:32:17.145 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:17.146 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:32:17.146 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:17.146 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:17.146 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:17.146 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:11457 3.1d 3:bcad7303:::rbd_data.112d92027590.0000000000000001:f [read 135168~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:32:17.146 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:25.654867+0000", 2024-09-14T06:32:17.146 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 171.47024605199999, 2024-09-14T06:32:17.146 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052921902, 2024-09-14T06:32:17.146 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:32:17.146 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:32:17.146 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:32:17.146 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:32:17.146 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:32:17.146 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:17.147 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 11457 2024-09-14T06:32:17.147 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:17.147 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:32:17.147 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:17.147 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:32:17.147 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654867+0000", 2024-09-14T06:32:17.147 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:17.147 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:17.147 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:17.147 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:32:17.147 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654867+0000", 2024-09-14T06:32:17.147 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:17.147 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:17.148 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:17.148 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:32:17.148 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654972+0000", 2024-09-14T06:32:17.148 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00010584 2024-09-14T06:32:17.148 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:17.148 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:17.148 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:32:17.148 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654975+0000", 2024-09-14T06:32:17.148 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.1169999999999998e-06 2024-09-14T06:32:17.148 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:17.148 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:17.148 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:32:17.149 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654979+0000", 2024-09-14T06:32:17.149 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.1960000000000001e-06 2024-09-14T06:32:17.149 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:17.149 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:17.149 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:32:17.149 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654982+0000", 2024-09-14T06:32:17.149 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.2990000000000001e-06 2024-09-14T06:32:17.149 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:17.149 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:17.149 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:32:17.150 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.707684+0000", 2024-09-14T06:32:17.150 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052701979000000003 2024-09-14T06:32:17.150 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:17.150 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:17.150 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:32:17.150 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.707728+0000", 2024-09-14T06:32:17.150 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.3529999999999998e-05 2024-09-14T06:32:17.150 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:17.150 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:17.150 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:32:17.150 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.707789+0000", 2024-09-14T06:32:17.150 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.0940999999999998e-05 2024-09-14T06:32:17.150 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:17.151 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:32:17.151 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:17.151 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:17.151 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:17.151 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:11458 3.1d 3:bcad7303:::rbd_data.112d92027590.0000000000000001:f [read 145408~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:32:17.151 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:25.654987+0000", 2024-09-14T06:32:17.151 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 171.47012565399999, 2024-09-14T06:32:17.151 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.05291655, 2024-09-14T06:32:17.151 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:32:17.151 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:32:17.151 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:32:17.151 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:32:17.151 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:32:17.151 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:17.152 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 11458 2024-09-14T06:32:17.152 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:17.152 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:32:17.152 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:17.152 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:32:17.152 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654987+0000", 2024-09-14T06:32:17.152 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:17.152 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:17.152 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:17.152 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:32:17.152 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654987+0000", 2024-09-14T06:32:17.152 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:17.152 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:17.152 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:17.153 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:32:17.153 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655077+0000", 2024-09-14T06:32:17.153 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.0253000000000005e-05 2024-09-14T06:32:17.153 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:17.153 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:17.153 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:32:17.153 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655079+0000", 2024-09-14T06:32:17.153 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.9989999999999998e-06 2024-09-14T06:32:17.153 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:17.153 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:17.153 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:32:17.153 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655083+0000", 2024-09-14T06:32:17.153 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.9280000000000003e-06 2024-09-14T06:32:17.154 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:17.154 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:17.154 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:32:17.154 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655086+0000", 2024-09-14T06:32:17.154 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.056e-06 2024-09-14T06:32:17.154 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:17.154 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:17.154 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:32:17.154 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.707805+0000", 2024-09-14T06:32:17.154 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052718627999999997 2024-09-14T06:32:17.154 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:17.154 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:17.154 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:32:17.154 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.707848+0000", 2024-09-14T06:32:17.155 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.2728999999999997e-05 2024-09-14T06:32:17.155 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:17.155 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:17.155 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:32:17.155 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.707904+0000", 2024-09-14T06:32:17.155 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.5957e-05 2024-09-14T06:32:17.155 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:17.155 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:32:17.155 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:17.155 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:17.155 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:17.156 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:11459 3.1d 3:bcad7303:::rbd_data.112d92027590.0000000000000001:f [read 155648~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:32:17.156 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:25.655091+0000", 2024-09-14T06:32:17.156 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 171.47002164599999, 2024-09-14T06:32:17.156 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052944242000000002, 2024-09-14T06:32:17.156 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:32:17.156 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:32:17.156 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:32:17.156 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:32:17.156 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:32:17.156 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:17.156 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 11459 2024-09-14T06:32:17.156 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:17.156 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:32:17.157 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:17.157 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:32:17.157 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655091+0000", 2024-09-14T06:32:17.157 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:17.157 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:17.157 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:17.157 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:32:17.157 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655091+0000", 2024-09-14T06:32:17.157 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:17.157 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:17.157 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:17.157 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:32:17.157 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655237+0000", 2024-09-14T06:32:17.158 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00014570100000000001 2024-09-14T06:32:17.158 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:17.158 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:17.158 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:32:17.158 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655239+0000", 2024-09-14T06:32:17.158 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.8670000000000001e-06 2024-09-14T06:32:17.158 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:17.158 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:17.158 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:32:17.158 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655243+0000", 2024-09-14T06:32:17.158 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.9829999999999998e-06 2024-09-14T06:32:17.158 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:17.158 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:17.159 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:32:17.159 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655246+0000", 2024-09-14T06:32:17.159 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.1540000000000002e-06 2024-09-14T06:32:17.159 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:17.159 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:17.159 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:32:17.159 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.707934+0000", 2024-09-14T06:32:17.159 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052688286000000001 2024-09-14T06:32:17.159 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:17.159 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:17.159 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:32:17.159 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.707977+0000", 2024-09-14T06:32:17.159 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.3291999999999998e-05 2024-09-14T06:32:17.160 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:17.160 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:17.160 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:32:17.160 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.708035+0000", 2024-09-14T06:32:17.160 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.7958999999999999e-05 2024-09-14T06:32:17.160 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:17.160 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:32:17.160 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:17.160 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:17.160 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:17.160 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:11460 3.1d 3:bcad7303:::rbd_data.112d92027590.0000000000000001:f [read 165888~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:32:17.160 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:25.655250+0000", 2024-09-14T06:32:17.160 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 171.46986234900001, 2024-09-14T06:32:17.161 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052910698999999999, 2024-09-14T06:32:17.161 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:32:17.161 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:32:17.161 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:32:17.161 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:32:17.161 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:32:17.161 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:17.161 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 11460 2024-09-14T06:32:17.161 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:17.161 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:32:17.161 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:17.161 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:32:17.161 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655250+0000", 2024-09-14T06:32:17.161 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:17.162 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:17.162 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:17.162 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:32:17.162 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655250+0000", 2024-09-14T06:32:17.162 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:17.162 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:17.162 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:17.162 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:32:17.162 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655356+0000", 2024-09-14T06:32:17.162 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00010535900000000001 2024-09-14T06:32:17.162 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:17.162 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:17.162 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:32:17.162 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655357+0000", 2024-09-14T06:32:17.163 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.8080000000000001e-06 2024-09-14T06:32:17.163 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:17.163 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:17.163 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:32:17.163 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655361+0000", 2024-09-14T06:32:17.163 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.861e-06 2024-09-14T06:32:17.163 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:17.163 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:17.163 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:32:17.163 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655364+0000", 2024-09-14T06:32:17.163 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.1089999999999998e-06 2024-09-14T06:32:17.163 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:17.163 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:17.163 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:32:17.164 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.708065+0000", 2024-09-14T06:32:17.164 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052700595000000003 2024-09-14T06:32:17.164 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:17.164 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:17.164 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:32:17.164 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.708108+0000", 2024-09-14T06:32:17.164 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.2484999999999997e-05 2024-09-14T06:32:17.164 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:17.164 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:17.164 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:32:17.164 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.708161+0000", 2024-09-14T06:32:17.164 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.3482000000000001e-05 2024-09-14T06:32:17.164 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:17.164 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:32:17.164 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:17.165 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:17.165 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:17.165 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:11461 3.1d 3:bcad7303:::rbd_data.112d92027590.0000000000000001:f [read 176128~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:32:17.165 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:25.655370+0000", 2024-09-14T06:32:17.165 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 171.469743067, 2024-09-14T06:32:17.165 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052915141999999998, 2024-09-14T06:32:17.165 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:32:17.165 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:32:17.165 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:32:17.165 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:32:17.165 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:32:17.165 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:17.165 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 11461 2024-09-14T06:32:17.165 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:17.165 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:32:17.166 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:17.166 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:32:17.166 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655370+0000", 2024-09-14T06:32:17.166 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:17.166 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:17.166 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:17.166 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:32:17.166 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655370+0000", 2024-09-14T06:32:17.166 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:17.166 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:17.166 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:17.166 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:32:17.166 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655394+0000", 2024-09-14T06:32:17.166 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.4553999999999999e-05 2024-09-14T06:32:17.166 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:17.167 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:17.167 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:32:17.167 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655396+0000", 2024-09-14T06:32:17.167 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.8950000000000001e-06 2024-09-14T06:32:17.167 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:17.167 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:17.167 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:32:17.167 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655401+0000", 2024-09-14T06:32:17.167 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.0089999999999998e-06 2024-09-14T06:32:17.167 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:17.167 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:17.167 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:32:17.167 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655404+0000", 2024-09-14T06:32:17.167 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.3400000000000002e-06 2024-09-14T06:32:17.167 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:17.168 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:17.168 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:32:17.168 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.708186+0000", 2024-09-14T06:32:17.168 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052781726000000001 2024-09-14T06:32:17.168 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:17.168 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:17.168 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:32:17.168 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.708229+0000", 2024-09-14T06:32:17.168 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.2670999999999999e-05 2024-09-14T06:32:17.168 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:17.168 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:17.168 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:32:17.168 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.708285+0000", 2024-09-14T06:32:17.168 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.5946999999999998e-05 2024-09-14T06:32:17.168 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:17.169 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:32:17.169 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:17.169 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:17.169 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:17.169 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:11462 3.1d 3:bcad7303:::rbd_data.112d92027590.0000000000000001:f [read 186368~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:32:17.169 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:25.655410+0000", 2024-09-14T06:32:17.169 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 171.46970258900001, 2024-09-14T06:32:17.169 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052995258000000003, 2024-09-14T06:32:17.169 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:32:17.169 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:32:17.169 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:32:17.169 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:32:17.169 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:32:17.170 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:17.170 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 11462 2024-09-14T06:32:17.170 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:17.170 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:32:17.170 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:17.170 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:32:17.170 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655410+0000", 2024-09-14T06:32:17.170 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:17.170 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:17.170 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:17.170 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:32:17.170 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655410+0000", 2024-09-14T06:32:17.170 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:17.170 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:17.170 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:17.171 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:32:17.171 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655625+0000", 2024-09-14T06:32:17.171 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00021442600000000001 2024-09-14T06:32:17.171 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:17.171 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:17.171 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:32:17.171 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655626+0000", 2024-09-14T06:32:17.171 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.815e-06 2024-09-14T06:32:17.171 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:17.171 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:17.171 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:32:17.171 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655631+0000", 2024-09-14T06:32:17.171 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.651e-06 2024-09-14T06:32:17.171 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:17.172 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:17.172 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:32:17.172 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655634+0000", 2024-09-14T06:32:17.172 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.2449999999999998e-06 2024-09-14T06:32:17.172 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:17.172 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:17.172 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:32:17.172 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.708312+0000", 2024-09-14T06:32:17.172 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052677412 2024-09-14T06:32:17.172 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:17.172 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:17.172 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:32:17.172 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.708352+0000", 2024-09-14T06:32:17.172 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.0729999999999998e-05 2024-09-14T06:32:17.173 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:17.173 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:17.173 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:32:17.173 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.708405+0000", 2024-09-14T06:32:17.173 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.2979000000000002e-05 2024-09-14T06:32:17.173 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:17.173 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:32:17.173 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:17.173 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:17.173 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:17.173 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:11463 3.1d 3:bcad7303:::rbd_data.112d92027590.0000000000000001:f [read 196608~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:32:17.173 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:25.655639+0000", 2024-09-14T06:32:17.173 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 171.469473443, 2024-09-14T06:32:17.173 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052888859000000003, 2024-09-14T06:32:17.174 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:32:17.174 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:32:17.174 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:32:17.174 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:32:17.174 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:32:17.174 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:17.174 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 11463 2024-09-14T06:32:17.174 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:17.174 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:32:17.174 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:17.174 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:32:17.174 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655639+0000", 2024-09-14T06:32:17.175 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:17.175 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:17.175 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:17.175 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:32:17.175 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655639+0000", 2024-09-14T06:32:17.175 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:17.175 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:17.175 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:17.175 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:32:17.175 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655640+0000", 2024-09-14T06:32:17.175 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.08e-07 2024-09-14T06:32:17.175 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:17.175 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:17.176 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:32:17.176 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655641+0000", 2024-09-14T06:32:17.176 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.156e-06 2024-09-14T06:32:17.176 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:17.176 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:17.176 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:32:17.176 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655648+0000", 2024-09-14T06:32:17.176 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.8460000000000002e-06 2024-09-14T06:32:17.176 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:17.176 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:17.176 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:32:17.176 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655651+0000", 2024-09-14T06:32:17.176 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.0079999999999998e-06 2024-09-14T06:32:17.176 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:17.177 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:17.177 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:32:17.177 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.708429+0000", 2024-09-14T06:32:17.177 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052778459 2024-09-14T06:32:17.177 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:17.177 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:17.177 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:32:17.177 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.708473+0000", 2024-09-14T06:32:17.177 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.3782999999999997e-05 2024-09-14T06:32:17.177 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:17.177 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:17.177 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:32:17.177 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.708528+0000", 2024-09-14T06:32:17.177 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.5198999999999998e-05 2024-09-14T06:32:17.178 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:17.178 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:32:17.178 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:17.178 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:17.178 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:17.178 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:11464 3.1d 3:bcad7303:::rbd_data.112d92027590.0000000000000001:f [read 206848~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:32:17.178 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:25.655654+0000", 2024-09-14T06:32:17.178 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 171.46945857399999, 2024-09-14T06:32:17.178 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052981845999999999, 2024-09-14T06:32:17.178 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:32:17.178 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:32:17.178 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:32:17.178 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:32:17.178 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:32:17.179 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:17.179 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 11464 2024-09-14T06:32:17.179 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:17.179 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:32:17.179 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:17.179 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:32:17.179 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655654+0000", 2024-09-14T06:32:17.179 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:17.179 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:17.179 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:17.179 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:32:17.180 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655654+0000", 2024-09-14T06:32:17.180 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:17.180 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:17.180 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:17.180 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:32:17.180 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655946+0000", 2024-09-14T06:32:17.180 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00029148200000000003 2024-09-14T06:32:17.180 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:17.180 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:17.180 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:32:17.180 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655948+0000", 2024-09-14T06:32:17.180 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.973e-06 2024-09-14T06:32:17.181 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:17.181 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:17.181 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:32:17.181 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655953+0000", 2024-09-14T06:32:17.181 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.1240000000000004e-06 2024-09-14T06:32:17.181 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:17.181 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:17.181 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:32:17.181 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655956+0000", 2024-09-14T06:32:17.181 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.315e-06 2024-09-14T06:32:17.181 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:17.181 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:17.181 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:32:17.181 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.708543+0000", 2024-09-14T06:32:17.182 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052587416999999997 2024-09-14T06:32:17.182 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:17.182 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:17.182 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:32:17.182 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.708584+0000", 2024-09-14T06:32:17.182 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.1004000000000002e-05 2024-09-14T06:32:17.182 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:17.182 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:17.182 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:32:17.182 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.708636+0000", 2024-09-14T06:32:17.182 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.1530999999999998e-05 2024-09-14T06:32:17.182 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:17.182 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:32:17.182 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:17.183 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:17.183 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:17.183 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:11465 3.1d 3:bcad7303:::rbd_data.112d92027590.0000000000000001:f [read 217088~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:32:17.183 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:25.655961+0000", 2024-09-14T06:32:17.183 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 171.46915142200001, 2024-09-14T06:32:17.183 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052798682999999999, 2024-09-14T06:32:17.183 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:32:17.183 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:32:17.183 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:32:17.183 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:32:17.183 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:32:17.183 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:17.183 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 11465 2024-09-14T06:32:17.183 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:17.184 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:32:17.184 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:17.184 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:32:17.184 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655961+0000", 2024-09-14T06:32:17.184 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:17.184 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:17.184 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:17.184 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:32:17.184 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655961+0000", 2024-09-14T06:32:17.184 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:17.184 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:17.184 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:17.185 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:32:17.185 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.656114+0000", 2024-09-14T06:32:17.185 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.000152745 2024-09-14T06:32:17.185 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:17.185 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:17.185 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:32:17.185 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.656116+0000", 2024-09-14T06:32:17.185 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.804e-06 2024-09-14T06:32:17.185 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:17.185 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:17.185 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:32:17.185 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.656120+0000", 2024-09-14T06:32:17.185 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.4700000000000004e-06 2024-09-14T06:32:17.186 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:17.186 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:17.186 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:32:17.186 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.656123+0000", 2024-09-14T06:32:17.186 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.1269999999999999e-06 2024-09-14T06:32:17.186 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:17.186 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:17.186 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:32:17.186 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.708660+0000", 2024-09-14T06:32:17.186 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052536919000000001 2024-09-14T06:32:17.186 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:17.186 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:17.186 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:32:17.186 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.708703+0000", 2024-09-14T06:32:17.187 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.3077000000000001e-05 2024-09-14T06:32:17.187 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:17.187 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:17.187 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:32:17.187 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.708760+0000", 2024-09-14T06:32:17.187 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.6541e-05 2024-09-14T06:32:17.187 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:17.187 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:32:17.187 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:17.187 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:17.187 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:17.187 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:11466 3.1d 3:bcad7303:::rbd_data.112d92027590.0000000000000001:f [read 227328~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:32:17.187 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:25.656129+0000", 2024-09-14T06:32:17.187 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 171.46898403399999, 2024-09-14T06:32:17.187 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052750987999999999, 2024-09-14T06:32:17.188 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:32:17.188 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:32:17.188 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:32:17.188 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:32:17.188 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:32:17.188 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:17.188 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 11466 2024-09-14T06:32:17.188 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:17.188 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:32:17.188 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:17.188 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:32:17.188 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.656129+0000", 2024-09-14T06:32:17.189 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:17.189 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:17.189 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:17.189 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:32:17.189 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.656129+0000", 2024-09-14T06:32:17.189 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:17.189 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:17.189 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:17.189 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:32:17.189 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.656347+0000", 2024-09-14T06:32:17.189 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.000218054 2024-09-14T06:32:17.189 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:17.189 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:17.189 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:32:17.190 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.656349+0000", 2024-09-14T06:32:17.190 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.9199999999999998e-06 2024-09-14T06:32:17.190 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:17.190 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:17.190 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:32:17.190 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.656353+0000", 2024-09-14T06:32:17.190 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.9210000000000002e-06 2024-09-14T06:32:17.190 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:17.190 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:17.190 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:32:17.191 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.656356+0000", 2024-09-14T06:32:17.191 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.7670000000000001e-06 2024-09-14T06:32:17.191 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:17.191 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:17.191 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:32:17.191 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.708787+0000", 2024-09-14T06:32:17.191 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052430840999999999 2024-09-14T06:32:17.191 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:17.191 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:17.191 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:32:17.191 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.708827+0000", 2024-09-14T06:32:17.191 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.9538e-05 2024-09-14T06:32:17.191 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:17.191 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:17.192 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:32:17.192 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.708880+0000", 2024-09-14T06:32:17.192 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.2947e-05 2024-09-14T06:32:17.192 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:17.192 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:32:17.192 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:17.192 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:17.192 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:17.192 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:11467 3.1d 3:bcad7303:::rbd_data.112d92027590.0000000000000001:f [read 237568~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:32:17.192 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:25.656361+0000", 2024-09-14T06:32:17.192 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 171.46875138600001, 2024-09-14T06:32:17.192 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052643898000000001, 2024-09-14T06:32:17.192 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:32:17.192 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:32:17.193 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:32:17.193 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:32:17.193 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:32:17.193 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:17.193 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 11467 2024-09-14T06:32:17.193 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:17.193 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:32:17.193 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:17.193 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:32:17.193 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.656361+0000", 2024-09-14T06:32:17.193 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:17.193 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:17.193 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:17.193 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:32:17.193 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.656361+0000", 2024-09-14T06:32:17.194 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:17.194 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:17.194 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:17.194 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:32:17.194 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.656855+0000", 2024-09-14T06:32:17.194 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00049372000000000001 2024-09-14T06:32:17.194 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:17.194 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:17.194 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:32:17.194 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.656859+0000", 2024-09-14T06:32:17.194 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.1300000000000003e-06 2024-09-14T06:32:17.194 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:17.194 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:17.194 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:32:17.194 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.656863+0000", 2024-09-14T06:32:17.195 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.2629999999999997e-06 2024-09-14T06:32:17.195 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:17.195 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:17.195 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:32:17.195 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.656867+0000", 2024-09-14T06:32:17.195 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.3220000000000001e-06 2024-09-14T06:32:17.195 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:17.195 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:17.195 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:32:17.195 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.708903+0000", 2024-09-14T06:32:17.195 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052035855999999998 2024-09-14T06:32:17.195 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:17.195 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:17.195 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:32:17.196 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.708949+0000", 2024-09-14T06:32:17.196 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.6276999999999998e-05 2024-09-14T06:32:17.196 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:17.196 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:17.196 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:32:17.196 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.709005+0000", 2024-09-14T06:32:17.196 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.6329999999999998e-05 2024-09-14T06:32:17.196 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:17.196 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:32:17.196 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:17.196 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:17.196 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:17.196 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:11480 3.1d 3:bcad7303:::rbd_data.112d92027590.0000000000000001:f [read 370688~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:32:17.197 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:25.658563+0000", 2024-09-14T06:32:17.197 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 171.466549622, 2024-09-14T06:32:17.197 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052429164, 2024-09-14T06:32:17.197 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:32:17.197 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:32:17.197 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:32:17.197 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:32:17.197 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:32:17.197 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:17.197 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 11480 2024-09-14T06:32:17.197 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:17.197 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:32:17.197 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:17.198 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:32:17.198 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.658563+0000", 2024-09-14T06:32:17.198 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:17.198 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:17.198 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:17.198 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:32:17.198 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.658563+0000", 2024-09-14T06:32:17.198 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:17.198 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:17.198 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:17.199 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:32:17.199 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.658658+0000", 2024-09-14T06:32:17.199 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.5346999999999994e-05 2024-09-14T06:32:17.199 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:17.199 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:17.199 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:32:17.199 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.658662+0000", 2024-09-14T06:32:17.199 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.14e-06 2024-09-14T06:32:17.199 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:17.199 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:17.199 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:32:17.199 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.658669+0000", 2024-09-14T06:32:17.200 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.7379999999999994e-06 2024-09-14T06:32:17.200 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:17.200 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:17.200 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:32:17.200 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.658675+0000", 2024-09-14T06:32:17.200 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.5430000000000001e-06 2024-09-14T06:32:17.200 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:17.200 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:17.200 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:32:17.200 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.710824+0000", 2024-09-14T06:32:17.200 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052148847999999998 2024-09-14T06:32:17.200 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:17.200 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:17.200 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:32:17.201 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.710864+0000", 2024-09-14T06:32:17.201 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.9925999999999997e-05 2024-09-14T06:32:17.201 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:17.201 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:17.201 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:32:17.201 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.710992+0000", 2024-09-14T06:32:17.201 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.000128622 2024-09-14T06:32:17.201 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:17.201 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:32:17.201 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:17.201 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:17.201 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:17.201 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:11481 3.1d 3:bcad7303:::rbd_data.112d92027590.0000000000000001:f [read 380928~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:32:17.202 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:25.658684+0000", 2024-09-14T06:32:17.202 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 171.466428552, 2024-09-14T06:32:17.202 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052508565, 2024-09-14T06:32:17.202 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:32:17.202 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:32:17.202 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:32:17.202 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:32:17.202 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:32:17.202 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:17.202 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 11481 2024-09-14T06:32:17.202 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:17.202 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:32:17.202 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:17.202 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:32:17.203 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.658684+0000", 2024-09-14T06:32:17.203 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:17.203 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:17.203 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:17.203 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:32:17.203 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.658684+0000", 2024-09-14T06:32:17.203 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:17.203 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:17.203 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:17.203 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:32:17.203 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.658715+0000", 2024-09-14T06:32:17.203 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.0703000000000001e-05 2024-09-14T06:32:17.203 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:17.203 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:17.204 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:32:17.204 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.658718+0000", 2024-09-14T06:32:17.204 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.0709999999999999e-06 2024-09-14T06:32:17.204 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:17.204 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:17.204 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:32:17.204 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.658725+0000", 2024-09-14T06:32:17.204 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.2339999999999997e-06 2024-09-14T06:32:17.204 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:17.204 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:17.204 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:32:17.204 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.658731+0000", 2024-09-14T06:32:17.204 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.6450000000000002e-06 2024-09-14T06:32:17.205 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:17.205 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:17.205 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:32:17.205 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.711020+0000", 2024-09-14T06:32:17.205 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052288886 2024-09-14T06:32:17.205 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:17.205 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:17.205 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:32:17.205 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.711061+0000", 2024-09-14T06:32:17.205 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.1084000000000002e-05 2024-09-14T06:32:17.205 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:17.205 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:17.205 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:32:17.205 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.711193+0000", 2024-09-14T06:32:17.206 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00013194199999999999 2024-09-14T06:32:17.206 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:17.206 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:32:17.206 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:17.206 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:17.206 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:17.206 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:11482 3.1d 3:bcad7303:::rbd_data.112d92027590.0000000000000001:f [read 391168~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:32:17.206 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:25.658740+0000", 2024-09-14T06:32:17.206 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 171.46637308699999, 2024-09-14T06:32:17.206 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052637015000000002, 2024-09-14T06:32:17.206 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:32:17.206 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:32:17.207 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:32:17.207 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:32:17.207 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:32:17.207 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:17.207 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 11482 2024-09-14T06:32:17.207 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:17.207 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:32:17.207 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:17.207 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:32:17.207 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.658740+0000", 2024-09-14T06:32:17.207 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:17.207 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:17.207 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:17.208 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:32:17.208 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.658740+0000", 2024-09-14T06:32:17.208 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:17.208 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:17.208 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:17.208 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:32:17.208 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.658880+0000", 2024-09-14T06:32:17.208 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00014036100000000001 2024-09-14T06:32:17.208 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:17.208 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:17.208 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:32:17.208 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.658883+0000", 2024-09-14T06:32:17.209 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.3969999999999998e-06 2024-09-14T06:32:17.209 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:17.209 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:17.209 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:32:17.209 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.658890+0000", 2024-09-14T06:32:17.209 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.1500000000000002e-06 2024-09-14T06:32:17.209 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:17.209 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:17.209 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:32:17.209 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.658897+0000", 2024-09-14T06:32:17.209 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.1260000000000003e-06 2024-09-14T06:32:17.209 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:17.210 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:17.210 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:32:17.210 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.711207+0000", 2024-09-14T06:32:17.210 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052310777000000003 2024-09-14T06:32:17.210 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:17.210 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:17.210 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:32:17.210 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.711249+0000", 2024-09-14T06:32:17.210 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.1786999999999998e-05 2024-09-14T06:32:17.210 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:17.210 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:17.210 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:32:17.210 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.711377+0000", 2024-09-14T06:32:17.210 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.000127417 2024-09-14T06:32:17.211 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:17.211 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:32:17.211 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:17.211 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:17.211 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:17.211 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:11483 3.1d 3:bcad7303:::rbd_data.112d92027590.0000000000000001:f [read 401408~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:32:17.211 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:25.658906+0000", 2024-09-14T06:32:17.211 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 171.46620630300001, 2024-09-14T06:32:17.211 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052595109000000001, 2024-09-14T06:32:17.211 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:32:17.211 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:32:17.211 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:32:17.211 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:32:17.211 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:32:17.212 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:17.212 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 11483 2024-09-14T06:32:17.212 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:17.212 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:32:17.212 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:17.212 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:32:17.212 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.658906+0000", 2024-09-14T06:32:17.212 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:17.212 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:17.212 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:17.212 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:32:17.212 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.658906+0000", 2024-09-14T06:32:17.212 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:17.212 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:17.213 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:17.213 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:32:17.213 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.664985+0000", 2024-09-14T06:32:17.213 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.0060787250000000001 2024-09-14T06:32:17.213 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:17.213 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:17.213 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:32:17.213 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.664991+0000", 2024-09-14T06:32:17.213 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.7030000000000003e-06 2024-09-14T06:32:17.213 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:17.213 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:17.213 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:32:17.213 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.664999+0000", 2024-09-14T06:32:17.214 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.4430000000000003e-06 2024-09-14T06:32:17.214 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:17.214 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:17.214 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:32:17.214 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.665006+0000", 2024-09-14T06:32:17.214 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.0840000000000003e-06 2024-09-14T06:32:17.214 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:17.214 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:17.214 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:32:17.214 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.711402+0000", 2024-09-14T06:32:17.214 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046395714999999997 2024-09-14T06:32:17.214 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:17.214 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:17.214 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:32:17.215 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.711443+0000", 2024-09-14T06:32:17.215 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.0911999999999997e-05 2024-09-14T06:32:17.215 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:17.215 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:17.215 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:32:17.215 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.711501+0000", 2024-09-14T06:32:17.215 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.8526999999999997e-05 2024-09-14T06:32:17.215 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:17.215 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:32:17.215 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:17.215 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:17.215 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:17.215 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:214569 3.2 3:479633bd:::rbd_object_map.112d3bd84cb0:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e170)", 2024-09-14T06:32:17.215 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:31:34.933901+0000", 2024-09-14T06:32:17.216 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 42.191211918999997, 2024-09-14T06:32:17.216 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.68157511400000004, 2024-09-14T06:32:17.216 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:32:17.216 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:32:17.216 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:32:17.216 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:32:17.216 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:32:17.216 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:17.216 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 214569 2024-09-14T06:32:17.217 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:17.217 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:32:17.217 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:17.217 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:32:17.217 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:31:34.933901+0000", 2024-09-14T06:32:17.217 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:17.217 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:17.217 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:17.217 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:32:17.218 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:31:34.933901+0000", 2024-09-14T06:32:17.218 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:17.218 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:17.218 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:17.218 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:32:17.218 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:31:34.933902+0000", 2024-09-14T06:32:17.218 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.7439999999999999e-06 2024-09-14T06:32:17.218 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:17.218 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:17.219 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:32:17.219 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:31:34.933910+0000", 2024-09-14T06:32:17.219 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.8569999999999995e-06 2024-09-14T06:32:17.219 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:17.219 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:17.219 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:32:17.220 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:31:34.933932+0000", 2024-09-14T06:32:17.220 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.1997999999999998e-05 2024-09-14T06:32:17.220 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:17.220 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:17.220 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:32:17.220 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:31:34.933941+0000", 2024-09-14T06:32:17.221 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.7180000000000002e-06 2024-09-14T06:32:17.221 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:17.221 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:17.221 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:32:17.221 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:31:34.934013+0000", 2024-09-14T06:32:17.221 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.1761000000000006e-05 2024-09-14T06:32:17.221 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:17.222 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:17.222 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "waiting for peered", 2024-09-14T06:32:17.222 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:31:34.934022+0000", 2024-09-14T06:32:17.222 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.8740000000000001e-06 2024-09-14T06:32:17.222 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:17.222 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:17.223 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:32:17.223 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:31:35.608803+0000", 2024-09-14T06:32:17.223 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.67478164299999999 2024-09-14T06:32:17.223 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:17.223 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:17.223 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:32:17.223 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:31:35.608964+0000", 2024-09-14T06:32:17.224 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00016112900000000001 2024-09-14T06:32:17.224 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:17.224 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:17.224 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "waiting for subops from 6", 2024-09-14T06:32:17.224 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:31:35.609399+0000", 2024-09-14T06:32:17.224 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00043454000000000002 2024-09-14T06:32:17.225 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:17.225 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:17.225 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "op_commit", 2024-09-14T06:32:17.225 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:31:35.610495+0000", 2024-09-14T06:32:17.225 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.001096404 2024-09-14T06:32:17.225 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:17.226 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:17.226 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:32:17.226 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:31:35.615409+0000", 2024-09-14T06:32:17.226 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.0049133199999999997 2024-09-14T06:32:17.226 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:17.226 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:17.226 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "commit_sent", 2024-09-14T06:32:17.227 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:31:35.615444+0000", 2024-09-14T06:32:17.227 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.5457000000000003e-05 2024-09-14T06:32:17.227 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:17.227 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:17.227 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:32:17.227 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:31:35.615476+0000", 2024-09-14T06:32:17.227 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.1668999999999998e-05 2024-09-14T06:32:17.227 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:17.227 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:32:17.227 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:17.228 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:17.228 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:32:17.228 INFO:teuthology.orchestra.run.smithi138.stdout:} 2024-09-14T06:32:17.229 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-14T06:32:17.229 INFO:tasks.ceph.osd.2.smithi094.stderr:2024-09-14T06:32:17.228+0000 7faf3269b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33649) UID: 0 2024-09-14T06:32:17.330 INFO:tasks.ceph.osd.6.smithi138.stderr:2024-09-14T06:32:17.329+0000 7f8cd8849640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33620) UID: 0 2024-09-14T06:32:17.373 INFO:teuthology.orchestra.run.smithi138.stdout:{ 2024-09-14T06:32:17.373 INFO:teuthology.orchestra.run.smithi138.stdout: "ops": [], 2024-09-14T06:32:17.374 INFO:teuthology.orchestra.run.smithi138.stdout: "num_ops": 0 2024-09-14T06:32:17.374 INFO:teuthology.orchestra.run.smithi138.stdout:} 2024-09-14T06:32:17.382 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-14T06:32:17.431 INFO:tasks.ceph.osd.3.smithi094.stderr:2024-09-14T06:32:17.430+0000 7feeeafe8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33655) UID: 0 2024-09-14T06:32:17.525 INFO:teuthology.orchestra.run.smithi138.stdout:{ 2024-09-14T06:32:17.525 INFO:teuthology.orchestra.run.smithi138.stdout: "ops": [], 2024-09-14T06:32:17.525 INFO:teuthology.orchestra.run.smithi138.stdout: "complaint_time": 30, 2024-09-14T06:32:17.525 INFO:teuthology.orchestra.run.smithi138.stdout: "num_blocked_ops": 0 2024-09-14T06:32:17.526 INFO:teuthology.orchestra.run.smithi138.stdout:} 2024-09-14T06:32:17.532 INFO:tasks.ceph.osd.4.smithi138.stderr:2024-09-14T06:32:17.531+0000 7f6c653c8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33615) UID: 0 2024-09-14T06:32:17.534 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-14T06:32:17.584 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-14T06:32:17.631 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-14T06:32:17.632 INFO:tasks.ceph.osd.5.smithi138.stderr:2024-09-14T06:32:17.631+0000 7fe13746f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33618) UID: 0 2024-09-14T06:32:17.671 INFO:teuthology.orchestra.run.smithi094.stderr:reweighted osd.3 to 1 (10000) 2024-09-14T06:32:17.679 INFO:teuthology.orchestra.run.smithi138.stdout:{ 2024-09-14T06:32:17.679 INFO:teuthology.orchestra.run.smithi138.stdout: "size": 20, 2024-09-14T06:32:17.680 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 600, 2024-09-14T06:32:17.680 INFO:teuthology.orchestra.run.smithi138.stdout: "ops": [ 2024-09-14T06:32:17.680 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:17.680 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:98505 3.10 3.1bf56f90 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T06:32:17.680 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:30:07.045453+0000", 2024-09-14T06:32:17.680 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 130.63259869999999, 2024-09-14T06:32:17.680 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032144440000000003, 2024-09-14T06:32:17.680 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:32:17.680 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:32:17.680 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "reached pg", 2024-09-14T06:32:17.680 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:32:17.680 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:32:17.680 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:17.681 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 98505 2024-09-14T06:32:17.681 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:17.681 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:32:17.681 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:17.681 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:32:17.681 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045453+0000", 2024-09-14T06:32:17.681 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:17.681 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:17.681 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:17.681 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:32:17.681 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045453+0000", 2024-09-14T06:32:17.681 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:17.682 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:17.682 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:17.682 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:32:17.682 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045453+0000", 2024-09-14T06:32:17.682 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.36e-07 2024-09-14T06:32:17.682 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:17.682 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:17.682 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:32:17.682 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045454+0000", 2024-09-14T06:32:17.682 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.1299999999999998e-07 2024-09-14T06:32:17.682 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:17.682 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:17.682 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:32:17.682 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045457+0000", 2024-09-14T06:32:17.682 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.8559999999999998e-06 2024-09-14T06:32:17.683 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:17.683 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:17.683 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:32:17.683 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045460+0000", 2024-09-14T06:32:17.683 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.8750000000000002e-06 2024-09-14T06:32:17.683 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:17.683 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:17.683 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:32:17.683 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.077592+0000", 2024-09-14T06:32:17.683 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032131727999999998 2024-09-14T06:32:17.683 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:17.683 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:17.683 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:32:17.683 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.077597+0000", 2024-09-14T06:32:17.684 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.8319999999999996e-06 2024-09-14T06:32:17.684 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:17.684 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:32:17.684 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:17.684 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:17.684 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:17.684 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:98506 3.10 3.1bf56f90 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T06:32:17.684 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:30:07.045481+0000", 2024-09-14T06:32:17.684 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 130.632570445, 2024-09-14T06:32:17.684 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032142759999999999, 2024-09-14T06:32:17.684 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:32:17.684 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:32:17.684 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "reached pg", 2024-09-14T06:32:17.685 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:32:17.685 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:32:17.685 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:17.685 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 98506 2024-09-14T06:32:17.685 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:17.685 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:32:17.685 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:17.685 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:32:17.685 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045481+0000", 2024-09-14T06:32:17.685 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:17.685 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:17.685 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:17.685 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:32:17.685 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045481+0000", 2024-09-14T06:32:17.686 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:17.686 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:17.686 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:17.686 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:32:17.686 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045481+0000", 2024-09-14T06:32:17.686 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.8500000000000001e-07 2024-09-14T06:32:17.686 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:17.686 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:17.686 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:32:17.686 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045483+0000", 2024-09-14T06:32:17.686 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.8250000000000001e-06 2024-09-14T06:32:17.686 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:17.686 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:17.687 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:32:17.687 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045489+0000", 2024-09-14T06:32:17.687 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.5740000000000003e-06 2024-09-14T06:32:17.687 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:17.687 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:17.687 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:32:17.687 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045495+0000", 2024-09-14T06:32:17.687 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.7690000000000002e-06 2024-09-14T06:32:17.687 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:17.687 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:17.687 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:32:17.687 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.077619+0000", 2024-09-14T06:32:17.687 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032124621999999999 2024-09-14T06:32:17.687 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:17.688 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:17.688 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:32:17.688 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.077624+0000", 2024-09-14T06:32:17.688 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.3850000000000004e-06 2024-09-14T06:32:17.688 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:17.688 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:32:17.688 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:17.688 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:17.688 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:17.688 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:98507 3.10 3.1bf56f90 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T06:32:17.688 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:30:07.045502+0000", 2024-09-14T06:32:17.688 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 130.63254902400001, 2024-09-14T06:32:17.688 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032147219999999997, 2024-09-14T06:32:17.689 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:32:17.689 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:32:17.689 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "reached pg", 2024-09-14T06:32:17.689 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:32:17.689 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:32:17.689 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:17.689 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 98507 2024-09-14T06:32:17.689 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:17.689 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:32:17.689 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:17.689 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:32:17.689 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045502+0000", 2024-09-14T06:32:17.689 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:17.690 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:17.690 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:17.690 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:32:17.690 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045502+0000", 2024-09-14T06:32:17.690 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:17.690 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:17.690 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:17.690 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:32:17.690 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045503+0000", 2024-09-14T06:32:17.690 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.4700000000000002e-07 2024-09-14T06:32:17.690 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:17.690 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:17.690 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:32:17.690 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045509+0000", 2024-09-14T06:32:17.691 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.7329999999999997e-06 2024-09-14T06:32:17.691 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:17.691 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:17.691 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:32:17.691 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045515+0000", 2024-09-14T06:32:17.691 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.8030000000000002e-06 2024-09-14T06:32:17.691 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:17.691 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:17.691 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:32:17.691 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045522+0000", 2024-09-14T06:32:17.691 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.0369999999999997e-06 2024-09-14T06:32:17.691 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:17.691 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:17.692 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:32:17.692 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.077641+0000", 2024-09-14T06:32:17.692 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032119091000000002 2024-09-14T06:32:17.692 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:17.692 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:17.692 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:32:17.692 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.077650+0000", 2024-09-14T06:32:17.692 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.2090000000000005e-06 2024-09-14T06:32:17.692 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:17.693 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:32:17.693 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:17.693 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:17.693 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:17.693 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:98508 3.10 3.1bf56f90 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T06:32:17.693 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:30:07.045530+0000", 2024-09-14T06:32:17.693 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 130.63252140700001, 2024-09-14T06:32:17.693 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032198773999999999, 2024-09-14T06:32:17.693 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:32:17.693 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:32:17.693 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "reached pg", 2024-09-14T06:32:17.693 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:32:17.693 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:32:17.694 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:17.694 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 98508 2024-09-14T06:32:17.694 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:17.694 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:32:17.694 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:17.694 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:32:17.694 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045530+0000", 2024-09-14T06:32:17.694 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:17.694 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:17.694 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:17.694 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:32:17.694 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045530+0000", 2024-09-14T06:32:17.694 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:17.694 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:17.695 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:17.695 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:32:17.695 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045530+0000", 2024-09-14T06:32:17.695 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.3099999999999998e-07 2024-09-14T06:32:17.695 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:17.695 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:17.695 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:32:17.695 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045539+0000", 2024-09-14T06:32:17.695 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.4780000000000008e-06 2024-09-14T06:32:17.695 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:17.695 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:17.695 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:32:17.695 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045546+0000", 2024-09-14T06:32:17.695 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.3259999999999998e-06 2024-09-14T06:32:17.695 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:17.696 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:17.696 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:32:17.696 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045551+0000", 2024-09-14T06:32:17.696 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.9960000000000001e-06 2024-09-14T06:32:17.696 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:17.696 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:17.696 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:32:17.696 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.077716+0000", 2024-09-14T06:32:17.696 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032165081999999998 2024-09-14T06:32:17.696 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:17.696 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:17.696 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:32:17.696 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.077729+0000", 2024-09-14T06:32:17.697 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.2461e-05 2024-09-14T06:32:17.697 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:17.697 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:32:17.697 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:17.697 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:17.697 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:17.697 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:98509 3.10 3.1bf56f90 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T06:32:17.697 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:30:07.045558+0000", 2024-09-14T06:32:17.697 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 130.63249320599999, 2024-09-14T06:32:17.697 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032242198, 2024-09-14T06:32:17.697 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:32:17.697 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:32:17.697 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "reached pg", 2024-09-14T06:32:17.697 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:32:17.698 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:32:17.698 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:17.698 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 98509 2024-09-14T06:32:17.698 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:17.698 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:32:17.698 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:17.698 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:32:17.698 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045558+0000", 2024-09-14T06:32:17.698 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:17.698 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:17.698 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:17.698 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:32:17.698 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045558+0000", 2024-09-14T06:32:17.698 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:17.699 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:17.699 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:17.699 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:32:17.699 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045559+0000", 2024-09-14T06:32:17.699 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.27e-07 2024-09-14T06:32:17.699 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:17.699 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:17.699 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:32:17.699 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045560+0000", 2024-09-14T06:32:17.699 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.6959999999999999e-06 2024-09-14T06:32:17.699 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:17.699 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:17.699 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:32:17.700 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045567+0000", 2024-09-14T06:32:17.700 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.6900000000000003e-06 2024-09-14T06:32:17.700 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:17.700 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:17.700 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:32:17.700 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045571+0000", 2024-09-14T06:32:17.700 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.5750000000000002e-06 2024-09-14T06:32:17.700 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:17.700 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:17.700 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:32:17.700 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.077789+0000", 2024-09-14T06:32:17.700 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032217186000000002 2024-09-14T06:32:17.700 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:17.701 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:17.701 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:32:17.701 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.077800+0000", 2024-09-14T06:32:17.701 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.1623999999999999e-05 2024-09-14T06:32:17.701 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:17.701 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:32:17.701 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:17.701 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:17.701 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:17.701 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:98510 3.10 3.1bf56f90 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T06:32:17.701 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:30:07.045580+0000", 2024-09-14T06:32:17.701 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 130.63247139200001, 2024-09-14T06:32:17.701 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032299845000000001, 2024-09-14T06:32:17.702 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:32:17.702 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:32:17.702 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "reached pg", 2024-09-14T06:32:17.702 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:32:17.702 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:32:17.702 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:17.702 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 98510 2024-09-14T06:32:17.702 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:17.702 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:32:17.702 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:17.702 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:32:17.702 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045580+0000", 2024-09-14T06:32:17.702 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:17.702 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:17.703 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:17.703 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:32:17.703 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045580+0000", 2024-09-14T06:32:17.703 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:17.703 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:17.703 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:17.703 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:32:17.703 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045580+0000", 2024-09-14T06:32:17.703 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.0400000000000002e-07 2024-09-14T06:32:17.703 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:17.703 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:17.703 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:32:17.703 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045582+0000", 2024-09-14T06:32:17.703 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.7600000000000001e-06 2024-09-14T06:32:17.704 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:17.704 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:17.704 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:32:17.704 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045588+0000", 2024-09-14T06:32:17.704 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.5650000000000001e-06 2024-09-14T06:32:17.704 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:17.704 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:17.704 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:32:17.704 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045597+0000", 2024-09-14T06:32:17.704 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.465e-06 2024-09-14T06:32:17.704 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:17.704 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:17.704 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:32:17.705 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.077871+0000", 2024-09-14T06:32:17.705 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032273562999999998 2024-09-14T06:32:17.705 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:17.705 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:17.705 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:32:17.705 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.077880+0000", 2024-09-14T06:32:17.705 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.0880000000000001e-06 2024-09-14T06:32:17.705 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:17.705 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:32:17.705 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:17.705 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:17.705 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:17.705 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:98511 3.10 3.1bf56f90 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T06:32:17.706 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:30:07.045610+0000", 2024-09-14T06:32:17.706 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 130.63244154500001, 2024-09-14T06:32:17.706 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032338894, 2024-09-14T06:32:17.706 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:32:17.706 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:32:17.706 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "reached pg", 2024-09-14T06:32:17.706 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:32:17.706 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:32:17.706 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:17.706 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 98511 2024-09-14T06:32:17.706 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:17.706 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:32:17.706 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:17.706 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:32:17.706 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045610+0000", 2024-09-14T06:32:17.707 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:17.707 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:17.707 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:17.707 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:32:17.707 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045610+0000", 2024-09-14T06:32:17.707 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:17.707 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:17.707 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:17.707 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:32:17.707 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045610+0000", 2024-09-14T06:32:17.707 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.1600000000000002e-07 2024-09-14T06:32:17.707 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:17.707 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:17.707 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:32:17.708 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045612+0000", 2024-09-14T06:32:17.708 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.1150000000000001e-06 2024-09-14T06:32:17.708 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:17.708 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:17.708 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:32:17.708 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045618+0000", 2024-09-14T06:32:17.708 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.0499999999999997e-06 2024-09-14T06:32:17.708 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:17.708 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:17.708 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:32:17.708 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045623+0000", 2024-09-14T06:32:17.708 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.6770000000000003e-06 2024-09-14T06:32:17.708 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:17.709 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:17.709 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:32:17.709 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.077928+0000", 2024-09-14T06:32:17.709 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032305111999999997 2024-09-14T06:32:17.709 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:17.709 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:17.709 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:32:17.709 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.077949+0000", 2024-09-14T06:32:17.709 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.0523999999999999e-05 2024-09-14T06:32:17.709 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:17.709 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:32:17.709 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:17.709 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:17.709 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:17.710 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:98512 3.10 3.1bf56f90 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T06:32:17.710 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:30:07.045630+0000", 2024-09-14T06:32:17.710 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 130.63242154, 2024-09-14T06:32:17.710 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032346632, 2024-09-14T06:32:17.710 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:32:17.710 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:32:17.710 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "reached pg", 2024-09-14T06:32:17.710 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:32:17.710 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:32:17.710 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:17.710 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 98512 2024-09-14T06:32:17.710 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:17.710 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:32:17.711 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:17.711 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:32:17.711 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045630+0000", 2024-09-14T06:32:17.711 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:17.711 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:17.711 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:17.711 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:32:17.711 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045630+0000", 2024-09-14T06:32:17.711 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:17.711 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:17.711 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:17.711 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:32:17.711 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045630+0000", 2024-09-14T06:32:17.711 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.8200000000000001e-07 2024-09-14T06:32:17.712 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:17.712 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:17.712 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:32:17.712 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045632+0000", 2024-09-14T06:32:17.712 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.7430000000000001e-06 2024-09-14T06:32:17.712 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:17.712 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:17.712 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:32:17.712 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045637+0000", 2024-09-14T06:32:17.712 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.4550000000000003e-06 2024-09-14T06:32:17.712 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:17.712 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:17.712 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:32:17.713 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045644+0000", 2024-09-14T06:32:17.713 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.1330000000000001e-06 2024-09-14T06:32:17.713 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:17.713 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:17.713 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:32:17.713 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.077968+0000", 2024-09-14T06:32:17.713 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032323401000000002 2024-09-14T06:32:17.713 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:17.713 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:17.713 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:32:17.713 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.077976+0000", 2024-09-14T06:32:17.713 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.5180000000000004e-06 2024-09-14T06:32:17.713 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:17.714 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:32:17.714 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:17.714 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:17.714 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:17.714 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:98513 3.10 3.1bf56f90 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T06:32:17.714 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:30:07.045652+0000", 2024-09-14T06:32:17.714 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 130.63239897599999, 2024-09-14T06:32:17.714 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032353436999999999, 2024-09-14T06:32:17.714 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:32:17.714 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:32:17.714 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "reached pg", 2024-09-14T06:32:17.714 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:32:17.714 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:32:17.714 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:17.715 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 98513 2024-09-14T06:32:17.715 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:17.715 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:32:17.715 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:17.715 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:32:17.715 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045652+0000", 2024-09-14T06:32:17.715 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:17.715 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:17.715 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:17.715 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:32:17.715 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045652+0000", 2024-09-14T06:32:17.715 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:17.715 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:17.715 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:17.716 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:32:17.716 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045653+0000", 2024-09-14T06:32:17.716 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.3700000000000001e-07 2024-09-14T06:32:17.716 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:17.716 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:17.716 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:32:17.716 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045655+0000", 2024-09-14T06:32:17.716 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.854e-06 2024-09-14T06:32:17.716 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:17.716 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:17.716 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:32:17.716 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045661+0000", 2024-09-14T06:32:17.716 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.5509999999999996e-06 2024-09-14T06:32:17.716 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:17.717 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:17.717 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:32:17.717 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045668+0000", 2024-09-14T06:32:17.717 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.1980000000000004e-06 2024-09-14T06:32:17.717 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:17.717 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:17.717 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:32:17.717 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.077997+0000", 2024-09-14T06:32:17.717 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032328575999999998 2024-09-14T06:32:17.717 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:17.717 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:17.717 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:32:17.717 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.078006+0000", 2024-09-14T06:32:17.718 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.8210000000000008e-06 2024-09-14T06:32:17.718 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:17.718 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:32:17.718 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:17.718 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:17.718 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:17.718 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:98514 3.10 3.1bf56f90 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T06:32:17.718 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:30:07.045699+0000", 2024-09-14T06:32:17.718 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 130.63235264100001, 2024-09-14T06:32:17.718 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032330223999999998, 2024-09-14T06:32:17.718 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:32:17.718 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:32:17.718 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "reached pg", 2024-09-14T06:32:17.718 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:32:17.719 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:32:17.719 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:17.719 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 98514 2024-09-14T06:32:17.719 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:17.719 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:32:17.719 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:17.719 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:32:17.719 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045699+0000", 2024-09-14T06:32:17.719 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:17.719 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:17.719 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:17.719 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:32:17.719 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045699+0000", 2024-09-14T06:32:17.719 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:17.720 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:17.720 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:17.720 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:32:17.720 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045699+0000", 2024-09-14T06:32:17.720 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.3200000000000001e-07 2024-09-14T06:32:17.720 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:17.720 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:17.720 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:32:17.720 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045701+0000", 2024-09-14T06:32:17.720 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.762e-06 2024-09-14T06:32:17.720 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:17.720 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:17.720 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:32:17.721 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045705+0000", 2024-09-14T06:32:17.721 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.2359999999999998e-06 2024-09-14T06:32:17.721 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:17.721 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:17.721 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:32:17.721 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045708+0000", 2024-09-14T06:32:17.721 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.796e-06 2024-09-14T06:32:17.721 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:17.721 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:17.721 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:32:17.721 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.078023+0000", 2024-09-14T06:32:17.721 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032315213000000002 2024-09-14T06:32:17.721 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:17.721 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:17.722 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:32:17.722 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.078029+0000", 2024-09-14T06:32:17.722 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.8850000000000004e-06 2024-09-14T06:32:17.722 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:17.722 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:32:17.722 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:17.722 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:17.722 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:17.722 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:98515 3.10 3.1bf56f90 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T06:32:17.722 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:30:07.045712+0000", 2024-09-14T06:32:17.722 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 130.63233933000001, 2024-09-14T06:32:17.722 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032346871999999999, 2024-09-14T06:32:17.722 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:32:17.722 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:32:17.722 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "reached pg", 2024-09-14T06:32:17.723 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:32:17.723 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:32:17.723 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:17.723 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 98515 2024-09-14T06:32:17.723 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:17.723 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:32:17.723 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:17.723 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:32:17.723 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045712+0000", 2024-09-14T06:32:17.723 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:17.723 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:17.723 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:17.723 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:32:17.723 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045712+0000", 2024-09-14T06:32:17.724 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:17.724 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:17.724 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:17.724 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:32:17.724 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045712+0000", 2024-09-14T06:32:17.724 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.2100000000000001e-07 2024-09-14T06:32:17.724 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:17.724 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:17.724 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:32:17.724 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045713+0000", 2024-09-14T06:32:17.724 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.068e-06 2024-09-14T06:32:17.724 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:17.724 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:17.725 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:32:17.725 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045716+0000", 2024-09-14T06:32:17.725 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.971e-06 2024-09-14T06:32:17.725 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:17.725 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:17.725 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:32:17.725 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045719+0000", 2024-09-14T06:32:17.725 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.3879999999999998e-06 2024-09-14T06:32:17.725 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:17.725 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:17.725 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:32:17.725 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.078049+0000", 2024-09-14T06:32:17.725 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032330553999999997 2024-09-14T06:32:17.726 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:17.726 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:17.726 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:32:17.726 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.078059+0000", 2024-09-14T06:32:17.726 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.6700000000000006e-06 2024-09-14T06:32:17.726 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:17.726 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:32:17.726 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:17.726 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:17.726 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:17.726 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:98516 3.10 3.1bf56f90 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T06:32:17.726 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:30:07.045722+0000", 2024-09-14T06:32:17.726 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 130.63232957700001, 2024-09-14T06:32:17.726 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032374334999999997, 2024-09-14T06:32:17.726 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:32:17.727 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:32:17.727 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "reached pg", 2024-09-14T06:32:17.727 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:32:17.727 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:32:17.727 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:17.727 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 98516 2024-09-14T06:32:17.727 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:17.727 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:32:17.727 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:17.727 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:32:17.727 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045722+0000", 2024-09-14T06:32:17.727 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:17.727 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:17.727 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:17.728 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:32:17.728 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045722+0000", 2024-09-14T06:32:17.728 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:17.728 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:17.728 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:17.728 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:32:17.728 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045722+0000", 2024-09-14T06:32:17.728 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.8999999999999998e-07 2024-09-14T06:32:17.728 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:17.728 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:17.728 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:32:17.728 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045723+0000", 2024-09-14T06:32:17.728 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.0049999999999999e-06 2024-09-14T06:32:17.728 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:17.729 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:17.729 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:32:17.729 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045726+0000", 2024-09-14T06:32:17.729 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.9019999999999999e-06 2024-09-14T06:32:17.729 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:17.729 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:17.729 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:32:17.729 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045728+0000", 2024-09-14T06:32:17.729 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.3290000000000001e-06 2024-09-14T06:32:17.729 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:17.729 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:17.729 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:32:17.729 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.078090+0000", 2024-09-14T06:32:17.730 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032362123 2024-09-14T06:32:17.730 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:17.730 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:17.730 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:32:17.730 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.078096+0000", 2024-09-14T06:32:17.730 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.6860000000000003e-06 2024-09-14T06:32:17.730 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:17.730 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:32:17.730 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:17.730 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:17.730 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:17.730 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:98517 3.10 3.1bf56f90 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T06:32:17.730 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:30:07.045731+0000", 2024-09-14T06:32:17.731 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 130.632319846, 2024-09-14T06:32:17.731 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.03241227, 2024-09-14T06:32:17.731 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:32:17.731 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:32:17.731 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "reached pg", 2024-09-14T06:32:17.731 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:32:17.731 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:32:17.731 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:17.731 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 98517 2024-09-14T06:32:17.731 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:17.731 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:32:17.731 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:17.731 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:32:17.731 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045731+0000", 2024-09-14T06:32:17.732 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:17.732 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:17.732 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:17.732 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:32:17.732 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045731+0000", 2024-09-14T06:32:17.732 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:17.732 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:17.732 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:17.732 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:32:17.732 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045732+0000", 2024-09-14T06:32:17.732 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.0699999999999999e-07 2024-09-14T06:32:17.732 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:17.732 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:17.733 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:32:17.733 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045733+0000", 2024-09-14T06:32:17.733 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.9199999999999999e-07 2024-09-14T06:32:17.733 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:17.733 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:17.733 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:32:17.733 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045735+0000", 2024-09-14T06:32:17.733 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.8150000000000002e-06 2024-09-14T06:32:17.733 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:17.733 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:17.733 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:32:17.733 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045738+0000", 2024-09-14T06:32:17.733 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.3640000000000002e-06 2024-09-14T06:32:17.733 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:17.734 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:17.734 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:32:17.734 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.078126+0000", 2024-09-14T06:32:17.734 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032388206000000003 2024-09-14T06:32:17.734 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:17.734 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:17.734 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:32:17.734 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.078144+0000", 2024-09-14T06:32:17.734 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.7686e-05 2024-09-14T06:32:17.734 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:17.734 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:32:17.734 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:17.734 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:17.734 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:17.735 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:98518 3.10 3.1bf56f90 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T06:32:17.735 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:30:07.045741+0000", 2024-09-14T06:32:17.735 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 130.632310451, 2024-09-14T06:32:17.735 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032451793999999999, 2024-09-14T06:32:17.735 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:32:17.735 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:32:17.735 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "reached pg", 2024-09-14T06:32:17.735 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:32:17.735 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:32:17.735 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:17.735 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 98518 2024-09-14T06:32:17.735 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:17.735 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:32:17.735 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:17.736 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:32:17.736 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045741+0000", 2024-09-14T06:32:17.736 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:17.736 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:17.736 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:17.736 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:32:17.736 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045741+0000", 2024-09-14T06:32:17.736 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:17.736 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:17.736 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:17.736 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:32:17.736 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045741+0000", 2024-09-14T06:32:17.737 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.05e-07 2024-09-14T06:32:17.737 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:17.737 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:17.737 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:32:17.737 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045746+0000", 2024-09-14T06:32:17.737 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.2259999999999996e-06 2024-09-14T06:32:17.737 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:17.737 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:17.737 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:32:17.737 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045750+0000", 2024-09-14T06:32:17.737 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.2490000000000001e-06 2024-09-14T06:32:17.737 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:17.737 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:17.737 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:32:17.738 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045752+0000", 2024-09-14T06:32:17.738 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.3980000000000002e-06 2024-09-14T06:32:17.738 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:17.738 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:17.738 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:32:17.738 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.078184+0000", 2024-09-14T06:32:17.738 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032431593000000002 2024-09-14T06:32:17.738 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:17.738 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:17.738 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:32:17.738 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.078193+0000", 2024-09-14T06:32:17.738 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.1230000000000006e-06 2024-09-14T06:32:17.738 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:17.739 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:32:17.739 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:17.739 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:17.739 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:17.739 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:98519 3.10 3.1bf56f90 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T06:32:17.739 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:30:07.045755+0000", 2024-09-14T06:32:17.739 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 130.63229602600001, 2024-09-14T06:32:17.739 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032489431999999999, 2024-09-14T06:32:17.739 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:32:17.739 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:32:17.739 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "reached pg", 2024-09-14T06:32:17.739 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:32:17.739 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:32:17.740 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:17.740 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 98519 2024-09-14T06:32:17.740 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:17.740 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:32:17.740 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:17.740 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:32:17.740 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045755+0000", 2024-09-14T06:32:17.740 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:17.740 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:17.740 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:17.740 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:32:17.740 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045755+0000", 2024-09-14T06:32:17.740 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:17.740 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:17.741 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:17.741 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:32:17.741 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045756+0000", 2024-09-14T06:32:17.741 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.23e-07 2024-09-14T06:32:17.741 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:17.741 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:17.741 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:32:17.741 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045757+0000", 2024-09-14T06:32:17.741 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.1060000000000001e-06 2024-09-14T06:32:17.741 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:17.741 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:17.741 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:32:17.741 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045760+0000", 2024-09-14T06:32:17.742 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.988e-06 2024-09-14T06:32:17.742 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:17.742 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:17.742 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:32:17.742 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045765+0000", 2024-09-14T06:32:17.742 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.2689999999999999e-06 2024-09-14T06:32:17.742 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:17.742 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:17.742 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:32:17.742 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.078234+0000", 2024-09-14T06:32:17.742 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032468915000000001 2024-09-14T06:32:17.742 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:17.742 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:17.742 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:32:17.743 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.078245+0000", 2024-09-14T06:32:17.743 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.0930999999999999e-05 2024-09-14T06:32:17.743 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:17.743 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:32:17.743 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:17.743 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:17.743 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:17.743 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:98520 3.10 3.1bf56f90 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T06:32:17.743 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:30:07.045768+0000", 2024-09-14T06:32:17.743 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 130.632283148, 2024-09-14T06:32:17.743 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032531379999999999, 2024-09-14T06:32:17.743 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:32:17.743 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:32:17.744 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "reached pg", 2024-09-14T06:32:17.744 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:32:17.744 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:32:17.744 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:17.744 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 98520 2024-09-14T06:32:17.744 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:17.744 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:32:17.744 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:17.744 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:32:17.744 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045768+0000", 2024-09-14T06:32:17.744 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:17.744 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:17.744 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:17.745 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:32:17.745 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045768+0000", 2024-09-14T06:32:17.745 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:17.745 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:17.745 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:17.745 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:32:17.745 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045768+0000", 2024-09-14T06:32:17.745 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.22e-07 2024-09-14T06:32:17.745 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:17.745 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:17.745 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:32:17.745 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045773+0000", 2024-09-14T06:32:17.745 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.3470000000000001e-06 2024-09-14T06:32:17.745 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:17.746 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:17.746 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:32:17.746 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045776+0000", 2024-09-14T06:32:17.746 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.1319999999999998e-06 2024-09-14T06:32:17.746 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:17.746 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:17.746 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:32:17.746 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045779+0000", 2024-09-14T06:32:17.746 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.9749999999999999e-06 2024-09-14T06:32:17.746 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:17.746 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:17.746 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:32:17.746 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.078289+0000", 2024-09-14T06:32:17.747 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032509693999999999 2024-09-14T06:32:17.747 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:17.747 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:17.747 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:32:17.747 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.078300+0000", 2024-09-14T06:32:17.747 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.101e-05 2024-09-14T06:32:17.747 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:17.747 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:32:17.747 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:17.747 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:17.747 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:17.747 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:98521 3.10 3.1bf56f90 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T06:32:17.747 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:30:07.045782+0000", 2024-09-14T06:32:17.747 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 130.63226896800001, 2024-09-14T06:32:17.748 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032566109000000003, 2024-09-14T06:32:17.748 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:32:17.748 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:32:17.748 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "reached pg", 2024-09-14T06:32:17.748 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:32:17.748 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:32:17.748 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:17.748 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 98521 2024-09-14T06:32:17.748 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:17.748 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:32:17.748 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:17.748 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:32:17.748 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045782+0000", 2024-09-14T06:32:17.749 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:17.749 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:17.749 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:17.749 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:32:17.749 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045782+0000", 2024-09-14T06:32:17.749 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:17.749 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:17.749 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:17.749 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:32:17.749 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045783+0000", 2024-09-14T06:32:17.749 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.2499999999999999e-07 2024-09-14T06:32:17.749 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:17.749 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:17.749 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:32:17.750 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045784+0000", 2024-09-14T06:32:17.750 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.0109999999999999e-06 2024-09-14T06:32:17.750 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:17.750 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:17.750 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:32:17.750 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045786+0000", 2024-09-14T06:32:17.750 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.8289999999999999e-06 2024-09-14T06:32:17.750 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:17.750 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:17.750 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:32:17.750 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045793+0000", 2024-09-14T06:32:17.750 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.579e-06 2024-09-14T06:32:17.750 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:17.750 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:17.751 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:32:17.751 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.078338+0000", 2024-09-14T06:32:17.751 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032545389000000001 2024-09-14T06:32:17.751 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:17.751 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:17.751 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:32:17.751 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.078348+0000", 2024-09-14T06:32:17.751 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.0076e-05 2024-09-14T06:32:17.751 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:17.751 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:32:17.751 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:17.751 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:17.751 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:17.752 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:98522 3.10 3.1bf56f90 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T06:32:17.752 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:30:07.045796+0000", 2024-09-14T06:32:17.752 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 130.63225506000001, 2024-09-14T06:32:17.752 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032577846000000001, 2024-09-14T06:32:17.752 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:32:17.752 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:32:17.752 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "reached pg", 2024-09-14T06:32:17.752 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:32:17.752 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:32:17.752 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:17.752 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 98522 2024-09-14T06:32:17.752 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:17.752 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:32:17.752 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:17.753 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:32:17.753 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045796+0000", 2024-09-14T06:32:17.753 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:17.753 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:17.753 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:17.753 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:32:17.753 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045796+0000", 2024-09-14T06:32:17.753 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:17.753 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:17.753 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:17.753 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:32:17.753 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045796+0000", 2024-09-14T06:32:17.753 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.2100000000000001e-07 2024-09-14T06:32:17.754 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:17.754 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:17.754 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:32:17.754 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045797+0000", 2024-09-14T06:32:17.754 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.6899999999999996e-07 2024-09-14T06:32:17.754 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:17.754 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:17.754 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:32:17.754 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045800+0000", 2024-09-14T06:32:17.754 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.9979999999999999e-06 2024-09-14T06:32:17.754 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:17.754 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:17.754 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:32:17.754 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045803+0000", 2024-09-14T06:32:17.755 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.2850000000000001e-06 2024-09-14T06:32:17.755 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:17.755 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:17.755 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:32:17.755 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.078366+0000", 2024-09-14T06:32:17.755 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032562932000000003 2024-09-14T06:32:17.755 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:17.755 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:17.755 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:32:17.755 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.078374+0000", 2024-09-14T06:32:17.755 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.4409999999999993e-06 2024-09-14T06:32:17.755 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:17.756 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:32:17.756 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:17.756 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:17.756 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:17.756 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:98523 3.10 3.1bf56f90 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T06:32:17.756 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:30:07.045806+0000", 2024-09-14T06:32:17.756 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 130.632245517, 2024-09-14T06:32:17.756 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032611167000000003, 2024-09-14T06:32:17.756 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:32:17.756 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:32:17.756 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "reached pg", 2024-09-14T06:32:17.756 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:32:17.756 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:32:17.756 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:17.757 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 98523 2024-09-14T06:32:17.757 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:17.757 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:32:17.757 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:17.757 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:32:17.757 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045806+0000", 2024-09-14T06:32:17.757 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:17.757 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:17.757 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:17.757 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:32:17.757 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045806+0000", 2024-09-14T06:32:17.757 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:17.758 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:17.758 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:17.758 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:32:17.758 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045806+0000", 2024-09-14T06:32:17.758 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.2000000000000001e-07 2024-09-14T06:32:17.758 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:17.758 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:17.758 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:32:17.758 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045807+0000", 2024-09-14T06:32:17.758 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.4e-07 2024-09-14T06:32:17.758 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:17.758 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:17.758 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:32:17.758 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045810+0000", 2024-09-14T06:32:17.759 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.9229999999999998e-06 2024-09-14T06:32:17.759 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:17.759 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:17.759 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:32:17.759 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045812+0000", 2024-09-14T06:32:17.759 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.2220000000000001e-06 2024-09-14T06:32:17.759 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:17.759 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:17.759 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:32:17.759 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.078407+0000", 2024-09-14T06:32:17.759 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032594513999999998 2024-09-14T06:32:17.759 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:17.759 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:17.760 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:32:17.760 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.078417+0000", 2024-09-14T06:32:17.760 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.0348e-05 2024-09-14T06:32:17.760 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:17.760 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:32:17.760 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:17.760 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:17.760 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:17.760 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:98524 3.10 3.1bf56f90 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T06:32:17.760 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:30:07.045815+0000", 2024-09-14T06:32:17.760 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 130.63223614899999, 2024-09-14T06:32:17.760 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032643208999999999, 2024-09-14T06:32:17.760 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:32:17.761 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:32:17.761 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "reached pg", 2024-09-14T06:32:17.761 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:32:17.761 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:32:17.761 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:17.761 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 98524 2024-09-14T06:32:17.761 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:17.761 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:32:17.761 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:17.761 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:32:17.761 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045815+0000", 2024-09-14T06:32:17.761 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:17.761 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:17.761 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:17.761 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:32:17.762 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045815+0000", 2024-09-14T06:32:17.762 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:17.762 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:17.762 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:17.762 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:32:17.762 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045815+0000", 2024-09-14T06:32:17.762 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.11e-07 2024-09-14T06:32:17.762 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:17.762 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:17.762 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:32:17.762 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045816+0000", 2024-09-14T06:32:17.762 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.9099999999999991e-07 2024-09-14T06:32:17.762 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:17.763 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:17.763 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:32:17.763 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045819+0000", 2024-09-14T06:32:17.763 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.9399999999999998e-06 2024-09-14T06:32:17.763 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:17.763 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:17.763 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:32:17.763 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045822+0000", 2024-09-14T06:32:17.763 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.3470000000000001e-06 2024-09-14T06:32:17.763 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:17.763 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:17.763 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:32:17.763 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.078449+0000", 2024-09-14T06:32:17.764 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032627529000000002 2024-09-14T06:32:17.764 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:17.764 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:17.764 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:32:17.764 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.078458+0000", 2024-09-14T06:32:17.764 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.1910000000000006e-06 2024-09-14T06:32:17.764 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:17.764 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:32:17.764 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:17.764 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:17.764 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:32:17.764 INFO:teuthology.orchestra.run.smithi138.stdout:} 2024-09-14T06:32:17.766 INFO:tasks.ceph.ceph_manager.ceph:waiting for recovery to complete 2024-09-14T06:32:17.766 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-14T06:32:17.767 INFO:teuthology.orchestra.run.smithi094.stdout:osd.0: {} 2024-09-14T06:32:17.767 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-14T06:32:17.767 INFO:teuthology.orchestra.run.smithi094.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-14T06:32:17.768 INFO:teuthology.orchestra.run.smithi094.stdout:osd.1: {} 2024-09-14T06:32:17.768 INFO:teuthology.orchestra.run.smithi094.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-14T06:32:17.768 INFO:teuthology.orchestra.run.smithi094.stdout:osd.2: {} 2024-09-14T06:32:17.768 INFO:teuthology.orchestra.run.smithi094.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-14T06:32:17.769 INFO:tasks.ceph.osd.1.smithi094.stderr:2024-09-14T06:32:17.764+0000 7f328e47a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33641) UID: 0 2024-09-14T06:32:17.776 INFO:teuthology.orchestra.run.smithi094.stdout:osd.3: {} 2024-09-14T06:32:17.776 INFO:teuthology.orchestra.run.smithi094.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-14T06:32:17.793 INFO:teuthology.orchestra.run.smithi094.stdout:osd.4: {} 2024-09-14T06:32:17.793 INFO:teuthology.orchestra.run.smithi094.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-14T06:32:17.811 INFO:teuthology.orchestra.run.smithi094.stdout:osd.5: {} 2024-09-14T06:32:17.812 INFO:teuthology.orchestra.run.smithi094.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-14T06:32:17.829 INFO:teuthology.orchestra.run.smithi094.stdout:osd.6: {} 2024-09-14T06:32:17.829 INFO:teuthology.orchestra.run.smithi094.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-14T06:32:17.847 INFO:teuthology.orchestra.run.smithi094.stdout:osd.7: {} 2024-09-14T06:32:17.847 INFO:teuthology.orchestra.run.smithi094.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-14T06:32:17.900 INFO:tasks.ceph.osd.2.smithi094.stderr:2024-09-14T06:32:17.899+0000 7faf3269b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33649) UID: 0 2024-09-14T06:32:17.911 INFO:teuthology.orchestra.run.smithi138.stdout:ERROR: (22) Invalid argument 2024-09-14T06:32:17.911 INFO:teuthology.orchestra.run.smithi138.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:32:17.919 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-14T06:32:17.966 INFO:tasks.ceph.osd.3.smithi094.stderr:2024-09-14T06:32:17.965+0000 7feeeafe8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33655) UID: 0 2024-09-14T06:32:18.021 INFO:teuthology.orchestra.run.smithi138.stdout:ERROR: (22) Invalid argument 2024-09-14T06:32:18.021 INFO:teuthology.orchestra.run.smithi138.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:32:18.029 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-14T06:32:18.064 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-14T06:32:18.065 INFO:teuthology.orchestra.run.smithi094.stderr:dumped all 2024-09-14T06:32:18.066 INFO:tasks.ceph.osd.2.smithi094.stderr:2024-09-14T06:32:18.065+0000 7faf3269b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33649) UID: 0 2024-09-14T06:32:18.077 INFO:teuthology.orchestra.run.smithi094.stdout:{"pg_ready":true,"pg_map":{"version":314,"stamp":"2024-09-14T06:32:17.671689+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":2005522731,"num_objects":2155,"num_object_clones":0,"num_object_copies":4310,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":156,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2155,"num_whiteouts":0,"num_read":230908,"num_read_kb":2177045,"num_write":34663,"num_write_kb":1479442,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4133,"num_bytes_recovered":555504806,"num_keys_recovered":466,"num_objects_omap":44,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":24900,"ondisk_log_size":24900,"up":82,"acting":81,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":7,"num_per_pool_osds":7,"num_per_pool_omap_osds":7,"kb":660602880,"kb_used":568096,"kb_used_data":144028,"kb_used_omap":10,"kb_used_meta":423605,"kb_avail":660034784,"statfs":{"total":676457349120,"available":675875618816,"internally_reserved":0,"allocated":147484672,"data_stored":1278102089,"data_compressed":79322459,"data_compressed_allocated":106147840,"data_compressed_original":1250764178,"omap_allocated":11157,"internal_metadata":433771627},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-124802491,"num_objects":-263,"num_object_clones":0,"num_object_copies":-526,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":156,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-263,"num_whiteouts":0,"num_read":1221,"num_read_kb":2116,"num_write":480,"num_write_kb":279,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.013232"},"pg_stats":[{"pgid":"3.1f","version":"207'728","reported_seq":7785,"reported_epoch":207,"state":"active+clean","last_fresh":"2024-09-14T06:32:12.761779+0000","last_change":"2024-09-14T06:32:12.659020+0000","last_active":"2024-09-14T06:32:12.761779+0000","last_peered":"2024-09-14T06:32:12.761779+0000","last_clean":"2024-09-14T06:32:12.761779+0000","last_became_active":"2024-09-14T06:29:00.579883+0000","last_became_peered":"2024-09-14T06:29:00.579883+0000","last_unstale":"2024-09-14T06:32:12.761779+0000","last_undegraded":"2024-09-14T06:32:12.761779+0000","last_fullsized":"2024-09-14T06:32:12.761779+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":728,"log_dups_size":0,"ondisk_log_size":728,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T10:15:02.779710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00040664400000000001,"stat_sum":{"num_bytes":91062694,"num_objects":69,"num_object_clones":0,"num_object_copies":138,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":69,"num_whiteouts":0,"num_read":8088,"num_read_kb":86571,"num_write":1057,"num_write_kb":49526,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"57","length":"1"},{"start":"59","length":"1"},{"start":"5b","length":"2"}]},{"pgid":"3.1e","version":"210'709","reported_seq":9445,"reported_epoch":210,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-14T06:32:15.632212+0000","last_change":"2024-09-14T06:32:15.631533+0000","last_active":"2024-09-14T06:32:15.632212+0000","last_peered":"2024-09-14T06:32:15.632212+0000","last_clean":"2024-09-14T06:32:12.761016+0000","last_became_active":"2024-09-14T06:32:15.628941+0000","last_became_peered":"2024-09-14T06:32:15.628941+0000","last_unstale":"2024-09-14T06:32:15.632212+0000","last_undegraded":"2024-09-14T06:32:15.612555+0000","last_fullsized":"2024-09-14T06:32:15.610111+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":200,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":709,"log_dups_size":0,"ondisk_log_size":709,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":1,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00042255199999999998,"stat_sum":{"num_bytes":59789316,"num_objects":61,"num_object_clones":0,"num_object_copies":122,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":156,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":61,"num_whiteouts":0,"num_read":8272,"num_read_kb":67628,"num_write":951,"num_write_kb":40402,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":557,"num_bytes_recovered":70268016,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3],"avail_no_missing":["3","1"],"object_location_counts":[{"shards":"1,3","objects":61},{"shards":"3","objects":-1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"57","length":"1"},{"start":"59","length":"1"},{"start":"5b","length":"1"}]},{"pgid":"3.1d","version":"208'941","reported_seq":6948,"reported_epoch":208,"state":"active+clean","last_fresh":"2024-09-14T06:32:13.662308+0000","last_change":"2024-09-14T06:32:13.662308+0000","last_active":"2024-09-14T06:32:13.662308+0000","last_peered":"2024-09-14T06:32:13.662308+0000","last_clean":"2024-09-14T06:32:13.662308+0000","last_became_active":"2024-09-14T06:29:00.579345+0000","last_became_peered":"2024-09-14T06:29:00.579345+0000","last_unstale":"2024-09-14T06:32:13.662308+0000","last_undegraded":"2024-09-14T06:32:13.662308+0000","last_fullsized":"2024-09-14T06:32:13.662308+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":941,"log_dups_size":0,"ondisk_log_size":941,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T16:59:53.722458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00044556000000000001,"stat_sum":{"num_bytes":84988890,"num_objects":77,"num_object_clones":0,"num_object_copies":154,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":77,"num_whiteouts":0,"num_read":7600,"num_read_kb":64149,"num_write":1423,"num_write_kb":53271,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"55","length":"1"},{"start":"57","length":"1"},{"start":"59","length":"1"},{"start":"5b","length":"2"}]},{"pgid":"3.1c","version":"208'631","reported_seq":5169,"reported_epoch":208,"state":"active+clean","last_fresh":"2024-09-14T06:32:13.752475+0000","last_change":"2024-09-14T06:32:13.662895+0000","last_active":"2024-09-14T06:32:13.752475+0000","last_peered":"2024-09-14T06:32:13.752475+0000","last_clean":"2024-09-14T06:32:13.752475+0000","last_became_active":"2024-09-14T06:32:04.618831+0000","last_became_peered":"2024-09-14T06:32:04.618831+0000","last_unstale":"2024-09-14T06:32:13.752475+0000","last_undegraded":"2024-09-14T06:32:13.752475+0000","last_fullsized":"2024-09-14T06:32:13.752475+0000","mapping_epoch":199,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":200,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":631,"log_dups_size":0,"ondisk_log_size":631,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T09:13:28.581097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00046365099999999998,"stat_sum":{"num_bytes":41533486,"num_objects":67,"num_object_clones":0,"num_object_copies":134,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":67,"num_whiteouts":0,"num_read":4225,"num_read_kb":43151,"num_write":750,"num_write_kb":45533,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":319,"num_bytes_recovered":32800897,"num_keys_recovered":223,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"55","length":"1"},{"start":"57","length":"1"},{"start":"59","length":"1"},{"start":"5b","length":"2"}]},{"pgid":"3.1b","version":"207'842","reported_seq":10952,"reported_epoch":207,"state":"active+clean","last_fresh":"2024-09-14T06:32:12.756464+0000","last_change":"2024-09-14T06:32:12.659531+0000","last_active":"2024-09-14T06:32:12.756464+0000","last_peered":"2024-09-14T06:32:12.756464+0000","last_clean":"2024-09-14T06:32:12.756464+0000","last_became_active":"2024-09-14T06:29:00.586183+0000","last_became_peered":"2024-09-14T06:29:00.586183+0000","last_unstale":"2024-09-14T06:32:12.756464+0000","last_undegraded":"2024-09-14T06:32:12.756464+0000","last_fullsized":"2024-09-14T06:32:12.756464+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":842,"log_dups_size":0,"ondisk_log_size":842,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T18:21:13.120536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00072037100000000003,"stat_sum":{"num_bytes":77932251,"num_objects":83,"num_object_clones":0,"num_object_copies":166,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":83,"num_whiteouts":0,"num_read":11697,"num_read_kb":107915,"num_write":1351,"num_write_kb":55091,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"57","length":"1"},{"start":"59","length":"1"},{"start":"5b","length":"2"}]},{"pgid":"3.1a","version":"208'1515","reported_seq":9700,"reported_epoch":208,"state":"active+clean","last_fresh":"2024-09-14T06:32:13.759334+0000","last_change":"2024-09-14T06:32:13.660594+0000","last_active":"2024-09-14T06:32:13.759334+0000","last_peered":"2024-09-14T06:32:13.759334+0000","last_clean":"2024-09-14T06:32:13.759334+0000","last_became_active":"2024-09-14T06:29:00.576206+0000","last_became_peered":"2024-09-14T06:29:00.576206+0000","last_unstale":"2024-09-14T06:32:13.759334+0000","last_undegraded":"2024-09-14T06:32:13.759334+0000","last_fullsized":"2024-09-14T06:32:13.759334+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":1515,"log_dups_size":0,"ondisk_log_size":1515,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T16:41:21.694673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050120000000000004,"stat_sum":{"num_bytes":67140503,"num_objects":74,"num_object_clones":0,"num_object_copies":148,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":74,"num_whiteouts":0,"num_read":12616,"num_read_kb":82257,"num_write":2294,"num_write_kb":55655,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"55","length":"1"},{"start":"57","length":"1"},{"start":"59","length":"1"},{"start":"5b","length":"2"}]},{"pgid":"3.19","version":"208'490","reported_seq":6058,"reported_epoch":208,"state":"active+clean","last_fresh":"2024-09-14T06:32:13.741675+0000","last_change":"2024-09-14T06:32:13.659799+0000","last_active":"2024-09-14T06:32:13.741675+0000","last_peered":"2024-09-14T06:32:13.741675+0000","last_clean":"2024-09-14T06:32:13.741675+0000","last_became_active":"2024-09-14T06:31:53.744800+0000","last_became_peered":"2024-09-14T06:31:53.744800+0000","last_unstale":"2024-09-14T06:32:13.741675+0000","last_undegraded":"2024-09-14T06:32:13.741675+0000","last_fullsized":"2024-09-14T06:32:13.741675+0000","mapping_epoch":188,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":189,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":490,"log_dups_size":0,"ondisk_log_size":490,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T09:48:33.686457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049229199999999997,"stat_sum":{"num_bytes":56946579,"num_objects":58,"num_object_clones":0,"num_object_copies":116,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":58,"num_whiteouts":0,"num_read":5078,"num_read_kb":67752,"num_write":693,"num_write_kb":43868,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":282,"num_bytes_recovered":41927489,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"55","length":"1"},{"start":"57","length":"1"},{"start":"59","length":"1"},{"start":"5b","length":"2"}]},{"pgid":"3.18","version":"208'463","reported_seq":5839,"reported_epoch":210,"state":"active+clean","last_fresh":"2024-09-14T06:32:15.610510+0000","last_change":"2024-09-14T06:32:13.662308+0000","last_active":"2024-09-14T06:32:15.610510+0000","last_peered":"2024-09-14T06:32:15.610510+0000","last_clean":"2024-09-14T06:32:15.610510+0000","last_became_active":"2024-09-14T06:31:53.745159+0000","last_became_peered":"2024-09-14T06:31:53.745159+0000","last_unstale":"2024-09-14T06:32:15.610510+0000","last_undegraded":"2024-09-14T06:32:15.610510+0000","last_fullsized":"2024-09-14T06:32:15.610510+0000","mapping_epoch":188,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":189,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":463,"log_dups_size":0,"ondisk_log_size":463,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T13:28:24.561144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00063330400000000001,"stat_sum":{"num_bytes":68306983,"num_objects":70,"num_object_clones":0,"num_object_copies":140,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":70,"num_whiteouts":0,"num_read":4455,"num_read_kb":78892,"num_write":589,"num_write_kb":51576,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":310,"num_bytes_recovered":56082976,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"55","length":"1"},{"start":"57","length":"1"},{"start":"59","length":"1"},{"start":"5b","length":"1"}]},{"pgid":"3.17","version":"207'856","reported_seq":8950,"reported_epoch":207,"state":"active+clean","last_fresh":"2024-09-14T06:32:12.659399+0000","last_change":"2024-09-14T06:32:12.659399+0000","last_active":"2024-09-14T06:32:12.659399+0000","last_peered":"2024-09-14T06:32:12.659399+0000","last_clean":"2024-09-14T06:32:12.659399+0000","last_became_active":"2024-09-14T06:29:00.579714+0000","last_became_peered":"2024-09-14T06:29:00.579714+0000","last_unstale":"2024-09-14T06:32:12.659399+0000","last_undegraded":"2024-09-14T06:32:12.659399+0000","last_fullsized":"2024-09-14T06:32:12.659399+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":856,"log_dups_size":0,"ondisk_log_size":856,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T10:36:04.398636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069585199999999995,"stat_sum":{"num_bytes":78451450,"num_objects":83,"num_object_clones":0,"num_object_copies":166,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":83,"num_whiteouts":0,"num_read":10072,"num_read_kb":68437,"num_write":992,"num_write_kb":48449,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"57","length":"1"},{"start":"59","length":"1"},{"start":"5b","length":"2"}]},{"pgid":"3.16","version":"203'1035","reported_seq":3574,"reported_epoch":208,"state":"active+clean","last_fresh":"2024-09-14T06:32:13.661922+0000","last_change":"2024-09-14T06:32:13.662104+0000","last_active":"2024-09-14T06:32:13.661922+0000","last_peered":"2024-09-14T06:32:13.661922+0000","last_clean":"2024-09-14T06:32:13.661922+0000","last_became_active":"2024-09-14T06:29:00.580070+0000","last_became_peered":"2024-09-14T06:29:00.580070+0000","last_unstale":"2024-09-14T06:32:13.661922+0000","last_undegraded":"2024-09-14T06:32:13.661922+0000","last_fullsized":"2024-09-14T06:32:13.661922+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":1035,"log_dups_size":0,"ondisk_log_size":1035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T14:17:44.932374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00033417000000000001,"stat_sum":{"num_bytes":42065946,"num_objects":68,"num_object_clones":0,"num_object_copies":136,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":68,"num_whiteouts":0,"num_read":5281,"num_read_kb":38236,"num_write":1630,"num_write_kb":37231,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"55","length":"1"},{"start":"57","length":"1"},{"start":"59","length":"1"},{"start":"5b","length":"2"}]},{"pgid":"3.15","version":"208'547","reported_seq":6205,"reported_epoch":208,"state":"active+clean","last_fresh":"2024-09-14T06:32:13.732385+0000","last_change":"2024-09-14T06:32:13.664759+0000","last_active":"2024-09-14T06:32:13.732385+0000","last_peered":"2024-09-14T06:32:13.732385+0000","last_clean":"2024-09-14T06:32:13.732385+0000","last_became_active":"2024-09-14T06:31:53.746091+0000","last_became_peered":"2024-09-14T06:31:53.746091+0000","last_unstale":"2024-09-14T06:32:13.732385+0000","last_undegraded":"2024-09-14T06:32:13.732385+0000","last_fullsized":"2024-09-14T06:32:13.732385+0000","mapping_epoch":188,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":189,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":547,"log_dups_size":0,"ondisk_log_size":547,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T09:15:05.610444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00144721,"stat_sum":{"num_bytes":79873129,"num_objects":64,"num_object_clones":0,"num_object_copies":128,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":4896,"num_read_kb":74723,"num_write":602,"num_write_kb":51185,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":271,"num_bytes_recovered":49981473,"num_keys_recovered":33,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"55","length":"1"},{"start":"57","length":"1"},{"start":"59","length":"1"},{"start":"5b","length":"2"}]},{"pgid":"3.14","version":"208'1024","reported_seq":5706,"reported_epoch":208,"state":"active+clean","last_fresh":"2024-09-14T06:32:13.660985+0000","last_change":"2024-09-14T06:32:13.660985+0000","last_active":"2024-09-14T06:32:13.660985+0000","last_peered":"2024-09-14T06:32:13.660985+0000","last_clean":"2024-09-14T06:32:13.660985+0000","last_became_active":"2024-09-14T06:29:00.582595+0000","last_became_peered":"2024-09-14T06:29:00.582595+0000","last_unstale":"2024-09-14T06:32:13.660985+0000","last_undegraded":"2024-09-14T06:32:13.660985+0000","last_fullsized":"2024-09-14T06:32:13.660985+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":1024,"log_dups_size":0,"ondisk_log_size":1024,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T07:52:57.903676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065583600000000003,"stat_sum":{"num_bytes":51834579,"num_objects":55,"num_object_clones":0,"num_object_copies":110,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":55,"num_whiteouts":0,"num_read":8562,"num_read_kb":46606,"num_write":1473,"num_write_kb":29812,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"55","length":"1"},{"start":"57","length":"1"},{"start":"59","length":"1"},{"start":"5b","length":"2"}]},{"pgid":"2.7","version":"0'0","reported_seq":34,"reported_epoch":208,"state":"active+clean","last_fresh":"2024-09-14T06:32:13.663307+0000","last_change":"2024-09-14T06:32:01.612470+0000","last_active":"2024-09-14T06:32:13.663307+0000","last_peered":"2024-09-14T06:32:13.663307+0000","last_clean":"2024-09-14T06:32:13.663307+0000","last_became_active":"2024-09-14T06:32:01.612121+0000","last_became_peered":"2024-09-14T06:32:01.612121+0000","last_unstale":"2024-09-14T06:32:13.663307+0000","last_undegraded":"2024-09-14T06:32:13.663307+0000","last_fullsized":"2024-09-14T06:32:13.663307+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:28:50.205350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:28:50.205350+0000","last_clean_scrub_stamp":"2024-09-14T06:28:50.205350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T13:40:52.537942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.6","version":"207'681","reported_seq":7916,"reported_epoch":207,"state":"active+clean","last_fresh":"2024-09-14T06:32:12.758188+0000","last_change":"2024-09-14T06:32:12.659011+0000","last_active":"2024-09-14T06:32:12.758188+0000","last_peered":"2024-09-14T06:32:12.758188+0000","last_clean":"2024-09-14T06:32:12.758188+0000","last_became_active":"2024-09-14T06:29:00.578283+0000","last_became_peered":"2024-09-14T06:29:00.578283+0000","last_unstale":"2024-09-14T06:32:12.758188+0000","last_undegraded":"2024-09-14T06:32:12.758188+0000","last_fullsized":"2024-09-14T06:32:12.758188+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":681,"log_dups_size":0,"ondisk_log_size":681,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T16:33:06.750083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00068571699999999997,"stat_sum":{"num_bytes":66750303,"num_objects":65,"num_object_clones":0,"num_object_copies":130,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":65,"num_whiteouts":0,"num_read":7993,"num_read_kb":78244,"num_write":771,"num_write_kb":48270,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"57","length":"1"},{"start":"59","length":"1"},{"start":"5b","length":"2"}]},{"pgid":"2.6","version":"0'0","reported_seq":399,"reported_epoch":208,"state":"active+clean","last_fresh":"2024-09-14T06:32:13.659575+0000","last_change":"2024-09-14T06:32:01.608456+0000","last_active":"2024-09-14T06:32:13.659575+0000","last_peered":"2024-09-14T06:32:13.659575+0000","last_clean":"2024-09-14T06:32:13.659575+0000","last_became_active":"2024-09-14T06:32:01.608178+0000","last_became_peered":"2024-09-14T06:32:01.608178+0000","last_unstale":"2024-09-14T06:32:13.659575+0000","last_undegraded":"2024-09-14T06:32:13.659575+0000","last_fullsized":"2024-09-14T06:32:13.659575+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:28:50.205350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:28:50.205350+0000","last_clean_scrub_stamp":"2024-09-14T06:28:50.205350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T10:16:40.480031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065018200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"207'753","reported_seq":4127,"reported_epoch":210,"state":"active+clean","last_fresh":"2024-09-14T06:32:15.610254+0000","last_change":"2024-09-14T06:32:13.662724+0000","last_active":"2024-09-14T06:32:15.610254+0000","last_peered":"2024-09-14T06:32:15.610254+0000","last_clean":"2024-09-14T06:32:15.610254+0000","last_became_active":"2024-09-14T06:31:54.747923+0000","last_became_peered":"2024-09-14T06:31:54.747923+0000","last_unstale":"2024-09-14T06:32:15.610254+0000","last_undegraded":"2024-09-14T06:32:15.610254+0000","last_fullsized":"2024-09-14T06:32:15.610254+0000","mapping_epoch":189,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":190,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":753,"log_dups_size":0,"ondisk_log_size":753,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T13:55:37.796703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00044561900000000002,"stat_sum":{"num_bytes":38285856,"num_objects":51,"num_object_clones":0,"num_object_copies":102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":51,"num_whiteouts":0,"num_read":4380,"num_read_kb":40951,"num_write":1098,"num_write_kb":36111,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":183,"num_bytes_recovered":13912513,"num_keys_recovered":22,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"55","length":"1"},{"start":"57","length":"1"},{"start":"59","length":"1"},{"start":"5b","length":"1"}]},{"pgid":"2.5","version":"0'0","reported_seq":393,"reported_epoch":208,"state":"active+clean","last_fresh":"2024-09-14T06:32:13.663731+0000","last_change":"2024-09-14T06:28:53.640035+0000","last_active":"2024-09-14T06:32:13.663731+0000","last_peered":"2024-09-14T06:32:13.663731+0000","last_clean":"2024-09-14T06:32:13.663731+0000","last_became_active":"2024-09-14T06:28:51.308058+0000","last_became_peered":"2024-09-14T06:28:51.308058+0000","last_unstale":"2024-09-14T06:32:13.663731+0000","last_undegraded":"2024-09-14T06:32:13.663731+0000","last_fullsized":"2024-09-14T06:32:13.663731+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:28:50.205350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:28:50.205350+0000","last_clean_scrub_stamp":"2024-09-14T06:28:50.205350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T13:35:48.544057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00037824000000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"207'613","reported_seq":3312,"reported_epoch":207,"state":"peering","last_fresh":"2024-09-14T06:32:12.756338+0000","last_change":"2024-09-14T06:32:13.659971+0000","last_active":"2024-09-14T06:32:13.659971+0000","last_peered":"2024-09-14T06:32:12.756338+0000","last_clean":"2024-09-14T06:32:12.756338+0000","last_became_active":"2024-09-14T06:31:41.604731+0000","last_became_peered":"2024-09-14T06:31:41.604731+0000","last_unstale":"2024-09-14T06:32:12.756338+0000","last_undegraded":"2024-09-14T06:32:12.756338+0000","last_fullsized":"2024-09-14T06:32:12.756338+0000","mapping_epoch":208,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":613,"log_dups_size":0,"ondisk_log_size":613,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00058330999999999997,"stat_sum":{"num_bytes":37139964,"num_objects":64,"num_object_clones":0,"num_object_copies":128,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":2589,"num_read_kb":37386,"num_write":807,"num_write_kb":40850,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":153,"num_bytes_recovered":15692987,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2,5],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"57","length":"1"},{"start":"59","length":"1"},{"start":"5b","length":"2"}]},{"pgid":"2.4","version":"0'0","reported_seq":393,"reported_epoch":208,"state":"active+clean","last_fresh":"2024-09-14T06:32:13.659521+0000","last_change":"2024-09-14T06:28:53.226072+0000","last_active":"2024-09-14T06:32:13.659521+0000","last_peered":"2024-09-14T06:32:13.659521+0000","last_clean":"2024-09-14T06:32:13.659521+0000","last_became_active":"2024-09-14T06:28:51.308072+0000","last_became_peered":"2024-09-14T06:28:51.308072+0000","last_unstale":"2024-09-14T06:32:13.659521+0000","last_undegraded":"2024-09-14T06:32:13.659521+0000","last_fullsized":"2024-09-14T06:32:13.659521+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:28:50.205350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:28:50.205350+0000","last_clean_scrub_stamp":"2024-09-14T06:28:50.205350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T13:15:09.052328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066197399999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"208'827","reported_seq":5247,"reported_epoch":208,"state":"active+clean","last_fresh":"2024-09-14T06:32:13.749810+0000","last_change":"2024-09-14T06:32:13.662584+0000","last_active":"2024-09-14T06:32:13.749810+0000","last_peered":"2024-09-14T06:32:13.749810+0000","last_clean":"2024-09-14T06:32:13.749810+0000","last_became_active":"2024-09-14T06:31:54.744373+0000","last_became_peered":"2024-09-14T06:31:54.744373+0000","last_unstale":"2024-09-14T06:32:13.749810+0000","last_undegraded":"2024-09-14T06:32:13.749810+0000","last_fullsized":"2024-09-14T06:32:13.749810+0000","mapping_epoch":189,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":190,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":827,"log_dups_size":0,"ondisk_log_size":827,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T10:48:07.580270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041746399999999999,"stat_sum":{"num_bytes":43262791,"num_objects":67,"num_object_clones":0,"num_object_copies":134,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":67,"num_whiteouts":0,"num_read":4507,"num_read_kb":42629,"num_write":1188,"num_write_kb":49494,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":330,"num_bytes_recovered":28798893,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"55","length":"1"},{"start":"57","length":"1"},{"start":"59","length":"1"},{"start":"5b","length":"2"}]},{"pgid":"3.3","version":"208'468","reported_seq":10079,"reported_epoch":208,"state":"active+clean","last_fresh":"2024-09-14T06:32:13.745191+0000","last_change":"2024-09-14T06:32:13.661140+0000","last_active":"2024-09-14T06:32:13.745191+0000","last_peered":"2024-09-14T06:32:13.745191+0000","last_clean":"2024-09-14T06:32:13.745191+0000","last_became_active":"2024-09-14T06:29:00.584678+0000","last_became_peered":"2024-09-14T06:29:00.584678+0000","last_unstale":"2024-09-14T06:32:13.745191+0000","last_undegraded":"2024-09-14T06:32:13.745191+0000","last_fullsized":"2024-09-14T06:32:13.745191+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":468,"log_dups_size":0,"ondisk_log_size":468,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T13:20:16.770689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00082891200000000001,"stat_sum":{"num_bytes":62895515,"num_objects":56,"num_object_clones":0,"num_object_copies":112,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":56,"num_whiteouts":0,"num_read":9151,"num_read_kb":88923,"num_write":541,"num_write_kb":51562,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"55","length":"1"},{"start":"57","length":"1"},{"start":"59","length":"1"},{"start":"5b","length":"2"}]},{"pgid":"2.2","version":"22'2","reported_seq":395,"reported_epoch":208,"state":"active+clean","last_fresh":"2024-09-14T06:32:13.661683+0000","last_change":"2024-09-14T06:28:53.231428+0000","last_active":"2024-09-14T06:32:13.661683+0000","last_peered":"2024-09-14T06:32:13.661683+0000","last_clean":"2024-09-14T06:32:13.661683+0000","last_became_active":"2024-09-14T06:28:51.222937+0000","last_became_peered":"2024-09-14T06:28:51.222937+0000","last_unstale":"2024-09-14T06:32:13.661683+0000","last_undegraded":"2024-09-14T06:32:13.661683+0000","last_fullsized":"2024-09-14T06:32:13.661683+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:28:50.205350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:28:50.205350+0000","last_clean_scrub_stamp":"2024-09-14T06:28:50.205350+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T14:27:37.233976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010072970000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.0","version":"208'994","reported_seq":6568,"reported_epoch":208,"state":"active+clean","last_fresh":"2024-09-14T06:32:13.659853+0000","last_change":"2024-09-14T06:32:13.659853+0000","last_active":"2024-09-14T06:32:13.659853+0000","last_peered":"2024-09-14T06:32:13.659853+0000","last_clean":"2024-09-14T06:32:13.659853+0000","last_became_active":"2024-09-14T06:29:00.574873+0000","last_became_peered":"2024-09-14T06:29:00.574873+0000","last_unstale":"2024-09-14T06:32:13.659853+0000","last_undegraded":"2024-09-14T06:32:13.659853+0000","last_fullsized":"2024-09-14T06:32:13.659853+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":994,"log_dups_size":0,"ondisk_log_size":994,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T13:32:24.881003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00058771500000000005,"stat_sum":{"num_bytes":74732015,"num_objects":77,"num_object_clones":0,"num_object_copies":154,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":77,"num_whiteouts":0,"num_read":8014,"num_read_kb":85277,"num_write":1533,"num_write_kb":58684,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"55","length":"1"},{"start":"57","length":"1"},{"start":"59","length":"1"},{"start":"5b","length":"2"}]},{"pgid":"2.1","version":"0'0","reported_seq":49,"reported_epoch":205,"state":"active+clean","last_fresh":"2024-09-14T06:32:10.679671+0000","last_change":"2024-09-14T06:31:50.607764+0000","last_active":"2024-09-14T06:32:10.679671+0000","last_peered":"2024-09-14T06:32:10.679671+0000","last_clean":"2024-09-14T06:32:10.679671+0000","last_became_active":"2024-09-14T06:31:50.607447+0000","last_became_peered":"2024-09-14T06:31:50.607447+0000","last_unstale":"2024-09-14T06:32:10.679671+0000","last_undegraded":"2024-09-14T06:32:10.679671+0000","last_fullsized":"2024-09-14T06:32:10.679671+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":186,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:28:50.205350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:28:50.205350+0000","last_clean_scrub_stamp":"2024-09-14T06:28:50.205350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T09:11:53.601856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053403100000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1","version":"203'695","reported_seq":11304,"reported_epoch":207,"state":"active+clean","last_fresh":"2024-09-14T06:32:12.657918+0000","last_change":"2024-09-14T06:32:12.658168+0000","last_active":"2024-09-14T06:32:12.657918+0000","last_peered":"2024-09-14T06:32:12.657918+0000","last_clean":"2024-09-14T06:32:12.657918+0000","last_became_active":"2024-09-14T06:30:57.768238+0000","last_became_peered":"2024-09-14T06:30:57.768238+0000","last_unstale":"2024-09-14T06:32:12.657918+0000","last_undegraded":"2024-09-14T06:32:12.657918+0000","last_fullsized":"2024-09-14T06:32:12.657918+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":695,"log_dups_size":0,"ondisk_log_size":695,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T07:49:50.576428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00090325799999999999,"stat_sum":{"num_bytes":93666435,"num_objects":66,"num_object_clones":0,"num_object_copies":132,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":66,"num_whiteouts":0,"num_read":11016,"num_read_kb":112206,"num_write":870,"num_write_kb":47387,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":183,"num_bytes_recovered":30431994,"num_keys_recovered":22,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"57","length":"1"},{"start":"59","length":"1"},{"start":"5b","length":"2"}]},{"pgid":"2.0","version":"0'0","reported_seq":393,"reported_epoch":208,"state":"active+clean","last_fresh":"2024-09-14T06:32:13.662840+0000","last_change":"2024-09-14T06:28:53.640096+0000","last_active":"2024-09-14T06:32:13.662840+0000","last_peered":"2024-09-14T06:32:13.662840+0000","last_clean":"2024-09-14T06:32:13.662840+0000","last_became_active":"2024-09-14T06:28:51.223069+0000","last_became_peered":"2024-09-14T06:28:51.223069+0000","last_unstale":"2024-09-14T06:32:13.662840+0000","last_undegraded":"2024-09-14T06:32:13.662840+0000","last_fullsized":"2024-09-14T06:32:13.662840+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:28:50.205350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:28:50.205350+0000","last_clean_scrub_stamp":"2024-09-14T06:28:50.205350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T15:13:00.471509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00043714600000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"210'1558","reported_seq":7376,"reported_epoch":210,"state":"active+clean","last_fresh":"2024-09-14T06:32:15.640212+0000","last_change":"2024-09-14T06:32:13.662240+0000","last_active":"2024-09-14T06:32:15.640212+0000","last_peered":"2024-09-14T06:32:15.640212+0000","last_clean":"2024-09-14T06:32:15.640212+0000","last_became_active":"2024-09-14T06:31:54.744160+0000","last_became_peered":"2024-09-14T06:31:54.744160+0000","last_unstale":"2024-09-14T06:32:15.640212+0000","last_undegraded":"2024-09-14T06:32:15.640212+0000","last_fullsized":"2024-09-14T06:32:15.640212+0000","mapping_epoch":189,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":190,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":1558,"log_dups_size":0,"ondisk_log_size":1558,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T12:41:51.808698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00055251900000000001,"stat_sum":{"num_bytes":66459400,"num_objects":80,"num_object_clones":0,"num_object_copies":160,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":80,"num_whiteouts":0,"num_read":10653,"num_read_kb":70538,"num_write":2490,"num_write_kb":53554,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":327,"num_bytes_recovered":50047063,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"55","length":"1"},{"start":"57","length":"1"},{"start":"59","length":"1"},{"start":"5b","length":"1"}]},{"pgid":"2.3","version":"20'1","reported_seq":394,"reported_epoch":208,"state":"active+clean","last_fresh":"2024-09-14T06:32:13.661658+0000","last_change":"2024-09-14T06:28:53.230933+0000","last_active":"2024-09-14T06:32:13.661658+0000","last_peered":"2024-09-14T06:32:13.661658+0000","last_clean":"2024-09-14T06:32:13.661658+0000","last_became_active":"2024-09-14T06:28:51.222874+0000","last_became_peered":"2024-09-14T06:28:51.222874+0000","last_unstale":"2024-09-14T06:32:13.661658+0000","last_undegraded":"2024-09-14T06:32:13.661658+0000","last_fullsized":"2024-09-14T06:32:13.661658+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:28:50.205350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:28:50.205350+0000","last_clean_scrub_stamp":"2024-09-14T06:28:50.205350+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T12:04:25.190171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050808899999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'72","reported_seq":484,"reported_epoch":208,"state":"active+clean","last_fresh":"2024-09-14T06:32:13.662987+0000","last_change":"2024-09-14T06:28:44.839885+0000","last_active":"2024-09-14T06:32:13.662987+0000","last_peered":"2024-09-14T06:32:13.662987+0000","last_clean":"2024-09-14T06:32:13.662987+0000","last_became_active":"2024-09-14T06:28:44.839573+0000","last_became_peered":"2024-09-14T06:28:44.839573+0000","last_unstale":"2024-09-14T06:32:13.662987+0000","last_undegraded":"2024-09-14T06:32:13.662987+0000","last_fullsized":"2024-09-14T06:32:13.662987+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:28:43.828494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:28:43.828494+0000","last_clean_scrub_stamp":"2024-09-14T06:28:43.828494+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T14:41:25.787293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.8","version":"208'586","reported_seq":6123,"reported_epoch":210,"state":"active+clean","last_fresh":"2024-09-14T06:32:15.610718+0000","last_change":"2024-09-14T06:32:13.662649+0000","last_active":"2024-09-14T06:32:15.610718+0000","last_peered":"2024-09-14T06:32:15.610718+0000","last_clean":"2024-09-14T06:32:15.610718+0000","last_became_active":"2024-09-14T06:31:55.754694+0000","last_became_peered":"2024-09-14T06:31:55.754694+0000","last_unstale":"2024-09-14T06:32:15.610718+0000","last_undegraded":"2024-09-14T06:32:15.610718+0000","last_fullsized":"2024-09-14T06:32:15.610718+0000","mapping_epoch":190,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":191,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":586,"log_dups_size":0,"ondisk_log_size":586,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T12:30:44.816178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097722299999999994,"stat_sum":{"num_bytes":78517473,"num_objects":71,"num_object_clones":0,"num_object_copies":142,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":71,"num_whiteouts":0,"num_read":5608,"num_read_kb":90696,"num_write":834,"num_write_kb":57685,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":319,"num_bytes_recovered":60352269,"num_keys_recovered":22,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"55","length":"1"},{"start":"57","length":"1"},{"start":"59","length":"1"},{"start":"5b","length":"1"}]},{"pgid":"3.9","version":"208'516","reported_seq":4063,"reported_epoch":208,"state":"active+clean","last_fresh":"2024-09-14T06:32:13.660676+0000","last_change":"2024-09-14T06:32:13.660676+0000","last_active":"2024-09-14T06:32:13.660676+0000","last_peered":"2024-09-14T06:32:13.660676+0000","last_clean":"2024-09-14T06:32:13.660676+0000","last_became_active":"2024-09-14T06:29:00.573179+0000","last_became_peered":"2024-09-14T06:29:00.573179+0000","last_unstale":"2024-09-14T06:32:13.660676+0000","last_undegraded":"2024-09-14T06:32:13.660676+0000","last_fullsized":"2024-09-14T06:32:13.660676+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":516,"log_dups_size":0,"ondisk_log_size":516,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T18:24:55.803054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054865399999999998,"stat_sum":{"num_bytes":42231296,"num_objects":66,"num_object_clones":0,"num_object_copies":132,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":66,"num_whiteouts":0,"num_read":3634,"num_read_kb":64262,"num_write":699,"num_write_kb":44923,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"55","length":"1"},{"start":"57","length":"1"},{"start":"59","length":"1"},{"start":"5b","length":"2"}]},{"pgid":"3.a","version":"208'738","reported_seq":4416,"reported_epoch":208,"state":"active+clean","last_fresh":"2024-09-14T06:32:13.757393+0000","last_change":"2024-09-14T06:32:13.661025+0000","last_active":"2024-09-14T06:32:13.757393+0000","last_peered":"2024-09-14T06:32:13.757393+0000","last_clean":"2024-09-14T06:32:13.757393+0000","last_became_active":"2024-09-14T06:32:04.617283+0000","last_became_peered":"2024-09-14T06:32:04.617283+0000","last_unstale":"2024-09-14T06:32:13.757393+0000","last_undegraded":"2024-09-14T06:32:13.757393+0000","last_fullsized":"2024-09-14T06:32:13.757393+0000","mapping_epoch":199,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":200,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":738,"log_dups_size":0,"ondisk_log_size":738,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T15:40:09.086871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070773799999999997,"stat_sum":{"num_bytes":46401995,"num_objects":67,"num_object_clones":0,"num_object_copies":134,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":67,"num_whiteouts":0,"num_read":4807,"num_read_kb":57874,"num_write":1106,"num_write_kb":47517,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":161,"num_bytes_recovered":21723371,"num_keys_recovered":33,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"55","length":"1"},{"start":"57","length":"1"},{"start":"59","length":"1"},{"start":"5b","length":"2"}]},{"pgid":"3.b","version":"210'1477","reported_seq":8243,"reported_epoch":210,"state":"active+clean","last_fresh":"2024-09-14T06:32:15.683852+0000","last_change":"2024-09-14T06:32:13.662503+0000","last_active":"2024-09-14T06:32:15.683852+0000","last_peered":"2024-09-14T06:32:15.683852+0000","last_clean":"2024-09-14T06:32:15.683852+0000","last_became_active":"2024-09-14T06:31:55.756219+0000","last_became_peered":"2024-09-14T06:31:55.756219+0000","last_unstale":"2024-09-14T06:32:15.683852+0000","last_undegraded":"2024-09-14T06:32:15.683852+0000","last_fullsized":"2024-09-14T06:32:15.683852+0000","mapping_epoch":190,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":191,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":1477,"log_dups_size":0,"ondisk_log_size":1477,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T08:46:05.977011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000634164,"stat_sum":{"num_bytes":58790990,"num_objects":73,"num_object_clones":0,"num_object_copies":146,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":73,"num_whiteouts":0,"num_read":12454,"num_read_kb":63624,"num_write":2349,"num_write_kb":43651,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":291,"num_bytes_recovered":37370632,"num_keys_recovered":66,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"55","length":"1"},{"start":"57","length":"1"},{"start":"59","length":"1"},{"start":"5b","length":"1"}]},{"pgid":"3.c","version":"208'547","reported_seq":5890,"reported_epoch":208,"state":"active+clean","last_fresh":"2024-09-14T06:32:13.736039+0000","last_change":"2024-09-14T06:32:13.662856+0000","last_active":"2024-09-14T06:32:13.736039+0000","last_peered":"2024-09-14T06:32:13.736039+0000","last_clean":"2024-09-14T06:32:13.736039+0000","last_became_active":"2024-09-14T06:31:53.747355+0000","last_became_peered":"2024-09-14T06:31:53.747355+0000","last_unstale":"2024-09-14T06:32:13.736039+0000","last_undegraded":"2024-09-14T06:32:13.736039+0000","last_fullsized":"2024-09-14T06:32:13.736039+0000","mapping_epoch":188,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":189,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":547,"log_dups_size":0,"ondisk_log_size":547,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T16:05:49.827036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00037372000000000002,"stat_sum":{"num_bytes":51667677,"num_objects":60,"num_object_clones":0,"num_object_copies":120,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":4805,"num_read_kb":50398,"num_write":722,"num_write_kb":40822,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":293,"num_bytes_recovered":36712421,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"55","length":"1"},{"start":"57","length":"1"},{"start":"59","length":"1"},{"start":"5b","length":"2"}]},{"pgid":"3.d","version":"208'482","reported_seq":8794,"reported_epoch":208,"state":"active+clean","last_fresh":"2024-09-14T06:32:13.745059+0000","last_change":"2024-09-14T06:32:13.664738+0000","last_active":"2024-09-14T06:32:13.745059+0000","last_peered":"2024-09-14T06:32:13.745059+0000","last_clean":"2024-09-14T06:32:13.745059+0000","last_became_active":"2024-09-14T06:29:00.580362+0000","last_became_peered":"2024-09-14T06:29:00.580362+0000","last_unstale":"2024-09-14T06:32:13.745059+0000","last_undegraded":"2024-09-14T06:32:13.745059+0000","last_fullsized":"2024-09-14T06:32:13.745059+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":482,"log_dups_size":0,"ondisk_log_size":482,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T10:55:42.483074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010157390000000001,"stat_sum":{"num_bytes":95841531,"num_objects":72,"num_object_clones":0,"num_object_copies":144,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":72,"num_whiteouts":0,"num_read":8023,"num_read_kb":93542,"num_write":644,"num_write_kb":41026,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"55","length":"1"},{"start":"57","length":"1"},{"start":"59","length":"1"},{"start":"5b","length":"2"}]},{"pgid":"3.e","version":"207'520","reported_seq":4739,"reported_epoch":208,"state":"active+clean","last_fresh":"2024-09-14T06:32:13.663598+0000","last_change":"2024-09-14T06:32:13.663783+0000","last_active":"2024-09-14T06:32:13.663598+0000","last_peered":"2024-09-14T06:32:13.663598+0000","last_clean":"2024-09-14T06:32:13.663598+0000","last_became_active":"2024-09-14T06:29:00.586374+0000","last_became_peered":"2024-09-14T06:29:00.586374+0000","last_unstale":"2024-09-14T06:32:13.663598+0000","last_undegraded":"2024-09-14T06:32:13.663598+0000","last_fullsized":"2024-09-14T06:32:13.663598+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":520,"log_dups_size":0,"ondisk_log_size":520,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T16:01:48.654194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053448299999999999,"stat_sum":{"num_bytes":48343618,"num_objects":63,"num_object_clones":0,"num_object_copies":126,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":63,"num_whiteouts":0,"num_read":3977,"num_read_kb":61134,"num_write":690,"num_write_kb":43787,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"55","length":"1"},{"start":"57","length":"1"},{"start":"59","length":"1"},{"start":"5b","length":"2"}]},{"pgid":"3.f","version":"208'619","reported_seq":5340,"reported_epoch":208,"state":"active+clean","last_fresh":"2024-09-14T06:32:13.755426+0000","last_change":"2024-09-14T06:32:13.663765+0000","last_active":"2024-09-14T06:32:13.755426+0000","last_peered":"2024-09-14T06:32:13.755426+0000","last_clean":"2024-09-14T06:32:13.755426+0000","last_became_active":"2024-09-14T06:29:00.586491+0000","last_became_peered":"2024-09-14T06:29:00.586491+0000","last_unstale":"2024-09-14T06:32:13.755426+0000","last_undegraded":"2024-09-14T06:32:13.755426+0000","last_fullsized":"2024-09-14T06:32:13.755426+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":619,"log_dups_size":0,"ondisk_log_size":619,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T12:18:46.267108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057572199999999997,"stat_sum":{"num_bytes":29756210,"num_objects":49,"num_object_clones":0,"num_object_copies":98,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":49,"num_whiteouts":0,"num_read":5724,"num_read_kb":45272,"num_write":755,"num_write_kb":36122,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"55","length":"1"},{"start":"57","length":"1"},{"start":"59","length":"1"},{"start":"5b","length":"2"}]},{"pgid":"3.10","version":"207'821","reported_seq":6164,"reported_epoch":207,"state":"active+clean","last_fresh":"2024-09-14T06:32:12.658587+0000","last_change":"2024-09-14T06:32:12.658587+0000","last_active":"2024-09-14T06:32:12.658587+0000","last_peered":"2024-09-14T06:32:12.658587+0000","last_clean":"2024-09-14T06:32:12.658587+0000","last_became_active":"2024-09-14T06:32:04.621617+0000","last_became_peered":"2024-09-14T06:32:04.621617+0000","last_unstale":"2024-09-14T06:32:12.658587+0000","last_undegraded":"2024-09-14T06:32:12.658587+0000","last_fullsized":"2024-09-14T06:32:12.658587+0000","mapping_epoch":199,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":200,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":821,"log_dups_size":0,"ondisk_log_size":821,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T14:57:15.286788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00038959599999999997,"stat_sum":{"num_bytes":37997520,"num_objects":65,"num_object_clones":0,"num_object_copies":130,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":65,"num_whiteouts":0,"num_read":6618,"num_read_kb":34017,"num_write":969,"num_write_kb":36637,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":154,"num_bytes_recovered":9401812,"num_keys_recovered":45,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"57","length":"1"},{"start":"59","length":"1"},{"start":"5b","length":"2"}]},{"pgid":"3.11","version":"208'619","reported_seq":10896,"reported_epoch":208,"state":"active+clean","last_fresh":"2024-09-14T06:32:13.663466+0000","last_change":"2024-09-14T06:32:13.663466+0000","last_active":"2024-09-14T06:32:13.663466+0000","last_peered":"2024-09-14T06:32:13.663466+0000","last_clean":"2024-09-14T06:32:13.663466+0000","last_became_active":"2024-09-14T06:29:00.586480+0000","last_became_peered":"2024-09-14T06:29:00.586480+0000","last_unstale":"2024-09-14T06:32:13.663466+0000","last_undegraded":"2024-09-14T06:32:13.663466+0000","last_fullsized":"2024-09-14T06:32:13.663466+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":619,"log_dups_size":0,"ondisk_log_size":619,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T15:11:50.174862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00055159800000000004,"stat_sum":{"num_bytes":103911764,"num_objects":74,"num_object_clones":0,"num_object_copies":148,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":74,"num_whiteouts":0,"num_read":10178,"num_read_kb":109171,"num_write":746,"num_write_kb":48292,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"55","length":"1"},{"start":"57","length":"1"},{"start":"59","length":"1"},{"start":"5b","length":"2"}]},{"pgid":"3.12","version":"207'653","reported_seq":6922,"reported_epoch":207,"state":"active+clean","last_fresh":"2024-09-14T06:32:12.658199+0000","last_change":"2024-09-14T06:32:12.658199+0000","last_active":"2024-09-14T06:32:12.658199+0000","last_peered":"2024-09-14T06:32:12.658199+0000","last_clean":"2024-09-14T06:32:12.658199+0000","last_became_active":"2024-09-14T06:29:00.579826+0000","last_became_peered":"2024-09-14T06:29:00.579826+0000","last_unstale":"2024-09-14T06:32:12.658199+0000","last_undegraded":"2024-09-14T06:32:12.658199+0000","last_fullsized":"2024-09-14T06:32:12.658199+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":653,"log_dups_size":0,"ondisk_log_size":653,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T14:44:57.259429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000643591,"stat_sum":{"num_bytes":42483919,"num_objects":73,"num_object_clones":0,"num_object_copies":146,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":73,"num_whiteouts":0,"num_read":6842,"num_read_kb":58865,"num_write":819,"num_write_kb":42420,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"57","length":"1"},{"start":"59","length":"1"},{"start":"5b","length":"2"}]},{"pgid":"3.13","version":"208'877","reported_seq":8244,"reported_epoch":208,"state":"active+clean","last_fresh":"2024-09-14T06:32:13.742462+0000","last_change":"2024-09-14T06:32:13.663533+0000","last_active":"2024-09-14T06:32:13.742462+0000","last_peered":"2024-09-14T06:32:13.742462+0000","last_clean":"2024-09-14T06:32:13.742462+0000","last_became_active":"2024-09-14T06:29:00.586520+0000","last_became_peered":"2024-09-14T06:29:00.586520+0000","last_unstale":"2024-09-14T06:32:13.742462+0000","last_undegraded":"2024-09-14T06:32:13.742462+0000","last_fullsized":"2024-09-14T06:32:13.742462+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":877,"log_dups_size":0,"ondisk_log_size":877,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T18:07:27.711675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064785299999999999,"stat_sum":{"num_bytes":85870266,"num_objects":63,"num_object_clones":0,"num_object_copies":126,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":63,"num_whiteouts":0,"num_read":9027,"num_read_kb":65664,"num_write":1124,"num_write_kb":47419,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"55","length":"1"},{"start":"57","length":"1"},{"start":"59","length":"1"},{"start":"5b","length":"2"}]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":2004932344,"num_objects":2151,"num_object_clones":0,"num_object_copies":4302,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":156,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2151,"num_whiteouts":0,"num_read":230842,"num_read_kb":2176990,"num_write":34560,"num_write_kb":1477812,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4133,"num_bytes_recovered":555504806,"num_keys_recovered":466,"num_objects_omap":44,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":134471680,"data_stored":1321310178,"data_compressed":82361941,"data_compressed_allocated":109842432,"data_compressed_original":1301915750,"omap_allocated":0,"internal_metadata":0},"log_size":24825,"ondisk_log_size":24825,"up":64,"acting":63,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":131072,"data_stored":1180736,"data_compressed":65166,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509485,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":76548,"kb_used_data":18836,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94295292,"statfs":{"total":96636764160,"available":96558379008,"internally_reserved":0,"allocated":19288064,"data_stored":161499942,"data_compressed":9753290,"data_compressed_allocated":13131776,"data_compressed_original":157359019,"omap_allocated":1590,"internal_metadata":58915274},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":0,"seq":64424509484,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509486,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":86008,"kb_used_data":22828,"kb_used_omap":1,"kb_used_meta":63102,"kb_avail":94285832,"statfs":{"total":96636764160,"available":96548691968,"internally_reserved":0,"allocated":23375872,"data_stored":202683279,"data_compressed":12507586,"data_compressed_allocated":16994304,"data_compressed_original":198429030,"omap_allocated":1595,"internal_metadata":64616901},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509485,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":99176,"kb_used_data":27652,"kb_used_omap":1,"kb_used_meta":71422,"kb_avail":94272664,"statfs":{"total":96636764160,"available":96535207936,"internally_reserved":0,"allocated":28315648,"data_stored":256783818,"data_compressed":15993453,"data_compressed_allocated":21262336,"data_compressed_original":252034110,"omap_allocated":1595,"internal_metadata":73136581},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509486,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":80772,"kb_used_data":19056,"kb_used_omap":1,"kb_used_meta":61694,"kb_avail":94291068,"statfs":{"total":96636764160,"available":96554053632,"internally_reserved":0,"allocated":19513344,"data_stored":170274174,"data_compressed":10352043,"data_compressed_allocated":13901824,"data_compressed_original":166626935,"omap_allocated":1595,"internal_metadata":63175109},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":14,"seq":60129542189,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":34940,"kb_used_data":7908,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94336900,"statfs":{"total":96636764160,"available":96600985600,"internally_reserved":0,"allocated":8097792,"data_stored":57649402,"data_compressed":3691919,"data_compressed_allocated":4816896,"data_compressed_original":55842929,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542188,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":96496,"kb_used_data":25040,"kb_used_omap":1,"kb_used_meta":71422,"kb_avail":94275344,"statfs":{"total":96636764160,"available":96537952256,"internally_reserved":0,"allocated":25640960,"data_stored":221544868,"data_compressed":13894558,"data_compressed_allocated":18673664,"data_compressed_original":216700251,"omap_allocated":1595,"internal_metadata":73136581},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574894,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":94156,"kb_used_data":22708,"kb_used_omap":1,"kb_used_meta":71422,"kb_avail":94277684,"statfs":{"total":96636764160,"available":96540348416,"internally_reserved":0,"allocated":23252992,"data_stored":207666606,"data_compressed":13129610,"data_compressed_allocated":17367040,"data_compressed_original":203771904,"omap_allocated":1593,"internal_metadata":73136583},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32583,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32583,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":22642688,"data_stored":215695918,"data_compressed":13584983,"data_compressed_allocated":18202624,"data_compressed_original":212092251,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":16781312,"data_stored":167924327,"data_compressed":10246884,"data_compressed_allocated":13713408,"data_compressed_original":165521015,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":5492736,"data_stored":55472628,"data_compressed":3620216,"data_compressed_allocated":4710400,"data_compressed_original":54884465,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":20717568,"data_stored":206432339,"data_compressed":13129610,"data_compressed_allocated":17367040,"data_compressed_original":203771904,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":25223168,"data_stored":252662737,"data_compressed":15819805,"data_compressed_allocated":20889600,"data_compressed_original":249326654,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":20480000,"data_stored":199416160,"data_compressed":12404973,"data_compressed_allocated":16728064,"data_compressed_original":196491622,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":6631424,"data_stored":65369250,"data_compressed":3936333,"data_compressed_allocated":5361664,"data_compressed_original":64393940,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16502784,"data_stored":158336819,"data_compressed":9619137,"data_compressed_allocated":12869632,"data_compressed_original":155433899,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-14T06:32:18.079 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-14T06:32:18.132 INFO:teuthology.orchestra.run.smithi138.stdout:ERROR: (22) Invalid argument 2024-09-14T06:32:18.132 INFO:teuthology.orchestra.run.smithi138.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:32:18.141 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-14T06:32:18.167 INFO:tasks.ceph.osd.5.smithi138.stderr:2024-09-14T06:32:18.166+0000 7fe13746f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33618) UID: 0 2024-09-14T06:32:18.266 INFO:teuthology.orchestra.run.smithi094.stdout:ERROR: (22) Invalid argument 2024-09-14T06:32:18.266 INFO:teuthology.orchestra.run.smithi094.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:32:18.269 INFO:tasks.ceph.osd.6.smithi138.stderr:2024-09-14T06:32:18.268+0000 7f8cd8849640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33620) UID: 0 2024-09-14T06:32:18.275 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-14T06:32:18.369 INFO:tasks.ceph.osd.0.smithi094.stderr:2024-09-14T06:32:18.368+0000 7f07c9333640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33636) UID: 0 2024-09-14T06:32:18.380 INFO:teuthology.orchestra.run.smithi094.stdout:ERROR: (22) Invalid argument 2024-09-14T06:32:18.380 INFO:teuthology.orchestra.run.smithi094.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:32:18.389 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-14T06:32:18.411 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-14T06:32:18.411 INFO:teuthology.orchestra.run.smithi094.stderr:dumped all 2024-09-14T06:32:18.423 INFO:teuthology.orchestra.run.smithi094.stdout:{"pg_ready":true,"pg_map":{"version":314,"stamp":"2024-09-14T06:32:17.671689+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":2005522731,"num_objects":2155,"num_object_clones":0,"num_object_copies":4310,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":156,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2155,"num_whiteouts":0,"num_read":230908,"num_read_kb":2177045,"num_write":34663,"num_write_kb":1479442,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4133,"num_bytes_recovered":555504806,"num_keys_recovered":466,"num_objects_omap":44,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":24900,"ondisk_log_size":24900,"up":82,"acting":81,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":7,"num_per_pool_osds":7,"num_per_pool_omap_osds":7,"kb":660602880,"kb_used":568096,"kb_used_data":144028,"kb_used_omap":10,"kb_used_meta":423605,"kb_avail":660034784,"statfs":{"total":676457349120,"available":675875618816,"internally_reserved":0,"allocated":147484672,"data_stored":1278102089,"data_compressed":79322459,"data_compressed_allocated":106147840,"data_compressed_original":1250764178,"omap_allocated":11157,"internal_metadata":433771627},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-124802491,"num_objects":-263,"num_object_clones":0,"num_object_copies":-526,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":156,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-263,"num_whiteouts":0,"num_read":1221,"num_read_kb":2116,"num_write":480,"num_write_kb":279,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.013232"},"pg_stats":[{"pgid":"3.1f","version":"207'728","reported_seq":7785,"reported_epoch":207,"state":"active+clean","last_fresh":"2024-09-14T06:32:12.761779+0000","last_change":"2024-09-14T06:32:12.659020+0000","last_active":"2024-09-14T06:32:12.761779+0000","last_peered":"2024-09-14T06:32:12.761779+0000","last_clean":"2024-09-14T06:32:12.761779+0000","last_became_active":"2024-09-14T06:29:00.579883+0000","last_became_peered":"2024-09-14T06:29:00.579883+0000","last_unstale":"2024-09-14T06:32:12.761779+0000","last_undegraded":"2024-09-14T06:32:12.761779+0000","last_fullsized":"2024-09-14T06:32:12.761779+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":728,"log_dups_size":0,"ondisk_log_size":728,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T10:15:02.779710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00040664400000000001,"stat_sum":{"num_bytes":91062694,"num_objects":69,"num_object_clones":0,"num_object_copies":138,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":69,"num_whiteouts":0,"num_read":8088,"num_read_kb":86571,"num_write":1057,"num_write_kb":49526,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"57","length":"1"},{"start":"59","length":"1"},{"start":"5b","length":"2"}]},{"pgid":"3.1e","version":"210'709","reported_seq":9445,"reported_epoch":210,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-14T06:32:15.632212+0000","last_change":"2024-09-14T06:32:15.631533+0000","last_active":"2024-09-14T06:32:15.632212+0000","last_peered":"2024-09-14T06:32:15.632212+0000","last_clean":"2024-09-14T06:32:12.761016+0000","last_became_active":"2024-09-14T06:32:15.628941+0000","last_became_peered":"2024-09-14T06:32:15.628941+0000","last_unstale":"2024-09-14T06:32:15.632212+0000","last_undegraded":"2024-09-14T06:32:15.612555+0000","last_fullsized":"2024-09-14T06:32:15.610111+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":200,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":709,"log_dups_size":0,"ondisk_log_size":709,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":1,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00042255199999999998,"stat_sum":{"num_bytes":59789316,"num_objects":61,"num_object_clones":0,"num_object_copies":122,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":156,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":61,"num_whiteouts":0,"num_read":8272,"num_read_kb":67628,"num_write":951,"num_write_kb":40402,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":557,"num_bytes_recovered":70268016,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3],"avail_no_missing":["3","1"],"object_location_counts":[{"shards":"1,3","objects":61},{"shards":"3","objects":-1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"57","length":"1"},{"start":"59","length":"1"},{"start":"5b","length":"1"}]},{"pgid":"3.1d","version":"208'941","reported_seq":6948,"reported_epoch":208,"state":"active+clean","last_fresh":"2024-09-14T06:32:13.662308+0000","last_change":"2024-09-14T06:32:13.662308+0000","last_active":"2024-09-14T06:32:13.662308+0000","last_peered":"2024-09-14T06:32:13.662308+0000","last_clean":"2024-09-14T06:32:13.662308+0000","last_became_active":"2024-09-14T06:29:00.579345+0000","last_became_peered":"2024-09-14T06:29:00.579345+0000","last_unstale":"2024-09-14T06:32:13.662308+0000","last_undegraded":"2024-09-14T06:32:13.662308+0000","last_fullsized":"2024-09-14T06:32:13.662308+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":941,"log_dups_size":0,"ondisk_log_size":941,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T16:59:53.722458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00044556000000000001,"stat_sum":{"num_bytes":84988890,"num_objects":77,"num_object_clones":0,"num_object_copies":154,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":77,"num_whiteouts":0,"num_read":7600,"num_read_kb":64149,"num_write":1423,"num_write_kb":53271,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"55","length":"1"},{"start":"57","length":"1"},{"start":"59","length":"1"},{"start":"5b","length":"2"}]},{"pgid":"3.1c","version":"208'631","reported_seq":5169,"reported_epoch":208,"state":"active+clean","last_fresh":"2024-09-14T06:32:13.752475+0000","last_change":"2024-09-14T06:32:13.662895+0000","last_active":"2024-09-14T06:32:13.752475+0000","last_peered":"2024-09-14T06:32:13.752475+0000","last_clean":"2024-09-14T06:32:13.752475+0000","last_became_active":"2024-09-14T06:32:04.618831+0000","last_became_peered":"2024-09-14T06:32:04.618831+0000","last_unstale":"2024-09-14T06:32:13.752475+0000","last_undegraded":"2024-09-14T06:32:13.752475+0000","last_fullsized":"2024-09-14T06:32:13.752475+0000","mapping_epoch":199,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":200,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":631,"log_dups_size":0,"ondisk_log_size":631,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T09:13:28.581097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00046365099999999998,"stat_sum":{"num_bytes":41533486,"num_objects":67,"num_object_clones":0,"num_object_copies":134,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":67,"num_whiteouts":0,"num_read":4225,"num_read_kb":43151,"num_write":750,"num_write_kb":45533,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":319,"num_bytes_recovered":32800897,"num_keys_recovered":223,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"55","length":"1"},{"start":"57","length":"1"},{"start":"59","length":"1"},{"start":"5b","length":"2"}]},{"pgid":"3.1b","version":"207'842","reported_seq":10952,"reported_epoch":207,"state":"active+clean","last_fresh":"2024-09-14T06:32:12.756464+0000","last_change":"2024-09-14T06:32:12.659531+0000","last_active":"2024-09-14T06:32:12.756464+0000","last_peered":"2024-09-14T06:32:12.756464+0000","last_clean":"2024-09-14T06:32:12.756464+0000","last_became_active":"2024-09-14T06:29:00.586183+0000","last_became_peered":"2024-09-14T06:29:00.586183+0000","last_unstale":"2024-09-14T06:32:12.756464+0000","last_undegraded":"2024-09-14T06:32:12.756464+0000","last_fullsized":"2024-09-14T06:32:12.756464+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":842,"log_dups_size":0,"ondisk_log_size":842,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T18:21:13.120536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00072037100000000003,"stat_sum":{"num_bytes":77932251,"num_objects":83,"num_object_clones":0,"num_object_copies":166,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":83,"num_whiteouts":0,"num_read":11697,"num_read_kb":107915,"num_write":1351,"num_write_kb":55091,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"57","length":"1"},{"start":"59","length":"1"},{"start":"5b","length":"2"}]},{"pgid":"3.1a","version":"208'1515","reported_seq":9700,"reported_epoch":208,"state":"active+clean","last_fresh":"2024-09-14T06:32:13.759334+0000","last_change":"2024-09-14T06:32:13.660594+0000","last_active":"2024-09-14T06:32:13.759334+0000","last_peered":"2024-09-14T06:32:13.759334+0000","last_clean":"2024-09-14T06:32:13.759334+0000","last_became_active":"2024-09-14T06:29:00.576206+0000","last_became_peered":"2024-09-14T06:29:00.576206+0000","last_unstale":"2024-09-14T06:32:13.759334+0000","last_undegraded":"2024-09-14T06:32:13.759334+0000","last_fullsized":"2024-09-14T06:32:13.759334+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":1515,"log_dups_size":0,"ondisk_log_size":1515,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T16:41:21.694673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050120000000000004,"stat_sum":{"num_bytes":67140503,"num_objects":74,"num_object_clones":0,"num_object_copies":148,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":74,"num_whiteouts":0,"num_read":12616,"num_read_kb":82257,"num_write":2294,"num_write_kb":55655,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"55","length":"1"},{"start":"57","length":"1"},{"start":"59","length":"1"},{"start":"5b","length":"2"}]},{"pgid":"3.19","version":"208'490","reported_seq":6058,"reported_epoch":208,"state":"active+clean","last_fresh":"2024-09-14T06:32:13.741675+0000","last_change":"2024-09-14T06:32:13.659799+0000","last_active":"2024-09-14T06:32:13.741675+0000","last_peered":"2024-09-14T06:32:13.741675+0000","last_clean":"2024-09-14T06:32:13.741675+0000","last_became_active":"2024-09-14T06:31:53.744800+0000","last_became_peered":"2024-09-14T06:31:53.744800+0000","last_unstale":"2024-09-14T06:32:13.741675+0000","last_undegraded":"2024-09-14T06:32:13.741675+0000","last_fullsized":"2024-09-14T06:32:13.741675+0000","mapping_epoch":188,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":189,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":490,"log_dups_size":0,"ondisk_log_size":490,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T09:48:33.686457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049229199999999997,"stat_sum":{"num_bytes":56946579,"num_objects":58,"num_object_clones":0,"num_object_copies":116,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":58,"num_whiteouts":0,"num_read":5078,"num_read_kb":67752,"num_write":693,"num_write_kb":43868,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":282,"num_bytes_recovered":41927489,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"55","length":"1"},{"start":"57","length":"1"},{"start":"59","length":"1"},{"start":"5b","length":"2"}]},{"pgid":"3.18","version":"208'463","reported_seq":5839,"reported_epoch":210,"state":"active+clean","last_fresh":"2024-09-14T06:32:15.610510+0000","last_change":"2024-09-14T06:32:13.662308+0000","last_active":"2024-09-14T06:32:15.610510+0000","last_peered":"2024-09-14T06:32:15.610510+0000","last_clean":"2024-09-14T06:32:15.610510+0000","last_became_active":"2024-09-14T06:31:53.745159+0000","last_became_peered":"2024-09-14T06:31:53.745159+0000","last_unstale":"2024-09-14T06:32:15.610510+0000","last_undegraded":"2024-09-14T06:32:15.610510+0000","last_fullsized":"2024-09-14T06:32:15.610510+0000","mapping_epoch":188,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":189,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":463,"log_dups_size":0,"ondisk_log_size":463,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T13:28:24.561144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00063330400000000001,"stat_sum":{"num_bytes":68306983,"num_objects":70,"num_object_clones":0,"num_object_copies":140,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":70,"num_whiteouts":0,"num_read":4455,"num_read_kb":78892,"num_write":589,"num_write_kb":51576,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":310,"num_bytes_recovered":56082976,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"55","length":"1"},{"start":"57","length":"1"},{"start":"59","length":"1"},{"start":"5b","length":"1"}]},{"pgid":"3.17","version":"207'856","reported_seq":8950,"reported_epoch":207,"state":"active+clean","last_fresh":"2024-09-14T06:32:12.659399+0000","last_change":"2024-09-14T06:32:12.659399+0000","last_active":"2024-09-14T06:32:12.659399+0000","last_peered":"2024-09-14T06:32:12.659399+0000","last_clean":"2024-09-14T06:32:12.659399+0000","last_became_active":"2024-09-14T06:29:00.579714+0000","last_became_peered":"2024-09-14T06:29:00.579714+0000","last_unstale":"2024-09-14T06:32:12.659399+0000","last_undegraded":"2024-09-14T06:32:12.659399+0000","last_fullsized":"2024-09-14T06:32:12.659399+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":856,"log_dups_size":0,"ondisk_log_size":856,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T10:36:04.398636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069585199999999995,"stat_sum":{"num_bytes":78451450,"num_objects":83,"num_object_clones":0,"num_object_copies":166,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":83,"num_whiteouts":0,"num_read":10072,"num_read_kb":68437,"num_write":992,"num_write_kb":48449,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"57","length":"1"},{"start":"59","length":"1"},{"start":"5b","length":"2"}]},{"pgid":"3.16","version":"203'1035","reported_seq":3574,"reported_epoch":208,"state":"active+clean","last_fresh":"2024-09-14T06:32:13.661922+0000","last_change":"2024-09-14T06:32:13.662104+0000","last_active":"2024-09-14T06:32:13.661922+0000","last_peered":"2024-09-14T06:32:13.661922+0000","last_clean":"2024-09-14T06:32:13.661922+0000","last_became_active":"2024-09-14T06:29:00.580070+0000","last_became_peered":"2024-09-14T06:29:00.580070+0000","last_unstale":"2024-09-14T06:32:13.661922+0000","last_undegraded":"2024-09-14T06:32:13.661922+0000","last_fullsized":"2024-09-14T06:32:13.661922+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":1035,"log_dups_size":0,"ondisk_log_size":1035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T14:17:44.932374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00033417000000000001,"stat_sum":{"num_bytes":42065946,"num_objects":68,"num_object_clones":0,"num_object_copies":136,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":68,"num_whiteouts":0,"num_read":5281,"num_read_kb":38236,"num_write":1630,"num_write_kb":37231,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"55","length":"1"},{"start":"57","length":"1"},{"start":"59","length":"1"},{"start":"5b","length":"2"}]},{"pgid":"3.15","version":"208'547","reported_seq":6205,"reported_epoch":208,"state":"active+clean","last_fresh":"2024-09-14T06:32:13.732385+0000","last_change":"2024-09-14T06:32:13.664759+0000","last_active":"2024-09-14T06:32:13.732385+0000","last_peered":"2024-09-14T06:32:13.732385+0000","last_clean":"2024-09-14T06:32:13.732385+0000","last_became_active":"2024-09-14T06:31:53.746091+0000","last_became_peered":"2024-09-14T06:31:53.746091+0000","last_unstale":"2024-09-14T06:32:13.732385+0000","last_undegraded":"2024-09-14T06:32:13.732385+0000","last_fullsized":"2024-09-14T06:32:13.732385+0000","mapping_epoch":188,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":189,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":547,"log_dups_size":0,"ondisk_log_size":547,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T09:15:05.610444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00144721,"stat_sum":{"num_bytes":79873129,"num_objects":64,"num_object_clones":0,"num_object_copies":128,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":4896,"num_read_kb":74723,"num_write":602,"num_write_kb":51185,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":271,"num_bytes_recovered":49981473,"num_keys_recovered":33,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"55","length":"1"},{"start":"57","length":"1"},{"start":"59","length":"1"},{"start":"5b","length":"2"}]},{"pgid":"3.14","version":"208'1024","reported_seq":5706,"reported_epoch":208,"state":"active+clean","last_fresh":"2024-09-14T06:32:13.660985+0000","last_change":"2024-09-14T06:32:13.660985+0000","last_active":"2024-09-14T06:32:13.660985+0000","last_peered":"2024-09-14T06:32:13.660985+0000","last_clean":"2024-09-14T06:32:13.660985+0000","last_became_active":"2024-09-14T06:29:00.582595+0000","last_became_peered":"2024-09-14T06:29:00.582595+0000","last_unstale":"2024-09-14T06:32:13.660985+0000","last_undegraded":"2024-09-14T06:32:13.660985+0000","last_fullsized":"2024-09-14T06:32:13.660985+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":1024,"log_dups_size":0,"ondisk_log_size":1024,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T07:52:57.903676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065583600000000003,"stat_sum":{"num_bytes":51834579,"num_objects":55,"num_object_clones":0,"num_object_copies":110,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":55,"num_whiteouts":0,"num_read":8562,"num_read_kb":46606,"num_write":1473,"num_write_kb":29812,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"55","length":"1"},{"start":"57","length":"1"},{"start":"59","length":"1"},{"start":"5b","length":"2"}]},{"pgid":"2.7","version":"0'0","reported_seq":34,"reported_epoch":208,"state":"active+clean","last_fresh":"2024-09-14T06:32:13.663307+0000","last_change":"2024-09-14T06:32:01.612470+0000","last_active":"2024-09-14T06:32:13.663307+0000","last_peered":"2024-09-14T06:32:13.663307+0000","last_clean":"2024-09-14T06:32:13.663307+0000","last_became_active":"2024-09-14T06:32:01.612121+0000","last_became_peered":"2024-09-14T06:32:01.612121+0000","last_unstale":"2024-09-14T06:32:13.663307+0000","last_undegraded":"2024-09-14T06:32:13.663307+0000","last_fullsized":"2024-09-14T06:32:13.663307+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:28:50.205350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:28:50.205350+0000","last_clean_scrub_stamp":"2024-09-14T06:28:50.205350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T13:40:52.537942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.6","version":"207'681","reported_seq":7916,"reported_epoch":207,"state":"active+clean","last_fresh":"2024-09-14T06:32:12.758188+0000","last_change":"2024-09-14T06:32:12.659011+0000","last_active":"2024-09-14T06:32:12.758188+0000","last_peered":"2024-09-14T06:32:12.758188+0000","last_clean":"2024-09-14T06:32:12.758188+0000","last_became_active":"2024-09-14T06:29:00.578283+0000","last_became_peered":"2024-09-14T06:29:00.578283+0000","last_unstale":"2024-09-14T06:32:12.758188+0000","last_undegraded":"2024-09-14T06:32:12.758188+0000","last_fullsized":"2024-09-14T06:32:12.758188+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":681,"log_dups_size":0,"ondisk_log_size":681,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T16:33:06.750083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00068571699999999997,"stat_sum":{"num_bytes":66750303,"num_objects":65,"num_object_clones":0,"num_object_copies":130,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":65,"num_whiteouts":0,"num_read":7993,"num_read_kb":78244,"num_write":771,"num_write_kb":48270,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"57","length":"1"},{"start":"59","length":"1"},{"start":"5b","length":"2"}]},{"pgid":"2.6","version":"0'0","reported_seq":399,"reported_epoch":208,"state":"active+clean","last_fresh":"2024-09-14T06:32:13.659575+0000","last_change":"2024-09-14T06:32:01.608456+0000","last_active":"2024-09-14T06:32:13.659575+0000","last_peered":"2024-09-14T06:32:13.659575+0000","last_clean":"2024-09-14T06:32:13.659575+0000","last_became_active":"2024-09-14T06:32:01.608178+0000","last_became_peered":"2024-09-14T06:32:01.608178+0000","last_unstale":"2024-09-14T06:32:13.659575+0000","last_undegraded":"2024-09-14T06:32:13.659575+0000","last_fullsized":"2024-09-14T06:32:13.659575+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:28:50.205350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:28:50.205350+0000","last_clean_scrub_stamp":"2024-09-14T06:28:50.205350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T10:16:40.480031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065018200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"207'753","reported_seq":4127,"reported_epoch":210,"state":"active+clean","last_fresh":"2024-09-14T06:32:15.610254+0000","last_change":"2024-09-14T06:32:13.662724+0000","last_active":"2024-09-14T06:32:15.610254+0000","last_peered":"2024-09-14T06:32:15.610254+0000","last_clean":"2024-09-14T06:32:15.610254+0000","last_became_active":"2024-09-14T06:31:54.747923+0000","last_became_peered":"2024-09-14T06:31:54.747923+0000","last_unstale":"2024-09-14T06:32:15.610254+0000","last_undegraded":"2024-09-14T06:32:15.610254+0000","last_fullsized":"2024-09-14T06:32:15.610254+0000","mapping_epoch":189,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":190,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":753,"log_dups_size":0,"ondisk_log_size":753,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T13:55:37.796703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00044561900000000002,"stat_sum":{"num_bytes":38285856,"num_objects":51,"num_object_clones":0,"num_object_copies":102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":51,"num_whiteouts":0,"num_read":4380,"num_read_kb":40951,"num_write":1098,"num_write_kb":36111,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":183,"num_bytes_recovered":13912513,"num_keys_recovered":22,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"55","length":"1"},{"start":"57","length":"1"},{"start":"59","length":"1"},{"start":"5b","length":"1"}]},{"pgid":"2.5","version":"0'0","reported_seq":393,"reported_epoch":208,"state":"active+clean","last_fresh":"2024-09-14T06:32:13.663731+0000","last_change":"2024-09-14T06:28:53.640035+0000","last_active":"2024-09-14T06:32:13.663731+0000","last_peered":"2024-09-14T06:32:13.663731+0000","last_clean":"2024-09-14T06:32:13.663731+0000","last_became_active":"2024-09-14T06:28:51.308058+0000","last_became_peered":"2024-09-14T06:28:51.308058+0000","last_unstale":"2024-09-14T06:32:13.663731+0000","last_undegraded":"2024-09-14T06:32:13.663731+0000","last_fullsized":"2024-09-14T06:32:13.663731+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:28:50.205350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:28:50.205350+0000","last_clean_scrub_stamp":"2024-09-14T06:28:50.205350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T13:35:48.544057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00037824000000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"207'613","reported_seq":3312,"reported_epoch":207,"state":"peering","last_fresh":"2024-09-14T06:32:12.756338+0000","last_change":"2024-09-14T06:32:13.659971+0000","last_active":"2024-09-14T06:32:13.659971+0000","last_peered":"2024-09-14T06:32:12.756338+0000","last_clean":"2024-09-14T06:32:12.756338+0000","last_became_active":"2024-09-14T06:31:41.604731+0000","last_became_peered":"2024-09-14T06:31:41.604731+0000","last_unstale":"2024-09-14T06:32:12.756338+0000","last_undegraded":"2024-09-14T06:32:12.756338+0000","last_fullsized":"2024-09-14T06:32:12.756338+0000","mapping_epoch":208,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":613,"log_dups_size":0,"ondisk_log_size":613,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00058330999999999997,"stat_sum":{"num_bytes":37139964,"num_objects":64,"num_object_clones":0,"num_object_copies":128,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":2589,"num_read_kb":37386,"num_write":807,"num_write_kb":40850,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":153,"num_bytes_recovered":15692987,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2,5],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"57","length":"1"},{"start":"59","length":"1"},{"start":"5b","length":"2"}]},{"pgid":"2.4","version":"0'0","reported_seq":393,"reported_epoch":208,"state":"active+clean","last_fresh":"2024-09-14T06:32:13.659521+0000","last_change":"2024-09-14T06:28:53.226072+0000","last_active":"2024-09-14T06:32:13.659521+0000","last_peered":"2024-09-14T06:32:13.659521+0000","last_clean":"2024-09-14T06:32:13.659521+0000","last_became_active":"2024-09-14T06:28:51.308072+0000","last_became_peered":"2024-09-14T06:28:51.308072+0000","last_unstale":"2024-09-14T06:32:13.659521+0000","last_undegraded":"2024-09-14T06:32:13.659521+0000","last_fullsized":"2024-09-14T06:32:13.659521+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:28:50.205350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:28:50.205350+0000","last_clean_scrub_stamp":"2024-09-14T06:28:50.205350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T13:15:09.052328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066197399999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"208'827","reported_seq":5247,"reported_epoch":208,"state":"active+clean","last_fresh":"2024-09-14T06:32:13.749810+0000","last_change":"2024-09-14T06:32:13.662584+0000","last_active":"2024-09-14T06:32:13.749810+0000","last_peered":"2024-09-14T06:32:13.749810+0000","last_clean":"2024-09-14T06:32:13.749810+0000","last_became_active":"2024-09-14T06:31:54.744373+0000","last_became_peered":"2024-09-14T06:31:54.744373+0000","last_unstale":"2024-09-14T06:32:13.749810+0000","last_undegraded":"2024-09-14T06:32:13.749810+0000","last_fullsized":"2024-09-14T06:32:13.749810+0000","mapping_epoch":189,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":190,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":827,"log_dups_size":0,"ondisk_log_size":827,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T10:48:07.580270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041746399999999999,"stat_sum":{"num_bytes":43262791,"num_objects":67,"num_object_clones":0,"num_object_copies":134,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":67,"num_whiteouts":0,"num_read":4507,"num_read_kb":42629,"num_write":1188,"num_write_kb":49494,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":330,"num_bytes_recovered":28798893,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"55","length":"1"},{"start":"57","length":"1"},{"start":"59","length":"1"},{"start":"5b","length":"2"}]},{"pgid":"3.3","version":"208'468","reported_seq":10079,"reported_epoch":208,"state":"active+clean","last_fresh":"2024-09-14T06:32:13.745191+0000","last_change":"2024-09-14T06:32:13.661140+0000","last_active":"2024-09-14T06:32:13.745191+0000","last_peered":"2024-09-14T06:32:13.745191+0000","last_clean":"2024-09-14T06:32:13.745191+0000","last_became_active":"2024-09-14T06:29:00.584678+0000","last_became_peered":"2024-09-14T06:29:00.584678+0000","last_unstale":"2024-09-14T06:32:13.745191+0000","last_undegraded":"2024-09-14T06:32:13.745191+0000","last_fullsized":"2024-09-14T06:32:13.745191+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":468,"log_dups_size":0,"ondisk_log_size":468,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T13:20:16.770689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00082891200000000001,"stat_sum":{"num_bytes":62895515,"num_objects":56,"num_object_clones":0,"num_object_copies":112,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":56,"num_whiteouts":0,"num_read":9151,"num_read_kb":88923,"num_write":541,"num_write_kb":51562,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"55","length":"1"},{"start":"57","length":"1"},{"start":"59","length":"1"},{"start":"5b","length":"2"}]},{"pgid":"2.2","version":"22'2","reported_seq":395,"reported_epoch":208,"state":"active+clean","last_fresh":"2024-09-14T06:32:13.661683+0000","last_change":"2024-09-14T06:28:53.231428+0000","last_active":"2024-09-14T06:32:13.661683+0000","last_peered":"2024-09-14T06:32:13.661683+0000","last_clean":"2024-09-14T06:32:13.661683+0000","last_became_active":"2024-09-14T06:28:51.222937+0000","last_became_peered":"2024-09-14T06:28:51.222937+0000","last_unstale":"2024-09-14T06:32:13.661683+0000","last_undegraded":"2024-09-14T06:32:13.661683+0000","last_fullsized":"2024-09-14T06:32:13.661683+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:28:50.205350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:28:50.205350+0000","last_clean_scrub_stamp":"2024-09-14T06:28:50.205350+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T14:27:37.233976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010072970000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.0","version":"208'994","reported_seq":6568,"reported_epoch":208,"state":"active+clean","last_fresh":"2024-09-14T06:32:13.659853+0000","last_change":"2024-09-14T06:32:13.659853+0000","last_active":"2024-09-14T06:32:13.659853+0000","last_peered":"2024-09-14T06:32:13.659853+0000","last_clean":"2024-09-14T06:32:13.659853+0000","last_became_active":"2024-09-14T06:29:00.574873+0000","last_became_peered":"2024-09-14T06:29:00.574873+0000","last_unstale":"2024-09-14T06:32:13.659853+0000","last_undegraded":"2024-09-14T06:32:13.659853+0000","last_fullsized":"2024-09-14T06:32:13.659853+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":994,"log_dups_size":0,"ondisk_log_size":994,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T13:32:24.881003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00058771500000000005,"stat_sum":{"num_bytes":74732015,"num_objects":77,"num_object_clones":0,"num_object_copies":154,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":77,"num_whiteouts":0,"num_read":8014,"num_read_kb":85277,"num_write":1533,"num_write_kb":58684,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"55","length":"1"},{"start":"57","length":"1"},{"start":"59","length":"1"},{"start":"5b","length":"2"}]},{"pgid":"2.1","version":"0'0","reported_seq":49,"reported_epoch":205,"state":"active+clean","last_fresh":"2024-09-14T06:32:10.679671+0000","last_change":"2024-09-14T06:31:50.607764+0000","last_active":"2024-09-14T06:32:10.679671+0000","last_peered":"2024-09-14T06:32:10.679671+0000","last_clean":"2024-09-14T06:32:10.679671+0000","last_became_active":"2024-09-14T06:31:50.607447+0000","last_became_peered":"2024-09-14T06:31:50.607447+0000","last_unstale":"2024-09-14T06:32:10.679671+0000","last_undegraded":"2024-09-14T06:32:10.679671+0000","last_fullsized":"2024-09-14T06:32:10.679671+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":186,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:28:50.205350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:28:50.205350+0000","last_clean_scrub_stamp":"2024-09-14T06:28:50.205350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T09:11:53.601856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053403100000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1","version":"203'695","reported_seq":11304,"reported_epoch":207,"state":"active+clean","last_fresh":"2024-09-14T06:32:12.657918+0000","last_change":"2024-09-14T06:32:12.658168+0000","last_active":"2024-09-14T06:32:12.657918+0000","last_peered":"2024-09-14T06:32:12.657918+0000","last_clean":"2024-09-14T06:32:12.657918+0000","last_became_active":"2024-09-14T06:30:57.768238+0000","last_became_peered":"2024-09-14T06:30:57.768238+0000","last_unstale":"2024-09-14T06:32:12.657918+0000","last_undegraded":"2024-09-14T06:32:12.657918+0000","last_fullsized":"2024-09-14T06:32:12.657918+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":695,"log_dups_size":0,"ondisk_log_size":695,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T07:49:50.576428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00090325799999999999,"stat_sum":{"num_bytes":93666435,"num_objects":66,"num_object_clones":0,"num_object_copies":132,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":66,"num_whiteouts":0,"num_read":11016,"num_read_kb":112206,"num_write":870,"num_write_kb":47387,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":183,"num_bytes_recovered":30431994,"num_keys_recovered":22,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"57","length":"1"},{"start":"59","length":"1"},{"start":"5b","length":"2"}]},{"pgid":"2.0","version":"0'0","reported_seq":393,"reported_epoch":208,"state":"active+clean","last_fresh":"2024-09-14T06:32:13.662840+0000","last_change":"2024-09-14T06:28:53.640096+0000","last_active":"2024-09-14T06:32:13.662840+0000","last_peered":"2024-09-14T06:32:13.662840+0000","last_clean":"2024-09-14T06:32:13.662840+0000","last_became_active":"2024-09-14T06:28:51.223069+0000","last_became_peered":"2024-09-14T06:28:51.223069+0000","last_unstale":"2024-09-14T06:32:13.662840+0000","last_undegraded":"2024-09-14T06:32:13.662840+0000","last_fullsized":"2024-09-14T06:32:13.662840+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:28:50.205350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:28:50.205350+0000","last_clean_scrub_stamp":"2024-09-14T06:28:50.205350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T15:13:00.471509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00043714600000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"210'1558","reported_seq":7376,"reported_epoch":210,"state":"active+clean","last_fresh":"2024-09-14T06:32:15.640212+0000","last_change":"2024-09-14T06:32:13.662240+0000","last_active":"2024-09-14T06:32:15.640212+0000","last_peered":"2024-09-14T06:32:15.640212+0000","last_clean":"2024-09-14T06:32:15.640212+0000","last_became_active":"2024-09-14T06:31:54.744160+0000","last_became_peered":"2024-09-14T06:31:54.744160+0000","last_unstale":"2024-09-14T06:32:15.640212+0000","last_undegraded":"2024-09-14T06:32:15.640212+0000","last_fullsized":"2024-09-14T06:32:15.640212+0000","mapping_epoch":189,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":190,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":1558,"log_dups_size":0,"ondisk_log_size":1558,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T12:41:51.808698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00055251900000000001,"stat_sum":{"num_bytes":66459400,"num_objects":80,"num_object_clones":0,"num_object_copies":160,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":80,"num_whiteouts":0,"num_read":10653,"num_read_kb":70538,"num_write":2490,"num_write_kb":53554,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":327,"num_bytes_recovered":50047063,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"55","length":"1"},{"start":"57","length":"1"},{"start":"59","length":"1"},{"start":"5b","length":"1"}]},{"pgid":"2.3","version":"20'1","reported_seq":394,"reported_epoch":208,"state":"active+clean","last_fresh":"2024-09-14T06:32:13.661658+0000","last_change":"2024-09-14T06:28:53.230933+0000","last_active":"2024-09-14T06:32:13.661658+0000","last_peered":"2024-09-14T06:32:13.661658+0000","last_clean":"2024-09-14T06:32:13.661658+0000","last_became_active":"2024-09-14T06:28:51.222874+0000","last_became_peered":"2024-09-14T06:28:51.222874+0000","last_unstale":"2024-09-14T06:32:13.661658+0000","last_undegraded":"2024-09-14T06:32:13.661658+0000","last_fullsized":"2024-09-14T06:32:13.661658+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:28:50.205350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:28:50.205350+0000","last_clean_scrub_stamp":"2024-09-14T06:28:50.205350+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T12:04:25.190171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050808899999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'72","reported_seq":484,"reported_epoch":208,"state":"active+clean","last_fresh":"2024-09-14T06:32:13.662987+0000","last_change":"2024-09-14T06:28:44.839885+0000","last_active":"2024-09-14T06:32:13.662987+0000","last_peered":"2024-09-14T06:32:13.662987+0000","last_clean":"2024-09-14T06:32:13.662987+0000","last_became_active":"2024-09-14T06:28:44.839573+0000","last_became_peered":"2024-09-14T06:28:44.839573+0000","last_unstale":"2024-09-14T06:32:13.662987+0000","last_undegraded":"2024-09-14T06:32:13.662987+0000","last_fullsized":"2024-09-14T06:32:13.662987+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:28:43.828494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:28:43.828494+0000","last_clean_scrub_stamp":"2024-09-14T06:28:43.828494+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T14:41:25.787293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.8","version":"208'586","reported_seq":6123,"reported_epoch":210,"state":"active+clean","last_fresh":"2024-09-14T06:32:15.610718+0000","last_change":"2024-09-14T06:32:13.662649+0000","last_active":"2024-09-14T06:32:15.610718+0000","last_peered":"2024-09-14T06:32:15.610718+0000","last_clean":"2024-09-14T06:32:15.610718+0000","last_became_active":"2024-09-14T06:31:55.754694+0000","last_became_peered":"2024-09-14T06:31:55.754694+0000","last_unstale":"2024-09-14T06:32:15.610718+0000","last_undegraded":"2024-09-14T06:32:15.610718+0000","last_fullsized":"2024-09-14T06:32:15.610718+0000","mapping_epoch":190,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":191,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":586,"log_dups_size":0,"ondisk_log_size":586,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T12:30:44.816178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097722299999999994,"stat_sum":{"num_bytes":78517473,"num_objects":71,"num_object_clones":0,"num_object_copies":142,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":71,"num_whiteouts":0,"num_read":5608,"num_read_kb":90696,"num_write":834,"num_write_kb":57685,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":319,"num_bytes_recovered":60352269,"num_keys_recovered":22,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"55","length":"1"},{"start":"57","length":"1"},{"start":"59","length":"1"},{"start":"5b","length":"1"}]},{"pgid":"3.9","version":"208'516","reported_seq":4063,"reported_epoch":208,"state":"active+clean","last_fresh":"2024-09-14T06:32:13.660676+0000","last_change":"2024-09-14T06:32:13.660676+0000","last_active":"2024-09-14T06:32:13.660676+0000","last_peered":"2024-09-14T06:32:13.660676+0000","last_clean":"2024-09-14T06:32:13.660676+0000","last_became_active":"2024-09-14T06:29:00.573179+0000","last_became_peered":"2024-09-14T06:29:00.573179+0000","last_unstale":"2024-09-14T06:32:13.660676+0000","last_undegraded":"2024-09-14T06:32:13.660676+0000","last_fullsized":"2024-09-14T06:32:13.660676+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":516,"log_dups_size":0,"ondisk_log_size":516,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T18:24:55.803054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054865399999999998,"stat_sum":{"num_bytes":42231296,"num_objects":66,"num_object_clones":0,"num_object_copies":132,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":66,"num_whiteouts":0,"num_read":3634,"num_read_kb":64262,"num_write":699,"num_write_kb":44923,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"55","length":"1"},{"start":"57","length":"1"},{"start":"59","length":"1"},{"start":"5b","length":"2"}]},{"pgid":"3.a","version":"208'738","reported_seq":4416,"reported_epoch":208,"state":"active+clean","last_fresh":"2024-09-14T06:32:13.757393+0000","last_change":"2024-09-14T06:32:13.661025+0000","last_active":"2024-09-14T06:32:13.757393+0000","last_peered":"2024-09-14T06:32:13.757393+0000","last_clean":"2024-09-14T06:32:13.757393+0000","last_became_active":"2024-09-14T06:32:04.617283+0000","last_became_peered":"2024-09-14T06:32:04.617283+0000","last_unstale":"2024-09-14T06:32:13.757393+0000","last_undegraded":"2024-09-14T06:32:13.757393+0000","last_fullsized":"2024-09-14T06:32:13.757393+0000","mapping_epoch":199,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":200,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":738,"log_dups_size":0,"ondisk_log_size":738,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T15:40:09.086871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070773799999999997,"stat_sum":{"num_bytes":46401995,"num_objects":67,"num_object_clones":0,"num_object_copies":134,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":67,"num_whiteouts":0,"num_read":4807,"num_read_kb":57874,"num_write":1106,"num_write_kb":47517,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":161,"num_bytes_recovered":21723371,"num_keys_recovered":33,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"55","length":"1"},{"start":"57","length":"1"},{"start":"59","length":"1"},{"start":"5b","length":"2"}]},{"pgid":"3.b","version":"210'1477","reported_seq":8243,"reported_epoch":210,"state":"active+clean","last_fresh":"2024-09-14T06:32:15.683852+0000","last_change":"2024-09-14T06:32:13.662503+0000","last_active":"2024-09-14T06:32:15.683852+0000","last_peered":"2024-09-14T06:32:15.683852+0000","last_clean":"2024-09-14T06:32:15.683852+0000","last_became_active":"2024-09-14T06:31:55.756219+0000","last_became_peered":"2024-09-14T06:31:55.756219+0000","last_unstale":"2024-09-14T06:32:15.683852+0000","last_undegraded":"2024-09-14T06:32:15.683852+0000","last_fullsized":"2024-09-14T06:32:15.683852+0000","mapping_epoch":190,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":191,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":1477,"log_dups_size":0,"ondisk_log_size":1477,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T08:46:05.977011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000634164,"stat_sum":{"num_bytes":58790990,"num_objects":73,"num_object_clones":0,"num_object_copies":146,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":73,"num_whiteouts":0,"num_read":12454,"num_read_kb":63624,"num_write":2349,"num_write_kb":43651,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":291,"num_bytes_recovered":37370632,"num_keys_recovered":66,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"55","length":"1"},{"start":"57","length":"1"},{"start":"59","length":"1"},{"start":"5b","length":"1"}]},{"pgid":"3.c","version":"208'547","reported_seq":5890,"reported_epoch":208,"state":"active+clean","last_fresh":"2024-09-14T06:32:13.736039+0000","last_change":"2024-09-14T06:32:13.662856+0000","last_active":"2024-09-14T06:32:13.736039+0000","last_peered":"2024-09-14T06:32:13.736039+0000","last_clean":"2024-09-14T06:32:13.736039+0000","last_became_active":"2024-09-14T06:31:53.747355+0000","last_became_peered":"2024-09-14T06:31:53.747355+0000","last_unstale":"2024-09-14T06:32:13.736039+0000","last_undegraded":"2024-09-14T06:32:13.736039+0000","last_fullsized":"2024-09-14T06:32:13.736039+0000","mapping_epoch":188,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":189,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":547,"log_dups_size":0,"ondisk_log_size":547,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T16:05:49.827036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00037372000000000002,"stat_sum":{"num_bytes":51667677,"num_objects":60,"num_object_clones":0,"num_object_copies":120,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":4805,"num_read_kb":50398,"num_write":722,"num_write_kb":40822,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":293,"num_bytes_recovered":36712421,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"55","length":"1"},{"start":"57","length":"1"},{"start":"59","length":"1"},{"start":"5b","length":"2"}]},{"pgid":"3.d","version":"208'482","reported_seq":8794,"reported_epoch":208,"state":"active+clean","last_fresh":"2024-09-14T06:32:13.745059+0000","last_change":"2024-09-14T06:32:13.664738+0000","last_active":"2024-09-14T06:32:13.745059+0000","last_peered":"2024-09-14T06:32:13.745059+0000","last_clean":"2024-09-14T06:32:13.745059+0000","last_became_active":"2024-09-14T06:29:00.580362+0000","last_became_peered":"2024-09-14T06:29:00.580362+0000","last_unstale":"2024-09-14T06:32:13.745059+0000","last_undegraded":"2024-09-14T06:32:13.745059+0000","last_fullsized":"2024-09-14T06:32:13.745059+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":482,"log_dups_size":0,"ondisk_log_size":482,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T10:55:42.483074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010157390000000001,"stat_sum":{"num_bytes":95841531,"num_objects":72,"num_object_clones":0,"num_object_copies":144,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":72,"num_whiteouts":0,"num_read":8023,"num_read_kb":93542,"num_write":644,"num_write_kb":41026,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"55","length":"1"},{"start":"57","length":"1"},{"start":"59","length":"1"},{"start":"5b","length":"2"}]},{"pgid":"3.e","version":"207'520","reported_seq":4739,"reported_epoch":208,"state":"active+clean","last_fresh":"2024-09-14T06:32:13.663598+0000","last_change":"2024-09-14T06:32:13.663783+0000","last_active":"2024-09-14T06:32:13.663598+0000","last_peered":"2024-09-14T06:32:13.663598+0000","last_clean":"2024-09-14T06:32:13.663598+0000","last_became_active":"2024-09-14T06:29:00.586374+0000","last_became_peered":"2024-09-14T06:29:00.586374+0000","last_unstale":"2024-09-14T06:32:13.663598+0000","last_undegraded":"2024-09-14T06:32:13.663598+0000","last_fullsized":"2024-09-14T06:32:13.663598+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":520,"log_dups_size":0,"ondisk_log_size":520,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T16:01:48.654194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053448299999999999,"stat_sum":{"num_bytes":48343618,"num_objects":63,"num_object_clones":0,"num_object_copies":126,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":63,"num_whiteouts":0,"num_read":3977,"num_read_kb":61134,"num_write":690,"num_write_kb":43787,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"55","length":"1"},{"start":"57","length":"1"},{"start":"59","length":"1"},{"start":"5b","length":"2"}]},{"pgid":"3.f","version":"208'619","reported_seq":5340,"reported_epoch":208,"state":"active+clean","last_fresh":"2024-09-14T06:32:13.755426+0000","last_change":"2024-09-14T06:32:13.663765+0000","last_active":"2024-09-14T06:32:13.755426+0000","last_peered":"2024-09-14T06:32:13.755426+0000","last_clean":"2024-09-14T06:32:13.755426+0000","last_became_active":"2024-09-14T06:29:00.586491+0000","last_became_peered":"2024-09-14T06:29:00.586491+0000","last_unstale":"2024-09-14T06:32:13.755426+0000","last_undegraded":"2024-09-14T06:32:13.755426+0000","last_fullsized":"2024-09-14T06:32:13.755426+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":619,"log_dups_size":0,"ondisk_log_size":619,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T12:18:46.267108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057572199999999997,"stat_sum":{"num_bytes":29756210,"num_objects":49,"num_object_clones":0,"num_object_copies":98,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":49,"num_whiteouts":0,"num_read":5724,"num_read_kb":45272,"num_write":755,"num_write_kb":36122,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"55","length":"1"},{"start":"57","length":"1"},{"start":"59","length":"1"},{"start":"5b","length":"2"}]},{"pgid":"3.10","version":"207'821","reported_seq":6164,"reported_epoch":207,"state":"active+clean","last_fresh":"2024-09-14T06:32:12.658587+0000","last_change":"2024-09-14T06:32:12.658587+0000","last_active":"2024-09-14T06:32:12.658587+0000","last_peered":"2024-09-14T06:32:12.658587+0000","last_clean":"2024-09-14T06:32:12.658587+0000","last_became_active":"2024-09-14T06:32:04.621617+0000","last_became_peered":"2024-09-14T06:32:04.621617+0000","last_unstale":"2024-09-14T06:32:12.658587+0000","last_undegraded":"2024-09-14T06:32:12.658587+0000","last_fullsized":"2024-09-14T06:32:12.658587+0000","mapping_epoch":199,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":200,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":821,"log_dups_size":0,"ondisk_log_size":821,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T14:57:15.286788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00038959599999999997,"stat_sum":{"num_bytes":37997520,"num_objects":65,"num_object_clones":0,"num_object_copies":130,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":65,"num_whiteouts":0,"num_read":6618,"num_read_kb":34017,"num_write":969,"num_write_kb":36637,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":154,"num_bytes_recovered":9401812,"num_keys_recovered":45,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"57","length":"1"},{"start":"59","length":"1"},{"start":"5b","length":"2"}]},{"pgid":"3.11","version":"208'619","reported_seq":10896,"reported_epoch":208,"state":"active+clean","last_fresh":"2024-09-14T06:32:13.663466+0000","last_change":"2024-09-14T06:32:13.663466+0000","last_active":"2024-09-14T06:32:13.663466+0000","last_peered":"2024-09-14T06:32:13.663466+0000","last_clean":"2024-09-14T06:32:13.663466+0000","last_became_active":"2024-09-14T06:29:00.586480+0000","last_became_peered":"2024-09-14T06:29:00.586480+0000","last_unstale":"2024-09-14T06:32:13.663466+0000","last_undegraded":"2024-09-14T06:32:13.663466+0000","last_fullsized":"2024-09-14T06:32:13.663466+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":619,"log_dups_size":0,"ondisk_log_size":619,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T15:11:50.174862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00055159800000000004,"stat_sum":{"num_bytes":103911764,"num_objects":74,"num_object_clones":0,"num_object_copies":148,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":74,"num_whiteouts":0,"num_read":10178,"num_read_kb":109171,"num_write":746,"num_write_kb":48292,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"55","length":"1"},{"start":"57","length":"1"},{"start":"59","length":"1"},{"start":"5b","length":"2"}]},{"pgid":"3.12","version":"207'653","reported_seq":6922,"reported_epoch":207,"state":"active+clean","last_fresh":"2024-09-14T06:32:12.658199+0000","last_change":"2024-09-14T06:32:12.658199+0000","last_active":"2024-09-14T06:32:12.658199+0000","last_peered":"2024-09-14T06:32:12.658199+0000","last_clean":"2024-09-14T06:32:12.658199+0000","last_became_active":"2024-09-14T06:29:00.579826+0000","last_became_peered":"2024-09-14T06:29:00.579826+0000","last_unstale":"2024-09-14T06:32:12.658199+0000","last_undegraded":"2024-09-14T06:32:12.658199+0000","last_fullsized":"2024-09-14T06:32:12.658199+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":653,"log_dups_size":0,"ondisk_log_size":653,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T14:44:57.259429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000643591,"stat_sum":{"num_bytes":42483919,"num_objects":73,"num_object_clones":0,"num_object_copies":146,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":73,"num_whiteouts":0,"num_read":6842,"num_read_kb":58865,"num_write":819,"num_write_kb":42420,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"57","length":"1"},{"start":"59","length":"1"},{"start":"5b","length":"2"}]},{"pgid":"3.13","version":"208'877","reported_seq":8244,"reported_epoch":208,"state":"active+clean","last_fresh":"2024-09-14T06:32:13.742462+0000","last_change":"2024-09-14T06:32:13.663533+0000","last_active":"2024-09-14T06:32:13.742462+0000","last_peered":"2024-09-14T06:32:13.742462+0000","last_clean":"2024-09-14T06:32:13.742462+0000","last_became_active":"2024-09-14T06:29:00.586520+0000","last_became_peered":"2024-09-14T06:29:00.586520+0000","last_unstale":"2024-09-14T06:32:13.742462+0000","last_undegraded":"2024-09-14T06:32:13.742462+0000","last_fullsized":"2024-09-14T06:32:13.742462+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":877,"log_dups_size":0,"ondisk_log_size":877,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T18:07:27.711675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064785299999999999,"stat_sum":{"num_bytes":85870266,"num_objects":63,"num_object_clones":0,"num_object_copies":126,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":63,"num_whiteouts":0,"num_read":9027,"num_read_kb":65664,"num_write":1124,"num_write_kb":47419,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"55","length":"1"},{"start":"57","length":"1"},{"start":"59","length":"1"},{"start":"5b","length":"2"}]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":2004932344,"num_objects":2151,"num_object_clones":0,"num_object_copies":4302,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":156,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2151,"num_whiteouts":0,"num_read":230842,"num_read_kb":2176990,"num_write":34560,"num_write_kb":1477812,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4133,"num_bytes_recovered":555504806,"num_keys_recovered":466,"num_objects_omap":44,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":134471680,"data_stored":1321310178,"data_compressed":82361941,"data_compressed_allocated":109842432,"data_compressed_original":1301915750,"omap_allocated":0,"internal_metadata":0},"log_size":24825,"ondisk_log_size":24825,"up":64,"acting":63,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":131072,"data_stored":1180736,"data_compressed":65166,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509485,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":76548,"kb_used_data":18836,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94295292,"statfs":{"total":96636764160,"available":96558379008,"internally_reserved":0,"allocated":19288064,"data_stored":161499942,"data_compressed":9753290,"data_compressed_allocated":13131776,"data_compressed_original":157359019,"omap_allocated":1590,"internal_metadata":58915274},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":0,"seq":64424509484,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509486,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":86008,"kb_used_data":22828,"kb_used_omap":1,"kb_used_meta":63102,"kb_avail":94285832,"statfs":{"total":96636764160,"available":96548691968,"internally_reserved":0,"allocated":23375872,"data_stored":202683279,"data_compressed":12507586,"data_compressed_allocated":16994304,"data_compressed_original":198429030,"omap_allocated":1595,"internal_metadata":64616901},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509485,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":99176,"kb_used_data":27652,"kb_used_omap":1,"kb_used_meta":71422,"kb_avail":94272664,"statfs":{"total":96636764160,"available":96535207936,"internally_reserved":0,"allocated":28315648,"data_stored":256783818,"data_compressed":15993453,"data_compressed_allocated":21262336,"data_compressed_original":252034110,"omap_allocated":1595,"internal_metadata":73136581},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509486,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":80772,"kb_used_data":19056,"kb_used_omap":1,"kb_used_meta":61694,"kb_avail":94291068,"statfs":{"total":96636764160,"available":96554053632,"internally_reserved":0,"allocated":19513344,"data_stored":170274174,"data_compressed":10352043,"data_compressed_allocated":13901824,"data_compressed_original":166626935,"omap_allocated":1595,"internal_metadata":63175109},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":14,"seq":60129542189,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":34940,"kb_used_data":7908,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94336900,"statfs":{"total":96636764160,"available":96600985600,"internally_reserved":0,"allocated":8097792,"data_stored":57649402,"data_compressed":3691919,"data_compressed_allocated":4816896,"data_compressed_original":55842929,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542188,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":96496,"kb_used_data":25040,"kb_used_omap":1,"kb_used_meta":71422,"kb_avail":94275344,"statfs":{"total":96636764160,"available":96537952256,"internally_reserved":0,"allocated":25640960,"data_stored":221544868,"data_compressed":13894558,"data_compressed_allocated":18673664,"data_compressed_original":216700251,"omap_allocated":1595,"internal_metadata":73136581},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574894,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":94156,"kb_used_data":22708,"kb_used_omap":1,"kb_used_meta":71422,"kb_avail":94277684,"statfs":{"total":96636764160,"available":96540348416,"internally_reserved":0,"allocated":23252992,"data_stored":207666606,"data_compressed":13129610,"data_compressed_allocated":17367040,"data_compressed_original":203771904,"omap_allocated":1593,"internal_metadata":73136583},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32583,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32583,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":22642688,"data_stored":215695918,"data_compressed":13584983,"data_compressed_allocated":18202624,"data_compressed_original":212092251,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":16781312,"data_stored":167924327,"data_compressed":10246884,"data_compressed_allocated":13713408,"data_compressed_original":165521015,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":5492736,"data_stored":55472628,"data_compressed":3620216,"data_compressed_allocated":4710400,"data_compressed_original":54884465,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":20717568,"data_stored":206432339,"data_compressed":13129610,"data_compressed_allocated":17367040,"data_compressed_original":203771904,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":25223168,"data_stored":252662737,"data_compressed":15819805,"data_compressed_allocated":20889600,"data_compressed_original":249326654,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":20480000,"data_stored":199416160,"data_compressed":12404973,"data_compressed_allocated":16728064,"data_compressed_original":196491622,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":6631424,"data_stored":65369250,"data_compressed":3936333,"data_compressed_allocated":5361664,"data_compressed_original":64393940,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16502784,"data_stored":158336819,"data_compressed":9619137,"data_compressed_allocated":12869632,"data_compressed_original":155433899,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-14T06:32:18.425 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-14T06:32:18.508 INFO:tasks.ceph.osd.1.smithi094.stderr:2024-09-14T06:32:18.507+0000 7f328e47a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33641) UID: 0 2024-09-14T06:32:18.515 INFO:teuthology.orchestra.run.smithi094.stdout:ERROR: (22) Invalid argument 2024-09-14T06:32:18.515 INFO:teuthology.orchestra.run.smithi094.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:32:18.524 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-14T06:32:18.570 INFO:tasks.ceph.osd.5.smithi138.stderr:2024-09-14T06:32:18.569+0000 7fe13746f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33618) UID: 0 2024-09-14T06:32:18.669 INFO:teuthology.orchestra.run.smithi094.stdout:ERROR: (22) Invalid argument 2024-09-14T06:32:18.669 INFO:teuthology.orchestra.run.smithi094.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:32:18.671 INFO:tasks.ceph.osd.4.smithi138.stderr:2024-09-14T06:32:18.670+0000 7f6c653c8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33615) UID: 0 2024-09-14T06:32:18.678 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-14T06:32:18.686 INFO:teuthology.orchestra.run.smithi094.stderr:noscrub is unset 2024-09-14T06:32:18.773 INFO:tasks.ceph.osd.3.smithi094.stderr:2024-09-14T06:32:18.771+0000 7feeeafe8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33655) UID: 0 2024-09-14T06:32:18.816 INFO:teuthology.orchestra.run.smithi094.stdout:ERROR: (22) Invalid argument 2024-09-14T06:32:18.816 INFO:teuthology.orchestra.run.smithi094.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:32:18.829 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-14T06:32:18.873 INFO:tasks.ceph.osd.7.smithi138.stderr:2024-09-14T06:32:18.872+0000 7f3c6a21d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33623) UID: 0 2024-09-14T06:32:18.885 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-14T06:32:18.885 INFO:teuthology.orchestra.run.smithi094.stdout:{"fsid":"200ad3da-bd63-4e04-af74-efb5ca0af75e","health":{"status":"HEALTH_WARN","checks":{"OSDMAP_FLAGS":{"severity":"HEALTH_WARN","summary":{"message":"nodeep-scrub flag(s) set","count":12},"muted":false},"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 156/4310 objects degraded (3.619%), 1 pg degraded","count":1},"muted":false}},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":228,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":213,"num_osds":8,"num_up_osds":8,"osd_up_since":1726295322,"num_in_osds":7,"osd_in_since":1726295520,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":39},{"state_name":"peering","count":1},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":1}],"num_pgs":41,"num_pools":3,"num_objects":2155,"data_bytes":2005522731,"bytes_used":581730304,"bytes_avail":675875618816,"bytes_total":676457349120,"inactive_pgs_ratio":0.024390242993831635,"degraded_objects":156,"degraded_total":4310,"degraded_ratio":0.036194895591647333,"read_bytes_sec":541600,"write_bytes_sec":71411,"read_op_per_sec":305,"write_op_per_sec":119},"fsmap":{"epoch":1,"btime":"2024-09-14T06:28:30:774849+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":5,"modified":"2024-09-14T06:29:45.290986+0000","services":{}},"progress_events":{}} 2024-09-14T06:32:18.898 INFO:tasks.ceph.ceph_manager.ceph:no progress seen, keeping timeout for now 2024-09-14T06:32:18.899 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-14T06:32:18.974 INFO:tasks.ceph.osd.5.smithi138.stderr:2024-09-14T06:32:18.973+0000 7fe13746f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33618) UID: 0 2024-09-14T06:32:18.977 INFO:teuthology.orchestra.run.smithi094.stdout:ERROR: (22) Invalid argument 2024-09-14T06:32:18.977 INFO:teuthology.orchestra.run.smithi094.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:32:18.986 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-14T06:32:19.075 INFO:tasks.ceph.osd.7.smithi138.stderr:2024-09-14T06:32:19.074+0000 7f3c6a21d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33623) UID: 0 2024-09-14T06:32:19.132 INFO:teuthology.orchestra.run.smithi094.stdout:ERROR: (22) Invalid argument 2024-09-14T06:32:19.132 INFO:teuthology.orchestra.run.smithi094.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:32:19.141 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-14T06:32:19.184 INFO:tasks.ceph.osd.0.smithi094.stderr:2024-09-14T06:32:19.183+0000 7f07c9333640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33636) UID: 0 2024-09-14T06:32:19.234 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-14T06:32:19.235 INFO:teuthology.orchestra.run.smithi094.stderr:dumped all 2024-09-14T06:32:19.245 INFO:teuthology.orchestra.run.smithi094.stdout:ERROR: (22) Invalid argument 2024-09-14T06:32:19.245 INFO:teuthology.orchestra.run.smithi094.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:32:19.246 INFO:teuthology.orchestra.run.smithi094.stdout:{"pg_ready":true,"pg_map":{"version":315,"stamp":"2024-09-14T06:32:18.685182+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":2000819503,"num_objects":2127,"num_object_clones":0,"num_object_copies":4254,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":156,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2127,"num_whiteouts":0,"num_read":230978,"num_read_kb":2177104,"num_write":34697,"num_write_kb":1479447,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4133,"num_bytes_recovered":555504806,"num_keys_recovered":466,"num_objects_omap":44,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":24935,"ondisk_log_size":24935,"up":82,"acting":81,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":85,"num_osds":7,"num_per_pool_osds":7,"num_per_pool_omap_osds":7,"kb":660602880,"kb_used":569416,"kb_used_data":145360,"kb_used_omap":10,"kb_used_meta":423605,"kb_avail":660033464,"statfs":{"total":676457349120,"available":675874267136,"internally_reserved":0,"allocated":148848640,"data_stored":1300777354,"data_compressed":80416070,"data_compressed_allocated":107327488,"data_compressed_original":1273345426,"omap_allocated":11157,"internal_metadata":433771627},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-43563940,"num_objects":-72,"num_object_clones":0,"num_object_copies":-144,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":156,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-72,"num_whiteouts":0,"num_read":109,"num_read_kb":86,"num_write":91,"num_write_kb":10,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.025568"},"pg_stats":[{"pgid":"3.1f","version":"211'731","reported_seq":7808,"reported_epoch":212,"state":"active+clean","last_fresh":"2024-09-14T06:32:17.687926+0000","last_change":"2024-09-14T06:32:16.675117+0000","last_active":"2024-09-14T06:32:17.687926+0000","last_peered":"2024-09-14T06:32:17.687926+0000","last_clean":"2024-09-14T06:32:17.687926+0000","last_became_active":"2024-09-14T06:29:00.579883+0000","last_became_peered":"2024-09-14T06:29:00.579883+0000","last_unstale":"2024-09-14T06:32:17.687926+0000","last_undegraded":"2024-09-14T06:32:17.687926+0000","last_fullsized":"2024-09-14T06:32:17.687926+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":731,"log_dups_size":0,"ondisk_log_size":731,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T15:33:15.819830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00060207499999999998,"stat_sum":{"num_bytes":90557590,"num_objects":66,"num_object_clones":0,"num_object_copies":132,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":66,"num_whiteouts":0,"num_read":8088,"num_read_kb":86571,"num_write":1060,"num_write_kb":49526,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"53","length":"1"},{"start":"55","length":"1"},{"start":"57","length":"1"},{"start":"59","length":"1"},{"start":"5b","length":"1"}]},{"pgid":"3.1e","version":"210'709","reported_seq":9445,"reported_epoch":210,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-14T06:32:15.632212+0000","last_change":"2024-09-14T06:32:15.631533+0000","last_active":"2024-09-14T06:32:15.632212+0000","last_peered":"2024-09-14T06:32:15.632212+0000","last_clean":"2024-09-14T06:32:12.761016+0000","last_became_active":"2024-09-14T06:32:15.628941+0000","last_became_peered":"2024-09-14T06:32:15.628941+0000","last_unstale":"2024-09-14T06:32:15.632212+0000","last_undegraded":"2024-09-14T06:32:15.612555+0000","last_fullsized":"2024-09-14T06:32:15.610111+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":200,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":709,"log_dups_size":0,"ondisk_log_size":709,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":1,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00042255199999999998,"stat_sum":{"num_bytes":59789316,"num_objects":61,"num_object_clones":0,"num_object_copies":122,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":156,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":61,"num_whiteouts":0,"num_read":8272,"num_read_kb":67628,"num_write":951,"num_write_kb":40402,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":557,"num_bytes_recovered":70268016,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3],"avail_no_missing":["3","1"],"object_location_counts":[{"shards":"1,3","objects":61},{"shards":"3","objects":-1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"57","length":"1"},{"start":"59","length":"1"},{"start":"5b","length":"1"}]},{"pgid":"3.1d","version":"208'941","reported_seq":6948,"reported_epoch":208,"state":"active+clean","last_fresh":"2024-09-14T06:32:13.662308+0000","last_change":"2024-09-14T06:32:13.662308+0000","last_active":"2024-09-14T06:32:13.662308+0000","last_peered":"2024-09-14T06:32:13.662308+0000","last_clean":"2024-09-14T06:32:13.662308+0000","last_became_active":"2024-09-14T06:29:00.579345+0000","last_became_peered":"2024-09-14T06:29:00.579345+0000","last_unstale":"2024-09-14T06:32:13.662308+0000","last_undegraded":"2024-09-14T06:32:13.662308+0000","last_fullsized":"2024-09-14T06:32:13.662308+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":941,"log_dups_size":0,"ondisk_log_size":941,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T16:59:53.722458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00044556000000000001,"stat_sum":{"num_bytes":84988890,"num_objects":77,"num_object_clones":0,"num_object_copies":154,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":77,"num_whiteouts":0,"num_read":7600,"num_read_kb":64149,"num_write":1423,"num_write_kb":53271,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"55","length":"1"},{"start":"57","length":"1"},{"start":"59","length":"1"},{"start":"5b","length":"2"}]},{"pgid":"3.1c","version":"208'631","reported_seq":5169,"reported_epoch":208,"state":"active+clean","last_fresh":"2024-09-14T06:32:13.752475+0000","last_change":"2024-09-14T06:32:13.662895+0000","last_active":"2024-09-14T06:32:13.752475+0000","last_peered":"2024-09-14T06:32:13.752475+0000","last_clean":"2024-09-14T06:32:13.752475+0000","last_became_active":"2024-09-14T06:32:04.618831+0000","last_became_peered":"2024-09-14T06:32:04.618831+0000","last_unstale":"2024-09-14T06:32:13.752475+0000","last_undegraded":"2024-09-14T06:32:13.752475+0000","last_fullsized":"2024-09-14T06:32:13.752475+0000","mapping_epoch":199,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":200,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":631,"log_dups_size":0,"ondisk_log_size":631,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T09:13:28.581097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00046365099999999998,"stat_sum":{"num_bytes":41533486,"num_objects":67,"num_object_clones":0,"num_object_copies":134,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":67,"num_whiteouts":0,"num_read":4225,"num_read_kb":43151,"num_write":750,"num_write_kb":45533,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":319,"num_bytes_recovered":32800897,"num_keys_recovered":223,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"55","length":"1"},{"start":"57","length":"1"},{"start":"59","length":"1"},{"start":"5b","length":"2"}]},{"pgid":"3.1b","version":"211'848","reported_seq":10978,"reported_epoch":212,"state":"active+clean","last_fresh":"2024-09-14T06:32:17.688246+0000","last_change":"2024-09-14T06:32:16.675101+0000","last_active":"2024-09-14T06:32:17.688246+0000","last_peered":"2024-09-14T06:32:17.688246+0000","last_clean":"2024-09-14T06:32:17.688246+0000","last_became_active":"2024-09-14T06:29:00.586183+0000","last_became_peered":"2024-09-14T06:29:00.586183+0000","last_unstale":"2024-09-14T06:32:17.688246+0000","last_undegraded":"2024-09-14T06:32:17.688246+0000","last_fullsized":"2024-09-14T06:32:17.688246+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":848,"log_dups_size":0,"ondisk_log_size":848,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T15:10:42.430832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001113331,"stat_sum":{"num_bytes":76920933,"num_objects":77,"num_object_clones":0,"num_object_copies":154,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":77,"num_whiteouts":0,"num_read":11697,"num_read_kb":107915,"num_write":1357,"num_write_kb":55091,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"53","length":"1"},{"start":"55","length":"1"},{"start":"57","length":"1"},{"start":"59","length":"1"},{"start":"5b","length":"1"}]},{"pgid":"3.1a","version":"208'1515","reported_seq":9700,"reported_epoch":208,"state":"active+clean","last_fresh":"2024-09-14T06:32:13.759334+0000","last_change":"2024-09-14T06:32:13.660594+0000","last_active":"2024-09-14T06:32:13.759334+0000","last_peered":"2024-09-14T06:32:13.759334+0000","last_clean":"2024-09-14T06:32:13.759334+0000","last_became_active":"2024-09-14T06:29:00.576206+0000","last_became_peered":"2024-09-14T06:29:00.576206+0000","last_unstale":"2024-09-14T06:32:13.759334+0000","last_undegraded":"2024-09-14T06:32:13.759334+0000","last_fullsized":"2024-09-14T06:32:13.759334+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":1515,"log_dups_size":0,"ondisk_log_size":1515,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T16:41:21.694673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050120000000000004,"stat_sum":{"num_bytes":67140503,"num_objects":74,"num_object_clones":0,"num_object_copies":148,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":74,"num_whiteouts":0,"num_read":12616,"num_read_kb":82257,"num_write":2294,"num_write_kb":55655,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"55","length":"1"},{"start":"57","length":"1"},{"start":"59","length":"1"},{"start":"5b","length":"2"}]},{"pgid":"3.19","version":"208'490","reported_seq":6058,"reported_epoch":208,"state":"active+clean","last_fresh":"2024-09-14T06:32:13.741675+0000","last_change":"2024-09-14T06:32:13.659799+0000","last_active":"2024-09-14T06:32:13.741675+0000","last_peered":"2024-09-14T06:32:13.741675+0000","last_clean":"2024-09-14T06:32:13.741675+0000","last_became_active":"2024-09-14T06:31:53.744800+0000","last_became_peered":"2024-09-14T06:31:53.744800+0000","last_unstale":"2024-09-14T06:32:13.741675+0000","last_undegraded":"2024-09-14T06:32:13.741675+0000","last_fullsized":"2024-09-14T06:32:13.741675+0000","mapping_epoch":188,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":189,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":490,"log_dups_size":0,"ondisk_log_size":490,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T09:48:33.686457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049229199999999997,"stat_sum":{"num_bytes":56946579,"num_objects":58,"num_object_clones":0,"num_object_copies":116,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":58,"num_whiteouts":0,"num_read":5078,"num_read_kb":67752,"num_write":693,"num_write_kb":43868,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":282,"num_bytes_recovered":41927489,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"55","length":"1"},{"start":"57","length":"1"},{"start":"59","length":"1"},{"start":"5b","length":"2"}]},{"pgid":"3.18","version":"208'463","reported_seq":5839,"reported_epoch":210,"state":"active+clean","last_fresh":"2024-09-14T06:32:15.610510+0000","last_change":"2024-09-14T06:32:13.662308+0000","last_active":"2024-09-14T06:32:15.610510+0000","last_peered":"2024-09-14T06:32:15.610510+0000","last_clean":"2024-09-14T06:32:15.610510+0000","last_became_active":"2024-09-14T06:31:53.745159+0000","last_became_peered":"2024-09-14T06:31:53.745159+0000","last_unstale":"2024-09-14T06:32:15.610510+0000","last_undegraded":"2024-09-14T06:32:15.610510+0000","last_fullsized":"2024-09-14T06:32:15.610510+0000","mapping_epoch":188,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":189,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":463,"log_dups_size":0,"ondisk_log_size":463,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T13:28:24.561144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00063330400000000001,"stat_sum":{"num_bytes":68306983,"num_objects":70,"num_object_clones":0,"num_object_copies":140,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":70,"num_whiteouts":0,"num_read":4455,"num_read_kb":78892,"num_write":589,"num_write_kb":51576,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":310,"num_bytes_recovered":56082976,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"55","length":"1"},{"start":"57","length":"1"},{"start":"59","length":"1"},{"start":"5b","length":"1"}]},{"pgid":"3.17","version":"212'867","reported_seq":9002,"reported_epoch":212,"state":"active+clean","last_fresh":"2024-09-14T06:32:17.725724+0000","last_change":"2024-09-14T06:32:16.675334+0000","last_active":"2024-09-14T06:32:17.725724+0000","last_peered":"2024-09-14T06:32:17.725724+0000","last_clean":"2024-09-14T06:32:17.725724+0000","last_became_active":"2024-09-14T06:29:00.579714+0000","last_became_peered":"2024-09-14T06:29:00.579714+0000","last_unstale":"2024-09-14T06:32:17.725724+0000","last_undegraded":"2024-09-14T06:32:17.725724+0000","last_fullsized":"2024-09-14T06:32:17.725724+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":867,"log_dups_size":0,"ondisk_log_size":867,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T13:34:00.327619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00052952999999999995,"stat_sum":{"num_bytes":77132538,"num_objects":77,"num_object_clones":0,"num_object_copies":154,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":77,"num_whiteouts":0,"num_read":10142,"num_read_kb":68496,"num_write":1004,"num_write_kb":48454,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"53","length":"1"},{"start":"55","length":"1"},{"start":"57","length":"1"},{"start":"59","length":"1"},{"start":"5b","length":"1"}]},{"pgid":"3.16","version":"203'1035","reported_seq":3574,"reported_epoch":208,"state":"active+clean","last_fresh":"2024-09-14T06:32:13.661922+0000","last_change":"2024-09-14T06:32:13.662104+0000","last_active":"2024-09-14T06:32:13.661922+0000","last_peered":"2024-09-14T06:32:13.661922+0000","last_clean":"2024-09-14T06:32:13.661922+0000","last_became_active":"2024-09-14T06:29:00.580070+0000","last_became_peered":"2024-09-14T06:29:00.580070+0000","last_unstale":"2024-09-14T06:32:13.661922+0000","last_undegraded":"2024-09-14T06:32:13.661922+0000","last_fullsized":"2024-09-14T06:32:13.661922+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":1035,"log_dups_size":0,"ondisk_log_size":1035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T14:17:44.932374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00033417000000000001,"stat_sum":{"num_bytes":42065946,"num_objects":68,"num_object_clones":0,"num_object_copies":136,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":68,"num_whiteouts":0,"num_read":5281,"num_read_kb":38236,"num_write":1630,"num_write_kb":37231,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"55","length":"1"},{"start":"57","length":"1"},{"start":"59","length":"1"},{"start":"5b","length":"2"}]},{"pgid":"3.15","version":"208'547","reported_seq":6205,"reported_epoch":208,"state":"active+clean","last_fresh":"2024-09-14T06:32:13.732385+0000","last_change":"2024-09-14T06:32:13.664759+0000","last_active":"2024-09-14T06:32:13.732385+0000","last_peered":"2024-09-14T06:32:13.732385+0000","last_clean":"2024-09-14T06:32:13.732385+0000","last_became_active":"2024-09-14T06:31:53.746091+0000","last_became_peered":"2024-09-14T06:31:53.746091+0000","last_unstale":"2024-09-14T06:32:13.732385+0000","last_undegraded":"2024-09-14T06:32:13.732385+0000","last_fullsized":"2024-09-14T06:32:13.732385+0000","mapping_epoch":188,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":189,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":547,"log_dups_size":0,"ondisk_log_size":547,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T09:15:05.610444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00144721,"stat_sum":{"num_bytes":79873129,"num_objects":64,"num_object_clones":0,"num_object_copies":128,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":4896,"num_read_kb":74723,"num_write":602,"num_write_kb":51185,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":271,"num_bytes_recovered":49981473,"num_keys_recovered":33,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"55","length":"1"},{"start":"57","length":"1"},{"start":"59","length":"1"},{"start":"5b","length":"2"}]},{"pgid":"3.14","version":"208'1024","reported_seq":5706,"reported_epoch":208,"state":"active+clean","last_fresh":"2024-09-14T06:32:13.660985+0000","last_change":"2024-09-14T06:32:13.660985+0000","last_active":"2024-09-14T06:32:13.660985+0000","last_peered":"2024-09-14T06:32:13.660985+0000","last_clean":"2024-09-14T06:32:13.660985+0000","last_became_active":"2024-09-14T06:29:00.582595+0000","last_became_peered":"2024-09-14T06:29:00.582595+0000","last_unstale":"2024-09-14T06:32:13.660985+0000","last_undegraded":"2024-09-14T06:32:13.660985+0000","last_fullsized":"2024-09-14T06:32:13.660985+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":1024,"log_dups_size":0,"ondisk_log_size":1024,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T07:52:57.903676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065583600000000003,"stat_sum":{"num_bytes":51834579,"num_objects":55,"num_object_clones":0,"num_object_copies":110,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":55,"num_whiteouts":0,"num_read":8562,"num_read_kb":46606,"num_write":1473,"num_write_kb":29812,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"55","length":"1"},{"start":"57","length":"1"},{"start":"59","length":"1"},{"start":"5b","length":"2"}]},{"pgid":"2.7","version":"0'0","reported_seq":34,"reported_epoch":208,"state":"active+clean","last_fresh":"2024-09-14T06:32:13.663307+0000","last_change":"2024-09-14T06:32:01.612470+0000","last_active":"2024-09-14T06:32:13.663307+0000","last_peered":"2024-09-14T06:32:13.663307+0000","last_clean":"2024-09-14T06:32:13.663307+0000","last_became_active":"2024-09-14T06:32:01.612121+0000","last_became_peered":"2024-09-14T06:32:01.612121+0000","last_unstale":"2024-09-14T06:32:13.663307+0000","last_undegraded":"2024-09-14T06:32:13.663307+0000","last_fullsized":"2024-09-14T06:32:13.663307+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:28:50.205350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:28:50.205350+0000","last_clean_scrub_stamp":"2024-09-14T06:28:50.205350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T13:40:52.537942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.6","version":"212'687","reported_seq":7942,"reported_epoch":212,"state":"active+clean","last_fresh":"2024-09-14T06:32:17.687977+0000","last_change":"2024-09-14T06:32:16.674174+0000","last_active":"2024-09-14T06:32:17.687977+0000","last_peered":"2024-09-14T06:32:17.687977+0000","last_clean":"2024-09-14T06:32:17.687977+0000","last_became_active":"2024-09-14T06:29:00.578283+0000","last_became_peered":"2024-09-14T06:29:00.578283+0000","last_unstale":"2024-09-14T06:32:17.687977+0000","last_undegraded":"2024-09-14T06:32:17.687977+0000","last_fullsized":"2024-09-14T06:32:17.687977+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":687,"log_dups_size":0,"ondisk_log_size":687,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T15:54:45.137038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00107485,"stat_sum":{"num_bytes":65980763,"num_objects":61,"num_object_clones":0,"num_object_copies":122,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":61,"num_whiteouts":0,"num_read":7993,"num_read_kb":78244,"num_write":775,"num_write_kb":48270,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"53","length":"1"},{"start":"55","length":"1"},{"start":"57","length":"1"},{"start":"59","length":"1"},{"start":"5b","length":"1"}]},{"pgid":"2.6","version":"0'0","reported_seq":399,"reported_epoch":208,"state":"active+clean","last_fresh":"2024-09-14T06:32:13.659575+0000","last_change":"2024-09-14T06:32:01.608456+0000","last_active":"2024-09-14T06:32:13.659575+0000","last_peered":"2024-09-14T06:32:13.659575+0000","last_clean":"2024-09-14T06:32:13.659575+0000","last_became_active":"2024-09-14T06:32:01.608178+0000","last_became_peered":"2024-09-14T06:32:01.608178+0000","last_unstale":"2024-09-14T06:32:13.659575+0000","last_undegraded":"2024-09-14T06:32:13.659575+0000","last_fullsized":"2024-09-14T06:32:13.659575+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:28:50.205350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:28:50.205350+0000","last_clean_scrub_stamp":"2024-09-14T06:28:50.205350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T10:16:40.480031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065018200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"207'753","reported_seq":4127,"reported_epoch":210,"state":"active+clean","last_fresh":"2024-09-14T06:32:15.610254+0000","last_change":"2024-09-14T06:32:13.662724+0000","last_active":"2024-09-14T06:32:15.610254+0000","last_peered":"2024-09-14T06:32:15.610254+0000","last_clean":"2024-09-14T06:32:15.610254+0000","last_became_active":"2024-09-14T06:31:54.747923+0000","last_became_peered":"2024-09-14T06:31:54.747923+0000","last_unstale":"2024-09-14T06:32:15.610254+0000","last_undegraded":"2024-09-14T06:32:15.610254+0000","last_fullsized":"2024-09-14T06:32:15.610254+0000","mapping_epoch":189,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":190,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":753,"log_dups_size":0,"ondisk_log_size":753,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T13:55:37.796703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00044561900000000002,"stat_sum":{"num_bytes":38285856,"num_objects":51,"num_object_clones":0,"num_object_copies":102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":51,"num_whiteouts":0,"num_read":4380,"num_read_kb":40951,"num_write":1098,"num_write_kb":36111,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":183,"num_bytes_recovered":13912513,"num_keys_recovered":22,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"55","length":"1"},{"start":"57","length":"1"},{"start":"59","length":"1"},{"start":"5b","length":"1"}]},{"pgid":"2.5","version":"0'0","reported_seq":393,"reported_epoch":208,"state":"active+clean","last_fresh":"2024-09-14T06:32:13.663731+0000","last_change":"2024-09-14T06:28:53.640035+0000","last_active":"2024-09-14T06:32:13.663731+0000","last_peered":"2024-09-14T06:32:13.663731+0000","last_clean":"2024-09-14T06:32:13.663731+0000","last_became_active":"2024-09-14T06:28:51.308058+0000","last_became_peered":"2024-09-14T06:28:51.308058+0000","last_unstale":"2024-09-14T06:32:13.663731+0000","last_undegraded":"2024-09-14T06:32:13.663731+0000","last_fullsized":"2024-09-14T06:32:13.663731+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:28:50.205350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:28:50.205350+0000","last_clean_scrub_stamp":"2024-09-14T06:28:50.205350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T13:35:48.544057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00037824000000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"207'613","reported_seq":3312,"reported_epoch":207,"state":"peering","last_fresh":"2024-09-14T06:32:12.756338+0000","last_change":"2024-09-14T06:32:13.659971+0000","last_active":"2024-09-14T06:32:13.659971+0000","last_peered":"2024-09-14T06:32:12.756338+0000","last_clean":"2024-09-14T06:32:12.756338+0000","last_became_active":"2024-09-14T06:31:41.604731+0000","last_became_peered":"2024-09-14T06:31:41.604731+0000","last_unstale":"2024-09-14T06:32:12.756338+0000","last_undegraded":"2024-09-14T06:32:12.756338+0000","last_fullsized":"2024-09-14T06:32:12.756338+0000","mapping_epoch":208,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":613,"log_dups_size":0,"ondisk_log_size":613,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00058330999999999997,"stat_sum":{"num_bytes":37139964,"num_objects":64,"num_object_clones":0,"num_object_copies":128,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":2589,"num_read_kb":37386,"num_write":807,"num_write_kb":40850,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":153,"num_bytes_recovered":15692987,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2,5],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"57","length":"1"},{"start":"59","length":"1"},{"start":"5b","length":"2"}]},{"pgid":"2.4","version":"0'0","reported_seq":393,"reported_epoch":208,"state":"active+clean","last_fresh":"2024-09-14T06:32:13.659521+0000","last_change":"2024-09-14T06:28:53.226072+0000","last_active":"2024-09-14T06:32:13.659521+0000","last_peered":"2024-09-14T06:32:13.659521+0000","last_clean":"2024-09-14T06:32:13.659521+0000","last_became_active":"2024-09-14T06:28:51.308072+0000","last_became_peered":"2024-09-14T06:28:51.308072+0000","last_unstale":"2024-09-14T06:32:13.659521+0000","last_undegraded":"2024-09-14T06:32:13.659521+0000","last_fullsized":"2024-09-14T06:32:13.659521+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:28:50.205350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:28:50.205350+0000","last_clean_scrub_stamp":"2024-09-14T06:28:50.205350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T13:15:09.052328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066197399999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"208'827","reported_seq":5247,"reported_epoch":208,"state":"active+clean","last_fresh":"2024-09-14T06:32:13.749810+0000","last_change":"2024-09-14T06:32:13.662584+0000","last_active":"2024-09-14T06:32:13.749810+0000","last_peered":"2024-09-14T06:32:13.749810+0000","last_clean":"2024-09-14T06:32:13.749810+0000","last_became_active":"2024-09-14T06:31:54.744373+0000","last_became_peered":"2024-09-14T06:31:54.744373+0000","last_unstale":"2024-09-14T06:32:13.749810+0000","last_undegraded":"2024-09-14T06:32:13.749810+0000","last_fullsized":"2024-09-14T06:32:13.749810+0000","mapping_epoch":189,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":190,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":827,"log_dups_size":0,"ondisk_log_size":827,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T10:48:07.580270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041746399999999999,"stat_sum":{"num_bytes":43262791,"num_objects":67,"num_object_clones":0,"num_object_copies":134,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":67,"num_whiteouts":0,"num_read":4507,"num_read_kb":42629,"num_write":1188,"num_write_kb":49494,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":330,"num_bytes_recovered":28798893,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"55","length":"1"},{"start":"57","length":"1"},{"start":"59","length":"1"},{"start":"5b","length":"2"}]},{"pgid":"3.3","version":"208'468","reported_seq":10079,"reported_epoch":208,"state":"active+clean","last_fresh":"2024-09-14T06:32:13.745191+0000","last_change":"2024-09-14T06:32:13.661140+0000","last_active":"2024-09-14T06:32:13.745191+0000","last_peered":"2024-09-14T06:32:13.745191+0000","last_clean":"2024-09-14T06:32:13.745191+0000","last_became_active":"2024-09-14T06:29:00.584678+0000","last_became_peered":"2024-09-14T06:29:00.584678+0000","last_unstale":"2024-09-14T06:32:13.745191+0000","last_undegraded":"2024-09-14T06:32:13.745191+0000","last_fullsized":"2024-09-14T06:32:13.745191+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":468,"log_dups_size":0,"ondisk_log_size":468,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T13:20:16.770689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00082891200000000001,"stat_sum":{"num_bytes":62895515,"num_objects":56,"num_object_clones":0,"num_object_copies":112,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":56,"num_whiteouts":0,"num_read":9151,"num_read_kb":88923,"num_write":541,"num_write_kb":51562,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"55","length":"1"},{"start":"57","length":"1"},{"start":"59","length":"1"},{"start":"5b","length":"2"}]},{"pgid":"2.2","version":"22'2","reported_seq":395,"reported_epoch":208,"state":"active+clean","last_fresh":"2024-09-14T06:32:13.661683+0000","last_change":"2024-09-14T06:28:53.231428+0000","last_active":"2024-09-14T06:32:13.661683+0000","last_peered":"2024-09-14T06:32:13.661683+0000","last_clean":"2024-09-14T06:32:13.661683+0000","last_became_active":"2024-09-14T06:28:51.222937+0000","last_became_peered":"2024-09-14T06:28:51.222937+0000","last_unstale":"2024-09-14T06:32:13.661683+0000","last_undegraded":"2024-09-14T06:32:13.661683+0000","last_fullsized":"2024-09-14T06:32:13.661683+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:28:50.205350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:28:50.205350+0000","last_clean_scrub_stamp":"2024-09-14T06:28:50.205350+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T14:27:37.233976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010072970000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.0","version":"208'994","reported_seq":6568,"reported_epoch":208,"state":"active+clean","last_fresh":"2024-09-14T06:32:13.659853+0000","last_change":"2024-09-14T06:32:13.659853+0000","last_active":"2024-09-14T06:32:13.659853+0000","last_peered":"2024-09-14T06:32:13.659853+0000","last_clean":"2024-09-14T06:32:13.659853+0000","last_became_active":"2024-09-14T06:29:00.574873+0000","last_became_peered":"2024-09-14T06:29:00.574873+0000","last_unstale":"2024-09-14T06:32:13.659853+0000","last_undegraded":"2024-09-14T06:32:13.659853+0000","last_fullsized":"2024-09-14T06:32:13.659853+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":994,"log_dups_size":0,"ondisk_log_size":994,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T13:32:24.881003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00058771500000000005,"stat_sum":{"num_bytes":74732015,"num_objects":77,"num_object_clones":0,"num_object_copies":154,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":77,"num_whiteouts":0,"num_read":8014,"num_read_kb":85277,"num_write":1533,"num_write_kb":58684,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"55","length":"1"},{"start":"57","length":"1"},{"start":"59","length":"1"},{"start":"5b","length":"2"}]},{"pgid":"2.1","version":"0'0","reported_seq":17,"reported_epoch":212,"state":"active+clean","last_fresh":"2024-09-14T06:32:17.674606+0000","last_change":"2024-09-14T06:32:14.665376+0000","last_active":"2024-09-14T06:32:17.674606+0000","last_peered":"2024-09-14T06:32:17.674606+0000","last_clean":"2024-09-14T06:32:17.674606+0000","last_became_active":"2024-09-14T06:32:14.665081+0000","last_became_peered":"2024-09-14T06:32:14.665081+0000","last_unstale":"2024-09-14T06:32:17.674606+0000","last_undegraded":"2024-09-14T06:32:17.674606+0000","last_fullsized":"2024-09-14T06:32:17.674606+0000","mapping_epoch":208,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":209,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:28:50.205350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:28:50.205350+0000","last_clean_scrub_stamp":"2024-09-14T06:28:50.205350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T16:35:12.869944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053403100000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1","version":"208'696","reported_seq":11325,"reported_epoch":212,"state":"active+clean","last_fresh":"2024-09-14T06:32:17.687869+0000","last_change":"2024-09-14T06:32:16.673919+0000","last_active":"2024-09-14T06:32:17.687869+0000","last_peered":"2024-09-14T06:32:17.687869+0000","last_clean":"2024-09-14T06:32:17.687869+0000","last_became_active":"2024-09-14T06:30:57.768238+0000","last_became_peered":"2024-09-14T06:30:57.768238+0000","last_unstale":"2024-09-14T06:32:17.687869+0000","last_undegraded":"2024-09-14T06:32:17.687869+0000","last_fullsized":"2024-09-14T06:32:17.687869+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":696,"log_dups_size":0,"ondisk_log_size":696,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T09:26:24.780770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001653273,"stat_sum":{"num_bytes":93623859,"num_objects":65,"num_object_clones":0,"num_object_copies":130,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":65,"num_whiteouts":0,"num_read":11016,"num_read_kb":112206,"num_write":871,"num_write_kb":47387,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":183,"num_bytes_recovered":30431994,"num_keys_recovered":22,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"53","length":"1"},{"start":"55","length":"1"},{"start":"57","length":"1"},{"start":"59","length":"1"},{"start":"5b","length":"1"}]},{"pgid":"2.0","version":"0'0","reported_seq":393,"reported_epoch":208,"state":"active+clean","last_fresh":"2024-09-14T06:32:13.662840+0000","last_change":"2024-09-14T06:28:53.640096+0000","last_active":"2024-09-14T06:32:13.662840+0000","last_peered":"2024-09-14T06:32:13.662840+0000","last_clean":"2024-09-14T06:32:13.662840+0000","last_became_active":"2024-09-14T06:28:51.223069+0000","last_became_peered":"2024-09-14T06:28:51.223069+0000","last_unstale":"2024-09-14T06:32:13.662840+0000","last_undegraded":"2024-09-14T06:32:13.662840+0000","last_fullsized":"2024-09-14T06:32:13.662840+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:28:50.205350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:28:50.205350+0000","last_clean_scrub_stamp":"2024-09-14T06:28:50.205350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T15:13:00.471509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00043714600000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"210'1558","reported_seq":7376,"reported_epoch":210,"state":"active+clean","last_fresh":"2024-09-14T06:32:15.640212+0000","last_change":"2024-09-14T06:32:13.662240+0000","last_active":"2024-09-14T06:32:15.640212+0000","last_peered":"2024-09-14T06:32:15.640212+0000","last_clean":"2024-09-14T06:32:15.640212+0000","last_became_active":"2024-09-14T06:31:54.744160+0000","last_became_peered":"2024-09-14T06:31:54.744160+0000","last_unstale":"2024-09-14T06:32:15.640212+0000","last_undegraded":"2024-09-14T06:32:15.640212+0000","last_fullsized":"2024-09-14T06:32:15.640212+0000","mapping_epoch":189,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":190,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":1558,"log_dups_size":0,"ondisk_log_size":1558,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T12:41:51.808698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00055251900000000001,"stat_sum":{"num_bytes":66459400,"num_objects":80,"num_object_clones":0,"num_object_copies":160,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":80,"num_whiteouts":0,"num_read":10653,"num_read_kb":70538,"num_write":2490,"num_write_kb":53554,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":327,"num_bytes_recovered":50047063,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"55","length":"1"},{"start":"57","length":"1"},{"start":"59","length":"1"},{"start":"5b","length":"1"}]},{"pgid":"2.3","version":"20'1","reported_seq":394,"reported_epoch":208,"state":"active+clean","last_fresh":"2024-09-14T06:32:13.661658+0000","last_change":"2024-09-14T06:28:53.230933+0000","last_active":"2024-09-14T06:32:13.661658+0000","last_peered":"2024-09-14T06:32:13.661658+0000","last_clean":"2024-09-14T06:32:13.661658+0000","last_became_active":"2024-09-14T06:28:51.222874+0000","last_became_peered":"2024-09-14T06:28:51.222874+0000","last_unstale":"2024-09-14T06:32:13.661658+0000","last_undegraded":"2024-09-14T06:32:13.661658+0000","last_fullsized":"2024-09-14T06:32:13.661658+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:28:50.205350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:28:50.205350+0000","last_clean_scrub_stamp":"2024-09-14T06:28:50.205350+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T12:04:25.190171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050808899999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'72","reported_seq":484,"reported_epoch":208,"state":"active+clean","last_fresh":"2024-09-14T06:32:13.662987+0000","last_change":"2024-09-14T06:28:44.839885+0000","last_active":"2024-09-14T06:32:13.662987+0000","last_peered":"2024-09-14T06:32:13.662987+0000","last_clean":"2024-09-14T06:32:13.662987+0000","last_became_active":"2024-09-14T06:28:44.839573+0000","last_became_peered":"2024-09-14T06:28:44.839573+0000","last_unstale":"2024-09-14T06:32:13.662987+0000","last_undegraded":"2024-09-14T06:32:13.662987+0000","last_fullsized":"2024-09-14T06:32:13.662987+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:28:43.828494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:28:43.828494+0000","last_clean_scrub_stamp":"2024-09-14T06:28:43.828494+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T14:41:25.787293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.8","version":"208'586","reported_seq":6123,"reported_epoch":210,"state":"active+clean","last_fresh":"2024-09-14T06:32:15.610718+0000","last_change":"2024-09-14T06:32:13.662649+0000","last_active":"2024-09-14T06:32:15.610718+0000","last_peered":"2024-09-14T06:32:15.610718+0000","last_clean":"2024-09-14T06:32:15.610718+0000","last_became_active":"2024-09-14T06:31:55.754694+0000","last_became_peered":"2024-09-14T06:31:55.754694+0000","last_unstale":"2024-09-14T06:32:15.610718+0000","last_undegraded":"2024-09-14T06:32:15.610718+0000","last_fullsized":"2024-09-14T06:32:15.610718+0000","mapping_epoch":190,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":191,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":586,"log_dups_size":0,"ondisk_log_size":586,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T12:30:44.816178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097722299999999994,"stat_sum":{"num_bytes":78517473,"num_objects":71,"num_object_clones":0,"num_object_copies":142,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":71,"num_whiteouts":0,"num_read":5608,"num_read_kb":90696,"num_write":834,"num_write_kb":57685,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":319,"num_bytes_recovered":60352269,"num_keys_recovered":22,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"55","length":"1"},{"start":"57","length":"1"},{"start":"59","length":"1"},{"start":"5b","length":"1"}]},{"pgid":"3.9","version":"208'516","reported_seq":4063,"reported_epoch":208,"state":"active+clean","last_fresh":"2024-09-14T06:32:13.660676+0000","last_change":"2024-09-14T06:32:13.660676+0000","last_active":"2024-09-14T06:32:13.660676+0000","last_peered":"2024-09-14T06:32:13.660676+0000","last_clean":"2024-09-14T06:32:13.660676+0000","last_became_active":"2024-09-14T06:29:00.573179+0000","last_became_peered":"2024-09-14T06:29:00.573179+0000","last_unstale":"2024-09-14T06:32:13.660676+0000","last_undegraded":"2024-09-14T06:32:13.660676+0000","last_fullsized":"2024-09-14T06:32:13.660676+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":516,"log_dups_size":0,"ondisk_log_size":516,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T18:24:55.803054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054865399999999998,"stat_sum":{"num_bytes":42231296,"num_objects":66,"num_object_clones":0,"num_object_copies":132,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":66,"num_whiteouts":0,"num_read":3634,"num_read_kb":64262,"num_write":699,"num_write_kb":44923,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"55","length":"1"},{"start":"57","length":"1"},{"start":"59","length":"1"},{"start":"5b","length":"2"}]},{"pgid":"3.a","version":"208'738","reported_seq":4416,"reported_epoch":208,"state":"active+clean","last_fresh":"2024-09-14T06:32:13.757393+0000","last_change":"2024-09-14T06:32:13.661025+0000","last_active":"2024-09-14T06:32:13.757393+0000","last_peered":"2024-09-14T06:32:13.757393+0000","last_clean":"2024-09-14T06:32:13.757393+0000","last_became_active":"2024-09-14T06:32:04.617283+0000","last_became_peered":"2024-09-14T06:32:04.617283+0000","last_unstale":"2024-09-14T06:32:13.757393+0000","last_undegraded":"2024-09-14T06:32:13.757393+0000","last_fullsized":"2024-09-14T06:32:13.757393+0000","mapping_epoch":199,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":200,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":738,"log_dups_size":0,"ondisk_log_size":738,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T15:40:09.086871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070773799999999997,"stat_sum":{"num_bytes":46401995,"num_objects":67,"num_object_clones":0,"num_object_copies":134,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":67,"num_whiteouts":0,"num_read":4807,"num_read_kb":57874,"num_write":1106,"num_write_kb":47517,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":161,"num_bytes_recovered":21723371,"num_keys_recovered":33,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"55","length":"1"},{"start":"57","length":"1"},{"start":"59","length":"1"},{"start":"5b","length":"2"}]},{"pgid":"3.b","version":"210'1477","reported_seq":8243,"reported_epoch":210,"state":"active+clean","last_fresh":"2024-09-14T06:32:15.683852+0000","last_change":"2024-09-14T06:32:13.662503+0000","last_active":"2024-09-14T06:32:15.683852+0000","last_peered":"2024-09-14T06:32:15.683852+0000","last_clean":"2024-09-14T06:32:15.683852+0000","last_became_active":"2024-09-14T06:31:55.756219+0000","last_became_peered":"2024-09-14T06:31:55.756219+0000","last_unstale":"2024-09-14T06:32:15.683852+0000","last_undegraded":"2024-09-14T06:32:15.683852+0000","last_fullsized":"2024-09-14T06:32:15.683852+0000","mapping_epoch":190,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":191,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":1477,"log_dups_size":0,"ondisk_log_size":1477,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T08:46:05.977011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000634164,"stat_sum":{"num_bytes":58790990,"num_objects":73,"num_object_clones":0,"num_object_copies":146,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":73,"num_whiteouts":0,"num_read":12454,"num_read_kb":63624,"num_write":2349,"num_write_kb":43651,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":291,"num_bytes_recovered":37370632,"num_keys_recovered":66,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"55","length":"1"},{"start":"57","length":"1"},{"start":"59","length":"1"},{"start":"5b","length":"1"}]},{"pgid":"3.c","version":"208'547","reported_seq":5890,"reported_epoch":208,"state":"active+clean","last_fresh":"2024-09-14T06:32:13.736039+0000","last_change":"2024-09-14T06:32:13.662856+0000","last_active":"2024-09-14T06:32:13.736039+0000","last_peered":"2024-09-14T06:32:13.736039+0000","last_clean":"2024-09-14T06:32:13.736039+0000","last_became_active":"2024-09-14T06:31:53.747355+0000","last_became_peered":"2024-09-14T06:31:53.747355+0000","last_unstale":"2024-09-14T06:32:13.736039+0000","last_undegraded":"2024-09-14T06:32:13.736039+0000","last_fullsized":"2024-09-14T06:32:13.736039+0000","mapping_epoch":188,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":189,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":547,"log_dups_size":0,"ondisk_log_size":547,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T16:05:49.827036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00037372000000000002,"stat_sum":{"num_bytes":51667677,"num_objects":60,"num_object_clones":0,"num_object_copies":120,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":4805,"num_read_kb":50398,"num_write":722,"num_write_kb":40822,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":293,"num_bytes_recovered":36712421,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"55","length":"1"},{"start":"57","length":"1"},{"start":"59","length":"1"},{"start":"5b","length":"2"}]},{"pgid":"3.d","version":"208'482","reported_seq":8794,"reported_epoch":208,"state":"active+clean","last_fresh":"2024-09-14T06:32:13.745059+0000","last_change":"2024-09-14T06:32:13.664738+0000","last_active":"2024-09-14T06:32:13.745059+0000","last_peered":"2024-09-14T06:32:13.745059+0000","last_clean":"2024-09-14T06:32:13.745059+0000","last_became_active":"2024-09-14T06:29:00.580362+0000","last_became_peered":"2024-09-14T06:29:00.580362+0000","last_unstale":"2024-09-14T06:32:13.745059+0000","last_undegraded":"2024-09-14T06:32:13.745059+0000","last_fullsized":"2024-09-14T06:32:13.745059+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":482,"log_dups_size":0,"ondisk_log_size":482,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T10:55:42.483074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010157390000000001,"stat_sum":{"num_bytes":95841531,"num_objects":72,"num_object_clones":0,"num_object_copies":144,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":72,"num_whiteouts":0,"num_read":8023,"num_read_kb":93542,"num_write":644,"num_write_kb":41026,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"55","length":"1"},{"start":"57","length":"1"},{"start":"59","length":"1"},{"start":"5b","length":"2"}]},{"pgid":"3.e","version":"207'520","reported_seq":4739,"reported_epoch":208,"state":"active+clean","last_fresh":"2024-09-14T06:32:13.663598+0000","last_change":"2024-09-14T06:32:13.663783+0000","last_active":"2024-09-14T06:32:13.663598+0000","last_peered":"2024-09-14T06:32:13.663598+0000","last_clean":"2024-09-14T06:32:13.663598+0000","last_became_active":"2024-09-14T06:29:00.586374+0000","last_became_peered":"2024-09-14T06:29:00.586374+0000","last_unstale":"2024-09-14T06:32:13.663598+0000","last_undegraded":"2024-09-14T06:32:13.663598+0000","last_fullsized":"2024-09-14T06:32:13.663598+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":520,"log_dups_size":0,"ondisk_log_size":520,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T16:01:48.654194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053448299999999999,"stat_sum":{"num_bytes":48343618,"num_objects":63,"num_object_clones":0,"num_object_copies":126,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":63,"num_whiteouts":0,"num_read":3977,"num_read_kb":61134,"num_write":690,"num_write_kb":43787,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"55","length":"1"},{"start":"57","length":"1"},{"start":"59","length":"1"},{"start":"5b","length":"2"}]},{"pgid":"3.f","version":"208'619","reported_seq":5340,"reported_epoch":208,"state":"active+clean","last_fresh":"2024-09-14T06:32:13.755426+0000","last_change":"2024-09-14T06:32:13.663765+0000","last_active":"2024-09-14T06:32:13.755426+0000","last_peered":"2024-09-14T06:32:13.755426+0000","last_clean":"2024-09-14T06:32:13.755426+0000","last_became_active":"2024-09-14T06:29:00.586491+0000","last_became_peered":"2024-09-14T06:29:00.586491+0000","last_unstale":"2024-09-14T06:32:13.755426+0000","last_undegraded":"2024-09-14T06:32:13.755426+0000","last_fullsized":"2024-09-14T06:32:13.755426+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":619,"log_dups_size":0,"ondisk_log_size":619,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T12:18:46.267108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057572199999999997,"stat_sum":{"num_bytes":29756210,"num_objects":49,"num_object_clones":0,"num_object_copies":98,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":49,"num_whiteouts":0,"num_read":5724,"num_read_kb":45272,"num_write":755,"num_write_kb":36122,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"55","length":"1"},{"start":"57","length":"1"},{"start":"59","length":"1"},{"start":"5b","length":"2"}]},{"pgid":"3.10","version":"211'826","reported_seq":6189,"reported_epoch":212,"state":"active+clean","last_fresh":"2024-09-14T06:32:17.688002+0000","last_change":"2024-09-14T06:32:16.673350+0000","last_active":"2024-09-14T06:32:17.688002+0000","last_peered":"2024-09-14T06:32:17.688002+0000","last_clean":"2024-09-14T06:32:17.688002+0000","last_became_active":"2024-09-14T06:32:04.621617+0000","last_became_peered":"2024-09-14T06:32:04.621617+0000","last_unstale":"2024-09-14T06:32:17.688002+0000","last_undegraded":"2024-09-14T06:32:17.688002+0000","last_fullsized":"2024-09-14T06:32:17.688002+0000","mapping_epoch":199,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":200,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":826,"log_dups_size":0,"ondisk_log_size":826,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T14:22:36.187885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001412967,"stat_sum":{"num_bytes":37504683,"num_objects":60,"num_object_clones":0,"num_object_copies":120,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":6618,"num_read_kb":34017,"num_write":974,"num_write_kb":36637,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":154,"num_bytes_recovered":9401812,"num_keys_recovered":45,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"53","length":"1"},{"start":"55","length":"1"},{"start":"57","length":"1"},{"start":"59","length":"1"},{"start":"5b","length":"1"}]},{"pgid":"3.11","version":"208'619","reported_seq":10896,"reported_epoch":208,"state":"active+clean","last_fresh":"2024-09-14T06:32:13.663466+0000","last_change":"2024-09-14T06:32:13.663466+0000","last_active":"2024-09-14T06:32:13.663466+0000","last_peered":"2024-09-14T06:32:13.663466+0000","last_clean":"2024-09-14T06:32:13.663466+0000","last_became_active":"2024-09-14T06:29:00.586480+0000","last_became_peered":"2024-09-14T06:29:00.586480+0000","last_unstale":"2024-09-14T06:32:13.663466+0000","last_undegraded":"2024-09-14T06:32:13.663466+0000","last_fullsized":"2024-09-14T06:32:13.663466+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":619,"log_dups_size":0,"ondisk_log_size":619,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T15:11:50.174862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00055159800000000004,"stat_sum":{"num_bytes":103911764,"num_objects":74,"num_object_clones":0,"num_object_copies":148,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":74,"num_whiteouts":0,"num_read":10178,"num_read_kb":109171,"num_write":746,"num_write_kb":48292,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"55","length":"1"},{"start":"57","length":"1"},{"start":"59","length":"1"},{"start":"5b","length":"2"}]},{"pgid":"3.12","version":"211'656","reported_seq":6945,"reported_epoch":212,"state":"active+clean","last_fresh":"2024-09-14T06:32:17.687957+0000","last_change":"2024-09-14T06:32:16.673655+0000","last_active":"2024-09-14T06:32:17.687957+0000","last_peered":"2024-09-14T06:32:17.687957+0000","last_clean":"2024-09-14T06:32:17.687957+0000","last_became_active":"2024-09-14T06:29:00.579826+0000","last_became_peered":"2024-09-14T06:29:00.579826+0000","last_unstale":"2024-09-14T06:32:17.687957+0000","last_undegraded":"2024-09-14T06:32:17.687957+0000","last_fullsized":"2024-09-14T06:32:17.687957+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":656,"log_dups_size":0,"ondisk_log_size":656,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T16:19:03.689356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010101960000000001,"stat_sum":{"num_bytes":41920978,"num_objects":70,"num_object_clones":0,"num_object_copies":140,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":70,"num_whiteouts":0,"num_read":6842,"num_read_kb":58865,"num_write":822,"num_write_kb":42420,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"53","length":"1"},{"start":"55","length":"1"},{"start":"57","length":"1"},{"start":"59","length":"1"},{"start":"5b","length":"1"}]},{"pgid":"3.13","version":"208'877","reported_seq":8244,"reported_epoch":208,"state":"active+clean","last_fresh":"2024-09-14T06:32:13.742462+0000","last_change":"2024-09-14T06:32:13.663533+0000","last_active":"2024-09-14T06:32:13.742462+0000","last_peered":"2024-09-14T06:32:13.742462+0000","last_clean":"2024-09-14T06:32:13.742462+0000","last_became_active":"2024-09-14T06:29:00.586520+0000","last_became_peered":"2024-09-14T06:29:00.586520+0000","last_unstale":"2024-09-14T06:32:13.742462+0000","last_undegraded":"2024-09-14T06:32:13.742462+0000","last_fullsized":"2024-09-14T06:32:13.742462+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":877,"log_dups_size":0,"ondisk_log_size":877,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T18:07:27.711675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064785299999999999,"stat_sum":{"num_bytes":85870266,"num_objects":63,"num_object_clones":0,"num_object_copies":126,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":63,"num_whiteouts":0,"num_read":9027,"num_read_kb":65664,"num_write":1124,"num_write_kb":47419,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"55","length":"1"},{"start":"57","length":"1"},{"start":"59","length":"1"},{"start":"5b","length":"2"}]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":2000229116,"num_objects":2123,"num_object_clones":0,"num_object_copies":4246,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":156,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2123,"num_whiteouts":0,"num_read":230912,"num_read_kb":2177049,"num_write":34594,"num_write_kb":1477817,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4133,"num_bytes_recovered":555504806,"num_keys_recovered":466,"num_objects_omap":44,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":136265728,"data_stored":1348808628,"data_compressed":83798035,"data_compressed_allocated":111538176,"data_compressed_original":1329358950,"omap_allocated":0,"internal_metadata":0},"log_size":24860,"ondisk_log_size":24860,"up":64,"acting":63,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":131072,"data_stored":1180736,"data_compressed":65166,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509485,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":76548,"kb_used_data":18836,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94295292,"statfs":{"total":96636764160,"available":96558379008,"internally_reserved":0,"allocated":19288064,"data_stored":161499942,"data_compressed":9753290,"data_compressed_allocated":13131776,"data_compressed_original":157359019,"omap_allocated":1590,"internal_metadata":58915274},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":0,"seq":64424509484,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509486,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":86008,"kb_used_data":22828,"kb_used_omap":1,"kb_used_meta":63102,"kb_avail":94285832,"statfs":{"total":96636764160,"available":96548691968,"internally_reserved":0,"allocated":23375872,"data_stored":202683279,"data_compressed":12507586,"data_compressed_allocated":16994304,"data_compressed_original":198429030,"omap_allocated":1595,"internal_metadata":64616901},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509485,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":99176,"kb_used_data":27652,"kb_used_omap":1,"kb_used_meta":71422,"kb_avail":94272664,"statfs":{"total":96636764160,"available":96535207936,"internally_reserved":0,"allocated":28315648,"data_stored":256783818,"data_compressed":15993453,"data_compressed_allocated":21262336,"data_compressed_original":252034110,"omap_allocated":1595,"internal_metadata":73136581},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509486,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":80772,"kb_used_data":19056,"kb_used_omap":1,"kb_used_meta":61694,"kb_avail":94291068,"statfs":{"total":96636764160,"available":96554053632,"internally_reserved":0,"allocated":19513344,"data_stored":170274174,"data_compressed":10352043,"data_compressed_allocated":13901824,"data_compressed_original":166626935,"omap_allocated":1595,"internal_metadata":63175109},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":14,"seq":60129542190,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37712,"kb_used_data":10684,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94334128,"statfs":{"total":96636764160,"available":96598147072,"internally_reserved":0,"allocated":10940416,"data_stored":88882747,"data_compressed":5419630,"data_compressed_allocated":7172096,"data_compressed_original":86718577,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542189,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":95044,"kb_used_data":23596,"kb_used_omap":1,"kb_used_meta":71422,"kb_avail":94276796,"statfs":{"total":96636764160,"available":96539439104,"internally_reserved":0,"allocated":24162304,"data_stored":212986788,"data_compressed":13260458,"data_compressed_allocated":17498112,"data_compressed_original":208405851,"omap_allocated":1595,"internal_metadata":73136581},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574894,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":94156,"kb_used_data":22708,"kb_used_omap":1,"kb_used_meta":71422,"kb_avail":94277684,"statfs":{"total":96636764160,"available":96540348416,"internally_reserved":0,"allocated":23252992,"data_stored":207666606,"data_compressed":13129610,"data_compressed_allocated":17367040,"data_compressed_original":203771904,"omap_allocated":1593,"internal_metadata":73136583},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32583,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32583,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":21553152,"data_stored":211156003,"data_compressed":13227875,"data_compressed_allocated":17453056,"data_compressed_original":207832411,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":16781312,"data_stored":167924327,"data_compressed":10246884,"data_compressed_allocated":13713408,"data_compressed_original":165521015,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8376320,"data_stored":87510993,"data_compressed":5413418,"data_compressed_allocated":7155712,"data_compressed_original":86587505,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":20717568,"data_stored":206432339,"data_compressed":13129610,"data_compressed_allocated":17367040,"data_compressed_original":203771904,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":25223168,"data_stored":252662737,"data_compressed":15819805,"data_compressed_allocated":20889600,"data_compressed_original":249326654,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":20480000,"data_stored":199416160,"data_compressed":12404973,"data_compressed_allocated":16728064,"data_compressed_original":196491622,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":6631424,"data_stored":65369250,"data_compressed":3936333,"data_compressed_allocated":5361664,"data_compressed_original":64393940,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16502784,"data_stored":158336819,"data_compressed":9619137,"data_compressed_allocated":12869632,"data_compressed_original":155433899,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-14T06:32:19.254 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-14T06:32:19.295 INFO:tasks.ceph.osd.3.smithi094.stderr:2024-09-14T06:32:19.294+0000 7feeeafe8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33655) UID: 0 2024-09-14T06:32:19.377 INFO:tasks.ceph.osd.7.smithi138.stderr:2024-09-14T06:32:19.376+0000 7f3c6a21d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33623) UID: 0 2024-09-14T06:32:19.396 INFO:teuthology.orchestra.run.smithi094.stdout:ERROR: (22) Invalid argument 2024-09-14T06:32:19.397 INFO:teuthology.orchestra.run.smithi094.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:32:19.405 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-14T06:32:19.478 INFO:tasks.ceph.osd.4.smithi138.stderr:2024-09-14T06:32:19.477+0000 7f6c653c8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33615) UID: 0 2024-09-14T06:32:19.508 INFO:teuthology.orchestra.run.smithi094.stdout:ERROR: (22) Invalid argument 2024-09-14T06:32:19.508 INFO:teuthology.orchestra.run.smithi094.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:32:19.517 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-14T06:32:19.579 INFO:tasks.ceph.osd.5.smithi138.stderr:2024-09-14T06:32:19.578+0000 7fe13746f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33618) UID: 0 2024-09-14T06:32:19.619 INFO:teuthology.orchestra.run.smithi094.stdout:ERROR: (22) Invalid argument 2024-09-14T06:32:19.619 INFO:teuthology.orchestra.run.smithi094.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:32:19.628 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-14T06:32:19.681 INFO:tasks.ceph.osd.6.smithi138.stderr:2024-09-14T06:32:19.680+0000 7f8cd8849640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33620) UID: 0 2024-09-14T06:32:19.741 INFO:teuthology.orchestra.run.smithi094.stdout:ERROR: (22) Invalid argument 2024-09-14T06:32:19.741 INFO:teuthology.orchestra.run.smithi094.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:32:19.751 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-14T06:32:19.796 INFO:tasks.ceph.osd.5.smithi138.stderr:2024-09-14T06:32:19.795+0000 7fe13746f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33618) UID: 0 2024-09-14T06:32:19.860 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-14T06:32:19.865 INFO:teuthology.orchestra.run.smithi138.stdout:ERROR: (22) Invalid argument 2024-09-14T06:32:19.865 INFO:teuthology.orchestra.run.smithi138.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:32:19.874 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-14T06:32:19.904 INFO:tasks.ceph.osd.1.smithi094.stderr:2024-09-14T06:32:19.903+0000 7f328e47a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33641) UID: 0 2024-09-14T06:32:19.980 INFO:teuthology.orchestra.run.smithi094.stdout:osd.0: {} 2024-09-14T06:32:19.980 INFO:teuthology.orchestra.run.smithi094.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-14T06:32:19.983 INFO:tasks.ceph.osd.5.smithi138.stderr:2024-09-14T06:32:19.982+0000 7fe13746f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33618) UID: 0 2024-09-14T06:32:19.997 INFO:teuthology.orchestra.run.smithi094.stdout:osd.1: {} 2024-09-14T06:32:19.997 INFO:teuthology.orchestra.run.smithi094.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-14T06:32:20.012 INFO:teuthology.orchestra.run.smithi094.stdout:osd.2: {} 2024-09-14T06:32:20.013 INFO:teuthology.orchestra.run.smithi094.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-14T06:32:20.018 INFO:teuthology.orchestra.run.smithi138.stdout:ERROR: (22) Invalid argument 2024-09-14T06:32:20.018 INFO:teuthology.orchestra.run.smithi138.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:32:20.027 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-14T06:32:20.028 INFO:teuthology.orchestra.run.smithi094.stdout:osd.3: {} 2024-09-14T06:32:20.028 INFO:teuthology.orchestra.run.smithi094.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-14T06:32:20.044 INFO:teuthology.orchestra.run.smithi094.stdout:osd.4: {} 2024-09-14T06:32:20.045 INFO:teuthology.orchestra.run.smithi094.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-14T06:32:20.061 INFO:teuthology.orchestra.run.smithi094.stdout:osd.5: {} 2024-09-14T06:32:20.061 INFO:teuthology.orchestra.run.smithi094.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-14T06:32:20.077 INFO:teuthology.orchestra.run.smithi094.stdout:osd.6: {} 2024-09-14T06:32:20.077 INFO:teuthology.orchestra.run.smithi094.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-14T06:32:20.093 INFO:teuthology.orchestra.run.smithi094.stdout:osd.7: {} 2024-09-14T06:32:20.093 INFO:teuthology.orchestra.run.smithi094.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-14T06:32:20.111 INFO:tasks.ceph.osd.4.smithi138.stderr:2024-09-14T06:32:20.110+0000 7f6c653c8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33615) UID: 0 2024-09-14T06:32:20.172 INFO:teuthology.orchestra.run.smithi138.stdout:{ 2024-09-14T06:32:20.172 INFO:teuthology.orchestra.run.smithi138.stdout: "size": 20, 2024-09-14T06:32:20.172 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 600, 2024-09-14T06:32:20.172 INFO:teuthology.orchestra.run.smithi138.stdout: "ops": [ 2024-09-14T06:32:20.172 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:20.172 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:19328 3.3 3:c34e5e51:::rbd_data.112d92027590.0000000000000024:f [read 1926144~1024 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:32:20.172 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:27.059768+0000", 2024-09-14T06:32:20.172 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 173.11021267800001, 2024-09-14T06:32:20.172 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046369147999999999, 2024-09-14T06:32:20.172 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:32:20.172 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:32:20.173 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:32:20.173 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:32:20.173 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:32:20.173 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:20.173 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 19328 2024-09-14T06:32:20.173 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:20.173 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:32:20.173 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:20.173 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:32:20.173 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.059768+0000", 2024-09-14T06:32:20.173 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:20.173 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:20.173 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:20.173 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:32:20.174 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.059768+0000", 2024-09-14T06:32:20.174 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:20.174 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:20.174 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:20.174 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:32:20.174 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.059874+0000", 2024-09-14T06:32:20.174 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00010551400000000001 2024-09-14T06:32:20.174 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:20.174 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:20.174 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:32:20.174 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.059877+0000", 2024-09-14T06:32:20.174 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.8279999999999999e-06 2024-09-14T06:32:20.174 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:20.174 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:20.176 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:32:20.176 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.059884+0000", 2024-09-14T06:32:20.176 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.6389999999999992e-06 2024-09-14T06:32:20.176 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:20.176 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:20.176 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:32:20.176 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.059890+0000", 2024-09-14T06:32:20.176 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.49e-06 2024-09-14T06:32:20.176 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:20.176 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:20.177 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:32:20.177 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.105804+0000", 2024-09-14T06:32:20.177 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.045914242000000001 2024-09-14T06:32:20.177 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:20.177 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:20.177 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:32:20.177 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.105886+0000", 2024-09-14T06:32:20.177 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.2305999999999994e-05 2024-09-14T06:32:20.177 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:20.177 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:20.177 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:32:20.177 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.106138+0000", 2024-09-14T06:32:20.177 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.000251129 2024-09-14T06:32:20.177 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:20.178 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:32:20.178 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:20.178 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:20.178 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:20.178 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:19329 3.3 3:c34e5e51:::rbd_data.112d92027590.0000000000000024:f [read 1936384~1024 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:32:20.178 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:27.059899+0000", 2024-09-14T06:32:20.178 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 173.11008215499999, 2024-09-14T06:32:20.178 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046572122000000001, 2024-09-14T06:32:20.178 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:32:20.178 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:32:20.178 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:32:20.178 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:32:20.178 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:32:20.178 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:20.179 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 19329 2024-09-14T06:32:20.179 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:20.179 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:32:20.179 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:20.179 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:32:20.179 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.059899+0000", 2024-09-14T06:32:20.179 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:20.179 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:20.179 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:20.179 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:32:20.179 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.059899+0000", 2024-09-14T06:32:20.179 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:20.179 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:20.179 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:20.179 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:32:20.179 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062501+0000", 2024-09-14T06:32:20.180 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.0026016419999999999 2024-09-14T06:32:20.180 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:20.180 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:20.180 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:32:20.180 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062514+0000", 2024-09-14T06:32:20.180 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.2947e-05 2024-09-14T06:32:20.180 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:20.180 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:20.180 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:32:20.180 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062523+0000", 2024-09-14T06:32:20.180 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.5519999999999993e-06 2024-09-14T06:32:20.180 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:20.180 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:20.180 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:32:20.181 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062534+0000", 2024-09-14T06:32:20.181 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.1416e-05 2024-09-14T06:32:20.181 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:20.181 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:20.181 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:32:20.181 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.106179+0000", 2024-09-14T06:32:20.181 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.043644049999999997 2024-09-14T06:32:20.181 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:20.181 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:20.181 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:32:20.181 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.106262+0000", 2024-09-14T06:32:20.181 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.3178999999999998e-05 2024-09-14T06:32:20.181 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:20.182 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:20.182 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:32:20.182 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.106471+0000", 2024-09-14T06:32:20.182 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00020933599999999999 2024-09-14T06:32:20.182 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:20.182 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:32:20.182 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:20.182 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:20.182 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:20.182 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:19351 3.3 3:c34e5e51:::rbd_data.112d92027590.0000000000000024:f [read 1159168~1024 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:32:20.182 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:27.062766+0000", 2024-09-14T06:32:20.182 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 173.10721493, 2024-09-14T06:32:20.182 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046099454999999998, 2024-09-14T06:32:20.182 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:32:20.182 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:32:20.182 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:32:20.183 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:32:20.183 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:32:20.183 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:20.183 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 19351 2024-09-14T06:32:20.183 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:20.183 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:32:20.183 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:20.183 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:32:20.183 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062766+0000", 2024-09-14T06:32:20.183 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:20.183 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:20.183 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:20.183 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:32:20.183 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062766+0000", 2024-09-14T06:32:20.183 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:20.184 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:20.184 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:20.184 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:32:20.184 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062766+0000", 2024-09-14T06:32:20.184 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.6e-07 2024-09-14T06:32:20.184 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:20.184 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:20.184 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:32:20.184 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062767+0000", 2024-09-14T06:32:20.184 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.0070000000000001e-06 2024-09-14T06:32:20.184 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:20.184 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:20.184 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:32:20.184 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062785+0000", 2024-09-14T06:32:20.184 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.7884999999999999e-05 2024-09-14T06:32:20.185 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:20.185 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:20.185 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:32:20.185 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062790+0000", 2024-09-14T06:32:20.185 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.0900000000000004e-06 2024-09-14T06:32:20.185 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:20.185 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:20.185 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:32:20.185 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.108735+0000", 2024-09-14T06:32:20.185 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.045944411999999997 2024-09-14T06:32:20.185 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:20.185 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:20.185 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:32:20.185 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.108809+0000", 2024-09-14T06:32:20.186 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.4157000000000002e-05 2024-09-14T06:32:20.186 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:20.186 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:20.186 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:32:20.186 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.108866+0000", 2024-09-14T06:32:20.186 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.6644000000000002e-05 2024-09-14T06:32:20.186 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:20.186 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:32:20.186 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:20.186 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:20.186 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:20.186 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:19352 3.3 3:c34e5e51:::rbd_data.112d92027590.0000000000000024:f [read 1169408~1024 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:32:20.186 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:27.062798+0000", 2024-09-14T06:32:20.186 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 173.10718351899999, 2024-09-14T06:32:20.186 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046255900000000003, 2024-09-14T06:32:20.187 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:32:20.187 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:32:20.187 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:32:20.187 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:32:20.187 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:32:20.187 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:20.187 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 19352 2024-09-14T06:32:20.187 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:20.187 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:32:20.187 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:20.187 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:32:20.187 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062798+0000", 2024-09-14T06:32:20.187 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:20.187 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:20.187 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:20.188 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:32:20.188 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062798+0000", 2024-09-14T06:32:20.188 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:20.188 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:20.188 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:20.188 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:32:20.188 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062798+0000", 2024-09-14T06:32:20.188 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.37e-07 2024-09-14T06:32:20.188 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:20.188 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:20.188 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:32:20.188 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062800+0000", 2024-09-14T06:32:20.188 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.0269999999999998e-06 2024-09-14T06:32:20.189 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:20.189 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:20.189 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:32:20.189 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062806+0000", 2024-09-14T06:32:20.189 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.6690000000000003e-06 2024-09-14T06:32:20.189 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:20.189 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:20.189 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:32:20.189 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062810+0000", 2024-09-14T06:32:20.189 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.3449999999999999e-06 2024-09-14T06:32:20.189 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:20.189 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:20.189 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:32:20.190 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.108892+0000", 2024-09-14T06:32:20.190 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046081715000000002 2024-09-14T06:32:20.190 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:20.190 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:20.190 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:32:20.190 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.108940+0000", 2024-09-14T06:32:20.190 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.7815999999999998e-05 2024-09-14T06:32:20.190 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:20.190 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:20.190 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:32:20.191 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.109053+0000", 2024-09-14T06:32:20.191 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.000113791 2024-09-14T06:32:20.191 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:20.191 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:32:20.191 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:20.191 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:20.191 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:20.191 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:19353 3.3 3:c34e5e51:::rbd_data.112d92027590.0000000000000024:f [read 1179648~1024 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:32:20.191 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:27.062816+0000", 2024-09-14T06:32:20.191 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 173.10716500000001, 2024-09-14T06:32:20.191 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046481256999999998, 2024-09-14T06:32:20.191 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:32:20.191 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:32:20.191 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:32:20.192 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:32:20.192 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:32:20.192 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:20.192 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 19353 2024-09-14T06:32:20.192 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:20.192 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:32:20.192 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:20.192 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:32:20.192 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062816+0000", 2024-09-14T06:32:20.192 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:20.192 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:20.192 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:20.192 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:32:20.192 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062816+0000", 2024-09-14T06:32:20.192 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:20.193 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:20.193 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:20.193 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:32:20.193 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062816+0000", 2024-09-14T06:32:20.193 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.2899999999999999e-07 2024-09-14T06:32:20.193 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:20.193 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:20.193 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:32:20.193 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062818+0000", 2024-09-14T06:32:20.193 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.699e-06 2024-09-14T06:32:20.193 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:20.193 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:20.193 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:32:20.194 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062825+0000", 2024-09-14T06:32:20.194 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.878e-06 2024-09-14T06:32:20.194 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:20.194 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:20.194 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:32:20.194 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062830+0000", 2024-09-14T06:32:20.194 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.5410000000000002e-06 2024-09-14T06:32:20.194 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:20.194 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:20.194 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:32:20.194 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.109096+0000", 2024-09-14T06:32:20.194 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046266517 2024-09-14T06:32:20.194 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:20.194 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:20.194 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:32:20.195 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.109190+0000", 2024-09-14T06:32:20.195 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.4000999999999997e-05 2024-09-14T06:32:20.195 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:20.195 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:20.195 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:32:20.195 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.109297+0000", 2024-09-14T06:32:20.195 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.000107292 2024-09-14T06:32:20.195 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:20.195 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:32:20.195 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:20.195 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:20.195 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:20.195 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:19354 3.3 3:c34e5e51:::rbd_data.112d92027590.0000000000000024:f [read 1189888~1024 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:32:20.195 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:27.062842+0000", 2024-09-14T06:32:20.195 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 173.107139545, 2024-09-14T06:32:20.195 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046594257, 2024-09-14T06:32:20.196 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:32:20.196 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:32:20.196 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:32:20.196 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:32:20.196 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:32:20.196 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:20.196 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 19354 2024-09-14T06:32:20.196 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:20.196 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:32:20.196 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:20.196 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:32:20.196 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062842+0000", 2024-09-14T06:32:20.196 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:20.196 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:20.197 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:20.197 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:32:20.197 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062842+0000", 2024-09-14T06:32:20.197 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:20.197 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:20.197 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:20.197 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:32:20.197 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062842+0000", 2024-09-14T06:32:20.197 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.7399999999999999e-07 2024-09-14T06:32:20.197 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:20.197 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:20.197 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:32:20.197 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062859+0000", 2024-09-14T06:32:20.197 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.7010000000000001e-05 2024-09-14T06:32:20.197 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:20.198 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:20.198 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:32:20.198 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062866+0000", 2024-09-14T06:32:20.198 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.9990000000000002e-06 2024-09-14T06:32:20.198 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:20.198 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:20.198 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:32:20.198 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062873+0000", 2024-09-14T06:32:20.198 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.8569999999999997e-06 2024-09-14T06:32:20.198 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:20.198 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:20.198 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:32:20.198 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.109335+0000", 2024-09-14T06:32:20.198 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046461943999999998 2024-09-14T06:32:20.198 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:20.199 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:20.199 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:32:20.199 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.109377+0000", 2024-09-14T06:32:20.199 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.2558999999999997e-05 2024-09-14T06:32:20.199 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:20.199 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:20.199 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:32:20.199 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.109436+0000", 2024-09-14T06:32:20.199 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.8514000000000003e-05 2024-09-14T06:32:20.199 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:20.199 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:32:20.199 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:20.199 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:20.199 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:20.200 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:19355 3.3 3:c34e5e51:::rbd_data.112d92027590.0000000000000024:f [read 1200128~1024 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:32:20.200 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:27.062884+0000", 2024-09-14T06:32:20.200 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 173.107096872, 2024-09-14T06:32:20.200 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046681650999999998, 2024-09-14T06:32:20.200 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:32:20.200 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:32:20.200 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:32:20.200 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:32:20.200 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:32:20.200 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:20.200 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 19355 2024-09-14T06:32:20.200 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:20.200 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:32:20.200 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:20.201 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:32:20.201 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062884+0000", 2024-09-14T06:32:20.201 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:20.201 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:20.201 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:20.201 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:32:20.201 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062884+0000", 2024-09-14T06:32:20.201 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:20.201 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:20.201 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:20.201 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:32:20.201 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062885+0000", 2024-09-14T06:32:20.201 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.8000000000000001e-07 2024-09-14T06:32:20.201 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:20.202 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:20.202 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:32:20.202 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062886+0000", 2024-09-14T06:32:20.202 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.8920000000000001e-06 2024-09-14T06:32:20.202 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:20.202 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:20.202 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:32:20.202 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062892+0000", 2024-09-14T06:32:20.202 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.84e-06 2024-09-14T06:32:20.202 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:20.202 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:20.202 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:32:20.202 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062899+0000", 2024-09-14T06:32:20.202 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.5339999999999996e-06 2024-09-14T06:32:20.202 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:20.202 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:20.203 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:32:20.203 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.109466+0000", 2024-09-14T06:32:20.203 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046566919999999998 2024-09-14T06:32:20.203 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:20.203 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:20.203 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:32:20.203 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.109510+0000", 2024-09-14T06:32:20.203 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.4153000000000002e-05 2024-09-14T06:32:20.203 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:20.203 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:20.203 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:32:20.203 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.109566+0000", 2024-09-14T06:32:20.203 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.5931999999999999e-05 2024-09-14T06:32:20.203 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:20.203 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:32:20.204 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:20.204 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:20.204 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:20.204 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:19356 3.3 3:c34e5e51:::rbd_data.112d92027590.0000000000000024:f [read 1210368~1024 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:32:20.204 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:27.062908+0000", 2024-09-14T06:32:20.204 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 173.10707343499999, 2024-09-14T06:32:20.204 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046807498000000003, 2024-09-14T06:32:20.204 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:32:20.204 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:32:20.204 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:32:20.204 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:32:20.204 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:32:20.204 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:20.204 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 19356 2024-09-14T06:32:20.205 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:20.205 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:32:20.205 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:20.205 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:32:20.205 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062908+0000", 2024-09-14T06:32:20.205 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:20.205 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:20.205 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:20.205 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:32:20.205 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062908+0000", 2024-09-14T06:32:20.205 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:20.205 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:20.205 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:20.205 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:32:20.206 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062908+0000", 2024-09-14T06:32:20.206 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.5699999999999998e-07 2024-09-14T06:32:20.206 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:20.206 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:20.206 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:32:20.206 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062910+0000", 2024-09-14T06:32:20.206 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.7710000000000001e-06 2024-09-14T06:32:20.206 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:20.206 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:20.206 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:32:20.206 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062920+0000", 2024-09-14T06:32:20.206 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.0654e-05 2024-09-14T06:32:20.206 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:20.206 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:20.207 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:32:20.207 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062931+0000", 2024-09-14T06:32:20.207 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.075e-05 2024-09-14T06:32:20.207 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:20.207 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:20.207 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:32:20.207 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.109595+0000", 2024-09-14T06:32:20.207 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046663713000000002 2024-09-14T06:32:20.207 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:20.207 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:20.207 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:32:20.207 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.109632+0000", 2024-09-14T06:32:20.207 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.7505000000000001e-05 2024-09-14T06:32:20.207 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:20.207 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:20.207 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:32:20.208 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.109715+0000", 2024-09-14T06:32:20.208 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.2748000000000001e-05 2024-09-14T06:32:20.208 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:20.208 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:32:20.208 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:20.208 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:20.208 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:20.208 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:19357 3.3 3:c34e5e51:::rbd_data.112d92027590.0000000000000024:f [read 1220608~1024 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:32:20.208 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:27.062939+0000", 2024-09-14T06:32:20.208 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 173.107041987, 2024-09-14T06:32:20.208 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046920508, 2024-09-14T06:32:20.208 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:32:20.208 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:32:20.208 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:32:20.208 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:32:20.209 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:32:20.209 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:20.209 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 19357 2024-09-14T06:32:20.209 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:20.209 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:32:20.209 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:20.209 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:32:20.209 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062939+0000", 2024-09-14T06:32:20.209 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:20.209 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:20.209 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:20.209 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:32:20.209 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062939+0000", 2024-09-14T06:32:20.210 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:20.210 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:20.210 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:20.210 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:32:20.210 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062940+0000", 2024-09-14T06:32:20.210 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.32e-07 2024-09-14T06:32:20.210 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:20.210 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:20.210 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:32:20.210 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062942+0000", 2024-09-14T06:32:20.210 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.1919999999999999e-06 2024-09-14T06:32:20.210 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:20.210 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:20.210 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:32:20.211 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062948+0000", 2024-09-14T06:32:20.211 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.1149999999999999e-06 2024-09-14T06:32:20.211 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:20.211 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:20.211 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:32:20.211 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062960+0000", 2024-09-14T06:32:20.211 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.1895e-05 2024-09-14T06:32:20.211 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:20.211 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:20.211 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:32:20.211 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.109704+0000", 2024-09-14T06:32:20.211 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046744584999999998 2024-09-14T06:32:20.211 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:20.211 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:20.211 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:32:20.212 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.109777+0000", 2024-09-14T06:32:20.212 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.2282999999999999e-05 2024-09-14T06:32:20.212 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:20.212 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:20.212 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:32:20.212 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.109860+0000", 2024-09-14T06:32:20.212 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.3005999999999998e-05 2024-09-14T06:32:20.212 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:20.212 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:32:20.212 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:20.212 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:20.212 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:20.212 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:19358 3.3 3:c34e5e51:::rbd_data.112d92027590.0000000000000024:f [read 1230848~1024 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:32:20.212 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:27.062967+0000", 2024-09-14T06:32:20.213 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 173.107014357, 2024-09-14T06:32:20.213 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047144471, 2024-09-14T06:32:20.213 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:32:20.213 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:32:20.213 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:32:20.213 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:32:20.213 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:32:20.213 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:20.213 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 19358 2024-09-14T06:32:20.213 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:20.213 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:32:20.213 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:20.213 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:32:20.213 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062967+0000", 2024-09-14T06:32:20.214 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:20.214 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:20.214 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:20.214 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:32:20.214 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062967+0000", 2024-09-14T06:32:20.214 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:20.214 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:20.214 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:20.214 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:32:20.214 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062967+0000", 2024-09-14T06:32:20.214 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.89e-07 2024-09-14T06:32:20.214 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:20.214 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:20.215 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:32:20.215 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062969+0000", 2024-09-14T06:32:20.215 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.9929999999999998e-06 2024-09-14T06:32:20.215 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:20.215 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:20.215 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:32:20.215 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062975+0000", 2024-09-14T06:32:20.215 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.9030000000000001e-06 2024-09-14T06:32:20.215 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:20.215 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:20.215 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:32:20.215 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062979+0000", 2024-09-14T06:32:20.215 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.2250000000000002e-06 2024-09-14T06:32:20.215 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:20.215 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:20.216 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:32:20.216 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.109934+0000", 2024-09-14T06:32:20.216 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046954410000000002 2024-09-14T06:32:20.216 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:20.216 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:20.216 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:32:20.216 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.110007+0000", 2024-09-14T06:32:20.216 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.3829000000000001e-05 2024-09-14T06:32:20.216 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:20.216 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:20.216 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:32:20.216 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.110111+0000", 2024-09-14T06:32:20.216 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00010372199999999999 2024-09-14T06:32:20.217 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:20.217 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:32:20.217 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:20.217 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:20.217 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:20.217 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:19359 3.3 3:c34e5e51:::rbd_data.112d92027590.0000000000000024:f [read 1241088~1024 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:32:20.217 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:27.062987+0000", 2024-09-14T06:32:20.217 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 173.106994267, 2024-09-14T06:32:20.217 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047297182, 2024-09-14T06:32:20.217 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:32:20.217 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:32:20.217 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:32:20.217 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:32:20.217 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:32:20.217 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:20.218 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 19359 2024-09-14T06:32:20.218 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:20.218 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:32:20.218 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:20.218 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:32:20.218 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062987+0000", 2024-09-14T06:32:20.218 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:20.218 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:20.218 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:20.218 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:32:20.218 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062987+0000", 2024-09-14T06:32:20.218 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:20.218 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:20.218 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:20.219 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:32:20.219 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062987+0000", 2024-09-14T06:32:20.219 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.2500000000000001e-07 2024-09-14T06:32:20.219 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:20.219 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:20.219 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:32:20.219 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062989+0000", 2024-09-14T06:32:20.219 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.6080000000000001e-06 2024-09-14T06:32:20.219 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:20.219 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:20.219 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:32:20.219 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062994+0000", 2024-09-14T06:32:20.219 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.5520000000000004e-06 2024-09-14T06:32:20.219 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:20.220 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:20.220 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:32:20.220 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062999+0000", 2024-09-14T06:32:20.220 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.3930000000000001e-06 2024-09-14T06:32:20.220 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:20.220 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:20.220 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:32:20.220 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.110142+0000", 2024-09-14T06:32:20.220 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047143154 2024-09-14T06:32:20.220 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:20.220 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:20.220 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:32:20.220 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.110219+0000", 2024-09-14T06:32:20.221 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.6737999999999996e-05 2024-09-14T06:32:20.221 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:20.221 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:20.221 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:32:20.221 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.110284+0000", 2024-09-14T06:32:20.221 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.5412000000000003e-05 2024-09-14T06:32:20.221 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:20.221 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:32:20.221 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:20.221 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:20.221 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:20.221 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:19360 3.3 3:c34e5e51:::rbd_data.112d92027590.0000000000000024:f [read 1251328~1024 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:32:20.221 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:27.063017+0000", 2024-09-14T06:32:20.221 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 173.10696380600001, 2024-09-14T06:32:20.221 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047453823999999999, 2024-09-14T06:32:20.222 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:32:20.222 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:32:20.222 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:32:20.222 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:32:20.222 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:32:20.222 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:20.222 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 19360 2024-09-14T06:32:20.222 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:20.222 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:32:20.222 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:20.222 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:32:20.222 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.063017+0000", 2024-09-14T06:32:20.222 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:20.223 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:20.223 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:20.223 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:32:20.223 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.063017+0000", 2024-09-14T06:32:20.223 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:20.223 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:20.223 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:20.223 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:32:20.223 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.064977+0000", 2024-09-14T06:32:20.223 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.0019597080000000001 2024-09-14T06:32:20.223 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:20.223 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:20.223 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:32:20.223 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.064987+0000", 2024-09-14T06:32:20.224 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.9620000000000005e-06 2024-09-14T06:32:20.224 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:20.224 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:20.224 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:32:20.224 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065000+0000", 2024-09-14T06:32:20.224 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.3173000000000001e-05 2024-09-14T06:32:20.224 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:20.224 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:20.224 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:32:20.224 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065005+0000", 2024-09-14T06:32:20.224 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.9960000000000001e-06 2024-09-14T06:32:20.224 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:20.224 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:20.225 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:32:20.225 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.110314+0000", 2024-09-14T06:32:20.225 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.04530899 2024-09-14T06:32:20.225 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:20.225 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:20.225 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:32:20.225 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.110398+0000", 2024-09-14T06:32:20.225 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.3539000000000001e-05 2024-09-14T06:32:20.225 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:20.225 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:20.225 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:32:20.225 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.110471+0000", 2024-09-14T06:32:20.225 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.3455999999999996e-05 2024-09-14T06:32:20.225 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:20.225 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:32:20.226 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:20.226 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:20.226 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:20.226 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:19365 3.3 3:c34e5e51:::rbd_data.112d92027590.0000000000000024:f [read 1302528~1024 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:32:20.226 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:27.065058+0000", 2024-09-14T06:32:20.226 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 173.104923136, 2024-09-14T06:32:20.226 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046105526000000001, 2024-09-14T06:32:20.226 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:32:20.226 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:32:20.226 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:32:20.226 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:32:20.226 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:32:20.226 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:20.226 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 19365 2024-09-14T06:32:20.226 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:20.226 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:32:20.227 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:20.227 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:32:20.227 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065058+0000", 2024-09-14T06:32:20.227 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:20.227 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:20.227 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:20.227 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:32:20.227 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065058+0000", 2024-09-14T06:32:20.227 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:20.227 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:20.227 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:20.227 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:32:20.227 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065058+0000", 2024-09-14T06:32:20.227 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.1899999999999999e-07 2024-09-14T06:32:20.228 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:20.228 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:20.228 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:32:20.228 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065059+0000", 2024-09-14T06:32:20.228 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.1299999999999998e-07 2024-09-14T06:32:20.228 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:20.228 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:20.228 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:32:20.228 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065063+0000", 2024-09-14T06:32:20.228 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.1099999999999996e-06 2024-09-14T06:32:20.228 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:20.228 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:20.228 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:32:20.228 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065070+0000", 2024-09-14T06:32:20.229 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.973e-06 2024-09-14T06:32:20.229 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:20.229 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:20.229 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:32:20.229 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.111014+0000", 2024-09-14T06:32:20.229 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.045943373000000003 2024-09-14T06:32:20.229 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:20.229 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:20.229 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:32:20.229 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.111073+0000", 2024-09-14T06:32:20.229 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.8987999999999998e-05 2024-09-14T06:32:20.229 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:20.229 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:20.229 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:32:20.230 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.111163+0000", 2024-09-14T06:32:20.230 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.0950000000000002e-05 2024-09-14T06:32:20.230 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:20.230 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:32:20.230 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:20.230 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:20.230 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:20.230 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:19366 3.3 3:c34e5e51:::rbd_data.112d92027590.0000000000000024:f [read 1312768~1024 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:32:20.230 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:27.065074+0000", 2024-09-14T06:32:20.230 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 173.10490728600001, 2024-09-14T06:32:20.230 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046289858000000003, 2024-09-14T06:32:20.230 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:32:20.230 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:32:20.230 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:32:20.231 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:32:20.231 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:32:20.231 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:20.231 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 19366 2024-09-14T06:32:20.231 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:20.231 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:32:20.231 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:20.231 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:32:20.231 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065074+0000", 2024-09-14T06:32:20.231 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:20.231 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:20.231 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:20.231 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:32:20.231 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065074+0000", 2024-09-14T06:32:20.231 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:20.232 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:20.232 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:20.232 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:32:20.232 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065074+0000", 2024-09-14T06:32:20.232 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.3900000000000001e-07 2024-09-14T06:32:20.232 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:20.232 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:20.232 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:32:20.232 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065078+0000", 2024-09-14T06:32:20.232 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.3229999999999999e-06 2024-09-14T06:32:20.232 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:20.232 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:20.232 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:32:20.232 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065082+0000", 2024-09-14T06:32:20.233 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.8299999999999998e-06 2024-09-14T06:32:20.233 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:20.233 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:20.233 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:32:20.233 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065085+0000", 2024-09-14T06:32:20.233 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.8660000000000002e-06 2024-09-14T06:32:20.233 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:20.233 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:20.233 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:32:20.233 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.111270+0000", 2024-09-14T06:32:20.233 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046185035999999999 2024-09-14T06:32:20.233 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:20.233 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:20.233 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:32:20.234 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.111311+0000", 2024-09-14T06:32:20.234 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.0935e-05 2024-09-14T06:32:20.234 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:20.234 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:20.234 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:32:20.234 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.111364+0000", 2024-09-14T06:32:20.234 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.2629e-05 2024-09-14T06:32:20.234 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:20.234 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:32:20.234 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:20.234 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:20.234 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:20.234 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:19367 3.3 3:c34e5e51:::rbd_data.112d92027590.0000000000000024:f [read 1323008~1024 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:32:20.234 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:27.065115+0000", 2024-09-14T06:32:20.234 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 173.104865725, 2024-09-14T06:32:20.235 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046407003000000002, 2024-09-14T06:32:20.235 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:32:20.235 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:32:20.235 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:32:20.235 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:32:20.235 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:32:20.235 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:20.235 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 19367 2024-09-14T06:32:20.235 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:20.235 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:32:20.235 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:20.235 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:32:20.235 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065115+0000", 2024-09-14T06:32:20.235 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:20.236 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:20.236 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:20.236 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:32:20.236 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065115+0000", 2024-09-14T06:32:20.236 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:20.236 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:20.236 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:20.236 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:32:20.236 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065116+0000", 2024-09-14T06:32:20.236 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.4999999999999998e-07 2024-09-14T06:32:20.236 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:20.236 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:20.236 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:32:20.236 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065117+0000", 2024-09-14T06:32:20.237 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.189e-06 2024-09-14T06:32:20.237 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:20.237 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:20.237 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:32:20.237 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065122+0000", 2024-09-14T06:32:20.237 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.3709999999999999e-06 2024-09-14T06:32:20.237 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:20.237 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:20.237 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:32:20.237 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065125+0000", 2024-09-14T06:32:20.237 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.8940000000000002e-06 2024-09-14T06:32:20.237 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:20.237 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:20.237 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:32:20.237 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.111402+0000", 2024-09-14T06:32:20.238 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046276645999999998 2024-09-14T06:32:20.238 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:20.238 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:20.238 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:32:20.238 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.111445+0000", 2024-09-14T06:32:20.238 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.3288000000000003e-05 2024-09-14T06:32:20.238 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:20.238 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:20.238 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:32:20.238 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.111522+0000", 2024-09-14T06:32:20.238 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.7164999999999997e-05 2024-09-14T06:32:20.238 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:20.238 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:32:20.238 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:20.239 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:20.239 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:20.239 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:19368 3.3 3:c34e5e51:::rbd_data.112d92027590.0000000000000024:f [read 1333248~1024 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:32:20.239 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:27.065130+0000", 2024-09-14T06:32:20.239 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 173.104851435, 2024-09-14T06:32:20.239 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046507275000000001, 2024-09-14T06:32:20.239 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:32:20.239 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:32:20.239 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:32:20.239 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:32:20.239 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:32:20.239 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:20.239 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 19368 2024-09-14T06:32:20.239 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:20.239 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:32:20.240 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:20.240 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:32:20.240 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065130+0000", 2024-09-14T06:32:20.240 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:20.240 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:20.240 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:20.240 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:32:20.240 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065130+0000", 2024-09-14T06:32:20.240 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:20.240 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:20.240 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:20.240 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:32:20.240 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065130+0000", 2024-09-14T06:32:20.241 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.0800000000000001e-07 2024-09-14T06:32:20.241 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:20.241 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:20.241 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:32:20.241 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065131+0000", 2024-09-14T06:32:20.241 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.8700000000000004e-07 2024-09-14T06:32:20.241 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:20.241 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:20.241 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:32:20.241 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065134+0000", 2024-09-14T06:32:20.241 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.1039999999999998e-06 2024-09-14T06:32:20.241 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:20.241 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:20.241 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:32:20.242 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065136+0000", 2024-09-14T06:32:20.242 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.3949999999999999e-06 2024-09-14T06:32:20.242 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:20.242 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:20.242 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:32:20.242 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.111541+0000", 2024-09-14T06:32:20.242 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046404206000000003 2024-09-14T06:32:20.242 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:20.242 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:20.242 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:32:20.243 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.111582+0000", 2024-09-14T06:32:20.243 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.1051999999999999e-05 2024-09-14T06:32:20.243 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:20.243 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:20.243 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:32:20.243 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.111637+0000", 2024-09-14T06:32:20.243 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.5322999999999999e-05 2024-09-14T06:32:20.243 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:20.243 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:32:20.243 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:20.243 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:20.243 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:20.243 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:19369 3.3 3:c34e5e51:::rbd_data.112d92027590.0000000000000024:f [read 1343488~1024 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:32:20.243 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:27.065139+0000", 2024-09-14T06:32:20.244 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 173.10484161799999, 2024-09-14T06:32:20.244 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046682169000000003, 2024-09-14T06:32:20.244 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:32:20.244 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:32:20.244 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:32:20.244 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:32:20.244 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:32:20.244 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:20.244 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 19369 2024-09-14T06:32:20.244 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:20.244 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:32:20.244 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:20.245 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:32:20.245 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065139+0000", 2024-09-14T06:32:20.245 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:20.245 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:20.245 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:20.245 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:32:20.245 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065139+0000", 2024-09-14T06:32:20.245 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:20.245 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:20.245 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:20.245 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:32:20.245 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065140+0000", 2024-09-14T06:32:20.245 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.8299999999999998e-07 2024-09-14T06:32:20.245 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:20.246 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:20.246 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:32:20.246 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065141+0000", 2024-09-14T06:32:20.246 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.3900000000000003e-07 2024-09-14T06:32:20.246 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:20.246 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:20.246 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:32:20.246 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065144+0000", 2024-09-14T06:32:20.246 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.7409999999999998e-06 2024-09-14T06:32:20.246 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:20.246 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:20.246 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:32:20.246 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065147+0000", 2024-09-14T06:32:20.246 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.4150000000000002e-06 2024-09-14T06:32:20.246 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:20.247 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:20.247 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:32:20.247 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.111673+0000", 2024-09-14T06:32:20.247 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046526004000000003 2024-09-14T06:32:20.247 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:20.247 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:20.247 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:32:20.247 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.111749+0000", 2024-09-14T06:32:20.247 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.5741999999999994e-05 2024-09-14T06:32:20.247 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:20.247 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:20.247 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:32:20.247 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.111822+0000", 2024-09-14T06:32:20.247 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.3045000000000003e-05 2024-09-14T06:32:20.248 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:20.248 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:32:20.248 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:20.248 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:20.248 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:20.248 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:19370 3.3 3:c34e5e51:::rbd_data.112d92027590.0000000000000024:f [read 1353728~1024 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:32:20.248 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:27.065150+0000", 2024-09-14T06:32:20.248 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 173.104830889, 2024-09-14T06:32:20.248 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046897556999999999, 2024-09-14T06:32:20.248 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:32:20.248 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:32:20.248 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:32:20.248 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:32:20.248 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:32:20.249 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:20.249 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 19370 2024-09-14T06:32:20.249 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:20.249 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:32:20.249 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:20.249 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:32:20.249 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065150+0000", 2024-09-14T06:32:20.249 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:20.249 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:20.249 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:20.249 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:32:20.249 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065150+0000", 2024-09-14T06:32:20.249 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:20.249 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:20.250 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:20.250 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:32:20.250 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065150+0000", 2024-09-14T06:32:20.250 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.0200000000000001e-07 2024-09-14T06:32:20.250 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:20.250 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:20.250 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:32:20.250 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065151+0000", 2024-09-14T06:32:20.250 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.1100000000000004e-07 2024-09-14T06:32:20.250 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:20.250 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:20.250 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:32:20.250 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065154+0000", 2024-09-14T06:32:20.250 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.9639999999999999e-06 2024-09-14T06:32:20.250 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:20.251 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:20.251 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:32:20.251 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065157+0000", 2024-09-14T06:32:20.251 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.4260000000000002e-06 2024-09-14T06:32:20.251 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:20.251 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:20.251 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:32:20.251 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.111861+0000", 2024-09-14T06:32:20.251 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046704268 2024-09-14T06:32:20.251 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:20.251 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:20.251 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:32:20.251 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.111939+0000", 2024-09-14T06:32:20.251 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.8231000000000004e-05 2024-09-14T06:32:20.252 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:20.252 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:20.252 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:32:20.252 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.112048+0000", 2024-09-14T06:32:20.252 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00010855500000000001 2024-09-14T06:32:20.252 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:20.252 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:32:20.252 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:20.252 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:20.252 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:20.252 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:19371 3.3 3:c34e5e51:::rbd_data.112d92027590.0000000000000024:f [read 1363968~1024 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:32:20.252 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:27.065161+0000", 2024-09-14T06:32:20.252 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 173.10482023899999, 2024-09-14T06:32:20.252 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047021944000000003, 2024-09-14T06:32:20.253 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:32:20.253 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:32:20.253 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:32:20.253 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:32:20.253 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:32:20.253 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:20.253 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 19371 2024-09-14T06:32:20.253 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:20.253 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:32:20.253 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:20.253 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:32:20.253 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065161+0000", 2024-09-14T06:32:20.253 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:20.253 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:20.254 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:20.254 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:32:20.254 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065161+0000", 2024-09-14T06:32:20.254 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:20.254 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:20.254 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:20.254 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:32:20.254 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065161+0000", 2024-09-14T06:32:20.254 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.0200000000000001e-07 2024-09-14T06:32:20.254 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:20.254 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:20.254 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:32:20.254 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065166+0000", 2024-09-14T06:32:20.254 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.9869999999999999e-06 2024-09-14T06:32:20.255 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:20.255 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:20.255 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:32:20.255 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065169+0000", 2024-09-14T06:32:20.255 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.3799999999999998e-06 2024-09-14T06:32:20.255 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:20.255 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:20.255 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:32:20.255 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065172+0000", 2024-09-14T06:32:20.255 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.4830000000000002e-06 2024-09-14T06:32:20.255 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:20.255 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:20.255 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:32:20.255 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.112071+0000", 2024-09-14T06:32:20.256 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046899529000000002 2024-09-14T06:32:20.256 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:20.256 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:20.256 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:32:20.256 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.112128+0000", 2024-09-14T06:32:20.256 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.6341999999999997e-05 2024-09-14T06:32:20.256 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:20.256 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:20.256 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:32:20.256 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.112183+0000", 2024-09-14T06:32:20.256 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.5021000000000001e-05 2024-09-14T06:32:20.256 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:20.256 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:32:20.256 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:20.256 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:20.257 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:20.257 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:19372 3.3 3:c34e5e51:::rbd_data.112d92027590.0000000000000024:f [read 1374208~1024 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:32:20.257 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:27.065176+0000", 2024-09-14T06:32:20.257 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 173.104805089, 2024-09-14T06:32:20.257 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047188889999999997, 2024-09-14T06:32:20.257 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:32:20.257 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:32:20.257 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:32:20.257 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:32:20.257 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:32:20.257 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:20.257 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 19372 2024-09-14T06:32:20.257 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:20.257 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:32:20.257 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:20.258 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:32:20.258 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065176+0000", 2024-09-14T06:32:20.258 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:20.258 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:20.258 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:20.258 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:32:20.258 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065176+0000", 2024-09-14T06:32:20.258 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:20.258 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:20.258 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:20.258 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:32:20.258 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.071117+0000", 2024-09-14T06:32:20.258 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.0059404990000000001 2024-09-14T06:32:20.258 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:20.258 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:20.259 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:32:20.259 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.071120+0000", 2024-09-14T06:32:20.259 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.4340000000000001e-06 2024-09-14T06:32:20.259 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:20.259 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:20.259 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:32:20.259 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.071129+0000", 2024-09-14T06:32:20.259 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.8389999999999996e-06 2024-09-14T06:32:20.259 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:20.259 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:20.259 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:32:20.259 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.071134+0000", 2024-09-14T06:32:20.259 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.412e-06 2024-09-14T06:32:20.259 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:20.260 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:20.260 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:32:20.260 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.112221+0000", 2024-09-14T06:32:20.260 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.041086473999999998 2024-09-14T06:32:20.260 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:20.260 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:20.260 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:32:20.260 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.112299+0000", 2024-09-14T06:32:20.260 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.8746999999999997e-05 2024-09-14T06:32:20.260 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:20.260 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:20.260 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:32:20.260 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.112365+0000", 2024-09-14T06:32:20.260 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.5485000000000001e-05 2024-09-14T06:32:20.261 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:20.261 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:32:20.261 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:20.261 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:20.261 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:32:20.261 INFO:teuthology.orchestra.run.smithi138.stdout:} 2024-09-14T06:32:20.262 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-14T06:32:20.263 INFO:tasks.ceph.osd.5.smithi138.stderr:2024-09-14T06:32:20.261+0000 7fe13746f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33618) UID: 0 2024-09-14T06:32:20.362 INFO:tasks.ceph.osd.5.smithi138.stderr:2024-09-14T06:32:20.361+0000 7fe13746f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33618) UID: 0 2024-09-14T06:32:20.406 INFO:teuthology.orchestra.run.smithi138.stdout:{ 2024-09-14T06:32:20.406 INFO:teuthology.orchestra.run.smithi138.stdout: "ops": [], 2024-09-14T06:32:20.406 INFO:teuthology.orchestra.run.smithi138.stdout: "num_ops": 0 2024-09-14T06:32:20.406 INFO:teuthology.orchestra.run.smithi138.stdout:} 2024-09-14T06:32:20.415 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-14T06:32:20.464 INFO:tasks.ceph.osd.2.smithi094.stderr:2024-09-14T06:32:20.463+0000 7faf3269b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33649) UID: 0 2024-09-14T06:32:20.558 INFO:teuthology.orchestra.run.smithi138.stdout:{ 2024-09-14T06:32:20.559 INFO:teuthology.orchestra.run.smithi138.stdout: "ops": [], 2024-09-14T06:32:20.559 INFO:teuthology.orchestra.run.smithi138.stdout: "complaint_time": 30, 2024-09-14T06:32:20.559 INFO:teuthology.orchestra.run.smithi138.stdout: "num_blocked_ops": 0 2024-09-14T06:32:20.559 INFO:teuthology.orchestra.run.smithi138.stdout:} 2024-09-14T06:32:20.565 INFO:tasks.ceph.osd.0.smithi094.stderr:2024-09-14T06:32:20.564+0000 7f07c9333640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33636) UID: 0 2024-09-14T06:32:20.567 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-14T06:32:20.666 INFO:tasks.ceph.osd.6.smithi138.stderr:2024-09-14T06:32:20.665+0000 7f8cd8849640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33620) UID: 0 2024-09-14T06:32:20.678 INFO:teuthology.orchestra.run.smithi138.stdout:{ 2024-09-14T06:32:20.678 INFO:teuthology.orchestra.run.smithi138.stdout: "size": 20, 2024-09-14T06:32:20.679 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 600, 2024-09-14T06:32:20.679 INFO:teuthology.orchestra.run.smithi138.stdout: "ops": [ 2024-09-14T06:32:20.679 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:20.679 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:11453 3.1d 3:bcad7303:::rbd_data.112d92027590.0000000000000001:f [read 94208~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:32:20.679 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:25.654375+0000", 2024-09-14T06:32:20.679 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 175.022684156, 2024-09-14T06:32:20.679 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052449318000000002, 2024-09-14T06:32:20.679 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:32:20.679 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:32:20.679 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:32:20.679 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:32:20.679 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:32:20.679 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:20.680 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 11453 2024-09-14T06:32:20.680 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:20.680 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:32:20.680 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:20.680 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:32:20.680 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654375+0000", 2024-09-14T06:32:20.680 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:20.680 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:20.680 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:20.680 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:32:20.680 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654375+0000", 2024-09-14T06:32:20.680 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:20.680 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:20.680 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:20.681 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:32:20.681 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654496+0000", 2024-09-14T06:32:20.681 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00012081 2024-09-14T06:32:20.681 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:20.681 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:20.681 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:32:20.681 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654497+0000", 2024-09-14T06:32:20.681 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.8789999999999999e-06 2024-09-14T06:32:20.681 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:20.681 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:20.681 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:32:20.681 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654501+0000", 2024-09-14T06:32:20.681 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.8999999999999999e-06 2024-09-14T06:32:20.681 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:20.681 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:20.682 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:32:20.682 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654505+0000", 2024-09-14T06:32:20.682 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.2229999999999999e-06 2024-09-14T06:32:20.682 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:20.682 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:20.682 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:32:20.682 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.705917+0000", 2024-09-14T06:32:20.682 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.051412501999999999 2024-09-14T06:32:20.682 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:20.682 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:20.682 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:32:20.682 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.705999+0000", 2024-09-14T06:32:20.682 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.1551999999999994e-05 2024-09-14T06:32:20.682 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:20.682 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:20.683 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:32:20.683 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.706824+0000", 2024-09-14T06:32:20.683 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00082545199999999996 2024-09-14T06:32:20.683 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:20.683 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:32:20.683 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:20.683 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:20.683 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:20.683 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:11454 3.1d 3:bcad7303:::rbd_data.112d92027590.0000000000000001:f [read 104448~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:32:20.683 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:25.654510+0000", 2024-09-14T06:32:20.683 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 175.02254936599999, 2024-09-14T06:32:20.683 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052610713000000003, 2024-09-14T06:32:20.683 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:32:20.683 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:32:20.683 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:32:20.684 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:32:20.684 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:32:20.684 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:20.684 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 11454 2024-09-14T06:32:20.684 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:20.684 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:32:20.684 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:20.684 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:32:20.684 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654510+0000", 2024-09-14T06:32:20.684 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:20.684 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:20.684 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:20.684 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:32:20.685 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654510+0000", 2024-09-14T06:32:20.685 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:20.685 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:20.685 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:20.685 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:32:20.685 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654825+0000", 2024-09-14T06:32:20.685 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00031522400000000002 2024-09-14T06:32:20.685 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:20.685 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:20.685 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:32:20.685 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654827+0000", 2024-09-14T06:32:20.685 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.9709999999999998e-06 2024-09-14T06:32:20.685 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:20.685 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:20.685 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:32:20.686 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654832+0000", 2024-09-14T06:32:20.686 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.4099999999999999e-06 2024-09-14T06:32:20.686 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:20.686 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:20.686 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:32:20.686 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654835+0000", 2024-09-14T06:32:20.686 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.14e-06 2024-09-14T06:32:20.686 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:20.686 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:20.686 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:32:20.686 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.706851+0000", 2024-09-14T06:32:20.686 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052015761000000001 2024-09-14T06:32:20.686 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:20.686 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:20.687 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:32:20.687 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.706934+0000", 2024-09-14T06:32:20.687 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.2484999999999994e-05 2024-09-14T06:32:20.687 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:20.687 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:20.687 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:32:20.687 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.707120+0000", 2024-09-14T06:32:20.687 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00018672200000000001 2024-09-14T06:32:20.687 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:20.687 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:32:20.687 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:20.687 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:20.687 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:20.687 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:11455 3.1d 3:bcad7303:::rbd_data.112d92027590.0000000000000001:f [read 114688~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:32:20.687 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:25.654841+0000", 2024-09-14T06:32:20.688 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 175.02221803699999, 2024-09-14T06:32:20.688 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052579898, 2024-09-14T06:32:20.688 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:32:20.688 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:32:20.688 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:32:20.688 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:32:20.688 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:32:20.688 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:20.688 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 11455 2024-09-14T06:32:20.688 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:20.688 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:32:20.688 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:20.688 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:32:20.689 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654841+0000", 2024-09-14T06:32:20.689 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:20.689 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:20.689 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:20.689 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:32:20.689 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654841+0000", 2024-09-14T06:32:20.689 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:20.689 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:20.689 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:20.689 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:32:20.689 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654841+0000", 2024-09-14T06:32:20.689 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.7300000000000002e-07 2024-09-14T06:32:20.689 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:20.689 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:20.690 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:32:20.690 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654845+0000", 2024-09-14T06:32:20.690 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.0520000000000003e-06 2024-09-14T06:32:20.690 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:20.690 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:20.690 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:32:20.690 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654849+0000", 2024-09-14T06:32:20.690 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.0459999999999999e-06 2024-09-14T06:32:20.690 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:20.690 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:20.690 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:32:20.690 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654852+0000", 2024-09-14T06:32:20.690 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.9830000000000001e-06 2024-09-14T06:32:20.690 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:20.690 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:20.691 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:32:20.691 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.707159+0000", 2024-09-14T06:32:20.691 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052306826000000001 2024-09-14T06:32:20.691 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:20.691 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:20.691 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:32:20.691 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.707242+0000", 2024-09-14T06:32:20.691 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.2595000000000005e-05 2024-09-14T06:32:20.691 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:20.691 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:20.691 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:32:20.692 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.707421+0000", 2024-09-14T06:32:20.692 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.000179123 2024-09-14T06:32:20.692 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:20.692 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:32:20.692 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:20.692 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:20.692 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:20.692 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:11456 3.1d 3:bcad7303:::rbd_data.112d92027590.0000000000000001:f [read 124928~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:32:20.692 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:25.654856+0000", 2024-09-14T06:32:20.692 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 175.02220319099999, 2024-09-14T06:32:20.692 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052801580000000001, 2024-09-14T06:32:20.692 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:32:20.693 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:32:20.693 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:32:20.693 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:32:20.693 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:32:20.693 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:20.693 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 11456 2024-09-14T06:32:20.693 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:20.693 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:32:20.693 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:20.693 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:32:20.693 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654856+0000", 2024-09-14T06:32:20.693 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:20.693 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:20.693 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:20.693 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:32:20.694 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654856+0000", 2024-09-14T06:32:20.694 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:20.694 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:20.694 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:20.694 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:32:20.694 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654856+0000", 2024-09-14T06:32:20.694 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.29e-07 2024-09-14T06:32:20.694 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:20.694 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:20.694 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:32:20.694 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654857+0000", 2024-09-14T06:32:20.694 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.187e-06 2024-09-14T06:32:20.694 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:20.694 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:20.695 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:32:20.695 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654861+0000", 2024-09-14T06:32:20.695 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.3850000000000002e-06 2024-09-14T06:32:20.695 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:20.695 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:20.695 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:32:20.695 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654863+0000", 2024-09-14T06:32:20.695 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.9500000000000001e-06 2024-09-14T06:32:20.695 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:20.695 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:20.695 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:32:20.695 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.707456+0000", 2024-09-14T06:32:20.695 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052592434 2024-09-14T06:32:20.695 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:20.695 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:20.696 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:32:20.696 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.707516+0000", 2024-09-14T06:32:20.696 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.0167999999999997e-05 2024-09-14T06:32:20.696 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:20.696 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:20.696 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:32:20.696 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.707657+0000", 2024-09-14T06:32:20.696 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00014122700000000001 2024-09-14T06:32:20.696 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:20.696 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:32:20.696 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:20.696 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:20.696 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:20.696 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:11457 3.1d 3:bcad7303:::rbd_data.112d92027590.0000000000000001:f [read 135168~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:32:20.697 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:25.654867+0000", 2024-09-14T06:32:20.697 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 175.02219230599999, 2024-09-14T06:32:20.697 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052921902, 2024-09-14T06:32:20.697 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:32:20.697 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:32:20.697 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:32:20.697 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:32:20.697 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:32:20.697 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:20.697 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 11457 2024-09-14T06:32:20.697 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:20.697 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:32:20.697 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:20.697 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:32:20.697 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654867+0000", 2024-09-14T06:32:20.698 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:20.698 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:20.698 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:20.698 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:32:20.698 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654867+0000", 2024-09-14T06:32:20.698 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:20.698 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:20.698 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:20.698 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:32:20.698 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654972+0000", 2024-09-14T06:32:20.698 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00010584 2024-09-14T06:32:20.698 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:20.698 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:20.698 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:32:20.698 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654975+0000", 2024-09-14T06:32:20.699 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.1169999999999998e-06 2024-09-14T06:32:20.699 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:20.699 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:20.699 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:32:20.699 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654979+0000", 2024-09-14T06:32:20.699 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.1960000000000001e-06 2024-09-14T06:32:20.699 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:20.699 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:20.699 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:32:20.699 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654982+0000", 2024-09-14T06:32:20.699 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.2990000000000001e-06 2024-09-14T06:32:20.699 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:20.699 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:20.699 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:32:20.699 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.707684+0000", 2024-09-14T06:32:20.700 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052701979000000003 2024-09-14T06:32:20.700 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:20.700 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:20.700 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:32:20.700 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.707728+0000", 2024-09-14T06:32:20.700 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.3529999999999998e-05 2024-09-14T06:32:20.700 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:20.700 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:20.700 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:32:20.700 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.707789+0000", 2024-09-14T06:32:20.700 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.0940999999999998e-05 2024-09-14T06:32:20.700 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:20.700 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:32:20.700 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:20.700 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:20.700 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:20.701 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:11458 3.1d 3:bcad7303:::rbd_data.112d92027590.0000000000000001:f [read 145408~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:32:20.701 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:25.654987+0000", 2024-09-14T06:32:20.701 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 175.02207190799999, 2024-09-14T06:32:20.701 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.05291655, 2024-09-14T06:32:20.701 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:32:20.701 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:32:20.701 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:32:20.701 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:32:20.701 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:32:20.701 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:20.701 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 11458 2024-09-14T06:32:20.701 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:20.701 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:32:20.701 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:20.702 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:32:20.702 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654987+0000", 2024-09-14T06:32:20.702 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:20.702 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:20.702 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:20.702 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:32:20.702 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654987+0000", 2024-09-14T06:32:20.702 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:20.702 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:20.702 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:20.702 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:32:20.702 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655077+0000", 2024-09-14T06:32:20.702 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.0253000000000005e-05 2024-09-14T06:32:20.702 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:20.702 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:20.702 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:32:20.703 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655079+0000", 2024-09-14T06:32:20.703 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.9989999999999998e-06 2024-09-14T06:32:20.703 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:20.703 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:20.703 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:32:20.703 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655083+0000", 2024-09-14T06:32:20.703 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.9280000000000003e-06 2024-09-14T06:32:20.703 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:20.703 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:20.703 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:32:20.703 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655086+0000", 2024-09-14T06:32:20.703 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.056e-06 2024-09-14T06:32:20.703 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:20.703 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:20.704 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:32:20.704 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.707805+0000", 2024-09-14T06:32:20.704 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052718627999999997 2024-09-14T06:32:20.704 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:20.704 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:20.704 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:32:20.704 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.707848+0000", 2024-09-14T06:32:20.704 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.2728999999999997e-05 2024-09-14T06:32:20.704 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:20.704 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:20.704 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:32:20.704 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.707904+0000", 2024-09-14T06:32:20.704 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.5957e-05 2024-09-14T06:32:20.704 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:20.705 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:32:20.705 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:20.705 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:20.705 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:20.705 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:11459 3.1d 3:bcad7303:::rbd_data.112d92027590.0000000000000001:f [read 155648~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:32:20.705 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:25.655091+0000", 2024-09-14T06:32:20.705 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 175.02196789999999, 2024-09-14T06:32:20.705 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052944242000000002, 2024-09-14T06:32:20.705 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:32:20.705 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:32:20.705 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:32:20.705 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:32:20.705 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:32:20.705 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:20.706 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 11459 2024-09-14T06:32:20.706 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:20.706 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:32:20.706 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:20.706 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:32:20.706 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655091+0000", 2024-09-14T06:32:20.706 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:20.706 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:20.706 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:20.706 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:32:20.706 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655091+0000", 2024-09-14T06:32:20.706 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:20.706 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:20.706 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:20.707 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:32:20.707 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655237+0000", 2024-09-14T06:32:20.707 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00014570100000000001 2024-09-14T06:32:20.707 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:20.707 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:20.707 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:32:20.707 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655239+0000", 2024-09-14T06:32:20.707 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.8670000000000001e-06 2024-09-14T06:32:20.707 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:20.707 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:20.707 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:32:20.707 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655243+0000", 2024-09-14T06:32:20.707 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.9829999999999998e-06 2024-09-14T06:32:20.707 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:20.707 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:20.708 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:32:20.708 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655246+0000", 2024-09-14T06:32:20.708 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.1540000000000002e-06 2024-09-14T06:32:20.708 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:20.708 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:20.708 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:32:20.708 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.707934+0000", 2024-09-14T06:32:20.708 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052688286000000001 2024-09-14T06:32:20.708 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:20.708 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:20.708 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:32:20.708 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.707977+0000", 2024-09-14T06:32:20.708 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.3291999999999998e-05 2024-09-14T06:32:20.708 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:20.709 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:20.709 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:32:20.709 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.708035+0000", 2024-09-14T06:32:20.709 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.7958999999999999e-05 2024-09-14T06:32:20.709 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:20.709 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:32:20.709 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:20.709 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:20.709 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:20.709 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:11460 3.1d 3:bcad7303:::rbd_data.112d92027590.0000000000000001:f [read 165888~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:32:20.709 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:25.655250+0000", 2024-09-14T06:32:20.709 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 175.02180860300001, 2024-09-14T06:32:20.709 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052910698999999999, 2024-09-14T06:32:20.709 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:32:20.709 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:32:20.710 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:32:20.710 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:32:20.710 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:32:20.710 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:20.710 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 11460 2024-09-14T06:32:20.710 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:20.710 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:32:20.710 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:20.710 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:32:20.710 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655250+0000", 2024-09-14T06:32:20.710 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:20.710 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:20.710 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:20.710 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:32:20.711 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655250+0000", 2024-09-14T06:32:20.711 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:20.711 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:20.711 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:20.711 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:32:20.711 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655356+0000", 2024-09-14T06:32:20.711 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00010535900000000001 2024-09-14T06:32:20.711 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:20.711 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:20.711 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:32:20.711 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655357+0000", 2024-09-14T06:32:20.711 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.8080000000000001e-06 2024-09-14T06:32:20.711 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:20.711 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:20.711 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:32:20.712 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655361+0000", 2024-09-14T06:32:20.712 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.861e-06 2024-09-14T06:32:20.712 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:20.712 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:20.712 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:32:20.712 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655364+0000", 2024-09-14T06:32:20.712 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.1089999999999998e-06 2024-09-14T06:32:20.712 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:20.712 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:20.712 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:32:20.712 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.708065+0000", 2024-09-14T06:32:20.712 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052700595000000003 2024-09-14T06:32:20.712 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:20.712 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:20.713 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:32:20.713 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.708108+0000", 2024-09-14T06:32:20.713 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.2484999999999997e-05 2024-09-14T06:32:20.713 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:20.713 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:20.713 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:32:20.713 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.708161+0000", 2024-09-14T06:32:20.713 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.3482000000000001e-05 2024-09-14T06:32:20.713 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:20.713 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:32:20.713 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:20.713 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:20.713 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:20.713 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:11461 3.1d 3:bcad7303:::rbd_data.112d92027590.0000000000000001:f [read 176128~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:32:20.713 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:25.655370+0000", 2024-09-14T06:32:20.714 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 175.021689321, 2024-09-14T06:32:20.714 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052915141999999998, 2024-09-14T06:32:20.714 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:32:20.714 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:32:20.714 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:32:20.714 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:32:20.714 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:32:20.714 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:20.714 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 11461 2024-09-14T06:32:20.714 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:20.714 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:32:20.714 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:20.714 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:32:20.714 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655370+0000", 2024-09-14T06:32:20.715 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:20.715 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:20.715 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:20.715 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:32:20.715 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655370+0000", 2024-09-14T06:32:20.715 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:20.715 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:20.715 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:20.715 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:32:20.715 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655394+0000", 2024-09-14T06:32:20.715 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.4553999999999999e-05 2024-09-14T06:32:20.715 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:20.715 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:20.715 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:32:20.716 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655396+0000", 2024-09-14T06:32:20.716 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.8950000000000001e-06 2024-09-14T06:32:20.716 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:20.716 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:20.716 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:32:20.716 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655401+0000", 2024-09-14T06:32:20.716 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.0089999999999998e-06 2024-09-14T06:32:20.716 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:20.716 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:20.716 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:32:20.716 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655404+0000", 2024-09-14T06:32:20.716 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.3400000000000002e-06 2024-09-14T06:32:20.716 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:20.717 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:20.717 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:32:20.717 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.708186+0000", 2024-09-14T06:32:20.717 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052781726000000001 2024-09-14T06:32:20.717 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:20.717 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:20.717 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:32:20.717 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.708229+0000", 2024-09-14T06:32:20.717 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.2670999999999999e-05 2024-09-14T06:32:20.717 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:20.717 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:20.717 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:32:20.717 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.708285+0000", 2024-09-14T06:32:20.717 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.5946999999999998e-05 2024-09-14T06:32:20.718 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:20.718 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:32:20.718 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:20.718 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:20.718 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:20.718 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:11462 3.1d 3:bcad7303:::rbd_data.112d92027590.0000000000000001:f [read 186368~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:32:20.718 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:25.655410+0000", 2024-09-14T06:32:20.718 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 175.02164884300001, 2024-09-14T06:32:20.718 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052995258000000003, 2024-09-14T06:32:20.718 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:32:20.718 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:32:20.718 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:32:20.718 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:32:20.718 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:32:20.719 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:20.719 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 11462 2024-09-14T06:32:20.719 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:20.719 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:32:20.719 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:20.719 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:32:20.719 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655410+0000", 2024-09-14T06:32:20.719 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:20.719 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:20.719 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:20.719 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:32:20.719 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655410+0000", 2024-09-14T06:32:20.719 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:20.719 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:20.720 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:20.720 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:32:20.720 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655625+0000", 2024-09-14T06:32:20.720 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00021442600000000001 2024-09-14T06:32:20.720 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:20.720 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:20.720 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:32:20.720 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655626+0000", 2024-09-14T06:32:20.720 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.815e-06 2024-09-14T06:32:20.720 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:20.720 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:20.720 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:32:20.720 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655631+0000", 2024-09-14T06:32:20.720 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.651e-06 2024-09-14T06:32:20.721 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:20.721 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:20.721 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:32:20.721 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655634+0000", 2024-09-14T06:32:20.721 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.2449999999999998e-06 2024-09-14T06:32:20.721 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:20.721 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:20.721 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:32:20.721 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.708312+0000", 2024-09-14T06:32:20.721 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052677412 2024-09-14T06:32:20.721 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:20.721 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:20.721 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:32:20.721 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.708352+0000", 2024-09-14T06:32:20.722 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.0729999999999998e-05 2024-09-14T06:32:20.722 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:20.722 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:20.722 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:32:20.722 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.708405+0000", 2024-09-14T06:32:20.722 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.2979000000000002e-05 2024-09-14T06:32:20.722 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:20.722 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:32:20.722 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:20.722 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:20.722 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:20.722 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:11463 3.1d 3:bcad7303:::rbd_data.112d92027590.0000000000000001:f [read 196608~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:32:20.722 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:25.655639+0000", 2024-09-14T06:32:20.722 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 175.021419697, 2024-09-14T06:32:20.722 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052888859000000003, 2024-09-14T06:32:20.723 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:32:20.723 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:32:20.723 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:32:20.723 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:32:20.723 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:32:20.723 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:20.723 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 11463 2024-09-14T06:32:20.723 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:20.723 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:32:20.723 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:20.723 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:32:20.723 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655639+0000", 2024-09-14T06:32:20.723 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:20.723 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:20.723 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:20.724 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:32:20.724 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655639+0000", 2024-09-14T06:32:20.724 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:20.724 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:20.724 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:20.724 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:32:20.724 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655640+0000", 2024-09-14T06:32:20.724 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.08e-07 2024-09-14T06:32:20.724 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:20.724 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:20.724 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:32:20.724 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655641+0000", 2024-09-14T06:32:20.724 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.156e-06 2024-09-14T06:32:20.724 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:20.725 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:20.725 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:32:20.725 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655648+0000", 2024-09-14T06:32:20.725 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.8460000000000002e-06 2024-09-14T06:32:20.725 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:20.725 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:20.725 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:32:20.725 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655651+0000", 2024-09-14T06:32:20.725 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.0079999999999998e-06 2024-09-14T06:32:20.725 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:20.725 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:20.725 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:32:20.725 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.708429+0000", 2024-09-14T06:32:20.725 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052778459 2024-09-14T06:32:20.725 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:20.726 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:20.726 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:32:20.726 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.708473+0000", 2024-09-14T06:32:20.726 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.3782999999999997e-05 2024-09-14T06:32:20.726 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:20.726 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:20.726 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:32:20.726 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.708528+0000", 2024-09-14T06:32:20.726 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.5198999999999998e-05 2024-09-14T06:32:20.726 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:20.726 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:32:20.726 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:20.726 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:20.726 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:20.726 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:11464 3.1d 3:bcad7303:::rbd_data.112d92027590.0000000000000001:f [read 206848~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:32:20.727 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:25.655654+0000", 2024-09-14T06:32:20.727 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 175.02140482799999, 2024-09-14T06:32:20.727 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052981845999999999, 2024-09-14T06:32:20.727 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:32:20.727 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:32:20.727 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:32:20.727 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:32:20.727 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:32:20.727 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:20.727 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 11464 2024-09-14T06:32:20.727 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:20.727 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:32:20.727 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:20.727 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:32:20.728 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655654+0000", 2024-09-14T06:32:20.728 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:20.728 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:20.728 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:20.728 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:32:20.728 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655654+0000", 2024-09-14T06:32:20.728 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:20.728 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:20.728 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:20.728 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:32:20.728 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655946+0000", 2024-09-14T06:32:20.728 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00029148200000000003 2024-09-14T06:32:20.728 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:20.728 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:20.728 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:32:20.729 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655948+0000", 2024-09-14T06:32:20.729 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.973e-06 2024-09-14T06:32:20.729 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:20.729 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:20.729 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:32:20.729 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655953+0000", 2024-09-14T06:32:20.729 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.1240000000000004e-06 2024-09-14T06:32:20.729 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:20.729 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:20.729 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:32:20.729 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655956+0000", 2024-09-14T06:32:20.729 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.315e-06 2024-09-14T06:32:20.729 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:20.729 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:20.730 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:32:20.730 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.708543+0000", 2024-09-14T06:32:20.730 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052587416999999997 2024-09-14T06:32:20.730 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:20.730 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:20.730 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:32:20.730 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.708584+0000", 2024-09-14T06:32:20.730 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.1004000000000002e-05 2024-09-14T06:32:20.730 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:20.730 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:20.730 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:32:20.730 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.708636+0000", 2024-09-14T06:32:20.730 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.1530999999999998e-05 2024-09-14T06:32:20.730 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:20.730 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:32:20.731 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:20.731 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:20.731 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:20.731 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:11465 3.1d 3:bcad7303:::rbd_data.112d92027590.0000000000000001:f [read 217088~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:32:20.731 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:25.655961+0000", 2024-09-14T06:32:20.731 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 175.02109767600001, 2024-09-14T06:32:20.731 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052798682999999999, 2024-09-14T06:32:20.731 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:32:20.731 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:32:20.731 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:32:20.731 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:32:20.731 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:32:20.731 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:20.731 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 11465 2024-09-14T06:32:20.731 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:20.732 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:32:20.732 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:20.732 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:32:20.732 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655961+0000", 2024-09-14T06:32:20.732 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:20.732 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:20.732 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:20.732 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:32:20.732 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655961+0000", 2024-09-14T06:32:20.732 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:20.732 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:20.732 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:20.732 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:32:20.732 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.656114+0000", 2024-09-14T06:32:20.733 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.000152745 2024-09-14T06:32:20.733 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:20.733 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:20.733 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:32:20.733 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.656116+0000", 2024-09-14T06:32:20.733 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.804e-06 2024-09-14T06:32:20.733 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:20.733 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:20.733 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:32:20.733 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.656120+0000", 2024-09-14T06:32:20.733 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.4700000000000004e-06 2024-09-14T06:32:20.733 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:20.733 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:20.733 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:32:20.734 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.656123+0000", 2024-09-14T06:32:20.734 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.1269999999999999e-06 2024-09-14T06:32:20.734 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:20.734 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:20.734 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:32:20.734 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.708660+0000", 2024-09-14T06:32:20.734 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052536919000000001 2024-09-14T06:32:20.734 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:20.734 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:20.734 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:32:20.734 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.708703+0000", 2024-09-14T06:32:20.734 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.3077000000000001e-05 2024-09-14T06:32:20.734 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:20.734 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:20.734 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:32:20.735 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.708760+0000", 2024-09-14T06:32:20.735 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.6541e-05 2024-09-14T06:32:20.735 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:20.735 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:32:20.735 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:20.735 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:20.735 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:20.735 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:11466 3.1d 3:bcad7303:::rbd_data.112d92027590.0000000000000001:f [read 227328~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:32:20.735 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:25.656129+0000", 2024-09-14T06:32:20.735 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 175.02093028799999, 2024-09-14T06:32:20.735 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052750987999999999, 2024-09-14T06:32:20.735 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:32:20.735 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:32:20.735 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:32:20.736 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:32:20.736 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:32:20.736 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:20.736 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 11466 2024-09-14T06:32:20.736 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:20.736 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:32:20.736 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:20.736 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:32:20.736 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.656129+0000", 2024-09-14T06:32:20.736 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:20.736 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:20.736 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:20.736 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:32:20.736 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.656129+0000", 2024-09-14T06:32:20.737 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:20.737 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:20.737 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:20.737 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:32:20.737 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.656347+0000", 2024-09-14T06:32:20.737 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.000218054 2024-09-14T06:32:20.737 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:20.737 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:20.737 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:32:20.737 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.656349+0000", 2024-09-14T06:32:20.737 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.9199999999999998e-06 2024-09-14T06:32:20.737 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:20.737 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:20.737 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:32:20.737 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.656353+0000", 2024-09-14T06:32:20.738 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.9210000000000002e-06 2024-09-14T06:32:20.738 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:20.738 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:20.738 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:32:20.738 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.656356+0000", 2024-09-14T06:32:20.738 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.7670000000000001e-06 2024-09-14T06:32:20.738 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:20.738 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:20.738 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:32:20.738 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.708787+0000", 2024-09-14T06:32:20.738 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052430840999999999 2024-09-14T06:32:20.738 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:20.738 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:20.738 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:32:20.738 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.708827+0000", 2024-09-14T06:32:20.739 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.9538e-05 2024-09-14T06:32:20.739 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:20.739 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:20.739 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:32:20.739 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.708880+0000", 2024-09-14T06:32:20.739 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.2947e-05 2024-09-14T06:32:20.739 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:20.739 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:32:20.739 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:20.739 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:20.739 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:20.739 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:11467 3.1d 3:bcad7303:::rbd_data.112d92027590.0000000000000001:f [read 237568~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:32:20.739 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:25.656361+0000", 2024-09-14T06:32:20.739 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 175.02069764000001, 2024-09-14T06:32:20.740 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052643898000000001, 2024-09-14T06:32:20.740 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:32:20.740 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:32:20.740 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:32:20.740 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:32:20.740 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:32:20.740 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:20.740 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 11467 2024-09-14T06:32:20.740 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:20.740 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:32:20.740 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:20.740 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:32:20.740 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.656361+0000", 2024-09-14T06:32:20.740 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:20.741 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:20.741 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:20.741 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:32:20.741 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.656361+0000", 2024-09-14T06:32:20.741 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:20.741 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:20.741 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:20.741 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:32:20.741 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.656855+0000", 2024-09-14T06:32:20.741 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00049372000000000001 2024-09-14T06:32:20.741 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:20.741 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:20.741 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:32:20.741 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.656859+0000", 2024-09-14T06:32:20.742 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.1300000000000003e-06 2024-09-14T06:32:20.742 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:20.742 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:20.742 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:32:20.742 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.656863+0000", 2024-09-14T06:32:20.742 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.2629999999999997e-06 2024-09-14T06:32:20.742 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:20.742 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:20.742 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:32:20.742 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.656867+0000", 2024-09-14T06:32:20.742 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.3220000000000001e-06 2024-09-14T06:32:20.742 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:20.742 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:20.742 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:32:20.743 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.708903+0000", 2024-09-14T06:32:20.743 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052035855999999998 2024-09-14T06:32:20.743 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:20.743 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:20.743 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:32:20.743 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.708949+0000", 2024-09-14T06:32:20.743 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.6276999999999998e-05 2024-09-14T06:32:20.743 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:20.743 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:20.743 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:32:20.743 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.709005+0000", 2024-09-14T06:32:20.743 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.6329999999999998e-05 2024-09-14T06:32:20.743 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:20.743 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:32:20.743 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:20.744 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:20.744 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:20.744 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:11480 3.1d 3:bcad7303:::rbd_data.112d92027590.0000000000000001:f [read 370688~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:32:20.744 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:25.658563+0000", 2024-09-14T06:32:20.744 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 175.018495876, 2024-09-14T06:32:20.744 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052429164, 2024-09-14T06:32:20.744 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:32:20.744 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:32:20.744 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:32:20.744 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:32:20.744 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:32:20.744 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:20.744 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 11480 2024-09-14T06:32:20.745 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:20.745 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:32:20.745 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:20.745 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:32:20.745 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.658563+0000", 2024-09-14T06:32:20.745 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:20.745 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:20.745 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:20.745 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:32:20.745 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.658563+0000", 2024-09-14T06:32:20.745 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:20.745 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:20.745 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:20.745 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:32:20.746 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.658658+0000", 2024-09-14T06:32:20.746 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.5346999999999994e-05 2024-09-14T06:32:20.746 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:20.746 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:20.746 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:32:20.746 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.658662+0000", 2024-09-14T06:32:20.746 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.14e-06 2024-09-14T06:32:20.746 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:20.746 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:20.746 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:32:20.746 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.658669+0000", 2024-09-14T06:32:20.746 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.7379999999999994e-06 2024-09-14T06:32:20.746 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:20.746 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:20.746 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:32:20.747 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.658675+0000", 2024-09-14T06:32:20.747 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.5430000000000001e-06 2024-09-14T06:32:20.747 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:20.747 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:20.747 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:32:20.747 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.710824+0000", 2024-09-14T06:32:20.747 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052148847999999998 2024-09-14T06:32:20.747 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:20.747 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:20.747 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:32:20.747 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.710864+0000", 2024-09-14T06:32:20.747 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.9925999999999997e-05 2024-09-14T06:32:20.747 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:20.748 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:20.748 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:32:20.748 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.710992+0000", 2024-09-14T06:32:20.748 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.000128622 2024-09-14T06:32:20.748 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:20.748 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:32:20.748 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:20.748 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:20.748 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:20.748 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:11481 3.1d 3:bcad7303:::rbd_data.112d92027590.0000000000000001:f [read 380928~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:32:20.748 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:25.658684+0000", 2024-09-14T06:32:20.748 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 175.018374806, 2024-09-14T06:32:20.748 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052508565, 2024-09-14T06:32:20.748 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:32:20.748 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:32:20.749 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:32:20.749 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:32:20.749 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:32:20.749 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:20.749 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 11481 2024-09-14T06:32:20.749 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:20.749 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:32:20.749 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:20.749 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:32:20.749 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.658684+0000", 2024-09-14T06:32:20.749 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:20.749 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:20.749 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:20.749 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:32:20.750 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.658684+0000", 2024-09-14T06:32:20.750 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:20.750 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:20.750 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:20.750 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:32:20.750 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.658715+0000", 2024-09-14T06:32:20.750 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.0703000000000001e-05 2024-09-14T06:32:20.750 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:20.750 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:20.750 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:32:20.750 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.658718+0000", 2024-09-14T06:32:20.750 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.0709999999999999e-06 2024-09-14T06:32:20.750 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:20.750 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:20.750 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:32:20.751 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.658725+0000", 2024-09-14T06:32:20.751 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.2339999999999997e-06 2024-09-14T06:32:20.751 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:20.751 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:20.751 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:32:20.751 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.658731+0000", 2024-09-14T06:32:20.751 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.6450000000000002e-06 2024-09-14T06:32:20.751 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:20.751 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:20.751 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:32:20.751 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.711020+0000", 2024-09-14T06:32:20.751 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052288886 2024-09-14T06:32:20.751 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:20.751 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:20.752 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:32:20.752 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.711061+0000", 2024-09-14T06:32:20.752 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.1084000000000002e-05 2024-09-14T06:32:20.752 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:20.752 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:20.752 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:32:20.752 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.711193+0000", 2024-09-14T06:32:20.752 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00013194199999999999 2024-09-14T06:32:20.752 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:20.752 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:32:20.752 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:20.752 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:20.752 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:20.752 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:11482 3.1d 3:bcad7303:::rbd_data.112d92027590.0000000000000001:f [read 391168~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:32:20.753 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:25.658740+0000", 2024-09-14T06:32:20.753 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 175.01831934099999, 2024-09-14T06:32:20.753 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052637015000000002, 2024-09-14T06:32:20.753 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:32:20.753 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:32:20.753 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:32:20.753 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:32:20.753 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:32:20.753 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:20.753 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 11482 2024-09-14T06:32:20.753 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:20.753 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:32:20.753 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:20.753 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:32:20.754 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.658740+0000", 2024-09-14T06:32:20.754 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:20.754 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:20.754 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:20.754 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:32:20.754 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.658740+0000", 2024-09-14T06:32:20.754 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:20.754 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:20.754 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:20.754 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:32:20.754 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.658880+0000", 2024-09-14T06:32:20.754 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00014036100000000001 2024-09-14T06:32:20.754 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:20.754 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:20.755 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:32:20.755 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.658883+0000", 2024-09-14T06:32:20.755 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.3969999999999998e-06 2024-09-14T06:32:20.755 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:20.755 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:20.755 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:32:20.755 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.658890+0000", 2024-09-14T06:32:20.755 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.1500000000000002e-06 2024-09-14T06:32:20.755 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:20.755 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:20.755 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:32:20.755 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.658897+0000", 2024-09-14T06:32:20.755 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.1260000000000003e-06 2024-09-14T06:32:20.755 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:20.756 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:20.756 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:32:20.756 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.711207+0000", 2024-09-14T06:32:20.756 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052310777000000003 2024-09-14T06:32:20.756 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:20.756 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:20.756 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:32:20.756 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.711249+0000", 2024-09-14T06:32:20.756 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.1786999999999998e-05 2024-09-14T06:32:20.756 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:20.756 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:20.756 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:32:20.756 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.711377+0000", 2024-09-14T06:32:20.757 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.000127417 2024-09-14T06:32:20.757 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:20.757 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:32:20.757 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:20.757 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:20.757 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:20.757 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:11483 3.1d 3:bcad7303:::rbd_data.112d92027590.0000000000000001:f [read 401408~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:32:20.757 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:25.658906+0000", 2024-09-14T06:32:20.757 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 175.01815255700001, 2024-09-14T06:32:20.757 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052595109000000001, 2024-09-14T06:32:20.757 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:32:20.757 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:32:20.757 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:32:20.758 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:32:20.758 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:32:20.758 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:20.758 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 11483 2024-09-14T06:32:20.758 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:20.758 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:32:20.758 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:20.758 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:32:20.758 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.658906+0000", 2024-09-14T06:32:20.758 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:20.758 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:20.758 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:20.758 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:32:20.758 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.658906+0000", 2024-09-14T06:32:20.759 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:20.759 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:20.759 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:20.759 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:32:20.759 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.664985+0000", 2024-09-14T06:32:20.759 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.0060787250000000001 2024-09-14T06:32:20.759 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:20.759 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:20.759 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:32:20.759 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.664991+0000", 2024-09-14T06:32:20.759 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.7030000000000003e-06 2024-09-14T06:32:20.759 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:20.759 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:20.759 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:32:20.760 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.664999+0000", 2024-09-14T06:32:20.760 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.4430000000000003e-06 2024-09-14T06:32:20.760 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:20.760 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:20.760 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:32:20.760 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.665006+0000", 2024-09-14T06:32:20.760 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.0840000000000003e-06 2024-09-14T06:32:20.760 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:20.760 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:20.760 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:32:20.760 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.711402+0000", 2024-09-14T06:32:20.760 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046395714999999997 2024-09-14T06:32:20.760 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:20.760 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:20.760 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:32:20.761 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.711443+0000", 2024-09-14T06:32:20.761 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.0911999999999997e-05 2024-09-14T06:32:20.761 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:20.761 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:20.761 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:32:20.761 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.711501+0000", 2024-09-14T06:32:20.761 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.8526999999999997e-05 2024-09-14T06:32:20.761 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:20.761 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:32:20.761 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:20.761 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:20.761 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:20.761 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:214569 3.2 3:479633bd:::rbd_object_map.112d3bd84cb0:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e170)", 2024-09-14T06:32:20.761 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:31:34.933901+0000", 2024-09-14T06:32:20.762 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 45.743158172999998, 2024-09-14T06:32:20.762 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.68157511400000004, 2024-09-14T06:32:20.762 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:32:20.762 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:32:20.762 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:32:20.762 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:32:20.762 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:32:20.762 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:20.762 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 214569 2024-09-14T06:32:20.762 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:20.762 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:32:20.762 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:20.762 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:32:20.762 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:31:34.933901+0000", 2024-09-14T06:32:20.762 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:20.763 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:20.763 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:20.763 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:32:20.763 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:31:34.933901+0000", 2024-09-14T06:32:20.763 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:20.763 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:20.763 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:20.763 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:32:20.763 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:31:34.933902+0000", 2024-09-14T06:32:20.763 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.7439999999999999e-06 2024-09-14T06:32:20.763 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:20.763 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:20.763 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:32:20.763 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:31:34.933910+0000", 2024-09-14T06:32:20.763 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.8569999999999995e-06 2024-09-14T06:32:20.763 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:20.763 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:20.764 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:32:20.764 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:31:34.933932+0000", 2024-09-14T06:32:20.764 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.1997999999999998e-05 2024-09-14T06:32:20.764 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:20.764 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:20.764 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:32:20.764 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:31:34.933941+0000", 2024-09-14T06:32:20.764 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.7180000000000002e-06 2024-09-14T06:32:20.764 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:20.764 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:20.764 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:32:20.764 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:31:34.934013+0000", 2024-09-14T06:32:20.764 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.1761000000000006e-05 2024-09-14T06:32:20.764 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:20.764 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:20.764 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "waiting for peered", 2024-09-14T06:32:20.765 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:31:34.934022+0000", 2024-09-14T06:32:20.765 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.8740000000000001e-06 2024-09-14T06:32:20.765 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:20.765 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:20.765 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:32:20.765 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:31:35.608803+0000", 2024-09-14T06:32:20.765 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.67478164299999999 2024-09-14T06:32:20.765 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:20.765 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:20.765 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:32:20.765 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:31:35.608964+0000", 2024-09-14T06:32:20.765 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00016112900000000001 2024-09-14T06:32:20.765 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:20.765 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:20.766 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "waiting for subops from 6", 2024-09-14T06:32:20.766 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:31:35.609399+0000", 2024-09-14T06:32:20.766 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00043454000000000002 2024-09-14T06:32:20.766 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:20.766 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:20.766 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "op_commit", 2024-09-14T06:32:20.766 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:31:35.610495+0000", 2024-09-14T06:32:20.766 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.001096404 2024-09-14T06:32:20.766 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:20.766 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:20.766 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:32:20.766 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:31:35.615409+0000", 2024-09-14T06:32:20.766 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.0049133199999999997 2024-09-14T06:32:20.766 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:20.766 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:20.767 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "commit_sent", 2024-09-14T06:32:20.767 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:31:35.615444+0000", 2024-09-14T06:32:20.767 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.5457000000000003e-05 2024-09-14T06:32:20.767 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:20.767 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:20.767 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:32:20.767 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:31:35.615476+0000", 2024-09-14T06:32:20.767 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.1668999999999998e-05 2024-09-14T06:32:20.767 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:20.767 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:32:20.767 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:20.767 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:20.767 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:32:20.767 INFO:teuthology.orchestra.run.smithi138.stdout:} 2024-09-14T06:32:20.768 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-14T06:32:20.770 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-14T06:32:20.770 INFO:tasks.ceph.osd.4.smithi138.stderr:2024-09-14T06:32:20.768+0000 7f6c653c8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33615) UID: 0 2024-09-14T06:32:20.870 INFO:tasks.ceph.osd.5.smithi138.stderr:2024-09-14T06:32:20.869+0000 7fe13746f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33618) UID: 0 2024-09-14T06:32:20.914 INFO:teuthology.orchestra.run.smithi138.stdout:{ 2024-09-14T06:32:20.915 INFO:teuthology.orchestra.run.smithi138.stdout: "ops": [], 2024-09-14T06:32:20.915 INFO:teuthology.orchestra.run.smithi138.stdout: "num_ops": 0 2024-09-14T06:32:20.915 INFO:teuthology.orchestra.run.smithi138.stdout:} 2024-09-14T06:32:20.923 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-14T06:32:21.008 INFO:tasks.ceph.osd.5.smithi138.stderr:2024-09-14T06:32:21.007+0000 7fe13746f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33618) UID: 0 2024-09-14T06:32:21.066 INFO:teuthology.orchestra.run.smithi138.stdout:{ 2024-09-14T06:32:21.066 INFO:teuthology.orchestra.run.smithi138.stdout: "ops": [], 2024-09-14T06:32:21.066 INFO:teuthology.orchestra.run.smithi138.stdout: "complaint_time": 30, 2024-09-14T06:32:21.067 INFO:teuthology.orchestra.run.smithi138.stdout: "num_blocked_ops": 0 2024-09-14T06:32:21.067 INFO:teuthology.orchestra.run.smithi138.stdout:} 2024-09-14T06:32:21.071 INFO:tasks.ceph.osd.6.smithi138.stderr:2024-09-14T06:32:21.070+0000 7f8cd8849640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33620) UID: 0 2024-09-14T06:32:21.075 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-14T06:32:21.171 INFO:tasks.ceph.osd.5.smithi138.stderr:2024-09-14T06:32:21.170+0000 7fe13746f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33618) UID: 0 2024-09-14T06:32:21.219 INFO:teuthology.orchestra.run.smithi138.stdout:{ 2024-09-14T06:32:21.220 INFO:teuthology.orchestra.run.smithi138.stdout: "size": 20, 2024-09-14T06:32:21.220 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 600, 2024-09-14T06:32:21.220 INFO:teuthology.orchestra.run.smithi138.stdout: "ops": [ 2024-09-14T06:32:21.220 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:21.220 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:98505 3.10 3.1bf56f90 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T06:32:21.220 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:30:07.045453+0000", 2024-09-14T06:32:21.220 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 134.17271861099999, 2024-09-14T06:32:21.220 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032144440000000003, 2024-09-14T06:32:21.220 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:32:21.220 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:32:21.220 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "reached pg", 2024-09-14T06:32:21.220 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:32:21.221 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:32:21.221 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:21.221 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 98505 2024-09-14T06:32:21.221 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:21.221 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:32:21.221 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:21.221 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:32:21.221 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045453+0000", 2024-09-14T06:32:21.221 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:21.221 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:21.221 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:21.221 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:32:21.221 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045453+0000", 2024-09-14T06:32:21.221 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:21.222 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:21.222 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:21.222 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:32:21.222 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045453+0000", 2024-09-14T06:32:21.222 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.36e-07 2024-09-14T06:32:21.222 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:21.222 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:21.222 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:32:21.222 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045454+0000", 2024-09-14T06:32:21.222 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.1299999999999998e-07 2024-09-14T06:32:21.222 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:21.222 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:21.222 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:32:21.222 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045457+0000", 2024-09-14T06:32:21.223 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.8559999999999998e-06 2024-09-14T06:32:21.223 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:21.223 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:21.223 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:32:21.223 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045460+0000", 2024-09-14T06:32:21.223 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.8750000000000002e-06 2024-09-14T06:32:21.223 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:21.223 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:21.223 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:32:21.223 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.077592+0000", 2024-09-14T06:32:21.223 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032131727999999998 2024-09-14T06:32:21.223 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:21.223 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:21.224 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:32:21.224 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.077597+0000", 2024-09-14T06:32:21.224 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.8319999999999996e-06 2024-09-14T06:32:21.224 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:21.224 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:32:21.224 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:21.224 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:21.224 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:21.224 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:98506 3.10 3.1bf56f90 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T06:32:21.224 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:30:07.045481+0000", 2024-09-14T06:32:21.224 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 134.172690356, 2024-09-14T06:32:21.224 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032142759999999999, 2024-09-14T06:32:21.224 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:32:21.225 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:32:21.225 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "reached pg", 2024-09-14T06:32:21.225 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:32:21.225 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:32:21.225 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:21.225 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 98506 2024-09-14T06:32:21.225 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:21.225 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:32:21.225 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:21.225 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:32:21.225 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045481+0000", 2024-09-14T06:32:21.225 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:21.225 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:21.225 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:21.226 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:32:21.226 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045481+0000", 2024-09-14T06:32:21.226 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:21.226 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:21.226 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:21.226 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:32:21.226 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045481+0000", 2024-09-14T06:32:21.226 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.8500000000000001e-07 2024-09-14T06:32:21.226 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:21.226 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:21.226 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:32:21.226 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045483+0000", 2024-09-14T06:32:21.226 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.8250000000000001e-06 2024-09-14T06:32:21.226 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:21.227 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:21.227 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:32:21.227 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045489+0000", 2024-09-14T06:32:21.227 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.5740000000000003e-06 2024-09-14T06:32:21.227 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:21.227 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:21.227 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:32:21.227 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045495+0000", 2024-09-14T06:32:21.227 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.7690000000000002e-06 2024-09-14T06:32:21.227 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:21.227 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:21.227 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:32:21.227 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.077619+0000", 2024-09-14T06:32:21.227 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032124621999999999 2024-09-14T06:32:21.227 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:21.228 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:21.228 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:32:21.228 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.077624+0000", 2024-09-14T06:32:21.228 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.3850000000000004e-06 2024-09-14T06:32:21.228 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:21.228 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:32:21.228 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:21.228 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:21.228 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:21.228 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:98507 3.10 3.1bf56f90 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T06:32:21.228 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:30:07.045502+0000", 2024-09-14T06:32:21.228 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 134.17266893499999, 2024-09-14T06:32:21.228 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032147219999999997, 2024-09-14T06:32:21.228 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:32:21.228 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:32:21.229 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "reached pg", 2024-09-14T06:32:21.229 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:32:21.229 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:32:21.229 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:21.229 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 98507 2024-09-14T06:32:21.229 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:21.229 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:32:21.229 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:21.229 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:32:21.229 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045502+0000", 2024-09-14T06:32:21.229 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:21.229 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:21.229 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:21.229 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:32:21.230 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045502+0000", 2024-09-14T06:32:21.230 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:21.230 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:21.230 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:21.230 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:32:21.230 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045503+0000", 2024-09-14T06:32:21.230 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.4700000000000002e-07 2024-09-14T06:32:21.230 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:21.230 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:21.230 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:32:21.231 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045509+0000", 2024-09-14T06:32:21.231 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.7329999999999997e-06 2024-09-14T06:32:21.231 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:21.231 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:21.231 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:32:21.231 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045515+0000", 2024-09-14T06:32:21.231 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.8030000000000002e-06 2024-09-14T06:32:21.231 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:21.231 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:21.231 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:32:21.231 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045522+0000", 2024-09-14T06:32:21.231 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.0369999999999997e-06 2024-09-14T06:32:21.231 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:21.231 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:21.232 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:32:21.232 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.077641+0000", 2024-09-14T06:32:21.232 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032119091000000002 2024-09-14T06:32:21.232 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:21.232 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:21.232 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:32:21.232 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.077650+0000", 2024-09-14T06:32:21.232 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.2090000000000005e-06 2024-09-14T06:32:21.232 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:21.232 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:32:21.232 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:21.232 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:21.232 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:21.232 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:98508 3.10 3.1bf56f90 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T06:32:21.232 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:30:07.045530+0000", 2024-09-14T06:32:21.233 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 134.17264131799999, 2024-09-14T06:32:21.233 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032198773999999999, 2024-09-14T06:32:21.233 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:32:21.233 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:32:21.233 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "reached pg", 2024-09-14T06:32:21.233 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:32:21.233 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:32:21.233 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:21.233 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 98508 2024-09-14T06:32:21.233 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:21.233 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:32:21.233 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:21.233 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:32:21.233 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045530+0000", 2024-09-14T06:32:21.234 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:21.234 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:21.234 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:21.234 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:32:21.234 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045530+0000", 2024-09-14T06:32:21.234 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:21.234 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:21.234 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:21.234 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:32:21.234 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045530+0000", 2024-09-14T06:32:21.234 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.3099999999999998e-07 2024-09-14T06:32:21.234 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:21.234 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:21.235 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:32:21.235 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045539+0000", 2024-09-14T06:32:21.235 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.4780000000000008e-06 2024-09-14T06:32:21.235 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:21.235 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:21.235 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:32:21.235 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045546+0000", 2024-09-14T06:32:21.235 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.3259999999999998e-06 2024-09-14T06:32:21.235 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:21.235 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:21.235 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:32:21.235 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045551+0000", 2024-09-14T06:32:21.235 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.9960000000000001e-06 2024-09-14T06:32:21.236 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:21.236 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:21.236 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:32:21.236 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.077716+0000", 2024-09-14T06:32:21.236 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032165081999999998 2024-09-14T06:32:21.236 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:21.236 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:21.236 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:32:21.236 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.077729+0000", 2024-09-14T06:32:21.236 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.2461e-05 2024-09-14T06:32:21.236 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:21.236 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:32:21.236 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:21.236 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:21.236 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:21.237 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:98509 3.10 3.1bf56f90 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T06:32:21.237 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:30:07.045558+0000", 2024-09-14T06:32:21.237 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 134.172613117, 2024-09-14T06:32:21.237 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032242198, 2024-09-14T06:32:21.237 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:32:21.237 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:32:21.237 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "reached pg", 2024-09-14T06:32:21.237 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:32:21.237 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:32:21.237 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:21.237 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 98509 2024-09-14T06:32:21.237 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:21.237 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:32:21.238 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:21.238 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:32:21.238 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045558+0000", 2024-09-14T06:32:21.238 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:21.238 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:21.238 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:21.238 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:32:21.238 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045558+0000", 2024-09-14T06:32:21.238 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:21.238 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:21.239 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:21.239 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:32:21.239 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045559+0000", 2024-09-14T06:32:21.239 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.27e-07 2024-09-14T06:32:21.239 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:21.239 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:21.239 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:32:21.240 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045560+0000", 2024-09-14T06:32:21.240 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.6959999999999999e-06 2024-09-14T06:32:21.240 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:21.240 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:21.240 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:32:21.240 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045567+0000", 2024-09-14T06:32:21.241 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.6900000000000003e-06 2024-09-14T06:32:21.241 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:21.241 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:21.241 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:32:21.241 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045571+0000", 2024-09-14T06:32:21.241 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.5750000000000002e-06 2024-09-14T06:32:21.241 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:21.241 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:21.241 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:32:21.241 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.077789+0000", 2024-09-14T06:32:21.242 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032217186000000002 2024-09-14T06:32:21.242 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:21.242 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:21.242 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:32:21.242 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.077800+0000", 2024-09-14T06:32:21.242 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.1623999999999999e-05 2024-09-14T06:32:21.242 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:21.242 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:32:21.242 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:21.242 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:21.242 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:21.242 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:98510 3.10 3.1bf56f90 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T06:32:21.242 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:30:07.045580+0000", 2024-09-14T06:32:21.243 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 134.17259130299999, 2024-09-14T06:32:21.243 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032299845000000001, 2024-09-14T06:32:21.243 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:32:21.243 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:32:21.243 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "reached pg", 2024-09-14T06:32:21.243 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:32:21.243 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:32:21.243 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:21.243 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 98510 2024-09-14T06:32:21.244 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:21.244 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:32:21.244 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:21.244 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:32:21.244 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045580+0000", 2024-09-14T06:32:21.244 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:21.244 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:21.244 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:21.244 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:32:21.245 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045580+0000", 2024-09-14T06:32:21.245 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:21.245 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:21.245 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:21.245 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:32:21.245 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045580+0000", 2024-09-14T06:32:21.245 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.0400000000000002e-07 2024-09-14T06:32:21.245 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:21.245 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:21.245 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:32:21.245 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045582+0000", 2024-09-14T06:32:21.245 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.7600000000000001e-06 2024-09-14T06:32:21.245 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:21.246 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:21.246 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:32:21.246 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045588+0000", 2024-09-14T06:32:21.246 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.5650000000000001e-06 2024-09-14T06:32:21.246 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:21.246 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:21.246 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:32:21.246 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045597+0000", 2024-09-14T06:32:21.246 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.465e-06 2024-09-14T06:32:21.246 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:21.246 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:21.246 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:32:21.247 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.077871+0000", 2024-09-14T06:32:21.247 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032273562999999998 2024-09-14T06:32:21.247 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:21.247 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:21.247 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:32:21.247 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.077880+0000", 2024-09-14T06:32:21.247 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.0880000000000001e-06 2024-09-14T06:32:21.247 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:21.247 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:32:21.247 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:21.247 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:21.247 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:21.247 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:98511 3.10 3.1bf56f90 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T06:32:21.247 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:30:07.045610+0000", 2024-09-14T06:32:21.248 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 134.17256145600001, 2024-09-14T06:32:21.248 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032338894, 2024-09-14T06:32:21.248 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:32:21.248 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:32:21.248 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "reached pg", 2024-09-14T06:32:21.248 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:32:21.248 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:32:21.248 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:21.248 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 98511 2024-09-14T06:32:21.248 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:21.248 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:32:21.248 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:21.248 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:32:21.248 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045610+0000", 2024-09-14T06:32:21.249 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:21.249 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:21.249 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:21.249 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:32:21.249 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045610+0000", 2024-09-14T06:32:21.249 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:21.249 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:21.249 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:21.249 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:32:21.249 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045610+0000", 2024-09-14T06:32:21.249 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.1600000000000002e-07 2024-09-14T06:32:21.249 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:21.249 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:21.250 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:32:21.250 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045612+0000", 2024-09-14T06:32:21.250 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.1150000000000001e-06 2024-09-14T06:32:21.250 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:21.250 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:21.250 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:32:21.250 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045618+0000", 2024-09-14T06:32:21.250 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.0499999999999997e-06 2024-09-14T06:32:21.250 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:21.250 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:21.250 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:32:21.250 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045623+0000", 2024-09-14T06:32:21.251 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.6770000000000003e-06 2024-09-14T06:32:21.251 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:21.251 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:21.251 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:32:21.251 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.077928+0000", 2024-09-14T06:32:21.251 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032305111999999997 2024-09-14T06:32:21.251 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:21.251 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:21.251 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:32:21.251 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.077949+0000", 2024-09-14T06:32:21.251 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.0523999999999999e-05 2024-09-14T06:32:21.251 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:21.251 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:32:21.252 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:21.252 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:21.252 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:21.252 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:98512 3.10 3.1bf56f90 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T06:32:21.252 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:30:07.045630+0000", 2024-09-14T06:32:21.252 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 134.172541451, 2024-09-14T06:32:21.252 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032346632, 2024-09-14T06:32:21.252 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:32:21.252 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:32:21.252 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "reached pg", 2024-09-14T06:32:21.252 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:32:21.253 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:32:21.253 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:21.253 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 98512 2024-09-14T06:32:21.253 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:21.253 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:32:21.253 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:21.253 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:32:21.253 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045630+0000", 2024-09-14T06:32:21.253 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:21.253 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:21.254 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:21.254 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:32:21.254 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045630+0000", 2024-09-14T06:32:21.254 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:21.254 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:21.254 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:21.254 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:32:21.254 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045630+0000", 2024-09-14T06:32:21.254 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.8200000000000001e-07 2024-09-14T06:32:21.254 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:21.254 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:21.254 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:32:21.254 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045632+0000", 2024-09-14T06:32:21.254 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.7430000000000001e-06 2024-09-14T06:32:21.255 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:21.255 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:21.255 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:32:21.255 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045637+0000", 2024-09-14T06:32:21.255 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.4550000000000003e-06 2024-09-14T06:32:21.255 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:21.255 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:21.255 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:32:21.255 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045644+0000", 2024-09-14T06:32:21.255 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.1330000000000001e-06 2024-09-14T06:32:21.255 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:21.255 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:21.255 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:32:21.255 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.077968+0000", 2024-09-14T06:32:21.255 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032323401000000002 2024-09-14T06:32:21.256 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:21.256 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:21.256 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:32:21.256 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.077976+0000", 2024-09-14T06:32:21.256 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.5180000000000004e-06 2024-09-14T06:32:21.256 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:21.256 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:32:21.256 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:21.256 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:21.256 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:21.256 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:98513 3.10 3.1bf56f90 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T06:32:21.256 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:30:07.045652+0000", 2024-09-14T06:32:21.256 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 134.172518887, 2024-09-14T06:32:21.256 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032353436999999999, 2024-09-14T06:32:21.257 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:32:21.257 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:32:21.257 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "reached pg", 2024-09-14T06:32:21.257 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:32:21.257 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:32:21.257 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:21.257 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 98513 2024-09-14T06:32:21.257 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:21.257 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:32:21.258 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:21.258 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:32:21.258 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045652+0000", 2024-09-14T06:32:21.258 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:21.258 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:21.258 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:21.258 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:32:21.259 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045652+0000", 2024-09-14T06:32:21.259 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:21.259 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:21.259 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:21.259 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:32:21.259 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045653+0000", 2024-09-14T06:32:21.260 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.3700000000000001e-07 2024-09-14T06:32:21.260 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:21.260 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:21.260 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:32:21.260 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045655+0000", 2024-09-14T06:32:21.260 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.854e-06 2024-09-14T06:32:21.260 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:21.260 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:21.260 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:32:21.260 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045661+0000", 2024-09-14T06:32:21.261 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.5509999999999996e-06 2024-09-14T06:32:21.261 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:21.261 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:21.261 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:32:21.261 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045668+0000", 2024-09-14T06:32:21.261 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.1980000000000004e-06 2024-09-14T06:32:21.261 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:21.262 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:21.262 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:32:21.262 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.077997+0000", 2024-09-14T06:32:21.262 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032328575999999998 2024-09-14T06:32:21.262 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:21.262 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:21.262 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:32:21.263 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.078006+0000", 2024-09-14T06:32:21.263 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.8210000000000008e-06 2024-09-14T06:32:21.263 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:21.263 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:32:21.263 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:21.263 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:21.263 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:21.263 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:98514 3.10 3.1bf56f90 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T06:32:21.264 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:30:07.045699+0000", 2024-09-14T06:32:21.264 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 134.17247255199999, 2024-09-14T06:32:21.264 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032330223999999998, 2024-09-14T06:32:21.264 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:32:21.264 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:32:21.264 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "reached pg", 2024-09-14T06:32:21.264 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:32:21.264 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:32:21.264 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:21.264 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 98514 2024-09-14T06:32:21.264 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:21.264 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:32:21.265 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:21.265 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:32:21.265 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045699+0000", 2024-09-14T06:32:21.265 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:21.265 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:21.265 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:21.265 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:32:21.265 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045699+0000", 2024-09-14T06:32:21.265 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:21.265 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:21.265 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:21.265 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:32:21.265 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045699+0000", 2024-09-14T06:32:21.266 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.3200000000000001e-07 2024-09-14T06:32:21.266 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:21.266 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:21.266 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:32:21.266 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045701+0000", 2024-09-14T06:32:21.266 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.762e-06 2024-09-14T06:32:21.266 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:21.266 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:21.266 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:32:21.266 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045705+0000", 2024-09-14T06:32:21.266 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.2359999999999998e-06 2024-09-14T06:32:21.266 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:21.266 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:21.266 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:32:21.266 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045708+0000", 2024-09-14T06:32:21.267 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.796e-06 2024-09-14T06:32:21.267 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:21.267 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:21.267 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:32:21.267 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.078023+0000", 2024-09-14T06:32:21.267 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032315213000000002 2024-09-14T06:32:21.267 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:21.267 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:21.267 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:32:21.267 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.078029+0000", 2024-09-14T06:32:21.267 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.8850000000000004e-06 2024-09-14T06:32:21.267 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:21.267 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:32:21.267 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:21.267 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:21.268 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:21.268 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:98515 3.10 3.1bf56f90 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T06:32:21.268 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:30:07.045712+0000", 2024-09-14T06:32:21.268 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 134.17245924100001, 2024-09-14T06:32:21.268 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032346871999999999, 2024-09-14T06:32:21.268 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:32:21.268 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:32:21.268 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "reached pg", 2024-09-14T06:32:21.268 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:32:21.268 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:32:21.268 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:21.268 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 98515 2024-09-14T06:32:21.268 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:21.269 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:32:21.269 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:21.269 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:32:21.269 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045712+0000", 2024-09-14T06:32:21.269 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:21.269 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:21.269 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:21.269 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:32:21.269 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045712+0000", 2024-09-14T06:32:21.269 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:21.269 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:21.269 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:21.270 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:32:21.270 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045712+0000", 2024-09-14T06:32:21.270 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.2100000000000001e-07 2024-09-14T06:32:21.270 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:21.270 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:21.270 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:32:21.270 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045713+0000", 2024-09-14T06:32:21.270 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.068e-06 2024-09-14T06:32:21.270 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:21.270 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:21.270 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:32:21.270 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045716+0000", 2024-09-14T06:32:21.271 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.971e-06 2024-09-14T06:32:21.271 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:21.271 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:21.271 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:32:21.271 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045719+0000", 2024-09-14T06:32:21.271 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.3879999999999998e-06 2024-09-14T06:32:21.271 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:21.271 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:21.271 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:32:21.271 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.078049+0000", 2024-09-14T06:32:21.271 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032330553999999997 2024-09-14T06:32:21.271 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:21.271 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:21.272 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:32:21.272 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.078059+0000", 2024-09-14T06:32:21.272 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.6700000000000006e-06 2024-09-14T06:32:21.272 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:21.272 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:32:21.272 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:21.272 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:21.272 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:21.272 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:98516 3.10 3.1bf56f90 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T06:32:21.272 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:30:07.045722+0000", 2024-09-14T06:32:21.272 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 134.17244948800001, 2024-09-14T06:32:21.272 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032374334999999997, 2024-09-14T06:32:21.272 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:32:21.272 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:32:21.273 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "reached pg", 2024-09-14T06:32:21.273 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:32:21.273 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:32:21.273 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:21.273 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 98516 2024-09-14T06:32:21.273 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:21.273 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:32:21.273 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:21.273 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:32:21.273 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045722+0000", 2024-09-14T06:32:21.273 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:21.273 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:21.273 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:21.273 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:32:21.274 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045722+0000", 2024-09-14T06:32:21.274 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:21.274 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:21.274 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:21.274 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:32:21.274 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045722+0000", 2024-09-14T06:32:21.274 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.8999999999999998e-07 2024-09-14T06:32:21.274 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:21.274 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:21.274 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:32:21.274 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045723+0000", 2024-09-14T06:32:21.274 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.0049999999999999e-06 2024-09-14T06:32:21.274 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:21.275 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:21.275 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:32:21.275 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045726+0000", 2024-09-14T06:32:21.275 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.9019999999999999e-06 2024-09-14T06:32:21.275 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:21.275 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:21.275 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:32:21.275 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045728+0000", 2024-09-14T06:32:21.275 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.3290000000000001e-06 2024-09-14T06:32:21.275 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:21.275 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:21.275 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:32:21.275 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.078090+0000", 2024-09-14T06:32:21.275 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032362123 2024-09-14T06:32:21.276 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:21.276 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:21.276 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:32:21.276 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.078096+0000", 2024-09-14T06:32:21.276 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.6860000000000003e-06 2024-09-14T06:32:21.276 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:21.276 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:32:21.276 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:21.276 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:21.276 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:21.276 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:98517 3.10 3.1bf56f90 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T06:32:21.276 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:30:07.045731+0000", 2024-09-14T06:32:21.276 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 134.17243975700001, 2024-09-14T06:32:21.276 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.03241227, 2024-09-14T06:32:21.277 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:32:21.277 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:32:21.277 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "reached pg", 2024-09-14T06:32:21.277 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:32:21.277 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:32:21.277 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:21.277 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 98517 2024-09-14T06:32:21.277 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:21.277 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:32:21.277 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:21.277 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:32:21.277 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045731+0000", 2024-09-14T06:32:21.277 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:21.278 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:21.278 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:21.278 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:32:21.278 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045731+0000", 2024-09-14T06:32:21.278 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:21.278 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:21.278 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:21.278 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:32:21.278 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045732+0000", 2024-09-14T06:32:21.278 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.0699999999999999e-07 2024-09-14T06:32:21.278 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:21.278 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:21.278 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:32:21.278 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045733+0000", 2024-09-14T06:32:21.278 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.9199999999999999e-07 2024-09-14T06:32:21.279 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:21.279 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:21.279 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:32:21.279 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045735+0000", 2024-09-14T06:32:21.279 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.8150000000000002e-06 2024-09-14T06:32:21.279 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:21.279 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:21.279 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:32:21.279 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045738+0000", 2024-09-14T06:32:21.279 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.3640000000000002e-06 2024-09-14T06:32:21.279 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:21.279 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:21.279 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:32:21.279 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.078126+0000", 2024-09-14T06:32:21.280 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032388206000000003 2024-09-14T06:32:21.280 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:21.280 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:21.280 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:32:21.280 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.078144+0000", 2024-09-14T06:32:21.280 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.7686e-05 2024-09-14T06:32:21.280 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:21.280 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:32:21.280 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:21.280 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:21.280 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:21.280 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:98518 3.10 3.1bf56f90 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T06:32:21.280 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:30:07.045741+0000", 2024-09-14T06:32:21.280 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 134.172430362, 2024-09-14T06:32:21.280 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032451793999999999, 2024-09-14T06:32:21.281 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:32:21.281 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:32:21.281 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "reached pg", 2024-09-14T06:32:21.281 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:32:21.281 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:32:21.281 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:21.281 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 98518 2024-09-14T06:32:21.281 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:21.281 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:32:21.281 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:21.281 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:32:21.281 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045741+0000", 2024-09-14T06:32:21.282 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:21.282 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:21.282 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:21.282 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:32:21.282 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045741+0000", 2024-09-14T06:32:21.282 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:21.282 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:21.282 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:21.282 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:32:21.282 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045741+0000", 2024-09-14T06:32:21.282 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.05e-07 2024-09-14T06:32:21.282 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:21.282 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:21.282 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:32:21.282 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045746+0000", 2024-09-14T06:32:21.283 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.2259999999999996e-06 2024-09-14T06:32:21.283 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:21.283 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:21.283 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:32:21.283 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045750+0000", 2024-09-14T06:32:21.283 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.2490000000000001e-06 2024-09-14T06:32:21.283 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:21.283 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:21.283 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:32:21.283 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045752+0000", 2024-09-14T06:32:21.283 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.3980000000000002e-06 2024-09-14T06:32:21.283 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:21.283 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:21.283 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:32:21.283 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.078184+0000", 2024-09-14T06:32:21.283 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032431593000000002 2024-09-14T06:32:21.284 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:21.284 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:21.284 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:32:21.284 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.078193+0000", 2024-09-14T06:32:21.284 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.1230000000000006e-06 2024-09-14T06:32:21.284 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:21.284 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:32:21.284 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:21.284 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:21.284 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:21.284 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:98519 3.10 3.1bf56f90 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T06:32:21.284 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:30:07.045755+0000", 2024-09-14T06:32:21.284 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 134.17241593700001, 2024-09-14T06:32:21.284 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032489431999999999, 2024-09-14T06:32:21.285 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:32:21.285 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:32:21.285 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "reached pg", 2024-09-14T06:32:21.285 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:32:21.285 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:32:21.285 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:21.285 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 98519 2024-09-14T06:32:21.285 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:21.285 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:32:21.285 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:21.285 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:32:21.285 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045755+0000", 2024-09-14T06:32:21.285 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:21.285 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:21.286 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:21.286 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:32:21.286 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045755+0000", 2024-09-14T06:32:21.286 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:21.286 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:21.286 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:21.286 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:32:21.286 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045756+0000", 2024-09-14T06:32:21.286 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.23e-07 2024-09-14T06:32:21.286 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:21.286 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:21.286 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:32:21.286 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045757+0000", 2024-09-14T06:32:21.286 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.1060000000000001e-06 2024-09-14T06:32:21.286 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:21.287 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:21.287 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:32:21.287 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045760+0000", 2024-09-14T06:32:21.287 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.988e-06 2024-09-14T06:32:21.287 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:21.287 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:21.287 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:32:21.287 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045765+0000", 2024-09-14T06:32:21.287 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.2689999999999999e-06 2024-09-14T06:32:21.287 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:21.288 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:21.288 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:32:21.288 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.078234+0000", 2024-09-14T06:32:21.288 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032468915000000001 2024-09-14T06:32:21.288 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:21.288 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:21.288 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:32:21.288 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.078245+0000", 2024-09-14T06:32:21.288 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.0930999999999999e-05 2024-09-14T06:32:21.288 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:21.288 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:32:21.288 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:21.288 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:21.288 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:21.289 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:98520 3.10 3.1bf56f90 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T06:32:21.289 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:30:07.045768+0000", 2024-09-14T06:32:21.289 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 134.172403059, 2024-09-14T06:32:21.289 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032531379999999999, 2024-09-14T06:32:21.289 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:32:21.289 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:32:21.289 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "reached pg", 2024-09-14T06:32:21.289 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:32:21.289 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:32:21.289 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:21.289 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 98520 2024-09-14T06:32:21.289 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:21.289 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:32:21.290 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:21.290 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:32:21.290 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045768+0000", 2024-09-14T06:32:21.290 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:21.290 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:21.290 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:21.290 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:32:21.290 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045768+0000", 2024-09-14T06:32:21.290 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:21.290 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:21.290 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:21.290 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:32:21.290 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045768+0000", 2024-09-14T06:32:21.290 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.22e-07 2024-09-14T06:32:21.291 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:21.291 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:21.291 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:32:21.291 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045773+0000", 2024-09-14T06:32:21.291 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.3470000000000001e-06 2024-09-14T06:32:21.291 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:21.291 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:21.291 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:32:21.291 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045776+0000", 2024-09-14T06:32:21.291 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.1319999999999998e-06 2024-09-14T06:32:21.291 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:21.291 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:21.291 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:32:21.291 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045779+0000", 2024-09-14T06:32:21.291 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.9749999999999999e-06 2024-09-14T06:32:21.292 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:21.292 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:21.292 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:32:21.292 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.078289+0000", 2024-09-14T06:32:21.292 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032509693999999999 2024-09-14T06:32:21.292 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:21.292 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:21.292 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:32:21.292 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.078300+0000", 2024-09-14T06:32:21.292 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.101e-05 2024-09-14T06:32:21.292 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:21.292 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:32:21.292 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:21.293 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:21.293 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:21.293 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:98521 3.10 3.1bf56f90 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T06:32:21.293 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:30:07.045782+0000", 2024-09-14T06:32:21.293 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 134.17238887900001, 2024-09-14T06:32:21.293 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032566109000000003, 2024-09-14T06:32:21.293 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:32:21.293 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:32:21.293 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "reached pg", 2024-09-14T06:32:21.293 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:32:21.293 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:32:21.293 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:21.294 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 98521 2024-09-14T06:32:21.294 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:21.294 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:32:21.294 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:21.294 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:32:21.294 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045782+0000", 2024-09-14T06:32:21.294 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:21.294 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:21.294 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:21.294 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:32:21.294 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045782+0000", 2024-09-14T06:32:21.294 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:21.294 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:21.295 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:21.295 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:32:21.295 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045783+0000", 2024-09-14T06:32:21.295 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.2499999999999999e-07 2024-09-14T06:32:21.295 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:21.295 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:21.295 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:32:21.295 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045784+0000", 2024-09-14T06:32:21.295 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.0109999999999999e-06 2024-09-14T06:32:21.295 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:21.295 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:21.295 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:32:21.295 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045786+0000", 2024-09-14T06:32:21.295 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.8289999999999999e-06 2024-09-14T06:32:21.295 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:21.295 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:21.296 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:32:21.296 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045793+0000", 2024-09-14T06:32:21.296 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.579e-06 2024-09-14T06:32:21.296 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:21.296 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:21.296 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:32:21.296 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.078338+0000", 2024-09-14T06:32:21.296 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032545389000000001 2024-09-14T06:32:21.296 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:21.296 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:21.296 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:32:21.296 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.078348+0000", 2024-09-14T06:32:21.296 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.0076e-05 2024-09-14T06:32:21.296 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:21.296 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:32:21.296 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:21.297 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:21.297 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:21.297 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:98522 3.10 3.1bf56f90 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T06:32:21.297 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:30:07.045796+0000", 2024-09-14T06:32:21.297 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 134.17237497100001, 2024-09-14T06:32:21.297 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032577846000000001, 2024-09-14T06:32:21.297 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:32:21.297 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:32:21.297 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "reached pg", 2024-09-14T06:32:21.298 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:32:21.298 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:32:21.298 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:21.298 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 98522 2024-09-14T06:32:21.298 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:21.298 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:32:21.298 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:21.298 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:32:21.298 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045796+0000", 2024-09-14T06:32:21.298 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:21.298 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:21.298 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:21.298 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:32:21.299 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045796+0000", 2024-09-14T06:32:21.299 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:21.299 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:21.299 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:21.299 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:32:21.299 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045796+0000", 2024-09-14T06:32:21.299 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.2100000000000001e-07 2024-09-14T06:32:21.299 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:21.299 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:21.299 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:32:21.299 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045797+0000", 2024-09-14T06:32:21.299 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.6899999999999996e-07 2024-09-14T06:32:21.299 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:21.299 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:21.299 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:32:21.300 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045800+0000", 2024-09-14T06:32:21.300 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.9979999999999999e-06 2024-09-14T06:32:21.300 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:21.300 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:21.300 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:32:21.300 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045803+0000", 2024-09-14T06:32:21.300 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.2850000000000001e-06 2024-09-14T06:32:21.300 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:21.300 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:21.300 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:32:21.300 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.078366+0000", 2024-09-14T06:32:21.301 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032562932000000003 2024-09-14T06:32:21.301 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:21.301 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:21.301 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:32:21.301 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.078374+0000", 2024-09-14T06:32:21.301 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.4409999999999993e-06 2024-09-14T06:32:21.301 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:21.301 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:32:21.301 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:21.301 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:21.301 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:21.301 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:98523 3.10 3.1bf56f90 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T06:32:21.301 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:30:07.045806+0000", 2024-09-14T06:32:21.302 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 134.17236542800001, 2024-09-14T06:32:21.302 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032611167000000003, 2024-09-14T06:32:21.302 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:32:21.302 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:32:21.302 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "reached pg", 2024-09-14T06:32:21.302 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:32:21.302 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:32:21.302 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:21.302 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 98523 2024-09-14T06:32:21.302 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:21.302 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:32:21.302 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:21.302 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:32:21.302 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045806+0000", 2024-09-14T06:32:21.302 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:21.302 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:21.303 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:21.303 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:32:21.303 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045806+0000", 2024-09-14T06:32:21.303 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:21.303 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:21.303 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:21.303 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:32:21.303 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045806+0000", 2024-09-14T06:32:21.303 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.2000000000000001e-07 2024-09-14T06:32:21.303 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:21.304 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:21.304 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:32:21.304 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045807+0000", 2024-09-14T06:32:21.304 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.4e-07 2024-09-14T06:32:21.304 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:21.304 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:21.304 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:32:21.304 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045810+0000", 2024-09-14T06:32:21.304 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.9229999999999998e-06 2024-09-14T06:32:21.304 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:21.304 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:21.304 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:32:21.304 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045812+0000", 2024-09-14T06:32:21.305 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.2220000000000001e-06 2024-09-14T06:32:21.305 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:21.305 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:21.305 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:32:21.305 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.078407+0000", 2024-09-14T06:32:21.305 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032594513999999998 2024-09-14T06:32:21.305 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:21.305 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:21.305 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:32:21.305 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.078417+0000", 2024-09-14T06:32:21.305 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.0348e-05 2024-09-14T06:32:21.306 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:21.306 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:32:21.306 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:21.306 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:21.306 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:21.306 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:98524 3.10 3.1bf56f90 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T06:32:21.306 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:30:07.045815+0000", 2024-09-14T06:32:21.306 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 134.17235606, 2024-09-14T06:32:21.306 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032643208999999999, 2024-09-14T06:32:21.306 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:32:21.306 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:32:21.306 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "reached pg", 2024-09-14T06:32:21.306 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:32:21.306 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:32:21.307 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:21.307 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 98524 2024-09-14T06:32:21.307 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:21.307 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:32:21.307 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:21.307 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:32:21.307 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045815+0000", 2024-09-14T06:32:21.307 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:21.307 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:21.307 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:21.307 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:32:21.307 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045815+0000", 2024-09-14T06:32:21.307 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:21.308 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:21.308 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:21.308 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:32:21.308 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045815+0000", 2024-09-14T06:32:21.308 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.11e-07 2024-09-14T06:32:21.308 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:21.308 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:21.308 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:32:21.308 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045816+0000", 2024-09-14T06:32:21.308 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.9099999999999991e-07 2024-09-14T06:32:21.308 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:21.308 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:21.308 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:32:21.308 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045819+0000", 2024-09-14T06:32:21.308 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.9399999999999998e-06 2024-09-14T06:32:21.308 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:21.309 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:21.309 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:32:21.309 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045822+0000", 2024-09-14T06:32:21.309 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.3470000000000001e-06 2024-09-14T06:32:21.309 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:21.309 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:21.309 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:32:21.309 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.078449+0000", 2024-09-14T06:32:21.309 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032627529000000002 2024-09-14T06:32:21.309 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:21.309 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:21.309 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:32:21.309 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.078458+0000", 2024-09-14T06:32:21.309 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.1910000000000006e-06 2024-09-14T06:32:21.310 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:21.310 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:32:21.310 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:21.310 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:21.310 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:32:21.310 INFO:teuthology.orchestra.run.smithi138.stdout:} 2024-09-14T06:32:21.311 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-14T06:32:21.312 INFO:tasks.ceph.osd.6.smithi138.stderr:2024-09-14T06:32:21.310+0000 7f8cd8849640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33620) UID: 0 2024-09-14T06:32:21.411 INFO:tasks.ceph.osd.0.smithi094.stderr:2024-09-14T06:32:21.411+0000 7f07c9333640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33636) UID: 0 2024-09-14T06:32:21.454 INFO:teuthology.orchestra.run.smithi138.stdout:{ 2024-09-14T06:32:21.454 INFO:teuthology.orchestra.run.smithi138.stdout: "ops": [], 2024-09-14T06:32:21.454 INFO:teuthology.orchestra.run.smithi138.stdout: "num_ops": 0 2024-09-14T06:32:21.454 INFO:teuthology.orchestra.run.smithi138.stdout:} 2024-09-14T06:32:21.463 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-14T06:32:21.512 INFO:tasks.ceph.osd.3.smithi094.stderr:2024-09-14T06:32:21.511+0000 7feeeafe8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33655) UID: 0 2024-09-14T06:32:21.566 INFO:teuthology.orchestra.run.smithi138.stdout:{ 2024-09-14T06:32:21.566 INFO:teuthology.orchestra.run.smithi138.stdout: "ops": [], 2024-09-14T06:32:21.566 INFO:teuthology.orchestra.run.smithi138.stdout: "complaint_time": 30, 2024-09-14T06:32:21.566 INFO:teuthology.orchestra.run.smithi138.stdout: "num_blocked_ops": 0 2024-09-14T06:32:21.566 INFO:teuthology.orchestra.run.smithi138.stdout:} 2024-09-14T06:32:21.575 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-14T06:32:21.613 INFO:tasks.ceph.osd.3.smithi094.stderr:2024-09-14T06:32:21.612+0000 7feeeafe8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33655) UID: 0 2024-09-14T06:32:21.680 INFO:teuthology.orchestra.run.smithi138.stdout:{ 2024-09-14T06:32:21.680 INFO:teuthology.orchestra.run.smithi138.stdout: "size": 20, 2024-09-14T06:32:21.680 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 600, 2024-09-14T06:32:21.680 INFO:teuthology.orchestra.run.smithi138.stdout: "ops": [ 2024-09-14T06:32:21.680 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:21.680 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:53244 3.f 3:f71f51a7:::rbd_data.112dca2f6ae1.0000000000000015:17 [read 1863680~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:32:21.680 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:44.369399+0000", 2024-09-14T06:32:21.680 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 157.30873544400001, 2024-09-14T06:32:21.680 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047291189999999997, 2024-09-14T06:32:21.680 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:32:21.681 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:32:21.681 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:32:21.681 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:32:21.681 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:32:21.681 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:21.681 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 53244 2024-09-14T06:32:21.681 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:21.681 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:32:21.682 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:21.682 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:32:21.682 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369399+0000", 2024-09-14T06:32:21.682 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:21.682 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:21.682 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:21.682 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:32:21.682 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369399+0000", 2024-09-14T06:32:21.682 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:21.682 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:21.683 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:21.683 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:32:21.683 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369400+0000", 2024-09-14T06:32:21.683 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.0699999999999999e-07 2024-09-14T06:32:21.683 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:21.683 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:21.683 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:32:21.683 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369401+0000", 2024-09-14T06:32:21.683 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.4300000000000001e-06 2024-09-14T06:32:21.683 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:21.683 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:21.683 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:32:21.684 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369404+0000", 2024-09-14T06:32:21.684 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.061e-06 2024-09-14T06:32:21.684 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:21.684 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:21.684 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:32:21.684 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369407+0000", 2024-09-14T06:32:21.684 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.6139999999999999e-06 2024-09-14T06:32:21.684 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:21.684 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:21.684 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:32:21.684 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.416517+0000", 2024-09-14T06:32:21.684 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047110338000000002 2024-09-14T06:32:21.684 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:21.684 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:21.685 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:32:21.685 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.416616+0000", 2024-09-14T06:32:21.685 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.8651000000000001e-05 2024-09-14T06:32:21.685 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:21.685 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:21.685 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:32:21.685 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.416691+0000", 2024-09-14T06:32:21.685 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.4888999999999994e-05 2024-09-14T06:32:21.685 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:21.685 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:32:21.685 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:21.685 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:21.685 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:21.686 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:53245 3.f 3:f71f51a7:::rbd_data.112dca2f6ae1.0000000000000015:17 [read 1888256~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:32:21.686 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:44.369410+0000", 2024-09-14T06:32:21.686 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 157.30872515600001, 2024-09-14T06:32:21.686 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047399275999999997, 2024-09-14T06:32:21.686 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:32:21.686 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:32:21.686 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:32:21.686 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:32:21.686 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:32:21.686 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:21.686 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 53245 2024-09-14T06:32:21.686 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:21.686 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:32:21.686 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:21.687 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:32:21.687 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369410+0000", 2024-09-14T06:32:21.687 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:21.687 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:21.687 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:21.687 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:32:21.687 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369410+0000", 2024-09-14T06:32:21.687 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:21.687 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:21.687 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:21.687 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:32:21.687 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369410+0000", 2024-09-14T06:32:21.687 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.1899999999999999e-07 2024-09-14T06:32:21.687 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:21.688 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:21.688 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:32:21.688 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369411+0000", 2024-09-14T06:32:21.688 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.0300000000000001e-06 2024-09-14T06:32:21.688 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:21.688 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:21.688 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:32:21.688 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369414+0000", 2024-09-14T06:32:21.688 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.1099999999999999e-06 2024-09-14T06:32:21.688 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:21.688 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:21.688 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:32:21.688 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369417+0000", 2024-09-14T06:32:21.688 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.6189999999999998e-06 2024-09-14T06:32:21.688 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:21.689 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:21.689 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:32:21.689 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.416718+0000", 2024-09-14T06:32:21.689 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047301322999999999 2024-09-14T06:32:21.689 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:21.689 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:21.689 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:32:21.689 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.416759+0000", 2024-09-14T06:32:21.689 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.1009e-05 2024-09-14T06:32:21.689 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:21.689 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:21.689 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:32:21.689 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.416809+0000", 2024-09-14T06:32:21.690 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.9966000000000002e-05 2024-09-14T06:32:21.690 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:21.690 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:32:21.690 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:21.690 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:21.690 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:21.690 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:53246 3.f 3:f71f51a7:::rbd_data.112dca2f6ae1.0000000000000015:17 [read 1912832~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:32:21.690 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:44.369420+0000", 2024-09-14T06:32:21.690 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 157.30871521200001, 2024-09-14T06:32:21.690 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047529463000000001, 2024-09-14T06:32:21.690 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:32:21.690 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:32:21.690 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:32:21.690 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:32:21.691 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:32:21.691 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:21.691 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 53246 2024-09-14T06:32:21.691 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:21.691 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:32:21.691 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:21.691 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:32:21.691 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369420+0000", 2024-09-14T06:32:21.691 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:21.691 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:21.691 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:21.691 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:32:21.691 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369420+0000", 2024-09-14T06:32:21.691 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:21.691 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:21.692 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:21.692 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:32:21.692 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369420+0000", 2024-09-14T06:32:21.692 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.1299999999999999e-07 2024-09-14T06:32:21.692 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:21.692 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:21.692 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:32:21.692 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369425+0000", 2024-09-14T06:32:21.692 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.0130000000000001e-06 2024-09-14T06:32:21.692 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:21.692 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:21.692 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:32:21.692 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369428+0000", 2024-09-14T06:32:21.692 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.349e-06 2024-09-14T06:32:21.693 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:21.693 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:21.693 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:32:21.693 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369431+0000", 2024-09-14T06:32:21.693 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.7389999999999999e-06 2024-09-14T06:32:21.693 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:21.693 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:21.693 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:32:21.693 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.416834+0000", 2024-09-14T06:32:21.693 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047402669000000001 2024-09-14T06:32:21.693 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:21.693 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:21.693 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:32:21.693 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.416882+0000", 2024-09-14T06:32:21.694 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.8068999999999997e-05 2024-09-14T06:32:21.694 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:21.694 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:21.694 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:32:21.694 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.416949+0000", 2024-09-14T06:32:21.694 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.7410999999999995e-05 2024-09-14T06:32:21.694 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:21.694 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:32:21.694 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:21.694 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:21.694 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:21.694 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:53247 3.f 3:f71f51a7:::rbd_data.112dca2f6ae1.0000000000000015:17 [read 1937408~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:32:21.694 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:44.369434+0000", 2024-09-14T06:32:21.694 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 157.308700448, 2024-09-14T06:32:21.694 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047652264999999999, 2024-09-14T06:32:21.695 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:32:21.695 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:32:21.695 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:32:21.695 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:32:21.695 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:32:21.695 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:21.695 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 53247 2024-09-14T06:32:21.695 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:21.695 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:32:21.695 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:21.695 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:32:21.695 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369434+0000", 2024-09-14T06:32:21.695 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:21.695 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:21.696 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:21.696 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:32:21.696 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369434+0000", 2024-09-14T06:32:21.696 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:21.696 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:21.696 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:21.696 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:32:21.696 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369435+0000", 2024-09-14T06:32:21.696 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.2100000000000001e-07 2024-09-14T06:32:21.696 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:21.696 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:21.696 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:32:21.696 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369436+0000", 2024-09-14T06:32:21.696 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.122e-06 2024-09-14T06:32:21.697 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:21.697 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:21.697 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:32:21.697 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369439+0000", 2024-09-14T06:32:21.697 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.0400000000000001e-06 2024-09-14T06:32:21.697 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:21.697 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:21.697 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:32:21.697 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369441+0000", 2024-09-14T06:32:21.697 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.728e-06 2024-09-14T06:32:21.697 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:21.698 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:21.698 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:32:21.698 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.416977+0000", 2024-09-14T06:32:21.698 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047535344 2024-09-14T06:32:21.698 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:21.698 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:21.698 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:32:21.698 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.417029+0000", 2024-09-14T06:32:21.698 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.2037999999999999e-05 2024-09-14T06:32:21.698 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:21.698 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:21.698 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:32:21.698 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.417087+0000", 2024-09-14T06:32:21.698 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.7772000000000002e-05 2024-09-14T06:32:21.699 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:21.699 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:32:21.699 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:21.699 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:21.699 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:21.699 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:53248 3.f 3:f71f51a7:::rbd_data.112dca2f6ae1.0000000000000015:17 [read 1961984~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:32:21.699 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:44.369445+0000", 2024-09-14T06:32:21.699 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 157.308689896, 2024-09-14T06:32:21.699 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047798489, 2024-09-14T06:32:21.700 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:32:21.700 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:32:21.700 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:32:21.700 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:32:21.700 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:32:21.700 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:21.700 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 53248 2024-09-14T06:32:21.700 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:21.700 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:32:21.700 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:21.700 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:32:21.701 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369445+0000", 2024-09-14T06:32:21.701 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:21.701 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:21.701 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:21.701 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:32:21.701 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369445+0000", 2024-09-14T06:32:21.701 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:21.701 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:21.701 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:21.701 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:32:21.701 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369445+0000", 2024-09-14T06:32:21.701 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.0699999999999999e-07 2024-09-14T06:32:21.701 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:21.701 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:21.702 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:32:21.702 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369446+0000", 2024-09-14T06:32:21.702 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.1000000000000001e-06 2024-09-14T06:32:21.702 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:21.702 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:21.702 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:32:21.702 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369449+0000", 2024-09-14T06:32:21.702 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.1039999999999998e-06 2024-09-14T06:32:21.702 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:21.702 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:21.702 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:32:21.702 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369452+0000", 2024-09-14T06:32:21.703 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.6199999999999999e-06 2024-09-14T06:32:21.703 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:21.703 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:21.703 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:32:21.703 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.417114+0000", 2024-09-14T06:32:21.703 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047661901999999999 2024-09-14T06:32:21.703 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:21.703 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:21.703 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:32:21.704 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.417181+0000", 2024-09-14T06:32:21.704 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.7385999999999995e-05 2024-09-14T06:32:21.704 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:21.704 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:21.704 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:32:21.704 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.417243+0000", 2024-09-14T06:32:21.704 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.2169999999999996e-05 2024-09-14T06:32:21.704 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:21.704 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:32:21.705 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:21.705 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:21.705 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:21.705 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:53249 3.f 3:f71f51a7:::rbd_data.112dca2f6ae1.0000000000000015:17 [read 1986560~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:32:21.705 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:44.369455+0000", 2024-09-14T06:32:21.705 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 157.30867958499999, 2024-09-14T06:32:21.705 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047890493999999999, 2024-09-14T06:32:21.705 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:32:21.705 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:32:21.705 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:32:21.705 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:32:21.705 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:32:21.705 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:21.705 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 53249 2024-09-14T06:32:21.706 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:21.706 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:32:21.706 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:21.706 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:32:21.706 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369455+0000", 2024-09-14T06:32:21.706 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:21.706 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:21.706 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:21.706 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:32:21.706 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369455+0000", 2024-09-14T06:32:21.706 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:21.706 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:21.706 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:21.706 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:32:21.707 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369455+0000", 2024-09-14T06:32:21.707 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.0800000000000001e-07 2024-09-14T06:32:21.707 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:21.707 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:21.707 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:32:21.707 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369456+0000", 2024-09-14T06:32:21.707 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.0729999999999999e-06 2024-09-14T06:32:21.707 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:21.707 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:21.707 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:32:21.707 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369460+0000", 2024-09-14T06:32:21.707 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.027e-06 2024-09-14T06:32:21.707 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:21.707 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:21.708 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:32:21.708 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369462+0000", 2024-09-14T06:32:21.708 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.5320000000000001e-06 2024-09-14T06:32:21.708 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:21.708 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:21.708 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:32:21.708 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.417257+0000", 2024-09-14T06:32:21.708 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.04779489 2024-09-14T06:32:21.708 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:21.708 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:21.708 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:32:21.708 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.417296+0000", 2024-09-14T06:32:21.708 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.8658000000000002e-05 2024-09-14T06:32:21.709 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:21.709 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:21.709 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:32:21.709 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.417346+0000", 2024-09-14T06:32:21.709 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.0105999999999998e-05 2024-09-14T06:32:21.709 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:21.709 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:32:21.709 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:21.709 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:21.709 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:21.709 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:53250 3.f 3:f71f51a7:::rbd_data.112dca2f6ae1.0000000000000015:17 [read 2011136~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:32:21.709 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:44.369465+0000", 2024-09-14T06:32:21.709 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 157.30866961500001, 2024-09-14T06:32:21.709 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.048032230000000002, 2024-09-14T06:32:21.710 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:32:21.710 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:32:21.710 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:32:21.710 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:32:21.710 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:32:21.710 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:21.710 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 53250 2024-09-14T06:32:21.710 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:21.710 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:32:21.710 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:21.710 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:32:21.710 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369465+0000", 2024-09-14T06:32:21.710 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:21.711 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:21.711 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:21.711 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:32:21.711 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369465+0000", 2024-09-14T06:32:21.711 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:21.711 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:21.711 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:21.711 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:32:21.711 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369465+0000", 2024-09-14T06:32:21.711 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.2100000000000001e-07 2024-09-14T06:32:21.711 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:21.711 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:21.711 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:32:21.711 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369466+0000", 2024-09-14T06:32:21.711 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.0640000000000001e-06 2024-09-14T06:32:21.712 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:21.712 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:21.712 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:32:21.712 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369469+0000", 2024-09-14T06:32:21.712 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.0170000000000001e-06 2024-09-14T06:32:21.712 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:21.712 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:21.712 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:32:21.712 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369472+0000", 2024-09-14T06:32:21.712 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.6189999999999998e-06 2024-09-14T06:32:21.712 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:21.712 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:21.712 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:32:21.712 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.417383+0000", 2024-09-14T06:32:21.713 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047911092000000002 2024-09-14T06:32:21.713 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:21.713 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:21.713 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:32:21.713 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.417430+0000", 2024-09-14T06:32:21.713 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.6898999999999999e-05 2024-09-14T06:32:21.713 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:21.713 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:21.713 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:32:21.713 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.417497+0000", 2024-09-14T06:32:21.713 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.7317999999999995e-05 2024-09-14T06:32:21.713 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:21.713 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:32:21.714 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:21.714 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:21.714 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:21.714 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:53251 3.f 3:f71f51a7:::rbd_data.112dca2f6ae1.0000000000000015:17 [read 2035712~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:32:21.714 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:44.369475+0000", 2024-09-14T06:32:21.714 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 157.308659619, 2024-09-14T06:32:21.714 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.048154542000000002, 2024-09-14T06:32:21.714 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:32:21.714 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:32:21.714 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:32:21.714 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:32:21.714 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:32:21.714 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:21.714 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 53251 2024-09-14T06:32:21.714 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:21.715 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:32:21.715 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:21.715 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:32:21.715 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369475+0000", 2024-09-14T06:32:21.715 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:21.715 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:21.715 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:21.715 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:32:21.715 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369475+0000", 2024-09-14T06:32:21.715 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:21.715 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:21.715 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:21.715 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:32:21.715 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369475+0000", 2024-09-14T06:32:21.716 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.0100000000000001e-07 2024-09-14T06:32:21.716 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:21.716 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:21.716 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:32:21.716 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369476+0000", 2024-09-14T06:32:21.716 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.0580000000000001e-06 2024-09-14T06:32:21.716 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:21.716 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:21.716 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:32:21.716 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369479+0000", 2024-09-14T06:32:21.716 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.9950000000000001e-06 2024-09-14T06:32:21.716 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:21.717 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:21.717 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:32:21.717 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369482+0000", 2024-09-14T06:32:21.717 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.6340000000000001e-06 2024-09-14T06:32:21.717 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:21.717 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:21.717 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:32:21.717 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.417530+0000", 2024-09-14T06:32:21.717 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.048047535000000002 2024-09-14T06:32:21.717 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:21.717 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:21.718 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:32:21.718 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.417574+0000", 2024-09-14T06:32:21.718 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.4255999999999998e-05 2024-09-14T06:32:21.718 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:21.718 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:21.718 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:32:21.718 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.417630+0000", 2024-09-14T06:32:21.718 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.5862999999999997e-05 2024-09-14T06:32:21.718 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:21.718 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:32:21.718 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:21.718 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:21.719 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:21.719 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:53252 3.f 3:f71f51a7:::rbd_data.112dca2f6ae1.0000000000000015:17 [read 2060288~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:32:21.719 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:44.369485+0000", 2024-09-14T06:32:21.719 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 157.30864977100001, 2024-09-14T06:32:21.719 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.048254116, 2024-09-14T06:32:21.719 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:32:21.719 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:32:21.719 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:32:21.719 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:32:21.719 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:32:21.719 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:21.719 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 53252 2024-09-14T06:32:21.719 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:21.719 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:32:21.720 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:21.720 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:32:21.720 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369485+0000", 2024-09-14T06:32:21.720 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:21.720 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:21.720 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:21.720 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:32:21.720 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369485+0000", 2024-09-14T06:32:21.720 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:21.720 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:21.720 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:21.720 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:32:21.721 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369485+0000", 2024-09-14T06:32:21.721 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.1e-07 2024-09-14T06:32:21.721 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:21.721 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:21.721 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:32:21.721 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369487+0000", 2024-09-14T06:32:21.721 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.42e-06 2024-09-14T06:32:21.721 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:21.721 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:21.721 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:32:21.721 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369490+0000", 2024-09-14T06:32:21.721 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.9639999999999999e-06 2024-09-14T06:32:21.721 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:21.721 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:21.722 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:32:21.722 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369492+0000", 2024-09-14T06:32:21.722 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.5679999999999998e-06 2024-09-14T06:32:21.722 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:21.722 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:21.722 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:32:21.722 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.417645+0000", 2024-09-14T06:32:21.722 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.048152753 2024-09-14T06:32:21.722 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:21.722 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:21.722 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:32:21.722 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.417687+0000", 2024-09-14T06:32:21.722 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.1900999999999997e-05 2024-09-14T06:32:21.722 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:21.723 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:21.723 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:32:21.723 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.417739+0000", 2024-09-14T06:32:21.723 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.2299999999999997e-05 2024-09-14T06:32:21.723 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:21.723 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:32:21.723 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:21.723 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:21.723 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:21.723 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:53253 3.f 3:f71f51a7:::rbd_data.112dca2f6ae1.0000000000000015:17 [read 2084864~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:32:21.724 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:44.369495+0000", 2024-09-14T06:32:21.724 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 157.30863964700001, 2024-09-14T06:32:21.724 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.048366785000000002, 2024-09-14T06:32:21.724 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:32:21.724 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:32:21.724 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:32:21.724 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:32:21.724 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:32:21.724 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:21.724 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 53253 2024-09-14T06:32:21.725 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:21.725 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:32:21.725 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:21.725 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:32:21.725 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369495+0000", 2024-09-14T06:32:21.725 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:21.725 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:21.725 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:21.725 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:32:21.725 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369495+0000", 2024-09-14T06:32:21.725 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:21.725 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:21.725 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:21.726 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:32:21.726 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370388+0000", 2024-09-14T06:32:21.726 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00089287999999999998 2024-09-14T06:32:21.726 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:21.726 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:21.726 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:32:21.726 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370392+0000", 2024-09-14T06:32:21.726 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.2130000000000002e-06 2024-09-14T06:32:21.726 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:21.726 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:21.726 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:32:21.727 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370401+0000", 2024-09-14T06:32:21.727 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.7649999999999999e-06 2024-09-14T06:32:21.727 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:21.727 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:21.727 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:32:21.727 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370406+0000", 2024-09-14T06:32:21.727 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.0159999999999999e-06 2024-09-14T06:32:21.727 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:21.727 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:21.727 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:32:21.727 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.417764+0000", 2024-09-14T06:32:21.727 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047358348000000001 2024-09-14T06:32:21.727 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:21.728 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:21.728 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:32:21.728 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.417806+0000", 2024-09-14T06:32:21.728 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.1149999999999997e-05 2024-09-14T06:32:21.728 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:21.728 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:21.728 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:32:21.728 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.417862+0000", 2024-09-14T06:32:21.728 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.6412999999999997e-05 2024-09-14T06:32:21.728 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:21.728 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:32:21.728 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:21.728 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:21.728 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:21.728 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:53340 3.f 3:f71f51a7:::rbd_data.112dca2f6ae1.0000000000000015:17 [read 1056768~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:32:21.729 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:44.370421+0000", 2024-09-14T06:32:21.729 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 157.307713767, 2024-09-14T06:32:21.729 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047566313999999998, 2024-09-14T06:32:21.729 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:32:21.729 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:32:21.729 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:32:21.729 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:32:21.729 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:32:21.729 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:21.729 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 53340 2024-09-14T06:32:21.729 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:21.729 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:32:21.730 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:21.730 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:32:21.730 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370421+0000", 2024-09-14T06:32:21.730 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:21.730 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:21.730 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:21.730 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:32:21.730 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370421+0000", 2024-09-14T06:32:21.730 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:21.730 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:21.730 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:21.730 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:32:21.730 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370422+0000", 2024-09-14T06:32:21.730 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.3799999999999996e-07 2024-09-14T06:32:21.731 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:21.731 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:21.731 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:32:21.731 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370424+0000", 2024-09-14T06:32:21.731 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.6919999999999998e-06 2024-09-14T06:32:21.731 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:21.731 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:21.731 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:32:21.731 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370432+0000", 2024-09-14T06:32:21.731 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.2439999999999996e-06 2024-09-14T06:32:21.731 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:21.731 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:21.731 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:32:21.731 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370437+0000", 2024-09-14T06:32:21.731 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.5389999999999998e-06 2024-09-14T06:32:21.731 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:21.732 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:21.732 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:32:21.732 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.417888+0000", 2024-09-14T06:32:21.732 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047450432000000001 2024-09-14T06:32:21.732 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:21.732 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:21.732 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:32:21.732 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.417935+0000", 2024-09-14T06:32:21.732 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.7454999999999999e-05 2024-09-14T06:32:21.732 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:21.732 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:21.732 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:32:21.732 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.417987+0000", 2024-09-14T06:32:21.732 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.2213999999999998e-05 2024-09-14T06:32:21.733 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:21.733 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:32:21.733 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:21.733 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:21.733 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:21.733 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:53341 3.f 3:f71f51a7:::rbd_data.112dca2f6ae1.0000000000000015:17 [read 1081344~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:32:21.733 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:44.370446+0000", 2024-09-14T06:32:21.733 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 157.30768849399999, 2024-09-14T06:32:21.733 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047665723, 2024-09-14T06:32:21.733 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:32:21.733 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:32:21.733 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:32:21.733 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:32:21.733 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:32:21.734 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:21.734 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 53341 2024-09-14T06:32:21.734 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:21.734 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:32:21.734 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:21.734 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:32:21.734 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370446+0000", 2024-09-14T06:32:21.734 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:21.734 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:21.734 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:21.734 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:32:21.734 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370446+0000", 2024-09-14T06:32:21.734 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:21.735 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:21.735 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:21.735 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:32:21.735 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370447+0000", 2024-09-14T06:32:21.735 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.6100000000000002e-07 2024-09-14T06:32:21.735 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:21.735 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:21.735 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:32:21.735 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370448+0000", 2024-09-14T06:32:21.735 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.7370000000000001e-06 2024-09-14T06:32:21.735 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:21.735 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:21.735 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:32:21.735 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370462+0000", 2024-09-14T06:32:21.735 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.3227000000000001e-05 2024-09-14T06:32:21.736 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:21.736 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:21.736 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:32:21.736 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370467+0000", 2024-09-14T06:32:21.736 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.6540000000000004e-06 2024-09-14T06:32:21.736 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:21.736 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:21.736 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:32:21.736 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.418013+0000", 2024-09-14T06:32:21.736 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047545325999999999 2024-09-14T06:32:21.736 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:21.736 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:21.736 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:32:21.736 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.418054+0000", 2024-09-14T06:32:21.736 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.1696999999999997e-05 2024-09-14T06:32:21.737 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:21.737 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:21.737 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:32:21.737 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.418112+0000", 2024-09-14T06:32:21.737 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.7720999999999998e-05 2024-09-14T06:32:21.737 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:21.737 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:32:21.737 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:21.737 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:21.737 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:21.737 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:53342 3.f 3:f71f51a7:::rbd_data.112dca2f6ae1.0000000000000015:17 [read 1105920~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:32:21.737 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:44.370475+0000", 2024-09-14T06:32:21.738 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 157.307659433, 2024-09-14T06:32:21.738 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047748848000000003, 2024-09-14T06:32:21.738 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:32:21.738 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:32:21.738 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:32:21.738 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:32:21.738 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:32:21.738 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:21.738 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 53342 2024-09-14T06:32:21.738 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:21.738 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:32:21.738 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:21.738 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:32:21.738 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370475+0000", 2024-09-14T06:32:21.739 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:21.739 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:21.739 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:21.739 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:32:21.739 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370475+0000", 2024-09-14T06:32:21.739 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:21.739 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:21.739 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:21.739 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:32:21.739 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370476+0000", 2024-09-14T06:32:21.739 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.4700000000000002e-07 2024-09-14T06:32:21.739 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:21.739 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:21.740 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:32:21.740 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370478+0000", 2024-09-14T06:32:21.740 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.5890000000000001e-06 2024-09-14T06:32:21.740 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:21.740 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:21.740 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:32:21.740 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370486+0000", 2024-09-14T06:32:21.740 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.3470000000000001e-06 2024-09-14T06:32:21.740 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:21.740 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:21.740 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:32:21.740 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370491+0000", 2024-09-14T06:32:21.740 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.5720000000000002e-06 2024-09-14T06:32:21.740 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:21.740 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:21.741 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:32:21.741 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.418135+0000", 2024-09-14T06:32:21.741 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047644094999999997 2024-09-14T06:32:21.741 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:21.741 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:21.741 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:32:21.741 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.418174+0000", 2024-09-14T06:32:21.741 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.8204000000000002e-05 2024-09-14T06:32:21.741 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:21.741 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:21.741 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:32:21.742 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.418224+0000", 2024-09-14T06:32:21.742 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.0593999999999997e-05 2024-09-14T06:32:21.742 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:21.742 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:32:21.742 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:21.742 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:21.742 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:21.742 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:53343 3.f 3:f71f51a7:::rbd_data.112dca2f6ae1.0000000000000015:17 [read 1130496~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:32:21.743 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:44.370498+0000", 2024-09-14T06:32:21.743 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 157.307636557, 2024-09-14T06:32:21.743 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047841574999999997, 2024-09-14T06:32:21.743 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:32:21.743 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:32:21.743 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:32:21.743 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:32:21.743 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:32:21.743 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:21.743 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 53343 2024-09-14T06:32:21.744 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:21.744 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:32:21.744 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:21.744 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:32:21.744 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370498+0000", 2024-09-14T06:32:21.744 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:21.744 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:21.744 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:21.744 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:32:21.744 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370498+0000", 2024-09-14T06:32:21.744 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:21.744 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:21.745 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:21.745 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:32:21.745 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370499+0000", 2024-09-14T06:32:21.745 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.9999999999999998e-07 2024-09-14T06:32:21.745 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:21.745 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:21.745 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:32:21.745 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370501+0000", 2024-09-14T06:32:21.745 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.0310000000000001e-06 2024-09-14T06:32:21.745 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:21.745 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:21.746 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:32:21.746 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370507+0000", 2024-09-14T06:32:21.746 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.4169999999999997e-06 2024-09-14T06:32:21.746 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:21.746 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:21.746 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:32:21.746 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370516+0000", 2024-09-14T06:32:21.746 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.7190000000000007e-06 2024-09-14T06:32:21.746 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:21.746 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:21.746 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:32:21.746 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.418247+0000", 2024-09-14T06:32:21.746 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047731190999999999 2024-09-14T06:32:21.746 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:21.746 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:21.747 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:32:21.747 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.418286+0000", 2024-09-14T06:32:21.747 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.8988e-05 2024-09-14T06:32:21.747 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:21.747 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:21.747 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:32:21.747 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.418340+0000", 2024-09-14T06:32:21.747 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.3829000000000002e-05 2024-09-14T06:32:21.747 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:21.747 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:32:21.747 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:21.748 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:21.748 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:21.748 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:53344 3.f 3:f71f51a7:::rbd_data.112dca2f6ae1.0000000000000015:17 [read 1155072~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:32:21.748 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:44.370524+0000", 2024-09-14T06:32:21.748 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 157.30761078699999, 2024-09-14T06:32:21.748 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047972262000000002, 2024-09-14T06:32:21.748 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:32:21.748 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:32:21.748 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:32:21.748 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:32:21.748 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:32:21.748 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:21.748 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 53344 2024-09-14T06:32:21.749 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:21.749 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:32:21.749 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:21.749 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:32:21.749 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370524+0000", 2024-09-14T06:32:21.749 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:21.749 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:21.749 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:21.749 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:32:21.749 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370524+0000", 2024-09-14T06:32:21.749 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:21.749 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:21.749 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:21.749 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:32:21.750 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370618+0000", 2024-09-14T06:32:21.750 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.4425999999999993e-05 2024-09-14T06:32:21.750 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:21.750 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:21.750 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:32:21.750 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370622+0000", 2024-09-14T06:32:21.750 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.3519999999999998e-06 2024-09-14T06:32:21.750 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:21.750 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:21.750 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:32:21.751 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370631+0000", 2024-09-14T06:32:21.751 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.9299999999999992e-06 2024-09-14T06:32:21.751 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:21.751 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:21.751 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:32:21.751 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370637+0000", 2024-09-14T06:32:21.752 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.3269999999999997e-06 2024-09-14T06:32:21.752 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:21.752 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:21.752 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:32:21.752 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.418372+0000", 2024-09-14T06:32:21.752 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047734764999999998 2024-09-14T06:32:21.753 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:21.753 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:21.753 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:32:21.753 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.418431+0000", 2024-09-14T06:32:21.753 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.8696999999999997e-05 2024-09-14T06:32:21.753 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:21.754 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:21.754 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:32:21.754 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.418496+0000", 2024-09-14T06:32:21.754 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.5765000000000005e-05 2024-09-14T06:32:21.754 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:21.754 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:32:21.754 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:21.754 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:21.754 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:21.754 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:53345 3.f 3:f71f51a7:::rbd_data.112dca2f6ae1.0000000000000015:17 [read 1179648~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:32:21.754 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:44.370647+0000", 2024-09-14T06:32:21.754 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 157.30748732000001, 2024-09-14T06:32:21.754 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047953775999999997, 2024-09-14T06:32:21.755 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:32:21.755 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:32:21.755 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:32:21.755 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:32:21.755 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:32:21.755 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:21.755 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 53345 2024-09-14T06:32:21.755 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:21.755 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:32:21.755 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:21.755 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:32:21.755 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370647+0000", 2024-09-14T06:32:21.756 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:21.756 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:21.756 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:21.756 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:32:21.756 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370647+0000", 2024-09-14T06:32:21.756 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:21.756 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:21.756 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:21.756 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:32:21.756 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.372346+0000", 2024-09-14T06:32:21.756 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.001698284 2024-09-14T06:32:21.756 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:21.756 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:21.756 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:32:21.757 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.372351+0000", 2024-09-14T06:32:21.757 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.8890000000000001e-06 2024-09-14T06:32:21.757 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:21.757 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:21.757 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:32:21.757 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.372362+0000", 2024-09-14T06:32:21.757 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.0941000000000001e-05 2024-09-14T06:32:21.757 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:21.757 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:21.757 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:32:21.757 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.372369+0000", 2024-09-14T06:32:21.757 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.362e-06 2024-09-14T06:32:21.758 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:21.758 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:21.758 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:32:21.758 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.418510+0000", 2024-09-14T06:32:21.758 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046140914999999998 2024-09-14T06:32:21.758 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:21.758 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:21.758 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:32:21.759 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.418549+0000", 2024-09-14T06:32:21.759 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.8717999999999998e-05 2024-09-14T06:32:21.759 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:21.759 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:21.759 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:32:21.759 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.418601+0000", 2024-09-14T06:32:21.759 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.2667000000000002e-05 2024-09-14T06:32:21.759 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:21.759 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:32:21.759 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:21.759 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:21.760 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:21.760 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:53353 3.f 3:f71f51a7:::rbd_data.112dca2f6ae1.0000000000000015:17 [read 1376256~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:32:21.760 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:44.372545+0000", 2024-09-14T06:32:21.760 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 157.30559024499999, 2024-09-14T06:32:21.760 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047364232999999999, 2024-09-14T06:32:21.760 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:32:21.760 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:32:21.760 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:32:21.760 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:32:21.760 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:32:21.761 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:21.761 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 53353 2024-09-14T06:32:21.761 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:21.761 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:32:21.761 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:21.761 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:32:21.761 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.372545+0000", 2024-09-14T06:32:21.761 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:21.761 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:21.761 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:21.761 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:32:21.761 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.372545+0000", 2024-09-14T06:32:21.761 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:21.761 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:21.762 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:21.762 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:32:21.762 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.372545+0000", 2024-09-14T06:32:21.762 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.27e-07 2024-09-14T06:32:21.762 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:21.762 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:21.762 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:32:21.762 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.372547+0000", 2024-09-14T06:32:21.762 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.0600000000000002e-06 2024-09-14T06:32:21.762 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:21.762 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:21.763 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:32:21.763 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.372553+0000", 2024-09-14T06:32:21.763 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.7980000000000002e-06 2024-09-14T06:32:21.763 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:21.763 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:21.763 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:32:21.763 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.372557+0000", 2024-09-14T06:32:21.763 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.7389999999999999e-06 2024-09-14T06:32:21.763 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:21.763 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:21.763 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:32:21.763 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.419811+0000", 2024-09-14T06:32:21.763 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047253310999999999 2024-09-14T06:32:21.764 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:21.764 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:21.764 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:32:21.764 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.419854+0000", 2024-09-14T06:32:21.764 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.3448000000000001e-05 2024-09-14T06:32:21.764 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:21.764 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:21.764 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:32:21.764 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.419909+0000", 2024-09-14T06:32:21.765 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.4549999999999998e-05 2024-09-14T06:32:21.765 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:21.765 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:32:21.765 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:21.765 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:21.765 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:21.765 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:53354 3.f 3:f71f51a7:::rbd_data.112dca2f6ae1.0000000000000015:17 [read 1400832~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:32:21.765 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:44.372563+0000", 2024-09-14T06:32:21.765 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 157.305571808, 2024-09-14T06:32:21.765 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047482233999999998, 2024-09-14T06:32:21.765 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:32:21.765 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:32:21.765 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:32:21.765 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:32:21.766 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:32:21.766 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:21.766 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 53354 2024-09-14T06:32:21.766 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:21.766 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:32:21.766 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:21.766 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:32:21.766 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.372563+0000", 2024-09-14T06:32:21.766 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:21.766 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:21.766 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:21.766 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:32:21.766 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.372563+0000", 2024-09-14T06:32:21.766 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:21.767 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:21.767 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:21.767 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:32:21.767 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.375009+0000", 2024-09-14T06:32:21.767 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.0024457680000000001 2024-09-14T06:32:21.767 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:21.767 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:21.767 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:32:21.767 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.375014+0000", 2024-09-14T06:32:21.767 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.0599999999999998e-06 2024-09-14T06:32:21.767 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:21.767 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:21.767 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:32:21.767 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.375026+0000", 2024-09-14T06:32:21.768 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.1783999999999999e-05 2024-09-14T06:32:21.768 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:21.768 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:21.768 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:32:21.768 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.375035+0000", 2024-09-14T06:32:21.768 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.7799999999999995e-06 2024-09-14T06:32:21.768 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:21.769 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:21.769 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:32:21.769 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.419942+0000", 2024-09-14T06:32:21.769 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.044906982999999998 2024-09-14T06:32:21.769 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:21.769 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:21.769 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:32:21.769 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.419989+0000", 2024-09-14T06:32:21.769 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.7104999999999997e-05 2024-09-14T06:32:21.769 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:21.769 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:21.769 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:32:21.770 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.420045+0000", 2024-09-14T06:32:21.770 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.5754000000000002e-05 2024-09-14T06:32:21.770 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:21.770 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:32:21.770 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:21.770 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:21.770 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:21.770 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:225577 3.15 3:ac159a70:::rbd_header.112daa9ec306:head [watch reconnect cookie 140616827151664 gen 4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e188)", 2024-09-14T06:32:21.770 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:31:52.733493+0000", 2024-09-14T06:32:21.770 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 28.944641344000001, 2024-09-14T06:32:21.770 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.0156593789999999, 2024-09-14T06:32:21.770 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:32:21.770 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:32:21.771 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:32:21.771 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:32:21.771 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:32:21.771 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:21.771 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 225577 2024-09-14T06:32:21.771 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:21.771 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:32:21.771 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:21.772 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:32:21.772 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:31:52.733493+0000", 2024-09-14T06:32:21.772 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:21.772 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:21.772 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:21.772 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:32:21.772 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:31:52.733493+0000", 2024-09-14T06:32:21.772 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:21.772 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:21.772 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:21.772 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:32:21.773 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:31:52.733495+0000", 2024-09-14T06:32:21.773 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.0059999999999999e-06 2024-09-14T06:32:21.773 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:21.773 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:21.773 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:32:21.773 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:31:52.733504+0000", 2024-09-14T06:32:21.773 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.5490000000000006e-06 2024-09-14T06:32:21.773 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:21.773 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:21.773 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:32:21.773 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:31:52.733522+0000", 2024-09-14T06:32:21.773 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.7660999999999999e-05 2024-09-14T06:32:21.774 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:21.774 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:21.774 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:32:21.774 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:31:52.733531+0000", 2024-09-14T06:32:21.774 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.7180000000000008e-06 2024-09-14T06:32:21.774 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:21.774 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:21.774 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:32:21.774 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:31:52.733598+0000", 2024-09-14T06:32:21.774 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.6490999999999997e-05 2024-09-14T06:32:21.774 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:21.774 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:21.774 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "op must wait for map", 2024-09-14T06:32:21.775 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:31:52.733605+0000", 2024-09-14T06:32:21.775 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.6109999999999996e-06 2024-09-14T06:32:21.775 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:21.775 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:21.775 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:32:21.775 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:31:52.737908+0000", 2024-09-14T06:32:21.775 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.0043023050000000002 2024-09-14T06:32:21.775 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:21.775 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:21.775 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "waiting for peered", 2024-09-14T06:32:21.775 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:31:52.737923+0000", 2024-09-14T06:32:21.775 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.488e-05 2024-09-14T06:32:21.776 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:21.776 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:21.776 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:32:21.776 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:31:53.746776+0000", 2024-09-14T06:32:21.776 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.0088535359999999 2024-09-14T06:32:21.776 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:21.776 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:21.776 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:32:21.776 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:31:53.746878+0000", 2024-09-14T06:32:21.776 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.000101788 2024-09-14T06:32:21.776 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:21.776 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:21.776 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:32:21.776 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:31:53.749153+0000", 2024-09-14T06:32:21.777 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.0022748339999999999 2024-09-14T06:32:21.777 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:21.777 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:32:21.777 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:21.777 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:21.777 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:21.777 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:225581 3.15 3:ab407115:::rbd_object_map.112d838b9898:head [call lock.unlock in=10b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e188)", 2024-09-14T06:32:21.777 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:31:52.743472+0000", 2024-09-14T06:32:21.777 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 28.934662305, 2024-09-14T06:32:21.777 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.007270501, 2024-09-14T06:32:21.777 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:32:21.777 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:32:21.777 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:32:21.777 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:32:21.778 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:32:21.778 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:21.778 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 225581 2024-09-14T06:32:21.778 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:21.778 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:32:21.778 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:21.778 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:32:21.778 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:31:52.743472+0000", 2024-09-14T06:32:21.778 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:21.778 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:21.778 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:21.778 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:32:21.778 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:31:52.743472+0000", 2024-09-14T06:32:21.778 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:21.779 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:21.779 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:21.779 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:32:21.779 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:31:52.743473+0000", 2024-09-14T06:32:21.779 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.1999999999999998e-07 2024-09-14T06:32:21.779 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:21.779 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:21.779 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:32:21.779 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:31:52.743478+0000", 2024-09-14T06:32:21.779 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.515e-06 2024-09-14T06:32:21.779 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:21.779 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:21.779 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:32:21.779 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:31:52.743489+0000", 2024-09-14T06:32:21.779 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.1411e-05 2024-09-14T06:32:21.780 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:21.780 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:21.780 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:32:21.780 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:31:52.743494+0000", 2024-09-14T06:32:21.780 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.1649999999999997e-06 2024-09-14T06:32:21.780 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:21.780 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:21.780 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:32:21.780 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:31:52.743548+0000", 2024-09-14T06:32:21.780 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.3758999999999998e-05 2024-09-14T06:32:21.780 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:21.780 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:21.780 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "waiting for peered", 2024-09-14T06:32:21.780 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:31:52.743558+0000", 2024-09-14T06:32:21.780 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.0336999999999999e-05 2024-09-14T06:32:21.781 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:21.781 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:21.781 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:32:21.781 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:31:53.747716+0000", 2024-09-14T06:32:21.781 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.004157035 2024-09-14T06:32:21.781 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:21.781 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:21.781 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:32:21.781 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:31:53.747831+0000", 2024-09-14T06:32:21.781 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.000115394 2024-09-14T06:32:21.781 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:21.781 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:21.781 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "waiting for subops from 3", 2024-09-14T06:32:21.781 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:31:53.748002+0000", 2024-09-14T06:32:21.782 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00017085099999999999 2024-09-14T06:32:21.782 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:21.782 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:21.782 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:32:21.782 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:31:53.749536+0000", 2024-09-14T06:32:21.782 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.0015339990000000001 2024-09-14T06:32:21.782 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:21.782 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:21.782 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "op_commit", 2024-09-14T06:32:21.782 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:31:53.750672+0000", 2024-09-14T06:32:21.782 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.001135973 2024-09-14T06:32:21.782 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:21.782 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:21.782 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "commit_sent", 2024-09-14T06:32:21.783 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:31:53.750704+0000", 2024-09-14T06:32:21.783 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.1791000000000001e-05 2024-09-14T06:32:21.783 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:21.783 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:21.783 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:32:21.783 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:31:53.750743+0000", 2024-09-14T06:32:21.783 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.9450999999999999e-05 2024-09-14T06:32:21.783 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:21.783 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:32:21.783 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:21.783 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:21.783 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:32:21.783 INFO:teuthology.orchestra.run.smithi138.stdout:} 2024-09-14T06:32:21.785 INFO:teuthology.orchestra.run.smithi094.stderr:nodeep-scrub is unset 2024-09-14T06:32:21.785 INFO:tasks.ceph.osd.7.smithi138.stderr:2024-09-14T06:32:21.783+0000 7f3c6a21d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33623) UID: 0 2024-09-14T06:32:21.786 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-14T06:32:21.886 INFO:tasks.ceph.osd.4.smithi138.stderr:2024-09-14T06:32:21.885+0000 7f6c653c8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33615) UID: 0 2024-09-14T06:32:21.929 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-14T06:32:21.930 INFO:teuthology.orchestra.run.smithi094.stdout: "ops": [], 2024-09-14T06:32:21.930 INFO:teuthology.orchestra.run.smithi094.stdout: "num_ops": 0 2024-09-14T06:32:21.930 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-14T06:32:21.938 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-14T06:32:21.987 INFO:tasks.ceph.osd.2.smithi094.stderr:2024-09-14T06:32:21.986+0000 7faf3269b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33649) UID: 0 2024-09-14T06:32:22.041 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-14T06:32:22.041 INFO:teuthology.orchestra.run.smithi094.stdout: "ops": [], 2024-09-14T06:32:22.041 INFO:teuthology.orchestra.run.smithi094.stdout: "complaint_time": 30, 2024-09-14T06:32:22.041 INFO:teuthology.orchestra.run.smithi094.stdout: "num_blocked_ops": 0 2024-09-14T06:32:22.041 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-14T06:32:22.049 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-14T06:32:22.090 INFO:tasks.ceph.osd.0.smithi094.stderr:2024-09-14T06:32:22.089+0000 7f07c9333640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33636) UID: 0 2024-09-14T06:32:22.105 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-14T06:32:22.188 INFO:tasks.ceph.osd.7.smithi138.stderr:2024-09-14T06:32:22.187+0000 7f3c6a21d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33623) UID: 0 2024-09-14T06:32:22.195 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-14T06:32:22.195 INFO:teuthology.orchestra.run.smithi094.stdout: "size": 20, 2024-09-14T06:32:22.195 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 600, 2024-09-14T06:32:22.195 INFO:teuthology.orchestra.run.smithi094.stdout: "ops": [ 2024-09-14T06:32:22.195 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:22.195 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:115193 3.1 3:856be886:::rbd_data.112d4d24c558.0000000000000004:29 [read 1523712~16384 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e96)", 2024-09-14T06:32:22.195 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:30:17.939854+0000", 2024-09-14T06:32:22.195 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 124.25358614300001, 2024-09-14T06:32:22.196 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.073897516999999996, 2024-09-14T06:32:22.196 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:32:22.196 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:32:22.196 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:32:22.196 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:32:22.196 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:32:22.196 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:22.196 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 115193 2024-09-14T06:32:22.196 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:22.196 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:32:22.196 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:22.196 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:32:22.196 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.939854+0000", 2024-09-14T06:32:22.196 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:22.197 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:22.197 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:22.197 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:32:22.197 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.939854+0000", 2024-09-14T06:32:22.197 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:22.197 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:22.197 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:22.197 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:32:22.197 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.939854+0000", 2024-09-14T06:32:22.197 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.9500000000000002e-07 2024-09-14T06:32:22.197 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:22.197 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:22.197 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:32:22.197 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.939856+0000", 2024-09-14T06:32:22.198 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.0959999999999999e-06 2024-09-14T06:32:22.198 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:22.198 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:22.198 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:32:22.198 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.939863+0000", 2024-09-14T06:32:22.198 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.1129999999999997e-06 2024-09-14T06:32:22.198 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:22.198 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:22.198 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:32:22.198 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.939867+0000", 2024-09-14T06:32:22.198 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.8350000000000003e-06 2024-09-14T06:32:22.198 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:22.198 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:22.198 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:32:22.198 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.013626+0000", 2024-09-14T06:32:22.199 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.073758799999999999 2024-09-14T06:32:22.199 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:22.199 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:22.199 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:32:22.199 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.013690+0000", 2024-09-14T06:32:22.199 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.3918999999999996e-05 2024-09-14T06:32:22.199 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:22.199 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:22.199 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:32:22.199 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.013751+0000", 2024-09-14T06:32:22.199 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.1159000000000002e-05 2024-09-14T06:32:22.199 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:22.199 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:32:22.199 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:22.199 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:22.199 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:22.200 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:115194 3.1 3:856be886:::rbd_data.112d4d24c558.0000000000000004:29 [read 1769472~16384 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e96)", 2024-09-14T06:32:22.200 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:30:17.939874+0000", 2024-09-14T06:32:22.200 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 124.25356583200001, 2024-09-14T06:32:22.200 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.074016988000000006, 2024-09-14T06:32:22.200 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:32:22.200 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:32:22.200 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:32:22.200 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:32:22.200 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:32:22.200 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:22.200 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 115194 2024-09-14T06:32:22.201 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:22.201 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:32:22.201 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:22.201 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:32:22.201 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.939874+0000", 2024-09-14T06:32:22.201 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:22.201 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:22.201 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:22.201 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:32:22.202 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.939874+0000", 2024-09-14T06:32:22.202 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:22.202 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:22.202 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:22.202 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:32:22.202 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.939875+0000", 2024-09-14T06:32:22.202 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.2600000000000002e-07 2024-09-14T06:32:22.202 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:22.202 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:22.202 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:32:22.202 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.939877+0000", 2024-09-14T06:32:22.203 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.9420000000000002e-06 2024-09-14T06:32:22.203 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:22.203 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:22.203 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:32:22.203 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.939882+0000", 2024-09-14T06:32:22.203 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.8749999999999997e-06 2024-09-14T06:32:22.203 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:22.203 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:22.203 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:32:22.203 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.939887+0000", 2024-09-14T06:32:22.204 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.741e-06 2024-09-14T06:32:22.204 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:22.204 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:22.204 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:32:22.204 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.013767+0000", 2024-09-14T06:32:22.204 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.073879720999999995 2024-09-14T06:32:22.204 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:22.204 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:22.204 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:32:22.204 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.013832+0000", 2024-09-14T06:32:22.204 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.5312000000000001e-05 2024-09-14T06:32:22.204 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:22.205 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:22.205 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:32:22.205 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.013891+0000", 2024-09-14T06:32:22.205 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.8870999999999999e-05 2024-09-14T06:32:22.205 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:22.205 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:32:22.205 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:22.205 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:22.205 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:22.205 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:115195 3.1 3:856be886:::rbd_data.112d4d24c558.0000000000000004:29 [read 2015232~16384 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e96)", 2024-09-14T06:32:22.205 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:30:17.939903+0000", 2024-09-14T06:32:22.205 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 124.253536647, 2024-09-14T06:32:22.205 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.074128199000000006, 2024-09-14T06:32:22.206 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:32:22.206 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:32:22.206 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:32:22.206 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:32:22.206 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:32:22.206 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:22.206 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 115195 2024-09-14T06:32:22.206 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:22.206 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:32:22.206 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:22.207 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:32:22.207 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.939903+0000", 2024-09-14T06:32:22.207 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:22.207 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:22.207 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:22.207 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:32:22.208 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.939903+0000", 2024-09-14T06:32:22.208 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:22.208 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:22.208 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:22.208 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:32:22.208 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.939904+0000", 2024-09-14T06:32:22.209 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.5799999999999999e-07 2024-09-14T06:32:22.209 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:22.209 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:22.209 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:32:22.209 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.939906+0000", 2024-09-14T06:32:22.210 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.2350000000000002e-06 2024-09-14T06:32:22.210 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:22.210 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:22.210 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:32:22.210 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.939913+0000", 2024-09-14T06:32:22.210 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.5649999999999998e-06 2024-09-14T06:32:22.210 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:22.210 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:22.210 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:32:22.210 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.939918+0000", 2024-09-14T06:32:22.210 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.784e-06 2024-09-14T06:32:22.211 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:22.211 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:22.211 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:32:22.211 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.013921+0000", 2024-09-14T06:32:22.211 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.074002911000000005 2024-09-14T06:32:22.211 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:22.211 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:22.211 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:32:22.211 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.013971+0000", 2024-09-14T06:32:22.211 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.9551e-05 2024-09-14T06:32:22.211 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:22.211 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:22.212 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:32:22.212 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.014031+0000", 2024-09-14T06:32:22.212 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.0594999999999998e-05 2024-09-14T06:32:22.212 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:22.212 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:32:22.212 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:22.212 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:22.212 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:22.212 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:115196 3.1 3:856be886:::rbd_data.112d4d24c558.0000000000000004:29 [read 2260992~16384 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e96)", 2024-09-14T06:32:22.212 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:30:17.939925+0000", 2024-09-14T06:32:22.212 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 124.253514455, 2024-09-14T06:32:22.212 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.074320137999999994, 2024-09-14T06:32:22.212 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:32:22.213 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:32:22.213 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:32:22.213 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:32:22.213 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:32:22.213 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:22.213 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 115196 2024-09-14T06:32:22.213 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:22.213 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:32:22.213 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:22.213 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:32:22.213 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.939925+0000", 2024-09-14T06:32:22.213 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:22.213 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:22.213 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:22.214 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:32:22.214 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.939925+0000", 2024-09-14T06:32:22.214 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:22.214 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:22.214 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:22.214 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:32:22.214 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.939926+0000", 2024-09-14T06:32:22.214 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.9999999999999998e-07 2024-09-14T06:32:22.214 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:22.214 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:22.214 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:32:22.214 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.939928+0000", 2024-09-14T06:32:22.214 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.9549999999999999e-06 2024-09-14T06:32:22.214 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:22.215 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:22.215 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:32:22.215 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.939934+0000", 2024-09-14T06:32:22.215 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.127e-06 2024-09-14T06:32:22.215 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:22.215 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:22.215 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:32:22.215 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.939939+0000", 2024-09-14T06:32:22.215 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.775e-06 2024-09-14T06:32:22.215 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:22.215 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:22.215 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:32:22.215 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.014075+0000", 2024-09-14T06:32:22.215 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.074136598999999997 2024-09-14T06:32:22.215 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:22.216 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:22.216 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:32:22.216 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.014162+0000", 2024-09-14T06:32:22.216 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.6477999999999994e-05 2024-09-14T06:32:22.216 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:22.216 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:22.216 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:32:22.216 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.014246+0000", 2024-09-14T06:32:22.216 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.3703999999999997e-05 2024-09-14T06:32:22.216 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:22.216 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:32:22.216 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:22.216 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:22.217 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:22.217 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:115197 3.1 3:856be886:::rbd_data.112d4d24c558.0000000000000004:29 [read 2506752~16384 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e96)", 2024-09-14T06:32:22.217 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:30:17.939969+0000", 2024-09-14T06:32:22.217 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 124.25347097, 2024-09-14T06:32:22.217 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.074528544000000002, 2024-09-14T06:32:22.217 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:32:22.217 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:32:22.217 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:32:22.217 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:32:22.218 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:32:22.218 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:22.218 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 115197 2024-09-14T06:32:22.218 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:22.218 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:32:22.218 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:22.218 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:32:22.218 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.939969+0000", 2024-09-14T06:32:22.218 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:22.218 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:22.218 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:22.218 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:32:22.218 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.939969+0000", 2024-09-14T06:32:22.219 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:22.219 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:22.219 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:22.219 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:32:22.219 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.939970+0000", 2024-09-14T06:32:22.219 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.3600000000000003e-07 2024-09-14T06:32:22.219 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:22.219 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:22.219 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:32:22.219 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.939972+0000", 2024-09-14T06:32:22.219 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.3180000000000001e-06 2024-09-14T06:32:22.219 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:22.219 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:22.219 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:32:22.220 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.939979+0000", 2024-09-14T06:32:22.220 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.8800000000000002e-06 2024-09-14T06:32:22.220 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:22.220 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:22.220 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:32:22.220 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.939984+0000", 2024-09-14T06:32:22.220 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.3510000000000001e-06 2024-09-14T06:32:22.220 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:22.220 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:22.220 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:32:22.220 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.014302+0000", 2024-09-14T06:32:22.220 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.074317416999999997 2024-09-14T06:32:22.220 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:22.220 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:22.221 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:32:22.221 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.014392+0000", 2024-09-14T06:32:22.221 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.0181999999999998e-05 2024-09-14T06:32:22.221 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:22.221 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:22.221 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:32:22.221 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.014497+0000", 2024-09-14T06:32:22.221 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00010576 2024-09-14T06:32:22.221 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:22.221 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:32:22.221 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:22.221 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:22.221 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:22.222 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:115198 3.1 3:856be886:::rbd_data.112d4d24c558.0000000000000004:29 [read 2752512~16384 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e96)", 2024-09-14T06:32:22.222 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:30:17.940004+0000", 2024-09-14T06:32:22.222 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 124.25343569, 2024-09-14T06:32:22.222 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.074736367999999997, 2024-09-14T06:32:22.222 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:32:22.222 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:32:22.222 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:32:22.222 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:32:22.222 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:32:22.222 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:22.222 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 115198 2024-09-14T06:32:22.222 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:22.222 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:32:22.222 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:22.222 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:32:22.223 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940004+0000", 2024-09-14T06:32:22.223 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:22.223 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:22.223 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:22.223 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:32:22.223 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940004+0000", 2024-09-14T06:32:22.223 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:22.223 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:22.223 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:22.223 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:32:22.223 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940005+0000", 2024-09-14T06:32:22.223 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.2200000000000004e-07 2024-09-14T06:32:22.223 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:22.223 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:22.224 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:32:22.224 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940007+0000", 2024-09-14T06:32:22.224 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.322e-06 2024-09-14T06:32:22.224 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:22.224 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:22.224 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:32:22.224 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940014+0000", 2024-09-14T06:32:22.224 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.6220000000000003e-06 2024-09-14T06:32:22.224 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:22.224 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:22.224 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:32:22.224 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940019+0000", 2024-09-14T06:32:22.224 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.6380000000000001e-06 2024-09-14T06:32:22.224 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:22.225 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:22.225 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:32:22.225 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.014562+0000", 2024-09-14T06:32:22.225 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.074542320999999995 2024-09-14T06:32:22.225 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:22.225 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:22.225 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:32:22.225 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.014646+0000", 2024-09-14T06:32:22.225 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.4079999999999995e-05 2024-09-14T06:32:22.225 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:22.225 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:22.225 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:32:22.225 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.014741+0000", 2024-09-14T06:32:22.225 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.4763000000000001e-05 2024-09-14T06:32:22.225 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:22.226 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:32:22.226 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:22.226 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:22.226 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:22.226 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:115199 3.1 3:856be886:::rbd_data.112d4d24c558.0000000000000004:29 [read 2998272~16384 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e96)", 2024-09-14T06:32:22.226 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:30:17.940040+0000", 2024-09-14T06:32:22.226 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 124.253399935, 2024-09-14T06:32:22.226 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.074957134999999994, 2024-09-14T06:32:22.226 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:32:22.226 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:32:22.226 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:32:22.226 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:32:22.226 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:32:22.227 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:22.227 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 115199 2024-09-14T06:32:22.227 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:22.227 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:32:22.227 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:22.227 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:32:22.227 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940040+0000", 2024-09-14T06:32:22.227 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:22.227 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:22.227 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:22.227 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:32:22.227 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940040+0000", 2024-09-14T06:32:22.227 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:22.227 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:22.227 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:22.228 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:32:22.228 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940041+0000", 2024-09-14T06:32:22.228 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.4000000000000001e-07 2024-09-14T06:32:22.228 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:22.228 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:22.228 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:32:22.228 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940046+0000", 2024-09-14T06:32:22.228 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.7250000000000002e-06 2024-09-14T06:32:22.228 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:22.228 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:22.228 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:32:22.228 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940053+0000", 2024-09-14T06:32:22.228 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.6109999999999999e-06 2024-09-14T06:32:22.229 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:22.229 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:22.229 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:32:22.229 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940058+0000", 2024-09-14T06:32:22.229 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.1590000000000001e-06 2024-09-14T06:32:22.229 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:22.229 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:22.229 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:32:22.229 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.014768+0000", 2024-09-14T06:32:22.229 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.074709624000000002 2024-09-14T06:32:22.229 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:22.229 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:22.229 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:32:22.229 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.014907+0000", 2024-09-14T06:32:22.230 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000139665 2024-09-14T06:32:22.230 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:22.230 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:22.230 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:32:22.230 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.014997+0000", 2024-09-14T06:32:22.230 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.9710999999999995e-05 2024-09-14T06:32:22.230 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:22.230 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:32:22.230 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:22.230 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:22.230 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:22.230 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:115200 3.1 3:856be886:::rbd_data.112d4d24c558.0000000000000004:29 [read 3244032~16384 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e96)", 2024-09-14T06:32:22.230 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:30:17.940080+0000", 2024-09-14T06:32:22.230 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 124.25335973, 2024-09-14T06:32:22.231 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.075137291999999994, 2024-09-14T06:32:22.231 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:32:22.231 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:32:22.231 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:32:22.231 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:32:22.231 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:32:22.231 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:22.231 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 115200 2024-09-14T06:32:22.231 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:22.231 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:32:22.231 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:22.231 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:32:22.231 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940080+0000", 2024-09-14T06:32:22.231 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:22.231 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:22.232 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:22.232 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:32:22.232 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940080+0000", 2024-09-14T06:32:22.232 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:22.232 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:22.232 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:22.232 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:32:22.232 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940081+0000", 2024-09-14T06:32:22.232 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.0800000000000004e-07 2024-09-14T06:32:22.232 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:22.232 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:22.232 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:32:22.232 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940083+0000", 2024-09-14T06:32:22.232 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.3920000000000001e-06 2024-09-14T06:32:22.233 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:22.233 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:22.233 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:32:22.233 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940091+0000", 2024-09-14T06:32:22.233 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.1389999999999995e-06 2024-09-14T06:32:22.233 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:22.233 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:22.233 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:32:22.233 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940097+0000", 2024-09-14T06:32:22.233 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.5550000000000002e-06 2024-09-14T06:32:22.233 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:22.233 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:22.233 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:32:22.233 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.015025+0000", 2024-09-14T06:32:22.234 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.074928579999999995 2024-09-14T06:32:22.234 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:22.234 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:22.234 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:32:22.234 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.015096+0000", 2024-09-14T06:32:22.234 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.0705999999999997e-05 2024-09-14T06:32:22.234 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:22.234 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:22.234 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:32:22.234 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.015217+0000", 2024-09-14T06:32:22.234 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00012131200000000001 2024-09-14T06:32:22.234 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:22.234 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:32:22.234 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:22.235 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:22.235 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:22.235 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:115201 3.1 3:856be886:::rbd_data.112d4d24c558.0000000000000004:29 [read 3489792~16384 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e96)", 2024-09-14T06:32:22.235 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:30:17.940113+0000", 2024-09-14T06:32:22.235 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 124.253326957, 2024-09-14T06:32:22.235 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.075246899000000006, 2024-09-14T06:32:22.235 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:32:22.235 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:32:22.235 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:32:22.235 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:32:22.235 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:32:22.235 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:22.235 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 115201 2024-09-14T06:32:22.235 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:22.235 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:32:22.236 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:22.236 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:32:22.236 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940113+0000", 2024-09-14T06:32:22.236 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:22.236 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:22.236 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:22.236 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:32:22.236 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940113+0000", 2024-09-14T06:32:22.236 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:22.236 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:22.236 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:22.236 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:32:22.236 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940114+0000", 2024-09-14T06:32:22.236 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.1500000000000004e-07 2024-09-14T06:32:22.237 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:22.237 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:22.237 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:32:22.237 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940116+0000", 2024-09-14T06:32:22.237 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.2809999999999998e-06 2024-09-14T06:32:22.237 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:22.237 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:22.237 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:32:22.237 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940123+0000", 2024-09-14T06:32:22.237 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.9229999999999996e-06 2024-09-14T06:32:22.237 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:22.237 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:22.237 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:32:22.237 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940128+0000", 2024-09-14T06:32:22.237 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.7790000000000003e-06 2024-09-14T06:32:22.238 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:22.238 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:22.238 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:32:22.238 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.015253+0000", 2024-09-14T06:32:22.238 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.075125792999999996 2024-09-14T06:32:22.238 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:22.238 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:22.238 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:32:22.239 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.015297+0000", 2024-09-14T06:32:22.239 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.3847000000000002e-05 2024-09-14T06:32:22.239 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:22.239 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:22.239 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:32:22.239 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.015360+0000", 2024-09-14T06:32:22.239 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.2661000000000002e-05 2024-09-14T06:32:22.239 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:22.240 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:32:22.240 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:22.240 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:22.240 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:22.240 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:115202 3.1 3:856be886:::rbd_data.112d4d24c558.0000000000000004:29 [read 3735552~16384 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e96)", 2024-09-14T06:32:22.240 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:30:17.940135+0000", 2024-09-14T06:32:22.241 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 124.25330517800001, 2024-09-14T06:32:22.241 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.075451105000000004, 2024-09-14T06:32:22.241 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:32:22.241 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:32:22.241 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:32:22.241 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:32:22.242 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:32:22.242 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:22.242 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 115202 2024-09-14T06:32:22.242 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:22.242 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:32:22.242 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:22.243 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:32:22.243 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940135+0000", 2024-09-14T06:32:22.243 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:22.243 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:22.243 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:22.243 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:32:22.243 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940135+0000", 2024-09-14T06:32:22.244 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:22.244 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:22.244 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:22.244 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:32:22.244 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940135+0000", 2024-09-14T06:32:22.244 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.0999999999999999e-07 2024-09-14T06:32:22.244 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:22.245 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:22.245 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:32:22.245 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940137+0000", 2024-09-14T06:32:22.245 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.0020000000000001e-06 2024-09-14T06:32:22.245 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:22.245 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:22.246 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:32:22.246 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940143+0000", 2024-09-14T06:32:22.246 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.9850000000000003e-06 2024-09-14T06:32:22.246 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:22.246 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:22.246 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:32:22.246 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940148+0000", 2024-09-14T06:32:22.247 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.9880000000000004e-06 2024-09-14T06:32:22.247 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:22.247 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:22.247 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:32:22.247 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.015400+0000", 2024-09-14T06:32:22.247 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.075252186999999998 2024-09-14T06:32:22.247 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:22.247 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:22.248 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:32:22.248 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.015481+0000", 2024-09-14T06:32:22.248 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.0698000000000006e-05 2024-09-14T06:32:22.248 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:22.248 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:22.248 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:32:22.249 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.015586+0000", 2024-09-14T06:32:22.249 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00010483499999999999 2024-09-14T06:32:22.249 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:22.249 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:32:22.249 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:22.249 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:22.249 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:22.250 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:115203 3.1 3:856be886:::rbd_data.112d4d24c558.0000000000000004:29 [read 3981312~16384 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e96)", 2024-09-14T06:32:22.250 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:30:17.940156+0000", 2024-09-14T06:32:22.250 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 124.25328395299999, 2024-09-14T06:32:22.250 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.075620860999999998, 2024-09-14T06:32:22.250 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:32:22.250 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:32:22.251 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:32:22.251 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:32:22.251 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:32:22.251 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:22.251 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 115203 2024-09-14T06:32:22.251 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:22.252 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:32:22.252 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:22.252 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:32:22.252 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940156+0000", 2024-09-14T06:32:22.252 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:22.252 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:22.252 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:22.253 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:32:22.253 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940156+0000", 2024-09-14T06:32:22.253 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:22.253 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:22.253 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:22.253 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:32:22.253 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940156+0000", 2024-09-14T06:32:22.253 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.1500000000000005e-07 2024-09-14T06:32:22.253 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:22.254 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:22.254 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:32:22.254 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940158+0000", 2024-09-14T06:32:22.254 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.967e-06 2024-09-14T06:32:22.254 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:22.254 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:22.254 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:32:22.254 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940165+0000", 2024-09-14T06:32:22.254 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.6939999999999998e-06 2024-09-14T06:32:22.254 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:22.254 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:22.254 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:32:22.254 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940170+0000", 2024-09-14T06:32:22.254 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.7539999999999997e-06 2024-09-14T06:32:22.255 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:22.255 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:22.255 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:32:22.255 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.015633+0000", 2024-09-14T06:32:22.255 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.075463334000000007 2024-09-14T06:32:22.255 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:22.255 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:22.255 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:32:22.255 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.015693+0000", 2024-09-14T06:32:22.255 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.9367000000000003e-05 2024-09-14T06:32:22.255 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:22.256 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:22.256 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:32:22.256 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.015777+0000", 2024-09-14T06:32:22.256 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.4229999999999998e-05 2024-09-14T06:32:22.256 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:22.256 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:32:22.257 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:22.257 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:22.257 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:22.257 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:115204 3.1 3:856be886:::rbd_data.112d4d24c558.0000000000000004:29 [read 4227072~16384 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e96)", 2024-09-14T06:32:22.257 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:30:17.940176+0000", 2024-09-14T06:32:22.257 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 124.25326342, 2024-09-14T06:32:22.258 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.075785505000000003, 2024-09-14T06:32:22.258 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:32:22.258 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:32:22.258 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:32:22.258 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:32:22.258 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:32:22.259 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:22.259 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 115204 2024-09-14T06:32:22.259 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:22.259 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:32:22.259 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:22.259 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:32:22.260 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940176+0000", 2024-09-14T06:32:22.260 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:22.260 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:22.260 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:22.260 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:32:22.260 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940176+0000", 2024-09-14T06:32:22.260 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:22.260 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:22.260 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:22.261 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:32:22.261 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940177+0000", 2024-09-14T06:32:22.261 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.7899999999999999e-07 2024-09-14T06:32:22.261 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:22.261 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:22.261 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:32:22.261 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940179+0000", 2024-09-14T06:32:22.261 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.103e-06 2024-09-14T06:32:22.261 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:22.262 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:22.262 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:32:22.262 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940185+0000", 2024-09-14T06:32:22.262 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.7239999999999997e-06 2024-09-14T06:32:22.262 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:22.262 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:22.262 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:32:22.262 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940190+0000", 2024-09-14T06:32:22.262 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.7269999999999998e-06 2024-09-14T06:32:22.263 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:22.263 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:22.263 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:32:22.263 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.015814+0000", 2024-09-14T06:32:22.263 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.075624114000000006 2024-09-14T06:32:22.263 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:22.263 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:22.263 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:32:22.263 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.015862+0000", 2024-09-14T06:32:22.263 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.8770999999999998e-05 2024-09-14T06:32:22.264 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:22.264 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:22.264 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:32:22.264 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.015962+0000", 2024-09-14T06:32:22.264 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.9586999999999994e-05 2024-09-14T06:32:22.264 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:22.264 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:32:22.264 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:22.264 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:22.264 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:22.265 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:115205 3.1 3:856be886:::rbd_data.112d4d24c558.0000000000000004:29 [read 4472832~16384 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e96)", 2024-09-14T06:32:22.265 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:30:17.940197+0000", 2024-09-14T06:32:22.265 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 124.253242711, 2024-09-14T06:32:22.265 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.076068920999999998, 2024-09-14T06:32:22.265 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:32:22.265 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:32:22.266 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:32:22.266 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:32:22.266 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:32:22.266 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:22.266 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 115205 2024-09-14T06:32:22.266 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:22.266 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:32:22.266 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:22.266 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:32:22.267 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940197+0000", 2024-09-14T06:32:22.267 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:22.267 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:22.267 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:22.267 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:32:22.267 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940197+0000", 2024-09-14T06:32:22.267 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:22.267 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:22.267 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:22.267 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:32:22.267 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940198+0000", 2024-09-14T06:32:22.267 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.5999999999999999e-07 2024-09-14T06:32:22.268 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:22.268 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:22.268 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:32:22.268 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940200+0000", 2024-09-14T06:32:22.268 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.0899999999999999e-06 2024-09-14T06:32:22.268 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:22.268 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:22.268 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:32:22.268 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940206+0000", 2024-09-14T06:32:22.268 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.8710000000000002e-06 2024-09-14T06:32:22.269 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:22.269 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:22.269 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:32:22.269 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940211+0000", 2024-09-14T06:32:22.269 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.9230000000000001e-06 2024-09-14T06:32:22.269 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:22.269 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:22.269 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:32:22.269 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.016059+0000", 2024-09-14T06:32:22.269 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.075848051999999999 2024-09-14T06:32:22.270 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:22.270 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:22.270 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:32:22.270 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.016151+0000", 2024-09-14T06:32:22.270 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.1982000000000001e-05 2024-09-14T06:32:22.270 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:22.270 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:22.270 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:32:22.270 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.016266+0000", 2024-09-14T06:32:22.270 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000115543 2024-09-14T06:32:22.270 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:22.270 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:32:22.270 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:22.271 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:22.271 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:22.271 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:115206 3.1 3:856be886:::rbd_data.112d4d24c558.0000000000000004:29 [read 4718592~16384 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e96)", 2024-09-14T06:32:22.271 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:30:17.940223+0000", 2024-09-14T06:32:22.271 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 124.25321662100001, 2024-09-14T06:32:22.271 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.076265115999999994, 2024-09-14T06:32:22.271 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:32:22.271 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:32:22.271 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:32:22.271 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:32:22.271 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:32:22.271 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:22.271 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 115206 2024-09-14T06:32:22.271 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:22.271 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:32:22.272 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:22.272 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:32:22.272 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940223+0000", 2024-09-14T06:32:22.272 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:22.272 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:22.272 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:22.272 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:32:22.272 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940223+0000", 2024-09-14T06:32:22.272 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:22.272 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:22.272 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:22.272 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:32:22.272 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940224+0000", 2024-09-14T06:32:22.272 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.0399999999999996e-07 2024-09-14T06:32:22.273 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:22.273 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:22.273 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:32:22.273 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940226+0000", 2024-09-14T06:32:22.273 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.1780000000000002e-06 2024-09-14T06:32:22.273 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:22.273 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:22.273 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:32:22.273 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940232+0000", 2024-09-14T06:32:22.273 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.5010000000000001e-06 2024-09-14T06:32:22.273 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:22.273 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:22.273 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:32:22.274 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940237+0000", 2024-09-14T06:32:22.274 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.0250000000000002e-06 2024-09-14T06:32:22.274 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:22.274 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:22.274 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:32:22.274 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.016305+0000", 2024-09-14T06:32:22.274 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.076067262999999996 2024-09-14T06:32:22.274 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:22.274 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:22.274 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:32:22.274 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.016383+0000", 2024-09-14T06:32:22.274 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.8578000000000006e-05 2024-09-14T06:32:22.274 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:22.274 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:22.275 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:32:22.275 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.016488+0000", 2024-09-14T06:32:22.275 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000105067 2024-09-14T06:32:22.275 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:22.275 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:32:22.275 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:22.275 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:22.275 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:22.275 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:115207 3.1 3:856be886:::rbd_data.112d4d24c558.0000000000000004:29 [read 4964352~16384 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e96)", 2024-09-14T06:32:22.275 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:30:17.940244+0000", 2024-09-14T06:32:22.275 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 124.25319550499999, 2024-09-14T06:32:22.275 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.076371154999999996, 2024-09-14T06:32:22.276 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:32:22.276 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:32:22.276 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:32:22.276 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:32:22.276 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:32:22.276 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:22.276 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 115207 2024-09-14T06:32:22.276 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:22.276 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:32:22.276 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:22.276 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:32:22.276 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940244+0000", 2024-09-14T06:32:22.276 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:22.277 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:22.277 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:22.277 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:32:22.277 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940244+0000", 2024-09-14T06:32:22.277 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:22.277 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:22.277 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:22.277 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:32:22.277 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940245+0000", 2024-09-14T06:32:22.277 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.1200000000000003e-07 2024-09-14T06:32:22.277 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:22.277 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:22.277 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:32:22.278 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940247+0000", 2024-09-14T06:32:22.278 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.0030000000000001e-06 2024-09-14T06:32:22.278 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:22.278 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:22.278 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:32:22.278 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940252+0000", 2024-09-14T06:32:22.278 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.5520000000000004e-06 2024-09-14T06:32:22.278 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:22.278 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:22.278 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:32:22.278 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940257+0000", 2024-09-14T06:32:22.278 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.6759999999999998e-06 2024-09-14T06:32:22.279 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:22.279 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:22.279 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:32:22.279 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.016521+0000", 2024-09-14T06:32:22.279 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.076263499999999998 2024-09-14T06:32:22.279 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:22.279 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:22.279 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:32:22.279 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.016562+0000", 2024-09-14T06:32:22.279 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.0967000000000003e-05 2024-09-14T06:32:22.279 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:22.279 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:22.279 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:32:22.279 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.016616+0000", 2024-09-14T06:32:22.279 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.3944999999999999e-05 2024-09-14T06:32:22.280 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:22.280 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:32:22.280 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:22.280 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:22.280 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:22.280 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:115208 3.1 3:856be886:::rbd_data.112d4d24c558.0000000000000004:29 [read 5210112~16384 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e96)", 2024-09-14T06:32:22.280 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:30:17.940264+0000", 2024-09-14T06:32:22.280 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 124.253175648, 2024-09-14T06:32:22.280 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.076558288000000002, 2024-09-14T06:32:22.280 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:32:22.280 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:32:22.280 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:32:22.281 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:32:22.281 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:32:22.281 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:22.281 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 115208 2024-09-14T06:32:22.281 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:22.281 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:32:22.281 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:22.281 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:32:22.281 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940264+0000", 2024-09-14T06:32:22.281 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:22.281 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:22.281 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:22.281 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:32:22.282 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940264+0000", 2024-09-14T06:32:22.282 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:22.282 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:22.282 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:22.282 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:32:22.282 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940265+0000", 2024-09-14T06:32:22.282 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.0500000000000004e-07 2024-09-14T06:32:22.282 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:22.282 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:22.282 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:32:22.282 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940267+0000", 2024-09-14T06:32:22.282 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.9810000000000002e-06 2024-09-14T06:32:22.282 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:22.283 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:22.283 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:32:22.283 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940272+0000", 2024-09-14T06:32:22.283 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.6999999999999996e-06 2024-09-14T06:32:22.283 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:22.283 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:22.283 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:32:22.283 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940277+0000", 2024-09-14T06:32:22.283 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.6480000000000002e-06 2024-09-14T06:32:22.283 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:22.283 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:22.284 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:32:22.284 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.016650+0000", 2024-09-14T06:32:22.284 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.076373089000000005 2024-09-14T06:32:22.284 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:22.284 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:22.284 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:32:22.284 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.016724+0000", 2024-09-14T06:32:22.284 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.4105999999999998e-05 2024-09-14T06:32:22.284 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:22.284 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:22.284 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:32:22.284 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.016823+0000", 2024-09-14T06:32:22.284 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.8258999999999996e-05 2024-09-14T06:32:22.285 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:22.285 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:32:22.285 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:22.285 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:22.285 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:22.285 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:115209 3.1 3:856be886:::rbd_data.112d4d24c558.0000000000000004:29 [read 5455872~16384 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e96)", 2024-09-14T06:32:22.285 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:30:17.940283+0000", 2024-09-14T06:32:22.285 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 124.25315662200001, 2024-09-14T06:32:22.285 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.076724029999999999, 2024-09-14T06:32:22.285 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:32:22.285 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:32:22.285 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:32:22.285 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:32:22.285 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:32:22.285 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:22.286 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 115209 2024-09-14T06:32:22.286 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:22.286 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:32:22.286 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:22.286 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:32:22.286 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940283+0000", 2024-09-14T06:32:22.286 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:22.286 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:22.286 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:22.286 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:32:22.286 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940283+0000", 2024-09-14T06:32:22.286 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:22.287 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:22.287 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:22.287 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:32:22.287 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940284+0000", 2024-09-14T06:32:22.287 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.6700000000000003e-07 2024-09-14T06:32:22.287 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:22.287 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:22.287 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:32:22.287 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940286+0000", 2024-09-14T06:32:22.287 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.9520000000000001e-06 2024-09-14T06:32:22.287 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:22.287 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:22.287 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:32:22.287 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940292+0000", 2024-09-14T06:32:22.288 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.7339999999999996e-06 2024-09-14T06:32:22.288 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:22.288 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:22.288 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:32:22.288 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940296+0000", 2024-09-14T06:32:22.288 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.7700000000000001e-06 2024-09-14T06:32:22.288 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:22.288 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:22.288 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:32:22.288 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.016859+0000", 2024-09-14T06:32:22.288 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.07656317 2024-09-14T06:32:22.288 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:22.288 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:22.289 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:32:22.289 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.016929+0000", 2024-09-14T06:32:22.289 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.9679999999999997e-05 2024-09-14T06:32:22.289 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:22.289 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:22.289 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:32:22.289 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.017007+0000", 2024-09-14T06:32:22.289 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.8157000000000004e-05 2024-09-14T06:32:22.289 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:22.289 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:32:22.289 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:22.290 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:22.290 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:22.290 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:115210 3.1 3:856be886:::rbd_data.112d4d24c558.0000000000000004:29 [read 5701632~16384 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e96)", 2024-09-14T06:32:22.290 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:30:17.940303+0000", 2024-09-14T06:32:22.290 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 124.25313710499999, 2024-09-14T06:32:22.290 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.076821358000000006, 2024-09-14T06:32:22.290 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:32:22.290 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:32:22.290 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:32:22.290 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:32:22.290 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:32:22.290 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:22.290 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 115210 2024-09-14T06:32:22.290 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:22.290 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:32:22.291 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:22.291 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:32:22.291 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940303+0000", 2024-09-14T06:32:22.291 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:22.291 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:22.291 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:22.291 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:32:22.291 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940303+0000", 2024-09-14T06:32:22.291 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:22.291 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:22.291 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:22.291 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:32:22.291 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940303+0000", 2024-09-14T06:32:22.291 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.3000000000000001e-07 2024-09-14T06:32:22.291 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:22.292 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:22.292 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:32:22.292 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940306+0000", 2024-09-14T06:32:22.292 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.255e-06 2024-09-14T06:32:22.292 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:22.292 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:22.292 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:32:22.292 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940312+0000", 2024-09-14T06:32:22.292 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.9710000000000001e-06 2024-09-14T06:32:22.292 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:22.292 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:22.292 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:32:22.292 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940316+0000", 2024-09-14T06:32:22.292 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.5340000000000001e-06 2024-09-14T06:32:22.293 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:22.293 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:22.293 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:32:22.293 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.017033+0000", 2024-09-14T06:32:22.293 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.076717169000000002 2024-09-14T06:32:22.293 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:22.293 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:22.293 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:32:22.293 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.017073+0000", 2024-09-14T06:32:22.293 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.9582000000000002e-05 2024-09-14T06:32:22.293 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:22.293 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:22.293 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:32:22.294 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.017124+0000", 2024-09-14T06:32:22.294 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.1316999999999997e-05 2024-09-14T06:32:22.294 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:22.294 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:32:22.294 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:22.294 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:22.294 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:22.294 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:115219 3.1 3:856be886:::rbd_data.112d4d24c558.0000000000000004:29 [read 65536~16384 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e96)", 2024-09-14T06:32:22.294 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:30:17.940323+0000", 2024-09-14T06:32:22.294 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 124.25311724399999, 2024-09-14T06:32:22.294 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.076915205, 2024-09-14T06:32:22.294 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:32:22.294 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:32:22.294 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:32:22.295 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:32:22.295 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:32:22.295 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:22.295 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 115219 2024-09-14T06:32:22.295 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:22.295 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:32:22.295 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:22.295 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:32:22.295 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940323+0000", 2024-09-14T06:32:22.295 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:22.295 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:22.295 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:22.295 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:32:22.296 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940323+0000", 2024-09-14T06:32:22.296 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:22.296 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:22.296 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:22.296 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:32:22.296 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940323+0000", 2024-09-14T06:32:22.296 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.0800000000000005e-07 2024-09-14T06:32:22.296 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:22.296 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:22.296 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:32:22.296 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940328+0000", 2024-09-14T06:32:22.296 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.887e-06 2024-09-14T06:32:22.296 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:22.296 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:22.297 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:32:22.297 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940334+0000", 2024-09-14T06:32:22.297 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.3269999999999997e-06 2024-09-14T06:32:22.297 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:22.297 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:22.297 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:32:22.297 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940339+0000", 2024-09-14T06:32:22.297 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.7960000000000003e-06 2024-09-14T06:32:22.297 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:22.297 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:22.297 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:32:22.297 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.017148+0000", 2024-09-14T06:32:22.297 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.076809125000000006 2024-09-14T06:32:22.298 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:22.298 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:22.298 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:32:22.298 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.017186+0000", 2024-09-14T06:32:22.298 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.7283000000000002e-05 2024-09-14T06:32:22.298 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:22.298 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:22.298 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:32:22.298 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.017238+0000", 2024-09-14T06:32:22.298 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.2278999999999999e-05 2024-09-14T06:32:22.298 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:22.298 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:32:22.298 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:22.298 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:22.299 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:22.299 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:115220 3.1 3:856be886:::rbd_data.112d4d24c558.0000000000000004:29 [read 311296~16384 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e96)", 2024-09-14T06:32:22.299 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:30:17.940346+0000", 2024-09-14T06:32:22.299 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 124.253094115, 2024-09-14T06:32:22.299 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.077033293000000003, 2024-09-14T06:32:22.299 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:32:22.299 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:32:22.299 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:32:22.299 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:32:22.299 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:32:22.299 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:22.299 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 115220 2024-09-14T06:32:22.299 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:22.299 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:32:22.299 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:22.299 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:32:22.300 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940346+0000", 2024-09-14T06:32:22.300 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:22.300 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:22.300 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:22.300 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:32:22.300 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940346+0000", 2024-09-14T06:32:22.300 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:22.300 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:22.300 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:22.300 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:32:22.300 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940346+0000", 2024-09-14T06:32:22.300 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.4300000000000003e-07 2024-09-14T06:32:22.300 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:22.301 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:22.301 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:32:22.301 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940348+0000", 2024-09-14T06:32:22.301 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.0030000000000001e-06 2024-09-14T06:32:22.301 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:22.301 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:22.301 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:32:22.301 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.004655+0000", 2024-09-14T06:32:22.301 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.064306323999999998 2024-09-14T06:32:22.302 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:22.302 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:22.302 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:32:22.302 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.004671+0000", 2024-09-14T06:32:22.302 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.6589999999999999e-05 2024-09-14T06:32:22.302 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:22.302 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:22.302 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:32:22.302 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.017262+0000", 2024-09-14T06:32:22.302 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.012590570000000001 2024-09-14T06:32:22.302 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:22.302 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:22.303 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:32:22.303 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.017321+0000", 2024-09-14T06:32:22.303 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.9210999999999999e-05 2024-09-14T06:32:22.303 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:22.303 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:22.303 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:32:22.303 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.017379+0000", 2024-09-14T06:32:22.303 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.8051999999999999e-05 2024-09-14T06:32:22.303 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:22.303 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:32:22.303 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:22.303 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:22.303 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:32:22.304 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-14T06:32:22.304 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-14T06:32:22.306 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-14T06:32:22.306 INFO:teuthology.orchestra.run.smithi094.stdout:osd.0: {} 2024-09-14T06:32:22.306 INFO:teuthology.orchestra.run.smithi094.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-14T06:32:22.307 INFO:teuthology.orchestra.run.smithi094.stdout:osd.1: {} 2024-09-14T06:32:22.307 INFO:teuthology.orchestra.run.smithi094.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-14T06:32:22.307 INFO:teuthology.orchestra.run.smithi094.stdout:osd.2: {} 2024-09-14T06:32:22.308 INFO:teuthology.orchestra.run.smithi094.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-14T06:32:22.308 INFO:teuthology.orchestra.run.smithi094.stdout:osd.3: {} 2024-09-14T06:32:22.308 INFO:teuthology.orchestra.run.smithi094.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-14T06:32:22.309 INFO:tasks.ceph.osd.0.smithi094.stderr:2024-09-14T06:32:22.304+0000 7f07c9333640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33636) UID: 0 2024-09-14T06:32:22.319 INFO:teuthology.orchestra.run.smithi094.stdout:osd.4: {} 2024-09-14T06:32:22.319 INFO:teuthology.orchestra.run.smithi094.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-14T06:32:22.336 INFO:teuthology.orchestra.run.smithi094.stdout:osd.5: {} 2024-09-14T06:32:22.336 INFO:teuthology.orchestra.run.smithi094.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-14T06:32:22.354 INFO:teuthology.orchestra.run.smithi094.stdout:osd.6: {} 2024-09-14T06:32:22.354 INFO:teuthology.orchestra.run.smithi094.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-14T06:32:22.370 INFO:teuthology.orchestra.run.smithi094.stdout:osd.7: {} 2024-09-14T06:32:22.371 INFO:teuthology.orchestra.run.smithi094.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-14T06:32:22.423 INFO:tasks.ceph.osd.0.smithi094.stderr:2024-09-14T06:32:22.422+0000 7f07c9333640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33636) UID: 0 2024-09-14T06:32:22.426 INFO:teuthology.orchestra.run.smithi094.stdout:ERROR: (22) Invalid argument 2024-09-14T06:32:22.426 INFO:teuthology.orchestra.run.smithi094.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:32:22.435 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-14T06:32:22.519 INFO:tasks.ceph.osd.2.smithi094.stderr:2024-09-14T06:32:22.518+0000 7faf3269b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33649) UID: 0 2024-09-14T06:32:22.580 INFO:teuthology.orchestra.run.smithi094.stdout:ERROR: (22) Invalid argument 2024-09-14T06:32:22.580 INFO:teuthology.orchestra.run.smithi094.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:32:22.589 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-14T06:32:22.616 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-14T06:32:22.617 INFO:teuthology.orchestra.run.smithi094.stderr:dumped all 2024-09-14T06:32:22.618 INFO:tasks.ceph.osd.1.smithi094.stderr:2024-09-14T06:32:22.613+0000 7f328e47a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33641) UID: 0 2024-09-14T06:32:22.628 INFO:teuthology.orchestra.run.smithi094.stdout:{"pg_ready":true,"pg_map":{"version":320,"stamp":"2024-09-14T06:32:21.702508+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1944867999,"num_objects":1977,"num_object_clones":0,"num_object_copies":3954,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1977,"num_whiteouts":0,"num_read":231260,"num_read_kb":2177343,"num_write":34903,"num_write_kb":1479469,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4467,"num_bytes_recovered":588103625,"num_keys_recovered":466,"num_objects_omap":42,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":25143,"ondisk_log_size":25143,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":85,"num_osds":7,"num_per_pool_osds":7,"num_per_pool_omap_osds":7,"kb":660602880,"kb_used":562792,"kb_used_data":138940,"kb_used_omap":10,"kb_used_meta":423605,"kb_avail":660040088,"statfs":{"total":676457349120,"available":675881050112,"internally_reserved":0,"allocated":142274560,"data_stored":1250290574,"data_compressed":77375366,"data_compressed_allocated":102125568,"data_compressed_original":1224070546,"omap_allocated":11157,"internal_metadata":433771627},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-60654732,"num_objects":-178,"num_object_clones":0,"num_object_copies":-356,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-156,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-178,"num_whiteouts":0,"num_read":352,"num_read_kb":298,"num_write":240,"num_write_kb":27,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":334,"num_bytes_recovered":32598819,"num_keys_recovered":0,"num_objects_omap":-2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.030819"},"pg_stats":[{"pgid":"3.1f","version":"211'731","reported_seq":7808,"reported_epoch":212,"state":"active+clean","last_fresh":"2024-09-14T06:32:17.687926+0000","last_change":"2024-09-14T06:32:16.675117+0000","last_active":"2024-09-14T06:32:17.687926+0000","last_peered":"2024-09-14T06:32:17.687926+0000","last_clean":"2024-09-14T06:32:17.687926+0000","last_became_active":"2024-09-14T06:29:00.579883+0000","last_became_peered":"2024-09-14T06:29:00.579883+0000","last_unstale":"2024-09-14T06:32:17.687926+0000","last_undegraded":"2024-09-14T06:32:17.687926+0000","last_fullsized":"2024-09-14T06:32:17.687926+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":731,"log_dups_size":0,"ondisk_log_size":731,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T15:33:15.819830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00060207499999999998,"stat_sum":{"num_bytes":90557590,"num_objects":66,"num_object_clones":0,"num_object_copies":132,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":66,"num_whiteouts":0,"num_read":8088,"num_read_kb":86571,"num_write":1060,"num_write_kb":49526,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"53","length":"1"},{"start":"55","length":"1"},{"start":"57","length":"1"},{"start":"59","length":"1"},{"start":"5b","length":"1"}]},{"pgid":"3.1e","version":"215'721","reported_seq":9811,"reported_epoch":215,"state":"active+clean","last_fresh":"2024-09-14T06:32:20.683254+0000","last_change":"2024-09-14T06:32:20.618019+0000","last_active":"2024-09-14T06:32:20.683254+0000","last_peered":"2024-09-14T06:32:20.683254+0000","last_clean":"2024-09-14T06:32:20.683254+0000","last_became_active":"2024-09-14T06:32:17.679198+0000","last_became_peered":"2024-09-14T06:32:17.679198+0000","last_unstale":"2024-09-14T06:32:20.683254+0000","last_undegraded":"2024-09-14T06:32:20.683254+0000","last_fullsized":"2024-09-14T06:32:20.683254+0000","mapping_epoch":211,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":212,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":721,"log_dups_size":0,"ondisk_log_size":721,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T14:42:15.707761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00087564100000000003,"stat_sum":{"num_bytes":34202102,"num_objects":49,"num_object_clones":0,"num_object_copies":98,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":49,"num_whiteouts":0,"num_read":8274,"num_read_kb":67629,"num_write":963,"num_write_kb":40402,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":713,"num_bytes_recovered":87148703,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"4d","length":"1"},{"start":"4f","length":"1"},{"start":"51","length":"1"},{"start":"53","length":"1"},{"start":"55","length":"1"}]},{"pgid":"3.1d","version":"213'951","reported_seq":6980,"reported_epoch":213,"state":"active+clean","last_fresh":"2024-09-14T06:32:18.816743+0000","last_change":"2024-09-14T06:32:18.693003+0000","last_active":"2024-09-14T06:32:18.816743+0000","last_peered":"2024-09-14T06:32:18.816743+0000","last_clean":"2024-09-14T06:32:18.816743+0000","last_became_active":"2024-09-14T06:29:00.579345+0000","last_became_peered":"2024-09-14T06:29:00.579345+0000","last_unstale":"2024-09-14T06:32:18.816743+0000","last_undegraded":"2024-09-14T06:32:18.816743+0000","last_fullsized":"2024-09-14T06:32:18.816743+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":951,"log_dups_size":0,"ondisk_log_size":951,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T06:49:45.277635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00058411300000000004,"stat_sum":{"num_bytes":84053567,"num_objects":67,"num_object_clones":0,"num_object_copies":134,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":67,"num_whiteouts":0,"num_read":7603,"num_read_kb":64150,"num_write":1433,"num_write_kb":53271,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"51","length":"1"},{"start":"53","length":"1"},{"start":"55","length":"1"},{"start":"57","length":"1"},{"start":"59","length":"1"},{"start":"5b","length":"1"}]},{"pgid":"3.1c","version":"213'661","reported_seq":5296,"reported_epoch":213,"state":"active+clean","last_fresh":"2024-09-14T06:32:18.819104+0000","last_change":"2024-09-14T06:32:18.692709+0000","last_active":"2024-09-14T06:32:18.819104+0000","last_peered":"2024-09-14T06:32:18.819104+0000","last_clean":"2024-09-14T06:32:18.819104+0000","last_became_active":"2024-09-14T06:32:04.618831+0000","last_became_peered":"2024-09-14T06:32:04.618831+0000","last_unstale":"2024-09-14T06:32:18.819104+0000","last_undegraded":"2024-09-14T06:32:18.819104+0000","last_fullsized":"2024-09-14T06:32:18.819104+0000","mapping_epoch":199,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":200,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":661,"log_dups_size":0,"ondisk_log_size":661,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T07:41:34.156871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049958700000000001,"stat_sum":{"num_bytes":40890488,"num_objects":60,"num_object_clones":0,"num_object_copies":120,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":4408,"num_read_kb":43312,"num_write":777,"num_write_kb":45539,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":319,"num_bytes_recovered":32800897,"num_keys_recovered":223,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"51","length":"1"},{"start":"53","length":"1"},{"start":"55","length":"1"},{"start":"57","length":"1"},{"start":"59","length":"1"},{"start":"5b","length":"1"}]},{"pgid":"3.1b","version":"211'848","reported_seq":10978,"reported_epoch":212,"state":"active+clean","last_fresh":"2024-09-14T06:32:17.688246+0000","last_change":"2024-09-14T06:32:16.675101+0000","last_active":"2024-09-14T06:32:17.688246+0000","last_peered":"2024-09-14T06:32:17.688246+0000","last_clean":"2024-09-14T06:32:17.688246+0000","last_became_active":"2024-09-14T06:29:00.586183+0000","last_became_peered":"2024-09-14T06:29:00.586183+0000","last_unstale":"2024-09-14T06:32:17.688246+0000","last_undegraded":"2024-09-14T06:32:17.688246+0000","last_fullsized":"2024-09-14T06:32:17.688246+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":848,"log_dups_size":0,"ondisk_log_size":848,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T15:10:42.430832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001113331,"stat_sum":{"num_bytes":76920933,"num_objects":77,"num_object_clones":0,"num_object_copies":154,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":77,"num_whiteouts":0,"num_read":11697,"num_read_kb":107915,"num_write":1357,"num_write_kb":55091,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"53","length":"1"},{"start":"55","length":"1"},{"start":"57","length":"1"},{"start":"59","length":"1"},{"start":"5b","length":"1"}]},{"pgid":"3.1a","version":"213'1522","reported_seq":9739,"reported_epoch":213,"state":"active+clean","last_fresh":"2024-09-14T06:32:18.813477+0000","last_change":"2024-09-14T06:32:18.687520+0000","last_active":"2024-09-14T06:32:18.813477+0000","last_peered":"2024-09-14T06:32:18.813477+0000","last_clean":"2024-09-14T06:32:18.813477+0000","last_became_active":"2024-09-14T06:29:00.576206+0000","last_became_peered":"2024-09-14T06:29:00.576206+0000","last_unstale":"2024-09-14T06:32:18.813477+0000","last_undegraded":"2024-09-14T06:32:18.813477+0000","last_fullsized":"2024-09-14T06:32:18.813477+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":1522,"log_dups_size":0,"ondisk_log_size":1522,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T14:15:40.076141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000631658,"stat_sum":{"num_bytes":66356871,"num_objects":67,"num_object_clones":0,"num_object_copies":134,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":67,"num_whiteouts":0,"num_read":12628,"num_read_kb":82269,"num_write":2301,"num_write_kb":55655,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"51","length":"1"},{"start":"53","length":"1"},{"start":"55","length":"1"},{"start":"57","length":"1"},{"start":"59","length":"1"},{"start":"5b","length":"1"}]},{"pgid":"3.19","version":"213'493","reported_seq":6083,"reported_epoch":213,"state":"active+clean","last_fresh":"2024-09-14T06:32:18.840327+0000","last_change":"2024-09-14T06:32:18.684241+0000","last_active":"2024-09-14T06:32:18.840327+0000","last_peered":"2024-09-14T06:32:18.840327+0000","last_clean":"2024-09-14T06:32:18.840327+0000","last_became_active":"2024-09-14T06:31:53.744800+0000","last_became_peered":"2024-09-14T06:31:53.744800+0000","last_unstale":"2024-09-14T06:32:18.840327+0000","last_undegraded":"2024-09-14T06:32:18.840327+0000","last_fullsized":"2024-09-14T06:32:18.840327+0000","mapping_epoch":188,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":189,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":493,"log_dups_size":0,"ondisk_log_size":493,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T17:14:42.801160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00052827500000000003,"stat_sum":{"num_bytes":56438675,"num_objects":55,"num_object_clones":0,"num_object_copies":110,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":55,"num_whiteouts":0,"num_read":5081,"num_read_kb":67753,"num_write":696,"num_write_kb":43868,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":282,"num_bytes_recovered":41927489,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"51","length":"1"},{"start":"53","length":"1"},{"start":"55","length":"1"},{"start":"57","length":"1"},{"start":"59","length":"1"},{"start":"5b","length":"1"}]},{"pgid":"3.18","version":"215'474","reported_seq":5880,"reported_epoch":215,"state":"active+clean","last_fresh":"2024-09-14T06:32:20.681362+0000","last_change":"2024-09-14T06:32:20.617466+0000","last_active":"2024-09-14T06:32:20.681362+0000","last_peered":"2024-09-14T06:32:20.681362+0000","last_clean":"2024-09-14T06:32:20.681362+0000","last_became_active":"2024-09-14T06:31:53.745159+0000","last_became_peered":"2024-09-14T06:31:53.745159+0000","last_unstale":"2024-09-14T06:32:20.681362+0000","last_undegraded":"2024-09-14T06:32:20.681362+0000","last_fullsized":"2024-09-14T06:32:20.681362+0000","mapping_epoch":188,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":189,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":474,"log_dups_size":0,"ondisk_log_size":474,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T07:57:49.108695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054220099999999997,"stat_sum":{"num_bytes":59946747,"num_objects":60,"num_object_clones":0,"num_object_copies":120,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":4455,"num_read_kb":78892,"num_write":599,"num_write_kb":51576,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":310,"num_bytes_recovered":56082976,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"4d","length":"1"},{"start":"4f","length":"1"},{"start":"51","length":"1"},{"start":"53","length":"1"},{"start":"55","length":"1"}]},{"pgid":"3.17","version":"212'867","reported_seq":9002,"reported_epoch":212,"state":"active+clean","last_fresh":"2024-09-14T06:32:17.725724+0000","last_change":"2024-09-14T06:32:16.675334+0000","last_active":"2024-09-14T06:32:17.725724+0000","last_peered":"2024-09-14T06:32:17.725724+0000","last_clean":"2024-09-14T06:32:17.725724+0000","last_became_active":"2024-09-14T06:29:00.579714+0000","last_became_peered":"2024-09-14T06:29:00.579714+0000","last_unstale":"2024-09-14T06:32:17.725724+0000","last_undegraded":"2024-09-14T06:32:17.725724+0000","last_fullsized":"2024-09-14T06:32:17.725724+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":867,"log_dups_size":0,"ondisk_log_size":867,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T13:34:00.327619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00052952999999999995,"stat_sum":{"num_bytes":77132538,"num_objects":77,"num_object_clones":0,"num_object_copies":154,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":77,"num_whiteouts":0,"num_read":10142,"num_read_kb":68496,"num_write":1004,"num_write_kb":48454,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"53","length":"1"},{"start":"55","length":"1"},{"start":"57","length":"1"},{"start":"59","length":"1"},{"start":"5b","length":"1"}]},{"pgid":"3.16","version":"213'1041","reported_seq":3600,"reported_epoch":213,"state":"active+clean","last_fresh":"2024-09-14T06:32:18.792112+0000","last_change":"2024-09-14T06:32:18.692881+0000","last_active":"2024-09-14T06:32:18.792112+0000","last_peered":"2024-09-14T06:32:18.792112+0000","last_clean":"2024-09-14T06:32:18.792112+0000","last_became_active":"2024-09-14T06:29:00.580070+0000","last_became_peered":"2024-09-14T06:29:00.580070+0000","last_unstale":"2024-09-14T06:32:18.792112+0000","last_undegraded":"2024-09-14T06:32:18.792112+0000","last_fullsized":"2024-09-14T06:32:18.792112+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":1041,"log_dups_size":0,"ondisk_log_size":1041,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T11:55:58.672916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050943400000000002,"stat_sum":{"num_bytes":41312662,"num_objects":62,"num_object_clones":0,"num_object_copies":124,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":62,"num_whiteouts":0,"num_read":5281,"num_read_kb":38236,"num_write":1636,"num_write_kb":37231,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"51","length":"1"},{"start":"53","length":"1"},{"start":"55","length":"1"},{"start":"57","length":"1"},{"start":"59","length":"1"},{"start":"5b","length":"1"}]},{"pgid":"3.15","version":"213'554","reported_seq":6235,"reported_epoch":213,"state":"active+clean","last_fresh":"2024-09-14T06:32:18.775731+0000","last_change":"2024-09-14T06:32:18.775731+0000","last_active":"2024-09-14T06:32:18.775731+0000","last_peered":"2024-09-14T06:32:18.775731+0000","last_clean":"2024-09-14T06:32:18.775731+0000","last_became_active":"2024-09-14T06:31:53.746091+0000","last_became_peered":"2024-09-14T06:31:53.746091+0000","last_unstale":"2024-09-14T06:32:18.775731+0000","last_undegraded":"2024-09-14T06:32:18.775731+0000","last_fullsized":"2024-09-14T06:32:18.775731+0000","mapping_epoch":188,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":189,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":554,"log_dups_size":0,"ondisk_log_size":554,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T14:56:49.797528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001603075,"stat_sum":{"num_bytes":79316073,"num_objects":60,"num_object_clones":0,"num_object_copies":120,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":4897,"num_read_kb":74724,"num_write":607,"num_write_kb":51185,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":271,"num_bytes_recovered":49981473,"num_keys_recovered":33,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"51","length":"1"},{"start":"53","length":"1"},{"start":"55","length":"1"},{"start":"57","length":"1"},{"start":"59","length":"1"},{"start":"5b","length":"1"}]},{"pgid":"3.14","version":"213'1043","reported_seq":5749,"reported_epoch":213,"state":"active+clean","last_fresh":"2024-09-14T06:32:18.807986+0000","last_change":"2024-09-14T06:32:18.688072+0000","last_active":"2024-09-14T06:32:18.807986+0000","last_peered":"2024-09-14T06:32:18.807986+0000","last_clean":"2024-09-14T06:32:18.807986+0000","last_became_active":"2024-09-14T06:29:00.582595+0000","last_became_peered":"2024-09-14T06:29:00.582595+0000","last_unstale":"2024-09-14T06:32:18.807986+0000","last_undegraded":"2024-09-14T06:32:18.807986+0000","last_fullsized":"2024-09-14T06:32:18.807986+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":1043,"log_dups_size":0,"ondisk_log_size":1043,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T14:01:59.640498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076514300000000003,"stat_sum":{"num_bytes":50793843,"num_objects":47,"num_object_clones":0,"num_object_copies":94,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":47,"num_whiteouts":0,"num_read":8593,"num_read_kb":46630,"num_write":1494,"num_write_kb":29817,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"51","length":"1"},{"start":"53","length":"1"},{"start":"55","length":"1"},{"start":"57","length":"1"},{"start":"59","length":"1"},{"start":"5b","length":"1"}]},{"pgid":"2.7","version":"0'0","reported_seq":44,"reported_epoch":213,"state":"active+clean","last_fresh":"2024-09-14T06:32:18.774473+0000","last_change":"2024-09-14T06:32:01.612470+0000","last_active":"2024-09-14T06:32:18.774473+0000","last_peered":"2024-09-14T06:32:18.774473+0000","last_clean":"2024-09-14T06:32:18.774473+0000","last_became_active":"2024-09-14T06:32:01.612121+0000","last_became_peered":"2024-09-14T06:32:01.612121+0000","last_unstale":"2024-09-14T06:32:18.774473+0000","last_undegraded":"2024-09-14T06:32:18.774473+0000","last_fullsized":"2024-09-14T06:32:18.774473+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:28:50.205350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:28:50.205350+0000","last_clean_scrub_stamp":"2024-09-14T06:28:50.205350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T13:40:52.537942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.6","version":"212'687","reported_seq":7942,"reported_epoch":212,"state":"active+clean","last_fresh":"2024-09-14T06:32:17.687977+0000","last_change":"2024-09-14T06:32:16.674174+0000","last_active":"2024-09-14T06:32:17.687977+0000","last_peered":"2024-09-14T06:32:17.687977+0000","last_clean":"2024-09-14T06:32:17.687977+0000","last_became_active":"2024-09-14T06:29:00.578283+0000","last_became_peered":"2024-09-14T06:29:00.578283+0000","last_unstale":"2024-09-14T06:32:17.687977+0000","last_undegraded":"2024-09-14T06:32:17.687977+0000","last_fullsized":"2024-09-14T06:32:17.687977+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":687,"log_dups_size":0,"ondisk_log_size":687,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T15:54:45.137038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00107485,"stat_sum":{"num_bytes":65980763,"num_objects":61,"num_object_clones":0,"num_object_copies":122,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":61,"num_whiteouts":0,"num_read":7993,"num_read_kb":78244,"num_write":775,"num_write_kb":48270,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"53","length":"1"},{"start":"55","length":"1"},{"start":"57","length":"1"},{"start":"59","length":"1"},{"start":"5b","length":"1"}]},{"pgid":"2.6","version":"0'0","reported_seq":409,"reported_epoch":213,"state":"active+clean","last_fresh":"2024-09-14T06:32:18.684058+0000","last_change":"2024-09-14T06:32:01.608456+0000","last_active":"2024-09-14T06:32:18.684058+0000","last_peered":"2024-09-14T06:32:18.684058+0000","last_clean":"2024-09-14T06:32:18.684058+0000","last_became_active":"2024-09-14T06:32:01.608178+0000","last_became_peered":"2024-09-14T06:32:01.608178+0000","last_unstale":"2024-09-14T06:32:18.684058+0000","last_undegraded":"2024-09-14T06:32:18.684058+0000","last_fullsized":"2024-09-14T06:32:18.684058+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:28:50.205350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:28:50.205350+0000","last_clean_scrub_stamp":"2024-09-14T06:28:50.205350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T10:16:40.480031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065018200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"213'757","reported_seq":4161,"reported_epoch":215,"state":"active+clean","last_fresh":"2024-09-14T06:32:20.617997+0000","last_change":"2024-09-14T06:32:20.618192+0000","last_active":"2024-09-14T06:32:20.617997+0000","last_peered":"2024-09-14T06:32:20.617997+0000","last_clean":"2024-09-14T06:32:20.617997+0000","last_became_active":"2024-09-14T06:31:54.747923+0000","last_became_peered":"2024-09-14T06:31:54.747923+0000","last_unstale":"2024-09-14T06:32:20.617997+0000","last_undegraded":"2024-09-14T06:32:20.617997+0000","last_fullsized":"2024-09-14T06:32:20.617997+0000","mapping_epoch":189,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":190,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":757,"log_dups_size":0,"ondisk_log_size":757,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T14:50:58.127377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00096002999999999995,"stat_sum":{"num_bytes":37624111,"num_objects":47,"num_object_clones":0,"num_object_copies":94,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":47,"num_whiteouts":0,"num_read":4380,"num_read_kb":40951,"num_write":1102,"num_write_kb":36111,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":183,"num_bytes_recovered":13912513,"num_keys_recovered":22,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"4d","length":"1"},{"start":"4f","length":"1"},{"start":"51","length":"1"},{"start":"53","length":"1"},{"start":"55","length":"1"}]},{"pgid":"2.5","version":"0'0","reported_seq":403,"reported_epoch":213,"state":"active+clean","last_fresh":"2024-09-14T06:32:18.774592+0000","last_change":"2024-09-14T06:28:53.640035+0000","last_active":"2024-09-14T06:32:18.774592+0000","last_peered":"2024-09-14T06:32:18.774592+0000","last_clean":"2024-09-14T06:32:18.774592+0000","last_became_active":"2024-09-14T06:28:51.308058+0000","last_became_peered":"2024-09-14T06:28:51.308058+0000","last_unstale":"2024-09-14T06:32:18.774592+0000","last_undegraded":"2024-09-14T06:32:18.774592+0000","last_fullsized":"2024-09-14T06:32:18.774592+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:28:50.205350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:28:50.205350+0000","last_clean_scrub_stamp":"2024-09-14T06:28:50.205350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T13:35:48.544057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00037824000000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"213'620","reported_seq":3711,"reported_epoch":213,"state":"active+clean","last_fresh":"2024-09-14T06:32:18.799963+0000","last_change":"2024-09-14T06:32:18.684244+0000","last_active":"2024-09-14T06:32:18.799963+0000","last_peered":"2024-09-14T06:32:18.799963+0000","last_clean":"2024-09-14T06:32:18.799963+0000","last_became_active":"2024-09-14T06:32:17.675470+0000","last_became_peered":"2024-09-14T06:32:17.675470+0000","last_unstale":"2024-09-14T06:32:18.799963+0000","last_undegraded":"2024-09-14T06:32:18.799963+0000","last_fullsized":"2024-09-14T06:32:18.799963+0000","mapping_epoch":211,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":212,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":620,"log_dups_size":0,"ondisk_log_size":620,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T10:44:48.979718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00052450000000000001,"stat_sum":{"num_bytes":35993068,"num_objects":57,"num_object_clones":0,"num_object_copies":114,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":57,"num_whiteouts":0,"num_read":2592,"num_read_kb":37387,"num_write":814,"num_write_kb":40850,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":331,"num_bytes_recovered":31411119,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"51","length":"1"},{"start":"53","length":"1"},{"start":"55","length":"1"},{"start":"57","length":"1"},{"start":"59","length":"1"},{"start":"5b","length":"1"}]},{"pgid":"2.4","version":"0'0","reported_seq":403,"reported_epoch":213,"state":"active+clean","last_fresh":"2024-09-14T06:32:18.683953+0000","last_change":"2024-09-14T06:28:53.226072+0000","last_active":"2024-09-14T06:32:18.683953+0000","last_peered":"2024-09-14T06:32:18.683953+0000","last_clean":"2024-09-14T06:32:18.683953+0000","last_became_active":"2024-09-14T06:28:51.308072+0000","last_became_peered":"2024-09-14T06:28:51.308072+0000","last_unstale":"2024-09-14T06:32:18.683953+0000","last_undegraded":"2024-09-14T06:32:18.683953+0000","last_fullsized":"2024-09-14T06:32:18.683953+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:28:50.205350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:28:50.205350+0000","last_clean_scrub_stamp":"2024-09-14T06:28:50.205350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T13:15:09.052328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066197399999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"213'830","reported_seq":5270,"reported_epoch":213,"state":"active+clean","last_fresh":"2024-09-14T06:32:18.693280+0000","last_change":"2024-09-14T06:32:18.693280+0000","last_active":"2024-09-14T06:32:18.693280+0000","last_peered":"2024-09-14T06:32:18.693280+0000","last_clean":"2024-09-14T06:32:18.693280+0000","last_became_active":"2024-09-14T06:31:54.744373+0000","last_became_peered":"2024-09-14T06:31:54.744373+0000","last_unstale":"2024-09-14T06:32:18.693280+0000","last_undegraded":"2024-09-14T06:32:18.693280+0000","last_fullsized":"2024-09-14T06:32:18.693280+0000","mapping_epoch":189,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":190,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":830,"log_dups_size":0,"ondisk_log_size":830,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T08:59:08.886122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00083983799999999998,"stat_sum":{"num_bytes":42904196,"num_objects":64,"num_object_clones":0,"num_object_copies":128,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":4507,"num_read_kb":42629,"num_write":1191,"num_write_kb":49494,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":330,"num_bytes_recovered":28798893,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"51","length":"1"},{"start":"53","length":"1"},{"start":"55","length":"1"},{"start":"57","length":"1"},{"start":"59","length":"1"},{"start":"5b","length":"1"}]},{"pgid":"3.3","version":"213'482","reported_seq":10122,"reported_epoch":213,"state":"active+clean","last_fresh":"2024-09-14T06:32:18.836145+0000","last_change":"2024-09-14T06:32:18.687930+0000","last_active":"2024-09-14T06:32:18.836145+0000","last_peered":"2024-09-14T06:32:18.836145+0000","last_clean":"2024-09-14T06:32:18.836145+0000","last_became_active":"2024-09-14T06:29:00.584678+0000","last_became_peered":"2024-09-14T06:29:00.584678+0000","last_unstale":"2024-09-14T06:32:18.836145+0000","last_undegraded":"2024-09-14T06:32:18.836145+0000","last_fullsized":"2024-09-14T06:32:18.836145+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":482,"log_dups_size":0,"ondisk_log_size":482,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T18:18:06.483246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00080170399999999998,"stat_sum":{"num_bytes":62215579,"num_objects":51,"num_object_clones":0,"num_object_copies":102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":51,"num_whiteouts":0,"num_read":9164,"num_read_kb":88936,"num_write":555,"num_write_kb":51568,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"51","length":"1"},{"start":"53","length":"1"},{"start":"55","length":"1"},{"start":"57","length":"1"},{"start":"59","length":"1"},{"start":"5b","length":"1"}]},{"pgid":"2.2","version":"22'2","reported_seq":405,"reported_epoch":213,"state":"active+clean","last_fresh":"2024-09-14T06:32:18.692172+0000","last_change":"2024-09-14T06:28:53.231428+0000","last_active":"2024-09-14T06:32:18.692172+0000","last_peered":"2024-09-14T06:32:18.692172+0000","last_clean":"2024-09-14T06:32:18.692172+0000","last_became_active":"2024-09-14T06:28:51.222937+0000","last_became_peered":"2024-09-14T06:28:51.222937+0000","last_unstale":"2024-09-14T06:32:18.692172+0000","last_undegraded":"2024-09-14T06:32:18.692172+0000","last_fullsized":"2024-09-14T06:32:18.692172+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:28:50.205350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:28:50.205350+0000","last_clean_scrub_stamp":"2024-09-14T06:28:50.205350+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T14:27:37.233976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010072970000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.0","version":"213'1000","reported_seq":6594,"reported_epoch":213,"state":"active+clean","last_fresh":"2024-09-14T06:32:18.773421+0000","last_change":"2024-09-14T06:32:18.684676+0000","last_active":"2024-09-14T06:32:18.773421+0000","last_peered":"2024-09-14T06:32:18.773421+0000","last_clean":"2024-09-14T06:32:18.773421+0000","last_became_active":"2024-09-14T06:29:00.574873+0000","last_became_peered":"2024-09-14T06:29:00.574873+0000","last_unstale":"2024-09-14T06:32:18.773421+0000","last_undegraded":"2024-09-14T06:32:18.773421+0000","last_fullsized":"2024-09-14T06:32:18.773421+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":1000,"log_dups_size":0,"ondisk_log_size":1000,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T14:21:46.942649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00084098799999999998,"stat_sum":{"num_bytes":73791467,"num_objects":71,"num_object_clones":0,"num_object_copies":142,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":71,"num_whiteouts":0,"num_read":8014,"num_read_kb":85277,"num_write":1539,"num_write_kb":58684,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"51","length":"1"},{"start":"53","length":"1"},{"start":"55","length":"1"},{"start":"57","length":"1"},{"start":"59","length":"1"},{"start":"5b","length":"1"}]},{"pgid":"2.1","version":"0'0","reported_seq":17,"reported_epoch":212,"state":"active+clean","last_fresh":"2024-09-14T06:32:17.674606+0000","last_change":"2024-09-14T06:32:14.665376+0000","last_active":"2024-09-14T06:32:17.674606+0000","last_peered":"2024-09-14T06:32:17.674606+0000","last_clean":"2024-09-14T06:32:17.674606+0000","last_became_active":"2024-09-14T06:32:14.665081+0000","last_became_peered":"2024-09-14T06:32:14.665081+0000","last_unstale":"2024-09-14T06:32:17.674606+0000","last_undegraded":"2024-09-14T06:32:17.674606+0000","last_fullsized":"2024-09-14T06:32:17.674606+0000","mapping_epoch":208,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":209,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:28:50.205350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:28:50.205350+0000","last_clean_scrub_stamp":"2024-09-14T06:28:50.205350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T16:35:12.869944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053403100000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1","version":"208'696","reported_seq":11325,"reported_epoch":212,"state":"active+clean","last_fresh":"2024-09-14T06:32:17.687869+0000","last_change":"2024-09-14T06:32:16.673919+0000","last_active":"2024-09-14T06:32:17.687869+0000","last_peered":"2024-09-14T06:32:17.687869+0000","last_clean":"2024-09-14T06:32:17.687869+0000","last_became_active":"2024-09-14T06:30:57.768238+0000","last_became_peered":"2024-09-14T06:30:57.768238+0000","last_unstale":"2024-09-14T06:32:17.687869+0000","last_undegraded":"2024-09-14T06:32:17.687869+0000","last_fullsized":"2024-09-14T06:32:17.687869+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":696,"log_dups_size":0,"ondisk_log_size":696,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T09:26:24.780770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001653273,"stat_sum":{"num_bytes":93623859,"num_objects":65,"num_object_clones":0,"num_object_copies":130,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":65,"num_whiteouts":0,"num_read":11016,"num_read_kb":112206,"num_write":871,"num_write_kb":47387,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":183,"num_bytes_recovered":30431994,"num_keys_recovered":22,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"53","length":"1"},{"start":"55","length":"1"},{"start":"57","length":"1"},{"start":"59","length":"1"},{"start":"5b","length":"1"}]},{"pgid":"2.0","version":"0'0","reported_seq":403,"reported_epoch":213,"state":"active+clean","last_fresh":"2024-09-14T06:32:18.773749+0000","last_change":"2024-09-14T06:28:53.640096+0000","last_active":"2024-09-14T06:32:18.773749+0000","last_peered":"2024-09-14T06:32:18.773749+0000","last_clean":"2024-09-14T06:32:18.773749+0000","last_became_active":"2024-09-14T06:28:51.223069+0000","last_became_peered":"2024-09-14T06:28:51.223069+0000","last_unstale":"2024-09-14T06:32:18.773749+0000","last_undegraded":"2024-09-14T06:32:18.773749+0000","last_fullsized":"2024-09-14T06:32:18.773749+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:28:50.205350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:28:50.205350+0000","last_clean_scrub_stamp":"2024-09-14T06:28:50.205350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T15:13:00.471509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00043714600000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"213'1573","reported_seq":7425,"reported_epoch":215,"state":"active+clean","last_fresh":"2024-09-14T06:32:20.617385+0000","last_change":"2024-09-14T06:32:20.617539+0000","last_active":"2024-09-14T06:32:20.617385+0000","last_peered":"2024-09-14T06:32:20.617385+0000","last_clean":"2024-09-14T06:32:20.617385+0000","last_became_active":"2024-09-14T06:31:54.744160+0000","last_became_peered":"2024-09-14T06:31:54.744160+0000","last_unstale":"2024-09-14T06:32:20.617385+0000","last_undegraded":"2024-09-14T06:32:20.617385+0000","last_fullsized":"2024-09-14T06:32:20.617385+0000","mapping_epoch":189,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":190,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":1573,"log_dups_size":0,"ondisk_log_size":1573,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T18:19:29.477948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049309400000000002,"stat_sum":{"num_bytes":66318026,"num_objects":76,"num_object_clones":0,"num_object_copies":152,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":76,"num_whiteouts":0,"num_read":10684,"num_read_kb":70562,"num_write":2507,"num_write_kb":53559,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":327,"num_bytes_recovered":50047063,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"4d","length":"1"},{"start":"4f","length":"1"},{"start":"51","length":"1"},{"start":"53","length":"1"},{"start":"55","length":"1"}]},{"pgid":"2.3","version":"20'1","reported_seq":404,"reported_epoch":213,"state":"active+clean","last_fresh":"2024-09-14T06:32:18.692057+0000","last_change":"2024-09-14T06:28:53.230933+0000","last_active":"2024-09-14T06:32:18.692057+0000","last_peered":"2024-09-14T06:32:18.692057+0000","last_clean":"2024-09-14T06:32:18.692057+0000","last_became_active":"2024-09-14T06:28:51.222874+0000","last_became_peered":"2024-09-14T06:28:51.222874+0000","last_unstale":"2024-09-14T06:32:18.692057+0000","last_undegraded":"2024-09-14T06:32:18.692057+0000","last_fullsized":"2024-09-14T06:32:18.692057+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:28:50.205350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:28:50.205350+0000","last_clean_scrub_stamp":"2024-09-14T06:28:50.205350+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T12:04:25.190171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050808899999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'72","reported_seq":494,"reported_epoch":213,"state":"active+clean","last_fresh":"2024-09-14T06:32:18.773848+0000","last_change":"2024-09-14T06:28:44.839885+0000","last_active":"2024-09-14T06:32:18.773848+0000","last_peered":"2024-09-14T06:32:18.773848+0000","last_clean":"2024-09-14T06:32:18.773848+0000","last_became_active":"2024-09-14T06:28:44.839573+0000","last_became_peered":"2024-09-14T06:28:44.839573+0000","last_unstale":"2024-09-14T06:32:18.773848+0000","last_undegraded":"2024-09-14T06:32:18.773848+0000","last_fullsized":"2024-09-14T06:32:18.773848+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:28:43.828494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:28:43.828494+0000","last_clean_scrub_stamp":"2024-09-14T06:28:43.828494+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T14:41:25.787293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.8","version":"213'591","reported_seq":6158,"reported_epoch":215,"state":"active+clean","last_fresh":"2024-09-14T06:32:20.617831+0000","last_change":"2024-09-14T06:32:20.618029+0000","last_active":"2024-09-14T06:32:20.617831+0000","last_peered":"2024-09-14T06:32:20.617831+0000","last_clean":"2024-09-14T06:32:20.617831+0000","last_became_active":"2024-09-14T06:31:55.754694+0000","last_became_peered":"2024-09-14T06:31:55.754694+0000","last_unstale":"2024-09-14T06:32:20.617831+0000","last_undegraded":"2024-09-14T06:32:20.617831+0000","last_fullsized":"2024-09-14T06:32:20.617831+0000","mapping_epoch":190,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":191,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":591,"log_dups_size":0,"ondisk_log_size":591,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T12:28:27.689422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011106860000000001,"stat_sum":{"num_bytes":77868079,"num_objects":66,"num_object_clones":0,"num_object_copies":132,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":66,"num_whiteouts":0,"num_read":5608,"num_read_kb":90696,"num_write":839,"num_write_kb":57685,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":319,"num_bytes_recovered":60352269,"num_keys_recovered":22,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"4d","length":"1"},{"start":"4f","length":"1"},{"start":"51","length":"1"},{"start":"53","length":"1"},{"start":"55","length":"1"}]},{"pgid":"3.9","version":"213'519","reported_seq":4086,"reported_epoch":213,"state":"active+clean","last_fresh":"2024-09-14T06:32:18.774037+0000","last_change":"2024-09-14T06:32:18.687776+0000","last_active":"2024-09-14T06:32:18.774037+0000","last_peered":"2024-09-14T06:32:18.774037+0000","last_clean":"2024-09-14T06:32:18.774037+0000","last_became_active":"2024-09-14T06:29:00.573179+0000","last_became_peered":"2024-09-14T06:29:00.573179+0000","last_unstale":"2024-09-14T06:32:18.774037+0000","last_undegraded":"2024-09-14T06:32:18.774037+0000","last_fullsized":"2024-09-14T06:32:18.774037+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":519,"log_dups_size":0,"ondisk_log_size":519,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T17:49:01.809466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00083231600000000005,"stat_sum":{"num_bytes":42020611,"num_objects":63,"num_object_clones":0,"num_object_copies":126,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":63,"num_whiteouts":0,"num_read":3634,"num_read_kb":64262,"num_write":702,"num_write_kb":44923,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"51","length":"1"},{"start":"53","length":"1"},{"start":"55","length":"1"},{"start":"57","length":"1"},{"start":"59","length":"1"},{"start":"5b","length":"1"}]},{"pgid":"3.a","version":"213'744","reported_seq":4442,"reported_epoch":213,"state":"active+clean","last_fresh":"2024-09-14T06:32:18.795243+0000","last_change":"2024-09-14T06:32:18.687544+0000","last_active":"2024-09-14T06:32:18.795243+0000","last_peered":"2024-09-14T06:32:18.795243+0000","last_clean":"2024-09-14T06:32:18.795243+0000","last_became_active":"2024-09-14T06:32:04.617283+0000","last_became_peered":"2024-09-14T06:32:04.617283+0000","last_unstale":"2024-09-14T06:32:18.795243+0000","last_undegraded":"2024-09-14T06:32:18.795243+0000","last_fullsized":"2024-09-14T06:32:18.795243+0000","mapping_epoch":199,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":200,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":744,"log_dups_size":0,"ondisk_log_size":744,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T13:17:33.073025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064405899999999999,"stat_sum":{"num_bytes":45746635,"num_objects":61,"num_object_clones":0,"num_object_copies":122,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":61,"num_whiteouts":0,"num_read":4807,"num_read_kb":57874,"num_write":1112,"num_write_kb":47517,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":161,"num_bytes_recovered":21723371,"num_keys_recovered":33,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"51","length":"1"},{"start":"53","length":"1"},{"start":"55","length":"1"},{"start":"57","length":"1"},{"start":"59","length":"1"},{"start":"5b","length":"1"}]},{"pgid":"3.b","version":"215'1482","reported_seq":8278,"reported_epoch":215,"state":"active+clean","last_fresh":"2024-09-14T06:32:20.617756+0000","last_change":"2024-09-14T06:32:20.617756+0000","last_active":"2024-09-14T06:32:20.617756+0000","last_peered":"2024-09-14T06:32:20.617756+0000","last_clean":"2024-09-14T06:32:20.617756+0000","last_became_active":"2024-09-14T06:31:55.756219+0000","last_became_peered":"2024-09-14T06:31:55.756219+0000","last_unstale":"2024-09-14T06:32:20.617756+0000","last_undegraded":"2024-09-14T06:32:20.617756+0000","last_fullsized":"2024-09-14T06:32:20.617756+0000","mapping_epoch":190,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":191,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":1482,"log_dups_size":0,"ondisk_log_size":1482,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T12:53:28.647405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070281100000000002,"stat_sum":{"num_bytes":51970960,"num_objects":68,"num_object_clones":0,"num_object_copies":136,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":68,"num_whiteouts":0,"num_read":12454,"num_read_kb":63624,"num_write":2354,"num_write_kb":43651,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":291,"num_bytes_recovered":37370632,"num_keys_recovered":66,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"4d","length":"1"},{"start":"4f","length":"1"},{"start":"51","length":"1"},{"start":"53","length":"1"},{"start":"55","length":"1"}]},{"pgid":"3.c","version":"213'555","reported_seq":5918,"reported_epoch":213,"state":"active+clean","last_fresh":"2024-09-14T06:32:18.805179+0000","last_change":"2024-09-14T06:32:18.693249+0000","last_active":"2024-09-14T06:32:18.805179+0000","last_peered":"2024-09-14T06:32:18.805179+0000","last_clean":"2024-09-14T06:32:18.805179+0000","last_became_active":"2024-09-14T06:31:53.747355+0000","last_became_peered":"2024-09-14T06:31:53.747355+0000","last_unstale":"2024-09-14T06:32:18.805179+0000","last_undegraded":"2024-09-14T06:32:18.805179+0000","last_fullsized":"2024-09-14T06:32:18.805179+0000","mapping_epoch":188,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":189,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":555,"log_dups_size":0,"ondisk_log_size":555,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T08:00:16.188042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00080405799999999996,"stat_sum":{"num_bytes":50451282,"num_objects":52,"num_object_clones":0,"num_object_copies":104,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":52,"num_whiteouts":0,"num_read":4805,"num_read_kb":50398,"num_write":730,"num_write_kb":40822,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":293,"num_bytes_recovered":36712421,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"51","length":"1"},{"start":"53","length":"1"},{"start":"55","length":"1"},{"start":"57","length":"1"},{"start":"59","length":"1"},{"start":"5b","length":"1"}]},{"pgid":"3.d","version":"213'493","reported_seq":8825,"reported_epoch":213,"state":"active+clean","last_fresh":"2024-09-14T06:32:18.812802+0000","last_change":"2024-09-14T06:32:18.775680+0000","last_active":"2024-09-14T06:32:18.812802+0000","last_peered":"2024-09-14T06:32:18.812802+0000","last_clean":"2024-09-14T06:32:18.812802+0000","last_became_active":"2024-09-14T06:29:00.580362+0000","last_became_peered":"2024-09-14T06:29:00.580362+0000","last_unstale":"2024-09-14T06:32:18.812802+0000","last_undegraded":"2024-09-14T06:32:18.812802+0000","last_fullsized":"2024-09-14T06:32:18.812802+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":493,"log_dups_size":0,"ondisk_log_size":493,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T15:09:07.195460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012061820000000001,"stat_sum":{"num_bytes":94395867,"num_objects":61,"num_object_clones":0,"num_object_copies":122,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":61,"num_whiteouts":0,"num_read":8023,"num_read_kb":93542,"num_write":655,"num_write_kb":41026,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"51","length":"1"},{"start":"53","length":"1"},{"start":"55","length":"1"},{"start":"57","length":"1"},{"start":"59","length":"1"},{"start":"5b","length":"1"}]},{"pgid":"3.e","version":"213'524","reported_seq":4763,"reported_epoch":213,"state":"active+clean","last_fresh":"2024-09-14T06:32:18.785147+0000","last_change":"2024-09-14T06:32:18.775977+0000","last_active":"2024-09-14T06:32:18.785147+0000","last_peered":"2024-09-14T06:32:18.785147+0000","last_clean":"2024-09-14T06:32:18.785147+0000","last_became_active":"2024-09-14T06:29:00.586374+0000","last_became_peered":"2024-09-14T06:29:00.586374+0000","last_unstale":"2024-09-14T06:32:18.785147+0000","last_undegraded":"2024-09-14T06:32:18.785147+0000","last_fullsized":"2024-09-14T06:32:18.785147+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":524,"log_dups_size":0,"ondisk_log_size":524,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T07:02:56.188332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001389209,"stat_sum":{"num_bytes":47759500,"num_objects":59,"num_object_clones":0,"num_object_copies":118,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":59,"num_whiteouts":0,"num_read":3977,"num_read_kb":61134,"num_write":694,"num_write_kb":43787,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"51","length":"1"},{"start":"53","length":"1"},{"start":"55","length":"1"},{"start":"57","length":"1"},{"start":"59","length":"1"},{"start":"5b","length":"1"}]},{"pgid":"3.f","version":"213'624","reported_seq":5365,"reported_epoch":213,"state":"active+clean","last_fresh":"2024-09-14T06:32:18.800655+0000","last_change":"2024-09-14T06:32:18.774823+0000","last_active":"2024-09-14T06:32:18.800655+0000","last_peered":"2024-09-14T06:32:18.800655+0000","last_clean":"2024-09-14T06:32:18.800655+0000","last_became_active":"2024-09-14T06:29:00.586491+0000","last_became_peered":"2024-09-14T06:29:00.586491+0000","last_unstale":"2024-09-14T06:32:18.800655+0000","last_undegraded":"2024-09-14T06:32:18.800655+0000","last_fullsized":"2024-09-14T06:32:18.800655+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":624,"log_dups_size":0,"ondisk_log_size":624,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T10:34:32.641118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00056132899999999995,"stat_sum":{"num_bytes":29256498,"num_objects":44,"num_object_clones":0,"num_object_copies":88,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":44,"num_whiteouts":0,"num_read":5724,"num_read_kb":45272,"num_write":760,"num_write_kb":36122,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"51","length":"1"},{"start":"53","length":"1"},{"start":"55","length":"1"},{"start":"57","length":"1"},{"start":"59","length":"1"},{"start":"5b","length":"1"}]},{"pgid":"3.10","version":"211'826","reported_seq":6189,"reported_epoch":212,"state":"active+clean","last_fresh":"2024-09-14T06:32:17.688002+0000","last_change":"2024-09-14T06:32:16.673350+0000","last_active":"2024-09-14T06:32:17.688002+0000","last_peered":"2024-09-14T06:32:17.688002+0000","last_clean":"2024-09-14T06:32:17.688002+0000","last_became_active":"2024-09-14T06:32:04.621617+0000","last_became_peered":"2024-09-14T06:32:04.621617+0000","last_unstale":"2024-09-14T06:32:17.688002+0000","last_undegraded":"2024-09-14T06:32:17.688002+0000","last_fullsized":"2024-09-14T06:32:17.688002+0000","mapping_epoch":199,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":200,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":826,"log_dups_size":0,"ondisk_log_size":826,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T14:22:36.187885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001412967,"stat_sum":{"num_bytes":37504683,"num_objects":60,"num_object_clones":0,"num_object_copies":120,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":6618,"num_read_kb":34017,"num_write":974,"num_write_kb":36637,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":154,"num_bytes_recovered":9401812,"num_keys_recovered":45,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"53","length":"1"},{"start":"55","length":"1"},{"start":"57","length":"1"},{"start":"59","length":"1"},{"start":"5b","length":"1"}]},{"pgid":"3.11","version":"213'622","reported_seq":10919,"reported_epoch":213,"state":"active+clean","last_fresh":"2024-09-14T06:32:18.797455+0000","last_change":"2024-09-14T06:32:18.774471+0000","last_active":"2024-09-14T06:32:18.797455+0000","last_peered":"2024-09-14T06:32:18.797455+0000","last_clean":"2024-09-14T06:32:18.797455+0000","last_became_active":"2024-09-14T06:29:00.586480+0000","last_became_peered":"2024-09-14T06:29:00.586480+0000","last_unstale":"2024-09-14T06:32:18.797455+0000","last_undegraded":"2024-09-14T06:32:18.797455+0000","last_fullsized":"2024-09-14T06:32:18.797455+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":622,"log_dups_size":0,"ondisk_log_size":622,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T12:16:24.553171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000454676,"stat_sum":{"num_bytes":103838036,"num_objects":71,"num_object_clones":0,"num_object_copies":142,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":71,"num_whiteouts":0,"num_read":10178,"num_read_kb":109171,"num_write":749,"num_write_kb":48292,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"51","length":"1"},{"start":"53","length":"1"},{"start":"55","length":"1"},{"start":"57","length":"1"},{"start":"59","length":"1"},{"start":"5b","length":"1"}]},{"pgid":"3.12","version":"211'656","reported_seq":6945,"reported_epoch":212,"state":"active+clean","last_fresh":"2024-09-14T06:32:17.687957+0000","last_change":"2024-09-14T06:32:16.673655+0000","last_active":"2024-09-14T06:32:17.687957+0000","last_peered":"2024-09-14T06:32:17.687957+0000","last_clean":"2024-09-14T06:32:17.687957+0000","last_became_active":"2024-09-14T06:29:00.579826+0000","last_became_peered":"2024-09-14T06:29:00.579826+0000","last_unstale":"2024-09-14T06:32:17.687957+0000","last_undegraded":"2024-09-14T06:32:17.687957+0000","last_fullsized":"2024-09-14T06:32:17.687957+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":656,"log_dups_size":0,"ondisk_log_size":656,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T16:19:03.689356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010101960000000001,"stat_sum":{"num_bytes":41920978,"num_objects":70,"num_object_clones":0,"num_object_copies":140,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":70,"num_whiteouts":0,"num_read":6842,"num_read_kb":58865,"num_write":822,"num_write_kb":42420,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"53","length":"1"},{"start":"55","length":"1"},{"start":"57","length":"1"},{"start":"59","length":"1"},{"start":"5b","length":"1"}]},{"pgid":"3.13","version":"213'881","reported_seq":8268,"reported_epoch":213,"state":"active+clean","last_fresh":"2024-09-14T06:32:18.793155+0000","last_change":"2024-09-14T06:32:18.774526+0000","last_active":"2024-09-14T06:32:18.793155+0000","last_peered":"2024-09-14T06:32:18.793155+0000","last_clean":"2024-09-14T06:32:18.793155+0000","last_became_active":"2024-09-14T06:29:00.586520+0000","last_became_peered":"2024-09-14T06:29:00.586520+0000","last_unstale":"2024-09-14T06:32:18.793155+0000","last_undegraded":"2024-09-14T06:32:18.793155+0000","last_fullsized":"2024-09-14T06:32:18.793155+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":881,"log_dups_size":0,"ondisk_log_size":881,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T07:50:39.112354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00045281999999999999,"stat_sum":{"num_bytes":85171325,"num_objects":59,"num_object_clones":0,"num_object_copies":118,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":59,"num_whiteouts":0,"num_read":9027,"num_read_kb":65664,"num_write":1128,"num_write_kb":47419,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"51","length":"1"},{"start":"53","length":"1"},{"start":"55","length":"1"},{"start":"57","length":"1"},{"start":"59","length":"1"},{"start":"5b","length":"1"}]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":1944277612,"num_objects":1973,"num_object_clones":0,"num_object_copies":3946,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1973,"num_whiteouts":0,"num_read":231194,"num_read_kb":2177288,"num_write":34800,"num_write_kb":1477839,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4467,"num_bytes_recovered":588103625,"num_keys_recovered":466,"num_objects_omap":42,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":129601536,"data_stored":1303292470,"data_compressed":81030869,"data_compressed_allocated":106987520,"data_compressed_original":1285551886,"omap_allocated":0,"internal_metadata":0},"log_size":25068,"ondisk_log_size":25068,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":131072,"data_stored":1180736,"data_compressed":65166,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509486,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":75908,"kb_used_data":18348,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94295932,"statfs":{"total":96636764160,"available":96559034368,"internally_reserved":0,"allocated":18788352,"data_stored":158536586,"data_compressed":9576918,"data_compressed_allocated":12656640,"data_compressed_original":154434475,"omap_allocated":1590,"internal_metadata":58915274},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":6,"up_from":0,"seq":64424509485,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509487,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":83672,"kb_used_data":20548,"kb_used_omap":1,"kb_used_meta":63102,"kb_avail":94288168,"statfs":{"total":96636764160,"available":96551084032,"internally_reserved":0,"allocated":21041152,"data_stored":181912149,"data_compressed":11304189,"data_compressed_allocated":15073280,"data_compressed_original":178014566,"omap_allocated":1595,"internal_metadata":64616901},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509486,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":98252,"kb_used_data":26756,"kb_used_omap":1,"kb_used_meta":71422,"kb_avail":94273588,"statfs":{"total":96636764160,"available":96536154112,"internally_reserved":0,"allocated":27398144,"data_stored":252237055,"data_compressed":15741943,"data_compressed_allocated":20656128,"data_compressed_original":247770174,"omap_allocated":1595,"internal_metadata":73136581},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509487,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":78660,"kb_used_data":16936,"kb_used_omap":1,"kb_used_meta":61694,"kb_avail":94293180,"statfs":{"total":96636764160,"available":96556216320,"internally_reserved":0,"allocated":17342464,"data_stored":150833496,"data_compressed":9194714,"data_compressed_allocated":12169216,"data_compressed_original":147560055,"omap_allocated":1595,"internal_metadata":63175109},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":14,"seq":60129542190,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37712,"kb_used_data":10684,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94334128,"statfs":{"total":96636764160,"available":96598147072,"internally_reserved":0,"allocated":10940416,"data_stored":88882747,"data_compressed":5419630,"data_compressed_allocated":7172096,"data_compressed_original":86718577,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542189,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":95044,"kb_used_data":23596,"kb_used_omap":1,"kb_used_meta":71422,"kb_avail":94276796,"statfs":{"total":96636764160,"available":96539439104,"internally_reserved":0,"allocated":24162304,"data_stored":212986788,"data_compressed":13260458,"data_compressed_allocated":17498112,"data_compressed_original":208405851,"omap_allocated":1595,"internal_metadata":73136581},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574895,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":93544,"kb_used_data":22072,"kb_used_omap":1,"kb_used_meta":71422,"kb_avail":94278296,"statfs":{"total":96636764160,"available":96540975104,"internally_reserved":0,"allocated":22601728,"data_stored":204901753,"data_compressed":12877514,"data_compressed_allocated":16900096,"data_compressed_original":201166848,"omap_allocated":1593,"internal_metadata":73136583},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32583,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32583,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":21553152,"data_stored":211156003,"data_compressed":13227875,"data_compressed_allocated":17453056,"data_compressed_original":207832411,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":14577664,"data_stored":149308278,"data_compressed":9174476,"data_compressed_allocated":12111872,"data_compressed_original":147395871,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8376320,"data_stored":87510993,"data_compressed":5413418,"data_compressed_allocated":7155712,"data_compressed_original":86587505,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":19726336,"data_stored":202807534,"data_compressed":12784815,"data_compressed_allocated":16781312,"data_compressed_original":200466432,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":24580096,"data_stored":250585122,"data_compressed":15727977,"data_compressed_allocated":20590592,"data_compressed_original":247532606,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":18219008,"data_stored":180225292,"data_compressed":11258943,"data_compressed_allocated":14995456,"data_compressed_original":177727846,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":6631424,"data_stored":65369250,"data_compressed":3936333,"data_compressed_allocated":5361664,"data_compressed_original":64393940,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":15937536,"data_stored":156329998,"data_compressed":9507032,"data_compressed_allocated":12537856,"data_compressed_original":153615275,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-14T06:32:22.630 INFO:tasks.ceph.ceph_manager.ceph:recovered! 2024-09-14T06:32:22.630 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-14T06:32:22.630 INFO:tasks.thrashosds.thrasher:primary_affinity 2024-09-14T06:32:22.630 INFO:tasks.thrashosds.thrasher:Setting osd 5 primary_affinity to 0.000000 2024-09-14T06:32:22.631 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 5 0 2024-09-14T06:32:22.713 INFO:tasks.ceph.osd.0.smithi094.stderr:2024-09-14T06:32:22.712+0000 7f07c9333640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33636) UID: 0 2024-09-14T06:32:22.724 INFO:teuthology.orchestra.run.smithi094.stdout:ERROR: (22) Invalid argument 2024-09-14T06:32:22.724 INFO:teuthology.orchestra.run.smithi094.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:32:22.735 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-14T06:32:22.819 INFO:tasks.ceph.osd.1.smithi094.stderr:2024-09-14T06:32:22.818+0000 7f328e47a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33641) UID: 0 2024-09-14T06:32:22.890 INFO:teuthology.orchestra.run.smithi094.stdout:ERROR: (22) Invalid argument 2024-09-14T06:32:22.890 INFO:teuthology.orchestra.run.smithi094.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:32:22.899 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-14T06:32:22.940 INFO:tasks.ceph.osd.0.smithi094.stderr:2024-09-14T06:32:22.939+0000 7f07c9333640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33636) UID: 0 2024-09-14T06:32:23.014 INFO:tasks.ceph.osd.3.smithi094.stderr:2024-09-14T06:32:23.013+0000 7feeeafe8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33655) UID: 0 2024-09-14T06:32:23.042 INFO:teuthology.orchestra.run.smithi094.stdout:ERROR: (22) Invalid argument 2024-09-14T06:32:23.043 INFO:teuthology.orchestra.run.smithi094.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:32:23.051 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-14T06:32:23.115 INFO:tasks.ceph.osd.7.smithi138.stderr:2024-09-14T06:32:23.114+0000 7f3c6a21d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33623) UID: 0 2024-09-14T06:32:23.194 INFO:teuthology.orchestra.run.smithi094.stdout:ERROR: (22) Invalid argument 2024-09-14T06:32:23.195 INFO:teuthology.orchestra.run.smithi094.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:32:23.203 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-14T06:32:23.247 INFO:tasks.ceph.osd.2.smithi094.stderr:2024-09-14T06:32:23.246+0000 7faf3269b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33649) UID: 0 2024-09-14T06:32:23.305 INFO:teuthology.orchestra.run.smithi094.stdout:ERROR: (22) Invalid argument 2024-09-14T06:32:23.305 INFO:teuthology.orchestra.run.smithi094.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:32:23.314 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-14T06:32:23.355 INFO:tasks.ceph.osd.2.smithi094.stderr:2024-09-14T06:32:23.354+0000 7faf3269b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33649) UID: 0 2024-09-14T06:32:23.417 INFO:tasks.ceph.osd.2.smithi094.stderr:2024-09-14T06:32:23.416+0000 7faf3269b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33649) UID: 0 2024-09-14T06:32:23.456 INFO:teuthology.orchestra.run.smithi094.stdout:ERROR: (22) Invalid argument 2024-09-14T06:32:23.457 INFO:teuthology.orchestra.run.smithi094.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:32:23.465 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-14T06:32:23.549 INFO:tasks.ceph.osd.3.smithi094.stderr:2024-09-14T06:32:23.548+0000 7feeeafe8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33655) UID: 0 2024-09-14T06:32:23.608 INFO:teuthology.orchestra.run.smithi094.stdout:ERROR: (22) Invalid argument 2024-09-14T06:32:23.608 INFO:teuthology.orchestra.run.smithi094.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:32:23.616 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-14T06:32:23.619 INFO:tasks.ceph.osd.5.smithi138.stderr:2024-09-14T06:32:23.618+0000 7fe13746f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33618) UID: 0 2024-09-14T06:32:23.712 INFO:teuthology.orchestra.run.smithi094.stderr:set osd.5 primary-affinity to 0 (802) 2024-09-14T06:32:23.719 INFO:tasks.ceph.osd.4.smithi138.stderr:2024-09-14T06:32:23.718+0000 7f6c653c8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33615) UID: 0 2024-09-14T06:32:23.737 INFO:teuthology.orchestra.run.smithi138.stdout:ERROR: (22) Invalid argument 2024-09-14T06:32:23.737 INFO:teuthology.orchestra.run.smithi138.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:32:23.746 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-14T06:32:23.786 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-14T06:32:23.831 INFO:tasks.ceph.osd.7.smithi138.stderr:2024-09-14T06:32:23.830+0000 7f3c6a21d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33623) UID: 0 2024-09-14T06:32:23.892 INFO:teuthology.orchestra.run.smithi138.stdout:ERROR: (22) Invalid argument 2024-09-14T06:32:23.892 INFO:teuthology.orchestra.run.smithi138.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:32:23.901 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-14T06:32:23.920 INFO:tasks.ceph.osd.1.smithi094.stderr:2024-09-14T06:32:23.919+0000 7f328e47a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33641) UID: 0 2024-09-14T06:32:24.022 INFO:tasks.ceph.osd.3.smithi094.stderr:2024-09-14T06:32:24.020+0000 7feeeafe8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33655) UID: 0 2024-09-14T06:32:24.044 INFO:teuthology.orchestra.run.smithi138.stdout:ERROR: (22) Invalid argument 2024-09-14T06:32:24.045 INFO:teuthology.orchestra.run.smithi138.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:32:24.053 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-14T06:32:24.123 INFO:tasks.ceph.osd.3.smithi094.stderr:2024-09-14T06:32:24.121+0000 7feeeafe8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33655) UID: 0 2024-09-14T06:32:24.155 INFO:teuthology.orchestra.run.smithi138.stdout:ERROR: (22) Invalid argument 2024-09-14T06:32:24.155 INFO:teuthology.orchestra.run.smithi138.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:32:24.163 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-14T06:32:24.223 INFO:tasks.ceph.osd.4.smithi138.stderr:2024-09-14T06:32:24.222+0000 7f6c653c8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33615) UID: 0 2024-09-14T06:32:24.265 INFO:teuthology.orchestra.run.smithi138.stdout:ERROR: (22) Invalid argument 2024-09-14T06:32:24.266 INFO:teuthology.orchestra.run.smithi138.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:32:24.274 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-14T06:32:24.358 INFO:tasks.ceph.osd.6.smithi138.stderr:2024-09-14T06:32:24.357+0000 7f8cd8849640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33620) UID: 0 2024-09-14T06:32:24.383 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-14T06:32:24.417 INFO:teuthology.orchestra.run.smithi138.stdout:ERROR: (22) Invalid argument 2024-09-14T06:32:24.417 INFO:teuthology.orchestra.run.smithi138.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:32:24.424 INFO:tasks.ceph.osd.7.smithi138.stderr:2024-09-14T06:32:24.423+0000 7f3c6a21d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33623) UID: 0 2024-09-14T06:32:24.425 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-14T06:32:24.504 INFO:teuthology.orchestra.run.smithi094.stdout:osd.0: {} 2024-09-14T06:32:24.504 INFO:teuthology.orchestra.run.smithi094.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-14T06:32:24.521 INFO:teuthology.orchestra.run.smithi094.stdout:osd.1: {} 2024-09-14T06:32:24.521 INFO:teuthology.orchestra.run.smithi094.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-14T06:32:24.525 INFO:tasks.ceph.osd.6.smithi138.stderr:2024-09-14T06:32:24.524+0000 7f8cd8849640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33620) UID: 0 2024-09-14T06:32:24.536 INFO:teuthology.orchestra.run.smithi094.stdout:osd.2: {} 2024-09-14T06:32:24.537 INFO:teuthology.orchestra.run.smithi094.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-14T06:32:24.552 INFO:teuthology.orchestra.run.smithi094.stdout:osd.3: {} 2024-09-14T06:32:24.553 INFO:teuthology.orchestra.run.smithi094.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-14T06:32:24.569 INFO:teuthology.orchestra.run.smithi094.stdout:osd.4: {} 2024-09-14T06:32:24.569 INFO:teuthology.orchestra.run.smithi094.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-14T06:32:24.572 INFO:teuthology.orchestra.run.smithi138.stdout:ERROR: (22) Invalid argument 2024-09-14T06:32:24.572 INFO:teuthology.orchestra.run.smithi138.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:32:24.581 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-14T06:32:24.585 INFO:teuthology.orchestra.run.smithi094.stdout:osd.5: {} 2024-09-14T06:32:24.586 INFO:teuthology.orchestra.run.smithi094.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-14T06:32:24.602 INFO:teuthology.orchestra.run.smithi094.stdout:osd.6: {} 2024-09-14T06:32:24.602 INFO:teuthology.orchestra.run.smithi094.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-14T06:32:24.618 INFO:teuthology.orchestra.run.smithi094.stdout:osd.7: {} 2024-09-14T06:32:24.619 INFO:teuthology.orchestra.run.smithi094.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-14T06:32:24.627 INFO:tasks.ceph.osd.2.smithi094.stderr:2024-09-14T06:32:24.626+0000 7faf3269b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33649) UID: 0 2024-09-14T06:32:24.714 INFO:teuthology.orchestra.run.smithi094.stderr:noscrub is set 2024-09-14T06:32:24.727 INFO:tasks.ceph.osd.3.smithi094.stderr:2024-09-14T06:32:24.726+0000 7feeeafe8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33655) UID: 0 2024-09-14T06:32:24.729 INFO:teuthology.orchestra.run.smithi138.stdout:{ 2024-09-14T06:32:24.729 INFO:teuthology.orchestra.run.smithi138.stdout: "ops": [], 2024-09-14T06:32:24.729 INFO:teuthology.orchestra.run.smithi138.stdout: "complaint_time": 30, 2024-09-14T06:32:24.729 INFO:teuthology.orchestra.run.smithi138.stdout: "num_blocked_ops": 0 2024-09-14T06:32:24.729 INFO:teuthology.orchestra.run.smithi138.stdout:} 2024-09-14T06:32:24.739 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-14T06:32:24.828 INFO:tasks.ceph.osd.0.smithi094.stderr:2024-09-14T06:32:24.827+0000 7f07c9333640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33636) UID: 0 2024-09-14T06:32:24.856 INFO:teuthology.orchestra.run.smithi138.stdout:{ 2024-09-14T06:32:24.856 INFO:teuthology.orchestra.run.smithi138.stdout: "size": 20, 2024-09-14T06:32:24.857 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 600, 2024-09-14T06:32:24.857 INFO:teuthology.orchestra.run.smithi138.stdout: "ops": [ 2024-09-14T06:32:24.857 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:24.857 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:98505 3.10 3.1bf56f90 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T06:32:24.857 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:30:07.045453+0000", 2024-09-14T06:32:24.857 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 137.80948686799999, 2024-09-14T06:32:24.857 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032144440000000003, 2024-09-14T06:32:24.857 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:32:24.857 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:32:24.857 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "reached pg", 2024-09-14T06:32:24.857 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:32:24.857 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:32:24.858 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:24.858 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 98505 2024-09-14T06:32:24.858 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:24.858 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:32:24.858 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:24.858 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:32:24.858 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045453+0000", 2024-09-14T06:32:24.858 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:24.858 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:24.858 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:24.858 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:32:24.858 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045453+0000", 2024-09-14T06:32:24.858 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:24.859 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:24.859 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:24.859 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:32:24.859 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045453+0000", 2024-09-14T06:32:24.859 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.36e-07 2024-09-14T06:32:24.859 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:24.859 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:24.859 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:32:24.859 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045454+0000", 2024-09-14T06:32:24.860 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.1299999999999998e-07 2024-09-14T06:32:24.860 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:24.860 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:24.860 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:32:24.860 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045457+0000", 2024-09-14T06:32:24.860 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.8559999999999998e-06 2024-09-14T06:32:24.860 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:24.860 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:24.861 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:32:24.861 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045460+0000", 2024-09-14T06:32:24.861 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.8750000000000002e-06 2024-09-14T06:32:24.861 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:24.861 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:24.861 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:32:24.861 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.077592+0000", 2024-09-14T06:32:24.861 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032131727999999998 2024-09-14T06:32:24.861 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:24.861 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:24.861 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:32:24.861 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.077597+0000", 2024-09-14T06:32:24.862 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.8319999999999996e-06 2024-09-14T06:32:24.862 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:24.862 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:32:24.862 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:24.862 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:24.862 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:24.862 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:98506 3.10 3.1bf56f90 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T06:32:24.862 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:30:07.045481+0000", 2024-09-14T06:32:24.862 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 137.809458613, 2024-09-14T06:32:24.862 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032142759999999999, 2024-09-14T06:32:24.862 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:32:24.862 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:32:24.862 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "reached pg", 2024-09-14T06:32:24.862 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:32:24.863 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:32:24.863 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:24.863 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 98506 2024-09-14T06:32:24.863 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:24.863 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:32:24.863 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:24.863 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:32:24.863 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045481+0000", 2024-09-14T06:32:24.863 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:24.863 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:24.863 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:24.863 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:32:24.863 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045481+0000", 2024-09-14T06:32:24.863 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:24.864 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:24.864 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:24.864 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:32:24.864 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045481+0000", 2024-09-14T06:32:24.864 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.8500000000000001e-07 2024-09-14T06:32:24.864 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:24.864 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:24.864 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:32:24.865 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045483+0000", 2024-09-14T06:32:24.865 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.8250000000000001e-06 2024-09-14T06:32:24.865 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:24.865 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:24.865 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:32:24.865 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045489+0000", 2024-09-14T06:32:24.865 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.5740000000000003e-06 2024-09-14T06:32:24.865 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:24.865 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:24.865 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:32:24.866 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045495+0000", 2024-09-14T06:32:24.866 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.7690000000000002e-06 2024-09-14T06:32:24.866 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:24.866 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:24.866 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:32:24.866 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.077619+0000", 2024-09-14T06:32:24.866 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032124621999999999 2024-09-14T06:32:24.866 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:24.866 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:24.866 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:32:24.866 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.077624+0000", 2024-09-14T06:32:24.866 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.3850000000000004e-06 2024-09-14T06:32:24.866 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:24.866 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:32:24.867 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:24.867 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:24.867 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:24.867 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:98507 3.10 3.1bf56f90 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T06:32:24.867 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:30:07.045502+0000", 2024-09-14T06:32:24.867 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 137.80943719199999, 2024-09-14T06:32:24.867 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032147219999999997, 2024-09-14T06:32:24.867 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:32:24.867 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:32:24.867 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "reached pg", 2024-09-14T06:32:24.867 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:32:24.867 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:32:24.867 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:24.867 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 98507 2024-09-14T06:32:24.868 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:24.868 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:32:24.868 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:24.868 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:32:24.868 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045502+0000", 2024-09-14T06:32:24.868 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:24.868 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:24.868 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:24.868 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:32:24.868 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045502+0000", 2024-09-14T06:32:24.868 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:24.868 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:24.868 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:24.868 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:32:24.869 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045503+0000", 2024-09-14T06:32:24.869 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.4700000000000002e-07 2024-09-14T06:32:24.869 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:24.869 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:24.869 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:32:24.869 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045509+0000", 2024-09-14T06:32:24.870 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.7329999999999997e-06 2024-09-14T06:32:24.870 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:24.870 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:24.870 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:32:24.870 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045515+0000", 2024-09-14T06:32:24.870 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.8030000000000002e-06 2024-09-14T06:32:24.870 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:24.870 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:24.870 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:32:24.870 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045522+0000", 2024-09-14T06:32:24.870 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.0369999999999997e-06 2024-09-14T06:32:24.870 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:24.871 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:24.871 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:32:24.871 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.077641+0000", 2024-09-14T06:32:24.871 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032119091000000002 2024-09-14T06:32:24.871 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:24.871 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:24.871 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:32:24.871 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.077650+0000", 2024-09-14T06:32:24.871 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.2090000000000005e-06 2024-09-14T06:32:24.871 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:24.871 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:32:24.871 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:24.871 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:24.871 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:24.872 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:98508 3.10 3.1bf56f90 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T06:32:24.872 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:30:07.045530+0000", 2024-09-14T06:32:24.872 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 137.80940957499999, 2024-09-14T06:32:24.872 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032198773999999999, 2024-09-14T06:32:24.872 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:32:24.872 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:32:24.872 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "reached pg", 2024-09-14T06:32:24.872 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:32:24.872 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:32:24.872 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:24.872 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 98508 2024-09-14T06:32:24.872 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:24.872 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:32:24.872 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:24.873 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:32:24.873 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045530+0000", 2024-09-14T06:32:24.873 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:24.873 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:24.873 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:24.873 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:32:24.873 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045530+0000", 2024-09-14T06:32:24.873 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:24.873 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:24.873 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:24.873 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:32:24.876 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045530+0000", 2024-09-14T06:32:24.877 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.3099999999999998e-07 2024-09-14T06:32:24.877 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:24.877 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:24.877 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:32:24.877 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045539+0000", 2024-09-14T06:32:24.877 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.4780000000000008e-06 2024-09-14T06:32:24.877 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:24.877 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:24.877 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:32:24.877 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045546+0000", 2024-09-14T06:32:24.877 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.3259999999999998e-06 2024-09-14T06:32:24.877 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:24.877 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:24.878 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:32:24.878 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045551+0000", 2024-09-14T06:32:24.878 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.9960000000000001e-06 2024-09-14T06:32:24.878 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:24.878 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:24.878 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:32:24.878 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.077716+0000", 2024-09-14T06:32:24.878 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032165081999999998 2024-09-14T06:32:24.878 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:24.878 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:24.878 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:32:24.878 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.077729+0000", 2024-09-14T06:32:24.878 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.2461e-05 2024-09-14T06:32:24.879 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:24.879 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:32:24.879 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:24.879 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:24.879 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:24.879 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:98509 3.10 3.1bf56f90 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T06:32:24.879 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:30:07.045558+0000", 2024-09-14T06:32:24.880 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 137.809381374, 2024-09-14T06:32:24.880 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032242198, 2024-09-14T06:32:24.880 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:32:24.880 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:32:24.880 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "reached pg", 2024-09-14T06:32:24.880 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:32:24.880 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:32:24.881 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:24.881 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 98509 2024-09-14T06:32:24.881 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:24.881 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:32:24.881 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:24.881 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:32:24.881 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045558+0000", 2024-09-14T06:32:24.882 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:24.882 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:24.882 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:24.882 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:32:24.882 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045558+0000", 2024-09-14T06:32:24.882 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:24.883 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:24.883 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:24.883 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:32:24.883 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045559+0000", 2024-09-14T06:32:24.883 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.27e-07 2024-09-14T06:32:24.883 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:24.883 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:24.884 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:32:24.884 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045560+0000", 2024-09-14T06:32:24.884 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.6959999999999999e-06 2024-09-14T06:32:24.884 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:24.884 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:24.884 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:32:24.884 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045567+0000", 2024-09-14T06:32:24.884 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.6900000000000003e-06 2024-09-14T06:32:24.884 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:24.884 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:24.884 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:32:24.884 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045571+0000", 2024-09-14T06:32:24.884 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.5750000000000002e-06 2024-09-14T06:32:24.884 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:24.885 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:24.885 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:32:24.885 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.077789+0000", 2024-09-14T06:32:24.885 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032217186000000002 2024-09-14T06:32:24.885 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:24.885 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:24.885 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:32:24.885 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.077800+0000", 2024-09-14T06:32:24.885 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.1623999999999999e-05 2024-09-14T06:32:24.885 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:24.885 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:32:24.885 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:24.885 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:24.886 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:24.886 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:98510 3.10 3.1bf56f90 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T06:32:24.886 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:30:07.045580+0000", 2024-09-14T06:32:24.886 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 137.80935955999999, 2024-09-14T06:32:24.886 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032299845000000001, 2024-09-14T06:32:24.886 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:32:24.886 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:32:24.886 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "reached pg", 2024-09-14T06:32:24.886 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:32:24.886 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:32:24.886 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:24.886 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 98510 2024-09-14T06:32:24.886 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:24.886 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:32:24.887 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:24.887 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:32:24.887 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045580+0000", 2024-09-14T06:32:24.887 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:24.887 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:24.887 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:24.887 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:32:24.887 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045580+0000", 2024-09-14T06:32:24.887 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:24.887 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:24.887 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:24.887 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:32:24.887 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045580+0000", 2024-09-14T06:32:24.888 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.0400000000000002e-07 2024-09-14T06:32:24.888 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:24.888 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:24.888 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:32:24.888 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045582+0000", 2024-09-14T06:32:24.888 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.7600000000000001e-06 2024-09-14T06:32:24.888 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:24.888 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:24.888 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:32:24.888 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045588+0000", 2024-09-14T06:32:24.888 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.5650000000000001e-06 2024-09-14T06:32:24.888 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:24.888 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:24.889 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:32:24.889 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045597+0000", 2024-09-14T06:32:24.889 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.465e-06 2024-09-14T06:32:24.889 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:24.889 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:24.889 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:32:24.889 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.077871+0000", 2024-09-14T06:32:24.889 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032273562999999998 2024-09-14T06:32:24.889 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:24.889 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:24.889 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:32:24.889 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.077880+0000", 2024-09-14T06:32:24.889 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.0880000000000001e-06 2024-09-14T06:32:24.890 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:24.890 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:32:24.890 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:24.890 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:24.890 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:24.890 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:98511 3.10 3.1bf56f90 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T06:32:24.890 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:30:07.045610+0000", 2024-09-14T06:32:24.890 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 137.80932971300001, 2024-09-14T06:32:24.890 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032338894, 2024-09-14T06:32:24.890 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:32:24.890 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:32:24.890 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "reached pg", 2024-09-14T06:32:24.890 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:32:24.891 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:32:24.891 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:24.891 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 98511 2024-09-14T06:32:24.891 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:24.891 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:32:24.891 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:24.891 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:32:24.891 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045610+0000", 2024-09-14T06:32:24.891 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:24.891 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:24.891 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:24.891 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:32:24.891 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045610+0000", 2024-09-14T06:32:24.891 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:24.892 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:24.892 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:24.892 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:32:24.892 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045610+0000", 2024-09-14T06:32:24.892 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.1600000000000002e-07 2024-09-14T06:32:24.892 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:24.892 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:24.892 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:32:24.892 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045612+0000", 2024-09-14T06:32:24.892 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.1150000000000001e-06 2024-09-14T06:32:24.892 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:24.892 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:24.892 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:32:24.893 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045618+0000", 2024-09-14T06:32:24.893 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.0499999999999997e-06 2024-09-14T06:32:24.893 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:24.893 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:24.893 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:32:24.893 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045623+0000", 2024-09-14T06:32:24.893 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.6770000000000003e-06 2024-09-14T06:32:24.893 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:24.893 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:24.893 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:32:24.893 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.077928+0000", 2024-09-14T06:32:24.893 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032305111999999997 2024-09-14T06:32:24.893 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:24.894 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:24.894 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:32:24.894 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.077949+0000", 2024-09-14T06:32:24.894 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.0523999999999999e-05 2024-09-14T06:32:24.894 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:24.894 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:32:24.894 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:24.894 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:24.894 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:24.894 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:98512 3.10 3.1bf56f90 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T06:32:24.894 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:30:07.045630+0000", 2024-09-14T06:32:24.894 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 137.809309708, 2024-09-14T06:32:24.894 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032346632, 2024-09-14T06:32:24.895 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:32:24.895 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:32:24.895 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "reached pg", 2024-09-14T06:32:24.895 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:32:24.895 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:32:24.895 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:24.895 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 98512 2024-09-14T06:32:24.895 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:24.895 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:32:24.895 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:24.895 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:32:24.895 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045630+0000", 2024-09-14T06:32:24.895 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:24.896 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:24.896 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:24.896 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:32:24.896 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045630+0000", 2024-09-14T06:32:24.896 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:24.896 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:24.896 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:24.896 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:32:24.896 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045630+0000", 2024-09-14T06:32:24.896 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.8200000000000001e-07 2024-09-14T06:32:24.896 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:24.896 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:24.896 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:32:24.897 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045632+0000", 2024-09-14T06:32:24.897 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.7430000000000001e-06 2024-09-14T06:32:24.897 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:24.897 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:24.897 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:32:24.897 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045637+0000", 2024-09-14T06:32:24.897 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.4550000000000003e-06 2024-09-14T06:32:24.897 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:24.897 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:24.897 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:32:24.897 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045644+0000", 2024-09-14T06:32:24.897 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.1330000000000001e-06 2024-09-14T06:32:24.897 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:24.897 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:24.898 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:32:24.898 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.077968+0000", 2024-09-14T06:32:24.898 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032323401000000002 2024-09-14T06:32:24.898 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:24.898 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:24.898 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:32:24.898 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.077976+0000", 2024-09-14T06:32:24.898 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.5180000000000004e-06 2024-09-14T06:32:24.898 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:24.898 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:32:24.898 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:24.899 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:24.899 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:24.899 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:98513 3.10 3.1bf56f90 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T06:32:24.899 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:30:07.045652+0000", 2024-09-14T06:32:24.899 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 137.809287144, 2024-09-14T06:32:24.899 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032353436999999999, 2024-09-14T06:32:24.899 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:32:24.899 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:32:24.899 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "reached pg", 2024-09-14T06:32:24.899 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:32:24.899 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:32:24.899 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:24.900 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 98513 2024-09-14T06:32:24.900 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:24.900 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:32:24.900 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:24.900 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:32:24.900 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045652+0000", 2024-09-14T06:32:24.900 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:24.900 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:24.900 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:24.900 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:32:24.900 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045652+0000", 2024-09-14T06:32:24.900 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:24.900 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:24.901 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:24.901 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:32:24.901 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045653+0000", 2024-09-14T06:32:24.901 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.3700000000000001e-07 2024-09-14T06:32:24.901 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:24.901 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:24.901 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:32:24.901 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045655+0000", 2024-09-14T06:32:24.901 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.854e-06 2024-09-14T06:32:24.901 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:24.901 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:24.902 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:32:24.902 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045661+0000", 2024-09-14T06:32:24.902 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.5509999999999996e-06 2024-09-14T06:32:24.902 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:24.902 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:24.902 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:32:24.902 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045668+0000", 2024-09-14T06:32:24.902 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.1980000000000004e-06 2024-09-14T06:32:24.902 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:24.902 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:24.903 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:32:24.903 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.077997+0000", 2024-09-14T06:32:24.903 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032328575999999998 2024-09-14T06:32:24.903 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:24.903 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:24.903 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:32:24.903 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.078006+0000", 2024-09-14T06:32:24.903 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.8210000000000008e-06 2024-09-14T06:32:24.903 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:24.903 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:32:24.903 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:24.903 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:24.903 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:24.903 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:98514 3.10 3.1bf56f90 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T06:32:24.904 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:30:07.045699+0000", 2024-09-14T06:32:24.904 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 137.80924080899999, 2024-09-14T06:32:24.904 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032330223999999998, 2024-09-14T06:32:24.904 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:32:24.904 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:32:24.904 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "reached pg", 2024-09-14T06:32:24.904 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:32:24.904 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:32:24.904 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:24.904 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 98514 2024-09-14T06:32:24.904 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:24.904 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:32:24.904 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:24.905 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:32:24.905 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045699+0000", 2024-09-14T06:32:24.905 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:24.905 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:24.905 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:24.905 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:32:24.905 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045699+0000", 2024-09-14T06:32:24.905 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:24.905 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:24.905 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:24.905 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:32:24.905 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045699+0000", 2024-09-14T06:32:24.905 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.3200000000000001e-07 2024-09-14T06:32:24.906 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:24.906 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:24.906 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:32:24.906 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045701+0000", 2024-09-14T06:32:24.906 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.762e-06 2024-09-14T06:32:24.906 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:24.906 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:24.906 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:32:24.906 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045705+0000", 2024-09-14T06:32:24.906 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.2359999999999998e-06 2024-09-14T06:32:24.906 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:24.906 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:24.906 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:32:24.906 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045708+0000", 2024-09-14T06:32:24.907 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.796e-06 2024-09-14T06:32:24.907 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:24.907 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:24.907 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:32:24.907 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.078023+0000", 2024-09-14T06:32:24.907 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032315213000000002 2024-09-14T06:32:24.907 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:24.907 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:24.907 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:32:24.907 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.078029+0000", 2024-09-14T06:32:24.907 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.8850000000000004e-06 2024-09-14T06:32:24.907 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:24.908 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:32:24.908 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:24.908 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:24.908 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:24.908 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:98515 3.10 3.1bf56f90 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T06:32:24.908 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:30:07.045712+0000", 2024-09-14T06:32:24.908 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 137.80922749800001, 2024-09-14T06:32:24.908 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032346871999999999, 2024-09-14T06:32:24.908 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:32:24.908 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:32:24.908 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "reached pg", 2024-09-14T06:32:24.908 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:32:24.909 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:32:24.909 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:24.909 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 98515 2024-09-14T06:32:24.909 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:24.909 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:32:24.909 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:24.909 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:32:24.909 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045712+0000", 2024-09-14T06:32:24.909 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:24.909 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:24.909 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:24.909 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:32:24.909 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045712+0000", 2024-09-14T06:32:24.909 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:24.909 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:24.910 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:24.910 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:32:24.910 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045712+0000", 2024-09-14T06:32:24.910 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.2100000000000001e-07 2024-09-14T06:32:24.910 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:24.910 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:24.910 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:32:24.910 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045713+0000", 2024-09-14T06:32:24.910 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.068e-06 2024-09-14T06:32:24.910 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:24.910 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:24.910 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:32:24.910 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045716+0000", 2024-09-14T06:32:24.911 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.971e-06 2024-09-14T06:32:24.911 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:24.911 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:24.911 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:32:24.911 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045719+0000", 2024-09-14T06:32:24.911 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.3879999999999998e-06 2024-09-14T06:32:24.911 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:24.911 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:24.912 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:32:24.912 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.078049+0000", 2024-09-14T06:32:24.912 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032330553999999997 2024-09-14T06:32:24.912 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:24.912 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:24.912 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:32:24.912 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.078059+0000", 2024-09-14T06:32:24.912 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.6700000000000006e-06 2024-09-14T06:32:24.912 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:24.912 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:32:24.912 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:24.912 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:24.912 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:24.913 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:98516 3.10 3.1bf56f90 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T06:32:24.913 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:30:07.045722+0000", 2024-09-14T06:32:24.913 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 137.80921774500001, 2024-09-14T06:32:24.913 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032374334999999997, 2024-09-14T06:32:24.913 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:32:24.913 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:32:24.913 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "reached pg", 2024-09-14T06:32:24.913 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:32:24.913 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:32:24.913 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:24.913 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 98516 2024-09-14T06:32:24.913 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:24.913 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:32:24.914 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:24.914 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:32:24.914 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045722+0000", 2024-09-14T06:32:24.914 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:24.914 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:24.914 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:24.914 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:32:24.914 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045722+0000", 2024-09-14T06:32:24.914 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:24.914 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:24.914 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:24.914 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:32:24.914 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045722+0000", 2024-09-14T06:32:24.914 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.8999999999999998e-07 2024-09-14T06:32:24.915 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:24.915 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:24.915 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:32:24.915 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045723+0000", 2024-09-14T06:32:24.915 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.0049999999999999e-06 2024-09-14T06:32:24.915 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:24.915 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:24.915 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:32:24.915 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045726+0000", 2024-09-14T06:32:24.915 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.9019999999999999e-06 2024-09-14T06:32:24.915 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:24.915 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:24.915 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:32:24.916 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045728+0000", 2024-09-14T06:32:24.916 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.3290000000000001e-06 2024-09-14T06:32:24.916 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:24.916 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:24.916 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:32:24.916 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.078090+0000", 2024-09-14T06:32:24.916 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032362123 2024-09-14T06:32:24.916 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:24.916 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:24.916 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:32:24.916 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.078096+0000", 2024-09-14T06:32:24.916 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.6860000000000003e-06 2024-09-14T06:32:24.916 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:24.917 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:32:24.917 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:24.917 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:24.917 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:24.917 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:98517 3.10 3.1bf56f90 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T06:32:24.917 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:30:07.045731+0000", 2024-09-14T06:32:24.917 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 137.80920801400001, 2024-09-14T06:32:24.917 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.03241227, 2024-09-14T06:32:24.917 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:32:24.917 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:32:24.917 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "reached pg", 2024-09-14T06:32:24.917 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:32:24.917 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:32:24.918 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:24.918 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 98517 2024-09-14T06:32:24.918 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:24.918 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:32:24.918 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:24.918 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:32:24.918 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045731+0000", 2024-09-14T06:32:24.918 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:24.918 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:24.918 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:24.918 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:32:24.918 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045731+0000", 2024-09-14T06:32:24.918 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:24.919 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:24.919 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:24.919 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:32:24.919 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045732+0000", 2024-09-14T06:32:24.919 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.0699999999999999e-07 2024-09-14T06:32:24.919 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:24.919 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:24.919 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:32:24.919 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045733+0000", 2024-09-14T06:32:24.919 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.9199999999999999e-07 2024-09-14T06:32:24.919 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:24.920 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:24.920 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:32:24.920 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045735+0000", 2024-09-14T06:32:24.920 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.8150000000000002e-06 2024-09-14T06:32:24.920 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:24.920 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:24.920 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:32:24.920 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045738+0000", 2024-09-14T06:32:24.920 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.3640000000000002e-06 2024-09-14T06:32:24.920 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:24.920 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:24.920 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:32:24.920 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.078126+0000", 2024-09-14T06:32:24.921 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032388206000000003 2024-09-14T06:32:24.921 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:24.921 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:24.921 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:32:24.921 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.078144+0000", 2024-09-14T06:32:24.921 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.7686e-05 2024-09-14T06:32:24.921 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:24.921 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:32:24.921 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:24.921 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:24.921 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:24.921 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:98518 3.10 3.1bf56f90 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T06:32:24.921 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:30:07.045741+0000", 2024-09-14T06:32:24.921 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 137.809198619, 2024-09-14T06:32:24.922 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032451793999999999, 2024-09-14T06:32:24.922 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:32:24.922 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:32:24.922 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "reached pg", 2024-09-14T06:32:24.922 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:32:24.922 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:32:24.922 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:24.922 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 98518 2024-09-14T06:32:24.922 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:24.922 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:32:24.922 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:24.922 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:32:24.922 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045741+0000", 2024-09-14T06:32:24.922 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:24.923 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:24.923 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:24.923 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:32:24.923 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045741+0000", 2024-09-14T06:32:24.923 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:24.923 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:24.923 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:24.923 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:32:24.923 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045741+0000", 2024-09-14T06:32:24.923 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.05e-07 2024-09-14T06:32:24.923 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:24.924 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:24.924 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:32:24.924 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045746+0000", 2024-09-14T06:32:24.924 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.2259999999999996e-06 2024-09-14T06:32:24.924 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:24.924 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:24.924 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:32:24.924 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045750+0000", 2024-09-14T06:32:24.924 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.2490000000000001e-06 2024-09-14T06:32:24.924 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:24.924 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:24.924 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:32:24.924 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045752+0000", 2024-09-14T06:32:24.925 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.3980000000000002e-06 2024-09-14T06:32:24.925 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:24.925 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:24.925 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:32:24.925 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.078184+0000", 2024-09-14T06:32:24.925 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032431593000000002 2024-09-14T06:32:24.925 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:24.925 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:24.925 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:32:24.925 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.078193+0000", 2024-09-14T06:32:24.925 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.1230000000000006e-06 2024-09-14T06:32:24.925 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:24.925 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:32:24.926 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:24.926 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:24.926 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:24.926 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:98519 3.10 3.1bf56f90 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T06:32:24.926 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:30:07.045755+0000", 2024-09-14T06:32:24.926 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 137.80918419400001, 2024-09-14T06:32:24.926 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032489431999999999, 2024-09-14T06:32:24.926 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:32:24.926 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:32:24.926 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "reached pg", 2024-09-14T06:32:24.926 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:32:24.926 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:32:24.926 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:24.927 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 98519 2024-09-14T06:32:24.927 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:24.927 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:32:24.927 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:24.927 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:32:24.927 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045755+0000", 2024-09-14T06:32:24.927 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:24.927 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:24.927 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:24.927 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:32:24.927 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045755+0000", 2024-09-14T06:32:24.927 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:24.928 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:24.928 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:24.928 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:32:24.928 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045756+0000", 2024-09-14T06:32:24.928 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.23e-07 2024-09-14T06:32:24.928 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:24.928 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:24.928 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:32:24.928 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045757+0000", 2024-09-14T06:32:24.928 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.1060000000000001e-06 2024-09-14T06:32:24.928 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:24.928 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:24.928 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:32:24.929 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045760+0000", 2024-09-14T06:32:24.929 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.988e-06 2024-09-14T06:32:24.929 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:24.929 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:24.929 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:32:24.929 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045765+0000", 2024-09-14T06:32:24.929 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.2689999999999999e-06 2024-09-14T06:32:24.929 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:24.929 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:24.929 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:32:24.929 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.078234+0000", 2024-09-14T06:32:24.930 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032468915000000001 2024-09-14T06:32:24.930 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:24.930 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:24.930 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:32:24.930 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.078245+0000", 2024-09-14T06:32:24.930 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.0930999999999999e-05 2024-09-14T06:32:24.930 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:24.930 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:32:24.930 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:24.930 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:24.931 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:24.931 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:98520 3.10 3.1bf56f90 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T06:32:24.931 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:30:07.045768+0000", 2024-09-14T06:32:24.931 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 137.809171316, 2024-09-14T06:32:24.931 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032531379999999999, 2024-09-14T06:32:24.931 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:32:24.931 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:32:24.931 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "reached pg", 2024-09-14T06:32:24.931 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:32:24.931 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:32:24.932 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:24.932 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 98520 2024-09-14T06:32:24.932 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:24.932 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:32:24.932 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:24.932 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:32:24.932 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045768+0000", 2024-09-14T06:32:24.932 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:24.932 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:24.932 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:24.933 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:32:24.933 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045768+0000", 2024-09-14T06:32:24.933 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:24.933 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:24.933 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:24.933 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:32:24.933 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045768+0000", 2024-09-14T06:32:24.933 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.22e-07 2024-09-14T06:32:24.933 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:24.933 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:24.933 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:32:24.933 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045773+0000", 2024-09-14T06:32:24.933 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.3470000000000001e-06 2024-09-14T06:32:24.934 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:24.934 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:24.934 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:32:24.934 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045776+0000", 2024-09-14T06:32:24.934 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.1319999999999998e-06 2024-09-14T06:32:24.934 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:24.934 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:24.934 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:32:24.934 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045779+0000", 2024-09-14T06:32:24.934 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.9749999999999999e-06 2024-09-14T06:32:24.934 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:24.934 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:24.934 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:32:24.935 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.078289+0000", 2024-09-14T06:32:24.935 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032509693999999999 2024-09-14T06:32:24.935 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:24.935 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:24.935 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:32:24.935 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.078300+0000", 2024-09-14T06:32:24.935 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.101e-05 2024-09-14T06:32:24.935 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:24.935 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:32:24.935 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:24.935 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:24.935 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:24.935 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:98521 3.10 3.1bf56f90 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T06:32:24.935 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:30:07.045782+0000", 2024-09-14T06:32:24.936 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 137.80915713600001, 2024-09-14T06:32:24.936 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032566109000000003, 2024-09-14T06:32:24.936 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:32:24.936 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:32:24.936 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "reached pg", 2024-09-14T06:32:24.936 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:32:24.936 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:32:24.936 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:24.936 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 98521 2024-09-14T06:32:24.936 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:24.936 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:32:24.936 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:24.937 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:32:24.937 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045782+0000", 2024-09-14T06:32:24.937 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:24.937 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:24.937 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:24.937 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:32:24.937 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045782+0000", 2024-09-14T06:32:24.937 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:24.937 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:24.937 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:24.937 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:32:24.937 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045783+0000", 2024-09-14T06:32:24.937 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.2499999999999999e-07 2024-09-14T06:32:24.938 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:24.938 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:24.938 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:32:24.938 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045784+0000", 2024-09-14T06:32:24.938 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.0109999999999999e-06 2024-09-14T06:32:24.938 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:24.938 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:24.938 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:32:24.938 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045786+0000", 2024-09-14T06:32:24.938 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.8289999999999999e-06 2024-09-14T06:32:24.938 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:24.938 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:24.938 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:32:24.938 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045793+0000", 2024-09-14T06:32:24.939 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.579e-06 2024-09-14T06:32:24.939 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:24.939 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:24.939 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:32:24.939 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.078338+0000", 2024-09-14T06:32:24.939 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032545389000000001 2024-09-14T06:32:24.939 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:24.939 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:24.939 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:32:24.939 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.078348+0000", 2024-09-14T06:32:24.939 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.0076e-05 2024-09-14T06:32:24.939 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:24.939 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:32:24.939 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:24.940 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:24.940 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:24.940 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:98522 3.10 3.1bf56f90 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T06:32:24.940 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:30:07.045796+0000", 2024-09-14T06:32:24.940 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 137.80914322800001, 2024-09-14T06:32:24.940 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032577846000000001, 2024-09-14T06:32:24.940 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:32:24.940 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:32:24.940 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "reached pg", 2024-09-14T06:32:24.940 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:32:24.940 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:32:24.940 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:24.940 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 98522 2024-09-14T06:32:24.940 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:24.941 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:32:24.941 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:24.941 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:32:24.941 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045796+0000", 2024-09-14T06:32:24.941 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:24.941 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:24.941 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:24.941 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:32:24.941 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045796+0000", 2024-09-14T06:32:24.941 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:24.941 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:24.941 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:24.941 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:32:24.942 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045796+0000", 2024-09-14T06:32:24.942 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.2100000000000001e-07 2024-09-14T06:32:24.942 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:24.942 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:24.942 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:32:24.942 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045797+0000", 2024-09-14T06:32:24.942 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.6899999999999996e-07 2024-09-14T06:32:24.942 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:24.942 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:24.942 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:32:24.942 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045800+0000", 2024-09-14T06:32:24.942 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.9979999999999999e-06 2024-09-14T06:32:24.942 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:24.943 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:24.943 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:32:24.943 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045803+0000", 2024-09-14T06:32:24.943 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.2850000000000001e-06 2024-09-14T06:32:24.943 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:24.943 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:24.943 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:32:24.943 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.078366+0000", 2024-09-14T06:32:24.943 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032562932000000003 2024-09-14T06:32:24.943 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:24.943 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:24.943 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:32:24.943 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.078374+0000", 2024-09-14T06:32:24.943 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.4409999999999993e-06 2024-09-14T06:32:24.944 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:24.944 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:32:24.944 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:24.944 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:24.944 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:24.944 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:98523 3.10 3.1bf56f90 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T06:32:24.944 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:30:07.045806+0000", 2024-09-14T06:32:24.944 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 137.80913368500001, 2024-09-14T06:32:24.944 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032611167000000003, 2024-09-14T06:32:24.944 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:32:24.944 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:32:24.944 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "reached pg", 2024-09-14T06:32:24.945 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:32:24.945 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:32:24.945 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:24.945 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 98523 2024-09-14T06:32:24.945 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:24.945 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:32:24.945 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:24.945 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:32:24.945 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045806+0000", 2024-09-14T06:32:24.945 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:24.946 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:24.946 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:24.946 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:32:24.946 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045806+0000", 2024-09-14T06:32:24.946 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:24.946 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:24.946 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:24.946 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:32:24.946 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045806+0000", 2024-09-14T06:32:24.946 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.2000000000000001e-07 2024-09-14T06:32:24.946 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:24.946 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:24.947 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:32:24.947 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045807+0000", 2024-09-14T06:32:24.947 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.4e-07 2024-09-14T06:32:24.947 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:24.947 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:24.947 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:32:24.947 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045810+0000", 2024-09-14T06:32:24.947 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.9229999999999998e-06 2024-09-14T06:32:24.947 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:24.947 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:24.947 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:32:24.947 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045812+0000", 2024-09-14T06:32:24.948 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.2220000000000001e-06 2024-09-14T06:32:24.948 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:24.948 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:24.948 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:32:24.948 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.078407+0000", 2024-09-14T06:32:24.948 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032594513999999998 2024-09-14T06:32:24.948 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:24.948 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:24.948 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:32:24.948 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.078417+0000", 2024-09-14T06:32:24.948 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.0348e-05 2024-09-14T06:32:24.948 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:24.948 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:32:24.948 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:24.949 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:24.949 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:24.949 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:98524 3.10 3.1bf56f90 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T06:32:24.949 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:30:07.045815+0000", 2024-09-14T06:32:24.949 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 137.809124317, 2024-09-14T06:32:24.949 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032643208999999999, 2024-09-14T06:32:24.949 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:32:24.949 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:32:24.949 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "reached pg", 2024-09-14T06:32:24.949 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:32:24.949 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:32:24.949 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:24.950 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 98524 2024-09-14T06:32:24.950 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:24.950 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:32:24.950 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:24.950 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:32:24.950 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045815+0000", 2024-09-14T06:32:24.950 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:24.950 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:24.950 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:24.950 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:32:24.950 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045815+0000", 2024-09-14T06:32:24.950 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:24.950 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:24.950 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:24.951 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:32:24.951 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045815+0000", 2024-09-14T06:32:24.951 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.11e-07 2024-09-14T06:32:24.951 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:24.951 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:24.951 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:32:24.951 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045816+0000", 2024-09-14T06:32:24.951 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.9099999999999991e-07 2024-09-14T06:32:24.951 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:24.951 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:24.952 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:32:24.952 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045819+0000", 2024-09-14T06:32:24.952 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.9399999999999998e-06 2024-09-14T06:32:24.952 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:24.952 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:24.952 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:32:24.952 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045822+0000", 2024-09-14T06:32:24.952 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.3470000000000001e-06 2024-09-14T06:32:24.952 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:24.952 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:24.952 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:32:24.952 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.078449+0000", 2024-09-14T06:32:24.952 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032627529000000002 2024-09-14T06:32:24.953 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:24.953 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:24.953 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:32:24.953 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.078458+0000", 2024-09-14T06:32:24.953 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.1910000000000006e-06 2024-09-14T06:32:24.953 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:24.953 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:32:24.953 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:24.953 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:24.953 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:32:24.953 INFO:teuthology.orchestra.run.smithi138.stdout:} 2024-09-14T06:32:24.955 INFO:tasks.ceph.osd.0.smithi094.stderr:2024-09-14T06:32:24.954+0000 7f07c9333640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33636) UID: 0 2024-09-14T06:32:24.956 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-14T06:32:25.056 INFO:tasks.ceph.osd.6.smithi138.stderr:2024-09-14T06:32:25.055+0000 7f8cd8849640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33620) UID: 0 2024-09-14T06:32:25.058 INFO:teuthology.orchestra.run.smithi138.stdout:{ 2024-09-14T06:32:25.058 INFO:teuthology.orchestra.run.smithi138.stdout: "ops": [], 2024-09-14T06:32:25.058 INFO:teuthology.orchestra.run.smithi138.stdout: "num_ops": 0 2024-09-14T06:32:25.058 INFO:teuthology.orchestra.run.smithi138.stdout:} 2024-09-14T06:32:25.067 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-14T06:32:25.157 INFO:tasks.ceph.osd.5.smithi138.stderr:2024-09-14T06:32:25.156+0000 7fe13746f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33618) UID: 0 2024-09-14T06:32:25.210 INFO:teuthology.orchestra.run.smithi138.stdout:{ 2024-09-14T06:32:25.210 INFO:teuthology.orchestra.run.smithi138.stdout: "ops": [], 2024-09-14T06:32:25.210 INFO:teuthology.orchestra.run.smithi138.stdout: "complaint_time": 30, 2024-09-14T06:32:25.210 INFO:teuthology.orchestra.run.smithi138.stdout: "num_blocked_ops": 0 2024-09-14T06:32:25.210 INFO:teuthology.orchestra.run.smithi138.stdout:} 2024-09-14T06:32:25.219 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-14T06:32:25.258 INFO:tasks.ceph.osd.1.smithi094.stderr:2024-09-14T06:32:25.258+0000 7f328e47a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33641) UID: 0 2024-09-14T06:32:25.359 INFO:tasks.ceph.osd.3.smithi094.stderr:2024-09-14T06:32:25.358+0000 7feeeafe8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33655) UID: 0 2024-09-14T06:32:25.365 INFO:teuthology.orchestra.run.smithi138.stdout:{ 2024-09-14T06:32:25.365 INFO:teuthology.orchestra.run.smithi138.stdout: "size": 20, 2024-09-14T06:32:25.365 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 600, 2024-09-14T06:32:25.365 INFO:teuthology.orchestra.run.smithi138.stdout: "ops": [ 2024-09-14T06:32:25.365 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:25.366 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:53244 3.f 3:f71f51a7:::rbd_data.112dca2f6ae1.0000000000000015:17 [read 1863680~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:32:25.366 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:44.369399+0000", 2024-09-14T06:32:25.366 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 160.99433146800001, 2024-09-14T06:32:25.366 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047291189999999997, 2024-09-14T06:32:25.366 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:32:25.366 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:32:25.366 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:32:25.366 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:32:25.366 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:32:25.366 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:25.366 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 53244 2024-09-14T06:32:25.366 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:25.366 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:32:25.367 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:25.367 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:32:25.367 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369399+0000", 2024-09-14T06:32:25.367 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:25.367 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:25.367 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:25.367 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:32:25.367 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369399+0000", 2024-09-14T06:32:25.367 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:25.367 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:25.367 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:25.367 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:32:25.367 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369400+0000", 2024-09-14T06:32:25.368 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.0699999999999999e-07 2024-09-14T06:32:25.368 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:25.368 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:25.368 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:32:25.368 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369401+0000", 2024-09-14T06:32:25.368 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.4300000000000001e-06 2024-09-14T06:32:25.368 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:25.368 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:25.368 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:32:25.368 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369404+0000", 2024-09-14T06:32:25.368 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.061e-06 2024-09-14T06:32:25.368 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:25.369 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:25.369 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:32:25.369 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369407+0000", 2024-09-14T06:32:25.369 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.6139999999999999e-06 2024-09-14T06:32:25.369 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:25.369 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:25.369 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:32:25.369 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.416517+0000", 2024-09-14T06:32:25.369 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047110338000000002 2024-09-14T06:32:25.369 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:25.369 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:25.369 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:32:25.369 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.416616+0000", 2024-09-14T06:32:25.369 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.8651000000000001e-05 2024-09-14T06:32:25.370 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:25.370 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:25.370 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:32:25.370 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.416691+0000", 2024-09-14T06:32:25.370 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.4888999999999994e-05 2024-09-14T06:32:25.370 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:25.370 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:32:25.370 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:25.370 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:25.370 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:25.370 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:53245 3.f 3:f71f51a7:::rbd_data.112dca2f6ae1.0000000000000015:17 [read 1888256~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:32:25.370 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:44.369410+0000", 2024-09-14T06:32:25.370 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 160.99432117999999, 2024-09-14T06:32:25.371 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047399275999999997, 2024-09-14T06:32:25.371 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:32:25.371 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:32:25.371 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:32:25.371 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:32:25.371 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:32:25.371 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:25.371 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 53245 2024-09-14T06:32:25.371 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:25.371 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:32:25.371 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:25.371 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:32:25.371 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369410+0000", 2024-09-14T06:32:25.371 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:25.372 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:25.372 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:25.372 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:32:25.372 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369410+0000", 2024-09-14T06:32:25.372 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:25.372 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:25.372 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:25.372 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:32:25.372 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369410+0000", 2024-09-14T06:32:25.372 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.1899999999999999e-07 2024-09-14T06:32:25.372 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:25.372 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:25.372 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:32:25.373 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369411+0000", 2024-09-14T06:32:25.373 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.0300000000000001e-06 2024-09-14T06:32:25.373 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:25.373 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:25.373 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:32:25.373 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369414+0000", 2024-09-14T06:32:25.373 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.1099999999999999e-06 2024-09-14T06:32:25.373 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:25.373 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:25.373 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:32:25.373 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369417+0000", 2024-09-14T06:32:25.373 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.6189999999999998e-06 2024-09-14T06:32:25.373 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:25.374 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:25.374 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:32:25.374 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.416718+0000", 2024-09-14T06:32:25.374 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047301322999999999 2024-09-14T06:32:25.374 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:25.374 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:25.374 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:32:25.374 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.416759+0000", 2024-09-14T06:32:25.374 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.1009e-05 2024-09-14T06:32:25.374 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:25.374 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:25.374 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:32:25.374 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.416809+0000", 2024-09-14T06:32:25.374 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.9966000000000002e-05 2024-09-14T06:32:25.375 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:25.375 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:32:25.375 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:25.375 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:25.375 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:25.375 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:53246 3.f 3:f71f51a7:::rbd_data.112dca2f6ae1.0000000000000015:17 [read 1912832~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:32:25.375 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:44.369420+0000", 2024-09-14T06:32:25.375 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 160.99431123599999, 2024-09-14T06:32:25.375 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047529463000000001, 2024-09-14T06:32:25.375 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:32:25.375 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:32:25.375 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:32:25.375 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:32:25.376 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:32:25.376 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:25.376 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 53246 2024-09-14T06:32:25.376 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:25.376 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:32:25.376 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:25.376 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:32:25.376 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369420+0000", 2024-09-14T06:32:25.376 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:25.376 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:25.376 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:25.376 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:32:25.376 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369420+0000", 2024-09-14T06:32:25.376 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:25.377 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:25.377 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:25.377 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:32:25.377 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369420+0000", 2024-09-14T06:32:25.377 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.1299999999999999e-07 2024-09-14T06:32:25.377 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:25.377 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:25.377 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:32:25.377 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369425+0000", 2024-09-14T06:32:25.377 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.0130000000000001e-06 2024-09-14T06:32:25.377 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:25.377 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:25.378 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:32:25.378 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369428+0000", 2024-09-14T06:32:25.378 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.349e-06 2024-09-14T06:32:25.378 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:25.378 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:25.378 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:32:25.378 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369431+0000", 2024-09-14T06:32:25.378 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.7389999999999999e-06 2024-09-14T06:32:25.378 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:25.378 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:25.378 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:32:25.378 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.416834+0000", 2024-09-14T06:32:25.378 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047402669000000001 2024-09-14T06:32:25.379 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:25.379 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:25.379 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:32:25.379 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.416882+0000", 2024-09-14T06:32:25.379 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.8068999999999997e-05 2024-09-14T06:32:25.379 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:25.379 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:25.379 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:32:25.379 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.416949+0000", 2024-09-14T06:32:25.379 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.7410999999999995e-05 2024-09-14T06:32:25.379 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:25.379 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:32:25.379 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:25.379 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:25.379 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:25.380 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:53247 3.f 3:f71f51a7:::rbd_data.112dca2f6ae1.0000000000000015:17 [read 1937408~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:32:25.380 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:44.369434+0000", 2024-09-14T06:32:25.380 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 160.994296472, 2024-09-14T06:32:25.380 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047652264999999999, 2024-09-14T06:32:25.380 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:32:25.380 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:32:25.380 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:32:25.380 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:32:25.380 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:32:25.380 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:25.380 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 53247 2024-09-14T06:32:25.380 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:25.381 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:32:25.381 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:25.381 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:32:25.381 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369434+0000", 2024-09-14T06:32:25.381 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:25.381 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:25.381 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:25.381 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:32:25.381 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369434+0000", 2024-09-14T06:32:25.381 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:25.381 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:25.381 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:25.381 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:32:25.382 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369435+0000", 2024-09-14T06:32:25.382 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.2100000000000001e-07 2024-09-14T06:32:25.382 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:25.382 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:25.382 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:32:25.382 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369436+0000", 2024-09-14T06:32:25.382 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.122e-06 2024-09-14T06:32:25.382 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:25.382 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:25.382 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:32:25.382 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369439+0000", 2024-09-14T06:32:25.382 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.0400000000000001e-06 2024-09-14T06:32:25.382 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:25.382 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:25.383 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:32:25.383 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369441+0000", 2024-09-14T06:32:25.383 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.728e-06 2024-09-14T06:32:25.383 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:25.383 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:25.383 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:32:25.383 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.416977+0000", 2024-09-14T06:32:25.383 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047535344 2024-09-14T06:32:25.383 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:25.383 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:25.383 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:32:25.383 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.417029+0000", 2024-09-14T06:32:25.383 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.2037999999999999e-05 2024-09-14T06:32:25.383 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:25.384 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:25.384 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:32:25.384 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.417087+0000", 2024-09-14T06:32:25.384 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.7772000000000002e-05 2024-09-14T06:32:25.384 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:25.384 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:32:25.384 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:25.384 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:25.384 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:25.384 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:53248 3.f 3:f71f51a7:::rbd_data.112dca2f6ae1.0000000000000015:17 [read 1961984~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:32:25.384 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:44.369445+0000", 2024-09-14T06:32:25.384 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 160.99428592000001, 2024-09-14T06:32:25.385 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047798489, 2024-09-14T06:32:25.385 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:32:25.385 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:32:25.385 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:32:25.385 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:32:25.385 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:32:25.385 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:25.385 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 53248 2024-09-14T06:32:25.385 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:25.385 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:32:25.385 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:25.386 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:32:25.386 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369445+0000", 2024-09-14T06:32:25.386 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:25.386 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:25.386 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:25.386 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:32:25.386 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369445+0000", 2024-09-14T06:32:25.386 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:25.386 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:25.386 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:25.386 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:32:25.386 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369445+0000", 2024-09-14T06:32:25.386 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.0699999999999999e-07 2024-09-14T06:32:25.386 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:25.386 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:25.387 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:32:25.387 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369446+0000", 2024-09-14T06:32:25.387 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.1000000000000001e-06 2024-09-14T06:32:25.387 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:25.387 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:25.387 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:32:25.387 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369449+0000", 2024-09-14T06:32:25.387 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.1039999999999998e-06 2024-09-14T06:32:25.387 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:25.387 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:25.387 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:32:25.387 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369452+0000", 2024-09-14T06:32:25.387 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.6199999999999999e-06 2024-09-14T06:32:25.387 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:25.387 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:25.388 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:32:25.388 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.417114+0000", 2024-09-14T06:32:25.388 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047661901999999999 2024-09-14T06:32:25.388 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:25.388 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:25.388 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:32:25.388 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.417181+0000", 2024-09-14T06:32:25.388 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.7385999999999995e-05 2024-09-14T06:32:25.388 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:25.388 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:25.388 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:32:25.388 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.417243+0000", 2024-09-14T06:32:25.388 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.2169999999999996e-05 2024-09-14T06:32:25.389 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:25.389 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:32:25.389 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:25.389 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:25.389 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:25.389 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:53249 3.f 3:f71f51a7:::rbd_data.112dca2f6ae1.0000000000000015:17 [read 1986560~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:32:25.389 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:44.369455+0000", 2024-09-14T06:32:25.389 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 160.994275609, 2024-09-14T06:32:25.389 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047890493999999999, 2024-09-14T06:32:25.389 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:32:25.389 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:32:25.389 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:32:25.389 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:32:25.389 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:32:25.390 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:25.390 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 53249 2024-09-14T06:32:25.390 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:25.390 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:32:25.390 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:25.390 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:32:25.390 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369455+0000", 2024-09-14T06:32:25.390 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:25.390 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:25.390 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:25.390 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:32:25.390 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369455+0000", 2024-09-14T06:32:25.390 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:25.390 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:25.391 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:25.391 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:32:25.391 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369455+0000", 2024-09-14T06:32:25.391 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.0800000000000001e-07 2024-09-14T06:32:25.391 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:25.391 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:25.391 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:32:25.391 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369456+0000", 2024-09-14T06:32:25.391 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.0729999999999999e-06 2024-09-14T06:32:25.391 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:25.391 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:25.391 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:32:25.391 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369460+0000", 2024-09-14T06:32:25.391 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.027e-06 2024-09-14T06:32:25.392 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:25.392 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:25.392 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:32:25.392 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369462+0000", 2024-09-14T06:32:25.392 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.5320000000000001e-06 2024-09-14T06:32:25.392 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:25.392 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:25.392 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:32:25.392 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.417257+0000", 2024-09-14T06:32:25.392 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.04779489 2024-09-14T06:32:25.392 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:25.392 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:25.392 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:32:25.393 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.417296+0000", 2024-09-14T06:32:25.393 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.8658000000000002e-05 2024-09-14T06:32:25.393 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:25.393 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:25.393 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:32:25.393 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.417346+0000", 2024-09-14T06:32:25.393 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.0105999999999998e-05 2024-09-14T06:32:25.393 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:25.393 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:32:25.393 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:25.393 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:25.393 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:25.393 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:53250 3.f 3:f71f51a7:::rbd_data.112dca2f6ae1.0000000000000015:17 [read 2011136~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:32:25.393 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:44.369465+0000", 2024-09-14T06:32:25.394 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 160.99426563899999, 2024-09-14T06:32:25.394 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.048032230000000002, 2024-09-14T06:32:25.394 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:32:25.394 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:32:25.394 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:32:25.394 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:32:25.394 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:32:25.394 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:25.394 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 53250 2024-09-14T06:32:25.394 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:25.394 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:32:25.395 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:25.395 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:32:25.395 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369465+0000", 2024-09-14T06:32:25.395 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:25.395 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:25.395 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:25.395 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:32:25.395 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369465+0000", 2024-09-14T06:32:25.395 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:25.395 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:25.395 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:25.395 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:32:25.395 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369465+0000", 2024-09-14T06:32:25.395 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.2100000000000001e-07 2024-09-14T06:32:25.396 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:25.396 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:25.396 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:32:25.396 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369466+0000", 2024-09-14T06:32:25.396 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.0640000000000001e-06 2024-09-14T06:32:25.396 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:25.396 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:25.396 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:32:25.396 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369469+0000", 2024-09-14T06:32:25.396 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.0170000000000001e-06 2024-09-14T06:32:25.396 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:25.396 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:25.396 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:32:25.397 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369472+0000", 2024-09-14T06:32:25.397 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.6189999999999998e-06 2024-09-14T06:32:25.397 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:25.397 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:25.397 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:32:25.397 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.417383+0000", 2024-09-14T06:32:25.397 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047911092000000002 2024-09-14T06:32:25.397 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:25.397 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:25.397 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:32:25.397 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.417430+0000", 2024-09-14T06:32:25.397 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.6898999999999999e-05 2024-09-14T06:32:25.397 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:25.398 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:25.398 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:32:25.398 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.417497+0000", 2024-09-14T06:32:25.398 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.7317999999999995e-05 2024-09-14T06:32:25.398 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:25.398 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:32:25.398 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:25.398 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:25.398 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:25.398 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:53251 3.f 3:f71f51a7:::rbd_data.112dca2f6ae1.0000000000000015:17 [read 2035712~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:32:25.398 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:44.369475+0000", 2024-09-14T06:32:25.398 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 160.994255643, 2024-09-14T06:32:25.398 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.048154542000000002, 2024-09-14T06:32:25.399 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:32:25.399 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:32:25.399 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:32:25.399 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:32:25.399 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:32:25.399 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:25.399 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 53251 2024-09-14T06:32:25.399 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:25.399 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:32:25.399 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:25.399 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:32:25.399 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369475+0000", 2024-09-14T06:32:25.399 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:25.399 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:25.400 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:25.400 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:32:25.400 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369475+0000", 2024-09-14T06:32:25.400 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:25.400 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:25.400 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:25.400 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:32:25.400 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369475+0000", 2024-09-14T06:32:25.400 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.0100000000000001e-07 2024-09-14T06:32:25.400 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:25.401 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:25.401 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:32:25.401 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369476+0000", 2024-09-14T06:32:25.401 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.0580000000000001e-06 2024-09-14T06:32:25.401 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:25.401 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:25.401 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:32:25.401 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369479+0000", 2024-09-14T06:32:25.401 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.9950000000000001e-06 2024-09-14T06:32:25.401 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:25.401 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:25.401 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:32:25.401 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369482+0000", 2024-09-14T06:32:25.402 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.6340000000000001e-06 2024-09-14T06:32:25.402 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:25.402 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:25.402 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:32:25.402 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.417530+0000", 2024-09-14T06:32:25.402 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.048047535000000002 2024-09-14T06:32:25.402 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:25.402 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:25.402 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:32:25.402 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.417574+0000", 2024-09-14T06:32:25.402 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.4255999999999998e-05 2024-09-14T06:32:25.402 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:25.403 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:25.403 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:32:25.403 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.417630+0000", 2024-09-14T06:32:25.403 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.5862999999999997e-05 2024-09-14T06:32:25.403 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:25.403 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:32:25.403 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:25.403 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:25.403 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:25.403 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:53252 3.f 3:f71f51a7:::rbd_data.112dca2f6ae1.0000000000000015:17 [read 2060288~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:32:25.403 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:44.369485+0000", 2024-09-14T06:32:25.404 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 160.99424579500001, 2024-09-14T06:32:25.404 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.048254116, 2024-09-14T06:32:25.404 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:32:25.404 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:32:25.404 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:32:25.404 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:32:25.404 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:32:25.404 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:25.404 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 53252 2024-09-14T06:32:25.405 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:25.405 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:32:25.405 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:25.405 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:32:25.405 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369485+0000", 2024-09-14T06:32:25.405 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:25.405 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:25.405 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:25.405 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:32:25.405 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369485+0000", 2024-09-14T06:32:25.405 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:25.405 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:25.405 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:25.405 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:32:25.406 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369485+0000", 2024-09-14T06:32:25.406 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.1e-07 2024-09-14T06:32:25.406 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:25.406 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:25.406 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:32:25.406 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369487+0000", 2024-09-14T06:32:25.406 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.42e-06 2024-09-14T06:32:25.406 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:25.406 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:25.406 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:32:25.406 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369490+0000", 2024-09-14T06:32:25.406 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.9639999999999999e-06 2024-09-14T06:32:25.406 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:25.406 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:25.407 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:32:25.407 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369492+0000", 2024-09-14T06:32:25.407 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.5679999999999998e-06 2024-09-14T06:32:25.407 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:25.407 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:25.407 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:32:25.407 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.417645+0000", 2024-09-14T06:32:25.407 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.048152753 2024-09-14T06:32:25.407 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:25.407 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:25.407 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:32:25.407 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.417687+0000", 2024-09-14T06:32:25.407 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.1900999999999997e-05 2024-09-14T06:32:25.407 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:25.408 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:25.408 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:32:25.408 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.417739+0000", 2024-09-14T06:32:25.408 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.2299999999999997e-05 2024-09-14T06:32:25.408 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:25.408 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:32:25.408 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:25.408 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:25.408 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:25.408 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:53253 3.f 3:f71f51a7:::rbd_data.112dca2f6ae1.0000000000000015:17 [read 2084864~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:32:25.408 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:44.369495+0000", 2024-09-14T06:32:25.408 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 160.99423567100001, 2024-09-14T06:32:25.409 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.048366785000000002, 2024-09-14T06:32:25.409 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:32:25.409 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:32:25.409 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:32:25.409 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:32:25.409 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:32:25.409 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:25.409 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 53253 2024-09-14T06:32:25.409 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:25.409 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:32:25.410 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:25.410 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:32:25.410 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369495+0000", 2024-09-14T06:32:25.410 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:25.410 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:25.410 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:25.410 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:32:25.410 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369495+0000", 2024-09-14T06:32:25.410 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:25.410 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:25.410 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:25.410 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:32:25.411 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370388+0000", 2024-09-14T06:32:25.411 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00089287999999999998 2024-09-14T06:32:25.411 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:25.411 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:25.411 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:32:25.411 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370392+0000", 2024-09-14T06:32:25.411 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.2130000000000002e-06 2024-09-14T06:32:25.411 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:25.411 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:25.411 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:32:25.411 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370401+0000", 2024-09-14T06:32:25.411 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.7649999999999999e-06 2024-09-14T06:32:25.412 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:25.412 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:25.412 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:32:25.412 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370406+0000", 2024-09-14T06:32:25.412 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.0159999999999999e-06 2024-09-14T06:32:25.412 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:25.412 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:25.412 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:32:25.412 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.417764+0000", 2024-09-14T06:32:25.412 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047358348000000001 2024-09-14T06:32:25.412 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:25.412 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:25.413 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:32:25.413 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.417806+0000", 2024-09-14T06:32:25.413 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.1149999999999997e-05 2024-09-14T06:32:25.413 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:25.413 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:25.413 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:32:25.413 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.417862+0000", 2024-09-14T06:32:25.413 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.6412999999999997e-05 2024-09-14T06:32:25.413 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:25.414 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:32:25.414 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:25.414 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:25.414 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:25.414 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:53340 3.f 3:f71f51a7:::rbd_data.112dca2f6ae1.0000000000000015:17 [read 1056768~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:32:25.414 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:44.370421+0000", 2024-09-14T06:32:25.414 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 160.993309791, 2024-09-14T06:32:25.414 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047566313999999998, 2024-09-14T06:32:25.414 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:32:25.414 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:32:25.414 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:32:25.414 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:32:25.414 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:32:25.414 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:25.415 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 53340 2024-09-14T06:32:25.415 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:25.415 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:32:25.415 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:25.415 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:32:25.415 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370421+0000", 2024-09-14T06:32:25.415 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:25.415 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:25.415 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:25.415 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:32:25.415 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370421+0000", 2024-09-14T06:32:25.415 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:25.416 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:25.416 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:25.416 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:32:25.416 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370422+0000", 2024-09-14T06:32:25.416 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.3799999999999996e-07 2024-09-14T06:32:25.416 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:25.416 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:25.416 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:32:25.416 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370424+0000", 2024-09-14T06:32:25.416 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.6919999999999998e-06 2024-09-14T06:32:25.416 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:25.416 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:25.416 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:32:25.416 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370432+0000", 2024-09-14T06:32:25.417 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.2439999999999996e-06 2024-09-14T06:32:25.417 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:25.417 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:25.417 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:32:25.417 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370437+0000", 2024-09-14T06:32:25.417 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.5389999999999998e-06 2024-09-14T06:32:25.417 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:25.417 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:25.417 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:32:25.417 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.417888+0000", 2024-09-14T06:32:25.417 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047450432000000001 2024-09-14T06:32:25.417 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:25.418 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:25.418 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:32:25.418 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.417935+0000", 2024-09-14T06:32:25.418 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.7454999999999999e-05 2024-09-14T06:32:25.418 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:25.418 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:25.418 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:32:25.418 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.417987+0000", 2024-09-14T06:32:25.418 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.2213999999999998e-05 2024-09-14T06:32:25.418 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:25.418 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:32:25.418 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:25.418 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:25.419 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:25.419 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:53341 3.f 3:f71f51a7:::rbd_data.112dca2f6ae1.0000000000000015:17 [read 1081344~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:32:25.419 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:44.370446+0000", 2024-09-14T06:32:25.419 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 160.993284518, 2024-09-14T06:32:25.419 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047665723, 2024-09-14T06:32:25.419 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:32:25.419 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:32:25.419 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:32:25.419 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:32:25.419 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:32:25.419 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:25.419 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 53341 2024-09-14T06:32:25.419 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:25.419 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:32:25.419 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:25.420 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:32:25.420 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370446+0000", 2024-09-14T06:32:25.420 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:25.420 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:25.420 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:25.420 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:32:25.420 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370446+0000", 2024-09-14T06:32:25.420 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:25.420 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:25.420 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:25.420 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:32:25.420 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370447+0000", 2024-09-14T06:32:25.420 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.6100000000000002e-07 2024-09-14T06:32:25.421 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:25.421 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:25.421 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:32:25.421 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370448+0000", 2024-09-14T06:32:25.421 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.7370000000000001e-06 2024-09-14T06:32:25.421 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:25.421 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:25.421 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:32:25.421 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370462+0000", 2024-09-14T06:32:25.421 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.3227000000000001e-05 2024-09-14T06:32:25.422 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:25.422 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:25.422 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:32:25.422 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370467+0000", 2024-09-14T06:32:25.422 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.6540000000000004e-06 2024-09-14T06:32:25.422 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:25.422 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:25.422 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:32:25.422 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.418013+0000", 2024-09-14T06:32:25.422 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047545325999999999 2024-09-14T06:32:25.422 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:25.422 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:25.422 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:32:25.423 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.418054+0000", 2024-09-14T06:32:25.423 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.1696999999999997e-05 2024-09-14T06:32:25.423 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:25.423 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:25.423 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:32:25.423 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.418112+0000", 2024-09-14T06:32:25.423 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.7720999999999998e-05 2024-09-14T06:32:25.423 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:25.423 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:32:25.423 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:25.423 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:25.423 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:25.423 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:53342 3.f 3:f71f51a7:::rbd_data.112dca2f6ae1.0000000000000015:17 [read 1105920~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:32:25.423 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:44.370475+0000", 2024-09-14T06:32:25.424 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 160.993255457, 2024-09-14T06:32:25.424 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047748848000000003, 2024-09-14T06:32:25.424 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:32:25.424 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:32:25.424 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:32:25.424 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:32:25.424 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:32:25.424 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:25.424 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 53342 2024-09-14T06:32:25.424 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:25.424 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:32:25.424 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:25.425 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:32:25.425 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370475+0000", 2024-09-14T06:32:25.425 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:25.425 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:25.425 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:25.425 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:32:25.425 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370475+0000", 2024-09-14T06:32:25.425 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:25.425 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:25.425 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:25.426 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:32:25.426 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370476+0000", 2024-09-14T06:32:25.426 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.4700000000000002e-07 2024-09-14T06:32:25.426 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:25.426 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:25.426 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:32:25.426 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370478+0000", 2024-09-14T06:32:25.426 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.5890000000000001e-06 2024-09-14T06:32:25.426 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:25.426 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:25.426 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:32:25.426 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370486+0000", 2024-09-14T06:32:25.427 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.3470000000000001e-06 2024-09-14T06:32:25.427 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:25.427 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:25.427 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:32:25.427 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370491+0000", 2024-09-14T06:32:25.427 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.5720000000000002e-06 2024-09-14T06:32:25.427 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:25.427 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:25.427 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:32:25.427 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.418135+0000", 2024-09-14T06:32:25.427 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047644094999999997 2024-09-14T06:32:25.427 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:25.427 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:25.428 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:32:25.428 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.418174+0000", 2024-09-14T06:32:25.428 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.8204000000000002e-05 2024-09-14T06:32:25.428 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:25.428 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:25.428 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:32:25.428 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.418224+0000", 2024-09-14T06:32:25.428 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.0593999999999997e-05 2024-09-14T06:32:25.428 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:25.428 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:32:25.429 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:25.429 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:25.429 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:25.429 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:53343 3.f 3:f71f51a7:::rbd_data.112dca2f6ae1.0000000000000015:17 [read 1130496~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:32:25.429 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:44.370498+0000", 2024-09-14T06:32:25.429 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 160.993232581, 2024-09-14T06:32:25.429 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047841574999999997, 2024-09-14T06:32:25.429 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:32:25.429 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:32:25.429 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:32:25.429 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:32:25.429 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:32:25.429 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:25.430 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 53343 2024-09-14T06:32:25.430 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:25.430 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:32:25.430 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:25.430 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:32:25.430 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370498+0000", 2024-09-14T06:32:25.430 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:25.430 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:25.430 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:25.430 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:32:25.430 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370498+0000", 2024-09-14T06:32:25.430 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:25.430 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:25.430 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:25.431 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:32:25.431 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370499+0000", 2024-09-14T06:32:25.431 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.9999999999999998e-07 2024-09-14T06:32:25.431 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:25.431 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:25.431 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:32:25.431 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370501+0000", 2024-09-14T06:32:25.431 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.0310000000000001e-06 2024-09-14T06:32:25.431 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:25.431 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:25.431 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:32:25.431 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370507+0000", 2024-09-14T06:32:25.431 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.4169999999999997e-06 2024-09-14T06:32:25.432 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:25.432 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:25.432 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:32:25.432 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370516+0000", 2024-09-14T06:32:25.432 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.7190000000000007e-06 2024-09-14T06:32:25.432 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:25.432 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:25.432 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:32:25.432 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.418247+0000", 2024-09-14T06:32:25.432 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047731190999999999 2024-09-14T06:32:25.432 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:25.432 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:25.432 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:32:25.433 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.418286+0000", 2024-09-14T06:32:25.433 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.8988e-05 2024-09-14T06:32:25.433 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:25.433 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:25.433 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:32:25.433 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.418340+0000", 2024-09-14T06:32:25.433 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.3829000000000002e-05 2024-09-14T06:32:25.433 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:25.433 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:32:25.433 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:25.433 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:25.433 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:25.433 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:53344 3.f 3:f71f51a7:::rbd_data.112dca2f6ae1.0000000000000015:17 [read 1155072~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:32:25.433 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:44.370524+0000", 2024-09-14T06:32:25.434 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 160.99320681099999, 2024-09-14T06:32:25.434 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047972262000000002, 2024-09-14T06:32:25.434 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:32:25.434 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:32:25.434 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:32:25.434 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:32:25.434 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:32:25.434 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:25.434 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 53344 2024-09-14T06:32:25.434 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:25.434 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:32:25.435 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:25.435 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:32:25.435 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370524+0000", 2024-09-14T06:32:25.435 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:25.435 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:25.435 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:25.435 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:32:25.435 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370524+0000", 2024-09-14T06:32:25.435 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:25.435 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:25.435 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:25.435 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:32:25.435 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370618+0000", 2024-09-14T06:32:25.436 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.4425999999999993e-05 2024-09-14T06:32:25.436 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:25.436 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:25.436 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:32:25.436 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370622+0000", 2024-09-14T06:32:25.436 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.3519999999999998e-06 2024-09-14T06:32:25.436 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:25.436 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:25.436 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:32:25.436 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370631+0000", 2024-09-14T06:32:25.436 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.9299999999999992e-06 2024-09-14T06:32:25.436 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:25.436 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:25.437 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:32:25.437 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370637+0000", 2024-09-14T06:32:25.437 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.3269999999999997e-06 2024-09-14T06:32:25.437 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:25.437 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:25.437 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:32:25.437 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.418372+0000", 2024-09-14T06:32:25.437 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047734764999999998 2024-09-14T06:32:25.437 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:25.437 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:25.437 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:32:25.437 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.418431+0000", 2024-09-14T06:32:25.437 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.8696999999999997e-05 2024-09-14T06:32:25.438 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:25.438 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:25.438 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:32:25.438 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.418496+0000", 2024-09-14T06:32:25.438 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.5765000000000005e-05 2024-09-14T06:32:25.438 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:25.438 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:32:25.438 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:25.438 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:25.438 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:25.438 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:53345 3.f 3:f71f51a7:::rbd_data.112dca2f6ae1.0000000000000015:17 [read 1179648~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:32:25.438 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:44.370647+0000", 2024-09-14T06:32:25.438 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 160.99308334400001, 2024-09-14T06:32:25.439 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047953775999999997, 2024-09-14T06:32:25.439 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:32:25.439 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:32:25.439 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:32:25.439 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:32:25.439 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:32:25.439 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:25.439 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 53345 2024-09-14T06:32:25.439 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:25.439 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:32:25.439 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:25.439 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:32:25.439 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370647+0000", 2024-09-14T06:32:25.439 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:25.440 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:25.440 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:25.440 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:32:25.440 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370647+0000", 2024-09-14T06:32:25.440 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:25.440 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:25.440 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:25.440 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:32:25.440 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.372346+0000", 2024-09-14T06:32:25.440 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.001698284 2024-09-14T06:32:25.440 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:25.440 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:25.441 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:32:25.441 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.372351+0000", 2024-09-14T06:32:25.441 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.8890000000000001e-06 2024-09-14T06:32:25.441 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:25.441 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:25.441 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:32:25.441 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.372362+0000", 2024-09-14T06:32:25.441 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.0941000000000001e-05 2024-09-14T06:32:25.441 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:25.441 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:25.441 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:32:25.441 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.372369+0000", 2024-09-14T06:32:25.441 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.362e-06 2024-09-14T06:32:25.442 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:25.442 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:25.442 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:32:25.442 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.418510+0000", 2024-09-14T06:32:25.442 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046140914999999998 2024-09-14T06:32:25.442 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:25.442 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:25.442 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:32:25.442 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.418549+0000", 2024-09-14T06:32:25.442 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.8717999999999998e-05 2024-09-14T06:32:25.442 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:25.443 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:25.443 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:32:25.443 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.418601+0000", 2024-09-14T06:32:25.443 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.2667000000000002e-05 2024-09-14T06:32:25.443 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:25.443 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:32:25.444 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:25.444 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:25.444 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:25.444 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:53353 3.f 3:f71f51a7:::rbd_data.112dca2f6ae1.0000000000000015:17 [read 1376256~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:32:25.444 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:44.372545+0000", 2024-09-14T06:32:25.444 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 160.991186269, 2024-09-14T06:32:25.444 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047364232999999999, 2024-09-14T06:32:25.444 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:32:25.444 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:32:25.445 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:32:25.445 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:32:25.445 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:32:25.445 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:25.445 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 53353 2024-09-14T06:32:25.445 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:25.445 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:32:25.445 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:25.445 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:32:25.445 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.372545+0000", 2024-09-14T06:32:25.445 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:25.445 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:25.445 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:25.445 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:32:25.446 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.372545+0000", 2024-09-14T06:32:25.446 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:25.446 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:25.446 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:25.446 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:32:25.446 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.372545+0000", 2024-09-14T06:32:25.446 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.27e-07 2024-09-14T06:32:25.446 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:25.446 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:25.446 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:32:25.446 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.372547+0000", 2024-09-14T06:32:25.446 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.0600000000000002e-06 2024-09-14T06:32:25.447 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:25.447 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:25.447 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:32:25.447 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.372553+0000", 2024-09-14T06:32:25.447 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.7980000000000002e-06 2024-09-14T06:32:25.447 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:25.447 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:25.447 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:32:25.447 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.372557+0000", 2024-09-14T06:32:25.447 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.7389999999999999e-06 2024-09-14T06:32:25.447 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:25.447 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:25.447 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:32:25.448 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.419811+0000", 2024-09-14T06:32:25.448 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047253310999999999 2024-09-14T06:32:25.448 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:25.448 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:25.448 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:32:25.448 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.419854+0000", 2024-09-14T06:32:25.448 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.3448000000000001e-05 2024-09-14T06:32:25.448 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:25.448 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:25.448 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:32:25.448 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.419909+0000", 2024-09-14T06:32:25.448 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.4549999999999998e-05 2024-09-14T06:32:25.448 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:25.449 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:32:25.449 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:25.449 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:25.449 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:25.449 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:53354 3.f 3:f71f51a7:::rbd_data.112dca2f6ae1.0000000000000015:17 [read 1400832~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:32:25.449 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:44.372563+0000", 2024-09-14T06:32:25.449 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 160.991167832, 2024-09-14T06:32:25.449 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047482233999999998, 2024-09-14T06:32:25.449 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:32:25.449 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:32:25.449 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:32:25.449 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:32:25.449 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:32:25.450 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:25.450 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 53354 2024-09-14T06:32:25.450 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:25.450 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:32:25.450 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:25.450 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:32:25.450 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.372563+0000", 2024-09-14T06:32:25.450 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:25.450 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:25.450 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:25.450 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:32:25.450 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.372563+0000", 2024-09-14T06:32:25.450 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:25.450 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:25.451 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:25.451 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:32:25.451 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.375009+0000", 2024-09-14T06:32:25.451 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.0024457680000000001 2024-09-14T06:32:25.451 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:25.451 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:25.451 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:32:25.451 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.375014+0000", 2024-09-14T06:32:25.451 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.0599999999999998e-06 2024-09-14T06:32:25.451 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:25.451 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:25.451 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:32:25.451 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.375026+0000", 2024-09-14T06:32:25.451 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.1783999999999999e-05 2024-09-14T06:32:25.452 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:25.452 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:25.452 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:32:25.452 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.375035+0000", 2024-09-14T06:32:25.452 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.7799999999999995e-06 2024-09-14T06:32:25.452 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:25.452 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:25.452 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:32:25.452 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.419942+0000", 2024-09-14T06:32:25.452 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.044906982999999998 2024-09-14T06:32:25.452 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:25.452 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:25.452 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:32:25.453 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.419989+0000", 2024-09-14T06:32:25.453 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.7104999999999997e-05 2024-09-14T06:32:25.453 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:25.453 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:25.453 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:32:25.453 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.420045+0000", 2024-09-14T06:32:25.453 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.5754000000000002e-05 2024-09-14T06:32:25.453 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:25.453 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:32:25.453 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:25.453 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:25.453 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:25.453 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:225577 3.15 3:ac159a70:::rbd_header.112daa9ec306:head [watch reconnect cookie 140616827151664 gen 4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e188)", 2024-09-14T06:32:25.453 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:31:52.733493+0000", 2024-09-14T06:32:25.454 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 32.630237368000003, 2024-09-14T06:32:25.454 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.0156593789999999, 2024-09-14T06:32:25.454 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:32:25.454 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:32:25.454 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:32:25.454 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:32:25.454 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:32:25.454 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:25.454 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 225577 2024-09-14T06:32:25.454 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:25.454 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:32:25.454 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:25.454 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:32:25.454 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:31:52.733493+0000", 2024-09-14T06:32:25.455 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:25.455 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:25.455 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:25.455 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:32:25.455 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:31:52.733493+0000", 2024-09-14T06:32:25.455 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:25.455 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:25.455 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:25.455 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:32:25.455 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:31:52.733495+0000", 2024-09-14T06:32:25.455 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.0059999999999999e-06 2024-09-14T06:32:25.456 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:25.456 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:25.456 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:32:25.456 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:31:52.733504+0000", 2024-09-14T06:32:25.456 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.5490000000000006e-06 2024-09-14T06:32:25.456 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:25.457 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:25.457 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:32:25.457 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:31:52.733522+0000", 2024-09-14T06:32:25.457 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.7660999999999999e-05 2024-09-14T06:32:25.457 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:25.458 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:25.458 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:32:25.458 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:31:52.733531+0000", 2024-09-14T06:32:25.458 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.7180000000000008e-06 2024-09-14T06:32:25.458 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:25.458 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:25.458 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:32:25.459 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:31:52.733598+0000", 2024-09-14T06:32:25.459 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.6490999999999997e-05 2024-09-14T06:32:25.459 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:25.459 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:25.459 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "op must wait for map", 2024-09-14T06:32:25.459 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:31:52.733605+0000", 2024-09-14T06:32:25.459 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.6109999999999996e-06 2024-09-14T06:32:25.460 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:25.460 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:25.460 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:32:25.460 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:31:52.737908+0000", 2024-09-14T06:32:25.460 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.0043023050000000002 2024-09-14T06:32:25.460 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:25.460 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:25.461 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "waiting for peered", 2024-09-14T06:32:25.461 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:31:52.737923+0000", 2024-09-14T06:32:25.461 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.488e-05 2024-09-14T06:32:25.461 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:25.461 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:25.461 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:32:25.462 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:31:53.746776+0000", 2024-09-14T06:32:25.462 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.0088535359999999 2024-09-14T06:32:25.462 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:25.462 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:25.462 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:32:25.463 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:31:53.746878+0000", 2024-09-14T06:32:25.463 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.000101788 2024-09-14T06:32:25.463 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:25.463 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:25.463 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:32:25.463 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:31:53.749153+0000", 2024-09-14T06:32:25.463 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.0022748339999999999 2024-09-14T06:32:25.464 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:25.464 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:32:25.464 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:25.464 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:25.464 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:25.464 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:225581 3.15 3:ab407115:::rbd_object_map.112d838b9898:head [call lock.unlock in=10b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e188)", 2024-09-14T06:32:25.464 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:31:52.743472+0000", 2024-09-14T06:32:25.464 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 32.620258329000002, 2024-09-14T06:32:25.464 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.007270501, 2024-09-14T06:32:25.464 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:32:25.464 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:32:25.465 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:32:25.465 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:32:25.465 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:32:25.465 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:25.465 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 225581 2024-09-14T06:32:25.465 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:25.465 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:32:25.465 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:25.465 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:32:25.465 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:31:52.743472+0000", 2024-09-14T06:32:25.465 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:25.465 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:25.465 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:25.466 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:32:25.466 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:31:52.743472+0000", 2024-09-14T06:32:25.466 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:25.466 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:25.466 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:25.466 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:32:25.466 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:31:52.743473+0000", 2024-09-14T06:32:25.466 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.1999999999999998e-07 2024-09-14T06:32:25.466 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:25.466 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:25.466 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:32:25.466 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:31:52.743478+0000", 2024-09-14T06:32:25.466 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.515e-06 2024-09-14T06:32:25.466 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:25.467 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:25.467 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:32:25.467 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:31:52.743489+0000", 2024-09-14T06:32:25.467 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.1411e-05 2024-09-14T06:32:25.467 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:25.467 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:25.467 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:32:25.467 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:31:52.743494+0000", 2024-09-14T06:32:25.467 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.1649999999999997e-06 2024-09-14T06:32:25.467 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:25.467 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:25.467 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:32:25.467 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:31:52.743548+0000", 2024-09-14T06:32:25.468 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.3758999999999998e-05 2024-09-14T06:32:25.468 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:25.468 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:25.468 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "waiting for peered", 2024-09-14T06:32:25.468 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:31:52.743558+0000", 2024-09-14T06:32:25.468 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.0336999999999999e-05 2024-09-14T06:32:25.468 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:25.468 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:25.468 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:32:25.468 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:31:53.747716+0000", 2024-09-14T06:32:25.468 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.004157035 2024-09-14T06:32:25.468 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:25.469 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:25.469 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:32:25.469 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:31:53.747831+0000", 2024-09-14T06:32:25.469 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.000115394 2024-09-14T06:32:25.469 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:25.469 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:25.469 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "waiting for subops from 3", 2024-09-14T06:32:25.469 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:31:53.748002+0000", 2024-09-14T06:32:25.469 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00017085099999999999 2024-09-14T06:32:25.469 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:25.469 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:25.469 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:32:25.470 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:31:53.749536+0000", 2024-09-14T06:32:25.470 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.0015339990000000001 2024-09-14T06:32:25.470 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:25.470 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:25.470 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "op_commit", 2024-09-14T06:32:25.470 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:31:53.750672+0000", 2024-09-14T06:32:25.470 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.001135973 2024-09-14T06:32:25.470 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:25.470 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:25.470 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "commit_sent", 2024-09-14T06:32:25.470 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:31:53.750704+0000", 2024-09-14T06:32:25.470 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.1791000000000001e-05 2024-09-14T06:32:25.470 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:25.471 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:25.471 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:32:25.471 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:31:53.750743+0000", 2024-09-14T06:32:25.471 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.9450999999999999e-05 2024-09-14T06:32:25.471 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:25.471 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:32:25.471 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:25.471 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:25.471 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:32:25.471 INFO:teuthology.orchestra.run.smithi138.stdout:} 2024-09-14T06:32:25.473 INFO:tasks.ceph.osd.4.smithi138.stderr:2024-09-14T06:32:25.471+0000 7f6c653c8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33615) UID: 0 2024-09-14T06:32:25.473 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-14T06:32:25.573 INFO:tasks.ceph.osd.7.smithi138.stderr:2024-09-14T06:32:25.572+0000 7f3c6a21d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33623) UID: 0 2024-09-14T06:32:25.577 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-14T06:32:25.578 INFO:teuthology.orchestra.run.smithi094.stdout: "ops": [], 2024-09-14T06:32:25.578 INFO:teuthology.orchestra.run.smithi094.stdout: "num_ops": 0 2024-09-14T06:32:25.578 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-14T06:32:25.586 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-14T06:32:25.674 INFO:tasks.ceph.osd.0.smithi094.stderr:2024-09-14T06:32:25.673+0000 7f07c9333640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33636) UID: 0 2024-09-14T06:32:25.700 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-14T06:32:25.700 INFO:teuthology.orchestra.run.smithi094.stdout: "ops": [], 2024-09-14T06:32:25.700 INFO:teuthology.orchestra.run.smithi094.stdout: "complaint_time": 30, 2024-09-14T06:32:25.700 INFO:teuthology.orchestra.run.smithi094.stdout: "num_blocked_ops": 0 2024-09-14T06:32:25.701 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-14T06:32:25.707 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-14T06:32:25.775 INFO:tasks.ceph.osd.6.smithi138.stderr:2024-09-14T06:32:25.774+0000 7f8cd8849640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33620) UID: 0 2024-09-14T06:32:25.852 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-14T06:32:25.852 INFO:teuthology.orchestra.run.smithi094.stdout: "size": 20, 2024-09-14T06:32:25.852 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 600, 2024-09-14T06:32:25.852 INFO:teuthology.orchestra.run.smithi094.stdout: "ops": [ 2024-09-14T06:32:25.853 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:25.853 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:115193 3.1 3:856be886:::rbd_data.112d4d24c558.0000000000000004:29 [read 1523712~16384 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e96)", 2024-09-14T06:32:25.853 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:30:17.939854+0000", 2024-09-14T06:32:25.853 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 127.90969167, 2024-09-14T06:32:25.853 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.073897516999999996, 2024-09-14T06:32:25.854 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:32:25.854 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:32:25.854 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:32:25.854 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:32:25.854 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:32:25.855 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:25.855 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 115193 2024-09-14T06:32:25.855 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:25.855 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:32:25.855 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:25.855 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:32:25.856 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.939854+0000", 2024-09-14T06:32:25.856 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:25.856 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:25.856 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:25.856 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:32:25.856 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.939854+0000", 2024-09-14T06:32:25.857 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:25.857 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:25.857 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:25.857 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:32:25.857 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.939854+0000", 2024-09-14T06:32:25.857 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.9500000000000002e-07 2024-09-14T06:32:25.858 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:25.858 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:25.858 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:32:25.858 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.939856+0000", 2024-09-14T06:32:25.858 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.0959999999999999e-06 2024-09-14T06:32:25.859 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:25.859 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:25.859 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:32:25.859 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.939863+0000", 2024-09-14T06:32:25.859 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.1129999999999997e-06 2024-09-14T06:32:25.860 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:25.860 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:25.860 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:32:25.860 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.939867+0000", 2024-09-14T06:32:25.860 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.8350000000000003e-06 2024-09-14T06:32:25.860 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:25.860 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:25.861 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:32:25.861 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.013626+0000", 2024-09-14T06:32:25.861 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.073758799999999999 2024-09-14T06:32:25.861 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:25.861 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:25.861 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:32:25.862 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.013690+0000", 2024-09-14T06:32:25.862 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.3918999999999996e-05 2024-09-14T06:32:25.862 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:25.862 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:25.862 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:32:25.862 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.013751+0000", 2024-09-14T06:32:25.862 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.1159000000000002e-05 2024-09-14T06:32:25.862 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:25.862 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:32:25.863 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:25.863 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:25.863 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:25.863 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:115194 3.1 3:856be886:::rbd_data.112d4d24c558.0000000000000004:29 [read 1769472~16384 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e96)", 2024-09-14T06:32:25.863 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:30:17.939874+0000", 2024-09-14T06:32:25.863 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 127.909671359, 2024-09-14T06:32:25.863 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.074016988000000006, 2024-09-14T06:32:25.863 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:32:25.863 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:32:25.863 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:32:25.863 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:32:25.863 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:32:25.863 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:25.863 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 115194 2024-09-14T06:32:25.864 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:25.864 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:32:25.864 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:25.864 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:32:25.864 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.939874+0000", 2024-09-14T06:32:25.864 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:25.864 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:25.864 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:25.864 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:32:25.864 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.939874+0000", 2024-09-14T06:32:25.864 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:25.864 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:25.865 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:25.865 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:32:25.865 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.939875+0000", 2024-09-14T06:32:25.865 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.2600000000000002e-07 2024-09-14T06:32:25.865 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:25.865 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:25.865 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:32:25.865 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.939877+0000", 2024-09-14T06:32:25.865 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.9420000000000002e-06 2024-09-14T06:32:25.865 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:25.865 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:25.865 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:32:25.865 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.939882+0000", 2024-09-14T06:32:25.866 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.8749999999999997e-06 2024-09-14T06:32:25.866 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:25.866 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:25.866 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:32:25.866 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.939887+0000", 2024-09-14T06:32:25.866 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.741e-06 2024-09-14T06:32:25.866 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:25.866 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:25.866 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:32:25.866 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.013767+0000", 2024-09-14T06:32:25.866 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.073879720999999995 2024-09-14T06:32:25.866 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:25.866 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:25.867 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:32:25.867 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.013832+0000", 2024-09-14T06:32:25.867 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.5312000000000001e-05 2024-09-14T06:32:25.867 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:25.867 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:25.867 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:32:25.867 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.013891+0000", 2024-09-14T06:32:25.867 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.8870999999999999e-05 2024-09-14T06:32:25.867 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:25.867 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:32:25.867 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:25.867 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:25.868 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:25.868 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:115195 3.1 3:856be886:::rbd_data.112d4d24c558.0000000000000004:29 [read 2015232~16384 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e96)", 2024-09-14T06:32:25.868 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:30:17.939903+0000", 2024-09-14T06:32:25.868 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 127.909642174, 2024-09-14T06:32:25.868 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.074128199000000006, 2024-09-14T06:32:25.868 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:32:25.868 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:32:25.868 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:32:25.868 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:32:25.868 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:32:25.868 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:25.868 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 115195 2024-09-14T06:32:25.869 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:25.869 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:32:25.869 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:25.869 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:32:25.869 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.939903+0000", 2024-09-14T06:32:25.869 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:25.869 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:25.869 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:25.869 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:32:25.869 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.939903+0000", 2024-09-14T06:32:25.869 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:25.869 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:25.870 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:25.870 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:32:25.870 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.939904+0000", 2024-09-14T06:32:25.870 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.5799999999999999e-07 2024-09-14T06:32:25.870 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:25.870 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:25.870 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:32:25.870 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.939906+0000", 2024-09-14T06:32:25.870 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.2350000000000002e-06 2024-09-14T06:32:25.870 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:25.870 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:25.870 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:32:25.870 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.939913+0000", 2024-09-14T06:32:25.871 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.5649999999999998e-06 2024-09-14T06:32:25.871 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:25.871 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:25.871 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:32:25.871 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.939918+0000", 2024-09-14T06:32:25.871 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.784e-06 2024-09-14T06:32:25.871 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:25.871 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:25.871 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:32:25.871 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.013921+0000", 2024-09-14T06:32:25.871 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.074002911000000005 2024-09-14T06:32:25.871 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:25.871 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:25.871 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:32:25.872 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.013971+0000", 2024-09-14T06:32:25.872 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.9551e-05 2024-09-14T06:32:25.872 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:25.872 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:25.872 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:32:25.872 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.014031+0000", 2024-09-14T06:32:25.872 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.0594999999999998e-05 2024-09-14T06:32:25.872 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:25.872 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:32:25.872 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:25.872 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:25.872 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:25.873 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:115196 3.1 3:856be886:::rbd_data.112d4d24c558.0000000000000004:29 [read 2260992~16384 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e96)", 2024-09-14T06:32:25.873 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:30:17.939925+0000", 2024-09-14T06:32:25.873 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 127.909619982, 2024-09-14T06:32:25.873 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.074320137999999994, 2024-09-14T06:32:25.873 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:32:25.873 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:32:25.873 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:32:25.873 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:32:25.873 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:32:25.873 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:25.873 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 115196 2024-09-14T06:32:25.874 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:25.874 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:32:25.874 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:25.874 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:32:25.874 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.939925+0000", 2024-09-14T06:32:25.874 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:25.874 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:25.874 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:25.874 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:32:25.874 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.939925+0000", 2024-09-14T06:32:25.874 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:25.874 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:25.874 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:25.875 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:32:25.875 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.939926+0000", 2024-09-14T06:32:25.875 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.9999999999999998e-07 2024-09-14T06:32:25.875 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:25.875 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:25.875 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:32:25.875 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.939928+0000", 2024-09-14T06:32:25.875 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.9549999999999999e-06 2024-09-14T06:32:25.875 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:25.875 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:25.875 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:32:25.875 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.939934+0000", 2024-09-14T06:32:25.875 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.127e-06 2024-09-14T06:32:25.875 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:25.876 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:25.876 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:32:25.876 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.939939+0000", 2024-09-14T06:32:25.876 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.775e-06 2024-09-14T06:32:25.876 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:25.876 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:25.876 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:32:25.876 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.014075+0000", 2024-09-14T06:32:25.876 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.074136598999999997 2024-09-14T06:32:25.876 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:25.876 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:25.876 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:32:25.876 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.014162+0000", 2024-09-14T06:32:25.877 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.6477999999999994e-05 2024-09-14T06:32:25.877 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:25.877 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:25.877 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:32:25.877 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.014246+0000", 2024-09-14T06:32:25.877 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.3703999999999997e-05 2024-09-14T06:32:25.877 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:25.877 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:32:25.877 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:25.877 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:25.877 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:25.877 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:115197 3.1 3:856be886:::rbd_data.112d4d24c558.0000000000000004:29 [read 2506752~16384 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e96)", 2024-09-14T06:32:25.877 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:30:17.939969+0000", 2024-09-14T06:32:25.877 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 127.909576497, 2024-09-14T06:32:25.878 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.074528544000000002, 2024-09-14T06:32:25.878 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:32:25.878 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:32:25.878 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:32:25.878 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:32:25.878 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:32:25.878 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:25.878 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 115197 2024-09-14T06:32:25.878 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:25.878 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:32:25.878 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:25.878 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:32:25.878 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.939969+0000", 2024-09-14T06:32:25.879 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:25.879 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:25.879 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:25.879 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:32:25.879 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.939969+0000", 2024-09-14T06:32:25.879 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:25.879 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:25.879 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:25.879 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:32:25.879 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.939970+0000", 2024-09-14T06:32:25.880 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.3600000000000003e-07 2024-09-14T06:32:25.880 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:25.880 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:25.880 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:32:25.880 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.939972+0000", 2024-09-14T06:32:25.880 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.3180000000000001e-06 2024-09-14T06:32:25.880 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:25.880 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:25.880 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:32:25.880 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.939979+0000", 2024-09-14T06:32:25.880 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.8800000000000002e-06 2024-09-14T06:32:25.880 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:25.881 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:25.881 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:32:25.881 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.939984+0000", 2024-09-14T06:32:25.881 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.3510000000000001e-06 2024-09-14T06:32:25.881 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:25.881 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:25.881 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:32:25.881 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.014302+0000", 2024-09-14T06:32:25.881 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.074317416999999997 2024-09-14T06:32:25.881 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:25.882 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:25.882 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:32:25.882 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.014392+0000", 2024-09-14T06:32:25.882 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.0181999999999998e-05 2024-09-14T06:32:25.882 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:25.882 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:25.882 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:32:25.882 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.014497+0000", 2024-09-14T06:32:25.882 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00010576 2024-09-14T06:32:25.882 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:25.882 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:32:25.883 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:25.883 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:25.883 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:25.883 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:115198 3.1 3:856be886:::rbd_data.112d4d24c558.0000000000000004:29 [read 2752512~16384 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e96)", 2024-09-14T06:32:25.883 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:30:17.940004+0000", 2024-09-14T06:32:25.883 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 127.909541217, 2024-09-14T06:32:25.883 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.074736367999999997, 2024-09-14T06:32:25.883 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:32:25.883 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:32:25.883 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:32:25.883 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:32:25.883 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:32:25.883 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:25.884 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 115198 2024-09-14T06:32:25.884 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:25.884 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:32:25.884 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:25.884 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:32:25.884 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940004+0000", 2024-09-14T06:32:25.884 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:25.884 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:25.884 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:25.884 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:32:25.884 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940004+0000", 2024-09-14T06:32:25.884 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:25.884 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:25.885 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:25.885 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:32:25.885 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940005+0000", 2024-09-14T06:32:25.885 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.2200000000000004e-07 2024-09-14T06:32:25.885 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:25.885 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:25.885 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:32:25.885 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940007+0000", 2024-09-14T06:32:25.885 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.322e-06 2024-09-14T06:32:25.885 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:25.885 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:25.885 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:32:25.886 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940014+0000", 2024-09-14T06:32:25.886 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.6220000000000003e-06 2024-09-14T06:32:25.886 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:25.886 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:25.886 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:32:25.886 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940019+0000", 2024-09-14T06:32:25.886 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.6380000000000001e-06 2024-09-14T06:32:25.886 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:25.886 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:25.886 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:32:25.886 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.014562+0000", 2024-09-14T06:32:25.887 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.074542320999999995 2024-09-14T06:32:25.887 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:25.887 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:25.887 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:32:25.887 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.014646+0000", 2024-09-14T06:32:25.887 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.4079999999999995e-05 2024-09-14T06:32:25.887 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:25.887 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:25.887 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:32:25.887 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.014741+0000", 2024-09-14T06:32:25.887 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.4763000000000001e-05 2024-09-14T06:32:25.887 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:25.887 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:32:25.887 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:25.888 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:25.888 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:25.888 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:115199 3.1 3:856be886:::rbd_data.112d4d24c558.0000000000000004:29 [read 2998272~16384 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e96)", 2024-09-14T06:32:25.888 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:30:17.940040+0000", 2024-09-14T06:32:25.888 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 127.909505462, 2024-09-14T06:32:25.888 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.074957134999999994, 2024-09-14T06:32:25.888 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:32:25.888 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:32:25.888 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:32:25.888 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:32:25.888 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:32:25.888 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:25.888 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 115199 2024-09-14T06:32:25.889 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:25.889 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:32:25.889 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:25.889 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:32:25.889 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940040+0000", 2024-09-14T06:32:25.889 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:25.889 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:25.889 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:25.889 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:32:25.889 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940040+0000", 2024-09-14T06:32:25.889 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:25.889 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:25.889 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:25.890 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:32:25.890 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940041+0000", 2024-09-14T06:32:25.890 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.4000000000000001e-07 2024-09-14T06:32:25.890 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:25.890 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:25.890 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:32:25.890 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940046+0000", 2024-09-14T06:32:25.890 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.7250000000000002e-06 2024-09-14T06:32:25.890 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:25.890 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:25.890 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:32:25.890 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940053+0000", 2024-09-14T06:32:25.890 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.6109999999999999e-06 2024-09-14T06:32:25.890 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:25.891 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:25.891 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:32:25.891 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940058+0000", 2024-09-14T06:32:25.891 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.1590000000000001e-06 2024-09-14T06:32:25.891 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:25.891 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:25.891 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:32:25.891 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.014768+0000", 2024-09-14T06:32:25.891 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.074709624000000002 2024-09-14T06:32:25.891 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:25.891 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:25.891 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:32:25.891 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.014907+0000", 2024-09-14T06:32:25.892 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000139665 2024-09-14T06:32:25.892 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:25.892 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:25.892 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:32:25.892 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.014997+0000", 2024-09-14T06:32:25.892 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.9710999999999995e-05 2024-09-14T06:32:25.892 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:25.892 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:32:25.892 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:25.892 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:25.892 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:25.892 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:115200 3.1 3:856be886:::rbd_data.112d4d24c558.0000000000000004:29 [read 3244032~16384 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e96)", 2024-09-14T06:32:25.892 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:30:17.940080+0000", 2024-09-14T06:32:25.892 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 127.90946525699999, 2024-09-14T06:32:25.893 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.075137291999999994, 2024-09-14T06:32:25.893 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:32:25.893 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:32:25.893 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:32:25.893 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:32:25.893 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:32:25.893 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:25.893 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 115200 2024-09-14T06:32:25.893 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:25.894 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:32:25.894 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:25.894 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:32:25.894 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940080+0000", 2024-09-14T06:32:25.894 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:25.894 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:25.894 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:25.894 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:32:25.894 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940080+0000", 2024-09-14T06:32:25.895 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:25.895 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:25.895 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:25.895 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:32:25.895 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940081+0000", 2024-09-14T06:32:25.895 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.0800000000000004e-07 2024-09-14T06:32:25.895 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:25.895 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:25.896 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:32:25.896 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940083+0000", 2024-09-14T06:32:25.896 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.3920000000000001e-06 2024-09-14T06:32:25.896 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:25.896 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:25.896 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:32:25.896 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940091+0000", 2024-09-14T06:32:25.896 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.1389999999999995e-06 2024-09-14T06:32:25.897 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:25.897 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:25.897 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:32:25.897 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940097+0000", 2024-09-14T06:32:25.897 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.5550000000000002e-06 2024-09-14T06:32:25.897 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:25.897 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:25.897 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:32:25.898 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.015025+0000", 2024-09-14T06:32:25.898 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.074928579999999995 2024-09-14T06:32:25.898 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:25.898 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:25.898 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:32:25.898 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.015096+0000", 2024-09-14T06:32:25.898 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.0705999999999997e-05 2024-09-14T06:32:25.898 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:25.898 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:25.899 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:32:25.899 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.015217+0000", 2024-09-14T06:32:25.899 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00012131200000000001 2024-09-14T06:32:25.899 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:25.899 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:32:25.899 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:25.899 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:25.899 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:25.900 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:115201 3.1 3:856be886:::rbd_data.112d4d24c558.0000000000000004:29 [read 3489792~16384 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e96)", 2024-09-14T06:32:25.900 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:30:17.940113+0000", 2024-09-14T06:32:25.900 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 127.90943248400001, 2024-09-14T06:32:25.900 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.075246899000000006, 2024-09-14T06:32:25.900 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:32:25.900 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:32:25.900 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:32:25.900 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:32:25.901 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:32:25.901 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:25.901 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 115201 2024-09-14T06:32:25.901 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:25.901 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:32:25.901 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:25.901 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:32:25.901 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940113+0000", 2024-09-14T06:32:25.902 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:25.902 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:25.902 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:25.902 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:32:25.902 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940113+0000", 2024-09-14T06:32:25.902 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:25.902 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:25.902 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:25.903 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:32:25.903 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940114+0000", 2024-09-14T06:32:25.903 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.1500000000000004e-07 2024-09-14T06:32:25.903 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:25.903 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:25.903 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:32:25.903 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940116+0000", 2024-09-14T06:32:25.903 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.2809999999999998e-06 2024-09-14T06:32:25.904 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:25.904 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:25.904 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:32:25.904 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940123+0000", 2024-09-14T06:32:25.904 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.9229999999999996e-06 2024-09-14T06:32:25.904 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:25.904 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:25.904 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:32:25.904 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940128+0000", 2024-09-14T06:32:25.905 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.7790000000000003e-06 2024-09-14T06:32:25.905 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:25.905 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:25.905 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:32:25.905 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.015253+0000", 2024-09-14T06:32:25.905 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.075125792999999996 2024-09-14T06:32:25.905 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:25.906 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:25.906 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:32:25.906 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.015297+0000", 2024-09-14T06:32:25.906 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.3847000000000002e-05 2024-09-14T06:32:25.906 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:25.906 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:25.906 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:32:25.907 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.015360+0000", 2024-09-14T06:32:25.907 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.2661000000000002e-05 2024-09-14T06:32:25.907 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:25.907 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:32:25.907 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:25.907 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:25.907 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:25.907 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:115202 3.1 3:856be886:::rbd_data.112d4d24c558.0000000000000004:29 [read 3735552~16384 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e96)", 2024-09-14T06:32:25.907 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:30:17.940135+0000", 2024-09-14T06:32:25.908 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 127.909410705, 2024-09-14T06:32:25.908 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.075451105000000004, 2024-09-14T06:32:25.908 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:32:25.908 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:32:25.908 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:32:25.908 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:32:25.908 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:32:25.908 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:25.908 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 115202 2024-09-14T06:32:25.909 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:25.909 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:32:25.909 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:25.909 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:32:25.909 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940135+0000", 2024-09-14T06:32:25.909 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:25.909 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:25.909 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:25.909 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:32:25.909 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940135+0000", 2024-09-14T06:32:25.909 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:25.909 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:25.909 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:25.910 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:32:25.910 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940135+0000", 2024-09-14T06:32:25.910 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.0999999999999999e-07 2024-09-14T06:32:25.910 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:25.910 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:25.910 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:32:25.910 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940137+0000", 2024-09-14T06:32:25.910 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.0020000000000001e-06 2024-09-14T06:32:25.910 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:25.910 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:25.910 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:32:25.910 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940143+0000", 2024-09-14T06:32:25.910 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.9850000000000003e-06 2024-09-14T06:32:25.910 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:25.910 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:25.911 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:32:25.911 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940148+0000", 2024-09-14T06:32:25.911 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.9880000000000004e-06 2024-09-14T06:32:25.911 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:25.911 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:25.911 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:32:25.911 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.015400+0000", 2024-09-14T06:32:25.911 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.075252186999999998 2024-09-14T06:32:25.911 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:25.911 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:25.912 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:32:25.912 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.015481+0000", 2024-09-14T06:32:25.912 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.0698000000000006e-05 2024-09-14T06:32:25.912 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:25.912 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:25.912 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:32:25.912 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.015586+0000", 2024-09-14T06:32:25.912 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00010483499999999999 2024-09-14T06:32:25.912 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:25.912 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:32:25.912 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:25.912 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:25.913 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:25.913 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:115203 3.1 3:856be886:::rbd_data.112d4d24c558.0000000000000004:29 [read 3981312~16384 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e96)", 2024-09-14T06:32:25.913 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:30:17.940156+0000", 2024-09-14T06:32:25.913 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 127.90938948, 2024-09-14T06:32:25.913 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.075620860999999998, 2024-09-14T06:32:25.913 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:32:25.913 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:32:25.913 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:32:25.913 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:32:25.913 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:32:25.913 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:25.913 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 115203 2024-09-14T06:32:25.914 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:25.914 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:32:25.914 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:25.914 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:32:25.914 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940156+0000", 2024-09-14T06:32:25.914 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:25.914 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:25.914 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:25.914 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:32:25.914 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940156+0000", 2024-09-14T06:32:25.914 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:25.915 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:25.915 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:25.915 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:32:25.915 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940156+0000", 2024-09-14T06:32:25.915 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.1500000000000005e-07 2024-09-14T06:32:25.915 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:25.915 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:25.915 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:32:25.916 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940158+0000", 2024-09-14T06:32:25.916 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.967e-06 2024-09-14T06:32:25.916 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:25.916 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:25.916 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:32:25.916 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940165+0000", 2024-09-14T06:32:25.916 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.6939999999999998e-06 2024-09-14T06:32:25.916 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:25.916 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:25.916 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:32:25.916 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940170+0000", 2024-09-14T06:32:25.916 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.7539999999999997e-06 2024-09-14T06:32:25.916 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:25.916 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:25.917 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:32:25.917 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.015633+0000", 2024-09-14T06:32:25.917 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.075463334000000007 2024-09-14T06:32:25.917 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:25.917 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:25.917 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:32:25.917 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.015693+0000", 2024-09-14T06:32:25.917 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.9367000000000003e-05 2024-09-14T06:32:25.917 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:25.917 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:25.917 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:32:25.917 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.015777+0000", 2024-09-14T06:32:25.917 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.4229999999999998e-05 2024-09-14T06:32:25.918 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:25.918 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:32:25.918 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:25.918 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:25.918 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:25.918 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:115204 3.1 3:856be886:::rbd_data.112d4d24c558.0000000000000004:29 [read 4227072~16384 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e96)", 2024-09-14T06:32:25.918 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:30:17.940176+0000", 2024-09-14T06:32:25.918 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 127.909368947, 2024-09-14T06:32:25.918 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.075785505000000003, 2024-09-14T06:32:25.918 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:32:25.918 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:32:25.918 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:32:25.918 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:32:25.918 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:32:25.919 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:25.919 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 115204 2024-09-14T06:32:25.919 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:25.919 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:32:25.919 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:25.919 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:32:25.919 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940176+0000", 2024-09-14T06:32:25.919 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:25.919 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:25.919 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:25.919 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:32:25.920 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940176+0000", 2024-09-14T06:32:25.920 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:25.920 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:25.920 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:25.920 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:32:25.920 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940177+0000", 2024-09-14T06:32:25.920 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.7899999999999999e-07 2024-09-14T06:32:25.920 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:25.920 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:25.920 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:32:25.920 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940179+0000", 2024-09-14T06:32:25.920 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.103e-06 2024-09-14T06:32:25.921 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:25.921 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:25.921 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:32:25.921 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940185+0000", 2024-09-14T06:32:25.921 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.7239999999999997e-06 2024-09-14T06:32:25.921 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:25.921 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:25.921 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:32:25.921 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940190+0000", 2024-09-14T06:32:25.921 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.7269999999999998e-06 2024-09-14T06:32:25.921 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:25.921 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:25.921 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:32:25.922 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.015814+0000", 2024-09-14T06:32:25.922 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.075624114000000006 2024-09-14T06:32:25.922 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:25.922 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:25.922 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:32:25.922 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.015862+0000", 2024-09-14T06:32:25.922 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.8770999999999998e-05 2024-09-14T06:32:25.922 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:25.922 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:25.922 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:32:25.922 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.015962+0000", 2024-09-14T06:32:25.922 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.9586999999999994e-05 2024-09-14T06:32:25.923 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:25.923 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:32:25.923 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:25.923 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:25.923 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:25.923 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:115205 3.1 3:856be886:::rbd_data.112d4d24c558.0000000000000004:29 [read 4472832~16384 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e96)", 2024-09-14T06:32:25.923 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:30:17.940197+0000", 2024-09-14T06:32:25.923 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 127.90934823800001, 2024-09-14T06:32:25.923 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.076068920999999998, 2024-09-14T06:32:25.923 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:32:25.923 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:32:25.923 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:32:25.924 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:32:25.924 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:32:25.924 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:25.924 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 115205 2024-09-14T06:32:25.924 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:25.924 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:32:25.924 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:25.924 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:32:25.924 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940197+0000", 2024-09-14T06:32:25.924 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:25.924 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:25.924 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:25.924 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:32:25.925 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940197+0000", 2024-09-14T06:32:25.925 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:25.925 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:25.925 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:25.925 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:32:25.925 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940198+0000", 2024-09-14T06:32:25.925 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.5999999999999999e-07 2024-09-14T06:32:25.925 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:25.925 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:25.925 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:32:25.925 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940200+0000", 2024-09-14T06:32:25.925 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.0899999999999999e-06 2024-09-14T06:32:25.925 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:25.926 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:25.926 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:32:25.926 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940206+0000", 2024-09-14T06:32:25.926 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.8710000000000002e-06 2024-09-14T06:32:25.926 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:25.926 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:25.926 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:32:25.926 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940211+0000", 2024-09-14T06:32:25.926 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.9230000000000001e-06 2024-09-14T06:32:25.926 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:25.926 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:25.926 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:32:25.927 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.016059+0000", 2024-09-14T06:32:25.927 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.075848051999999999 2024-09-14T06:32:25.927 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:25.927 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:25.927 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:32:25.927 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.016151+0000", 2024-09-14T06:32:25.927 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.1982000000000001e-05 2024-09-14T06:32:25.927 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:25.927 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:25.927 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:32:25.927 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.016266+0000", 2024-09-14T06:32:25.927 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000115543 2024-09-14T06:32:25.928 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:25.928 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:32:25.928 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:25.928 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:25.928 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:25.928 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:115206 3.1 3:856be886:::rbd_data.112d4d24c558.0000000000000004:29 [read 4718592~16384 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e96)", 2024-09-14T06:32:25.928 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:30:17.940223+0000", 2024-09-14T06:32:25.928 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 127.909322148, 2024-09-14T06:32:25.928 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.076265115999999994, 2024-09-14T06:32:25.928 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:32:25.928 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:32:25.928 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:32:25.929 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:32:25.929 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:32:25.929 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:25.929 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 115206 2024-09-14T06:32:25.929 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:25.929 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:32:25.929 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:25.929 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:32:25.929 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940223+0000", 2024-09-14T06:32:25.930 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:25.930 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:25.930 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:25.930 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:32:25.930 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940223+0000", 2024-09-14T06:32:25.930 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:25.930 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:25.930 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:25.930 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:32:25.930 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940224+0000", 2024-09-14T06:32:25.930 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.0399999999999996e-07 2024-09-14T06:32:25.930 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:25.930 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:25.931 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:32:25.931 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940226+0000", 2024-09-14T06:32:25.931 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.1780000000000002e-06 2024-09-14T06:32:25.931 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:25.931 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:25.931 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:32:25.931 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940232+0000", 2024-09-14T06:32:25.931 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.5010000000000001e-06 2024-09-14T06:32:25.931 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:25.931 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:25.931 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:32:25.931 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940237+0000", 2024-09-14T06:32:25.931 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.0250000000000002e-06 2024-09-14T06:32:25.932 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:25.932 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:25.932 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:32:25.932 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.016305+0000", 2024-09-14T06:32:25.932 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.076067262999999996 2024-09-14T06:32:25.932 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:25.932 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:25.932 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:32:25.932 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.016383+0000", 2024-09-14T06:32:25.932 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.8578000000000006e-05 2024-09-14T06:32:25.932 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:25.932 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:25.932 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:32:25.932 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.016488+0000", 2024-09-14T06:32:25.933 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000105067 2024-09-14T06:32:25.933 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:25.933 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:32:25.933 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:25.933 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:25.933 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:25.933 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:115207 3.1 3:856be886:::rbd_data.112d4d24c558.0000000000000004:29 [read 4964352~16384 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e96)", 2024-09-14T06:32:25.933 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:30:17.940244+0000", 2024-09-14T06:32:25.933 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 127.909301032, 2024-09-14T06:32:25.933 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.076371154999999996, 2024-09-14T06:32:25.933 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:32:25.933 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:32:25.933 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:32:25.934 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:32:25.934 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:32:25.934 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:25.934 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 115207 2024-09-14T06:32:25.934 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:25.934 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:32:25.934 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:25.934 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:32:25.934 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940244+0000", 2024-09-14T06:32:25.934 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:25.934 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:25.934 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:25.934 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:32:25.935 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940244+0000", 2024-09-14T06:32:25.935 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:25.935 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:25.935 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:25.935 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:32:25.935 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940245+0000", 2024-09-14T06:32:25.935 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.1200000000000003e-07 2024-09-14T06:32:25.935 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:25.935 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:25.935 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:32:25.935 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940247+0000", 2024-09-14T06:32:25.935 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.0030000000000001e-06 2024-09-14T06:32:25.936 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:25.936 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:25.936 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:32:25.936 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940252+0000", 2024-09-14T06:32:25.936 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.5520000000000004e-06 2024-09-14T06:32:25.936 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:25.936 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:25.936 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:32:25.936 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940257+0000", 2024-09-14T06:32:25.936 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.6759999999999998e-06 2024-09-14T06:32:25.937 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:25.937 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:25.937 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:32:25.937 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.016521+0000", 2024-09-14T06:32:25.937 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.076263499999999998 2024-09-14T06:32:25.937 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:25.937 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:25.937 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:32:25.937 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.016562+0000", 2024-09-14T06:32:25.937 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.0967000000000003e-05 2024-09-14T06:32:25.937 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:25.937 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:25.937 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:32:25.938 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.016616+0000", 2024-09-14T06:32:25.938 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.3944999999999999e-05 2024-09-14T06:32:25.938 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:25.938 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:32:25.938 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:25.938 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:25.938 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:25.938 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:115208 3.1 3:856be886:::rbd_data.112d4d24c558.0000000000000004:29 [read 5210112~16384 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e96)", 2024-09-14T06:32:25.938 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:30:17.940264+0000", 2024-09-14T06:32:25.938 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 127.909281175, 2024-09-14T06:32:25.938 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.076558288000000002, 2024-09-14T06:32:25.938 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:32:25.938 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:32:25.938 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:32:25.938 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:32:25.939 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:32:25.939 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:25.939 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 115208 2024-09-14T06:32:25.939 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:25.939 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:32:25.939 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:25.939 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:32:25.939 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940264+0000", 2024-09-14T06:32:25.939 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:25.939 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:25.939 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:25.939 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:32:25.939 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940264+0000", 2024-09-14T06:32:25.940 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:25.940 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:25.940 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:25.940 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:32:25.940 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940265+0000", 2024-09-14T06:32:25.940 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.0500000000000004e-07 2024-09-14T06:32:25.940 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:25.940 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:25.940 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:32:25.940 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940267+0000", 2024-09-14T06:32:25.940 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.9810000000000002e-06 2024-09-14T06:32:25.940 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:25.940 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:25.941 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:32:25.941 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940272+0000", 2024-09-14T06:32:25.941 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.6999999999999996e-06 2024-09-14T06:32:25.941 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:25.941 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:25.941 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:32:25.941 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940277+0000", 2024-09-14T06:32:25.941 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.6480000000000002e-06 2024-09-14T06:32:25.941 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:25.941 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:25.941 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:32:25.941 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.016650+0000", 2024-09-14T06:32:25.942 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.076373089000000005 2024-09-14T06:32:25.942 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:25.942 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:25.942 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:32:25.942 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.016724+0000", 2024-09-14T06:32:25.942 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.4105999999999998e-05 2024-09-14T06:32:25.942 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:25.942 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:25.942 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:32:25.942 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.016823+0000", 2024-09-14T06:32:25.942 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.8258999999999996e-05 2024-09-14T06:32:25.942 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:25.942 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:32:25.943 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:25.943 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:25.943 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:25.943 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:115209 3.1 3:856be886:::rbd_data.112d4d24c558.0000000000000004:29 [read 5455872~16384 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e96)", 2024-09-14T06:32:25.943 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:30:17.940283+0000", 2024-09-14T06:32:25.943 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 127.909262149, 2024-09-14T06:32:25.943 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.076724029999999999, 2024-09-14T06:32:25.943 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:32:25.943 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:32:25.943 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:32:25.943 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:32:25.943 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:32:25.943 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:25.943 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 115209 2024-09-14T06:32:25.944 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:25.944 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:32:25.944 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:25.944 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:32:25.944 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940283+0000", 2024-09-14T06:32:25.944 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:25.944 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:25.944 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:25.944 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:32:25.944 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940283+0000", 2024-09-14T06:32:25.944 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:25.944 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:25.945 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:25.945 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:32:25.945 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940284+0000", 2024-09-14T06:32:25.945 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.6700000000000003e-07 2024-09-14T06:32:25.945 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:25.945 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:25.945 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:32:25.945 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940286+0000", 2024-09-14T06:32:25.945 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.9520000000000001e-06 2024-09-14T06:32:25.945 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:25.945 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:25.945 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:32:25.945 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940292+0000", 2024-09-14T06:32:25.945 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.7339999999999996e-06 2024-09-14T06:32:25.946 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:25.946 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:25.946 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:32:25.946 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940296+0000", 2024-09-14T06:32:25.946 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.7700000000000001e-06 2024-09-14T06:32:25.946 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:25.946 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:25.946 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:32:25.946 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.016859+0000", 2024-09-14T06:32:25.946 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.07656317 2024-09-14T06:32:25.946 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:25.946 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:25.946 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:32:25.946 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.016929+0000", 2024-09-14T06:32:25.947 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.9679999999999997e-05 2024-09-14T06:32:25.947 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:25.947 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:25.947 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:32:25.947 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.017007+0000", 2024-09-14T06:32:25.947 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.8157000000000004e-05 2024-09-14T06:32:25.947 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:25.947 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:32:25.947 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:25.947 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:25.947 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:25.947 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:115210 3.1 3:856be886:::rbd_data.112d4d24c558.0000000000000004:29 [read 5701632~16384 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e96)", 2024-09-14T06:32:25.947 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:30:17.940303+0000", 2024-09-14T06:32:25.947 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 127.909242632, 2024-09-14T06:32:25.948 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.076821358000000006, 2024-09-14T06:32:25.948 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:32:25.948 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:32:25.948 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:32:25.948 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:32:25.948 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:32:25.948 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:25.948 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 115210 2024-09-14T06:32:25.948 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:25.948 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:32:25.948 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:25.949 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:32:25.949 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940303+0000", 2024-09-14T06:32:25.949 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:25.949 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:25.952 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:25.952 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:32:25.952 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940303+0000", 2024-09-14T06:32:25.952 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:25.952 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:25.952 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:25.952 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:32:25.952 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940303+0000", 2024-09-14T06:32:25.952 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.3000000000000001e-07 2024-09-14T06:32:25.952 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:25.952 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:25.953 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:32:25.953 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940306+0000", 2024-09-14T06:32:25.953 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.255e-06 2024-09-14T06:32:25.953 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:25.953 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:25.953 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:32:25.953 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940312+0000", 2024-09-14T06:32:25.953 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.9710000000000001e-06 2024-09-14T06:32:25.953 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:25.953 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:25.953 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:32:25.953 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940316+0000", 2024-09-14T06:32:25.953 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.5340000000000001e-06 2024-09-14T06:32:25.954 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:25.954 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:25.954 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:32:25.954 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.017033+0000", 2024-09-14T06:32:25.954 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.076717169000000002 2024-09-14T06:32:25.954 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:25.954 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:25.954 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:32:25.954 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.017073+0000", 2024-09-14T06:32:25.954 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.9582000000000002e-05 2024-09-14T06:32:25.954 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:25.954 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:25.954 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:32:25.954 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.017124+0000", 2024-09-14T06:32:25.955 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.1316999999999997e-05 2024-09-14T06:32:25.955 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:25.955 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:32:25.955 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:25.955 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:25.955 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:25.955 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:115219 3.1 3:856be886:::rbd_data.112d4d24c558.0000000000000004:29 [read 65536~16384 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e96)", 2024-09-14T06:32:25.955 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:30:17.940323+0000", 2024-09-14T06:32:25.955 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 127.909222771, 2024-09-14T06:32:25.955 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.076915205, 2024-09-14T06:32:25.955 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:32:25.956 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:32:25.956 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:32:25.956 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:32:25.956 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:32:25.956 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:25.956 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 115219 2024-09-14T06:32:25.956 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:25.956 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:32:25.956 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:25.956 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:32:25.956 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940323+0000", 2024-09-14T06:32:25.956 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:25.957 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:25.957 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:25.957 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:32:25.957 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940323+0000", 2024-09-14T06:32:25.957 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:25.957 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:25.957 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:25.957 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:32:25.957 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940323+0000", 2024-09-14T06:32:25.957 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.0800000000000005e-07 2024-09-14T06:32:25.957 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:25.957 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:25.958 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:32:25.958 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940328+0000", 2024-09-14T06:32:25.958 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.887e-06 2024-09-14T06:32:25.958 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:25.958 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:25.958 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:32:25.958 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940334+0000", 2024-09-14T06:32:25.958 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.3269999999999997e-06 2024-09-14T06:32:25.958 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:25.958 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:25.958 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:32:25.958 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940339+0000", 2024-09-14T06:32:25.958 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.7960000000000003e-06 2024-09-14T06:32:25.958 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:25.959 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:25.959 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:32:25.959 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.017148+0000", 2024-09-14T06:32:25.959 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.076809125000000006 2024-09-14T06:32:25.959 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:25.959 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:25.959 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:32:25.959 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.017186+0000", 2024-09-14T06:32:25.959 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.7283000000000002e-05 2024-09-14T06:32:25.959 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:25.959 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:25.959 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:32:25.960 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.017238+0000", 2024-09-14T06:32:25.960 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.2278999999999999e-05 2024-09-14T06:32:25.960 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:25.960 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:32:25.960 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:25.960 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:25.960 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:25.960 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:115220 3.1 3:856be886:::rbd_data.112d4d24c558.0000000000000004:29 [read 311296~16384 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e96)", 2024-09-14T06:32:25.960 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:30:17.940346+0000", 2024-09-14T06:32:25.960 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 127.909199642, 2024-09-14T06:32:25.960 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.077033293000000003, 2024-09-14T06:32:25.960 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:32:25.961 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:32:25.961 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:32:25.961 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:32:25.961 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:32:25.961 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:25.961 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 115220 2024-09-14T06:32:25.961 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:25.961 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:32:25.961 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:25.961 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:32:25.961 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940346+0000", 2024-09-14T06:32:25.961 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:25.962 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:25.962 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:25.962 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:32:25.962 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940346+0000", 2024-09-14T06:32:25.962 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:25.962 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:25.962 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:25.962 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:32:25.962 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940346+0000", 2024-09-14T06:32:25.962 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.4300000000000003e-07 2024-09-14T06:32:25.962 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:25.962 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:25.962 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:32:25.963 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940348+0000", 2024-09-14T06:32:25.963 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.0030000000000001e-06 2024-09-14T06:32:25.963 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:25.963 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:25.963 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:32:25.963 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.004655+0000", 2024-09-14T06:32:25.963 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.064306323999999998 2024-09-14T06:32:25.963 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:25.963 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:25.963 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:32:25.963 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.004671+0000", 2024-09-14T06:32:25.963 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.6589999999999999e-05 2024-09-14T06:32:25.964 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:25.964 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:25.964 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:32:25.964 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.017262+0000", 2024-09-14T06:32:25.964 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.012590570000000001 2024-09-14T06:32:25.964 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:25.964 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:25.964 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:32:25.964 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.017321+0000", 2024-09-14T06:32:25.964 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.9210999999999999e-05 2024-09-14T06:32:25.964 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:25.964 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:25.964 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:32:25.965 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.017379+0000", 2024-09-14T06:32:25.965 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.8051999999999999e-05 2024-09-14T06:32:25.965 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:25.965 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:32:25.965 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:25.965 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:25.965 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:32:25.965 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-14T06:32:25.967 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-14T06:32:25.967 INFO:tasks.ceph.osd.2.smithi094.stderr:2024-09-14T06:32:25.966+0000 7faf3269b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33649) UID: 0 2024-09-14T06:32:26.067 INFO:tasks.ceph.osd.2.smithi094.stderr:2024-09-14T06:32:26.066+0000 7faf3269b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33649) UID: 0 2024-09-14T06:32:26.111 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-14T06:32:26.111 INFO:teuthology.orchestra.run.smithi094.stdout: "ops": [], 2024-09-14T06:32:26.112 INFO:teuthology.orchestra.run.smithi094.stdout: "num_ops": 0 2024-09-14T06:32:26.112 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-14T06:32:26.120 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-14T06:32:26.168 INFO:tasks.ceph.osd.5.smithi138.stderr:2024-09-14T06:32:26.167+0000 7fe13746f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33618) UID: 0 2024-09-14T06:32:26.263 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-14T06:32:26.263 INFO:teuthology.orchestra.run.smithi094.stdout: "ops": [], 2024-09-14T06:32:26.263 INFO:teuthology.orchestra.run.smithi094.stdout: "complaint_time": 30, 2024-09-14T06:32:26.263 INFO:teuthology.orchestra.run.smithi094.stdout: "num_blocked_ops": 0 2024-09-14T06:32:26.263 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-14T06:32:26.268 INFO:tasks.ceph.osd.5.smithi138.stderr:2024-09-14T06:32:26.267+0000 7fe13746f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33618) UID: 0 2024-09-14T06:32:26.271 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-14T06:32:26.370 INFO:tasks.ceph.osd.6.smithi138.stderr:2024-09-14T06:32:26.369+0000 7f8cd8849640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33620) UID: 0 2024-09-14T06:32:26.375 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-14T06:32:26.375 INFO:teuthology.orchestra.run.smithi094.stdout: "size": 20, 2024-09-14T06:32:26.375 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 600, 2024-09-14T06:32:26.376 INFO:teuthology.orchestra.run.smithi094.stdout: "ops": [ 2024-09-14T06:32:26.376 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:26.376 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:14918 3.0 3:05c39535:::rbd_data.112d92027590.0000000000000009:f [read 795648~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:32:26.376 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:26.083834+0000", 2024-09-14T06:32:26.376 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 180.28977083000001, 2024-09-14T06:32:26.376 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.041776287000000002, 2024-09-14T06:32:26.376 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:32:26.376 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:32:26.376 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:32:26.377 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:32:26.377 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:32:26.377 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:26.377 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 14918 2024-09-14T06:32:26.377 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:26.377 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:32:26.377 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:26.377 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:32:26.377 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.083834+0000", 2024-09-14T06:32:26.377 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:26.377 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:26.378 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:26.378 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:32:26.378 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.083834+0000", 2024-09-14T06:32:26.378 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:26.378 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:26.378 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:26.378 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:32:26.378 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.083834+0000", 2024-09-14T06:32:26.378 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.0100000000000001e-07 2024-09-14T06:32:26.378 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:26.378 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:26.378 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:32:26.379 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.083835+0000", 2024-09-14T06:32:26.379 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.3900000000000003e-07 2024-09-14T06:32:26.379 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:26.379 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:26.379 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:32:26.379 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.083838+0000", 2024-09-14T06:32:26.379 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.9830000000000001e-06 2024-09-14T06:32:26.379 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:26.379 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:26.379 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:32:26.379 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.083840+0000", 2024-09-14T06:32:26.379 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.4679999999999999e-06 2024-09-14T06:32:26.379 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:26.379 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:26.380 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:32:26.380 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.125455+0000", 2024-09-14T06:32:26.380 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.041614578999999999 2024-09-14T06:32:26.380 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:26.380 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:26.380 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:32:26.380 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.125496+0000", 2024-09-14T06:32:26.380 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.1047000000000002e-05 2024-09-14T06:32:26.380 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:26.380 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:26.380 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:32:26.380 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.125610+0000", 2024-09-14T06:32:26.380 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00011407 2024-09-14T06:32:26.380 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:26.381 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:32:26.381 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:26.381 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:26.381 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:26.381 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:14919 3.0 3:05c39535:::rbd_data.112d92027590.0000000000000009:f [read 805888~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:32:26.381 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:26.083843+0000", 2024-09-14T06:32:26.381 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 180.28976108200001, 2024-09-14T06:32:26.381 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.041992356000000002, 2024-09-14T06:32:26.381 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:32:26.381 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:32:26.381 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:32:26.381 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:32:26.382 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:32:26.382 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:26.382 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 14919 2024-09-14T06:32:26.382 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:26.382 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:32:26.382 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:26.382 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:32:26.382 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.083843+0000", 2024-09-14T06:32:26.382 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:26.382 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:26.382 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:26.382 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:32:26.382 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.083843+0000", 2024-09-14T06:32:26.383 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:26.383 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:26.383 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:26.383 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:32:26.383 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.083844+0000", 2024-09-14T06:32:26.383 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.3300000000000001e-07 2024-09-14T06:32:26.383 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:26.383 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:26.383 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:32:26.383 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.083845+0000", 2024-09-14T06:32:26.383 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.4729999999999999e-06 2024-09-14T06:32:26.383 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:26.383 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:26.383 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:32:26.384 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.083850+0000", 2024-09-14T06:32:26.384 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.0130000000000001e-06 2024-09-14T06:32:26.384 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:26.384 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:26.384 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:32:26.384 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.083854+0000", 2024-09-14T06:32:26.384 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.4759999999999998e-06 2024-09-14T06:32:26.384 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:26.384 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:26.384 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:32:26.384 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.125636+0000", 2024-09-14T06:32:26.384 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.041782683000000001 2024-09-14T06:32:26.384 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:26.384 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:26.385 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:32:26.385 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.125680+0000", 2024-09-14T06:32:26.385 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.3614e-05 2024-09-14T06:32:26.385 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:26.385 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:26.385 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:32:26.385 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.125836+0000", 2024-09-14T06:32:26.385 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000155864 2024-09-14T06:32:26.385 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:26.385 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:32:26.385 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:26.385 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:26.385 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:26.385 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:14920 3.0 3:05c39535:::rbd_data.112d92027590.0000000000000009:f [read 816128~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:32:26.386 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:26.083868+0000", 2024-09-14T06:32:26.386 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 180.28973642099999, 2024-09-14T06:32:26.386 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.042166771999999998, 2024-09-14T06:32:26.386 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:32:26.386 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:32:26.386 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:32:26.386 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:32:26.386 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:32:26.386 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:26.386 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 14920 2024-09-14T06:32:26.386 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:26.386 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:32:26.386 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:26.386 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:32:26.386 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.083868+0000", 2024-09-14T06:32:26.387 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:26.387 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:26.387 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:26.387 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:32:26.387 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.083868+0000", 2024-09-14T06:32:26.387 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:26.387 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:26.387 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:26.387 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:32:26.387 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.083869+0000", 2024-09-14T06:32:26.387 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.99e-07 2024-09-14T06:32:26.387 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:26.387 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:26.387 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:32:26.388 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.083870+0000", 2024-09-14T06:32:26.388 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.782e-06 2024-09-14T06:32:26.388 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:26.388 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:26.388 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:32:26.388 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.083876+0000", 2024-09-14T06:32:26.388 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.3809999999999998e-06 2024-09-14T06:32:26.388 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:26.388 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:26.388 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:32:26.388 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.083882+0000", 2024-09-14T06:32:26.388 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.9479999999999996e-06 2024-09-14T06:32:26.388 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:26.389 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:26.389 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:32:26.389 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.125878+0000", 2024-09-14T06:32:26.389 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.041996264999999998 2024-09-14T06:32:26.389 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:26.389 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:26.389 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:32:26.389 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.125929+0000", 2024-09-14T06:32:26.389 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.0498999999999999e-05 2024-09-14T06:32:26.389 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:26.389 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:26.389 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:32:26.390 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.126035+0000", 2024-09-14T06:32:26.390 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000106298 2024-09-14T06:32:26.390 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:26.390 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:32:26.390 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:26.390 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:26.390 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:26.390 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:14921 3.0 3:05c39535:::rbd_data.112d92027590.0000000000000009:f [read 826368~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:32:26.390 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:26.083889+0000", 2024-09-14T06:32:26.390 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 180.289715411, 2024-09-14T06:32:26.390 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.042435309999999997, 2024-09-14T06:32:26.390 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:32:26.391 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:32:26.391 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:32:26.391 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:32:26.391 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:32:26.391 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:26.391 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 14921 2024-09-14T06:32:26.391 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:26.391 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:32:26.391 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:26.391 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:32:26.391 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.083889+0000", 2024-09-14T06:32:26.391 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:26.392 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:26.392 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:26.392 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:32:26.392 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.083889+0000", 2024-09-14T06:32:26.392 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:26.392 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:26.392 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:26.392 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:32:26.392 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.086033+0000", 2024-09-14T06:32:26.392 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0021444179999999999 2024-09-14T06:32:26.392 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:26.392 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:26.392 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:32:26.392 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.086038+0000", 2024-09-14T06:32:26.393 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.0969999999999999e-06 2024-09-14T06:32:26.393 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:26.393 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:26.393 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:32:26.393 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.086046+0000", 2024-09-14T06:32:26.393 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.9440000000000005e-06 2024-09-14T06:32:26.393 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:26.393 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:26.393 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:32:26.393 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.086050+0000", 2024-09-14T06:32:26.393 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.69e-06 2024-09-14T06:32:26.393 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:26.393 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:26.393 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:32:26.394 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.126080+0000", 2024-09-14T06:32:26.394 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.040030237000000003 2024-09-14T06:32:26.394 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:26.394 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:26.394 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:32:26.394 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.126163+0000", 2024-09-14T06:32:26.394 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.2071999999999996e-05 2024-09-14T06:32:26.394 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:26.394 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:26.394 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:32:26.394 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.126324+0000", 2024-09-14T06:32:26.394 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00016185200000000001 2024-09-14T06:32:26.394 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:26.394 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:32:26.395 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:26.395 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:26.395 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:26.395 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:55627 3.0 3:06490baf:::rbd_data.112dca2f6ae1.0000000000000030:17 [read 2002944~4096 [fadvise_nocache]] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:32:26.395 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:44.599966+0000", 2024-09-14T06:32:26.395 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 161.773638463, 2024-09-14T06:32:26.395 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.041868546999999999, 2024-09-14T06:32:26.395 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:32:26.395 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:32:26.395 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:32:26.395 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:32:26.395 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:32:26.395 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:26.395 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 55627 2024-09-14T06:32:26.395 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:26.395 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:32:26.396 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:26.396 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:32:26.396 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.599966+0000", 2024-09-14T06:32:26.396 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:26.396 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:26.396 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:26.396 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:32:26.396 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.599966+0000", 2024-09-14T06:32:26.396 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:26.396 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:26.396 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:26.396 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:32:26.396 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.599966+0000", 2024-09-14T06:32:26.396 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.0900000000000001e-07 2024-09-14T06:32:26.397 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:26.397 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:26.397 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:32:26.397 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.599967+0000", 2024-09-14T06:32:26.397 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.64e-07 2024-09-14T06:32:26.397 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:26.397 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:26.397 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:32:26.397 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.599970+0000", 2024-09-14T06:32:26.397 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.9670000000000001e-06 2024-09-14T06:32:26.397 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:26.397 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:26.398 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:32:26.398 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.599973+0000", 2024-09-14T06:32:26.398 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.632e-06 2024-09-14T06:32:26.398 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:26.398 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:26.398 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:32:26.398 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.641665+0000", 2024-09-14T06:32:26.398 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.041692547000000003 2024-09-14T06:32:26.398 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:26.398 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:26.398 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:32:26.398 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.641740+0000", 2024-09-14T06:32:26.398 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.4559000000000002e-05 2024-09-14T06:32:26.398 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:26.399 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:26.399 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:32:26.399 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.641835+0000", 2024-09-14T06:32:26.399 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.4668999999999998e-05 2024-09-14T06:32:26.399 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:26.399 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:32:26.399 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:26.399 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:26.399 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:26.399 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:55628 3.0 3:06490baf:::rbd_data.112dca2f6ae1.0000000000000030:17 [read 2027520~4096 [fadvise_nocache]] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:32:26.399 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:44.599976+0000", 2024-09-14T06:32:26.399 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 161.77362867599999, 2024-09-14T06:32:26.399 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.041991621, 2024-09-14T06:32:26.399 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:32:26.400 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:32:26.400 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:32:26.400 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:32:26.400 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:32:26.400 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:26.400 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 55628 2024-09-14T06:32:26.400 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:26.400 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:32:26.400 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:26.400 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:32:26.400 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.599976+0000", 2024-09-14T06:32:26.400 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:26.400 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:26.400 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:26.401 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:32:26.401 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.599976+0000", 2024-09-14T06:32:26.401 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:26.401 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:26.401 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:26.401 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:32:26.401 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.599976+0000", 2024-09-14T06:32:26.401 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.1299999999999999e-07 2024-09-14T06:32:26.401 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:26.401 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:26.401 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:32:26.401 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.599977+0000", 2024-09-14T06:32:26.401 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.7999999999999993e-07 2024-09-14T06:32:26.402 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:26.402 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:26.402 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:32:26.402 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.599980+0000", 2024-09-14T06:32:26.402 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.01e-06 2024-09-14T06:32:26.402 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:26.402 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:26.402 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:32:26.402 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.599983+0000", 2024-09-14T06:32:26.402 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.5409999999999999e-06 2024-09-14T06:32:26.402 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:26.402 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:26.402 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:32:26.403 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.641869+0000", 2024-09-14T06:32:26.403 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.041886235000000001 2024-09-14T06:32:26.403 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:26.403 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:26.403 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:32:26.403 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.641913+0000", 2024-09-14T06:32:26.403 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.4357000000000002e-05 2024-09-14T06:32:26.403 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:26.403 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:26.403 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:32:26.403 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.641967+0000", 2024-09-14T06:32:26.403 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.4284999999999999e-05 2024-09-14T06:32:26.403 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:26.403 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:32:26.404 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:26.404 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:26.404 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:26.404 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:55629 3.0 3:06490baf:::rbd_data.112dca2f6ae1.0000000000000030:17 [read 2052096~4096 [fadvise_nocache]] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:32:26.404 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:44.599985+0000", 2024-09-14T06:32:26.404 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 161.77361897, 2024-09-14T06:32:26.404 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.042183375000000002, 2024-09-14T06:32:26.404 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:32:26.404 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:32:26.404 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:32:26.404 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:32:26.404 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:32:26.405 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:26.405 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 55629 2024-09-14T06:32:26.405 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:26.405 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:32:26.405 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:26.405 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:32:26.405 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.599985+0000", 2024-09-14T06:32:26.405 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:26.405 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:26.405 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:26.405 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:32:26.405 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.599985+0000", 2024-09-14T06:32:26.405 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:26.405 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:26.406 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:26.406 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:32:26.406 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.599986+0000", 2024-09-14T06:32:26.406 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.05e-07 2024-09-14T06:32:26.406 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:26.406 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:26.406 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:32:26.406 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.599987+0000", 2024-09-14T06:32:26.406 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.8400000000000002e-07 2024-09-14T06:32:26.406 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:26.406 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:26.406 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:32:26.406 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.599990+0000", 2024-09-14T06:32:26.406 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.5640000000000001e-06 2024-09-14T06:32:26.407 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:26.407 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:26.407 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:32:26.407 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.599993+0000", 2024-09-14T06:32:26.407 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.615e-06 2024-09-14T06:32:26.407 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:26.407 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:26.407 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:32:26.407 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.642003+0000", 2024-09-14T06:32:26.407 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.042010345999999997 2024-09-14T06:32:26.407 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:26.407 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:26.407 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:32:26.408 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.642080+0000", 2024-09-14T06:32:26.408 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.6538999999999993e-05 2024-09-14T06:32:26.408 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:26.408 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:26.408 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:32:26.408 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.642169+0000", 2024-09-14T06:32:26.408 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.9122000000000005e-05 2024-09-14T06:32:26.408 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:26.408 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:32:26.408 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:26.408 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:26.409 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:26.409 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:55630 3.0 3:06490baf:::rbd_data.112dca2f6ae1.0000000000000030:17 [read 2076672~4096 [fadvise_nocache]] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:32:26.409 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:44.599996+0000", 2024-09-14T06:32:26.409 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 161.77360849199999, 2024-09-14T06:32:26.409 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.042299615999999998, 2024-09-14T06:32:26.409 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:32:26.409 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:32:26.409 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:32:26.409 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:32:26.409 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:32:26.409 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:26.409 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 55630 2024-09-14T06:32:26.409 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:26.410 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:32:26.410 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:26.410 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:32:26.410 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.599996+0000", 2024-09-14T06:32:26.410 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:26.410 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:26.410 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:26.410 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:32:26.410 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.599996+0000", 2024-09-14T06:32:26.410 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:26.410 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:26.410 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:26.410 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:32:26.411 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.599996+0000", 2024-09-14T06:32:26.411 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.1500000000000001e-07 2024-09-14T06:32:26.411 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:26.411 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:26.411 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:32:26.411 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.599997+0000", 2024-09-14T06:32:26.411 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.33e-07 2024-09-14T06:32:26.411 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:26.411 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:26.411 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:32:26.411 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600000+0000", 2024-09-14T06:32:26.411 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.0240000000000002e-06 2024-09-14T06:32:26.411 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:26.411 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:26.412 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:32:26.412 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600003+0000", 2024-09-14T06:32:26.412 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.3640000000000002e-06 2024-09-14T06:32:26.412 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:26.412 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:26.412 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:32:26.412 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.642200+0000", 2024-09-14T06:32:26.412 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.042197571000000003 2024-09-14T06:32:26.412 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:26.412 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:26.412 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:32:26.412 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.642243+0000", 2024-09-14T06:32:26.412 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.2765000000000001e-05 2024-09-14T06:32:26.413 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:26.413 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:26.413 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:32:26.413 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.642296+0000", 2024-09-14T06:32:26.413 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.2744000000000002e-05 2024-09-14T06:32:26.413 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:26.413 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:32:26.413 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:26.413 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:26.413 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:26.413 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:55631 3.0 3:06490baf:::rbd_data.112dca2f6ae1.0000000000000030:17 [read 1048576~4096 [fadvise_nocache]] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:32:26.413 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:44.600006+0000", 2024-09-14T06:32:26.413 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 161.77359890400001, 2024-09-14T06:32:26.414 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.042487905999999999, 2024-09-14T06:32:26.414 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:32:26.414 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:32:26.414 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:32:26.414 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:32:26.414 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:32:26.414 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:26.414 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 55631 2024-09-14T06:32:26.414 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:26.414 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:32:26.414 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:26.414 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:32:26.414 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600006+0000", 2024-09-14T06:32:26.415 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:26.415 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:26.415 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:26.415 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:32:26.415 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600006+0000", 2024-09-14T06:32:26.415 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:26.415 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:26.415 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:26.415 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:32:26.415 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600006+0000", 2024-09-14T06:32:26.415 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.0800000000000001e-07 2024-09-14T06:32:26.415 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:26.415 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:26.415 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:32:26.415 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600007+0000", 2024-09-14T06:32:26.416 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.7199999999999997e-07 2024-09-14T06:32:26.416 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:26.416 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:26.416 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:32:26.416 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600010+0000", 2024-09-14T06:32:26.416 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.0280000000000001e-06 2024-09-14T06:32:26.416 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:26.416 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:26.416 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:32:26.416 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600012+0000", 2024-09-14T06:32:26.417 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.475e-06 2024-09-14T06:32:26.417 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:26.417 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:26.417 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:32:26.417 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.642329+0000", 2024-09-14T06:32:26.417 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.042317157000000001 2024-09-14T06:32:26.417 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:26.417 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:26.417 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:32:26.417 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.642403+0000", 2024-09-14T06:32:26.417 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.3941000000000002e-05 2024-09-14T06:32:26.417 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:26.417 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:26.417 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:32:26.418 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.642493+0000", 2024-09-14T06:32:26.418 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.0124999999999995e-05 2024-09-14T06:32:26.418 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:26.418 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:32:26.418 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:26.418 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:26.418 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:26.418 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:55632 3.0 3:06490baf:::rbd_data.112dca2f6ae1.0000000000000030:17 [read 1073152~4096 [fadvise_nocache]] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:32:26.418 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:44.600015+0000", 2024-09-14T06:32:26.418 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 161.77358915100001, 2024-09-14T06:32:26.418 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.042604922000000003, 2024-09-14T06:32:26.418 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:32:26.418 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:32:26.418 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:32:26.419 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:32:26.419 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:32:26.419 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:26.419 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 55632 2024-09-14T06:32:26.419 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:26.419 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:32:26.419 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:26.419 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:32:26.419 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600015+0000", 2024-09-14T06:32:26.419 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:26.419 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:26.419 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:26.420 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:32:26.420 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600015+0000", 2024-09-14T06:32:26.420 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:26.420 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:26.420 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:26.420 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:32:26.420 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600016+0000", 2024-09-14T06:32:26.420 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.1e-07 2024-09-14T06:32:26.420 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:26.420 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:26.420 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:32:26.420 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600017+0000", 2024-09-14T06:32:26.420 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.9900000000000009e-07 2024-09-14T06:32:26.421 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:26.421 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:26.421 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:32:26.421 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600020+0000", 2024-09-14T06:32:26.421 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.4170000000000001e-06 2024-09-14T06:32:26.421 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:26.421 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:26.421 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:32:26.421 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600023+0000", 2024-09-14T06:32:26.421 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.588e-06 2024-09-14T06:32:26.421 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:26.421 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:26.421 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:32:26.421 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.642526+0000", 2024-09-14T06:32:26.422 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.042503236999999999 2024-09-14T06:32:26.422 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:26.422 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:26.422 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:32:26.422 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.642568+0000", 2024-09-14T06:32:26.422 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.1925000000000003e-05 2024-09-14T06:32:26.422 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:26.422 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:26.422 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:32:26.422 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.642620+0000", 2024-09-14T06:32:26.422 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.2546000000000001e-05 2024-09-14T06:32:26.422 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:26.422 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:32:26.423 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:26.423 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:26.423 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:26.423 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:55633 3.0 3:06490baf:::rbd_data.112dca2f6ae1.0000000000000030:17 [read 1097728~4096 [fadvise_nocache]] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:32:26.423 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:44.600026+0000", 2024-09-14T06:32:26.423 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 161.77357888200001, 2024-09-14T06:32:26.423 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.042797474000000002, 2024-09-14T06:32:26.423 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:32:26.423 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:32:26.423 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:32:26.423 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:32:26.423 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:32:26.423 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:26.423 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 55633 2024-09-14T06:32:26.423 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:26.424 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:32:26.424 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:26.424 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:32:26.424 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600026+0000", 2024-09-14T06:32:26.424 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:26.424 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:26.424 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:26.424 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:32:26.424 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600026+0000", 2024-09-14T06:32:26.424 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:26.424 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:26.424 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:26.424 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:32:26.425 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600026+0000", 2024-09-14T06:32:26.425 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.3099999999999999e-07 2024-09-14T06:32:26.425 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:26.425 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:26.425 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:32:26.425 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600029+0000", 2024-09-14T06:32:26.425 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.304e-06 2024-09-14T06:32:26.425 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:26.425 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:26.425 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:32:26.425 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600034+0000", 2024-09-14T06:32:26.425 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.1429999999999997e-06 2024-09-14T06:32:26.425 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:26.425 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:26.426 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:32:26.426 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600038+0000", 2024-09-14T06:32:26.426 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.692e-06 2024-09-14T06:32:26.426 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:26.426 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:26.426 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:32:26.426 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.642654+0000", 2024-09-14T06:32:26.426 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.042615564000000002 2024-09-14T06:32:26.426 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:26.426 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:26.426 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:32:26.426 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.642728+0000", 2024-09-14T06:32:26.426 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.4018999999999997e-05 2024-09-14T06:32:26.426 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:26.427 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:26.427 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:32:26.427 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.642823+0000", 2024-09-14T06:32:26.427 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.5520999999999996e-05 2024-09-14T06:32:26.427 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:26.427 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:32:26.427 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:26.427 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:26.427 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:26.427 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:55634 3.0 3:06490baf:::rbd_data.112dca2f6ae1.0000000000000030:17 [read 1122304~4096 [fadvise_nocache]] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:32:26.427 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:44.600042+0000", 2024-09-14T06:32:26.427 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 161.77356240500001, 2024-09-14T06:32:26.427 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.042903729000000002, 2024-09-14T06:32:26.428 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:32:26.428 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:32:26.428 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:32:26.428 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:32:26.428 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:32:26.428 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:26.428 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 55634 2024-09-14T06:32:26.428 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:26.428 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:32:26.428 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:26.428 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:32:26.428 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600042+0000", 2024-09-14T06:32:26.428 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:26.428 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:26.429 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:26.429 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:32:26.429 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600042+0000", 2024-09-14T06:32:26.429 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:26.429 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:26.429 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:26.429 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:32:26.429 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600043+0000", 2024-09-14T06:32:26.429 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.7000000000000005e-07 2024-09-14T06:32:26.429 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:26.429 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:26.429 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:32:26.429 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600044+0000", 2024-09-14T06:32:26.430 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.742e-06 2024-09-14T06:32:26.430 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:26.430 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:26.430 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:32:26.430 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600050+0000", 2024-09-14T06:32:26.430 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.0800000000000002e-06 2024-09-14T06:32:26.430 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:26.430 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:26.430 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:32:26.430 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600055+0000", 2024-09-14T06:32:26.430 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.3889999999999998e-06 2024-09-14T06:32:26.430 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:26.430 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:26.430 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:32:26.431 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.642845+0000", 2024-09-14T06:32:26.431 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.042790040000000001 2024-09-14T06:32:26.431 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:26.431 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:26.431 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:32:26.431 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.642888+0000", 2024-09-14T06:32:26.431 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.2790000000000002e-05 2024-09-14T06:32:26.431 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:26.431 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:26.431 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:32:26.431 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.642946+0000", 2024-09-14T06:32:26.431 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.8118000000000002e-05 2024-09-14T06:32:26.431 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:26.432 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:32:26.432 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:26.432 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:26.432 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:26.432 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:55635 3.0 3:06490baf:::rbd_data.112dca2f6ae1.0000000000000030:17 [read 1146880~4096 [fadvise_nocache]] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:32:26.432 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:44.600061+0000", 2024-09-14T06:32:26.432 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 161.77354383400001, 2024-09-14T06:32:26.432 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.043092695, 2024-09-14T06:32:26.432 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:32:26.432 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:32:26.432 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:32:26.432 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:32:26.433 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:32:26.433 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:26.433 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 55635 2024-09-14T06:32:26.433 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:26.433 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:32:26.433 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:26.433 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:32:26.433 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600061+0000", 2024-09-14T06:32:26.433 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:26.433 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:26.433 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:26.434 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:32:26.434 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600061+0000", 2024-09-14T06:32:26.434 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:26.434 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:26.434 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:26.434 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:32:26.434 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600061+0000", 2024-09-14T06:32:26.434 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.3099999999999999e-07 2024-09-14T06:32:26.434 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:26.434 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:26.434 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:32:26.434 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600063+0000", 2024-09-14T06:32:26.435 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.6220000000000001e-06 2024-09-14T06:32:26.435 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:26.435 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:26.435 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:32:26.435 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600068+0000", 2024-09-14T06:32:26.435 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.4790000000000004e-06 2024-09-14T06:32:26.435 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:26.435 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:26.435 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:32:26.435 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600072+0000", 2024-09-14T06:32:26.435 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.3100000000000002e-06 2024-09-14T06:32:26.435 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:26.435 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:26.436 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:32:26.436 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.642987+0000", 2024-09-14T06:32:26.436 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.042914489 2024-09-14T06:32:26.436 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:26.436 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:26.436 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:32:26.436 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.643062+0000", 2024-09-14T06:32:26.436 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.5011000000000004e-05 2024-09-14T06:32:26.436 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:26.436 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:26.436 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:32:26.436 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.643153+0000", 2024-09-14T06:32:26.436 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.1452999999999993e-05 2024-09-14T06:32:26.436 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:26.437 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:32:26.437 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:26.437 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:26.437 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:26.437 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:55636 3.0 3:06490baf:::rbd_data.112dca2f6ae1.0000000000000030:17 [read 1171456~4096 [fadvise_nocache]] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:32:26.437 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:44.600078+0000", 2024-09-14T06:32:26.437 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 161.773526422, 2024-09-14T06:32:26.437 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.043207966, 2024-09-14T06:32:26.437 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:32:26.437 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:32:26.437 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:32:26.437 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:32:26.437 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:32:26.438 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:26.438 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 55636 2024-09-14T06:32:26.438 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:26.438 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:32:26.438 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:26.438 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:32:26.438 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600078+0000", 2024-09-14T06:32:26.438 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:26.438 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:26.438 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:26.438 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:32:26.438 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600078+0000", 2024-09-14T06:32:26.438 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:26.438 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:26.439 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:26.439 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:32:26.439 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600078+0000", 2024-09-14T06:32:26.439 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.34e-07 2024-09-14T06:32:26.439 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:26.439 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:26.439 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:32:26.439 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600080+0000", 2024-09-14T06:32:26.439 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.641e-06 2024-09-14T06:32:26.439 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:26.439 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:26.439 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:32:26.439 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600086+0000", 2024-09-14T06:32:26.439 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.3860000000000004e-06 2024-09-14T06:32:26.440 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:26.440 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:26.440 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:32:26.440 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600091+0000", 2024-09-14T06:32:26.440 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.5970000000000002e-06 2024-09-14T06:32:26.440 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:26.440 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:26.440 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:32:26.440 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.643184+0000", 2024-09-14T06:32:26.440 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.043092893 2024-09-14T06:32:26.440 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:26.440 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:26.440 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:32:26.440 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.643228+0000", 2024-09-14T06:32:26.441 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.4536999999999997e-05 2024-09-14T06:32:26.441 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:26.441 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:26.441 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:32:26.441 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.643286+0000", 2024-09-14T06:32:26.441 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.7578000000000003e-05 2024-09-14T06:32:26.441 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:26.441 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:32:26.441 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:26.441 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:26.441 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:26.441 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:55637 3.0 3:06490baf:::rbd_data.112dca2f6ae1.0000000000000030:17 [read 1196032~4096 [fadvise_nocache]] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:32:26.442 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:44.600097+0000", 2024-09-14T06:32:26.442 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 161.77350785799999, 2024-09-14T06:32:26.442 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.043399124999999997, 2024-09-14T06:32:26.442 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:32:26.442 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:32:26.442 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:32:26.442 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:32:26.442 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:32:26.442 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:26.442 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 55637 2024-09-14T06:32:26.442 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:26.442 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:32:26.442 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:26.442 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:32:26.442 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600097+0000", 2024-09-14T06:32:26.443 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:26.443 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:26.443 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:26.443 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:32:26.443 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600097+0000", 2024-09-14T06:32:26.443 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:26.443 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:26.443 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:26.443 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:32:26.443 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.604372+0000", 2024-09-14T06:32:26.443 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0042751489999999998 2024-09-14T06:32:26.443 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:26.443 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:26.444 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:32:26.444 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.604377+0000", 2024-09-14T06:32:26.444 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.7899999999999999e-06 2024-09-14T06:32:26.444 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:26.444 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:26.444 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:32:26.444 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.604389+0000", 2024-09-14T06:32:26.444 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.2197e-05 2024-09-14T06:32:26.444 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:26.444 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:26.444 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:32:26.444 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.604397+0000", 2024-09-14T06:32:26.444 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.8730000000000007e-06 2024-09-14T06:32:26.444 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:26.445 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:26.445 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:32:26.445 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.643324+0000", 2024-09-14T06:32:26.445 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.038926897000000002 2024-09-14T06:32:26.445 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:26.445 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:26.445 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:32:26.445 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.643402+0000", 2024-09-14T06:32:26.445 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.8063000000000001e-05 2024-09-14T06:32:26.445 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:26.445 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:26.445 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:32:26.445 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.643496+0000", 2024-09-14T06:32:26.445 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.4155999999999998e-05 2024-09-14T06:32:26.446 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:26.446 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:32:26.446 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:26.446 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:26.446 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:26.446 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:55657 3.0 3:06490baf:::rbd_data.112dca2f6ae1.0000000000000030:17 [read 1687552~4096 [fadvise_nocache]] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:32:26.446 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:44.604841+0000", 2024-09-14T06:32:26.446 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 161.76876312900001, 2024-09-14T06:32:26.446 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.041890087999999999, 2024-09-14T06:32:26.446 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:32:26.446 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:32:26.446 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:32:26.446 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:32:26.446 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:32:26.447 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:26.447 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 55657 2024-09-14T06:32:26.447 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:26.447 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:32:26.447 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:26.447 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:32:26.447 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.604841+0000", 2024-09-14T06:32:26.447 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:26.447 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:26.447 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:26.447 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:32:26.447 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.604841+0000", 2024-09-14T06:32:26.447 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:26.447 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:26.447 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:26.448 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:32:26.448 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.604842+0000", 2024-09-14T06:32:26.448 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.0900000000000001e-07 2024-09-14T06:32:26.448 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:26.448 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:26.448 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:32:26.448 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.604846+0000", 2024-09-14T06:32:26.448 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.5759999999999999e-06 2024-09-14T06:32:26.448 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:26.448 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:26.448 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:32:26.448 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.604850+0000", 2024-09-14T06:32:26.448 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.405e-06 2024-09-14T06:32:26.449 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:26.449 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:26.449 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:32:26.449 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.604852+0000", 2024-09-14T06:32:26.449 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.8590000000000001e-06 2024-09-14T06:32:26.449 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:26.449 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:26.449 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:32:26.449 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.646633+0000", 2024-09-14T06:32:26.449 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.041780772000000001 2024-09-14T06:32:26.449 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:26.449 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:26.449 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:32:26.450 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.646677+0000", 2024-09-14T06:32:26.450 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.4115e-05 2024-09-14T06:32:26.450 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:26.450 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:26.450 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:32:26.450 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.646731+0000", 2024-09-14T06:32:26.450 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.4151999999999999e-05 2024-09-14T06:32:26.450 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:26.450 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:32:26.450 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:26.450 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:26.450 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:26.450 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:55658 3.0 3:06490baf:::rbd_data.112dca2f6ae1.0000000000000030:17 [read 1712128~4096 [fadvise_nocache]] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:32:26.451 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:44.604856+0000", 2024-09-14T06:32:26.451 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 161.76874852899999, 2024-09-14T06:32:26.451 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.041986227000000001, 2024-09-14T06:32:26.451 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:32:26.451 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:32:26.451 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:32:26.451 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:32:26.451 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:32:26.451 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:26.451 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 55658 2024-09-14T06:32:26.451 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:26.451 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:32:26.451 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:26.451 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:32:26.452 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.604856+0000", 2024-09-14T06:32:26.452 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:26.452 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:26.452 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:26.452 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:32:26.452 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.604856+0000", 2024-09-14T06:32:26.452 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:26.452 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:26.452 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:26.452 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:32:26.452 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.604856+0000", 2024-09-14T06:32:26.452 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.23e-07 2024-09-14T06:32:26.452 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:26.453 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:26.453 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:32:26.453 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.604857+0000", 2024-09-14T06:32:26.453 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.9999999999999995e-07 2024-09-14T06:32:26.453 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:26.453 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:26.453 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:32:26.453 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.604861+0000", 2024-09-14T06:32:26.453 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.9890000000000003e-06 2024-09-14T06:32:26.453 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:26.453 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:26.453 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:32:26.453 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.604864+0000", 2024-09-14T06:32:26.453 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.7429999999999998e-06 2024-09-14T06:32:26.454 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:26.454 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:26.454 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:32:26.454 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.646744+0000", 2024-09-14T06:32:26.454 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.041880325000000003 2024-09-14T06:32:26.454 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:26.454 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:26.454 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:32:26.454 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.646784+0000", 2024-09-14T06:32:26.454 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.9604000000000002e-05 2024-09-14T06:32:26.454 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:26.454 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:26.454 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:32:26.454 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.646842+0000", 2024-09-14T06:32:26.455 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.8343e-05 2024-09-14T06:32:26.455 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:26.455 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:32:26.455 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:26.455 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:26.455 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:26.455 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:55659 3.0 3:06490baf:::rbd_data.112dca2f6ae1.0000000000000030:17 [read 1736704~4096 [fadvise_nocache]] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:32:26.455 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:44.604867+0000", 2024-09-14T06:32:26.455 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 161.76873737599999, 2024-09-14T06:32:26.455 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.042097082000000001, 2024-09-14T06:32:26.455 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:32:26.455 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:32:26.455 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:32:26.455 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:32:26.456 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:32:26.456 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:26.456 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 55659 2024-09-14T06:32:26.456 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:26.456 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:32:26.456 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:26.456 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:32:26.456 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.604867+0000", 2024-09-14T06:32:26.456 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:26.456 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:26.456 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:26.456 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:32:26.456 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.604867+0000", 2024-09-14T06:32:26.456 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:26.457 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:26.457 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:26.457 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:32:26.457 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.604867+0000", 2024-09-14T06:32:26.457 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.2000000000000001e-07 2024-09-14T06:32:26.457 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:26.457 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:26.457 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:32:26.457 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.604868+0000", 2024-09-14T06:32:26.457 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.04e-06 2024-09-14T06:32:26.457 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:26.457 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:26.457 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:32:26.457 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.604871+0000", 2024-09-14T06:32:26.457 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.9689999999999999e-06 2024-09-14T06:32:26.458 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:26.458 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:26.458 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:32:26.458 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.604874+0000", 2024-09-14T06:32:26.458 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.5069999999999999e-06 2024-09-14T06:32:26.458 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:26.458 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:26.458 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:32:26.458 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.646874+0000", 2024-09-14T06:32:26.458 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.042000513000000003 2024-09-14T06:32:26.458 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:26.458 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:26.458 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:32:26.458 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.646914+0000", 2024-09-14T06:32:26.458 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.9443999999999997e-05 2024-09-14T06:32:26.459 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:26.459 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:26.459 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:32:26.459 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.646964+0000", 2024-09-14T06:32:26.459 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.0389000000000002e-05 2024-09-14T06:32:26.459 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:26.459 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:32:26.459 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:26.459 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:26.459 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:26.459 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:55660 3.0 3:06490baf:::rbd_data.112dca2f6ae1.0000000000000030:17 [read 1761280~4096 [fadvise_nocache]] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:32:26.459 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:44.604877+0000", 2024-09-14T06:32:26.459 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 161.768727627, 2024-09-14T06:32:26.460 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.042196349000000001, 2024-09-14T06:32:26.460 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:32:26.460 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:32:26.460 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:32:26.460 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:32:26.460 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:32:26.460 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:26.460 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 55660 2024-09-14T06:32:26.460 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:26.460 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:32:26.460 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:26.460 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:32:26.460 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.604877+0000", 2024-09-14T06:32:26.461 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:26.461 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:26.461 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:26.461 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:32:26.461 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.604877+0000", 2024-09-14T06:32:26.461 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:26.461 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:26.461 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:26.461 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:32:26.461 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.605810+0000", 2024-09-14T06:32:26.461 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00093336899999999997 2024-09-14T06:32:26.461 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:26.461 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:26.461 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:32:26.462 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.605815+0000", 2024-09-14T06:32:26.462 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.6140000000000002e-06 2024-09-14T06:32:26.462 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:26.462 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:26.462 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:32:26.462 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.605825+0000", 2024-09-14T06:32:26.462 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.8889999999999997e-06 2024-09-14T06:32:26.462 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:26.462 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:26.462 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:32:26.462 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.605831+0000", 2024-09-14T06:32:26.462 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.9800000000000003e-06 2024-09-14T06:32:26.462 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:26.463 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:26.463 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:32:26.463 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.646989+0000", 2024-09-14T06:32:26.463 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.041157878000000002 2024-09-14T06:32:26.463 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:26.463 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:26.463 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:32:26.463 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.647026+0000", 2024-09-14T06:32:26.463 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.7305000000000003e-05 2024-09-14T06:32:26.463 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:26.463 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:26.463 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:32:26.463 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.647073+0000", 2024-09-14T06:32:26.463 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.7314000000000001e-05 2024-09-14T06:32:26.464 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:26.464 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:32:26.464 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:26.464 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:26.464 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:26.464 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:55666 3.0 3:06490baf:::rbd_data.112dca2f6ae1.0000000000000030:17 [read 1908736~4096 [fadvise_nocache]] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:32:26.464 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:44.605898+0000", 2024-09-14T06:32:26.464 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 161.76770688100001, 2024-09-14T06:32:26.464 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.041862887000000001, 2024-09-14T06:32:26.464 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:32:26.464 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:32:26.464 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:32:26.464 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:32:26.464 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:32:26.465 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:26.465 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 55666 2024-09-14T06:32:26.465 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:26.465 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:32:26.465 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:26.465 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:32:26.465 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.605898+0000", 2024-09-14T06:32:26.465 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:26.465 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:26.465 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:26.465 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:32:26.465 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.605898+0000", 2024-09-14T06:32:26.465 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:26.465 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:26.466 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:26.466 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:32:26.466 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.611874+0000", 2024-09-14T06:32:26.466 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0059764049999999997 2024-09-14T06:32:26.466 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:26.466 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:26.466 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:32:26.466 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.611879+0000", 2024-09-14T06:32:26.466 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.8749999999999999e-06 2024-09-14T06:32:26.466 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:26.466 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:26.466 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:32:26.466 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.611887+0000", 2024-09-14T06:32:26.467 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.4109999999999996e-06 2024-09-14T06:32:26.467 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:26.467 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:26.467 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:32:26.467 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.611893+0000", 2024-09-14T06:32:26.467 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.2959999999999998e-06 2024-09-14T06:32:26.467 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:26.467 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:26.467 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:32:26.467 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.647674+0000", 2024-09-14T06:32:26.467 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.035781281999999998 2024-09-14T06:32:26.467 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:26.467 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:26.468 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:32:26.468 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.647711+0000", 2024-09-14T06:32:26.468 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.7434000000000001e-05 2024-09-14T06:32:26.468 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:26.468 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:26.468 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:32:26.468 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.647760+0000", 2024-09-14T06:32:26.468 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.9184000000000002e-05 2024-09-14T06:32:26.468 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:26.468 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:32:26.468 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:26.468 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:26.468 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:32:26.468 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-14T06:32:26.470 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-14T06:32:26.471 INFO:tasks.ceph.osd.3.smithi094.stderr:2024-09-14T06:32:26.470+0000 7feeeafe8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33655) UID: 0 2024-09-14T06:32:26.572 INFO:tasks.ceph.osd.7.smithi138.stderr:2024-09-14T06:32:26.571+0000 7f3c6a21d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33623) UID: 0 2024-09-14T06:32:26.572 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-14T06:32:26.573 INFO:teuthology.orchestra.run.smithi094.stdout: "ops": [], 2024-09-14T06:32:26.573 INFO:teuthology.orchestra.run.smithi094.stdout: "num_ops": 0 2024-09-14T06:32:26.573 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-14T06:32:26.581 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-14T06:32:26.631 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-14T06:32:26.675 INFO:tasks.ceph.osd.3.smithi094.stderr:2024-09-14T06:32:26.674+0000 7feeeafe8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33655) UID: 0 2024-09-14T06:32:26.685 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-14T06:32:26.686 INFO:teuthology.orchestra.run.smithi094.stdout: "ops": [], 2024-09-14T06:32:26.686 INFO:teuthology.orchestra.run.smithi094.stdout: "complaint_time": 30, 2024-09-14T06:32:26.686 INFO:teuthology.orchestra.run.smithi094.stdout: "num_blocked_ops": 0 2024-09-14T06:32:26.833 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-14T06:32:26.833 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-14T06:32:26.835 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-14T06:32:26.835 INFO:teuthology.orchestra.run.smithi094.stdout:osd.0: {} 2024-09-14T06:32:26.836 INFO:teuthology.orchestra.run.smithi094.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-14T06:32:26.836 INFO:teuthology.orchestra.run.smithi094.stdout:osd.1: {} 2024-09-14T06:32:26.836 INFO:teuthology.orchestra.run.smithi094.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-14T06:32:26.836 INFO:teuthology.orchestra.run.smithi094.stdout:osd.2: {} 2024-09-14T06:32:26.837 INFO:teuthology.orchestra.run.smithi094.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-14T06:32:26.837 INFO:teuthology.orchestra.run.smithi094.stdout:osd.3: {} 2024-09-14T06:32:26.837 INFO:teuthology.orchestra.run.smithi094.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-14T06:32:26.838 INFO:tasks.ceph.osd.7.smithi138.stderr:2024-09-14T06:32:26.833+0000 7f3c6a21d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33623) UID: 0 2024-09-14T06:32:26.838 INFO:teuthology.orchestra.run.smithi094.stdout:osd.4: {} 2024-09-14T06:32:26.838 INFO:teuthology.orchestra.run.smithi094.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-14T06:32:26.855 INFO:teuthology.orchestra.run.smithi094.stdout:osd.5: {} 2024-09-14T06:32:26.855 INFO:teuthology.orchestra.run.smithi094.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-14T06:32:26.873 INFO:teuthology.orchestra.run.smithi094.stdout:osd.6: {} 2024-09-14T06:32:26.873 INFO:teuthology.orchestra.run.smithi094.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-14T06:32:26.890 INFO:teuthology.orchestra.run.smithi094.stdout:osd.7: {} 2024-09-14T06:32:26.890 INFO:teuthology.orchestra.run.smithi094.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-14T06:32:26.943 INFO:tasks.ceph.osd.1.smithi094.stderr:2024-09-14T06:32:26.942+0000 7f328e47a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33641) UID: 0 2024-09-14T06:32:26.944 INFO:teuthology.orchestra.run.smithi094.stdout:ERROR: (22) Invalid argument 2024-09-14T06:32:26.944 INFO:teuthology.orchestra.run.smithi094.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:32:26.955 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-14T06:32:27.040 INFO:tasks.ceph.osd.1.smithi094.stderr:2024-09-14T06:32:27.039+0000 7f328e47a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33641) UID: 0 2024-09-14T06:32:27.101 INFO:teuthology.orchestra.run.smithi094.stdout:ERROR: (22) Invalid argument 2024-09-14T06:32:27.101 INFO:teuthology.orchestra.run.smithi094.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:32:27.109 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-14T06:32:27.151 INFO:tasks.ceph.osd.3.smithi094.stderr:2024-09-14T06:32:27.150+0000 7feeeafe8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33655) UID: 0 2024-09-14T06:32:27.242 INFO:tasks.ceph.osd.6.smithi138.stderr:2024-09-14T06:32:27.241+0000 7f8cd8849640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33620) UID: 0 2024-09-14T06:32:27.252 INFO:teuthology.orchestra.run.smithi094.stdout:ERROR: (22) Invalid argument 2024-09-14T06:32:27.252 INFO:teuthology.orchestra.run.smithi094.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:32:27.261 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-14T06:32:27.345 INFO:tasks.ceph.osd.1.smithi094.stderr:2024-09-14T06:32:27.344+0000 7f328e47a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33641) UID: 0 2024-09-14T06:32:27.364 INFO:teuthology.orchestra.run.smithi094.stdout:ERROR: (22) Invalid argument 2024-09-14T06:32:27.364 INFO:teuthology.orchestra.run.smithi094.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:32:27.373 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-14T06:32:27.446 INFO:tasks.ceph.osd.1.smithi094.stderr:2024-09-14T06:32:27.446+0000 7f328e47a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33641) UID: 0 2024-09-14T06:32:27.488 INFO:teuthology.orchestra.run.smithi138.stdout:ERROR: (22) Invalid argument 2024-09-14T06:32:27.488 INFO:teuthology.orchestra.run.smithi138.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:32:27.498 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-14T06:32:27.549 INFO:tasks.ceph.osd.2.smithi094.stderr:2024-09-14T06:32:27.548+0000 7faf3269b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33649) UID: 0 2024-09-14T06:32:27.600 INFO:teuthology.orchestra.run.smithi138.stdout:ERROR: (22) Invalid argument 2024-09-14T06:32:27.601 INFO:teuthology.orchestra.run.smithi138.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:32:27.609 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-14T06:32:27.652 INFO:tasks.ceph.osd.7.smithi138.stderr:2024-09-14T06:32:27.651+0000 7f3c6a21d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33623) UID: 0 2024-09-14T06:32:27.712 INFO:teuthology.orchestra.run.smithi138.stdout:ERROR: (22) Invalid argument 2024-09-14T06:32:27.713 INFO:teuthology.orchestra.run.smithi138.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:32:27.722 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-14T06:32:27.727 INFO:teuthology.orchestra.run.smithi094.stderr:nodeep-scrub is set 2024-09-14T06:32:27.763 INFO:tasks.ceph.osd.4.smithi138.stderr:2024-09-14T06:32:27.762+0000 7f6c653c8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33615) UID: 0 2024-09-14T06:32:27.852 INFO:tasks.ceph.osd.3.smithi094.stderr:2024-09-14T06:32:27.851+0000 7feeeafe8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33655) UID: 0 2024-09-14T06:32:27.874 INFO:teuthology.orchestra.run.smithi138.stdout:ERROR: (22) Invalid argument 2024-09-14T06:32:27.874 INFO:teuthology.orchestra.run.smithi138.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:32:27.883 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-14T06:32:27.953 INFO:tasks.ceph.osd.3.smithi094.stderr:2024-09-14T06:32:27.952+0000 7feeeafe8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33655) UID: 0 2024-09-14T06:32:27.985 INFO:teuthology.orchestra.run.smithi138.stdout:ERROR: (22) Invalid argument 2024-09-14T06:32:27.985 INFO:teuthology.orchestra.run.smithi138.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:32:27.994 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-14T06:32:28.054 INFO:tasks.ceph.osd.1.smithi094.stderr:2024-09-14T06:32:28.053+0000 7f328e47a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33641) UID: 0 2024-09-14T06:32:28.096 INFO:teuthology.orchestra.run.smithi138.stdout:ERROR: (22) Invalid argument 2024-09-14T06:32:28.096 INFO:teuthology.orchestra.run.smithi138.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:32:28.104 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-14T06:32:28.154 INFO:tasks.ceph.osd.6.smithi138.stderr:2024-09-14T06:32:28.154+0000 7f8cd8849640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33620) UID: 0 2024-09-14T06:32:28.206 INFO:teuthology.orchestra.run.smithi138.stdout:ERROR: (22) Invalid argument 2024-09-14T06:32:28.206 INFO:teuthology.orchestra.run.smithi138.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:32:28.214 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-14T06:32:28.255 INFO:tasks.ceph.osd.4.smithi138.stderr:2024-09-14T06:32:28.254+0000 7f6c653c8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33615) UID: 0 2024-09-14T06:32:28.356 INFO:tasks.ceph.osd.7.smithi138.stderr:2024-09-14T06:32:28.355+0000 7f3c6a21d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33623) UID: 0 2024-09-14T06:32:28.357 INFO:teuthology.orchestra.run.smithi138.stdout:ERROR: (22) Invalid argument 2024-09-14T06:32:28.357 INFO:teuthology.orchestra.run.smithi138.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:32:28.366 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-14T06:32:28.457 INFO:tasks.ceph.osd.7.smithi138.stderr:2024-09-14T06:32:28.456+0000 7f3c6a21d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33623) UID: 0 2024-09-14T06:32:28.509 INFO:teuthology.orchestra.run.smithi138.stdout:ERROR: (22) Invalid argument 2024-09-14T06:32:28.509 INFO:teuthology.orchestra.run.smithi138.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:32:28.518 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-14T06:32:28.559 INFO:tasks.ceph.osd.3.smithi094.stderr:2024-09-14T06:32:28.558+0000 7feeeafe8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33655) UID: 0 2024-09-14T06:32:28.659 INFO:tasks.ceph.osd.4.smithi138.stderr:2024-09-14T06:32:28.658+0000 7f6c653c8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33615) UID: 0 2024-09-14T06:32:28.662 INFO:teuthology.orchestra.run.smithi138.stdout:ERROR: (22) Invalid argument 2024-09-14T06:32:28.662 INFO:teuthology.orchestra.run.smithi138.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:32:28.671 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-14T06:32:28.727 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 4, 5, 7, 3] out_osds: [6] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-14T06:32:28.728 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-14T06:32:28.728 INFO:tasks.thrashosds.thrasher:trim_stale_osdmaps 2024-09-14T06:32:28.728 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 'trim stale osdmaps' 2024-09-14T06:32:28.760 INFO:tasks.ceph.osd.5.smithi138.stderr:2024-09-14T06:32:28.759+0000 7fe13746f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33618) UID: 0 2024-09-14T06:32:28.821 INFO:teuthology.orchestra.run.smithi138.stdout:ERROR: (22) Invalid argument 2024-09-14T06:32:28.821 INFO:teuthology.orchestra.run.smithi138.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:32:28.831 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-14T06:32:28.861 INFO:tasks.ceph.osd.1.smithi094.stderr:2024-09-14T06:32:28.860+0000 7f328e47a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33641) UID: 0 2024-09-14T06:32:28.883 INFO:teuthology.orchestra.run.smithi094.stderr:no valid command found; 10 closest matches: 2024-09-14T06:32:28.883 INFO:teuthology.orchestra.run.smithi094.stderr:0 2024-09-14T06:32:28.883 INFO:teuthology.orchestra.run.smithi094.stderr:1 2024-09-14T06:32:28.883 INFO:teuthology.orchestra.run.smithi094.stderr:2 2024-09-14T06:32:28.883 INFO:teuthology.orchestra.run.smithi094.stderr:abort 2024-09-14T06:32:28.883 INFO:teuthology.orchestra.run.smithi094.stderr:assert 2024-09-14T06:32:28.884 INFO:teuthology.orchestra.run.smithi094.stderr:bench [] [] [] [] 2024-09-14T06:32:28.884 INFO:teuthology.orchestra.run.smithi094.stderr:bluefs debug_inject_read_zeros 2024-09-14T06:32:28.884 INFO:teuthology.orchestra.run.smithi094.stderr:bluefs files list 2024-09-14T06:32:28.884 INFO:teuthology.orchestra.run.smithi094.stderr:bluefs stats 2024-09-14T06:32:28.884 INFO:teuthology.orchestra.run.smithi094.stderr:bluestore allocator dump block 2024-09-14T06:32:28.884 INFO:teuthology.orchestra.run.smithi094.stderr:Error EINVAL: invalid command 2024-09-14T06:32:28.886 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-14T06:32:28.887 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-14T06:32:28.887 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 'trim stale osdmaps' 2024-09-14T06:32:28.902 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-14T06:32:28.971 INFO:tasks.ceph.osd.2.smithi094.stderr:2024-09-14T06:32:28.970+0000 7faf3269b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33649) UID: 0 2024-09-14T06:32:28.974 INFO:teuthology.orchestra.run.smithi138.stdout:ERROR: (22) Invalid argument 2024-09-14T06:32:28.974 INFO:teuthology.orchestra.run.smithi138.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:32:28.982 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-14T06:32:29.063 INFO:teuthology.orchestra.run.smithi094.stderr:no valid command found; 10 closest matches: 2024-09-14T06:32:29.063 INFO:teuthology.orchestra.run.smithi094.stderr:0 2024-09-14T06:32:29.063 INFO:teuthology.orchestra.run.smithi094.stderr:1 2024-09-14T06:32:29.063 INFO:teuthology.orchestra.run.smithi094.stderr:2 2024-09-14T06:32:29.063 INFO:teuthology.orchestra.run.smithi094.stderr:abort 2024-09-14T06:32:29.063 INFO:teuthology.orchestra.run.smithi094.stderr:assert 2024-09-14T06:32:29.063 INFO:teuthology.orchestra.run.smithi094.stderr:bench [] [] [] [] 2024-09-14T06:32:29.063 INFO:teuthology.orchestra.run.smithi094.stderr:bluefs debug_inject_read_zeros 2024-09-14T06:32:29.063 INFO:teuthology.orchestra.run.smithi094.stderr:bluefs files list 2024-09-14T06:32:29.063 INFO:teuthology.orchestra.run.smithi094.stderr:bluefs stats 2024-09-14T06:32:29.063 INFO:teuthology.orchestra.run.smithi094.stderr:bluestore allocator dump block 2024-09-14T06:32:29.064 INFO:teuthology.orchestra.run.smithi094.stderr:Error EINVAL: invalid command 2024-09-14T06:32:29.064 INFO:tasks.ceph.osd.5.smithi138.stderr:2024-09-14T06:32:29.062+0000 7fe13746f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33618) UID: 0 2024-09-14T06:32:29.066 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-14T06:32:29.066 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-14T06:32:29.067 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 'trim stale osdmaps' 2024-09-14T06:32:29.067 INFO:teuthology.orchestra.run.smithi094.stdout:osd.0: {} 2024-09-14T06:32:29.067 INFO:teuthology.orchestra.run.smithi094.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-14T06:32:29.089 INFO:teuthology.orchestra.run.smithi094.stdout:osd.1: {} 2024-09-14T06:32:29.089 INFO:teuthology.orchestra.run.smithi094.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-14T06:32:29.106 INFO:teuthology.orchestra.run.smithi094.stdout:osd.2: {} 2024-09-14T06:32:29.106 INFO:teuthology.orchestra.run.smithi094.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-14T06:32:29.107 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-14T06:32:29.107 INFO:teuthology.orchestra.run.smithi094.stdout: "ops": [], 2024-09-14T06:32:29.107 INFO:teuthology.orchestra.run.smithi094.stdout: "num_ops": 0 2024-09-14T06:32:29.107 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-14T06:32:29.116 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-14T06:32:29.122 INFO:teuthology.orchestra.run.smithi094.stdout:osd.3: {} 2024-09-14T06:32:29.123 INFO:teuthology.orchestra.run.smithi094.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-14T06:32:29.140 INFO:teuthology.orchestra.run.smithi094.stdout:osd.4: {} 2024-09-14T06:32:29.141 INFO:teuthology.orchestra.run.smithi094.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-14T06:32:29.158 INFO:teuthology.orchestra.run.smithi094.stdout:osd.5: {} 2024-09-14T06:32:29.158 INFO:teuthology.orchestra.run.smithi094.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-14T06:32:29.163 INFO:tasks.ceph.osd.3.smithi094.stderr:2024-09-14T06:32:29.162+0000 7feeeafe8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33655) UID: 0 2024-09-14T06:32:29.176 INFO:teuthology.orchestra.run.smithi094.stdout:osd.6: {} 2024-09-14T06:32:29.176 INFO:teuthology.orchestra.run.smithi094.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-14T06:32:29.193 INFO:teuthology.orchestra.run.smithi094.stdout:osd.7: {} 2024-09-14T06:32:29.193 INFO:teuthology.orchestra.run.smithi094.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-14T06:32:29.204 INFO:teuthology.orchestra.run.smithi094.stderr:no valid command found; 10 closest matches: 2024-09-14T06:32:29.204 INFO:teuthology.orchestra.run.smithi094.stderr:0 2024-09-14T06:32:29.204 INFO:teuthology.orchestra.run.smithi094.stderr:1 2024-09-14T06:32:29.205 INFO:teuthology.orchestra.run.smithi094.stderr:2 2024-09-14T06:32:29.205 INFO:teuthology.orchestra.run.smithi094.stderr:abort 2024-09-14T06:32:29.205 INFO:teuthology.orchestra.run.smithi094.stderr:assert 2024-09-14T06:32:29.205 INFO:teuthology.orchestra.run.smithi094.stderr:bench [] [] [] [] 2024-09-14T06:32:29.205 INFO:teuthology.orchestra.run.smithi094.stderr:bluefs debug_inject_read_zeros 2024-09-14T06:32:29.205 INFO:teuthology.orchestra.run.smithi094.stderr:bluefs files list 2024-09-14T06:32:29.205 INFO:teuthology.orchestra.run.smithi094.stderr:bluefs stats 2024-09-14T06:32:29.205 INFO:teuthology.orchestra.run.smithi094.stderr:bluestore allocator dump block 2024-09-14T06:32:29.205 INFO:teuthology.orchestra.run.smithi094.stderr:Error EINVAL: invalid command 2024-09-14T06:32:29.208 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-14T06:32:29.208 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-14T06:32:29.209 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 'trim stale osdmaps' 2024-09-14T06:32:29.231 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-14T06:32:29.231 INFO:teuthology.orchestra.run.smithi094.stdout: "ops": [], 2024-09-14T06:32:29.231 INFO:teuthology.orchestra.run.smithi094.stdout: "complaint_time": 30, 2024-09-14T06:32:29.232 INFO:teuthology.orchestra.run.smithi094.stdout: "num_blocked_ops": 0 2024-09-14T06:32:29.232 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-14T06:32:29.240 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-14T06:32:29.264 INFO:tasks.ceph.osd.7.smithi138.stderr:2024-09-14T06:32:29.263+0000 7f3c6a21d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33623) UID: 0 2024-09-14T06:32:29.366 INFO:tasks.ceph.osd.2.smithi094.stderr:2024-09-14T06:32:29.365+0000 7faf3269b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33649) UID: 0 2024-09-14T06:32:29.392 INFO:teuthology.orchestra.run.smithi094.stderr:no valid command found; 10 closest matches: 2024-09-14T06:32:29.392 INFO:teuthology.orchestra.run.smithi094.stderr:0 2024-09-14T06:32:29.392 INFO:teuthology.orchestra.run.smithi094.stderr:1 2024-09-14T06:32:29.392 INFO:teuthology.orchestra.run.smithi094.stderr:2 2024-09-14T06:32:29.392 INFO:teuthology.orchestra.run.smithi094.stderr:abort 2024-09-14T06:32:29.392 INFO:teuthology.orchestra.run.smithi094.stderr:assert 2024-09-14T06:32:29.392 INFO:teuthology.orchestra.run.smithi094.stderr:bench [] [] [] [] 2024-09-14T06:32:29.392 INFO:teuthology.orchestra.run.smithi094.stderr:bluefs debug_inject_read_zeros 2024-09-14T06:32:29.392 INFO:teuthology.orchestra.run.smithi094.stderr:bluefs files list 2024-09-14T06:32:29.392 INFO:teuthology.orchestra.run.smithi094.stderr:bluefs stats 2024-09-14T06:32:29.392 INFO:teuthology.orchestra.run.smithi094.stderr:bluestore allocator dump block 2024-09-14T06:32:29.392 INFO:teuthology.orchestra.run.smithi094.stderr:Error EINVAL: invalid command 2024-09-14T06:32:29.395 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-14T06:32:29.395 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-14T06:32:29.395 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 'trim stale osdmaps' 2024-09-14T06:32:29.407 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-14T06:32:29.407 INFO:teuthology.orchestra.run.smithi094.stdout: "size": 20, 2024-09-14T06:32:29.408 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 600, 2024-09-14T06:32:29.408 INFO:teuthology.orchestra.run.smithi094.stdout: "ops": [ 2024-09-14T06:32:29.408 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:29.408 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:115193 3.1 3:856be886:::rbd_data.112d4d24c558.0000000000000004:29 [read 1523712~16384 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e96)", 2024-09-14T06:32:29.408 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:30:17.939854+0000", 2024-09-14T06:32:29.408 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 131.465858126, 2024-09-14T06:32:29.408 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.073897516999999996, 2024-09-14T06:32:29.408 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:32:29.408 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:32:29.408 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:32:29.408 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:32:29.408 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:32:29.408 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:29.408 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 115193 2024-09-14T06:32:29.409 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:29.409 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:32:29.409 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:29.409 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:32:29.409 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.939854+0000", 2024-09-14T06:32:29.409 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:29.409 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:29.409 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:29.409 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:32:29.409 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.939854+0000", 2024-09-14T06:32:29.409 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:29.409 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:29.409 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:29.409 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:32:29.409 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.939854+0000", 2024-09-14T06:32:29.410 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.9500000000000002e-07 2024-09-14T06:32:29.410 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:29.410 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:29.410 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:32:29.410 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.939856+0000", 2024-09-14T06:32:29.410 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.0959999999999999e-06 2024-09-14T06:32:29.410 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:29.410 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:29.410 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:32:29.410 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.939863+0000", 2024-09-14T06:32:29.410 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.1129999999999997e-06 2024-09-14T06:32:29.410 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:29.410 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:29.410 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:32:29.411 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.939867+0000", 2024-09-14T06:32:29.411 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.8350000000000003e-06 2024-09-14T06:32:29.411 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:29.411 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:29.411 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:32:29.411 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.013626+0000", 2024-09-14T06:32:29.411 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.073758799999999999 2024-09-14T06:32:29.411 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:29.411 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:29.411 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:32:29.411 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.013690+0000", 2024-09-14T06:32:29.411 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.3918999999999996e-05 2024-09-14T06:32:29.411 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:29.411 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:29.411 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:32:29.411 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.013751+0000", 2024-09-14T06:32:29.412 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.1159000000000002e-05 2024-09-14T06:32:29.412 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:29.412 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:32:29.412 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:29.412 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:29.412 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:29.412 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:115194 3.1 3:856be886:::rbd_data.112d4d24c558.0000000000000004:29 [read 1769472~16384 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e96)", 2024-09-14T06:32:29.412 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:30:17.939874+0000", 2024-09-14T06:32:29.412 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 131.46583781499999, 2024-09-14T06:32:29.412 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.074016988000000006, 2024-09-14T06:32:29.412 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:32:29.412 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:32:29.412 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:32:29.412 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:32:29.413 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:32:29.413 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:29.413 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 115194 2024-09-14T06:32:29.413 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:29.413 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:32:29.413 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:29.413 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:32:29.413 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.939874+0000", 2024-09-14T06:32:29.413 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:29.413 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:29.413 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:29.413 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:32:29.414 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.939874+0000", 2024-09-14T06:32:29.414 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:29.414 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:29.414 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:29.414 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:32:29.414 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.939875+0000", 2024-09-14T06:32:29.414 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.2600000000000002e-07 2024-09-14T06:32:29.414 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:29.414 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:29.414 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:32:29.415 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.939877+0000", 2024-09-14T06:32:29.415 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.9420000000000002e-06 2024-09-14T06:32:29.415 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:29.415 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:29.415 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:32:29.415 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.939882+0000", 2024-09-14T06:32:29.415 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.8749999999999997e-06 2024-09-14T06:32:29.415 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:29.415 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:29.415 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:32:29.415 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.939887+0000", 2024-09-14T06:32:29.415 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.741e-06 2024-09-14T06:32:29.415 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:29.415 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:29.416 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:32:29.416 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.013767+0000", 2024-09-14T06:32:29.416 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.073879720999999995 2024-09-14T06:32:29.416 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:29.416 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:29.416 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:32:29.416 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.013832+0000", 2024-09-14T06:32:29.416 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.5312000000000001e-05 2024-09-14T06:32:29.416 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:29.416 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:29.416 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:32:29.416 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.013891+0000", 2024-09-14T06:32:29.416 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.8870999999999999e-05 2024-09-14T06:32:29.417 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:29.417 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:32:29.417 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:29.417 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:29.417 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:29.417 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:115195 3.1 3:856be886:::rbd_data.112d4d24c558.0000000000000004:29 [read 2015232~16384 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e96)", 2024-09-14T06:32:29.417 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:30:17.939903+0000", 2024-09-14T06:32:29.417 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 131.46580863, 2024-09-14T06:32:29.417 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.074128199000000006, 2024-09-14T06:32:29.417 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:32:29.417 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:32:29.417 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:32:29.417 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:32:29.417 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:32:29.418 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:29.418 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 115195 2024-09-14T06:32:29.418 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:29.418 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:32:29.418 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:29.418 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:32:29.418 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.939903+0000", 2024-09-14T06:32:29.418 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:29.418 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:29.418 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:29.418 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:32:29.418 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.939903+0000", 2024-09-14T06:32:29.418 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:29.418 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:29.419 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:29.419 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:32:29.419 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.939904+0000", 2024-09-14T06:32:29.419 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.5799999999999999e-07 2024-09-14T06:32:29.419 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:29.419 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:29.419 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:32:29.419 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.939906+0000", 2024-09-14T06:32:29.419 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.2350000000000002e-06 2024-09-14T06:32:29.419 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:29.419 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:29.419 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:32:29.419 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.939913+0000", 2024-09-14T06:32:29.419 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.5649999999999998e-06 2024-09-14T06:32:29.419 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:29.419 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:29.420 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:32:29.420 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.939918+0000", 2024-09-14T06:32:29.420 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.784e-06 2024-09-14T06:32:29.420 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:29.420 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:29.420 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:32:29.420 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.013921+0000", 2024-09-14T06:32:29.420 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.074002911000000005 2024-09-14T06:32:29.420 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:29.420 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:29.420 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:32:29.420 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.013971+0000", 2024-09-14T06:32:29.420 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.9551e-05 2024-09-14T06:32:29.420 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:29.420 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:29.421 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:32:29.421 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.014031+0000", 2024-09-14T06:32:29.421 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.0594999999999998e-05 2024-09-14T06:32:29.421 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:29.421 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:32:29.421 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:29.421 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:29.421 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:29.421 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:115196 3.1 3:856be886:::rbd_data.112d4d24c558.0000000000000004:29 [read 2260992~16384 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e96)", 2024-09-14T06:32:29.421 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:30:17.939925+0000", 2024-09-14T06:32:29.421 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 131.46578643800001, 2024-09-14T06:32:29.421 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.074320137999999994, 2024-09-14T06:32:29.422 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:32:29.422 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:32:29.422 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:32:29.422 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:32:29.422 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:32:29.422 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:29.422 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 115196 2024-09-14T06:32:29.422 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:29.422 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:32:29.422 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:29.422 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:32:29.422 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.939925+0000", 2024-09-14T06:32:29.422 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:29.423 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:29.423 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:29.423 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:32:29.423 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.939925+0000", 2024-09-14T06:32:29.423 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:29.423 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:29.423 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:29.423 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:32:29.423 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.939926+0000", 2024-09-14T06:32:29.423 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.9999999999999998e-07 2024-09-14T06:32:29.423 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:29.423 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:29.423 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:32:29.423 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.939928+0000", 2024-09-14T06:32:29.423 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.9549999999999999e-06 2024-09-14T06:32:29.424 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:29.424 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:29.424 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:32:29.424 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.939934+0000", 2024-09-14T06:32:29.424 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.127e-06 2024-09-14T06:32:29.424 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:29.424 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:29.424 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:32:29.424 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.939939+0000", 2024-09-14T06:32:29.424 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.775e-06 2024-09-14T06:32:29.424 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:29.425 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:29.425 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:32:29.425 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.014075+0000", 2024-09-14T06:32:29.425 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.074136598999999997 2024-09-14T06:32:29.425 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:29.425 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:29.425 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:32:29.425 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.014162+0000", 2024-09-14T06:32:29.425 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.6477999999999994e-05 2024-09-14T06:32:29.425 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:29.426 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:29.426 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:32:29.426 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.014246+0000", 2024-09-14T06:32:29.426 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.3703999999999997e-05 2024-09-14T06:32:29.426 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:29.426 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:32:29.426 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:29.426 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:29.426 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:29.426 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:115197 3.1 3:856be886:::rbd_data.112d4d24c558.0000000000000004:29 [read 2506752~16384 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e96)", 2024-09-14T06:32:29.426 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:30:17.939969+0000", 2024-09-14T06:32:29.426 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 131.46574295299999, 2024-09-14T06:32:29.426 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.074528544000000002, 2024-09-14T06:32:29.426 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:32:29.426 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:32:29.427 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:32:29.427 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:32:29.427 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:32:29.427 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:29.427 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 115197 2024-09-14T06:32:29.427 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:29.427 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:32:29.427 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:29.427 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:32:29.428 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.939969+0000", 2024-09-14T06:32:29.428 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:29.428 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:29.428 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:29.428 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:32:29.428 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.939969+0000", 2024-09-14T06:32:29.428 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:29.428 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:29.428 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:29.428 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:32:29.429 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.939970+0000", 2024-09-14T06:32:29.429 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.3600000000000003e-07 2024-09-14T06:32:29.429 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:29.429 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:29.429 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:32:29.429 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.939972+0000", 2024-09-14T06:32:29.429 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.3180000000000001e-06 2024-09-14T06:32:29.429 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:29.430 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:29.430 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:32:29.430 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.939979+0000", 2024-09-14T06:32:29.430 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.8800000000000002e-06 2024-09-14T06:32:29.430 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:29.430 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:29.430 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:32:29.431 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.939984+0000", 2024-09-14T06:32:29.431 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.3510000000000001e-06 2024-09-14T06:32:29.431 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:29.431 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:29.431 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:32:29.431 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.014302+0000", 2024-09-14T06:32:29.431 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.074317416999999997 2024-09-14T06:32:29.431 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:29.431 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:29.431 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:32:29.431 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.014392+0000", 2024-09-14T06:32:29.432 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.0181999999999998e-05 2024-09-14T06:32:29.432 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:29.432 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:29.432 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:32:29.432 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.014497+0000", 2024-09-14T06:32:29.432 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00010576 2024-09-14T06:32:29.432 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:29.432 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:32:29.432 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:29.433 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:29.433 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:29.433 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:115198 3.1 3:856be886:::rbd_data.112d4d24c558.0000000000000004:29 [read 2752512~16384 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e96)", 2024-09-14T06:32:29.433 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:30:17.940004+0000", 2024-09-14T06:32:29.433 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 131.465707673, 2024-09-14T06:32:29.433 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.074736367999999997, 2024-09-14T06:32:29.433 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:32:29.433 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:32:29.433 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:32:29.433 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:32:29.433 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:32:29.434 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:29.434 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 115198 2024-09-14T06:32:29.434 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:29.434 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:32:29.434 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:29.434 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:32:29.434 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940004+0000", 2024-09-14T06:32:29.434 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:29.434 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:29.434 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:29.435 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:32:29.435 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940004+0000", 2024-09-14T06:32:29.435 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:29.435 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:29.436 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:29.436 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:32:29.436 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940005+0000", 2024-09-14T06:32:29.436 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.2200000000000004e-07 2024-09-14T06:32:29.436 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:29.436 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:29.436 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:32:29.436 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940007+0000", 2024-09-14T06:32:29.436 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.322e-06 2024-09-14T06:32:29.436 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:29.436 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:29.436 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:32:29.437 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940014+0000", 2024-09-14T06:32:29.437 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.6220000000000003e-06 2024-09-14T06:32:29.437 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:29.437 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:29.437 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:32:29.437 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940019+0000", 2024-09-14T06:32:29.437 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.6380000000000001e-06 2024-09-14T06:32:29.437 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:29.437 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:29.437 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:32:29.437 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.014562+0000", 2024-09-14T06:32:29.438 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.074542320999999995 2024-09-14T06:32:29.438 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:29.438 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:29.438 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:32:29.438 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.014646+0000", 2024-09-14T06:32:29.438 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.4079999999999995e-05 2024-09-14T06:32:29.438 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:29.438 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:29.438 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:32:29.438 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.014741+0000", 2024-09-14T06:32:29.438 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.4763000000000001e-05 2024-09-14T06:32:29.439 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:29.439 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:32:29.439 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:29.439 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:29.439 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:29.439 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:115199 3.1 3:856be886:::rbd_data.112d4d24c558.0000000000000004:29 [read 2998272~16384 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e96)", 2024-09-14T06:32:29.439 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:30:17.940040+0000", 2024-09-14T06:32:29.439 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 131.465671918, 2024-09-14T06:32:29.439 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.074957134999999994, 2024-09-14T06:32:29.439 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:32:29.440 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:32:29.440 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:32:29.440 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:32:29.440 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:32:29.440 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:29.440 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 115199 2024-09-14T06:32:29.440 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:29.440 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:32:29.440 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:29.440 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:32:29.440 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940040+0000", 2024-09-14T06:32:29.440 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:29.441 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:29.441 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:29.441 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:32:29.441 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940040+0000", 2024-09-14T06:32:29.441 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:29.441 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:29.441 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:29.441 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:32:29.441 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940041+0000", 2024-09-14T06:32:29.441 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.4000000000000001e-07 2024-09-14T06:32:29.441 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:29.441 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:29.441 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:32:29.441 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940046+0000", 2024-09-14T06:32:29.442 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.7250000000000002e-06 2024-09-14T06:32:29.442 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:29.442 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:29.442 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:32:29.442 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940053+0000", 2024-09-14T06:32:29.442 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.6109999999999999e-06 2024-09-14T06:32:29.442 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:29.442 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:29.442 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:32:29.442 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940058+0000", 2024-09-14T06:32:29.442 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.1590000000000001e-06 2024-09-14T06:32:29.443 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:29.443 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:29.443 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:32:29.443 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.014768+0000", 2024-09-14T06:32:29.443 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.074709624000000002 2024-09-14T06:32:29.443 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:29.443 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:29.443 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:32:29.443 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.014907+0000", 2024-09-14T06:32:29.443 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000139665 2024-09-14T06:32:29.443 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:29.443 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:29.443 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:32:29.443 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.014997+0000", 2024-09-14T06:32:29.444 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.9710999999999995e-05 2024-09-14T06:32:29.444 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:29.444 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:32:29.444 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:29.444 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:29.444 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:29.444 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:115200 3.1 3:856be886:::rbd_data.112d4d24c558.0000000000000004:29 [read 3244032~16384 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e96)", 2024-09-14T06:32:29.444 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:30:17.940080+0000", 2024-09-14T06:32:29.444 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 131.46563171299999, 2024-09-14T06:32:29.444 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.075137291999999994, 2024-09-14T06:32:29.444 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:32:29.444 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:32:29.444 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:32:29.445 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:32:29.445 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:32:29.445 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:29.445 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 115200 2024-09-14T06:32:29.445 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:29.445 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:32:29.445 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:29.445 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:32:29.445 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940080+0000", 2024-09-14T06:32:29.445 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:29.445 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:29.445 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:29.445 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:32:29.446 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940080+0000", 2024-09-14T06:32:29.446 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:29.446 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:29.446 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:29.446 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:32:29.446 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940081+0000", 2024-09-14T06:32:29.446 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.0800000000000004e-07 2024-09-14T06:32:29.446 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:29.446 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:29.447 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:32:29.447 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940083+0000", 2024-09-14T06:32:29.447 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.3920000000000001e-06 2024-09-14T06:32:29.447 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:29.447 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:29.447 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:32:29.447 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940091+0000", 2024-09-14T06:32:29.447 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.1389999999999995e-06 2024-09-14T06:32:29.447 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:29.447 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:29.447 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:32:29.447 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940097+0000", 2024-09-14T06:32:29.448 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.5550000000000002e-06 2024-09-14T06:32:29.448 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:29.448 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:29.448 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:32:29.448 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.015025+0000", 2024-09-14T06:32:29.448 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.074928579999999995 2024-09-14T06:32:29.448 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:29.448 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:29.448 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:32:29.448 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.015096+0000", 2024-09-14T06:32:29.448 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.0705999999999997e-05 2024-09-14T06:32:29.448 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:29.449 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:29.449 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:32:29.449 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.015217+0000", 2024-09-14T06:32:29.449 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00012131200000000001 2024-09-14T06:32:29.449 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:29.449 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:32:29.449 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:29.449 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:29.449 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:29.449 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:115201 3.1 3:856be886:::rbd_data.112d4d24c558.0000000000000004:29 [read 3489792~16384 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e96)", 2024-09-14T06:32:29.449 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:30:17.940113+0000", 2024-09-14T06:32:29.449 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 131.46559894000001, 2024-09-14T06:32:29.449 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.075246899000000006, 2024-09-14T06:32:29.450 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:32:29.450 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:32:29.450 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:32:29.450 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:32:29.450 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:32:29.450 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:29.450 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 115201 2024-09-14T06:32:29.450 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:29.450 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:32:29.450 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:29.450 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:32:29.450 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940113+0000", 2024-09-14T06:32:29.450 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:29.450 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:29.451 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:29.451 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:32:29.451 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940113+0000", 2024-09-14T06:32:29.451 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:29.451 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:29.451 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:29.451 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:32:29.451 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940114+0000", 2024-09-14T06:32:29.451 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.1500000000000004e-07 2024-09-14T06:32:29.451 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:29.451 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:29.452 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:32:29.452 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940116+0000", 2024-09-14T06:32:29.452 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.2809999999999998e-06 2024-09-14T06:32:29.452 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:29.452 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:29.452 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:32:29.452 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940123+0000", 2024-09-14T06:32:29.452 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.9229999999999996e-06 2024-09-14T06:32:29.452 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:29.452 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:29.452 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:32:29.452 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940128+0000", 2024-09-14T06:32:29.453 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.7790000000000003e-06 2024-09-14T06:32:29.453 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:29.453 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:29.453 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:32:29.453 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.015253+0000", 2024-09-14T06:32:29.453 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.075125792999999996 2024-09-14T06:32:29.453 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:29.453 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:29.453 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:32:29.453 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.015297+0000", 2024-09-14T06:32:29.453 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.3847000000000002e-05 2024-09-14T06:32:29.453 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:29.453 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:29.454 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:32:29.454 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.015360+0000", 2024-09-14T06:32:29.454 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.2661000000000002e-05 2024-09-14T06:32:29.454 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:29.454 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:32:29.454 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:29.454 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:29.454 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:29.454 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:115202 3.1 3:856be886:::rbd_data.112d4d24c558.0000000000000004:29 [read 3735552~16384 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e96)", 2024-09-14T06:32:29.454 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:30:17.940135+0000", 2024-09-14T06:32:29.454 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 131.465577161, 2024-09-14T06:32:29.454 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.075451105000000004, 2024-09-14T06:32:29.454 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:32:29.454 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:32:29.455 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:32:29.455 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:32:29.455 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:32:29.455 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:29.455 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 115202 2024-09-14T06:32:29.455 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:29.455 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:32:29.455 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:29.455 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:32:29.455 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940135+0000", 2024-09-14T06:32:29.455 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:29.455 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:29.456 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:29.456 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:32:29.456 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940135+0000", 2024-09-14T06:32:29.456 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:29.456 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:29.456 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:29.456 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:32:29.456 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940135+0000", 2024-09-14T06:32:29.456 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.0999999999999999e-07 2024-09-14T06:32:29.456 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:29.456 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:29.456 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:32:29.457 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940137+0000", 2024-09-14T06:32:29.457 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.0020000000000001e-06 2024-09-14T06:32:29.457 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:29.457 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:29.457 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:32:29.457 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940143+0000", 2024-09-14T06:32:29.457 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.9850000000000003e-06 2024-09-14T06:32:29.457 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:29.457 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:29.457 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:32:29.457 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940148+0000", 2024-09-14T06:32:29.458 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.9880000000000004e-06 2024-09-14T06:32:29.458 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:29.458 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:29.458 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:32:29.458 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.015400+0000", 2024-09-14T06:32:29.458 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.075252186999999998 2024-09-14T06:32:29.458 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:29.458 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:29.458 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:32:29.458 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.015481+0000", 2024-09-14T06:32:29.458 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.0698000000000006e-05 2024-09-14T06:32:29.458 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:29.459 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:29.459 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:32:29.459 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.015586+0000", 2024-09-14T06:32:29.459 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00010483499999999999 2024-09-14T06:32:29.459 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:29.459 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:32:29.459 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:29.459 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:29.459 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:29.459 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:115203 3.1 3:856be886:::rbd_data.112d4d24c558.0000000000000004:29 [read 3981312~16384 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e96)", 2024-09-14T06:32:29.459 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:30:17.940156+0000", 2024-09-14T06:32:29.459 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 131.46555593599999, 2024-09-14T06:32:29.460 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.075620860999999998, 2024-09-14T06:32:29.460 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:32:29.460 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:32:29.460 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:32:29.460 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:32:29.461 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:32:29.461 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:29.461 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 115203 2024-09-14T06:32:29.461 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:29.461 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:32:29.461 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:29.461 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:32:29.461 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940156+0000", 2024-09-14T06:32:29.461 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:29.461 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:29.461 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:29.461 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:32:29.461 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940156+0000", 2024-09-14T06:32:29.462 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:29.462 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:29.462 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:29.462 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:32:29.462 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940156+0000", 2024-09-14T06:32:29.462 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.1500000000000005e-07 2024-09-14T06:32:29.462 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:29.462 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:29.462 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:32:29.462 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940158+0000", 2024-09-14T06:32:29.462 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.967e-06 2024-09-14T06:32:29.462 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:29.462 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:29.463 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:32:29.463 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940165+0000", 2024-09-14T06:32:29.463 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.6939999999999998e-06 2024-09-14T06:32:29.463 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:29.463 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:29.463 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:32:29.463 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940170+0000", 2024-09-14T06:32:29.463 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.7539999999999997e-06 2024-09-14T06:32:29.463 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:29.463 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:29.463 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:32:29.464 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.015633+0000", 2024-09-14T06:32:29.464 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.075463334000000007 2024-09-14T06:32:29.464 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:29.464 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:29.464 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:32:29.464 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.015693+0000", 2024-09-14T06:32:29.464 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.9367000000000003e-05 2024-09-14T06:32:29.464 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:29.464 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:29.464 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:32:29.464 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.015777+0000", 2024-09-14T06:32:29.464 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.4229999999999998e-05 2024-09-14T06:32:29.465 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:29.465 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:32:29.465 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:29.465 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:29.465 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:29.465 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:115204 3.1 3:856be886:::rbd_data.112d4d24c558.0000000000000004:29 [read 4227072~16384 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e96)", 2024-09-14T06:32:29.465 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:30:17.940176+0000", 2024-09-14T06:32:29.465 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 131.46553540299999, 2024-09-14T06:32:29.465 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.075785505000000003, 2024-09-14T06:32:29.465 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:32:29.465 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:32:29.466 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:32:29.466 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:32:29.466 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:32:29.466 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:29.466 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 115204 2024-09-14T06:32:29.466 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:29.466 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:32:29.466 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:29.466 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:32:29.466 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940176+0000", 2024-09-14T06:32:29.466 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:29.467 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:29.467 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:29.467 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:32:29.467 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940176+0000", 2024-09-14T06:32:29.467 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:29.467 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:29.467 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:29.467 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:32:29.467 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940177+0000", 2024-09-14T06:32:29.467 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.7899999999999999e-07 2024-09-14T06:32:29.467 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:29.468 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:29.468 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:32:29.468 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940179+0000", 2024-09-14T06:32:29.468 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.103e-06 2024-09-14T06:32:29.468 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:29.468 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:29.468 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:32:29.468 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940185+0000", 2024-09-14T06:32:29.468 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.7239999999999997e-06 2024-09-14T06:32:29.468 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:29.468 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:29.468 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:32:29.468 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940190+0000", 2024-09-14T06:32:29.468 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.7269999999999998e-06 2024-09-14T06:32:29.469 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:29.469 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:29.469 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:32:29.469 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.015814+0000", 2024-09-14T06:32:29.469 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.075624114000000006 2024-09-14T06:32:29.469 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:29.469 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:29.469 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:32:29.469 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.015862+0000", 2024-09-14T06:32:29.469 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.8770999999999998e-05 2024-09-14T06:32:29.469 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:29.469 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:29.469 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:32:29.469 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.015962+0000", 2024-09-14T06:32:29.470 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.9586999999999994e-05 2024-09-14T06:32:29.470 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:29.470 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:32:29.470 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:29.470 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:29.470 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:29.470 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:115205 3.1 3:856be886:::rbd_data.112d4d24c558.0000000000000004:29 [read 4472832~16384 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e96)", 2024-09-14T06:32:29.470 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:30:17.940197+0000", 2024-09-14T06:32:29.470 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 131.46551469400001, 2024-09-14T06:32:29.470 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.076068920999999998, 2024-09-14T06:32:29.470 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:32:29.471 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:32:29.471 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:32:29.471 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:32:29.471 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:32:29.471 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:29.471 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 115205 2024-09-14T06:32:29.471 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:29.471 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:32:29.471 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:29.471 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:32:29.471 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940197+0000", 2024-09-14T06:32:29.471 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:29.472 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:29.472 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:29.472 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:32:29.472 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940197+0000", 2024-09-14T06:32:29.472 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:29.472 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:29.472 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:29.472 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:32:29.472 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940198+0000", 2024-09-14T06:32:29.472 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.5999999999999999e-07 2024-09-14T06:32:29.472 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:29.472 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:29.472 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:32:29.473 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940200+0000", 2024-09-14T06:32:29.473 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.0899999999999999e-06 2024-09-14T06:32:29.473 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:29.473 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:29.473 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:32:29.473 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940206+0000", 2024-09-14T06:32:29.473 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.8710000000000002e-06 2024-09-14T06:32:29.473 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:29.473 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:29.473 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:32:29.473 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940211+0000", 2024-09-14T06:32:29.473 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.9230000000000001e-06 2024-09-14T06:32:29.474 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:29.474 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:29.474 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:32:29.474 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.016059+0000", 2024-09-14T06:32:29.474 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.075848051999999999 2024-09-14T06:32:29.474 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:29.474 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:29.474 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:32:29.474 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.016151+0000", 2024-09-14T06:32:29.474 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.1982000000000001e-05 2024-09-14T06:32:29.474 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:29.474 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:29.474 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:32:29.474 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.016266+0000", 2024-09-14T06:32:29.475 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000115543 2024-09-14T06:32:29.475 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:29.475 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:32:29.475 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:29.475 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:29.475 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:29.475 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:115206 3.1 3:856be886:::rbd_data.112d4d24c558.0000000000000004:29 [read 4718592~16384 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e96)", 2024-09-14T06:32:29.475 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:30:17.940223+0000", 2024-09-14T06:32:29.475 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 131.465488604, 2024-09-14T06:32:29.475 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.076265115999999994, 2024-09-14T06:32:29.475 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:32:29.475 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:32:29.475 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:32:29.476 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:32:29.476 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:32:29.476 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:29.476 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 115206 2024-09-14T06:32:29.476 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:29.476 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:32:29.476 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:29.476 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:32:29.476 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940223+0000", 2024-09-14T06:32:29.476 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:29.476 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:29.476 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:29.477 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:32:29.477 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940223+0000", 2024-09-14T06:32:29.477 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:29.477 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:29.477 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:29.477 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:32:29.477 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940224+0000", 2024-09-14T06:32:29.477 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.0399999999999996e-07 2024-09-14T06:32:29.477 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:29.477 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:29.477 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:32:29.477 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940226+0000", 2024-09-14T06:32:29.477 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.1780000000000002e-06 2024-09-14T06:32:29.477 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:29.478 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:29.478 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:32:29.478 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940232+0000", 2024-09-14T06:32:29.478 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.5010000000000001e-06 2024-09-14T06:32:29.478 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:29.478 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:29.478 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:32:29.478 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940237+0000", 2024-09-14T06:32:29.478 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.0250000000000002e-06 2024-09-14T06:32:29.478 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:29.478 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:29.478 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:32:29.479 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.016305+0000", 2024-09-14T06:32:29.479 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.076067262999999996 2024-09-14T06:32:29.479 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:29.479 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:29.479 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:32:29.479 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.016383+0000", 2024-09-14T06:32:29.479 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.8578000000000006e-05 2024-09-14T06:32:29.479 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:29.479 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:29.479 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:32:29.479 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.016488+0000", 2024-09-14T06:32:29.479 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000105067 2024-09-14T06:32:29.479 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:29.479 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:32:29.479 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:29.480 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:29.480 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:29.480 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:115207 3.1 3:856be886:::rbd_data.112d4d24c558.0000000000000004:29 [read 4964352~16384 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e96)", 2024-09-14T06:32:29.480 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:30:17.940244+0000", 2024-09-14T06:32:29.480 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 131.465467488, 2024-09-14T06:32:29.480 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.076371154999999996, 2024-09-14T06:32:29.480 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:32:29.480 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:32:29.480 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:32:29.480 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:32:29.480 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:32:29.480 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:29.480 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 115207 2024-09-14T06:32:29.481 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:29.481 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:32:29.481 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:29.481 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:32:29.481 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940244+0000", 2024-09-14T06:32:29.481 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:29.481 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:29.481 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:29.481 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:32:29.481 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940244+0000", 2024-09-14T06:32:29.481 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:29.481 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:29.482 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:29.482 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:32:29.482 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940245+0000", 2024-09-14T06:32:29.482 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.1200000000000003e-07 2024-09-14T06:32:29.482 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:29.482 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:29.482 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:32:29.482 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940247+0000", 2024-09-14T06:32:29.482 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.0030000000000001e-06 2024-09-14T06:32:29.482 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:29.482 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:29.482 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:32:29.483 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940252+0000", 2024-09-14T06:32:29.483 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.5520000000000004e-06 2024-09-14T06:32:29.483 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:29.483 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:29.483 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:32:29.483 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940257+0000", 2024-09-14T06:32:29.483 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.6759999999999998e-06 2024-09-14T06:32:29.483 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:29.483 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:29.483 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:32:29.483 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.016521+0000", 2024-09-14T06:32:29.484 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.076263499999999998 2024-09-14T06:32:29.484 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:29.484 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:29.484 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:32:29.484 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.016562+0000", 2024-09-14T06:32:29.484 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.0967000000000003e-05 2024-09-14T06:32:29.484 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:29.484 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:29.484 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:32:29.484 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.016616+0000", 2024-09-14T06:32:29.484 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.3944999999999999e-05 2024-09-14T06:32:29.484 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:29.484 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:32:29.484 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:29.485 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:29.485 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:29.485 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:115208 3.1 3:856be886:::rbd_data.112d4d24c558.0000000000000004:29 [read 5210112~16384 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e96)", 2024-09-14T06:32:29.485 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:30:17.940264+0000", 2024-09-14T06:32:29.485 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 131.46544763099999, 2024-09-14T06:32:29.485 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.076558288000000002, 2024-09-14T06:32:29.485 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:32:29.485 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:32:29.485 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:32:29.485 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:32:29.485 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:32:29.485 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:29.485 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 115208 2024-09-14T06:32:29.485 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:29.486 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:32:29.486 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:29.486 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:32:29.486 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940264+0000", 2024-09-14T06:32:29.486 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:29.486 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:29.486 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:29.486 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:32:29.486 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940264+0000", 2024-09-14T06:32:29.486 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:29.486 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:29.486 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:29.486 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:32:29.487 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940265+0000", 2024-09-14T06:32:29.487 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.0500000000000004e-07 2024-09-14T06:32:29.487 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:29.487 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:29.487 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:32:29.487 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940267+0000", 2024-09-14T06:32:29.487 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.9810000000000002e-06 2024-09-14T06:32:29.487 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:29.487 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:29.487 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:32:29.487 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940272+0000", 2024-09-14T06:32:29.487 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.6999999999999996e-06 2024-09-14T06:32:29.488 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:29.488 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:29.488 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:32:29.488 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940277+0000", 2024-09-14T06:32:29.488 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.6480000000000002e-06 2024-09-14T06:32:29.488 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:29.488 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:29.488 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:32:29.488 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.016650+0000", 2024-09-14T06:32:29.488 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.076373089000000005 2024-09-14T06:32:29.488 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:29.488 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:29.488 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:32:29.489 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.016724+0000", 2024-09-14T06:32:29.489 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.4105999999999998e-05 2024-09-14T06:32:29.489 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:29.489 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:29.489 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:32:29.489 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.016823+0000", 2024-09-14T06:32:29.489 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.8258999999999996e-05 2024-09-14T06:32:29.489 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:29.489 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:32:29.490 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:29.490 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:29.490 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:29.490 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:115209 3.1 3:856be886:::rbd_data.112d4d24c558.0000000000000004:29 [read 5455872~16384 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e96)", 2024-09-14T06:32:29.490 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:30:17.940283+0000", 2024-09-14T06:32:29.490 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 131.465428605, 2024-09-14T06:32:29.490 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.076724029999999999, 2024-09-14T06:32:29.490 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:32:29.490 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:32:29.490 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:32:29.490 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:32:29.490 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:32:29.491 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:29.491 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 115209 2024-09-14T06:32:29.491 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:29.491 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:32:29.491 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:29.491 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:32:29.491 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940283+0000", 2024-09-14T06:32:29.491 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:29.491 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:29.491 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:29.491 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:32:29.491 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940283+0000", 2024-09-14T06:32:29.491 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:29.491 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:29.491 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:29.492 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:32:29.492 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940284+0000", 2024-09-14T06:32:29.492 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.6700000000000003e-07 2024-09-14T06:32:29.492 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:29.492 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:29.492 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:32:29.492 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940286+0000", 2024-09-14T06:32:29.492 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.9520000000000001e-06 2024-09-14T06:32:29.492 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:29.492 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:29.492 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:32:29.492 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940292+0000", 2024-09-14T06:32:29.492 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.7339999999999996e-06 2024-09-14T06:32:29.492 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:29.493 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:29.493 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:32:29.493 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940296+0000", 2024-09-14T06:32:29.493 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.7700000000000001e-06 2024-09-14T06:32:29.493 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:29.493 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:29.493 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:32:29.493 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.016859+0000", 2024-09-14T06:32:29.493 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.07656317 2024-09-14T06:32:29.493 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:29.493 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:29.493 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:32:29.494 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.016929+0000", 2024-09-14T06:32:29.494 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.9679999999999997e-05 2024-09-14T06:32:29.494 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:29.494 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:29.494 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:32:29.494 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.017007+0000", 2024-09-14T06:32:29.494 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.8157000000000004e-05 2024-09-14T06:32:29.494 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:29.494 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:32:29.494 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:29.494 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:29.495 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:29.495 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:115210 3.1 3:856be886:::rbd_data.112d4d24c558.0000000000000004:29 [read 5701632~16384 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e96)", 2024-09-14T06:32:29.495 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:30:17.940303+0000", 2024-09-14T06:32:29.495 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 131.465409088, 2024-09-14T06:32:29.495 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.076821358000000006, 2024-09-14T06:32:29.495 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:32:29.495 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:32:29.495 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:32:29.495 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:32:29.495 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:32:29.495 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:29.495 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 115210 2024-09-14T06:32:29.495 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:29.496 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:32:29.496 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:29.496 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:32:29.496 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940303+0000", 2024-09-14T06:32:29.496 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:29.496 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:29.496 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:29.496 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:32:29.496 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940303+0000", 2024-09-14T06:32:29.496 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:29.496 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:29.496 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:29.496 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:32:29.496 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940303+0000", 2024-09-14T06:32:29.497 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.3000000000000001e-07 2024-09-14T06:32:29.497 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:29.497 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:29.497 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:32:29.497 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940306+0000", 2024-09-14T06:32:29.497 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.255e-06 2024-09-14T06:32:29.497 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:29.497 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:29.497 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:32:29.497 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940312+0000", 2024-09-14T06:32:29.497 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.9710000000000001e-06 2024-09-14T06:32:29.497 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:29.497 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:29.498 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:32:29.498 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940316+0000", 2024-09-14T06:32:29.498 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.5340000000000001e-06 2024-09-14T06:32:29.498 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:29.498 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:29.498 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:32:29.498 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.017033+0000", 2024-09-14T06:32:29.498 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.076717169000000002 2024-09-14T06:32:29.498 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:29.498 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:29.499 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:32:29.499 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.017073+0000", 2024-09-14T06:32:29.499 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.9582000000000002e-05 2024-09-14T06:32:29.499 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:29.499 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:29.499 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:32:29.499 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.017124+0000", 2024-09-14T06:32:29.499 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.1316999999999997e-05 2024-09-14T06:32:29.499 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:29.499 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:32:29.499 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:29.499 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:29.499 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:29.499 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:115219 3.1 3:856be886:::rbd_data.112d4d24c558.0000000000000004:29 [read 65536~16384 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e96)", 2024-09-14T06:32:29.499 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:30:17.940323+0000", 2024-09-14T06:32:29.500 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 131.465389227, 2024-09-14T06:32:29.500 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.076915205, 2024-09-14T06:32:29.500 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:32:29.500 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:32:29.500 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:32:29.500 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:32:29.500 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:32:29.500 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:29.501 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 115219 2024-09-14T06:32:29.501 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:29.501 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:32:29.501 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:29.501 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:32:29.501 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940323+0000", 2024-09-14T06:32:29.501 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:29.501 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:29.501 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:29.501 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:32:29.501 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940323+0000", 2024-09-14T06:32:29.502 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:29.502 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:29.502 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:29.502 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:32:29.502 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940323+0000", 2024-09-14T06:32:29.502 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.0800000000000005e-07 2024-09-14T06:32:29.502 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:29.502 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:29.502 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:32:29.502 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940328+0000", 2024-09-14T06:32:29.502 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.887e-06 2024-09-14T06:32:29.503 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:29.503 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:29.503 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:32:29.503 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940334+0000", 2024-09-14T06:32:29.503 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.3269999999999997e-06 2024-09-14T06:32:29.503 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:29.503 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:29.503 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:32:29.503 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940339+0000", 2024-09-14T06:32:29.503 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.7960000000000003e-06 2024-09-14T06:32:29.503 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:29.503 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:29.504 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:32:29.504 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.017148+0000", 2024-09-14T06:32:29.504 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.076809125000000006 2024-09-14T06:32:29.504 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:29.504 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:29.504 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:32:29.504 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.017186+0000", 2024-09-14T06:32:29.504 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.7283000000000002e-05 2024-09-14T06:32:29.505 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:29.505 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:29.505 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:32:29.505 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.017238+0000", 2024-09-14T06:32:29.505 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.2278999999999999e-05 2024-09-14T06:32:29.505 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:29.505 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:32:29.505 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:29.506 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:29.506 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:29.506 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:115220 3.1 3:856be886:::rbd_data.112d4d24c558.0000000000000004:29 [read 311296~16384 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e96)", 2024-09-14T06:32:29.506 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:30:17.940346+0000", 2024-09-14T06:32:29.506 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 131.465366098, 2024-09-14T06:32:29.506 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.077033293000000003, 2024-09-14T06:32:29.506 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:32:29.506 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:32:29.506 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:32:29.506 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:32:29.506 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:32:29.506 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:29.506 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 115220 2024-09-14T06:32:29.506 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:29.507 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:32:29.507 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:29.507 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:32:29.507 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940346+0000", 2024-09-14T06:32:29.507 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:29.507 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:29.507 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:29.507 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:32:29.507 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940346+0000", 2024-09-14T06:32:29.507 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:29.507 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:29.507 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:29.507 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:32:29.508 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940346+0000", 2024-09-14T06:32:29.508 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.4300000000000003e-07 2024-09-14T06:32:29.508 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:29.508 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:29.508 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:32:29.508 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940348+0000", 2024-09-14T06:32:29.508 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.0030000000000001e-06 2024-09-14T06:32:29.508 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:29.508 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:29.509 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:32:29.509 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.004655+0000", 2024-09-14T06:32:29.509 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.064306323999999998 2024-09-14T06:32:29.509 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:29.509 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:29.509 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:32:29.509 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.004671+0000", 2024-09-14T06:32:29.510 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.6589999999999999e-05 2024-09-14T06:32:29.510 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:29.510 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:29.510 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:32:29.510 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.017262+0000", 2024-09-14T06:32:29.510 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.012590570000000001 2024-09-14T06:32:29.510 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:29.511 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:29.511 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:32:29.511 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.017321+0000", 2024-09-14T06:32:29.511 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.9210999999999999e-05 2024-09-14T06:32:29.511 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:29.511 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:29.511 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:32:29.511 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.017379+0000", 2024-09-14T06:32:29.511 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.8051999999999999e-05 2024-09-14T06:32:29.511 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:29.511 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:32:29.511 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:29.511 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:29.511 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:32:29.512 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-14T06:32:29.513 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-14T06:32:29.515 INFO:tasks.ceph.osd.2.smithi094.stderr:2024-09-14T06:32:29.512+0000 7faf3269b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33649) UID: 0 2024-09-14T06:32:29.614 INFO:tasks.ceph.osd.5.smithi138.stderr:2024-09-14T06:32:29.613+0000 7fe13746f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33618) UID: 0 2024-09-14T06:32:29.646 INFO:teuthology.orchestra.run.smithi094.stderr:no valid command found; 10 closest matches: 2024-09-14T06:32:29.646 INFO:teuthology.orchestra.run.smithi094.stderr:0 2024-09-14T06:32:29.646 INFO:teuthology.orchestra.run.smithi094.stderr:1 2024-09-14T06:32:29.646 INFO:teuthology.orchestra.run.smithi094.stderr:2 2024-09-14T06:32:29.647 INFO:teuthology.orchestra.run.smithi094.stderr:abort 2024-09-14T06:32:29.647 INFO:teuthology.orchestra.run.smithi094.stderr:assert 2024-09-14T06:32:29.647 INFO:teuthology.orchestra.run.smithi094.stderr:bench [] [] [] [] 2024-09-14T06:32:29.647 INFO:teuthology.orchestra.run.smithi094.stderr:bluefs debug_inject_read_zeros 2024-09-14T06:32:29.647 INFO:teuthology.orchestra.run.smithi094.stderr:bluefs files list 2024-09-14T06:32:29.647 INFO:teuthology.orchestra.run.smithi094.stderr:bluefs stats 2024-09-14T06:32:29.647 INFO:teuthology.orchestra.run.smithi094.stderr:bluestore allocator dump block 2024-09-14T06:32:29.647 INFO:teuthology.orchestra.run.smithi094.stderr:Error EINVAL: invalid command 2024-09-14T06:32:29.649 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-14T06:32:29.650 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-14T06:32:29.650 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 'trim stale osdmaps' 2024-09-14T06:32:29.661 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-14T06:32:29.662 INFO:teuthology.orchestra.run.smithi094.stdout: "ops": [], 2024-09-14T06:32:29.662 INFO:teuthology.orchestra.run.smithi094.stdout: "num_ops": 0 2024-09-14T06:32:29.662 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-14T06:32:29.671 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-14T06:32:29.744 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-14T06:32:29.755 INFO:tasks.ceph.osd.0.smithi094.stderr:2024-09-14T06:32:29.754+0000 7f07c9333640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33636) UID: 0 2024-09-14T06:32:29.790 INFO:teuthology.orchestra.run.smithi094.stderr:no valid command found; 10 closest matches: 2024-09-14T06:32:29.790 INFO:teuthology.orchestra.run.smithi094.stderr:0 2024-09-14T06:32:29.791 INFO:teuthology.orchestra.run.smithi094.stderr:1 2024-09-14T06:32:29.791 INFO:teuthology.orchestra.run.smithi094.stderr:2 2024-09-14T06:32:29.791 INFO:teuthology.orchestra.run.smithi094.stderr:abort 2024-09-14T06:32:29.791 INFO:teuthology.orchestra.run.smithi094.stderr:assert 2024-09-14T06:32:29.791 INFO:teuthology.orchestra.run.smithi094.stderr:bench [] [] [] [] 2024-09-14T06:32:29.791 INFO:teuthology.orchestra.run.smithi094.stderr:bluefs debug_inject_read_zeros 2024-09-14T06:32:29.791 INFO:teuthology.orchestra.run.smithi094.stderr:bluefs files list 2024-09-14T06:32:29.791 INFO:teuthology.orchestra.run.smithi094.stderr:bluefs stats 2024-09-14T06:32:29.791 INFO:teuthology.orchestra.run.smithi094.stderr:bluestore allocator dump block 2024-09-14T06:32:29.791 INFO:teuthology.orchestra.run.smithi094.stderr:Error EINVAL: invalid command 2024-09-14T06:32:29.793 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-14T06:32:29.796 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-14T06:32:29.796 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 'trim stale osdmaps' 2024-09-14T06:32:29.824 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-14T06:32:29.825 INFO:teuthology.orchestra.run.smithi094.stdout: "ops": [], 2024-09-14T06:32:29.825 INFO:teuthology.orchestra.run.smithi094.stdout: "complaint_time": 30, 2024-09-14T06:32:29.825 INFO:teuthology.orchestra.run.smithi094.stdout: "num_blocked_ops": 0 2024-09-14T06:32:29.825 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-14T06:32:29.825 INFO:tasks.ceph.osd.2.smithi094.stderr:2024-09-14T06:32:29.823+0000 7faf3269b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33649) UID: 0 2024-09-14T06:32:29.839 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-14T06:32:29.923 INFO:tasks.ceph.osd.1.smithi094.stderr:2024-09-14T06:32:29.922+0000 7f328e47a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33641) UID: 0 2024-09-14T06:32:29.984 INFO:teuthology.orchestra.run.smithi094.stderr:no valid command found; 10 closest matches: 2024-09-14T06:32:29.985 INFO:teuthology.orchestra.run.smithi094.stderr:0 2024-09-14T06:32:29.985 INFO:teuthology.orchestra.run.smithi094.stderr:1 2024-09-14T06:32:29.985 INFO:teuthology.orchestra.run.smithi094.stderr:2 2024-09-14T06:32:29.985 INFO:teuthology.orchestra.run.smithi094.stderr:abort 2024-09-14T06:32:29.985 INFO:teuthology.orchestra.run.smithi094.stderr:assert 2024-09-14T06:32:29.985 INFO:teuthology.orchestra.run.smithi094.stderr:bench [] [] [] [] 2024-09-14T06:32:29.985 INFO:teuthology.orchestra.run.smithi094.stderr:bluefs debug_inject_read_zeros 2024-09-14T06:32:29.985 INFO:teuthology.orchestra.run.smithi094.stderr:bluefs files list 2024-09-14T06:32:29.985 INFO:teuthology.orchestra.run.smithi094.stderr:bluefs stats 2024-09-14T06:32:29.985 INFO:teuthology.orchestra.run.smithi094.stderr:bluestore allocator dump block 2024-09-14T06:32:29.985 INFO:teuthology.orchestra.run.smithi094.stderr:Error EINVAL: invalid command 2024-09-14T06:32:29.986 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-14T06:32:29.986 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-14T06:32:29.992 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-14T06:32:29.992 INFO:teuthology.orchestra.run.smithi094.stdout: "size": 20, 2024-09-14T06:32:29.992 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 600, 2024-09-14T06:32:29.992 INFO:teuthology.orchestra.run.smithi094.stdout: "ops": [ 2024-09-14T06:32:29.992 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:29.992 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:14918 3.0 3:05c39535:::rbd_data.112d92027590.0000000000000009:f [read 795648~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:32:29.992 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:26.083834+0000", 2024-09-14T06:32:29.992 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 183.90578929599999, 2024-09-14T06:32:29.992 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.041776287000000002, 2024-09-14T06:32:29.992 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:32:29.992 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:32:29.992 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:32:29.993 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:32:29.993 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:32:29.993 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:29.993 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 14918 2024-09-14T06:32:29.993 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:29.993 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:32:29.993 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:29.993 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:32:29.993 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.083834+0000", 2024-09-14T06:32:29.993 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:29.993 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:29.993 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:29.993 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:32:29.993 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.083834+0000", 2024-09-14T06:32:29.993 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:29.994 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:29.994 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:29.994 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:32:29.994 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.083834+0000", 2024-09-14T06:32:29.994 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.0100000000000001e-07 2024-09-14T06:32:29.994 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:29.994 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:29.994 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:32:29.994 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.083835+0000", 2024-09-14T06:32:29.994 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.3900000000000003e-07 2024-09-14T06:32:29.994 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:29.994 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:29.994 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:32:29.994 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.083838+0000", 2024-09-14T06:32:29.995 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.9830000000000001e-06 2024-09-14T06:32:29.995 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:29.995 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:29.995 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:32:29.995 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.083840+0000", 2024-09-14T06:32:29.995 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.4679999999999999e-06 2024-09-14T06:32:29.995 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:29.995 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:29.995 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:32:29.995 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.125455+0000", 2024-09-14T06:32:29.995 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.041614578999999999 2024-09-14T06:32:29.995 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:29.995 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:29.995 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:32:29.995 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.125496+0000", 2024-09-14T06:32:29.995 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.1047000000000002e-05 2024-09-14T06:32:29.996 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:29.996 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:29.996 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:32:29.996 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.125610+0000", 2024-09-14T06:32:29.996 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00011407 2024-09-14T06:32:29.996 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:29.996 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:32:29.996 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:29.996 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:29.996 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:29.996 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:14919 3.0 3:05c39535:::rbd_data.112d92027590.0000000000000009:f [read 805888~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:32:29.996 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:26.083843+0000", 2024-09-14T06:32:29.996 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 183.905779548, 2024-09-14T06:32:29.996 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.041992356000000002, 2024-09-14T06:32:29.997 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:32:29.997 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:32:29.997 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:32:29.997 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:32:29.997 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:32:29.997 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:29.997 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 14919 2024-09-14T06:32:29.997 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:29.997 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:32:29.997 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:29.997 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:32:29.997 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.083843+0000", 2024-09-14T06:32:29.997 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:29.997 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:29.998 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:29.998 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:32:29.998 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.083843+0000", 2024-09-14T06:32:29.998 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:29.998 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:29.998 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:29.998 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:32:29.998 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.083844+0000", 2024-09-14T06:32:29.998 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.3300000000000001e-07 2024-09-14T06:32:29.998 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:29.998 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:29.998 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:32:29.998 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.083845+0000", 2024-09-14T06:32:29.998 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.4729999999999999e-06 2024-09-14T06:32:29.999 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:29.999 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:29.999 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:32:29.999 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.083850+0000", 2024-09-14T06:32:29.999 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.0130000000000001e-06 2024-09-14T06:32:29.999 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:29.999 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:29.999 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:32:29.999 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.083854+0000", 2024-09-14T06:32:29.999 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.4759999999999998e-06 2024-09-14T06:32:29.999 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:29.999 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:29.999 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:32:30.000 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.125636+0000", 2024-09-14T06:32:30.000 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.041782683000000001 2024-09-14T06:32:30.000 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:30.000 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:30.000 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:32:30.000 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.125680+0000", 2024-09-14T06:32:30.000 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.3614e-05 2024-09-14T06:32:30.000 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:30.000 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:30.000 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:32:30.000 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.125836+0000", 2024-09-14T06:32:30.000 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000155864 2024-09-14T06:32:30.000 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:30.000 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:32:30.000 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:30.001 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:30.001 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:30.001 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:14920 3.0 3:05c39535:::rbd_data.112d92027590.0000000000000009:f [read 816128~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:32:30.001 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:26.083868+0000", 2024-09-14T06:32:30.001 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 183.905754887, 2024-09-14T06:32:30.001 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.042166771999999998, 2024-09-14T06:32:30.001 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:32:30.001 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:32:30.001 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:32:30.001 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:32:30.001 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:32:30.001 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:30.001 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 14920 2024-09-14T06:32:30.002 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:30.002 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:32:30.002 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:30.002 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:32:30.002 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.083868+0000", 2024-09-14T06:32:30.002 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:30.002 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:30.002 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:30.002 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:32:30.002 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.083868+0000", 2024-09-14T06:32:30.002 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:30.002 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:30.002 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:30.002 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:32:30.003 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.083869+0000", 2024-09-14T06:32:30.003 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.99e-07 2024-09-14T06:32:30.003 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:30.003 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:30.003 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:32:30.003 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.083870+0000", 2024-09-14T06:32:30.003 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.782e-06 2024-09-14T06:32:30.003 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:30.003 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:30.003 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:32:30.003 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.083876+0000", 2024-09-14T06:32:30.003 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.3809999999999998e-06 2024-09-14T06:32:30.003 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:30.003 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:30.003 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:32:30.004 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.083882+0000", 2024-09-14T06:32:30.004 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.9479999999999996e-06 2024-09-14T06:32:30.004 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:30.004 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:30.004 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:32:30.004 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.125878+0000", 2024-09-14T06:32:30.004 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.041996264999999998 2024-09-14T06:32:30.004 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:30.004 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:30.004 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:32:30.004 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.125929+0000", 2024-09-14T06:32:30.004 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.0498999999999999e-05 2024-09-14T06:32:30.004 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:30.004 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:30.004 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:32:30.005 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.126035+0000", 2024-09-14T06:32:30.005 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000106298 2024-09-14T06:32:30.005 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:30.005 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:32:30.005 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:30.005 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:30.005 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:30.005 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:14921 3.0 3:05c39535:::rbd_data.112d92027590.0000000000000009:f [read 826368~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:32:30.005 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:26.083889+0000", 2024-09-14T06:32:30.005 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 183.90573387699999, 2024-09-14T06:32:30.005 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.042435309999999997, 2024-09-14T06:32:30.005 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:32:30.005 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:32:30.005 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:32:30.005 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:32:30.006 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:32:30.006 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:30.006 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 14921 2024-09-14T06:32:30.006 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:30.006 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:32:30.006 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:30.006 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:32:30.006 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.083889+0000", 2024-09-14T06:32:30.006 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:30.006 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:30.006 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:30.006 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:32:30.006 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.083889+0000", 2024-09-14T06:32:30.006 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:30.006 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:30.007 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:30.007 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:32:30.007 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.086033+0000", 2024-09-14T06:32:30.007 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0021444179999999999 2024-09-14T06:32:30.007 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:30.007 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:30.007 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:32:30.007 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.086038+0000", 2024-09-14T06:32:30.007 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.0969999999999999e-06 2024-09-14T06:32:30.007 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:30.007 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:30.007 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:32:30.007 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.086046+0000", 2024-09-14T06:32:30.007 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.9440000000000005e-06 2024-09-14T06:32:30.007 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:30.007 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:30.008 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:32:30.008 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.086050+0000", 2024-09-14T06:32:30.008 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.69e-06 2024-09-14T06:32:30.008 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:30.008 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:30.008 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:32:30.008 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.126080+0000", 2024-09-14T06:32:30.008 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.040030237000000003 2024-09-14T06:32:30.008 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:30.008 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:30.008 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:32:30.008 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.126163+0000", 2024-09-14T06:32:30.008 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.2071999999999996e-05 2024-09-14T06:32:30.008 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:30.009 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:30.009 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:32:30.009 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.126324+0000", 2024-09-14T06:32:30.009 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00016185200000000001 2024-09-14T06:32:30.009 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:30.009 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:32:30.009 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:30.009 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:30.009 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:30.009 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:55627 3.0 3:06490baf:::rbd_data.112dca2f6ae1.0000000000000030:17 [read 2002944~4096 [fadvise_nocache]] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:32:30.009 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:44.599966+0000", 2024-09-14T06:32:30.009 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 165.38965692900001, 2024-09-14T06:32:30.009 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.041868546999999999, 2024-09-14T06:32:30.009 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:32:30.009 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:32:30.010 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:32:30.010 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:32:30.010 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:32:30.010 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:30.010 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 55627 2024-09-14T06:32:30.010 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:30.010 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:32:30.010 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:30.010 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:32:30.010 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.599966+0000", 2024-09-14T06:32:30.010 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:30.010 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:30.010 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:30.010 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:32:30.010 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.599966+0000", 2024-09-14T06:32:30.010 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:30.011 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:30.011 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:30.011 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:32:30.011 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.599966+0000", 2024-09-14T06:32:30.011 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.0900000000000001e-07 2024-09-14T06:32:30.011 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:30.011 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:30.011 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:32:30.011 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.599967+0000", 2024-09-14T06:32:30.011 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.64e-07 2024-09-14T06:32:30.011 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:30.011 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:30.011 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:32:30.011 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.599970+0000", 2024-09-14T06:32:30.011 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.9670000000000001e-06 2024-09-14T06:32:30.012 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:30.012 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:30.012 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:32:30.012 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.599973+0000", 2024-09-14T06:32:30.012 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.632e-06 2024-09-14T06:32:30.012 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:30.012 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:30.012 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:32:30.012 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.641665+0000", 2024-09-14T06:32:30.012 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.041692547000000003 2024-09-14T06:32:30.012 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:30.012 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:30.012 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:32:30.012 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.641740+0000", 2024-09-14T06:32:30.012 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.4559000000000002e-05 2024-09-14T06:32:30.013 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:30.013 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:30.013 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:32:30.013 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.641835+0000", 2024-09-14T06:32:30.013 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.4668999999999998e-05 2024-09-14T06:32:30.013 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:30.013 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:32:30.013 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:30.013 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:30.013 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:30.013 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:55628 3.0 3:06490baf:::rbd_data.112dca2f6ae1.0000000000000030:17 [read 2027520~4096 [fadvise_nocache]] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:32:30.013 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:44.599976+0000", 2024-09-14T06:32:30.013 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 165.389647142, 2024-09-14T06:32:30.013 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.041991621, 2024-09-14T06:32:30.013 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:32:30.014 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:32:30.014 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:32:30.014 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:32:30.014 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:32:30.014 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:30.014 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 55628 2024-09-14T06:32:30.014 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:30.014 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:32:30.014 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:30.014 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:32:30.014 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.599976+0000", 2024-09-14T06:32:30.014 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:30.014 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:30.014 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:30.015 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:32:30.015 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.599976+0000", 2024-09-14T06:32:30.015 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:30.015 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:30.015 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:30.015 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:32:30.015 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.599976+0000", 2024-09-14T06:32:30.015 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.1299999999999999e-07 2024-09-14T06:32:30.015 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:30.015 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:30.015 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:32:30.015 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.599977+0000", 2024-09-14T06:32:30.015 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.7999999999999993e-07 2024-09-14T06:32:30.015 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:30.016 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:30.016 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:32:30.016 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.599980+0000", 2024-09-14T06:32:30.016 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.01e-06 2024-09-14T06:32:30.016 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:30.016 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:30.016 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:32:30.016 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.599983+0000", 2024-09-14T06:32:30.016 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.5409999999999999e-06 2024-09-14T06:32:30.016 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:30.016 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:30.016 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:32:30.017 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.641869+0000", 2024-09-14T06:32:30.017 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.041886235000000001 2024-09-14T06:32:30.017 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:30.017 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:30.017 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:32:30.017 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.641913+0000", 2024-09-14T06:32:30.017 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.4357000000000002e-05 2024-09-14T06:32:30.017 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:30.017 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:30.017 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:32:30.017 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.641967+0000", 2024-09-14T06:32:30.017 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.4284999999999999e-05 2024-09-14T06:32:30.017 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:30.017 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:32:30.018 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:30.018 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:30.018 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:30.018 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:55629 3.0 3:06490baf:::rbd_data.112dca2f6ae1.0000000000000030:17 [read 2052096~4096 [fadvise_nocache]] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:32:30.018 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:44.599985+0000", 2024-09-14T06:32:30.018 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 165.38963743599999, 2024-09-14T06:32:30.018 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.042183375000000002, 2024-09-14T06:32:30.018 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:32:30.018 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:32:30.018 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:32:30.018 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:32:30.018 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:32:30.018 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:30.018 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 55629 2024-09-14T06:32:30.019 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:30.019 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:32:30.019 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:30.019 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:32:30.019 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.599985+0000", 2024-09-14T06:32:30.019 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:30.019 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:30.019 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:30.019 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:32:30.019 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.599985+0000", 2024-09-14T06:32:30.019 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:30.019 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:30.019 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:30.019 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:32:30.020 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.599986+0000", 2024-09-14T06:32:30.020 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.05e-07 2024-09-14T06:32:30.020 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:30.020 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:30.020 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:32:30.020 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.599987+0000", 2024-09-14T06:32:30.020 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.8400000000000002e-07 2024-09-14T06:32:30.020 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:30.020 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:30.020 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:32:30.020 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.599990+0000", 2024-09-14T06:32:30.020 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.5640000000000001e-06 2024-09-14T06:32:30.020 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:30.020 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:30.020 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:32:30.021 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.599993+0000", 2024-09-14T06:32:30.021 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.615e-06 2024-09-14T06:32:30.021 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:30.021 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:30.021 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:32:30.021 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.642003+0000", 2024-09-14T06:32:30.021 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.042010345999999997 2024-09-14T06:32:30.021 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:30.021 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:30.021 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:32:30.021 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.642080+0000", 2024-09-14T06:32:30.021 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.6538999999999993e-05 2024-09-14T06:32:30.021 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:30.021 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:30.022 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:32:30.022 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.642169+0000", 2024-09-14T06:32:30.022 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.9122000000000005e-05 2024-09-14T06:32:30.022 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:30.022 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:32:30.022 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:30.022 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:30.022 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:30.022 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:55630 3.0 3:06490baf:::rbd_data.112dca2f6ae1.0000000000000030:17 [read 2076672~4096 [fadvise_nocache]] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:32:30.022 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:44.599996+0000", 2024-09-14T06:32:30.022 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 165.38962695800001, 2024-09-14T06:32:30.022 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.042299615999999998, 2024-09-14T06:32:30.022 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:32:30.022 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:32:30.023 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:32:30.023 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:32:30.023 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:32:30.023 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:30.023 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 55630 2024-09-14T06:32:30.023 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:30.023 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:32:30.023 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:30.023 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:32:30.023 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.599996+0000", 2024-09-14T06:32:30.023 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:30.023 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:30.023 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:30.023 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:32:30.023 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.599996+0000", 2024-09-14T06:32:30.024 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:30.024 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:30.024 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:30.024 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:32:30.024 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.599996+0000", 2024-09-14T06:32:30.024 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.1500000000000001e-07 2024-09-14T06:32:30.024 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:30.024 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:30.024 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:32:30.024 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.599997+0000", 2024-09-14T06:32:30.024 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.33e-07 2024-09-14T06:32:30.024 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:30.024 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:30.024 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:32:30.025 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600000+0000", 2024-09-14T06:32:30.025 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.0240000000000002e-06 2024-09-14T06:32:30.025 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:30.025 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:30.025 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:32:30.025 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600003+0000", 2024-09-14T06:32:30.025 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.3640000000000002e-06 2024-09-14T06:32:30.025 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:30.025 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:30.025 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:32:30.025 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.642200+0000", 2024-09-14T06:32:30.025 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.042197571000000003 2024-09-14T06:32:30.025 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:30.026 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:30.026 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:32:30.026 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.642243+0000", 2024-09-14T06:32:30.026 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.2765000000000001e-05 2024-09-14T06:32:30.026 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:30.026 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:30.026 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:32:30.026 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.642296+0000", 2024-09-14T06:32:30.026 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.2744000000000002e-05 2024-09-14T06:32:30.026 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:30.026 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:32:30.026 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:30.026 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:30.026 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:30.027 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:55631 3.0 3:06490baf:::rbd_data.112dca2f6ae1.0000000000000030:17 [read 1048576~4096 [fadvise_nocache]] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:32:30.027 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:44.600006+0000", 2024-09-14T06:32:30.027 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 165.38961737, 2024-09-14T06:32:30.027 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.042487905999999999, 2024-09-14T06:32:30.027 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:32:30.027 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:32:30.027 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:32:30.027 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:32:30.027 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:32:30.027 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:30.027 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 55631 2024-09-14T06:32:30.027 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:30.027 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:32:30.027 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:30.028 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:32:30.028 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600006+0000", 2024-09-14T06:32:30.028 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:30.028 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:30.028 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:30.028 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:32:30.028 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600006+0000", 2024-09-14T06:32:30.028 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:30.028 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:30.028 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:30.028 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:32:30.028 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600006+0000", 2024-09-14T06:32:30.028 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.0800000000000001e-07 2024-09-14T06:32:30.028 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:30.029 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:30.029 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:32:30.029 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600007+0000", 2024-09-14T06:32:30.029 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.7199999999999997e-07 2024-09-14T06:32:30.029 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:30.029 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:30.029 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:32:30.029 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600010+0000", 2024-09-14T06:32:30.029 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.0280000000000001e-06 2024-09-14T06:32:30.029 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:30.029 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:30.029 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:32:30.029 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600012+0000", 2024-09-14T06:32:30.030 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.475e-06 2024-09-14T06:32:30.030 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:30.030 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:30.030 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:32:30.030 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.642329+0000", 2024-09-14T06:32:30.030 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.042317157000000001 2024-09-14T06:32:30.030 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:30.030 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:30.030 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:32:30.030 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.642403+0000", 2024-09-14T06:32:30.030 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.3941000000000002e-05 2024-09-14T06:32:30.030 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:30.030 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:30.031 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:32:30.031 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.642493+0000", 2024-09-14T06:32:30.031 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.0124999999999995e-05 2024-09-14T06:32:30.031 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:30.031 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:32:30.031 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:30.031 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:30.031 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:30.031 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:55632 3.0 3:06490baf:::rbd_data.112dca2f6ae1.0000000000000030:17 [read 1073152~4096 [fadvise_nocache]] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:32:30.031 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:44.600015+0000", 2024-09-14T06:32:30.031 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 165.389607617, 2024-09-14T06:32:30.031 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.042604922000000003, 2024-09-14T06:32:30.031 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:32:30.031 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:32:30.032 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:32:30.032 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:32:30.032 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:32:30.032 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:30.032 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 55632 2024-09-14T06:32:30.032 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:30.032 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:32:30.032 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:30.032 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:32:30.032 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600015+0000", 2024-09-14T06:32:30.032 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:30.032 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:30.033 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:30.033 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:32:30.033 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600015+0000", 2024-09-14T06:32:30.033 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:30.033 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:30.033 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:30.033 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:32:30.033 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600016+0000", 2024-09-14T06:32:30.033 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.1e-07 2024-09-14T06:32:30.033 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:30.033 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:30.033 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:32:30.033 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600017+0000", 2024-09-14T06:32:30.033 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.9900000000000009e-07 2024-09-14T06:32:30.034 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:30.034 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:30.034 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:32:30.034 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600020+0000", 2024-09-14T06:32:30.034 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.4170000000000001e-06 2024-09-14T06:32:30.034 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:30.034 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:30.034 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:32:30.034 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600023+0000", 2024-09-14T06:32:30.034 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.588e-06 2024-09-14T06:32:30.034 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:30.034 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:30.034 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:32:30.034 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.642526+0000", 2024-09-14T06:32:30.034 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.042503236999999999 2024-09-14T06:32:30.034 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:30.035 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:30.035 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:32:30.035 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.642568+0000", 2024-09-14T06:32:30.035 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.1925000000000003e-05 2024-09-14T06:32:30.035 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:30.035 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:30.035 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:32:30.035 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.642620+0000", 2024-09-14T06:32:30.035 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.2546000000000001e-05 2024-09-14T06:32:30.035 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:30.035 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:32:30.035 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:30.035 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:30.035 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:30.036 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:55633 3.0 3:06490baf:::rbd_data.112dca2f6ae1.0000000000000030:17 [read 1097728~4096 [fadvise_nocache]] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:32:30.036 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:44.600026+0000", 2024-09-14T06:32:30.036 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 165.389597348, 2024-09-14T06:32:30.036 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.042797474000000002, 2024-09-14T06:32:30.036 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:32:30.036 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:32:30.036 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:32:30.036 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:32:30.036 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:32:30.036 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:30.036 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 55633 2024-09-14T06:32:30.036 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:30.036 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:32:30.036 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:30.037 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:32:30.037 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600026+0000", 2024-09-14T06:32:30.037 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:30.037 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:30.037 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:30.037 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:32:30.037 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600026+0000", 2024-09-14T06:32:30.037 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:30.037 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:30.037 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:30.037 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:32:30.037 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600026+0000", 2024-09-14T06:32:30.037 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.3099999999999999e-07 2024-09-14T06:32:30.038 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:30.038 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:30.038 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:32:30.038 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600029+0000", 2024-09-14T06:32:30.038 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.304e-06 2024-09-14T06:32:30.038 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:30.038 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:30.038 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:32:30.038 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600034+0000", 2024-09-14T06:32:30.038 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.1429999999999997e-06 2024-09-14T06:32:30.038 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:30.038 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:30.038 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:32:30.038 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600038+0000", 2024-09-14T06:32:30.039 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.692e-06 2024-09-14T06:32:30.039 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:30.039 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:30.039 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:32:30.039 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.642654+0000", 2024-09-14T06:32:30.039 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.042615564000000002 2024-09-14T06:32:30.039 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:30.039 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:30.039 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:32:30.039 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.642728+0000", 2024-09-14T06:32:30.039 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.4018999999999997e-05 2024-09-14T06:32:30.039 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:30.039 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:30.039 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:32:30.040 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.642823+0000", 2024-09-14T06:32:30.040 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.5520999999999996e-05 2024-09-14T06:32:30.040 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:30.040 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:32:30.040 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:30.040 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:30.040 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:30.040 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:55634 3.0 3:06490baf:::rbd_data.112dca2f6ae1.0000000000000030:17 [read 1122304~4096 [fadvise_nocache]] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:32:30.040 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:44.600042+0000", 2024-09-14T06:32:30.040 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 165.38958087099999, 2024-09-14T06:32:30.040 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.042903729000000002, 2024-09-14T06:32:30.040 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:32:30.040 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:32:30.040 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:32:30.041 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:32:30.041 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:32:30.041 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:30.041 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 55634 2024-09-14T06:32:30.041 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:30.041 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:32:30.041 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:30.041 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:32:30.041 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600042+0000", 2024-09-14T06:32:30.041 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:30.041 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:30.041 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:30.041 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:32:30.042 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600042+0000", 2024-09-14T06:32:30.042 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:30.042 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:30.042 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:30.042 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:32:30.042 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600043+0000", 2024-09-14T06:32:30.042 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.7000000000000005e-07 2024-09-14T06:32:30.042 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:30.042 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:30.042 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:32:30.042 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600044+0000", 2024-09-14T06:32:30.042 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.742e-06 2024-09-14T06:32:30.043 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:30.043 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:30.043 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:32:30.043 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600050+0000", 2024-09-14T06:32:30.043 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.0800000000000002e-06 2024-09-14T06:32:30.043 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:30.043 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:30.043 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:32:30.043 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600055+0000", 2024-09-14T06:32:30.043 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.3889999999999998e-06 2024-09-14T06:32:30.043 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:30.043 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:30.043 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:32:30.043 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.642845+0000", 2024-09-14T06:32:30.043 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.042790040000000001 2024-09-14T06:32:30.043 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:30.044 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:30.044 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:32:30.044 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.642888+0000", 2024-09-14T06:32:30.044 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.2790000000000002e-05 2024-09-14T06:32:30.044 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:30.044 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:30.044 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:32:30.044 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.642946+0000", 2024-09-14T06:32:30.044 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.8118000000000002e-05 2024-09-14T06:32:30.044 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:30.044 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:32:30.044 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:30.044 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:30.044 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:30.044 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:55635 3.0 3:06490baf:::rbd_data.112dca2f6ae1.0000000000000030:17 [read 1146880~4096 [fadvise_nocache]] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:32:30.045 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:44.600061+0000", 2024-09-14T06:32:30.045 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 165.38956229999999, 2024-09-14T06:32:30.045 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.043092695, 2024-09-14T06:32:30.045 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:32:30.045 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:32:30.045 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:32:30.045 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:32:30.045 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:32:30.045 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:30.045 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 55635 2024-09-14T06:32:30.045 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:30.045 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:32:30.045 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:30.045 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:32:30.045 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600061+0000", 2024-09-14T06:32:30.046 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:30.046 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:30.046 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:30.046 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:32:30.046 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600061+0000", 2024-09-14T06:32:30.046 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:30.046 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:30.046 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:30.046 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:32:30.046 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600061+0000", 2024-09-14T06:32:30.046 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.3099999999999999e-07 2024-09-14T06:32:30.046 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:30.046 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:30.046 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:32:30.046 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600063+0000", 2024-09-14T06:32:30.046 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.6220000000000001e-06 2024-09-14T06:32:30.047 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:30.047 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:30.047 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:32:30.047 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600068+0000", 2024-09-14T06:32:30.047 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.4790000000000004e-06 2024-09-14T06:32:30.047 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:30.047 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:30.047 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:32:30.047 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600072+0000", 2024-09-14T06:32:30.047 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.3100000000000002e-06 2024-09-14T06:32:30.047 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:30.047 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:30.047 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:32:30.047 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.642987+0000", 2024-09-14T06:32:30.048 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.042914489 2024-09-14T06:32:30.048 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:30.048 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:30.048 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:32:30.048 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.643062+0000", 2024-09-14T06:32:30.048 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.5011000000000004e-05 2024-09-14T06:32:30.048 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:30.048 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:30.048 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:32:30.048 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.643153+0000", 2024-09-14T06:32:30.048 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.1452999999999993e-05 2024-09-14T06:32:30.048 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:30.048 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:32:30.048 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:30.048 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:30.048 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:30.049 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:55636 3.0 3:06490baf:::rbd_data.112dca2f6ae1.0000000000000030:17 [read 1171456~4096 [fadvise_nocache]] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:32:30.049 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:44.600078+0000", 2024-09-14T06:32:30.049 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 165.38954488799999, 2024-09-14T06:32:30.049 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.043207966, 2024-09-14T06:32:30.049 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:32:30.049 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:32:30.049 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:32:30.049 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:32:30.049 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:32:30.049 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:30.049 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 55636 2024-09-14T06:32:30.049 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:30.049 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:32:30.050 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:30.050 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:32:30.050 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600078+0000", 2024-09-14T06:32:30.050 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:30.050 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:30.050 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:30.050 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:32:30.050 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600078+0000", 2024-09-14T06:32:30.050 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:30.050 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:30.050 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:30.050 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:32:30.050 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600078+0000", 2024-09-14T06:32:30.050 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.34e-07 2024-09-14T06:32:30.050 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:30.051 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:30.051 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:32:30.051 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600080+0000", 2024-09-14T06:32:30.051 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.641e-06 2024-09-14T06:32:30.051 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:30.051 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:30.051 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:32:30.051 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600086+0000", 2024-09-14T06:32:30.051 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.3860000000000004e-06 2024-09-14T06:32:30.051 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:30.051 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:30.051 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:32:30.051 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600091+0000", 2024-09-14T06:32:30.051 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.5970000000000002e-06 2024-09-14T06:32:30.051 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:30.051 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:30.052 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:32:30.052 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.643184+0000", 2024-09-14T06:32:30.052 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.043092893 2024-09-14T06:32:30.052 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:30.052 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:30.052 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:32:30.052 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.643228+0000", 2024-09-14T06:32:30.052 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.4536999999999997e-05 2024-09-14T06:32:30.052 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:30.052 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:30.052 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:32:30.052 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.643286+0000", 2024-09-14T06:32:30.052 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.7578000000000003e-05 2024-09-14T06:32:30.052 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:30.053 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:32:30.053 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:30.053 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:30.053 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:30.053 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:55637 3.0 3:06490baf:::rbd_data.112dca2f6ae1.0000000000000030:17 [read 1196032~4096 [fadvise_nocache]] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:32:30.053 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:44.600097+0000", 2024-09-14T06:32:30.053 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 165.389526324, 2024-09-14T06:32:30.053 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.043399124999999997, 2024-09-14T06:32:30.053 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:32:30.053 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:32:30.053 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:32:30.053 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:32:30.053 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:32:30.053 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:30.053 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 55637 2024-09-14T06:32:30.054 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:30.054 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:32:30.054 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:30.054 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:32:30.054 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600097+0000", 2024-09-14T06:32:30.054 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:30.054 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:30.054 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:30.054 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:32:30.054 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600097+0000", 2024-09-14T06:32:30.054 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:30.054 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:30.054 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:30.054 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:32:30.054 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.604372+0000", 2024-09-14T06:32:30.054 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0042751489999999998 2024-09-14T06:32:30.055 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:30.055 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:30.055 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:32:30.055 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.604377+0000", 2024-09-14T06:32:30.055 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.7899999999999999e-06 2024-09-14T06:32:30.055 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:30.055 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:30.055 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:32:30.055 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.604389+0000", 2024-09-14T06:32:30.055 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.2197e-05 2024-09-14T06:32:30.055 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:30.055 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:30.055 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:32:30.055 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.604397+0000", 2024-09-14T06:32:30.055 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.8730000000000007e-06 2024-09-14T06:32:30.056 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:30.056 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:30.056 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:32:30.056 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.643324+0000", 2024-09-14T06:32:30.056 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.038926897000000002 2024-09-14T06:32:30.056 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:30.056 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:30.056 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:32:30.056 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.643402+0000", 2024-09-14T06:32:30.056 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.8063000000000001e-05 2024-09-14T06:32:30.056 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:30.056 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:30.056 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:32:30.056 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.643496+0000", 2024-09-14T06:32:30.056 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.4155999999999998e-05 2024-09-14T06:32:30.057 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:30.057 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:32:30.057 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:30.057 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:30.057 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:30.057 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:55657 3.0 3:06490baf:::rbd_data.112dca2f6ae1.0000000000000030:17 [read 1687552~4096 [fadvise_nocache]] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:32:30.057 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:44.604841+0000", 2024-09-14T06:32:30.057 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 165.38478159499999, 2024-09-14T06:32:30.057 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.041890087999999999, 2024-09-14T06:32:30.057 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:32:30.057 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:32:30.058 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:32:30.058 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:32:30.058 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:32:30.058 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:30.058 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 55657 2024-09-14T06:32:30.058 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:30.058 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:32:30.058 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:30.058 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:32:30.058 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.604841+0000", 2024-09-14T06:32:30.058 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:30.058 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:30.058 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:30.058 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:32:30.059 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.604841+0000", 2024-09-14T06:32:30.059 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:30.059 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:30.059 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:30.059 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:32:30.059 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.604842+0000", 2024-09-14T06:32:30.059 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.0900000000000001e-07 2024-09-14T06:32:30.059 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:30.059 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:30.059 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:32:30.059 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.604846+0000", 2024-09-14T06:32:30.059 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.5759999999999999e-06 2024-09-14T06:32:30.059 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:30.059 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:30.060 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:32:30.060 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.604850+0000", 2024-09-14T06:32:30.060 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.405e-06 2024-09-14T06:32:30.060 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:30.060 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:30.060 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:32:30.060 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.604852+0000", 2024-09-14T06:32:30.060 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.8590000000000001e-06 2024-09-14T06:32:30.060 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:30.060 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:30.060 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:32:30.060 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.646633+0000", 2024-09-14T06:32:30.060 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.041780772000000001 2024-09-14T06:32:30.060 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:30.061 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:30.061 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:32:30.061 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.646677+0000", 2024-09-14T06:32:30.061 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.4115e-05 2024-09-14T06:32:30.061 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:30.061 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:30.061 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:32:30.061 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.646731+0000", 2024-09-14T06:32:30.061 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.4151999999999999e-05 2024-09-14T06:32:30.061 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:30.061 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:32:30.061 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:30.061 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:30.061 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:30.062 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:55658 3.0 3:06490baf:::rbd_data.112dca2f6ae1.0000000000000030:17 [read 1712128~4096 [fadvise_nocache]] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:32:30.062 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:44.604856+0000", 2024-09-14T06:32:30.062 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 165.38476699500001, 2024-09-14T06:32:30.062 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.041986227000000001, 2024-09-14T06:32:30.062 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:32:30.062 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:32:30.062 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:32:30.062 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:32:30.062 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:32:30.062 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:30.062 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 55658 2024-09-14T06:32:30.062 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:30.062 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:32:30.063 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:30.063 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:32:30.063 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.604856+0000", 2024-09-14T06:32:30.063 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:30.063 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:30.063 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:30.063 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:32:30.063 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.604856+0000", 2024-09-14T06:32:30.063 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:30.063 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:30.063 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:30.063 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:32:30.063 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.604856+0000", 2024-09-14T06:32:30.063 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.23e-07 2024-09-14T06:32:30.064 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:30.064 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:30.064 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:32:30.064 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.604857+0000", 2024-09-14T06:32:30.064 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.9999999999999995e-07 2024-09-14T06:32:30.064 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:30.064 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:30.064 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:32:30.064 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.604861+0000", 2024-09-14T06:32:30.064 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.9890000000000003e-06 2024-09-14T06:32:30.064 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:30.064 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:30.064 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:32:30.064 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.604864+0000", 2024-09-14T06:32:30.064 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.7429999999999998e-06 2024-09-14T06:32:30.065 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:30.065 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:30.065 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:32:30.065 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.646744+0000", 2024-09-14T06:32:30.065 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.041880325000000003 2024-09-14T06:32:30.065 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:30.065 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:30.065 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:32:30.065 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.646784+0000", 2024-09-14T06:32:30.065 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.9604000000000002e-05 2024-09-14T06:32:30.065 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:30.065 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:30.065 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:32:30.066 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.646842+0000", 2024-09-14T06:32:30.066 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.8343e-05 2024-09-14T06:32:30.066 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:30.066 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:32:30.066 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:30.066 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:30.066 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:30.066 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:55659 3.0 3:06490baf:::rbd_data.112dca2f6ae1.0000000000000030:17 [read 1736704~4096 [fadvise_nocache]] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:32:30.066 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:44.604867+0000", 2024-09-14T06:32:30.066 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 165.384755842, 2024-09-14T06:32:30.066 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.042097082000000001, 2024-09-14T06:32:30.066 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:32:30.066 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:32:30.066 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:32:30.066 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:32:30.067 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:32:30.067 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:30.067 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 55659 2024-09-14T06:32:30.067 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:30.067 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:32:30.067 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:30.067 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:32:30.067 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.604867+0000", 2024-09-14T06:32:30.067 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:30.067 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:30.067 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:30.067 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:32:30.067 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.604867+0000", 2024-09-14T06:32:30.067 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:30.067 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:30.068 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:30.068 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:32:30.068 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.604867+0000", 2024-09-14T06:32:30.068 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.2000000000000001e-07 2024-09-14T06:32:30.068 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:30.068 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:30.068 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:32:30.068 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.604868+0000", 2024-09-14T06:32:30.068 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.04e-06 2024-09-14T06:32:30.068 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:30.068 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:30.068 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:32:30.068 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.604871+0000", 2024-09-14T06:32:30.068 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.9689999999999999e-06 2024-09-14T06:32:30.069 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:30.069 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:30.069 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:32:30.069 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.604874+0000", 2024-09-14T06:32:30.069 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.5069999999999999e-06 2024-09-14T06:32:30.069 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:30.069 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:30.069 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:32:30.069 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.646874+0000", 2024-09-14T06:32:30.069 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.042000513000000003 2024-09-14T06:32:30.069 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:30.069 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:30.069 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:32:30.069 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.646914+0000", 2024-09-14T06:32:30.070 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.9443999999999997e-05 2024-09-14T06:32:30.070 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:30.070 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:30.070 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:32:30.070 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.646964+0000", 2024-09-14T06:32:30.070 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.0389000000000002e-05 2024-09-14T06:32:30.070 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:30.070 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:32:30.070 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:30.070 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:30.070 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:30.070 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:55660 3.0 3:06490baf:::rbd_data.112dca2f6ae1.0000000000000030:17 [read 1761280~4096 [fadvise_nocache]] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:32:30.070 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:44.604877+0000", 2024-09-14T06:32:30.071 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 165.38474609299999, 2024-09-14T06:32:30.071 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.042196349000000001, 2024-09-14T06:32:30.071 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:32:30.071 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:32:30.071 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:32:30.071 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:32:30.071 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:32:30.071 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:30.071 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 55660 2024-09-14T06:32:30.071 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:30.071 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:32:30.071 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:30.071 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:32:30.071 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.604877+0000", 2024-09-14T06:32:30.071 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:30.071 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:30.072 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:30.072 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:32:30.072 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.604877+0000", 2024-09-14T06:32:30.072 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:30.072 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:30.072 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:30.072 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:32:30.072 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.605810+0000", 2024-09-14T06:32:30.072 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00093336899999999997 2024-09-14T06:32:30.072 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:30.072 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:30.072 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:32:30.072 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.605815+0000", 2024-09-14T06:32:30.072 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.6140000000000002e-06 2024-09-14T06:32:30.073 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:30.073 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:30.073 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:32:30.073 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.605825+0000", 2024-09-14T06:32:30.073 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.8889999999999997e-06 2024-09-14T06:32:30.073 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:30.073 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:30.073 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:32:30.073 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.605831+0000", 2024-09-14T06:32:30.073 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.9800000000000003e-06 2024-09-14T06:32:30.073 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:30.073 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:30.074 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:32:30.074 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.646989+0000", 2024-09-14T06:32:30.074 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.041157878000000002 2024-09-14T06:32:30.074 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:30.074 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:30.074 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:32:30.074 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.647026+0000", 2024-09-14T06:32:30.074 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.7305000000000003e-05 2024-09-14T06:32:30.074 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:30.074 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:30.074 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:32:30.074 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.647073+0000", 2024-09-14T06:32:30.074 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.7314000000000001e-05 2024-09-14T06:32:30.074 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:30.074 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:32:30.075 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:30.075 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:30.075 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:30.075 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:55666 3.0 3:06490baf:::rbd_data.112dca2f6ae1.0000000000000030:17 [read 1908736~4096 [fadvise_nocache]] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:32:30.075 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:44.605898+0000", 2024-09-14T06:32:30.075 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 165.383725347, 2024-09-14T06:32:30.075 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.041862887000000001, 2024-09-14T06:32:30.075 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:32:30.075 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:32:30.075 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:32:30.075 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:32:30.075 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:32:30.075 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:30.075 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 55666 2024-09-14T06:32:30.076 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:30.076 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:32:30.076 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:30.076 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:32:30.076 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.605898+0000", 2024-09-14T06:32:30.076 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:30.076 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:30.076 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:30.076 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:32:30.076 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.605898+0000", 2024-09-14T06:32:30.076 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:30.076 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:30.076 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:30.077 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:32:30.077 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.611874+0000", 2024-09-14T06:32:30.077 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0059764049999999997 2024-09-14T06:32:30.077 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:30.077 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:30.077 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:32:30.077 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.611879+0000", 2024-09-14T06:32:30.077 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.8749999999999999e-06 2024-09-14T06:32:30.077 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:30.077 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:30.077 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:32:30.077 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.611887+0000", 2024-09-14T06:32:30.077 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.4109999999999996e-06 2024-09-14T06:32:30.078 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:30.078 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:30.078 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:32:30.078 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.611893+0000", 2024-09-14T06:32:30.078 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.2959999999999998e-06 2024-09-14T06:32:30.078 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:30.078 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:30.078 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:32:30.078 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.647674+0000", 2024-09-14T06:32:30.078 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.035781281999999998 2024-09-14T06:32:30.078 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:30.078 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:30.078 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:32:30.078 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.647711+0000", 2024-09-14T06:32:30.079 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.7434000000000001e-05 2024-09-14T06:32:30.079 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:30.079 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:30.079 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:32:30.079 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.647760+0000", 2024-09-14T06:32:30.079 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.9184000000000002e-05 2024-09-14T06:32:30.079 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:30.079 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:32:30.079 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:30.079 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:30.079 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:32:30.079 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-14T06:32:30.081 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-14T06:32:30.081 INFO:tasks.ceph.osd.3.smithi094.stderr:2024-09-14T06:32:30.079+0000 7feeeafe8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33655) UID: 0 2024-09-14T06:32:30.181 INFO:tasks.ceph.osd.3.smithi094.stderr:2024-09-14T06:32:30.181+0000 7feeeafe8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33655) UID: 0 2024-09-14T06:32:30.225 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-14T06:32:30.225 INFO:teuthology.orchestra.run.smithi094.stdout: "ops": [], 2024-09-14T06:32:30.225 INFO:teuthology.orchestra.run.smithi094.stdout: "num_ops": 0 2024-09-14T06:32:30.225 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-14T06:32:30.233 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-14T06:32:30.318 INFO:tasks.ceph.osd.2.smithi094.stderr:2024-09-14T06:32:30.317+0000 7faf3269b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33649) UID: 0 2024-09-14T06:32:30.376 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-14T06:32:30.377 INFO:teuthology.orchestra.run.smithi094.stdout: "ops": [], 2024-09-14T06:32:30.377 INFO:teuthology.orchestra.run.smithi094.stdout: "complaint_time": 30, 2024-09-14T06:32:30.377 INFO:teuthology.orchestra.run.smithi094.stdout: "num_blocked_ops": 0 2024-09-14T06:32:30.377 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-14T06:32:30.383 INFO:tasks.ceph.osd.7.smithi138.stderr:2024-09-14T06:32:30.382+0000 7f3c6a21d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33623) UID: 0 2024-09-14T06:32:30.385 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-14T06:32:30.483 INFO:tasks.ceph.osd.4.smithi138.stderr:2024-09-14T06:32:30.482+0000 7f6c653c8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33615) UID: 0 2024-09-14T06:32:30.529 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-14T06:32:30.529 INFO:teuthology.orchestra.run.smithi094.stdout: "size": 20, 2024-09-14T06:32:30.529 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 600, 2024-09-14T06:32:30.529 INFO:teuthology.orchestra.run.smithi094.stdout: "ops": [ 2024-09-14T06:32:30.529 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:30.530 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4397.0:45722 3.9 e64/23 3:91fed098:::rbd_data.112d5a737e72.0000000000000001:head v 64'61, mlcod=62'59)", 2024-09-14T06:32:30.530 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:43.864269+0000", 2024-09-14T06:32:30.530 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 166.66337261699999, 2024-09-14T06:32:30.530 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.003055224, 2024-09-14T06:32:30.530 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:32:30.530 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:32:30.530 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:32:30.530 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:32:30.530 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:30.530 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:32:30.530 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.864269+0000", 2024-09-14T06:32:30.530 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:30.530 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:30.530 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:30.530 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:32:30.531 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.864269+0000", 2024-09-14T06:32:30.531 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:30.531 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:30.531 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:30.531 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:32:30.531 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.864269+0000", 2024-09-14T06:32:30.531 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.41e-07 2024-09-14T06:32:30.531 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:30.531 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:30.531 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:32:30.531 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.865490+0000", 2024-09-14T06:32:30.531 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.001220552 2024-09-14T06:32:30.531 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:30.531 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:30.531 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:32:30.532 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.865506+0000", 2024-09-14T06:32:30.532 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.6263999999999999e-05 2024-09-14T06:32:30.532 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:30.532 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:30.532 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:32:30.532 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.865516+0000", 2024-09-14T06:32:30.532 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.0091999999999999e-05 2024-09-14T06:32:30.532 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:30.532 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:30.532 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:32:30.532 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.865583+0000", 2024-09-14T06:32:30.532 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.6676999999999999e-05 2024-09-14T06:32:30.532 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:30.532 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:30.533 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:32:30.533 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.865610+0000", 2024-09-14T06:32:30.533 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.6529999999999998e-05 2024-09-14T06:32:30.533 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:30.533 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:30.533 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-14T06:32:30.533 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.867281+0000", 2024-09-14T06:32:30.533 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.001671203 2024-09-14T06:32:30.533 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:30.533 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:30.533 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:32:30.533 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.867324+0000", 2024-09-14T06:32:30.533 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.3665000000000003e-05 2024-09-14T06:32:30.534 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:30.534 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:32:30.534 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:30.534 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:30.534 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:30.534 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4397.0:74241 3.0 e73/23 3:06e93fb9:::rbd_data.112d80ca42af.0000000000000000:head v 73'102, mlcod=71'100)", 2024-09-14T06:32:30.534 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:53.876730+0000", 2024-09-14T06:32:30.534 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 156.650911517, 2024-09-14T06:32:30.534 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0079470329999999992, 2024-09-14T06:32:30.534 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:32:30.534 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:32:30.534 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:32:30.534 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:32:30.534 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:30.534 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:32:30.535 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:53.876730+0000", 2024-09-14T06:32:30.535 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:30.535 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:30.535 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:30.535 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:32:30.535 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:53.876730+0000", 2024-09-14T06:32:30.535 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:30.535 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:30.535 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:30.535 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:32:30.535 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:53.876731+0000", 2024-09-14T06:32:30.535 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.0500000000000004e-07 2024-09-14T06:32:30.535 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:30.535 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:30.536 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:32:30.536 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:53.879616+0000", 2024-09-14T06:32:30.536 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0028856680000000001 2024-09-14T06:32:30.536 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:30.536 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:30.536 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:32:30.536 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:53.879643+0000", 2024-09-14T06:32:30.536 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.6120999999999999e-05 2024-09-14T06:32:30.536 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:30.536 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:30.536 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:32:30.536 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:53.879654+0000", 2024-09-14T06:32:30.536 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.1586999999999999e-05 2024-09-14T06:32:30.536 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:30.537 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:30.537 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:32:30.537 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:53.879756+0000", 2024-09-14T06:32:30.537 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00010223899999999999 2024-09-14T06:32:30.537 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:30.537 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:30.537 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:32:30.537 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:53.879783+0000", 2024-09-14T06:32:30.537 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.6913000000000001e-05 2024-09-14T06:32:30.537 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:30.537 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:30.537 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-14T06:32:30.537 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:53.884594+0000", 2024-09-14T06:32:30.537 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0048110729999999999 2024-09-14T06:32:30.538 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:30.538 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:30.538 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:32:30.538 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:53.884677+0000", 2024-09-14T06:32:30.538 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.2927000000000001e-05 2024-09-14T06:32:30.538 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:30.538 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:32:30.538 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:30.538 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:30.538 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:30.538 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4397.0:74243 3.9 e73/23 3:94977f81:::rbd_data.112d80ca42af.0000000000000002:head v 73'94, mlcod=71'92)", 2024-09-14T06:32:30.538 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:53.885156+0000", 2024-09-14T06:32:30.538 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 156.642485599, 2024-09-14T06:32:30.538 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0096009860000000006, 2024-09-14T06:32:30.538 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:32:30.538 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:32:30.539 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:32:30.539 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:32:30.539 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:30.539 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:32:30.539 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:53.885156+0000", 2024-09-14T06:32:30.539 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:30.539 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:30.539 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:30.539 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:32:30.539 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:53.885156+0000", 2024-09-14T06:32:30.539 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:30.539 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:30.539 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:30.540 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:32:30.540 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:53.885156+0000", 2024-09-14T06:32:30.540 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.23e-07 2024-09-14T06:32:30.540 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:30.540 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:30.540 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:32:30.540 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:53.889694+0000", 2024-09-14T06:32:30.540 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0045378709999999997 2024-09-14T06:32:30.540 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:30.540 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:30.540 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:32:30.540 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:53.889720+0000", 2024-09-14T06:32:30.540 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.5259e-05 2024-09-14T06:32:30.540 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:30.541 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:30.541 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:32:30.541 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:53.889731+0000", 2024-09-14T06:32:30.541 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.1739000000000001e-05 2024-09-14T06:32:30.541 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:30.541 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:30.541 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:32:30.541 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:53.889822+0000", 2024-09-14T06:32:30.541 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.1200999999999996e-05 2024-09-14T06:32:30.541 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:30.541 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:30.541 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:32:30.542 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:53.889850+0000", 2024-09-14T06:32:30.542 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.7084999999999999e-05 2024-09-14T06:32:30.542 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:30.542 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:30.542 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-14T06:32:30.542 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:53.894703+0000", 2024-09-14T06:32:30.542 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0048532690000000003 2024-09-14T06:32:30.542 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:30.542 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:30.542 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:32:30.542 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:53.894757+0000", 2024-09-14T06:32:30.542 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.4339000000000003e-05 2024-09-14T06:32:30.542 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:30.542 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:32:30.542 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:30.542 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:30.543 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:30.543 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4397.0:74249 3.0 e73/23 3:009b0225:::rbd_data.112d80ca42af.0000000000000009:head v 73'103, mlcod=71'101)", 2024-09-14T06:32:30.543 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:53.893219+0000", 2024-09-14T06:32:30.543 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 156.634423121, 2024-09-14T06:32:30.543 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0065381980000000003, 2024-09-14T06:32:30.543 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:32:30.543 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:32:30.543 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:32:30.543 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:32:30.543 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:30.543 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:32:30.543 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:53.893219+0000", 2024-09-14T06:32:30.543 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:30.543 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:30.544 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:30.544 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:32:30.544 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:53.893219+0000", 2024-09-14T06:32:30.544 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:30.544 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:30.544 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:30.544 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:32:30.544 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:53.893221+0000", 2024-09-14T06:32:30.544 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.4329999999999998e-06 2024-09-14T06:32:30.544 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:30.544 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:30.544 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:32:30.544 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:53.895968+0000", 2024-09-14T06:32:30.544 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.002746746 2024-09-14T06:32:30.545 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:30.545 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:30.545 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:32:30.545 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:53.895988+0000", 2024-09-14T06:32:30.545 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.0227000000000001e-05 2024-09-14T06:32:30.545 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:30.545 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:30.545 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:32:30.545 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:53.896000+0000", 2024-09-14T06:32:30.545 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.2359e-05 2024-09-14T06:32:30.545 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:30.545 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:30.545 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:32:30.545 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:53.896084+0000", 2024-09-14T06:32:30.545 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.3659999999999995e-05 2024-09-14T06:32:30.546 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:30.546 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:30.546 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:32:30.546 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:53.896109+0000", 2024-09-14T06:32:30.546 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.5259999999999999e-05 2024-09-14T06:32:30.546 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:30.546 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:30.546 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-14T06:32:30.546 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:53.899704+0000", 2024-09-14T06:32:30.546 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0035942209999999999 2024-09-14T06:32:30.546 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:30.546 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:30.546 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:32:30.546 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:53.899757+0000", 2024-09-14T06:32:30.546 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.3291999999999997e-05 2024-09-14T06:32:30.546 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:30.547 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:32:30.547 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:30.547 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:30.547 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:30.547 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4397.0:78263 3.0 e77/23 3:00899b22:::rbd_data.112ddfbec471.0000000000000016:head v 77'104, mlcod=73'102)", 2024-09-14T06:32:30.547 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:57.634578+0000", 2024-09-14T06:32:30.547 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 152.89306406899999, 2024-09-14T06:32:30.547 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.002372454, 2024-09-14T06:32:30.547 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:32:30.547 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:32:30.547 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:32:30.547 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:32:30.547 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:30.547 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:32:30.547 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:57.634578+0000", 2024-09-14T06:32:30.547 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:30.548 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:30.548 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:30.548 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:32:30.548 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:57.634578+0000", 2024-09-14T06:32:30.548 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:30.548 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:30.548 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:30.548 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:32:30.548 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:57.634578+0000", 2024-09-14T06:32:30.548 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.5699999999999998e-07 2024-09-14T06:32:30.548 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:30.548 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:30.548 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:32:30.548 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:57.635347+0000", 2024-09-14T06:32:30.549 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00076848300000000003 2024-09-14T06:32:30.549 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:30.549 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:30.549 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:32:30.549 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:57.635361+0000", 2024-09-14T06:32:30.549 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.4238e-05 2024-09-14T06:32:30.549 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:30.549 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:30.549 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:32:30.549 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:57.635371+0000", 2024-09-14T06:32:30.549 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.0087e-05 2024-09-14T06:32:30.549 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:30.549 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:30.549 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:32:30.550 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:57.635439+0000", 2024-09-14T06:32:30.550 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.7953000000000005e-05 2024-09-14T06:32:30.550 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:30.550 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:30.550 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:32:30.550 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:57.635464+0000", 2024-09-14T06:32:30.550 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.5062000000000002e-05 2024-09-14T06:32:30.550 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:30.550 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:30.550 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-14T06:32:30.550 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:57.636914+0000", 2024-09-14T06:32:30.550 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.001449625 2024-09-14T06:32:30.550 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:30.550 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:30.550 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:32:30.550 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:57.636950+0000", 2024-09-14T06:32:30.551 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.6548999999999999e-05 2024-09-14T06:32:30.551 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:30.551 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:32:30.551 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:30.551 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:30.551 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:30.551 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4397.0:78604 3.9 e77/23 3:94c62a0e:::rbd_data.112ddfbec471.000000000000002c:head v 77'96, mlcod=73'94)", 2024-09-14T06:32:30.551 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:57.677568+0000", 2024-09-14T06:32:30.551 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 152.850073681, 2024-09-14T06:32:30.551 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.002564846, 2024-09-14T06:32:30.551 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:32:30.551 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:32:30.551 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:32:30.551 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:32:30.551 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:30.552 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:32:30.552 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:57.677568+0000", 2024-09-14T06:32:30.552 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:30.552 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:30.552 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:30.552 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:32:30.552 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:57.677568+0000", 2024-09-14T06:32:30.552 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:30.552 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:30.552 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:30.552 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:32:30.552 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:57.677568+0000", 2024-09-14T06:32:30.552 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.8900000000000001e-07 2024-09-14T06:32:30.552 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:30.552 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:30.552 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:32:30.553 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:57.678488+0000", 2024-09-14T06:32:30.553 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00091963899999999998 2024-09-14T06:32:30.553 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:30.553 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:30.553 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:32:30.553 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:57.678500+0000", 2024-09-14T06:32:30.553 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.1902000000000001e-05 2024-09-14T06:32:30.553 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:30.553 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:30.553 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:32:30.553 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:57.678507+0000", 2024-09-14T06:32:30.553 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.7739999999999999e-06 2024-09-14T06:32:30.553 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:30.553 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:30.554 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:32:30.554 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:57.678586+0000", 2024-09-14T06:32:30.554 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.9103000000000005e-05 2024-09-14T06:32:30.554 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:30.554 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:30.554 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:32:30.554 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:57.678614+0000", 2024-09-14T06:32:30.554 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.7912e-05 2024-09-14T06:32:30.554 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:30.554 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:30.554 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-14T06:32:30.554 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:57.680075+0000", 2024-09-14T06:32:30.554 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0014613370000000001 2024-09-14T06:32:30.554 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:30.555 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:30.555 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:32:30.555 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:57.680133+0000", 2024-09-14T06:32:30.555 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.7890000000000003e-05 2024-09-14T06:32:30.555 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:30.555 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:32:30.555 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:30.555 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:30.555 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:30.555 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4397.0:149885 3.9 e122/23 3:951acef3:::rbd_data.112d1bb7b269.0000000000000005:head v 122'176, mlcod=122'172)", 2024-09-14T06:32:30.555 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:30:44.739092+0000", 2024-09-14T06:32:30.555 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 105.788549371, 2024-09-14T06:32:30.555 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0028236659999999998, 2024-09-14T06:32:30.555 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:32:30.555 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:32:30.556 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:32:30.556 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:32:30.556 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:30.556 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:32:30.556 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.739092+0000", 2024-09-14T06:32:30.556 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:30.557 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:30.557 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:30.557 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:32:30.557 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.739092+0000", 2024-09-14T06:32:30.557 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:30.557 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:30.557 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:30.557 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:32:30.557 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.739093+0000", 2024-09-14T06:32:30.557 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.0109999999999999e-06 2024-09-14T06:32:30.557 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:30.557 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:30.557 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:32:30.557 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.740189+0000", 2024-09-14T06:32:30.557 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00109527 2024-09-14T06:32:30.558 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:30.558 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:30.558 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:32:30.558 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.740199+0000", 2024-09-14T06:32:30.558 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.0458999999999999e-05 2024-09-14T06:32:30.558 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:30.558 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:30.558 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:32:30.558 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.740207+0000", 2024-09-14T06:32:30.558 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.4499999999999998e-06 2024-09-14T06:32:30.558 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:30.558 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:30.558 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:32:30.558 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.740295+0000", 2024-09-14T06:32:30.559 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.8740999999999996e-05 2024-09-14T06:32:30.559 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:30.559 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:30.559 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:32:30.559 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.740317+0000", 2024-09-14T06:32:30.559 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.1523e-05 2024-09-14T06:32:30.559 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:30.559 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:30.559 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-14T06:32:30.560 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.741865+0000", 2024-09-14T06:32:30.560 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.001548315 2024-09-14T06:32:30.560 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:30.560 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:30.560 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:32:30.560 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.741916+0000", 2024-09-14T06:32:30.560 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.0896999999999997e-05 2024-09-14T06:32:30.560 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:30.560 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:32:30.560 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:30.560 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:30.560 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:30.560 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4397.0:149933 3.4 e122/23 3:234f5229:::rbd_data.112d1bb7b269.0000000000000009:head v 122'221, mlcod=120'218)", 2024-09-14T06:32:30.560 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:30:44.743849+0000", 2024-09-14T06:32:30.561 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 105.78379295400001, 2024-09-14T06:32:30.561 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0031479519999999999, 2024-09-14T06:32:30.561 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:32:30.561 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:32:30.561 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:32:30.561 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:32:30.561 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:30.561 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:32:30.561 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.743849+0000", 2024-09-14T06:32:30.561 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:30.561 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:30.561 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:30.561 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:32:30.561 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.743849+0000", 2024-09-14T06:32:30.561 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:30.562 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:30.562 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:30.562 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:32:30.562 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.743850+0000", 2024-09-14T06:32:30.562 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.279e-06 2024-09-14T06:32:30.562 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:30.562 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:30.562 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:32:30.562 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.745151+0000", 2024-09-14T06:32:30.562 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.001300495 2024-09-14T06:32:30.562 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:30.562 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:30.562 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:32:30.562 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.745167+0000", 2024-09-14T06:32:30.562 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.6478e-05 2024-09-14T06:32:30.563 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:30.563 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:30.563 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:32:30.563 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.745175+0000", 2024-09-14T06:32:30.563 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.2679999999999994e-06 2024-09-14T06:32:30.563 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:30.563 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:30.563 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:32:30.563 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.745226+0000", 2024-09-14T06:32:30.563 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.0192999999999999e-05 2024-09-14T06:32:30.563 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:30.563 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:30.563 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:32:30.563 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.745247+0000", 2024-09-14T06:32:30.563 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.1767e-05 2024-09-14T06:32:30.563 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:30.564 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:30.564 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-14T06:32:30.564 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.746952+0000", 2024-09-14T06:32:30.564 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0017042769999999999 2024-09-14T06:32:30.564 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:30.564 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:30.564 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:32:30.564 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.746997+0000", 2024-09-14T06:32:30.564 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.5194999999999997e-05 2024-09-14T06:32:30.564 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:30.564 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:32:30.564 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:30.564 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:30.564 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:30.565 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4397.0:150494 3.9 e122/23 3:955152a4:::rbd_object_map.112d1bb7b269.0000000000000038:head v 122'194, mlcod=122'190)", 2024-09-14T06:32:30.565 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:30:44.781585+0000", 2024-09-14T06:32:30.565 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 105.74605706200001, 2024-09-14T06:32:30.565 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0026957499999999998, 2024-09-14T06:32:30.565 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:32:30.565 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:32:30.565 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:32:30.565 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:32:30.565 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:30.565 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:32:30.565 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.781585+0000", 2024-09-14T06:32:30.565 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:30.565 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:30.565 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:30.565 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:32:30.566 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.781585+0000", 2024-09-14T06:32:30.566 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:30.566 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:30.566 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:30.566 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:32:30.566 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.781585+0000", 2024-09-14T06:32:30.566 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.5000000000000002e-07 2024-09-14T06:32:30.566 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:30.566 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:30.566 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:32:30.566 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.781594+0000", 2024-09-14T06:32:30.566 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.4640000000000006e-06 2024-09-14T06:32:30.566 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:30.566 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:30.566 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:32:30.567 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.781604+0000", 2024-09-14T06:32:30.567 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.0071999999999999e-05 2024-09-14T06:32:30.567 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:30.567 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:30.567 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:32:30.567 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.781610+0000", 2024-09-14T06:32:30.567 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.3609999999999997e-06 2024-09-14T06:32:30.567 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:30.567 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:30.567 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:32:30.567 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.781676+0000", 2024-09-14T06:32:30.567 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.5963000000000005e-05 2024-09-14T06:32:30.567 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:30.567 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:30.567 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:32:30.568 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.781696+0000", 2024-09-14T06:32:30.568 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.9278000000000001e-05 2024-09-14T06:32:30.568 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:30.568 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:30.568 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-14T06:32:30.568 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.784231+0000", 2024-09-14T06:32:30.568 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.002535093 2024-09-14T06:32:30.568 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:30.568 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:30.568 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:32:30.568 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.784280+0000", 2024-09-14T06:32:30.568 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.9869e-05 2024-09-14T06:32:30.568 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:30.568 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:32:30.568 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:30.568 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:30.569 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:30.569 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4397.0:150898 3.9 e122/23 3:97c98f90:::rbd_data.112d1bb7b269.0000000000000025:head v 122'206, mlcod=122'203)", 2024-09-14T06:32:30.569 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:30:44.807232+0000", 2024-09-14T06:32:30.569 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 105.720410145, 2024-09-14T06:32:30.569 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0022297889999999998, 2024-09-14T06:32:30.569 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:32:30.569 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:32:30.569 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:32:30.569 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:32:30.569 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:30.569 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:32:30.569 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.807232+0000", 2024-09-14T06:32:30.569 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:30.569 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:30.569 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:30.569 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:32:30.570 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.807232+0000", 2024-09-14T06:32:30.570 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:30.570 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:30.570 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:30.570 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:32:30.570 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.807233+0000", 2024-09-14T06:32:30.570 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.5710000000000001e-06 2024-09-14T06:32:30.570 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:30.570 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:30.570 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:32:30.570 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.807944+0000", 2024-09-14T06:32:30.570 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00071056299999999999 2024-09-14T06:32:30.570 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:30.570 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:30.571 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:32:30.571 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.807961+0000", 2024-09-14T06:32:30.571 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.7689e-05 2024-09-14T06:32:30.571 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:30.571 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:30.571 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:32:30.571 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.807971+0000", 2024-09-14T06:32:30.571 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.2e-06 2024-09-14T06:32:30.571 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:30.571 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:30.571 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:32:30.571 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.808050+0000", 2024-09-14T06:32:30.571 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.9308999999999996e-05 2024-09-14T06:32:30.571 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:30.571 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:30.571 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:32:30.572 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.808074+0000", 2024-09-14T06:32:30.572 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.4147000000000001e-05 2024-09-14T06:32:30.572 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:30.572 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:30.572 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-14T06:32:30.572 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.809415+0000", 2024-09-14T06:32:30.572 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.001341257 2024-09-14T06:32:30.572 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:30.572 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:30.572 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:32:30.572 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.809461+0000", 2024-09-14T06:32:30.572 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.6053000000000001e-05 2024-09-14T06:32:30.572 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:30.572 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:32:30.573 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:30.573 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:30.573 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:30.573 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4397.0:202551 3.9 e166/23 3:965c09b6:::rbd_data.112d3bd84cb0.000000000000000a:head v 166'416, mlcod=164'414)", 2024-09-14T06:32:30.573 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:31:30.657011+0000", 2024-09-14T06:32:30.573 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 59.870631111999998, 2024-09-14T06:32:30.573 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.002509262, 2024-09-14T06:32:30.573 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:32:30.573 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:32:30.573 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:32:30.573 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:32:30.573 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:30.573 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:32:30.573 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:30.657011+0000", 2024-09-14T06:32:30.573 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:30.573 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:30.574 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:30.574 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:32:30.574 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:30.657011+0000", 2024-09-14T06:32:30.574 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:30.574 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:30.574 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:30.574 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:32:30.574 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:30.657011+0000", 2024-09-14T06:32:30.574 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.08e-07 2024-09-14T06:32:30.574 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:30.574 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:30.574 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:32:30.574 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:30.658467+0000", 2024-09-14T06:32:30.575 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.001455708 2024-09-14T06:32:30.575 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:30.575 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:30.575 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:32:30.575 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:30.658484+0000", 2024-09-14T06:32:30.575 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.6844e-05 2024-09-14T06:32:30.575 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:30.575 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:30.575 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:32:30.575 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:30.658493+0000", 2024-09-14T06:32:30.575 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.2790000000000004e-06 2024-09-14T06:32:30.575 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:30.576 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:30.576 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:32:30.576 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:30.658559+0000", 2024-09-14T06:32:30.576 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.6459999999999997e-05 2024-09-14T06:32:30.576 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:30.576 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:30.576 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:32:30.576 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:30.658574+0000", 2024-09-14T06:32:30.576 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.5048000000000001e-05 2024-09-14T06:32:30.576 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:30.576 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:30.576 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-14T06:32:30.576 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:30.659473+0000", 2024-09-14T06:32:30.576 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00089842800000000005 2024-09-14T06:32:30.576 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:30.577 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:30.577 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:32:30.577 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:30.659520+0000", 2024-09-14T06:32:30.577 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.7086999999999998e-05 2024-09-14T06:32:30.577 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:30.577 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:32:30.577 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:30.577 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:30.577 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:30.577 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4397.0:202665 3.0 e166/23 3:00eeadfb:::rbd_data.112d3bd84cb0.000000000000000e:head v 166'920, mlcod=163'918)", 2024-09-14T06:32:30.577 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:31:30.662434+0000", 2024-09-14T06:32:30.577 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 59.865207323, 2024-09-14T06:32:30.577 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.002522121, 2024-09-14T06:32:30.577 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:32:30.578 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:32:30.578 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:32:30.578 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:32:30.578 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:30.578 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:32:30.578 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:30.662434+0000", 2024-09-14T06:32:30.578 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:30.578 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:30.578 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:30.578 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:32:30.578 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:30.662434+0000", 2024-09-14T06:32:30.578 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:30.578 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:30.578 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:30.578 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:32:30.578 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:30.662436+0000", 2024-09-14T06:32:30.579 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.257e-06 2024-09-14T06:32:30.579 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:30.579 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:30.579 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:32:30.579 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:30.662527+0000", 2024-09-14T06:32:30.579 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.1644000000000006e-05 2024-09-14T06:32:30.579 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:30.579 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:30.579 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:32:30.579 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:30.662546+0000", 2024-09-14T06:32:30.579 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.8998e-05 2024-09-14T06:32:30.579 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:30.579 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:30.579 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:32:30.579 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:30.662556+0000", 2024-09-14T06:32:30.579 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.9809999999999998e-06 2024-09-14T06:32:30.580 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:30.580 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:30.580 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:32:30.580 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:30.662621+0000", 2024-09-14T06:32:30.580 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.4597000000000005e-05 2024-09-14T06:32:30.580 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:30.580 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:30.580 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:32:30.580 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:30.662643+0000", 2024-09-14T06:32:30.580 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.1608e-05 2024-09-14T06:32:30.580 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:30.580 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:30.580 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-14T06:32:30.580 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:30.664909+0000", 2024-09-14T06:32:30.580 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0022668689999999999 2024-09-14T06:32:30.581 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:30.581 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:30.581 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:32:30.581 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:30.664957+0000", 2024-09-14T06:32:30.581 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.7166999999999998e-05 2024-09-14T06:32:30.581 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:30.581 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:32:30.581 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:30.581 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:30.581 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:30.581 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4397.0:204589 3.4 e166/23 3:234552f0:::rbd_data.112d3bd84cb0.000000000000003f:head v 166'499, mlcod=165'497)", 2024-09-14T06:32:30.581 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:31:30.772894+0000", 2024-09-14T06:32:30.581 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 59.754747467000001, 2024-09-14T06:32:30.581 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0034564650000000001, 2024-09-14T06:32:30.581 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:32:30.581 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:32:30.582 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:32:30.582 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:32:30.582 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:30.582 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:32:30.582 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:30.772894+0000", 2024-09-14T06:32:30.582 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:30.582 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:30.582 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:30.582 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:32:30.582 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:30.772894+0000", 2024-09-14T06:32:30.582 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:30.582 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:30.582 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:30.582 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:32:30.582 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:30.772896+0000", 2024-09-14T06:32:30.582 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.5379999999999999e-06 2024-09-14T06:32:30.583 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:30.583 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:30.583 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:32:30.583 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:30.773042+0000", 2024-09-14T06:32:30.583 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000146107 2024-09-14T06:32:30.583 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:30.583 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:30.583 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:32:30.583 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:30.773061+0000", 2024-09-14T06:32:30.583 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.8862999999999999e-05 2024-09-14T06:32:30.583 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:30.583 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:30.583 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:32:30.583 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:30.773071+0000", 2024-09-14T06:32:30.583 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.0227000000000001e-05 2024-09-14T06:32:30.583 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:30.584 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:30.584 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:32:30.584 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:30.773149+0000", 2024-09-14T06:32:30.584 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.8071999999999994e-05 2024-09-14T06:32:30.584 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:30.584 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:30.584 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:32:30.584 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:30.773171+0000", 2024-09-14T06:32:30.584 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.1699999999999999e-05 2024-09-14T06:32:30.584 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:30.584 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:30.584 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-14T06:32:30.584 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:30.776285+0000", 2024-09-14T06:32:30.584 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0031142050000000001 2024-09-14T06:32:30.584 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:30.584 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:30.585 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:32:30.585 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:30.776351+0000", 2024-09-14T06:32:30.585 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.5753000000000006e-05 2024-09-14T06:32:30.585 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:30.585 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:32:30.585 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:30.585 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:30.585 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:30.585 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4397.0:204598 3.4 e166/23 3:245d069e:::rbd_data.112d3bd84cb0.0000000000000041:head v 166'500, mlcod=165'497)", 2024-09-14T06:32:30.585 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:31:30.775247+0000", 2024-09-14T06:32:30.585 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 59.752394344000002, 2024-09-14T06:32:30.585 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00318235, 2024-09-14T06:32:30.585 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:32:30.585 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:32:30.585 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:32:30.586 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:32:30.586 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:30.586 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:32:30.586 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:30.775247+0000", 2024-09-14T06:32:30.586 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:30.586 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:30.586 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:30.586 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:32:30.586 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:30.775247+0000", 2024-09-14T06:32:30.586 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:30.586 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:30.586 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:30.586 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:32:30.586 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:30.775249+0000", 2024-09-14T06:32:30.587 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.265e-06 2024-09-14T06:32:30.587 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:30.587 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:30.587 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:32:30.587 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:30.775340+0000", 2024-09-14T06:32:30.587 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.1131999999999994e-05 2024-09-14T06:32:30.587 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:30.587 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:30.587 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:32:30.587 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:30.775352+0000", 2024-09-14T06:32:30.587 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.2294e-05 2024-09-14T06:32:30.587 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:30.587 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:30.587 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:32:30.587 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:30.775361+0000", 2024-09-14T06:32:30.588 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.6950000000000006e-06 2024-09-14T06:32:30.588 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:30.588 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:30.588 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:32:30.588 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:30.775439+0000", 2024-09-14T06:32:30.588 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.7910000000000005e-05 2024-09-14T06:32:30.588 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:30.588 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:30.588 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:32:30.588 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:30.775460+0000", 2024-09-14T06:32:30.588 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.0900999999999999e-05 2024-09-14T06:32:30.588 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:30.588 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:30.588 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-14T06:32:30.589 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:30.778385+0000", 2024-09-14T06:32:30.589 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.002925539 2024-09-14T06:32:30.589 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:30.589 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:30.589 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:32:30.589 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:30.778430+0000", 2024-09-14T06:32:30.589 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.4613999999999997e-05 2024-09-14T06:32:30.589 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:30.589 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:32:30.589 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:30.589 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:30.589 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:30.589 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4397.0:204975 3.4 e166/23 3:20972d18:::rbd_data.112d3bd84cb0.0000000000000046:head v 166'501, mlcod=166'499)", 2024-09-14T06:32:30.589 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:31:30.787253+0000", 2024-09-14T06:32:30.590 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 59.740388314999997, 2024-09-14T06:32:30.590 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.002248318, 2024-09-14T06:32:30.590 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:32:30.590 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:32:30.590 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:32:30.590 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:32:30.590 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:30.590 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:32:30.590 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:30.787253+0000", 2024-09-14T06:32:30.590 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:30.590 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:30.590 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:30.590 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:32:30.590 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:30.787253+0000", 2024-09-14T06:32:30.590 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:30.590 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:30.591 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:30.591 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:32:30.591 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:30.787255+0000", 2024-09-14T06:32:30.591 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.271e-06 2024-09-14T06:32:30.591 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:30.591 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:30.591 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:32:30.591 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:30.787442+0000", 2024-09-14T06:32:30.591 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00018744000000000001 2024-09-14T06:32:30.591 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:30.591 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:30.591 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:32:30.591 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:30.787461+0000", 2024-09-14T06:32:30.591 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.9054000000000001e-05 2024-09-14T06:32:30.591 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:30.592 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:30.592 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:32:30.592 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:30.787470+0000", 2024-09-14T06:32:30.592 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.0929999999999992e-06 2024-09-14T06:32:30.592 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:30.592 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:30.592 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:32:30.592 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:30.788004+0000", 2024-09-14T06:32:30.592 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00053359399999999997 2024-09-14T06:32:30.592 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:30.592 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:30.592 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:32:30.592 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:30.788034+0000", 2024-09-14T06:32:30.592 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.9635e-05 2024-09-14T06:32:30.592 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:30.593 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:30.593 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-14T06:32:30.593 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:30.789457+0000", 2024-09-14T06:32:30.593 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.001423487 2024-09-14T06:32:30.593 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:30.593 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:30.593 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:32:30.593 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:30.789502+0000", 2024-09-14T06:32:30.593 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.4743999999999997e-05 2024-09-14T06:32:30.593 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:30.593 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:32:30.594 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:30.594 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:30.594 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:30.594 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4397.0:212188 3.4 e166/23 3:23871bc7:::rbd_data.112d3bd84cb0.00000000000000fa:head v 166'508, mlcod=166'506)", 2024-09-14T06:32:30.594 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:31:31.391415+0000", 2024-09-14T06:32:30.594 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 59.136226733000001, 2024-09-14T06:32:30.594 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0022039049999999999, 2024-09-14T06:32:30.594 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:32:30.594 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:32:30.594 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:32:30.594 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:32:30.594 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:30.594 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:32:30.594 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:31.391415+0000", 2024-09-14T06:32:30.594 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:30.595 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:30.595 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:30.595 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:32:30.595 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:31.391415+0000", 2024-09-14T06:32:30.595 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:30.595 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:30.595 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:30.595 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:32:30.595 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:31.391416+0000", 2024-09-14T06:32:30.595 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.44e-06 2024-09-14T06:32:30.595 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:30.595 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:30.595 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:32:30.595 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:31.391554+0000", 2024-09-14T06:32:30.596 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000137119 2024-09-14T06:32:30.596 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:30.596 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:30.596 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:32:30.596 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:31.391570+0000", 2024-09-14T06:32:30.596 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.6827e-05 2024-09-14T06:32:30.596 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:30.596 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:30.596 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:32:30.596 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:31.391580+0000", 2024-09-14T06:32:30.596 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.3139999999999993e-06 2024-09-14T06:32:30.596 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:30.596 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:30.596 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:32:30.596 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:31.391664+0000", 2024-09-14T06:32:30.597 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.3757999999999993e-05 2024-09-14T06:32:30.597 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:30.597 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:30.597 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:32:30.597 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:31.391692+0000", 2024-09-14T06:32:30.597 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.8898e-05 2024-09-14T06:32:30.597 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:30.597 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:30.597 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-14T06:32:30.597 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:31.393584+0000", 2024-09-14T06:32:30.597 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.001891752 2024-09-14T06:32:30.597 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:30.597 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:30.597 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:32:30.597 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:31.393619+0000", 2024-09-14T06:32:30.597 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.4796999999999999e-05 2024-09-14T06:32:30.598 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:30.598 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:32:30.598 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:30.598 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:30.598 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:30.598 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "MOSDPGPush(3.5 169/168 [PushOp(3:a7c12b08:::rbd_data.112ddfbec471.000000000000000e:head, version: 77'100, data_included: [0~1048576], data_size: 1048576, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(3:a7c12b08:::rbd_data.112ddfbec471.000000000000000e:head@77'100, size: 1048576, copy_subset: [(0, 1048576)], clone_subset: {}, snapset: 0=[]:{}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 1048576, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2024-09-14T06:32:30.598 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:31:34.540707+0000", 2024-09-14T06:32:30.598 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 55.986934968, 2024-09-14T06:32:30.598 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0023637369999999999, 2024-09-14T06:32:30.598 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:32:30.598 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:32:30.598 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:32:30.598 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:32:30.598 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:30.598 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:32:30.599 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:34.540707+0000", 2024-09-14T06:32:30.599 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:30.599 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:30.599 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:30.599 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:32:30.599 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:34.540707+0000", 2024-09-14T06:32:30.599 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:30.599 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:30.599 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:30.599 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:32:30.599 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:34.540708+0000", 2024-09-14T06:32:30.599 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.7199999999999998e-07 2024-09-14T06:32:30.599 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:30.600 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:30.600 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:32:30.600 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:34.542150+0000", 2024-09-14T06:32:30.600 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0014423229999999999 2024-09-14T06:32:30.600 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:30.600 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:30.600 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:32:30.600 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:34.542169+0000", 2024-09-14T06:32:30.600 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.8756000000000001e-05 2024-09-14T06:32:30.600 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:30.600 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:30.600 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:32:30.600 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:34.542180+0000", 2024-09-14T06:32:30.600 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.1051e-05 2024-09-14T06:32:30.600 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:30.600 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:30.601 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:32:30.601 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:34.542275+0000", 2024-09-14T06:32:30.601 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.5122000000000002e-05 2024-09-14T06:32:30.601 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:30.601 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:30.601 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:32:30.601 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:34.542284+0000", 2024-09-14T06:32:30.601 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.9190000000000005e-06 2024-09-14T06:32:30.601 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:30.601 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:30.601 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:32:30.601 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:34.543071+0000", 2024-09-14T06:32:30.601 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00078679399999999995 2024-09-14T06:32:30.601 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:30.602 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:32:30.602 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:30.602 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:30.602 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:30.602 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "MOSDPGPush(3.5 169/168 [PushOp(3:a7a54476:::rbd_data.112d1bb7b269.0000000000000038:head, version: 122'581, data_included: [0~524288], data_size: 524288, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(3:a7a54476:::rbd_data.112d1bb7b269.0000000000000038:head@122'581, size: 524288, copy_subset: [(0, 524288)], clone_subset: {}, snapset: 0=[]:{}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 524288, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2024-09-14T06:32:30.602 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:31:34.667757+0000", 2024-09-14T06:32:30.602 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 55.859884921999999, 2024-09-14T06:32:30.602 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.002750382, 2024-09-14T06:32:30.602 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:32:30.602 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:32:30.602 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:32:30.602 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:32:30.602 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:30.602 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:32:30.602 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:34.667757+0000", 2024-09-14T06:32:30.603 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:30.603 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:30.603 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:30.603 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:32:30.603 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:34.667757+0000", 2024-09-14T06:32:30.603 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:30.603 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:30.603 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:30.603 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:32:30.603 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:34.667758+0000", 2024-09-14T06:32:30.603 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.3599999999999999e-06 2024-09-14T06:32:30.603 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:30.603 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:30.603 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:32:30.603 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:34.669375+0000", 2024-09-14T06:32:30.603 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.001616644 2024-09-14T06:32:30.604 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:30.604 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:30.604 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:32:30.604 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:34.669404+0000", 2024-09-14T06:32:30.604 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.9300999999999999e-05 2024-09-14T06:32:30.604 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:30.604 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:30.604 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:32:30.604 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:34.669428+0000", 2024-09-14T06:32:30.604 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.3788e-05 2024-09-14T06:32:30.604 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:30.604 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:30.604 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:32:30.604 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:34.669543+0000", 2024-09-14T06:32:30.604 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000115462 2024-09-14T06:32:30.604 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:30.605 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:30.605 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:32:30.605 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:34.669559+0000", 2024-09-14T06:32:30.605 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.5167999999999999e-05 2024-09-14T06:32:30.605 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:30.605 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:30.605 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:32:30.605 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:34.670507+0000", 2024-09-14T06:32:30.605 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00094865900000000003 2024-09-14T06:32:30.605 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:30.605 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:32:30.605 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:30.605 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:30.605 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:30.605 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "MOSDPGPush(3.5 170/168 [PushOp(3:a162138c:::rbd_data.112d1bb7b269.000000000000003c:head, version: 122'582, data_included: [0~524288], data_size: 524288, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(3:a162138c:::rbd_data.112d1bb7b269.000000000000003c:head@122'582, size: 524288, copy_subset: [(0, 524288)], clone_subset: {}, snapset: 0=[]:{}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 524288, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2024-09-14T06:32:30.606 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:31:34.670110+0000", 2024-09-14T06:32:30.606 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 55.857531973, 2024-09-14T06:32:30.606 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0030066229999999999, 2024-09-14T06:32:30.606 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:32:30.606 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:32:30.606 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:32:30.606 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:32:30.606 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:30.606 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:32:30.606 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:34.670110+0000", 2024-09-14T06:32:30.606 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:30.606 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:30.606 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:30.606 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:32:30.606 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:34.670110+0000", 2024-09-14T06:32:30.607 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:30.607 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:30.607 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:30.607 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:32:30.607 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:34.670111+0000", 2024-09-14T06:32:30.607 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.6499999999999998e-07 2024-09-14T06:32:30.607 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:30.607 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:30.607 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:32:30.607 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:34.671507+0000", 2024-09-14T06:32:30.607 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.001396284 2024-09-14T06:32:30.607 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:30.607 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:30.607 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:32:30.608 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:34.671530+0000", 2024-09-14T06:32:30.608 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.3023e-05 2024-09-14T06:32:30.608 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:30.608 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:30.608 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:32:30.608 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:34.671548+0000", 2024-09-14T06:32:30.608 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.8116000000000001e-05 2024-09-14T06:32:30.608 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:30.608 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:30.608 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:32:30.608 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:34.671775+0000", 2024-09-14T06:32:30.608 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00022699600000000001 2024-09-14T06:32:30.608 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:30.608 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:30.608 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:32:30.609 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:34.671790+0000", 2024-09-14T06:32:30.609 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.539e-05 2024-09-14T06:32:30.609 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:30.609 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:30.609 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:32:30.609 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:34.673116+0000", 2024-09-14T06:32:30.609 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0013259490000000001 2024-09-14T06:32:30.609 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:30.609 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:32:30.609 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:30.609 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:30.609 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:30.609 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4397.0:232694 3.9 e207/23 3:95763440:::rbd_data.112d838b9898.0000000000000006:head v 207'513, mlcod=203'511)", 2024-09-14T06:32:30.609 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:32:12.742327+0000", 2024-09-14T06:32:30.610 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 17.785315013000002, 2024-09-14T06:32:30.610 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.002504952, 2024-09-14T06:32:30.610 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:32:30.612 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:32:30.612 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:32:30.612 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:32:30.612 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:30.612 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:32:30.612 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:12.742327+0000", 2024-09-14T06:32:30.613 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:30.613 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:30.613 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:30.613 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:32:30.613 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:12.742327+0000", 2024-09-14T06:32:30.613 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:30.613 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:30.613 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:30.613 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:32:30.613 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:12.742327+0000", 2024-09-14T06:32:30.613 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.3999999999999997e-07 2024-09-14T06:32:30.613 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:30.613 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:30.613 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:32:30.613 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:12.742332+0000", 2024-09-14T06:32:30.614 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.3569999999999997e-06 2024-09-14T06:32:30.614 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:30.614 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:30.614 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:32:30.614 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:12.742360+0000", 2024-09-14T06:32:30.614 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.7650000000000001e-05 2024-09-14T06:32:30.614 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:30.614 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:30.614 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:32:30.614 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:12.742370+0000", 2024-09-14T06:32:30.614 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.0115000000000001e-05 2024-09-14T06:32:30.614 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:30.614 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:30.614 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:32:30.614 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:12.742433+0000", 2024-09-14T06:32:30.615 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.2364000000000001e-05 2024-09-14T06:32:30.615 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:30.615 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:30.615 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:32:30.615 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:12.742451+0000", 2024-09-14T06:32:30.615 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.8250999999999999e-05 2024-09-14T06:32:30.615 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:30.615 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:30.615 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-14T06:32:30.615 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:12.744776+0000", 2024-09-14T06:32:30.615 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0023253839999999998 2024-09-14T06:32:30.615 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:30.615 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:30.615 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:32:30.615 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:12.744832+0000", 2024-09-14T06:32:30.616 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.5491000000000001e-05 2024-09-14T06:32:30.616 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:30.616 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:32:30.616 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:30.616 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:30.616 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:32:30.616 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-14T06:32:30.617 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-14T06:32:30.618 INFO:teuthology.orchestra.run.smithi094.stderr:noscrub is unset 2024-09-14T06:32:30.618 INFO:tasks.ceph.osd.6.smithi138.stderr:2024-09-14T06:32:30.616+0000 7f8cd8849640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33620) UID: 0 2024-09-14T06:32:30.717 INFO:tasks.ceph.osd.4.smithi138.stderr:2024-09-14T06:32:30.716+0000 7f6c653c8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33615) UID: 0 2024-09-14T06:32:30.730 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-14T06:32:30.730 INFO:teuthology.orchestra.run.smithi094.stdout: "ops": [], 2024-09-14T06:32:30.730 INFO:teuthology.orchestra.run.smithi094.stdout: "num_ops": 0 2024-09-14T06:32:30.731 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-14T06:32:30.741 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-14T06:32:30.818 INFO:tasks.ceph.osd.2.smithi094.stderr:2024-09-14T06:32:30.817+0000 7faf3269b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33649) UID: 0 2024-09-14T06:32:30.844 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-14T06:32:30.844 INFO:teuthology.orchestra.run.smithi094.stdout: "ops": [], 2024-09-14T06:32:30.844 INFO:teuthology.orchestra.run.smithi094.stdout: "complaint_time": 30, 2024-09-14T06:32:30.844 INFO:teuthology.orchestra.run.smithi094.stdout: "num_blocked_ops": 0 2024-09-14T06:32:30.844 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-14T06:32:30.853 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-14T06:32:30.937 INFO:tasks.ceph.osd.1.smithi094.stderr:2024-09-14T06:32:30.936+0000 7f328e47a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33641) UID: 0 2024-09-14T06:32:30.997 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-14T06:32:30.998 INFO:teuthology.orchestra.run.smithi094.stdout: "size": 20, 2024-09-14T06:32:30.998 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 600, 2024-09-14T06:32:30.998 INFO:teuthology.orchestra.run.smithi094.stdout: "ops": [ 2024-09-14T06:32:30.998 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:30.998 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:44426 3.2 3:4603ad7f:::rbd_data.112dca2f6ae1.0000000000000005:17 [read 892928~4096 [fadvise_nocache]] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:32:30.998 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:43.854977+0000", 2024-09-14T06:32:30.998 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 167.140988459, 2024-09-14T06:32:30.998 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.063369237999999994, 2024-09-14T06:32:30.998 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:32:30.998 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:32:30.998 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:32:30.998 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:32:30.998 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:32:30.998 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:30.998 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 44426 2024-09-14T06:32:30.998 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:30.999 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:32:30.999 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:30.999 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:32:30.999 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.854977+0000", 2024-09-14T06:32:30.999 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:30.999 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:30.999 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:30.999 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:32:30.999 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.854977+0000", 2024-09-14T06:32:30.999 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:30.999 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:30.999 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:30.999 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:32:30.999 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.854977+0000", 2024-09-14T06:32:30.999 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.1199999999999999e-07 2024-09-14T06:32:31.000 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:31.000 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:31.000 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:32:31.000 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.854980+0000", 2024-09-14T06:32:31.000 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.27e-06 2024-09-14T06:32:31.000 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:31.000 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:31.000 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:32:31.000 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.854983+0000", 2024-09-14T06:32:31.000 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.1650000000000002e-06 2024-09-14T06:32:31.000 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:31.000 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:31.000 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:32:31.000 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.854989+0000", 2024-09-14T06:32:31.001 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.6520000000000003e-06 2024-09-14T06:32:31.001 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:31.001 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:31.001 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:32:31.001 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.918143+0000", 2024-09-14T06:32:31.001 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.063154541999999994 2024-09-14T06:32:31.001 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:31.001 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:31.001 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:32:31.001 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.918209+0000", 2024-09-14T06:32:31.001 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.5529000000000002e-05 2024-09-14T06:32:31.001 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:31.001 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:31.001 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:32:31.002 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.918346+0000", 2024-09-14T06:32:31.002 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000136868 2024-09-14T06:32:31.002 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:31.002 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:32:31.002 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:31.002 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:31.002 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:31.002 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:44427 3.2 3:4603ad7f:::rbd_data.112dca2f6ae1.0000000000000005:17 [read 917504~4096 [fadvise_nocache]] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:32:31.002 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:43.854992+0000", 2024-09-14T06:32:31.002 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 167.14097281299999, 2024-09-14T06:32:31.002 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.063607792999999996, 2024-09-14T06:32:31.002 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:32:31.002 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:32:31.002 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:32:31.003 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:32:31.003 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:32:31.003 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:31.003 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 44427 2024-09-14T06:32:31.003 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:31.003 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:32:31.003 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:31.003 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:32:31.003 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.854992+0000", 2024-09-14T06:32:31.003 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:31.003 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:31.003 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:31.003 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:32:31.003 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.854992+0000", 2024-09-14T06:32:31.004 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:31.004 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:31.004 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:31.004 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:32:31.004 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.854993+0000", 2024-09-14T06:32:31.004 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.1400000000000001e-07 2024-09-14T06:32:31.004 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:31.004 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:31.004 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:32:31.004 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.854994+0000", 2024-09-14T06:32:31.004 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.091e-06 2024-09-14T06:32:31.004 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:31.004 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:31.004 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:32:31.004 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855001+0000", 2024-09-14T06:32:31.005 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.204e-06 2024-09-14T06:32:31.005 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:31.005 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:31.005 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:32:31.005 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855003+0000", 2024-09-14T06:32:31.005 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.514e-06 2024-09-14T06:32:31.005 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:31.005 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:31.005 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:32:31.005 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.918380+0000", 2024-09-14T06:32:31.005 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.063376738000000002 2024-09-14T06:32:31.005 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:31.005 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:31.005 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:32:31.006 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.918453+0000", 2024-09-14T06:32:31.006 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.3291e-05 2024-09-14T06:32:31.006 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:31.006 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:31.006 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:32:31.006 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.918600+0000", 2024-09-14T06:32:31.006 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00014674100000000001 2024-09-14T06:32:31.006 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:31.006 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:32:31.006 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:31.006 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:31.006 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:31.006 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:44428 3.2 3:4603ad7f:::rbd_data.112dca2f6ae1.0000000000000005:17 [read 942080~4096 [fadvise_nocache]] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:32:31.006 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:43.855006+0000", 2024-09-14T06:32:31.006 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 167.14095876299999, 2024-09-14T06:32:31.007 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.063846321999999997, 2024-09-14T06:32:31.007 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:32:31.007 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:32:31.007 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:32:31.007 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:32:31.007 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:32:31.007 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:31.007 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 44428 2024-09-14T06:32:31.007 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:31.007 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:32:31.007 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:31.007 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:32:31.007 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855006+0000", 2024-09-14T06:32:31.007 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:31.007 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:31.007 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:31.008 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:32:31.008 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855006+0000", 2024-09-14T06:32:31.008 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:31.008 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:31.008 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:31.008 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:32:31.008 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855007+0000", 2024-09-14T06:32:31.008 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.1199999999999999e-07 2024-09-14T06:32:31.008 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:31.008 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:31.008 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:32:31.008 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855011+0000", 2024-09-14T06:32:31.008 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.9539999999999998e-06 2024-09-14T06:32:31.008 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:31.008 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:31.009 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:32:31.009 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855014+0000", 2024-09-14T06:32:31.009 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.2930000000000001e-06 2024-09-14T06:32:31.009 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:31.009 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:31.009 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:32:31.009 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855016+0000", 2024-09-14T06:32:31.009 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.3039999999999999e-06 2024-09-14T06:32:31.009 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:31.009 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:31.009 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:32:31.009 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.918634+0000", 2024-09-14T06:32:31.009 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.063617928000000004 2024-09-14T06:32:31.010 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:31.010 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:31.010 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:32:31.010 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.918711+0000", 2024-09-14T06:32:31.010 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.6545000000000007e-05 2024-09-14T06:32:31.010 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:31.010 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:31.010 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:32:31.010 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.918853+0000", 2024-09-14T06:32:31.010 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00014208600000000001 2024-09-14T06:32:31.010 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:31.010 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:32:31.010 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:31.010 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:31.011 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:31.011 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:44429 3.2 3:4603ad7f:::rbd_data.112dca2f6ae1.0000000000000005:17 [read 966656~4096 [fadvise_nocache]] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:32:31.011 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:43.855019+0000", 2024-09-14T06:32:31.011 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 167.14094609, 2024-09-14T06:32:31.011 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.064059226999999996, 2024-09-14T06:32:31.011 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:32:31.011 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:32:31.011 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:32:31.011 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:32:31.011 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:32:31.011 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:31.011 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 44429 2024-09-14T06:32:31.011 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:31.011 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:32:31.012 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:31.012 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:32:31.012 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855019+0000", 2024-09-14T06:32:31.012 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:31.012 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:31.012 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:31.012 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:32:31.012 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855019+0000", 2024-09-14T06:32:31.012 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:31.012 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:31.012 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:31.012 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:32:31.012 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855019+0000", 2024-09-14T06:32:31.013 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.28e-07 2024-09-14T06:32:31.013 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:31.013 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:31.013 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:32:31.013 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855020+0000", 2024-09-14T06:32:31.013 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.0330000000000001e-06 2024-09-14T06:32:31.013 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:31.013 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:31.013 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:32:31.013 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855023+0000", 2024-09-14T06:32:31.013 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.79e-06 2024-09-14T06:32:31.013 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:31.013 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:31.013 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:32:31.013 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855025+0000", 2024-09-14T06:32:31.013 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.216e-06 2024-09-14T06:32:31.014 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:31.014 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:31.014 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:32:31.014 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.918886+0000", 2024-09-14T06:32:31.014 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.063861037999999995 2024-09-14T06:32:31.014 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:31.014 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:31.014 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:32:31.014 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.918952+0000", 2024-09-14T06:32:31.014 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.5993000000000003e-05 2024-09-14T06:32:31.014 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:31.014 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:31.014 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:32:31.015 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.919078+0000", 2024-09-14T06:32:31.015 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00012592899999999999 2024-09-14T06:32:31.015 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:31.015 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:32:31.015 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:31.015 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:31.015 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:31.015 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:44430 3.2 3:4603ad7f:::rbd_data.112dca2f6ae1.0000000000000005:17 [read 991232~4096 [fadvise_nocache]] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:32:31.015 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:43.855028+0000", 2024-09-14T06:32:31.015 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 167.14093707999999, 2024-09-14T06:32:31.015 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.064284346000000006, 2024-09-14T06:32:31.015 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:32:31.015 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:32:31.015 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:32:31.015 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:32:31.015 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:32:31.016 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:31.016 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 44430 2024-09-14T06:32:31.016 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:31.016 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:32:31.016 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:31.016 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:32:31.016 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855028+0000", 2024-09-14T06:32:31.016 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:31.016 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:31.016 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:31.016 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:32:31.016 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855028+0000", 2024-09-14T06:32:31.016 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:31.016 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:31.016 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:31.017 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:32:31.017 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855028+0000", 2024-09-14T06:32:31.017 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.1199999999999999e-07 2024-09-14T06:32:31.017 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:31.017 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:31.020 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:32:31.020 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855029+0000", 2024-09-14T06:32:31.020 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.1100000000000004e-07 2024-09-14T06:32:31.020 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:31.020 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:31.020 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:32:31.020 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855032+0000", 2024-09-14T06:32:31.020 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.7889999999999999e-06 2024-09-14T06:32:31.020 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:31.020 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:31.020 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:32:31.020 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855034+0000", 2024-09-14T06:32:31.021 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.125e-06 2024-09-14T06:32:31.021 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:31.021 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:31.021 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:32:31.021 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.919110+0000", 2024-09-14T06:32:31.021 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.064076413999999998 2024-09-14T06:32:31.021 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:31.021 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:31.021 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:32:31.021 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.919186+0000", 2024-09-14T06:32:31.021 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.5784999999999994e-05 2024-09-14T06:32:31.021 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:31.021 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:31.021 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:32:31.022 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.919312+0000", 2024-09-14T06:32:31.022 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00012611 2024-09-14T06:32:31.022 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:31.022 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:32:31.022 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:31.022 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:31.022 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:31.022 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:44431 3.2 3:4603ad7f:::rbd_data.112dca2f6ae1.0000000000000005:17 [read 1015808~4096 [fadvise_nocache]] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:32:31.022 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:43.855037+0000", 2024-09-14T06:32:31.022 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 167.14092821200001, 2024-09-14T06:32:31.022 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.064469755000000004, 2024-09-14T06:32:31.022 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:32:31.022 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:32:31.022 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:32:31.022 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:32:31.023 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:32:31.023 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:31.023 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 44431 2024-09-14T06:32:31.023 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:31.023 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:32:31.023 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:31.023 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:32:31.023 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855037+0000", 2024-09-14T06:32:31.023 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:31.023 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:31.023 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:31.023 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:32:31.023 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855037+0000", 2024-09-14T06:32:31.023 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:31.023 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:31.024 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:31.024 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:32:31.024 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855037+0000", 2024-09-14T06:32:31.024 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.1e-07 2024-09-14T06:32:31.024 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:31.024 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:31.024 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:32:31.024 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855038+0000", 2024-09-14T06:32:31.024 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.9400000000000004e-07 2024-09-14T06:32:31.024 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:31.024 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:31.024 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:32:31.024 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855041+0000", 2024-09-14T06:32:31.024 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.9639999999999999e-06 2024-09-14T06:32:31.024 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:31.024 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:31.025 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:32:31.025 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855043+0000", 2024-09-14T06:32:31.025 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.2259999999999999e-06 2024-09-14T06:32:31.025 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:31.025 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:31.025 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:32:31.025 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.919347+0000", 2024-09-14T06:32:31.025 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.064304091999999993 2024-09-14T06:32:31.025 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:31.025 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:31.025 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:32:31.025 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.919393+0000", 2024-09-14T06:32:31.025 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.5340999999999998e-05 2024-09-14T06:32:31.025 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:31.025 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:31.025 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:32:31.026 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.919507+0000", 2024-09-14T06:32:31.026 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000114028 2024-09-14T06:32:31.026 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:31.026 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:32:31.026 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:31.026 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:31.026 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:31.026 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:44432 3.2 3:4603ad7f:::rbd_data.112dca2f6ae1.0000000000000005:17 [read 1040384~4096 [fadvise_nocache]] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:32:31.026 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:43.855046+0000", 2024-09-14T06:32:31.026 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 167.14091914400001, 2024-09-14T06:32:31.026 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.064639253999999993, 2024-09-14T06:32:31.026 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:32:31.026 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:32:31.026 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:32:31.026 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:32:31.027 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:32:31.027 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:31.027 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 44432 2024-09-14T06:32:31.027 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:31.027 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:32:31.027 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:31.027 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:32:31.027 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855046+0000", 2024-09-14T06:32:31.027 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:31.027 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:31.027 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:31.027 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:32:31.027 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855046+0000", 2024-09-14T06:32:31.027 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:31.027 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:31.027 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:31.028 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:32:31.028 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855046+0000", 2024-09-14T06:32:31.028 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.05e-07 2024-09-14T06:32:31.028 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:31.028 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:31.028 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:32:31.028 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855047+0000", 2024-09-14T06:32:31.028 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.879999999999999e-07 2024-09-14T06:32:31.028 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:31.028 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:31.028 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:32:31.028 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855050+0000", 2024-09-14T06:32:31.028 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.8459999999999999e-06 2024-09-14T06:32:31.028 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:31.028 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:31.029 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:32:31.029 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855052+0000", 2024-09-14T06:32:31.029 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.136e-06 2024-09-14T06:32:31.029 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:31.029 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:31.029 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:32:31.029 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.919536+0000", 2024-09-14T06:32:31.029 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.064484132 2024-09-14T06:32:31.029 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:31.029 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:31.029 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:32:31.029 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.919611+0000", 2024-09-14T06:32:31.029 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.4407999999999996e-05 2024-09-14T06:32:31.029 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:31.029 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:31.029 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:32:31.030 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.919685+0000", 2024-09-14T06:32:31.030 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.4538999999999999e-05 2024-09-14T06:32:31.030 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:31.030 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:32:31.030 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:31.030 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:31.030 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:31.030 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:44518 3.2 3:4190d882:::rbd_data.112dca2f6ae1.0000000000000002:17 [read 12288~4096 [fadvise_nocache]] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:32:31.030 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:43.855055+0000", 2024-09-14T06:32:31.030 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 167.14091026, 2024-09-14T06:32:31.030 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.064894317000000007, 2024-09-14T06:32:31.030 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:32:31.030 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:32:31.030 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:32:31.030 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:32:31.031 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:32:31.031 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:31.031 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 44518 2024-09-14T06:32:31.031 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:31.031 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:32:31.031 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:31.031 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:32:31.031 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855055+0000", 2024-09-14T06:32:31.031 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:31.031 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:31.031 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:31.031 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:32:31.031 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855055+0000", 2024-09-14T06:32:31.031 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:31.032 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:31.032 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:31.032 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:32:31.032 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855055+0000", 2024-09-14T06:32:31.032 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.0699999999999999e-07 2024-09-14T06:32:31.032 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:31.032 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:31.032 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:32:31.032 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855056+0000", 2024-09-14T06:32:31.032 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.9400000000000004e-07 2024-09-14T06:32:31.032 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:31.032 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:31.032 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:32:31.032 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855059+0000", 2024-09-14T06:32:31.032 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.8870000000000001e-06 2024-09-14T06:32:31.033 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:31.033 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:31.033 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:32:31.033 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855061+0000", 2024-09-14T06:32:31.033 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.1919999999999999e-06 2024-09-14T06:32:31.033 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:31.033 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:31.033 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:32:31.033 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.919719+0000", 2024-09-14T06:32:31.033 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.064658013 2024-09-14T06:32:31.033 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:31.033 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:31.033 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:32:31.034 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.919788+0000", 2024-09-14T06:32:31.034 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.9282000000000005e-05 2024-09-14T06:32:31.034 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:31.034 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:31.034 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:32:31.034 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.919949+0000", 2024-09-14T06:32:31.034 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00016084199999999999 2024-09-14T06:32:31.034 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:31.034 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:32:31.034 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:31.034 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:31.034 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:31.034 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:44519 3.2 3:4190d882:::rbd_data.112dca2f6ae1.0000000000000002:17 [read 36864~4096 [fadvise_nocache]] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:32:31.034 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:43.855064+0000", 2024-09-14T06:32:31.035 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 167.14090128300001, 2024-09-14T06:32:31.035 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.065081865000000003, 2024-09-14T06:32:31.035 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:32:31.035 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:32:31.035 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:32:31.035 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:32:31.035 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:32:31.035 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:31.035 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 44519 2024-09-14T06:32:31.035 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:31.035 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:32:31.035 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:31.035 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:32:31.035 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855064+0000", 2024-09-14T06:32:31.035 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:31.036 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:31.036 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:31.036 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:32:31.036 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855064+0000", 2024-09-14T06:32:31.036 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:31.036 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:31.036 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:31.036 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:32:31.036 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855064+0000", 2024-09-14T06:32:31.036 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.1e-07 2024-09-14T06:32:31.036 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:31.036 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:31.036 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:32:31.036 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855065+0000", 2024-09-14T06:32:31.036 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.035e-06 2024-09-14T06:32:31.037 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:31.037 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:31.037 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:32:31.037 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855068+0000", 2024-09-14T06:32:31.037 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.8339999999999999e-06 2024-09-14T06:32:31.037 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:31.037 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:31.037 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:32:31.037 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855071+0000", 2024-09-14T06:32:31.037 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.3450000000000002e-06 2024-09-14T06:32:31.037 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:31.037 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:31.037 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:32:31.037 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.919979+0000", 2024-09-14T06:32:31.037 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.064907647999999998 2024-09-14T06:32:31.038 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:31.038 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:31.038 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:32:31.038 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.920026+0000", 2024-09-14T06:32:31.038 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.7595000000000001e-05 2024-09-14T06:32:31.038 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:31.038 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:31.038 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:32:31.038 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.920146+0000", 2024-09-14T06:32:31.038 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00011919800000000001 2024-09-14T06:32:31.038 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:31.038 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:32:31.038 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:31.038 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:31.038 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:31.038 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:44520 3.2 3:4190d882:::rbd_data.112dca2f6ae1.0000000000000002:17 [read 61440~4096 [fadvise_nocache]] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:32:31.039 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:43.855075+0000", 2024-09-14T06:32:31.039 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 167.140890232, 2024-09-14T06:32:31.039 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.065335078000000005, 2024-09-14T06:32:31.039 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:32:31.039 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:32:31.039 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:32:31.039 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:32:31.039 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:32:31.039 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:31.039 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 44520 2024-09-14T06:32:31.039 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:31.039 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:32:31.039 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:31.039 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:32:31.039 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855075+0000", 2024-09-14T06:32:31.040 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:31.040 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:31.040 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:31.040 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:32:31.040 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855075+0000", 2024-09-14T06:32:31.040 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:31.040 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:31.040 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:31.040 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:32:31.040 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855075+0000", 2024-09-14T06:32:31.040 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.1e-07 2024-09-14T06:32:31.040 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:31.040 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:31.041 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:32:31.041 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855076+0000", 2024-09-14T06:32:31.041 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.0839999999999999e-06 2024-09-14T06:32:31.041 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:31.041 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:31.041 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:32:31.041 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855079+0000", 2024-09-14T06:32:31.041 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.9359999999999999e-06 2024-09-14T06:32:31.041 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:31.041 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:31.041 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:32:31.041 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855084+0000", 2024-09-14T06:32:31.041 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.1730000000000003e-06 2024-09-14T06:32:31.041 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:31.042 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:31.042 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:32:31.042 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.920175+0000", 2024-09-14T06:32:31.042 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.065090234999999996 2024-09-14T06:32:31.042 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:31.042 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:31.042 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:32:31.042 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.920250+0000", 2024-09-14T06:32:31.042 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.5807000000000001e-05 2024-09-14T06:32:31.042 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:31.042 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:31.042 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:32:31.042 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.920410+0000", 2024-09-14T06:32:31.043 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00015963300000000001 2024-09-14T06:32:31.043 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:31.043 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:32:31.043 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:31.043 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:31.043 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:31.043 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:44521 3.2 3:4190d882:::rbd_data.112dca2f6ae1.0000000000000002:17 [read 86016~4096 [fadvise_nocache]] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:32:31.043 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:43.855087+0000", 2024-09-14T06:32:31.043 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 167.140877751, 2024-09-14T06:32:31.043 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.065563264999999996, 2024-09-14T06:32:31.043 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:32:31.043 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:32:31.043 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:32:31.044 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:32:31.044 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:32:31.044 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:31.044 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 44521 2024-09-14T06:32:31.044 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:31.044 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:32:31.044 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:31.044 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:32:31.044 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855087+0000", 2024-09-14T06:32:31.044 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:31.044 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:31.044 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:31.044 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:32:31.044 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855087+0000", 2024-09-14T06:32:31.044 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:31.045 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:31.045 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:31.045 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:32:31.045 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855088+0000", 2024-09-14T06:32:31.045 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.3699999999999999e-07 2024-09-14T06:32:31.045 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:31.045 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:31.045 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:32:31.045 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855089+0000", 2024-09-14T06:32:31.045 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.9900000000000009e-07 2024-09-14T06:32:31.045 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:31.045 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:31.045 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:32:31.045 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855091+0000", 2024-09-14T06:32:31.045 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.8940000000000002e-06 2024-09-14T06:32:31.046 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:31.046 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:31.046 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:32:31.046 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855094+0000", 2024-09-14T06:32:31.046 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.351e-06 2024-09-14T06:32:31.046 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:31.046 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:31.046 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:32:31.046 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.920437+0000", 2024-09-14T06:32:31.046 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.065343221000000007 2024-09-14T06:32:31.046 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:31.046 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:31.046 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:32:31.047 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.920481+0000", 2024-09-14T06:32:31.047 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.4431000000000002e-05 2024-09-14T06:32:31.047 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:31.047 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:31.047 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:32:31.047 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.920651+0000", 2024-09-14T06:32:31.047 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00016913200000000001 2024-09-14T06:32:31.047 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:31.047 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:32:31.047 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:31.047 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:31.047 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:31.047 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:44522 3.2 3:4190d882:::rbd_data.112dca2f6ae1.0000000000000002:17 [read 110592~4096 [fadvise_nocache]] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:32:31.047 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:43.855098+0000", 2024-09-14T06:32:31.047 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 167.14086752399999, 2024-09-14T06:32:31.047 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.065827628999999999, 2024-09-14T06:32:31.048 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:32:31.048 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:32:31.048 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:32:31.048 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:32:31.048 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:32:31.048 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:31.048 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 44522 2024-09-14T06:32:31.048 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:31.048 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:32:31.048 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:31.048 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:32:31.048 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855098+0000", 2024-09-14T06:32:31.048 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:31.048 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:31.048 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:31.049 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:32:31.049 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855098+0000", 2024-09-14T06:32:31.049 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:31.049 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:31.049 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:31.049 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:32:31.049 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855098+0000", 2024-09-14T06:32:31.049 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.1e-07 2024-09-14T06:32:31.049 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:31.049 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:31.049 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:32:31.049 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855099+0000", 2024-09-14T06:32:31.049 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.012e-06 2024-09-14T06:32:31.049 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:31.050 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:31.050 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:32:31.050 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855102+0000", 2024-09-14T06:32:31.050 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.9380000000000001e-06 2024-09-14T06:32:31.050 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:31.050 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:31.050 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:32:31.050 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855108+0000", 2024-09-14T06:32:31.050 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.4339999999999997e-06 2024-09-14T06:32:31.050 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:31.050 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:31.050 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:32:31.050 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.920681+0000", 2024-09-14T06:32:31.051 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.065572943999999994 2024-09-14T06:32:31.051 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:31.051 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:31.051 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:32:31.051 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.920724+0000", 2024-09-14T06:32:31.051 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.2877000000000003e-05 2024-09-14T06:32:31.051 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:31.051 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:31.051 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:32:31.051 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.920925+0000", 2024-09-14T06:32:31.051 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00020121399999999999 2024-09-14T06:32:31.051 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:31.051 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:32:31.051 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:31.052 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:31.052 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:31.052 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:44523 3.2 3:4190d882:::rbd_data.112dca2f6ae1.0000000000000002:17 [read 135168~4096 [fadvise_nocache]] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:32:31.052 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:43.855111+0000", 2024-09-14T06:32:31.052 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 167.14085380200001, 2024-09-14T06:32:31.052 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.066028198999999996, 2024-09-14T06:32:31.052 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:32:31.052 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:32:31.052 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:32:31.052 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:32:31.052 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:32:31.052 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:31.052 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 44523 2024-09-14T06:32:31.052 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:31.052 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:32:31.053 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:31.053 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:32:31.053 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855111+0000", 2024-09-14T06:32:31.053 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:31.053 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:31.053 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:31.053 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:32:31.053 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855111+0000", 2024-09-14T06:32:31.053 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:31.053 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:31.053 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:31.053 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:32:31.053 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855112+0000", 2024-09-14T06:32:31.054 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.1199999999999999e-07 2024-09-14T06:32:31.054 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:31.054 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:31.054 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:32:31.054 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855115+0000", 2024-09-14T06:32:31.054 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.3380000000000001e-06 2024-09-14T06:32:31.054 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:31.054 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:31.054 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:32:31.054 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855118+0000", 2024-09-14T06:32:31.054 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.366e-06 2024-09-14T06:32:31.054 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:31.054 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:31.054 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:32:31.055 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855121+0000", 2024-09-14T06:32:31.055 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.8329999999999998e-06 2024-09-14T06:32:31.055 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:31.055 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:31.055 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:32:31.055 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.920956+0000", 2024-09-14T06:32:31.055 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.065835251999999997 2024-09-14T06:32:31.055 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:31.055 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:31.055 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:32:31.055 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.921001+0000", 2024-09-14T06:32:31.055 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.4778999999999999e-05 2024-09-14T06:32:31.055 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:31.055 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:31.055 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:32:31.056 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.921139+0000", 2024-09-14T06:32:31.056 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000138419 2024-09-14T06:32:31.056 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:31.056 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:32:31.056 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:31.056 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:31.056 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:31.056 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:44524 3.2 3:4190d882:::rbd_data.112dca2f6ae1.0000000000000002:17 [read 159744~4096 [fadvise_nocache]] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:32:31.056 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:43.855124+0000", 2024-09-14T06:32:31.056 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 167.14084082299999, 2024-09-14T06:32:31.056 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.066249261000000004, 2024-09-14T06:32:31.056 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:32:31.056 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:32:31.056 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:32:31.056 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:32:31.056 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:32:31.057 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:31.057 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 44524 2024-09-14T06:32:31.057 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:31.057 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:32:31.057 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:31.057 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:32:31.057 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855124+0000", 2024-09-14T06:32:31.057 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:31.057 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:31.057 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:31.057 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:32:31.057 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855124+0000", 2024-09-14T06:32:31.057 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:31.057 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:31.058 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:31.058 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:32:31.058 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861086+0000", 2024-09-14T06:32:31.058 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0059613829999999998 2024-09-14T06:32:31.058 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:31.058 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:31.058 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:32:31.058 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861100+0000", 2024-09-14T06:32:31.058 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.4644000000000001e-05 2024-09-14T06:32:31.058 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:31.058 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:31.058 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:32:31.058 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861109+0000", 2024-09-14T06:32:31.058 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.541e-06 2024-09-14T06:32:31.059 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:31.059 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:31.059 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:32:31.059 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861117+0000", 2024-09-14T06:32:31.059 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.1049999999999995e-06 2024-09-14T06:32:31.059 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:31.059 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:31.059 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:32:31.059 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.921169+0000", 2024-09-14T06:32:31.059 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.060052405000000003 2024-09-14T06:32:31.059 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:31.059 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:31.059 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:32:31.059 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.921213+0000", 2024-09-14T06:32:31.059 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.3565e-05 2024-09-14T06:32:31.059 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:31.060 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:31.060 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:32:31.060 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.921374+0000", 2024-09-14T06:32:31.060 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00016061800000000001 2024-09-14T06:32:31.060 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:31.060 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:32:31.060 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:31.060 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:31.060 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:31.060 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:44537 3.2 3:4190d882:::rbd_data.112dca2f6ae1.0000000000000002:17 [read 479232~4096 [fadvise_nocache]] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:32:31.060 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:43.861405+0000", 2024-09-14T06:32:31.060 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 167.13456022, 2024-09-14T06:32:31.060 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.063351795000000002, 2024-09-14T06:32:31.060 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:32:31.061 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:32:31.061 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:32:31.061 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:32:31.061 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:32:31.061 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:31.061 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 44537 2024-09-14T06:32:31.061 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:31.061 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:32:31.061 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:31.061 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:32:31.061 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861405+0000", 2024-09-14T06:32:31.061 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:31.061 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:31.062 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:31.062 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:32:31.062 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861405+0000", 2024-09-14T06:32:31.062 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:31.062 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:31.062 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:31.062 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:32:31.062 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861405+0000", 2024-09-14T06:32:31.062 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.9099999999999999e-07 2024-09-14T06:32:31.062 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:31.062 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:31.062 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:32:31.062 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861407+0000", 2024-09-14T06:32:31.063 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.075e-06 2024-09-14T06:32:31.063 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:31.063 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:31.063 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:32:31.063 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861414+0000", 2024-09-14T06:32:31.063 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.4659999999999996e-06 2024-09-14T06:32:31.063 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:31.063 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:31.063 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:32:31.063 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861418+0000", 2024-09-14T06:32:31.063 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.5519999999999998e-06 2024-09-14T06:32:31.063 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:31.063 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:31.064 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:32:31.064 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.924474+0000", 2024-09-14T06:32:31.064 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.063056012999999994 2024-09-14T06:32:31.064 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:31.064 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:31.064 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:32:31.064 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.924556+0000", 2024-09-14T06:32:31.064 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.1181999999999996e-05 2024-09-14T06:32:31.064 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:31.064 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:31.064 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:32:31.064 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.924757+0000", 2024-09-14T06:32:31.064 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00020111599999999999 2024-09-14T06:32:31.064 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:31.064 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:32:31.065 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:31.065 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:31.065 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:31.065 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:44538 3.2 3:4190d882:::rbd_data.112dca2f6ae1.0000000000000002:17 [read 503808~4096 [fadvise_nocache]] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:32:31.065 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:43.861425+0000", 2024-09-14T06:32:31.065 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 167.134540179, 2024-09-14T06:32:31.065 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.063627125000000007, 2024-09-14T06:32:31.065 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:32:31.065 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:32:31.065 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:32:31.065 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:32:31.065 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:32:31.065 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:31.066 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 44538 2024-09-14T06:32:31.066 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:31.066 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:32:31.066 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:31.066 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:32:31.066 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861425+0000", 2024-09-14T06:32:31.066 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:31.066 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:31.066 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:31.066 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:32:31.066 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861425+0000", 2024-09-14T06:32:31.066 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:31.066 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:31.066 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:31.066 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:32:31.067 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861425+0000", 2024-09-14T06:32:31.067 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.9999999999999998e-07 2024-09-14T06:32:31.067 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:31.067 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:31.067 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:32:31.067 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861427+0000", 2024-09-14T06:32:31.067 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.9829999999999999e-06 2024-09-14T06:32:31.067 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:31.067 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:31.067 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:32:31.067 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861433+0000", 2024-09-14T06:32:31.067 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.2889999999999997e-06 2024-09-14T06:32:31.067 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:31.068 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:31.068 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:32:31.068 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861454+0000", 2024-09-14T06:32:31.068 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.1844999999999998e-05 2024-09-14T06:32:31.068 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:31.068 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:31.068 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:32:31.068 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.924786+0000", 2024-09-14T06:32:31.068 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.063331497 2024-09-14T06:32:31.068 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:31.068 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:31.068 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:32:31.068 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.924864+0000", 2024-09-14T06:32:31.068 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.7744000000000006e-05 2024-09-14T06:32:31.069 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:31.069 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:31.069 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:32:31.069 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.925052+0000", 2024-09-14T06:32:31.069 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00018836699999999999 2024-09-14T06:32:31.069 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:31.069 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:32:31.069 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:31.069 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:31.069 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:31.069 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:44539 3.2 3:4190d882:::rbd_data.112dca2f6ae1.0000000000000002:17 [read 528384~4096 [fadvise_nocache]] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:32:31.069 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:43.861462+0000", 2024-09-14T06:32:31.069 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 167.134502852, 2024-09-14T06:32:31.069 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.063855705999999998, 2024-09-14T06:32:31.069 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:32:31.070 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:32:31.070 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:32:31.070 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:32:31.070 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:32:31.070 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:31.070 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 44539 2024-09-14T06:32:31.070 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:31.070 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:32:31.070 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:31.070 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:32:31.070 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861462+0000", 2024-09-14T06:32:31.070 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:31.070 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:31.070 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:31.071 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:32:31.071 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861462+0000", 2024-09-14T06:32:31.071 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:31.071 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:31.071 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:31.071 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:32:31.071 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861463+0000", 2024-09-14T06:32:31.071 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.5200000000000002e-07 2024-09-14T06:32:31.071 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:31.071 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:31.071 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:32:31.071 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861465+0000", 2024-09-14T06:32:31.071 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.5189999999999999e-06 2024-09-14T06:32:31.071 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:31.072 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:31.072 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:32:31.072 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861471+0000", 2024-09-14T06:32:31.072 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.6899999999999997e-06 2024-09-14T06:32:31.072 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:31.072 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:31.072 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:32:31.072 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861475+0000", 2024-09-14T06:32:31.072 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.5730000000000001e-06 2024-09-14T06:32:31.072 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:31.072 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:31.072 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:32:31.072 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.925085+0000", 2024-09-14T06:32:31.073 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.063609902999999995 2024-09-14T06:32:31.073 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:31.073 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:31.073 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:32:31.073 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.925145+0000", 2024-09-14T06:32:31.073 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.9383999999999999e-05 2024-09-14T06:32:31.073 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:31.073 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:31.073 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:32:31.073 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.925318+0000", 2024-09-14T06:32:31.073 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00017318499999999999 2024-09-14T06:32:31.073 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:31.073 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:32:31.073 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:31.074 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:31.074 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:31.074 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:44540 3.2 3:4190d882:::rbd_data.112dca2f6ae1.0000000000000002:17 [read 552960~4096 [fadvise_nocache]] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:32:31.074 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:43.861483+0000", 2024-09-14T06:32:31.074 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 167.134482421, 2024-09-14T06:32:31.074 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.064065681999999999, 2024-09-14T06:32:31.074 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:32:31.074 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:32:31.074 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:32:31.074 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:32:31.074 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:32:31.074 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:31.074 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 44540 2024-09-14T06:32:31.074 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:31.074 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:32:31.075 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:31.075 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:32:31.075 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861483+0000", 2024-09-14T06:32:31.075 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:31.075 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:31.075 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:31.075 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:32:31.075 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861483+0000", 2024-09-14T06:32:31.075 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:31.075 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:31.075 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:31.075 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:32:31.075 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.867079+0000", 2024-09-14T06:32:31.075 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0055966480000000001 2024-09-14T06:32:31.076 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:31.076 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:31.076 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:32:31.076 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.867086+0000", 2024-09-14T06:32:31.076 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.387e-06 2024-09-14T06:32:31.076 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:31.076 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:31.076 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:32:31.076 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.867094+0000", 2024-09-14T06:32:31.076 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.9419999999999995e-06 2024-09-14T06:32:31.076 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:31.076 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:31.076 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:32:31.076 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.867109+0000", 2024-09-14T06:32:31.076 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.5818999999999999e-05 2024-09-14T06:32:31.077 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:31.077 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:31.077 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:32:31.077 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.925354+0000", 2024-09-14T06:32:31.077 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.058244047 2024-09-14T06:32:31.077 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:31.077 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:31.077 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:32:31.077 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.925406+0000", 2024-09-14T06:32:31.077 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.2661999999999998e-05 2024-09-14T06:32:31.077 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:31.077 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:31.077 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:32:31.077 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.925548+0000", 2024-09-14T06:32:31.078 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000142177 2024-09-14T06:32:31.078 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:31.078 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:32:31.078 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:31.078 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:31.078 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:31.078 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:225587 3.7 3:e112e2b3:::rbd_id.image_client.0-clone42:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e189)", 2024-09-14T06:32:31.078 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:31:53.874591+0000", 2024-09-14T06:32:31.078 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 37.121374203999999, 2024-09-14T06:32:31.078 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.875541507, 2024-09-14T06:32:31.078 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:32:31.078 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:32:31.078 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:32:31.078 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:32:31.078 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:32:31.079 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:31.079 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 225587 2024-09-14T06:32:31.079 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:31.079 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:32:31.079 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:31.079 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:32:31.079 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:53.874591+0000", 2024-09-14T06:32:31.079 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:31.079 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:31.079 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:31.080 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:32:31.080 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:53.874591+0000", 2024-09-14T06:32:31.080 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:31.080 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:31.080 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:31.080 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:32:31.080 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:53.874592+0000", 2024-09-14T06:32:31.080 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.5209999999999999e-06 2024-09-14T06:32:31.080 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:31.080 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:31.080 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:32:31.080 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:53.874599+0000", 2024-09-14T06:32:31.080 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.6290000000000004e-06 2024-09-14T06:32:31.080 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:31.081 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:31.081 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:32:31.081 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:53.874618+0000", 2024-09-14T06:32:31.081 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.8950999999999999e-05 2024-09-14T06:32:31.081 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:31.081 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:31.081 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:32:31.081 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:53.874626+0000", 2024-09-14T06:32:31.081 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.3960000000000006e-06 2024-09-14T06:32:31.081 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:31.081 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:31.081 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:32:31.081 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:53.874699+0000", 2024-09-14T06:32:31.082 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.2329999999999994e-05 2024-09-14T06:32:31.082 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:31.082 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:31.082 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "waiting for peered", 2024-09-14T06:32:31.082 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:53.874711+0000", 2024-09-14T06:32:31.082 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.2473999999999999e-05 2024-09-14T06:32:31.082 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:31.082 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:31.082 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:32:31.082 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:54.749578+0000", 2024-09-14T06:32:31.082 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.87486642100000001 2024-09-14T06:32:31.082 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:31.082 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:31.082 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:32:31.083 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:54.749706+0000", 2024-09-14T06:32:31.083 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00012856200000000001 2024-09-14T06:32:31.083 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:31.083 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:31.083 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:32:31.083 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:54.750132+0000", 2024-09-14T06:32:31.083 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00042622299999999999 2024-09-14T06:32:31.083 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:31.083 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:32:31.083 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:31.083 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:31.083 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:31.083 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:233119 3.1e 3:7bae5948:::rbd_data.112df8d32353.000000000000000f:head [delete] snapc 53=[] ondisk+write+known_if_redirected+supports_pool_eio e211)", 2024-09-14T06:32:31.083 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:32:16.760588+0000", 2024-09-14T06:32:31.083 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 14.235377217, 2024-09-14T06:32:31.084 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.92349815700000004, 2024-09-14T06:32:31.084 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:32:31.084 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:32:31.084 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:32:31.084 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:32:31.084 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:32:31.084 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:31.084 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 233119 2024-09-14T06:32:31.084 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:31.084 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:32:31.084 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:31.084 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:32:31.084 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:16.760588+0000", 2024-09-14T06:32:31.084 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:31.084 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:31.085 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:31.085 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:32:31.085 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:16.760588+0000", 2024-09-14T06:32:31.085 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:31.085 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:31.085 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:31.085 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:32:31.085 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:16.760588+0000", 2024-09-14T06:32:31.085 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.3900000000000005e-07 2024-09-14T06:32:31.085 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:31.085 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:31.085 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:32:31.085 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:16.760591+0000", 2024-09-14T06:32:31.085 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.0070000000000002e-06 2024-09-14T06:32:31.086 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:31.086 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:31.086 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:32:31.086 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:16.760598+0000", 2024-09-14T06:32:31.086 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.0509999999999999e-06 2024-09-14T06:32:31.086 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:31.086 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:31.086 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:32:31.086 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:16.760605+0000", 2024-09-14T06:32:31.086 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.0310000000000004e-06 2024-09-14T06:32:31.086 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:31.086 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:31.086 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:32:31.086 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:16.760668+0000", 2024-09-14T06:32:31.086 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.3699000000000001e-05 2024-09-14T06:32:31.087 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:31.087 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:31.087 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "waiting for peered", 2024-09-14T06:32:31.087 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:16.760678+0000", 2024-09-14T06:32:31.087 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.0122999999999999e-05 2024-09-14T06:32:31.087 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:31.087 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:31.087 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:32:31.087 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:17.681459+0000", 2024-09-14T06:32:31.087 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.92078067399999997 2024-09-14T06:32:31.087 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:31.087 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:31.087 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:32:31.087 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:17.681700+0000", 2024-09-14T06:32:31.087 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00024128800000000001 2024-09-14T06:32:31.087 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:31.087 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:31.088 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "waiting for subops from 2", 2024-09-14T06:32:31.088 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:17.681860+0000", 2024-09-14T06:32:31.088 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00015995500000000001 2024-09-14T06:32:31.088 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:31.088 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:31.088 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:32:31.088 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:17.683399+0000", 2024-09-14T06:32:31.088 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0015388209999999999 2024-09-14T06:32:31.088 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:31.088 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:31.088 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "op_commit", 2024-09-14T06:32:31.088 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:17.683687+0000", 2024-09-14T06:32:31.088 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00028772299999999998 2024-09-14T06:32:31.088 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:31.088 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:31.089 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-14T06:32:31.089 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:17.684032+0000", 2024-09-14T06:32:31.089 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000344836 2024-09-14T06:32:31.089 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:31.089 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:31.089 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:32:31.089 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:17.684086+0000", 2024-09-14T06:32:31.089 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.4410000000000003e-05 2024-09-14T06:32:31.089 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:31.089 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:32:31.089 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:31.089 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:31.089 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:32:31.089 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-14T06:32:31.091 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-14T06:32:31.092 INFO:tasks.ceph.osd.3.smithi094.stderr:2024-09-14T06:32:31.090+0000 7feeeafe8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33655) UID: 0 2024-09-14T06:32:31.192 INFO:tasks.ceph.osd.1.smithi094.stderr:2024-09-14T06:32:31.191+0000 7f328e47a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33641) UID: 0 2024-09-14T06:32:31.197 INFO:teuthology.orchestra.run.smithi138.stdout:{ 2024-09-14T06:32:31.197 INFO:teuthology.orchestra.run.smithi138.stdout: "ops": [], 2024-09-14T06:32:31.197 INFO:teuthology.orchestra.run.smithi138.stdout: "num_ops": 0 2024-09-14T06:32:31.197 INFO:teuthology.orchestra.run.smithi138.stdout:} 2024-09-14T06:32:31.206 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-14T06:32:31.206 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-14T06:32:31.293 INFO:tasks.ceph.osd.4.smithi138.stderr:2024-09-14T06:32:31.292+0000 7f6c653c8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33615) UID: 0 2024-09-14T06:32:31.310 INFO:teuthology.orchestra.run.smithi138.stdout:{ 2024-09-14T06:32:31.310 INFO:teuthology.orchestra.run.smithi138.stdout: "ops": [], 2024-09-14T06:32:31.310 INFO:teuthology.orchestra.run.smithi138.stdout: "complaint_time": 30, 2024-09-14T06:32:31.311 INFO:teuthology.orchestra.run.smithi138.stdout: "num_blocked_ops": 0 2024-09-14T06:32:31.311 INFO:teuthology.orchestra.run.smithi138.stdout:} 2024-09-14T06:32:31.319 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-14T06:32:31.326 INFO:teuthology.orchestra.run.smithi094.stdout:osd.0: {} 2024-09-14T06:32:31.326 INFO:teuthology.orchestra.run.smithi094.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-14T06:32:31.343 INFO:teuthology.orchestra.run.smithi094.stdout:osd.1: {} 2024-09-14T06:32:31.343 INFO:teuthology.orchestra.run.smithi094.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-14T06:32:31.359 INFO:teuthology.orchestra.run.smithi094.stdout:osd.2: {} 2024-09-14T06:32:31.360 INFO:teuthology.orchestra.run.smithi094.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-14T06:32:31.375 INFO:teuthology.orchestra.run.smithi094.stdout:osd.3: {} 2024-09-14T06:32:31.376 INFO:teuthology.orchestra.run.smithi094.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-14T06:32:31.392 INFO:teuthology.orchestra.run.smithi094.stdout:osd.4: {} 2024-09-14T06:32:31.392 INFO:teuthology.orchestra.run.smithi094.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-14T06:32:31.394 INFO:tasks.ceph.osd.2.smithi094.stderr:2024-09-14T06:32:31.393+0000 7faf3269b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33649) UID: 0 2024-09-14T06:32:31.408 INFO:teuthology.orchestra.run.smithi094.stdout:osd.5: {} 2024-09-14T06:32:31.408 INFO:teuthology.orchestra.run.smithi094.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-14T06:32:31.425 INFO:teuthology.orchestra.run.smithi094.stdout:osd.6: {} 2024-09-14T06:32:31.425 INFO:teuthology.orchestra.run.smithi094.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-14T06:32:31.441 INFO:teuthology.orchestra.run.smithi094.stdout:osd.7: {} 2024-09-14T06:32:31.442 INFO:teuthology.orchestra.run.smithi094.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-14T06:32:31.464 INFO:teuthology.orchestra.run.smithi138.stdout:ERROR: (22) Invalid argument 2024-09-14T06:32:31.465 INFO:teuthology.orchestra.run.smithi138.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:32:31.473 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-14T06:32:31.517 INFO:tasks.ceph.osd.4.smithi138.stderr:2024-09-14T06:32:31.516+0000 7f6c653c8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33615) UID: 0 2024-09-14T06:32:31.576 INFO:teuthology.orchestra.run.smithi138.stdout:ERROR: (22) Invalid argument 2024-09-14T06:32:31.577 INFO:teuthology.orchestra.run.smithi138.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:32:31.585 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-14T06:32:31.595 INFO:tasks.ceph.osd.0.smithi094.stderr:2024-09-14T06:32:31.593+0000 7f07c9333640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33636) UID: 0 2024-09-14T06:32:31.697 INFO:tasks.ceph.osd.1.smithi094.stderr:2024-09-14T06:32:31.695+0000 7f328e47a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33641) UID: 0 2024-09-14T06:32:31.728 INFO:teuthology.orchestra.run.smithi138.stdout:ERROR: (22) Invalid argument 2024-09-14T06:32:31.728 INFO:teuthology.orchestra.run.smithi138.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:32:31.737 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-14T06:32:31.797 INFO:tasks.ceph.osd.5.smithi138.stderr:2024-09-14T06:32:31.796+0000 7fe13746f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33618) UID: 0 2024-09-14T06:32:31.857 INFO:teuthology.orchestra.run.smithi138.stdout:ERROR: (22) Invalid argument 2024-09-14T06:32:31.858 INFO:teuthology.orchestra.run.smithi138.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:32:31.867 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-14T06:32:31.898 INFO:tasks.ceph.osd.1.smithi094.stderr:2024-09-14T06:32:31.896+0000 7f328e47a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33641) UID: 0 2024-09-14T06:32:32.001 INFO:tasks.ceph.osd.0.smithi094.stderr:2024-09-14T06:32:31.999+0000 7f07c9333640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33636) UID: 0 2024-09-14T06:32:32.009 INFO:teuthology.orchestra.run.smithi138.stdout:ERROR: (22) Invalid argument 2024-09-14T06:32:32.009 INFO:teuthology.orchestra.run.smithi138.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:32:32.018 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-14T06:32:32.101 INFO:tasks.ceph.osd.5.smithi138.stderr:2024-09-14T06:32:32.100+0000 7fe13746f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33618) UID: 0 2024-09-14T06:32:32.121 INFO:teuthology.orchestra.run.smithi138.stdout:ERROR: (22) Invalid argument 2024-09-14T06:32:32.121 INFO:teuthology.orchestra.run.smithi138.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:32:32.129 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-14T06:32:32.213 INFO:tasks.ceph.osd.6.smithi138.stderr:2024-09-14T06:32:32.212+0000 7f8cd8849640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33620) UID: 0 2024-09-14T06:32:32.272 INFO:teuthology.orchestra.run.smithi138.stdout:ERROR: (22) Invalid argument 2024-09-14T06:32:32.272 INFO:teuthology.orchestra.run.smithi138.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:32:32.281 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-14T06:32:32.303 INFO:tasks.ceph.osd.2.smithi094.stderr:2024-09-14T06:32:32.302+0000 7faf3269b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33649) UID: 0 2024-09-14T06:32:32.404 INFO:tasks.ceph.osd.6.smithi138.stderr:2024-09-14T06:32:32.403+0000 7f8cd8849640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33620) UID: 0 2024-09-14T06:32:32.425 INFO:teuthology.orchestra.run.smithi138.stdout:ERROR: (22) Invalid argument 2024-09-14T06:32:32.425 INFO:teuthology.orchestra.run.smithi138.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:32:32.433 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-14T06:32:32.517 INFO:tasks.ceph.osd.5.smithi138.stderr:2024-09-14T06:32:32.516+0000 7fe13746f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33618) UID: 0 2024-09-14T06:32:32.575 INFO:teuthology.orchestra.run.smithi138.stdout:ERROR: (22) Invalid argument 2024-09-14T06:32:32.576 INFO:teuthology.orchestra.run.smithi138.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:32:32.584 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-14T06:32:32.605 INFO:tasks.ceph.osd.1.smithi094.stderr:2024-09-14T06:32:32.603+0000 7f328e47a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33641) UID: 0 2024-09-14T06:32:32.628 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-14T06:32:32.706 INFO:tasks.ceph.osd.7.smithi138.stderr:2024-09-14T06:32:32.705+0000 7f3c6a21d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33623) UID: 0 2024-09-14T06:32:32.728 INFO:teuthology.orchestra.run.smithi138.stdout:ERROR: (22) Invalid argument 2024-09-14T06:32:32.728 INFO:teuthology.orchestra.run.smithi138.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:32:32.737 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-14T06:32:32.806 INFO:tasks.ceph.osd.3.smithi094.stderr:2024-09-14T06:32:32.804+0000 7feeeafe8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33655) UID: 0 2024-09-14T06:32:32.870 INFO:teuthology.orchestra.run.smithi094.stdout:ERROR: (22) Invalid argument 2024-09-14T06:32:32.870 INFO:teuthology.orchestra.run.smithi094.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:32:32.880 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-14T06:32:32.921 INFO:tasks.ceph.osd.3.smithi094.stderr:2024-09-14T06:32:32.919+0000 7feeeafe8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33655) UID: 0 2024-09-14T06:32:33.008 INFO:tasks.ceph.osd.2.smithi094.stderr:2024-09-14T06:32:33.006+0000 7faf3269b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33649) UID: 0 2024-09-14T06:32:33.023 INFO:teuthology.orchestra.run.smithi094.stdout:ERROR: (22) Invalid argument 2024-09-14T06:32:33.023 INFO:teuthology.orchestra.run.smithi094.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:32:33.032 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-14T06:32:33.109 INFO:tasks.ceph.osd.6.smithi138.stderr:2024-09-14T06:32:33.108+0000 7f8cd8849640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33620) UID: 0 2024-09-14T06:32:33.174 INFO:teuthology.orchestra.run.smithi094.stdout:ERROR: (22) Invalid argument 2024-09-14T06:32:33.175 INFO:teuthology.orchestra.run.smithi094.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:32:33.183 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-14T06:32:33.227 INFO:tasks.ceph.osd.3.smithi094.stderr:2024-09-14T06:32:33.225+0000 7feeeafe8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33655) UID: 0 2024-09-14T06:32:33.287 INFO:teuthology.orchestra.run.smithi094.stdout:ERROR: (22) Invalid argument 2024-09-14T06:32:33.287 INFO:teuthology.orchestra.run.smithi094.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:32:33.295 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-14T06:32:33.311 INFO:tasks.ceph.osd.5.smithi138.stderr:2024-09-14T06:32:33.310+0000 7fe13746f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33618) UID: 0 2024-09-14T06:32:33.412 INFO:tasks.ceph.osd.7.smithi138.stderr:2024-09-14T06:32:33.411+0000 7f3c6a21d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33623) UID: 0 2024-09-14T06:32:33.437 INFO:teuthology.orchestra.run.smithi094.stdout:ERROR: (22) Invalid argument 2024-09-14T06:32:33.437 INFO:teuthology.orchestra.run.smithi094.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:32:33.445 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-14T06:32:33.453 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-14T06:32:33.532 INFO:tasks.ceph.osd.3.smithi094.stderr:2024-09-14T06:32:33.530+0000 7feeeafe8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33655) UID: 0 2024-09-14T06:32:33.551 INFO:teuthology.orchestra.run.smithi094.stdout:ERROR: (22) Invalid argument 2024-09-14T06:32:33.551 INFO:teuthology.orchestra.run.smithi094.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:32:33.560 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-14T06:32:33.610 INFO:teuthology.orchestra.run.smithi094.stdout:osd.0: {} 2024-09-14T06:32:33.610 INFO:teuthology.orchestra.run.smithi094.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-14T06:32:33.614 INFO:tasks.ceph.osd.5.smithi138.stderr:2024-09-14T06:32:33.613+0000 7fe13746f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33618) UID: 0 2024-09-14T06:32:33.628 INFO:teuthology.orchestra.run.smithi094.stdout:osd.1: {} 2024-09-14T06:32:33.628 INFO:teuthology.orchestra.run.smithi094.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-14T06:32:33.644 INFO:teuthology.orchestra.run.smithi094.stdout:osd.2: {} 2024-09-14T06:32:33.644 INFO:teuthology.orchestra.run.smithi094.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-14T06:32:33.661 INFO:teuthology.orchestra.run.smithi094.stdout:osd.3: {} 2024-09-14T06:32:33.661 INFO:teuthology.orchestra.run.smithi094.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-14T06:32:33.678 INFO:teuthology.orchestra.run.smithi094.stdout:osd.4: {} 2024-09-14T06:32:33.678 INFO:teuthology.orchestra.run.smithi094.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-14T06:32:33.695 INFO:teuthology.orchestra.run.smithi094.stdout:osd.5: {} 2024-09-14T06:32:33.696 INFO:teuthology.orchestra.run.smithi094.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-14T06:32:33.707 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-14T06:32:33.708 INFO:teuthology.orchestra.run.smithi094.stdout: "ops": [], 2024-09-14T06:32:33.708 INFO:teuthology.orchestra.run.smithi094.stdout: "num_ops": 0 2024-09-14T06:32:33.708 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-14T06:32:33.713 INFO:teuthology.orchestra.run.smithi094.stdout:osd.6: {} 2024-09-14T06:32:33.713 INFO:teuthology.orchestra.run.smithi094.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-14T06:32:33.715 INFO:tasks.ceph.osd.5.smithi138.stderr:2024-09-14T06:32:33.714+0000 7fe13746f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33618) UID: 0 2024-09-14T06:32:33.717 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-14T06:32:33.730 INFO:teuthology.orchestra.run.smithi094.stdout:osd.7: {} 2024-09-14T06:32:33.730 INFO:teuthology.orchestra.run.smithi094.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-14T06:32:33.752 INFO:teuthology.orchestra.run.smithi094.stderr:nodeep-scrub is unset 2024-09-14T06:32:33.816 INFO:tasks.ceph.osd.1.smithi094.stderr:2024-09-14T06:32:33.814+0000 7f328e47a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33641) UID: 0 2024-09-14T06:32:33.850 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-14T06:32:33.850 INFO:teuthology.orchestra.run.smithi094.stdout: "ops": [], 2024-09-14T06:32:33.850 INFO:teuthology.orchestra.run.smithi094.stdout: "complaint_time": 30, 2024-09-14T06:32:33.851 INFO:teuthology.orchestra.run.smithi094.stdout: "num_blocked_ops": 0 2024-09-14T06:32:33.851 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-14T06:32:33.860 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-14T06:32:33.944 INFO:tasks.ceph.osd.0.smithi094.stderr:2024-09-14T06:32:33.942+0000 7f07c9333640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33636) UID: 0 2024-09-14T06:32:34.005 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-14T06:32:34.005 INFO:teuthology.orchestra.run.smithi094.stdout: "size": 20, 2024-09-14T06:32:34.005 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 600, 2024-09-14T06:32:34.005 INFO:teuthology.orchestra.run.smithi094.stdout: "ops": [ 2024-09-14T06:32:34.005 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:34.005 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4397.0:45722 3.9 e64/23 3:91fed098:::rbd_data.112d5a737e72.0000000000000001:head v 64'61, mlcod=62'59)", 2024-09-14T06:32:34.006 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:43.864269+0000", 2024-09-14T06:32:34.006 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 170.13866294799999, 2024-09-14T06:32:34.006 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.003055224, 2024-09-14T06:32:34.006 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:32:34.006 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:32:34.007 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:32:34.007 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:32:34.007 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:34.007 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:32:34.007 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.864269+0000", 2024-09-14T06:32:34.008 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:34.008 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:34.008 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:34.008 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:32:34.008 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.864269+0000", 2024-09-14T06:32:34.008 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:34.009 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:34.009 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:34.009 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:32:34.009 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.864269+0000", 2024-09-14T06:32:34.009 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.41e-07 2024-09-14T06:32:34.009 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:34.010 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:34.010 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:32:34.010 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.865490+0000", 2024-09-14T06:32:34.010 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.001220552 2024-09-14T06:32:34.010 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:34.010 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:34.010 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:32:34.011 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.865506+0000", 2024-09-14T06:32:34.011 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.6263999999999999e-05 2024-09-14T06:32:34.011 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:34.011 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:34.011 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:32:34.011 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.865516+0000", 2024-09-14T06:32:34.012 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.0091999999999999e-05 2024-09-14T06:32:34.012 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:34.012 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:34.012 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:32:34.012 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.865583+0000", 2024-09-14T06:32:34.012 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.6676999999999999e-05 2024-09-14T06:32:34.013 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:34.013 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:34.013 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:32:34.013 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.865610+0000", 2024-09-14T06:32:34.013 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.6529999999999998e-05 2024-09-14T06:32:34.013 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:34.013 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:34.013 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-14T06:32:34.013 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.867281+0000", 2024-09-14T06:32:34.016 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.001671203 2024-09-14T06:32:34.016 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:34.016 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:34.016 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:32:34.017 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.867324+0000", 2024-09-14T06:32:34.017 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.3665000000000003e-05 2024-09-14T06:32:34.017 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:34.017 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:32:34.017 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:34.017 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:34.018 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:34.018 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4397.0:74241 3.0 e73/23 3:06e93fb9:::rbd_data.112d80ca42af.0000000000000000:head v 73'102, mlcod=71'100)", 2024-09-14T06:32:34.018 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:53.876730+0000", 2024-09-14T06:32:34.018 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 160.12620184799999, 2024-09-14T06:32:34.018 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0079470329999999992, 2024-09-14T06:32:34.018 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:32:34.018 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:32:34.018 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:32:34.019 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:32:34.019 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:34.019 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:32:34.019 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:53.876730+0000", 2024-09-14T06:32:34.019 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:34.019 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:34.019 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:34.020 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:32:34.020 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:53.876730+0000", 2024-09-14T06:32:34.020 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:34.020 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:34.020 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:34.020 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:32:34.021 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:53.876731+0000", 2024-09-14T06:32:34.021 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.0500000000000004e-07 2024-09-14T06:32:34.021 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:34.021 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:34.021 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:32:34.021 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:53.879616+0000", 2024-09-14T06:32:34.022 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0028856680000000001 2024-09-14T06:32:34.022 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:34.022 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:34.022 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:32:34.022 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:53.879643+0000", 2024-09-14T06:32:34.022 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.6120999999999999e-05 2024-09-14T06:32:34.023 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:34.023 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:34.023 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:32:34.023 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:53.879654+0000", 2024-09-14T06:32:34.023 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.1586999999999999e-05 2024-09-14T06:32:34.023 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:34.023 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:34.023 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:32:34.024 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:53.879756+0000", 2024-09-14T06:32:34.024 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00010223899999999999 2024-09-14T06:32:34.024 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:34.024 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:34.024 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:32:34.024 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:53.879783+0000", 2024-09-14T06:32:34.024 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.6913000000000001e-05 2024-09-14T06:32:34.025 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:34.025 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:34.025 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-14T06:32:34.025 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:53.884594+0000", 2024-09-14T06:32:34.025 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0048110729999999999 2024-09-14T06:32:34.025 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:34.026 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:34.026 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:32:34.026 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:53.884677+0000", 2024-09-14T06:32:34.026 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.2927000000000001e-05 2024-09-14T06:32:34.026 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:34.026 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:32:34.027 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:34.027 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:34.027 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:34.027 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4397.0:74243 3.9 e73/23 3:94977f81:::rbd_data.112d80ca42af.0000000000000002:head v 73'94, mlcod=71'92)", 2024-09-14T06:32:34.027 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:53.885156+0000", 2024-09-14T06:32:34.027 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 160.11777592999999, 2024-09-14T06:32:34.027 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0096009860000000006, 2024-09-14T06:32:34.028 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:32:34.028 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:32:34.028 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:32:34.028 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:32:34.028 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:34.028 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:32:34.029 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:53.885156+0000", 2024-09-14T06:32:34.029 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:34.029 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:34.029 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:34.029 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:32:34.029 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:53.885156+0000", 2024-09-14T06:32:34.030 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:34.030 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:34.030 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:34.030 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:32:34.030 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:53.885156+0000", 2024-09-14T06:32:34.030 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.23e-07 2024-09-14T06:32:34.031 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:34.031 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:34.031 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:32:34.031 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:53.889694+0000", 2024-09-14T06:32:34.031 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0045378709999999997 2024-09-14T06:32:34.031 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:34.031 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:34.031 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:32:34.031 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:53.889720+0000", 2024-09-14T06:32:34.031 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.5259e-05 2024-09-14T06:32:34.031 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:34.031 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:34.032 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:32:34.032 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:53.889731+0000", 2024-09-14T06:32:34.032 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.1739000000000001e-05 2024-09-14T06:32:34.032 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:34.032 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:34.032 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:32:34.032 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:53.889822+0000", 2024-09-14T06:32:34.032 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.1200999999999996e-05 2024-09-14T06:32:34.032 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:34.032 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:34.033 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:32:34.033 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:53.889850+0000", 2024-09-14T06:32:34.033 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.7084999999999999e-05 2024-09-14T06:32:34.033 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:34.033 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:34.033 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-14T06:32:34.033 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:53.894703+0000", 2024-09-14T06:32:34.033 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0048532690000000003 2024-09-14T06:32:34.033 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:34.033 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:34.033 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:32:34.033 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:53.894757+0000", 2024-09-14T06:32:34.033 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.4339000000000003e-05 2024-09-14T06:32:34.034 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:34.034 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:32:34.034 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:34.034 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:34.034 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:34.034 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4397.0:74249 3.0 e73/23 3:009b0225:::rbd_data.112d80ca42af.0000000000000009:head v 73'103, mlcod=71'101)", 2024-09-14T06:32:34.034 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:53.893219+0000", 2024-09-14T06:32:34.034 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 160.10971345199999, 2024-09-14T06:32:34.034 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0065381980000000003, 2024-09-14T06:32:34.034 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:32:34.034 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:32:34.034 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:32:34.034 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:32:34.034 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:34.035 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:32:34.035 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:53.893219+0000", 2024-09-14T06:32:34.035 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:34.035 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:34.035 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:34.035 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:32:34.035 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:53.893219+0000", 2024-09-14T06:32:34.035 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:34.035 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:34.035 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:34.035 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:32:34.035 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:53.893221+0000", 2024-09-14T06:32:34.035 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.4329999999999998e-06 2024-09-14T06:32:34.035 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:34.036 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:34.036 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:32:34.036 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:53.895968+0000", 2024-09-14T06:32:34.036 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.002746746 2024-09-14T06:32:34.036 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:34.036 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:34.036 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:32:34.036 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:53.895988+0000", 2024-09-14T06:32:34.036 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.0227000000000001e-05 2024-09-14T06:32:34.036 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:34.036 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:34.036 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:32:34.037 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:53.896000+0000", 2024-09-14T06:32:34.037 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.2359e-05 2024-09-14T06:32:34.037 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:34.037 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:34.037 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:32:34.037 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:53.896084+0000", 2024-09-14T06:32:34.037 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.3659999999999995e-05 2024-09-14T06:32:34.037 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:34.037 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:34.037 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:32:34.037 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:53.896109+0000", 2024-09-14T06:32:34.037 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.5259999999999999e-05 2024-09-14T06:32:34.037 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:34.038 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:34.038 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-14T06:32:34.038 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:53.899704+0000", 2024-09-14T06:32:34.038 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0035942209999999999 2024-09-14T06:32:34.038 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:34.038 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:34.038 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:32:34.038 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:53.899757+0000", 2024-09-14T06:32:34.038 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.3291999999999997e-05 2024-09-14T06:32:34.038 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:34.038 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:32:34.038 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:34.038 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:34.038 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:34.038 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4397.0:78263 3.0 e77/23 3:00899b22:::rbd_data.112ddfbec471.0000000000000016:head v 77'104, mlcod=73'102)", 2024-09-14T06:32:34.039 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:57.634578+0000", 2024-09-14T06:32:34.039 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 156.36835439999999, 2024-09-14T06:32:34.039 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.002372454, 2024-09-14T06:32:34.039 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:32:34.039 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:32:34.039 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:32:34.039 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:32:34.039 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:34.039 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:32:34.039 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:57.634578+0000", 2024-09-14T06:32:34.039 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:34.039 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:34.039 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:34.040 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:32:34.040 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:57.634578+0000", 2024-09-14T06:32:34.040 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:34.040 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:34.040 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:34.040 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:32:34.040 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:57.634578+0000", 2024-09-14T06:32:34.040 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.5699999999999998e-07 2024-09-14T06:32:34.040 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:34.040 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:34.040 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:32:34.040 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:57.635347+0000", 2024-09-14T06:32:34.040 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00076848300000000003 2024-09-14T06:32:34.041 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:34.041 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:34.041 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:32:34.041 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:57.635361+0000", 2024-09-14T06:32:34.041 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.4238e-05 2024-09-14T06:32:34.041 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:34.041 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:34.042 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:32:34.042 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:57.635371+0000", 2024-09-14T06:32:34.042 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.0087e-05 2024-09-14T06:32:34.042 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:34.042 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:34.042 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:32:34.043 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:57.635439+0000", 2024-09-14T06:32:34.043 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.7953000000000005e-05 2024-09-14T06:32:34.043 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:34.043 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:34.043 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:32:34.044 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:57.635464+0000", 2024-09-14T06:32:34.044 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.5062000000000002e-05 2024-09-14T06:32:34.044 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:34.044 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:34.044 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-14T06:32:34.045 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:57.636914+0000", 2024-09-14T06:32:34.045 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.001449625 2024-09-14T06:32:34.045 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:34.045 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:34.045 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:32:34.045 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:57.636950+0000", 2024-09-14T06:32:34.046 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.6548999999999999e-05 2024-09-14T06:32:34.046 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:34.046 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:32:34.046 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:34.046 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:34.046 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:34.046 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4397.0:78604 3.9 e77/23 3:94c62a0e:::rbd_data.112ddfbec471.000000000000002c:head v 77'96, mlcod=73'94)", 2024-09-14T06:32:34.047 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:57.677568+0000", 2024-09-14T06:32:34.047 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 156.32536401199999, 2024-09-14T06:32:34.047 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.002564846, 2024-09-14T06:32:34.047 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:32:34.047 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:32:34.047 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:32:34.048 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:32:34.048 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:34.048 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:32:34.048 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:57.677568+0000", 2024-09-14T06:32:34.048 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:34.048 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:34.049 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:34.049 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:32:34.049 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:57.677568+0000", 2024-09-14T06:32:34.049 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:34.049 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:34.049 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:34.050 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:32:34.050 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:57.677568+0000", 2024-09-14T06:32:34.050 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.8900000000000001e-07 2024-09-14T06:32:34.050 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:34.050 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:34.050 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:32:34.051 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:57.678488+0000", 2024-09-14T06:32:34.051 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00091963899999999998 2024-09-14T06:32:34.051 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:34.051 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:34.051 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:32:34.052 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:57.678500+0000", 2024-09-14T06:32:34.052 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.1902000000000001e-05 2024-09-14T06:32:34.052 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:34.052 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:34.052 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:32:34.052 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:57.678507+0000", 2024-09-14T06:32:34.052 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.7739999999999999e-06 2024-09-14T06:32:34.053 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:34.053 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:34.053 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:32:34.053 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:57.678586+0000", 2024-09-14T06:32:34.053 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.9103000000000005e-05 2024-09-14T06:32:34.053 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:34.053 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:34.053 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:32:34.054 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:57.678614+0000", 2024-09-14T06:32:34.054 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.7912e-05 2024-09-14T06:32:34.054 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:34.054 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:34.054 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-14T06:32:34.054 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:57.680075+0000", 2024-09-14T06:32:34.055 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0014613370000000001 2024-09-14T06:32:34.055 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:34.055 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:34.055 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:32:34.055 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:57.680133+0000", 2024-09-14T06:32:34.055 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.7890000000000003e-05 2024-09-14T06:32:34.056 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:34.056 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:32:34.056 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:34.056 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:34.056 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:34.056 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4397.0:149885 3.9 e122/23 3:951acef3:::rbd_data.112d1bb7b269.0000000000000005:head v 122'176, mlcod=122'172)", 2024-09-14T06:32:34.057 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:30:44.739092+0000", 2024-09-14T06:32:34.057 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 109.263839702, 2024-09-14T06:32:34.057 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0028236659999999998, 2024-09-14T06:32:34.057 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:32:34.057 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:32:34.058 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:32:34.058 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:32:34.058 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:34.058 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:32:34.058 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.739092+0000", 2024-09-14T06:32:34.058 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:34.059 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:34.059 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:34.059 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:32:34.059 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.739092+0000", 2024-09-14T06:32:34.059 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:34.060 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:34.060 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:34.060 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:32:34.060 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.739093+0000", 2024-09-14T06:32:34.060 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.0109999999999999e-06 2024-09-14T06:32:34.060 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:34.060 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:34.060 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:32:34.061 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.740189+0000", 2024-09-14T06:32:34.061 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00109527 2024-09-14T06:32:34.061 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:34.061 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:34.061 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:32:34.061 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.740199+0000", 2024-09-14T06:32:34.061 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.0458999999999999e-05 2024-09-14T06:32:34.061 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:34.062 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:34.062 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:32:34.062 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.740207+0000", 2024-09-14T06:32:34.062 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.4499999999999998e-06 2024-09-14T06:32:34.062 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:34.062 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:34.062 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:32:34.062 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.740295+0000", 2024-09-14T06:32:34.063 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.8740999999999996e-05 2024-09-14T06:32:34.063 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:34.063 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:34.063 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:32:34.063 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.740317+0000", 2024-09-14T06:32:34.063 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.1523e-05 2024-09-14T06:32:34.063 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:34.063 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:34.064 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-14T06:32:34.064 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.741865+0000", 2024-09-14T06:32:34.064 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.001548315 2024-09-14T06:32:34.064 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:34.064 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:34.064 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:32:34.065 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.741916+0000", 2024-09-14T06:32:34.065 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.0896999999999997e-05 2024-09-14T06:32:34.065 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:34.065 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:32:34.065 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:34.065 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:34.066 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:34.066 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4397.0:149933 3.4 e122/23 3:234f5229:::rbd_data.112d1bb7b269.0000000000000009:head v 122'221, mlcod=120'218)", 2024-09-14T06:32:34.066 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:30:44.743849+0000", 2024-09-14T06:32:34.066 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 109.259083285, 2024-09-14T06:32:34.066 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0031479519999999999, 2024-09-14T06:32:34.066 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:32:34.066 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:32:34.066 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:32:34.066 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:32:34.067 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:34.067 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:32:34.067 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.743849+0000", 2024-09-14T06:32:34.067 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:34.067 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:34.067 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:34.067 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:32:34.067 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.743849+0000", 2024-09-14T06:32:34.067 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:34.067 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:34.067 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:34.067 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:32:34.067 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.743850+0000", 2024-09-14T06:32:34.068 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.279e-06 2024-09-14T06:32:34.068 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:34.068 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:34.068 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:32:34.068 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.745151+0000", 2024-09-14T06:32:34.068 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.001300495 2024-09-14T06:32:34.068 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:34.068 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:34.068 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:32:34.068 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.745167+0000", 2024-09-14T06:32:34.068 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.6478e-05 2024-09-14T06:32:34.069 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:34.069 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:34.069 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:32:34.069 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.745175+0000", 2024-09-14T06:32:34.069 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.2679999999999994e-06 2024-09-14T06:32:34.069 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:34.069 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:34.069 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:32:34.069 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.745226+0000", 2024-09-14T06:32:34.069 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.0192999999999999e-05 2024-09-14T06:32:34.069 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:34.069 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:34.069 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:32:34.070 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.745247+0000", 2024-09-14T06:32:34.070 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.1767e-05 2024-09-14T06:32:34.070 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:34.070 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:34.070 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-14T06:32:34.070 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.746952+0000", 2024-09-14T06:32:34.070 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0017042769999999999 2024-09-14T06:32:34.070 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:34.070 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:34.070 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:32:34.070 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.746997+0000", 2024-09-14T06:32:34.070 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.5194999999999997e-05 2024-09-14T06:32:34.070 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:34.070 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:32:34.071 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:34.071 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:34.071 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:34.071 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4397.0:150494 3.9 e122/23 3:955152a4:::rbd_object_map.112d1bb7b269.0000000000000038:head v 122'194, mlcod=122'190)", 2024-09-14T06:32:34.071 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:30:44.781585+0000", 2024-09-14T06:32:34.071 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 109.221347393, 2024-09-14T06:32:34.071 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0026957499999999998, 2024-09-14T06:32:34.071 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:32:34.071 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:32:34.071 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:32:34.071 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:32:34.071 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:34.071 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:32:34.071 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.781585+0000", 2024-09-14T06:32:34.072 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:34.072 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:34.072 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:34.072 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:32:34.072 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.781585+0000", 2024-09-14T06:32:34.072 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:34.072 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:34.072 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:34.072 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:32:34.072 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.781585+0000", 2024-09-14T06:32:34.072 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.5000000000000002e-07 2024-09-14T06:32:34.072 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:34.072 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:34.073 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:32:34.073 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.781594+0000", 2024-09-14T06:32:34.073 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.4640000000000006e-06 2024-09-14T06:32:34.073 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:34.073 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:34.073 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:32:34.073 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.781604+0000", 2024-09-14T06:32:34.073 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.0071999999999999e-05 2024-09-14T06:32:34.073 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:34.073 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:34.073 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:32:34.073 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.781610+0000", 2024-09-14T06:32:34.073 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.3609999999999997e-06 2024-09-14T06:32:34.073 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:34.073 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:34.074 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:32:34.074 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.781676+0000", 2024-09-14T06:32:34.074 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.5963000000000005e-05 2024-09-14T06:32:34.074 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:34.074 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:34.074 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:32:34.074 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.781696+0000", 2024-09-14T06:32:34.074 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.9278000000000001e-05 2024-09-14T06:32:34.074 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:34.074 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:34.074 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-14T06:32:34.074 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.784231+0000", 2024-09-14T06:32:34.075 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.002535093 2024-09-14T06:32:34.075 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:34.075 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:34.075 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:32:34.075 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.784280+0000", 2024-09-14T06:32:34.075 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.9869e-05 2024-09-14T06:32:34.075 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:34.075 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:32:34.075 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:34.075 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:34.075 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:34.075 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4397.0:150898 3.9 e122/23 3:97c98f90:::rbd_data.112d1bb7b269.0000000000000025:head v 122'206, mlcod=122'203)", 2024-09-14T06:32:34.075 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:30:44.807232+0000", 2024-09-14T06:32:34.075 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 109.195700476, 2024-09-14T06:32:34.075 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0022297889999999998, 2024-09-14T06:32:34.076 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:32:34.076 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:32:34.076 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:32:34.076 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:32:34.076 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:34.076 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:32:34.076 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.807232+0000", 2024-09-14T06:32:34.076 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:34.076 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:34.076 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:34.076 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:32:34.076 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.807232+0000", 2024-09-14T06:32:34.076 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:34.077 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:34.077 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:34.077 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:32:34.077 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.807233+0000", 2024-09-14T06:32:34.077 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.5710000000000001e-06 2024-09-14T06:32:34.078 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:34.078 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:34.078 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:32:34.078 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.807944+0000", 2024-09-14T06:32:34.078 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00071056299999999999 2024-09-14T06:32:34.078 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:34.078 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:34.078 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:32:34.078 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.807961+0000", 2024-09-14T06:32:34.078 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.7689e-05 2024-09-14T06:32:34.078 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:34.078 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:34.079 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:32:34.079 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.807971+0000", 2024-09-14T06:32:34.079 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.2e-06 2024-09-14T06:32:34.079 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:34.079 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:34.079 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:32:34.079 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.808050+0000", 2024-09-14T06:32:34.079 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.9308999999999996e-05 2024-09-14T06:32:34.079 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:34.079 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:34.079 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:32:34.079 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.808074+0000", 2024-09-14T06:32:34.079 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.4147000000000001e-05 2024-09-14T06:32:34.079 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:34.079 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:34.080 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-14T06:32:34.080 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.809415+0000", 2024-09-14T06:32:34.080 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.001341257 2024-09-14T06:32:34.080 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:34.080 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:34.080 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:32:34.080 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.809461+0000", 2024-09-14T06:32:34.080 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.6053000000000001e-05 2024-09-14T06:32:34.080 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:34.080 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:32:34.080 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:34.080 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:34.081 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:34.081 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4397.0:202551 3.9 e166/23 3:965c09b6:::rbd_data.112d3bd84cb0.000000000000000a:head v 166'416, mlcod=164'414)", 2024-09-14T06:32:34.081 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:31:30.657011+0000", 2024-09-14T06:32:34.081 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 63.345921443000002, 2024-09-14T06:32:34.081 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.002509262, 2024-09-14T06:32:34.081 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:32:34.081 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:32:34.081 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:32:34.081 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:32:34.081 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:34.081 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:32:34.081 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:30.657011+0000", 2024-09-14T06:32:34.081 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:34.082 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:34.082 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:34.082 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:32:34.082 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:30.657011+0000", 2024-09-14T06:32:34.082 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:34.082 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:34.082 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:34.082 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:32:34.082 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:30.657011+0000", 2024-09-14T06:32:34.082 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.08e-07 2024-09-14T06:32:34.082 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:34.082 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:34.082 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:32:34.082 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:30.658467+0000", 2024-09-14T06:32:34.082 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.001455708 2024-09-14T06:32:34.083 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:34.083 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:34.083 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:32:34.083 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:30.658484+0000", 2024-09-14T06:32:34.083 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.6844e-05 2024-09-14T06:32:34.083 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:34.083 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:34.083 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:32:34.083 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:30.658493+0000", 2024-09-14T06:32:34.083 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.2790000000000004e-06 2024-09-14T06:32:34.083 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:34.083 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:34.083 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:32:34.083 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:30.658559+0000", 2024-09-14T06:32:34.084 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.6459999999999997e-05 2024-09-14T06:32:34.084 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:34.084 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:34.084 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:32:34.084 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:30.658574+0000", 2024-09-14T06:32:34.084 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.5048000000000001e-05 2024-09-14T06:32:34.084 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:34.084 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:34.084 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-14T06:32:34.084 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:30.659473+0000", 2024-09-14T06:32:34.084 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00089842800000000005 2024-09-14T06:32:34.084 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:34.085 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:34.085 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:32:34.085 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:30.659520+0000", 2024-09-14T06:32:34.085 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.7086999999999998e-05 2024-09-14T06:32:34.085 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:34.085 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:32:34.085 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:34.085 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:34.085 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:34.085 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4397.0:202665 3.0 e166/23 3:00eeadfb:::rbd_data.112d3bd84cb0.000000000000000e:head v 166'920, mlcod=163'918)", 2024-09-14T06:32:34.085 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:31:30.662434+0000", 2024-09-14T06:32:34.085 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 63.340497654000004, 2024-09-14T06:32:34.085 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.002522121, 2024-09-14T06:32:34.086 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:32:34.086 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:32:34.086 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:32:34.086 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:32:34.086 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:34.086 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:32:34.086 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:30.662434+0000", 2024-09-14T06:32:34.086 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:34.086 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:34.086 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:34.086 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:32:34.086 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:30.662434+0000", 2024-09-14T06:32:34.086 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:34.086 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:34.087 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:34.087 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:32:34.087 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:30.662436+0000", 2024-09-14T06:32:34.087 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.257e-06 2024-09-14T06:32:34.087 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:34.087 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:34.087 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:32:34.087 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:30.662527+0000", 2024-09-14T06:32:34.087 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.1644000000000006e-05 2024-09-14T06:32:34.087 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:34.087 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:34.087 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:32:34.087 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:30.662546+0000", 2024-09-14T06:32:34.088 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.8998e-05 2024-09-14T06:32:34.088 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:34.088 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:34.088 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:32:34.088 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:30.662556+0000", 2024-09-14T06:32:34.088 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.9809999999999998e-06 2024-09-14T06:32:34.088 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:34.088 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:34.088 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:32:34.089 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:30.662621+0000", 2024-09-14T06:32:34.089 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.4597000000000005e-05 2024-09-14T06:32:34.089 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:34.089 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:34.089 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:32:34.089 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:30.662643+0000", 2024-09-14T06:32:34.089 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.1608e-05 2024-09-14T06:32:34.089 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:34.089 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:34.089 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-14T06:32:34.089 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:30.664909+0000", 2024-09-14T06:32:34.089 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0022668689999999999 2024-09-14T06:32:34.089 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:34.089 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:34.090 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:32:34.090 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:30.664957+0000", 2024-09-14T06:32:34.090 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.7166999999999998e-05 2024-09-14T06:32:34.090 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:34.090 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:32:34.090 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:34.090 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:34.090 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:34.090 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4397.0:204589 3.4 e166/23 3:234552f0:::rbd_data.112d3bd84cb0.000000000000003f:head v 166'499, mlcod=165'497)", 2024-09-14T06:32:34.090 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:31:30.772894+0000", 2024-09-14T06:32:34.090 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 63.230037797999998, 2024-09-14T06:32:34.090 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0034564650000000001, 2024-09-14T06:32:34.090 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:32:34.090 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:32:34.090 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:32:34.091 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:32:34.091 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:34.091 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:32:34.091 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:30.772894+0000", 2024-09-14T06:32:34.091 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:34.091 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:34.091 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:34.091 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:32:34.091 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:30.772894+0000", 2024-09-14T06:32:34.091 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:34.091 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:34.091 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:34.091 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:32:34.091 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:30.772896+0000", 2024-09-14T06:32:34.092 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.5379999999999999e-06 2024-09-14T06:32:34.092 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:34.092 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:34.092 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:32:34.092 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:30.773042+0000", 2024-09-14T06:32:34.092 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000146107 2024-09-14T06:32:34.092 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:34.092 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:34.092 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:32:34.092 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:30.773061+0000", 2024-09-14T06:32:34.092 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.8862999999999999e-05 2024-09-14T06:32:34.092 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:34.092 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:34.092 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:32:34.093 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:30.773071+0000", 2024-09-14T06:32:34.093 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.0227000000000001e-05 2024-09-14T06:32:34.093 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:34.093 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:34.093 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:32:34.093 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:30.773149+0000", 2024-09-14T06:32:34.093 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.8071999999999994e-05 2024-09-14T06:32:34.093 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:34.093 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:34.093 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:32:34.093 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:30.773171+0000", 2024-09-14T06:32:34.093 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.1699999999999999e-05 2024-09-14T06:32:34.094 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:34.094 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:34.094 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-14T06:32:34.094 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:30.776285+0000", 2024-09-14T06:32:34.094 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0031142050000000001 2024-09-14T06:32:34.094 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:34.094 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:34.094 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:32:34.094 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:30.776351+0000", 2024-09-14T06:32:34.094 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.5753000000000006e-05 2024-09-14T06:32:34.094 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:34.094 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:32:34.094 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:34.094 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:34.094 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:34.094 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4397.0:204598 3.4 e166/23 3:245d069e:::rbd_data.112d3bd84cb0.0000000000000041:head v 166'500, mlcod=165'497)", 2024-09-14T06:32:34.095 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:31:30.775247+0000", 2024-09-14T06:32:34.095 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 63.227684674999999, 2024-09-14T06:32:34.095 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00318235, 2024-09-14T06:32:34.095 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:32:34.095 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:32:34.095 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:32:34.095 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:32:34.095 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:34.095 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:32:34.095 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:30.775247+0000", 2024-09-14T06:32:34.095 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:34.095 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:34.095 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:34.095 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:32:34.095 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:30.775247+0000", 2024-09-14T06:32:34.096 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:34.096 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:34.096 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:34.096 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:32:34.096 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:30.775249+0000", 2024-09-14T06:32:34.096 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.265e-06 2024-09-14T06:32:34.096 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:34.096 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:34.096 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:32:34.096 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:30.775340+0000", 2024-09-14T06:32:34.096 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.1131999999999994e-05 2024-09-14T06:32:34.096 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:34.096 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:34.096 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:32:34.097 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:30.775352+0000", 2024-09-14T06:32:34.097 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.2294e-05 2024-09-14T06:32:34.097 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:34.097 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:34.097 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:32:34.097 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:30.775361+0000", 2024-09-14T06:32:34.097 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.6950000000000006e-06 2024-09-14T06:32:34.097 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:34.097 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:34.097 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:32:34.097 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:30.775439+0000", 2024-09-14T06:32:34.097 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.7910000000000005e-05 2024-09-14T06:32:34.097 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:34.097 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:34.097 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:32:34.098 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:30.775460+0000", 2024-09-14T06:32:34.098 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.0900999999999999e-05 2024-09-14T06:32:34.098 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:34.098 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:34.098 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-14T06:32:34.098 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:30.778385+0000", 2024-09-14T06:32:34.098 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.002925539 2024-09-14T06:32:34.098 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:34.098 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:34.098 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:32:34.098 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:30.778430+0000", 2024-09-14T06:32:34.098 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.4613999999999997e-05 2024-09-14T06:32:34.099 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:34.099 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:32:34.099 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:34.099 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:34.099 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:34.099 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4397.0:204975 3.4 e166/23 3:20972d18:::rbd_data.112d3bd84cb0.0000000000000046:head v 166'501, mlcod=166'499)", 2024-09-14T06:32:34.099 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:31:30.787253+0000", 2024-09-14T06:32:34.099 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 63.215678646000001, 2024-09-14T06:32:34.099 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.002248318, 2024-09-14T06:32:34.099 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:32:34.099 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:32:34.099 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:32:34.099 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:32:34.099 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:34.100 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:32:34.100 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:30.787253+0000", 2024-09-14T06:32:34.100 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:34.100 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:34.100 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:34.100 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:32:34.100 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:30.787253+0000", 2024-09-14T06:32:34.100 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:34.100 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:34.100 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:34.100 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:32:34.100 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:30.787255+0000", 2024-09-14T06:32:34.100 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.271e-06 2024-09-14T06:32:34.100 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:34.100 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:34.101 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:32:34.101 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:30.787442+0000", 2024-09-14T06:32:34.101 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00018744000000000001 2024-09-14T06:32:34.101 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:34.101 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:34.101 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:32:34.101 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:30.787461+0000", 2024-09-14T06:32:34.101 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.9054000000000001e-05 2024-09-14T06:32:34.101 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:34.101 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:34.101 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:32:34.101 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:30.787470+0000", 2024-09-14T06:32:34.101 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.0929999999999992e-06 2024-09-14T06:32:34.101 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:34.101 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:34.102 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:32:34.102 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:30.788004+0000", 2024-09-14T06:32:34.102 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00053359399999999997 2024-09-14T06:32:34.102 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:34.102 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:34.102 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:32:34.102 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:30.788034+0000", 2024-09-14T06:32:34.102 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.9635e-05 2024-09-14T06:32:34.102 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:34.102 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:34.102 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-14T06:32:34.102 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:30.789457+0000", 2024-09-14T06:32:34.102 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.001423487 2024-09-14T06:32:34.102 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:34.102 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:34.102 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:32:34.103 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:30.789502+0000", 2024-09-14T06:32:34.103 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.4743999999999997e-05 2024-09-14T06:32:34.103 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:34.103 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:32:34.103 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:34.103 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:34.103 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:34.103 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4397.0:212188 3.4 e166/23 3:23871bc7:::rbd_data.112d3bd84cb0.00000000000000fa:head v 166'508, mlcod=166'506)", 2024-09-14T06:32:34.103 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:31:31.391415+0000", 2024-09-14T06:32:34.103 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 62.611517063999997, 2024-09-14T06:32:34.103 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0022039049999999999, 2024-09-14T06:32:34.103 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:32:34.103 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:32:34.104 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:32:34.104 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:32:34.104 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:34.104 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:32:34.104 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:31.391415+0000", 2024-09-14T06:32:34.104 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:34.104 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:34.104 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:34.104 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:32:34.104 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:31.391415+0000", 2024-09-14T06:32:34.104 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:34.104 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:34.104 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:34.104 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:32:34.104 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:31.391416+0000", 2024-09-14T06:32:34.105 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.44e-06 2024-09-14T06:32:34.105 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:34.105 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:34.105 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:32:34.105 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:31.391554+0000", 2024-09-14T06:32:34.105 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000137119 2024-09-14T06:32:34.105 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:34.105 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:34.105 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:32:34.105 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:31.391570+0000", 2024-09-14T06:32:34.105 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.6827e-05 2024-09-14T06:32:34.105 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:34.105 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:34.106 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:32:34.106 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:31.391580+0000", 2024-09-14T06:32:34.106 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.3139999999999993e-06 2024-09-14T06:32:34.106 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:34.106 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:34.106 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:32:34.106 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:31.391664+0000", 2024-09-14T06:32:34.106 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.3757999999999993e-05 2024-09-14T06:32:34.106 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:34.106 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:34.106 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:32:34.106 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:31.391692+0000", 2024-09-14T06:32:34.106 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.8898e-05 2024-09-14T06:32:34.107 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:34.107 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:34.107 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-14T06:32:34.107 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:31.393584+0000", 2024-09-14T06:32:34.107 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.001891752 2024-09-14T06:32:34.107 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:34.107 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:34.107 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:32:34.107 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:31.393619+0000", 2024-09-14T06:32:34.107 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.4796999999999999e-05 2024-09-14T06:32:34.107 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:34.107 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:32:34.107 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:34.107 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:34.108 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:34.108 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "MOSDPGPush(3.5 169/168 [PushOp(3:a7c12b08:::rbd_data.112ddfbec471.000000000000000e:head, version: 77'100, data_included: [0~1048576], data_size: 1048576, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(3:a7c12b08:::rbd_data.112ddfbec471.000000000000000e:head@77'100, size: 1048576, copy_subset: [(0, 1048576)], clone_subset: {}, snapset: 0=[]:{}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 1048576, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2024-09-14T06:32:34.108 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:31:34.540707+0000", 2024-09-14T06:32:34.108 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 59.462225298999996, 2024-09-14T06:32:34.108 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0023637369999999999, 2024-09-14T06:32:34.108 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:32:34.108 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:32:34.108 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:32:34.108 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:32:34.109 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:34.109 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:32:34.109 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:34.540707+0000", 2024-09-14T06:32:34.109 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:34.109 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:34.109 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:34.109 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:32:34.109 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:34.540707+0000", 2024-09-14T06:32:34.109 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:34.109 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:34.110 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:34.110 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:32:34.110 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:34.540708+0000", 2024-09-14T06:32:34.110 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.7199999999999998e-07 2024-09-14T06:32:34.110 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:34.110 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:34.110 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:32:34.110 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:34.542150+0000", 2024-09-14T06:32:34.110 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0014423229999999999 2024-09-14T06:32:34.110 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:34.110 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:34.110 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:32:34.110 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:34.542169+0000", 2024-09-14T06:32:34.111 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.8756000000000001e-05 2024-09-14T06:32:34.111 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:34.111 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:34.111 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:32:34.111 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:34.542180+0000", 2024-09-14T06:32:34.111 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.1051e-05 2024-09-14T06:32:34.111 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:34.111 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:34.111 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:32:34.111 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:34.542275+0000", 2024-09-14T06:32:34.111 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.5122000000000002e-05 2024-09-14T06:32:34.111 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:34.111 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:34.111 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:32:34.112 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:34.542284+0000", 2024-09-14T06:32:34.112 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.9190000000000005e-06 2024-09-14T06:32:34.112 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:34.112 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:34.112 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:32:34.112 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:34.543071+0000", 2024-09-14T06:32:34.112 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00078679399999999995 2024-09-14T06:32:34.112 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:34.112 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:32:34.112 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:34.112 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:34.112 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:34.112 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "MOSDPGPush(3.5 169/168 [PushOp(3:a7a54476:::rbd_data.112d1bb7b269.0000000000000038:head, version: 122'581, data_included: [0~524288], data_size: 524288, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(3:a7a54476:::rbd_data.112d1bb7b269.0000000000000038:head@122'581, size: 524288, copy_subset: [(0, 524288)], clone_subset: {}, snapset: 0=[]:{}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 524288, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2024-09-14T06:32:34.112 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:31:34.667757+0000", 2024-09-14T06:32:34.113 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 59.335175253000003, 2024-09-14T06:32:34.113 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.002750382, 2024-09-14T06:32:34.113 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:32:34.113 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:32:34.113 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:32:34.113 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:32:34.113 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:34.113 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:32:34.113 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:34.667757+0000", 2024-09-14T06:32:34.113 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:34.113 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:34.113 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:34.113 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:32:34.114 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:34.667757+0000", 2024-09-14T06:32:34.114 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:34.114 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:34.114 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:34.114 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:32:34.114 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:34.667758+0000", 2024-09-14T06:32:34.114 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.3599999999999999e-06 2024-09-14T06:32:34.114 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:34.114 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:34.114 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:32:34.114 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:34.669375+0000", 2024-09-14T06:32:34.114 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.001616644 2024-09-14T06:32:34.114 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:34.114 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:34.115 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:32:34.115 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:34.669404+0000", 2024-09-14T06:32:34.115 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.9300999999999999e-05 2024-09-14T06:32:34.115 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:34.115 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:34.115 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:32:34.115 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:34.669428+0000", 2024-09-14T06:32:34.115 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.3788e-05 2024-09-14T06:32:34.115 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:34.115 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:34.115 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:32:34.115 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:34.669543+0000", 2024-09-14T06:32:34.115 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000115462 2024-09-14T06:32:34.115 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:34.115 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:34.116 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:32:34.116 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:34.669559+0000", 2024-09-14T06:32:34.116 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.5167999999999999e-05 2024-09-14T06:32:34.116 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:34.116 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:34.116 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:32:34.116 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:34.670507+0000", 2024-09-14T06:32:34.116 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00094865900000000003 2024-09-14T06:32:34.116 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:34.116 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:32:34.116 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:34.116 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:34.116 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:34.116 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "MOSDPGPush(3.5 170/168 [PushOp(3:a162138c:::rbd_data.112d1bb7b269.000000000000003c:head, version: 122'582, data_included: [0~524288], data_size: 524288, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(3:a162138c:::rbd_data.112d1bb7b269.000000000000003c:head@122'582, size: 524288, copy_subset: [(0, 524288)], clone_subset: {}, snapset: 0=[]:{}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 524288, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2024-09-14T06:32:34.117 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:31:34.670110+0000", 2024-09-14T06:32:34.117 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 59.332822303999997, 2024-09-14T06:32:34.117 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0030066229999999999, 2024-09-14T06:32:34.117 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:32:34.117 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:32:34.117 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:32:34.117 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:32:34.117 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:34.117 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:32:34.117 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:34.670110+0000", 2024-09-14T06:32:34.117 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:34.117 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:34.117 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:34.117 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:32:34.117 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:34.670110+0000", 2024-09-14T06:32:34.117 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:34.118 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:34.118 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:34.118 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:32:34.118 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:34.670111+0000", 2024-09-14T06:32:34.118 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.6499999999999998e-07 2024-09-14T06:32:34.118 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:34.118 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:34.118 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:32:34.118 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:34.671507+0000", 2024-09-14T06:32:34.118 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.001396284 2024-09-14T06:32:34.118 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:34.118 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:34.118 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:32:34.118 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:34.671530+0000", 2024-09-14T06:32:34.119 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.3023e-05 2024-09-14T06:32:34.119 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:34.119 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:34.119 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:32:34.119 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:34.671548+0000", 2024-09-14T06:32:34.119 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.8116000000000001e-05 2024-09-14T06:32:34.119 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:34.119 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:34.119 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:32:34.119 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:34.671775+0000", 2024-09-14T06:32:34.119 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00022699600000000001 2024-09-14T06:32:34.119 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:34.119 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:34.120 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:32:34.120 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:34.671790+0000", 2024-09-14T06:32:34.120 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.539e-05 2024-09-14T06:32:34.120 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:34.120 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:34.121 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:32:34.121 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:34.673116+0000", 2024-09-14T06:32:34.121 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0013259490000000001 2024-09-14T06:32:34.121 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:34.121 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:32:34.121 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:34.121 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:34.121 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:34.121 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4397.0:232694 3.9 e207/23 3:95763440:::rbd_data.112d838b9898.0000000000000006:head v 207'513, mlcod=203'511)", 2024-09-14T06:32:34.121 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:32:12.742327+0000", 2024-09-14T06:32:34.121 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 21.260605343999998, 2024-09-14T06:32:34.121 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.002504952, 2024-09-14T06:32:34.121 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:32:34.121 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:32:34.122 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:32:34.122 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:32:34.122 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:34.122 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:32:34.122 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:12.742327+0000", 2024-09-14T06:32:34.122 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:34.122 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:34.122 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:34.122 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:32:34.122 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:12.742327+0000", 2024-09-14T06:32:34.122 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:34.122 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:34.122 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:34.122 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:32:34.122 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:12.742327+0000", 2024-09-14T06:32:34.123 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.3999999999999997e-07 2024-09-14T06:32:34.123 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:34.123 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:34.123 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:32:34.123 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:12.742332+0000", 2024-09-14T06:32:34.123 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.3569999999999997e-06 2024-09-14T06:32:34.123 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:34.123 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:34.123 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:32:34.123 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:12.742360+0000", 2024-09-14T06:32:34.123 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.7650000000000001e-05 2024-09-14T06:32:34.123 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:34.123 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:34.123 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:32:34.123 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:12.742370+0000", 2024-09-14T06:32:34.124 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.0115000000000001e-05 2024-09-14T06:32:34.124 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:34.124 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:34.124 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:32:34.124 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:12.742433+0000", 2024-09-14T06:32:34.124 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.2364000000000001e-05 2024-09-14T06:32:34.124 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:34.124 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:34.124 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:32:34.124 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:12.742451+0000", 2024-09-14T06:32:34.124 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.8250999999999999e-05 2024-09-14T06:32:34.124 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:34.124 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:34.124 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-14T06:32:34.124 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:12.744776+0000", 2024-09-14T06:32:34.125 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0023253839999999998 2024-09-14T06:32:34.125 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:34.125 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:34.125 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:32:34.125 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:12.744832+0000", 2024-09-14T06:32:34.125 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.5491000000000001e-05 2024-09-14T06:32:34.125 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:34.125 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:32:34.125 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:34.125 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:34.125 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:32:34.125 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-14T06:32:34.127 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-14T06:32:34.127 INFO:tasks.ceph.osd.5.smithi138.stderr:2024-09-14T06:32:34.125+0000 7fe13746f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33618) UID: 0 2024-09-14T06:32:34.227 INFO:tasks.ceph.osd.4.smithi138.stderr:2024-09-14T06:32:34.226+0000 7f6c653c8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33615) UID: 0 2024-09-14T06:32:34.270 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-14T06:32:34.271 INFO:teuthology.orchestra.run.smithi094.stdout: "ops": [], 2024-09-14T06:32:34.271 INFO:teuthology.orchestra.run.smithi094.stdout: "num_ops": 0 2024-09-14T06:32:34.271 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-14T06:32:34.279 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-14T06:32:34.328 INFO:tasks.ceph.osd.6.smithi138.stderr:2024-09-14T06:32:34.327+0000 7f8cd8849640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33620) UID: 0 2024-09-14T06:32:34.381 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-14T06:32:34.381 INFO:teuthology.orchestra.run.smithi094.stdout: "ops": [], 2024-09-14T06:32:34.381 INFO:teuthology.orchestra.run.smithi094.stdout: "complaint_time": 30, 2024-09-14T06:32:34.382 INFO:teuthology.orchestra.run.smithi094.stdout: "num_blocked_ops": 0 2024-09-14T06:32:34.382 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-14T06:32:34.389 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-14T06:32:34.433 INFO:tasks.ceph.osd.0.smithi094.stderr:2024-09-14T06:32:34.431+0000 7f07c9333640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33636) UID: 0 2024-09-14T06:32:34.494 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-14T06:32:34.494 INFO:teuthology.orchestra.run.smithi094.stdout: "size": 20, 2024-09-14T06:32:34.494 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 600, 2024-09-14T06:32:34.494 INFO:teuthology.orchestra.run.smithi094.stdout: "ops": [ 2024-09-14T06:32:34.495 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:34.495 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:44426 3.2 3:4603ad7f:::rbd_data.112dca2f6ae1.0000000000000005:17 [read 892928~4096 [fadvise_nocache]] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:32:34.495 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:43.854977+0000", 2024-09-14T06:32:34.495 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 170.636384284, 2024-09-14T06:32:34.495 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.063369237999999994, 2024-09-14T06:32:34.495 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:32:34.495 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:32:34.495 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:32:34.495 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:32:34.495 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:32:34.495 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:34.495 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 44426 2024-09-14T06:32:34.495 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:34.495 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:32:34.495 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:34.496 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:32:34.496 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.854977+0000", 2024-09-14T06:32:34.496 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:34.496 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:34.496 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:34.496 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:32:34.496 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.854977+0000", 2024-09-14T06:32:34.496 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:34.496 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:34.496 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:34.496 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:32:34.496 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.854977+0000", 2024-09-14T06:32:34.496 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.1199999999999999e-07 2024-09-14T06:32:34.496 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:34.497 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:34.497 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:32:34.497 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.854980+0000", 2024-09-14T06:32:34.497 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.27e-06 2024-09-14T06:32:34.497 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:34.497 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:34.497 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:32:34.497 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.854983+0000", 2024-09-14T06:32:34.497 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.1650000000000002e-06 2024-09-14T06:32:34.497 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:34.497 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:34.497 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:32:34.497 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.854989+0000", 2024-09-14T06:32:34.497 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.6520000000000003e-06 2024-09-14T06:32:34.497 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:34.498 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:34.498 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:32:34.498 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.918143+0000", 2024-09-14T06:32:34.498 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.063154541999999994 2024-09-14T06:32:34.498 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:34.498 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:34.498 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:32:34.498 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.918209+0000", 2024-09-14T06:32:34.498 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.5529000000000002e-05 2024-09-14T06:32:34.498 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:34.498 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:34.498 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:32:34.498 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.918346+0000", 2024-09-14T06:32:34.498 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000136868 2024-09-14T06:32:34.499 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:34.499 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:32:34.499 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:34.499 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:34.499 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:34.499 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:44427 3.2 3:4603ad7f:::rbd_data.112dca2f6ae1.0000000000000005:17 [read 917504~4096 [fadvise_nocache]] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:32:34.499 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:43.854992+0000", 2024-09-14T06:32:34.499 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 170.63636863799999, 2024-09-14T06:32:34.499 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.063607792999999996, 2024-09-14T06:32:34.499 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:32:34.499 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:32:34.500 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:32:34.500 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:32:34.500 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:32:34.500 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:34.500 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 44427 2024-09-14T06:32:34.500 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:34.500 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:32:34.500 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:34.500 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:32:34.500 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.854992+0000", 2024-09-14T06:32:34.500 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:34.500 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:34.501 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:34.501 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:32:34.501 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.854992+0000", 2024-09-14T06:32:34.501 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:34.501 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:34.501 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:34.501 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:32:34.501 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.854993+0000", 2024-09-14T06:32:34.501 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.1400000000000001e-07 2024-09-14T06:32:34.501 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:34.501 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:34.501 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:32:34.501 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.854994+0000", 2024-09-14T06:32:34.501 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.091e-06 2024-09-14T06:32:34.502 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:34.502 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:34.502 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:32:34.502 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855001+0000", 2024-09-14T06:32:34.502 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.204e-06 2024-09-14T06:32:34.502 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:34.502 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:34.502 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:32:34.502 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855003+0000", 2024-09-14T06:32:34.502 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.514e-06 2024-09-14T06:32:34.502 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:34.502 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:34.502 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:32:34.502 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.918380+0000", 2024-09-14T06:32:34.503 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.063376738000000002 2024-09-14T06:32:34.503 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:34.503 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:34.503 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:32:34.503 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.918453+0000", 2024-09-14T06:32:34.503 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.3291e-05 2024-09-14T06:32:34.503 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:34.503 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:34.503 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:32:34.503 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.918600+0000", 2024-09-14T06:32:34.503 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00014674100000000001 2024-09-14T06:32:34.503 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:34.503 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:32:34.503 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:34.504 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:34.504 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:34.504 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:44428 3.2 3:4603ad7f:::rbd_data.112dca2f6ae1.0000000000000005:17 [read 942080~4096 [fadvise_nocache]] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:32:34.504 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:43.855006+0000", 2024-09-14T06:32:34.504 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 170.63635458799999, 2024-09-14T06:32:34.504 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.063846321999999997, 2024-09-14T06:32:34.504 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:32:34.504 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:32:34.504 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:32:34.504 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:32:34.504 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:32:34.504 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:34.504 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 44428 2024-09-14T06:32:34.504 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:34.504 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:32:34.504 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:34.505 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:32:34.505 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855006+0000", 2024-09-14T06:32:34.505 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:34.505 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:34.505 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:34.505 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:32:34.505 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855006+0000", 2024-09-14T06:32:34.505 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:34.506 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:34.506 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:34.506 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:32:34.506 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855007+0000", 2024-09-14T06:32:34.506 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.1199999999999999e-07 2024-09-14T06:32:34.506 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:34.506 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:34.506 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:32:34.506 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855011+0000", 2024-09-14T06:32:34.506 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.9539999999999998e-06 2024-09-14T06:32:34.506 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:34.506 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:34.506 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:32:34.506 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855014+0000", 2024-09-14T06:32:34.507 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.2930000000000001e-06 2024-09-14T06:32:34.507 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:34.507 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:34.507 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:32:34.507 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855016+0000", 2024-09-14T06:32:34.507 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.3039999999999999e-06 2024-09-14T06:32:34.507 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:34.507 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:34.507 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:32:34.507 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.918634+0000", 2024-09-14T06:32:34.507 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.063617928000000004 2024-09-14T06:32:34.507 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:34.507 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:34.508 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:32:34.508 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.918711+0000", 2024-09-14T06:32:34.508 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.6545000000000007e-05 2024-09-14T06:32:34.508 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:34.508 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:34.508 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:32:34.508 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.918853+0000", 2024-09-14T06:32:34.508 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00014208600000000001 2024-09-14T06:32:34.508 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:34.508 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:32:34.508 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:34.508 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:34.508 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:34.508 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:44429 3.2 3:4603ad7f:::rbd_data.112dca2f6ae1.0000000000000005:17 [read 966656~4096 [fadvise_nocache]] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:32:34.509 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:43.855019+0000", 2024-09-14T06:32:34.509 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 170.636341915, 2024-09-14T06:32:34.509 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.064059226999999996, 2024-09-14T06:32:34.509 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:32:34.509 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:32:34.509 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:32:34.509 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:32:34.509 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:32:34.509 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:34.509 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 44429 2024-09-14T06:32:34.509 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:34.509 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:32:34.509 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:34.510 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:32:34.510 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855019+0000", 2024-09-14T06:32:34.510 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:34.510 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:34.510 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:34.510 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:32:34.510 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855019+0000", 2024-09-14T06:32:34.510 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:34.510 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:34.510 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:34.510 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:32:34.510 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855019+0000", 2024-09-14T06:32:34.510 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.28e-07 2024-09-14T06:32:34.510 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:34.511 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:34.511 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:32:34.511 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855020+0000", 2024-09-14T06:32:34.511 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.0330000000000001e-06 2024-09-14T06:32:34.511 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:34.511 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:34.511 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:32:34.511 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855023+0000", 2024-09-14T06:32:34.511 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.79e-06 2024-09-14T06:32:34.511 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:34.511 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:34.511 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:32:34.511 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855025+0000", 2024-09-14T06:32:34.511 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.216e-06 2024-09-14T06:32:34.511 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:34.511 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:34.512 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:32:34.512 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.918886+0000", 2024-09-14T06:32:34.512 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.063861037999999995 2024-09-14T06:32:34.512 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:34.512 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:34.512 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:32:34.512 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.918952+0000", 2024-09-14T06:32:34.512 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.5993000000000003e-05 2024-09-14T06:32:34.512 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:34.512 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:34.512 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:32:34.512 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.919078+0000", 2024-09-14T06:32:34.512 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00012592899999999999 2024-09-14T06:32:34.512 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:34.513 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:32:34.513 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:34.513 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:34.513 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:34.513 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:44430 3.2 3:4603ad7f:::rbd_data.112dca2f6ae1.0000000000000005:17 [read 991232~4096 [fadvise_nocache]] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:32:34.513 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:43.855028+0000", 2024-09-14T06:32:34.513 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 170.63633290499999, 2024-09-14T06:32:34.513 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.064284346000000006, 2024-09-14T06:32:34.513 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:32:34.513 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:32:34.513 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:32:34.513 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:32:34.513 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:32:34.513 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:34.514 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 44430 2024-09-14T06:32:34.514 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:34.514 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:32:34.514 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:34.514 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:32:34.514 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855028+0000", 2024-09-14T06:32:34.514 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:34.514 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:34.514 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:34.514 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:32:34.514 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855028+0000", 2024-09-14T06:32:34.514 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:34.514 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:34.514 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:34.514 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:32:34.515 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855028+0000", 2024-09-14T06:32:34.515 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.1199999999999999e-07 2024-09-14T06:32:34.515 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:34.515 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:34.515 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:32:34.515 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855029+0000", 2024-09-14T06:32:34.515 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.1100000000000004e-07 2024-09-14T06:32:34.515 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:34.515 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:34.515 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:32:34.515 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855032+0000", 2024-09-14T06:32:34.515 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.7889999999999999e-06 2024-09-14T06:32:34.515 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:34.515 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:34.515 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:32:34.516 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855034+0000", 2024-09-14T06:32:34.516 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.125e-06 2024-09-14T06:32:34.516 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:34.516 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:34.516 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:32:34.516 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.919110+0000", 2024-09-14T06:32:34.516 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.064076413999999998 2024-09-14T06:32:34.516 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:34.516 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:34.516 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:32:34.516 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.919186+0000", 2024-09-14T06:32:34.516 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.5784999999999994e-05 2024-09-14T06:32:34.516 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:34.517 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:34.517 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:32:34.517 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.919312+0000", 2024-09-14T06:32:34.517 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00012611 2024-09-14T06:32:34.517 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:34.517 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:32:34.517 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:34.517 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:34.517 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:34.517 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:44431 3.2 3:4603ad7f:::rbd_data.112dca2f6ae1.0000000000000005:17 [read 1015808~4096 [fadvise_nocache]] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:32:34.517 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:43.855037+0000", 2024-09-14T06:32:34.517 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 170.63632403700001, 2024-09-14T06:32:34.517 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.064469755000000004, 2024-09-14T06:32:34.517 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:32:34.518 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:32:34.518 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:32:34.518 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:32:34.518 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:32:34.518 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:34.518 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 44431 2024-09-14T06:32:34.518 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:34.518 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:32:34.518 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:34.518 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:32:34.518 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855037+0000", 2024-09-14T06:32:34.518 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:34.518 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:34.518 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:34.519 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:32:34.519 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855037+0000", 2024-09-14T06:32:34.519 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:34.519 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:34.519 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:34.519 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:32:34.519 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855037+0000", 2024-09-14T06:32:34.519 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.1e-07 2024-09-14T06:32:34.519 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:34.519 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:34.519 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:32:34.519 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855038+0000", 2024-09-14T06:32:34.519 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.9400000000000004e-07 2024-09-14T06:32:34.519 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:34.519 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:34.520 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:32:34.520 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855041+0000", 2024-09-14T06:32:34.520 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.9639999999999999e-06 2024-09-14T06:32:34.520 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:34.520 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:34.520 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:32:34.520 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855043+0000", 2024-09-14T06:32:34.520 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.2259999999999999e-06 2024-09-14T06:32:34.520 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:34.520 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:34.521 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:32:34.521 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.919347+0000", 2024-09-14T06:32:34.521 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.064304091999999993 2024-09-14T06:32:34.521 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:34.521 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:34.521 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:32:34.521 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.919393+0000", 2024-09-14T06:32:34.521 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.5340999999999998e-05 2024-09-14T06:32:34.521 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:34.522 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:34.522 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:32:34.522 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.919507+0000", 2024-09-14T06:32:34.522 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000114028 2024-09-14T06:32:34.522 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:34.522 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:32:34.522 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:34.522 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:34.522 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:34.522 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:44432 3.2 3:4603ad7f:::rbd_data.112dca2f6ae1.0000000000000005:17 [read 1040384~4096 [fadvise_nocache]] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:32:34.522 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:43.855046+0000", 2024-09-14T06:32:34.522 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 170.63631496900001, 2024-09-14T06:32:34.522 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.064639253999999993, 2024-09-14T06:32:34.522 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:32:34.523 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:32:34.523 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:32:34.523 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:32:34.523 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:32:34.523 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:34.523 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 44432 2024-09-14T06:32:34.523 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:34.523 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:32:34.523 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:34.523 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:32:34.523 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855046+0000", 2024-09-14T06:32:34.523 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:34.523 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:34.524 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:34.524 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:32:34.524 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855046+0000", 2024-09-14T06:32:34.524 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:34.524 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:34.524 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:34.524 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:32:34.524 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855046+0000", 2024-09-14T06:32:34.524 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.05e-07 2024-09-14T06:32:34.524 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:34.524 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:34.524 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:32:34.524 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855047+0000", 2024-09-14T06:32:34.525 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.879999999999999e-07 2024-09-14T06:32:34.525 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:34.525 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:34.525 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:32:34.525 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855050+0000", 2024-09-14T06:32:34.525 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.8459999999999999e-06 2024-09-14T06:32:34.525 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:34.525 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:34.525 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:32:34.525 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855052+0000", 2024-09-14T06:32:34.525 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.136e-06 2024-09-14T06:32:34.525 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:34.525 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:34.525 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:32:34.526 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.919536+0000", 2024-09-14T06:32:34.526 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.064484132 2024-09-14T06:32:34.526 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:34.526 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:34.526 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:32:34.526 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.919611+0000", 2024-09-14T06:32:34.526 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.4407999999999996e-05 2024-09-14T06:32:34.526 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:34.526 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:34.526 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:32:34.526 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.919685+0000", 2024-09-14T06:32:34.526 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.4538999999999999e-05 2024-09-14T06:32:34.526 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:34.526 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:32:34.526 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:34.526 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:34.527 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:34.527 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:44518 3.2 3:4190d882:::rbd_data.112dca2f6ae1.0000000000000002:17 [read 12288~4096 [fadvise_nocache]] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:32:34.527 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:43.855055+0000", 2024-09-14T06:32:34.527 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 170.636306085, 2024-09-14T06:32:34.527 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.064894317000000007, 2024-09-14T06:32:34.527 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:32:34.527 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:32:34.527 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:32:34.527 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:32:34.527 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:32:34.527 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:34.527 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 44518 2024-09-14T06:32:34.527 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:34.527 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:32:34.528 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:34.528 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:32:34.528 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855055+0000", 2024-09-14T06:32:34.528 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:34.528 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:34.528 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:34.528 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:32:34.528 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855055+0000", 2024-09-14T06:32:34.528 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:34.528 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:34.528 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:34.528 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:32:34.528 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855055+0000", 2024-09-14T06:32:34.528 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.0699999999999999e-07 2024-09-14T06:32:34.529 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:34.529 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:34.529 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:32:34.529 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855056+0000", 2024-09-14T06:32:34.529 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.9400000000000004e-07 2024-09-14T06:32:34.529 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:34.529 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:34.529 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:32:34.529 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855059+0000", 2024-09-14T06:32:34.530 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.8870000000000001e-06 2024-09-14T06:32:34.530 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:34.530 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:34.530 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:32:34.530 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855061+0000", 2024-09-14T06:32:34.530 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.1919999999999999e-06 2024-09-14T06:32:34.530 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:34.530 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:34.530 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:32:34.530 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.919719+0000", 2024-09-14T06:32:34.530 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.064658013 2024-09-14T06:32:34.530 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:34.530 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:34.530 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:32:34.530 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.919788+0000", 2024-09-14T06:32:34.531 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.9282000000000005e-05 2024-09-14T06:32:34.531 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:34.531 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:34.531 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:32:34.531 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.919949+0000", 2024-09-14T06:32:34.531 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00016084199999999999 2024-09-14T06:32:34.531 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:34.531 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:32:34.531 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:34.531 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:34.531 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:34.531 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:44519 3.2 3:4190d882:::rbd_data.112dca2f6ae1.0000000000000002:17 [read 36864~4096 [fadvise_nocache]] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:32:34.531 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:43.855064+0000", 2024-09-14T06:32:34.531 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 170.63629710800001, 2024-09-14T06:32:34.531 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.065081865000000003, 2024-09-14T06:32:34.532 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:32:34.532 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:32:34.532 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:32:34.532 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:32:34.532 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:32:34.532 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:34.532 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 44519 2024-09-14T06:32:34.532 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:34.532 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:32:34.532 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:34.532 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:32:34.532 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855064+0000", 2024-09-14T06:32:34.532 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:34.532 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:34.533 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:34.533 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:32:34.533 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855064+0000", 2024-09-14T06:32:34.533 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:34.533 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:34.533 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:34.533 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:32:34.533 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855064+0000", 2024-09-14T06:32:34.533 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.1e-07 2024-09-14T06:32:34.534 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:34.534 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:34.534 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:32:34.534 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855065+0000", 2024-09-14T06:32:34.534 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.035e-06 2024-09-14T06:32:34.534 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:34.534 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:34.534 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:32:34.534 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855068+0000", 2024-09-14T06:32:34.534 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.8339999999999999e-06 2024-09-14T06:32:34.534 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:34.534 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:34.534 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:32:34.534 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855071+0000", 2024-09-14T06:32:34.534 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.3450000000000002e-06 2024-09-14T06:32:34.535 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:34.535 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:34.535 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:32:34.535 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.919979+0000", 2024-09-14T06:32:34.535 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.064907647999999998 2024-09-14T06:32:34.535 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:34.535 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:34.535 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:32:34.535 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.920026+0000", 2024-09-14T06:32:34.535 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.7595000000000001e-05 2024-09-14T06:32:34.535 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:34.535 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:34.535 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:32:34.535 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.920146+0000", 2024-09-14T06:32:34.536 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00011919800000000001 2024-09-14T06:32:34.536 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:34.536 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:32:34.536 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:34.536 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:34.536 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:34.536 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:44520 3.2 3:4190d882:::rbd_data.112dca2f6ae1.0000000000000002:17 [read 61440~4096 [fadvise_nocache]] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:32:34.536 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:43.855075+0000", 2024-09-14T06:32:34.536 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 170.63628605700001, 2024-09-14T06:32:34.536 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.065335078000000005, 2024-09-14T06:32:34.536 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:32:34.536 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:32:34.537 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:32:34.537 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:32:34.537 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:32:34.537 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:34.537 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 44520 2024-09-14T06:32:34.537 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:34.537 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:32:34.537 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:34.537 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:32:34.537 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855075+0000", 2024-09-14T06:32:34.537 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:34.537 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:34.537 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:34.538 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:32:34.538 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855075+0000", 2024-09-14T06:32:34.538 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:34.538 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:34.538 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:34.538 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:32:34.538 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855075+0000", 2024-09-14T06:32:34.538 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.1e-07 2024-09-14T06:32:34.538 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:34.538 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:34.538 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:32:34.538 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855076+0000", 2024-09-14T06:32:34.538 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.0839999999999999e-06 2024-09-14T06:32:34.539 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:34.539 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:34.539 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:32:34.539 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855079+0000", 2024-09-14T06:32:34.539 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.9359999999999999e-06 2024-09-14T06:32:34.539 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:34.539 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:34.539 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:32:34.539 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855084+0000", 2024-09-14T06:32:34.539 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.1730000000000003e-06 2024-09-14T06:32:34.539 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:34.539 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:34.539 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:32:34.539 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.920175+0000", 2024-09-14T06:32:34.540 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.065090234999999996 2024-09-14T06:32:34.540 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:34.540 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:34.540 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:32:34.540 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.920250+0000", 2024-09-14T06:32:34.540 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.5807000000000001e-05 2024-09-14T06:32:34.540 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:34.540 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:34.540 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:32:34.540 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.920410+0000", 2024-09-14T06:32:34.540 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00015963300000000001 2024-09-14T06:32:34.540 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:34.541 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:32:34.541 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:34.541 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:34.541 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:34.541 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:44521 3.2 3:4190d882:::rbd_data.112dca2f6ae1.0000000000000002:17 [read 86016~4096 [fadvise_nocache]] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:32:34.541 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:43.855087+0000", 2024-09-14T06:32:34.541 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 170.63627357600001, 2024-09-14T06:32:34.541 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.065563264999999996, 2024-09-14T06:32:34.541 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:32:34.541 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:32:34.541 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:32:34.541 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:32:34.541 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:32:34.542 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:34.542 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 44521 2024-09-14T06:32:34.542 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:34.542 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:32:34.542 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:34.542 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:32:34.542 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855087+0000", 2024-09-14T06:32:34.542 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:34.542 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:34.542 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:34.542 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:32:34.542 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855087+0000", 2024-09-14T06:32:34.542 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:34.542 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:34.543 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:34.543 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:32:34.543 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855088+0000", 2024-09-14T06:32:34.543 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.3699999999999999e-07 2024-09-14T06:32:34.543 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:34.543 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:34.543 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:32:34.543 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855089+0000", 2024-09-14T06:32:34.543 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.9900000000000009e-07 2024-09-14T06:32:34.543 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:34.543 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:34.543 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:32:34.544 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855091+0000", 2024-09-14T06:32:34.544 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.8940000000000002e-06 2024-09-14T06:32:34.544 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:34.544 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:34.544 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:32:34.544 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855094+0000", 2024-09-14T06:32:34.544 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.351e-06 2024-09-14T06:32:34.544 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:34.544 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:34.544 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:32:34.544 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.920437+0000", 2024-09-14T06:32:34.544 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.065343221000000007 2024-09-14T06:32:34.545 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:34.545 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:34.545 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:32:34.545 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.920481+0000", 2024-09-14T06:32:34.545 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.4431000000000002e-05 2024-09-14T06:32:34.545 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:34.545 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:34.545 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:32:34.545 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.920651+0000", 2024-09-14T06:32:34.545 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00016913200000000001 2024-09-14T06:32:34.545 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:34.545 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:32:34.545 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:34.545 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:34.545 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:34.546 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:44522 3.2 3:4190d882:::rbd_data.112dca2f6ae1.0000000000000002:17 [read 110592~4096 [fadvise_nocache]] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:32:34.546 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:43.855098+0000", 2024-09-14T06:32:34.546 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 170.63626334899999, 2024-09-14T06:32:34.546 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.065827628999999999, 2024-09-14T06:32:34.546 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:32:34.546 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:32:34.546 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:32:34.546 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:32:34.546 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:32:34.546 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:34.546 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 44522 2024-09-14T06:32:34.546 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:34.546 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:32:34.547 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:34.547 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:32:34.547 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855098+0000", 2024-09-14T06:32:34.547 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:34.547 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:34.547 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:34.547 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:32:34.547 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855098+0000", 2024-09-14T06:32:34.547 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:34.547 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:34.548 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:34.548 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:32:34.548 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855098+0000", 2024-09-14T06:32:34.548 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.1e-07 2024-09-14T06:32:34.548 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:34.548 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:34.548 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:32:34.548 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855099+0000", 2024-09-14T06:32:34.548 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.012e-06 2024-09-14T06:32:34.548 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:34.548 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:34.548 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:32:34.548 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855102+0000", 2024-09-14T06:32:34.548 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.9380000000000001e-06 2024-09-14T06:32:34.548 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:34.549 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:34.549 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:32:34.549 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855108+0000", 2024-09-14T06:32:34.549 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.4339999999999997e-06 2024-09-14T06:32:34.549 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:34.549 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:34.549 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:32:34.549 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.920681+0000", 2024-09-14T06:32:34.549 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.065572943999999994 2024-09-14T06:32:34.549 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:34.549 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:34.549 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:32:34.549 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.920724+0000", 2024-09-14T06:32:34.549 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.2877000000000003e-05 2024-09-14T06:32:34.549 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:34.549 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:34.550 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:32:34.550 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.920925+0000", 2024-09-14T06:32:34.550 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00020121399999999999 2024-09-14T06:32:34.550 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:34.550 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:32:34.550 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:34.550 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:34.550 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:34.550 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:44523 3.2 3:4190d882:::rbd_data.112dca2f6ae1.0000000000000002:17 [read 135168~4096 [fadvise_nocache]] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:32:34.550 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:43.855111+0000", 2024-09-14T06:32:34.550 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 170.63624962700001, 2024-09-14T06:32:34.550 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.066028198999999996, 2024-09-14T06:32:34.550 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:32:34.551 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:32:34.551 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:32:34.551 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:32:34.551 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:32:34.551 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:34.551 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 44523 2024-09-14T06:32:34.551 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:34.551 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:32:34.551 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:34.551 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:32:34.551 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855111+0000", 2024-09-14T06:32:34.551 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:34.551 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:34.551 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:34.552 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:32:34.552 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855111+0000", 2024-09-14T06:32:34.552 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:34.552 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:34.552 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:34.552 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:32:34.552 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855112+0000", 2024-09-14T06:32:34.552 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.1199999999999999e-07 2024-09-14T06:32:34.552 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:34.552 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:34.552 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:32:34.552 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855115+0000", 2024-09-14T06:32:34.552 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.3380000000000001e-06 2024-09-14T06:32:34.552 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:34.553 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:34.553 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:32:34.553 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855118+0000", 2024-09-14T06:32:34.553 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.366e-06 2024-09-14T06:32:34.553 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:34.553 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:34.553 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:32:34.553 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855121+0000", 2024-09-14T06:32:34.553 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.8329999999999998e-06 2024-09-14T06:32:34.553 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:34.553 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:34.553 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:32:34.553 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.920956+0000", 2024-09-14T06:32:34.553 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.065835251999999997 2024-09-14T06:32:34.554 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:34.554 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:34.554 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:32:34.554 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.921001+0000", 2024-09-14T06:32:34.554 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.4778999999999999e-05 2024-09-14T06:32:34.554 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:34.554 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:34.554 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:32:34.554 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.921139+0000", 2024-09-14T06:32:34.554 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000138419 2024-09-14T06:32:34.554 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:34.554 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:32:34.554 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:34.554 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:34.554 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:34.555 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:44524 3.2 3:4190d882:::rbd_data.112dca2f6ae1.0000000000000002:17 [read 159744~4096 [fadvise_nocache]] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:32:34.555 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:43.855124+0000", 2024-09-14T06:32:34.555 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 170.63623664799999, 2024-09-14T06:32:34.555 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.066249261000000004, 2024-09-14T06:32:34.555 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:32:34.555 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:32:34.555 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:32:34.555 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:32:34.555 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:32:34.555 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:34.555 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 44524 2024-09-14T06:32:34.556 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:34.556 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:32:34.556 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:34.556 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:32:34.556 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855124+0000", 2024-09-14T06:32:34.556 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:34.556 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:34.556 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:34.556 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:32:34.556 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855124+0000", 2024-09-14T06:32:34.556 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:34.556 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:34.556 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:34.557 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:32:34.557 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861086+0000", 2024-09-14T06:32:34.557 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0059613829999999998 2024-09-14T06:32:34.557 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:34.557 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:34.557 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:32:34.557 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861100+0000", 2024-09-14T06:32:34.557 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.4644000000000001e-05 2024-09-14T06:32:34.557 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:34.557 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:34.557 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:32:34.557 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861109+0000", 2024-09-14T06:32:34.557 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.541e-06 2024-09-14T06:32:34.557 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:34.558 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:34.558 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:32:34.558 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861117+0000", 2024-09-14T06:32:34.558 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.1049999999999995e-06 2024-09-14T06:32:34.558 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:34.558 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:34.558 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:32:34.558 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.921169+0000", 2024-09-14T06:32:34.558 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.060052405000000003 2024-09-14T06:32:34.558 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:34.558 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:34.558 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:32:34.558 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.921213+0000", 2024-09-14T06:32:34.558 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.3565e-05 2024-09-14T06:32:34.558 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:34.559 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:34.559 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:32:34.559 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.921374+0000", 2024-09-14T06:32:34.559 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00016061800000000001 2024-09-14T06:32:34.559 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:34.559 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:32:34.559 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:34.559 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:34.559 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:34.559 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:44537 3.2 3:4190d882:::rbd_data.112dca2f6ae1.0000000000000002:17 [read 479232~4096 [fadvise_nocache]] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:32:34.559 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:43.861405+0000", 2024-09-14T06:32:34.559 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 170.629956045, 2024-09-14T06:32:34.559 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.063351795000000002, 2024-09-14T06:32:34.560 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:32:34.560 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:32:34.560 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:32:34.560 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:32:34.560 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:32:34.560 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:34.560 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 44537 2024-09-14T06:32:34.560 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:34.560 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:32:34.560 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:34.560 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:32:34.560 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861405+0000", 2024-09-14T06:32:34.560 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:34.560 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:34.561 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:34.561 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:32:34.561 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861405+0000", 2024-09-14T06:32:34.561 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:34.561 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:34.561 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:34.561 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:32:34.561 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861405+0000", 2024-09-14T06:32:34.561 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.9099999999999999e-07 2024-09-14T06:32:34.561 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:34.561 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:34.561 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:32:34.561 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861407+0000", 2024-09-14T06:32:34.561 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.075e-06 2024-09-14T06:32:34.562 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:34.562 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:34.562 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:32:34.562 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861414+0000", 2024-09-14T06:32:34.562 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.4659999999999996e-06 2024-09-14T06:32:34.562 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:34.562 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:34.562 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:32:34.562 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861418+0000", 2024-09-14T06:32:34.562 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.5519999999999998e-06 2024-09-14T06:32:34.562 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:34.562 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:34.562 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:32:34.562 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.924474+0000", 2024-09-14T06:32:34.562 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.063056012999999994 2024-09-14T06:32:34.563 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:34.563 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:34.563 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:32:34.563 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.924556+0000", 2024-09-14T06:32:34.563 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.1181999999999996e-05 2024-09-14T06:32:34.563 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:34.563 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:34.563 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:32:34.563 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.924757+0000", 2024-09-14T06:32:34.563 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00020111599999999999 2024-09-14T06:32:34.563 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:34.563 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:32:34.563 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:34.563 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:34.563 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:34.563 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:44538 3.2 3:4190d882:::rbd_data.112dca2f6ae1.0000000000000002:17 [read 503808~4096 [fadvise_nocache]] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:32:34.564 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:43.861425+0000", 2024-09-14T06:32:34.564 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 170.629936004, 2024-09-14T06:32:34.564 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.063627125000000007, 2024-09-14T06:32:34.564 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:32:34.564 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:32:34.564 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:32:34.564 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:32:34.564 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:32:34.564 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:34.564 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 44538 2024-09-14T06:32:34.564 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:34.564 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:32:34.564 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:34.565 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:32:34.565 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861425+0000", 2024-09-14T06:32:34.565 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:34.565 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:34.565 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:34.565 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:32:34.565 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861425+0000", 2024-09-14T06:32:34.565 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:34.565 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:34.565 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:34.565 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:32:34.565 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861425+0000", 2024-09-14T06:32:34.565 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.9999999999999998e-07 2024-09-14T06:32:34.566 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:34.566 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:34.566 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:32:34.566 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861427+0000", 2024-09-14T06:32:34.566 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.9829999999999999e-06 2024-09-14T06:32:34.566 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:34.566 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:34.566 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:32:34.566 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861433+0000", 2024-09-14T06:32:34.566 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.2889999999999997e-06 2024-09-14T06:32:34.566 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:34.566 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:34.566 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:32:34.567 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861454+0000", 2024-09-14T06:32:34.567 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.1844999999999998e-05 2024-09-14T06:32:34.567 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:34.567 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:34.567 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:32:34.567 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.924786+0000", 2024-09-14T06:32:34.567 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.063331497 2024-09-14T06:32:34.567 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:34.567 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:34.567 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:32:34.567 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.924864+0000", 2024-09-14T06:32:34.567 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.7744000000000006e-05 2024-09-14T06:32:34.567 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:34.567 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:34.568 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:32:34.568 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.925052+0000", 2024-09-14T06:32:34.568 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00018836699999999999 2024-09-14T06:32:34.568 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:34.568 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:32:34.568 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:34.568 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:34.568 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:34.568 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:44539 3.2 3:4190d882:::rbd_data.112dca2f6ae1.0000000000000002:17 [read 528384~4096 [fadvise_nocache]] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:32:34.568 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:43.861462+0000", 2024-09-14T06:32:34.568 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 170.629898677, 2024-09-14T06:32:34.568 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.063855705999999998, 2024-09-14T06:32:34.568 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:32:34.569 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:32:34.569 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:32:34.569 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:32:34.569 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:32:34.569 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:34.569 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 44539 2024-09-14T06:32:34.569 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:34.569 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:32:34.569 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:34.569 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:32:34.569 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861462+0000", 2024-09-14T06:32:34.569 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:34.569 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:34.569 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:34.569 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:32:34.570 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861462+0000", 2024-09-14T06:32:34.570 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:34.570 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:34.570 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:34.570 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:32:34.570 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861463+0000", 2024-09-14T06:32:34.570 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.5200000000000002e-07 2024-09-14T06:32:34.570 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:34.570 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:34.570 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:32:34.570 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861465+0000", 2024-09-14T06:32:34.570 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.5189999999999999e-06 2024-09-14T06:32:34.570 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:34.570 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:34.571 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:32:34.571 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861471+0000", 2024-09-14T06:32:34.571 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.6899999999999997e-06 2024-09-14T06:32:34.571 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:34.571 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:34.571 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:32:34.571 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861475+0000", 2024-09-14T06:32:34.571 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.5730000000000001e-06 2024-09-14T06:32:34.571 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:34.571 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:34.571 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:32:34.571 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.925085+0000", 2024-09-14T06:32:34.571 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.063609902999999995 2024-09-14T06:32:34.571 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:34.571 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:34.572 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:32:34.572 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.925145+0000", 2024-09-14T06:32:34.572 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.9383999999999999e-05 2024-09-14T06:32:34.572 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:34.572 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:34.572 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:32:34.572 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.925318+0000", 2024-09-14T06:32:34.572 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00017318499999999999 2024-09-14T06:32:34.572 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:34.572 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:32:34.572 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:34.572 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:34.572 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:34.572 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:44540 3.2 3:4190d882:::rbd_data.112dca2f6ae1.0000000000000002:17 [read 552960~4096 [fadvise_nocache]] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:32:34.572 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:43.861483+0000", 2024-09-14T06:32:34.573 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 170.629878246, 2024-09-14T06:32:34.573 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.064065681999999999, 2024-09-14T06:32:34.573 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:32:34.573 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:32:34.573 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:32:34.573 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:32:34.573 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:32:34.573 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:34.573 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 44540 2024-09-14T06:32:34.573 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:34.573 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:32:34.573 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:34.573 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:32:34.573 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861483+0000", 2024-09-14T06:32:34.573 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:34.574 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:34.574 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:34.574 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:32:34.574 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861483+0000", 2024-09-14T06:32:34.574 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:34.574 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:34.574 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:34.574 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:32:34.574 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.867079+0000", 2024-09-14T06:32:34.574 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0055966480000000001 2024-09-14T06:32:34.574 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:34.574 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:34.574 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:32:34.574 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.867086+0000", 2024-09-14T06:32:34.575 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.387e-06 2024-09-14T06:32:34.575 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:34.575 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:34.575 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:32:34.575 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.867094+0000", 2024-09-14T06:32:34.575 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.9419999999999995e-06 2024-09-14T06:32:34.575 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:34.575 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:34.575 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:32:34.575 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.867109+0000", 2024-09-14T06:32:34.575 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.5818999999999999e-05 2024-09-14T06:32:34.575 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:34.575 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:34.576 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:32:34.576 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.925354+0000", 2024-09-14T06:32:34.576 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.058244047 2024-09-14T06:32:34.576 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:34.576 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:34.576 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:32:34.576 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.925406+0000", 2024-09-14T06:32:34.576 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.2661999999999998e-05 2024-09-14T06:32:34.576 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:34.576 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:34.576 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:32:34.576 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.925548+0000", 2024-09-14T06:32:34.577 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000142177 2024-09-14T06:32:34.577 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:34.577 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:32:34.577 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:34.577 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:34.577 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:34.577 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:225587 3.7 3:e112e2b3:::rbd_id.image_client.0-clone42:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e189)", 2024-09-14T06:32:34.577 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:31:53.874591+0000", 2024-09-14T06:32:34.577 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 40.616770029000001, 2024-09-14T06:32:34.577 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.875541507, 2024-09-14T06:32:34.577 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:32:34.577 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:32:34.577 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:32:34.577 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:32:34.578 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:32:34.578 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:34.578 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 225587 2024-09-14T06:32:34.578 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:34.578 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:32:34.578 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:34.578 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:32:34.578 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:53.874591+0000", 2024-09-14T06:32:34.578 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:34.578 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:34.578 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:34.578 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:32:34.578 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:53.874591+0000", 2024-09-14T06:32:34.578 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:34.578 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:34.579 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:34.579 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:32:34.579 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:53.874592+0000", 2024-09-14T06:32:34.579 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.5209999999999999e-06 2024-09-14T06:32:34.579 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:34.579 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:34.579 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:32:34.579 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:53.874599+0000", 2024-09-14T06:32:34.579 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.6290000000000004e-06 2024-09-14T06:32:34.579 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:34.579 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:34.579 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:32:34.580 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:53.874618+0000", 2024-09-14T06:32:34.580 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.8950999999999999e-05 2024-09-14T06:32:34.580 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:34.580 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:34.580 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:32:34.580 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:53.874626+0000", 2024-09-14T06:32:34.580 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.3960000000000006e-06 2024-09-14T06:32:34.580 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:34.580 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:34.580 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:32:34.580 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:53.874699+0000", 2024-09-14T06:32:34.580 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.2329999999999994e-05 2024-09-14T06:32:34.581 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:34.581 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:34.581 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "waiting for peered", 2024-09-14T06:32:34.581 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:53.874711+0000", 2024-09-14T06:32:34.581 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.2473999999999999e-05 2024-09-14T06:32:34.581 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:34.581 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:34.581 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:32:34.581 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:54.749578+0000", 2024-09-14T06:32:34.581 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.87486642100000001 2024-09-14T06:32:34.581 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:34.581 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:34.581 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:32:34.581 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:54.749706+0000", 2024-09-14T06:32:34.581 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00012856200000000001 2024-09-14T06:32:34.582 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:34.582 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:34.582 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:32:34.582 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:54.750132+0000", 2024-09-14T06:32:34.582 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00042622299999999999 2024-09-14T06:32:34.582 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:34.582 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:32:34.582 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:34.582 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:34.582 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:34.582 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:233119 3.1e 3:7bae5948:::rbd_data.112df8d32353.000000000000000f:head [delete] snapc 53=[] ondisk+write+known_if_redirected+supports_pool_eio e211)", 2024-09-14T06:32:34.582 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:32:16.760588+0000", 2024-09-14T06:32:34.582 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 17.730773041999999, 2024-09-14T06:32:34.582 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.92349815700000004, 2024-09-14T06:32:34.583 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:32:34.583 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:32:34.583 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:32:34.583 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:32:34.583 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:32:34.583 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:34.583 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 233119 2024-09-14T06:32:34.583 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:34.583 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:32:34.583 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:34.583 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:32:34.583 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:16.760588+0000", 2024-09-14T06:32:34.583 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:34.584 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:34.584 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:34.584 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:32:34.584 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:16.760588+0000", 2024-09-14T06:32:34.584 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:34.584 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:34.584 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:34.584 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:32:34.584 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:16.760588+0000", 2024-09-14T06:32:34.584 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.3900000000000005e-07 2024-09-14T06:32:34.584 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:34.584 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:34.584 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:32:34.585 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:16.760591+0000", 2024-09-14T06:32:34.585 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.0070000000000002e-06 2024-09-14T06:32:34.585 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:34.585 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:34.585 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:32:34.585 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:16.760598+0000", 2024-09-14T06:32:34.585 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.0509999999999999e-06 2024-09-14T06:32:34.585 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:34.585 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:34.585 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:32:34.585 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:16.760605+0000", 2024-09-14T06:32:34.585 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.0310000000000004e-06 2024-09-14T06:32:34.585 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:34.585 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:34.586 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:32:34.586 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:16.760668+0000", 2024-09-14T06:32:34.586 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.3699000000000001e-05 2024-09-14T06:32:34.586 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:34.586 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:34.586 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "waiting for peered", 2024-09-14T06:32:34.586 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:16.760678+0000", 2024-09-14T06:32:34.586 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.0122999999999999e-05 2024-09-14T06:32:34.586 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:34.586 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:34.586 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:32:34.586 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:17.681459+0000", 2024-09-14T06:32:34.586 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.92078067399999997 2024-09-14T06:32:34.586 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:34.586 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:34.587 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:32:34.587 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:17.681700+0000", 2024-09-14T06:32:34.587 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00024128800000000001 2024-09-14T06:32:34.587 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:34.587 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:34.587 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "waiting for subops from 2", 2024-09-14T06:32:34.587 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:17.681860+0000", 2024-09-14T06:32:34.587 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00015995500000000001 2024-09-14T06:32:34.587 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:34.587 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:34.587 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:32:34.587 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:17.683399+0000", 2024-09-14T06:32:34.587 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0015388209999999999 2024-09-14T06:32:34.588 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:34.588 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:34.588 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "op_commit", 2024-09-14T06:32:34.588 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:17.683687+0000", 2024-09-14T06:32:34.588 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00028772299999999998 2024-09-14T06:32:34.588 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:34.588 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:34.588 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-14T06:32:34.588 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:17.684032+0000", 2024-09-14T06:32:34.588 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000344836 2024-09-14T06:32:34.588 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:34.588 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:34.588 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:32:34.588 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:17.684086+0000", 2024-09-14T06:32:34.588 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.4410000000000003e-05 2024-09-14T06:32:34.589 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:34.589 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:32:34.589 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:34.589 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:34.589 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:32:34.589 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-14T06:32:34.591 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-14T06:32:34.591 INFO:tasks.ceph.osd.0.smithi094.stderr:2024-09-14T06:32:34.588+0000 7f07c9333640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33636) UID: 0 2024-09-14T06:32:34.691 INFO:tasks.ceph.osd.3.smithi094.stderr:2024-09-14T06:32:34.689+0000 7feeeafe8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33655) UID: 0 2024-09-14T06:32:34.706 INFO:teuthology.orchestra.run.smithi138.stdout:{ 2024-09-14T06:32:34.706 INFO:teuthology.orchestra.run.smithi138.stdout: "ops": [], 2024-09-14T06:32:34.706 INFO:teuthology.orchestra.run.smithi138.stdout: "num_ops": 0 2024-09-14T06:32:34.706 INFO:teuthology.orchestra.run.smithi138.stdout:} 2024-09-14T06:32:34.715 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-14T06:32:34.791 INFO:tasks.ceph.osd.2.smithi094.stderr:2024-09-14T06:32:34.789+0000 7faf3269b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33649) UID: 0 2024-09-14T06:32:34.828 INFO:teuthology.orchestra.run.smithi138.stdout:{ 2024-09-14T06:32:34.828 INFO:teuthology.orchestra.run.smithi138.stdout: "ops": [], 2024-09-14T06:32:34.828 INFO:teuthology.orchestra.run.smithi138.stdout: "complaint_time": 30, 2024-09-14T06:32:34.828 INFO:teuthology.orchestra.run.smithi138.stdout: "num_blocked_ops": 0 2024-09-14T06:32:34.828 INFO:teuthology.orchestra.run.smithi138.stdout:} 2024-09-14T06:32:34.838 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-14T06:32:34.892 INFO:tasks.ceph.osd.2.smithi094.stderr:2024-09-14T06:32:34.890+0000 7faf3269b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33649) UID: 0 2024-09-14T06:32:34.948 INFO:teuthology.orchestra.run.smithi138.stdout:{ 2024-09-14T06:32:34.948 INFO:teuthology.orchestra.run.smithi138.stdout: "size": 20, 2024-09-14T06:32:34.948 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 600, 2024-09-14T06:32:34.948 INFO:teuthology.orchestra.run.smithi138.stdout: "ops": [ 2024-09-14T06:32:34.949 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:34.949 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:19328 3.3 3:c34e5e51:::rbd_data.112d92027590.0000000000000024:f [read 1926144~1024 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:32:34.949 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:27.059768+0000", 2024-09-14T06:32:34.949 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 187.88653484400001, 2024-09-14T06:32:34.949 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046369147999999999, 2024-09-14T06:32:34.949 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:32:34.949 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:32:34.949 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:32:34.949 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:32:34.949 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:32:34.949 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:34.949 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 19328 2024-09-14T06:32:34.949 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:34.949 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:32:34.950 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:34.950 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:32:34.950 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.059768+0000", 2024-09-14T06:32:34.950 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:34.950 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:34.950 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:34.950 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:32:34.950 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.059768+0000", 2024-09-14T06:32:34.950 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:34.950 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:34.950 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:34.950 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:32:34.950 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.059874+0000", 2024-09-14T06:32:34.950 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00010551400000000001 2024-09-14T06:32:34.951 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:34.951 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:34.951 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:32:34.951 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.059877+0000", 2024-09-14T06:32:34.951 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.8279999999999999e-06 2024-09-14T06:32:34.951 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:34.951 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:34.951 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:32:34.951 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.059884+0000", 2024-09-14T06:32:34.951 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.6389999999999992e-06 2024-09-14T06:32:34.951 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:34.951 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:34.951 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:32:34.951 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.059890+0000", 2024-09-14T06:32:34.951 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.49e-06 2024-09-14T06:32:34.952 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:34.952 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:34.952 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:32:34.952 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.105804+0000", 2024-09-14T06:32:34.952 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.045914242000000001 2024-09-14T06:32:34.952 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:34.952 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:34.952 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:32:34.952 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.105886+0000", 2024-09-14T06:32:34.952 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.2305999999999994e-05 2024-09-14T06:32:34.952 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:34.952 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:34.952 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:32:34.952 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.106138+0000", 2024-09-14T06:32:34.952 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.000251129 2024-09-14T06:32:34.953 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:34.953 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:32:34.953 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:34.953 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:34.953 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:34.953 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:19329 3.3 3:c34e5e51:::rbd_data.112d92027590.0000000000000024:f [read 1936384~1024 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:32:34.953 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:27.059899+0000", 2024-09-14T06:32:34.953 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 187.88640432099999, 2024-09-14T06:32:34.953 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046572122000000001, 2024-09-14T06:32:34.953 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:32:34.953 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:32:34.953 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:32:34.953 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:32:34.953 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:32:34.953 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:34.954 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 19329 2024-09-14T06:32:34.954 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:34.954 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:32:34.954 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:34.954 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:32:34.954 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.059899+0000", 2024-09-14T06:32:34.954 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:34.954 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:34.954 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:34.954 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:32:34.954 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.059899+0000", 2024-09-14T06:32:34.954 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:34.954 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:34.954 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:34.954 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:32:34.954 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062501+0000", 2024-09-14T06:32:34.955 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.0026016419999999999 2024-09-14T06:32:34.955 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:34.955 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:34.955 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:32:34.955 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062514+0000", 2024-09-14T06:32:34.955 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.2947e-05 2024-09-14T06:32:34.955 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:34.955 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:34.955 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:32:34.955 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062523+0000", 2024-09-14T06:32:34.955 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.5519999999999993e-06 2024-09-14T06:32:34.955 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:34.955 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:34.955 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:32:34.955 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062534+0000", 2024-09-14T06:32:34.956 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.1416e-05 2024-09-14T06:32:34.956 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:34.956 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:34.956 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:32:34.956 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.106179+0000", 2024-09-14T06:32:34.956 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.043644049999999997 2024-09-14T06:32:34.956 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:34.956 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:34.956 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:32:34.956 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.106262+0000", 2024-09-14T06:32:34.956 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.3178999999999998e-05 2024-09-14T06:32:34.956 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:34.956 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:34.956 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:32:34.957 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.106471+0000", 2024-09-14T06:32:34.957 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00020933599999999999 2024-09-14T06:32:34.957 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:34.957 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:32:34.957 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:34.957 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:34.957 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:34.957 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:19351 3.3 3:c34e5e51:::rbd_data.112d92027590.0000000000000024:f [read 1159168~1024 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:32:34.957 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:27.062766+0000", 2024-09-14T06:32:34.957 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 187.883537096, 2024-09-14T06:32:34.957 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046099454999999998, 2024-09-14T06:32:34.957 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:32:34.957 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:32:34.957 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:32:34.958 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:32:34.958 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:32:34.958 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:34.958 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 19351 2024-09-14T06:32:34.958 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:34.958 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:32:34.958 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:34.958 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:32:34.958 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062766+0000", 2024-09-14T06:32:34.958 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:34.958 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:34.958 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:34.958 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:32:34.958 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062766+0000", 2024-09-14T06:32:34.958 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:34.958 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:34.959 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:34.959 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:32:34.959 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062766+0000", 2024-09-14T06:32:34.959 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.6e-07 2024-09-14T06:32:34.959 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:34.959 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:34.959 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:32:34.959 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062767+0000", 2024-09-14T06:32:34.959 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.0070000000000001e-06 2024-09-14T06:32:34.959 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:34.959 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:34.959 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:32:34.959 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062785+0000", 2024-09-14T06:32:34.959 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.7884999999999999e-05 2024-09-14T06:32:34.959 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:34.959 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:34.960 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:32:34.960 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062790+0000", 2024-09-14T06:32:34.960 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.0900000000000004e-06 2024-09-14T06:32:34.960 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:34.960 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:34.960 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:32:34.960 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.108735+0000", 2024-09-14T06:32:34.960 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.045944411999999997 2024-09-14T06:32:34.960 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:34.960 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:34.960 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:32:34.960 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.108809+0000", 2024-09-14T06:32:34.960 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.4157000000000002e-05 2024-09-14T06:32:34.960 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:34.960 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:34.961 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:32:34.961 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.108866+0000", 2024-09-14T06:32:34.961 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.6644000000000002e-05 2024-09-14T06:32:34.961 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:34.961 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:32:34.961 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:34.961 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:34.961 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:34.961 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:19352 3.3 3:c34e5e51:::rbd_data.112d92027590.0000000000000024:f [read 1169408~1024 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:32:34.961 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:27.062798+0000", 2024-09-14T06:32:34.961 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 187.88350568499999, 2024-09-14T06:32:34.961 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046255900000000003, 2024-09-14T06:32:34.962 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:32:34.962 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:32:34.962 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:32:34.962 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:32:34.962 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:32:34.962 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:34.962 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 19352 2024-09-14T06:32:34.962 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:34.962 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:32:34.962 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:34.962 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:32:34.962 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062798+0000", 2024-09-14T06:32:34.962 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:34.962 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:34.962 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:34.963 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:32:34.963 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062798+0000", 2024-09-14T06:32:34.963 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:34.963 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:34.963 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:34.963 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:32:34.963 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062798+0000", 2024-09-14T06:32:34.963 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.37e-07 2024-09-14T06:32:34.963 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:34.963 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:34.963 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:32:34.963 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062800+0000", 2024-09-14T06:32:34.963 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.0269999999999998e-06 2024-09-14T06:32:34.963 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:34.963 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:34.964 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:32:34.964 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062806+0000", 2024-09-14T06:32:34.964 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.6690000000000003e-06 2024-09-14T06:32:34.964 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:34.964 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:34.964 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:32:34.964 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062810+0000", 2024-09-14T06:32:34.964 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.3449999999999999e-06 2024-09-14T06:32:34.964 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:34.964 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:34.964 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:32:34.964 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.108892+0000", 2024-09-14T06:32:34.964 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046081715000000002 2024-09-14T06:32:34.964 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:34.964 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:34.965 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:32:34.965 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.108940+0000", 2024-09-14T06:32:34.965 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.7815999999999998e-05 2024-09-14T06:32:34.965 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:34.965 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:34.965 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:32:34.965 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.109053+0000", 2024-09-14T06:32:34.965 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.000113791 2024-09-14T06:32:34.965 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:34.965 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:32:34.965 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:34.965 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:34.965 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:34.965 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:19353 3.3 3:c34e5e51:::rbd_data.112d92027590.0000000000000024:f [read 1179648~1024 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:32:34.966 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:27.062816+0000", 2024-09-14T06:32:34.966 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 187.88348716600001, 2024-09-14T06:32:34.966 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046481256999999998, 2024-09-14T06:32:34.966 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:32:34.966 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:32:34.966 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:32:34.966 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:32:34.966 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:32:34.966 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:34.966 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 19353 2024-09-14T06:32:34.966 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:34.966 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:32:34.966 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:34.966 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:32:34.966 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062816+0000", 2024-09-14T06:32:34.966 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:34.967 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:34.967 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:34.967 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:32:34.967 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062816+0000", 2024-09-14T06:32:34.967 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:34.967 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:34.967 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:34.967 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:32:34.967 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062816+0000", 2024-09-14T06:32:34.967 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.2899999999999999e-07 2024-09-14T06:32:34.967 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:34.967 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:34.967 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:32:34.967 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062818+0000", 2024-09-14T06:32:34.967 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.699e-06 2024-09-14T06:32:34.968 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:34.968 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:34.968 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:32:34.968 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062825+0000", 2024-09-14T06:32:34.968 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.878e-06 2024-09-14T06:32:34.968 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:34.968 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:34.968 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:32:34.968 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062830+0000", 2024-09-14T06:32:34.968 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.5410000000000002e-06 2024-09-14T06:32:34.968 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:34.968 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:34.968 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:32:34.968 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.109096+0000", 2024-09-14T06:32:34.968 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046266517 2024-09-14T06:32:34.969 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:34.969 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:34.969 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:32:34.969 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.109190+0000", 2024-09-14T06:32:34.969 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.4000999999999997e-05 2024-09-14T06:32:34.969 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:34.969 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:34.969 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:32:34.969 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.109297+0000", 2024-09-14T06:32:34.969 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.000107292 2024-09-14T06:32:34.969 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:34.969 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:32:34.969 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:34.970 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:34.970 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:34.970 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:19354 3.3 3:c34e5e51:::rbd_data.112d92027590.0000000000000024:f [read 1189888~1024 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:32:34.970 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:27.062842+0000", 2024-09-14T06:32:34.970 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 187.883461711, 2024-09-14T06:32:34.970 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046594257, 2024-09-14T06:32:34.970 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:32:34.970 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:32:34.970 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:32:34.970 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:32:34.970 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:32:34.970 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:34.970 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 19354 2024-09-14T06:32:34.970 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:34.970 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:32:34.971 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:34.971 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:32:34.971 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062842+0000", 2024-09-14T06:32:34.971 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:34.971 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:34.971 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:34.971 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:32:34.971 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062842+0000", 2024-09-14T06:32:34.971 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:34.971 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:34.971 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:34.971 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:32:34.971 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062842+0000", 2024-09-14T06:32:34.971 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.7399999999999999e-07 2024-09-14T06:32:34.971 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:34.971 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:34.971 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:32:34.972 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062859+0000", 2024-09-14T06:32:34.972 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.7010000000000001e-05 2024-09-14T06:32:34.972 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:34.972 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:34.972 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:32:34.972 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062866+0000", 2024-09-14T06:32:34.972 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.9990000000000002e-06 2024-09-14T06:32:34.972 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:34.972 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:34.972 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:32:34.972 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062873+0000", 2024-09-14T06:32:34.972 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.8569999999999997e-06 2024-09-14T06:32:34.972 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:34.972 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:34.972 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:32:34.973 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.109335+0000", 2024-09-14T06:32:34.973 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046461943999999998 2024-09-14T06:32:34.973 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:34.973 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:34.973 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:32:34.973 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.109377+0000", 2024-09-14T06:32:34.973 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.2558999999999997e-05 2024-09-14T06:32:34.973 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:34.973 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:34.973 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:32:34.973 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.109436+0000", 2024-09-14T06:32:34.973 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.8514000000000003e-05 2024-09-14T06:32:34.973 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:34.973 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:32:34.973 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:34.974 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:34.974 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:34.974 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:19355 3.3 3:c34e5e51:::rbd_data.112d92027590.0000000000000024:f [read 1200128~1024 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:32:34.974 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:27.062884+0000", 2024-09-14T06:32:34.974 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 187.883419038, 2024-09-14T06:32:34.974 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046681650999999998, 2024-09-14T06:32:34.974 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:32:34.974 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:32:34.974 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:32:34.974 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:32:34.974 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:32:34.974 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:34.974 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 19355 2024-09-14T06:32:34.975 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:34.975 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:32:34.975 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:34.975 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:32:34.975 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062884+0000", 2024-09-14T06:32:34.975 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:34.975 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:34.975 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:34.975 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:32:34.975 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062884+0000", 2024-09-14T06:32:34.975 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:34.975 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:34.975 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:34.975 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:32:34.976 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062885+0000", 2024-09-14T06:32:34.976 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.8000000000000001e-07 2024-09-14T06:32:34.976 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:34.976 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:34.976 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:32:34.976 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062886+0000", 2024-09-14T06:32:34.976 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.8920000000000001e-06 2024-09-14T06:32:34.976 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:34.976 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:34.976 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:32:34.976 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062892+0000", 2024-09-14T06:32:34.976 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.84e-06 2024-09-14T06:32:34.976 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:34.977 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:34.977 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:32:34.977 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062899+0000", 2024-09-14T06:32:34.977 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.5339999999999996e-06 2024-09-14T06:32:34.977 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:34.977 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:34.977 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:32:34.977 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.109466+0000", 2024-09-14T06:32:34.977 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046566919999999998 2024-09-14T06:32:34.977 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:34.977 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:34.977 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:32:34.977 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.109510+0000", 2024-09-14T06:32:34.977 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.4153000000000002e-05 2024-09-14T06:32:34.978 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:34.978 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:34.978 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:32:34.978 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.109566+0000", 2024-09-14T06:32:34.978 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.5931999999999999e-05 2024-09-14T06:32:34.978 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:34.978 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:32:34.978 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:34.978 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:34.978 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:34.978 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:19356 3.3 3:c34e5e51:::rbd_data.112d92027590.0000000000000024:f [read 1210368~1024 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:32:34.978 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:27.062908+0000", 2024-09-14T06:32:34.978 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 187.88339560099999, 2024-09-14T06:32:34.978 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046807498000000003, 2024-09-14T06:32:34.979 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:32:34.979 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:32:34.979 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:32:34.979 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:32:34.979 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:32:34.979 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:34.979 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 19356 2024-09-14T06:32:34.979 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:34.979 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:32:34.979 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:34.979 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:32:34.979 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062908+0000", 2024-09-14T06:32:34.979 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:34.979 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:34.979 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:34.979 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:32:34.980 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062908+0000", 2024-09-14T06:32:34.980 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:34.980 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:34.980 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:34.980 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:32:34.980 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062908+0000", 2024-09-14T06:32:34.980 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.5699999999999998e-07 2024-09-14T06:32:34.980 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:34.980 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:34.980 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:32:34.980 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062910+0000", 2024-09-14T06:32:34.980 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.7710000000000001e-06 2024-09-14T06:32:34.980 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:34.981 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:34.981 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:32:34.981 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062920+0000", 2024-09-14T06:32:34.981 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.0654e-05 2024-09-14T06:32:34.981 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:34.981 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:34.981 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:32:34.981 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062931+0000", 2024-09-14T06:32:34.981 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.075e-05 2024-09-14T06:32:34.981 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:34.981 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:34.981 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:32:34.981 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.109595+0000", 2024-09-14T06:32:34.982 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046663713000000002 2024-09-14T06:32:34.982 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:34.982 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:34.982 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:32:34.982 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.109632+0000", 2024-09-14T06:32:34.982 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.7505000000000001e-05 2024-09-14T06:32:34.982 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:34.982 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:34.982 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:32:34.982 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.109715+0000", 2024-09-14T06:32:34.982 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.2748000000000001e-05 2024-09-14T06:32:34.982 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:34.982 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:32:34.982 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:34.982 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:34.983 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:34.983 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:19357 3.3 3:c34e5e51:::rbd_data.112d92027590.0000000000000024:f [read 1220608~1024 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:32:34.983 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:27.062939+0000", 2024-09-14T06:32:34.983 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 187.883364153, 2024-09-14T06:32:34.983 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046920508, 2024-09-14T06:32:34.983 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:32:34.983 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:32:34.983 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:32:34.983 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:32:34.983 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:32:34.983 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:34.983 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 19357 2024-09-14T06:32:34.983 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:34.983 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:32:34.983 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:34.983 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:32:34.984 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062939+0000", 2024-09-14T06:32:34.984 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:34.984 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:34.984 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:34.984 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:32:34.984 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062939+0000", 2024-09-14T06:32:34.984 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:34.984 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:34.984 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:34.984 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:32:34.984 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062940+0000", 2024-09-14T06:32:34.984 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.32e-07 2024-09-14T06:32:34.984 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:34.984 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:34.985 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:32:34.985 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062942+0000", 2024-09-14T06:32:34.985 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.1919999999999999e-06 2024-09-14T06:32:34.985 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:34.985 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:34.985 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:32:34.985 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062948+0000", 2024-09-14T06:32:34.985 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.1149999999999999e-06 2024-09-14T06:32:34.985 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:34.985 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:34.985 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:32:34.985 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062960+0000", 2024-09-14T06:32:34.985 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.1895e-05 2024-09-14T06:32:34.985 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:34.985 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:34.986 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:32:34.986 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.109704+0000", 2024-09-14T06:32:34.986 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046744584999999998 2024-09-14T06:32:34.986 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:34.986 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:34.986 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:32:34.986 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.109777+0000", 2024-09-14T06:32:34.986 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.2282999999999999e-05 2024-09-14T06:32:34.986 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:34.986 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:34.986 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:32:34.986 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.109860+0000", 2024-09-14T06:32:34.986 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.3005999999999998e-05 2024-09-14T06:32:34.986 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:34.986 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:32:34.986 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:34.987 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:34.987 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:34.987 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:19358 3.3 3:c34e5e51:::rbd_data.112d92027590.0000000000000024:f [read 1230848~1024 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:32:34.987 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:27.062967+0000", 2024-09-14T06:32:34.987 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 187.883336523, 2024-09-14T06:32:34.987 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047144471, 2024-09-14T06:32:34.987 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:32:34.987 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:32:34.987 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:32:34.987 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:32:34.987 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:32:34.987 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:34.987 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 19358 2024-09-14T06:32:34.987 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:34.987 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:32:34.987 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:34.988 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:32:34.988 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062967+0000", 2024-09-14T06:32:34.988 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:34.988 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:34.988 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:34.988 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:32:34.988 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062967+0000", 2024-09-14T06:32:34.988 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:34.988 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:34.988 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:34.988 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:32:34.988 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062967+0000", 2024-09-14T06:32:34.988 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.89e-07 2024-09-14T06:32:34.988 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:34.989 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:34.989 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:32:34.989 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062969+0000", 2024-09-14T06:32:34.989 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.9929999999999998e-06 2024-09-14T06:32:34.989 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:34.989 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:34.989 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:32:34.989 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062975+0000", 2024-09-14T06:32:34.989 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.9030000000000001e-06 2024-09-14T06:32:34.989 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:34.989 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:34.989 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:32:34.989 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062979+0000", 2024-09-14T06:32:34.989 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.2250000000000002e-06 2024-09-14T06:32:34.990 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:34.990 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:34.990 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:32:34.990 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.109934+0000", 2024-09-14T06:32:34.990 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046954410000000002 2024-09-14T06:32:34.990 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:34.990 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:34.990 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:32:34.990 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.110007+0000", 2024-09-14T06:32:34.990 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.3829000000000001e-05 2024-09-14T06:32:34.990 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:34.990 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:34.990 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:32:34.990 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.110111+0000", 2024-09-14T06:32:34.990 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00010372199999999999 2024-09-14T06:32:34.990 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:34.991 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:32:34.991 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:34.991 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:34.991 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:34.991 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:19359 3.3 3:c34e5e51:::rbd_data.112d92027590.0000000000000024:f [read 1241088~1024 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:32:34.991 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:27.062987+0000", 2024-09-14T06:32:34.991 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 187.883316433, 2024-09-14T06:32:34.991 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047297182, 2024-09-14T06:32:34.991 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:32:34.991 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:32:34.991 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:32:34.991 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:32:34.991 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:32:34.991 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:34.992 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 19359 2024-09-14T06:32:34.992 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:34.992 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:32:34.992 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:34.992 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:32:34.992 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062987+0000", 2024-09-14T06:32:34.992 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:34.992 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:34.992 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:34.992 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:32:34.992 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062987+0000", 2024-09-14T06:32:34.992 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:34.992 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:34.992 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:34.992 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:32:34.993 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062987+0000", 2024-09-14T06:32:34.993 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.2500000000000001e-07 2024-09-14T06:32:34.993 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:34.993 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:34.993 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:32:34.993 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062989+0000", 2024-09-14T06:32:34.993 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.6080000000000001e-06 2024-09-14T06:32:34.993 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:34.993 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:34.993 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:32:34.993 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062994+0000", 2024-09-14T06:32:34.993 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.5520000000000004e-06 2024-09-14T06:32:34.994 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:34.994 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:34.994 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:32:34.994 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062999+0000", 2024-09-14T06:32:34.994 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.3930000000000001e-06 2024-09-14T06:32:34.994 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:34.994 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:34.994 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:32:34.994 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.110142+0000", 2024-09-14T06:32:34.994 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047143154 2024-09-14T06:32:34.994 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:34.994 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:34.994 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:32:34.995 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.110219+0000", 2024-09-14T06:32:34.995 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.6737999999999996e-05 2024-09-14T06:32:34.995 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:34.995 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:34.995 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:32:34.995 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.110284+0000", 2024-09-14T06:32:34.995 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.5412000000000003e-05 2024-09-14T06:32:34.995 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:34.995 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:32:34.995 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:34.995 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:34.995 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:34.995 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:19360 3.3 3:c34e5e51:::rbd_data.112d92027590.0000000000000024:f [read 1251328~1024 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:32:34.995 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:27.063017+0000", 2024-09-14T06:32:34.996 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 187.88328597200001, 2024-09-14T06:32:34.996 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047453823999999999, 2024-09-14T06:32:34.996 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:32:34.996 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:32:34.996 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:32:34.996 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:32:34.996 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:32:34.996 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:34.996 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 19360 2024-09-14T06:32:34.996 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:34.996 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:32:34.996 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:34.997 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:32:34.997 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.063017+0000", 2024-09-14T06:32:34.997 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:34.997 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:34.997 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:34.997 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:32:34.997 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.063017+0000", 2024-09-14T06:32:34.997 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:34.997 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:34.997 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:34.997 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:32:34.997 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.064977+0000", 2024-09-14T06:32:34.997 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.0019597080000000001 2024-09-14T06:32:34.998 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:34.998 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:34.998 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:32:34.998 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.064987+0000", 2024-09-14T06:32:34.998 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.9620000000000005e-06 2024-09-14T06:32:34.998 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:34.998 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:34.998 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:32:34.998 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065000+0000", 2024-09-14T06:32:34.998 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.3173000000000001e-05 2024-09-14T06:32:34.998 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:34.998 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:34.998 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:32:34.998 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065005+0000", 2024-09-14T06:32:34.999 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.9960000000000001e-06 2024-09-14T06:32:34.999 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:34.999 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:34.999 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:32:34.999 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.110314+0000", 2024-09-14T06:32:34.999 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.04530899 2024-09-14T06:32:34.999 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:34.999 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:34.999 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:32:34.999 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.110398+0000", 2024-09-14T06:32:34.999 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.3539000000000001e-05 2024-09-14T06:32:34.999 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:34.999 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:35.000 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:32:35.000 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.110471+0000", 2024-09-14T06:32:35.000 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.3455999999999996e-05 2024-09-14T06:32:35.000 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:35.000 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:32:35.000 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:35.000 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:35.000 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:35.000 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:19365 3.3 3:c34e5e51:::rbd_data.112d92027590.0000000000000024:f [read 1302528~1024 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:32:35.000 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:27.065058+0000", 2024-09-14T06:32:35.000 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 187.881245302, 2024-09-14T06:32:35.000 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046105526000000001, 2024-09-14T06:32:35.000 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:32:35.001 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:32:35.001 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:32:35.001 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:32:35.001 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:32:35.001 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:35.001 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 19365 2024-09-14T06:32:35.001 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:35.001 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:32:35.001 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:35.001 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:32:35.001 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065058+0000", 2024-09-14T06:32:35.001 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:35.001 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:35.001 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:35.002 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:32:35.002 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065058+0000", 2024-09-14T06:32:35.002 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:35.002 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:35.002 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:35.002 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:32:35.002 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065058+0000", 2024-09-14T06:32:35.002 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.1899999999999999e-07 2024-09-14T06:32:35.002 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:35.002 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:35.002 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:32:35.002 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065059+0000", 2024-09-14T06:32:35.002 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.1299999999999998e-07 2024-09-14T06:32:35.002 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:35.002 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:35.003 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:32:35.003 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065063+0000", 2024-09-14T06:32:35.003 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.1099999999999996e-06 2024-09-14T06:32:35.003 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:35.003 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:35.003 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:32:35.003 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065070+0000", 2024-09-14T06:32:35.003 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.973e-06 2024-09-14T06:32:35.003 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:35.003 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:35.003 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:32:35.003 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.111014+0000", 2024-09-14T06:32:35.003 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.045943373000000003 2024-09-14T06:32:35.003 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:35.004 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:35.004 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:32:35.004 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.111073+0000", 2024-09-14T06:32:35.004 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.8987999999999998e-05 2024-09-14T06:32:35.004 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:35.004 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:35.004 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:32:35.004 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.111163+0000", 2024-09-14T06:32:35.004 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.0950000000000002e-05 2024-09-14T06:32:35.004 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:35.004 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:32:35.004 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:35.004 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:35.004 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:35.005 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:19366 3.3 3:c34e5e51:::rbd_data.112d92027590.0000000000000024:f [read 1312768~1024 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:32:35.005 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:27.065074+0000", 2024-09-14T06:32:35.005 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 187.88122945200001, 2024-09-14T06:32:35.005 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046289858000000003, 2024-09-14T06:32:35.005 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:32:35.005 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:32:35.005 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:32:35.005 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:32:35.005 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:32:35.005 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:35.005 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 19366 2024-09-14T06:32:35.005 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:35.005 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:32:35.005 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:35.005 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:32:35.006 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065074+0000", 2024-09-14T06:32:35.006 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:35.006 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:35.006 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:35.006 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:32:35.006 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065074+0000", 2024-09-14T06:32:35.006 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:35.006 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:35.006 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:35.006 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:32:35.006 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065074+0000", 2024-09-14T06:32:35.006 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.3900000000000001e-07 2024-09-14T06:32:35.006 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:35.006 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:35.006 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:32:35.007 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065078+0000", 2024-09-14T06:32:35.007 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.3229999999999999e-06 2024-09-14T06:32:35.007 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:35.007 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:35.007 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:32:35.007 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065082+0000", 2024-09-14T06:32:35.007 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.8299999999999998e-06 2024-09-14T06:32:35.007 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:35.007 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:35.007 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:32:35.007 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065085+0000", 2024-09-14T06:32:35.007 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.8660000000000002e-06 2024-09-14T06:32:35.007 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:35.007 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:35.007 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:32:35.007 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.111270+0000", 2024-09-14T06:32:35.008 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046185035999999999 2024-09-14T06:32:35.008 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:35.008 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:35.008 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:32:35.008 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.111311+0000", 2024-09-14T06:32:35.008 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.0935e-05 2024-09-14T06:32:35.008 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:35.008 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:35.008 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:32:35.008 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.111364+0000", 2024-09-14T06:32:35.008 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.2629e-05 2024-09-14T06:32:35.008 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:35.008 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:32:35.008 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:35.009 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:35.009 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:35.009 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:19367 3.3 3:c34e5e51:::rbd_data.112d92027590.0000000000000024:f [read 1323008~1024 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:32:35.009 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:27.065115+0000", 2024-09-14T06:32:35.009 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 187.881187891, 2024-09-14T06:32:35.009 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046407003000000002, 2024-09-14T06:32:35.009 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:32:35.009 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:32:35.009 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:32:35.009 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:32:35.009 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:32:35.009 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:35.009 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 19367 2024-09-14T06:32:35.009 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:35.010 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:32:35.010 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:35.010 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:32:35.010 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065115+0000", 2024-09-14T06:32:35.010 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:35.010 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:35.010 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:35.010 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:32:35.010 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065115+0000", 2024-09-14T06:32:35.010 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:35.010 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:35.010 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:35.010 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:32:35.010 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065116+0000", 2024-09-14T06:32:35.010 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.4999999999999998e-07 2024-09-14T06:32:35.011 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:35.011 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:35.011 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:32:35.011 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065117+0000", 2024-09-14T06:32:35.011 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.189e-06 2024-09-14T06:32:35.011 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:35.011 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:35.011 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:32:35.011 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065122+0000", 2024-09-14T06:32:35.011 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.3709999999999999e-06 2024-09-14T06:32:35.011 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:35.011 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:35.011 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:32:35.011 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065125+0000", 2024-09-14T06:32:35.012 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.8940000000000002e-06 2024-09-14T06:32:35.012 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:35.012 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:35.012 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:32:35.012 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.111402+0000", 2024-09-14T06:32:35.012 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046276645999999998 2024-09-14T06:32:35.012 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:35.012 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:35.012 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:32:35.012 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.111445+0000", 2024-09-14T06:32:35.012 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.3288000000000003e-05 2024-09-14T06:32:35.012 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:35.012 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:35.012 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:32:35.012 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.111522+0000", 2024-09-14T06:32:35.013 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.7164999999999997e-05 2024-09-14T06:32:35.013 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:35.013 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:32:35.013 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:35.013 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:35.013 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:35.013 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:19368 3.3 3:c34e5e51:::rbd_data.112d92027590.0000000000000024:f [read 1333248~1024 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:32:35.013 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:27.065130+0000", 2024-09-14T06:32:35.013 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 187.881173601, 2024-09-14T06:32:35.013 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046507275000000001, 2024-09-14T06:32:35.013 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:32:35.013 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:32:35.013 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:32:35.013 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:32:35.013 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:32:35.013 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:35.014 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 19368 2024-09-14T06:32:35.014 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:35.014 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:32:35.014 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:35.014 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:32:35.014 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065130+0000", 2024-09-14T06:32:35.014 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:35.014 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:35.014 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:35.014 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:32:35.014 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065130+0000", 2024-09-14T06:32:35.014 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:35.014 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:35.014 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:35.014 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:32:35.014 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065130+0000", 2024-09-14T06:32:35.015 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.0800000000000001e-07 2024-09-14T06:32:35.015 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:35.015 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:35.015 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:32:35.015 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065131+0000", 2024-09-14T06:32:35.015 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.8700000000000004e-07 2024-09-14T06:32:35.015 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:35.015 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:35.015 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:32:35.015 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065134+0000", 2024-09-14T06:32:35.015 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.1039999999999998e-06 2024-09-14T06:32:35.015 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:35.015 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:35.016 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:32:35.016 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065136+0000", 2024-09-14T06:32:35.016 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.3949999999999999e-06 2024-09-14T06:32:35.016 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:35.016 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:35.016 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:32:35.016 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.111541+0000", 2024-09-14T06:32:35.016 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046404206000000003 2024-09-14T06:32:35.016 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:35.016 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:35.016 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:32:35.016 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.111582+0000", 2024-09-14T06:32:35.016 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.1051999999999999e-05 2024-09-14T06:32:35.016 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:35.017 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:35.017 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:32:35.017 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.111637+0000", 2024-09-14T06:32:35.017 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.5322999999999999e-05 2024-09-14T06:32:35.017 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:35.017 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:32:35.017 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:35.017 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:35.017 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:35.017 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:19369 3.3 3:c34e5e51:::rbd_data.112d92027590.0000000000000024:f [read 1343488~1024 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:32:35.017 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:27.065139+0000", 2024-09-14T06:32:35.017 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 187.88116378399999, 2024-09-14T06:32:35.017 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046682169000000003, 2024-09-14T06:32:35.017 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:32:35.018 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:32:35.018 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:32:35.018 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:32:35.018 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:32:35.018 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:35.018 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 19369 2024-09-14T06:32:35.018 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:35.018 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:32:35.018 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:35.018 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:32:35.018 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065139+0000", 2024-09-14T06:32:35.018 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:35.018 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:35.018 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:35.019 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:32:35.019 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065139+0000", 2024-09-14T06:32:35.019 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:35.019 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:35.019 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:35.019 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:32:35.019 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065140+0000", 2024-09-14T06:32:35.019 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.8299999999999998e-07 2024-09-14T06:32:35.019 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:35.019 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:35.019 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:32:35.019 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065141+0000", 2024-09-14T06:32:35.019 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.3900000000000003e-07 2024-09-14T06:32:35.020 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:35.020 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:35.020 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:32:35.020 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065144+0000", 2024-09-14T06:32:35.020 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.7409999999999998e-06 2024-09-14T06:32:35.020 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:35.020 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:35.020 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:32:35.020 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065147+0000", 2024-09-14T06:32:35.020 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.4150000000000002e-06 2024-09-14T06:32:35.020 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:35.020 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:35.020 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:32:35.021 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.111673+0000", 2024-09-14T06:32:35.021 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046526004000000003 2024-09-14T06:32:35.021 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:35.021 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:35.021 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:32:35.021 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.111749+0000", 2024-09-14T06:32:35.021 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.5741999999999994e-05 2024-09-14T06:32:35.021 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:35.021 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:35.021 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:32:35.021 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.111822+0000", 2024-09-14T06:32:35.021 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.3045000000000003e-05 2024-09-14T06:32:35.021 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:35.021 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:32:35.022 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:35.022 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:35.022 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:35.022 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:19370 3.3 3:c34e5e51:::rbd_data.112d92027590.0000000000000024:f [read 1353728~1024 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:32:35.022 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:27.065150+0000", 2024-09-14T06:32:35.022 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 187.881153055, 2024-09-14T06:32:35.022 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046897556999999999, 2024-09-14T06:32:35.022 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:32:35.022 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:32:35.022 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:32:35.022 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:32:35.022 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:32:35.022 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:35.022 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 19370 2024-09-14T06:32:35.022 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:35.023 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:32:35.023 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:35.023 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:32:35.023 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065150+0000", 2024-09-14T06:32:35.023 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:35.023 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:35.023 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:35.023 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:32:35.023 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065150+0000", 2024-09-14T06:32:35.023 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:35.023 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:35.023 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:35.023 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:32:35.023 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065150+0000", 2024-09-14T06:32:35.023 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.0200000000000001e-07 2024-09-14T06:32:35.024 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:35.024 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:35.024 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:32:35.024 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065151+0000", 2024-09-14T06:32:35.024 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.1100000000000004e-07 2024-09-14T06:32:35.024 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:35.024 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:35.024 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:32:35.024 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065154+0000", 2024-09-14T06:32:35.024 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.9639999999999999e-06 2024-09-14T06:32:35.024 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:35.024 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:35.024 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:32:35.024 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065157+0000", 2024-09-14T06:32:35.025 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.4260000000000002e-06 2024-09-14T06:32:35.025 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:35.025 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:35.025 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:32:35.025 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.111861+0000", 2024-09-14T06:32:35.025 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046704268 2024-09-14T06:32:35.025 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:35.025 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:35.025 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:32:35.025 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.111939+0000", 2024-09-14T06:32:35.025 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.8231000000000004e-05 2024-09-14T06:32:35.025 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:35.025 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:35.026 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:32:35.026 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.112048+0000", 2024-09-14T06:32:35.026 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00010855500000000001 2024-09-14T06:32:35.026 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:35.026 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:32:35.026 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:35.026 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:35.026 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:35.026 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:19371 3.3 3:c34e5e51:::rbd_data.112d92027590.0000000000000024:f [read 1363968~1024 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:32:35.026 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:27.065161+0000", 2024-09-14T06:32:35.026 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 187.88114240499999, 2024-09-14T06:32:35.026 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047021944000000003, 2024-09-14T06:32:35.026 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:32:35.026 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:32:35.027 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:32:35.027 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:32:35.027 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:32:35.027 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:35.027 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 19371 2024-09-14T06:32:35.027 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:35.027 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:32:35.027 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:35.027 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:32:35.027 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065161+0000", 2024-09-14T06:32:35.027 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:35.027 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:35.027 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:35.027 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:32:35.028 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065161+0000", 2024-09-14T06:32:35.028 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:35.028 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:35.028 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:35.028 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:32:35.028 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065161+0000", 2024-09-14T06:32:35.028 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.0200000000000001e-07 2024-09-14T06:32:35.028 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:35.028 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:35.028 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:32:35.028 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065166+0000", 2024-09-14T06:32:35.028 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.9869999999999999e-06 2024-09-14T06:32:35.028 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:35.028 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:35.029 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:32:35.029 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065169+0000", 2024-09-14T06:32:35.029 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.3799999999999998e-06 2024-09-14T06:32:35.029 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:35.029 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:35.029 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:32:35.029 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065172+0000", 2024-09-14T06:32:35.029 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.4830000000000002e-06 2024-09-14T06:32:35.029 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:35.029 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:35.029 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:32:35.029 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.112071+0000", 2024-09-14T06:32:35.029 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046899529000000002 2024-09-14T06:32:35.030 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:35.030 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:35.030 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:32:35.030 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.112128+0000", 2024-09-14T06:32:35.030 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.6341999999999997e-05 2024-09-14T06:32:35.030 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:35.030 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:35.030 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:32:35.030 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.112183+0000", 2024-09-14T06:32:35.030 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.5021000000000001e-05 2024-09-14T06:32:35.030 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:35.030 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:32:35.030 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:35.030 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:35.031 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:35.031 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:19372 3.3 3:c34e5e51:::rbd_data.112d92027590.0000000000000024:f [read 1374208~1024 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:32:35.031 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:27.065176+0000", 2024-09-14T06:32:35.031 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 187.881127255, 2024-09-14T06:32:35.031 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047188889999999997, 2024-09-14T06:32:35.031 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:32:35.031 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:32:35.031 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:32:35.031 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:32:35.031 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:32:35.031 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:35.031 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 19372 2024-09-14T06:32:35.031 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:35.031 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:32:35.031 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:35.032 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:32:35.032 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065176+0000", 2024-09-14T06:32:35.032 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:35.032 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:35.032 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:35.032 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:32:35.032 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065176+0000", 2024-09-14T06:32:35.032 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:35.032 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:35.032 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:35.032 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:32:35.032 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.071117+0000", 2024-09-14T06:32:35.032 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.0059404990000000001 2024-09-14T06:32:35.032 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:35.032 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:35.033 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:32:35.033 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.071120+0000", 2024-09-14T06:32:35.033 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.4340000000000001e-06 2024-09-14T06:32:35.033 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:35.033 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:35.033 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:32:35.033 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.071129+0000", 2024-09-14T06:32:35.033 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.8389999999999996e-06 2024-09-14T06:32:35.033 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:35.033 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:35.033 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:32:35.033 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.071134+0000", 2024-09-14T06:32:35.033 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.412e-06 2024-09-14T06:32:35.033 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:35.033 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:35.034 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:32:35.034 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.112221+0000", 2024-09-14T06:32:35.034 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.041086473999999998 2024-09-14T06:32:35.034 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:35.034 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:35.034 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:32:35.034 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.112299+0000", 2024-09-14T06:32:35.034 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.8746999999999997e-05 2024-09-14T06:32:35.034 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:35.034 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:35.034 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:32:35.034 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.112365+0000", 2024-09-14T06:32:35.034 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.5485000000000001e-05 2024-09-14T06:32:35.034 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:35.034 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:32:35.035 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:35.035 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:35.035 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:32:35.035 INFO:teuthology.orchestra.run.smithi138.stdout:} 2024-09-14T06:32:35.035 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 4, 5, 7, 3] out_osds: [6] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-14T06:32:35.035 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 0 1 2024-09-14T06:32:35.037 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-14T06:32:35.037 INFO:tasks.ceph.osd.4.smithi138.stderr:2024-09-14T06:32:35.035+0000 7f6c653c8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33615) UID: 0 2024-09-14T06:32:35.137 INFO:tasks.ceph.osd.7.smithi138.stderr:2024-09-14T06:32:35.136+0000 7f3c6a21d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33623) UID: 0 2024-09-14T06:32:35.181 INFO:teuthology.orchestra.run.smithi138.stdout:{ 2024-09-14T06:32:35.181 INFO:teuthology.orchestra.run.smithi138.stdout: "ops": [], 2024-09-14T06:32:35.181 INFO:teuthology.orchestra.run.smithi138.stdout: "num_ops": 0 2024-09-14T06:32:35.181 INFO:teuthology.orchestra.run.smithi138.stdout:} 2024-09-14T06:32:35.190 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-14T06:32:35.275 INFO:tasks.ceph.osd.4.smithi138.stderr:2024-09-14T06:32:35.274+0000 7f6c653c8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33615) UID: 0 2024-09-14T06:32:35.335 INFO:teuthology.orchestra.run.smithi138.stdout:{ 2024-09-14T06:32:35.335 INFO:teuthology.orchestra.run.smithi138.stdout: "ops": [], 2024-09-14T06:32:35.335 INFO:teuthology.orchestra.run.smithi138.stdout: "complaint_time": 30, 2024-09-14T06:32:35.335 INFO:teuthology.orchestra.run.smithi138.stdout: "num_blocked_ops": 0 2024-09-14T06:32:35.335 INFO:teuthology.orchestra.run.smithi138.stdout:} 2024-09-14T06:32:35.339 INFO:tasks.ceph.osd.1.smithi094.stderr:2024-09-14T06:32:35.337+0000 7f328e47a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33641) UID: 0 2024-09-14T06:32:35.344 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-14T06:32:35.441 INFO:tasks.ceph.osd.2.smithi094.stderr:2024-09-14T06:32:35.439+0000 7faf3269b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33649) UID: 0 2024-09-14T06:32:35.488 INFO:teuthology.orchestra.run.smithi138.stdout:{ 2024-09-14T06:32:35.488 INFO:teuthology.orchestra.run.smithi138.stdout: "size": 20, 2024-09-14T06:32:35.488 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 600, 2024-09-14T06:32:35.488 INFO:teuthology.orchestra.run.smithi138.stdout: "ops": [ 2024-09-14T06:32:35.488 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:35.488 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:11453 3.1d 3:bcad7303:::rbd_data.112d92027590.0000000000000001:f [read 94208~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:32:35.488 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:25.654375+0000", 2024-09-14T06:32:35.488 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 189.832077083, 2024-09-14T06:32:35.488 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052449318000000002, 2024-09-14T06:32:35.489 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:32:35.489 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:32:35.489 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:32:35.489 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:32:35.489 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:32:35.489 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:35.489 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 11453 2024-09-14T06:32:35.489 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:35.489 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:32:35.489 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:35.489 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:32:35.489 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654375+0000", 2024-09-14T06:32:35.489 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:35.489 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:35.489 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:35.489 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:32:35.490 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654375+0000", 2024-09-14T06:32:35.490 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:35.490 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:35.490 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:35.490 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:32:35.490 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654496+0000", 2024-09-14T06:32:35.490 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00012081 2024-09-14T06:32:35.490 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:35.490 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:35.490 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:32:35.490 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654497+0000", 2024-09-14T06:32:35.490 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.8789999999999999e-06 2024-09-14T06:32:35.490 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:35.490 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:35.491 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:32:35.491 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654501+0000", 2024-09-14T06:32:35.491 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.8999999999999999e-06 2024-09-14T06:32:35.491 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:35.491 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:35.491 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:32:35.491 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654505+0000", 2024-09-14T06:32:35.491 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.2229999999999999e-06 2024-09-14T06:32:35.491 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:35.491 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:35.491 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:32:35.491 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.705917+0000", 2024-09-14T06:32:35.491 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.051412501999999999 2024-09-14T06:32:35.492 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:35.492 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:35.492 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:32:35.492 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.705999+0000", 2024-09-14T06:32:35.492 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.1551999999999994e-05 2024-09-14T06:32:35.492 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:35.492 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:35.492 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:32:35.492 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.706824+0000", 2024-09-14T06:32:35.492 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00082545199999999996 2024-09-14T06:32:35.492 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:35.492 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:32:35.492 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:35.492 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:35.492 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:35.492 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:11454 3.1d 3:bcad7303:::rbd_data.112d92027590.0000000000000001:f [read 104448~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:32:35.493 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:25.654510+0000", 2024-09-14T06:32:35.493 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 189.831942293, 2024-09-14T06:32:35.493 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052610713000000003, 2024-09-14T06:32:35.493 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:32:35.493 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:32:35.493 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:32:35.493 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:32:35.493 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:32:35.493 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:35.493 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 11454 2024-09-14T06:32:35.493 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:35.493 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:32:35.493 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:35.493 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:32:35.494 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654510+0000", 2024-09-14T06:32:35.494 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:35.494 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:35.494 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:35.494 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:32:35.494 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654510+0000", 2024-09-14T06:32:35.494 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:35.494 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:35.494 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:35.494 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:32:35.494 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654825+0000", 2024-09-14T06:32:35.494 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00031522400000000002 2024-09-14T06:32:35.494 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:35.494 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:35.494 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:32:35.495 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654827+0000", 2024-09-14T06:32:35.495 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.9709999999999998e-06 2024-09-14T06:32:35.495 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:35.495 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:35.495 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:32:35.495 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654832+0000", 2024-09-14T06:32:35.495 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.4099999999999999e-06 2024-09-14T06:32:35.495 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:35.495 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:35.495 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:32:35.495 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654835+0000", 2024-09-14T06:32:35.495 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.14e-06 2024-09-14T06:32:35.495 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:35.495 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:35.495 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:32:35.495 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.706851+0000", 2024-09-14T06:32:35.496 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052015761000000001 2024-09-14T06:32:35.496 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:35.496 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:35.496 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:32:35.496 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.706934+0000", 2024-09-14T06:32:35.496 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.2484999999999994e-05 2024-09-14T06:32:35.496 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:35.496 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:35.496 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:32:35.496 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.707120+0000", 2024-09-14T06:32:35.496 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00018672200000000001 2024-09-14T06:32:35.496 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:35.496 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:32:35.496 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:35.496 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:35.497 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:35.497 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:11455 3.1d 3:bcad7303:::rbd_data.112d92027590.0000000000000001:f [read 114688~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:32:35.497 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:25.654841+0000", 2024-09-14T06:32:35.497 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 189.83161096399999, 2024-09-14T06:32:35.497 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052579898, 2024-09-14T06:32:35.497 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:32:35.497 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:32:35.497 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:32:35.497 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:32:35.497 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:32:35.497 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:35.497 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 11455 2024-09-14T06:32:35.497 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:35.497 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:32:35.497 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:35.497 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:32:35.498 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654841+0000", 2024-09-14T06:32:35.498 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:35.498 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:35.498 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:35.498 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:32:35.498 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654841+0000", 2024-09-14T06:32:35.498 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:35.498 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:35.498 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:35.498 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:32:35.498 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654841+0000", 2024-09-14T06:32:35.498 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.7300000000000002e-07 2024-09-14T06:32:35.498 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:35.498 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:35.498 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:32:35.498 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654845+0000", 2024-09-14T06:32:35.499 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.0520000000000003e-06 2024-09-14T06:32:35.499 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:35.499 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:35.499 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:32:35.499 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654849+0000", 2024-09-14T06:32:35.499 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.0459999999999999e-06 2024-09-14T06:32:35.499 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:35.499 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:35.499 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:32:35.499 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654852+0000", 2024-09-14T06:32:35.499 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.9830000000000001e-06 2024-09-14T06:32:35.499 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:35.499 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:35.499 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:32:35.500 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.707159+0000", 2024-09-14T06:32:35.500 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052306826000000001 2024-09-14T06:32:35.500 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:35.500 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:35.500 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:32:35.500 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.707242+0000", 2024-09-14T06:32:35.500 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.2595000000000005e-05 2024-09-14T06:32:35.500 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:35.500 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:35.500 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:32:35.500 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.707421+0000", 2024-09-14T06:32:35.500 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.000179123 2024-09-14T06:32:35.500 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:35.501 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:32:35.501 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:35.501 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:35.501 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:35.501 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:11456 3.1d 3:bcad7303:::rbd_data.112d92027590.0000000000000001:f [read 124928~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:32:35.501 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:25.654856+0000", 2024-09-14T06:32:35.501 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 189.83159611799999, 2024-09-14T06:32:35.501 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052801580000000001, 2024-09-14T06:32:35.501 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:32:35.501 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:32:35.501 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:32:35.501 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:32:35.501 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:32:35.501 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:35.502 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 11456 2024-09-14T06:32:35.502 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:35.502 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:32:35.502 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:35.502 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:32:35.502 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654856+0000", 2024-09-14T06:32:35.502 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:35.502 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:35.502 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:35.502 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:32:35.502 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654856+0000", 2024-09-14T06:32:35.502 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:35.502 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:35.502 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:35.503 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:32:35.503 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654856+0000", 2024-09-14T06:32:35.503 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.29e-07 2024-09-14T06:32:35.503 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:35.503 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:35.503 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:32:35.503 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654857+0000", 2024-09-14T06:32:35.503 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.187e-06 2024-09-14T06:32:35.503 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:35.503 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:35.503 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:32:35.503 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654861+0000", 2024-09-14T06:32:35.503 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.3850000000000002e-06 2024-09-14T06:32:35.503 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:35.503 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:35.503 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:32:35.504 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654863+0000", 2024-09-14T06:32:35.504 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.9500000000000001e-06 2024-09-14T06:32:35.504 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:35.504 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:35.504 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:32:35.504 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.707456+0000", 2024-09-14T06:32:35.504 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052592434 2024-09-14T06:32:35.504 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:35.504 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:35.504 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:32:35.504 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.707516+0000", 2024-09-14T06:32:35.504 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.0167999999999997e-05 2024-09-14T06:32:35.504 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:35.504 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:35.504 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:32:35.505 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.707657+0000", 2024-09-14T06:32:35.505 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00014122700000000001 2024-09-14T06:32:35.505 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:35.505 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:32:35.505 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:35.505 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:35.505 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:35.505 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:11457 3.1d 3:bcad7303:::rbd_data.112d92027590.0000000000000001:f [read 135168~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:32:35.505 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:25.654867+0000", 2024-09-14T06:32:35.505 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 189.831585233, 2024-09-14T06:32:35.505 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052921902, 2024-09-14T06:32:35.505 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:32:35.505 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:32:35.505 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:32:35.505 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:32:35.505 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:32:35.506 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:35.506 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 11457 2024-09-14T06:32:35.506 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:35.506 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:32:35.506 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:35.506 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:32:35.506 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654867+0000", 2024-09-14T06:32:35.506 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:35.506 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:35.506 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:35.506 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:32:35.506 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654867+0000", 2024-09-14T06:32:35.506 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:35.506 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:35.506 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:35.506 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:32:35.507 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654972+0000", 2024-09-14T06:32:35.507 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00010584 2024-09-14T06:32:35.507 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:35.507 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:35.507 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:32:35.507 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654975+0000", 2024-09-14T06:32:35.507 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.1169999999999998e-06 2024-09-14T06:32:35.507 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:35.507 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:35.507 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:32:35.507 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654979+0000", 2024-09-14T06:32:35.507 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.1960000000000001e-06 2024-09-14T06:32:35.507 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:35.507 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:35.507 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:32:35.507 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654982+0000", 2024-09-14T06:32:35.507 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.2990000000000001e-06 2024-09-14T06:32:35.508 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:35.508 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:35.508 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:32:35.508 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.707684+0000", 2024-09-14T06:32:35.508 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052701979000000003 2024-09-14T06:32:35.508 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:35.508 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:35.508 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:32:35.508 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.707728+0000", 2024-09-14T06:32:35.508 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.3529999999999998e-05 2024-09-14T06:32:35.508 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:35.508 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:35.508 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:32:35.509 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.707789+0000", 2024-09-14T06:32:35.509 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.0940999999999998e-05 2024-09-14T06:32:35.509 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:35.509 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:32:35.509 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:35.509 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:35.509 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:35.509 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:11458 3.1d 3:bcad7303:::rbd_data.112d92027590.0000000000000001:f [read 145408~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:32:35.509 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:25.654987+0000", 2024-09-14T06:32:35.509 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 189.83146483499999, 2024-09-14T06:32:35.509 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.05291655, 2024-09-14T06:32:35.509 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:32:35.509 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:32:35.509 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:32:35.510 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:32:35.510 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:32:35.510 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:35.510 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 11458 2024-09-14T06:32:35.510 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:35.510 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:32:35.510 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:35.510 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:32:35.510 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654987+0000", 2024-09-14T06:32:35.510 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:35.510 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:35.510 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:35.510 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:32:35.510 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654987+0000", 2024-09-14T06:32:35.510 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:35.511 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:35.511 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:35.511 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:32:35.511 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655077+0000", 2024-09-14T06:32:35.511 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.0253000000000005e-05 2024-09-14T06:32:35.511 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:35.511 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:35.511 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:32:35.511 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655079+0000", 2024-09-14T06:32:35.511 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.9989999999999998e-06 2024-09-14T06:32:35.511 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:35.511 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:35.511 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:32:35.511 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655083+0000", 2024-09-14T06:32:35.511 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.9280000000000003e-06 2024-09-14T06:32:35.511 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:35.512 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:35.512 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:32:35.512 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655086+0000", 2024-09-14T06:32:35.512 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.056e-06 2024-09-14T06:32:35.512 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:35.512 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:35.512 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:32:35.512 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.707805+0000", 2024-09-14T06:32:35.512 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052718627999999997 2024-09-14T06:32:35.512 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:35.512 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:35.512 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:32:35.512 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.707848+0000", 2024-09-14T06:32:35.512 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.2728999999999997e-05 2024-09-14T06:32:35.512 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:35.512 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:35.513 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:32:35.513 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.707904+0000", 2024-09-14T06:32:35.513 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.5957e-05 2024-09-14T06:32:35.513 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:35.513 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:32:35.513 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:35.513 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:35.513 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:35.513 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:11459 3.1d 3:bcad7303:::rbd_data.112d92027590.0000000000000001:f [read 155648~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:32:35.513 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:25.655091+0000", 2024-09-14T06:32:35.513 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 189.831360827, 2024-09-14T06:32:35.513 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052944242000000002, 2024-09-14T06:32:35.516 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:32:35.516 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:32:35.516 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:32:35.516 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:32:35.516 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:32:35.516 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:35.516 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 11459 2024-09-14T06:32:35.516 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:35.517 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:32:35.517 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:35.517 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:32:35.517 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655091+0000", 2024-09-14T06:32:35.517 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:35.517 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:35.517 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:35.517 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:32:35.517 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655091+0000", 2024-09-14T06:32:35.517 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:35.517 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:35.517 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:35.517 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:32:35.517 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655237+0000", 2024-09-14T06:32:35.517 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00014570100000000001 2024-09-14T06:32:35.517 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:35.517 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:35.518 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:32:35.518 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655239+0000", 2024-09-14T06:32:35.518 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.8670000000000001e-06 2024-09-14T06:32:35.518 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:35.518 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:35.518 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:32:35.518 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655243+0000", 2024-09-14T06:32:35.518 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.9829999999999998e-06 2024-09-14T06:32:35.518 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:35.518 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:35.518 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:32:35.518 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655246+0000", 2024-09-14T06:32:35.518 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.1540000000000002e-06 2024-09-14T06:32:35.518 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:35.518 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:35.518 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:32:35.519 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.707934+0000", 2024-09-14T06:32:35.519 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052688286000000001 2024-09-14T06:32:35.519 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:35.519 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:35.519 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:32:35.519 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.707977+0000", 2024-09-14T06:32:35.519 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.3291999999999998e-05 2024-09-14T06:32:35.519 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:35.519 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:35.519 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:32:35.519 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.708035+0000", 2024-09-14T06:32:35.519 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.7958999999999999e-05 2024-09-14T06:32:35.519 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:35.519 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:32:35.519 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:35.520 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:35.520 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:35.520 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:11460 3.1d 3:bcad7303:::rbd_data.112d92027590.0000000000000001:f [read 165888~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:32:35.520 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:25.655250+0000", 2024-09-14T06:32:35.520 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 189.83120152999999, 2024-09-14T06:32:35.520 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052910698999999999, 2024-09-14T06:32:35.520 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:32:35.520 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:32:35.520 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:32:35.520 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:32:35.520 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:32:35.520 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:35.520 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 11460 2024-09-14T06:32:35.520 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:35.521 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:32:35.521 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:35.521 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:32:35.521 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655250+0000", 2024-09-14T06:32:35.521 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:35.521 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:35.521 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:35.521 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:32:35.521 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655250+0000", 2024-09-14T06:32:35.521 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:35.521 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:35.521 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:35.521 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:32:35.521 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655356+0000", 2024-09-14T06:32:35.521 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00010535900000000001 2024-09-14T06:32:35.522 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:35.522 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:35.522 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:32:35.522 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655357+0000", 2024-09-14T06:32:35.522 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.8080000000000001e-06 2024-09-14T06:32:35.522 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:35.522 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:35.522 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:32:35.522 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655361+0000", 2024-09-14T06:32:35.522 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.861e-06 2024-09-14T06:32:35.522 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:35.522 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:35.522 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:32:35.522 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655364+0000", 2024-09-14T06:32:35.522 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.1089999999999998e-06 2024-09-14T06:32:35.522 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:35.523 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:35.523 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:32:35.523 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.708065+0000", 2024-09-14T06:32:35.523 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052700595000000003 2024-09-14T06:32:35.523 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:35.523 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:35.523 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:32:35.523 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.708108+0000", 2024-09-14T06:32:35.523 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.2484999999999997e-05 2024-09-14T06:32:35.523 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:35.523 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:35.523 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:32:35.523 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.708161+0000", 2024-09-14T06:32:35.523 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.3482000000000001e-05 2024-09-14T06:32:35.523 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:35.523 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:32:35.524 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:35.524 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:35.524 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:35.524 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:11461 3.1d 3:bcad7303:::rbd_data.112d92027590.0000000000000001:f [read 176128~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:32:35.524 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:25.655370+0000", 2024-09-14T06:32:35.524 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 189.831082248, 2024-09-14T06:32:35.524 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052915141999999998, 2024-09-14T06:32:35.524 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:32:35.524 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:32:35.524 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:32:35.524 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:32:35.524 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:32:35.524 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:35.524 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 11461 2024-09-14T06:32:35.524 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:35.524 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:32:35.525 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:35.525 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:32:35.525 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655370+0000", 2024-09-14T06:32:35.525 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:35.525 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:35.525 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:35.525 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:32:35.525 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655370+0000", 2024-09-14T06:32:35.525 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:35.525 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:35.525 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:35.525 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:32:35.525 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655394+0000", 2024-09-14T06:32:35.526 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.4553999999999999e-05 2024-09-14T06:32:35.526 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:35.526 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:35.526 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:32:35.526 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655396+0000", 2024-09-14T06:32:35.526 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.8950000000000001e-06 2024-09-14T06:32:35.526 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:35.526 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:35.526 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:32:35.526 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655401+0000", 2024-09-14T06:32:35.526 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.0089999999999998e-06 2024-09-14T06:32:35.526 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:35.526 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:35.526 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:32:35.527 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655404+0000", 2024-09-14T06:32:35.527 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.3400000000000002e-06 2024-09-14T06:32:35.527 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:35.527 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:35.527 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:32:35.527 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.708186+0000", 2024-09-14T06:32:35.527 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052781726000000001 2024-09-14T06:32:35.527 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:35.527 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:35.527 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:32:35.527 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.708229+0000", 2024-09-14T06:32:35.527 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.2670999999999999e-05 2024-09-14T06:32:35.527 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:35.527 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:35.527 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:32:35.527 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.708285+0000", 2024-09-14T06:32:35.528 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.5946999999999998e-05 2024-09-14T06:32:35.528 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:35.528 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:32:35.528 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:35.528 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:35.528 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:35.528 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:11462 3.1d 3:bcad7303:::rbd_data.112d92027590.0000000000000001:f [read 186368~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:32:35.528 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:25.655410+0000", 2024-09-14T06:32:35.528 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 189.83104177000001, 2024-09-14T06:32:35.528 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052995258000000003, 2024-09-14T06:32:35.528 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:32:35.528 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:32:35.528 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:32:35.528 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:32:35.529 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:32:35.529 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:35.529 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 11462 2024-09-14T06:32:35.529 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:35.529 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:32:35.529 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:35.529 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:32:35.529 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655410+0000", 2024-09-14T06:32:35.529 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:35.529 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:35.529 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:35.529 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:32:35.529 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655410+0000", 2024-09-14T06:32:35.529 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:35.530 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:35.530 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:35.530 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:32:35.530 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655625+0000", 2024-09-14T06:32:35.530 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00021442600000000001 2024-09-14T06:32:35.530 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:35.530 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:35.530 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:32:35.530 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655626+0000", 2024-09-14T06:32:35.530 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.815e-06 2024-09-14T06:32:35.530 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:35.530 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:35.530 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:32:35.530 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655631+0000", 2024-09-14T06:32:35.530 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.651e-06 2024-09-14T06:32:35.531 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:35.531 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:35.531 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:32:35.531 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655634+0000", 2024-09-14T06:32:35.531 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.2449999999999998e-06 2024-09-14T06:32:35.531 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:35.531 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:35.531 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:32:35.531 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.708312+0000", 2024-09-14T06:32:35.531 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052677412 2024-09-14T06:32:35.531 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:35.531 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:35.531 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:32:35.531 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.708352+0000", 2024-09-14T06:32:35.532 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.0729999999999998e-05 2024-09-14T06:32:35.532 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:35.532 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:35.532 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:32:35.532 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.708405+0000", 2024-09-14T06:32:35.532 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.2979000000000002e-05 2024-09-14T06:32:35.532 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:35.532 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:32:35.532 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:35.532 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:35.532 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:35.532 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:11463 3.1d 3:bcad7303:::rbd_data.112d92027590.0000000000000001:f [read 196608~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:32:35.532 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:25.655639+0000", 2024-09-14T06:32:35.532 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 189.830812624, 2024-09-14T06:32:35.532 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052888859000000003, 2024-09-14T06:32:35.533 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:32:35.533 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:32:35.533 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:32:35.533 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:32:35.533 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:32:35.533 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:35.533 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 11463 2024-09-14T06:32:35.533 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:35.533 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:32:35.533 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:35.533 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:32:35.533 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655639+0000", 2024-09-14T06:32:35.533 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:35.533 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:35.534 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:35.534 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:32:35.534 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655639+0000", 2024-09-14T06:32:35.534 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:35.534 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:35.534 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:35.534 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:32:35.534 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655640+0000", 2024-09-14T06:32:35.534 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.08e-07 2024-09-14T06:32:35.534 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:35.534 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:35.534 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:32:35.534 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655641+0000", 2024-09-14T06:32:35.534 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.156e-06 2024-09-14T06:32:35.534 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:35.534 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:35.535 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:32:35.535 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655648+0000", 2024-09-14T06:32:35.535 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.8460000000000002e-06 2024-09-14T06:32:35.535 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:35.535 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:35.535 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:32:35.535 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655651+0000", 2024-09-14T06:32:35.535 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.0079999999999998e-06 2024-09-14T06:32:35.535 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:35.535 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:35.535 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:32:35.535 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.708429+0000", 2024-09-14T06:32:35.535 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052778459 2024-09-14T06:32:35.535 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:35.535 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:35.536 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:32:35.536 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.708473+0000", 2024-09-14T06:32:35.536 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.3782999999999997e-05 2024-09-14T06:32:35.536 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:35.536 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:35.536 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:32:35.536 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.708528+0000", 2024-09-14T06:32:35.536 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.5198999999999998e-05 2024-09-14T06:32:35.536 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:35.536 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:32:35.536 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:35.536 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:35.536 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:35.536 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:11464 3.1d 3:bcad7303:::rbd_data.112d92027590.0000000000000001:f [read 206848~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:32:35.536 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:25.655654+0000", 2024-09-14T06:32:35.537 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 189.83079775499999, 2024-09-14T06:32:35.537 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052981845999999999, 2024-09-14T06:32:35.537 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:32:35.537 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:32:35.537 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:32:35.537 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:32:35.537 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:32:35.537 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:35.537 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 11464 2024-09-14T06:32:35.537 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:35.537 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:32:35.537 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:35.537 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:32:35.538 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655654+0000", 2024-09-14T06:32:35.538 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:35.538 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:35.538 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:35.538 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:32:35.538 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655654+0000", 2024-09-14T06:32:35.538 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:35.538 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:35.538 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:35.538 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:32:35.538 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655946+0000", 2024-09-14T06:32:35.538 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00029148200000000003 2024-09-14T06:32:35.538 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:35.539 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:35.539 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:32:35.539 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655948+0000", 2024-09-14T06:32:35.539 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.973e-06 2024-09-14T06:32:35.539 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:35.539 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:35.539 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:32:35.539 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655953+0000", 2024-09-14T06:32:35.539 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.1240000000000004e-06 2024-09-14T06:32:35.539 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:35.539 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:35.539 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:32:35.539 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655956+0000", 2024-09-14T06:32:35.539 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.315e-06 2024-09-14T06:32:35.539 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:35.540 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:35.540 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:32:35.540 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.708543+0000", 2024-09-14T06:32:35.540 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052587416999999997 2024-09-14T06:32:35.540 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:35.540 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:35.540 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:32:35.540 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.708584+0000", 2024-09-14T06:32:35.540 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.1004000000000002e-05 2024-09-14T06:32:35.540 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:35.540 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:35.540 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:32:35.540 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.708636+0000", 2024-09-14T06:32:35.540 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.1530999999999998e-05 2024-09-14T06:32:35.541 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:35.541 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:32:35.541 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:35.541 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:35.541 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:35.541 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:11465 3.1d 3:bcad7303:::rbd_data.112d92027590.0000000000000001:f [read 217088~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:32:35.541 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:25.655961+0000", 2024-09-14T06:32:35.541 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 189.83049060299999, 2024-09-14T06:32:35.541 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052798682999999999, 2024-09-14T06:32:35.541 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:32:35.541 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:32:35.541 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:32:35.541 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:32:35.541 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:32:35.541 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:35.542 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 11465 2024-09-14T06:32:35.542 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:35.542 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:32:35.542 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:35.542 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:32:35.542 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655961+0000", 2024-09-14T06:32:35.542 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:35.542 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:35.542 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:35.542 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:32:35.542 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655961+0000", 2024-09-14T06:32:35.542 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:35.542 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:35.542 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:35.542 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:32:35.543 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.656114+0000", 2024-09-14T06:32:35.543 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.000152745 2024-09-14T06:32:35.543 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:35.543 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:35.543 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:32:35.543 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.656116+0000", 2024-09-14T06:32:35.543 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.804e-06 2024-09-14T06:32:35.543 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:35.543 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:35.543 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:32:35.543 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.656120+0000", 2024-09-14T06:32:35.543 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.4700000000000004e-06 2024-09-14T06:32:35.543 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:35.544 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:35.544 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:32:35.544 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.656123+0000", 2024-09-14T06:32:35.544 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.1269999999999999e-06 2024-09-14T06:32:35.544 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:35.544 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:35.544 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:32:35.544 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.708660+0000", 2024-09-14T06:32:35.544 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052536919000000001 2024-09-14T06:32:35.544 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:35.544 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:35.544 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:32:35.544 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.708703+0000", 2024-09-14T06:32:35.544 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.3077000000000001e-05 2024-09-14T06:32:35.544 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:35.544 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:35.545 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:32:35.545 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.708760+0000", 2024-09-14T06:32:35.545 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.6541e-05 2024-09-14T06:32:35.545 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:35.545 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:32:35.545 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:35.545 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:35.545 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:35.545 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:11466 3.1d 3:bcad7303:::rbd_data.112d92027590.0000000000000001:f [read 227328~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:32:35.545 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:25.656129+0000", 2024-09-14T06:32:35.545 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 189.83032321499999, 2024-09-14T06:32:35.545 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052750987999999999, 2024-09-14T06:32:35.545 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:32:35.545 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:32:35.545 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:32:35.546 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:32:35.546 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:32:35.546 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:35.546 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 11466 2024-09-14T06:32:35.546 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:35.546 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:32:35.546 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:35.546 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:32:35.546 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.656129+0000", 2024-09-14T06:32:35.546 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:35.546 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:35.546 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:35.546 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:32:35.547 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.656129+0000", 2024-09-14T06:32:35.547 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:35.547 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:35.547 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:35.547 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:32:35.547 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.656347+0000", 2024-09-14T06:32:35.547 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.000218054 2024-09-14T06:32:35.547 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:35.547 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:35.547 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:32:35.547 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.656349+0000", 2024-09-14T06:32:35.547 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.9199999999999998e-06 2024-09-14T06:32:35.547 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:35.547 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:35.547 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:32:35.548 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.656353+0000", 2024-09-14T06:32:35.548 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.9210000000000002e-06 2024-09-14T06:32:35.548 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:35.548 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:35.548 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:32:35.548 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.656356+0000", 2024-09-14T06:32:35.548 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.7670000000000001e-06 2024-09-14T06:32:35.548 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:35.548 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:35.548 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:32:35.548 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.708787+0000", 2024-09-14T06:32:35.548 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052430840999999999 2024-09-14T06:32:35.548 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:35.548 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:35.549 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:32:35.549 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.708827+0000", 2024-09-14T06:32:35.549 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.9538e-05 2024-09-14T06:32:35.549 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:35.549 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:35.549 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:32:35.549 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.708880+0000", 2024-09-14T06:32:35.549 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.2947e-05 2024-09-14T06:32:35.549 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:35.549 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:32:35.549 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:35.549 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:35.549 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:35.549 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:11467 3.1d 3:bcad7303:::rbd_data.112d92027590.0000000000000001:f [read 237568~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:32:35.550 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:25.656361+0000", 2024-09-14T06:32:35.550 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 189.83009056700001, 2024-09-14T06:32:35.550 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052643898000000001, 2024-09-14T06:32:35.550 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:32:35.550 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:32:35.550 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:32:35.550 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:32:35.550 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:32:35.550 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:35.550 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 11467 2024-09-14T06:32:35.551 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:35.551 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:32:35.551 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:35.551 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:32:35.551 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.656361+0000", 2024-09-14T06:32:35.551 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:35.551 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:35.551 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:35.551 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:32:35.551 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.656361+0000", 2024-09-14T06:32:35.551 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:35.551 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:35.551 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:35.551 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:32:35.552 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.656855+0000", 2024-09-14T06:32:35.552 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00049372000000000001 2024-09-14T06:32:35.552 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:35.552 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:35.552 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:32:35.552 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.656859+0000", 2024-09-14T06:32:35.552 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.1300000000000003e-06 2024-09-14T06:32:35.552 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:35.552 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:35.552 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:32:35.552 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.656863+0000", 2024-09-14T06:32:35.552 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.2629999999999997e-06 2024-09-14T06:32:35.552 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:35.552 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:35.552 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:32:35.553 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.656867+0000", 2024-09-14T06:32:35.553 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.3220000000000001e-06 2024-09-14T06:32:35.553 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:35.553 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:35.553 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:32:35.553 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.708903+0000", 2024-09-14T06:32:35.553 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052035855999999998 2024-09-14T06:32:35.553 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:35.553 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:35.553 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:32:35.553 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.708949+0000", 2024-09-14T06:32:35.553 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.6276999999999998e-05 2024-09-14T06:32:35.554 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:35.554 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:35.554 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:32:35.554 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.709005+0000", 2024-09-14T06:32:35.554 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.6329999999999998e-05 2024-09-14T06:32:35.554 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:35.554 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:32:35.554 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:35.554 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:35.554 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:35.554 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:11480 3.1d 3:bcad7303:::rbd_data.112d92027590.0000000000000001:f [read 370688~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:32:35.554 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:25.658563+0000", 2024-09-14T06:32:35.554 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 189.82788880300001, 2024-09-14T06:32:35.554 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052429164, 2024-09-14T06:32:35.555 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:32:35.555 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:32:35.555 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:32:35.555 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:32:35.555 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:32:35.555 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:35.555 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 11480 2024-09-14T06:32:35.555 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:35.555 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:32:35.555 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:35.555 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:32:35.555 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.658563+0000", 2024-09-14T06:32:35.555 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:35.555 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:35.555 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:35.556 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:32:35.556 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.658563+0000", 2024-09-14T06:32:35.556 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:35.556 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:35.556 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:35.556 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:32:35.556 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.658658+0000", 2024-09-14T06:32:35.556 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.5346999999999994e-05 2024-09-14T06:32:35.556 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:35.556 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:35.556 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:32:35.556 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.658662+0000", 2024-09-14T06:32:35.557 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.14e-06 2024-09-14T06:32:35.557 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:35.557 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:35.557 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:32:35.557 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.658669+0000", 2024-09-14T06:32:35.557 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.7379999999999994e-06 2024-09-14T06:32:35.557 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:35.557 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:35.557 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:32:35.557 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.658675+0000", 2024-09-14T06:32:35.557 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.5430000000000001e-06 2024-09-14T06:32:35.557 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:35.557 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:35.558 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:32:35.558 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.710824+0000", 2024-09-14T06:32:35.558 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052148847999999998 2024-09-14T06:32:35.558 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:35.558 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:35.558 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:32:35.558 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.710864+0000", 2024-09-14T06:32:35.558 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.9925999999999997e-05 2024-09-14T06:32:35.558 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:35.558 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:35.558 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:32:35.558 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.710992+0000", 2024-09-14T06:32:35.558 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.000128622 2024-09-14T06:32:35.559 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:35.559 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:32:35.559 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:35.559 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:35.559 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:35.559 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:11481 3.1d 3:bcad7303:::rbd_data.112d92027590.0000000000000001:f [read 380928~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:32:35.559 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:25.658684+0000", 2024-09-14T06:32:35.559 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 189.827767733, 2024-09-14T06:32:35.559 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052508565, 2024-09-14T06:32:35.559 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:32:35.559 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:32:35.559 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:32:35.559 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:32:35.559 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:32:35.560 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:35.560 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 11481 2024-09-14T06:32:35.560 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:35.560 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:32:35.560 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:35.560 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:32:35.560 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.658684+0000", 2024-09-14T06:32:35.560 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:35.560 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:35.560 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:35.560 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:32:35.560 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.658684+0000", 2024-09-14T06:32:35.561 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:35.561 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:35.561 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:35.561 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:32:35.561 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.658715+0000", 2024-09-14T06:32:35.561 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.0703000000000001e-05 2024-09-14T06:32:35.561 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:35.561 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:35.561 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:32:35.561 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.658718+0000", 2024-09-14T06:32:35.561 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.0709999999999999e-06 2024-09-14T06:32:35.561 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:35.561 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:35.561 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:32:35.562 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.658725+0000", 2024-09-14T06:32:35.562 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.2339999999999997e-06 2024-09-14T06:32:35.562 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:35.562 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:35.562 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:32:35.562 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.658731+0000", 2024-09-14T06:32:35.562 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.6450000000000002e-06 2024-09-14T06:32:35.562 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:35.562 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:35.562 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:32:35.562 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.711020+0000", 2024-09-14T06:32:35.562 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052288886 2024-09-14T06:32:35.562 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:35.562 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:35.563 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:32:35.563 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.711061+0000", 2024-09-14T06:32:35.563 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.1084000000000002e-05 2024-09-14T06:32:35.563 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:35.563 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:35.563 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:32:35.563 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.711193+0000", 2024-09-14T06:32:35.563 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00013194199999999999 2024-09-14T06:32:35.563 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:35.563 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:32:35.563 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:35.563 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:35.563 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:35.563 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:11482 3.1d 3:bcad7303:::rbd_data.112d92027590.0000000000000001:f [read 391168~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:32:35.564 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:25.658740+0000", 2024-09-14T06:32:35.564 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 189.827712268, 2024-09-14T06:32:35.564 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052637015000000002, 2024-09-14T06:32:35.564 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:32:35.564 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:32:35.564 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:32:35.564 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:32:35.564 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:32:35.564 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:35.564 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 11482 2024-09-14T06:32:35.564 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:35.564 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:32:35.564 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:35.564 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:32:35.565 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.658740+0000", 2024-09-14T06:32:35.565 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:35.565 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:35.565 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:35.565 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:32:35.565 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.658740+0000", 2024-09-14T06:32:35.565 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:35.565 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:35.565 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:35.565 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:32:35.565 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.658880+0000", 2024-09-14T06:32:35.565 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00014036100000000001 2024-09-14T06:32:35.565 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:35.565 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:35.565 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:32:35.566 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.658883+0000", 2024-09-14T06:32:35.566 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.3969999999999998e-06 2024-09-14T06:32:35.566 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:35.566 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:35.566 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:32:35.566 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.658890+0000", 2024-09-14T06:32:35.566 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.1500000000000002e-06 2024-09-14T06:32:35.566 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:35.566 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:35.566 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:32:35.566 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.658897+0000", 2024-09-14T06:32:35.566 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.1260000000000003e-06 2024-09-14T06:32:35.566 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:35.566 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:35.566 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:32:35.567 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.711207+0000", 2024-09-14T06:32:35.567 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052310777000000003 2024-09-14T06:32:35.567 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:35.567 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:35.567 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:32:35.567 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.711249+0000", 2024-09-14T06:32:35.567 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.1786999999999998e-05 2024-09-14T06:32:35.567 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:35.567 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:35.567 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:32:35.567 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.711377+0000", 2024-09-14T06:32:35.567 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.000127417 2024-09-14T06:32:35.567 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:35.568 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:32:35.568 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:35.568 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:35.568 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:35.568 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:11483 3.1d 3:bcad7303:::rbd_data.112d92027590.0000000000000001:f [read 401408~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:32:35.568 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:25.658906+0000", 2024-09-14T06:32:35.568 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 189.82754548400001, 2024-09-14T06:32:35.568 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052595109000000001, 2024-09-14T06:32:35.568 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:32:35.568 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:32:35.568 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:32:35.568 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:32:35.568 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:32:35.569 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:35.569 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 11483 2024-09-14T06:32:35.569 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:35.569 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:32:35.569 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:35.569 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:32:35.569 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.658906+0000", 2024-09-14T06:32:35.569 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:35.569 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:35.569 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:35.569 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:32:35.569 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.658906+0000", 2024-09-14T06:32:35.569 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:35.570 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:35.570 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:35.570 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:32:35.570 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.664985+0000", 2024-09-14T06:32:35.570 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.0060787250000000001 2024-09-14T06:32:35.570 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:35.570 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:35.570 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:32:35.570 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.664991+0000", 2024-09-14T06:32:35.570 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.7030000000000003e-06 2024-09-14T06:32:35.570 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:35.570 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:35.570 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:32:35.571 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.664999+0000", 2024-09-14T06:32:35.571 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.4430000000000003e-06 2024-09-14T06:32:35.571 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:35.571 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:35.571 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:32:35.571 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.665006+0000", 2024-09-14T06:32:35.571 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.0840000000000003e-06 2024-09-14T06:32:35.571 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:35.571 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:35.571 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:32:35.571 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.711402+0000", 2024-09-14T06:32:35.571 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046395714999999997 2024-09-14T06:32:35.571 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:35.571 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:35.572 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:32:35.572 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.711443+0000", 2024-09-14T06:32:35.572 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.0911999999999997e-05 2024-09-14T06:32:35.572 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:35.572 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:35.572 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:32:35.572 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.711501+0000", 2024-09-14T06:32:35.572 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.8526999999999997e-05 2024-09-14T06:32:35.572 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:35.572 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:32:35.572 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:35.572 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:35.573 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:35.573 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:214569 3.2 3:479633bd:::rbd_object_map.112d3bd84cb0:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e170)", 2024-09-14T06:32:35.573 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:31:34.933901+0000", 2024-09-14T06:32:35.573 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 60.552551100000002, 2024-09-14T06:32:35.573 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.68157511400000004, 2024-09-14T06:32:35.573 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:32:35.573 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:32:35.573 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:32:35.573 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:32:35.573 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:32:35.573 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:35.573 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 214569 2024-09-14T06:32:35.573 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:35.573 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:32:35.574 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:35.574 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:32:35.574 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:31:34.933901+0000", 2024-09-14T06:32:35.574 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:35.574 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:35.574 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:35.574 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:32:35.574 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:31:34.933901+0000", 2024-09-14T06:32:35.574 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:35.574 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:35.574 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:35.574 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:32:35.574 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:31:34.933902+0000", 2024-09-14T06:32:35.574 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.7439999999999999e-06 2024-09-14T06:32:35.575 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:35.575 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:35.575 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:32:35.575 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:31:34.933910+0000", 2024-09-14T06:32:35.575 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.8569999999999995e-06 2024-09-14T06:32:35.575 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:35.575 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:35.575 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:32:35.575 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:31:34.933932+0000", 2024-09-14T06:32:35.575 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.1997999999999998e-05 2024-09-14T06:32:35.575 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:35.575 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:35.575 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:32:35.575 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:31:34.933941+0000", 2024-09-14T06:32:35.575 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.7180000000000002e-06 2024-09-14T06:32:35.575 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:35.576 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:35.576 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:32:35.576 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:31:34.934013+0000", 2024-09-14T06:32:35.576 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.1761000000000006e-05 2024-09-14T06:32:35.576 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:35.576 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:35.576 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "waiting for peered", 2024-09-14T06:32:35.576 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:31:34.934022+0000", 2024-09-14T06:32:35.576 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.8740000000000001e-06 2024-09-14T06:32:35.576 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:35.576 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:35.576 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:32:35.576 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:31:35.608803+0000", 2024-09-14T06:32:35.576 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.67478164299999999 2024-09-14T06:32:35.576 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:35.576 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:35.577 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:32:35.577 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:31:35.608964+0000", 2024-09-14T06:32:35.577 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00016112900000000001 2024-09-14T06:32:35.577 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:35.577 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:35.577 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "waiting for subops from 6", 2024-09-14T06:32:35.577 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:31:35.609399+0000", 2024-09-14T06:32:35.577 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00043454000000000002 2024-09-14T06:32:35.577 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:35.577 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:35.577 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "op_commit", 2024-09-14T06:32:35.577 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:31:35.610495+0000", 2024-09-14T06:32:35.577 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.001096404 2024-09-14T06:32:35.577 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:35.577 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:35.578 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:32:35.578 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:31:35.615409+0000", 2024-09-14T06:32:35.578 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.0049133199999999997 2024-09-14T06:32:35.578 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:35.578 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:35.578 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "commit_sent", 2024-09-14T06:32:35.578 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:31:35.615444+0000", 2024-09-14T06:32:35.578 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.5457000000000003e-05 2024-09-14T06:32:35.578 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:35.578 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:35.578 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:32:35.578 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:31:35.615476+0000", 2024-09-14T06:32:35.578 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.1668999999999998e-05 2024-09-14T06:32:35.578 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:35.578 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:32:35.578 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:35.579 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:35.579 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:32:35.579 INFO:teuthology.orchestra.run.smithi138.stdout:} 2024-09-14T06:32:35.580 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-14T06:32:35.581 INFO:tasks.ceph.osd.5.smithi138.stderr:2024-09-14T06:32:35.579+0000 7fe13746f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33618) UID: 0 2024-09-14T06:32:35.610 INFO:teuthology.orchestra.run.smithi094.stderr:reweighted osd.0 to 1 (10000) 2024-09-14T06:32:35.629 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 1 1 2024-09-14T06:32:35.713 INFO:tasks.ceph.osd.0.smithi094.stderr:2024-09-14T06:32:35.711+0000 7f07c9333640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33636) UID: 0 2024-09-14T06:32:35.722 INFO:teuthology.orchestra.run.smithi138.stdout:{ 2024-09-14T06:32:35.722 INFO:teuthology.orchestra.run.smithi138.stdout: "ops": [], 2024-09-14T06:32:35.722 INFO:teuthology.orchestra.run.smithi138.stdout: "num_ops": 0 2024-09-14T06:32:35.722 INFO:teuthology.orchestra.run.smithi138.stdout:} 2024-09-14T06:32:35.731 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-14T06:32:35.742 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-14T06:32:35.767 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-14T06:32:35.787 INFO:tasks.ceph.osd.1.smithi094.stderr:2024-09-14T06:32:35.785+0000 7f328e47a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33641) UID: 0 2024-09-14T06:32:35.834 INFO:teuthology.orchestra.run.smithi138.stdout:{ 2024-09-14T06:32:35.834 INFO:teuthology.orchestra.run.smithi138.stdout: "ops": [], 2024-09-14T06:32:35.834 INFO:teuthology.orchestra.run.smithi138.stdout: "complaint_time": 30, 2024-09-14T06:32:35.834 INFO:teuthology.orchestra.run.smithi138.stdout: "num_blocked_ops": 0 2024-09-14T06:32:35.834 INFO:teuthology.orchestra.run.smithi138.stdout:} 2024-09-14T06:32:35.843 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-14T06:32:35.869 INFO:teuthology.orchestra.run.smithi094.stdout:osd.0: {} 2024-09-14T06:32:35.869 INFO:teuthology.orchestra.run.smithi094.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-14T06:32:35.886 INFO:tasks.ceph.osd.6.smithi138.stderr:2024-09-14T06:32:35.885+0000 7f8cd8849640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33620) UID: 0 2024-09-14T06:32:35.886 INFO:teuthology.orchestra.run.smithi094.stdout:osd.1: {} 2024-09-14T06:32:35.886 INFO:teuthology.orchestra.run.smithi094.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-14T06:32:35.903 INFO:teuthology.orchestra.run.smithi094.stdout:osd.2: {} 2024-09-14T06:32:35.904 INFO:teuthology.orchestra.run.smithi094.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-14T06:32:35.919 INFO:teuthology.orchestra.run.smithi094.stdout:osd.3: {} 2024-09-14T06:32:35.920 INFO:teuthology.orchestra.run.smithi094.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-14T06:32:35.936 INFO:teuthology.orchestra.run.smithi094.stdout:osd.4: {} 2024-09-14T06:32:35.936 INFO:teuthology.orchestra.run.smithi094.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-14T06:32:35.948 INFO:teuthology.orchestra.run.smithi138.stdout:{ 2024-09-14T06:32:35.948 INFO:teuthology.orchestra.run.smithi138.stdout: "size": 20, 2024-09-14T06:32:35.948 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 600, 2024-09-14T06:32:35.948 INFO:teuthology.orchestra.run.smithi138.stdout: "ops": [ 2024-09-14T06:32:35.948 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:35.948 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:98505 3.10 3.1bf56f90 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T06:32:35.948 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:30:07.045453+0000", 2024-09-14T06:32:35.948 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 148.90109566300001, 2024-09-14T06:32:35.948 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032144440000000003, 2024-09-14T06:32:35.948 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:32:35.948 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:32:35.949 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "reached pg", 2024-09-14T06:32:35.949 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:32:35.949 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:32:35.949 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:35.949 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 98505 2024-09-14T06:32:35.949 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:35.949 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:32:35.949 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:35.949 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:32:35.949 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045453+0000", 2024-09-14T06:32:35.949 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:35.949 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:35.949 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:35.949 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:32:35.950 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045453+0000", 2024-09-14T06:32:35.950 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:35.950 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:35.950 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:35.950 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:32:35.950 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045453+0000", 2024-09-14T06:32:35.950 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.36e-07 2024-09-14T06:32:35.950 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:35.950 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:35.950 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:32:35.950 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045454+0000", 2024-09-14T06:32:35.950 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.1299999999999998e-07 2024-09-14T06:32:35.950 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:35.950 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:35.951 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:32:35.951 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045457+0000", 2024-09-14T06:32:35.951 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.8559999999999998e-06 2024-09-14T06:32:35.951 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:35.951 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:35.951 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:32:35.951 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045460+0000", 2024-09-14T06:32:35.951 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.8750000000000002e-06 2024-09-14T06:32:35.951 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:35.951 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:35.951 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:32:35.951 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.077592+0000", 2024-09-14T06:32:35.951 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032131727999999998 2024-09-14T06:32:35.951 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:35.952 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:35.952 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:32:35.952 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.077597+0000", 2024-09-14T06:32:35.952 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.8319999999999996e-06 2024-09-14T06:32:35.952 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:35.952 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:32:35.952 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:35.952 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:35.952 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:35.952 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:98506 3.10 3.1bf56f90 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T06:32:35.952 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:30:07.045481+0000", 2024-09-14T06:32:35.953 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 148.90106740799999, 2024-09-14T06:32:35.953 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032142759999999999, 2024-09-14T06:32:35.953 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:32:35.953 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:32:35.953 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "reached pg", 2024-09-14T06:32:35.953 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:32:35.953 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:32:35.953 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:35.953 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 98506 2024-09-14T06:32:35.953 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:35.953 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:32:35.953 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:35.953 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:32:35.953 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045481+0000", 2024-09-14T06:32:35.954 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:35.954 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:35.954 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:35.954 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:32:35.954 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045481+0000", 2024-09-14T06:32:35.954 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:35.954 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:35.954 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:35.954 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:32:35.954 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045481+0000", 2024-09-14T06:32:35.954 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.8500000000000001e-07 2024-09-14T06:32:35.954 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:35.954 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:35.955 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:32:35.955 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045483+0000", 2024-09-14T06:32:35.955 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.8250000000000001e-06 2024-09-14T06:32:35.955 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:35.955 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:35.955 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:32:35.955 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045489+0000", 2024-09-14T06:32:35.955 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.5740000000000003e-06 2024-09-14T06:32:35.955 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:35.955 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:35.955 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:32:35.955 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045495+0000", 2024-09-14T06:32:35.955 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.7690000000000002e-06 2024-09-14T06:32:35.956 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:35.956 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:35.956 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:32:35.956 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.077619+0000", 2024-09-14T06:32:35.956 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032124621999999999 2024-09-14T06:32:35.956 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:35.956 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:35.956 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:32:35.956 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.077624+0000", 2024-09-14T06:32:35.956 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.3850000000000004e-06 2024-09-14T06:32:35.956 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:35.956 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:32:35.956 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:35.957 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:35.957 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:35.957 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:98507 3.10 3.1bf56f90 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T06:32:35.957 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:30:07.045502+0000", 2024-09-14T06:32:35.957 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 148.901045987, 2024-09-14T06:32:35.957 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032147219999999997, 2024-09-14T06:32:35.957 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:32:35.957 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:32:35.957 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "reached pg", 2024-09-14T06:32:35.957 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:32:35.957 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:32:35.957 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:35.957 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 98507 2024-09-14T06:32:35.957 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:35.958 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:32:35.958 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:35.958 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:32:35.958 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045502+0000", 2024-09-14T06:32:35.958 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:35.958 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:35.958 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:35.958 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:32:35.958 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045502+0000", 2024-09-14T06:32:35.958 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:35.958 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:35.958 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:35.958 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:32:35.958 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045503+0000", 2024-09-14T06:32:35.958 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.4700000000000002e-07 2024-09-14T06:32:35.959 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:35.959 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:35.959 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:32:35.959 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045509+0000", 2024-09-14T06:32:35.959 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.7329999999999997e-06 2024-09-14T06:32:35.959 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:35.959 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:35.959 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:32:35.959 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045515+0000", 2024-09-14T06:32:35.959 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.8030000000000002e-06 2024-09-14T06:32:35.959 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:35.959 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:35.959 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:32:35.959 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045522+0000", 2024-09-14T06:32:35.960 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.0369999999999997e-06 2024-09-14T06:32:35.960 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:35.960 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:35.960 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:32:35.960 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.077641+0000", 2024-09-14T06:32:35.960 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032119091000000002 2024-09-14T06:32:35.960 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:35.960 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:35.960 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:32:35.960 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.077650+0000", 2024-09-14T06:32:35.960 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.2090000000000005e-06 2024-09-14T06:32:35.960 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:35.960 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:32:35.960 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:35.961 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:35.961 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:35.961 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:98508 3.10 3.1bf56f90 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T06:32:35.961 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:30:07.045530+0000", 2024-09-14T06:32:35.961 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 148.90101837, 2024-09-14T06:32:35.961 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032198773999999999, 2024-09-14T06:32:35.961 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:32:35.961 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:32:35.961 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "reached pg", 2024-09-14T06:32:35.961 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:32:35.961 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:32:35.961 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:35.961 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 98508 2024-09-14T06:32:35.961 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:35.962 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:32:35.962 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:35.962 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:32:35.962 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045530+0000", 2024-09-14T06:32:35.962 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:35.962 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:35.962 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:35.962 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:32:35.962 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045530+0000", 2024-09-14T06:32:35.962 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:35.962 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:35.962 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:35.962 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:32:35.962 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045530+0000", 2024-09-14T06:32:35.963 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.3099999999999998e-07 2024-09-14T06:32:35.963 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:35.963 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:35.963 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:32:35.963 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045539+0000", 2024-09-14T06:32:35.963 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.4780000000000008e-06 2024-09-14T06:32:35.963 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:35.963 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:35.963 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:32:35.963 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045546+0000", 2024-09-14T06:32:35.963 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.3259999999999998e-06 2024-09-14T06:32:35.963 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:35.963 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:35.963 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:32:35.963 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045551+0000", 2024-09-14T06:32:35.964 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.9960000000000001e-06 2024-09-14T06:32:35.964 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:35.964 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:35.964 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:32:35.964 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.077716+0000", 2024-09-14T06:32:35.964 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032165081999999998 2024-09-14T06:32:35.964 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:35.964 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:35.964 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:32:35.964 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.077729+0000", 2024-09-14T06:32:35.964 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.2461e-05 2024-09-14T06:32:35.964 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:35.964 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:32:35.965 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:35.965 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:35.965 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:35.965 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:98509 3.10 3.1bf56f90 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T06:32:35.965 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:30:07.045558+0000", 2024-09-14T06:32:35.965 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 148.90099016900001, 2024-09-14T06:32:35.965 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032242198, 2024-09-14T06:32:35.965 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:32:35.965 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:32:35.965 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "reached pg", 2024-09-14T06:32:35.965 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:32:35.965 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:32:35.966 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:35.966 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 98509 2024-09-14T06:32:35.966 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:35.966 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:32:35.966 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:35.966 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:32:35.966 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045558+0000", 2024-09-14T06:32:35.966 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:35.966 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:35.966 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:35.966 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:32:35.966 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045558+0000", 2024-09-14T06:32:35.966 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:35.966 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:35.967 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:35.967 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:32:35.967 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045559+0000", 2024-09-14T06:32:35.967 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.27e-07 2024-09-14T06:32:35.967 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:35.967 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:35.967 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:32:35.967 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045560+0000", 2024-09-14T06:32:35.967 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.6959999999999999e-06 2024-09-14T06:32:35.967 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:35.967 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:35.967 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:32:35.968 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045567+0000", 2024-09-14T06:32:35.968 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.6900000000000003e-06 2024-09-14T06:32:35.968 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:35.968 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:35.968 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:32:35.968 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045571+0000", 2024-09-14T06:32:35.968 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.5750000000000002e-06 2024-09-14T06:32:35.968 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:35.968 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:35.968 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:32:35.968 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.077789+0000", 2024-09-14T06:32:35.968 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032217186000000002 2024-09-14T06:32:35.968 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:35.968 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:35.969 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:32:35.969 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.077800+0000", 2024-09-14T06:32:35.969 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.1623999999999999e-05 2024-09-14T06:32:35.969 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:35.969 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:32:35.969 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:35.969 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:35.969 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:35.969 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:98510 3.10 3.1bf56f90 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T06:32:35.969 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:30:07.045580+0000", 2024-09-14T06:32:35.969 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 148.900968355, 2024-09-14T06:32:35.970 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032299845000000001, 2024-09-14T06:32:35.970 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:32:35.970 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:32:35.970 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "reached pg", 2024-09-14T06:32:35.970 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:32:35.970 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:32:35.970 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:35.970 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 98510 2024-09-14T06:32:35.970 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:35.970 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:32:35.970 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:35.970 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:32:35.970 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045580+0000", 2024-09-14T06:32:35.970 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:35.971 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:35.971 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:35.971 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:32:35.971 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045580+0000", 2024-09-14T06:32:35.971 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:35.971 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:35.971 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:35.971 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:32:35.971 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045580+0000", 2024-09-14T06:32:35.971 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.0400000000000002e-07 2024-09-14T06:32:35.971 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:35.971 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:35.971 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:32:35.971 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045582+0000", 2024-09-14T06:32:35.972 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.7600000000000001e-06 2024-09-14T06:32:35.972 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:35.972 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:35.972 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:32:35.972 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045588+0000", 2024-09-14T06:32:35.972 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.5650000000000001e-06 2024-09-14T06:32:35.972 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:35.972 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:35.972 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:32:35.972 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045597+0000", 2024-09-14T06:32:35.972 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.465e-06 2024-09-14T06:32:35.972 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:35.972 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:35.972 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:32:35.972 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.077871+0000", 2024-09-14T06:32:35.973 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032273562999999998 2024-09-14T06:32:35.973 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:35.973 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:35.973 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:32:35.973 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.077880+0000", 2024-09-14T06:32:35.973 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.0880000000000001e-06 2024-09-14T06:32:35.973 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:35.973 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:32:35.973 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:35.973 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:35.973 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:35.973 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:98511 3.10 3.1bf56f90 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T06:32:35.973 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:30:07.045610+0000", 2024-09-14T06:32:35.974 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 148.900938508, 2024-09-14T06:32:35.974 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032338894, 2024-09-14T06:32:35.974 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:32:35.974 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:32:35.974 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "reached pg", 2024-09-14T06:32:35.974 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:32:35.974 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:32:35.974 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:35.974 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 98511 2024-09-14T06:32:35.974 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:35.974 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:32:35.974 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:35.974 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:32:35.974 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045610+0000", 2024-09-14T06:32:35.975 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:35.975 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:35.975 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:35.975 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:32:35.975 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045610+0000", 2024-09-14T06:32:35.975 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:35.975 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:35.975 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:35.975 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:32:35.975 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045610+0000", 2024-09-14T06:32:35.975 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.1600000000000002e-07 2024-09-14T06:32:35.975 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:35.975 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:35.975 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:32:35.976 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045612+0000", 2024-09-14T06:32:35.976 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.1150000000000001e-06 2024-09-14T06:32:35.976 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:35.976 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:35.976 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:32:35.976 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045618+0000", 2024-09-14T06:32:35.976 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.0499999999999997e-06 2024-09-14T06:32:35.976 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:35.976 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:35.976 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:32:35.976 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045623+0000", 2024-09-14T06:32:35.976 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.6770000000000003e-06 2024-09-14T06:32:35.976 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:35.976 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:35.977 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:32:35.977 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.077928+0000", 2024-09-14T06:32:35.977 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032305111999999997 2024-09-14T06:32:35.977 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:35.977 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:35.977 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:32:35.977 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.077949+0000", 2024-09-14T06:32:35.977 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.0523999999999999e-05 2024-09-14T06:32:35.977 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:35.977 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:32:35.977 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:35.977 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:35.977 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:35.977 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:98512 3.10 3.1bf56f90 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T06:32:35.978 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:30:07.045630+0000", 2024-09-14T06:32:35.978 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 148.90091850300001, 2024-09-14T06:32:35.978 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032346632, 2024-09-14T06:32:35.978 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:32:35.978 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:32:35.978 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "reached pg", 2024-09-14T06:32:35.978 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:32:35.978 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:32:35.978 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:35.978 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 98512 2024-09-14T06:32:35.978 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:35.978 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:32:35.978 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:35.978 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:32:35.978 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045630+0000", 2024-09-14T06:32:35.979 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:35.979 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:35.979 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:35.979 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:32:35.979 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045630+0000", 2024-09-14T06:32:35.979 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:35.979 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:35.979 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:35.979 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:32:35.979 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045630+0000", 2024-09-14T06:32:35.979 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.8200000000000001e-07 2024-09-14T06:32:35.979 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:35.979 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:35.979 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:32:35.979 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045632+0000", 2024-09-14T06:32:35.980 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.7430000000000001e-06 2024-09-14T06:32:35.980 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:35.980 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:35.980 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:32:35.980 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045637+0000", 2024-09-14T06:32:35.980 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.4550000000000003e-06 2024-09-14T06:32:35.980 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:35.980 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:35.980 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:32:35.980 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045644+0000", 2024-09-14T06:32:35.980 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.1330000000000001e-06 2024-09-14T06:32:35.980 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:35.980 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:35.980 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:32:35.981 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.077968+0000", 2024-09-14T06:32:35.981 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032323401000000002 2024-09-14T06:32:35.981 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:35.981 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:35.981 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:32:35.981 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.077976+0000", 2024-09-14T06:32:35.981 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.5180000000000004e-06 2024-09-14T06:32:35.981 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:35.981 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:32:35.981 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:35.981 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:35.981 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:35.981 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:98513 3.10 3.1bf56f90 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T06:32:35.981 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:30:07.045652+0000", 2024-09-14T06:32:35.982 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 148.90089593900001, 2024-09-14T06:32:35.982 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032353436999999999, 2024-09-14T06:32:35.982 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:32:35.982 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:32:35.982 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "reached pg", 2024-09-14T06:32:35.982 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:32:35.982 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:32:35.982 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:35.982 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 98513 2024-09-14T06:32:35.982 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:35.982 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:32:35.982 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:35.982 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:32:35.982 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045652+0000", 2024-09-14T06:32:35.982 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:35.983 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:35.983 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:35.983 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:32:35.983 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045652+0000", 2024-09-14T06:32:35.983 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:35.983 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:35.983 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:35.983 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:32:35.983 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045653+0000", 2024-09-14T06:32:35.983 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.3700000000000001e-07 2024-09-14T06:32:35.983 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:35.983 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:35.983 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:32:35.983 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045655+0000", 2024-09-14T06:32:35.984 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.854e-06 2024-09-14T06:32:35.984 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:35.984 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:35.984 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:32:35.984 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045661+0000", 2024-09-14T06:32:35.984 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.5509999999999996e-06 2024-09-14T06:32:35.984 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:35.984 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:35.984 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:32:35.984 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045668+0000", 2024-09-14T06:32:35.984 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.1980000000000004e-06 2024-09-14T06:32:35.984 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:35.984 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:35.985 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:32:35.985 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.077997+0000", 2024-09-14T06:32:35.985 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032328575999999998 2024-09-14T06:32:35.985 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:35.985 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:35.985 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:32:35.985 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.078006+0000", 2024-09-14T06:32:35.985 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.8210000000000008e-06 2024-09-14T06:32:35.985 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:35.985 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:32:35.985 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:35.985 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:35.985 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:35.985 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:98514 3.10 3.1bf56f90 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T06:32:35.986 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:30:07.045699+0000", 2024-09-14T06:32:35.986 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 148.900849604, 2024-09-14T06:32:35.986 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032330223999999998, 2024-09-14T06:32:35.986 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:32:35.986 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:32:35.986 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "reached pg", 2024-09-14T06:32:35.986 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:32:35.986 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:32:35.986 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:35.986 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 98514 2024-09-14T06:32:35.986 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:35.986 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:32:35.986 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:35.986 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:32:35.986 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045699+0000", 2024-09-14T06:32:35.987 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:35.987 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:35.987 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:35.987 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:32:35.987 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045699+0000", 2024-09-14T06:32:35.987 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:35.987 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:35.987 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:35.987 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:32:35.987 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045699+0000", 2024-09-14T06:32:35.987 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.3200000000000001e-07 2024-09-14T06:32:35.987 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:35.987 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:35.987 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:32:35.988 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045701+0000", 2024-09-14T06:32:35.988 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.762e-06 2024-09-14T06:32:35.988 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:35.988 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:35.988 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:32:35.988 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045705+0000", 2024-09-14T06:32:35.988 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.2359999999999998e-06 2024-09-14T06:32:35.988 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:35.988 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:35.988 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:32:35.988 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045708+0000", 2024-09-14T06:32:35.988 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.796e-06 2024-09-14T06:32:35.988 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:35.988 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:35.988 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:32:35.989 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.078023+0000", 2024-09-14T06:32:35.989 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032315213000000002 2024-09-14T06:32:35.989 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:35.989 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:35.989 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:32:35.989 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.078029+0000", 2024-09-14T06:32:35.989 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.8850000000000004e-06 2024-09-14T06:32:35.989 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:35.989 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:32:35.989 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:35.989 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:35.989 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:35.989 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:98515 3.10 3.1bf56f90 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T06:32:35.989 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:30:07.045712+0000", 2024-09-14T06:32:35.990 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 148.900836293, 2024-09-14T06:32:35.990 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032346871999999999, 2024-09-14T06:32:35.990 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:32:35.990 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:32:35.990 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "reached pg", 2024-09-14T06:32:35.990 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:32:35.990 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:32:35.990 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:35.990 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 98515 2024-09-14T06:32:35.990 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:35.990 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:32:35.990 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:35.990 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:32:35.991 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045712+0000", 2024-09-14T06:32:35.991 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:35.991 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:35.991 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:35.991 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:32:35.991 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045712+0000", 2024-09-14T06:32:35.991 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:35.991 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:35.991 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:35.991 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:32:35.991 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045712+0000", 2024-09-14T06:32:35.991 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.2100000000000001e-07 2024-09-14T06:32:35.991 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:35.991 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:35.991 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:32:35.992 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045713+0000", 2024-09-14T06:32:35.992 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.068e-06 2024-09-14T06:32:35.992 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:35.992 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:35.992 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:32:35.992 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045716+0000", 2024-09-14T06:32:35.992 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.971e-06 2024-09-14T06:32:35.992 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:35.992 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:35.992 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:32:35.992 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045719+0000", 2024-09-14T06:32:35.992 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.3879999999999998e-06 2024-09-14T06:32:35.992 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:35.992 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:35.993 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:32:35.993 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.078049+0000", 2024-09-14T06:32:35.993 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032330553999999997 2024-09-14T06:32:35.993 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:35.993 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:35.993 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:32:35.993 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.078059+0000", 2024-09-14T06:32:35.993 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.6700000000000006e-06 2024-09-14T06:32:35.993 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:35.993 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:32:35.993 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:35.993 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:35.993 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:35.994 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:98516 3.10 3.1bf56f90 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T06:32:35.994 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:30:07.045722+0000", 2024-09-14T06:32:35.994 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 148.90082654, 2024-09-14T06:32:35.994 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032374334999999997, 2024-09-14T06:32:35.994 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:32:35.994 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:32:35.994 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "reached pg", 2024-09-14T06:32:35.994 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:32:35.994 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:32:35.994 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:35.994 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 98516 2024-09-14T06:32:35.994 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:35.994 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:32:35.994 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:35.995 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:32:35.995 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045722+0000", 2024-09-14T06:32:35.995 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:35.995 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:35.995 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:35.995 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:32:35.995 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045722+0000", 2024-09-14T06:32:35.995 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:35.995 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:35.995 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:35.995 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:32:35.995 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045722+0000", 2024-09-14T06:32:35.995 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.8999999999999998e-07 2024-09-14T06:32:35.995 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:35.995 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:35.996 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:32:35.996 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045723+0000", 2024-09-14T06:32:35.996 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.0049999999999999e-06 2024-09-14T06:32:35.996 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:35.996 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:35.996 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:32:35.996 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045726+0000", 2024-09-14T06:32:35.996 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.9019999999999999e-06 2024-09-14T06:32:35.996 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:35.996 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:35.996 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:32:35.996 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045728+0000", 2024-09-14T06:32:35.996 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.3290000000000001e-06 2024-09-14T06:32:35.996 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:35.996 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:35.997 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:32:35.997 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.078090+0000", 2024-09-14T06:32:35.997 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032362123 2024-09-14T06:32:35.997 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:35.997 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:35.997 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:32:35.997 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.078096+0000", 2024-09-14T06:32:35.997 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.6860000000000003e-06 2024-09-14T06:32:35.997 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:35.997 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:32:35.997 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:35.997 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:35.997 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:35.997 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:98517 3.10 3.1bf56f90 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T06:32:35.998 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:30:07.045731+0000", 2024-09-14T06:32:35.998 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 148.90081680899999, 2024-09-14T06:32:35.998 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.03241227, 2024-09-14T06:32:35.998 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:32:35.998 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:32:35.998 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "reached pg", 2024-09-14T06:32:35.998 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:32:35.998 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:32:35.998 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:35.998 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 98517 2024-09-14T06:32:35.998 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:35.998 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:32:35.998 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:35.998 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:32:35.999 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045731+0000", 2024-09-14T06:32:35.999 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:35.999 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:35.999 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:35.999 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:32:35.999 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045731+0000", 2024-09-14T06:32:35.999 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:35.999 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:35.999 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:35.999 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:32:35.999 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045732+0000", 2024-09-14T06:32:35.999 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.0699999999999999e-07 2024-09-14T06:32:35.999 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:35.999 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:36.000 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:32:36.000 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045733+0000", 2024-09-14T06:32:36.000 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.9199999999999999e-07 2024-09-14T06:32:36.000 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:36.000 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:36.000 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:32:36.000 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045735+0000", 2024-09-14T06:32:36.000 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.8150000000000002e-06 2024-09-14T06:32:36.000 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:36.000 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:36.000 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:32:36.000 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045738+0000", 2024-09-14T06:32:36.000 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.3640000000000002e-06 2024-09-14T06:32:36.000 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:36.001 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:36.001 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:32:36.001 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.078126+0000", 2024-09-14T06:32:36.001 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032388206000000003 2024-09-14T06:32:36.001 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:36.001 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:36.001 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:32:36.001 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.078144+0000", 2024-09-14T06:32:36.001 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.7686e-05 2024-09-14T06:32:36.001 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:36.001 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:32:36.001 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:36.001 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:36.002 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:36.002 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:98518 3.10 3.1bf56f90 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T06:32:36.002 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:30:07.045741+0000", 2024-09-14T06:32:36.002 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 148.90080741400001, 2024-09-14T06:32:36.002 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032451793999999999, 2024-09-14T06:32:36.002 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:32:36.002 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:32:36.002 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "reached pg", 2024-09-14T06:32:36.002 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:32:36.002 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:32:36.002 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:36.002 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 98518 2024-09-14T06:32:36.002 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:36.002 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:32:36.002 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:36.003 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:32:36.003 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045741+0000", 2024-09-14T06:32:36.003 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:36.003 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:36.003 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:36.003 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:32:36.003 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045741+0000", 2024-09-14T06:32:36.003 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:36.003 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:36.003 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:36.003 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:32:36.003 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045741+0000", 2024-09-14T06:32:36.003 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.05e-07 2024-09-14T06:32:36.004 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:36.004 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:36.004 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:32:36.004 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045746+0000", 2024-09-14T06:32:36.004 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.2259999999999996e-06 2024-09-14T06:32:36.004 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:36.004 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:36.004 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:32:36.004 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045750+0000", 2024-09-14T06:32:36.004 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.2490000000000001e-06 2024-09-14T06:32:36.004 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:36.004 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:36.004 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:32:36.004 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045752+0000", 2024-09-14T06:32:36.005 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.3980000000000002e-06 2024-09-14T06:32:36.005 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:36.005 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:36.005 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:32:36.005 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.078184+0000", 2024-09-14T06:32:36.005 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032431593000000002 2024-09-14T06:32:36.005 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:36.005 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:36.005 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:32:36.005 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.078193+0000", 2024-09-14T06:32:36.005 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.1230000000000006e-06 2024-09-14T06:32:36.005 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:36.005 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:32:36.005 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:36.005 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:36.006 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:36.006 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:98519 3.10 3.1bf56f90 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T06:32:36.006 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:30:07.045755+0000", 2024-09-14T06:32:36.006 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 148.900792989, 2024-09-14T06:32:36.006 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032489431999999999, 2024-09-14T06:32:36.006 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:32:36.006 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:32:36.006 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "reached pg", 2024-09-14T06:32:36.006 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:32:36.006 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:32:36.006 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:36.006 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 98519 2024-09-14T06:32:36.006 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:36.006 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:32:36.006 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:36.006 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:32:36.007 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045755+0000", 2024-09-14T06:32:36.007 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:36.007 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:36.007 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:36.007 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:32:36.007 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045755+0000", 2024-09-14T06:32:36.007 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:36.007 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:36.007 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:36.007 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:32:36.007 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045756+0000", 2024-09-14T06:32:36.007 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.23e-07 2024-09-14T06:32:36.007 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:36.007 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:36.007 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:32:36.007 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045757+0000", 2024-09-14T06:32:36.008 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.1060000000000001e-06 2024-09-14T06:32:36.008 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:36.008 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:36.008 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:32:36.008 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045760+0000", 2024-09-14T06:32:36.008 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.988e-06 2024-09-14T06:32:36.008 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:36.008 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:36.008 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:32:36.008 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045765+0000", 2024-09-14T06:32:36.008 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.2689999999999999e-06 2024-09-14T06:32:36.008 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:36.008 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:36.009 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:32:36.009 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.078234+0000", 2024-09-14T06:32:36.009 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032468915000000001 2024-09-14T06:32:36.009 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:36.009 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:36.009 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:32:36.009 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.078245+0000", 2024-09-14T06:32:36.009 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.0930999999999999e-05 2024-09-14T06:32:36.009 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:36.009 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:32:36.009 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:36.009 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:36.009 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:36.009 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:98520 3.10 3.1bf56f90 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T06:32:36.009 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:30:07.045768+0000", 2024-09-14T06:32:36.010 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 148.90078011099999, 2024-09-14T06:32:36.010 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032531379999999999, 2024-09-14T06:32:36.010 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:32:36.010 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:32:36.010 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "reached pg", 2024-09-14T06:32:36.010 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:32:36.010 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:32:36.010 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:36.010 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 98520 2024-09-14T06:32:36.010 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:36.010 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:32:36.010 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:36.010 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:32:36.010 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045768+0000", 2024-09-14T06:32:36.011 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:36.011 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:36.011 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:36.011 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:32:36.011 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045768+0000", 2024-09-14T06:32:36.011 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:36.011 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:36.011 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:36.011 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:32:36.011 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045768+0000", 2024-09-14T06:32:36.011 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.22e-07 2024-09-14T06:32:36.011 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:36.011 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:36.011 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:32:36.012 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045773+0000", 2024-09-14T06:32:36.012 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.3470000000000001e-06 2024-09-14T06:32:36.012 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:36.012 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:36.012 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:32:36.012 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045776+0000", 2024-09-14T06:32:36.012 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.1319999999999998e-06 2024-09-14T06:32:36.012 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:36.012 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:36.012 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:32:36.012 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045779+0000", 2024-09-14T06:32:36.012 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.9749999999999999e-06 2024-09-14T06:32:36.012 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:36.013 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:36.013 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:32:36.013 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.078289+0000", 2024-09-14T06:32:36.013 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032509693999999999 2024-09-14T06:32:36.013 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:36.013 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:36.013 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:32:36.013 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.078300+0000", 2024-09-14T06:32:36.013 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.101e-05 2024-09-14T06:32:36.013 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:36.013 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:32:36.013 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:36.013 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:36.013 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:36.014 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:98521 3.10 3.1bf56f90 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T06:32:36.014 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:30:07.045782+0000", 2024-09-14T06:32:36.014 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 148.900765931, 2024-09-14T06:32:36.014 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032566109000000003, 2024-09-14T06:32:36.014 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:32:36.014 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:32:36.014 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "reached pg", 2024-09-14T06:32:36.014 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:32:36.014 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:32:36.014 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:36.014 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 98521 2024-09-14T06:32:36.014 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:36.014 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:32:36.014 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:36.015 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:32:36.015 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045782+0000", 2024-09-14T06:32:36.015 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:36.015 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:36.015 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:36.015 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:32:36.015 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045782+0000", 2024-09-14T06:32:36.015 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:36.015 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:36.015 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:36.015 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:32:36.015 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045783+0000", 2024-09-14T06:32:36.015 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.2499999999999999e-07 2024-09-14T06:32:36.015 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:36.016 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:36.016 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:32:36.016 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045784+0000", 2024-09-14T06:32:36.016 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.0109999999999999e-06 2024-09-14T06:32:36.016 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:36.016 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:36.016 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:32:36.016 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045786+0000", 2024-09-14T06:32:36.016 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.8289999999999999e-06 2024-09-14T06:32:36.016 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:36.016 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:36.016 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:32:36.016 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045793+0000", 2024-09-14T06:32:36.016 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.579e-06 2024-09-14T06:32:36.017 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:36.017 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:36.017 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:32:36.017 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.078338+0000", 2024-09-14T06:32:36.017 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032545389000000001 2024-09-14T06:32:36.017 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:36.017 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:36.017 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:32:36.017 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.078348+0000", 2024-09-14T06:32:36.017 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.0076e-05 2024-09-14T06:32:36.017 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:36.017 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:32:36.017 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:36.018 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:36.018 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:36.018 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:98522 3.10 3.1bf56f90 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T06:32:36.018 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:30:07.045796+0000", 2024-09-14T06:32:36.018 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 148.900752023, 2024-09-14T06:32:36.018 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032577846000000001, 2024-09-14T06:32:36.018 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:32:36.018 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:32:36.018 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "reached pg", 2024-09-14T06:32:36.018 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:32:36.018 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:32:36.018 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:36.018 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 98522 2024-09-14T06:32:36.018 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:36.018 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:32:36.019 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:36.019 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:32:36.019 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045796+0000", 2024-09-14T06:32:36.019 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:36.019 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:36.019 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:36.019 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:32:36.019 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045796+0000", 2024-09-14T06:32:36.019 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:36.019 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:36.019 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:36.019 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:32:36.019 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045796+0000", 2024-09-14T06:32:36.019 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.2100000000000001e-07 2024-09-14T06:32:36.019 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:36.020 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:36.020 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:32:36.020 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045797+0000", 2024-09-14T06:32:36.020 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.6899999999999996e-07 2024-09-14T06:32:36.020 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:36.020 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:36.020 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:32:36.020 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045800+0000", 2024-09-14T06:32:36.020 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.9979999999999999e-06 2024-09-14T06:32:36.020 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:36.020 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:36.020 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:32:36.020 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045803+0000", 2024-09-14T06:32:36.021 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.2850000000000001e-06 2024-09-14T06:32:36.021 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:36.021 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:36.021 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:32:36.021 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.078366+0000", 2024-09-14T06:32:36.021 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032562932000000003 2024-09-14T06:32:36.021 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:36.021 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:36.021 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:32:36.021 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.078374+0000", 2024-09-14T06:32:36.021 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.4409999999999993e-06 2024-09-14T06:32:36.021 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:36.021 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:32:36.021 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:36.021 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:36.022 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:36.022 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:98523 3.10 3.1bf56f90 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T06:32:36.022 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:30:07.045806+0000", 2024-09-14T06:32:36.022 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 148.90074247999999, 2024-09-14T06:32:36.022 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032611167000000003, 2024-09-14T06:32:36.022 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:32:36.022 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:32:36.022 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "reached pg", 2024-09-14T06:32:36.022 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:32:36.022 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:32:36.022 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:36.022 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 98523 2024-09-14T06:32:36.022 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:36.022 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:32:36.023 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:36.023 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:32:36.023 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045806+0000", 2024-09-14T06:32:36.023 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:36.023 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:36.023 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:36.023 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:32:36.023 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045806+0000", 2024-09-14T06:32:36.023 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:36.023 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:36.023 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:36.023 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:32:36.023 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045806+0000", 2024-09-14T06:32:36.023 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.2000000000000001e-07 2024-09-14T06:32:36.024 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:36.024 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:36.024 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:32:36.024 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045807+0000", 2024-09-14T06:32:36.024 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.4e-07 2024-09-14T06:32:36.024 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:36.024 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:36.024 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:32:36.024 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045810+0000", 2024-09-14T06:32:36.024 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.9229999999999998e-06 2024-09-14T06:32:36.024 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:36.024 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:36.024 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:32:36.024 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045812+0000", 2024-09-14T06:32:36.025 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.2220000000000001e-06 2024-09-14T06:32:36.025 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:36.025 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:36.025 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:32:36.025 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.078407+0000", 2024-09-14T06:32:36.025 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032594513999999998 2024-09-14T06:32:36.025 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:36.025 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:36.025 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:32:36.025 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.078417+0000", 2024-09-14T06:32:36.025 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.0348e-05 2024-09-14T06:32:36.025 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:36.025 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:32:36.025 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:36.025 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:36.026 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:36.026 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:98524 3.10 3.1bf56f90 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T06:32:36.026 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:30:07.045815+0000", 2024-09-14T06:32:36.026 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 148.90073311200001, 2024-09-14T06:32:36.026 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032643208999999999, 2024-09-14T06:32:36.026 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:32:36.026 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:32:36.026 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "reached pg", 2024-09-14T06:32:36.026 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:32:36.026 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:32:36.026 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:36.026 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 98524 2024-09-14T06:32:36.026 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:36.026 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:32:36.027 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:36.027 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:32:36.027 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045815+0000", 2024-09-14T06:32:36.027 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:36.027 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:36.027 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:36.027 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:32:36.027 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045815+0000", 2024-09-14T06:32:36.027 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:36.027 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:36.027 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:36.027 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:32:36.027 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045815+0000", 2024-09-14T06:32:36.027 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.11e-07 2024-09-14T06:32:36.027 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:36.028 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:36.028 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:32:36.028 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045816+0000", 2024-09-14T06:32:36.028 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.9099999999999991e-07 2024-09-14T06:32:36.028 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:36.028 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:36.028 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:32:36.028 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045819+0000", 2024-09-14T06:32:36.028 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.9399999999999998e-06 2024-09-14T06:32:36.028 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:36.028 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:36.028 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:32:36.028 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045822+0000", 2024-09-14T06:32:36.028 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.3470000000000001e-06 2024-09-14T06:32:36.028 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:36.028 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:36.029 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:32:36.029 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.078449+0000", 2024-09-14T06:32:36.029 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032627529000000002 2024-09-14T06:32:36.029 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:36.029 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:36.029 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:32:36.029 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.078458+0000", 2024-09-14T06:32:36.029 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.1910000000000006e-06 2024-09-14T06:32:36.029 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:36.029 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:32:36.029 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:36.029 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:36.029 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:32:36.029 INFO:teuthology.orchestra.run.smithi138.stdout:} 2024-09-14T06:32:36.030 INFO:teuthology.orchestra.run.smithi094.stdout:osd.5: {} 2024-09-14T06:32:36.031 INFO:teuthology.orchestra.run.smithi094.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-14T06:32:36.031 INFO:teuthology.orchestra.run.smithi094.stdout:osd.6: {} 2024-09-14T06:32:36.032 INFO:teuthology.orchestra.run.smithi094.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-14T06:32:36.032 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-14T06:32:36.032 INFO:teuthology.orchestra.run.smithi094.stdout:osd.7: {} 2024-09-14T06:32:36.033 INFO:teuthology.orchestra.run.smithi094.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-14T06:32:36.033 INFO:tasks.ceph.osd.3.smithi094.stderr:2024-09-14T06:32:36.029+0000 7feeeafe8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33655) UID: 0 2024-09-14T06:32:36.132 INFO:tasks.ceph.osd.2.smithi094.stderr:2024-09-14T06:32:36.130+0000 7faf3269b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33649) UID: 0 2024-09-14T06:32:36.174 INFO:teuthology.orchestra.run.smithi138.stdout:ERROR: (22) Invalid argument 2024-09-14T06:32:36.174 INFO:teuthology.orchestra.run.smithi138.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:32:36.183 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-14T06:32:36.232 INFO:tasks.ceph.osd.4.smithi138.stderr:2024-09-14T06:32:36.231+0000 7f6c653c8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33615) UID: 0 2024-09-14T06:32:36.285 INFO:teuthology.orchestra.run.smithi138.stdout:ERROR: (22) Invalid argument 2024-09-14T06:32:36.285 INFO:teuthology.orchestra.run.smithi138.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:32:36.293 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-14T06:32:36.334 INFO:tasks.ceph.osd.5.smithi138.stderr:2024-09-14T06:32:36.333+0000 7fe13746f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33618) UID: 0 2024-09-14T06:32:36.434 INFO:tasks.ceph.osd.1.smithi094.stderr:2024-09-14T06:32:36.432+0000 7f328e47a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33641) UID: 0 2024-09-14T06:32:36.436 INFO:teuthology.orchestra.run.smithi138.stdout:ERROR: (22) Invalid argument 2024-09-14T06:32:36.436 INFO:teuthology.orchestra.run.smithi138.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:32:36.445 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-14T06:32:36.534 INFO:tasks.ceph.osd.3.smithi094.stderr:2024-09-14T06:32:36.532+0000 7feeeafe8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33655) UID: 0 2024-09-14T06:32:36.550 INFO:teuthology.orchestra.run.smithi094.stdout:ERROR: (22) Invalid argument 2024-09-14T06:32:36.550 INFO:teuthology.orchestra.run.smithi094.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:32:36.559 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-14T06:32:36.612 INFO:teuthology.orchestra.run.smithi094.stderr:reweighted osd.1 to 1 (10000) 2024-09-14T06:32:36.612 INFO:teuthology.orchestra.run.smithi094.stderr:noscrub is set 2024-09-14T06:32:36.629 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 2 1 2024-09-14T06:32:36.635 INFO:tasks.ceph.osd.7.smithi138.stderr:2024-09-14T06:32:36.634+0000 7f3c6a21d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33623) UID: 0 2024-09-14T06:32:36.707 INFO:teuthology.orchestra.run.smithi094.stdout:ERROR: (22) Invalid argument 2024-09-14T06:32:36.707 INFO:teuthology.orchestra.run.smithi094.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:32:36.717 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-14T06:32:36.758 INFO:tasks.ceph.osd.2.smithi094.stderr:2024-09-14T06:32:36.756+0000 7faf3269b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33649) UID: 0 2024-09-14T06:32:36.837 INFO:tasks.ceph.osd.2.smithi094.stderr:2024-09-14T06:32:36.835+0000 7faf3269b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33649) UID: 0 2024-09-14T06:32:36.862 INFO:teuthology.orchestra.run.smithi094.stdout:ERROR: (22) Invalid argument 2024-09-14T06:32:36.862 INFO:teuthology.orchestra.run.smithi094.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:32:36.871 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-14T06:32:36.955 INFO:tasks.ceph.osd.3.smithi094.stderr:2024-09-14T06:32:36.953+0000 7feeeafe8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33655) UID: 0 2024-09-14T06:32:37.014 INFO:teuthology.orchestra.run.smithi094.stdout:ERROR: (22) Invalid argument 2024-09-14T06:32:37.014 INFO:teuthology.orchestra.run.smithi094.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:32:37.023 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-14T06:32:37.064 INFO:tasks.ceph.osd.3.smithi094.stderr:2024-09-14T06:32:37.062+0000 7feeeafe8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33655) UID: 0 2024-09-14T06:32:37.140 INFO:tasks.ceph.osd.2.smithi094.stderr:2024-09-14T06:32:37.138+0000 7faf3269b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33649) UID: 0 2024-09-14T06:32:37.166 INFO:teuthology.orchestra.run.smithi094.stdout:ERROR: (22) Invalid argument 2024-09-14T06:32:37.166 INFO:teuthology.orchestra.run.smithi094.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:32:37.175 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-14T06:32:37.241 INFO:tasks.ceph.osd.4.smithi138.stderr:2024-09-14T06:32:37.240+0000 7f6c653c8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33615) UID: 0 2024-09-14T06:32:37.320 INFO:teuthology.orchestra.run.smithi094.stdout:ERROR: (22) Invalid argument 2024-09-14T06:32:37.320 INFO:teuthology.orchestra.run.smithi094.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:32:37.328 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-14T06:32:37.342 INFO:tasks.ceph.osd.6.smithi138.stderr:2024-09-14T06:32:37.341+0000 7f8cd8849640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33620) UID: 0 2024-09-14T06:32:37.431 INFO:teuthology.orchestra.run.smithi094.stdout:ERROR: (22) Invalid argument 2024-09-14T06:32:37.431 INFO:teuthology.orchestra.run.smithi094.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:32:37.440 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-14T06:32:37.443 INFO:tasks.ceph.osd.1.smithi094.stderr:2024-09-14T06:32:37.441+0000 7f328e47a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33641) UID: 0 2024-09-14T06:32:37.543 INFO:teuthology.orchestra.run.smithi094.stdout:ERROR: (22) Invalid argument 2024-09-14T06:32:37.543 INFO:teuthology.orchestra.run.smithi094.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:32:37.544 INFO:tasks.ceph.osd.2.smithi094.stderr:2024-09-14T06:32:37.541+0000 7faf3269b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33649) UID: 0 2024-09-14T06:32:37.551 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-14T06:32:37.616 INFO:teuthology.orchestra.run.smithi094.stderr:reweighted osd.2 to 1 (10000) 2024-09-14T06:32:37.636 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 4 1 2024-09-14T06:32:37.644 INFO:tasks.ceph.osd.6.smithi138.stderr:2024-09-14T06:32:37.643+0000 7f8cd8849640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33620) UID: 0 2024-09-14T06:32:37.710 INFO:teuthology.orchestra.run.smithi094.stdout:ERROR: (22) Invalid argument 2024-09-14T06:32:37.711 INFO:teuthology.orchestra.run.smithi094.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:32:37.721 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-14T06:32:37.762 INFO:tasks.ceph.osd.0.smithi094.stderr:2024-09-14T06:32:37.760+0000 7f07c9333640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33636) UID: 0 2024-09-14T06:32:37.846 INFO:tasks.ceph.osd.2.smithi094.stderr:2024-09-14T06:32:37.844+0000 7faf3269b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33649) UID: 0 2024-09-14T06:32:37.883 INFO:teuthology.orchestra.run.smithi094.stdout:ERROR: (22) Invalid argument 2024-09-14T06:32:37.883 INFO:teuthology.orchestra.run.smithi094.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:32:37.894 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-14T06:32:37.978 INFO:tasks.ceph.osd.1.smithi094.stderr:2024-09-14T06:32:37.976+0000 7f328e47a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33641) UID: 0 2024-09-14T06:32:38.035 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-14T06:32:38.047 INFO:tasks.ceph.osd.4.smithi138.stderr:2024-09-14T06:32:38.046+0000 7f6c653c8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33615) UID: 0 2024-09-14T06:32:38.055 INFO:teuthology.orchestra.run.smithi094.stdout:ERROR: (22) Invalid argument 2024-09-14T06:32:38.055 INFO:teuthology.orchestra.run.smithi094.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:32:38.066 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-14T06:32:38.150 INFO:tasks.ceph.osd.2.smithi094.stderr:2024-09-14T06:32:38.148+0000 7faf3269b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33649) UID: 0 2024-09-14T06:32:38.165 INFO:teuthology.orchestra.run.smithi094.stdout:osd.0: {} 2024-09-14T06:32:38.165 INFO:teuthology.orchestra.run.smithi094.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-14T06:32:38.226 INFO:teuthology.orchestra.run.smithi094.stdout:ERROR: (22) Invalid argument 2024-09-14T06:32:38.226 INFO:teuthology.orchestra.run.smithi094.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:32:38.236 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-14T06:32:38.278 INFO:tasks.ceph.osd.0.smithi094.stderr:2024-09-14T06:32:38.276+0000 7f07c9333640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33636) UID: 0 2024-09-14T06:32:38.351 INFO:tasks.ceph.osd.5.smithi138.stderr:2024-09-14T06:32:38.350+0000 7fe13746f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33618) UID: 0 2024-09-14T06:32:38.354 INFO:teuthology.orchestra.run.smithi138.stdout:ERROR: (22) Invalid argument 2024-09-14T06:32:38.354 INFO:teuthology.orchestra.run.smithi138.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:32:38.364 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-14T06:32:38.388 INFO:teuthology.orchestra.run.smithi094.stdout:osd.1: {} 2024-09-14T06:32:38.388 INFO:teuthology.orchestra.run.smithi094.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-14T06:32:38.409 INFO:teuthology.orchestra.run.smithi094.stdout:osd.2: {} 2024-09-14T06:32:38.409 INFO:teuthology.orchestra.run.smithi094.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-14T06:32:38.425 INFO:teuthology.orchestra.run.smithi094.stdout:osd.3: {} 2024-09-14T06:32:38.425 INFO:teuthology.orchestra.run.smithi094.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-14T06:32:38.444 INFO:teuthology.orchestra.run.smithi094.stdout:osd.4: {} 2024-09-14T06:32:38.445 INFO:teuthology.orchestra.run.smithi094.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-14T06:32:38.452 INFO:tasks.ceph.osd.1.smithi094.stderr:2024-09-14T06:32:38.450+0000 7f328e47a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33641) UID: 0 2024-09-14T06:32:38.462 INFO:teuthology.orchestra.run.smithi094.stdout:osd.5: {} 2024-09-14T06:32:38.462 INFO:teuthology.orchestra.run.smithi094.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-14T06:32:38.478 INFO:teuthology.orchestra.run.smithi094.stdout:osd.6: {} 2024-09-14T06:32:38.479 INFO:teuthology.orchestra.run.smithi094.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-14T06:32:38.495 INFO:teuthology.orchestra.run.smithi094.stdout:osd.7: {} 2024-09-14T06:32:38.495 INFO:teuthology.orchestra.run.smithi094.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-14T06:32:38.509 INFO:teuthology.orchestra.run.smithi138.stdout:{ 2024-09-14T06:32:38.509 INFO:teuthology.orchestra.run.smithi138.stdout: "ops": [], 2024-09-14T06:32:38.509 INFO:teuthology.orchestra.run.smithi138.stdout: "complaint_time": 30, 2024-09-14T06:32:38.509 INFO:teuthology.orchestra.run.smithi138.stdout: "num_blocked_ops": 0 2024-09-14T06:32:38.510 INFO:teuthology.orchestra.run.smithi138.stdout:} 2024-09-14T06:32:38.518 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-14T06:32:38.553 INFO:tasks.ceph.osd.1.smithi094.stderr:2024-09-14T06:32:38.551+0000 7f328e47a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33641) UID: 0 2024-09-14T06:32:38.622 INFO:teuthology.orchestra.run.smithi138.stdout:{ 2024-09-14T06:32:38.622 INFO:teuthology.orchestra.run.smithi138.stdout: "size": 20, 2024-09-14T06:32:38.623 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 600, 2024-09-14T06:32:38.623 INFO:teuthology.orchestra.run.smithi138.stdout: "ops": [ 2024-09-14T06:32:38.623 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:38.623 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:19328 3.3 3:c34e5e51:::rbd_data.112d92027590.0000000000000024:f [read 1926144~1024 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:32:38.623 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:27.059768+0000", 2024-09-14T06:32:38.623 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 191.560412579, 2024-09-14T06:32:38.623 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046369147999999999, 2024-09-14T06:32:38.623 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:32:38.623 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:32:38.623 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:32:38.623 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:32:38.623 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:32:38.623 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:38.624 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 19328 2024-09-14T06:32:38.624 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:38.624 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:32:38.624 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:38.624 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:32:38.624 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.059768+0000", 2024-09-14T06:32:38.624 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:38.624 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:38.624 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:38.624 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:32:38.624 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.059768+0000", 2024-09-14T06:32:38.624 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:38.624 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:38.624 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:38.625 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:32:38.625 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.059874+0000", 2024-09-14T06:32:38.625 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00010551400000000001 2024-09-14T06:32:38.625 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:38.625 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:38.625 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:32:38.625 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.059877+0000", 2024-09-14T06:32:38.625 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.8279999999999999e-06 2024-09-14T06:32:38.625 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:38.625 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:38.625 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:32:38.625 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.059884+0000", 2024-09-14T06:32:38.626 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.6389999999999992e-06 2024-09-14T06:32:38.626 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:38.626 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:38.626 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:32:38.626 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.059890+0000", 2024-09-14T06:32:38.626 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.49e-06 2024-09-14T06:32:38.626 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:38.626 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:38.626 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:32:38.626 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.105804+0000", 2024-09-14T06:32:38.626 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.045914242000000001 2024-09-14T06:32:38.626 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:38.626 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:38.626 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:32:38.626 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.105886+0000", 2024-09-14T06:32:38.627 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.2305999999999994e-05 2024-09-14T06:32:38.627 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:38.627 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:38.627 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:32:38.627 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.106138+0000", 2024-09-14T06:32:38.627 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.000251129 2024-09-14T06:32:38.627 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:38.627 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:32:38.627 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:38.627 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:38.628 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:38.628 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:19329 3.3 3:c34e5e51:::rbd_data.112d92027590.0000000000000024:f [read 1936384~1024 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:32:38.628 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:27.059899+0000", 2024-09-14T06:32:38.628 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 191.56028205600001, 2024-09-14T06:32:38.628 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046572122000000001, 2024-09-14T06:32:38.628 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:32:38.628 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:32:38.628 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:32:38.628 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:32:38.628 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:32:38.628 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:38.628 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 19329 2024-09-14T06:32:38.628 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:38.628 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:32:38.629 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:38.629 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:32:38.629 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.059899+0000", 2024-09-14T06:32:38.629 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:38.629 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:38.629 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:38.629 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:32:38.629 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.059899+0000", 2024-09-14T06:32:38.629 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:38.629 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:38.629 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:38.629 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:32:38.630 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062501+0000", 2024-09-14T06:32:38.630 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.0026016419999999999 2024-09-14T06:32:38.630 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:38.630 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:38.630 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:32:38.630 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062514+0000", 2024-09-14T06:32:38.630 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.2947e-05 2024-09-14T06:32:38.630 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:38.630 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:38.630 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:32:38.630 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062523+0000", 2024-09-14T06:32:38.630 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.5519999999999993e-06 2024-09-14T06:32:38.630 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:38.630 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:38.631 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:32:38.631 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062534+0000", 2024-09-14T06:32:38.631 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.1416e-05 2024-09-14T06:32:38.631 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:38.631 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:38.631 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:32:38.631 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.106179+0000", 2024-09-14T06:32:38.631 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.043644049999999997 2024-09-14T06:32:38.631 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:38.631 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:38.631 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:32:38.631 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.106262+0000", 2024-09-14T06:32:38.631 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.3178999999999998e-05 2024-09-14T06:32:38.631 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:38.632 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:38.632 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:32:38.632 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.106471+0000", 2024-09-14T06:32:38.632 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00020933599999999999 2024-09-14T06:32:38.632 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:38.632 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:32:38.632 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:38.632 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:38.632 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:38.632 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:19351 3.3 3:c34e5e51:::rbd_data.112d92027590.0000000000000024:f [read 1159168~1024 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:32:38.632 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:27.062766+0000", 2024-09-14T06:32:38.632 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 191.55741483099999, 2024-09-14T06:32:38.632 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046099454999999998, 2024-09-14T06:32:38.632 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:32:38.633 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:32:38.633 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:32:38.633 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:32:38.633 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:32:38.633 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:38.633 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 19351 2024-09-14T06:32:38.633 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:38.633 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:32:38.633 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:38.633 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:32:38.633 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062766+0000", 2024-09-14T06:32:38.633 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:38.633 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:38.633 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:38.634 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:32:38.634 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062766+0000", 2024-09-14T06:32:38.634 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:38.634 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:38.634 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:38.634 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:32:38.634 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062766+0000", 2024-09-14T06:32:38.634 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.6e-07 2024-09-14T06:32:38.634 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:38.634 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:38.634 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:32:38.634 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062767+0000", 2024-09-14T06:32:38.634 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.0070000000000001e-06 2024-09-14T06:32:38.634 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:38.634 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:38.635 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:32:38.635 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062785+0000", 2024-09-14T06:32:38.635 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.7884999999999999e-05 2024-09-14T06:32:38.635 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:38.635 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:38.635 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:32:38.635 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062790+0000", 2024-09-14T06:32:38.635 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.0900000000000004e-06 2024-09-14T06:32:38.635 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:38.635 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:38.635 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:32:38.635 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.108735+0000", 2024-09-14T06:32:38.636 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.045944411999999997 2024-09-14T06:32:38.636 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:38.636 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:38.636 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:32:38.636 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.108809+0000", 2024-09-14T06:32:38.636 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.4157000000000002e-05 2024-09-14T06:32:38.636 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:38.636 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:38.636 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:32:38.636 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.108866+0000", 2024-09-14T06:32:38.636 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.6644000000000002e-05 2024-09-14T06:32:38.636 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:38.636 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:32:38.636 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:38.636 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:38.637 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:38.637 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:19352 3.3 3:c34e5e51:::rbd_data.112d92027590.0000000000000024:f [read 1169408~1024 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:32:38.637 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:27.062798+0000", 2024-09-14T06:32:38.637 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 191.55738342000001, 2024-09-14T06:32:38.637 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046255900000000003, 2024-09-14T06:32:38.637 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:32:38.637 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:32:38.637 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:32:38.637 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:32:38.637 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:32:38.637 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:38.637 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 19352 2024-09-14T06:32:38.638 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:38.638 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:32:38.638 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:38.638 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:32:38.638 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062798+0000", 2024-09-14T06:32:38.638 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:38.638 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:38.638 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:38.638 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:32:38.638 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062798+0000", 2024-09-14T06:32:38.638 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:38.638 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:38.638 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:38.638 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:32:38.638 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062798+0000", 2024-09-14T06:32:38.639 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.37e-07 2024-09-14T06:32:38.639 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:38.639 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:38.639 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:32:38.639 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062800+0000", 2024-09-14T06:32:38.639 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.0269999999999998e-06 2024-09-14T06:32:38.639 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:38.639 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:38.639 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:32:38.639 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062806+0000", 2024-09-14T06:32:38.639 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.6690000000000003e-06 2024-09-14T06:32:38.639 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:38.639 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:38.640 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:32:38.640 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062810+0000", 2024-09-14T06:32:38.640 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.3449999999999999e-06 2024-09-14T06:32:38.640 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:38.640 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:38.640 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:32:38.640 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.108892+0000", 2024-09-14T06:32:38.640 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046081715000000002 2024-09-14T06:32:38.640 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:38.640 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:38.641 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:32:38.641 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.108940+0000", 2024-09-14T06:32:38.641 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.7815999999999998e-05 2024-09-14T06:32:38.641 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:38.641 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:38.641 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:32:38.641 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.109053+0000", 2024-09-14T06:32:38.641 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.000113791 2024-09-14T06:32:38.641 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:38.641 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:32:38.641 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:38.641 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:38.641 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:38.641 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:19353 3.3 3:c34e5e51:::rbd_data.112d92027590.0000000000000024:f [read 1179648~1024 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:32:38.642 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:27.062816+0000", 2024-09-14T06:32:38.642 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 191.557364901, 2024-09-14T06:32:38.642 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046481256999999998, 2024-09-14T06:32:38.642 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:32:38.642 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:32:38.642 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:32:38.642 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:32:38.642 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:32:38.642 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:38.642 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 19353 2024-09-14T06:32:38.642 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:38.642 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:32:38.642 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:38.642 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:32:38.643 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062816+0000", 2024-09-14T06:32:38.643 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:38.643 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:38.643 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:38.643 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:32:38.643 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062816+0000", 2024-09-14T06:32:38.643 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:38.643 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:38.643 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:38.643 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:32:38.643 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062816+0000", 2024-09-14T06:32:38.643 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.2899999999999999e-07 2024-09-14T06:32:38.643 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:38.643 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:38.644 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:32:38.644 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062818+0000", 2024-09-14T06:32:38.644 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.699e-06 2024-09-14T06:32:38.644 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:38.644 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:38.644 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:32:38.644 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062825+0000", 2024-09-14T06:32:38.644 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.878e-06 2024-09-14T06:32:38.644 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:38.644 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:38.644 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:32:38.644 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062830+0000", 2024-09-14T06:32:38.644 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.5410000000000002e-06 2024-09-14T06:32:38.644 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:38.645 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:38.645 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:32:38.645 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.109096+0000", 2024-09-14T06:32:38.645 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046266517 2024-09-14T06:32:38.645 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:38.645 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:38.645 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:32:38.645 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.109190+0000", 2024-09-14T06:32:38.645 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.4000999999999997e-05 2024-09-14T06:32:38.645 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:38.645 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:38.645 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:32:38.645 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.109297+0000", 2024-09-14T06:32:38.645 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.000107292 2024-09-14T06:32:38.646 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:38.646 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:32:38.646 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:38.646 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:38.646 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:38.646 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:19354 3.3 3:c34e5e51:::rbd_data.112d92027590.0000000000000024:f [read 1189888~1024 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:32:38.646 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:27.062842+0000", 2024-09-14T06:32:38.646 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 191.55733944599999, 2024-09-14T06:32:38.646 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046594257, 2024-09-14T06:32:38.646 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:32:38.646 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:32:38.646 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:32:38.646 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:32:38.646 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:32:38.646 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:38.647 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 19354 2024-09-14T06:32:38.647 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:38.647 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:32:38.647 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:38.647 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:32:38.647 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062842+0000", 2024-09-14T06:32:38.647 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:38.647 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:38.647 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:38.647 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:32:38.647 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062842+0000", 2024-09-14T06:32:38.647 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:38.647 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:38.647 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:38.648 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:32:38.648 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062842+0000", 2024-09-14T06:32:38.648 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.7399999999999999e-07 2024-09-14T06:32:38.648 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:38.648 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:38.648 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:32:38.648 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062859+0000", 2024-09-14T06:32:38.648 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.7010000000000001e-05 2024-09-14T06:32:38.648 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:38.648 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:38.648 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:32:38.648 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062866+0000", 2024-09-14T06:32:38.648 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.9990000000000002e-06 2024-09-14T06:32:38.648 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:38.649 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:38.649 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:32:38.649 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062873+0000", 2024-09-14T06:32:38.649 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.8569999999999997e-06 2024-09-14T06:32:38.649 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:38.649 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:38.649 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:32:38.649 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.109335+0000", 2024-09-14T06:32:38.649 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046461943999999998 2024-09-14T06:32:38.649 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:38.649 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:38.649 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:32:38.650 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.109377+0000", 2024-09-14T06:32:38.650 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.2558999999999997e-05 2024-09-14T06:32:38.650 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:38.650 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:38.650 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:32:38.650 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.109436+0000", 2024-09-14T06:32:38.650 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.8514000000000003e-05 2024-09-14T06:32:38.650 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:38.650 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:32:38.650 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:38.651 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:38.651 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:38.651 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:19355 3.3 3:c34e5e51:::rbd_data.112d92027590.0000000000000024:f [read 1200128~1024 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:32:38.651 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:27.062884+0000", 2024-09-14T06:32:38.651 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 191.55729677299999, 2024-09-14T06:32:38.651 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046681650999999998, 2024-09-14T06:32:38.651 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:32:38.651 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:32:38.651 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:32:38.651 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:32:38.651 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:32:38.651 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:38.652 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 19355 2024-09-14T06:32:38.652 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:38.652 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:32:38.652 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:38.652 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:32:38.652 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062884+0000", 2024-09-14T06:32:38.652 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:38.652 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:38.652 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:38.652 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:32:38.652 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062884+0000", 2024-09-14T06:32:38.652 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:38.652 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:38.652 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:38.653 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:32:38.653 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062885+0000", 2024-09-14T06:32:38.653 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.8000000000000001e-07 2024-09-14T06:32:38.653 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:38.653 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:38.653 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:32:38.653 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062886+0000", 2024-09-14T06:32:38.653 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.8920000000000001e-06 2024-09-14T06:32:38.653 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:38.653 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:38.653 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:32:38.653 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062892+0000", 2024-09-14T06:32:38.653 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.84e-06 2024-09-14T06:32:38.653 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:38.653 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:38.654 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:32:38.654 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062899+0000", 2024-09-14T06:32:38.654 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.5339999999999996e-06 2024-09-14T06:32:38.654 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:38.654 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:38.654 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:32:38.654 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.109466+0000", 2024-09-14T06:32:38.654 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046566919999999998 2024-09-14T06:32:38.654 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:38.654 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:38.654 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:32:38.654 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.109510+0000", 2024-09-14T06:32:38.654 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.4153000000000002e-05 2024-09-14T06:32:38.654 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:38.654 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:38.654 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:32:38.655 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.109566+0000", 2024-09-14T06:32:38.655 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.5931999999999999e-05 2024-09-14T06:32:38.655 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:38.655 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:32:38.655 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:38.655 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:38.655 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:38.655 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:19356 3.3 3:c34e5e51:::rbd_data.112d92027590.0000000000000024:f [read 1210368~1024 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:32:38.655 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:27.062908+0000", 2024-09-14T06:32:38.655 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 191.55727333600001, 2024-09-14T06:32:38.655 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046807498000000003, 2024-09-14T06:32:38.655 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:32:38.655 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:32:38.655 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:32:38.655 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:32:38.656 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:32:38.656 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:38.656 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 19356 2024-09-14T06:32:38.656 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:38.656 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:32:38.656 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:38.656 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:32:38.656 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062908+0000", 2024-09-14T06:32:38.656 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:38.656 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:38.656 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:38.656 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:32:38.656 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062908+0000", 2024-09-14T06:32:38.656 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:38.657 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:38.657 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:38.657 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:32:38.657 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062908+0000", 2024-09-14T06:32:38.657 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.5699999999999998e-07 2024-09-14T06:32:38.657 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:38.657 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:38.657 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:32:38.657 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062910+0000", 2024-09-14T06:32:38.657 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.7710000000000001e-06 2024-09-14T06:32:38.657 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:38.657 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:38.657 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:32:38.657 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062920+0000", 2024-09-14T06:32:38.657 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.0654e-05 2024-09-14T06:32:38.658 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:38.658 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:38.658 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:32:38.658 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062931+0000", 2024-09-14T06:32:38.658 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.075e-05 2024-09-14T06:32:38.658 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:38.658 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:38.658 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:32:38.658 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.109595+0000", 2024-09-14T06:32:38.658 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046663713000000002 2024-09-14T06:32:38.658 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:38.658 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:38.658 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:32:38.659 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.109632+0000", 2024-09-14T06:32:38.659 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.7505000000000001e-05 2024-09-14T06:32:38.659 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:38.659 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:38.659 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:32:38.659 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.109715+0000", 2024-09-14T06:32:38.659 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.2748000000000001e-05 2024-09-14T06:32:38.659 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:38.659 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:32:38.659 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:38.659 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:38.659 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:38.660 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:19357 3.3 3:c34e5e51:::rbd_data.112d92027590.0000000000000024:f [read 1220608~1024 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:32:38.660 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:27.062939+0000", 2024-09-14T06:32:38.660 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 191.55724188799999, 2024-09-14T06:32:38.660 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046920508, 2024-09-14T06:32:38.660 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:32:38.660 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:32:38.660 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:32:38.660 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:32:38.660 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:32:38.660 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:38.660 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 19357 2024-09-14T06:32:38.660 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:38.660 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:32:38.661 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:38.661 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:32:38.661 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062939+0000", 2024-09-14T06:32:38.661 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:38.661 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:38.661 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:38.661 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:32:38.662 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062939+0000", 2024-09-14T06:32:38.662 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:38.662 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:38.662 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:38.662 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:32:38.662 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062940+0000", 2024-09-14T06:32:38.662 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.32e-07 2024-09-14T06:32:38.663 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:38.663 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:38.663 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:32:38.663 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062942+0000", 2024-09-14T06:32:38.663 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.1919999999999999e-06 2024-09-14T06:32:38.663 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:38.663 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:38.663 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:32:38.663 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062948+0000", 2024-09-14T06:32:38.663 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.1149999999999999e-06 2024-09-14T06:32:38.663 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:38.663 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:38.664 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:32:38.664 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062960+0000", 2024-09-14T06:32:38.664 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.1895e-05 2024-09-14T06:32:38.664 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:38.664 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:38.664 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:32:38.664 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.109704+0000", 2024-09-14T06:32:38.664 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046744584999999998 2024-09-14T06:32:38.664 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:38.664 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:38.665 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:32:38.665 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.109777+0000", 2024-09-14T06:32:38.665 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.2282999999999999e-05 2024-09-14T06:32:38.665 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:38.665 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:38.665 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:32:38.665 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.109860+0000", 2024-09-14T06:32:38.665 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.3005999999999998e-05 2024-09-14T06:32:38.665 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:38.665 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:32:38.665 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:38.665 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:38.666 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:38.666 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:19358 3.3 3:c34e5e51:::rbd_data.112d92027590.0000000000000024:f [read 1230848~1024 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:32:38.666 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:27.062967+0000", 2024-09-14T06:32:38.666 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 191.55721425799999, 2024-09-14T06:32:38.666 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047144471, 2024-09-14T06:32:38.666 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:32:38.666 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:32:38.666 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:32:38.666 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:32:38.666 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:32:38.666 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:38.666 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 19358 2024-09-14T06:32:38.666 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:38.667 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:32:38.667 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:38.667 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:32:38.667 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062967+0000", 2024-09-14T06:32:38.667 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:38.667 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:38.667 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:38.667 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:32:38.667 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062967+0000", 2024-09-14T06:32:38.667 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:38.667 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:38.667 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:38.668 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:32:38.668 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062967+0000", 2024-09-14T06:32:38.668 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.89e-07 2024-09-14T06:32:38.668 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:38.668 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:38.668 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:32:38.668 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062969+0000", 2024-09-14T06:32:38.668 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.9929999999999998e-06 2024-09-14T06:32:38.668 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:38.668 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:38.668 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:32:38.668 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062975+0000", 2024-09-14T06:32:38.668 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.9030000000000001e-06 2024-09-14T06:32:38.668 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:38.669 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:38.669 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:32:38.669 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062979+0000", 2024-09-14T06:32:38.669 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.2250000000000002e-06 2024-09-14T06:32:38.669 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:38.669 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:38.669 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:32:38.669 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.109934+0000", 2024-09-14T06:32:38.669 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046954410000000002 2024-09-14T06:32:38.669 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:38.669 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:38.669 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:32:38.669 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.110007+0000", 2024-09-14T06:32:38.670 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.3829000000000001e-05 2024-09-14T06:32:38.670 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:38.670 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:38.670 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:32:38.670 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.110111+0000", 2024-09-14T06:32:38.670 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00010372199999999999 2024-09-14T06:32:38.670 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:38.670 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:32:38.670 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:38.670 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:38.671 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:38.671 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:19359 3.3 3:c34e5e51:::rbd_data.112d92027590.0000000000000024:f [read 1241088~1024 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:32:38.671 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:27.062987+0000", 2024-09-14T06:32:38.671 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 191.557194168, 2024-09-14T06:32:38.671 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047297182, 2024-09-14T06:32:38.671 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:32:38.671 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:32:38.671 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:32:38.671 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:32:38.671 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:32:38.672 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:38.672 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 19359 2024-09-14T06:32:38.672 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:38.672 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:32:38.672 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:38.672 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:32:38.672 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062987+0000", 2024-09-14T06:32:38.672 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:38.672 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:38.672 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:38.672 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:32:38.672 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062987+0000", 2024-09-14T06:32:38.673 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:38.673 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:38.673 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:38.673 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:32:38.673 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062987+0000", 2024-09-14T06:32:38.673 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.2500000000000001e-07 2024-09-14T06:32:38.673 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:38.673 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:38.673 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:32:38.673 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062989+0000", 2024-09-14T06:32:38.674 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.6080000000000001e-06 2024-09-14T06:32:38.674 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:38.674 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:38.674 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:32:38.674 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062994+0000", 2024-09-14T06:32:38.674 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.5520000000000004e-06 2024-09-14T06:32:38.674 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:38.674 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:38.674 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:32:38.674 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062999+0000", 2024-09-14T06:32:38.674 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.3930000000000001e-06 2024-09-14T06:32:38.675 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:38.675 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:38.675 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:32:38.675 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.110142+0000", 2024-09-14T06:32:38.675 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047143154 2024-09-14T06:32:38.675 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:38.675 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:38.675 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:32:38.675 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.110219+0000", 2024-09-14T06:32:38.675 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.6737999999999996e-05 2024-09-14T06:32:38.675 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:38.675 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:38.675 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:32:38.675 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.110284+0000", 2024-09-14T06:32:38.675 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.5412000000000003e-05 2024-09-14T06:32:38.676 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:38.676 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:32:38.676 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:38.676 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:38.676 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:38.676 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:19360 3.3 3:c34e5e51:::rbd_data.112d92027590.0000000000000024:f [read 1251328~1024 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:32:38.676 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:27.063017+0000", 2024-09-14T06:32:38.676 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 191.557163707, 2024-09-14T06:32:38.676 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047453823999999999, 2024-09-14T06:32:38.676 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:32:38.676 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:32:38.676 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:32:38.676 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:32:38.677 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:32:38.677 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:38.677 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 19360 2024-09-14T06:32:38.677 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:38.677 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:32:38.677 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:38.677 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:32:38.677 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.063017+0000", 2024-09-14T06:32:38.677 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:38.677 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:38.677 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:38.677 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:32:38.677 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.063017+0000", 2024-09-14T06:32:38.678 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:38.678 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:38.678 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:38.678 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:32:38.678 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.064977+0000", 2024-09-14T06:32:38.678 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.0019597080000000001 2024-09-14T06:32:38.678 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:38.678 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:38.678 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:32:38.678 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.064987+0000", 2024-09-14T06:32:38.678 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.9620000000000005e-06 2024-09-14T06:32:38.678 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:38.678 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:38.678 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:32:38.678 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065000+0000", 2024-09-14T06:32:38.678 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.3173000000000001e-05 2024-09-14T06:32:38.679 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:38.679 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:38.679 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:32:38.679 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065005+0000", 2024-09-14T06:32:38.679 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.9960000000000001e-06 2024-09-14T06:32:38.679 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:38.679 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:38.679 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:32:38.679 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.110314+0000", 2024-09-14T06:32:38.679 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.04530899 2024-09-14T06:32:38.679 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:38.679 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:38.679 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:32:38.679 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.110398+0000", 2024-09-14T06:32:38.679 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.3539000000000001e-05 2024-09-14T06:32:38.680 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:38.680 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:38.680 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:32:38.680 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.110471+0000", 2024-09-14T06:32:38.680 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.3455999999999996e-05 2024-09-14T06:32:38.680 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:38.680 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:32:38.680 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:38.680 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:38.680 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:38.680 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:19365 3.3 3:c34e5e51:::rbd_data.112d92027590.0000000000000024:f [read 1302528~1024 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:32:38.680 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:27.065058+0000", 2024-09-14T06:32:38.680 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 191.55512303699999, 2024-09-14T06:32:38.680 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046105526000000001, 2024-09-14T06:32:38.680 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:32:38.681 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:32:38.681 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:32:38.681 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:32:38.681 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:32:38.681 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:38.681 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 19365 2024-09-14T06:32:38.681 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:38.681 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:32:38.681 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:38.681 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:32:38.681 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065058+0000", 2024-09-14T06:32:38.681 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:38.681 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:38.681 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:38.681 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:32:38.682 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065058+0000", 2024-09-14T06:32:38.682 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:38.682 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:38.682 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:38.682 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:32:38.682 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065058+0000", 2024-09-14T06:32:38.682 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.1899999999999999e-07 2024-09-14T06:32:38.682 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:38.682 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:38.682 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:32:38.682 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065059+0000", 2024-09-14T06:32:38.683 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.1299999999999998e-07 2024-09-14T06:32:38.683 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:38.683 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:38.683 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:32:38.683 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065063+0000", 2024-09-14T06:32:38.683 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.1099999999999996e-06 2024-09-14T06:32:38.683 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:38.683 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:38.683 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:32:38.683 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065070+0000", 2024-09-14T06:32:38.683 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.973e-06 2024-09-14T06:32:38.683 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:38.683 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:38.683 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:32:38.684 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.111014+0000", 2024-09-14T06:32:38.684 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.045943373000000003 2024-09-14T06:32:38.684 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:38.684 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:38.684 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:32:38.684 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.111073+0000", 2024-09-14T06:32:38.684 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.8987999999999998e-05 2024-09-14T06:32:38.684 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:38.684 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:38.684 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:32:38.684 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.111163+0000", 2024-09-14T06:32:38.684 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.0950000000000002e-05 2024-09-14T06:32:38.684 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:38.685 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:32:38.685 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:38.685 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:38.685 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:38.685 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:19366 3.3 3:c34e5e51:::rbd_data.112d92027590.0000000000000024:f [read 1312768~1024 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:32:38.685 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:27.065074+0000", 2024-09-14T06:32:38.685 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 191.555107187, 2024-09-14T06:32:38.685 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046289858000000003, 2024-09-14T06:32:38.685 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:32:38.685 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:32:38.685 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:32:38.686 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:32:38.686 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:32:38.686 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:38.686 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 19366 2024-09-14T06:32:38.686 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:38.686 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:32:38.686 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:38.686 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:32:38.686 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065074+0000", 2024-09-14T06:32:38.686 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:38.686 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:38.686 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:38.686 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:32:38.687 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065074+0000", 2024-09-14T06:32:38.687 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:38.687 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:38.687 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:38.687 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:32:38.687 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065074+0000", 2024-09-14T06:32:38.687 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.3900000000000001e-07 2024-09-14T06:32:38.687 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:38.687 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:38.687 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:32:38.687 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065078+0000", 2024-09-14T06:32:38.687 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.3229999999999999e-06 2024-09-14T06:32:38.687 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:38.688 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:38.688 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:32:38.688 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065082+0000", 2024-09-14T06:32:38.688 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.8299999999999998e-06 2024-09-14T06:32:38.688 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:38.688 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:38.688 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:32:38.688 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065085+0000", 2024-09-14T06:32:38.688 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.8660000000000002e-06 2024-09-14T06:32:38.688 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:38.688 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:38.688 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:32:38.688 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.111270+0000", 2024-09-14T06:32:38.689 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046185035999999999 2024-09-14T06:32:38.689 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:38.689 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:38.689 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:32:38.689 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.111311+0000", 2024-09-14T06:32:38.689 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.0935e-05 2024-09-14T06:32:38.689 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:38.689 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:38.689 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:32:38.689 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.111364+0000", 2024-09-14T06:32:38.689 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.2629e-05 2024-09-14T06:32:38.689 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:38.690 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:32:38.690 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:38.690 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:38.690 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:38.690 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:19367 3.3 3:c34e5e51:::rbd_data.112d92027590.0000000000000024:f [read 1323008~1024 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:32:38.690 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:27.065115+0000", 2024-09-14T06:32:38.690 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 191.55506562599999, 2024-09-14T06:32:38.690 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046407003000000002, 2024-09-14T06:32:38.690 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:32:38.690 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:32:38.690 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:32:38.690 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:32:38.691 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:32:38.691 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:38.691 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 19367 2024-09-14T06:32:38.691 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:38.691 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:32:38.691 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:38.691 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:32:38.691 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065115+0000", 2024-09-14T06:32:38.691 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:38.691 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:38.691 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:38.691 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:32:38.691 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065115+0000", 2024-09-14T06:32:38.692 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:38.692 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:38.692 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:38.692 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:32:38.692 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065116+0000", 2024-09-14T06:32:38.692 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.4999999999999998e-07 2024-09-14T06:32:38.692 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:38.692 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:38.692 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:32:38.692 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065117+0000", 2024-09-14T06:32:38.693 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.189e-06 2024-09-14T06:32:38.693 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:38.693 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:38.693 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:32:38.693 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065122+0000", 2024-09-14T06:32:38.693 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.3709999999999999e-06 2024-09-14T06:32:38.693 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:38.693 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:38.693 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:32:38.693 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065125+0000", 2024-09-14T06:32:38.693 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.8940000000000002e-06 2024-09-14T06:32:38.693 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:38.693 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:38.693 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:32:38.693 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.111402+0000", 2024-09-14T06:32:38.694 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046276645999999998 2024-09-14T06:32:38.694 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:38.694 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:38.694 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:32:38.694 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.111445+0000", 2024-09-14T06:32:38.694 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.3288000000000003e-05 2024-09-14T06:32:38.694 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:38.694 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:38.694 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:32:38.694 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.111522+0000", 2024-09-14T06:32:38.694 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.7164999999999997e-05 2024-09-14T06:32:38.694 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:38.694 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:32:38.695 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:38.695 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:38.695 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:38.695 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:19368 3.3 3:c34e5e51:::rbd_data.112d92027590.0000000000000024:f [read 1333248~1024 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:32:38.695 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:27.065130+0000", 2024-09-14T06:32:38.695 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 191.55505133599999, 2024-09-14T06:32:38.695 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046507275000000001, 2024-09-14T06:32:38.695 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:32:38.695 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:32:38.695 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:32:38.695 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:32:38.695 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:32:38.695 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:38.696 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 19368 2024-09-14T06:32:38.696 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:38.696 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:32:38.696 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:38.696 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:32:38.696 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065130+0000", 2024-09-14T06:32:38.696 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:38.696 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:38.696 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:38.696 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:32:38.697 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065130+0000", 2024-09-14T06:32:38.697 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:38.697 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:38.697 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:38.697 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:32:38.697 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065130+0000", 2024-09-14T06:32:38.697 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.0800000000000001e-07 2024-09-14T06:32:38.697 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:38.697 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:38.697 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:32:38.697 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065131+0000", 2024-09-14T06:32:38.698 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.8700000000000004e-07 2024-09-14T06:32:38.698 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:38.698 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:38.698 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:32:38.698 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065134+0000", 2024-09-14T06:32:38.698 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.1039999999999998e-06 2024-09-14T06:32:38.698 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:38.698 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:38.698 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:32:38.698 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065136+0000", 2024-09-14T06:32:38.698 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.3949999999999999e-06 2024-09-14T06:32:38.698 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:38.698 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:38.699 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:32:38.699 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.111541+0000", 2024-09-14T06:32:38.699 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046404206000000003 2024-09-14T06:32:38.699 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:38.699 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:38.699 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:32:38.699 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.111582+0000", 2024-09-14T06:32:38.700 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.1051999999999999e-05 2024-09-14T06:32:38.700 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:38.700 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:38.700 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:32:38.700 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.111637+0000", 2024-09-14T06:32:38.700 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.5322999999999999e-05 2024-09-14T06:32:38.700 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:38.701 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:32:38.701 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:38.701 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:38.701 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:38.701 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:19369 3.3 3:c34e5e51:::rbd_data.112d92027590.0000000000000024:f [read 1343488~1024 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:32:38.701 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:27.065139+0000", 2024-09-14T06:32:38.702 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 191.55504151900001, 2024-09-14T06:32:38.702 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046682169000000003, 2024-09-14T06:32:38.702 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:32:38.702 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:32:38.702 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:32:38.702 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:32:38.702 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:32:38.702 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:38.702 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 19369 2024-09-14T06:32:38.702 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:38.703 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:32:38.703 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:38.703 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:32:38.703 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065139+0000", 2024-09-14T06:32:38.703 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:38.703 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:38.704 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:38.704 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:32:38.704 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065139+0000", 2024-09-14T06:32:38.704 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:38.704 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:38.704 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:38.704 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:32:38.705 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065140+0000", 2024-09-14T06:32:38.705 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.8299999999999998e-07 2024-09-14T06:32:38.705 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:38.705 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:38.705 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:32:38.705 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065141+0000", 2024-09-14T06:32:38.705 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.3900000000000003e-07 2024-09-14T06:32:38.705 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:38.705 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:38.705 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:32:38.705 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065144+0000", 2024-09-14T06:32:38.705 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.7409999999999998e-06 2024-09-14T06:32:38.706 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:38.706 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:38.706 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:32:38.706 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065147+0000", 2024-09-14T06:32:38.706 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.4150000000000002e-06 2024-09-14T06:32:38.706 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:38.706 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:38.706 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:32:38.706 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.111673+0000", 2024-09-14T06:32:38.706 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046526004000000003 2024-09-14T06:32:38.706 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:38.706 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:38.706 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:32:38.707 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.111749+0000", 2024-09-14T06:32:38.707 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.5741999999999994e-05 2024-09-14T06:32:38.707 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:38.707 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:38.707 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:32:38.707 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.111822+0000", 2024-09-14T06:32:38.707 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.3045000000000003e-05 2024-09-14T06:32:38.707 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:38.707 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:32:38.707 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:38.707 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:38.707 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:38.708 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:19370 3.3 3:c34e5e51:::rbd_data.112d92027590.0000000000000024:f [read 1353728~1024 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:32:38.708 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:27.065150+0000", 2024-09-14T06:32:38.708 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 191.55503078999999, 2024-09-14T06:32:38.708 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046897556999999999, 2024-09-14T06:32:38.708 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:32:38.708 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:32:38.708 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:32:38.708 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:32:38.708 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:32:38.708 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:38.708 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 19370 2024-09-14T06:32:38.708 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:38.709 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:32:38.709 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:38.709 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:32:38.709 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065150+0000", 2024-09-14T06:32:38.709 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:38.709 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:38.709 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:38.709 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:32:38.709 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065150+0000", 2024-09-14T06:32:38.709 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:38.710 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:38.710 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:38.710 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:32:38.710 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065150+0000", 2024-09-14T06:32:38.710 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.0200000000000001e-07 2024-09-14T06:32:38.710 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:38.710 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:38.710 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:32:38.710 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065151+0000", 2024-09-14T06:32:38.710 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.1100000000000004e-07 2024-09-14T06:32:38.710 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:38.710 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:38.710 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:32:38.710 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065154+0000", 2024-09-14T06:32:38.710 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.9639999999999999e-06 2024-09-14T06:32:38.711 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:38.711 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:38.711 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:32:38.711 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065157+0000", 2024-09-14T06:32:38.711 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.4260000000000002e-06 2024-09-14T06:32:38.711 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:38.711 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:38.711 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:32:38.711 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.111861+0000", 2024-09-14T06:32:38.711 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046704268 2024-09-14T06:32:38.711 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:38.711 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:38.711 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:32:38.711 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.111939+0000", 2024-09-14T06:32:38.711 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.8231000000000004e-05 2024-09-14T06:32:38.712 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:38.712 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:38.712 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:32:38.712 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.112048+0000", 2024-09-14T06:32:38.712 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00010855500000000001 2024-09-14T06:32:38.712 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:38.712 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:32:38.712 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:38.712 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:38.712 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:38.712 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:19371 3.3 3:c34e5e51:::rbd_data.112d92027590.0000000000000024:f [read 1363968~1024 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:32:38.712 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:27.065161+0000", 2024-09-14T06:32:38.712 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 191.55502014000001, 2024-09-14T06:32:38.712 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047021944000000003, 2024-09-14T06:32:38.713 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:32:38.713 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:32:38.713 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:32:38.713 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:32:38.713 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:32:38.713 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:38.713 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 19371 2024-09-14T06:32:38.713 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:38.713 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:32:38.713 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:38.713 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:32:38.714 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065161+0000", 2024-09-14T06:32:38.714 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:38.714 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:38.714 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:38.714 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:32:38.714 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065161+0000", 2024-09-14T06:32:38.714 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:38.714 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:38.714 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:38.714 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:32:38.714 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065161+0000", 2024-09-14T06:32:38.714 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.0200000000000001e-07 2024-09-14T06:32:38.714 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:38.714 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:38.714 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:32:38.714 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065166+0000", 2024-09-14T06:32:38.715 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.9869999999999999e-06 2024-09-14T06:32:38.715 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:38.715 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:38.715 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:32:38.715 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065169+0000", 2024-09-14T06:32:38.715 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.3799999999999998e-06 2024-09-14T06:32:38.715 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:38.715 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:38.715 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:32:38.715 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065172+0000", 2024-09-14T06:32:38.715 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.4830000000000002e-06 2024-09-14T06:32:38.715 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:38.715 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:38.715 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:32:38.715 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.112071+0000", 2024-09-14T06:32:38.716 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046899529000000002 2024-09-14T06:32:38.716 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:38.716 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:38.716 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:32:38.716 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.112128+0000", 2024-09-14T06:32:38.716 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.6341999999999997e-05 2024-09-14T06:32:38.716 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:38.716 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:38.716 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:32:38.716 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.112183+0000", 2024-09-14T06:32:38.716 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.5021000000000001e-05 2024-09-14T06:32:38.716 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:38.716 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:32:38.716 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:38.716 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:38.717 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:38.717 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:19372 3.3 3:c34e5e51:::rbd_data.112d92027590.0000000000000024:f [read 1374208~1024 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:32:38.717 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:27.065176+0000", 2024-09-14T06:32:38.717 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 191.55500498999999, 2024-09-14T06:32:38.717 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047188889999999997, 2024-09-14T06:32:38.717 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:32:38.717 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:32:38.717 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:32:38.717 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:32:38.717 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:32:38.717 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:38.717 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 19372 2024-09-14T06:32:38.717 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:38.717 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:32:38.718 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:38.718 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:32:38.718 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065176+0000", 2024-09-14T06:32:38.718 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:38.718 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:38.718 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:38.718 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:32:38.718 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065176+0000", 2024-09-14T06:32:38.718 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:38.718 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:38.718 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:38.718 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:32:38.718 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.071117+0000", 2024-09-14T06:32:38.718 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.0059404990000000001 2024-09-14T06:32:38.719 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:38.719 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:38.719 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:32:38.719 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.071120+0000", 2024-09-14T06:32:38.719 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.4340000000000001e-06 2024-09-14T06:32:38.719 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:38.719 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:38.719 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:32:38.720 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.071129+0000", 2024-09-14T06:32:38.720 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.8389999999999996e-06 2024-09-14T06:32:38.720 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:38.720 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:38.720 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:32:38.721 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.071134+0000", 2024-09-14T06:32:38.721 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.412e-06 2024-09-14T06:32:38.721 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:38.721 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:38.721 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:32:38.721 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.112221+0000", 2024-09-14T06:32:38.721 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.041086473999999998 2024-09-14T06:32:38.722 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:38.722 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:38.722 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:32:38.722 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.112299+0000", 2024-09-14T06:32:38.722 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.8746999999999997e-05 2024-09-14T06:32:38.722 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:38.722 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:38.722 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:32:38.722 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.112365+0000", 2024-09-14T06:32:38.723 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.5485000000000001e-05 2024-09-14T06:32:38.723 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:38.723 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:32:38.723 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:38.723 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:38.723 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:32:38.724 INFO:teuthology.orchestra.run.smithi138.stdout:} 2024-09-14T06:32:38.724 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-14T06:32:38.726 INFO:tasks.ceph.osd.3.smithi094.stderr:2024-09-14T06:32:38.724+0000 7feeeafe8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33655) UID: 0 2024-09-14T06:32:38.727 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-14T06:32:38.771 INFO:teuthology.orchestra.run.smithi094.stderr:reweighted osd.4 to 1 (10000) 2024-09-14T06:32:38.789 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 5 1 2024-09-14T06:32:38.833 INFO:tasks.ceph.osd.2.smithi094.stderr:2024-09-14T06:32:38.831+0000 7faf3269b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33649) UID: 0 2024-09-14T06:32:38.838 INFO:teuthology.orchestra.run.smithi138.stdout:{ 2024-09-14T06:32:38.838 INFO:teuthology.orchestra.run.smithi138.stdout: "ops": [], 2024-09-14T06:32:38.838 INFO:teuthology.orchestra.run.smithi138.stdout: "num_ops": 0 2024-09-14T06:32:38.838 INFO:teuthology.orchestra.run.smithi138.stdout:} 2024-09-14T06:32:38.847 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-14T06:32:38.928 INFO:tasks.ceph.osd.0.smithi094.stderr:2024-09-14T06:32:38.926+0000 7f07c9333640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33636) UID: 0 2024-09-14T06:32:38.951 INFO:teuthology.orchestra.run.smithi138.stdout:{ 2024-09-14T06:32:38.951 INFO:teuthology.orchestra.run.smithi138.stdout: "ops": [], 2024-09-14T06:32:38.951 INFO:teuthology.orchestra.run.smithi138.stdout: "complaint_time": 30, 2024-09-14T06:32:38.952 INFO:teuthology.orchestra.run.smithi138.stdout: "num_blocked_ops": 0 2024-09-14T06:32:38.952 INFO:teuthology.orchestra.run.smithi138.stdout:} 2024-09-14T06:32:38.961 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-14T06:32:39.029 INFO:tasks.ceph.osd.6.smithi138.stderr:2024-09-14T06:32:39.028+0000 7f8cd8849640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33620) UID: 0 2024-09-14T06:32:39.065 INFO:teuthology.orchestra.run.smithi138.stdout:{ 2024-09-14T06:32:39.066 INFO:teuthology.orchestra.run.smithi138.stdout: "size": 20, 2024-09-14T06:32:39.066 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 600, 2024-09-14T06:32:39.066 INFO:teuthology.orchestra.run.smithi138.stdout: "ops": [ 2024-09-14T06:32:39.066 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:39.066 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:11453 3.1d 3:bcad7303:::rbd_data.112d92027590.0000000000000001:f [read 94208~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:32:39.066 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:25.654375+0000", 2024-09-14T06:32:39.066 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 193.40971620799999, 2024-09-14T06:32:39.066 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052449318000000002, 2024-09-14T06:32:39.066 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:32:39.066 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:32:39.066 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:32:39.066 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:32:39.067 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:32:39.067 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:39.067 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 11453 2024-09-14T06:32:39.067 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:39.067 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:32:39.067 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:39.067 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:32:39.067 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654375+0000", 2024-09-14T06:32:39.067 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:39.067 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:39.067 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:39.067 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:32:39.067 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654375+0000", 2024-09-14T06:32:39.067 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:39.068 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:39.068 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:39.068 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:32:39.068 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654496+0000", 2024-09-14T06:32:39.068 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00012081 2024-09-14T06:32:39.068 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:39.068 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:39.068 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:32:39.068 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654497+0000", 2024-09-14T06:32:39.068 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.8789999999999999e-06 2024-09-14T06:32:39.068 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:39.068 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:39.068 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:32:39.069 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654501+0000", 2024-09-14T06:32:39.069 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.8999999999999999e-06 2024-09-14T06:32:39.069 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:39.069 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:39.069 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:32:39.069 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654505+0000", 2024-09-14T06:32:39.069 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.2229999999999999e-06 2024-09-14T06:32:39.069 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:39.069 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:39.069 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:32:39.069 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.705917+0000", 2024-09-14T06:32:39.069 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.051412501999999999 2024-09-14T06:32:39.069 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:39.070 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:39.070 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:32:39.070 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.705999+0000", 2024-09-14T06:32:39.070 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.1551999999999994e-05 2024-09-14T06:32:39.070 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:39.070 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:39.070 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:32:39.070 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.706824+0000", 2024-09-14T06:32:39.070 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00082545199999999996 2024-09-14T06:32:39.070 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:39.070 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:32:39.070 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:39.070 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:39.070 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:39.070 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:11454 3.1d 3:bcad7303:::rbd_data.112d92027590.0000000000000001:f [read 104448~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:32:39.071 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:25.654510+0000", 2024-09-14T06:32:39.071 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 193.40958141799999, 2024-09-14T06:32:39.071 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052610713000000003, 2024-09-14T06:32:39.071 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:32:39.071 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:32:39.071 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:32:39.071 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:32:39.071 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:32:39.071 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:39.071 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 11454 2024-09-14T06:32:39.071 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:39.071 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:32:39.071 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:39.071 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:32:39.072 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654510+0000", 2024-09-14T06:32:39.072 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:39.072 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:39.072 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:39.072 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:32:39.072 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654510+0000", 2024-09-14T06:32:39.072 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:39.072 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:39.072 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:39.072 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:32:39.072 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654825+0000", 2024-09-14T06:32:39.072 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00031522400000000002 2024-09-14T06:32:39.072 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:39.073 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:39.073 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:32:39.073 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654827+0000", 2024-09-14T06:32:39.073 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.9709999999999998e-06 2024-09-14T06:32:39.073 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:39.073 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:39.073 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:32:39.073 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654832+0000", 2024-09-14T06:32:39.073 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.4099999999999999e-06 2024-09-14T06:32:39.073 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:39.073 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:39.073 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:32:39.073 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654835+0000", 2024-09-14T06:32:39.074 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.14e-06 2024-09-14T06:32:39.074 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:39.074 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:39.074 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:32:39.074 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.706851+0000", 2024-09-14T06:32:39.074 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052015761000000001 2024-09-14T06:32:39.074 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:39.074 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:39.074 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:32:39.074 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.706934+0000", 2024-09-14T06:32:39.074 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.2484999999999994e-05 2024-09-14T06:32:39.074 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:39.074 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:39.075 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:32:39.075 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.707120+0000", 2024-09-14T06:32:39.075 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00018672200000000001 2024-09-14T06:32:39.075 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:39.075 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:32:39.075 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:39.076 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:39.076 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:39.076 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:11455 3.1d 3:bcad7303:::rbd_data.112d92027590.0000000000000001:f [read 114688~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:32:39.076 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:25.654841+0000", 2024-09-14T06:32:39.076 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 193.40925008900001, 2024-09-14T06:32:39.076 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052579898, 2024-09-14T06:32:39.077 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:32:39.077 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:32:39.077 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:32:39.077 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:32:39.077 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:32:39.077 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:39.078 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 11455 2024-09-14T06:32:39.078 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:39.078 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:32:39.078 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:39.078 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:32:39.078 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654841+0000", 2024-09-14T06:32:39.079 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:39.079 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:39.079 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:39.079 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:32:39.079 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654841+0000", 2024-09-14T06:32:39.079 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:39.080 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:39.080 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:39.080 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:32:39.080 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654841+0000", 2024-09-14T06:32:39.080 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.7300000000000002e-07 2024-09-14T06:32:39.080 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:39.081 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:39.081 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:32:39.081 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654845+0000", 2024-09-14T06:32:39.081 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.0520000000000003e-06 2024-09-14T06:32:39.081 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:39.081 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:39.082 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:32:39.082 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654849+0000", 2024-09-14T06:32:39.082 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.0459999999999999e-06 2024-09-14T06:32:39.082 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:39.082 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:39.082 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:32:39.083 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654852+0000", 2024-09-14T06:32:39.083 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.9830000000000001e-06 2024-09-14T06:32:39.083 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:39.083 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:39.083 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:32:39.083 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.707159+0000", 2024-09-14T06:32:39.084 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052306826000000001 2024-09-14T06:32:39.084 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:39.084 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:39.084 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:32:39.084 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.707242+0000", 2024-09-14T06:32:39.084 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.2595000000000005e-05 2024-09-14T06:32:39.085 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:39.085 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:39.085 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:32:39.085 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.707421+0000", 2024-09-14T06:32:39.085 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.000179123 2024-09-14T06:32:39.085 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:39.086 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:32:39.086 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:39.086 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:39.086 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:39.086 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:11456 3.1d 3:bcad7303:::rbd_data.112d92027590.0000000000000001:f [read 124928~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:32:39.086 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:25.654856+0000", 2024-09-14T06:32:39.087 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 193.40923524300001, 2024-09-14T06:32:39.087 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052801580000000001, 2024-09-14T06:32:39.087 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:32:39.087 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:32:39.087 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:32:39.087 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:32:39.088 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:32:39.088 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:39.088 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 11456 2024-09-14T06:32:39.088 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:39.088 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:32:39.088 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:39.089 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:32:39.089 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654856+0000", 2024-09-14T06:32:39.089 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:39.089 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:39.089 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:39.090 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:32:39.090 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654856+0000", 2024-09-14T06:32:39.090 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:39.090 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:39.090 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:39.090 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:32:39.091 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654856+0000", 2024-09-14T06:32:39.091 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.29e-07 2024-09-14T06:32:39.091 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:39.091 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:39.091 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:32:39.091 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654857+0000", 2024-09-14T06:32:39.092 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.187e-06 2024-09-14T06:32:39.092 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:39.092 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:39.092 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:32:39.092 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654861+0000", 2024-09-14T06:32:39.093 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.3850000000000002e-06 2024-09-14T06:32:39.093 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:39.093 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:39.093 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:32:39.093 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654863+0000", 2024-09-14T06:32:39.093 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.9500000000000001e-06 2024-09-14T06:32:39.094 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:39.094 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:39.094 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:32:39.094 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.707456+0000", 2024-09-14T06:32:39.094 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052592434 2024-09-14T06:32:39.094 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:39.095 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:39.095 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:32:39.095 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.707516+0000", 2024-09-14T06:32:39.095 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.0167999999999997e-05 2024-09-14T06:32:39.095 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:39.095 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:39.096 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:32:39.096 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.707657+0000", 2024-09-14T06:32:39.096 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00014122700000000001 2024-09-14T06:32:39.096 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:39.096 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:32:39.097 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:39.097 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:39.097 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:39.097 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:11457 3.1d 3:bcad7303:::rbd_data.112d92027590.0000000000000001:f [read 135168~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:32:39.097 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:25.654867+0000", 2024-09-14T06:32:39.098 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 193.40922435799999, 2024-09-14T06:32:39.098 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052921902, 2024-09-14T06:32:39.098 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:32:39.098 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:32:39.098 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:32:39.099 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:32:39.099 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:32:39.099 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:39.099 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 11457 2024-09-14T06:32:39.099 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:39.099 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:32:39.100 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:39.100 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:32:39.100 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654867+0000", 2024-09-14T06:32:39.100 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:39.100 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:39.100 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:39.100 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:32:39.101 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654867+0000", 2024-09-14T06:32:39.101 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:39.101 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:39.101 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:39.101 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:32:39.101 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654972+0000", 2024-09-14T06:32:39.102 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00010584 2024-09-14T06:32:39.102 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:39.102 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:39.102 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:32:39.102 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654975+0000", 2024-09-14T06:32:39.103 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.1169999999999998e-06 2024-09-14T06:32:39.103 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:39.103 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:39.103 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:32:39.103 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654979+0000", 2024-09-14T06:32:39.103 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.1960000000000001e-06 2024-09-14T06:32:39.103 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:39.104 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:39.104 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:32:39.104 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654982+0000", 2024-09-14T06:32:39.104 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.2990000000000001e-06 2024-09-14T06:32:39.104 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:39.104 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:39.105 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:32:39.105 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.707684+0000", 2024-09-14T06:32:39.105 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052701979000000003 2024-09-14T06:32:39.105 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:39.105 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:39.106 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:32:39.106 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.707728+0000", 2024-09-14T06:32:39.106 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.3529999999999998e-05 2024-09-14T06:32:39.106 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:39.106 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:39.106 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:32:39.107 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.707789+0000", 2024-09-14T06:32:39.107 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.0940999999999998e-05 2024-09-14T06:32:39.107 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:39.107 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:32:39.107 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:39.108 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:39.108 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:39.108 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:11458 3.1d 3:bcad7303:::rbd_data.112d92027590.0000000000000001:f [read 145408~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:32:39.108 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:25.654987+0000", 2024-09-14T06:32:39.108 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 193.40910396000001, 2024-09-14T06:32:39.109 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.05291655, 2024-09-14T06:32:39.109 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:32:39.109 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:32:39.109 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:32:39.109 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:32:39.110 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:32:39.110 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:39.110 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 11458 2024-09-14T06:32:39.110 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:39.110 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:32:39.110 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:39.111 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:32:39.111 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654987+0000", 2024-09-14T06:32:39.111 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:39.111 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:39.111 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:39.112 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:32:39.112 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654987+0000", 2024-09-14T06:32:39.112 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:39.112 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:39.112 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:39.112 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:32:39.113 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655077+0000", 2024-09-14T06:32:39.113 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.0253000000000005e-05 2024-09-14T06:32:39.113 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:39.113 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:39.113 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:32:39.113 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655079+0000", 2024-09-14T06:32:39.114 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.9989999999999998e-06 2024-09-14T06:32:39.114 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:39.114 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:39.114 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:32:39.114 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655083+0000", 2024-09-14T06:32:39.115 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.9280000000000003e-06 2024-09-14T06:32:39.115 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:39.115 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:39.115 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:32:39.115 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655086+0000", 2024-09-14T06:32:39.115 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.056e-06 2024-09-14T06:32:39.116 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:39.116 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:39.116 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:32:39.116 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.707805+0000", 2024-09-14T06:32:39.116 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052718627999999997 2024-09-14T06:32:39.116 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:39.117 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:39.117 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:32:39.117 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.707848+0000", 2024-09-14T06:32:39.117 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.2728999999999997e-05 2024-09-14T06:32:39.117 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:39.117 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:39.118 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:32:39.118 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.707904+0000", 2024-09-14T06:32:39.118 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.5957e-05 2024-09-14T06:32:39.118 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:39.118 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:32:39.118 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:39.118 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:39.118 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:39.118 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:11459 3.1d 3:bcad7303:::rbd_data.112d92027590.0000000000000001:f [read 155648~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:32:39.118 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:25.655091+0000", 2024-09-14T06:32:39.119 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 193.40899995199999, 2024-09-14T06:32:39.119 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052944242000000002, 2024-09-14T06:32:39.119 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:32:39.119 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:32:39.119 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:32:39.119 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:32:39.119 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:32:39.119 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:39.119 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 11459 2024-09-14T06:32:39.119 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:39.119 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:32:39.120 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:39.120 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:32:39.120 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655091+0000", 2024-09-14T06:32:39.120 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:39.120 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:39.120 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:39.120 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:32:39.120 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655091+0000", 2024-09-14T06:32:39.120 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:39.120 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:39.121 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:39.121 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:32:39.121 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655237+0000", 2024-09-14T06:32:39.121 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00014570100000000001 2024-09-14T06:32:39.121 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:39.121 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:39.121 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:32:39.121 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655239+0000", 2024-09-14T06:32:39.121 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.8670000000000001e-06 2024-09-14T06:32:39.121 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:39.121 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:39.122 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:32:39.122 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655243+0000", 2024-09-14T06:32:39.122 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.9829999999999998e-06 2024-09-14T06:32:39.122 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:39.122 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:39.122 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:32:39.122 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655246+0000", 2024-09-14T06:32:39.122 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.1540000000000002e-06 2024-09-14T06:32:39.122 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:39.123 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:39.123 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:32:39.123 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.707934+0000", 2024-09-14T06:32:39.123 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052688286000000001 2024-09-14T06:32:39.123 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:39.123 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:39.123 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:32:39.123 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.707977+0000", 2024-09-14T06:32:39.123 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.3291999999999998e-05 2024-09-14T06:32:39.124 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:39.124 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:39.124 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:32:39.124 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.708035+0000", 2024-09-14T06:32:39.124 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.7958999999999999e-05 2024-09-14T06:32:39.124 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:39.124 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:32:39.124 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:39.124 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:39.124 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:39.124 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:11460 3.1d 3:bcad7303:::rbd_data.112d92027590.0000000000000001:f [read 165888~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:32:39.125 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:25.655250+0000", 2024-09-14T06:32:39.125 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 193.40884065500001, 2024-09-14T06:32:39.125 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052910698999999999, 2024-09-14T06:32:39.125 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:32:39.125 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:32:39.125 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:32:39.125 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:32:39.125 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:32:39.125 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:39.125 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 11460 2024-09-14T06:32:39.125 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:39.125 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:32:39.125 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:39.125 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:32:39.126 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655250+0000", 2024-09-14T06:32:39.126 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:39.126 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:39.126 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:39.126 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:32:39.126 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655250+0000", 2024-09-14T06:32:39.126 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:39.126 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:39.126 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:39.126 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:32:39.126 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655356+0000", 2024-09-14T06:32:39.126 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00010535900000000001 2024-09-14T06:32:39.126 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:39.126 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:39.127 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:32:39.127 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655357+0000", 2024-09-14T06:32:39.127 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.8080000000000001e-06 2024-09-14T06:32:39.127 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:39.127 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:39.127 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:32:39.127 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655361+0000", 2024-09-14T06:32:39.127 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.861e-06 2024-09-14T06:32:39.127 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:39.127 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:39.127 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:32:39.127 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655364+0000", 2024-09-14T06:32:39.127 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.1089999999999998e-06 2024-09-14T06:32:39.127 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:39.128 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:39.128 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:32:39.128 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.708065+0000", 2024-09-14T06:32:39.128 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052700595000000003 2024-09-14T06:32:39.128 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:39.128 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:39.128 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:32:39.128 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.708108+0000", 2024-09-14T06:32:39.128 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.2484999999999997e-05 2024-09-14T06:32:39.128 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:39.128 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:39.128 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:32:39.128 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.708161+0000", 2024-09-14T06:32:39.128 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.3482000000000001e-05 2024-09-14T06:32:39.129 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:39.129 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:32:39.129 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:39.129 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:39.129 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:39.129 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:11461 3.1d 3:bcad7303:::rbd_data.112d92027590.0000000000000001:f [read 176128~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:32:39.129 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:25.655370+0000", 2024-09-14T06:32:39.129 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 193.40872137299999, 2024-09-14T06:32:39.129 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052915141999999998, 2024-09-14T06:32:39.129 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:32:39.129 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:32:39.129 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:32:39.129 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:32:39.130 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:32:39.130 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:39.130 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 11461 2024-09-14T06:32:39.130 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:39.130 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:32:39.130 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:39.130 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:32:39.130 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655370+0000", 2024-09-14T06:32:39.130 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:39.130 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:39.130 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:39.130 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:32:39.130 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655370+0000", 2024-09-14T06:32:39.130 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:39.130 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:39.131 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:39.131 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:32:39.131 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655394+0000", 2024-09-14T06:32:39.131 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.4553999999999999e-05 2024-09-14T06:32:39.131 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:39.131 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:39.131 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:32:39.131 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655396+0000", 2024-09-14T06:32:39.131 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.8950000000000001e-06 2024-09-14T06:32:39.131 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:39.131 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:39.131 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:32:39.131 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655401+0000", 2024-09-14T06:32:39.131 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.0089999999999998e-06 2024-09-14T06:32:39.131 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:39.132 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:39.132 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:32:39.132 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655404+0000", 2024-09-14T06:32:39.132 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.3400000000000002e-06 2024-09-14T06:32:39.132 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:39.132 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:39.132 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:32:39.132 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.708186+0000", 2024-09-14T06:32:39.132 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052781726000000001 2024-09-14T06:32:39.132 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:39.132 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:39.132 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:32:39.132 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.708229+0000", 2024-09-14T06:32:39.132 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.2670999999999999e-05 2024-09-14T06:32:39.133 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:39.133 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:39.133 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:32:39.133 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.708285+0000", 2024-09-14T06:32:39.133 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.5946999999999998e-05 2024-09-14T06:32:39.133 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:39.133 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:32:39.133 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:39.133 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:39.133 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:39.133 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:11462 3.1d 3:bcad7303:::rbd_data.112d92027590.0000000000000001:f [read 186368~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:32:39.133 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:25.655410+0000", 2024-09-14T06:32:39.133 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 193.408680895, 2024-09-14T06:32:39.133 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052995258000000003, 2024-09-14T06:32:39.134 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:32:39.134 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:32:39.134 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:32:39.134 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:32:39.134 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:32:39.134 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:39.134 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 11462 2024-09-14T06:32:39.134 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:39.134 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:32:39.134 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:39.134 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:32:39.134 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655410+0000", 2024-09-14T06:32:39.134 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:39.134 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:39.134 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:39.135 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:32:39.135 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655410+0000", 2024-09-14T06:32:39.135 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:39.135 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:39.135 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:39.135 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:32:39.135 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655625+0000", 2024-09-14T06:32:39.135 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00021442600000000001 2024-09-14T06:32:39.135 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:39.135 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:39.135 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:32:39.135 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655626+0000", 2024-09-14T06:32:39.135 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.815e-06 2024-09-14T06:32:39.135 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:39.136 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:39.136 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:32:39.136 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655631+0000", 2024-09-14T06:32:39.136 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.651e-06 2024-09-14T06:32:39.136 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:39.136 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:39.136 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:32:39.136 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655634+0000", 2024-09-14T06:32:39.136 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.2449999999999998e-06 2024-09-14T06:32:39.136 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:39.136 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:39.136 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:32:39.136 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.708312+0000", 2024-09-14T06:32:39.137 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052677412 2024-09-14T06:32:39.137 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:39.137 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:39.137 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:32:39.137 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.708352+0000", 2024-09-14T06:32:39.137 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.0729999999999998e-05 2024-09-14T06:32:39.137 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:39.137 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:39.137 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:32:39.137 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.708405+0000", 2024-09-14T06:32:39.137 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.2979000000000002e-05 2024-09-14T06:32:39.137 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:39.137 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:32:39.137 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:39.138 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:39.138 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:39.138 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:11463 3.1d 3:bcad7303:::rbd_data.112d92027590.0000000000000001:f [read 196608~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:32:39.138 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:25.655639+0000", 2024-09-14T06:32:39.138 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 193.40845174899999, 2024-09-14T06:32:39.138 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052888859000000003, 2024-09-14T06:32:39.138 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:32:39.138 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:32:39.138 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:32:39.138 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:32:39.138 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:32:39.138 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:39.138 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 11463 2024-09-14T06:32:39.138 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:39.138 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:32:39.139 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:39.139 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:32:39.139 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655639+0000", 2024-09-14T06:32:39.139 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:39.139 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:39.139 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:39.139 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:32:39.139 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655639+0000", 2024-09-14T06:32:39.139 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:39.139 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:39.139 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:39.139 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:32:39.139 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655640+0000", 2024-09-14T06:32:39.139 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.08e-07 2024-09-14T06:32:39.140 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:39.140 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:39.140 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:32:39.140 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655641+0000", 2024-09-14T06:32:39.140 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.156e-06 2024-09-14T06:32:39.140 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:39.140 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:39.140 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:32:39.140 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655648+0000", 2024-09-14T06:32:39.140 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.8460000000000002e-06 2024-09-14T06:32:39.140 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:39.140 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:39.140 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:32:39.140 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655651+0000", 2024-09-14T06:32:39.140 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.0079999999999998e-06 2024-09-14T06:32:39.141 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:39.141 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:39.141 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:32:39.141 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.708429+0000", 2024-09-14T06:32:39.141 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052778459 2024-09-14T06:32:39.141 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:39.141 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:39.141 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:32:39.141 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.708473+0000", 2024-09-14T06:32:39.141 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.3782999999999997e-05 2024-09-14T06:32:39.141 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:39.141 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:39.141 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:32:39.142 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.708528+0000", 2024-09-14T06:32:39.142 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.5198999999999998e-05 2024-09-14T06:32:39.142 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:39.142 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:32:39.142 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:39.142 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:39.142 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:39.142 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:11464 3.1d 3:bcad7303:::rbd_data.112d92027590.0000000000000001:f [read 206848~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:32:39.142 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:25.655654+0000", 2024-09-14T06:32:39.142 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 193.40843688000001, 2024-09-14T06:32:39.142 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052981845999999999, 2024-09-14T06:32:39.142 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:32:39.142 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:32:39.142 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:32:39.142 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:32:39.142 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:32:39.143 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:39.143 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 11464 2024-09-14T06:32:39.143 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:39.143 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:32:39.143 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:39.143 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:32:39.143 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655654+0000", 2024-09-14T06:32:39.143 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:39.143 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:39.143 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:39.143 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:32:39.143 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655654+0000", 2024-09-14T06:32:39.143 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:39.143 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:39.143 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:39.144 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:32:39.144 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655946+0000", 2024-09-14T06:32:39.144 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00029148200000000003 2024-09-14T06:32:39.144 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:39.144 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:39.144 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:32:39.144 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655948+0000", 2024-09-14T06:32:39.144 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.973e-06 2024-09-14T06:32:39.144 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:39.144 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:39.144 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:32:39.144 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655953+0000", 2024-09-14T06:32:39.144 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.1240000000000004e-06 2024-09-14T06:32:39.144 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:39.145 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:39.145 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:32:39.145 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655956+0000", 2024-09-14T06:32:39.145 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.315e-06 2024-09-14T06:32:39.145 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:39.145 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:39.145 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:32:39.145 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.708543+0000", 2024-09-14T06:32:39.145 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052587416999999997 2024-09-14T06:32:39.145 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:39.145 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:39.145 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:32:39.145 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.708584+0000", 2024-09-14T06:32:39.145 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.1004000000000002e-05 2024-09-14T06:32:39.146 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:39.146 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:39.146 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:32:39.146 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.708636+0000", 2024-09-14T06:32:39.146 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.1530999999999998e-05 2024-09-14T06:32:39.146 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:39.146 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:32:39.146 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:39.146 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:39.146 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:39.146 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:11465 3.1d 3:bcad7303:::rbd_data.112d92027590.0000000000000001:f [read 217088~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:32:39.146 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:25.655961+0000", 2024-09-14T06:32:39.146 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 193.40812972800001, 2024-09-14T06:32:39.146 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052798682999999999, 2024-09-14T06:32:39.146 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:32:39.147 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:32:39.147 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:32:39.147 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:32:39.147 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:32:39.147 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:39.147 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 11465 2024-09-14T06:32:39.147 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:39.147 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:32:39.147 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:39.147 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:32:39.147 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655961+0000", 2024-09-14T06:32:39.147 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:39.147 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:39.148 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:39.148 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:32:39.148 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655961+0000", 2024-09-14T06:32:39.148 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:39.148 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:39.148 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:39.148 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:32:39.148 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.656114+0000", 2024-09-14T06:32:39.148 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.000152745 2024-09-14T06:32:39.148 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:39.148 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:39.148 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:32:39.149 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.656116+0000", 2024-09-14T06:32:39.149 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.804e-06 2024-09-14T06:32:39.149 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:39.149 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:39.149 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:32:39.149 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.656120+0000", 2024-09-14T06:32:39.149 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.4700000000000004e-06 2024-09-14T06:32:39.149 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:39.149 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:39.149 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:32:39.149 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.656123+0000", 2024-09-14T06:32:39.149 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.1269999999999999e-06 2024-09-14T06:32:39.149 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:39.149 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:39.150 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:32:39.150 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.708660+0000", 2024-09-14T06:32:39.150 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052536919000000001 2024-09-14T06:32:39.150 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:39.150 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:39.150 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:32:39.150 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.708703+0000", 2024-09-14T06:32:39.150 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.3077000000000001e-05 2024-09-14T06:32:39.150 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:39.150 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:39.150 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:32:39.150 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.708760+0000", 2024-09-14T06:32:39.150 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.6541e-05 2024-09-14T06:32:39.151 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:39.151 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:32:39.151 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:39.151 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:39.151 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:39.151 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:11466 3.1d 3:bcad7303:::rbd_data.112d92027590.0000000000000001:f [read 227328~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:32:39.151 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:25.656129+0000", 2024-09-14T06:32:39.151 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 193.40796234000001, 2024-09-14T06:32:39.151 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052750987999999999, 2024-09-14T06:32:39.151 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:32:39.151 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:32:39.151 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:32:39.151 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:32:39.151 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:32:39.151 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:39.152 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 11466 2024-09-14T06:32:39.152 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:39.152 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:32:39.152 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:39.152 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:32:39.152 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.656129+0000", 2024-09-14T06:32:39.152 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:39.152 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:39.152 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:39.152 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:32:39.152 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.656129+0000", 2024-09-14T06:32:39.152 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:39.152 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:39.153 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:39.153 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:32:39.153 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.656347+0000", 2024-09-14T06:32:39.153 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.000218054 2024-09-14T06:32:39.153 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:39.153 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:39.153 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:32:39.153 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.656349+0000", 2024-09-14T06:32:39.153 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.9199999999999998e-06 2024-09-14T06:32:39.153 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:39.153 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:39.153 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:32:39.153 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.656353+0000", 2024-09-14T06:32:39.154 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.9210000000000002e-06 2024-09-14T06:32:39.154 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:39.154 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:39.154 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:32:39.154 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.656356+0000", 2024-09-14T06:32:39.154 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.7670000000000001e-06 2024-09-14T06:32:39.154 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:39.154 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:39.154 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:32:39.154 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.708787+0000", 2024-09-14T06:32:39.154 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052430840999999999 2024-09-14T06:32:39.154 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:39.154 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:39.154 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:32:39.155 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.708827+0000", 2024-09-14T06:32:39.155 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.9538e-05 2024-09-14T06:32:39.155 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:39.155 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:39.155 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:32:39.155 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.708880+0000", 2024-09-14T06:32:39.155 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.2947e-05 2024-09-14T06:32:39.155 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:39.155 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:32:39.155 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:39.155 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:39.155 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:39.155 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:11467 3.1d 3:bcad7303:::rbd_data.112d92027590.0000000000000001:f [read 237568~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:32:39.155 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:25.656361+0000", 2024-09-14T06:32:39.155 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 193.407729692, 2024-09-14T06:32:39.156 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052643898000000001, 2024-09-14T06:32:39.156 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:32:39.156 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:32:39.156 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:32:39.156 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:32:39.156 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:32:39.156 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:39.156 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 11467 2024-09-14T06:32:39.156 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:39.156 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:32:39.156 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:39.156 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:32:39.156 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.656361+0000", 2024-09-14T06:32:39.157 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:39.157 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:39.157 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:39.157 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:32:39.157 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.656361+0000", 2024-09-14T06:32:39.157 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:39.157 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:39.157 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:39.157 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:32:39.157 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.656855+0000", 2024-09-14T06:32:39.157 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00049372000000000001 2024-09-14T06:32:39.157 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:39.157 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:39.157 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:32:39.158 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.656859+0000", 2024-09-14T06:32:39.158 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.1300000000000003e-06 2024-09-14T06:32:39.158 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:39.158 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:39.158 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:32:39.158 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.656863+0000", 2024-09-14T06:32:39.158 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.2629999999999997e-06 2024-09-14T06:32:39.158 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:39.158 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:39.158 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:32:39.158 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.656867+0000", 2024-09-14T06:32:39.158 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.3220000000000001e-06 2024-09-14T06:32:39.158 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:39.158 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:39.159 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:32:39.159 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.708903+0000", 2024-09-14T06:32:39.159 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052035855999999998 2024-09-14T06:32:39.159 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:39.159 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:39.159 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:32:39.159 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.708949+0000", 2024-09-14T06:32:39.159 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.6276999999999998e-05 2024-09-14T06:32:39.159 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:39.159 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:39.159 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:32:39.159 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.709005+0000", 2024-09-14T06:32:39.159 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.6329999999999998e-05 2024-09-14T06:32:39.159 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:39.159 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:32:39.160 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:39.160 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:39.160 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:39.160 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:11480 3.1d 3:bcad7303:::rbd_data.112d92027590.0000000000000001:f [read 370688~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:32:39.160 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:25.658563+0000", 2024-09-14T06:32:39.160 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 193.405527928, 2024-09-14T06:32:39.160 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052429164, 2024-09-14T06:32:39.160 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:32:39.160 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:32:39.160 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:32:39.160 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:32:39.160 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:32:39.160 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:39.161 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 11480 2024-09-14T06:32:39.161 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:39.161 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:32:39.161 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:39.161 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:32:39.161 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.658563+0000", 2024-09-14T06:32:39.161 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:39.161 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:39.161 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:39.161 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:32:39.161 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.658563+0000", 2024-09-14T06:32:39.161 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:39.161 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:39.161 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:39.161 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:32:39.161 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.658658+0000", 2024-09-14T06:32:39.162 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.5346999999999994e-05 2024-09-14T06:32:39.162 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:39.162 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:39.162 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:32:39.162 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.658662+0000", 2024-09-14T06:32:39.162 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.14e-06 2024-09-14T06:32:39.162 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:39.162 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:39.162 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:32:39.162 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.658669+0000", 2024-09-14T06:32:39.162 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.7379999999999994e-06 2024-09-14T06:32:39.162 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:39.162 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:39.163 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:32:39.163 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.658675+0000", 2024-09-14T06:32:39.163 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.5430000000000001e-06 2024-09-14T06:32:39.163 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:39.163 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:39.163 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:32:39.163 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.710824+0000", 2024-09-14T06:32:39.163 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052148847999999998 2024-09-14T06:32:39.163 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:39.163 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:39.164 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:32:39.164 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.710864+0000", 2024-09-14T06:32:39.164 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.9925999999999997e-05 2024-09-14T06:32:39.164 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:39.164 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:39.164 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:32:39.164 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.710992+0000", 2024-09-14T06:32:39.164 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.000128622 2024-09-14T06:32:39.164 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:39.164 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:32:39.164 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:39.164 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:39.164 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:39.164 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:11481 3.1d 3:bcad7303:::rbd_data.112d92027590.0000000000000001:f [read 380928~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:32:39.165 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:25.658684+0000", 2024-09-14T06:32:39.165 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 193.40540685799999, 2024-09-14T06:32:39.165 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052508565, 2024-09-14T06:32:39.165 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:32:39.165 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:32:39.165 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:32:39.165 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:32:39.165 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:32:39.165 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:39.165 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 11481 2024-09-14T06:32:39.165 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:39.165 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:32:39.165 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:39.166 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:32:39.166 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.658684+0000", 2024-09-14T06:32:39.166 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:39.166 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:39.166 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:39.166 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:32:39.166 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.658684+0000", 2024-09-14T06:32:39.166 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:39.166 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:39.166 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:39.166 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:32:39.166 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.658715+0000", 2024-09-14T06:32:39.166 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.0703000000000001e-05 2024-09-14T06:32:39.166 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:39.166 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:39.167 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:32:39.167 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.658718+0000", 2024-09-14T06:32:39.167 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.0709999999999999e-06 2024-09-14T06:32:39.167 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:39.167 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:39.167 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:32:39.167 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.658725+0000", 2024-09-14T06:32:39.167 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.2339999999999997e-06 2024-09-14T06:32:39.167 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:39.167 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:39.167 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:32:39.167 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.658731+0000", 2024-09-14T06:32:39.167 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.6450000000000002e-06 2024-09-14T06:32:39.167 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:39.168 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:39.168 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:32:39.168 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.711020+0000", 2024-09-14T06:32:39.168 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052288886 2024-09-14T06:32:39.168 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:39.168 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:39.168 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:32:39.168 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.711061+0000", 2024-09-14T06:32:39.168 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.1084000000000002e-05 2024-09-14T06:32:39.168 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:39.168 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:39.168 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:32:39.168 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.711193+0000", 2024-09-14T06:32:39.168 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00013194199999999999 2024-09-14T06:32:39.169 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:39.169 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:32:39.169 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:39.169 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:39.169 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:39.169 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:11482 3.1d 3:bcad7303:::rbd_data.112d92027590.0000000000000001:f [read 391168~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:32:39.169 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:25.658740+0000", 2024-09-14T06:32:39.169 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 193.40535139299999, 2024-09-14T06:32:39.169 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052637015000000002, 2024-09-14T06:32:39.169 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:32:39.169 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:32:39.169 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:32:39.169 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:32:39.170 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:32:39.170 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:39.170 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 11482 2024-09-14T06:32:39.170 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:39.170 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:32:39.170 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:39.170 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:32:39.170 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.658740+0000", 2024-09-14T06:32:39.170 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:39.170 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:39.170 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:39.170 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:32:39.170 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.658740+0000", 2024-09-14T06:32:39.170 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:39.171 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:39.171 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:39.171 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:32:39.171 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.658880+0000", 2024-09-14T06:32:39.171 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00014036100000000001 2024-09-14T06:32:39.171 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:39.171 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:39.171 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:32:39.171 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.658883+0000", 2024-09-14T06:32:39.171 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.3969999999999998e-06 2024-09-14T06:32:39.171 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:39.171 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:39.171 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:32:39.171 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.658890+0000", 2024-09-14T06:32:39.172 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.1500000000000002e-06 2024-09-14T06:32:39.172 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:39.172 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:39.172 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:32:39.172 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.658897+0000", 2024-09-14T06:32:39.172 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.1260000000000003e-06 2024-09-14T06:32:39.172 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:39.172 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:39.172 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:32:39.172 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.711207+0000", 2024-09-14T06:32:39.172 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052310777000000003 2024-09-14T06:32:39.172 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:39.172 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:39.173 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:32:39.173 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.711249+0000", 2024-09-14T06:32:39.173 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.1786999999999998e-05 2024-09-14T06:32:39.173 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:39.173 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:39.173 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:32:39.173 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.711377+0000", 2024-09-14T06:32:39.173 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.000127417 2024-09-14T06:32:39.173 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:39.173 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:32:39.173 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:39.173 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:39.173 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:39.173 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:11483 3.1d 3:bcad7303:::rbd_data.112d92027590.0000000000000001:f [read 401408~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:32:39.174 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:25.658906+0000", 2024-09-14T06:32:39.174 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 193.405184609, 2024-09-14T06:32:39.174 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052595109000000001, 2024-09-14T06:32:39.174 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:32:39.174 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:32:39.174 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:32:39.174 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:32:39.174 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:32:39.174 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:39.174 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 11483 2024-09-14T06:32:39.174 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:39.174 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:32:39.174 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:39.174 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:32:39.175 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.658906+0000", 2024-09-14T06:32:39.175 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:39.175 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:39.175 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:39.175 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:32:39.175 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.658906+0000", 2024-09-14T06:32:39.175 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:39.175 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:39.175 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:39.175 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:32:39.175 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.664985+0000", 2024-09-14T06:32:39.175 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.0060787250000000001 2024-09-14T06:32:39.175 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:39.175 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:39.176 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:32:39.176 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.664991+0000", 2024-09-14T06:32:39.176 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.7030000000000003e-06 2024-09-14T06:32:39.176 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:39.176 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:39.176 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:32:39.176 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.664999+0000", 2024-09-14T06:32:39.176 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.4430000000000003e-06 2024-09-14T06:32:39.176 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:39.176 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:39.176 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:32:39.176 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.665006+0000", 2024-09-14T06:32:39.176 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.0840000000000003e-06 2024-09-14T06:32:39.177 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:39.177 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:39.177 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:32:39.177 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.711402+0000", 2024-09-14T06:32:39.177 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046395714999999997 2024-09-14T06:32:39.177 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:39.177 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:39.177 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:32:39.177 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.711443+0000", 2024-09-14T06:32:39.177 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.0911999999999997e-05 2024-09-14T06:32:39.177 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:39.177 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:39.177 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:32:39.177 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.711501+0000", 2024-09-14T06:32:39.178 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.8526999999999997e-05 2024-09-14T06:32:39.178 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:39.178 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:32:39.178 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:39.178 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:39.178 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:39.178 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:214569 3.2 3:479633bd:::rbd_object_map.112d3bd84cb0:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e170)", 2024-09-14T06:32:39.178 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:31:34.933901+0000", 2024-09-14T06:32:39.178 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 64.130190225000007, 2024-09-14T06:32:39.178 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.68157511400000004, 2024-09-14T06:32:39.178 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:32:39.178 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:32:39.178 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:32:39.178 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:32:39.179 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:32:39.179 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:39.179 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 214569 2024-09-14T06:32:39.179 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:39.179 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:32:39.179 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:39.179 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:32:39.179 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:31:34.933901+0000", 2024-09-14T06:32:39.179 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:39.179 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:39.179 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:39.179 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:32:39.179 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:31:34.933901+0000", 2024-09-14T06:32:39.179 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:39.180 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:39.180 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:39.180 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:32:39.180 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:31:34.933902+0000", 2024-09-14T06:32:39.180 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.7439999999999999e-06 2024-09-14T06:32:39.180 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:39.180 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:39.180 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:32:39.180 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:31:34.933910+0000", 2024-09-14T06:32:39.180 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.8569999999999995e-06 2024-09-14T06:32:39.180 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:39.180 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:39.180 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:32:39.181 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:31:34.933932+0000", 2024-09-14T06:32:39.181 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.1997999999999998e-05 2024-09-14T06:32:39.181 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:39.181 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:39.181 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:32:39.181 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:31:34.933941+0000", 2024-09-14T06:32:39.181 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.7180000000000002e-06 2024-09-14T06:32:39.181 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:39.181 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:39.181 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:32:39.181 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:31:34.934013+0000", 2024-09-14T06:32:39.181 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.1761000000000006e-05 2024-09-14T06:32:39.181 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:39.182 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:39.182 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "waiting for peered", 2024-09-14T06:32:39.182 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:31:34.934022+0000", 2024-09-14T06:32:39.182 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.8740000000000001e-06 2024-09-14T06:32:39.182 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:39.182 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:39.182 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:32:39.182 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:31:35.608803+0000", 2024-09-14T06:32:39.182 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.67478164299999999 2024-09-14T06:32:39.182 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:39.182 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:39.182 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:32:39.182 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:31:35.608964+0000", 2024-09-14T06:32:39.182 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00016112900000000001 2024-09-14T06:32:39.183 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:39.183 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:39.183 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "waiting for subops from 6", 2024-09-14T06:32:39.183 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:31:35.609399+0000", 2024-09-14T06:32:39.183 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00043454000000000002 2024-09-14T06:32:39.183 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:39.183 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:39.183 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "op_commit", 2024-09-14T06:32:39.183 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:31:35.610495+0000", 2024-09-14T06:32:39.183 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.001096404 2024-09-14T06:32:39.183 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:39.183 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:39.183 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:32:39.183 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:31:35.615409+0000", 2024-09-14T06:32:39.183 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.0049133199999999997 2024-09-14T06:32:39.184 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:39.184 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:39.184 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "commit_sent", 2024-09-14T06:32:39.184 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:31:35.615444+0000", 2024-09-14T06:32:39.184 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.5457000000000003e-05 2024-09-14T06:32:39.184 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:39.184 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:39.184 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:32:39.184 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:31:35.615476+0000", 2024-09-14T06:32:39.184 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.1668999999999998e-05 2024-09-14T06:32:39.184 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:39.184 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:32:39.184 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:39.184 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:39.184 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:32:39.185 INFO:teuthology.orchestra.run.smithi138.stdout:} 2024-09-14T06:32:39.186 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-14T06:32:39.186 INFO:tasks.ceph.osd.5.smithi138.stderr:2024-09-14T06:32:39.185+0000 7fe13746f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33618) UID: 0 2024-09-14T06:32:39.287 INFO:tasks.ceph.osd.4.smithi138.stderr:2024-09-14T06:32:39.286+0000 7f6c653c8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33615) UID: 0 2024-09-14T06:32:39.332 INFO:teuthology.orchestra.run.smithi138.stdout:{ 2024-09-14T06:32:39.333 INFO:teuthology.orchestra.run.smithi138.stdout: "ops": [], 2024-09-14T06:32:39.333 INFO:teuthology.orchestra.run.smithi138.stdout: "num_ops": 0 2024-09-14T06:32:39.333 INFO:teuthology.orchestra.run.smithi138.stdout:} 2024-09-14T06:32:39.341 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-14T06:32:39.426 INFO:tasks.ceph.osd.5.smithi138.stderr:2024-09-14T06:32:39.425+0000 7fe13746f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33618) UID: 0 2024-09-14T06:32:39.485 INFO:teuthology.orchestra.run.smithi138.stdout:{ 2024-09-14T06:32:39.485 INFO:teuthology.orchestra.run.smithi138.stdout: "ops": [], 2024-09-14T06:32:39.485 INFO:teuthology.orchestra.run.smithi138.stdout: "complaint_time": 30, 2024-09-14T06:32:39.485 INFO:teuthology.orchestra.run.smithi138.stdout: "num_blocked_ops": 0 2024-09-14T06:32:39.485 INFO:teuthology.orchestra.run.smithi138.stdout:} 2024-09-14T06:32:39.488 INFO:tasks.ceph.osd.4.smithi138.stderr:2024-09-14T06:32:39.487+0000 7f6c653c8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33615) UID: 0 2024-09-14T06:32:39.493 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-14T06:32:39.590 INFO:tasks.ceph.osd.0.smithi094.stderr:2024-09-14T06:32:39.587+0000 7f07c9333640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33636) UID: 0 2024-09-14T06:32:39.636 INFO:teuthology.orchestra.run.smithi138.stdout:{ 2024-09-14T06:32:39.637 INFO:teuthology.orchestra.run.smithi138.stdout: "size": 20, 2024-09-14T06:32:39.637 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 600, 2024-09-14T06:32:39.637 INFO:teuthology.orchestra.run.smithi138.stdout: "ops": [ 2024-09-14T06:32:39.637 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:39.637 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:98505 3.10 3.1bf56f90 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T06:32:39.637 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:30:07.045453+0000", 2024-09-14T06:32:39.637 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 152.589500993, 2024-09-14T06:32:39.637 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032144440000000003, 2024-09-14T06:32:39.637 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:32:39.637 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:32:39.637 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "reached pg", 2024-09-14T06:32:39.637 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:32:39.637 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:32:39.638 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:39.638 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 98505 2024-09-14T06:32:39.638 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:39.638 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:32:39.638 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:39.638 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:32:39.638 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045453+0000", 2024-09-14T06:32:39.638 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:39.638 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:39.638 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:39.638 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:32:39.638 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045453+0000", 2024-09-14T06:32:39.638 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:39.639 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:39.639 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:39.639 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:32:39.639 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045453+0000", 2024-09-14T06:32:39.639 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.36e-07 2024-09-14T06:32:39.639 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:39.639 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:39.639 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:32:39.639 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045454+0000", 2024-09-14T06:32:39.639 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.1299999999999998e-07 2024-09-14T06:32:39.639 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:39.639 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:39.639 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:32:39.639 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045457+0000", 2024-09-14T06:32:39.640 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.8559999999999998e-06 2024-09-14T06:32:39.640 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:39.640 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:39.640 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:32:39.640 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045460+0000", 2024-09-14T06:32:39.640 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.8750000000000002e-06 2024-09-14T06:32:39.640 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:39.640 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:39.640 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:32:39.640 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.077592+0000", 2024-09-14T06:32:39.640 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032131727999999998 2024-09-14T06:32:39.640 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:39.640 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:39.640 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:32:39.641 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.077597+0000", 2024-09-14T06:32:39.641 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.8319999999999996e-06 2024-09-14T06:32:39.641 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:39.641 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:32:39.641 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:39.641 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:39.641 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:39.641 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:98506 3.10 3.1bf56f90 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T06:32:39.641 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:30:07.045481+0000", 2024-09-14T06:32:39.641 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 152.58947273800001, 2024-09-14T06:32:39.641 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032142759999999999, 2024-09-14T06:32:39.641 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:32:39.641 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:32:39.642 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "reached pg", 2024-09-14T06:32:39.642 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:32:39.642 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:32:39.642 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:39.642 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 98506 2024-09-14T06:32:39.642 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:39.642 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:32:39.642 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:39.642 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:32:39.642 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045481+0000", 2024-09-14T06:32:39.642 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:39.642 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:39.642 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:39.642 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:32:39.642 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045481+0000", 2024-09-14T06:32:39.643 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:39.643 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:39.643 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:39.643 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:32:39.643 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045481+0000", 2024-09-14T06:32:39.643 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.8500000000000001e-07 2024-09-14T06:32:39.643 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:39.643 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:39.643 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:32:39.643 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045483+0000", 2024-09-14T06:32:39.643 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.8250000000000001e-06 2024-09-14T06:32:39.643 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:39.643 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:39.644 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:32:39.644 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045489+0000", 2024-09-14T06:32:39.644 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.5740000000000003e-06 2024-09-14T06:32:39.644 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:39.644 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:39.644 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:32:39.644 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045495+0000", 2024-09-14T06:32:39.644 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.7690000000000002e-06 2024-09-14T06:32:39.644 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:39.644 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:39.644 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:32:39.644 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.077619+0000", 2024-09-14T06:32:39.644 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032124621999999999 2024-09-14T06:32:39.644 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:39.645 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:39.645 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:32:39.645 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.077624+0000", 2024-09-14T06:32:39.645 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.3850000000000004e-06 2024-09-14T06:32:39.645 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:39.645 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:32:39.645 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:39.645 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:39.645 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:39.645 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:98507 3.10 3.1bf56f90 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T06:32:39.645 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:30:07.045502+0000", 2024-09-14T06:32:39.645 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 152.589451317, 2024-09-14T06:32:39.645 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032147219999999997, 2024-09-14T06:32:39.645 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:32:39.646 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:32:39.646 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "reached pg", 2024-09-14T06:32:39.646 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:32:39.646 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:32:39.646 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:39.646 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 98507 2024-09-14T06:32:39.646 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:39.646 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:32:39.646 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:39.646 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:32:39.646 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045502+0000", 2024-09-14T06:32:39.646 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:39.646 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:39.647 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:39.647 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:32:39.647 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045502+0000", 2024-09-14T06:32:39.647 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:39.647 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:39.647 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:39.647 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:32:39.647 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045503+0000", 2024-09-14T06:32:39.647 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.4700000000000002e-07 2024-09-14T06:32:39.647 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:39.647 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:39.647 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:32:39.647 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045509+0000", 2024-09-14T06:32:39.647 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.7329999999999997e-06 2024-09-14T06:32:39.648 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:39.648 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:39.648 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:32:39.648 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045515+0000", 2024-09-14T06:32:39.648 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.8030000000000002e-06 2024-09-14T06:32:39.648 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:39.648 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:39.648 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:32:39.648 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045522+0000", 2024-09-14T06:32:39.648 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.0369999999999997e-06 2024-09-14T06:32:39.648 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:39.648 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:39.648 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:32:39.648 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.077641+0000", 2024-09-14T06:32:39.648 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032119091000000002 2024-09-14T06:32:39.649 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:39.649 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:39.649 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:32:39.649 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.077650+0000", 2024-09-14T06:32:39.649 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.2090000000000005e-06 2024-09-14T06:32:39.649 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:39.649 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:32:39.649 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:39.649 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:39.649 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:39.649 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:98508 3.10 3.1bf56f90 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T06:32:39.649 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:30:07.045530+0000", 2024-09-14T06:32:39.649 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 152.5894237, 2024-09-14T06:32:39.650 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032198773999999999, 2024-09-14T06:32:39.650 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:32:39.650 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:32:39.650 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "reached pg", 2024-09-14T06:32:39.650 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:32:39.650 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:32:39.650 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:39.650 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 98508 2024-09-14T06:32:39.650 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:39.650 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:32:39.650 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:39.650 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:32:39.650 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045530+0000", 2024-09-14T06:32:39.650 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:39.651 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:39.651 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:39.651 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:32:39.651 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045530+0000", 2024-09-14T06:32:39.651 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:39.651 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:39.651 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:39.651 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:32:39.651 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045530+0000", 2024-09-14T06:32:39.651 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.3099999999999998e-07 2024-09-14T06:32:39.651 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:39.651 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:39.651 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:32:39.651 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045539+0000", 2024-09-14T06:32:39.652 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.4780000000000008e-06 2024-09-14T06:32:39.652 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:39.652 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:39.652 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:32:39.652 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045546+0000", 2024-09-14T06:32:39.652 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.3259999999999998e-06 2024-09-14T06:32:39.652 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:39.652 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:39.652 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:32:39.652 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045551+0000", 2024-09-14T06:32:39.652 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.9960000000000001e-06 2024-09-14T06:32:39.652 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:39.652 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:39.652 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:32:39.653 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.077716+0000", 2024-09-14T06:32:39.653 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032165081999999998 2024-09-14T06:32:39.653 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:39.653 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:39.653 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:32:39.653 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.077729+0000", 2024-09-14T06:32:39.653 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.2461e-05 2024-09-14T06:32:39.653 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:39.653 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:32:39.653 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:39.653 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:39.653 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:39.654 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:98509 3.10 3.1bf56f90 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T06:32:39.654 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:30:07.045558+0000", 2024-09-14T06:32:39.654 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 152.58939549900001, 2024-09-14T06:32:39.654 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032242198, 2024-09-14T06:32:39.654 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:32:39.654 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:32:39.654 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "reached pg", 2024-09-14T06:32:39.654 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:32:39.654 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:32:39.654 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:39.654 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 98509 2024-09-14T06:32:39.654 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:39.654 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:32:39.655 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:39.655 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:32:39.655 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045558+0000", 2024-09-14T06:32:39.655 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:39.655 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:39.655 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:39.655 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:32:39.655 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045558+0000", 2024-09-14T06:32:39.655 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:39.655 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:39.655 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:39.655 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:32:39.655 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045559+0000", 2024-09-14T06:32:39.655 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.27e-07 2024-09-14T06:32:39.656 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:39.656 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:39.656 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:32:39.656 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045560+0000", 2024-09-14T06:32:39.656 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.6959999999999999e-06 2024-09-14T06:32:39.656 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:39.656 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:39.656 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:32:39.656 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045567+0000", 2024-09-14T06:32:39.656 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.6900000000000003e-06 2024-09-14T06:32:39.656 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:39.656 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:39.656 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:32:39.656 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045571+0000", 2024-09-14T06:32:39.657 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.5750000000000002e-06 2024-09-14T06:32:39.657 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:39.657 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:39.657 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:32:39.657 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.077789+0000", 2024-09-14T06:32:39.657 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032217186000000002 2024-09-14T06:32:39.657 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:39.657 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:39.657 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:32:39.657 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.077800+0000", 2024-09-14T06:32:39.657 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.1623999999999999e-05 2024-09-14T06:32:39.657 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:39.657 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:32:39.658 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:39.658 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:39.658 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:39.658 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:98510 3.10 3.1bf56f90 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T06:32:39.658 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:30:07.045580+0000", 2024-09-14T06:32:39.658 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 152.589373685, 2024-09-14T06:32:39.658 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032299845000000001, 2024-09-14T06:32:39.658 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:32:39.658 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:32:39.658 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "reached pg", 2024-09-14T06:32:39.658 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:32:39.658 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:32:39.658 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:39.658 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 98510 2024-09-14T06:32:39.659 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:39.659 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:32:39.659 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:39.659 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:32:39.659 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045580+0000", 2024-09-14T06:32:39.659 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:39.659 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:39.659 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:39.659 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:32:39.659 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045580+0000", 2024-09-14T06:32:39.659 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:39.659 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:39.659 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:39.659 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:32:39.660 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045580+0000", 2024-09-14T06:32:39.660 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.0400000000000002e-07 2024-09-14T06:32:39.660 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:39.660 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:39.660 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:32:39.660 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045582+0000", 2024-09-14T06:32:39.660 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.7600000000000001e-06 2024-09-14T06:32:39.660 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:39.660 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:39.660 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:32:39.660 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045588+0000", 2024-09-14T06:32:39.660 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.5650000000000001e-06 2024-09-14T06:32:39.660 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:39.661 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:39.661 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:32:39.661 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045597+0000", 2024-09-14T06:32:39.661 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.465e-06 2024-09-14T06:32:39.661 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:39.661 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:39.661 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:32:39.661 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.077871+0000", 2024-09-14T06:32:39.661 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032273562999999998 2024-09-14T06:32:39.661 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:39.661 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:39.661 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:32:39.661 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.077880+0000", 2024-09-14T06:32:39.661 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.0880000000000001e-06 2024-09-14T06:32:39.662 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:39.662 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:32:39.662 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:39.662 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:39.662 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:39.662 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:98511 3.10 3.1bf56f90 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T06:32:39.662 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:30:07.045610+0000", 2024-09-14T06:32:39.662 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 152.58934383799999, 2024-09-14T06:32:39.662 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032338894, 2024-09-14T06:32:39.662 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:32:39.662 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:32:39.662 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "reached pg", 2024-09-14T06:32:39.662 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:32:39.663 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:32:39.663 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:39.663 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 98511 2024-09-14T06:32:39.663 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:39.663 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:32:39.663 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:39.663 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:32:39.663 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045610+0000", 2024-09-14T06:32:39.663 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:39.663 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:39.663 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:39.663 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:32:39.663 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045610+0000", 2024-09-14T06:32:39.664 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:39.664 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:39.664 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:39.664 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:32:39.664 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045610+0000", 2024-09-14T06:32:39.664 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.1600000000000002e-07 2024-09-14T06:32:39.664 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:39.664 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:39.664 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:32:39.664 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045612+0000", 2024-09-14T06:32:39.664 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.1150000000000001e-06 2024-09-14T06:32:39.664 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:39.664 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:39.664 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:32:39.665 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045618+0000", 2024-09-14T06:32:39.665 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.0499999999999997e-06 2024-09-14T06:32:39.665 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:39.665 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:39.665 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:32:39.665 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045623+0000", 2024-09-14T06:32:39.665 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.6770000000000003e-06 2024-09-14T06:32:39.665 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:39.665 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:39.665 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:32:39.665 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.077928+0000", 2024-09-14T06:32:39.665 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032305111999999997 2024-09-14T06:32:39.665 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:39.666 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:39.666 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:32:39.666 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.077949+0000", 2024-09-14T06:32:39.666 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.0523999999999999e-05 2024-09-14T06:32:39.666 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:39.666 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:32:39.666 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:39.666 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:39.666 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:39.666 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:98512 3.10 3.1bf56f90 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T06:32:39.666 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:30:07.045630+0000", 2024-09-14T06:32:39.666 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 152.58932383300001, 2024-09-14T06:32:39.666 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032346632, 2024-09-14T06:32:39.667 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:32:39.667 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:32:39.667 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "reached pg", 2024-09-14T06:32:39.667 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:32:39.667 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:32:39.667 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:39.667 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 98512 2024-09-14T06:32:39.667 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:39.667 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:32:39.667 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:39.667 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:32:39.667 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045630+0000", 2024-09-14T06:32:39.667 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:39.667 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:39.668 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:39.668 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:32:39.668 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045630+0000", 2024-09-14T06:32:39.668 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:39.668 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:39.668 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:39.668 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:32:39.668 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045630+0000", 2024-09-14T06:32:39.668 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.8200000000000001e-07 2024-09-14T06:32:39.668 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:39.668 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:39.668 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:32:39.668 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045632+0000", 2024-09-14T06:32:39.668 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.7430000000000001e-06 2024-09-14T06:32:39.669 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:39.669 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:39.669 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:32:39.669 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045637+0000", 2024-09-14T06:32:39.669 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.4550000000000003e-06 2024-09-14T06:32:39.669 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:39.669 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:39.669 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:32:39.669 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045644+0000", 2024-09-14T06:32:39.669 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.1330000000000001e-06 2024-09-14T06:32:39.669 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:39.669 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:39.669 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:32:39.670 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.077968+0000", 2024-09-14T06:32:39.670 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032323401000000002 2024-09-14T06:32:39.670 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:39.670 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:39.670 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:32:39.670 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.077976+0000", 2024-09-14T06:32:39.670 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.5180000000000004e-06 2024-09-14T06:32:39.670 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:39.670 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:32:39.670 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:39.670 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:39.670 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:39.670 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:98513 3.10 3.1bf56f90 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T06:32:39.671 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:30:07.045652+0000", 2024-09-14T06:32:39.671 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 152.589301269, 2024-09-14T06:32:39.671 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032353436999999999, 2024-09-14T06:32:39.671 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:32:39.671 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:32:39.671 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "reached pg", 2024-09-14T06:32:39.671 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:32:39.671 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:32:39.671 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:39.671 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 98513 2024-09-14T06:32:39.671 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:39.671 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:32:39.671 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:39.671 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:32:39.672 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045652+0000", 2024-09-14T06:32:39.672 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:39.672 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:39.672 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:39.672 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:32:39.672 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045652+0000", 2024-09-14T06:32:39.672 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:39.672 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:39.672 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:39.672 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:32:39.672 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045653+0000", 2024-09-14T06:32:39.672 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.3700000000000001e-07 2024-09-14T06:32:39.672 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:39.672 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:39.673 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:32:39.673 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045655+0000", 2024-09-14T06:32:39.673 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.854e-06 2024-09-14T06:32:39.673 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:39.673 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:39.673 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:32:39.673 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045661+0000", 2024-09-14T06:32:39.673 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.5509999999999996e-06 2024-09-14T06:32:39.673 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:39.673 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:39.673 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:32:39.673 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045668+0000", 2024-09-14T06:32:39.673 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.1980000000000004e-06 2024-09-14T06:32:39.673 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:39.674 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:39.674 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:32:39.674 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.077997+0000", 2024-09-14T06:32:39.674 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032328575999999998 2024-09-14T06:32:39.674 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:39.674 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:39.674 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:32:39.674 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.078006+0000", 2024-09-14T06:32:39.674 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.8210000000000008e-06 2024-09-14T06:32:39.674 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:39.674 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:32:39.674 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:39.674 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:39.674 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:39.675 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:98514 3.10 3.1bf56f90 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T06:32:39.675 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:30:07.045699+0000", 2024-09-14T06:32:39.675 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 152.589254934, 2024-09-14T06:32:39.675 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032330223999999998, 2024-09-14T06:32:39.675 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:32:39.675 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:32:39.675 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "reached pg", 2024-09-14T06:32:39.675 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:32:39.675 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:32:39.675 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:39.675 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 98514 2024-09-14T06:32:39.675 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:39.675 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:32:39.675 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:39.676 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:32:39.676 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045699+0000", 2024-09-14T06:32:39.676 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:39.676 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:39.676 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:39.676 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:32:39.676 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045699+0000", 2024-09-14T06:32:39.676 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:39.676 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:39.676 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:39.676 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:32:39.676 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045699+0000", 2024-09-14T06:32:39.676 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.3200000000000001e-07 2024-09-14T06:32:39.677 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:39.677 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:39.677 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:32:39.677 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045701+0000", 2024-09-14T06:32:39.677 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.762e-06 2024-09-14T06:32:39.677 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:39.677 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:39.677 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:32:39.677 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045705+0000", 2024-09-14T06:32:39.677 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.2359999999999998e-06 2024-09-14T06:32:39.677 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:39.677 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:39.677 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:32:39.677 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045708+0000", 2024-09-14T06:32:39.678 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.796e-06 2024-09-14T06:32:39.678 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:39.678 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:39.678 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:32:39.678 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.078023+0000", 2024-09-14T06:32:39.678 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032315213000000002 2024-09-14T06:32:39.678 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:39.678 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:39.678 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:32:39.678 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.078029+0000", 2024-09-14T06:32:39.678 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.8850000000000004e-06 2024-09-14T06:32:39.678 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:39.678 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:32:39.679 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:39.679 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:39.679 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:39.679 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:98515 3.10 3.1bf56f90 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T06:32:39.679 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:30:07.045712+0000", 2024-09-14T06:32:39.679 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 152.58924162299999, 2024-09-14T06:32:39.679 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032346871999999999, 2024-09-14T06:32:39.679 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:32:39.679 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:32:39.679 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "reached pg", 2024-09-14T06:32:39.679 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:32:39.679 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:32:39.679 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:39.679 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 98515 2024-09-14T06:32:39.680 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:39.680 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:32:39.680 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:39.680 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:32:39.680 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045712+0000", 2024-09-14T06:32:39.680 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:39.680 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:39.680 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:39.680 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:32:39.680 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045712+0000", 2024-09-14T06:32:39.680 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:39.680 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:39.680 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:39.681 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:32:39.681 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045712+0000", 2024-09-14T06:32:39.681 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.2100000000000001e-07 2024-09-14T06:32:39.681 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:39.681 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:39.681 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:32:39.681 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045713+0000", 2024-09-14T06:32:39.681 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.068e-06 2024-09-14T06:32:39.681 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:39.681 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:39.681 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:32:39.681 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045716+0000", 2024-09-14T06:32:39.681 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.971e-06 2024-09-14T06:32:39.682 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:39.682 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:39.682 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:32:39.682 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045719+0000", 2024-09-14T06:32:39.682 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.3879999999999998e-06 2024-09-14T06:32:39.682 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:39.682 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:39.682 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:32:39.682 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.078049+0000", 2024-09-14T06:32:39.682 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032330553999999997 2024-09-14T06:32:39.682 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:39.682 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:39.682 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:32:39.682 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.078059+0000", 2024-09-14T06:32:39.683 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.6700000000000006e-06 2024-09-14T06:32:39.683 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:39.683 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:32:39.683 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:39.683 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:39.683 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:39.683 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:98516 3.10 3.1bf56f90 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T06:32:39.683 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:30:07.045722+0000", 2024-09-14T06:32:39.683 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 152.58923186999999, 2024-09-14T06:32:39.683 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032374334999999997, 2024-09-14T06:32:39.683 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:32:39.683 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:32:39.683 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "reached pg", 2024-09-14T06:32:39.683 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:32:39.683 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:32:39.684 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:39.684 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 98516 2024-09-14T06:32:39.684 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:39.684 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:32:39.684 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:39.684 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:32:39.684 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045722+0000", 2024-09-14T06:32:39.684 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:39.684 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:39.684 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:39.684 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:32:39.684 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045722+0000", 2024-09-14T06:32:39.684 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:39.684 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:39.684 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:39.685 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:32:39.685 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045722+0000", 2024-09-14T06:32:39.685 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.8999999999999998e-07 2024-09-14T06:32:39.685 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:39.685 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:39.685 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:32:39.685 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045723+0000", 2024-09-14T06:32:39.685 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.0049999999999999e-06 2024-09-14T06:32:39.685 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:39.685 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:39.685 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:32:39.685 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045726+0000", 2024-09-14T06:32:39.685 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.9019999999999999e-06 2024-09-14T06:32:39.685 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:39.686 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:39.686 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:32:39.686 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045728+0000", 2024-09-14T06:32:39.686 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.3290000000000001e-06 2024-09-14T06:32:39.686 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:39.686 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:39.686 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:32:39.686 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.078090+0000", 2024-09-14T06:32:39.686 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032362123 2024-09-14T06:32:39.686 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:39.686 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:39.686 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:32:39.686 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.078096+0000", 2024-09-14T06:32:39.686 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.6860000000000003e-06 2024-09-14T06:32:39.687 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:39.687 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:32:39.687 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:39.687 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:39.687 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:39.687 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:98517 3.10 3.1bf56f90 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T06:32:39.687 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:30:07.045731+0000", 2024-09-14T06:32:39.687 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 152.58922213899999, 2024-09-14T06:32:39.687 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.03241227, 2024-09-14T06:32:39.687 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:32:39.687 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:32:39.687 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "reached pg", 2024-09-14T06:32:39.687 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:32:39.687 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:32:39.687 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:39.688 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 98517 2024-09-14T06:32:39.688 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:39.688 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:32:39.688 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:39.688 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:32:39.688 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045731+0000", 2024-09-14T06:32:39.688 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:39.688 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:39.688 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:39.688 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:32:39.688 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045731+0000", 2024-09-14T06:32:39.688 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:39.688 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:39.688 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:39.689 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:32:39.689 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045732+0000", 2024-09-14T06:32:39.689 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.0699999999999999e-07 2024-09-14T06:32:39.689 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:39.689 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:39.689 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:32:39.689 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045733+0000", 2024-09-14T06:32:39.689 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.9199999999999999e-07 2024-09-14T06:32:39.689 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:39.689 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:39.689 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:32:39.689 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045735+0000", 2024-09-14T06:32:39.689 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.8150000000000002e-06 2024-09-14T06:32:39.689 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:39.689 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:39.690 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:32:39.690 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045738+0000", 2024-09-14T06:32:39.690 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.3640000000000002e-06 2024-09-14T06:32:39.690 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:39.690 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:39.690 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:32:39.690 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.078126+0000", 2024-09-14T06:32:39.690 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032388206000000003 2024-09-14T06:32:39.690 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:39.690 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:39.690 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:32:39.690 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.078144+0000", 2024-09-14T06:32:39.690 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.7686e-05 2024-09-14T06:32:39.691 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:39.691 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:32:39.691 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:39.691 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:39.691 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:39.691 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:98518 3.10 3.1bf56f90 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T06:32:39.691 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:30:07.045741+0000", 2024-09-14T06:32:39.691 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 152.58921274400001, 2024-09-14T06:32:39.691 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032451793999999999, 2024-09-14T06:32:39.691 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:32:39.691 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:32:39.691 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "reached pg", 2024-09-14T06:32:39.691 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:32:39.691 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:32:39.691 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:39.691 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 98518 2024-09-14T06:32:39.692 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:39.692 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:32:39.692 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:39.692 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:32:39.692 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045741+0000", 2024-09-14T06:32:39.692 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:39.692 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:39.692 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:39.692 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:32:39.692 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045741+0000", 2024-09-14T06:32:39.692 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:39.692 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:39.692 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:39.692 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:32:39.693 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045741+0000", 2024-09-14T06:32:39.693 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.05e-07 2024-09-14T06:32:39.693 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:39.693 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:39.693 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:32:39.693 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045746+0000", 2024-09-14T06:32:39.693 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.2259999999999996e-06 2024-09-14T06:32:39.693 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:39.693 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:39.693 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:32:39.693 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045750+0000", 2024-09-14T06:32:39.693 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.2490000000000001e-06 2024-09-14T06:32:39.693 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:39.693 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:39.694 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:32:39.694 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045752+0000", 2024-09-14T06:32:39.694 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.3980000000000002e-06 2024-09-14T06:32:39.694 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:39.694 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:39.694 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:32:39.694 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.078184+0000", 2024-09-14T06:32:39.694 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032431593000000002 2024-09-14T06:32:39.694 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:39.694 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:39.694 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:32:39.694 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.078193+0000", 2024-09-14T06:32:39.694 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.1230000000000006e-06 2024-09-14T06:32:39.695 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:39.695 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:32:39.695 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:39.695 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:39.695 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:39.695 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:98519 3.10 3.1bf56f90 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T06:32:39.695 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:30:07.045755+0000", 2024-09-14T06:32:39.695 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 152.58919831899999, 2024-09-14T06:32:39.695 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032489431999999999, 2024-09-14T06:32:39.695 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:32:39.695 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:32:39.695 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "reached pg", 2024-09-14T06:32:39.695 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:32:39.696 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:32:39.696 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:39.696 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 98519 2024-09-14T06:32:39.696 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:39.696 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:32:39.696 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:39.696 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:32:39.696 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045755+0000", 2024-09-14T06:32:39.696 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:39.696 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:39.696 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:39.696 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:32:39.696 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045755+0000", 2024-09-14T06:32:39.697 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:39.697 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:39.697 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:39.697 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:32:39.697 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045756+0000", 2024-09-14T06:32:39.697 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.23e-07 2024-09-14T06:32:39.697 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:39.697 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:39.697 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:32:39.697 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045757+0000", 2024-09-14T06:32:39.697 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.1060000000000001e-06 2024-09-14T06:32:39.697 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:39.697 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:39.697 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:32:39.698 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045760+0000", 2024-09-14T06:32:39.698 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.988e-06 2024-09-14T06:32:39.698 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:39.698 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:39.698 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:32:39.698 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045765+0000", 2024-09-14T06:32:39.698 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.2689999999999999e-06 2024-09-14T06:32:39.698 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:39.698 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:39.698 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:32:39.698 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.078234+0000", 2024-09-14T06:32:39.698 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032468915000000001 2024-09-14T06:32:39.698 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:39.698 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:39.699 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:32:39.699 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.078245+0000", 2024-09-14T06:32:39.699 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.0930999999999999e-05 2024-09-14T06:32:39.699 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:39.699 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:32:39.699 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:39.699 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:39.699 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:39.699 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:98520 3.10 3.1bf56f90 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T06:32:39.699 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:30:07.045768+0000", 2024-09-14T06:32:39.699 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 152.58918544100001, 2024-09-14T06:32:39.699 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032531379999999999, 2024-09-14T06:32:39.699 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:32:39.699 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:32:39.699 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "reached pg", 2024-09-14T06:32:39.700 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:32:39.700 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:32:39.700 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:39.700 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 98520 2024-09-14T06:32:39.700 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:39.700 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:32:39.700 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:39.700 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:32:39.700 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045768+0000", 2024-09-14T06:32:39.700 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:39.700 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:39.700 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:39.700 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:32:39.701 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045768+0000", 2024-09-14T06:32:39.701 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:39.701 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:39.701 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:39.701 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:32:39.701 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045768+0000", 2024-09-14T06:32:39.701 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.22e-07 2024-09-14T06:32:39.701 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:39.701 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:39.701 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:32:39.701 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045773+0000", 2024-09-14T06:32:39.701 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.3470000000000001e-06 2024-09-14T06:32:39.701 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:39.701 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:39.702 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:32:39.702 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045776+0000", 2024-09-14T06:32:39.702 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.1319999999999998e-06 2024-09-14T06:32:39.702 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:39.702 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:39.702 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:32:39.702 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045779+0000", 2024-09-14T06:32:39.702 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.9749999999999999e-06 2024-09-14T06:32:39.702 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:39.702 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:39.702 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:32:39.702 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.078289+0000", 2024-09-14T06:32:39.702 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032509693999999999 2024-09-14T06:32:39.703 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:39.703 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:39.703 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:32:39.703 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.078300+0000", 2024-09-14T06:32:39.703 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.101e-05 2024-09-14T06:32:39.703 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:39.703 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:32:39.703 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:39.703 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:39.703 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:39.703 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:98521 3.10 3.1bf56f90 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T06:32:39.703 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:30:07.045782+0000", 2024-09-14T06:32:39.703 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 152.58917126099999, 2024-09-14T06:32:39.703 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032566109000000003, 2024-09-14T06:32:39.704 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:32:39.704 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:32:39.704 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "reached pg", 2024-09-14T06:32:39.704 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:32:39.704 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:32:39.704 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:39.704 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 98521 2024-09-14T06:32:39.704 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:39.704 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:32:39.704 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:39.704 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:32:39.704 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045782+0000", 2024-09-14T06:32:39.704 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:39.705 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:39.705 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:39.705 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:32:39.705 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045782+0000", 2024-09-14T06:32:39.705 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:39.705 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:39.705 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:39.705 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:32:39.705 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045783+0000", 2024-09-14T06:32:39.705 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.2499999999999999e-07 2024-09-14T06:32:39.705 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:39.705 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:39.705 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:32:39.705 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045784+0000", 2024-09-14T06:32:39.705 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.0109999999999999e-06 2024-09-14T06:32:39.706 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:39.706 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:39.706 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:32:39.706 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045786+0000", 2024-09-14T06:32:39.706 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.8289999999999999e-06 2024-09-14T06:32:39.706 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:39.706 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:39.706 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:32:39.706 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045793+0000", 2024-09-14T06:32:39.706 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.579e-06 2024-09-14T06:32:39.706 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:39.706 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:39.706 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:32:39.706 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.078338+0000", 2024-09-14T06:32:39.707 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032545389000000001 2024-09-14T06:32:39.707 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:39.707 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:39.707 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:32:39.707 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.078348+0000", 2024-09-14T06:32:39.707 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.0076e-05 2024-09-14T06:32:39.707 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:39.707 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:32:39.707 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:39.707 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:39.707 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:39.707 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:98522 3.10 3.1bf56f90 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T06:32:39.707 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:30:07.045796+0000", 2024-09-14T06:32:39.707 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 152.58915735299999, 2024-09-14T06:32:39.707 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032577846000000001, 2024-09-14T06:32:39.708 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:32:39.708 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:32:39.708 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "reached pg", 2024-09-14T06:32:39.708 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:32:39.708 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:32:39.708 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:39.708 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 98522 2024-09-14T06:32:39.708 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:39.708 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:32:39.708 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:39.708 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:32:39.708 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045796+0000", 2024-09-14T06:32:39.708 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:39.709 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:39.709 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:39.709 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:32:39.709 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045796+0000", 2024-09-14T06:32:39.709 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:39.709 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:39.709 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:39.709 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:32:39.709 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045796+0000", 2024-09-14T06:32:39.709 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.2100000000000001e-07 2024-09-14T06:32:39.709 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:39.709 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:39.710 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:32:39.710 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045797+0000", 2024-09-14T06:32:39.710 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.6899999999999996e-07 2024-09-14T06:32:39.710 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:39.710 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:39.710 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:32:39.710 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045800+0000", 2024-09-14T06:32:39.710 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.9979999999999999e-06 2024-09-14T06:32:39.710 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:39.710 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:39.710 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:32:39.710 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045803+0000", 2024-09-14T06:32:39.710 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.2850000000000001e-06 2024-09-14T06:32:39.710 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:39.711 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:39.711 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:32:39.711 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.078366+0000", 2024-09-14T06:32:39.711 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032562932000000003 2024-09-14T06:32:39.711 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:39.711 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:39.711 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:32:39.711 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.078374+0000", 2024-09-14T06:32:39.711 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.4409999999999993e-06 2024-09-14T06:32:39.711 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:39.711 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:32:39.711 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:39.711 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:39.712 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:39.712 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:98523 3.10 3.1bf56f90 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T06:32:39.712 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:30:07.045806+0000", 2024-09-14T06:32:39.712 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 152.58914780999999, 2024-09-14T06:32:39.712 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032611167000000003, 2024-09-14T06:32:39.712 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:32:39.712 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:32:39.712 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "reached pg", 2024-09-14T06:32:39.712 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:32:39.712 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:32:39.712 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:39.712 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 98523 2024-09-14T06:32:39.712 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:39.713 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:32:39.713 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:39.713 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:32:39.713 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045806+0000", 2024-09-14T06:32:39.713 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:39.713 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:39.713 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:39.713 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:32:39.713 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045806+0000", 2024-09-14T06:32:39.713 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:39.713 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:39.713 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:39.713 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:32:39.713 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045806+0000", 2024-09-14T06:32:39.714 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.2000000000000001e-07 2024-09-14T06:32:39.714 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:39.714 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:39.714 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:32:39.714 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045807+0000", 2024-09-14T06:32:39.714 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.4e-07 2024-09-14T06:32:39.714 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:39.714 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:39.714 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:32:39.714 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045810+0000", 2024-09-14T06:32:39.714 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.9229999999999998e-06 2024-09-14T06:32:39.714 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:39.714 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:39.714 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:32:39.715 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045812+0000", 2024-09-14T06:32:39.715 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.2220000000000001e-06 2024-09-14T06:32:39.715 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:39.715 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:39.715 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:32:39.715 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.078407+0000", 2024-09-14T06:32:39.715 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032594513999999998 2024-09-14T06:32:39.715 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:39.715 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:39.715 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:32:39.715 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.078417+0000", 2024-09-14T06:32:39.715 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.0348e-05 2024-09-14T06:32:39.715 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:39.716 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:32:39.716 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:39.716 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:39.716 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:39.716 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:98524 3.10 3.1bf56f90 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T06:32:39.716 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:30:07.045815+0000", 2024-09-14T06:32:39.716 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 152.58913844200001, 2024-09-14T06:32:39.716 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032643208999999999, 2024-09-14T06:32:39.716 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:32:39.716 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:32:39.716 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "reached pg", 2024-09-14T06:32:39.716 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:32:39.716 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:32:39.716 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:39.717 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 98524 2024-09-14T06:32:39.717 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:39.717 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:32:39.717 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:39.717 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:32:39.717 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045815+0000", 2024-09-14T06:32:39.717 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:39.717 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:39.717 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:39.717 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:32:39.717 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045815+0000", 2024-09-14T06:32:39.717 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:39.717 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:39.718 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:39.718 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:32:39.718 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045815+0000", 2024-09-14T06:32:39.718 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.11e-07 2024-09-14T06:32:39.718 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:39.718 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:39.718 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:32:39.718 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045816+0000", 2024-09-14T06:32:39.718 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.9099999999999991e-07 2024-09-14T06:32:39.718 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:39.718 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:39.718 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:32:39.718 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045819+0000", 2024-09-14T06:32:39.718 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.9399999999999998e-06 2024-09-14T06:32:39.718 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:39.719 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:39.719 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:32:39.719 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045822+0000", 2024-09-14T06:32:39.719 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.3470000000000001e-06 2024-09-14T06:32:39.719 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:39.719 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:39.719 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:32:39.719 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.078449+0000", 2024-09-14T06:32:39.719 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032627529000000002 2024-09-14T06:32:39.719 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:39.719 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:39.719 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:32:39.719 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.078458+0000", 2024-09-14T06:32:39.719 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.1910000000000006e-06 2024-09-14T06:32:39.719 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:39.720 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:32:39.720 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:39.720 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:39.720 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:32:39.720 INFO:teuthology.orchestra.run.smithi138.stdout:} 2024-09-14T06:32:39.721 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-14T06:32:39.722 INFO:tasks.ceph.osd.4.smithi138.stderr:2024-09-14T06:32:39.720+0000 7f6c653c8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33615) UID: 0 2024-09-14T06:32:39.776 INFO:teuthology.orchestra.run.smithi094.stderr:nodeep-scrub is set 2024-09-14T06:32:39.776 INFO:teuthology.orchestra.run.smithi094.stderr:reweighted osd.5 to 1 (10000) 2024-09-14T06:32:39.795 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 7 1 2024-09-14T06:32:39.836 INFO:tasks.ceph.osd.1.smithi094.stderr:2024-09-14T06:32:39.834+0000 7f328e47a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33641) UID: 0 2024-09-14T06:32:39.883 INFO:teuthology.orchestra.run.smithi138.stdout:{ 2024-09-14T06:32:39.883 INFO:teuthology.orchestra.run.smithi138.stdout: "ops": [], 2024-09-14T06:32:39.883 INFO:teuthology.orchestra.run.smithi138.stdout: "num_ops": 0 2024-09-14T06:32:39.884 INFO:teuthology.orchestra.run.smithi138.stdout:} 2024-09-14T06:32:39.892 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-14T06:32:39.936 INFO:tasks.ceph.osd.5.smithi138.stderr:2024-09-14T06:32:39.935+0000 7fe13746f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33618) UID: 0 2024-09-14T06:32:39.997 INFO:teuthology.orchestra.run.smithi138.stdout:{ 2024-09-14T06:32:39.998 INFO:teuthology.orchestra.run.smithi138.stdout: "ops": [], 2024-09-14T06:32:39.998 INFO:teuthology.orchestra.run.smithi138.stdout: "complaint_time": 30, 2024-09-14T06:32:39.998 INFO:teuthology.orchestra.run.smithi138.stdout: "num_blocked_ops": 0 2024-09-14T06:32:39.998 INFO:teuthology.orchestra.run.smithi138.stdout:} 2024-09-14T06:32:40.006 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-14T06:32:40.047 INFO:tasks.ceph.osd.7.smithi138.stderr:2024-09-14T06:32:40.046+0000 7f3c6a21d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33623) UID: 0 2024-09-14T06:32:40.124 INFO:tasks.ceph.osd.3.smithi094.stderr:2024-09-14T06:32:40.122+0000 7feeeafe8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33655) UID: 0 2024-09-14T06:32:40.151 INFO:teuthology.orchestra.run.smithi138.stdout:{ 2024-09-14T06:32:40.151 INFO:teuthology.orchestra.run.smithi138.stdout: "size": 20, 2024-09-14T06:32:40.151 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 600, 2024-09-14T06:32:40.151 INFO:teuthology.orchestra.run.smithi138.stdout: "ops": [ 2024-09-14T06:32:40.151 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:40.151 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:53244 3.f 3:f71f51a7:::rbd_data.112dca2f6ae1.0000000000000015:17 [read 1863680~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:32:40.152 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:44.369399+0000", 2024-09-14T06:32:40.152 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 175.78011809200001, 2024-09-14T06:32:40.152 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047291189999999997, 2024-09-14T06:32:40.152 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:32:40.152 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:32:40.152 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:32:40.152 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:32:40.152 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:32:40.152 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:40.152 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 53244 2024-09-14T06:32:40.152 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:40.152 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:32:40.152 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:40.153 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:32:40.153 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369399+0000", 2024-09-14T06:32:40.153 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:40.153 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:40.153 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:40.153 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:32:40.153 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369399+0000", 2024-09-14T06:32:40.153 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:40.153 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:40.153 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:40.153 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:32:40.153 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369400+0000", 2024-09-14T06:32:40.153 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.0699999999999999e-07 2024-09-14T06:32:40.154 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:40.154 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:40.154 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:32:40.154 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369401+0000", 2024-09-14T06:32:40.154 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.4300000000000001e-06 2024-09-14T06:32:40.154 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:40.154 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:40.154 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:32:40.154 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369404+0000", 2024-09-14T06:32:40.154 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.061e-06 2024-09-14T06:32:40.154 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:40.154 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:40.154 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:32:40.154 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369407+0000", 2024-09-14T06:32:40.155 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.6139999999999999e-06 2024-09-14T06:32:40.155 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:40.155 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:40.155 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:32:40.155 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.416517+0000", 2024-09-14T06:32:40.155 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047110338000000002 2024-09-14T06:32:40.155 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:40.155 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:40.155 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:32:40.155 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.416616+0000", 2024-09-14T06:32:40.155 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.8651000000000001e-05 2024-09-14T06:32:40.155 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:40.155 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:40.156 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:32:40.156 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.416691+0000", 2024-09-14T06:32:40.156 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.4888999999999994e-05 2024-09-14T06:32:40.156 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:40.156 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:32:40.156 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:40.156 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:40.156 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:40.156 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:53245 3.f 3:f71f51a7:::rbd_data.112dca2f6ae1.0000000000000015:17 [read 1888256~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:32:40.156 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:44.369410+0000", 2024-09-14T06:32:40.156 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 175.78010780400001, 2024-09-14T06:32:40.156 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047399275999999997, 2024-09-14T06:32:40.156 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:32:40.157 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:32:40.157 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:32:40.157 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:32:40.157 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:32:40.157 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:40.157 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 53245 2024-09-14T06:32:40.157 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:40.157 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:32:40.157 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:40.157 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:32:40.157 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369410+0000", 2024-09-14T06:32:40.157 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:40.157 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:40.157 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:40.158 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:32:40.158 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369410+0000", 2024-09-14T06:32:40.158 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:40.158 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:40.158 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:40.158 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:32:40.158 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369410+0000", 2024-09-14T06:32:40.158 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.1899999999999999e-07 2024-09-14T06:32:40.158 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:40.158 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:40.158 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:32:40.158 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369411+0000", 2024-09-14T06:32:40.158 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.0300000000000001e-06 2024-09-14T06:32:40.159 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:40.159 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:40.159 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:32:40.159 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369414+0000", 2024-09-14T06:32:40.159 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.1099999999999999e-06 2024-09-14T06:32:40.159 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:40.159 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:40.159 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:32:40.159 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369417+0000", 2024-09-14T06:32:40.159 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.6189999999999998e-06 2024-09-14T06:32:40.159 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:40.159 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:40.159 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:32:40.159 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.416718+0000", 2024-09-14T06:32:40.160 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047301322999999999 2024-09-14T06:32:40.160 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:40.160 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:40.160 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:32:40.160 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.416759+0000", 2024-09-14T06:32:40.160 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.1009e-05 2024-09-14T06:32:40.160 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:40.160 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:40.160 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:32:40.160 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.416809+0000", 2024-09-14T06:32:40.160 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.9966000000000002e-05 2024-09-14T06:32:40.160 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:40.160 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:32:40.160 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:40.161 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:40.161 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:40.161 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:53246 3.f 3:f71f51a7:::rbd_data.112dca2f6ae1.0000000000000015:17 [read 1912832~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:32:40.161 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:44.369420+0000", 2024-09-14T06:32:40.161 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 175.78009786000001, 2024-09-14T06:32:40.161 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047529463000000001, 2024-09-14T06:32:40.161 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:32:40.161 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:32:40.161 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:32:40.161 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:32:40.161 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:32:40.161 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:40.161 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 53246 2024-09-14T06:32:40.162 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:40.162 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:32:40.162 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:40.162 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:32:40.162 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369420+0000", 2024-09-14T06:32:40.162 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:40.162 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:40.162 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:40.162 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:32:40.162 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369420+0000", 2024-09-14T06:32:40.162 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:40.162 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:40.162 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:40.162 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:32:40.162 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369420+0000", 2024-09-14T06:32:40.163 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.1299999999999999e-07 2024-09-14T06:32:40.163 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:40.163 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:40.163 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:32:40.163 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369425+0000", 2024-09-14T06:32:40.163 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.0130000000000001e-06 2024-09-14T06:32:40.163 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:40.163 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:40.163 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:32:40.163 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369428+0000", 2024-09-14T06:32:40.163 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.349e-06 2024-09-14T06:32:40.163 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:40.163 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:40.163 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:32:40.164 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369431+0000", 2024-09-14T06:32:40.164 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.7389999999999999e-06 2024-09-14T06:32:40.164 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:40.164 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:40.164 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:32:40.164 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.416834+0000", 2024-09-14T06:32:40.164 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047402669000000001 2024-09-14T06:32:40.164 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:40.164 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:40.164 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:32:40.164 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.416882+0000", 2024-09-14T06:32:40.164 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.8068999999999997e-05 2024-09-14T06:32:40.164 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:40.165 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:40.165 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:32:40.165 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.416949+0000", 2024-09-14T06:32:40.165 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.7410999999999995e-05 2024-09-14T06:32:40.165 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:40.165 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:32:40.165 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:40.165 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:40.165 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:40.165 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:53247 3.f 3:f71f51a7:::rbd_data.112dca2f6ae1.0000000000000015:17 [read 1937408~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:32:40.165 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:44.369434+0000", 2024-09-14T06:32:40.165 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 175.780083096, 2024-09-14T06:32:40.165 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047652264999999999, 2024-09-14T06:32:40.166 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:32:40.166 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:32:40.166 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:32:40.166 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:32:40.166 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:32:40.166 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:40.166 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 53247 2024-09-14T06:32:40.166 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:40.166 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:32:40.166 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:40.166 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:32:40.166 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369434+0000", 2024-09-14T06:32:40.166 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:40.166 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:40.167 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:40.167 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:32:40.167 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369434+0000", 2024-09-14T06:32:40.167 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:40.167 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:40.167 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:40.167 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:32:40.167 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369435+0000", 2024-09-14T06:32:40.167 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.2100000000000001e-07 2024-09-14T06:32:40.167 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:40.167 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:40.167 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:32:40.167 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369436+0000", 2024-09-14T06:32:40.167 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.122e-06 2024-09-14T06:32:40.168 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:40.168 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:40.168 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:32:40.168 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369439+0000", 2024-09-14T06:32:40.168 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.0400000000000001e-06 2024-09-14T06:32:40.168 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:40.168 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:40.168 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:32:40.168 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369441+0000", 2024-09-14T06:32:40.168 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.728e-06 2024-09-14T06:32:40.168 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:40.168 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:40.168 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:32:40.169 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.416977+0000", 2024-09-14T06:32:40.169 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047535344 2024-09-14T06:32:40.169 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:40.169 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:40.169 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:32:40.169 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.417029+0000", 2024-09-14T06:32:40.169 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.2037999999999999e-05 2024-09-14T06:32:40.169 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:40.169 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:40.169 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:32:40.169 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.417087+0000", 2024-09-14T06:32:40.169 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.7772000000000002e-05 2024-09-14T06:32:40.169 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:40.170 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:32:40.170 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:40.170 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:40.170 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:40.170 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:53248 3.f 3:f71f51a7:::rbd_data.112dca2f6ae1.0000000000000015:17 [read 1961984~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:32:40.170 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:44.369445+0000", 2024-09-14T06:32:40.170 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 175.78007254400001, 2024-09-14T06:32:40.170 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047798489, 2024-09-14T06:32:40.170 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:32:40.170 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:32:40.170 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:32:40.170 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:32:40.170 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:32:40.170 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:40.171 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 53248 2024-09-14T06:32:40.171 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:40.171 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:32:40.171 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:40.171 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:32:40.171 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369445+0000", 2024-09-14T06:32:40.171 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:40.171 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:40.171 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:40.171 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:32:40.171 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369445+0000", 2024-09-14T06:32:40.171 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:40.171 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:40.171 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:40.172 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:32:40.172 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369445+0000", 2024-09-14T06:32:40.172 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.0699999999999999e-07 2024-09-14T06:32:40.172 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:40.172 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:40.172 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:32:40.172 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369446+0000", 2024-09-14T06:32:40.172 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.1000000000000001e-06 2024-09-14T06:32:40.172 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:40.172 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:40.172 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:32:40.172 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369449+0000", 2024-09-14T06:32:40.173 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.1039999999999998e-06 2024-09-14T06:32:40.173 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:40.173 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:40.173 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:32:40.173 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369452+0000", 2024-09-14T06:32:40.173 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.6199999999999999e-06 2024-09-14T06:32:40.173 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:40.173 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:40.173 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:32:40.173 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.417114+0000", 2024-09-14T06:32:40.173 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047661901999999999 2024-09-14T06:32:40.173 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:40.173 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:40.173 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:32:40.174 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.417181+0000", 2024-09-14T06:32:40.174 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.7385999999999995e-05 2024-09-14T06:32:40.174 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:40.174 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:40.174 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:32:40.174 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.417243+0000", 2024-09-14T06:32:40.174 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.2169999999999996e-05 2024-09-14T06:32:40.174 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:40.174 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:32:40.174 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:40.174 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:40.174 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:40.174 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:53249 3.f 3:f71f51a7:::rbd_data.112dca2f6ae1.0000000000000015:17 [read 1986560~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:32:40.174 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:44.369455+0000", 2024-09-14T06:32:40.175 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 175.780062233, 2024-09-14T06:32:40.175 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047890493999999999, 2024-09-14T06:32:40.175 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:32:40.175 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:32:40.175 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:32:40.175 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:32:40.175 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:32:40.175 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:40.175 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 53249 2024-09-14T06:32:40.175 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:40.175 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:32:40.175 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:40.175 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:32:40.175 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369455+0000", 2024-09-14T06:32:40.176 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:40.176 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:40.176 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:40.176 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:32:40.176 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369455+0000", 2024-09-14T06:32:40.176 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:40.176 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:40.176 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:40.176 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:32:40.176 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369455+0000", 2024-09-14T06:32:40.176 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.0800000000000001e-07 2024-09-14T06:32:40.176 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:40.176 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:40.177 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:32:40.177 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369456+0000", 2024-09-14T06:32:40.177 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.0729999999999999e-06 2024-09-14T06:32:40.177 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:40.177 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:40.177 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:32:40.177 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369460+0000", 2024-09-14T06:32:40.177 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.027e-06 2024-09-14T06:32:40.177 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:40.177 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:40.177 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:32:40.177 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369462+0000", 2024-09-14T06:32:40.177 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.5320000000000001e-06 2024-09-14T06:32:40.177 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:40.178 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:40.178 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:32:40.178 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.417257+0000", 2024-09-14T06:32:40.178 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.04779489 2024-09-14T06:32:40.178 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:40.178 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:40.178 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:32:40.178 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.417296+0000", 2024-09-14T06:32:40.178 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.8658000000000002e-05 2024-09-14T06:32:40.178 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:40.178 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:40.178 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:32:40.178 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.417346+0000", 2024-09-14T06:32:40.178 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.0105999999999998e-05 2024-09-14T06:32:40.178 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:40.179 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:32:40.179 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:40.179 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:40.179 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:40.179 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:53250 3.f 3:f71f51a7:::rbd_data.112dca2f6ae1.0000000000000015:17 [read 2011136~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:32:40.179 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:44.369465+0000", 2024-09-14T06:32:40.179 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 175.78005226299999, 2024-09-14T06:32:40.179 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.048032230000000002, 2024-09-14T06:32:40.179 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:32:40.179 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:32:40.179 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:32:40.179 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:32:40.179 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:32:40.180 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:40.180 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 53250 2024-09-14T06:32:40.180 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:40.180 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:32:40.180 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:40.180 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:32:40.180 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369465+0000", 2024-09-14T06:32:40.180 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:40.180 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:40.180 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:40.180 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:32:40.180 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369465+0000", 2024-09-14T06:32:40.180 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:40.180 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:40.181 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:40.181 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:32:40.181 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369465+0000", 2024-09-14T06:32:40.181 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.2100000000000001e-07 2024-09-14T06:32:40.181 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:40.181 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:40.181 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:32:40.181 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369466+0000", 2024-09-14T06:32:40.181 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.0640000000000001e-06 2024-09-14T06:32:40.181 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:40.181 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:40.181 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:32:40.181 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369469+0000", 2024-09-14T06:32:40.181 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.0170000000000001e-06 2024-09-14T06:32:40.182 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:40.182 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:40.182 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:32:40.182 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369472+0000", 2024-09-14T06:32:40.182 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.6189999999999998e-06 2024-09-14T06:32:40.182 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:40.182 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:40.182 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:32:40.182 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.417383+0000", 2024-09-14T06:32:40.182 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047911092000000002 2024-09-14T06:32:40.182 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:40.182 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:40.182 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:32:40.182 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.417430+0000", 2024-09-14T06:32:40.183 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.6898999999999999e-05 2024-09-14T06:32:40.183 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:40.183 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:40.183 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:32:40.183 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.417497+0000", 2024-09-14T06:32:40.183 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.7317999999999995e-05 2024-09-14T06:32:40.183 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:40.183 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:32:40.183 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:40.183 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:40.183 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:40.183 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:53251 3.f 3:f71f51a7:::rbd_data.112dca2f6ae1.0000000000000015:17 [read 2035712~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:32:40.183 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:44.369475+0000", 2024-09-14T06:32:40.183 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 175.780042267, 2024-09-14T06:32:40.184 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.048154542000000002, 2024-09-14T06:32:40.184 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:32:40.184 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:32:40.184 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:32:40.184 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:32:40.184 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:32:40.184 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:40.184 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 53251 2024-09-14T06:32:40.184 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:40.184 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:32:40.184 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:40.184 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:32:40.184 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369475+0000", 2024-09-14T06:32:40.184 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:40.185 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:40.185 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:40.185 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:32:40.185 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369475+0000", 2024-09-14T06:32:40.185 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:40.185 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:40.185 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:40.185 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:32:40.185 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369475+0000", 2024-09-14T06:32:40.185 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.0100000000000001e-07 2024-09-14T06:32:40.185 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:40.185 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:40.185 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:32:40.185 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369476+0000", 2024-09-14T06:32:40.186 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.0580000000000001e-06 2024-09-14T06:32:40.186 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:40.186 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:40.186 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:32:40.186 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369479+0000", 2024-09-14T06:32:40.186 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.9950000000000001e-06 2024-09-14T06:32:40.186 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:40.186 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:40.186 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:32:40.186 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369482+0000", 2024-09-14T06:32:40.186 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.6340000000000001e-06 2024-09-14T06:32:40.186 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:40.186 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:40.186 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:32:40.187 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.417530+0000", 2024-09-14T06:32:40.187 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.048047535000000002 2024-09-14T06:32:40.187 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:40.187 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:40.187 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:32:40.187 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.417574+0000", 2024-09-14T06:32:40.187 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.4255999999999998e-05 2024-09-14T06:32:40.187 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:40.187 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:40.187 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:32:40.187 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.417630+0000", 2024-09-14T06:32:40.187 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.5862999999999997e-05 2024-09-14T06:32:40.187 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:40.187 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:32:40.188 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:40.188 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:40.188 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:40.188 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:53252 3.f 3:f71f51a7:::rbd_data.112dca2f6ae1.0000000000000015:17 [read 2060288~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:32:40.188 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:44.369485+0000", 2024-09-14T06:32:40.188 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 175.78003241900001, 2024-09-14T06:32:40.188 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.048254116, 2024-09-14T06:32:40.188 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:32:40.188 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:32:40.188 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:32:40.188 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:32:40.188 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:32:40.188 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:40.189 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 53252 2024-09-14T06:32:40.189 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:40.189 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:32:40.189 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:40.189 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:32:40.189 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369485+0000", 2024-09-14T06:32:40.189 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:40.189 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:40.189 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:40.189 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:32:40.189 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369485+0000", 2024-09-14T06:32:40.189 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:40.189 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:40.189 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:40.190 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:32:40.190 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369485+0000", 2024-09-14T06:32:40.190 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.1e-07 2024-09-14T06:32:40.190 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:40.190 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:40.190 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:32:40.190 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369487+0000", 2024-09-14T06:32:40.190 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.42e-06 2024-09-14T06:32:40.190 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:40.190 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:40.190 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:32:40.190 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369490+0000", 2024-09-14T06:32:40.190 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.9639999999999999e-06 2024-09-14T06:32:40.190 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:40.191 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:40.191 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:32:40.191 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369492+0000", 2024-09-14T06:32:40.191 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.5679999999999998e-06 2024-09-14T06:32:40.191 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:40.191 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:40.191 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:32:40.191 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.417645+0000", 2024-09-14T06:32:40.191 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.048152753 2024-09-14T06:32:40.191 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:40.191 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:40.191 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:32:40.191 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.417687+0000", 2024-09-14T06:32:40.191 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.1900999999999997e-05 2024-09-14T06:32:40.192 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:40.192 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:40.192 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:32:40.192 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.417739+0000", 2024-09-14T06:32:40.192 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.2299999999999997e-05 2024-09-14T06:32:40.192 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:40.192 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:32:40.192 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:40.192 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:40.192 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:40.192 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:53253 3.f 3:f71f51a7:::rbd_data.112dca2f6ae1.0000000000000015:17 [read 2084864~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:32:40.192 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:44.369495+0000", 2024-09-14T06:32:40.192 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 175.78002229500001, 2024-09-14T06:32:40.193 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.048366785000000002, 2024-09-14T06:32:40.193 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:32:40.193 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:32:40.193 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:32:40.193 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:32:40.193 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:32:40.193 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:40.193 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 53253 2024-09-14T06:32:40.193 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:40.193 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:32:40.193 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:40.193 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:32:40.193 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369495+0000", 2024-09-14T06:32:40.194 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:40.194 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:40.194 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:40.194 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:32:40.194 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369495+0000", 2024-09-14T06:32:40.194 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:40.194 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:40.194 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:40.194 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:32:40.194 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370388+0000", 2024-09-14T06:32:40.194 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00089287999999999998 2024-09-14T06:32:40.194 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:40.194 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:40.194 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:32:40.194 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370392+0000", 2024-09-14T06:32:40.195 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.2130000000000002e-06 2024-09-14T06:32:40.195 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:40.195 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:40.195 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:32:40.195 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370401+0000", 2024-09-14T06:32:40.195 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.7649999999999999e-06 2024-09-14T06:32:40.195 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:40.195 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:40.195 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:32:40.195 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370406+0000", 2024-09-14T06:32:40.195 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.0159999999999999e-06 2024-09-14T06:32:40.195 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:40.196 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:40.196 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:32:40.196 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.417764+0000", 2024-09-14T06:32:40.196 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047358348000000001 2024-09-14T06:32:40.196 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:40.196 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:40.196 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:32:40.196 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.417806+0000", 2024-09-14T06:32:40.196 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.1149999999999997e-05 2024-09-14T06:32:40.196 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:40.196 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:40.196 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:32:40.196 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.417862+0000", 2024-09-14T06:32:40.197 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.6412999999999997e-05 2024-09-14T06:32:40.197 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:40.197 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:32:40.197 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:40.197 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:40.197 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:40.197 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:53340 3.f 3:f71f51a7:::rbd_data.112dca2f6ae1.0000000000000015:17 [read 1056768~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:32:40.197 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:44.370421+0000", 2024-09-14T06:32:40.197 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 175.779096415, 2024-09-14T06:32:40.197 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047566313999999998, 2024-09-14T06:32:40.197 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:32:40.197 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:32:40.197 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:32:40.198 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:32:40.198 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:32:40.198 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:40.198 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 53340 2024-09-14T06:32:40.198 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:40.198 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:32:40.198 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:40.198 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:32:40.198 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370421+0000", 2024-09-14T06:32:40.198 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:40.198 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:40.198 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:40.199 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:32:40.199 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370421+0000", 2024-09-14T06:32:40.199 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:40.199 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:40.199 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:40.199 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:32:40.199 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370422+0000", 2024-09-14T06:32:40.199 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.3799999999999996e-07 2024-09-14T06:32:40.199 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:40.199 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:40.199 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:32:40.199 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370424+0000", 2024-09-14T06:32:40.199 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.6919999999999998e-06 2024-09-14T06:32:40.200 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:40.200 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:40.200 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:32:40.200 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370432+0000", 2024-09-14T06:32:40.200 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.2439999999999996e-06 2024-09-14T06:32:40.200 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:40.200 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:40.200 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:32:40.200 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370437+0000", 2024-09-14T06:32:40.200 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.5389999999999998e-06 2024-09-14T06:32:40.200 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:40.200 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:40.200 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:32:40.201 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.417888+0000", 2024-09-14T06:32:40.201 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047450432000000001 2024-09-14T06:32:40.201 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:40.201 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:40.201 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:32:40.201 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.417935+0000", 2024-09-14T06:32:40.201 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.7454999999999999e-05 2024-09-14T06:32:40.201 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:40.201 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:40.201 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:32:40.201 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.417987+0000", 2024-09-14T06:32:40.201 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.2213999999999998e-05 2024-09-14T06:32:40.201 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:40.202 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:32:40.202 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:40.202 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:40.202 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:40.202 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:53341 3.f 3:f71f51a7:::rbd_data.112dca2f6ae1.0000000000000015:17 [read 1081344~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:32:40.202 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:44.370446+0000", 2024-09-14T06:32:40.202 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 175.77907114199999, 2024-09-14T06:32:40.202 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047665723, 2024-09-14T06:32:40.202 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:32:40.202 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:32:40.202 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:32:40.202 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:32:40.202 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:32:40.203 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:40.203 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 53341 2024-09-14T06:32:40.203 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:40.203 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:32:40.203 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:40.203 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:32:40.203 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370446+0000", 2024-09-14T06:32:40.203 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:40.203 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:40.203 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:40.203 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:32:40.203 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370446+0000", 2024-09-14T06:32:40.203 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:40.203 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:40.204 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:40.204 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:32:40.204 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370447+0000", 2024-09-14T06:32:40.204 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.6100000000000002e-07 2024-09-14T06:32:40.204 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:40.204 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:40.204 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:32:40.204 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370448+0000", 2024-09-14T06:32:40.204 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.7370000000000001e-06 2024-09-14T06:32:40.204 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:40.204 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:40.204 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:32:40.204 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370462+0000", 2024-09-14T06:32:40.204 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.3227000000000001e-05 2024-09-14T06:32:40.205 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:40.205 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:40.205 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:32:40.205 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370467+0000", 2024-09-14T06:32:40.205 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.6540000000000004e-06 2024-09-14T06:32:40.205 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:40.205 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:40.205 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:32:40.205 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.418013+0000", 2024-09-14T06:32:40.205 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047545325999999999 2024-09-14T06:32:40.205 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:40.205 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:40.205 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:32:40.205 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.418054+0000", 2024-09-14T06:32:40.206 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.1696999999999997e-05 2024-09-14T06:32:40.206 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:40.206 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:40.206 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:32:40.206 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.418112+0000", 2024-09-14T06:32:40.206 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.7720999999999998e-05 2024-09-14T06:32:40.206 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:40.206 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:32:40.206 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:40.206 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:40.206 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:40.206 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:53342 3.f 3:f71f51a7:::rbd_data.112dca2f6ae1.0000000000000015:17 [read 1105920~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:32:40.206 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:44.370475+0000", 2024-09-14T06:32:40.206 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 175.779042081, 2024-09-14T06:32:40.207 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047748848000000003, 2024-09-14T06:32:40.207 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:32:40.207 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:32:40.207 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:32:40.207 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:32:40.207 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:32:40.207 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:40.207 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 53342 2024-09-14T06:32:40.207 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:40.207 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:32:40.207 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:40.207 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:32:40.207 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370475+0000", 2024-09-14T06:32:40.207 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:40.207 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:40.208 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:40.208 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:32:40.208 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370475+0000", 2024-09-14T06:32:40.208 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:40.208 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:40.208 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:40.208 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:32:40.208 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370476+0000", 2024-09-14T06:32:40.208 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.4700000000000002e-07 2024-09-14T06:32:40.208 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:40.208 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:40.208 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:32:40.208 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370478+0000", 2024-09-14T06:32:40.209 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.5890000000000001e-06 2024-09-14T06:32:40.209 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:40.209 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:40.209 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:32:40.209 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370486+0000", 2024-09-14T06:32:40.209 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.3470000000000001e-06 2024-09-14T06:32:40.209 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:40.209 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:40.209 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:32:40.209 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370491+0000", 2024-09-14T06:32:40.209 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.5720000000000002e-06 2024-09-14T06:32:40.209 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:40.209 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:40.210 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:32:40.210 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.418135+0000", 2024-09-14T06:32:40.210 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047644094999999997 2024-09-14T06:32:40.210 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:40.210 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:40.210 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:32:40.210 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.418174+0000", 2024-09-14T06:32:40.210 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.8204000000000002e-05 2024-09-14T06:32:40.210 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:40.210 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:40.210 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:32:40.210 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.418224+0000", 2024-09-14T06:32:40.210 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.0593999999999997e-05 2024-09-14T06:32:40.210 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:40.210 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:32:40.211 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:40.211 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:40.211 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:40.211 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:53343 3.f 3:f71f51a7:::rbd_data.112dca2f6ae1.0000000000000015:17 [read 1130496~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:32:40.211 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:44.370498+0000", 2024-09-14T06:32:40.211 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 175.779019205, 2024-09-14T06:32:40.211 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047841574999999997, 2024-09-14T06:32:40.211 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:32:40.211 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:32:40.211 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:32:40.211 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:32:40.211 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:32:40.211 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:40.211 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 53343 2024-09-14T06:32:40.212 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:40.212 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:32:40.212 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:40.212 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:32:40.212 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370498+0000", 2024-09-14T06:32:40.212 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:40.212 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:40.212 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:40.212 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:32:40.212 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370498+0000", 2024-09-14T06:32:40.212 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:40.212 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:40.212 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:40.212 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:32:40.213 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370499+0000", 2024-09-14T06:32:40.213 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.9999999999999998e-07 2024-09-14T06:32:40.213 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:40.213 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:40.213 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:32:40.213 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370501+0000", 2024-09-14T06:32:40.213 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.0310000000000001e-06 2024-09-14T06:32:40.213 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:40.213 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:40.213 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:32:40.213 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370507+0000", 2024-09-14T06:32:40.213 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.4169999999999997e-06 2024-09-14T06:32:40.213 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:40.213 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:40.214 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:32:40.214 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370516+0000", 2024-09-14T06:32:40.214 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.7190000000000007e-06 2024-09-14T06:32:40.214 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:40.214 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:40.214 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:32:40.214 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.418247+0000", 2024-09-14T06:32:40.214 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047731190999999999 2024-09-14T06:32:40.214 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:40.214 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:40.214 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:32:40.214 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.418286+0000", 2024-09-14T06:32:40.214 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.8988e-05 2024-09-14T06:32:40.214 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:40.214 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:40.215 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:32:40.215 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.418340+0000", 2024-09-14T06:32:40.215 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.3829000000000002e-05 2024-09-14T06:32:40.215 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:40.215 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:32:40.215 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:40.215 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:40.215 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:40.215 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:53344 3.f 3:f71f51a7:::rbd_data.112dca2f6ae1.0000000000000015:17 [read 1155072~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:32:40.215 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:44.370524+0000", 2024-09-14T06:32:40.215 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 175.77899343499999, 2024-09-14T06:32:40.215 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047972262000000002, 2024-09-14T06:32:40.215 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:32:40.215 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:32:40.216 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:32:40.216 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:32:40.216 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:32:40.216 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:40.216 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 53344 2024-09-14T06:32:40.216 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:40.216 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:32:40.216 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:40.216 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:32:40.216 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370524+0000", 2024-09-14T06:32:40.216 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:40.216 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:40.216 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:40.216 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:32:40.216 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370524+0000", 2024-09-14T06:32:40.217 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:40.217 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:40.217 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:40.217 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:32:40.217 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370618+0000", 2024-09-14T06:32:40.217 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.4425999999999993e-05 2024-09-14T06:32:40.217 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:40.217 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:40.217 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:32:40.217 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370622+0000", 2024-09-14T06:32:40.217 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.3519999999999998e-06 2024-09-14T06:32:40.217 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:40.217 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:40.217 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:32:40.218 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370631+0000", 2024-09-14T06:32:40.218 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.9299999999999992e-06 2024-09-14T06:32:40.218 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:40.218 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:40.218 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:32:40.218 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370637+0000", 2024-09-14T06:32:40.218 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.3269999999999997e-06 2024-09-14T06:32:40.218 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:40.218 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:40.218 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:32:40.218 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.418372+0000", 2024-09-14T06:32:40.218 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047734764999999998 2024-09-14T06:32:40.218 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:40.218 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:40.218 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:32:40.219 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.418431+0000", 2024-09-14T06:32:40.219 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.8696999999999997e-05 2024-09-14T06:32:40.219 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:40.219 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:40.219 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:32:40.219 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.418496+0000", 2024-09-14T06:32:40.219 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.5765000000000005e-05 2024-09-14T06:32:40.219 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:40.219 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:32:40.219 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:40.219 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:40.219 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:40.219 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:53345 3.f 3:f71f51a7:::rbd_data.112dca2f6ae1.0000000000000015:17 [read 1179648~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:32:40.219 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:44.370647+0000", 2024-09-14T06:32:40.220 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 175.77886996800001, 2024-09-14T06:32:40.220 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047953775999999997, 2024-09-14T06:32:40.220 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:32:40.220 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:32:40.220 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:32:40.220 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:32:40.220 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:32:40.220 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:40.220 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 53345 2024-09-14T06:32:40.220 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:40.220 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:32:40.220 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:40.220 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:32:40.220 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370647+0000", 2024-09-14T06:32:40.220 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:40.221 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:40.221 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:40.221 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:32:40.221 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370647+0000", 2024-09-14T06:32:40.221 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:40.221 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:40.221 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:40.221 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:32:40.221 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.372346+0000", 2024-09-14T06:32:40.221 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.001698284 2024-09-14T06:32:40.221 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:40.221 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:40.221 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:32:40.222 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.372351+0000", 2024-09-14T06:32:40.222 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.8890000000000001e-06 2024-09-14T06:32:40.222 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:40.222 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:40.222 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:32:40.222 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.372362+0000", 2024-09-14T06:32:40.222 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.0941000000000001e-05 2024-09-14T06:32:40.222 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:40.222 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:40.222 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:32:40.222 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.372369+0000", 2024-09-14T06:32:40.222 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.362e-06 2024-09-14T06:32:40.222 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:40.222 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:40.223 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:32:40.223 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.418510+0000", 2024-09-14T06:32:40.223 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046140914999999998 2024-09-14T06:32:40.223 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:40.223 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:40.223 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:32:40.223 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.418549+0000", 2024-09-14T06:32:40.223 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.8717999999999998e-05 2024-09-14T06:32:40.223 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:40.223 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:40.223 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:32:40.223 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.418601+0000", 2024-09-14T06:32:40.223 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.2667000000000002e-05 2024-09-14T06:32:40.223 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:40.224 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:32:40.224 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:40.224 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:40.224 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:40.224 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:53353 3.f 3:f71f51a7:::rbd_data.112dca2f6ae1.0000000000000015:17 [read 1376256~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:32:40.224 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:44.372545+0000", 2024-09-14T06:32:40.224 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 175.77697289299999, 2024-09-14T06:32:40.224 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047364232999999999, 2024-09-14T06:32:40.224 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:32:40.224 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:32:40.224 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:32:40.224 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:32:40.224 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:32:40.224 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:40.225 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 53353 2024-09-14T06:32:40.225 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:40.225 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:32:40.225 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:40.225 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:32:40.225 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.372545+0000", 2024-09-14T06:32:40.225 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:40.225 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:40.225 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:40.225 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:32:40.225 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.372545+0000", 2024-09-14T06:32:40.225 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:40.225 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:40.225 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:40.226 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:32:40.226 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.372545+0000", 2024-09-14T06:32:40.226 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.27e-07 2024-09-14T06:32:40.226 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:40.226 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:40.226 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:32:40.226 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.372547+0000", 2024-09-14T06:32:40.226 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.0600000000000002e-06 2024-09-14T06:32:40.226 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:40.226 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:40.226 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:32:40.226 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.372553+0000", 2024-09-14T06:32:40.226 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.7980000000000002e-06 2024-09-14T06:32:40.226 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:40.226 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:40.227 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:32:40.227 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.372557+0000", 2024-09-14T06:32:40.227 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.7389999999999999e-06 2024-09-14T06:32:40.227 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:40.227 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:40.227 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:32:40.227 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.419811+0000", 2024-09-14T06:32:40.227 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047253310999999999 2024-09-14T06:32:40.227 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:40.227 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:40.227 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:32:40.227 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.419854+0000", 2024-09-14T06:32:40.227 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.3448000000000001e-05 2024-09-14T06:32:40.227 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:40.228 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:40.228 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:32:40.228 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.419909+0000", 2024-09-14T06:32:40.228 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.4549999999999998e-05 2024-09-14T06:32:40.228 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:40.228 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:32:40.228 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:40.228 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:40.228 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:40.228 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:53354 3.f 3:f71f51a7:::rbd_data.112dca2f6ae1.0000000000000015:17 [read 1400832~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:32:40.228 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:44.372563+0000", 2024-09-14T06:32:40.228 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 175.776954456, 2024-09-14T06:32:40.228 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047482233999999998, 2024-09-14T06:32:40.228 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:32:40.228 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:32:40.229 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:32:40.229 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:32:40.229 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:32:40.229 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:40.229 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 53354 2024-09-14T06:32:40.229 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:40.229 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:32:40.229 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:40.229 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:32:40.229 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.372563+0000", 2024-09-14T06:32:40.229 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:40.229 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:40.229 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:40.229 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:32:40.230 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.372563+0000", 2024-09-14T06:32:40.230 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:40.230 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:40.230 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:40.230 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:32:40.230 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.375009+0000", 2024-09-14T06:32:40.230 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.0024457680000000001 2024-09-14T06:32:40.230 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:40.230 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:40.230 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:32:40.230 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.375014+0000", 2024-09-14T06:32:40.230 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.0599999999999998e-06 2024-09-14T06:32:40.230 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:40.230 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:40.231 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:32:40.231 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.375026+0000", 2024-09-14T06:32:40.231 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.1783999999999999e-05 2024-09-14T06:32:40.231 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:40.231 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:40.231 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:32:40.231 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.375035+0000", 2024-09-14T06:32:40.231 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.7799999999999995e-06 2024-09-14T06:32:40.231 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:40.231 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:40.231 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:32:40.231 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.419942+0000", 2024-09-14T06:32:40.231 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.044906982999999998 2024-09-14T06:32:40.231 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:40.232 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:40.232 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:32:40.232 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.419989+0000", 2024-09-14T06:32:40.232 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.7104999999999997e-05 2024-09-14T06:32:40.232 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:40.232 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:40.232 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:32:40.232 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.420045+0000", 2024-09-14T06:32:40.232 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.5754000000000002e-05 2024-09-14T06:32:40.232 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:40.232 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:32:40.232 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:40.232 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:40.232 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:40.233 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:225577 3.15 3:ac159a70:::rbd_header.112daa9ec306:head [watch reconnect cookie 140616827151664 gen 4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e188)", 2024-09-14T06:32:40.233 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:31:52.733493+0000", 2024-09-14T06:32:40.233 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 47.416023992, 2024-09-14T06:32:40.233 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.0156593789999999, 2024-09-14T06:32:40.233 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:32:40.233 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:32:40.233 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:32:40.233 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:32:40.233 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:32:40.233 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:40.233 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 225577 2024-09-14T06:32:40.233 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:40.233 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:32:40.234 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:40.234 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:32:40.234 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:31:52.733493+0000", 2024-09-14T06:32:40.234 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:40.234 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:40.234 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:40.234 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:32:40.234 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:31:52.733493+0000", 2024-09-14T06:32:40.234 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:40.234 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:40.234 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:40.234 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:32:40.234 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:31:52.733495+0000", 2024-09-14T06:32:40.234 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.0059999999999999e-06 2024-09-14T06:32:40.234 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:40.235 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:40.235 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:32:40.235 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:31:52.733504+0000", 2024-09-14T06:32:40.235 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.5490000000000006e-06 2024-09-14T06:32:40.235 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:40.235 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:40.235 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:32:40.235 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:31:52.733522+0000", 2024-09-14T06:32:40.235 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.7660999999999999e-05 2024-09-14T06:32:40.235 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:40.235 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:40.235 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:32:40.235 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:31:52.733531+0000", 2024-09-14T06:32:40.235 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.7180000000000008e-06 2024-09-14T06:32:40.235 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:40.236 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:40.236 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:32:40.236 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:31:52.733598+0000", 2024-09-14T06:32:40.236 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.6490999999999997e-05 2024-09-14T06:32:40.236 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:40.236 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:40.236 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "op must wait for map", 2024-09-14T06:32:40.236 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:31:52.733605+0000", 2024-09-14T06:32:40.236 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.6109999999999996e-06 2024-09-14T06:32:40.236 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:40.236 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:40.236 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:32:40.236 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:31:52.737908+0000", 2024-09-14T06:32:40.237 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.0043023050000000002 2024-09-14T06:32:40.237 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:40.237 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:40.237 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "waiting for peered", 2024-09-14T06:32:40.237 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:31:52.737923+0000", 2024-09-14T06:32:40.237 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.488e-05 2024-09-14T06:32:40.237 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:40.237 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:40.237 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:32:40.237 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:31:53.746776+0000", 2024-09-14T06:32:40.237 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.0088535359999999 2024-09-14T06:32:40.237 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:40.237 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:40.237 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:32:40.237 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:31:53.746878+0000", 2024-09-14T06:32:40.238 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.000101788 2024-09-14T06:32:40.238 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:40.238 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:40.238 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:32:40.238 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:31:53.749153+0000", 2024-09-14T06:32:40.238 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.0022748339999999999 2024-09-14T06:32:40.238 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:40.238 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:32:40.238 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:40.238 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:40.238 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:40.238 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:225581 3.15 3:ab407115:::rbd_object_map.112d838b9898:head [call lock.unlock in=10b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e188)", 2024-09-14T06:32:40.238 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:31:52.743472+0000", 2024-09-14T06:32:40.238 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 47.406044952999999, 2024-09-14T06:32:40.238 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.007270501, 2024-09-14T06:32:40.238 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:32:40.239 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:32:40.239 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:32:40.239 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:32:40.239 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:32:40.239 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:40.239 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 225581 2024-09-14T06:32:40.239 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:40.239 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:32:40.239 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:40.239 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:32:40.239 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:31:52.743472+0000", 2024-09-14T06:32:40.239 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:40.239 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:40.239 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:40.239 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:32:40.240 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:31:52.743472+0000", 2024-09-14T06:32:40.240 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:40.240 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:40.240 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:40.240 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:32:40.240 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:31:52.743473+0000", 2024-09-14T06:32:40.240 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.1999999999999998e-07 2024-09-14T06:32:40.240 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:40.240 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:40.240 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:32:40.240 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:31:52.743478+0000", 2024-09-14T06:32:40.240 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.515e-06 2024-09-14T06:32:40.240 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:40.240 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:40.241 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:32:40.241 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:31:52.743489+0000", 2024-09-14T06:32:40.241 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.1411e-05 2024-09-14T06:32:40.241 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:40.241 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:40.241 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:32:40.241 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:31:52.743494+0000", 2024-09-14T06:32:40.241 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.1649999999999997e-06 2024-09-14T06:32:40.241 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:40.241 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:40.241 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:32:40.241 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:31:52.743548+0000", 2024-09-14T06:32:40.241 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.3758999999999998e-05 2024-09-14T06:32:40.241 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:40.241 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:40.242 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "waiting for peered", 2024-09-14T06:32:40.242 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:31:52.743558+0000", 2024-09-14T06:32:40.242 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.0336999999999999e-05 2024-09-14T06:32:40.242 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:40.242 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:40.242 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:32:40.242 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:31:53.747716+0000", 2024-09-14T06:32:40.242 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.004157035 2024-09-14T06:32:40.242 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:40.242 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:40.242 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:32:40.242 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:31:53.747831+0000", 2024-09-14T06:32:40.242 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.000115394 2024-09-14T06:32:40.242 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:40.242 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:40.243 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "waiting for subops from 3", 2024-09-14T06:32:40.243 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:31:53.748002+0000", 2024-09-14T06:32:40.243 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00017085099999999999 2024-09-14T06:32:40.243 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:40.243 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:40.243 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:32:40.243 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:31:53.749536+0000", 2024-09-14T06:32:40.243 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.0015339990000000001 2024-09-14T06:32:40.243 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:40.243 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:40.243 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "op_commit", 2024-09-14T06:32:40.243 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:31:53.750672+0000", 2024-09-14T06:32:40.243 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.001135973 2024-09-14T06:32:40.243 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:40.243 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:40.244 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "commit_sent", 2024-09-14T06:32:40.244 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:31:53.750704+0000", 2024-09-14T06:32:40.244 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.1791000000000001e-05 2024-09-14T06:32:40.244 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:40.244 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:40.244 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:32:40.244 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:31:53.750743+0000", 2024-09-14T06:32:40.244 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.9450999999999999e-05 2024-09-14T06:32:40.244 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:40.244 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:32:40.244 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:40.244 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:40.244 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:32:40.244 INFO:teuthology.orchestra.run.smithi138.stdout:} 2024-09-14T06:32:40.246 INFO:tasks.ceph.osd.4.smithi138.stderr:2024-09-14T06:32:40.245+0000 7f6c653c8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33615) UID: 0 2024-09-14T06:32:40.246 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-14T06:32:40.346 INFO:tasks.ceph.osd.2.smithi094.stderr:2024-09-14T06:32:40.344+0000 7faf3269b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33649) UID: 0 2024-09-14T06:32:40.351 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-14T06:32:40.351 INFO:teuthology.orchestra.run.smithi094.stdout: "ops": [], 2024-09-14T06:32:40.351 INFO:teuthology.orchestra.run.smithi094.stdout: "num_ops": 0 2024-09-14T06:32:40.351 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-14T06:32:40.360 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-14T06:32:40.447 INFO:tasks.ceph.osd.3.smithi094.stderr:2024-09-14T06:32:40.445+0000 7feeeafe8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33655) UID: 0 2024-09-14T06:32:40.503 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-14T06:32:40.504 INFO:teuthology.orchestra.run.smithi094.stdout: "ops": [], 2024-09-14T06:32:40.504 INFO:teuthology.orchestra.run.smithi094.stdout: "complaint_time": 30, 2024-09-14T06:32:40.504 INFO:teuthology.orchestra.run.smithi094.stdout: "num_blocked_ops": 0 2024-09-14T06:32:40.504 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-14T06:32:40.506 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-14T06:32:40.512 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-14T06:32:40.553 INFO:tasks.ceph.osd.2.smithi094.stderr:2024-09-14T06:32:40.551+0000 7faf3269b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33649) UID: 0 2024-09-14T06:32:40.611 INFO:teuthology.orchestra.run.smithi094.stderr:reweighted osd.7 to 1 (10000) 2024-09-14T06:32:40.633 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 3 1 2024-09-14T06:32:40.636 INFO:teuthology.orchestra.run.smithi094.stdout:osd.0: {} 2024-09-14T06:32:40.636 INFO:teuthology.orchestra.run.smithi094.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-14T06:32:40.656 INFO:teuthology.orchestra.run.smithi094.stdout:osd.1: {} 2024-09-14T06:32:40.656 INFO:teuthology.orchestra.run.smithi094.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-14T06:32:40.657 INFO:tasks.ceph.osd.3.smithi094.stderr:2024-09-14T06:32:40.654+0000 7feeeafe8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33655) UID: 0 2024-09-14T06:32:40.670 INFO:teuthology.orchestra.run.smithi094.stdout:ERROR: (22) Invalid argument 2024-09-14T06:32:40.670 INFO:teuthology.orchestra.run.smithi094.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:32:40.674 INFO:teuthology.orchestra.run.smithi094.stdout:osd.2: {} 2024-09-14T06:32:40.674 INFO:teuthology.orchestra.run.smithi094.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-14T06:32:40.681 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-14T06:32:40.693 INFO:teuthology.orchestra.run.smithi094.stdout:osd.3: {} 2024-09-14T06:32:40.693 INFO:teuthology.orchestra.run.smithi094.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-14T06:32:40.713 INFO:teuthology.orchestra.run.smithi094.stdout:osd.4: {} 2024-09-14T06:32:40.713 INFO:teuthology.orchestra.run.smithi094.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-14T06:32:40.746 INFO:teuthology.orchestra.run.smithi094.stdout:osd.5: {} 2024-09-14T06:32:40.746 INFO:teuthology.orchestra.run.smithi094.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-14T06:32:40.749 INFO:tasks.ceph.osd.2.smithi094.stderr:2024-09-14T06:32:40.747+0000 7faf3269b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33649) UID: 0 2024-09-14T06:32:40.763 INFO:teuthology.orchestra.run.smithi094.stdout:osd.6: {} 2024-09-14T06:32:40.763 INFO:teuthology.orchestra.run.smithi094.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-14T06:32:40.788 INFO:teuthology.orchestra.run.smithi094.stdout:osd.7: {} 2024-09-14T06:32:40.789 INFO:teuthology.orchestra.run.smithi094.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-14T06:32:40.808 INFO:teuthology.orchestra.run.smithi094.stdout:ERROR: (22) Invalid argument 2024-09-14T06:32:40.808 INFO:teuthology.orchestra.run.smithi094.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:32:40.818 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-14T06:32:40.860 INFO:tasks.ceph.osd.2.smithi094.stderr:2024-09-14T06:32:40.857+0000 7faf3269b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33649) UID: 0 2024-09-14T06:32:40.951 INFO:tasks.ceph.osd.1.smithi094.stderr:2024-09-14T06:32:40.949+0000 7f328e47a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33641) UID: 0 2024-09-14T06:32:40.964 INFO:teuthology.orchestra.run.smithi094.stdout:ERROR: (22) Invalid argument 2024-09-14T06:32:40.964 INFO:teuthology.orchestra.run.smithi094.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:32:40.973 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-14T06:32:41.057 INFO:tasks.ceph.osd.2.smithi094.stderr:2024-09-14T06:32:41.055+0000 7faf3269b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33649) UID: 0 2024-09-14T06:32:41.117 INFO:teuthology.orchestra.run.smithi094.stdout:ERROR: (22) Invalid argument 2024-09-14T06:32:41.117 INFO:teuthology.orchestra.run.smithi094.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:32:41.126 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-14T06:32:41.167 INFO:tasks.ceph.osd.1.smithi094.stderr:2024-09-14T06:32:41.165+0000 7f328e47a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33641) UID: 0 2024-09-14T06:32:41.254 INFO:tasks.ceph.osd.1.smithi094.stderr:2024-09-14T06:32:41.252+0000 7f328e47a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33641) UID: 0 2024-09-14T06:32:41.269 INFO:teuthology.orchestra.run.smithi094.stdout:ERROR: (22) Invalid argument 2024-09-14T06:32:41.269 INFO:teuthology.orchestra.run.smithi094.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:32:41.278 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-14T06:32:41.355 INFO:tasks.ceph.osd.6.smithi138.stderr:2024-09-14T06:32:41.355+0000 7f8cd8849640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33620) UID: 0 2024-09-14T06:32:41.421 INFO:teuthology.orchestra.run.smithi094.stdout:ERROR: (22) Invalid argument 2024-09-14T06:32:41.422 INFO:teuthology.orchestra.run.smithi094.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:32:41.430 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-14T06:32:41.474 INFO:tasks.ceph.osd.1.smithi094.stderr:2024-09-14T06:32:41.472+0000 7f328e47a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33641) UID: 0 2024-09-14T06:32:41.533 INFO:teuthology.orchestra.run.smithi094.stdout:ERROR: (22) Invalid argument 2024-09-14T06:32:41.533 INFO:teuthology.orchestra.run.smithi094.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:32:41.542 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-14T06:32:41.557 INFO:tasks.ceph.osd.4.smithi138.stderr:2024-09-14T06:32:41.556+0000 7f6c653c8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33615) UID: 0 2024-09-14T06:32:41.613 INFO:teuthology.orchestra.run.smithi094.stderr:reweighted osd.3 to 1 (10000) 2024-09-14T06:32:41.630 INFO:tasks.ceph.ceph_manager.ceph:waiting for recovery to complete 2024-09-14T06:32:41.631 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-14T06:32:41.672 INFO:tasks.ceph.osd.2.smithi094.stderr:2024-09-14T06:32:41.670+0000 7faf3269b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33649) UID: 0 2024-09-14T06:32:41.699 INFO:teuthology.orchestra.run.smithi094.stdout:ERROR: (22) Invalid argument 2024-09-14T06:32:41.700 INFO:teuthology.orchestra.run.smithi094.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:32:41.711 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-14T06:32:41.795 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-14T06:32:41.797 INFO:tasks.ceph.osd.1.smithi094.stderr:2024-09-14T06:32:41.795+0000 7f328e47a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33641) UID: 0 2024-09-14T06:32:41.860 INFO:tasks.ceph.osd.7.smithi138.stderr:2024-09-14T06:32:41.859+0000 7f3c6a21d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33623) UID: 0 2024-09-14T06:32:41.863 INFO:teuthology.orchestra.run.smithi094.stdout:ERROR: (22) Invalid argument 2024-09-14T06:32:41.863 INFO:teuthology.orchestra.run.smithi094.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:32:41.872 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-14T06:32:41.961 INFO:tasks.ceph.osd.4.smithi138.stderr:2024-09-14T06:32:41.960+0000 7f6c653c8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33615) UID: 0 2024-09-14T06:32:41.977 INFO:teuthology.orchestra.run.smithi094.stdout:ERROR: (22) Invalid argument 2024-09-14T06:32:41.977 INFO:teuthology.orchestra.run.smithi094.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:32:41.983 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-14T06:32:41.984 INFO:teuthology.orchestra.run.smithi094.stderr:dumped all 2024-09-14T06:32:41.987 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-14T06:32:41.996 INFO:teuthology.orchestra.run.smithi094.stdout:{"pg_ready":true,"pg_map":{"version":350,"stamp":"2024-09-14T06:32:41.615714+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1146673686,"num_objects":1337,"num_object_clones":0,"num_object_copies":2674,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1337,"num_whiteouts":0,"num_read":235111,"num_read_kb":2180650,"num_write":36132,"num_write_kb":1479702,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5001,"num_bytes_recovered":627008899,"num_keys_recovered":523,"num_objects_omap":25,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":26444,"ondisk_log_size":26444,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":86,"num_osds":7,"num_per_pool_osds":7,"num_per_pool_omap_osds":7,"kb":660602880,"kb_used":531452,"kb_used_data":97940,"kb_used_omap":10,"kb_used_meta":433333,"kb_avail":660071428,"statfs":{"total":676457349120,"available":675913142272,"internally_reserved":0,"allocated":100290560,"data_stored":801083826,"data_compressed":49130229,"data_compressed_allocated":64483328,"data_compressed_original":778778715,"omap_allocated":11157,"internal_metadata":443733099},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-139998817,"num_objects":-81,"num_object_clones":0,"num_object_copies":-162,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-81,"num_whiteouts":0,"num_read":756,"num_read_kb":649,"num_write":187,"num_write_kb":42,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":534,"num_bytes_recovered":38905274,"num_keys_recovered":57,"num_objects_omap":-3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.001847"},"pg_stats":[{"pgid":"3.1f","version":"229'757","reported_seq":7959,"reported_epoch":232,"state":"active+clean","last_fresh":"2024-09-14T06:32:37.615314+0000","last_change":"2024-09-14T06:32:35.617295+0000","last_active":"2024-09-14T06:32:37.615314+0000","last_peered":"2024-09-14T06:32:37.615314+0000","last_clean":"2024-09-14T06:32:37.615314+0000","last_became_active":"2024-09-14T06:29:00.579883+0000","last_became_peered":"2024-09-14T06:29:00.579883+0000","last_unstale":"2024-09-14T06:32:37.615314+0000","last_undegraded":"2024-09-14T06:32:37.615314+0000","last_fullsized":"2024-09-14T06:32:37.615314+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":757,"log_dups_size":0,"ondisk_log_size":757,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T16:08:30.160203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010119669999999999,"stat_sum":{"num_bytes":42655632,"num_objects":41,"num_object_clones":0,"num_object_copies":82,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":41,"num_whiteouts":0,"num_read":8088,"num_read_kb":86571,"num_write":1085,"num_write_kb":49526,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"31","length":"1"},{"start":"33","length":"1"},{"start":"35","length":"1"},{"start":"37","length":"1"},{"start":"39","length":"1"},{"start":"3b","length":"1"},{"start":"3d","length":"1"}]},{"pgid":"3.1e","version":"235'750","reported_seq":9967,"reported_epoch":235,"state":"active+clean","last_fresh":"2024-09-14T06:32:40.713486+0000","last_change":"2024-09-14T06:32:40.629980+0000","last_active":"2024-09-14T06:32:40.713486+0000","last_peered":"2024-09-14T06:32:40.713486+0000","last_clean":"2024-09-14T06:32:40.713486+0000","last_became_active":"2024-09-14T06:32:17.679198+0000","last_became_peered":"2024-09-14T06:32:17.679198+0000","last_unstale":"2024-09-14T06:32:40.713486+0000","last_undegraded":"2024-09-14T06:32:40.713486+0000","last_fullsized":"2024-09-14T06:32:40.713486+0000","mapping_epoch":211,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":212,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":750,"log_dups_size":0,"ondisk_log_size":750,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T12:02:59.537260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0025465459999999998,"stat_sum":{"num_bytes":28440871,"num_objects":32,"num_object_clones":0,"num_object_copies":64,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":32,"num_whiteouts":0,"num_read":8310,"num_read_kb":67655,"num_write":993,"num_write_kb":40407,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":713,"num_bytes_recovered":87148703,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2d","length":"1"},{"start":"2f","length":"1"},{"start":"31","length":"1"},{"start":"33","length":"1"}]},{"pgid":"3.1d","version":"232'990","reported_seq":7153,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-09-14T06:32:38.770178+0000","last_change":"2024-09-14T06:32:38.770453+0000","last_active":"2024-09-14T06:32:38.770178+0000","last_peered":"2024-09-14T06:32:38.770178+0000","last_clean":"2024-09-14T06:32:38.770178+0000","last_became_active":"2024-09-14T06:32:24.716363+0000","last_became_peered":"2024-09-14T06:32:24.716363+0000","last_unstale":"2024-09-14T06:32:38.770178+0000","last_undegraded":"2024-09-14T06:32:38.770178+0000","last_fullsized":"2024-09-14T06:32:38.770178+0000","mapping_epoch":218,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":219,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":990,"log_dups_size":0,"ondisk_log_size":990,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T18:09:51.623186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000555775,"stat_sum":{"num_bytes":43960317,"num_objects":41,"num_object_clones":0,"num_object_copies":82,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":41,"num_whiteouts":0,"num_read":7636,"num_read_kb":64175,"num_write":1472,"num_write_kb":53276,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"2f","length":"1"},{"start":"31","length":"1"},{"start":"33","length":"1"},{"start":"35","length":"1"},{"start":"37","length":"1"},{"start":"39","length":"1"},{"start":"3b","length":"1"},{"start":"3d","length":"1"}]},{"pgid":"3.1c","version":"232'714","reported_seq":5603,"reported_epoch":233,"state":"creating+peering","last_fresh":"2024-09-14T06:32:38.181188+0000","last_change":"2024-09-14T06:32:38.842933+0000","last_active":"2024-09-14T06:32:38.842933+0000","last_peered":"2024-09-14T06:32:38.181188+0000","last_clean":"2024-09-14T06:32:34.845986+0000","last_became_active":"2024-09-14T06:32:37.633793+0000","last_became_peered":"2024-09-14T06:32:37.633793+0000","last_unstale":"2024-09-14T06:32:38.181188+0000","last_undegraded":"2024-09-14T06:32:38.181188+0000","last_fullsized":"2024-09-14T06:32:38.181188+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":219,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":714,"log_dups_size":0,"ondisk_log_size":714,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00080438500000000002,"stat_sum":{"num_bytes":17988923,"num_objects":45,"num_object_clones":0,"num_object_copies":90,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":45,"num_whiteouts":0,"num_read":4661,"num_read_kb":43543,"num_write":840,"num_write_kb":45545,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":505,"num_bytes_recovered":44145197,"num_keys_recovered":280,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"33","length":"1"},{"start":"35","length":"1"},{"start":"37","length":"1"},{"start":"39","length":"1"},{"start":"3b","length":"1"},{"start":"3d","length":"1"}]},{"pgid":"3.1b","version":"232'885","reported_seq":11185,"reported_epoch":232,"state":"active+clean","last_fresh":"2024-09-14T06:32:37.657534+0000","last_change":"2024-09-14T06:32:35.618821+0000","last_active":"2024-09-14T06:32:37.657534+0000","last_peered":"2024-09-14T06:32:37.657534+0000","last_clean":"2024-09-14T06:32:37.657534+0000","last_became_active":"2024-09-14T06:29:00.586183+0000","last_became_peered":"2024-09-14T06:29:00.586183+0000","last_unstale":"2024-09-14T06:32:37.657534+0000","last_undegraded":"2024-09-14T06:32:37.657534+0000","last_fullsized":"2024-09-14T06:32:37.657534+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":885,"log_dups_size":0,"ondisk_log_size":885,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T17:31:31.893152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0017418970000000001,"stat_sum":{"num_bytes":56512327,"num_objects":58,"num_object_clones":0,"num_object_copies":116,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":58,"num_whiteouts":0,"num_read":11822,"num_read_kb":108019,"num_write":1393,"num_write_kb":55098,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"31","length":"1"},{"start":"33","length":"1"},{"start":"35","length":"1"},{"start":"37","length":"1"},{"start":"39","length":"1"},{"start":"3b","length":"1"},{"start":"3d","length":"1"}]},{"pgid":"3.1a","version":"229'1561","reported_seq":10010,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-09-14T06:32:38.770108+0000","last_change":"2024-09-14T06:32:38.770493+0000","last_active":"2024-09-14T06:32:38.770108+0000","last_peered":"2024-09-14T06:32:38.770108+0000","last_clean":"2024-09-14T06:32:38.770108+0000","last_became_active":"2024-09-14T06:29:00.576206+0000","last_became_peered":"2024-09-14T06:29:00.576206+0000","last_unstale":"2024-09-14T06:32:38.770108+0000","last_undegraded":"2024-09-14T06:32:38.770108+0000","last_fullsized":"2024-09-14T06:32:38.770108+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":1561,"log_dups_size":0,"ondisk_log_size":1561,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T15:26:39.408140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00068763400000000001,"stat_sum":{"num_bytes":39956900,"num_objects":50,"num_object_clones":0,"num_object_copies":100,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":50,"num_whiteouts":0,"num_read":12790,"num_read_kb":82416,"num_write":2344,"num_write_kb":55665,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"2f","length":"1"},{"start":"31","length":"1"},{"start":"33","length":"1"},{"start":"35","length":"1"},{"start":"37","length":"1"},{"start":"39","length":"1"},{"start":"3b","length":"1"},{"start":"3d","length":"1"}]},{"pgid":"3.19","version":"232'506","reported_seq":6223,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-09-14T06:32:38.772136+0000","last_change":"2024-09-14T06:32:38.772406+0000","last_active":"2024-09-14T06:32:38.772136+0000","last_peered":"2024-09-14T06:32:38.772136+0000","last_clean":"2024-09-14T06:32:38.772136+0000","last_became_active":"2024-09-14T06:31:53.744800+0000","last_became_peered":"2024-09-14T06:31:53.744800+0000","last_unstale":"2024-09-14T06:32:38.772136+0000","last_undegraded":"2024-09-14T06:32:38.772136+0000","last_fullsized":"2024-09-14T06:32:38.772136+0000","mapping_epoch":188,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":189,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":506,"log_dups_size":0,"ondisk_log_size":506,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T07:55:27.874216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076022100000000005,"stat_sum":{"num_bytes":34593958,"num_objects":43,"num_object_clones":0,"num_object_copies":86,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":43,"num_whiteouts":0,"num_read":5084,"num_read_kb":67754,"num_write":708,"num_write_kb":43868,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":282,"num_bytes_recovered":41927489,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2f","length":"1"},{"start":"31","length":"1"},{"start":"33","length":"1"},{"start":"35","length":"1"},{"start":"37","length":"1"},{"start":"39","length":"1"},{"start":"3b","length":"1"},{"start":"3d","length":"1"}]},{"pgid":"3.18","version":"234'499","reported_seq":6027,"reported_epoch":235,"state":"active+clean","last_fresh":"2024-09-14T06:32:40.729078+0000","last_change":"2024-09-14T06:32:40.629026+0000","last_active":"2024-09-14T06:32:40.729078+0000","last_peered":"2024-09-14T06:32:40.729078+0000","last_clean":"2024-09-14T06:32:40.729078+0000","last_became_active":"2024-09-14T06:31:53.745159+0000","last_became_peered":"2024-09-14T06:31:53.745159+0000","last_unstale":"2024-09-14T06:32:40.729078+0000","last_undegraded":"2024-09-14T06:32:40.729078+0000","last_fullsized":"2024-09-14T06:32:40.729078+0000","mapping_epoch":188,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":189,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":499,"log_dups_size":0,"ondisk_log_size":499,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T08:21:11.501994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0016595749999999999,"stat_sum":{"num_bytes":22128117,"num_objects":35,"num_object_clones":0,"num_object_copies":70,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":35,"num_whiteouts":0,"num_read":4458,"num_read_kb":78893,"num_write":624,"num_write_kb":51576,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":310,"num_bytes_recovered":56082976,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2d","length":"1"},{"start":"2f","length":"1"},{"start":"31","length":"1"},{"start":"33","length":"1"}]},{"pgid":"3.17","version":"229'912","reported_seq":9228,"reported_epoch":232,"state":"active+clean","last_fresh":"2024-09-14T06:32:37.615378+0000","last_change":"2024-09-14T06:32:35.617536+0000","last_active":"2024-09-14T06:32:37.615378+0000","last_peered":"2024-09-14T06:32:37.615378+0000","last_clean":"2024-09-14T06:32:37.615378+0000","last_became_active":"2024-09-14T06:29:00.579714+0000","last_became_peered":"2024-09-14T06:29:00.579714+0000","last_unstale":"2024-09-14T06:32:37.615378+0000","last_undegraded":"2024-09-14T06:32:37.615378+0000","last_fullsized":"2024-09-14T06:32:37.615378+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":912,"log_dups_size":0,"ondisk_log_size":912,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T15:19:00.788559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001073657,"stat_sum":{"num_bytes":59778817,"num_objects":60,"num_object_clones":0,"num_object_copies":120,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":10280,"num_read_kb":68614,"num_write":1044,"num_write_kb":48460,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"31","length":"1"},{"start":"33","length":"1"},{"start":"35","length":"1"},{"start":"37","length":"1"},{"start":"39","length":"1"},{"start":"3b","length":"1"},{"start":"3d","length":"1"}]},{"pgid":"3.16","version":"229'1095","reported_seq":3858,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-09-14T06:32:38.775723+0000","last_change":"2024-09-14T06:32:38.775964+0000","last_active":"2024-09-14T06:32:38.775723+0000","last_peered":"2024-09-14T06:32:38.775723+0000","last_clean":"2024-09-14T06:32:38.775723+0000","last_became_active":"2024-09-14T06:32:24.718745+0000","last_became_peered":"2024-09-14T06:32:24.718745+0000","last_unstale":"2024-09-14T06:32:38.775723+0000","last_undegraded":"2024-09-14T06:32:38.775723+0000","last_fullsized":"2024-09-14T06:32:38.775723+0000","mapping_epoch":218,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":219,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":1095,"log_dups_size":0,"ondisk_log_size":1095,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T16:37:11.927791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00116008,"stat_sum":{"num_bytes":17620777,"num_objects":40,"num_object_clones":0,"num_object_copies":80,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":40,"num_whiteouts":0,"num_read":5485,"num_read_kb":38411,"num_write":1687,"num_write_kb":37242,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"2f","length":"1"},{"start":"31","length":"1"},{"start":"33","length":"1"},{"start":"35","length":"1"},{"start":"37","length":"1"},{"start":"39","length":"1"},{"start":"3b","length":"1"},{"start":"3d","length":"1"}]},{"pgid":"3.15","version":"232'585","reported_seq":6395,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-09-14T06:32:38.776125+0000","last_change":"2024-09-14T06:32:38.776355+0000","last_active":"2024-09-14T06:32:38.776125+0000","last_peered":"2024-09-14T06:32:38.776125+0000","last_clean":"2024-09-14T06:32:38.776125+0000","last_became_active":"2024-09-14T06:31:53.746091+0000","last_became_peered":"2024-09-14T06:31:53.746091+0000","last_unstale":"2024-09-14T06:32:38.776125+0000","last_undegraded":"2024-09-14T06:32:38.776125+0000","last_fullsized":"2024-09-14T06:32:38.776125+0000","mapping_epoch":188,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":189,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":585,"log_dups_size":0,"ondisk_log_size":585,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T10:43:55.843374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001230535,"stat_sum":{"num_bytes":35165078,"num_objects":41,"num_object_clones":0,"num_object_copies":82,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":41,"num_whiteouts":0,"num_read":4928,"num_read_kb":74748,"num_write":639,"num_write_kb":51190,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":271,"num_bytes_recovered":49981473,"num_keys_recovered":33,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"2f","length":"1"},{"start":"31","length":"1"},{"start":"33","length":"1"},{"start":"35","length":"1"},{"start":"37","length":"1"},{"start":"39","length":"1"},{"start":"3b","length":"1"},{"start":"3d","length":"1"}]},{"pgid":"3.14","version":"228'1077","reported_seq":5955,"reported_epoch":228,"state":"active+clean","last_fresh":"2024-09-14T06:32:33.764872+0000","last_change":"2024-09-14T06:32:33.764872+0000","last_active":"2024-09-14T06:32:33.764872+0000","last_peered":"2024-09-14T06:32:33.764872+0000","last_clean":"2024-09-14T06:32:33.764872+0000","last_became_active":"2024-09-14T06:29:00.582595+0000","last_became_peered":"2024-09-14T06:29:00.582595+0000","last_unstale":"2024-09-14T06:32:33.764872+0000","last_undegraded":"2024-09-14T06:32:33.764872+0000","last_fullsized":"2024-09-14T06:32:33.764872+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":1077,"log_dups_size":0,"ondisk_log_size":1077,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T16:53:26.698465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011374860000000001,"stat_sum":{"num_bytes":35195867,"num_objects":35,"num_object_clones":0,"num_object_copies":70,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":35,"num_whiteouts":0,"num_read":8768,"num_read_kb":46782,"num_write":1522,"num_write_kb":29823,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"35","length":"1"},{"start":"37","length":"1"},{"start":"39","length":"1"},{"start":"3b","length":"1"},{"start":"3d","length":"1"},{"start":"3f","length":"1"},{"start":"41","length":"1"},{"start":"43","length":"1"}]},{"pgid":"2.7","version":"0'0","reported_seq":84,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-09-14T06:32:38.775097+0000","last_change":"2024-09-14T06:32:01.612470+0000","last_active":"2024-09-14T06:32:38.775097+0000","last_peered":"2024-09-14T06:32:38.775097+0000","last_clean":"2024-09-14T06:32:38.775097+0000","last_became_active":"2024-09-14T06:32:01.612121+0000","last_became_peered":"2024-09-14T06:32:01.612121+0000","last_unstale":"2024-09-14T06:32:38.775097+0000","last_undegraded":"2024-09-14T06:32:38.775097+0000","last_fullsized":"2024-09-14T06:32:38.775097+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:28:50.205350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:28:50.205350+0000","last_clean_scrub_stamp":"2024-09-14T06:28:50.205350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T13:40:52.537942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.6","version":"229'784","reported_seq":8312,"reported_epoch":232,"state":"active+clean","last_fresh":"2024-09-14T06:32:37.615311+0000","last_change":"2024-09-14T06:32:35.617521+0000","last_active":"2024-09-14T06:32:37.615311+0000","last_peered":"2024-09-14T06:32:37.615311+0000","last_clean":"2024-09-14T06:32:37.615311+0000","last_became_active":"2024-09-14T06:29:00.578283+0000","last_became_peered":"2024-09-14T06:29:00.578283+0000","last_unstale":"2024-09-14T06:32:37.615311+0000","last_undegraded":"2024-09-14T06:32:37.615311+0000","last_fullsized":"2024-09-14T06:32:37.615311+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":784,"log_dups_size":0,"ondisk_log_size":784,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T06:44:25.975585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012089189999999999,"stat_sum":{"num_bytes":25415186,"num_objects":35,"num_object_clones":0,"num_object_copies":70,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":35,"num_whiteouts":0,"num_read":8373,"num_read_kb":78571,"num_write":846,"num_write_kb":48287,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"31","length":"1"},{"start":"33","length":"1"},{"start":"35","length":"1"},{"start":"37","length":"1"},{"start":"39","length":"1"},{"start":"3b","length":"1"},{"start":"3d","length":"1"}]},{"pgid":"2.6","version":"0'0","reported_seq":449,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-09-14T06:32:38.770882+0000","last_change":"2024-09-14T06:32:01.608456+0000","last_active":"2024-09-14T06:32:38.770882+0000","last_peered":"2024-09-14T06:32:38.770882+0000","last_clean":"2024-09-14T06:32:38.770882+0000","last_became_active":"2024-09-14T06:32:01.608178+0000","last_became_peered":"2024-09-14T06:32:01.608178+0000","last_unstale":"2024-09-14T06:32:38.770882+0000","last_undegraded":"2024-09-14T06:32:38.770882+0000","last_fullsized":"2024-09-14T06:32:38.770882+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:28:50.205350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:28:50.205350+0000","last_clean_scrub_stamp":"2024-09-14T06:28:50.205350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T10:16:40.480031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065018200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"235'785","reported_seq":4315,"reported_epoch":235,"state":"active+clean","last_fresh":"2024-09-14T06:32:40.628084+0000","last_change":"2024-09-14T06:32:40.628084+0000","last_active":"2024-09-14T06:32:40.628084+0000","last_peered":"2024-09-14T06:32:40.628084+0000","last_clean":"2024-09-14T06:32:40.628084+0000","last_became_active":"2024-09-14T06:31:54.747923+0000","last_became_peered":"2024-09-14T06:31:54.747923+0000","last_unstale":"2024-09-14T06:32:40.628084+0000","last_undegraded":"2024-09-14T06:32:40.628084+0000","last_fullsized":"2024-09-14T06:32:40.628084+0000","mapping_epoch":189,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":190,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":785,"log_dups_size":0,"ondisk_log_size":785,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T17:36:19.829390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00112521,"stat_sum":{"num_bytes":24555353,"num_objects":30,"num_object_clones":0,"num_object_copies":60,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":30,"num_whiteouts":0,"num_read":4414,"num_read_kb":40976,"num_write":1132,"num_write_kb":36116,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":183,"num_bytes_recovered":13912513,"num_keys_recovered":22,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2d","length":"1"},{"start":"2f","length":"1"},{"start":"31","length":"1"},{"start":"33","length":"1"}]},{"pgid":"2.5","version":"0'0","reported_seq":443,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-09-14T06:32:38.775360+0000","last_change":"2024-09-14T06:28:53.640035+0000","last_active":"2024-09-14T06:32:38.775360+0000","last_peered":"2024-09-14T06:32:38.775360+0000","last_clean":"2024-09-14T06:32:38.775360+0000","last_became_active":"2024-09-14T06:28:51.308058+0000","last_became_peered":"2024-09-14T06:28:51.308058+0000","last_unstale":"2024-09-14T06:32:38.775360+0000","last_undegraded":"2024-09-14T06:32:38.775360+0000","last_fullsized":"2024-09-14T06:32:38.775360+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:28:50.205350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:28:50.205350+0000","last_clean_scrub_stamp":"2024-09-14T06:28:50.205350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T13:35:48.544057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00037824000000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"229'652","reported_seq":3872,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-09-14T06:32:38.770678+0000","last_change":"2024-09-14T06:32:38.770948+0000","last_active":"2024-09-14T06:32:38.770678+0000","last_peered":"2024-09-14T06:32:38.770678+0000","last_clean":"2024-09-14T06:32:38.770678+0000","last_became_active":"2024-09-14T06:32:17.675470+0000","last_became_peered":"2024-09-14T06:32:17.675470+0000","last_unstale":"2024-09-14T06:32:38.770678+0000","last_undegraded":"2024-09-14T06:32:38.770678+0000","last_fullsized":"2024-09-14T06:32:38.770678+0000","mapping_epoch":211,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":212,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":652,"log_dups_size":0,"ondisk_log_size":652,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T11:31:07.192934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00058196400000000003,"stat_sum":{"num_bytes":13383727,"num_objects":36,"num_object_clones":0,"num_object_copies":72,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":36,"num_whiteouts":0,"num_read":2623,"num_read_kb":37411,"num_write":848,"num_write_kb":40855,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":331,"num_bytes_recovered":31411119,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2f","length":"1"},{"start":"31","length":"1"},{"start":"33","length":"1"},{"start":"35","length":"1"},{"start":"37","length":"1"},{"start":"39","length":"1"},{"start":"3b","length":"1"},{"start":"3d","length":"1"}]},{"pgid":"2.4","version":"0'0","reported_seq":443,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-09-14T06:32:38.770585+0000","last_change":"2024-09-14T06:28:53.226072+0000","last_active":"2024-09-14T06:32:38.770585+0000","last_peered":"2024-09-14T06:32:38.770585+0000","last_clean":"2024-09-14T06:32:38.770585+0000","last_became_active":"2024-09-14T06:28:51.308072+0000","last_became_peered":"2024-09-14T06:28:51.308072+0000","last_unstale":"2024-09-14T06:32:38.770585+0000","last_undegraded":"2024-09-14T06:32:38.770585+0000","last_fullsized":"2024-09-14T06:32:38.770585+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:28:50.205350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:28:50.205350+0000","last_clean_scrub_stamp":"2024-09-14T06:28:50.205350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T13:15:09.052328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066197399999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"229'854","reported_seq":5413,"reported_epoch":235,"state":"active+clean","last_fresh":"2024-09-14T06:32:40.628189+0000","last_change":"2024-09-14T06:32:40.628504+0000","last_active":"2024-09-14T06:32:40.628189+0000","last_peered":"2024-09-14T06:32:40.628189+0000","last_clean":"2024-09-14T06:32:40.628189+0000","last_became_active":"2024-09-14T06:32:24.716458+0000","last_became_peered":"2024-09-14T06:32:24.716458+0000","last_unstale":"2024-09-14T06:32:40.628189+0000","last_undegraded":"2024-09-14T06:32:40.628189+0000","last_fullsized":"2024-09-14T06:32:40.628189+0000","mapping_epoch":218,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":219,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":854,"log_dups_size":0,"ondisk_log_size":854,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T15:55:23.510859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001426797,"stat_sum":{"num_bytes":27242971,"num_objects":41,"num_object_clones":0,"num_object_copies":82,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":41,"num_whiteouts":0,"num_read":4510,"num_read_kb":42630,"num_write":1214,"num_write_kb":49494,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":330,"num_bytes_recovered":28798893,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2d","length":"1"},{"start":"2f","length":"1"},{"start":"31","length":"1"},{"start":"33","length":"1"}]},{"pgid":"3.3","version":"232'563","reported_seq":10394,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-09-14T06:32:38.770613+0000","last_change":"2024-09-14T06:32:38.770801+0000","last_active":"2024-09-14T06:32:38.770613+0000","last_peered":"2024-09-14T06:32:38.770613+0000","last_clean":"2024-09-14T06:32:38.770613+0000","last_became_active":"2024-09-14T06:29:00.584678+0000","last_became_peered":"2024-09-14T06:29:00.584678+0000","last_unstale":"2024-09-14T06:32:38.770613+0000","last_undegraded":"2024-09-14T06:32:38.770613+0000","last_fullsized":"2024-09-14T06:32:38.770613+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":563,"log_dups_size":0,"ondisk_log_size":563,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T18:11:37.882356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00082310700000000001,"stat_sum":{"num_bytes":49423207,"num_objects":34,"num_object_clones":0,"num_object_copies":68,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":34,"num_whiteouts":0,"num_read":9263,"num_read_kb":89033,"num_write":636,"num_write_kb":51616,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"2f","length":"1"},{"start":"31","length":"1"},{"start":"33","length":"1"},{"start":"35","length":"1"},{"start":"37","length":"1"},{"start":"39","length":"1"},{"start":"3b","length":"1"},{"start":"3d","length":"1"}]},{"pgid":"2.2","version":"22'2","reported_seq":58,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-09-14T06:32:38.770093+0000","last_change":"2024-09-14T06:32:24.716937+0000","last_active":"2024-09-14T06:32:38.770093+0000","last_peered":"2024-09-14T06:32:38.770093+0000","last_clean":"2024-09-14T06:32:38.770093+0000","last_became_active":"2024-09-14T06:32:24.716539+0000","last_became_peered":"2024-09-14T06:32:24.716539+0000","last_unstale":"2024-09-14T06:32:38.770093+0000","last_undegraded":"2024-09-14T06:32:38.770093+0000","last_fullsized":"2024-09-14T06:32:38.770093+0000","mapping_epoch":218,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":219,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:28:50.205350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:28:50.205350+0000","last_clean_scrub_stamp":"2024-09-14T06:28:50.205350+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T07:16:29.639469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"232'1062","reported_seq":6861,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-09-14T06:32:38.770343+0000","last_change":"2024-09-14T06:32:38.770573+0000","last_active":"2024-09-14T06:32:38.770343+0000","last_peered":"2024-09-14T06:32:38.770343+0000","last_clean":"2024-09-14T06:32:38.770343+0000","last_became_active":"2024-09-14T06:29:00.574873+0000","last_became_peered":"2024-09-14T06:29:00.574873+0000","last_unstale":"2024-09-14T06:32:38.770343+0000","last_undegraded":"2024-09-14T06:32:38.770343+0000","last_fullsized":"2024-09-14T06:32:38.770343+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":1062,"log_dups_size":0,"ondisk_log_size":1062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T15:03:28.889369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062675899999999995,"stat_sum":{"num_bytes":44082536,"num_objects":52,"num_object_clones":0,"num_object_copies":104,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":52,"num_whiteouts":0,"num_read":8251,"num_read_kb":85477,"num_write":1600,"num_write_kb":58700,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2f","length":"1"},{"start":"31","length":"1"},{"start":"33","length":"1"},{"start":"35","length":"1"},{"start":"37","length":"1"},{"start":"39","length":"1"},{"start":"3b","length":"1"},{"start":"3d","length":"1"}]},{"pgid":"2.1","version":"0'0","reported_seq":55,"reported_epoch":231,"state":"active+clean","last_fresh":"2024-09-14T06:32:36.614259+0000","last_change":"2024-09-14T06:32:14.665376+0000","last_active":"2024-09-14T06:32:36.614259+0000","last_peered":"2024-09-14T06:32:36.614259+0000","last_clean":"2024-09-14T06:32:36.614259+0000","last_became_active":"2024-09-14T06:32:14.665081+0000","last_became_peered":"2024-09-14T06:32:14.665081+0000","last_unstale":"2024-09-14T06:32:36.614259+0000","last_undegraded":"2024-09-14T06:32:36.614259+0000","last_fullsized":"2024-09-14T06:32:36.614259+0000","mapping_epoch":208,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":209,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:28:50.205350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:28:50.205350+0000","last_clean_scrub_stamp":"2024-09-14T06:28:50.205350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T16:35:12.869944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053403100000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1","version":"229'733","reported_seq":11492,"reported_epoch":232,"state":"active+clean","last_fresh":"2024-09-14T06:32:37.615156+0000","last_change":"2024-09-14T06:32:35.616947+0000","last_active":"2024-09-14T06:32:37.615156+0000","last_peered":"2024-09-14T06:32:37.615156+0000","last_clean":"2024-09-14T06:32:37.615156+0000","last_became_active":"2024-09-14T06:30:57.768238+0000","last_became_peered":"2024-09-14T06:30:57.768238+0000","last_unstale":"2024-09-14T06:32:37.615156+0000","last_undegraded":"2024-09-14T06:32:37.615156+0000","last_fullsized":"2024-09-14T06:32:37.615156+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":733,"log_dups_size":0,"ondisk_log_size":733,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T12:59:08.997641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00072126999999999998,"stat_sum":{"num_bytes":54607005,"num_objects":40,"num_object_clones":0,"num_object_copies":80,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":40,"num_whiteouts":0,"num_read":11049,"num_read_kb":112231,"num_write":909,"num_write_kb":47392,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":183,"num_bytes_recovered":30431994,"num_keys_recovered":22,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"31","length":"1"},{"start":"33","length":"1"},{"start":"35","length":"1"},{"start":"37","length":"1"},{"start":"39","length":"1"},{"start":"3b","length":"1"},{"start":"3d","length":"1"}]},{"pgid":"2.0","version":"0'0","reported_seq":443,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-09-14T06:32:38.774071+0000","last_change":"2024-09-14T06:28:53.640096+0000","last_active":"2024-09-14T06:32:38.774071+0000","last_peered":"2024-09-14T06:32:38.774071+0000","last_clean":"2024-09-14T06:32:38.774071+0000","last_became_active":"2024-09-14T06:28:51.223069+0000","last_became_peered":"2024-09-14T06:28:51.223069+0000","last_unstale":"2024-09-14T06:32:38.774071+0000","last_undegraded":"2024-09-14T06:32:38.774071+0000","last_fullsized":"2024-09-14T06:32:38.774071+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:28:50.205350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:28:50.205350+0000","last_clean_scrub_stamp":"2024-09-14T06:28:50.205350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T15:13:00.471509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00043714600000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"234'1598","reported_seq":7570,"reported_epoch":235,"state":"active+clean","last_fresh":"2024-09-14T06:32:40.628278+0000","last_change":"2024-09-14T06:32:40.628508+0000","last_active":"2024-09-14T06:32:40.628278+0000","last_peered":"2024-09-14T06:32:40.628278+0000","last_clean":"2024-09-14T06:32:40.628278+0000","last_became_active":"2024-09-14T06:31:54.744160+0000","last_became_peered":"2024-09-14T06:31:54.744160+0000","last_unstale":"2024-09-14T06:32:40.628278+0000","last_undegraded":"2024-09-14T06:32:40.628278+0000","last_fullsized":"2024-09-14T06:32:40.628278+0000","mapping_epoch":189,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":190,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":1598,"log_dups_size":0,"ondisk_log_size":1598,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T13:47:24.444561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001372971,"stat_sum":{"num_bytes":43675338,"num_objects":51,"num_object_clones":0,"num_object_copies":102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":51,"num_whiteouts":0,"num_read":10684,"num_read_kb":70562,"num_write":2532,"num_write_kb":53559,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":327,"num_bytes_recovered":50047063,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2d","length":"1"},{"start":"2f","length":"1"},{"start":"31","length":"1"},{"start":"33","length":"1"}]},{"pgid":"2.3","version":"20'1","reported_seq":40,"reported_epoch":231,"state":"active+clean","last_fresh":"2024-09-14T06:32:36.613863+0000","last_change":"2024-09-14T06:32:24.717893+0000","last_active":"2024-09-14T06:32:36.613863+0000","last_peered":"2024-09-14T06:32:36.613863+0000","last_clean":"2024-09-14T06:32:36.613863+0000","last_became_active":"2024-09-14T06:32:24.717658+0000","last_became_peered":"2024-09-14T06:32:24.717658+0000","last_unstale":"2024-09-14T06:32:36.613863+0000","last_undegraded":"2024-09-14T06:32:36.613863+0000","last_fullsized":"2024-09-14T06:32:36.613863+0000","mapping_epoch":218,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":219,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:28:50.205350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:28:50.205350+0000","last_clean_scrub_stamp":"2024-09-14T06:28:50.205350+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T10:36:16.650799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.0","version":"17'72","reported_seq":534,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-09-14T06:32:38.774173+0000","last_change":"2024-09-14T06:28:44.839885+0000","last_active":"2024-09-14T06:32:38.774173+0000","last_peered":"2024-09-14T06:32:38.774173+0000","last_clean":"2024-09-14T06:32:38.774173+0000","last_became_active":"2024-09-14T06:28:44.839573+0000","last_became_peered":"2024-09-14T06:28:44.839573+0000","last_unstale":"2024-09-14T06:32:38.774173+0000","last_undegraded":"2024-09-14T06:32:38.774173+0000","last_fullsized":"2024-09-14T06:32:38.774173+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:28:43.828494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:28:43.828494+0000","last_clean_scrub_stamp":"2024-09-14T06:28:43.828494+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T14:41:25.787293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.8","version":"234'611","reported_seq":6301,"reported_epoch":235,"state":"active+clean","last_fresh":"2024-09-14T06:32:40.629274+0000","last_change":"2024-09-14T06:32:40.629478+0000","last_active":"2024-09-14T06:32:40.629274+0000","last_peered":"2024-09-14T06:32:40.629274+0000","last_clean":"2024-09-14T06:32:40.629274+0000","last_became_active":"2024-09-14T06:31:55.754694+0000","last_became_peered":"2024-09-14T06:31:55.754694+0000","last_unstale":"2024-09-14T06:32:40.629274+0000","last_undegraded":"2024-09-14T06:32:40.629274+0000","last_fullsized":"2024-09-14T06:32:40.629274+0000","mapping_epoch":190,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":191,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":611,"log_dups_size":0,"ondisk_log_size":611,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T14:01:10.585303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0017852390000000001,"stat_sum":{"num_bytes":45016747,"num_objects":46,"num_object_clones":0,"num_object_copies":92,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":46,"num_whiteouts":0,"num_read":5613,"num_read_kb":90698,"num_write":859,"num_write_kb":57685,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":319,"num_bytes_recovered":60352269,"num_keys_recovered":22,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2d","length":"1"},{"start":"2f","length":"1"},{"start":"31","length":"1"},{"start":"33","length":"1"}]},{"pgid":"3.9","version":"232'548","reported_seq":4250,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-09-14T06:32:38.770678+0000","last_change":"2024-09-14T06:32:38.771289+0000","last_active":"2024-09-14T06:32:38.770678+0000","last_peered":"2024-09-14T06:32:38.770678+0000","last_clean":"2024-09-14T06:32:38.770678+0000","last_became_active":"2024-09-14T06:29:00.573179+0000","last_became_peered":"2024-09-14T06:29:00.573179+0000","last_unstale":"2024-09-14T06:32:38.770678+0000","last_undegraded":"2024-09-14T06:32:38.770678+0000","last_fullsized":"2024-09-14T06:32:38.770678+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":548,"log_dups_size":0,"ondisk_log_size":548,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T14:58:02.404316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014824910000000001,"stat_sum":{"num_bytes":19760470,"num_objects":45,"num_object_clones":0,"num_object_copies":90,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":45,"num_whiteouts":0,"num_read":3675,"num_read_kb":64290,"num_write":733,"num_write_kb":44928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"2f","length":"1"},{"start":"31","length":"1"},{"start":"33","length":"1"},{"start":"35","length":"1"},{"start":"37","length":"1"},{"start":"39","length":"1"},{"start":"3b","length":"1"},{"start":"3d","length":"1"}]},{"pgid":"3.a","version":"229'809","reported_seq":4776,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-09-14T06:32:38.770939+0000","last_change":"2024-09-14T06:32:38.771148+0000","last_active":"2024-09-14T06:32:38.770939+0000","last_peered":"2024-09-14T06:32:38.770939+0000","last_clean":"2024-09-14T06:32:38.770939+0000","last_became_active":"2024-09-14T06:32:04.617283+0000","last_became_peered":"2024-09-14T06:32:04.617283+0000","last_unstale":"2024-09-14T06:32:38.770939+0000","last_undegraded":"2024-09-14T06:32:38.770939+0000","last_fullsized":"2024-09-14T06:32:38.770939+0000","mapping_epoch":199,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":200,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":809,"log_dups_size":0,"ondisk_log_size":809,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T08:56:00.098170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00096903099999999997,"stat_sum":{"num_bytes":38682031,"num_objects":40,"num_object_clones":0,"num_object_copies":80,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":40,"num_whiteouts":0,"num_read":5155,"num_read_kb":58177,"num_write":1165,"num_write_kb":47529,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":161,"num_bytes_recovered":21723371,"num_keys_recovered":33,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"2f","length":"1"},{"start":"31","length":"1"},{"start":"33","length":"1"},{"start":"35","length":"1"},{"start":"37","length":"1"},{"start":"39","length":"1"},{"start":"3b","length":"1"},{"start":"3d","length":"1"}]},{"pgid":"3.b","version":"235'1551","reported_seq":8610,"reported_epoch":235,"state":"active+clean","last_fresh":"2024-09-14T06:32:40.629928+0000","last_change":"2024-09-14T06:32:40.629928+0000","last_active":"2024-09-14T06:32:40.629928+0000","last_peered":"2024-09-14T06:32:40.629928+0000","last_clean":"2024-09-14T06:32:40.629928+0000","last_became_active":"2024-09-14T06:31:55.756219+0000","last_became_peered":"2024-09-14T06:31:55.756219+0000","last_unstale":"2024-09-14T06:32:40.629928+0000","last_undegraded":"2024-09-14T06:32:40.629928+0000","last_fullsized":"2024-09-14T06:32:40.629928+0000","mapping_epoch":190,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":191,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":1551,"log_dups_size":0,"ondisk_log_size":1551,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T17:34:08.601295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013340330000000001,"stat_sum":{"num_bytes":21757058,"num_objects":41,"num_object_clones":0,"num_object_copies":82,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":41,"num_whiteouts":0,"num_read":12802,"num_read_kb":63928,"num_write":2413,"num_write_kb":43663,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":291,"num_bytes_recovered":37370632,"num_keys_recovered":66,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2d","length":"1"},{"start":"2f","length":"1"},{"start":"31","length":"1"},{"start":"33","length":"1"}]},{"pgid":"3.c","version":"235'573","reported_seq":6079,"reported_epoch":235,"state":"active+clean","last_fresh":"2024-09-14T06:32:40.746714+0000","last_change":"2024-09-14T06:32:40.627600+0000","last_active":"2024-09-14T06:32:40.746714+0000","last_peered":"2024-09-14T06:32:40.746714+0000","last_clean":"2024-09-14T06:32:40.746714+0000","last_became_active":"2024-09-14T06:32:24.716421+0000","last_became_peered":"2024-09-14T06:32:24.716421+0000","last_unstale":"2024-09-14T06:32:40.746714+0000","last_undegraded":"2024-09-14T06:32:40.746714+0000","last_fullsized":"2024-09-14T06:32:40.746714+0000","mapping_epoch":218,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":219,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":573,"log_dups_size":0,"ondisk_log_size":573,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T13:12:43.583648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061805300000000003,"stat_sum":{"num_bytes":26190419,"num_objects":36,"num_object_clones":0,"num_object_copies":72,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":36,"num_whiteouts":0,"num_read":4817,"num_read_kb":50405,"num_write":748,"num_write_kb":40823,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":293,"num_bytes_recovered":36712421,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2d","length":"1"},{"start":"2f","length":"1"},{"start":"31","length":"1"},{"start":"33","length":"1"}]},{"pgid":"3.d","version":"232'513","reported_seq":8972,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-09-14T06:32:38.775655+0000","last_change":"2024-09-14T06:32:38.775937+0000","last_active":"2024-09-14T06:32:38.775655+0000","last_peered":"2024-09-14T06:32:38.775655+0000","last_clean":"2024-09-14T06:32:38.775655+0000","last_became_active":"2024-09-14T06:29:00.580362+0000","last_became_peered":"2024-09-14T06:29:00.580362+0000","last_unstale":"2024-09-14T06:32:38.775655+0000","last_undegraded":"2024-09-14T06:32:38.775655+0000","last_fullsized":"2024-09-14T06:32:38.775655+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":513,"log_dups_size":0,"ondisk_log_size":513,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T10:40:58.585595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013537169999999999,"stat_sum":{"num_bytes":45657425,"num_objects":42,"num_object_clones":0,"num_object_copies":84,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":42,"num_whiteouts":0,"num_read":8026,"num_read_kb":93543,"num_write":674,"num_write_kb":41026,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"2f","length":"1"},{"start":"31","length":"1"},{"start":"33","length":"1"},{"start":"35","length":"1"},{"start":"37","length":"1"},{"start":"39","length":"1"},{"start":"3b","length":"1"},{"start":"3d","length":"1"}]},{"pgid":"3.e","version":"229'538","reported_seq":5239,"reported_epoch":232,"state":"peering","last_fresh":"2024-09-14T06:32:37.900918+0000","last_change":"2024-09-14T06:32:38.774414+0000","last_active":"2024-09-14T06:32:38.774414+0000","last_peered":"2024-09-14T06:32:37.900918+0000","last_clean":"2024-09-14T06:32:34.846079+0000","last_became_active":"2024-09-14T06:32:37.632816+0000","last_became_peered":"2024-09-14T06:32:37.632816+0000","last_unstale":"2024-09-14T06:32:37.900918+0000","last_undegraded":"2024-09-14T06:32:37.900918+0000","last_fullsized":"2024-09-14T06:32:37.617910+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":538,"log_dups_size":0,"ondisk_log_size":538,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0030274130000000001,"stat_sum":{"num_bytes":25133180,"num_objects":45,"num_object_clones":0,"num_object_copies":90,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":45,"num_whiteouts":0,"num_read":3980,"num_read_kb":61135,"num_write":708,"num_write_kb":43787,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":171,"num_bytes_recovered":10913227,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1,4],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"33","length":"1"},{"start":"35","length":"1"},{"start":"37","length":"1"},{"start":"39","length":"1"},{"start":"3b","length":"1"},{"start":"3d","length":"1"}]},{"pgid":"3.f","version":"232'663","reported_seq":5604,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-09-14T06:32:38.776055+0000","last_change":"2024-09-14T06:32:38.776260+0000","last_active":"2024-09-14T06:32:38.776055+0000","last_peered":"2024-09-14T06:32:38.776055+0000","last_clean":"2024-09-14T06:32:38.776055+0000","last_became_active":"2024-09-14T06:29:00.586491+0000","last_became_peered":"2024-09-14T06:29:00.586491+0000","last_unstale":"2024-09-14T06:32:38.776055+0000","last_undegraded":"2024-09-14T06:32:38.776055+0000","last_fullsized":"2024-09-14T06:32:38.776055+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":663,"log_dups_size":0,"ondisk_log_size":663,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T15:02:02.032487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013964019999999999,"stat_sum":{"num_bytes":17931678,"num_objects":26,"num_object_clones":0,"num_object_copies":52,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":0,"num_read":5903,"num_read_kb":45425,"num_write":794,"num_write_kb":36128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"2f","length":"1"},{"start":"31","length":"1"},{"start":"33","length":"1"},{"start":"35","length":"1"},{"start":"37","length":"1"},{"start":"39","length":"1"},{"start":"3b","length":"1"},{"start":"3d","length":"1"}]},{"pgid":"3.10","version":"228'892","reported_seq":6524,"reported_epoch":232,"state":"active+clean","last_fresh":"2024-09-14T06:32:37.615102+0000","last_change":"2024-09-14T06:32:35.617047+0000","last_active":"2024-09-14T06:32:37.615102+0000","last_peered":"2024-09-14T06:32:37.615102+0000","last_clean":"2024-09-14T06:32:37.615102+0000","last_became_active":"2024-09-14T06:32:04.621617+0000","last_became_peered":"2024-09-14T06:32:04.621617+0000","last_unstale":"2024-09-14T06:32:37.615102+0000","last_undegraded":"2024-09-14T06:32:37.615102+0000","last_fullsized":"2024-09-14T06:32:37.615102+0000","mapping_epoch":199,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":200,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":892,"log_dups_size":0,"ondisk_log_size":892,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T14:19:32.601273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00081382600000000002,"stat_sum":{"num_bytes":28229344,"num_objects":36,"num_object_clones":0,"num_object_copies":72,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":36,"num_whiteouts":0,"num_read":6967,"num_read_kb":34320,"num_write":1030,"num_write_kb":36649,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":154,"num_bytes_recovered":9401812,"num_keys_recovered":45,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"31","length":"1"},{"start":"33","length":"1"},{"start":"35","length":"1"},{"start":"37","length":"1"},{"start":"39","length":"1"},{"start":"3b","length":"1"},{"start":"3d","length":"1"}]},{"pgid":"3.11","version":"227'652","reported_seq":11425,"reported_epoch":232,"state":"peering","last_fresh":"2024-09-14T06:32:38.469234+0000","last_change":"2024-09-14T06:32:38.775405+0000","last_active":"2024-09-14T06:32:38.775405+0000","last_peered":"2024-09-14T06:32:38.469234+0000","last_clean":"2024-09-14T06:32:34.841771+0000","last_became_active":"2024-09-14T06:32:37.635777+0000","last_became_peered":"2024-09-14T06:32:37.635777+0000","last_unstale":"2024-09-14T06:32:38.469234+0000","last_undegraded":"2024-09-14T06:32:38.469234+0000","last_fullsized":"2024-09-14T06:32:37.617586+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":652,"log_dups_size":0,"ondisk_log_size":652,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00055383899999999998,"stat_sum":{"num_bytes":73789373,"num_objects":52,"num_object_clones":0,"num_object_copies":104,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":52,"num_whiteouts":0,"num_read":10209,"num_read_kb":109195,"num_write":781,"num_write_kb":48297,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":177,"num_bytes_recovered":16647747,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1,4],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"33","length":"1"},{"start":"35","length":"1"},{"start":"37","length":"1"},{"start":"39","length":"1"},{"start":"3b","length":"1"},{"start":"3d","length":"1"}]},{"pgid":"3.12","version":"229'694","reported_seq":7113,"reported_epoch":232,"state":"active+clean","last_fresh":"2024-09-14T06:32:37.615180+0000","last_change":"2024-09-14T06:32:35.617225+0000","last_active":"2024-09-14T06:32:37.615180+0000","last_peered":"2024-09-14T06:32:37.615180+0000","last_clean":"2024-09-14T06:32:37.615180+0000","last_became_active":"2024-09-14T06:29:00.579826+0000","last_became_peered":"2024-09-14T06:29:00.579826+0000","last_unstale":"2024-09-14T06:32:37.615180+0000","last_undegraded":"2024-09-14T06:32:37.615180+0000","last_fullsized":"2024-09-14T06:32:37.615180+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":694,"log_dups_size":0,"ondisk_log_size":694,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T17:20:16.256422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00094657200000000002,"stat_sum":{"num_bytes":33207124,"num_objects":43,"num_object_clones":0,"num_object_copies":86,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":43,"num_whiteouts":0,"num_read":6875,"num_read_kb":58890,"num_write":862,"num_write_kb":42425,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"31","length":"1"},{"start":"33","length":"1"},{"start":"35","length":"1"},{"start":"37","length":"1"},{"start":"39","length":"1"},{"start":"3b","length":"1"},{"start":"3d","length":"1"}]},{"pgid":"3.13","version":"227'963","reported_seq":8691,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-09-14T06:32:38.774432+0000","last_change":"2024-09-14T06:32:38.774663+0000","last_active":"2024-09-14T06:32:38.774432+0000","last_peered":"2024-09-14T06:32:38.774432+0000","last_clean":"2024-09-14T06:32:38.774432+0000","last_became_active":"2024-09-14T06:29:00.586520+0000","last_became_peered":"2024-09-14T06:29:00.586520+0000","last_unstale":"2024-09-14T06:32:38.774432+0000","last_undegraded":"2024-09-14T06:32:38.774432+0000","last_fullsized":"2024-09-14T06:32:38.774432+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":963,"log_dups_size":0,"ondisk_log_size":963,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T09:17:05.867426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049198799999999995,"stat_sum":{"num_bytes":54345543,"num_objects":41,"num_object_clones":0,"num_object_copies":82,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":41,"num_whiteouts":0,"num_read":9546,"num_read_kb":66117,"num_write":1194,"num_write_kb":47437,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"2f","length":"1"},{"start":"31","length":"1"},{"start":"33","length":"1"},{"start":"35","length":"1"},{"start":"37","length":"1"},{"start":"39","length":"1"},{"start":"3b","length":"1"},{"start":"3d","length":"1"}]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":1146083299,"num_objects":1333,"num_object_clones":0,"num_object_copies":2666,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1333,"num_whiteouts":0,"num_read":235045,"num_read_kb":2180595,"num_write":36029,"num_write_kb":1478072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5001,"num_bytes_recovered":627008899,"num_keys_recovered":523,"num_objects_omap":25,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":87437312,"data_stored":858645747,"data_compressed":53183589,"data_compressed_allocated":69992448,"data_compressed_original":844987183,"omap_allocated":0,"internal_metadata":0},"log_size":26369,"ondisk_log_size":26369,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":131072,"data_stored":1180736,"data_compressed":65166,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509490,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":76548,"kb_used_data":13428,"kb_used_omap":1,"kb_used_meta":63102,"kb_avail":94295292,"statfs":{"total":96636764160,"available":96558379008,"internally_reserved":0,"allocated":13750272,"data_stored":106349888,"data_compressed":6344321,"data_compressed_allocated":8302592,"data_compressed_original":102843715,"omap_allocated":1590,"internal_metadata":64616906},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":0,"seq":64424509490,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509491,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":80552,"kb_used_data":13260,"kb_used_omap":1,"kb_used_meta":67262,"kb_avail":94291288,"statfs":{"total":96636764160,"available":96554278912,"internally_reserved":0,"allocated":13578240,"data_stored":101872488,"data_compressed":6174405,"data_compressed_allocated":8257536,"data_compressed_original":98530128,"omap_allocated":1595,"internal_metadata":68876741},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509490,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":94376,"kb_used_data":18764,"kb_used_omap":1,"kb_used_meta":75582,"kb_avail":94277464,"statfs":{"total":96636764160,"available":96540123136,"internally_reserved":0,"allocated":19214336,"data_stored":164438257,"data_compressed":10361619,"data_compressed_allocated":13443072,"data_compressed_original":160805636,"omap_allocated":1595,"internal_metadata":77396421},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509491,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":64456,"kb_used_data":15216,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94307384,"statfs":{"total":96636764160,"available":96570761216,"internally_reserved":0,"allocated":15581184,"data_stored":129195838,"data_compressed":8098621,"data_compressed_allocated":10567680,"data_compressed_original":126127129,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":14,"seq":60129542194,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":38912,"kb_used_data":7716,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94332928,"statfs":{"total":96636764160,"available":96596918272,"internally_reserved":0,"allocated":7901184,"data_stored":55608356,"data_compressed":3234230,"data_compressed_allocated":4337664,"data_compressed_original":53662503,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542193,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":86292,"kb_used_data":14844,"kb_used_omap":1,"kb_used_meta":71422,"kb_avail":94285548,"statfs":{"total":96636764160,"available":96548401152,"internally_reserved":0,"allocated":15200256,"data_stored":120701857,"data_compressed":7276960,"data_compressed_allocated":9498624,"data_compressed_original":116981124,"omap_allocated":1595,"internal_metadata":73136581},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574899,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":90316,"kb_used_data":14712,"kb_used_omap":1,"kb_used_meta":75582,"kb_avail":94281524,"statfs":{"total":96636764160,"available":96544280576,"internally_reserved":0,"allocated":15065088,"data_stored":122917142,"data_compressed":7640073,"data_compressed_allocated":10076160,"data_compressed_original":119828480,"omap_allocated":1593,"internal_metadata":77396423},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32583,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32583,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12333056,"data_stored":118738265,"data_compressed":7244377,"data_compressed_allocated":9453568,"data_compressed_original":116407684,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":13070336,"data_stored":130840294,"data_compressed":8281791,"data_compressed_allocated":10809344,"data_compressed_original":129109017,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":5111808,"data_stored":54241309,"data_compressed":3234230,"data_compressed_allocated":4337664,"data_compressed_original":53662503,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12222464,"data_stored":121506055,"data_compressed":7639096,"data_compressed_allocated":10072064,"data_compressed_original":119808000,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":16412672,"data_stored":163065033,"data_compressed":10361619,"data_compressed_allocated":13443072,"data_compressed_original":160805636,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":10772480,"data_stored":100499245,"data_compressed":6174405,"data_compressed_allocated":8257536,"data_compressed_original":98530128,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":6631424,"data_stored":65369250,"data_compressed":3936333,"data_compressed_allocated":5361664,"data_compressed_original":64393940,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":10883072,"data_stored":104386296,"data_compressed":6311738,"data_compressed_allocated":8257536,"data_compressed_original":102270275,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-14T06:32:41.999 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-14T06:32:42.062 INFO:tasks.ceph.osd.4.smithi138.stderr:2024-09-14T06:32:42.061+0000 7f6c653c8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33615) UID: 0 2024-09-14T06:32:42.092 INFO:teuthology.orchestra.run.smithi138.stdout:ERROR: (22) Invalid argument 2024-09-14T06:32:42.093 INFO:teuthology.orchestra.run.smithi138.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:32:42.101 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-14T06:32:42.162 INFO:tasks.ceph.osd.0.smithi094.stderr:2024-09-14T06:32:42.161+0000 7f07c9333640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33636) UID: 0 2024-09-14T06:32:42.244 INFO:teuthology.orchestra.run.smithi138.stdout:ERROR: (22) Invalid argument 2024-09-14T06:32:42.244 INFO:teuthology.orchestra.run.smithi138.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:32:42.253 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-14T06:32:42.263 INFO:tasks.ceph.osd.2.smithi094.stderr:2024-09-14T06:32:42.261+0000 7faf3269b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33649) UID: 0 2024-09-14T06:32:42.322 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-14T06:32:42.322 INFO:teuthology.orchestra.run.smithi094.stderr:dumped all 2024-09-14T06:32:42.334 INFO:teuthology.orchestra.run.smithi094.stdout:{"pg_ready":true,"pg_map":{"version":350,"stamp":"2024-09-14T06:32:41.615714+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1146673686,"num_objects":1337,"num_object_clones":0,"num_object_copies":2674,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1337,"num_whiteouts":0,"num_read":235111,"num_read_kb":2180650,"num_write":36132,"num_write_kb":1479702,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5001,"num_bytes_recovered":627008899,"num_keys_recovered":523,"num_objects_omap":25,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":26444,"ondisk_log_size":26444,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":86,"num_osds":7,"num_per_pool_osds":7,"num_per_pool_omap_osds":7,"kb":660602880,"kb_used":531452,"kb_used_data":97940,"kb_used_omap":10,"kb_used_meta":433333,"kb_avail":660071428,"statfs":{"total":676457349120,"available":675913142272,"internally_reserved":0,"allocated":100290560,"data_stored":801083826,"data_compressed":49130229,"data_compressed_allocated":64483328,"data_compressed_original":778778715,"omap_allocated":11157,"internal_metadata":443733099},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-139998817,"num_objects":-81,"num_object_clones":0,"num_object_copies":-162,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-81,"num_whiteouts":0,"num_read":756,"num_read_kb":649,"num_write":187,"num_write_kb":42,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":534,"num_bytes_recovered":38905274,"num_keys_recovered":57,"num_objects_omap":-3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.001847"},"pg_stats":[{"pgid":"3.1f","version":"229'757","reported_seq":7959,"reported_epoch":232,"state":"active+clean","last_fresh":"2024-09-14T06:32:37.615314+0000","last_change":"2024-09-14T06:32:35.617295+0000","last_active":"2024-09-14T06:32:37.615314+0000","last_peered":"2024-09-14T06:32:37.615314+0000","last_clean":"2024-09-14T06:32:37.615314+0000","last_became_active":"2024-09-14T06:29:00.579883+0000","last_became_peered":"2024-09-14T06:29:00.579883+0000","last_unstale":"2024-09-14T06:32:37.615314+0000","last_undegraded":"2024-09-14T06:32:37.615314+0000","last_fullsized":"2024-09-14T06:32:37.615314+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":757,"log_dups_size":0,"ondisk_log_size":757,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T16:08:30.160203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010119669999999999,"stat_sum":{"num_bytes":42655632,"num_objects":41,"num_object_clones":0,"num_object_copies":82,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":41,"num_whiteouts":0,"num_read":8088,"num_read_kb":86571,"num_write":1085,"num_write_kb":49526,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"31","length":"1"},{"start":"33","length":"1"},{"start":"35","length":"1"},{"start":"37","length":"1"},{"start":"39","length":"1"},{"start":"3b","length":"1"},{"start":"3d","length":"1"}]},{"pgid":"3.1e","version":"235'750","reported_seq":9967,"reported_epoch":235,"state":"active+clean","last_fresh":"2024-09-14T06:32:40.713486+0000","last_change":"2024-09-14T06:32:40.629980+0000","last_active":"2024-09-14T06:32:40.713486+0000","last_peered":"2024-09-14T06:32:40.713486+0000","last_clean":"2024-09-14T06:32:40.713486+0000","last_became_active":"2024-09-14T06:32:17.679198+0000","last_became_peered":"2024-09-14T06:32:17.679198+0000","last_unstale":"2024-09-14T06:32:40.713486+0000","last_undegraded":"2024-09-14T06:32:40.713486+0000","last_fullsized":"2024-09-14T06:32:40.713486+0000","mapping_epoch":211,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":212,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":750,"log_dups_size":0,"ondisk_log_size":750,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T12:02:59.537260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0025465459999999998,"stat_sum":{"num_bytes":28440871,"num_objects":32,"num_object_clones":0,"num_object_copies":64,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":32,"num_whiteouts":0,"num_read":8310,"num_read_kb":67655,"num_write":993,"num_write_kb":40407,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":713,"num_bytes_recovered":87148703,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2d","length":"1"},{"start":"2f","length":"1"},{"start":"31","length":"1"},{"start":"33","length":"1"}]},{"pgid":"3.1d","version":"232'990","reported_seq":7153,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-09-14T06:32:38.770178+0000","last_change":"2024-09-14T06:32:38.770453+0000","last_active":"2024-09-14T06:32:38.770178+0000","last_peered":"2024-09-14T06:32:38.770178+0000","last_clean":"2024-09-14T06:32:38.770178+0000","last_became_active":"2024-09-14T06:32:24.716363+0000","last_became_peered":"2024-09-14T06:32:24.716363+0000","last_unstale":"2024-09-14T06:32:38.770178+0000","last_undegraded":"2024-09-14T06:32:38.770178+0000","last_fullsized":"2024-09-14T06:32:38.770178+0000","mapping_epoch":218,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":219,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":990,"log_dups_size":0,"ondisk_log_size":990,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T18:09:51.623186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000555775,"stat_sum":{"num_bytes":43960317,"num_objects":41,"num_object_clones":0,"num_object_copies":82,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":41,"num_whiteouts":0,"num_read":7636,"num_read_kb":64175,"num_write":1472,"num_write_kb":53276,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"2f","length":"1"},{"start":"31","length":"1"},{"start":"33","length":"1"},{"start":"35","length":"1"},{"start":"37","length":"1"},{"start":"39","length":"1"},{"start":"3b","length":"1"},{"start":"3d","length":"1"}]},{"pgid":"3.1c","version":"232'714","reported_seq":5603,"reported_epoch":233,"state":"creating+peering","last_fresh":"2024-09-14T06:32:38.181188+0000","last_change":"2024-09-14T06:32:38.842933+0000","last_active":"2024-09-14T06:32:38.842933+0000","last_peered":"2024-09-14T06:32:38.181188+0000","last_clean":"2024-09-14T06:32:34.845986+0000","last_became_active":"2024-09-14T06:32:37.633793+0000","last_became_peered":"2024-09-14T06:32:37.633793+0000","last_unstale":"2024-09-14T06:32:38.181188+0000","last_undegraded":"2024-09-14T06:32:38.181188+0000","last_fullsized":"2024-09-14T06:32:38.181188+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":219,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":714,"log_dups_size":0,"ondisk_log_size":714,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00080438500000000002,"stat_sum":{"num_bytes":17988923,"num_objects":45,"num_object_clones":0,"num_object_copies":90,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":45,"num_whiteouts":0,"num_read":4661,"num_read_kb":43543,"num_write":840,"num_write_kb":45545,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":505,"num_bytes_recovered":44145197,"num_keys_recovered":280,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"33","length":"1"},{"start":"35","length":"1"},{"start":"37","length":"1"},{"start":"39","length":"1"},{"start":"3b","length":"1"},{"start":"3d","length":"1"}]},{"pgid":"3.1b","version":"232'885","reported_seq":11185,"reported_epoch":232,"state":"active+clean","last_fresh":"2024-09-14T06:32:37.657534+0000","last_change":"2024-09-14T06:32:35.618821+0000","last_active":"2024-09-14T06:32:37.657534+0000","last_peered":"2024-09-14T06:32:37.657534+0000","last_clean":"2024-09-14T06:32:37.657534+0000","last_became_active":"2024-09-14T06:29:00.586183+0000","last_became_peered":"2024-09-14T06:29:00.586183+0000","last_unstale":"2024-09-14T06:32:37.657534+0000","last_undegraded":"2024-09-14T06:32:37.657534+0000","last_fullsized":"2024-09-14T06:32:37.657534+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":885,"log_dups_size":0,"ondisk_log_size":885,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T17:31:31.893152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0017418970000000001,"stat_sum":{"num_bytes":56512327,"num_objects":58,"num_object_clones":0,"num_object_copies":116,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":58,"num_whiteouts":0,"num_read":11822,"num_read_kb":108019,"num_write":1393,"num_write_kb":55098,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"31","length":"1"},{"start":"33","length":"1"},{"start":"35","length":"1"},{"start":"37","length":"1"},{"start":"39","length":"1"},{"start":"3b","length":"1"},{"start":"3d","length":"1"}]},{"pgid":"3.1a","version":"229'1561","reported_seq":10010,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-09-14T06:32:38.770108+0000","last_change":"2024-09-14T06:32:38.770493+0000","last_active":"2024-09-14T06:32:38.770108+0000","last_peered":"2024-09-14T06:32:38.770108+0000","last_clean":"2024-09-14T06:32:38.770108+0000","last_became_active":"2024-09-14T06:29:00.576206+0000","last_became_peered":"2024-09-14T06:29:00.576206+0000","last_unstale":"2024-09-14T06:32:38.770108+0000","last_undegraded":"2024-09-14T06:32:38.770108+0000","last_fullsized":"2024-09-14T06:32:38.770108+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":1561,"log_dups_size":0,"ondisk_log_size":1561,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T15:26:39.408140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00068763400000000001,"stat_sum":{"num_bytes":39956900,"num_objects":50,"num_object_clones":0,"num_object_copies":100,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":50,"num_whiteouts":0,"num_read":12790,"num_read_kb":82416,"num_write":2344,"num_write_kb":55665,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"2f","length":"1"},{"start":"31","length":"1"},{"start":"33","length":"1"},{"start":"35","length":"1"},{"start":"37","length":"1"},{"start":"39","length":"1"},{"start":"3b","length":"1"},{"start":"3d","length":"1"}]},{"pgid":"3.19","version":"232'506","reported_seq":6223,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-09-14T06:32:38.772136+0000","last_change":"2024-09-14T06:32:38.772406+0000","last_active":"2024-09-14T06:32:38.772136+0000","last_peered":"2024-09-14T06:32:38.772136+0000","last_clean":"2024-09-14T06:32:38.772136+0000","last_became_active":"2024-09-14T06:31:53.744800+0000","last_became_peered":"2024-09-14T06:31:53.744800+0000","last_unstale":"2024-09-14T06:32:38.772136+0000","last_undegraded":"2024-09-14T06:32:38.772136+0000","last_fullsized":"2024-09-14T06:32:38.772136+0000","mapping_epoch":188,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":189,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":506,"log_dups_size":0,"ondisk_log_size":506,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T07:55:27.874216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076022100000000005,"stat_sum":{"num_bytes":34593958,"num_objects":43,"num_object_clones":0,"num_object_copies":86,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":43,"num_whiteouts":0,"num_read":5084,"num_read_kb":67754,"num_write":708,"num_write_kb":43868,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":282,"num_bytes_recovered":41927489,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2f","length":"1"},{"start":"31","length":"1"},{"start":"33","length":"1"},{"start":"35","length":"1"},{"start":"37","length":"1"},{"start":"39","length":"1"},{"start":"3b","length":"1"},{"start":"3d","length":"1"}]},{"pgid":"3.18","version":"234'499","reported_seq":6027,"reported_epoch":235,"state":"active+clean","last_fresh":"2024-09-14T06:32:40.729078+0000","last_change":"2024-09-14T06:32:40.629026+0000","last_active":"2024-09-14T06:32:40.729078+0000","last_peered":"2024-09-14T06:32:40.729078+0000","last_clean":"2024-09-14T06:32:40.729078+0000","last_became_active":"2024-09-14T06:31:53.745159+0000","last_became_peered":"2024-09-14T06:31:53.745159+0000","last_unstale":"2024-09-14T06:32:40.729078+0000","last_undegraded":"2024-09-14T06:32:40.729078+0000","last_fullsized":"2024-09-14T06:32:40.729078+0000","mapping_epoch":188,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":189,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":499,"log_dups_size":0,"ondisk_log_size":499,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T08:21:11.501994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0016595749999999999,"stat_sum":{"num_bytes":22128117,"num_objects":35,"num_object_clones":0,"num_object_copies":70,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":35,"num_whiteouts":0,"num_read":4458,"num_read_kb":78893,"num_write":624,"num_write_kb":51576,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":310,"num_bytes_recovered":56082976,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2d","length":"1"},{"start":"2f","length":"1"},{"start":"31","length":"1"},{"start":"33","length":"1"}]},{"pgid":"3.17","version":"229'912","reported_seq":9228,"reported_epoch":232,"state":"active+clean","last_fresh":"2024-09-14T06:32:37.615378+0000","last_change":"2024-09-14T06:32:35.617536+0000","last_active":"2024-09-14T06:32:37.615378+0000","last_peered":"2024-09-14T06:32:37.615378+0000","last_clean":"2024-09-14T06:32:37.615378+0000","last_became_active":"2024-09-14T06:29:00.579714+0000","last_became_peered":"2024-09-14T06:29:00.579714+0000","last_unstale":"2024-09-14T06:32:37.615378+0000","last_undegraded":"2024-09-14T06:32:37.615378+0000","last_fullsized":"2024-09-14T06:32:37.615378+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":912,"log_dups_size":0,"ondisk_log_size":912,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T15:19:00.788559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001073657,"stat_sum":{"num_bytes":59778817,"num_objects":60,"num_object_clones":0,"num_object_copies":120,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":10280,"num_read_kb":68614,"num_write":1044,"num_write_kb":48460,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"31","length":"1"},{"start":"33","length":"1"},{"start":"35","length":"1"},{"start":"37","length":"1"},{"start":"39","length":"1"},{"start":"3b","length":"1"},{"start":"3d","length":"1"}]},{"pgid":"3.16","version":"229'1095","reported_seq":3858,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-09-14T06:32:38.775723+0000","last_change":"2024-09-14T06:32:38.775964+0000","last_active":"2024-09-14T06:32:38.775723+0000","last_peered":"2024-09-14T06:32:38.775723+0000","last_clean":"2024-09-14T06:32:38.775723+0000","last_became_active":"2024-09-14T06:32:24.718745+0000","last_became_peered":"2024-09-14T06:32:24.718745+0000","last_unstale":"2024-09-14T06:32:38.775723+0000","last_undegraded":"2024-09-14T06:32:38.775723+0000","last_fullsized":"2024-09-14T06:32:38.775723+0000","mapping_epoch":218,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":219,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":1095,"log_dups_size":0,"ondisk_log_size":1095,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T16:37:11.927791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00116008,"stat_sum":{"num_bytes":17620777,"num_objects":40,"num_object_clones":0,"num_object_copies":80,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":40,"num_whiteouts":0,"num_read":5485,"num_read_kb":38411,"num_write":1687,"num_write_kb":37242,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"2f","length":"1"},{"start":"31","length":"1"},{"start":"33","length":"1"},{"start":"35","length":"1"},{"start":"37","length":"1"},{"start":"39","length":"1"},{"start":"3b","length":"1"},{"start":"3d","length":"1"}]},{"pgid":"3.15","version":"232'585","reported_seq":6395,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-09-14T06:32:38.776125+0000","last_change":"2024-09-14T06:32:38.776355+0000","last_active":"2024-09-14T06:32:38.776125+0000","last_peered":"2024-09-14T06:32:38.776125+0000","last_clean":"2024-09-14T06:32:38.776125+0000","last_became_active":"2024-09-14T06:31:53.746091+0000","last_became_peered":"2024-09-14T06:31:53.746091+0000","last_unstale":"2024-09-14T06:32:38.776125+0000","last_undegraded":"2024-09-14T06:32:38.776125+0000","last_fullsized":"2024-09-14T06:32:38.776125+0000","mapping_epoch":188,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":189,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":585,"log_dups_size":0,"ondisk_log_size":585,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T10:43:55.843374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001230535,"stat_sum":{"num_bytes":35165078,"num_objects":41,"num_object_clones":0,"num_object_copies":82,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":41,"num_whiteouts":0,"num_read":4928,"num_read_kb":74748,"num_write":639,"num_write_kb":51190,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":271,"num_bytes_recovered":49981473,"num_keys_recovered":33,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"2f","length":"1"},{"start":"31","length":"1"},{"start":"33","length":"1"},{"start":"35","length":"1"},{"start":"37","length":"1"},{"start":"39","length":"1"},{"start":"3b","length":"1"},{"start":"3d","length":"1"}]},{"pgid":"3.14","version":"228'1077","reported_seq":5955,"reported_epoch":228,"state":"active+clean","last_fresh":"2024-09-14T06:32:33.764872+0000","last_change":"2024-09-14T06:32:33.764872+0000","last_active":"2024-09-14T06:32:33.764872+0000","last_peered":"2024-09-14T06:32:33.764872+0000","last_clean":"2024-09-14T06:32:33.764872+0000","last_became_active":"2024-09-14T06:29:00.582595+0000","last_became_peered":"2024-09-14T06:29:00.582595+0000","last_unstale":"2024-09-14T06:32:33.764872+0000","last_undegraded":"2024-09-14T06:32:33.764872+0000","last_fullsized":"2024-09-14T06:32:33.764872+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":1077,"log_dups_size":0,"ondisk_log_size":1077,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T16:53:26.698465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011374860000000001,"stat_sum":{"num_bytes":35195867,"num_objects":35,"num_object_clones":0,"num_object_copies":70,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":35,"num_whiteouts":0,"num_read":8768,"num_read_kb":46782,"num_write":1522,"num_write_kb":29823,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"35","length":"1"},{"start":"37","length":"1"},{"start":"39","length":"1"},{"start":"3b","length":"1"},{"start":"3d","length":"1"},{"start":"3f","length":"1"},{"start":"41","length":"1"},{"start":"43","length":"1"}]},{"pgid":"2.7","version":"0'0","reported_seq":84,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-09-14T06:32:38.775097+0000","last_change":"2024-09-14T06:32:01.612470+0000","last_active":"2024-09-14T06:32:38.775097+0000","last_peered":"2024-09-14T06:32:38.775097+0000","last_clean":"2024-09-14T06:32:38.775097+0000","last_became_active":"2024-09-14T06:32:01.612121+0000","last_became_peered":"2024-09-14T06:32:01.612121+0000","last_unstale":"2024-09-14T06:32:38.775097+0000","last_undegraded":"2024-09-14T06:32:38.775097+0000","last_fullsized":"2024-09-14T06:32:38.775097+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:28:50.205350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:28:50.205350+0000","last_clean_scrub_stamp":"2024-09-14T06:28:50.205350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T13:40:52.537942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.6","version":"229'784","reported_seq":8312,"reported_epoch":232,"state":"active+clean","last_fresh":"2024-09-14T06:32:37.615311+0000","last_change":"2024-09-14T06:32:35.617521+0000","last_active":"2024-09-14T06:32:37.615311+0000","last_peered":"2024-09-14T06:32:37.615311+0000","last_clean":"2024-09-14T06:32:37.615311+0000","last_became_active":"2024-09-14T06:29:00.578283+0000","last_became_peered":"2024-09-14T06:29:00.578283+0000","last_unstale":"2024-09-14T06:32:37.615311+0000","last_undegraded":"2024-09-14T06:32:37.615311+0000","last_fullsized":"2024-09-14T06:32:37.615311+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":784,"log_dups_size":0,"ondisk_log_size":784,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T06:44:25.975585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012089189999999999,"stat_sum":{"num_bytes":25415186,"num_objects":35,"num_object_clones":0,"num_object_copies":70,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":35,"num_whiteouts":0,"num_read":8373,"num_read_kb":78571,"num_write":846,"num_write_kb":48287,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"31","length":"1"},{"start":"33","length":"1"},{"start":"35","length":"1"},{"start":"37","length":"1"},{"start":"39","length":"1"},{"start":"3b","length":"1"},{"start":"3d","length":"1"}]},{"pgid":"2.6","version":"0'0","reported_seq":449,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-09-14T06:32:38.770882+0000","last_change":"2024-09-14T06:32:01.608456+0000","last_active":"2024-09-14T06:32:38.770882+0000","last_peered":"2024-09-14T06:32:38.770882+0000","last_clean":"2024-09-14T06:32:38.770882+0000","last_became_active":"2024-09-14T06:32:01.608178+0000","last_became_peered":"2024-09-14T06:32:01.608178+0000","last_unstale":"2024-09-14T06:32:38.770882+0000","last_undegraded":"2024-09-14T06:32:38.770882+0000","last_fullsized":"2024-09-14T06:32:38.770882+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:28:50.205350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:28:50.205350+0000","last_clean_scrub_stamp":"2024-09-14T06:28:50.205350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T10:16:40.480031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065018200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"235'785","reported_seq":4315,"reported_epoch":235,"state":"active+clean","last_fresh":"2024-09-14T06:32:40.628084+0000","last_change":"2024-09-14T06:32:40.628084+0000","last_active":"2024-09-14T06:32:40.628084+0000","last_peered":"2024-09-14T06:32:40.628084+0000","last_clean":"2024-09-14T06:32:40.628084+0000","last_became_active":"2024-09-14T06:31:54.747923+0000","last_became_peered":"2024-09-14T06:31:54.747923+0000","last_unstale":"2024-09-14T06:32:40.628084+0000","last_undegraded":"2024-09-14T06:32:40.628084+0000","last_fullsized":"2024-09-14T06:32:40.628084+0000","mapping_epoch":189,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":190,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":785,"log_dups_size":0,"ondisk_log_size":785,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T17:36:19.829390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00112521,"stat_sum":{"num_bytes":24555353,"num_objects":30,"num_object_clones":0,"num_object_copies":60,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":30,"num_whiteouts":0,"num_read":4414,"num_read_kb":40976,"num_write":1132,"num_write_kb":36116,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":183,"num_bytes_recovered":13912513,"num_keys_recovered":22,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2d","length":"1"},{"start":"2f","length":"1"},{"start":"31","length":"1"},{"start":"33","length":"1"}]},{"pgid":"2.5","version":"0'0","reported_seq":443,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-09-14T06:32:38.775360+0000","last_change":"2024-09-14T06:28:53.640035+0000","last_active":"2024-09-14T06:32:38.775360+0000","last_peered":"2024-09-14T06:32:38.775360+0000","last_clean":"2024-09-14T06:32:38.775360+0000","last_became_active":"2024-09-14T06:28:51.308058+0000","last_became_peered":"2024-09-14T06:28:51.308058+0000","last_unstale":"2024-09-14T06:32:38.775360+0000","last_undegraded":"2024-09-14T06:32:38.775360+0000","last_fullsized":"2024-09-14T06:32:38.775360+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:28:50.205350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:28:50.205350+0000","last_clean_scrub_stamp":"2024-09-14T06:28:50.205350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T13:35:48.544057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00037824000000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"229'652","reported_seq":3872,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-09-14T06:32:38.770678+0000","last_change":"2024-09-14T06:32:38.770948+0000","last_active":"2024-09-14T06:32:38.770678+0000","last_peered":"2024-09-14T06:32:38.770678+0000","last_clean":"2024-09-14T06:32:38.770678+0000","last_became_active":"2024-09-14T06:32:17.675470+0000","last_became_peered":"2024-09-14T06:32:17.675470+0000","last_unstale":"2024-09-14T06:32:38.770678+0000","last_undegraded":"2024-09-14T06:32:38.770678+0000","last_fullsized":"2024-09-14T06:32:38.770678+0000","mapping_epoch":211,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":212,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":652,"log_dups_size":0,"ondisk_log_size":652,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T11:31:07.192934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00058196400000000003,"stat_sum":{"num_bytes":13383727,"num_objects":36,"num_object_clones":0,"num_object_copies":72,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":36,"num_whiteouts":0,"num_read":2623,"num_read_kb":37411,"num_write":848,"num_write_kb":40855,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":331,"num_bytes_recovered":31411119,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2f","length":"1"},{"start":"31","length":"1"},{"start":"33","length":"1"},{"start":"35","length":"1"},{"start":"37","length":"1"},{"start":"39","length":"1"},{"start":"3b","length":"1"},{"start":"3d","length":"1"}]},{"pgid":"2.4","version":"0'0","reported_seq":443,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-09-14T06:32:38.770585+0000","last_change":"2024-09-14T06:28:53.226072+0000","last_active":"2024-09-14T06:32:38.770585+0000","last_peered":"2024-09-14T06:32:38.770585+0000","last_clean":"2024-09-14T06:32:38.770585+0000","last_became_active":"2024-09-14T06:28:51.308072+0000","last_became_peered":"2024-09-14T06:28:51.308072+0000","last_unstale":"2024-09-14T06:32:38.770585+0000","last_undegraded":"2024-09-14T06:32:38.770585+0000","last_fullsized":"2024-09-14T06:32:38.770585+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:28:50.205350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:28:50.205350+0000","last_clean_scrub_stamp":"2024-09-14T06:28:50.205350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T13:15:09.052328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066197399999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"229'854","reported_seq":5413,"reported_epoch":235,"state":"active+clean","last_fresh":"2024-09-14T06:32:40.628189+0000","last_change":"2024-09-14T06:32:40.628504+0000","last_active":"2024-09-14T06:32:40.628189+0000","last_peered":"2024-09-14T06:32:40.628189+0000","last_clean":"2024-09-14T06:32:40.628189+0000","last_became_active":"2024-09-14T06:32:24.716458+0000","last_became_peered":"2024-09-14T06:32:24.716458+0000","last_unstale":"2024-09-14T06:32:40.628189+0000","last_undegraded":"2024-09-14T06:32:40.628189+0000","last_fullsized":"2024-09-14T06:32:40.628189+0000","mapping_epoch":218,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":219,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":854,"log_dups_size":0,"ondisk_log_size":854,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T15:55:23.510859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001426797,"stat_sum":{"num_bytes":27242971,"num_objects":41,"num_object_clones":0,"num_object_copies":82,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":41,"num_whiteouts":0,"num_read":4510,"num_read_kb":42630,"num_write":1214,"num_write_kb":49494,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":330,"num_bytes_recovered":28798893,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2d","length":"1"},{"start":"2f","length":"1"},{"start":"31","length":"1"},{"start":"33","length":"1"}]},{"pgid":"3.3","version":"232'563","reported_seq":10394,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-09-14T06:32:38.770613+0000","last_change":"2024-09-14T06:32:38.770801+0000","last_active":"2024-09-14T06:32:38.770613+0000","last_peered":"2024-09-14T06:32:38.770613+0000","last_clean":"2024-09-14T06:32:38.770613+0000","last_became_active":"2024-09-14T06:29:00.584678+0000","last_became_peered":"2024-09-14T06:29:00.584678+0000","last_unstale":"2024-09-14T06:32:38.770613+0000","last_undegraded":"2024-09-14T06:32:38.770613+0000","last_fullsized":"2024-09-14T06:32:38.770613+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":563,"log_dups_size":0,"ondisk_log_size":563,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T18:11:37.882356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00082310700000000001,"stat_sum":{"num_bytes":49423207,"num_objects":34,"num_object_clones":0,"num_object_copies":68,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":34,"num_whiteouts":0,"num_read":9263,"num_read_kb":89033,"num_write":636,"num_write_kb":51616,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"2f","length":"1"},{"start":"31","length":"1"},{"start":"33","length":"1"},{"start":"35","length":"1"},{"start":"37","length":"1"},{"start":"39","length":"1"},{"start":"3b","length":"1"},{"start":"3d","length":"1"}]},{"pgid":"2.2","version":"22'2","reported_seq":58,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-09-14T06:32:38.770093+0000","last_change":"2024-09-14T06:32:24.716937+0000","last_active":"2024-09-14T06:32:38.770093+0000","last_peered":"2024-09-14T06:32:38.770093+0000","last_clean":"2024-09-14T06:32:38.770093+0000","last_became_active":"2024-09-14T06:32:24.716539+0000","last_became_peered":"2024-09-14T06:32:24.716539+0000","last_unstale":"2024-09-14T06:32:38.770093+0000","last_undegraded":"2024-09-14T06:32:38.770093+0000","last_fullsized":"2024-09-14T06:32:38.770093+0000","mapping_epoch":218,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":219,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:28:50.205350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:28:50.205350+0000","last_clean_scrub_stamp":"2024-09-14T06:28:50.205350+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T07:16:29.639469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"232'1062","reported_seq":6861,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-09-14T06:32:38.770343+0000","last_change":"2024-09-14T06:32:38.770573+0000","last_active":"2024-09-14T06:32:38.770343+0000","last_peered":"2024-09-14T06:32:38.770343+0000","last_clean":"2024-09-14T06:32:38.770343+0000","last_became_active":"2024-09-14T06:29:00.574873+0000","last_became_peered":"2024-09-14T06:29:00.574873+0000","last_unstale":"2024-09-14T06:32:38.770343+0000","last_undegraded":"2024-09-14T06:32:38.770343+0000","last_fullsized":"2024-09-14T06:32:38.770343+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":1062,"log_dups_size":0,"ondisk_log_size":1062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T15:03:28.889369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062675899999999995,"stat_sum":{"num_bytes":44082536,"num_objects":52,"num_object_clones":0,"num_object_copies":104,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":52,"num_whiteouts":0,"num_read":8251,"num_read_kb":85477,"num_write":1600,"num_write_kb":58700,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2f","length":"1"},{"start":"31","length":"1"},{"start":"33","length":"1"},{"start":"35","length":"1"},{"start":"37","length":"1"},{"start":"39","length":"1"},{"start":"3b","length":"1"},{"start":"3d","length":"1"}]},{"pgid":"2.1","version":"0'0","reported_seq":55,"reported_epoch":231,"state":"active+clean","last_fresh":"2024-09-14T06:32:36.614259+0000","last_change":"2024-09-14T06:32:14.665376+0000","last_active":"2024-09-14T06:32:36.614259+0000","last_peered":"2024-09-14T06:32:36.614259+0000","last_clean":"2024-09-14T06:32:36.614259+0000","last_became_active":"2024-09-14T06:32:14.665081+0000","last_became_peered":"2024-09-14T06:32:14.665081+0000","last_unstale":"2024-09-14T06:32:36.614259+0000","last_undegraded":"2024-09-14T06:32:36.614259+0000","last_fullsized":"2024-09-14T06:32:36.614259+0000","mapping_epoch":208,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":209,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:28:50.205350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:28:50.205350+0000","last_clean_scrub_stamp":"2024-09-14T06:28:50.205350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T16:35:12.869944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053403100000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1","version":"229'733","reported_seq":11492,"reported_epoch":232,"state":"active+clean","last_fresh":"2024-09-14T06:32:37.615156+0000","last_change":"2024-09-14T06:32:35.616947+0000","last_active":"2024-09-14T06:32:37.615156+0000","last_peered":"2024-09-14T06:32:37.615156+0000","last_clean":"2024-09-14T06:32:37.615156+0000","last_became_active":"2024-09-14T06:30:57.768238+0000","last_became_peered":"2024-09-14T06:30:57.768238+0000","last_unstale":"2024-09-14T06:32:37.615156+0000","last_undegraded":"2024-09-14T06:32:37.615156+0000","last_fullsized":"2024-09-14T06:32:37.615156+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":733,"log_dups_size":0,"ondisk_log_size":733,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T12:59:08.997641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00072126999999999998,"stat_sum":{"num_bytes":54607005,"num_objects":40,"num_object_clones":0,"num_object_copies":80,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":40,"num_whiteouts":0,"num_read":11049,"num_read_kb":112231,"num_write":909,"num_write_kb":47392,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":183,"num_bytes_recovered":30431994,"num_keys_recovered":22,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"31","length":"1"},{"start":"33","length":"1"},{"start":"35","length":"1"},{"start":"37","length":"1"},{"start":"39","length":"1"},{"start":"3b","length":"1"},{"start":"3d","length":"1"}]},{"pgid":"2.0","version":"0'0","reported_seq":443,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-09-14T06:32:38.774071+0000","last_change":"2024-09-14T06:28:53.640096+0000","last_active":"2024-09-14T06:32:38.774071+0000","last_peered":"2024-09-14T06:32:38.774071+0000","last_clean":"2024-09-14T06:32:38.774071+0000","last_became_active":"2024-09-14T06:28:51.223069+0000","last_became_peered":"2024-09-14T06:28:51.223069+0000","last_unstale":"2024-09-14T06:32:38.774071+0000","last_undegraded":"2024-09-14T06:32:38.774071+0000","last_fullsized":"2024-09-14T06:32:38.774071+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:28:50.205350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:28:50.205350+0000","last_clean_scrub_stamp":"2024-09-14T06:28:50.205350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T15:13:00.471509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00043714600000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"234'1598","reported_seq":7570,"reported_epoch":235,"state":"active+clean","last_fresh":"2024-09-14T06:32:40.628278+0000","last_change":"2024-09-14T06:32:40.628508+0000","last_active":"2024-09-14T06:32:40.628278+0000","last_peered":"2024-09-14T06:32:40.628278+0000","last_clean":"2024-09-14T06:32:40.628278+0000","last_became_active":"2024-09-14T06:31:54.744160+0000","last_became_peered":"2024-09-14T06:31:54.744160+0000","last_unstale":"2024-09-14T06:32:40.628278+0000","last_undegraded":"2024-09-14T06:32:40.628278+0000","last_fullsized":"2024-09-14T06:32:40.628278+0000","mapping_epoch":189,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":190,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":1598,"log_dups_size":0,"ondisk_log_size":1598,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T13:47:24.444561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001372971,"stat_sum":{"num_bytes":43675338,"num_objects":51,"num_object_clones":0,"num_object_copies":102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":51,"num_whiteouts":0,"num_read":10684,"num_read_kb":70562,"num_write":2532,"num_write_kb":53559,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":327,"num_bytes_recovered":50047063,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2d","length":"1"},{"start":"2f","length":"1"},{"start":"31","length":"1"},{"start":"33","length":"1"}]},{"pgid":"2.3","version":"20'1","reported_seq":40,"reported_epoch":231,"state":"active+clean","last_fresh":"2024-09-14T06:32:36.613863+0000","last_change":"2024-09-14T06:32:24.717893+0000","last_active":"2024-09-14T06:32:36.613863+0000","last_peered":"2024-09-14T06:32:36.613863+0000","last_clean":"2024-09-14T06:32:36.613863+0000","last_became_active":"2024-09-14T06:32:24.717658+0000","last_became_peered":"2024-09-14T06:32:24.717658+0000","last_unstale":"2024-09-14T06:32:36.613863+0000","last_undegraded":"2024-09-14T06:32:36.613863+0000","last_fullsized":"2024-09-14T06:32:36.613863+0000","mapping_epoch":218,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":219,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:28:50.205350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:28:50.205350+0000","last_clean_scrub_stamp":"2024-09-14T06:28:50.205350+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T10:36:16.650799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.0","version":"17'72","reported_seq":534,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-09-14T06:32:38.774173+0000","last_change":"2024-09-14T06:28:44.839885+0000","last_active":"2024-09-14T06:32:38.774173+0000","last_peered":"2024-09-14T06:32:38.774173+0000","last_clean":"2024-09-14T06:32:38.774173+0000","last_became_active":"2024-09-14T06:28:44.839573+0000","last_became_peered":"2024-09-14T06:28:44.839573+0000","last_unstale":"2024-09-14T06:32:38.774173+0000","last_undegraded":"2024-09-14T06:32:38.774173+0000","last_fullsized":"2024-09-14T06:32:38.774173+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:28:43.828494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:28:43.828494+0000","last_clean_scrub_stamp":"2024-09-14T06:28:43.828494+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T14:41:25.787293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.8","version":"234'611","reported_seq":6301,"reported_epoch":235,"state":"active+clean","last_fresh":"2024-09-14T06:32:40.629274+0000","last_change":"2024-09-14T06:32:40.629478+0000","last_active":"2024-09-14T06:32:40.629274+0000","last_peered":"2024-09-14T06:32:40.629274+0000","last_clean":"2024-09-14T06:32:40.629274+0000","last_became_active":"2024-09-14T06:31:55.754694+0000","last_became_peered":"2024-09-14T06:31:55.754694+0000","last_unstale":"2024-09-14T06:32:40.629274+0000","last_undegraded":"2024-09-14T06:32:40.629274+0000","last_fullsized":"2024-09-14T06:32:40.629274+0000","mapping_epoch":190,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":191,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":611,"log_dups_size":0,"ondisk_log_size":611,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T14:01:10.585303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0017852390000000001,"stat_sum":{"num_bytes":45016747,"num_objects":46,"num_object_clones":0,"num_object_copies":92,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":46,"num_whiteouts":0,"num_read":5613,"num_read_kb":90698,"num_write":859,"num_write_kb":57685,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":319,"num_bytes_recovered":60352269,"num_keys_recovered":22,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2d","length":"1"},{"start":"2f","length":"1"},{"start":"31","length":"1"},{"start":"33","length":"1"}]},{"pgid":"3.9","version":"232'548","reported_seq":4250,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-09-14T06:32:38.770678+0000","last_change":"2024-09-14T06:32:38.771289+0000","last_active":"2024-09-14T06:32:38.770678+0000","last_peered":"2024-09-14T06:32:38.770678+0000","last_clean":"2024-09-14T06:32:38.770678+0000","last_became_active":"2024-09-14T06:29:00.573179+0000","last_became_peered":"2024-09-14T06:29:00.573179+0000","last_unstale":"2024-09-14T06:32:38.770678+0000","last_undegraded":"2024-09-14T06:32:38.770678+0000","last_fullsized":"2024-09-14T06:32:38.770678+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":548,"log_dups_size":0,"ondisk_log_size":548,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T14:58:02.404316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014824910000000001,"stat_sum":{"num_bytes":19760470,"num_objects":45,"num_object_clones":0,"num_object_copies":90,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":45,"num_whiteouts":0,"num_read":3675,"num_read_kb":64290,"num_write":733,"num_write_kb":44928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"2f","length":"1"},{"start":"31","length":"1"},{"start":"33","length":"1"},{"start":"35","length":"1"},{"start":"37","length":"1"},{"start":"39","length":"1"},{"start":"3b","length":"1"},{"start":"3d","length":"1"}]},{"pgid":"3.a","version":"229'809","reported_seq":4776,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-09-14T06:32:38.770939+0000","last_change":"2024-09-14T06:32:38.771148+0000","last_active":"2024-09-14T06:32:38.770939+0000","last_peered":"2024-09-14T06:32:38.770939+0000","last_clean":"2024-09-14T06:32:38.770939+0000","last_became_active":"2024-09-14T06:32:04.617283+0000","last_became_peered":"2024-09-14T06:32:04.617283+0000","last_unstale":"2024-09-14T06:32:38.770939+0000","last_undegraded":"2024-09-14T06:32:38.770939+0000","last_fullsized":"2024-09-14T06:32:38.770939+0000","mapping_epoch":199,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":200,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":809,"log_dups_size":0,"ondisk_log_size":809,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T08:56:00.098170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00096903099999999997,"stat_sum":{"num_bytes":38682031,"num_objects":40,"num_object_clones":0,"num_object_copies":80,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":40,"num_whiteouts":0,"num_read":5155,"num_read_kb":58177,"num_write":1165,"num_write_kb":47529,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":161,"num_bytes_recovered":21723371,"num_keys_recovered":33,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"2f","length":"1"},{"start":"31","length":"1"},{"start":"33","length":"1"},{"start":"35","length":"1"},{"start":"37","length":"1"},{"start":"39","length":"1"},{"start":"3b","length":"1"},{"start":"3d","length":"1"}]},{"pgid":"3.b","version":"235'1551","reported_seq":8610,"reported_epoch":235,"state":"active+clean","last_fresh":"2024-09-14T06:32:40.629928+0000","last_change":"2024-09-14T06:32:40.629928+0000","last_active":"2024-09-14T06:32:40.629928+0000","last_peered":"2024-09-14T06:32:40.629928+0000","last_clean":"2024-09-14T06:32:40.629928+0000","last_became_active":"2024-09-14T06:31:55.756219+0000","last_became_peered":"2024-09-14T06:31:55.756219+0000","last_unstale":"2024-09-14T06:32:40.629928+0000","last_undegraded":"2024-09-14T06:32:40.629928+0000","last_fullsized":"2024-09-14T06:32:40.629928+0000","mapping_epoch":190,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":191,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":1551,"log_dups_size":0,"ondisk_log_size":1551,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T17:34:08.601295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013340330000000001,"stat_sum":{"num_bytes":21757058,"num_objects":41,"num_object_clones":0,"num_object_copies":82,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":41,"num_whiteouts":0,"num_read":12802,"num_read_kb":63928,"num_write":2413,"num_write_kb":43663,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":291,"num_bytes_recovered":37370632,"num_keys_recovered":66,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2d","length":"1"},{"start":"2f","length":"1"},{"start":"31","length":"1"},{"start":"33","length":"1"}]},{"pgid":"3.c","version":"235'573","reported_seq":6079,"reported_epoch":235,"state":"active+clean","last_fresh":"2024-09-14T06:32:40.746714+0000","last_change":"2024-09-14T06:32:40.627600+0000","last_active":"2024-09-14T06:32:40.746714+0000","last_peered":"2024-09-14T06:32:40.746714+0000","last_clean":"2024-09-14T06:32:40.746714+0000","last_became_active":"2024-09-14T06:32:24.716421+0000","last_became_peered":"2024-09-14T06:32:24.716421+0000","last_unstale":"2024-09-14T06:32:40.746714+0000","last_undegraded":"2024-09-14T06:32:40.746714+0000","last_fullsized":"2024-09-14T06:32:40.746714+0000","mapping_epoch":218,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":219,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":573,"log_dups_size":0,"ondisk_log_size":573,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T13:12:43.583648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061805300000000003,"stat_sum":{"num_bytes":26190419,"num_objects":36,"num_object_clones":0,"num_object_copies":72,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":36,"num_whiteouts":0,"num_read":4817,"num_read_kb":50405,"num_write":748,"num_write_kb":40823,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":293,"num_bytes_recovered":36712421,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2d","length":"1"},{"start":"2f","length":"1"},{"start":"31","length":"1"},{"start":"33","length":"1"}]},{"pgid":"3.d","version":"232'513","reported_seq":8972,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-09-14T06:32:38.775655+0000","last_change":"2024-09-14T06:32:38.775937+0000","last_active":"2024-09-14T06:32:38.775655+0000","last_peered":"2024-09-14T06:32:38.775655+0000","last_clean":"2024-09-14T06:32:38.775655+0000","last_became_active":"2024-09-14T06:29:00.580362+0000","last_became_peered":"2024-09-14T06:29:00.580362+0000","last_unstale":"2024-09-14T06:32:38.775655+0000","last_undegraded":"2024-09-14T06:32:38.775655+0000","last_fullsized":"2024-09-14T06:32:38.775655+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":513,"log_dups_size":0,"ondisk_log_size":513,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T10:40:58.585595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013537169999999999,"stat_sum":{"num_bytes":45657425,"num_objects":42,"num_object_clones":0,"num_object_copies":84,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":42,"num_whiteouts":0,"num_read":8026,"num_read_kb":93543,"num_write":674,"num_write_kb":41026,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"2f","length":"1"},{"start":"31","length":"1"},{"start":"33","length":"1"},{"start":"35","length":"1"},{"start":"37","length":"1"},{"start":"39","length":"1"},{"start":"3b","length":"1"},{"start":"3d","length":"1"}]},{"pgid":"3.e","version":"229'538","reported_seq":5239,"reported_epoch":232,"state":"peering","last_fresh":"2024-09-14T06:32:37.900918+0000","last_change":"2024-09-14T06:32:38.774414+0000","last_active":"2024-09-14T06:32:38.774414+0000","last_peered":"2024-09-14T06:32:37.900918+0000","last_clean":"2024-09-14T06:32:34.846079+0000","last_became_active":"2024-09-14T06:32:37.632816+0000","last_became_peered":"2024-09-14T06:32:37.632816+0000","last_unstale":"2024-09-14T06:32:37.900918+0000","last_undegraded":"2024-09-14T06:32:37.900918+0000","last_fullsized":"2024-09-14T06:32:37.617910+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":538,"log_dups_size":0,"ondisk_log_size":538,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0030274130000000001,"stat_sum":{"num_bytes":25133180,"num_objects":45,"num_object_clones":0,"num_object_copies":90,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":45,"num_whiteouts":0,"num_read":3980,"num_read_kb":61135,"num_write":708,"num_write_kb":43787,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":171,"num_bytes_recovered":10913227,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1,4],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"33","length":"1"},{"start":"35","length":"1"},{"start":"37","length":"1"},{"start":"39","length":"1"},{"start":"3b","length":"1"},{"start":"3d","length":"1"}]},{"pgid":"3.f","version":"232'663","reported_seq":5604,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-09-14T06:32:38.776055+0000","last_change":"2024-09-14T06:32:38.776260+0000","last_active":"2024-09-14T06:32:38.776055+0000","last_peered":"2024-09-14T06:32:38.776055+0000","last_clean":"2024-09-14T06:32:38.776055+0000","last_became_active":"2024-09-14T06:29:00.586491+0000","last_became_peered":"2024-09-14T06:29:00.586491+0000","last_unstale":"2024-09-14T06:32:38.776055+0000","last_undegraded":"2024-09-14T06:32:38.776055+0000","last_fullsized":"2024-09-14T06:32:38.776055+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":663,"log_dups_size":0,"ondisk_log_size":663,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T15:02:02.032487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013964019999999999,"stat_sum":{"num_bytes":17931678,"num_objects":26,"num_object_clones":0,"num_object_copies":52,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":0,"num_read":5903,"num_read_kb":45425,"num_write":794,"num_write_kb":36128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"2f","length":"1"},{"start":"31","length":"1"},{"start":"33","length":"1"},{"start":"35","length":"1"},{"start":"37","length":"1"},{"start":"39","length":"1"},{"start":"3b","length":"1"},{"start":"3d","length":"1"}]},{"pgid":"3.10","version":"228'892","reported_seq":6524,"reported_epoch":232,"state":"active+clean","last_fresh":"2024-09-14T06:32:37.615102+0000","last_change":"2024-09-14T06:32:35.617047+0000","last_active":"2024-09-14T06:32:37.615102+0000","last_peered":"2024-09-14T06:32:37.615102+0000","last_clean":"2024-09-14T06:32:37.615102+0000","last_became_active":"2024-09-14T06:32:04.621617+0000","last_became_peered":"2024-09-14T06:32:04.621617+0000","last_unstale":"2024-09-14T06:32:37.615102+0000","last_undegraded":"2024-09-14T06:32:37.615102+0000","last_fullsized":"2024-09-14T06:32:37.615102+0000","mapping_epoch":199,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":200,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":892,"log_dups_size":0,"ondisk_log_size":892,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T14:19:32.601273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00081382600000000002,"stat_sum":{"num_bytes":28229344,"num_objects":36,"num_object_clones":0,"num_object_copies":72,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":36,"num_whiteouts":0,"num_read":6967,"num_read_kb":34320,"num_write":1030,"num_write_kb":36649,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":154,"num_bytes_recovered":9401812,"num_keys_recovered":45,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"31","length":"1"},{"start":"33","length":"1"},{"start":"35","length":"1"},{"start":"37","length":"1"},{"start":"39","length":"1"},{"start":"3b","length":"1"},{"start":"3d","length":"1"}]},{"pgid":"3.11","version":"227'652","reported_seq":11425,"reported_epoch":232,"state":"peering","last_fresh":"2024-09-14T06:32:38.469234+0000","last_change":"2024-09-14T06:32:38.775405+0000","last_active":"2024-09-14T06:32:38.775405+0000","last_peered":"2024-09-14T06:32:38.469234+0000","last_clean":"2024-09-14T06:32:34.841771+0000","last_became_active":"2024-09-14T06:32:37.635777+0000","last_became_peered":"2024-09-14T06:32:37.635777+0000","last_unstale":"2024-09-14T06:32:38.469234+0000","last_undegraded":"2024-09-14T06:32:38.469234+0000","last_fullsized":"2024-09-14T06:32:37.617586+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":652,"log_dups_size":0,"ondisk_log_size":652,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00055383899999999998,"stat_sum":{"num_bytes":73789373,"num_objects":52,"num_object_clones":0,"num_object_copies":104,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":52,"num_whiteouts":0,"num_read":10209,"num_read_kb":109195,"num_write":781,"num_write_kb":48297,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":177,"num_bytes_recovered":16647747,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1,4],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"33","length":"1"},{"start":"35","length":"1"},{"start":"37","length":"1"},{"start":"39","length":"1"},{"start":"3b","length":"1"},{"start":"3d","length":"1"}]},{"pgid":"3.12","version":"229'694","reported_seq":7113,"reported_epoch":232,"state":"active+clean","last_fresh":"2024-09-14T06:32:37.615180+0000","last_change":"2024-09-14T06:32:35.617225+0000","last_active":"2024-09-14T06:32:37.615180+0000","last_peered":"2024-09-14T06:32:37.615180+0000","last_clean":"2024-09-14T06:32:37.615180+0000","last_became_active":"2024-09-14T06:29:00.579826+0000","last_became_peered":"2024-09-14T06:29:00.579826+0000","last_unstale":"2024-09-14T06:32:37.615180+0000","last_undegraded":"2024-09-14T06:32:37.615180+0000","last_fullsized":"2024-09-14T06:32:37.615180+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":694,"log_dups_size":0,"ondisk_log_size":694,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T17:20:16.256422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00094657200000000002,"stat_sum":{"num_bytes":33207124,"num_objects":43,"num_object_clones":0,"num_object_copies":86,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":43,"num_whiteouts":0,"num_read":6875,"num_read_kb":58890,"num_write":862,"num_write_kb":42425,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"31","length":"1"},{"start":"33","length":"1"},{"start":"35","length":"1"},{"start":"37","length":"1"},{"start":"39","length":"1"},{"start":"3b","length":"1"},{"start":"3d","length":"1"}]},{"pgid":"3.13","version":"227'963","reported_seq":8691,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-09-14T06:32:38.774432+0000","last_change":"2024-09-14T06:32:38.774663+0000","last_active":"2024-09-14T06:32:38.774432+0000","last_peered":"2024-09-14T06:32:38.774432+0000","last_clean":"2024-09-14T06:32:38.774432+0000","last_became_active":"2024-09-14T06:29:00.586520+0000","last_became_peered":"2024-09-14T06:29:00.586520+0000","last_unstale":"2024-09-14T06:32:38.774432+0000","last_undegraded":"2024-09-14T06:32:38.774432+0000","last_fullsized":"2024-09-14T06:32:38.774432+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":963,"log_dups_size":0,"ondisk_log_size":963,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T09:17:05.867426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049198799999999995,"stat_sum":{"num_bytes":54345543,"num_objects":41,"num_object_clones":0,"num_object_copies":82,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":41,"num_whiteouts":0,"num_read":9546,"num_read_kb":66117,"num_write":1194,"num_write_kb":47437,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"2f","length":"1"},{"start":"31","length":"1"},{"start":"33","length":"1"},{"start":"35","length":"1"},{"start":"37","length":"1"},{"start":"39","length":"1"},{"start":"3b","length":"1"},{"start":"3d","length":"1"}]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":1146083299,"num_objects":1333,"num_object_clones":0,"num_object_copies":2666,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1333,"num_whiteouts":0,"num_read":235045,"num_read_kb":2180595,"num_write":36029,"num_write_kb":1478072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5001,"num_bytes_recovered":627008899,"num_keys_recovered":523,"num_objects_omap":25,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":87437312,"data_stored":858645747,"data_compressed":53183589,"data_compressed_allocated":69992448,"data_compressed_original":844987183,"omap_allocated":0,"internal_metadata":0},"log_size":26369,"ondisk_log_size":26369,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":131072,"data_stored":1180736,"data_compressed":65166,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509490,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":76548,"kb_used_data":13428,"kb_used_omap":1,"kb_used_meta":63102,"kb_avail":94295292,"statfs":{"total":96636764160,"available":96558379008,"internally_reserved":0,"allocated":13750272,"data_stored":106349888,"data_compressed":6344321,"data_compressed_allocated":8302592,"data_compressed_original":102843715,"omap_allocated":1590,"internal_metadata":64616906},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":0,"seq":64424509490,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509491,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":80552,"kb_used_data":13260,"kb_used_omap":1,"kb_used_meta":67262,"kb_avail":94291288,"statfs":{"total":96636764160,"available":96554278912,"internally_reserved":0,"allocated":13578240,"data_stored":101872488,"data_compressed":6174405,"data_compressed_allocated":8257536,"data_compressed_original":98530128,"omap_allocated":1595,"internal_metadata":68876741},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509490,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":94376,"kb_used_data":18764,"kb_used_omap":1,"kb_used_meta":75582,"kb_avail":94277464,"statfs":{"total":96636764160,"available":96540123136,"internally_reserved":0,"allocated":19214336,"data_stored":164438257,"data_compressed":10361619,"data_compressed_allocated":13443072,"data_compressed_original":160805636,"omap_allocated":1595,"internal_metadata":77396421},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509491,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":64456,"kb_used_data":15216,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94307384,"statfs":{"total":96636764160,"available":96570761216,"internally_reserved":0,"allocated":15581184,"data_stored":129195838,"data_compressed":8098621,"data_compressed_allocated":10567680,"data_compressed_original":126127129,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":14,"seq":60129542194,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":38912,"kb_used_data":7716,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94332928,"statfs":{"total":96636764160,"available":96596918272,"internally_reserved":0,"allocated":7901184,"data_stored":55608356,"data_compressed":3234230,"data_compressed_allocated":4337664,"data_compressed_original":53662503,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542193,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":86292,"kb_used_data":14844,"kb_used_omap":1,"kb_used_meta":71422,"kb_avail":94285548,"statfs":{"total":96636764160,"available":96548401152,"internally_reserved":0,"allocated":15200256,"data_stored":120701857,"data_compressed":7276960,"data_compressed_allocated":9498624,"data_compressed_original":116981124,"omap_allocated":1595,"internal_metadata":73136581},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574899,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":90316,"kb_used_data":14712,"kb_used_omap":1,"kb_used_meta":75582,"kb_avail":94281524,"statfs":{"total":96636764160,"available":96544280576,"internally_reserved":0,"allocated":15065088,"data_stored":122917142,"data_compressed":7640073,"data_compressed_allocated":10076160,"data_compressed_original":119828480,"omap_allocated":1593,"internal_metadata":77396423},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32583,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32583,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12333056,"data_stored":118738265,"data_compressed":7244377,"data_compressed_allocated":9453568,"data_compressed_original":116407684,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":13070336,"data_stored":130840294,"data_compressed":8281791,"data_compressed_allocated":10809344,"data_compressed_original":129109017,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":5111808,"data_stored":54241309,"data_compressed":3234230,"data_compressed_allocated":4337664,"data_compressed_original":53662503,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12222464,"data_stored":121506055,"data_compressed":7639096,"data_compressed_allocated":10072064,"data_compressed_original":119808000,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":16412672,"data_stored":163065033,"data_compressed":10361619,"data_compressed_allocated":13443072,"data_compressed_original":160805636,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":10772480,"data_stored":100499245,"data_compressed":6174405,"data_compressed_allocated":8257536,"data_compressed_original":98530128,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":6631424,"data_stored":65369250,"data_compressed":3936333,"data_compressed_allocated":5361664,"data_compressed_original":64393940,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":10883072,"data_stored":104386296,"data_compressed":6311738,"data_compressed_allocated":8257536,"data_compressed_original":102270275,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-14T06:32:42.336 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-14T06:32:42.357 INFO:teuthology.orchestra.run.smithi138.stdout:ERROR: (22) Invalid argument 2024-09-14T06:32:42.357 INFO:teuthology.orchestra.run.smithi138.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:32:42.365 INFO:tasks.ceph.osd.5.smithi138.stderr:2024-09-14T06:32:42.363+0000 7fe13746f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33618) UID: 0 2024-09-14T06:32:42.366 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-14T06:32:42.465 INFO:tasks.ceph.osd.2.smithi094.stderr:2024-09-14T06:32:42.463+0000 7faf3269b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33649) UID: 0 2024-09-14T06:32:42.510 INFO:teuthology.orchestra.run.smithi138.stdout:ERROR: (22) Invalid argument 2024-09-14T06:32:42.510 INFO:teuthology.orchestra.run.smithi138.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:32:42.519 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-14T06:32:42.565 INFO:tasks.ceph.osd.5.smithi138.stderr:2024-09-14T06:32:42.564+0000 7fe13746f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33618) UID: 0 2024-09-14T06:32:42.618 INFO:teuthology.orchestra.run.smithi094.stderr:noscrub is unset 2024-09-14T06:32:42.623 INFO:teuthology.orchestra.run.smithi138.stdout:ERROR: (22) Invalid argument 2024-09-14T06:32:42.623 INFO:teuthology.orchestra.run.smithi138.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:32:42.633 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-14T06:32:42.677 INFO:tasks.ceph.osd.0.smithi094.stderr:2024-09-14T06:32:42.673+0000 7f07c9333640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33636) UID: 0 2024-09-14T06:32:42.751 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-14T06:32:42.751 INFO:teuthology.orchestra.run.smithi094.stdout:{"fsid":"200ad3da-bd63-4e04-af74-efb5ca0af75e","health":{"status":"HEALTH_WARN","checks":{"OSDMAP_FLAGS":{"severity":"HEALTH_WARN","summary":{"message":"nodeep-scrub flag(s) set","count":12},"muted":false}},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":251,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":237,"num_osds":8,"num_up_osds":8,"osd_up_since":1726295322,"num_in_osds":7,"osd_in_since":1726295520,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":38},{"state_name":"peering","count":2},{"state_name":"creating+peering","count":1}],"num_pgs":41,"num_pools":3,"num_objects":1337,"data_bytes":1146673686,"bytes_used":544206848,"bytes_avail":675913142272,"bytes_total":676457349120,"inactive_pgs_ratio":0.073170728981494904,"recovering_objects_per_sec":133,"recovering_bytes_per_sec":9724443,"recovering_keys_per_sec":14,"num_objects_recovered":534,"num_bytes_recovered":38905274,"num_keys_recovered":57,"read_bytes_sec":166111,"write_bytes_sec":10749,"read_op_per_sec":188,"write_op_per_sec":46},"fsmap":{"epoch":1,"btime":"2024-09-14T06:28:30:774849+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":5,"modified":"2024-09-14T06:29:45.290986+0000","services":{}},"progress_events":{}} 2024-09-14T06:32:42.764 INFO:tasks.ceph.ceph_manager.ceph:making progress, resetting timeout 2024-09-14T06:32:42.765 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-14T06:32:42.786 INFO:teuthology.orchestra.run.smithi138.stdout:ERROR: (22) Invalid argument 2024-09-14T06:32:42.787 INFO:teuthology.orchestra.run.smithi138.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:32:42.795 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-14T06:32:42.801 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-14T06:32:42.806 INFO:tasks.ceph.osd.1.smithi094.stderr:2024-09-14T06:32:42.805+0000 7f328e47a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33641) UID: 0 2024-09-14T06:32:42.868 INFO:tasks.ceph.osd.0.smithi094.stderr:2024-09-14T06:32:42.866+0000 7f07c9333640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33636) UID: 0 2024-09-14T06:32:42.897 INFO:teuthology.orchestra.run.smithi138.stdout:ERROR: (22) Invalid argument 2024-09-14T06:32:42.897 INFO:teuthology.orchestra.run.smithi138.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:32:42.906 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-14T06:32:42.970 INFO:tasks.ceph.osd.2.smithi094.stderr:2024-09-14T06:32:42.968+0000 7faf3269b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33649) UID: 0 2024-09-14T06:32:43.007 INFO:teuthology.orchestra.run.smithi138.stdout:ERROR: (22) Invalid argument 2024-09-14T06:32:43.007 INFO:teuthology.orchestra.run.smithi138.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:32:43.015 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-14T06:32:43.071 INFO:tasks.ceph.osd.3.smithi094.stderr:2024-09-14T06:32:43.069+0000 7feeeafe8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33655) UID: 0 2024-09-14T06:32:43.092 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-14T06:32:43.092 INFO:teuthology.orchestra.run.smithi094.stderr:dumped all 2024-09-14T06:32:43.104 INFO:teuthology.orchestra.run.smithi094.stdout:{"pg_ready":true,"pg_map":{"version":351,"stamp":"2024-09-14T06:32:42.614937+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1146673686,"num_objects":1337,"num_object_clones":0,"num_object_copies":2674,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1337,"num_whiteouts":0,"num_read":235111,"num_read_kb":2180650,"num_write":36132,"num_write_kb":1479702,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5001,"num_bytes_recovered":627008899,"num_keys_recovered":523,"num_objects_omap":25,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":26444,"ondisk_log_size":26444,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":86,"num_osds":7,"num_per_pool_osds":7,"num_per_pool_omap_osds":7,"kb":660602880,"kb_used":531452,"kb_used_data":97940,"kb_used_omap":10,"kb_used_meta":433333,"kb_avail":660071428,"statfs":{"total":676457349120,"available":675913142272,"internally_reserved":0,"allocated":100290560,"data_stored":801083826,"data_compressed":49130229,"data_compressed_allocated":64483328,"data_compressed_original":778778715,"omap_allocated":11157,"internal_metadata":443733099},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-115732005,"num_objects":-56,"num_object_clones":0,"num_object_copies":-112,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-56,"num_whiteouts":0,"num_read":461,"num_read_kb":395,"num_write":129,"num_write_kb":29,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":534,"num_bytes_recovered":38905274,"num_keys_recovered":57,"num_objects_omap":-2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"3.846497"},"pg_stats":[{"pgid":"3.1f","version":"229'757","reported_seq":7959,"reported_epoch":232,"state":"active+clean","last_fresh":"2024-09-14T06:32:37.615314+0000","last_change":"2024-09-14T06:32:35.617295+0000","last_active":"2024-09-14T06:32:37.615314+0000","last_peered":"2024-09-14T06:32:37.615314+0000","last_clean":"2024-09-14T06:32:37.615314+0000","last_became_active":"2024-09-14T06:29:00.579883+0000","last_became_peered":"2024-09-14T06:29:00.579883+0000","last_unstale":"2024-09-14T06:32:37.615314+0000","last_undegraded":"2024-09-14T06:32:37.615314+0000","last_fullsized":"2024-09-14T06:32:37.615314+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":757,"log_dups_size":0,"ondisk_log_size":757,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T16:08:30.160203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010119669999999999,"stat_sum":{"num_bytes":42655632,"num_objects":41,"num_object_clones":0,"num_object_copies":82,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":41,"num_whiteouts":0,"num_read":8088,"num_read_kb":86571,"num_write":1085,"num_write_kb":49526,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"31","length":"1"},{"start":"33","length":"1"},{"start":"35","length":"1"},{"start":"37","length":"1"},{"start":"39","length":"1"},{"start":"3b","length":"1"},{"start":"3d","length":"1"}]},{"pgid":"3.1e","version":"235'750","reported_seq":9967,"reported_epoch":235,"state":"active+clean","last_fresh":"2024-09-14T06:32:40.713486+0000","last_change":"2024-09-14T06:32:40.629980+0000","last_active":"2024-09-14T06:32:40.713486+0000","last_peered":"2024-09-14T06:32:40.713486+0000","last_clean":"2024-09-14T06:32:40.713486+0000","last_became_active":"2024-09-14T06:32:17.679198+0000","last_became_peered":"2024-09-14T06:32:17.679198+0000","last_unstale":"2024-09-14T06:32:40.713486+0000","last_undegraded":"2024-09-14T06:32:40.713486+0000","last_fullsized":"2024-09-14T06:32:40.713486+0000","mapping_epoch":211,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":212,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":750,"log_dups_size":0,"ondisk_log_size":750,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T12:02:59.537260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0025465459999999998,"stat_sum":{"num_bytes":28440871,"num_objects":32,"num_object_clones":0,"num_object_copies":64,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":32,"num_whiteouts":0,"num_read":8310,"num_read_kb":67655,"num_write":993,"num_write_kb":40407,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":713,"num_bytes_recovered":87148703,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2d","length":"1"},{"start":"2f","length":"1"},{"start":"31","length":"1"},{"start":"33","length":"1"}]},{"pgid":"3.1d","version":"232'990","reported_seq":7153,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-09-14T06:32:38.770178+0000","last_change":"2024-09-14T06:32:38.770453+0000","last_active":"2024-09-14T06:32:38.770178+0000","last_peered":"2024-09-14T06:32:38.770178+0000","last_clean":"2024-09-14T06:32:38.770178+0000","last_became_active":"2024-09-14T06:32:24.716363+0000","last_became_peered":"2024-09-14T06:32:24.716363+0000","last_unstale":"2024-09-14T06:32:38.770178+0000","last_undegraded":"2024-09-14T06:32:38.770178+0000","last_fullsized":"2024-09-14T06:32:38.770178+0000","mapping_epoch":218,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":219,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":990,"log_dups_size":0,"ondisk_log_size":990,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T18:09:51.623186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000555775,"stat_sum":{"num_bytes":43960317,"num_objects":41,"num_object_clones":0,"num_object_copies":82,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":41,"num_whiteouts":0,"num_read":7636,"num_read_kb":64175,"num_write":1472,"num_write_kb":53276,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"2f","length":"1"},{"start":"31","length":"1"},{"start":"33","length":"1"},{"start":"35","length":"1"},{"start":"37","length":"1"},{"start":"39","length":"1"},{"start":"3b","length":"1"},{"start":"3d","length":"1"}]},{"pgid":"3.1c","version":"232'714","reported_seq":5603,"reported_epoch":233,"state":"creating+peering","last_fresh":"2024-09-14T06:32:38.181188+0000","last_change":"2024-09-14T06:32:38.842933+0000","last_active":"2024-09-14T06:32:38.842933+0000","last_peered":"2024-09-14T06:32:38.181188+0000","last_clean":"2024-09-14T06:32:34.845986+0000","last_became_active":"2024-09-14T06:32:37.633793+0000","last_became_peered":"2024-09-14T06:32:37.633793+0000","last_unstale":"2024-09-14T06:32:38.181188+0000","last_undegraded":"2024-09-14T06:32:38.181188+0000","last_fullsized":"2024-09-14T06:32:38.181188+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":219,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":714,"log_dups_size":0,"ondisk_log_size":714,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00080438500000000002,"stat_sum":{"num_bytes":17988923,"num_objects":45,"num_object_clones":0,"num_object_copies":90,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":45,"num_whiteouts":0,"num_read":4661,"num_read_kb":43543,"num_write":840,"num_write_kb":45545,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":505,"num_bytes_recovered":44145197,"num_keys_recovered":280,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"33","length":"1"},{"start":"35","length":"1"},{"start":"37","length":"1"},{"start":"39","length":"1"},{"start":"3b","length":"1"},{"start":"3d","length":"1"}]},{"pgid":"3.1b","version":"232'885","reported_seq":11185,"reported_epoch":232,"state":"active+clean","last_fresh":"2024-09-14T06:32:37.657534+0000","last_change":"2024-09-14T06:32:35.618821+0000","last_active":"2024-09-14T06:32:37.657534+0000","last_peered":"2024-09-14T06:32:37.657534+0000","last_clean":"2024-09-14T06:32:37.657534+0000","last_became_active":"2024-09-14T06:29:00.586183+0000","last_became_peered":"2024-09-14T06:29:00.586183+0000","last_unstale":"2024-09-14T06:32:37.657534+0000","last_undegraded":"2024-09-14T06:32:37.657534+0000","last_fullsized":"2024-09-14T06:32:37.657534+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":885,"log_dups_size":0,"ondisk_log_size":885,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T17:31:31.893152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0017418970000000001,"stat_sum":{"num_bytes":56512327,"num_objects":58,"num_object_clones":0,"num_object_copies":116,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":58,"num_whiteouts":0,"num_read":11822,"num_read_kb":108019,"num_write":1393,"num_write_kb":55098,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"31","length":"1"},{"start":"33","length":"1"},{"start":"35","length":"1"},{"start":"37","length":"1"},{"start":"39","length":"1"},{"start":"3b","length":"1"},{"start":"3d","length":"1"}]},{"pgid":"3.1a","version":"229'1561","reported_seq":10010,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-09-14T06:32:38.770108+0000","last_change":"2024-09-14T06:32:38.770493+0000","last_active":"2024-09-14T06:32:38.770108+0000","last_peered":"2024-09-14T06:32:38.770108+0000","last_clean":"2024-09-14T06:32:38.770108+0000","last_became_active":"2024-09-14T06:29:00.576206+0000","last_became_peered":"2024-09-14T06:29:00.576206+0000","last_unstale":"2024-09-14T06:32:38.770108+0000","last_undegraded":"2024-09-14T06:32:38.770108+0000","last_fullsized":"2024-09-14T06:32:38.770108+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":1561,"log_dups_size":0,"ondisk_log_size":1561,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T15:26:39.408140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00068763400000000001,"stat_sum":{"num_bytes":39956900,"num_objects":50,"num_object_clones":0,"num_object_copies":100,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":50,"num_whiteouts":0,"num_read":12790,"num_read_kb":82416,"num_write":2344,"num_write_kb":55665,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"2f","length":"1"},{"start":"31","length":"1"},{"start":"33","length":"1"},{"start":"35","length":"1"},{"start":"37","length":"1"},{"start":"39","length":"1"},{"start":"3b","length":"1"},{"start":"3d","length":"1"}]},{"pgid":"3.19","version":"232'506","reported_seq":6223,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-09-14T06:32:38.772136+0000","last_change":"2024-09-14T06:32:38.772406+0000","last_active":"2024-09-14T06:32:38.772136+0000","last_peered":"2024-09-14T06:32:38.772136+0000","last_clean":"2024-09-14T06:32:38.772136+0000","last_became_active":"2024-09-14T06:31:53.744800+0000","last_became_peered":"2024-09-14T06:31:53.744800+0000","last_unstale":"2024-09-14T06:32:38.772136+0000","last_undegraded":"2024-09-14T06:32:38.772136+0000","last_fullsized":"2024-09-14T06:32:38.772136+0000","mapping_epoch":188,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":189,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":506,"log_dups_size":0,"ondisk_log_size":506,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T07:55:27.874216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076022100000000005,"stat_sum":{"num_bytes":34593958,"num_objects":43,"num_object_clones":0,"num_object_copies":86,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":43,"num_whiteouts":0,"num_read":5084,"num_read_kb":67754,"num_write":708,"num_write_kb":43868,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":282,"num_bytes_recovered":41927489,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2f","length":"1"},{"start":"31","length":"1"},{"start":"33","length":"1"},{"start":"35","length":"1"},{"start":"37","length":"1"},{"start":"39","length":"1"},{"start":"3b","length":"1"},{"start":"3d","length":"1"}]},{"pgid":"3.18","version":"234'499","reported_seq":6027,"reported_epoch":235,"state":"active+clean","last_fresh":"2024-09-14T06:32:40.729078+0000","last_change":"2024-09-14T06:32:40.629026+0000","last_active":"2024-09-14T06:32:40.729078+0000","last_peered":"2024-09-14T06:32:40.729078+0000","last_clean":"2024-09-14T06:32:40.729078+0000","last_became_active":"2024-09-14T06:31:53.745159+0000","last_became_peered":"2024-09-14T06:31:53.745159+0000","last_unstale":"2024-09-14T06:32:40.729078+0000","last_undegraded":"2024-09-14T06:32:40.729078+0000","last_fullsized":"2024-09-14T06:32:40.729078+0000","mapping_epoch":188,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":189,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":499,"log_dups_size":0,"ondisk_log_size":499,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T08:21:11.501994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0016595749999999999,"stat_sum":{"num_bytes":22128117,"num_objects":35,"num_object_clones":0,"num_object_copies":70,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":35,"num_whiteouts":0,"num_read":4458,"num_read_kb":78893,"num_write":624,"num_write_kb":51576,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":310,"num_bytes_recovered":56082976,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2d","length":"1"},{"start":"2f","length":"1"},{"start":"31","length":"1"},{"start":"33","length":"1"}]},{"pgid":"3.17","version":"229'912","reported_seq":9228,"reported_epoch":232,"state":"active+clean","last_fresh":"2024-09-14T06:32:37.615378+0000","last_change":"2024-09-14T06:32:35.617536+0000","last_active":"2024-09-14T06:32:37.615378+0000","last_peered":"2024-09-14T06:32:37.615378+0000","last_clean":"2024-09-14T06:32:37.615378+0000","last_became_active":"2024-09-14T06:29:00.579714+0000","last_became_peered":"2024-09-14T06:29:00.579714+0000","last_unstale":"2024-09-14T06:32:37.615378+0000","last_undegraded":"2024-09-14T06:32:37.615378+0000","last_fullsized":"2024-09-14T06:32:37.615378+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":912,"log_dups_size":0,"ondisk_log_size":912,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T15:19:00.788559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001073657,"stat_sum":{"num_bytes":59778817,"num_objects":60,"num_object_clones":0,"num_object_copies":120,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":10280,"num_read_kb":68614,"num_write":1044,"num_write_kb":48460,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"31","length":"1"},{"start":"33","length":"1"},{"start":"35","length":"1"},{"start":"37","length":"1"},{"start":"39","length":"1"},{"start":"3b","length":"1"},{"start":"3d","length":"1"}]},{"pgid":"3.16","version":"229'1095","reported_seq":3858,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-09-14T06:32:38.775723+0000","last_change":"2024-09-14T06:32:38.775964+0000","last_active":"2024-09-14T06:32:38.775723+0000","last_peered":"2024-09-14T06:32:38.775723+0000","last_clean":"2024-09-14T06:32:38.775723+0000","last_became_active":"2024-09-14T06:32:24.718745+0000","last_became_peered":"2024-09-14T06:32:24.718745+0000","last_unstale":"2024-09-14T06:32:38.775723+0000","last_undegraded":"2024-09-14T06:32:38.775723+0000","last_fullsized":"2024-09-14T06:32:38.775723+0000","mapping_epoch":218,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":219,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":1095,"log_dups_size":0,"ondisk_log_size":1095,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T16:37:11.927791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00116008,"stat_sum":{"num_bytes":17620777,"num_objects":40,"num_object_clones":0,"num_object_copies":80,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":40,"num_whiteouts":0,"num_read":5485,"num_read_kb":38411,"num_write":1687,"num_write_kb":37242,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"2f","length":"1"},{"start":"31","length":"1"},{"start":"33","length":"1"},{"start":"35","length":"1"},{"start":"37","length":"1"},{"start":"39","length":"1"},{"start":"3b","length":"1"},{"start":"3d","length":"1"}]},{"pgid":"3.15","version":"232'585","reported_seq":6395,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-09-14T06:32:38.776125+0000","last_change":"2024-09-14T06:32:38.776355+0000","last_active":"2024-09-14T06:32:38.776125+0000","last_peered":"2024-09-14T06:32:38.776125+0000","last_clean":"2024-09-14T06:32:38.776125+0000","last_became_active":"2024-09-14T06:31:53.746091+0000","last_became_peered":"2024-09-14T06:31:53.746091+0000","last_unstale":"2024-09-14T06:32:38.776125+0000","last_undegraded":"2024-09-14T06:32:38.776125+0000","last_fullsized":"2024-09-14T06:32:38.776125+0000","mapping_epoch":188,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":189,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":585,"log_dups_size":0,"ondisk_log_size":585,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T10:43:55.843374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001230535,"stat_sum":{"num_bytes":35165078,"num_objects":41,"num_object_clones":0,"num_object_copies":82,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":41,"num_whiteouts":0,"num_read":4928,"num_read_kb":74748,"num_write":639,"num_write_kb":51190,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":271,"num_bytes_recovered":49981473,"num_keys_recovered":33,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"2f","length":"1"},{"start":"31","length":"1"},{"start":"33","length":"1"},{"start":"35","length":"1"},{"start":"37","length":"1"},{"start":"39","length":"1"},{"start":"3b","length":"1"},{"start":"3d","length":"1"}]},{"pgid":"3.14","version":"228'1077","reported_seq":5955,"reported_epoch":228,"state":"active+clean","last_fresh":"2024-09-14T06:32:33.764872+0000","last_change":"2024-09-14T06:32:33.764872+0000","last_active":"2024-09-14T06:32:33.764872+0000","last_peered":"2024-09-14T06:32:33.764872+0000","last_clean":"2024-09-14T06:32:33.764872+0000","last_became_active":"2024-09-14T06:29:00.582595+0000","last_became_peered":"2024-09-14T06:29:00.582595+0000","last_unstale":"2024-09-14T06:32:33.764872+0000","last_undegraded":"2024-09-14T06:32:33.764872+0000","last_fullsized":"2024-09-14T06:32:33.764872+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":1077,"log_dups_size":0,"ondisk_log_size":1077,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T16:53:26.698465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011374860000000001,"stat_sum":{"num_bytes":35195867,"num_objects":35,"num_object_clones":0,"num_object_copies":70,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":35,"num_whiteouts":0,"num_read":8768,"num_read_kb":46782,"num_write":1522,"num_write_kb":29823,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"35","length":"1"},{"start":"37","length":"1"},{"start":"39","length":"1"},{"start":"3b","length":"1"},{"start":"3d","length":"1"},{"start":"3f","length":"1"},{"start":"41","length":"1"},{"start":"43","length":"1"}]},{"pgid":"2.7","version":"0'0","reported_seq":84,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-09-14T06:32:38.775097+0000","last_change":"2024-09-14T06:32:01.612470+0000","last_active":"2024-09-14T06:32:38.775097+0000","last_peered":"2024-09-14T06:32:38.775097+0000","last_clean":"2024-09-14T06:32:38.775097+0000","last_became_active":"2024-09-14T06:32:01.612121+0000","last_became_peered":"2024-09-14T06:32:01.612121+0000","last_unstale":"2024-09-14T06:32:38.775097+0000","last_undegraded":"2024-09-14T06:32:38.775097+0000","last_fullsized":"2024-09-14T06:32:38.775097+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:28:50.205350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:28:50.205350+0000","last_clean_scrub_stamp":"2024-09-14T06:28:50.205350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T13:40:52.537942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.6","version":"229'784","reported_seq":8312,"reported_epoch":232,"state":"active+clean","last_fresh":"2024-09-14T06:32:37.615311+0000","last_change":"2024-09-14T06:32:35.617521+0000","last_active":"2024-09-14T06:32:37.615311+0000","last_peered":"2024-09-14T06:32:37.615311+0000","last_clean":"2024-09-14T06:32:37.615311+0000","last_became_active":"2024-09-14T06:29:00.578283+0000","last_became_peered":"2024-09-14T06:29:00.578283+0000","last_unstale":"2024-09-14T06:32:37.615311+0000","last_undegraded":"2024-09-14T06:32:37.615311+0000","last_fullsized":"2024-09-14T06:32:37.615311+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":784,"log_dups_size":0,"ondisk_log_size":784,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T06:44:25.975585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012089189999999999,"stat_sum":{"num_bytes":25415186,"num_objects":35,"num_object_clones":0,"num_object_copies":70,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":35,"num_whiteouts":0,"num_read":8373,"num_read_kb":78571,"num_write":846,"num_write_kb":48287,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"31","length":"1"},{"start":"33","length":"1"},{"start":"35","length":"1"},{"start":"37","length":"1"},{"start":"39","length":"1"},{"start":"3b","length":"1"},{"start":"3d","length":"1"}]},{"pgid":"2.6","version":"0'0","reported_seq":449,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-09-14T06:32:38.770882+0000","last_change":"2024-09-14T06:32:01.608456+0000","last_active":"2024-09-14T06:32:38.770882+0000","last_peered":"2024-09-14T06:32:38.770882+0000","last_clean":"2024-09-14T06:32:38.770882+0000","last_became_active":"2024-09-14T06:32:01.608178+0000","last_became_peered":"2024-09-14T06:32:01.608178+0000","last_unstale":"2024-09-14T06:32:38.770882+0000","last_undegraded":"2024-09-14T06:32:38.770882+0000","last_fullsized":"2024-09-14T06:32:38.770882+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:28:50.205350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:28:50.205350+0000","last_clean_scrub_stamp":"2024-09-14T06:28:50.205350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T10:16:40.480031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065018200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"235'785","reported_seq":4315,"reported_epoch":235,"state":"active+clean","last_fresh":"2024-09-14T06:32:40.628084+0000","last_change":"2024-09-14T06:32:40.628084+0000","last_active":"2024-09-14T06:32:40.628084+0000","last_peered":"2024-09-14T06:32:40.628084+0000","last_clean":"2024-09-14T06:32:40.628084+0000","last_became_active":"2024-09-14T06:31:54.747923+0000","last_became_peered":"2024-09-14T06:31:54.747923+0000","last_unstale":"2024-09-14T06:32:40.628084+0000","last_undegraded":"2024-09-14T06:32:40.628084+0000","last_fullsized":"2024-09-14T06:32:40.628084+0000","mapping_epoch":189,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":190,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":785,"log_dups_size":0,"ondisk_log_size":785,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T17:36:19.829390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00112521,"stat_sum":{"num_bytes":24555353,"num_objects":30,"num_object_clones":0,"num_object_copies":60,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":30,"num_whiteouts":0,"num_read":4414,"num_read_kb":40976,"num_write":1132,"num_write_kb":36116,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":183,"num_bytes_recovered":13912513,"num_keys_recovered":22,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2d","length":"1"},{"start":"2f","length":"1"},{"start":"31","length":"1"},{"start":"33","length":"1"}]},{"pgid":"2.5","version":"0'0","reported_seq":443,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-09-14T06:32:38.775360+0000","last_change":"2024-09-14T06:28:53.640035+0000","last_active":"2024-09-14T06:32:38.775360+0000","last_peered":"2024-09-14T06:32:38.775360+0000","last_clean":"2024-09-14T06:32:38.775360+0000","last_became_active":"2024-09-14T06:28:51.308058+0000","last_became_peered":"2024-09-14T06:28:51.308058+0000","last_unstale":"2024-09-14T06:32:38.775360+0000","last_undegraded":"2024-09-14T06:32:38.775360+0000","last_fullsized":"2024-09-14T06:32:38.775360+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:28:50.205350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:28:50.205350+0000","last_clean_scrub_stamp":"2024-09-14T06:28:50.205350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T13:35:48.544057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00037824000000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"229'652","reported_seq":3872,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-09-14T06:32:38.770678+0000","last_change":"2024-09-14T06:32:38.770948+0000","last_active":"2024-09-14T06:32:38.770678+0000","last_peered":"2024-09-14T06:32:38.770678+0000","last_clean":"2024-09-14T06:32:38.770678+0000","last_became_active":"2024-09-14T06:32:17.675470+0000","last_became_peered":"2024-09-14T06:32:17.675470+0000","last_unstale":"2024-09-14T06:32:38.770678+0000","last_undegraded":"2024-09-14T06:32:38.770678+0000","last_fullsized":"2024-09-14T06:32:38.770678+0000","mapping_epoch":211,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":212,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":652,"log_dups_size":0,"ondisk_log_size":652,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T11:31:07.192934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00058196400000000003,"stat_sum":{"num_bytes":13383727,"num_objects":36,"num_object_clones":0,"num_object_copies":72,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":36,"num_whiteouts":0,"num_read":2623,"num_read_kb":37411,"num_write":848,"num_write_kb":40855,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":331,"num_bytes_recovered":31411119,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2f","length":"1"},{"start":"31","length":"1"},{"start":"33","length":"1"},{"start":"35","length":"1"},{"start":"37","length":"1"},{"start":"39","length":"1"},{"start":"3b","length":"1"},{"start":"3d","length":"1"}]},{"pgid":"2.4","version":"0'0","reported_seq":443,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-09-14T06:32:38.770585+0000","last_change":"2024-09-14T06:28:53.226072+0000","last_active":"2024-09-14T06:32:38.770585+0000","last_peered":"2024-09-14T06:32:38.770585+0000","last_clean":"2024-09-14T06:32:38.770585+0000","last_became_active":"2024-09-14T06:28:51.308072+0000","last_became_peered":"2024-09-14T06:28:51.308072+0000","last_unstale":"2024-09-14T06:32:38.770585+0000","last_undegraded":"2024-09-14T06:32:38.770585+0000","last_fullsized":"2024-09-14T06:32:38.770585+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:28:50.205350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:28:50.205350+0000","last_clean_scrub_stamp":"2024-09-14T06:28:50.205350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T13:15:09.052328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066197399999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"229'854","reported_seq":5413,"reported_epoch":235,"state":"active+clean","last_fresh":"2024-09-14T06:32:40.628189+0000","last_change":"2024-09-14T06:32:40.628504+0000","last_active":"2024-09-14T06:32:40.628189+0000","last_peered":"2024-09-14T06:32:40.628189+0000","last_clean":"2024-09-14T06:32:40.628189+0000","last_became_active":"2024-09-14T06:32:24.716458+0000","last_became_peered":"2024-09-14T06:32:24.716458+0000","last_unstale":"2024-09-14T06:32:40.628189+0000","last_undegraded":"2024-09-14T06:32:40.628189+0000","last_fullsized":"2024-09-14T06:32:40.628189+0000","mapping_epoch":218,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":219,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":854,"log_dups_size":0,"ondisk_log_size":854,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T15:55:23.510859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001426797,"stat_sum":{"num_bytes":27242971,"num_objects":41,"num_object_clones":0,"num_object_copies":82,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":41,"num_whiteouts":0,"num_read":4510,"num_read_kb":42630,"num_write":1214,"num_write_kb":49494,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":330,"num_bytes_recovered":28798893,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2d","length":"1"},{"start":"2f","length":"1"},{"start":"31","length":"1"},{"start":"33","length":"1"}]},{"pgid":"3.3","version":"232'563","reported_seq":10394,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-09-14T06:32:38.770613+0000","last_change":"2024-09-14T06:32:38.770801+0000","last_active":"2024-09-14T06:32:38.770613+0000","last_peered":"2024-09-14T06:32:38.770613+0000","last_clean":"2024-09-14T06:32:38.770613+0000","last_became_active":"2024-09-14T06:29:00.584678+0000","last_became_peered":"2024-09-14T06:29:00.584678+0000","last_unstale":"2024-09-14T06:32:38.770613+0000","last_undegraded":"2024-09-14T06:32:38.770613+0000","last_fullsized":"2024-09-14T06:32:38.770613+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":563,"log_dups_size":0,"ondisk_log_size":563,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T18:11:37.882356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00082310700000000001,"stat_sum":{"num_bytes":49423207,"num_objects":34,"num_object_clones":0,"num_object_copies":68,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":34,"num_whiteouts":0,"num_read":9263,"num_read_kb":89033,"num_write":636,"num_write_kb":51616,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"2f","length":"1"},{"start":"31","length":"1"},{"start":"33","length":"1"},{"start":"35","length":"1"},{"start":"37","length":"1"},{"start":"39","length":"1"},{"start":"3b","length":"1"},{"start":"3d","length":"1"}]},{"pgid":"2.2","version":"22'2","reported_seq":58,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-09-14T06:32:38.770093+0000","last_change":"2024-09-14T06:32:24.716937+0000","last_active":"2024-09-14T06:32:38.770093+0000","last_peered":"2024-09-14T06:32:38.770093+0000","last_clean":"2024-09-14T06:32:38.770093+0000","last_became_active":"2024-09-14T06:32:24.716539+0000","last_became_peered":"2024-09-14T06:32:24.716539+0000","last_unstale":"2024-09-14T06:32:38.770093+0000","last_undegraded":"2024-09-14T06:32:38.770093+0000","last_fullsized":"2024-09-14T06:32:38.770093+0000","mapping_epoch":218,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":219,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:28:50.205350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:28:50.205350+0000","last_clean_scrub_stamp":"2024-09-14T06:28:50.205350+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T07:16:29.639469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"232'1062","reported_seq":6861,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-09-14T06:32:38.770343+0000","last_change":"2024-09-14T06:32:38.770573+0000","last_active":"2024-09-14T06:32:38.770343+0000","last_peered":"2024-09-14T06:32:38.770343+0000","last_clean":"2024-09-14T06:32:38.770343+0000","last_became_active":"2024-09-14T06:29:00.574873+0000","last_became_peered":"2024-09-14T06:29:00.574873+0000","last_unstale":"2024-09-14T06:32:38.770343+0000","last_undegraded":"2024-09-14T06:32:38.770343+0000","last_fullsized":"2024-09-14T06:32:38.770343+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":1062,"log_dups_size":0,"ondisk_log_size":1062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T15:03:28.889369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062675899999999995,"stat_sum":{"num_bytes":44082536,"num_objects":52,"num_object_clones":0,"num_object_copies":104,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":52,"num_whiteouts":0,"num_read":8251,"num_read_kb":85477,"num_write":1600,"num_write_kb":58700,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2f","length":"1"},{"start":"31","length":"1"},{"start":"33","length":"1"},{"start":"35","length":"1"},{"start":"37","length":"1"},{"start":"39","length":"1"},{"start":"3b","length":"1"},{"start":"3d","length":"1"}]},{"pgid":"2.1","version":"0'0","reported_seq":55,"reported_epoch":231,"state":"active+clean","last_fresh":"2024-09-14T06:32:36.614259+0000","last_change":"2024-09-14T06:32:14.665376+0000","last_active":"2024-09-14T06:32:36.614259+0000","last_peered":"2024-09-14T06:32:36.614259+0000","last_clean":"2024-09-14T06:32:36.614259+0000","last_became_active":"2024-09-14T06:32:14.665081+0000","last_became_peered":"2024-09-14T06:32:14.665081+0000","last_unstale":"2024-09-14T06:32:36.614259+0000","last_undegraded":"2024-09-14T06:32:36.614259+0000","last_fullsized":"2024-09-14T06:32:36.614259+0000","mapping_epoch":208,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":209,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:28:50.205350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:28:50.205350+0000","last_clean_scrub_stamp":"2024-09-14T06:28:50.205350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T16:35:12.869944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053403100000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1","version":"229'733","reported_seq":11492,"reported_epoch":232,"state":"active+clean","last_fresh":"2024-09-14T06:32:37.615156+0000","last_change":"2024-09-14T06:32:35.616947+0000","last_active":"2024-09-14T06:32:37.615156+0000","last_peered":"2024-09-14T06:32:37.615156+0000","last_clean":"2024-09-14T06:32:37.615156+0000","last_became_active":"2024-09-14T06:30:57.768238+0000","last_became_peered":"2024-09-14T06:30:57.768238+0000","last_unstale":"2024-09-14T06:32:37.615156+0000","last_undegraded":"2024-09-14T06:32:37.615156+0000","last_fullsized":"2024-09-14T06:32:37.615156+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":733,"log_dups_size":0,"ondisk_log_size":733,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T12:59:08.997641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00072126999999999998,"stat_sum":{"num_bytes":54607005,"num_objects":40,"num_object_clones":0,"num_object_copies":80,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":40,"num_whiteouts":0,"num_read":11049,"num_read_kb":112231,"num_write":909,"num_write_kb":47392,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":183,"num_bytes_recovered":30431994,"num_keys_recovered":22,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"31","length":"1"},{"start":"33","length":"1"},{"start":"35","length":"1"},{"start":"37","length":"1"},{"start":"39","length":"1"},{"start":"3b","length":"1"},{"start":"3d","length":"1"}]},{"pgid":"2.0","version":"0'0","reported_seq":443,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-09-14T06:32:38.774071+0000","last_change":"2024-09-14T06:28:53.640096+0000","last_active":"2024-09-14T06:32:38.774071+0000","last_peered":"2024-09-14T06:32:38.774071+0000","last_clean":"2024-09-14T06:32:38.774071+0000","last_became_active":"2024-09-14T06:28:51.223069+0000","last_became_peered":"2024-09-14T06:28:51.223069+0000","last_unstale":"2024-09-14T06:32:38.774071+0000","last_undegraded":"2024-09-14T06:32:38.774071+0000","last_fullsized":"2024-09-14T06:32:38.774071+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:28:50.205350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:28:50.205350+0000","last_clean_scrub_stamp":"2024-09-14T06:28:50.205350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T15:13:00.471509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00043714600000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"234'1598","reported_seq":7570,"reported_epoch":235,"state":"active+clean","last_fresh":"2024-09-14T06:32:40.628278+0000","last_change":"2024-09-14T06:32:40.628508+0000","last_active":"2024-09-14T06:32:40.628278+0000","last_peered":"2024-09-14T06:32:40.628278+0000","last_clean":"2024-09-14T06:32:40.628278+0000","last_became_active":"2024-09-14T06:31:54.744160+0000","last_became_peered":"2024-09-14T06:31:54.744160+0000","last_unstale":"2024-09-14T06:32:40.628278+0000","last_undegraded":"2024-09-14T06:32:40.628278+0000","last_fullsized":"2024-09-14T06:32:40.628278+0000","mapping_epoch":189,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":190,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":1598,"log_dups_size":0,"ondisk_log_size":1598,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T13:47:24.444561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001372971,"stat_sum":{"num_bytes":43675338,"num_objects":51,"num_object_clones":0,"num_object_copies":102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":51,"num_whiteouts":0,"num_read":10684,"num_read_kb":70562,"num_write":2532,"num_write_kb":53559,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":327,"num_bytes_recovered":50047063,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2d","length":"1"},{"start":"2f","length":"1"},{"start":"31","length":"1"},{"start":"33","length":"1"}]},{"pgid":"2.3","version":"20'1","reported_seq":40,"reported_epoch":231,"state":"active+clean","last_fresh":"2024-09-14T06:32:36.613863+0000","last_change":"2024-09-14T06:32:24.717893+0000","last_active":"2024-09-14T06:32:36.613863+0000","last_peered":"2024-09-14T06:32:36.613863+0000","last_clean":"2024-09-14T06:32:36.613863+0000","last_became_active":"2024-09-14T06:32:24.717658+0000","last_became_peered":"2024-09-14T06:32:24.717658+0000","last_unstale":"2024-09-14T06:32:36.613863+0000","last_undegraded":"2024-09-14T06:32:36.613863+0000","last_fullsized":"2024-09-14T06:32:36.613863+0000","mapping_epoch":218,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":219,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:28:50.205350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:28:50.205350+0000","last_clean_scrub_stamp":"2024-09-14T06:28:50.205350+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T10:36:16.650799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.0","version":"17'72","reported_seq":534,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-09-14T06:32:38.774173+0000","last_change":"2024-09-14T06:28:44.839885+0000","last_active":"2024-09-14T06:32:38.774173+0000","last_peered":"2024-09-14T06:32:38.774173+0000","last_clean":"2024-09-14T06:32:38.774173+0000","last_became_active":"2024-09-14T06:28:44.839573+0000","last_became_peered":"2024-09-14T06:28:44.839573+0000","last_unstale":"2024-09-14T06:32:38.774173+0000","last_undegraded":"2024-09-14T06:32:38.774173+0000","last_fullsized":"2024-09-14T06:32:38.774173+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:28:43.828494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:28:43.828494+0000","last_clean_scrub_stamp":"2024-09-14T06:28:43.828494+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T14:41:25.787293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.8","version":"234'611","reported_seq":6301,"reported_epoch":235,"state":"active+clean","last_fresh":"2024-09-14T06:32:40.629274+0000","last_change":"2024-09-14T06:32:40.629478+0000","last_active":"2024-09-14T06:32:40.629274+0000","last_peered":"2024-09-14T06:32:40.629274+0000","last_clean":"2024-09-14T06:32:40.629274+0000","last_became_active":"2024-09-14T06:31:55.754694+0000","last_became_peered":"2024-09-14T06:31:55.754694+0000","last_unstale":"2024-09-14T06:32:40.629274+0000","last_undegraded":"2024-09-14T06:32:40.629274+0000","last_fullsized":"2024-09-14T06:32:40.629274+0000","mapping_epoch":190,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":191,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":611,"log_dups_size":0,"ondisk_log_size":611,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T14:01:10.585303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0017852390000000001,"stat_sum":{"num_bytes":45016747,"num_objects":46,"num_object_clones":0,"num_object_copies":92,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":46,"num_whiteouts":0,"num_read":5613,"num_read_kb":90698,"num_write":859,"num_write_kb":57685,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":319,"num_bytes_recovered":60352269,"num_keys_recovered":22,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2d","length":"1"},{"start":"2f","length":"1"},{"start":"31","length":"1"},{"start":"33","length":"1"}]},{"pgid":"3.9","version":"232'548","reported_seq":4250,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-09-14T06:32:38.770678+0000","last_change":"2024-09-14T06:32:38.771289+0000","last_active":"2024-09-14T06:32:38.770678+0000","last_peered":"2024-09-14T06:32:38.770678+0000","last_clean":"2024-09-14T06:32:38.770678+0000","last_became_active":"2024-09-14T06:29:00.573179+0000","last_became_peered":"2024-09-14T06:29:00.573179+0000","last_unstale":"2024-09-14T06:32:38.770678+0000","last_undegraded":"2024-09-14T06:32:38.770678+0000","last_fullsized":"2024-09-14T06:32:38.770678+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":548,"log_dups_size":0,"ondisk_log_size":548,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T14:58:02.404316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014824910000000001,"stat_sum":{"num_bytes":19760470,"num_objects":45,"num_object_clones":0,"num_object_copies":90,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":45,"num_whiteouts":0,"num_read":3675,"num_read_kb":64290,"num_write":733,"num_write_kb":44928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"2f","length":"1"},{"start":"31","length":"1"},{"start":"33","length":"1"},{"start":"35","length":"1"},{"start":"37","length":"1"},{"start":"39","length":"1"},{"start":"3b","length":"1"},{"start":"3d","length":"1"}]},{"pgid":"3.a","version":"229'809","reported_seq":4776,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-09-14T06:32:38.770939+0000","last_change":"2024-09-14T06:32:38.771148+0000","last_active":"2024-09-14T06:32:38.770939+0000","last_peered":"2024-09-14T06:32:38.770939+0000","last_clean":"2024-09-14T06:32:38.770939+0000","last_became_active":"2024-09-14T06:32:04.617283+0000","last_became_peered":"2024-09-14T06:32:04.617283+0000","last_unstale":"2024-09-14T06:32:38.770939+0000","last_undegraded":"2024-09-14T06:32:38.770939+0000","last_fullsized":"2024-09-14T06:32:38.770939+0000","mapping_epoch":199,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":200,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":809,"log_dups_size":0,"ondisk_log_size":809,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T08:56:00.098170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00096903099999999997,"stat_sum":{"num_bytes":38682031,"num_objects":40,"num_object_clones":0,"num_object_copies":80,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":40,"num_whiteouts":0,"num_read":5155,"num_read_kb":58177,"num_write":1165,"num_write_kb":47529,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":161,"num_bytes_recovered":21723371,"num_keys_recovered":33,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"2f","length":"1"},{"start":"31","length":"1"},{"start":"33","length":"1"},{"start":"35","length":"1"},{"start":"37","length":"1"},{"start":"39","length":"1"},{"start":"3b","length":"1"},{"start":"3d","length":"1"}]},{"pgid":"3.b","version":"235'1551","reported_seq":8610,"reported_epoch":235,"state":"active+clean","last_fresh":"2024-09-14T06:32:40.629928+0000","last_change":"2024-09-14T06:32:40.629928+0000","last_active":"2024-09-14T06:32:40.629928+0000","last_peered":"2024-09-14T06:32:40.629928+0000","last_clean":"2024-09-14T06:32:40.629928+0000","last_became_active":"2024-09-14T06:31:55.756219+0000","last_became_peered":"2024-09-14T06:31:55.756219+0000","last_unstale":"2024-09-14T06:32:40.629928+0000","last_undegraded":"2024-09-14T06:32:40.629928+0000","last_fullsized":"2024-09-14T06:32:40.629928+0000","mapping_epoch":190,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":191,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":1551,"log_dups_size":0,"ondisk_log_size":1551,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T17:34:08.601295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013340330000000001,"stat_sum":{"num_bytes":21757058,"num_objects":41,"num_object_clones":0,"num_object_copies":82,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":41,"num_whiteouts":0,"num_read":12802,"num_read_kb":63928,"num_write":2413,"num_write_kb":43663,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":291,"num_bytes_recovered":37370632,"num_keys_recovered":66,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2d","length":"1"},{"start":"2f","length":"1"},{"start":"31","length":"1"},{"start":"33","length":"1"}]},{"pgid":"3.c","version":"235'573","reported_seq":6079,"reported_epoch":235,"state":"active+clean","last_fresh":"2024-09-14T06:32:40.746714+0000","last_change":"2024-09-14T06:32:40.627600+0000","last_active":"2024-09-14T06:32:40.746714+0000","last_peered":"2024-09-14T06:32:40.746714+0000","last_clean":"2024-09-14T06:32:40.746714+0000","last_became_active":"2024-09-14T06:32:24.716421+0000","last_became_peered":"2024-09-14T06:32:24.716421+0000","last_unstale":"2024-09-14T06:32:40.746714+0000","last_undegraded":"2024-09-14T06:32:40.746714+0000","last_fullsized":"2024-09-14T06:32:40.746714+0000","mapping_epoch":218,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":219,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":573,"log_dups_size":0,"ondisk_log_size":573,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T13:12:43.583648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061805300000000003,"stat_sum":{"num_bytes":26190419,"num_objects":36,"num_object_clones":0,"num_object_copies":72,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":36,"num_whiteouts":0,"num_read":4817,"num_read_kb":50405,"num_write":748,"num_write_kb":40823,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":293,"num_bytes_recovered":36712421,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2d","length":"1"},{"start":"2f","length":"1"},{"start":"31","length":"1"},{"start":"33","length":"1"}]},{"pgid":"3.d","version":"232'513","reported_seq":8972,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-09-14T06:32:38.775655+0000","last_change":"2024-09-14T06:32:38.775937+0000","last_active":"2024-09-14T06:32:38.775655+0000","last_peered":"2024-09-14T06:32:38.775655+0000","last_clean":"2024-09-14T06:32:38.775655+0000","last_became_active":"2024-09-14T06:29:00.580362+0000","last_became_peered":"2024-09-14T06:29:00.580362+0000","last_unstale":"2024-09-14T06:32:38.775655+0000","last_undegraded":"2024-09-14T06:32:38.775655+0000","last_fullsized":"2024-09-14T06:32:38.775655+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":513,"log_dups_size":0,"ondisk_log_size":513,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T10:40:58.585595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013537169999999999,"stat_sum":{"num_bytes":45657425,"num_objects":42,"num_object_clones":0,"num_object_copies":84,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":42,"num_whiteouts":0,"num_read":8026,"num_read_kb":93543,"num_write":674,"num_write_kb":41026,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"2f","length":"1"},{"start":"31","length":"1"},{"start":"33","length":"1"},{"start":"35","length":"1"},{"start":"37","length":"1"},{"start":"39","length":"1"},{"start":"3b","length":"1"},{"start":"3d","length":"1"}]},{"pgid":"3.e","version":"229'538","reported_seq":5239,"reported_epoch":232,"state":"peering","last_fresh":"2024-09-14T06:32:37.900918+0000","last_change":"2024-09-14T06:32:38.774414+0000","last_active":"2024-09-14T06:32:38.774414+0000","last_peered":"2024-09-14T06:32:37.900918+0000","last_clean":"2024-09-14T06:32:34.846079+0000","last_became_active":"2024-09-14T06:32:37.632816+0000","last_became_peered":"2024-09-14T06:32:37.632816+0000","last_unstale":"2024-09-14T06:32:37.900918+0000","last_undegraded":"2024-09-14T06:32:37.900918+0000","last_fullsized":"2024-09-14T06:32:37.617910+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":538,"log_dups_size":0,"ondisk_log_size":538,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0030274130000000001,"stat_sum":{"num_bytes":25133180,"num_objects":45,"num_object_clones":0,"num_object_copies":90,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":45,"num_whiteouts":0,"num_read":3980,"num_read_kb":61135,"num_write":708,"num_write_kb":43787,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":171,"num_bytes_recovered":10913227,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1,4],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"33","length":"1"},{"start":"35","length":"1"},{"start":"37","length":"1"},{"start":"39","length":"1"},{"start":"3b","length":"1"},{"start":"3d","length":"1"}]},{"pgid":"3.f","version":"232'663","reported_seq":5604,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-09-14T06:32:38.776055+0000","last_change":"2024-09-14T06:32:38.776260+0000","last_active":"2024-09-14T06:32:38.776055+0000","last_peered":"2024-09-14T06:32:38.776055+0000","last_clean":"2024-09-14T06:32:38.776055+0000","last_became_active":"2024-09-14T06:29:00.586491+0000","last_became_peered":"2024-09-14T06:29:00.586491+0000","last_unstale":"2024-09-14T06:32:38.776055+0000","last_undegraded":"2024-09-14T06:32:38.776055+0000","last_fullsized":"2024-09-14T06:32:38.776055+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":663,"log_dups_size":0,"ondisk_log_size":663,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T15:02:02.032487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013964019999999999,"stat_sum":{"num_bytes":17931678,"num_objects":26,"num_object_clones":0,"num_object_copies":52,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":0,"num_read":5903,"num_read_kb":45425,"num_write":794,"num_write_kb":36128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"2f","length":"1"},{"start":"31","length":"1"},{"start":"33","length":"1"},{"start":"35","length":"1"},{"start":"37","length":"1"},{"start":"39","length":"1"},{"start":"3b","length":"1"},{"start":"3d","length":"1"}]},{"pgid":"3.10","version":"228'892","reported_seq":6524,"reported_epoch":232,"state":"active+clean","last_fresh":"2024-09-14T06:32:37.615102+0000","last_change":"2024-09-14T06:32:35.617047+0000","last_active":"2024-09-14T06:32:37.615102+0000","last_peered":"2024-09-14T06:32:37.615102+0000","last_clean":"2024-09-14T06:32:37.615102+0000","last_became_active":"2024-09-14T06:32:04.621617+0000","last_became_peered":"2024-09-14T06:32:04.621617+0000","last_unstale":"2024-09-14T06:32:37.615102+0000","last_undegraded":"2024-09-14T06:32:37.615102+0000","last_fullsized":"2024-09-14T06:32:37.615102+0000","mapping_epoch":199,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":200,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":892,"log_dups_size":0,"ondisk_log_size":892,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T14:19:32.601273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00081382600000000002,"stat_sum":{"num_bytes":28229344,"num_objects":36,"num_object_clones":0,"num_object_copies":72,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":36,"num_whiteouts":0,"num_read":6967,"num_read_kb":34320,"num_write":1030,"num_write_kb":36649,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":154,"num_bytes_recovered":9401812,"num_keys_recovered":45,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"31","length":"1"},{"start":"33","length":"1"},{"start":"35","length":"1"},{"start":"37","length":"1"},{"start":"39","length":"1"},{"start":"3b","length":"1"},{"start":"3d","length":"1"}]},{"pgid":"3.11","version":"227'652","reported_seq":11425,"reported_epoch":232,"state":"peering","last_fresh":"2024-09-14T06:32:38.469234+0000","last_change":"2024-09-14T06:32:38.775405+0000","last_active":"2024-09-14T06:32:38.775405+0000","last_peered":"2024-09-14T06:32:38.469234+0000","last_clean":"2024-09-14T06:32:34.841771+0000","last_became_active":"2024-09-14T06:32:37.635777+0000","last_became_peered":"2024-09-14T06:32:37.635777+0000","last_unstale":"2024-09-14T06:32:38.469234+0000","last_undegraded":"2024-09-14T06:32:38.469234+0000","last_fullsized":"2024-09-14T06:32:37.617586+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":652,"log_dups_size":0,"ondisk_log_size":652,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00055383899999999998,"stat_sum":{"num_bytes":73789373,"num_objects":52,"num_object_clones":0,"num_object_copies":104,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":52,"num_whiteouts":0,"num_read":10209,"num_read_kb":109195,"num_write":781,"num_write_kb":48297,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":177,"num_bytes_recovered":16647747,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1,4],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"33","length":"1"},{"start":"35","length":"1"},{"start":"37","length":"1"},{"start":"39","length":"1"},{"start":"3b","length":"1"},{"start":"3d","length":"1"}]},{"pgid":"3.12","version":"229'694","reported_seq":7113,"reported_epoch":232,"state":"active+clean","last_fresh":"2024-09-14T06:32:37.615180+0000","last_change":"2024-09-14T06:32:35.617225+0000","last_active":"2024-09-14T06:32:37.615180+0000","last_peered":"2024-09-14T06:32:37.615180+0000","last_clean":"2024-09-14T06:32:37.615180+0000","last_became_active":"2024-09-14T06:29:00.579826+0000","last_became_peered":"2024-09-14T06:29:00.579826+0000","last_unstale":"2024-09-14T06:32:37.615180+0000","last_undegraded":"2024-09-14T06:32:37.615180+0000","last_fullsized":"2024-09-14T06:32:37.615180+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":694,"log_dups_size":0,"ondisk_log_size":694,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T17:20:16.256422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00094657200000000002,"stat_sum":{"num_bytes":33207124,"num_objects":43,"num_object_clones":0,"num_object_copies":86,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":43,"num_whiteouts":0,"num_read":6875,"num_read_kb":58890,"num_write":862,"num_write_kb":42425,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"31","length":"1"},{"start":"33","length":"1"},{"start":"35","length":"1"},{"start":"37","length":"1"},{"start":"39","length":"1"},{"start":"3b","length":"1"},{"start":"3d","length":"1"}]},{"pgid":"3.13","version":"227'963","reported_seq":8691,"reported_epoch":233,"state":"active+clean","last_fresh":"2024-09-14T06:32:38.774432+0000","last_change":"2024-09-14T06:32:38.774663+0000","last_active":"2024-09-14T06:32:38.774432+0000","last_peered":"2024-09-14T06:32:38.774432+0000","last_clean":"2024-09-14T06:32:38.774432+0000","last_became_active":"2024-09-14T06:29:00.586520+0000","last_became_peered":"2024-09-14T06:29:00.586520+0000","last_unstale":"2024-09-14T06:32:38.774432+0000","last_undegraded":"2024-09-14T06:32:38.774432+0000","last_fullsized":"2024-09-14T06:32:38.774432+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":963,"log_dups_size":0,"ondisk_log_size":963,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T09:17:05.867426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049198799999999995,"stat_sum":{"num_bytes":54345543,"num_objects":41,"num_object_clones":0,"num_object_copies":82,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":41,"num_whiteouts":0,"num_read":9546,"num_read_kb":66117,"num_write":1194,"num_write_kb":47437,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"2f","length":"1"},{"start":"31","length":"1"},{"start":"33","length":"1"},{"start":"35","length":"1"},{"start":"37","length":"1"},{"start":"39","length":"1"},{"start":"3b","length":"1"},{"start":"3d","length":"1"}]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":1146083299,"num_objects":1333,"num_object_clones":0,"num_object_copies":2666,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1333,"num_whiteouts":0,"num_read":235045,"num_read_kb":2180595,"num_write":36029,"num_write_kb":1478072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5001,"num_bytes_recovered":627008899,"num_keys_recovered":523,"num_objects_omap":25,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":87437312,"data_stored":858645747,"data_compressed":53183589,"data_compressed_allocated":69992448,"data_compressed_original":844987183,"omap_allocated":0,"internal_metadata":0},"log_size":26369,"ondisk_log_size":26369,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":131072,"data_stored":1180736,"data_compressed":65166,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509490,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":76548,"kb_used_data":13428,"kb_used_omap":1,"kb_used_meta":63102,"kb_avail":94295292,"statfs":{"total":96636764160,"available":96558379008,"internally_reserved":0,"allocated":13750272,"data_stored":106349888,"data_compressed":6344321,"data_compressed_allocated":8302592,"data_compressed_original":102843715,"omap_allocated":1590,"internal_metadata":64616906},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":0,"seq":64424509490,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509491,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":80552,"kb_used_data":13260,"kb_used_omap":1,"kb_used_meta":67262,"kb_avail":94291288,"statfs":{"total":96636764160,"available":96554278912,"internally_reserved":0,"allocated":13578240,"data_stored":101872488,"data_compressed":6174405,"data_compressed_allocated":8257536,"data_compressed_original":98530128,"omap_allocated":1595,"internal_metadata":68876741},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509490,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":94376,"kb_used_data":18764,"kb_used_omap":1,"kb_used_meta":75582,"kb_avail":94277464,"statfs":{"total":96636764160,"available":96540123136,"internally_reserved":0,"allocated":19214336,"data_stored":164438257,"data_compressed":10361619,"data_compressed_allocated":13443072,"data_compressed_original":160805636,"omap_allocated":1595,"internal_metadata":77396421},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509491,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":64456,"kb_used_data":15216,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94307384,"statfs":{"total":96636764160,"available":96570761216,"internally_reserved":0,"allocated":15581184,"data_stored":129195838,"data_compressed":8098621,"data_compressed_allocated":10567680,"data_compressed_original":126127129,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":14,"seq":60129542194,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":38912,"kb_used_data":7716,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94332928,"statfs":{"total":96636764160,"available":96596918272,"internally_reserved":0,"allocated":7901184,"data_stored":55608356,"data_compressed":3234230,"data_compressed_allocated":4337664,"data_compressed_original":53662503,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542193,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":86292,"kb_used_data":14844,"kb_used_omap":1,"kb_used_meta":71422,"kb_avail":94285548,"statfs":{"total":96636764160,"available":96548401152,"internally_reserved":0,"allocated":15200256,"data_stored":120701857,"data_compressed":7276960,"data_compressed_allocated":9498624,"data_compressed_original":116981124,"omap_allocated":1595,"internal_metadata":73136581},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574899,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":90316,"kb_used_data":14712,"kb_used_omap":1,"kb_used_meta":75582,"kb_avail":94281524,"statfs":{"total":96636764160,"available":96544280576,"internally_reserved":0,"allocated":15065088,"data_stored":122917142,"data_compressed":7640073,"data_compressed_allocated":10076160,"data_compressed_original":119828480,"omap_allocated":1593,"internal_metadata":77396423},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32583,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32583,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12333056,"data_stored":118738265,"data_compressed":7244377,"data_compressed_allocated":9453568,"data_compressed_original":116407684,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":13070336,"data_stored":130840294,"data_compressed":8281791,"data_compressed_allocated":10809344,"data_compressed_original":129109017,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":5111808,"data_stored":54241309,"data_compressed":3234230,"data_compressed_allocated":4337664,"data_compressed_original":53662503,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12222464,"data_stored":121506055,"data_compressed":7639096,"data_compressed_allocated":10072064,"data_compressed_original":119808000,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":16412672,"data_stored":163065033,"data_compressed":10361619,"data_compressed_allocated":13443072,"data_compressed_original":160805636,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":10772480,"data_stored":100499245,"data_compressed":6174405,"data_compressed_allocated":8257536,"data_compressed_original":98530128,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":6631424,"data_stored":65369250,"data_compressed":3936333,"data_compressed_allocated":5361664,"data_compressed_original":64393940,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":10883072,"data_stored":104386296,"data_compressed":6311738,"data_compressed_allocated":8257536,"data_compressed_original":102270275,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-14T06:32:43.118 INFO:teuthology.orchestra.run.smithi138.stdout:ERROR: (22) Invalid argument 2024-09-14T06:32:43.119 INFO:teuthology.orchestra.run.smithi138.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:32:43.127 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-14T06:32:43.128 INFO:teuthology.orchestra.run.smithi094.stdout:osd.0: {} 2024-09-14T06:32:43.129 INFO:teuthology.orchestra.run.smithi094.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-14T06:32:43.145 INFO:teuthology.orchestra.run.smithi094.stdout:osd.1: {} 2024-09-14T06:32:43.145 INFO:teuthology.orchestra.run.smithi094.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-14T06:32:43.161 INFO:teuthology.orchestra.run.smithi094.stdout:osd.2: {} 2024-09-14T06:32:43.161 INFO:teuthology.orchestra.run.smithi094.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-14T06:32:43.172 INFO:tasks.ceph.osd.6.smithi138.stderr:2024-09-14T06:32:43.171+0000 7f8cd8849640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33620) UID: 0 2024-09-14T06:32:43.177 INFO:teuthology.orchestra.run.smithi094.stdout:osd.3: {} 2024-09-14T06:32:43.177 INFO:teuthology.orchestra.run.smithi094.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-14T06:32:43.194 INFO:teuthology.orchestra.run.smithi094.stdout:osd.4: {} 2024-09-14T06:32:43.194 INFO:teuthology.orchestra.run.smithi094.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-14T06:32:43.210 INFO:teuthology.orchestra.run.smithi094.stdout:osd.5: {} 2024-09-14T06:32:43.210 INFO:teuthology.orchestra.run.smithi094.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-14T06:32:43.226 INFO:teuthology.orchestra.run.smithi094.stdout:osd.6: {} 2024-09-14T06:32:43.226 INFO:teuthology.orchestra.run.smithi094.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-14T06:32:43.232 INFO:teuthology.orchestra.run.smithi138.stdout:ERROR: (22) Invalid argument 2024-09-14T06:32:43.232 INFO:teuthology.orchestra.run.smithi138.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:32:43.242 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-14T06:32:43.243 INFO:teuthology.orchestra.run.smithi094.stdout:osd.7: {} 2024-09-14T06:32:43.243 INFO:teuthology.orchestra.run.smithi094.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-14T06:32:43.294 INFO:tasks.ceph.osd.1.smithi094.stderr:2024-09-14T06:32:43.293+0000 7f328e47a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33641) UID: 0 2024-09-14T06:32:43.373 INFO:tasks.ceph.osd.1.smithi094.stderr:2024-09-14T06:32:43.372+0000 7f328e47a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33641) UID: 0 2024-09-14T06:32:43.385 INFO:teuthology.orchestra.run.smithi138.stdout:{ 2024-09-14T06:32:43.385 INFO:teuthology.orchestra.run.smithi138.stdout: "ops": [], 2024-09-14T06:32:43.385 INFO:teuthology.orchestra.run.smithi138.stdout: "complaint_time": 30, 2024-09-14T06:32:43.385 INFO:teuthology.orchestra.run.smithi138.stdout: "num_blocked_ops": 0 2024-09-14T06:32:43.385 INFO:teuthology.orchestra.run.smithi138.stdout:} 2024-09-14T06:32:43.393 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-14T06:32:43.474 INFO:tasks.ceph.osd.7.smithi138.stderr:2024-09-14T06:32:43.473+0000 7f3c6a21d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33623) UID: 0 2024-09-14T06:32:43.497 INFO:teuthology.orchestra.run.smithi138.stdout:{ 2024-09-14T06:32:43.497 INFO:teuthology.orchestra.run.smithi138.stdout: "size": 20, 2024-09-14T06:32:43.498 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 600, 2024-09-14T06:32:43.498 INFO:teuthology.orchestra.run.smithi138.stdout: "ops": [ 2024-09-14T06:32:43.498 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:43.498 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:53244 3.f 3:f71f51a7:::rbd_data.112dca2f6ae1.0000000000000015:17 [read 1863680~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:32:43.498 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:44.369399+0000", 2024-09-14T06:32:43.498 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 179.126356762, 2024-09-14T06:32:43.498 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047291189999999997, 2024-09-14T06:32:43.498 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:32:43.498 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:32:43.498 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:32:43.498 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:32:43.498 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:32:43.498 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:43.499 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 53244 2024-09-14T06:32:43.499 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:43.499 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:32:43.499 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:43.499 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:32:43.499 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369399+0000", 2024-09-14T06:32:43.499 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:43.499 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:43.499 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:43.499 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:32:43.499 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369399+0000", 2024-09-14T06:32:43.499 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:43.499 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:43.499 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:43.499 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:32:43.499 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369400+0000", 2024-09-14T06:32:43.500 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.0699999999999999e-07 2024-09-14T06:32:43.500 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:43.500 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:43.500 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:32:43.500 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369401+0000", 2024-09-14T06:32:43.500 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.4300000000000001e-06 2024-09-14T06:32:43.500 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:43.500 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:43.500 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:32:43.500 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369404+0000", 2024-09-14T06:32:43.500 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.061e-06 2024-09-14T06:32:43.500 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:43.500 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:43.500 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:32:43.501 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369407+0000", 2024-09-14T06:32:43.501 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.6139999999999999e-06 2024-09-14T06:32:43.501 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:43.501 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:43.501 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:32:43.501 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.416517+0000", 2024-09-14T06:32:43.501 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047110338000000002 2024-09-14T06:32:43.501 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:43.501 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:43.501 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:32:43.501 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.416616+0000", 2024-09-14T06:32:43.501 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.8651000000000001e-05 2024-09-14T06:32:43.501 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:43.502 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:43.502 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:32:43.502 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.416691+0000", 2024-09-14T06:32:43.502 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.4888999999999994e-05 2024-09-14T06:32:43.502 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:43.502 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:32:43.502 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:43.502 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:43.502 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:43.502 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:53245 3.f 3:f71f51a7:::rbd_data.112dca2f6ae1.0000000000000015:17 [read 1888256~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:32:43.502 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:44.369410+0000", 2024-09-14T06:32:43.502 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 179.126346474, 2024-09-14T06:32:43.502 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047399275999999997, 2024-09-14T06:32:43.502 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:32:43.502 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:32:43.502 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:32:43.503 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:32:43.503 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:32:43.503 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:43.503 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 53245 2024-09-14T06:32:43.503 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:43.503 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:32:43.503 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:43.503 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:32:43.503 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369410+0000", 2024-09-14T06:32:43.503 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:43.503 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:43.503 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:43.503 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:32:43.503 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369410+0000", 2024-09-14T06:32:43.504 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:43.504 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:43.504 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:43.504 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:32:43.504 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369410+0000", 2024-09-14T06:32:43.504 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.1899999999999999e-07 2024-09-14T06:32:43.504 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:43.504 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:43.504 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:32:43.504 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369411+0000", 2024-09-14T06:32:43.504 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.0300000000000001e-06 2024-09-14T06:32:43.504 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:43.504 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:43.504 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:32:43.504 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369414+0000", 2024-09-14T06:32:43.505 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.1099999999999999e-06 2024-09-14T06:32:43.505 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:43.505 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:43.505 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:32:43.505 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369417+0000", 2024-09-14T06:32:43.505 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.6189999999999998e-06 2024-09-14T06:32:43.505 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:43.505 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:43.505 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:32:43.505 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.416718+0000", 2024-09-14T06:32:43.505 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047301322999999999 2024-09-14T06:32:43.505 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:43.505 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:43.506 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:32:43.506 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.416759+0000", 2024-09-14T06:32:43.506 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.1009e-05 2024-09-14T06:32:43.506 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:43.506 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:43.506 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:32:43.506 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.416809+0000", 2024-09-14T06:32:43.506 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.9966000000000002e-05 2024-09-14T06:32:43.506 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:43.506 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:32:43.506 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:43.506 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:43.506 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:43.506 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:53246 3.f 3:f71f51a7:::rbd_data.112dca2f6ae1.0000000000000015:17 [read 1912832~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:32:43.506 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:44.369420+0000", 2024-09-14T06:32:43.507 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 179.12633653, 2024-09-14T06:32:43.507 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047529463000000001, 2024-09-14T06:32:43.507 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:32:43.507 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:32:43.507 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:32:43.507 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:32:43.507 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:32:43.507 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:43.507 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 53246 2024-09-14T06:32:43.507 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:43.507 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:32:43.507 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:43.507 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:32:43.507 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369420+0000", 2024-09-14T06:32:43.507 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:43.507 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:43.508 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:43.508 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:32:43.508 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369420+0000", 2024-09-14T06:32:43.508 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:43.508 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:43.508 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:43.508 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:32:43.508 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369420+0000", 2024-09-14T06:32:43.508 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.1299999999999999e-07 2024-09-14T06:32:43.508 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:43.508 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:43.508 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:32:43.508 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369425+0000", 2024-09-14T06:32:43.508 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.0130000000000001e-06 2024-09-14T06:32:43.508 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:43.509 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:43.509 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:32:43.509 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369428+0000", 2024-09-14T06:32:43.509 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.349e-06 2024-09-14T06:32:43.509 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:43.509 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:43.509 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:32:43.509 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369431+0000", 2024-09-14T06:32:43.509 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.7389999999999999e-06 2024-09-14T06:32:43.509 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:43.509 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:43.509 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:32:43.509 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.416834+0000", 2024-09-14T06:32:43.509 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047402669000000001 2024-09-14T06:32:43.509 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:43.509 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:43.510 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:32:43.510 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.416882+0000", 2024-09-14T06:32:43.510 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.8068999999999997e-05 2024-09-14T06:32:43.510 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:43.510 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:43.510 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:32:43.510 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.416949+0000", 2024-09-14T06:32:43.510 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.7410999999999995e-05 2024-09-14T06:32:43.510 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:43.510 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:32:43.510 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:43.510 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:43.511 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:43.511 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:53247 3.f 3:f71f51a7:::rbd_data.112dca2f6ae1.0000000000000015:17 [read 1937408~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:32:43.511 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:44.369434+0000", 2024-09-14T06:32:43.511 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 179.12632176599999, 2024-09-14T06:32:43.511 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047652264999999999, 2024-09-14T06:32:43.511 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:32:43.511 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:32:43.511 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:32:43.511 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:32:43.511 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:32:43.511 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:43.511 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 53247 2024-09-14T06:32:43.511 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:43.511 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:32:43.511 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:43.511 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:32:43.512 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369434+0000", 2024-09-14T06:32:43.512 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:43.512 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:43.512 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:43.512 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:32:43.512 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369434+0000", 2024-09-14T06:32:43.512 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:43.512 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:43.512 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:43.512 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:32:43.512 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369435+0000", 2024-09-14T06:32:43.512 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.2100000000000001e-07 2024-09-14T06:32:43.512 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:43.512 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:43.512 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:32:43.513 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369436+0000", 2024-09-14T06:32:43.513 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.122e-06 2024-09-14T06:32:43.513 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:43.513 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:43.513 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:32:43.513 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369439+0000", 2024-09-14T06:32:43.513 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.0400000000000001e-06 2024-09-14T06:32:43.513 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:43.513 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:43.513 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:32:43.513 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369441+0000", 2024-09-14T06:32:43.513 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.728e-06 2024-09-14T06:32:43.513 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:43.513 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:43.513 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:32:43.514 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.416977+0000", 2024-09-14T06:32:43.514 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047535344 2024-09-14T06:32:43.514 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:43.514 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:43.514 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:32:43.514 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.417029+0000", 2024-09-14T06:32:43.514 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.2037999999999999e-05 2024-09-14T06:32:43.514 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:43.514 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:43.514 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:32:43.514 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.417087+0000", 2024-09-14T06:32:43.514 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.7772000000000002e-05 2024-09-14T06:32:43.514 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:43.514 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:32:43.514 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:43.514 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:43.515 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:43.515 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:53248 3.f 3:f71f51a7:::rbd_data.112dca2f6ae1.0000000000000015:17 [read 1961984~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:32:43.515 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:44.369445+0000", 2024-09-14T06:32:43.515 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 179.126311214, 2024-09-14T06:32:43.515 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047798489, 2024-09-14T06:32:43.515 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:32:43.515 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:32:43.515 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:32:43.515 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:32:43.515 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:32:43.515 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:43.515 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 53248 2024-09-14T06:32:43.515 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:43.515 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:32:43.515 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:43.515 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:32:43.516 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369445+0000", 2024-09-14T06:32:43.516 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:43.516 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:43.516 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:43.516 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:32:43.516 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369445+0000", 2024-09-14T06:32:43.516 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:43.516 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:43.516 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:43.516 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:32:43.516 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369445+0000", 2024-09-14T06:32:43.516 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.0699999999999999e-07 2024-09-14T06:32:43.516 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:43.516 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:43.516 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:32:43.516 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369446+0000", 2024-09-14T06:32:43.517 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.1000000000000001e-06 2024-09-14T06:32:43.517 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:43.517 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:43.517 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:32:43.517 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369449+0000", 2024-09-14T06:32:43.517 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.1039999999999998e-06 2024-09-14T06:32:43.517 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:43.517 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:43.517 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:32:43.517 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369452+0000", 2024-09-14T06:32:43.517 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.6199999999999999e-06 2024-09-14T06:32:43.517 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:43.517 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:43.517 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:32:43.517 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.417114+0000", 2024-09-14T06:32:43.518 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047661901999999999 2024-09-14T06:32:43.518 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:43.518 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:43.518 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:32:43.518 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.417181+0000", 2024-09-14T06:32:43.518 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.7385999999999995e-05 2024-09-14T06:32:43.518 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:43.518 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:43.518 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:32:43.518 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.417243+0000", 2024-09-14T06:32:43.518 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.2169999999999996e-05 2024-09-14T06:32:43.518 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:43.518 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:32:43.518 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:43.518 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:43.518 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:43.519 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:53249 3.f 3:f71f51a7:::rbd_data.112dca2f6ae1.0000000000000015:17 [read 1986560~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:32:43.519 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:44.369455+0000", 2024-09-14T06:32:43.519 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 179.12630090299999, 2024-09-14T06:32:43.519 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047890493999999999, 2024-09-14T06:32:43.519 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:32:43.519 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:32:43.519 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:32:43.519 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:32:43.519 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:32:43.519 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:43.519 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 53249 2024-09-14T06:32:43.519 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:43.519 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:32:43.519 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:43.519 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:32:43.519 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369455+0000", 2024-09-14T06:32:43.520 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:43.520 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:43.520 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:43.520 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:32:43.520 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369455+0000", 2024-09-14T06:32:43.520 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:43.520 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:43.520 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:43.520 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:32:43.520 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369455+0000", 2024-09-14T06:32:43.520 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.0800000000000001e-07 2024-09-14T06:32:43.520 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:43.520 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:43.520 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:32:43.520 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369456+0000", 2024-09-14T06:32:43.521 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.0729999999999999e-06 2024-09-14T06:32:43.521 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:43.521 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:43.521 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:32:43.521 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369460+0000", 2024-09-14T06:32:43.521 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.027e-06 2024-09-14T06:32:43.521 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:43.521 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:43.521 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:32:43.521 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369462+0000", 2024-09-14T06:32:43.521 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.5320000000000001e-06 2024-09-14T06:32:43.521 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:43.521 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:43.521 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:32:43.521 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.417257+0000", 2024-09-14T06:32:43.521 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.04779489 2024-09-14T06:32:43.522 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:43.522 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:43.522 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:32:43.522 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.417296+0000", 2024-09-14T06:32:43.522 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.8658000000000002e-05 2024-09-14T06:32:43.522 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:43.522 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:43.522 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:32:43.522 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.417346+0000", 2024-09-14T06:32:43.522 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.0105999999999998e-05 2024-09-14T06:32:43.522 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:43.522 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:32:43.522 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:43.522 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:43.522 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:43.522 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:53250 3.f 3:f71f51a7:::rbd_data.112dca2f6ae1.0000000000000015:17 [read 2011136~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:32:43.523 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:44.369465+0000", 2024-09-14T06:32:43.523 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 179.12629093300001, 2024-09-14T06:32:43.523 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.048032230000000002, 2024-09-14T06:32:43.523 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:32:43.523 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:32:43.523 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:32:43.523 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:32:43.523 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:32:43.523 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:43.523 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 53250 2024-09-14T06:32:43.523 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:43.523 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:32:43.523 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:43.523 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:32:43.523 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369465+0000", 2024-09-14T06:32:43.523 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:43.524 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:43.524 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:43.524 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:32:43.524 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369465+0000", 2024-09-14T06:32:43.524 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:43.524 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:43.524 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:43.524 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:32:43.524 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369465+0000", 2024-09-14T06:32:43.524 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.2100000000000001e-07 2024-09-14T06:32:43.524 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:43.524 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:43.524 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:32:43.524 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369466+0000", 2024-09-14T06:32:43.524 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.0640000000000001e-06 2024-09-14T06:32:43.524 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:43.525 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:43.525 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:32:43.525 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369469+0000", 2024-09-14T06:32:43.525 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.0170000000000001e-06 2024-09-14T06:32:43.525 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:43.525 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:43.525 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:32:43.525 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369472+0000", 2024-09-14T06:32:43.525 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.6189999999999998e-06 2024-09-14T06:32:43.525 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:43.525 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:43.525 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:32:43.525 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.417383+0000", 2024-09-14T06:32:43.525 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047911092000000002 2024-09-14T06:32:43.525 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:43.526 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:43.526 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:32:43.526 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.417430+0000", 2024-09-14T06:32:43.526 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.6898999999999999e-05 2024-09-14T06:32:43.526 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:43.526 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:43.526 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:32:43.526 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.417497+0000", 2024-09-14T06:32:43.526 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.7317999999999995e-05 2024-09-14T06:32:43.526 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:43.526 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:32:43.526 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:43.526 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:43.526 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:43.526 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:53251 3.f 3:f71f51a7:::rbd_data.112dca2f6ae1.0000000000000015:17 [read 2035712~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:32:43.526 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:44.369475+0000", 2024-09-14T06:32:43.527 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 179.12628093699999, 2024-09-14T06:32:43.527 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.048154542000000002, 2024-09-14T06:32:43.527 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:32:43.527 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:32:43.527 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:32:43.527 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:32:43.527 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:32:43.527 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:43.527 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 53251 2024-09-14T06:32:43.527 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:43.527 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:32:43.527 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:43.527 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:32:43.527 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369475+0000", 2024-09-14T06:32:43.527 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:43.527 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:43.527 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:43.528 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:32:43.528 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369475+0000", 2024-09-14T06:32:43.528 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:43.528 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:43.528 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:43.528 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:32:43.528 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369475+0000", 2024-09-14T06:32:43.528 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.0100000000000001e-07 2024-09-14T06:32:43.528 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:43.528 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:43.528 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:32:43.528 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369476+0000", 2024-09-14T06:32:43.528 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.0580000000000001e-06 2024-09-14T06:32:43.528 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:43.528 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:43.529 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:32:43.529 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369479+0000", 2024-09-14T06:32:43.529 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.9950000000000001e-06 2024-09-14T06:32:43.529 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:43.529 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:43.529 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:32:43.529 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369482+0000", 2024-09-14T06:32:43.529 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.6340000000000001e-06 2024-09-14T06:32:43.529 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:43.529 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:43.529 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:32:43.529 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.417530+0000", 2024-09-14T06:32:43.529 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.048047535000000002 2024-09-14T06:32:43.529 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:43.530 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:43.530 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:32:43.530 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.417574+0000", 2024-09-14T06:32:43.530 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.4255999999999998e-05 2024-09-14T06:32:43.530 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:43.530 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:43.530 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:32:43.530 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.417630+0000", 2024-09-14T06:32:43.530 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.5862999999999997e-05 2024-09-14T06:32:43.530 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:43.530 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:32:43.530 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:43.530 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:43.530 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:43.531 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:53252 3.f 3:f71f51a7:::rbd_data.112dca2f6ae1.0000000000000015:17 [read 2060288~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:32:43.531 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:44.369485+0000", 2024-09-14T06:32:43.531 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 179.126271089, 2024-09-14T06:32:43.531 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.048254116, 2024-09-14T06:32:43.531 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:32:43.531 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:32:43.531 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:32:43.531 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:32:43.531 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:32:43.531 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:43.531 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 53252 2024-09-14T06:32:43.531 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:43.531 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:32:43.531 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:43.531 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:32:43.532 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369485+0000", 2024-09-14T06:32:43.532 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:43.532 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:43.532 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:43.532 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:32:43.532 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369485+0000", 2024-09-14T06:32:43.532 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:43.532 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:43.532 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:43.532 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:32:43.532 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369485+0000", 2024-09-14T06:32:43.532 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.1e-07 2024-09-14T06:32:43.532 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:43.532 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:43.532 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:32:43.532 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369487+0000", 2024-09-14T06:32:43.533 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.42e-06 2024-09-14T06:32:43.533 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:43.533 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:43.533 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:32:43.533 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369490+0000", 2024-09-14T06:32:43.533 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.9639999999999999e-06 2024-09-14T06:32:43.533 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:43.533 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:43.533 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:32:43.533 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369492+0000", 2024-09-14T06:32:43.533 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.5679999999999998e-06 2024-09-14T06:32:43.533 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:43.533 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:43.533 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:32:43.534 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.417645+0000", 2024-09-14T06:32:43.534 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.048152753 2024-09-14T06:32:43.534 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:43.534 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:43.534 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:32:43.534 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.417687+0000", 2024-09-14T06:32:43.534 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.1900999999999997e-05 2024-09-14T06:32:43.534 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:43.534 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:43.534 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:32:43.534 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.417739+0000", 2024-09-14T06:32:43.534 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.2299999999999997e-05 2024-09-14T06:32:43.534 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:43.534 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:32:43.534 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:43.535 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:43.535 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:43.535 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:53253 3.f 3:f71f51a7:::rbd_data.112dca2f6ae1.0000000000000015:17 [read 2084864~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:32:43.535 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:44.369495+0000", 2024-09-14T06:32:43.535 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 179.126260965, 2024-09-14T06:32:43.535 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.048366785000000002, 2024-09-14T06:32:43.535 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:32:43.535 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:32:43.535 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:32:43.535 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:32:43.535 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:32:43.535 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:43.535 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 53253 2024-09-14T06:32:43.535 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:43.535 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:32:43.535 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:43.536 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:32:43.536 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369495+0000", 2024-09-14T06:32:43.536 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:43.536 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:43.536 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:43.536 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:32:43.536 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369495+0000", 2024-09-14T06:32:43.536 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:43.536 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:43.536 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:43.536 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:32:43.536 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370388+0000", 2024-09-14T06:32:43.536 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00089287999999999998 2024-09-14T06:32:43.536 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:43.537 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:43.537 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:32:43.537 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370392+0000", 2024-09-14T06:32:43.537 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.2130000000000002e-06 2024-09-14T06:32:43.537 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:43.537 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:43.537 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:32:43.537 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370401+0000", 2024-09-14T06:32:43.537 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.7649999999999999e-06 2024-09-14T06:32:43.537 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:43.537 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:43.537 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:32:43.537 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370406+0000", 2024-09-14T06:32:43.537 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.0159999999999999e-06 2024-09-14T06:32:43.538 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:43.538 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:43.538 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:32:43.538 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.417764+0000", 2024-09-14T06:32:43.538 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047358348000000001 2024-09-14T06:32:43.538 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:43.538 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:43.538 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:32:43.538 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.417806+0000", 2024-09-14T06:32:43.538 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.1149999999999997e-05 2024-09-14T06:32:43.538 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:43.538 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:43.538 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:32:43.539 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.417862+0000", 2024-09-14T06:32:43.539 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.6412999999999997e-05 2024-09-14T06:32:43.539 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:43.539 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:32:43.539 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:43.539 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:43.539 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:43.539 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:53340 3.f 3:f71f51a7:::rbd_data.112dca2f6ae1.0000000000000015:17 [read 1056768~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:32:43.539 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:44.370421+0000", 2024-09-14T06:32:43.539 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 179.12533508499999, 2024-09-14T06:32:43.539 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047566313999999998, 2024-09-14T06:32:43.539 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:32:43.539 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:32:43.539 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:32:43.539 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:32:43.540 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:32:43.540 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:43.540 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 53340 2024-09-14T06:32:43.540 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:43.540 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:32:43.540 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:43.540 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:32:43.540 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370421+0000", 2024-09-14T06:32:43.540 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:43.540 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:43.540 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:43.540 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:32:43.540 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370421+0000", 2024-09-14T06:32:43.540 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:43.540 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:43.540 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:43.541 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:32:43.541 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370422+0000", 2024-09-14T06:32:43.541 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.3799999999999996e-07 2024-09-14T06:32:43.541 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:43.541 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:43.541 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:32:43.541 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370424+0000", 2024-09-14T06:32:43.541 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.6919999999999998e-06 2024-09-14T06:32:43.541 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:43.541 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:43.541 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:32:43.541 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370432+0000", 2024-09-14T06:32:43.541 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.2439999999999996e-06 2024-09-14T06:32:43.541 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:43.542 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:43.542 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:32:43.542 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370437+0000", 2024-09-14T06:32:43.542 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.5389999999999998e-06 2024-09-14T06:32:43.542 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:43.542 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:43.542 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:32:43.542 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.417888+0000", 2024-09-14T06:32:43.542 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047450432000000001 2024-09-14T06:32:43.542 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:43.542 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:43.542 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:32:43.542 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.417935+0000", 2024-09-14T06:32:43.542 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.7454999999999999e-05 2024-09-14T06:32:43.543 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:43.543 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:43.543 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:32:43.543 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.417987+0000", 2024-09-14T06:32:43.543 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.2213999999999998e-05 2024-09-14T06:32:43.543 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:43.543 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:32:43.543 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:43.543 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:43.543 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:43.543 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:53341 3.f 3:f71f51a7:::rbd_data.112dca2f6ae1.0000000000000015:17 [read 1081344~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:32:43.543 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:44.370446+0000", 2024-09-14T06:32:43.543 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 179.12530981200001, 2024-09-14T06:32:43.543 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047665723, 2024-09-14T06:32:43.543 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:32:43.544 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:32:43.544 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:32:43.544 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:32:43.544 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:32:43.544 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:43.544 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 53341 2024-09-14T06:32:43.544 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:43.544 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:32:43.544 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:43.544 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:32:43.544 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370446+0000", 2024-09-14T06:32:43.544 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:43.544 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:43.544 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:43.544 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:32:43.545 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370446+0000", 2024-09-14T06:32:43.545 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:43.545 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:43.545 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:43.545 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:32:43.545 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370447+0000", 2024-09-14T06:32:43.545 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.6100000000000002e-07 2024-09-14T06:32:43.545 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:43.545 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:43.545 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:32:43.545 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370448+0000", 2024-09-14T06:32:43.545 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.7370000000000001e-06 2024-09-14T06:32:43.545 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:43.545 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:43.545 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:32:43.546 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370462+0000", 2024-09-14T06:32:43.546 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.3227000000000001e-05 2024-09-14T06:32:43.546 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:43.546 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:43.546 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:32:43.546 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370467+0000", 2024-09-14T06:32:43.546 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.6540000000000004e-06 2024-09-14T06:32:43.546 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:43.546 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:43.546 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:32:43.546 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.418013+0000", 2024-09-14T06:32:43.546 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047545325999999999 2024-09-14T06:32:43.546 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:43.546 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:43.546 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:32:43.546 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.418054+0000", 2024-09-14T06:32:43.547 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.1696999999999997e-05 2024-09-14T06:32:43.547 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:43.547 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:43.547 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:32:43.547 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.418112+0000", 2024-09-14T06:32:43.547 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.7720999999999998e-05 2024-09-14T06:32:43.547 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:43.547 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:32:43.547 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:43.547 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:43.547 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:43.547 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:53342 3.f 3:f71f51a7:::rbd_data.112dca2f6ae1.0000000000000015:17 [read 1105920~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:32:43.547 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:44.370475+0000", 2024-09-14T06:32:43.547 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 179.12528075099999, 2024-09-14T06:32:43.547 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047748848000000003, 2024-09-14T06:32:43.547 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:32:43.548 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:32:43.548 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:32:43.548 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:32:43.548 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:32:43.548 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:43.548 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 53342 2024-09-14T06:32:43.548 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:43.548 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:32:43.548 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:43.548 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:32:43.548 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370475+0000", 2024-09-14T06:32:43.548 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:43.548 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:43.548 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:43.548 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:32:43.549 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370475+0000", 2024-09-14T06:32:43.549 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:43.549 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:43.549 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:43.549 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:32:43.549 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370476+0000", 2024-09-14T06:32:43.549 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.4700000000000002e-07 2024-09-14T06:32:43.549 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:43.549 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:43.549 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:32:43.549 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370478+0000", 2024-09-14T06:32:43.549 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.5890000000000001e-06 2024-09-14T06:32:43.549 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:43.550 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:43.550 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:32:43.550 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370486+0000", 2024-09-14T06:32:43.550 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.3470000000000001e-06 2024-09-14T06:32:43.550 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:43.550 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:43.550 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:32:43.550 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370491+0000", 2024-09-14T06:32:43.550 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.5720000000000002e-06 2024-09-14T06:32:43.550 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:43.550 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:43.550 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:32:43.550 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.418135+0000", 2024-09-14T06:32:43.551 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047644094999999997 2024-09-14T06:32:43.551 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:43.551 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:43.551 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:32:43.551 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.418174+0000", 2024-09-14T06:32:43.551 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.8204000000000002e-05 2024-09-14T06:32:43.551 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:43.551 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:43.551 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:32:43.551 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.418224+0000", 2024-09-14T06:32:43.551 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.0593999999999997e-05 2024-09-14T06:32:43.551 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:43.551 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:32:43.551 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:43.551 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:43.552 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:43.552 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:53343 3.f 3:f71f51a7:::rbd_data.112dca2f6ae1.0000000000000015:17 [read 1130496~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:32:43.552 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:44.370498+0000", 2024-09-14T06:32:43.552 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 179.12525787499999, 2024-09-14T06:32:43.552 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047841574999999997, 2024-09-14T06:32:43.552 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:32:43.552 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:32:43.552 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:32:43.552 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:32:43.552 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:32:43.552 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:43.552 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 53343 2024-09-14T06:32:43.552 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:43.552 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:32:43.552 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:43.553 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:32:43.553 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370498+0000", 2024-09-14T06:32:43.553 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:43.553 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:43.553 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:43.553 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:32:43.553 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370498+0000", 2024-09-14T06:32:43.553 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:43.553 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:43.553 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:43.553 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:32:43.553 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370499+0000", 2024-09-14T06:32:43.553 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.9999999999999998e-07 2024-09-14T06:32:43.553 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:43.553 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:43.554 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:32:43.554 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370501+0000", 2024-09-14T06:32:43.554 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.0310000000000001e-06 2024-09-14T06:32:43.554 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:43.554 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:43.554 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:32:43.554 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370507+0000", 2024-09-14T06:32:43.554 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.4169999999999997e-06 2024-09-14T06:32:43.554 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:43.554 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:43.554 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:32:43.554 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370516+0000", 2024-09-14T06:32:43.554 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.7190000000000007e-06 2024-09-14T06:32:43.554 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:43.554 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:43.554 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:32:43.554 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.418247+0000", 2024-09-14T06:32:43.555 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047731190999999999 2024-09-14T06:32:43.555 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:43.555 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:43.555 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:32:43.555 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.418286+0000", 2024-09-14T06:32:43.555 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.8988e-05 2024-09-14T06:32:43.555 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:43.555 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:43.555 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:32:43.555 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.418340+0000", 2024-09-14T06:32:43.555 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.3829000000000002e-05 2024-09-14T06:32:43.555 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:43.555 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:32:43.555 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:43.555 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:43.556 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:43.556 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:53344 3.f 3:f71f51a7:::rbd_data.112dca2f6ae1.0000000000000015:17 [read 1155072~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:32:43.556 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:44.370524+0000", 2024-09-14T06:32:43.556 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 179.12523210500001, 2024-09-14T06:32:43.556 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047972262000000002, 2024-09-14T06:32:43.556 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:32:43.556 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:32:43.556 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:32:43.556 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:32:43.556 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:32:43.556 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:43.556 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 53344 2024-09-14T06:32:43.556 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:43.556 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:32:43.556 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:43.556 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:32:43.557 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370524+0000", 2024-09-14T06:32:43.557 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:43.557 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:43.557 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:43.557 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:32:43.557 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370524+0000", 2024-09-14T06:32:43.557 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:43.557 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:43.557 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:43.557 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:32:43.557 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370618+0000", 2024-09-14T06:32:43.557 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.4425999999999993e-05 2024-09-14T06:32:43.557 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:43.557 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:43.557 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:32:43.558 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370622+0000", 2024-09-14T06:32:43.558 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.3519999999999998e-06 2024-09-14T06:32:43.558 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:43.558 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:43.558 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:32:43.558 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370631+0000", 2024-09-14T06:32:43.558 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.9299999999999992e-06 2024-09-14T06:32:43.558 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:43.558 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:43.558 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:32:43.558 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370637+0000", 2024-09-14T06:32:43.558 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.3269999999999997e-06 2024-09-14T06:32:43.558 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:43.558 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:43.558 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:32:43.558 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.418372+0000", 2024-09-14T06:32:43.558 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047734764999999998 2024-09-14T06:32:43.559 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:43.559 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:43.559 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:32:43.559 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.418431+0000", 2024-09-14T06:32:43.559 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.8696999999999997e-05 2024-09-14T06:32:43.559 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:43.559 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:43.559 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:32:43.559 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.418496+0000", 2024-09-14T06:32:43.559 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.5765000000000005e-05 2024-09-14T06:32:43.559 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:43.559 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:32:43.559 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:43.559 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:43.559 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:43.559 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:53345 3.f 3:f71f51a7:::rbd_data.112dca2f6ae1.0000000000000015:17 [read 1179648~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:32:43.560 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:44.370647+0000", 2024-09-14T06:32:43.560 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 179.125108638, 2024-09-14T06:32:43.560 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047953775999999997, 2024-09-14T06:32:43.560 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:32:43.560 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:32:43.560 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:32:43.560 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:32:43.560 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:32:43.560 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:43.560 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 53345 2024-09-14T06:32:43.560 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:43.560 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:32:43.560 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:43.560 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:32:43.560 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370647+0000", 2024-09-14T06:32:43.561 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:43.561 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:43.561 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:43.561 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:32:43.561 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370647+0000", 2024-09-14T06:32:43.561 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:43.561 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:43.561 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:43.561 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:32:43.561 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.372346+0000", 2024-09-14T06:32:43.561 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.001698284 2024-09-14T06:32:43.561 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:43.561 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:43.561 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:32:43.562 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.372351+0000", 2024-09-14T06:32:43.562 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.8890000000000001e-06 2024-09-14T06:32:43.562 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:43.562 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:43.562 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:32:43.562 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.372362+0000", 2024-09-14T06:32:43.562 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.0941000000000001e-05 2024-09-14T06:32:43.562 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:43.562 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:43.562 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:32:43.562 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.372369+0000", 2024-09-14T06:32:43.562 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.362e-06 2024-09-14T06:32:43.562 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:43.562 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:43.562 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:32:43.562 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.418510+0000", 2024-09-14T06:32:43.563 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046140914999999998 2024-09-14T06:32:43.563 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:43.563 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:43.563 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:32:43.563 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.418549+0000", 2024-09-14T06:32:43.563 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.8717999999999998e-05 2024-09-14T06:32:43.563 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:43.563 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:43.563 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:32:43.563 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.418601+0000", 2024-09-14T06:32:43.563 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.2667000000000002e-05 2024-09-14T06:32:43.563 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:43.563 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:32:43.563 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:43.563 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:43.563 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:43.564 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:53353 3.f 3:f71f51a7:::rbd_data.112dca2f6ae1.0000000000000015:17 [read 1376256~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:32:43.564 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:44.372545+0000", 2024-09-14T06:32:43.564 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 179.12321156300001, 2024-09-14T06:32:43.564 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047364232999999999, 2024-09-14T06:32:43.564 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:32:43.564 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:32:43.564 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:32:43.564 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:32:43.564 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:32:43.564 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:43.564 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 53353 2024-09-14T06:32:43.564 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:43.564 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:32:43.564 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:43.564 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:32:43.565 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.372545+0000", 2024-09-14T06:32:43.565 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:43.565 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:43.565 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:43.565 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:32:43.565 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.372545+0000", 2024-09-14T06:32:43.565 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:43.565 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:43.565 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:43.565 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:32:43.565 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.372545+0000", 2024-09-14T06:32:43.565 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.27e-07 2024-09-14T06:32:43.565 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:43.565 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:43.565 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:32:43.565 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.372547+0000", 2024-09-14T06:32:43.566 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.0600000000000002e-06 2024-09-14T06:32:43.566 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:43.566 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:43.566 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:32:43.566 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.372553+0000", 2024-09-14T06:32:43.566 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.7980000000000002e-06 2024-09-14T06:32:43.566 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:43.566 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:43.566 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:32:43.566 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.372557+0000", 2024-09-14T06:32:43.566 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.7389999999999999e-06 2024-09-14T06:32:43.566 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:43.566 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:43.566 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:32:43.566 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.419811+0000", 2024-09-14T06:32:43.566 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047253310999999999 2024-09-14T06:32:43.567 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:43.567 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:43.567 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:32:43.567 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.419854+0000", 2024-09-14T06:32:43.567 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.3448000000000001e-05 2024-09-14T06:32:43.567 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:43.567 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:43.567 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:32:43.567 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.419909+0000", 2024-09-14T06:32:43.567 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.4549999999999998e-05 2024-09-14T06:32:43.567 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:43.567 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:32:43.567 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:43.567 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:43.567 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:43.567 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:53354 3.f 3:f71f51a7:::rbd_data.112dca2f6ae1.0000000000000015:17 [read 1400832~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:32:43.568 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:44.372563+0000", 2024-09-14T06:32:43.568 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 179.12319312599999, 2024-09-14T06:32:43.568 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047482233999999998, 2024-09-14T06:32:43.568 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:32:43.568 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:32:43.568 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:32:43.568 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:32:43.568 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:32:43.568 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:43.568 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 53354 2024-09-14T06:32:43.568 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:43.568 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:32:43.568 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:43.568 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:32:43.568 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.372563+0000", 2024-09-14T06:32:43.568 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:43.569 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:43.569 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:43.569 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:32:43.569 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.372563+0000", 2024-09-14T06:32:43.569 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:43.569 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:43.569 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:43.569 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:32:43.569 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.375009+0000", 2024-09-14T06:32:43.569 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.0024457680000000001 2024-09-14T06:32:43.569 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:43.569 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:43.569 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:32:43.569 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.375014+0000", 2024-09-14T06:32:43.570 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.0599999999999998e-06 2024-09-14T06:32:43.570 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:43.570 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:43.570 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:32:43.570 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.375026+0000", 2024-09-14T06:32:43.570 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.1783999999999999e-05 2024-09-14T06:32:43.570 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:43.570 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:43.570 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:32:43.570 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.375035+0000", 2024-09-14T06:32:43.570 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.7799999999999995e-06 2024-09-14T06:32:43.570 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:43.570 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:43.570 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:32:43.571 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.419942+0000", 2024-09-14T06:32:43.571 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.044906982999999998 2024-09-14T06:32:43.571 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:43.571 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:43.571 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:32:43.571 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.419989+0000", 2024-09-14T06:32:43.571 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.7104999999999997e-05 2024-09-14T06:32:43.571 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:43.571 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:43.571 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:32:43.571 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.420045+0000", 2024-09-14T06:32:43.571 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.5754000000000002e-05 2024-09-14T06:32:43.571 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:43.571 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:32:43.571 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:43.571 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:43.571 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:43.572 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:225577 3.15 3:ac159a70:::rbd_header.112daa9ec306:head [watch reconnect cookie 140616827151664 gen 4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e188)", 2024-09-14T06:32:43.572 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:31:52.733493+0000", 2024-09-14T06:32:43.572 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 50.762262661999998, 2024-09-14T06:32:43.572 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.0156593789999999, 2024-09-14T06:32:43.572 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:32:43.572 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:32:43.572 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:32:43.572 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:32:43.572 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:32:43.572 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:43.572 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 225577 2024-09-14T06:32:43.572 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:43.572 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:32:43.572 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:43.572 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:32:43.573 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:31:52.733493+0000", 2024-09-14T06:32:43.573 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:43.573 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:43.573 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:43.573 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:32:43.573 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:31:52.733493+0000", 2024-09-14T06:32:43.573 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:43.573 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:43.573 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:43.573 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:32:43.573 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:31:52.733495+0000", 2024-09-14T06:32:43.573 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.0059999999999999e-06 2024-09-14T06:32:43.573 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:43.573 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:43.573 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:32:43.573 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:31:52.733504+0000", 2024-09-14T06:32:43.574 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.5490000000000006e-06 2024-09-14T06:32:43.574 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:43.574 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:43.574 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:32:43.574 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:31:52.733522+0000", 2024-09-14T06:32:43.574 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.7660999999999999e-05 2024-09-14T06:32:43.574 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:43.574 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:43.574 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:32:43.574 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:31:52.733531+0000", 2024-09-14T06:32:43.574 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.7180000000000008e-06 2024-09-14T06:32:43.574 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:43.574 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:43.574 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:32:43.574 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:31:52.733598+0000", 2024-09-14T06:32:43.574 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.6490999999999997e-05 2024-09-14T06:32:43.575 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:43.575 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:43.575 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "op must wait for map", 2024-09-14T06:32:43.575 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:31:52.733605+0000", 2024-09-14T06:32:43.575 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.6109999999999996e-06 2024-09-14T06:32:43.575 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:43.575 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:43.575 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:32:43.575 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:31:52.737908+0000", 2024-09-14T06:32:43.575 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.0043023050000000002 2024-09-14T06:32:43.575 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:43.575 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:43.575 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "waiting for peered", 2024-09-14T06:32:43.575 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:31:52.737923+0000", 2024-09-14T06:32:43.576 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.488e-05 2024-09-14T06:32:43.576 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:43.576 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:43.576 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:32:43.576 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:31:53.746776+0000", 2024-09-14T06:32:43.576 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.0088535359999999 2024-09-14T06:32:43.576 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:43.576 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:43.576 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:32:43.576 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:31:53.746878+0000", 2024-09-14T06:32:43.576 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.000101788 2024-09-14T06:32:43.576 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:43.576 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:43.576 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:32:43.577 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:31:53.749153+0000", 2024-09-14T06:32:43.577 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.0022748339999999999 2024-09-14T06:32:43.577 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:43.577 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:32:43.577 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:43.577 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:43.577 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:43.577 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:225581 3.15 3:ab407115:::rbd_object_map.112d838b9898:head [call lock.unlock in=10b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e188)", 2024-09-14T06:32:43.577 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:31:52.743472+0000", 2024-09-14T06:32:43.577 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 50.752283622999997, 2024-09-14T06:32:43.577 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.007270501, 2024-09-14T06:32:43.577 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:32:43.577 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:32:43.577 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:32:43.577 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:32:43.578 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:32:43.578 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:43.578 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 225581 2024-09-14T06:32:43.578 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:43.578 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:32:43.578 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:43.578 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:32:43.578 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:31:52.743472+0000", 2024-09-14T06:32:43.578 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:43.578 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:43.578 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:43.578 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:32:43.578 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:31:52.743472+0000", 2024-09-14T06:32:43.578 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:43.578 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:43.578 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:43.579 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:32:43.579 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:31:52.743473+0000", 2024-09-14T06:32:43.579 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.1999999999999998e-07 2024-09-14T06:32:43.579 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:43.579 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:43.579 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:32:43.579 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:31:52.743478+0000", 2024-09-14T06:32:43.579 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.515e-06 2024-09-14T06:32:43.579 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:43.579 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:43.579 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:32:43.579 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:31:52.743489+0000", 2024-09-14T06:32:43.579 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.1411e-05 2024-09-14T06:32:43.579 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:43.579 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:43.580 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:32:43.580 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:31:52.743494+0000", 2024-09-14T06:32:43.580 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.1649999999999997e-06 2024-09-14T06:32:43.580 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:43.580 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:43.580 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:32:43.580 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:31:52.743548+0000", 2024-09-14T06:32:43.580 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.3758999999999998e-05 2024-09-14T06:32:43.580 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:43.580 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:43.580 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "waiting for peered", 2024-09-14T06:32:43.580 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:31:52.743558+0000", 2024-09-14T06:32:43.580 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.0336999999999999e-05 2024-09-14T06:32:43.580 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:43.580 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:43.581 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:32:43.581 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:31:53.747716+0000", 2024-09-14T06:32:43.581 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.004157035 2024-09-14T06:32:43.581 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:43.581 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:43.581 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:32:43.581 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:31:53.747831+0000", 2024-09-14T06:32:43.581 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.000115394 2024-09-14T06:32:43.581 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:43.581 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:43.581 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "waiting for subops from 3", 2024-09-14T06:32:43.581 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:31:53.748002+0000", 2024-09-14T06:32:43.581 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00017085099999999999 2024-09-14T06:32:43.581 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:43.581 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:43.582 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:32:43.582 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:31:53.749536+0000", 2024-09-14T06:32:43.582 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.0015339990000000001 2024-09-14T06:32:43.582 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:43.582 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:43.582 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "op_commit", 2024-09-14T06:32:43.582 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:31:53.750672+0000", 2024-09-14T06:32:43.582 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.001135973 2024-09-14T06:32:43.582 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:43.582 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:43.582 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "commit_sent", 2024-09-14T06:32:43.582 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:31:53.750704+0000", 2024-09-14T06:32:43.582 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.1791000000000001e-05 2024-09-14T06:32:43.582 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:43.582 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:43.582 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:32:43.582 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:31:53.750743+0000", 2024-09-14T06:32:43.583 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.9450999999999999e-05 2024-09-14T06:32:43.583 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:43.583 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:32:43.583 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:43.583 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:43.583 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:32:43.583 INFO:teuthology.orchestra.run.smithi138.stdout:} 2024-09-14T06:32:43.584 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-14T06:32:43.585 INFO:tasks.ceph.osd.1.smithi094.stderr:2024-09-14T06:32:43.583+0000 7f328e47a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33641) UID: 0 2024-09-14T06:32:43.685 INFO:tasks.ceph.osd.5.smithi138.stderr:2024-09-14T06:32:43.684+0000 7fe13746f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33618) UID: 0 2024-09-14T06:32:43.702 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-14T06:32:43.702 INFO:teuthology.orchestra.run.smithi094.stdout: "ops": [ 2024-09-14T06:32:43.702 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:43.702 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:237116 3.1 3:80a992c4:::rbd_data.112d92ae6cee.000000000000000c:head [delete] snapc 27=[] ondisk+write+known_if_redirected+supports_pool_eio e238)", 2024-09-14T06:32:43.702 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:32:43.695946+0000", 2024-09-14T06:32:43.702 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 0.0054066469999999997, 2024-09-14T06:32:43.702 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0054241280000000003, 2024-09-14T06:32:43.702 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:32:43.702 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:32:43.702 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "waiting for sub ops", 2024-09-14T06:32:43.702 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:32:43.702 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:32:43.703 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:43.703 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 237116 2024-09-14T06:32:43.703 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:43.703 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:32:43.703 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:43.703 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:32:43.703 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:43.695946+0000", 2024-09-14T06:32:43.703 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:43.703 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:43.703 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:43.703 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:32:43.703 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:43.695946+0000", 2024-09-14T06:32:43.703 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:43.703 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:43.704 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:43.704 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:32:43.704 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:43.695947+0000", 2024-09-14T06:32:43.704 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.1999999999999999e-07 2024-09-14T06:32:43.704 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:43.704 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:43.704 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:32:43.704 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:43.695950+0000", 2024-09-14T06:32:43.704 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.5090000000000002e-06 2024-09-14T06:32:43.704 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:43.704 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:43.704 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:32:43.704 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:43.695960+0000", 2024-09-14T06:32:43.705 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.008e-05 2024-09-14T06:32:43.705 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:43.705 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:43.705 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:32:43.705 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:43.695965+0000", 2024-09-14T06:32:43.705 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.1529999999999996e-06 2024-09-14T06:32:43.705 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:43.705 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:43.705 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:32:43.705 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:43.696013+0000", 2024-09-14T06:32:43.705 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.8015e-05 2024-09-14T06:32:43.705 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:43.705 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:43.706 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:32:43.706 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:43.696099+0000", 2024-09-14T06:32:43.706 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.5865999999999994e-05 2024-09-14T06:32:43.706 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:43.706 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:43.706 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "waiting for subops from 4", 2024-09-14T06:32:43.706 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:43.696178+0000", 2024-09-14T06:32:43.706 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.8608000000000004e-05 2024-09-14T06:32:43.706 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:43.706 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:43.706 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "op_commit", 2024-09-14T06:32:43.706 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:43.696893+0000", 2024-09-14T06:32:43.706 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00071556899999999999 2024-09-14T06:32:43.706 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:43.707 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:32:43.707 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:43.707 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:43.707 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-09-14T06:32:43.707 INFO:teuthology.orchestra.run.smithi094.stdout: "num_ops": 1 2024-09-14T06:32:43.707 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-14T06:32:43.712 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-14T06:32:43.786 INFO:tasks.ceph.osd.0.smithi094.stderr:2024-09-14T06:32:43.785+0000 7f07c9333640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33636) UID: 0 2024-09-14T06:32:43.819 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-14T06:32:43.819 INFO:teuthology.orchestra.run.smithi094.stdout: "ops": [], 2024-09-14T06:32:43.820 INFO:teuthology.orchestra.run.smithi094.stdout: "complaint_time": 30, 2024-09-14T06:32:43.820 INFO:teuthology.orchestra.run.smithi094.stdout: "num_blocked_ops": 0 2024-09-14T06:32:43.820 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-14T06:32:43.828 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-14T06:32:43.886 INFO:tasks.ceph.osd.7.smithi138.stderr:2024-09-14T06:32:43.886+0000 7f3c6a21d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33623) UID: 0 2024-09-14T06:32:43.972 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-14T06:32:43.972 INFO:teuthology.orchestra.run.smithi094.stdout: "size": 20, 2024-09-14T06:32:43.972 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 600, 2024-09-14T06:32:43.972 INFO:teuthology.orchestra.run.smithi094.stdout: "ops": [ 2024-09-14T06:32:43.972 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:43.972 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:115193 3.1 3:856be886:::rbd_data.112d4d24c558.0000000000000004:29 [read 1523712~16384 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e96)", 2024-09-14T06:32:43.973 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:30:17.939854+0000", 2024-09-14T06:32:43.973 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 146.030698239, 2024-09-14T06:32:43.973 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.073897516999999996, 2024-09-14T06:32:43.973 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:32:43.973 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:32:43.973 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:32:43.973 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:32:43.973 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:32:43.973 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:43.973 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 115193 2024-09-14T06:32:43.973 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:43.973 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:32:43.973 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:43.974 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:32:43.974 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.939854+0000", 2024-09-14T06:32:43.974 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:43.974 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:43.974 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:43.974 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:32:43.974 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.939854+0000", 2024-09-14T06:32:43.974 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:43.974 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:43.974 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:43.974 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:32:43.974 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.939854+0000", 2024-09-14T06:32:43.974 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.9500000000000002e-07 2024-09-14T06:32:43.974 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:43.974 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:43.975 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:32:43.975 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.939856+0000", 2024-09-14T06:32:43.975 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.0959999999999999e-06 2024-09-14T06:32:43.975 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:43.975 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:43.975 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:32:43.975 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.939863+0000", 2024-09-14T06:32:43.975 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.1129999999999997e-06 2024-09-14T06:32:43.975 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:43.975 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:43.975 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:32:43.975 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.939867+0000", 2024-09-14T06:32:43.976 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.8350000000000003e-06 2024-09-14T06:32:43.976 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:43.976 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:43.976 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:32:43.976 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.013626+0000", 2024-09-14T06:32:43.976 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.073758799999999999 2024-09-14T06:32:43.976 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:43.976 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:43.976 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:32:43.976 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.013690+0000", 2024-09-14T06:32:43.976 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.3918999999999996e-05 2024-09-14T06:32:43.976 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:43.976 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:43.976 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:32:43.977 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.013751+0000", 2024-09-14T06:32:43.977 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.1159000000000002e-05 2024-09-14T06:32:43.977 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:43.977 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:32:43.977 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:43.977 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:43.977 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:43.977 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:115194 3.1 3:856be886:::rbd_data.112d4d24c558.0000000000000004:29 [read 1769472~16384 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e96)", 2024-09-14T06:32:43.977 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:30:17.939874+0000", 2024-09-14T06:32:43.977 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 146.03067792799999, 2024-09-14T06:32:43.977 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.074016988000000006, 2024-09-14T06:32:43.977 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:32:43.977 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:32:43.978 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:32:43.978 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:32:43.978 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:32:43.978 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:43.978 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 115194 2024-09-14T06:32:43.978 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:43.978 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:32:43.978 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:43.978 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:32:43.978 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.939874+0000", 2024-09-14T06:32:43.978 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:43.978 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:43.978 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:43.978 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:32:43.979 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.939874+0000", 2024-09-14T06:32:43.979 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:43.979 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:43.979 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:43.979 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:32:43.979 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.939875+0000", 2024-09-14T06:32:43.979 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.2600000000000002e-07 2024-09-14T06:32:43.979 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:43.979 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:43.979 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:32:43.979 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.939877+0000", 2024-09-14T06:32:43.979 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.9420000000000002e-06 2024-09-14T06:32:43.979 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:43.979 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:43.979 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:32:43.980 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.939882+0000", 2024-09-14T06:32:43.980 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.8749999999999997e-06 2024-09-14T06:32:43.980 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:43.980 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:43.980 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:32:43.980 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.939887+0000", 2024-09-14T06:32:43.980 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.741e-06 2024-09-14T06:32:43.980 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:43.980 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:43.980 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:32:43.980 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.013767+0000", 2024-09-14T06:32:43.980 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.073879720999999995 2024-09-14T06:32:43.980 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:43.981 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:43.981 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:32:43.981 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.013832+0000", 2024-09-14T06:32:43.981 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.5312000000000001e-05 2024-09-14T06:32:43.981 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:43.981 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:43.981 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:32:43.981 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.013891+0000", 2024-09-14T06:32:43.981 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.8870999999999999e-05 2024-09-14T06:32:43.981 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:43.981 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:32:43.981 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:43.981 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:43.981 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:43.981 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:115195 3.1 3:856be886:::rbd_data.112d4d24c558.0000000000000004:29 [read 2015232~16384 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e96)", 2024-09-14T06:32:43.982 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:30:17.939903+0000", 2024-09-14T06:32:43.982 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 146.030648743, 2024-09-14T06:32:43.982 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.074128199000000006, 2024-09-14T06:32:43.982 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:32:43.982 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:32:43.982 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:32:43.982 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:32:43.982 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:32:43.982 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:43.982 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 115195 2024-09-14T06:32:43.982 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:43.982 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:32:43.982 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:43.982 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:32:43.983 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.939903+0000", 2024-09-14T06:32:43.983 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:43.983 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:43.983 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:43.983 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:32:43.983 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.939903+0000", 2024-09-14T06:32:43.983 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:43.983 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:43.983 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:43.983 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:32:43.983 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.939904+0000", 2024-09-14T06:32:43.983 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.5799999999999999e-07 2024-09-14T06:32:43.983 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:43.983 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:43.984 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:32:43.984 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.939906+0000", 2024-09-14T06:32:43.984 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.2350000000000002e-06 2024-09-14T06:32:43.984 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:43.984 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:43.984 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:32:43.984 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.939913+0000", 2024-09-14T06:32:43.984 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.5649999999999998e-06 2024-09-14T06:32:43.984 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:43.984 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:43.984 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:32:43.984 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.939918+0000", 2024-09-14T06:32:43.984 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.784e-06 2024-09-14T06:32:43.984 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:43.985 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:43.985 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:32:43.985 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.013921+0000", 2024-09-14T06:32:43.985 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.074002911000000005 2024-09-14T06:32:43.985 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:43.985 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:43.985 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:32:43.985 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.013971+0000", 2024-09-14T06:32:43.985 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.9551e-05 2024-09-14T06:32:43.985 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:43.985 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:43.985 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:32:43.985 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.014031+0000", 2024-09-14T06:32:43.985 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.0594999999999998e-05 2024-09-14T06:32:43.986 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:43.986 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:32:43.986 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:43.986 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:43.986 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:43.986 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:115196 3.1 3:856be886:::rbd_data.112d4d24c558.0000000000000004:29 [read 2260992~16384 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e96)", 2024-09-14T06:32:43.986 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:30:17.939925+0000", 2024-09-14T06:32:43.986 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 146.03062655100001, 2024-09-14T06:32:43.986 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.074320137999999994, 2024-09-14T06:32:43.986 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:32:43.986 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:32:43.986 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:32:43.986 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:32:43.987 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:32:43.987 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:43.987 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 115196 2024-09-14T06:32:43.987 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:43.987 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:32:43.987 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:43.987 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:32:43.987 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.939925+0000", 2024-09-14T06:32:43.987 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:43.987 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:43.987 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:43.987 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:32:43.987 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.939925+0000", 2024-09-14T06:32:43.987 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:43.987 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:43.988 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:43.988 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:32:43.988 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.939926+0000", 2024-09-14T06:32:43.988 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.9999999999999998e-07 2024-09-14T06:32:43.988 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:43.988 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:43.988 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:32:43.988 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.939928+0000", 2024-09-14T06:32:43.988 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.9549999999999999e-06 2024-09-14T06:32:43.988 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:43.988 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:43.988 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:32:43.988 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.939934+0000", 2024-09-14T06:32:43.989 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.127e-06 2024-09-14T06:32:43.989 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:43.989 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:43.989 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:32:43.989 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.939939+0000", 2024-09-14T06:32:43.989 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.775e-06 2024-09-14T06:32:43.989 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:43.989 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:43.989 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:32:43.989 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.014075+0000", 2024-09-14T06:32:43.989 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.074136598999999997 2024-09-14T06:32:43.989 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:43.989 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:43.989 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:32:43.989 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.014162+0000", 2024-09-14T06:32:43.990 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.6477999999999994e-05 2024-09-14T06:32:43.990 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:43.990 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:43.990 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:32:43.990 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.014246+0000", 2024-09-14T06:32:43.990 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.3703999999999997e-05 2024-09-14T06:32:43.990 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:43.990 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:32:43.990 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:43.990 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:43.990 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:43.990 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:115197 3.1 3:856be886:::rbd_data.112d4d24c558.0000000000000004:29 [read 2506752~16384 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e96)", 2024-09-14T06:32:43.990 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:30:17.939969+0000", 2024-09-14T06:32:43.991 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 146.03058306599999, 2024-09-14T06:32:43.991 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.074528544000000002, 2024-09-14T06:32:43.991 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:32:43.991 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:32:43.991 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:32:43.991 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:32:43.991 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:32:43.991 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:43.991 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 115197 2024-09-14T06:32:43.991 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:43.991 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:32:43.991 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:43.991 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:32:43.991 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.939969+0000", 2024-09-14T06:32:43.991 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:43.992 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:43.992 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:43.992 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:32:43.992 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.939969+0000", 2024-09-14T06:32:43.992 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:43.992 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:43.992 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:43.992 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:32:43.992 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.939970+0000", 2024-09-14T06:32:43.992 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.3600000000000003e-07 2024-09-14T06:32:43.992 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:43.992 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:43.992 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:32:43.992 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.939972+0000", 2024-09-14T06:32:43.993 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.3180000000000001e-06 2024-09-14T06:32:43.993 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:43.993 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:43.993 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:32:43.993 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.939979+0000", 2024-09-14T06:32:43.993 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.8800000000000002e-06 2024-09-14T06:32:43.993 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:43.993 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:43.993 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:32:43.993 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.939984+0000", 2024-09-14T06:32:43.993 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.3510000000000001e-06 2024-09-14T06:32:43.993 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:43.993 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:43.993 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:32:43.994 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.014302+0000", 2024-09-14T06:32:43.994 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.074317416999999997 2024-09-14T06:32:43.994 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:43.994 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:43.994 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:32:43.994 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.014392+0000", 2024-09-14T06:32:43.994 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.0181999999999998e-05 2024-09-14T06:32:43.994 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:43.994 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:43.994 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:32:43.995 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.014497+0000", 2024-09-14T06:32:43.995 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00010576 2024-09-14T06:32:43.995 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:43.995 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:32:43.995 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:43.995 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:43.995 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:43.995 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:115198 3.1 3:856be886:::rbd_data.112d4d24c558.0000000000000004:29 [read 2752512~16384 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e96)", 2024-09-14T06:32:43.995 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:30:17.940004+0000", 2024-09-14T06:32:43.995 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 146.030547786, 2024-09-14T06:32:43.995 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.074736367999999997, 2024-09-14T06:32:43.996 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:32:43.996 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:32:43.996 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:32:43.996 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:32:43.996 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:32:43.996 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:43.996 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 115198 2024-09-14T06:32:43.996 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:43.996 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:32:43.996 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:43.996 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:32:43.996 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940004+0000", 2024-09-14T06:32:43.996 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:43.997 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:43.997 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:43.997 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:32:43.997 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940004+0000", 2024-09-14T06:32:43.997 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:43.997 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:43.997 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:43.997 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:32:43.997 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940005+0000", 2024-09-14T06:32:43.997 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.2200000000000004e-07 2024-09-14T06:32:43.997 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:43.997 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:43.997 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:32:43.997 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940007+0000", 2024-09-14T06:32:43.997 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.322e-06 2024-09-14T06:32:43.998 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:43.998 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:43.998 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:32:43.998 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940014+0000", 2024-09-14T06:32:43.998 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.6220000000000003e-06 2024-09-14T06:32:43.998 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:43.998 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:43.998 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:32:43.998 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940019+0000", 2024-09-14T06:32:43.998 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.6380000000000001e-06 2024-09-14T06:32:43.998 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:43.998 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:43.998 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:32:43.998 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.014562+0000", 2024-09-14T06:32:43.999 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.074542320999999995 2024-09-14T06:32:43.999 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:43.999 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:43.999 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:32:43.999 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.014646+0000", 2024-09-14T06:32:43.999 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.4079999999999995e-05 2024-09-14T06:32:43.999 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:43.999 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:43.999 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:32:43.999 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.014741+0000", 2024-09-14T06:32:43.999 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.4763000000000001e-05 2024-09-14T06:32:43.999 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:43.999 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:32:43.999 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:44.000 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:44.000 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:44.000 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:115199 3.1 3:856be886:::rbd_data.112d4d24c558.0000000000000004:29 [read 2998272~16384 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e96)", 2024-09-14T06:32:44.000 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:30:17.940040+0000", 2024-09-14T06:32:44.000 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 146.030512031, 2024-09-14T06:32:44.000 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.074957134999999994, 2024-09-14T06:32:44.000 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:32:44.000 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:32:44.000 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:32:44.000 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:32:44.000 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:32:44.000 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:44.000 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 115199 2024-09-14T06:32:44.000 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:44.001 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:32:44.001 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:44.001 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:32:44.001 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940040+0000", 2024-09-14T06:32:44.001 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:44.001 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:44.001 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:44.001 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:32:44.001 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940040+0000", 2024-09-14T06:32:44.001 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:44.001 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:44.001 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:44.002 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:32:44.002 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940041+0000", 2024-09-14T06:32:44.002 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.4000000000000001e-07 2024-09-14T06:32:44.002 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:44.002 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:44.002 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:32:44.002 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940046+0000", 2024-09-14T06:32:44.002 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.7250000000000002e-06 2024-09-14T06:32:44.002 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:44.002 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:44.002 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:32:44.002 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940053+0000", 2024-09-14T06:32:44.002 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.6109999999999999e-06 2024-09-14T06:32:44.002 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:44.003 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:44.003 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:32:44.003 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940058+0000", 2024-09-14T06:32:44.003 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.1590000000000001e-06 2024-09-14T06:32:44.003 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:44.003 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:44.003 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:32:44.003 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.014768+0000", 2024-09-14T06:32:44.003 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.074709624000000002 2024-09-14T06:32:44.003 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:44.003 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:44.003 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:32:44.003 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.014907+0000", 2024-09-14T06:32:44.003 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000139665 2024-09-14T06:32:44.004 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:44.004 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:44.004 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:32:44.004 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.014997+0000", 2024-09-14T06:32:44.004 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.9710999999999995e-05 2024-09-14T06:32:44.004 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:44.004 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:32:44.004 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:44.004 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:44.004 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:44.004 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:115200 3.1 3:856be886:::rbd_data.112d4d24c558.0000000000000004:29 [read 3244032~16384 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e96)", 2024-09-14T06:32:44.004 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:30:17.940080+0000", 2024-09-14T06:32:44.004 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 146.030471826, 2024-09-14T06:32:44.005 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.075137291999999994, 2024-09-14T06:32:44.005 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:32:44.005 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:32:44.005 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:32:44.005 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:32:44.005 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:32:44.005 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:44.005 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 115200 2024-09-14T06:32:44.005 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:44.005 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:32:44.005 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:44.005 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:32:44.005 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940080+0000", 2024-09-14T06:32:44.005 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:44.006 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:44.006 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:44.006 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:32:44.006 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940080+0000", 2024-09-14T06:32:44.006 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:44.006 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:44.006 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:44.006 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:32:44.006 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940081+0000", 2024-09-14T06:32:44.006 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.0800000000000004e-07 2024-09-14T06:32:44.006 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:44.006 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:44.006 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:32:44.006 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940083+0000", 2024-09-14T06:32:44.007 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.3920000000000001e-06 2024-09-14T06:32:44.007 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:44.007 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:44.007 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:32:44.007 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940091+0000", 2024-09-14T06:32:44.007 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.1389999999999995e-06 2024-09-14T06:32:44.007 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:44.007 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:44.007 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:32:44.007 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940097+0000", 2024-09-14T06:32:44.007 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.5550000000000002e-06 2024-09-14T06:32:44.007 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:44.007 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:44.007 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:32:44.007 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.015025+0000", 2024-09-14T06:32:44.008 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.074928579999999995 2024-09-14T06:32:44.008 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:44.008 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:44.008 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:32:44.008 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.015096+0000", 2024-09-14T06:32:44.008 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.0705999999999997e-05 2024-09-14T06:32:44.008 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:44.008 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:44.008 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:32:44.008 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.015217+0000", 2024-09-14T06:32:44.008 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00012131200000000001 2024-09-14T06:32:44.008 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:44.008 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:32:44.008 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:44.009 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:44.009 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:44.009 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:115201 3.1 3:856be886:::rbd_data.112d4d24c558.0000000000000004:29 [read 3489792~16384 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e96)", 2024-09-14T06:32:44.009 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:30:17.940113+0000", 2024-09-14T06:32:44.009 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 146.03043905300001, 2024-09-14T06:32:44.009 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.075246899000000006, 2024-09-14T06:32:44.009 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:32:44.009 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:32:44.009 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:32:44.009 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:32:44.009 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:32:44.009 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:44.009 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 115201 2024-09-14T06:32:44.009 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:44.010 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:32:44.010 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:44.010 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:32:44.010 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940113+0000", 2024-09-14T06:32:44.010 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:44.010 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:44.010 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:44.010 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:32:44.010 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940113+0000", 2024-09-14T06:32:44.010 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:44.010 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:44.010 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:44.010 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:32:44.010 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940114+0000", 2024-09-14T06:32:44.011 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.1500000000000004e-07 2024-09-14T06:32:44.011 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:44.011 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:44.011 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:32:44.011 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940116+0000", 2024-09-14T06:32:44.011 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.2809999999999998e-06 2024-09-14T06:32:44.011 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:44.011 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:44.011 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:32:44.011 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940123+0000", 2024-09-14T06:32:44.011 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.9229999999999996e-06 2024-09-14T06:32:44.011 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:44.011 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:44.011 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:32:44.012 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940128+0000", 2024-09-14T06:32:44.012 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.7790000000000003e-06 2024-09-14T06:32:44.012 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:44.012 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:44.012 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:32:44.012 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.015253+0000", 2024-09-14T06:32:44.012 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.075125792999999996 2024-09-14T06:32:44.012 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:44.012 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:44.012 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:32:44.012 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.015297+0000", 2024-09-14T06:32:44.012 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.3847000000000002e-05 2024-09-14T06:32:44.012 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:44.013 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:44.013 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:32:44.013 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.015360+0000", 2024-09-14T06:32:44.013 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.2661000000000002e-05 2024-09-14T06:32:44.013 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:44.013 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:32:44.013 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:44.013 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:44.013 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:44.013 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:115202 3.1 3:856be886:::rbd_data.112d4d24c558.0000000000000004:29 [read 3735552~16384 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e96)", 2024-09-14T06:32:44.013 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:30:17.940135+0000", 2024-09-14T06:32:44.013 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 146.030417274, 2024-09-14T06:32:44.013 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.075451105000000004, 2024-09-14T06:32:44.013 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:32:44.014 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:32:44.014 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:32:44.014 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:32:44.014 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:32:44.014 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:44.014 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 115202 2024-09-14T06:32:44.014 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:44.014 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:32:44.014 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:44.014 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:32:44.014 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940135+0000", 2024-09-14T06:32:44.014 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:44.014 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:44.014 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:44.015 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:32:44.015 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940135+0000", 2024-09-14T06:32:44.015 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:44.015 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:44.015 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:44.015 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:32:44.015 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940135+0000", 2024-09-14T06:32:44.015 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.0999999999999999e-07 2024-09-14T06:32:44.015 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:44.015 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:44.015 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:32:44.015 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940137+0000", 2024-09-14T06:32:44.015 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.0020000000000001e-06 2024-09-14T06:32:44.015 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:44.015 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:44.016 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:32:44.016 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940143+0000", 2024-09-14T06:32:44.016 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.9850000000000003e-06 2024-09-14T06:32:44.016 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:44.016 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:44.016 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:32:44.016 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940148+0000", 2024-09-14T06:32:44.016 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.9880000000000004e-06 2024-09-14T06:32:44.016 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:44.016 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:44.016 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:32:44.016 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.015400+0000", 2024-09-14T06:32:44.016 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.075252186999999998 2024-09-14T06:32:44.016 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:44.017 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:44.017 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:32:44.017 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.015481+0000", 2024-09-14T06:32:44.017 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.0698000000000006e-05 2024-09-14T06:32:44.017 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:44.017 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:44.017 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:32:44.017 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.015586+0000", 2024-09-14T06:32:44.017 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00010483499999999999 2024-09-14T06:32:44.017 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:44.017 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:32:44.017 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:44.017 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:44.017 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:44.018 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:115203 3.1 3:856be886:::rbd_data.112d4d24c558.0000000000000004:29 [read 3981312~16384 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e96)", 2024-09-14T06:32:44.018 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:30:17.940156+0000", 2024-09-14T06:32:44.018 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 146.03039604899999, 2024-09-14T06:32:44.018 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.075620860999999998, 2024-09-14T06:32:44.018 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:32:44.018 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:32:44.018 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:32:44.018 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:32:44.018 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:32:44.018 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:44.018 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 115203 2024-09-14T06:32:44.018 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:44.018 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:32:44.019 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:44.019 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:32:44.019 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940156+0000", 2024-09-14T06:32:44.019 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:44.019 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:44.019 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:44.019 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:32:44.019 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940156+0000", 2024-09-14T06:32:44.019 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:44.019 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:44.019 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:44.019 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:32:44.019 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940156+0000", 2024-09-14T06:32:44.019 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.1500000000000005e-07 2024-09-14T06:32:44.020 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:44.020 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:44.020 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:32:44.020 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940158+0000", 2024-09-14T06:32:44.020 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.967e-06 2024-09-14T06:32:44.020 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:44.020 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:44.020 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:32:44.020 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940165+0000", 2024-09-14T06:32:44.020 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.6939999999999998e-06 2024-09-14T06:32:44.020 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:44.020 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:44.020 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:32:44.021 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940170+0000", 2024-09-14T06:32:44.021 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.7539999999999997e-06 2024-09-14T06:32:44.021 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:44.021 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:44.021 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:32:44.021 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.015633+0000", 2024-09-14T06:32:44.021 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.075463334000000007 2024-09-14T06:32:44.021 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:44.021 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:44.021 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:32:44.021 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.015693+0000", 2024-09-14T06:32:44.021 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.9367000000000003e-05 2024-09-14T06:32:44.022 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:44.022 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:44.022 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:32:44.022 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.015777+0000", 2024-09-14T06:32:44.022 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.4229999999999998e-05 2024-09-14T06:32:44.022 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:44.022 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:32:44.022 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:44.022 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:44.022 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:44.022 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:115204 3.1 3:856be886:::rbd_data.112d4d24c558.0000000000000004:29 [read 4227072~16384 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e96)", 2024-09-14T06:32:44.022 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:30:17.940176+0000", 2024-09-14T06:32:44.022 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 146.03037551599999, 2024-09-14T06:32:44.023 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.075785505000000003, 2024-09-14T06:32:44.023 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:32:44.023 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:32:44.023 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:32:44.023 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:32:44.023 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:32:44.023 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:44.023 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 115204 2024-09-14T06:32:44.023 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:44.023 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:32:44.023 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:44.023 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:32:44.023 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940176+0000", 2024-09-14T06:32:44.024 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:44.024 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:44.024 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:44.024 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:32:44.024 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940176+0000", 2024-09-14T06:32:44.024 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:44.024 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:44.024 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:44.024 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:32:44.024 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940177+0000", 2024-09-14T06:32:44.024 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.7899999999999999e-07 2024-09-14T06:32:44.024 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:44.024 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:44.024 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:32:44.024 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940179+0000", 2024-09-14T06:32:44.025 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.103e-06 2024-09-14T06:32:44.025 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:44.025 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:44.025 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:32:44.025 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940185+0000", 2024-09-14T06:32:44.025 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.7239999999999997e-06 2024-09-14T06:32:44.025 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:44.025 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:44.025 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:32:44.025 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940190+0000", 2024-09-14T06:32:44.025 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.7269999999999998e-06 2024-09-14T06:32:44.025 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:44.025 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:44.025 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:32:44.026 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.015814+0000", 2024-09-14T06:32:44.026 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.075624114000000006 2024-09-14T06:32:44.026 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:44.026 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:44.026 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:32:44.026 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.015862+0000", 2024-09-14T06:32:44.026 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.8770999999999998e-05 2024-09-14T06:32:44.026 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:44.026 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:44.026 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:32:44.026 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.015962+0000", 2024-09-14T06:32:44.026 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.9586999999999994e-05 2024-09-14T06:32:44.026 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:44.026 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:32:44.026 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:44.026 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:44.027 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:44.027 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:115205 3.1 3:856be886:::rbd_data.112d4d24c558.0000000000000004:29 [read 4472832~16384 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e96)", 2024-09-14T06:32:44.027 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:30:17.940197+0000", 2024-09-14T06:32:44.027 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 146.03035480700001, 2024-09-14T06:32:44.027 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.076068920999999998, 2024-09-14T06:32:44.027 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:32:44.027 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:32:44.027 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:32:44.027 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:32:44.027 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:32:44.027 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:44.027 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 115205 2024-09-14T06:32:44.027 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:44.027 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:32:44.027 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:44.028 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:32:44.028 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940197+0000", 2024-09-14T06:32:44.028 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:44.028 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:44.028 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:44.028 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:32:44.028 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940197+0000", 2024-09-14T06:32:44.028 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:44.028 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:44.028 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:44.028 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:32:44.028 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940198+0000", 2024-09-14T06:32:44.028 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.5999999999999999e-07 2024-09-14T06:32:44.028 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:44.029 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:44.029 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:32:44.029 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940200+0000", 2024-09-14T06:32:44.029 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.0899999999999999e-06 2024-09-14T06:32:44.029 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:44.029 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:44.029 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:32:44.029 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940206+0000", 2024-09-14T06:32:44.029 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.8710000000000002e-06 2024-09-14T06:32:44.029 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:44.029 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:44.029 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:32:44.029 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940211+0000", 2024-09-14T06:32:44.029 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.9230000000000001e-06 2024-09-14T06:32:44.029 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:44.030 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:44.030 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:32:44.030 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.016059+0000", 2024-09-14T06:32:44.030 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.075848051999999999 2024-09-14T06:32:44.030 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:44.030 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:44.030 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:32:44.030 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.016151+0000", 2024-09-14T06:32:44.030 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.1982000000000001e-05 2024-09-14T06:32:44.030 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:44.030 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:44.030 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:32:44.030 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.016266+0000", 2024-09-14T06:32:44.030 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000115543 2024-09-14T06:32:44.030 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:44.031 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:32:44.031 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:44.031 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:44.031 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:44.031 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:115206 3.1 3:856be886:::rbd_data.112d4d24c558.0000000000000004:29 [read 4718592~16384 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e96)", 2024-09-14T06:32:44.031 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:30:17.940223+0000", 2024-09-14T06:32:44.031 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 146.030328717, 2024-09-14T06:32:44.031 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.076265115999999994, 2024-09-14T06:32:44.031 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:32:44.031 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:32:44.031 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:32:44.031 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:32:44.031 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:32:44.031 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:44.031 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 115206 2024-09-14T06:32:44.032 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:44.032 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:32:44.032 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:44.032 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:32:44.032 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940223+0000", 2024-09-14T06:32:44.032 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:44.032 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:44.032 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:44.032 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:32:44.032 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940223+0000", 2024-09-14T06:32:44.032 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:44.032 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:44.032 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:44.032 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:32:44.032 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940224+0000", 2024-09-14T06:32:44.033 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.0399999999999996e-07 2024-09-14T06:32:44.033 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:44.033 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:44.033 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:32:44.033 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940226+0000", 2024-09-14T06:32:44.033 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.1780000000000002e-06 2024-09-14T06:32:44.033 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:44.033 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:44.033 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:32:44.033 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940232+0000", 2024-09-14T06:32:44.033 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.5010000000000001e-06 2024-09-14T06:32:44.033 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:44.033 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:44.033 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:32:44.033 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940237+0000", 2024-09-14T06:32:44.034 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.0250000000000002e-06 2024-09-14T06:32:44.034 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:44.034 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:44.034 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:32:44.034 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.016305+0000", 2024-09-14T06:32:44.034 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.076067262999999996 2024-09-14T06:32:44.034 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:44.034 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:44.034 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:32:44.034 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.016383+0000", 2024-09-14T06:32:44.034 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.8578000000000006e-05 2024-09-14T06:32:44.034 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:44.034 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:44.034 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:32:44.034 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.016488+0000", 2024-09-14T06:32:44.035 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000105067 2024-09-14T06:32:44.035 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:44.035 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:32:44.035 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:44.035 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:44.035 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:44.035 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:115207 3.1 3:856be886:::rbd_data.112d4d24c558.0000000000000004:29 [read 4964352~16384 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e96)", 2024-09-14T06:32:44.035 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:30:17.940244+0000", 2024-09-14T06:32:44.035 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 146.030307601, 2024-09-14T06:32:44.035 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.076371154999999996, 2024-09-14T06:32:44.035 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:32:44.035 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:32:44.035 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:32:44.036 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:32:44.036 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:32:44.036 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:44.036 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 115207 2024-09-14T06:32:44.036 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:44.036 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:32:44.036 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:44.036 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:32:44.036 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940244+0000", 2024-09-14T06:32:44.036 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:44.036 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:44.036 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:44.036 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:32:44.036 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940244+0000", 2024-09-14T06:32:44.036 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:44.037 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:44.037 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:44.037 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:32:44.037 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940245+0000", 2024-09-14T06:32:44.037 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.1200000000000003e-07 2024-09-14T06:32:44.037 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:44.037 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:44.037 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:32:44.037 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940247+0000", 2024-09-14T06:32:44.037 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.0030000000000001e-06 2024-09-14T06:32:44.037 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:44.037 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:44.037 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:32:44.037 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940252+0000", 2024-09-14T06:32:44.038 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.5520000000000004e-06 2024-09-14T06:32:44.038 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:44.038 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:44.038 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:32:44.038 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940257+0000", 2024-09-14T06:32:44.038 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.6759999999999998e-06 2024-09-14T06:32:44.038 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:44.038 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:44.038 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:32:44.038 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.016521+0000", 2024-09-14T06:32:44.038 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.076263499999999998 2024-09-14T06:32:44.038 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:44.038 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:44.038 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:32:44.038 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.016562+0000", 2024-09-14T06:32:44.039 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.0967000000000003e-05 2024-09-14T06:32:44.039 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:44.039 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:44.039 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:32:44.039 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.016616+0000", 2024-09-14T06:32:44.039 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.3944999999999999e-05 2024-09-14T06:32:44.039 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:44.039 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:32:44.039 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:44.039 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:44.039 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:44.039 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:115208 3.1 3:856be886:::rbd_data.112d4d24c558.0000000000000004:29 [read 5210112~16384 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e96)", 2024-09-14T06:32:44.039 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:30:17.940264+0000", 2024-09-14T06:32:44.039 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 146.03028774399999, 2024-09-14T06:32:44.039 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.076558288000000002, 2024-09-14T06:32:44.039 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:32:44.040 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:32:44.040 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:32:44.040 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:32:44.040 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:32:44.040 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:44.040 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 115208 2024-09-14T06:32:44.040 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:44.040 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:32:44.040 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:44.040 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:32:44.040 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940264+0000", 2024-09-14T06:32:44.040 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:44.040 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:44.040 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:44.040 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:32:44.041 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940264+0000", 2024-09-14T06:32:44.041 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:44.041 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:44.041 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:44.041 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:32:44.041 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940265+0000", 2024-09-14T06:32:44.041 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.0500000000000004e-07 2024-09-14T06:32:44.041 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:44.041 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:44.041 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:32:44.041 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940267+0000", 2024-09-14T06:32:44.041 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.9810000000000002e-06 2024-09-14T06:32:44.041 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:44.041 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:44.042 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:32:44.042 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940272+0000", 2024-09-14T06:32:44.042 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.6999999999999996e-06 2024-09-14T06:32:44.042 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:44.042 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:44.042 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:32:44.042 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940277+0000", 2024-09-14T06:32:44.042 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.6480000000000002e-06 2024-09-14T06:32:44.042 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:44.042 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:44.042 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:32:44.042 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.016650+0000", 2024-09-14T06:32:44.042 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.076373089000000005 2024-09-14T06:32:44.042 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:44.042 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:44.043 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:32:44.043 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.016724+0000", 2024-09-14T06:32:44.043 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.4105999999999998e-05 2024-09-14T06:32:44.043 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:44.043 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:44.043 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:32:44.043 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.016823+0000", 2024-09-14T06:32:44.043 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.8258999999999996e-05 2024-09-14T06:32:44.043 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:44.043 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:32:44.043 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:44.043 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:44.043 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:44.043 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:115209 3.1 3:856be886:::rbd_data.112d4d24c558.0000000000000004:29 [read 5455872~16384 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e96)", 2024-09-14T06:32:44.043 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:30:17.940283+0000", 2024-09-14T06:32:44.043 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 146.030268718, 2024-09-14T06:32:44.043 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.076724029999999999, 2024-09-14T06:32:44.044 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:32:44.044 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:32:44.044 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:32:44.044 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:32:44.044 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:32:44.044 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:44.044 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 115209 2024-09-14T06:32:44.044 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:44.044 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:32:44.044 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:44.044 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:32:44.044 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940283+0000", 2024-09-14T06:32:44.044 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:44.044 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:44.044 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:44.045 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:32:44.045 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940283+0000", 2024-09-14T06:32:44.045 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:44.045 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:44.045 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:44.045 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:32:44.045 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940284+0000", 2024-09-14T06:32:44.045 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.6700000000000003e-07 2024-09-14T06:32:44.045 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:44.045 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:44.045 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:32:44.045 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940286+0000", 2024-09-14T06:32:44.045 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.9520000000000001e-06 2024-09-14T06:32:44.045 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:44.045 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:44.046 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:32:44.046 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940292+0000", 2024-09-14T06:32:44.046 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.7339999999999996e-06 2024-09-14T06:32:44.046 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:44.046 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:44.046 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:32:44.046 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940296+0000", 2024-09-14T06:32:44.046 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.7700000000000001e-06 2024-09-14T06:32:44.046 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:44.046 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:44.046 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:32:44.046 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.016859+0000", 2024-09-14T06:32:44.046 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.07656317 2024-09-14T06:32:44.046 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:44.046 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:44.047 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:32:44.047 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.016929+0000", 2024-09-14T06:32:44.047 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.9679999999999997e-05 2024-09-14T06:32:44.047 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:44.047 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:44.047 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:32:44.047 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.017007+0000", 2024-09-14T06:32:44.047 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.8157000000000004e-05 2024-09-14T06:32:44.047 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:44.047 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:32:44.047 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:44.047 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:44.047 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:44.047 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:115210 3.1 3:856be886:::rbd_data.112d4d24c558.0000000000000004:29 [read 5701632~16384 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e96)", 2024-09-14T06:32:44.047 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:30:17.940303+0000", 2024-09-14T06:32:44.047 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 146.030249201, 2024-09-14T06:32:44.048 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.076821358000000006, 2024-09-14T06:32:44.048 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:32:44.048 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:32:44.048 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:32:44.048 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:32:44.048 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:32:44.048 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:44.048 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 115210 2024-09-14T06:32:44.048 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:44.048 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:32:44.048 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:44.048 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:32:44.048 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940303+0000", 2024-09-14T06:32:44.048 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:44.048 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:44.048 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:44.049 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:32:44.049 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940303+0000", 2024-09-14T06:32:44.049 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:44.049 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:44.049 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:44.049 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:32:44.049 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940303+0000", 2024-09-14T06:32:44.049 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.3000000000000001e-07 2024-09-14T06:32:44.049 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:44.049 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:44.049 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:32:44.049 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940306+0000", 2024-09-14T06:32:44.049 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.255e-06 2024-09-14T06:32:44.049 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:44.049 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:44.050 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:32:44.050 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940312+0000", 2024-09-14T06:32:44.050 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.9710000000000001e-06 2024-09-14T06:32:44.050 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:44.050 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:44.050 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:32:44.050 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940316+0000", 2024-09-14T06:32:44.050 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.5340000000000001e-06 2024-09-14T06:32:44.050 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:44.050 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:44.050 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:32:44.050 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.017033+0000", 2024-09-14T06:32:44.050 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.076717169000000002 2024-09-14T06:32:44.050 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:44.050 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:44.051 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:32:44.051 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.017073+0000", 2024-09-14T06:32:44.051 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.9582000000000002e-05 2024-09-14T06:32:44.051 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:44.051 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:44.051 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:32:44.051 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.017124+0000", 2024-09-14T06:32:44.051 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.1316999999999997e-05 2024-09-14T06:32:44.051 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:44.051 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:32:44.051 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:44.051 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:44.051 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:44.051 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:115219 3.1 3:856be886:::rbd_data.112d4d24c558.0000000000000004:29 [read 65536~16384 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e96)", 2024-09-14T06:32:44.051 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:30:17.940323+0000", 2024-09-14T06:32:44.052 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 146.03022934000001, 2024-09-14T06:32:44.052 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.076915205, 2024-09-14T06:32:44.052 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:32:44.052 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:32:44.052 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:32:44.052 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:32:44.052 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:32:44.052 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:44.052 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 115219 2024-09-14T06:32:44.052 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:44.052 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:32:44.052 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:44.052 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:32:44.053 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940323+0000", 2024-09-14T06:32:44.053 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:44.053 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:44.053 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:44.053 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:32:44.053 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940323+0000", 2024-09-14T06:32:44.053 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:44.053 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:44.053 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:44.053 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:32:44.053 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940323+0000", 2024-09-14T06:32:44.053 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.0800000000000005e-07 2024-09-14T06:32:44.053 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:44.054 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:44.054 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:32:44.054 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940328+0000", 2024-09-14T06:32:44.054 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.887e-06 2024-09-14T06:32:44.054 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:44.054 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:44.054 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:32:44.054 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940334+0000", 2024-09-14T06:32:44.054 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.3269999999999997e-06 2024-09-14T06:32:44.054 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:44.054 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:44.054 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:32:44.054 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940339+0000", 2024-09-14T06:32:44.054 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.7960000000000003e-06 2024-09-14T06:32:44.055 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:44.055 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:44.055 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:32:44.055 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.017148+0000", 2024-09-14T06:32:44.055 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.076809125000000006 2024-09-14T06:32:44.055 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:44.055 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:44.055 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:32:44.055 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.017186+0000", 2024-09-14T06:32:44.055 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.7283000000000002e-05 2024-09-14T06:32:44.055 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:44.055 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:44.055 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:32:44.055 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.017238+0000", 2024-09-14T06:32:44.055 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.2278999999999999e-05 2024-09-14T06:32:44.056 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:44.056 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:32:44.056 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:44.056 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:44.056 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:44.056 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:115220 3.1 3:856be886:::rbd_data.112d4d24c558.0000000000000004:29 [read 311296~16384 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e96)", 2024-09-14T06:32:44.056 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:30:17.940346+0000", 2024-09-14T06:32:44.056 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 146.03020621100001, 2024-09-14T06:32:44.056 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.077033293000000003, 2024-09-14T06:32:44.056 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:32:44.056 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:32:44.056 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:32:44.056 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:32:44.056 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:32:44.056 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:44.057 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 115220 2024-09-14T06:32:44.057 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:44.057 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:32:44.057 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:44.057 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:32:44.057 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940346+0000", 2024-09-14T06:32:44.057 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:44.057 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:44.057 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:44.057 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:32:44.057 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940346+0000", 2024-09-14T06:32:44.057 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:44.057 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:44.057 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:44.058 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:32:44.058 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940346+0000", 2024-09-14T06:32:44.058 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.4300000000000003e-07 2024-09-14T06:32:44.058 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:44.058 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:44.058 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:32:44.058 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940348+0000", 2024-09-14T06:32:44.058 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.0030000000000001e-06 2024-09-14T06:32:44.058 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:44.058 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:44.058 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:32:44.058 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.004655+0000", 2024-09-14T06:32:44.058 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.064306323999999998 2024-09-14T06:32:44.058 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:44.058 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:44.059 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:32:44.059 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.004671+0000", 2024-09-14T06:32:44.059 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.6589999999999999e-05 2024-09-14T06:32:44.059 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:44.059 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:44.059 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:32:44.059 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.017262+0000", 2024-09-14T06:32:44.059 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.012590570000000001 2024-09-14T06:32:44.059 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:44.059 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:44.059 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:32:44.059 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.017321+0000", 2024-09-14T06:32:44.059 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.9210999999999999e-05 2024-09-14T06:32:44.059 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:44.060 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:44.060 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:32:44.060 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.017379+0000", 2024-09-14T06:32:44.060 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.8051999999999999e-05 2024-09-14T06:32:44.060 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:44.060 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:32:44.060 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:44.060 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:44.060 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:32:44.060 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-14T06:32:44.061 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-14T06:32:44.062 INFO:tasks.ceph.osd.5.smithi138.stderr:2024-09-14T06:32:44.060+0000 7fe13746f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33618) UID: 0 2024-09-14T06:32:44.162 INFO:tasks.ceph.osd.5.smithi138.stderr:2024-09-14T06:32:44.162+0000 7fe13746f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33618) UID: 0 2024-09-14T06:32:44.165 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-14T06:32:44.165 INFO:teuthology.orchestra.run.smithi094.stdout: "ops": [], 2024-09-14T06:32:44.165 INFO:teuthology.orchestra.run.smithi094.stdout: "num_ops": 0 2024-09-14T06:32:44.165 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-14T06:32:44.174 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-14T06:32:44.263 INFO:tasks.ceph.osd.6.smithi138.stderr:2024-09-14T06:32:44.263+0000 7f8cd8849640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33620) UID: 0 2024-09-14T06:32:44.276 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-14T06:32:44.276 INFO:teuthology.orchestra.run.smithi094.stdout: "ops": [], 2024-09-14T06:32:44.276 INFO:teuthology.orchestra.run.smithi094.stdout: "complaint_time": 30, 2024-09-14T06:32:44.276 INFO:teuthology.orchestra.run.smithi094.stdout: "num_blocked_ops": 0 2024-09-14T06:32:44.276 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-14T06:32:44.284 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-14T06:32:44.364 INFO:tasks.ceph.osd.0.smithi094.stderr:2024-09-14T06:32:44.363+0000 7f07c9333640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33636) UID: 0 2024-09-14T06:32:44.388 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-14T06:32:44.389 INFO:teuthology.orchestra.run.smithi094.stdout: "size": 20, 2024-09-14T06:32:44.389 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 600, 2024-09-14T06:32:44.389 INFO:teuthology.orchestra.run.smithi094.stdout: "ops": [ 2024-09-14T06:32:44.389 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:44.389 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:14919 3.0 3:05c39535:::rbd_data.112d92027590.0000000000000009:f [read 805888~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:32:44.389 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:26.083843+0000", 2024-09-14T06:32:44.389 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 198.30312297, 2024-09-14T06:32:44.389 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.041992356000000002, 2024-09-14T06:32:44.389 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:32:44.389 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:32:44.389 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:32:44.389 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:32:44.389 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:32:44.390 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:44.390 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 14919 2024-09-14T06:32:44.390 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:44.390 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:32:44.390 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:44.390 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:32:44.390 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.083843+0000", 2024-09-14T06:32:44.390 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:44.390 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:44.390 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:44.390 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:32:44.390 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.083843+0000", 2024-09-14T06:32:44.390 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:44.390 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:44.391 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:44.391 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:32:44.391 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.083844+0000", 2024-09-14T06:32:44.391 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.3300000000000001e-07 2024-09-14T06:32:44.391 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:44.391 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:44.391 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:32:44.391 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.083845+0000", 2024-09-14T06:32:44.391 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.4729999999999999e-06 2024-09-14T06:32:44.391 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:44.391 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:44.391 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:32:44.391 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.083850+0000", 2024-09-14T06:32:44.391 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.0130000000000001e-06 2024-09-14T06:32:44.391 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:44.392 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:44.392 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:32:44.392 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.083854+0000", 2024-09-14T06:32:44.392 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.4759999999999998e-06 2024-09-14T06:32:44.392 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:44.392 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:44.392 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:32:44.392 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.125636+0000", 2024-09-14T06:32:44.392 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.041782683000000001 2024-09-14T06:32:44.392 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:44.392 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:44.392 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:32:44.392 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.125680+0000", 2024-09-14T06:32:44.392 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.3614e-05 2024-09-14T06:32:44.392 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:44.393 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:44.393 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:32:44.393 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.125836+0000", 2024-09-14T06:32:44.393 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000155864 2024-09-14T06:32:44.393 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:44.393 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:32:44.393 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:44.393 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:44.393 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:44.393 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:14920 3.0 3:05c39535:::rbd_data.112d92027590.0000000000000009:f [read 816128~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:32:44.393 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:26.083868+0000", 2024-09-14T06:32:44.393 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 198.30309830900001, 2024-09-14T06:32:44.393 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.042166771999999998, 2024-09-14T06:32:44.393 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:32:44.394 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:32:44.394 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:32:44.394 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:32:44.394 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:32:44.394 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:44.394 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 14920 2024-09-14T06:32:44.394 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:44.394 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:32:44.394 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:44.394 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:32:44.394 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.083868+0000", 2024-09-14T06:32:44.394 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:44.394 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:44.394 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:44.394 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:32:44.395 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.083868+0000", 2024-09-14T06:32:44.395 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:44.395 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:44.395 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:44.395 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:32:44.395 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.083869+0000", 2024-09-14T06:32:44.395 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.99e-07 2024-09-14T06:32:44.395 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:44.395 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:44.395 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:32:44.395 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.083870+0000", 2024-09-14T06:32:44.395 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.782e-06 2024-09-14T06:32:44.395 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:44.395 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:44.396 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:32:44.396 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.083876+0000", 2024-09-14T06:32:44.396 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.3809999999999998e-06 2024-09-14T06:32:44.396 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:44.396 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:44.396 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:32:44.396 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.083882+0000", 2024-09-14T06:32:44.396 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.9479999999999996e-06 2024-09-14T06:32:44.396 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:44.396 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:44.396 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:32:44.396 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.125878+0000", 2024-09-14T06:32:44.396 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.041996264999999998 2024-09-14T06:32:44.396 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:44.396 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:44.396 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:32:44.397 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.125929+0000", 2024-09-14T06:32:44.397 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.0498999999999999e-05 2024-09-14T06:32:44.397 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:44.397 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:44.397 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:32:44.397 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.126035+0000", 2024-09-14T06:32:44.397 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000106298 2024-09-14T06:32:44.397 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:44.397 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:32:44.397 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:44.397 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:44.397 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:44.397 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:14921 3.0 3:05c39535:::rbd_data.112d92027590.0000000000000009:f [read 826368~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:32:44.397 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:26.083889+0000", 2024-09-14T06:32:44.397 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 198.30307729899999, 2024-09-14T06:32:44.398 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.042435309999999997, 2024-09-14T06:32:44.398 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:32:44.398 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:32:44.398 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:32:44.398 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:32:44.398 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:32:44.398 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:44.398 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 14921 2024-09-14T06:32:44.398 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:44.398 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:32:44.399 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:44.399 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:32:44.399 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.083889+0000", 2024-09-14T06:32:44.399 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:44.399 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:44.399 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:44.399 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:32:44.399 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.083889+0000", 2024-09-14T06:32:44.399 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:44.399 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:44.399 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:44.399 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:32:44.399 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.086033+0000", 2024-09-14T06:32:44.399 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0021444179999999999 2024-09-14T06:32:44.400 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:44.400 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:44.400 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:32:44.400 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.086038+0000", 2024-09-14T06:32:44.400 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.0969999999999999e-06 2024-09-14T06:32:44.400 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:44.400 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:44.400 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:32:44.400 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.086046+0000", 2024-09-14T06:32:44.400 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.9440000000000005e-06 2024-09-14T06:32:44.400 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:44.400 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:44.400 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:32:44.400 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.086050+0000", 2024-09-14T06:32:44.400 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.69e-06 2024-09-14T06:32:44.401 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:44.401 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:44.401 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:32:44.401 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.126080+0000", 2024-09-14T06:32:44.401 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.040030237000000003 2024-09-14T06:32:44.401 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:44.401 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:44.401 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:32:44.401 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.126163+0000", 2024-09-14T06:32:44.401 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.2071999999999996e-05 2024-09-14T06:32:44.401 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:44.401 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:44.401 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:32:44.401 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.126324+0000", 2024-09-14T06:32:44.401 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00016185200000000001 2024-09-14T06:32:44.402 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:44.402 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:32:44.402 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:44.402 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:44.402 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:44.402 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:55627 3.0 3:06490baf:::rbd_data.112dca2f6ae1.0000000000000030:17 [read 2002944~4096 [fadvise_nocache]] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:32:44.402 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:44.599966+0000", 2024-09-14T06:32:44.402 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 179.78700035099999, 2024-09-14T06:32:44.402 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.041868546999999999, 2024-09-14T06:32:44.402 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:32:44.402 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:32:44.402 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:32:44.402 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:32:44.402 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:32:44.403 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:44.403 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 55627 2024-09-14T06:32:44.403 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:44.403 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:32:44.403 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:44.403 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:32:44.403 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.599966+0000", 2024-09-14T06:32:44.403 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:44.403 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:44.403 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:44.403 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:32:44.403 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.599966+0000", 2024-09-14T06:32:44.403 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:44.403 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:44.403 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:44.404 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:32:44.404 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.599966+0000", 2024-09-14T06:32:44.404 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.0900000000000001e-07 2024-09-14T06:32:44.404 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:44.404 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:44.404 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:32:44.404 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.599967+0000", 2024-09-14T06:32:44.404 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.64e-07 2024-09-14T06:32:44.404 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:44.404 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:44.404 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:32:44.404 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.599970+0000", 2024-09-14T06:32:44.404 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.9670000000000001e-06 2024-09-14T06:32:44.404 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:44.404 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:44.405 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:32:44.405 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.599973+0000", 2024-09-14T06:32:44.405 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.632e-06 2024-09-14T06:32:44.405 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:44.405 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:44.405 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:32:44.405 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.641665+0000", 2024-09-14T06:32:44.405 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.041692547000000003 2024-09-14T06:32:44.405 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:44.405 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:44.405 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:32:44.405 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.641740+0000", 2024-09-14T06:32:44.405 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.4559000000000002e-05 2024-09-14T06:32:44.405 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:44.405 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:44.406 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:32:44.406 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.641835+0000", 2024-09-14T06:32:44.406 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.4668999999999998e-05 2024-09-14T06:32:44.406 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:44.406 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:32:44.406 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:44.406 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:44.406 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:44.406 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:55628 3.0 3:06490baf:::rbd_data.112dca2f6ae1.0000000000000030:17 [read 2027520~4096 [fadvise_nocache]] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:32:44.406 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:44.599976+0000", 2024-09-14T06:32:44.406 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 179.78699056400001, 2024-09-14T06:32:44.406 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.041991621, 2024-09-14T06:32:44.406 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:32:44.406 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:32:44.406 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:32:44.406 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:32:44.407 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:32:44.407 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:44.407 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 55628 2024-09-14T06:32:44.407 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:44.407 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:32:44.407 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:44.407 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:32:44.407 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.599976+0000", 2024-09-14T06:32:44.407 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:44.407 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:44.407 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:44.407 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:32:44.407 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.599976+0000", 2024-09-14T06:32:44.407 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:44.407 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:44.408 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:44.408 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:32:44.408 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.599976+0000", 2024-09-14T06:32:44.408 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.1299999999999999e-07 2024-09-14T06:32:44.408 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:44.408 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:44.408 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:32:44.408 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.599977+0000", 2024-09-14T06:32:44.408 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.7999999999999993e-07 2024-09-14T06:32:44.408 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:44.408 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:44.408 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:32:44.408 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.599980+0000", 2024-09-14T06:32:44.408 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.01e-06 2024-09-14T06:32:44.408 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:44.409 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:44.409 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:32:44.409 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.599983+0000", 2024-09-14T06:32:44.409 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.5409999999999999e-06 2024-09-14T06:32:44.409 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:44.409 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:44.409 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:32:44.409 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.641869+0000", 2024-09-14T06:32:44.409 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.041886235000000001 2024-09-14T06:32:44.409 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:44.409 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:44.409 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:32:44.409 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.641913+0000", 2024-09-14T06:32:44.409 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.4357000000000002e-05 2024-09-14T06:32:44.409 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:44.409 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:44.410 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:32:44.410 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.641967+0000", 2024-09-14T06:32:44.410 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.4284999999999999e-05 2024-09-14T06:32:44.410 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:44.410 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:32:44.410 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:44.410 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:44.410 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:44.410 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:55629 3.0 3:06490baf:::rbd_data.112dca2f6ae1.0000000000000030:17 [read 2052096~4096 [fadvise_nocache]] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:32:44.410 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:44.599985+0000", 2024-09-14T06:32:44.410 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 179.78698085799999, 2024-09-14T06:32:44.410 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.042183375000000002, 2024-09-14T06:32:44.410 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:32:44.410 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:32:44.411 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:32:44.411 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:32:44.411 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:32:44.411 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:44.411 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 55629 2024-09-14T06:32:44.411 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:44.411 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:32:44.411 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:44.411 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:32:44.411 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.599985+0000", 2024-09-14T06:32:44.411 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:44.411 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:44.411 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:44.411 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:32:44.412 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.599985+0000", 2024-09-14T06:32:44.412 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:44.412 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:44.412 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:44.412 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:32:44.412 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.599986+0000", 2024-09-14T06:32:44.412 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.05e-07 2024-09-14T06:32:44.412 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:44.412 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:44.412 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:32:44.412 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.599987+0000", 2024-09-14T06:32:44.412 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.8400000000000002e-07 2024-09-14T06:32:44.412 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:44.412 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:44.412 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:32:44.412 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.599990+0000", 2024-09-14T06:32:44.413 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.5640000000000001e-06 2024-09-14T06:32:44.413 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:44.413 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:44.413 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:32:44.413 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.599993+0000", 2024-09-14T06:32:44.413 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.615e-06 2024-09-14T06:32:44.413 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:44.413 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:44.413 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:32:44.413 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.642003+0000", 2024-09-14T06:32:44.413 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.042010345999999997 2024-09-14T06:32:44.413 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:44.413 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:44.413 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:32:44.413 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.642080+0000", 2024-09-14T06:32:44.413 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.6538999999999993e-05 2024-09-14T06:32:44.414 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:44.414 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:44.414 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:32:44.414 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.642169+0000", 2024-09-14T06:32:44.414 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.9122000000000005e-05 2024-09-14T06:32:44.414 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:44.414 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:32:44.414 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:44.414 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:44.414 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:44.414 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:55630 3.0 3:06490baf:::rbd_data.112dca2f6ae1.0000000000000030:17 [read 2076672~4096 [fadvise_nocache]] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:32:44.414 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:44.599996+0000", 2024-09-14T06:32:44.414 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 179.78697038000001, 2024-09-14T06:32:44.414 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.042299615999999998, 2024-09-14T06:32:44.414 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:32:44.415 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:32:44.415 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:32:44.415 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:32:44.415 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:32:44.415 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:44.415 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 55630 2024-09-14T06:32:44.415 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:44.415 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:32:44.415 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:44.415 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:32:44.415 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.599996+0000", 2024-09-14T06:32:44.415 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:44.415 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:44.415 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:44.416 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:32:44.416 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.599996+0000", 2024-09-14T06:32:44.416 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:44.416 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:44.416 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:44.416 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:32:44.416 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.599996+0000", 2024-09-14T06:32:44.416 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.1500000000000001e-07 2024-09-14T06:32:44.416 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:44.416 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:44.416 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:32:44.416 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.599997+0000", 2024-09-14T06:32:44.416 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.33e-07 2024-09-14T06:32:44.416 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:44.416 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:44.417 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:32:44.417 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600000+0000", 2024-09-14T06:32:44.417 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.0240000000000002e-06 2024-09-14T06:32:44.417 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:44.417 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:44.417 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:32:44.417 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600003+0000", 2024-09-14T06:32:44.417 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.3640000000000002e-06 2024-09-14T06:32:44.417 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:44.417 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:44.417 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:32:44.417 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.642200+0000", 2024-09-14T06:32:44.417 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.042197571000000003 2024-09-14T06:32:44.417 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:44.418 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:44.418 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:32:44.418 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.642243+0000", 2024-09-14T06:32:44.418 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.2765000000000001e-05 2024-09-14T06:32:44.418 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:44.418 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:44.418 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:32:44.418 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.642296+0000", 2024-09-14T06:32:44.418 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.2744000000000002e-05 2024-09-14T06:32:44.418 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:44.418 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:32:44.418 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:44.418 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:44.418 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:44.418 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:55631 3.0 3:06490baf:::rbd_data.112dca2f6ae1.0000000000000030:17 [read 1048576~4096 [fadvise_nocache]] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:32:44.419 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:44.600006+0000", 2024-09-14T06:32:44.419 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 179.786960792, 2024-09-14T06:32:44.419 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.042487905999999999, 2024-09-14T06:32:44.419 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:32:44.419 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:32:44.419 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:32:44.419 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:32:44.419 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:32:44.419 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:44.419 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 55631 2024-09-14T06:32:44.419 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:44.419 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:32:44.419 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:44.419 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:32:44.419 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600006+0000", 2024-09-14T06:32:44.420 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:44.420 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:44.420 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:44.420 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:32:44.420 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600006+0000", 2024-09-14T06:32:44.420 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:44.420 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:44.420 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:44.420 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:32:44.420 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600006+0000", 2024-09-14T06:32:44.420 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.0800000000000001e-07 2024-09-14T06:32:44.420 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:44.420 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:44.421 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:32:44.421 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600007+0000", 2024-09-14T06:32:44.421 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.7199999999999997e-07 2024-09-14T06:32:44.421 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:44.421 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:44.421 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:32:44.421 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600010+0000", 2024-09-14T06:32:44.421 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.0280000000000001e-06 2024-09-14T06:32:44.421 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:44.421 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:44.421 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:32:44.421 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600012+0000", 2024-09-14T06:32:44.421 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.475e-06 2024-09-14T06:32:44.421 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:44.422 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:44.422 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:32:44.422 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.642329+0000", 2024-09-14T06:32:44.422 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.042317157000000001 2024-09-14T06:32:44.422 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:44.422 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:44.422 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:32:44.422 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.642403+0000", 2024-09-14T06:32:44.422 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.3941000000000002e-05 2024-09-14T06:32:44.422 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:44.422 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:44.422 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:32:44.422 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.642493+0000", 2024-09-14T06:32:44.422 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.0124999999999995e-05 2024-09-14T06:32:44.423 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:44.423 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:32:44.423 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:44.423 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:44.423 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:44.423 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:55632 3.0 3:06490baf:::rbd_data.112dca2f6ae1.0000000000000030:17 [read 1073152~4096 [fadvise_nocache]] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:32:44.423 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:44.600015+0000", 2024-09-14T06:32:44.423 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 179.786951039, 2024-09-14T06:32:44.423 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.042604922000000003, 2024-09-14T06:32:44.423 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:32:44.423 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:32:44.423 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:32:44.423 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:32:44.423 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:32:44.423 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:44.423 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 55632 2024-09-14T06:32:44.424 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:44.424 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:32:44.424 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:44.424 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:32:44.424 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600015+0000", 2024-09-14T06:32:44.424 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:44.424 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:44.424 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:44.424 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:32:44.424 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600015+0000", 2024-09-14T06:32:44.424 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:44.424 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:44.424 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:44.424 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:32:44.424 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600016+0000", 2024-09-14T06:32:44.425 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.1e-07 2024-09-14T06:32:44.425 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:44.425 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:44.425 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:32:44.425 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600017+0000", 2024-09-14T06:32:44.425 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.9900000000000009e-07 2024-09-14T06:32:44.425 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:44.425 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:44.425 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:32:44.425 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600020+0000", 2024-09-14T06:32:44.425 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.4170000000000001e-06 2024-09-14T06:32:44.425 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:44.425 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:44.425 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:32:44.425 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600023+0000", 2024-09-14T06:32:44.426 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.588e-06 2024-09-14T06:32:44.426 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:44.426 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:44.426 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:32:44.426 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.642526+0000", 2024-09-14T06:32:44.426 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.042503236999999999 2024-09-14T06:32:44.426 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:44.426 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:44.426 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:32:44.426 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.642568+0000", 2024-09-14T06:32:44.426 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.1925000000000003e-05 2024-09-14T06:32:44.426 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:44.426 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:44.426 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:32:44.427 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.642620+0000", 2024-09-14T06:32:44.427 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.2546000000000001e-05 2024-09-14T06:32:44.427 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:44.427 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:32:44.427 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:44.427 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:44.427 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:44.427 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:55633 3.0 3:06490baf:::rbd_data.112dca2f6ae1.0000000000000030:17 [read 1097728~4096 [fadvise_nocache]] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:32:44.427 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:44.600026+0000", 2024-09-14T06:32:44.427 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 179.78694077, 2024-09-14T06:32:44.427 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.042797474000000002, 2024-09-14T06:32:44.427 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:32:44.427 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:32:44.427 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:32:44.427 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:32:44.428 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:32:44.428 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:44.428 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 55633 2024-09-14T06:32:44.428 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:44.428 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:32:44.428 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:44.428 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:32:44.428 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600026+0000", 2024-09-14T06:32:44.428 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:44.428 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:44.428 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:44.428 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:32:44.428 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600026+0000", 2024-09-14T06:32:44.428 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:44.429 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:44.429 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:44.429 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:32:44.429 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600026+0000", 2024-09-14T06:32:44.429 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.3099999999999999e-07 2024-09-14T06:32:44.429 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:44.429 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:44.429 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:32:44.429 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600029+0000", 2024-09-14T06:32:44.429 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.304e-06 2024-09-14T06:32:44.429 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:44.429 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:44.429 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:32:44.429 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600034+0000", 2024-09-14T06:32:44.430 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.1429999999999997e-06 2024-09-14T06:32:44.430 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:44.430 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:44.430 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:32:44.430 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600038+0000", 2024-09-14T06:32:44.430 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.692e-06 2024-09-14T06:32:44.430 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:44.430 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:44.430 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:32:44.430 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.642654+0000", 2024-09-14T06:32:44.430 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.042615564000000002 2024-09-14T06:32:44.430 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:44.430 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:44.430 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:32:44.430 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.642728+0000", 2024-09-14T06:32:44.431 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.4018999999999997e-05 2024-09-14T06:32:44.431 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:44.431 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:44.431 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:32:44.431 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.642823+0000", 2024-09-14T06:32:44.431 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.5520999999999996e-05 2024-09-14T06:32:44.431 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:44.431 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:32:44.431 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:44.431 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:44.431 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:44.431 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:55634 3.0 3:06490baf:::rbd_data.112dca2f6ae1.0000000000000030:17 [read 1122304~4096 [fadvise_nocache]] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:32:44.431 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:44.600042+0000", 2024-09-14T06:32:44.431 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 179.786924293, 2024-09-14T06:32:44.431 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.042903729000000002, 2024-09-14T06:32:44.432 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:32:44.432 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:32:44.432 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:32:44.432 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:32:44.432 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:32:44.432 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:44.432 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 55634 2024-09-14T06:32:44.432 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:44.432 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:32:44.432 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:44.432 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:32:44.432 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600042+0000", 2024-09-14T06:32:44.432 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:44.432 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:44.432 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:44.433 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:32:44.433 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600042+0000", 2024-09-14T06:32:44.433 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:44.433 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:44.433 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:44.433 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:32:44.433 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600043+0000", 2024-09-14T06:32:44.433 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.7000000000000005e-07 2024-09-14T06:32:44.433 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:44.433 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:44.433 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:32:44.433 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600044+0000", 2024-09-14T06:32:44.433 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.742e-06 2024-09-14T06:32:44.434 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:44.434 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:44.434 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:32:44.434 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600050+0000", 2024-09-14T06:32:44.434 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.0800000000000002e-06 2024-09-14T06:32:44.434 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:44.434 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:44.434 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:32:44.434 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600055+0000", 2024-09-14T06:32:44.434 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.3889999999999998e-06 2024-09-14T06:32:44.434 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:44.434 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:44.434 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:32:44.434 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.642845+0000", 2024-09-14T06:32:44.434 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.042790040000000001 2024-09-14T06:32:44.434 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:44.435 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:44.435 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:32:44.435 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.642888+0000", 2024-09-14T06:32:44.435 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.2790000000000002e-05 2024-09-14T06:32:44.435 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:44.435 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:44.435 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:32:44.435 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.642946+0000", 2024-09-14T06:32:44.435 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.8118000000000002e-05 2024-09-14T06:32:44.435 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:44.435 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:32:44.435 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:44.435 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:44.435 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:44.435 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:55635 3.0 3:06490baf:::rbd_data.112dca2f6ae1.0000000000000030:17 [read 1146880~4096 [fadvise_nocache]] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:32:44.436 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:44.600061+0000", 2024-09-14T06:32:44.436 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 179.786905722, 2024-09-14T06:32:44.436 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.043092695, 2024-09-14T06:32:44.436 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:32:44.436 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:32:44.436 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:32:44.436 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:32:44.436 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:32:44.436 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:44.436 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 55635 2024-09-14T06:32:44.436 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:44.436 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:32:44.436 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:44.436 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:32:44.437 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600061+0000", 2024-09-14T06:32:44.437 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:44.437 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:44.437 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:44.437 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:32:44.437 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600061+0000", 2024-09-14T06:32:44.437 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:44.437 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:44.437 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:44.437 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:32:44.437 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600061+0000", 2024-09-14T06:32:44.437 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.3099999999999999e-07 2024-09-14T06:32:44.437 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:44.438 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:44.438 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:32:44.438 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600063+0000", 2024-09-14T06:32:44.438 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.6220000000000001e-06 2024-09-14T06:32:44.438 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:44.438 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:44.438 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:32:44.438 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600068+0000", 2024-09-14T06:32:44.438 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.4790000000000004e-06 2024-09-14T06:32:44.438 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:44.438 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:44.438 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:32:44.438 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600072+0000", 2024-09-14T06:32:44.438 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.3100000000000002e-06 2024-09-14T06:32:44.438 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:44.439 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:44.439 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:32:44.439 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.642987+0000", 2024-09-14T06:32:44.439 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.042914489 2024-09-14T06:32:44.439 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:44.439 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:44.439 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:32:44.439 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.643062+0000", 2024-09-14T06:32:44.439 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.5011000000000004e-05 2024-09-14T06:32:44.439 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:44.439 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:44.439 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:32:44.439 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.643153+0000", 2024-09-14T06:32:44.439 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.1452999999999993e-05 2024-09-14T06:32:44.440 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:44.440 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:32:44.440 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:44.440 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:44.440 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:44.440 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:55636 3.0 3:06490baf:::rbd_data.112dca2f6ae1.0000000000000030:17 [read 1171456~4096 [fadvise_nocache]] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:32:44.440 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:44.600078+0000", 2024-09-14T06:32:44.440 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 179.78688830999999, 2024-09-14T06:32:44.440 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.043207966, 2024-09-14T06:32:44.440 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:32:44.440 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:32:44.440 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:32:44.440 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:32:44.440 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:32:44.440 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:44.441 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 55636 2024-09-14T06:32:44.441 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:44.441 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:32:44.441 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:44.441 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:32:44.441 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600078+0000", 2024-09-14T06:32:44.441 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:44.441 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:44.441 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:44.441 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:32:44.441 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600078+0000", 2024-09-14T06:32:44.441 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:44.441 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:44.441 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:44.441 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:32:44.442 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600078+0000", 2024-09-14T06:32:44.442 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.34e-07 2024-09-14T06:32:44.442 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:44.442 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:44.442 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:32:44.442 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600080+0000", 2024-09-14T06:32:44.442 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.641e-06 2024-09-14T06:32:44.442 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:44.442 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:44.442 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:32:44.442 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600086+0000", 2024-09-14T06:32:44.442 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.3860000000000004e-06 2024-09-14T06:32:44.442 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:44.442 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:44.443 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:32:44.443 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600091+0000", 2024-09-14T06:32:44.443 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.5970000000000002e-06 2024-09-14T06:32:44.443 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:44.443 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:44.443 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:32:44.443 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.643184+0000", 2024-09-14T06:32:44.443 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.043092893 2024-09-14T06:32:44.443 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:44.443 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:44.443 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:32:44.443 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.643228+0000", 2024-09-14T06:32:44.443 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.4536999999999997e-05 2024-09-14T06:32:44.443 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:44.443 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:44.444 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:32:44.444 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.643286+0000", 2024-09-14T06:32:44.444 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.7578000000000003e-05 2024-09-14T06:32:44.444 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:44.444 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:32:44.444 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:44.444 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:44.444 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:44.444 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:55637 3.0 3:06490baf:::rbd_data.112dca2f6ae1.0000000000000030:17 [read 1196032~4096 [fadvise_nocache]] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:32:44.444 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:44.600097+0000", 2024-09-14T06:32:44.444 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 179.78686974600001, 2024-09-14T06:32:44.444 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.043399124999999997, 2024-09-14T06:32:44.444 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:32:44.444 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:32:44.444 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:32:44.445 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:32:44.445 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:32:44.445 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:44.445 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 55637 2024-09-14T06:32:44.445 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:44.445 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:32:44.445 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:44.445 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:32:44.445 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600097+0000", 2024-09-14T06:32:44.445 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:44.445 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:44.445 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:44.445 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:32:44.446 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600097+0000", 2024-09-14T06:32:44.446 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:44.446 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:44.446 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:44.446 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:32:44.446 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.604372+0000", 2024-09-14T06:32:44.446 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0042751489999999998 2024-09-14T06:32:44.446 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:44.446 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:44.446 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:32:44.446 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.604377+0000", 2024-09-14T06:32:44.446 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.7899999999999999e-06 2024-09-14T06:32:44.446 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:44.446 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:44.446 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:32:44.447 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.604389+0000", 2024-09-14T06:32:44.447 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.2197e-05 2024-09-14T06:32:44.447 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:44.447 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:44.447 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:32:44.447 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.604397+0000", 2024-09-14T06:32:44.447 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.8730000000000007e-06 2024-09-14T06:32:44.447 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:44.447 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:44.447 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:32:44.447 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.643324+0000", 2024-09-14T06:32:44.447 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.038926897000000002 2024-09-14T06:32:44.447 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:44.447 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:44.447 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:32:44.447 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.643402+0000", 2024-09-14T06:32:44.448 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.8063000000000001e-05 2024-09-14T06:32:44.448 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:44.448 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:44.448 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:32:44.448 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.643496+0000", 2024-09-14T06:32:44.448 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.4155999999999998e-05 2024-09-14T06:32:44.448 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:44.448 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:32:44.448 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:44.448 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:44.448 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:44.448 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:55657 3.0 3:06490baf:::rbd_data.112dca2f6ae1.0000000000000030:17 [read 1687552~4096 [fadvise_nocache]] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:32:44.448 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:44.604841+0000", 2024-09-14T06:32:44.448 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 179.782125017, 2024-09-14T06:32:44.449 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.041890087999999999, 2024-09-14T06:32:44.449 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:32:44.449 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:32:44.449 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:32:44.449 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:32:44.449 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:32:44.449 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:44.449 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 55657 2024-09-14T06:32:44.449 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:44.449 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:32:44.449 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:44.449 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:32:44.450 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.604841+0000", 2024-09-14T06:32:44.450 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:44.450 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:44.450 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:44.450 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:32:44.450 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.604841+0000", 2024-09-14T06:32:44.450 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:44.450 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:44.450 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:44.450 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:32:44.450 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.604842+0000", 2024-09-14T06:32:44.450 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.0900000000000001e-07 2024-09-14T06:32:44.450 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:44.451 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:44.451 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:32:44.451 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.604846+0000", 2024-09-14T06:32:44.451 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.5759999999999999e-06 2024-09-14T06:32:44.451 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:44.451 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:44.451 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:32:44.451 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.604850+0000", 2024-09-14T06:32:44.451 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.405e-06 2024-09-14T06:32:44.451 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:44.451 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:44.451 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:32:44.451 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.604852+0000", 2024-09-14T06:32:44.451 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.8590000000000001e-06 2024-09-14T06:32:44.452 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:44.452 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:44.452 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:32:44.452 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.646633+0000", 2024-09-14T06:32:44.452 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.041780772000000001 2024-09-14T06:32:44.452 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:44.452 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:44.452 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:32:44.452 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.646677+0000", 2024-09-14T06:32:44.452 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.4115e-05 2024-09-14T06:32:44.452 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:44.452 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:44.452 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:32:44.452 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.646731+0000", 2024-09-14T06:32:44.452 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.4151999999999999e-05 2024-09-14T06:32:44.452 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:44.453 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:32:44.453 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:44.453 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:44.453 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:44.453 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:55658 3.0 3:06490baf:::rbd_data.112dca2f6ae1.0000000000000030:17 [read 1712128~4096 [fadvise_nocache]] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:32:44.453 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:44.604856+0000", 2024-09-14T06:32:44.453 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 179.78211041700001, 2024-09-14T06:32:44.453 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.041986227000000001, 2024-09-14T06:32:44.453 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:32:44.453 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:32:44.453 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:32:44.453 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:32:44.453 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:32:44.453 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:44.453 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 55658 2024-09-14T06:32:44.454 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:44.454 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:32:44.454 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:44.454 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:32:44.454 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.604856+0000", 2024-09-14T06:32:44.454 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:44.454 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:44.454 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:44.454 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:32:44.454 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.604856+0000", 2024-09-14T06:32:44.454 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:44.454 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:44.454 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:44.454 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:32:44.455 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.604856+0000", 2024-09-14T06:32:44.455 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.23e-07 2024-09-14T06:32:44.455 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:44.455 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:44.455 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:32:44.455 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.604857+0000", 2024-09-14T06:32:44.455 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.9999999999999995e-07 2024-09-14T06:32:44.455 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:44.455 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:44.455 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:32:44.455 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.604861+0000", 2024-09-14T06:32:44.455 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.9890000000000003e-06 2024-09-14T06:32:44.455 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:44.455 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:44.455 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:32:44.456 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.604864+0000", 2024-09-14T06:32:44.456 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.7429999999999998e-06 2024-09-14T06:32:44.456 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:44.456 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:44.456 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:32:44.456 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.646744+0000", 2024-09-14T06:32:44.456 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.041880325000000003 2024-09-14T06:32:44.456 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:44.456 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:44.456 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:32:44.456 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.646784+0000", 2024-09-14T06:32:44.456 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.9604000000000002e-05 2024-09-14T06:32:44.456 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:44.456 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:44.456 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:32:44.456 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.646842+0000", 2024-09-14T06:32:44.457 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.8343e-05 2024-09-14T06:32:44.457 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:44.457 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:32:44.457 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:44.457 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:44.457 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:44.457 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:55659 3.0 3:06490baf:::rbd_data.112dca2f6ae1.0000000000000030:17 [read 1736704~4096 [fadvise_nocache]] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:32:44.457 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:44.604867+0000", 2024-09-14T06:32:44.457 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 179.78209926400001, 2024-09-14T06:32:44.457 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.042097082000000001, 2024-09-14T06:32:44.457 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:32:44.457 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:32:44.457 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:32:44.457 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:32:44.457 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:32:44.458 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:44.458 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 55659 2024-09-14T06:32:44.458 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:44.458 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:32:44.458 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:44.458 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:32:44.458 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.604867+0000", 2024-09-14T06:32:44.458 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:44.458 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:44.458 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:44.458 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:32:44.458 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.604867+0000", 2024-09-14T06:32:44.458 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:44.458 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:44.458 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:44.458 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:32:44.459 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.604867+0000", 2024-09-14T06:32:44.459 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.2000000000000001e-07 2024-09-14T06:32:44.459 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:44.459 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:44.459 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:32:44.459 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.604868+0000", 2024-09-14T06:32:44.459 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.04e-06 2024-09-14T06:32:44.459 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:44.459 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:44.459 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:32:44.459 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.604871+0000", 2024-09-14T06:32:44.459 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.9689999999999999e-06 2024-09-14T06:32:44.459 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:44.459 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:44.459 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:32:44.459 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.604874+0000", 2024-09-14T06:32:44.459 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.5069999999999999e-06 2024-09-14T06:32:44.460 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:44.460 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:44.460 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:32:44.460 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.646874+0000", 2024-09-14T06:32:44.460 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.042000513000000003 2024-09-14T06:32:44.460 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:44.460 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:44.460 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:32:44.460 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.646914+0000", 2024-09-14T06:32:44.460 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.9443999999999997e-05 2024-09-14T06:32:44.460 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:44.460 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:44.460 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:32:44.460 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.646964+0000", 2024-09-14T06:32:44.461 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.0389000000000002e-05 2024-09-14T06:32:44.461 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:44.461 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:32:44.461 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:44.461 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:44.461 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:44.461 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:55660 3.0 3:06490baf:::rbd_data.112dca2f6ae1.0000000000000030:17 [read 1761280~4096 [fadvise_nocache]] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:32:44.461 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:44.604877+0000", 2024-09-14T06:32:44.461 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 179.782089515, 2024-09-14T06:32:44.461 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.042196349000000001, 2024-09-14T06:32:44.461 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:32:44.461 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:32:44.461 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:32:44.461 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:32:44.461 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:32:44.462 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:44.462 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 55660 2024-09-14T06:32:44.462 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:44.462 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:32:44.462 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:44.462 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:32:44.462 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.604877+0000", 2024-09-14T06:32:44.462 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:44.462 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:44.462 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:44.462 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:32:44.462 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.604877+0000", 2024-09-14T06:32:44.462 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:44.462 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:44.462 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:44.462 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:32:44.463 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.605810+0000", 2024-09-14T06:32:44.463 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00093336899999999997 2024-09-14T06:32:44.463 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:44.463 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:44.463 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:32:44.463 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.605815+0000", 2024-09-14T06:32:44.463 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.6140000000000002e-06 2024-09-14T06:32:44.463 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:44.463 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:44.463 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:32:44.463 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.605825+0000", 2024-09-14T06:32:44.463 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.8889999999999997e-06 2024-09-14T06:32:44.463 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:44.463 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:44.463 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:32:44.464 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.605831+0000", 2024-09-14T06:32:44.464 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.9800000000000003e-06 2024-09-14T06:32:44.464 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:44.464 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:44.464 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:32:44.464 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.646989+0000", 2024-09-14T06:32:44.464 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.041157878000000002 2024-09-14T06:32:44.464 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:44.464 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:44.464 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:32:44.464 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.647026+0000", 2024-09-14T06:32:44.464 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.7305000000000003e-05 2024-09-14T06:32:44.464 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:44.464 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:44.464 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:32:44.464 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.647073+0000", 2024-09-14T06:32:44.465 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.7314000000000001e-05 2024-09-14T06:32:44.465 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:44.465 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:32:44.465 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:44.465 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:44.465 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:44.465 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:55666 3.0 3:06490baf:::rbd_data.112dca2f6ae1.0000000000000030:17 [read 1908736~4096 [fadvise_nocache]] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:32:44.465 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:44.605898+0000", 2024-09-14T06:32:44.465 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 179.781068769, 2024-09-14T06:32:44.465 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.041862887000000001, 2024-09-14T06:32:44.465 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:32:44.465 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:32:44.465 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:32:44.465 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:32:44.465 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:32:44.465 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:44.466 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 55666 2024-09-14T06:32:44.466 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:44.466 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:32:44.466 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:44.466 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:32:44.466 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.605898+0000", 2024-09-14T06:32:44.466 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:44.466 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:44.466 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:44.466 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:32:44.466 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.605898+0000", 2024-09-14T06:32:44.466 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:44.466 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:44.466 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:44.466 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:32:44.467 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.611874+0000", 2024-09-14T06:32:44.467 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0059764049999999997 2024-09-14T06:32:44.467 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:44.467 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:44.467 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:32:44.467 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.611879+0000", 2024-09-14T06:32:44.467 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.8749999999999999e-06 2024-09-14T06:32:44.467 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:44.467 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:44.467 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:32:44.467 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.611887+0000", 2024-09-14T06:32:44.467 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.4109999999999996e-06 2024-09-14T06:32:44.467 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:44.467 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:44.467 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:32:44.468 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.611893+0000", 2024-09-14T06:32:44.468 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.2959999999999998e-06 2024-09-14T06:32:44.468 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:44.468 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:44.468 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:32:44.468 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.647674+0000", 2024-09-14T06:32:44.468 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.035781281999999998 2024-09-14T06:32:44.468 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:44.468 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:44.468 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:32:44.468 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.647711+0000", 2024-09-14T06:32:44.468 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.7434000000000001e-05 2024-09-14T06:32:44.468 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:44.468 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:44.468 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:32:44.469 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.647760+0000", 2024-09-14T06:32:44.469 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.9184000000000002e-05 2024-09-14T06:32:44.469 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:44.469 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:32:44.469 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:44.469 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:44.469 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:44.469 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:236341 3.1c 3:3831950c:::rbd_directory:head [call rbd.dir_get_id in=13b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e233)", 2024-09-14T06:32:44.469 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:32:38.788743+0000", 2024-09-14T06:32:44.469 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 5.59822329, 2024-09-14T06:32:44.469 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.99031488700000003, 2024-09-14T06:32:44.469 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:32:44.469 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:32:44.469 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:32:44.469 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:32:44.470 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:32:44.470 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:44.470 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 236341 2024-09-14T06:32:44.470 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:44.470 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:32:44.470 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:44.470 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:32:44.470 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:38.788743+0000", 2024-09-14T06:32:44.470 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:44.470 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:44.470 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:44.470 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:32:44.470 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:38.788743+0000", 2024-09-14T06:32:44.470 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:44.471 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:44.471 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:44.471 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:32:44.471 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:38.788745+0000", 2024-09-14T06:32:44.471 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.623e-06 2024-09-14T06:32:44.471 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:44.471 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:44.471 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:32:44.471 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:38.788753+0000", 2024-09-14T06:32:44.471 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.8420000000000005e-06 2024-09-14T06:32:44.471 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:44.471 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:44.471 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:32:44.471 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:38.788877+0000", 2024-09-14T06:32:44.472 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00012487899999999999 2024-09-14T06:32:44.472 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:44.472 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:44.472 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:32:44.472 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:38.788886+0000", 2024-09-14T06:32:44.472 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.2379999999999997e-06 2024-09-14T06:32:44.472 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:44.472 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:44.472 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:32:44.472 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:38.788947+0000", 2024-09-14T06:32:44.472 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.1721000000000001e-05 2024-09-14T06:32:44.472 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:44.472 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:44.472 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "waiting for peered", 2024-09-14T06:32:44.472 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:38.788955+0000", 2024-09-14T06:32:44.473 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.2579999999999998e-06 2024-09-14T06:32:44.473 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:44.473 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:44.473 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:32:44.473 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:39.778650+0000", 2024-09-14T06:32:44.473 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.989694988 2024-09-14T06:32:44.473 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:44.473 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:44.473 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:32:44.473 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:39.778811+0000", 2024-09-14T06:32:44.473 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000161092 2024-09-14T06:32:44.473 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:44.473 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:44.473 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:32:44.473 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:39.779058+0000", 2024-09-14T06:32:44.473 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00024724600000000001 2024-09-14T06:32:44.474 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:44.474 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:32:44.474 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:44.474 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:44.474 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:32:44.474 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-14T06:32:44.475 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-14T06:32:44.476 INFO:tasks.ceph.osd.0.smithi094.stderr:2024-09-14T06:32:44.474+0000 7f07c9333640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33636) UID: 0 2024-09-14T06:32:44.575 INFO:tasks.ceph.osd.2.smithi094.stderr:2024-09-14T06:32:44.574+0000 7faf3269b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33649) UID: 0 2024-09-14T06:32:44.622 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-14T06:32:44.622 INFO:teuthology.orchestra.run.smithi094.stdout: "ops": [], 2024-09-14T06:32:44.622 INFO:teuthology.orchestra.run.smithi094.stdout: "num_ops": 0 2024-09-14T06:32:44.622 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-14T06:32:44.633 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-14T06:32:44.634 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-14T06:32:44.676 INFO:tasks.ceph.osd.6.smithi138.stderr:2024-09-14T06:32:44.675+0000 7f8cd8849640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33620) UID: 0 2024-09-14T06:32:44.777 INFO:tasks.ceph.osd.1.smithi094.stderr:2024-09-14T06:32:44.776+0000 7f328e47a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33641) UID: 0 2024-09-14T06:32:44.828 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-14T06:32:44.828 INFO:teuthology.orchestra.run.smithi094.stdout: "ops": [], 2024-09-14T06:32:44.828 INFO:teuthology.orchestra.run.smithi094.stdout: "complaint_time": 30, 2024-09-14T06:32:44.828 INFO:teuthology.orchestra.run.smithi094.stdout: "num_blocked_ops": 0 2024-09-14T06:32:44.828 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-14T06:32:44.839 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-14T06:32:44.880 INFO:tasks.ceph.osd.1.smithi094.stderr:2024-09-14T06:32:44.879+0000 7f328e47a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33641) UID: 0 2024-09-14T06:32:44.979 INFO:tasks.ceph.osd.2.smithi094.stderr:2024-09-14T06:32:44.978+0000 7faf3269b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33649) UID: 0 2024-09-14T06:32:44.985 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-14T06:32:44.985 INFO:teuthology.orchestra.run.smithi094.stdout: "size": 20, 2024-09-14T06:32:44.985 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 600, 2024-09-14T06:32:44.985 INFO:teuthology.orchestra.run.smithi094.stdout: "ops": [ 2024-09-14T06:32:44.985 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:44.985 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4397.0:45722 3.9 e64/23 3:91fed098:::rbd_data.112d5a737e72.0000000000000001:head v 64'61, mlcod=62'59)", 2024-09-14T06:32:44.985 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:43.864269+0000", 2024-09-14T06:32:44.985 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 181.11889987800001, 2024-09-14T06:32:44.985 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.003055224, 2024-09-14T06:32:44.985 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:32:44.986 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:32:44.986 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:32:44.986 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:32:44.986 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:44.986 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:32:44.986 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.864269+0000", 2024-09-14T06:32:44.986 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:44.986 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:44.986 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:44.986 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:32:44.986 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.864269+0000", 2024-09-14T06:32:44.986 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:44.986 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:44.986 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:44.986 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:32:44.987 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.864269+0000", 2024-09-14T06:32:44.987 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.41e-07 2024-09-14T06:32:44.987 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:44.987 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:44.987 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:32:44.987 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.865490+0000", 2024-09-14T06:32:44.987 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.001220552 2024-09-14T06:32:44.987 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:44.987 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:44.987 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:32:44.987 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.865506+0000", 2024-09-14T06:32:44.987 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.6263999999999999e-05 2024-09-14T06:32:44.987 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:44.987 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:44.987 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:32:44.988 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.865516+0000", 2024-09-14T06:32:44.988 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.0091999999999999e-05 2024-09-14T06:32:44.988 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:44.988 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:44.988 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:32:44.988 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.865583+0000", 2024-09-14T06:32:44.988 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.6676999999999999e-05 2024-09-14T06:32:44.988 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:44.988 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:44.988 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:32:44.988 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.865610+0000", 2024-09-14T06:32:44.988 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.6529999999999998e-05 2024-09-14T06:32:44.988 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:44.988 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:44.988 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-14T06:32:44.989 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.867281+0000", 2024-09-14T06:32:44.989 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.001671203 2024-09-14T06:32:44.989 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:44.989 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:44.989 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:32:44.989 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.867324+0000", 2024-09-14T06:32:44.989 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.3665000000000003e-05 2024-09-14T06:32:44.989 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:44.989 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:32:44.989 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:44.989 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:44.989 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:44.989 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4397.0:74241 3.0 e73/23 3:06e93fb9:::rbd_data.112d80ca42af.0000000000000000:head v 73'102, mlcod=71'100)", 2024-09-14T06:32:44.990 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:53.876730+0000", 2024-09-14T06:32:44.990 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 171.10643877800001, 2024-09-14T06:32:44.990 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0079470329999999992, 2024-09-14T06:32:44.990 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:32:44.990 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:32:44.990 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:32:44.990 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:32:44.990 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:44.990 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:32:44.990 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:53.876730+0000", 2024-09-14T06:32:44.990 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:44.990 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:44.990 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:44.990 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:32:44.990 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:53.876730+0000", 2024-09-14T06:32:44.990 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:44.991 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:44.991 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:44.991 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:32:44.991 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:53.876731+0000", 2024-09-14T06:32:44.991 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.0500000000000004e-07 2024-09-14T06:32:44.991 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:44.991 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:44.991 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:32:44.991 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:53.879616+0000", 2024-09-14T06:32:44.991 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0028856680000000001 2024-09-14T06:32:44.991 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:44.991 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:44.991 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:32:44.991 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:53.879643+0000", 2024-09-14T06:32:44.991 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.6120999999999999e-05 2024-09-14T06:32:44.992 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:44.992 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:44.992 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:32:44.992 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:53.879654+0000", 2024-09-14T06:32:44.992 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.1586999999999999e-05 2024-09-14T06:32:44.992 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:44.992 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:44.992 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:32:44.992 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:53.879756+0000", 2024-09-14T06:32:44.992 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00010223899999999999 2024-09-14T06:32:44.992 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:44.992 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:44.992 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:32:44.992 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:53.879783+0000", 2024-09-14T06:32:44.992 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.6913000000000001e-05 2024-09-14T06:32:44.993 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:44.993 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:44.993 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-14T06:32:44.993 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:53.884594+0000", 2024-09-14T06:32:44.993 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0048110729999999999 2024-09-14T06:32:44.993 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:44.993 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:44.993 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:32:44.993 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:53.884677+0000", 2024-09-14T06:32:44.993 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.2927000000000001e-05 2024-09-14T06:32:44.993 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:44.993 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:32:44.993 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:44.993 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:44.994 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:44.994 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4397.0:74243 3.9 e73/23 3:94977f81:::rbd_data.112d80ca42af.0000000000000002:head v 73'94, mlcod=71'92)", 2024-09-14T06:32:44.994 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:53.885156+0000", 2024-09-14T06:32:44.994 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 171.09801286000001, 2024-09-14T06:32:44.994 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0096009860000000006, 2024-09-14T06:32:44.994 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:32:44.994 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:32:44.994 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:32:44.994 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:32:44.994 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:44.994 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:32:44.994 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:53.885156+0000", 2024-09-14T06:32:44.994 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:44.994 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:44.994 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:44.994 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:32:44.995 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:53.885156+0000", 2024-09-14T06:32:44.995 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:44.995 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:44.995 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:44.995 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:32:44.995 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:53.885156+0000", 2024-09-14T06:32:44.995 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.23e-07 2024-09-14T06:32:44.995 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:44.995 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:44.995 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:32:44.995 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:53.889694+0000", 2024-09-14T06:32:44.995 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0045378709999999997 2024-09-14T06:32:44.995 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:44.995 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:44.995 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:32:44.995 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:53.889720+0000", 2024-09-14T06:32:44.996 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.5259e-05 2024-09-14T06:32:44.996 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:44.996 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:44.996 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:32:44.996 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:53.889731+0000", 2024-09-14T06:32:44.996 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.1739000000000001e-05 2024-09-14T06:32:44.996 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:44.996 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:44.996 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:32:44.996 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:53.889822+0000", 2024-09-14T06:32:44.996 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.1200999999999996e-05 2024-09-14T06:32:44.996 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:44.996 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:44.997 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:32:44.997 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:53.889850+0000", 2024-09-14T06:32:44.997 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.7084999999999999e-05 2024-09-14T06:32:44.997 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:44.997 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:44.997 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-14T06:32:44.997 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:53.894703+0000", 2024-09-14T06:32:44.997 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0048532690000000003 2024-09-14T06:32:44.997 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:44.997 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:44.997 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:32:44.997 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:53.894757+0000", 2024-09-14T06:32:44.997 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.4339000000000003e-05 2024-09-14T06:32:44.997 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:44.998 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:32:44.998 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:44.998 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:44.998 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:44.998 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4397.0:74249 3.0 e73/23 3:009b0225:::rbd_data.112d80ca42af.0000000000000009:head v 73'103, mlcod=71'101)", 2024-09-14T06:32:44.998 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:53.893219+0000", 2024-09-14T06:32:44.998 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 171.08995038200001, 2024-09-14T06:32:44.998 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0065381980000000003, 2024-09-14T06:32:44.998 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:32:44.998 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:32:44.998 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:32:44.998 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:32:44.998 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:44.998 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:32:44.999 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:53.893219+0000", 2024-09-14T06:32:44.999 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:44.999 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:44.999 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:44.999 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:32:44.999 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:53.893219+0000", 2024-09-14T06:32:44.999 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:44.999 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:44.999 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:44.999 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:32:44.999 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:53.893221+0000", 2024-09-14T06:32:44.999 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.4329999999999998e-06 2024-09-14T06:32:44.999 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:44.999 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:44.999 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:32:45.000 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:53.895968+0000", 2024-09-14T06:32:45.000 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.002746746 2024-09-14T06:32:45.000 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:45.000 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:45.000 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:32:45.000 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:53.895988+0000", 2024-09-14T06:32:45.000 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.0227000000000001e-05 2024-09-14T06:32:45.000 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:45.000 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:45.001 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:32:45.001 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:53.896000+0000", 2024-09-14T06:32:45.001 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.2359e-05 2024-09-14T06:32:45.001 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:45.001 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:45.001 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:32:45.001 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:53.896084+0000", 2024-09-14T06:32:45.001 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.3659999999999995e-05 2024-09-14T06:32:45.001 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:45.001 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:45.001 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:32:45.001 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:53.896109+0000", 2024-09-14T06:32:45.001 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.5259999999999999e-05 2024-09-14T06:32:45.001 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:45.002 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:45.002 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-14T06:32:45.002 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:53.899704+0000", 2024-09-14T06:32:45.002 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0035942209999999999 2024-09-14T06:32:45.002 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:45.002 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:45.002 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:32:45.002 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:53.899757+0000", 2024-09-14T06:32:45.002 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.3291999999999997e-05 2024-09-14T06:32:45.002 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:45.002 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:32:45.002 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:45.002 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:45.002 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:45.002 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4397.0:78604 3.9 e77/23 3:94c62a0e:::rbd_data.112ddfbec471.000000000000002c:head v 77'96, mlcod=73'94)", 2024-09-14T06:32:45.003 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:57.677568+0000", 2024-09-14T06:32:45.003 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 167.30560094200001, 2024-09-14T06:32:45.003 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.002564846, 2024-09-14T06:32:45.003 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:32:45.003 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:32:45.003 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:32:45.003 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:32:45.003 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:45.003 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:32:45.003 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:57.677568+0000", 2024-09-14T06:32:45.003 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:45.003 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:45.003 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:45.003 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:32:45.003 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:57.677568+0000", 2024-09-14T06:32:45.004 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:45.004 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:45.004 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:45.004 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:32:45.004 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:57.677568+0000", 2024-09-14T06:32:45.004 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.8900000000000001e-07 2024-09-14T06:32:45.004 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:45.004 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:45.004 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:32:45.004 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:57.678488+0000", 2024-09-14T06:32:45.004 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00091963899999999998 2024-09-14T06:32:45.005 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:45.005 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:45.005 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:32:45.005 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:57.678500+0000", 2024-09-14T06:32:45.005 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.1902000000000001e-05 2024-09-14T06:32:45.005 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:45.005 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:45.005 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:32:45.005 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:57.678507+0000", 2024-09-14T06:32:45.005 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.7739999999999999e-06 2024-09-14T06:32:45.005 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:45.005 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:45.005 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:32:45.005 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:57.678586+0000", 2024-09-14T06:32:45.005 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.9103000000000005e-05 2024-09-14T06:32:45.006 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:45.006 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:45.006 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:32:45.006 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:57.678614+0000", 2024-09-14T06:32:45.006 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.7912e-05 2024-09-14T06:32:45.006 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:45.006 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:45.006 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-14T06:32:45.006 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:57.680075+0000", 2024-09-14T06:32:45.006 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0014613370000000001 2024-09-14T06:32:45.006 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:45.006 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:45.006 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:32:45.006 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:57.680133+0000", 2024-09-14T06:32:45.007 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.7890000000000003e-05 2024-09-14T06:32:45.007 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:45.007 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:32:45.007 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:45.007 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:45.007 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:45.007 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4397.0:149885 3.9 e122/23 3:951acef3:::rbd_data.112d1bb7b269.0000000000000005:head v 122'176, mlcod=122'172)", 2024-09-14T06:32:45.007 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:30:44.739092+0000", 2024-09-14T06:32:45.007 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 120.244076632, 2024-09-14T06:32:45.007 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0028236659999999998, 2024-09-14T06:32:45.007 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:32:45.007 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:32:45.007 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:32:45.007 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:32:45.007 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:45.008 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:32:45.008 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.739092+0000", 2024-09-14T06:32:45.008 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:45.008 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:45.008 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:45.008 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:32:45.008 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.739092+0000", 2024-09-14T06:32:45.008 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:45.008 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:45.008 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:45.008 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:32:45.008 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.739093+0000", 2024-09-14T06:32:45.008 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.0109999999999999e-06 2024-09-14T06:32:45.008 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:45.008 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:45.009 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:32:45.009 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.740189+0000", 2024-09-14T06:32:45.009 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00109527 2024-09-14T06:32:45.009 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:45.009 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:45.009 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:32:45.009 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.740199+0000", 2024-09-14T06:32:45.009 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.0458999999999999e-05 2024-09-14T06:32:45.009 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:45.009 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:45.009 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:32:45.009 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.740207+0000", 2024-09-14T06:32:45.009 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.4499999999999998e-06 2024-09-14T06:32:45.010 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:45.010 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:45.010 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:32:45.010 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.740295+0000", 2024-09-14T06:32:45.010 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.8740999999999996e-05 2024-09-14T06:32:45.010 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:45.010 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:45.010 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:32:45.010 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.740317+0000", 2024-09-14T06:32:45.010 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.1523e-05 2024-09-14T06:32:45.010 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:45.010 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:45.010 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-14T06:32:45.010 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.741865+0000", 2024-09-14T06:32:45.011 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.001548315 2024-09-14T06:32:45.011 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:45.011 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:45.011 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:32:45.011 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.741916+0000", 2024-09-14T06:32:45.011 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.0896999999999997e-05 2024-09-14T06:32:45.011 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:45.011 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:32:45.011 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:45.011 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:45.011 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:45.011 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4397.0:149933 3.4 e122/23 3:234f5229:::rbd_data.112d1bb7b269.0000000000000009:head v 122'221, mlcod=120'218)", 2024-09-14T06:32:45.011 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:30:44.743849+0000", 2024-09-14T06:32:45.011 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 120.23932021500001, 2024-09-14T06:32:45.011 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0031479519999999999, 2024-09-14T06:32:45.011 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:32:45.012 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:32:45.012 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:32:45.012 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:32:45.012 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:45.012 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:32:45.012 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.743849+0000", 2024-09-14T06:32:45.012 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:45.012 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:45.012 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:45.012 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:32:45.012 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.743849+0000", 2024-09-14T06:32:45.012 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:45.012 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:45.012 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:45.012 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:32:45.013 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.743850+0000", 2024-09-14T06:32:45.013 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.279e-06 2024-09-14T06:32:45.013 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:45.013 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:45.013 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:32:45.013 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.745151+0000", 2024-09-14T06:32:45.013 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.001300495 2024-09-14T06:32:45.013 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:45.013 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:45.013 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:32:45.013 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.745167+0000", 2024-09-14T06:32:45.013 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.6478e-05 2024-09-14T06:32:45.013 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:45.014 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:45.014 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:32:45.014 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.745175+0000", 2024-09-14T06:32:45.014 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.2679999999999994e-06 2024-09-14T06:32:45.014 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:45.014 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:45.014 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:32:45.014 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.745226+0000", 2024-09-14T06:32:45.014 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.0192999999999999e-05 2024-09-14T06:32:45.014 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:45.014 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:45.014 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:32:45.014 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.745247+0000", 2024-09-14T06:32:45.014 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.1767e-05 2024-09-14T06:32:45.014 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:45.015 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:45.015 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-14T06:32:45.015 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.746952+0000", 2024-09-14T06:32:45.015 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0017042769999999999 2024-09-14T06:32:45.015 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:45.015 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:45.015 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:32:45.015 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.746997+0000", 2024-09-14T06:32:45.015 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.5194999999999997e-05 2024-09-14T06:32:45.015 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:45.015 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:32:45.015 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:45.015 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:45.015 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:45.015 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4397.0:150494 3.9 e122/23 3:955152a4:::rbd_object_map.112d1bb7b269.0000000000000038:head v 122'194, mlcod=122'190)", 2024-09-14T06:32:45.016 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:30:44.781585+0000", 2024-09-14T06:32:45.016 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 120.20158432300001, 2024-09-14T06:32:45.016 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0026957499999999998, 2024-09-14T06:32:45.016 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:32:45.016 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:32:45.016 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:32:45.016 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:32:45.016 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:45.016 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:32:45.016 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.781585+0000", 2024-09-14T06:32:45.016 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:45.016 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:45.016 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:45.016 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:32:45.017 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.781585+0000", 2024-09-14T06:32:45.017 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:45.017 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:45.017 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:45.017 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:32:45.017 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.781585+0000", 2024-09-14T06:32:45.017 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.5000000000000002e-07 2024-09-14T06:32:45.017 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:45.017 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:45.017 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:32:45.017 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.781594+0000", 2024-09-14T06:32:45.018 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.4640000000000006e-06 2024-09-14T06:32:45.018 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:45.018 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:45.018 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:32:45.018 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.781604+0000", 2024-09-14T06:32:45.018 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.0071999999999999e-05 2024-09-14T06:32:45.018 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:45.018 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:45.018 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:32:45.018 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.781610+0000", 2024-09-14T06:32:45.018 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.3609999999999997e-06 2024-09-14T06:32:45.018 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:45.018 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:45.018 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:32:45.018 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.781676+0000", 2024-09-14T06:32:45.018 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.5963000000000005e-05 2024-09-14T06:32:45.019 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:45.019 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:45.019 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:32:45.019 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.781696+0000", 2024-09-14T06:32:45.019 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.9278000000000001e-05 2024-09-14T06:32:45.019 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:45.019 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:45.019 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-14T06:32:45.019 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.784231+0000", 2024-09-14T06:32:45.019 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.002535093 2024-09-14T06:32:45.019 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:45.019 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:45.019 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:32:45.019 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.784280+0000", 2024-09-14T06:32:45.019 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.9869e-05 2024-09-14T06:32:45.020 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:45.020 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:32:45.020 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:45.020 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:45.020 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:45.020 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4397.0:204589 3.4 e166/23 3:234552f0:::rbd_data.112d3bd84cb0.000000000000003f:head v 166'499, mlcod=165'497)", 2024-09-14T06:32:45.020 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:31:30.772894+0000", 2024-09-14T06:32:45.020 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 74.210274728000002, 2024-09-14T06:32:45.020 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0034564650000000001, 2024-09-14T06:32:45.020 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:32:45.020 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:32:45.020 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:32:45.020 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:32:45.020 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:45.020 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:32:45.021 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:30.772894+0000", 2024-09-14T06:32:45.021 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:45.021 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:45.021 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:45.021 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:32:45.021 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:30.772894+0000", 2024-09-14T06:32:45.021 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:45.021 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:45.021 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:45.021 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:32:45.021 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:30.772896+0000", 2024-09-14T06:32:45.021 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.5379999999999999e-06 2024-09-14T06:32:45.022 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:45.022 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:45.022 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:32:45.022 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:30.773042+0000", 2024-09-14T06:32:45.022 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000146107 2024-09-14T06:32:45.022 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:45.022 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:45.022 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:32:45.022 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:30.773061+0000", 2024-09-14T06:32:45.022 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.8862999999999999e-05 2024-09-14T06:32:45.022 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:45.022 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:45.022 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:32:45.022 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:30.773071+0000", 2024-09-14T06:32:45.022 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.0227000000000001e-05 2024-09-14T06:32:45.022 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:45.023 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:45.023 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:32:45.023 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:30.773149+0000", 2024-09-14T06:32:45.023 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.8071999999999994e-05 2024-09-14T06:32:45.023 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:45.023 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:45.023 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:32:45.023 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:30.773171+0000", 2024-09-14T06:32:45.023 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.1699999999999999e-05 2024-09-14T06:32:45.023 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:45.023 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:45.023 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-14T06:32:45.023 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:30.776285+0000", 2024-09-14T06:32:45.023 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0031142050000000001 2024-09-14T06:32:45.023 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:45.023 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:45.024 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:32:45.024 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:30.776351+0000", 2024-09-14T06:32:45.024 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.5753000000000006e-05 2024-09-14T06:32:45.024 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:45.024 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:32:45.024 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:45.024 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:45.024 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:45.024 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4397.0:204598 3.4 e166/23 3:245d069e:::rbd_data.112d3bd84cb0.0000000000000041:head v 166'500, mlcod=165'497)", 2024-09-14T06:32:45.024 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:31:30.775247+0000", 2024-09-14T06:32:45.024 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 74.207921604999996, 2024-09-14T06:32:45.024 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00318235, 2024-09-14T06:32:45.024 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:32:45.024 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:32:45.025 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:32:45.025 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:32:45.025 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:45.025 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:32:45.025 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:30.775247+0000", 2024-09-14T06:32:45.025 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:45.025 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:45.025 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:45.025 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:32:45.025 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:30.775247+0000", 2024-09-14T06:32:45.025 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:45.025 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:45.025 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:45.025 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:32:45.025 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:30.775249+0000", 2024-09-14T06:32:45.026 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.265e-06 2024-09-14T06:32:45.026 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:45.026 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:45.026 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:32:45.026 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:30.775340+0000", 2024-09-14T06:32:45.026 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.1131999999999994e-05 2024-09-14T06:32:45.026 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:45.026 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:45.026 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:32:45.026 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:30.775352+0000", 2024-09-14T06:32:45.026 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.2294e-05 2024-09-14T06:32:45.026 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:45.026 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:45.026 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:32:45.027 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:30.775361+0000", 2024-09-14T06:32:45.027 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.6950000000000006e-06 2024-09-14T06:32:45.027 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:45.027 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:45.027 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:32:45.027 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:30.775439+0000", 2024-09-14T06:32:45.027 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.7910000000000005e-05 2024-09-14T06:32:45.027 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:45.027 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:45.027 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:32:45.027 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:30.775460+0000", 2024-09-14T06:32:45.027 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.0900999999999999e-05 2024-09-14T06:32:45.027 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:45.027 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:45.027 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-14T06:32:45.027 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:30.778385+0000", 2024-09-14T06:32:45.028 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.002925539 2024-09-14T06:32:45.028 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:45.028 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:45.028 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:32:45.028 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:30.778430+0000", 2024-09-14T06:32:45.028 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.4613999999999997e-05 2024-09-14T06:32:45.028 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:45.028 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:32:45.028 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:45.028 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:45.028 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:45.028 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "MOSDPGPush(3.5 169/168 [PushOp(3:a7a54476:::rbd_data.112d1bb7b269.0000000000000038:head, version: 122'581, data_included: [0~524288], data_size: 524288, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(3:a7a54476:::rbd_data.112d1bb7b269.0000000000000038:head@122'581, size: 524288, copy_subset: [(0, 524288)], clone_subset: {}, snapset: 0=[]:{}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 524288, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2024-09-14T06:32:45.028 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:31:34.667757+0000", 2024-09-14T06:32:45.028 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 70.315412183000007, 2024-09-14T06:32:45.028 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.002750382, 2024-09-14T06:32:45.029 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:32:45.029 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:32:45.029 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:32:45.029 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:32:45.029 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:45.029 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:32:45.029 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:34.667757+0000", 2024-09-14T06:32:45.029 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:45.029 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:45.029 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:45.029 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:32:45.029 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:34.667757+0000", 2024-09-14T06:32:45.029 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:45.029 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:45.029 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:45.030 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:32:45.030 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:34.667758+0000", 2024-09-14T06:32:45.030 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.3599999999999999e-06 2024-09-14T06:32:45.030 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:45.030 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:45.030 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:32:45.030 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:34.669375+0000", 2024-09-14T06:32:45.030 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.001616644 2024-09-14T06:32:45.030 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:45.030 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:45.030 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:32:45.030 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:34.669404+0000", 2024-09-14T06:32:45.030 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.9300999999999999e-05 2024-09-14T06:32:45.030 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:45.031 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:45.031 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:32:45.031 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:34.669428+0000", 2024-09-14T06:32:45.031 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.3788e-05 2024-09-14T06:32:45.031 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:45.031 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:45.031 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:32:45.031 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:34.669543+0000", 2024-09-14T06:32:45.031 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000115462 2024-09-14T06:32:45.031 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:45.031 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:45.031 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:32:45.031 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:34.669559+0000", 2024-09-14T06:32:45.032 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.5167999999999999e-05 2024-09-14T06:32:45.032 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:45.032 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:45.032 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:32:45.032 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:34.670507+0000", 2024-09-14T06:32:45.032 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00094865900000000003 2024-09-14T06:32:45.032 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:45.032 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:32:45.032 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:45.032 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:45.032 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:45.032 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "MOSDPGPush(3.5 170/168 [PushOp(3:a162138c:::rbd_data.112d1bb7b269.000000000000003c:head, version: 122'582, data_included: [0~524288], data_size: 524288, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(3:a162138c:::rbd_data.112d1bb7b269.000000000000003c:head@122'582, size: 524288, copy_subset: [(0, 524288)], clone_subset: {}, snapset: 0=[]:{}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 524288, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2024-09-14T06:32:45.032 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:31:34.670110+0000", 2024-09-14T06:32:45.033 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 70.313059233999994, 2024-09-14T06:32:45.033 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0030066229999999999, 2024-09-14T06:32:45.033 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:32:45.033 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:32:45.033 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:32:45.033 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:32:45.033 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:45.033 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:32:45.033 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:34.670110+0000", 2024-09-14T06:32:45.033 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:45.033 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:45.033 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:45.033 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:32:45.033 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:34.670110+0000", 2024-09-14T06:32:45.034 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:45.034 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:45.034 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:45.034 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:32:45.034 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:34.670111+0000", 2024-09-14T06:32:45.034 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.6499999999999998e-07 2024-09-14T06:32:45.034 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:45.034 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:45.034 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:32:45.034 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:34.671507+0000", 2024-09-14T06:32:45.034 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.001396284 2024-09-14T06:32:45.034 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:45.034 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:45.035 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:32:45.035 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:34.671530+0000", 2024-09-14T06:32:45.035 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.3023e-05 2024-09-14T06:32:45.035 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:45.035 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:45.035 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:32:45.035 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:34.671548+0000", 2024-09-14T06:32:45.035 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.8116000000000001e-05 2024-09-14T06:32:45.035 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:45.035 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:45.035 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:32:45.035 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:34.671775+0000", 2024-09-14T06:32:45.035 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00022699600000000001 2024-09-14T06:32:45.036 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:45.036 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:45.036 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:32:45.036 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:34.671790+0000", 2024-09-14T06:32:45.036 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.539e-05 2024-09-14T06:32:45.036 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:45.036 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:45.036 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:32:45.036 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:34.673116+0000", 2024-09-14T06:32:45.036 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0013259490000000001 2024-09-14T06:32:45.036 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:45.036 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:32:45.036 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:45.036 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:45.036 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:45.037 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:236487 3.14 3:2d502bc2:::rbd_data.112d97471187.000000000000008e:head [delete] snapc 2f=[] ondisk+write+known_if_redirected+supports_pool_eio e234)", 2024-09-14T06:32:45.037 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:32:39.870309+0000", 2024-09-14T06:32:45.037 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 5.1128597060000001, 2024-09-14T06:32:45.037 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0042368170000000004, 2024-09-14T06:32:45.037 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:32:45.037 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:32:45.037 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:32:45.037 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:32:45.037 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:32:45.037 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:45.037 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 236487 2024-09-14T06:32:45.037 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:45.038 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:32:45.038 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:45.038 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:32:45.038 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:39.870309+0000", 2024-09-14T06:32:45.038 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:45.038 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:45.038 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:45.038 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:32:45.038 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:39.870309+0000", 2024-09-14T06:32:45.038 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:45.038 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:45.038 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:45.038 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:32:45.038 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:39.870311+0000", 2024-09-14T06:32:45.038 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.0080000000000001e-06 2024-09-14T06:32:45.039 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:45.039 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:45.039 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:32:45.039 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:39.870316+0000", 2024-09-14T06:32:45.039 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.7160000000000002e-06 2024-09-14T06:32:45.039 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:45.039 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:45.039 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:32:45.039 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:39.870327+0000", 2024-09-14T06:32:45.039 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.0995999999999999e-05 2024-09-14T06:32:45.039 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:45.039 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:45.040 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:32:45.040 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:39.870335+0000", 2024-09-14T06:32:45.040 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.2139999999999996e-06 2024-09-14T06:32:45.040 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:45.040 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:45.040 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:32:45.040 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:39.870403+0000", 2024-09-14T06:32:45.040 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.7966999999999995e-05 2024-09-14T06:32:45.040 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:45.040 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:45.040 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:32:45.040 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:39.870543+0000", 2024-09-14T06:32:45.040 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00014018799999999999 2024-09-14T06:32:45.040 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:45.041 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:45.041 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "waiting for subops from 7", 2024-09-14T06:32:45.041 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:39.870669+0000", 2024-09-14T06:32:45.041 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000125778 2024-09-14T06:32:45.041 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:45.041 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:45.041 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "op_commit", 2024-09-14T06:32:45.041 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:39.871410+0000", 2024-09-14T06:32:45.041 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00074113599999999999 2024-09-14T06:32:45.041 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:45.041 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:45.041 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:32:45.041 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:39.874439+0000", 2024-09-14T06:32:45.042 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0030287259999999998 2024-09-14T06:32:45.042 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:45.042 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:45.042 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-14T06:32:45.042 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:39.874495+0000", 2024-09-14T06:32:45.042 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.5855e-05 2024-09-14T06:32:45.042 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:45.042 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:45.042 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:32:45.042 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:39.874546+0000", 2024-09-14T06:32:45.042 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.1233000000000002e-05 2024-09-14T06:32:45.042 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:45.042 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:32:45.042 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:45.042 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:45.043 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:45.043 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:236490 3.14 3:2e5aef87:::rbd_data.112d97471187.00000000000000ca:head [delete] snapc 2f=[] ondisk+write+known_if_redirected+supports_pool_eio e234)", 2024-09-14T06:32:45.043 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:32:39.872130+0000", 2024-09-14T06:32:45.043 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 5.111039055, 2024-09-14T06:32:45.043 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0048885589999999998, 2024-09-14T06:32:45.043 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:32:45.043 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:32:45.043 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:32:45.043 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:32:45.043 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:32:45.043 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:45.043 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 236490 2024-09-14T06:32:45.043 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:45.044 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:32:45.044 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:45.044 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:32:45.044 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:39.872130+0000", 2024-09-14T06:32:45.044 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:45.044 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:45.044 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:45.044 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:32:45.044 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:39.872130+0000", 2024-09-14T06:32:45.044 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:45.044 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:45.044 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:45.044 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:32:45.044 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:39.872131+0000", 2024-09-14T06:32:45.045 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.0720000000000001e-06 2024-09-14T06:32:45.045 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:45.045 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:45.045 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:32:45.045 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:39.872136+0000", 2024-09-14T06:32:45.045 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.8790000000000002e-06 2024-09-14T06:32:45.045 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:45.045 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:45.045 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:32:45.045 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:39.872150+0000", 2024-09-14T06:32:45.045 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.435e-05 2024-09-14T06:32:45.045 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:45.045 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:45.045 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:32:45.045 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:39.872158+0000", 2024-09-14T06:32:45.045 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.571e-06 2024-09-14T06:32:45.046 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:45.046 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:45.046 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:32:45.046 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:39.872234+0000", 2024-09-14T06:32:45.046 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.606e-05 2024-09-14T06:32:45.046 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:45.046 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:45.046 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:32:45.046 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:39.872356+0000", 2024-09-14T06:32:45.046 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00012250499999999999 2024-09-14T06:32:45.046 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:45.046 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:45.046 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "waiting for subops from 7", 2024-09-14T06:32:45.046 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:39.872480+0000", 2024-09-14T06:32:45.046 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00012316700000000001 2024-09-14T06:32:45.046 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:45.047 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:45.047 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "op_commit", 2024-09-14T06:32:45.047 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:39.873051+0000", 2024-09-14T06:32:45.047 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00057178100000000005 2024-09-14T06:32:45.047 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:45.047 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:45.047 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:32:45.047 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:39.876925+0000", 2024-09-14T06:32:45.047 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0038739239999999999 2024-09-14T06:32:45.047 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:45.047 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:45.047 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-14T06:32:45.047 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:39.876974+0000", 2024-09-14T06:32:45.047 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.8207000000000001e-05 2024-09-14T06:32:45.047 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:45.048 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:45.048 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:32:45.048 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:39.877019+0000", 2024-09-14T06:32:45.048 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.5043000000000002e-05 2024-09-14T06:32:45.048 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:45.048 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:32:45.048 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:45.048 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:45.048 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:45.048 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:237283 3.14 3:2f6164a9:::rbd_data.112d401cec05.000000000000000a:head [delete] snapc 25=[] ondisk+write+known_if_redirected+supports_pool_eio e239)", 2024-09-14T06:32:45.048 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:32:44.693535+0000", 2024-09-14T06:32:45.048 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 0.28963400099999997, 2024-09-14T06:32:45.048 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0070517319999999998, 2024-09-14T06:32:45.048 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:32:45.048 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:32:45.048 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:32:45.049 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:32:45.049 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:32:45.049 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:45.049 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 237283 2024-09-14T06:32:45.049 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:45.049 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:32:45.049 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:45.049 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:32:45.049 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:44.693535+0000", 2024-09-14T06:32:45.049 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:45.049 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:45.049 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:45.049 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:32:45.049 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:44.693535+0000", 2024-09-14T06:32:45.049 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:45.049 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:45.050 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:45.050 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:32:45.050 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:44.693537+0000", 2024-09-14T06:32:45.050 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.4649999999999999e-06 2024-09-14T06:32:45.050 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:45.050 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:45.050 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:32:45.050 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:44.693541+0000", 2024-09-14T06:32:45.050 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.4950000000000002e-06 2024-09-14T06:32:45.050 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:45.050 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:45.050 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:32:45.050 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:44.693553+0000", 2024-09-14T06:32:45.050 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.2238e-05 2024-09-14T06:32:45.051 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:45.051 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:45.051 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:32:45.051 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:44.693561+0000", 2024-09-14T06:32:45.051 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.4379999999999998e-06 2024-09-14T06:32:45.051 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:45.051 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:45.051 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:32:45.051 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:44.693633+0000", 2024-09-14T06:32:45.051 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.2234e-05 2024-09-14T06:32:45.051 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:45.051 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:45.051 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "op must wait for map", 2024-09-14T06:32:45.051 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:44.693642+0000", 2024-09-14T06:32:45.051 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.6770000000000001e-06 2024-09-14T06:32:45.052 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:45.052 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:45.052 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:32:45.052 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:44.695664+0000", 2024-09-14T06:32:45.052 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.002022252 2024-09-14T06:32:45.052 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:45.052 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:45.052 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:32:45.052 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:44.695825+0000", 2024-09-14T06:32:45.052 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00016144099999999999 2024-09-14T06:32:45.052 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:45.052 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:45.052 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "waiting for subops from 7", 2024-09-14T06:32:45.052 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:44.695947+0000", 2024-09-14T06:32:45.053 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000121674 2024-09-14T06:32:45.053 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:45.053 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:45.053 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "op_commit", 2024-09-14T06:32:45.053 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:44.696853+0000", 2024-09-14T06:32:45.053 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00090605000000000004 2024-09-14T06:32:45.053 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:45.053 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:45.053 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:32:45.053 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:44.700467+0000", 2024-09-14T06:32:45.053 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0036137909999999999 2024-09-14T06:32:45.053 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:45.053 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:45.054 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-14T06:32:45.054 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:44.700533+0000", 2024-09-14T06:32:45.054 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.6583999999999998e-05 2024-09-14T06:32:45.054 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:45.054 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:45.054 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:32:45.054 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:44.700587+0000", 2024-09-14T06:32:45.054 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.3393000000000002e-05 2024-09-14T06:32:45.054 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:45.054 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:32:45.054 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:45.054 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:45.054 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:45.054 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4397.0:237335 3.4 e239/211 3:24ee60df:::rbd_data.112d401cec05.000000000000003e:head v 239'657, mlcod=239'655)", 2024-09-14T06:32:45.054 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:32:44.712143+0000", 2024-09-14T06:32:45.055 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 0.27102631300000002, 2024-09-14T06:32:45.055 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0025601270000000001, 2024-09-14T06:32:45.055 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:32:45.055 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:32:45.055 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:32:45.055 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:32:45.055 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:45.055 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:32:45.055 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:44.712143+0000", 2024-09-14T06:32:45.055 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:45.055 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:45.055 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:45.055 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:32:45.055 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:44.712143+0000", 2024-09-14T06:32:45.056 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:45.056 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:45.056 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:45.056 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:32:45.056 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:44.712144+0000", 2024-09-14T06:32:45.056 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.3790000000000001e-06 2024-09-14T06:32:45.056 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:45.056 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:45.056 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:32:45.056 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:44.712153+0000", 2024-09-14T06:32:45.056 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.9250000000000001e-06 2024-09-14T06:32:45.056 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:45.056 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:45.057 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:32:45.057 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:44.712171+0000", 2024-09-14T06:32:45.057 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.7754999999999999e-05 2024-09-14T06:32:45.057 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:45.057 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:45.057 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:32:45.057 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:44.712181+0000", 2024-09-14T06:32:45.057 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.0030000000000001e-05 2024-09-14T06:32:45.057 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:45.057 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:45.057 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:32:45.057 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:44.712262+0000", 2024-09-14T06:32:45.057 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.1197000000000002e-05 2024-09-14T06:32:45.057 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:45.057 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:45.058 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:32:45.058 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:44.712288+0000", 2024-09-14T06:32:45.058 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.6089999999999999e-05 2024-09-14T06:32:45.058 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:45.058 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:45.058 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-14T06:32:45.058 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:44.714657+0000", 2024-09-14T06:32:45.058 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.002368794 2024-09-14T06:32:45.058 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:45.058 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:45.059 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:32:45.059 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:44.714703+0000", 2024-09-14T06:32:45.059 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.5957000000000001e-05 2024-09-14T06:32:45.059 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:45.059 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:32:45.059 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:45.059 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:45.059 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:45.059 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:237364 3.14 3:2a8d6252:::rbd_data.112d401cec05.000000000000005a:head [delete] snapc 25=[] ondisk+write+known_if_redirected+supports_pool_eio e239)", 2024-09-14T06:32:45.059 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:32:44.724057+0000", 2024-09-14T06:32:45.059 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 0.259111859, 2024-09-14T06:32:45.059 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.005405923, 2024-09-14T06:32:45.059 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:32:45.060 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:32:45.060 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:32:45.060 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:32:45.060 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:32:45.060 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:45.060 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 237364 2024-09-14T06:32:45.060 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:45.060 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:32:45.060 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:45.060 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:32:45.060 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:44.724057+0000", 2024-09-14T06:32:45.060 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:45.060 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:45.060 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:45.061 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:32:45.061 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:44.724057+0000", 2024-09-14T06:32:45.061 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:45.061 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:45.061 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:45.061 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:32:45.061 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:44.724059+0000", 2024-09-14T06:32:45.061 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.854e-06 2024-09-14T06:32:45.061 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:45.061 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:45.061 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:32:45.061 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:44.724066+0000", 2024-09-14T06:32:45.061 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.4410000000000004e-06 2024-09-14T06:32:45.061 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:45.062 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:45.062 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:32:45.062 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:44.724086+0000", 2024-09-14T06:32:45.062 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.9128e-05 2024-09-14T06:32:45.062 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:45.062 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:45.062 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:32:45.062 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:44.724095+0000", 2024-09-14T06:32:45.062 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.4900000000000006e-06 2024-09-14T06:32:45.062 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:45.062 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:45.062 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:32:45.062 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:44.724172+0000", 2024-09-14T06:32:45.062 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.7226999999999998e-05 2024-09-14T06:32:45.062 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:45.062 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:45.063 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:32:45.063 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:44.724334+0000", 2024-09-14T06:32:45.063 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000161545 2024-09-14T06:32:45.063 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:45.063 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:45.063 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "waiting for subops from 7", 2024-09-14T06:32:45.063 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:44.724472+0000", 2024-09-14T06:32:45.063 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00013770300000000001 2024-09-14T06:32:45.063 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:45.063 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:45.063 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "op_commit", 2024-09-14T06:32:45.063 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:44.725367+0000", 2024-09-14T06:32:45.063 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00089564699999999996 2024-09-14T06:32:45.063 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:45.064 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:45.064 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:32:45.064 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:44.729349+0000", 2024-09-14T06:32:45.064 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0039818539999999999 2024-09-14T06:32:45.064 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:45.064 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:45.064 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-14T06:32:45.064 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:44.729410+0000", 2024-09-14T06:32:45.064 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.0523000000000003e-05 2024-09-14T06:32:45.064 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:45.064 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:45.064 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:32:45.064 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:44.729463+0000", 2024-09-14T06:32:45.064 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.3511000000000003e-05 2024-09-14T06:32:45.064 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:45.065 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:32:45.065 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:45.065 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:45.065 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:45.065 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:237397 3.14 3:2c456f9d:::rbd_data.112d401cec05.000000000000007b:head [delete] snapc 25=[] ondisk+write+known_if_redirected+supports_pool_eio e239)", 2024-09-14T06:32:45.065 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:32:44.737527+0000", 2024-09-14T06:32:45.065 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 0.24564254099999999, 2024-09-14T06:32:45.065 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0025555869999999998, 2024-09-14T06:32:45.065 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:32:45.065 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:32:45.065 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:32:45.065 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:32:45.065 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:32:45.065 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:45.065 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 237397 2024-09-14T06:32:45.065 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:45.066 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:32:45.066 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:45.066 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:32:45.066 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:44.737527+0000", 2024-09-14T06:32:45.066 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:45.066 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:45.066 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:45.066 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:32:45.066 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:44.737527+0000", 2024-09-14T06:32:45.066 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:45.066 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:45.066 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:45.066 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:32:45.066 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:44.737528+0000", 2024-09-14T06:32:45.066 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.3689999999999999e-06 2024-09-14T06:32:45.067 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:45.067 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:45.067 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:32:45.067 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:44.737533+0000", 2024-09-14T06:32:45.067 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.1839999999999998e-06 2024-09-14T06:32:45.067 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:45.067 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:45.067 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:32:45.067 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:44.737546+0000", 2024-09-14T06:32:45.067 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.306e-05 2024-09-14T06:32:45.067 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:45.067 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:45.067 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:32:45.067 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:44.737554+0000", 2024-09-14T06:32:45.067 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.8480000000000001e-06 2024-09-14T06:32:45.067 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:45.068 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:45.068 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:32:45.068 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:44.737617+0000", 2024-09-14T06:32:45.068 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.2718999999999994e-05 2024-09-14T06:32:45.068 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:45.068 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:45.068 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:32:45.068 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:44.737766+0000", 2024-09-14T06:32:45.068 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000149748 2024-09-14T06:32:45.068 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:45.068 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:45.068 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "waiting for subops from 7", 2024-09-14T06:32:45.068 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:44.737950+0000", 2024-09-14T06:32:45.069 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00018370800000000001 2024-09-14T06:32:45.069 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:45.069 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:45.069 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "op_commit", 2024-09-14T06:32:45.069 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:44.738844+0000", 2024-09-14T06:32:45.069 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00089411099999999995 2024-09-14T06:32:45.069 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:45.069 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:45.069 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:32:45.069 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:44.739967+0000", 2024-09-14T06:32:45.069 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0011227520000000001 2024-09-14T06:32:45.069 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:45.069 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:45.069 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-14T06:32:45.069 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:44.740027+0000", 2024-09-14T06:32:45.070 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.9964999999999999e-05 2024-09-14T06:32:45.070 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:45.070 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:45.070 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:32:45.070 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:44.740082+0000", 2024-09-14T06:32:45.070 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.5123000000000001e-05 2024-09-14T06:32:45.070 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:45.070 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:32:45.070 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:45.070 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:45.070 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:45.070 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:237438 3.14 3:2b4d012e:::rbd_data.112d401cec05.00000000000000ae:head [delete] snapc 25=[] ondisk+write+known_if_redirected+supports_pool_eio e239)", 2024-09-14T06:32:45.070 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:32:44.755579+0000", 2024-09-14T06:32:45.070 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 0.22758973299999999, 2024-09-14T06:32:45.070 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0056954090000000002, 2024-09-14T06:32:45.071 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:32:45.071 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:32:45.071 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:32:45.071 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:32:45.071 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:32:45.071 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:45.071 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 237438 2024-09-14T06:32:45.071 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:45.071 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:32:45.071 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:45.071 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:32:45.071 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:44.755579+0000", 2024-09-14T06:32:45.071 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:45.071 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:45.071 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:45.072 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:32:45.072 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:44.755579+0000", 2024-09-14T06:32:45.072 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:45.072 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:45.072 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:45.072 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:32:45.072 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:44.755581+0000", 2024-09-14T06:32:45.072 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.9099999999999999e-06 2024-09-14T06:32:45.072 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:45.072 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:45.072 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:32:45.072 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:44.755588+0000", 2024-09-14T06:32:45.072 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.9580000000000001e-06 2024-09-14T06:32:45.072 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:45.072 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:45.073 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:32:45.073 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:44.755605+0000", 2024-09-14T06:32:45.073 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.6481e-05 2024-09-14T06:32:45.073 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:45.073 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:45.073 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:32:45.073 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:44.755613+0000", 2024-09-14T06:32:45.073 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.2490000000000001e-06 2024-09-14T06:32:45.073 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:45.073 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:45.073 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:32:45.073 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:44.755706+0000", 2024-09-14T06:32:45.073 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.3346999999999999e-05 2024-09-14T06:32:45.074 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:45.074 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:45.074 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:32:45.074 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:44.755858+0000", 2024-09-14T06:32:45.074 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00015165600000000001 2024-09-14T06:32:45.074 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:45.074 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:45.074 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "waiting for subops from 7", 2024-09-14T06:32:45.074 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:44.755984+0000", 2024-09-14T06:32:45.074 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000125907 2024-09-14T06:32:45.074 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:45.074 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:45.074 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "op_commit", 2024-09-14T06:32:45.074 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:44.756892+0000", 2024-09-14T06:32:45.074 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00090802199999999997 2024-09-14T06:32:45.075 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:45.075 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:45.075 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:32:45.075 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:44.761135+0000", 2024-09-14T06:32:45.075 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.004243251 2024-09-14T06:32:45.075 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:45.075 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:45.075 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-14T06:32:45.075 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:44.761211+0000", 2024-09-14T06:32:45.075 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.6027999999999998e-05 2024-09-14T06:32:45.075 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:45.075 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:45.075 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:32:45.075 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:44.761275+0000", 2024-09-14T06:32:45.075 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.3600000000000001e-05 2024-09-14T06:32:45.075 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:45.076 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:32:45.076 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:45.076 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:45.076 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:45.076 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:237445 3.14 3:2e12370a:::rbd_data.112d401cec05.00000000000000b5:head [delete] snapc 25=[] ondisk+write+known_if_redirected+supports_pool_eio e239)", 2024-09-14T06:32:45.076 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:32:44.757373+0000", 2024-09-14T06:32:45.076 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 0.22579590999999999, 2024-09-14T06:32:45.076 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0063576639999999999, 2024-09-14T06:32:45.076 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:32:45.076 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:32:45.076 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:32:45.076 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:32:45.076 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:32:45.077 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:45.077 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 237445 2024-09-14T06:32:45.077 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:45.077 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:32:45.077 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:45.077 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:32:45.077 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:44.757373+0000", 2024-09-14T06:32:45.077 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:45.077 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:45.077 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:45.077 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:32:45.077 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:44.757373+0000", 2024-09-14T06:32:45.077 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:45.077 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:45.078 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:45.078 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:32:45.078 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:44.757374+0000", 2024-09-14T06:32:45.078 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.359e-06 2024-09-14T06:32:45.078 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:45.078 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:45.078 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:32:45.078 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:44.757381+0000", 2024-09-14T06:32:45.078 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.7769999999999997e-06 2024-09-14T06:32:45.078 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:45.078 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:45.078 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:32:45.078 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:44.757397+0000", 2024-09-14T06:32:45.078 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.5610000000000001e-05 2024-09-14T06:32:45.079 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:45.079 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:45.079 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:32:45.079 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:44.757406+0000", 2024-09-14T06:32:45.079 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.8300000000000002e-06 2024-09-14T06:32:45.079 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:45.079 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:45.079 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:32:45.079 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:44.757462+0000", 2024-09-14T06:32:45.079 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.6348000000000003e-05 2024-09-14T06:32:45.079 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:45.079 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:45.079 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:32:45.079 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:44.757579+0000", 2024-09-14T06:32:45.080 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000116747 2024-09-14T06:32:45.080 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:45.080 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:45.080 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "waiting for subops from 7", 2024-09-14T06:32:45.080 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:44.757704+0000", 2024-09-14T06:32:45.080 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00012546899999999999 2024-09-14T06:32:45.080 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:45.080 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:45.080 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "op_commit", 2024-09-14T06:32:45.080 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:44.758510+0000", 2024-09-14T06:32:45.080 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00080536099999999999 2024-09-14T06:32:45.080 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:45.080 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:45.080 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:32:45.081 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:44.763596+0000", 2024-09-14T06:32:45.081 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0050862049999999999 2024-09-14T06:32:45.081 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:45.081 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:45.081 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-14T06:32:45.081 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:44.763673+0000", 2024-09-14T06:32:45.081 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.7464000000000002e-05 2024-09-14T06:32:45.081 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:45.081 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:45.081 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:32:45.081 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:44.763731+0000", 2024-09-14T06:32:45.081 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.7494000000000002e-05 2024-09-14T06:32:45.081 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:45.081 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:32:45.082 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:45.082 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:45.082 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:32:45.082 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-14T06:32:45.083 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-14T06:32:45.084 INFO:tasks.ceph.osd.4.smithi138.stderr:2024-09-14T06:32:45.082+0000 7f6c653c8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33615) UID: 0 2024-09-14T06:32:45.184 INFO:tasks.ceph.osd.6.smithi138.stderr:2024-09-14T06:32:45.183+0000 7f8cd8849640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33620) UID: 0 2024-09-14T06:32:45.226 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-14T06:32:45.226 INFO:teuthology.orchestra.run.smithi094.stdout: "ops": [], 2024-09-14T06:32:45.226 INFO:teuthology.orchestra.run.smithi094.stdout: "num_ops": 0 2024-09-14T06:32:45.226 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-14T06:32:45.234 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-14T06:32:45.254 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-14T06:32:45.321 INFO:tasks.ceph.osd.2.smithi094.stderr:2024-09-14T06:32:45.320+0000 7faf3269b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33649) UID: 0 2024-09-14T06:32:45.339 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-14T06:32:45.339 INFO:teuthology.orchestra.run.smithi094.stdout: "ops": [], 2024-09-14T06:32:45.340 INFO:teuthology.orchestra.run.smithi094.stdout: "complaint_time": 30, 2024-09-14T06:32:45.340 INFO:teuthology.orchestra.run.smithi094.stdout: "num_blocked_ops": 0 2024-09-14T06:32:45.340 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-14T06:32:45.349 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-14T06:32:45.390 INFO:tasks.ceph.osd.0.smithi094.stderr:2024-09-14T06:32:45.389+0000 7f07c9333640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33636) UID: 0 2024-09-14T06:32:45.399 INFO:teuthology.orchestra.run.smithi094.stdout:osd.0: {} 2024-09-14T06:32:45.399 INFO:teuthology.orchestra.run.smithi094.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-14T06:32:45.416 INFO:teuthology.orchestra.run.smithi094.stdout:osd.1: {} 2024-09-14T06:32:45.416 INFO:teuthology.orchestra.run.smithi094.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-14T06:32:45.432 INFO:teuthology.orchestra.run.smithi094.stdout:osd.2: {} 2024-09-14T06:32:45.432 INFO:teuthology.orchestra.run.smithi094.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-14T06:32:45.448 INFO:teuthology.orchestra.run.smithi094.stdout:osd.3: {} 2024-09-14T06:32:45.448 INFO:teuthology.orchestra.run.smithi094.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-14T06:32:45.465 INFO:teuthology.orchestra.run.smithi094.stdout:osd.4: {} 2024-09-14T06:32:45.466 INFO:teuthology.orchestra.run.smithi094.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-14T06:32:45.482 INFO:teuthology.orchestra.run.smithi094.stdout:osd.5: {} 2024-09-14T06:32:45.483 INFO:teuthology.orchestra.run.smithi094.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-14T06:32:45.486 INFO:tasks.ceph.osd.4.smithi138.stderr:2024-09-14T06:32:45.485+0000 7f6c653c8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33615) UID: 0 2024-09-14T06:32:45.494 INFO:teuthology.orchestra.run.smithi094.stdout:ERROR: (22) Invalid argument 2024-09-14T06:32:45.494 INFO:teuthology.orchestra.run.smithi094.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:32:45.499 INFO:teuthology.orchestra.run.smithi094.stdout:osd.6: {} 2024-09-14T06:32:45.499 INFO:teuthology.orchestra.run.smithi094.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-14T06:32:45.503 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-14T06:32:45.516 INFO:teuthology.orchestra.run.smithi094.stdout:osd.7: {} 2024-09-14T06:32:45.516 INFO:teuthology.orchestra.run.smithi094.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-14T06:32:45.587 INFO:tasks.ceph.osd.7.smithi138.stderr:2024-09-14T06:32:45.585+0000 7f3c6a21d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33623) UID: 0 2024-09-14T06:32:45.609 INFO:teuthology.orchestra.run.smithi138.stdout:ERROR: (22) Invalid argument 2024-09-14T06:32:45.609 INFO:teuthology.orchestra.run.smithi138.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:32:45.617 INFO:teuthology.orchestra.run.smithi094.stderr:nodeep-scrub is unset 2024-09-14T06:32:45.619 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-14T06:32:45.687 INFO:tasks.ceph.osd.3.smithi094.stderr:2024-09-14T06:32:45.686+0000 7feeeafe8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33655) UID: 0 2024-09-14T06:32:45.767 INFO:teuthology.orchestra.run.smithi138.stdout:ERROR: (22) Invalid argument 2024-09-14T06:32:45.767 INFO:teuthology.orchestra.run.smithi138.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:32:45.776 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-14T06:32:45.819 INFO:tasks.ceph.osd.5.smithi138.stderr:2024-09-14T06:32:45.818+0000 7fe13746f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33618) UID: 0 2024-09-14T06:32:45.879 INFO:teuthology.orchestra.run.smithi138.stdout:ERROR: (22) Invalid argument 2024-09-14T06:32:45.879 INFO:teuthology.orchestra.run.smithi138.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:32:45.888 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-14T06:32:45.929 INFO:tasks.ceph.osd.5.smithi138.stderr:2024-09-14T06:32:45.928+0000 7fe13746f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33618) UID: 0 2024-09-14T06:32:45.989 INFO:tasks.ceph.osd.0.smithi094.stderr:2024-09-14T06:32:45.988+0000 7f07c9333640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33636) UID: 0 2024-09-14T06:32:46.031 INFO:teuthology.orchestra.run.smithi138.stdout:ERROR: (22) Invalid argument 2024-09-14T06:32:46.031 INFO:teuthology.orchestra.run.smithi138.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:32:46.039 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-14T06:32:46.090 INFO:tasks.ceph.osd.2.smithi094.stderr:2024-09-14T06:32:46.089+0000 7faf3269b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33649) UID: 0 2024-09-14T06:32:46.106 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-14T06:32:46.143 INFO:teuthology.orchestra.run.smithi138.stdout:ERROR: (22) Invalid argument 2024-09-14T06:32:46.143 INFO:teuthology.orchestra.run.smithi138.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:32:46.152 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-14T06:32:46.191 INFO:tasks.ceph.osd.0.smithi094.stderr:2024-09-14T06:32:46.190+0000 7f07c9333640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33636) UID: 0 2024-09-14T06:32:46.255 INFO:teuthology.orchestra.run.smithi138.stdout:ERROR: (22) Invalid argument 2024-09-14T06:32:46.255 INFO:teuthology.orchestra.run.smithi138.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:32:46.264 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-14T06:32:46.292 INFO:tasks.ceph.osd.1.smithi094.stderr:2024-09-14T06:32:46.291+0000 7f328e47a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33641) UID: 0 2024-09-14T06:32:46.367 INFO:teuthology.orchestra.run.smithi138.stdout:ERROR: (22) Invalid argument 2024-09-14T06:32:46.367 INFO:teuthology.orchestra.run.smithi138.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:32:46.375 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-14T06:32:46.392 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-14T06:32:46.393 INFO:teuthology.orchestra.run.smithi094.stderr:dumped all 2024-09-14T06:32:46.404 INFO:teuthology.orchestra.run.smithi094.stdout:{"pg_ready":true,"pg_map":{"version":356,"stamp":"2024-09-14T06:32:45.613000+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":937904146,"num_objects":1223,"num_object_clones":0,"num_object_copies":2446,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1223,"num_whiteouts":0,"num_read":236060,"num_read_kb":2181472,"num_write":36380,"num_write_kb":1479756,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5149,"num_bytes_recovered":633091608,"num_keys_recovered":534,"num_objects_omap":20,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":26713,"ondisk_log_size":26713,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":7,"num_per_pool_osds":7,"num_per_pool_omap_osds":7,"kb":660602880,"kb_used":520196,"kb_used_data":86560,"kb_used_omap":10,"kb_used_meta":433333,"kb_avail":660082684,"statfs":{"total":676457349120,"available":675924668416,"internally_reserved":0,"allocated":88637440,"data_stored":687244143,"data_compressed":41350952,"data_compressed_allocated":54673408,"data_compressed_original":666447181,"omap_allocated":11157,"internal_metadata":443733099},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-208769540,"num_objects":-114,"num_object_clones":0,"num_object_copies":-228,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-114,"num_whiteouts":0,"num_read":949,"num_read_kb":822,"num_write":248,"num_write_kb":54,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":148,"num_bytes_recovered":6082709,"num_keys_recovered":11,"num_objects_omap":-5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"3.997285"},"pg_stats":[{"pgid":"3.1f","version":"237'798","reported_seq":8105,"reported_epoch":237,"state":"active+clean","last_fresh":"2024-09-14T06:32:42.723600+0000","last_change":"2024-09-14T06:32:42.622370+0000","last_active":"2024-09-14T06:32:42.723600+0000","last_peered":"2024-09-14T06:32:42.723600+0000","last_clean":"2024-09-14T06:32:42.723600+0000","last_became_active":"2024-09-14T06:29:00.579883+0000","last_became_peered":"2024-09-14T06:29:00.579883+0000","last_unstale":"2024-09-14T06:32:42.723600+0000","last_undegraded":"2024-09-14T06:32:42.723600+0000","last_fullsized":"2024-09-14T06:32:42.723600+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":798,"log_dups_size":0,"ondisk_log_size":798,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T14:44:05.442430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015915510000000001,"stat_sum":{"num_bytes":25212990,"num_objects":33,"num_object_clones":0,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":33,"num_whiteouts":0,"num_read":8292,"num_read_kb":86746,"num_write":1122,"num_write_kb":49537,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"29","length":"1"},{"start":"2b","length":"1"},{"start":"2d","length":"1"},{"start":"2f","length":"1"},{"start":"31","length":"1"},{"start":"33","length":"1"}]},{"pgid":"3.1e","version":"235'750","reported_seq":9967,"reported_epoch":235,"state":"active+clean","last_fresh":"2024-09-14T06:32:40.713486+0000","last_change":"2024-09-14T06:32:40.629980+0000","last_active":"2024-09-14T06:32:40.713486+0000","last_peered":"2024-09-14T06:32:40.713486+0000","last_clean":"2024-09-14T06:32:40.713486+0000","last_became_active":"2024-09-14T06:32:17.679198+0000","last_became_peered":"2024-09-14T06:32:17.679198+0000","last_unstale":"2024-09-14T06:32:40.713486+0000","last_undegraded":"2024-09-14T06:32:40.713486+0000","last_fullsized":"2024-09-14T06:32:40.713486+0000","mapping_epoch":211,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":212,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":750,"log_dups_size":0,"ondisk_log_size":750,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T12:02:59.537260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0025465459999999998,"stat_sum":{"num_bytes":28440871,"num_objects":32,"num_object_clones":0,"num_object_copies":64,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":32,"num_whiteouts":0,"num_read":8310,"num_read_kb":67655,"num_write":993,"num_write_kb":40407,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":713,"num_bytes_recovered":87148703,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2d","length":"1"},{"start":"2f","length":"1"},{"start":"31","length":"1"},{"start":"33","length":"1"}]},{"pgid":"3.1d","version":"234'993","reported_seq":7186,"reported_epoch":238,"state":"active+clean","last_fresh":"2024-09-14T06:32:43.621605+0000","last_change":"2024-09-14T06:32:43.621824+0000","last_active":"2024-09-14T06:32:43.621605+0000","last_peered":"2024-09-14T06:32:43.621605+0000","last_clean":"2024-09-14T06:32:43.621605+0000","last_became_active":"2024-09-14T06:32:24.716363+0000","last_became_peered":"2024-09-14T06:32:24.716363+0000","last_unstale":"2024-09-14T06:32:43.621605+0000","last_undegraded":"2024-09-14T06:32:43.621605+0000","last_fullsized":"2024-09-14T06:32:43.621605+0000","mapping_epoch":218,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":219,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":993,"log_dups_size":0,"ondisk_log_size":993,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T07:10:14.350088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00081397399999999997,"stat_sum":{"num_bytes":42985469,"num_objects":38,"num_object_clones":0,"num_object_copies":76,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":38,"num_whiteouts":0,"num_read":7636,"num_read_kb":64175,"num_write":1475,"num_write_kb":53276,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"27","length":"1"},{"start":"29","length":"1"},{"start":"2b","length":"1"},{"start":"2d","length":"1"},{"start":"2f","length":"1"},{"start":"31","length":"1"},{"start":"33","length":"1"}]},{"pgid":"3.1c","version":"238'720","reported_seq":5671,"reported_epoch":238,"state":"active+clean","last_fresh":"2024-09-14T06:32:43.723861+0000","last_change":"2024-09-14T06:32:43.618395+0000","last_active":"2024-09-14T06:32:43.723861+0000","last_peered":"2024-09-14T06:32:43.723861+0000","last_clean":"2024-09-14T06:32:43.723861+0000","last_became_active":"2024-09-14T06:32:39.775788+0000","last_became_peered":"2024-09-14T06:32:39.775788+0000","last_unstale":"2024-09-14T06:32:43.723861+0000","last_undegraded":"2024-09-14T06:32:43.723861+0000","last_fullsized":"2024-09-14T06:32:43.723861+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":720,"log_dups_size":0,"ondisk_log_size":720,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T06:34:45.769015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097621199999999996,"stat_sum":{"num_bytes":17726779,"num_objects":44,"num_object_clones":0,"num_object_copies":88,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":44,"num_whiteouts":0,"num_read":4689,"num_read_kb":43569,"num_write":851,"num_write_kb":45545,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":505,"num_bytes_recovered":44145197,"num_keys_recovered":280,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"27","length":"1"},{"start":"29","length":"1"},{"start":"2b","length":"1"},{"start":"2d","length":"1"},{"start":"2f","length":"1"},{"start":"31","length":"1"},{"start":"33","length":"1"}]},{"pgid":"3.1b","version":"236'910","reported_seq":11274,"reported_epoch":237,"state":"active+clean","last_fresh":"2024-09-14T06:32:42.621027+0000","last_change":"2024-09-14T06:32:42.621201+0000","last_active":"2024-09-14T06:32:42.621027+0000","last_peered":"2024-09-14T06:32:42.621027+0000","last_clean":"2024-09-14T06:32:42.621027+0000","last_became_active":"2024-09-14T06:29:00.586183+0000","last_became_peered":"2024-09-14T06:29:00.586183+0000","last_unstale":"2024-09-14T06:32:42.621027+0000","last_undegraded":"2024-09-14T06:32:42.621027+0000","last_fullsized":"2024-09-14T06:32:42.621027+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":910,"log_dups_size":0,"ondisk_log_size":910,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T09:43:18.628500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062806799999999996,"stat_sum":{"num_bytes":20631325,"num_objects":49,"num_object_clones":0,"num_object_copies":98,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":49,"num_whiteouts":0,"num_read":11906,"num_read_kb":108092,"num_write":1414,"num_write_kb":55102,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"29","length":"1"},{"start":"2b","length":"1"},{"start":"2d","length":"1"},{"start":"2f","length":"1"},{"start":"31","length":"1"},{"start":"33","length":"1"}]},{"pgid":"3.1a","version":"237'1585","reported_seq":10165,"reported_epoch":238,"state":"active+clean","last_fresh":"2024-09-14T06:32:43.727460+0000","last_change":"2024-09-14T06:32:43.621551+0000","last_active":"2024-09-14T06:32:43.727460+0000","last_peered":"2024-09-14T06:32:43.727460+0000","last_clean":"2024-09-14T06:32:43.727460+0000","last_became_active":"2024-09-14T06:29:00.576206+0000","last_became_peered":"2024-09-14T06:29:00.576206+0000","last_unstale":"2024-09-14T06:32:43.727460+0000","last_undegraded":"2024-09-14T06:32:43.727460+0000","last_fullsized":"2024-09-14T06:32:43.727460+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":1585,"log_dups_size":0,"ondisk_log_size":1585,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T17:39:46.417447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059520000000000005,"stat_sum":{"num_bytes":31076772,"num_objects":47,"num_object_clones":0,"num_object_copies":94,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":47,"num_whiteouts":0,"num_read":12993,"num_read_kb":82597,"num_write":2363,"num_write_kb":55671,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"27","length":"1"},{"start":"29","length":"1"},{"start":"2b","length":"1"},{"start":"2d","length":"1"},{"start":"2f","length":"1"},{"start":"31","length":"1"},{"start":"33","length":"1"}]},{"pgid":"3.19","version":"238'509","reported_seq":6258,"reported_epoch":238,"state":"active+clean","last_fresh":"2024-09-14T06:32:43.618317+0000","last_change":"2024-09-14T06:32:43.618317+0000","last_active":"2024-09-14T06:32:43.618317+0000","last_peered":"2024-09-14T06:32:43.618317+0000","last_clean":"2024-09-14T06:32:43.618317+0000","last_became_active":"2024-09-14T06:31:53.744800+0000","last_became_peered":"2024-09-14T06:31:53.744800+0000","last_unstale":"2024-09-14T06:32:43.618317+0000","last_undegraded":"2024-09-14T06:32:43.618317+0000","last_fullsized":"2024-09-14T06:32:43.618317+0000","mapping_epoch":188,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":189,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":509,"log_dups_size":0,"ondisk_log_size":509,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T08:37:31.957932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011132309999999999,"stat_sum":{"num_bytes":34253926,"num_objects":40,"num_object_clones":0,"num_object_copies":80,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":40,"num_whiteouts":0,"num_read":5087,"num_read_kb":67755,"num_write":711,"num_write_kb":43868,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":282,"num_bytes_recovered":41927489,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"27","length":"1"},{"start":"29","length":"1"},{"start":"2b","length":"1"},{"start":"2d","length":"1"},{"start":"2f","length":"1"},{"start":"31","length":"1"},{"start":"33","length":"1"}]},{"pgid":"3.18","version":"234'499","reported_seq":6027,"reported_epoch":235,"state":"active+clean","last_fresh":"2024-09-14T06:32:40.729078+0000","last_change":"2024-09-14T06:32:40.629026+0000","last_active":"2024-09-14T06:32:40.729078+0000","last_peered":"2024-09-14T06:32:40.729078+0000","last_clean":"2024-09-14T06:32:40.729078+0000","last_became_active":"2024-09-14T06:31:53.745159+0000","last_became_peered":"2024-09-14T06:31:53.745159+0000","last_unstale":"2024-09-14T06:32:40.729078+0000","last_undegraded":"2024-09-14T06:32:40.729078+0000","last_fullsized":"2024-09-14T06:32:40.729078+0000","mapping_epoch":188,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":189,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":499,"log_dups_size":0,"ondisk_log_size":499,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T08:21:11.501994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0016595749999999999,"stat_sum":{"num_bytes":22128117,"num_objects":35,"num_object_clones":0,"num_object_copies":70,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":35,"num_whiteouts":0,"num_read":4458,"num_read_kb":78893,"num_write":624,"num_write_kb":51576,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":310,"num_bytes_recovered":56082976,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2d","length":"1"},{"start":"2f","length":"1"},{"start":"31","length":"1"},{"start":"33","length":"1"}]},{"pgid":"3.17","version":"237'931","reported_seq":9281,"reported_epoch":237,"state":"active+clean","last_fresh":"2024-09-14T06:32:42.716907+0000","last_change":"2024-09-14T06:32:42.621220+0000","last_active":"2024-09-14T06:32:42.716907+0000","last_peered":"2024-09-14T06:32:42.716907+0000","last_clean":"2024-09-14T06:32:42.716907+0000","last_became_active":"2024-09-14T06:29:00.579714+0000","last_became_peered":"2024-09-14T06:29:00.579714+0000","last_unstale":"2024-09-14T06:32:42.716907+0000","last_undegraded":"2024-09-14T06:32:42.716907+0000","last_fullsized":"2024-09-14T06:32:42.716907+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":931,"log_dups_size":0,"ondisk_log_size":931,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T07:25:42.063845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00084899900000000002,"stat_sum":{"num_bytes":49692439,"num_objects":52,"num_object_clones":0,"num_object_copies":104,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":52,"num_whiteouts":0,"num_read":10311,"num_read_kb":68638,"num_write":1065,"num_write_kb":48465,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"29","length":"1"},{"start":"2b","length":"1"},{"start":"2d","length":"1"},{"start":"2f","length":"1"},{"start":"31","length":"1"},{"start":"33","length":"1"}]},{"pgid":"3.16","version":"235'1124","reported_seq":3988,"reported_epoch":238,"state":"active+clean","last_fresh":"2024-09-14T06:32:43.622619+0000","last_change":"2024-09-14T06:32:43.622812+0000","last_active":"2024-09-14T06:32:43.622619+0000","last_peered":"2024-09-14T06:32:43.622619+0000","last_clean":"2024-09-14T06:32:43.622619+0000","last_became_active":"2024-09-14T06:32:24.718745+0000","last_became_peered":"2024-09-14T06:32:24.718745+0000","last_unstale":"2024-09-14T06:32:43.622619+0000","last_undegraded":"2024-09-14T06:32:43.622619+0000","last_fullsized":"2024-09-14T06:32:43.622619+0000","mapping_epoch":218,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":219,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":1124,"log_dups_size":0,"ondisk_log_size":1124,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T09:15:35.275119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00086667799999999996,"stat_sum":{"num_bytes":14704807,"num_objects":32,"num_object_clones":0,"num_object_copies":64,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":32,"num_whiteouts":0,"num_read":5658,"num_read_kb":38562,"num_write":1711,"num_write_kb":37248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"27","length":"1"},{"start":"29","length":"1"},{"start":"2b","length":"1"},{"start":"2d","length":"1"},{"start":"2f","length":"1"},{"start":"31","length":"1"},{"start":"33","length":"1"}]},{"pgid":"3.15","version":"236'587","reported_seq":6427,"reported_epoch":238,"state":"active+clean","last_fresh":"2024-09-14T06:32:43.621901+0000","last_change":"2024-09-14T06:32:43.622184+0000","last_active":"2024-09-14T06:32:43.621901+0000","last_peered":"2024-09-14T06:32:43.621901+0000","last_clean":"2024-09-14T06:32:43.621901+0000","last_became_active":"2024-09-14T06:31:53.746091+0000","last_became_peered":"2024-09-14T06:31:53.746091+0000","last_unstale":"2024-09-14T06:32:43.621901+0000","last_undegraded":"2024-09-14T06:32:43.621901+0000","last_fullsized":"2024-09-14T06:32:43.621901+0000","mapping_epoch":188,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":189,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":587,"log_dups_size":0,"ondisk_log_size":587,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T12:56:00.840746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00058041699999999996,"stat_sum":{"num_bytes":34132886,"num_objects":39,"num_object_clones":0,"num_object_copies":78,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":39,"num_whiteouts":0,"num_read":4928,"num_read_kb":74748,"num_write":641,"num_write_kb":51190,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":271,"num_bytes_recovered":49981473,"num_keys_recovered":33,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"27","length":"1"},{"start":"29","length":"1"},{"start":"2b","length":"1"},{"start":"2d","length":"1"},{"start":"2f","length":"1"},{"start":"31","length":"1"},{"start":"33","length":"1"}]},{"pgid":"3.14","version":"237'1083","reported_seq":6012,"reported_epoch":237,"state":"active+clean","last_fresh":"2024-09-14T06:32:42.703832+0000","last_change":"2024-09-14T06:32:42.701857+0000","last_active":"2024-09-14T06:32:42.703832+0000","last_peered":"2024-09-14T06:32:42.703832+0000","last_clean":"2024-09-14T06:32:42.703832+0000","last_became_active":"2024-09-14T06:32:39.776600+0000","last_became_peered":"2024-09-14T06:32:39.776600+0000","last_unstale":"2024-09-14T06:32:42.703832+0000","last_undegraded":"2024-09-14T06:32:42.703832+0000","last_fullsized":"2024-09-14T06:32:42.703832+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":1083,"log_dups_size":0,"ondisk_log_size":1083,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T18:28:14.973851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021024889999999999,"stat_sum":{"num_bytes":17246307,"num_objects":29,"num_object_clones":0,"num_object_copies":58,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":29,"num_whiteouts":0,"num_read":8770,"num_read_kb":46783,"num_write":1528,"num_write_kb":29823,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":148,"num_bytes_recovered":6082709,"num_keys_recovered":11,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"29","length":"1"},{"start":"2b","length":"1"},{"start":"2d","length":"1"},{"start":"2f","length":"1"},{"start":"31","length":"1"},{"start":"33","length":"1"}]},{"pgid":"2.7","version":"0'0","reported_seq":94,"reported_epoch":238,"state":"active+clean","last_fresh":"2024-09-14T06:32:43.621919+0000","last_change":"2024-09-14T06:32:01.612470+0000","last_active":"2024-09-14T06:32:43.621919+0000","last_peered":"2024-09-14T06:32:43.621919+0000","last_clean":"2024-09-14T06:32:43.621919+0000","last_became_active":"2024-09-14T06:32:01.612121+0000","last_became_peered":"2024-09-14T06:32:01.612121+0000","last_unstale":"2024-09-14T06:32:43.621919+0000","last_undegraded":"2024-09-14T06:32:43.621919+0000","last_fullsized":"2024-09-14T06:32:43.621919+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:28:50.205350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:28:50.205350+0000","last_clean_scrub_stamp":"2024-09-14T06:28:50.205350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T13:40:52.537942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.6","version":"237'793","reported_seq":8351,"reported_epoch":237,"state":"active+clean","last_fresh":"2024-09-14T06:32:42.702096+0000","last_change":"2024-09-14T06:32:42.620969+0000","last_active":"2024-09-14T06:32:42.702096+0000","last_peered":"2024-09-14T06:32:42.702096+0000","last_clean":"2024-09-14T06:32:42.702096+0000","last_became_active":"2024-09-14T06:29:00.578283+0000","last_became_peered":"2024-09-14T06:29:00.578283+0000","last_unstale":"2024-09-14T06:32:42.702096+0000","last_undegraded":"2024-09-14T06:32:42.702096+0000","last_fullsized":"2024-09-14T06:32:42.702096+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":793,"log_dups_size":0,"ondisk_log_size":793,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T15:20:25.657119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00078026400000000002,"stat_sum":{"num_bytes":15699474,"num_objects":31,"num_object_clones":0,"num_object_copies":62,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":31,"num_whiteouts":0,"num_read":8373,"num_read_kb":78571,"num_write":850,"num_write_kb":48287,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"29","length":"1"},{"start":"2b","length":"1"},{"start":"2d","length":"1"},{"start":"2f","length":"1"},{"start":"31","length":"1"},{"start":"33","length":"1"}]},{"pgid":"2.6","version":"0'0","reported_seq":459,"reported_epoch":238,"state":"active+clean","last_fresh":"2024-09-14T06:32:43.617992+0000","last_change":"2024-09-14T06:32:01.608456+0000","last_active":"2024-09-14T06:32:43.617992+0000","last_peered":"2024-09-14T06:32:43.617992+0000","last_clean":"2024-09-14T06:32:43.617992+0000","last_became_active":"2024-09-14T06:32:01.608178+0000","last_became_peered":"2024-09-14T06:32:01.608178+0000","last_unstale":"2024-09-14T06:32:43.617992+0000","last_undegraded":"2024-09-14T06:32:43.617992+0000","last_fullsized":"2024-09-14T06:32:43.617992+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:28:50.205350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:28:50.205350+0000","last_clean_scrub_stamp":"2024-09-14T06:28:50.205350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T10:16:40.480031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065018200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"235'785","reported_seq":4315,"reported_epoch":235,"state":"active+clean","last_fresh":"2024-09-14T06:32:40.628084+0000","last_change":"2024-09-14T06:32:40.628084+0000","last_active":"2024-09-14T06:32:40.628084+0000","last_peered":"2024-09-14T06:32:40.628084+0000","last_clean":"2024-09-14T06:32:40.628084+0000","last_became_active":"2024-09-14T06:31:54.747923+0000","last_became_peered":"2024-09-14T06:31:54.747923+0000","last_unstale":"2024-09-14T06:32:40.628084+0000","last_undegraded":"2024-09-14T06:32:40.628084+0000","last_fullsized":"2024-09-14T06:32:40.628084+0000","mapping_epoch":189,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":190,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":785,"log_dups_size":0,"ondisk_log_size":785,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T17:36:19.829390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00112521,"stat_sum":{"num_bytes":24555353,"num_objects":30,"num_object_clones":0,"num_object_copies":60,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":30,"num_whiteouts":0,"num_read":4414,"num_read_kb":40976,"num_write":1132,"num_write_kb":36116,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":183,"num_bytes_recovered":13912513,"num_keys_recovered":22,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2d","length":"1"},{"start":"2f","length":"1"},{"start":"31","length":"1"},{"start":"33","length":"1"}]},{"pgid":"2.5","version":"0'0","reported_seq":453,"reported_epoch":238,"state":"active+clean","last_fresh":"2024-09-14T06:32:43.621844+0000","last_change":"2024-09-14T06:28:53.640035+0000","last_active":"2024-09-14T06:32:43.621844+0000","last_peered":"2024-09-14T06:32:43.621844+0000","last_clean":"2024-09-14T06:32:43.621844+0000","last_became_active":"2024-09-14T06:28:51.308058+0000","last_became_peered":"2024-09-14T06:28:51.308058+0000","last_unstale":"2024-09-14T06:32:43.621844+0000","last_undegraded":"2024-09-14T06:32:43.621844+0000","last_fullsized":"2024-09-14T06:32:43.621844+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:28:50.205350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:28:50.205350+0000","last_clean_scrub_stamp":"2024-09-14T06:28:50.205350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T13:35:48.544057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00037824000000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"236'654","reported_seq":3904,"reported_epoch":238,"state":"active+clean","last_fresh":"2024-09-14T06:32:43.618374+0000","last_change":"2024-09-14T06:32:43.618632+0000","last_active":"2024-09-14T06:32:43.618374+0000","last_peered":"2024-09-14T06:32:43.618374+0000","last_clean":"2024-09-14T06:32:43.618374+0000","last_became_active":"2024-09-14T06:32:17.675470+0000","last_became_peered":"2024-09-14T06:32:17.675470+0000","last_unstale":"2024-09-14T06:32:43.618374+0000","last_undegraded":"2024-09-14T06:32:43.618374+0000","last_fullsized":"2024-09-14T06:32:43.618374+0000","mapping_epoch":211,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":212,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":654,"log_dups_size":0,"ondisk_log_size":654,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T13:18:03.757852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088633400000000002,"stat_sum":{"num_bytes":12728367,"num_objects":34,"num_object_clones":0,"num_object_copies":68,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":34,"num_whiteouts":0,"num_read":2623,"num_read_kb":37411,"num_write":850,"num_write_kb":40855,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":331,"num_bytes_recovered":31411119,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"27","length":"1"},{"start":"29","length":"1"},{"start":"2b","length":"1"},{"start":"2d","length":"1"},{"start":"2f","length":"1"},{"start":"31","length":"1"},{"start":"33","length":"1"}]},{"pgid":"2.4","version":"0'0","reported_seq":453,"reported_epoch":238,"state":"active+clean","last_fresh":"2024-09-14T06:32:43.617968+0000","last_change":"2024-09-14T06:28:53.226072+0000","last_active":"2024-09-14T06:32:43.617968+0000","last_peered":"2024-09-14T06:32:43.617968+0000","last_clean":"2024-09-14T06:32:43.617968+0000","last_became_active":"2024-09-14T06:28:51.308072+0000","last_became_peered":"2024-09-14T06:28:51.308072+0000","last_unstale":"2024-09-14T06:32:43.617968+0000","last_undegraded":"2024-09-14T06:32:43.617968+0000","last_fullsized":"2024-09-14T06:32:43.617968+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:28:50.205350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:28:50.205350+0000","last_clean_scrub_stamp":"2024-09-14T06:28:50.205350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T13:15:09.052328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066197399999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"229'854","reported_seq":5413,"reported_epoch":235,"state":"active+clean","last_fresh":"2024-09-14T06:32:40.628189+0000","last_change":"2024-09-14T06:32:40.628504+0000","last_active":"2024-09-14T06:32:40.628189+0000","last_peered":"2024-09-14T06:32:40.628189+0000","last_clean":"2024-09-14T06:32:40.628189+0000","last_became_active":"2024-09-14T06:32:24.716458+0000","last_became_peered":"2024-09-14T06:32:24.716458+0000","last_unstale":"2024-09-14T06:32:40.628189+0000","last_undegraded":"2024-09-14T06:32:40.628189+0000","last_fullsized":"2024-09-14T06:32:40.628189+0000","mapping_epoch":218,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":219,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":854,"log_dups_size":0,"ondisk_log_size":854,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T15:55:23.510859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001426797,"stat_sum":{"num_bytes":27242971,"num_objects":41,"num_object_clones":0,"num_object_copies":82,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":41,"num_whiteouts":0,"num_read":4510,"num_read_kb":42630,"num_write":1214,"num_write_kb":49494,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":330,"num_bytes_recovered":28798893,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2d","length":"1"},{"start":"2f","length":"1"},{"start":"31","length":"1"},{"start":"33","length":"1"}]},{"pgid":"3.3","version":"238'588","reported_seq":10470,"reported_epoch":238,"state":"active+clean","last_fresh":"2024-09-14T06:32:43.720879+0000","last_change":"2024-09-14T06:32:43.622014+0000","last_active":"2024-09-14T06:32:43.720879+0000","last_peered":"2024-09-14T06:32:43.720879+0000","last_clean":"2024-09-14T06:32:43.720879+0000","last_became_active":"2024-09-14T06:29:00.584678+0000","last_became_peered":"2024-09-14T06:29:00.584678+0000","last_unstale":"2024-09-14T06:32:43.720879+0000","last_undegraded":"2024-09-14T06:32:43.720879+0000","last_fullsized":"2024-09-14T06:32:43.720879+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":588,"log_dups_size":0,"ondisk_log_size":588,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T13:05:58.655890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00089673599999999997,"stat_sum":{"num_bytes":48374477,"num_objects":29,"num_object_clones":0,"num_object_copies":58,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":29,"num_whiteouts":0,"num_read":9295,"num_read_kb":89064,"num_write":661,"num_write_kb":51631,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"27","length":"1"},{"start":"29","length":"1"},{"start":"2b","length":"1"},{"start":"2d","length":"1"},{"start":"2f","length":"1"},{"start":"31","length":"1"},{"start":"33","length":"1"}]},{"pgid":"2.2","version":"22'2","reported_seq":68,"reported_epoch":238,"state":"active+clean","last_fresh":"2024-09-14T06:32:43.618124+0000","last_change":"2024-09-14T06:32:24.716937+0000","last_active":"2024-09-14T06:32:43.618124+0000","last_peered":"2024-09-14T06:32:43.618124+0000","last_clean":"2024-09-14T06:32:43.618124+0000","last_became_active":"2024-09-14T06:32:24.716539+0000","last_became_peered":"2024-09-14T06:32:24.716539+0000","last_unstale":"2024-09-14T06:32:43.618124+0000","last_undegraded":"2024-09-14T06:32:43.618124+0000","last_fullsized":"2024-09-14T06:32:43.618124+0000","mapping_epoch":218,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":219,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:28:50.205350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:28:50.205350+0000","last_clean_scrub_stamp":"2024-09-14T06:28:50.205350+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T07:16:29.639469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"238'1066","reported_seq":6896,"reported_epoch":238,"state":"active+clean","last_fresh":"2024-09-14T06:32:43.617730+0000","last_change":"2024-09-14T06:32:43.617730+0000","last_active":"2024-09-14T06:32:43.617730+0000","last_peered":"2024-09-14T06:32:43.617730+0000","last_clean":"2024-09-14T06:32:43.617730+0000","last_became_active":"2024-09-14T06:29:00.574873+0000","last_became_peered":"2024-09-14T06:29:00.574873+0000","last_unstale":"2024-09-14T06:32:43.617730+0000","last_undegraded":"2024-09-14T06:32:43.617730+0000","last_fullsized":"2024-09-14T06:32:43.617730+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":1066,"log_dups_size":0,"ondisk_log_size":1066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T08:13:39.775536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061334200000000001,"stat_sum":{"num_bytes":43049554,"num_objects":48,"num_object_clones":0,"num_object_copies":96,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":48,"num_whiteouts":0,"num_read":8253,"num_read_kb":85478,"num_write":1604,"num_write_kb":58700,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"27","length":"1"},{"start":"29","length":"1"},{"start":"2b","length":"1"},{"start":"2d","length":"1"},{"start":"2f","length":"1"},{"start":"31","length":"1"},{"start":"33","length":"1"}]},{"pgid":"2.1","version":"0'0","reported_seq":67,"reported_epoch":237,"state":"active+clean","last_fresh":"2024-09-14T06:32:42.699579+0000","last_change":"2024-09-14T06:32:14.665376+0000","last_active":"2024-09-14T06:32:42.699579+0000","last_peered":"2024-09-14T06:32:42.699579+0000","last_clean":"2024-09-14T06:32:42.699579+0000","last_became_active":"2024-09-14T06:32:14.665081+0000","last_became_peered":"2024-09-14T06:32:14.665081+0000","last_unstale":"2024-09-14T06:32:42.699579+0000","last_undegraded":"2024-09-14T06:32:42.699579+0000","last_fullsized":"2024-09-14T06:32:42.699579+0000","mapping_epoch":208,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":209,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:28:50.205350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:28:50.205350+0000","last_clean_scrub_stamp":"2024-09-14T06:28:50.205350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T16:35:12.869944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053403100000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1","version":"237'739","reported_seq":11529,"reported_epoch":237,"state":"active+clean","last_fresh":"2024-09-14T06:32:42.749374+0000","last_change":"2024-09-14T06:32:42.620695+0000","last_active":"2024-09-14T06:32:42.749374+0000","last_peered":"2024-09-14T06:32:42.749374+0000","last_clean":"2024-09-14T06:32:42.749374+0000","last_became_active":"2024-09-14T06:30:57.768238+0000","last_became_peered":"2024-09-14T06:30:57.768238+0000","last_unstale":"2024-09-14T06:32:42.749374+0000","last_undegraded":"2024-09-14T06:32:42.749374+0000","last_fullsized":"2024-09-14T06:32:42.749374+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":739,"log_dups_size":0,"ondisk_log_size":739,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T18:00:33.732750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054797400000000001,"stat_sum":{"num_bytes":40541999,"num_objects":36,"num_object_clones":0,"num_object_copies":72,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":36,"num_whiteouts":0,"num_read":11055,"num_read_kb":112235,"num_write":915,"num_write_kb":47393,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":183,"num_bytes_recovered":30431994,"num_keys_recovered":22,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"29","length":"1"},{"start":"2b","length":"1"},{"start":"2d","length":"1"},{"start":"2f","length":"1"},{"start":"31","length":"1"},{"start":"33","length":"1"}]},{"pgid":"2.0","version":"0'0","reported_seq":453,"reported_epoch":238,"state":"active+clean","last_fresh":"2024-09-14T06:32:43.621327+0000","last_change":"2024-09-14T06:28:53.640096+0000","last_active":"2024-09-14T06:32:43.621327+0000","last_peered":"2024-09-14T06:32:43.621327+0000","last_clean":"2024-09-14T06:32:43.621327+0000","last_became_active":"2024-09-14T06:28:51.223069+0000","last_became_peered":"2024-09-14T06:28:51.223069+0000","last_unstale":"2024-09-14T06:32:43.621327+0000","last_undegraded":"2024-09-14T06:32:43.621327+0000","last_fullsized":"2024-09-14T06:32:43.621327+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:28:50.205350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:28:50.205350+0000","last_clean_scrub_stamp":"2024-09-14T06:28:50.205350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T15:13:00.471509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00043714600000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"234'1598","reported_seq":7570,"reported_epoch":235,"state":"active+clean","last_fresh":"2024-09-14T06:32:40.628278+0000","last_change":"2024-09-14T06:32:40.628508+0000","last_active":"2024-09-14T06:32:40.628278+0000","last_peered":"2024-09-14T06:32:40.628278+0000","last_clean":"2024-09-14T06:32:40.628278+0000","last_became_active":"2024-09-14T06:31:54.744160+0000","last_became_peered":"2024-09-14T06:31:54.744160+0000","last_unstale":"2024-09-14T06:32:40.628278+0000","last_undegraded":"2024-09-14T06:32:40.628278+0000","last_fullsized":"2024-09-14T06:32:40.628278+0000","mapping_epoch":189,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":190,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":1598,"log_dups_size":0,"ondisk_log_size":1598,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T13:47:24.444561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001372971,"stat_sum":{"num_bytes":43675338,"num_objects":51,"num_object_clones":0,"num_object_copies":102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":51,"num_whiteouts":0,"num_read":10684,"num_read_kb":70562,"num_write":2532,"num_write_kb":53559,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":327,"num_bytes_recovered":50047063,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2d","length":"1"},{"start":"2f","length":"1"},{"start":"31","length":"1"},{"start":"33","length":"1"}]},{"pgid":"2.3","version":"20'1","reported_seq":52,"reported_epoch":237,"state":"active+clean","last_fresh":"2024-09-14T06:32:42.699608+0000","last_change":"2024-09-14T06:32:24.717893+0000","last_active":"2024-09-14T06:32:42.699608+0000","last_peered":"2024-09-14T06:32:42.699608+0000","last_clean":"2024-09-14T06:32:42.699608+0000","last_became_active":"2024-09-14T06:32:24.717658+0000","last_became_peered":"2024-09-14T06:32:24.717658+0000","last_unstale":"2024-09-14T06:32:42.699608+0000","last_undegraded":"2024-09-14T06:32:42.699608+0000","last_fullsized":"2024-09-14T06:32:42.699608+0000","mapping_epoch":218,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":219,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:28:50.205350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:28:50.205350+0000","last_clean_scrub_stamp":"2024-09-14T06:28:50.205350+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T10:36:16.650799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.0","version":"17'72","reported_seq":544,"reported_epoch":238,"state":"active+clean","last_fresh":"2024-09-14T06:32:43.621370+0000","last_change":"2024-09-14T06:28:44.839885+0000","last_active":"2024-09-14T06:32:43.621370+0000","last_peered":"2024-09-14T06:32:43.621370+0000","last_clean":"2024-09-14T06:32:43.621370+0000","last_became_active":"2024-09-14T06:28:44.839573+0000","last_became_peered":"2024-09-14T06:28:44.839573+0000","last_unstale":"2024-09-14T06:32:43.621370+0000","last_undegraded":"2024-09-14T06:32:43.621370+0000","last_fullsized":"2024-09-14T06:32:43.621370+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:28:43.828494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:28:43.828494+0000","last_clean_scrub_stamp":"2024-09-14T06:28:43.828494+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T14:41:25.787293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.8","version":"234'611","reported_seq":6301,"reported_epoch":235,"state":"active+clean","last_fresh":"2024-09-14T06:32:40.629274+0000","last_change":"2024-09-14T06:32:40.629478+0000","last_active":"2024-09-14T06:32:40.629274+0000","last_peered":"2024-09-14T06:32:40.629274+0000","last_clean":"2024-09-14T06:32:40.629274+0000","last_became_active":"2024-09-14T06:31:55.754694+0000","last_became_peered":"2024-09-14T06:31:55.754694+0000","last_unstale":"2024-09-14T06:32:40.629274+0000","last_undegraded":"2024-09-14T06:32:40.629274+0000","last_fullsized":"2024-09-14T06:32:40.629274+0000","mapping_epoch":190,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":191,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":611,"log_dups_size":0,"ondisk_log_size":611,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T14:01:10.585303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0017852390000000001,"stat_sum":{"num_bytes":45016747,"num_objects":46,"num_object_clones":0,"num_object_copies":92,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":46,"num_whiteouts":0,"num_read":5613,"num_read_kb":90698,"num_write":859,"num_write_kb":57685,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":319,"num_bytes_recovered":60352269,"num_keys_recovered":22,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2d","length":"1"},{"start":"2f","length":"1"},{"start":"31","length":"1"},{"start":"33","length":"1"}]},{"pgid":"3.9","version":"235'551","reported_seq":4285,"reported_epoch":238,"state":"active+clean","last_fresh":"2024-09-14T06:32:43.621206+0000","last_change":"2024-09-14T06:32:43.621434+0000","last_active":"2024-09-14T06:32:43.621206+0000","last_peered":"2024-09-14T06:32:43.621206+0000","last_clean":"2024-09-14T06:32:43.621206+0000","last_became_active":"2024-09-14T06:29:00.573179+0000","last_became_peered":"2024-09-14T06:29:00.573179+0000","last_unstale":"2024-09-14T06:32:43.621206+0000","last_undegraded":"2024-09-14T06:32:43.621206+0000","last_fullsized":"2024-09-14T06:32:43.621206+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":551,"log_dups_size":0,"ondisk_log_size":551,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T06:40:38.359137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000498223,"stat_sum":{"num_bytes":18711878,"num_objects":42,"num_object_clones":0,"num_object_copies":84,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":42,"num_whiteouts":0,"num_read":3678,"num_read_kb":64291,"num_write":736,"num_write_kb":44928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"27","length":"1"},{"start":"29","length":"1"},{"start":"2b","length":"1"},{"start":"2d","length":"1"},{"start":"2f","length":"1"},{"start":"31","length":"1"},{"start":"33","length":"1"}]},{"pgid":"3.a","version":"238'813","reported_seq":4810,"reported_epoch":238,"state":"active+clean","last_fresh":"2024-09-14T06:32:43.621730+0000","last_change":"2024-09-14T06:32:43.621730+0000","last_active":"2024-09-14T06:32:43.621730+0000","last_peered":"2024-09-14T06:32:43.621730+0000","last_clean":"2024-09-14T06:32:43.621730+0000","last_became_active":"2024-09-14T06:32:04.617283+0000","last_became_peered":"2024-09-14T06:32:04.617283+0000","last_unstale":"2024-09-14T06:32:43.621730+0000","last_undegraded":"2024-09-14T06:32:43.621730+0000","last_fullsized":"2024-09-14T06:32:43.621730+0000","mapping_epoch":199,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":200,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":813,"log_dups_size":0,"ondisk_log_size":813,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T17:18:32.137675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077279300000000003,"stat_sum":{"num_bytes":37097096,"num_objects":36,"num_object_clones":0,"num_object_copies":72,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":36,"num_whiteouts":0,"num_read":5155,"num_read_kb":58177,"num_write":1169,"num_write_kb":47529,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":161,"num_bytes_recovered":21723371,"num_keys_recovered":33,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"27","length":"1"},{"start":"29","length":"1"},{"start":"2b","length":"1"},{"start":"2d","length":"1"},{"start":"2f","length":"1"},{"start":"31","length":"1"},{"start":"33","length":"1"}]},{"pgid":"3.b","version":"235'1551","reported_seq":8610,"reported_epoch":235,"state":"active+clean","last_fresh":"2024-09-14T06:32:40.629928+0000","last_change":"2024-09-14T06:32:40.629928+0000","last_active":"2024-09-14T06:32:40.629928+0000","last_peered":"2024-09-14T06:32:40.629928+0000","last_clean":"2024-09-14T06:32:40.629928+0000","last_became_active":"2024-09-14T06:31:55.756219+0000","last_became_peered":"2024-09-14T06:31:55.756219+0000","last_unstale":"2024-09-14T06:32:40.629928+0000","last_undegraded":"2024-09-14T06:32:40.629928+0000","last_fullsized":"2024-09-14T06:32:40.629928+0000","mapping_epoch":190,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":191,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":1551,"log_dups_size":0,"ondisk_log_size":1551,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T17:34:08.601295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013340330000000001,"stat_sum":{"num_bytes":21757058,"num_objects":41,"num_object_clones":0,"num_object_copies":82,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":41,"num_whiteouts":0,"num_read":12802,"num_read_kb":63928,"num_write":2413,"num_write_kb":43663,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":291,"num_bytes_recovered":37370632,"num_keys_recovered":66,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2d","length":"1"},{"start":"2f","length":"1"},{"start":"31","length":"1"},{"start":"33","length":"1"}]},{"pgid":"3.c","version":"235'573","reported_seq":6079,"reported_epoch":235,"state":"active+clean","last_fresh":"2024-09-14T06:32:40.746714+0000","last_change":"2024-09-14T06:32:40.627600+0000","last_active":"2024-09-14T06:32:40.746714+0000","last_peered":"2024-09-14T06:32:40.746714+0000","last_clean":"2024-09-14T06:32:40.746714+0000","last_became_active":"2024-09-14T06:32:24.716421+0000","last_became_peered":"2024-09-14T06:32:24.716421+0000","last_unstale":"2024-09-14T06:32:40.746714+0000","last_undegraded":"2024-09-14T06:32:40.746714+0000","last_fullsized":"2024-09-14T06:32:40.746714+0000","mapping_epoch":218,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":219,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":573,"log_dups_size":0,"ondisk_log_size":573,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T13:12:43.583648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061805300000000003,"stat_sum":{"num_bytes":26190419,"num_objects":36,"num_object_clones":0,"num_object_copies":72,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":36,"num_whiteouts":0,"num_read":4817,"num_read_kb":50405,"num_write":748,"num_write_kb":40823,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":293,"num_bytes_recovered":36712421,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2d","length":"1"},{"start":"2f","length":"1"},{"start":"31","length":"1"},{"start":"33","length":"1"}]},{"pgid":"3.d","version":"238'518","reported_seq":9007,"reported_epoch":238,"state":"active+clean","last_fresh":"2024-09-14T06:32:43.699130+0000","last_change":"2024-09-14T06:32:43.622462+0000","last_active":"2024-09-14T06:32:43.699130+0000","last_peered":"2024-09-14T06:32:43.699130+0000","last_clean":"2024-09-14T06:32:43.699130+0000","last_became_active":"2024-09-14T06:29:00.580362+0000","last_became_peered":"2024-09-14T06:29:00.580362+0000","last_unstale":"2024-09-14T06:32:43.699130+0000","last_undegraded":"2024-09-14T06:32:43.699130+0000","last_fullsized":"2024-09-14T06:32:43.699130+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":518,"log_dups_size":0,"ondisk_log_size":518,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T08:43:46.800196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00084532499999999996,"stat_sum":{"num_bytes":27519222,"num_objects":37,"num_object_clones":0,"num_object_copies":74,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":37,"num_whiteouts":0,"num_read":8026,"num_read_kb":93543,"num_write":679,"num_write_kb":41026,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"27","length":"1"},{"start":"29","length":"1"},{"start":"2b","length":"1"},{"start":"2d","length":"1"},{"start":"2f","length":"1"},{"start":"31","length":"1"},{"start":"33","length":"1"}]},{"pgid":"3.e","version":"238'544","reported_seq":5291,"reported_epoch":238,"state":"active+clean","last_fresh":"2024-09-14T06:32:43.623493+0000","last_change":"2024-09-14T06:32:43.623493+0000","last_active":"2024-09-14T06:32:43.623493+0000","last_peered":"2024-09-14T06:32:43.623493+0000","last_clean":"2024-09-14T06:32:43.623493+0000","last_became_active":"2024-09-14T06:32:39.777524+0000","last_became_peered":"2024-09-14T06:32:39.777524+0000","last_unstale":"2024-09-14T06:32:43.623493+0000","last_undegraded":"2024-09-14T06:32:43.623493+0000","last_fullsized":"2024-09-14T06:32:43.623493+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":544,"log_dups_size":0,"ondisk_log_size":544,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T17:41:20.292335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001295593,"stat_sum":{"num_bytes":14958569,"num_objects":39,"num_object_clones":0,"num_object_copies":78,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":39,"num_whiteouts":0,"num_read":3982,"num_read_kb":61136,"num_write":714,"num_write_kb":43787,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":171,"num_bytes_recovered":10913227,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"27","length":"1"},{"start":"29","length":"1"},{"start":"2b","length":"1"},{"start":"2d","length":"1"},{"start":"2f","length":"1"},{"start":"31","length":"1"},{"start":"33","length":"1"}]},{"pgid":"3.f","version":"238'687","reported_seq":5729,"reported_epoch":238,"state":"active+clean","last_fresh":"2024-09-14T06:32:43.714749+0000","last_change":"2024-09-14T06:32:43.623386+0000","last_active":"2024-09-14T06:32:43.714749+0000","last_peered":"2024-09-14T06:32:43.714749+0000","last_clean":"2024-09-14T06:32:43.714749+0000","last_became_active":"2024-09-14T06:29:00.586491+0000","last_became_peered":"2024-09-14T06:29:00.586491+0000","last_unstale":"2024-09-14T06:32:43.714749+0000","last_undegraded":"2024-09-14T06:32:43.714749+0000","last_fullsized":"2024-09-14T06:32:43.714749+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":687,"log_dups_size":0,"ondisk_log_size":687,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T13:24:23.935392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014244769999999999,"stat_sum":{"num_bytes":17046942,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":6076,"num_read_kb":45576,"num_write":813,"num_write_kb":36134,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"27","length":"1"},{"start":"29","length":"1"},{"start":"2b","length":"1"},{"start":"2d","length":"1"},{"start":"2f","length":"1"},{"start":"31","length":"1"},{"start":"33","length":"1"}]},{"pgid":"3.10","version":"236'899","reported_seq":6563,"reported_epoch":237,"state":"active+clean","last_fresh":"2024-09-14T06:32:42.620400+0000","last_change":"2024-09-14T06:32:42.620647+0000","last_active":"2024-09-14T06:32:42.620400+0000","last_peered":"2024-09-14T06:32:42.620400+0000","last_clean":"2024-09-14T06:32:42.620400+0000","last_became_active":"2024-09-14T06:32:04.621617+0000","last_became_peered":"2024-09-14T06:32:04.621617+0000","last_unstale":"2024-09-14T06:32:42.620400+0000","last_undegraded":"2024-09-14T06:32:42.620400+0000","last_fullsized":"2024-09-14T06:32:42.620400+0000","mapping_epoch":199,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":200,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":899,"log_dups_size":0,"ondisk_log_size":899,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T16:44:12.269688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051763799999999995,"stat_sum":{"num_bytes":18989605,"num_objects":29,"num_object_clones":0,"num_object_copies":58,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":29,"num_whiteouts":0,"num_read":6970,"num_read_kb":34321,"num_write":1037,"num_write_kb":36649,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":154,"num_bytes_recovered":9401812,"num_keys_recovered":45,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"29","length":"1"},{"start":"2b","length":"1"},{"start":"2d","length":"1"},{"start":"2f","length":"1"},{"start":"31","length":"1"},{"start":"33","length":"1"}]},{"pgid":"3.11","version":"238'660","reported_seq":11478,"reported_epoch":238,"state":"active+clean","last_fresh":"2024-09-14T06:32:43.696263+0000","last_change":"2024-09-14T06:32:43.622647+0000","last_active":"2024-09-14T06:32:43.696263+0000","last_peered":"2024-09-14T06:32:43.696263+0000","last_clean":"2024-09-14T06:32:43.696263+0000","last_became_active":"2024-09-14T06:32:39.778010+0000","last_became_peered":"2024-09-14T06:32:39.778010+0000","last_unstale":"2024-09-14T06:32:43.696263+0000","last_undegraded":"2024-09-14T06:32:43.696263+0000","last_fullsized":"2024-09-14T06:32:43.696263+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":660,"log_dups_size":0,"ondisk_log_size":660,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T09:41:48.034081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076344800000000001,"stat_sum":{"num_bytes":50834137,"num_objects":44,"num_object_clones":0,"num_object_copies":88,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":44,"num_whiteouts":0,"num_read":10209,"num_read_kb":109195,"num_write":789,"num_write_kb":48297,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":177,"num_bytes_recovered":16647747,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"27","length":"1"},{"start":"29","length":"1"},{"start":"2b","length":"1"},{"start":"2d","length":"1"},{"start":"2f","length":"1"},{"start":"31","length":"1"},{"start":"33","length":"1"}]},{"pgid":"3.12","version":"237'695","reported_seq":7144,"reported_epoch":237,"state":"active+clean","last_fresh":"2024-09-14T06:32:42.620909+0000","last_change":"2024-09-14T06:32:42.620909+0000","last_active":"2024-09-14T06:32:42.620909+0000","last_peered":"2024-09-14T06:32:42.620909+0000","last_clean":"2024-09-14T06:32:42.620909+0000","last_became_active":"2024-09-14T06:29:00.579826+0000","last_became_peered":"2024-09-14T06:29:00.579826+0000","last_unstale":"2024-09-14T06:32:42.620909+0000","last_undegraded":"2024-09-14T06:32:42.620909+0000","last_fullsized":"2024-09-14T06:32:42.620909+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":695,"log_dups_size":0,"ondisk_log_size":695,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T18:25:05.755525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00075932500000000004,"stat_sum":{"num_bytes":24818516,"num_objects":42,"num_object_clones":0,"num_object_copies":84,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":42,"num_whiteouts":0,"num_read":6875,"num_read_kb":58890,"num_write":863,"num_write_kb":42425,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"29","length":"1"},{"start":"2b","length":"1"},{"start":"2d","length":"1"},{"start":"2f","length":"1"},{"start":"31","length":"1"},{"start":"33","length":"1"}]},{"pgid":"3.13","version":"238'970","reported_seq":8728,"reported_epoch":238,"state":"active+clean","last_fresh":"2024-09-14T06:32:43.704236+0000","last_change":"2024-09-14T06:32:43.622369+0000","last_active":"2024-09-14T06:32:43.704236+0000","last_peered":"2024-09-14T06:32:43.704236+0000","last_clean":"2024-09-14T06:32:43.704236+0000","last_became_active":"2024-09-14T06:29:00.586520+0000","last_became_peered":"2024-09-14T06:29:00.586520+0000","last_unstale":"2024-09-14T06:32:43.704236+0000","last_undegraded":"2024-09-14T06:32:43.704236+0000","last_fullsized":"2024-09-14T06:32:43.704236+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":970,"log_dups_size":0,"ondisk_log_size":970,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T12:28:10.034097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00082331099999999996,"stat_sum":{"num_bytes":40273349,"num_objects":34,"num_object_clones":0,"num_object_copies":68,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":34,"num_whiteouts":0,"num_read":9546,"num_read_kb":66117,"num_write":1201,"num_write_kb":47437,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"27","length":"1"},{"start":"29","length":"1"},{"start":"2b","length":"1"},{"start":"2d","length":"1"},{"start":"2f","length":"1"},{"start":"31","length":"1"},{"start":"33","length":"1"}]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":937313759,"num_objects":1219,"num_object_clones":0,"num_object_copies":2438,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1219,"num_whiteouts":0,"num_read":235994,"num_read_kb":2181417,"num_write":36277,"num_write_kb":1478126,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5149,"num_bytes_recovered":633091608,"num_keys_recovered":534,"num_objects_omap":20,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":74141696,"data_stored":736743878,"data_compressed":44801348,"data_compressed_allocated":59424768,"data_compressed_original":725188858,"omap_allocated":0,"internal_metadata":0},"log_size":26638,"ondisk_log_size":26638,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":131072,"data_stored":1180736,"data_compressed":65166,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509491,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":75096,"kb_used_data":11968,"kb_used_omap":1,"kb_used_meta":63102,"kb_avail":94296744,"statfs":{"total":96636764160,"available":96559865856,"internally_reserved":0,"allocated":12255232,"data_stored":93297315,"data_compressed":5468001,"data_compressed_allocated":7200768,"data_compressed_original":90126128,"omap_allocated":1590,"internal_metadata":64616906},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":0,"seq":64424509491,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509492,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":79612,"kb_used_data":12316,"kb_used_omap":1,"kb_used_meta":67262,"kb_avail":94292228,"statfs":{"total":96636764160,"available":96555241472,"internally_reserved":0,"allocated":12611584,"data_stored":91841273,"data_compressed":5557659,"data_compressed_allocated":7454720,"data_compressed_original":88637452,"omap_allocated":1595,"internal_metadata":68876741},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509491,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":88432,"kb_used_data":12828,"kb_used_omap":1,"kb_used_meta":75582,"kb_avail":94283408,"statfs":{"total":96636764160,"available":96546209792,"internally_reserved":0,"allocated":13135872,"data_stored":106616304,"data_compressed":6468186,"data_compressed_allocated":8470528,"data_compressed_original":103846041,"omap_allocated":1595,"internal_metadata":77396421},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509492,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":63672,"kb_used_data":14436,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94308168,"statfs":{"total":96636764160,"available":96571564032,"internally_reserved":0,"allocated":14782464,"data_stored":121071457,"data_compressed":7493926,"data_compressed_allocated":9818112,"data_compressed_original":118051925,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":14,"seq":60129542195,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":38940,"kb_used_data":7744,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94332900,"statfs":{"total":96636764160,"available":96596889600,"internally_reserved":0,"allocated":7929856,"data_stored":54710491,"data_compressed":3152243,"data_compressed_allocated":4280320,"data_compressed_original":52716327,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542194,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":84128,"kb_used_data":12556,"kb_used_omap":1,"kb_used_meta":71422,"kb_avail":94287712,"statfs":{"total":96636764160,"available":96550617088,"internally_reserved":0,"allocated":12857344,"data_stored":96790161,"data_compressed":5570864,"data_compressed_allocated":7372800,"data_compressed_original":93240828,"omap_allocated":1595,"internal_metadata":73136581},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574899,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":90316,"kb_used_data":14712,"kb_used_omap":1,"kb_used_meta":75582,"kb_avail":94281524,"statfs":{"total":96636764160,"available":96544280576,"internally_reserved":0,"allocated":15065088,"data_stored":122917142,"data_compressed":7640073,"data_compressed_allocated":10076160,"data_compressed_original":119828480,"omap_allocated":1593,"internal_metadata":77396423},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32583,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32583,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":9744384,"data_stored":94574967,"data_compressed":5535176,"data_compressed_allocated":7319552,"data_compressed_original":92601852,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":11714560,"data_stored":118758735,"data_compressed":7407455,"data_compressed_allocated":9711616,"data_compressed_original":117216341,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":5058560,"data_stored":53289916,"data_compressed":3151615,"data_compressed_allocated":4276224,"data_compressed_original":52704039,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12222464,"data_stored":121506055,"data_compressed":7639096,"data_compressed_allocated":10072064,"data_compressed_original":119808000,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":10072064,"data_stored":103857913,"data_compressed":6344819,"data_compressed_allocated":8323072,"data_compressed_original":102531442,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":9555968,"data_stored":89273137,"data_compressed":5452184,"data_compressed_allocated":7327744,"data_compressed_original":87519244,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":6631424,"data_stored":65369250,"data_compressed":3936333,"data_compressed_allocated":5361664,"data_compressed_original":64393940,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":9142272,"data_stored":90113905,"data_compressed":5334670,"data_compressed_allocated":7032832,"data_compressed_original":88414000,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-14T06:32:46.407 INFO:tasks.ceph.ceph_manager.ceph:recovered! 2024-09-14T06:32:46.407 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-14T06:32:46.407 INFO:tasks.thrashosds.thrasher:thrash_pg_upmap_items 2024-09-14T06:32:46.407 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump -f json-pretty 2024-09-14T06:32:46.419 INFO:tasks.ceph.osd.5.smithi138.stderr:2024-09-14T06:32:46.418+0000 7fe13746f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33618) UID: 0 2024-09-14T06:32:46.478 INFO:teuthology.orchestra.run.smithi138.stdout:ERROR: (22) Invalid argument 2024-09-14T06:32:46.478 INFO:teuthology.orchestra.run.smithi138.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:32:46.487 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-14T06:32:46.528 INFO:tasks.ceph.osd.5.smithi138.stderr:2024-09-14T06:32:46.527+0000 7fe13746f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33618) UID: 0 2024-09-14T06:32:46.596 INFO:tasks.ceph.osd.7.smithi138.stderr:2024-09-14T06:32:46.595+0000 7f3c6a21d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33623) UID: 0 2024-09-14T06:32:46.635 INFO:teuthology.orchestra.run.smithi138.stdout:ERROR: (22) Invalid argument 2024-09-14T06:32:46.635 INFO:teuthology.orchestra.run.smithi138.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:32:46.644 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-14T06:32:46.729 INFO:tasks.ceph.osd.6.smithi138.stderr:2024-09-14T06:32:46.728+0000 7f8cd8849640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33620) UID: 0 2024-09-14T06:32:46.770 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-14T06:32:46.771 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-14T06:32:46.771 INFO:teuthology.orchestra.run.smithi094.stdout: "epoch": 241, 2024-09-14T06:32:46.771 INFO:teuthology.orchestra.run.smithi094.stdout: "fsid": "200ad3da-bd63-4e04-af74-efb5ca0af75e", 2024-09-14T06:32:46.771 INFO:teuthology.orchestra.run.smithi094.stdout: "created": "2024-09-14T06:28:30.782046+0000", 2024-09-14T06:32:46.771 INFO:teuthology.orchestra.run.smithi094.stdout: "modified": "2024-09-14T06:32:46.612244+0000", 2024-09-14T06:32:46.771 INFO:teuthology.orchestra.run.smithi094.stdout: "last_up_change": "2024-09-14T06:28:42.827239+0000", 2024-09-14T06:32:46.771 INFO:teuthology.orchestra.run.smithi094.stdout: "last_in_change": "2024-09-14T06:32:00.601027+0000", 2024-09-14T06:32:46.771 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2024-09-14T06:32:46.771 INFO:teuthology.orchestra.run.smithi094.stdout: "flags_num": 5799936, 2024-09-14T06:32:46.771 INFO:teuthology.orchestra.run.smithi094.stdout: "flags_set": [ 2024-09-14T06:32:46.771 INFO:teuthology.orchestra.run.smithi094.stdout: "pglog_hardlimit", 2024-09-14T06:32:46.771 INFO:teuthology.orchestra.run.smithi094.stdout: "purged_snapdirs", 2024-09-14T06:32:46.771 INFO:teuthology.orchestra.run.smithi094.stdout: "recovery_deletes", 2024-09-14T06:32:46.771 INFO:teuthology.orchestra.run.smithi094.stdout: "sortbitwise" 2024-09-14T06:32:46.772 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-09-14T06:32:46.772 INFO:teuthology.orchestra.run.smithi094.stdout: "crush_version": 5, 2024-09-14T06:32:46.772 INFO:teuthology.orchestra.run.smithi094.stdout: "full_ratio": 0.94999998807907104, 2024-09-14T06:32:46.772 INFO:teuthology.orchestra.run.smithi094.stdout: "backfillfull_ratio": 0.89999997615814209, 2024-09-14T06:32:46.772 INFO:teuthology.orchestra.run.smithi094.stdout: "nearfull_ratio": 0.85000002384185791, 2024-09-14T06:32:46.772 INFO:teuthology.orchestra.run.smithi094.stdout: "cluster_snapshot": "", 2024-09-14T06:32:46.772 INFO:teuthology.orchestra.run.smithi094.stdout: "pool_max": 3, 2024-09-14T06:32:46.772 INFO:teuthology.orchestra.run.smithi094.stdout: "max_osd": 8, 2024-09-14T06:32:46.772 INFO:teuthology.orchestra.run.smithi094.stdout: "require_min_compat_client": "luminous", 2024-09-14T06:32:46.772 INFO:teuthology.orchestra.run.smithi094.stdout: "min_compat_client": "luminous", 2024-09-14T06:32:46.772 INFO:teuthology.orchestra.run.smithi094.stdout: "require_osd_release": "squid", 2024-09-14T06:32:46.772 INFO:teuthology.orchestra.run.smithi094.stdout: "allow_crimson": false, 2024-09-14T06:32:46.772 INFO:teuthology.orchestra.run.smithi094.stdout: "pools": [ 2024-09-14T06:32:46.772 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:46.773 INFO:teuthology.orchestra.run.smithi094.stdout: "pool": 1, 2024-09-14T06:32:46.773 INFO:teuthology.orchestra.run.smithi094.stdout: "pool_name": ".mgr", 2024-09-14T06:32:46.773 INFO:teuthology.orchestra.run.smithi094.stdout: "create_time": "2024-09-14T06:28:43.295990+0000", 2024-09-14T06:32:46.773 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-14T06:32:46.773 INFO:teuthology.orchestra.run.smithi094.stdout: "flags_names": "hashpspool", 2024-09-14T06:32:46.773 INFO:teuthology.orchestra.run.smithi094.stdout: "type": 1, 2024-09-14T06:32:46.773 INFO:teuthology.orchestra.run.smithi094.stdout: "size": 2, 2024-09-14T06:32:46.773 INFO:teuthology.orchestra.run.smithi094.stdout: "min_size": 1, 2024-09-14T06:32:46.773 INFO:teuthology.orchestra.run.smithi094.stdout: "crush_rule": 0, 2024-09-14T06:32:46.773 INFO:teuthology.orchestra.run.smithi094.stdout: "peering_crush_bucket_count": 0, 2024-09-14T06:32:46.773 INFO:teuthology.orchestra.run.smithi094.stdout: "peering_crush_bucket_target": 0, 2024-09-14T06:32:46.773 INFO:teuthology.orchestra.run.smithi094.stdout: "peering_crush_bucket_barrier": 0, 2024-09-14T06:32:46.773 INFO:teuthology.orchestra.run.smithi094.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-14T06:32:46.774 INFO:teuthology.orchestra.run.smithi094.stdout: "is_stretch_pool": false, 2024-09-14T06:32:46.774 INFO:teuthology.orchestra.run.smithi094.stdout: "object_hash": 2, 2024-09-14T06:32:46.774 INFO:teuthology.orchestra.run.smithi094.stdout: "pg_autoscale_mode": "off", 2024-09-14T06:32:46.774 INFO:teuthology.orchestra.run.smithi094.stdout: "pg_num": 1, 2024-09-14T06:32:46.774 INFO:teuthology.orchestra.run.smithi094.stdout: "pg_placement_num": 1, 2024-09-14T06:32:46.774 INFO:teuthology.orchestra.run.smithi094.stdout: "pg_placement_num_target": 1, 2024-09-14T06:32:46.774 INFO:teuthology.orchestra.run.smithi094.stdout: "pg_num_target": 1, 2024-09-14T06:32:46.774 INFO:teuthology.orchestra.run.smithi094.stdout: "pg_num_pending": 1, 2024-09-14T06:32:46.774 INFO:teuthology.orchestra.run.smithi094.stdout: "last_pg_merge_meta": { 2024-09-14T06:32:46.774 INFO:teuthology.orchestra.run.smithi094.stdout: "source_pgid": "0.0", 2024-09-14T06:32:46.774 INFO:teuthology.orchestra.run.smithi094.stdout: "ready_epoch": 0, 2024-09-14T06:32:46.774 INFO:teuthology.orchestra.run.smithi094.stdout: "last_epoch_started": 0, 2024-09-14T06:32:46.774 INFO:teuthology.orchestra.run.smithi094.stdout: "last_epoch_clean": 0, 2024-09-14T06:32:46.774 INFO:teuthology.orchestra.run.smithi094.stdout: "source_version": "0'0", 2024-09-14T06:32:46.775 INFO:teuthology.orchestra.run.smithi094.stdout: "target_version": "0'0" 2024-09-14T06:32:46.775 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:46.775 INFO:teuthology.orchestra.run.smithi094.stdout: "last_change": "18", 2024-09-14T06:32:46.775 INFO:teuthology.orchestra.run.smithi094.stdout: "last_force_op_resend": "0", 2024-09-14T06:32:46.775 INFO:teuthology.orchestra.run.smithi094.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-14T06:32:46.775 INFO:teuthology.orchestra.run.smithi094.stdout: "last_force_op_resend_preluminous": "0", 2024-09-14T06:32:46.775 INFO:teuthology.orchestra.run.smithi094.stdout: "auid": 0, 2024-09-14T06:32:46.775 INFO:teuthology.orchestra.run.smithi094.stdout: "snap_mode": "selfmanaged", 2024-09-14T06:32:46.775 INFO:teuthology.orchestra.run.smithi094.stdout: "snap_seq": 0, 2024-09-14T06:32:46.775 INFO:teuthology.orchestra.run.smithi094.stdout: "snap_epoch": 0, 2024-09-14T06:32:46.775 INFO:teuthology.orchestra.run.smithi094.stdout: "pool_snaps": [], 2024-09-14T06:32:46.775 INFO:teuthology.orchestra.run.smithi094.stdout: "removed_snaps": "[]", 2024-09-14T06:32:46.775 INFO:teuthology.orchestra.run.smithi094.stdout: "quota_max_bytes": 0, 2024-09-14T06:32:46.775 INFO:teuthology.orchestra.run.smithi094.stdout: "quota_max_objects": 0, 2024-09-14T06:32:46.775 INFO:teuthology.orchestra.run.smithi094.stdout: "tiers": [], 2024-09-14T06:32:46.776 INFO:teuthology.orchestra.run.smithi094.stdout: "tier_of": -1, 2024-09-14T06:32:46.776 INFO:teuthology.orchestra.run.smithi094.stdout: "read_tier": -1, 2024-09-14T06:32:46.776 INFO:teuthology.orchestra.run.smithi094.stdout: "write_tier": -1, 2024-09-14T06:32:46.776 INFO:teuthology.orchestra.run.smithi094.stdout: "cache_mode": "none", 2024-09-14T06:32:46.776 INFO:teuthology.orchestra.run.smithi094.stdout: "target_max_bytes": 0, 2024-09-14T06:32:46.776 INFO:teuthology.orchestra.run.smithi094.stdout: "target_max_objects": 0, 2024-09-14T06:32:46.776 INFO:teuthology.orchestra.run.smithi094.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-14T06:32:46.776 INFO:teuthology.orchestra.run.smithi094.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-14T06:32:46.776 INFO:teuthology.orchestra.run.smithi094.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-14T06:32:46.776 INFO:teuthology.orchestra.run.smithi094.stdout: "cache_min_flush_age": 0, 2024-09-14T06:32:46.777 INFO:teuthology.orchestra.run.smithi094.stdout: "cache_min_evict_age": 0, 2024-09-14T06:32:46.777 INFO:teuthology.orchestra.run.smithi094.stdout: "erasure_code_profile": "", 2024-09-14T06:32:46.777 INFO:teuthology.orchestra.run.smithi094.stdout: "hit_set_params": { 2024-09-14T06:32:46.777 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "none" 2024-09-14T06:32:46.777 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:46.777 INFO:teuthology.orchestra.run.smithi094.stdout: "hit_set_period": 0, 2024-09-14T06:32:46.777 INFO:teuthology.orchestra.run.smithi094.stdout: "hit_set_count": 0, 2024-09-14T06:32:46.777 INFO:teuthology.orchestra.run.smithi094.stdout: "use_gmt_hitset": true, 2024-09-14T06:32:46.778 INFO:teuthology.orchestra.run.smithi094.stdout: "min_read_recency_for_promote": 0, 2024-09-14T06:32:46.778 INFO:teuthology.orchestra.run.smithi094.stdout: "min_write_recency_for_promote": 0, 2024-09-14T06:32:46.778 INFO:teuthology.orchestra.run.smithi094.stdout: "hit_set_grade_decay_rate": 0, 2024-09-14T06:32:46.778 INFO:teuthology.orchestra.run.smithi094.stdout: "hit_set_search_last_n": 0, 2024-09-14T06:32:46.778 INFO:teuthology.orchestra.run.smithi094.stdout: "grade_table": [], 2024-09-14T06:32:46.778 INFO:teuthology.orchestra.run.smithi094.stdout: "stripe_width": 0, 2024-09-14T06:32:46.778 INFO:teuthology.orchestra.run.smithi094.stdout: "expected_num_objects": 0, 2024-09-14T06:32:46.778 INFO:teuthology.orchestra.run.smithi094.stdout: "fast_read": false, 2024-09-14T06:32:46.778 INFO:teuthology.orchestra.run.smithi094.stdout: "options": { 2024-09-14T06:32:46.778 INFO:teuthology.orchestra.run.smithi094.stdout: "pg_num_max": 32, 2024-09-14T06:32:46.778 INFO:teuthology.orchestra.run.smithi094.stdout: "pg_num_min": 1 2024-09-14T06:32:46.778 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:46.778 INFO:teuthology.orchestra.run.smithi094.stdout: "application_metadata": { 2024-09-14T06:32:46.779 INFO:teuthology.orchestra.run.smithi094.stdout: "mgr": {} 2024-09-14T06:32:46.779 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:46.779 INFO:teuthology.orchestra.run.smithi094.stdout: "read_balance": { 2024-09-14T06:32:46.779 INFO:teuthology.orchestra.run.smithi094.stdout: "score_type": "Fair distribution", 2024-09-14T06:32:46.779 INFO:teuthology.orchestra.run.smithi094.stdout: "score_acting": 6.9000000953674316, 2024-09-14T06:32:46.779 INFO:teuthology.orchestra.run.smithi094.stdout: "score_stable": 6.9000000953674316, 2024-09-14T06:32:46.779 INFO:teuthology.orchestra.run.smithi094.stdout: "optimal_score": 0.28999999165534973, 2024-09-14T06:32:46.779 INFO:teuthology.orchestra.run.smithi094.stdout: "raw_score_acting": 2, 2024-09-14T06:32:46.779 INFO:teuthology.orchestra.run.smithi094.stdout: "raw_score_stable": 2, 2024-09-14T06:32:46.779 INFO:teuthology.orchestra.run.smithi094.stdout: "primary_affinity_weighted": 0.85513687133789062, 2024-09-14T06:32:46.779 INFO:teuthology.orchestra.run.smithi094.stdout: "average_primary_affinity": 0.98000001907348633, 2024-09-14T06:32:46.779 INFO:teuthology.orchestra.run.smithi094.stdout: "average_primary_affinity_weighted": 0.86000001430511475 2024-09-14T06:32:46.779 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:46.779 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:46.779 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:46.780 INFO:teuthology.orchestra.run.smithi094.stdout: "pool": 2, 2024-09-14T06:32:46.780 INFO:teuthology.orchestra.run.smithi094.stdout: "pool_name": "rbd", 2024-09-14T06:32:46.780 INFO:teuthology.orchestra.run.smithi094.stdout: "create_time": "2024-09-14T06:28:49.451714+0000", 2024-09-14T06:32:46.780 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 8193, 2024-09-14T06:32:46.780 INFO:teuthology.orchestra.run.smithi094.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-09-14T06:32:46.780 INFO:teuthology.orchestra.run.smithi094.stdout: "type": 1, 2024-09-14T06:32:46.780 INFO:teuthology.orchestra.run.smithi094.stdout: "size": 2, 2024-09-14T06:32:46.780 INFO:teuthology.orchestra.run.smithi094.stdout: "min_size": 1, 2024-09-14T06:32:46.780 INFO:teuthology.orchestra.run.smithi094.stdout: "crush_rule": 0, 2024-09-14T06:32:46.780 INFO:teuthology.orchestra.run.smithi094.stdout: "peering_crush_bucket_count": 0, 2024-09-14T06:32:46.780 INFO:teuthology.orchestra.run.smithi094.stdout: "peering_crush_bucket_target": 0, 2024-09-14T06:32:46.780 INFO:teuthology.orchestra.run.smithi094.stdout: "peering_crush_bucket_barrier": 0, 2024-09-14T06:32:46.780 INFO:teuthology.orchestra.run.smithi094.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-14T06:32:46.781 INFO:teuthology.orchestra.run.smithi094.stdout: "is_stretch_pool": false, 2024-09-14T06:32:46.781 INFO:teuthology.orchestra.run.smithi094.stdout: "object_hash": 2, 2024-09-14T06:32:46.781 INFO:teuthology.orchestra.run.smithi094.stdout: "pg_autoscale_mode": "off", 2024-09-14T06:32:46.781 INFO:teuthology.orchestra.run.smithi094.stdout: "pg_num": 8, 2024-09-14T06:32:46.781 INFO:teuthology.orchestra.run.smithi094.stdout: "pg_placement_num": 8, 2024-09-14T06:32:46.781 INFO:teuthology.orchestra.run.smithi094.stdout: "pg_placement_num_target": 8, 2024-09-14T06:32:46.781 INFO:teuthology.orchestra.run.smithi094.stdout: "pg_num_target": 8, 2024-09-14T06:32:46.781 INFO:teuthology.orchestra.run.smithi094.stdout: "pg_num_pending": 8, 2024-09-14T06:32:46.781 INFO:teuthology.orchestra.run.smithi094.stdout: "last_pg_merge_meta": { 2024-09-14T06:32:46.781 INFO:teuthology.orchestra.run.smithi094.stdout: "source_pgid": "0.0", 2024-09-14T06:32:46.781 INFO:teuthology.orchestra.run.smithi094.stdout: "ready_epoch": 0, 2024-09-14T06:32:46.781 INFO:teuthology.orchestra.run.smithi094.stdout: "last_epoch_started": 0, 2024-09-14T06:32:46.781 INFO:teuthology.orchestra.run.smithi094.stdout: "last_epoch_clean": 0, 2024-09-14T06:32:46.781 INFO:teuthology.orchestra.run.smithi094.stdout: "source_version": "0'0", 2024-09-14T06:32:46.782 INFO:teuthology.orchestra.run.smithi094.stdout: "target_version": "0'0" 2024-09-14T06:32:46.782 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:46.782 INFO:teuthology.orchestra.run.smithi094.stdout: "last_change": "22", 2024-09-14T06:32:46.782 INFO:teuthology.orchestra.run.smithi094.stdout: "last_force_op_resend": "0", 2024-09-14T06:32:46.782 INFO:teuthology.orchestra.run.smithi094.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-14T06:32:46.782 INFO:teuthology.orchestra.run.smithi094.stdout: "last_force_op_resend_preluminous": "0", 2024-09-14T06:32:46.782 INFO:teuthology.orchestra.run.smithi094.stdout: "auid": 0, 2024-09-14T06:32:46.782 INFO:teuthology.orchestra.run.smithi094.stdout: "snap_mode": "selfmanaged", 2024-09-14T06:32:46.782 INFO:teuthology.orchestra.run.smithi094.stdout: "snap_seq": 2, 2024-09-14T06:32:46.782 INFO:teuthology.orchestra.run.smithi094.stdout: "snap_epoch": 22, 2024-09-14T06:32:46.782 INFO:teuthology.orchestra.run.smithi094.stdout: "pool_snaps": [], 2024-09-14T06:32:46.782 INFO:teuthology.orchestra.run.smithi094.stdout: "removed_snaps": "[]", 2024-09-14T06:32:46.782 INFO:teuthology.orchestra.run.smithi094.stdout: "quota_max_bytes": 0, 2024-09-14T06:32:46.782 INFO:teuthology.orchestra.run.smithi094.stdout: "quota_max_objects": 0, 2024-09-14T06:32:46.783 INFO:teuthology.orchestra.run.smithi094.stdout: "tiers": [], 2024-09-14T06:32:46.783 INFO:teuthology.orchestra.run.smithi094.stdout: "tier_of": -1, 2024-09-14T06:32:46.783 INFO:teuthology.orchestra.run.smithi094.stdout: "read_tier": -1, 2024-09-14T06:32:46.783 INFO:teuthology.orchestra.run.smithi094.stdout: "write_tier": -1, 2024-09-14T06:32:46.783 INFO:teuthology.orchestra.run.smithi094.stdout: "cache_mode": "none", 2024-09-14T06:32:46.783 INFO:teuthology.orchestra.run.smithi094.stdout: "target_max_bytes": 0, 2024-09-14T06:32:46.783 INFO:teuthology.orchestra.run.smithi094.stdout: "target_max_objects": 0, 2024-09-14T06:32:46.783 INFO:teuthology.orchestra.run.smithi094.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-14T06:32:46.783 INFO:teuthology.orchestra.run.smithi094.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-14T06:32:46.783 INFO:teuthology.orchestra.run.smithi094.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-14T06:32:46.783 INFO:teuthology.orchestra.run.smithi094.stdout: "cache_min_flush_age": 0, 2024-09-14T06:32:46.784 INFO:teuthology.orchestra.run.smithi094.stdout: "cache_min_evict_age": 0, 2024-09-14T06:32:46.784 INFO:teuthology.orchestra.run.smithi094.stdout: "erasure_code_profile": "", 2024-09-14T06:32:46.784 INFO:teuthology.orchestra.run.smithi094.stdout: "hit_set_params": { 2024-09-14T06:32:46.784 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "none" 2024-09-14T06:32:46.784 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:46.784 INFO:teuthology.orchestra.run.smithi094.stdout: "hit_set_period": 0, 2024-09-14T06:32:46.784 INFO:teuthology.orchestra.run.smithi094.stdout: "hit_set_count": 0, 2024-09-14T06:32:46.784 INFO:teuthology.orchestra.run.smithi094.stdout: "use_gmt_hitset": true, 2024-09-14T06:32:46.784 INFO:teuthology.orchestra.run.smithi094.stdout: "min_read_recency_for_promote": 0, 2024-09-14T06:32:46.784 INFO:teuthology.orchestra.run.smithi094.stdout: "min_write_recency_for_promote": 0, 2024-09-14T06:32:46.784 INFO:teuthology.orchestra.run.smithi094.stdout: "hit_set_grade_decay_rate": 0, 2024-09-14T06:32:46.784 INFO:teuthology.orchestra.run.smithi094.stdout: "hit_set_search_last_n": 0, 2024-09-14T06:32:46.784 INFO:teuthology.orchestra.run.smithi094.stdout: "grade_table": [], 2024-09-14T06:32:46.785 INFO:teuthology.orchestra.run.smithi094.stdout: "stripe_width": 0, 2024-09-14T06:32:46.785 INFO:teuthology.orchestra.run.smithi094.stdout: "expected_num_objects": 0, 2024-09-14T06:32:46.785 INFO:teuthology.orchestra.run.smithi094.stdout: "fast_read": false, 2024-09-14T06:32:46.785 INFO:teuthology.orchestra.run.smithi094.stdout: "options": {}, 2024-09-14T06:32:46.785 INFO:teuthology.orchestra.run.smithi094.stdout: "application_metadata": { 2024-09-14T06:32:46.785 INFO:teuthology.orchestra.run.smithi094.stdout: "rbd": {} 2024-09-14T06:32:46.785 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:46.785 INFO:teuthology.orchestra.run.smithi094.stdout: "read_balance": { 2024-09-14T06:32:46.785 INFO:teuthology.orchestra.run.smithi094.stdout: "score_type": "Fair distribution", 2024-09-14T06:32:46.785 INFO:teuthology.orchestra.run.smithi094.stdout: "score_acting": 2.630000114440918, 2024-09-14T06:32:46.785 INFO:teuthology.orchestra.run.smithi094.stdout: "score_stable": 2.630000114440918, 2024-09-14T06:32:46.785 INFO:teuthology.orchestra.run.smithi094.stdout: "optimal_score": 1, 2024-09-14T06:32:46.785 INFO:teuthology.orchestra.run.smithi094.stdout: "raw_score_acting": 2.630000114440918, 2024-09-14T06:32:46.786 INFO:teuthology.orchestra.run.smithi094.stdout: "raw_score_stable": 2.630000114440918, 2024-09-14T06:32:46.786 INFO:teuthology.orchestra.run.smithi094.stdout: "primary_affinity_weighted": 0.85513687133789062, 2024-09-14T06:32:46.786 INFO:teuthology.orchestra.run.smithi094.stdout: "average_primary_affinity": 0.98000001907348633, 2024-09-14T06:32:46.786 INFO:teuthology.orchestra.run.smithi094.stdout: "average_primary_affinity_weighted": 0.86000001430511475 2024-09-14T06:32:46.786 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:46.786 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:46.786 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:46.786 INFO:teuthology.orchestra.run.smithi094.stdout: "pool": 3, 2024-09-14T06:32:46.786 INFO:teuthology.orchestra.run.smithi094.stdout: "pool_name": "pool_client.0", 2024-09-14T06:32:46.786 INFO:teuthology.orchestra.run.smithi094.stdout: "create_time": "2024-09-14T06:28:59.688535+0000", 2024-09-14T06:32:46.786 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 8193, 2024-09-14T06:32:46.786 INFO:teuthology.orchestra.run.smithi094.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-09-14T06:32:46.786 INFO:teuthology.orchestra.run.smithi094.stdout: "type": 1, 2024-09-14T06:32:46.786 INFO:teuthology.orchestra.run.smithi094.stdout: "size": 2, 2024-09-14T06:32:46.787 INFO:teuthology.orchestra.run.smithi094.stdout: "min_size": 1, 2024-09-14T06:32:46.787 INFO:teuthology.orchestra.run.smithi094.stdout: "crush_rule": 0, 2024-09-14T06:32:46.787 INFO:teuthology.orchestra.run.smithi094.stdout: "peering_crush_bucket_count": 0, 2024-09-14T06:32:46.787 INFO:teuthology.orchestra.run.smithi094.stdout: "peering_crush_bucket_target": 0, 2024-09-14T06:32:46.787 INFO:teuthology.orchestra.run.smithi094.stdout: "peering_crush_bucket_barrier": 0, 2024-09-14T06:32:46.787 INFO:teuthology.orchestra.run.smithi094.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-14T06:32:46.787 INFO:teuthology.orchestra.run.smithi094.stdout: "is_stretch_pool": false, 2024-09-14T06:32:46.787 INFO:teuthology.orchestra.run.smithi094.stdout: "object_hash": 2, 2024-09-14T06:32:46.787 INFO:teuthology.orchestra.run.smithi094.stdout: "pg_autoscale_mode": "off", 2024-09-14T06:32:46.787 INFO:teuthology.orchestra.run.smithi094.stdout: "pg_num": 32, 2024-09-14T06:32:46.787 INFO:teuthology.orchestra.run.smithi094.stdout: "pg_placement_num": 32, 2024-09-14T06:32:46.787 INFO:teuthology.orchestra.run.smithi094.stdout: "pg_placement_num_target": 32, 2024-09-14T06:32:46.787 INFO:teuthology.orchestra.run.smithi094.stdout: "pg_num_target": 32, 2024-09-14T06:32:46.788 INFO:teuthology.orchestra.run.smithi094.stdout: "pg_num_pending": 32, 2024-09-14T06:32:46.788 INFO:teuthology.orchestra.run.smithi094.stdout: "last_pg_merge_meta": { 2024-09-14T06:32:46.788 INFO:teuthology.orchestra.run.smithi094.stdout: "source_pgid": "0.0", 2024-09-14T06:32:46.788 INFO:teuthology.orchestra.run.smithi094.stdout: "ready_epoch": 0, 2024-09-14T06:32:46.788 INFO:teuthology.orchestra.run.smithi094.stdout: "last_epoch_started": 0, 2024-09-14T06:32:46.788 INFO:teuthology.orchestra.run.smithi094.stdout: "last_epoch_clean": 0, 2024-09-14T06:32:46.788 INFO:teuthology.orchestra.run.smithi094.stdout: "source_version": "0'0", 2024-09-14T06:32:46.788 INFO:teuthology.orchestra.run.smithi094.stdout: "target_version": "0'0" 2024-09-14T06:32:46.788 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:46.788 INFO:teuthology.orchestra.run.smithi094.stdout: "last_change": "241", 2024-09-14T06:32:46.788 INFO:teuthology.orchestra.run.smithi094.stdout: "last_force_op_resend": "0", 2024-09-14T06:32:46.788 INFO:teuthology.orchestra.run.smithi094.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-14T06:32:46.788 INFO:teuthology.orchestra.run.smithi094.stdout: "last_force_op_resend_preluminous": "0", 2024-09-14T06:32:46.789 INFO:teuthology.orchestra.run.smithi094.stdout: "auid": 0, 2024-09-14T06:32:46.789 INFO:teuthology.orchestra.run.smithi094.stdout: "snap_mode": "selfmanaged", 2024-09-14T06:32:46.789 INFO:teuthology.orchestra.run.smithi094.stdout: "snap_seq": 92, 2024-09-14T06:32:46.789 INFO:teuthology.orchestra.run.smithi094.stdout: "snap_epoch": 241, 2024-09-14T06:32:46.789 INFO:teuthology.orchestra.run.smithi094.stdout: "pool_snaps": [], 2024-09-14T06:32:46.789 INFO:teuthology.orchestra.run.smithi094.stdout: "removed_snaps": "[]", 2024-09-14T06:32:46.789 INFO:teuthology.orchestra.run.smithi094.stdout: "quota_max_bytes": 0, 2024-09-14T06:32:46.789 INFO:teuthology.orchestra.run.smithi094.stdout: "quota_max_objects": 0, 2024-09-14T06:32:46.789 INFO:teuthology.orchestra.run.smithi094.stdout: "tiers": [], 2024-09-14T06:32:46.789 INFO:teuthology.orchestra.run.smithi094.stdout: "tier_of": -1, 2024-09-14T06:32:46.789 INFO:teuthology.orchestra.run.smithi094.stdout: "read_tier": -1, 2024-09-14T06:32:46.789 INFO:teuthology.orchestra.run.smithi094.stdout: "write_tier": -1, 2024-09-14T06:32:46.789 INFO:teuthology.orchestra.run.smithi094.stdout: "cache_mode": "none", 2024-09-14T06:32:46.789 INFO:teuthology.orchestra.run.smithi094.stdout: "target_max_bytes": 0, 2024-09-14T06:32:46.790 INFO:teuthology.orchestra.run.smithi094.stdout: "target_max_objects": 0, 2024-09-14T06:32:46.790 INFO:teuthology.orchestra.run.smithi094.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-14T06:32:46.790 INFO:teuthology.orchestra.run.smithi094.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-14T06:32:46.790 INFO:teuthology.orchestra.run.smithi094.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-14T06:32:46.790 INFO:teuthology.orchestra.run.smithi094.stdout: "cache_min_flush_age": 0, 2024-09-14T06:32:46.790 INFO:teuthology.orchestra.run.smithi094.stdout: "cache_min_evict_age": 0, 2024-09-14T06:32:46.790 INFO:teuthology.orchestra.run.smithi094.stdout: "erasure_code_profile": "", 2024-09-14T06:32:46.790 INFO:teuthology.orchestra.run.smithi094.stdout: "hit_set_params": { 2024-09-14T06:32:46.790 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "none" 2024-09-14T06:32:46.790 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:46.790 INFO:teuthology.orchestra.run.smithi094.stdout: "hit_set_period": 0, 2024-09-14T06:32:46.790 INFO:teuthology.orchestra.run.smithi094.stdout: "hit_set_count": 0, 2024-09-14T06:32:46.790 INFO:teuthology.orchestra.run.smithi094.stdout: "use_gmt_hitset": true, 2024-09-14T06:32:46.790 INFO:teuthology.orchestra.run.smithi094.stdout: "min_read_recency_for_promote": 0, 2024-09-14T06:32:46.791 INFO:teuthology.orchestra.run.smithi094.stdout: "min_write_recency_for_promote": 0, 2024-09-14T06:32:46.791 INFO:teuthology.orchestra.run.smithi094.stdout: "hit_set_grade_decay_rate": 0, 2024-09-14T06:32:46.791 INFO:teuthology.orchestra.run.smithi094.stdout: "hit_set_search_last_n": 0, 2024-09-14T06:32:46.791 INFO:teuthology.orchestra.run.smithi094.stdout: "grade_table": [], 2024-09-14T06:32:46.791 INFO:teuthology.orchestra.run.smithi094.stdout: "stripe_width": 0, 2024-09-14T06:32:46.791 INFO:teuthology.orchestra.run.smithi094.stdout: "expected_num_objects": 0, 2024-09-14T06:32:46.791 INFO:teuthology.orchestra.run.smithi094.stdout: "fast_read": false, 2024-09-14T06:32:46.791 INFO:teuthology.orchestra.run.smithi094.stdout: "options": {}, 2024-09-14T06:32:46.791 INFO:teuthology.orchestra.run.smithi094.stdout: "application_metadata": { 2024-09-14T06:32:46.791 INFO:teuthology.orchestra.run.smithi094.stdout: "rbd": {} 2024-09-14T06:32:46.791 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:46.792 INFO:teuthology.orchestra.run.smithi094.stdout: "read_balance": { 2024-09-14T06:32:46.792 INFO:teuthology.orchestra.run.smithi094.stdout: "score_type": "Fair distribution", 2024-09-14T06:32:46.792 INFO:teuthology.orchestra.run.smithi094.stdout: "score_acting": 1.75, 2024-09-14T06:32:46.792 INFO:teuthology.orchestra.run.smithi094.stdout: "score_stable": 1.75, 2024-09-14T06:32:46.792 INFO:teuthology.orchestra.run.smithi094.stdout: "optimal_score": 1, 2024-09-14T06:32:46.792 INFO:teuthology.orchestra.run.smithi094.stdout: "raw_score_acting": 1.75, 2024-09-14T06:32:46.792 INFO:teuthology.orchestra.run.smithi094.stdout: "raw_score_stable": 1.75, 2024-09-14T06:32:46.792 INFO:teuthology.orchestra.run.smithi094.stdout: "primary_affinity_weighted": 0.85513687133789062, 2024-09-14T06:32:46.792 INFO:teuthology.orchestra.run.smithi094.stdout: "average_primary_affinity": 0.98000001907348633, 2024-09-14T06:32:46.792 INFO:teuthology.orchestra.run.smithi094.stdout: "average_primary_affinity_weighted": 0.86000001430511475 2024-09-14T06:32:46.792 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:46.792 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:46.793 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-09-14T06:32:46.793 INFO:teuthology.orchestra.run.smithi094.stdout: "osds": [ 2024-09-14T06:32:46.793 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:46.793 INFO:teuthology.orchestra.run.smithi094.stdout: "osd": 0, 2024-09-14T06:32:46.793 INFO:teuthology.orchestra.run.smithi094.stdout: "uuid": "8b22a249-b842-406a-a02c-c0f31f57b635", 2024-09-14T06:32:46.793 INFO:teuthology.orchestra.run.smithi094.stdout: "up": 1, 2024-09-14T06:32:46.793 INFO:teuthology.orchestra.run.smithi094.stdout: "in": 1, 2024-09-14T06:32:46.793 INFO:teuthology.orchestra.run.smithi094.stdout: "weight": 1, 2024-09-14T06:32:46.793 INFO:teuthology.orchestra.run.smithi094.stdout: "primary_affinity": 1, 2024-09-14T06:32:46.793 INFO:teuthology.orchestra.run.smithi094.stdout: "last_clean_begin": 0, 2024-09-14T06:32:46.793 INFO:teuthology.orchestra.run.smithi094.stdout: "last_clean_end": 0, 2024-09-14T06:32:46.793 INFO:teuthology.orchestra.run.smithi094.stdout: "up_from": 14, 2024-09-14T06:32:46.793 INFO:teuthology.orchestra.run.smithi094.stdout: "up_thru": 199, 2024-09-14T06:32:46.793 INFO:teuthology.orchestra.run.smithi094.stdout: "down_at": 0, 2024-09-14T06:32:46.794 INFO:teuthology.orchestra.run.smithi094.stdout: "lost_at": 0, 2024-09-14T06:32:46.794 INFO:teuthology.orchestra.run.smithi094.stdout: "public_addrs": { 2024-09-14T06:32:46.794 INFO:teuthology.orchestra.run.smithi094.stdout: "addrvec": [ 2024-09-14T06:32:46.794 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:46.794 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v2", 2024-09-14T06:32:46.794 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.94:6824", 2024-09-14T06:32:46.794 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 1809564923 2024-09-14T06:32:46.794 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:46.794 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:46.794 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v1", 2024-09-14T06:32:46.794 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.94:6825", 2024-09-14T06:32:46.794 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 1809564923 2024-09-14T06:32:46.794 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:46.795 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:32:46.795 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:46.795 INFO:teuthology.orchestra.run.smithi094.stdout: "cluster_addrs": { 2024-09-14T06:32:46.795 INFO:teuthology.orchestra.run.smithi094.stdout: "addrvec": [ 2024-09-14T06:32:46.795 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:46.795 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v2", 2024-09-14T06:32:46.795 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.94:6826", 2024-09-14T06:32:46.795 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 1809564923 2024-09-14T06:32:46.795 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:46.795 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:46.795 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v1", 2024-09-14T06:32:46.796 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.94:6827", 2024-09-14T06:32:46.796 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 1809564923 2024-09-14T06:32:46.796 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:46.796 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:32:46.796 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:46.796 INFO:teuthology.orchestra.run.smithi094.stdout: "heartbeat_back_addrs": { 2024-09-14T06:32:46.796 INFO:teuthology.orchestra.run.smithi094.stdout: "addrvec": [ 2024-09-14T06:32:46.796 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:46.796 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v2", 2024-09-14T06:32:46.796 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.94:6830", 2024-09-14T06:32:46.796 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 1809564923 2024-09-14T06:32:46.796 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:46.797 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:46.797 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v1", 2024-09-14T06:32:46.797 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.94:6831", 2024-09-14T06:32:46.797 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 1809564923 2024-09-14T06:32:46.797 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:46.797 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:32:46.797 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:46.797 INFO:teuthology.orchestra.run.smithi094.stdout: "heartbeat_front_addrs": { 2024-09-14T06:32:46.797 INFO:teuthology.orchestra.run.smithi094.stdout: "addrvec": [ 2024-09-14T06:32:46.797 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:46.797 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v2", 2024-09-14T06:32:46.797 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.94:6828", 2024-09-14T06:32:46.798 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 1809564923 2024-09-14T06:32:46.798 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:46.798 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:46.798 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v1", 2024-09-14T06:32:46.798 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.94:6829", 2024-09-14T06:32:46.798 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 1809564923 2024-09-14T06:32:46.798 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:46.798 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:32:46.798 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:46.798 INFO:teuthology.orchestra.run.smithi094.stdout: "public_addr": "172.21.15.94:6825/1809564923", 2024-09-14T06:32:46.798 INFO:teuthology.orchestra.run.smithi094.stdout: "cluster_addr": "172.21.15.94:6827/1809564923", 2024-09-14T06:32:46.798 INFO:teuthology.orchestra.run.smithi094.stdout: "heartbeat_back_addr": "172.21.15.94:6831/1809564923", 2024-09-14T06:32:46.798 INFO:teuthology.orchestra.run.smithi094.stdout: "heartbeat_front_addr": "172.21.15.94:6829/1809564923", 2024-09-14T06:32:46.798 INFO:teuthology.orchestra.run.smithi094.stdout: "state": [ 2024-09-14T06:32:46.799 INFO:teuthology.orchestra.run.smithi094.stdout: "exists", 2024-09-14T06:32:46.799 INFO:teuthology.orchestra.run.smithi094.stdout: "up" 2024-09-14T06:32:46.799 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:32:46.799 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:46.799 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:46.799 INFO:teuthology.orchestra.run.smithi094.stdout: "osd": 1, 2024-09-14T06:32:46.799 INFO:teuthology.orchestra.run.smithi094.stdout: "uuid": "02251a03-84a0-414d-a980-fac4fd043cb6", 2024-09-14T06:32:46.799 INFO:teuthology.orchestra.run.smithi094.stdout: "up": 1, 2024-09-14T06:32:46.799 INFO:teuthology.orchestra.run.smithi094.stdout: "in": 1, 2024-09-14T06:32:46.799 INFO:teuthology.orchestra.run.smithi094.stdout: "weight": 1, 2024-09-14T06:32:46.799 INFO:teuthology.orchestra.run.smithi094.stdout: "primary_affinity": 0.841094970703125, 2024-09-14T06:32:46.799 INFO:teuthology.orchestra.run.smithi094.stdout: "last_clean_begin": 0, 2024-09-14T06:32:46.799 INFO:teuthology.orchestra.run.smithi094.stdout: "last_clean_end": 0, 2024-09-14T06:32:46.799 INFO:teuthology.orchestra.run.smithi094.stdout: "up_from": 15, 2024-09-14T06:32:46.800 INFO:teuthology.orchestra.run.smithi094.stdout: "up_thru": 233, 2024-09-14T06:32:46.800 INFO:teuthology.orchestra.run.smithi094.stdout: "down_at": 0, 2024-09-14T06:32:46.800 INFO:teuthology.orchestra.run.smithi094.stdout: "lost_at": 0, 2024-09-14T06:32:46.800 INFO:teuthology.orchestra.run.smithi094.stdout: "public_addrs": { 2024-09-14T06:32:46.800 INFO:teuthology.orchestra.run.smithi094.stdout: "addrvec": [ 2024-09-14T06:32:46.800 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:46.800 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v2", 2024-09-14T06:32:46.800 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.94:6816", 2024-09-14T06:32:46.800 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 736402652 2024-09-14T06:32:46.800 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:46.800 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:46.800 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v1", 2024-09-14T06:32:46.801 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.94:6817", 2024-09-14T06:32:46.801 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 736402652 2024-09-14T06:32:46.801 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:46.801 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:32:46.801 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:46.801 INFO:teuthology.orchestra.run.smithi094.stdout: "cluster_addrs": { 2024-09-14T06:32:46.801 INFO:teuthology.orchestra.run.smithi094.stdout: "addrvec": [ 2024-09-14T06:32:46.801 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:46.801 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v2", 2024-09-14T06:32:46.801 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.94:6818", 2024-09-14T06:32:46.801 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 736402652 2024-09-14T06:32:46.802 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:46.802 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:46.802 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v1", 2024-09-14T06:32:46.802 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.94:6819", 2024-09-14T06:32:46.802 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 736402652 2024-09-14T06:32:46.802 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:46.802 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:32:46.802 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:46.802 INFO:teuthology.orchestra.run.smithi094.stdout: "heartbeat_back_addrs": { 2024-09-14T06:32:46.802 INFO:teuthology.orchestra.run.smithi094.stdout: "addrvec": [ 2024-09-14T06:32:46.802 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:46.802 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v2", 2024-09-14T06:32:46.802 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.94:6822", 2024-09-14T06:32:46.802 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 736402652 2024-09-14T06:32:46.803 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:46.803 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:46.803 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v1", 2024-09-14T06:32:46.803 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.94:6823", 2024-09-14T06:32:46.803 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 736402652 2024-09-14T06:32:46.803 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:46.803 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:32:46.803 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:46.803 INFO:teuthology.orchestra.run.smithi094.stdout: "heartbeat_front_addrs": { 2024-09-14T06:32:46.803 INFO:teuthology.orchestra.run.smithi094.stdout: "addrvec": [ 2024-09-14T06:32:46.803 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:46.803 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v2", 2024-09-14T06:32:46.804 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.94:6820", 2024-09-14T06:32:46.804 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 736402652 2024-09-14T06:32:46.804 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:46.804 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:46.804 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v1", 2024-09-14T06:32:46.804 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.94:6821", 2024-09-14T06:32:46.804 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 736402652 2024-09-14T06:32:46.804 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:46.804 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:32:46.804 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:46.804 INFO:teuthology.orchestra.run.smithi094.stdout: "public_addr": "172.21.15.94:6817/736402652", 2024-09-14T06:32:46.804 INFO:teuthology.orchestra.run.smithi094.stdout: "cluster_addr": "172.21.15.94:6819/736402652", 2024-09-14T06:32:46.804 INFO:teuthology.orchestra.run.smithi094.stdout: "heartbeat_back_addr": "172.21.15.94:6823/736402652", 2024-09-14T06:32:46.805 INFO:teuthology.orchestra.run.smithi094.stdout: "heartbeat_front_addr": "172.21.15.94:6821/736402652", 2024-09-14T06:32:46.805 INFO:teuthology.orchestra.run.smithi094.stdout: "state": [ 2024-09-14T06:32:46.805 INFO:teuthology.orchestra.run.smithi094.stdout: "exists", 2024-09-14T06:32:46.805 INFO:teuthology.orchestra.run.smithi094.stdout: "up" 2024-09-14T06:32:46.805 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:32:46.805 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:46.805 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:46.805 INFO:teuthology.orchestra.run.smithi094.stdout: "osd": 2, 2024-09-14T06:32:46.805 INFO:teuthology.orchestra.run.smithi094.stdout: "uuid": "d45bf470-bb29-4739-8294-cd122b2b4102", 2024-09-14T06:32:46.805 INFO:teuthology.orchestra.run.smithi094.stdout: "up": 1, 2024-09-14T06:32:46.805 INFO:teuthology.orchestra.run.smithi094.stdout: "in": 1, 2024-09-14T06:32:46.805 INFO:teuthology.orchestra.run.smithi094.stdout: "weight": 1, 2024-09-14T06:32:46.805 INFO:teuthology.orchestra.run.smithi094.stdout: "primary_affinity": 1, 2024-09-14T06:32:46.805 INFO:teuthology.orchestra.run.smithi094.stdout: "last_clean_begin": 0, 2024-09-14T06:32:46.805 INFO:teuthology.orchestra.run.smithi094.stdout: "last_clean_end": 0, 2024-09-14T06:32:46.806 INFO:teuthology.orchestra.run.smithi094.stdout: "up_from": 14, 2024-09-14T06:32:46.806 INFO:teuthology.orchestra.run.smithi094.stdout: "up_thru": 233, 2024-09-14T06:32:46.806 INFO:teuthology.orchestra.run.smithi094.stdout: "down_at": 0, 2024-09-14T06:32:46.806 INFO:teuthology.orchestra.run.smithi094.stdout: "lost_at": 0, 2024-09-14T06:32:46.806 INFO:teuthology.orchestra.run.smithi094.stdout: "public_addrs": { 2024-09-14T06:32:46.806 INFO:teuthology.orchestra.run.smithi094.stdout: "addrvec": [ 2024-09-14T06:32:46.806 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:46.806 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v2", 2024-09-14T06:32:46.806 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.94:6808", 2024-09-14T06:32:46.806 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 1063204782 2024-09-14T06:32:46.806 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:46.806 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:46.806 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v1", 2024-09-14T06:32:46.807 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.94:6809", 2024-09-14T06:32:46.807 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 1063204782 2024-09-14T06:32:46.807 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:46.807 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:32:46.807 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:46.807 INFO:teuthology.orchestra.run.smithi094.stdout: "cluster_addrs": { 2024-09-14T06:32:46.807 INFO:teuthology.orchestra.run.smithi094.stdout: "addrvec": [ 2024-09-14T06:32:46.807 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:46.807 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v2", 2024-09-14T06:32:46.807 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.94:6810", 2024-09-14T06:32:46.807 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 1063204782 2024-09-14T06:32:46.808 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:46.808 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:46.808 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v1", 2024-09-14T06:32:46.808 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.94:6811", 2024-09-14T06:32:46.808 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 1063204782 2024-09-14T06:32:46.808 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:46.808 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:32:46.808 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:46.808 INFO:teuthology.orchestra.run.smithi094.stdout: "heartbeat_back_addrs": { 2024-09-14T06:32:46.808 INFO:teuthology.orchestra.run.smithi094.stdout: "addrvec": [ 2024-09-14T06:32:46.808 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:46.808 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v2", 2024-09-14T06:32:46.808 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.94:6814", 2024-09-14T06:32:46.809 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 1063204782 2024-09-14T06:32:46.809 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:46.809 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:46.809 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v1", 2024-09-14T06:32:46.809 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.94:6815", 2024-09-14T06:32:46.809 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 1063204782 2024-09-14T06:32:46.809 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:46.809 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:32:46.809 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:46.810 INFO:teuthology.orchestra.run.smithi094.stdout: "heartbeat_front_addrs": { 2024-09-14T06:32:46.810 INFO:teuthology.orchestra.run.smithi094.stdout: "addrvec": [ 2024-09-14T06:32:46.810 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:46.810 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v2", 2024-09-14T06:32:46.810 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.94:6812", 2024-09-14T06:32:46.810 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 1063204782 2024-09-14T06:32:46.810 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:46.810 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:46.810 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v1", 2024-09-14T06:32:46.810 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.94:6813", 2024-09-14T06:32:46.810 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 1063204782 2024-09-14T06:32:46.810 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:46.810 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:32:46.810 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:46.810 INFO:teuthology.orchestra.run.smithi094.stdout: "public_addr": "172.21.15.94:6809/1063204782", 2024-09-14T06:32:46.811 INFO:teuthology.orchestra.run.smithi094.stdout: "cluster_addr": "172.21.15.94:6811/1063204782", 2024-09-14T06:32:46.811 INFO:teuthology.orchestra.run.smithi094.stdout: "heartbeat_back_addr": "172.21.15.94:6815/1063204782", 2024-09-14T06:32:46.811 INFO:teuthology.orchestra.run.smithi094.stdout: "heartbeat_front_addr": "172.21.15.94:6813/1063204782", 2024-09-14T06:32:46.811 INFO:teuthology.orchestra.run.smithi094.stdout: "state": [ 2024-09-14T06:32:46.811 INFO:teuthology.orchestra.run.smithi094.stdout: "exists", 2024-09-14T06:32:46.811 INFO:teuthology.orchestra.run.smithi094.stdout: "up" 2024-09-14T06:32:46.811 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:32:46.811 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:46.811 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:46.811 INFO:teuthology.orchestra.run.smithi094.stdout: "osd": 3, 2024-09-14T06:32:46.811 INFO:teuthology.orchestra.run.smithi094.stdout: "uuid": "456ec2d9-c438-4243-b931-335704f661eb", 2024-09-14T06:32:46.811 INFO:teuthology.orchestra.run.smithi094.stdout: "up": 1, 2024-09-14T06:32:46.811 INFO:teuthology.orchestra.run.smithi094.stdout: "in": 1, 2024-09-14T06:32:46.811 INFO:teuthology.orchestra.run.smithi094.stdout: "weight": 1, 2024-09-14T06:32:46.811 INFO:teuthology.orchestra.run.smithi094.stdout: "primary_affinity": 1, 2024-09-14T06:32:46.811 INFO:teuthology.orchestra.run.smithi094.stdout: "last_clean_begin": 0, 2024-09-14T06:32:46.812 INFO:teuthology.orchestra.run.smithi094.stdout: "last_clean_end": 0, 2024-09-14T06:32:46.812 INFO:teuthology.orchestra.run.smithi094.stdout: "up_from": 13, 2024-09-14T06:32:46.812 INFO:teuthology.orchestra.run.smithi094.stdout: "up_thru": 218, 2024-09-14T06:32:46.812 INFO:teuthology.orchestra.run.smithi094.stdout: "down_at": 0, 2024-09-14T06:32:46.812 INFO:teuthology.orchestra.run.smithi094.stdout: "lost_at": 0, 2024-09-14T06:32:46.812 INFO:teuthology.orchestra.run.smithi094.stdout: "public_addrs": { 2024-09-14T06:32:46.812 INFO:teuthology.orchestra.run.smithi094.stdout: "addrvec": [ 2024-09-14T06:32:46.812 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:46.812 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v2", 2024-09-14T06:32:46.812 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.94:6800", 2024-09-14T06:32:46.812 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 3382046754 2024-09-14T06:32:46.812 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:46.812 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:46.812 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v1", 2024-09-14T06:32:46.813 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.94:6801", 2024-09-14T06:32:46.813 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 3382046754 2024-09-14T06:32:46.813 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:46.813 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:32:46.813 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:46.813 INFO:teuthology.orchestra.run.smithi094.stdout: "cluster_addrs": { 2024-09-14T06:32:46.813 INFO:teuthology.orchestra.run.smithi094.stdout: "addrvec": [ 2024-09-14T06:32:46.813 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:46.813 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v2", 2024-09-14T06:32:46.813 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.94:6802", 2024-09-14T06:32:46.813 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 3382046754 2024-09-14T06:32:46.813 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:46.813 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:46.814 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v1", 2024-09-14T06:32:46.814 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.94:6803", 2024-09-14T06:32:46.814 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 3382046754 2024-09-14T06:32:46.814 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:46.814 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:32:46.814 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:46.814 INFO:teuthology.orchestra.run.smithi094.stdout: "heartbeat_back_addrs": { 2024-09-14T06:32:46.814 INFO:teuthology.orchestra.run.smithi094.stdout: "addrvec": [ 2024-09-14T06:32:46.814 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:46.814 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v2", 2024-09-14T06:32:46.814 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.94:6806", 2024-09-14T06:32:46.814 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 3382046754 2024-09-14T06:32:46.814 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:46.814 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:46.814 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v1", 2024-09-14T06:32:46.815 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.94:6807", 2024-09-14T06:32:46.815 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 3382046754 2024-09-14T06:32:46.815 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:46.815 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:32:46.815 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:46.815 INFO:teuthology.orchestra.run.smithi094.stdout: "heartbeat_front_addrs": { 2024-09-14T06:32:46.815 INFO:teuthology.orchestra.run.smithi094.stdout: "addrvec": [ 2024-09-14T06:32:46.815 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:46.815 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v2", 2024-09-14T06:32:46.815 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.94:6804", 2024-09-14T06:32:46.815 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 3382046754 2024-09-14T06:32:46.815 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:46.815 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:46.816 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v1", 2024-09-14T06:32:46.816 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.94:6805", 2024-09-14T06:32:46.816 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 3382046754 2024-09-14T06:32:46.816 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:46.816 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:32:46.816 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:46.816 INFO:teuthology.orchestra.run.smithi094.stdout: "public_addr": "172.21.15.94:6801/3382046754", 2024-09-14T06:32:46.816 INFO:teuthology.orchestra.run.smithi094.stdout: "cluster_addr": "172.21.15.94:6803/3382046754", 2024-09-14T06:32:46.816 INFO:teuthology.orchestra.run.smithi094.stdout: "heartbeat_back_addr": "172.21.15.94:6807/3382046754", 2024-09-14T06:32:46.816 INFO:teuthology.orchestra.run.smithi094.stdout: "heartbeat_front_addr": "172.21.15.94:6805/3382046754", 2024-09-14T06:32:46.816 INFO:teuthology.orchestra.run.smithi094.stdout: "state": [ 2024-09-14T06:32:46.816 INFO:teuthology.orchestra.run.smithi094.stdout: "exists", 2024-09-14T06:32:46.816 INFO:teuthology.orchestra.run.smithi094.stdout: "up" 2024-09-14T06:32:46.816 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:32:46.816 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:46.816 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:46.817 INFO:teuthology.orchestra.run.smithi094.stdout: "osd": 4, 2024-09-14T06:32:46.817 INFO:teuthology.orchestra.run.smithi094.stdout: "uuid": "2dab948e-c6e4-4437-bf17-017d1ac6821c", 2024-09-14T06:32:46.817 INFO:teuthology.orchestra.run.smithi094.stdout: "up": 1, 2024-09-14T06:32:46.817 INFO:teuthology.orchestra.run.smithi094.stdout: "in": 1, 2024-09-14T06:32:46.817 INFO:teuthology.orchestra.run.smithi094.stdout: "weight": 1, 2024-09-14T06:32:46.817 INFO:teuthology.orchestra.run.smithi094.stdout: "primary_affinity": 1, 2024-09-14T06:32:46.817 INFO:teuthology.orchestra.run.smithi094.stdout: "last_clean_begin": 0, 2024-09-14T06:32:46.817 INFO:teuthology.orchestra.run.smithi094.stdout: "last_clean_end": 0, 2024-09-14T06:32:46.817 INFO:teuthology.orchestra.run.smithi094.stdout: "up_from": 15, 2024-09-14T06:32:46.817 INFO:teuthology.orchestra.run.smithi094.stdout: "up_thru": 232, 2024-09-14T06:32:46.817 INFO:teuthology.orchestra.run.smithi094.stdout: "down_at": 0, 2024-09-14T06:32:46.817 INFO:teuthology.orchestra.run.smithi094.stdout: "lost_at": 0, 2024-09-14T06:32:46.817 INFO:teuthology.orchestra.run.smithi094.stdout: "public_addrs": { 2024-09-14T06:32:46.817 INFO:teuthology.orchestra.run.smithi094.stdout: "addrvec": [ 2024-09-14T06:32:46.817 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:46.818 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v2", 2024-09-14T06:32:46.818 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.138:6810", 2024-09-14T06:32:46.818 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 501726729 2024-09-14T06:32:46.818 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:46.818 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:46.818 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v1", 2024-09-14T06:32:46.818 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.138:6811", 2024-09-14T06:32:46.818 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 501726729 2024-09-14T06:32:46.818 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:46.818 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:32:46.818 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:46.818 INFO:teuthology.orchestra.run.smithi094.stdout: "cluster_addrs": { 2024-09-14T06:32:46.819 INFO:teuthology.orchestra.run.smithi094.stdout: "addrvec": [ 2024-09-14T06:32:46.819 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:46.819 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v2", 2024-09-14T06:32:46.819 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.138:6812", 2024-09-14T06:32:46.819 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 501726729 2024-09-14T06:32:46.819 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:46.819 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:46.819 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v1", 2024-09-14T06:32:46.819 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.138:6813", 2024-09-14T06:32:46.819 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 501726729 2024-09-14T06:32:46.819 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:46.819 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:32:46.819 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:46.819 INFO:teuthology.orchestra.run.smithi094.stdout: "heartbeat_back_addrs": { 2024-09-14T06:32:46.819 INFO:teuthology.orchestra.run.smithi094.stdout: "addrvec": [ 2024-09-14T06:32:46.820 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:46.820 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v2", 2024-09-14T06:32:46.820 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.138:6816", 2024-09-14T06:32:46.820 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 501726729 2024-09-14T06:32:46.820 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:46.820 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:46.820 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v1", 2024-09-14T06:32:46.820 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.138:6817", 2024-09-14T06:32:46.820 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 501726729 2024-09-14T06:32:46.820 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:46.820 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:32:46.820 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:46.820 INFO:teuthology.orchestra.run.smithi094.stdout: "heartbeat_front_addrs": { 2024-09-14T06:32:46.820 INFO:teuthology.orchestra.run.smithi094.stdout: "addrvec": [ 2024-09-14T06:32:46.821 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:46.821 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v2", 2024-09-14T06:32:46.821 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.138:6814", 2024-09-14T06:32:46.821 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 501726729 2024-09-14T06:32:46.821 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:46.821 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:46.821 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v1", 2024-09-14T06:32:46.821 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.138:6815", 2024-09-14T06:32:46.821 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 501726729 2024-09-14T06:32:46.821 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:46.821 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:32:46.821 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:46.821 INFO:teuthology.orchestra.run.smithi094.stdout: "public_addr": "172.21.15.138:6811/501726729", 2024-09-14T06:32:46.822 INFO:teuthology.orchestra.run.smithi094.stdout: "cluster_addr": "172.21.15.138:6813/501726729", 2024-09-14T06:32:46.822 INFO:teuthology.orchestra.run.smithi094.stdout: "heartbeat_back_addr": "172.21.15.138:6817/501726729", 2024-09-14T06:32:46.822 INFO:teuthology.orchestra.run.smithi094.stdout: "heartbeat_front_addr": "172.21.15.138:6815/501726729", 2024-09-14T06:32:46.822 INFO:teuthology.orchestra.run.smithi094.stdout: "state": [ 2024-09-14T06:32:46.822 INFO:teuthology.orchestra.run.smithi094.stdout: "exists", 2024-09-14T06:32:46.822 INFO:teuthology.orchestra.run.smithi094.stdout: "up" 2024-09-14T06:32:46.822 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:32:46.822 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:46.822 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:46.822 INFO:teuthology.orchestra.run.smithi094.stdout: "osd": 5, 2024-09-14T06:32:46.822 INFO:teuthology.orchestra.run.smithi094.stdout: "uuid": "8caf3aa3-ad87-4d33-b82e-4c15195176fd", 2024-09-14T06:32:46.823 INFO:teuthology.orchestra.run.smithi094.stdout: "up": 1, 2024-09-14T06:32:46.823 INFO:teuthology.orchestra.run.smithi094.stdout: "in": 1, 2024-09-14T06:32:46.823 INFO:teuthology.orchestra.run.smithi094.stdout: "weight": 1, 2024-09-14T06:32:46.823 INFO:teuthology.orchestra.run.smithi094.stdout: "primary_affinity": 0, 2024-09-14T06:32:46.823 INFO:teuthology.orchestra.run.smithi094.stdout: "last_clean_begin": 0, 2024-09-14T06:32:46.823 INFO:teuthology.orchestra.run.smithi094.stdout: "last_clean_end": 0, 2024-09-14T06:32:46.823 INFO:teuthology.orchestra.run.smithi094.stdout: "up_from": 15, 2024-09-14T06:32:46.823 INFO:teuthology.orchestra.run.smithi094.stdout: "up_thru": 199, 2024-09-14T06:32:46.823 INFO:teuthology.orchestra.run.smithi094.stdout: "down_at": 0, 2024-09-14T06:32:46.823 INFO:teuthology.orchestra.run.smithi094.stdout: "lost_at": 0, 2024-09-14T06:32:46.823 INFO:teuthology.orchestra.run.smithi094.stdout: "public_addrs": { 2024-09-14T06:32:46.823 INFO:teuthology.orchestra.run.smithi094.stdout: "addrvec": [ 2024-09-14T06:32:46.823 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:46.823 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v2", 2024-09-14T06:32:46.824 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.138:6826", 2024-09-14T06:32:46.824 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 1599292735 2024-09-14T06:32:46.824 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:46.824 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:46.824 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v1", 2024-09-14T06:32:46.824 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.138:6827", 2024-09-14T06:32:46.824 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 1599292735 2024-09-14T06:32:46.824 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:46.824 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:32:46.824 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:46.824 INFO:teuthology.orchestra.run.smithi094.stdout: "cluster_addrs": { 2024-09-14T06:32:46.824 INFO:teuthology.orchestra.run.smithi094.stdout: "addrvec": [ 2024-09-14T06:32:46.824 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:46.825 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v2", 2024-09-14T06:32:46.825 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.138:6828", 2024-09-14T06:32:46.825 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 1599292735 2024-09-14T06:32:46.825 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:46.825 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:46.825 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v1", 2024-09-14T06:32:46.825 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.138:6829", 2024-09-14T06:32:46.825 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 1599292735 2024-09-14T06:32:46.825 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:46.825 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:32:46.825 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:46.825 INFO:teuthology.orchestra.run.smithi094.stdout: "heartbeat_back_addrs": { 2024-09-14T06:32:46.826 INFO:teuthology.orchestra.run.smithi094.stdout: "addrvec": [ 2024-09-14T06:32:46.826 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:46.826 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v2", 2024-09-14T06:32:46.826 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.138:6832", 2024-09-14T06:32:46.826 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 1599292735 2024-09-14T06:32:46.826 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:46.826 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:46.826 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v1", 2024-09-14T06:32:46.826 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.138:6833", 2024-09-14T06:32:46.826 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 1599292735 2024-09-14T06:32:46.826 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:46.826 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:32:46.826 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:46.827 INFO:teuthology.orchestra.run.smithi094.stdout: "heartbeat_front_addrs": { 2024-09-14T06:32:46.827 INFO:teuthology.orchestra.run.smithi094.stdout: "addrvec": [ 2024-09-14T06:32:46.827 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:46.827 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v2", 2024-09-14T06:32:46.827 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.138:6830", 2024-09-14T06:32:46.827 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 1599292735 2024-09-14T06:32:46.827 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:46.827 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:46.827 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v1", 2024-09-14T06:32:46.827 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.138:6831", 2024-09-14T06:32:46.827 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 1599292735 2024-09-14T06:32:46.827 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:46.827 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:32:46.827 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:46.827 INFO:teuthology.orchestra.run.smithi094.stdout: "public_addr": "172.21.15.138:6827/1599292735", 2024-09-14T06:32:46.828 INFO:teuthology.orchestra.run.smithi094.stdout: "cluster_addr": "172.21.15.138:6829/1599292735", 2024-09-14T06:32:46.828 INFO:teuthology.orchestra.run.smithi094.stdout: "heartbeat_back_addr": "172.21.15.138:6833/1599292735", 2024-09-14T06:32:46.828 INFO:teuthology.orchestra.run.smithi094.stdout: "heartbeat_front_addr": "172.21.15.138:6831/1599292735", 2024-09-14T06:32:46.828 INFO:teuthology.orchestra.run.smithi094.stdout: "state": [ 2024-09-14T06:32:46.828 INFO:teuthology.orchestra.run.smithi094.stdout: "exists", 2024-09-14T06:32:46.828 INFO:teuthology.orchestra.run.smithi094.stdout: "up" 2024-09-14T06:32:46.828 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:32:46.828 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:46.828 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:46.828 INFO:teuthology.orchestra.run.smithi094.stdout: "osd": 6, 2024-09-14T06:32:46.828 INFO:teuthology.orchestra.run.smithi094.stdout: "uuid": "e55d11fd-433d-4462-93a7-d5e84560221b", 2024-09-14T06:32:46.828 INFO:teuthology.orchestra.run.smithi094.stdout: "up": 1, 2024-09-14T06:32:46.828 INFO:teuthology.orchestra.run.smithi094.stdout: "in": 0, 2024-09-14T06:32:46.828 INFO:teuthology.orchestra.run.smithi094.stdout: "weight": 0, 2024-09-14T06:32:46.829 INFO:teuthology.orchestra.run.smithi094.stdout: "primary_affinity": 1, 2024-09-14T06:32:46.829 INFO:teuthology.orchestra.run.smithi094.stdout: "last_clean_begin": 0, 2024-09-14T06:32:46.829 INFO:teuthology.orchestra.run.smithi094.stdout: "last_clean_end": 0, 2024-09-14T06:32:46.829 INFO:teuthology.orchestra.run.smithi094.stdout: "up_from": 15, 2024-09-14T06:32:46.829 INFO:teuthology.orchestra.run.smithi094.stdout: "up_thru": 176, 2024-09-14T06:32:46.829 INFO:teuthology.orchestra.run.smithi094.stdout: "down_at": 0, 2024-09-14T06:32:46.829 INFO:teuthology.orchestra.run.smithi094.stdout: "lost_at": 0, 2024-09-14T06:32:46.829 INFO:teuthology.orchestra.run.smithi094.stdout: "public_addrs": { 2024-09-14T06:32:46.829 INFO:teuthology.orchestra.run.smithi094.stdout: "addrvec": [ 2024-09-14T06:32:46.829 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:46.829 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v2", 2024-09-14T06:32:46.829 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.138:6818", 2024-09-14T06:32:46.829 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 163531479 2024-09-14T06:32:46.829 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:46.829 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:46.830 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v1", 2024-09-14T06:32:46.830 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.138:6819", 2024-09-14T06:32:46.830 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 163531479 2024-09-14T06:32:46.830 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:46.830 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:32:46.830 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:46.830 INFO:teuthology.orchestra.run.smithi094.stdout: "cluster_addrs": { 2024-09-14T06:32:46.830 INFO:teuthology.orchestra.run.smithi094.stdout: "addrvec": [ 2024-09-14T06:32:46.830 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:46.830 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v2", 2024-09-14T06:32:46.830 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.138:6820", 2024-09-14T06:32:46.830 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 163531479 2024-09-14T06:32:46.830 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:46.831 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:46.831 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v1", 2024-09-14T06:32:46.831 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.138:6821", 2024-09-14T06:32:46.831 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 163531479 2024-09-14T06:32:46.831 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:46.831 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:32:46.831 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:46.831 INFO:teuthology.orchestra.run.smithi094.stdout: "heartbeat_back_addrs": { 2024-09-14T06:32:46.831 INFO:teuthology.orchestra.run.smithi094.stdout: "addrvec": [ 2024-09-14T06:32:46.831 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:46.831 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v2", 2024-09-14T06:32:46.831 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.138:6824", 2024-09-14T06:32:46.831 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 163531479 2024-09-14T06:32:46.831 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:46.831 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:46.832 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v1", 2024-09-14T06:32:46.832 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.138:6825", 2024-09-14T06:32:46.832 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 163531479 2024-09-14T06:32:46.832 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:46.832 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:32:46.832 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:46.832 INFO:teuthology.orchestra.run.smithi094.stdout: "heartbeat_front_addrs": { 2024-09-14T06:32:46.832 INFO:teuthology.orchestra.run.smithi094.stdout: "addrvec": [ 2024-09-14T06:32:46.832 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:46.832 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v2", 2024-09-14T06:32:46.832 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.138:6822", 2024-09-14T06:32:46.832 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 163531479 2024-09-14T06:32:46.832 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:46.832 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:46.832 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v1", 2024-09-14T06:32:46.833 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.138:6823", 2024-09-14T06:32:46.833 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 163531479 2024-09-14T06:32:46.833 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:46.833 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:32:46.833 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:46.833 INFO:teuthology.orchestra.run.smithi094.stdout: "public_addr": "172.21.15.138:6819/163531479", 2024-09-14T06:32:46.833 INFO:teuthology.orchestra.run.smithi094.stdout: "cluster_addr": "172.21.15.138:6821/163531479", 2024-09-14T06:32:46.833 INFO:teuthology.orchestra.run.smithi094.stdout: "heartbeat_back_addr": "172.21.15.138:6825/163531479", 2024-09-14T06:32:46.833 INFO:teuthology.orchestra.run.smithi094.stdout: "heartbeat_front_addr": "172.21.15.138:6823/163531479", 2024-09-14T06:32:46.833 INFO:teuthology.orchestra.run.smithi094.stdout: "state": [ 2024-09-14T06:32:46.833 INFO:teuthology.orchestra.run.smithi094.stdout: "exists", 2024-09-14T06:32:46.833 INFO:teuthology.orchestra.run.smithi094.stdout: "up" 2024-09-14T06:32:46.833 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:32:46.833 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:46.833 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:46.834 INFO:teuthology.orchestra.run.smithi094.stdout: "osd": 7, 2024-09-14T06:32:46.834 INFO:teuthology.orchestra.run.smithi094.stdout: "uuid": "ba14ba69-2c6c-4d09-81e2-2593dc3a7ea9", 2024-09-14T06:32:46.834 INFO:teuthology.orchestra.run.smithi094.stdout: "up": 1, 2024-09-14T06:32:46.834 INFO:teuthology.orchestra.run.smithi094.stdout: "in": 1, 2024-09-14T06:32:46.834 INFO:teuthology.orchestra.run.smithi094.stdout: "weight": 1, 2024-09-14T06:32:46.834 INFO:teuthology.orchestra.run.smithi094.stdout: "primary_affinity": 1, 2024-09-14T06:32:46.834 INFO:teuthology.orchestra.run.smithi094.stdout: "last_clean_begin": 0, 2024-09-14T06:32:46.834 INFO:teuthology.orchestra.run.smithi094.stdout: "last_clean_end": 0, 2024-09-14T06:32:46.834 INFO:teuthology.orchestra.run.smithi094.stdout: "up_from": 15, 2024-09-14T06:32:46.834 INFO:teuthology.orchestra.run.smithi094.stdout: "up_thru": 233, 2024-09-14T06:32:46.834 INFO:teuthology.orchestra.run.smithi094.stdout: "down_at": 0, 2024-09-14T06:32:46.834 INFO:teuthology.orchestra.run.smithi094.stdout: "lost_at": 0, 2024-09-14T06:32:46.834 INFO:teuthology.orchestra.run.smithi094.stdout: "public_addrs": { 2024-09-14T06:32:46.834 INFO:teuthology.orchestra.run.smithi094.stdout: "addrvec": [ 2024-09-14T06:32:46.834 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:46.835 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v2", 2024-09-14T06:32:46.835 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.138:6800", 2024-09-14T06:32:46.835 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 218507235 2024-09-14T06:32:46.835 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:46.835 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:46.835 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v1", 2024-09-14T06:32:46.835 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.138:6801", 2024-09-14T06:32:46.835 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 218507235 2024-09-14T06:32:46.835 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:46.835 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:32:46.835 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:46.835 INFO:teuthology.orchestra.run.smithi094.stdout: "cluster_addrs": { 2024-09-14T06:32:46.835 INFO:teuthology.orchestra.run.smithi094.stdout: "addrvec": [ 2024-09-14T06:32:46.835 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:46.836 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v2", 2024-09-14T06:32:46.836 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.138:6802", 2024-09-14T06:32:46.836 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 218507235 2024-09-14T06:32:46.836 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:46.836 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:46.836 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v1", 2024-09-14T06:32:46.836 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.138:6803", 2024-09-14T06:32:46.836 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 218507235 2024-09-14T06:32:46.836 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:46.836 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:32:46.836 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:46.836 INFO:teuthology.orchestra.run.smithi094.stdout: "heartbeat_back_addrs": { 2024-09-14T06:32:46.837 INFO:teuthology.orchestra.run.smithi094.stdout: "addrvec": [ 2024-09-14T06:32:46.837 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:46.837 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v2", 2024-09-14T06:32:46.837 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.138:6806", 2024-09-14T06:32:46.837 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 218507235 2024-09-14T06:32:46.837 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:46.837 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:46.837 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v1", 2024-09-14T06:32:46.837 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.138:6807", 2024-09-14T06:32:46.837 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 218507235 2024-09-14T06:32:46.837 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:46.837 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:32:46.837 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:46.838 INFO:teuthology.orchestra.run.smithi094.stdout: "heartbeat_front_addrs": { 2024-09-14T06:32:46.838 INFO:teuthology.orchestra.run.smithi094.stdout: "addrvec": [ 2024-09-14T06:32:46.838 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:46.838 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v2", 2024-09-14T06:32:46.838 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.138:6804", 2024-09-14T06:32:46.838 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 218507235 2024-09-14T06:32:46.838 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:46.838 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:46.838 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v1", 2024-09-14T06:32:46.838 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.138:6805", 2024-09-14T06:32:46.838 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 218507235 2024-09-14T06:32:46.838 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:46.838 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:32:46.838 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:46.839 INFO:teuthology.orchestra.run.smithi094.stdout: "public_addr": "172.21.15.138:6801/218507235", 2024-09-14T06:32:46.839 INFO:teuthology.orchestra.run.smithi094.stdout: "cluster_addr": "172.21.15.138:6803/218507235", 2024-09-14T06:32:46.839 INFO:teuthology.orchestra.run.smithi094.stdout: "heartbeat_back_addr": "172.21.15.138:6807/218507235", 2024-09-14T06:32:46.839 INFO:teuthology.orchestra.run.smithi094.stdout: "heartbeat_front_addr": "172.21.15.138:6805/218507235", 2024-09-14T06:32:46.839 INFO:teuthology.orchestra.run.smithi094.stdout: "state": [ 2024-09-14T06:32:46.839 INFO:teuthology.orchestra.run.smithi094.stdout: "exists", 2024-09-14T06:32:46.839 INFO:teuthology.orchestra.run.smithi094.stdout: "up" 2024-09-14T06:32:46.839 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:32:46.839 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:46.839 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-09-14T06:32:46.839 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_xinfo": [ 2024-09-14T06:32:46.839 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:46.839 INFO:teuthology.orchestra.run.smithi094.stdout: "osd": 0, 2024-09-14T06:32:46.839 INFO:teuthology.orchestra.run.smithi094.stdout: "down_stamp": "0.000000", 2024-09-14T06:32:46.840 INFO:teuthology.orchestra.run.smithi094.stdout: "laggy_probability": 0, 2024-09-14T06:32:46.840 INFO:teuthology.orchestra.run.smithi094.stdout: "laggy_interval": 0, 2024-09-14T06:32:46.840 INFO:teuthology.orchestra.run.smithi094.stdout: "features": 4540719139924082687, 2024-09-14T06:32:46.840 INFO:teuthology.orchestra.run.smithi094.stdout: "old_weight": 0, 2024-09-14T06:32:46.840 INFO:teuthology.orchestra.run.smithi094.stdout: "last_purged_snaps_scrub": "2024-09-14T06:28:39.906639+0000", 2024-09-14T06:32:46.840 INFO:teuthology.orchestra.run.smithi094.stdout: "dead_epoch": 0 2024-09-14T06:32:46.840 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:46.840 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:46.840 INFO:teuthology.orchestra.run.smithi094.stdout: "osd": 1, 2024-09-14T06:32:46.840 INFO:teuthology.orchestra.run.smithi094.stdout: "down_stamp": "0.000000", 2024-09-14T06:32:46.840 INFO:teuthology.orchestra.run.smithi094.stdout: "laggy_probability": 0, 2024-09-14T06:32:46.840 INFO:teuthology.orchestra.run.smithi094.stdout: "laggy_interval": 0, 2024-09-14T06:32:46.840 INFO:teuthology.orchestra.run.smithi094.stdout: "features": 4540719139924082687, 2024-09-14T06:32:46.840 INFO:teuthology.orchestra.run.smithi094.stdout: "old_weight": 0, 2024-09-14T06:32:46.841 INFO:teuthology.orchestra.run.smithi094.stdout: "last_purged_snaps_scrub": "2024-09-14T06:28:40.541279+0000", 2024-09-14T06:32:46.841 INFO:teuthology.orchestra.run.smithi094.stdout: "dead_epoch": 0 2024-09-14T06:32:46.841 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:46.841 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:46.841 INFO:teuthology.orchestra.run.smithi094.stdout: "osd": 2, 2024-09-14T06:32:46.841 INFO:teuthology.orchestra.run.smithi094.stdout: "down_stamp": "0.000000", 2024-09-14T06:32:46.841 INFO:teuthology.orchestra.run.smithi094.stdout: "laggy_probability": 0, 2024-09-14T06:32:46.841 INFO:teuthology.orchestra.run.smithi094.stdout: "laggy_interval": 0, 2024-09-14T06:32:46.841 INFO:teuthology.orchestra.run.smithi094.stdout: "features": 4540719139924082687, 2024-09-14T06:32:46.841 INFO:teuthology.orchestra.run.smithi094.stdout: "old_weight": 0, 2024-09-14T06:32:46.841 INFO:teuthology.orchestra.run.smithi094.stdout: "last_purged_snaps_scrub": "2024-09-14T06:28:40.025350+0000", 2024-09-14T06:32:46.841 INFO:teuthology.orchestra.run.smithi094.stdout: "dead_epoch": 0 2024-09-14T06:32:46.841 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:46.842 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:46.842 INFO:teuthology.orchestra.run.smithi094.stdout: "osd": 3, 2024-09-14T06:32:46.842 INFO:teuthology.orchestra.run.smithi094.stdout: "down_stamp": "0.000000", 2024-09-14T06:32:46.842 INFO:teuthology.orchestra.run.smithi094.stdout: "laggy_probability": 0, 2024-09-14T06:32:46.842 INFO:teuthology.orchestra.run.smithi094.stdout: "laggy_interval": 0, 2024-09-14T06:32:46.842 INFO:teuthology.orchestra.run.smithi094.stdout: "features": 4540719139924082687, 2024-09-14T06:32:46.842 INFO:teuthology.orchestra.run.smithi094.stdout: "old_weight": 0, 2024-09-14T06:32:46.842 INFO:teuthology.orchestra.run.smithi094.stdout: "last_purged_snaps_scrub": "2024-09-14T06:28:39.217944+0000", 2024-09-14T06:32:46.842 INFO:teuthology.orchestra.run.smithi094.stdout: "dead_epoch": 0 2024-09-14T06:32:46.842 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:46.842 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:46.842 INFO:teuthology.orchestra.run.smithi094.stdout: "osd": 4, 2024-09-14T06:32:46.842 INFO:teuthology.orchestra.run.smithi094.stdout: "down_stamp": "0.000000", 2024-09-14T06:32:46.842 INFO:teuthology.orchestra.run.smithi094.stdout: "laggy_probability": 0, 2024-09-14T06:32:46.842 INFO:teuthology.orchestra.run.smithi094.stdout: "laggy_interval": 0, 2024-09-14T06:32:46.842 INFO:teuthology.orchestra.run.smithi094.stdout: "features": 4540719139924082687, 2024-09-14T06:32:46.843 INFO:teuthology.orchestra.run.smithi094.stdout: "old_weight": 0, 2024-09-14T06:32:46.843 INFO:teuthology.orchestra.run.smithi094.stdout: "last_purged_snaps_scrub": "2024-09-14T06:28:40.185898+0000", 2024-09-14T06:32:46.843 INFO:teuthology.orchestra.run.smithi094.stdout: "dead_epoch": 0 2024-09-14T06:32:46.843 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:46.843 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:46.843 INFO:teuthology.orchestra.run.smithi094.stdout: "osd": 5, 2024-09-14T06:32:46.843 INFO:teuthology.orchestra.run.smithi094.stdout: "down_stamp": "0.000000", 2024-09-14T06:32:46.843 INFO:teuthology.orchestra.run.smithi094.stdout: "laggy_probability": 0, 2024-09-14T06:32:46.843 INFO:teuthology.orchestra.run.smithi094.stdout: "laggy_interval": 0, 2024-09-14T06:32:46.843 INFO:teuthology.orchestra.run.smithi094.stdout: "features": 4540719139924082687, 2024-09-14T06:32:46.843 INFO:teuthology.orchestra.run.smithi094.stdout: "old_weight": 0, 2024-09-14T06:32:46.843 INFO:teuthology.orchestra.run.smithi094.stdout: "last_purged_snaps_scrub": "2024-09-14T06:28:40.783504+0000", 2024-09-14T06:32:46.843 INFO:teuthology.orchestra.run.smithi094.stdout: "dead_epoch": 0 2024-09-14T06:32:46.843 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:46.843 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:46.844 INFO:teuthology.orchestra.run.smithi094.stdout: "osd": 6, 2024-09-14T06:32:46.844 INFO:teuthology.orchestra.run.smithi094.stdout: "down_stamp": "0.000000", 2024-09-14T06:32:46.844 INFO:teuthology.orchestra.run.smithi094.stdout: "laggy_probability": 0, 2024-09-14T06:32:46.844 INFO:teuthology.orchestra.run.smithi094.stdout: "laggy_interval": 0, 2024-09-14T06:32:46.844 INFO:teuthology.orchestra.run.smithi094.stdout: "features": 4540719139924082687, 2024-09-14T06:32:46.844 INFO:teuthology.orchestra.run.smithi094.stdout: "old_weight": 65536, 2024-09-14T06:32:46.844 INFO:teuthology.orchestra.run.smithi094.stdout: "last_purged_snaps_scrub": "2024-09-14T06:28:40.527305+0000", 2024-09-14T06:32:46.844 INFO:teuthology.orchestra.run.smithi094.stdout: "dead_epoch": 0 2024-09-14T06:32:46.844 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:46.844 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:46.844 INFO:teuthology.orchestra.run.smithi094.stdout: "osd": 7, 2024-09-14T06:32:46.844 INFO:teuthology.orchestra.run.smithi094.stdout: "down_stamp": "0.000000", 2024-09-14T06:32:46.844 INFO:teuthology.orchestra.run.smithi094.stdout: "laggy_probability": 0, 2024-09-14T06:32:46.845 INFO:teuthology.orchestra.run.smithi094.stdout: "laggy_interval": 0, 2024-09-14T06:32:46.845 INFO:teuthology.orchestra.run.smithi094.stdout: "features": 4540719139924082687, 2024-09-14T06:32:46.845 INFO:teuthology.orchestra.run.smithi094.stdout: "old_weight": 0, 2024-09-14T06:32:46.845 INFO:teuthology.orchestra.run.smithi094.stdout: "last_purged_snaps_scrub": "2024-09-14T06:28:40.349676+0000", 2024-09-14T06:32:46.845 INFO:teuthology.orchestra.run.smithi094.stdout: "dead_epoch": 0 2024-09-14T06:32:46.845 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:46.845 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-09-14T06:32:46.845 INFO:teuthology.orchestra.run.smithi094.stdout: "pg_upmap": [], 2024-09-14T06:32:46.845 INFO:teuthology.orchestra.run.smithi094.stdout: "pg_upmap_items": [ 2024-09-14T06:32:46.845 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:46.845 INFO:teuthology.orchestra.run.smithi094.stdout: "pgid": "3.e", 2024-09-14T06:32:46.845 INFO:teuthology.orchestra.run.smithi094.stdout: "mappings": [ 2024-09-14T06:32:46.845 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:46.846 INFO:teuthology.orchestra.run.smithi094.stdout: "from": 4, 2024-09-14T06:32:46.846 INFO:teuthology.orchestra.run.smithi094.stdout: "to": 1 2024-09-14T06:32:46.846 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:46.846 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:32:46.846 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:46.846 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:46.846 INFO:teuthology.orchestra.run.smithi094.stdout: "pgid": "3.11", 2024-09-14T06:32:46.846 INFO:teuthology.orchestra.run.smithi094.stdout: "mappings": [ 2024-09-14T06:32:46.846 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:46.846 INFO:teuthology.orchestra.run.smithi094.stdout: "from": 4, 2024-09-14T06:32:46.846 INFO:teuthology.orchestra.run.smithi094.stdout: "to": 1 2024-09-14T06:32:46.846 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:46.846 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:32:46.846 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:46.846 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:46.847 INFO:teuthology.orchestra.run.smithi094.stdout: "pgid": "3.14", 2024-09-14T06:32:46.847 INFO:teuthology.orchestra.run.smithi094.stdout: "mappings": [ 2024-09-14T06:32:46.847 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:46.847 INFO:teuthology.orchestra.run.smithi094.stdout: "from": 4, 2024-09-14T06:32:46.847 INFO:teuthology.orchestra.run.smithi094.stdout: "to": 2 2024-09-14T06:32:46.847 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:46.847 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:32:46.847 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:46.847 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:46.847 INFO:teuthology.orchestra.run.smithi094.stdout: "pgid": "3.1c", 2024-09-14T06:32:46.847 INFO:teuthology.orchestra.run.smithi094.stdout: "mappings": [ 2024-09-14T06:32:46.847 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:46.847 INFO:teuthology.orchestra.run.smithi094.stdout: "from": 4, 2024-09-14T06:32:46.847 INFO:teuthology.orchestra.run.smithi094.stdout: "to": 1 2024-09-14T06:32:46.847 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:46.847 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:32:46.848 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:46.848 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-09-14T06:32:46.848 INFO:teuthology.orchestra.run.smithi094.stdout: "pg_upmap_primaries": [], 2024-09-14T06:32:46.848 INFO:teuthology.orchestra.run.smithi094.stdout: "pg_temp": [], 2024-09-14T06:32:46.848 INFO:teuthology.orchestra.run.smithi094.stdout: "primary_temp": [], 2024-09-14T06:32:46.848 INFO:teuthology.orchestra.run.smithi094.stdout: "blocklist": {}, 2024-09-14T06:32:46.848 INFO:teuthology.orchestra.run.smithi094.stdout: "range_blocklist": {}, 2024-09-14T06:32:46.848 INFO:teuthology.orchestra.run.smithi094.stdout: "erasure_code_profiles": { 2024-09-14T06:32:46.848 INFO:teuthology.orchestra.run.smithi094.stdout: "default": { 2024-09-14T06:32:46.848 INFO:teuthology.orchestra.run.smithi094.stdout: "crush-failure-domain": "osd", 2024-09-14T06:32:46.848 INFO:teuthology.orchestra.run.smithi094.stdout: "k": "2", 2024-09-14T06:32:46.848 INFO:teuthology.orchestra.run.smithi094.stdout: "m": "1", 2024-09-14T06:32:46.848 INFO:teuthology.orchestra.run.smithi094.stdout: "plugin": "jerasure", 2024-09-14T06:32:46.848 INFO:teuthology.orchestra.run.smithi094.stdout: "technique": "reed_sol_van" 2024-09-14T06:32:46.848 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:46.849 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:46.849 INFO:teuthology.orchestra.run.smithi094.stdout: "removed_snaps_queue": [ 2024-09-14T06:32:46.849 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:46.849 INFO:teuthology.orchestra.run.smithi094.stdout: "pool": 3, 2024-09-14T06:32:46.849 INFO:teuthology.orchestra.run.smithi094.stdout: "snaps": [ 2024-09-14T06:32:46.849 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:46.849 INFO:teuthology.orchestra.run.smithi094.stdout: "begin": 33, 2024-09-14T06:32:46.849 INFO:teuthology.orchestra.run.smithi094.stdout: "length": 1 2024-09-14T06:32:46.849 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:46.849 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:46.849 INFO:teuthology.orchestra.run.smithi094.stdout: "begin": 35, 2024-09-14T06:32:46.849 INFO:teuthology.orchestra.run.smithi094.stdout: "length": 1 2024-09-14T06:32:46.849 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:46.849 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:46.850 INFO:teuthology.orchestra.run.smithi094.stdout: "begin": 37, 2024-09-14T06:32:46.850 INFO:teuthology.orchestra.run.smithi094.stdout: "length": 1 2024-09-14T06:32:46.850 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:46.850 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:46.850 INFO:teuthology.orchestra.run.smithi094.stdout: "begin": 39, 2024-09-14T06:32:46.850 INFO:teuthology.orchestra.run.smithi094.stdout: "length": 1 2024-09-14T06:32:46.850 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:46.850 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:46.850 INFO:teuthology.orchestra.run.smithi094.stdout: "begin": 41, 2024-09-14T06:32:46.850 INFO:teuthology.orchestra.run.smithi094.stdout: "length": 1 2024-09-14T06:32:46.850 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:46.850 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:46.850 INFO:teuthology.orchestra.run.smithi094.stdout: "begin": 43, 2024-09-14T06:32:46.850 INFO:teuthology.orchestra.run.smithi094.stdout: "length": 1 2024-09-14T06:32:46.850 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:46.850 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:46.851 INFO:teuthology.orchestra.run.smithi094.stdout: "begin": 45, 2024-09-14T06:32:46.851 INFO:teuthology.orchestra.run.smithi094.stdout: "length": 1 2024-09-14T06:32:46.851 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:46.851 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:46.851 INFO:teuthology.orchestra.run.smithi094.stdout: "begin": 47, 2024-09-14T06:32:46.851 INFO:teuthology.orchestra.run.smithi094.stdout: "length": 1 2024-09-14T06:32:46.851 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:46.851 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:46.851 INFO:teuthology.orchestra.run.smithi094.stdout: "begin": 49, 2024-09-14T06:32:46.851 INFO:teuthology.orchestra.run.smithi094.stdout: "length": 1 2024-09-14T06:32:46.851 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:46.851 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:32:46.851 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:46.851 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-09-14T06:32:46.851 INFO:teuthology.orchestra.run.smithi094.stdout: "new_removed_snaps": [ 2024-09-14T06:32:46.852 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:46.852 INFO:teuthology.orchestra.run.smithi094.stdout: "pool": 3, 2024-09-14T06:32:46.852 INFO:teuthology.orchestra.run.smithi094.stdout: "snaps": [ 2024-09-14T06:32:46.852 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:46.852 INFO:teuthology.orchestra.run.smithi094.stdout: "begin": 33, 2024-09-14T06:32:46.852 INFO:teuthology.orchestra.run.smithi094.stdout: "length": 1 2024-09-14T06:32:46.852 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:46.852 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:32:46.852 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:46.852 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-09-14T06:32:46.852 INFO:teuthology.orchestra.run.smithi094.stdout: "new_purged_snaps": [], 2024-09-14T06:32:46.852 INFO:teuthology.orchestra.run.smithi094.stdout: "crush_node_flags": {}, 2024-09-14T06:32:46.852 INFO:teuthology.orchestra.run.smithi094.stdout: "device_class_flags": {}, 2024-09-14T06:32:46.852 INFO:teuthology.orchestra.run.smithi094.stdout: "stretch_mode": { 2024-09-14T06:32:46.853 INFO:teuthology.orchestra.run.smithi094.stdout: "stretch_mode_enabled": false, 2024-09-14T06:32:46.853 INFO:teuthology.orchestra.run.smithi094.stdout: "stretch_bucket_count": 0, 2024-09-14T06:32:46.853 INFO:teuthology.orchestra.run.smithi094.stdout: "degraded_stretch_mode": 0, 2024-09-14T06:32:46.853 INFO:teuthology.orchestra.run.smithi094.stdout: "recovering_stretch_mode": 0, 2024-09-14T06:32:46.853 INFO:teuthology.orchestra.run.smithi094.stdout: "stretch_mode_bucket": 0 2024-09-14T06:32:46.853 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:46.853 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-14T06:32:46.855 INFO:teuthology.orchestra.run.smithi138.stdout:ERROR: (22) Invalid argument 2024-09-14T06:32:46.855 INFO:teuthology.orchestra.run.smithi138.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:32:46.856 INFO:tasks.thrashosds.thrasher:j is {'epoch': 241, 'fsid': '200ad3da-bd63-4e04-af74-efb5ca0af75e', 'created': '2024-09-14T06:28:30.782046+0000', 'modified': '2024-09-14T06:32:46.612244+0000', 'last_up_change': '2024-09-14T06:28:42.827239+0000', 'last_in_change': '2024-09-14T06:32:00.601027+0000', 'flags': 'sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit', 'flags_num': 5799936, 'flags_set': ['pglog_hardlimit', 'purged_snapdirs', 'recovery_deletes', 'sortbitwise'], 'crush_version': 5, 'full_ratio': 0.949999988079071, 'backfillfull_ratio': 0.8999999761581421, 'nearfull_ratio': 0.8500000238418579, 'cluster_snapshot': '', 'pool_max': 3, 'max_osd': 8, 'require_min_compat_client': 'luminous', 'min_compat_client': 'luminous', 'require_osd_release': 'squid', 'allow_crimson': False, 'pools': [{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-14T06:28:43.295990+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 6.900000095367432, 'score_stable': 6.900000095367432, 'optimal_score': 0.28999999165534973, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 0.8551368713378906, 'average_primary_affinity': 0.9800000190734863, 'average_primary_affinity_weighted': 0.8600000143051147}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2024-09-14T06:28:49.451714+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 2, 'snap_epoch': 22, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.630000114440918, 'score_stable': 2.630000114440918, 'optimal_score': 1, 'raw_score_acting': 2.630000114440918, 'raw_score_stable': 2.630000114440918, 'primary_affinity_weighted': 0.8551368713378906, 'average_primary_affinity': 0.9800000190734863, 'average_primary_affinity_weighted': 0.8600000143051147}}, {'pool': 3, 'pool_name': 'pool_client.0', 'create_time': '2024-09-14T06:28:59.688535+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '241', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 92, 'snap_epoch': 241, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 0.8551368713378906, 'average_primary_affinity': 0.9800000190734863, 'average_primary_affinity_weighted': 0.8600000143051147}}], 'osds': [{'osd': 0, 'uuid': '8b22a249-b842-406a-a02c-c0f31f57b635', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 199, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.94:6824', 'nonce': 1809564923}, {'type': 'v1', 'addr': '172.21.15.94:6825', 'nonce': 1809564923}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.94:6826', 'nonce': 1809564923}, {'type': 'v1', 'addr': '172.21.15.94:6827', 'nonce': 1809564923}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.94:6830', 'nonce': 1809564923}, {'type': 'v1', 'addr': '172.21.15.94:6831', 'nonce': 1809564923}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.94:6828', 'nonce': 1809564923}, {'type': 'v1', 'addr': '172.21.15.94:6829', 'nonce': 1809564923}]}, 'public_addr': '172.21.15.94:6825/1809564923', 'cluster_addr': '172.21.15.94:6827/1809564923', 'heartbeat_back_addr': '172.21.15.94:6831/1809564923', 'heartbeat_front_addr': '172.21.15.94:6829/1809564923', 'state': ['exists', 'up']}, {'osd': 1, 'uuid': '02251a03-84a0-414d-a980-fac4fd043cb6', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 0.841094970703125, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 233, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.94:6816', 'nonce': 736402652}, {'type': 'v1', 'addr': '172.21.15.94:6817', 'nonce': 736402652}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.94:6818', 'nonce': 736402652}, {'type': 'v1', 'addr': '172.21.15.94:6819', 'nonce': 736402652}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.94:6822', 'nonce': 736402652}, {'type': 'v1', 'addr': '172.21.15.94:6823', 'nonce': 736402652}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.94:6820', 'nonce': 736402652}, {'type': 'v1', 'addr': '172.21.15.94:6821', 'nonce': 736402652}]}, 'public_addr': '172.21.15.94:6817/736402652', 'cluster_addr': '172.21.15.94:6819/736402652', 'heartbeat_back_addr': '172.21.15.94:6823/736402652', 'heartbeat_front_addr': '172.21.15.94:6821/736402652', 'state': ['exists', 'up']}, {'osd': 2, 'uuid': 'd45bf470-bb29-4739-8294-cd122b2b4102', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 233, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.94:6808', 'nonce': 1063204782}, {'type': 'v1', 'addr': '172.21.15.94:6809', 'nonce': 1063204782}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.94:6810', 'nonce': 1063204782}, {'type': 'v1', 'addr': '172.21.15.94:6811', 'nonce': 1063204782}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.94:6814', 'nonce': 1063204782}, {'type': 'v1', 'addr': '172.21.15.94:6815', 'nonce': 1063204782}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.94:6812', 'nonce': 1063204782}, {'type': 'v1', 'addr': '172.21.15.94:6813', 'nonce': 1063204782}]}, 'public_addr': '172.21.15.94:6809/1063204782', 'cluster_addr': '172.21.15.94:6811/1063204782', 'heartbeat_back_addr': '172.21.15.94:6815/1063204782', 'heartbeat_front_addr': '172.21.15.94:6813/1063204782', 'state': ['exists', 'up']}, {'osd': 3, 'uuid': '456ec2d9-c438-4243-b931-335704f661eb', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 218, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.94:6800', 'nonce': 3382046754}, {'type': 'v1', 'addr': '172.21.15.94:6801', 'nonce': 3382046754}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.94:6802', 'nonce': 3382046754}, {'type': 'v1', 'addr': '172.21.15.94:6803', 'nonce': 3382046754}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.94:6806', 'nonce': 3382046754}, {'type': 'v1', 'addr': '172.21.15.94:6807', 'nonce': 3382046754}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.94:6804', 'nonce': 3382046754}, {'type': 'v1', 'addr': '172.21.15.94:6805', 'nonce': 3382046754}]}, 'public_addr': '172.21.15.94:6801/3382046754', 'cluster_addr': '172.21.15.94:6803/3382046754', 'heartbeat_back_addr': '172.21.15.94:6807/3382046754', 'heartbeat_front_addr': '172.21.15.94:6805/3382046754', 'state': ['exists', 'up']}, {'osd': 4, 'uuid': '2dab948e-c6e4-4437-bf17-017d1ac6821c', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 232, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.138:6810', 'nonce': 501726729}, {'type': 'v1', 'addr': '172.21.15.138:6811', 'nonce': 501726729}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.138:6812', 'nonce': 501726729}, {'type': 'v1', 'addr': '172.21.15.138:6813', 'nonce': 501726729}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.138:6816', 'nonce': 501726729}, {'type': 'v1', 'addr': '172.21.15.138:6817', 'nonce': 501726729}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.138:6814', 'nonce': 501726729}, {'type': 'v1', 'addr': '172.21.15.138:6815', 'nonce': 501726729}]}, 'public_addr': '172.21.15.138:6811/501726729', 'cluster_addr': '172.21.15.138:6813/501726729', 'heartbeat_back_addr': '172.21.15.138:6817/501726729', 'heartbeat_front_addr': '172.21.15.138:6815/501726729', 'state': ['exists', 'up']}, {'osd': 5, 'uuid': '8caf3aa3-ad87-4d33-b82e-4c15195176fd', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 0, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 199, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.138:6826', 'nonce': 1599292735}, {'type': 'v1', 'addr': '172.21.15.138:6827', 'nonce': 1599292735}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.138:6828', 'nonce': 1599292735}, {'type': 'v1', 'addr': '172.21.15.138:6829', 'nonce': 1599292735}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.138:6832', 'nonce': 1599292735}, {'type': 'v1', 'addr': '172.21.15.138:6833', 'nonce': 1599292735}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.138:6830', 'nonce': 1599292735}, {'type': 'v1', 'addr': '172.21.15.138:6831', 'nonce': 1599292735}]}, 'public_addr': '172.21.15.138:6827/1599292735', 'cluster_addr': '172.21.15.138:6829/1599292735', 'heartbeat_back_addr': '172.21.15.138:6833/1599292735', 'heartbeat_front_addr': '172.21.15.138:6831/1599292735', 'state': ['exists', 'up']}, {'osd': 6, 'uuid': 'e55d11fd-433d-4462-93a7-d5e84560221b', 'up': 1, 'in': 0, 'weight': 0, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 176, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.138:6818', 'nonce': 163531479}, {'type': 'v1', 'addr': '172.21.15.138:6819', 'nonce': 163531479}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.138:6820', 'nonce': 163531479}, {'type': 'v1', 'addr': '172.21.15.138:6821', 'nonce': 163531479}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.138:6824', 'nonce': 163531479}, {'type': 'v1', 'addr': '172.21.15.138:6825', 'nonce': 163531479}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.138:6822', 'nonce': 163531479}, {'type': 'v1', 'addr': '172.21.15.138:6823', 'nonce': 163531479}]}, 'public_addr': '172.21.15.138:6819/163531479', 'cluster_addr': '172.21.15.138:6821/163531479', 'heartbeat_back_addr': '172.21.15.138:6825/163531479', 'heartbeat_front_addr': '172.21.15.138:6823/163531479', 'state': ['exists', 'up']}, {'osd': 7, 'uuid': 'ba14ba69-2c6c-4d09-81e2-2593dc3a7ea9', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 233, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.138:6800', 'nonce': 218507235}, {'type': 'v1', 'addr': '172.21.15.138:6801', 'nonce': 218507235}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.138:6802', 'nonce': 218507235}, {'type': 'v1', 'addr': '172.21.15.138:6803', 'nonce': 218507235}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.138:6806', 'nonce': 218507235}, {'type': 'v1', 'addr': '172.21.15.138:6807', 'nonce': 218507235}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.138:6804', 'nonce': 218507235}, {'type': 'v1', 'addr': '172.21.15.138:6805', 'nonce': 218507235}]}, 'public_addr': '172.21.15.138:6801/218507235', 'cluster_addr': '172.21.15.138:6803/218507235', 'heartbeat_back_addr': '172.21.15.138:6807/218507235', 'heartbeat_front_addr': '172.21.15.138:6805/218507235', 'state': ['exists', 'up']}], 'osd_xinfo': [{'osd': 0, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-14T06:28:39.906639+0000', 'dead_epoch': 0}, {'osd': 1, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-14T06:28:40.541279+0000', 'dead_epoch': 0}, {'osd': 2, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-14T06:28:40.025350+0000', 'dead_epoch': 0}, {'osd': 3, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-14T06:28:39.217944+0000', 'dead_epoch': 0}, {'osd': 4, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-14T06:28:40.185898+0000', 'dead_epoch': 0}, {'osd': 5, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-14T06:28:40.783504+0000', 'dead_epoch': 0}, {'osd': 6, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 65536, 'last_purged_snaps_scrub': '2024-09-14T06:28:40.527305+0000', 'dead_epoch': 0}, {'osd': 7, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-14T06:28:40.349676+0000', 'dead_epoch': 0}], 'pg_upmap': [], 'pg_upmap_items': [{'pgid': '3.e', 'mappings': [{'from': 4, 'to': 1}]}, {'pgid': '3.11', 'mappings': [{'from': 4, 'to': 1}]}, {'pgid': '3.14', 'mappings': [{'from': 4, 'to': 2}]}, {'pgid': '3.1c', 'mappings': [{'from': 4, 'to': 1}]}], 'pg_upmap_primaries': [], 'pg_temp': [], 'primary_temp': [], 'blocklist': {}, 'range_blocklist': {}, 'erasure_code_profiles': {'default': {'crush-failure-domain': 'osd', 'k': '2', 'm': '1', 'plugin': 'jerasure', 'technique': 'reed_sol_van'}}, 'removed_snaps_queue': [{'pool': 3, 'snaps': [{'begin': 33, 'length': 1}, {'begin': 35, 'length': 1}, {'begin': 37, 'length': 1}, {'begin': 39, 'length': 1}, {'begin': 41, 'length': 1}, {'begin': 43, 'length': 1}, {'begin': 45, 'length': 1}, {'begin': 47, 'length': 1}, {'begin': 49, 'length': 1}]}], 'new_removed_snaps': [{'pool': 3, 'snaps': [{'begin': 33, 'length': 1}]}], 'new_purged_snaps': [], 'crush_node_flags': {}, 'device_class_flags': {}, 'stretch_mode': {'stretch_mode_enabled': False, 'stretch_bucket_count': 0, 'degraded_stretch_mode': 0, 'recovering_stretch_mode': 0, 'stretch_mode_bucket': 0}} 2024-09-14T06:32:46.856 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-14T06:32:46.857 INFO:tasks.ceph.osd.2.smithi094.stderr:2024-09-14T06:32:46.853+0000 7faf3269b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33649) UID: 0 2024-09-14T06:32:46.857 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-14T06:32:46.955 INFO:tasks.ceph.osd.5.smithi138.stderr:2024-09-14T06:32:46.954+0000 7fe13746f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33618) UID: 0 2024-09-14T06:32:46.998 INFO:teuthology.orchestra.run.smithi138.stdout:ERROR: (22) Invalid argument 2024-09-14T06:32:46.998 INFO:teuthology.orchestra.run.smithi138.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:32:47.007 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-14T06:32:47.056 INFO:tasks.ceph.osd.1.smithi094.stderr:2024-09-14T06:32:47.055+0000 7f328e47a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33641) UID: 0 2024-09-14T06:32:47.150 INFO:teuthology.orchestra.run.smithi138.stdout:ERROR: (22) Invalid argument 2024-09-14T06:32:47.150 INFO:teuthology.orchestra.run.smithi138.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:32:47.156 INFO:tasks.ceph.osd.5.smithi138.stderr:2024-09-14T06:32:47.155+0000 7fe13746f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33618) UID: 0 2024-09-14T06:32:47.159 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-14T06:32:47.182 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-14T06:32:47.183 INFO:teuthology.orchestra.run.smithi094.stderr:dumped all 2024-09-14T06:32:47.194 INFO:teuthology.orchestra.run.smithi094.stdout:{"pg_ready":true,"pg_map":{"version":357,"stamp":"2024-09-14T06:32:46.614169+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":907951857,"num_objects":1118,"num_object_clones":0,"num_object_copies":2236,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1118,"num_whiteouts":0,"num_read":236328,"num_read_kb":2181694,"num_write":36538,"num_write_kb":1479776,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5149,"num_bytes_recovered":633091608,"num_keys_recovered":534,"num_objects_omap":19,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":26870,"ondisk_log_size":26870,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":7,"num_per_pool_osds":7,"num_per_pool_omap_osds":7,"kb":660602880,"kb_used":518352,"kb_used_data":84576,"kb_used_omap":10,"kb_used_meta":433333,"kb_avail":660084528,"statfs":{"total":676457349120,"available":675926556672,"internally_reserved":0,"allocated":86605824,"data_stored":664017061,"data_compressed":39932661,"data_compressed_allocated":52813824,"data_compressed_original":643358029,"omap_allocated":11157,"internal_metadata":443733099},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-238721829,"num_objects":-219,"num_object_clones":0,"num_object_copies":-438,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-219,"num_whiteouts":0,"num_read":1217,"num_read_kb":1044,"num_write":406,"num_write_kb":74,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":148,"num_bytes_recovered":6082709,"num_keys_recovered":11,"num_objects_omap":-6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"3.999232"},"pg_stats":[{"pgid":"3.1f","version":"237'798","reported_seq":8105,"reported_epoch":237,"state":"active+clean","last_fresh":"2024-09-14T06:32:42.723600+0000","last_change":"2024-09-14T06:32:42.622370+0000","last_active":"2024-09-14T06:32:42.723600+0000","last_peered":"2024-09-14T06:32:42.723600+0000","last_clean":"2024-09-14T06:32:42.723600+0000","last_became_active":"2024-09-14T06:29:00.579883+0000","last_became_peered":"2024-09-14T06:29:00.579883+0000","last_unstale":"2024-09-14T06:32:42.723600+0000","last_undegraded":"2024-09-14T06:32:42.723600+0000","last_fullsized":"2024-09-14T06:32:42.723600+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":798,"log_dups_size":0,"ondisk_log_size":798,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T14:44:05.442430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015915510000000001,"stat_sum":{"num_bytes":25212990,"num_objects":33,"num_object_clones":0,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":33,"num_whiteouts":0,"num_read":8292,"num_read_kb":86746,"num_write":1122,"num_write_kb":49537,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"29","length":"1"},{"start":"2b","length":"1"},{"start":"2d","length":"1"},{"start":"2f","length":"1"},{"start":"31","length":"1"},{"start":"33","length":"1"}]},{"pgid":"3.1e","version":"240'766","reported_seq":10020,"reported_epoch":240,"state":"active+clean","last_fresh":"2024-09-14T06:32:45.710763+0000","last_change":"2024-09-14T06:32:45.616743+0000","last_active":"2024-09-14T06:32:45.710763+0000","last_peered":"2024-09-14T06:32:45.710763+0000","last_clean":"2024-09-14T06:32:45.710763+0000","last_became_active":"2024-09-14T06:32:17.679198+0000","last_became_peered":"2024-09-14T06:32:17.679198+0000","last_unstale":"2024-09-14T06:32:45.710763+0000","last_undegraded":"2024-09-14T06:32:45.710763+0000","last_fullsized":"2024-09-14T06:32:45.710763+0000","mapping_epoch":211,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":212,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":766,"log_dups_size":0,"ondisk_log_size":766,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T16:26:58.408347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014875559999999999,"stat_sum":{"num_bytes":18807079,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":0,"num_read":8313,"num_read_kb":67656,"num_write":1009,"num_write_kb":40407,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":713,"num_bytes_recovered":87148703,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"23","length":"1"},{"start":"25","length":"1"},{"start":"27","length":"1"},{"start":"29","length":"1"},{"start":"2b","length":"1"},{"start":"2d","length":"1"},{"start":"2f","length":"1"},{"start":"31","length":"1"}]},{"pgid":"3.1d","version":"234'993","reported_seq":7186,"reported_epoch":238,"state":"active+clean","last_fresh":"2024-09-14T06:32:43.621605+0000","last_change":"2024-09-14T06:32:43.621824+0000","last_active":"2024-09-14T06:32:43.621605+0000","last_peered":"2024-09-14T06:32:43.621605+0000","last_clean":"2024-09-14T06:32:43.621605+0000","last_became_active":"2024-09-14T06:32:24.716363+0000","last_became_peered":"2024-09-14T06:32:24.716363+0000","last_unstale":"2024-09-14T06:32:43.621605+0000","last_undegraded":"2024-09-14T06:32:43.621605+0000","last_fullsized":"2024-09-14T06:32:43.621605+0000","mapping_epoch":218,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":219,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":993,"log_dups_size":0,"ondisk_log_size":993,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T07:10:14.350088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00081397399999999997,"stat_sum":{"num_bytes":42985469,"num_objects":38,"num_object_clones":0,"num_object_copies":76,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":38,"num_whiteouts":0,"num_read":7636,"num_read_kb":64175,"num_write":1475,"num_write_kb":53276,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"27","length":"1"},{"start":"29","length":"1"},{"start":"2b","length":"1"},{"start":"2d","length":"1"},{"start":"2f","length":"1"},{"start":"31","length":"1"},{"start":"33","length":"1"}]},{"pgid":"3.1c","version":"238'720","reported_seq":5671,"reported_epoch":238,"state":"active+clean","last_fresh":"2024-09-14T06:32:43.723861+0000","last_change":"2024-09-14T06:32:43.618395+0000","last_active":"2024-09-14T06:32:43.723861+0000","last_peered":"2024-09-14T06:32:43.723861+0000","last_clean":"2024-09-14T06:32:43.723861+0000","last_became_active":"2024-09-14T06:32:39.775788+0000","last_became_peered":"2024-09-14T06:32:39.775788+0000","last_unstale":"2024-09-14T06:32:43.723861+0000","last_undegraded":"2024-09-14T06:32:43.723861+0000","last_fullsized":"2024-09-14T06:32:43.723861+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":720,"log_dups_size":0,"ondisk_log_size":720,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T06:34:45.769015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097621199999999996,"stat_sum":{"num_bytes":17726779,"num_objects":44,"num_object_clones":0,"num_object_copies":88,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":44,"num_whiteouts":0,"num_read":4689,"num_read_kb":43569,"num_write":851,"num_write_kb":45545,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":505,"num_bytes_recovered":44145197,"num_keys_recovered":280,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"27","length":"1"},{"start":"29","length":"1"},{"start":"2b","length":"1"},{"start":"2d","length":"1"},{"start":"2f","length":"1"},{"start":"31","length":"1"},{"start":"33","length":"1"}]},{"pgid":"3.1b","version":"236'910","reported_seq":11274,"reported_epoch":237,"state":"active+clean","last_fresh":"2024-09-14T06:32:42.621027+0000","last_change":"2024-09-14T06:32:42.621201+0000","last_active":"2024-09-14T06:32:42.621027+0000","last_peered":"2024-09-14T06:32:42.621027+0000","last_clean":"2024-09-14T06:32:42.621027+0000","last_became_active":"2024-09-14T06:29:00.586183+0000","last_became_peered":"2024-09-14T06:29:00.586183+0000","last_unstale":"2024-09-14T06:32:42.621027+0000","last_undegraded":"2024-09-14T06:32:42.621027+0000","last_fullsized":"2024-09-14T06:32:42.621027+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":910,"log_dups_size":0,"ondisk_log_size":910,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T09:43:18.628500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062806799999999996,"stat_sum":{"num_bytes":20631325,"num_objects":49,"num_object_clones":0,"num_object_copies":98,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":49,"num_whiteouts":0,"num_read":11906,"num_read_kb":108092,"num_write":1414,"num_write_kb":55102,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"29","length":"1"},{"start":"2b","length":"1"},{"start":"2d","length":"1"},{"start":"2f","length":"1"},{"start":"31","length":"1"},{"start":"33","length":"1"}]},{"pgid":"3.1a","version":"237'1585","reported_seq":10165,"reported_epoch":238,"state":"active+clean","last_fresh":"2024-09-14T06:32:43.727460+0000","last_change":"2024-09-14T06:32:43.621551+0000","last_active":"2024-09-14T06:32:43.727460+0000","last_peered":"2024-09-14T06:32:43.727460+0000","last_clean":"2024-09-14T06:32:43.727460+0000","last_became_active":"2024-09-14T06:29:00.576206+0000","last_became_peered":"2024-09-14T06:29:00.576206+0000","last_unstale":"2024-09-14T06:32:43.727460+0000","last_undegraded":"2024-09-14T06:32:43.727460+0000","last_fullsized":"2024-09-14T06:32:43.727460+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":1585,"log_dups_size":0,"ondisk_log_size":1585,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T17:39:46.417447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059520000000000005,"stat_sum":{"num_bytes":31076772,"num_objects":47,"num_object_clones":0,"num_object_copies":94,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":47,"num_whiteouts":0,"num_read":12993,"num_read_kb":82597,"num_write":2363,"num_write_kb":55671,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"27","length":"1"},{"start":"29","length":"1"},{"start":"2b","length":"1"},{"start":"2d","length":"1"},{"start":"2f","length":"1"},{"start":"31","length":"1"},{"start":"33","length":"1"}]},{"pgid":"3.19","version":"238'509","reported_seq":6258,"reported_epoch":238,"state":"active+clean","last_fresh":"2024-09-14T06:32:43.618317+0000","last_change":"2024-09-14T06:32:43.618317+0000","last_active":"2024-09-14T06:32:43.618317+0000","last_peered":"2024-09-14T06:32:43.618317+0000","last_clean":"2024-09-14T06:32:43.618317+0000","last_became_active":"2024-09-14T06:31:53.744800+0000","last_became_peered":"2024-09-14T06:31:53.744800+0000","last_unstale":"2024-09-14T06:32:43.618317+0000","last_undegraded":"2024-09-14T06:32:43.618317+0000","last_fullsized":"2024-09-14T06:32:43.618317+0000","mapping_epoch":188,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":189,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":509,"log_dups_size":0,"ondisk_log_size":509,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T08:37:31.957932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011132309999999999,"stat_sum":{"num_bytes":34253926,"num_objects":40,"num_object_clones":0,"num_object_copies":80,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":40,"num_whiteouts":0,"num_read":5087,"num_read_kb":67755,"num_write":711,"num_write_kb":43868,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":282,"num_bytes_recovered":41927489,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"27","length":"1"},{"start":"29","length":"1"},{"start":"2b","length":"1"},{"start":"2d","length":"1"},{"start":"2f","length":"1"},{"start":"31","length":"1"},{"start":"33","length":"1"}]},{"pgid":"3.18","version":"239'507","reported_seq":6070,"reported_epoch":240,"state":"active+clean","last_fresh":"2024-09-14T06:32:45.614930+0000","last_change":"2024-09-14T06:32:45.615125+0000","last_active":"2024-09-14T06:32:45.614930+0000","last_peered":"2024-09-14T06:32:45.614930+0000","last_clean":"2024-09-14T06:32:45.614930+0000","last_became_active":"2024-09-14T06:31:53.745159+0000","last_became_peered":"2024-09-14T06:31:53.745159+0000","last_unstale":"2024-09-14T06:32:45.614930+0000","last_undegraded":"2024-09-14T06:32:45.614930+0000","last_fullsized":"2024-09-14T06:32:45.614930+0000","mapping_epoch":188,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":189,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":507,"log_dups_size":0,"ondisk_log_size":507,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T06:56:44.217447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00091913499999999996,"stat_sum":{"num_bytes":20393701,"num_objects":27,"num_object_clones":0,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":27,"num_whiteouts":0,"num_read":4458,"num_read_kb":78893,"num_write":632,"num_write_kb":51576,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":310,"num_bytes_recovered":56082976,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"23","length":"1"},{"start":"25","length":"1"},{"start":"27","length":"1"},{"start":"29","length":"1"},{"start":"2b","length":"1"},{"start":"2d","length":"1"},{"start":"2f","length":"1"},{"start":"31","length":"1"}]},{"pgid":"3.17","version":"237'931","reported_seq":9281,"reported_epoch":237,"state":"active+clean","last_fresh":"2024-09-14T06:32:42.716907+0000","last_change":"2024-09-14T06:32:42.621220+0000","last_active":"2024-09-14T06:32:42.716907+0000","last_peered":"2024-09-14T06:32:42.716907+0000","last_clean":"2024-09-14T06:32:42.716907+0000","last_became_active":"2024-09-14T06:29:00.579714+0000","last_became_peered":"2024-09-14T06:29:00.579714+0000","last_unstale":"2024-09-14T06:32:42.716907+0000","last_undegraded":"2024-09-14T06:32:42.716907+0000","last_fullsized":"2024-09-14T06:32:42.716907+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":931,"log_dups_size":0,"ondisk_log_size":931,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T07:25:42.063845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00084899900000000002,"stat_sum":{"num_bytes":49692439,"num_objects":52,"num_object_clones":0,"num_object_copies":104,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":52,"num_whiteouts":0,"num_read":10311,"num_read_kb":68638,"num_write":1065,"num_write_kb":48465,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"29","length":"1"},{"start":"2b","length":"1"},{"start":"2d","length":"1"},{"start":"2f","length":"1"},{"start":"31","length":"1"},{"start":"33","length":"1"}]},{"pgid":"3.16","version":"235'1124","reported_seq":3988,"reported_epoch":238,"state":"active+clean","last_fresh":"2024-09-14T06:32:43.622619+0000","last_change":"2024-09-14T06:32:43.622812+0000","last_active":"2024-09-14T06:32:43.622619+0000","last_peered":"2024-09-14T06:32:43.622619+0000","last_clean":"2024-09-14T06:32:43.622619+0000","last_became_active":"2024-09-14T06:32:24.718745+0000","last_became_peered":"2024-09-14T06:32:24.718745+0000","last_unstale":"2024-09-14T06:32:43.622619+0000","last_undegraded":"2024-09-14T06:32:43.622619+0000","last_fullsized":"2024-09-14T06:32:43.622619+0000","mapping_epoch":218,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":219,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":1124,"log_dups_size":0,"ondisk_log_size":1124,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T09:15:35.275119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00086667799999999996,"stat_sum":{"num_bytes":14704807,"num_objects":32,"num_object_clones":0,"num_object_copies":64,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":32,"num_whiteouts":0,"num_read":5658,"num_read_kb":38562,"num_write":1711,"num_write_kb":37248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"27","length":"1"},{"start":"29","length":"1"},{"start":"2b","length":"1"},{"start":"2d","length":"1"},{"start":"2f","length":"1"},{"start":"31","length":"1"},{"start":"33","length":"1"}]},{"pgid":"3.15","version":"236'587","reported_seq":6427,"reported_epoch":238,"state":"active+clean","last_fresh":"2024-09-14T06:32:43.621901+0000","last_change":"2024-09-14T06:32:43.622184+0000","last_active":"2024-09-14T06:32:43.621901+0000","last_peered":"2024-09-14T06:32:43.621901+0000","last_clean":"2024-09-14T06:32:43.621901+0000","last_became_active":"2024-09-14T06:31:53.746091+0000","last_became_peered":"2024-09-14T06:31:53.746091+0000","last_unstale":"2024-09-14T06:32:43.621901+0000","last_undegraded":"2024-09-14T06:32:43.621901+0000","last_fullsized":"2024-09-14T06:32:43.621901+0000","mapping_epoch":188,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":189,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":587,"log_dups_size":0,"ondisk_log_size":587,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T12:56:00.840746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00058041699999999996,"stat_sum":{"num_bytes":34132886,"num_objects":39,"num_object_clones":0,"num_object_copies":78,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":39,"num_whiteouts":0,"num_read":4928,"num_read_kb":74748,"num_write":641,"num_write_kb":51190,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":271,"num_bytes_recovered":49981473,"num_keys_recovered":33,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"27","length":"1"},{"start":"29","length":"1"},{"start":"2b","length":"1"},{"start":"2d","length":"1"},{"start":"2f","length":"1"},{"start":"31","length":"1"},{"start":"33","length":"1"}]},{"pgid":"3.14","version":"237'1083","reported_seq":6012,"reported_epoch":237,"state":"active+clean","last_fresh":"2024-09-14T06:32:42.703832+0000","last_change":"2024-09-14T06:32:42.701857+0000","last_active":"2024-09-14T06:32:42.703832+0000","last_peered":"2024-09-14T06:32:42.703832+0000","last_clean":"2024-09-14T06:32:42.703832+0000","last_became_active":"2024-09-14T06:32:39.776600+0000","last_became_peered":"2024-09-14T06:32:39.776600+0000","last_unstale":"2024-09-14T06:32:42.703832+0000","last_undegraded":"2024-09-14T06:32:42.703832+0000","last_fullsized":"2024-09-14T06:32:42.703832+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":1083,"log_dups_size":0,"ondisk_log_size":1083,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T18:28:14.973851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021024889999999999,"stat_sum":{"num_bytes":17246307,"num_objects":29,"num_object_clones":0,"num_object_copies":58,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":29,"num_whiteouts":0,"num_read":8770,"num_read_kb":46783,"num_write":1528,"num_write_kb":29823,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":148,"num_bytes_recovered":6082709,"num_keys_recovered":11,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"29","length":"1"},{"start":"2b","length":"1"},{"start":"2d","length":"1"},{"start":"2f","length":"1"},{"start":"31","length":"1"},{"start":"33","length":"1"}]},{"pgid":"2.7","version":"0'0","reported_seq":94,"reported_epoch":238,"state":"active+clean","last_fresh":"2024-09-14T06:32:43.621919+0000","last_change":"2024-09-14T06:32:01.612470+0000","last_active":"2024-09-14T06:32:43.621919+0000","last_peered":"2024-09-14T06:32:43.621919+0000","last_clean":"2024-09-14T06:32:43.621919+0000","last_became_active":"2024-09-14T06:32:01.612121+0000","last_became_peered":"2024-09-14T06:32:01.612121+0000","last_unstale":"2024-09-14T06:32:43.621919+0000","last_undegraded":"2024-09-14T06:32:43.621919+0000","last_fullsized":"2024-09-14T06:32:43.621919+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:28:50.205350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:28:50.205350+0000","last_clean_scrub_stamp":"2024-09-14T06:28:50.205350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T13:40:52.537942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.6","version":"237'793","reported_seq":8351,"reported_epoch":237,"state":"active+clean","last_fresh":"2024-09-14T06:32:42.702096+0000","last_change":"2024-09-14T06:32:42.620969+0000","last_active":"2024-09-14T06:32:42.702096+0000","last_peered":"2024-09-14T06:32:42.702096+0000","last_clean":"2024-09-14T06:32:42.702096+0000","last_became_active":"2024-09-14T06:29:00.578283+0000","last_became_peered":"2024-09-14T06:29:00.578283+0000","last_unstale":"2024-09-14T06:32:42.702096+0000","last_undegraded":"2024-09-14T06:32:42.702096+0000","last_fullsized":"2024-09-14T06:32:42.702096+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":793,"log_dups_size":0,"ondisk_log_size":793,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T15:20:25.657119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00078026400000000002,"stat_sum":{"num_bytes":15699474,"num_objects":31,"num_object_clones":0,"num_object_copies":62,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":31,"num_whiteouts":0,"num_read":8373,"num_read_kb":78571,"num_write":850,"num_write_kb":48287,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"29","length":"1"},{"start":"2b","length":"1"},{"start":"2d","length":"1"},{"start":"2f","length":"1"},{"start":"31","length":"1"},{"start":"33","length":"1"}]},{"pgid":"2.6","version":"0'0","reported_seq":459,"reported_epoch":238,"state":"active+clean","last_fresh":"2024-09-14T06:32:43.617992+0000","last_change":"2024-09-14T06:32:01.608456+0000","last_active":"2024-09-14T06:32:43.617992+0000","last_peered":"2024-09-14T06:32:43.617992+0000","last_clean":"2024-09-14T06:32:43.617992+0000","last_became_active":"2024-09-14T06:32:01.608178+0000","last_became_peered":"2024-09-14T06:32:01.608178+0000","last_unstale":"2024-09-14T06:32:43.617992+0000","last_undegraded":"2024-09-14T06:32:43.617992+0000","last_fullsized":"2024-09-14T06:32:43.617992+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:28:50.205350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:28:50.205350+0000","last_clean_scrub_stamp":"2024-09-14T06:28:50.205350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T10:16:40.480031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065018200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"239'794","reported_seq":4359,"reported_epoch":240,"state":"active+clean","last_fresh":"2024-09-14T06:32:45.616237+0000","last_change":"2024-09-14T06:32:45.616441+0000","last_active":"2024-09-14T06:32:45.616237+0000","last_peered":"2024-09-14T06:32:45.616237+0000","last_clean":"2024-09-14T06:32:45.616237+0000","last_became_active":"2024-09-14T06:31:54.747923+0000","last_became_peered":"2024-09-14T06:31:54.747923+0000","last_unstale":"2024-09-14T06:32:45.616237+0000","last_undegraded":"2024-09-14T06:32:45.616237+0000","last_fullsized":"2024-09-14T06:32:45.616237+0000","mapping_epoch":189,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":190,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":794,"log_dups_size":0,"ondisk_log_size":794,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T15:55:14.570071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015576920000000001,"stat_sum":{"num_bytes":22196057,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":4414,"num_read_kb":40976,"num_write":1141,"num_write_kb":36116,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":183,"num_bytes_recovered":13912513,"num_keys_recovered":22,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"23","length":"1"},{"start":"25","length":"1"},{"start":"27","length":"1"},{"start":"29","length":"1"},{"start":"2b","length":"1"},{"start":"2d","length":"1"},{"start":"2f","length":"1"},{"start":"31","length":"1"}]},{"pgid":"2.5","version":"0'0","reported_seq":453,"reported_epoch":238,"state":"active+clean","last_fresh":"2024-09-14T06:32:43.621844+0000","last_change":"2024-09-14T06:28:53.640035+0000","last_active":"2024-09-14T06:32:43.621844+0000","last_peered":"2024-09-14T06:32:43.621844+0000","last_clean":"2024-09-14T06:32:43.621844+0000","last_became_active":"2024-09-14T06:28:51.308058+0000","last_became_peered":"2024-09-14T06:28:51.308058+0000","last_unstale":"2024-09-14T06:32:43.621844+0000","last_undegraded":"2024-09-14T06:32:43.621844+0000","last_fullsized":"2024-09-14T06:32:43.621844+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:28:50.205350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:28:50.205350+0000","last_clean_scrub_stamp":"2024-09-14T06:28:50.205350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T13:35:48.544057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00037824000000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"236'654","reported_seq":3904,"reported_epoch":238,"state":"active+clean","last_fresh":"2024-09-14T06:32:43.618374+0000","last_change":"2024-09-14T06:32:43.618632+0000","last_active":"2024-09-14T06:32:43.618374+0000","last_peered":"2024-09-14T06:32:43.618374+0000","last_clean":"2024-09-14T06:32:43.618374+0000","last_became_active":"2024-09-14T06:32:17.675470+0000","last_became_peered":"2024-09-14T06:32:17.675470+0000","last_unstale":"2024-09-14T06:32:43.618374+0000","last_undegraded":"2024-09-14T06:32:43.618374+0000","last_fullsized":"2024-09-14T06:32:43.618374+0000","mapping_epoch":211,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":212,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":654,"log_dups_size":0,"ondisk_log_size":654,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T13:18:03.757852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088633400000000002,"stat_sum":{"num_bytes":12728367,"num_objects":34,"num_object_clones":0,"num_object_copies":68,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":34,"num_whiteouts":0,"num_read":2623,"num_read_kb":37411,"num_write":850,"num_write_kb":40855,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":331,"num_bytes_recovered":31411119,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"27","length":"1"},{"start":"29","length":"1"},{"start":"2b","length":"1"},{"start":"2d","length":"1"},{"start":"2f","length":"1"},{"start":"31","length":"1"},{"start":"33","length":"1"}]},{"pgid":"2.4","version":"0'0","reported_seq":453,"reported_epoch":238,"state":"active+clean","last_fresh":"2024-09-14T06:32:43.617968+0000","last_change":"2024-09-14T06:28:53.226072+0000","last_active":"2024-09-14T06:32:43.617968+0000","last_peered":"2024-09-14T06:32:43.617968+0000","last_clean":"2024-09-14T06:32:43.617968+0000","last_became_active":"2024-09-14T06:28:51.308072+0000","last_became_peered":"2024-09-14T06:28:51.308072+0000","last_unstale":"2024-09-14T06:32:43.617968+0000","last_undegraded":"2024-09-14T06:32:43.617968+0000","last_fullsized":"2024-09-14T06:32:43.617968+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:28:50.205350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:28:50.205350+0000","last_clean_scrub_stamp":"2024-09-14T06:28:50.205350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T13:15:09.052328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066197399999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"240'871","reported_seq":5465,"reported_epoch":240,"state":"active+clean","last_fresh":"2024-09-14T06:32:45.616133+0000","last_change":"2024-09-14T06:32:45.616133+0000","last_active":"2024-09-14T06:32:45.616133+0000","last_peered":"2024-09-14T06:32:45.616133+0000","last_clean":"2024-09-14T06:32:45.616133+0000","last_became_active":"2024-09-14T06:32:24.716458+0000","last_became_peered":"2024-09-14T06:32:24.716458+0000","last_unstale":"2024-09-14T06:32:45.616133+0000","last_undegraded":"2024-09-14T06:32:45.616133+0000","last_fullsized":"2024-09-14T06:32:45.616133+0000","mapping_epoch":218,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":219,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":871,"log_dups_size":0,"ondisk_log_size":871,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T18:22:52.383908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012874539999999999,"stat_sum":{"num_bytes":21863990,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":4510,"num_read_kb":42630,"num_write":1231,"num_write_kb":49494,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":330,"num_bytes_recovered":28798893,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"23","length":"1"},{"start":"25","length":"1"},{"start":"27","length":"1"},{"start":"29","length":"1"},{"start":"2b","length":"1"},{"start":"2d","length":"1"},{"start":"2f","length":"1"},{"start":"31","length":"1"}]},{"pgid":"3.3","version":"238'588","reported_seq":10470,"reported_epoch":238,"state":"active+clean","last_fresh":"2024-09-14T06:32:43.720879+0000","last_change":"2024-09-14T06:32:43.622014+0000","last_active":"2024-09-14T06:32:43.720879+0000","last_peered":"2024-09-14T06:32:43.720879+0000","last_clean":"2024-09-14T06:32:43.720879+0000","last_became_active":"2024-09-14T06:29:00.584678+0000","last_became_peered":"2024-09-14T06:29:00.584678+0000","last_unstale":"2024-09-14T06:32:43.720879+0000","last_undegraded":"2024-09-14T06:32:43.720879+0000","last_fullsized":"2024-09-14T06:32:43.720879+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":588,"log_dups_size":0,"ondisk_log_size":588,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T13:05:58.655890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00089673599999999997,"stat_sum":{"num_bytes":48374477,"num_objects":29,"num_object_clones":0,"num_object_copies":58,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":29,"num_whiteouts":0,"num_read":9295,"num_read_kb":89064,"num_write":661,"num_write_kb":51631,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"27","length":"1"},{"start":"29","length":"1"},{"start":"2b","length":"1"},{"start":"2d","length":"1"},{"start":"2f","length":"1"},{"start":"31","length":"1"},{"start":"33","length":"1"}]},{"pgid":"2.2","version":"22'2","reported_seq":68,"reported_epoch":238,"state":"active+clean","last_fresh":"2024-09-14T06:32:43.618124+0000","last_change":"2024-09-14T06:32:24.716937+0000","last_active":"2024-09-14T06:32:43.618124+0000","last_peered":"2024-09-14T06:32:43.618124+0000","last_clean":"2024-09-14T06:32:43.618124+0000","last_became_active":"2024-09-14T06:32:24.716539+0000","last_became_peered":"2024-09-14T06:32:24.716539+0000","last_unstale":"2024-09-14T06:32:43.618124+0000","last_undegraded":"2024-09-14T06:32:43.618124+0000","last_fullsized":"2024-09-14T06:32:43.618124+0000","mapping_epoch":218,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":219,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:28:50.205350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:28:50.205350+0000","last_clean_scrub_stamp":"2024-09-14T06:28:50.205350+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T07:16:29.639469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"238'1066","reported_seq":6896,"reported_epoch":238,"state":"active+clean","last_fresh":"2024-09-14T06:32:43.617730+0000","last_change":"2024-09-14T06:32:43.617730+0000","last_active":"2024-09-14T06:32:43.617730+0000","last_peered":"2024-09-14T06:32:43.617730+0000","last_clean":"2024-09-14T06:32:43.617730+0000","last_became_active":"2024-09-14T06:29:00.574873+0000","last_became_peered":"2024-09-14T06:29:00.574873+0000","last_unstale":"2024-09-14T06:32:43.617730+0000","last_undegraded":"2024-09-14T06:32:43.617730+0000","last_fullsized":"2024-09-14T06:32:43.617730+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":1066,"log_dups_size":0,"ondisk_log_size":1066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T08:13:39.775536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061334200000000001,"stat_sum":{"num_bytes":43049554,"num_objects":48,"num_object_clones":0,"num_object_copies":96,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":48,"num_whiteouts":0,"num_read":8253,"num_read_kb":85478,"num_write":1604,"num_write_kb":58700,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"27","length":"1"},{"start":"29","length":"1"},{"start":"2b","length":"1"},{"start":"2d","length":"1"},{"start":"2f","length":"1"},{"start":"31","length":"1"},{"start":"33","length":"1"}]},{"pgid":"2.1","version":"0'0","reported_seq":67,"reported_epoch":237,"state":"active+clean","last_fresh":"2024-09-14T06:32:42.699579+0000","last_change":"2024-09-14T06:32:14.665376+0000","last_active":"2024-09-14T06:32:42.699579+0000","last_peered":"2024-09-14T06:32:42.699579+0000","last_clean":"2024-09-14T06:32:42.699579+0000","last_became_active":"2024-09-14T06:32:14.665081+0000","last_became_peered":"2024-09-14T06:32:14.665081+0000","last_unstale":"2024-09-14T06:32:42.699579+0000","last_undegraded":"2024-09-14T06:32:42.699579+0000","last_fullsized":"2024-09-14T06:32:42.699579+0000","mapping_epoch":208,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":209,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:28:50.205350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:28:50.205350+0000","last_clean_scrub_stamp":"2024-09-14T06:28:50.205350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T16:35:12.869944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053403100000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1","version":"237'739","reported_seq":11529,"reported_epoch":237,"state":"active+clean","last_fresh":"2024-09-14T06:32:42.749374+0000","last_change":"2024-09-14T06:32:42.620695+0000","last_active":"2024-09-14T06:32:42.749374+0000","last_peered":"2024-09-14T06:32:42.749374+0000","last_clean":"2024-09-14T06:32:42.749374+0000","last_became_active":"2024-09-14T06:30:57.768238+0000","last_became_peered":"2024-09-14T06:30:57.768238+0000","last_unstale":"2024-09-14T06:32:42.749374+0000","last_undegraded":"2024-09-14T06:32:42.749374+0000","last_fullsized":"2024-09-14T06:32:42.749374+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":739,"log_dups_size":0,"ondisk_log_size":739,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T18:00:33.732750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054797400000000001,"stat_sum":{"num_bytes":40541999,"num_objects":36,"num_object_clones":0,"num_object_copies":72,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":36,"num_whiteouts":0,"num_read":11055,"num_read_kb":112235,"num_write":915,"num_write_kb":47393,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":183,"num_bytes_recovered":30431994,"num_keys_recovered":22,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"29","length":"1"},{"start":"2b","length":"1"},{"start":"2d","length":"1"},{"start":"2f","length":"1"},{"start":"31","length":"1"},{"start":"33","length":"1"}]},{"pgid":"2.0","version":"0'0","reported_seq":453,"reported_epoch":238,"state":"active+clean","last_fresh":"2024-09-14T06:32:43.621327+0000","last_change":"2024-09-14T06:28:53.640096+0000","last_active":"2024-09-14T06:32:43.621327+0000","last_peered":"2024-09-14T06:32:43.621327+0000","last_clean":"2024-09-14T06:32:43.621327+0000","last_became_active":"2024-09-14T06:28:51.223069+0000","last_became_peered":"2024-09-14T06:28:51.223069+0000","last_unstale":"2024-09-14T06:32:43.621327+0000","last_undegraded":"2024-09-14T06:32:43.621327+0000","last_fullsized":"2024-09-14T06:32:43.621327+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:28:50.205350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:28:50.205350+0000","last_clean_scrub_stamp":"2024-09-14T06:28:50.205350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T15:13:00.471509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00043714600000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"240'1618","reported_seq":7630,"reported_epoch":240,"state":"active+clean","last_fresh":"2024-09-14T06:32:45.727613+0000","last_change":"2024-09-14T06:32:45.615694+0000","last_active":"2024-09-14T06:32:45.727613+0000","last_peered":"2024-09-14T06:32:45.727613+0000","last_clean":"2024-09-14T06:32:45.727613+0000","last_became_active":"2024-09-14T06:31:54.744160+0000","last_became_peered":"2024-09-14T06:31:54.744160+0000","last_unstale":"2024-09-14T06:32:45.727613+0000","last_undegraded":"2024-09-14T06:32:45.727613+0000","last_fullsized":"2024-09-14T06:32:45.727613+0000","mapping_epoch":189,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":190,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":1618,"log_dups_size":0,"ondisk_log_size":1618,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T15:47:19.578642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013338759999999999,"stat_sum":{"num_bytes":42049101,"num_objects":42,"num_object_clones":0,"num_object_copies":84,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":42,"num_whiteouts":0,"num_read":10717,"num_read_kb":70587,"num_write":2554,"num_write_kb":53564,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":327,"num_bytes_recovered":50047063,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"23","length":"1"},{"start":"25","length":"1"},{"start":"27","length":"1"},{"start":"29","length":"1"},{"start":"2b","length":"1"},{"start":"2d","length":"1"},{"start":"2f","length":"1"},{"start":"31","length":"1"}]},{"pgid":"2.3","version":"20'1","reported_seq":52,"reported_epoch":237,"state":"active+clean","last_fresh":"2024-09-14T06:32:42.699608+0000","last_change":"2024-09-14T06:32:24.717893+0000","last_active":"2024-09-14T06:32:42.699608+0000","last_peered":"2024-09-14T06:32:42.699608+0000","last_clean":"2024-09-14T06:32:42.699608+0000","last_became_active":"2024-09-14T06:32:24.717658+0000","last_became_peered":"2024-09-14T06:32:24.717658+0000","last_unstale":"2024-09-14T06:32:42.699608+0000","last_undegraded":"2024-09-14T06:32:42.699608+0000","last_fullsized":"2024-09-14T06:32:42.699608+0000","mapping_epoch":218,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":219,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:28:50.205350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:28:50.205350+0000","last_clean_scrub_stamp":"2024-09-14T06:28:50.205350+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T10:36:16.650799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.0","version":"17'72","reported_seq":544,"reported_epoch":238,"state":"active+clean","last_fresh":"2024-09-14T06:32:43.621370+0000","last_change":"2024-09-14T06:28:44.839885+0000","last_active":"2024-09-14T06:32:43.621370+0000","last_peered":"2024-09-14T06:32:43.621370+0000","last_clean":"2024-09-14T06:32:43.621370+0000","last_became_active":"2024-09-14T06:28:44.839573+0000","last_became_peered":"2024-09-14T06:28:44.839573+0000","last_unstale":"2024-09-14T06:32:43.621370+0000","last_undegraded":"2024-09-14T06:32:43.621370+0000","last_fullsized":"2024-09-14T06:32:43.621370+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:28:43.828494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:28:43.828494+0000","last_clean_scrub_stamp":"2024-09-14T06:28:43.828494+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T14:41:25.787293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.8","version":"239'630","reported_seq":6357,"reported_epoch":240,"state":"active+clean","last_fresh":"2024-09-14T06:32:45.615825+0000","last_change":"2024-09-14T06:32:45.616002+0000","last_active":"2024-09-14T06:32:45.615825+0000","last_peered":"2024-09-14T06:32:45.615825+0000","last_clean":"2024-09-14T06:32:45.615825+0000","last_became_active":"2024-09-14T06:31:55.754694+0000","last_became_peered":"2024-09-14T06:31:55.754694+0000","last_unstale":"2024-09-14T06:32:45.615825+0000","last_undegraded":"2024-09-14T06:32:45.615825+0000","last_fullsized":"2024-09-14T06:32:45.615825+0000","mapping_epoch":190,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":191,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":630,"log_dups_size":0,"ondisk_log_size":630,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T09:33:14.274787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013620310000000001,"stat_sum":{"num_bytes":40534367,"num_objects":27,"num_object_clones":0,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":27,"num_whiteouts":0,"num_read":5616,"num_read_kb":90699,"num_write":878,"num_write_kb":57685,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":319,"num_bytes_recovered":60352269,"num_keys_recovered":22,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"23","length":"1"},{"start":"25","length":"1"},{"start":"27","length":"1"},{"start":"29","length":"1"},{"start":"2b","length":"1"},{"start":"2d","length":"1"},{"start":"2f","length":"1"},{"start":"31","length":"1"}]},{"pgid":"3.9","version":"235'551","reported_seq":4285,"reported_epoch":238,"state":"active+clean","last_fresh":"2024-09-14T06:32:43.621206+0000","last_change":"2024-09-14T06:32:43.621434+0000","last_active":"2024-09-14T06:32:43.621206+0000","last_peered":"2024-09-14T06:32:43.621206+0000","last_clean":"2024-09-14T06:32:43.621206+0000","last_became_active":"2024-09-14T06:29:00.573179+0000","last_became_peered":"2024-09-14T06:29:00.573179+0000","last_unstale":"2024-09-14T06:32:43.621206+0000","last_undegraded":"2024-09-14T06:32:43.621206+0000","last_fullsized":"2024-09-14T06:32:43.621206+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":551,"log_dups_size":0,"ondisk_log_size":551,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T06:40:38.359137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000498223,"stat_sum":{"num_bytes":18711878,"num_objects":42,"num_object_clones":0,"num_object_copies":84,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":42,"num_whiteouts":0,"num_read":3678,"num_read_kb":64291,"num_write":736,"num_write_kb":44928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"27","length":"1"},{"start":"29","length":"1"},{"start":"2b","length":"1"},{"start":"2d","length":"1"},{"start":"2f","length":"1"},{"start":"31","length":"1"},{"start":"33","length":"1"}]},{"pgid":"3.a","version":"238'813","reported_seq":4810,"reported_epoch":238,"state":"active+clean","last_fresh":"2024-09-14T06:32:43.621730+0000","last_change":"2024-09-14T06:32:43.621730+0000","last_active":"2024-09-14T06:32:43.621730+0000","last_peered":"2024-09-14T06:32:43.621730+0000","last_clean":"2024-09-14T06:32:43.621730+0000","last_became_active":"2024-09-14T06:32:04.617283+0000","last_became_peered":"2024-09-14T06:32:04.617283+0000","last_unstale":"2024-09-14T06:32:43.621730+0000","last_undegraded":"2024-09-14T06:32:43.621730+0000","last_fullsized":"2024-09-14T06:32:43.621730+0000","mapping_epoch":199,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":200,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":813,"log_dups_size":0,"ondisk_log_size":813,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T17:18:32.137675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077279300000000003,"stat_sum":{"num_bytes":37097096,"num_objects":36,"num_object_clones":0,"num_object_copies":72,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":36,"num_whiteouts":0,"num_read":5155,"num_read_kb":58177,"num_write":1169,"num_write_kb":47529,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":161,"num_bytes_recovered":21723371,"num_keys_recovered":33,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"27","length":"1"},{"start":"29","length":"1"},{"start":"2b","length":"1"},{"start":"2d","length":"1"},{"start":"2f","length":"1"},{"start":"31","length":"1"},{"start":"33","length":"1"}]},{"pgid":"3.b","version":"240'1597","reported_seq":8766,"reported_epoch":240,"state":"active+clean","last_fresh":"2024-09-14T06:32:45.701596+0000","last_change":"2024-09-14T06:32:45.616508+0000","last_active":"2024-09-14T06:32:45.701596+0000","last_peered":"2024-09-14T06:32:45.701596+0000","last_clean":"2024-09-14T06:32:45.701596+0000","last_became_active":"2024-09-14T06:31:55.756219+0000","last_became_peered":"2024-09-14T06:31:55.756219+0000","last_unstale":"2024-09-14T06:32:45.701596+0000","last_undegraded":"2024-09-14T06:32:45.701596+0000","last_fullsized":"2024-09-14T06:32:45.701596+0000","mapping_epoch":190,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":191,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":1597,"log_dups_size":0,"ondisk_log_size":1597,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T13:14:53.941038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015328919999999999,"stat_sum":{"num_bytes":19633489,"num_objects":27,"num_object_clones":0,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":27,"num_whiteouts":0,"num_read":13006,"num_read_kb":64103,"num_write":2456,"num_write_kb":43674,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":291,"num_bytes_recovered":37370632,"num_keys_recovered":66,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"23","length":"1"},{"start":"25","length":"1"},{"start":"27","length":"1"},{"start":"29","length":"1"},{"start":"2b","length":"1"},{"start":"2d","length":"1"},{"start":"2f","length":"1"},{"start":"31","length":"1"}]},{"pgid":"3.c","version":"239'595","reported_seq":6139,"reported_epoch":240,"state":"active+clean","last_fresh":"2024-09-14T06:32:45.615554+0000","last_change":"2024-09-14T06:32:45.615730+0000","last_active":"2024-09-14T06:32:45.615554+0000","last_peered":"2024-09-14T06:32:45.615554+0000","last_clean":"2024-09-14T06:32:45.615554+0000","last_became_active":"2024-09-14T06:32:24.716421+0000","last_became_peered":"2024-09-14T06:32:24.716421+0000","last_unstale":"2024-09-14T06:32:45.615554+0000","last_undegraded":"2024-09-14T06:32:45.615554+0000","last_fullsized":"2024-09-14T06:32:45.615554+0000","mapping_epoch":218,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":219,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":595,"log_dups_size":0,"ondisk_log_size":595,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T16:29:13.260361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013141590000000001,"stat_sum":{"num_bytes":23576801,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":4842,"num_read_kb":50425,"num_write":772,"num_write_kb":40827,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":293,"num_bytes_recovered":36712421,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"23","length":"1"},{"start":"25","length":"1"},{"start":"27","length":"1"},{"start":"29","length":"1"},{"start":"2b","length":"1"},{"start":"2d","length":"1"},{"start":"2f","length":"1"},{"start":"31","length":"1"}]},{"pgid":"3.d","version":"238'518","reported_seq":9007,"reported_epoch":238,"state":"active+clean","last_fresh":"2024-09-14T06:32:43.699130+0000","last_change":"2024-09-14T06:32:43.622462+0000","last_active":"2024-09-14T06:32:43.699130+0000","last_peered":"2024-09-14T06:32:43.699130+0000","last_clean":"2024-09-14T06:32:43.699130+0000","last_became_active":"2024-09-14T06:29:00.580362+0000","last_became_peered":"2024-09-14T06:29:00.580362+0000","last_unstale":"2024-09-14T06:32:43.699130+0000","last_undegraded":"2024-09-14T06:32:43.699130+0000","last_fullsized":"2024-09-14T06:32:43.699130+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":518,"log_dups_size":0,"ondisk_log_size":518,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T08:43:46.800196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00084532499999999996,"stat_sum":{"num_bytes":27519222,"num_objects":37,"num_object_clones":0,"num_object_copies":74,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":37,"num_whiteouts":0,"num_read":8026,"num_read_kb":93543,"num_write":679,"num_write_kb":41026,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"27","length":"1"},{"start":"29","length":"1"},{"start":"2b","length":"1"},{"start":"2d","length":"1"},{"start":"2f","length":"1"},{"start":"31","length":"1"},{"start":"33","length":"1"}]},{"pgid":"3.e","version":"238'544","reported_seq":5291,"reported_epoch":238,"state":"active+clean","last_fresh":"2024-09-14T06:32:43.623493+0000","last_change":"2024-09-14T06:32:43.623493+0000","last_active":"2024-09-14T06:32:43.623493+0000","last_peered":"2024-09-14T06:32:43.623493+0000","last_clean":"2024-09-14T06:32:43.623493+0000","last_became_active":"2024-09-14T06:32:39.777524+0000","last_became_peered":"2024-09-14T06:32:39.777524+0000","last_unstale":"2024-09-14T06:32:43.623493+0000","last_undegraded":"2024-09-14T06:32:43.623493+0000","last_fullsized":"2024-09-14T06:32:43.623493+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":544,"log_dups_size":0,"ondisk_log_size":544,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T17:41:20.292335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001295593,"stat_sum":{"num_bytes":14958569,"num_objects":39,"num_object_clones":0,"num_object_copies":78,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":39,"num_whiteouts":0,"num_read":3982,"num_read_kb":61136,"num_write":714,"num_write_kb":43787,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":171,"num_bytes_recovered":10913227,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"27","length":"1"},{"start":"29","length":"1"},{"start":"2b","length":"1"},{"start":"2d","length":"1"},{"start":"2f","length":"1"},{"start":"31","length":"1"},{"start":"33","length":"1"}]},{"pgid":"3.f","version":"238'687","reported_seq":5729,"reported_epoch":238,"state":"active+clean","last_fresh":"2024-09-14T06:32:43.714749+0000","last_change":"2024-09-14T06:32:43.623386+0000","last_active":"2024-09-14T06:32:43.714749+0000","last_peered":"2024-09-14T06:32:43.714749+0000","last_clean":"2024-09-14T06:32:43.714749+0000","last_became_active":"2024-09-14T06:29:00.586491+0000","last_became_peered":"2024-09-14T06:29:00.586491+0000","last_unstale":"2024-09-14T06:32:43.714749+0000","last_undegraded":"2024-09-14T06:32:43.714749+0000","last_fullsized":"2024-09-14T06:32:43.714749+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":687,"log_dups_size":0,"ondisk_log_size":687,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T13:24:23.935392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014244769999999999,"stat_sum":{"num_bytes":17046942,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":6076,"num_read_kb":45576,"num_write":813,"num_write_kb":36134,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"27","length":"1"},{"start":"29","length":"1"},{"start":"2b","length":"1"},{"start":"2d","length":"1"},{"start":"2f","length":"1"},{"start":"31","length":"1"},{"start":"33","length":"1"}]},{"pgid":"3.10","version":"236'899","reported_seq":6563,"reported_epoch":237,"state":"active+clean","last_fresh":"2024-09-14T06:32:42.620400+0000","last_change":"2024-09-14T06:32:42.620647+0000","last_active":"2024-09-14T06:32:42.620400+0000","last_peered":"2024-09-14T06:32:42.620400+0000","last_clean":"2024-09-14T06:32:42.620400+0000","last_became_active":"2024-09-14T06:32:04.621617+0000","last_became_peered":"2024-09-14T06:32:04.621617+0000","last_unstale":"2024-09-14T06:32:42.620400+0000","last_undegraded":"2024-09-14T06:32:42.620400+0000","last_fullsized":"2024-09-14T06:32:42.620400+0000","mapping_epoch":199,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":200,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":899,"log_dups_size":0,"ondisk_log_size":899,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T16:44:12.269688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051763799999999995,"stat_sum":{"num_bytes":18989605,"num_objects":29,"num_object_clones":0,"num_object_copies":58,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":29,"num_whiteouts":0,"num_read":6970,"num_read_kb":34321,"num_write":1037,"num_write_kb":36649,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":154,"num_bytes_recovered":9401812,"num_keys_recovered":45,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"29","length":"1"},{"start":"2b","length":"1"},{"start":"2d","length":"1"},{"start":"2f","length":"1"},{"start":"31","length":"1"},{"start":"33","length":"1"}]},{"pgid":"3.11","version":"238'660","reported_seq":11478,"reported_epoch":238,"state":"active+clean","last_fresh":"2024-09-14T06:32:43.696263+0000","last_change":"2024-09-14T06:32:43.622647+0000","last_active":"2024-09-14T06:32:43.696263+0000","last_peered":"2024-09-14T06:32:43.696263+0000","last_clean":"2024-09-14T06:32:43.696263+0000","last_became_active":"2024-09-14T06:32:39.778010+0000","last_became_peered":"2024-09-14T06:32:39.778010+0000","last_unstale":"2024-09-14T06:32:43.696263+0000","last_undegraded":"2024-09-14T06:32:43.696263+0000","last_fullsized":"2024-09-14T06:32:43.696263+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":660,"log_dups_size":0,"ondisk_log_size":660,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T09:41:48.034081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076344800000000001,"stat_sum":{"num_bytes":50834137,"num_objects":44,"num_object_clones":0,"num_object_copies":88,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":44,"num_whiteouts":0,"num_read":10209,"num_read_kb":109195,"num_write":789,"num_write_kb":48297,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":177,"num_bytes_recovered":16647747,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"27","length":"1"},{"start":"29","length":"1"},{"start":"2b","length":"1"},{"start":"2d","length":"1"},{"start":"2f","length":"1"},{"start":"31","length":"1"},{"start":"33","length":"1"}]},{"pgid":"3.12","version":"237'695","reported_seq":7144,"reported_epoch":237,"state":"active+clean","last_fresh":"2024-09-14T06:32:42.620909+0000","last_change":"2024-09-14T06:32:42.620909+0000","last_active":"2024-09-14T06:32:42.620909+0000","last_peered":"2024-09-14T06:32:42.620909+0000","last_clean":"2024-09-14T06:32:42.620909+0000","last_became_active":"2024-09-14T06:29:00.579826+0000","last_became_peered":"2024-09-14T06:29:00.579826+0000","last_unstale":"2024-09-14T06:32:42.620909+0000","last_undegraded":"2024-09-14T06:32:42.620909+0000","last_fullsized":"2024-09-14T06:32:42.620909+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":695,"log_dups_size":0,"ondisk_log_size":695,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T18:25:05.755525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00075932500000000004,"stat_sum":{"num_bytes":24818516,"num_objects":42,"num_object_clones":0,"num_object_copies":84,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":42,"num_whiteouts":0,"num_read":6875,"num_read_kb":58890,"num_write":863,"num_write_kb":42425,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"29","length":"1"},{"start":"2b","length":"1"},{"start":"2d","length":"1"},{"start":"2f","length":"1"},{"start":"31","length":"1"},{"start":"33","length":"1"}]},{"pgid":"3.13","version":"238'970","reported_seq":8728,"reported_epoch":238,"state":"active+clean","last_fresh":"2024-09-14T06:32:43.704236+0000","last_change":"2024-09-14T06:32:43.622369+0000","last_active":"2024-09-14T06:32:43.704236+0000","last_peered":"2024-09-14T06:32:43.704236+0000","last_clean":"2024-09-14T06:32:43.704236+0000","last_became_active":"2024-09-14T06:29:00.586520+0000","last_became_peered":"2024-09-14T06:29:00.586520+0000","last_unstale":"2024-09-14T06:32:43.704236+0000","last_undegraded":"2024-09-14T06:32:43.704236+0000","last_fullsized":"2024-09-14T06:32:43.704236+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":970,"log_dups_size":0,"ondisk_log_size":970,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T12:28:10.034097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00082331099999999996,"stat_sum":{"num_bytes":40273349,"num_objects":34,"num_object_clones":0,"num_object_copies":68,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":34,"num_whiteouts":0,"num_read":9546,"num_read_kb":66117,"num_write":1201,"num_write_kb":47437,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"27","length":"1"},{"start":"29","length":"1"},{"start":"2b","length":"1"},{"start":"2d","length":"1"},{"start":"2f","length":"1"},{"start":"31","length":"1"},{"start":"33","length":"1"}]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":907361470,"num_objects":1114,"num_object_clones":0,"num_object_copies":2228,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1114,"num_whiteouts":0,"num_read":236262,"num_read_kb":2181639,"num_write":36435,"num_write_kb":1478146,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5149,"num_bytes_recovered":633091608,"num_keys_recovered":534,"num_objects_omap":19,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":72040448,"data_stored":713447386,"data_compressed":43381329,"data_compressed_allocated":57565184,"data_compressed_original":702062842,"omap_allocated":0,"internal_metadata":0},"log_size":26795,"ondisk_log_size":26795,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":131072,"data_stored":1180736,"data_compressed":65166,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509491,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":75096,"kb_used_data":11968,"kb_used_omap":1,"kb_used_meta":63102,"kb_avail":94296744,"statfs":{"total":96636764160,"available":96559865856,"internally_reserved":0,"allocated":12255232,"data_stored":93297315,"data_compressed":5468001,"data_compressed_allocated":7200768,"data_compressed_original":90126128,"omap_allocated":1590,"internal_metadata":64616906},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":0,"seq":64424509491,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509492,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":79612,"kb_used_data":12316,"kb_used_omap":1,"kb_used_meta":67262,"kb_avail":94292228,"statfs":{"total":96636764160,"available":96555241472,"internally_reserved":0,"allocated":12611584,"data_stored":91841273,"data_compressed":5557659,"data_compressed_allocated":7454720,"data_compressed_original":88637452,"omap_allocated":1595,"internal_metadata":68876741},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509491,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":88432,"kb_used_data":12828,"kb_used_omap":1,"kb_used_meta":75582,"kb_avail":94283408,"statfs":{"total":96636764160,"available":96546209792,"internally_reserved":0,"allocated":13135872,"data_stored":106616304,"data_compressed":6468186,"data_compressed_allocated":8470528,"data_compressed_original":103846041,"omap_allocated":1595,"internal_metadata":77396421},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509492,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":63672,"kb_used_data":14436,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94308168,"statfs":{"total":96636764160,"available":96571564032,"internally_reserved":0,"allocated":14782464,"data_stored":121071457,"data_compressed":7493926,"data_compressed_allocated":9818112,"data_compressed_original":118051925,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":14,"seq":60129542195,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":38940,"kb_used_data":7744,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94332900,"statfs":{"total":96636764160,"available":96596889600,"internally_reserved":0,"allocated":7929856,"data_stored":54710491,"data_compressed":3152243,"data_compressed_allocated":4280320,"data_compressed_original":52716327,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542194,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":84128,"kb_used_data":12556,"kb_used_omap":1,"kb_used_meta":71422,"kb_avail":94287712,"statfs":{"total":96636764160,"available":96550617088,"internally_reserved":0,"allocated":12857344,"data_stored":96790161,"data_compressed":5570864,"data_compressed_allocated":7372800,"data_compressed_original":93240828,"omap_allocated":1595,"internal_metadata":73136581},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574900,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":88472,"kb_used_data":12728,"kb_used_omap":1,"kb_used_meta":75582,"kb_avail":94283368,"statfs":{"total":96636764160,"available":96546168832,"internally_reserved":0,"allocated":13033472,"data_stored":99690060,"data_compressed":6221782,"data_compressed_allocated":8216576,"data_compressed_original":96739328,"omap_allocated":1593,"internal_metadata":77396423},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32583,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32583,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":9744384,"data_stored":94574967,"data_compressed":5535176,"data_compressed_allocated":7319552,"data_compressed_original":92601852,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":11714560,"data_stored":118758735,"data_compressed":7407455,"data_compressed_allocated":9711616,"data_compressed_original":117216341,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":5058560,"data_stored":53289916,"data_compressed":3151615,"data_compressed_allocated":4276224,"data_compressed_original":52704039,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":10121216,"data_stored":98209563,"data_compressed":6219077,"data_compressed_allocated":8212480,"data_compressed_original":96681984,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":10072064,"data_stored":103857913,"data_compressed":6344819,"data_compressed_allocated":8323072,"data_compressed_original":102531442,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":9555968,"data_stored":89273137,"data_compressed":5452184,"data_compressed_allocated":7327744,"data_compressed_original":87519244,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":6631424,"data_stored":65369250,"data_compressed":3936333,"data_compressed_allocated":5361664,"data_compressed_original":64393940,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":9142272,"data_stored":90113905,"data_compressed":5334670,"data_compressed_allocated":7032832,"data_compressed_original":88414000,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-14T06:32:47.197 INFO:tasks.thrashosds.thrasher:Setting 3.a to [1, 3, 4, 2] 2024-09-14T06:32:47.197 INFO:tasks.thrashosds.thrasher:cmd ['osd', 'pg-upmap-items', '3.a', '1', '3', '4', '2'] 2024-09-14T06:32:47.197 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pg-upmap-items 3.a 1 3 4 2 2024-09-14T06:32:47.265 INFO:teuthology.orchestra.run.smithi094.stdout:ERROR: (22) Invalid argument 2024-09-14T06:32:47.265 INFO:teuthology.orchestra.run.smithi094.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:32:47.266 INFO:tasks.ceph.osd.2.smithi094.stderr:2024-09-14T06:32:47.264+0000 7faf3269b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33649) UID: 0 2024-09-14T06:32:47.275 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-14T06:32:47.359 INFO:tasks.ceph.osd.5.smithi138.stderr:2024-09-14T06:32:47.357+0000 7fe13746f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33618) UID: 0 2024-09-14T06:32:47.421 INFO:teuthology.orchestra.run.smithi094.stdout:ERROR: (22) Invalid argument 2024-09-14T06:32:47.421 INFO:teuthology.orchestra.run.smithi094.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:32:47.430 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-14T06:32:47.459 INFO:tasks.ceph.osd.6.smithi138.stderr:2024-09-14T06:32:47.459+0000 7f8cd8849640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33620) UID: 0 2024-09-14T06:32:47.527 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-14T06:32:47.534 INFO:teuthology.orchestra.run.smithi094.stdout:ERROR: (22) Invalid argument 2024-09-14T06:32:47.534 INFO:teuthology.orchestra.run.smithi094.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:32:47.543 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-14T06:32:47.560 INFO:tasks.ceph.osd.5.smithi138.stderr:2024-09-14T06:32:47.559+0000 7fe13746f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33618) UID: 0 2024-09-14T06:32:47.618 INFO:teuthology.orchestra.run.smithi094.stderr:set 3.a pg_upmap_items mapping to [1->3,4->2] 2024-09-14T06:32:47.633 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-14T06:32:47.662 INFO:teuthology.orchestra.run.smithi094.stdout:osd.0: {} 2024-09-14T06:32:47.662 INFO:teuthology.orchestra.run.smithi094.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-14T06:32:47.663 INFO:tasks.ceph.osd.1.smithi094.stderr:2024-09-14T06:32:47.661+0000 7f328e47a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33641) UID: 0 2024-09-14T06:32:47.683 INFO:teuthology.orchestra.run.smithi094.stdout:osd.1: {} 2024-09-14T06:32:47.683 INFO:teuthology.orchestra.run.smithi094.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-14T06:32:47.704 INFO:teuthology.orchestra.run.smithi094.stdout:osd.2: {} 2024-09-14T06:32:47.704 INFO:teuthology.orchestra.run.smithi094.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-14T06:32:47.713 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-14T06:32:47.713 INFO:teuthology.orchestra.run.smithi094.stdout: "ops": [ 2024-09-14T06:32:47.713 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:47.713 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:238175 3.0 3:06e93fb9:::rbd_data.112d80ca42af.0000000000000000:head [delete] snapc 1f=[] ondisk+write+known_if_redirected+supports_pool_eio e242)", 2024-09-14T06:32:47.713 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:32:47.709710+0000", 2024-09-14T06:32:47.714 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 0.0025241579999999999, 2024-09-14T06:32:47.714 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0025510519999999998, 2024-09-14T06:32:47.714 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:32:47.714 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:32:47.714 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "waiting for sub ops", 2024-09-14T06:32:47.714 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:32:47.714 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:32:47.714 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:47.715 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 238175 2024-09-14T06:32:47.715 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:47.715 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:32:47.715 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:47.715 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:32:47.715 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:47.709710+0000", 2024-09-14T06:32:47.715 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:47.715 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:47.715 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:47.716 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:32:47.716 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:47.709710+0000", 2024-09-14T06:32:47.716 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:47.716 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:47.716 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:47.716 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:32:47.716 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:47.709712+0000", 2024-09-14T06:32:47.716 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.632e-06 2024-09-14T06:32:47.716 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:47.716 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:47.716 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:32:47.716 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:47.709717+0000", 2024-09-14T06:32:47.716 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.1830000000000002e-06 2024-09-14T06:32:47.716 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:47.716 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:47.717 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:32:47.717 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:47.709732+0000", 2024-09-14T06:32:47.717 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.4758e-05 2024-09-14T06:32:47.717 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:47.717 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:47.717 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:32:47.717 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:47.709740+0000", 2024-09-14T06:32:47.717 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.0069999999999997e-06 2024-09-14T06:32:47.717 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:47.717 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:47.717 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:32:47.717 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:47.709817+0000", 2024-09-14T06:32:47.718 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.7564000000000005e-05 2024-09-14T06:32:47.718 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:47.718 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:47.718 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:32:47.718 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:47.709961+0000", 2024-09-14T06:32:47.718 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00014346500000000001 2024-09-14T06:32:47.718 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:47.718 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:47.718 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "waiting for subops from 2", 2024-09-14T06:32:47.718 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:47.710058+0000", 2024-09-14T06:32:47.718 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.7231000000000005e-05 2024-09-14T06:32:47.718 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:47.718 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:32:47.718 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:47.719 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:47.719 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:47.719 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:238184 3.0 3:009b0225:::rbd_data.112d80ca42af.0000000000000009:head [delete] snapc 1f=[] ondisk+write+known_if_redirected+supports_pool_eio e242)", 2024-09-14T06:32:47.719 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:32:47.709816+0000", 2024-09-14T06:32:47.719 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 0.0024186609999999999, 2024-09-14T06:32:47.719 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0025144719999999998, 2024-09-14T06:32:47.719 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:32:47.719 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:32:47.719 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "waiting for sub ops", 2024-09-14T06:32:47.719 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:32:47.719 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:32:47.719 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:47.719 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 238184 2024-09-14T06:32:47.719 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:47.720 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:32:47.720 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:47.720 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:32:47.720 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:47.709816+0000", 2024-09-14T06:32:47.720 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:47.720 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:47.720 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:47.720 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:32:47.720 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:47.709816+0000", 2024-09-14T06:32:47.720 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:47.720 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:47.721 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:47.721 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:32:47.721 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:47.709817+0000", 2024-09-14T06:32:47.721 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.0109999999999999e-06 2024-09-14T06:32:47.721 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:47.721 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:47.721 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:32:47.721 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:47.709822+0000", 2024-09-14T06:32:47.722 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.9150000000000004e-06 2024-09-14T06:32:47.722 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:47.722 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:47.722 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:32:47.722 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:47.709834+0000", 2024-09-14T06:32:47.722 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.2179e-05 2024-09-14T06:32:47.722 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:47.722 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:47.722 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:32:47.722 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:47.709841+0000", 2024-09-14T06:32:47.722 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.6699999999999994e-06 2024-09-14T06:32:47.722 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:47.722 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:47.722 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:32:47.722 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:47.711240+0000", 2024-09-14T06:32:47.722 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0013988259999999999 2024-09-14T06:32:47.723 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:47.723 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:47.723 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:32:47.723 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:47.711384+0000", 2024-09-14T06:32:47.723 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00014398699999999999 2024-09-14T06:32:47.723 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:47.723 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:47.723 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "waiting for subops from 2", 2024-09-14T06:32:47.723 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:47.711520+0000", 2024-09-14T06:32:47.723 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000135477 2024-09-14T06:32:47.723 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:47.723 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:32:47.723 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:47.723 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:47.724 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:47.724 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4397.0:238183 3.1a e242/23 3:5d4fd0bd:::rbd_data.112d80ca42af.0000000000000008:head v 242'1599, mlcod=241'1597)", 2024-09-14T06:32:47.724 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:32:47.710282+0000", 2024-09-14T06:32:47.724 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 0.0019520640000000001, 2024-09-14T06:32:47.724 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0021121680000000002, 2024-09-14T06:32:47.724 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:32:47.724 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:32:47.724 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:32:47.724 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:32:47.724 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:47.724 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:32:47.724 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:47.710282+0000", 2024-09-14T06:32:47.724 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:47.724 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:47.724 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:47.725 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:32:47.725 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:47.710282+0000", 2024-09-14T06:32:47.725 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:47.725 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:47.725 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:47.725 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:32:47.725 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:47.710283+0000", 2024-09-14T06:32:47.725 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.8700000000000002e-07 2024-09-14T06:32:47.725 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:47.725 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:47.725 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:32:47.725 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:47.710286+0000", 2024-09-14T06:32:47.726 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.7419999999999999e-06 2024-09-14T06:32:47.726 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:47.726 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:47.726 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:32:47.726 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:47.710298+0000", 2024-09-14T06:32:47.726 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.2061e-05 2024-09-14T06:32:47.726 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:47.726 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:47.726 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:32:47.726 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:47.710305+0000", 2024-09-14T06:32:47.726 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.9099999999999999e-06 2024-09-14T06:32:47.726 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:47.726 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:47.727 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:32:47.727 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:47.710737+0000", 2024-09-14T06:32:47.727 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00043147200000000002 2024-09-14T06:32:47.727 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:47.727 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:47.727 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:32:47.727 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:47.710760+0000", 2024-09-14T06:32:47.727 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.2889999999999999e-05 2024-09-14T06:32:47.727 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:47.727 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:32:47.727 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:47.727 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:47.727 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:47.727 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4397.0:238178 3.8 e242/190 3:1177d9cb:::rbd_data.112d80ca42af.0000000000000003:head v 242'637, mlcod=241'635)", 2024-09-14T06:32:47.728 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:32:47.710373+0000", 2024-09-14T06:32:47.728 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 0.001861786, 2024-09-14T06:32:47.728 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0020708520000000002, 2024-09-14T06:32:47.728 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:32:47.728 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:32:47.728 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:32:47.728 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:32:47.728 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:47.728 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:32:47.728 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:47.710373+0000", 2024-09-14T06:32:47.728 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:47.728 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:47.729 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:47.729 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:32:47.729 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:47.710373+0000", 2024-09-14T06:32:47.729 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:47.729 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:47.729 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:47.729 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:32:47.729 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:47.710373+0000", 2024-09-14T06:32:47.729 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.48e-07 2024-09-14T06:32:47.729 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:47.729 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:47.729 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:32:47.730 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:47.710377+0000", 2024-09-14T06:32:47.730 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.1729999999999997e-06 2024-09-14T06:32:47.730 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:47.730 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:47.730 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:32:47.730 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:47.710390+0000", 2024-09-14T06:32:47.730 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.2808e-05 2024-09-14T06:32:47.730 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:47.730 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:47.730 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:32:47.730 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:47.710397+0000", 2024-09-14T06:32:47.730 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.9580000000000001e-06 2024-09-14T06:32:47.730 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:47.730 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:47.730 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:32:47.731 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:47.711226+0000", 2024-09-14T06:32:47.731 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00082919199999999997 2024-09-14T06:32:47.731 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:47.731 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:47.731 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:32:47.731 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:47.711247+0000", 2024-09-14T06:32:47.731 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.1216000000000002e-05 2024-09-14T06:32:47.731 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:47.731 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:32:47.731 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:47.731 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:47.731 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:47.731 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4397.0:238180 3.8 e242/190 3:1740cb0d:::rbd_data.112d80ca42af.0000000000000005:head v 242'638, mlcod=241'635)", 2024-09-14T06:32:47.731 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:32:47.711826+0000", 2024-09-14T06:32:47.732 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 0.00040834999999999998, 2024-09-14T06:32:47.732 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00066646999999999995, 2024-09-14T06:32:47.732 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:32:47.732 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:32:47.732 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:32:47.732 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:32:47.732 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:47.732 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:32:47.732 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:47.711826+0000", 2024-09-14T06:32:47.732 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:47.732 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:47.732 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:47.732 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:32:47.732 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:47.711826+0000", 2024-09-14T06:32:47.733 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:47.733 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:47.733 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:47.733 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:32:47.733 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:47.711827+0000", 2024-09-14T06:32:47.733 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.5899999999999995e-07 2024-09-14T06:32:47.733 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:47.733 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:47.733 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:32:47.733 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:47.711832+0000", 2024-09-14T06:32:47.733 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.5319999999999997e-06 2024-09-14T06:32:47.734 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:47.734 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:47.734 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:32:47.734 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:47.711842+0000", 2024-09-14T06:32:47.734 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.0132999999999999e-05 2024-09-14T06:32:47.734 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:47.734 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:47.734 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:32:47.734 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:47.711850+0000", 2024-09-14T06:32:47.734 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.5809999999999999e-06 2024-09-14T06:32:47.734 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:47.734 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:47.734 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:32:47.734 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:47.712119+0000", 2024-09-14T06:32:47.734 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000269432 2024-09-14T06:32:47.735 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:47.735 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:47.735 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:32:47.735 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:47.712140+0000", 2024-09-14T06:32:47.735 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.0741e-05 2024-09-14T06:32:47.735 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:47.735 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:32:47.735 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:47.735 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:47.735 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-09-14T06:32:47.735 INFO:teuthology.orchestra.run.smithi094.stdout: "num_ops": 5 2024-09-14T06:32:47.735 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-14T06:32:47.736 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-14T06:32:47.737 INFO:teuthology.orchestra.run.smithi094.stdout:osd.3: {} 2024-09-14T06:32:47.737 INFO:teuthology.orchestra.run.smithi094.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-14T06:32:47.747 INFO:teuthology.orchestra.run.smithi094.stdout:osd.4: {} 2024-09-14T06:32:47.748 INFO:teuthology.orchestra.run.smithi094.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-14T06:32:47.761 INFO:tasks.ceph.osd.4.smithi138.stderr:2024-09-14T06:32:47.760+0000 7f6c653c8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33615) UID: 0 2024-09-14T06:32:47.765 INFO:teuthology.orchestra.run.smithi094.stdout:osd.5: {} 2024-09-14T06:32:47.766 INFO:teuthology.orchestra.run.smithi094.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-14T06:32:47.783 INFO:teuthology.orchestra.run.smithi094.stdout:osd.6: {} 2024-09-14T06:32:47.783 INFO:teuthology.orchestra.run.smithi094.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-14T06:32:47.800 INFO:teuthology.orchestra.run.smithi094.stdout:osd.7: {} 2024-09-14T06:32:47.801 INFO:teuthology.orchestra.run.smithi094.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-14T06:32:47.860 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-14T06:32:47.860 INFO:teuthology.orchestra.run.smithi094.stdout: "ops": [], 2024-09-14T06:32:47.861 INFO:teuthology.orchestra.run.smithi094.stdout: "complaint_time": 30, 2024-09-14T06:32:47.861 INFO:teuthology.orchestra.run.smithi094.stdout: "num_blocked_ops": 0 2024-09-14T06:32:47.861 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-14T06:32:47.862 INFO:tasks.ceph.osd.1.smithi094.stderr:2024-09-14T06:32:47.861+0000 7f328e47a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33641) UID: 0 2024-09-14T06:32:47.870 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-14T06:32:47.963 INFO:tasks.ceph.osd.5.smithi138.stderr:2024-09-14T06:32:47.962+0000 7fe13746f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33618) UID: 0 2024-09-14T06:32:48.016 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-14T06:32:48.016 INFO:teuthology.orchestra.run.smithi094.stdout: "size": 20, 2024-09-14T06:32:48.016 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 600, 2024-09-14T06:32:48.016 INFO:teuthology.orchestra.run.smithi094.stdout: "ops": [ 2024-09-14T06:32:48.016 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:48.016 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:14919 3.0 3:05c39535:::rbd_data.112d92027590.0000000000000009:f [read 805888~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:32:48.016 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:26.083843+0000", 2024-09-14T06:32:48.016 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 201.93043572299999, 2024-09-14T06:32:48.017 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.041992356000000002, 2024-09-14T06:32:48.017 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:32:48.017 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:32:48.017 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:32:48.017 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:32:48.017 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:32:48.017 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:48.017 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 14919 2024-09-14T06:32:48.017 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:48.017 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:32:48.017 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:48.017 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:32:48.017 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.083843+0000", 2024-09-14T06:32:48.017 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:48.017 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:48.018 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:48.018 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:32:48.018 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.083843+0000", 2024-09-14T06:32:48.018 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:48.018 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:48.018 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:48.018 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:32:48.018 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.083844+0000", 2024-09-14T06:32:48.018 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.3300000000000001e-07 2024-09-14T06:32:48.018 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:48.018 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:48.018 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:32:48.018 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.083845+0000", 2024-09-14T06:32:48.018 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.4729999999999999e-06 2024-09-14T06:32:48.018 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:48.019 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:48.019 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:32:48.019 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.083850+0000", 2024-09-14T06:32:48.019 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.0130000000000001e-06 2024-09-14T06:32:48.019 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:48.019 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:48.019 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:32:48.019 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.083854+0000", 2024-09-14T06:32:48.019 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.4759999999999998e-06 2024-09-14T06:32:48.019 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:48.019 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:48.019 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:32:48.019 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.125636+0000", 2024-09-14T06:32:48.019 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.041782683000000001 2024-09-14T06:32:48.019 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:48.020 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:48.020 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:32:48.020 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.125680+0000", 2024-09-14T06:32:48.020 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.3614e-05 2024-09-14T06:32:48.020 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:48.020 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:48.020 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:32:48.020 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.125836+0000", 2024-09-14T06:32:48.020 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000155864 2024-09-14T06:32:48.020 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:48.020 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:32:48.020 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:48.020 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:48.020 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:48.020 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:14920 3.0 3:05c39535:::rbd_data.112d92027590.0000000000000009:f [read 816128~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:32:48.021 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:26.083868+0000", 2024-09-14T06:32:48.021 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 201.93041106199999, 2024-09-14T06:32:48.021 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.042166771999999998, 2024-09-14T06:32:48.021 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:32:48.021 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:32:48.021 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:32:48.021 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:32:48.021 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:32:48.021 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:48.021 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 14920 2024-09-14T06:32:48.021 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:48.021 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:32:48.022 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:48.022 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:32:48.022 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.083868+0000", 2024-09-14T06:32:48.022 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:48.022 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:48.022 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:48.022 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:32:48.022 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.083868+0000", 2024-09-14T06:32:48.022 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:48.022 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:48.022 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:48.022 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:32:48.022 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.083869+0000", 2024-09-14T06:32:48.022 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.99e-07 2024-09-14T06:32:48.022 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:48.023 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:48.023 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:32:48.023 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.083870+0000", 2024-09-14T06:32:48.023 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.782e-06 2024-09-14T06:32:48.023 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:48.023 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:48.023 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:32:48.023 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.083876+0000", 2024-09-14T06:32:48.023 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.3809999999999998e-06 2024-09-14T06:32:48.023 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:48.023 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:48.024 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:32:48.024 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.083882+0000", 2024-09-14T06:32:48.024 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.9479999999999996e-06 2024-09-14T06:32:48.024 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:48.024 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:48.024 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:32:48.024 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.125878+0000", 2024-09-14T06:32:48.024 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.041996264999999998 2024-09-14T06:32:48.024 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:48.024 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:48.024 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:32:48.024 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.125929+0000", 2024-09-14T06:32:48.024 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.0498999999999999e-05 2024-09-14T06:32:48.024 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:48.025 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:48.025 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:32:48.025 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.126035+0000", 2024-09-14T06:32:48.025 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000106298 2024-09-14T06:32:48.025 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:48.025 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:32:48.025 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:48.025 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:48.025 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:48.025 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:14921 3.0 3:05c39535:::rbd_data.112d92027590.0000000000000009:f [read 826368~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:32:48.025 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:26.083889+0000", 2024-09-14T06:32:48.025 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 201.93039005200001, 2024-09-14T06:32:48.025 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.042435309999999997, 2024-09-14T06:32:48.026 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:32:48.026 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:32:48.026 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:32:48.026 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:32:48.026 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:32:48.026 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:48.026 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 14921 2024-09-14T06:32:48.026 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:48.026 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:32:48.026 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:48.026 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:32:48.026 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.083889+0000", 2024-09-14T06:32:48.026 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:48.026 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:48.027 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:48.027 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:32:48.027 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.083889+0000", 2024-09-14T06:32:48.027 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:48.027 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:48.027 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:48.027 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:32:48.027 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.086033+0000", 2024-09-14T06:32:48.027 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0021444179999999999 2024-09-14T06:32:48.027 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:48.027 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:48.027 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:32:48.027 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.086038+0000", 2024-09-14T06:32:48.028 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.0969999999999999e-06 2024-09-14T06:32:48.028 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:48.028 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:48.028 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:32:48.028 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.086046+0000", 2024-09-14T06:32:48.028 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.9440000000000005e-06 2024-09-14T06:32:48.028 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:48.028 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:48.028 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:32:48.028 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.086050+0000", 2024-09-14T06:32:48.028 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.69e-06 2024-09-14T06:32:48.028 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:48.028 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:48.028 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:32:48.029 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.126080+0000", 2024-09-14T06:32:48.029 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.040030237000000003 2024-09-14T06:32:48.029 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:48.029 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:48.029 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:32:48.029 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.126163+0000", 2024-09-14T06:32:48.029 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.2071999999999996e-05 2024-09-14T06:32:48.029 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:48.029 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:48.029 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:32:48.029 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.126324+0000", 2024-09-14T06:32:48.029 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00016185200000000001 2024-09-14T06:32:48.029 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:48.030 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:32:48.030 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:48.030 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:48.030 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:48.030 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:55627 3.0 3:06490baf:::rbd_data.112dca2f6ae1.0000000000000030:17 [read 2002944~4096 [fadvise_nocache]] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:32:48.030 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:44.599966+0000", 2024-09-14T06:32:48.030 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 183.414313104, 2024-09-14T06:32:48.030 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.041868546999999999, 2024-09-14T06:32:48.030 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:32:48.030 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:32:48.030 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:32:48.030 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:32:48.030 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:32:48.030 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:48.030 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 55627 2024-09-14T06:32:48.031 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:48.031 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:32:48.031 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:48.031 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:32:48.031 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.599966+0000", 2024-09-14T06:32:48.031 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:48.031 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:48.031 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:48.031 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:32:48.031 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.599966+0000", 2024-09-14T06:32:48.031 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:48.031 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:48.031 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:48.031 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:32:48.031 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.599966+0000", 2024-09-14T06:32:48.032 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.0900000000000001e-07 2024-09-14T06:32:48.032 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:48.032 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:48.032 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:32:48.032 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.599967+0000", 2024-09-14T06:32:48.032 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.64e-07 2024-09-14T06:32:48.032 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:48.032 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:48.032 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:32:48.032 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.599970+0000", 2024-09-14T06:32:48.032 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.9670000000000001e-06 2024-09-14T06:32:48.032 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:48.032 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:48.032 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:32:48.032 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.599973+0000", 2024-09-14T06:32:48.033 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.632e-06 2024-09-14T06:32:48.033 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:48.033 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:48.033 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:32:48.033 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.641665+0000", 2024-09-14T06:32:48.033 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.041692547000000003 2024-09-14T06:32:48.033 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:48.033 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:48.033 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:32:48.033 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.641740+0000", 2024-09-14T06:32:48.033 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.4559000000000002e-05 2024-09-14T06:32:48.033 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:48.033 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:48.033 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:32:48.034 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.641835+0000", 2024-09-14T06:32:48.034 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.4668999999999998e-05 2024-09-14T06:32:48.034 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:48.034 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:32:48.034 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:48.034 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:48.034 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:48.034 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:55628 3.0 3:06490baf:::rbd_data.112dca2f6ae1.0000000000000030:17 [read 2027520~4096 [fadvise_nocache]] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:32:48.034 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:44.599976+0000", 2024-09-14T06:32:48.034 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 183.41430331699999, 2024-09-14T06:32:48.034 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.041991621, 2024-09-14T06:32:48.034 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:32:48.034 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:32:48.035 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:32:48.035 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:32:48.035 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:32:48.035 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:48.035 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 55628 2024-09-14T06:32:48.035 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:48.035 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:32:48.035 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:48.035 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:32:48.035 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.599976+0000", 2024-09-14T06:32:48.035 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:48.035 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:48.035 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:48.036 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:32:48.036 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.599976+0000", 2024-09-14T06:32:48.036 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:48.036 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:48.036 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:48.036 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:32:48.036 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.599976+0000", 2024-09-14T06:32:48.036 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.1299999999999999e-07 2024-09-14T06:32:48.036 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:48.036 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:48.036 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:32:48.036 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.599977+0000", 2024-09-14T06:32:48.036 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.7999999999999993e-07 2024-09-14T06:32:48.036 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:48.037 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:48.037 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:32:48.037 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.599980+0000", 2024-09-14T06:32:48.037 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.01e-06 2024-09-14T06:32:48.037 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:48.037 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:48.037 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:32:48.037 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.599983+0000", 2024-09-14T06:32:48.037 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.5409999999999999e-06 2024-09-14T06:32:48.037 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:48.037 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:48.037 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:32:48.037 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.641869+0000", 2024-09-14T06:32:48.037 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.041886235000000001 2024-09-14T06:32:48.037 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:48.038 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:48.038 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:32:48.038 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.641913+0000", 2024-09-14T06:32:48.038 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.4357000000000002e-05 2024-09-14T06:32:48.038 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:48.038 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:48.038 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:32:48.038 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.641967+0000", 2024-09-14T06:32:48.038 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.4284999999999999e-05 2024-09-14T06:32:48.038 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:48.038 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:32:48.038 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:48.038 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:48.038 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:48.038 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:55629 3.0 3:06490baf:::rbd_data.112dca2f6ae1.0000000000000030:17 [read 2052096~4096 [fadvise_nocache]] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:32:48.039 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:44.599985+0000", 2024-09-14T06:32:48.039 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 183.41429361100001, 2024-09-14T06:32:48.039 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.042183375000000002, 2024-09-14T06:32:48.039 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:32:48.039 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:32:48.039 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:32:48.039 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:32:48.039 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:32:48.039 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:48.039 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 55629 2024-09-14T06:32:48.039 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:48.039 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:32:48.039 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:48.039 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:32:48.039 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.599985+0000", 2024-09-14T06:32:48.040 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:48.040 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:48.040 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:48.040 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:32:48.040 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.599985+0000", 2024-09-14T06:32:48.040 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:48.040 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:48.040 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:48.040 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:32:48.040 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.599986+0000", 2024-09-14T06:32:48.040 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.05e-07 2024-09-14T06:32:48.040 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:48.040 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:48.040 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:32:48.041 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.599987+0000", 2024-09-14T06:32:48.041 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.8400000000000002e-07 2024-09-14T06:32:48.041 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:48.041 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:48.041 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:32:48.041 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.599990+0000", 2024-09-14T06:32:48.041 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.5640000000000001e-06 2024-09-14T06:32:48.041 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:48.041 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:48.041 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:32:48.041 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.599993+0000", 2024-09-14T06:32:48.041 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.615e-06 2024-09-14T06:32:48.041 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:48.041 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:48.041 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:32:48.041 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.642003+0000", 2024-09-14T06:32:48.042 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.042010345999999997 2024-09-14T06:32:48.042 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:48.042 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:48.042 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:32:48.042 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.642080+0000", 2024-09-14T06:32:48.042 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.6538999999999993e-05 2024-09-14T06:32:48.042 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:48.042 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:48.042 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:32:48.042 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.642169+0000", 2024-09-14T06:32:48.042 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.9122000000000005e-05 2024-09-14T06:32:48.042 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:48.042 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:32:48.042 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:48.042 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:48.043 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:48.043 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:55630 3.0 3:06490baf:::rbd_data.112dca2f6ae1.0000000000000030:17 [read 2076672~4096 [fadvise_nocache]] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:32:48.043 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:44.599996+0000", 2024-09-14T06:32:48.043 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 183.414283133, 2024-09-14T06:32:48.043 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.042299615999999998, 2024-09-14T06:32:48.043 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:32:48.043 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:32:48.043 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:32:48.043 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:32:48.043 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:32:48.043 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:48.043 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 55630 2024-09-14T06:32:48.043 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:48.044 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:32:48.044 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:48.044 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:32:48.044 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.599996+0000", 2024-09-14T06:32:48.044 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:48.044 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:48.044 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:48.044 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:32:48.044 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.599996+0000", 2024-09-14T06:32:48.044 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:48.044 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:48.044 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:48.044 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:32:48.044 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.599996+0000", 2024-09-14T06:32:48.045 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.1500000000000001e-07 2024-09-14T06:32:48.045 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:48.045 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:48.045 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:32:48.045 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.599997+0000", 2024-09-14T06:32:48.045 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.33e-07 2024-09-14T06:32:48.045 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:48.045 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:48.045 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:32:48.045 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600000+0000", 2024-09-14T06:32:48.045 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.0240000000000002e-06 2024-09-14T06:32:48.045 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:48.045 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:48.045 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:32:48.046 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600003+0000", 2024-09-14T06:32:48.046 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.3640000000000002e-06 2024-09-14T06:32:48.046 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:48.046 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:48.046 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:32:48.046 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.642200+0000", 2024-09-14T06:32:48.046 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.042197571000000003 2024-09-14T06:32:48.046 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:48.046 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:48.046 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:32:48.046 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.642243+0000", 2024-09-14T06:32:48.046 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.2765000000000001e-05 2024-09-14T06:32:48.046 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:48.046 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:48.046 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:32:48.047 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.642296+0000", 2024-09-14T06:32:48.047 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.2744000000000002e-05 2024-09-14T06:32:48.047 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:48.047 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:32:48.047 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:48.047 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:48.047 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:48.047 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:55631 3.0 3:06490baf:::rbd_data.112dca2f6ae1.0000000000000030:17 [read 1048576~4096 [fadvise_nocache]] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:32:48.047 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:44.600006+0000", 2024-09-14T06:32:48.047 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 183.41427354499999, 2024-09-14T06:32:48.047 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.042487905999999999, 2024-09-14T06:32:48.047 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:32:48.047 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:32:48.047 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:32:48.047 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:32:48.048 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:32:48.048 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:48.048 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 55631 2024-09-14T06:32:48.048 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:48.048 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:32:48.048 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:48.048 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:32:48.048 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600006+0000", 2024-09-14T06:32:48.048 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:48.048 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:48.048 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:48.048 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:32:48.048 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600006+0000", 2024-09-14T06:32:48.049 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:48.049 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:48.049 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:48.049 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:32:48.049 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600006+0000", 2024-09-14T06:32:48.049 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.0800000000000001e-07 2024-09-14T06:32:48.049 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:48.049 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:48.049 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:32:48.049 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600007+0000", 2024-09-14T06:32:48.049 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.7199999999999997e-07 2024-09-14T06:32:48.049 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:48.049 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:48.049 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:32:48.049 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600010+0000", 2024-09-14T06:32:48.050 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.0280000000000001e-06 2024-09-14T06:32:48.050 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:48.050 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:48.050 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:32:48.050 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600012+0000", 2024-09-14T06:32:48.050 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.475e-06 2024-09-14T06:32:48.050 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:48.050 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:48.050 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:32:48.050 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.642329+0000", 2024-09-14T06:32:48.050 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.042317157000000001 2024-09-14T06:32:48.050 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:48.050 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:48.051 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:32:48.051 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.642403+0000", 2024-09-14T06:32:48.051 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.3941000000000002e-05 2024-09-14T06:32:48.051 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:48.051 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:48.051 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:32:48.051 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.642493+0000", 2024-09-14T06:32:48.051 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.0124999999999995e-05 2024-09-14T06:32:48.051 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:48.051 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:32:48.051 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:48.051 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:48.051 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:48.051 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:55632 3.0 3:06490baf:::rbd_data.112dca2f6ae1.0000000000000030:17 [read 1073152~4096 [fadvise_nocache]] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:32:48.051 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:44.600015+0000", 2024-09-14T06:32:48.051 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 183.41426379200001, 2024-09-14T06:32:48.052 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.042604922000000003, 2024-09-14T06:32:48.052 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:32:48.052 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:32:48.052 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:32:48.052 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:32:48.052 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:32:48.052 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:48.052 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 55632 2024-09-14T06:32:48.052 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:48.052 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:32:48.052 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:48.052 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:32:48.052 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600015+0000", 2024-09-14T06:32:48.053 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:48.053 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:48.053 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:48.053 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:32:48.053 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600015+0000", 2024-09-14T06:32:48.053 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:48.053 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:48.053 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:48.053 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:32:48.053 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600016+0000", 2024-09-14T06:32:48.053 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.1e-07 2024-09-14T06:32:48.053 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:48.053 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:48.053 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:32:48.054 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600017+0000", 2024-09-14T06:32:48.054 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.9900000000000009e-07 2024-09-14T06:32:48.054 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:48.054 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:48.054 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:32:48.054 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600020+0000", 2024-09-14T06:32:48.054 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.4170000000000001e-06 2024-09-14T06:32:48.054 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:48.054 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:48.054 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:32:48.054 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600023+0000", 2024-09-14T06:32:48.054 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.588e-06 2024-09-14T06:32:48.054 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:48.054 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:48.055 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:32:48.055 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.642526+0000", 2024-09-14T06:32:48.055 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.042503236999999999 2024-09-14T06:32:48.055 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:48.055 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:48.055 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:32:48.055 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.642568+0000", 2024-09-14T06:32:48.055 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.1925000000000003e-05 2024-09-14T06:32:48.055 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:48.055 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:48.055 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:32:48.055 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.642620+0000", 2024-09-14T06:32:48.055 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.2546000000000001e-05 2024-09-14T06:32:48.055 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:48.056 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:32:48.056 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:48.056 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:48.056 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:48.056 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:55633 3.0 3:06490baf:::rbd_data.112dca2f6ae1.0000000000000030:17 [read 1097728~4096 [fadvise_nocache]] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:32:48.056 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:44.600026+0000", 2024-09-14T06:32:48.056 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 183.41425352300001, 2024-09-14T06:32:48.056 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.042797474000000002, 2024-09-14T06:32:48.056 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:32:48.056 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:32:48.056 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:32:48.056 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:32:48.056 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:32:48.056 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:48.056 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 55633 2024-09-14T06:32:48.057 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:48.057 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:32:48.057 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:48.057 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:32:48.057 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600026+0000", 2024-09-14T06:32:48.057 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:48.057 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:48.057 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:48.057 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:32:48.057 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600026+0000", 2024-09-14T06:32:48.057 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:48.057 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:48.057 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:48.057 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:32:48.058 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600026+0000", 2024-09-14T06:32:48.058 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.3099999999999999e-07 2024-09-14T06:32:48.058 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:48.058 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:48.058 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:32:48.058 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600029+0000", 2024-09-14T06:32:48.058 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.304e-06 2024-09-14T06:32:48.058 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:48.058 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:48.058 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:32:48.058 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600034+0000", 2024-09-14T06:32:48.058 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.1429999999999997e-06 2024-09-14T06:32:48.058 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:48.059 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:48.059 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:32:48.059 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600038+0000", 2024-09-14T06:32:48.059 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.692e-06 2024-09-14T06:32:48.059 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:48.059 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:48.059 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:32:48.059 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.642654+0000", 2024-09-14T06:32:48.059 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.042615564000000002 2024-09-14T06:32:48.059 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:48.059 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:48.059 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:32:48.059 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.642728+0000", 2024-09-14T06:32:48.060 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.4018999999999997e-05 2024-09-14T06:32:48.060 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:48.060 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:48.060 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:32:48.060 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.642823+0000", 2024-09-14T06:32:48.060 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.5520999999999996e-05 2024-09-14T06:32:48.060 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:48.060 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:32:48.060 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:48.060 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:48.060 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:48.060 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:55634 3.0 3:06490baf:::rbd_data.112dca2f6ae1.0000000000000030:17 [read 1122304~4096 [fadvise_nocache]] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:32:48.060 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:44.600042+0000", 2024-09-14T06:32:48.060 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 183.41423704600001, 2024-09-14T06:32:48.061 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.042903729000000002, 2024-09-14T06:32:48.061 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:32:48.061 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:32:48.061 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:32:48.061 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:32:48.061 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:32:48.061 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:48.061 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 55634 2024-09-14T06:32:48.061 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:48.061 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:32:48.061 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:48.061 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:32:48.061 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600042+0000", 2024-09-14T06:32:48.062 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:48.062 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:48.062 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:48.062 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:32:48.062 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600042+0000", 2024-09-14T06:32:48.062 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:48.062 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:48.062 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:48.062 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:32:48.062 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600043+0000", 2024-09-14T06:32:48.062 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.7000000000000005e-07 2024-09-14T06:32:48.062 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:48.062 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:48.063 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:32:48.063 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600044+0000", 2024-09-14T06:32:48.063 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.742e-06 2024-09-14T06:32:48.063 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:48.063 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:48.063 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:32:48.063 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600050+0000", 2024-09-14T06:32:48.063 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.0800000000000002e-06 2024-09-14T06:32:48.063 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:48.063 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:48.063 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:32:48.063 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600055+0000", 2024-09-14T06:32:48.063 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.3889999999999998e-06 2024-09-14T06:32:48.063 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:48.064 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:48.064 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:32:48.064 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.642845+0000", 2024-09-14T06:32:48.064 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.042790040000000001 2024-09-14T06:32:48.064 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:48.064 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:48.064 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:32:48.064 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.642888+0000", 2024-09-14T06:32:48.064 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.2790000000000002e-05 2024-09-14T06:32:48.064 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:48.064 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:48.064 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:32:48.065 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.642946+0000", 2024-09-14T06:32:48.065 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.8118000000000002e-05 2024-09-14T06:32:48.065 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:48.065 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:32:48.065 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:48.065 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:48.065 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:48.065 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:55635 3.0 3:06490baf:::rbd_data.112dca2f6ae1.0000000000000030:17 [read 1146880~4096 [fadvise_nocache]] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:32:48.065 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:44.600061+0000", 2024-09-14T06:32:48.065 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 183.41421847500001, 2024-09-14T06:32:48.065 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.043092695, 2024-09-14T06:32:48.065 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:32:48.066 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:32:48.066 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:32:48.066 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:32:48.066 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:32:48.066 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:48.066 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 55635 2024-09-14T06:32:48.066 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:48.066 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:32:48.066 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:48.066 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:32:48.066 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600061+0000", 2024-09-14T06:32:48.066 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:48.066 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:48.067 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:48.067 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:32:48.067 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600061+0000", 2024-09-14T06:32:48.067 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:48.067 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:48.067 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:48.067 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:32:48.067 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600061+0000", 2024-09-14T06:32:48.067 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.3099999999999999e-07 2024-09-14T06:32:48.067 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:48.067 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:48.067 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:32:48.067 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600063+0000", 2024-09-14T06:32:48.067 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.6220000000000001e-06 2024-09-14T06:32:48.067 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:48.067 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:48.068 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:32:48.068 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600068+0000", 2024-09-14T06:32:48.068 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.4790000000000004e-06 2024-09-14T06:32:48.068 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:48.068 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:48.068 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:32:48.068 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600072+0000", 2024-09-14T06:32:48.068 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.3100000000000002e-06 2024-09-14T06:32:48.068 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:48.068 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:48.068 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:32:48.068 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.642987+0000", 2024-09-14T06:32:48.068 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.042914489 2024-09-14T06:32:48.068 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:48.068 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:48.069 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:32:48.069 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.643062+0000", 2024-09-14T06:32:48.069 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.5011000000000004e-05 2024-09-14T06:32:48.069 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:48.069 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:48.069 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:32:48.069 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.643153+0000", 2024-09-14T06:32:48.069 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.1452999999999993e-05 2024-09-14T06:32:48.069 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:48.069 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:32:48.069 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:48.069 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:48.069 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:48.069 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:55636 3.0 3:06490baf:::rbd_data.112dca2f6ae1.0000000000000030:17 [read 1171456~4096 [fadvise_nocache]] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:32:48.070 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:44.600078+0000", 2024-09-14T06:32:48.070 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 183.41420106300001, 2024-09-14T06:32:48.070 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.043207966, 2024-09-14T06:32:48.070 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:32:48.070 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:32:48.070 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:32:48.070 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:32:48.070 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:32:48.070 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:48.070 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 55636 2024-09-14T06:32:48.070 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:48.070 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:32:48.070 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:48.070 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:32:48.070 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600078+0000", 2024-09-14T06:32:48.071 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:48.071 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:48.071 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:48.071 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:32:48.071 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600078+0000", 2024-09-14T06:32:48.071 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:48.071 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:48.071 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:48.071 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:32:48.071 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600078+0000", 2024-09-14T06:32:48.071 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.34e-07 2024-09-14T06:32:48.071 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:48.071 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:48.071 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:32:48.072 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600080+0000", 2024-09-14T06:32:48.072 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.641e-06 2024-09-14T06:32:48.072 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:48.072 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:48.072 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:32:48.072 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600086+0000", 2024-09-14T06:32:48.072 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.3860000000000004e-06 2024-09-14T06:32:48.072 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:48.072 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:48.072 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:32:48.072 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600091+0000", 2024-09-14T06:32:48.072 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.5970000000000002e-06 2024-09-14T06:32:48.072 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:48.072 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:48.073 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:32:48.073 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.643184+0000", 2024-09-14T06:32:48.073 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.043092893 2024-09-14T06:32:48.073 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:48.073 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:48.073 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:32:48.073 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.643228+0000", 2024-09-14T06:32:48.073 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.4536999999999997e-05 2024-09-14T06:32:48.073 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:48.073 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:48.073 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:32:48.073 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.643286+0000", 2024-09-14T06:32:48.073 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.7578000000000003e-05 2024-09-14T06:32:48.073 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:48.074 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:32:48.074 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:48.074 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:48.074 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:48.074 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:55637 3.0 3:06490baf:::rbd_data.112dca2f6ae1.0000000000000030:17 [read 1196032~4096 [fadvise_nocache]] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:32:48.074 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:44.600097+0000", 2024-09-14T06:32:48.074 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 183.41418249899999, 2024-09-14T06:32:48.074 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.043399124999999997, 2024-09-14T06:32:48.074 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:32:48.074 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:32:48.074 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:32:48.074 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:32:48.074 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:32:48.074 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:48.074 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 55637 2024-09-14T06:32:48.074 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:48.075 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:32:48.075 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:48.075 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:32:48.075 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600097+0000", 2024-09-14T06:32:48.075 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:48.075 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:48.075 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:48.075 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:32:48.075 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600097+0000", 2024-09-14T06:32:48.075 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:48.075 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:48.075 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:48.075 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:32:48.075 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.604372+0000", 2024-09-14T06:32:48.076 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0042751489999999998 2024-09-14T06:32:48.076 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:48.076 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:48.076 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:32:48.076 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.604377+0000", 2024-09-14T06:32:48.076 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.7899999999999999e-06 2024-09-14T06:32:48.076 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:48.076 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:48.076 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:32:48.076 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.604389+0000", 2024-09-14T06:32:48.076 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.2197e-05 2024-09-14T06:32:48.076 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:48.076 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:48.076 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:32:48.077 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.604397+0000", 2024-09-14T06:32:48.077 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.8730000000000007e-06 2024-09-14T06:32:48.077 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:48.077 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:48.077 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:32:48.077 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.643324+0000", 2024-09-14T06:32:48.077 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.038926897000000002 2024-09-14T06:32:48.077 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:48.077 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:48.077 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:32:48.077 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.643402+0000", 2024-09-14T06:32:48.077 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.8063000000000001e-05 2024-09-14T06:32:48.077 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:48.078 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:48.078 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:32:48.078 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.643496+0000", 2024-09-14T06:32:48.078 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.4155999999999998e-05 2024-09-14T06:32:48.078 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:48.078 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:32:48.078 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:48.078 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:48.078 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:48.078 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:55657 3.0 3:06490baf:::rbd_data.112dca2f6ae1.0000000000000030:17 [read 1687552~4096 [fadvise_nocache]] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:32:48.078 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:44.604841+0000", 2024-09-14T06:32:48.078 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 183.40943777000001, 2024-09-14T06:32:48.078 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.041890087999999999, 2024-09-14T06:32:48.078 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:32:48.079 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:32:48.079 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:32:48.079 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:32:48.079 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:32:48.079 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:48.079 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 55657 2024-09-14T06:32:48.079 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:48.079 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:32:48.079 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:48.079 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:32:48.079 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.604841+0000", 2024-09-14T06:32:48.079 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:48.079 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:48.079 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:48.079 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:32:48.080 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.604841+0000", 2024-09-14T06:32:48.080 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:48.080 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:48.080 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:48.080 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:32:48.080 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.604842+0000", 2024-09-14T06:32:48.080 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.0900000000000001e-07 2024-09-14T06:32:48.080 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:48.080 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:48.080 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:32:48.080 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.604846+0000", 2024-09-14T06:32:48.080 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.5759999999999999e-06 2024-09-14T06:32:48.080 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:48.080 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:48.081 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:32:48.081 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.604850+0000", 2024-09-14T06:32:48.081 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.405e-06 2024-09-14T06:32:48.081 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:48.081 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:48.081 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:32:48.081 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.604852+0000", 2024-09-14T06:32:48.081 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.8590000000000001e-06 2024-09-14T06:32:48.081 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:48.081 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:48.081 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:32:48.081 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.646633+0000", 2024-09-14T06:32:48.081 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.041780772000000001 2024-09-14T06:32:48.081 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:48.082 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:48.082 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:32:48.082 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.646677+0000", 2024-09-14T06:32:48.082 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.4115e-05 2024-09-14T06:32:48.082 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:48.082 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:48.082 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:32:48.082 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.646731+0000", 2024-09-14T06:32:48.082 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.4151999999999999e-05 2024-09-14T06:32:48.082 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:48.082 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:32:48.082 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:48.082 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:48.082 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:48.082 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:55658 3.0 3:06490baf:::rbd_data.112dca2f6ae1.0000000000000030:17 [read 1712128~4096 [fadvise_nocache]] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:32:48.083 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:44.604856+0000", 2024-09-14T06:32:48.083 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 183.40942317, 2024-09-14T06:32:48.083 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.041986227000000001, 2024-09-14T06:32:48.083 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:32:48.083 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:32:48.083 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:32:48.083 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:32:48.083 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:32:48.083 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:48.083 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 55658 2024-09-14T06:32:48.083 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:48.083 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:32:48.083 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:48.083 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:32:48.084 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.604856+0000", 2024-09-14T06:32:48.084 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:48.084 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:48.084 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:48.084 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:32:48.084 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.604856+0000", 2024-09-14T06:32:48.084 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:48.084 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:48.084 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:48.084 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:32:48.084 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.604856+0000", 2024-09-14T06:32:48.084 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.23e-07 2024-09-14T06:32:48.084 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:48.085 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:48.085 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:32:48.085 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.604857+0000", 2024-09-14T06:32:48.085 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.9999999999999995e-07 2024-09-14T06:32:48.085 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:48.085 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:48.085 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:32:48.085 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.604861+0000", 2024-09-14T06:32:48.085 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.9890000000000003e-06 2024-09-14T06:32:48.085 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:48.085 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:48.085 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:32:48.085 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.604864+0000", 2024-09-14T06:32:48.086 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.7429999999999998e-06 2024-09-14T06:32:48.086 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:48.086 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:48.086 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:32:48.086 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.646744+0000", 2024-09-14T06:32:48.086 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.041880325000000003 2024-09-14T06:32:48.086 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:48.086 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:48.086 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:32:48.086 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.646784+0000", 2024-09-14T06:32:48.086 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.9604000000000002e-05 2024-09-14T06:32:48.086 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:48.086 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:48.087 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:32:48.087 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.646842+0000", 2024-09-14T06:32:48.087 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.8343e-05 2024-09-14T06:32:48.087 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:48.087 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:32:48.087 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:48.087 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:48.087 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:48.087 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:55659 3.0 3:06490baf:::rbd_data.112dca2f6ae1.0000000000000030:17 [read 1736704~4096 [fadvise_nocache]] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:32:48.087 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:44.604867+0000", 2024-09-14T06:32:48.087 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 183.40941201699999, 2024-09-14T06:32:48.087 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.042097082000000001, 2024-09-14T06:32:48.087 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:32:48.087 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:32:48.088 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:32:48.088 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:32:48.088 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:32:48.088 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:48.088 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 55659 2024-09-14T06:32:48.088 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:48.088 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:32:48.088 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:48.088 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:32:48.088 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.604867+0000", 2024-09-14T06:32:48.088 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:48.088 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:48.088 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:48.088 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:32:48.089 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.604867+0000", 2024-09-14T06:32:48.089 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:48.089 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:48.089 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:48.089 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:32:48.089 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.604867+0000", 2024-09-14T06:32:48.089 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.2000000000000001e-07 2024-09-14T06:32:48.089 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:48.089 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:48.089 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:32:48.089 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.604868+0000", 2024-09-14T06:32:48.089 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.04e-06 2024-09-14T06:32:48.089 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:48.090 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:48.090 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:32:48.090 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.604871+0000", 2024-09-14T06:32:48.090 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.9689999999999999e-06 2024-09-14T06:32:48.090 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:48.090 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:48.090 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:32:48.090 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.604874+0000", 2024-09-14T06:32:48.090 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.5069999999999999e-06 2024-09-14T06:32:48.090 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:48.090 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:48.090 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:32:48.090 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.646874+0000", 2024-09-14T06:32:48.090 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.042000513000000003 2024-09-14T06:32:48.091 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:48.091 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:48.091 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:32:48.091 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.646914+0000", 2024-09-14T06:32:48.091 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.9443999999999997e-05 2024-09-14T06:32:48.091 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:48.091 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:48.091 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:32:48.091 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.646964+0000", 2024-09-14T06:32:48.091 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.0389000000000002e-05 2024-09-14T06:32:48.091 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:48.091 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:32:48.091 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:48.091 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:48.092 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:48.092 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:55660 3.0 3:06490baf:::rbd_data.112dca2f6ae1.0000000000000030:17 [read 1761280~4096 [fadvise_nocache]] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:32:48.092 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:44.604877+0000", 2024-09-14T06:32:48.092 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 183.40940226800001, 2024-09-14T06:32:48.092 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.042196349000000001, 2024-09-14T06:32:48.092 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:32:48.092 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:32:48.092 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:32:48.092 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:32:48.092 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:32:48.092 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:48.092 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 55660 2024-09-14T06:32:48.092 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:48.092 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:32:48.093 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:48.093 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:32:48.093 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.604877+0000", 2024-09-14T06:32:48.093 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:48.093 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:48.093 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:48.093 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:32:48.093 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.604877+0000", 2024-09-14T06:32:48.093 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:48.093 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:48.093 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:48.093 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:32:48.093 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.605810+0000", 2024-09-14T06:32:48.093 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00093336899999999997 2024-09-14T06:32:48.093 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:48.094 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:48.094 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:32:48.094 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.605815+0000", 2024-09-14T06:32:48.094 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.6140000000000002e-06 2024-09-14T06:32:48.094 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:48.094 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:48.094 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:32:48.094 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.605825+0000", 2024-09-14T06:32:48.094 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.8889999999999997e-06 2024-09-14T06:32:48.094 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:48.094 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:48.094 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:32:48.094 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.605831+0000", 2024-09-14T06:32:48.094 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.9800000000000003e-06 2024-09-14T06:32:48.095 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:48.095 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:48.095 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:32:48.095 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.646989+0000", 2024-09-14T06:32:48.095 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.041157878000000002 2024-09-14T06:32:48.095 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:48.095 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:48.095 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:32:48.095 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.647026+0000", 2024-09-14T06:32:48.095 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.7305000000000003e-05 2024-09-14T06:32:48.095 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:48.095 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:48.095 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:32:48.095 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.647073+0000", 2024-09-14T06:32:48.095 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.7314000000000001e-05 2024-09-14T06:32:48.096 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:48.096 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:32:48.096 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:48.096 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:48.096 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:48.096 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:55666 3.0 3:06490baf:::rbd_data.112dca2f6ae1.0000000000000030:17 [read 1908736~4096 [fadvise_nocache]] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:32:48.096 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:44.605898+0000", 2024-09-14T06:32:48.096 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 183.40838152200001, 2024-09-14T06:32:48.096 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.041862887000000001, 2024-09-14T06:32:48.096 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:32:48.096 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:32:48.096 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:32:48.096 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:32:48.097 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:32:48.097 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:48.097 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 55666 2024-09-14T06:32:48.097 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:48.097 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:32:48.097 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:48.097 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:32:48.097 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.605898+0000", 2024-09-14T06:32:48.097 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:48.097 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:48.097 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:48.097 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:32:48.097 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.605898+0000", 2024-09-14T06:32:48.097 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:48.098 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:48.098 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:48.098 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:32:48.098 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.611874+0000", 2024-09-14T06:32:48.098 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0059764049999999997 2024-09-14T06:32:48.098 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:48.098 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:48.098 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:32:48.098 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.611879+0000", 2024-09-14T06:32:48.098 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.8749999999999999e-06 2024-09-14T06:32:48.098 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:48.098 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:48.098 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:32:48.099 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.611887+0000", 2024-09-14T06:32:48.099 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.4109999999999996e-06 2024-09-14T06:32:48.099 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:48.099 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:48.099 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:32:48.099 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.611893+0000", 2024-09-14T06:32:48.099 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.2959999999999998e-06 2024-09-14T06:32:48.099 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:48.099 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:48.099 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:32:48.099 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.647674+0000", 2024-09-14T06:32:48.099 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.035781281999999998 2024-09-14T06:32:48.099 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:48.100 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:48.100 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:32:48.100 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.647711+0000", 2024-09-14T06:32:48.100 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.7434000000000001e-05 2024-09-14T06:32:48.100 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:48.100 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:48.100 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:32:48.100 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.647760+0000", 2024-09-14T06:32:48.100 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.9184000000000002e-05 2024-09-14T06:32:48.100 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:48.100 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:32:48.100 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:48.100 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:48.101 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:48.101 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:236341 3.1c 3:3831950c:::rbd_directory:head [call rbd.dir_get_id in=13b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e233)", 2024-09-14T06:32:48.101 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:32:38.788743+0000", 2024-09-14T06:32:48.101 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 9.225536043, 2024-09-14T06:32:48.101 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.99031488700000003, 2024-09-14T06:32:48.101 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:32:48.101 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:32:48.101 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:32:48.101 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:32:48.101 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:32:48.101 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:48.101 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 236341 2024-09-14T06:32:48.101 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:48.101 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:32:48.102 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:48.102 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:32:48.102 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:38.788743+0000", 2024-09-14T06:32:48.102 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:48.102 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:48.102 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:48.102 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:32:48.102 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:38.788743+0000", 2024-09-14T06:32:48.102 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:48.102 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:48.102 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:48.102 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:32:48.102 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:38.788745+0000", 2024-09-14T06:32:48.102 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.623e-06 2024-09-14T06:32:48.102 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:48.103 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:48.103 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:32:48.103 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:38.788753+0000", 2024-09-14T06:32:48.103 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.8420000000000005e-06 2024-09-14T06:32:48.103 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:48.103 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:48.103 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:32:48.103 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:38.788877+0000", 2024-09-14T06:32:48.103 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00012487899999999999 2024-09-14T06:32:48.103 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:48.103 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:48.103 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:32:48.103 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:38.788886+0000", 2024-09-14T06:32:48.103 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.2379999999999997e-06 2024-09-14T06:32:48.103 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:48.103 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:48.104 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:32:48.104 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:38.788947+0000", 2024-09-14T06:32:48.104 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.1721000000000001e-05 2024-09-14T06:32:48.104 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:48.104 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:48.104 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "waiting for peered", 2024-09-14T06:32:48.104 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:38.788955+0000", 2024-09-14T06:32:48.104 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.2579999999999998e-06 2024-09-14T06:32:48.104 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:48.104 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:48.104 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:32:48.104 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:39.778650+0000", 2024-09-14T06:32:48.104 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.989694988 2024-09-14T06:32:48.104 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:48.104 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:48.105 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:32:48.105 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:39.778811+0000", 2024-09-14T06:32:48.105 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000161092 2024-09-14T06:32:48.105 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:48.105 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:48.105 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:32:48.105 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:39.779058+0000", 2024-09-14T06:32:48.105 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00024724600000000001 2024-09-14T06:32:48.105 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:48.105 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:32:48.105 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:48.105 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:48.105 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:32:48.105 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-14T06:32:48.107 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-14T06:32:48.107 INFO:tasks.ceph.osd.2.smithi094.stderr:2024-09-14T06:32:48.106+0000 7faf3269b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33649) UID: 0 2024-09-14T06:32:48.208 INFO:tasks.ceph.osd.3.smithi094.stderr:2024-09-14T06:32:48.207+0000 7feeeafe8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33655) UID: 0 2024-09-14T06:32:48.252 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-14T06:32:48.252 INFO:teuthology.orchestra.run.smithi094.stdout: "ops": [ 2024-09-14T06:32:48.252 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:48.252 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:238182 3.a 3.3eafbe6a (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e242)", 2024-09-14T06:32:48.252 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:32:47.709833+0000", 2024-09-14T06:32:48.252 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 0.54139736599999999, 2024-09-14T06:32:48.252 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.54141737400000001, 2024-09-14T06:32:48.252 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:32:48.252 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:32:48.252 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "waiting for peered", 2024-09-14T06:32:48.252 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:32:48.253 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:32:48.253 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:48.253 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 238182 2024-09-14T06:32:48.253 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:48.253 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:32:48.253 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:48.253 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:32:48.253 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:47.709833+0000", 2024-09-14T06:32:48.253 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:48.253 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:48.253 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:48.253 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:32:48.253 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:47.709833+0000", 2024-09-14T06:32:48.254 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:48.254 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:48.254 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:48.254 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:32:48.254 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:47.709835+0000", 2024-09-14T06:32:48.254 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.0020000000000001e-06 2024-09-14T06:32:48.254 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:48.254 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:48.254 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:32:48.254 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:47.709843+0000", 2024-09-14T06:32:48.254 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.0779999999999996e-06 2024-09-14T06:32:48.254 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:48.254 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:48.254 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:32:48.255 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:47.709864+0000", 2024-09-14T06:32:48.255 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.0605999999999999e-05 2024-09-14T06:32:48.255 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:48.255 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:48.255 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:32:48.255 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:47.709874+0000", 2024-09-14T06:32:48.255 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.9340000000000001e-06 2024-09-14T06:32:48.255 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:48.255 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:48.255 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:32:48.255 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:47.709948+0000", 2024-09-14T06:32:48.255 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.3796999999999998e-05 2024-09-14T06:32:48.255 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:48.256 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:48.256 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "waiting for peered", 2024-09-14T06:32:48.256 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:47.709956+0000", 2024-09-14T06:32:48.256 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.7760000000000003e-06 2024-09-14T06:32:48.256 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:48.256 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:32:48.256 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:48.256 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:48.256 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-09-14T06:32:48.256 INFO:teuthology.orchestra.run.smithi094.stdout: "num_ops": 1 2024-09-14T06:32:48.256 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-14T06:32:48.260 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-14T06:32:48.309 INFO:tasks.ceph.osd.5.smithi138.stderr:2024-09-14T06:32:48.308+0000 7fe13746f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33618) UID: 0 2024-09-14T06:32:48.403 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-14T06:32:48.404 INFO:teuthology.orchestra.run.smithi094.stdout: "ops": [], 2024-09-14T06:32:48.404 INFO:teuthology.orchestra.run.smithi094.stdout: "complaint_time": 30, 2024-09-14T06:32:48.404 INFO:teuthology.orchestra.run.smithi094.stdout: "num_blocked_ops": 0 2024-09-14T06:32:48.404 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-14T06:32:48.410 INFO:tasks.ceph.osd.6.smithi138.stderr:2024-09-14T06:32:48.409+0000 7f8cd8849640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33620) UID: 0 2024-09-14T06:32:48.412 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-14T06:32:48.511 INFO:tasks.ceph.osd.2.smithi094.stderr:2024-09-14T06:32:48.510+0000 7faf3269b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33649) UID: 0 2024-09-14T06:32:48.516 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-14T06:32:48.516 INFO:teuthology.orchestra.run.smithi094.stdout: "size": 20, 2024-09-14T06:32:48.517 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 600, 2024-09-14T06:32:48.517 INFO:teuthology.orchestra.run.smithi094.stdout: "ops": [ 2024-09-14T06:32:48.517 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:48.517 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4397.0:45722 3.9 e64/23 3:91fed098:::rbd_data.112d5a737e72.0000000000000001:head v 64'61, mlcod=62'59)", 2024-09-14T06:32:48.517 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:43.864269+0000", 2024-09-14T06:32:48.517 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 184.650623529, 2024-09-14T06:32:48.517 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.003055224, 2024-09-14T06:32:48.517 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:32:48.517 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:32:48.517 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:32:48.517 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:32:48.517 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:48.518 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:32:48.518 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.864269+0000", 2024-09-14T06:32:48.518 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:48.518 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:48.518 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:48.518 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:32:48.518 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.864269+0000", 2024-09-14T06:32:48.518 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:48.518 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:48.518 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:48.518 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:32:48.518 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.864269+0000", 2024-09-14T06:32:48.518 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.41e-07 2024-09-14T06:32:48.518 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:48.519 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:48.519 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:32:48.519 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.865490+0000", 2024-09-14T06:32:48.519 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.001220552 2024-09-14T06:32:48.519 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:48.519 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:48.519 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:32:48.519 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.865506+0000", 2024-09-14T06:32:48.519 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.6263999999999999e-05 2024-09-14T06:32:48.519 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:48.519 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:48.519 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:32:48.519 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.865516+0000", 2024-09-14T06:32:48.519 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.0091999999999999e-05 2024-09-14T06:32:48.519 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:48.519 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:48.520 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:32:48.520 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.865583+0000", 2024-09-14T06:32:48.520 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.6676999999999999e-05 2024-09-14T06:32:48.520 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:48.520 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:48.520 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:32:48.520 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.865610+0000", 2024-09-14T06:32:48.520 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.6529999999999998e-05 2024-09-14T06:32:48.520 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:48.520 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:48.520 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-14T06:32:48.520 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.867281+0000", 2024-09-14T06:32:48.520 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.001671203 2024-09-14T06:32:48.520 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:48.520 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:48.521 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:32:48.521 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.867324+0000", 2024-09-14T06:32:48.521 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.3665000000000003e-05 2024-09-14T06:32:48.521 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:48.521 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:32:48.521 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:48.521 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:48.521 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:48.521 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4397.0:74241 3.0 e73/23 3:06e93fb9:::rbd_data.112d80ca42af.0000000000000000:head v 73'102, mlcod=71'100)", 2024-09-14T06:32:48.521 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:53.876730+0000", 2024-09-14T06:32:48.521 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 174.638162429, 2024-09-14T06:32:48.521 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0079470329999999992, 2024-09-14T06:32:48.521 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:32:48.521 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:32:48.522 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:32:48.522 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:32:48.522 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:48.522 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:32:48.522 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:53.876730+0000", 2024-09-14T06:32:48.522 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:48.522 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:48.522 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:48.522 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:32:48.522 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:53.876730+0000", 2024-09-14T06:32:48.522 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:48.522 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:48.522 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:48.523 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:32:48.523 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:53.876731+0000", 2024-09-14T06:32:48.523 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.0500000000000004e-07 2024-09-14T06:32:48.523 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:48.523 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:48.523 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:32:48.523 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:53.879616+0000", 2024-09-14T06:32:48.523 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0028856680000000001 2024-09-14T06:32:48.523 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:48.523 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:48.523 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:32:48.523 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:53.879643+0000", 2024-09-14T06:32:48.523 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.6120999999999999e-05 2024-09-14T06:32:48.523 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:48.523 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:48.524 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:32:48.524 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:53.879654+0000", 2024-09-14T06:32:48.524 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.1586999999999999e-05 2024-09-14T06:32:48.524 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:48.524 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:48.524 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:32:48.524 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:53.879756+0000", 2024-09-14T06:32:48.524 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00010223899999999999 2024-09-14T06:32:48.524 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:48.524 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:48.524 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:32:48.524 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:53.879783+0000", 2024-09-14T06:32:48.524 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.6913000000000001e-05 2024-09-14T06:32:48.525 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:48.525 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:48.525 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-14T06:32:48.525 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:53.884594+0000", 2024-09-14T06:32:48.525 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0048110729999999999 2024-09-14T06:32:48.525 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:48.525 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:48.525 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:32:48.525 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:53.884677+0000", 2024-09-14T06:32:48.525 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.2927000000000001e-05 2024-09-14T06:32:48.525 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:48.525 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:32:48.525 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:48.525 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:48.526 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:48.526 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4397.0:74243 3.9 e73/23 3:94977f81:::rbd_data.112d80ca42af.0000000000000002:head v 73'94, mlcod=71'92)", 2024-09-14T06:32:48.526 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:53.885156+0000", 2024-09-14T06:32:48.526 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 174.629736511, 2024-09-14T06:32:48.526 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0096009860000000006, 2024-09-14T06:32:48.526 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:32:48.526 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:32:48.526 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:32:48.526 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:32:48.526 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:48.526 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:32:48.526 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:53.885156+0000", 2024-09-14T06:32:48.526 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:48.527 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:48.527 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:48.527 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:32:48.527 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:53.885156+0000", 2024-09-14T06:32:48.527 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:48.527 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:48.527 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:48.527 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:32:48.527 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:53.885156+0000", 2024-09-14T06:32:48.527 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.23e-07 2024-09-14T06:32:48.527 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:48.527 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:48.527 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:32:48.527 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:53.889694+0000", 2024-09-14T06:32:48.527 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0045378709999999997 2024-09-14T06:32:48.528 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:48.528 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:48.528 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:32:48.528 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:53.889720+0000", 2024-09-14T06:32:48.528 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.5259e-05 2024-09-14T06:32:48.528 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:48.528 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:48.528 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:32:48.528 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:53.889731+0000", 2024-09-14T06:32:48.528 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.1739000000000001e-05 2024-09-14T06:32:48.528 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:48.528 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:48.528 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:32:48.528 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:53.889822+0000", 2024-09-14T06:32:48.529 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.1200999999999996e-05 2024-09-14T06:32:48.529 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:48.529 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:48.529 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:32:48.529 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:53.889850+0000", 2024-09-14T06:32:48.529 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.7084999999999999e-05 2024-09-14T06:32:48.529 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:48.529 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:48.529 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-14T06:32:48.529 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:53.894703+0000", 2024-09-14T06:32:48.529 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0048532690000000003 2024-09-14T06:32:48.529 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:48.529 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:48.530 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:32:48.530 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:53.894757+0000", 2024-09-14T06:32:48.530 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.4339000000000003e-05 2024-09-14T06:32:48.530 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:48.530 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:32:48.530 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:48.530 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:48.530 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:48.530 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4397.0:74249 3.0 e73/23 3:009b0225:::rbd_data.112d80ca42af.0000000000000009:head v 73'103, mlcod=71'101)", 2024-09-14T06:32:48.530 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:53.893219+0000", 2024-09-14T06:32:48.530 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 174.62167403300001, 2024-09-14T06:32:48.530 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0065381980000000003, 2024-09-14T06:32:48.531 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:32:48.531 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:32:48.531 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:32:48.531 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:32:48.531 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:48.531 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:32:48.531 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:53.893219+0000", 2024-09-14T06:32:48.531 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:48.531 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:48.531 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:48.531 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:32:48.531 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:53.893219+0000", 2024-09-14T06:32:48.531 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:48.532 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:48.532 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:48.532 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:32:48.532 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:53.893221+0000", 2024-09-14T06:32:48.532 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.4329999999999998e-06 2024-09-14T06:32:48.532 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:48.532 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:48.532 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:32:48.532 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:53.895968+0000", 2024-09-14T06:32:48.532 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.002746746 2024-09-14T06:32:48.532 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:48.532 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:48.532 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:32:48.533 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:53.895988+0000", 2024-09-14T06:32:48.533 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.0227000000000001e-05 2024-09-14T06:32:48.533 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:48.533 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:48.533 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:32:48.533 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:53.896000+0000", 2024-09-14T06:32:48.533 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.2359e-05 2024-09-14T06:32:48.533 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:48.533 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:48.533 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:32:48.533 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:53.896084+0000", 2024-09-14T06:32:48.533 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.3659999999999995e-05 2024-09-14T06:32:48.533 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:48.533 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:48.533 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:32:48.534 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:53.896109+0000", 2024-09-14T06:32:48.534 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.5259999999999999e-05 2024-09-14T06:32:48.534 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:48.534 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:48.534 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-14T06:32:48.534 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:53.899704+0000", 2024-09-14T06:32:48.534 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0035942209999999999 2024-09-14T06:32:48.534 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:48.534 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:48.534 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:32:48.534 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:53.899757+0000", 2024-09-14T06:32:48.534 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.3291999999999997e-05 2024-09-14T06:32:48.534 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:48.534 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:32:48.535 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:48.535 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:48.535 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:48.535 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4397.0:149885 3.9 e122/23 3:951acef3:::rbd_data.112d1bb7b269.0000000000000005:head v 122'176, mlcod=122'172)", 2024-09-14T06:32:48.535 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:30:44.739092+0000", 2024-09-14T06:32:48.535 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 123.775800283, 2024-09-14T06:32:48.535 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0028236659999999998, 2024-09-14T06:32:48.535 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:32:48.535 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:32:48.535 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:32:48.535 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:32:48.535 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:48.535 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:32:48.535 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.739092+0000", 2024-09-14T06:32:48.535 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:48.536 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:48.536 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:48.536 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:32:48.536 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.739092+0000", 2024-09-14T06:32:48.536 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:48.536 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:48.536 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:48.536 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:32:48.536 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.739093+0000", 2024-09-14T06:32:48.536 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.0109999999999999e-06 2024-09-14T06:32:48.536 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:48.536 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:48.536 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:32:48.536 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.740189+0000", 2024-09-14T06:32:48.537 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00109527 2024-09-14T06:32:48.537 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:48.537 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:48.537 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:32:48.537 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.740199+0000", 2024-09-14T06:32:48.537 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.0458999999999999e-05 2024-09-14T06:32:48.537 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:48.537 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:48.537 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:32:48.537 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.740207+0000", 2024-09-14T06:32:48.537 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.4499999999999998e-06 2024-09-14T06:32:48.537 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:48.537 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:48.537 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:32:48.538 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.740295+0000", 2024-09-14T06:32:48.538 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.8740999999999996e-05 2024-09-14T06:32:48.538 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:48.538 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:48.538 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:32:48.538 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.740317+0000", 2024-09-14T06:32:48.538 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.1523e-05 2024-09-14T06:32:48.538 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:48.538 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:48.538 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-14T06:32:48.538 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.741865+0000", 2024-09-14T06:32:48.538 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.001548315 2024-09-14T06:32:48.538 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:48.538 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:48.538 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:32:48.538 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.741916+0000", 2024-09-14T06:32:48.539 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.0896999999999997e-05 2024-09-14T06:32:48.539 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:48.539 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:32:48.539 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:48.539 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:48.539 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:48.539 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4397.0:149933 3.4 e122/23 3:234f5229:::rbd_data.112d1bb7b269.0000000000000009:head v 122'221, mlcod=120'218)", 2024-09-14T06:32:48.539 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:30:44.743849+0000", 2024-09-14T06:32:48.539 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 123.771043866, 2024-09-14T06:32:48.539 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0031479519999999999, 2024-09-14T06:32:48.539 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:32:48.539 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:32:48.539 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:32:48.539 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:32:48.540 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:48.540 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:32:48.540 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.743849+0000", 2024-09-14T06:32:48.540 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:48.540 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:48.540 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:48.540 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:32:48.540 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.743849+0000", 2024-09-14T06:32:48.540 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:48.540 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:48.540 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:48.540 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:32:48.540 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.743850+0000", 2024-09-14T06:32:48.541 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.279e-06 2024-09-14T06:32:48.541 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:48.541 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:48.541 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:32:48.541 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.745151+0000", 2024-09-14T06:32:48.541 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.001300495 2024-09-14T06:32:48.541 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:48.541 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:48.541 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:32:48.541 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.745167+0000", 2024-09-14T06:32:48.541 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.6478e-05 2024-09-14T06:32:48.541 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:48.542 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:48.542 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:32:48.542 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.745175+0000", 2024-09-14T06:32:48.542 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.2679999999999994e-06 2024-09-14T06:32:48.542 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:48.542 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:48.542 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:32:48.542 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.745226+0000", 2024-09-14T06:32:48.542 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.0192999999999999e-05 2024-09-14T06:32:48.542 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:48.542 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:48.542 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:32:48.542 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.745247+0000", 2024-09-14T06:32:48.542 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.1767e-05 2024-09-14T06:32:48.543 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:48.543 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:48.543 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-14T06:32:48.543 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.746952+0000", 2024-09-14T06:32:48.543 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0017042769999999999 2024-09-14T06:32:48.543 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:48.543 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:48.543 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:32:48.543 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.746997+0000", 2024-09-14T06:32:48.543 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.5194999999999997e-05 2024-09-14T06:32:48.543 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:48.543 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:32:48.543 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:48.543 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:48.543 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:48.544 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4397.0:150494 3.9 e122/23 3:955152a4:::rbd_object_map.112d1bb7b269.0000000000000038:head v 122'194, mlcod=122'190)", 2024-09-14T06:32:48.544 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:30:44.781585+0000", 2024-09-14T06:32:48.544 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 123.733307974, 2024-09-14T06:32:48.544 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0026957499999999998, 2024-09-14T06:32:48.544 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:32:48.544 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:32:48.544 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:32:48.544 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:32:48.544 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:48.544 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:32:48.544 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.781585+0000", 2024-09-14T06:32:48.544 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:48.544 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:48.544 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:48.545 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:32:48.545 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.781585+0000", 2024-09-14T06:32:48.545 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:48.545 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:48.545 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:48.545 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:32:48.545 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.781585+0000", 2024-09-14T06:32:48.545 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.5000000000000002e-07 2024-09-14T06:32:48.545 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:48.545 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:48.545 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:32:48.545 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.781594+0000", 2024-09-14T06:32:48.545 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.4640000000000006e-06 2024-09-14T06:32:48.546 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:48.546 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:48.546 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:32:48.546 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.781604+0000", 2024-09-14T06:32:48.546 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.0071999999999999e-05 2024-09-14T06:32:48.546 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:48.546 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:48.546 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:32:48.546 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.781610+0000", 2024-09-14T06:32:48.546 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.3609999999999997e-06 2024-09-14T06:32:48.546 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:48.546 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:48.546 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:32:48.546 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.781676+0000", 2024-09-14T06:32:48.547 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.5963000000000005e-05 2024-09-14T06:32:48.547 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:48.547 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:48.547 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:32:48.547 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.781696+0000", 2024-09-14T06:32:48.547 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.9278000000000001e-05 2024-09-14T06:32:48.547 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:48.547 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:48.547 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-14T06:32:48.547 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.784231+0000", 2024-09-14T06:32:48.547 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.002535093 2024-09-14T06:32:48.547 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:48.547 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:48.547 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:32:48.547 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.784280+0000", 2024-09-14T06:32:48.547 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.9869e-05 2024-09-14T06:32:48.548 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:48.548 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:32:48.548 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:48.548 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:48.548 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:48.548 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4397.0:204589 3.4 e166/23 3:234552f0:::rbd_data.112d3bd84cb0.000000000000003f:head v 166'499, mlcod=165'497)", 2024-09-14T06:32:48.548 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:31:30.772894+0000", 2024-09-14T06:32:48.548 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 77.741998378999995, 2024-09-14T06:32:48.548 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0034564650000000001, 2024-09-14T06:32:48.548 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:32:48.548 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:32:48.548 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:32:48.548 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:32:48.548 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:48.548 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:32:48.549 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:30.772894+0000", 2024-09-14T06:32:48.549 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:48.549 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:48.549 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:48.549 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:32:48.549 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:30.772894+0000", 2024-09-14T06:32:48.549 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:48.549 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:48.549 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:48.549 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:32:48.549 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:30.772896+0000", 2024-09-14T06:32:48.549 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.5379999999999999e-06 2024-09-14T06:32:48.549 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:48.550 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:48.550 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:32:48.550 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:30.773042+0000", 2024-09-14T06:32:48.550 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000146107 2024-09-14T06:32:48.550 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:48.550 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:48.550 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:32:48.550 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:30.773061+0000", 2024-09-14T06:32:48.550 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.8862999999999999e-05 2024-09-14T06:32:48.550 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:48.550 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:48.550 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:32:48.550 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:30.773071+0000", 2024-09-14T06:32:48.550 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.0227000000000001e-05 2024-09-14T06:32:48.551 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:48.551 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:48.551 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:32:48.551 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:30.773149+0000", 2024-09-14T06:32:48.551 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.8071999999999994e-05 2024-09-14T06:32:48.551 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:48.551 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:48.551 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:32:48.551 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:30.773171+0000", 2024-09-14T06:32:48.551 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.1699999999999999e-05 2024-09-14T06:32:48.551 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:48.551 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:48.551 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-14T06:32:48.551 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:30.776285+0000", 2024-09-14T06:32:48.552 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0031142050000000001 2024-09-14T06:32:48.552 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:48.552 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:48.552 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:32:48.552 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:30.776351+0000", 2024-09-14T06:32:48.552 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.5753000000000006e-05 2024-09-14T06:32:48.552 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:48.552 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:32:48.552 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:48.552 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:48.552 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:48.552 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4397.0:204598 3.4 e166/23 3:245d069e:::rbd_data.112d3bd84cb0.0000000000000041:head v 166'500, mlcod=165'497)", 2024-09-14T06:32:48.552 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:31:30.775247+0000", 2024-09-14T06:32:48.552 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 77.739645256000003, 2024-09-14T06:32:48.553 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00318235, 2024-09-14T06:32:48.553 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:32:48.553 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:32:48.553 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:32:48.553 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:32:48.553 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:48.553 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:32:48.553 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:30.775247+0000", 2024-09-14T06:32:48.553 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:48.553 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:48.553 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:48.553 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:32:48.553 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:30.775247+0000", 2024-09-14T06:32:48.553 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:48.554 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:48.554 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:48.554 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:32:48.554 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:30.775249+0000", 2024-09-14T06:32:48.554 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.265e-06 2024-09-14T06:32:48.554 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:48.554 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:48.554 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:32:48.554 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:30.775340+0000", 2024-09-14T06:32:48.554 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.1131999999999994e-05 2024-09-14T06:32:48.554 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:48.554 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:48.554 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:32:48.554 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:30.775352+0000", 2024-09-14T06:32:48.555 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.2294e-05 2024-09-14T06:32:48.555 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:48.555 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:48.555 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:32:48.555 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:30.775361+0000", 2024-09-14T06:32:48.555 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.6950000000000006e-06 2024-09-14T06:32:48.555 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:48.555 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:48.555 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:32:48.555 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:30.775439+0000", 2024-09-14T06:32:48.555 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.7910000000000005e-05 2024-09-14T06:32:48.555 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:48.555 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:48.555 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:32:48.556 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:30.775460+0000", 2024-09-14T06:32:48.556 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.0900999999999999e-05 2024-09-14T06:32:48.556 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:48.556 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:48.556 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-14T06:32:48.556 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:30.778385+0000", 2024-09-14T06:32:48.556 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.002925539 2024-09-14T06:32:48.556 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:48.556 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:48.556 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:32:48.556 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:30.778430+0000", 2024-09-14T06:32:48.556 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.4613999999999997e-05 2024-09-14T06:32:48.556 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:48.557 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:32:48.557 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:48.557 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:48.557 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:48.557 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "MOSDPGPush(3.5 169/168 [PushOp(3:a7a54476:::rbd_data.112d1bb7b269.0000000000000038:head, version: 122'581, data_included: [0~524288], data_size: 524288, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(3:a7a54476:::rbd_data.112d1bb7b269.0000000000000038:head@122'581, size: 524288, copy_subset: [(0, 524288)], clone_subset: {}, snapset: 0=[]:{}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 524288, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2024-09-14T06:32:48.557 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:31:34.667757+0000", 2024-09-14T06:32:48.557 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 73.847135833999999, 2024-09-14T06:32:48.557 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.002750382, 2024-09-14T06:32:48.557 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:32:48.557 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:32:48.557 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:32:48.557 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:32:48.557 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:48.557 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:32:48.558 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:34.667757+0000", 2024-09-14T06:32:48.558 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:48.558 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:48.558 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:48.558 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:32:48.558 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:34.667757+0000", 2024-09-14T06:32:48.558 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:48.558 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:48.558 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:48.558 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:32:48.558 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:34.667758+0000", 2024-09-14T06:32:48.558 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.3599999999999999e-06 2024-09-14T06:32:48.558 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:48.559 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:48.559 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:32:48.559 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:34.669375+0000", 2024-09-14T06:32:48.559 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.001616644 2024-09-14T06:32:48.559 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:48.559 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:48.559 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:32:48.559 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:34.669404+0000", 2024-09-14T06:32:48.559 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.9300999999999999e-05 2024-09-14T06:32:48.559 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:48.559 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:48.559 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:32:48.559 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:34.669428+0000", 2024-09-14T06:32:48.559 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.3788e-05 2024-09-14T06:32:48.560 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:48.560 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:48.560 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:32:48.560 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:34.669543+0000", 2024-09-14T06:32:48.560 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000115462 2024-09-14T06:32:48.560 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:48.560 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:48.560 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:32:48.560 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:34.669559+0000", 2024-09-14T06:32:48.560 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.5167999999999999e-05 2024-09-14T06:32:48.560 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:48.560 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:48.560 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:32:48.560 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:34.670507+0000", 2024-09-14T06:32:48.561 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00094865900000000003 2024-09-14T06:32:48.561 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:48.561 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:32:48.561 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:48.561 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:48.561 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:48.561 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "MOSDPGPush(3.5 170/168 [PushOp(3:a162138c:::rbd_data.112d1bb7b269.000000000000003c:head, version: 122'582, data_included: [0~524288], data_size: 524288, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(3:a162138c:::rbd_data.112d1bb7b269.000000000000003c:head@122'582, size: 524288, copy_subset: [(0, 524288)], clone_subset: {}, snapset: 0=[]:{}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 524288, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2024-09-14T06:32:48.561 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:31:34.670110+0000", 2024-09-14T06:32:48.561 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 73.844782885000001, 2024-09-14T06:32:48.561 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0030066229999999999, 2024-09-14T06:32:48.561 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:32:48.561 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:32:48.561 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:32:48.562 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:32:48.562 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:48.562 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:32:48.562 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:34.670110+0000", 2024-09-14T06:32:48.562 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:48.562 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:48.562 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:48.562 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:32:48.562 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:34.670110+0000", 2024-09-14T06:32:48.562 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:48.562 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:48.562 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:48.562 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:32:48.562 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:34.670111+0000", 2024-09-14T06:32:48.563 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.6499999999999998e-07 2024-09-14T06:32:48.563 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:48.563 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:48.563 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:32:48.563 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:34.671507+0000", 2024-09-14T06:32:48.563 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.001396284 2024-09-14T06:32:48.563 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:48.564 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:48.564 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:32:48.564 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:34.671530+0000", 2024-09-14T06:32:48.564 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.3023e-05 2024-09-14T06:32:48.564 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:48.564 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:48.564 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:32:48.564 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:34.671548+0000", 2024-09-14T06:32:48.564 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.8116000000000001e-05 2024-09-14T06:32:48.564 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:48.564 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:48.564 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:32:48.564 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:34.671775+0000", 2024-09-14T06:32:48.565 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00022699600000000001 2024-09-14T06:32:48.565 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:48.565 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:48.565 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:32:48.565 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:34.671790+0000", 2024-09-14T06:32:48.565 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.539e-05 2024-09-14T06:32:48.565 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:48.565 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:48.565 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:32:48.565 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:34.673116+0000", 2024-09-14T06:32:48.565 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0013259490000000001 2024-09-14T06:32:48.565 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:48.565 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:32:48.565 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:48.565 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:48.566 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:48.566 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:236487 3.14 3:2d502bc2:::rbd_data.112d97471187.000000000000008e:head [delete] snapc 2f=[] ondisk+write+known_if_redirected+supports_pool_eio e234)", 2024-09-14T06:32:48.566 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:32:39.870309+0000", 2024-09-14T06:32:48.566 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 8.6445833570000001, 2024-09-14T06:32:48.566 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0042368170000000004, 2024-09-14T06:32:48.566 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:32:48.566 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:32:48.566 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:32:48.566 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:32:48.566 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:32:48.566 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:48.566 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 236487 2024-09-14T06:32:48.566 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:48.566 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:32:48.566 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:48.566 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:32:48.567 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:39.870309+0000", 2024-09-14T06:32:48.567 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:48.567 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:48.567 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:48.567 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:32:48.567 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:39.870309+0000", 2024-09-14T06:32:48.567 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:48.567 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:48.567 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:48.567 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:32:48.567 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:39.870311+0000", 2024-09-14T06:32:48.567 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.0080000000000001e-06 2024-09-14T06:32:48.567 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:48.567 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:48.568 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:32:48.568 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:39.870316+0000", 2024-09-14T06:32:48.568 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.7160000000000002e-06 2024-09-14T06:32:48.568 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:48.568 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:48.568 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:32:48.568 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:39.870327+0000", 2024-09-14T06:32:48.568 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.0995999999999999e-05 2024-09-14T06:32:48.568 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:48.568 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:48.568 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:32:48.568 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:39.870335+0000", 2024-09-14T06:32:48.568 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.2139999999999996e-06 2024-09-14T06:32:48.568 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:48.569 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:48.569 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:32:48.569 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:39.870403+0000", 2024-09-14T06:32:48.569 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.7966999999999995e-05 2024-09-14T06:32:48.569 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:48.569 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:48.569 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:32:48.569 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:39.870543+0000", 2024-09-14T06:32:48.569 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00014018799999999999 2024-09-14T06:32:48.569 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:48.569 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:48.569 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "waiting for subops from 7", 2024-09-14T06:32:48.569 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:39.870669+0000", 2024-09-14T06:32:48.569 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000125778 2024-09-14T06:32:48.570 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:48.570 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:48.570 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "op_commit", 2024-09-14T06:32:48.570 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:39.871410+0000", 2024-09-14T06:32:48.570 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00074113599999999999 2024-09-14T06:32:48.570 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:48.570 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:48.570 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:32:48.570 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:39.874439+0000", 2024-09-14T06:32:48.570 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0030287259999999998 2024-09-14T06:32:48.570 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:48.570 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:48.570 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-14T06:32:48.570 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:39.874495+0000", 2024-09-14T06:32:48.570 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.5855e-05 2024-09-14T06:32:48.571 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:48.571 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:48.571 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:32:48.571 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:39.874546+0000", 2024-09-14T06:32:48.571 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.1233000000000002e-05 2024-09-14T06:32:48.571 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:48.571 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:32:48.571 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:48.571 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:48.571 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:48.571 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:236490 3.14 3:2e5aef87:::rbd_data.112d97471187.00000000000000ca:head [delete] snapc 2f=[] ondisk+write+known_if_redirected+supports_pool_eio e234)", 2024-09-14T06:32:48.571 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:32:39.872130+0000", 2024-09-14T06:32:48.571 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 8.6427627059999992, 2024-09-14T06:32:48.571 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0048885589999999998, 2024-09-14T06:32:48.571 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:32:48.572 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:32:48.572 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:32:48.572 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:32:48.572 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:32:48.572 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:48.572 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 236490 2024-09-14T06:32:48.572 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:48.572 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:32:48.572 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:48.572 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:32:48.572 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:39.872130+0000", 2024-09-14T06:32:48.572 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:48.572 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:48.572 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:48.573 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:32:48.573 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:39.872130+0000", 2024-09-14T06:32:48.573 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:48.573 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:48.573 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:48.573 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:32:48.573 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:39.872131+0000", 2024-09-14T06:32:48.573 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.0720000000000001e-06 2024-09-14T06:32:48.573 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:48.573 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:48.573 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:32:48.573 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:39.872136+0000", 2024-09-14T06:32:48.573 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.8790000000000002e-06 2024-09-14T06:32:48.573 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:48.573 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:48.574 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:32:48.574 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:39.872150+0000", 2024-09-14T06:32:48.574 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.435e-05 2024-09-14T06:32:48.574 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:48.574 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:48.574 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:32:48.574 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:39.872158+0000", 2024-09-14T06:32:48.574 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.571e-06 2024-09-14T06:32:48.574 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:48.574 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:48.574 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:32:48.574 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:39.872234+0000", 2024-09-14T06:32:48.574 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.606e-05 2024-09-14T06:32:48.574 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:48.574 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:48.575 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:32:48.575 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:39.872356+0000", 2024-09-14T06:32:48.575 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00012250499999999999 2024-09-14T06:32:48.575 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:48.575 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:48.575 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "waiting for subops from 7", 2024-09-14T06:32:48.575 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:39.872480+0000", 2024-09-14T06:32:48.575 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00012316700000000001 2024-09-14T06:32:48.575 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:48.575 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:48.575 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "op_commit", 2024-09-14T06:32:48.575 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:39.873051+0000", 2024-09-14T06:32:48.575 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00057178100000000005 2024-09-14T06:32:48.575 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:48.575 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:48.575 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:32:48.576 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:39.876925+0000", 2024-09-14T06:32:48.576 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0038739239999999999 2024-09-14T06:32:48.576 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:48.576 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:48.576 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-14T06:32:48.576 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:39.876974+0000", 2024-09-14T06:32:48.576 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.8207000000000001e-05 2024-09-14T06:32:48.576 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:48.576 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:48.576 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:32:48.576 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:39.877019+0000", 2024-09-14T06:32:48.576 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.5043000000000002e-05 2024-09-14T06:32:48.576 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:48.576 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:32:48.577 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:48.577 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:48.577 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:48.577 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:237283 3.14 3:2f6164a9:::rbd_data.112d401cec05.000000000000000a:head [delete] snapc 25=[] ondisk+write+known_if_redirected+supports_pool_eio e239)", 2024-09-14T06:32:48.577 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:32:44.693535+0000", 2024-09-14T06:32:48.577 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 3.8213576520000001, 2024-09-14T06:32:48.577 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0070517319999999998, 2024-09-14T06:32:48.577 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:32:48.577 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:32:48.577 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:32:48.577 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:32:48.577 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:32:48.577 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:48.577 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 237283 2024-09-14T06:32:48.578 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:48.578 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:32:48.578 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:48.578 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:32:48.578 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:44.693535+0000", 2024-09-14T06:32:48.578 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:48.578 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:48.578 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:48.578 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:32:48.578 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:44.693535+0000", 2024-09-14T06:32:48.578 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:48.578 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:48.578 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:48.578 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:32:48.578 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:44.693537+0000", 2024-09-14T06:32:48.578 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.4649999999999999e-06 2024-09-14T06:32:48.579 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:48.579 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:48.579 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:32:48.579 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:44.693541+0000", 2024-09-14T06:32:48.579 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.4950000000000002e-06 2024-09-14T06:32:48.579 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:48.579 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:48.579 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:32:48.579 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:44.693553+0000", 2024-09-14T06:32:48.579 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.2238e-05 2024-09-14T06:32:48.579 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:48.579 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:48.579 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:32:48.579 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:44.693561+0000", 2024-09-14T06:32:48.580 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.4379999999999998e-06 2024-09-14T06:32:48.580 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:48.580 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:48.580 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:32:48.580 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:44.693633+0000", 2024-09-14T06:32:48.580 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.2234e-05 2024-09-14T06:32:48.580 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:48.580 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:48.580 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "op must wait for map", 2024-09-14T06:32:48.580 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:44.693642+0000", 2024-09-14T06:32:48.580 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.6770000000000001e-06 2024-09-14T06:32:48.580 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:48.580 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:48.580 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:32:48.580 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:44.695664+0000", 2024-09-14T06:32:48.581 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.002022252 2024-09-14T06:32:48.581 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:48.581 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:48.581 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:32:48.581 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:44.695825+0000", 2024-09-14T06:32:48.581 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00016144099999999999 2024-09-14T06:32:48.581 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:48.581 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:48.581 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "waiting for subops from 7", 2024-09-14T06:32:48.581 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:44.695947+0000", 2024-09-14T06:32:48.581 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000121674 2024-09-14T06:32:48.581 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:48.581 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:48.581 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "op_commit", 2024-09-14T06:32:48.582 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:44.696853+0000", 2024-09-14T06:32:48.582 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00090605000000000004 2024-09-14T06:32:48.582 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:48.582 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:48.582 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:32:48.582 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:44.700467+0000", 2024-09-14T06:32:48.582 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0036137909999999999 2024-09-14T06:32:48.582 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:48.582 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:48.582 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-14T06:32:48.582 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:44.700533+0000", 2024-09-14T06:32:48.582 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.6583999999999998e-05 2024-09-14T06:32:48.582 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:48.583 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:48.583 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:32:48.583 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:44.700587+0000", 2024-09-14T06:32:48.583 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.3393000000000002e-05 2024-09-14T06:32:48.583 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:48.583 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:32:48.583 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:48.583 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:48.583 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:48.583 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:237364 3.14 3:2a8d6252:::rbd_data.112d401cec05.000000000000005a:head [delete] snapc 25=[] ondisk+write+known_if_redirected+supports_pool_eio e239)", 2024-09-14T06:32:48.583 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:32:44.724057+0000", 2024-09-14T06:32:48.583 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 3.79083551, 2024-09-14T06:32:48.583 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.005405923, 2024-09-14T06:32:48.583 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:32:48.583 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:32:48.583 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:32:48.584 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:32:48.584 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:32:48.584 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:48.584 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 237364 2024-09-14T06:32:48.584 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:48.584 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:32:48.584 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:48.584 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:32:48.584 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:44.724057+0000", 2024-09-14T06:32:48.584 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:48.584 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:48.584 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:48.585 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:32:48.585 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:44.724057+0000", 2024-09-14T06:32:48.585 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:48.585 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:48.585 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:48.585 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:32:48.585 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:44.724059+0000", 2024-09-14T06:32:48.585 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.854e-06 2024-09-14T06:32:48.585 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:48.585 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:48.585 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:32:48.585 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:44.724066+0000", 2024-09-14T06:32:48.585 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.4410000000000004e-06 2024-09-14T06:32:48.586 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:48.586 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:48.586 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:32:48.586 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:44.724086+0000", 2024-09-14T06:32:48.586 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.9128e-05 2024-09-14T06:32:48.586 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:48.586 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:48.586 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:32:48.586 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:44.724095+0000", 2024-09-14T06:32:48.586 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.4900000000000006e-06 2024-09-14T06:32:48.586 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:48.586 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:48.586 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:32:48.587 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:44.724172+0000", 2024-09-14T06:32:48.587 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.7226999999999998e-05 2024-09-14T06:32:48.587 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:48.587 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:48.587 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:32:48.587 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:44.724334+0000", 2024-09-14T06:32:48.587 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000161545 2024-09-14T06:32:48.587 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:48.587 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:48.587 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "waiting for subops from 7", 2024-09-14T06:32:48.587 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:44.724472+0000", 2024-09-14T06:32:48.587 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00013770300000000001 2024-09-14T06:32:48.587 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:48.587 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:48.588 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "op_commit", 2024-09-14T06:32:48.588 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:44.725367+0000", 2024-09-14T06:32:48.588 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00089564699999999996 2024-09-14T06:32:48.588 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:48.588 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:48.588 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:32:48.588 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:44.729349+0000", 2024-09-14T06:32:48.588 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0039818539999999999 2024-09-14T06:32:48.588 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:48.588 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:48.588 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-14T06:32:48.588 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:44.729410+0000", 2024-09-14T06:32:48.588 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.0523000000000003e-05 2024-09-14T06:32:48.588 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:48.589 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:48.589 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:32:48.589 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:44.729463+0000", 2024-09-14T06:32:48.589 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.3511000000000003e-05 2024-09-14T06:32:48.589 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:48.589 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:32:48.589 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:48.589 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:48.589 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:48.589 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:237438 3.14 3:2b4d012e:::rbd_data.112d401cec05.00000000000000ae:head [delete] snapc 25=[] ondisk+write+known_if_redirected+supports_pool_eio e239)", 2024-09-14T06:32:48.589 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:32:44.755579+0000", 2024-09-14T06:32:48.589 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 3.7593133839999999, 2024-09-14T06:32:48.589 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0056954090000000002, 2024-09-14T06:32:48.589 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:32:48.589 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:32:48.590 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:32:48.590 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:32:48.590 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:32:48.590 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:48.590 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 237438 2024-09-14T06:32:48.590 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:48.590 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:32:48.590 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:48.590 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:32:48.590 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:44.755579+0000", 2024-09-14T06:32:48.590 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:48.590 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:48.591 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:48.591 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:32:48.591 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:44.755579+0000", 2024-09-14T06:32:48.591 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:48.591 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:48.591 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:48.591 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:32:48.591 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:44.755581+0000", 2024-09-14T06:32:48.591 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.9099999999999999e-06 2024-09-14T06:32:48.591 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:48.591 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:48.591 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:32:48.591 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:44.755588+0000", 2024-09-14T06:32:48.591 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.9580000000000001e-06 2024-09-14T06:32:48.591 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:48.592 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:48.592 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:32:48.592 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:44.755605+0000", 2024-09-14T06:32:48.592 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.6481e-05 2024-09-14T06:32:48.592 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:48.592 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:48.592 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:32:48.592 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:44.755613+0000", 2024-09-14T06:32:48.592 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.2490000000000001e-06 2024-09-14T06:32:48.592 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:48.592 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:48.592 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:32:48.592 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:44.755706+0000", 2024-09-14T06:32:48.592 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.3346999999999999e-05 2024-09-14T06:32:48.593 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:48.593 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:48.593 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:32:48.593 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:44.755858+0000", 2024-09-14T06:32:48.593 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00015165600000000001 2024-09-14T06:32:48.593 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:48.593 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:48.593 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "waiting for subops from 7", 2024-09-14T06:32:48.593 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:44.755984+0000", 2024-09-14T06:32:48.593 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000125907 2024-09-14T06:32:48.593 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:48.593 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:48.593 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "op_commit", 2024-09-14T06:32:48.593 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:44.756892+0000", 2024-09-14T06:32:48.594 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00090802199999999997 2024-09-14T06:32:48.594 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:48.594 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:48.594 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:32:48.594 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:44.761135+0000", 2024-09-14T06:32:48.594 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.004243251 2024-09-14T06:32:48.594 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:48.594 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:48.594 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-14T06:32:48.594 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:44.761211+0000", 2024-09-14T06:32:48.594 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.6027999999999998e-05 2024-09-14T06:32:48.594 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:48.594 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:48.594 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:32:48.595 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:44.761275+0000", 2024-09-14T06:32:48.595 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.3600000000000001e-05 2024-09-14T06:32:48.595 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:48.595 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:32:48.595 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:48.595 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:48.595 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:48.595 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:237445 3.14 3:2e12370a:::rbd_data.112d401cec05.00000000000000b5:head [delete] snapc 25=[] ondisk+write+known_if_redirected+supports_pool_eio e239)", 2024-09-14T06:32:48.595 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:32:44.757373+0000", 2024-09-14T06:32:48.595 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 3.7575195610000001, 2024-09-14T06:32:48.595 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0063576639999999999, 2024-09-14T06:32:48.595 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:32:48.595 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:32:48.595 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:32:48.596 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:32:48.596 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:32:48.596 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:48.596 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 237445 2024-09-14T06:32:48.596 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:48.596 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:32:48.596 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:48.596 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:32:48.596 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:44.757373+0000", 2024-09-14T06:32:48.596 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:48.596 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:48.596 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:48.596 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:32:48.596 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:44.757373+0000", 2024-09-14T06:32:48.596 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:48.597 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:48.597 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:48.597 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:32:48.597 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:44.757374+0000", 2024-09-14T06:32:48.597 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.359e-06 2024-09-14T06:32:48.597 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:48.597 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:48.597 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:32:48.597 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:44.757381+0000", 2024-09-14T06:32:48.597 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.7769999999999997e-06 2024-09-14T06:32:48.597 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:48.597 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:48.597 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:32:48.597 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:44.757397+0000", 2024-09-14T06:32:48.598 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.5610000000000001e-05 2024-09-14T06:32:48.598 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:48.598 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:48.598 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:32:48.598 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:44.757406+0000", 2024-09-14T06:32:48.598 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.8300000000000002e-06 2024-09-14T06:32:48.598 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:48.598 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:48.598 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:32:48.598 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:44.757462+0000", 2024-09-14T06:32:48.598 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.6348000000000003e-05 2024-09-14T06:32:48.598 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:48.598 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:48.599 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:32:48.599 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:44.757579+0000", 2024-09-14T06:32:48.599 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000116747 2024-09-14T06:32:48.599 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:48.599 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:48.599 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "waiting for subops from 7", 2024-09-14T06:32:48.599 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:44.757704+0000", 2024-09-14T06:32:48.599 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00012546899999999999 2024-09-14T06:32:48.599 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:48.599 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:48.599 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "op_commit", 2024-09-14T06:32:48.599 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:44.758510+0000", 2024-09-14T06:32:48.599 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00080536099999999999 2024-09-14T06:32:48.599 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:48.600 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:48.600 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:32:48.600 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:44.763596+0000", 2024-09-14T06:32:48.600 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0050862049999999999 2024-09-14T06:32:48.600 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:48.600 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:48.600 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-14T06:32:48.600 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:44.763673+0000", 2024-09-14T06:32:48.600 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.7464000000000002e-05 2024-09-14T06:32:48.600 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:48.600 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:48.600 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:32:48.600 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:44.763731+0000", 2024-09-14T06:32:48.600 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.7494000000000002e-05 2024-09-14T06:32:48.600 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:48.601 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:32:48.601 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:48.601 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:48.601 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:48.601 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4397.0:238175 3.0 e242/23 3:06e93fb9:::rbd_data.112d80ca42af.0000000000000000:head v 242'1084, mlcod=241'1082)", 2024-09-14T06:32:48.601 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:32:47.710122+0000", 2024-09-14T06:32:48.601 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 0.80477103299999997, 2024-09-14T06:32:48.601 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.002719062, 2024-09-14T06:32:48.601 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:32:48.601 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:32:48.601 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:32:48.601 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:32:48.601 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:48.602 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:32:48.602 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:47.710122+0000", 2024-09-14T06:32:48.602 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:48.602 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:48.602 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:48.602 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:32:48.602 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:47.710122+0000", 2024-09-14T06:32:48.602 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:48.602 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:48.602 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:48.602 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:32:48.602 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:47.710123+0000", 2024-09-14T06:32:48.602 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.3600000000000002e-07 2024-09-14T06:32:48.602 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:48.603 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:48.603 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:32:48.603 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:47.710129+0000", 2024-09-14T06:32:48.603 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.7789999999999998e-06 2024-09-14T06:32:48.603 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:48.603 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:48.603 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:32:48.603 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:47.710142+0000", 2024-09-14T06:32:48.603 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.2904e-05 2024-09-14T06:32:48.603 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:48.603 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:48.603 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:32:48.603 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:47.710149+0000", 2024-09-14T06:32:48.603 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.1559999999999998e-06 2024-09-14T06:32:48.603 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:48.603 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:48.604 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:32:48.604 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:47.710205+0000", 2024-09-14T06:32:48.604 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.5377000000000002e-05 2024-09-14T06:32:48.604 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:48.604 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:48.604 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:32:48.604 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:47.710229+0000", 2024-09-14T06:32:48.604 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.3836e-05 2024-09-14T06:32:48.604 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:48.604 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:48.604 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-14T06:32:48.604 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:47.712786+0000", 2024-09-14T06:32:48.604 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0025572009999999998 2024-09-14T06:32:48.604 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:48.604 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:48.605 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:32:48.605 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:47.712841+0000", 2024-09-14T06:32:48.605 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.4972999999999997e-05 2024-09-14T06:32:48.605 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:48.605 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:32:48.605 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:48.605 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:48.605 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:48.605 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4397.0:238177 3.9 e242/23 3:94977f81:::rbd_data.112d80ca42af.0000000000000002:head v 242'570, mlcod=241'568)", 2024-09-14T06:32:48.605 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:32:47.711459+0000", 2024-09-14T06:32:48.605 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 0.80343413799999996, 2024-09-14T06:32:48.605 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.002852551, 2024-09-14T06:32:48.605 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:32:48.605 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:32:48.606 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:32:48.606 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:32:48.606 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:48.606 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:32:48.606 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:47.711459+0000", 2024-09-14T06:32:48.606 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:48.606 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:48.606 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:48.606 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:32:48.606 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:47.711459+0000", 2024-09-14T06:32:48.606 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:48.606 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:48.606 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:48.606 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:32:48.606 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:47.711459+0000", 2024-09-14T06:32:48.607 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.2600000000000002e-07 2024-09-14T06:32:48.607 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:48.607 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:48.607 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:32:48.607 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:47.711465+0000", 2024-09-14T06:32:48.607 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.4310000000000002e-06 2024-09-14T06:32:48.607 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:48.607 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:48.607 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:32:48.607 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:47.711474+0000", 2024-09-14T06:32:48.607 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.5070000000000006e-06 2024-09-14T06:32:48.607 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:48.607 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:48.607 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:32:48.607 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:47.711480+0000", 2024-09-14T06:32:48.608 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.8839999999999999e-06 2024-09-14T06:32:48.608 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:48.608 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:48.608 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:32:48.608 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:47.711638+0000", 2024-09-14T06:32:48.608 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00015806800000000001 2024-09-14T06:32:48.608 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:48.608 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:48.608 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:32:48.608 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:47.711660+0000", 2024-09-14T06:32:48.608 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.1481e-05 2024-09-14T06:32:48.608 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:48.608 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:48.609 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-14T06:32:48.609 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:47.714263+0000", 2024-09-14T06:32:48.609 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0026039380000000001 2024-09-14T06:32:48.609 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:48.609 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:48.609 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:32:48.609 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:47.714311+0000", 2024-09-14T06:32:48.609 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.7615999999999999e-05 2024-09-14T06:32:48.609 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:48.609 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:32:48.609 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:48.609 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:48.609 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:48.609 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:238198 3.14 3:2d396db2:::rbd_data.112d80ca42af.0000000000000017:head [delete] snapc 1f=[] ondisk+write+known_if_redirected+supports_pool_eio e242)", 2024-09-14T06:32:48.609 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:32:47.719965+0000", 2024-09-14T06:32:48.610 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 0.79492792199999995, 2024-09-14T06:32:48.610 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0039876809999999999, 2024-09-14T06:32:48.610 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:32:48.610 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:32:48.610 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:32:48.610 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:32:48.610 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:32:48.610 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:48.610 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 238198 2024-09-14T06:32:48.610 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:48.610 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:32:48.610 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:48.610 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:32:48.610 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:47.719965+0000", 2024-09-14T06:32:48.611 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:48.611 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:48.611 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:48.611 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:32:48.611 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:47.719965+0000", 2024-09-14T06:32:48.611 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:48.611 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:48.611 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:48.611 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:32:48.611 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:47.719966+0000", 2024-09-14T06:32:48.611 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.319e-06 2024-09-14T06:32:48.611 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:48.611 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:48.611 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:32:48.611 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:47.719972+0000", 2024-09-14T06:32:48.612 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.4709999999999998e-06 2024-09-14T06:32:48.612 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:48.612 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:48.612 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:32:48.612 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:47.719982+0000", 2024-09-14T06:32:48.612 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.029e-05 2024-09-14T06:32:48.612 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:48.612 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:48.612 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:32:48.612 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:47.719988+0000", 2024-09-14T06:32:48.612 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.6050000000000003e-06 2024-09-14T06:32:48.612 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:48.612 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:48.612 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:32:48.613 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:47.720064+0000", 2024-09-14T06:32:48.613 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.5893e-05 2024-09-14T06:32:48.613 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:48.613 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:48.613 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:32:48.613 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:47.720199+0000", 2024-09-14T06:32:48.613 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00013506100000000001 2024-09-14T06:32:48.613 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:48.613 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:48.613 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "waiting for subops from 7", 2024-09-14T06:32:48.614 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:47.720325+0000", 2024-09-14T06:32:48.614 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000125456 2024-09-14T06:32:48.614 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:48.614 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:48.614 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "op_commit", 2024-09-14T06:32:48.614 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:47.722213+0000", 2024-09-14T06:32:48.614 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0018876260000000001 2024-09-14T06:32:48.614 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:48.614 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:48.614 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:32:48.614 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:47.723839+0000", 2024-09-14T06:32:48.614 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0016260210000000001 2024-09-14T06:32:48.615 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:48.615 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:48.615 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-14T06:32:48.615 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:47.723898+0000", 2024-09-14T06:32:48.615 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.9728000000000001e-05 2024-09-14T06:32:48.615 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:48.615 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:48.615 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:32:48.615 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:47.723952+0000", 2024-09-14T06:32:48.615 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.4211e-05 2024-09-14T06:32:48.615 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:48.615 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:32:48.615 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:48.615 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:48.615 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:32:48.615 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-14T06:32:48.617 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-14T06:32:48.617 INFO:tasks.ceph.osd.3.smithi094.stderr:2024-09-14T06:32:48.615+0000 7feeeafe8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33655) UID: 0 2024-09-14T06:32:48.619 INFO:teuthology.orchestra.run.smithi094.stderr:noscrub is set 2024-09-14T06:32:48.718 INFO:tasks.ceph.osd.7.smithi138.stderr:2024-09-14T06:32:48.717+0000 7f3c6a21d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33623) UID: 0 2024-09-14T06:32:48.722 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-14T06:32:48.722 INFO:teuthology.orchestra.run.smithi094.stdout: "ops": [], 2024-09-14T06:32:48.722 INFO:teuthology.orchestra.run.smithi094.stdout: "num_ops": 0 2024-09-14T06:32:48.723 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-14T06:32:48.731 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-14T06:32:48.819 INFO:tasks.ceph.osd.4.smithi138.stderr:2024-09-14T06:32:48.818+0000 7f6c653c8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33615) UID: 0 2024-09-14T06:32:48.835 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-14T06:32:48.835 INFO:teuthology.orchestra.run.smithi094.stdout: "ops": [], 2024-09-14T06:32:48.835 INFO:teuthology.orchestra.run.smithi094.stdout: "complaint_time": 30, 2024-09-14T06:32:48.835 INFO:teuthology.orchestra.run.smithi094.stdout: "num_blocked_ops": 0 2024-09-14T06:32:48.835 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-14T06:32:48.844 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-14T06:32:48.920 INFO:tasks.ceph.osd.7.smithi138.stderr:2024-09-14T06:32:48.919+0000 7f3c6a21d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33623) UID: 0 2024-09-14T06:32:48.949 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-14T06:32:48.949 INFO:teuthology.orchestra.run.smithi094.stdout: "size": 20, 2024-09-14T06:32:48.949 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 600, 2024-09-14T06:32:48.949 INFO:teuthology.orchestra.run.smithi094.stdout: "ops": [ 2024-09-14T06:32:48.950 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:48.950 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:44426 3.2 3:4603ad7f:::rbd_data.112dca2f6ae1.0000000000000005:17 [read 892928~4096 [fadvise_nocache]] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:32:48.950 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:43.854977+0000", 2024-09-14T06:32:48.950 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 185.09269890600001, 2024-09-14T06:32:48.950 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.063369237999999994, 2024-09-14T06:32:48.950 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:32:48.950 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:32:48.950 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:32:48.950 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:32:48.950 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:32:48.950 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:48.950 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 44426 2024-09-14T06:32:48.950 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:48.950 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:32:48.950 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:48.951 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:32:48.951 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.854977+0000", 2024-09-14T06:32:48.951 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:48.951 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:48.951 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:48.951 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:32:48.951 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.854977+0000", 2024-09-14T06:32:48.951 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:48.951 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:48.951 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:48.951 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:32:48.951 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.854977+0000", 2024-09-14T06:32:48.951 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.1199999999999999e-07 2024-09-14T06:32:48.951 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:48.951 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:48.952 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:32:48.952 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.854980+0000", 2024-09-14T06:32:48.952 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.27e-06 2024-09-14T06:32:48.952 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:48.952 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:48.952 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:32:48.952 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.854983+0000", 2024-09-14T06:32:48.952 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.1650000000000002e-06 2024-09-14T06:32:48.952 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:48.952 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:48.952 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:32:48.952 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.854989+0000", 2024-09-14T06:32:48.952 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.6520000000000003e-06 2024-09-14T06:32:48.952 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:48.952 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:48.953 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:32:48.953 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.918143+0000", 2024-09-14T06:32:48.953 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.063154541999999994 2024-09-14T06:32:48.953 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:48.953 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:48.953 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:32:48.953 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.918209+0000", 2024-09-14T06:32:48.953 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.5529000000000002e-05 2024-09-14T06:32:48.953 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:48.953 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:48.953 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:32:48.953 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.918346+0000", 2024-09-14T06:32:48.953 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000136868 2024-09-14T06:32:48.953 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:48.954 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:32:48.954 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:48.954 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:48.954 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:48.954 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:44427 3.2 3:4603ad7f:::rbd_data.112dca2f6ae1.0000000000000005:17 [read 917504~4096 [fadvise_nocache]] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:32:48.954 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:43.854992+0000", 2024-09-14T06:32:48.954 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 185.09268326, 2024-09-14T06:32:48.954 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.063607792999999996, 2024-09-14T06:32:48.954 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:32:48.954 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:32:48.954 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:32:48.954 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:32:48.954 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:32:48.954 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:48.955 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 44427 2024-09-14T06:32:48.955 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:48.955 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:32:48.955 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:48.955 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:32:48.955 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.854992+0000", 2024-09-14T06:32:48.955 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:48.955 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:48.955 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:48.955 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:32:48.955 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.854992+0000", 2024-09-14T06:32:48.955 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:48.955 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:48.955 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:48.956 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:32:48.956 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.854993+0000", 2024-09-14T06:32:48.956 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.1400000000000001e-07 2024-09-14T06:32:48.956 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:48.956 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:48.956 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:32:48.956 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.854994+0000", 2024-09-14T06:32:48.956 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.091e-06 2024-09-14T06:32:48.956 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:48.956 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:48.956 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:32:48.956 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855001+0000", 2024-09-14T06:32:48.956 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.204e-06 2024-09-14T06:32:48.956 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:48.957 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:48.957 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:32:48.957 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855003+0000", 2024-09-14T06:32:48.957 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.514e-06 2024-09-14T06:32:48.957 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:48.957 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:48.957 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:32:48.957 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.918380+0000", 2024-09-14T06:32:48.957 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.063376738000000002 2024-09-14T06:32:48.957 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:48.957 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:48.957 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:32:48.957 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.918453+0000", 2024-09-14T06:32:48.957 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.3291e-05 2024-09-14T06:32:48.958 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:48.958 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:48.958 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:32:48.958 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.918600+0000", 2024-09-14T06:32:48.958 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00014674100000000001 2024-09-14T06:32:48.958 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:48.958 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:32:48.958 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:48.958 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:48.958 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:48.958 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:44428 3.2 3:4603ad7f:::rbd_data.112dca2f6ae1.0000000000000005:17 [read 942080~4096 [fadvise_nocache]] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:32:48.958 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:43.855006+0000", 2024-09-14T06:32:48.958 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 185.09266921, 2024-09-14T06:32:48.958 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.063846321999999997, 2024-09-14T06:32:48.958 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:32:48.959 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:32:48.959 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:32:48.959 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:32:48.959 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:32:48.959 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:48.959 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 44428 2024-09-14T06:32:48.959 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:48.959 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:32:48.959 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:48.959 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:32:48.959 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855006+0000", 2024-09-14T06:32:48.959 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:48.959 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:48.959 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:48.959 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:32:48.960 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855006+0000", 2024-09-14T06:32:48.960 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:48.960 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:48.960 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:48.960 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:32:48.960 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855007+0000", 2024-09-14T06:32:48.960 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.1199999999999999e-07 2024-09-14T06:32:48.960 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:48.960 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:48.960 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:32:48.960 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855011+0000", 2024-09-14T06:32:48.960 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.9539999999999998e-06 2024-09-14T06:32:48.960 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:48.960 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:48.961 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:32:48.961 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855014+0000", 2024-09-14T06:32:48.961 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.2930000000000001e-06 2024-09-14T06:32:48.961 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:48.961 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:48.961 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:32:48.961 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855016+0000", 2024-09-14T06:32:48.961 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.3039999999999999e-06 2024-09-14T06:32:48.961 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:48.961 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:48.961 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:32:48.961 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.918634+0000", 2024-09-14T06:32:48.961 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.063617928000000004 2024-09-14T06:32:48.962 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:48.962 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:48.962 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:32:48.962 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.918711+0000", 2024-09-14T06:32:48.962 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.6545000000000007e-05 2024-09-14T06:32:48.962 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:48.962 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:48.962 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:32:48.962 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.918853+0000", 2024-09-14T06:32:48.962 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00014208600000000001 2024-09-14T06:32:48.962 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:48.962 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:32:48.962 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:48.962 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:48.963 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:48.963 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:44429 3.2 3:4603ad7f:::rbd_data.112dca2f6ae1.0000000000000005:17 [read 966656~4096 [fadvise_nocache]] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:32:48.963 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:43.855019+0000", 2024-09-14T06:32:48.963 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 185.09265653700001, 2024-09-14T06:32:48.963 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.064059226999999996, 2024-09-14T06:32:48.963 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:32:48.963 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:32:48.963 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:32:48.963 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:32:48.963 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:32:48.963 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:48.963 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 44429 2024-09-14T06:32:48.963 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:48.963 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:32:48.964 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:48.964 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:32:48.964 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855019+0000", 2024-09-14T06:32:48.964 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:48.964 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:48.964 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:48.964 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:32:48.964 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855019+0000", 2024-09-14T06:32:48.964 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:48.964 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:48.965 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:48.965 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:32:48.965 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855019+0000", 2024-09-14T06:32:48.965 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.28e-07 2024-09-14T06:32:48.965 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:48.965 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:48.965 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:32:48.965 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855020+0000", 2024-09-14T06:32:48.965 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.0330000000000001e-06 2024-09-14T06:32:48.965 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:48.965 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:48.965 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:32:48.965 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855023+0000", 2024-09-14T06:32:48.965 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.79e-06 2024-09-14T06:32:48.966 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:48.966 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:48.966 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:32:48.966 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855025+0000", 2024-09-14T06:32:48.966 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.216e-06 2024-09-14T06:32:48.966 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:48.966 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:48.966 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:32:48.966 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.918886+0000", 2024-09-14T06:32:48.966 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.063861037999999995 2024-09-14T06:32:48.966 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:48.966 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:48.966 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:32:48.966 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.918952+0000", 2024-09-14T06:32:48.966 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.5993000000000003e-05 2024-09-14T06:32:48.966 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:48.967 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:48.967 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:32:48.967 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.919078+0000", 2024-09-14T06:32:48.967 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00012592899999999999 2024-09-14T06:32:48.967 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:48.967 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:32:48.967 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:48.967 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:48.967 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:48.967 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:44430 3.2 3:4603ad7f:::rbd_data.112dca2f6ae1.0000000000000005:17 [read 991232~4096 [fadvise_nocache]] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:32:48.967 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:43.855028+0000", 2024-09-14T06:32:48.967 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 185.092647527, 2024-09-14T06:32:48.967 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.064284346000000006, 2024-09-14T06:32:48.967 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:32:48.967 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:32:48.967 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:32:48.968 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:32:48.968 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:32:48.968 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:48.968 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 44430 2024-09-14T06:32:48.968 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:48.968 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:32:48.968 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:48.968 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:32:48.968 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855028+0000", 2024-09-14T06:32:48.968 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:48.968 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:48.968 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:48.968 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:32:48.968 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855028+0000", 2024-09-14T06:32:48.969 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:48.969 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:48.969 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:48.969 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:32:48.969 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855028+0000", 2024-09-14T06:32:48.969 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.1199999999999999e-07 2024-09-14T06:32:48.969 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:48.969 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:48.969 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:32:48.969 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855029+0000", 2024-09-14T06:32:48.969 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.1100000000000004e-07 2024-09-14T06:32:48.969 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:48.969 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:48.969 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:32:48.970 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855032+0000", 2024-09-14T06:32:48.970 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.7889999999999999e-06 2024-09-14T06:32:48.970 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:48.970 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:48.970 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:32:48.970 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855034+0000", 2024-09-14T06:32:48.970 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.125e-06 2024-09-14T06:32:48.970 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:48.970 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:48.970 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:32:48.970 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.919110+0000", 2024-09-14T06:32:48.970 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.064076413999999998 2024-09-14T06:32:48.970 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:48.970 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:48.970 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:32:48.970 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.919186+0000", 2024-09-14T06:32:48.971 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.5784999999999994e-05 2024-09-14T06:32:48.971 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:48.971 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:48.971 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:32:48.971 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.919312+0000", 2024-09-14T06:32:48.971 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00012611 2024-09-14T06:32:48.971 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:48.971 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:32:48.971 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:48.971 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:48.971 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:48.971 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:44431 3.2 3:4603ad7f:::rbd_data.112dca2f6ae1.0000000000000005:17 [read 1015808~4096 [fadvise_nocache]] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:32:48.971 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:43.855037+0000", 2024-09-14T06:32:48.971 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 185.09263865899999, 2024-09-14T06:32:48.971 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.064469755000000004, 2024-09-14T06:32:48.971 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:32:48.972 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:32:48.972 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:32:48.972 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:32:48.972 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:32:48.972 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:48.972 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 44431 2024-09-14T06:32:48.972 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:48.972 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:32:48.972 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:48.972 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:32:48.972 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855037+0000", 2024-09-14T06:32:48.972 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:48.972 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:48.972 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:48.972 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:32:48.973 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855037+0000", 2024-09-14T06:32:48.973 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:48.973 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:48.973 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:48.973 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:32:48.973 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855037+0000", 2024-09-14T06:32:48.973 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.1e-07 2024-09-14T06:32:48.973 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:48.973 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:48.973 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:32:48.973 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855038+0000", 2024-09-14T06:32:48.973 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.9400000000000004e-07 2024-09-14T06:32:48.973 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:48.973 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:48.974 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:32:48.974 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855041+0000", 2024-09-14T06:32:48.974 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.9639999999999999e-06 2024-09-14T06:32:48.974 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:48.974 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:48.974 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:32:48.974 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855043+0000", 2024-09-14T06:32:48.974 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.2259999999999999e-06 2024-09-14T06:32:48.974 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:48.974 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:48.974 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:32:48.974 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.919347+0000", 2024-09-14T06:32:48.974 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.064304091999999993 2024-09-14T06:32:48.974 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:48.974 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:48.974 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:32:48.975 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.919393+0000", 2024-09-14T06:32:48.975 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.5340999999999998e-05 2024-09-14T06:32:48.975 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:48.975 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:48.975 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:32:48.975 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.919507+0000", 2024-09-14T06:32:48.975 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000114028 2024-09-14T06:32:48.975 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:48.975 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:32:48.975 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:48.975 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:48.975 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:48.975 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:44432 3.2 3:4603ad7f:::rbd_data.112dca2f6ae1.0000000000000005:17 [read 1040384~4096 [fadvise_nocache]] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:32:48.975 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:43.855046+0000", 2024-09-14T06:32:48.975 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 185.09262959099999, 2024-09-14T06:32:48.975 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.064639253999999993, 2024-09-14T06:32:48.976 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:32:48.976 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:32:48.976 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:32:48.976 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:32:48.976 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:32:48.976 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:48.976 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 44432 2024-09-14T06:32:48.976 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:48.976 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:32:48.976 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:48.976 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:32:48.976 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855046+0000", 2024-09-14T06:32:48.976 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:48.976 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:48.977 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:48.977 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:32:48.977 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855046+0000", 2024-09-14T06:32:48.977 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:48.977 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:48.977 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:48.977 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:32:48.977 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855046+0000", 2024-09-14T06:32:48.977 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.05e-07 2024-09-14T06:32:48.977 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:48.977 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:48.977 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:32:48.977 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855047+0000", 2024-09-14T06:32:48.977 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.879999999999999e-07 2024-09-14T06:32:48.978 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:48.978 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:48.978 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:32:48.978 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855050+0000", 2024-09-14T06:32:48.978 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.8459999999999999e-06 2024-09-14T06:32:48.978 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:48.978 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:48.978 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:32:48.978 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855052+0000", 2024-09-14T06:32:48.978 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.136e-06 2024-09-14T06:32:48.978 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:48.978 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:48.978 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:32:48.978 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.919536+0000", 2024-09-14T06:32:48.978 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.064484132 2024-09-14T06:32:48.978 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:48.979 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:48.979 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:32:48.979 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.919611+0000", 2024-09-14T06:32:48.979 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.4407999999999996e-05 2024-09-14T06:32:48.979 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:48.979 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:48.979 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:32:48.979 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.919685+0000", 2024-09-14T06:32:48.979 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.4538999999999999e-05 2024-09-14T06:32:48.979 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:48.979 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:32:48.979 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:48.979 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:48.979 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:48.980 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:44518 3.2 3:4190d882:::rbd_data.112dca2f6ae1.0000000000000002:17 [read 12288~4096 [fadvise_nocache]] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:32:48.980 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:43.855055+0000", 2024-09-14T06:32:48.980 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 185.09262070700001, 2024-09-14T06:32:48.980 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.064894317000000007, 2024-09-14T06:32:48.980 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:32:48.980 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:32:48.980 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:32:48.980 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:32:48.980 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:32:48.980 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:48.980 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 44518 2024-09-14T06:32:48.980 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:48.980 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:32:48.980 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:48.981 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:32:48.981 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855055+0000", 2024-09-14T06:32:48.981 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:48.981 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:48.981 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:48.981 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:32:48.981 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855055+0000", 2024-09-14T06:32:48.981 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:48.981 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:48.981 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:48.981 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:32:48.981 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855055+0000", 2024-09-14T06:32:48.981 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.0699999999999999e-07 2024-09-14T06:32:48.981 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:48.982 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:48.982 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:32:48.982 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855056+0000", 2024-09-14T06:32:48.982 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.9400000000000004e-07 2024-09-14T06:32:48.982 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:48.982 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:48.982 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:32:48.982 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855059+0000", 2024-09-14T06:32:48.982 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.8870000000000001e-06 2024-09-14T06:32:48.982 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:48.982 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:48.982 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:32:48.982 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855061+0000", 2024-09-14T06:32:48.982 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.1919999999999999e-06 2024-09-14T06:32:48.982 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:48.983 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:48.983 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:32:48.983 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.919719+0000", 2024-09-14T06:32:48.983 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.064658013 2024-09-14T06:32:48.983 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:48.983 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:48.983 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:32:48.983 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.919788+0000", 2024-09-14T06:32:48.983 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.9282000000000005e-05 2024-09-14T06:32:48.983 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:48.983 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:48.983 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:32:48.983 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.919949+0000", 2024-09-14T06:32:48.983 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00016084199999999999 2024-09-14T06:32:48.984 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:48.984 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:32:48.984 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:48.984 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:48.984 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:48.984 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:44519 3.2 3:4190d882:::rbd_data.112dca2f6ae1.0000000000000002:17 [read 36864~4096 [fadvise_nocache]] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:32:48.984 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:43.855064+0000", 2024-09-14T06:32:48.984 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 185.09261172999999, 2024-09-14T06:32:48.984 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.065081865000000003, 2024-09-14T06:32:48.984 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:32:48.984 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:32:48.984 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:32:48.984 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:32:48.984 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:32:48.984 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:48.985 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 44519 2024-09-14T06:32:48.985 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:48.985 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:32:48.985 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:48.985 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:32:48.985 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855064+0000", 2024-09-14T06:32:48.985 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:48.985 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:48.985 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:48.985 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:32:48.985 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855064+0000", 2024-09-14T06:32:48.985 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:48.985 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:48.985 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:48.986 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:32:48.986 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855064+0000", 2024-09-14T06:32:48.986 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.1e-07 2024-09-14T06:32:48.986 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:48.986 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:48.986 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:32:48.986 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855065+0000", 2024-09-14T06:32:48.986 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.035e-06 2024-09-14T06:32:48.986 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:48.986 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:48.986 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:32:48.986 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855068+0000", 2024-09-14T06:32:48.986 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.8339999999999999e-06 2024-09-14T06:32:48.986 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:48.986 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:48.987 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:32:48.987 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855071+0000", 2024-09-14T06:32:48.987 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.3450000000000002e-06 2024-09-14T06:32:48.987 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:48.987 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:48.987 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:32:48.987 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.919979+0000", 2024-09-14T06:32:48.987 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.064907647999999998 2024-09-14T06:32:48.987 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:48.987 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:48.987 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:32:48.987 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.920026+0000", 2024-09-14T06:32:48.987 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.7595000000000001e-05 2024-09-14T06:32:48.987 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:48.988 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:48.988 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:32:48.988 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.920146+0000", 2024-09-14T06:32:48.988 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00011919800000000001 2024-09-14T06:32:48.988 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:48.988 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:32:48.988 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:48.988 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:48.988 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:48.988 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:44520 3.2 3:4190d882:::rbd_data.112dca2f6ae1.0000000000000002:17 [read 61440~4096 [fadvise_nocache]] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:32:48.988 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:43.855075+0000", 2024-09-14T06:32:48.988 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 185.09260067899999, 2024-09-14T06:32:48.988 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.065335078000000005, 2024-09-14T06:32:48.989 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:32:48.989 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:32:48.989 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:32:48.989 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:32:48.989 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:32:48.989 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:48.989 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 44520 2024-09-14T06:32:48.989 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:48.989 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:32:48.989 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:48.989 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:32:48.989 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855075+0000", 2024-09-14T06:32:48.989 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:48.990 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:48.990 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:48.990 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:32:48.990 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855075+0000", 2024-09-14T06:32:48.990 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:48.990 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:48.990 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:48.990 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:32:48.990 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855075+0000", 2024-09-14T06:32:48.990 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.1e-07 2024-09-14T06:32:48.990 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:48.990 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:48.990 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:32:48.990 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855076+0000", 2024-09-14T06:32:48.991 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.0839999999999999e-06 2024-09-14T06:32:48.991 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:48.991 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:48.991 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:32:48.991 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855079+0000", 2024-09-14T06:32:48.991 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.9359999999999999e-06 2024-09-14T06:32:48.991 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:48.991 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:48.991 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:32:48.991 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855084+0000", 2024-09-14T06:32:48.991 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.1730000000000003e-06 2024-09-14T06:32:48.991 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:48.991 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:48.992 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:32:48.992 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.920175+0000", 2024-09-14T06:32:48.992 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.065090234999999996 2024-09-14T06:32:48.992 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:48.992 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:48.992 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:32:48.992 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.920250+0000", 2024-09-14T06:32:48.992 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.5807000000000001e-05 2024-09-14T06:32:48.992 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:48.992 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:48.992 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:32:48.992 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.920410+0000", 2024-09-14T06:32:48.992 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00015963300000000001 2024-09-14T06:32:48.993 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:48.993 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:32:48.993 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:48.993 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:48.993 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:48.993 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:44521 3.2 3:4190d882:::rbd_data.112dca2f6ae1.0000000000000002:17 [read 86016~4096 [fadvise_nocache]] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:32:48.993 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:43.855087+0000", 2024-09-14T06:32:48.993 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 185.09258819799999, 2024-09-14T06:32:48.993 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.065563264999999996, 2024-09-14T06:32:48.993 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:32:48.993 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:32:48.993 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:32:48.994 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:32:48.994 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:32:48.994 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:48.994 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 44521 2024-09-14T06:32:48.994 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:48.994 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:32:48.994 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:48.994 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:32:48.994 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855087+0000", 2024-09-14T06:32:48.994 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:48.994 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:48.994 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:48.994 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:32:48.994 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855087+0000", 2024-09-14T06:32:48.994 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:48.995 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:48.995 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:48.995 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:32:48.995 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855088+0000", 2024-09-14T06:32:48.995 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.3699999999999999e-07 2024-09-14T06:32:48.995 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:48.995 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:48.995 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:32:48.995 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855089+0000", 2024-09-14T06:32:48.995 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.9900000000000009e-07 2024-09-14T06:32:48.995 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:48.995 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:48.995 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:32:48.995 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855091+0000", 2024-09-14T06:32:48.995 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.8940000000000002e-06 2024-09-14T06:32:48.996 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:48.996 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:48.996 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:32:48.996 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855094+0000", 2024-09-14T06:32:48.996 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.351e-06 2024-09-14T06:32:48.996 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:48.996 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:48.996 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:32:48.996 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.920437+0000", 2024-09-14T06:32:48.996 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.065343221000000007 2024-09-14T06:32:48.996 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:48.996 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:48.996 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:32:48.996 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.920481+0000", 2024-09-14T06:32:48.996 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.4431000000000002e-05 2024-09-14T06:32:48.997 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:48.997 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:48.997 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:32:48.997 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.920651+0000", 2024-09-14T06:32:48.997 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00016913200000000001 2024-09-14T06:32:48.997 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:48.997 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:32:48.997 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:48.997 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:48.997 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:48.997 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:44522 3.2 3:4190d882:::rbd_data.112dca2f6ae1.0000000000000002:17 [read 110592~4096 [fadvise_nocache]] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:32:48.997 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:43.855098+0000", 2024-09-14T06:32:48.997 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 185.092577971, 2024-09-14T06:32:48.997 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.065827628999999999, 2024-09-14T06:32:48.998 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:32:48.998 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:32:48.998 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:32:48.998 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:32:48.998 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:32:48.998 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:48.998 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 44522 2024-09-14T06:32:48.998 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:48.998 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:32:48.998 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:48.998 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:32:48.998 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855098+0000", 2024-09-14T06:32:48.998 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:48.998 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:48.998 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:48.999 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:32:48.999 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855098+0000", 2024-09-14T06:32:48.999 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:48.999 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:48.999 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:48.999 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:32:48.999 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855098+0000", 2024-09-14T06:32:48.999 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.1e-07 2024-09-14T06:32:48.999 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:48.999 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:48.999 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:32:48.999 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855099+0000", 2024-09-14T06:32:48.999 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.012e-06 2024-09-14T06:32:48.999 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:48.999 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:49.000 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:32:49.000 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855102+0000", 2024-09-14T06:32:49.000 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.9380000000000001e-06 2024-09-14T06:32:49.000 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:49.000 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:49.000 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:32:49.000 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855108+0000", 2024-09-14T06:32:49.000 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.4339999999999997e-06 2024-09-14T06:32:49.000 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:49.000 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:49.000 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:32:49.000 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.920681+0000", 2024-09-14T06:32:49.000 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.065572943999999994 2024-09-14T06:32:49.000 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:49.000 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:49.001 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:32:49.001 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.920724+0000", 2024-09-14T06:32:49.001 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.2877000000000003e-05 2024-09-14T06:32:49.001 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:49.001 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:49.001 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:32:49.001 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.920925+0000", 2024-09-14T06:32:49.001 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00020121399999999999 2024-09-14T06:32:49.001 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:49.001 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:32:49.001 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:49.001 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:49.001 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:49.001 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:44523 3.2 3:4190d882:::rbd_data.112dca2f6ae1.0000000000000002:17 [read 135168~4096 [fadvise_nocache]] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:32:49.002 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:43.855111+0000", 2024-09-14T06:32:49.002 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 185.09256424899999, 2024-09-14T06:32:49.002 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.066028198999999996, 2024-09-14T06:32:49.002 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:32:49.002 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:32:49.002 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:32:49.002 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:32:49.002 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:32:49.002 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:49.002 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 44523 2024-09-14T06:32:49.002 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:49.002 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:32:49.002 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:49.002 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:32:49.002 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855111+0000", 2024-09-14T06:32:49.003 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:49.003 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:49.003 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:49.003 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:32:49.003 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855111+0000", 2024-09-14T06:32:49.003 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:49.003 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:49.003 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:49.003 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:32:49.003 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855112+0000", 2024-09-14T06:32:49.003 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.1199999999999999e-07 2024-09-14T06:32:49.003 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:49.003 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:49.003 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:32:49.004 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855115+0000", 2024-09-14T06:32:49.004 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.3380000000000001e-06 2024-09-14T06:32:49.004 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:49.004 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:49.004 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:32:49.004 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855118+0000", 2024-09-14T06:32:49.004 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.366e-06 2024-09-14T06:32:49.004 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:49.004 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:49.004 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:32:49.004 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855121+0000", 2024-09-14T06:32:49.004 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.8329999999999998e-06 2024-09-14T06:32:49.004 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:49.004 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:49.004 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:32:49.005 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.920956+0000", 2024-09-14T06:32:49.005 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.065835251999999997 2024-09-14T06:32:49.005 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:49.005 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:49.005 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:32:49.005 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.921001+0000", 2024-09-14T06:32:49.005 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.4778999999999999e-05 2024-09-14T06:32:49.005 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:49.005 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:49.005 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:32:49.005 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.921139+0000", 2024-09-14T06:32:49.005 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000138419 2024-09-14T06:32:49.005 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:49.005 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:32:49.005 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:49.006 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:49.006 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:49.006 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:44524 3.2 3:4190d882:::rbd_data.112dca2f6ae1.0000000000000002:17 [read 159744~4096 [fadvise_nocache]] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:32:49.006 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:43.855124+0000", 2024-09-14T06:32:49.006 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 185.09255127, 2024-09-14T06:32:49.006 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.066249261000000004, 2024-09-14T06:32:49.006 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:32:49.006 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:32:49.006 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:32:49.006 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:32:49.006 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:32:49.006 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:49.006 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 44524 2024-09-14T06:32:49.006 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:49.006 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:32:49.006 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:49.007 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:32:49.007 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855124+0000", 2024-09-14T06:32:49.007 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:49.007 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:49.007 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:49.007 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:32:49.007 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855124+0000", 2024-09-14T06:32:49.007 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:49.007 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:49.007 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:49.007 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:32:49.007 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861086+0000", 2024-09-14T06:32:49.007 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0059613829999999998 2024-09-14T06:32:49.007 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:49.007 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:49.008 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:32:49.008 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861100+0000", 2024-09-14T06:32:49.008 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.4644000000000001e-05 2024-09-14T06:32:49.008 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:49.008 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:49.008 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:32:49.008 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861109+0000", 2024-09-14T06:32:49.008 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.541e-06 2024-09-14T06:32:49.008 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:49.008 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:49.008 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:32:49.008 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861117+0000", 2024-09-14T06:32:49.009 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.1049999999999995e-06 2024-09-14T06:32:49.009 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:49.009 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:49.009 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:32:49.009 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.921169+0000", 2024-09-14T06:32:49.009 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.060052405000000003 2024-09-14T06:32:49.009 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:49.009 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:49.009 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:32:49.009 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.921213+0000", 2024-09-14T06:32:49.009 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.3565e-05 2024-09-14T06:32:49.009 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:49.009 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:49.009 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:32:49.010 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.921374+0000", 2024-09-14T06:32:49.010 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00016061800000000001 2024-09-14T06:32:49.010 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:49.010 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:32:49.010 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:49.010 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:49.010 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:49.010 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:44537 3.2 3:4190d882:::rbd_data.112dca2f6ae1.0000000000000002:17 [read 479232~4096 [fadvise_nocache]] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:32:49.010 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:43.861405+0000", 2024-09-14T06:32:49.010 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 185.08627066700001, 2024-09-14T06:32:49.010 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.063351795000000002, 2024-09-14T06:32:49.010 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:32:49.010 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:32:49.011 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:32:49.011 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:32:49.011 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:32:49.011 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:49.011 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 44537 2024-09-14T06:32:49.011 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:49.011 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:32:49.011 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:49.011 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:32:49.011 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861405+0000", 2024-09-14T06:32:49.011 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:49.011 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:49.011 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:49.011 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:32:49.012 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861405+0000", 2024-09-14T06:32:49.012 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:49.012 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:49.012 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:49.012 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:32:49.012 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861405+0000", 2024-09-14T06:32:49.012 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.9099999999999999e-07 2024-09-14T06:32:49.012 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:49.012 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:49.012 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:32:49.012 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861407+0000", 2024-09-14T06:32:49.012 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.075e-06 2024-09-14T06:32:49.012 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:49.012 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:49.012 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:32:49.013 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861414+0000", 2024-09-14T06:32:49.013 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.4659999999999996e-06 2024-09-14T06:32:49.013 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:49.013 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:49.013 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:32:49.013 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861418+0000", 2024-09-14T06:32:49.013 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.5519999999999998e-06 2024-09-14T06:32:49.013 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:49.013 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:49.013 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:32:49.013 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.924474+0000", 2024-09-14T06:32:49.013 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.063056012999999994 2024-09-14T06:32:49.013 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:49.013 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:49.014 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:32:49.014 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.924556+0000", 2024-09-14T06:32:49.014 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.1181999999999996e-05 2024-09-14T06:32:49.014 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:49.014 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:49.014 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:32:49.014 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.924757+0000", 2024-09-14T06:32:49.015 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00020111599999999999 2024-09-14T06:32:49.015 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:49.015 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:32:49.015 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:49.015 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:49.015 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:49.015 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:44538 3.2 3:4190d882:::rbd_data.112dca2f6ae1.0000000000000002:17 [read 503808~4096 [fadvise_nocache]] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:32:49.015 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:43.861425+0000", 2024-09-14T06:32:49.015 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 185.08625062600001, 2024-09-14T06:32:49.015 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.063627125000000007, 2024-09-14T06:32:49.015 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:32:49.015 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:32:49.015 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:32:49.015 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:32:49.016 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:32:49.016 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:49.016 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 44538 2024-09-14T06:32:49.016 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:49.016 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:32:49.016 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:49.016 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:32:49.016 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861425+0000", 2024-09-14T06:32:49.016 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:49.016 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:49.016 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:49.016 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:32:49.016 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861425+0000", 2024-09-14T06:32:49.016 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:49.017 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:49.017 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:49.017 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:32:49.017 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861425+0000", 2024-09-14T06:32:49.017 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.9999999999999998e-07 2024-09-14T06:32:49.017 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:49.017 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:49.017 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:32:49.017 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861427+0000", 2024-09-14T06:32:49.017 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.9829999999999999e-06 2024-09-14T06:32:49.017 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:49.017 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:49.017 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:32:49.017 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861433+0000", 2024-09-14T06:32:49.018 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.2889999999999997e-06 2024-09-14T06:32:49.018 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:49.018 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:49.018 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:32:49.018 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861454+0000", 2024-09-14T06:32:49.018 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.1844999999999998e-05 2024-09-14T06:32:49.018 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:49.018 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:49.018 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:32:49.018 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.924786+0000", 2024-09-14T06:32:49.018 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.063331497 2024-09-14T06:32:49.018 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:49.018 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:49.019 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:32:49.019 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.924864+0000", 2024-09-14T06:32:49.019 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.7744000000000006e-05 2024-09-14T06:32:49.019 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:49.019 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:49.019 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:32:49.019 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.925052+0000", 2024-09-14T06:32:49.019 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00018836699999999999 2024-09-14T06:32:49.019 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:49.019 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:32:49.019 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:49.019 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:49.019 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:49.019 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:44539 3.2 3:4190d882:::rbd_data.112dca2f6ae1.0000000000000002:17 [read 528384~4096 [fadvise_nocache]] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:32:49.020 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:43.861462+0000", 2024-09-14T06:32:49.020 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 185.08621329900001, 2024-09-14T06:32:49.020 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.063855705999999998, 2024-09-14T06:32:49.020 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:32:49.020 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:32:49.020 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:32:49.020 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:32:49.020 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:32:49.020 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:49.020 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 44539 2024-09-14T06:32:49.020 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:49.020 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:32:49.020 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:49.020 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:32:49.020 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861462+0000", 2024-09-14T06:32:49.021 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:49.021 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:49.021 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:49.021 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:32:49.021 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861462+0000", 2024-09-14T06:32:49.021 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:49.021 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:49.021 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:49.021 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:32:49.021 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861463+0000", 2024-09-14T06:32:49.021 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.5200000000000002e-07 2024-09-14T06:32:49.021 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:49.021 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:49.021 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:32:49.022 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861465+0000", 2024-09-14T06:32:49.022 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.5189999999999999e-06 2024-09-14T06:32:49.022 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:49.022 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:49.022 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:32:49.022 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861471+0000", 2024-09-14T06:32:49.022 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.6899999999999997e-06 2024-09-14T06:32:49.022 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:49.022 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:49.022 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:32:49.022 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861475+0000", 2024-09-14T06:32:49.022 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.5730000000000001e-06 2024-09-14T06:32:49.022 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:49.022 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:49.023 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:32:49.023 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.925085+0000", 2024-09-14T06:32:49.023 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.063609902999999995 2024-09-14T06:32:49.023 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:49.023 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:49.023 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:32:49.023 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.925145+0000", 2024-09-14T06:32:49.023 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.9383999999999999e-05 2024-09-14T06:32:49.023 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:49.023 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:49.023 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:32:49.023 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.925318+0000", 2024-09-14T06:32:49.023 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00017318499999999999 2024-09-14T06:32:49.023 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:49.023 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:32:49.023 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:49.024 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:49.024 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:49.024 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:44540 3.2 3:4190d882:::rbd_data.112dca2f6ae1.0000000000000002:17 [read 552960~4096 [fadvise_nocache]] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:32:49.024 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:43.861483+0000", 2024-09-14T06:32:49.024 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 185.08619286800001, 2024-09-14T06:32:49.024 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.064065681999999999, 2024-09-14T06:32:49.024 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:32:49.024 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:32:49.024 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:32:49.024 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:32:49.024 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:32:49.024 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:49.024 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 44540 2024-09-14T06:32:49.025 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:49.025 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:32:49.025 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:49.025 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:32:49.025 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861483+0000", 2024-09-14T06:32:49.025 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:49.025 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:49.025 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:49.025 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:32:49.025 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861483+0000", 2024-09-14T06:32:49.025 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:49.025 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:49.025 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:49.025 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:32:49.025 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.867079+0000", 2024-09-14T06:32:49.026 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0055966480000000001 2024-09-14T06:32:49.026 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:49.026 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:49.026 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:32:49.026 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.867086+0000", 2024-09-14T06:32:49.026 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.387e-06 2024-09-14T06:32:49.026 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:49.026 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:49.026 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:32:49.026 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.867094+0000", 2024-09-14T06:32:49.026 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.9419999999999995e-06 2024-09-14T06:32:49.026 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:49.026 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:49.026 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:32:49.026 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.867109+0000", 2024-09-14T06:32:49.026 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.5818999999999999e-05 2024-09-14T06:32:49.027 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:49.027 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:49.027 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:32:49.027 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.925354+0000", 2024-09-14T06:32:49.027 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.058244047 2024-09-14T06:32:49.027 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:49.027 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:49.027 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:32:49.027 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.925406+0000", 2024-09-14T06:32:49.027 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.2661999999999998e-05 2024-09-14T06:32:49.027 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:49.027 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:49.027 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:32:49.027 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.925548+0000", 2024-09-14T06:32:49.027 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000142177 2024-09-14T06:32:49.027 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:49.028 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:32:49.028 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:49.028 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:49.028 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:49.028 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:225587 3.7 3:e112e2b3:::rbd_id.image_client.0-clone42:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e189)", 2024-09-14T06:32:49.028 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:31:53.874591+0000", 2024-09-14T06:32:49.028 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 55.073084651000002, 2024-09-14T06:32:49.028 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.875541507, 2024-09-14T06:32:49.028 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:32:49.028 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:32:49.028 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:32:49.028 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:32:49.028 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:32:49.028 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:49.029 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 225587 2024-09-14T06:32:49.029 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:49.029 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:32:49.029 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:49.029 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:32:49.029 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:53.874591+0000", 2024-09-14T06:32:49.029 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:49.029 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:49.029 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:49.029 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:32:49.029 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:53.874591+0000", 2024-09-14T06:32:49.029 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:49.029 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:49.030 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:49.030 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:32:49.030 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:53.874592+0000", 2024-09-14T06:32:49.030 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.5209999999999999e-06 2024-09-14T06:32:49.030 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:49.030 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:49.030 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:32:49.030 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:53.874599+0000", 2024-09-14T06:32:49.030 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.6290000000000004e-06 2024-09-14T06:32:49.030 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:49.030 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:49.030 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:32:49.030 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:53.874618+0000", 2024-09-14T06:32:49.030 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.8950999999999999e-05 2024-09-14T06:32:49.030 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:49.031 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:49.031 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:32:49.031 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:53.874626+0000", 2024-09-14T06:32:49.031 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.3960000000000006e-06 2024-09-14T06:32:49.031 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:49.031 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:49.031 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:32:49.031 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:53.874699+0000", 2024-09-14T06:32:49.031 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.2329999999999994e-05 2024-09-14T06:32:49.031 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:49.031 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:49.031 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "waiting for peered", 2024-09-14T06:32:49.031 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:53.874711+0000", 2024-09-14T06:32:49.031 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.2473999999999999e-05 2024-09-14T06:32:49.032 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:49.032 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:49.032 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:32:49.032 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:54.749578+0000", 2024-09-14T06:32:49.032 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.87486642100000001 2024-09-14T06:32:49.032 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:49.032 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:49.032 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:32:49.032 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:54.749706+0000", 2024-09-14T06:32:49.032 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00012856200000000001 2024-09-14T06:32:49.032 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:49.032 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:49.032 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:32:49.032 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:54.750132+0000", 2024-09-14T06:32:49.033 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00042622299999999999 2024-09-14T06:32:49.033 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:49.033 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:32:49.033 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:49.033 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:49.033 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:49.033 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:233119 3.1e 3:7bae5948:::rbd_data.112df8d32353.000000000000000f:head [delete] snapc 53=[] ondisk+write+known_if_redirected+supports_pool_eio e211)", 2024-09-14T06:32:49.033 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:32:16.760588+0000", 2024-09-14T06:32:49.033 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 32.187087664000003, 2024-09-14T06:32:49.033 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.92349815700000004, 2024-09-14T06:32:49.033 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:32:49.033 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:32:49.033 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:32:49.033 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:32:49.034 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:32:49.034 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:49.034 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 233119 2024-09-14T06:32:49.034 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:49.034 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:32:49.034 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:49.034 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:32:49.034 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:16.760588+0000", 2024-09-14T06:32:49.034 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:49.034 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:49.034 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:49.034 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:32:49.034 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:16.760588+0000", 2024-09-14T06:32:49.034 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:49.034 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:49.034 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:49.035 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:32:49.035 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:16.760588+0000", 2024-09-14T06:32:49.035 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.3900000000000005e-07 2024-09-14T06:32:49.035 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:49.035 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:49.035 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:32:49.035 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:16.760591+0000", 2024-09-14T06:32:49.035 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.0070000000000002e-06 2024-09-14T06:32:49.035 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:49.035 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:49.035 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:32:49.035 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:16.760598+0000", 2024-09-14T06:32:49.035 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.0509999999999999e-06 2024-09-14T06:32:49.035 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:49.035 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:49.035 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:32:49.036 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:16.760605+0000", 2024-09-14T06:32:49.036 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.0310000000000004e-06 2024-09-14T06:32:49.036 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:49.036 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:49.036 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:32:49.036 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:16.760668+0000", 2024-09-14T06:32:49.036 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.3699000000000001e-05 2024-09-14T06:32:49.036 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:49.036 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:49.036 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "waiting for peered", 2024-09-14T06:32:49.036 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:16.760678+0000", 2024-09-14T06:32:49.036 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.0122999999999999e-05 2024-09-14T06:32:49.036 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:49.036 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:49.036 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:32:49.037 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:17.681459+0000", 2024-09-14T06:32:49.037 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.92078067399999997 2024-09-14T06:32:49.037 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:49.037 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:49.037 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:32:49.037 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:17.681700+0000", 2024-09-14T06:32:49.037 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00024128800000000001 2024-09-14T06:32:49.037 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:49.037 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:49.037 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "waiting for subops from 2", 2024-09-14T06:32:49.037 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:17.681860+0000", 2024-09-14T06:32:49.037 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00015995500000000001 2024-09-14T06:32:49.037 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:49.037 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:49.038 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:32:49.038 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:17.683399+0000", 2024-09-14T06:32:49.038 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0015388209999999999 2024-09-14T06:32:49.038 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:49.038 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:49.038 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "op_commit", 2024-09-14T06:32:49.038 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:17.683687+0000", 2024-09-14T06:32:49.038 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00028772299999999998 2024-09-14T06:32:49.038 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:49.038 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:49.038 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-14T06:32:49.038 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:17.684032+0000", 2024-09-14T06:32:49.038 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000344836 2024-09-14T06:32:49.038 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:49.038 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:49.039 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:32:49.039 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:17.684086+0000", 2024-09-14T06:32:49.039 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.4410000000000003e-05 2024-09-14T06:32:49.039 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:49.039 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:32:49.039 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:49.039 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:49.039 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:32:49.039 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-14T06:32:49.040 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-14T06:32:49.041 INFO:tasks.ceph.osd.7.smithi138.stderr:2024-09-14T06:32:49.040+0000 7f3c6a21d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33623) UID: 0 2024-09-14T06:32:49.141 INFO:tasks.ceph.osd.3.smithi094.stderr:2024-09-14T06:32:49.140+0000 7feeeafe8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33655) UID: 0 2024-09-14T06:32:49.155 INFO:teuthology.orchestra.run.smithi138.stdout:{ 2024-09-14T06:32:49.155 INFO:teuthology.orchestra.run.smithi138.stdout: "ops": [], 2024-09-14T06:32:49.155 INFO:teuthology.orchestra.run.smithi138.stdout: "num_ops": 0 2024-09-14T06:32:49.155 INFO:teuthology.orchestra.run.smithi138.stdout:} 2024-09-14T06:32:49.164 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-14T06:32:49.242 INFO:tasks.ceph.osd.6.smithi138.stderr:2024-09-14T06:32:49.241+0000 7f8cd8849640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33620) UID: 0 2024-09-14T06:32:49.268 INFO:teuthology.orchestra.run.smithi138.stdout:{ 2024-09-14T06:32:49.268 INFO:teuthology.orchestra.run.smithi138.stdout: "ops": [], 2024-09-14T06:32:49.269 INFO:teuthology.orchestra.run.smithi138.stdout: "complaint_time": 30, 2024-09-14T06:32:49.269 INFO:teuthology.orchestra.run.smithi138.stdout: "num_blocked_ops": 0 2024-09-14T06:32:49.269 INFO:teuthology.orchestra.run.smithi138.stdout:} 2024-09-14T06:32:49.277 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-14T06:32:49.343 INFO:tasks.ceph.osd.1.smithi094.stderr:2024-09-14T06:32:49.342+0000 7f328e47a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33641) UID: 0 2024-09-14T06:32:49.422 INFO:teuthology.orchestra.run.smithi138.stdout:{ 2024-09-14T06:32:49.422 INFO:teuthology.orchestra.run.smithi138.stdout: "size": 20, 2024-09-14T06:32:49.422 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 600, 2024-09-14T06:32:49.422 INFO:teuthology.orchestra.run.smithi138.stdout: "ops": [ 2024-09-14T06:32:49.422 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:49.422 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:19328 3.3 3:c34e5e51:::rbd_data.112d92027590.0000000000000024:f [read 1926144~1024 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:32:49.422 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:27.059768+0000", 2024-09-14T06:32:49.422 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 202.36035909700001, 2024-09-14T06:32:49.422 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046369147999999999, 2024-09-14T06:32:49.422 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:32:49.423 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:32:49.423 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:32:49.423 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:32:49.423 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:32:49.423 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:49.423 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 19328 2024-09-14T06:32:49.423 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:49.423 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:32:49.423 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:49.423 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:32:49.423 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.059768+0000", 2024-09-14T06:32:49.423 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:49.423 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:49.423 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:49.423 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:32:49.423 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.059768+0000", 2024-09-14T06:32:49.424 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:49.424 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:49.424 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:49.424 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:32:49.424 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.059874+0000", 2024-09-14T06:32:49.424 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00010551400000000001 2024-09-14T06:32:49.424 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:49.424 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:49.424 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:32:49.424 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.059877+0000", 2024-09-14T06:32:49.424 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.8279999999999999e-06 2024-09-14T06:32:49.424 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:49.424 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:49.424 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:32:49.425 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.059884+0000", 2024-09-14T06:32:49.425 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.6389999999999992e-06 2024-09-14T06:32:49.425 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:49.425 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:49.425 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:32:49.425 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.059890+0000", 2024-09-14T06:32:49.425 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.49e-06 2024-09-14T06:32:49.425 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:49.425 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:49.425 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:32:49.425 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.105804+0000", 2024-09-14T06:32:49.425 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.045914242000000001 2024-09-14T06:32:49.425 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:49.425 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:49.425 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:32:49.426 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.105886+0000", 2024-09-14T06:32:49.426 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.2305999999999994e-05 2024-09-14T06:32:49.426 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:49.426 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:49.426 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:32:49.426 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.106138+0000", 2024-09-14T06:32:49.426 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.000251129 2024-09-14T06:32:49.426 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:49.426 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:32:49.426 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:49.426 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:49.426 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:49.426 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:19329 3.3 3:c34e5e51:::rbd_data.112d92027590.0000000000000024:f [read 1936384~1024 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:32:49.426 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:27.059899+0000", 2024-09-14T06:32:49.426 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 202.36022857399999, 2024-09-14T06:32:49.427 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046572122000000001, 2024-09-14T06:32:49.427 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:32:49.427 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:32:49.427 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:32:49.427 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:32:49.427 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:32:49.427 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:49.427 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 19329 2024-09-14T06:32:49.427 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:49.427 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:32:49.427 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:49.427 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:32:49.427 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.059899+0000", 2024-09-14T06:32:49.427 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:49.427 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:49.428 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:49.428 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:32:49.428 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.059899+0000", 2024-09-14T06:32:49.428 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:49.428 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:49.428 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:49.428 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:32:49.428 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062501+0000", 2024-09-14T06:32:49.428 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.0026016419999999999 2024-09-14T06:32:49.428 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:49.428 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:49.428 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:32:49.428 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062514+0000", 2024-09-14T06:32:49.429 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.2947e-05 2024-09-14T06:32:49.429 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:49.429 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:49.429 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:32:49.429 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062523+0000", 2024-09-14T06:32:49.429 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.5519999999999993e-06 2024-09-14T06:32:49.429 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:49.429 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:49.430 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:32:49.430 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062534+0000", 2024-09-14T06:32:49.430 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.1416e-05 2024-09-14T06:32:49.430 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:49.430 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:49.430 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:32:49.430 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.106179+0000", 2024-09-14T06:32:49.430 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.043644049999999997 2024-09-14T06:32:49.430 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:49.430 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:49.430 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:32:49.431 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.106262+0000", 2024-09-14T06:32:49.431 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.3178999999999998e-05 2024-09-14T06:32:49.431 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:49.431 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:49.431 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:32:49.431 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.106471+0000", 2024-09-14T06:32:49.431 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00020933599999999999 2024-09-14T06:32:49.431 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:49.431 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:32:49.431 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:49.431 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:49.432 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:49.432 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:19351 3.3 3:c34e5e51:::rbd_data.112d92027590.0000000000000024:f [read 1159168~1024 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:32:49.432 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:27.062766+0000", 2024-09-14T06:32:49.432 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 202.357361349, 2024-09-14T06:32:49.432 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046099454999999998, 2024-09-14T06:32:49.432 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:32:49.432 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:32:49.432 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:32:49.432 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:32:49.432 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:32:49.432 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:49.432 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 19351 2024-09-14T06:32:49.432 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:49.433 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:32:49.433 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:49.433 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:32:49.433 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062766+0000", 2024-09-14T06:32:49.433 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:49.433 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:49.433 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:49.433 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:32:49.433 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062766+0000", 2024-09-14T06:32:49.434 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:49.434 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:49.434 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:49.434 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:32:49.434 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062766+0000", 2024-09-14T06:32:49.434 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.6e-07 2024-09-14T06:32:49.434 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:49.434 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:49.434 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:32:49.434 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062767+0000", 2024-09-14T06:32:49.434 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.0070000000000001e-06 2024-09-14T06:32:49.434 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:49.434 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:49.434 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:32:49.435 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062785+0000", 2024-09-14T06:32:49.435 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.7884999999999999e-05 2024-09-14T06:32:49.435 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:49.435 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:49.435 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:32:49.435 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062790+0000", 2024-09-14T06:32:49.435 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.0900000000000004e-06 2024-09-14T06:32:49.435 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:49.435 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:49.435 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:32:49.435 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.108735+0000", 2024-09-14T06:32:49.435 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.045944411999999997 2024-09-14T06:32:49.435 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:49.436 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:49.436 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:32:49.436 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.108809+0000", 2024-09-14T06:32:49.436 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.4157000000000002e-05 2024-09-14T06:32:49.436 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:49.436 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:49.436 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:32:49.436 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.108866+0000", 2024-09-14T06:32:49.436 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.6644000000000002e-05 2024-09-14T06:32:49.436 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:49.436 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:32:49.437 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:49.437 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:49.437 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:49.437 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:19352 3.3 3:c34e5e51:::rbd_data.112d92027590.0000000000000024:f [read 1169408~1024 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:32:49.437 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:27.062798+0000", 2024-09-14T06:32:49.437 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 202.35732993799999, 2024-09-14T06:32:49.437 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046255900000000003, 2024-09-14T06:32:49.437 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:32:49.437 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:32:49.437 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:32:49.437 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:32:49.437 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:32:49.437 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:49.438 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 19352 2024-09-14T06:32:49.438 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:49.438 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:32:49.438 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:49.438 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:32:49.438 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062798+0000", 2024-09-14T06:32:49.438 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:49.438 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:49.438 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:49.438 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:32:49.438 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062798+0000", 2024-09-14T06:32:49.438 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:49.438 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:49.438 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:49.439 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:32:49.439 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062798+0000", 2024-09-14T06:32:49.439 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.37e-07 2024-09-14T06:32:49.439 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:49.439 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:49.439 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:32:49.439 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062800+0000", 2024-09-14T06:32:49.439 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.0269999999999998e-06 2024-09-14T06:32:49.439 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:49.439 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:49.439 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:32:49.439 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062806+0000", 2024-09-14T06:32:49.439 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.6690000000000003e-06 2024-09-14T06:32:49.440 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:49.440 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:49.440 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:32:49.440 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062810+0000", 2024-09-14T06:32:49.440 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.3449999999999999e-06 2024-09-14T06:32:49.440 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:49.440 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:49.440 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:32:49.440 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.108892+0000", 2024-09-14T06:32:49.440 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046081715000000002 2024-09-14T06:32:49.440 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:49.440 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:49.440 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:32:49.440 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.108940+0000", 2024-09-14T06:32:49.441 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.7815999999999998e-05 2024-09-14T06:32:49.441 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:49.441 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:49.441 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:32:49.441 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.109053+0000", 2024-09-14T06:32:49.441 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.000113791 2024-09-14T06:32:49.441 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:49.441 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:32:49.441 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:49.441 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:49.441 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:49.441 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:19353 3.3 3:c34e5e51:::rbd_data.112d92027590.0000000000000024:f [read 1179648~1024 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:32:49.441 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:27.062816+0000", 2024-09-14T06:32:49.442 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 202.35731141900001, 2024-09-14T06:32:49.442 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046481256999999998, 2024-09-14T06:32:49.442 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:32:49.442 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:32:49.442 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:32:49.442 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:32:49.442 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:32:49.442 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:49.442 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 19353 2024-09-14T06:32:49.442 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:49.442 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:32:49.442 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:49.442 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:32:49.442 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062816+0000", 2024-09-14T06:32:49.443 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:49.443 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:49.443 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:49.443 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:32:49.443 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062816+0000", 2024-09-14T06:32:49.443 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:49.443 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:49.443 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:49.443 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:32:49.443 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062816+0000", 2024-09-14T06:32:49.443 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.2899999999999999e-07 2024-09-14T06:32:49.443 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:49.443 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:49.443 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:32:49.444 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062818+0000", 2024-09-14T06:32:49.444 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.699e-06 2024-09-14T06:32:49.444 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:49.444 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:49.444 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:32:49.444 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062825+0000", 2024-09-14T06:32:49.444 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.878e-06 2024-09-14T06:32:49.444 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:49.444 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:49.444 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:32:49.444 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062830+0000", 2024-09-14T06:32:49.444 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.5410000000000002e-06 2024-09-14T06:32:49.444 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:49.445 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:49.445 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:32:49.445 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.109096+0000", 2024-09-14T06:32:49.445 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046266517 2024-09-14T06:32:49.445 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:49.445 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:49.445 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:32:49.445 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.109190+0000", 2024-09-14T06:32:49.445 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.4000999999999997e-05 2024-09-14T06:32:49.445 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:49.445 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:49.445 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:32:49.445 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.109297+0000", 2024-09-14T06:32:49.445 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.000107292 2024-09-14T06:32:49.446 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:49.446 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:32:49.446 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:49.446 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:49.446 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:49.446 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:19354 3.3 3:c34e5e51:::rbd_data.112d92027590.0000000000000024:f [read 1189888~1024 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:32:49.446 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:27.062842+0000", 2024-09-14T06:32:49.446 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 202.357285964, 2024-09-14T06:32:49.446 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046594257, 2024-09-14T06:32:49.446 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:32:49.446 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:32:49.446 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:32:49.446 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:32:49.446 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:32:49.447 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:49.447 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 19354 2024-09-14T06:32:49.447 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:49.447 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:32:49.447 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:49.447 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:32:49.447 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062842+0000", 2024-09-14T06:32:49.447 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:49.447 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:49.447 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:49.447 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:32:49.447 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062842+0000", 2024-09-14T06:32:49.447 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:49.447 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:49.448 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:49.448 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:32:49.448 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062842+0000", 2024-09-14T06:32:49.448 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.7399999999999999e-07 2024-09-14T06:32:49.448 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:49.448 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:49.448 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:32:49.448 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062859+0000", 2024-09-14T06:32:49.448 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.7010000000000001e-05 2024-09-14T06:32:49.448 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:49.448 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:49.448 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:32:49.448 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062866+0000", 2024-09-14T06:32:49.448 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.9990000000000002e-06 2024-09-14T06:32:49.449 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:49.449 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:49.449 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:32:49.449 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062873+0000", 2024-09-14T06:32:49.449 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.8569999999999997e-06 2024-09-14T06:32:49.449 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:49.449 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:49.449 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:32:49.449 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.109335+0000", 2024-09-14T06:32:49.449 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046461943999999998 2024-09-14T06:32:49.449 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:49.449 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:49.449 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:32:49.449 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.109377+0000", 2024-09-14T06:32:49.450 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.2558999999999997e-05 2024-09-14T06:32:49.450 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:49.450 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:49.450 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:32:49.450 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.109436+0000", 2024-09-14T06:32:49.450 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.8514000000000003e-05 2024-09-14T06:32:49.450 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:49.450 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:32:49.450 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:49.450 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:49.450 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:49.450 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:19355 3.3 3:c34e5e51:::rbd_data.112d92027590.0000000000000024:f [read 1200128~1024 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:32:49.450 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:27.062884+0000", 2024-09-14T06:32:49.450 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 202.357243291, 2024-09-14T06:32:49.451 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046681650999999998, 2024-09-14T06:32:49.451 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:32:49.451 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:32:49.451 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:32:49.451 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:32:49.451 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:32:49.451 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:49.451 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 19355 2024-09-14T06:32:49.451 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:49.451 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:32:49.451 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:49.451 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:32:49.451 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062884+0000", 2024-09-14T06:32:49.452 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:49.452 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:49.452 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:49.452 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:32:49.452 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062884+0000", 2024-09-14T06:32:49.452 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:49.452 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:49.452 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:49.452 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:32:49.452 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062885+0000", 2024-09-14T06:32:49.452 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.8000000000000001e-07 2024-09-14T06:32:49.452 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:49.452 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:49.453 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:32:49.453 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062886+0000", 2024-09-14T06:32:49.453 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.8920000000000001e-06 2024-09-14T06:32:49.453 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:49.453 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:49.453 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:32:49.453 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062892+0000", 2024-09-14T06:32:49.453 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.84e-06 2024-09-14T06:32:49.453 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:49.453 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:49.453 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:32:49.453 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062899+0000", 2024-09-14T06:32:49.453 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.5339999999999996e-06 2024-09-14T06:32:49.454 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:49.454 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:49.454 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:32:49.454 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.109466+0000", 2024-09-14T06:32:49.454 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046566919999999998 2024-09-14T06:32:49.454 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:49.454 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:49.454 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:32:49.454 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.109510+0000", 2024-09-14T06:32:49.454 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.4153000000000002e-05 2024-09-14T06:32:49.454 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:49.454 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:49.454 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:32:49.454 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.109566+0000", 2024-09-14T06:32:49.455 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.5931999999999999e-05 2024-09-14T06:32:49.455 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:49.455 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:32:49.455 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:49.455 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:49.455 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:49.455 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:19356 3.3 3:c34e5e51:::rbd_data.112d92027590.0000000000000024:f [read 1210368~1024 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:32:49.455 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:27.062908+0000", 2024-09-14T06:32:49.455 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 202.35721985399999, 2024-09-14T06:32:49.455 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046807498000000003, 2024-09-14T06:32:49.455 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:32:49.455 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:32:49.455 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:32:49.456 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:32:49.456 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:32:49.456 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:49.456 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 19356 2024-09-14T06:32:49.456 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:49.456 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:32:49.456 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:49.456 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:32:49.456 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062908+0000", 2024-09-14T06:32:49.456 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:49.456 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:49.456 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:49.456 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:32:49.457 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062908+0000", 2024-09-14T06:32:49.457 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:49.457 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:49.457 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:49.457 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:32:49.457 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062908+0000", 2024-09-14T06:32:49.457 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.5699999999999998e-07 2024-09-14T06:32:49.457 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:49.457 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:49.457 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:32:49.457 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062910+0000", 2024-09-14T06:32:49.457 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.7710000000000001e-06 2024-09-14T06:32:49.458 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:49.458 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:49.458 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:32:49.458 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062920+0000", 2024-09-14T06:32:49.458 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.0654e-05 2024-09-14T06:32:49.458 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:49.458 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:49.458 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:32:49.458 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062931+0000", 2024-09-14T06:32:49.458 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.075e-05 2024-09-14T06:32:49.458 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:49.458 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:49.458 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:32:49.458 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.109595+0000", 2024-09-14T06:32:49.459 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046663713000000002 2024-09-14T06:32:49.459 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:49.459 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:49.459 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:32:49.459 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.109632+0000", 2024-09-14T06:32:49.459 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.7505000000000001e-05 2024-09-14T06:32:49.459 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:49.459 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:49.459 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:32:49.459 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.109715+0000", 2024-09-14T06:32:49.459 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.2748000000000001e-05 2024-09-14T06:32:49.459 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:49.459 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:32:49.459 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:49.460 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:49.460 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:49.460 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:19357 3.3 3:c34e5e51:::rbd_data.112d92027590.0000000000000024:f [read 1220608~1024 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:32:49.460 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:27.062939+0000", 2024-09-14T06:32:49.460 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 202.35718840600001, 2024-09-14T06:32:49.460 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046920508, 2024-09-14T06:32:49.460 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:32:49.460 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:32:49.460 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:32:49.460 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:32:49.460 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:32:49.460 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:49.460 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 19357 2024-09-14T06:32:49.461 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:49.461 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:32:49.461 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:49.461 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:32:49.461 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062939+0000", 2024-09-14T06:32:49.461 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:49.461 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:49.461 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:49.461 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:32:49.461 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062939+0000", 2024-09-14T06:32:49.461 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:49.461 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:49.461 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:49.461 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:32:49.462 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062940+0000", 2024-09-14T06:32:49.462 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.32e-07 2024-09-14T06:32:49.462 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:49.462 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:49.462 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:32:49.462 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062942+0000", 2024-09-14T06:32:49.462 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.1919999999999999e-06 2024-09-14T06:32:49.462 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:49.462 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:49.462 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:32:49.462 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062948+0000", 2024-09-14T06:32:49.462 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.1149999999999999e-06 2024-09-14T06:32:49.462 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:49.463 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:49.463 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:32:49.463 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062960+0000", 2024-09-14T06:32:49.463 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.1895e-05 2024-09-14T06:32:49.463 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:49.463 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:49.463 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:32:49.463 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.109704+0000", 2024-09-14T06:32:49.463 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046744584999999998 2024-09-14T06:32:49.463 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:49.463 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:49.463 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:32:49.463 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.109777+0000", 2024-09-14T06:32:49.463 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.2282999999999999e-05 2024-09-14T06:32:49.464 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:49.464 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:49.464 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:32:49.464 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.109860+0000", 2024-09-14T06:32:49.464 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.3005999999999998e-05 2024-09-14T06:32:49.464 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:49.464 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:32:49.464 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:49.464 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:49.464 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:49.464 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:19358 3.3 3:c34e5e51:::rbd_data.112d92027590.0000000000000024:f [read 1230848~1024 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:32:49.464 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:27.062967+0000", 2024-09-14T06:32:49.464 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 202.357160776, 2024-09-14T06:32:49.464 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047144471, 2024-09-14T06:32:49.465 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:32:49.465 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:32:49.465 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:32:49.465 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:32:49.465 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:32:49.465 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:49.465 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 19358 2024-09-14T06:32:49.465 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:49.465 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:32:49.465 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:49.465 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:32:49.465 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062967+0000", 2024-09-14T06:32:49.465 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:49.466 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:49.466 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:49.466 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:32:49.466 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062967+0000", 2024-09-14T06:32:49.466 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:49.466 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:49.466 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:49.466 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:32:49.466 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062967+0000", 2024-09-14T06:32:49.466 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.89e-07 2024-09-14T06:32:49.466 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:49.466 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:49.467 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:32:49.467 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062969+0000", 2024-09-14T06:32:49.467 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.9929999999999998e-06 2024-09-14T06:32:49.467 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:49.467 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:49.467 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:32:49.467 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062975+0000", 2024-09-14T06:32:49.467 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.9030000000000001e-06 2024-09-14T06:32:49.467 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:49.467 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:49.467 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:32:49.467 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062979+0000", 2024-09-14T06:32:49.467 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.2250000000000002e-06 2024-09-14T06:32:49.468 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:49.468 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:49.468 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:32:49.468 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.109934+0000", 2024-09-14T06:32:49.468 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046954410000000002 2024-09-14T06:32:49.468 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:49.468 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:49.468 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:32:49.468 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.110007+0000", 2024-09-14T06:32:49.468 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.3829000000000001e-05 2024-09-14T06:32:49.468 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:49.468 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:49.468 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:32:49.468 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.110111+0000", 2024-09-14T06:32:49.469 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00010372199999999999 2024-09-14T06:32:49.469 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:49.469 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:32:49.469 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:49.469 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:49.469 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:49.469 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:19359 3.3 3:c34e5e51:::rbd_data.112d92027590.0000000000000024:f [read 1241088~1024 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:32:49.469 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:27.062987+0000", 2024-09-14T06:32:49.469 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 202.35714068600001, 2024-09-14T06:32:49.469 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047297182, 2024-09-14T06:32:49.469 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:32:49.469 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:32:49.469 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:32:49.469 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:32:49.470 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:32:49.470 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:49.470 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 19359 2024-09-14T06:32:49.470 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:49.470 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:32:49.470 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:49.470 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:32:49.470 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062987+0000", 2024-09-14T06:32:49.470 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:49.470 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:49.470 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:49.470 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:32:49.470 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062987+0000", 2024-09-14T06:32:49.470 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:49.471 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:49.471 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:49.471 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:32:49.471 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062987+0000", 2024-09-14T06:32:49.471 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.2500000000000001e-07 2024-09-14T06:32:49.471 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:49.471 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:49.471 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:32:49.471 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062989+0000", 2024-09-14T06:32:49.471 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.6080000000000001e-06 2024-09-14T06:32:49.471 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:49.471 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:49.471 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:32:49.471 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062994+0000", 2024-09-14T06:32:49.472 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.5520000000000004e-06 2024-09-14T06:32:49.472 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:49.472 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:49.472 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:32:49.472 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062999+0000", 2024-09-14T06:32:49.472 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.3930000000000001e-06 2024-09-14T06:32:49.472 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:49.472 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:49.472 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:32:49.472 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.110142+0000", 2024-09-14T06:32:49.472 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047143154 2024-09-14T06:32:49.472 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:49.472 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:49.472 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:32:49.472 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.110219+0000", 2024-09-14T06:32:49.473 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.6737999999999996e-05 2024-09-14T06:32:49.473 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:49.473 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:49.473 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:32:49.473 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.110284+0000", 2024-09-14T06:32:49.473 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.5412000000000003e-05 2024-09-14T06:32:49.473 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:49.473 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:32:49.473 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:49.473 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:49.473 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:49.473 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:19360 3.3 3:c34e5e51:::rbd_data.112d92027590.0000000000000024:f [read 1251328~1024 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:32:49.474 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:27.063017+0000", 2024-09-14T06:32:49.474 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 202.35711022500001, 2024-09-14T06:32:49.474 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047453823999999999, 2024-09-14T06:32:49.474 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:32:49.474 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:32:49.474 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:32:49.474 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:32:49.474 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:32:49.474 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:49.474 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 19360 2024-09-14T06:32:49.474 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:49.474 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:32:49.474 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:49.474 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:32:49.475 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.063017+0000", 2024-09-14T06:32:49.475 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:49.475 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:49.475 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:49.475 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:32:49.475 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.063017+0000", 2024-09-14T06:32:49.475 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:49.475 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:49.475 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:49.475 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:32:49.475 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.064977+0000", 2024-09-14T06:32:49.475 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.0019597080000000001 2024-09-14T06:32:49.475 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:49.476 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:49.476 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:32:49.476 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.064987+0000", 2024-09-14T06:32:49.476 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.9620000000000005e-06 2024-09-14T06:32:49.476 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:49.476 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:49.476 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:32:49.476 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065000+0000", 2024-09-14T06:32:49.476 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.3173000000000001e-05 2024-09-14T06:32:49.476 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:49.476 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:49.476 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:32:49.477 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065005+0000", 2024-09-14T06:32:49.477 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.9960000000000001e-06 2024-09-14T06:32:49.477 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:49.477 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:49.477 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:32:49.477 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.110314+0000", 2024-09-14T06:32:49.477 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.04530899 2024-09-14T06:32:49.477 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:49.477 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:49.477 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:32:49.477 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.110398+0000", 2024-09-14T06:32:49.477 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.3539000000000001e-05 2024-09-14T06:32:49.478 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:49.478 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:49.478 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:32:49.478 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.110471+0000", 2024-09-14T06:32:49.478 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.3455999999999996e-05 2024-09-14T06:32:49.478 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:49.478 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:32:49.478 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:49.478 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:49.478 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:49.478 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:19365 3.3 3:c34e5e51:::rbd_data.112d92027590.0000000000000024:f [read 1302528~1024 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:32:49.478 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:27.065058+0000", 2024-09-14T06:32:49.478 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 202.355069555, 2024-09-14T06:32:49.479 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046105526000000001, 2024-09-14T06:32:49.479 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:32:49.479 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:32:49.479 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:32:49.479 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:32:49.479 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:32:49.479 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:49.479 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 19365 2024-09-14T06:32:49.479 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:49.479 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:32:49.479 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:49.479 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:32:49.479 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065058+0000", 2024-09-14T06:32:49.479 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:49.480 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:49.480 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:49.480 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:32:49.480 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065058+0000", 2024-09-14T06:32:49.480 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:49.480 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:49.480 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:49.480 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:32:49.480 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065058+0000", 2024-09-14T06:32:49.480 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.1899999999999999e-07 2024-09-14T06:32:49.480 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:49.480 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:49.481 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:32:49.481 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065059+0000", 2024-09-14T06:32:49.481 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.1299999999999998e-07 2024-09-14T06:32:49.481 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:49.481 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:49.481 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:32:49.481 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065063+0000", 2024-09-14T06:32:49.481 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.1099999999999996e-06 2024-09-14T06:32:49.481 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:49.481 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:49.481 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:32:49.481 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065070+0000", 2024-09-14T06:32:49.481 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.973e-06 2024-09-14T06:32:49.482 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:49.482 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:49.482 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:32:49.482 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.111014+0000", 2024-09-14T06:32:49.482 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.045943373000000003 2024-09-14T06:32:49.482 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:49.482 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:49.482 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:32:49.482 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.111073+0000", 2024-09-14T06:32:49.482 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.8987999999999998e-05 2024-09-14T06:32:49.482 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:49.482 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:49.482 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:32:49.482 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.111163+0000", 2024-09-14T06:32:49.483 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.0950000000000002e-05 2024-09-14T06:32:49.483 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:49.483 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:32:49.483 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:49.483 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:49.483 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:49.483 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:19366 3.3 3:c34e5e51:::rbd_data.112d92027590.0000000000000024:f [read 1312768~1024 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:32:49.483 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:27.065074+0000", 2024-09-14T06:32:49.483 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 202.35505370499999, 2024-09-14T06:32:49.483 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046289858000000003, 2024-09-14T06:32:49.483 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:32:49.483 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:32:49.483 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:32:49.483 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:32:49.484 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:32:49.484 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:49.484 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 19366 2024-09-14T06:32:49.484 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:49.484 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:32:49.484 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:49.484 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:32:49.484 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065074+0000", 2024-09-14T06:32:49.484 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:49.484 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:49.484 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:49.484 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:32:49.484 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065074+0000", 2024-09-14T06:32:49.484 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:49.484 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:49.485 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:49.485 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:32:49.485 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065074+0000", 2024-09-14T06:32:49.485 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.3900000000000001e-07 2024-09-14T06:32:49.485 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:49.485 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:49.485 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:32:49.485 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065078+0000", 2024-09-14T06:32:49.485 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.3229999999999999e-06 2024-09-14T06:32:49.485 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:49.485 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:49.485 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:32:49.485 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065082+0000", 2024-09-14T06:32:49.485 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.8299999999999998e-06 2024-09-14T06:32:49.486 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:49.486 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:49.486 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:32:49.486 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065085+0000", 2024-09-14T06:32:49.486 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.8660000000000002e-06 2024-09-14T06:32:49.486 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:49.486 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:49.486 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:32:49.486 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.111270+0000", 2024-09-14T06:32:49.486 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046185035999999999 2024-09-14T06:32:49.486 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:49.486 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:49.486 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:32:49.486 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.111311+0000", 2024-09-14T06:32:49.486 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.0935e-05 2024-09-14T06:32:49.486 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:49.487 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:49.487 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:32:49.487 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.111364+0000", 2024-09-14T06:32:49.487 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.2629e-05 2024-09-14T06:32:49.487 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:49.487 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:32:49.487 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:49.487 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:49.487 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:49.487 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:19367 3.3 3:c34e5e51:::rbd_data.112d92027590.0000000000000024:f [read 1323008~1024 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:32:49.487 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:27.065115+0000", 2024-09-14T06:32:49.487 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 202.355012144, 2024-09-14T06:32:49.487 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046407003000000002, 2024-09-14T06:32:49.487 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:32:49.487 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:32:49.488 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:32:49.488 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:32:49.488 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:32:49.488 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:49.488 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 19367 2024-09-14T06:32:49.488 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:49.488 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:32:49.488 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:49.488 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:32:49.488 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065115+0000", 2024-09-14T06:32:49.488 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:49.488 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:49.488 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:49.489 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:32:49.489 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065115+0000", 2024-09-14T06:32:49.489 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:49.489 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:49.489 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:49.489 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:32:49.489 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065116+0000", 2024-09-14T06:32:49.489 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.4999999999999998e-07 2024-09-14T06:32:49.489 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:49.489 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:49.489 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:32:49.489 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065117+0000", 2024-09-14T06:32:49.489 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.189e-06 2024-09-14T06:32:49.490 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:49.490 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:49.490 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:32:49.490 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065122+0000", 2024-09-14T06:32:49.490 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.3709999999999999e-06 2024-09-14T06:32:49.490 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:49.490 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:49.490 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:32:49.490 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065125+0000", 2024-09-14T06:32:49.490 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.8940000000000002e-06 2024-09-14T06:32:49.490 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:49.490 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:49.490 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:32:49.490 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.111402+0000", 2024-09-14T06:32:49.491 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046276645999999998 2024-09-14T06:32:49.491 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:49.491 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:49.491 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:32:49.491 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.111445+0000", 2024-09-14T06:32:49.491 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.3288000000000003e-05 2024-09-14T06:32:49.491 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:49.491 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:49.491 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:32:49.491 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.111522+0000", 2024-09-14T06:32:49.491 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.7164999999999997e-05 2024-09-14T06:32:49.491 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:49.491 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:32:49.491 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:49.491 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:49.491 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:49.492 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:19368 3.3 3:c34e5e51:::rbd_data.112d92027590.0000000000000024:f [read 1333248~1024 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:32:49.492 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:27.065130+0000", 2024-09-14T06:32:49.492 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 202.354997854, 2024-09-14T06:32:49.492 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046507275000000001, 2024-09-14T06:32:49.492 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:32:49.492 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:32:49.492 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:32:49.492 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:32:49.492 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:32:49.492 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:49.492 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 19368 2024-09-14T06:32:49.492 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:49.492 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:32:49.492 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:49.493 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:32:49.493 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065130+0000", 2024-09-14T06:32:49.493 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:49.493 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:49.493 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:49.493 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:32:49.493 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065130+0000", 2024-09-14T06:32:49.493 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:49.493 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:49.493 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:49.493 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:32:49.493 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065130+0000", 2024-09-14T06:32:49.493 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.0800000000000001e-07 2024-09-14T06:32:49.494 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:49.494 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:49.494 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:32:49.494 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065131+0000", 2024-09-14T06:32:49.494 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.8700000000000004e-07 2024-09-14T06:32:49.494 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:49.494 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:49.494 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:32:49.494 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065134+0000", 2024-09-14T06:32:49.494 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.1039999999999998e-06 2024-09-14T06:32:49.494 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:49.494 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:49.494 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:32:49.495 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065136+0000", 2024-09-14T06:32:49.495 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.3949999999999999e-06 2024-09-14T06:32:49.495 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:49.495 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:49.495 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:32:49.495 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.111541+0000", 2024-09-14T06:32:49.495 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046404206000000003 2024-09-14T06:32:49.495 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:49.495 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:49.495 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:32:49.495 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.111582+0000", 2024-09-14T06:32:49.495 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.1051999999999999e-05 2024-09-14T06:32:49.496 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:49.496 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:49.496 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:32:49.496 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.111637+0000", 2024-09-14T06:32:49.496 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.5322999999999999e-05 2024-09-14T06:32:49.496 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:49.496 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:32:49.496 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:49.496 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:49.496 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:49.496 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:19369 3.3 3:c34e5e51:::rbd_data.112d92027590.0000000000000024:f [read 1343488~1024 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:32:49.496 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:27.065139+0000", 2024-09-14T06:32:49.497 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 202.354988037, 2024-09-14T06:32:49.497 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046682169000000003, 2024-09-14T06:32:49.497 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:32:49.497 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:32:49.497 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:32:49.497 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:32:49.497 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:32:49.497 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:49.497 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 19369 2024-09-14T06:32:49.497 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:49.497 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:32:49.497 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:49.497 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:32:49.498 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065139+0000", 2024-09-14T06:32:49.498 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:49.498 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:49.498 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:49.498 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:32:49.498 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065139+0000", 2024-09-14T06:32:49.498 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:49.498 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:49.498 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:49.498 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:32:49.498 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065140+0000", 2024-09-14T06:32:49.498 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.8299999999999998e-07 2024-09-14T06:32:49.498 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:49.499 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:49.499 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:32:49.499 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065141+0000", 2024-09-14T06:32:49.499 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.3900000000000003e-07 2024-09-14T06:32:49.499 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:49.499 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:49.499 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:32:49.499 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065144+0000", 2024-09-14T06:32:49.499 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.7409999999999998e-06 2024-09-14T06:32:49.499 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:49.499 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:49.499 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:32:49.499 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065147+0000", 2024-09-14T06:32:49.499 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.4150000000000002e-06 2024-09-14T06:32:49.500 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:49.500 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:49.500 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:32:49.500 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.111673+0000", 2024-09-14T06:32:49.500 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046526004000000003 2024-09-14T06:32:49.500 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:49.500 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:49.500 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:32:49.500 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.111749+0000", 2024-09-14T06:32:49.500 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.5741999999999994e-05 2024-09-14T06:32:49.500 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:49.500 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:49.500 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:32:49.501 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.111822+0000", 2024-09-14T06:32:49.501 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.3045000000000003e-05 2024-09-14T06:32:49.501 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:49.501 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:32:49.501 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:49.501 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:49.501 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:49.501 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:19370 3.3 3:c34e5e51:::rbd_data.112d92027590.0000000000000024:f [read 1353728~1024 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:32:49.501 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:27.065150+0000", 2024-09-14T06:32:49.501 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 202.354977308, 2024-09-14T06:32:49.501 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046897556999999999, 2024-09-14T06:32:49.501 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:32:49.501 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:32:49.502 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:32:49.502 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:32:49.502 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:32:49.502 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:49.502 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 19370 2024-09-14T06:32:49.502 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:49.502 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:32:49.502 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:49.502 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:32:49.502 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065150+0000", 2024-09-14T06:32:49.502 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:49.502 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:49.502 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:49.502 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:32:49.503 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065150+0000", 2024-09-14T06:32:49.503 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:49.503 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:49.503 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:49.503 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:32:49.503 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065150+0000", 2024-09-14T06:32:49.503 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.0200000000000001e-07 2024-09-14T06:32:49.503 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:49.503 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:49.503 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:32:49.503 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065151+0000", 2024-09-14T06:32:49.503 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.1100000000000004e-07 2024-09-14T06:32:49.503 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:49.504 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:49.504 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:32:49.504 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065154+0000", 2024-09-14T06:32:49.504 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.9639999999999999e-06 2024-09-14T06:32:49.504 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:49.504 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:49.504 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:32:49.504 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065157+0000", 2024-09-14T06:32:49.504 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.4260000000000002e-06 2024-09-14T06:32:49.504 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:49.504 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:49.504 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:32:49.505 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.111861+0000", 2024-09-14T06:32:49.505 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046704268 2024-09-14T06:32:49.505 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:49.505 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:49.505 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:32:49.505 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.111939+0000", 2024-09-14T06:32:49.505 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.8231000000000004e-05 2024-09-14T06:32:49.505 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:49.505 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:49.505 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:32:49.505 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.112048+0000", 2024-09-14T06:32:49.505 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00010855500000000001 2024-09-14T06:32:49.505 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:49.506 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:32:49.506 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:49.506 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:49.506 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:49.506 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:19371 3.3 3:c34e5e51:::rbd_data.112d92027590.0000000000000024:f [read 1363968~1024 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:32:49.506 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:27.065161+0000", 2024-09-14T06:32:49.506 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 202.354966658, 2024-09-14T06:32:49.506 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047021944000000003, 2024-09-14T06:32:49.506 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:32:49.506 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:32:49.506 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:32:49.506 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:32:49.507 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:32:49.507 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:49.507 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 19371 2024-09-14T06:32:49.507 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:49.507 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:32:49.507 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:49.507 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:32:49.507 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065161+0000", 2024-09-14T06:32:49.507 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:49.507 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:49.507 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:49.507 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:32:49.507 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065161+0000", 2024-09-14T06:32:49.507 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:49.508 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:49.508 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:49.508 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:32:49.508 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065161+0000", 2024-09-14T06:32:49.508 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.0200000000000001e-07 2024-09-14T06:32:49.508 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:49.508 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:49.508 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:32:49.508 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065166+0000", 2024-09-14T06:32:49.508 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.9869999999999999e-06 2024-09-14T06:32:49.508 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:49.508 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:49.508 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:32:49.509 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065169+0000", 2024-09-14T06:32:49.509 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.3799999999999998e-06 2024-09-14T06:32:49.509 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:49.509 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:49.509 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:32:49.509 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065172+0000", 2024-09-14T06:32:49.509 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.4830000000000002e-06 2024-09-14T06:32:49.509 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:49.509 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:49.509 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:32:49.509 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.112071+0000", 2024-09-14T06:32:49.509 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046899529000000002 2024-09-14T06:32:49.509 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:49.509 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:49.510 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:32:49.510 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.112128+0000", 2024-09-14T06:32:49.510 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.6341999999999997e-05 2024-09-14T06:32:49.510 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:49.510 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:49.510 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:32:49.510 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.112183+0000", 2024-09-14T06:32:49.510 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.5021000000000001e-05 2024-09-14T06:32:49.510 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:49.510 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:32:49.510 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:49.510 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:49.510 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:49.510 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:19372 3.3 3:c34e5e51:::rbd_data.112d92027590.0000000000000024:f [read 1374208~1024 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:32:49.510 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:27.065176+0000", 2024-09-14T06:32:49.511 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 202.354951508, 2024-09-14T06:32:49.511 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047188889999999997, 2024-09-14T06:32:49.511 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:32:49.511 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:32:49.511 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:32:49.511 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:32:49.511 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:32:49.511 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:49.511 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 19372 2024-09-14T06:32:49.511 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:49.511 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:32:49.511 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:49.511 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:32:49.511 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065176+0000", 2024-09-14T06:32:49.512 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:49.512 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:49.512 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:49.512 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:32:49.512 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065176+0000", 2024-09-14T06:32:49.512 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:49.512 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:49.512 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:49.512 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:32:49.512 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.071117+0000", 2024-09-14T06:32:49.512 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.0059404990000000001 2024-09-14T06:32:49.512 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:49.512 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:49.512 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:32:49.513 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.071120+0000", 2024-09-14T06:32:49.513 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.4340000000000001e-06 2024-09-14T06:32:49.513 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:49.513 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:49.513 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:32:49.513 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.071129+0000", 2024-09-14T06:32:49.513 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.8389999999999996e-06 2024-09-14T06:32:49.513 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:49.513 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:49.513 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:32:49.513 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.071134+0000", 2024-09-14T06:32:49.513 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.412e-06 2024-09-14T06:32:49.513 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:49.514 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:49.514 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:32:49.514 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.112221+0000", 2024-09-14T06:32:49.514 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.041086473999999998 2024-09-14T06:32:49.514 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:49.514 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:49.514 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:32:49.514 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.112299+0000", 2024-09-14T06:32:49.514 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.8746999999999997e-05 2024-09-14T06:32:49.514 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:49.514 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:49.514 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:32:49.514 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.112365+0000", 2024-09-14T06:32:49.514 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.5485000000000001e-05 2024-09-14T06:32:49.515 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:49.515 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:32:49.515 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:49.515 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:49.515 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:32:49.515 INFO:teuthology.orchestra.run.smithi138.stdout:} 2024-09-14T06:32:49.517 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-14T06:32:49.517 INFO:tasks.ceph.osd.3.smithi094.stderr:2024-09-14T06:32:49.515+0000 7feeeafe8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33655) UID: 0 2024-09-14T06:32:49.618 INFO:tasks.ceph.osd.0.smithi094.stderr:2024-09-14T06:32:49.617+0000 7f07c9333640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33636) UID: 0 2024-09-14T06:32:49.621 INFO:teuthology.orchestra.run.smithi138.stdout:{ 2024-09-14T06:32:49.621 INFO:teuthology.orchestra.run.smithi138.stdout: "ops": [], 2024-09-14T06:32:49.621 INFO:teuthology.orchestra.run.smithi138.stdout: "num_ops": 0 2024-09-14T06:32:49.621 INFO:teuthology.orchestra.run.smithi138.stdout:} 2024-09-14T06:32:49.631 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-14T06:32:49.718 INFO:tasks.ceph.osd.6.smithi138.stderr:2024-09-14T06:32:49.717+0000 7f8cd8849640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33620) UID: 0 2024-09-14T06:32:49.737 INFO:teuthology.orchestra.run.smithi138.stdout:{ 2024-09-14T06:32:49.737 INFO:teuthology.orchestra.run.smithi138.stdout: "ops": [], 2024-09-14T06:32:49.737 INFO:teuthology.orchestra.run.smithi138.stdout: "complaint_time": 30, 2024-09-14T06:32:49.737 INFO:teuthology.orchestra.run.smithi138.stdout: "num_blocked_ops": 0 2024-09-14T06:32:49.737 INFO:teuthology.orchestra.run.smithi138.stdout:} 2024-09-14T06:32:49.746 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-14T06:32:49.813 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-14T06:32:49.857 INFO:tasks.ceph.osd.0.smithi094.stderr:2024-09-14T06:32:49.856+0000 7f07c9333640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33636) UID: 0 2024-09-14T06:32:49.890 INFO:teuthology.orchestra.run.smithi138.stdout:{ 2024-09-14T06:32:49.890 INFO:teuthology.orchestra.run.smithi138.stdout: "size": 20, 2024-09-14T06:32:49.890 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 600, 2024-09-14T06:32:49.890 INFO:teuthology.orchestra.run.smithi138.stdout: "ops": [ 2024-09-14T06:32:49.890 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:49.890 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:11453 3.1d 3:bcad7303:::rbd_data.112d92027590.0000000000000001:f [read 94208~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:32:49.890 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:25.654375+0000", 2024-09-14T06:32:49.890 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 204.23398229200001, 2024-09-14T06:32:49.890 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052449318000000002, 2024-09-14T06:32:49.891 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:32:49.891 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:32:49.891 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:32:49.891 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:32:49.891 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:32:49.891 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:49.891 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 11453 2024-09-14T06:32:49.891 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:49.891 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:32:49.891 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:49.891 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:32:49.891 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654375+0000", 2024-09-14T06:32:49.891 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:49.892 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:49.892 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:49.892 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:32:49.892 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654375+0000", 2024-09-14T06:32:49.892 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:49.892 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:49.892 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:49.892 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:32:49.892 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654496+0000", 2024-09-14T06:32:49.892 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00012081 2024-09-14T06:32:49.892 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:49.892 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:49.892 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:32:49.892 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654497+0000", 2024-09-14T06:32:49.893 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.8789999999999999e-06 2024-09-14T06:32:49.893 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:49.893 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:49.893 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:32:49.893 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654501+0000", 2024-09-14T06:32:49.893 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.8999999999999999e-06 2024-09-14T06:32:49.893 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:49.893 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:49.893 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:32:49.893 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654505+0000", 2024-09-14T06:32:49.893 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.2229999999999999e-06 2024-09-14T06:32:49.893 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:49.894 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:49.894 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:32:49.894 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.705917+0000", 2024-09-14T06:32:49.894 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.051412501999999999 2024-09-14T06:32:49.894 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:49.894 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:49.894 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:32:49.894 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.705999+0000", 2024-09-14T06:32:49.894 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.1551999999999994e-05 2024-09-14T06:32:49.894 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:49.894 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:49.894 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:32:49.894 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.706824+0000", 2024-09-14T06:32:49.894 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00082545199999999996 2024-09-14T06:32:49.895 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:49.895 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:32:49.895 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:49.895 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:49.895 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:49.895 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:11454 3.1d 3:bcad7303:::rbd_data.112d92027590.0000000000000001:f [read 104448~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:32:49.895 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:25.654510+0000", 2024-09-14T06:32:49.895 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 204.233847502, 2024-09-14T06:32:49.895 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052610713000000003, 2024-09-14T06:32:49.895 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:32:49.895 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:32:49.895 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:32:49.895 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:32:49.896 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:32:49.896 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:49.896 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 11454 2024-09-14T06:32:49.896 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:49.896 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:32:49.896 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:49.896 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:32:49.896 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654510+0000", 2024-09-14T06:32:49.896 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:49.896 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:49.896 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:49.896 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:32:49.896 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654510+0000", 2024-09-14T06:32:49.896 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:49.897 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:49.897 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:49.897 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:32:49.897 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654825+0000", 2024-09-14T06:32:49.897 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00031522400000000002 2024-09-14T06:32:49.897 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:49.897 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:49.897 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:32:49.897 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654827+0000", 2024-09-14T06:32:49.897 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.9709999999999998e-06 2024-09-14T06:32:49.897 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:49.897 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:49.897 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:32:49.897 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654832+0000", 2024-09-14T06:32:49.898 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.4099999999999999e-06 2024-09-14T06:32:49.898 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:49.898 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:49.898 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:32:49.898 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654835+0000", 2024-09-14T06:32:49.898 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.14e-06 2024-09-14T06:32:49.898 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:49.898 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:49.898 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:32:49.898 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.706851+0000", 2024-09-14T06:32:49.898 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052015761000000001 2024-09-14T06:32:49.898 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:49.898 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:49.898 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:32:49.899 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.706934+0000", 2024-09-14T06:32:49.899 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.2484999999999994e-05 2024-09-14T06:32:49.899 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:49.899 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:49.899 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:32:49.899 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.707120+0000", 2024-09-14T06:32:49.899 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00018672200000000001 2024-09-14T06:32:49.899 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:49.899 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:32:49.899 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:49.899 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:49.899 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:49.899 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:11455 3.1d 3:bcad7303:::rbd_data.112d92027590.0000000000000001:f [read 114688~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:32:49.899 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:25.654841+0000", 2024-09-14T06:32:49.900 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 204.233516173, 2024-09-14T06:32:49.900 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052579898, 2024-09-14T06:32:49.900 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:32:49.900 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:32:49.900 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:32:49.900 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:32:49.900 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:32:49.900 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:49.900 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 11455 2024-09-14T06:32:49.900 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:49.900 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:32:49.900 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:49.900 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:32:49.900 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654841+0000", 2024-09-14T06:32:49.901 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:49.901 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:49.901 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:49.901 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:32:49.901 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654841+0000", 2024-09-14T06:32:49.901 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:49.901 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:49.901 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:49.901 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:32:49.901 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654841+0000", 2024-09-14T06:32:49.901 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.7300000000000002e-07 2024-09-14T06:32:49.901 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:49.901 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:49.901 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:32:49.901 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654845+0000", 2024-09-14T06:32:49.902 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.0520000000000003e-06 2024-09-14T06:32:49.902 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:49.902 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:49.902 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:32:49.902 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654849+0000", 2024-09-14T06:32:49.902 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.0459999999999999e-06 2024-09-14T06:32:49.902 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:49.902 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:49.902 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:32:49.902 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654852+0000", 2024-09-14T06:32:49.902 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.9830000000000001e-06 2024-09-14T06:32:49.902 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:49.902 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:49.902 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:32:49.902 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.707159+0000", 2024-09-14T06:32:49.903 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052306826000000001 2024-09-14T06:32:49.903 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:49.903 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:49.903 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:32:49.903 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.707242+0000", 2024-09-14T06:32:49.903 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.2595000000000005e-05 2024-09-14T06:32:49.903 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:49.903 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:49.903 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:32:49.903 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.707421+0000", 2024-09-14T06:32:49.903 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.000179123 2024-09-14T06:32:49.903 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:49.903 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:32:49.903 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:49.903 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:49.904 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:49.904 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:11456 3.1d 3:bcad7303:::rbd_data.112d92027590.0000000000000001:f [read 124928~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:32:49.904 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:25.654856+0000", 2024-09-14T06:32:49.904 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 204.233501327, 2024-09-14T06:32:49.904 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052801580000000001, 2024-09-14T06:32:49.904 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:32:49.904 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:32:49.904 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:32:49.904 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:32:49.904 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:32:49.904 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:49.904 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 11456 2024-09-14T06:32:49.904 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:49.904 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:32:49.905 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:49.905 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:32:49.905 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654856+0000", 2024-09-14T06:32:49.905 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:49.905 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:49.905 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:49.905 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:32:49.905 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654856+0000", 2024-09-14T06:32:49.905 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:49.905 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:49.905 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:49.905 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:32:49.905 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654856+0000", 2024-09-14T06:32:49.905 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.29e-07 2024-09-14T06:32:49.906 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:49.906 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:49.906 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:32:49.906 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654857+0000", 2024-09-14T06:32:49.906 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.187e-06 2024-09-14T06:32:49.906 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:49.906 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:49.906 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:32:49.906 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654861+0000", 2024-09-14T06:32:49.906 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.3850000000000002e-06 2024-09-14T06:32:49.906 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:49.906 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:49.906 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:32:49.907 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654863+0000", 2024-09-14T06:32:49.907 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.9500000000000001e-06 2024-09-14T06:32:49.907 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:49.907 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:49.907 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:32:49.907 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.707456+0000", 2024-09-14T06:32:49.907 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052592434 2024-09-14T06:32:49.907 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:49.907 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:49.907 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:32:49.907 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.707516+0000", 2024-09-14T06:32:49.907 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.0167999999999997e-05 2024-09-14T06:32:49.907 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:49.907 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:49.907 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:32:49.908 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.707657+0000", 2024-09-14T06:32:49.908 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00014122700000000001 2024-09-14T06:32:49.908 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:49.908 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:32:49.908 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:49.908 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:49.908 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:49.908 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:11457 3.1d 3:bcad7303:::rbd_data.112d92027590.0000000000000001:f [read 135168~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:32:49.908 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:25.654867+0000", 2024-09-14T06:32:49.908 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 204.233490442, 2024-09-14T06:32:49.908 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052921902, 2024-09-14T06:32:49.908 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:32:49.908 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:32:49.909 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:32:49.909 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:32:49.909 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:32:49.909 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:49.909 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 11457 2024-09-14T06:32:49.909 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:49.909 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:32:49.909 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:49.909 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:32:49.909 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654867+0000", 2024-09-14T06:32:49.909 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:49.909 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:49.909 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:49.909 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:32:49.910 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654867+0000", 2024-09-14T06:32:49.910 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:49.910 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:49.910 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:49.910 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:32:49.910 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654972+0000", 2024-09-14T06:32:49.910 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00010584 2024-09-14T06:32:49.910 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:49.910 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:49.910 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:32:49.910 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654975+0000", 2024-09-14T06:32:49.910 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.1169999999999998e-06 2024-09-14T06:32:49.910 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:49.910 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:49.911 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:32:49.911 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654979+0000", 2024-09-14T06:32:49.911 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.1960000000000001e-06 2024-09-14T06:32:49.911 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:49.911 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:49.911 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:32:49.911 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654982+0000", 2024-09-14T06:32:49.911 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.2990000000000001e-06 2024-09-14T06:32:49.911 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:49.911 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:49.911 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:32:49.911 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.707684+0000", 2024-09-14T06:32:49.911 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052701979000000003 2024-09-14T06:32:49.911 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:49.912 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:49.912 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:32:49.912 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.707728+0000", 2024-09-14T06:32:49.912 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.3529999999999998e-05 2024-09-14T06:32:49.912 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:49.912 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:49.912 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:32:49.912 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.707789+0000", 2024-09-14T06:32:49.912 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.0940999999999998e-05 2024-09-14T06:32:49.912 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:49.912 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:32:49.912 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:49.912 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:49.912 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:49.913 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:11458 3.1d 3:bcad7303:::rbd_data.112d92027590.0000000000000001:f [read 145408~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:32:49.913 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:25.654987+0000", 2024-09-14T06:32:49.913 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 204.233370044, 2024-09-14T06:32:49.913 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.05291655, 2024-09-14T06:32:49.913 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:32:49.913 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:32:49.913 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:32:49.913 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:32:49.913 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:32:49.913 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:49.913 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 11458 2024-09-14T06:32:49.913 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:49.913 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:32:49.914 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:49.914 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:32:49.914 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654987+0000", 2024-09-14T06:32:49.914 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:49.914 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:49.914 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:49.914 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:32:49.914 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654987+0000", 2024-09-14T06:32:49.914 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:49.914 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:49.914 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:49.914 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:32:49.914 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655077+0000", 2024-09-14T06:32:49.914 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.0253000000000005e-05 2024-09-14T06:32:49.914 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:49.915 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:49.915 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:32:49.915 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655079+0000", 2024-09-14T06:32:49.915 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.9989999999999998e-06 2024-09-14T06:32:49.915 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:49.915 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:49.915 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:32:49.915 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655083+0000", 2024-09-14T06:32:49.915 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.9280000000000003e-06 2024-09-14T06:32:49.915 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:49.915 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:49.915 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:32:49.915 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655086+0000", 2024-09-14T06:32:49.915 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.056e-06 2024-09-14T06:32:49.916 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:49.916 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:49.916 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:32:49.916 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.707805+0000", 2024-09-14T06:32:49.916 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052718627999999997 2024-09-14T06:32:49.916 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:49.916 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:49.916 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:32:49.916 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.707848+0000", 2024-09-14T06:32:49.916 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.2728999999999997e-05 2024-09-14T06:32:49.916 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:49.916 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:49.916 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:32:49.917 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.707904+0000", 2024-09-14T06:32:49.917 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.5957e-05 2024-09-14T06:32:49.917 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:49.917 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:32:49.917 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:49.917 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:49.917 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:49.917 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:11459 3.1d 3:bcad7303:::rbd_data.112d92027590.0000000000000001:f [read 155648~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:32:49.917 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:25.655091+0000", 2024-09-14T06:32:49.918 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 204.233266036, 2024-09-14T06:32:49.918 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052944242000000002, 2024-09-14T06:32:49.918 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:32:49.918 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:32:49.918 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:32:49.918 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:32:49.918 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:32:49.918 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:49.918 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 11459 2024-09-14T06:32:49.919 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:49.919 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:32:49.919 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:49.919 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:32:49.919 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655091+0000", 2024-09-14T06:32:49.919 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:49.919 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:49.919 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:49.919 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:32:49.919 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655091+0000", 2024-09-14T06:32:49.919 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:49.919 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:49.919 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:49.920 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:32:49.920 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655237+0000", 2024-09-14T06:32:49.920 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00014570100000000001 2024-09-14T06:32:49.920 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:49.920 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:49.920 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:32:49.920 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655239+0000", 2024-09-14T06:32:49.920 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.8670000000000001e-06 2024-09-14T06:32:49.920 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:49.920 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:49.920 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:32:49.920 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655243+0000", 2024-09-14T06:32:49.920 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.9829999999999998e-06 2024-09-14T06:32:49.920 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:49.921 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:49.921 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:32:49.921 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655246+0000", 2024-09-14T06:32:49.921 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.1540000000000002e-06 2024-09-14T06:32:49.921 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:49.921 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:49.921 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:32:49.921 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.707934+0000", 2024-09-14T06:32:49.921 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052688286000000001 2024-09-14T06:32:49.921 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:49.921 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:49.921 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:32:49.921 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.707977+0000", 2024-09-14T06:32:49.922 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.3291999999999998e-05 2024-09-14T06:32:49.922 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:49.922 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:49.922 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:32:49.922 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.708035+0000", 2024-09-14T06:32:49.922 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.7958999999999999e-05 2024-09-14T06:32:49.922 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:49.922 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:32:49.922 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:49.922 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:49.922 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:49.922 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:11460 3.1d 3:bcad7303:::rbd_data.112d92027590.0000000000000001:f [read 165888~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:32:49.922 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:25.655250+0000", 2024-09-14T06:32:49.922 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 204.23310673899999, 2024-09-14T06:32:49.923 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052910698999999999, 2024-09-14T06:32:49.923 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:32:49.923 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:32:49.923 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:32:49.923 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:32:49.923 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:32:49.923 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:49.923 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 11460 2024-09-14T06:32:49.923 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:49.923 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:32:49.923 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:49.923 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:32:49.923 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655250+0000", 2024-09-14T06:32:49.923 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:49.924 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:49.924 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:49.924 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:32:49.924 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655250+0000", 2024-09-14T06:32:49.924 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:49.924 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:49.924 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:49.924 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:32:49.924 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655356+0000", 2024-09-14T06:32:49.924 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00010535900000000001 2024-09-14T06:32:49.924 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:49.924 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:49.924 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:32:49.925 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655357+0000", 2024-09-14T06:32:49.925 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.8080000000000001e-06 2024-09-14T06:32:49.925 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:49.925 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:49.925 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:32:49.925 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655361+0000", 2024-09-14T06:32:49.925 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.861e-06 2024-09-14T06:32:49.925 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:49.925 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:49.925 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:32:49.925 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655364+0000", 2024-09-14T06:32:49.925 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.1089999999999998e-06 2024-09-14T06:32:49.925 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:49.926 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:49.926 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:32:49.926 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.708065+0000", 2024-09-14T06:32:49.926 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052700595000000003 2024-09-14T06:32:49.926 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:49.926 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:49.926 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:32:49.926 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.708108+0000", 2024-09-14T06:32:49.926 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.2484999999999997e-05 2024-09-14T06:32:49.926 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:49.926 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:49.926 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:32:49.926 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.708161+0000", 2024-09-14T06:32:49.926 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.3482000000000001e-05 2024-09-14T06:32:49.926 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:49.927 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:32:49.927 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:49.927 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:49.927 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:49.927 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:11461 3.1d 3:bcad7303:::rbd_data.112d92027590.0000000000000001:f [read 176128~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:32:49.927 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:25.655370+0000", 2024-09-14T06:32:49.927 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 204.23298745700001, 2024-09-14T06:32:49.927 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052915141999999998, 2024-09-14T06:32:49.927 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:32:49.927 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:32:49.927 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:32:49.927 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:32:49.927 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:32:49.927 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:49.927 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 11461 2024-09-14T06:32:49.928 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:49.928 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:32:49.928 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:49.928 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:32:49.928 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655370+0000", 2024-09-14T06:32:49.928 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:49.928 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:49.928 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:49.928 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:32:49.928 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655370+0000", 2024-09-14T06:32:49.928 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:49.928 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:49.928 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:49.928 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:32:49.928 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655394+0000", 2024-09-14T06:32:49.929 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.4553999999999999e-05 2024-09-14T06:32:49.929 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:49.929 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:49.929 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:32:49.929 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655396+0000", 2024-09-14T06:32:49.929 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.8950000000000001e-06 2024-09-14T06:32:49.929 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:49.929 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:49.929 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:32:49.929 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655401+0000", 2024-09-14T06:32:49.929 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.0089999999999998e-06 2024-09-14T06:32:49.929 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:49.929 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:49.930 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:32:49.930 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655404+0000", 2024-09-14T06:32:49.930 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.3400000000000002e-06 2024-09-14T06:32:49.930 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:49.930 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:49.930 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:32:49.930 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.708186+0000", 2024-09-14T06:32:49.930 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052781726000000001 2024-09-14T06:32:49.930 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:49.930 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:49.930 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:32:49.930 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.708229+0000", 2024-09-14T06:32:49.930 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.2670999999999999e-05 2024-09-14T06:32:49.930 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:49.931 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:49.931 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:32:49.931 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.708285+0000", 2024-09-14T06:32:49.931 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.5946999999999998e-05 2024-09-14T06:32:49.931 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:49.931 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:32:49.931 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:49.931 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:49.931 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:49.931 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:11462 3.1d 3:bcad7303:::rbd_data.112d92027590.0000000000000001:f [read 186368~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:32:49.931 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:25.655410+0000", 2024-09-14T06:32:49.931 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 204.23294697899999, 2024-09-14T06:32:49.931 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052995258000000003, 2024-09-14T06:32:49.932 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:32:49.932 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:32:49.932 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:32:49.932 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:32:49.932 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:32:49.932 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:49.932 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 11462 2024-09-14T06:32:49.932 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:49.932 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:32:49.932 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:49.932 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:32:49.932 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655410+0000", 2024-09-14T06:32:49.933 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:49.933 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:49.933 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:49.933 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:32:49.933 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655410+0000", 2024-09-14T06:32:49.933 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:49.933 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:49.933 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:49.933 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:32:49.933 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655625+0000", 2024-09-14T06:32:49.933 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00021442600000000001 2024-09-14T06:32:49.933 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:49.933 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:49.934 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:32:49.934 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655626+0000", 2024-09-14T06:32:49.934 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.815e-06 2024-09-14T06:32:49.934 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:49.934 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:49.934 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:32:49.934 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655631+0000", 2024-09-14T06:32:49.934 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.651e-06 2024-09-14T06:32:49.934 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:49.934 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:49.934 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:32:49.934 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655634+0000", 2024-09-14T06:32:49.934 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.2449999999999998e-06 2024-09-14T06:32:49.935 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:49.935 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:49.935 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:32:49.935 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.708312+0000", 2024-09-14T06:32:49.935 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052677412 2024-09-14T06:32:49.935 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:49.935 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:49.935 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:32:49.935 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.708352+0000", 2024-09-14T06:32:49.935 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.0729999999999998e-05 2024-09-14T06:32:49.935 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:49.935 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:49.935 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:32:49.935 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.708405+0000", 2024-09-14T06:32:49.936 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.2979000000000002e-05 2024-09-14T06:32:49.936 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:49.936 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:32:49.936 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:49.936 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:49.936 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:49.936 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:11463 3.1d 3:bcad7303:::rbd_data.112d92027590.0000000000000001:f [read 196608~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:32:49.936 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:25.655639+0000", 2024-09-14T06:32:49.936 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 204.23271783300001, 2024-09-14T06:32:49.936 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052888859000000003, 2024-09-14T06:32:49.936 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:32:49.936 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:32:49.936 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:32:49.936 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:32:49.937 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:32:49.937 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:49.937 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 11463 2024-09-14T06:32:49.937 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:49.937 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:32:49.937 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:49.937 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:32:49.937 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655639+0000", 2024-09-14T06:32:49.937 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:49.937 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:49.937 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:49.937 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:32:49.937 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655639+0000", 2024-09-14T06:32:49.938 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:49.938 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:49.938 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:49.938 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:32:49.938 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655640+0000", 2024-09-14T06:32:49.938 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.08e-07 2024-09-14T06:32:49.938 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:49.938 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:49.938 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:32:49.938 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655641+0000", 2024-09-14T06:32:49.938 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.156e-06 2024-09-14T06:32:49.938 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:49.938 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:49.939 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:32:49.939 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655648+0000", 2024-09-14T06:32:49.939 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.8460000000000002e-06 2024-09-14T06:32:49.939 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:49.939 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:49.939 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:32:49.939 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655651+0000", 2024-09-14T06:32:49.939 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.0079999999999998e-06 2024-09-14T06:32:49.939 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:49.939 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:49.939 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:32:49.939 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.708429+0000", 2024-09-14T06:32:49.939 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052778459 2024-09-14T06:32:49.940 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:49.940 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:49.940 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:32:49.940 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.708473+0000", 2024-09-14T06:32:49.940 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.3782999999999997e-05 2024-09-14T06:32:49.940 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:49.940 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:49.940 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:32:49.940 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.708528+0000", 2024-09-14T06:32:49.940 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.5198999999999998e-05 2024-09-14T06:32:49.940 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:49.940 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:32:49.940 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:49.941 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:49.941 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:49.941 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:11464 3.1d 3:bcad7303:::rbd_data.112d92027590.0000000000000001:f [read 206848~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:32:49.941 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:25.655654+0000", 2024-09-14T06:32:49.941 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 204.232702964, 2024-09-14T06:32:49.941 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052981845999999999, 2024-09-14T06:32:49.941 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:32:49.941 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:32:49.941 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:32:49.941 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:32:49.941 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:32:49.941 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:49.941 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 11464 2024-09-14T06:32:49.942 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:49.942 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:32:49.942 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:49.942 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:32:49.942 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655654+0000", 2024-09-14T06:32:49.942 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:49.942 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:49.942 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:49.942 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:32:49.942 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655654+0000", 2024-09-14T06:32:49.942 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:49.942 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:49.942 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:49.942 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:32:49.942 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655946+0000", 2024-09-14T06:32:49.943 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00029148200000000003 2024-09-14T06:32:49.943 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:49.943 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:49.943 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:32:49.943 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655948+0000", 2024-09-14T06:32:49.943 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.973e-06 2024-09-14T06:32:49.943 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:49.943 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:49.943 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:32:49.943 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655953+0000", 2024-09-14T06:32:49.943 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.1240000000000004e-06 2024-09-14T06:32:49.943 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:49.943 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:49.943 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:32:49.944 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655956+0000", 2024-09-14T06:32:49.944 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.315e-06 2024-09-14T06:32:49.944 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:49.944 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:49.944 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:32:49.944 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.708543+0000", 2024-09-14T06:32:49.944 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052587416999999997 2024-09-14T06:32:49.944 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:49.944 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:49.944 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:32:49.944 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.708584+0000", 2024-09-14T06:32:49.944 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.1004000000000002e-05 2024-09-14T06:32:49.944 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:49.945 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:49.945 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:32:49.945 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.708636+0000", 2024-09-14T06:32:49.945 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.1530999999999998e-05 2024-09-14T06:32:49.945 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:49.945 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:32:49.945 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:49.945 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:49.945 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:49.945 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:11465 3.1d 3:bcad7303:::rbd_data.112d92027590.0000000000000001:f [read 217088~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:32:49.945 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:25.655961+0000", 2024-09-14T06:32:49.945 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 204.23239581199999, 2024-09-14T06:32:49.945 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052798682999999999, 2024-09-14T06:32:49.945 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:32:49.946 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:32:49.946 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:32:49.946 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:32:49.946 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:32:49.946 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:49.946 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 11465 2024-09-14T06:32:49.946 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:49.946 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:32:49.946 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:49.946 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:32:49.946 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655961+0000", 2024-09-14T06:32:49.946 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:49.946 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:49.946 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:49.947 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:32:49.947 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655961+0000", 2024-09-14T06:32:49.947 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:49.947 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:49.947 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:49.947 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:32:49.947 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.656114+0000", 2024-09-14T06:32:49.947 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.000152745 2024-09-14T06:32:49.947 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:49.947 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:49.947 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:32:49.947 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.656116+0000", 2024-09-14T06:32:49.947 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.804e-06 2024-09-14T06:32:49.948 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:49.948 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:49.948 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:32:49.948 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.656120+0000", 2024-09-14T06:32:49.948 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.4700000000000004e-06 2024-09-14T06:32:49.948 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:49.948 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:49.948 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:32:49.948 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.656123+0000", 2024-09-14T06:32:49.948 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.1269999999999999e-06 2024-09-14T06:32:49.948 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:49.948 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:49.948 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:32:49.949 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.708660+0000", 2024-09-14T06:32:49.949 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052536919000000001 2024-09-14T06:32:49.949 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:49.949 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:49.949 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:32:49.949 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.708703+0000", 2024-09-14T06:32:49.949 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.3077000000000001e-05 2024-09-14T06:32:49.949 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:49.949 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:49.949 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:32:49.949 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.708760+0000", 2024-09-14T06:32:49.949 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.6541e-05 2024-09-14T06:32:49.949 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:49.949 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:32:49.950 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:49.950 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:49.950 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:49.950 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:11466 3.1d 3:bcad7303:::rbd_data.112d92027590.0000000000000001:f [read 227328~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:32:49.950 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:25.656129+0000", 2024-09-14T06:32:49.950 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 204.232228424, 2024-09-14T06:32:49.950 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052750987999999999, 2024-09-14T06:32:49.950 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:32:49.950 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:32:49.950 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:32:49.950 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:32:49.950 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:32:49.951 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:49.951 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 11466 2024-09-14T06:32:49.951 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:49.951 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:32:49.951 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:49.951 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:32:49.951 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.656129+0000", 2024-09-14T06:32:49.951 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:49.951 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:49.951 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:49.951 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:32:49.951 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.656129+0000", 2024-09-14T06:32:49.951 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:49.951 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:49.952 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:49.952 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:32:49.952 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.656347+0000", 2024-09-14T06:32:49.952 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.000218054 2024-09-14T06:32:49.952 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:49.952 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:49.952 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:32:49.952 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.656349+0000", 2024-09-14T06:32:49.952 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.9199999999999998e-06 2024-09-14T06:32:49.952 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:49.952 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:49.952 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:32:49.952 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.656353+0000", 2024-09-14T06:32:49.953 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.9210000000000002e-06 2024-09-14T06:32:49.953 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:49.953 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:49.953 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:32:49.953 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.656356+0000", 2024-09-14T06:32:49.953 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.7670000000000001e-06 2024-09-14T06:32:49.953 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:49.953 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:49.953 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:32:49.953 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.708787+0000", 2024-09-14T06:32:49.953 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052430840999999999 2024-09-14T06:32:49.953 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:49.953 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:49.953 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:32:49.954 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.708827+0000", 2024-09-14T06:32:49.954 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.9538e-05 2024-09-14T06:32:49.954 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:49.954 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:49.954 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:32:49.954 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.708880+0000", 2024-09-14T06:32:49.954 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.2947e-05 2024-09-14T06:32:49.954 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:49.954 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:32:49.954 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:49.954 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:49.954 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:49.954 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:11467 3.1d 3:bcad7303:::rbd_data.112d92027590.0000000000000001:f [read 237568~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:32:49.955 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:25.656361+0000", 2024-09-14T06:32:49.955 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 204.23199577599999, 2024-09-14T06:32:49.955 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052643898000000001, 2024-09-14T06:32:49.955 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:32:49.955 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:32:49.955 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:32:49.955 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:32:49.955 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:32:49.955 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:49.955 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 11467 2024-09-14T06:32:49.955 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:49.955 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:32:49.955 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:49.955 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:32:49.955 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.656361+0000", 2024-09-14T06:32:49.956 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:49.956 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:49.956 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:49.956 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:32:49.956 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.656361+0000", 2024-09-14T06:32:49.956 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:49.956 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:49.956 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:49.956 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:32:49.956 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.656855+0000", 2024-09-14T06:32:49.956 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00049372000000000001 2024-09-14T06:32:49.956 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:49.956 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:49.956 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:32:49.957 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.656859+0000", 2024-09-14T06:32:49.957 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.1300000000000003e-06 2024-09-14T06:32:49.957 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:49.957 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:49.957 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:32:49.957 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.656863+0000", 2024-09-14T06:32:49.957 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.2629999999999997e-06 2024-09-14T06:32:49.957 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:49.957 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:49.957 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:32:49.957 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.656867+0000", 2024-09-14T06:32:49.957 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.3220000000000001e-06 2024-09-14T06:32:49.957 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:49.957 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:49.958 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:32:49.958 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.708903+0000", 2024-09-14T06:32:49.958 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052035855999999998 2024-09-14T06:32:49.958 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:49.958 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:49.958 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:32:49.958 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.708949+0000", 2024-09-14T06:32:49.958 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.6276999999999998e-05 2024-09-14T06:32:49.958 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:49.958 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:49.958 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:32:49.958 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.709005+0000", 2024-09-14T06:32:49.958 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.6329999999999998e-05 2024-09-14T06:32:49.958 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:49.959 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:32:49.959 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:49.959 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:49.959 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:49.959 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:11480 3.1d 3:bcad7303:::rbd_data.112d92027590.0000000000000001:f [read 370688~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:32:49.959 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:25.658563+0000", 2024-09-14T06:32:49.959 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 204.22979401200001, 2024-09-14T06:32:49.959 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052429164, 2024-09-14T06:32:49.959 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:32:49.959 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:32:49.959 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:32:49.959 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:32:49.959 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:32:49.959 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:49.960 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 11480 2024-09-14T06:32:49.960 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:49.960 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:32:49.960 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:49.960 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:32:49.960 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.658563+0000", 2024-09-14T06:32:49.960 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:49.960 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:49.960 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:49.960 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:32:49.960 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.658563+0000", 2024-09-14T06:32:49.960 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:49.960 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:49.960 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:49.960 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:32:49.961 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.658658+0000", 2024-09-14T06:32:49.961 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.5346999999999994e-05 2024-09-14T06:32:49.961 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:49.961 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:49.961 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:32:49.961 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.658662+0000", 2024-09-14T06:32:49.961 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.14e-06 2024-09-14T06:32:49.961 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:49.961 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:49.961 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:32:49.961 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.658669+0000", 2024-09-14T06:32:49.961 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.7379999999999994e-06 2024-09-14T06:32:49.961 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:49.962 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:49.962 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:32:49.962 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.658675+0000", 2024-09-14T06:32:49.962 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.5430000000000001e-06 2024-09-14T06:32:49.962 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:49.962 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:49.962 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:32:49.962 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.710824+0000", 2024-09-14T06:32:49.962 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052148847999999998 2024-09-14T06:32:49.962 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:49.962 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:49.962 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:32:49.962 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.710864+0000", 2024-09-14T06:32:49.962 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.9925999999999997e-05 2024-09-14T06:32:49.963 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:49.963 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:49.963 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:32:49.963 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.710992+0000", 2024-09-14T06:32:49.963 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.000128622 2024-09-14T06:32:49.963 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:49.963 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:32:49.963 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:49.963 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:49.963 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:49.963 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:11481 3.1d 3:bcad7303:::rbd_data.112d92027590.0000000000000001:f [read 380928~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:32:49.963 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:25.658684+0000", 2024-09-14T06:32:49.963 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 204.22967294200001, 2024-09-14T06:32:49.963 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052508565, 2024-09-14T06:32:49.964 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:32:49.964 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:32:49.964 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:32:49.964 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:32:49.964 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:32:49.964 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:49.964 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 11481 2024-09-14T06:32:49.964 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:49.964 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:32:49.964 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:49.964 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:32:49.964 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.658684+0000", 2024-09-14T06:32:49.964 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:49.965 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:49.965 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:49.965 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:32:49.965 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.658684+0000", 2024-09-14T06:32:49.965 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:49.965 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:49.965 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:49.965 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:32:49.965 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.658715+0000", 2024-09-14T06:32:49.965 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.0703000000000001e-05 2024-09-14T06:32:49.965 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:49.965 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:49.965 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:32:49.965 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.658718+0000", 2024-09-14T06:32:49.966 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.0709999999999999e-06 2024-09-14T06:32:49.966 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:49.966 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:49.966 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:32:49.966 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.658725+0000", 2024-09-14T06:32:49.966 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.2339999999999997e-06 2024-09-14T06:32:49.966 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:49.966 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:49.966 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:32:49.966 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.658731+0000", 2024-09-14T06:32:49.966 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.6450000000000002e-06 2024-09-14T06:32:49.966 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:49.966 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:49.966 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:32:49.966 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.711020+0000", 2024-09-14T06:32:49.966 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052288886 2024-09-14T06:32:49.967 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:49.967 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:49.967 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:32:49.967 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.711061+0000", 2024-09-14T06:32:49.967 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.1084000000000002e-05 2024-09-14T06:32:49.967 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:49.967 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:49.967 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:32:49.967 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.711193+0000", 2024-09-14T06:32:49.967 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00013194199999999999 2024-09-14T06:32:49.967 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:49.967 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:32:49.967 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:49.967 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:49.968 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:49.968 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:11482 3.1d 3:bcad7303:::rbd_data.112d92027590.0000000000000001:f [read 391168~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:32:49.968 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:25.658740+0000", 2024-09-14T06:32:49.968 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 204.22961747700001, 2024-09-14T06:32:49.968 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052637015000000002, 2024-09-14T06:32:49.968 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:32:49.968 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:32:49.968 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:32:49.968 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:32:49.968 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:32:49.968 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:49.968 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 11482 2024-09-14T06:32:49.968 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:49.969 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:32:49.969 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:49.969 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:32:49.969 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.658740+0000", 2024-09-14T06:32:49.969 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:49.969 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:49.969 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:49.969 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:32:49.969 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.658740+0000", 2024-09-14T06:32:49.969 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:49.969 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:49.969 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:49.969 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:32:49.969 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.658880+0000", 2024-09-14T06:32:49.970 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00014036100000000001 2024-09-14T06:32:49.970 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:49.970 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:49.970 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:32:49.970 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.658883+0000", 2024-09-14T06:32:49.970 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.3969999999999998e-06 2024-09-14T06:32:49.970 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:49.970 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:49.970 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:32:49.970 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.658890+0000", 2024-09-14T06:32:49.970 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.1500000000000002e-06 2024-09-14T06:32:49.970 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:49.970 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:49.970 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:32:49.971 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.658897+0000", 2024-09-14T06:32:49.971 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.1260000000000003e-06 2024-09-14T06:32:49.971 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:49.971 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:49.971 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:32:49.971 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.711207+0000", 2024-09-14T06:32:49.971 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052310777000000003 2024-09-14T06:32:49.971 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:49.971 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:49.971 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:32:49.971 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.711249+0000", 2024-09-14T06:32:49.971 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.1786999999999998e-05 2024-09-14T06:32:49.971 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:49.971 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:49.971 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:32:49.972 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.711377+0000", 2024-09-14T06:32:49.972 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.000127417 2024-09-14T06:32:49.972 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:49.972 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:32:49.972 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:49.972 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:49.972 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:49.972 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:11483 3.1d 3:bcad7303:::rbd_data.112d92027590.0000000000000001:f [read 401408~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:32:49.972 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:25.658906+0000", 2024-09-14T06:32:49.972 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 204.22945069299999, 2024-09-14T06:32:49.972 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052595109000000001, 2024-09-14T06:32:49.972 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:32:49.972 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:32:49.973 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:32:49.973 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:32:49.973 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:32:49.973 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:49.973 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 11483 2024-09-14T06:32:49.973 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:49.973 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:32:49.973 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:49.973 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:32:49.973 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.658906+0000", 2024-09-14T06:32:49.973 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:49.973 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:49.973 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:49.973 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:32:49.974 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.658906+0000", 2024-09-14T06:32:49.974 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:49.974 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:49.974 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:49.974 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:32:49.974 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.664985+0000", 2024-09-14T06:32:49.974 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.0060787250000000001 2024-09-14T06:32:49.974 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:49.974 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:49.974 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:32:49.974 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.664991+0000", 2024-09-14T06:32:49.974 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.7030000000000003e-06 2024-09-14T06:32:49.974 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:49.974 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:49.975 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:32:49.975 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.664999+0000", 2024-09-14T06:32:49.975 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.4430000000000003e-06 2024-09-14T06:32:49.975 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:49.975 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:49.975 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:32:49.975 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.665006+0000", 2024-09-14T06:32:49.975 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.0840000000000003e-06 2024-09-14T06:32:49.975 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:49.975 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:49.975 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:32:49.975 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.711402+0000", 2024-09-14T06:32:49.975 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046395714999999997 2024-09-14T06:32:49.975 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:49.976 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:49.976 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:32:49.976 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.711443+0000", 2024-09-14T06:32:49.976 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.0911999999999997e-05 2024-09-14T06:32:49.976 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:49.976 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:49.976 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:32:49.976 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.711501+0000", 2024-09-14T06:32:49.976 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.8526999999999997e-05 2024-09-14T06:32:49.976 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:49.976 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:32:49.976 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:49.976 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:49.977 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:49.977 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:214569 3.2 3:479633bd:::rbd_object_map.112d3bd84cb0:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e170)", 2024-09-14T06:32:49.977 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:31:34.933901+0000", 2024-09-14T06:32:49.977 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 74.954456308999994, 2024-09-14T06:32:49.977 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.68157511400000004, 2024-09-14T06:32:49.977 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:32:49.977 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:32:49.977 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:32:49.977 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:32:49.977 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:32:49.977 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:49.977 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 214569 2024-09-14T06:32:49.977 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:49.978 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:32:49.978 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:49.978 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:32:49.978 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:31:34.933901+0000", 2024-09-14T06:32:49.978 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:49.978 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:49.978 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:49.978 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:32:49.978 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:31:34.933901+0000", 2024-09-14T06:32:49.978 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:49.978 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:49.978 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:49.978 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:32:49.978 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:31:34.933902+0000", 2024-09-14T06:32:49.978 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.7439999999999999e-06 2024-09-14T06:32:49.979 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:49.979 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:49.979 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:32:49.979 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:31:34.933910+0000", 2024-09-14T06:32:49.979 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.8569999999999995e-06 2024-09-14T06:32:49.979 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:49.979 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:49.979 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:32:49.979 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:31:34.933932+0000", 2024-09-14T06:32:49.979 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.1997999999999998e-05 2024-09-14T06:32:49.979 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:49.979 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:49.979 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:32:49.979 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:31:34.933941+0000", 2024-09-14T06:32:49.980 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.7180000000000002e-06 2024-09-14T06:32:49.980 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:49.980 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:49.980 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:32:49.980 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:31:34.934013+0000", 2024-09-14T06:32:49.980 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.1761000000000006e-05 2024-09-14T06:32:49.980 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:49.980 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:49.980 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "waiting for peered", 2024-09-14T06:32:49.980 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:31:34.934022+0000", 2024-09-14T06:32:49.980 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.8740000000000001e-06 2024-09-14T06:32:49.980 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:49.980 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:49.981 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:32:49.981 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:31:35.608803+0000", 2024-09-14T06:32:49.981 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.67478164299999999 2024-09-14T06:32:49.981 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:49.981 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:49.981 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:32:49.981 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:31:35.608964+0000", 2024-09-14T06:32:49.981 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00016112900000000001 2024-09-14T06:32:49.981 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:49.982 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:49.982 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "waiting for subops from 6", 2024-09-14T06:32:49.982 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:31:35.609399+0000", 2024-09-14T06:32:49.982 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00043454000000000002 2024-09-14T06:32:49.982 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:49.982 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:49.982 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "op_commit", 2024-09-14T06:32:49.982 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:31:35.610495+0000", 2024-09-14T06:32:49.982 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.001096404 2024-09-14T06:32:49.982 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:49.982 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:49.982 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:32:49.982 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:31:35.615409+0000", 2024-09-14T06:32:49.982 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.0049133199999999997 2024-09-14T06:32:49.983 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:49.983 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:49.983 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "commit_sent", 2024-09-14T06:32:49.983 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:31:35.615444+0000", 2024-09-14T06:32:49.983 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.5457000000000003e-05 2024-09-14T06:32:49.983 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:49.983 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:49.983 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:32:49.983 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:31:35.615476+0000", 2024-09-14T06:32:49.983 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.1668999999999998e-05 2024-09-14T06:32:49.983 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:49.983 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:32:49.983 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:49.983 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:49.984 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:32:49.984 INFO:teuthology.orchestra.run.smithi138.stdout:} 2024-09-14T06:32:49.985 INFO:teuthology.orchestra.run.smithi094.stdout:osd.0: {} 2024-09-14T06:32:49.985 INFO:teuthology.orchestra.run.smithi094.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-14T06:32:49.985 INFO:tasks.ceph.osd.0.smithi094.stderr:2024-09-14T06:32:49.984+0000 7f07c9333640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33636) UID: 0 2024-09-14T06:32:49.986 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-14T06:32:49.986 INFO:teuthology.orchestra.run.smithi094.stdout:osd.1: {} 2024-09-14T06:32:49.987 INFO:teuthology.orchestra.run.smithi094.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-14T06:32:50.002 INFO:teuthology.orchestra.run.smithi094.stdout:osd.2: {} 2024-09-14T06:32:50.002 INFO:teuthology.orchestra.run.smithi094.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-14T06:32:50.018 INFO:teuthology.orchestra.run.smithi094.stdout:osd.3: {} 2024-09-14T06:32:50.019 INFO:teuthology.orchestra.run.smithi094.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-14T06:32:50.034 INFO:teuthology.orchestra.run.smithi094.stdout:osd.4: {} 2024-09-14T06:32:50.035 INFO:teuthology.orchestra.run.smithi094.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-14T06:32:50.051 INFO:teuthology.orchestra.run.smithi094.stdout:osd.5: {} 2024-09-14T06:32:50.051 INFO:teuthology.orchestra.run.smithi094.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-14T06:32:50.067 INFO:teuthology.orchestra.run.smithi094.stdout:osd.6: {} 2024-09-14T06:32:50.067 INFO:teuthology.orchestra.run.smithi094.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-14T06:32:50.083 INFO:teuthology.orchestra.run.smithi094.stdout:osd.7: {} 2024-09-14T06:32:50.083 INFO:teuthology.orchestra.run.smithi094.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-14T06:32:50.085 INFO:tasks.ceph.osd.0.smithi094.stderr:2024-09-14T06:32:50.084+0000 7f07c9333640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33636) UID: 0 2024-09-14T06:32:50.091 INFO:teuthology.orchestra.run.smithi138.stdout:ERROR: (22) Invalid argument 2024-09-14T06:32:50.091 INFO:teuthology.orchestra.run.smithi138.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:32:50.100 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-14T06:32:50.186 INFO:tasks.ceph.osd.2.smithi094.stderr:2024-09-14T06:32:50.185+0000 7faf3269b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33649) UID: 0 2024-09-14T06:32:50.203 INFO:teuthology.orchestra.run.smithi138.stdout:ERROR: (22) Invalid argument 2024-09-14T06:32:50.203 INFO:teuthology.orchestra.run.smithi138.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:32:50.212 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-14T06:32:50.287 INFO:tasks.ceph.osd.2.smithi094.stderr:2024-09-14T06:32:50.286+0000 7faf3269b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33649) UID: 0 2024-09-14T06:32:50.313 INFO:teuthology.orchestra.run.smithi138.stdout:ERROR: (22) Invalid argument 2024-09-14T06:32:50.313 INFO:teuthology.orchestra.run.smithi138.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:32:50.322 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-14T06:32:50.387 INFO:tasks.ceph.osd.7.smithi138.stderr:2024-09-14T06:32:50.386+0000 7f3c6a21d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33623) UID: 0 2024-09-14T06:32:50.425 INFO:teuthology.orchestra.run.smithi138.stdout:ERROR: (22) Invalid argument 2024-09-14T06:32:50.425 INFO:teuthology.orchestra.run.smithi138.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:32:50.433 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-14T06:32:50.517 INFO:tasks.ceph.osd.4.smithi138.stderr:2024-09-14T06:32:50.516+0000 7f6c653c8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33615) UID: 0 2024-09-14T06:32:50.576 INFO:teuthology.orchestra.run.smithi138.stdout:ERROR: (22) Invalid argument 2024-09-14T06:32:50.576 INFO:teuthology.orchestra.run.smithi138.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:32:50.585 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-14T06:32:50.626 INFO:tasks.ceph.osd.4.smithi138.stderr:2024-09-14T06:32:50.625+0000 7f6c653c8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33615) UID: 0 2024-09-14T06:32:50.634 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-14T06:32:50.691 INFO:tasks.ceph.osd.7.smithi138.stderr:2024-09-14T06:32:50.690+0000 7f3c6a21d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33623) UID: 0 2024-09-14T06:32:50.729 INFO:teuthology.orchestra.run.smithi138.stdout:ERROR: (22) Invalid argument 2024-09-14T06:32:50.729 INFO:teuthology.orchestra.run.smithi138.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:32:50.738 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-14T06:32:50.792 INFO:tasks.ceph.osd.4.smithi138.stderr:2024-09-14T06:32:50.792+0000 7f6c653c8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33615) UID: 0 2024-09-14T06:32:50.843 INFO:teuthology.orchestra.run.smithi094.stdout:ERROR: (22) Invalid argument 2024-09-14T06:32:50.844 INFO:teuthology.orchestra.run.smithi094.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:32:50.853 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-14T06:32:50.893 INFO:tasks.ceph.osd.4.smithi138.stderr:2024-09-14T06:32:50.892+0000 7f6c653c8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33615) UID: 0 2024-09-14T06:32:50.957 INFO:teuthology.orchestra.run.smithi094.stdout:ERROR: (22) Invalid argument 2024-09-14T06:32:50.958 INFO:teuthology.orchestra.run.smithi094.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:32:50.966 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-14T06:32:50.994 INFO:tasks.ceph.osd.5.smithi138.stderr:2024-09-14T06:32:50.994+0000 7fe13746f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33618) UID: 0 2024-09-14T06:32:51.069 INFO:teuthology.orchestra.run.smithi094.stdout:ERROR: (22) Invalid argument 2024-09-14T06:32:51.069 INFO:teuthology.orchestra.run.smithi094.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:32:51.077 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-14T06:32:51.120 INFO:tasks.ceph.osd.2.smithi094.stderr:2024-09-14T06:32:51.119+0000 7faf3269b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33649) UID: 0 2024-09-14T06:32:51.179 INFO:teuthology.orchestra.run.smithi094.stdout:ERROR: (22) Invalid argument 2024-09-14T06:32:51.180 INFO:teuthology.orchestra.run.smithi094.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:32:51.188 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-14T06:32:51.195 INFO:tasks.ceph.osd.4.smithi138.stderr:2024-09-14T06:32:51.194+0000 7f6c653c8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33615) UID: 0 2024-09-14T06:32:51.297 INFO:tasks.ceph.osd.1.smithi094.stderr:2024-09-14T06:32:51.296+0000 7f328e47a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33641) UID: 0 2024-09-14T06:32:51.332 INFO:teuthology.orchestra.run.smithi094.stdout:ERROR: (22) Invalid argument 2024-09-14T06:32:51.333 INFO:teuthology.orchestra.run.smithi094.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:32:51.342 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-14T06:32:51.397 INFO:tasks.ceph.osd.5.smithi138.stderr:2024-09-14T06:32:51.397+0000 7fe13746f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33618) UID: 0 2024-09-14T06:32:51.485 INFO:teuthology.orchestra.run.smithi094.stdout:ERROR: (22) Invalid argument 2024-09-14T06:32:51.485 INFO:teuthology.orchestra.run.smithi094.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:32:51.494 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-14T06:32:51.498 INFO:tasks.ceph.osd.7.smithi138.stderr:2024-09-14T06:32:51.497+0000 7f3c6a21d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33623) UID: 0 2024-09-14T06:32:51.596 INFO:teuthology.orchestra.run.smithi094.stdout:ERROR: (22) Invalid argument 2024-09-14T06:32:51.596 INFO:teuthology.orchestra.run.smithi094.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:32:51.599 INFO:tasks.ceph.osd.3.smithi094.stderr:2024-09-14T06:32:51.598+0000 7feeeafe8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33655) UID: 0 2024-09-14T06:32:51.605 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-14T06:32:51.615 INFO:teuthology.orchestra.run.smithi094.stderr:nodeep-scrub is set 2024-09-14T06:32:51.702 INFO:tasks.ceph.osd.4.smithi138.stderr:2024-09-14T06:32:51.700+0000 7f6c653c8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33615) UID: 0 2024-09-14T06:32:51.748 INFO:teuthology.orchestra.run.smithi094.stdout:ERROR: (22) Invalid argument 2024-09-14T06:32:51.748 INFO:teuthology.orchestra.run.smithi094.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:32:51.761 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-14T06:32:51.803 INFO:tasks.ceph.osd.4.smithi138.stderr:2024-09-14T06:32:51.801+0000 7f6c653c8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33615) UID: 0 2024-09-14T06:32:51.876 INFO:teuthology.orchestra.run.smithi094.stdout:ERROR: (22) Invalid argument 2024-09-14T06:32:51.876 INFO:teuthology.orchestra.run.smithi094.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:32:51.885 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-14T06:32:51.929 INFO:tasks.ceph.osd.2.smithi094.stderr:2024-09-14T06:32:51.928+0000 7faf3269b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33649) UID: 0 2024-09-14T06:32:51.989 INFO:teuthology.orchestra.run.smithi094.stdout:ERROR: (22) Invalid argument 2024-09-14T06:32:51.989 INFO:teuthology.orchestra.run.smithi094.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:32:51.998 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-14T06:32:52.039 INFO:tasks.ceph.osd.0.smithi094.stderr:2024-09-14T06:32:52.038+0000 7f07c9333640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33636) UID: 0 2024-09-14T06:32:52.094 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-14T06:32:52.106 INFO:tasks.ceph.osd.6.smithi138.stderr:2024-09-14T06:32:52.105+0000 7f8cd8849640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33620) UID: 0 2024-09-14T06:32:52.143 INFO:teuthology.orchestra.run.smithi094.stdout:ERROR: (22) Invalid argument 2024-09-14T06:32:52.143 INFO:teuthology.orchestra.run.smithi094.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:32:52.152 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-14T06:32:52.207 INFO:tasks.ceph.osd.0.smithi094.stderr:2024-09-14T06:32:52.206+0000 7f07c9333640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33636) UID: 0 2024-09-14T06:32:52.233 INFO:teuthology.orchestra.run.smithi094.stdout:osd.0: {} 2024-09-14T06:32:52.233 INFO:teuthology.orchestra.run.smithi094.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-14T06:32:52.265 INFO:teuthology.orchestra.run.smithi094.stdout:osd.1: {} 2024-09-14T06:32:52.265 INFO:teuthology.orchestra.run.smithi094.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-14T06:32:52.297 INFO:teuthology.orchestra.run.smithi094.stdout:ERROR: (22) Invalid argument 2024-09-14T06:32:52.297 INFO:teuthology.orchestra.run.smithi094.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:32:52.297 INFO:teuthology.orchestra.run.smithi094.stdout:osd.2: {} 2024-09-14T06:32:52.298 INFO:teuthology.orchestra.run.smithi094.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-14T06:32:52.308 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-14T06:32:52.309 INFO:tasks.ceph.osd.7.smithi138.stderr:2024-09-14T06:32:52.307+0000 7f3c6a21d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33623) UID: 0 2024-09-14T06:32:52.318 INFO:teuthology.orchestra.run.smithi094.stdout:osd.3: {} 2024-09-14T06:32:52.318 INFO:teuthology.orchestra.run.smithi094.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-14T06:32:52.335 INFO:teuthology.orchestra.run.smithi094.stdout:osd.4: {} 2024-09-14T06:32:52.335 INFO:teuthology.orchestra.run.smithi094.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-14T06:32:52.352 INFO:teuthology.orchestra.run.smithi094.stdout:osd.5: {} 2024-09-14T06:32:52.352 INFO:teuthology.orchestra.run.smithi094.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-14T06:32:52.369 INFO:teuthology.orchestra.run.smithi094.stdout:osd.6: {} 2024-09-14T06:32:52.369 INFO:teuthology.orchestra.run.smithi094.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-14T06:32:52.385 INFO:teuthology.orchestra.run.smithi094.stdout:osd.7: {} 2024-09-14T06:32:52.385 INFO:teuthology.orchestra.run.smithi094.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-14T06:32:52.409 INFO:tasks.ceph.osd.5.smithi138.stderr:2024-09-14T06:32:52.408+0000 7fe13746f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33618) UID: 0 2024-09-14T06:32:52.426 INFO:teuthology.orchestra.run.smithi138.stdout:{ 2024-09-14T06:32:52.426 INFO:teuthology.orchestra.run.smithi138.stdout: "ops": [], 2024-09-14T06:32:52.426 INFO:teuthology.orchestra.run.smithi138.stdout: "num_ops": 0 2024-09-14T06:32:52.426 INFO:teuthology.orchestra.run.smithi138.stdout:} 2024-09-14T06:32:52.435 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-14T06:32:52.510 INFO:tasks.ceph.osd.1.smithi094.stderr:2024-09-14T06:32:52.509+0000 7f328e47a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33641) UID: 0 2024-09-14T06:32:52.579 INFO:teuthology.orchestra.run.smithi138.stdout:{ 2024-09-14T06:32:52.579 INFO:teuthology.orchestra.run.smithi138.stdout: "ops": [], 2024-09-14T06:32:52.580 INFO:teuthology.orchestra.run.smithi138.stdout: "complaint_time": 30, 2024-09-14T06:32:52.580 INFO:teuthology.orchestra.run.smithi138.stdout: "num_blocked_ops": 0 2024-09-14T06:32:52.580 INFO:teuthology.orchestra.run.smithi138.stdout:} 2024-09-14T06:32:52.588 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-14T06:32:52.632 INFO:tasks.ceph.osd.4.smithi138.stderr:2024-09-14T06:32:52.631+0000 7f6c653c8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33615) UID: 0 2024-09-14T06:32:52.640 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 4, 5, 7, 3] out_osds: [6] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-14T06:32:52.641 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-14T06:32:52.641 INFO:tasks.thrashosds.thrasher:thrash_pg_upmap_items 2024-09-14T06:32:52.641 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump -f json-pretty 2024-09-14T06:32:52.698 INFO:teuthology.orchestra.run.smithi138.stdout:{ 2024-09-14T06:32:52.698 INFO:teuthology.orchestra.run.smithi138.stdout: "size": 20, 2024-09-14T06:32:52.698 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 600, 2024-09-14T06:32:52.698 INFO:teuthology.orchestra.run.smithi138.stdout: "ops": [ 2024-09-14T06:32:52.698 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:52.698 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:19328 3.3 3:c34e5e51:::rbd_data.112d92027590.0000000000000024:f [read 1926144~1024 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:32:52.698 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:27.059768+0000", 2024-09-14T06:32:52.698 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 205.63657279500001, 2024-09-14T06:32:52.699 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046369147999999999, 2024-09-14T06:32:52.699 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:32:52.699 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:32:52.699 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:32:52.699 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:32:52.699 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:32:52.699 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:52.699 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 19328 2024-09-14T06:32:52.699 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:52.699 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:32:52.699 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:52.699 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:32:52.699 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.059768+0000", 2024-09-14T06:32:52.699 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:52.700 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:52.700 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:52.700 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:32:52.700 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.059768+0000", 2024-09-14T06:32:52.700 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:52.700 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:52.700 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:52.700 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:32:52.700 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.059874+0000", 2024-09-14T06:32:52.700 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00010551400000000001 2024-09-14T06:32:52.700 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:52.700 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:52.700 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:32:52.700 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.059877+0000", 2024-09-14T06:32:52.701 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.8279999999999999e-06 2024-09-14T06:32:52.701 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:52.701 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:52.701 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:32:52.701 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.059884+0000", 2024-09-14T06:32:52.701 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.6389999999999992e-06 2024-09-14T06:32:52.701 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:52.701 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:52.701 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:32:52.701 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.059890+0000", 2024-09-14T06:32:52.701 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.49e-06 2024-09-14T06:32:52.701 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:52.701 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:52.702 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:32:52.702 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.105804+0000", 2024-09-14T06:32:52.702 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.045914242000000001 2024-09-14T06:32:52.702 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:52.702 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:52.702 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:32:52.702 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.105886+0000", 2024-09-14T06:32:52.702 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.2305999999999994e-05 2024-09-14T06:32:52.702 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:52.702 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:52.702 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:32:52.702 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.106138+0000", 2024-09-14T06:32:52.702 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.000251129 2024-09-14T06:32:52.703 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:52.703 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:32:52.703 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:52.703 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:52.703 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:52.703 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:19329 3.3 3:c34e5e51:::rbd_data.112d92027590.0000000000000024:f [read 1936384~1024 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:32:52.703 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:27.059899+0000", 2024-09-14T06:32:52.703 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 205.63644227200001, 2024-09-14T06:32:52.703 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046572122000000001, 2024-09-14T06:32:52.703 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:32:52.703 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:32:52.703 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:32:52.703 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:32:52.703 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:32:52.704 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:52.704 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 19329 2024-09-14T06:32:52.704 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:52.704 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:32:52.704 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:52.704 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:32:52.704 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.059899+0000", 2024-09-14T06:32:52.704 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:52.704 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:52.704 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:52.704 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:32:52.704 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.059899+0000", 2024-09-14T06:32:52.704 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:52.704 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:52.705 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:52.705 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:32:52.705 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062501+0000", 2024-09-14T06:32:52.705 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.0026016419999999999 2024-09-14T06:32:52.705 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:52.705 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:52.705 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:32:52.706 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062514+0000", 2024-09-14T06:32:52.706 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.2947e-05 2024-09-14T06:32:52.706 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:52.706 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:52.706 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:32:52.706 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062523+0000", 2024-09-14T06:32:52.706 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.5519999999999993e-06 2024-09-14T06:32:52.706 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:52.706 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:52.706 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:32:52.706 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062534+0000", 2024-09-14T06:32:52.706 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.1416e-05 2024-09-14T06:32:52.707 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:52.707 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:52.707 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:32:52.707 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.106179+0000", 2024-09-14T06:32:52.707 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.043644049999999997 2024-09-14T06:32:52.707 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:52.707 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:52.707 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:32:52.707 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.106262+0000", 2024-09-14T06:32:52.707 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.3178999999999998e-05 2024-09-14T06:32:52.707 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:52.707 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:52.707 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:32:52.707 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.106471+0000", 2024-09-14T06:32:52.708 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00020933599999999999 2024-09-14T06:32:52.708 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:52.708 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:32:52.708 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:52.708 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:52.708 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:52.708 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:19351 3.3 3:c34e5e51:::rbd_data.112d92027590.0000000000000024:f [read 1159168~1024 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:32:52.708 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:27.062766+0000", 2024-09-14T06:32:52.708 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 205.63357504699999, 2024-09-14T06:32:52.708 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046099454999999998, 2024-09-14T06:32:52.708 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:32:52.708 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:32:52.708 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:32:52.708 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:32:52.709 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:32:52.709 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:52.709 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 19351 2024-09-14T06:32:52.709 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:52.709 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:32:52.709 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:52.709 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:32:52.709 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062766+0000", 2024-09-14T06:32:52.709 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:52.709 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:52.709 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:52.709 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:32:52.709 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062766+0000", 2024-09-14T06:32:52.709 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:52.710 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:52.710 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:52.710 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:32:52.710 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062766+0000", 2024-09-14T06:32:52.710 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.6e-07 2024-09-14T06:32:52.710 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:52.710 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:52.710 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:32:52.710 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062767+0000", 2024-09-14T06:32:52.710 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.0070000000000001e-06 2024-09-14T06:32:52.710 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:52.710 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:52.710 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:32:52.710 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062785+0000", 2024-09-14T06:32:52.711 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.7884999999999999e-05 2024-09-14T06:32:52.711 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:52.711 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:52.711 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:32:52.711 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062790+0000", 2024-09-14T06:32:52.711 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.0900000000000004e-06 2024-09-14T06:32:52.711 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:52.711 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:52.711 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:32:52.711 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.108735+0000", 2024-09-14T06:32:52.711 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.045944411999999997 2024-09-14T06:32:52.711 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:52.711 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:52.711 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:32:52.712 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.108809+0000", 2024-09-14T06:32:52.712 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.4157000000000002e-05 2024-09-14T06:32:52.712 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:52.712 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:52.712 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:32:52.712 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.108866+0000", 2024-09-14T06:32:52.712 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.6644000000000002e-05 2024-09-14T06:32:52.712 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:52.712 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:32:52.712 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:52.712 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:52.712 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:52.713 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:19352 3.3 3:c34e5e51:::rbd_data.112d92027590.0000000000000024:f [read 1169408~1024 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:32:52.713 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:27.062798+0000", 2024-09-14T06:32:52.713 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 205.63354363600001, 2024-09-14T06:32:52.713 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046255900000000003, 2024-09-14T06:32:52.713 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:32:52.713 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:32:52.713 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:32:52.713 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:32:52.713 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:32:52.713 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:52.713 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 19352 2024-09-14T06:32:52.713 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:52.713 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:32:52.714 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:52.714 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:32:52.714 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062798+0000", 2024-09-14T06:32:52.714 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:52.714 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:52.714 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:52.714 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:32:52.714 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062798+0000", 2024-09-14T06:32:52.714 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:52.714 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:52.714 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:52.714 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:32:52.714 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062798+0000", 2024-09-14T06:32:52.714 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.37e-07 2024-09-14T06:32:52.715 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:52.715 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:52.715 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:32:52.715 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062800+0000", 2024-09-14T06:32:52.715 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.0269999999999998e-06 2024-09-14T06:32:52.715 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:52.715 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:52.715 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:32:52.715 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062806+0000", 2024-09-14T06:32:52.715 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.6690000000000003e-06 2024-09-14T06:32:52.715 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:52.715 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:52.716 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:32:52.716 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062810+0000", 2024-09-14T06:32:52.716 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.3449999999999999e-06 2024-09-14T06:32:52.716 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:52.716 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:52.716 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:32:52.716 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.108892+0000", 2024-09-14T06:32:52.716 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046081715000000002 2024-09-14T06:32:52.716 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:52.716 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:52.716 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:32:52.716 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.108940+0000", 2024-09-14T06:32:52.716 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.7815999999999998e-05 2024-09-14T06:32:52.717 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:52.717 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:52.717 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:32:52.717 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.109053+0000", 2024-09-14T06:32:52.717 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.000113791 2024-09-14T06:32:52.717 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:52.717 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:32:52.717 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:52.717 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:52.717 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:52.717 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:19353 3.3 3:c34e5e51:::rbd_data.112d92027590.0000000000000024:f [read 1179648~1024 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:32:52.717 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:27.062816+0000", 2024-09-14T06:32:52.718 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 205.633525117, 2024-09-14T06:32:52.718 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046481256999999998, 2024-09-14T06:32:52.718 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:32:52.718 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:32:52.718 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:32:52.718 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:32:52.718 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:32:52.718 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:52.718 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 19353 2024-09-14T06:32:52.718 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:52.718 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:32:52.719 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:52.719 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:32:52.719 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062816+0000", 2024-09-14T06:32:52.719 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:52.719 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:52.719 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:52.719 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:32:52.719 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062816+0000", 2024-09-14T06:32:52.719 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:52.719 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:52.719 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:52.719 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:32:52.719 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062816+0000", 2024-09-14T06:32:52.719 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.2899999999999999e-07 2024-09-14T06:32:52.720 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:52.720 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:52.720 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:32:52.720 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062818+0000", 2024-09-14T06:32:52.720 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.699e-06 2024-09-14T06:32:52.720 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:52.720 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:52.720 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:32:52.720 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062825+0000", 2024-09-14T06:32:52.720 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.878e-06 2024-09-14T06:32:52.720 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:52.720 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:52.720 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:32:52.721 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062830+0000", 2024-09-14T06:32:52.721 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.5410000000000002e-06 2024-09-14T06:32:52.721 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:52.721 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:52.721 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:32:52.721 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.109096+0000", 2024-09-14T06:32:52.721 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046266517 2024-09-14T06:32:52.721 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:52.721 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:52.721 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:32:52.721 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.109190+0000", 2024-09-14T06:32:52.721 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.4000999999999997e-05 2024-09-14T06:32:52.722 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:52.722 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:52.722 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:32:52.722 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.109297+0000", 2024-09-14T06:32:52.722 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.000107292 2024-09-14T06:32:52.722 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:52.722 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:32:52.722 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:52.722 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:52.722 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:52.722 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:19354 3.3 3:c34e5e51:::rbd_data.112d92027590.0000000000000024:f [read 1189888~1024 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:32:52.722 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:27.062842+0000", 2024-09-14T06:32:52.722 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 205.63349966199999, 2024-09-14T06:32:52.722 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046594257, 2024-09-14T06:32:52.723 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:32:52.723 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:32:52.723 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:32:52.723 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:32:52.723 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:32:52.723 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:52.723 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 19354 2024-09-14T06:32:52.723 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:52.723 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:32:52.723 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:52.723 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:32:52.723 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062842+0000", 2024-09-14T06:32:52.724 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:52.724 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:52.724 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:52.724 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:32:52.724 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062842+0000", 2024-09-14T06:32:52.724 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:52.724 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:52.724 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:52.724 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:32:52.724 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062842+0000", 2024-09-14T06:32:52.724 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.7399999999999999e-07 2024-09-14T06:32:52.724 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:52.724 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:52.725 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:32:52.725 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062859+0000", 2024-09-14T06:32:52.725 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.7010000000000001e-05 2024-09-14T06:32:52.725 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:52.725 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:52.725 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:32:52.725 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062866+0000", 2024-09-14T06:32:52.725 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.9990000000000002e-06 2024-09-14T06:32:52.725 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:52.725 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:52.725 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:32:52.725 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062873+0000", 2024-09-14T06:32:52.725 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.8569999999999997e-06 2024-09-14T06:32:52.726 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:52.726 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:52.726 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:32:52.726 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.109335+0000", 2024-09-14T06:32:52.726 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046461943999999998 2024-09-14T06:32:52.726 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:52.726 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:52.726 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:32:52.726 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.109377+0000", 2024-09-14T06:32:52.726 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.2558999999999997e-05 2024-09-14T06:32:52.726 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:52.726 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:52.726 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:32:52.726 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.109436+0000", 2024-09-14T06:32:52.726 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.8514000000000003e-05 2024-09-14T06:32:52.727 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:52.727 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:32:52.727 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:52.727 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:52.727 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:52.727 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:19355 3.3 3:c34e5e51:::rbd_data.112d92027590.0000000000000024:f [read 1200128~1024 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:32:52.727 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:27.062884+0000", 2024-09-14T06:32:52.727 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 205.633456989, 2024-09-14T06:32:52.727 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046681650999999998, 2024-09-14T06:32:52.727 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:32:52.727 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:32:52.727 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:32:52.727 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:32:52.728 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:32:52.728 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:52.728 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 19355 2024-09-14T06:32:52.728 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:52.728 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:32:52.728 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:52.728 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:32:52.728 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062884+0000", 2024-09-14T06:32:52.728 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:52.728 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:52.728 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:52.728 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:32:52.728 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062884+0000", 2024-09-14T06:32:52.728 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:52.729 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:52.729 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:52.729 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:32:52.729 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062885+0000", 2024-09-14T06:32:52.729 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.8000000000000001e-07 2024-09-14T06:32:52.729 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:52.729 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:52.729 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:32:52.729 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062886+0000", 2024-09-14T06:32:52.729 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.8920000000000001e-06 2024-09-14T06:32:52.729 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:52.729 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:52.729 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:32:52.730 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062892+0000", 2024-09-14T06:32:52.730 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.84e-06 2024-09-14T06:32:52.730 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:52.730 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:52.730 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:32:52.730 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062899+0000", 2024-09-14T06:32:52.730 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.5339999999999996e-06 2024-09-14T06:32:52.730 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:52.730 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:52.730 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:32:52.730 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.109466+0000", 2024-09-14T06:32:52.730 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046566919999999998 2024-09-14T06:32:52.730 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:52.730 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:52.731 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:32:52.731 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.109510+0000", 2024-09-14T06:32:52.731 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.4153000000000002e-05 2024-09-14T06:32:52.731 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:52.731 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:52.731 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:32:52.731 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.109566+0000", 2024-09-14T06:32:52.731 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.5931999999999999e-05 2024-09-14T06:32:52.731 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:52.731 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:32:52.731 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:52.731 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:52.731 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:52.732 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:19356 3.3 3:c34e5e51:::rbd_data.112d92027590.0000000000000024:f [read 1210368~1024 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:32:52.732 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:27.062908+0000", 2024-09-14T06:32:52.732 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 205.63343355200001, 2024-09-14T06:32:52.732 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046807498000000003, 2024-09-14T06:32:52.732 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:32:52.732 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:32:52.732 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:32:52.732 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:32:52.732 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:32:52.732 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:52.732 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 19356 2024-09-14T06:32:52.732 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:52.732 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:32:52.732 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:52.733 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:32:52.733 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062908+0000", 2024-09-14T06:32:52.733 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:52.733 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:52.733 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:52.733 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:32:52.733 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062908+0000", 2024-09-14T06:32:52.733 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:52.733 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:52.733 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:52.733 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:32:52.733 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062908+0000", 2024-09-14T06:32:52.733 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.5699999999999998e-07 2024-09-14T06:32:52.734 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:52.734 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:52.734 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:32:52.734 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062910+0000", 2024-09-14T06:32:52.734 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.7710000000000001e-06 2024-09-14T06:32:52.734 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:52.734 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:52.734 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:32:52.734 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062920+0000", 2024-09-14T06:32:52.734 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.0654e-05 2024-09-14T06:32:52.734 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:52.734 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:52.734 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:32:52.734 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062931+0000", 2024-09-14T06:32:52.735 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.075e-05 2024-09-14T06:32:52.735 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:52.735 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:52.735 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:32:52.735 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.109595+0000", 2024-09-14T06:32:52.735 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046663713000000002 2024-09-14T06:32:52.735 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:52.735 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:52.735 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:32:52.735 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.109632+0000", 2024-09-14T06:32:52.735 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.7505000000000001e-05 2024-09-14T06:32:52.735 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:52.735 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:52.736 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:32:52.736 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.109715+0000", 2024-09-14T06:32:52.736 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.2748000000000001e-05 2024-09-14T06:32:52.736 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:52.736 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:32:52.736 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:52.736 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:52.736 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:52.736 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:19357 3.3 3:c34e5e51:::rbd_data.112d92027590.0000000000000024:f [read 1220608~1024 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:32:52.736 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:27.062939+0000", 2024-09-14T06:32:52.736 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 205.633402104, 2024-09-14T06:32:52.736 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046920508, 2024-09-14T06:32:52.736 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:32:52.736 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:32:52.737 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:32:52.737 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:32:52.737 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:32:52.737 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:52.737 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 19357 2024-09-14T06:32:52.737 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:52.737 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:32:52.737 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:52.737 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:32:52.737 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062939+0000", 2024-09-14T06:32:52.737 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:52.737 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:52.737 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:52.738 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:32:52.738 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062939+0000", 2024-09-14T06:32:52.738 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:52.738 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:52.738 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:52.738 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:32:52.738 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062940+0000", 2024-09-14T06:32:52.738 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.32e-07 2024-09-14T06:32:52.738 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:52.738 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:52.738 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:32:52.738 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062942+0000", 2024-09-14T06:32:52.738 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.1919999999999999e-06 2024-09-14T06:32:52.739 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:52.739 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:52.739 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:32:52.739 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062948+0000", 2024-09-14T06:32:52.739 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.1149999999999999e-06 2024-09-14T06:32:52.739 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:52.739 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:52.739 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:32:52.739 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062960+0000", 2024-09-14T06:32:52.739 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.1895e-05 2024-09-14T06:32:52.739 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:52.739 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:52.739 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:32:52.739 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.109704+0000", 2024-09-14T06:32:52.739 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046744584999999998 2024-09-14T06:32:52.740 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:52.740 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:52.740 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:32:52.740 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.109777+0000", 2024-09-14T06:32:52.740 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.2282999999999999e-05 2024-09-14T06:32:52.740 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:52.740 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:52.740 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:32:52.740 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.109860+0000", 2024-09-14T06:32:52.740 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.3005999999999998e-05 2024-09-14T06:32:52.740 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:52.740 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:32:52.740 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:52.740 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:52.741 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:52.741 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:19358 3.3 3:c34e5e51:::rbd_data.112d92027590.0000000000000024:f [read 1230848~1024 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:32:52.741 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:27.062967+0000", 2024-09-14T06:32:52.741 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 205.63337447399999, 2024-09-14T06:32:52.741 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047144471, 2024-09-14T06:32:52.741 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:32:52.741 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:32:52.741 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:32:52.741 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:32:52.741 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:32:52.741 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:52.741 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 19358 2024-09-14T06:32:52.741 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:52.741 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:32:52.742 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:52.742 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:32:52.742 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062967+0000", 2024-09-14T06:32:52.742 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:52.742 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:52.742 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:52.742 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:32:52.742 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062967+0000", 2024-09-14T06:32:52.742 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:52.742 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:52.742 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:52.742 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:32:52.742 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062967+0000", 2024-09-14T06:32:52.742 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.89e-07 2024-09-14T06:32:52.743 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:52.743 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:52.743 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:32:52.743 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062969+0000", 2024-09-14T06:32:52.743 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.9929999999999998e-06 2024-09-14T06:32:52.743 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:52.743 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:52.743 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:32:52.743 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062975+0000", 2024-09-14T06:32:52.743 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.9030000000000001e-06 2024-09-14T06:32:52.744 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:52.744 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:52.744 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:32:52.744 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062979+0000", 2024-09-14T06:32:52.744 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.2250000000000002e-06 2024-09-14T06:32:52.744 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:52.744 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:52.744 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:32:52.744 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.109934+0000", 2024-09-14T06:32:52.744 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046954410000000002 2024-09-14T06:32:52.744 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:52.744 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:52.744 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:32:52.745 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.110007+0000", 2024-09-14T06:32:52.745 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.3829000000000001e-05 2024-09-14T06:32:52.745 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:52.745 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:52.745 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:32:52.745 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.110111+0000", 2024-09-14T06:32:52.745 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00010372199999999999 2024-09-14T06:32:52.745 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:52.745 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:32:52.745 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:52.745 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:52.745 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:52.745 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:19359 3.3 3:c34e5e51:::rbd_data.112d92027590.0000000000000024:f [read 1241088~1024 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:32:52.745 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:27.062987+0000", 2024-09-14T06:32:52.746 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 205.633354384, 2024-09-14T06:32:52.746 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047297182, 2024-09-14T06:32:52.746 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:32:52.746 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:32:52.746 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:32:52.746 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:32:52.746 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:32:52.746 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:52.746 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 19359 2024-09-14T06:32:52.746 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:52.746 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:32:52.746 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:52.746 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:32:52.746 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062987+0000", 2024-09-14T06:32:52.747 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:52.747 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:52.747 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:52.747 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:32:52.747 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062987+0000", 2024-09-14T06:32:52.747 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:52.747 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:52.747 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:52.747 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:32:52.747 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062987+0000", 2024-09-14T06:32:52.747 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.2500000000000001e-07 2024-09-14T06:32:52.747 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:52.747 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:52.747 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:32:52.748 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062989+0000", 2024-09-14T06:32:52.748 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.6080000000000001e-06 2024-09-14T06:32:52.748 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:52.748 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:52.748 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:32:52.748 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062994+0000", 2024-09-14T06:32:52.748 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.5520000000000004e-06 2024-09-14T06:32:52.748 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:52.748 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:52.748 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:32:52.748 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062999+0000", 2024-09-14T06:32:52.748 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.3930000000000001e-06 2024-09-14T06:32:52.748 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:52.749 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:52.749 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:32:52.749 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.110142+0000", 2024-09-14T06:32:52.749 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047143154 2024-09-14T06:32:52.749 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:52.749 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:52.749 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:32:52.749 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.110219+0000", 2024-09-14T06:32:52.749 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.6737999999999996e-05 2024-09-14T06:32:52.749 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:52.749 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:52.749 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:32:52.749 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.110284+0000", 2024-09-14T06:32:52.750 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.5412000000000003e-05 2024-09-14T06:32:52.750 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:52.750 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:32:52.750 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:52.750 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:52.750 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:52.750 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:19360 3.3 3:c34e5e51:::rbd_data.112d92027590.0000000000000024:f [read 1251328~1024 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:32:52.750 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:27.063017+0000", 2024-09-14T06:32:52.750 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 205.63332392300001, 2024-09-14T06:32:52.750 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047453823999999999, 2024-09-14T06:32:52.750 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:32:52.750 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:32:52.750 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:32:52.750 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:32:52.751 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:32:52.751 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:52.751 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 19360 2024-09-14T06:32:52.751 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:52.751 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:32:52.751 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:52.751 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:32:52.751 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.063017+0000", 2024-09-14T06:32:52.751 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:52.751 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:52.751 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:52.752 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:32:52.752 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.063017+0000", 2024-09-14T06:32:52.752 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:52.752 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:52.752 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:52.752 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:32:52.752 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.064977+0000", 2024-09-14T06:32:52.752 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.0019597080000000001 2024-09-14T06:32:52.752 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:52.752 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:52.752 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:32:52.752 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.064987+0000", 2024-09-14T06:32:52.752 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.9620000000000005e-06 2024-09-14T06:32:52.752 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:52.753 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:52.753 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:32:52.753 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065000+0000", 2024-09-14T06:32:52.753 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.3173000000000001e-05 2024-09-14T06:32:52.753 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:52.753 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:52.753 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:32:52.753 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065005+0000", 2024-09-14T06:32:52.753 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.9960000000000001e-06 2024-09-14T06:32:52.753 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:52.753 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:52.753 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:32:52.753 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.110314+0000", 2024-09-14T06:32:52.754 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.04530899 2024-09-14T06:32:52.754 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:52.754 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:52.754 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:32:52.754 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.110398+0000", 2024-09-14T06:32:52.754 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.3539000000000001e-05 2024-09-14T06:32:52.754 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:52.754 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:52.754 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:32:52.754 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.110471+0000", 2024-09-14T06:32:52.754 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.3455999999999996e-05 2024-09-14T06:32:52.754 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:52.754 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:32:52.754 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:52.754 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:52.755 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:52.755 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:19365 3.3 3:c34e5e51:::rbd_data.112d92027590.0000000000000024:f [read 1302528~1024 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:32:52.755 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:27.065058+0000", 2024-09-14T06:32:52.755 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 205.63128325299999, 2024-09-14T06:32:52.755 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046105526000000001, 2024-09-14T06:32:52.755 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:32:52.755 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:32:52.755 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:32:52.755 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:32:52.755 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:32:52.755 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:52.755 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 19365 2024-09-14T06:32:52.755 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:52.755 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:32:52.756 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:52.756 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:32:52.756 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065058+0000", 2024-09-14T06:32:52.756 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:52.756 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:52.756 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:52.756 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:32:52.756 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065058+0000", 2024-09-14T06:32:52.756 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:52.756 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:52.756 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:52.756 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:32:52.756 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065058+0000", 2024-09-14T06:32:52.757 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.1899999999999999e-07 2024-09-14T06:32:52.757 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:52.757 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:52.757 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:32:52.757 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065059+0000", 2024-09-14T06:32:52.757 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.1299999999999998e-07 2024-09-14T06:32:52.757 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:52.757 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:52.757 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:32:52.757 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065063+0000", 2024-09-14T06:32:52.757 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.1099999999999996e-06 2024-09-14T06:32:52.757 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:52.757 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:52.758 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:32:52.758 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065070+0000", 2024-09-14T06:32:52.758 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.973e-06 2024-09-14T06:32:52.758 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:52.758 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:52.758 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:32:52.758 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.111014+0000", 2024-09-14T06:32:52.758 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.045943373000000003 2024-09-14T06:32:52.758 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:52.758 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:52.758 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:32:52.759 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.111073+0000", 2024-09-14T06:32:52.759 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.8987999999999998e-05 2024-09-14T06:32:52.759 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:52.759 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:52.759 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:32:52.759 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.111163+0000", 2024-09-14T06:32:52.759 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.0950000000000002e-05 2024-09-14T06:32:52.759 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:52.759 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:32:52.759 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:52.759 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:52.759 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:52.759 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:19366 3.3 3:c34e5e51:::rbd_data.112d92027590.0000000000000024:f [read 1312768~1024 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:32:52.759 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:27.065074+0000", 2024-09-14T06:32:52.760 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 205.63126740300001, 2024-09-14T06:32:52.760 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046289858000000003, 2024-09-14T06:32:52.760 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:32:52.760 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:32:52.760 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:32:52.760 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:32:52.760 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:32:52.760 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:52.760 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 19366 2024-09-14T06:32:52.760 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:52.760 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:32:52.760 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:52.761 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:32:52.761 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065074+0000", 2024-09-14T06:32:52.761 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:52.761 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:52.761 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:52.761 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:32:52.761 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065074+0000", 2024-09-14T06:32:52.761 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:52.761 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:52.761 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:52.761 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:32:52.761 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065074+0000", 2024-09-14T06:32:52.761 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.3900000000000001e-07 2024-09-14T06:32:52.762 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:52.762 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:52.762 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:32:52.762 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065078+0000", 2024-09-14T06:32:52.762 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.3229999999999999e-06 2024-09-14T06:32:52.762 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:52.762 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:52.762 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:32:52.762 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065082+0000", 2024-09-14T06:32:52.762 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.8299999999999998e-06 2024-09-14T06:32:52.762 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:52.762 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:52.762 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:32:52.762 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065085+0000", 2024-09-14T06:32:52.763 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.8660000000000002e-06 2024-09-14T06:32:52.763 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:52.763 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:52.763 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:32:52.763 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.111270+0000", 2024-09-14T06:32:52.763 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046185035999999999 2024-09-14T06:32:52.763 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:52.763 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:52.763 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:32:52.763 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.111311+0000", 2024-09-14T06:32:52.763 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.0935e-05 2024-09-14T06:32:52.763 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:52.763 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:52.763 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:32:52.764 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.111364+0000", 2024-09-14T06:32:52.764 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.2629e-05 2024-09-14T06:32:52.764 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:52.764 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:32:52.764 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:52.764 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:52.764 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:52.764 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:19367 3.3 3:c34e5e51:::rbd_data.112d92027590.0000000000000024:f [read 1323008~1024 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:32:52.764 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:27.065115+0000", 2024-09-14T06:32:52.764 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 205.63122584199999, 2024-09-14T06:32:52.764 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046407003000000002, 2024-09-14T06:32:52.764 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:32:52.764 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:32:52.765 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:32:52.765 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:32:52.765 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:32:52.765 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:52.765 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 19367 2024-09-14T06:32:52.765 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:52.765 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:32:52.765 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:52.765 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:32:52.765 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065115+0000", 2024-09-14T06:32:52.765 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:52.765 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:52.765 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:52.766 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:32:52.766 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065115+0000", 2024-09-14T06:32:52.766 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:52.766 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:52.766 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:52.766 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:32:52.766 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065116+0000", 2024-09-14T06:32:52.766 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.4999999999999998e-07 2024-09-14T06:32:52.766 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:52.766 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:52.766 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:32:52.766 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065117+0000", 2024-09-14T06:32:52.766 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.189e-06 2024-09-14T06:32:52.766 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:52.767 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:52.767 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:32:52.767 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065122+0000", 2024-09-14T06:32:52.767 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.3709999999999999e-06 2024-09-14T06:32:52.767 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:52.767 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:52.767 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:32:52.767 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065125+0000", 2024-09-14T06:32:52.767 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.8940000000000002e-06 2024-09-14T06:32:52.767 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:52.767 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:52.767 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:32:52.768 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.111402+0000", 2024-09-14T06:32:52.768 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046276645999999998 2024-09-14T06:32:52.768 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:52.768 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:52.768 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:32:52.768 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.111445+0000", 2024-09-14T06:32:52.768 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.3288000000000003e-05 2024-09-14T06:32:52.768 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:52.768 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:52.768 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:32:52.768 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.111522+0000", 2024-09-14T06:32:52.768 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.7164999999999997e-05 2024-09-14T06:32:52.768 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:52.769 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:32:52.769 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:52.769 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:52.769 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:52.769 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:19368 3.3 3:c34e5e51:::rbd_data.112d92027590.0000000000000024:f [read 1333248~1024 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:32:52.769 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:27.065130+0000", 2024-09-14T06:32:52.769 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 205.631211552, 2024-09-14T06:32:52.769 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046507275000000001, 2024-09-14T06:32:52.769 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:32:52.769 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:32:52.769 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:32:52.769 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:32:52.769 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:32:52.770 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:52.770 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 19368 2024-09-14T06:32:52.770 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:52.770 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:32:52.770 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:52.770 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:32:52.770 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065130+0000", 2024-09-14T06:32:52.770 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:52.770 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:52.770 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:52.770 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:32:52.770 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065130+0000", 2024-09-14T06:32:52.770 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:52.770 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:52.771 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:52.771 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:32:52.771 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065130+0000", 2024-09-14T06:32:52.771 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.0800000000000001e-07 2024-09-14T06:32:52.771 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:52.771 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:52.771 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:32:52.771 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065131+0000", 2024-09-14T06:32:52.771 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.8700000000000004e-07 2024-09-14T06:32:52.771 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:52.771 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:52.771 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:32:52.771 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065134+0000", 2024-09-14T06:32:52.771 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.1039999999999998e-06 2024-09-14T06:32:52.772 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:52.772 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:52.772 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:32:52.772 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065136+0000", 2024-09-14T06:32:52.772 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.3949999999999999e-06 2024-09-14T06:32:52.772 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:52.772 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:52.772 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:32:52.772 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.111541+0000", 2024-09-14T06:32:52.772 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046404206000000003 2024-09-14T06:32:52.772 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:52.772 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:52.773 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:32:52.773 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.111582+0000", 2024-09-14T06:32:52.773 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.1051999999999999e-05 2024-09-14T06:32:52.773 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:52.773 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:52.773 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:32:52.773 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.111637+0000", 2024-09-14T06:32:52.773 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.5322999999999999e-05 2024-09-14T06:32:52.773 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:52.773 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:32:52.773 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:52.773 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:52.773 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:52.773 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:19369 3.3 3:c34e5e51:::rbd_data.112d92027590.0000000000000024:f [read 1343488~1024 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:32:52.774 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:27.065139+0000", 2024-09-14T06:32:52.774 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 205.63120173499999, 2024-09-14T06:32:52.774 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046682169000000003, 2024-09-14T06:32:52.774 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:32:52.774 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:32:52.774 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:32:52.774 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:32:52.774 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:32:52.774 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:52.774 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 19369 2024-09-14T06:32:52.774 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:52.774 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:32:52.774 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:52.775 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:32:52.775 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065139+0000", 2024-09-14T06:32:52.775 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:52.775 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:52.775 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:52.775 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:32:52.775 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065139+0000", 2024-09-14T06:32:52.775 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:52.775 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:52.775 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:52.775 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:32:52.775 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065140+0000", 2024-09-14T06:32:52.776 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.8299999999999998e-07 2024-09-14T06:32:52.776 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:52.776 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:52.776 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:32:52.776 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065141+0000", 2024-09-14T06:32:52.776 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.3900000000000003e-07 2024-09-14T06:32:52.776 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:52.776 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:52.776 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:32:52.776 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065144+0000", 2024-09-14T06:32:52.776 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.7409999999999998e-06 2024-09-14T06:32:52.776 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:52.777 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:52.777 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:32:52.777 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065147+0000", 2024-09-14T06:32:52.777 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.4150000000000002e-06 2024-09-14T06:32:52.777 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:52.777 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:52.777 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:32:52.777 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.111673+0000", 2024-09-14T06:32:52.777 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046526004000000003 2024-09-14T06:32:52.777 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:52.777 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:52.777 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:32:52.777 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.111749+0000", 2024-09-14T06:32:52.777 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.5741999999999994e-05 2024-09-14T06:32:52.778 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:52.778 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:52.778 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:32:52.778 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.111822+0000", 2024-09-14T06:32:52.778 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.3045000000000003e-05 2024-09-14T06:32:52.778 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:52.778 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:32:52.778 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:52.778 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:52.778 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:52.778 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:19370 3.3 3:c34e5e51:::rbd_data.112d92027590.0000000000000024:f [read 1353728~1024 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:32:52.778 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:27.065150+0000", 2024-09-14T06:32:52.778 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 205.63119100599999, 2024-09-14T06:32:52.779 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046897556999999999, 2024-09-14T06:32:52.779 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:32:52.779 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:32:52.779 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:32:52.779 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:32:52.779 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:32:52.779 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:52.779 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 19370 2024-09-14T06:32:52.779 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:52.779 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:32:52.779 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:52.779 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:32:52.779 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065150+0000", 2024-09-14T06:32:52.779 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:52.779 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:52.780 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:52.780 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:32:52.780 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065150+0000", 2024-09-14T06:32:52.780 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:52.780 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:52.780 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:52.780 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:32:52.780 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065150+0000", 2024-09-14T06:32:52.780 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.0200000000000001e-07 2024-09-14T06:32:52.780 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:52.780 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:52.780 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:32:52.780 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065151+0000", 2024-09-14T06:32:52.780 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.1100000000000004e-07 2024-09-14T06:32:52.781 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:52.781 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:52.781 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:32:52.781 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065154+0000", 2024-09-14T06:32:52.781 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.9639999999999999e-06 2024-09-14T06:32:52.781 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:52.781 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:52.781 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:32:52.781 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065157+0000", 2024-09-14T06:32:52.781 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.4260000000000002e-06 2024-09-14T06:32:52.781 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:52.781 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:52.781 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:32:52.781 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.111861+0000", 2024-09-14T06:32:52.782 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046704268 2024-09-14T06:32:52.782 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:52.782 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:52.782 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:32:52.782 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.111939+0000", 2024-09-14T06:32:52.782 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.8231000000000004e-05 2024-09-14T06:32:52.782 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:52.782 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:52.782 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:32:52.782 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.112048+0000", 2024-09-14T06:32:52.782 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00010855500000000001 2024-09-14T06:32:52.782 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:52.782 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:32:52.783 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:52.783 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:52.783 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:52.783 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:19371 3.3 3:c34e5e51:::rbd_data.112d92027590.0000000000000024:f [read 1363968~1024 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:32:52.783 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:27.065161+0000", 2024-09-14T06:32:52.783 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 205.63118035599999, 2024-09-14T06:32:52.783 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047021944000000003, 2024-09-14T06:32:52.783 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:32:52.783 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:32:52.783 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:32:52.783 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:32:52.783 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:32:52.783 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:52.784 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 19371 2024-09-14T06:32:52.784 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:52.784 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:32:52.784 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:52.784 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:32:52.784 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065161+0000", 2024-09-14T06:32:52.784 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:52.784 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:52.784 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:52.784 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:32:52.784 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065161+0000", 2024-09-14T06:32:52.784 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:52.784 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:52.785 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:52.785 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:32:52.785 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065161+0000", 2024-09-14T06:32:52.785 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.0200000000000001e-07 2024-09-14T06:32:52.785 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:52.785 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:52.785 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:32:52.785 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065166+0000", 2024-09-14T06:32:52.785 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.9869999999999999e-06 2024-09-14T06:32:52.785 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:52.785 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:52.785 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:32:52.785 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065169+0000", 2024-09-14T06:32:52.786 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.3799999999999998e-06 2024-09-14T06:32:52.786 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:52.786 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:52.786 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:32:52.786 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065172+0000", 2024-09-14T06:32:52.786 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.4830000000000002e-06 2024-09-14T06:32:52.786 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:52.786 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:52.786 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:32:52.786 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.112071+0000", 2024-09-14T06:32:52.786 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046899529000000002 2024-09-14T06:32:52.787 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:52.787 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:52.787 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:32:52.787 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.112128+0000", 2024-09-14T06:32:52.787 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.6341999999999997e-05 2024-09-14T06:32:52.787 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:52.787 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:52.787 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:32:52.787 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.112183+0000", 2024-09-14T06:32:52.787 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.5021000000000001e-05 2024-09-14T06:32:52.787 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:52.787 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:32:52.787 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:52.788 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:52.788 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:52.788 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:19372 3.3 3:c34e5e51:::rbd_data.112d92027590.0000000000000024:f [read 1374208~1024 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:32:52.788 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:27.065176+0000", 2024-09-14T06:32:52.788 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 205.63116520599999, 2024-09-14T06:32:52.788 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047188889999999997, 2024-09-14T06:32:52.788 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:32:52.788 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:32:52.788 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:32:52.788 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:32:52.788 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:32:52.788 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:52.788 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 19372 2024-09-14T06:32:52.789 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:52.789 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:32:52.789 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:52.789 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:32:52.789 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065176+0000", 2024-09-14T06:32:52.789 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:52.789 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:52.789 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:52.789 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:32:52.789 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065176+0000", 2024-09-14T06:32:52.789 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:52.789 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:52.790 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:52.790 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:32:52.790 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.071117+0000", 2024-09-14T06:32:52.790 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.0059404990000000001 2024-09-14T06:32:52.790 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:52.790 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:52.790 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:32:52.790 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.071120+0000", 2024-09-14T06:32:52.790 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.4340000000000001e-06 2024-09-14T06:32:52.790 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:52.790 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:52.790 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:32:52.790 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.071129+0000", 2024-09-14T06:32:52.790 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.8389999999999996e-06 2024-09-14T06:32:52.790 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:52.791 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:52.791 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:32:52.791 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.071134+0000", 2024-09-14T06:32:52.791 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.412e-06 2024-09-14T06:32:52.791 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:52.791 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:52.791 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:32:52.791 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.112221+0000", 2024-09-14T06:32:52.791 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.041086473999999998 2024-09-14T06:32:52.791 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:52.791 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:52.791 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:32:52.791 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.112299+0000", 2024-09-14T06:32:52.791 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.8746999999999997e-05 2024-09-14T06:32:52.792 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:52.792 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:52.792 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:32:52.792 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.112365+0000", 2024-09-14T06:32:52.792 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.5485000000000001e-05 2024-09-14T06:32:52.792 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:52.792 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:32:52.792 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:52.792 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:52.792 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:32:52.792 INFO:teuthology.orchestra.run.smithi138.stdout:} 2024-09-14T06:32:52.794 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-14T06:32:52.794 INFO:tasks.ceph.osd.5.smithi138.stderr:2024-09-14T06:32:52.792+0000 7fe13746f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33618) UID: 0 2024-09-14T06:32:52.894 INFO:tasks.ceph.osd.0.smithi094.stderr:2024-09-14T06:32:52.893+0000 7f07c9333640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33636) UID: 0 2024-09-14T06:32:52.937 INFO:teuthology.orchestra.run.smithi138.stdout:{ 2024-09-14T06:32:52.937 INFO:teuthology.orchestra.run.smithi138.stdout: "ops": [], 2024-09-14T06:32:52.937 INFO:teuthology.orchestra.run.smithi138.stdout: "num_ops": 0 2024-09-14T06:32:52.938 INFO:teuthology.orchestra.run.smithi138.stdout:} 2024-09-14T06:32:52.941 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-14T06:32:52.941 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-14T06:32:52.941 INFO:teuthology.orchestra.run.smithi094.stdout: "epoch": 247, 2024-09-14T06:32:52.941 INFO:teuthology.orchestra.run.smithi094.stdout: "fsid": "200ad3da-bd63-4e04-af74-efb5ca0af75e", 2024-09-14T06:32:52.941 INFO:teuthology.orchestra.run.smithi094.stdout: "created": "2024-09-14T06:28:30.782046+0000", 2024-09-14T06:32:52.941 INFO:teuthology.orchestra.run.smithi094.stdout: "modified": "2024-09-14T06:32:52.628499+0000", 2024-09-14T06:32:52.941 INFO:teuthology.orchestra.run.smithi094.stdout: "last_up_change": "2024-09-14T06:28:42.827239+0000", 2024-09-14T06:32:52.941 INFO:teuthology.orchestra.run.smithi094.stdout: "last_in_change": "2024-09-14T06:32:00.601027+0000", 2024-09-14T06:32:52.942 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": "noscrub,nodeep-scrub,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2024-09-14T06:32:52.942 INFO:teuthology.orchestra.run.smithi094.stdout: "flags_num": 5806080, 2024-09-14T06:32:52.942 INFO:teuthology.orchestra.run.smithi094.stdout: "flags_set": [ 2024-09-14T06:32:52.942 INFO:teuthology.orchestra.run.smithi094.stdout: "nodeep-scrub", 2024-09-14T06:32:52.942 INFO:teuthology.orchestra.run.smithi094.stdout: "noscrub", 2024-09-14T06:32:52.942 INFO:teuthology.orchestra.run.smithi094.stdout: "pglog_hardlimit", 2024-09-14T06:32:52.942 INFO:teuthology.orchestra.run.smithi094.stdout: "purged_snapdirs", 2024-09-14T06:32:52.942 INFO:teuthology.orchestra.run.smithi094.stdout: "recovery_deletes", 2024-09-14T06:32:52.942 INFO:teuthology.orchestra.run.smithi094.stdout: "sortbitwise" 2024-09-14T06:32:52.942 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-09-14T06:32:52.942 INFO:teuthology.orchestra.run.smithi094.stdout: "crush_version": 5, 2024-09-14T06:32:52.942 INFO:teuthology.orchestra.run.smithi094.stdout: "full_ratio": 0.94999998807907104, 2024-09-14T06:32:52.942 INFO:teuthology.orchestra.run.smithi094.stdout: "backfillfull_ratio": 0.89999997615814209, 2024-09-14T06:32:52.943 INFO:teuthology.orchestra.run.smithi094.stdout: "nearfull_ratio": 0.85000002384185791, 2024-09-14T06:32:52.943 INFO:teuthology.orchestra.run.smithi094.stdout: "cluster_snapshot": "", 2024-09-14T06:32:52.943 INFO:teuthology.orchestra.run.smithi094.stdout: "pool_max": 3, 2024-09-14T06:32:52.943 INFO:teuthology.orchestra.run.smithi094.stdout: "max_osd": 8, 2024-09-14T06:32:52.943 INFO:teuthology.orchestra.run.smithi094.stdout: "require_min_compat_client": "luminous", 2024-09-14T06:32:52.943 INFO:teuthology.orchestra.run.smithi094.stdout: "min_compat_client": "luminous", 2024-09-14T06:32:52.943 INFO:teuthology.orchestra.run.smithi094.stdout: "require_osd_release": "squid", 2024-09-14T06:32:52.943 INFO:teuthology.orchestra.run.smithi094.stdout: "allow_crimson": false, 2024-09-14T06:32:52.943 INFO:teuthology.orchestra.run.smithi094.stdout: "pools": [ 2024-09-14T06:32:52.943 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:52.943 INFO:teuthology.orchestra.run.smithi094.stdout: "pool": 1, 2024-09-14T06:32:52.943 INFO:teuthology.orchestra.run.smithi094.stdout: "pool_name": ".mgr", 2024-09-14T06:32:52.943 INFO:teuthology.orchestra.run.smithi094.stdout: "create_time": "2024-09-14T06:28:43.295990+0000", 2024-09-14T06:32:52.943 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-09-14T06:32:52.944 INFO:teuthology.orchestra.run.smithi094.stdout: "flags_names": "hashpspool", 2024-09-14T06:32:52.944 INFO:teuthology.orchestra.run.smithi094.stdout: "type": 1, 2024-09-14T06:32:52.944 INFO:teuthology.orchestra.run.smithi094.stdout: "size": 2, 2024-09-14T06:32:52.944 INFO:teuthology.orchestra.run.smithi094.stdout: "min_size": 1, 2024-09-14T06:32:52.944 INFO:teuthology.orchestra.run.smithi094.stdout: "crush_rule": 0, 2024-09-14T06:32:52.944 INFO:teuthology.orchestra.run.smithi094.stdout: "peering_crush_bucket_count": 0, 2024-09-14T06:32:52.944 INFO:teuthology.orchestra.run.smithi094.stdout: "peering_crush_bucket_target": 0, 2024-09-14T06:32:52.944 INFO:teuthology.orchestra.run.smithi094.stdout: "peering_crush_bucket_barrier": 0, 2024-09-14T06:32:52.944 INFO:teuthology.orchestra.run.smithi094.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-14T06:32:52.944 INFO:teuthology.orchestra.run.smithi094.stdout: "is_stretch_pool": false, 2024-09-14T06:32:52.945 INFO:teuthology.orchestra.run.smithi094.stdout: "object_hash": 2, 2024-09-14T06:32:52.945 INFO:teuthology.orchestra.run.smithi094.stdout: "pg_autoscale_mode": "off", 2024-09-14T06:32:52.945 INFO:teuthology.orchestra.run.smithi094.stdout: "pg_num": 1, 2024-09-14T06:32:52.945 INFO:teuthology.orchestra.run.smithi094.stdout: "pg_placement_num": 1, 2024-09-14T06:32:52.945 INFO:teuthology.orchestra.run.smithi094.stdout: "pg_placement_num_target": 1, 2024-09-14T06:32:52.945 INFO:teuthology.orchestra.run.smithi094.stdout: "pg_num_target": 1, 2024-09-14T06:32:52.945 INFO:teuthology.orchestra.run.smithi094.stdout: "pg_num_pending": 1, 2024-09-14T06:32:52.945 INFO:teuthology.orchestra.run.smithi094.stdout: "last_pg_merge_meta": { 2024-09-14T06:32:52.946 INFO:teuthology.orchestra.run.smithi094.stdout: "source_pgid": "0.0", 2024-09-14T06:32:52.946 INFO:teuthology.orchestra.run.smithi094.stdout: "ready_epoch": 0, 2024-09-14T06:32:52.946 INFO:teuthology.orchestra.run.smithi094.stdout: "last_epoch_started": 0, 2024-09-14T06:32:52.946 INFO:teuthology.orchestra.run.smithi094.stdout: "last_epoch_clean": 0, 2024-09-14T06:32:52.946 INFO:teuthology.orchestra.run.smithi094.stdout: "source_version": "0'0", 2024-09-14T06:32:52.946 INFO:teuthology.orchestra.run.smithi094.stdout: "target_version": "0'0" 2024-09-14T06:32:52.946 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:52.946 INFO:teuthology.orchestra.run.smithi094.stdout: "last_change": "18", 2024-09-14T06:32:52.946 INFO:teuthology.orchestra.run.smithi094.stdout: "last_force_op_resend": "0", 2024-09-14T06:32:52.946 INFO:teuthology.orchestra.run.smithi094.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-14T06:32:52.946 INFO:teuthology.orchestra.run.smithi094.stdout: "last_force_op_resend_preluminous": "0", 2024-09-14T06:32:52.947 INFO:teuthology.orchestra.run.smithi094.stdout: "auid": 0, 2024-09-14T06:32:52.947 INFO:teuthology.orchestra.run.smithi094.stdout: "snap_mode": "selfmanaged", 2024-09-14T06:32:52.947 INFO:teuthology.orchestra.run.smithi094.stdout: "snap_seq": 0, 2024-09-14T06:32:52.947 INFO:teuthology.orchestra.run.smithi094.stdout: "snap_epoch": 0, 2024-09-14T06:32:52.947 INFO:teuthology.orchestra.run.smithi094.stdout: "pool_snaps": [], 2024-09-14T06:32:52.947 INFO:teuthology.orchestra.run.smithi094.stdout: "removed_snaps": "[]", 2024-09-14T06:32:52.947 INFO:teuthology.orchestra.run.smithi094.stdout: "quota_max_bytes": 0, 2024-09-14T06:32:52.947 INFO:teuthology.orchestra.run.smithi094.stdout: "quota_max_objects": 0, 2024-09-14T06:32:52.947 INFO:teuthology.orchestra.run.smithi094.stdout: "tiers": [], 2024-09-14T06:32:52.947 INFO:teuthology.orchestra.run.smithi094.stdout: "tier_of": -1, 2024-09-14T06:32:52.947 INFO:teuthology.orchestra.run.smithi094.stdout: "read_tier": -1, 2024-09-14T06:32:52.948 INFO:teuthology.orchestra.run.smithi094.stdout: "write_tier": -1, 2024-09-14T06:32:52.948 INFO:teuthology.orchestra.run.smithi094.stdout: "cache_mode": "none", 2024-09-14T06:32:52.948 INFO:teuthology.orchestra.run.smithi094.stdout: "target_max_bytes": 0, 2024-09-14T06:32:52.948 INFO:teuthology.orchestra.run.smithi094.stdout: "target_max_objects": 0, 2024-09-14T06:32:52.948 INFO:teuthology.orchestra.run.smithi094.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-14T06:32:52.948 INFO:teuthology.orchestra.run.smithi094.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-14T06:32:52.948 INFO:teuthology.orchestra.run.smithi094.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-14T06:32:52.948 INFO:teuthology.orchestra.run.smithi094.stdout: "cache_min_flush_age": 0, 2024-09-14T06:32:52.948 INFO:teuthology.orchestra.run.smithi094.stdout: "cache_min_evict_age": 0, 2024-09-14T06:32:52.948 INFO:teuthology.orchestra.run.smithi094.stdout: "erasure_code_profile": "", 2024-09-14T06:32:52.948 INFO:teuthology.orchestra.run.smithi094.stdout: "hit_set_params": { 2024-09-14T06:32:52.948 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "none" 2024-09-14T06:32:52.949 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:52.949 INFO:teuthology.orchestra.run.smithi094.stdout: "hit_set_period": 0, 2024-09-14T06:32:52.949 INFO:teuthology.orchestra.run.smithi094.stdout: "hit_set_count": 0, 2024-09-14T06:32:52.949 INFO:teuthology.orchestra.run.smithi094.stdout: "use_gmt_hitset": true, 2024-09-14T06:32:52.949 INFO:teuthology.orchestra.run.smithi094.stdout: "min_read_recency_for_promote": 0, 2024-09-14T06:32:52.949 INFO:teuthology.orchestra.run.smithi094.stdout: "min_write_recency_for_promote": 0, 2024-09-14T06:32:52.949 INFO:teuthology.orchestra.run.smithi094.stdout: "hit_set_grade_decay_rate": 0, 2024-09-14T06:32:52.949 INFO:teuthology.orchestra.run.smithi094.stdout: "hit_set_search_last_n": 0, 2024-09-14T06:32:52.949 INFO:teuthology.orchestra.run.smithi094.stdout: "grade_table": [], 2024-09-14T06:32:52.949 INFO:teuthology.orchestra.run.smithi094.stdout: "stripe_width": 0, 2024-09-14T06:32:52.949 INFO:teuthology.orchestra.run.smithi094.stdout: "expected_num_objects": 0, 2024-09-14T06:32:52.950 INFO:teuthology.orchestra.run.smithi094.stdout: "fast_read": false, 2024-09-14T06:32:52.950 INFO:teuthology.orchestra.run.smithi094.stdout: "options": { 2024-09-14T06:32:52.950 INFO:teuthology.orchestra.run.smithi094.stdout: "pg_num_max": 32, 2024-09-14T06:32:52.950 INFO:teuthology.orchestra.run.smithi094.stdout: "pg_num_min": 1 2024-09-14T06:32:52.950 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:52.950 INFO:teuthology.orchestra.run.smithi094.stdout: "application_metadata": { 2024-09-14T06:32:52.950 INFO:teuthology.orchestra.run.smithi094.stdout: "mgr": {} 2024-09-14T06:32:52.950 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:52.950 INFO:teuthology.orchestra.run.smithi094.stdout: "read_balance": { 2024-09-14T06:32:52.950 INFO:teuthology.orchestra.run.smithi094.stdout: "score_type": "Fair distribution", 2024-09-14T06:32:52.950 INFO:teuthology.orchestra.run.smithi094.stdout: "score_acting": 6.9000000953674316, 2024-09-14T06:32:52.951 INFO:teuthology.orchestra.run.smithi094.stdout: "score_stable": 6.9000000953674316, 2024-09-14T06:32:52.951 INFO:teuthology.orchestra.run.smithi094.stdout: "optimal_score": 0.28999999165534973, 2024-09-14T06:32:52.951 INFO:teuthology.orchestra.run.smithi094.stdout: "raw_score_acting": 2, 2024-09-14T06:32:52.951 INFO:teuthology.orchestra.run.smithi094.stdout: "raw_score_stable": 2, 2024-09-14T06:32:52.951 INFO:teuthology.orchestra.run.smithi094.stdout: "primary_affinity_weighted": 0.85513687133789062, 2024-09-14T06:32:52.951 INFO:teuthology.orchestra.run.smithi094.stdout: "average_primary_affinity": 0.98000001907348633, 2024-09-14T06:32:52.951 INFO:teuthology.orchestra.run.smithi094.stdout: "average_primary_affinity_weighted": 0.86000001430511475 2024-09-14T06:32:52.951 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:52.951 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:52.951 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:52.951 INFO:teuthology.orchestra.run.smithi094.stdout: "pool": 2, 2024-09-14T06:32:52.952 INFO:teuthology.orchestra.run.smithi094.stdout: "pool_name": "rbd", 2024-09-14T06:32:52.952 INFO:teuthology.orchestra.run.smithi094.stdout: "create_time": "2024-09-14T06:28:49.451714+0000", 2024-09-14T06:32:52.952 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 8193, 2024-09-14T06:32:52.952 INFO:teuthology.orchestra.run.smithi094.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-09-14T06:32:52.952 INFO:teuthology.orchestra.run.smithi094.stdout: "type": 1, 2024-09-14T06:32:52.952 INFO:teuthology.orchestra.run.smithi094.stdout: "size": 2, 2024-09-14T06:32:52.952 INFO:teuthology.orchestra.run.smithi094.stdout: "min_size": 1, 2024-09-14T06:32:52.952 INFO:teuthology.orchestra.run.smithi094.stdout: "crush_rule": 0, 2024-09-14T06:32:52.952 INFO:teuthology.orchestra.run.smithi094.stdout: "peering_crush_bucket_count": 0, 2024-09-14T06:32:52.952 INFO:teuthology.orchestra.run.smithi094.stdout: "peering_crush_bucket_target": 0, 2024-09-14T06:32:52.952 INFO:teuthology.orchestra.run.smithi094.stdout: "peering_crush_bucket_barrier": 0, 2024-09-14T06:32:52.953 INFO:teuthology.orchestra.run.smithi094.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-14T06:32:52.953 INFO:teuthology.orchestra.run.smithi094.stdout: "is_stretch_pool": false, 2024-09-14T06:32:52.953 INFO:teuthology.orchestra.run.smithi094.stdout: "object_hash": 2, 2024-09-14T06:32:52.953 INFO:teuthology.orchestra.run.smithi094.stdout: "pg_autoscale_mode": "off", 2024-09-14T06:32:52.953 INFO:teuthology.orchestra.run.smithi094.stdout: "pg_num": 8, 2024-09-14T06:32:52.953 INFO:teuthology.orchestra.run.smithi094.stdout: "pg_placement_num": 8, 2024-09-14T06:32:52.953 INFO:teuthology.orchestra.run.smithi094.stdout: "pg_placement_num_target": 8, 2024-09-14T06:32:52.953 INFO:teuthology.orchestra.run.smithi094.stdout: "pg_num_target": 8, 2024-09-14T06:32:52.953 INFO:teuthology.orchestra.run.smithi094.stdout: "pg_num_pending": 8, 2024-09-14T06:32:52.953 INFO:teuthology.orchestra.run.smithi094.stdout: "last_pg_merge_meta": { 2024-09-14T06:32:52.953 INFO:teuthology.orchestra.run.smithi094.stdout: "source_pgid": "0.0", 2024-09-14T06:32:52.954 INFO:teuthology.orchestra.run.smithi094.stdout: "ready_epoch": 0, 2024-09-14T06:32:52.954 INFO:teuthology.orchestra.run.smithi094.stdout: "last_epoch_started": 0, 2024-09-14T06:32:52.954 INFO:teuthology.orchestra.run.smithi094.stdout: "last_epoch_clean": 0, 2024-09-14T06:32:52.954 INFO:teuthology.orchestra.run.smithi094.stdout: "source_version": "0'0", 2024-09-14T06:32:52.954 INFO:teuthology.orchestra.run.smithi094.stdout: "target_version": "0'0" 2024-09-14T06:32:52.954 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:52.954 INFO:teuthology.orchestra.run.smithi094.stdout: "last_change": "22", 2024-09-14T06:32:52.954 INFO:teuthology.orchestra.run.smithi094.stdout: "last_force_op_resend": "0", 2024-09-14T06:32:52.954 INFO:teuthology.orchestra.run.smithi094.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-14T06:32:52.954 INFO:teuthology.orchestra.run.smithi094.stdout: "last_force_op_resend_preluminous": "0", 2024-09-14T06:32:52.954 INFO:teuthology.orchestra.run.smithi094.stdout: "auid": 0, 2024-09-14T06:32:52.954 INFO:teuthology.orchestra.run.smithi094.stdout: "snap_mode": "selfmanaged", 2024-09-14T06:32:52.954 INFO:teuthology.orchestra.run.smithi094.stdout: "snap_seq": 2, 2024-09-14T06:32:52.955 INFO:teuthology.orchestra.run.smithi094.stdout: "snap_epoch": 22, 2024-09-14T06:32:52.955 INFO:teuthology.orchestra.run.smithi094.stdout: "pool_snaps": [], 2024-09-14T06:32:52.955 INFO:teuthology.orchestra.run.smithi094.stdout: "removed_snaps": "[]", 2024-09-14T06:32:52.955 INFO:teuthology.orchestra.run.smithi094.stdout: "quota_max_bytes": 0, 2024-09-14T06:32:52.955 INFO:teuthology.orchestra.run.smithi094.stdout: "quota_max_objects": 0, 2024-09-14T06:32:52.955 INFO:teuthology.orchestra.run.smithi094.stdout: "tiers": [], 2024-09-14T06:32:52.955 INFO:teuthology.orchestra.run.smithi094.stdout: "tier_of": -1, 2024-09-14T06:32:52.955 INFO:teuthology.orchestra.run.smithi094.stdout: "read_tier": -1, 2024-09-14T06:32:52.955 INFO:teuthology.orchestra.run.smithi094.stdout: "write_tier": -1, 2024-09-14T06:32:52.955 INFO:teuthology.orchestra.run.smithi094.stdout: "cache_mode": "none", 2024-09-14T06:32:52.955 INFO:teuthology.orchestra.run.smithi094.stdout: "target_max_bytes": 0, 2024-09-14T06:32:52.955 INFO:teuthology.orchestra.run.smithi094.stdout: "target_max_objects": 0, 2024-09-14T06:32:52.956 INFO:teuthology.orchestra.run.smithi094.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-14T06:32:52.956 INFO:teuthology.orchestra.run.smithi094.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-14T06:32:52.956 INFO:teuthology.orchestra.run.smithi094.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-14T06:32:52.956 INFO:teuthology.orchestra.run.smithi094.stdout: "cache_min_flush_age": 0, 2024-09-14T06:32:52.956 INFO:teuthology.orchestra.run.smithi094.stdout: "cache_min_evict_age": 0, 2024-09-14T06:32:52.956 INFO:teuthology.orchestra.run.smithi094.stdout: "erasure_code_profile": "", 2024-09-14T06:32:52.956 INFO:teuthology.orchestra.run.smithi094.stdout: "hit_set_params": { 2024-09-14T06:32:52.956 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "none" 2024-09-14T06:32:52.956 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:52.956 INFO:teuthology.orchestra.run.smithi094.stdout: "hit_set_period": 0, 2024-09-14T06:32:52.956 INFO:teuthology.orchestra.run.smithi094.stdout: "hit_set_count": 0, 2024-09-14T06:32:52.956 INFO:teuthology.orchestra.run.smithi094.stdout: "use_gmt_hitset": true, 2024-09-14T06:32:52.957 INFO:teuthology.orchestra.run.smithi094.stdout: "min_read_recency_for_promote": 0, 2024-09-14T06:32:52.957 INFO:teuthology.orchestra.run.smithi094.stdout: "min_write_recency_for_promote": 0, 2024-09-14T06:32:52.957 INFO:teuthology.orchestra.run.smithi094.stdout: "hit_set_grade_decay_rate": 0, 2024-09-14T06:32:52.957 INFO:teuthology.orchestra.run.smithi094.stdout: "hit_set_search_last_n": 0, 2024-09-14T06:32:52.957 INFO:teuthology.orchestra.run.smithi094.stdout: "grade_table": [], 2024-09-14T06:32:52.957 INFO:teuthology.orchestra.run.smithi094.stdout: "stripe_width": 0, 2024-09-14T06:32:52.957 INFO:teuthology.orchestra.run.smithi094.stdout: "expected_num_objects": 0, 2024-09-14T06:32:52.957 INFO:teuthology.orchestra.run.smithi094.stdout: "fast_read": false, 2024-09-14T06:32:52.957 INFO:teuthology.orchestra.run.smithi094.stdout: "options": {}, 2024-09-14T06:32:52.957 INFO:teuthology.orchestra.run.smithi094.stdout: "application_metadata": { 2024-09-14T06:32:52.958 INFO:teuthology.orchestra.run.smithi094.stdout: "rbd": {} 2024-09-14T06:32:52.958 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:52.958 INFO:teuthology.orchestra.run.smithi094.stdout: "read_balance": { 2024-09-14T06:32:52.958 INFO:teuthology.orchestra.run.smithi094.stdout: "score_type": "Fair distribution", 2024-09-14T06:32:52.958 INFO:teuthology.orchestra.run.smithi094.stdout: "score_acting": 2.630000114440918, 2024-09-14T06:32:52.958 INFO:teuthology.orchestra.run.smithi094.stdout: "score_stable": 2.630000114440918, 2024-09-14T06:32:52.958 INFO:teuthology.orchestra.run.smithi094.stdout: "optimal_score": 1, 2024-09-14T06:32:52.958 INFO:teuthology.orchestra.run.smithi094.stdout: "raw_score_acting": 2.630000114440918, 2024-09-14T06:32:52.958 INFO:teuthology.orchestra.run.smithi094.stdout: "raw_score_stable": 2.630000114440918, 2024-09-14T06:32:52.958 INFO:teuthology.orchestra.run.smithi094.stdout: "primary_affinity_weighted": 0.85513687133789062, 2024-09-14T06:32:52.958 INFO:teuthology.orchestra.run.smithi094.stdout: "average_primary_affinity": 0.98000001907348633, 2024-09-14T06:32:52.958 INFO:teuthology.orchestra.run.smithi094.stdout: "average_primary_affinity_weighted": 0.86000001430511475 2024-09-14T06:32:52.959 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:52.959 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:52.959 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:52.959 INFO:teuthology.orchestra.run.smithi094.stdout: "pool": 3, 2024-09-14T06:32:52.959 INFO:teuthology.orchestra.run.smithi094.stdout: "pool_name": "pool_client.0", 2024-09-14T06:32:52.959 INFO:teuthology.orchestra.run.smithi094.stdout: "create_time": "2024-09-14T06:28:59.688535+0000", 2024-09-14T06:32:52.959 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 8193, 2024-09-14T06:32:52.959 INFO:teuthology.orchestra.run.smithi094.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-09-14T06:32:52.959 INFO:teuthology.orchestra.run.smithi094.stdout: "type": 1, 2024-09-14T06:32:52.959 INFO:teuthology.orchestra.run.smithi094.stdout: "size": 2, 2024-09-14T06:32:52.959 INFO:teuthology.orchestra.run.smithi094.stdout: "min_size": 1, 2024-09-14T06:32:52.960 INFO:teuthology.orchestra.run.smithi094.stdout: "crush_rule": 0, 2024-09-14T06:32:52.960 INFO:teuthology.orchestra.run.smithi094.stdout: "peering_crush_bucket_count": 0, 2024-09-14T06:32:52.960 INFO:teuthology.orchestra.run.smithi094.stdout: "peering_crush_bucket_target": 0, 2024-09-14T06:32:52.960 INFO:teuthology.orchestra.run.smithi094.stdout: "peering_crush_bucket_barrier": 0, 2024-09-14T06:32:52.960 INFO:teuthology.orchestra.run.smithi094.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-14T06:32:52.960 INFO:teuthology.orchestra.run.smithi094.stdout: "is_stretch_pool": false, 2024-09-14T06:32:52.960 INFO:teuthology.orchestra.run.smithi094.stdout: "object_hash": 2, 2024-09-14T06:32:52.960 INFO:teuthology.orchestra.run.smithi094.stdout: "pg_autoscale_mode": "off", 2024-09-14T06:32:52.960 INFO:teuthology.orchestra.run.smithi094.stdout: "pg_num": 32, 2024-09-14T06:32:52.960 INFO:teuthology.orchestra.run.smithi094.stdout: "pg_placement_num": 32, 2024-09-14T06:32:52.961 INFO:teuthology.orchestra.run.smithi094.stdout: "pg_placement_num_target": 32, 2024-09-14T06:32:52.961 INFO:teuthology.orchestra.run.smithi094.stdout: "pg_num_target": 32, 2024-09-14T06:32:52.961 INFO:teuthology.orchestra.run.smithi094.stdout: "pg_num_pending": 32, 2024-09-14T06:32:52.961 INFO:teuthology.orchestra.run.smithi094.stdout: "last_pg_merge_meta": { 2024-09-14T06:32:52.961 INFO:teuthology.orchestra.run.smithi094.stdout: "source_pgid": "0.0", 2024-09-14T06:32:52.961 INFO:teuthology.orchestra.run.smithi094.stdout: "ready_epoch": 0, 2024-09-14T06:32:52.961 INFO:teuthology.orchestra.run.smithi094.stdout: "last_epoch_started": 0, 2024-09-14T06:32:52.961 INFO:teuthology.orchestra.run.smithi094.stdout: "last_epoch_clean": 0, 2024-09-14T06:32:52.961 INFO:teuthology.orchestra.run.smithi094.stdout: "source_version": "0'0", 2024-09-14T06:32:52.961 INFO:teuthology.orchestra.run.smithi094.stdout: "target_version": "0'0" 2024-09-14T06:32:52.961 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:52.962 INFO:teuthology.orchestra.run.smithi094.stdout: "last_change": "247", 2024-09-14T06:32:52.962 INFO:teuthology.orchestra.run.smithi094.stdout: "last_force_op_resend": "0", 2024-09-14T06:32:52.962 INFO:teuthology.orchestra.run.smithi094.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-14T06:32:52.962 INFO:teuthology.orchestra.run.smithi094.stdout: "last_force_op_resend_preluminous": "0", 2024-09-14T06:32:52.962 INFO:teuthology.orchestra.run.smithi094.stdout: "auid": 0, 2024-09-14T06:32:52.962 INFO:teuthology.orchestra.run.smithi094.stdout: "snap_mode": "selfmanaged", 2024-09-14T06:32:52.962 INFO:teuthology.orchestra.run.smithi094.stdout: "snap_seq": 92, 2024-09-14T06:32:52.962 INFO:teuthology.orchestra.run.smithi094.stdout: "snap_epoch": 247, 2024-09-14T06:32:52.962 INFO:teuthology.orchestra.run.smithi094.stdout: "pool_snaps": [], 2024-09-14T06:32:52.962 INFO:teuthology.orchestra.run.smithi094.stdout: "removed_snaps": "[]", 2024-09-14T06:32:52.962 INFO:teuthology.orchestra.run.smithi094.stdout: "quota_max_bytes": 0, 2024-09-14T06:32:52.962 INFO:teuthology.orchestra.run.smithi094.stdout: "quota_max_objects": 0, 2024-09-14T06:32:52.963 INFO:teuthology.orchestra.run.smithi094.stdout: "tiers": [], 2024-09-14T06:32:52.963 INFO:teuthology.orchestra.run.smithi094.stdout: "tier_of": -1, 2024-09-14T06:32:52.963 INFO:teuthology.orchestra.run.smithi094.stdout: "read_tier": -1, 2024-09-14T06:32:52.963 INFO:teuthology.orchestra.run.smithi094.stdout: "write_tier": -1, 2024-09-14T06:32:52.963 INFO:teuthology.orchestra.run.smithi094.stdout: "cache_mode": "none", 2024-09-14T06:32:52.963 INFO:teuthology.orchestra.run.smithi094.stdout: "target_max_bytes": 0, 2024-09-14T06:32:52.963 INFO:teuthology.orchestra.run.smithi094.stdout: "target_max_objects": 0, 2024-09-14T06:32:52.963 INFO:teuthology.orchestra.run.smithi094.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-14T06:32:52.963 INFO:teuthology.orchestra.run.smithi094.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-14T06:32:52.963 INFO:teuthology.orchestra.run.smithi094.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-14T06:32:52.963 INFO:teuthology.orchestra.run.smithi094.stdout: "cache_min_flush_age": 0, 2024-09-14T06:32:52.963 INFO:teuthology.orchestra.run.smithi094.stdout: "cache_min_evict_age": 0, 2024-09-14T06:32:52.964 INFO:teuthology.orchestra.run.smithi094.stdout: "erasure_code_profile": "", 2024-09-14T06:32:52.964 INFO:teuthology.orchestra.run.smithi094.stdout: "hit_set_params": { 2024-09-14T06:32:52.964 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "none" 2024-09-14T06:32:52.964 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:52.964 INFO:teuthology.orchestra.run.smithi094.stdout: "hit_set_period": 0, 2024-09-14T06:32:52.964 INFO:teuthology.orchestra.run.smithi094.stdout: "hit_set_count": 0, 2024-09-14T06:32:52.964 INFO:teuthology.orchestra.run.smithi094.stdout: "use_gmt_hitset": true, 2024-09-14T06:32:52.964 INFO:teuthology.orchestra.run.smithi094.stdout: "min_read_recency_for_promote": 0, 2024-09-14T06:32:52.964 INFO:teuthology.orchestra.run.smithi094.stdout: "min_write_recency_for_promote": 0, 2024-09-14T06:32:52.964 INFO:teuthology.orchestra.run.smithi094.stdout: "hit_set_grade_decay_rate": 0, 2024-09-14T06:32:52.964 INFO:teuthology.orchestra.run.smithi094.stdout: "hit_set_search_last_n": 0, 2024-09-14T06:32:52.965 INFO:teuthology.orchestra.run.smithi094.stdout: "grade_table": [], 2024-09-14T06:32:52.965 INFO:teuthology.orchestra.run.smithi094.stdout: "stripe_width": 0, 2024-09-14T06:32:52.965 INFO:teuthology.orchestra.run.smithi094.stdout: "expected_num_objects": 0, 2024-09-14T06:32:52.965 INFO:teuthology.orchestra.run.smithi094.stdout: "fast_read": false, 2024-09-14T06:32:52.965 INFO:teuthology.orchestra.run.smithi094.stdout: "options": {}, 2024-09-14T06:32:52.965 INFO:teuthology.orchestra.run.smithi094.stdout: "application_metadata": { 2024-09-14T06:32:52.965 INFO:teuthology.orchestra.run.smithi094.stdout: "rbd": {} 2024-09-14T06:32:52.965 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:52.965 INFO:teuthology.orchestra.run.smithi094.stdout: "read_balance": { 2024-09-14T06:32:52.965 INFO:teuthology.orchestra.run.smithi094.stdout: "score_type": "Fair distribution", 2024-09-14T06:32:52.965 INFO:teuthology.orchestra.run.smithi094.stdout: "score_acting": 1.75, 2024-09-14T06:32:52.966 INFO:teuthology.orchestra.run.smithi094.stdout: "score_stable": 1.75, 2024-09-14T06:32:52.966 INFO:teuthology.orchestra.run.smithi094.stdout: "optimal_score": 1, 2024-09-14T06:32:52.966 INFO:teuthology.orchestra.run.smithi094.stdout: "raw_score_acting": 1.75, 2024-09-14T06:32:52.966 INFO:teuthology.orchestra.run.smithi094.stdout: "raw_score_stable": 1.75, 2024-09-14T06:32:52.966 INFO:teuthology.orchestra.run.smithi094.stdout: "primary_affinity_weighted": 0.85513687133789062, 2024-09-14T06:32:52.966 INFO:teuthology.orchestra.run.smithi094.stdout: "average_primary_affinity": 0.98000001907348633, 2024-09-14T06:32:52.966 INFO:teuthology.orchestra.run.smithi094.stdout: "average_primary_affinity_weighted": 0.86000001430511475 2024-09-14T06:32:52.966 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:52.966 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:52.966 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-09-14T06:32:52.966 INFO:teuthology.orchestra.run.smithi094.stdout: "osds": [ 2024-09-14T06:32:52.966 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:52.967 INFO:teuthology.orchestra.run.smithi094.stdout: "osd": 0, 2024-09-14T06:32:52.967 INFO:teuthology.orchestra.run.smithi094.stdout: "uuid": "8b22a249-b842-406a-a02c-c0f31f57b635", 2024-09-14T06:32:52.967 INFO:teuthology.orchestra.run.smithi094.stdout: "up": 1, 2024-09-14T06:32:52.967 INFO:teuthology.orchestra.run.smithi094.stdout: "in": 1, 2024-09-14T06:32:52.967 INFO:teuthology.orchestra.run.smithi094.stdout: "weight": 1, 2024-09-14T06:32:52.967 INFO:teuthology.orchestra.run.smithi094.stdout: "primary_affinity": 1, 2024-09-14T06:32:52.967 INFO:teuthology.orchestra.run.smithi094.stdout: "last_clean_begin": 0, 2024-09-14T06:32:52.967 INFO:teuthology.orchestra.run.smithi094.stdout: "last_clean_end": 0, 2024-09-14T06:32:52.967 INFO:teuthology.orchestra.run.smithi094.stdout: "up_from": 14, 2024-09-14T06:32:52.967 INFO:teuthology.orchestra.run.smithi094.stdout: "up_thru": 199, 2024-09-14T06:32:52.967 INFO:teuthology.orchestra.run.smithi094.stdout: "down_at": 0, 2024-09-14T06:32:52.967 INFO:teuthology.orchestra.run.smithi094.stdout: "lost_at": 0, 2024-09-14T06:32:52.968 INFO:teuthology.orchestra.run.smithi094.stdout: "public_addrs": { 2024-09-14T06:32:52.968 INFO:teuthology.orchestra.run.smithi094.stdout: "addrvec": [ 2024-09-14T06:32:52.968 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:52.968 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v2", 2024-09-14T06:32:52.968 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.94:6824", 2024-09-14T06:32:52.968 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 1809564923 2024-09-14T06:32:52.968 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:52.968 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:52.968 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v1", 2024-09-14T06:32:52.968 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.94:6825", 2024-09-14T06:32:52.968 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 1809564923 2024-09-14T06:32:52.968 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:52.969 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:32:52.969 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:52.969 INFO:teuthology.orchestra.run.smithi094.stdout: "cluster_addrs": { 2024-09-14T06:32:52.969 INFO:teuthology.orchestra.run.smithi094.stdout: "addrvec": [ 2024-09-14T06:32:52.969 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:52.969 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v2", 2024-09-14T06:32:52.969 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.94:6826", 2024-09-14T06:32:52.969 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 1809564923 2024-09-14T06:32:52.969 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:52.969 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:52.969 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v1", 2024-09-14T06:32:52.969 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.94:6827", 2024-09-14T06:32:52.970 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 1809564923 2024-09-14T06:32:52.970 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:52.970 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:32:52.970 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:52.970 INFO:teuthology.orchestra.run.smithi094.stdout: "heartbeat_back_addrs": { 2024-09-14T06:32:52.970 INFO:teuthology.orchestra.run.smithi094.stdout: "addrvec": [ 2024-09-14T06:32:52.970 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:52.970 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v2", 2024-09-14T06:32:52.970 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.94:6830", 2024-09-14T06:32:52.970 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 1809564923 2024-09-14T06:32:52.970 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:52.970 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:52.971 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v1", 2024-09-14T06:32:52.971 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.94:6831", 2024-09-14T06:32:52.971 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 1809564923 2024-09-14T06:32:52.971 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:52.971 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:32:52.971 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:52.971 INFO:teuthology.orchestra.run.smithi094.stdout: "heartbeat_front_addrs": { 2024-09-14T06:32:52.971 INFO:teuthology.orchestra.run.smithi094.stdout: "addrvec": [ 2024-09-14T06:32:52.971 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:52.971 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v2", 2024-09-14T06:32:52.971 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.94:6828", 2024-09-14T06:32:52.971 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 1809564923 2024-09-14T06:32:52.972 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:52.972 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:52.972 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v1", 2024-09-14T06:32:52.972 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.94:6829", 2024-09-14T06:32:52.972 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 1809564923 2024-09-14T06:32:52.972 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:52.972 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:32:52.972 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:52.972 INFO:teuthology.orchestra.run.smithi094.stdout: "public_addr": "172.21.15.94:6825/1809564923", 2024-09-14T06:32:52.972 INFO:teuthology.orchestra.run.smithi094.stdout: "cluster_addr": "172.21.15.94:6827/1809564923", 2024-09-14T06:32:52.972 INFO:teuthology.orchestra.run.smithi094.stdout: "heartbeat_back_addr": "172.21.15.94:6831/1809564923", 2024-09-14T06:32:52.972 INFO:teuthology.orchestra.run.smithi094.stdout: "heartbeat_front_addr": "172.21.15.94:6829/1809564923", 2024-09-14T06:32:52.973 INFO:teuthology.orchestra.run.smithi094.stdout: "state": [ 2024-09-14T06:32:52.973 INFO:teuthology.orchestra.run.smithi094.stdout: "exists", 2024-09-14T06:32:52.973 INFO:teuthology.orchestra.run.smithi094.stdout: "up" 2024-09-14T06:32:52.973 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:32:52.973 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:52.973 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:52.973 INFO:teuthology.orchestra.run.smithi094.stdout: "osd": 1, 2024-09-14T06:32:52.973 INFO:teuthology.orchestra.run.smithi094.stdout: "uuid": "02251a03-84a0-414d-a980-fac4fd043cb6", 2024-09-14T06:32:52.973 INFO:teuthology.orchestra.run.smithi094.stdout: "up": 1, 2024-09-14T06:32:52.973 INFO:teuthology.orchestra.run.smithi094.stdout: "in": 1, 2024-09-14T06:32:52.973 INFO:teuthology.orchestra.run.smithi094.stdout: "weight": 1, 2024-09-14T06:32:52.973 INFO:teuthology.orchestra.run.smithi094.stdout: "primary_affinity": 0.841094970703125, 2024-09-14T06:32:52.974 INFO:teuthology.orchestra.run.smithi094.stdout: "last_clean_begin": 0, 2024-09-14T06:32:52.974 INFO:teuthology.orchestra.run.smithi094.stdout: "last_clean_end": 0, 2024-09-14T06:32:52.974 INFO:teuthology.orchestra.run.smithi094.stdout: "up_from": 15, 2024-09-14T06:32:52.974 INFO:teuthology.orchestra.run.smithi094.stdout: "up_thru": 244, 2024-09-14T06:32:52.974 INFO:teuthology.orchestra.run.smithi094.stdout: "down_at": 0, 2024-09-14T06:32:52.974 INFO:teuthology.orchestra.run.smithi094.stdout: "lost_at": 0, 2024-09-14T06:32:52.974 INFO:teuthology.orchestra.run.smithi094.stdout: "public_addrs": { 2024-09-14T06:32:52.974 INFO:teuthology.orchestra.run.smithi094.stdout: "addrvec": [ 2024-09-14T06:32:52.974 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:52.974 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v2", 2024-09-14T06:32:52.974 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.94:6816", 2024-09-14T06:32:52.974 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 736402652 2024-09-14T06:32:52.975 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:52.975 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:52.975 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v1", 2024-09-14T06:32:52.975 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.94:6817", 2024-09-14T06:32:52.975 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 736402652 2024-09-14T06:32:52.975 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:52.975 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:32:52.975 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:52.975 INFO:teuthology.orchestra.run.smithi094.stdout: "cluster_addrs": { 2024-09-14T06:32:52.975 INFO:teuthology.orchestra.run.smithi094.stdout: "addrvec": [ 2024-09-14T06:32:52.975 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:52.975 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v2", 2024-09-14T06:32:52.976 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.94:6818", 2024-09-14T06:32:52.976 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 736402652 2024-09-14T06:32:52.976 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:52.976 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:52.976 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v1", 2024-09-14T06:32:52.976 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.94:6819", 2024-09-14T06:32:52.976 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 736402652 2024-09-14T06:32:52.976 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:52.976 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:32:52.976 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:52.976 INFO:teuthology.orchestra.run.smithi094.stdout: "heartbeat_back_addrs": { 2024-09-14T06:32:52.976 INFO:teuthology.orchestra.run.smithi094.stdout: "addrvec": [ 2024-09-14T06:32:52.977 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:52.977 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v2", 2024-09-14T06:32:52.977 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.94:6822", 2024-09-14T06:32:52.977 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 736402652 2024-09-14T06:32:52.977 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:52.977 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:52.977 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v1", 2024-09-14T06:32:52.977 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.94:6823", 2024-09-14T06:32:52.977 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 736402652 2024-09-14T06:32:52.977 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:52.977 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:32:52.978 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:52.978 INFO:teuthology.orchestra.run.smithi094.stdout: "heartbeat_front_addrs": { 2024-09-14T06:32:52.978 INFO:teuthology.orchestra.run.smithi094.stdout: "addrvec": [ 2024-09-14T06:32:52.978 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:52.978 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v2", 2024-09-14T06:32:52.978 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.94:6820", 2024-09-14T06:32:52.978 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 736402652 2024-09-14T06:32:52.978 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:52.978 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:52.978 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v1", 2024-09-14T06:32:52.978 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.94:6821", 2024-09-14T06:32:52.978 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 736402652 2024-09-14T06:32:52.979 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:52.979 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:32:52.979 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:52.979 INFO:teuthology.orchestra.run.smithi094.stdout: "public_addr": "172.21.15.94:6817/736402652", 2024-09-14T06:32:52.979 INFO:teuthology.orchestra.run.smithi094.stdout: "cluster_addr": "172.21.15.94:6819/736402652", 2024-09-14T06:32:52.979 INFO:teuthology.orchestra.run.smithi094.stdout: "heartbeat_back_addr": "172.21.15.94:6823/736402652", 2024-09-14T06:32:52.979 INFO:teuthology.orchestra.run.smithi094.stdout: "heartbeat_front_addr": "172.21.15.94:6821/736402652", 2024-09-14T06:32:52.979 INFO:teuthology.orchestra.run.smithi094.stdout: "state": [ 2024-09-14T06:32:52.979 INFO:teuthology.orchestra.run.smithi094.stdout: "exists", 2024-09-14T06:32:52.979 INFO:teuthology.orchestra.run.smithi094.stdout: "up" 2024-09-14T06:32:52.979 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:32:52.979 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:52.980 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:52.980 INFO:teuthology.orchestra.run.smithi094.stdout: "osd": 2, 2024-09-14T06:32:52.980 INFO:teuthology.orchestra.run.smithi094.stdout: "uuid": "d45bf470-bb29-4739-8294-cd122b2b4102", 2024-09-14T06:32:52.980 INFO:teuthology.orchestra.run.smithi094.stdout: "up": 1, 2024-09-14T06:32:52.980 INFO:teuthology.orchestra.run.smithi094.stdout: "in": 1, 2024-09-14T06:32:52.980 INFO:teuthology.orchestra.run.smithi094.stdout: "weight": 1, 2024-09-14T06:32:52.980 INFO:teuthology.orchestra.run.smithi094.stdout: "primary_affinity": 1, 2024-09-14T06:32:52.980 INFO:teuthology.orchestra.run.smithi094.stdout: "last_clean_begin": 0, 2024-09-14T06:32:52.980 INFO:teuthology.orchestra.run.smithi094.stdout: "last_clean_end": 0, 2024-09-14T06:32:52.980 INFO:teuthology.orchestra.run.smithi094.stdout: "up_from": 14, 2024-09-14T06:32:52.980 INFO:teuthology.orchestra.run.smithi094.stdout: "up_thru": 245, 2024-09-14T06:32:52.980 INFO:teuthology.orchestra.run.smithi094.stdout: "down_at": 0, 2024-09-14T06:32:52.981 INFO:teuthology.orchestra.run.smithi094.stdout: "lost_at": 0, 2024-09-14T06:32:52.981 INFO:teuthology.orchestra.run.smithi094.stdout: "public_addrs": { 2024-09-14T06:32:52.981 INFO:teuthology.orchestra.run.smithi094.stdout: "addrvec": [ 2024-09-14T06:32:52.981 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:52.981 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v2", 2024-09-14T06:32:52.981 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.94:6808", 2024-09-14T06:32:52.981 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 1063204782 2024-09-14T06:32:52.981 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:52.981 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:52.981 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v1", 2024-09-14T06:32:52.981 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.94:6809", 2024-09-14T06:32:52.981 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 1063204782 2024-09-14T06:32:52.982 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:52.982 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:32:52.982 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:52.982 INFO:teuthology.orchestra.run.smithi094.stdout: "cluster_addrs": { 2024-09-14T06:32:52.982 INFO:teuthology.orchestra.run.smithi094.stdout: "addrvec": [ 2024-09-14T06:32:52.982 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:52.982 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v2", 2024-09-14T06:32:52.982 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.94:6810", 2024-09-14T06:32:52.982 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 1063204782 2024-09-14T06:32:52.982 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:52.982 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:52.983 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v1", 2024-09-14T06:32:52.983 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.94:6811", 2024-09-14T06:32:52.983 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 1063204782 2024-09-14T06:32:52.983 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:52.983 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:32:52.983 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:52.983 INFO:teuthology.orchestra.run.smithi094.stdout: "heartbeat_back_addrs": { 2024-09-14T06:32:52.983 INFO:teuthology.orchestra.run.smithi094.stdout: "addrvec": [ 2024-09-14T06:32:52.983 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:52.983 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v2", 2024-09-14T06:32:52.983 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.94:6814", 2024-09-14T06:32:52.983 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 1063204782 2024-09-14T06:32:52.984 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:52.984 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:52.984 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v1", 2024-09-14T06:32:52.984 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.94:6815", 2024-09-14T06:32:52.984 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 1063204782 2024-09-14T06:32:52.984 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:52.984 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:32:52.984 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:52.984 INFO:teuthology.orchestra.run.smithi094.stdout: "heartbeat_front_addrs": { 2024-09-14T06:32:52.984 INFO:teuthology.orchestra.run.smithi094.stdout: "addrvec": [ 2024-09-14T06:32:52.984 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:52.985 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v2", 2024-09-14T06:32:52.985 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.94:6812", 2024-09-14T06:32:52.985 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 1063204782 2024-09-14T06:32:52.985 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:52.985 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:52.985 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v1", 2024-09-14T06:32:52.985 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.94:6813", 2024-09-14T06:32:52.985 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 1063204782 2024-09-14T06:32:52.985 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:52.985 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:32:52.985 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:52.985 INFO:teuthology.orchestra.run.smithi094.stdout: "public_addr": "172.21.15.94:6809/1063204782", 2024-09-14T06:32:52.986 INFO:teuthology.orchestra.run.smithi094.stdout: "cluster_addr": "172.21.15.94:6811/1063204782", 2024-09-14T06:32:52.986 INFO:teuthology.orchestra.run.smithi094.stdout: "heartbeat_back_addr": "172.21.15.94:6815/1063204782", 2024-09-14T06:32:52.986 INFO:teuthology.orchestra.run.smithi094.stdout: "heartbeat_front_addr": "172.21.15.94:6813/1063204782", 2024-09-14T06:32:52.986 INFO:teuthology.orchestra.run.smithi094.stdout: "state": [ 2024-09-14T06:32:52.986 INFO:teuthology.orchestra.run.smithi094.stdout: "exists", 2024-09-14T06:32:52.986 INFO:teuthology.orchestra.run.smithi094.stdout: "up" 2024-09-14T06:32:52.986 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:32:52.986 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:52.986 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:52.986 INFO:teuthology.orchestra.run.smithi094.stdout: "osd": 3, 2024-09-14T06:32:52.986 INFO:teuthology.orchestra.run.smithi094.stdout: "uuid": "456ec2d9-c438-4243-b931-335704f661eb", 2024-09-14T06:32:52.987 INFO:teuthology.orchestra.run.smithi094.stdout: "up": 1, 2024-09-14T06:32:52.987 INFO:teuthology.orchestra.run.smithi094.stdout: "in": 1, 2024-09-14T06:32:52.987 INFO:teuthology.orchestra.run.smithi094.stdout: "weight": 1, 2024-09-14T06:32:52.987 INFO:teuthology.orchestra.run.smithi094.stdout: "primary_affinity": 1, 2024-09-14T06:32:52.987 INFO:teuthology.orchestra.run.smithi094.stdout: "last_clean_begin": 0, 2024-09-14T06:32:52.987 INFO:teuthology.orchestra.run.smithi094.stdout: "last_clean_end": 0, 2024-09-14T06:32:52.987 INFO:teuthology.orchestra.run.smithi094.stdout: "up_from": 13, 2024-09-14T06:32:52.987 INFO:teuthology.orchestra.run.smithi094.stdout: "up_thru": 218, 2024-09-14T06:32:52.987 INFO:teuthology.orchestra.run.smithi094.stdout: "down_at": 0, 2024-09-14T06:32:52.987 INFO:teuthology.orchestra.run.smithi094.stdout: "lost_at": 0, 2024-09-14T06:32:52.987 INFO:teuthology.orchestra.run.smithi094.stdout: "public_addrs": { 2024-09-14T06:32:52.987 INFO:teuthology.orchestra.run.smithi094.stdout: "addrvec": [ 2024-09-14T06:32:52.988 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:52.988 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v2", 2024-09-14T06:32:52.988 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.94:6800", 2024-09-14T06:32:52.988 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 3382046754 2024-09-14T06:32:52.988 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:52.988 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:52.988 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v1", 2024-09-14T06:32:52.988 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.94:6801", 2024-09-14T06:32:52.988 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 3382046754 2024-09-14T06:32:52.988 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:52.988 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:32:52.988 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:52.989 INFO:teuthology.orchestra.run.smithi094.stdout: "cluster_addrs": { 2024-09-14T06:32:52.989 INFO:teuthology.orchestra.run.smithi094.stdout: "addrvec": [ 2024-09-14T06:32:52.989 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:52.989 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v2", 2024-09-14T06:32:52.989 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.94:6802", 2024-09-14T06:32:52.989 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 3382046754 2024-09-14T06:32:52.989 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:52.989 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:52.989 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v1", 2024-09-14T06:32:52.989 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.94:6803", 2024-09-14T06:32:52.989 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 3382046754 2024-09-14T06:32:52.990 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:52.990 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:32:52.990 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:52.990 INFO:teuthology.orchestra.run.smithi094.stdout: "heartbeat_back_addrs": { 2024-09-14T06:32:52.990 INFO:teuthology.orchestra.run.smithi094.stdout: "addrvec": [ 2024-09-14T06:32:52.990 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:52.990 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v2", 2024-09-14T06:32:52.990 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.94:6806", 2024-09-14T06:32:52.990 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 3382046754 2024-09-14T06:32:52.990 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:52.990 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:52.991 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v1", 2024-09-14T06:32:52.991 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.94:6807", 2024-09-14T06:32:52.991 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 3382046754 2024-09-14T06:32:52.991 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:52.991 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:32:52.991 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:52.991 INFO:teuthology.orchestra.run.smithi094.stdout: "heartbeat_front_addrs": { 2024-09-14T06:32:52.991 INFO:teuthology.orchestra.run.smithi094.stdout: "addrvec": [ 2024-09-14T06:32:52.991 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:52.991 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v2", 2024-09-14T06:32:52.991 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.94:6804", 2024-09-14T06:32:52.991 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 3382046754 2024-09-14T06:32:52.992 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:52.992 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:52.992 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v1", 2024-09-14T06:32:52.992 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.94:6805", 2024-09-14T06:32:52.992 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 3382046754 2024-09-14T06:32:52.992 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:52.992 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:32:52.992 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:52.992 INFO:teuthology.orchestra.run.smithi094.stdout: "public_addr": "172.21.15.94:6801/3382046754", 2024-09-14T06:32:52.992 INFO:teuthology.orchestra.run.smithi094.stdout: "cluster_addr": "172.21.15.94:6803/3382046754", 2024-09-14T06:32:52.992 INFO:teuthology.orchestra.run.smithi094.stdout: "heartbeat_back_addr": "172.21.15.94:6807/3382046754", 2024-09-14T06:32:52.992 INFO:teuthology.orchestra.run.smithi094.stdout: "heartbeat_front_addr": "172.21.15.94:6805/3382046754", 2024-09-14T06:32:52.993 INFO:teuthology.orchestra.run.smithi094.stdout: "state": [ 2024-09-14T06:32:52.993 INFO:teuthology.orchestra.run.smithi094.stdout: "exists", 2024-09-14T06:32:52.993 INFO:teuthology.orchestra.run.smithi094.stdout: "up" 2024-09-14T06:32:52.993 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:32:52.993 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:52.993 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:52.993 INFO:teuthology.orchestra.run.smithi094.stdout: "osd": 4, 2024-09-14T06:32:52.993 INFO:teuthology.orchestra.run.smithi094.stdout: "uuid": "2dab948e-c6e4-4437-bf17-017d1ac6821c", 2024-09-14T06:32:52.993 INFO:teuthology.orchestra.run.smithi094.stdout: "up": 1, 2024-09-14T06:32:52.993 INFO:teuthology.orchestra.run.smithi094.stdout: "in": 1, 2024-09-14T06:32:52.993 INFO:teuthology.orchestra.run.smithi094.stdout: "weight": 1, 2024-09-14T06:32:52.993 INFO:teuthology.orchestra.run.smithi094.stdout: "primary_affinity": 1, 2024-09-14T06:32:52.994 INFO:teuthology.orchestra.run.smithi094.stdout: "last_clean_begin": 0, 2024-09-14T06:32:52.994 INFO:teuthology.orchestra.run.smithi094.stdout: "last_clean_end": 0, 2024-09-14T06:32:52.994 INFO:teuthology.orchestra.run.smithi094.stdout: "up_from": 15, 2024-09-14T06:32:52.994 INFO:teuthology.orchestra.run.smithi094.stdout: "up_thru": 232, 2024-09-14T06:32:52.994 INFO:teuthology.orchestra.run.smithi094.stdout: "down_at": 0, 2024-09-14T06:32:52.994 INFO:teuthology.orchestra.run.smithi094.stdout: "lost_at": 0, 2024-09-14T06:32:52.994 INFO:teuthology.orchestra.run.smithi094.stdout: "public_addrs": { 2024-09-14T06:32:52.994 INFO:teuthology.orchestra.run.smithi094.stdout: "addrvec": [ 2024-09-14T06:32:52.994 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:52.994 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v2", 2024-09-14T06:32:52.994 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.138:6810", 2024-09-14T06:32:52.994 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 501726729 2024-09-14T06:32:52.995 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:52.995 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:52.995 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v1", 2024-09-14T06:32:52.995 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.138:6811", 2024-09-14T06:32:52.995 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 501726729 2024-09-14T06:32:52.995 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:52.995 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:32:52.995 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:52.995 INFO:teuthology.orchestra.run.smithi094.stdout: "cluster_addrs": { 2024-09-14T06:32:52.995 INFO:teuthology.orchestra.run.smithi094.stdout: "addrvec": [ 2024-09-14T06:32:52.995 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:52.995 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v2", 2024-09-14T06:32:52.996 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.138:6812", 2024-09-14T06:32:52.996 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 501726729 2024-09-14T06:32:52.996 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:52.996 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:52.996 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v1", 2024-09-14T06:32:52.996 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.138:6813", 2024-09-14T06:32:52.996 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 501726729 2024-09-14T06:32:52.996 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:52.996 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:32:52.996 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:52.996 INFO:teuthology.orchestra.run.smithi094.stdout: "heartbeat_back_addrs": { 2024-09-14T06:32:52.996 INFO:teuthology.orchestra.run.smithi094.stdout: "addrvec": [ 2024-09-14T06:32:52.997 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:52.997 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v2", 2024-09-14T06:32:52.997 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.138:6816", 2024-09-14T06:32:52.997 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 501726729 2024-09-14T06:32:52.997 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:52.997 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:52.997 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v1", 2024-09-14T06:32:52.997 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.138:6817", 2024-09-14T06:32:52.997 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 501726729 2024-09-14T06:32:52.997 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:52.997 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:32:52.997 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:52.998 INFO:teuthology.orchestra.run.smithi094.stdout: "heartbeat_front_addrs": { 2024-09-14T06:32:52.998 INFO:teuthology.orchestra.run.smithi094.stdout: "addrvec": [ 2024-09-14T06:32:52.998 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:52.998 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v2", 2024-09-14T06:32:52.998 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.138:6814", 2024-09-14T06:32:52.998 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 501726729 2024-09-14T06:32:52.998 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:52.998 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:52.998 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v1", 2024-09-14T06:32:52.998 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.138:6815", 2024-09-14T06:32:52.998 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 501726729 2024-09-14T06:32:52.998 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:52.999 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:32:52.999 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:52.999 INFO:teuthology.orchestra.run.smithi094.stdout: "public_addr": "172.21.15.138:6811/501726729", 2024-09-14T06:32:52.999 INFO:teuthology.orchestra.run.smithi094.stdout: "cluster_addr": "172.21.15.138:6813/501726729", 2024-09-14T06:32:52.999 INFO:teuthology.orchestra.run.smithi094.stdout: "heartbeat_back_addr": "172.21.15.138:6817/501726729", 2024-09-14T06:32:52.999 INFO:teuthology.orchestra.run.smithi094.stdout: "heartbeat_front_addr": "172.21.15.138:6815/501726729", 2024-09-14T06:32:52.999 INFO:teuthology.orchestra.run.smithi094.stdout: "state": [ 2024-09-14T06:32:52.999 INFO:teuthology.orchestra.run.smithi094.stdout: "exists", 2024-09-14T06:32:52.999 INFO:teuthology.orchestra.run.smithi094.stdout: "up" 2024-09-14T06:32:52.999 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:32:52.999 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:52.999 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:53.000 INFO:teuthology.orchestra.run.smithi094.stdout: "osd": 5, 2024-09-14T06:32:53.000 INFO:teuthology.orchestra.run.smithi094.stdout: "uuid": "8caf3aa3-ad87-4d33-b82e-4c15195176fd", 2024-09-14T06:32:53.000 INFO:teuthology.orchestra.run.smithi094.stdout: "up": 1, 2024-09-14T06:32:53.000 INFO:teuthology.orchestra.run.smithi094.stdout: "in": 1, 2024-09-14T06:32:53.000 INFO:teuthology.orchestra.run.smithi094.stdout: "weight": 1, 2024-09-14T06:32:53.000 INFO:teuthology.orchestra.run.smithi094.stdout: "primary_affinity": 0, 2024-09-14T06:32:53.000 INFO:teuthology.orchestra.run.smithi094.stdout: "last_clean_begin": 0, 2024-09-14T06:32:53.000 INFO:teuthology.orchestra.run.smithi094.stdout: "last_clean_end": 0, 2024-09-14T06:32:53.000 INFO:teuthology.orchestra.run.smithi094.stdout: "up_from": 15, 2024-09-14T06:32:53.000 INFO:teuthology.orchestra.run.smithi094.stdout: "up_thru": 199, 2024-09-14T06:32:53.000 INFO:teuthology.orchestra.run.smithi094.stdout: "down_at": 0, 2024-09-14T06:32:53.001 INFO:teuthology.orchestra.run.smithi094.stdout: "lost_at": 0, 2024-09-14T06:32:53.001 INFO:teuthology.orchestra.run.smithi094.stdout: "public_addrs": { 2024-09-14T06:32:53.001 INFO:teuthology.orchestra.run.smithi094.stdout: "addrvec": [ 2024-09-14T06:32:53.001 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:53.001 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v2", 2024-09-14T06:32:53.001 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.138:6826", 2024-09-14T06:32:53.001 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 1599292735 2024-09-14T06:32:53.001 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:53.001 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:53.001 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v1", 2024-09-14T06:32:53.001 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.138:6827", 2024-09-14T06:32:53.001 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 1599292735 2024-09-14T06:32:53.001 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:53.002 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:32:53.002 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:53.002 INFO:teuthology.orchestra.run.smithi094.stdout: "cluster_addrs": { 2024-09-14T06:32:53.002 INFO:teuthology.orchestra.run.smithi094.stdout: "addrvec": [ 2024-09-14T06:32:53.002 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:53.002 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v2", 2024-09-14T06:32:53.002 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.138:6828", 2024-09-14T06:32:53.002 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 1599292735 2024-09-14T06:32:53.002 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:53.002 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:53.002 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v1", 2024-09-14T06:32:53.002 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.138:6829", 2024-09-14T06:32:53.003 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 1599292735 2024-09-14T06:32:53.003 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:53.003 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:32:53.003 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:53.003 INFO:teuthology.orchestra.run.smithi094.stdout: "heartbeat_back_addrs": { 2024-09-14T06:32:53.003 INFO:teuthology.orchestra.run.smithi094.stdout: "addrvec": [ 2024-09-14T06:32:53.003 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:53.003 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v2", 2024-09-14T06:32:53.003 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.138:6832", 2024-09-14T06:32:53.003 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 1599292735 2024-09-14T06:32:53.003 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:53.003 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:53.004 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v1", 2024-09-14T06:32:53.004 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.138:6833", 2024-09-14T06:32:53.004 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 1599292735 2024-09-14T06:32:53.004 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:53.004 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:32:53.004 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:53.004 INFO:teuthology.orchestra.run.smithi094.stdout: "heartbeat_front_addrs": { 2024-09-14T06:32:53.004 INFO:teuthology.orchestra.run.smithi094.stdout: "addrvec": [ 2024-09-14T06:32:53.004 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:53.004 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v2", 2024-09-14T06:32:53.004 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.138:6830", 2024-09-14T06:32:53.004 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 1599292735 2024-09-14T06:32:53.005 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:53.005 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:53.005 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v1", 2024-09-14T06:32:53.005 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.138:6831", 2024-09-14T06:32:53.005 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 1599292735 2024-09-14T06:32:53.005 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:53.005 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:32:53.005 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:53.005 INFO:teuthology.orchestra.run.smithi094.stdout: "public_addr": "172.21.15.138:6827/1599292735", 2024-09-14T06:32:53.005 INFO:teuthology.orchestra.run.smithi094.stdout: "cluster_addr": "172.21.15.138:6829/1599292735", 2024-09-14T06:32:53.005 INFO:teuthology.orchestra.run.smithi094.stdout: "heartbeat_back_addr": "172.21.15.138:6833/1599292735", 2024-09-14T06:32:53.005 INFO:teuthology.orchestra.run.smithi094.stdout: "heartbeat_front_addr": "172.21.15.138:6831/1599292735", 2024-09-14T06:32:53.006 INFO:teuthology.orchestra.run.smithi094.stdout: "state": [ 2024-09-14T06:32:53.006 INFO:teuthology.orchestra.run.smithi094.stdout: "exists", 2024-09-14T06:32:53.006 INFO:teuthology.orchestra.run.smithi094.stdout: "up" 2024-09-14T06:32:53.006 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:32:53.006 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:53.006 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:53.006 INFO:teuthology.orchestra.run.smithi094.stdout: "osd": 6, 2024-09-14T06:32:53.006 INFO:teuthology.orchestra.run.smithi094.stdout: "uuid": "e55d11fd-433d-4462-93a7-d5e84560221b", 2024-09-14T06:32:53.006 INFO:teuthology.orchestra.run.smithi094.stdout: "up": 1, 2024-09-14T06:32:53.006 INFO:teuthology.orchestra.run.smithi094.stdout: "in": 0, 2024-09-14T06:32:53.006 INFO:teuthology.orchestra.run.smithi094.stdout: "weight": 0, 2024-09-14T06:32:53.006 INFO:teuthology.orchestra.run.smithi094.stdout: "primary_affinity": 1, 2024-09-14T06:32:53.007 INFO:teuthology.orchestra.run.smithi094.stdout: "last_clean_begin": 0, 2024-09-14T06:32:53.007 INFO:teuthology.orchestra.run.smithi094.stdout: "last_clean_end": 0, 2024-09-14T06:32:53.007 INFO:teuthology.orchestra.run.smithi094.stdout: "up_from": 15, 2024-09-14T06:32:53.007 INFO:teuthology.orchestra.run.smithi094.stdout: "up_thru": 176, 2024-09-14T06:32:53.007 INFO:teuthology.orchestra.run.smithi094.stdout: "down_at": 0, 2024-09-14T06:32:53.007 INFO:teuthology.orchestra.run.smithi094.stdout: "lost_at": 0, 2024-09-14T06:32:53.007 INFO:teuthology.orchestra.run.smithi094.stdout: "public_addrs": { 2024-09-14T06:32:53.007 INFO:teuthology.orchestra.run.smithi094.stdout: "addrvec": [ 2024-09-14T06:32:53.007 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:53.007 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v2", 2024-09-14T06:32:53.007 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.138:6818", 2024-09-14T06:32:53.007 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 163531479 2024-09-14T06:32:53.008 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:53.008 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:53.008 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v1", 2024-09-14T06:32:53.008 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.138:6819", 2024-09-14T06:32:53.008 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 163531479 2024-09-14T06:32:53.008 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:53.008 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:32:53.008 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:53.008 INFO:teuthology.orchestra.run.smithi094.stdout: "cluster_addrs": { 2024-09-14T06:32:53.008 INFO:teuthology.orchestra.run.smithi094.stdout: "addrvec": [ 2024-09-14T06:32:53.008 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:53.009 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v2", 2024-09-14T06:32:53.009 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.138:6820", 2024-09-14T06:32:53.009 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 163531479 2024-09-14T06:32:53.009 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:53.009 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:53.009 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v1", 2024-09-14T06:32:53.009 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.138:6821", 2024-09-14T06:32:53.009 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 163531479 2024-09-14T06:32:53.009 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:53.009 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:32:53.009 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:53.009 INFO:teuthology.orchestra.run.smithi094.stdout: "heartbeat_back_addrs": { 2024-09-14T06:32:53.010 INFO:teuthology.orchestra.run.smithi094.stdout: "addrvec": [ 2024-09-14T06:32:53.010 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:53.010 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v2", 2024-09-14T06:32:53.010 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.138:6824", 2024-09-14T06:32:53.010 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 163531479 2024-09-14T06:32:53.010 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:53.010 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:53.010 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v1", 2024-09-14T06:32:53.010 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.138:6825", 2024-09-14T06:32:53.010 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 163531479 2024-09-14T06:32:53.010 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:53.010 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:32:53.011 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:53.011 INFO:teuthology.orchestra.run.smithi094.stdout: "heartbeat_front_addrs": { 2024-09-14T06:32:53.011 INFO:teuthology.orchestra.run.smithi094.stdout: "addrvec": [ 2024-09-14T06:32:53.011 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:53.011 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v2", 2024-09-14T06:32:53.011 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.138:6822", 2024-09-14T06:32:53.011 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 163531479 2024-09-14T06:32:53.011 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:53.011 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:53.011 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v1", 2024-09-14T06:32:53.011 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.138:6823", 2024-09-14T06:32:53.011 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 163531479 2024-09-14T06:32:53.012 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:53.012 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:32:53.012 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:53.012 INFO:teuthology.orchestra.run.smithi094.stdout: "public_addr": "172.21.15.138:6819/163531479", 2024-09-14T06:32:53.012 INFO:teuthology.orchestra.run.smithi094.stdout: "cluster_addr": "172.21.15.138:6821/163531479", 2024-09-14T06:32:53.012 INFO:teuthology.orchestra.run.smithi094.stdout: "heartbeat_back_addr": "172.21.15.138:6825/163531479", 2024-09-14T06:32:53.012 INFO:teuthology.orchestra.run.smithi094.stdout: "heartbeat_front_addr": "172.21.15.138:6823/163531479", 2024-09-14T06:32:53.012 INFO:teuthology.orchestra.run.smithi094.stdout: "state": [ 2024-09-14T06:32:53.012 INFO:teuthology.orchestra.run.smithi094.stdout: "exists", 2024-09-14T06:32:53.012 INFO:teuthology.orchestra.run.smithi094.stdout: "up" 2024-09-14T06:32:53.012 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:32:53.012 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:53.013 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:53.013 INFO:teuthology.orchestra.run.smithi094.stdout: "osd": 7, 2024-09-14T06:32:53.013 INFO:teuthology.orchestra.run.smithi094.stdout: "uuid": "ba14ba69-2c6c-4d09-81e2-2593dc3a7ea9", 2024-09-14T06:32:53.013 INFO:teuthology.orchestra.run.smithi094.stdout: "up": 1, 2024-09-14T06:32:53.013 INFO:teuthology.orchestra.run.smithi094.stdout: "in": 1, 2024-09-14T06:32:53.013 INFO:teuthology.orchestra.run.smithi094.stdout: "weight": 1, 2024-09-14T06:32:53.013 INFO:teuthology.orchestra.run.smithi094.stdout: "primary_affinity": 1, 2024-09-14T06:32:53.013 INFO:teuthology.orchestra.run.smithi094.stdout: "last_clean_begin": 0, 2024-09-14T06:32:53.013 INFO:teuthology.orchestra.run.smithi094.stdout: "last_clean_end": 0, 2024-09-14T06:32:53.013 INFO:teuthology.orchestra.run.smithi094.stdout: "up_from": 15, 2024-09-14T06:32:53.013 INFO:teuthology.orchestra.run.smithi094.stdout: "up_thru": 233, 2024-09-14T06:32:53.014 INFO:teuthology.orchestra.run.smithi094.stdout: "down_at": 0, 2024-09-14T06:32:53.014 INFO:teuthology.orchestra.run.smithi094.stdout: "lost_at": 0, 2024-09-14T06:32:53.014 INFO:teuthology.orchestra.run.smithi094.stdout: "public_addrs": { 2024-09-14T06:32:53.014 INFO:teuthology.orchestra.run.smithi094.stdout: "addrvec": [ 2024-09-14T06:32:53.014 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:53.014 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v2", 2024-09-14T06:32:53.014 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.138:6800", 2024-09-14T06:32:53.014 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 218507235 2024-09-14T06:32:53.014 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:53.014 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:53.014 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v1", 2024-09-14T06:32:53.014 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.138:6801", 2024-09-14T06:32:53.015 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 218507235 2024-09-14T06:32:53.015 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:53.015 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:32:53.015 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:53.015 INFO:teuthology.orchestra.run.smithi094.stdout: "cluster_addrs": { 2024-09-14T06:32:53.015 INFO:teuthology.orchestra.run.smithi094.stdout: "addrvec": [ 2024-09-14T06:32:53.015 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:53.015 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v2", 2024-09-14T06:32:53.015 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.138:6802", 2024-09-14T06:32:53.015 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 218507235 2024-09-14T06:32:53.015 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:53.016 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:53.016 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v1", 2024-09-14T06:32:53.016 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.138:6803", 2024-09-14T06:32:53.016 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 218507235 2024-09-14T06:32:53.016 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:53.016 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:32:53.016 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:53.016 INFO:teuthology.orchestra.run.smithi094.stdout: "heartbeat_back_addrs": { 2024-09-14T06:32:53.016 INFO:teuthology.orchestra.run.smithi094.stdout: "addrvec": [ 2024-09-14T06:32:53.016 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:53.016 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v2", 2024-09-14T06:32:53.017 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.138:6806", 2024-09-14T06:32:53.017 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 218507235 2024-09-14T06:32:53.017 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:53.017 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:53.017 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v1", 2024-09-14T06:32:53.017 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.138:6807", 2024-09-14T06:32:53.017 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 218507235 2024-09-14T06:32:53.017 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:53.017 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:32:53.017 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:53.017 INFO:teuthology.orchestra.run.smithi094.stdout: "heartbeat_front_addrs": { 2024-09-14T06:32:53.017 INFO:teuthology.orchestra.run.smithi094.stdout: "addrvec": [ 2024-09-14T06:32:53.018 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:53.018 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v2", 2024-09-14T06:32:53.018 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.138:6804", 2024-09-14T06:32:53.018 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 218507235 2024-09-14T06:32:53.018 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:53.018 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:53.018 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v1", 2024-09-14T06:32:53.018 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.138:6805", 2024-09-14T06:32:53.018 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 218507235 2024-09-14T06:32:53.018 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:53.018 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:32:53.019 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:53.019 INFO:teuthology.orchestra.run.smithi094.stdout: "public_addr": "172.21.15.138:6801/218507235", 2024-09-14T06:32:53.019 INFO:teuthology.orchestra.run.smithi094.stdout: "cluster_addr": "172.21.15.138:6803/218507235", 2024-09-14T06:32:53.019 INFO:teuthology.orchestra.run.smithi094.stdout: "heartbeat_back_addr": "172.21.15.138:6807/218507235", 2024-09-14T06:32:53.019 INFO:teuthology.orchestra.run.smithi094.stdout: "heartbeat_front_addr": "172.21.15.138:6805/218507235", 2024-09-14T06:32:53.019 INFO:teuthology.orchestra.run.smithi094.stdout: "state": [ 2024-09-14T06:32:53.019 INFO:teuthology.orchestra.run.smithi094.stdout: "exists", 2024-09-14T06:32:53.019 INFO:teuthology.orchestra.run.smithi094.stdout: "up" 2024-09-14T06:32:53.019 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:32:53.019 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:53.019 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-09-14T06:32:53.019 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_xinfo": [ 2024-09-14T06:32:53.020 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:53.020 INFO:teuthology.orchestra.run.smithi094.stdout: "osd": 0, 2024-09-14T06:32:53.020 INFO:teuthology.orchestra.run.smithi094.stdout: "down_stamp": "0.000000", 2024-09-14T06:32:53.020 INFO:teuthology.orchestra.run.smithi094.stdout: "laggy_probability": 0, 2024-09-14T06:32:53.020 INFO:teuthology.orchestra.run.smithi094.stdout: "laggy_interval": 0, 2024-09-14T06:32:53.020 INFO:teuthology.orchestra.run.smithi094.stdout: "features": 4540719139924082687, 2024-09-14T06:32:53.020 INFO:teuthology.orchestra.run.smithi094.stdout: "old_weight": 0, 2024-09-14T06:32:53.020 INFO:teuthology.orchestra.run.smithi094.stdout: "last_purged_snaps_scrub": "2024-09-14T06:28:39.906639+0000", 2024-09-14T06:32:53.020 INFO:teuthology.orchestra.run.smithi094.stdout: "dead_epoch": 0 2024-09-14T06:32:53.020 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:53.020 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:53.021 INFO:teuthology.orchestra.run.smithi094.stdout: "osd": 1, 2024-09-14T06:32:53.021 INFO:teuthology.orchestra.run.smithi094.stdout: "down_stamp": "0.000000", 2024-09-14T06:32:53.021 INFO:teuthology.orchestra.run.smithi094.stdout: "laggy_probability": 0, 2024-09-14T06:32:53.021 INFO:teuthology.orchestra.run.smithi094.stdout: "laggy_interval": 0, 2024-09-14T06:32:53.021 INFO:teuthology.orchestra.run.smithi094.stdout: "features": 4540719139924082687, 2024-09-14T06:32:53.021 INFO:teuthology.orchestra.run.smithi094.stdout: "old_weight": 0, 2024-09-14T06:32:53.021 INFO:teuthology.orchestra.run.smithi094.stdout: "last_purged_snaps_scrub": "2024-09-14T06:28:40.541279+0000", 2024-09-14T06:32:53.021 INFO:teuthology.orchestra.run.smithi094.stdout: "dead_epoch": 0 2024-09-14T06:32:53.021 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:53.021 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:53.021 INFO:teuthology.orchestra.run.smithi094.stdout: "osd": 2, 2024-09-14T06:32:53.021 INFO:teuthology.orchestra.run.smithi094.stdout: "down_stamp": "0.000000", 2024-09-14T06:32:53.022 INFO:teuthology.orchestra.run.smithi094.stdout: "laggy_probability": 0, 2024-09-14T06:32:53.022 INFO:teuthology.orchestra.run.smithi094.stdout: "laggy_interval": 0, 2024-09-14T06:32:53.022 INFO:teuthology.orchestra.run.smithi094.stdout: "features": 4540719139924082687, 2024-09-14T06:32:53.022 INFO:teuthology.orchestra.run.smithi094.stdout: "old_weight": 0, 2024-09-14T06:32:53.022 INFO:teuthology.orchestra.run.smithi094.stdout: "last_purged_snaps_scrub": "2024-09-14T06:28:40.025350+0000", 2024-09-14T06:32:53.022 INFO:teuthology.orchestra.run.smithi094.stdout: "dead_epoch": 0 2024-09-14T06:32:53.022 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:53.022 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:53.022 INFO:teuthology.orchestra.run.smithi094.stdout: "osd": 3, 2024-09-14T06:32:53.022 INFO:teuthology.orchestra.run.smithi094.stdout: "down_stamp": "0.000000", 2024-09-14T06:32:53.022 INFO:teuthology.orchestra.run.smithi094.stdout: "laggy_probability": 0, 2024-09-14T06:32:53.022 INFO:teuthology.orchestra.run.smithi094.stdout: "laggy_interval": 0, 2024-09-14T06:32:53.022 INFO:teuthology.orchestra.run.smithi094.stdout: "features": 4540719139924082687, 2024-09-14T06:32:53.023 INFO:teuthology.orchestra.run.smithi094.stdout: "old_weight": 0, 2024-09-14T06:32:53.023 INFO:teuthology.orchestra.run.smithi094.stdout: "last_purged_snaps_scrub": "2024-09-14T06:28:39.217944+0000", 2024-09-14T06:32:53.023 INFO:teuthology.orchestra.run.smithi094.stdout: "dead_epoch": 0 2024-09-14T06:32:53.023 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:53.023 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:53.023 INFO:teuthology.orchestra.run.smithi094.stdout: "osd": 4, 2024-09-14T06:32:53.023 INFO:teuthology.orchestra.run.smithi094.stdout: "down_stamp": "0.000000", 2024-09-14T06:32:53.023 INFO:teuthology.orchestra.run.smithi094.stdout: "laggy_probability": 0, 2024-09-14T06:32:53.023 INFO:teuthology.orchestra.run.smithi094.stdout: "laggy_interval": 0, 2024-09-14T06:32:53.023 INFO:teuthology.orchestra.run.smithi094.stdout: "features": 4540719139924082687, 2024-09-14T06:32:53.023 INFO:teuthology.orchestra.run.smithi094.stdout: "old_weight": 0, 2024-09-14T06:32:53.023 INFO:teuthology.orchestra.run.smithi094.stdout: "last_purged_snaps_scrub": "2024-09-14T06:28:40.185898+0000", 2024-09-14T06:32:53.024 INFO:teuthology.orchestra.run.smithi094.stdout: "dead_epoch": 0 2024-09-14T06:32:53.024 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:53.024 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:53.024 INFO:teuthology.orchestra.run.smithi094.stdout: "osd": 5, 2024-09-14T06:32:53.024 INFO:teuthology.orchestra.run.smithi094.stdout: "down_stamp": "0.000000", 2024-09-14T06:32:53.024 INFO:teuthology.orchestra.run.smithi094.stdout: "laggy_probability": 0, 2024-09-14T06:32:53.024 INFO:teuthology.orchestra.run.smithi094.stdout: "laggy_interval": 0, 2024-09-14T06:32:53.024 INFO:teuthology.orchestra.run.smithi094.stdout: "features": 4540719139924082687, 2024-09-14T06:32:53.024 INFO:teuthology.orchestra.run.smithi094.stdout: "old_weight": 0, 2024-09-14T06:32:53.024 INFO:teuthology.orchestra.run.smithi094.stdout: "last_purged_snaps_scrub": "2024-09-14T06:28:40.783504+0000", 2024-09-14T06:32:53.024 INFO:teuthology.orchestra.run.smithi094.stdout: "dead_epoch": 0 2024-09-14T06:32:53.024 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:53.024 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:53.025 INFO:teuthology.orchestra.run.smithi094.stdout: "osd": 6, 2024-09-14T06:32:53.025 INFO:teuthology.orchestra.run.smithi094.stdout: "down_stamp": "0.000000", 2024-09-14T06:32:53.025 INFO:teuthology.orchestra.run.smithi094.stdout: "laggy_probability": 0, 2024-09-14T06:32:53.025 INFO:teuthology.orchestra.run.smithi094.stdout: "laggy_interval": 0, 2024-09-14T06:32:53.025 INFO:teuthology.orchestra.run.smithi094.stdout: "features": 4540719139924082687, 2024-09-14T06:32:53.025 INFO:teuthology.orchestra.run.smithi094.stdout: "old_weight": 65536, 2024-09-14T06:32:53.025 INFO:teuthology.orchestra.run.smithi094.stdout: "last_purged_snaps_scrub": "2024-09-14T06:28:40.527305+0000", 2024-09-14T06:32:53.025 INFO:teuthology.orchestra.run.smithi094.stdout: "dead_epoch": 0 2024-09-14T06:32:53.025 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:53.025 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:53.025 INFO:teuthology.orchestra.run.smithi094.stdout: "osd": 7, 2024-09-14T06:32:53.025 INFO:teuthology.orchestra.run.smithi094.stdout: "down_stamp": "0.000000", 2024-09-14T06:32:53.026 INFO:teuthology.orchestra.run.smithi094.stdout: "laggy_probability": 0, 2024-09-14T06:32:53.026 INFO:teuthology.orchestra.run.smithi094.stdout: "laggy_interval": 0, 2024-09-14T06:32:53.026 INFO:teuthology.orchestra.run.smithi094.stdout: "features": 4540719139924082687, 2024-09-14T06:32:53.026 INFO:teuthology.orchestra.run.smithi094.stdout: "old_weight": 0, 2024-09-14T06:32:53.026 INFO:teuthology.orchestra.run.smithi094.stdout: "last_purged_snaps_scrub": "2024-09-14T06:28:40.349676+0000", 2024-09-14T06:32:53.026 INFO:teuthology.orchestra.run.smithi094.stdout: "dead_epoch": 0 2024-09-14T06:32:53.026 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:53.026 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-09-14T06:32:53.026 INFO:teuthology.orchestra.run.smithi094.stdout: "pg_upmap": [], 2024-09-14T06:32:53.026 INFO:teuthology.orchestra.run.smithi094.stdout: "pg_upmap_items": [ 2024-09-14T06:32:53.026 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:53.026 INFO:teuthology.orchestra.run.smithi094.stdout: "pgid": "3.a", 2024-09-14T06:32:53.026 INFO:teuthology.orchestra.run.smithi094.stdout: "mappings": [ 2024-09-14T06:32:53.027 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:53.027 INFO:teuthology.orchestra.run.smithi094.stdout: "from": 1, 2024-09-14T06:32:53.027 INFO:teuthology.orchestra.run.smithi094.stdout: "to": 3 2024-09-14T06:32:53.027 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:53.027 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:53.027 INFO:teuthology.orchestra.run.smithi094.stdout: "from": 4, 2024-09-14T06:32:53.027 INFO:teuthology.orchestra.run.smithi094.stdout: "to": 2 2024-09-14T06:32:53.027 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:53.027 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:32:53.027 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:53.027 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:53.027 INFO:teuthology.orchestra.run.smithi094.stdout: "pgid": "3.e", 2024-09-14T06:32:53.028 INFO:teuthology.orchestra.run.smithi094.stdout: "mappings": [ 2024-09-14T06:32:53.028 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:53.028 INFO:teuthology.orchestra.run.smithi094.stdout: "from": 4, 2024-09-14T06:32:53.028 INFO:teuthology.orchestra.run.smithi094.stdout: "to": 1 2024-09-14T06:32:53.028 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:53.028 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:32:53.028 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:53.028 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:53.028 INFO:teuthology.orchestra.run.smithi094.stdout: "pgid": "3.11", 2024-09-14T06:32:53.028 INFO:teuthology.orchestra.run.smithi094.stdout: "mappings": [ 2024-09-14T06:32:53.028 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:53.029 INFO:teuthology.orchestra.run.smithi094.stdout: "from": 4, 2024-09-14T06:32:53.029 INFO:teuthology.orchestra.run.smithi094.stdout: "to": 1 2024-09-14T06:32:53.029 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:53.029 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:32:53.029 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:53.029 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:53.029 INFO:teuthology.orchestra.run.smithi094.stdout: "pgid": "3.14", 2024-09-14T06:32:53.029 INFO:teuthology.orchestra.run.smithi094.stdout: "mappings": [ 2024-09-14T06:32:53.029 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:53.030 INFO:teuthology.orchestra.run.smithi094.stdout: "from": 4, 2024-09-14T06:32:53.030 INFO:teuthology.orchestra.run.smithi094.stdout: "to": 2 2024-09-14T06:32:53.030 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:53.030 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:32:53.030 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:53.030 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:53.030 INFO:teuthology.orchestra.run.smithi094.stdout: "pgid": "3.1c", 2024-09-14T06:32:53.030 INFO:teuthology.orchestra.run.smithi094.stdout: "mappings": [ 2024-09-14T06:32:53.030 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:53.030 INFO:teuthology.orchestra.run.smithi094.stdout: "from": 4, 2024-09-14T06:32:53.030 INFO:teuthology.orchestra.run.smithi094.stdout: "to": 1 2024-09-14T06:32:53.031 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:53.031 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:32:53.031 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:53.031 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-09-14T06:32:53.031 INFO:teuthology.orchestra.run.smithi094.stdout: "pg_upmap_primaries": [], 2024-09-14T06:32:53.031 INFO:teuthology.orchestra.run.smithi094.stdout: "pg_temp": [], 2024-09-14T06:32:53.031 INFO:teuthology.orchestra.run.smithi094.stdout: "primary_temp": [], 2024-09-14T06:32:53.031 INFO:teuthology.orchestra.run.smithi094.stdout: "blocklist": {}, 2024-09-14T06:32:53.031 INFO:teuthology.orchestra.run.smithi094.stdout: "range_blocklist": {}, 2024-09-14T06:32:53.031 INFO:teuthology.orchestra.run.smithi094.stdout: "erasure_code_profiles": { 2024-09-14T06:32:53.031 INFO:teuthology.orchestra.run.smithi094.stdout: "default": { 2024-09-14T06:32:53.031 INFO:teuthology.orchestra.run.smithi094.stdout: "crush-failure-domain": "osd", 2024-09-14T06:32:53.032 INFO:teuthology.orchestra.run.smithi094.stdout: "k": "2", 2024-09-14T06:32:53.032 INFO:teuthology.orchestra.run.smithi094.stdout: "m": "1", 2024-09-14T06:32:53.032 INFO:teuthology.orchestra.run.smithi094.stdout: "plugin": "jerasure", 2024-09-14T06:32:53.032 INFO:teuthology.orchestra.run.smithi094.stdout: "technique": "reed_sol_van" 2024-09-14T06:32:53.032 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:53.032 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:53.032 INFO:teuthology.orchestra.run.smithi094.stdout: "removed_snaps_queue": [ 2024-09-14T06:32:53.032 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:53.032 INFO:teuthology.orchestra.run.smithi094.stdout: "pool": 3, 2024-09-14T06:32:53.032 INFO:teuthology.orchestra.run.smithi094.stdout: "snaps": [ 2024-09-14T06:32:53.032 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:53.032 INFO:teuthology.orchestra.run.smithi094.stdout: "begin": 27, 2024-09-14T06:32:53.033 INFO:teuthology.orchestra.run.smithi094.stdout: "length": 1 2024-09-14T06:32:53.033 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:53.033 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:53.033 INFO:teuthology.orchestra.run.smithi094.stdout: "begin": 29, 2024-09-14T06:32:53.033 INFO:teuthology.orchestra.run.smithi094.stdout: "length": 1 2024-09-14T06:32:53.033 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:53.033 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:53.033 INFO:teuthology.orchestra.run.smithi094.stdout: "begin": 31, 2024-09-14T06:32:53.033 INFO:teuthology.orchestra.run.smithi094.stdout: "length": 1 2024-09-14T06:32:53.033 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:53.033 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:53.033 INFO:teuthology.orchestra.run.smithi094.stdout: "begin": 33, 2024-09-14T06:32:53.034 INFO:teuthology.orchestra.run.smithi094.stdout: "length": 1 2024-09-14T06:32:53.034 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:53.034 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:32:53.034 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:53.034 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-09-14T06:32:53.034 INFO:teuthology.orchestra.run.smithi094.stdout: "new_removed_snaps": [ 2024-09-14T06:32:53.034 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:53.034 INFO:teuthology.orchestra.run.smithi094.stdout: "pool": 3, 2024-09-14T06:32:53.034 INFO:teuthology.orchestra.run.smithi094.stdout: "snaps": [ 2024-09-14T06:32:53.034 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:53.034 INFO:teuthology.orchestra.run.smithi094.stdout: "begin": 27, 2024-09-14T06:32:53.034 INFO:teuthology.orchestra.run.smithi094.stdout: "length": 1 2024-09-14T06:32:53.034 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:53.035 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:32:53.035 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:53.035 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-09-14T06:32:53.035 INFO:teuthology.orchestra.run.smithi094.stdout: "new_purged_snaps": [], 2024-09-14T06:32:53.035 INFO:teuthology.orchestra.run.smithi094.stdout: "crush_node_flags": {}, 2024-09-14T06:32:53.035 INFO:teuthology.orchestra.run.smithi094.stdout: "device_class_flags": {}, 2024-09-14T06:32:53.035 INFO:teuthology.orchestra.run.smithi094.stdout: "stretch_mode": { 2024-09-14T06:32:53.035 INFO:teuthology.orchestra.run.smithi094.stdout: "stretch_mode_enabled": false, 2024-09-14T06:32:53.035 INFO:teuthology.orchestra.run.smithi094.stdout: "stretch_bucket_count": 0, 2024-09-14T06:32:53.035 INFO:teuthology.orchestra.run.smithi094.stdout: "degraded_stretch_mode": 0, 2024-09-14T06:32:53.035 INFO:teuthology.orchestra.run.smithi094.stdout: "recovering_stretch_mode": 0, 2024-09-14T06:32:53.035 INFO:teuthology.orchestra.run.smithi094.stdout: "stretch_mode_bucket": 0 2024-09-14T06:32:53.036 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:53.036 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-14T06:32:53.038 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-14T06:32:53.038 INFO:tasks.ceph.osd.7.smithi138.stderr:2024-09-14T06:32:53.036+0000 7f3c6a21d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33623) UID: 0 2024-09-14T06:32:53.039 INFO:tasks.thrashosds.thrasher:j is {'epoch': 247, 'fsid': '200ad3da-bd63-4e04-af74-efb5ca0af75e', 'created': '2024-09-14T06:28:30.782046+0000', 'modified': '2024-09-14T06:32:52.628499+0000', 'last_up_change': '2024-09-14T06:28:42.827239+0000', 'last_in_change': '2024-09-14T06:32:00.601027+0000', 'flags': 'noscrub,nodeep-scrub,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit', 'flags_num': 5806080, 'flags_set': ['nodeep-scrub', 'noscrub', 'pglog_hardlimit', 'purged_snapdirs', 'recovery_deletes', 'sortbitwise'], 'crush_version': 5, 'full_ratio': 0.949999988079071, 'backfillfull_ratio': 0.8999999761581421, 'nearfull_ratio': 0.8500000238418579, 'cluster_snapshot': '', 'pool_max': 3, 'max_osd': 8, 'require_min_compat_client': 'luminous', 'min_compat_client': 'luminous', 'require_osd_release': 'squid', 'allow_crimson': False, 'pools': [{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-14T06:28:43.295990+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 6.900000095367432, 'score_stable': 6.900000095367432, 'optimal_score': 0.28999999165534973, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 0.8551368713378906, 'average_primary_affinity': 0.9800000190734863, 'average_primary_affinity_weighted': 0.8600000143051147}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2024-09-14T06:28:49.451714+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 2, 'snap_epoch': 22, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.630000114440918, 'score_stable': 2.630000114440918, 'optimal_score': 1, 'raw_score_acting': 2.630000114440918, 'raw_score_stable': 2.630000114440918, 'primary_affinity_weighted': 0.8551368713378906, 'average_primary_affinity': 0.9800000190734863, 'average_primary_affinity_weighted': 0.8600000143051147}}, {'pool': 3, 'pool_name': 'pool_client.0', 'create_time': '2024-09-14T06:28:59.688535+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '247', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 92, 'snap_epoch': 247, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 0.8551368713378906, 'average_primary_affinity': 0.9800000190734863, 'average_primary_affinity_weighted': 0.8600000143051147}}], 'osds': [{'osd': 0, 'uuid': '8b22a249-b842-406a-a02c-c0f31f57b635', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 199, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.94:6824', 'nonce': 1809564923}, {'type': 'v1', 'addr': '172.21.15.94:6825', 'nonce': 1809564923}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.94:6826', 'nonce': 1809564923}, {'type': 'v1', 'addr': '172.21.15.94:6827', 'nonce': 1809564923}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.94:6830', 'nonce': 1809564923}, {'type': 'v1', 'addr': '172.21.15.94:6831', 'nonce': 1809564923}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.94:6828', 'nonce': 1809564923}, {'type': 'v1', 'addr': '172.21.15.94:6829', 'nonce': 1809564923}]}, 'public_addr': '172.21.15.94:6825/1809564923', 'cluster_addr': '172.21.15.94:6827/1809564923', 'heartbeat_back_addr': '172.21.15.94:6831/1809564923', 'heartbeat_front_addr': '172.21.15.94:6829/1809564923', 'state': ['exists', 'up']}, {'osd': 1, 'uuid': '02251a03-84a0-414d-a980-fac4fd043cb6', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 0.841094970703125, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 244, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.94:6816', 'nonce': 736402652}, {'type': 'v1', 'addr': '172.21.15.94:6817', 'nonce': 736402652}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.94:6818', 'nonce': 736402652}, {'type': 'v1', 'addr': '172.21.15.94:6819', 'nonce': 736402652}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.94:6822', 'nonce': 736402652}, {'type': 'v1', 'addr': '172.21.15.94:6823', 'nonce': 736402652}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.94:6820', 'nonce': 736402652}, {'type': 'v1', 'addr': '172.21.15.94:6821', 'nonce': 736402652}]}, 'public_addr': '172.21.15.94:6817/736402652', 'cluster_addr': '172.21.15.94:6819/736402652', 'heartbeat_back_addr': '172.21.15.94:6823/736402652', 'heartbeat_front_addr': '172.21.15.94:6821/736402652', 'state': ['exists', 'up']}, {'osd': 2, 'uuid': 'd45bf470-bb29-4739-8294-cd122b2b4102', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 245, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.94:6808', 'nonce': 1063204782}, {'type': 'v1', 'addr': '172.21.15.94:6809', 'nonce': 1063204782}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.94:6810', 'nonce': 1063204782}, {'type': 'v1', 'addr': '172.21.15.94:6811', 'nonce': 1063204782}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.94:6814', 'nonce': 1063204782}, {'type': 'v1', 'addr': '172.21.15.94:6815', 'nonce': 1063204782}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.94:6812', 'nonce': 1063204782}, {'type': 'v1', 'addr': '172.21.15.94:6813', 'nonce': 1063204782}]}, 'public_addr': '172.21.15.94:6809/1063204782', 'cluster_addr': '172.21.15.94:6811/1063204782', 'heartbeat_back_addr': '172.21.15.94:6815/1063204782', 'heartbeat_front_addr': '172.21.15.94:6813/1063204782', 'state': ['exists', 'up']}, {'osd': 3, 'uuid': '456ec2d9-c438-4243-b931-335704f661eb', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 218, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.94:6800', 'nonce': 3382046754}, {'type': 'v1', 'addr': '172.21.15.94:6801', 'nonce': 3382046754}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.94:6802', 'nonce': 3382046754}, {'type': 'v1', 'addr': '172.21.15.94:6803', 'nonce': 3382046754}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.94:6806', 'nonce': 3382046754}, {'type': 'v1', 'addr': '172.21.15.94:6807', 'nonce': 3382046754}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.94:6804', 'nonce': 3382046754}, {'type': 'v1', 'addr': '172.21.15.94:6805', 'nonce': 3382046754}]}, 'public_addr': '172.21.15.94:6801/3382046754', 'cluster_addr': '172.21.15.94:6803/3382046754', 'heartbeat_back_addr': '172.21.15.94:6807/3382046754', 'heartbeat_front_addr': '172.21.15.94:6805/3382046754', 'state': ['exists', 'up']}, {'osd': 4, 'uuid': '2dab948e-c6e4-4437-bf17-017d1ac6821c', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 232, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.138:6810', 'nonce': 501726729}, {'type': 'v1', 'addr': '172.21.15.138:6811', 'nonce': 501726729}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.138:6812', 'nonce': 501726729}, {'type': 'v1', 'addr': '172.21.15.138:6813', 'nonce': 501726729}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.138:6816', 'nonce': 501726729}, {'type': 'v1', 'addr': '172.21.15.138:6817', 'nonce': 501726729}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.138:6814', 'nonce': 501726729}, {'type': 'v1', 'addr': '172.21.15.138:6815', 'nonce': 501726729}]}, 'public_addr': '172.21.15.138:6811/501726729', 'cluster_addr': '172.21.15.138:6813/501726729', 'heartbeat_back_addr': '172.21.15.138:6817/501726729', 'heartbeat_front_addr': '172.21.15.138:6815/501726729', 'state': ['exists', 'up']}, {'osd': 5, 'uuid': '8caf3aa3-ad87-4d33-b82e-4c15195176fd', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 0, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 199, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.138:6826', 'nonce': 1599292735}, {'type': 'v1', 'addr': '172.21.15.138:6827', 'nonce': 1599292735}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.138:6828', 'nonce': 1599292735}, {'type': 'v1', 'addr': '172.21.15.138:6829', 'nonce': 1599292735}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.138:6832', 'nonce': 1599292735}, {'type': 'v1', 'addr': '172.21.15.138:6833', 'nonce': 1599292735}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.138:6830', 'nonce': 1599292735}, {'type': 'v1', 'addr': '172.21.15.138:6831', 'nonce': 1599292735}]}, 'public_addr': '172.21.15.138:6827/1599292735', 'cluster_addr': '172.21.15.138:6829/1599292735', 'heartbeat_back_addr': '172.21.15.138:6833/1599292735', 'heartbeat_front_addr': '172.21.15.138:6831/1599292735', 'state': ['exists', 'up']}, {'osd': 6, 'uuid': 'e55d11fd-433d-4462-93a7-d5e84560221b', 'up': 1, 'in': 0, 'weight': 0, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 176, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.138:6818', 'nonce': 163531479}, {'type': 'v1', 'addr': '172.21.15.138:6819', 'nonce': 163531479}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.138:6820', 'nonce': 163531479}, {'type': 'v1', 'addr': '172.21.15.138:6821', 'nonce': 163531479}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.138:6824', 'nonce': 163531479}, {'type': 'v1', 'addr': '172.21.15.138:6825', 'nonce': 163531479}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.138:6822', 'nonce': 163531479}, {'type': 'v1', 'addr': '172.21.15.138:6823', 'nonce': 163531479}]}, 'public_addr': '172.21.15.138:6819/163531479', 'cluster_addr': '172.21.15.138:6821/163531479', 'heartbeat_back_addr': '172.21.15.138:6825/163531479', 'heartbeat_front_addr': '172.21.15.138:6823/163531479', 'state': ['exists', 'up']}, {'osd': 7, 'uuid': 'ba14ba69-2c6c-4d09-81e2-2593dc3a7ea9', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 233, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.138:6800', 'nonce': 218507235}, {'type': 'v1', 'addr': '172.21.15.138:6801', 'nonce': 218507235}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.138:6802', 'nonce': 218507235}, {'type': 'v1', 'addr': '172.21.15.138:6803', 'nonce': 218507235}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.138:6806', 'nonce': 218507235}, {'type': 'v1', 'addr': '172.21.15.138:6807', 'nonce': 218507235}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.138:6804', 'nonce': 218507235}, {'type': 'v1', 'addr': '172.21.15.138:6805', 'nonce': 218507235}]}, 'public_addr': '172.21.15.138:6801/218507235', 'cluster_addr': '172.21.15.138:6803/218507235', 'heartbeat_back_addr': '172.21.15.138:6807/218507235', 'heartbeat_front_addr': '172.21.15.138:6805/218507235', 'state': ['exists', 'up']}], 'osd_xinfo': [{'osd': 0, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-14T06:28:39.906639+0000', 'dead_epoch': 0}, {'osd': 1, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-14T06:28:40.541279+0000', 'dead_epoch': 0}, {'osd': 2, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-14T06:28:40.025350+0000', 'dead_epoch': 0}, {'osd': 3, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-14T06:28:39.217944+0000', 'dead_epoch': 0}, {'osd': 4, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-14T06:28:40.185898+0000', 'dead_epoch': 0}, {'osd': 5, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-14T06:28:40.783504+0000', 'dead_epoch': 0}, {'osd': 6, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 65536, 'last_purged_snaps_scrub': '2024-09-14T06:28:40.527305+0000', 'dead_epoch': 0}, {'osd': 7, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-14T06:28:40.349676+0000', 'dead_epoch': 0}], 'pg_upmap': [], 'pg_upmap_items': [{'pgid': '3.a', 'mappings': [{'from': 1, 'to': 3}, {'from': 4, 'to': 2}]}, {'pgid': '3.e', 'mappings': [{'from': 4, 'to': 1}]}, {'pgid': '3.11', 'mappings': [{'from': 4, 'to': 1}]}, {'pgid': '3.14', 'mappings': [{'from': 4, 'to': 2}]}, {'pgid': '3.1c', 'mappings': [{'from': 4, 'to': 1}]}], 'pg_upmap_primaries': [], 'pg_temp': [], 'primary_temp': [], 'blocklist': {}, 'range_blocklist': {}, 'erasure_code_profiles': {'default': {'crush-failure-domain': 'osd', 'k': '2', 'm': '1', 'plugin': 'jerasure', 'technique': 'reed_sol_van'}}, 'removed_snaps_queue': [{'pool': 3, 'snaps': [{'begin': 27, 'length': 1}, {'begin': 29, 'length': 1}, {'begin': 31, 'length': 1}, {'begin': 33, 'length': 1}]}], 'new_removed_snaps': [{'pool': 3, 'snaps': [{'begin': 27, 'length': 1}]}], 'new_purged_snaps': [], 'crush_node_flags': {}, 'device_class_flags': {}, 'stretch_mode': {'stretch_mode_enabled': False, 'stretch_bucket_count': 0, 'degraded_stretch_mode': 0, 'recovering_stretch_mode': 0, 'stretch_mode_bucket': 0}} 2024-09-14T06:32:53.039 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-14T06:32:53.138 INFO:tasks.ceph.osd.2.smithi094.stderr:2024-09-14T06:32:53.136+0000 7faf3269b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33649) UID: 0 2024-09-14T06:32:53.181 INFO:teuthology.orchestra.run.smithi138.stdout:{ 2024-09-14T06:32:53.181 INFO:teuthology.orchestra.run.smithi138.stdout: "ops": [], 2024-09-14T06:32:53.181 INFO:teuthology.orchestra.run.smithi138.stdout: "complaint_time": 30, 2024-09-14T06:32:53.181 INFO:teuthology.orchestra.run.smithi138.stdout: "num_blocked_ops": 0 2024-09-14T06:32:53.181 INFO:teuthology.orchestra.run.smithi138.stdout:} 2024-09-14T06:32:53.190 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-14T06:32:53.239 INFO:tasks.ceph.osd.0.smithi094.stderr:2024-09-14T06:32:53.238+0000 7f07c9333640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33636) UID: 0 2024-09-14T06:32:53.294 INFO:teuthology.orchestra.run.smithi138.stdout:{ 2024-09-14T06:32:53.294 INFO:teuthology.orchestra.run.smithi138.stdout: "size": 20, 2024-09-14T06:32:53.294 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 600, 2024-09-14T06:32:53.294 INFO:teuthology.orchestra.run.smithi138.stdout: "ops": [ 2024-09-14T06:32:53.294 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:53.294 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:11453 3.1d 3:bcad7303:::rbd_data.112d92027590.0000000000000001:f [read 94208~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:32:53.294 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:25.654375+0000", 2024-09-14T06:32:53.295 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 207.637873896, 2024-09-14T06:32:53.295 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052449318000000002, 2024-09-14T06:32:53.295 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:32:53.295 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:32:53.295 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:32:53.295 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:32:53.295 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:32:53.295 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:53.295 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 11453 2024-09-14T06:32:53.295 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:53.295 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:32:53.295 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:53.296 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:32:53.296 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654375+0000", 2024-09-14T06:32:53.296 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:53.296 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:53.296 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:53.296 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:32:53.296 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654375+0000", 2024-09-14T06:32:53.296 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:53.296 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:53.296 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:53.296 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:32:53.296 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654496+0000", 2024-09-14T06:32:53.297 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00012081 2024-09-14T06:32:53.297 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:53.297 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:53.297 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:32:53.297 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654497+0000", 2024-09-14T06:32:53.297 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.8789999999999999e-06 2024-09-14T06:32:53.297 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:53.297 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:53.297 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:32:53.297 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654501+0000", 2024-09-14T06:32:53.297 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.8999999999999999e-06 2024-09-14T06:32:53.297 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:53.298 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:53.298 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:32:53.298 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654505+0000", 2024-09-14T06:32:53.298 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.2229999999999999e-06 2024-09-14T06:32:53.298 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:53.298 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:53.298 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:32:53.298 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.705917+0000", 2024-09-14T06:32:53.298 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.051412501999999999 2024-09-14T06:32:53.298 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:53.298 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:53.298 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:32:53.299 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.705999+0000", 2024-09-14T06:32:53.299 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.1551999999999994e-05 2024-09-14T06:32:53.299 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:53.299 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:53.299 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:32:53.299 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.706824+0000", 2024-09-14T06:32:53.299 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00082545199999999996 2024-09-14T06:32:53.299 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:53.299 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:32:53.299 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:53.299 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:53.300 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:53.300 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:11454 3.1d 3:bcad7303:::rbd_data.112d92027590.0000000000000001:f [read 104448~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:32:53.300 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:25.654510+0000", 2024-09-14T06:32:53.300 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 207.637739106, 2024-09-14T06:32:53.300 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052610713000000003, 2024-09-14T06:32:53.300 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:32:53.300 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:32:53.300 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:32:53.300 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:32:53.300 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:32:53.300 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:53.301 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 11454 2024-09-14T06:32:53.301 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:53.301 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:32:53.301 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:53.301 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:32:53.301 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654510+0000", 2024-09-14T06:32:53.301 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:53.301 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:53.301 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:53.301 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:32:53.301 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654510+0000", 2024-09-14T06:32:53.301 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:53.302 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:53.302 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:53.302 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:32:53.302 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654825+0000", 2024-09-14T06:32:53.302 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00031522400000000002 2024-09-14T06:32:53.302 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:53.302 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:53.302 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:32:53.302 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654827+0000", 2024-09-14T06:32:53.302 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.9709999999999998e-06 2024-09-14T06:32:53.302 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:53.302 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:53.303 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:32:53.303 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654832+0000", 2024-09-14T06:32:53.303 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.4099999999999999e-06 2024-09-14T06:32:53.303 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:53.303 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:53.303 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:32:53.303 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654835+0000", 2024-09-14T06:32:53.303 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.14e-06 2024-09-14T06:32:53.303 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:53.303 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:53.303 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:32:53.303 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.706851+0000", 2024-09-14T06:32:53.304 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052015761000000001 2024-09-14T06:32:53.304 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:53.304 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:53.304 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:32:53.304 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.706934+0000", 2024-09-14T06:32:53.304 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.2484999999999994e-05 2024-09-14T06:32:53.304 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:53.304 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:53.304 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:32:53.304 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.707120+0000", 2024-09-14T06:32:53.304 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00018672200000000001 2024-09-14T06:32:53.304 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:53.305 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:32:53.305 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:53.305 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:53.305 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:53.305 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:11455 3.1d 3:bcad7303:::rbd_data.112d92027590.0000000000000001:f [read 114688~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:32:53.305 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:25.654841+0000", 2024-09-14T06:32:53.305 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 207.63740777699999, 2024-09-14T06:32:53.305 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052579898, 2024-09-14T06:32:53.305 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:32:53.305 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:32:53.305 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:32:53.305 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:32:53.306 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:32:53.306 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:53.306 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 11455 2024-09-14T06:32:53.306 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:53.306 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:32:53.306 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:53.306 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:32:53.306 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654841+0000", 2024-09-14T06:32:53.306 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:53.306 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:53.306 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:53.307 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:32:53.307 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654841+0000", 2024-09-14T06:32:53.307 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:53.307 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:53.307 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:53.307 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:32:53.307 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654841+0000", 2024-09-14T06:32:53.307 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.7300000000000002e-07 2024-09-14T06:32:53.307 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:53.307 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:53.307 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:32:53.307 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654845+0000", 2024-09-14T06:32:53.307 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.0520000000000003e-06 2024-09-14T06:32:53.308 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:53.308 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:53.308 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:32:53.308 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654849+0000", 2024-09-14T06:32:53.308 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.0459999999999999e-06 2024-09-14T06:32:53.308 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:53.308 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:53.308 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:32:53.308 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654852+0000", 2024-09-14T06:32:53.308 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.9830000000000001e-06 2024-09-14T06:32:53.308 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:53.308 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:53.309 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:32:53.309 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.707159+0000", 2024-09-14T06:32:53.309 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052306826000000001 2024-09-14T06:32:53.309 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:53.309 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:53.309 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:32:53.309 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.707242+0000", 2024-09-14T06:32:53.309 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.2595000000000005e-05 2024-09-14T06:32:53.309 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:53.309 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:53.310 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:32:53.310 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.707421+0000", 2024-09-14T06:32:53.310 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.000179123 2024-09-14T06:32:53.310 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:53.310 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:32:53.310 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:53.310 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:53.310 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:53.310 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:11456 3.1d 3:bcad7303:::rbd_data.112d92027590.0000000000000001:f [read 124928~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:32:53.310 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:25.654856+0000", 2024-09-14T06:32:53.310 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 207.63739293099999, 2024-09-14T06:32:53.310 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052801580000000001, 2024-09-14T06:32:53.311 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:32:53.311 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:32:53.311 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:32:53.311 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:32:53.311 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:32:53.311 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:53.311 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 11456 2024-09-14T06:32:53.311 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:53.311 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:32:53.311 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:53.311 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:32:53.312 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654856+0000", 2024-09-14T06:32:53.312 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:53.312 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:53.312 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:53.312 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:32:53.312 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654856+0000", 2024-09-14T06:32:53.312 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:53.312 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:53.312 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:53.312 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:32:53.312 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654856+0000", 2024-09-14T06:32:53.313 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.29e-07 2024-09-14T06:32:53.313 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:53.313 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:53.313 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:32:53.313 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654857+0000", 2024-09-14T06:32:53.313 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.187e-06 2024-09-14T06:32:53.313 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:53.313 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:53.313 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:32:53.313 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654861+0000", 2024-09-14T06:32:53.313 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.3850000000000002e-06 2024-09-14T06:32:53.313 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:53.314 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:53.314 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:32:53.314 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654863+0000", 2024-09-14T06:32:53.314 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.9500000000000001e-06 2024-09-14T06:32:53.314 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:53.314 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:53.314 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:32:53.314 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.707456+0000", 2024-09-14T06:32:53.314 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052592434 2024-09-14T06:32:53.314 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:53.315 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:53.315 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:32:53.315 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.707516+0000", 2024-09-14T06:32:53.315 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.0167999999999997e-05 2024-09-14T06:32:53.315 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:53.315 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:53.315 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:32:53.315 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.707657+0000", 2024-09-14T06:32:53.315 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00014122700000000001 2024-09-14T06:32:53.315 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:53.315 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:32:53.315 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:53.316 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:53.316 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:53.316 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:11457 3.1d 3:bcad7303:::rbd_data.112d92027590.0000000000000001:f [read 135168~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:32:53.316 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:25.654867+0000", 2024-09-14T06:32:53.316 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 207.637382046, 2024-09-14T06:32:53.316 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052921902, 2024-09-14T06:32:53.316 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:32:53.316 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:32:53.316 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:32:53.316 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:32:53.316 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:32:53.317 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:53.317 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 11457 2024-09-14T06:32:53.317 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:53.317 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:32:53.317 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:53.317 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:32:53.317 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654867+0000", 2024-09-14T06:32:53.317 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:53.317 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:53.317 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:53.318 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:32:53.318 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654867+0000", 2024-09-14T06:32:53.318 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:53.318 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:53.318 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:53.318 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:32:53.318 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654972+0000", 2024-09-14T06:32:53.318 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00010584 2024-09-14T06:32:53.318 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:53.318 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:53.318 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:32:53.318 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654975+0000", 2024-09-14T06:32:53.319 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.1169999999999998e-06 2024-09-14T06:32:53.319 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:53.319 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:53.319 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:32:53.319 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654979+0000", 2024-09-14T06:32:53.319 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.1960000000000001e-06 2024-09-14T06:32:53.319 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:53.319 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:53.319 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:32:53.319 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654982+0000", 2024-09-14T06:32:53.319 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.2990000000000001e-06 2024-09-14T06:32:53.319 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:53.320 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:53.320 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:32:53.320 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.707684+0000", 2024-09-14T06:32:53.320 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052701979000000003 2024-09-14T06:32:53.320 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:53.320 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:53.320 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:32:53.320 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.707728+0000", 2024-09-14T06:32:53.321 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.3529999999999998e-05 2024-09-14T06:32:53.321 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:53.321 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:53.321 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:32:53.321 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.707789+0000", 2024-09-14T06:32:53.321 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.0940999999999998e-05 2024-09-14T06:32:53.321 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:53.321 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:32:53.321 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:53.321 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:53.321 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:53.322 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:11458 3.1d 3:bcad7303:::rbd_data.112d92027590.0000000000000001:f [read 145408~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:32:53.322 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:25.654987+0000", 2024-09-14T06:32:53.322 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 207.63726164799999, 2024-09-14T06:32:53.322 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.05291655, 2024-09-14T06:32:53.322 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:32:53.322 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:32:53.322 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:32:53.322 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:32:53.322 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:32:53.322 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:53.322 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 11458 2024-09-14T06:32:53.322 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:53.323 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:32:53.323 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:53.323 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:32:53.323 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654987+0000", 2024-09-14T06:32:53.323 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:53.323 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:53.323 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:53.323 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:32:53.323 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654987+0000", 2024-09-14T06:32:53.323 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:53.323 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:53.323 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:53.324 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:32:53.324 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655077+0000", 2024-09-14T06:32:53.324 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.0253000000000005e-05 2024-09-14T06:32:53.324 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:53.324 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:53.324 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:32:53.324 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655079+0000", 2024-09-14T06:32:53.324 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.9989999999999998e-06 2024-09-14T06:32:53.324 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:53.324 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:53.324 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:32:53.324 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655083+0000", 2024-09-14T06:32:53.325 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.9280000000000003e-06 2024-09-14T06:32:53.325 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:53.325 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:53.325 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:32:53.325 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655086+0000", 2024-09-14T06:32:53.325 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.056e-06 2024-09-14T06:32:53.325 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:53.325 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:53.325 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:32:53.325 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.707805+0000", 2024-09-14T06:32:53.325 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052718627999999997 2024-09-14T06:32:53.325 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:53.326 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:53.326 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:32:53.326 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.707848+0000", 2024-09-14T06:32:53.326 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.2728999999999997e-05 2024-09-14T06:32:53.326 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:53.326 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:53.326 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:32:53.328 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.707904+0000", 2024-09-14T06:32:53.328 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.5957e-05 2024-09-14T06:32:53.328 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:53.328 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:32:53.328 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:53.328 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:53.328 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:53.328 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:11459 3.1d 3:bcad7303:::rbd_data.112d92027590.0000000000000001:f [read 155648~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:32:53.328 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:25.655091+0000", 2024-09-14T06:32:53.328 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 207.63715764, 2024-09-14T06:32:53.328 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052944242000000002, 2024-09-14T06:32:53.329 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:32:53.329 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:32:53.329 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:32:53.329 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:32:53.329 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:32:53.329 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:53.329 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 11459 2024-09-14T06:32:53.329 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:53.329 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:32:53.329 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:53.329 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:32:53.330 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655091+0000", 2024-09-14T06:32:53.330 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:53.330 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:53.330 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:53.330 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:32:53.330 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655091+0000", 2024-09-14T06:32:53.330 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:53.330 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:53.330 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:53.330 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:32:53.330 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655237+0000", 2024-09-14T06:32:53.330 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00014570100000000001 2024-09-14T06:32:53.331 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:53.331 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:53.331 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:32:53.331 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655239+0000", 2024-09-14T06:32:53.331 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.8670000000000001e-06 2024-09-14T06:32:53.331 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:53.331 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:53.331 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:32:53.331 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655243+0000", 2024-09-14T06:32:53.331 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.9829999999999998e-06 2024-09-14T06:32:53.331 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:53.331 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:53.332 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:32:53.332 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655246+0000", 2024-09-14T06:32:53.332 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.1540000000000002e-06 2024-09-14T06:32:53.332 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:53.332 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:53.332 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:32:53.332 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.707934+0000", 2024-09-14T06:32:53.332 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052688286000000001 2024-09-14T06:32:53.332 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:53.332 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:53.332 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:32:53.333 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.707977+0000", 2024-09-14T06:32:53.333 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.3291999999999998e-05 2024-09-14T06:32:53.333 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:53.333 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:53.333 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:32:53.333 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.708035+0000", 2024-09-14T06:32:53.333 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.7958999999999999e-05 2024-09-14T06:32:53.333 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:53.333 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:32:53.333 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:53.333 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:53.333 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:53.334 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:11460 3.1d 3:bcad7303:::rbd_data.112d92027590.0000000000000001:f [read 165888~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:32:53.334 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:25.655250+0000", 2024-09-14T06:32:53.334 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 207.63699834299999, 2024-09-14T06:32:53.334 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052910698999999999, 2024-09-14T06:32:53.334 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:32:53.334 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:32:53.334 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:32:53.334 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:32:53.334 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:32:53.334 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:53.334 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 11460 2024-09-14T06:32:53.334 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:53.335 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:32:53.335 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:53.335 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:32:53.335 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655250+0000", 2024-09-14T06:32:53.335 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:53.335 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:53.335 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:53.335 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:32:53.335 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655250+0000", 2024-09-14T06:32:53.335 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:53.335 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:53.336 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:53.336 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:32:53.336 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655356+0000", 2024-09-14T06:32:53.336 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00010535900000000001 2024-09-14T06:32:53.336 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:53.336 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:53.336 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:32:53.336 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655357+0000", 2024-09-14T06:32:53.336 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.8080000000000001e-06 2024-09-14T06:32:53.337 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:53.337 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:53.337 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:32:53.337 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655361+0000", 2024-09-14T06:32:53.337 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.861e-06 2024-09-14T06:32:53.337 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:53.337 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:53.337 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:32:53.337 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655364+0000", 2024-09-14T06:32:53.337 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.1089999999999998e-06 2024-09-14T06:32:53.338 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:53.338 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:53.338 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:32:53.338 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.708065+0000", 2024-09-14T06:32:53.338 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052700595000000003 2024-09-14T06:32:53.338 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:53.338 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:53.338 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:32:53.338 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.708108+0000", 2024-09-14T06:32:53.339 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.2484999999999997e-05 2024-09-14T06:32:53.339 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:53.339 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:53.339 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:32:53.339 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.708161+0000", 2024-09-14T06:32:53.339 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.3482000000000001e-05 2024-09-14T06:32:53.339 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:53.339 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:32:53.339 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:53.339 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:53.339 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:53.339 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:11461 3.1d 3:bcad7303:::rbd_data.112d92027590.0000000000000001:f [read 176128~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:32:53.340 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:25.655370+0000", 2024-09-14T06:32:53.340 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 207.636879061, 2024-09-14T06:32:53.340 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052915141999999998, 2024-09-14T06:32:53.340 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:32:53.340 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:32:53.340 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:32:53.340 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:32:53.340 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:32:53.340 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:53.340 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 11461 2024-09-14T06:32:53.340 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:53.340 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:32:53.341 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:53.341 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:32:53.341 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655370+0000", 2024-09-14T06:32:53.341 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:53.341 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:53.341 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:53.341 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:32:53.341 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655370+0000", 2024-09-14T06:32:53.341 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:53.341 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:53.342 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:53.342 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:32:53.342 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655394+0000", 2024-09-14T06:32:53.342 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.4553999999999999e-05 2024-09-14T06:32:53.342 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:53.342 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:53.342 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:32:53.342 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655396+0000", 2024-09-14T06:32:53.342 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.8950000000000001e-06 2024-09-14T06:32:53.342 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:53.342 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:53.342 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:32:53.343 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655401+0000", 2024-09-14T06:32:53.343 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.0089999999999998e-06 2024-09-14T06:32:53.343 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:53.343 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:53.343 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:32:53.343 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655404+0000", 2024-09-14T06:32:53.343 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.3400000000000002e-06 2024-09-14T06:32:53.343 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:53.343 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:53.343 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:32:53.343 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.708186+0000", 2024-09-14T06:32:53.343 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052781726000000001 2024-09-14T06:32:53.344 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:53.344 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:53.344 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:32:53.344 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.708229+0000", 2024-09-14T06:32:53.344 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.2670999999999999e-05 2024-09-14T06:32:53.344 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:53.344 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:53.344 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:32:53.344 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.708285+0000", 2024-09-14T06:32:53.344 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.5946999999999998e-05 2024-09-14T06:32:53.344 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:53.344 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:32:53.345 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:53.345 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:53.345 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:53.345 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:11462 3.1d 3:bcad7303:::rbd_data.112d92027590.0000000000000001:f [read 186368~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:32:53.345 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:25.655410+0000", 2024-09-14T06:32:53.345 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 207.63683858300001, 2024-09-14T06:32:53.345 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052995258000000003, 2024-09-14T06:32:53.345 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:32:53.345 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:32:53.345 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:32:53.345 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:32:53.345 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:32:53.346 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:53.346 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 11462 2024-09-14T06:32:53.346 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:53.346 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:32:53.346 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:53.346 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:32:53.346 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655410+0000", 2024-09-14T06:32:53.346 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:53.346 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:53.346 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:53.346 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:32:53.347 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655410+0000", 2024-09-14T06:32:53.347 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:53.347 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:53.347 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:53.347 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:32:53.347 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655625+0000", 2024-09-14T06:32:53.347 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00021442600000000001 2024-09-14T06:32:53.347 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:53.347 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:53.347 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:32:53.347 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655626+0000", 2024-09-14T06:32:53.347 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.815e-06 2024-09-14T06:32:53.348 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:53.348 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:53.348 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:32:53.348 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655631+0000", 2024-09-14T06:32:53.348 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.651e-06 2024-09-14T06:32:53.348 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:53.348 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:53.348 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:32:53.348 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655634+0000", 2024-09-14T06:32:53.348 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.2449999999999998e-06 2024-09-14T06:32:53.348 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:53.349 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:53.349 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:32:53.349 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.708312+0000", 2024-09-14T06:32:53.349 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052677412 2024-09-14T06:32:53.349 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:53.349 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:53.349 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:32:53.349 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.708352+0000", 2024-09-14T06:32:53.349 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.0729999999999998e-05 2024-09-14T06:32:53.349 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:53.349 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:53.350 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:32:53.350 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.708405+0000", 2024-09-14T06:32:53.350 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.2979000000000002e-05 2024-09-14T06:32:53.350 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:53.350 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:32:53.350 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:53.350 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:53.350 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:53.350 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:11463 3.1d 3:bcad7303:::rbd_data.112d92027590.0000000000000001:f [read 196608~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:32:53.350 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:25.655639+0000", 2024-09-14T06:32:53.350 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 207.636609437, 2024-09-14T06:32:53.350 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052888859000000003, 2024-09-14T06:32:53.351 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:32:53.351 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:32:53.351 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:32:53.351 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:32:53.351 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:32:53.351 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:53.351 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 11463 2024-09-14T06:32:53.351 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:53.351 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:32:53.351 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:53.351 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:32:53.352 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655639+0000", 2024-09-14T06:32:53.352 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:53.352 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:53.352 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:53.352 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:32:53.352 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655639+0000", 2024-09-14T06:32:53.352 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:53.352 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:53.352 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:53.352 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:32:53.352 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655640+0000", 2024-09-14T06:32:53.352 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.08e-07 2024-09-14T06:32:53.353 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:53.353 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:53.353 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:32:53.353 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655641+0000", 2024-09-14T06:32:53.353 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.156e-06 2024-09-14T06:32:53.353 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:53.353 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:53.353 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:32:53.353 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655648+0000", 2024-09-14T06:32:53.353 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.8460000000000002e-06 2024-09-14T06:32:53.353 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:53.354 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:53.354 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:32:53.354 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655651+0000", 2024-09-14T06:32:53.354 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.0079999999999998e-06 2024-09-14T06:32:53.354 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:53.354 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:53.354 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:32:53.354 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.708429+0000", 2024-09-14T06:32:53.354 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052778459 2024-09-14T06:32:53.354 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:53.354 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:53.354 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:32:53.354 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.708473+0000", 2024-09-14T06:32:53.355 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.3782999999999997e-05 2024-09-14T06:32:53.355 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:53.355 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:53.355 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:32:53.355 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.708528+0000", 2024-09-14T06:32:53.355 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.5198999999999998e-05 2024-09-14T06:32:53.355 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:53.355 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:32:53.355 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:53.355 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:53.355 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:53.355 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:11464 3.1d 3:bcad7303:::rbd_data.112d92027590.0000000000000001:f [read 206848~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:32:53.355 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:25.655654+0000", 2024-09-14T06:32:53.356 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 207.63659456799999, 2024-09-14T06:32:53.356 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052981845999999999, 2024-09-14T06:32:53.356 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:32:53.356 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:32:53.356 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:32:53.356 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:32:53.356 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:32:53.356 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:53.356 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 11464 2024-09-14T06:32:53.356 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:53.356 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:32:53.356 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:53.357 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:32:53.357 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655654+0000", 2024-09-14T06:32:53.357 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:53.357 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:53.357 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:53.357 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:32:53.357 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655654+0000", 2024-09-14T06:32:53.357 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:53.357 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:53.357 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:53.357 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:32:53.357 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655946+0000", 2024-09-14T06:32:53.358 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00029148200000000003 2024-09-14T06:32:53.358 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:53.358 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:53.358 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:32:53.358 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655948+0000", 2024-09-14T06:32:53.358 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.973e-06 2024-09-14T06:32:53.358 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:53.358 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:53.358 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:32:53.358 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655953+0000", 2024-09-14T06:32:53.358 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.1240000000000004e-06 2024-09-14T06:32:53.358 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:53.358 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:53.358 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:32:53.359 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655956+0000", 2024-09-14T06:32:53.359 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.315e-06 2024-09-14T06:32:53.359 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:53.359 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:53.359 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:32:53.359 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.708543+0000", 2024-09-14T06:32:53.359 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052587416999999997 2024-09-14T06:32:53.359 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:53.359 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:53.359 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:32:53.359 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.708584+0000", 2024-09-14T06:32:53.359 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.1004000000000002e-05 2024-09-14T06:32:53.360 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:53.360 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:53.360 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:32:53.360 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.708636+0000", 2024-09-14T06:32:53.360 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.1530999999999998e-05 2024-09-14T06:32:53.360 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:53.360 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:32:53.360 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:53.360 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:53.360 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:53.361 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:11465 3.1d 3:bcad7303:::rbd_data.112d92027590.0000000000000001:f [read 217088~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:32:53.361 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:25.655961+0000", 2024-09-14T06:32:53.361 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 207.63628741599999, 2024-09-14T06:32:53.361 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052798682999999999, 2024-09-14T06:32:53.361 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:32:53.361 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:32:53.361 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:32:53.361 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:32:53.361 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:32:53.361 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:53.361 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 11465 2024-09-14T06:32:53.361 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:53.362 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:32:53.362 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:53.362 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:32:53.362 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655961+0000", 2024-09-14T06:32:53.362 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:53.362 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:53.362 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:53.362 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:32:53.362 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655961+0000", 2024-09-14T06:32:53.362 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:53.362 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:53.362 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:53.362 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:32:53.363 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.656114+0000", 2024-09-14T06:32:53.363 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.000152745 2024-09-14T06:32:53.363 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:53.363 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:53.363 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:32:53.363 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.656116+0000", 2024-09-14T06:32:53.363 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.804e-06 2024-09-14T06:32:53.363 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:53.363 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:53.363 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:32:53.364 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.656120+0000", 2024-09-14T06:32:53.364 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.4700000000000004e-06 2024-09-14T06:32:53.364 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:53.364 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:53.364 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:32:53.364 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.656123+0000", 2024-09-14T06:32:53.364 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.1269999999999999e-06 2024-09-14T06:32:53.364 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:53.364 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:53.364 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:32:53.364 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.708660+0000", 2024-09-14T06:32:53.365 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052536919000000001 2024-09-14T06:32:53.365 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:53.365 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:53.365 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:32:53.365 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.708703+0000", 2024-09-14T06:32:53.365 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.3077000000000001e-05 2024-09-14T06:32:53.365 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:53.365 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:53.365 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:32:53.365 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.708760+0000", 2024-09-14T06:32:53.365 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.6541e-05 2024-09-14T06:32:53.366 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:53.366 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:32:53.366 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:53.366 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:53.366 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:53.366 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:11466 3.1d 3:bcad7303:::rbd_data.112d92027590.0000000000000001:f [read 227328~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:32:53.366 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:25.656129+0000", 2024-09-14T06:32:53.366 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 207.63612002799999, 2024-09-14T06:32:53.366 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052750987999999999, 2024-09-14T06:32:53.366 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:32:53.366 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:32:53.366 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:32:53.367 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:32:53.367 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:32:53.367 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:53.367 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 11466 2024-09-14T06:32:53.367 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:53.367 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:32:53.367 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:53.367 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:32:53.367 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.656129+0000", 2024-09-14T06:32:53.367 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:53.367 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:53.367 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:53.368 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:32:53.368 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.656129+0000", 2024-09-14T06:32:53.368 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:53.368 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:53.368 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:53.368 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:32:53.368 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.656347+0000", 2024-09-14T06:32:53.368 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.000218054 2024-09-14T06:32:53.368 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:53.368 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:53.368 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:32:53.368 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.656349+0000", 2024-09-14T06:32:53.369 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.9199999999999998e-06 2024-09-14T06:32:53.369 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:53.369 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:53.369 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:32:53.369 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.656353+0000", 2024-09-14T06:32:53.369 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.9210000000000002e-06 2024-09-14T06:32:53.369 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:53.369 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:53.369 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:32:53.369 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.656356+0000", 2024-09-14T06:32:53.369 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.7670000000000001e-06 2024-09-14T06:32:53.369 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:53.370 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:53.370 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:32:53.370 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.708787+0000", 2024-09-14T06:32:53.370 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052430840999999999 2024-09-14T06:32:53.370 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:53.370 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:53.370 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:32:53.371 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.708827+0000", 2024-09-14T06:32:53.371 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.9538e-05 2024-09-14T06:32:53.371 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:53.371 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:53.371 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:32:53.371 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.708880+0000", 2024-09-14T06:32:53.371 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.2947e-05 2024-09-14T06:32:53.371 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:53.371 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:32:53.371 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:53.371 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:53.371 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:53.371 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:11467 3.1d 3:bcad7303:::rbd_data.112d92027590.0000000000000001:f [read 237568~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:32:53.372 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:25.656361+0000", 2024-09-14T06:32:53.372 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 207.63588738000001, 2024-09-14T06:32:53.372 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052643898000000001, 2024-09-14T06:32:53.372 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:32:53.372 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:32:53.372 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:32:53.372 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:32:53.372 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:32:53.372 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:53.372 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 11467 2024-09-14T06:32:53.372 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:53.372 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:32:53.372 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:53.373 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:32:53.373 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.656361+0000", 2024-09-14T06:32:53.373 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:53.373 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:53.373 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:53.373 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:32:53.373 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.656361+0000", 2024-09-14T06:32:53.373 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:53.373 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:53.373 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:53.373 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:32:53.374 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.656855+0000", 2024-09-14T06:32:53.374 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00049372000000000001 2024-09-14T06:32:53.374 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:53.374 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:53.374 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:32:53.374 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.656859+0000", 2024-09-14T06:32:53.374 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.1300000000000003e-06 2024-09-14T06:32:53.374 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:53.374 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:53.374 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:32:53.374 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.656863+0000", 2024-09-14T06:32:53.374 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.2629999999999997e-06 2024-09-14T06:32:53.374 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:53.375 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:53.375 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:32:53.375 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.656867+0000", 2024-09-14T06:32:53.375 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.3220000000000001e-06 2024-09-14T06:32:53.375 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:53.375 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:53.375 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:32:53.375 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.708903+0000", 2024-09-14T06:32:53.375 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052035855999999998 2024-09-14T06:32:53.375 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:53.375 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:53.375 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:32:53.375 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.708949+0000", 2024-09-14T06:32:53.376 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.6276999999999998e-05 2024-09-14T06:32:53.376 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:53.376 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:53.376 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:32:53.376 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.709005+0000", 2024-09-14T06:32:53.376 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.6329999999999998e-05 2024-09-14T06:32:53.376 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:53.376 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:32:53.376 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:53.376 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:53.376 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:53.376 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:11480 3.1d 3:bcad7303:::rbd_data.112d92027590.0000000000000001:f [read 370688~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:32:53.377 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:25.658563+0000", 2024-09-14T06:32:53.377 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 207.63368561600001, 2024-09-14T06:32:53.377 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052429164, 2024-09-14T06:32:53.377 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:32:53.377 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:32:53.377 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:32:53.377 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:32:53.377 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:32:53.377 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:53.377 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 11480 2024-09-14T06:32:53.377 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:53.378 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:32:53.378 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:53.378 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:32:53.378 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.658563+0000", 2024-09-14T06:32:53.378 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:53.378 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:53.378 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:53.378 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:32:53.378 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.658563+0000", 2024-09-14T06:32:53.378 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:53.378 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:53.378 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:53.379 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:32:53.379 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.658658+0000", 2024-09-14T06:32:53.379 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.5346999999999994e-05 2024-09-14T06:32:53.379 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:53.379 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:53.379 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:32:53.379 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.658662+0000", 2024-09-14T06:32:53.379 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.14e-06 2024-09-14T06:32:53.379 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:53.379 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:53.380 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:32:53.380 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.658669+0000", 2024-09-14T06:32:53.380 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.7379999999999994e-06 2024-09-14T06:32:53.380 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:53.380 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:53.380 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:32:53.380 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.658675+0000", 2024-09-14T06:32:53.380 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.5430000000000001e-06 2024-09-14T06:32:53.380 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:53.380 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:53.380 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:32:53.381 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.710824+0000", 2024-09-14T06:32:53.381 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052148847999999998 2024-09-14T06:32:53.381 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:53.381 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:53.381 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:32:53.381 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.710864+0000", 2024-09-14T06:32:53.381 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.9925999999999997e-05 2024-09-14T06:32:53.381 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:53.381 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:53.381 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:32:53.382 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.710992+0000", 2024-09-14T06:32:53.382 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.000128622 2024-09-14T06:32:53.382 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:53.382 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:32:53.382 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:53.382 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:53.382 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:53.382 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:11481 3.1d 3:bcad7303:::rbd_data.112d92027590.0000000000000001:f [read 380928~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:32:53.382 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:25.658684+0000", 2024-09-14T06:32:53.382 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 207.633564546, 2024-09-14T06:32:53.382 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052508565, 2024-09-14T06:32:53.382 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:32:53.383 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:32:53.383 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:32:53.383 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:32:53.383 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:32:53.383 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:53.383 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 11481 2024-09-14T06:32:53.383 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:53.383 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:32:53.383 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:53.383 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:32:53.383 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.658684+0000", 2024-09-14T06:32:53.384 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:53.384 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:53.384 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:53.384 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:32:53.384 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.658684+0000", 2024-09-14T06:32:53.384 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:53.384 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:53.384 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:53.384 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:32:53.384 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.658715+0000", 2024-09-14T06:32:53.384 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.0703000000000001e-05 2024-09-14T06:32:53.385 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:53.385 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:53.385 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:32:53.385 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.658718+0000", 2024-09-14T06:32:53.385 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.0709999999999999e-06 2024-09-14T06:32:53.385 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:53.385 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:53.385 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:32:53.385 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.658725+0000", 2024-09-14T06:32:53.385 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.2339999999999997e-06 2024-09-14T06:32:53.385 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:53.385 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:53.386 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:32:53.386 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.658731+0000", 2024-09-14T06:32:53.386 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.6450000000000002e-06 2024-09-14T06:32:53.386 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:53.386 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:53.386 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:32:53.386 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.711020+0000", 2024-09-14T06:32:53.386 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052288886 2024-09-14T06:32:53.386 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:53.386 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:53.386 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:32:53.386 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.711061+0000", 2024-09-14T06:32:53.387 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.1084000000000002e-05 2024-09-14T06:32:53.387 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:53.387 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:53.387 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:32:53.387 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.711193+0000", 2024-09-14T06:32:53.387 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00013194199999999999 2024-09-14T06:32:53.387 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:53.387 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:32:53.387 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:53.387 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:53.387 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:53.387 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:11482 3.1d 3:bcad7303:::rbd_data.112d92027590.0000000000000001:f [read 391168~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:32:53.388 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:25.658740+0000", 2024-09-14T06:32:53.388 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 207.633509081, 2024-09-14T06:32:53.388 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052637015000000002, 2024-09-14T06:32:53.388 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:32:53.388 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:32:53.388 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:32:53.388 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:32:53.388 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:32:53.388 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:53.388 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 11482 2024-09-14T06:32:53.388 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:53.388 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:32:53.388 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:53.389 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:32:53.389 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.658740+0000", 2024-09-14T06:32:53.389 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:53.389 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:53.389 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:53.389 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:32:53.389 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.658740+0000", 2024-09-14T06:32:53.389 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:53.389 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:53.389 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:53.389 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:32:53.389 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.658880+0000", 2024-09-14T06:32:53.390 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00014036100000000001 2024-09-14T06:32:53.390 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:53.390 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:53.390 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:32:53.390 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.658883+0000", 2024-09-14T06:32:53.390 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.3969999999999998e-06 2024-09-14T06:32:53.390 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:53.390 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:53.390 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:32:53.390 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.658890+0000", 2024-09-14T06:32:53.390 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.1500000000000002e-06 2024-09-14T06:32:53.390 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:53.391 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:53.391 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:32:53.391 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.658897+0000", 2024-09-14T06:32:53.391 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.1260000000000003e-06 2024-09-14T06:32:53.391 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:53.391 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:53.391 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:32:53.391 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.711207+0000", 2024-09-14T06:32:53.391 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052310777000000003 2024-09-14T06:32:53.391 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:53.391 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:53.391 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:32:53.392 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.711249+0000", 2024-09-14T06:32:53.392 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.1786999999999998e-05 2024-09-14T06:32:53.392 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:53.392 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:53.392 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:32:53.392 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.711377+0000", 2024-09-14T06:32:53.392 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.000127417 2024-09-14T06:32:53.392 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:53.392 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:32:53.392 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:53.393 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:53.393 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:53.393 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:11483 3.1d 3:bcad7303:::rbd_data.112d92027590.0000000000000001:f [read 401408~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:32:53.393 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:25.658906+0000", 2024-09-14T06:32:53.393 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 207.63334229700001, 2024-09-14T06:32:53.393 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052595109000000001, 2024-09-14T06:32:53.393 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:32:53.393 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:32:53.393 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:32:53.393 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:32:53.393 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:32:53.394 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:53.394 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 11483 2024-09-14T06:32:53.394 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:53.394 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:32:53.394 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:53.394 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:32:53.394 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.658906+0000", 2024-09-14T06:32:53.394 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:53.394 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:53.394 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:53.394 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:32:53.395 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.658906+0000", 2024-09-14T06:32:53.395 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:53.395 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:53.395 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:53.395 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:32:53.395 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.664985+0000", 2024-09-14T06:32:53.395 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.0060787250000000001 2024-09-14T06:32:53.395 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:53.395 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:53.395 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:32:53.395 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.664991+0000", 2024-09-14T06:32:53.396 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.7030000000000003e-06 2024-09-14T06:32:53.396 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:53.396 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:53.396 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:32:53.396 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.664999+0000", 2024-09-14T06:32:53.396 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.4430000000000003e-06 2024-09-14T06:32:53.396 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:53.396 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:53.396 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:32:53.396 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.665006+0000", 2024-09-14T06:32:53.397 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.0840000000000003e-06 2024-09-14T06:32:53.397 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:53.397 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:53.397 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:32:53.397 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.711402+0000", 2024-09-14T06:32:53.397 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046395714999999997 2024-09-14T06:32:53.397 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:53.397 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:53.397 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:32:53.397 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.711443+0000", 2024-09-14T06:32:53.397 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.0911999999999997e-05 2024-09-14T06:32:53.398 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:53.398 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:53.398 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:32:53.398 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.711501+0000", 2024-09-14T06:32:53.398 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.8526999999999997e-05 2024-09-14T06:32:53.398 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:53.398 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:32:53.398 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:53.398 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:53.398 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:53.398 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:214569 3.2 3:479633bd:::rbd_object_map.112d3bd84cb0:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e170)", 2024-09-14T06:32:53.399 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:31:34.933901+0000", 2024-09-14T06:32:53.399 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 78.358347913000003, 2024-09-14T06:32:53.399 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.68157511400000004, 2024-09-14T06:32:53.399 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:32:53.399 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:32:53.399 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:32:53.399 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:32:53.399 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:32:53.399 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:53.399 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 214569 2024-09-14T06:32:53.399 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:53.399 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:32:53.400 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:53.400 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:32:53.400 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:31:34.933901+0000", 2024-09-14T06:32:53.400 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:53.400 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:53.400 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:53.400 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:32:53.400 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:31:34.933901+0000", 2024-09-14T06:32:53.400 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:53.400 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:53.400 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:53.401 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:32:53.401 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:31:34.933902+0000", 2024-09-14T06:32:53.401 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.7439999999999999e-06 2024-09-14T06:32:53.401 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:53.401 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:53.401 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:32:53.401 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:31:34.933910+0000", 2024-09-14T06:32:53.401 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.8569999999999995e-06 2024-09-14T06:32:53.401 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:53.401 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:53.401 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:32:53.402 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:31:34.933932+0000", 2024-09-14T06:32:53.402 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.1997999999999998e-05 2024-09-14T06:32:53.402 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:53.402 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:53.402 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:32:53.402 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:31:34.933941+0000", 2024-09-14T06:32:53.402 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.7180000000000002e-06 2024-09-14T06:32:53.402 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:53.402 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:53.402 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:32:53.402 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:31:34.934013+0000", 2024-09-14T06:32:53.402 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.1761000000000006e-05 2024-09-14T06:32:53.402 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:53.402 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:53.403 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "waiting for peered", 2024-09-14T06:32:53.403 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:31:34.934022+0000", 2024-09-14T06:32:53.403 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.8740000000000001e-06 2024-09-14T06:32:53.403 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:53.403 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:53.403 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:32:53.403 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:31:35.608803+0000", 2024-09-14T06:32:53.403 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.67478164299999999 2024-09-14T06:32:53.403 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:53.403 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:53.403 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:32:53.403 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:31:35.608964+0000", 2024-09-14T06:32:53.404 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00016112900000000001 2024-09-14T06:32:53.404 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:53.404 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:53.404 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "waiting for subops from 6", 2024-09-14T06:32:53.404 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:31:35.609399+0000", 2024-09-14T06:32:53.404 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00043454000000000002 2024-09-14T06:32:53.404 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:53.404 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:53.404 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "op_commit", 2024-09-14T06:32:53.404 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:31:35.610495+0000", 2024-09-14T06:32:53.404 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.001096404 2024-09-14T06:32:53.404 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:53.405 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:53.405 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:32:53.405 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:31:35.615409+0000", 2024-09-14T06:32:53.405 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.0049133199999999997 2024-09-14T06:32:53.405 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:53.405 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:53.405 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "commit_sent", 2024-09-14T06:32:53.405 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:31:35.615444+0000", 2024-09-14T06:32:53.405 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.5457000000000003e-05 2024-09-14T06:32:53.405 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:53.405 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:53.406 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:32:53.406 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:31:35.615476+0000", 2024-09-14T06:32:53.406 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.1668999999999998e-05 2024-09-14T06:32:53.406 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:53.406 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:32:53.406 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:53.406 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:53.406 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:32:53.406 INFO:teuthology.orchestra.run.smithi138.stdout:} 2024-09-14T06:32:53.408 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-14T06:32:53.409 INFO:teuthology.orchestra.run.smithi094.stderr:dumped all 2024-09-14T06:32:53.409 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-14T06:32:53.411 INFO:teuthology.orchestra.run.smithi094.stdout:{"pg_ready":true,"pg_map":{"version":367,"stamp":"2024-09-14T06:32:53.327210+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":617201154,"num_objects":650,"num_object_clones":0,"num_object_copies":1300,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":650,"num_whiteouts":0,"num_read":237492,"num_read_kb":2182695,"num_write":37178,"num_write_kb":1479847,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5314,"num_bytes_recovered":637770699,"num_keys_recovered":546,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":27570,"ondisk_log_size":27570,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":84,"num_osds":7,"num_per_pool_osds":7,"num_per_pool_omap_osds":7,"kb":660602880,"kb_used":494480,"kb_used_data":56480,"kb_used_omap":10,"kb_used_meta":437493,"kb_avail":660108400,"statfs":{"total":676457349120,"available":675951001600,"internally_reserved":0,"allocated":57835520,"data_stored":316398979,"data_compressed":19468911,"data_compressed_allocated":26099712,"data_compressed_original":297544805,"omap_allocated":11157,"internal_metadata":447992939},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-70904318,"num_objects":-114,"num_object_clones":0,"num_object_copies":-228,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-114,"num_whiteouts":0,"num_read":492,"num_read_kb":418,"num_write":182,"num_write_kb":29,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":165,"num_bytes_recovered":4679091,"num_keys_recovered":12,"num_objects_omap":-3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.000786"},"pg_stats":[{"pgid":"3.1f","version":"246'843","reported_seq":8275,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-09-14T06:32:52.640912+0000","last_change":"2024-09-14T06:32:52.641050+0000","last_active":"2024-09-14T06:32:52.640912+0000","last_peered":"2024-09-14T06:32:52.640912+0000","last_clean":"2024-09-14T06:32:52.640912+0000","last_became_active":"2024-09-14T06:29:00.579883+0000","last_became_peered":"2024-09-14T06:29:00.579883+0000","last_unstale":"2024-09-14T06:32:52.640912+0000","last_undegraded":"2024-09-14T06:32:52.640912+0000","last_fullsized":"2024-09-14T06:32:52.640912+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":843,"log_dups_size":0,"ondisk_log_size":843,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T07:00:55.973615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013497400000000001,"stat_sum":{"num_bytes":20082011,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":0,"num_read":8465,"num_read_kb":86897,"num_write":1160,"num_write_kb":49543,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"1b","length":"1"},{"start":"1d","length":"1"},{"start":"1f","length":"1"},{"start":"21","length":"1"}]},{"pgid":"3.1e","version":"242'770","reported_seq":10049,"reported_epoch":245,"state":"active+clean","last_fresh":"2024-09-14T06:32:50.621418+0000","last_change":"2024-09-14T06:32:50.621612+0000","last_active":"2024-09-14T06:32:50.621418+0000","last_peered":"2024-09-14T06:32:50.621418+0000","last_clean":"2024-09-14T06:32:50.621418+0000","last_became_active":"2024-09-14T06:32:17.679198+0000","last_became_peered":"2024-09-14T06:32:17.679198+0000","last_unstale":"2024-09-14T06:32:50.621418+0000","last_undegraded":"2024-09-14T06:32:50.621418+0000","last_fullsized":"2024-09-14T06:32:50.621418+0000","mapping_epoch":211,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":212,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":770,"log_dups_size":0,"ondisk_log_size":770,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T09:22:00.073193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00068614299999999995,"stat_sum":{"num_bytes":13605158,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":8313,"num_read_kb":67656,"num_write":1013,"num_write_kb":40407,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":713,"num_bytes_recovered":87148703,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"1d","length":"1"},{"start":"1f","length":"1"},{"start":"21","length":"1"}]},{"pgid":"3.1d","version":"242'1010","reported_seq":7233,"reported_epoch":243,"state":"active+clean","last_fresh":"2024-09-14T06:32:48.620790+0000","last_change":"2024-09-14T06:32:47.618166+0000","last_active":"2024-09-14T06:32:48.620790+0000","last_peered":"2024-09-14T06:32:48.620790+0000","last_clean":"2024-09-14T06:32:48.620790+0000","last_became_active":"2024-09-14T06:32:24.716363+0000","last_became_peered":"2024-09-14T06:32:24.716363+0000","last_unstale":"2024-09-14T06:32:48.620790+0000","last_undegraded":"2024-09-14T06:32:48.620790+0000","last_fullsized":"2024-09-14T06:32:48.620790+0000","mapping_epoch":218,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":219,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":1010,"log_dups_size":0,"ondisk_log_size":1010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T14:47:05.271811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076405400000000001,"stat_sum":{"num_bytes":38727511,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":7636,"num_read_kb":64175,"num_write":1490,"num_write_kb":53276,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"1f","length":"1"},{"start":"21","length":"1"},{"start":"23","length":"1"},{"start":"25","length":"1"},{"start":"27","length":"1"},{"start":"29","length":"1"},{"start":"2b","length":"1"},{"start":"2d","length":"1"},{"start":"2f","length":"1"},{"start":"31","length":"1"}]},{"pgid":"3.1c","version":"242'743","reported_seq":5736,"reported_epoch":243,"state":"active+clean","last_fresh":"2024-09-14T06:32:48.618606+0000","last_change":"2024-09-14T06:32:47.618826+0000","last_active":"2024-09-14T06:32:48.618606+0000","last_peered":"2024-09-14T06:32:48.618606+0000","last_clean":"2024-09-14T06:32:48.618606+0000","last_became_active":"2024-09-14T06:32:39.775788+0000","last_became_peered":"2024-09-14T06:32:39.775788+0000","last_unstale":"2024-09-14T06:32:48.618606+0000","last_undegraded":"2024-09-14T06:32:48.618606+0000","last_fullsized":"2024-09-14T06:32:48.618606+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":743,"log_dups_size":0,"ondisk_log_size":743,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T18:15:02.841575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0009972850000000001,"stat_sum":{"num_bytes":11595051,"num_objects":27,"num_object_clones":0,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":27,"num_whiteouts":0,"num_read":4709,"num_read_kb":43589,"num_write":876,"num_write_kb":45545,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":505,"num_bytes_recovered":44145197,"num_keys_recovered":280,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"1f","length":"1"},{"start":"21","length":"1"},{"start":"23","length":"1"},{"start":"25","length":"1"},{"start":"27","length":"1"},{"start":"29","length":"1"},{"start":"2b","length":"1"},{"start":"2d","length":"1"},{"start":"2f","length":"1"},{"start":"31","length":"1"}]},{"pgid":"3.1b","version":"246'945","reported_seq":11364,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-09-14T06:32:52.640718+0000","last_change":"2024-09-14T06:32:52.640883+0000","last_active":"2024-09-14T06:32:52.640718+0000","last_peered":"2024-09-14T06:32:52.640718+0000","last_clean":"2024-09-14T06:32:52.640718+0000","last_became_active":"2024-09-14T06:29:00.586183+0000","last_became_peered":"2024-09-14T06:29:00.586183+0000","last_unstale":"2024-09-14T06:32:52.640718+0000","last_undegraded":"2024-09-14T06:32:52.640718+0000","last_fullsized":"2024-09-14T06:32:52.640718+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":945,"log_dups_size":0,"ondisk_log_size":945,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T07:51:04.778922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013021529999999999,"stat_sum":{"num_bytes":8472568,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":0,"num_read":11908,"num_read_kb":108093,"num_write":1449,"num_write_kb":55102,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"1b","length":"1"},{"start":"1d","length":"1"},{"start":"1f","length":"1"},{"start":"21","length":"1"}]},{"pgid":"3.1a","version":"242'1600","reported_seq":10232,"reported_epoch":243,"state":"active+clean","last_fresh":"2024-09-14T06:32:48.621293+0000","last_change":"2024-09-14T06:32:47.617872+0000","last_active":"2024-09-14T06:32:48.621293+0000","last_peered":"2024-09-14T06:32:48.621293+0000","last_clean":"2024-09-14T06:32:48.621293+0000","last_became_active":"2024-09-14T06:29:00.576206+0000","last_became_peered":"2024-09-14T06:29:00.576206+0000","last_unstale":"2024-09-14T06:32:48.621293+0000","last_undegraded":"2024-09-14T06:32:48.621293+0000","last_fullsized":"2024-09-14T06:32:48.621293+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":1600,"log_dups_size":0,"ondisk_log_size":1600,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T15:52:11.992787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000513255,"stat_sum":{"num_bytes":16886247,"num_objects":33,"num_object_clones":0,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":33,"num_whiteouts":0,"num_read":13015,"num_read_kb":82619,"num_write":2377,"num_write_kb":55671,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"1f","length":"1"},{"start":"21","length":"1"},{"start":"23","length":"1"},{"start":"25","length":"1"},{"start":"27","length":"1"},{"start":"29","length":"1"},{"start":"2b","length":"1"},{"start":"2d","length":"1"},{"start":"2f","length":"1"},{"start":"31","length":"1"}]},{"pgid":"3.19","version":"241'525","reported_seq":6306,"reported_epoch":243,"state":"active+clean","last_fresh":"2024-09-14T06:32:48.618538+0000","last_change":"2024-09-14T06:32:47.617922+0000","last_active":"2024-09-14T06:32:48.618538+0000","last_peered":"2024-09-14T06:32:48.618538+0000","last_clean":"2024-09-14T06:32:48.618538+0000","last_became_active":"2024-09-14T06:31:53.744800+0000","last_became_peered":"2024-09-14T06:31:53.744800+0000","last_unstale":"2024-09-14T06:32:48.618538+0000","last_undegraded":"2024-09-14T06:32:48.618538+0000","last_fullsized":"2024-09-14T06:32:48.618538+0000","mapping_epoch":188,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":189,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":525,"log_dups_size":0,"ondisk_log_size":525,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T10:14:12.753246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00048076999999999999,"stat_sum":{"num_bytes":29846614,"num_objects":25,"num_object_clones":0,"num_object_copies":50,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":0,"num_read":5090,"num_read_kb":67756,"num_write":726,"num_write_kb":43868,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":282,"num_bytes_recovered":41927489,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"1f","length":"1"},{"start":"21","length":"1"},{"start":"23","length":"1"},{"start":"25","length":"1"},{"start":"27","length":"1"},{"start":"29","length":"1"},{"start":"2b","length":"1"},{"start":"2d","length":"1"},{"start":"2f","length":"1"},{"start":"31","length":"1"}]},{"pgid":"3.18","version":"242'511","reported_seq":6099,"reported_epoch":245,"state":"active+clean","last_fresh":"2024-09-14T06:32:50.620110+0000","last_change":"2024-09-14T06:32:50.620289+0000","last_active":"2024-09-14T06:32:50.620110+0000","last_peered":"2024-09-14T06:32:50.620110+0000","last_clean":"2024-09-14T06:32:50.620110+0000","last_became_active":"2024-09-14T06:31:53.745159+0000","last_became_peered":"2024-09-14T06:31:53.745159+0000","last_unstale":"2024-09-14T06:32:50.620110+0000","last_undegraded":"2024-09-14T06:32:50.620110+0000","last_fullsized":"2024-09-14T06:32:50.620110+0000","mapping_epoch":188,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":189,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":511,"log_dups_size":0,"ondisk_log_size":511,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T14:49:56.682951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049795799999999995,"stat_sum":{"num_bytes":15150821,"num_objects":25,"num_object_clones":0,"num_object_copies":50,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":0,"num_read":4458,"num_read_kb":78893,"num_write":634,"num_write_kb":51576,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":310,"num_bytes_recovered":56082976,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"1d","length":"1"},{"start":"1f","length":"1"},{"start":"21","length":"1"}]},{"pgid":"3.17","version":"247'964","reported_seq":9389,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-09-14T06:32:52.750110+0000","last_change":"2024-09-14T06:32:52.641278+0000","last_active":"2024-09-14T06:32:52.750110+0000","last_peered":"2024-09-14T06:32:52.750110+0000","last_clean":"2024-09-14T06:32:52.750110+0000","last_became_active":"2024-09-14T06:29:00.579714+0000","last_became_peered":"2024-09-14T06:29:00.579714+0000","last_unstale":"2024-09-14T06:32:52.750110+0000","last_undegraded":"2024-09-14T06:32:52.750110+0000","last_fullsized":"2024-09-14T06:32:52.750110+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":964,"log_dups_size":0,"ondisk_log_size":964,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T09:55:26.864120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015440619999999999,"stat_sum":{"num_bytes":40592411,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":10381,"num_read_kb":68697,"num_write":1099,"num_write_kb":48470,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"1b","length":"1"},{"start":"1d","length":"1"},{"start":"1f","length":"1"},{"start":"21","length":"1"}]},{"pgid":"3.16","version":"241'1139","reported_seq":4033,"reported_epoch":243,"state":"active+clean","last_fresh":"2024-09-14T06:32:48.855681+0000","last_change":"2024-09-14T06:32:47.712851+0000","last_active":"2024-09-14T06:32:48.855681+0000","last_peered":"2024-09-14T06:32:48.855681+0000","last_clean":"2024-09-14T06:32:48.855681+0000","last_became_active":"2024-09-14T06:32:24.718745+0000","last_became_peered":"2024-09-14T06:32:24.718745+0000","last_unstale":"2024-09-14T06:32:48.855681+0000","last_undegraded":"2024-09-14T06:32:48.855681+0000","last_fullsized":"2024-09-14T06:32:48.855681+0000","mapping_epoch":218,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":219,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":1139,"log_dups_size":0,"ondisk_log_size":1139,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T09:17:26.636756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011263180000000001,"stat_sum":{"num_bytes":10730521,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":0,"num_read":5658,"num_read_kb":38562,"num_write":1726,"num_write_kb":37248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"1f","length":"1"},{"start":"21","length":"1"},{"start":"23","length":"1"},{"start":"25","length":"1"},{"start":"27","length":"1"},{"start":"29","length":"1"},{"start":"2b","length":"1"},{"start":"2d","length":"1"},{"start":"2f","length":"1"},{"start":"31","length":"1"}]},{"pgid":"3.15","version":"242'607","reported_seq":6477,"reported_epoch":243,"state":"active+clean","last_fresh":"2024-09-14T06:32:48.855407+0000","last_change":"2024-09-14T06:32:47.714319+0000","last_active":"2024-09-14T06:32:48.855407+0000","last_peered":"2024-09-14T06:32:48.855407+0000","last_clean":"2024-09-14T06:32:48.855407+0000","last_became_active":"2024-09-14T06:31:53.746091+0000","last_became_peered":"2024-09-14T06:31:53.746091+0000","last_unstale":"2024-09-14T06:32:48.855407+0000","last_undegraded":"2024-09-14T06:32:48.855407+0000","last_fullsized":"2024-09-14T06:32:48.855407+0000","mapping_epoch":188,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":189,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":607,"log_dups_size":0,"ondisk_log_size":607,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T07:11:58.263184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0027384229999999998,"stat_sum":{"num_bytes":12866454,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":4928,"num_read_kb":74748,"num_write":659,"num_write_kb":51190,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":271,"num_bytes_recovered":49981473,"num_keys_recovered":33,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"1f","length":"1"},{"start":"21","length":"1"},{"start":"23","length":"1"},{"start":"25","length":"1"},{"start":"27","length":"1"},{"start":"29","length":"1"},{"start":"2b","length":"1"},{"start":"2d","length":"1"},{"start":"2f","length":"1"},{"start":"31","length":"1"}]},{"pgid":"3.14","version":"246'1101","reported_seq":6085,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-09-14T06:32:52.637439+0000","last_change":"2024-09-14T06:32:52.637751+0000","last_active":"2024-09-14T06:32:52.637439+0000","last_peered":"2024-09-14T06:32:52.637439+0000","last_clean":"2024-09-14T06:32:52.637439+0000","last_became_active":"2024-09-14T06:32:39.776600+0000","last_became_peered":"2024-09-14T06:32:39.776600+0000","last_unstale":"2024-09-14T06:32:52.637439+0000","last_undegraded":"2024-09-14T06:32:52.637439+0000","last_fullsized":"2024-09-14T06:32:52.637439+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":1101,"log_dups_size":0,"ondisk_log_size":1101,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T17:44:01.106832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00048112200000000001,"stat_sum":{"num_bytes":8751218,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":8770,"num_read_kb":46783,"num_write":1545,"num_write_kb":29823,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":148,"num_bytes_recovered":6082709,"num_keys_recovered":11,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"1b","length":"1"},{"start":"1d","length":"1"},{"start":"1f","length":"1"},{"start":"21","length":"1"}]},{"pgid":"2.7","version":"0'0","reported_seq":104,"reported_epoch":243,"state":"active+clean","last_fresh":"2024-09-14T06:32:48.855541+0000","last_change":"2024-09-14T06:32:01.612470+0000","last_active":"2024-09-14T06:32:48.855541+0000","last_peered":"2024-09-14T06:32:48.855541+0000","last_clean":"2024-09-14T06:32:48.855541+0000","last_became_active":"2024-09-14T06:32:01.612121+0000","last_became_peered":"2024-09-14T06:32:01.612121+0000","last_unstale":"2024-09-14T06:32:48.855541+0000","last_undegraded":"2024-09-14T06:32:48.855541+0000","last_fullsized":"2024-09-14T06:32:48.855541+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:28:50.205350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:28:50.205350+0000","last_clean_scrub_stamp":"2024-09-14T06:28:50.205350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T13:40:52.537942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.6","version":"247'847","reported_seq":8467,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-09-14T06:32:52.724366+0000","last_change":"2024-09-14T06:32:52.641051+0000","last_active":"2024-09-14T06:32:52.724366+0000","last_peered":"2024-09-14T06:32:52.724366+0000","last_clean":"2024-09-14T06:32:52.724366+0000","last_became_active":"2024-09-14T06:29:00.578283+0000","last_became_peered":"2024-09-14T06:29:00.578283+0000","last_unstale":"2024-09-14T06:32:52.724366+0000","last_undegraded":"2024-09-14T06:32:52.724366+0000","last_fullsized":"2024-09-14T06:32:52.724366+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":847,"log_dups_size":0,"ondisk_log_size":847,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T17:46:49.815855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001358539,"stat_sum":{"num_bytes":8601020,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":8435,"num_read_kb":78619,"num_write":897,"num_write_kb":48297,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"1b","length":"1"},{"start":"1d","length":"1"},{"start":"1f","length":"1"},{"start":"21","length":"1"}]},{"pgid":"2.6","version":"0'0","reported_seq":469,"reported_epoch":243,"state":"active+clean","last_fresh":"2024-09-14T06:32:48.618754+0000","last_change":"2024-09-14T06:32:01.608456+0000","last_active":"2024-09-14T06:32:48.618754+0000","last_peered":"2024-09-14T06:32:48.618754+0000","last_clean":"2024-09-14T06:32:48.618754+0000","last_became_active":"2024-09-14T06:32:01.608178+0000","last_became_peered":"2024-09-14T06:32:01.608178+0000","last_unstale":"2024-09-14T06:32:48.618754+0000","last_undegraded":"2024-09-14T06:32:48.618754+0000","last_fullsized":"2024-09-14T06:32:48.618754+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:28:50.205350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:28:50.205350+0000","last_clean_scrub_stamp":"2024-09-14T06:28:50.205350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T10:16:40.480031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065018200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"242'798","reported_seq":4388,"reported_epoch":245,"state":"active+clean","last_fresh":"2024-09-14T06:32:50.621673+0000","last_change":"2024-09-14T06:32:50.621868+0000","last_active":"2024-09-14T06:32:50.621673+0000","last_peered":"2024-09-14T06:32:50.621673+0000","last_clean":"2024-09-14T06:32:50.621673+0000","last_became_active":"2024-09-14T06:31:54.747923+0000","last_became_peered":"2024-09-14T06:31:54.747923+0000","last_unstale":"2024-09-14T06:32:50.621673+0000","last_undegraded":"2024-09-14T06:32:50.621673+0000","last_fullsized":"2024-09-14T06:32:50.621673+0000","mapping_epoch":189,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":190,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":798,"log_dups_size":0,"ondisk_log_size":798,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T17:06:50.634057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00083445000000000004,"stat_sum":{"num_bytes":18657113,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":0,"num_read":4414,"num_read_kb":40976,"num_write":1145,"num_write_kb":36116,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":183,"num_bytes_recovered":13912513,"num_keys_recovered":22,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"1d","length":"1"},{"start":"1f","length":"1"},{"start":"21","length":"1"}]},{"pgid":"2.5","version":"0'0","reported_seq":463,"reported_epoch":243,"state":"active+clean","last_fresh":"2024-09-14T06:32:48.855553+0000","last_change":"2024-09-14T06:28:53.640035+0000","last_active":"2024-09-14T06:32:48.855553+0000","last_peered":"2024-09-14T06:32:48.855553+0000","last_clean":"2024-09-14T06:32:48.855553+0000","last_became_active":"2024-09-14T06:28:51.308058+0000","last_became_peered":"2024-09-14T06:28:51.308058+0000","last_unstale":"2024-09-14T06:32:48.855553+0000","last_undegraded":"2024-09-14T06:32:48.855553+0000","last_fullsized":"2024-09-14T06:32:48.855553+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:28:50.205350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:28:50.205350+0000","last_clean_scrub_stamp":"2024-09-14T06:28:50.205350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T13:35:48.544057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00037824000000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"242'669","reported_seq":3949,"reported_epoch":243,"state":"active+clean","last_fresh":"2024-09-14T06:32:48.618696+0000","last_change":"2024-09-14T06:32:47.619035+0000","last_active":"2024-09-14T06:32:48.618696+0000","last_peered":"2024-09-14T06:32:48.618696+0000","last_clean":"2024-09-14T06:32:48.618696+0000","last_became_active":"2024-09-14T06:32:17.675470+0000","last_became_peered":"2024-09-14T06:32:17.675470+0000","last_unstale":"2024-09-14T06:32:48.618696+0000","last_undegraded":"2024-09-14T06:32:48.618696+0000","last_fullsized":"2024-09-14T06:32:48.618696+0000","mapping_epoch":211,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":212,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":669,"log_dups_size":0,"ondisk_log_size":669,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T07:11:32.438178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00080866399999999995,"stat_sum":{"num_bytes":6445103,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":0,"num_read":2623,"num_read_kb":37411,"num_write":865,"num_write_kb":40855,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":331,"num_bytes_recovered":31411119,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"1f","length":"1"},{"start":"21","length":"1"},{"start":"23","length":"1"},{"start":"25","length":"1"},{"start":"27","length":"1"},{"start":"29","length":"1"},{"start":"2b","length":"1"},{"start":"2d","length":"1"},{"start":"2f","length":"1"},{"start":"31","length":"1"}]},{"pgid":"2.4","version":"0'0","reported_seq":463,"reported_epoch":243,"state":"active+clean","last_fresh":"2024-09-14T06:32:48.618741+0000","last_change":"2024-09-14T06:28:53.226072+0000","last_active":"2024-09-14T06:32:48.618741+0000","last_peered":"2024-09-14T06:32:48.618741+0000","last_clean":"2024-09-14T06:32:48.618741+0000","last_became_active":"2024-09-14T06:28:51.308072+0000","last_became_peered":"2024-09-14T06:28:51.308072+0000","last_unstale":"2024-09-14T06:32:48.618741+0000","last_undegraded":"2024-09-14T06:32:48.618741+0000","last_fullsized":"2024-09-14T06:32:48.618741+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:28:50.205350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:28:50.205350+0000","last_clean_scrub_stamp":"2024-09-14T06:28:50.205350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T13:15:09.052328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066197399999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"242'877","reported_seq":5496,"reported_epoch":245,"state":"active+clean","last_fresh":"2024-09-14T06:32:50.621836+0000","last_change":"2024-09-14T06:32:50.621995+0000","last_active":"2024-09-14T06:32:50.621836+0000","last_peered":"2024-09-14T06:32:50.621836+0000","last_clean":"2024-09-14T06:32:50.621836+0000","last_became_active":"2024-09-14T06:32:24.716458+0000","last_became_peered":"2024-09-14T06:32:24.716458+0000","last_unstale":"2024-09-14T06:32:50.621836+0000","last_undegraded":"2024-09-14T06:32:50.621836+0000","last_fullsized":"2024-09-14T06:32:50.621836+0000","mapping_epoch":218,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":219,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":877,"log_dups_size":0,"ondisk_log_size":877,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T14:17:39.014844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00092606700000000004,"stat_sum":{"num_bytes":13992787,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":0,"num_read":4510,"num_read_kb":42630,"num_write":1236,"num_write_kb":49494,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":330,"num_bytes_recovered":28798893,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"1d","length":"1"},{"start":"1f","length":"1"},{"start":"21","length":"1"}]},{"pgid":"3.3","version":"242'615","reported_seq":10542,"reported_epoch":243,"state":"active+clean","last_fresh":"2024-09-14T06:32:48.620876+0000","last_change":"2024-09-14T06:32:47.618264+0000","last_active":"2024-09-14T06:32:48.620876+0000","last_peered":"2024-09-14T06:32:48.620876+0000","last_clean":"2024-09-14T06:32:48.620876+0000","last_became_active":"2024-09-14T06:29:00.584678+0000","last_became_peered":"2024-09-14T06:29:00.584678+0000","last_unstale":"2024-09-14T06:32:48.620876+0000","last_undegraded":"2024-09-14T06:32:48.620876+0000","last_fullsized":"2024-09-14T06:32:48.620876+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":615,"log_dups_size":0,"ondisk_log_size":615,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T08:30:37.424996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067343999999999998,"stat_sum":{"num_bytes":25618330,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":0,"num_read":9318,"num_read_kb":89087,"num_write":688,"num_write_kb":51643,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"1f","length":"1"},{"start":"21","length":"1"},{"start":"23","length":"1"},{"start":"25","length":"1"},{"start":"27","length":"1"},{"start":"29","length":"1"},{"start":"2b","length":"1"},{"start":"2d","length":"1"},{"start":"2f","length":"1"},{"start":"31","length":"1"}]},{"pgid":"2.2","version":"22'2","reported_seq":78,"reported_epoch":243,"state":"active+clean","last_fresh":"2024-09-14T06:32:48.618587+0000","last_change":"2024-09-14T06:32:24.716937+0000","last_active":"2024-09-14T06:32:48.618587+0000","last_peered":"2024-09-14T06:32:48.618587+0000","last_clean":"2024-09-14T06:32:48.618587+0000","last_became_active":"2024-09-14T06:32:24.716539+0000","last_became_peered":"2024-09-14T06:32:24.716539+0000","last_unstale":"2024-09-14T06:32:48.618587+0000","last_undegraded":"2024-09-14T06:32:48.618587+0000","last_fullsized":"2024-09-14T06:32:48.618587+0000","mapping_epoch":218,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":219,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:28:50.205350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:28:50.205350+0000","last_clean_scrub_stamp":"2024-09-14T06:28:50.205350+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T07:16:29.639469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"242'1085","reported_seq":6945,"reported_epoch":243,"state":"active+clean","last_fresh":"2024-09-14T06:32:48.618400+0000","last_change":"2024-09-14T06:32:47.617517+0000","last_active":"2024-09-14T06:32:48.618400+0000","last_peered":"2024-09-14T06:32:48.618400+0000","last_clean":"2024-09-14T06:32:48.618400+0000","last_became_active":"2024-09-14T06:29:00.574873+0000","last_became_peered":"2024-09-14T06:29:00.574873+0000","last_unstale":"2024-09-14T06:32:48.618400+0000","last_undegraded":"2024-09-14T06:32:48.618400+0000","last_fullsized":"2024-09-14T06:32:48.618400+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":1085,"log_dups_size":0,"ondisk_log_size":1085,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T13:08:11.145355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00052732099999999997,"stat_sum":{"num_bytes":29508178,"num_objects":31,"num_object_clones":0,"num_object_copies":62,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":31,"num_whiteouts":0,"num_read":8253,"num_read_kb":85478,"num_write":1621,"num_write_kb":58700,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"1f","length":"1"},{"start":"21","length":"1"},{"start":"23","length":"1"},{"start":"25","length":"1"},{"start":"27","length":"1"},{"start":"29","length":"1"},{"start":"2b","length":"1"},{"start":"2d","length":"1"},{"start":"2f","length":"1"},{"start":"31","length":"1"}]},{"pgid":"2.1","version":"0'0","reported_seq":87,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-09-14T06:32:52.636916+0000","last_change":"2024-09-14T06:32:14.665376+0000","last_active":"2024-09-14T06:32:52.636916+0000","last_peered":"2024-09-14T06:32:52.636916+0000","last_clean":"2024-09-14T06:32:52.636916+0000","last_became_active":"2024-09-14T06:32:14.665081+0000","last_became_peered":"2024-09-14T06:32:14.665081+0000","last_unstale":"2024-09-14T06:32:52.636916+0000","last_undegraded":"2024-09-14T06:32:52.636916+0000","last_fullsized":"2024-09-14T06:32:52.636916+0000","mapping_epoch":208,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":209,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:28:50.205350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:28:50.205350+0000","last_clean_scrub_stamp":"2024-09-14T06:28:50.205350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T16:35:12.869944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053403100000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1","version":"247'810","reported_seq":11732,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-09-14T06:32:52.639996+0000","last_change":"2024-09-14T06:32:52.639996+0000","last_active":"2024-09-14T06:32:52.639996+0000","last_peered":"2024-09-14T06:32:52.639996+0000","last_clean":"2024-09-14T06:32:52.639996+0000","last_became_active":"2024-09-14T06:30:57.768238+0000","last_became_peered":"2024-09-14T06:30:57.768238+0000","last_unstale":"2024-09-14T06:32:52.639996+0000","last_undegraded":"2024-09-14T06:32:52.639996+0000","last_fullsized":"2024-09-14T06:32:52.639996+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":810,"log_dups_size":0,"ondisk_log_size":810,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T12:19:37.613581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051009899999999997,"stat_sum":{"num_bytes":13175713,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":11284,"num_read_kb":112430,"num_write":982,"num_write_kb":47408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":183,"num_bytes_recovered":30431994,"num_keys_recovered":22,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"1b","length":"1"},{"start":"1d","length":"1"},{"start":"1f","length":"1"},{"start":"21","length":"1"}]},{"pgid":"2.0","version":"0'0","reported_seq":463,"reported_epoch":243,"state":"active+clean","last_fresh":"2024-09-14T06:32:48.855132+0000","last_change":"2024-09-14T06:28:53.640096+0000","last_active":"2024-09-14T06:32:48.855132+0000","last_peered":"2024-09-14T06:32:48.855132+0000","last_clean":"2024-09-14T06:32:48.855132+0000","last_became_active":"2024-09-14T06:28:51.223069+0000","last_became_peered":"2024-09-14T06:28:51.223069+0000","last_unstale":"2024-09-14T06:32:48.855132+0000","last_undegraded":"2024-09-14T06:32:48.855132+0000","last_fullsized":"2024-09-14T06:32:48.855132+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:28:50.205350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:28:50.205350+0000","last_clean_scrub_stamp":"2024-09-14T06:28:50.205350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T15:13:00.471509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00043714600000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"242'1624","reported_seq":7663,"reported_epoch":245,"state":"active+clean","last_fresh":"2024-09-14T06:32:50.621185+0000","last_change":"2024-09-14T06:32:50.621460+0000","last_active":"2024-09-14T06:32:50.621185+0000","last_peered":"2024-09-14T06:32:50.621185+0000","last_clean":"2024-09-14T06:32:50.621185+0000","last_became_active":"2024-09-14T06:31:54.744160+0000","last_became_peered":"2024-09-14T06:31:54.744160+0000","last_unstale":"2024-09-14T06:32:50.621185+0000","last_undegraded":"2024-09-14T06:32:50.621185+0000","last_fullsized":"2024-09-14T06:32:50.621185+0000","mapping_epoch":189,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":190,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":1624,"log_dups_size":0,"ondisk_log_size":1624,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T14:41:40.053297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010607629999999999,"stat_sum":{"num_bytes":36953661,"num_objects":36,"num_object_clones":0,"num_object_copies":72,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":36,"num_whiteouts":0,"num_read":10720,"num_read_kb":70588,"num_write":2560,"num_write_kb":53564,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":327,"num_bytes_recovered":50047063,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"1d","length":"1"},{"start":"1f","length":"1"},{"start":"21","length":"1"}]},{"pgid":"2.3","version":"20'1","reported_seq":72,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-09-14T06:32:52.636988+0000","last_change":"2024-09-14T06:32:24.717893+0000","last_active":"2024-09-14T06:32:52.636988+0000","last_peered":"2024-09-14T06:32:52.636988+0000","last_clean":"2024-09-14T06:32:52.636988+0000","last_became_active":"2024-09-14T06:32:24.717658+0000","last_became_peered":"2024-09-14T06:32:24.717658+0000","last_unstale":"2024-09-14T06:32:52.636988+0000","last_undegraded":"2024-09-14T06:32:52.636988+0000","last_fullsized":"2024-09-14T06:32:52.636988+0000","mapping_epoch":218,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":219,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:28:50.205350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:28:50.205350+0000","last_clean_scrub_stamp":"2024-09-14T06:28:50.205350+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T10:36:16.650799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.0","version":"17'72","reported_seq":555,"reported_epoch":243,"state":"active+clean","last_fresh":"2024-09-14T06:32:48.855280+0000","last_change":"2024-09-14T06:28:44.839885+0000","last_active":"2024-09-14T06:32:48.855280+0000","last_peered":"2024-09-14T06:32:48.855280+0000","last_clean":"2024-09-14T06:32:48.855280+0000","last_became_active":"2024-09-14T06:28:44.839573+0000","last_became_peered":"2024-09-14T06:28:44.839573+0000","last_unstale":"2024-09-14T06:32:48.855280+0000","last_undegraded":"2024-09-14T06:32:48.855280+0000","last_fullsized":"2024-09-14T06:32:48.855280+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:28:43.828494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:28:43.828494+0000","last_clean_scrub_stamp":"2024-09-14T06:28:43.828494+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T14:41:25.787293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.8","version":"242'638","reported_seq":6390,"reported_epoch":245,"state":"active+clean","last_fresh":"2024-09-14T06:32:50.620140+0000","last_change":"2024-09-14T06:32:50.620333+0000","last_active":"2024-09-14T06:32:50.620140+0000","last_peered":"2024-09-14T06:32:50.620140+0000","last_clean":"2024-09-14T06:32:50.620140+0000","last_became_active":"2024-09-14T06:31:55.754694+0000","last_became_peered":"2024-09-14T06:31:55.754694+0000","last_unstale":"2024-09-14T06:32:50.620140+0000","last_undegraded":"2024-09-14T06:32:50.620140+0000","last_fullsized":"2024-09-14T06:32:50.620140+0000","mapping_epoch":190,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":191,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":638,"log_dups_size":0,"ondisk_log_size":638,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T13:42:11.297497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050606700000000002,"stat_sum":{"num_bytes":27427167,"num_objects":20,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":0,"num_read":5616,"num_read_kb":90699,"num_write":885,"num_write_kb":57685,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":319,"num_bytes_recovered":60352269,"num_keys_recovered":22,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"1d","length":"1"},{"start":"1f","length":"1"},{"start":"21","length":"1"}]},{"pgid":"3.9","version":"242'571","reported_seq":4335,"reported_epoch":243,"state":"active+clean","last_fresh":"2024-09-14T06:32:48.620714+0000","last_change":"2024-09-14T06:32:47.617870+0000","last_active":"2024-09-14T06:32:48.620714+0000","last_peered":"2024-09-14T06:32:48.620714+0000","last_clean":"2024-09-14T06:32:48.620714+0000","last_became_active":"2024-09-14T06:29:00.573179+0000","last_became_peered":"2024-09-14T06:29:00.573179+0000","last_unstale":"2024-09-14T06:32:48.620714+0000","last_undegraded":"2024-09-14T06:32:48.620714+0000","last_fullsized":"2024-09-14T06:32:48.620714+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":571,"log_dups_size":0,"ondisk_log_size":571,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T13:31:57.908327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00052833199999999996,"stat_sum":{"num_bytes":8921032,"num_objects":25,"num_object_clones":0,"num_object_copies":50,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":0,"num_read":3678,"num_read_kb":64291,"num_write":753,"num_write_kb":44928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"1f","length":"1"},{"start":"21","length":"1"},{"start":"23","length":"1"},{"start":"25","length":"1"},{"start":"27","length":"1"},{"start":"29","length":"1"},{"start":"2b","length":"1"},{"start":"2d","length":"1"},{"start":"2f","length":"1"},{"start":"31","length":"1"}]},{"pgid":"3.a","version":"246'860","reported_seq":5033,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-09-14T06:32:52.637990+0000","last_change":"2024-09-14T06:32:52.638197+0000","last_active":"2024-09-14T06:32:52.637990+0000","last_peered":"2024-09-14T06:32:52.637990+0000","last_clean":"2024-09-14T06:32:52.637990+0000","last_became_active":"2024-09-14T06:32:51.622296+0000","last_became_peered":"2024-09-14T06:32:51.622296+0000","last_unstale":"2024-09-14T06:32:52.637990+0000","last_undegraded":"2024-09-14T06:32:52.637990+0000","last_fullsized":"2024-09-14T06:32:52.637990+0000","mapping_epoch":245,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":246,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":860,"log_dups_size":0,"ondisk_log_size":860,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T07:36:20.769957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00060112000000000002,"stat_sum":{"num_bytes":26873464,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":5331,"num_read_kb":58329,"num_write":1209,"num_write_kb":47535,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":326,"num_bytes_recovered":26402462,"num_keys_recovered":45,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"1b","length":"1"},{"start":"1d","length":"1"},{"start":"1f","length":"1"},{"start":"21","length":"1"}]},{"pgid":"3.b","version":"241'1600","reported_seq":8794,"reported_epoch":245,"state":"active+clean","last_fresh":"2024-09-14T06:32:50.621523+0000","last_change":"2024-09-14T06:32:50.621737+0000","last_active":"2024-09-14T06:32:50.621523+0000","last_peered":"2024-09-14T06:32:50.621523+0000","last_clean":"2024-09-14T06:32:50.621523+0000","last_became_active":"2024-09-14T06:31:55.756219+0000","last_became_peered":"2024-09-14T06:31:55.756219+0000","last_unstale":"2024-09-14T06:32:50.621523+0000","last_undegraded":"2024-09-14T06:32:50.621523+0000","last_fullsized":"2024-09-14T06:32:50.621523+0000","mapping_epoch":190,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":191,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":1600,"log_dups_size":0,"ondisk_log_size":1600,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T12:17:28.102314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070935799999999999,"stat_sum":{"num_bytes":19338577,"num_objects":26,"num_object_clones":0,"num_object_copies":52,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":0,"num_read":13006,"num_read_kb":64103,"num_write":2457,"num_write_kb":43674,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":291,"num_bytes_recovered":37370632,"num_keys_recovered":66,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"1d","length":"1"},{"start":"1f","length":"1"},{"start":"21","length":"1"}]},{"pgid":"3.c","version":"241'598","reported_seq":6167,"reported_epoch":245,"state":"active+clean","last_fresh":"2024-09-14T06:32:50.621129+0000","last_change":"2024-09-14T06:32:50.621328+0000","last_active":"2024-09-14T06:32:50.621129+0000","last_peered":"2024-09-14T06:32:50.621129+0000","last_clean":"2024-09-14T06:32:50.621129+0000","last_became_active":"2024-09-14T06:32:24.716421+0000","last_became_peered":"2024-09-14T06:32:24.716421+0000","last_unstale":"2024-09-14T06:32:50.621129+0000","last_undegraded":"2024-09-14T06:32:50.621129+0000","last_fullsized":"2024-09-14T06:32:50.621129+0000","mapping_epoch":218,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":219,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":598,"log_dups_size":0,"ondisk_log_size":598,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T08:14:40.439782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001196377,"stat_sum":{"num_bytes":22002358,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":4842,"num_read_kb":50425,"num_write":774,"num_write_kb":40827,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":293,"num_bytes_recovered":36712421,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"1d","length":"1"},{"start":"1f","length":"1"},{"start":"21","length":"1"}]},{"pgid":"3.d","version":"241'536","reported_seq":9055,"reported_epoch":243,"state":"active+clean","last_fresh":"2024-09-14T06:32:48.855533+0000","last_change":"2024-09-14T06:32:47.714296+0000","last_active":"2024-09-14T06:32:48.855533+0000","last_peered":"2024-09-14T06:32:48.855533+0000","last_clean":"2024-09-14T06:32:48.855533+0000","last_became_active":"2024-09-14T06:29:00.580362+0000","last_became_peered":"2024-09-14T06:29:00.580362+0000","last_unstale":"2024-09-14T06:32:48.855533+0000","last_undegraded":"2024-09-14T06:32:48.855533+0000","last_fullsized":"2024-09-14T06:32:48.855533+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":536,"log_dups_size":0,"ondisk_log_size":536,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T07:25:49.985710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023500230000000001,"stat_sum":{"num_bytes":21387510,"num_objects":20,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":0,"num_read":8026,"num_read_kb":93543,"num_write":696,"num_write_kb":41026,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"1f","length":"1"},{"start":"21","length":"1"},{"start":"23","length":"1"},{"start":"25","length":"1"},{"start":"27","length":"1"},{"start":"29","length":"1"},{"start":"2b","length":"1"},{"start":"2d","length":"1"},{"start":"2f","length":"1"},{"start":"31","length":"1"}]},{"pgid":"3.e","version":"242'560","reported_seq":5338,"reported_epoch":243,"state":"active+clean","last_fresh":"2024-09-14T06:32:48.855686+0000","last_change":"2024-09-14T06:32:47.713085+0000","last_active":"2024-09-14T06:32:48.855686+0000","last_peered":"2024-09-14T06:32:48.855686+0000","last_clean":"2024-09-14T06:32:48.855686+0000","last_became_active":"2024-09-14T06:32:39.777524+0000","last_became_peered":"2024-09-14T06:32:39.777524+0000","last_unstale":"2024-09-14T06:32:48.855686+0000","last_undegraded":"2024-09-14T06:32:48.855686+0000","last_fullsized":"2024-09-14T06:32:48.855686+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":560,"log_dups_size":0,"ondisk_log_size":560,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T17:06:58.558516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012157699999999999,"stat_sum":{"num_bytes":7037150,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":3984,"num_read_kb":61137,"num_write":729,"num_write_kb":43787,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":171,"num_bytes_recovered":10913227,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"1f","length":"1"},{"start":"21","length":"1"},{"start":"23","length":"1"},{"start":"25","length":"1"},{"start":"27","length":"1"},{"start":"29","length":"1"},{"start":"2b","length":"1"},{"start":"2d","length":"1"},{"start":"2f","length":"1"},{"start":"31","length":"1"}]},{"pgid":"3.f","version":"241'695","reported_seq":5767,"reported_epoch":243,"state":"active+clean","last_fresh":"2024-09-14T06:32:48.855495+0000","last_change":"2024-09-14T06:32:47.712667+0000","last_active":"2024-09-14T06:32:48.855495+0000","last_peered":"2024-09-14T06:32:48.855495+0000","last_clean":"2024-09-14T06:32:48.855495+0000","last_became_active":"2024-09-14T06:29:00.586491+0000","last_became_peered":"2024-09-14T06:29:00.586491+0000","last_unstale":"2024-09-14T06:32:48.855495+0000","last_undegraded":"2024-09-14T06:32:48.855495+0000","last_fullsized":"2024-09-14T06:32:48.855495+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":695,"log_dups_size":0,"ondisk_log_size":695,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T14:17:31.563305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00090730400000000005,"stat_sum":{"num_bytes":15703454,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":0,"num_read":6076,"num_read_kb":45576,"num_write":819,"num_write_kb":36134,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"1f","length":"1"},{"start":"21","length":"1"},{"start":"23","length":"1"},{"start":"25","length":"1"},{"start":"27","length":"1"},{"start":"29","length":"1"},{"start":"2b","length":"1"},{"start":"2d","length":"1"},{"start":"2f","length":"1"},{"start":"31","length":"1"}]},{"pgid":"3.10","version":"246'940","reported_seq":6731,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-09-14T06:32:52.639687+0000","last_change":"2024-09-14T06:32:52.639914+0000","last_active":"2024-09-14T06:32:52.639687+0000","last_peered":"2024-09-14T06:32:52.639687+0000","last_clean":"2024-09-14T06:32:52.639687+0000","last_became_active":"2024-09-14T06:32:04.621617+0000","last_became_peered":"2024-09-14T06:32:04.621617+0000","last_unstale":"2024-09-14T06:32:52.639687+0000","last_undegraded":"2024-09-14T06:32:52.639687+0000","last_fullsized":"2024-09-14T06:32:52.639687+0000","mapping_epoch":199,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":200,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":940,"log_dups_size":0,"ondisk_log_size":940,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T11:34:20.051776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00048729300000000003,"stat_sum":{"num_bytes":9332925,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":0,"num_read":7145,"num_read_kb":34473,"num_write":1071,"num_write_kb":36655,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":154,"num_bytes_recovered":9401812,"num_keys_recovered":45,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"1b","length":"1"},{"start":"1d","length":"1"},{"start":"1f","length":"1"},{"start":"21","length":"1"}]},{"pgid":"3.11","version":"242'689","reported_seq":11541,"reported_epoch":243,"state":"active+clean","last_fresh":"2024-09-14T06:32:48.855132+0000","last_change":"2024-09-14T06:32:47.712372+0000","last_active":"2024-09-14T06:32:48.855132+0000","last_peered":"2024-09-14T06:32:48.855132+0000","last_clean":"2024-09-14T06:32:48.855132+0000","last_became_active":"2024-09-14T06:32:39.778010+0000","last_became_peered":"2024-09-14T06:32:39.778010+0000","last_unstale":"2024-09-14T06:32:48.855132+0000","last_undegraded":"2024-09-14T06:32:48.855132+0000","last_fullsized":"2024-09-14T06:32:48.855132+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":689,"log_dups_size":0,"ondisk_log_size":689,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T12:49:15.956762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000736785,"stat_sum":{"num_bytes":35702475,"num_objects":27,"num_object_clones":0,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":27,"num_whiteouts":0,"num_read":10240,"num_read_kb":109219,"num_write":819,"num_write_kb":48302,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":177,"num_bytes_recovered":16647747,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"1f","length":"1"},{"start":"21","length":"1"},{"start":"23","length":"1"},{"start":"25","length":"1"},{"start":"27","length":"1"},{"start":"29","length":"1"},{"start":"2b","length":"1"},{"start":"2d","length":"1"},{"start":"2f","length":"1"},{"start":"31","length":"1"}]},{"pgid":"3.12","version":"247'745","reported_seq":7319,"reported_epoch":247,"state":"active+clean","last_fresh":"2024-09-14T06:32:52.721309+0000","last_change":"2024-09-14T06:32:52.640803+0000","last_active":"2024-09-14T06:32:52.721309+0000","last_peered":"2024-09-14T06:32:52.721309+0000","last_clean":"2024-09-14T06:32:52.721309+0000","last_became_active":"2024-09-14T06:29:00.579826+0000","last_became_peered":"2024-09-14T06:29:00.579826+0000","last_unstale":"2024-09-14T06:32:52.721309+0000","last_undegraded":"2024-09-14T06:32:52.721309+0000","last_fullsized":"2024-09-14T06:32:52.721309+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":745,"log_dups_size":0,"ondisk_log_size":745,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T15:57:49.990431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001257242,"stat_sum":{"num_bytes":10186161,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":0,"num_read":7048,"num_read_kb":59041,"num_write":907,"num_write_kb":42431,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"1b","length":"1"},{"start":"1d","length":"1"},{"start":"1f","length":"1"},{"start":"21","length":"1"}]},{"pgid":"3.13","version":"241'980","reported_seq":8768,"reported_epoch":243,"state":"active+clean","last_fresh":"2024-09-14T06:32:48.855174+0000","last_change":"2024-09-14T06:32:47.712170+0000","last_active":"2024-09-14T06:32:48.855174+0000","last_peered":"2024-09-14T06:32:48.855174+0000","last_clean":"2024-09-14T06:32:48.855174+0000","last_became_active":"2024-09-14T06:29:00.586520+0000","last_became_peered":"2024-09-14T06:29:00.586520+0000","last_unstale":"2024-09-14T06:32:48.855174+0000","last_undegraded":"2024-09-14T06:32:48.855174+0000","last_fullsized":"2024-09-14T06:32:48.855174+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":980,"log_dups_size":0,"ondisk_log_size":980,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T13:35:17.848200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00056844600000000001,"stat_sum":{"num_bytes":32440004,"num_objects":27,"num_object_clones":0,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":27,"num_whiteouts":0,"num_read":9546,"num_read_kb":66117,"num_write":1208,"num_write_kb":47437,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"1f","length":"1"},{"start":"21","length":"1"},{"start":"23","length":"1"},{"start":"25","length":"1"},{"start":"27","length":"1"},{"start":"29","length":"1"},{"start":"2b","length":"1"},{"start":"2d","length":"1"},{"start":"2f","length":"1"},{"start":"31","length":"1"}]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":616610767,"num_objects":646,"num_object_clones":0,"num_object_copies":1292,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":646,"num_whiteouts":0,"num_read":237426,"num_read_kb":2182640,"num_write":37075,"num_write_kb":1478217,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5314,"num_bytes_recovered":637770699,"num_keys_recovered":546,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":43204608,"data_stored":363638516,"data_compressed":22940966,"data_compressed_allocated":30863360,"data_compressed_original":353976121,"omap_allocated":0,"internal_metadata":0},"log_size":27495,"ondisk_log_size":27495,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":131072,"data_stored":1180736,"data_compressed":65166,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509492,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":70820,"kb_used_data":7696,"kb_used_omap":1,"kb_used_meta":63102,"kb_avail":94301020,"statfs":{"total":96636764160,"available":96564244480,"internally_reserved":0,"allocated":7880704,"data_stored":43429711,"data_compressed":2389556,"data_compressed_allocated":3244032,"data_compressed_original":40621759,"omap_allocated":1590,"internal_metadata":64616906},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":0,"seq":64424509492,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509493,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":76388,"kb_used_data":9104,"kb_used_omap":1,"kb_used_meta":67262,"kb_avail":94295452,"statfs":{"total":96636764160,"available":96558542848,"internally_reserved":0,"allocated":9322496,"data_stored":52008830,"data_compressed":3312020,"data_compressed_allocated":4460544,"data_compressed_original":49037943,"omap_allocated":1595,"internal_metadata":68876741},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509492,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":84464,"kb_used_data":8852,"kb_used_omap":1,"kb_used_meta":75582,"kb_avail":94287376,"statfs":{"total":96636764160,"available":96550273024,"internally_reserved":0,"allocated":9064448,"data_stored":58295648,"data_compressed":3686011,"data_compressed_allocated":4825088,"data_compressed_original":55874291,"omap_allocated":1595,"internal_metadata":77396421},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509493,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":58712,"kb_used_data":9236,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94313128,"statfs":{"total":96636764160,"available":96576643072,"internally_reserved":0,"allocated":9457664,"data_stored":58006131,"data_compressed":3623205,"data_compressed_allocated":4841472,"data_compressed_original":55276789,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":14,"seq":60129542197,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":35948,"kb_used_data":4760,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94335892,"statfs":{"total":96636764160,"available":96599953408,"internally_reserved":0,"allocated":4874240,"data_stored":15856387,"data_compressed":905082,"data_compressed_allocated":1261568,"data_compressed_original":13939401,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542196,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":82660,"kb_used_data":6952,"kb_used_omap":1,"kb_used_meta":75582,"kb_avail":94289180,"statfs":{"total":96636764160,"available":96552120320,"internally_reserved":0,"allocated":7118848,"data_stored":29931553,"data_compressed":1710296,"data_compressed_allocated":2285568,"data_compressed_original":26962046,"omap_allocated":1595,"internal_metadata":77396421},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574901,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":85488,"kb_used_data":9880,"kb_used_omap":1,"kb_used_meta":75582,"kb_avail":94286352,"statfs":{"total":96636764160,"available":96549224448,"internally_reserved":0,"allocated":10117120,"data_stored":58870719,"data_compressed":3842741,"data_compressed_allocated":5181440,"data_compressed_original":55832576,"omap_allocated":1593,"internal_metadata":77396423},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32583,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32583,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":3563520,"data_stored":21062551,"data_compressed":1278601,"data_compressed_allocated":1732608,"data_compressed_original":19572862,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":6529024,"data_stored":56568374,"data_compressed":3623205,"data_compressed_allocated":4841472,"data_compressed_original":55276789,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":1888256,"data_stored":14387123,"data_compressed":905082,"data_compressed_allocated":1261568,"data_compressed_original":13939401,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":7168000,"data_stored":57420630,"data_compressed":3842741,"data_compressed_allocated":5181440,"data_compressed_original":55832576,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":6139904,"data_stored":56857910,"data_compressed":3686011,"data_compressed_allocated":4825088,"data_compressed_original":55874291,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":6393856,"data_stored":50571073,"data_compressed":3312020,"data_compressed_allocated":4460544,"data_compressed_original":49037943,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":6631424,"data_stored":65369250,"data_compressed":3936333,"data_compressed_allocated":5361664,"data_compressed_original":64393940,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4890624,"data_stored":41401605,"data_compressed":2356973,"data_compressed_allocated":3198976,"data_compressed_original":40048319,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-14T06:32:53.414 INFO:tasks.thrashosds.thrasher:Setting 2.6 to [3, 5, 4, 6] 2024-09-14T06:32:53.414 INFO:tasks.thrashosds.thrasher:cmd ['osd', 'pg-upmap-items', '2.6', '3', '5', '4', '6'] 2024-09-14T06:32:53.414 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pg-upmap-items 2.6 3 5 4 6 2024-09-14T06:32:53.414 INFO:tasks.ceph.osd.3.smithi094.stderr:2024-09-14T06:32:53.406+0000 7feeeafe8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33655) UID: 0 2024-09-14T06:32:53.508 INFO:tasks.ceph.osd.7.smithi138.stderr:2024-09-14T06:32:53.507+0000 7f3c6a21d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33623) UID: 0 2024-09-14T06:32:53.517 INFO:teuthology.orchestra.run.smithi138.stdout:{ 2024-09-14T06:32:53.517 INFO:teuthology.orchestra.run.smithi138.stdout: "ops": [], 2024-09-14T06:32:53.517 INFO:teuthology.orchestra.run.smithi138.stdout: "num_ops": 0 2024-09-14T06:32:53.517 INFO:teuthology.orchestra.run.smithi138.stdout:} 2024-09-14T06:32:53.526 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-14T06:32:53.609 INFO:tasks.ceph.osd.2.smithi094.stderr:2024-09-14T06:32:53.608+0000 7faf3269b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33649) UID: 0 2024-09-14T06:32:53.637 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-14T06:32:53.673 INFO:teuthology.orchestra.run.smithi138.stdout:{ 2024-09-14T06:32:53.674 INFO:teuthology.orchestra.run.smithi138.stdout: "ops": [], 2024-09-14T06:32:53.674 INFO:teuthology.orchestra.run.smithi138.stdout: "complaint_time": 30, 2024-09-14T06:32:53.674 INFO:teuthology.orchestra.run.smithi138.stdout: "num_blocked_ops": 0 2024-09-14T06:32:53.674 INFO:teuthology.orchestra.run.smithi138.stdout:} 2024-09-14T06:32:53.683 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-14T06:32:53.710 INFO:tasks.ceph.osd.1.smithi094.stderr:2024-09-14T06:32:53.709+0000 7f328e47a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33641) UID: 0 2024-09-14T06:32:53.801 INFO:teuthology.orchestra.run.smithi138.stdout:{ 2024-09-14T06:32:53.801 INFO:teuthology.orchestra.run.smithi138.stdout: "size": 20, 2024-09-14T06:32:53.801 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 600, 2024-09-14T06:32:53.802 INFO:teuthology.orchestra.run.smithi138.stdout: "ops": [ 2024-09-14T06:32:53.802 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:53.802 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:98505 3.10 3.1bf56f90 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T06:32:53.802 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:30:07.045453+0000", 2024-09-14T06:32:53.802 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 166.754555549, 2024-09-14T06:32:53.802 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032144440000000003, 2024-09-14T06:32:53.802 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:32:53.802 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:32:53.802 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "reached pg", 2024-09-14T06:32:53.802 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:32:53.802 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:32:53.802 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:53.803 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 98505 2024-09-14T06:32:53.803 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:53.803 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:32:53.803 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:53.803 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:32:53.803 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045453+0000", 2024-09-14T06:32:53.803 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:53.803 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:53.803 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:53.803 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:32:53.803 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045453+0000", 2024-09-14T06:32:53.803 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:53.803 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:53.804 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:53.804 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:32:53.804 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045453+0000", 2024-09-14T06:32:53.804 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.36e-07 2024-09-14T06:32:53.804 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:53.804 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:53.804 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:32:53.804 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045454+0000", 2024-09-14T06:32:53.804 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.1299999999999998e-07 2024-09-14T06:32:53.804 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:53.804 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:53.804 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:32:53.805 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045457+0000", 2024-09-14T06:32:53.805 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.8559999999999998e-06 2024-09-14T06:32:53.805 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:53.805 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:53.805 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:32:53.805 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045460+0000", 2024-09-14T06:32:53.805 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.8750000000000002e-06 2024-09-14T06:32:53.805 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:53.805 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:53.805 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:32:53.805 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.077592+0000", 2024-09-14T06:32:53.805 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032131727999999998 2024-09-14T06:32:53.805 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:53.805 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:53.806 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:32:53.806 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.077597+0000", 2024-09-14T06:32:53.806 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.8319999999999996e-06 2024-09-14T06:32:53.806 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:53.806 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:32:53.806 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:53.806 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:53.806 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:53.806 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:98506 3.10 3.1bf56f90 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T06:32:53.806 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:30:07.045481+0000", 2024-09-14T06:32:53.806 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 166.75452729400001, 2024-09-14T06:32:53.806 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032142759999999999, 2024-09-14T06:32:53.806 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:32:53.806 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:32:53.807 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "reached pg", 2024-09-14T06:32:53.807 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:32:53.807 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:32:53.807 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:53.807 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 98506 2024-09-14T06:32:53.807 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:53.807 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:32:53.807 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:53.807 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:32:53.807 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045481+0000", 2024-09-14T06:32:53.807 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:53.807 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:53.807 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:53.807 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:32:53.808 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045481+0000", 2024-09-14T06:32:53.808 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:53.808 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:53.808 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:53.808 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:32:53.808 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045481+0000", 2024-09-14T06:32:53.808 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.8500000000000001e-07 2024-09-14T06:32:53.808 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:53.808 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:53.808 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:32:53.808 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045483+0000", 2024-09-14T06:32:53.808 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.8250000000000001e-06 2024-09-14T06:32:53.808 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:53.809 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:53.809 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:32:53.809 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045489+0000", 2024-09-14T06:32:53.809 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.5740000000000003e-06 2024-09-14T06:32:53.809 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:53.809 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:53.809 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:32:53.809 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045495+0000", 2024-09-14T06:32:53.809 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.7690000000000002e-06 2024-09-14T06:32:53.809 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:53.809 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:53.809 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:32:53.809 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.077619+0000", 2024-09-14T06:32:53.809 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032124621999999999 2024-09-14T06:32:53.810 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:53.810 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:53.810 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:32:53.810 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.077624+0000", 2024-09-14T06:32:53.810 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.3850000000000004e-06 2024-09-14T06:32:53.810 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:53.810 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:32:53.810 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:53.810 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:53.810 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:53.810 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:98507 3.10 3.1bf56f90 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T06:32:53.810 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:30:07.045502+0000", 2024-09-14T06:32:53.810 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 166.754505873, 2024-09-14T06:32:53.810 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032147219999999997, 2024-09-14T06:32:53.811 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:32:53.811 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:32:53.811 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "reached pg", 2024-09-14T06:32:53.811 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:32:53.811 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:32:53.811 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:53.811 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 98507 2024-09-14T06:32:53.811 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:53.811 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:32:53.811 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:53.811 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:32:53.811 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045502+0000", 2024-09-14T06:32:53.811 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:53.812 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:53.812 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:53.812 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:32:53.812 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045502+0000", 2024-09-14T06:32:53.812 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:53.812 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:53.812 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:53.812 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:32:53.812 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045503+0000", 2024-09-14T06:32:53.812 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.4700000000000002e-07 2024-09-14T06:32:53.812 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:53.812 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:53.812 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:32:53.812 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045509+0000", 2024-09-14T06:32:53.813 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.7329999999999997e-06 2024-09-14T06:32:53.813 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:53.813 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:53.813 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:32:53.813 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045515+0000", 2024-09-14T06:32:53.813 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.8030000000000002e-06 2024-09-14T06:32:53.813 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:53.813 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:53.813 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:32:53.813 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045522+0000", 2024-09-14T06:32:53.814 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.0369999999999997e-06 2024-09-14T06:32:53.814 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:53.814 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:53.814 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:32:53.814 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.077641+0000", 2024-09-14T06:32:53.814 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032119091000000002 2024-09-14T06:32:53.814 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:53.814 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:53.814 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:32:53.814 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.077650+0000", 2024-09-14T06:32:53.814 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.2090000000000005e-06 2024-09-14T06:32:53.814 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:53.814 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:32:53.814 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:53.815 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:53.815 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:53.815 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:98508 3.10 3.1bf56f90 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T06:32:53.815 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:30:07.045530+0000", 2024-09-14T06:32:53.815 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 166.754478256, 2024-09-14T06:32:53.815 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032198773999999999, 2024-09-14T06:32:53.815 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:32:53.815 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:32:53.815 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "reached pg", 2024-09-14T06:32:53.815 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:32:53.815 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:32:53.815 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:53.815 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 98508 2024-09-14T06:32:53.815 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:53.816 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:32:53.816 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:53.816 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:32:53.816 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045530+0000", 2024-09-14T06:32:53.816 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:53.816 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:53.816 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:53.816 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:32:53.816 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045530+0000", 2024-09-14T06:32:53.816 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:53.816 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:53.816 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:53.816 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:32:53.817 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045530+0000", 2024-09-14T06:32:53.817 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.3099999999999998e-07 2024-09-14T06:32:53.817 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:53.817 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:53.817 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:32:53.817 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045539+0000", 2024-09-14T06:32:53.817 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.4780000000000008e-06 2024-09-14T06:32:53.817 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:53.817 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:53.817 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:32:53.817 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045546+0000", 2024-09-14T06:32:53.817 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.3259999999999998e-06 2024-09-14T06:32:53.817 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:53.818 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:53.818 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:32:53.818 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045551+0000", 2024-09-14T06:32:53.818 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.9960000000000001e-06 2024-09-14T06:32:53.818 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:53.818 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:53.818 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:32:53.818 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.077716+0000", 2024-09-14T06:32:53.818 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032165081999999998 2024-09-14T06:32:53.818 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:53.818 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:53.818 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:32:53.818 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.077729+0000", 2024-09-14T06:32:53.818 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.2461e-05 2024-09-14T06:32:53.819 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:53.819 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:32:53.819 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:53.819 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:53.819 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:53.819 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:98509 3.10 3.1bf56f90 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T06:32:53.819 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:30:07.045558+0000", 2024-09-14T06:32:53.819 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 166.75445005500001, 2024-09-14T06:32:53.819 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032242198, 2024-09-14T06:32:53.819 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:32:53.819 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:32:53.819 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "reached pg", 2024-09-14T06:32:53.819 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:32:53.819 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:32:53.820 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:53.820 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 98509 2024-09-14T06:32:53.820 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:53.820 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:32:53.820 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:53.820 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:32:53.820 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045558+0000", 2024-09-14T06:32:53.820 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:53.820 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:53.820 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:53.820 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:32:53.820 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045558+0000", 2024-09-14T06:32:53.820 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:53.821 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:53.821 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:53.821 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:32:53.821 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045559+0000", 2024-09-14T06:32:53.821 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.27e-07 2024-09-14T06:32:53.821 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:53.821 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:53.821 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:32:53.821 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045560+0000", 2024-09-14T06:32:53.821 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.6959999999999999e-06 2024-09-14T06:32:53.821 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:53.821 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:53.821 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:32:53.822 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045567+0000", 2024-09-14T06:32:53.822 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.6900000000000003e-06 2024-09-14T06:32:53.822 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:53.822 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:53.822 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:32:53.822 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045571+0000", 2024-09-14T06:32:53.822 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.5750000000000002e-06 2024-09-14T06:32:53.822 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:53.822 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:53.822 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:32:53.822 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.077789+0000", 2024-09-14T06:32:53.822 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032217186000000002 2024-09-14T06:32:53.823 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:53.823 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:53.823 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:32:53.823 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.077800+0000", 2024-09-14T06:32:53.823 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.1623999999999999e-05 2024-09-14T06:32:53.823 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:53.823 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:32:53.823 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:53.823 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:53.823 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:53.823 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:98510 3.10 3.1bf56f90 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T06:32:53.823 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:30:07.045580+0000", 2024-09-14T06:32:53.823 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 166.754428241, 2024-09-14T06:32:53.823 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032299845000000001, 2024-09-14T06:32:53.824 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:32:53.824 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:32:53.824 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "reached pg", 2024-09-14T06:32:53.824 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:32:53.824 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:32:53.824 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:53.824 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 98510 2024-09-14T06:32:53.824 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:53.824 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:32:53.824 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:53.824 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:32:53.824 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045580+0000", 2024-09-14T06:32:53.824 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:53.824 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:53.825 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:53.825 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:32:53.825 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045580+0000", 2024-09-14T06:32:53.825 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:53.825 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:53.825 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:53.825 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:32:53.825 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045580+0000", 2024-09-14T06:32:53.825 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.0400000000000002e-07 2024-09-14T06:32:53.825 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:53.825 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:53.825 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:32:53.825 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045582+0000", 2024-09-14T06:32:53.825 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.7600000000000001e-06 2024-09-14T06:32:53.825 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:53.826 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:53.826 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:32:53.826 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045588+0000", 2024-09-14T06:32:53.826 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.5650000000000001e-06 2024-09-14T06:32:53.826 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:53.826 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:53.826 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:32:53.826 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045597+0000", 2024-09-14T06:32:53.826 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.465e-06 2024-09-14T06:32:53.826 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:53.826 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:53.826 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:32:53.826 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.077871+0000", 2024-09-14T06:32:53.826 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032273562999999998 2024-09-14T06:32:53.826 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:53.827 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:53.827 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:32:53.827 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.077880+0000", 2024-09-14T06:32:53.827 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.0880000000000001e-06 2024-09-14T06:32:53.827 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:53.827 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:32:53.827 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:53.827 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:53.827 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:53.827 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:98511 3.10 3.1bf56f90 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T06:32:53.827 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:30:07.045610+0000", 2024-09-14T06:32:53.827 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 166.75439839399999, 2024-09-14T06:32:53.827 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032338894, 2024-09-14T06:32:53.827 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:32:53.827 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:32:53.828 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "reached pg", 2024-09-14T06:32:53.828 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:32:53.828 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:32:53.828 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:53.828 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 98511 2024-09-14T06:32:53.828 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:53.828 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:32:53.828 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:53.828 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:32:53.828 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045610+0000", 2024-09-14T06:32:53.828 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:53.828 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:53.828 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:53.828 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:32:53.829 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045610+0000", 2024-09-14T06:32:53.829 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:53.829 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:53.829 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:53.829 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:32:53.829 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045610+0000", 2024-09-14T06:32:53.829 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.1600000000000002e-07 2024-09-14T06:32:53.829 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:53.829 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:53.829 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:32:53.829 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045612+0000", 2024-09-14T06:32:53.829 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.1150000000000001e-06 2024-09-14T06:32:53.829 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:53.829 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:53.829 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:32:53.830 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045618+0000", 2024-09-14T06:32:53.830 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.0499999999999997e-06 2024-09-14T06:32:53.830 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:53.830 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:53.830 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:32:53.830 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045623+0000", 2024-09-14T06:32:53.830 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.6770000000000003e-06 2024-09-14T06:32:53.830 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:53.830 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:53.830 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:32:53.830 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.077928+0000", 2024-09-14T06:32:53.830 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032305111999999997 2024-09-14T06:32:53.830 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:53.830 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:53.831 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:32:53.831 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.077949+0000", 2024-09-14T06:32:53.831 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.0523999999999999e-05 2024-09-14T06:32:53.831 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:53.831 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:32:53.831 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:53.831 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:53.831 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:53.831 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:98512 3.10 3.1bf56f90 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T06:32:53.831 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:30:07.045630+0000", 2024-09-14T06:32:53.831 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 166.75437838900001, 2024-09-14T06:32:53.831 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032346632, 2024-09-14T06:32:53.831 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:32:53.831 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:32:53.831 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "reached pg", 2024-09-14T06:32:53.832 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:32:53.832 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:32:53.832 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:53.832 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 98512 2024-09-14T06:32:53.832 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:53.832 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:32:53.832 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:53.832 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:32:53.832 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045630+0000", 2024-09-14T06:32:53.832 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:53.832 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:53.832 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:53.832 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:32:53.832 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045630+0000", 2024-09-14T06:32:53.833 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:53.833 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:53.833 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:53.833 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:32:53.833 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045630+0000", 2024-09-14T06:32:53.833 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.8200000000000001e-07 2024-09-14T06:32:53.833 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:53.833 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:53.833 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:32:53.833 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045632+0000", 2024-09-14T06:32:53.833 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.7430000000000001e-06 2024-09-14T06:32:53.834 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:53.834 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:53.834 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:32:53.834 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045637+0000", 2024-09-14T06:32:53.834 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.4550000000000003e-06 2024-09-14T06:32:53.834 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:53.834 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:53.834 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:32:53.834 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045644+0000", 2024-09-14T06:32:53.834 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.1330000000000001e-06 2024-09-14T06:32:53.834 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:53.834 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:53.834 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:32:53.835 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.077968+0000", 2024-09-14T06:32:53.835 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032323401000000002 2024-09-14T06:32:53.835 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:53.835 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:53.835 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:32:53.835 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.077976+0000", 2024-09-14T06:32:53.835 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.5180000000000004e-06 2024-09-14T06:32:53.835 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:53.835 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:32:53.835 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:53.835 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:53.835 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:53.835 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:98513 3.10 3.1bf56f90 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T06:32:53.835 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:30:07.045652+0000", 2024-09-14T06:32:53.835 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 166.754355825, 2024-09-14T06:32:53.836 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032353436999999999, 2024-09-14T06:32:53.836 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:32:53.836 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:32:53.836 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "reached pg", 2024-09-14T06:32:53.836 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:32:53.836 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:32:53.836 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:53.836 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 98513 2024-09-14T06:32:53.836 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:53.836 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:32:53.836 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:53.836 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:32:53.837 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045652+0000", 2024-09-14T06:32:53.837 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:53.837 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:53.837 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:53.837 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:32:53.837 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045652+0000", 2024-09-14T06:32:53.837 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:53.837 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:53.837 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:53.837 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:32:53.837 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045653+0000", 2024-09-14T06:32:53.837 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.3700000000000001e-07 2024-09-14T06:32:53.837 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:53.837 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:53.838 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:32:53.838 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045655+0000", 2024-09-14T06:32:53.838 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.854e-06 2024-09-14T06:32:53.838 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:53.838 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:53.838 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:32:53.838 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045661+0000", 2024-09-14T06:32:53.838 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.5509999999999996e-06 2024-09-14T06:32:53.838 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:53.838 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:53.838 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:32:53.838 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045668+0000", 2024-09-14T06:32:53.839 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.1980000000000004e-06 2024-09-14T06:32:53.839 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:53.839 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:53.839 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:32:53.839 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.077997+0000", 2024-09-14T06:32:53.839 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032328575999999998 2024-09-14T06:32:53.839 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:53.839 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:53.839 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:32:53.839 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.078006+0000", 2024-09-14T06:32:53.839 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.8210000000000008e-06 2024-09-14T06:32:53.839 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:53.839 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:32:53.839 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:53.839 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:53.840 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:53.840 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:98514 3.10 3.1bf56f90 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T06:32:53.840 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:30:07.045699+0000", 2024-09-14T06:32:53.840 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 166.75430949, 2024-09-14T06:32:53.840 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032330223999999998, 2024-09-14T06:32:53.840 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:32:53.840 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:32:53.840 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "reached pg", 2024-09-14T06:32:53.840 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:32:53.840 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:32:53.840 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:53.840 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 98514 2024-09-14T06:32:53.840 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:53.841 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:32:53.841 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:53.841 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:32:53.841 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045699+0000", 2024-09-14T06:32:53.841 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:53.841 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:53.841 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:53.841 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:32:53.841 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045699+0000", 2024-09-14T06:32:53.841 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:53.841 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:53.841 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:53.841 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:32:53.841 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045699+0000", 2024-09-14T06:32:53.841 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.3200000000000001e-07 2024-09-14T06:32:53.842 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:53.842 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:53.842 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:32:53.842 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045701+0000", 2024-09-14T06:32:53.842 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.762e-06 2024-09-14T06:32:53.842 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:53.842 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:53.842 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:32:53.842 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045705+0000", 2024-09-14T06:32:53.842 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.2359999999999998e-06 2024-09-14T06:32:53.842 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:53.842 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:53.842 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:32:53.842 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045708+0000", 2024-09-14T06:32:53.843 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.796e-06 2024-09-14T06:32:53.843 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:53.843 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:53.843 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:32:53.843 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.078023+0000", 2024-09-14T06:32:53.843 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032315213000000002 2024-09-14T06:32:53.843 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:53.843 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:53.843 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:32:53.843 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.078029+0000", 2024-09-14T06:32:53.843 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.8850000000000004e-06 2024-09-14T06:32:53.843 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:53.843 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:32:53.843 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:53.843 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:53.844 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:53.844 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:98515 3.10 3.1bf56f90 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T06:32:53.844 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:30:07.045712+0000", 2024-09-14T06:32:53.844 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 166.75429617899999, 2024-09-14T06:32:53.844 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032346871999999999, 2024-09-14T06:32:53.844 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:32:53.844 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:32:53.844 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "reached pg", 2024-09-14T06:32:53.844 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:32:53.844 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:32:53.844 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:53.844 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 98515 2024-09-14T06:32:53.844 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:53.844 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:32:53.845 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:53.845 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:32:53.845 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045712+0000", 2024-09-14T06:32:53.845 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:53.845 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:53.845 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:53.845 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:32:53.845 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045712+0000", 2024-09-14T06:32:53.845 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:53.845 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:53.845 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:53.845 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:32:53.845 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045712+0000", 2024-09-14T06:32:53.845 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.2100000000000001e-07 2024-09-14T06:32:53.846 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:53.846 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:53.846 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:32:53.846 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045713+0000", 2024-09-14T06:32:53.846 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.068e-06 2024-09-14T06:32:53.846 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:53.846 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:53.846 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:32:53.846 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045716+0000", 2024-09-14T06:32:53.846 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.971e-06 2024-09-14T06:32:53.846 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:53.846 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:53.846 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:32:53.847 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045719+0000", 2024-09-14T06:32:53.847 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.3879999999999998e-06 2024-09-14T06:32:53.847 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:53.847 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:53.847 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:32:53.847 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.078049+0000", 2024-09-14T06:32:53.847 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032330553999999997 2024-09-14T06:32:53.847 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:53.847 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:53.847 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:32:53.847 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.078059+0000", 2024-09-14T06:32:53.847 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.6700000000000006e-06 2024-09-14T06:32:53.847 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:53.847 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:32:53.848 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:53.848 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:53.848 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:53.848 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:98516 3.10 3.1bf56f90 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T06:32:53.848 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:30:07.045722+0000", 2024-09-14T06:32:53.848 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 166.75428642599999, 2024-09-14T06:32:53.848 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032374334999999997, 2024-09-14T06:32:53.848 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:32:53.848 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:32:53.848 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "reached pg", 2024-09-14T06:32:53.848 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:32:53.848 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:32:53.848 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:53.848 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 98516 2024-09-14T06:32:53.848 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:53.849 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:32:53.849 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:53.849 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:32:53.849 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045722+0000", 2024-09-14T06:32:53.849 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:53.849 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:53.849 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:53.849 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:32:53.849 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045722+0000", 2024-09-14T06:32:53.849 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:53.849 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:53.849 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:53.849 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:32:53.849 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045722+0000", 2024-09-14T06:32:53.850 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.8999999999999998e-07 2024-09-14T06:32:53.850 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:53.850 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:53.850 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:32:53.850 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045723+0000", 2024-09-14T06:32:53.850 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.0049999999999999e-06 2024-09-14T06:32:53.850 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:53.850 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:53.850 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:32:53.850 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045726+0000", 2024-09-14T06:32:53.850 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.9019999999999999e-06 2024-09-14T06:32:53.850 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:53.850 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:53.850 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:32:53.851 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045728+0000", 2024-09-14T06:32:53.851 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.3290000000000001e-06 2024-09-14T06:32:53.851 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:53.851 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:53.851 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:32:53.851 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.078090+0000", 2024-09-14T06:32:53.851 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032362123 2024-09-14T06:32:53.851 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:53.851 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:53.851 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:32:53.851 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.078096+0000", 2024-09-14T06:32:53.851 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.6860000000000003e-06 2024-09-14T06:32:53.851 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:53.852 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:32:53.852 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:53.852 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:53.852 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:53.852 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:98517 3.10 3.1bf56f90 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T06:32:53.852 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:30:07.045731+0000", 2024-09-14T06:32:53.852 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 166.75427669499999, 2024-09-14T06:32:53.852 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.03241227, 2024-09-14T06:32:53.852 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:32:53.852 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:32:53.852 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "reached pg", 2024-09-14T06:32:53.852 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:32:53.852 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:32:53.852 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:53.853 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 98517 2024-09-14T06:32:53.853 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:53.853 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:32:53.853 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:53.853 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:32:53.853 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045731+0000", 2024-09-14T06:32:53.853 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:53.853 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:53.853 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:53.853 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:32:53.853 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045731+0000", 2024-09-14T06:32:53.853 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:53.853 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:53.854 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:53.854 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:32:53.854 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045732+0000", 2024-09-14T06:32:53.854 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.0699999999999999e-07 2024-09-14T06:32:53.854 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:53.854 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:53.854 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:32:53.854 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045733+0000", 2024-09-14T06:32:53.854 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.9199999999999999e-07 2024-09-14T06:32:53.854 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:53.854 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:53.854 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:32:53.855 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045735+0000", 2024-09-14T06:32:53.855 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.8150000000000002e-06 2024-09-14T06:32:53.855 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:53.855 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:53.855 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:32:53.855 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045738+0000", 2024-09-14T06:32:53.855 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.3640000000000002e-06 2024-09-14T06:32:53.855 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:53.855 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:53.855 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:32:53.855 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.078126+0000", 2024-09-14T06:32:53.855 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032388206000000003 2024-09-14T06:32:53.856 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:53.856 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:53.856 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:32:53.856 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.078144+0000", 2024-09-14T06:32:53.856 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.7686e-05 2024-09-14T06:32:53.856 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:53.856 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:32:53.856 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:53.856 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:53.856 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:53.856 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:98518 3.10 3.1bf56f90 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T06:32:53.856 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:30:07.045741+0000", 2024-09-14T06:32:53.856 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 166.75426730000001, 2024-09-14T06:32:53.857 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032451793999999999, 2024-09-14T06:32:53.857 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:32:53.857 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:32:53.857 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "reached pg", 2024-09-14T06:32:53.857 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:32:53.857 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:32:53.857 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:53.857 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 98518 2024-09-14T06:32:53.857 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:53.857 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:32:53.857 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:53.858 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:32:53.858 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045741+0000", 2024-09-14T06:32:53.858 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:53.858 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:53.858 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:53.858 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:32:53.858 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045741+0000", 2024-09-14T06:32:53.858 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:53.858 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:53.858 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:53.858 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:32:53.858 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045741+0000", 2024-09-14T06:32:53.859 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.05e-07 2024-09-14T06:32:53.859 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:53.859 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:53.859 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:32:53.859 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045746+0000", 2024-09-14T06:32:53.859 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.2259999999999996e-06 2024-09-14T06:32:53.859 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:53.859 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:53.859 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:32:53.859 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045750+0000", 2024-09-14T06:32:53.859 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.2490000000000001e-06 2024-09-14T06:32:53.859 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:53.859 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:53.859 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:32:53.860 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045752+0000", 2024-09-14T06:32:53.860 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.3980000000000002e-06 2024-09-14T06:32:53.860 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:53.860 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:53.860 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:32:53.860 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.078184+0000", 2024-09-14T06:32:53.860 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032431593000000002 2024-09-14T06:32:53.860 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:53.860 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:53.860 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:32:53.860 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.078193+0000", 2024-09-14T06:32:53.860 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.1230000000000006e-06 2024-09-14T06:32:53.861 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:53.861 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:32:53.861 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:53.861 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:53.861 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:53.861 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:98519 3.10 3.1bf56f90 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T06:32:53.861 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:30:07.045755+0000", 2024-09-14T06:32:53.861 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 166.75425287499999, 2024-09-14T06:32:53.861 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032489431999999999, 2024-09-14T06:32:53.861 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:32:53.861 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:32:53.861 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "reached pg", 2024-09-14T06:32:53.861 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:32:53.861 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:32:53.862 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:53.862 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 98519 2024-09-14T06:32:53.862 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:53.862 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:32:53.862 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:53.862 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:32:53.862 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045755+0000", 2024-09-14T06:32:53.862 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:53.862 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:53.862 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:53.862 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:32:53.862 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045755+0000", 2024-09-14T06:32:53.862 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:53.862 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:53.863 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:53.863 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:32:53.863 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045756+0000", 2024-09-14T06:32:53.863 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.23e-07 2024-09-14T06:32:53.863 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:53.863 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:53.863 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:32:53.863 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045757+0000", 2024-09-14T06:32:53.863 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.1060000000000001e-06 2024-09-14T06:32:53.863 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:53.863 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:53.863 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:32:53.863 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045760+0000", 2024-09-14T06:32:53.863 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.988e-06 2024-09-14T06:32:53.864 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:53.864 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:53.864 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:32:53.864 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045765+0000", 2024-09-14T06:32:53.864 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.2689999999999999e-06 2024-09-14T06:32:53.864 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:53.864 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:53.864 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:32:53.864 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.078234+0000", 2024-09-14T06:32:53.864 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032468915000000001 2024-09-14T06:32:53.864 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:53.864 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:53.864 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:32:53.864 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.078245+0000", 2024-09-14T06:32:53.865 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.0930999999999999e-05 2024-09-14T06:32:53.865 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:53.865 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:32:53.865 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:53.865 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:53.865 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:53.865 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:98520 3.10 3.1bf56f90 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T06:32:53.865 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:30:07.045768+0000", 2024-09-14T06:32:53.865 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 166.75423999700001, 2024-09-14T06:32:53.865 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032531379999999999, 2024-09-14T06:32:53.865 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:32:53.865 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:32:53.865 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "reached pg", 2024-09-14T06:32:53.865 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:32:53.866 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:32:53.866 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:53.866 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 98520 2024-09-14T06:32:53.866 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:53.866 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:32:53.866 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:53.866 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:32:53.866 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045768+0000", 2024-09-14T06:32:53.866 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:53.866 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:53.866 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:53.866 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:32:53.866 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045768+0000", 2024-09-14T06:32:53.867 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:53.867 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:53.867 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:53.867 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:32:53.867 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045768+0000", 2024-09-14T06:32:53.867 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.22e-07 2024-09-14T06:32:53.867 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:53.867 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:53.867 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:32:53.867 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045773+0000", 2024-09-14T06:32:53.867 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.3470000000000001e-06 2024-09-14T06:32:53.867 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:53.867 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:53.867 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:32:53.868 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045776+0000", 2024-09-14T06:32:53.868 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.1319999999999998e-06 2024-09-14T06:32:53.868 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:53.868 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:53.868 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:32:53.868 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045779+0000", 2024-09-14T06:32:53.868 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.9749999999999999e-06 2024-09-14T06:32:53.868 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:53.868 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:53.868 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:32:53.868 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.078289+0000", 2024-09-14T06:32:53.868 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032509693999999999 2024-09-14T06:32:53.868 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:53.868 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:53.868 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:32:53.869 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.078300+0000", 2024-09-14T06:32:53.869 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.101e-05 2024-09-14T06:32:53.869 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:53.869 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:32:53.869 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:53.869 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:53.869 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:53.869 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:98521 3.10 3.1bf56f90 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T06:32:53.869 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:30:07.045782+0000", 2024-09-14T06:32:53.869 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 166.75422581699999, 2024-09-14T06:32:53.869 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032566109000000003, 2024-09-14T06:32:53.869 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:32:53.869 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:32:53.869 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "reached pg", 2024-09-14T06:32:53.869 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:32:53.870 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:32:53.870 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:53.870 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 98521 2024-09-14T06:32:53.870 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:53.870 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:32:53.870 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:53.870 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:32:53.870 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045782+0000", 2024-09-14T06:32:53.870 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:53.870 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:53.870 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:53.870 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:32:53.870 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045782+0000", 2024-09-14T06:32:53.870 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:53.870 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:53.871 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:53.871 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:32:53.871 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045783+0000", 2024-09-14T06:32:53.871 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.2499999999999999e-07 2024-09-14T06:32:53.871 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:53.871 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:53.871 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:32:53.871 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045784+0000", 2024-09-14T06:32:53.871 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.0109999999999999e-06 2024-09-14T06:32:53.871 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:53.871 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:53.871 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:32:53.871 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045786+0000", 2024-09-14T06:32:53.871 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.8289999999999999e-06 2024-09-14T06:32:53.872 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:53.872 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:53.872 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:32:53.872 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045793+0000", 2024-09-14T06:32:53.872 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.579e-06 2024-09-14T06:32:53.872 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:53.872 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:53.872 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:32:53.872 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.078338+0000", 2024-09-14T06:32:53.872 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032545389000000001 2024-09-14T06:32:53.872 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:53.872 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:53.872 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:32:53.872 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.078348+0000", 2024-09-14T06:32:53.873 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.0076e-05 2024-09-14T06:32:53.873 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:53.873 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:32:53.873 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:53.873 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:53.873 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:53.873 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:98522 3.10 3.1bf56f90 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T06:32:53.873 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:30:07.045796+0000", 2024-09-14T06:32:53.873 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 166.75421190899999, 2024-09-14T06:32:53.873 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032577846000000001, 2024-09-14T06:32:53.873 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:32:53.873 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:32:53.873 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "reached pg", 2024-09-14T06:32:53.873 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:32:53.873 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:32:53.873 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:53.874 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 98522 2024-09-14T06:32:53.874 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:53.874 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:32:53.874 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:53.874 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:32:53.874 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045796+0000", 2024-09-14T06:32:53.874 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:53.874 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:53.874 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:53.874 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:32:53.874 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045796+0000", 2024-09-14T06:32:53.874 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:53.874 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:53.874 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:53.875 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:32:53.875 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045796+0000", 2024-09-14T06:32:53.875 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.2100000000000001e-07 2024-09-14T06:32:53.875 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:53.875 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:53.875 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:32:53.875 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045797+0000", 2024-09-14T06:32:53.875 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.6899999999999996e-07 2024-09-14T06:32:53.875 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:53.875 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:53.875 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:32:53.875 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045800+0000", 2024-09-14T06:32:53.875 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.9979999999999999e-06 2024-09-14T06:32:53.875 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:53.876 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:53.876 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:32:53.876 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045803+0000", 2024-09-14T06:32:53.876 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.2850000000000001e-06 2024-09-14T06:32:53.876 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:53.876 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:53.876 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:32:53.876 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.078366+0000", 2024-09-14T06:32:53.876 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032562932000000003 2024-09-14T06:32:53.876 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:53.876 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:53.876 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:32:53.876 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.078374+0000", 2024-09-14T06:32:53.876 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.4409999999999993e-06 2024-09-14T06:32:53.876 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:53.877 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:32:53.877 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:53.877 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:53.877 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:53.877 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:98523 3.10 3.1bf56f90 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T06:32:53.877 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:30:07.045806+0000", 2024-09-14T06:32:53.877 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 166.75420236599999, 2024-09-14T06:32:53.877 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032611167000000003, 2024-09-14T06:32:53.877 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:32:53.877 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:32:53.877 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "reached pg", 2024-09-14T06:32:53.877 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:32:53.877 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:32:53.877 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:53.877 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 98523 2024-09-14T06:32:53.878 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:53.878 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:32:53.878 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:53.878 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:32:53.878 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045806+0000", 2024-09-14T06:32:53.878 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:53.878 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:53.878 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:53.878 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:32:53.878 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045806+0000", 2024-09-14T06:32:53.878 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:53.878 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:53.878 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:53.878 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:32:53.878 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045806+0000", 2024-09-14T06:32:53.879 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.2000000000000001e-07 2024-09-14T06:32:53.879 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:53.879 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:53.879 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:32:53.879 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045807+0000", 2024-09-14T06:32:53.879 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.4e-07 2024-09-14T06:32:53.879 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:53.879 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:53.879 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:32:53.879 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045810+0000", 2024-09-14T06:32:53.879 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.9229999999999998e-06 2024-09-14T06:32:53.879 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:53.879 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:53.880 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:32:53.880 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045812+0000", 2024-09-14T06:32:53.880 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.2220000000000001e-06 2024-09-14T06:32:53.880 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:53.880 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:53.880 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:32:53.880 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.078407+0000", 2024-09-14T06:32:53.880 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032594513999999998 2024-09-14T06:32:53.880 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:53.880 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:53.880 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:32:53.880 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.078417+0000", 2024-09-14T06:32:53.880 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.0348e-05 2024-09-14T06:32:53.880 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:53.880 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:32:53.881 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:53.881 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:53.881 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:53.881 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:98524 3.10 3.1bf56f90 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T06:32:53.881 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:30:07.045815+0000", 2024-09-14T06:32:53.881 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 166.75419299800001, 2024-09-14T06:32:53.881 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032643208999999999, 2024-09-14T06:32:53.881 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:32:53.881 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:32:53.881 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "reached pg", 2024-09-14T06:32:53.881 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:32:53.881 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:32:53.881 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:53.881 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 98524 2024-09-14T06:32:53.881 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:53.882 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:32:53.882 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:53.882 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:32:53.882 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045815+0000", 2024-09-14T06:32:53.882 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:53.882 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:53.882 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:53.882 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:32:53.882 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045815+0000", 2024-09-14T06:32:53.882 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:53.882 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:53.882 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:53.882 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:32:53.882 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045815+0000", 2024-09-14T06:32:53.882 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.11e-07 2024-09-14T06:32:53.883 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:53.883 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:53.883 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:32:53.883 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045816+0000", 2024-09-14T06:32:53.883 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.9099999999999991e-07 2024-09-14T06:32:53.883 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:53.883 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:53.883 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:32:53.883 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045819+0000", 2024-09-14T06:32:53.883 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.9399999999999998e-06 2024-09-14T06:32:53.883 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:53.883 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:53.883 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:32:53.883 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045822+0000", 2024-09-14T06:32:53.883 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.3470000000000001e-06 2024-09-14T06:32:53.884 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:53.884 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:53.884 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:32:53.884 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.078449+0000", 2024-09-14T06:32:53.884 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032627529000000002 2024-09-14T06:32:53.884 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:53.884 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:53.884 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:32:53.884 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.078458+0000", 2024-09-14T06:32:53.884 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.1910000000000006e-06 2024-09-14T06:32:53.884 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:53.884 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:32:53.884 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:53.884 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:53.884 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:32:53.885 INFO:teuthology.orchestra.run.smithi138.stdout:} 2024-09-14T06:32:53.886 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-14T06:32:53.886 INFO:tasks.ceph.osd.3.smithi094.stderr:2024-09-14T06:32:53.884+0000 7feeeafe8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33655) UID: 0 2024-09-14T06:32:53.986 INFO:tasks.ceph.osd.5.smithi138.stderr:2024-09-14T06:32:53.985+0000 7fe13746f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33618) UID: 0 2024-09-14T06:32:53.988 INFO:teuthology.orchestra.run.smithi138.stdout:{ 2024-09-14T06:32:53.988 INFO:teuthology.orchestra.run.smithi138.stdout: "ops": [], 2024-09-14T06:32:53.988 INFO:teuthology.orchestra.run.smithi138.stdout: "num_ops": 0 2024-09-14T06:32:53.989 INFO:teuthology.orchestra.run.smithi138.stdout:} 2024-09-14T06:32:53.997 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-14T06:32:54.088 INFO:tasks.ceph.osd.3.smithi094.stderr:2024-09-14T06:32:54.087+0000 7feeeafe8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33655) UID: 0 2024-09-14T06:32:54.139 INFO:teuthology.orchestra.run.smithi138.stdout:{ 2024-09-14T06:32:54.139 INFO:teuthology.orchestra.run.smithi138.stdout: "ops": [], 2024-09-14T06:32:54.139 INFO:teuthology.orchestra.run.smithi138.stdout: "complaint_time": 30, 2024-09-14T06:32:54.140 INFO:teuthology.orchestra.run.smithi138.stdout: "num_blocked_ops": 0 2024-09-14T06:32:54.140 INFO:teuthology.orchestra.run.smithi138.stdout:} 2024-09-14T06:32:54.148 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-14T06:32:54.193 INFO:tasks.ceph.osd.7.smithi138.stderr:2024-09-14T06:32:54.192+0000 7f3c6a21d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33623) UID: 0 2024-09-14T06:32:54.252 INFO:teuthology.orchestra.run.smithi138.stdout:{ 2024-09-14T06:32:54.252 INFO:teuthology.orchestra.run.smithi138.stdout: "size": 20, 2024-09-14T06:32:54.252 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 600, 2024-09-14T06:32:54.252 INFO:teuthology.orchestra.run.smithi138.stdout: "ops": [ 2024-09-14T06:32:54.252 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:54.252 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:53244 3.f 3:f71f51a7:::rbd_data.112dca2f6ae1.0000000000000015:17 [read 1863680~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:32:54.252 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:44.369399+0000", 2024-09-14T06:32:54.252 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 189.880782853, 2024-09-14T06:32:54.252 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047291189999999997, 2024-09-14T06:32:54.252 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:32:54.252 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:32:54.252 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:32:54.252 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:32:54.253 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:32:54.253 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:54.253 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 53244 2024-09-14T06:32:54.253 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:54.253 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:32:54.253 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:54.253 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:32:54.253 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369399+0000", 2024-09-14T06:32:54.253 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:54.253 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:54.253 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:54.253 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:32:54.253 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369399+0000", 2024-09-14T06:32:54.253 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:54.254 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:54.254 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:54.254 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:32:54.254 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369400+0000", 2024-09-14T06:32:54.254 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.0699999999999999e-07 2024-09-14T06:32:54.254 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:54.254 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:54.254 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:32:54.254 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369401+0000", 2024-09-14T06:32:54.254 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.4300000000000001e-06 2024-09-14T06:32:54.254 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:54.254 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:54.254 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:32:54.255 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369404+0000", 2024-09-14T06:32:54.255 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.061e-06 2024-09-14T06:32:54.255 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:54.255 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:54.255 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:32:54.255 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369407+0000", 2024-09-14T06:32:54.255 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.6139999999999999e-06 2024-09-14T06:32:54.255 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:54.255 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:54.255 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:32:54.255 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.416517+0000", 2024-09-14T06:32:54.255 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047110338000000002 2024-09-14T06:32:54.255 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:54.256 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:54.256 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:32:54.256 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.416616+0000", 2024-09-14T06:32:54.256 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.8651000000000001e-05 2024-09-14T06:32:54.256 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:54.256 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:54.256 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:32:54.256 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.416691+0000", 2024-09-14T06:32:54.256 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.4888999999999994e-05 2024-09-14T06:32:54.256 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:54.256 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:32:54.256 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:54.256 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:54.256 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:54.257 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:53245 3.f 3:f71f51a7:::rbd_data.112dca2f6ae1.0000000000000015:17 [read 1888256~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:32:54.257 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:44.369410+0000", 2024-09-14T06:32:54.257 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 189.880772565, 2024-09-14T06:32:54.257 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047399275999999997, 2024-09-14T06:32:54.257 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:32:54.257 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:32:54.257 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:32:54.257 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:32:54.257 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:32:54.257 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:54.257 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 53245 2024-09-14T06:32:54.257 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:54.257 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:32:54.258 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:54.258 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:32:54.258 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369410+0000", 2024-09-14T06:32:54.258 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:54.258 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:54.258 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:54.258 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:32:54.258 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369410+0000", 2024-09-14T06:32:54.258 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:54.258 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:54.258 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:54.258 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:32:54.258 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369410+0000", 2024-09-14T06:32:54.259 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.1899999999999999e-07 2024-09-14T06:32:54.259 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:54.259 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:54.259 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:32:54.259 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369411+0000", 2024-09-14T06:32:54.259 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.0300000000000001e-06 2024-09-14T06:32:54.259 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:54.259 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:54.259 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:32:54.259 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369414+0000", 2024-09-14T06:32:54.259 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.1099999999999999e-06 2024-09-14T06:32:54.259 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:54.259 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:54.260 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:32:54.260 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369417+0000", 2024-09-14T06:32:54.260 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.6189999999999998e-06 2024-09-14T06:32:54.260 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:54.260 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:54.260 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:32:54.260 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.416718+0000", 2024-09-14T06:32:54.260 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047301322999999999 2024-09-14T06:32:54.260 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:54.260 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:54.260 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:32:54.260 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.416759+0000", 2024-09-14T06:32:54.260 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.1009e-05 2024-09-14T06:32:54.261 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:54.261 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:54.261 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:32:54.261 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.416809+0000", 2024-09-14T06:32:54.261 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.9966000000000002e-05 2024-09-14T06:32:54.261 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:54.261 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:32:54.261 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:54.261 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:54.261 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:54.261 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:53246 3.f 3:f71f51a7:::rbd_data.112dca2f6ae1.0000000000000015:17 [read 1912832~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:32:54.261 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:44.369420+0000", 2024-09-14T06:32:54.262 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 189.880762621, 2024-09-14T06:32:54.262 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047529463000000001, 2024-09-14T06:32:54.262 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:32:54.262 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:32:54.262 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:32:54.262 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:32:54.262 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:32:54.262 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:54.262 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 53246 2024-09-14T06:32:54.262 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:54.262 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:32:54.262 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:54.262 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:32:54.263 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369420+0000", 2024-09-14T06:32:54.263 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:54.263 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:54.263 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:54.263 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:32:54.263 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369420+0000", 2024-09-14T06:32:54.263 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:54.263 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:54.263 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:54.263 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:32:54.263 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369420+0000", 2024-09-14T06:32:54.263 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.1299999999999999e-07 2024-09-14T06:32:54.263 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:54.264 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:54.264 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:32:54.264 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369425+0000", 2024-09-14T06:32:54.264 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.0130000000000001e-06 2024-09-14T06:32:54.264 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:54.264 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:54.264 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:32:54.264 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369428+0000", 2024-09-14T06:32:54.264 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.349e-06 2024-09-14T06:32:54.264 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:54.264 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:54.264 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:32:54.264 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369431+0000", 2024-09-14T06:32:54.264 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.7389999999999999e-06 2024-09-14T06:32:54.264 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:54.265 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:54.265 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:32:54.265 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.416834+0000", 2024-09-14T06:32:54.265 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047402669000000001 2024-09-14T06:32:54.265 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:54.265 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:54.265 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:32:54.265 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.416882+0000", 2024-09-14T06:32:54.265 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.8068999999999997e-05 2024-09-14T06:32:54.265 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:54.265 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:54.265 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:32:54.265 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.416949+0000", 2024-09-14T06:32:54.265 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.7410999999999995e-05 2024-09-14T06:32:54.266 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:54.266 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:32:54.266 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:54.266 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:54.266 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:54.266 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:53247 3.f 3:f71f51a7:::rbd_data.112dca2f6ae1.0000000000000015:17 [read 1937408~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:32:54.266 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:44.369434+0000", 2024-09-14T06:32:54.266 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 189.88074785699999, 2024-09-14T06:32:54.266 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047652264999999999, 2024-09-14T06:32:54.266 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:32:54.266 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:32:54.266 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:32:54.266 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:32:54.267 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:32:54.267 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:54.267 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 53247 2024-09-14T06:32:54.267 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:54.267 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:32:54.267 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:54.267 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:32:54.267 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369434+0000", 2024-09-14T06:32:54.267 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:54.267 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:54.267 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:54.267 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:32:54.267 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369434+0000", 2024-09-14T06:32:54.267 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:54.268 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:54.268 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:54.268 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:32:54.268 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369435+0000", 2024-09-14T06:32:54.268 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.2100000000000001e-07 2024-09-14T06:32:54.268 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:54.268 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:54.268 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:32:54.268 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369436+0000", 2024-09-14T06:32:54.268 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.122e-06 2024-09-14T06:32:54.268 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:54.268 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:54.268 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:32:54.269 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369439+0000", 2024-09-14T06:32:54.269 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.0400000000000001e-06 2024-09-14T06:32:54.269 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:54.269 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:54.269 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:32:54.269 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369441+0000", 2024-09-14T06:32:54.269 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.728e-06 2024-09-14T06:32:54.269 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:54.269 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:54.269 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:32:54.269 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.416977+0000", 2024-09-14T06:32:54.269 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047535344 2024-09-14T06:32:54.269 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:54.269 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:54.270 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:32:54.270 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.417029+0000", 2024-09-14T06:32:54.270 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.2037999999999999e-05 2024-09-14T06:32:54.270 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:54.270 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:54.270 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:32:54.270 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.417087+0000", 2024-09-14T06:32:54.270 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.7772000000000002e-05 2024-09-14T06:32:54.270 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:54.270 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:32:54.270 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:54.270 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:54.270 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:54.271 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:53248 3.f 3:f71f51a7:::rbd_data.112dca2f6ae1.0000000000000015:17 [read 1961984~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:32:54.271 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:44.369445+0000", 2024-09-14T06:32:54.271 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 189.880737305, 2024-09-14T06:32:54.271 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047798489, 2024-09-14T06:32:54.271 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:32:54.271 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:32:54.271 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:32:54.271 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:32:54.271 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:32:54.271 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:54.271 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 53248 2024-09-14T06:32:54.271 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:54.271 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:32:54.271 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:54.271 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:32:54.272 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369445+0000", 2024-09-14T06:32:54.272 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:54.272 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:54.272 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:54.272 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:32:54.272 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369445+0000", 2024-09-14T06:32:54.272 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:54.272 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:54.272 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:54.272 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:32:54.272 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369445+0000", 2024-09-14T06:32:54.272 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.0699999999999999e-07 2024-09-14T06:32:54.272 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:54.272 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:54.273 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:32:54.273 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369446+0000", 2024-09-14T06:32:54.273 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.1000000000000001e-06 2024-09-14T06:32:54.273 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:54.273 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:54.273 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:32:54.273 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369449+0000", 2024-09-14T06:32:54.273 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.1039999999999998e-06 2024-09-14T06:32:54.273 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:54.273 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:54.273 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:32:54.273 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369452+0000", 2024-09-14T06:32:54.273 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.6199999999999999e-06 2024-09-14T06:32:54.273 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:54.273 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:54.274 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:32:54.274 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.417114+0000", 2024-09-14T06:32:54.274 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047661901999999999 2024-09-14T06:32:54.274 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:54.274 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:54.274 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:32:54.274 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.417181+0000", 2024-09-14T06:32:54.274 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.7385999999999995e-05 2024-09-14T06:32:54.274 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:54.274 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:54.274 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:32:54.274 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.417243+0000", 2024-09-14T06:32:54.274 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.2169999999999996e-05 2024-09-14T06:32:54.274 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:54.274 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:32:54.275 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:54.275 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:54.275 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:54.275 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:53249 3.f 3:f71f51a7:::rbd_data.112dca2f6ae1.0000000000000015:17 [read 1986560~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:32:54.275 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:44.369455+0000", 2024-09-14T06:32:54.275 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 189.88072699400001, 2024-09-14T06:32:54.275 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047890493999999999, 2024-09-14T06:32:54.275 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:32:54.275 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:32:54.275 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:32:54.275 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:32:54.275 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:32:54.275 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:54.275 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 53249 2024-09-14T06:32:54.275 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:54.276 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:32:54.276 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:54.276 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:32:54.276 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369455+0000", 2024-09-14T06:32:54.276 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:54.276 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:54.276 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:54.276 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:32:54.276 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369455+0000", 2024-09-14T06:32:54.276 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:54.276 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:54.276 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:54.276 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:32:54.276 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369455+0000", 2024-09-14T06:32:54.277 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.0800000000000001e-07 2024-09-14T06:32:54.277 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:54.277 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:54.277 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:32:54.277 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369456+0000", 2024-09-14T06:32:54.277 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.0729999999999999e-06 2024-09-14T06:32:54.277 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:54.277 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:54.277 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:32:54.277 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369460+0000", 2024-09-14T06:32:54.277 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.027e-06 2024-09-14T06:32:54.277 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:54.277 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:54.277 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:32:54.277 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369462+0000", 2024-09-14T06:32:54.278 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.5320000000000001e-06 2024-09-14T06:32:54.278 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:54.278 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:54.278 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:32:54.278 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.417257+0000", 2024-09-14T06:32:54.278 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.04779489 2024-09-14T06:32:54.278 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:54.278 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:54.278 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:32:54.278 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.417296+0000", 2024-09-14T06:32:54.278 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.8658000000000002e-05 2024-09-14T06:32:54.278 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:54.278 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:54.278 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:32:54.278 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.417346+0000", 2024-09-14T06:32:54.279 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.0105999999999998e-05 2024-09-14T06:32:54.279 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:54.279 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:32:54.279 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:54.279 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:54.279 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:54.279 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:53250 3.f 3:f71f51a7:::rbd_data.112dca2f6ae1.0000000000000015:17 [read 2011136~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:32:54.279 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:44.369465+0000", 2024-09-14T06:32:54.279 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 189.88071702400001, 2024-09-14T06:32:54.279 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.048032230000000002, 2024-09-14T06:32:54.279 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:32:54.279 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:32:54.279 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:32:54.279 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:32:54.279 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:32:54.280 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:54.280 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 53250 2024-09-14T06:32:54.280 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:54.280 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:32:54.280 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:54.280 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:32:54.280 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369465+0000", 2024-09-14T06:32:54.280 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:54.280 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:54.280 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:54.280 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:32:54.281 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369465+0000", 2024-09-14T06:32:54.281 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:54.281 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:54.281 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:54.281 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:32:54.281 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369465+0000", 2024-09-14T06:32:54.281 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.2100000000000001e-07 2024-09-14T06:32:54.281 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:54.281 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:54.281 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:32:54.281 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369466+0000", 2024-09-14T06:32:54.281 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.0640000000000001e-06 2024-09-14T06:32:54.281 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:54.281 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:54.282 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:32:54.282 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369469+0000", 2024-09-14T06:32:54.282 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.0170000000000001e-06 2024-09-14T06:32:54.282 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:54.282 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:54.282 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:32:54.282 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369472+0000", 2024-09-14T06:32:54.282 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.6189999999999998e-06 2024-09-14T06:32:54.282 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:54.282 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:54.282 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:32:54.282 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.417383+0000", 2024-09-14T06:32:54.282 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047911092000000002 2024-09-14T06:32:54.283 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:54.283 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:54.283 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:32:54.283 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.417430+0000", 2024-09-14T06:32:54.283 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.6898999999999999e-05 2024-09-14T06:32:54.283 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:54.283 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:54.283 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:32:54.283 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.417497+0000", 2024-09-14T06:32:54.283 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.7317999999999995e-05 2024-09-14T06:32:54.283 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:54.283 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:32:54.283 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:54.284 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:54.284 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:54.284 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:53251 3.f 3:f71f51a7:::rbd_data.112dca2f6ae1.0000000000000015:17 [read 2035712~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:32:54.284 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:44.369475+0000", 2024-09-14T06:32:54.284 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 189.88070702799999, 2024-09-14T06:32:54.284 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.048154542000000002, 2024-09-14T06:32:54.284 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:32:54.284 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:32:54.284 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:32:54.284 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:32:54.284 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:32:54.284 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:54.284 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 53251 2024-09-14T06:32:54.285 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:54.285 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:32:54.285 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:54.285 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:32:54.285 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369475+0000", 2024-09-14T06:32:54.285 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:54.285 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:54.285 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:54.285 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:32:54.285 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369475+0000", 2024-09-14T06:32:54.285 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:54.285 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:54.285 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:54.285 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:32:54.286 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369475+0000", 2024-09-14T06:32:54.286 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.0100000000000001e-07 2024-09-14T06:32:54.286 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:54.286 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:54.286 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:32:54.286 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369476+0000", 2024-09-14T06:32:54.286 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.0580000000000001e-06 2024-09-14T06:32:54.286 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:54.286 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:54.286 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:32:54.286 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369479+0000", 2024-09-14T06:32:54.286 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.9950000000000001e-06 2024-09-14T06:32:54.286 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:54.287 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:54.287 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:32:54.287 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369482+0000", 2024-09-14T06:32:54.287 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.6340000000000001e-06 2024-09-14T06:32:54.287 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:54.287 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:54.287 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:32:54.287 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.417530+0000", 2024-09-14T06:32:54.287 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.048047535000000002 2024-09-14T06:32:54.287 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:54.287 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:54.287 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:32:54.287 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.417574+0000", 2024-09-14T06:32:54.287 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.4255999999999998e-05 2024-09-14T06:32:54.288 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:54.288 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:54.288 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:32:54.288 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.417630+0000", 2024-09-14T06:32:54.288 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.5862999999999997e-05 2024-09-14T06:32:54.288 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:54.288 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:32:54.288 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:54.288 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:54.288 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:54.288 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:53252 3.f 3:f71f51a7:::rbd_data.112dca2f6ae1.0000000000000015:17 [read 2060288~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:32:54.288 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:44.369485+0000", 2024-09-14T06:32:54.288 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 189.88069718, 2024-09-14T06:32:54.289 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.048254116, 2024-09-14T06:32:54.289 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:32:54.289 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:32:54.289 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:32:54.289 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:32:54.289 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:32:54.289 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:54.289 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 53252 2024-09-14T06:32:54.289 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:54.289 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:32:54.289 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:54.289 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:32:54.289 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369485+0000", 2024-09-14T06:32:54.289 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:54.290 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:54.290 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:54.290 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:32:54.290 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369485+0000", 2024-09-14T06:32:54.290 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:54.290 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:54.290 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:54.290 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:32:54.290 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369485+0000", 2024-09-14T06:32:54.290 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.1e-07 2024-09-14T06:32:54.290 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:54.290 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:54.290 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:32:54.291 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369487+0000", 2024-09-14T06:32:54.291 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.42e-06 2024-09-14T06:32:54.291 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:54.291 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:54.291 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:32:54.291 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369490+0000", 2024-09-14T06:32:54.291 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.9639999999999999e-06 2024-09-14T06:32:54.291 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:54.291 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:54.291 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:32:54.291 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369492+0000", 2024-09-14T06:32:54.291 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.5679999999999998e-06 2024-09-14T06:32:54.291 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:54.292 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:54.292 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:32:54.292 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.417645+0000", 2024-09-14T06:32:54.292 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.048152753 2024-09-14T06:32:54.292 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:54.292 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:54.292 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:32:54.292 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.417687+0000", 2024-09-14T06:32:54.292 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.1900999999999997e-05 2024-09-14T06:32:54.292 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:54.292 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:54.292 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:32:54.292 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.417739+0000", 2024-09-14T06:32:54.293 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.2299999999999997e-05 2024-09-14T06:32:54.293 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:54.293 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:32:54.293 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:54.293 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:54.293 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:54.293 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:53253 3.f 3:f71f51a7:::rbd_data.112dca2f6ae1.0000000000000015:17 [read 2084864~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:32:54.293 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:44.369495+0000", 2024-09-14T06:32:54.293 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 189.880687056, 2024-09-14T06:32:54.293 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.048366785000000002, 2024-09-14T06:32:54.293 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:32:54.293 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:32:54.293 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:32:54.294 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:32:54.294 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:32:54.294 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:54.294 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 53253 2024-09-14T06:32:54.294 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:54.294 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:32:54.294 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:54.294 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:32:54.294 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369495+0000", 2024-09-14T06:32:54.294 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:54.294 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:54.294 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:54.294 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:32:54.295 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369495+0000", 2024-09-14T06:32:54.295 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:54.295 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:54.295 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:54.295 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:32:54.295 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370388+0000", 2024-09-14T06:32:54.295 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00089287999999999998 2024-09-14T06:32:54.295 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:54.295 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:54.295 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:32:54.295 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370392+0000", 2024-09-14T06:32:54.295 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.2130000000000002e-06 2024-09-14T06:32:54.295 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:54.295 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:54.296 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:32:54.296 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370401+0000", 2024-09-14T06:32:54.296 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.7649999999999999e-06 2024-09-14T06:32:54.296 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:54.296 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:54.296 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:32:54.296 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370406+0000", 2024-09-14T06:32:54.296 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.0159999999999999e-06 2024-09-14T06:32:54.296 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:54.296 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:54.296 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:32:54.296 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.417764+0000", 2024-09-14T06:32:54.296 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047358348000000001 2024-09-14T06:32:54.297 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:54.297 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:54.297 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:32:54.297 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.417806+0000", 2024-09-14T06:32:54.297 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.1149999999999997e-05 2024-09-14T06:32:54.297 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:54.297 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:54.297 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:32:54.297 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.417862+0000", 2024-09-14T06:32:54.297 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.6412999999999997e-05 2024-09-14T06:32:54.297 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:54.297 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:32:54.297 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:54.298 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:54.298 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:54.298 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:53340 3.f 3:f71f51a7:::rbd_data.112dca2f6ae1.0000000000000015:17 [read 1056768~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:32:54.298 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:44.370421+0000", 2024-09-14T06:32:54.298 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 189.87976117599999, 2024-09-14T06:32:54.298 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047566313999999998, 2024-09-14T06:32:54.298 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:32:54.298 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:32:54.298 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:32:54.298 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:32:54.298 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:32:54.298 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:54.298 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 53340 2024-09-14T06:32:54.298 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:54.298 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:32:54.299 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:54.299 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:32:54.299 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370421+0000", 2024-09-14T06:32:54.299 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:54.299 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:54.299 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:54.299 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:32:54.299 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370421+0000", 2024-09-14T06:32:54.299 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:54.299 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:54.299 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:54.299 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:32:54.299 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370422+0000", 2024-09-14T06:32:54.299 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.3799999999999996e-07 2024-09-14T06:32:54.300 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:54.300 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:54.300 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:32:54.300 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370424+0000", 2024-09-14T06:32:54.300 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.6919999999999998e-06 2024-09-14T06:32:54.300 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:54.300 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:54.300 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:32:54.300 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370432+0000", 2024-09-14T06:32:54.300 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.2439999999999996e-06 2024-09-14T06:32:54.300 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:54.300 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:54.300 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:32:54.300 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370437+0000", 2024-09-14T06:32:54.301 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.5389999999999998e-06 2024-09-14T06:32:54.301 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:54.301 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:54.301 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:32:54.301 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.417888+0000", 2024-09-14T06:32:54.301 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047450432000000001 2024-09-14T06:32:54.301 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:54.301 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:54.301 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:32:54.301 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.417935+0000", 2024-09-14T06:32:54.301 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.7454999999999999e-05 2024-09-14T06:32:54.301 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:54.301 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:54.302 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:32:54.302 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.417987+0000", 2024-09-14T06:32:54.302 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.2213999999999998e-05 2024-09-14T06:32:54.302 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:54.302 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:32:54.302 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:54.302 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:54.302 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:54.302 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:53341 3.f 3:f71f51a7:::rbd_data.112dca2f6ae1.0000000000000015:17 [read 1081344~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:32:54.302 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:44.370446+0000", 2024-09-14T06:32:54.302 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 189.87973590300001, 2024-09-14T06:32:54.302 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047665723, 2024-09-14T06:32:54.302 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:32:54.302 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:32:54.303 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:32:54.303 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:32:54.303 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:32:54.303 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:54.303 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 53341 2024-09-14T06:32:54.303 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:54.303 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:32:54.303 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:54.303 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:32:54.303 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370446+0000", 2024-09-14T06:32:54.303 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:54.303 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:54.303 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:54.303 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:32:54.303 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370446+0000", 2024-09-14T06:32:54.304 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:54.304 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:54.304 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:54.304 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:32:54.304 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370447+0000", 2024-09-14T06:32:54.304 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.6100000000000002e-07 2024-09-14T06:32:54.304 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:54.304 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:54.304 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:32:54.304 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370448+0000", 2024-09-14T06:32:54.304 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.7370000000000001e-06 2024-09-14T06:32:54.304 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:54.304 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:54.305 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:32:54.305 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370462+0000", 2024-09-14T06:32:54.305 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.3227000000000001e-05 2024-09-14T06:32:54.305 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:54.305 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:54.305 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:32:54.305 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370467+0000", 2024-09-14T06:32:54.305 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.6540000000000004e-06 2024-09-14T06:32:54.305 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:54.305 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:54.305 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:32:54.305 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.418013+0000", 2024-09-14T06:32:54.305 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047545325999999999 2024-09-14T06:32:54.305 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:54.306 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:54.306 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:32:54.306 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.418054+0000", 2024-09-14T06:32:54.306 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.1696999999999997e-05 2024-09-14T06:32:54.306 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:54.306 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:54.306 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:32:54.306 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.418112+0000", 2024-09-14T06:32:54.306 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.7720999999999998e-05 2024-09-14T06:32:54.306 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:54.306 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:32:54.306 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:54.306 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:54.306 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:54.306 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:53342 3.f 3:f71f51a7:::rbd_data.112dca2f6ae1.0000000000000015:17 [read 1105920~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:32:54.307 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:44.370475+0000", 2024-09-14T06:32:54.307 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 189.87970684199999, 2024-09-14T06:32:54.307 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047748848000000003, 2024-09-14T06:32:54.307 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:32:54.307 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:32:54.307 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:32:54.307 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:32:54.307 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:32:54.307 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:54.307 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 53342 2024-09-14T06:32:54.307 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:54.307 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:32:54.307 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:54.307 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:32:54.307 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370475+0000", 2024-09-14T06:32:54.308 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:54.308 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:54.308 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:54.308 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:32:54.308 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370475+0000", 2024-09-14T06:32:54.308 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:54.308 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:54.308 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:54.308 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:32:54.308 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370476+0000", 2024-09-14T06:32:54.308 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.4700000000000002e-07 2024-09-14T06:32:54.308 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:54.308 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:54.308 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:32:54.309 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370478+0000", 2024-09-14T06:32:54.309 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.5890000000000001e-06 2024-09-14T06:32:54.309 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:54.309 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:54.309 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:32:54.309 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370486+0000", 2024-09-14T06:32:54.309 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.3470000000000001e-06 2024-09-14T06:32:54.309 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:54.309 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:54.309 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:32:54.309 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370491+0000", 2024-09-14T06:32:54.309 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.5720000000000002e-06 2024-09-14T06:32:54.309 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:54.310 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:54.310 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:32:54.310 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.418135+0000", 2024-09-14T06:32:54.310 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047644094999999997 2024-09-14T06:32:54.310 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:54.310 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:54.310 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:32:54.310 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.418174+0000", 2024-09-14T06:32:54.310 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.8204000000000002e-05 2024-09-14T06:32:54.310 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:54.310 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:54.310 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:32:54.310 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.418224+0000", 2024-09-14T06:32:54.310 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.0593999999999997e-05 2024-09-14T06:32:54.311 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:54.311 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:32:54.311 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:54.311 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:54.311 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:54.311 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:53343 3.f 3:f71f51a7:::rbd_data.112dca2f6ae1.0000000000000015:17 [read 1130496~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:32:54.311 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:44.370498+0000", 2024-09-14T06:32:54.311 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 189.87968396599999, 2024-09-14T06:32:54.311 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047841574999999997, 2024-09-14T06:32:54.311 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:32:54.311 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:32:54.311 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:32:54.311 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:32:54.312 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:32:54.312 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:54.312 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 53343 2024-09-14T06:32:54.312 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:54.312 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:32:54.312 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:54.312 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:32:54.312 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370498+0000", 2024-09-14T06:32:54.312 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:54.312 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:54.312 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:54.312 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:32:54.312 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370498+0000", 2024-09-14T06:32:54.312 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:54.313 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:54.313 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:54.313 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:32:54.313 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370499+0000", 2024-09-14T06:32:54.313 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.9999999999999998e-07 2024-09-14T06:32:54.313 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:54.313 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:54.313 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:32:54.313 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370501+0000", 2024-09-14T06:32:54.313 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.0310000000000001e-06 2024-09-14T06:32:54.313 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:54.313 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:54.313 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:32:54.313 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370507+0000", 2024-09-14T06:32:54.314 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.4169999999999997e-06 2024-09-14T06:32:54.314 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:54.314 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:54.314 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:32:54.314 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370516+0000", 2024-09-14T06:32:54.314 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.7190000000000007e-06 2024-09-14T06:32:54.314 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:54.314 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:54.314 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:32:54.314 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.418247+0000", 2024-09-14T06:32:54.314 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047731190999999999 2024-09-14T06:32:54.314 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:54.314 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:54.314 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:32:54.315 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.418286+0000", 2024-09-14T06:32:54.315 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.8988e-05 2024-09-14T06:32:54.315 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:54.315 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:54.315 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:32:54.315 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.418340+0000", 2024-09-14T06:32:54.315 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.3829000000000002e-05 2024-09-14T06:32:54.315 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:54.315 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:32:54.315 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:54.315 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:54.315 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:54.315 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:53344 3.f 3:f71f51a7:::rbd_data.112dca2f6ae1.0000000000000015:17 [read 1155072~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:32:54.315 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:44.370524+0000", 2024-09-14T06:32:54.316 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 189.87965819600001, 2024-09-14T06:32:54.316 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047972262000000002, 2024-09-14T06:32:54.316 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:32:54.316 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:32:54.316 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:32:54.316 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:32:54.316 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:32:54.316 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:54.316 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 53344 2024-09-14T06:32:54.316 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:54.316 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:32:54.316 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:54.316 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:32:54.317 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370524+0000", 2024-09-14T06:32:54.317 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:54.317 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:54.317 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:54.317 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:32:54.317 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370524+0000", 2024-09-14T06:32:54.317 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:54.317 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:54.317 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:54.317 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:32:54.317 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370618+0000", 2024-09-14T06:32:54.318 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.4425999999999993e-05 2024-09-14T06:32:54.318 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:54.318 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:54.318 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:32:54.318 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370622+0000", 2024-09-14T06:32:54.318 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.3519999999999998e-06 2024-09-14T06:32:54.318 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:54.318 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:54.318 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:32:54.318 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370631+0000", 2024-09-14T06:32:54.318 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.9299999999999992e-06 2024-09-14T06:32:54.318 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:54.319 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:54.319 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:32:54.319 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370637+0000", 2024-09-14T06:32:54.319 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.3269999999999997e-06 2024-09-14T06:32:54.319 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:54.319 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:54.319 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:32:54.319 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.418372+0000", 2024-09-14T06:32:54.319 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047734764999999998 2024-09-14T06:32:54.319 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:54.319 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:54.319 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:32:54.319 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.418431+0000", 2024-09-14T06:32:54.319 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.8696999999999997e-05 2024-09-14T06:32:54.320 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:54.320 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:54.320 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:32:54.320 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.418496+0000", 2024-09-14T06:32:54.320 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.5765000000000005e-05 2024-09-14T06:32:54.320 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:54.320 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:32:54.320 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:54.320 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:54.320 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:54.320 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:53345 3.f 3:f71f51a7:::rbd_data.112dca2f6ae1.0000000000000015:17 [read 1179648~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:32:54.320 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:44.370647+0000", 2024-09-14T06:32:54.321 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 189.879534729, 2024-09-14T06:32:54.321 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047953775999999997, 2024-09-14T06:32:54.321 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:32:54.321 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:32:54.321 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:32:54.321 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:32:54.321 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:32:54.321 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:54.321 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 53345 2024-09-14T06:32:54.321 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:54.321 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:32:54.321 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:54.321 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:32:54.322 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370647+0000", 2024-09-14T06:32:54.322 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:54.322 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:54.322 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:54.322 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:32:54.322 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370647+0000", 2024-09-14T06:32:54.322 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:54.322 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:54.322 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:54.322 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:32:54.322 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.372346+0000", 2024-09-14T06:32:54.322 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.001698284 2024-09-14T06:32:54.322 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:54.323 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:54.323 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:32:54.323 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.372351+0000", 2024-09-14T06:32:54.323 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.8890000000000001e-06 2024-09-14T06:32:54.323 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:54.323 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:54.323 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:32:54.323 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.372362+0000", 2024-09-14T06:32:54.323 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.0941000000000001e-05 2024-09-14T06:32:54.323 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:54.323 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:54.323 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:32:54.324 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.372369+0000", 2024-09-14T06:32:54.324 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.362e-06 2024-09-14T06:32:54.324 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:54.324 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:54.324 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:32:54.324 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.418510+0000", 2024-09-14T06:32:54.324 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046140914999999998 2024-09-14T06:32:54.324 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:54.324 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:54.324 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:32:54.324 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.418549+0000", 2024-09-14T06:32:54.324 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.8717999999999998e-05 2024-09-14T06:32:54.325 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:54.325 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:54.325 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:32:54.325 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.418601+0000", 2024-09-14T06:32:54.325 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.2667000000000002e-05 2024-09-14T06:32:54.325 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:54.325 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:32:54.325 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:54.325 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:54.325 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:54.325 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:53353 3.f 3:f71f51a7:::rbd_data.112dca2f6ae1.0000000000000015:17 [read 1376256~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:32:54.325 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:44.372545+0000", 2024-09-14T06:32:54.325 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 189.87763765400001, 2024-09-14T06:32:54.326 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047364232999999999, 2024-09-14T06:32:54.326 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:32:54.326 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:32:54.326 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:32:54.326 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:32:54.326 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:32:54.326 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:54.326 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 53353 2024-09-14T06:32:54.326 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:54.326 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:32:54.326 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:54.326 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:32:54.326 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.372545+0000", 2024-09-14T06:32:54.326 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:54.326 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:54.327 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:54.327 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:32:54.327 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.372545+0000", 2024-09-14T06:32:54.327 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:54.327 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:54.327 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:54.327 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:32:54.327 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.372545+0000", 2024-09-14T06:32:54.327 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.27e-07 2024-09-14T06:32:54.327 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:54.327 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:54.327 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:32:54.327 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.372547+0000", 2024-09-14T06:32:54.327 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.0600000000000002e-06 2024-09-14T06:32:54.328 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:54.328 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:54.328 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:32:54.328 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.372553+0000", 2024-09-14T06:32:54.328 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.7980000000000002e-06 2024-09-14T06:32:54.328 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:54.328 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:54.328 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:32:54.328 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.372557+0000", 2024-09-14T06:32:54.328 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.7389999999999999e-06 2024-09-14T06:32:54.328 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:54.328 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:54.328 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:32:54.328 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.419811+0000", 2024-09-14T06:32:54.329 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047253310999999999 2024-09-14T06:32:54.329 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:54.329 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:54.329 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:32:54.329 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.419854+0000", 2024-09-14T06:32:54.329 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.3448000000000001e-05 2024-09-14T06:32:54.329 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:54.329 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:54.329 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:32:54.329 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.419909+0000", 2024-09-14T06:32:54.329 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.4549999999999998e-05 2024-09-14T06:32:54.329 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:54.329 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:32:54.330 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:54.330 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:54.330 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:54.330 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:53354 3.f 3:f71f51a7:::rbd_data.112dca2f6ae1.0000000000000015:17 [read 1400832~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:32:54.330 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:44.372563+0000", 2024-09-14T06:32:54.330 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 189.87761921699999, 2024-09-14T06:32:54.330 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047482233999999998, 2024-09-14T06:32:54.330 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:32:54.330 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:32:54.330 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:32:54.330 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:32:54.330 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:32:54.330 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:54.330 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 53354 2024-09-14T06:32:54.330 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:54.331 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:32:54.331 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:54.331 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:32:54.331 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.372563+0000", 2024-09-14T06:32:54.331 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:54.331 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:54.331 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:54.331 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:32:54.331 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.372563+0000", 2024-09-14T06:32:54.331 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:54.331 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:54.331 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:54.331 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:32:54.332 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.375009+0000", 2024-09-14T06:32:54.332 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.0024457680000000001 2024-09-14T06:32:54.332 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:54.332 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:54.332 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:32:54.332 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.375014+0000", 2024-09-14T06:32:54.332 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.0599999999999998e-06 2024-09-14T06:32:54.332 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:54.332 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:54.332 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:32:54.332 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.375026+0000", 2024-09-14T06:32:54.332 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.1783999999999999e-05 2024-09-14T06:32:54.332 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:54.333 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:54.333 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:32:54.333 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.375035+0000", 2024-09-14T06:32:54.333 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.7799999999999995e-06 2024-09-14T06:32:54.333 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:54.333 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:54.333 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:32:54.333 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.419942+0000", 2024-09-14T06:32:54.333 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.044906982999999998 2024-09-14T06:32:54.333 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:54.333 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:54.333 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:32:54.333 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.419989+0000", 2024-09-14T06:32:54.334 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.7104999999999997e-05 2024-09-14T06:32:54.334 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:54.334 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:54.334 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:32:54.334 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.420045+0000", 2024-09-14T06:32:54.334 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.5754000000000002e-05 2024-09-14T06:32:54.334 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:54.334 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:32:54.334 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:54.334 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:54.334 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:54.334 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:225577 3.15 3:ac159a70:::rbd_header.112daa9ec306:head [watch reconnect cookie 140616827151664 gen 4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e188)", 2024-09-14T06:32:54.334 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:31:52.733493+0000", 2024-09-14T06:32:54.334 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 61.516688752999997, 2024-09-14T06:32:54.335 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.0156593789999999, 2024-09-14T06:32:54.335 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:32:54.335 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:32:54.335 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:32:54.335 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:32:54.335 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:32:54.335 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:54.335 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 225577 2024-09-14T06:32:54.335 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:54.335 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:32:54.335 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:54.335 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:32:54.335 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:31:52.733493+0000", 2024-09-14T06:32:54.335 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:54.336 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:54.336 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:54.336 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:32:54.336 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:31:52.733493+0000", 2024-09-14T06:32:54.336 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:54.336 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:54.336 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:54.336 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:32:54.336 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:31:52.733495+0000", 2024-09-14T06:32:54.336 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.0059999999999999e-06 2024-09-14T06:32:54.336 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:54.336 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:54.336 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:32:54.336 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:31:52.733504+0000", 2024-09-14T06:32:54.337 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.5490000000000006e-06 2024-09-14T06:32:54.337 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:54.337 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:54.337 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:32:54.337 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:31:52.733522+0000", 2024-09-14T06:32:54.337 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.7660999999999999e-05 2024-09-14T06:32:54.337 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:54.337 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:54.337 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:32:54.337 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:31:52.733531+0000", 2024-09-14T06:32:54.337 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.7180000000000008e-06 2024-09-14T06:32:54.337 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:54.337 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:54.337 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:32:54.338 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:31:52.733598+0000", 2024-09-14T06:32:54.338 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.6490999999999997e-05 2024-09-14T06:32:54.338 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:54.338 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:54.338 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "op must wait for map", 2024-09-14T06:32:54.338 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:31:52.733605+0000", 2024-09-14T06:32:54.338 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.6109999999999996e-06 2024-09-14T06:32:54.338 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:54.338 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:54.338 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:32:54.338 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:31:52.737908+0000", 2024-09-14T06:32:54.338 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.0043023050000000002 2024-09-14T06:32:54.338 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:54.339 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:54.339 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "waiting for peered", 2024-09-14T06:32:54.339 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:31:52.737923+0000", 2024-09-14T06:32:54.339 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.488e-05 2024-09-14T06:32:54.339 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:54.339 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:54.339 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:32:54.339 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:31:53.746776+0000", 2024-09-14T06:32:54.339 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.0088535359999999 2024-09-14T06:32:54.339 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:54.339 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:54.339 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:32:54.339 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:31:53.746878+0000", 2024-09-14T06:32:54.340 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.000101788 2024-09-14T06:32:54.340 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:54.340 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:54.340 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:32:54.340 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:31:53.749153+0000", 2024-09-14T06:32:54.340 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.0022748339999999999 2024-09-14T06:32:54.340 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:54.340 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:32:54.340 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:54.340 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:54.340 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:54.340 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:225581 3.15 3:ab407115:::rbd_object_map.112d838b9898:head [call lock.unlock in=10b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e188)", 2024-09-14T06:32:54.340 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:31:52.743472+0000", 2024-09-14T06:32:54.340 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 61.506709714000003, 2024-09-14T06:32:54.341 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.007270501, 2024-09-14T06:32:54.341 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:32:54.341 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:32:54.341 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:32:54.341 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:32:54.341 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:32:54.341 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:54.341 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 225581 2024-09-14T06:32:54.341 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:54.341 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:32:54.341 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:54.341 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:32:54.341 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:31:52.743472+0000", 2024-09-14T06:32:54.341 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:54.342 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:54.342 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:54.342 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:32:54.342 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:31:52.743472+0000", 2024-09-14T06:32:54.342 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:54.342 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:54.342 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:54.342 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:32:54.342 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:31:52.743473+0000", 2024-09-14T06:32:54.342 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.1999999999999998e-07 2024-09-14T06:32:54.342 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:54.342 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:54.342 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:32:54.342 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:31:52.743478+0000", 2024-09-14T06:32:54.342 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.515e-06 2024-09-14T06:32:54.343 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:54.343 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:54.343 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:32:54.343 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:31:52.743489+0000", 2024-09-14T06:32:54.343 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.1411e-05 2024-09-14T06:32:54.343 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:54.343 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:54.343 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:32:54.343 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:31:52.743494+0000", 2024-09-14T06:32:54.343 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.1649999999999997e-06 2024-09-14T06:32:54.343 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:54.343 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:54.343 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:32:54.343 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:31:52.743548+0000", 2024-09-14T06:32:54.344 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.3758999999999998e-05 2024-09-14T06:32:54.344 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:54.344 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:54.344 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "waiting for peered", 2024-09-14T06:32:54.344 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:31:52.743558+0000", 2024-09-14T06:32:54.344 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.0336999999999999e-05 2024-09-14T06:32:54.344 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:54.344 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:54.344 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:32:54.344 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:31:53.747716+0000", 2024-09-14T06:32:54.344 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.004157035 2024-09-14T06:32:54.344 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:54.344 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:54.344 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:32:54.345 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:31:53.747831+0000", 2024-09-14T06:32:54.345 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.000115394 2024-09-14T06:32:54.345 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:54.345 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:54.345 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "waiting for subops from 3", 2024-09-14T06:32:54.345 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:31:53.748002+0000", 2024-09-14T06:32:54.345 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00017085099999999999 2024-09-14T06:32:54.345 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:54.345 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:54.345 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:32:54.345 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:31:53.749536+0000", 2024-09-14T06:32:54.345 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.0015339990000000001 2024-09-14T06:32:54.345 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:54.345 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:54.346 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "op_commit", 2024-09-14T06:32:54.346 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:31:53.750672+0000", 2024-09-14T06:32:54.346 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.001135973 2024-09-14T06:32:54.346 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:54.346 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:54.346 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "commit_sent", 2024-09-14T06:32:54.346 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:31:53.750704+0000", 2024-09-14T06:32:54.346 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.1791000000000001e-05 2024-09-14T06:32:54.346 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:54.346 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:54.346 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:32:54.346 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:31:53.750743+0000", 2024-09-14T06:32:54.346 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.9450999999999999e-05 2024-09-14T06:32:54.346 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:54.347 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:32:54.347 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:54.347 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:54.347 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:32:54.347 INFO:teuthology.orchestra.run.smithi138.stdout:} 2024-09-14T06:32:54.348 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-14T06:32:54.349 INFO:tasks.ceph.osd.1.smithi094.stderr:2024-09-14T06:32:54.347+0000 7f328e47a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33641) UID: 0 2024-09-14T06:32:54.397 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-14T06:32:54.449 INFO:tasks.ceph.osd.5.smithi138.stderr:2024-09-14T06:32:54.449+0000 7fe13746f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33618) UID: 0 2024-09-14T06:32:54.457 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-14T06:32:54.458 INFO:teuthology.orchestra.run.smithi094.stdout: "ops": [], 2024-09-14T06:32:54.458 INFO:teuthology.orchestra.run.smithi094.stdout: "num_ops": 0 2024-09-14T06:32:54.458 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-14T06:32:54.468 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-14T06:32:54.536 INFO:teuthology.orchestra.run.smithi094.stdout:osd.0: {} 2024-09-14T06:32:54.536 INFO:teuthology.orchestra.run.smithi094.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-14T06:32:54.550 INFO:tasks.ceph.osd.7.smithi138.stderr:2024-09-14T06:32:54.550+0000 7f3c6a21d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33623) UID: 0 2024-09-14T06:32:54.567 INFO:teuthology.orchestra.run.smithi094.stdout:osd.1: {} 2024-09-14T06:32:54.568 INFO:teuthology.orchestra.run.smithi094.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-14T06:32:54.589 INFO:teuthology.orchestra.run.smithi094.stdout:osd.2: {} 2024-09-14T06:32:54.589 INFO:teuthology.orchestra.run.smithi094.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-14T06:32:54.599 INFO:teuthology.orchestra.run.smithi094.stdout:ERROR: (22) Invalid argument 2024-09-14T06:32:54.599 INFO:teuthology.orchestra.run.smithi094.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:32:54.605 INFO:teuthology.orchestra.run.smithi094.stdout:osd.3: {} 2024-09-14T06:32:54.605 INFO:teuthology.orchestra.run.smithi094.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-14T06:32:54.608 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-14T06:32:54.624 INFO:teuthology.orchestra.run.smithi094.stdout:osd.4: {} 2024-09-14T06:32:54.624 INFO:teuthology.orchestra.run.smithi094.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-14T06:32:54.641 INFO:teuthology.orchestra.run.smithi094.stderr:set 2.6 pg_upmap_items mapping to [3->5,4->6] 2024-09-14T06:32:54.644 INFO:teuthology.orchestra.run.smithi094.stderr:noscrub is unset 2024-09-14T06:32:54.648 INFO:teuthology.orchestra.run.smithi094.stdout:osd.5: {} 2024-09-14T06:32:54.648 INFO:teuthology.orchestra.run.smithi094.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-14T06:32:54.651 INFO:tasks.ceph.osd.7.smithi138.stderr:2024-09-14T06:32:54.650+0000 7f3c6a21d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33623) UID: 0 2024-09-14T06:32:54.672 INFO:teuthology.orchestra.run.smithi094.stdout:osd.6: {} 2024-09-14T06:32:54.672 INFO:teuthology.orchestra.run.smithi094.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-14T06:32:54.692 INFO:teuthology.orchestra.run.smithi094.stdout:osd.7: {} 2024-09-14T06:32:54.692 INFO:teuthology.orchestra.run.smithi094.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-14T06:32:54.736 INFO:teuthology.orchestra.run.smithi094.stdout:ERROR: (22) Invalid argument 2024-09-14T06:32:54.736 INFO:teuthology.orchestra.run.smithi094.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:32:54.745 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-14T06:32:54.753 INFO:tasks.ceph.osd.6.smithi138.stderr:2024-09-14T06:32:54.751+0000 7f8cd8849640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33620) UID: 0 2024-09-14T06:32:54.854 INFO:tasks.ceph.osd.3.smithi094.stderr:2024-09-14T06:32:54.853+0000 7feeeafe8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33655) UID: 0 2024-09-14T06:32:54.888 INFO:teuthology.orchestra.run.smithi094.stdout:ERROR: (22) Invalid argument 2024-09-14T06:32:54.888 INFO:teuthology.orchestra.run.smithi094.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:32:54.897 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-14T06:32:54.954 INFO:tasks.ceph.osd.6.smithi138.stderr:2024-09-14T06:32:54.954+0000 7f8cd8849640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33620) UID: 0 2024-09-14T06:32:55.039 INFO:teuthology.orchestra.run.smithi094.stdout:ERROR: (22) Invalid argument 2024-09-14T06:32:55.039 INFO:teuthology.orchestra.run.smithi094.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:32:55.048 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-14T06:32:55.055 INFO:tasks.ceph.osd.6.smithi138.stderr:2024-09-14T06:32:55.055+0000 7f8cd8849640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33620) UID: 0 2024-09-14T06:32:55.150 INFO:teuthology.orchestra.run.smithi094.stdout:ERROR: (22) Invalid argument 2024-09-14T06:32:55.151 INFO:teuthology.orchestra.run.smithi094.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:32:55.156 INFO:tasks.ceph.osd.0.smithi094.stderr:2024-09-14T06:32:55.155+0000 7f07c9333640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33636) UID: 0 2024-09-14T06:32:55.160 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-14T06:32:55.257 INFO:tasks.ceph.osd.6.smithi138.stderr:2024-09-14T06:32:55.256+0000 7f8cd8849640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33620) UID: 0 2024-09-14T06:32:55.303 INFO:teuthology.orchestra.run.smithi094.stdout:ERROR: (22) Invalid argument 2024-09-14T06:32:55.304 INFO:teuthology.orchestra.run.smithi094.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:32:55.313 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-14T06:32:55.358 INFO:tasks.ceph.osd.5.smithi138.stderr:2024-09-14T06:32:55.358+0000 7fe13746f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33618) UID: 0 2024-09-14T06:32:55.416 INFO:teuthology.orchestra.run.smithi094.stdout:ERROR: (22) Invalid argument 2024-09-14T06:32:55.416 INFO:teuthology.orchestra.run.smithi094.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:32:55.425 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-14T06:32:55.468 INFO:tasks.ceph.osd.2.smithi094.stderr:2024-09-14T06:32:55.467+0000 7faf3269b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33649) UID: 0 2024-09-14T06:32:55.528 INFO:teuthology.orchestra.run.smithi094.stdout:ERROR: (22) Invalid argument 2024-09-14T06:32:55.528 INFO:teuthology.orchestra.run.smithi094.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:32:55.536 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-14T06:32:55.577 INFO:tasks.ceph.osd.2.smithi094.stderr:2024-09-14T06:32:55.576+0000 7faf3269b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33649) UID: 0 2024-09-14T06:32:55.661 INFO:tasks.ceph.osd.6.smithi138.stderr:2024-09-14T06:32:55.660+0000 7f8cd8849640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33620) UID: 0 2024-09-14T06:32:55.700 INFO:teuthology.orchestra.run.smithi094.stdout:ERROR: (22) Invalid argument 2024-09-14T06:32:55.701 INFO:teuthology.orchestra.run.smithi094.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:32:55.711 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-14T06:32:55.762 INFO:tasks.ceph.osd.4.smithi138.stderr:2024-09-14T06:32:55.761+0000 7f6c653c8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33615) UID: 0 2024-09-14T06:32:55.815 INFO:teuthology.orchestra.run.smithi094.stdout:ERROR: (22) Invalid argument 2024-09-14T06:32:55.816 INFO:teuthology.orchestra.run.smithi094.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:32:55.825 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-14T06:32:55.869 INFO:tasks.ceph.osd.1.smithi094.stderr:2024-09-14T06:32:55.868+0000 7f328e47a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33641) UID: 0 2024-09-14T06:32:55.928 INFO:teuthology.orchestra.run.smithi094.stdout:ERROR: (22) Invalid argument 2024-09-14T06:32:55.928 INFO:teuthology.orchestra.run.smithi094.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:32:55.936 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-14T06:32:55.980 INFO:tasks.ceph.osd.6.smithi138.stderr:2024-09-14T06:32:55.979+0000 7f8cd8849640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33620) UID: 0 2024-09-14T06:32:56.043 INFO:teuthology.orchestra.run.smithi138.stdout:ERROR: (22) Invalid argument 2024-09-14T06:32:56.043 INFO:teuthology.orchestra.run.smithi138.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:32:56.051 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-14T06:32:56.093 INFO:tasks.ceph.osd.6.smithi138.stderr:2024-09-14T06:32:56.092+0000 7f8cd8849640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33620) UID: 0 2024-09-14T06:32:56.165 INFO:tasks.ceph.osd.4.smithi138.stderr:2024-09-14T06:32:56.164+0000 7f6c653c8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33615) UID: 0 2024-09-14T06:32:56.195 INFO:teuthology.orchestra.run.smithi138.stdout:ERROR: (22) Invalid argument 2024-09-14T06:32:56.195 INFO:teuthology.orchestra.run.smithi138.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:32:56.204 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-14T06:32:56.266 INFO:tasks.ceph.osd.3.smithi094.stderr:2024-09-14T06:32:56.265+0000 7feeeafe8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33655) UID: 0 2024-09-14T06:32:56.348 INFO:teuthology.orchestra.run.smithi138.stdout:ERROR: (22) Invalid argument 2024-09-14T06:32:56.348 INFO:teuthology.orchestra.run.smithi138.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:32:56.357 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-14T06:32:56.366 INFO:tasks.ceph.osd.0.smithi094.stderr:2024-09-14T06:32:56.365+0000 7f07c9333640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33636) UID: 0 2024-09-14T06:32:56.460 INFO:teuthology.orchestra.run.smithi138.stdout:ERROR: (22) Invalid argument 2024-09-14T06:32:56.460 INFO:teuthology.orchestra.run.smithi138.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:32:56.467 INFO:tasks.ceph.osd.3.smithi094.stderr:2024-09-14T06:32:56.466+0000 7feeeafe8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33655) UID: 0 2024-09-14T06:32:56.469 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-14T06:32:56.568 INFO:tasks.ceph.osd.1.smithi094.stderr:2024-09-14T06:32:56.567+0000 7f328e47a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33641) UID: 0 2024-09-14T06:32:56.571 INFO:teuthology.orchestra.run.smithi138.stdout:ERROR: (22) Invalid argument 2024-09-14T06:32:56.572 INFO:teuthology.orchestra.run.smithi138.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:32:56.580 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-14T06:32:56.666 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-14T06:32:56.669 INFO:tasks.ceph.osd.1.smithi094.stderr:2024-09-14T06:32:56.668+0000 7f328e47a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33641) UID: 0 2024-09-14T06:32:56.693 INFO:teuthology.orchestra.run.smithi138.stdout:ERROR: (22) Invalid argument 2024-09-14T06:32:56.693 INFO:teuthology.orchestra.run.smithi138.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:32:56.703 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-14T06:32:56.706 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-14T06:32:56.770 INFO:tasks.ceph.osd.5.smithi138.stderr:2024-09-14T06:32:56.769+0000 7fe13746f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33618) UID: 0 2024-09-14T06:32:56.812 INFO:teuthology.orchestra.run.smithi138.stdout:ERROR: (22) Invalid argument 2024-09-14T06:32:56.813 INFO:teuthology.orchestra.run.smithi138.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:32:56.822 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-14T06:32:56.843 INFO:teuthology.orchestra.run.smithi094.stdout:osd.0: {} 2024-09-14T06:32:56.843 INFO:teuthology.orchestra.run.smithi094.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-14T06:32:56.859 INFO:teuthology.orchestra.run.smithi094.stdout:osd.1: {} 2024-09-14T06:32:56.860 INFO:teuthology.orchestra.run.smithi094.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-14T06:32:56.877 INFO:teuthology.orchestra.run.smithi094.stdout:osd.2: {} 2024-09-14T06:32:56.877 INFO:teuthology.orchestra.run.smithi094.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-14T06:32:56.893 INFO:teuthology.orchestra.run.smithi094.stdout:osd.3: {} 2024-09-14T06:32:56.893 INFO:teuthology.orchestra.run.smithi094.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-14T06:32:56.906 INFO:tasks.ceph.osd.7.smithi138.stderr:2024-09-14T06:32:56.905+0000 7f3c6a21d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33623) UID: 0 2024-09-14T06:32:56.911 INFO:teuthology.orchestra.run.smithi094.stdout:osd.4: {} 2024-09-14T06:32:56.911 INFO:teuthology.orchestra.run.smithi094.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-14T06:32:56.927 INFO:teuthology.orchestra.run.smithi094.stdout:osd.5: {} 2024-09-14T06:32:56.927 INFO:teuthology.orchestra.run.smithi094.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-14T06:32:56.944 INFO:teuthology.orchestra.run.smithi094.stdout:osd.6: {} 2024-09-14T06:32:56.944 INFO:teuthology.orchestra.run.smithi094.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-14T06:32:56.960 INFO:teuthology.orchestra.run.smithi094.stdout:osd.7: {} 2024-09-14T06:32:56.960 INFO:teuthology.orchestra.run.smithi094.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-14T06:32:56.967 INFO:teuthology.orchestra.run.smithi138.stdout:{ 2024-09-14T06:32:56.967 INFO:teuthology.orchestra.run.smithi138.stdout: "ops": [], 2024-09-14T06:32:56.968 INFO:teuthology.orchestra.run.smithi138.stdout: "complaint_time": 30, 2024-09-14T06:32:56.968 INFO:teuthology.orchestra.run.smithi138.stdout: "num_blocked_ops": 0 2024-09-14T06:32:56.968 INFO:teuthology.orchestra.run.smithi138.stdout:} 2024-09-14T06:32:56.971 INFO:tasks.ceph.osd.4.smithi138.stderr:2024-09-14T06:32:56.970+0000 7f6c653c8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33615) UID: 0 2024-09-14T06:32:56.976 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-14T06:32:57.072 INFO:tasks.ceph.osd.6.smithi138.stderr:2024-09-14T06:32:57.070+0000 7f8cd8849640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33620) UID: 0 2024-09-14T06:32:57.121 INFO:teuthology.orchestra.run.smithi138.stdout:{ 2024-09-14T06:32:57.121 INFO:teuthology.orchestra.run.smithi138.stdout: "size": 20, 2024-09-14T06:32:57.121 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 600, 2024-09-14T06:32:57.121 INFO:teuthology.orchestra.run.smithi138.stdout: "ops": [ 2024-09-14T06:32:57.121 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:57.121 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:98505 3.10 3.1bf56f90 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T06:32:57.121 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:30:07.045453+0000", 2024-09-14T06:32:57.121 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 170.07422051899999, 2024-09-14T06:32:57.121 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032144440000000003, 2024-09-14T06:32:57.121 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:32:57.122 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:32:57.122 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "reached pg", 2024-09-14T06:32:57.122 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:32:57.122 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:32:57.122 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:57.122 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 98505 2024-09-14T06:32:57.122 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:57.122 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:32:57.122 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:57.122 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:32:57.122 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045453+0000", 2024-09-14T06:32:57.122 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:57.122 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:57.122 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:57.122 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:32:57.123 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045453+0000", 2024-09-14T06:32:57.123 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:57.123 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:57.123 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:57.123 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:32:57.123 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045453+0000", 2024-09-14T06:32:57.123 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.36e-07 2024-09-14T06:32:57.123 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:57.123 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:57.123 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:32:57.123 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045454+0000", 2024-09-14T06:32:57.123 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.1299999999999998e-07 2024-09-14T06:32:57.123 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:57.123 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:57.123 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:32:57.123 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045457+0000", 2024-09-14T06:32:57.124 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.8559999999999998e-06 2024-09-14T06:32:57.124 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:57.124 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:57.124 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:32:57.124 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045460+0000", 2024-09-14T06:32:57.124 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.8750000000000002e-06 2024-09-14T06:32:57.124 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:57.124 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:57.124 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:32:57.124 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.077592+0000", 2024-09-14T06:32:57.124 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032131727999999998 2024-09-14T06:32:57.124 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:57.124 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:57.124 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:32:57.124 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.077597+0000", 2024-09-14T06:32:57.125 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.8319999999999996e-06 2024-09-14T06:32:57.125 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:57.125 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:32:57.125 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:57.125 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:57.125 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:57.125 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:98506 3.10 3.1bf56f90 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T06:32:57.125 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:30:07.045481+0000", 2024-09-14T06:32:57.125 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 170.074192264, 2024-09-14T06:32:57.125 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032142759999999999, 2024-09-14T06:32:57.125 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:32:57.125 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:32:57.125 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "reached pg", 2024-09-14T06:32:57.126 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:32:57.126 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:32:57.126 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:57.126 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 98506 2024-09-14T06:32:57.126 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:57.126 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:32:57.126 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:57.126 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:32:57.126 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045481+0000", 2024-09-14T06:32:57.126 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:57.126 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:57.126 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:57.126 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:32:57.126 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045481+0000", 2024-09-14T06:32:57.126 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:57.127 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:57.127 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:57.127 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:32:57.127 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045481+0000", 2024-09-14T06:32:57.127 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.8500000000000001e-07 2024-09-14T06:32:57.127 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:57.127 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:57.127 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:32:57.127 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045483+0000", 2024-09-14T06:32:57.127 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.8250000000000001e-06 2024-09-14T06:32:57.127 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:57.127 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:57.127 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:32:57.127 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045489+0000", 2024-09-14T06:32:57.127 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.5740000000000003e-06 2024-09-14T06:32:57.128 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:57.128 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:57.128 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:32:57.128 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045495+0000", 2024-09-14T06:32:57.128 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.7690000000000002e-06 2024-09-14T06:32:57.128 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:57.128 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:57.128 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:32:57.128 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.077619+0000", 2024-09-14T06:32:57.128 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032124621999999999 2024-09-14T06:32:57.128 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:57.128 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:57.128 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:32:57.128 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.077624+0000", 2024-09-14T06:32:57.128 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.3850000000000004e-06 2024-09-14T06:32:57.129 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:57.129 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:32:57.129 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:57.129 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:57.129 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:57.129 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:98507 3.10 3.1bf56f90 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T06:32:57.129 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:30:07.045502+0000", 2024-09-14T06:32:57.129 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 170.07417084299999, 2024-09-14T06:32:57.129 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032147219999999997, 2024-09-14T06:32:57.129 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:32:57.129 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:32:57.129 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "reached pg", 2024-09-14T06:32:57.129 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:32:57.129 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:32:57.129 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:57.130 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 98507 2024-09-14T06:32:57.130 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:57.130 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:32:57.130 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:57.130 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:32:57.130 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045502+0000", 2024-09-14T06:32:57.130 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:57.130 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:57.130 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:57.130 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:32:57.130 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045502+0000", 2024-09-14T06:32:57.130 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:57.130 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:57.130 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:57.131 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:32:57.131 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045503+0000", 2024-09-14T06:32:57.131 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.4700000000000002e-07 2024-09-14T06:32:57.131 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:57.131 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:57.131 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:32:57.131 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045509+0000", 2024-09-14T06:32:57.131 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.7329999999999997e-06 2024-09-14T06:32:57.131 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:57.131 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:57.131 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:32:57.131 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045515+0000", 2024-09-14T06:32:57.131 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.8030000000000002e-06 2024-09-14T06:32:57.131 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:57.131 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:57.131 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:32:57.132 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045522+0000", 2024-09-14T06:32:57.132 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.0369999999999997e-06 2024-09-14T06:32:57.132 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:57.132 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:57.132 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:32:57.132 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.077641+0000", 2024-09-14T06:32:57.132 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032119091000000002 2024-09-14T06:32:57.132 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:57.132 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:57.132 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:32:57.132 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.077650+0000", 2024-09-14T06:32:57.133 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.2090000000000005e-06 2024-09-14T06:32:57.133 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:57.133 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:32:57.133 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:57.133 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:57.133 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:57.133 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:98508 3.10 3.1bf56f90 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T06:32:57.133 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:30:07.045530+0000", 2024-09-14T06:32:57.133 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 170.07414322599999, 2024-09-14T06:32:57.133 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032198773999999999, 2024-09-14T06:32:57.133 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:32:57.133 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:32:57.133 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "reached pg", 2024-09-14T06:32:57.133 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:32:57.133 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:32:57.133 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:57.134 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 98508 2024-09-14T06:32:57.134 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:57.134 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:32:57.134 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:57.134 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:32:57.134 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045530+0000", 2024-09-14T06:32:57.134 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:57.134 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:57.134 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:57.134 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:32:57.134 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045530+0000", 2024-09-14T06:32:57.134 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:57.134 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:57.134 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:57.135 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:32:57.135 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045530+0000", 2024-09-14T06:32:57.135 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.3099999999999998e-07 2024-09-14T06:32:57.135 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:57.135 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:57.135 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:32:57.135 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045539+0000", 2024-09-14T06:32:57.135 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.4780000000000008e-06 2024-09-14T06:32:57.135 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:57.135 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:57.135 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:32:57.135 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045546+0000", 2024-09-14T06:32:57.136 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.3259999999999998e-06 2024-09-14T06:32:57.136 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:57.136 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:57.136 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:32:57.136 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045551+0000", 2024-09-14T06:32:57.137 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.9960000000000001e-06 2024-09-14T06:32:57.137 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:57.137 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:57.137 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:32:57.137 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.077716+0000", 2024-09-14T06:32:57.137 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032165081999999998 2024-09-14T06:32:57.137 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:57.137 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:57.138 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:32:57.138 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.077729+0000", 2024-09-14T06:32:57.138 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.2461e-05 2024-09-14T06:32:57.138 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:57.138 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:32:57.138 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:57.138 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:57.139 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:57.139 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:98509 3.10 3.1bf56f90 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T06:32:57.139 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:30:07.045558+0000", 2024-09-14T06:32:57.139 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 170.074115025, 2024-09-14T06:32:57.139 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032242198, 2024-09-14T06:32:57.139 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:32:57.139 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:32:57.139 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "reached pg", 2024-09-14T06:32:57.139 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:32:57.139 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:32:57.139 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:57.139 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 98509 2024-09-14T06:32:57.139 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:57.139 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:32:57.140 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:57.140 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:32:57.140 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045558+0000", 2024-09-14T06:32:57.140 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:57.140 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:57.140 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:57.140 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:32:57.140 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045558+0000", 2024-09-14T06:32:57.140 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:57.140 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:57.140 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:57.140 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:32:57.140 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045559+0000", 2024-09-14T06:32:57.140 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.27e-07 2024-09-14T06:32:57.140 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:57.141 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:57.141 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:32:57.141 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045560+0000", 2024-09-14T06:32:57.141 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.6959999999999999e-06 2024-09-14T06:32:57.141 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:57.141 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:57.141 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:32:57.141 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045567+0000", 2024-09-14T06:32:57.141 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.6900000000000003e-06 2024-09-14T06:32:57.141 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:57.141 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:57.141 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:32:57.141 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045571+0000", 2024-09-14T06:32:57.141 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.5750000000000002e-06 2024-09-14T06:32:57.142 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:57.142 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:57.142 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:32:57.142 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.077789+0000", 2024-09-14T06:32:57.142 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032217186000000002 2024-09-14T06:32:57.142 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:57.142 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:57.142 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:32:57.142 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.077800+0000", 2024-09-14T06:32:57.142 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.1623999999999999e-05 2024-09-14T06:32:57.142 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:57.142 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:32:57.142 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:57.142 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:57.142 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:57.143 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:98510 3.10 3.1bf56f90 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T06:32:57.143 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:30:07.045580+0000", 2024-09-14T06:32:57.143 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 170.07409321099999, 2024-09-14T06:32:57.143 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032299845000000001, 2024-09-14T06:32:57.143 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:32:57.143 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:32:57.143 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "reached pg", 2024-09-14T06:32:57.143 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:32:57.143 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:32:57.143 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:57.143 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 98510 2024-09-14T06:32:57.143 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:57.143 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:32:57.144 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:57.144 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:32:57.144 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045580+0000", 2024-09-14T06:32:57.144 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:57.144 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:57.144 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:57.144 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:32:57.144 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045580+0000", 2024-09-14T06:32:57.144 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:57.144 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:57.144 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:57.144 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:32:57.144 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045580+0000", 2024-09-14T06:32:57.144 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.0400000000000002e-07 2024-09-14T06:32:57.145 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:57.145 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:57.145 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:32:57.145 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045582+0000", 2024-09-14T06:32:57.145 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.7600000000000001e-06 2024-09-14T06:32:57.145 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:57.145 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:57.145 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:32:57.145 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045588+0000", 2024-09-14T06:32:57.146 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.5650000000000001e-06 2024-09-14T06:32:57.146 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:57.146 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:57.146 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:32:57.146 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045597+0000", 2024-09-14T06:32:57.146 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.465e-06 2024-09-14T06:32:57.146 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:57.146 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:57.146 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:32:57.146 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.077871+0000", 2024-09-14T06:32:57.146 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032273562999999998 2024-09-14T06:32:57.146 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:57.146 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:57.146 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:32:57.146 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.077880+0000", 2024-09-14T06:32:57.147 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.0880000000000001e-06 2024-09-14T06:32:57.147 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:57.147 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:32:57.147 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:57.147 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:57.147 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:57.147 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:98511 3.10 3.1bf56f90 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T06:32:57.147 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:30:07.045610+0000", 2024-09-14T06:32:57.147 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 170.07406336400001, 2024-09-14T06:32:57.147 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032338894, 2024-09-14T06:32:57.147 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:32:57.147 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:32:57.147 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "reached pg", 2024-09-14T06:32:57.147 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:32:57.147 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:32:57.147 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:57.147 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 98511 2024-09-14T06:32:57.148 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:57.148 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:32:57.148 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:57.148 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:32:57.148 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045610+0000", 2024-09-14T06:32:57.148 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:57.148 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:57.148 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:57.148 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:32:57.148 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045610+0000", 2024-09-14T06:32:57.148 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:57.148 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:57.148 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:57.148 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:32:57.148 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045610+0000", 2024-09-14T06:32:57.149 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.1600000000000002e-07 2024-09-14T06:32:57.149 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:57.149 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:57.149 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:32:57.149 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045612+0000", 2024-09-14T06:32:57.149 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.1150000000000001e-06 2024-09-14T06:32:57.149 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:57.149 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:57.149 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:32:57.149 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045618+0000", 2024-09-14T06:32:57.149 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.0499999999999997e-06 2024-09-14T06:32:57.149 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:57.149 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:57.150 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:32:57.150 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045623+0000", 2024-09-14T06:32:57.150 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.6770000000000003e-06 2024-09-14T06:32:57.150 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:57.150 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:57.150 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:32:57.150 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.077928+0000", 2024-09-14T06:32:57.150 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032305111999999997 2024-09-14T06:32:57.150 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:57.150 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:57.150 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:32:57.150 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.077949+0000", 2024-09-14T06:32:57.150 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.0523999999999999e-05 2024-09-14T06:32:57.150 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:57.150 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:32:57.150 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:57.151 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:57.151 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:57.151 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:98512 3.10 3.1bf56f90 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T06:32:57.151 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:30:07.045630+0000", 2024-09-14T06:32:57.151 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 170.074043359, 2024-09-14T06:32:57.151 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032346632, 2024-09-14T06:32:57.151 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:32:57.151 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:32:57.151 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "reached pg", 2024-09-14T06:32:57.151 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:32:57.151 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:32:57.151 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:57.151 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 98512 2024-09-14T06:32:57.151 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:57.151 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:32:57.152 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:57.152 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:32:57.152 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045630+0000", 2024-09-14T06:32:57.152 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:57.152 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:57.152 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:57.152 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:32:57.152 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045630+0000", 2024-09-14T06:32:57.152 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:57.152 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:57.152 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:57.152 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:32:57.153 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045630+0000", 2024-09-14T06:32:57.153 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.8200000000000001e-07 2024-09-14T06:32:57.153 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:57.153 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:57.153 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:32:57.153 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045632+0000", 2024-09-14T06:32:57.153 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.7430000000000001e-06 2024-09-14T06:32:57.153 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:57.153 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:57.153 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:32:57.153 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045637+0000", 2024-09-14T06:32:57.153 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.4550000000000003e-06 2024-09-14T06:32:57.153 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:57.153 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:57.153 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:32:57.154 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045644+0000", 2024-09-14T06:32:57.154 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.1330000000000001e-06 2024-09-14T06:32:57.154 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:57.154 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:57.154 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:32:57.154 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.077968+0000", 2024-09-14T06:32:57.154 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032323401000000002 2024-09-14T06:32:57.154 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:57.154 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:57.154 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:32:57.154 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.077976+0000", 2024-09-14T06:32:57.154 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.5180000000000004e-06 2024-09-14T06:32:57.154 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:57.154 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:32:57.154 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:57.155 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:57.155 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:57.155 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:98513 3.10 3.1bf56f90 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T06:32:57.155 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:30:07.045652+0000", 2024-09-14T06:32:57.155 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 170.074020795, 2024-09-14T06:32:57.155 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032353436999999999, 2024-09-14T06:32:57.155 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:32:57.155 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:32:57.155 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "reached pg", 2024-09-14T06:32:57.155 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:32:57.155 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:32:57.155 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:57.155 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 98513 2024-09-14T06:32:57.155 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:57.155 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:32:57.156 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:57.156 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:32:57.156 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045652+0000", 2024-09-14T06:32:57.156 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:57.156 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:57.156 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:57.156 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:32:57.156 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045652+0000", 2024-09-14T06:32:57.156 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:57.156 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:57.156 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:57.156 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:32:57.157 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045653+0000", 2024-09-14T06:32:57.157 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.3700000000000001e-07 2024-09-14T06:32:57.157 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:57.157 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:57.157 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:32:57.157 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045655+0000", 2024-09-14T06:32:57.157 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.854e-06 2024-09-14T06:32:57.157 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:57.157 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:57.157 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:32:57.157 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045661+0000", 2024-09-14T06:32:57.157 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.5509999999999996e-06 2024-09-14T06:32:57.157 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:57.157 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:57.158 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:32:57.158 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045668+0000", 2024-09-14T06:32:57.158 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.1980000000000004e-06 2024-09-14T06:32:57.158 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:57.158 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:57.158 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:32:57.158 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.077997+0000", 2024-09-14T06:32:57.158 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032328575999999998 2024-09-14T06:32:57.158 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:57.158 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:57.158 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:32:57.158 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.078006+0000", 2024-09-14T06:32:57.158 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.8210000000000008e-06 2024-09-14T06:32:57.158 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:57.158 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:32:57.159 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:57.159 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:57.159 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:57.159 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:98514 3.10 3.1bf56f90 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T06:32:57.159 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:30:07.045699+0000", 2024-09-14T06:32:57.159 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 170.07397445999999, 2024-09-14T06:32:57.159 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032330223999999998, 2024-09-14T06:32:57.160 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:32:57.160 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:32:57.160 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "reached pg", 2024-09-14T06:32:57.160 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:32:57.160 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:32:57.160 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:57.160 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 98514 2024-09-14T06:32:57.160 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:57.160 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:32:57.160 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:57.160 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:32:57.160 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045699+0000", 2024-09-14T06:32:57.160 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:57.160 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:57.160 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:57.160 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:32:57.161 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045699+0000", 2024-09-14T06:32:57.161 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:57.161 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:57.161 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:57.161 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:32:57.161 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045699+0000", 2024-09-14T06:32:57.161 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.3200000000000001e-07 2024-09-14T06:32:57.161 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:57.161 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:57.161 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:32:57.161 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045701+0000", 2024-09-14T06:32:57.161 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.762e-06 2024-09-14T06:32:57.161 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:57.161 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:57.161 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:32:57.161 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045705+0000", 2024-09-14T06:32:57.162 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.2359999999999998e-06 2024-09-14T06:32:57.162 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:57.162 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:57.162 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:32:57.162 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045708+0000", 2024-09-14T06:32:57.162 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.796e-06 2024-09-14T06:32:57.162 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:57.162 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:57.162 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:32:57.162 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.078023+0000", 2024-09-14T06:32:57.162 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032315213000000002 2024-09-14T06:32:57.162 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:57.162 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:57.162 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:32:57.163 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.078029+0000", 2024-09-14T06:32:57.163 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.8850000000000004e-06 2024-09-14T06:32:57.163 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:57.163 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:32:57.163 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:57.163 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:57.163 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:57.163 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:98515 3.10 3.1bf56f90 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T06:32:57.163 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:30:07.045712+0000", 2024-09-14T06:32:57.163 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 170.07396114900001, 2024-09-14T06:32:57.163 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032346871999999999, 2024-09-14T06:32:57.163 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:32:57.163 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:32:57.163 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "reached pg", 2024-09-14T06:32:57.163 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:32:57.164 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:32:57.164 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:57.164 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 98515 2024-09-14T06:32:57.164 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:57.164 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:32:57.164 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:57.164 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:32:57.164 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045712+0000", 2024-09-14T06:32:57.164 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:57.164 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:57.164 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:57.164 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:32:57.164 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045712+0000", 2024-09-14T06:32:57.165 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:57.165 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:57.165 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:57.165 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:32:57.165 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045712+0000", 2024-09-14T06:32:57.165 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.2100000000000001e-07 2024-09-14T06:32:57.165 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:57.165 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:57.165 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:32:57.165 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045713+0000", 2024-09-14T06:32:57.165 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.068e-06 2024-09-14T06:32:57.165 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:57.165 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:57.165 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:32:57.166 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045716+0000", 2024-09-14T06:32:57.166 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.971e-06 2024-09-14T06:32:57.166 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:57.166 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:57.166 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:32:57.166 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045719+0000", 2024-09-14T06:32:57.166 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.3879999999999998e-06 2024-09-14T06:32:57.166 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:57.166 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:57.166 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:32:57.166 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.078049+0000", 2024-09-14T06:32:57.166 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032330553999999997 2024-09-14T06:32:57.167 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:57.167 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:57.167 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:32:57.167 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.078059+0000", 2024-09-14T06:32:57.167 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.6700000000000006e-06 2024-09-14T06:32:57.167 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:57.167 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:32:57.167 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:57.167 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:57.167 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:57.167 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:98516 3.10 3.1bf56f90 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T06:32:57.167 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:30:07.045722+0000", 2024-09-14T06:32:57.167 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 170.07395139600001, 2024-09-14T06:32:57.167 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032374334999999997, 2024-09-14T06:32:57.167 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:32:57.167 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:32:57.168 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "reached pg", 2024-09-14T06:32:57.168 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:32:57.168 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:32:57.168 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:57.168 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 98516 2024-09-14T06:32:57.168 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:57.168 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:32:57.168 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:57.168 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:32:57.168 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045722+0000", 2024-09-14T06:32:57.168 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:57.168 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:57.168 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:57.169 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:32:57.169 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045722+0000", 2024-09-14T06:32:57.169 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:57.169 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:57.169 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:57.169 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:32:57.169 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045722+0000", 2024-09-14T06:32:57.169 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.8999999999999998e-07 2024-09-14T06:32:57.169 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:57.169 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:57.169 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:32:57.169 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045723+0000", 2024-09-14T06:32:57.169 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.0049999999999999e-06 2024-09-14T06:32:57.169 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:57.170 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:57.170 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:32:57.170 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045726+0000", 2024-09-14T06:32:57.170 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.9019999999999999e-06 2024-09-14T06:32:57.170 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:57.170 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:57.170 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:32:57.170 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045728+0000", 2024-09-14T06:32:57.170 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.3290000000000001e-06 2024-09-14T06:32:57.170 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:57.170 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:57.170 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:32:57.170 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.078090+0000", 2024-09-14T06:32:57.170 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032362123 2024-09-14T06:32:57.170 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:57.170 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:57.171 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:32:57.171 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.078096+0000", 2024-09-14T06:32:57.171 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.6860000000000003e-06 2024-09-14T06:32:57.171 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:57.171 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:32:57.171 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:57.171 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:57.171 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:57.171 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:98517 3.10 3.1bf56f90 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T06:32:57.171 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:30:07.045731+0000", 2024-09-14T06:32:57.171 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 170.07394166500001, 2024-09-14T06:32:57.171 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.03241227, 2024-09-14T06:32:57.171 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:32:57.171 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:32:57.171 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "reached pg", 2024-09-14T06:32:57.172 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:32:57.172 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:32:57.172 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:57.172 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 98517 2024-09-14T06:32:57.172 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:57.172 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:32:57.172 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:57.172 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:32:57.172 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045731+0000", 2024-09-14T06:32:57.172 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:57.173 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:57.173 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:57.173 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:32:57.173 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045731+0000", 2024-09-14T06:32:57.173 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:57.173 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:57.173 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:57.173 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:32:57.173 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045732+0000", 2024-09-14T06:32:57.173 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.0699999999999999e-07 2024-09-14T06:32:57.173 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:57.173 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:57.173 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:32:57.173 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045733+0000", 2024-09-14T06:32:57.174 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.9199999999999999e-07 2024-09-14T06:32:57.174 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:57.174 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:57.174 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:32:57.174 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045735+0000", 2024-09-14T06:32:57.174 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.8150000000000002e-06 2024-09-14T06:32:57.174 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:57.174 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:57.174 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:32:57.174 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045738+0000", 2024-09-14T06:32:57.174 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.3640000000000002e-06 2024-09-14T06:32:57.174 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:57.174 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:57.174 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:32:57.174 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.078126+0000", 2024-09-14T06:32:57.175 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032388206000000003 2024-09-14T06:32:57.175 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:57.175 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:57.175 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:32:57.175 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.078144+0000", 2024-09-14T06:32:57.175 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.7686e-05 2024-09-14T06:32:57.175 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:57.175 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:32:57.175 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:57.175 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:57.175 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:57.175 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:98518 3.10 3.1bf56f90 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T06:32:57.175 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:30:07.045741+0000", 2024-09-14T06:32:57.175 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 170.07393227, 2024-09-14T06:32:57.175 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032451793999999999, 2024-09-14T06:32:57.176 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:32:57.176 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:32:57.176 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "reached pg", 2024-09-14T06:32:57.176 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:32:57.176 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:32:57.176 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:57.176 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 98518 2024-09-14T06:32:57.176 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:57.176 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:32:57.176 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:57.176 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:32:57.176 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045741+0000", 2024-09-14T06:32:57.176 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:57.176 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:57.176 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:57.177 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:32:57.177 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045741+0000", 2024-09-14T06:32:57.177 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:57.177 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:57.177 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:57.177 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:32:57.177 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045741+0000", 2024-09-14T06:32:57.177 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.05e-07 2024-09-14T06:32:57.177 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:57.177 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:57.177 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:32:57.177 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045746+0000", 2024-09-14T06:32:57.177 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.2259999999999996e-06 2024-09-14T06:32:57.177 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:57.177 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:57.178 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:32:57.178 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045750+0000", 2024-09-14T06:32:57.178 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.2490000000000001e-06 2024-09-14T06:32:57.178 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:57.178 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:57.178 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:32:57.178 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045752+0000", 2024-09-14T06:32:57.178 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.3980000000000002e-06 2024-09-14T06:32:57.178 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:57.178 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:57.178 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:32:57.178 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.078184+0000", 2024-09-14T06:32:57.178 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032431593000000002 2024-09-14T06:32:57.178 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:57.178 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:57.179 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:32:57.179 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.078193+0000", 2024-09-14T06:32:57.179 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.1230000000000006e-06 2024-09-14T06:32:57.179 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:57.179 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:32:57.179 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:57.179 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:57.179 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:57.179 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:98519 3.10 3.1bf56f90 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T06:32:57.179 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:30:07.045755+0000", 2024-09-14T06:32:57.179 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 170.07391784500001, 2024-09-14T06:32:57.179 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032489431999999999, 2024-09-14T06:32:57.179 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:32:57.179 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:32:57.180 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "reached pg", 2024-09-14T06:32:57.180 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:32:57.180 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:32:57.180 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:57.180 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 98519 2024-09-14T06:32:57.180 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:57.180 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:32:57.180 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:57.180 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:32:57.180 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045755+0000", 2024-09-14T06:32:57.180 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:57.180 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:57.180 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:57.180 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:32:57.181 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045755+0000", 2024-09-14T06:32:57.181 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:57.181 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:57.181 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:57.181 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:32:57.181 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045756+0000", 2024-09-14T06:32:57.181 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.23e-07 2024-09-14T06:32:57.181 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:57.181 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:57.181 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:32:57.181 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045757+0000", 2024-09-14T06:32:57.181 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.1060000000000001e-06 2024-09-14T06:32:57.181 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:57.181 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:57.182 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:32:57.182 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045760+0000", 2024-09-14T06:32:57.182 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.988e-06 2024-09-14T06:32:57.182 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:57.182 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:57.182 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:32:57.182 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045765+0000", 2024-09-14T06:32:57.182 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.2689999999999999e-06 2024-09-14T06:32:57.182 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:57.182 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:57.182 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:32:57.182 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.078234+0000", 2024-09-14T06:32:57.182 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032468915000000001 2024-09-14T06:32:57.183 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:57.183 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:57.183 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:32:57.183 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.078245+0000", 2024-09-14T06:32:57.183 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.0930999999999999e-05 2024-09-14T06:32:57.183 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:57.183 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:32:57.183 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:57.183 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:57.183 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:57.183 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:98520 3.10 3.1bf56f90 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T06:32:57.183 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:30:07.045768+0000", 2024-09-14T06:32:57.183 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 170.073904967, 2024-09-14T06:32:57.184 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032531379999999999, 2024-09-14T06:32:57.184 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:32:57.184 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:32:57.184 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "reached pg", 2024-09-14T06:32:57.184 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:32:57.184 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:32:57.184 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:57.184 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 98520 2024-09-14T06:32:57.184 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:57.184 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:32:57.184 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:57.184 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:32:57.184 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045768+0000", 2024-09-14T06:32:57.184 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:57.185 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:57.185 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:57.185 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:32:57.185 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045768+0000", 2024-09-14T06:32:57.185 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:57.185 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:57.185 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:57.185 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:32:57.185 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045768+0000", 2024-09-14T06:32:57.185 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.22e-07 2024-09-14T06:32:57.185 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:57.185 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:57.185 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:32:57.185 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045773+0000", 2024-09-14T06:32:57.186 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.3470000000000001e-06 2024-09-14T06:32:57.186 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:57.186 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:57.186 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:32:57.186 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045776+0000", 2024-09-14T06:32:57.186 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.1319999999999998e-06 2024-09-14T06:32:57.186 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:57.186 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:57.186 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:32:57.186 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045779+0000", 2024-09-14T06:32:57.186 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.9749999999999999e-06 2024-09-14T06:32:57.186 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:57.186 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:57.186 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:32:57.186 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.078289+0000", 2024-09-14T06:32:57.186 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032509693999999999 2024-09-14T06:32:57.187 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:57.187 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:57.187 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:32:57.187 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.078300+0000", 2024-09-14T06:32:57.187 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.101e-05 2024-09-14T06:32:57.187 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:57.187 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:32:57.187 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:57.187 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:57.187 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:57.187 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:98521 3.10 3.1bf56f90 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T06:32:57.187 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:30:07.045782+0000", 2024-09-14T06:32:57.187 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 170.07389078700001, 2024-09-14T06:32:57.187 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032566109000000003, 2024-09-14T06:32:57.187 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:32:57.188 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:32:57.188 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "reached pg", 2024-09-14T06:32:57.188 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:32:57.188 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:32:57.188 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:57.188 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 98521 2024-09-14T06:32:57.188 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:57.188 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:32:57.188 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:57.188 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:32:57.188 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045782+0000", 2024-09-14T06:32:57.188 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:57.188 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:57.188 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:57.188 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:32:57.189 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045782+0000", 2024-09-14T06:32:57.189 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:57.189 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:57.189 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:57.189 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:32:57.189 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045783+0000", 2024-09-14T06:32:57.189 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.2499999999999999e-07 2024-09-14T06:32:57.189 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:57.189 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:57.189 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:32:57.189 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045784+0000", 2024-09-14T06:32:57.189 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.0109999999999999e-06 2024-09-14T06:32:57.189 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:57.189 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:57.189 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:32:57.189 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045786+0000", 2024-09-14T06:32:57.190 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.8289999999999999e-06 2024-09-14T06:32:57.190 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:57.190 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:57.190 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:32:57.190 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045793+0000", 2024-09-14T06:32:57.190 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.579e-06 2024-09-14T06:32:57.190 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:57.190 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:57.190 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:32:57.190 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.078338+0000", 2024-09-14T06:32:57.190 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032545389000000001 2024-09-14T06:32:57.190 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:57.190 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:57.190 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:32:57.190 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.078348+0000", 2024-09-14T06:32:57.191 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.0076e-05 2024-09-14T06:32:57.191 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:57.191 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:32:57.191 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:57.191 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:57.191 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:57.191 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:98522 3.10 3.1bf56f90 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T06:32:57.191 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:30:07.045796+0000", 2024-09-14T06:32:57.191 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 170.07387687900001, 2024-09-14T06:32:57.191 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032577846000000001, 2024-09-14T06:32:57.191 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:32:57.191 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:32:57.191 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "reached pg", 2024-09-14T06:32:57.192 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:32:57.192 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:32:57.192 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:57.192 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 98522 2024-09-14T06:32:57.192 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:57.192 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:32:57.192 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:57.192 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:32:57.192 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045796+0000", 2024-09-14T06:32:57.192 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:57.192 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:57.192 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:57.192 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:32:57.192 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045796+0000", 2024-09-14T06:32:57.193 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:57.193 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:57.193 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:57.193 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:32:57.193 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045796+0000", 2024-09-14T06:32:57.193 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.2100000000000001e-07 2024-09-14T06:32:57.193 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:57.193 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:57.193 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:32:57.193 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045797+0000", 2024-09-14T06:32:57.193 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.6899999999999996e-07 2024-09-14T06:32:57.193 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:57.193 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:57.194 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:32:57.194 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045800+0000", 2024-09-14T06:32:57.194 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.9979999999999999e-06 2024-09-14T06:32:57.194 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:57.194 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:57.194 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:32:57.194 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045803+0000", 2024-09-14T06:32:57.194 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.2850000000000001e-06 2024-09-14T06:32:57.194 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:57.194 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:57.194 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:32:57.194 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.078366+0000", 2024-09-14T06:32:57.194 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032562932000000003 2024-09-14T06:32:57.194 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:57.194 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:57.195 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:32:57.195 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.078374+0000", 2024-09-14T06:32:57.195 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.4409999999999993e-06 2024-09-14T06:32:57.195 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:57.195 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:32:57.195 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:57.195 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:57.195 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:57.195 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:98523 3.10 3.1bf56f90 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T06:32:57.195 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:30:07.045806+0000", 2024-09-14T06:32:57.195 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 170.07386733600001, 2024-09-14T06:32:57.195 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032611167000000003, 2024-09-14T06:32:57.195 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:32:57.195 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:32:57.196 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "reached pg", 2024-09-14T06:32:57.196 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:32:57.196 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:32:57.196 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:57.196 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 98523 2024-09-14T06:32:57.196 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:57.196 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:32:57.196 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:57.196 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:32:57.196 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045806+0000", 2024-09-14T06:32:57.196 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:57.196 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:57.196 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:57.197 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:32:57.197 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045806+0000", 2024-09-14T06:32:57.197 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:57.197 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:57.197 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:57.197 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:32:57.197 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045806+0000", 2024-09-14T06:32:57.197 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.2000000000000001e-07 2024-09-14T06:32:57.197 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:57.197 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:57.197 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:32:57.197 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045807+0000", 2024-09-14T06:32:57.197 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.4e-07 2024-09-14T06:32:57.198 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:57.198 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:57.198 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:32:57.198 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045810+0000", 2024-09-14T06:32:57.198 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.9229999999999998e-06 2024-09-14T06:32:57.198 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:57.198 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:57.198 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:32:57.198 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045812+0000", 2024-09-14T06:32:57.198 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.2220000000000001e-06 2024-09-14T06:32:57.198 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:57.198 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:57.198 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:32:57.199 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.078407+0000", 2024-09-14T06:32:57.199 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032594513999999998 2024-09-14T06:32:57.199 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:57.199 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:57.199 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:32:57.199 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.078417+0000", 2024-09-14T06:32:57.199 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.0348e-05 2024-09-14T06:32:57.199 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:57.199 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:32:57.199 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:57.199 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:57.199 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:57.200 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:98524 3.10 3.1bf56f90 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T06:32:57.200 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:30:07.045815+0000", 2024-09-14T06:32:57.200 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 170.073857968, 2024-09-14T06:32:57.200 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032643208999999999, 2024-09-14T06:32:57.200 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:32:57.200 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:32:57.200 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "reached pg", 2024-09-14T06:32:57.200 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:32:57.200 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:32:57.200 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:57.200 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 98524 2024-09-14T06:32:57.200 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:57.200 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:32:57.200 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:57.200 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:32:57.201 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045815+0000", 2024-09-14T06:32:57.201 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:57.201 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:57.201 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:57.201 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:32:57.201 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045815+0000", 2024-09-14T06:32:57.201 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:57.201 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:57.201 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:57.201 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:32:57.201 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045815+0000", 2024-09-14T06:32:57.201 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.11e-07 2024-09-14T06:32:57.201 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:57.201 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:57.201 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:32:57.201 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045816+0000", 2024-09-14T06:32:57.201 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.9099999999999991e-07 2024-09-14T06:32:57.202 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:57.202 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:57.202 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:32:57.202 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045819+0000", 2024-09-14T06:32:57.202 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.9399999999999998e-06 2024-09-14T06:32:57.202 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:57.202 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:57.202 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:32:57.202 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045822+0000", 2024-09-14T06:32:57.202 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.3470000000000001e-06 2024-09-14T06:32:57.202 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:57.202 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:57.202 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:32:57.202 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.078449+0000", 2024-09-14T06:32:57.202 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032627529000000002 2024-09-14T06:32:57.202 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:57.203 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:57.203 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:32:57.203 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.078458+0000", 2024-09-14T06:32:57.203 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.1910000000000006e-06 2024-09-14T06:32:57.203 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:57.203 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:32:57.203 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:57.203 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:57.203 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:32:57.203 INFO:teuthology.orchestra.run.smithi138.stdout:} 2024-09-14T06:32:57.204 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-14T06:32:57.205 INFO:tasks.ceph.osd.7.smithi138.stderr:2024-09-14T06:32:57.204+0000 7f3c6a21d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33623) UID: 0 2024-09-14T06:32:57.305 INFO:tasks.ceph.osd.7.smithi138.stderr:2024-09-14T06:32:57.304+0000 7f3c6a21d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33623) UID: 0 2024-09-14T06:32:57.348 INFO:teuthology.orchestra.run.smithi138.stdout:{ 2024-09-14T06:32:57.349 INFO:teuthology.orchestra.run.smithi138.stdout: "ops": [], 2024-09-14T06:32:57.349 INFO:teuthology.orchestra.run.smithi138.stdout: "num_ops": 0 2024-09-14T06:32:57.349 INFO:teuthology.orchestra.run.smithi138.stdout:} 2024-09-14T06:32:57.357 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-14T06:32:57.406 INFO:tasks.ceph.osd.2.smithi094.stderr:2024-09-14T06:32:57.405+0000 7faf3269b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33649) UID: 0 2024-09-14T06:32:57.502 INFO:teuthology.orchestra.run.smithi138.stdout:{ 2024-09-14T06:32:57.518 INFO:teuthology.orchestra.run.smithi138.stdout: "ops": [], 2024-09-14T06:32:57.518 INFO:teuthology.orchestra.run.smithi138.stdout: "complaint_time": 30, 2024-09-14T06:32:57.518 INFO:teuthology.orchestra.run.smithi138.stdout: "num_blocked_ops": 0 2024-09-14T06:32:57.518 INFO:teuthology.orchestra.run.smithi138.stdout:} 2024-09-14T06:32:57.519 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-14T06:32:57.520 INFO:tasks.ceph.osd.1.smithi094.stderr:2024-09-14T06:32:57.518+0000 7f328e47a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33641) UID: 0 2024-09-14T06:32:57.620 INFO:tasks.ceph.osd.0.smithi094.stderr:2024-09-14T06:32:57.619+0000 7f07c9333640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33636) UID: 0 2024-09-14T06:32:57.623 INFO:teuthology.orchestra.run.smithi138.stdout:{ 2024-09-14T06:32:57.623 INFO:teuthology.orchestra.run.smithi138.stdout: "size": 20, 2024-09-14T06:32:57.623 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 600, 2024-09-14T06:32:57.623 INFO:teuthology.orchestra.run.smithi138.stdout: "ops": [ 2024-09-14T06:32:57.624 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:57.624 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:53244 3.f 3:f71f51a7:::rbd_data.112dca2f6ae1.0000000000000015:17 [read 1863680~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:32:57.624 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:44.369399+0000", 2024-09-14T06:32:57.624 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 193.25227343500001, 2024-09-14T06:32:57.624 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047291189999999997, 2024-09-14T06:32:57.624 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:32:57.624 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:32:57.624 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:32:57.624 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:32:57.624 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:32:57.624 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:57.625 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 53244 2024-09-14T06:32:57.625 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:57.625 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:32:57.625 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:57.625 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:32:57.625 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369399+0000", 2024-09-14T06:32:57.625 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:57.625 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:57.625 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:57.625 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:32:57.625 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369399+0000", 2024-09-14T06:32:57.625 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:57.626 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:57.626 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:57.626 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:32:57.626 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369400+0000", 2024-09-14T06:32:57.626 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.0699999999999999e-07 2024-09-14T06:32:57.626 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:57.626 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:57.626 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:32:57.626 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369401+0000", 2024-09-14T06:32:57.626 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.4300000000000001e-06 2024-09-14T06:32:57.626 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:57.626 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:57.626 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:32:57.627 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369404+0000", 2024-09-14T06:32:57.627 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.061e-06 2024-09-14T06:32:57.627 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:57.627 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:57.627 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:32:57.627 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369407+0000", 2024-09-14T06:32:57.627 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.6139999999999999e-06 2024-09-14T06:32:57.627 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:57.627 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:57.627 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:32:57.627 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.416517+0000", 2024-09-14T06:32:57.627 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047110338000000002 2024-09-14T06:32:57.627 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:57.627 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:57.628 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:32:57.628 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.416616+0000", 2024-09-14T06:32:57.628 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.8651000000000001e-05 2024-09-14T06:32:57.628 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:57.628 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:57.628 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:32:57.628 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.416691+0000", 2024-09-14T06:32:57.628 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.4888999999999994e-05 2024-09-14T06:32:57.628 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:57.629 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:32:57.629 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:57.629 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:57.629 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:57.629 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:53245 3.f 3:f71f51a7:::rbd_data.112dca2f6ae1.0000000000000015:17 [read 1888256~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:32:57.629 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:44.369410+0000", 2024-09-14T06:32:57.629 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 193.25226314700001, 2024-09-14T06:32:57.629 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047399275999999997, 2024-09-14T06:32:57.629 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:32:57.629 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:32:57.629 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:32:57.630 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:32:57.630 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:32:57.630 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:57.630 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 53245 2024-09-14T06:32:57.630 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:57.630 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:32:57.630 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:57.630 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:32:57.630 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369410+0000", 2024-09-14T06:32:57.630 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:57.630 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:57.630 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:57.631 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:32:57.631 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369410+0000", 2024-09-14T06:32:57.631 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:57.631 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:57.631 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:57.631 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:32:57.631 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369410+0000", 2024-09-14T06:32:57.631 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.1899999999999999e-07 2024-09-14T06:32:57.631 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:57.631 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:57.631 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:32:57.632 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369411+0000", 2024-09-14T06:32:57.632 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.0300000000000001e-06 2024-09-14T06:32:57.632 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:57.632 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:57.632 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:32:57.632 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369414+0000", 2024-09-14T06:32:57.632 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.1099999999999999e-06 2024-09-14T06:32:57.632 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:57.632 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:57.632 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:32:57.632 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369417+0000", 2024-09-14T06:32:57.633 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.6189999999999998e-06 2024-09-14T06:32:57.633 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:57.633 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:57.633 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:32:57.633 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.416718+0000", 2024-09-14T06:32:57.633 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047301322999999999 2024-09-14T06:32:57.633 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:57.633 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:57.633 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:32:57.633 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.416759+0000", 2024-09-14T06:32:57.633 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.1009e-05 2024-09-14T06:32:57.633 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:57.634 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:57.634 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:32:57.634 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.416809+0000", 2024-09-14T06:32:57.634 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.9966000000000002e-05 2024-09-14T06:32:57.634 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:57.634 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:32:57.636 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:57.636 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:57.636 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:57.637 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:53246 3.f 3:f71f51a7:::rbd_data.112dca2f6ae1.0000000000000015:17 [read 1912832~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:32:57.637 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:44.369420+0000", 2024-09-14T06:32:57.637 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 193.25225320300001, 2024-09-14T06:32:57.637 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047529463000000001, 2024-09-14T06:32:57.637 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:32:57.637 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:32:57.637 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:32:57.637 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:32:57.637 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:32:57.637 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:57.637 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 53246 2024-09-14T06:32:57.638 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:57.638 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:32:57.638 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:57.638 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:32:57.638 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369420+0000", 2024-09-14T06:32:57.638 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:57.638 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:57.638 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:57.638 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:32:57.638 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369420+0000", 2024-09-14T06:32:57.638 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:57.638 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:57.638 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:57.638 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:32:57.639 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369420+0000", 2024-09-14T06:32:57.639 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.1299999999999999e-07 2024-09-14T06:32:57.639 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:57.639 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:57.639 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:32:57.639 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369425+0000", 2024-09-14T06:32:57.639 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.0130000000000001e-06 2024-09-14T06:32:57.639 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:57.639 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:57.639 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:32:57.639 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369428+0000", 2024-09-14T06:32:57.639 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.349e-06 2024-09-14T06:32:57.639 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:57.639 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:57.640 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:32:57.640 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369431+0000", 2024-09-14T06:32:57.640 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.7389999999999999e-06 2024-09-14T06:32:57.640 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:57.640 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:57.640 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:32:57.640 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.416834+0000", 2024-09-14T06:32:57.640 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047402669000000001 2024-09-14T06:32:57.640 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:57.640 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:57.640 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:32:57.640 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.416882+0000", 2024-09-14T06:32:57.640 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.8068999999999997e-05 2024-09-14T06:32:57.641 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:57.641 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:57.641 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:32:57.641 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.416949+0000", 2024-09-14T06:32:57.641 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.7410999999999995e-05 2024-09-14T06:32:57.641 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:57.641 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:32:57.641 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:57.641 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:57.641 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:57.641 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:53247 3.f 3:f71f51a7:::rbd_data.112dca2f6ae1.0000000000000015:17 [read 1937408~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:32:57.641 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:44.369434+0000", 2024-09-14T06:32:57.641 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 193.252238439, 2024-09-14T06:32:57.642 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047652264999999999, 2024-09-14T06:32:57.642 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:32:57.642 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:32:57.642 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:32:57.642 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:32:57.642 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:32:57.642 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:57.642 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 53247 2024-09-14T06:32:57.642 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:57.642 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:32:57.643 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:57.643 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:32:57.643 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369434+0000", 2024-09-14T06:32:57.643 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:57.643 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:57.643 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:57.643 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:32:57.643 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369434+0000", 2024-09-14T06:32:57.643 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:57.643 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:57.643 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:57.643 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:32:57.643 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369435+0000", 2024-09-14T06:32:57.643 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.2100000000000001e-07 2024-09-14T06:32:57.644 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:57.644 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:57.644 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:32:57.644 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369436+0000", 2024-09-14T06:32:57.644 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.122e-06 2024-09-14T06:32:57.644 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:57.644 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:57.644 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:32:57.644 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369439+0000", 2024-09-14T06:32:57.644 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.0400000000000001e-06 2024-09-14T06:32:57.644 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:57.644 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:57.644 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:32:57.644 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369441+0000", 2024-09-14T06:32:57.645 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.728e-06 2024-09-14T06:32:57.645 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:57.645 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:57.645 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:32:57.645 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.416977+0000", 2024-09-14T06:32:57.645 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047535344 2024-09-14T06:32:57.645 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:57.645 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:57.645 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:32:57.645 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.417029+0000", 2024-09-14T06:32:57.645 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.2037999999999999e-05 2024-09-14T06:32:57.645 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:57.645 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:57.646 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:32:57.646 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.417087+0000", 2024-09-14T06:32:57.646 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.7772000000000002e-05 2024-09-14T06:32:57.646 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:57.646 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:32:57.646 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:57.646 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:57.646 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:57.646 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:53248 3.f 3:f71f51a7:::rbd_data.112dca2f6ae1.0000000000000015:17 [read 1961984~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:32:57.646 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:44.369445+0000", 2024-09-14T06:32:57.646 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 193.252227887, 2024-09-14T06:32:57.646 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047798489, 2024-09-14T06:32:57.646 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:32:57.646 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:32:57.647 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:32:57.647 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:32:57.647 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:32:57.647 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:57.647 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 53248 2024-09-14T06:32:57.647 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:57.647 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:32:57.647 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:57.647 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:32:57.647 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369445+0000", 2024-09-14T06:32:57.648 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:57.648 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:57.648 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:57.648 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:32:57.648 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369445+0000", 2024-09-14T06:32:57.648 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:57.648 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:57.648 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:57.648 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:32:57.648 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369445+0000", 2024-09-14T06:32:57.648 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.0699999999999999e-07 2024-09-14T06:32:57.648 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:57.648 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:57.649 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:32:57.649 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369446+0000", 2024-09-14T06:32:57.649 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.1000000000000001e-06 2024-09-14T06:32:57.649 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:57.649 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:57.649 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:32:57.649 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369449+0000", 2024-09-14T06:32:57.649 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.1039999999999998e-06 2024-09-14T06:32:57.649 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:57.649 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:57.649 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:32:57.649 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369452+0000", 2024-09-14T06:32:57.649 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.6199999999999999e-06 2024-09-14T06:32:57.649 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:57.650 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:57.650 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:32:57.650 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.417114+0000", 2024-09-14T06:32:57.650 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047661901999999999 2024-09-14T06:32:57.650 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:57.650 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:57.650 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:32:57.650 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.417181+0000", 2024-09-14T06:32:57.650 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.7385999999999995e-05 2024-09-14T06:32:57.650 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:57.650 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:57.650 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:32:57.650 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.417243+0000", 2024-09-14T06:32:57.650 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.2169999999999996e-05 2024-09-14T06:32:57.651 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:57.651 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:32:57.651 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:57.651 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:57.651 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:57.651 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:53249 3.f 3:f71f51a7:::rbd_data.112dca2f6ae1.0000000000000015:17 [read 1986560~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:32:57.651 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:44.369455+0000", 2024-09-14T06:32:57.651 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 193.25221757599999, 2024-09-14T06:32:57.651 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047890493999999999, 2024-09-14T06:32:57.651 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:32:57.651 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:32:57.651 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:32:57.651 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:32:57.651 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:32:57.652 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:57.652 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 53249 2024-09-14T06:32:57.652 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:57.652 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:32:57.652 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:57.652 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:32:57.652 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369455+0000", 2024-09-14T06:32:57.652 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:57.652 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:57.652 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:57.652 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:32:57.652 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369455+0000", 2024-09-14T06:32:57.653 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:57.653 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:57.653 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:57.653 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:32:57.653 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369455+0000", 2024-09-14T06:32:57.653 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.0800000000000001e-07 2024-09-14T06:32:57.653 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:57.653 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:57.653 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:32:57.653 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369456+0000", 2024-09-14T06:32:57.653 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.0729999999999999e-06 2024-09-14T06:32:57.653 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:57.653 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:57.654 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:32:57.654 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369460+0000", 2024-09-14T06:32:57.654 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.027e-06 2024-09-14T06:32:57.654 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:57.654 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:57.654 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:32:57.654 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369462+0000", 2024-09-14T06:32:57.654 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.5320000000000001e-06 2024-09-14T06:32:57.654 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:57.654 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:57.654 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:32:57.654 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.417257+0000", 2024-09-14T06:32:57.654 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.04779489 2024-09-14T06:32:57.654 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:57.655 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:57.655 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:32:57.655 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.417296+0000", 2024-09-14T06:32:57.655 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.8658000000000002e-05 2024-09-14T06:32:57.655 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:57.655 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:57.655 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:32:57.655 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.417346+0000", 2024-09-14T06:32:57.655 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.0105999999999998e-05 2024-09-14T06:32:57.655 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:57.655 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:32:57.655 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:57.655 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:57.655 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:57.656 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:53250 3.f 3:f71f51a7:::rbd_data.112dca2f6ae1.0000000000000015:17 [read 2011136~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:32:57.656 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:44.369465+0000", 2024-09-14T06:32:57.656 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 193.25220760600001, 2024-09-14T06:32:57.656 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.048032230000000002, 2024-09-14T06:32:57.656 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:32:57.656 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:32:57.656 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:32:57.656 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:32:57.656 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:32:57.656 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:57.656 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 53250 2024-09-14T06:32:57.656 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:57.656 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:32:57.657 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:57.657 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:32:57.657 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369465+0000", 2024-09-14T06:32:57.657 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:57.657 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:57.657 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:57.657 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:32:57.657 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369465+0000", 2024-09-14T06:32:57.657 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:57.657 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:57.657 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:57.657 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:32:57.657 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369465+0000", 2024-09-14T06:32:57.658 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.2100000000000001e-07 2024-09-14T06:32:57.658 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:57.658 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:57.658 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:32:57.658 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369466+0000", 2024-09-14T06:32:57.658 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.0640000000000001e-06 2024-09-14T06:32:57.658 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:57.658 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:57.658 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:32:57.658 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369469+0000", 2024-09-14T06:32:57.658 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.0170000000000001e-06 2024-09-14T06:32:57.658 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:57.658 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:57.658 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:32:57.659 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369472+0000", 2024-09-14T06:32:57.659 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.6189999999999998e-06 2024-09-14T06:32:57.659 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:57.659 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:57.659 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:32:57.659 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.417383+0000", 2024-09-14T06:32:57.659 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047911092000000002 2024-09-14T06:32:57.659 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:57.659 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:57.659 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:32:57.659 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.417430+0000", 2024-09-14T06:32:57.659 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.6898999999999999e-05 2024-09-14T06:32:57.659 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:57.659 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:57.659 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:32:57.660 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.417497+0000", 2024-09-14T06:32:57.660 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.7317999999999995e-05 2024-09-14T06:32:57.660 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:57.660 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:32:57.660 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:57.660 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:57.660 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:57.660 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:53251 3.f 3:f71f51a7:::rbd_data.112dca2f6ae1.0000000000000015:17 [read 2035712~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:32:57.660 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:44.369475+0000", 2024-09-14T06:32:57.660 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 193.25219761, 2024-09-14T06:32:57.660 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.048154542000000002, 2024-09-14T06:32:57.660 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:32:57.660 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:32:57.661 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:32:57.661 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:32:57.661 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:32:57.661 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:57.661 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 53251 2024-09-14T06:32:57.661 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:57.661 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:32:57.661 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:57.661 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:32:57.661 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369475+0000", 2024-09-14T06:32:57.661 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:57.661 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:57.661 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:57.662 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:32:57.662 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369475+0000", 2024-09-14T06:32:57.662 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:57.662 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:57.662 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:57.662 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:32:57.662 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369475+0000", 2024-09-14T06:32:57.662 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.0100000000000001e-07 2024-09-14T06:32:57.662 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:57.662 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:57.662 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:32:57.662 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369476+0000", 2024-09-14T06:32:57.662 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.0580000000000001e-06 2024-09-14T06:32:57.663 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:57.663 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:57.663 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:32:57.663 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369479+0000", 2024-09-14T06:32:57.663 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.9950000000000001e-06 2024-09-14T06:32:57.663 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:57.663 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:57.663 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:32:57.663 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369482+0000", 2024-09-14T06:32:57.663 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.6340000000000001e-06 2024-09-14T06:32:57.663 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:57.663 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:57.663 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:32:57.663 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.417530+0000", 2024-09-14T06:32:57.664 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.048047535000000002 2024-09-14T06:32:57.664 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:57.664 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:57.664 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:32:57.664 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.417574+0000", 2024-09-14T06:32:57.664 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.4255999999999998e-05 2024-09-14T06:32:57.664 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:57.664 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:57.664 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:32:57.664 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.417630+0000", 2024-09-14T06:32:57.664 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.5862999999999997e-05 2024-09-14T06:32:57.664 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:57.664 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:32:57.665 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:57.665 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:57.665 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:57.665 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:53252 3.f 3:f71f51a7:::rbd_data.112dca2f6ae1.0000000000000015:17 [read 2060288~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:32:57.665 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:44.369485+0000", 2024-09-14T06:32:57.665 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 193.25218776200001, 2024-09-14T06:32:57.665 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.048254116, 2024-09-14T06:32:57.665 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:32:57.665 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:32:57.665 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:32:57.665 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:32:57.665 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:32:57.665 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:57.666 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 53252 2024-09-14T06:32:57.666 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:57.666 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:32:57.666 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:57.666 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:32:57.666 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369485+0000", 2024-09-14T06:32:57.666 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:57.666 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:57.666 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:57.666 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:32:57.666 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369485+0000", 2024-09-14T06:32:57.666 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:57.666 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:57.666 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:57.667 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:32:57.667 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369485+0000", 2024-09-14T06:32:57.667 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.1e-07 2024-09-14T06:32:57.667 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:57.667 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:57.667 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:32:57.667 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369487+0000", 2024-09-14T06:32:57.667 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.42e-06 2024-09-14T06:32:57.667 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:57.667 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:57.667 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:32:57.667 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369490+0000", 2024-09-14T06:32:57.667 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.9639999999999999e-06 2024-09-14T06:32:57.668 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:57.668 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:57.668 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:32:57.668 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369492+0000", 2024-09-14T06:32:57.668 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.5679999999999998e-06 2024-09-14T06:32:57.668 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:57.668 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:57.668 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:32:57.668 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.417645+0000", 2024-09-14T06:32:57.668 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.048152753 2024-09-14T06:32:57.668 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:57.668 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:57.668 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:32:57.668 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.417687+0000", 2024-09-14T06:32:57.669 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.1900999999999997e-05 2024-09-14T06:32:57.669 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:57.669 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:57.669 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:32:57.669 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.417739+0000", 2024-09-14T06:32:57.669 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.2299999999999997e-05 2024-09-14T06:32:57.669 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:57.669 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:32:57.669 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:57.669 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:57.669 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:57.669 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:53253 3.f 3:f71f51a7:::rbd_data.112dca2f6ae1.0000000000000015:17 [read 2084864~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:32:57.670 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:44.369495+0000", 2024-09-14T06:32:57.670 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 193.25217763800001, 2024-09-14T06:32:57.670 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.048366785000000002, 2024-09-14T06:32:57.670 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:32:57.670 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:32:57.670 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:32:57.670 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:32:57.670 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:32:57.670 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:57.670 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 53253 2024-09-14T06:32:57.670 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:57.670 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:32:57.670 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:57.670 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:32:57.671 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369495+0000", 2024-09-14T06:32:57.671 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:57.671 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:57.671 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:57.671 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:32:57.671 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369495+0000", 2024-09-14T06:32:57.671 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:57.671 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:57.671 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:57.671 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:32:57.671 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370388+0000", 2024-09-14T06:32:57.671 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00089287999999999998 2024-09-14T06:32:57.671 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:57.671 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:57.672 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:32:57.672 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370392+0000", 2024-09-14T06:32:57.672 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.2130000000000002e-06 2024-09-14T06:32:57.672 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:57.672 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:57.672 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:32:57.672 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370401+0000", 2024-09-14T06:32:57.672 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.7649999999999999e-06 2024-09-14T06:32:57.672 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:57.672 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:57.672 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:32:57.672 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370406+0000", 2024-09-14T06:32:57.672 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.0159999999999999e-06 2024-09-14T06:32:57.673 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:57.673 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:57.673 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:32:57.673 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.417764+0000", 2024-09-14T06:32:57.673 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047358348000000001 2024-09-14T06:32:57.673 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:57.673 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:57.673 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:32:57.673 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.417806+0000", 2024-09-14T06:32:57.673 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.1149999999999997e-05 2024-09-14T06:32:57.673 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:57.673 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:57.673 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:32:57.673 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.417862+0000", 2024-09-14T06:32:57.674 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.6412999999999997e-05 2024-09-14T06:32:57.674 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:57.674 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:32:57.674 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:57.674 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:57.674 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:57.674 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:53340 3.f 3:f71f51a7:::rbd_data.112dca2f6ae1.0000000000000015:17 [read 1056768~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:32:57.674 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:44.370421+0000", 2024-09-14T06:32:57.674 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 193.251251758, 2024-09-14T06:32:57.674 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047566313999999998, 2024-09-14T06:32:57.674 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:32:57.674 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:32:57.674 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:32:57.674 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:32:57.675 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:32:57.675 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:57.675 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 53340 2024-09-14T06:32:57.675 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:57.675 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:32:57.675 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:57.675 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:32:57.675 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370421+0000", 2024-09-14T06:32:57.675 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:57.675 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:57.675 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:57.675 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:32:57.675 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370421+0000", 2024-09-14T06:32:57.675 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:57.676 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:57.676 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:57.676 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:32:57.676 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370422+0000", 2024-09-14T06:32:57.676 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.3799999999999996e-07 2024-09-14T06:32:57.676 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:57.676 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:57.676 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:32:57.676 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370424+0000", 2024-09-14T06:32:57.676 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.6919999999999998e-06 2024-09-14T06:32:57.676 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:57.676 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:57.676 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:32:57.676 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370432+0000", 2024-09-14T06:32:57.677 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.2439999999999996e-06 2024-09-14T06:32:57.677 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:57.677 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:57.677 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:32:57.677 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370437+0000", 2024-09-14T06:32:57.677 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.5389999999999998e-06 2024-09-14T06:32:57.677 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:57.677 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:57.677 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:32:57.677 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.417888+0000", 2024-09-14T06:32:57.677 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047450432000000001 2024-09-14T06:32:57.677 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:57.677 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:57.678 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:32:57.678 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.417935+0000", 2024-09-14T06:32:57.678 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.7454999999999999e-05 2024-09-14T06:32:57.678 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:57.678 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:57.678 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:32:57.678 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.417987+0000", 2024-09-14T06:32:57.678 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.2213999999999998e-05 2024-09-14T06:32:57.678 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:57.678 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:32:57.678 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:57.678 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:57.678 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:57.679 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:53341 3.f 3:f71f51a7:::rbd_data.112dca2f6ae1.0000000000000015:17 [read 1081344~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:32:57.679 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:44.370446+0000", 2024-09-14T06:32:57.679 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 193.25122648499999, 2024-09-14T06:32:57.679 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047665723, 2024-09-14T06:32:57.679 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:32:57.679 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:32:57.679 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:32:57.679 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:32:57.679 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:32:57.679 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:57.679 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 53341 2024-09-14T06:32:57.679 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:57.679 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:32:57.679 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:57.680 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:32:57.680 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370446+0000", 2024-09-14T06:32:57.680 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:57.680 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:57.680 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:57.680 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:32:57.680 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370446+0000", 2024-09-14T06:32:57.680 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:57.680 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:57.680 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:57.680 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:32:57.681 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370447+0000", 2024-09-14T06:32:57.681 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.6100000000000002e-07 2024-09-14T06:32:57.681 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:57.681 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:57.681 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:32:57.681 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370448+0000", 2024-09-14T06:32:57.681 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.7370000000000001e-06 2024-09-14T06:32:57.681 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:57.681 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:57.681 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:32:57.681 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370462+0000", 2024-09-14T06:32:57.681 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.3227000000000001e-05 2024-09-14T06:32:57.682 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:57.682 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:57.682 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:32:57.682 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370467+0000", 2024-09-14T06:32:57.682 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.6540000000000004e-06 2024-09-14T06:32:57.682 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:57.682 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:57.682 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:32:57.682 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.418013+0000", 2024-09-14T06:32:57.682 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047545325999999999 2024-09-14T06:32:57.682 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:57.682 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:57.682 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:32:57.682 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.418054+0000", 2024-09-14T06:32:57.683 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.1696999999999997e-05 2024-09-14T06:32:57.683 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:57.683 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:57.683 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:32:57.683 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.418112+0000", 2024-09-14T06:32:57.683 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.7720999999999998e-05 2024-09-14T06:32:57.683 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:57.683 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:32:57.683 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:57.683 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:57.683 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:57.683 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:53342 3.f 3:f71f51a7:::rbd_data.112dca2f6ae1.0000000000000015:17 [read 1105920~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:32:57.683 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:44.370475+0000", 2024-09-14T06:32:57.683 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 193.251197424, 2024-09-14T06:32:57.684 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047748848000000003, 2024-09-14T06:32:57.684 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:32:57.684 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:32:57.684 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:32:57.684 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:32:57.684 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:32:57.684 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:57.684 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 53342 2024-09-14T06:32:57.684 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:57.684 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:32:57.684 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:57.684 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:32:57.684 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370475+0000", 2024-09-14T06:32:57.685 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:57.685 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:57.685 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:57.685 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:32:57.685 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370475+0000", 2024-09-14T06:32:57.685 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:57.685 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:57.685 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:57.685 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:32:57.685 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370476+0000", 2024-09-14T06:32:57.685 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.4700000000000002e-07 2024-09-14T06:32:57.685 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:57.685 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:57.686 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:32:57.686 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370478+0000", 2024-09-14T06:32:57.686 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.5890000000000001e-06 2024-09-14T06:32:57.686 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:57.686 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:57.686 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:32:57.686 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370486+0000", 2024-09-14T06:32:57.686 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.3470000000000001e-06 2024-09-14T06:32:57.686 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:57.686 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:57.686 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:32:57.686 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370491+0000", 2024-09-14T06:32:57.686 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.5720000000000002e-06 2024-09-14T06:32:57.687 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:57.687 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:57.687 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:32:57.687 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.418135+0000", 2024-09-14T06:32:57.687 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047644094999999997 2024-09-14T06:32:57.687 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:57.687 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:57.687 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:32:57.687 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.418174+0000", 2024-09-14T06:32:57.687 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.8204000000000002e-05 2024-09-14T06:32:57.687 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:57.687 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:57.687 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:32:57.688 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.418224+0000", 2024-09-14T06:32:57.688 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.0593999999999997e-05 2024-09-14T06:32:57.688 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:57.688 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:32:57.688 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:57.688 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:57.688 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:57.688 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:53343 3.f 3:f71f51a7:::rbd_data.112dca2f6ae1.0000000000000015:17 [read 1130496~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:32:57.688 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:44.370498+0000", 2024-09-14T06:32:57.688 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 193.25117454799999, 2024-09-14T06:32:57.688 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047841574999999997, 2024-09-14T06:32:57.688 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:32:57.689 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:32:57.689 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:32:57.689 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:32:57.689 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:32:57.689 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:57.689 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 53343 2024-09-14T06:32:57.689 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:57.689 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:32:57.689 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:57.689 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:32:57.689 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370498+0000", 2024-09-14T06:32:57.689 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:57.689 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:57.689 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:57.690 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:32:57.690 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370498+0000", 2024-09-14T06:32:57.690 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:57.690 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:57.690 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:57.690 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:32:57.690 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370499+0000", 2024-09-14T06:32:57.690 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.9999999999999998e-07 2024-09-14T06:32:57.690 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:57.690 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:57.690 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:32:57.690 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370501+0000", 2024-09-14T06:32:57.691 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.0310000000000001e-06 2024-09-14T06:32:57.691 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:57.691 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:57.691 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:32:57.691 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370507+0000", 2024-09-14T06:32:57.691 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.4169999999999997e-06 2024-09-14T06:32:57.691 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:57.691 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:57.691 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:32:57.691 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370516+0000", 2024-09-14T06:32:57.691 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.7190000000000007e-06 2024-09-14T06:32:57.691 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:57.691 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:57.691 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:32:57.692 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.418247+0000", 2024-09-14T06:32:57.692 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047731190999999999 2024-09-14T06:32:57.692 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:57.692 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:57.692 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:32:57.692 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.418286+0000", 2024-09-14T06:32:57.692 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.8988e-05 2024-09-14T06:32:57.692 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:57.692 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:57.692 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:32:57.692 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.418340+0000", 2024-09-14T06:32:57.692 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.3829000000000002e-05 2024-09-14T06:32:57.692 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:57.692 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:32:57.692 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:57.693 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:57.693 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:57.693 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:53344 3.f 3:f71f51a7:::rbd_data.112dca2f6ae1.0000000000000015:17 [read 1155072~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:32:57.693 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:44.370524+0000", 2024-09-14T06:32:57.693 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 193.25114877799999, 2024-09-14T06:32:57.693 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047972262000000002, 2024-09-14T06:32:57.693 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:32:57.693 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:32:57.693 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:32:57.693 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:32:57.693 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:32:57.693 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:57.693 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 53344 2024-09-14T06:32:57.693 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:57.693 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:32:57.694 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:57.694 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:32:57.694 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370524+0000", 2024-09-14T06:32:57.694 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:57.694 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:57.694 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:57.694 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:32:57.694 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370524+0000", 2024-09-14T06:32:57.694 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:57.694 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:57.694 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:57.694 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:32:57.694 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370618+0000", 2024-09-14T06:32:57.695 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.4425999999999993e-05 2024-09-14T06:32:57.695 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:57.695 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:57.695 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:32:57.695 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370622+0000", 2024-09-14T06:32:57.695 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.3519999999999998e-06 2024-09-14T06:32:57.695 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:57.695 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:57.695 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:32:57.695 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370631+0000", 2024-09-14T06:32:57.695 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.9299999999999992e-06 2024-09-14T06:32:57.695 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:57.695 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:57.695 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:32:57.696 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370637+0000", 2024-09-14T06:32:57.696 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.3269999999999997e-06 2024-09-14T06:32:57.696 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:57.696 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:57.696 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:32:57.696 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.418372+0000", 2024-09-14T06:32:57.696 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047734764999999998 2024-09-14T06:32:57.696 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:57.696 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:57.696 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:32:57.696 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.418431+0000", 2024-09-14T06:32:57.696 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.8696999999999997e-05 2024-09-14T06:32:57.696 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:57.696 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:57.697 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:32:57.697 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.418496+0000", 2024-09-14T06:32:57.697 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.5765000000000005e-05 2024-09-14T06:32:57.697 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:57.697 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:32:57.697 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:57.697 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:57.697 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:57.697 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:53345 3.f 3:f71f51a7:::rbd_data.112dca2f6ae1.0000000000000015:17 [read 1179648~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:32:57.697 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:44.370647+0000", 2024-09-14T06:32:57.697 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 193.25102531100001, 2024-09-14T06:32:57.697 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047953775999999997, 2024-09-14T06:32:57.697 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:32:57.698 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:32:57.698 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:32:57.698 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:32:57.698 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:32:57.698 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:57.698 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 53345 2024-09-14T06:32:57.698 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:57.698 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:32:57.698 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:57.698 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:32:57.698 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370647+0000", 2024-09-14T06:32:57.698 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:57.698 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:57.699 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:57.699 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:32:57.699 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370647+0000", 2024-09-14T06:32:57.699 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:57.699 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:57.699 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:57.699 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:32:57.699 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.372346+0000", 2024-09-14T06:32:57.699 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.001698284 2024-09-14T06:32:57.699 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:57.699 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:57.699 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:32:57.699 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.372351+0000", 2024-09-14T06:32:57.700 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.8890000000000001e-06 2024-09-14T06:32:57.700 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:57.700 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:57.700 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:32:57.700 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.372362+0000", 2024-09-14T06:32:57.700 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.0941000000000001e-05 2024-09-14T06:32:57.700 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:57.700 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:57.700 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:32:57.700 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.372369+0000", 2024-09-14T06:32:57.700 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.362e-06 2024-09-14T06:32:57.700 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:57.700 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:57.701 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:32:57.701 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.418510+0000", 2024-09-14T06:32:57.701 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046140914999999998 2024-09-14T06:32:57.701 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:57.701 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:57.701 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:32:57.701 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.418549+0000", 2024-09-14T06:32:57.701 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.8717999999999998e-05 2024-09-14T06:32:57.701 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:57.701 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:57.701 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:32:57.701 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.418601+0000", 2024-09-14T06:32:57.701 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.2667000000000002e-05 2024-09-14T06:32:57.702 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:57.702 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:32:57.702 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:57.702 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:57.702 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:57.702 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:53353 3.f 3:f71f51a7:::rbd_data.112dca2f6ae1.0000000000000015:17 [read 1376256~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:32:57.702 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:44.372545+0000", 2024-09-14T06:32:57.702 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 193.24912823599999, 2024-09-14T06:32:57.702 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047364232999999999, 2024-09-14T06:32:57.702 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:32:57.702 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:32:57.702 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:32:57.702 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:32:57.702 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:32:57.703 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:57.703 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 53353 2024-09-14T06:32:57.703 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:57.703 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:32:57.703 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:57.703 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:32:57.703 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.372545+0000", 2024-09-14T06:32:57.703 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:57.703 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:57.703 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:57.703 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:32:57.703 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.372545+0000", 2024-09-14T06:32:57.703 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:57.703 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:57.704 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:57.704 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:32:57.704 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.372545+0000", 2024-09-14T06:32:57.704 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.27e-07 2024-09-14T06:32:57.704 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:57.704 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:57.704 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:32:57.704 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.372547+0000", 2024-09-14T06:32:57.704 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.0600000000000002e-06 2024-09-14T06:32:57.704 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:57.704 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:57.704 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:32:57.704 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.372553+0000", 2024-09-14T06:32:57.704 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.7980000000000002e-06 2024-09-14T06:32:57.705 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:57.705 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:57.705 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:32:57.705 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.372557+0000", 2024-09-14T06:32:57.705 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.7389999999999999e-06 2024-09-14T06:32:57.705 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:57.705 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:57.705 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:32:57.705 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.419811+0000", 2024-09-14T06:32:57.705 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047253310999999999 2024-09-14T06:32:57.705 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:57.705 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:57.705 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:32:57.706 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.419854+0000", 2024-09-14T06:32:57.706 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.3448000000000001e-05 2024-09-14T06:32:57.706 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:57.706 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:57.706 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:32:57.706 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.419909+0000", 2024-09-14T06:32:57.706 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.4549999999999998e-05 2024-09-14T06:32:57.706 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:57.706 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:32:57.706 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:57.706 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:57.706 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:57.706 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:53354 3.f 3:f71f51a7:::rbd_data.112dca2f6ae1.0000000000000015:17 [read 1400832~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:32:57.706 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:44.372563+0000", 2024-09-14T06:32:57.707 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 193.249109799, 2024-09-14T06:32:57.707 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047482233999999998, 2024-09-14T06:32:57.707 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:32:57.707 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:32:57.707 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:32:57.707 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:32:57.707 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:32:57.707 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:57.707 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 53354 2024-09-14T06:32:57.707 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:57.707 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:32:57.707 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:57.707 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:32:57.707 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.372563+0000", 2024-09-14T06:32:57.707 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:57.708 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:57.708 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:57.708 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:32:57.708 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.372563+0000", 2024-09-14T06:32:57.708 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:57.708 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:57.708 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:57.708 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:32:57.708 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.375009+0000", 2024-09-14T06:32:57.708 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.0024457680000000001 2024-09-14T06:32:57.708 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:57.708 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:57.708 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:32:57.709 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.375014+0000", 2024-09-14T06:32:57.709 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.0599999999999998e-06 2024-09-14T06:32:57.709 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:57.709 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:57.709 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:32:57.709 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.375026+0000", 2024-09-14T06:32:57.709 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.1783999999999999e-05 2024-09-14T06:32:57.709 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:57.709 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:57.709 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:32:57.709 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.375035+0000", 2024-09-14T06:32:57.709 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.7799999999999995e-06 2024-09-14T06:32:57.709 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:57.710 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:57.710 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:32:57.710 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.419942+0000", 2024-09-14T06:32:57.710 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.044906982999999998 2024-09-14T06:32:57.710 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:57.710 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:57.710 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:32:57.710 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.419989+0000", 2024-09-14T06:32:57.710 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.7104999999999997e-05 2024-09-14T06:32:57.710 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:57.710 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:57.710 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:32:57.710 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.420045+0000", 2024-09-14T06:32:57.710 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.5754000000000002e-05 2024-09-14T06:32:57.711 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:57.711 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:32:57.711 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:57.711 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:57.711 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:57.711 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:225577 3.15 3:ac159a70:::rbd_header.112daa9ec306:head [watch reconnect cookie 140616827151664 gen 4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e188)", 2024-09-14T06:32:57.711 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:31:52.733493+0000", 2024-09-14T06:32:57.711 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 64.888179335000004, 2024-09-14T06:32:57.711 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.0156593789999999, 2024-09-14T06:32:57.711 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:32:57.711 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:32:57.711 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:32:57.711 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:32:57.711 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:32:57.712 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:57.712 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 225577 2024-09-14T06:32:57.712 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:57.712 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:32:57.712 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:57.712 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:32:57.712 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:31:52.733493+0000", 2024-09-14T06:32:57.712 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:57.712 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:57.712 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:57.712 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:32:57.712 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:31:52.733493+0000", 2024-09-14T06:32:57.712 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:57.713 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:57.713 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:57.713 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:32:57.713 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:31:52.733495+0000", 2024-09-14T06:32:57.713 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.0059999999999999e-06 2024-09-14T06:32:57.713 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:57.713 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:57.713 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:32:57.713 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:31:52.733504+0000", 2024-09-14T06:32:57.713 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.5490000000000006e-06 2024-09-14T06:32:57.713 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:57.713 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:57.713 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:32:57.713 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:31:52.733522+0000", 2024-09-14T06:32:57.714 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.7660999999999999e-05 2024-09-14T06:32:57.714 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:57.714 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:57.714 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:32:57.714 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:31:52.733531+0000", 2024-09-14T06:32:57.714 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.7180000000000008e-06 2024-09-14T06:32:57.714 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:57.714 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:57.714 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:32:57.714 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:31:52.733598+0000", 2024-09-14T06:32:57.714 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.6490999999999997e-05 2024-09-14T06:32:57.714 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:57.714 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:57.714 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "op must wait for map", 2024-09-14T06:32:57.715 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:31:52.733605+0000", 2024-09-14T06:32:57.715 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.6109999999999996e-06 2024-09-14T06:32:57.715 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:57.715 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:57.715 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:32:57.715 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:31:52.737908+0000", 2024-09-14T06:32:57.715 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.0043023050000000002 2024-09-14T06:32:57.715 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:57.715 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:57.715 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "waiting for peered", 2024-09-14T06:32:57.715 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:31:52.737923+0000", 2024-09-14T06:32:57.715 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.488e-05 2024-09-14T06:32:57.715 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:57.716 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:57.716 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:32:57.716 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:31:53.746776+0000", 2024-09-14T06:32:57.716 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.0088535359999999 2024-09-14T06:32:57.716 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:57.716 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:57.716 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:32:57.716 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:31:53.746878+0000", 2024-09-14T06:32:57.716 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.000101788 2024-09-14T06:32:57.716 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:57.716 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:57.716 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:32:57.716 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:31:53.749153+0000", 2024-09-14T06:32:57.717 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.0022748339999999999 2024-09-14T06:32:57.717 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:57.717 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:32:57.717 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:57.717 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:57.717 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:57.717 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:225581 3.15 3:ab407115:::rbd_object_map.112d838b9898:head [call lock.unlock in=10b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e188)", 2024-09-14T06:32:57.717 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:31:52.743472+0000", 2024-09-14T06:32:57.717 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 64.878200296000003, 2024-09-14T06:32:57.717 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.007270501, 2024-09-14T06:32:57.717 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:32:57.717 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:32:57.717 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:32:57.717 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:32:57.718 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:32:57.718 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:57.718 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 225581 2024-09-14T06:32:57.718 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:57.718 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:32:57.718 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:57.718 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:32:57.718 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:31:52.743472+0000", 2024-09-14T06:32:57.718 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:57.718 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:57.718 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:57.718 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:32:57.718 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:31:52.743472+0000", 2024-09-14T06:32:57.718 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:32:57.719 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:57.719 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:57.719 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:32:57.719 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:31:52.743473+0000", 2024-09-14T06:32:57.719 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.1999999999999998e-07 2024-09-14T06:32:57.719 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:57.719 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:57.719 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:32:57.719 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:31:52.743478+0000", 2024-09-14T06:32:57.719 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.515e-06 2024-09-14T06:32:57.719 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:57.719 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:57.719 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:32:57.719 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:31:52.743489+0000", 2024-09-14T06:32:57.720 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.1411e-05 2024-09-14T06:32:57.720 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:57.720 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:57.720 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:32:57.720 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:31:52.743494+0000", 2024-09-14T06:32:57.720 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.1649999999999997e-06 2024-09-14T06:32:57.720 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:57.720 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:57.720 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:32:57.720 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:31:52.743548+0000", 2024-09-14T06:32:57.720 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.3758999999999998e-05 2024-09-14T06:32:57.720 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:57.720 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:57.720 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "waiting for peered", 2024-09-14T06:32:57.721 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:31:52.743558+0000", 2024-09-14T06:32:57.721 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.0336999999999999e-05 2024-09-14T06:32:57.721 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:57.721 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:57.721 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:32:57.721 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:31:53.747716+0000", 2024-09-14T06:32:57.721 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.004157035 2024-09-14T06:32:57.721 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:57.721 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:57.721 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:32:57.721 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:31:53.747831+0000", 2024-09-14T06:32:57.721 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.000115394 2024-09-14T06:32:57.721 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:57.721 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:57.722 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "waiting for subops from 3", 2024-09-14T06:32:57.722 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:31:53.748002+0000", 2024-09-14T06:32:57.722 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00017085099999999999 2024-09-14T06:32:57.722 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:57.722 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:57.722 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:32:57.722 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:31:53.749536+0000", 2024-09-14T06:32:57.722 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.0015339990000000001 2024-09-14T06:32:57.722 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:57.722 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:57.722 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "op_commit", 2024-09-14T06:32:57.722 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:31:53.750672+0000", 2024-09-14T06:32:57.722 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.001135973 2024-09-14T06:32:57.722 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:57.723 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:57.723 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "commit_sent", 2024-09-14T06:32:57.723 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:31:53.750704+0000", 2024-09-14T06:32:57.723 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.1791000000000001e-05 2024-09-14T06:32:57.723 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:32:57.723 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:32:57.723 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:32:57.723 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:31:53.750743+0000", 2024-09-14T06:32:57.723 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.9450999999999999e-05 2024-09-14T06:32:57.723 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:57.723 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:32:57.723 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:57.723 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:32:57.723 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:32:57.724 INFO:teuthology.orchestra.run.smithi138.stdout:} 2024-09-14T06:32:57.725 INFO:teuthology.orchestra.run.smithi094.stderr:nodeep-scrub is unset 2024-09-14T06:32:57.726 INFO:tasks.ceph.osd.4.smithi138.stderr:2024-09-14T06:32:57.724+0000 7f6c653c8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33615) UID: 0 2024-09-14T06:32:57.726 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-14T06:32:57.826 INFO:tasks.ceph.osd.0.smithi094.stderr:2024-09-14T06:32:57.825+0000 7f07c9333640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33636) UID: 0 2024-09-14T06:32:57.884 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-14T06:32:57.884 INFO:teuthology.orchestra.run.smithi094.stdout: "ops": [], 2024-09-14T06:32:57.884 INFO:teuthology.orchestra.run.smithi094.stdout: "num_ops": 0 2024-09-14T06:32:57.884 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-14T06:32:57.892 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-14T06:32:57.933 INFO:tasks.ceph.osd.0.smithi094.stderr:2024-09-14T06:32:57.932+0000 7f07c9333640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33636) UID: 0 2024-09-14T06:32:58.028 INFO:tasks.ceph.osd.2.smithi094.stderr:2024-09-14T06:32:58.027+0000 7faf3269b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33649) UID: 0 2024-09-14T06:32:58.034 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-14T06:32:58.034 INFO:teuthology.orchestra.run.smithi094.stdout: "ops": [], 2024-09-14T06:32:58.034 INFO:teuthology.orchestra.run.smithi094.stdout: "complaint_time": 30, 2024-09-14T06:32:58.034 INFO:teuthology.orchestra.run.smithi094.stdout: "num_blocked_ops": 0 2024-09-14T06:32:58.034 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-14T06:32:58.042 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-14T06:32:58.129 INFO:tasks.ceph.osd.1.smithi094.stderr:2024-09-14T06:32:58.128+0000 7f328e47a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33641) UID: 0 2024-09-14T06:32:58.187 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-14T06:32:58.187 INFO:teuthology.orchestra.run.smithi094.stdout: "size": 20, 2024-09-14T06:32:58.187 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 600, 2024-09-14T06:32:58.188 INFO:teuthology.orchestra.run.smithi094.stdout: "ops": [ 2024-09-14T06:32:58.188 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:58.188 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:115193 3.1 3:856be886:::rbd_data.112d4d24c558.0000000000000004:29 [read 1523712~16384 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e96)", 2024-09-14T06:32:58.188 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:30:17.939854+0000", 2024-09-14T06:32:58.188 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 160.24584322800001, 2024-09-14T06:32:58.188 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.073897516999999996, 2024-09-14T06:32:58.188 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:32:58.188 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:32:58.188 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:32:58.188 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:32:58.188 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:32:58.188 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:58.188 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 115193 2024-09-14T06:32:58.189 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:58.189 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:32:58.189 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:58.189 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:32:58.189 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.939854+0000", 2024-09-14T06:32:58.189 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:58.189 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:58.189 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:58.189 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:32:58.189 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.939854+0000", 2024-09-14T06:32:58.189 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:58.189 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:58.189 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:58.189 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:32:58.190 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.939854+0000", 2024-09-14T06:32:58.190 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.9500000000000002e-07 2024-09-14T06:32:58.190 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:58.190 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:58.190 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:32:58.190 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.939856+0000", 2024-09-14T06:32:58.190 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.0959999999999999e-06 2024-09-14T06:32:58.190 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:58.190 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:58.190 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:32:58.190 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.939863+0000", 2024-09-14T06:32:58.190 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.1129999999999997e-06 2024-09-14T06:32:58.191 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:58.191 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:58.191 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:32:58.191 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.939867+0000", 2024-09-14T06:32:58.191 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.8350000000000003e-06 2024-09-14T06:32:58.191 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:58.191 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:58.191 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:32:58.191 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.013626+0000", 2024-09-14T06:32:58.191 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.073758799999999999 2024-09-14T06:32:58.191 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:58.191 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:58.191 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:32:58.192 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.013690+0000", 2024-09-14T06:32:58.192 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.3918999999999996e-05 2024-09-14T06:32:58.192 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:58.192 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:58.192 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:32:58.192 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.013751+0000", 2024-09-14T06:32:58.192 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.1159000000000002e-05 2024-09-14T06:32:58.192 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:58.192 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:32:58.192 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:58.192 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:58.192 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:58.192 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:115194 3.1 3:856be886:::rbd_data.112d4d24c558.0000000000000004:29 [read 1769472~16384 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e96)", 2024-09-14T06:32:58.193 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:30:17.939874+0000", 2024-09-14T06:32:58.193 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 160.245822917, 2024-09-14T06:32:58.193 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.074016988000000006, 2024-09-14T06:32:58.193 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:32:58.193 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:32:58.193 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:32:58.193 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:32:58.193 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:32:58.193 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:58.193 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 115194 2024-09-14T06:32:58.193 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:58.193 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:32:58.193 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:58.194 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:32:58.194 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.939874+0000", 2024-09-14T06:32:58.194 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:58.194 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:58.194 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:58.194 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:32:58.194 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.939874+0000", 2024-09-14T06:32:58.194 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:58.194 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:58.194 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:58.194 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:32:58.194 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.939875+0000", 2024-09-14T06:32:58.194 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.2600000000000002e-07 2024-09-14T06:32:58.194 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:58.195 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:58.195 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:32:58.195 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.939877+0000", 2024-09-14T06:32:58.195 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.9420000000000002e-06 2024-09-14T06:32:58.195 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:58.195 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:58.195 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:32:58.195 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.939882+0000", 2024-09-14T06:32:58.195 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.8749999999999997e-06 2024-09-14T06:32:58.195 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:58.195 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:58.195 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:32:58.195 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.939887+0000", 2024-09-14T06:32:58.196 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.741e-06 2024-09-14T06:32:58.196 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:58.196 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:58.196 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:32:58.196 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.013767+0000", 2024-09-14T06:32:58.196 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.073879720999999995 2024-09-14T06:32:58.196 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:58.196 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:58.196 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:32:58.196 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.013832+0000", 2024-09-14T06:32:58.196 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.5312000000000001e-05 2024-09-14T06:32:58.196 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:58.196 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:58.197 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:32:58.197 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.013891+0000", 2024-09-14T06:32:58.197 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.8870999999999999e-05 2024-09-14T06:32:58.197 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:58.197 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:32:58.197 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:58.197 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:58.197 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:58.197 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:115195 3.1 3:856be886:::rbd_data.112d4d24c558.0000000000000004:29 [read 2015232~16384 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e96)", 2024-09-14T06:32:58.197 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:30:17.939903+0000", 2024-09-14T06:32:58.197 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 160.24579373200001, 2024-09-14T06:32:58.197 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.074128199000000006, 2024-09-14T06:32:58.197 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:32:58.198 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:32:58.198 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:32:58.198 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:32:58.198 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:32:58.198 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:58.198 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 115195 2024-09-14T06:32:58.198 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:58.198 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:32:58.198 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:58.198 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:32:58.198 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.939903+0000", 2024-09-14T06:32:58.198 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:58.198 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:58.198 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:58.199 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:32:58.199 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.939903+0000", 2024-09-14T06:32:58.199 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:58.199 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:58.199 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:58.199 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:32:58.199 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.939904+0000", 2024-09-14T06:32:58.199 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.5799999999999999e-07 2024-09-14T06:32:58.199 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:58.199 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:58.199 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:32:58.199 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.939906+0000", 2024-09-14T06:32:58.199 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.2350000000000002e-06 2024-09-14T06:32:58.200 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:58.200 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:58.200 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:32:58.200 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.939913+0000", 2024-09-14T06:32:58.200 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.5649999999999998e-06 2024-09-14T06:32:58.200 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:58.200 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:58.200 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:32:58.200 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.939918+0000", 2024-09-14T06:32:58.200 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.784e-06 2024-09-14T06:32:58.200 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:58.200 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:58.200 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:32:58.201 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.013921+0000", 2024-09-14T06:32:58.201 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.074002911000000005 2024-09-14T06:32:58.201 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:58.201 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:58.201 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:32:58.201 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.013971+0000", 2024-09-14T06:32:58.201 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.9551e-05 2024-09-14T06:32:58.201 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:58.201 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:58.201 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:32:58.201 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.014031+0000", 2024-09-14T06:32:58.201 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.0594999999999998e-05 2024-09-14T06:32:58.201 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:58.201 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:32:58.202 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:58.202 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:58.202 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:58.202 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:115196 3.1 3:856be886:::rbd_data.112d4d24c558.0000000000000004:29 [read 2260992~16384 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e96)", 2024-09-14T06:32:58.202 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:30:17.939925+0000", 2024-09-14T06:32:58.202 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 160.24577153999999, 2024-09-14T06:32:58.202 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.074320137999999994, 2024-09-14T06:32:58.202 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:32:58.202 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:32:58.202 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:32:58.202 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:32:58.202 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:32:58.202 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:58.203 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 115196 2024-09-14T06:32:58.203 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:58.203 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:32:58.203 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:58.203 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:32:58.203 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.939925+0000", 2024-09-14T06:32:58.203 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:58.203 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:58.203 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:58.203 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:32:58.203 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.939925+0000", 2024-09-14T06:32:58.203 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:58.203 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:58.203 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:58.204 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:32:58.204 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.939926+0000", 2024-09-14T06:32:58.204 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.9999999999999998e-07 2024-09-14T06:32:58.204 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:58.204 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:58.204 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:32:58.204 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.939928+0000", 2024-09-14T06:32:58.204 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.9549999999999999e-06 2024-09-14T06:32:58.204 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:58.204 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:58.204 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:32:58.204 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.939934+0000", 2024-09-14T06:32:58.204 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.127e-06 2024-09-14T06:32:58.205 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:58.205 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:58.205 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:32:58.205 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.939939+0000", 2024-09-14T06:32:58.205 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.775e-06 2024-09-14T06:32:58.205 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:58.205 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:58.205 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:32:58.205 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.014075+0000", 2024-09-14T06:32:58.205 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.074136598999999997 2024-09-14T06:32:58.205 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:58.205 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:58.205 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:32:58.206 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.014162+0000", 2024-09-14T06:32:58.206 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.6477999999999994e-05 2024-09-14T06:32:58.206 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:58.206 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:58.206 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:32:58.206 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.014246+0000", 2024-09-14T06:32:58.206 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.3703999999999997e-05 2024-09-14T06:32:58.206 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:58.206 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:32:58.207 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:58.207 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:58.208 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:58.208 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:115197 3.1 3:856be886:::rbd_data.112d4d24c558.0000000000000004:29 [read 2506752~16384 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e96)", 2024-09-14T06:32:58.208 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:30:17.939969+0000", 2024-09-14T06:32:58.208 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 160.245728055, 2024-09-14T06:32:58.208 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.074528544000000002, 2024-09-14T06:32:58.208 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:32:58.208 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:32:58.208 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:32:58.208 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:32:58.208 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:32:58.208 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:58.208 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 115197 2024-09-14T06:32:58.208 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:58.209 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:32:58.209 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:58.209 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:32:58.209 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.939969+0000", 2024-09-14T06:32:58.209 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:58.209 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:58.209 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:58.209 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:32:58.209 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.939969+0000", 2024-09-14T06:32:58.209 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:58.209 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:58.209 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:58.209 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:32:58.210 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.939970+0000", 2024-09-14T06:32:58.210 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.3600000000000003e-07 2024-09-14T06:32:58.210 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:58.210 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:58.210 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:32:58.210 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.939972+0000", 2024-09-14T06:32:58.210 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.3180000000000001e-06 2024-09-14T06:32:58.210 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:58.210 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:58.210 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:32:58.210 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.939979+0000", 2024-09-14T06:32:58.210 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.8800000000000002e-06 2024-09-14T06:32:58.210 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:58.211 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:58.211 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:32:58.211 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.939984+0000", 2024-09-14T06:32:58.211 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.3510000000000001e-06 2024-09-14T06:32:58.211 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:58.211 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:58.211 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:32:58.211 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.014302+0000", 2024-09-14T06:32:58.211 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.074317416999999997 2024-09-14T06:32:58.211 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:58.211 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:58.211 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:32:58.211 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.014392+0000", 2024-09-14T06:32:58.212 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.0181999999999998e-05 2024-09-14T06:32:58.212 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:58.212 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:58.212 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:32:58.212 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.014497+0000", 2024-09-14T06:32:58.212 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00010576 2024-09-14T06:32:58.212 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:58.212 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:32:58.212 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:58.212 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:58.212 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:58.212 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:115198 3.1 3:856be886:::rbd_data.112d4d24c558.0000000000000004:29 [read 2752512~16384 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e96)", 2024-09-14T06:32:58.212 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:30:17.940004+0000", 2024-09-14T06:32:58.213 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 160.24569277500001, 2024-09-14T06:32:58.213 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.074736367999999997, 2024-09-14T06:32:58.213 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:32:58.213 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:32:58.213 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:32:58.213 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:32:58.213 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:32:58.213 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:58.213 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 115198 2024-09-14T06:32:58.213 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:58.213 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:32:58.213 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:58.213 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:32:58.214 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940004+0000", 2024-09-14T06:32:58.214 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:58.214 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:58.214 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:58.214 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:32:58.214 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940004+0000", 2024-09-14T06:32:58.214 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:58.214 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:58.214 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:58.214 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:32:58.214 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940005+0000", 2024-09-14T06:32:58.214 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.2200000000000004e-07 2024-09-14T06:32:58.214 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:58.215 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:58.215 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:32:58.215 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940007+0000", 2024-09-14T06:32:58.215 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.322e-06 2024-09-14T06:32:58.215 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:58.215 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:58.215 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:32:58.215 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940014+0000", 2024-09-14T06:32:58.215 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.6220000000000003e-06 2024-09-14T06:32:58.215 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:58.215 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:58.215 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:32:58.215 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940019+0000", 2024-09-14T06:32:58.215 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.6380000000000001e-06 2024-09-14T06:32:58.216 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:58.216 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:58.216 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:32:58.216 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.014562+0000", 2024-09-14T06:32:58.216 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.074542320999999995 2024-09-14T06:32:58.216 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:58.216 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:58.216 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:32:58.216 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.014646+0000", 2024-09-14T06:32:58.216 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.4079999999999995e-05 2024-09-14T06:32:58.216 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:58.216 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:58.216 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:32:58.217 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.014741+0000", 2024-09-14T06:32:58.217 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.4763000000000001e-05 2024-09-14T06:32:58.217 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:58.217 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:32:58.217 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:58.217 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:58.217 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:58.217 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:115199 3.1 3:856be886:::rbd_data.112d4d24c558.0000000000000004:29 [read 2998272~16384 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e96)", 2024-09-14T06:32:58.217 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:30:17.940040+0000", 2024-09-14T06:32:58.217 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 160.24565702000001, 2024-09-14T06:32:58.217 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.074957134999999994, 2024-09-14T06:32:58.217 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:32:58.218 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:32:58.218 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:32:58.218 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:32:58.218 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:32:58.218 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:58.218 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 115199 2024-09-14T06:32:58.218 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:58.218 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:32:58.218 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:58.218 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:32:58.218 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940040+0000", 2024-09-14T06:32:58.218 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:58.218 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:58.219 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:58.219 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:32:58.219 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940040+0000", 2024-09-14T06:32:58.219 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:58.219 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:58.219 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:58.219 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:32:58.219 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940041+0000", 2024-09-14T06:32:58.219 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.4000000000000001e-07 2024-09-14T06:32:58.219 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:58.219 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:58.219 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:32:58.219 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940046+0000", 2024-09-14T06:32:58.220 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.7250000000000002e-06 2024-09-14T06:32:58.220 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:58.220 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:58.220 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:32:58.220 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940053+0000", 2024-09-14T06:32:58.220 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.6109999999999999e-06 2024-09-14T06:32:58.220 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:58.220 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:58.220 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:32:58.220 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940058+0000", 2024-09-14T06:32:58.220 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.1590000000000001e-06 2024-09-14T06:32:58.220 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:58.220 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:58.220 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:32:58.221 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.014768+0000", 2024-09-14T06:32:58.221 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.074709624000000002 2024-09-14T06:32:58.221 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:58.221 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:58.221 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:32:58.221 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.014907+0000", 2024-09-14T06:32:58.221 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000139665 2024-09-14T06:32:58.221 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:58.221 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:58.221 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:32:58.221 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.014997+0000", 2024-09-14T06:32:58.221 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.9710999999999995e-05 2024-09-14T06:32:58.221 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:58.221 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:32:58.222 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:58.222 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:58.222 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:58.222 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:115200 3.1 3:856be886:::rbd_data.112d4d24c558.0000000000000004:29 [read 3244032~16384 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e96)", 2024-09-14T06:32:58.222 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:30:17.940080+0000", 2024-09-14T06:32:58.222 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 160.24561681500001, 2024-09-14T06:32:58.222 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.075137291999999994, 2024-09-14T06:32:58.222 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:32:58.222 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:32:58.222 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:32:58.222 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:32:58.222 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:32:58.222 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:58.222 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 115200 2024-09-14T06:32:58.223 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:58.223 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:32:58.223 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:58.223 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:32:58.223 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940080+0000", 2024-09-14T06:32:58.223 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:58.223 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:58.223 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:58.223 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:32:58.223 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940080+0000", 2024-09-14T06:32:58.223 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:58.223 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:58.223 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:58.223 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:32:58.224 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940081+0000", 2024-09-14T06:32:58.224 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.0800000000000004e-07 2024-09-14T06:32:58.224 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:58.224 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:58.224 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:32:58.224 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940083+0000", 2024-09-14T06:32:58.224 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.3920000000000001e-06 2024-09-14T06:32:58.224 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:58.224 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:58.224 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:32:58.224 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940091+0000", 2024-09-14T06:32:58.224 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.1389999999999995e-06 2024-09-14T06:32:58.224 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:58.225 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:58.225 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:32:58.225 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940097+0000", 2024-09-14T06:32:58.225 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.5550000000000002e-06 2024-09-14T06:32:58.225 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:58.225 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:58.225 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:32:58.225 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.015025+0000", 2024-09-14T06:32:58.225 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.074928579999999995 2024-09-14T06:32:58.225 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:58.225 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:58.225 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:32:58.225 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.015096+0000", 2024-09-14T06:32:58.225 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.0705999999999997e-05 2024-09-14T06:32:58.225 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:58.226 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:58.226 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:32:58.226 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.015217+0000", 2024-09-14T06:32:58.226 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00012131200000000001 2024-09-14T06:32:58.226 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:58.226 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:32:58.226 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:58.226 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:58.226 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:58.226 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:115201 3.1 3:856be886:::rbd_data.112d4d24c558.0000000000000004:29 [read 3489792~16384 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e96)", 2024-09-14T06:32:58.226 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:30:17.940113+0000", 2024-09-14T06:32:58.226 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 160.24558404199999, 2024-09-14T06:32:58.226 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.075246899000000006, 2024-09-14T06:32:58.227 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:32:58.227 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:32:58.227 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:32:58.227 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:32:58.227 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:32:58.227 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:58.227 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 115201 2024-09-14T06:32:58.227 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:58.227 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:32:58.227 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:58.227 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:32:58.227 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940113+0000", 2024-09-14T06:32:58.227 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:58.228 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:58.228 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:58.228 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:32:58.228 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940113+0000", 2024-09-14T06:32:58.228 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:58.228 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:58.228 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:58.228 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:32:58.228 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940114+0000", 2024-09-14T06:32:58.228 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.1500000000000004e-07 2024-09-14T06:32:58.228 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:58.228 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:58.228 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:32:58.229 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940116+0000", 2024-09-14T06:32:58.229 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.2809999999999998e-06 2024-09-14T06:32:58.229 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:58.229 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:58.229 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:32:58.229 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940123+0000", 2024-09-14T06:32:58.229 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.9229999999999996e-06 2024-09-14T06:32:58.229 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:58.229 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:58.229 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:32:58.229 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940128+0000", 2024-09-14T06:32:58.229 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.7790000000000003e-06 2024-09-14T06:32:58.229 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:58.230 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:58.230 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:32:58.230 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.015253+0000", 2024-09-14T06:32:58.230 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.075125792999999996 2024-09-14T06:32:58.230 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:58.230 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:58.230 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:32:58.230 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.015297+0000", 2024-09-14T06:32:58.230 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.3847000000000002e-05 2024-09-14T06:32:58.230 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:58.230 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:58.230 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:32:58.230 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.015360+0000", 2024-09-14T06:32:58.230 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.2661000000000002e-05 2024-09-14T06:32:58.230 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:58.231 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:32:58.231 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:58.231 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:58.231 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:58.231 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:115202 3.1 3:856be886:::rbd_data.112d4d24c558.0000000000000004:29 [read 3735552~16384 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e96)", 2024-09-14T06:32:58.231 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:30:17.940135+0000", 2024-09-14T06:32:58.231 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 160.24556226300001, 2024-09-14T06:32:58.231 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.075451105000000004, 2024-09-14T06:32:58.231 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:32:58.231 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:32:58.231 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:32:58.231 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:32:58.231 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:32:58.231 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:58.232 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 115202 2024-09-14T06:32:58.232 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:58.232 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:32:58.232 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:58.232 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:32:58.232 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940135+0000", 2024-09-14T06:32:58.232 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:58.232 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:58.232 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:58.232 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:32:58.232 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940135+0000", 2024-09-14T06:32:58.232 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:58.232 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:58.233 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:58.233 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:32:58.233 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940135+0000", 2024-09-14T06:32:58.233 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.0999999999999999e-07 2024-09-14T06:32:58.233 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:58.233 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:58.233 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:32:58.233 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940137+0000", 2024-09-14T06:32:58.233 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.0020000000000001e-06 2024-09-14T06:32:58.233 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:58.233 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:58.233 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:32:58.233 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940143+0000", 2024-09-14T06:32:58.234 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.9850000000000003e-06 2024-09-14T06:32:58.234 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:58.234 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:58.234 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:32:58.234 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940148+0000", 2024-09-14T06:32:58.234 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.9880000000000004e-06 2024-09-14T06:32:58.234 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:58.234 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:58.234 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:32:58.234 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.015400+0000", 2024-09-14T06:32:58.234 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.075252186999999998 2024-09-14T06:32:58.234 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:58.234 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:58.234 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:32:58.234 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.015481+0000", 2024-09-14T06:32:58.235 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.0698000000000006e-05 2024-09-14T06:32:58.235 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:58.235 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:58.235 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:32:58.235 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.015586+0000", 2024-09-14T06:32:58.235 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00010483499999999999 2024-09-14T06:32:58.235 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:58.235 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:32:58.235 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:58.235 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:58.235 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:58.236 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:115203 3.1 3:856be886:::rbd_data.112d4d24c558.0000000000000004:29 [read 3981312~16384 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e96)", 2024-09-14T06:32:58.236 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:30:17.940156+0000", 2024-09-14T06:32:58.236 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 160.245541038, 2024-09-14T06:32:58.236 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.075620860999999998, 2024-09-14T06:32:58.236 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:32:58.237 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:32:58.237 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:32:58.237 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:32:58.237 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:32:58.237 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:58.238 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 115203 2024-09-14T06:32:58.238 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:58.238 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:32:58.238 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:58.238 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:32:58.238 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940156+0000", 2024-09-14T06:32:58.239 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:58.239 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:58.239 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:58.239 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:32:58.239 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940156+0000", 2024-09-14T06:32:58.239 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:58.240 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:58.240 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:58.240 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:32:58.240 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940156+0000", 2024-09-14T06:32:58.240 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.1500000000000005e-07 2024-09-14T06:32:58.240 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:58.240 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:58.241 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:32:58.241 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940158+0000", 2024-09-14T06:32:58.241 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.967e-06 2024-09-14T06:32:58.241 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:58.241 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:58.241 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:32:58.241 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940165+0000", 2024-09-14T06:32:58.242 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.6939999999999998e-06 2024-09-14T06:32:58.242 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:58.242 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:58.242 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:32:58.242 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940170+0000", 2024-09-14T06:32:58.242 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.7539999999999997e-06 2024-09-14T06:32:58.242 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:58.242 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:58.243 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:32:58.243 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.015633+0000", 2024-09-14T06:32:58.243 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.075463334000000007 2024-09-14T06:32:58.243 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:58.243 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:58.243 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:32:58.243 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.015693+0000", 2024-09-14T06:32:58.243 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.9367000000000003e-05 2024-09-14T06:32:58.244 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:58.244 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:58.244 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:32:58.244 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.015777+0000", 2024-09-14T06:32:58.244 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.4229999999999998e-05 2024-09-14T06:32:58.244 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:58.244 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:32:58.244 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:58.244 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:58.244 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:58.244 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:115204 3.1 3:856be886:::rbd_data.112d4d24c558.0000000000000004:29 [read 4227072~16384 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e96)", 2024-09-14T06:32:58.244 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:30:17.940176+0000", 2024-09-14T06:32:58.245 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 160.245520505, 2024-09-14T06:32:58.245 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.075785505000000003, 2024-09-14T06:32:58.245 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:32:58.245 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:32:58.245 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:32:58.245 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:32:58.245 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:32:58.245 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:58.245 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 115204 2024-09-14T06:32:58.245 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:58.245 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:32:58.245 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:58.245 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:32:58.245 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940176+0000", 2024-09-14T06:32:58.246 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:58.246 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:58.246 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:58.246 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:32:58.246 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940176+0000", 2024-09-14T06:32:58.246 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:58.246 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:58.246 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:58.246 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:32:58.246 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940177+0000", 2024-09-14T06:32:58.246 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.7899999999999999e-07 2024-09-14T06:32:58.246 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:58.246 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:58.247 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:32:58.247 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940179+0000", 2024-09-14T06:32:58.247 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.103e-06 2024-09-14T06:32:58.247 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:58.247 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:58.247 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:32:58.247 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940185+0000", 2024-09-14T06:32:58.247 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.7239999999999997e-06 2024-09-14T06:32:58.247 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:58.247 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:58.247 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:32:58.247 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940190+0000", 2024-09-14T06:32:58.247 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.7269999999999998e-06 2024-09-14T06:32:58.247 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:58.248 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:58.248 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:32:58.248 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.015814+0000", 2024-09-14T06:32:58.248 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.075624114000000006 2024-09-14T06:32:58.248 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:58.248 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:58.248 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:32:58.248 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.015862+0000", 2024-09-14T06:32:58.248 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.8770999999999998e-05 2024-09-14T06:32:58.248 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:58.248 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:58.248 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:32:58.248 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.015962+0000", 2024-09-14T06:32:58.249 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.9586999999999994e-05 2024-09-14T06:32:58.249 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:58.249 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:32:58.249 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:58.249 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:58.249 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:58.249 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:115205 3.1 3:856be886:::rbd_data.112d4d24c558.0000000000000004:29 [read 4472832~16384 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e96)", 2024-09-14T06:32:58.249 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:30:17.940197+0000", 2024-09-14T06:32:58.249 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 160.24549979599999, 2024-09-14T06:32:58.250 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.076068920999999998, 2024-09-14T06:32:58.250 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:32:58.250 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:32:58.250 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:32:58.250 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:32:58.250 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:32:58.250 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:58.250 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 115205 2024-09-14T06:32:58.250 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:58.250 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:32:58.250 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:58.250 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:32:58.250 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940197+0000", 2024-09-14T06:32:58.251 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:58.251 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:58.251 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:58.251 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:32:58.251 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940197+0000", 2024-09-14T06:32:58.251 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:58.251 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:58.251 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:58.251 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:32:58.251 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940198+0000", 2024-09-14T06:32:58.251 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.5999999999999999e-07 2024-09-14T06:32:58.251 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:58.251 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:58.251 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:32:58.252 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940200+0000", 2024-09-14T06:32:58.252 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.0899999999999999e-06 2024-09-14T06:32:58.252 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:58.252 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:58.252 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:32:58.252 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940206+0000", 2024-09-14T06:32:58.252 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.8710000000000002e-06 2024-09-14T06:32:58.252 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:58.252 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:58.252 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:32:58.252 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940211+0000", 2024-09-14T06:32:58.252 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.9230000000000001e-06 2024-09-14T06:32:58.252 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:58.253 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:58.253 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:32:58.253 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.016059+0000", 2024-09-14T06:32:58.253 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.075848051999999999 2024-09-14T06:32:58.253 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:58.253 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:58.253 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:32:58.253 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.016151+0000", 2024-09-14T06:32:58.253 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.1982000000000001e-05 2024-09-14T06:32:58.253 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:58.253 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:58.253 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:32:58.253 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.016266+0000", 2024-09-14T06:32:58.254 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000115543 2024-09-14T06:32:58.254 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:58.254 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:32:58.254 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:58.254 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:58.254 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:58.254 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:115206 3.1 3:856be886:::rbd_data.112d4d24c558.0000000000000004:29 [read 4718592~16384 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e96)", 2024-09-14T06:32:58.254 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:30:17.940223+0000", 2024-09-14T06:32:58.254 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 160.24547370600001, 2024-09-14T06:32:58.254 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.076265115999999994, 2024-09-14T06:32:58.254 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:32:58.254 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:32:58.254 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:32:58.254 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:32:58.254 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:32:58.255 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:58.255 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 115206 2024-09-14T06:32:58.255 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:58.255 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:32:58.255 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:58.255 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:32:58.255 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940223+0000", 2024-09-14T06:32:58.255 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:58.255 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:58.255 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:58.255 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:32:58.255 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940223+0000", 2024-09-14T06:32:58.255 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:58.255 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:58.256 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:58.256 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:32:58.256 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940224+0000", 2024-09-14T06:32:58.256 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.0399999999999996e-07 2024-09-14T06:32:58.256 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:58.256 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:58.256 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:32:58.256 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940226+0000", 2024-09-14T06:32:58.256 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.1780000000000002e-06 2024-09-14T06:32:58.256 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:58.256 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:58.256 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:32:58.256 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940232+0000", 2024-09-14T06:32:58.257 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.5010000000000001e-06 2024-09-14T06:32:58.257 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:58.257 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:58.257 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:32:58.257 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940237+0000", 2024-09-14T06:32:58.257 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.0250000000000002e-06 2024-09-14T06:32:58.257 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:58.257 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:58.257 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:32:58.257 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.016305+0000", 2024-09-14T06:32:58.257 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.076067262999999996 2024-09-14T06:32:58.257 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:58.257 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:58.257 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:32:58.258 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.016383+0000", 2024-09-14T06:32:58.258 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.8578000000000006e-05 2024-09-14T06:32:58.258 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:58.258 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:58.258 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:32:58.258 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.016488+0000", 2024-09-14T06:32:58.258 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000105067 2024-09-14T06:32:58.258 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:58.258 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:32:58.258 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:58.258 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:58.258 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:58.258 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:115207 3.1 3:856be886:::rbd_data.112d4d24c558.0000000000000004:29 [read 4964352~16384 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e96)", 2024-09-14T06:32:58.258 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:30:17.940244+0000", 2024-09-14T06:32:58.259 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 160.24545259000001, 2024-09-14T06:32:58.259 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.076371154999999996, 2024-09-14T06:32:58.259 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:32:58.259 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:32:58.259 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:32:58.259 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:32:58.259 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:32:58.259 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:58.259 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 115207 2024-09-14T06:32:58.259 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:58.259 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:32:58.259 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:58.259 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:32:58.259 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940244+0000", 2024-09-14T06:32:58.260 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:58.260 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:58.260 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:58.260 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:32:58.260 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940244+0000", 2024-09-14T06:32:58.260 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:58.260 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:58.260 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:58.260 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:32:58.260 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940245+0000", 2024-09-14T06:32:58.260 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.1200000000000003e-07 2024-09-14T06:32:58.260 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:58.260 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:58.260 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:32:58.261 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940247+0000", 2024-09-14T06:32:58.261 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.0030000000000001e-06 2024-09-14T06:32:58.261 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:58.261 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:58.261 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:32:58.261 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940252+0000", 2024-09-14T06:32:58.261 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.5520000000000004e-06 2024-09-14T06:32:58.261 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:58.261 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:58.261 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:32:58.261 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940257+0000", 2024-09-14T06:32:58.261 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.6759999999999998e-06 2024-09-14T06:32:58.261 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:58.261 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:58.262 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:32:58.262 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.016521+0000", 2024-09-14T06:32:58.262 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.076263499999999998 2024-09-14T06:32:58.262 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:58.262 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:58.262 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:32:58.262 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.016562+0000", 2024-09-14T06:32:58.262 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.0967000000000003e-05 2024-09-14T06:32:58.262 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:58.262 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:58.262 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:32:58.262 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.016616+0000", 2024-09-14T06:32:58.262 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.3944999999999999e-05 2024-09-14T06:32:58.262 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:58.263 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:32:58.263 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:58.263 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:58.263 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:58.263 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:115208 3.1 3:856be886:::rbd_data.112d4d24c558.0000000000000004:29 [read 5210112~16384 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e96)", 2024-09-14T06:32:58.263 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:30:17.940264+0000", 2024-09-14T06:32:58.263 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 160.245432733, 2024-09-14T06:32:58.263 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.076558288000000002, 2024-09-14T06:32:58.263 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:32:58.263 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:32:58.263 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:32:58.263 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:32:58.263 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:32:58.264 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:58.264 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 115208 2024-09-14T06:32:58.264 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:58.264 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:32:58.264 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:58.264 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:32:58.264 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940264+0000", 2024-09-14T06:32:58.264 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:58.264 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:58.264 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:58.264 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:32:58.264 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940264+0000", 2024-09-14T06:32:58.264 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:58.264 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:58.265 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:58.265 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:32:58.265 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940265+0000", 2024-09-14T06:32:58.265 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.0500000000000004e-07 2024-09-14T06:32:58.265 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:58.265 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:58.265 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:32:58.265 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940267+0000", 2024-09-14T06:32:58.265 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.9810000000000002e-06 2024-09-14T06:32:58.265 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:58.265 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:58.265 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:32:58.265 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940272+0000", 2024-09-14T06:32:58.266 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.6999999999999996e-06 2024-09-14T06:32:58.266 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:58.266 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:58.266 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:32:58.266 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940277+0000", 2024-09-14T06:32:58.266 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.6480000000000002e-06 2024-09-14T06:32:58.266 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:58.266 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:58.266 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:32:58.266 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.016650+0000", 2024-09-14T06:32:58.266 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.076373089000000005 2024-09-14T06:32:58.266 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:58.266 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:58.266 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:32:58.267 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.016724+0000", 2024-09-14T06:32:58.267 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.4105999999999998e-05 2024-09-14T06:32:58.267 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:58.267 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:58.267 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:32:58.267 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.016823+0000", 2024-09-14T06:32:58.267 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.8258999999999996e-05 2024-09-14T06:32:58.267 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:58.267 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:32:58.267 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:58.267 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:58.267 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:58.267 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:115209 3.1 3:856be886:::rbd_data.112d4d24c558.0000000000000004:29 [read 5455872~16384 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e96)", 2024-09-14T06:32:58.267 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:30:17.940283+0000", 2024-09-14T06:32:58.268 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 160.24541370700001, 2024-09-14T06:32:58.268 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.076724029999999999, 2024-09-14T06:32:58.268 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:32:58.268 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:32:58.268 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:32:58.268 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:32:58.268 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:32:58.268 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:58.268 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 115209 2024-09-14T06:32:58.268 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:58.268 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:32:58.268 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:58.268 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:32:58.269 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940283+0000", 2024-09-14T06:32:58.269 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:58.269 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:58.269 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:58.269 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:32:58.269 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940283+0000", 2024-09-14T06:32:58.269 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:58.269 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:58.269 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:58.269 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:32:58.269 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940284+0000", 2024-09-14T06:32:58.269 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.6700000000000003e-07 2024-09-14T06:32:58.269 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:58.270 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:58.270 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:32:58.270 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940286+0000", 2024-09-14T06:32:58.270 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.9520000000000001e-06 2024-09-14T06:32:58.270 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:58.270 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:58.270 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:32:58.270 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940292+0000", 2024-09-14T06:32:58.270 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.7339999999999996e-06 2024-09-14T06:32:58.270 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:58.270 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:58.270 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:32:58.270 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940296+0000", 2024-09-14T06:32:58.270 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.7700000000000001e-06 2024-09-14T06:32:58.271 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:58.271 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:58.271 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:32:58.271 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.016859+0000", 2024-09-14T06:32:58.271 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.07656317 2024-09-14T06:32:58.271 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:58.271 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:58.271 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:32:58.271 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.016929+0000", 2024-09-14T06:32:58.271 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.9679999999999997e-05 2024-09-14T06:32:58.271 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:58.271 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:58.271 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:32:58.271 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.017007+0000", 2024-09-14T06:32:58.272 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.8157000000000004e-05 2024-09-14T06:32:58.272 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:58.272 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:32:58.272 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:58.272 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:58.272 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:58.272 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:115210 3.1 3:856be886:::rbd_data.112d4d24c558.0000000000000004:29 [read 5701632~16384 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e96)", 2024-09-14T06:32:58.272 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:30:17.940303+0000", 2024-09-14T06:32:58.272 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 160.24539419000001, 2024-09-14T06:32:58.272 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.076821358000000006, 2024-09-14T06:32:58.272 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:32:58.272 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:32:58.272 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:32:58.273 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:32:58.273 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:32:58.273 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:58.273 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 115210 2024-09-14T06:32:58.273 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:58.273 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:32:58.273 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:58.273 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:32:58.273 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940303+0000", 2024-09-14T06:32:58.273 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:58.273 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:58.273 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:58.273 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:32:58.273 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940303+0000", 2024-09-14T06:32:58.274 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:58.274 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:58.274 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:58.274 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:32:58.274 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940303+0000", 2024-09-14T06:32:58.274 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.3000000000000001e-07 2024-09-14T06:32:58.274 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:58.274 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:58.274 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:32:58.274 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940306+0000", 2024-09-14T06:32:58.274 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.255e-06 2024-09-14T06:32:58.274 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:58.274 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:58.274 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:32:58.275 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940312+0000", 2024-09-14T06:32:58.275 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.9710000000000001e-06 2024-09-14T06:32:58.275 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:58.275 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:58.275 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:32:58.275 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940316+0000", 2024-09-14T06:32:58.275 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.5340000000000001e-06 2024-09-14T06:32:58.275 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:58.275 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:58.275 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:32:58.275 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.017033+0000", 2024-09-14T06:32:58.275 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.076717169000000002 2024-09-14T06:32:58.275 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:58.275 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:58.275 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:32:58.276 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.017073+0000", 2024-09-14T06:32:58.276 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.9582000000000002e-05 2024-09-14T06:32:58.276 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:58.276 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:58.276 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:32:58.276 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.017124+0000", 2024-09-14T06:32:58.276 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.1316999999999997e-05 2024-09-14T06:32:58.276 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:58.276 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:32:58.276 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:58.276 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:58.276 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:58.276 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:115219 3.1 3:856be886:::rbd_data.112d4d24c558.0000000000000004:29 [read 65536~16384 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e96)", 2024-09-14T06:32:58.277 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:30:17.940323+0000", 2024-09-14T06:32:58.277 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 160.24537432899999, 2024-09-14T06:32:58.277 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.076915205, 2024-09-14T06:32:58.277 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:32:58.277 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:32:58.277 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:32:58.277 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:32:58.277 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:32:58.277 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:58.277 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 115219 2024-09-14T06:32:58.277 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:58.277 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:32:58.277 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:58.277 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:32:58.277 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940323+0000", 2024-09-14T06:32:58.278 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:58.278 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:58.278 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:58.278 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:32:58.278 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940323+0000", 2024-09-14T06:32:58.278 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:58.278 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:58.278 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:58.278 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:32:58.278 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940323+0000", 2024-09-14T06:32:58.278 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.0800000000000005e-07 2024-09-14T06:32:58.278 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:58.278 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:58.279 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:32:58.279 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940328+0000", 2024-09-14T06:32:58.279 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.887e-06 2024-09-14T06:32:58.279 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:58.279 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:58.279 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:32:58.279 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940334+0000", 2024-09-14T06:32:58.279 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.3269999999999997e-06 2024-09-14T06:32:58.279 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:58.279 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:58.279 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:32:58.279 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940339+0000", 2024-09-14T06:32:58.279 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.7960000000000003e-06 2024-09-14T06:32:58.279 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:58.280 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:58.280 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:32:58.280 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.017148+0000", 2024-09-14T06:32:58.280 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.076809125000000006 2024-09-14T06:32:58.280 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:58.280 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:58.280 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:32:58.280 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.017186+0000", 2024-09-14T06:32:58.280 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.7283000000000002e-05 2024-09-14T06:32:58.280 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:58.280 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:58.280 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:32:58.280 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.017238+0000", 2024-09-14T06:32:58.281 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.2278999999999999e-05 2024-09-14T06:32:58.281 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:58.281 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:32:58.281 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:58.281 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:58.281 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:58.281 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:115220 3.1 3:856be886:::rbd_data.112d4d24c558.0000000000000004:29 [read 311296~16384 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e96)", 2024-09-14T06:32:58.282 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:30:17.940346+0000", 2024-09-14T06:32:58.282 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 160.24535119999999, 2024-09-14T06:32:58.282 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.077033293000000003, 2024-09-14T06:32:58.282 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:32:58.282 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:32:58.282 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:32:58.282 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:32:58.282 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:32:58.283 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:58.283 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 115220 2024-09-14T06:32:58.283 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:58.283 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:32:58.283 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:58.283 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:32:58.283 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940346+0000", 2024-09-14T06:32:58.283 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:58.283 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:58.283 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:58.283 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:32:58.283 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940346+0000", 2024-09-14T06:32:58.283 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:58.283 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:58.284 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:58.284 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:32:58.284 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940346+0000", 2024-09-14T06:32:58.284 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.4300000000000003e-07 2024-09-14T06:32:58.284 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:58.284 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:58.284 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:32:58.284 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940348+0000", 2024-09-14T06:32:58.284 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.0030000000000001e-06 2024-09-14T06:32:58.284 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:58.284 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:58.284 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:32:58.284 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.004655+0000", 2024-09-14T06:32:58.284 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.064306323999999998 2024-09-14T06:32:58.285 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:58.285 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:58.285 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:32:58.285 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.004671+0000", 2024-09-14T06:32:58.285 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.6589999999999999e-05 2024-09-14T06:32:58.285 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:58.285 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:58.285 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:32:58.285 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.017262+0000", 2024-09-14T06:32:58.285 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.012590570000000001 2024-09-14T06:32:58.285 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:58.285 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:58.285 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:32:58.285 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.017321+0000", 2024-09-14T06:32:58.286 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.9210999999999999e-05 2024-09-14T06:32:58.286 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:58.286 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:58.286 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:32:58.286 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.017379+0000", 2024-09-14T06:32:58.286 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.8051999999999999e-05 2024-09-14T06:32:58.286 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:58.286 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:32:58.286 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:58.286 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:58.286 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:32:58.286 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-14T06:32:58.288 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-14T06:32:58.288 INFO:tasks.ceph.osd.1.smithi094.stderr:2024-09-14T06:32:58.286+0000 7f328e47a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33641) UID: 0 2024-09-14T06:32:58.389 INFO:tasks.ceph.osd.6.smithi138.stderr:2024-09-14T06:32:58.388+0000 7f8cd8849640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33620) UID: 0 2024-09-14T06:32:58.432 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-14T06:32:58.432 INFO:teuthology.orchestra.run.smithi094.stdout: "ops": [], 2024-09-14T06:32:58.432 INFO:teuthology.orchestra.run.smithi094.stdout: "num_ops": 0 2024-09-14T06:32:58.432 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-14T06:32:58.441 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-14T06:32:58.490 INFO:tasks.ceph.osd.7.smithi138.stderr:2024-09-14T06:32:58.489+0000 7f3c6a21d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33623) UID: 0 2024-09-14T06:32:58.544 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-14T06:32:58.544 INFO:teuthology.orchestra.run.smithi094.stdout: "ops": [], 2024-09-14T06:32:58.544 INFO:teuthology.orchestra.run.smithi094.stdout: "complaint_time": 30, 2024-09-14T06:32:58.544 INFO:teuthology.orchestra.run.smithi094.stdout: "num_blocked_ops": 0 2024-09-14T06:32:58.544 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-14T06:32:58.553 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-14T06:32:58.591 INFO:tasks.ceph.osd.6.smithi138.stderr:2024-09-14T06:32:58.591+0000 7f8cd8849640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33620) UID: 0 2024-09-14T06:32:58.657 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-14T06:32:58.657 INFO:teuthology.orchestra.run.smithi094.stdout: "size": 20, 2024-09-14T06:32:58.657 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 600, 2024-09-14T06:32:58.657 INFO:teuthology.orchestra.run.smithi094.stdout: "ops": [ 2024-09-14T06:32:58.657 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:58.657 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:14919 3.0 3:05c39535:::rbd_data.112d92027590.0000000000000009:f [read 805888~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:32:58.657 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:26.083843+0000", 2024-09-14T06:32:58.657 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 212.57110983499999, 2024-09-14T06:32:58.657 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.041992356000000002, 2024-09-14T06:32:58.657 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:32:58.657 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:32:58.657 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:32:58.658 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:32:58.658 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:32:58.658 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:58.658 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 14919 2024-09-14T06:32:58.658 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:58.658 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:32:58.658 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:58.658 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:32:58.658 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.083843+0000", 2024-09-14T06:32:58.658 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:58.658 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:58.658 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:58.658 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:32:58.658 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.083843+0000", 2024-09-14T06:32:58.659 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:58.659 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:58.659 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:58.659 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:32:58.659 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.083844+0000", 2024-09-14T06:32:58.659 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.3300000000000001e-07 2024-09-14T06:32:58.659 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:58.659 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:58.659 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:32:58.659 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.083845+0000", 2024-09-14T06:32:58.659 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.4729999999999999e-06 2024-09-14T06:32:58.659 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:58.660 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:58.660 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:32:58.660 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.083850+0000", 2024-09-14T06:32:58.660 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.0130000000000001e-06 2024-09-14T06:32:58.660 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:58.660 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:58.660 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:32:58.660 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.083854+0000", 2024-09-14T06:32:58.660 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.4759999999999998e-06 2024-09-14T06:32:58.660 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:58.660 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:58.660 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:32:58.660 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.125636+0000", 2024-09-14T06:32:58.661 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.041782683000000001 2024-09-14T06:32:58.661 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:58.661 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:58.661 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:32:58.661 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.125680+0000", 2024-09-14T06:32:58.661 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.3614e-05 2024-09-14T06:32:58.661 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:58.661 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:58.661 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:32:58.661 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.125836+0000", 2024-09-14T06:32:58.661 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000155864 2024-09-14T06:32:58.661 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:58.661 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:32:58.662 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:58.662 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:58.662 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:58.662 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:14920 3.0 3:05c39535:::rbd_data.112d92027590.0000000000000009:f [read 816128~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:32:58.662 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:26.083868+0000", 2024-09-14T06:32:58.662 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 212.57108517399999, 2024-09-14T06:32:58.662 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.042166771999999998, 2024-09-14T06:32:58.662 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:32:58.662 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:32:58.662 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:32:58.662 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:32:58.662 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:32:58.662 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:58.662 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 14920 2024-09-14T06:32:58.663 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:58.663 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:32:58.663 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:58.663 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:32:58.663 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.083868+0000", 2024-09-14T06:32:58.663 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:58.663 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:58.663 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:58.663 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:32:58.663 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.083868+0000", 2024-09-14T06:32:58.663 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:58.663 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:58.663 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:58.663 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:32:58.664 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.083869+0000", 2024-09-14T06:32:58.664 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.99e-07 2024-09-14T06:32:58.664 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:58.664 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:58.664 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:32:58.664 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.083870+0000", 2024-09-14T06:32:58.664 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.782e-06 2024-09-14T06:32:58.664 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:58.664 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:58.664 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:32:58.664 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.083876+0000", 2024-09-14T06:32:58.664 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.3809999999999998e-06 2024-09-14T06:32:58.664 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:58.665 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:58.665 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:32:58.665 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.083882+0000", 2024-09-14T06:32:58.665 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.9479999999999996e-06 2024-09-14T06:32:58.665 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:58.665 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:58.665 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:32:58.665 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.125878+0000", 2024-09-14T06:32:58.665 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.041996264999999998 2024-09-14T06:32:58.665 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:58.665 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:58.665 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:32:58.666 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.125929+0000", 2024-09-14T06:32:58.666 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.0498999999999999e-05 2024-09-14T06:32:58.666 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:58.666 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:58.666 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:32:58.666 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.126035+0000", 2024-09-14T06:32:58.666 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000106298 2024-09-14T06:32:58.666 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:58.666 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:32:58.666 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:58.666 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:58.666 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:58.666 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:14921 3.0 3:05c39535:::rbd_data.112d92027590.0000000000000009:f [read 826368~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:32:58.666 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:26.083889+0000", 2024-09-14T06:32:58.667 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 212.57106416400001, 2024-09-14T06:32:58.667 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.042435309999999997, 2024-09-14T06:32:58.667 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:32:58.667 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:32:58.667 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:32:58.667 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:32:58.667 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:32:58.667 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:58.667 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 14921 2024-09-14T06:32:58.667 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:58.667 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:32:58.667 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:58.667 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:32:58.667 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.083889+0000", 2024-09-14T06:32:58.668 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:58.668 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:58.668 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:58.668 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:32:58.668 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.083889+0000", 2024-09-14T06:32:58.668 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:58.668 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:58.668 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:58.668 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:32:58.668 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.086033+0000", 2024-09-14T06:32:58.668 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0021444179999999999 2024-09-14T06:32:58.668 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:58.668 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:58.668 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:32:58.669 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.086038+0000", 2024-09-14T06:32:58.669 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.0969999999999999e-06 2024-09-14T06:32:58.669 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:58.669 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:58.669 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:32:58.669 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.086046+0000", 2024-09-14T06:32:58.669 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.9440000000000005e-06 2024-09-14T06:32:58.669 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:58.669 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:58.669 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:32:58.669 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.086050+0000", 2024-09-14T06:32:58.669 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.69e-06 2024-09-14T06:32:58.669 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:58.669 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:58.670 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:32:58.670 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.126080+0000", 2024-09-14T06:32:58.670 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.040030237000000003 2024-09-14T06:32:58.670 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:58.670 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:58.670 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:32:58.670 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.126163+0000", 2024-09-14T06:32:58.670 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.2071999999999996e-05 2024-09-14T06:32:58.670 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:58.670 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:58.670 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:32:58.670 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.126324+0000", 2024-09-14T06:32:58.670 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00016185200000000001 2024-09-14T06:32:58.670 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:58.671 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:32:58.671 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:58.671 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:58.671 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:58.671 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:55627 3.0 3:06490baf:::rbd_data.112dca2f6ae1.0000000000000030:17 [read 2002944~4096 [fadvise_nocache]] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:32:58.671 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:44.599966+0000", 2024-09-14T06:32:58.671 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 194.054987216, 2024-09-14T06:32:58.671 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.041868546999999999, 2024-09-14T06:32:58.671 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:32:58.671 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:32:58.671 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:32:58.671 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:32:58.671 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:32:58.671 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:58.672 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 55627 2024-09-14T06:32:58.672 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:58.672 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:32:58.672 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:58.672 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:32:58.672 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.599966+0000", 2024-09-14T06:32:58.672 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:58.672 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:58.672 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:58.672 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:32:58.672 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.599966+0000", 2024-09-14T06:32:58.672 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:58.672 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:58.673 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:58.673 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:32:58.673 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.599966+0000", 2024-09-14T06:32:58.673 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.0900000000000001e-07 2024-09-14T06:32:58.673 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:58.673 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:58.673 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:32:58.673 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.599967+0000", 2024-09-14T06:32:58.673 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.64e-07 2024-09-14T06:32:58.673 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:58.673 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:58.673 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:32:58.673 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.599970+0000", 2024-09-14T06:32:58.673 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.9670000000000001e-06 2024-09-14T06:32:58.674 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:58.674 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:58.674 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:32:58.674 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.599973+0000", 2024-09-14T06:32:58.674 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.632e-06 2024-09-14T06:32:58.674 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:58.674 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:58.674 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:32:58.674 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.641665+0000", 2024-09-14T06:32:58.674 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.041692547000000003 2024-09-14T06:32:58.674 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:58.674 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:58.674 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:32:58.674 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.641740+0000", 2024-09-14T06:32:58.675 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.4559000000000002e-05 2024-09-14T06:32:58.675 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:58.675 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:58.675 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:32:58.675 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.641835+0000", 2024-09-14T06:32:58.675 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.4668999999999998e-05 2024-09-14T06:32:58.675 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:58.675 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:32:58.675 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:58.675 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:58.675 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:58.675 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:55628 3.0 3:06490baf:::rbd_data.112dca2f6ae1.0000000000000030:17 [read 2027520~4096 [fadvise_nocache]] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:32:58.675 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:44.599976+0000", 2024-09-14T06:32:58.676 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 194.05497742899999, 2024-09-14T06:32:58.676 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.041991621, 2024-09-14T06:32:58.676 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:32:58.676 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:32:58.676 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:32:58.676 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:32:58.676 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:32:58.676 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:58.676 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 55628 2024-09-14T06:32:58.676 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:58.676 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:32:58.676 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:58.676 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:32:58.676 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.599976+0000", 2024-09-14T06:32:58.677 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:58.677 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:58.677 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:58.677 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:32:58.677 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.599976+0000", 2024-09-14T06:32:58.677 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:58.677 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:58.677 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:58.677 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:32:58.677 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.599976+0000", 2024-09-14T06:32:58.677 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.1299999999999999e-07 2024-09-14T06:32:58.677 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:58.677 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:58.678 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:32:58.678 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.599977+0000", 2024-09-14T06:32:58.678 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.7999999999999993e-07 2024-09-14T06:32:58.678 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:58.678 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:58.678 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:32:58.678 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.599980+0000", 2024-09-14T06:32:58.678 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.01e-06 2024-09-14T06:32:58.678 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:58.678 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:58.678 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:32:58.678 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.599983+0000", 2024-09-14T06:32:58.678 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.5409999999999999e-06 2024-09-14T06:32:58.678 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:58.678 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:58.679 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:32:58.679 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.641869+0000", 2024-09-14T06:32:58.679 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.041886235000000001 2024-09-14T06:32:58.679 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:58.679 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:58.679 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:32:58.679 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.641913+0000", 2024-09-14T06:32:58.679 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.4357000000000002e-05 2024-09-14T06:32:58.679 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:58.679 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:58.679 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:32:58.679 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.641967+0000", 2024-09-14T06:32:58.679 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.4284999999999999e-05 2024-09-14T06:32:58.679 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:58.680 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:32:58.680 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:58.680 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:58.680 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:58.680 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:55629 3.0 3:06490baf:::rbd_data.112dca2f6ae1.0000000000000030:17 [read 2052096~4096 [fadvise_nocache]] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:32:58.680 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:44.599985+0000", 2024-09-14T06:32:58.680 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 194.054967723, 2024-09-14T06:32:58.680 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.042183375000000002, 2024-09-14T06:32:58.680 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:32:58.680 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:32:58.680 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:32:58.680 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:32:58.680 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:32:58.681 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:58.681 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 55629 2024-09-14T06:32:58.681 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:58.681 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:32:58.681 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:58.681 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:32:58.681 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.599985+0000", 2024-09-14T06:32:58.681 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:58.681 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:58.681 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:58.681 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:32:58.681 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.599985+0000", 2024-09-14T06:32:58.681 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:58.682 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:58.682 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:58.682 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:32:58.682 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.599986+0000", 2024-09-14T06:32:58.682 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.05e-07 2024-09-14T06:32:58.682 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:58.682 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:58.682 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:32:58.682 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.599987+0000", 2024-09-14T06:32:58.682 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.8400000000000002e-07 2024-09-14T06:32:58.682 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:58.682 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:58.682 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:32:58.682 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.599990+0000", 2024-09-14T06:32:58.683 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.5640000000000001e-06 2024-09-14T06:32:58.683 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:58.683 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:58.683 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:32:58.683 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.599993+0000", 2024-09-14T06:32:58.683 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.615e-06 2024-09-14T06:32:58.683 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:58.683 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:58.683 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:32:58.683 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.642003+0000", 2024-09-14T06:32:58.683 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.042010345999999997 2024-09-14T06:32:58.683 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:58.683 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:58.683 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:32:58.684 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.642080+0000", 2024-09-14T06:32:58.684 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.6538999999999993e-05 2024-09-14T06:32:58.684 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:58.684 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:58.684 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:32:58.684 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.642169+0000", 2024-09-14T06:32:58.684 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.9122000000000005e-05 2024-09-14T06:32:58.684 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:58.684 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:32:58.684 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:58.684 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:58.684 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:58.684 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:55630 3.0 3:06490baf:::rbd_data.112dca2f6ae1.0000000000000030:17 [read 2076672~4096 [fadvise_nocache]] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:32:58.684 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:44.599996+0000", 2024-09-14T06:32:58.685 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 194.054957245, 2024-09-14T06:32:58.685 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.042299615999999998, 2024-09-14T06:32:58.685 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:32:58.685 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:32:58.685 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:32:58.685 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:32:58.685 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:32:58.685 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:58.685 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 55630 2024-09-14T06:32:58.685 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:58.685 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:32:58.685 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:58.686 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:32:58.686 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.599996+0000", 2024-09-14T06:32:58.686 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:58.686 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:58.686 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:58.686 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:32:58.686 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.599996+0000", 2024-09-14T06:32:58.686 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:58.686 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:58.686 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:58.686 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:32:58.686 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.599996+0000", 2024-09-14T06:32:58.686 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.1500000000000001e-07 2024-09-14T06:32:58.686 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:58.687 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:58.687 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:32:58.687 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.599997+0000", 2024-09-14T06:32:58.687 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.33e-07 2024-09-14T06:32:58.687 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:58.687 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:58.687 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:32:58.687 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600000+0000", 2024-09-14T06:32:58.687 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.0240000000000002e-06 2024-09-14T06:32:58.687 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:58.687 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:58.687 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:32:58.687 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600003+0000", 2024-09-14T06:32:58.687 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.3640000000000002e-06 2024-09-14T06:32:58.688 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:58.688 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:58.688 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:32:58.688 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.642200+0000", 2024-09-14T06:32:58.688 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.042197571000000003 2024-09-14T06:32:58.688 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:58.688 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:58.688 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:32:58.688 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.642243+0000", 2024-09-14T06:32:58.688 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.2765000000000001e-05 2024-09-14T06:32:58.688 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:58.688 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:58.688 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:32:58.689 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.642296+0000", 2024-09-14T06:32:58.689 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.2744000000000002e-05 2024-09-14T06:32:58.689 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:58.689 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:32:58.689 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:58.689 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:58.689 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:58.689 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:55631 3.0 3:06490baf:::rbd_data.112dca2f6ae1.0000000000000030:17 [read 1048576~4096 [fadvise_nocache]] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:32:58.689 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:44.600006+0000", 2024-09-14T06:32:58.689 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 194.05494765700001, 2024-09-14T06:32:58.689 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.042487905999999999, 2024-09-14T06:32:58.689 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:32:58.689 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:32:58.690 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:32:58.690 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:32:58.690 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:32:58.690 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:58.690 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 55631 2024-09-14T06:32:58.690 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:58.690 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:32:58.690 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:58.690 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:32:58.690 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600006+0000", 2024-09-14T06:32:58.690 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:58.690 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:58.690 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:58.690 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:32:58.691 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600006+0000", 2024-09-14T06:32:58.691 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:58.691 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:58.691 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:58.691 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:32:58.691 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600006+0000", 2024-09-14T06:32:58.691 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.0800000000000001e-07 2024-09-14T06:32:58.691 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:58.691 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:58.691 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:32:58.691 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600007+0000", 2024-09-14T06:32:58.691 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.7199999999999997e-07 2024-09-14T06:32:58.691 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:58.692 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:58.692 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:32:58.692 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600010+0000", 2024-09-14T06:32:58.692 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.0280000000000001e-06 2024-09-14T06:32:58.692 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:58.692 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:58.692 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:32:58.692 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600012+0000", 2024-09-14T06:32:58.692 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.475e-06 2024-09-14T06:32:58.692 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:58.692 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:58.692 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:32:58.692 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.642329+0000", 2024-09-14T06:32:58.692 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.042317157000000001 2024-09-14T06:32:58.693 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:58.693 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:58.693 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:32:58.693 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.642403+0000", 2024-09-14T06:32:58.693 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.3941000000000002e-05 2024-09-14T06:32:58.693 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:58.693 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:58.693 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:32:58.693 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.642493+0000", 2024-09-14T06:32:58.693 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.0124999999999995e-05 2024-09-14T06:32:58.693 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:58.693 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:32:58.693 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:58.693 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:58.694 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:58.694 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:55632 3.0 3:06490baf:::rbd_data.112dca2f6ae1.0000000000000030:17 [read 1073152~4096 [fadvise_nocache]] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:32:58.694 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:44.600015+0000", 2024-09-14T06:32:58.694 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 194.05493790400001, 2024-09-14T06:32:58.694 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.042604922000000003, 2024-09-14T06:32:58.694 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:32:58.694 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:32:58.694 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:32:58.694 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:32:58.694 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:32:58.694 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:58.694 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 55632 2024-09-14T06:32:58.694 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:58.695 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:32:58.695 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:58.695 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:32:58.695 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600015+0000", 2024-09-14T06:32:58.695 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:58.695 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:58.695 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:58.695 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:32:58.695 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600015+0000", 2024-09-14T06:32:58.695 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:58.695 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:58.695 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:58.695 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:32:58.695 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600016+0000", 2024-09-14T06:32:58.696 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.1e-07 2024-09-14T06:32:58.696 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:58.696 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:58.696 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:32:58.696 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600017+0000", 2024-09-14T06:32:58.696 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.9900000000000009e-07 2024-09-14T06:32:58.696 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:58.696 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:58.696 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:32:58.696 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600020+0000", 2024-09-14T06:32:58.696 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.4170000000000001e-06 2024-09-14T06:32:58.696 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:58.696 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:58.697 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:32:58.697 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600023+0000", 2024-09-14T06:32:58.697 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.588e-06 2024-09-14T06:32:58.697 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:58.697 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:58.697 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:32:58.697 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.642526+0000", 2024-09-14T06:32:58.697 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.042503236999999999 2024-09-14T06:32:58.697 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:58.697 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:58.697 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:32:58.697 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.642568+0000", 2024-09-14T06:32:58.697 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.1925000000000003e-05 2024-09-14T06:32:58.697 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:58.698 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:58.698 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:32:58.698 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.642620+0000", 2024-09-14T06:32:58.698 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.2546000000000001e-05 2024-09-14T06:32:58.698 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:58.698 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:32:58.698 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:58.698 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:58.698 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:58.698 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:55633 3.0 3:06490baf:::rbd_data.112dca2f6ae1.0000000000000030:17 [read 1097728~4096 [fadvise_nocache]] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:32:58.698 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:44.600026+0000", 2024-09-14T06:32:58.698 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 194.05492763500001, 2024-09-14T06:32:58.698 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.042797474000000002, 2024-09-14T06:32:58.699 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:32:58.699 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:32:58.699 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:32:58.699 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:32:58.699 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:32:58.699 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:58.699 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 55633 2024-09-14T06:32:58.699 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:58.699 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:32:58.699 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:58.699 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:32:58.699 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600026+0000", 2024-09-14T06:32:58.699 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:58.700 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:58.700 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:58.700 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:32:58.700 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600026+0000", 2024-09-14T06:32:58.700 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:58.700 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:58.700 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:58.700 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:32:58.700 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600026+0000", 2024-09-14T06:32:58.700 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.3099999999999999e-07 2024-09-14T06:32:58.700 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:58.700 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:58.700 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:32:58.700 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600029+0000", 2024-09-14T06:32:58.701 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.304e-06 2024-09-14T06:32:58.701 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:58.701 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:58.701 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:32:58.701 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600034+0000", 2024-09-14T06:32:58.701 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.1429999999999997e-06 2024-09-14T06:32:58.701 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:58.701 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:58.701 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:32:58.701 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600038+0000", 2024-09-14T06:32:58.701 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.692e-06 2024-09-14T06:32:58.701 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:58.701 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:58.702 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:32:58.702 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.642654+0000", 2024-09-14T06:32:58.702 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.042615564000000002 2024-09-14T06:32:58.702 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:58.702 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:58.702 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:32:58.702 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.642728+0000", 2024-09-14T06:32:58.702 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.4018999999999997e-05 2024-09-14T06:32:58.702 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:58.702 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:58.702 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:32:58.702 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.642823+0000", 2024-09-14T06:32:58.702 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.5520999999999996e-05 2024-09-14T06:32:58.703 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:58.703 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:32:58.703 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:58.703 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:58.703 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:58.703 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:55634 3.0 3:06490baf:::rbd_data.112dca2f6ae1.0000000000000030:17 [read 1122304~4096 [fadvise_nocache]] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:32:58.703 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:44.600042+0000", 2024-09-14T06:32:58.703 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 194.05491115800001, 2024-09-14T06:32:58.703 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.042903729000000002, 2024-09-14T06:32:58.703 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:32:58.703 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:32:58.703 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:32:58.703 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:32:58.703 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:32:58.704 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:58.704 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 55634 2024-09-14T06:32:58.704 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:58.704 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:32:58.704 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:58.704 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:32:58.704 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600042+0000", 2024-09-14T06:32:58.704 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:58.704 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:58.704 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:58.704 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:32:58.704 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600042+0000", 2024-09-14T06:32:58.704 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:58.705 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:58.705 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:58.705 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:32:58.705 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600043+0000", 2024-09-14T06:32:58.705 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.7000000000000005e-07 2024-09-14T06:32:58.705 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:58.705 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:58.705 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:32:58.705 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600044+0000", 2024-09-14T06:32:58.705 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.742e-06 2024-09-14T06:32:58.705 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:58.705 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:58.705 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:32:58.705 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600050+0000", 2024-09-14T06:32:58.706 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.0800000000000002e-06 2024-09-14T06:32:58.706 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:58.706 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:58.706 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:32:58.706 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600055+0000", 2024-09-14T06:32:58.706 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.3889999999999998e-06 2024-09-14T06:32:58.706 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:58.706 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:58.706 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:32:58.706 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.642845+0000", 2024-09-14T06:32:58.706 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.042790040000000001 2024-09-14T06:32:58.706 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:58.706 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:58.706 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:32:58.707 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.642888+0000", 2024-09-14T06:32:58.707 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.2790000000000002e-05 2024-09-14T06:32:58.707 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:58.707 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:58.707 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:32:58.707 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.642946+0000", 2024-09-14T06:32:58.707 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.8118000000000002e-05 2024-09-14T06:32:58.707 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:58.707 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:32:58.707 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:58.707 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:58.707 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:58.707 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:55635 3.0 3:06490baf:::rbd_data.112dca2f6ae1.0000000000000030:17 [read 1146880~4096 [fadvise_nocache]] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:32:58.707 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:44.600061+0000", 2024-09-14T06:32:58.708 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 194.05489258700001, 2024-09-14T06:32:58.708 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.043092695, 2024-09-14T06:32:58.708 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:32:58.708 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:32:58.708 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:32:58.708 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:32:58.708 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:32:58.708 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:58.708 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 55635 2024-09-14T06:32:58.708 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:58.708 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:32:58.708 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:58.708 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:32:58.709 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600061+0000", 2024-09-14T06:32:58.709 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:58.709 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:58.709 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:58.709 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:32:58.709 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600061+0000", 2024-09-14T06:32:58.709 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:58.709 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:58.709 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:58.709 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:32:58.709 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600061+0000", 2024-09-14T06:32:58.709 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.3099999999999999e-07 2024-09-14T06:32:58.709 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:58.709 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:58.710 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:32:58.710 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600063+0000", 2024-09-14T06:32:58.710 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.6220000000000001e-06 2024-09-14T06:32:58.710 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:58.710 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:58.710 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:32:58.710 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600068+0000", 2024-09-14T06:32:58.710 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.4790000000000004e-06 2024-09-14T06:32:58.710 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:58.710 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:58.710 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:32:58.710 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600072+0000", 2024-09-14T06:32:58.710 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.3100000000000002e-06 2024-09-14T06:32:58.710 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:58.711 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:58.711 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:32:58.711 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.642987+0000", 2024-09-14T06:32:58.711 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.042914489 2024-09-14T06:32:58.711 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:58.711 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:58.711 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:32:58.711 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.643062+0000", 2024-09-14T06:32:58.711 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.5011000000000004e-05 2024-09-14T06:32:58.711 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:58.711 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:58.711 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:32:58.711 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.643153+0000", 2024-09-14T06:32:58.711 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.1452999999999993e-05 2024-09-14T06:32:58.711 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:58.712 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:32:58.712 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:58.712 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:58.712 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:58.712 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:55636 3.0 3:06490baf:::rbd_data.112dca2f6ae1.0000000000000030:17 [read 1171456~4096 [fadvise_nocache]] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:32:58.712 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:44.600078+0000", 2024-09-14T06:32:58.712 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 194.05487517500001, 2024-09-14T06:32:58.712 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.043207966, 2024-09-14T06:32:58.712 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:32:58.712 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:32:58.712 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:32:58.712 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:32:58.712 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:32:58.713 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:58.713 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 55636 2024-09-14T06:32:58.713 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:58.713 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:32:58.713 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:58.713 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:32:58.713 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600078+0000", 2024-09-14T06:32:58.713 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:58.713 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:58.713 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:58.713 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:32:58.713 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600078+0000", 2024-09-14T06:32:58.713 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:58.713 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:58.714 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:58.714 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:32:58.714 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600078+0000", 2024-09-14T06:32:58.714 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.34e-07 2024-09-14T06:32:58.714 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:58.714 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:58.714 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:32:58.714 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600080+0000", 2024-09-14T06:32:58.714 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.641e-06 2024-09-14T06:32:58.714 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:58.714 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:58.714 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:32:58.714 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600086+0000", 2024-09-14T06:32:58.714 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.3860000000000004e-06 2024-09-14T06:32:58.715 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:58.715 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:58.715 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:32:58.715 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600091+0000", 2024-09-14T06:32:58.715 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.5970000000000002e-06 2024-09-14T06:32:58.715 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:58.715 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:58.715 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:32:58.715 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.643184+0000", 2024-09-14T06:32:58.715 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.043092893 2024-09-14T06:32:58.715 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:58.715 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:58.715 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:32:58.715 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.643228+0000", 2024-09-14T06:32:58.716 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.4536999999999997e-05 2024-09-14T06:32:58.716 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:58.716 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:58.716 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:32:58.716 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.643286+0000", 2024-09-14T06:32:58.716 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.7578000000000003e-05 2024-09-14T06:32:58.716 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:58.716 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:32:58.716 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:58.716 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:58.716 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:58.716 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:55637 3.0 3:06490baf:::rbd_data.112dca2f6ae1.0000000000000030:17 [read 1196032~4096 [fadvise_nocache]] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:32:58.716 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:44.600097+0000", 2024-09-14T06:32:58.717 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 194.05485661099999, 2024-09-14T06:32:58.717 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.043399124999999997, 2024-09-14T06:32:58.717 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:32:58.717 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:32:58.717 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:32:58.717 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:32:58.717 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:32:58.717 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:58.717 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 55637 2024-09-14T06:32:58.717 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:58.717 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:32:58.717 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:58.717 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:32:58.717 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600097+0000", 2024-09-14T06:32:58.718 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:58.718 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:58.718 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:58.718 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:32:58.718 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600097+0000", 2024-09-14T06:32:58.718 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:58.718 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:58.718 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:58.718 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:32:58.718 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.604372+0000", 2024-09-14T06:32:58.718 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0042751489999999998 2024-09-14T06:32:58.718 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:58.718 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:58.718 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:32:58.719 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.604377+0000", 2024-09-14T06:32:58.719 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.7899999999999999e-06 2024-09-14T06:32:58.719 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:58.719 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:58.719 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:32:58.719 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.604389+0000", 2024-09-14T06:32:58.719 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.2197e-05 2024-09-14T06:32:58.719 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:58.719 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:58.719 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:32:58.719 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.604397+0000", 2024-09-14T06:32:58.719 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.8730000000000007e-06 2024-09-14T06:32:58.719 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:58.719 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:58.720 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:32:58.720 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.643324+0000", 2024-09-14T06:32:58.720 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.038926897000000002 2024-09-14T06:32:58.720 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:58.720 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:58.720 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:32:58.720 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.643402+0000", 2024-09-14T06:32:58.720 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.8063000000000001e-05 2024-09-14T06:32:58.720 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:58.720 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:58.720 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:32:58.720 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.643496+0000", 2024-09-14T06:32:58.720 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.4155999999999998e-05 2024-09-14T06:32:58.720 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:58.721 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:32:58.721 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:58.721 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:58.721 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:58.721 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:55657 3.0 3:06490baf:::rbd_data.112dca2f6ae1.0000000000000030:17 [read 1687552~4096 [fadvise_nocache]] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:32:58.721 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:44.604841+0000", 2024-09-14T06:32:58.721 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 194.05011188200001, 2024-09-14T06:32:58.721 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.041890087999999999, 2024-09-14T06:32:58.721 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:32:58.721 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:32:58.721 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:32:58.721 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:32:58.721 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:32:58.721 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:58.722 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 55657 2024-09-14T06:32:58.722 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:58.722 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:32:58.722 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:58.722 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:32:58.722 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.604841+0000", 2024-09-14T06:32:58.722 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:58.722 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:58.722 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:58.722 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:32:58.722 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.604841+0000", 2024-09-14T06:32:58.722 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:58.722 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:58.722 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:58.723 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:32:58.723 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.604842+0000", 2024-09-14T06:32:58.723 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.0900000000000001e-07 2024-09-14T06:32:58.723 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:58.723 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:58.723 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:32:58.723 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.604846+0000", 2024-09-14T06:32:58.723 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.5759999999999999e-06 2024-09-14T06:32:58.723 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:58.723 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:58.723 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:32:58.723 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.604850+0000", 2024-09-14T06:32:58.723 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.405e-06 2024-09-14T06:32:58.724 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:58.724 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:58.724 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:32:58.724 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.604852+0000", 2024-09-14T06:32:58.724 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.8590000000000001e-06 2024-09-14T06:32:58.724 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:58.724 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:58.724 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:32:58.724 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.646633+0000", 2024-09-14T06:32:58.724 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.041780772000000001 2024-09-14T06:32:58.724 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:58.724 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:58.724 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:32:58.724 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.646677+0000", 2024-09-14T06:32:58.725 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.4115e-05 2024-09-14T06:32:58.725 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:58.725 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:58.725 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:32:58.725 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.646731+0000", 2024-09-14T06:32:58.725 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.4151999999999999e-05 2024-09-14T06:32:58.725 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:58.725 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:32:58.725 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:58.725 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:58.725 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:58.725 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:55658 3.0 3:06490baf:::rbd_data.112dca2f6ae1.0000000000000030:17 [read 1712128~4096 [fadvise_nocache]] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:32:58.725 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:44.604856+0000", 2024-09-14T06:32:58.725 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 194.050097282, 2024-09-14T06:32:58.726 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.041986227000000001, 2024-09-14T06:32:58.726 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:32:58.726 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:32:58.726 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:32:58.726 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:32:58.726 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:32:58.726 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:58.726 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 55658 2024-09-14T06:32:58.726 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:58.726 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:32:58.726 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:58.726 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:32:58.726 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.604856+0000", 2024-09-14T06:32:58.726 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:58.727 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:58.727 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:58.727 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:32:58.727 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.604856+0000", 2024-09-14T06:32:58.727 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:58.727 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:58.727 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:58.727 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:32:58.727 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.604856+0000", 2024-09-14T06:32:58.727 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.23e-07 2024-09-14T06:32:58.727 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:58.727 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:58.727 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:32:58.727 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.604857+0000", 2024-09-14T06:32:58.728 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.9999999999999995e-07 2024-09-14T06:32:58.728 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:58.728 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:58.728 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:32:58.728 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.604861+0000", 2024-09-14T06:32:58.728 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.9890000000000003e-06 2024-09-14T06:32:58.728 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:58.728 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:58.728 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:32:58.728 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.604864+0000", 2024-09-14T06:32:58.728 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.7429999999999998e-06 2024-09-14T06:32:58.728 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:58.728 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:58.728 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:32:58.729 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.646744+0000", 2024-09-14T06:32:58.729 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.041880325000000003 2024-09-14T06:32:58.729 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:58.729 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:58.729 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:32:58.729 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.646784+0000", 2024-09-14T06:32:58.729 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.9604000000000002e-05 2024-09-14T06:32:58.729 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:58.729 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:58.729 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:32:58.729 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.646842+0000", 2024-09-14T06:32:58.729 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.8343e-05 2024-09-14T06:32:58.729 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:58.730 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:32:58.730 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:58.730 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:58.730 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:58.730 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:55659 3.0 3:06490baf:::rbd_data.112dca2f6ae1.0000000000000030:17 [read 1736704~4096 [fadvise_nocache]] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:32:58.730 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:44.604867+0000", 2024-09-14T06:32:58.730 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 194.05008612899999, 2024-09-14T06:32:58.730 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.042097082000000001, 2024-09-14T06:32:58.730 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:32:58.730 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:32:58.730 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:32:58.730 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:32:58.730 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:32:58.730 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:58.731 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 55659 2024-09-14T06:32:58.731 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:58.731 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:32:58.731 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:58.731 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:32:58.731 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.604867+0000", 2024-09-14T06:32:58.731 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:58.731 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:58.731 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:58.731 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:32:58.731 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.604867+0000", 2024-09-14T06:32:58.731 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:58.731 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:58.731 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:58.731 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:32:58.732 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.604867+0000", 2024-09-14T06:32:58.732 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.2000000000000001e-07 2024-09-14T06:32:58.732 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:58.732 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:58.732 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:32:58.732 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.604868+0000", 2024-09-14T06:32:58.732 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.04e-06 2024-09-14T06:32:58.732 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:58.732 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:58.732 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:32:58.732 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.604871+0000", 2024-09-14T06:32:58.732 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.9689999999999999e-06 2024-09-14T06:32:58.732 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:58.733 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:58.733 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:32:58.733 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.604874+0000", 2024-09-14T06:32:58.733 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.5069999999999999e-06 2024-09-14T06:32:58.733 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:58.733 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:58.733 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:32:58.733 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.646874+0000", 2024-09-14T06:32:58.733 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.042000513000000003 2024-09-14T06:32:58.733 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:58.733 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:58.733 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:32:58.733 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.646914+0000", 2024-09-14T06:32:58.733 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.9443999999999997e-05 2024-09-14T06:32:58.734 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:58.734 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:58.734 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:32:58.734 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.646964+0000", 2024-09-14T06:32:58.734 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.0389000000000002e-05 2024-09-14T06:32:58.734 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:58.734 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:32:58.734 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:58.734 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:58.734 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:58.734 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:55660 3.0 3:06490baf:::rbd_data.112dca2f6ae1.0000000000000030:17 [read 1761280~4096 [fadvise_nocache]] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:32:58.734 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:44.604877+0000", 2024-09-14T06:32:58.734 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 194.05007638000001, 2024-09-14T06:32:58.734 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.042196349000000001, 2024-09-14T06:32:58.735 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:32:58.735 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:32:58.735 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:32:58.735 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:32:58.735 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:32:58.735 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:58.735 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 55660 2024-09-14T06:32:58.735 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:58.735 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:32:58.735 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:58.735 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:32:58.735 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.604877+0000", 2024-09-14T06:32:58.735 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:58.735 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:58.736 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:58.736 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:32:58.736 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.604877+0000", 2024-09-14T06:32:58.736 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:58.736 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:58.736 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:58.736 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:32:58.736 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.605810+0000", 2024-09-14T06:32:58.736 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00093336899999999997 2024-09-14T06:32:58.736 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:58.737 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:58.737 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:32:58.737 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.605815+0000", 2024-09-14T06:32:58.737 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.6140000000000002e-06 2024-09-14T06:32:58.737 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:58.737 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:58.737 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:32:58.737 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.605825+0000", 2024-09-14T06:32:58.737 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.8889999999999997e-06 2024-09-14T06:32:58.737 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:58.737 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:58.737 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:32:58.737 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.605831+0000", 2024-09-14T06:32:58.737 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.9800000000000003e-06 2024-09-14T06:32:58.738 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:58.738 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:58.738 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:32:58.738 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.646989+0000", 2024-09-14T06:32:58.738 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.041157878000000002 2024-09-14T06:32:58.738 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:58.738 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:58.738 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:32:58.738 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.647026+0000", 2024-09-14T06:32:58.738 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.7305000000000003e-05 2024-09-14T06:32:58.738 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:58.738 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:58.738 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:32:58.738 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.647073+0000", 2024-09-14T06:32:58.739 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.7314000000000001e-05 2024-09-14T06:32:58.739 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:58.739 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:32:58.739 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:58.739 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:58.739 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:58.739 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:236341 3.1c 3:3831950c:::rbd_directory:head [call rbd.dir_get_id in=13b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e233)", 2024-09-14T06:32:58.739 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:32:38.788743+0000", 2024-09-14T06:32:58.739 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 19.866210155000001, 2024-09-14T06:32:58.739 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.99031488700000003, 2024-09-14T06:32:58.739 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:32:58.739 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:32:58.739 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:32:58.739 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:32:58.740 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:32:58.740 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:58.740 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 236341 2024-09-14T06:32:58.740 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:58.740 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:32:58.740 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:58.740 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:32:58.740 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:38.788743+0000", 2024-09-14T06:32:58.740 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:58.741 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:58.741 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:58.741 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:32:58.741 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:38.788743+0000", 2024-09-14T06:32:58.741 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:58.741 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:58.741 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:58.741 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:32:58.741 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:38.788745+0000", 2024-09-14T06:32:58.741 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.623e-06 2024-09-14T06:32:58.741 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:58.742 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:58.742 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:32:58.742 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:38.788753+0000", 2024-09-14T06:32:58.742 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.8420000000000005e-06 2024-09-14T06:32:58.742 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:58.742 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:58.742 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:32:58.742 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:38.788877+0000", 2024-09-14T06:32:58.742 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00012487899999999999 2024-09-14T06:32:58.742 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:58.742 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:58.742 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:32:58.742 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:38.788886+0000", 2024-09-14T06:32:58.743 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.2379999999999997e-06 2024-09-14T06:32:58.743 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:58.743 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:58.743 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:32:58.743 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:38.788947+0000", 2024-09-14T06:32:58.743 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.1721000000000001e-05 2024-09-14T06:32:58.743 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:58.743 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:58.743 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "waiting for peered", 2024-09-14T06:32:58.743 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:38.788955+0000", 2024-09-14T06:32:58.743 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.2579999999999998e-06 2024-09-14T06:32:58.743 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:58.743 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:58.744 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:32:58.744 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:39.778650+0000", 2024-09-14T06:32:58.744 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.989694988 2024-09-14T06:32:58.744 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:58.744 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:58.744 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:32:58.744 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:39.778811+0000", 2024-09-14T06:32:58.744 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000161092 2024-09-14T06:32:58.744 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:58.744 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:58.744 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:32:58.744 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:39.779058+0000", 2024-09-14T06:32:58.744 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00024724600000000001 2024-09-14T06:32:58.745 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:58.745 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:32:58.745 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:58.745 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:58.745 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:58.745 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:238182 3.a 3:567df57c:::rbd_data.112d80ca42af.0000000000000007:head [delete] snapc 1f=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e243)", 2024-09-14T06:32:58.745 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:32:48.618828+0000", 2024-09-14T06:32:58.745 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 10.036125597, 2024-09-14T06:32:58.745 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.0662279320000001, 2024-09-14T06:32:58.745 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:32:58.745 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:32:58.745 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:32:58.745 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:32:58.745 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:32:58.746 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:32:58.746 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 238182 2024-09-14T06:32:58.746 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:58.746 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:32:58.746 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:58.746 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:32:58.746 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:48.618828+0000", 2024-09-14T06:32:58.746 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:58.746 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:58.746 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:58.746 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:32:58.746 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:48.618828+0000", 2024-09-14T06:32:58.746 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:32:58.746 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:58.747 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:58.747 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:32:58.747 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:48.618829+0000", 2024-09-14T06:32:58.747 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.362e-06 2024-09-14T06:32:58.747 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:58.747 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:58.747 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:32:58.747 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:48.618833+0000", 2024-09-14T06:32:58.747 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.4789999999999998e-06 2024-09-14T06:32:58.747 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:58.748 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:58.748 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:32:58.748 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:48.618841+0000", 2024-09-14T06:32:58.748 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.0260000000000007e-06 2024-09-14T06:32:58.748 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:58.748 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:58.748 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:32:58.748 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:48.618850+0000", 2024-09-14T06:32:58.748 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.8720000000000008e-06 2024-09-14T06:32:58.748 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:58.748 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:58.748 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:32:58.748 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:48.619393+0000", 2024-09-14T06:32:58.749 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00054264099999999996 2024-09-14T06:32:58.749 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:58.749 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:58.749 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "waiting for peered", 2024-09-14T06:32:58.749 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:48.619404+0000", 2024-09-14T06:32:58.749 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.1113e-05 2024-09-14T06:32:58.749 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:58.749 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:58.749 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:32:58.749 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:49.683016+0000", 2024-09-14T06:32:58.749 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.063612344 2024-09-14T06:32:58.749 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:58.749 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:58.749 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:32:58.750 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:49.683195+0000", 2024-09-14T06:32:58.750 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00017872100000000001 2024-09-14T06:32:58.750 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:58.750 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:58.750 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "waiting for subops from 2,3", 2024-09-14T06:32:58.750 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:49.683310+0000", 2024-09-14T06:32:58.750 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000114541 2024-09-14T06:32:58.750 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:58.750 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:58.750 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:32:58.750 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:49.684783+0000", 2024-09-14T06:32:58.750 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0014735359999999999 2024-09-14T06:32:58.750 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:58.750 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:58.751 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:32:58.751 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:49.684864+0000", 2024-09-14T06:32:58.751 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.1166000000000002e-05 2024-09-14T06:32:58.751 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:58.751 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:58.751 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "op_commit", 2024-09-14T06:32:58.751 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:49.684887+0000", 2024-09-14T06:32:58.751 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.2132000000000001e-05 2024-09-14T06:32:58.751 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:58.751 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:58.751 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-14T06:32:58.751 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:49.684929+0000", 2024-09-14T06:32:58.751 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.2141999999999997e-05 2024-09-14T06:32:58.751 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:32:58.752 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:32:58.752 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:32:58.752 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:49.685056+0000", 2024-09-14T06:32:58.752 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000126857 2024-09-14T06:32:58.752 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:58.752 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:32:58.752 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:58.752 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:32:58.752 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:32:58.752 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-14T06:32:58.754 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-14T06:32:58.754 INFO:tasks.ceph.osd.7.smithi138.stderr:2024-09-14T06:32:58.752+0000 7f3c6a21d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33623) UID: 0 2024-09-14T06:32:58.854 INFO:tasks.ceph.osd.5.smithi138.stderr:2024-09-14T06:32:58.853+0000 7fe13746f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33618) UID: 0 2024-09-14T06:32:58.865 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-14T06:32:58.865 INFO:teuthology.orchestra.run.smithi094.stdout: "ops": [], 2024-09-14T06:32:58.865 INFO:teuthology.orchestra.run.smithi094.stdout: "num_ops": 0 2024-09-14T06:32:58.865 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-14T06:32:58.874 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-14T06:32:58.955 INFO:tasks.ceph.osd.1.smithi094.stderr:2024-09-14T06:32:58.954+0000 7f328e47a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33641) UID: 0 2024-09-14T06:32:58.972 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-14T06:32:58.976 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-14T06:32:58.976 INFO:teuthology.orchestra.run.smithi094.stdout: "ops": [], 2024-09-14T06:32:58.977 INFO:teuthology.orchestra.run.smithi094.stdout: "complaint_time": 30, 2024-09-14T06:32:58.977 INFO:teuthology.orchestra.run.smithi094.stdout: "num_blocked_ops": 0 2024-09-14T06:32:58.977 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-14T06:32:58.986 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-14T06:32:59.071 INFO:tasks.ceph.osd.1.smithi094.stderr:2024-09-14T06:32:59.070+0000 7f328e47a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33641) UID: 0 2024-09-14T06:32:59.109 INFO:teuthology.orchestra.run.smithi094.stdout:osd.0: {} 2024-09-14T06:32:59.109 INFO:teuthology.orchestra.run.smithi094.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-14T06:32:59.126 INFO:teuthology.orchestra.run.smithi094.stdout:osd.1: {} 2024-09-14T06:32:59.126 INFO:teuthology.orchestra.run.smithi094.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-14T06:32:59.142 INFO:teuthology.orchestra.run.smithi094.stdout:osd.2: {} 2024-09-14T06:32:59.142 INFO:teuthology.orchestra.run.smithi094.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-14T06:32:59.142 INFO:teuthology.orchestra.run.smithi094.stdout:ERROR: (22) Invalid argument 2024-09-14T06:32:59.142 INFO:teuthology.orchestra.run.smithi094.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:32:59.151 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-14T06:32:59.158 INFO:teuthology.orchestra.run.smithi094.stdout:osd.3: {} 2024-09-14T06:32:59.158 INFO:teuthology.orchestra.run.smithi094.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-14T06:32:59.159 INFO:tasks.ceph.osd.1.smithi094.stderr:2024-09-14T06:32:59.157+0000 7f328e47a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33641) UID: 0 2024-09-14T06:32:59.175 INFO:teuthology.orchestra.run.smithi094.stdout:osd.4: {} 2024-09-14T06:32:59.176 INFO:teuthology.orchestra.run.smithi094.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-14T06:32:59.193 INFO:teuthology.orchestra.run.smithi094.stdout:osd.5: {} 2024-09-14T06:32:59.193 INFO:teuthology.orchestra.run.smithi094.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-14T06:32:59.210 INFO:teuthology.orchestra.run.smithi094.stdout:osd.6: {} 2024-09-14T06:32:59.210 INFO:teuthology.orchestra.run.smithi094.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-14T06:32:59.228 INFO:teuthology.orchestra.run.smithi094.stdout:osd.7: {} 2024-09-14T06:32:59.228 INFO:teuthology.orchestra.run.smithi094.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-14T06:32:59.258 INFO:tasks.ceph.osd.4.smithi138.stderr:2024-09-14T06:32:59.257+0000 7f6c653c8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33615) UID: 0 2024-09-14T06:32:59.263 INFO:teuthology.orchestra.run.smithi094.stdout:ERROR: (22) Invalid argument 2024-09-14T06:32:59.263 INFO:teuthology.orchestra.run.smithi094.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:32:59.272 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-14T06:32:59.359 INFO:tasks.ceph.osd.7.smithi138.stderr:2024-09-14T06:32:59.357+0000 7f3c6a21d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33623) UID: 0 2024-09-14T06:32:59.415 INFO:teuthology.orchestra.run.smithi094.stdout:ERROR: (22) Invalid argument 2024-09-14T06:32:59.416 INFO:teuthology.orchestra.run.smithi094.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:32:59.424 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-14T06:32:59.459 INFO:tasks.ceph.osd.4.smithi138.stderr:2024-09-14T06:32:59.457+0000 7f6c653c8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33615) UID: 0 2024-09-14T06:32:59.526 INFO:teuthology.orchestra.run.smithi094.stdout:ERROR: (22) Invalid argument 2024-09-14T06:32:59.526 INFO:teuthology.orchestra.run.smithi094.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:32:59.535 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-14T06:32:59.560 INFO:tasks.ceph.osd.0.smithi094.stderr:2024-09-14T06:32:59.559+0000 7f07c9333640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33636) UID: 0 2024-09-14T06:32:59.651 INFO:teuthology.orchestra.run.smithi138.stdout:ERROR: (22) Invalid argument 2024-09-14T06:32:59.651 INFO:teuthology.orchestra.run.smithi138.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:32:59.660 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-14T06:32:59.661 INFO:tasks.ceph.osd.7.smithi138.stderr:2024-09-14T06:32:59.659+0000 7f3c6a21d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33623) UID: 0 2024-09-14T06:32:59.668 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 4, 5, 7, 3] out_osds: [6] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-14T06:32:59.668 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-14T06:32:59.668 INFO:tasks.thrashosds.thrasher:Removing osd 2, in_osds are: [0, 1, 2, 4, 5, 7, 3] 2024-09-14T06:32:59.668 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd out 2 2024-09-14T06:32:59.727 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-14T06:32:59.762 INFO:tasks.ceph.osd.5.smithi138.stderr:2024-09-14T06:32:59.760+0000 7fe13746f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33618) UID: 0 2024-09-14T06:32:59.771 INFO:teuthology.orchestra.run.smithi138.stdout:ERROR: (22) Invalid argument 2024-09-14T06:32:59.771 INFO:teuthology.orchestra.run.smithi138.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:32:59.781 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-14T06:32:59.863 INFO:tasks.ceph.osd.1.smithi094.stderr:2024-09-14T06:32:59.862+0000 7f328e47a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33641) UID: 0 2024-09-14T06:32:59.924 INFO:teuthology.orchestra.run.smithi138.stdout:ERROR: (22) Invalid argument 2024-09-14T06:32:59.924 INFO:teuthology.orchestra.run.smithi138.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:32:59.933 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-14T06:32:59.976 INFO:tasks.ceph.osd.4.smithi138.stderr:2024-09-14T06:32:59.974+0000 7f6c653c8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33615) UID: 0 2024-09-14T06:32:59.985 INFO:tasks.ceph.mon.a.smithi094.stderr:2024-09-14T06:32:59.984+0000 7f3f97784640 -1 mon.a@0(leader).osd e254 definitely_dead 0 2024-09-14T06:33:00.035 INFO:teuthology.orchestra.run.smithi138.stdout:ERROR: (22) Invalid argument 2024-09-14T06:33:00.035 INFO:teuthology.orchestra.run.smithi138.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:33:00.044 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-14T06:33:00.085 INFO:tasks.ceph.osd.4.smithi138.stderr:2024-09-14T06:33:00.083+0000 7f6c653c8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33615) UID: 0 2024-09-14T06:33:00.166 INFO:tasks.ceph.osd.3.smithi094.stderr:2024-09-14T06:33:00.165+0000 7feeeafe8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33655) UID: 0 2024-09-14T06:33:00.186 INFO:teuthology.orchestra.run.smithi138.stdout:ERROR: (22) Invalid argument 2024-09-14T06:33:00.186 INFO:teuthology.orchestra.run.smithi138.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:33:00.195 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-14T06:33:00.267 INFO:tasks.ceph.osd.0.smithi094.stderr:2024-09-14T06:33:00.265+0000 7f07c9333640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33636) UID: 0 2024-09-14T06:33:00.298 INFO:teuthology.orchestra.run.smithi138.stdout:ERROR: (22) Invalid argument 2024-09-14T06:33:00.298 INFO:teuthology.orchestra.run.smithi138.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:33:00.307 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-14T06:33:00.367 INFO:tasks.ceph.osd.6.smithi138.stderr:2024-09-14T06:33:00.365+0000 7f8cd8849640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33620) UID: 0 2024-09-14T06:33:00.409 INFO:teuthology.orchestra.run.smithi138.stdout:ERROR: (22) Invalid argument 2024-09-14T06:33:00.409 INFO:teuthology.orchestra.run.smithi138.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:33:00.418 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-14T06:33:00.502 INFO:tasks.ceph.osd.5.smithi138.stderr:2024-09-14T06:33:00.500+0000 7fe13746f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33618) UID: 0 2024-09-14T06:33:00.561 INFO:teuthology.orchestra.run.smithi138.stdout:ERROR: (22) Invalid argument 2024-09-14T06:33:00.561 INFO:teuthology.orchestra.run.smithi138.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:33:00.569 INFO:tasks.ceph.osd.1.smithi094.stderr:2024-09-14T06:33:00.568+0000 7f328e47a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33641) UID: 0 2024-09-14T06:33:00.570 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-14T06:33:00.616 INFO:teuthology.orchestra.run.smithi094.stderr:noscrub is set 2024-09-14T06:33:00.620 INFO:teuthology.orchestra.run.smithi094.stderr:marked out osd.2. 2024-09-14T06:33:00.680 INFO:tasks.ceph.osd.2.smithi094.stderr:2024-09-14T06:33:00.679+0000 7faf3269b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33649) UID: 0 2024-09-14T06:33:00.731 INFO:teuthology.orchestra.run.smithi138.stdout:ERROR: (22) Invalid argument 2024-09-14T06:33:00.731 INFO:teuthology.orchestra.run.smithi138.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:33:00.741 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-14T06:33:00.771 INFO:tasks.ceph.osd.0.smithi094.stderr:2024-09-14T06:33:00.770+0000 7f07c9333640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33636) UID: 0 2024-09-14T06:33:00.845 INFO:teuthology.orchestra.run.smithi138.stdout:ERROR: (22) Invalid argument 2024-09-14T06:33:00.845 INFO:teuthology.orchestra.run.smithi138.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:33:00.853 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-14T06:33:00.872 INFO:tasks.ceph.osd.3.smithi094.stderr:2024-09-14T06:33:00.871+0000 7feeeafe8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33655) UID: 0 2024-09-14T06:33:00.956 INFO:teuthology.orchestra.run.smithi138.stdout:ERROR: (22) Invalid argument 2024-09-14T06:33:00.956 INFO:teuthology.orchestra.run.smithi138.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:33:00.965 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-14T06:33:01.009 INFO:tasks.ceph.osd.4.smithi138.stderr:2024-09-14T06:33:01.007+0000 7f6c653c8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33615) UID: 0 2024-09-14T06:33:01.068 INFO:teuthology.orchestra.run.smithi138.stdout:ERROR: (22) Invalid argument 2024-09-14T06:33:01.068 INFO:teuthology.orchestra.run.smithi138.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:33:01.073 INFO:tasks.ceph.osd.1.smithi094.stderr:2024-09-14T06:33:01.072+0000 7f328e47a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33641) UID: 0 2024-09-14T06:33:01.076 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-14T06:33:01.175 INFO:tasks.ceph.osd.4.smithi138.stderr:2024-09-14T06:33:01.173+0000 7f6c653c8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33615) UID: 0 2024-09-14T06:33:01.187 INFO:teuthology.orchestra.run.smithi094.stdout:ERROR: (22) Invalid argument 2024-09-14T06:33:01.188 INFO:teuthology.orchestra.run.smithi094.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:33:01.197 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-14T06:33:01.240 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-14T06:33:01.275 INFO:tasks.ceph.osd.4.smithi138.stderr:2024-09-14T06:33:01.274+0000 7f6c653c8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33615) UID: 0 2024-09-14T06:33:01.301 INFO:teuthology.orchestra.run.smithi094.stdout:ERROR: (22) Invalid argument 2024-09-14T06:33:01.301 INFO:teuthology.orchestra.run.smithi094.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:33:01.311 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-14T06:33:01.376 INFO:tasks.ceph.osd.7.smithi138.stderr:2024-09-14T06:33:01.374+0000 7f3c6a21d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33623) UID: 0 2024-09-14T06:33:01.379 INFO:teuthology.orchestra.run.smithi094.stdout:osd.0: {} 2024-09-14T06:33:01.380 INFO:teuthology.orchestra.run.smithi094.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-14T06:33:01.399 INFO:teuthology.orchestra.run.smithi094.stdout:osd.1: {} 2024-09-14T06:33:01.399 INFO:teuthology.orchestra.run.smithi094.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-14T06:33:01.415 INFO:teuthology.orchestra.run.smithi094.stdout:osd.2: {} 2024-09-14T06:33:01.416 INFO:teuthology.orchestra.run.smithi094.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-14T06:33:01.432 INFO:teuthology.orchestra.run.smithi094.stdout:osd.3: {} 2024-09-14T06:33:01.432 INFO:teuthology.orchestra.run.smithi094.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-14T06:33:01.433 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-14T06:33:01.433 INFO:teuthology.orchestra.run.smithi094.stdout: "size": 20, 2024-09-14T06:33:01.433 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 600, 2024-09-14T06:33:01.433 INFO:teuthology.orchestra.run.smithi094.stdout: "ops": [ 2024-09-14T06:33:01.433 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:01.434 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:115193 3.1 3:856be886:::rbd_data.112d4d24c558.0000000000000004:29 [read 1523712~16384 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e96)", 2024-09-14T06:33:01.434 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:30:17.939854+0000", 2024-09-14T06:33:01.434 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 163.49097098600001, 2024-09-14T06:33:01.434 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.073897516999999996, 2024-09-14T06:33:01.434 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:33:01.434 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:33:01.434 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:33:01.434 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:33:01.434 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:33:01.434 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:33:01.434 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 115193 2024-09-14T06:33:01.434 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:01.434 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:33:01.434 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:01.435 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:33:01.435 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.939854+0000", 2024-09-14T06:33:01.435 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:33:01.435 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:01.435 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:01.435 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:33:01.435 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.939854+0000", 2024-09-14T06:33:01.435 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:33:01.435 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:01.435 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:01.435 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:33:01.435 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.939854+0000", 2024-09-14T06:33:01.435 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.9500000000000002e-07 2024-09-14T06:33:01.435 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:01.435 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:01.436 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:33:01.436 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.939856+0000", 2024-09-14T06:33:01.436 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.0959999999999999e-06 2024-09-14T06:33:01.436 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:01.436 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:01.436 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:33:01.436 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.939863+0000", 2024-09-14T06:33:01.436 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.1129999999999997e-06 2024-09-14T06:33:01.436 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:01.436 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:01.436 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:33:01.436 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.939867+0000", 2024-09-14T06:33:01.436 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.8350000000000003e-06 2024-09-14T06:33:01.436 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:01.436 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:01.437 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:33:01.437 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.013626+0000", 2024-09-14T06:33:01.437 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.073758799999999999 2024-09-14T06:33:01.437 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:01.437 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:01.437 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:33:01.437 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.013690+0000", 2024-09-14T06:33:01.437 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.3918999999999996e-05 2024-09-14T06:33:01.437 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:01.437 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:01.437 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:33:01.437 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.013751+0000", 2024-09-14T06:33:01.437 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.1159000000000002e-05 2024-09-14T06:33:01.437 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:33:01.438 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:33:01.438 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:33:01.438 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:01.438 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:01.438 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:115194 3.1 3:856be886:::rbd_data.112d4d24c558.0000000000000004:29 [read 1769472~16384 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e96)", 2024-09-14T06:33:01.438 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:30:17.939874+0000", 2024-09-14T06:33:01.438 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 163.49095067499999, 2024-09-14T06:33:01.438 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.074016988000000006, 2024-09-14T06:33:01.438 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:33:01.438 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:33:01.438 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:33:01.438 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:33:01.438 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:33:01.438 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:33:01.439 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 115194 2024-09-14T06:33:01.439 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:01.439 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:33:01.439 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:01.439 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:33:01.439 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.939874+0000", 2024-09-14T06:33:01.439 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:33:01.439 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:01.439 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:01.439 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:33:01.439 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.939874+0000", 2024-09-14T06:33:01.439 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:33:01.439 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:01.440 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:01.440 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:33:01.440 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.939875+0000", 2024-09-14T06:33:01.440 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.2600000000000002e-07 2024-09-14T06:33:01.440 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:01.440 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:01.440 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:33:01.440 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.939877+0000", 2024-09-14T06:33:01.440 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.9420000000000002e-06 2024-09-14T06:33:01.440 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:01.440 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:01.440 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:33:01.441 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.939882+0000", 2024-09-14T06:33:01.441 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.8749999999999997e-06 2024-09-14T06:33:01.441 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:01.441 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:01.441 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:33:01.441 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.939887+0000", 2024-09-14T06:33:01.441 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.741e-06 2024-09-14T06:33:01.441 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:01.441 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:01.441 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:33:01.441 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.013767+0000", 2024-09-14T06:33:01.441 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.073879720999999995 2024-09-14T06:33:01.441 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:01.442 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:01.442 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:33:01.442 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.013832+0000", 2024-09-14T06:33:01.442 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.5312000000000001e-05 2024-09-14T06:33:01.442 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:01.442 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:01.442 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:33:01.442 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.013891+0000", 2024-09-14T06:33:01.442 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.8870999999999999e-05 2024-09-14T06:33:01.442 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:33:01.442 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:33:01.442 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:33:01.442 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:01.443 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:01.443 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:115195 3.1 3:856be886:::rbd_data.112d4d24c558.0000000000000004:29 [read 2015232~16384 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e96)", 2024-09-14T06:33:01.443 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:30:17.939903+0000", 2024-09-14T06:33:01.443 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 163.49092149000001, 2024-09-14T06:33:01.443 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.074128199000000006, 2024-09-14T06:33:01.443 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:33:01.443 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:33:01.443 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:33:01.443 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:33:01.443 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:33:01.443 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:33:01.443 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 115195 2024-09-14T06:33:01.443 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:01.444 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:33:01.444 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:01.444 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:33:01.444 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.939903+0000", 2024-09-14T06:33:01.444 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:33:01.444 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:01.444 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:01.444 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:33:01.444 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.939903+0000", 2024-09-14T06:33:01.444 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:33:01.444 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:01.444 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:01.444 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:33:01.445 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.939904+0000", 2024-09-14T06:33:01.445 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.5799999999999999e-07 2024-09-14T06:33:01.445 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:01.445 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:01.445 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:33:01.445 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.939906+0000", 2024-09-14T06:33:01.445 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.2350000000000002e-06 2024-09-14T06:33:01.445 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:01.445 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:01.445 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:33:01.445 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.939913+0000", 2024-09-14T06:33:01.445 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.5649999999999998e-06 2024-09-14T06:33:01.445 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:01.445 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:01.446 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:33:01.446 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.939918+0000", 2024-09-14T06:33:01.446 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.784e-06 2024-09-14T06:33:01.446 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:01.446 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:01.446 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:33:01.446 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.013921+0000", 2024-09-14T06:33:01.446 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.074002911000000005 2024-09-14T06:33:01.446 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:01.446 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:01.446 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:33:01.446 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.013971+0000", 2024-09-14T06:33:01.446 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.9551e-05 2024-09-14T06:33:01.447 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:01.447 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:01.447 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:33:01.447 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.014031+0000", 2024-09-14T06:33:01.447 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.0594999999999998e-05 2024-09-14T06:33:01.447 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:33:01.447 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:33:01.447 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:33:01.447 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:01.447 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:01.447 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:115196 3.1 3:856be886:::rbd_data.112d4d24c558.0000000000000004:29 [read 2260992~16384 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e96)", 2024-09-14T06:33:01.447 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:30:17.939925+0000", 2024-09-14T06:33:01.447 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 163.49089929799999, 2024-09-14T06:33:01.447 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.074320137999999994, 2024-09-14T06:33:01.448 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:33:01.448 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:33:01.448 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:33:01.448 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:33:01.448 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:33:01.448 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:33:01.448 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 115196 2024-09-14T06:33:01.448 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:01.448 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:33:01.448 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:01.448 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:33:01.448 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.939925+0000", 2024-09-14T06:33:01.448 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:33:01.449 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:01.449 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:01.449 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:33:01.449 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.939925+0000", 2024-09-14T06:33:01.449 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:33:01.449 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:01.449 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:01.449 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:33:01.449 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.939926+0000", 2024-09-14T06:33:01.449 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.9999999999999998e-07 2024-09-14T06:33:01.449 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:01.449 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:01.449 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:33:01.450 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.939928+0000", 2024-09-14T06:33:01.450 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.9549999999999999e-06 2024-09-14T06:33:01.450 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:01.450 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:01.450 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:33:01.450 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.939934+0000", 2024-09-14T06:33:01.450 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.127e-06 2024-09-14T06:33:01.450 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:01.450 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:01.450 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:33:01.450 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.939939+0000", 2024-09-14T06:33:01.450 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.775e-06 2024-09-14T06:33:01.450 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:01.450 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:01.451 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:33:01.451 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.014075+0000", 2024-09-14T06:33:01.451 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.074136598999999997 2024-09-14T06:33:01.451 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:01.451 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:01.451 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:33:01.451 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.014162+0000", 2024-09-14T06:33:01.451 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.6477999999999994e-05 2024-09-14T06:33:01.451 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:01.451 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:01.451 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:33:01.451 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.014246+0000", 2024-09-14T06:33:01.452 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.3703999999999997e-05 2024-09-14T06:33:01.452 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:33:01.452 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:33:01.452 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:33:01.452 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:01.452 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:01.452 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:115197 3.1 3:856be886:::rbd_data.112d4d24c558.0000000000000004:29 [read 2506752~16384 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e96)", 2024-09-14T06:33:01.452 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:30:17.939969+0000", 2024-09-14T06:33:01.452 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 163.490855813, 2024-09-14T06:33:01.452 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.074528544000000002, 2024-09-14T06:33:01.452 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:33:01.452 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:33:01.452 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:33:01.452 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:33:01.453 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:33:01.453 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:33:01.453 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 115197 2024-09-14T06:33:01.453 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:01.453 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:33:01.453 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:01.453 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:33:01.453 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.939969+0000", 2024-09-14T06:33:01.453 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:33:01.453 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:01.453 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:01.453 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:33:01.454 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.939969+0000", 2024-09-14T06:33:01.454 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:33:01.454 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:01.454 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:01.454 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:33:01.454 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.939970+0000", 2024-09-14T06:33:01.454 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.3600000000000003e-07 2024-09-14T06:33:01.454 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:01.454 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:01.454 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:33:01.454 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.939972+0000", 2024-09-14T06:33:01.454 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.3180000000000001e-06 2024-09-14T06:33:01.454 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:01.454 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:01.455 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:33:01.455 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.939979+0000", 2024-09-14T06:33:01.455 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.8800000000000002e-06 2024-09-14T06:33:01.455 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:01.455 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:01.455 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:33:01.455 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.939984+0000", 2024-09-14T06:33:01.455 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.3510000000000001e-06 2024-09-14T06:33:01.455 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:01.455 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:01.455 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:33:01.455 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.014302+0000", 2024-09-14T06:33:01.455 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.074317416999999997 2024-09-14T06:33:01.456 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:01.456 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:01.456 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:33:01.456 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.014392+0000", 2024-09-14T06:33:01.456 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.0181999999999998e-05 2024-09-14T06:33:01.456 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:01.456 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:01.456 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:33:01.456 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.014497+0000", 2024-09-14T06:33:01.456 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00010576 2024-09-14T06:33:01.456 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:33:01.456 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:33:01.456 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:33:01.456 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:01.457 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:01.457 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:115198 3.1 3:856be886:::rbd_data.112d4d24c558.0000000000000004:29 [read 2752512~16384 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e96)", 2024-09-14T06:33:01.457 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:30:17.940004+0000", 2024-09-14T06:33:01.457 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 163.490820533, 2024-09-14T06:33:01.457 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.074736367999999997, 2024-09-14T06:33:01.457 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:33:01.457 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:33:01.457 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:33:01.457 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:33:01.457 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:33:01.457 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:33:01.457 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 115198 2024-09-14T06:33:01.457 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:01.458 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:33:01.458 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:01.458 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:33:01.458 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940004+0000", 2024-09-14T06:33:01.458 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:33:01.458 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:01.458 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:01.458 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:33:01.458 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940004+0000", 2024-09-14T06:33:01.458 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:33:01.458 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:01.458 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:01.458 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:33:01.458 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940005+0000", 2024-09-14T06:33:01.459 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.2200000000000004e-07 2024-09-14T06:33:01.459 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:01.459 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:01.459 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:33:01.459 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940007+0000", 2024-09-14T06:33:01.459 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.322e-06 2024-09-14T06:33:01.459 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:01.459 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:01.459 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:33:01.459 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940014+0000", 2024-09-14T06:33:01.459 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.6220000000000003e-06 2024-09-14T06:33:01.459 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:01.460 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:01.460 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:33:01.460 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940019+0000", 2024-09-14T06:33:01.460 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.6380000000000001e-06 2024-09-14T06:33:01.460 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:01.460 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:01.460 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:33:01.460 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.014562+0000", 2024-09-14T06:33:01.460 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.074542320999999995 2024-09-14T06:33:01.460 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:01.460 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:01.460 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:33:01.460 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.014646+0000", 2024-09-14T06:33:01.460 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.4079999999999995e-05 2024-09-14T06:33:01.461 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:01.461 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:01.461 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:33:01.461 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.014741+0000", 2024-09-14T06:33:01.461 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.4763000000000001e-05 2024-09-14T06:33:01.461 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:33:01.461 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:33:01.461 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:33:01.461 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:01.461 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:01.461 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:115199 3.1 3:856be886:::rbd_data.112d4d24c558.0000000000000004:29 [read 2998272~16384 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e96)", 2024-09-14T06:33:01.461 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:30:17.940040+0000", 2024-09-14T06:33:01.461 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 163.49078477800001, 2024-09-14T06:33:01.462 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.074957134999999994, 2024-09-14T06:33:01.462 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:33:01.462 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:33:01.462 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:33:01.462 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:33:01.462 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:33:01.462 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:33:01.462 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 115199 2024-09-14T06:33:01.462 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:01.462 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:33:01.462 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:01.462 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:33:01.462 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940040+0000", 2024-09-14T06:33:01.463 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:33:01.463 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:01.463 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:01.463 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:33:01.463 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940040+0000", 2024-09-14T06:33:01.463 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:33:01.463 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:01.463 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:01.463 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:33:01.463 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940041+0000", 2024-09-14T06:33:01.463 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.4000000000000001e-07 2024-09-14T06:33:01.463 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:01.463 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:01.464 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:33:01.464 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940046+0000", 2024-09-14T06:33:01.464 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.7250000000000002e-06 2024-09-14T06:33:01.464 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:01.464 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:01.464 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:33:01.464 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940053+0000", 2024-09-14T06:33:01.464 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.6109999999999999e-06 2024-09-14T06:33:01.464 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:01.464 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:01.464 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:33:01.464 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940058+0000", 2024-09-14T06:33:01.464 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.1590000000000001e-06 2024-09-14T06:33:01.465 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:01.465 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:01.465 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:33:01.465 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.014768+0000", 2024-09-14T06:33:01.465 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.074709624000000002 2024-09-14T06:33:01.465 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:01.465 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:01.465 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:33:01.465 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.014907+0000", 2024-09-14T06:33:01.465 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000139665 2024-09-14T06:33:01.465 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:01.465 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:01.465 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:33:01.466 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.014997+0000", 2024-09-14T06:33:01.466 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.9710999999999995e-05 2024-09-14T06:33:01.466 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:33:01.466 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:33:01.466 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:33:01.466 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:01.466 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:01.466 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:115200 3.1 3:856be886:::rbd_data.112d4d24c558.0000000000000004:29 [read 3244032~16384 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e96)", 2024-09-14T06:33:01.466 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:30:17.940080+0000", 2024-09-14T06:33:01.466 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 163.490744573, 2024-09-14T06:33:01.466 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.075137291999999994, 2024-09-14T06:33:01.466 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:33:01.466 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:33:01.466 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:33:01.466 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:33:01.467 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:33:01.467 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:33:01.467 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 115200 2024-09-14T06:33:01.467 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:01.467 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:33:01.467 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:01.467 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:33:01.467 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940080+0000", 2024-09-14T06:33:01.467 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:33:01.467 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:01.467 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:01.467 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:33:01.467 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940080+0000", 2024-09-14T06:33:01.468 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:33:01.468 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:01.468 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:01.468 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:33:01.468 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940081+0000", 2024-09-14T06:33:01.468 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.0800000000000004e-07 2024-09-14T06:33:01.468 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:01.468 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:01.468 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:33:01.468 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940083+0000", 2024-09-14T06:33:01.468 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.3920000000000001e-06 2024-09-14T06:33:01.468 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:01.468 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:01.468 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:33:01.468 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940091+0000", 2024-09-14T06:33:01.469 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.1389999999999995e-06 2024-09-14T06:33:01.469 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:01.469 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:01.469 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:33:01.469 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940097+0000", 2024-09-14T06:33:01.469 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.5550000000000002e-06 2024-09-14T06:33:01.469 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:01.469 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:01.469 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:33:01.469 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.015025+0000", 2024-09-14T06:33:01.469 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.074928579999999995 2024-09-14T06:33:01.469 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:01.469 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:01.470 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:33:01.470 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.015096+0000", 2024-09-14T06:33:01.470 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.0705999999999997e-05 2024-09-14T06:33:01.470 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:01.470 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:01.470 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:33:01.470 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.015217+0000", 2024-09-14T06:33:01.470 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00012131200000000001 2024-09-14T06:33:01.470 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:33:01.470 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:33:01.470 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:33:01.470 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:01.470 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:01.470 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:115201 3.1 3:856be886:::rbd_data.112d4d24c558.0000000000000004:29 [read 3489792~16384 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e96)", 2024-09-14T06:33:01.470 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:30:17.940113+0000", 2024-09-14T06:33:01.471 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 163.49071180000001, 2024-09-14T06:33:01.471 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.075246899000000006, 2024-09-14T06:33:01.471 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:33:01.471 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:33:01.471 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:33:01.471 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:33:01.471 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:33:01.471 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:33:01.471 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 115201 2024-09-14T06:33:01.471 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:01.471 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:33:01.471 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:01.471 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:33:01.472 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940113+0000", 2024-09-14T06:33:01.472 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:33:01.472 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:01.472 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:01.472 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:33:01.472 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940113+0000", 2024-09-14T06:33:01.472 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:33:01.472 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:01.472 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:01.472 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:33:01.472 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940114+0000", 2024-09-14T06:33:01.472 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.1500000000000004e-07 2024-09-14T06:33:01.472 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:01.472 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:01.473 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:33:01.473 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940116+0000", 2024-09-14T06:33:01.473 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.2809999999999998e-06 2024-09-14T06:33:01.473 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:01.473 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:01.473 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:33:01.473 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940123+0000", 2024-09-14T06:33:01.473 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.9229999999999996e-06 2024-09-14T06:33:01.473 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:01.473 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:01.473 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:33:01.473 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940128+0000", 2024-09-14T06:33:01.473 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.7790000000000003e-06 2024-09-14T06:33:01.474 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:01.474 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:01.474 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:33:01.474 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.015253+0000", 2024-09-14T06:33:01.474 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.075125792999999996 2024-09-14T06:33:01.474 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:01.474 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:01.474 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:33:01.474 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.015297+0000", 2024-09-14T06:33:01.474 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.3847000000000002e-05 2024-09-14T06:33:01.474 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:01.474 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:01.474 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:33:01.474 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.015360+0000", 2024-09-14T06:33:01.475 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.2661000000000002e-05 2024-09-14T06:33:01.475 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:33:01.475 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:33:01.475 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:33:01.475 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:01.475 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:01.475 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:115202 3.1 3:856be886:::rbd_data.112d4d24c558.0000000000000004:29 [read 3735552~16384 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e96)", 2024-09-14T06:33:01.475 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:30:17.940135+0000", 2024-09-14T06:33:01.475 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 163.49069002100001, 2024-09-14T06:33:01.475 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.075451105000000004, 2024-09-14T06:33:01.475 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:33:01.475 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:33:01.475 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:33:01.476 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:33:01.476 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:33:01.476 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:33:01.476 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 115202 2024-09-14T06:33:01.476 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:01.476 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:33:01.476 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:01.476 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:33:01.476 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940135+0000", 2024-09-14T06:33:01.476 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:33:01.476 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:01.476 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:01.476 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:33:01.477 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940135+0000", 2024-09-14T06:33:01.477 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:33:01.477 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:01.477 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:01.477 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:33:01.477 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940135+0000", 2024-09-14T06:33:01.477 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.0999999999999999e-07 2024-09-14T06:33:01.477 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:01.477 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:01.477 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:33:01.477 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940137+0000", 2024-09-14T06:33:01.477 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.0020000000000001e-06 2024-09-14T06:33:01.478 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:01.478 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:01.478 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:33:01.478 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940143+0000", 2024-09-14T06:33:01.478 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.9850000000000003e-06 2024-09-14T06:33:01.478 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:01.478 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:01.478 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:33:01.478 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940148+0000", 2024-09-14T06:33:01.479 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.9880000000000004e-06 2024-09-14T06:33:01.479 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:01.479 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:01.479 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:33:01.479 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.015400+0000", 2024-09-14T06:33:01.479 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.075252186999999998 2024-09-14T06:33:01.479 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:01.479 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:01.479 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:33:01.479 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.015481+0000", 2024-09-14T06:33:01.479 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.0698000000000006e-05 2024-09-14T06:33:01.479 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:01.479 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:01.480 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:33:01.480 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.015586+0000", 2024-09-14T06:33:01.480 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00010483499999999999 2024-09-14T06:33:01.480 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:33:01.480 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:33:01.480 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:33:01.480 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:01.480 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:01.480 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:115203 3.1 3:856be886:::rbd_data.112d4d24c558.0000000000000004:29 [read 3981312~16384 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e96)", 2024-09-14T06:33:01.480 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:30:17.940156+0000", 2024-09-14T06:33:01.480 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 163.49066879599999, 2024-09-14T06:33:01.480 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.075620860999999998, 2024-09-14T06:33:01.481 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:33:01.481 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:33:01.481 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:33:01.481 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:33:01.481 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:33:01.481 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:33:01.481 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 115203 2024-09-14T06:33:01.481 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:01.481 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:33:01.481 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:01.481 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:33:01.481 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940156+0000", 2024-09-14T06:33:01.481 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:33:01.482 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:01.482 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:01.482 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:33:01.482 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940156+0000", 2024-09-14T06:33:01.482 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:33:01.482 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:01.482 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:01.482 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:33:01.482 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940156+0000", 2024-09-14T06:33:01.482 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.1500000000000005e-07 2024-09-14T06:33:01.482 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:01.482 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:01.482 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:33:01.483 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940158+0000", 2024-09-14T06:33:01.483 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.967e-06 2024-09-14T06:33:01.483 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:01.483 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:01.483 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:33:01.483 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940165+0000", 2024-09-14T06:33:01.483 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.6939999999999998e-06 2024-09-14T06:33:01.483 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:01.483 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:01.483 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:33:01.483 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940170+0000", 2024-09-14T06:33:01.483 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.7539999999999997e-06 2024-09-14T06:33:01.483 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:01.483 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:01.484 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:33:01.484 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.015633+0000", 2024-09-14T06:33:01.484 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.075463334000000007 2024-09-14T06:33:01.484 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:01.484 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:01.484 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:33:01.484 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.015693+0000", 2024-09-14T06:33:01.484 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.9367000000000003e-05 2024-09-14T06:33:01.484 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:01.484 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:01.484 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:33:01.484 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.015777+0000", 2024-09-14T06:33:01.484 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.4229999999999998e-05 2024-09-14T06:33:01.484 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:33:01.485 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:33:01.485 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:33:01.485 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:01.485 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:01.485 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:115204 3.1 3:856be886:::rbd_data.112d4d24c558.0000000000000004:29 [read 4227072~16384 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e96)", 2024-09-14T06:33:01.485 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:30:17.940176+0000", 2024-09-14T06:33:01.485 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 163.490648263, 2024-09-14T06:33:01.485 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.075785505000000003, 2024-09-14T06:33:01.485 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:33:01.485 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:33:01.485 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:33:01.485 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:33:01.485 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:33:01.486 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:33:01.486 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 115204 2024-09-14T06:33:01.486 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:01.486 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:33:01.486 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:01.486 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:33:01.486 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940176+0000", 2024-09-14T06:33:01.486 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:33:01.486 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:01.486 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:01.486 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:33:01.486 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940176+0000", 2024-09-14T06:33:01.486 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:33:01.486 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:01.487 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:01.487 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:33:01.487 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940177+0000", 2024-09-14T06:33:01.487 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.7899999999999999e-07 2024-09-14T06:33:01.487 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:01.487 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:01.487 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:33:01.487 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940179+0000", 2024-09-14T06:33:01.487 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.103e-06 2024-09-14T06:33:01.487 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:01.487 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:01.487 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:33:01.487 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940185+0000", 2024-09-14T06:33:01.487 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.7239999999999997e-06 2024-09-14T06:33:01.488 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:01.488 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:01.488 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:33:01.488 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940190+0000", 2024-09-14T06:33:01.488 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.7269999999999998e-06 2024-09-14T06:33:01.488 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:01.488 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:01.488 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:33:01.488 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.015814+0000", 2024-09-14T06:33:01.488 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.075624114000000006 2024-09-14T06:33:01.488 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:01.488 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:01.488 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:33:01.489 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.015862+0000", 2024-09-14T06:33:01.489 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.8770999999999998e-05 2024-09-14T06:33:01.489 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:01.489 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:01.489 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:33:01.489 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.015962+0000", 2024-09-14T06:33:01.489 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.9586999999999994e-05 2024-09-14T06:33:01.489 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:33:01.489 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:33:01.489 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:33:01.489 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:01.489 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:01.489 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:115205 3.1 3:856be886:::rbd_data.112d4d24c558.0000000000000004:29 [read 4472832~16384 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e96)", 2024-09-14T06:33:01.490 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:30:17.940197+0000", 2024-09-14T06:33:01.490 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 163.49062755400001, 2024-09-14T06:33:01.490 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.076068920999999998, 2024-09-14T06:33:01.490 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:33:01.490 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:33:01.490 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:33:01.490 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:33:01.490 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:33:01.490 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:33:01.490 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 115205 2024-09-14T06:33:01.490 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:01.490 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:33:01.490 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:01.490 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:33:01.491 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940197+0000", 2024-09-14T06:33:01.491 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:33:01.491 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:01.491 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:01.491 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:33:01.491 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940197+0000", 2024-09-14T06:33:01.491 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:33:01.491 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:01.491 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:01.491 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:33:01.491 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940198+0000", 2024-09-14T06:33:01.491 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.5999999999999999e-07 2024-09-14T06:33:01.491 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:01.492 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:01.492 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:33:01.492 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940200+0000", 2024-09-14T06:33:01.492 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.0899999999999999e-06 2024-09-14T06:33:01.492 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:01.492 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:01.492 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:33:01.492 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940206+0000", 2024-09-14T06:33:01.492 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.8710000000000002e-06 2024-09-14T06:33:01.492 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:01.492 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:01.492 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:33:01.492 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940211+0000", 2024-09-14T06:33:01.493 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.9230000000000001e-06 2024-09-14T06:33:01.493 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:01.493 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:01.493 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:33:01.493 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.016059+0000", 2024-09-14T06:33:01.493 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.075848051999999999 2024-09-14T06:33:01.493 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:01.493 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:01.493 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:33:01.493 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.016151+0000", 2024-09-14T06:33:01.493 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.1982000000000001e-05 2024-09-14T06:33:01.493 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:01.493 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:01.493 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:33:01.494 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.016266+0000", 2024-09-14T06:33:01.494 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000115543 2024-09-14T06:33:01.494 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:33:01.494 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:33:01.494 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:33:01.494 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:01.494 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:01.494 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:115206 3.1 3:856be886:::rbd_data.112d4d24c558.0000000000000004:29 [read 4718592~16384 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e96)", 2024-09-14T06:33:01.494 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:30:17.940223+0000", 2024-09-14T06:33:01.494 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 163.49060146400001, 2024-09-14T06:33:01.494 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.076265115999999994, 2024-09-14T06:33:01.494 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:33:01.495 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:33:01.495 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:33:01.495 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:33:01.495 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:33:01.495 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:33:01.495 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 115206 2024-09-14T06:33:01.495 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:01.495 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:33:01.495 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:01.495 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:33:01.495 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940223+0000", 2024-09-14T06:33:01.496 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:33:01.496 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:01.496 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:01.496 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:33:01.496 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940223+0000", 2024-09-14T06:33:01.496 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:33:01.496 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:01.496 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:01.496 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:33:01.496 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940224+0000", 2024-09-14T06:33:01.496 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.0399999999999996e-07 2024-09-14T06:33:01.496 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:01.496 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:01.497 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:33:01.497 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940226+0000", 2024-09-14T06:33:01.497 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.1780000000000002e-06 2024-09-14T06:33:01.497 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:01.497 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:01.497 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:33:01.497 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940232+0000", 2024-09-14T06:33:01.497 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.5010000000000001e-06 2024-09-14T06:33:01.497 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:01.497 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:01.497 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:33:01.497 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940237+0000", 2024-09-14T06:33:01.497 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.0250000000000002e-06 2024-09-14T06:33:01.498 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:01.498 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:01.498 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:33:01.498 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.016305+0000", 2024-09-14T06:33:01.498 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.076067262999999996 2024-09-14T06:33:01.498 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:01.498 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:01.498 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:33:01.498 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.016383+0000", 2024-09-14T06:33:01.498 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.8578000000000006e-05 2024-09-14T06:33:01.498 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:01.498 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:01.498 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:33:01.499 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.016488+0000", 2024-09-14T06:33:01.499 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000105067 2024-09-14T06:33:01.499 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:33:01.499 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:33:01.499 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:33:01.499 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:01.499 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:01.499 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:115207 3.1 3:856be886:::rbd_data.112d4d24c558.0000000000000004:29 [read 4964352~16384 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e96)", 2024-09-14T06:33:01.499 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:30:17.940244+0000", 2024-09-14T06:33:01.499 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 163.49058034800001, 2024-09-14T06:33:01.499 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.076371154999999996, 2024-09-14T06:33:01.499 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:33:01.499 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:33:01.500 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:33:01.500 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:33:01.500 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:33:01.500 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:33:01.500 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 115207 2024-09-14T06:33:01.500 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:01.500 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:33:01.500 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:01.500 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:33:01.500 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940244+0000", 2024-09-14T06:33:01.500 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:33:01.500 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:01.500 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:01.500 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:33:01.500 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940244+0000", 2024-09-14T06:33:01.501 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:33:01.501 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:01.501 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:01.501 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:33:01.501 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940245+0000", 2024-09-14T06:33:01.501 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.1200000000000003e-07 2024-09-14T06:33:01.501 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:01.501 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:01.501 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:33:01.501 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940247+0000", 2024-09-14T06:33:01.501 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.0030000000000001e-06 2024-09-14T06:33:01.501 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:01.501 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:01.501 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:33:01.502 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940252+0000", 2024-09-14T06:33:01.502 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.5520000000000004e-06 2024-09-14T06:33:01.502 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:01.502 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:01.502 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:33:01.502 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940257+0000", 2024-09-14T06:33:01.502 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.6759999999999998e-06 2024-09-14T06:33:01.502 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:01.502 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:01.502 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:33:01.502 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.016521+0000", 2024-09-14T06:33:01.502 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.076263499999999998 2024-09-14T06:33:01.502 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:01.502 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:01.502 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:33:01.503 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.016562+0000", 2024-09-14T06:33:01.503 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.0967000000000003e-05 2024-09-14T06:33:01.503 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:01.503 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:01.503 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:33:01.503 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.016616+0000", 2024-09-14T06:33:01.503 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.3944999999999999e-05 2024-09-14T06:33:01.503 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:33:01.503 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:33:01.503 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:33:01.503 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:01.503 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:01.503 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:115208 3.1 3:856be886:::rbd_data.112d4d24c558.0000000000000004:29 [read 5210112~16384 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e96)", 2024-09-14T06:33:01.503 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:30:17.940264+0000", 2024-09-14T06:33:01.504 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 163.490560491, 2024-09-14T06:33:01.504 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.076558288000000002, 2024-09-14T06:33:01.504 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:33:01.504 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:33:01.504 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:33:01.504 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:33:01.504 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:33:01.504 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:33:01.504 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 115208 2024-09-14T06:33:01.504 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:01.504 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:33:01.504 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:01.504 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:33:01.504 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940264+0000", 2024-09-14T06:33:01.505 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:33:01.505 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:01.505 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:01.505 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:33:01.505 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940264+0000", 2024-09-14T06:33:01.505 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:33:01.505 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:01.505 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:01.505 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:33:01.505 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940265+0000", 2024-09-14T06:33:01.505 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.0500000000000004e-07 2024-09-14T06:33:01.505 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:01.505 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:01.505 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:33:01.506 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940267+0000", 2024-09-14T06:33:01.506 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.9810000000000002e-06 2024-09-14T06:33:01.506 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:01.506 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:01.506 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:33:01.506 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940272+0000", 2024-09-14T06:33:01.506 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.6999999999999996e-06 2024-09-14T06:33:01.506 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:01.506 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:01.506 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:33:01.506 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940277+0000", 2024-09-14T06:33:01.506 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.6480000000000002e-06 2024-09-14T06:33:01.506 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:01.506 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:01.506 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:33:01.507 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.016650+0000", 2024-09-14T06:33:01.507 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.076373089000000005 2024-09-14T06:33:01.507 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:01.507 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:01.507 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:33:01.507 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.016724+0000", 2024-09-14T06:33:01.507 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.4105999999999998e-05 2024-09-14T06:33:01.507 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:01.507 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:01.507 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:33:01.507 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.016823+0000", 2024-09-14T06:33:01.507 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.8258999999999996e-05 2024-09-14T06:33:01.507 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:33:01.507 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:33:01.507 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:33:01.508 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:01.508 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:01.508 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:115209 3.1 3:856be886:::rbd_data.112d4d24c558.0000000000000004:29 [read 5455872~16384 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e96)", 2024-09-14T06:33:01.508 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:30:17.940283+0000", 2024-09-14T06:33:01.508 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 163.49054146500001, 2024-09-14T06:33:01.508 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.076724029999999999, 2024-09-14T06:33:01.508 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:33:01.508 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:33:01.508 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:33:01.508 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:33:01.508 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:33:01.508 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:33:01.508 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 115209 2024-09-14T06:33:01.509 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:01.509 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:33:01.509 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:01.509 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:33:01.509 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940283+0000", 2024-09-14T06:33:01.509 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:33:01.509 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:01.509 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:01.509 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:33:01.509 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940283+0000", 2024-09-14T06:33:01.509 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:33:01.509 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:01.509 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:01.509 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:33:01.510 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940284+0000", 2024-09-14T06:33:01.510 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.6700000000000003e-07 2024-09-14T06:33:01.510 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:01.510 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:01.510 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:33:01.510 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940286+0000", 2024-09-14T06:33:01.510 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.9520000000000001e-06 2024-09-14T06:33:01.510 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:01.510 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:01.510 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:33:01.510 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940292+0000", 2024-09-14T06:33:01.510 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.7339999999999996e-06 2024-09-14T06:33:01.511 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:01.511 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:01.511 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:33:01.511 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940296+0000", 2024-09-14T06:33:01.511 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.7700000000000001e-06 2024-09-14T06:33:01.511 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:01.511 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:01.511 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:33:01.511 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.016859+0000", 2024-09-14T06:33:01.511 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.07656317 2024-09-14T06:33:01.511 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:01.511 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:01.511 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:33:01.511 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.016929+0000", 2024-09-14T06:33:01.512 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.9679999999999997e-05 2024-09-14T06:33:01.512 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:01.512 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:01.512 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:33:01.512 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.017007+0000", 2024-09-14T06:33:01.512 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.8157000000000004e-05 2024-09-14T06:33:01.512 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:33:01.512 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:33:01.512 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:33:01.512 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:01.512 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:01.512 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:115210 3.1 3:856be886:::rbd_data.112d4d24c558.0000000000000004:29 [read 5701632~16384 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e96)", 2024-09-14T06:33:01.512 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:30:17.940303+0000", 2024-09-14T06:33:01.512 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 163.49052194800001, 2024-09-14T06:33:01.513 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.076821358000000006, 2024-09-14T06:33:01.513 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:33:01.513 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:33:01.513 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:33:01.513 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:33:01.513 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:33:01.513 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:33:01.513 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 115210 2024-09-14T06:33:01.513 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:01.513 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:33:01.513 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:01.513 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:33:01.513 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940303+0000", 2024-09-14T06:33:01.514 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:33:01.514 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:01.514 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:01.514 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:33:01.514 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940303+0000", 2024-09-14T06:33:01.514 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:33:01.514 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:01.514 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:01.514 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:33:01.514 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940303+0000", 2024-09-14T06:33:01.514 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.3000000000000001e-07 2024-09-14T06:33:01.514 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:01.514 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:01.514 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:33:01.515 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940306+0000", 2024-09-14T06:33:01.515 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.255e-06 2024-09-14T06:33:01.515 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:01.515 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:01.515 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:33:01.515 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940312+0000", 2024-09-14T06:33:01.515 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.9710000000000001e-06 2024-09-14T06:33:01.515 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:01.515 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:01.515 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:33:01.515 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940316+0000", 2024-09-14T06:33:01.515 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.5340000000000001e-06 2024-09-14T06:33:01.515 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:01.515 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:01.516 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:33:01.516 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.017033+0000", 2024-09-14T06:33:01.516 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.076717169000000002 2024-09-14T06:33:01.516 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:01.516 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:01.516 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:33:01.516 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.017073+0000", 2024-09-14T06:33:01.516 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.9582000000000002e-05 2024-09-14T06:33:01.516 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:01.516 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:01.516 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:33:01.516 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.017124+0000", 2024-09-14T06:33:01.516 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.1316999999999997e-05 2024-09-14T06:33:01.516 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:33:01.517 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:33:01.517 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:33:01.517 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:01.517 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:01.517 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:115219 3.1 3:856be886:::rbd_data.112d4d24c558.0000000000000004:29 [read 65536~16384 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e96)", 2024-09-14T06:33:01.517 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:30:17.940323+0000", 2024-09-14T06:33:01.517 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 163.49050208700001, 2024-09-14T06:33:01.517 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.076915205, 2024-09-14T06:33:01.517 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:33:01.517 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:33:01.517 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:33:01.517 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:33:01.517 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:33:01.518 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:33:01.518 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 115219 2024-09-14T06:33:01.518 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:01.518 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:33:01.518 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:01.518 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:33:01.518 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940323+0000", 2024-09-14T06:33:01.519 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:33:01.519 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:01.520 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:01.520 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:33:01.520 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940323+0000", 2024-09-14T06:33:01.520 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:33:01.520 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:01.520 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:01.520 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:33:01.520 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940323+0000", 2024-09-14T06:33:01.520 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.0800000000000005e-07 2024-09-14T06:33:01.520 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:01.520 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:01.520 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:33:01.521 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940328+0000", 2024-09-14T06:33:01.521 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.887e-06 2024-09-14T06:33:01.521 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:01.521 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:01.521 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:33:01.521 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940334+0000", 2024-09-14T06:33:01.521 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.3269999999999997e-06 2024-09-14T06:33:01.521 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:01.521 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:01.521 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:33:01.521 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940339+0000", 2024-09-14T06:33:01.521 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.7960000000000003e-06 2024-09-14T06:33:01.521 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:01.522 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:01.522 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:33:01.522 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.017148+0000", 2024-09-14T06:33:01.522 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.076809125000000006 2024-09-14T06:33:01.522 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:01.522 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:01.522 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:33:01.522 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.017186+0000", 2024-09-14T06:33:01.522 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.7283000000000002e-05 2024-09-14T06:33:01.522 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:01.522 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:01.522 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:33:01.522 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.017238+0000", 2024-09-14T06:33:01.522 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.2278999999999999e-05 2024-09-14T06:33:01.523 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:33:01.523 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:33:01.523 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:33:01.523 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:01.523 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:01.523 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:115220 3.1 3:856be886:::rbd_data.112d4d24c558.0000000000000004:29 [read 311296~16384 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e96)", 2024-09-14T06:33:01.523 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:30:17.940346+0000", 2024-09-14T06:33:01.523 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 163.49047895800001, 2024-09-14T06:33:01.523 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.077033293000000003, 2024-09-14T06:33:01.523 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:33:01.523 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:33:01.523 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:33:01.523 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:33:01.524 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:33:01.524 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:33:01.524 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 115220 2024-09-14T06:33:01.524 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:01.524 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:33:01.524 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:01.524 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:33:01.524 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940346+0000", 2024-09-14T06:33:01.524 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:33:01.524 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:01.524 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:01.524 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:33:01.524 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940346+0000", 2024-09-14T06:33:01.524 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:33:01.525 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:01.525 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:01.525 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:33:01.525 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940346+0000", 2024-09-14T06:33:01.525 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.4300000000000003e-07 2024-09-14T06:33:01.525 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:01.525 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:01.525 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:33:01.525 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:17.940348+0000", 2024-09-14T06:33:01.525 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.0030000000000001e-06 2024-09-14T06:33:01.525 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:01.525 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:01.525 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:33:01.526 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.004655+0000", 2024-09-14T06:33:01.526 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.064306323999999998 2024-09-14T06:33:01.526 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:01.526 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:01.526 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:33:01.526 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.004671+0000", 2024-09-14T06:33:01.526 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.6589999999999999e-05 2024-09-14T06:33:01.526 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:01.526 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:01.526 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:33:01.526 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.017262+0000", 2024-09-14T06:33:01.526 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.012590570000000001 2024-09-14T06:33:01.526 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:01.526 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:01.527 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:33:01.527 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.017321+0000", 2024-09-14T06:33:01.527 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.9210999999999999e-05 2024-09-14T06:33:01.527 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:01.527 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:01.527 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:33:01.527 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:18.017379+0000", 2024-09-14T06:33:01.527 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.8051999999999999e-05 2024-09-14T06:33:01.527 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:33:01.527 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:33:01.527 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:33:01.527 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:33:01.527 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:33:01.527 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-14T06:33:01.529 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-14T06:33:01.529 INFO:teuthology.orchestra.run.smithi094.stdout:osd.4: {} 2024-09-14T06:33:01.530 INFO:teuthology.orchestra.run.smithi094.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-14T06:33:01.530 INFO:teuthology.orchestra.run.smithi094.stdout:osd.5: {} 2024-09-14T06:33:01.530 INFO:teuthology.orchestra.run.smithi094.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-14T06:33:01.531 INFO:teuthology.orchestra.run.smithi094.stdout:osd.6: {} 2024-09-14T06:33:01.531 INFO:teuthology.orchestra.run.smithi094.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-14T06:33:01.531 INFO:teuthology.orchestra.run.smithi094.stdout:osd.7: {} 2024-09-14T06:33:01.532 INFO:teuthology.orchestra.run.smithi094.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-14T06:33:01.532 INFO:tasks.ceph.osd.2.smithi094.stderr:2024-09-14T06:33:01.528+0000 7faf3269b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33649) UID: 0 2024-09-14T06:33:01.630 INFO:tasks.ceph.osd.2.smithi094.stderr:2024-09-14T06:33:01.629+0000 7faf3269b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33649) UID: 0 2024-09-14T06:33:01.678 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-14T06:33:01.678 INFO:teuthology.orchestra.run.smithi094.stdout: "ops": [], 2024-09-14T06:33:01.678 INFO:teuthology.orchestra.run.smithi094.stdout: "num_ops": 0 2024-09-14T06:33:01.678 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-14T06:33:01.687 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-14T06:33:01.731 INFO:tasks.ceph.osd.5.smithi138.stderr:2024-09-14T06:33:01.729+0000 7fe13746f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33618) UID: 0 2024-09-14T06:33:01.829 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-14T06:33:01.829 INFO:teuthology.orchestra.run.smithi094.stdout: "ops": [], 2024-09-14T06:33:01.829 INFO:teuthology.orchestra.run.smithi094.stdout: "complaint_time": 30, 2024-09-14T06:33:01.829 INFO:teuthology.orchestra.run.smithi094.stdout: "num_blocked_ops": 0 2024-09-14T06:33:01.829 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-14T06:33:01.831 INFO:tasks.ceph.osd.2.smithi094.stderr:2024-09-14T06:33:01.830+0000 7faf3269b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33649) UID: 0 2024-09-14T06:33:01.837 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-14T06:33:01.932 INFO:tasks.ceph.osd.2.smithi094.stderr:2024-09-14T06:33:01.931+0000 7faf3269b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33649) UID: 0 2024-09-14T06:33:01.983 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-14T06:33:01.983 INFO:teuthology.orchestra.run.smithi094.stdout: "size": 20, 2024-09-14T06:33:01.983 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 600, 2024-09-14T06:33:01.983 INFO:teuthology.orchestra.run.smithi094.stdout: "ops": [ 2024-09-14T06:33:01.983 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:01.983 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:14919 3.0 3:05c39535:::rbd_data.112d92027590.0000000000000009:f [read 805888~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:33:01.983 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:26.083843+0000", 2024-09-14T06:33:01.983 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 215.89717737800001, 2024-09-14T06:33:01.983 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.041992356000000002, 2024-09-14T06:33:01.983 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:33:01.983 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:33:01.983 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:33:01.984 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:33:01.984 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:33:01.984 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:33:01.984 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 14919 2024-09-14T06:33:01.984 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:01.984 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:33:01.984 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:01.984 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:33:01.984 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.083843+0000", 2024-09-14T06:33:01.984 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:33:01.984 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:01.984 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:01.984 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:33:01.985 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.083843+0000", 2024-09-14T06:33:01.985 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:33:01.985 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:01.985 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:01.985 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:33:01.985 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.083844+0000", 2024-09-14T06:33:01.985 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.3300000000000001e-07 2024-09-14T06:33:01.985 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:01.985 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:01.985 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:33:01.985 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.083845+0000", 2024-09-14T06:33:01.985 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.4729999999999999e-06 2024-09-14T06:33:01.985 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:01.986 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:01.986 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:33:01.986 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.083850+0000", 2024-09-14T06:33:01.986 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.0130000000000001e-06 2024-09-14T06:33:01.986 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:01.986 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:01.986 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:33:01.986 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.083854+0000", 2024-09-14T06:33:01.986 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.4759999999999998e-06 2024-09-14T06:33:01.986 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:01.986 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:01.986 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:33:01.986 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.125636+0000", 2024-09-14T06:33:01.986 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.041782683000000001 2024-09-14T06:33:01.987 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:01.987 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:01.987 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:33:01.987 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.125680+0000", 2024-09-14T06:33:01.987 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.3614e-05 2024-09-14T06:33:01.987 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:01.987 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:01.987 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:33:01.987 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.125836+0000", 2024-09-14T06:33:01.987 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000155864 2024-09-14T06:33:01.987 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:33:01.987 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:33:01.987 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:33:01.987 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:01.988 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:01.988 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:14920 3.0 3:05c39535:::rbd_data.112d92027590.0000000000000009:f [read 816128~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:33:01.988 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:26.083868+0000", 2024-09-14T06:33:01.988 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 215.89715271700001, 2024-09-14T06:33:01.988 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.042166771999999998, 2024-09-14T06:33:01.988 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:33:01.988 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:33:01.988 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:33:01.988 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:33:01.988 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:33:01.988 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:33:01.988 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 14920 2024-09-14T06:33:01.988 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:01.988 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:33:01.989 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:01.989 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:33:01.989 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.083868+0000", 2024-09-14T06:33:01.989 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:33:01.989 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:01.989 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:01.989 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:33:01.989 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.083868+0000", 2024-09-14T06:33:01.989 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:33:01.989 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:01.989 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:01.989 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:33:01.989 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.083869+0000", 2024-09-14T06:33:01.990 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.99e-07 2024-09-14T06:33:01.990 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:01.990 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:01.990 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:33:01.990 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.083870+0000", 2024-09-14T06:33:01.990 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.782e-06 2024-09-14T06:33:01.990 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:01.990 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:01.990 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:33:01.990 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.083876+0000", 2024-09-14T06:33:01.990 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.3809999999999998e-06 2024-09-14T06:33:01.990 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:01.990 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:01.990 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:33:01.991 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.083882+0000", 2024-09-14T06:33:01.991 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.9479999999999996e-06 2024-09-14T06:33:01.991 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:01.991 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:01.991 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:33:01.991 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.125878+0000", 2024-09-14T06:33:01.991 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.041996264999999998 2024-09-14T06:33:01.991 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:01.991 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:01.991 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:33:01.991 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.125929+0000", 2024-09-14T06:33:01.991 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.0498999999999999e-05 2024-09-14T06:33:01.991 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:01.991 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:01.992 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:33:01.992 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.126035+0000", 2024-09-14T06:33:01.992 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000106298 2024-09-14T06:33:01.992 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:33:01.992 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:33:01.992 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:33:01.992 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:01.992 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:01.992 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:14921 3.0 3:05c39535:::rbd_data.112d92027590.0000000000000009:f [read 826368~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:33:01.992 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:26.083889+0000", 2024-09-14T06:33:01.992 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 215.897131707, 2024-09-14T06:33:01.992 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.042435309999999997, 2024-09-14T06:33:01.992 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:33:01.993 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:33:01.993 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:33:01.993 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:33:01.993 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:33:01.993 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:33:01.993 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 14921 2024-09-14T06:33:01.993 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:01.993 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:33:01.993 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:01.993 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:33:01.993 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.083889+0000", 2024-09-14T06:33:01.993 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:33:01.993 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:01.993 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:01.994 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:33:01.994 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.083889+0000", 2024-09-14T06:33:01.994 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:33:01.994 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:01.994 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:01.994 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:33:01.994 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.086033+0000", 2024-09-14T06:33:01.994 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0021444179999999999 2024-09-14T06:33:01.994 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:01.994 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:01.994 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:33:01.994 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.086038+0000", 2024-09-14T06:33:01.994 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.0969999999999999e-06 2024-09-14T06:33:01.994 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:01.994 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:01.995 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:33:01.995 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.086046+0000", 2024-09-14T06:33:01.995 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.9440000000000005e-06 2024-09-14T06:33:01.995 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:01.995 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:01.995 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:33:01.995 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.086050+0000", 2024-09-14T06:33:01.995 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.69e-06 2024-09-14T06:33:01.995 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:01.995 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:01.995 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:33:01.995 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.126080+0000", 2024-09-14T06:33:01.995 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.040030237000000003 2024-09-14T06:33:01.995 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:01.996 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:01.996 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:33:01.996 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.126163+0000", 2024-09-14T06:33:01.996 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.2071999999999996e-05 2024-09-14T06:33:01.996 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:01.996 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:01.996 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:33:01.996 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:26.126324+0000", 2024-09-14T06:33:01.996 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00016185200000000001 2024-09-14T06:33:01.996 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:33:01.996 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:33:01.996 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:33:01.996 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:01.996 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:01.997 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:55627 3.0 3:06490baf:::rbd_data.112dca2f6ae1.0000000000000030:17 [read 2002944~4096 [fadvise_nocache]] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:33:01.997 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:44.599966+0000", 2024-09-14T06:33:01.997 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 197.38105475899999, 2024-09-14T06:33:01.997 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.041868546999999999, 2024-09-14T06:33:01.997 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:33:01.997 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:33:01.997 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:33:01.997 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:33:01.997 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:33:01.997 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:33:01.997 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 55627 2024-09-14T06:33:01.997 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:01.997 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:33:01.997 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:01.998 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:33:01.998 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.599966+0000", 2024-09-14T06:33:01.998 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:33:01.998 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:01.998 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:01.998 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:33:01.998 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.599966+0000", 2024-09-14T06:33:01.998 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:33:01.998 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:01.998 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:01.998 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:33:01.998 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.599966+0000", 2024-09-14T06:33:01.999 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.0900000000000001e-07 2024-09-14T06:33:01.999 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:01.999 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:01.999 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:33:01.999 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.599967+0000", 2024-09-14T06:33:01.999 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.64e-07 2024-09-14T06:33:01.999 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:01.999 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:01.999 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:33:01.999 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.599970+0000", 2024-09-14T06:33:01.999 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.9670000000000001e-06 2024-09-14T06:33:02.000 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:02.000 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:02.000 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:33:02.000 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.599973+0000", 2024-09-14T06:33:02.000 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.632e-06 2024-09-14T06:33:02.000 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:02.000 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:02.000 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:33:02.000 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.641665+0000", 2024-09-14T06:33:02.000 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.041692547000000003 2024-09-14T06:33:02.000 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:02.000 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:02.000 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:33:02.000 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.641740+0000", 2024-09-14T06:33:02.001 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.4559000000000002e-05 2024-09-14T06:33:02.001 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:02.001 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:02.001 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:33:02.001 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.641835+0000", 2024-09-14T06:33:02.001 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.4668999999999998e-05 2024-09-14T06:33:02.001 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:33:02.001 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:33:02.001 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:33:02.001 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:02.001 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:02.001 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:55628 3.0 3:06490baf:::rbd_data.112dca2f6ae1.0000000000000030:17 [read 2027520~4096 [fadvise_nocache]] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:33:02.001 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:44.599976+0000", 2024-09-14T06:33:02.001 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 197.38104497200001, 2024-09-14T06:33:02.002 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.041991621, 2024-09-14T06:33:02.002 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:33:02.002 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:33:02.002 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:33:02.002 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:33:02.002 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:33:02.002 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:33:02.002 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 55628 2024-09-14T06:33:02.002 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:02.002 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:33:02.002 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:02.002 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:33:02.002 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.599976+0000", 2024-09-14T06:33:02.002 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:33:02.002 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:02.003 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:02.003 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:33:02.003 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.599976+0000", 2024-09-14T06:33:02.003 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:33:02.003 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:02.003 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:02.003 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:33:02.003 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.599976+0000", 2024-09-14T06:33:02.003 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.1299999999999999e-07 2024-09-14T06:33:02.003 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:02.003 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:02.003 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:33:02.003 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.599977+0000", 2024-09-14T06:33:02.003 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.7999999999999993e-07 2024-09-14T06:33:02.004 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:02.004 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:02.004 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:33:02.004 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.599980+0000", 2024-09-14T06:33:02.004 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.01e-06 2024-09-14T06:33:02.004 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:02.004 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:02.004 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:33:02.004 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.599983+0000", 2024-09-14T06:33:02.004 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.5409999999999999e-06 2024-09-14T06:33:02.004 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:02.004 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:02.004 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:33:02.004 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.641869+0000", 2024-09-14T06:33:02.004 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.041886235000000001 2024-09-14T06:33:02.005 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:02.005 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:02.005 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:33:02.005 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.641913+0000", 2024-09-14T06:33:02.005 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.4357000000000002e-05 2024-09-14T06:33:02.005 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:02.005 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:02.005 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:33:02.005 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.641967+0000", 2024-09-14T06:33:02.005 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.4284999999999999e-05 2024-09-14T06:33:02.005 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:33:02.005 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:33:02.005 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:33:02.005 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:02.006 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:02.006 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:55629 3.0 3:06490baf:::rbd_data.112dca2f6ae1.0000000000000030:17 [read 2052096~4096 [fadvise_nocache]] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:33:02.006 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:44.599985+0000", 2024-09-14T06:33:02.006 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 197.381035266, 2024-09-14T06:33:02.006 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.042183375000000002, 2024-09-14T06:33:02.006 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:33:02.006 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:33:02.006 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:33:02.006 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:33:02.006 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:33:02.006 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:33:02.006 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 55629 2024-09-14T06:33:02.006 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:02.006 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:33:02.006 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:02.007 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:33:02.007 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.599985+0000", 2024-09-14T06:33:02.007 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:33:02.007 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:02.007 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:02.007 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:33:02.007 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.599985+0000", 2024-09-14T06:33:02.007 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:33:02.007 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:02.007 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:02.007 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:33:02.007 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.599986+0000", 2024-09-14T06:33:02.007 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.05e-07 2024-09-14T06:33:02.007 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:02.008 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:02.008 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:33:02.008 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.599987+0000", 2024-09-14T06:33:02.008 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.8400000000000002e-07 2024-09-14T06:33:02.008 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:02.008 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:02.008 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:33:02.008 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.599990+0000", 2024-09-14T06:33:02.008 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.5640000000000001e-06 2024-09-14T06:33:02.008 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:02.008 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:02.008 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:33:02.008 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.599993+0000", 2024-09-14T06:33:02.008 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.615e-06 2024-09-14T06:33:02.008 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:02.009 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:02.009 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:33:02.009 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.642003+0000", 2024-09-14T06:33:02.009 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.042010345999999997 2024-09-14T06:33:02.009 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:02.009 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:02.009 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:33:02.009 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.642080+0000", 2024-09-14T06:33:02.009 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.6538999999999993e-05 2024-09-14T06:33:02.009 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:02.009 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:02.009 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:33:02.009 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.642169+0000", 2024-09-14T06:33:02.009 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.9122000000000005e-05 2024-09-14T06:33:02.010 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:33:02.010 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:33:02.010 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:33:02.010 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:02.010 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:02.010 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:55630 3.0 3:06490baf:::rbd_data.112dca2f6ae1.0000000000000030:17 [read 2076672~4096 [fadvise_nocache]] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:33:02.010 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:44.599996+0000", 2024-09-14T06:33:02.010 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 197.38102478799999, 2024-09-14T06:33:02.010 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.042299615999999998, 2024-09-14T06:33:02.010 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:33:02.010 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:33:02.010 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:33:02.010 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:33:02.010 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:33:02.010 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:33:02.011 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 55630 2024-09-14T06:33:02.011 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:02.011 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:33:02.011 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:02.011 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:33:02.011 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.599996+0000", 2024-09-14T06:33:02.011 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:33:02.011 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:02.011 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:02.011 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:33:02.011 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.599996+0000", 2024-09-14T06:33:02.011 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:33:02.011 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:02.011 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:02.012 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:33:02.012 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.599996+0000", 2024-09-14T06:33:02.012 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.1500000000000001e-07 2024-09-14T06:33:02.012 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:02.012 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:02.012 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:33:02.012 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.599997+0000", 2024-09-14T06:33:02.012 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.33e-07 2024-09-14T06:33:02.012 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:02.012 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:02.012 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:33:02.012 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600000+0000", 2024-09-14T06:33:02.012 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.0240000000000002e-06 2024-09-14T06:33:02.012 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:02.013 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:02.013 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:33:02.013 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600003+0000", 2024-09-14T06:33:02.013 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.3640000000000002e-06 2024-09-14T06:33:02.013 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:02.013 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:02.013 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:33:02.013 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.642200+0000", 2024-09-14T06:33:02.013 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.042197571000000003 2024-09-14T06:33:02.013 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:02.013 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:02.013 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:33:02.013 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.642243+0000", 2024-09-14T06:33:02.013 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.2765000000000001e-05 2024-09-14T06:33:02.014 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:02.014 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:02.014 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:33:02.014 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.642296+0000", 2024-09-14T06:33:02.014 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.2744000000000002e-05 2024-09-14T06:33:02.014 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:33:02.014 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:33:02.014 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:33:02.014 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:02.014 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:02.014 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:55631 3.0 3:06490baf:::rbd_data.112dca2f6ae1.0000000000000030:17 [read 1048576~4096 [fadvise_nocache]] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:33:02.014 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:44.600006+0000", 2024-09-14T06:33:02.014 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 197.38101520000001, 2024-09-14T06:33:02.014 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.042487905999999999, 2024-09-14T06:33:02.015 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:33:02.015 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:33:02.015 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:33:02.015 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:33:02.015 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:33:02.015 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:33:02.015 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 55631 2024-09-14T06:33:02.015 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:02.015 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:33:02.015 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:02.015 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:33:02.015 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600006+0000", 2024-09-14T06:33:02.015 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:33:02.015 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:02.016 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:02.016 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:33:02.016 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600006+0000", 2024-09-14T06:33:02.016 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:33:02.016 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:02.016 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:02.016 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:33:02.016 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600006+0000", 2024-09-14T06:33:02.016 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.0800000000000001e-07 2024-09-14T06:33:02.016 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:02.016 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:02.016 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:33:02.016 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600007+0000", 2024-09-14T06:33:02.016 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.7199999999999997e-07 2024-09-14T06:33:02.017 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:02.017 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:02.017 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:33:02.017 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600010+0000", 2024-09-14T06:33:02.017 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.0280000000000001e-06 2024-09-14T06:33:02.017 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:02.017 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:02.017 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:33:02.017 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600012+0000", 2024-09-14T06:33:02.017 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.475e-06 2024-09-14T06:33:02.017 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:02.017 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:02.017 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:33:02.017 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.642329+0000", 2024-09-14T06:33:02.017 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.042317157000000001 2024-09-14T06:33:02.018 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:02.018 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:02.018 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:33:02.018 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.642403+0000", 2024-09-14T06:33:02.018 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.3941000000000002e-05 2024-09-14T06:33:02.018 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:02.018 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:02.018 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:33:02.018 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.642493+0000", 2024-09-14T06:33:02.018 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.0124999999999995e-05 2024-09-14T06:33:02.018 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:33:02.018 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:33:02.018 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:33:02.018 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:02.019 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:02.019 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:55632 3.0 3:06490baf:::rbd_data.112dca2f6ae1.0000000000000030:17 [read 1073152~4096 [fadvise_nocache]] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:33:02.019 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:44.600015+0000", 2024-09-14T06:33:02.019 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 197.38100544700001, 2024-09-14T06:33:02.019 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.042604922000000003, 2024-09-14T06:33:02.019 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:33:02.019 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:33:02.019 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:33:02.019 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:33:02.019 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:33:02.019 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:33:02.019 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 55632 2024-09-14T06:33:02.019 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:02.019 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:33:02.019 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:02.020 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:33:02.020 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600015+0000", 2024-09-14T06:33:02.020 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:33:02.020 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:02.020 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:02.020 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:33:02.020 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600015+0000", 2024-09-14T06:33:02.020 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:33:02.020 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:02.020 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:02.020 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:33:02.020 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600016+0000", 2024-09-14T06:33:02.020 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.1e-07 2024-09-14T06:33:02.020 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:02.021 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:02.021 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:33:02.021 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600017+0000", 2024-09-14T06:33:02.021 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.9900000000000009e-07 2024-09-14T06:33:02.021 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:02.021 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:02.021 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:33:02.021 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600020+0000", 2024-09-14T06:33:02.021 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.4170000000000001e-06 2024-09-14T06:33:02.021 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:02.021 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:02.021 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:33:02.021 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600023+0000", 2024-09-14T06:33:02.021 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.588e-06 2024-09-14T06:33:02.021 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:02.022 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:02.022 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:33:02.022 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.642526+0000", 2024-09-14T06:33:02.022 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.042503236999999999 2024-09-14T06:33:02.022 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:02.022 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:02.022 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:33:02.022 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.642568+0000", 2024-09-14T06:33:02.022 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.1925000000000003e-05 2024-09-14T06:33:02.022 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:02.022 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:02.022 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:33:02.022 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.642620+0000", 2024-09-14T06:33:02.022 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.2546000000000001e-05 2024-09-14T06:33:02.023 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:33:02.023 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:33:02.023 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:33:02.023 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:02.023 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:02.023 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:55633 3.0 3:06490baf:::rbd_data.112dca2f6ae1.0000000000000030:17 [read 1097728~4096 [fadvise_nocache]] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:33:02.023 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:44.600026+0000", 2024-09-14T06:33:02.023 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 197.38099517800001, 2024-09-14T06:33:02.023 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.042797474000000002, 2024-09-14T06:33:02.023 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:33:02.023 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:33:02.023 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:33:02.023 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:33:02.023 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:33:02.024 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:33:02.024 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 55633 2024-09-14T06:33:02.024 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:02.024 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:33:02.024 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:02.024 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:33:02.024 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600026+0000", 2024-09-14T06:33:02.024 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:33:02.024 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:02.024 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:02.024 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:33:02.024 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600026+0000", 2024-09-14T06:33:02.024 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:33:02.024 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:02.025 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:02.025 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:33:02.025 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600026+0000", 2024-09-14T06:33:02.025 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.3099999999999999e-07 2024-09-14T06:33:02.025 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:02.025 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:02.025 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:33:02.025 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600029+0000", 2024-09-14T06:33:02.025 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.304e-06 2024-09-14T06:33:02.025 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:02.025 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:02.025 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:33:02.025 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600034+0000", 2024-09-14T06:33:02.025 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.1429999999999997e-06 2024-09-14T06:33:02.025 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:02.026 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:02.026 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:33:02.026 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600038+0000", 2024-09-14T06:33:02.026 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.692e-06 2024-09-14T06:33:02.026 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:02.026 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:02.026 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:33:02.026 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.642654+0000", 2024-09-14T06:33:02.026 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.042615564000000002 2024-09-14T06:33:02.026 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:02.026 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:02.026 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:33:02.026 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.642728+0000", 2024-09-14T06:33:02.026 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.4018999999999997e-05 2024-09-14T06:33:02.027 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:02.027 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:02.027 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:33:02.027 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.642823+0000", 2024-09-14T06:33:02.027 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.5520999999999996e-05 2024-09-14T06:33:02.027 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:33:02.027 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:33:02.027 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:33:02.027 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:02.027 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:02.027 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:55634 3.0 3:06490baf:::rbd_data.112dca2f6ae1.0000000000000030:17 [read 1122304~4096 [fadvise_nocache]] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:33:02.027 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:44.600042+0000", 2024-09-14T06:33:02.027 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 197.380978701, 2024-09-14T06:33:02.027 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.042903729000000002, 2024-09-14T06:33:02.027 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:33:02.028 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:33:02.028 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:33:02.028 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:33:02.028 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:33:02.028 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:33:02.028 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 55634 2024-09-14T06:33:02.028 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:02.028 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:33:02.028 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:02.028 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:33:02.028 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600042+0000", 2024-09-14T06:33:02.028 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:33:02.028 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:02.028 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:02.029 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:33:02.029 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600042+0000", 2024-09-14T06:33:02.029 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:33:02.029 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:02.029 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:02.029 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:33:02.029 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600043+0000", 2024-09-14T06:33:02.029 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.7000000000000005e-07 2024-09-14T06:33:02.029 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:02.029 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:02.029 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:33:02.030 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600044+0000", 2024-09-14T06:33:02.030 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.742e-06 2024-09-14T06:33:02.030 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:02.030 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:02.030 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:33:02.030 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600050+0000", 2024-09-14T06:33:02.030 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.0800000000000002e-06 2024-09-14T06:33:02.030 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:02.030 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:02.030 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:33:02.030 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600055+0000", 2024-09-14T06:33:02.030 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.3889999999999998e-06 2024-09-14T06:33:02.030 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:02.031 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:02.031 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:33:02.031 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.642845+0000", 2024-09-14T06:33:02.031 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.042790040000000001 2024-09-14T06:33:02.031 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:02.031 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:02.031 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:33:02.031 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.642888+0000", 2024-09-14T06:33:02.031 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.2790000000000002e-05 2024-09-14T06:33:02.031 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:02.031 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:02.031 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:33:02.031 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.642946+0000", 2024-09-14T06:33:02.031 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.8118000000000002e-05 2024-09-14T06:33:02.032 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:33:02.032 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:33:02.032 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:33:02.032 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:02.032 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:02.032 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:55635 3.0 3:06490baf:::rbd_data.112dca2f6ae1.0000000000000030:17 [read 1146880~4096 [fadvise_nocache]] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:33:02.032 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:44.600061+0000", 2024-09-14T06:33:02.032 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 197.38096013000001, 2024-09-14T06:33:02.032 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.043092695, 2024-09-14T06:33:02.032 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:33:02.032 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:33:02.032 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:33:02.032 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:33:02.033 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:33:02.033 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:33:02.033 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 55635 2024-09-14T06:33:02.033 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:02.033 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:33:02.033 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:02.033 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:33:02.033 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600061+0000", 2024-09-14T06:33:02.033 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:33:02.033 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:02.033 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:02.033 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:33:02.033 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600061+0000", 2024-09-14T06:33:02.033 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:33:02.034 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:02.034 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:02.034 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:33:02.034 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600061+0000", 2024-09-14T06:33:02.034 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.3099999999999999e-07 2024-09-14T06:33:02.034 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:02.034 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:02.034 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:33:02.034 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600063+0000", 2024-09-14T06:33:02.034 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.6220000000000001e-06 2024-09-14T06:33:02.034 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:02.034 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:02.034 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:33:02.034 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600068+0000", 2024-09-14T06:33:02.034 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.4790000000000004e-06 2024-09-14T06:33:02.035 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:02.035 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:02.035 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:33:02.035 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600072+0000", 2024-09-14T06:33:02.035 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.3100000000000002e-06 2024-09-14T06:33:02.035 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:02.035 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:02.035 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:33:02.035 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.642987+0000", 2024-09-14T06:33:02.035 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.042914489 2024-09-14T06:33:02.035 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:02.035 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:02.035 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:33:02.035 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.643062+0000", 2024-09-14T06:33:02.035 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.5011000000000004e-05 2024-09-14T06:33:02.036 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:02.036 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:02.036 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:33:02.036 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.643153+0000", 2024-09-14T06:33:02.036 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.1452999999999993e-05 2024-09-14T06:33:02.036 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:33:02.036 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:33:02.036 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:33:02.036 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:02.036 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:02.036 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:55636 3.0 3:06490baf:::rbd_data.112dca2f6ae1.0000000000000030:17 [read 1171456~4096 [fadvise_nocache]] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:33:02.036 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:44.600078+0000", 2024-09-14T06:33:02.036 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 197.380942718, 2024-09-14T06:33:02.036 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.043207966, 2024-09-14T06:33:02.037 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:33:02.037 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:33:02.037 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:33:02.037 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:33:02.037 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:33:02.037 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:33:02.037 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 55636 2024-09-14T06:33:02.037 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:02.037 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:33:02.037 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:02.037 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:33:02.037 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600078+0000", 2024-09-14T06:33:02.037 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:33:02.037 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:02.038 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:02.038 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:33:02.038 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600078+0000", 2024-09-14T06:33:02.038 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:33:02.038 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:02.038 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:02.038 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:33:02.038 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600078+0000", 2024-09-14T06:33:02.038 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.34e-07 2024-09-14T06:33:02.038 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:02.038 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:02.038 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:33:02.038 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600080+0000", 2024-09-14T06:33:02.038 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.641e-06 2024-09-14T06:33:02.039 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:02.039 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:02.039 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:33:02.039 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600086+0000", 2024-09-14T06:33:02.039 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.3860000000000004e-06 2024-09-14T06:33:02.039 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:02.039 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:02.039 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:33:02.039 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600091+0000", 2024-09-14T06:33:02.039 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.5970000000000002e-06 2024-09-14T06:33:02.039 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:02.039 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:02.039 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:33:02.039 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.643184+0000", 2024-09-14T06:33:02.039 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.043092893 2024-09-14T06:33:02.040 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:02.040 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:02.040 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:33:02.040 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.643228+0000", 2024-09-14T06:33:02.040 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.4536999999999997e-05 2024-09-14T06:33:02.040 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:02.040 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:02.040 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:33:02.040 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.643286+0000", 2024-09-14T06:33:02.040 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.7578000000000003e-05 2024-09-14T06:33:02.040 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:33:02.040 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:33:02.040 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:33:02.041 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:02.041 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:02.041 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:55637 3.0 3:06490baf:::rbd_data.112dca2f6ae1.0000000000000030:17 [read 1196032~4096 [fadvise_nocache]] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:33:02.041 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:44.600097+0000", 2024-09-14T06:33:02.041 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 197.38092415400001, 2024-09-14T06:33:02.041 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.043399124999999997, 2024-09-14T06:33:02.041 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:33:02.041 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:33:02.041 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:33:02.041 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:33:02.041 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:33:02.041 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:33:02.041 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 55637 2024-09-14T06:33:02.041 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:02.041 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:33:02.042 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:02.042 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:33:02.042 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600097+0000", 2024-09-14T06:33:02.042 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:33:02.042 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:02.042 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:02.042 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:33:02.042 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.600097+0000", 2024-09-14T06:33:02.042 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:33:02.042 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:02.042 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:02.042 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:33:02.042 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.604372+0000", 2024-09-14T06:33:02.042 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0042751489999999998 2024-09-14T06:33:02.043 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:02.043 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:02.043 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:33:02.043 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.604377+0000", 2024-09-14T06:33:02.043 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.7899999999999999e-06 2024-09-14T06:33:02.043 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:02.043 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:02.043 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:33:02.043 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.604389+0000", 2024-09-14T06:33:02.043 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.2197e-05 2024-09-14T06:33:02.043 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:02.043 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:02.043 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:33:02.043 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.604397+0000", 2024-09-14T06:33:02.043 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.8730000000000007e-06 2024-09-14T06:33:02.044 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:02.044 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:02.044 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:33:02.044 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.643324+0000", 2024-09-14T06:33:02.044 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.038926897000000002 2024-09-14T06:33:02.044 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:02.044 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:02.044 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:33:02.044 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.643402+0000", 2024-09-14T06:33:02.044 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.8063000000000001e-05 2024-09-14T06:33:02.044 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:02.044 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:02.044 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:33:02.044 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.643496+0000", 2024-09-14T06:33:02.045 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.4155999999999998e-05 2024-09-14T06:33:02.045 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:33:02.045 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:33:02.045 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:33:02.045 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:02.045 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:02.045 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:55657 3.0 3:06490baf:::rbd_data.112dca2f6ae1.0000000000000030:17 [read 1687552~4096 [fadvise_nocache]] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:33:02.045 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:44.604841+0000", 2024-09-14T06:33:02.045 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 197.376179425, 2024-09-14T06:33:02.045 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.041890087999999999, 2024-09-14T06:33:02.045 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:33:02.045 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:33:02.045 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:33:02.045 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:33:02.046 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:33:02.046 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:33:02.046 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 55657 2024-09-14T06:33:02.046 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:02.046 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:33:02.046 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:02.046 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:33:02.046 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.604841+0000", 2024-09-14T06:33:02.046 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:33:02.046 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:02.046 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:02.046 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:33:02.046 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.604841+0000", 2024-09-14T06:33:02.046 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:33:02.047 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:02.047 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:02.047 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:33:02.047 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.604842+0000", 2024-09-14T06:33:02.047 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.0900000000000001e-07 2024-09-14T06:33:02.047 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:02.047 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:02.047 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:33:02.047 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.604846+0000", 2024-09-14T06:33:02.047 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.5759999999999999e-06 2024-09-14T06:33:02.047 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:02.047 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:02.047 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:33:02.048 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.604850+0000", 2024-09-14T06:33:02.048 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.405e-06 2024-09-14T06:33:02.048 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:02.048 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:02.048 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:33:02.048 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.604852+0000", 2024-09-14T06:33:02.048 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.8590000000000001e-06 2024-09-14T06:33:02.048 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:02.048 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:02.048 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:33:02.048 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.646633+0000", 2024-09-14T06:33:02.048 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.041780772000000001 2024-09-14T06:33:02.048 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:02.049 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:02.049 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:33:02.049 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.646677+0000", 2024-09-14T06:33:02.049 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.4115e-05 2024-09-14T06:33:02.049 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:02.049 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:02.049 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:33:02.049 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.646731+0000", 2024-09-14T06:33:02.049 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.4151999999999999e-05 2024-09-14T06:33:02.049 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:33:02.049 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:33:02.049 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:33:02.049 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:02.049 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:02.050 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:55658 3.0 3:06490baf:::rbd_data.112dca2f6ae1.0000000000000030:17 [read 1712128~4096 [fadvise_nocache]] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:33:02.050 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:44.604856+0000", 2024-09-14T06:33:02.050 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 197.37616482499999, 2024-09-14T06:33:02.050 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.041986227000000001, 2024-09-14T06:33:02.050 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:33:02.050 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:33:02.050 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:33:02.050 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:33:02.050 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:33:02.050 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:33:02.050 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 55658 2024-09-14T06:33:02.050 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:02.050 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:33:02.050 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:02.051 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:33:02.051 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.604856+0000", 2024-09-14T06:33:02.051 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:33:02.051 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:02.051 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:02.051 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:33:02.051 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.604856+0000", 2024-09-14T06:33:02.051 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:33:02.051 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:02.051 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:02.051 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:33:02.051 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.604856+0000", 2024-09-14T06:33:02.051 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.23e-07 2024-09-14T06:33:02.051 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:02.052 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:02.052 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:33:02.052 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.604857+0000", 2024-09-14T06:33:02.052 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.9999999999999995e-07 2024-09-14T06:33:02.052 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:02.052 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:02.052 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:33:02.052 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.604861+0000", 2024-09-14T06:33:02.052 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.9890000000000003e-06 2024-09-14T06:33:02.052 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:02.052 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:02.052 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:33:02.052 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.604864+0000", 2024-09-14T06:33:02.053 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.7429999999999998e-06 2024-09-14T06:33:02.053 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:02.053 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:02.053 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:33:02.053 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.646744+0000", 2024-09-14T06:33:02.053 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.041880325000000003 2024-09-14T06:33:02.053 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:02.053 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:02.053 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:33:02.053 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.646784+0000", 2024-09-14T06:33:02.053 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.9604000000000002e-05 2024-09-14T06:33:02.053 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:02.053 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:02.053 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:33:02.054 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.646842+0000", 2024-09-14T06:33:02.054 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.8343e-05 2024-09-14T06:33:02.054 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:33:02.054 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:33:02.054 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:33:02.054 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:02.054 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:02.054 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:55659 3.0 3:06490baf:::rbd_data.112dca2f6ae1.0000000000000030:17 [read 1736704~4096 [fadvise_nocache]] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:33:02.054 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:44.604867+0000", 2024-09-14T06:33:02.054 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 197.37615367199999, 2024-09-14T06:33:02.054 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.042097082000000001, 2024-09-14T06:33:02.054 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:33:02.054 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:33:02.054 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:33:02.054 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:33:02.055 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:33:02.055 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:33:02.055 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 55659 2024-09-14T06:33:02.055 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:02.055 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:33:02.055 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:02.055 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:33:02.055 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.604867+0000", 2024-09-14T06:33:02.055 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:33:02.055 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:02.055 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:02.055 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:33:02.055 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.604867+0000", 2024-09-14T06:33:02.055 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:33:02.056 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:02.056 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:02.056 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:33:02.056 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.604867+0000", 2024-09-14T06:33:02.056 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.2000000000000001e-07 2024-09-14T06:33:02.056 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:02.056 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:02.056 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:33:02.056 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.604868+0000", 2024-09-14T06:33:02.056 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.04e-06 2024-09-14T06:33:02.056 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:02.056 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:02.056 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:33:02.056 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.604871+0000", 2024-09-14T06:33:02.057 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.9689999999999999e-06 2024-09-14T06:33:02.057 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:02.057 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:02.057 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:33:02.057 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.604874+0000", 2024-09-14T06:33:02.057 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.5069999999999999e-06 2024-09-14T06:33:02.057 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:02.057 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:02.057 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:33:02.057 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.646874+0000", 2024-09-14T06:33:02.057 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.042000513000000003 2024-09-14T06:33:02.057 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:02.057 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:02.057 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:33:02.058 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.646914+0000", 2024-09-14T06:33:02.058 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.9443999999999997e-05 2024-09-14T06:33:02.058 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:02.058 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:02.058 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:33:02.058 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.646964+0000", 2024-09-14T06:33:02.058 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.0389000000000002e-05 2024-09-14T06:33:02.058 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:33:02.058 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:33:02.058 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:33:02.058 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:02.058 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:02.058 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:55660 3.0 3:06490baf:::rbd_data.112dca2f6ae1.0000000000000030:17 [read 1761280~4096 [fadvise_nocache]] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:33:02.058 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:44.604877+0000", 2024-09-14T06:33:02.059 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 197.376143923, 2024-09-14T06:33:02.059 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.042196349000000001, 2024-09-14T06:33:02.059 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:33:02.059 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:33:02.059 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:33:02.059 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:33:02.059 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:33:02.059 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:33:02.059 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 55660 2024-09-14T06:33:02.059 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:02.059 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:33:02.059 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:02.059 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:33:02.059 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.604877+0000", 2024-09-14T06:33:02.060 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:33:02.060 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:02.060 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:02.060 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:33:02.060 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.604877+0000", 2024-09-14T06:33:02.060 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:33:02.060 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:02.060 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:02.060 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:33:02.060 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.605810+0000", 2024-09-14T06:33:02.060 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00093336899999999997 2024-09-14T06:33:02.060 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:02.060 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:02.060 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:33:02.061 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.605815+0000", 2024-09-14T06:33:02.061 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.6140000000000002e-06 2024-09-14T06:33:02.061 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:02.061 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:02.061 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:33:02.061 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.605825+0000", 2024-09-14T06:33:02.061 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.8889999999999997e-06 2024-09-14T06:33:02.061 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:02.061 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:02.061 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:33:02.061 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.605831+0000", 2024-09-14T06:33:02.061 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.9800000000000003e-06 2024-09-14T06:33:02.061 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:02.061 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:02.062 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:33:02.062 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.646989+0000", 2024-09-14T06:33:02.062 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.041157878000000002 2024-09-14T06:33:02.062 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:02.062 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:02.062 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:33:02.062 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.647026+0000", 2024-09-14T06:33:02.062 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.7305000000000003e-05 2024-09-14T06:33:02.062 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:02.062 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:02.062 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:33:02.063 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:44.647073+0000", 2024-09-14T06:33:02.063 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.7314000000000001e-05 2024-09-14T06:33:02.063 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:33:02.063 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:33:02.063 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:33:02.063 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:02.063 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:02.063 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:236341 3.1c 3:3831950c:::rbd_directory:head [call rbd.dir_get_id in=13b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e233)", 2024-09-14T06:33:02.063 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:32:38.788743+0000", 2024-09-14T06:33:02.063 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 23.192277698000002, 2024-09-14T06:33:02.063 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.99031488700000003, 2024-09-14T06:33:02.063 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:33:02.064 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:33:02.064 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:33:02.064 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:33:02.064 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:33:02.064 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:33:02.064 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 236341 2024-09-14T06:33:02.064 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:02.064 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:33:02.064 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:02.064 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:33:02.064 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:38.788743+0000", 2024-09-14T06:33:02.064 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:33:02.064 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:02.064 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:02.065 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:33:02.065 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:38.788743+0000", 2024-09-14T06:33:02.065 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:33:02.065 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:02.065 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:02.065 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:33:02.065 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:38.788745+0000", 2024-09-14T06:33:02.065 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.623e-06 2024-09-14T06:33:02.065 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:02.065 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:02.065 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:33:02.065 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:38.788753+0000", 2024-09-14T06:33:02.065 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.8420000000000005e-06 2024-09-14T06:33:02.065 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:02.066 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:02.066 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:33:02.066 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:38.788877+0000", 2024-09-14T06:33:02.066 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00012487899999999999 2024-09-14T06:33:02.066 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:02.066 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:02.066 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:33:02.066 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:38.788886+0000", 2024-09-14T06:33:02.066 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.2379999999999997e-06 2024-09-14T06:33:02.066 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:02.066 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:02.066 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:33:02.066 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:38.788947+0000", 2024-09-14T06:33:02.066 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.1721000000000001e-05 2024-09-14T06:33:02.067 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:02.067 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:02.067 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "waiting for peered", 2024-09-14T06:33:02.067 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:38.788955+0000", 2024-09-14T06:33:02.067 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.2579999999999998e-06 2024-09-14T06:33:02.067 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:02.067 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:02.067 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:33:02.067 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:39.778650+0000", 2024-09-14T06:33:02.067 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.989694988 2024-09-14T06:33:02.067 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:02.067 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:02.067 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:33:02.067 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:39.778811+0000", 2024-09-14T06:33:02.067 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000161092 2024-09-14T06:33:02.068 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:02.068 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:02.068 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:33:02.068 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:39.779058+0000", 2024-09-14T06:33:02.068 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00024724600000000001 2024-09-14T06:33:02.068 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:33:02.068 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:33:02.068 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:33:02.068 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:02.068 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:02.068 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:238182 3.a 3:567df57c:::rbd_data.112d80ca42af.0000000000000007:head [delete] snapc 1f=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e243)", 2024-09-14T06:33:02.068 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:32:48.618828+0000", 2024-09-14T06:33:02.068 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 13.36219314, 2024-09-14T06:33:02.069 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.0662279320000001, 2024-09-14T06:33:02.069 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:33:02.069 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:33:02.069 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:33:02.069 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:33:02.069 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:33:02.069 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:33:02.069 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 238182 2024-09-14T06:33:02.069 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:02.069 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:33:02.069 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:02.069 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:33:02.069 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:48.618828+0000", 2024-09-14T06:33:02.070 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:33:02.070 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:02.070 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:02.070 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:33:02.070 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:48.618828+0000", 2024-09-14T06:33:02.070 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:33:02.070 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:02.070 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:02.070 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:33:02.070 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:48.618829+0000", 2024-09-14T06:33:02.070 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.362e-06 2024-09-14T06:33:02.070 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:02.070 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:02.070 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:33:02.070 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:48.618833+0000", 2024-09-14T06:33:02.071 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.4789999999999998e-06 2024-09-14T06:33:02.071 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:02.071 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:02.071 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:33:02.071 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:48.618841+0000", 2024-09-14T06:33:02.071 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.0260000000000007e-06 2024-09-14T06:33:02.071 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:02.071 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:02.071 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:33:02.071 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:48.618850+0000", 2024-09-14T06:33:02.071 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.8720000000000008e-06 2024-09-14T06:33:02.071 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:02.071 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:02.071 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:33:02.071 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:48.619393+0000", 2024-09-14T06:33:02.072 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00054264099999999996 2024-09-14T06:33:02.072 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:02.072 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:02.072 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "waiting for peered", 2024-09-14T06:33:02.072 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:48.619404+0000", 2024-09-14T06:33:02.072 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.1113e-05 2024-09-14T06:33:02.072 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:02.072 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:02.072 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:33:02.072 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:49.683016+0000", 2024-09-14T06:33:02.072 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.063612344 2024-09-14T06:33:02.072 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:02.072 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:02.072 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:33:02.072 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:49.683195+0000", 2024-09-14T06:33:02.073 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00017872100000000001 2024-09-14T06:33:02.073 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:02.073 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:02.073 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "waiting for subops from 2,3", 2024-09-14T06:33:02.073 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:49.683310+0000", 2024-09-14T06:33:02.073 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000114541 2024-09-14T06:33:02.073 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:02.073 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:02.073 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:33:02.073 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:49.684783+0000", 2024-09-14T06:33:02.073 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0014735359999999999 2024-09-14T06:33:02.073 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:02.073 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:02.073 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:33:02.074 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:49.684864+0000", 2024-09-14T06:33:02.074 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.1166000000000002e-05 2024-09-14T06:33:02.074 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:02.074 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:02.074 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "op_commit", 2024-09-14T06:33:02.074 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:49.684887+0000", 2024-09-14T06:33:02.074 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.2132000000000001e-05 2024-09-14T06:33:02.074 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:02.074 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:02.074 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-14T06:33:02.074 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:49.684929+0000", 2024-09-14T06:33:02.074 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.2141999999999997e-05 2024-09-14T06:33:02.074 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:02.074 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:02.074 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:33:02.075 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:49.685056+0000", 2024-09-14T06:33:02.075 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000126857 2024-09-14T06:33:02.075 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:33:02.075 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:33:02.075 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:33:02.075 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:33:02.075 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:33:02.075 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-14T06:33:02.077 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-14T06:33:02.077 INFO:tasks.ceph.osd.0.smithi094.stderr:2024-09-14T06:33:02.075+0000 7f07c9333640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33636) UID: 0 2024-09-14T06:33:02.177 INFO:tasks.ceph.osd.2.smithi094.stderr:2024-09-14T06:33:02.176+0000 7faf3269b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33649) UID: 0 2024-09-14T06:33:02.221 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-14T06:33:02.221 INFO:teuthology.orchestra.run.smithi094.stdout: "ops": [ 2024-09-14T06:33:02.221 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:02.221 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:240324 3.a 3.fa1caca (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e256)", 2024-09-14T06:33:02.221 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:33:01.618472+0000", 2024-09-14T06:33:02.221 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 0.60202822600000006, 2024-09-14T06:33:02.221 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.60204876200000002, 2024-09-14T06:33:02.221 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:33:02.221 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:33:02.222 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "waiting for peered", 2024-09-14T06:33:02.222 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:33:02.222 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:33:02.222 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:33:02.222 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 240324 2024-09-14T06:33:02.222 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:02.222 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:33:02.222 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:02.222 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:33:02.222 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:33:01.618472+0000", 2024-09-14T06:33:02.222 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:33:02.222 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:02.222 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:02.222 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:33:02.223 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:33:01.618472+0000", 2024-09-14T06:33:02.223 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:33:02.223 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:02.223 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:02.223 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:33:02.223 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:33:01.618474+0000", 2024-09-14T06:33:02.223 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.6750000000000001e-06 2024-09-14T06:33:02.223 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:02.223 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:02.223 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:33:02.223 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:33:01.618482+0000", 2024-09-14T06:33:02.223 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.8110000000000003e-06 2024-09-14T06:33:02.223 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:02.223 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:02.223 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:33:02.224 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:33:01.618506+0000", 2024-09-14T06:33:02.224 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.4037999999999999e-05 2024-09-14T06:33:02.224 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:02.224 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:02.224 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:33:02.224 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:33:01.618518+0000", 2024-09-14T06:33:02.224 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.2352e-05 2024-09-14T06:33:02.224 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:02.224 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:02.224 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:33:02.224 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:33:01.618610+0000", 2024-09-14T06:33:02.224 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.2021999999999994e-05 2024-09-14T06:33:02.224 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:02.225 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:02.225 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "op must wait for map", 2024-09-14T06:33:02.225 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:33:01.618621+0000", 2024-09-14T06:33:02.225 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.0753e-05 2024-09-14T06:33:02.225 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:02.225 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:02.225 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:33:02.225 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:33:01.621380+0000", 2024-09-14T06:33:02.225 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.002758768 2024-09-14T06:33:02.225 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:02.225 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:02.225 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "waiting for peered", 2024-09-14T06:33:02.225 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:33:01.621392+0000", 2024-09-14T06:33:02.226 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.2221e-05 2024-09-14T06:33:02.226 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:33:02.226 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:33:02.226 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:33:02.226 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:33:02.226 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-09-14T06:33:02.226 INFO:teuthology.orchestra.run.smithi094.stdout: "num_ops": 1 2024-09-14T06:33:02.226 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-14T06:33:02.230 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-14T06:33:02.314 INFO:tasks.ceph.osd.3.smithi094.stderr:2024-09-14T06:33:02.313+0000 7feeeafe8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33655) UID: 0 2024-09-14T06:33:02.373 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-14T06:33:02.374 INFO:teuthology.orchestra.run.smithi094.stdout: "ops": [], 2024-09-14T06:33:02.374 INFO:teuthology.orchestra.run.smithi094.stdout: "complaint_time": 30, 2024-09-14T06:33:02.374 INFO:teuthology.orchestra.run.smithi094.stdout: "num_blocked_ops": 0 2024-09-14T06:33:02.374 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-14T06:33:02.379 INFO:tasks.ceph.osd.1.smithi094.stderr:2024-09-14T06:33:02.378+0000 7f328e47a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33641) UID: 0 2024-09-14T06:33:02.382 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-14T06:33:02.480 INFO:tasks.ceph.osd.4.smithi138.stderr:2024-09-14T06:33:02.479+0000 7f6c653c8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33615) UID: 0 2024-09-14T06:33:02.527 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-14T06:33:02.527 INFO:teuthology.orchestra.run.smithi094.stdout: "size": 20, 2024-09-14T06:33:02.527 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 600, 2024-09-14T06:33:02.527 INFO:teuthology.orchestra.run.smithi094.stdout: "ops": [ 2024-09-14T06:33:02.527 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:02.527 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4397.0:74241 3.0 e73/23 3:06e93fb9:::rbd_data.112d80ca42af.0000000000000000:head v 73'102, mlcod=71'100)", 2024-09-14T06:33:02.527 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:53.876730+0000", 2024-09-14T06:33:02.527 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 188.64839712899999, 2024-09-14T06:33:02.527 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0079470329999999992, 2024-09-14T06:33:02.528 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:33:02.528 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:33:02.528 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:33:02.528 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:33:02.528 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:02.528 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:33:02.528 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:53.876730+0000", 2024-09-14T06:33:02.528 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:33:02.528 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:02.528 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:02.528 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:33:02.528 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:53.876730+0000", 2024-09-14T06:33:02.528 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:33:02.529 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:02.529 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:02.529 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:33:02.529 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:53.876731+0000", 2024-09-14T06:33:02.529 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.0500000000000004e-07 2024-09-14T06:33:02.529 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:02.529 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:02.529 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:33:02.529 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:53.879616+0000", 2024-09-14T06:33:02.529 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0028856680000000001 2024-09-14T06:33:02.529 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:02.529 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:02.529 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:33:02.530 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:53.879643+0000", 2024-09-14T06:33:02.530 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.6120999999999999e-05 2024-09-14T06:33:02.530 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:02.530 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:02.530 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:33:02.530 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:53.879654+0000", 2024-09-14T06:33:02.530 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.1586999999999999e-05 2024-09-14T06:33:02.530 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:02.530 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:02.530 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:33:02.530 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:53.879756+0000", 2024-09-14T06:33:02.530 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00010223899999999999 2024-09-14T06:33:02.530 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:02.530 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:02.531 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:33:02.531 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:53.879783+0000", 2024-09-14T06:33:02.531 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.6913000000000001e-05 2024-09-14T06:33:02.531 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:02.531 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:02.531 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-14T06:33:02.531 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:53.884594+0000", 2024-09-14T06:33:02.531 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0048110729999999999 2024-09-14T06:33:02.531 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:02.531 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:02.531 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:33:02.531 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:53.884677+0000", 2024-09-14T06:33:02.531 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.2927000000000001e-05 2024-09-14T06:33:02.531 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:33:02.531 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:33:02.532 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:33:02.532 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:02.532 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:02.532 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4397.0:74243 3.9 e73/23 3:94977f81:::rbd_data.112d80ca42af.0000000000000002:head v 73'94, mlcod=71'92)", 2024-09-14T06:33:02.532 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:53.885156+0000", 2024-09-14T06:33:02.532 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 188.63997121099999, 2024-09-14T06:33:02.532 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0096009860000000006, 2024-09-14T06:33:02.532 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:33:02.532 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:33:02.532 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:33:02.532 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:33:02.532 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:02.532 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:33:02.533 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:53.885156+0000", 2024-09-14T06:33:02.533 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:33:02.533 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:02.533 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:02.533 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:33:02.533 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:53.885156+0000", 2024-09-14T06:33:02.533 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:33:02.533 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:02.533 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:02.533 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:33:02.533 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:53.885156+0000", 2024-09-14T06:33:02.533 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.23e-07 2024-09-14T06:33:02.533 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:02.533 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:02.534 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:33:02.534 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:53.889694+0000", 2024-09-14T06:33:02.534 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0045378709999999997 2024-09-14T06:33:02.534 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:02.534 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:02.534 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:33:02.534 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:53.889720+0000", 2024-09-14T06:33:02.534 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.5259e-05 2024-09-14T06:33:02.534 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:02.534 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:02.534 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:33:02.534 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:53.889731+0000", 2024-09-14T06:33:02.534 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.1739000000000001e-05 2024-09-14T06:33:02.534 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:02.534 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:02.535 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:33:02.535 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:53.889822+0000", 2024-09-14T06:33:02.535 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.1200999999999996e-05 2024-09-14T06:33:02.535 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:02.535 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:02.535 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:33:02.535 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:53.889850+0000", 2024-09-14T06:33:02.535 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.7084999999999999e-05 2024-09-14T06:33:02.535 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:02.535 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:02.535 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-14T06:33:02.535 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:53.894703+0000", 2024-09-14T06:33:02.535 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0048532690000000003 2024-09-14T06:33:02.535 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:02.536 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:02.536 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:33:02.536 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:53.894757+0000", 2024-09-14T06:33:02.536 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.4339000000000003e-05 2024-09-14T06:33:02.536 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:33:02.536 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:33:02.536 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:33:02.536 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:02.536 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:02.536 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4397.0:74249 3.0 e73/23 3:009b0225:::rbd_data.112d80ca42af.0000000000000009:head v 73'103, mlcod=71'101)", 2024-09-14T06:33:02.536 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:53.893219+0000", 2024-09-14T06:33:02.536 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 188.63190873299999, 2024-09-14T06:33:02.536 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0065381980000000003, 2024-09-14T06:33:02.536 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:33:02.537 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:33:02.537 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:33:02.537 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:33:02.537 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:02.537 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:33:02.537 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:53.893219+0000", 2024-09-14T06:33:02.537 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:33:02.537 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:02.537 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:02.537 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:33:02.537 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:53.893219+0000", 2024-09-14T06:33:02.537 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:33:02.537 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:02.538 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:02.538 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:33:02.538 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:53.893221+0000", 2024-09-14T06:33:02.538 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.4329999999999998e-06 2024-09-14T06:33:02.538 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:02.538 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:02.538 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:33:02.538 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:53.895968+0000", 2024-09-14T06:33:02.538 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.002746746 2024-09-14T06:33:02.538 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:02.538 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:02.538 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:33:02.538 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:53.895988+0000", 2024-09-14T06:33:02.538 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.0227000000000001e-05 2024-09-14T06:33:02.538 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:02.539 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:02.539 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:33:02.539 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:53.896000+0000", 2024-09-14T06:33:02.539 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.2359e-05 2024-09-14T06:33:02.539 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:02.539 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:02.539 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:33:02.539 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:53.896084+0000", 2024-09-14T06:33:02.539 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.3659999999999995e-05 2024-09-14T06:33:02.539 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:02.539 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:02.539 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:33:02.539 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:53.896109+0000", 2024-09-14T06:33:02.539 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.5259999999999999e-05 2024-09-14T06:33:02.539 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:02.540 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:02.540 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-14T06:33:02.540 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:53.899704+0000", 2024-09-14T06:33:02.540 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0035942209999999999 2024-09-14T06:33:02.540 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:02.540 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:02.540 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:33:02.540 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:53.899757+0000", 2024-09-14T06:33:02.540 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.3291999999999997e-05 2024-09-14T06:33:02.540 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:33:02.540 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:33:02.540 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:33:02.540 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:02.540 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:02.541 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4397.0:149933 3.4 e122/23 3:234f5229:::rbd_data.112d1bb7b269.0000000000000009:head v 122'221, mlcod=120'218)", 2024-09-14T06:33:02.541 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:30:44.743849+0000", 2024-09-14T06:33:02.541 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 137.781278566, 2024-09-14T06:33:02.541 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0031479519999999999, 2024-09-14T06:33:02.541 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:33:02.541 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:33:02.541 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:33:02.541 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:33:02.541 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:02.541 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:33:02.541 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.743849+0000", 2024-09-14T06:33:02.541 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:33:02.541 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:02.541 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:02.542 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:33:02.542 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.743849+0000", 2024-09-14T06:33:02.542 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:33:02.542 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:02.542 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:02.542 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:33:02.542 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.743850+0000", 2024-09-14T06:33:02.542 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.279e-06 2024-09-14T06:33:02.542 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:02.542 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:02.542 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:33:02.542 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.745151+0000", 2024-09-14T06:33:02.542 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.001300495 2024-09-14T06:33:02.542 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:02.543 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:02.543 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:33:02.543 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.745167+0000", 2024-09-14T06:33:02.543 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.6478e-05 2024-09-14T06:33:02.543 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:02.543 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:02.543 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:33:02.543 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.745175+0000", 2024-09-14T06:33:02.543 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.2679999999999994e-06 2024-09-14T06:33:02.543 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:02.543 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:02.543 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:33:02.543 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.745226+0000", 2024-09-14T06:33:02.543 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.0192999999999999e-05 2024-09-14T06:33:02.543 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:02.544 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:02.544 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:33:02.544 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.745247+0000", 2024-09-14T06:33:02.544 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.1767e-05 2024-09-14T06:33:02.544 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:02.544 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:02.544 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-14T06:33:02.544 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.746952+0000", 2024-09-14T06:33:02.544 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0017042769999999999 2024-09-14T06:33:02.544 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:02.544 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:02.544 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:33:02.544 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:30:44.746997+0000", 2024-09-14T06:33:02.544 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.5194999999999997e-05 2024-09-14T06:33:02.544 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:33:02.545 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:33:02.545 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:33:02.545 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:02.545 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:02.545 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4397.0:204589 3.4 e166/23 3:234552f0:::rbd_data.112d3bd84cb0.000000000000003f:head v 166'499, mlcod=165'497)", 2024-09-14T06:33:02.545 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:31:30.772894+0000", 2024-09-14T06:33:02.545 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 91.752233079000007, 2024-09-14T06:33:02.545 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0034564650000000001, 2024-09-14T06:33:02.545 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:33:02.545 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:33:02.545 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:33:02.545 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:33:02.545 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:02.545 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:33:02.546 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:30.772894+0000", 2024-09-14T06:33:02.546 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:33:02.546 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:02.547 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:02.548 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:33:02.548 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:30.772894+0000", 2024-09-14T06:33:02.548 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:33:02.548 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:02.548 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:02.548 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:33:02.548 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:30.772896+0000", 2024-09-14T06:33:02.548 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.5379999999999999e-06 2024-09-14T06:33:02.548 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:02.548 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:02.548 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:33:02.548 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:30.773042+0000", 2024-09-14T06:33:02.548 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000146107 2024-09-14T06:33:02.549 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:02.549 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:02.549 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:33:02.549 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:30.773061+0000", 2024-09-14T06:33:02.549 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.8862999999999999e-05 2024-09-14T06:33:02.549 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:02.549 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:02.549 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:33:02.549 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:30.773071+0000", 2024-09-14T06:33:02.549 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.0227000000000001e-05 2024-09-14T06:33:02.549 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:02.549 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:02.549 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:33:02.549 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:30.773149+0000", 2024-09-14T06:33:02.550 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.8071999999999994e-05 2024-09-14T06:33:02.550 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:02.550 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:02.550 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:33:02.550 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:30.773171+0000", 2024-09-14T06:33:02.550 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.1699999999999999e-05 2024-09-14T06:33:02.550 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:02.550 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:02.550 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-14T06:33:02.550 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:30.776285+0000", 2024-09-14T06:33:02.550 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0031142050000000001 2024-09-14T06:33:02.550 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:02.550 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:02.550 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:33:02.551 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:30.776351+0000", 2024-09-14T06:33:02.551 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.5753000000000006e-05 2024-09-14T06:33:02.551 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:33:02.551 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:33:02.551 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:33:02.551 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:02.551 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:02.551 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4397.0:204598 3.4 e166/23 3:245d069e:::rbd_data.112d3bd84cb0.0000000000000041:head v 166'500, mlcod=165'497)", 2024-09-14T06:33:02.551 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:31:30.775247+0000", 2024-09-14T06:33:02.551 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 91.749879956000001, 2024-09-14T06:33:02.551 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00318235, 2024-09-14T06:33:02.551 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:33:02.551 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:33:02.552 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:33:02.552 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:33:02.552 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:02.552 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:33:02.552 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:30.775247+0000", 2024-09-14T06:33:02.552 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:33:02.552 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:02.552 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:02.552 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:33:02.552 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:30.775247+0000", 2024-09-14T06:33:02.552 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:33:02.552 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:02.552 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:02.552 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:33:02.553 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:30.775249+0000", 2024-09-14T06:33:02.553 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.265e-06 2024-09-14T06:33:02.553 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:02.553 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:02.553 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:33:02.553 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:30.775340+0000", 2024-09-14T06:33:02.553 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.1131999999999994e-05 2024-09-14T06:33:02.553 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:02.553 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:02.553 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:33:02.553 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:30.775352+0000", 2024-09-14T06:33:02.553 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.2294e-05 2024-09-14T06:33:02.553 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:02.553 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:02.554 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:33:02.554 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:30.775361+0000", 2024-09-14T06:33:02.554 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.6950000000000006e-06 2024-09-14T06:33:02.554 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:02.554 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:02.554 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:33:02.554 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:30.775439+0000", 2024-09-14T06:33:02.554 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.7910000000000005e-05 2024-09-14T06:33:02.554 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:02.554 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:02.554 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:33:02.554 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:30.775460+0000", 2024-09-14T06:33:02.554 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.0900999999999999e-05 2024-09-14T06:33:02.554 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:02.555 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:02.555 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-14T06:33:02.555 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:30.778385+0000", 2024-09-14T06:33:02.555 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.002925539 2024-09-14T06:33:02.555 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:02.555 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:02.555 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:33:02.555 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:30.778430+0000", 2024-09-14T06:33:02.555 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.4613999999999997e-05 2024-09-14T06:33:02.555 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:33:02.555 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:33:02.555 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:33:02.555 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:02.555 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:02.556 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:236487 3.14 3:2d502bc2:::rbd_data.112d97471187.000000000000008e:head [delete] snapc 2f=[] ondisk+write+known_if_redirected+supports_pool_eio e234)", 2024-09-14T06:33:02.556 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:32:39.870309+0000", 2024-09-14T06:33:02.556 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 22.654818057, 2024-09-14T06:33:02.556 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0042368170000000004, 2024-09-14T06:33:02.556 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:33:02.556 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:33:02.556 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:33:02.556 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:33:02.556 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:33:02.556 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:33:02.556 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 236487 2024-09-14T06:33:02.556 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:02.556 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:33:02.556 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:02.557 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:33:02.557 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:39.870309+0000", 2024-09-14T06:33:02.557 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:33:02.557 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:02.557 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:02.557 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:33:02.557 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:39.870309+0000", 2024-09-14T06:33:02.557 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:33:02.557 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:02.557 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:02.557 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:33:02.557 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:39.870311+0000", 2024-09-14T06:33:02.557 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.0080000000000001e-06 2024-09-14T06:33:02.558 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:02.558 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:02.558 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:33:02.558 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:39.870316+0000", 2024-09-14T06:33:02.558 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.7160000000000002e-06 2024-09-14T06:33:02.558 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:02.558 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:02.558 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:33:02.558 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:39.870327+0000", 2024-09-14T06:33:02.558 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.0995999999999999e-05 2024-09-14T06:33:02.558 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:02.558 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:02.558 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:33:02.559 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:39.870335+0000", 2024-09-14T06:33:02.559 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.2139999999999996e-06 2024-09-14T06:33:02.559 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:02.559 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:02.559 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:33:02.559 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:39.870403+0000", 2024-09-14T06:33:02.559 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.7966999999999995e-05 2024-09-14T06:33:02.559 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:02.559 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:02.559 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:33:02.559 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:39.870543+0000", 2024-09-14T06:33:02.559 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00014018799999999999 2024-09-14T06:33:02.559 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:02.559 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:02.559 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "waiting for subops from 7", 2024-09-14T06:33:02.560 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:39.870669+0000", 2024-09-14T06:33:02.560 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000125778 2024-09-14T06:33:02.560 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:02.560 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:02.560 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "op_commit", 2024-09-14T06:33:02.560 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:39.871410+0000", 2024-09-14T06:33:02.560 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00074113599999999999 2024-09-14T06:33:02.560 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:02.560 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:02.560 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:33:02.560 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:39.874439+0000", 2024-09-14T06:33:02.560 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0030287259999999998 2024-09-14T06:33:02.560 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:02.561 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:02.561 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-14T06:33:02.561 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:39.874495+0000", 2024-09-14T06:33:02.561 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.5855e-05 2024-09-14T06:33:02.561 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:02.561 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:02.561 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:33:02.561 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:39.874546+0000", 2024-09-14T06:33:02.561 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.1233000000000002e-05 2024-09-14T06:33:02.561 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:33:02.561 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:33:02.561 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:33:02.561 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:02.561 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:02.562 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:236490 3.14 3:2e5aef87:::rbd_data.112d97471187.00000000000000ca:head [delete] snapc 2f=[] ondisk+write+known_if_redirected+supports_pool_eio e234)", 2024-09-14T06:33:02.562 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:32:39.872130+0000", 2024-09-14T06:33:02.562 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 22.652997406000001, 2024-09-14T06:33:02.562 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0048885589999999998, 2024-09-14T06:33:02.562 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:33:02.562 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:33:02.562 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:33:02.562 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:33:02.562 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:33:02.562 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:33:02.562 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 236490 2024-09-14T06:33:02.562 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:02.562 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:33:02.562 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:02.563 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:33:02.563 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:39.872130+0000", 2024-09-14T06:33:02.563 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:33:02.563 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:02.563 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:02.563 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:33:02.563 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:39.872130+0000", 2024-09-14T06:33:02.563 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:33:02.563 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:02.563 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:02.563 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:33:02.563 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:39.872131+0000", 2024-09-14T06:33:02.563 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.0720000000000001e-06 2024-09-14T06:33:02.563 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:02.564 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:02.564 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:33:02.564 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:39.872136+0000", 2024-09-14T06:33:02.564 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.8790000000000002e-06 2024-09-14T06:33:02.564 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:02.564 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:02.564 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:33:02.564 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:39.872150+0000", 2024-09-14T06:33:02.564 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.435e-05 2024-09-14T06:33:02.565 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:02.565 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:02.565 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:33:02.565 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:39.872158+0000", 2024-09-14T06:33:02.565 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.571e-06 2024-09-14T06:33:02.565 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:02.565 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:02.565 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:33:02.565 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:39.872234+0000", 2024-09-14T06:33:02.565 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.606e-05 2024-09-14T06:33:02.565 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:02.565 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:02.565 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:33:02.566 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:39.872356+0000", 2024-09-14T06:33:02.566 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00012250499999999999 2024-09-14T06:33:02.566 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:02.566 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:02.566 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "waiting for subops from 7", 2024-09-14T06:33:02.566 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:39.872480+0000", 2024-09-14T06:33:02.566 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00012316700000000001 2024-09-14T06:33:02.566 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:02.566 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:02.566 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "op_commit", 2024-09-14T06:33:02.566 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:39.873051+0000", 2024-09-14T06:33:02.566 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00057178100000000005 2024-09-14T06:33:02.566 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:02.566 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:02.567 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:33:02.567 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:39.876925+0000", 2024-09-14T06:33:02.567 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0038739239999999999 2024-09-14T06:33:02.567 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:02.567 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:02.567 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-14T06:33:02.567 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:39.876974+0000", 2024-09-14T06:33:02.567 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.8207000000000001e-05 2024-09-14T06:33:02.567 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:02.567 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:02.567 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:33:02.567 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:39.877019+0000", 2024-09-14T06:33:02.567 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.5043000000000002e-05 2024-09-14T06:33:02.567 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:33:02.568 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:33:02.568 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:33:02.568 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:02.568 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:02.568 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:237283 3.14 3:2f6164a9:::rbd_data.112d401cec05.000000000000000a:head [delete] snapc 25=[] ondisk+write+known_if_redirected+supports_pool_eio e239)", 2024-09-14T06:33:02.568 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:32:44.693535+0000", 2024-09-14T06:33:02.568 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 17.831592352000001, 2024-09-14T06:33:02.568 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0070517319999999998, 2024-09-14T06:33:02.568 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:33:02.568 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:33:02.568 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:33:02.568 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:33:02.568 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:33:02.568 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:33:02.569 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 237283 2024-09-14T06:33:02.569 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:02.569 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:33:02.569 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:02.569 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:33:02.569 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:44.693535+0000", 2024-09-14T06:33:02.569 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:33:02.569 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:02.569 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:02.569 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:33:02.569 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:44.693535+0000", 2024-09-14T06:33:02.569 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:33:02.569 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:02.569 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:02.570 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:33:02.570 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:44.693537+0000", 2024-09-14T06:33:02.570 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.4649999999999999e-06 2024-09-14T06:33:02.570 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:02.570 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:02.570 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:33:02.570 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:44.693541+0000", 2024-09-14T06:33:02.570 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.4950000000000002e-06 2024-09-14T06:33:02.570 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:02.570 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:02.570 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:33:02.570 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:44.693553+0000", 2024-09-14T06:33:02.570 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.2238e-05 2024-09-14T06:33:02.570 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:02.571 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:02.571 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:33:02.571 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:44.693561+0000", 2024-09-14T06:33:02.571 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.4379999999999998e-06 2024-09-14T06:33:02.571 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:02.571 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:02.571 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:33:02.571 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:44.693633+0000", 2024-09-14T06:33:02.571 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.2234e-05 2024-09-14T06:33:02.571 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:02.571 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:02.571 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "op must wait for map", 2024-09-14T06:33:02.571 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:44.693642+0000", 2024-09-14T06:33:02.571 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.6770000000000001e-06 2024-09-14T06:33:02.572 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:02.572 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:02.572 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:33:02.572 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:44.695664+0000", 2024-09-14T06:33:02.572 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.002022252 2024-09-14T06:33:02.572 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:02.572 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:02.572 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:33:02.572 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:44.695825+0000", 2024-09-14T06:33:02.572 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00016144099999999999 2024-09-14T06:33:02.572 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:02.572 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:02.572 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "waiting for subops from 7", 2024-09-14T06:33:02.573 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:44.695947+0000", 2024-09-14T06:33:02.573 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000121674 2024-09-14T06:33:02.573 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:02.573 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:02.573 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "op_commit", 2024-09-14T06:33:02.573 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:44.696853+0000", 2024-09-14T06:33:02.573 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00090605000000000004 2024-09-14T06:33:02.573 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:02.573 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:02.573 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:33:02.573 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:44.700467+0000", 2024-09-14T06:33:02.573 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0036137909999999999 2024-09-14T06:33:02.573 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:02.574 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:02.574 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-14T06:33:02.574 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:44.700533+0000", 2024-09-14T06:33:02.574 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.6583999999999998e-05 2024-09-14T06:33:02.574 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:02.574 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:02.574 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:33:02.574 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:44.700587+0000", 2024-09-14T06:33:02.574 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.3393000000000002e-05 2024-09-14T06:33:02.574 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:33:02.574 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:33:02.574 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:33:02.574 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:02.575 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:02.575 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:237364 3.14 3:2a8d6252:::rbd_data.112d401cec05.000000000000005a:head [delete] snapc 25=[] ondisk+write+known_if_redirected+supports_pool_eio e239)", 2024-09-14T06:33:02.575 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:32:44.724057+0000", 2024-09-14T06:33:02.575 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 17.801070209999999, 2024-09-14T06:33:02.575 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.005405923, 2024-09-14T06:33:02.575 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:33:02.575 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:33:02.575 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:33:02.575 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:33:02.575 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:33:02.575 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:33:02.575 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 237364 2024-09-14T06:33:02.575 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:02.575 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:33:02.576 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:02.576 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:33:02.576 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:44.724057+0000", 2024-09-14T06:33:02.576 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:33:02.576 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:02.576 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:02.576 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:33:02.576 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:44.724057+0000", 2024-09-14T06:33:02.576 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:33:02.576 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:02.576 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:02.576 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:33:02.576 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:44.724059+0000", 2024-09-14T06:33:02.577 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.854e-06 2024-09-14T06:33:02.577 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:02.577 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:02.577 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:33:02.577 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:44.724066+0000", 2024-09-14T06:33:02.577 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.4410000000000004e-06 2024-09-14T06:33:02.577 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:02.577 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:02.577 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:33:02.577 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:44.724086+0000", 2024-09-14T06:33:02.577 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.9128e-05 2024-09-14T06:33:02.577 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:02.577 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:02.577 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:33:02.578 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:44.724095+0000", 2024-09-14T06:33:02.578 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.4900000000000006e-06 2024-09-14T06:33:02.578 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:02.578 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:02.578 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:33:02.578 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:44.724172+0000", 2024-09-14T06:33:02.578 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.7226999999999998e-05 2024-09-14T06:33:02.578 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:02.578 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:02.578 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:33:02.578 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:44.724334+0000", 2024-09-14T06:33:02.578 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000161545 2024-09-14T06:33:02.578 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:02.578 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:02.579 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "waiting for subops from 7", 2024-09-14T06:33:02.579 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:44.724472+0000", 2024-09-14T06:33:02.579 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00013770300000000001 2024-09-14T06:33:02.579 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:02.579 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:02.579 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "op_commit", 2024-09-14T06:33:02.579 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:44.725367+0000", 2024-09-14T06:33:02.579 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00089564699999999996 2024-09-14T06:33:02.579 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:02.579 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:02.579 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:33:02.579 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:44.729349+0000", 2024-09-14T06:33:02.579 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0039818539999999999 2024-09-14T06:33:02.579 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:02.580 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:02.580 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-14T06:33:02.580 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:44.729410+0000", 2024-09-14T06:33:02.580 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.0523000000000003e-05 2024-09-14T06:33:02.580 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:02.580 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:02.580 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:33:02.580 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:44.729463+0000", 2024-09-14T06:33:02.580 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.3511000000000003e-05 2024-09-14T06:33:02.580 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:33:02.580 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:33:02.580 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:33:02.580 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:02.581 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:02.581 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:237438 3.14 3:2b4d012e:::rbd_data.112d401cec05.00000000000000ae:head [delete] snapc 25=[] ondisk+write+known_if_redirected+supports_pool_eio e239)", 2024-09-14T06:33:02.581 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:32:44.755579+0000", 2024-09-14T06:33:02.581 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 17.769548084, 2024-09-14T06:33:02.581 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0056954090000000002, 2024-09-14T06:33:02.581 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:33:02.581 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:33:02.581 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:33:02.581 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:33:02.581 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:33:02.581 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:33:02.581 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 237438 2024-09-14T06:33:02.581 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:02.581 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:33:02.582 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:02.582 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:33:02.582 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:44.755579+0000", 2024-09-14T06:33:02.582 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:33:02.582 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:02.582 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:02.582 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:33:02.582 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:44.755579+0000", 2024-09-14T06:33:02.582 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:33:02.582 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:02.582 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:02.582 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:33:02.582 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:44.755581+0000", 2024-09-14T06:33:02.583 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.9099999999999999e-06 2024-09-14T06:33:02.583 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:02.583 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:02.583 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:33:02.583 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:44.755588+0000", 2024-09-14T06:33:02.583 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.9580000000000001e-06 2024-09-14T06:33:02.583 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:02.583 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:02.583 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:33:02.583 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:44.755605+0000", 2024-09-14T06:33:02.583 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.6481e-05 2024-09-14T06:33:02.583 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:02.583 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:02.583 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:33:02.584 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:44.755613+0000", 2024-09-14T06:33:02.584 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.2490000000000001e-06 2024-09-14T06:33:02.584 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:02.584 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:02.584 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:33:02.584 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:44.755706+0000", 2024-09-14T06:33:02.584 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.3346999999999999e-05 2024-09-14T06:33:02.584 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:02.584 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:02.584 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:33:02.584 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:44.755858+0000", 2024-09-14T06:33:02.584 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00015165600000000001 2024-09-14T06:33:02.584 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:02.584 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:02.585 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "waiting for subops from 7", 2024-09-14T06:33:02.585 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:44.755984+0000", 2024-09-14T06:33:02.585 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000125907 2024-09-14T06:33:02.585 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:02.585 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:02.585 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "op_commit", 2024-09-14T06:33:02.585 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:44.756892+0000", 2024-09-14T06:33:02.585 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00090802199999999997 2024-09-14T06:33:02.585 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:02.585 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:02.585 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:33:02.585 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:44.761135+0000", 2024-09-14T06:33:02.585 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.004243251 2024-09-14T06:33:02.586 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:02.586 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:02.586 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-14T06:33:02.586 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:44.761211+0000", 2024-09-14T06:33:02.586 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.6027999999999998e-05 2024-09-14T06:33:02.586 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:02.586 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:02.586 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:33:02.586 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:44.761275+0000", 2024-09-14T06:33:02.586 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.3600000000000001e-05 2024-09-14T06:33:02.586 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:33:02.586 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:33:02.586 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:33:02.586 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:02.587 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:02.587 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:237445 3.14 3:2e12370a:::rbd_data.112d401cec05.00000000000000b5:head [delete] snapc 25=[] ondisk+write+known_if_redirected+supports_pool_eio e239)", 2024-09-14T06:33:02.587 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:32:44.757373+0000", 2024-09-14T06:33:02.587 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 17.767754261, 2024-09-14T06:33:02.587 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0063576639999999999, 2024-09-14T06:33:02.587 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:33:02.587 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:33:02.587 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:33:02.587 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:33:02.587 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:33:02.587 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:33:02.587 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 237445 2024-09-14T06:33:02.587 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:02.588 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:33:02.588 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:02.588 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:33:02.588 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:44.757373+0000", 2024-09-14T06:33:02.588 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:33:02.588 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:02.588 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:02.588 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:33:02.588 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:44.757373+0000", 2024-09-14T06:33:02.588 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:33:02.588 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:02.588 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:02.588 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:33:02.588 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:44.757374+0000", 2024-09-14T06:33:02.589 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.359e-06 2024-09-14T06:33:02.589 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:02.589 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:02.589 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:33:02.589 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:44.757381+0000", 2024-09-14T06:33:02.589 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.7769999999999997e-06 2024-09-14T06:33:02.589 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:02.589 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:02.589 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:33:02.589 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:44.757397+0000", 2024-09-14T06:33:02.589 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.5610000000000001e-05 2024-09-14T06:33:02.589 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:02.589 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:02.589 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:33:02.589 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:44.757406+0000", 2024-09-14T06:33:02.590 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.8300000000000002e-06 2024-09-14T06:33:02.590 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:02.590 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:02.590 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:33:02.590 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:44.757462+0000", 2024-09-14T06:33:02.590 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.6348000000000003e-05 2024-09-14T06:33:02.590 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:02.590 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:02.590 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:33:02.590 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:44.757579+0000", 2024-09-14T06:33:02.590 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000116747 2024-09-14T06:33:02.590 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:02.590 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:02.590 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "waiting for subops from 7", 2024-09-14T06:33:02.591 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:44.757704+0000", 2024-09-14T06:33:02.591 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00012546899999999999 2024-09-14T06:33:02.591 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:02.591 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:02.591 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "op_commit", 2024-09-14T06:33:02.591 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:44.758510+0000", 2024-09-14T06:33:02.591 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00080536099999999999 2024-09-14T06:33:02.591 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:02.591 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:02.591 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:33:02.591 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:44.763596+0000", 2024-09-14T06:33:02.591 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0050862049999999999 2024-09-14T06:33:02.591 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:02.592 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:02.592 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-14T06:33:02.592 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:44.763673+0000", 2024-09-14T06:33:02.592 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.7464000000000002e-05 2024-09-14T06:33:02.592 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:02.592 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:02.592 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:33:02.592 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:44.763731+0000", 2024-09-14T06:33:02.592 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.7494000000000002e-05 2024-09-14T06:33:02.592 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:33:02.592 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:33:02.592 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:33:02.592 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:02.592 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:02.593 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:238182 3.a 3.3eafbe6a (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e242)", 2024-09-14T06:33:02.593 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:32:47.709833+0000", 2024-09-14T06:33:02.593 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 14.815294134, 2024-09-14T06:33:02.593 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.90868078100000005, 2024-09-14T06:33:02.593 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:33:02.593 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:33:02.593 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "reached pg", 2024-09-14T06:33:02.593 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:33:02.593 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:33:02.593 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:33:02.593 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 238182 2024-09-14T06:33:02.593 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:02.593 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:33:02.593 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:02.594 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:33:02.594 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:47.709833+0000", 2024-09-14T06:33:02.594 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:33:02.594 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:02.594 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:02.594 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:33:02.594 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:47.709833+0000", 2024-09-14T06:33:02.594 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:33:02.594 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:02.594 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:02.594 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:33:02.594 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:47.709835+0000", 2024-09-14T06:33:02.594 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.0020000000000001e-06 2024-09-14T06:33:02.594 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:02.595 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:02.595 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:33:02.595 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:47.709843+0000", 2024-09-14T06:33:02.595 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.0779999999999996e-06 2024-09-14T06:33:02.595 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:02.595 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:02.595 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:33:02.595 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:47.709864+0000", 2024-09-14T06:33:02.595 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.0605999999999999e-05 2024-09-14T06:33:02.595 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:02.595 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:02.595 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:33:02.595 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:47.709874+0000", 2024-09-14T06:33:02.595 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.9340000000000001e-06 2024-09-14T06:33:02.595 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:02.595 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:02.596 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:33:02.596 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:47.709948+0000", 2024-09-14T06:33:02.596 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.3796999999999998e-05 2024-09-14T06:33:02.596 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:02.596 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:02.596 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "waiting for peered", 2024-09-14T06:33:02.596 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:47.709956+0000", 2024-09-14T06:33:02.596 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.7760000000000003e-06 2024-09-14T06:33:02.596 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:02.596 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:02.596 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:33:02.596 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:48.618503+0000", 2024-09-14T06:33:02.596 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.90854614899999997 2024-09-14T06:33:02.597 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:02.597 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:02.597 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:33:02.597 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:48.618514+0000", 2024-09-14T06:33:02.597 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.1439e-05 2024-09-14T06:33:02.597 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:33:02.597 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:33:02.597 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:33:02.597 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:02.597 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:02.597 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:238198 3.14 3:2d396db2:::rbd_data.112d80ca42af.0000000000000017:head [delete] snapc 1f=[] ondisk+write+known_if_redirected+supports_pool_eio e242)", 2024-09-14T06:33:02.597 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:32:47.719965+0000", 2024-09-14T06:33:02.597 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 14.805162621999999, 2024-09-14T06:33:02.597 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0039876809999999999, 2024-09-14T06:33:02.598 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:33:02.598 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:33:02.598 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:33:02.598 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:33:02.598 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:33:02.598 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:33:02.598 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 238198 2024-09-14T06:33:02.598 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:02.598 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:33:02.598 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:02.598 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:33:02.598 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:47.719965+0000", 2024-09-14T06:33:02.598 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:33:02.598 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:02.598 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:02.599 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:33:02.599 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:47.719965+0000", 2024-09-14T06:33:02.599 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:33:02.599 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:02.599 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:02.599 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:33:02.599 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:47.719966+0000", 2024-09-14T06:33:02.599 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.319e-06 2024-09-14T06:33:02.599 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:02.599 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:02.599 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:33:02.599 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:47.719972+0000", 2024-09-14T06:33:02.599 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.4709999999999998e-06 2024-09-14T06:33:02.599 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:02.600 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:02.600 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:33:02.600 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:47.719982+0000", 2024-09-14T06:33:02.600 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.029e-05 2024-09-14T06:33:02.600 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:02.600 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:02.600 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:33:02.600 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:47.719988+0000", 2024-09-14T06:33:02.600 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.6050000000000003e-06 2024-09-14T06:33:02.600 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:02.600 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:02.600 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:33:02.600 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:47.720064+0000", 2024-09-14T06:33:02.600 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.5893e-05 2024-09-14T06:33:02.601 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:02.601 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:02.601 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:33:02.601 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:47.720199+0000", 2024-09-14T06:33:02.601 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00013506100000000001 2024-09-14T06:33:02.601 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:02.601 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:02.601 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "waiting for subops from 7", 2024-09-14T06:33:02.601 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:47.720325+0000", 2024-09-14T06:33:02.601 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000125456 2024-09-14T06:33:02.601 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:02.601 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:02.601 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "op_commit", 2024-09-14T06:33:02.602 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:47.722213+0000", 2024-09-14T06:33:02.602 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0018876260000000001 2024-09-14T06:33:02.602 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:02.602 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:02.602 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:33:02.602 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:47.723839+0000", 2024-09-14T06:33:02.602 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0016260210000000001 2024-09-14T06:33:02.602 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:02.602 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:02.602 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-14T06:33:02.602 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:47.723898+0000", 2024-09-14T06:33:02.602 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.9728000000000001e-05 2024-09-14T06:33:02.602 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:02.602 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:02.602 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:33:02.603 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:47.723952+0000", 2024-09-14T06:33:02.603 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.4211e-05 2024-09-14T06:33:02.603 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:33:02.603 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:33:02.603 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:33:02.603 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:02.603 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:02.603 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:238917 3.a 3:52903c9c:::rbd_data.112d5a737e72.000000000000005c:head [delete] snapc 19=[] ondisk+write+known_if_redirected+supports_pool_eio e248)", 2024-09-14T06:33:02.603 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:32:53.735625+0000", 2024-09-14T06:33:02.603 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 8.7895020620000004, 2024-09-14T06:33:02.603 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0035851860000000002, 2024-09-14T06:33:02.603 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:33:02.603 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:33:02.604 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:33:02.604 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:33:02.604 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:33:02.604 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:33:02.604 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 238917 2024-09-14T06:33:02.604 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:02.604 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:33:02.604 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:02.604 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:33:02.604 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:53.735625+0000", 2024-09-14T06:33:02.604 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:33:02.604 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:02.604 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:02.604 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:33:02.605 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:53.735625+0000", 2024-09-14T06:33:02.605 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:33:02.605 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:02.605 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:02.605 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:33:02.605 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:53.735627+0000", 2024-09-14T06:33:02.605 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.699e-06 2024-09-14T06:33:02.605 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:02.605 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:02.605 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:33:02.605 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:53.735633+0000", 2024-09-14T06:33:02.605 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.896e-06 2024-09-14T06:33:02.605 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:02.605 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:02.606 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:33:02.606 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:53.735649+0000", 2024-09-14T06:33:02.606 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.5573000000000002e-05 2024-09-14T06:33:02.606 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:02.606 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:02.606 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:33:02.606 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:53.735657+0000", 2024-09-14T06:33:02.606 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.0520000000000002e-06 2024-09-14T06:33:02.606 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:02.606 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:02.606 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:33:02.606 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:53.735724+0000", 2024-09-14T06:33:02.606 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.7594000000000004e-05 2024-09-14T06:33:02.606 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:02.607 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:02.607 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:33:02.607 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:53.735906+0000", 2024-09-14T06:33:02.607 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00018224800000000001 2024-09-14T06:33:02.607 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:02.607 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:02.607 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "waiting for subops from 3", 2024-09-14T06:33:02.607 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:53.736040+0000", 2024-09-14T06:33:02.607 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000133628 2024-09-14T06:33:02.607 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:02.607 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:02.607 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "op_commit", 2024-09-14T06:33:02.607 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:53.736863+0000", 2024-09-14T06:33:02.607 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00082262100000000005 2024-09-14T06:33:02.608 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:02.608 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:02.608 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:33:02.608 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:53.739079+0000", 2024-09-14T06:33:02.608 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.002216057 2024-09-14T06:33:02.608 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:02.608 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:02.608 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-14T06:33:02.608 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:53.739160+0000", 2024-09-14T06:33:02.608 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.1441999999999997e-05 2024-09-14T06:33:02.608 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:02.608 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:02.608 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:33:02.608 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:53.739211+0000", 2024-09-14T06:33:02.609 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.0376e-05 2024-09-14T06:33:02.609 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:33:02.609 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:33:02.609 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:33:02.609 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:02.609 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:02.609 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:239652 3.14 3:2fa4ee6b:::rbd_data.112d8f603570.0000000000000068:head [delete] snapc 11=[] ondisk+write+known_if_redirected+supports_pool_eio e252)", 2024-09-14T06:33:02.609 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:32:57.751430+0000", 2024-09-14T06:33:02.609 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 4.7736974129999998, 2024-09-14T06:33:02.609 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0032203599999999998, 2024-09-14T06:33:02.609 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:33:02.609 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:33:02.609 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:33:02.610 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:33:02.610 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:33:02.610 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:33:02.610 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 239652 2024-09-14T06:33:02.610 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:02.610 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:33:02.610 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:02.610 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:33:02.610 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:57.751430+0000", 2024-09-14T06:33:02.610 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:33:02.610 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:02.610 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:02.610 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:33:02.610 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:57.751430+0000", 2024-09-14T06:33:02.611 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:33:02.611 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:02.611 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:02.611 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:33:02.611 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:57.751431+0000", 2024-09-14T06:33:02.611 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.271e-06 2024-09-14T06:33:02.611 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:02.611 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:02.611 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:33:02.611 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:57.751436+0000", 2024-09-14T06:33:02.611 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.0520000000000001e-06 2024-09-14T06:33:02.611 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:02.611 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:02.611 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:33:02.611 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:57.751448+0000", 2024-09-14T06:33:02.611 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.1419e-05 2024-09-14T06:33:02.612 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:02.612 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:02.612 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:33:02.612 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:57.751455+0000", 2024-09-14T06:33:02.612 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.4420000000000001e-06 2024-09-14T06:33:02.612 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:02.612 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:02.612 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:33:02.612 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:57.751514+0000", 2024-09-14T06:33:02.612 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.9268999999999998e-05 2024-09-14T06:33:02.612 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:02.612 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:02.612 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:33:02.612 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:57.751650+0000", 2024-09-14T06:33:02.613 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000135332 2024-09-14T06:33:02.613 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:02.613 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:02.613 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "waiting for subops from 7", 2024-09-14T06:33:02.613 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:57.751765+0000", 2024-09-14T06:33:02.613 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000115109 2024-09-14T06:33:02.613 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:02.613 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:02.613 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "op_commit", 2024-09-14T06:33:02.613 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:57.753015+0000", 2024-09-14T06:33:02.613 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0012499919999999999 2024-09-14T06:33:02.613 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:02.613 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:02.613 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:33:02.614 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:57.754540+0000", 2024-09-14T06:33:02.614 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0015254400000000001 2024-09-14T06:33:02.614 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:02.614 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:02.614 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-14T06:33:02.614 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:57.754598+0000", 2024-09-14T06:33:02.614 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.7815999999999997e-05 2024-09-14T06:33:02.614 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:02.614 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:02.614 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:33:02.614 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:57.754650+0000", 2024-09-14T06:33:02.614 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.2218e-05 2024-09-14T06:33:02.614 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:33:02.614 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:33:02.615 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:33:02.615 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:02.615 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:02.615 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:239702 3.14 3:2ce24b3a:::rbd_data.112d8f603570.0000000000000130:head [delete] snapc 11=[] ondisk+write+known_if_redirected+supports_pool_eio e252)", 2024-09-14T06:33:02.615 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:32:57.764410+0000", 2024-09-14T06:33:02.615 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 4.7607172689999997, 2024-09-14T06:33:02.615 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0036256449999999998, 2024-09-14T06:33:02.615 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:33:02.615 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:33:02.615 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:33:02.615 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:33:02.615 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:33:02.615 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:33:02.615 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 239702 2024-09-14T06:33:02.615 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:02.616 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:33:02.616 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:02.616 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:33:02.616 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:57.764410+0000", 2024-09-14T06:33:02.616 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:33:02.616 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:02.616 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:02.616 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:33:02.616 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:57.764410+0000", 2024-09-14T06:33:02.616 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:33:02.616 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:02.616 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:02.616 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:33:02.617 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:57.764411+0000", 2024-09-14T06:33:02.617 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.062e-06 2024-09-14T06:33:02.617 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:02.617 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:02.617 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:33:02.617 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:57.764415+0000", 2024-09-14T06:33:02.617 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.0999999999999997e-06 2024-09-14T06:33:02.617 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:02.617 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:02.617 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:33:02.617 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:57.764429+0000", 2024-09-14T06:33:02.617 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.3365e-05 2024-09-14T06:33:02.617 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:02.618 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:02.618 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:33:02.618 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:57.764436+0000", 2024-09-14T06:33:02.618 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.9360000000000002e-06 2024-09-14T06:33:02.618 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:02.618 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:02.618 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:33:02.618 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:57.764483+0000", 2024-09-14T06:33:02.618 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.7747000000000002e-05 2024-09-14T06:33:02.618 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:02.618 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:02.618 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:33:02.618 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:57.764573+0000", 2024-09-14T06:33:02.618 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.9184999999999994e-05 2024-09-14T06:33:02.619 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:02.619 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:02.619 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "waiting for subops from 7", 2024-09-14T06:33:02.619 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:57.764657+0000", 2024-09-14T06:33:02.619 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.4374000000000002e-05 2024-09-14T06:33:02.619 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:02.619 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:02.619 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "op_commit", 2024-09-14T06:33:02.619 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:57.765242+0000", 2024-09-14T06:33:02.619 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00058536600000000003 2024-09-14T06:33:02.619 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:02.619 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:02.619 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:33:02.619 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:57.767916+0000", 2024-09-14T06:33:02.619 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.002673398 2024-09-14T06:33:02.620 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:02.620 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:02.620 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-14T06:33:02.620 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:57.767969+0000", 2024-09-14T06:33:02.620 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.3010000000000002e-05 2024-09-14T06:33:02.620 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:02.620 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:02.620 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:33:02.620 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:57.768036+0000", 2024-09-14T06:33:02.620 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.7101999999999996e-05 2024-09-14T06:33:02.620 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:33:02.620 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:33:02.620 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:33:02.621 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:02.621 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:02.621 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:239775 3.14 3:298708e5:::rbd_data.112d8f603570.0000000000000291:head [delete] snapc 11=[] ondisk+write+known_if_redirected+supports_pool_eio e252)", 2024-09-14T06:33:02.621 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:32:57.792966+0000", 2024-09-14T06:33:02.621 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 4.7321617219999998, 2024-09-14T06:33:02.621 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0054272879999999997, 2024-09-14T06:33:02.621 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:33:02.621 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:33:02.621 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:33:02.621 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:33:02.621 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:33:02.621 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:33:02.621 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 239775 2024-09-14T06:33:02.622 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:02.622 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:33:02.622 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:02.622 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:33:02.622 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:57.792966+0000", 2024-09-14T06:33:02.622 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:33:02.622 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:02.622 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:02.622 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:33:02.622 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:57.792966+0000", 2024-09-14T06:33:02.622 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:33:02.622 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:02.622 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:02.622 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:33:02.623 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:57.792967+0000", 2024-09-14T06:33:02.623 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.5600000000000001e-06 2024-09-14T06:33:02.623 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:02.623 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:02.623 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:33:02.623 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:57.792973+0000", 2024-09-14T06:33:02.623 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.7259999999999999e-06 2024-09-14T06:33:02.623 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:02.623 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:02.623 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:33:02.623 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:57.792990+0000", 2024-09-14T06:33:02.623 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.6651e-05 2024-09-14T06:33:02.623 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:02.623 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:02.623 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:33:02.624 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:57.792998+0000", 2024-09-14T06:33:02.624 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.3299999999999999e-06 2024-09-14T06:33:02.624 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:02.624 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:02.624 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:33:02.624 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:57.793066+0000", 2024-09-14T06:33:02.624 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.8182999999999994e-05 2024-09-14T06:33:02.624 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:02.624 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:02.624 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:33:02.624 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:57.793207+0000", 2024-09-14T06:33:02.624 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00014066600000000001 2024-09-14T06:33:02.624 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:02.624 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:02.624 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "waiting for subops from 7", 2024-09-14T06:33:02.625 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:57.793325+0000", 2024-09-14T06:33:02.625 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000118664 2024-09-14T06:33:02.625 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:02.625 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:02.625 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "op_commit", 2024-09-14T06:33:02.625 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:57.794251+0000", 2024-09-14T06:33:02.625 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00092560799999999996 2024-09-14T06:33:02.625 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:02.625 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:02.625 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:33:02.625 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:57.798268+0000", 2024-09-14T06:33:02.625 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0040170680000000004 2024-09-14T06:33:02.625 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:02.625 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:02.626 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-14T06:33:02.626 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:57.798337+0000", 2024-09-14T06:33:02.626 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.8813000000000007e-05 2024-09-14T06:33:02.626 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:02.626 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:02.626 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:33:02.626 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:57.798393+0000", 2024-09-14T06:33:02.626 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.6019e-05 2024-09-14T06:33:02.626 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:33:02.626 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:33:02.626 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:33:02.626 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:02.626 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:02.626 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:239927 3.a 3:51cf3b8a:::rbd_data.112d92027590.0000000000000006:head [delete] snapc f=[] ondisk+write+known_if_redirected+supports_pool_eio e253)", 2024-09-14T06:33:02.627 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:32:58.748865+0000", 2024-09-14T06:33:02.627 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 3.7762624059999998, 2024-09-14T06:33:02.627 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0074010889999999996, 2024-09-14T06:33:02.627 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:33:02.627 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:33:02.627 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:33:02.627 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:33:02.627 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:33:02.627 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:33:02.627 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 239927 2024-09-14T06:33:02.627 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:02.627 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:33:02.627 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:02.627 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:33:02.627 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:58.748865+0000", 2024-09-14T06:33:02.628 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:33:02.628 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:02.628 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:02.628 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:33:02.628 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:58.748865+0000", 2024-09-14T06:33:02.628 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:33:02.628 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:02.628 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:02.628 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:33:02.628 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:58.748867+0000", 2024-09-14T06:33:02.628 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.877e-06 2024-09-14T06:33:02.628 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:02.628 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:02.629 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:33:02.629 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:58.748874+0000", 2024-09-14T06:33:02.629 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.9110000000000004e-06 2024-09-14T06:33:02.629 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:02.629 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:02.629 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:33:02.629 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:58.748894+0000", 2024-09-14T06:33:02.629 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.0293e-05 2024-09-14T06:33:02.629 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:02.629 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:02.629 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:33:02.629 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:58.748904+0000", 2024-09-14T06:33:02.629 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.0395e-05 2024-09-14T06:33:02.629 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:02.629 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:02.629 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:33:02.630 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:58.748975+0000", 2024-09-14T06:33:02.630 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.0711999999999997e-05 2024-09-14T06:33:02.630 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:02.630 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:02.630 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "op must wait for map", 2024-09-14T06:33:02.630 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:58.748983+0000", 2024-09-14T06:33:02.630 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.7079999999999997e-06 2024-09-14T06:33:02.630 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:02.630 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:02.630 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:33:02.630 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:58.751692+0000", 2024-09-14T06:33:02.630 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0027086760000000001 2024-09-14T06:33:02.630 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:02.630 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:02.631 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:33:02.631 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:58.751828+0000", 2024-09-14T06:33:02.631 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00013688900000000001 2024-09-14T06:33:02.631 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:02.631 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:02.631 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:33:02.631 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:58.756266+0000", 2024-09-14T06:33:02.631 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0044376279999999999 2024-09-14T06:33:02.631 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:33:02.631 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:33:02.631 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:33:02.631 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:02.631 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:02.631 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:239960 3.a 3:518edc23:::rbd_data.112d92027590.0000000000000027:head [delete] snapc f=[] ondisk+write+known_if_redirected+supports_pool_eio e253)", 2024-09-14T06:33:02.631 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:32:58.759772+0000", 2024-09-14T06:33:02.632 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 3.7653555819999998, 2024-09-14T06:33:02.632 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0033111099999999999, 2024-09-14T06:33:02.632 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:33:02.632 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:33:02.632 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:33:02.632 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:33:02.632 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:33:02.632 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:33:02.632 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 239960 2024-09-14T06:33:02.632 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:02.632 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:33:02.632 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:02.632 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:33:02.632 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:58.759772+0000", 2024-09-14T06:33:02.633 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:33:02.633 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:02.633 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:02.633 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:33:02.633 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:58.759772+0000", 2024-09-14T06:33:02.633 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:33:02.633 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:02.633 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:02.633 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:33:02.633 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:58.759773+0000", 2024-09-14T06:33:02.633 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.85e-07 2024-09-14T06:33:02.633 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:02.633 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:02.633 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:33:02.633 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:58.759777+0000", 2024-09-14T06:33:02.634 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.0929999999999996e-06 2024-09-14T06:33:02.634 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:02.634 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:02.634 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:33:02.634 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:58.759785+0000", 2024-09-14T06:33:02.634 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.9209999999999992e-06 2024-09-14T06:33:02.634 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:02.634 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:02.634 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:33:02.634 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:58.759791+0000", 2024-09-14T06:33:02.634 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.2369999999999997e-06 2024-09-14T06:33:02.634 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:02.634 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:02.634 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:33:02.634 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:58.759987+0000", 2024-09-14T06:33:02.635 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00019611300000000001 2024-09-14T06:33:02.635 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:02.635 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:02.635 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:33:02.635 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:58.760119+0000", 2024-09-14T06:33:02.635 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00013234499999999999 2024-09-14T06:33:02.635 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:02.635 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:02.635 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:33:02.635 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:58.763083+0000", 2024-09-14T06:33:02.635 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0029635159999999998 2024-09-14T06:33:02.635 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:33:02.635 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:33:02.636 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:33:02.636 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:33:02.636 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:33:02.636 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-14T06:33:02.637 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-14T06:33:02.638 INFO:tasks.ceph.osd.7.smithi138.stderr:2024-09-14T06:33:02.635+0000 7f3c6a21d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33623) UID: 0 2024-09-14T06:33:02.638 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-14T06:33:02.738 INFO:tasks.ceph.osd.6.smithi138.stderr:2024-09-14T06:33:02.735+0000 7f8cd8849640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33620) UID: 0 2024-09-14T06:33:02.778 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-14T06:33:02.778 INFO:teuthology.orchestra.run.smithi094.stdout: "ops": [], 2024-09-14T06:33:02.778 INFO:teuthology.orchestra.run.smithi094.stdout: "num_ops": 0 2024-09-14T06:33:02.778 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-14T06:33:02.791 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-14T06:33:02.838 INFO:tasks.ceph.osd.4.smithi138.stderr:2024-09-14T06:33:02.836+0000 7f6c653c8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33615) UID: 0 2024-09-14T06:33:02.915 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-14T06:33:02.915 INFO:teuthology.orchestra.run.smithi094.stdout: "ops": [], 2024-09-14T06:33:02.915 INFO:teuthology.orchestra.run.smithi094.stdout: "complaint_time": 30, 2024-09-14T06:33:02.915 INFO:teuthology.orchestra.run.smithi094.stdout: "num_blocked_ops": 0 2024-09-14T06:33:02.916 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-14T06:33:02.925 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-14T06:33:02.970 INFO:tasks.ceph.osd.0.smithi094.stderr:2024-09-14T06:33:02.969+0000 7f07c9333640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33636) UID: 0 2024-09-14T06:33:03.037 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-14T06:33:03.037 INFO:teuthology.orchestra.run.smithi094.stdout: "size": 20, 2024-09-14T06:33:03.037 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 600, 2024-09-14T06:33:03.037 INFO:teuthology.orchestra.run.smithi094.stdout: "ops": [ 2024-09-14T06:33:03.037 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:03.038 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:44426 3.2 3:4603ad7f:::rbd_data.112dca2f6ae1.0000000000000005:17 [read 892928~4096 [fadvise_nocache]] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:33:03.038 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:43.854977+0000", 2024-09-14T06:33:03.038 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 199.180294506, 2024-09-14T06:33:03.038 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.063369237999999994, 2024-09-14T06:33:03.038 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:33:03.038 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:33:03.038 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:33:03.038 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:33:03.038 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:33:03.038 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:33:03.038 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 44426 2024-09-14T06:33:03.038 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:03.038 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:33:03.039 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:03.039 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:33:03.039 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.854977+0000", 2024-09-14T06:33:03.039 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:33:03.039 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:03.039 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:03.039 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:33:03.039 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.854977+0000", 2024-09-14T06:33:03.039 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:33:03.039 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:03.039 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:03.039 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:33:03.039 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.854977+0000", 2024-09-14T06:33:03.040 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.1199999999999999e-07 2024-09-14T06:33:03.040 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:03.040 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:03.040 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:33:03.040 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.854980+0000", 2024-09-14T06:33:03.040 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.27e-06 2024-09-14T06:33:03.040 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:03.040 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:03.040 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:33:03.040 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.854983+0000", 2024-09-14T06:33:03.040 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.1650000000000002e-06 2024-09-14T06:33:03.040 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:03.040 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:03.041 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:33:03.041 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.854989+0000", 2024-09-14T06:33:03.041 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.6520000000000003e-06 2024-09-14T06:33:03.041 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:03.041 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:03.041 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:33:03.041 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.918143+0000", 2024-09-14T06:33:03.041 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.063154541999999994 2024-09-14T06:33:03.041 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:03.041 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:03.041 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:33:03.041 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.918209+0000", 2024-09-14T06:33:03.041 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.5529000000000002e-05 2024-09-14T06:33:03.041 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:03.041 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:03.042 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:33:03.042 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.918346+0000", 2024-09-14T06:33:03.042 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000136868 2024-09-14T06:33:03.042 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:33:03.042 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:33:03.042 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:33:03.042 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:03.042 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:03.042 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:44427 3.2 3:4603ad7f:::rbd_data.112dca2f6ae1.0000000000000005:17 [read 917504~4096 [fadvise_nocache]] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:33:03.042 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:43.854992+0000", 2024-09-14T06:33:03.043 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 199.18027885999999, 2024-09-14T06:33:03.043 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.063607792999999996, 2024-09-14T06:33:03.043 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:33:03.043 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:33:03.043 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:33:03.043 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:33:03.043 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:33:03.043 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:33:03.043 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 44427 2024-09-14T06:33:03.043 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:03.043 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:33:03.043 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:03.044 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:33:03.044 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.854992+0000", 2024-09-14T06:33:03.044 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:33:03.044 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:03.044 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:03.044 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:33:03.044 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.854992+0000", 2024-09-14T06:33:03.044 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:33:03.044 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:03.044 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:03.044 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:33:03.044 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.854993+0000", 2024-09-14T06:33:03.044 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.1400000000000001e-07 2024-09-14T06:33:03.045 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:03.045 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:03.045 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:33:03.045 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.854994+0000", 2024-09-14T06:33:03.045 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.091e-06 2024-09-14T06:33:03.045 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:03.045 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:03.045 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:33:03.045 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855001+0000", 2024-09-14T06:33:03.045 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.204e-06 2024-09-14T06:33:03.045 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:03.045 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:03.045 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:33:03.045 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855003+0000", 2024-09-14T06:33:03.045 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.514e-06 2024-09-14T06:33:03.046 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:03.046 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:03.046 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:33:03.046 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.918380+0000", 2024-09-14T06:33:03.046 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.063376738000000002 2024-09-14T06:33:03.046 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:03.046 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:03.046 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:33:03.046 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.918453+0000", 2024-09-14T06:33:03.046 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.3291e-05 2024-09-14T06:33:03.046 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:03.046 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:03.046 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:33:03.046 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.918600+0000", 2024-09-14T06:33:03.047 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00014674100000000001 2024-09-14T06:33:03.047 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:33:03.047 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:33:03.047 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:33:03.047 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:03.047 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:03.047 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:44428 3.2 3:4603ad7f:::rbd_data.112dca2f6ae1.0000000000000005:17 [read 942080~4096 [fadvise_nocache]] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:33:03.047 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:43.855006+0000", 2024-09-14T06:33:03.047 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 199.18026481000001, 2024-09-14T06:33:03.047 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.063846321999999997, 2024-09-14T06:33:03.047 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:33:03.047 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:33:03.047 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:33:03.047 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:33:03.048 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:33:03.048 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:33:03.048 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 44428 2024-09-14T06:33:03.048 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:03.048 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:33:03.048 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:03.048 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:33:03.048 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855006+0000", 2024-09-14T06:33:03.048 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:33:03.048 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:03.048 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:03.048 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:33:03.048 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855006+0000", 2024-09-14T06:33:03.048 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:33:03.049 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:03.049 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:03.049 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:33:03.049 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855007+0000", 2024-09-14T06:33:03.049 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.1199999999999999e-07 2024-09-14T06:33:03.049 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:03.049 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:03.049 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:33:03.049 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855011+0000", 2024-09-14T06:33:03.049 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.9539999999999998e-06 2024-09-14T06:33:03.049 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:03.049 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:03.050 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:33:03.050 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855014+0000", 2024-09-14T06:33:03.050 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.2930000000000001e-06 2024-09-14T06:33:03.050 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:03.050 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:03.050 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:33:03.050 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855016+0000", 2024-09-14T06:33:03.050 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.3039999999999999e-06 2024-09-14T06:33:03.050 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:03.050 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:03.050 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:33:03.050 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.918634+0000", 2024-09-14T06:33:03.050 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.063617928000000004 2024-09-14T06:33:03.050 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:03.051 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:03.051 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:33:03.051 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.918711+0000", 2024-09-14T06:33:03.051 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.6545000000000007e-05 2024-09-14T06:33:03.051 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:03.051 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:03.051 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:33:03.051 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.918853+0000", 2024-09-14T06:33:03.051 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00014208600000000001 2024-09-14T06:33:03.051 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:33:03.051 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:33:03.051 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:33:03.051 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:03.051 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:03.052 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:44429 3.2 3:4603ad7f:::rbd_data.112dca2f6ae1.0000000000000005:17 [read 966656~4096 [fadvise_nocache]] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:33:03.052 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:43.855019+0000", 2024-09-14T06:33:03.052 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 199.180252137, 2024-09-14T06:33:03.052 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.064059226999999996, 2024-09-14T06:33:03.052 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:33:03.052 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:33:03.052 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:33:03.052 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:33:03.052 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:33:03.052 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:33:03.052 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 44429 2024-09-14T06:33:03.052 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:03.052 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:33:03.052 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:03.053 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:33:03.053 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855019+0000", 2024-09-14T06:33:03.053 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:33:03.053 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:03.053 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:03.053 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:33:03.053 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855019+0000", 2024-09-14T06:33:03.053 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:33:03.053 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:03.053 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:03.053 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:33:03.053 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855019+0000", 2024-09-14T06:33:03.053 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.28e-07 2024-09-14T06:33:03.053 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:03.054 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:03.054 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:33:03.054 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855020+0000", 2024-09-14T06:33:03.054 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.0330000000000001e-06 2024-09-14T06:33:03.054 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:03.054 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:03.054 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:33:03.055 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855023+0000", 2024-09-14T06:33:03.055 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.79e-06 2024-09-14T06:33:03.055 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:03.055 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:03.055 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:33:03.055 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855025+0000", 2024-09-14T06:33:03.055 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.216e-06 2024-09-14T06:33:03.055 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:03.055 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:03.055 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:33:03.055 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.918886+0000", 2024-09-14T06:33:03.055 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.063861037999999995 2024-09-14T06:33:03.055 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:03.055 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:03.056 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:33:03.056 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.918952+0000", 2024-09-14T06:33:03.056 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.5993000000000003e-05 2024-09-14T06:33:03.056 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:03.056 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:03.056 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:33:03.056 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.919078+0000", 2024-09-14T06:33:03.056 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00012592899999999999 2024-09-14T06:33:03.056 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:33:03.056 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:33:03.056 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:33:03.057 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:03.057 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:03.057 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:44430 3.2 3:4603ad7f:::rbd_data.112dca2f6ae1.0000000000000005:17 [read 991232~4096 [fadvise_nocache]] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:33:03.057 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:43.855028+0000", 2024-09-14T06:33:03.057 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 199.18024312700001, 2024-09-14T06:33:03.057 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.064284346000000006, 2024-09-14T06:33:03.057 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:33:03.057 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:33:03.057 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:33:03.057 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:33:03.057 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:33:03.057 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:33:03.057 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 44430 2024-09-14T06:33:03.058 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:03.058 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:33:03.058 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:03.058 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:33:03.058 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855028+0000", 2024-09-14T06:33:03.058 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:33:03.058 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:03.058 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:03.058 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:33:03.058 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855028+0000", 2024-09-14T06:33:03.058 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:33:03.058 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:03.058 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:03.059 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:33:03.059 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855028+0000", 2024-09-14T06:33:03.059 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.1199999999999999e-07 2024-09-14T06:33:03.059 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:03.059 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:03.059 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:33:03.059 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855029+0000", 2024-09-14T06:33:03.059 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.1100000000000004e-07 2024-09-14T06:33:03.059 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:03.059 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:03.059 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:33:03.059 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855032+0000", 2024-09-14T06:33:03.059 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.7889999999999999e-06 2024-09-14T06:33:03.059 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:03.060 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:03.060 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:33:03.060 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855034+0000", 2024-09-14T06:33:03.060 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.125e-06 2024-09-14T06:33:03.060 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:03.060 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:03.060 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:33:03.060 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.919110+0000", 2024-09-14T06:33:03.060 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.064076413999999998 2024-09-14T06:33:03.060 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:03.060 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:03.060 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:33:03.060 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.919186+0000", 2024-09-14T06:33:03.060 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.5784999999999994e-05 2024-09-14T06:33:03.061 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:03.061 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:03.061 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:33:03.061 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.919312+0000", 2024-09-14T06:33:03.061 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00012611 2024-09-14T06:33:03.061 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:33:03.061 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:33:03.061 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:33:03.061 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:03.061 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:03.061 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:44431 3.2 3:4603ad7f:::rbd_data.112dca2f6ae1.0000000000000005:17 [read 1015808~4096 [fadvise_nocache]] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:33:03.061 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:43.855037+0000", 2024-09-14T06:33:03.061 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 199.180234259, 2024-09-14T06:33:03.061 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.064469755000000004, 2024-09-14T06:33:03.062 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:33:03.062 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:33:03.062 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:33:03.062 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:33:03.062 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:33:03.062 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:33:03.062 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 44431 2024-09-14T06:33:03.062 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:03.062 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:33:03.062 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:03.062 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:33:03.062 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855037+0000", 2024-09-14T06:33:03.062 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:33:03.063 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:03.063 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:03.063 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:33:03.063 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855037+0000", 2024-09-14T06:33:03.063 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:33:03.063 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:03.063 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:03.063 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:33:03.063 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855037+0000", 2024-09-14T06:33:03.063 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.1e-07 2024-09-14T06:33:03.063 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:03.064 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:03.064 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:33:03.064 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855038+0000", 2024-09-14T06:33:03.064 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.9400000000000004e-07 2024-09-14T06:33:03.064 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:03.064 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:03.064 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:33:03.064 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855041+0000", 2024-09-14T06:33:03.064 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.9639999999999999e-06 2024-09-14T06:33:03.064 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:03.064 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:03.064 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:33:03.064 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855043+0000", 2024-09-14T06:33:03.064 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.2259999999999999e-06 2024-09-14T06:33:03.065 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:03.065 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:03.065 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:33:03.065 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.919347+0000", 2024-09-14T06:33:03.065 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.064304091999999993 2024-09-14T06:33:03.065 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:03.065 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:03.065 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:33:03.065 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.919393+0000", 2024-09-14T06:33:03.065 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.5340999999999998e-05 2024-09-14T06:33:03.065 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:03.065 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:03.066 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:33:03.066 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.919507+0000", 2024-09-14T06:33:03.066 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000114028 2024-09-14T06:33:03.066 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:33:03.066 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:33:03.066 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:33:03.066 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:03.067 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:03.067 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:44432 3.2 3:4603ad7f:::rbd_data.112dca2f6ae1.0000000000000005:17 [read 1040384~4096 [fadvise_nocache]] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:33:03.067 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:43.855046+0000", 2024-09-14T06:33:03.067 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 199.18022519100001, 2024-09-14T06:33:03.067 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.064639253999999993, 2024-09-14T06:33:03.067 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:33:03.067 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:33:03.067 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:33:03.067 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:33:03.067 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:33:03.067 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:33:03.067 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 44432 2024-09-14T06:33:03.067 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:03.068 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:33:03.068 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:03.068 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:33:03.068 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855046+0000", 2024-09-14T06:33:03.068 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:33:03.068 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:03.068 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:03.068 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:33:03.068 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855046+0000", 2024-09-14T06:33:03.068 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:33:03.068 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:03.069 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:03.069 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:33:03.069 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855046+0000", 2024-09-14T06:33:03.069 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.05e-07 2024-09-14T06:33:03.069 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:03.069 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:03.069 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:33:03.069 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855047+0000", 2024-09-14T06:33:03.069 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.879999999999999e-07 2024-09-14T06:33:03.069 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:03.069 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:03.069 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:33:03.069 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855050+0000", 2024-09-14T06:33:03.069 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.8459999999999999e-06 2024-09-14T06:33:03.070 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:03.070 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:03.070 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:33:03.070 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855052+0000", 2024-09-14T06:33:03.070 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.136e-06 2024-09-14T06:33:03.070 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:03.070 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:03.070 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:33:03.070 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.919536+0000", 2024-09-14T06:33:03.070 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.064484132 2024-09-14T06:33:03.071 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:03.071 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:03.071 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:33:03.071 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.919611+0000", 2024-09-14T06:33:03.071 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.4407999999999996e-05 2024-09-14T06:33:03.071 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:03.071 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:03.071 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:33:03.071 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.919685+0000", 2024-09-14T06:33:03.071 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.4538999999999999e-05 2024-09-14T06:33:03.071 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:33:03.071 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:33:03.071 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:33:03.072 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:03.072 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:03.072 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:44518 3.2 3:4190d882:::rbd_data.112dca2f6ae1.0000000000000002:17 [read 12288~4096 [fadvise_nocache]] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:33:03.072 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:43.855055+0000", 2024-09-14T06:33:03.072 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 199.18021630699999, 2024-09-14T06:33:03.072 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.064894317000000007, 2024-09-14T06:33:03.072 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:33:03.072 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:33:03.072 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:33:03.072 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:33:03.072 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:33:03.073 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:33:03.073 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 44518 2024-09-14T06:33:03.073 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:03.073 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:33:03.073 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:03.073 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:33:03.073 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855055+0000", 2024-09-14T06:33:03.073 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:33:03.073 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:03.073 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:03.073 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:33:03.074 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855055+0000", 2024-09-14T06:33:03.074 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:33:03.074 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:03.074 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:03.074 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:33:03.074 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855055+0000", 2024-09-14T06:33:03.074 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.0699999999999999e-07 2024-09-14T06:33:03.074 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:03.074 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:03.074 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:33:03.074 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855056+0000", 2024-09-14T06:33:03.074 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.9400000000000004e-07 2024-09-14T06:33:03.075 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:03.075 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:03.075 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:33:03.075 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855059+0000", 2024-09-14T06:33:03.075 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.8870000000000001e-06 2024-09-14T06:33:03.075 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:03.075 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:03.075 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:33:03.075 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855061+0000", 2024-09-14T06:33:03.075 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.1919999999999999e-06 2024-09-14T06:33:03.076 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:03.076 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:03.076 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:33:03.076 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.919719+0000", 2024-09-14T06:33:03.076 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.064658013 2024-09-14T06:33:03.076 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:03.076 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:03.076 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:33:03.076 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.919788+0000", 2024-09-14T06:33:03.076 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.9282000000000005e-05 2024-09-14T06:33:03.077 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:03.077 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:03.077 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:33:03.077 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.919949+0000", 2024-09-14T06:33:03.077 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00016084199999999999 2024-09-14T06:33:03.077 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:33:03.077 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:33:03.077 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:33:03.077 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:03.077 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:03.077 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:44519 3.2 3:4190d882:::rbd_data.112dca2f6ae1.0000000000000002:17 [read 36864~4096 [fadvise_nocache]] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:33:03.078 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:43.855064+0000", 2024-09-14T06:33:03.078 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 199.18020733, 2024-09-14T06:33:03.078 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.065081865000000003, 2024-09-14T06:33:03.078 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:33:03.078 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:33:03.078 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:33:03.078 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:33:03.078 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:33:03.078 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:33:03.078 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 44519 2024-09-14T06:33:03.078 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:03.079 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:33:03.079 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:03.079 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:33:03.079 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855064+0000", 2024-09-14T06:33:03.079 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:33:03.079 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:03.079 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:03.079 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:33:03.079 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855064+0000", 2024-09-14T06:33:03.079 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:33:03.079 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:03.079 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:03.079 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:33:03.079 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855064+0000", 2024-09-14T06:33:03.080 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.1e-07 2024-09-14T06:33:03.080 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:03.080 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:03.080 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:33:03.080 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855065+0000", 2024-09-14T06:33:03.080 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.035e-06 2024-09-14T06:33:03.080 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:03.080 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:03.080 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:33:03.080 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855068+0000", 2024-09-14T06:33:03.080 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.8339999999999999e-06 2024-09-14T06:33:03.080 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:03.080 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:03.080 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:33:03.081 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855071+0000", 2024-09-14T06:33:03.081 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.3450000000000002e-06 2024-09-14T06:33:03.081 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:03.081 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:03.081 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:33:03.081 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.919979+0000", 2024-09-14T06:33:03.081 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.064907647999999998 2024-09-14T06:33:03.081 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:03.081 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:03.081 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:33:03.081 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.920026+0000", 2024-09-14T06:33:03.081 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.7595000000000001e-05 2024-09-14T06:33:03.081 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:03.081 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:03.081 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:33:03.081 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.920146+0000", 2024-09-14T06:33:03.082 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00011919800000000001 2024-09-14T06:33:03.082 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:33:03.082 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:33:03.082 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:33:03.082 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:03.082 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:03.082 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:44520 3.2 3:4190d882:::rbd_data.112dca2f6ae1.0000000000000002:17 [read 61440~4096 [fadvise_nocache]] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:33:03.082 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:43.855075+0000", 2024-09-14T06:33:03.082 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 199.180196279, 2024-09-14T06:33:03.082 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.065335078000000005, 2024-09-14T06:33:03.082 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:33:03.082 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:33:03.082 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:33:03.082 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:33:03.082 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:33:03.083 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:33:03.083 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 44520 2024-09-14T06:33:03.083 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:03.083 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:33:03.083 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:03.083 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:33:03.083 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855075+0000", 2024-09-14T06:33:03.083 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:33:03.083 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:03.083 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:03.083 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:33:03.083 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855075+0000", 2024-09-14T06:33:03.083 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:33:03.083 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:03.083 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:03.084 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:33:03.084 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855075+0000", 2024-09-14T06:33:03.084 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.1e-07 2024-09-14T06:33:03.084 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:03.084 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:03.084 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:33:03.084 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855076+0000", 2024-09-14T06:33:03.084 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.0839999999999999e-06 2024-09-14T06:33:03.084 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:03.084 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:03.084 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:33:03.084 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855079+0000", 2024-09-14T06:33:03.084 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.9359999999999999e-06 2024-09-14T06:33:03.084 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:03.084 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:03.085 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:33:03.085 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855084+0000", 2024-09-14T06:33:03.085 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.1730000000000003e-06 2024-09-14T06:33:03.085 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:03.085 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:03.085 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:33:03.085 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.920175+0000", 2024-09-14T06:33:03.085 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.065090234999999996 2024-09-14T06:33:03.085 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:03.085 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:03.085 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:33:03.085 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.920250+0000", 2024-09-14T06:33:03.085 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.5807000000000001e-05 2024-09-14T06:33:03.085 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:03.086 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:03.086 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:33:03.086 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.920410+0000", 2024-09-14T06:33:03.086 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00015963300000000001 2024-09-14T06:33:03.086 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:33:03.086 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:33:03.086 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:33:03.086 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:03.086 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:03.086 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:44521 3.2 3:4190d882:::rbd_data.112dca2f6ae1.0000000000000002:17 [read 86016~4096 [fadvise_nocache]] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:33:03.086 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:43.855087+0000", 2024-09-14T06:33:03.086 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 199.180183798, 2024-09-14T06:33:03.086 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.065563264999999996, 2024-09-14T06:33:03.086 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:33:03.086 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:33:03.087 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:33:03.087 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:33:03.087 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:33:03.087 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:33:03.087 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 44521 2024-09-14T06:33:03.087 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:03.087 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:33:03.087 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:03.087 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:33:03.087 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855087+0000", 2024-09-14T06:33:03.087 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:33:03.087 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:03.087 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:03.087 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:33:03.087 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855087+0000", 2024-09-14T06:33:03.088 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:33:03.088 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:03.088 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:03.088 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:33:03.088 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855088+0000", 2024-09-14T06:33:03.088 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.3699999999999999e-07 2024-09-14T06:33:03.088 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:03.088 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:03.088 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:33:03.088 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855089+0000", 2024-09-14T06:33:03.088 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.9900000000000009e-07 2024-09-14T06:33:03.088 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:03.088 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:03.088 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:33:03.089 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855091+0000", 2024-09-14T06:33:03.089 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.8940000000000002e-06 2024-09-14T06:33:03.089 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:03.089 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:03.089 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:33:03.089 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855094+0000", 2024-09-14T06:33:03.089 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.351e-06 2024-09-14T06:33:03.089 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:03.089 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:03.089 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:33:03.089 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.920437+0000", 2024-09-14T06:33:03.089 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.065343221000000007 2024-09-14T06:33:03.089 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:03.089 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:03.090 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:33:03.090 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.920481+0000", 2024-09-14T06:33:03.090 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.4431000000000002e-05 2024-09-14T06:33:03.090 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:03.090 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:03.090 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:33:03.090 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.920651+0000", 2024-09-14T06:33:03.090 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00016913200000000001 2024-09-14T06:33:03.090 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:33:03.090 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:33:03.090 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:33:03.090 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:03.090 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:03.090 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:44522 3.2 3:4190d882:::rbd_data.112dca2f6ae1.0000000000000002:17 [read 110592~4096 [fadvise_nocache]] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:33:03.091 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:43.855098+0000", 2024-09-14T06:33:03.091 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 199.18017357100001, 2024-09-14T06:33:03.091 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.065827628999999999, 2024-09-14T06:33:03.091 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:33:03.091 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:33:03.091 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:33:03.091 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:33:03.091 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:33:03.091 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:33:03.091 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 44522 2024-09-14T06:33:03.091 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:03.091 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:33:03.091 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:03.091 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:33:03.091 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855098+0000", 2024-09-14T06:33:03.092 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:33:03.092 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:03.092 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:03.092 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:33:03.092 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855098+0000", 2024-09-14T06:33:03.092 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:33:03.092 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:03.092 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:03.092 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:33:03.092 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855098+0000", 2024-09-14T06:33:03.092 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.1e-07 2024-09-14T06:33:03.092 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:03.092 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:03.092 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:33:03.092 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855099+0000", 2024-09-14T06:33:03.093 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.012e-06 2024-09-14T06:33:03.093 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:03.093 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:03.093 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:33:03.093 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855102+0000", 2024-09-14T06:33:03.093 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.9380000000000001e-06 2024-09-14T06:33:03.093 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:03.093 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:03.093 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:33:03.093 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855108+0000", 2024-09-14T06:33:03.093 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.4339999999999997e-06 2024-09-14T06:33:03.093 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:03.093 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:03.093 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:33:03.094 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.920681+0000", 2024-09-14T06:33:03.094 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.065572943999999994 2024-09-14T06:33:03.094 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:03.094 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:03.094 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:33:03.094 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.920724+0000", 2024-09-14T06:33:03.094 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.2877000000000003e-05 2024-09-14T06:33:03.094 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:03.094 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:03.094 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:33:03.094 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.920925+0000", 2024-09-14T06:33:03.094 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00020121399999999999 2024-09-14T06:33:03.094 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:33:03.094 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:33:03.094 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:33:03.095 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:03.095 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:03.095 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:44523 3.2 3:4190d882:::rbd_data.112dca2f6ae1.0000000000000002:17 [read 135168~4096 [fadvise_nocache]] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:33:03.095 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:43.855111+0000", 2024-09-14T06:33:03.095 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 199.18015984900001, 2024-09-14T06:33:03.095 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.066028198999999996, 2024-09-14T06:33:03.095 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:33:03.095 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:33:03.095 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:33:03.095 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:33:03.095 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:33:03.095 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:33:03.095 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 44523 2024-09-14T06:33:03.095 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:03.095 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:33:03.096 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:03.096 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:33:03.096 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855111+0000", 2024-09-14T06:33:03.096 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:33:03.096 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:03.096 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:03.096 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:33:03.096 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855111+0000", 2024-09-14T06:33:03.096 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:33:03.096 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:03.096 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:03.096 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:33:03.096 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855112+0000", 2024-09-14T06:33:03.096 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.1199999999999999e-07 2024-09-14T06:33:03.096 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:03.097 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:03.097 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:33:03.097 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855115+0000", 2024-09-14T06:33:03.097 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.3380000000000001e-06 2024-09-14T06:33:03.097 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:03.097 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:03.097 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:33:03.097 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855118+0000", 2024-09-14T06:33:03.097 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.366e-06 2024-09-14T06:33:03.097 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:03.097 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:03.097 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:33:03.097 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855121+0000", 2024-09-14T06:33:03.097 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.8329999999999998e-06 2024-09-14T06:33:03.097 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:03.098 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:03.098 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:33:03.098 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.920956+0000", 2024-09-14T06:33:03.098 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.065835251999999997 2024-09-14T06:33:03.098 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:03.098 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:03.098 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:33:03.098 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.921001+0000", 2024-09-14T06:33:03.098 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.4778999999999999e-05 2024-09-14T06:33:03.098 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:03.098 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:03.098 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:33:03.098 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.921139+0000", 2024-09-14T06:33:03.098 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000138419 2024-09-14T06:33:03.099 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:33:03.099 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:33:03.099 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:33:03.099 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:03.099 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:03.099 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:44524 3.2 3:4190d882:::rbd_data.112dca2f6ae1.0000000000000002:17 [read 159744~4096 [fadvise_nocache]] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:33:03.099 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:43.855124+0000", 2024-09-14T06:33:03.099 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 199.18014686999999, 2024-09-14T06:33:03.099 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.066249261000000004, 2024-09-14T06:33:03.099 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:33:03.099 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:33:03.100 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:33:03.100 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:33:03.100 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:33:03.100 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:33:03.100 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 44524 2024-09-14T06:33:03.100 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:03.100 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:33:03.100 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:03.100 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:33:03.100 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855124+0000", 2024-09-14T06:33:03.100 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:33:03.100 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:03.101 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:03.101 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:33:03.101 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855124+0000", 2024-09-14T06:33:03.101 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:33:03.101 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:03.101 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:03.101 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:33:03.101 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861086+0000", 2024-09-14T06:33:03.101 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0059613829999999998 2024-09-14T06:33:03.101 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:03.101 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:03.101 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:33:03.101 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861100+0000", 2024-09-14T06:33:03.101 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.4644000000000001e-05 2024-09-14T06:33:03.102 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:03.102 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:03.102 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:33:03.102 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861109+0000", 2024-09-14T06:33:03.102 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.541e-06 2024-09-14T06:33:03.102 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:03.102 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:03.102 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:33:03.102 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861117+0000", 2024-09-14T06:33:03.102 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.1049999999999995e-06 2024-09-14T06:33:03.103 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:03.103 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:03.103 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:33:03.103 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.921169+0000", 2024-09-14T06:33:03.103 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.060052405000000003 2024-09-14T06:33:03.103 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:03.103 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:03.103 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:33:03.103 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.921213+0000", 2024-09-14T06:33:03.103 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.3565e-05 2024-09-14T06:33:03.103 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:03.103 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:03.103 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:33:03.103 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.921374+0000", 2024-09-14T06:33:03.103 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00016061800000000001 2024-09-14T06:33:03.104 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:33:03.104 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:33:03.104 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:33:03.104 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:03.104 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:03.104 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:44537 3.2 3:4190d882:::rbd_data.112dca2f6ae1.0000000000000002:17 [read 479232~4096 [fadvise_nocache]] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:33:03.104 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:43.861405+0000", 2024-09-14T06:33:03.104 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 199.17386626699999, 2024-09-14T06:33:03.104 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.063351795000000002, 2024-09-14T06:33:03.104 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:33:03.105 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:33:03.105 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:33:03.105 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:33:03.105 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:33:03.105 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:33:03.105 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 44537 2024-09-14T06:33:03.105 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:03.105 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:33:03.105 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:03.105 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:33:03.105 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861405+0000", 2024-09-14T06:33:03.105 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:33:03.105 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:03.105 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:03.106 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:33:03.106 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861405+0000", 2024-09-14T06:33:03.106 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:33:03.106 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:03.106 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:03.106 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:33:03.106 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861405+0000", 2024-09-14T06:33:03.106 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.9099999999999999e-07 2024-09-14T06:33:03.106 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:03.106 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:03.107 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:33:03.107 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861407+0000", 2024-09-14T06:33:03.107 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.075e-06 2024-09-14T06:33:03.107 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:03.107 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:03.107 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:33:03.107 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861414+0000", 2024-09-14T06:33:03.107 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.4659999999999996e-06 2024-09-14T06:33:03.107 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:03.107 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:03.107 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:33:03.107 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861418+0000", 2024-09-14T06:33:03.107 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.5519999999999998e-06 2024-09-14T06:33:03.108 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:03.108 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:03.108 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:33:03.108 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.924474+0000", 2024-09-14T06:33:03.108 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.063056012999999994 2024-09-14T06:33:03.108 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:03.108 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:03.108 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:33:03.108 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.924556+0000", 2024-09-14T06:33:03.108 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.1181999999999996e-05 2024-09-14T06:33:03.109 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:03.109 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:03.109 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:33:03.109 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.924757+0000", 2024-09-14T06:33:03.109 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00020111599999999999 2024-09-14T06:33:03.109 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:33:03.109 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:33:03.109 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:33:03.109 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:03.109 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:03.110 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:44538 3.2 3:4190d882:::rbd_data.112dca2f6ae1.0000000000000002:17 [read 503808~4096 [fadvise_nocache]] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:33:03.110 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:43.861425+0000", 2024-09-14T06:33:03.110 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 199.17384622599999, 2024-09-14T06:33:03.110 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.063627125000000007, 2024-09-14T06:33:03.110 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:33:03.110 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:33:03.110 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:33:03.110 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:33:03.110 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:33:03.110 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:33:03.110 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 44538 2024-09-14T06:33:03.111 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:03.111 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:33:03.111 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:03.111 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:33:03.111 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861425+0000", 2024-09-14T06:33:03.111 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:33:03.111 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:03.111 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:03.111 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:33:03.111 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861425+0000", 2024-09-14T06:33:03.112 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:33:03.112 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:03.112 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:03.112 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:33:03.112 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861425+0000", 2024-09-14T06:33:03.112 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.9999999999999998e-07 2024-09-14T06:33:03.112 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:03.112 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:03.112 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:33:03.112 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861427+0000", 2024-09-14T06:33:03.112 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.9829999999999999e-06 2024-09-14T06:33:03.112 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:03.112 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:03.113 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:33:03.113 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861433+0000", 2024-09-14T06:33:03.113 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.2889999999999997e-06 2024-09-14T06:33:03.113 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:03.113 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:03.113 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:33:03.113 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861454+0000", 2024-09-14T06:33:03.113 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.1844999999999998e-05 2024-09-14T06:33:03.113 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:03.113 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:03.113 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:33:03.113 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.924786+0000", 2024-09-14T06:33:03.113 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.063331497 2024-09-14T06:33:03.113 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:03.113 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:03.114 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:33:03.114 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.924864+0000", 2024-09-14T06:33:03.114 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.7744000000000006e-05 2024-09-14T06:33:03.114 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:03.114 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:03.114 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:33:03.114 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.925052+0000", 2024-09-14T06:33:03.114 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00018836699999999999 2024-09-14T06:33:03.114 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:33:03.114 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:33:03.114 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:33:03.115 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:03.115 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:03.115 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:44539 3.2 3:4190d882:::rbd_data.112dca2f6ae1.0000000000000002:17 [read 528384~4096 [fadvise_nocache]] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:33:03.115 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:43.861462+0000", 2024-09-14T06:33:03.115 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 199.17380889899999, 2024-09-14T06:33:03.115 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.063855705999999998, 2024-09-14T06:33:03.115 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:33:03.115 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:33:03.115 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:33:03.115 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:33:03.115 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:33:03.115 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:33:03.116 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 44539 2024-09-14T06:33:03.116 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:03.116 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:33:03.116 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:03.116 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:33:03.116 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861462+0000", 2024-09-14T06:33:03.116 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:33:03.116 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:03.116 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:03.116 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:33:03.116 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861462+0000", 2024-09-14T06:33:03.116 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:33:03.116 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:03.116 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:03.116 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:33:03.117 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861463+0000", 2024-09-14T06:33:03.117 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.5200000000000002e-07 2024-09-14T06:33:03.117 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:03.117 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:03.117 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:33:03.117 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861465+0000", 2024-09-14T06:33:03.117 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.5189999999999999e-06 2024-09-14T06:33:03.117 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:03.117 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:03.117 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:33:03.117 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861471+0000", 2024-09-14T06:33:03.117 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.6899999999999997e-06 2024-09-14T06:33:03.117 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:03.117 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:03.118 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:33:03.118 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861475+0000", 2024-09-14T06:33:03.118 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.5730000000000001e-06 2024-09-14T06:33:03.118 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:03.118 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:03.118 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:33:03.118 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.925085+0000", 2024-09-14T06:33:03.118 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.063609902999999995 2024-09-14T06:33:03.118 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:03.118 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:03.118 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:33:03.118 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.925145+0000", 2024-09-14T06:33:03.118 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.9383999999999999e-05 2024-09-14T06:33:03.118 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:03.119 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:03.119 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:33:03.119 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.925318+0000", 2024-09-14T06:33:03.119 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00017318499999999999 2024-09-14T06:33:03.119 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:33:03.119 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:33:03.119 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:33:03.119 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:03.119 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:03.119 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:44540 3.2 3:4190d882:::rbd_data.112dca2f6ae1.0000000000000002:17 [read 552960~4096 [fadvise_nocache]] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:33:03.119 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:43.861483+0000", 2024-09-14T06:33:03.119 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 199.173788468, 2024-09-14T06:33:03.119 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.064065681999999999, 2024-09-14T06:33:03.119 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:33:03.119 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:33:03.120 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:33:03.120 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:33:03.120 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:33:03.120 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:33:03.120 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 44540 2024-09-14T06:33:03.120 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:03.120 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:33:03.120 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:03.120 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:33:03.120 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861483+0000", 2024-09-14T06:33:03.120 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:33:03.120 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:03.120 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:03.121 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:33:03.121 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861483+0000", 2024-09-14T06:33:03.121 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:33:03.121 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:03.121 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:03.121 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:33:03.121 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.867079+0000", 2024-09-14T06:33:03.121 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0055966480000000001 2024-09-14T06:33:03.121 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:03.121 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:03.121 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:33:03.121 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.867086+0000", 2024-09-14T06:33:03.121 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.387e-06 2024-09-14T06:33:03.121 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:03.122 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:03.122 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:33:03.122 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.867094+0000", 2024-09-14T06:33:03.122 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.9419999999999995e-06 2024-09-14T06:33:03.122 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:03.122 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:03.122 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:33:03.122 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.867109+0000", 2024-09-14T06:33:03.122 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.5818999999999999e-05 2024-09-14T06:33:03.122 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:03.122 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:03.122 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:33:03.122 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.925354+0000", 2024-09-14T06:33:03.122 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.058244047 2024-09-14T06:33:03.123 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:03.123 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:03.123 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:33:03.123 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.925406+0000", 2024-09-14T06:33:03.123 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.2661999999999998e-05 2024-09-14T06:33:03.123 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:03.123 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:03.123 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:33:03.123 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.925548+0000", 2024-09-14T06:33:03.123 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000142177 2024-09-14T06:33:03.123 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:33:03.123 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:33:03.123 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:33:03.123 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:03.124 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:03.124 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:225587 3.7 3:e112e2b3:::rbd_id.image_client.0-clone42:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e189)", 2024-09-14T06:33:03.124 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:31:53.874591+0000", 2024-09-14T06:33:03.124 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 69.160680251000002, 2024-09-14T06:33:03.124 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.875541507, 2024-09-14T06:33:03.124 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:33:03.124 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:33:03.124 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:33:03.124 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:33:03.124 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:33:03.124 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:33:03.124 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 225587 2024-09-14T06:33:03.124 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:03.124 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:33:03.125 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:03.125 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:33:03.125 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:53.874591+0000", 2024-09-14T06:33:03.125 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:33:03.125 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:03.125 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:03.125 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:33:03.125 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:53.874591+0000", 2024-09-14T06:33:03.125 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:33:03.125 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:03.125 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:03.125 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:33:03.125 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:53.874592+0000", 2024-09-14T06:33:03.125 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.5209999999999999e-06 2024-09-14T06:33:03.125 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:03.126 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:03.126 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:33:03.126 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:53.874599+0000", 2024-09-14T06:33:03.126 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.6290000000000004e-06 2024-09-14T06:33:03.126 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:03.126 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:03.126 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:33:03.126 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:53.874618+0000", 2024-09-14T06:33:03.126 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.8950999999999999e-05 2024-09-14T06:33:03.126 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:03.126 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:03.126 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:33:03.126 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:53.874626+0000", 2024-09-14T06:33:03.126 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.3960000000000006e-06 2024-09-14T06:33:03.127 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:03.127 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:03.127 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:33:03.127 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:53.874699+0000", 2024-09-14T06:33:03.127 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.2329999999999994e-05 2024-09-14T06:33:03.127 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:03.127 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:03.127 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "waiting for peered", 2024-09-14T06:33:03.127 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:53.874711+0000", 2024-09-14T06:33:03.127 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.2473999999999999e-05 2024-09-14T06:33:03.127 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:03.127 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:03.127 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:33:03.127 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:54.749578+0000", 2024-09-14T06:33:03.128 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.87486642100000001 2024-09-14T06:33:03.128 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:03.128 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:03.128 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:33:03.128 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:54.749706+0000", 2024-09-14T06:33:03.128 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00012856200000000001 2024-09-14T06:33:03.128 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:03.128 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:03.128 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:33:03.128 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:54.750132+0000", 2024-09-14T06:33:03.128 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00042622299999999999 2024-09-14T06:33:03.128 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:33:03.128 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:33:03.128 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:33:03.128 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:03.129 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:03.129 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:233119 3.1e 3:7bae5948:::rbd_data.112df8d32353.000000000000000f:head [delete] snapc 53=[] ondisk+write+known_if_redirected+supports_pool_eio e211)", 2024-09-14T06:33:03.129 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:32:16.760588+0000", 2024-09-14T06:33:03.129 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 46.274683263999997, 2024-09-14T06:33:03.129 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.92349815700000004, 2024-09-14T06:33:03.129 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:33:03.129 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:33:03.129 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:33:03.129 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:33:03.129 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:33:03.129 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:33:03.129 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 233119 2024-09-14T06:33:03.129 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:03.129 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:33:03.130 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:03.130 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:33:03.130 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:16.760588+0000", 2024-09-14T06:33:03.130 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:33:03.130 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:03.130 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:03.130 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:33:03.130 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:16.760588+0000", 2024-09-14T06:33:03.130 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:33:03.130 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:03.130 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:03.130 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:33:03.130 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:16.760588+0000", 2024-09-14T06:33:03.130 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.3900000000000005e-07 2024-09-14T06:33:03.130 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:03.131 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:03.131 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:33:03.131 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:16.760591+0000", 2024-09-14T06:33:03.131 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.0070000000000002e-06 2024-09-14T06:33:03.131 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:03.131 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:03.131 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:33:03.131 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:16.760598+0000", 2024-09-14T06:33:03.131 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.0509999999999999e-06 2024-09-14T06:33:03.131 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:03.131 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:03.131 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:33:03.131 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:16.760605+0000", 2024-09-14T06:33:03.131 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.0310000000000004e-06 2024-09-14T06:33:03.131 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:03.132 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:03.132 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:33:03.132 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:16.760668+0000", 2024-09-14T06:33:03.132 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.3699000000000001e-05 2024-09-14T06:33:03.132 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:03.132 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:03.132 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "waiting for peered", 2024-09-14T06:33:03.132 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:16.760678+0000", 2024-09-14T06:33:03.132 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.0122999999999999e-05 2024-09-14T06:33:03.132 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:03.132 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:03.132 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:33:03.132 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:17.681459+0000", 2024-09-14T06:33:03.132 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.92078067399999997 2024-09-14T06:33:03.132 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:03.132 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:03.133 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:33:03.133 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:17.681700+0000", 2024-09-14T06:33:03.133 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00024128800000000001 2024-09-14T06:33:03.133 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:03.133 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:03.133 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "waiting for subops from 2", 2024-09-14T06:33:03.133 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:17.681860+0000", 2024-09-14T06:33:03.133 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00015995500000000001 2024-09-14T06:33:03.133 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:03.133 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:03.133 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:33:03.133 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:17.683399+0000", 2024-09-14T06:33:03.133 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0015388209999999999 2024-09-14T06:33:03.133 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:03.134 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:03.134 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "op_commit", 2024-09-14T06:33:03.134 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:17.683687+0000", 2024-09-14T06:33:03.134 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00028772299999999998 2024-09-14T06:33:03.134 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:03.134 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:03.134 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-14T06:33:03.134 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:17.684032+0000", 2024-09-14T06:33:03.134 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000344836 2024-09-14T06:33:03.134 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:03.134 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:03.134 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:33:03.134 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:17.684086+0000", 2024-09-14T06:33:03.134 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.4410000000000003e-05 2024-09-14T06:33:03.135 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:33:03.135 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:33:03.135 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:33:03.135 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:33:03.135 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:33:03.135 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-14T06:33:03.137 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-14T06:33:03.137 INFO:tasks.ceph.osd.0.smithi094.stderr:2024-09-14T06:33:03.135+0000 7f07c9333640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33636) UID: 0 2024-09-14T06:33:03.237 INFO:tasks.ceph.osd.6.smithi138.stderr:2024-09-14T06:33:03.235+0000 7f8cd8849640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33620) UID: 0 2024-09-14T06:33:03.250 INFO:teuthology.orchestra.run.smithi138.stdout:{ 2024-09-14T06:33:03.250 INFO:teuthology.orchestra.run.smithi138.stdout: "ops": [], 2024-09-14T06:33:03.250 INFO:teuthology.orchestra.run.smithi138.stdout: "num_ops": 0 2024-09-14T06:33:03.250 INFO:teuthology.orchestra.run.smithi138.stdout:} 2024-09-14T06:33:03.259 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-14T06:33:03.339 INFO:tasks.ceph.osd.3.smithi094.stderr:2024-09-14T06:33:03.338+0000 7feeeafe8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33655) UID: 0 2024-09-14T06:33:03.403 INFO:teuthology.orchestra.run.smithi138.stdout:{ 2024-09-14T06:33:03.403 INFO:teuthology.orchestra.run.smithi138.stdout: "ops": [], 2024-09-14T06:33:03.403 INFO:teuthology.orchestra.run.smithi138.stdout: "complaint_time": 30, 2024-09-14T06:33:03.404 INFO:teuthology.orchestra.run.smithi138.stdout: "num_blocked_ops": 0 2024-09-14T06:33:03.404 INFO:teuthology.orchestra.run.smithi138.stdout:} 2024-09-14T06:33:03.412 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-14T06:33:03.439 INFO:tasks.ceph.osd.2.smithi094.stderr:2024-09-14T06:33:03.439+0000 7faf3269b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33649) UID: 0 2024-09-14T06:33:03.516 INFO:teuthology.orchestra.run.smithi138.stdout:{ 2024-09-14T06:33:03.517 INFO:teuthology.orchestra.run.smithi138.stdout: "size": 20, 2024-09-14T06:33:03.517 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 600, 2024-09-14T06:33:03.517 INFO:teuthology.orchestra.run.smithi138.stdout: "ops": [ 2024-09-14T06:33:03.517 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:03.517 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:19328 3.3 3:c34e5e51:::rbd_data.112d92027590.0000000000000024:f [read 1926144~1024 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:33:03.517 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:27.059768+0000", 2024-09-14T06:33:03.517 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 216.45517342900001, 2024-09-14T06:33:03.517 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046369147999999999, 2024-09-14T06:33:03.517 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:33:03.517 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:33:03.517 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:33:03.517 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:33:03.517 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:33:03.518 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:33:03.518 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 19328 2024-09-14T06:33:03.518 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:03.518 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:33:03.518 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:03.518 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:33:03.518 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.059768+0000", 2024-09-14T06:33:03.518 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:33:03.518 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:03.518 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:03.518 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:33:03.518 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.059768+0000", 2024-09-14T06:33:03.518 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:33:03.518 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:03.519 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:03.519 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:33:03.519 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.059874+0000", 2024-09-14T06:33:03.519 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00010551400000000001 2024-09-14T06:33:03.519 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:03.519 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:03.519 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:33:03.519 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.059877+0000", 2024-09-14T06:33:03.519 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.8279999999999999e-06 2024-09-14T06:33:03.519 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:03.519 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:03.519 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:33:03.519 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.059884+0000", 2024-09-14T06:33:03.519 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.6389999999999992e-06 2024-09-14T06:33:03.520 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:03.520 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:03.520 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:33:03.520 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.059890+0000", 2024-09-14T06:33:03.520 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.49e-06 2024-09-14T06:33:03.520 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:03.520 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:03.520 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:33:03.520 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.105804+0000", 2024-09-14T06:33:03.520 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.045914242000000001 2024-09-14T06:33:03.521 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:03.521 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:03.521 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:33:03.521 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.105886+0000", 2024-09-14T06:33:03.521 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.2305999999999994e-05 2024-09-14T06:33:03.521 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:03.521 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:03.521 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:33:03.521 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.106138+0000", 2024-09-14T06:33:03.521 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.000251129 2024-09-14T06:33:03.521 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:33:03.521 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:33:03.521 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:33:03.521 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:03.522 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:03.522 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:19329 3.3 3:c34e5e51:::rbd_data.112d92027590.0000000000000024:f [read 1936384~1024 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:33:03.522 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:27.059899+0000", 2024-09-14T06:33:03.522 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 216.45504290599999, 2024-09-14T06:33:03.522 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046572122000000001, 2024-09-14T06:33:03.522 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:33:03.522 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:33:03.522 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:33:03.522 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:33:03.522 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:33:03.522 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:33:03.522 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 19329 2024-09-14T06:33:03.522 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:03.522 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:33:03.522 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:03.523 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:33:03.523 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.059899+0000", 2024-09-14T06:33:03.523 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:33:03.523 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:03.523 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:03.523 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:33:03.523 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.059899+0000", 2024-09-14T06:33:03.523 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:33:03.523 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:03.523 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:03.523 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:33:03.523 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062501+0000", 2024-09-14T06:33:03.523 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.0026016419999999999 2024-09-14T06:33:03.523 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:03.523 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:03.524 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:33:03.524 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062514+0000", 2024-09-14T06:33:03.524 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.2947e-05 2024-09-14T06:33:03.524 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:03.524 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:03.524 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:33:03.524 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062523+0000", 2024-09-14T06:33:03.524 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.5519999999999993e-06 2024-09-14T06:33:03.524 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:03.524 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:03.524 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:33:03.524 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062534+0000", 2024-09-14T06:33:03.524 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.1416e-05 2024-09-14T06:33:03.524 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:03.525 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:03.525 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:33:03.525 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.106179+0000", 2024-09-14T06:33:03.525 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.043644049999999997 2024-09-14T06:33:03.525 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:03.525 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:03.525 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:33:03.525 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.106262+0000", 2024-09-14T06:33:03.525 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.3178999999999998e-05 2024-09-14T06:33:03.525 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:03.525 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:03.525 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:33:03.525 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.106471+0000", 2024-09-14T06:33:03.525 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00020933599999999999 2024-09-14T06:33:03.526 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:33:03.526 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:33:03.526 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:33:03.526 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:03.526 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:03.526 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:19351 3.3 3:c34e5e51:::rbd_data.112d92027590.0000000000000024:f [read 1159168~1024 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:33:03.526 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:27.062766+0000", 2024-09-14T06:33:03.526 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 216.452175681, 2024-09-14T06:33:03.526 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046099454999999998, 2024-09-14T06:33:03.526 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:33:03.526 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:33:03.526 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:33:03.526 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:33:03.526 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:33:03.526 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:33:03.527 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 19351 2024-09-14T06:33:03.527 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:03.527 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:33:03.527 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:03.527 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:33:03.527 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062766+0000", 2024-09-14T06:33:03.527 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:33:03.527 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:03.527 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:03.527 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:33:03.527 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062766+0000", 2024-09-14T06:33:03.527 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:33:03.527 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:03.527 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:03.528 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:33:03.528 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062766+0000", 2024-09-14T06:33:03.528 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.6e-07 2024-09-14T06:33:03.528 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:03.528 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:03.528 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:33:03.528 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062767+0000", 2024-09-14T06:33:03.528 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.0070000000000001e-06 2024-09-14T06:33:03.528 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:03.528 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:03.528 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:33:03.528 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062785+0000", 2024-09-14T06:33:03.528 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.7884999999999999e-05 2024-09-14T06:33:03.528 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:03.529 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:03.529 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:33:03.529 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062790+0000", 2024-09-14T06:33:03.529 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.0900000000000004e-06 2024-09-14T06:33:03.529 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:03.529 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:03.529 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:33:03.529 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.108735+0000", 2024-09-14T06:33:03.529 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.045944411999999997 2024-09-14T06:33:03.529 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:03.529 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:03.529 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:33:03.529 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.108809+0000", 2024-09-14T06:33:03.529 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.4157000000000002e-05 2024-09-14T06:33:03.529 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:03.530 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:03.530 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:33:03.530 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.108866+0000", 2024-09-14T06:33:03.530 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.6644000000000002e-05 2024-09-14T06:33:03.530 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:33:03.530 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:33:03.530 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:33:03.530 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:03.530 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:03.530 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:19352 3.3 3:c34e5e51:::rbd_data.112d92027590.0000000000000024:f [read 1169408~1024 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:33:03.530 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:27.062798+0000", 2024-09-14T06:33:03.530 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 216.45214426999999, 2024-09-14T06:33:03.530 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046255900000000003, 2024-09-14T06:33:03.530 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:33:03.531 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:33:03.531 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:33:03.531 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:33:03.531 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:33:03.531 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:33:03.531 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 19352 2024-09-14T06:33:03.531 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:03.531 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:33:03.531 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:03.531 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:33:03.531 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062798+0000", 2024-09-14T06:33:03.531 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:33:03.531 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:03.531 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:03.532 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:33:03.532 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062798+0000", 2024-09-14T06:33:03.532 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:33:03.532 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:03.532 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:03.532 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:33:03.532 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062798+0000", 2024-09-14T06:33:03.532 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.37e-07 2024-09-14T06:33:03.532 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:03.532 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:03.532 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:33:03.532 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062800+0000", 2024-09-14T06:33:03.532 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.0269999999999998e-06 2024-09-14T06:33:03.532 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:03.533 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:03.533 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:33:03.533 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062806+0000", 2024-09-14T06:33:03.533 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.6690000000000003e-06 2024-09-14T06:33:03.533 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:03.533 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:03.533 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:33:03.533 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062810+0000", 2024-09-14T06:33:03.533 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.3449999999999999e-06 2024-09-14T06:33:03.533 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:03.533 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:03.533 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:33:03.533 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.108892+0000", 2024-09-14T06:33:03.533 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046081715000000002 2024-09-14T06:33:03.534 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:03.534 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:03.534 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:33:03.534 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.108940+0000", 2024-09-14T06:33:03.534 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.7815999999999998e-05 2024-09-14T06:33:03.534 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:03.534 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:03.534 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:33:03.534 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.109053+0000", 2024-09-14T06:33:03.534 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.000113791 2024-09-14T06:33:03.534 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:33:03.534 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:33:03.534 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:33:03.534 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:03.534 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:03.535 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:19353 3.3 3:c34e5e51:::rbd_data.112d92027590.0000000000000024:f [read 1179648~1024 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:33:03.535 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:27.062816+0000", 2024-09-14T06:33:03.535 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 216.45212575100001, 2024-09-14T06:33:03.535 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046481256999999998, 2024-09-14T06:33:03.535 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:33:03.535 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:33:03.535 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:33:03.535 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:33:03.535 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:33:03.535 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:33:03.535 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 19353 2024-09-14T06:33:03.535 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:03.535 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:33:03.535 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:03.536 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:33:03.536 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062816+0000", 2024-09-14T06:33:03.536 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:33:03.536 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:03.536 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:03.536 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:33:03.536 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062816+0000", 2024-09-14T06:33:03.536 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:33:03.536 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:03.536 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:03.536 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:33:03.536 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062816+0000", 2024-09-14T06:33:03.536 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.2899999999999999e-07 2024-09-14T06:33:03.537 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:03.537 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:03.537 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:33:03.537 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062818+0000", 2024-09-14T06:33:03.537 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.699e-06 2024-09-14T06:33:03.537 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:03.537 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:03.537 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:33:03.537 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062825+0000", 2024-09-14T06:33:03.537 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.878e-06 2024-09-14T06:33:03.537 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:03.537 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:03.537 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:33:03.537 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062830+0000", 2024-09-14T06:33:03.538 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.5410000000000002e-06 2024-09-14T06:33:03.538 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:03.538 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:03.538 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:33:03.538 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.109096+0000", 2024-09-14T06:33:03.538 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046266517 2024-09-14T06:33:03.538 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:03.538 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:03.538 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:33:03.538 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.109190+0000", 2024-09-14T06:33:03.538 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.4000999999999997e-05 2024-09-14T06:33:03.538 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:03.538 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:03.538 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:33:03.539 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.109297+0000", 2024-09-14T06:33:03.539 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.000107292 2024-09-14T06:33:03.539 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:33:03.539 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:33:03.539 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:33:03.539 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:03.539 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:03.539 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:19354 3.3 3:c34e5e51:::rbd_data.112d92027590.0000000000000024:f [read 1189888~1024 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:33:03.539 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:27.062842+0000", 2024-09-14T06:33:03.539 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 216.452100296, 2024-09-14T06:33:03.539 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046594257, 2024-09-14T06:33:03.539 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:33:03.539 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:33:03.540 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:33:03.540 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:33:03.540 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:33:03.540 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:33:03.540 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 19354 2024-09-14T06:33:03.540 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:03.540 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:33:03.540 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:03.540 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:33:03.540 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062842+0000", 2024-09-14T06:33:03.540 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:33:03.540 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:03.540 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:03.540 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:33:03.540 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062842+0000", 2024-09-14T06:33:03.541 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:33:03.541 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:03.541 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:03.541 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:33:03.541 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062842+0000", 2024-09-14T06:33:03.541 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.7399999999999999e-07 2024-09-14T06:33:03.541 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:03.541 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:03.541 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:33:03.542 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062859+0000", 2024-09-14T06:33:03.542 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.7010000000000001e-05 2024-09-14T06:33:03.542 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:03.542 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:03.542 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:33:03.542 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062866+0000", 2024-09-14T06:33:03.542 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.9990000000000002e-06 2024-09-14T06:33:03.542 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:03.542 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:03.542 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:33:03.542 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062873+0000", 2024-09-14T06:33:03.543 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.8569999999999997e-06 2024-09-14T06:33:03.543 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:03.543 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:03.543 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:33:03.543 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.109335+0000", 2024-09-14T06:33:03.543 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046461943999999998 2024-09-14T06:33:03.543 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:03.543 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:03.543 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:33:03.543 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.109377+0000", 2024-09-14T06:33:03.543 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.2558999999999997e-05 2024-09-14T06:33:03.543 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:03.543 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:03.544 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:33:03.544 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.109436+0000", 2024-09-14T06:33:03.544 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.8514000000000003e-05 2024-09-14T06:33:03.544 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:33:03.544 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:33:03.544 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:33:03.544 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:03.544 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:03.544 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:19355 3.3 3:c34e5e51:::rbd_data.112d92027590.0000000000000024:f [read 1200128~1024 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:33:03.544 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:27.062884+0000", 2024-09-14T06:33:03.544 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 216.452057623, 2024-09-14T06:33:03.545 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046681650999999998, 2024-09-14T06:33:03.545 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:33:03.545 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:33:03.545 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:33:03.545 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:33:03.545 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:33:03.545 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:33:03.545 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 19355 2024-09-14T06:33:03.545 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:03.545 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:33:03.545 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:03.546 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:33:03.546 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062884+0000", 2024-09-14T06:33:03.546 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:33:03.546 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:03.546 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:03.546 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:33:03.546 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062884+0000", 2024-09-14T06:33:03.546 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:33:03.546 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:03.546 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:03.546 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:33:03.547 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062885+0000", 2024-09-14T06:33:03.547 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.8000000000000001e-07 2024-09-14T06:33:03.547 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:03.547 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:03.547 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:33:03.547 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062886+0000", 2024-09-14T06:33:03.547 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.8920000000000001e-06 2024-09-14T06:33:03.547 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:03.547 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:03.547 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:33:03.547 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062892+0000", 2024-09-14T06:33:03.547 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.84e-06 2024-09-14T06:33:03.548 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:03.548 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:03.548 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:33:03.548 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062899+0000", 2024-09-14T06:33:03.548 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.5339999999999996e-06 2024-09-14T06:33:03.548 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:03.548 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:03.548 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:33:03.548 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.109466+0000", 2024-09-14T06:33:03.548 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046566919999999998 2024-09-14T06:33:03.548 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:03.548 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:03.549 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:33:03.549 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.109510+0000", 2024-09-14T06:33:03.549 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.4153000000000002e-05 2024-09-14T06:33:03.549 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:03.549 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:03.549 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:33:03.549 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.109566+0000", 2024-09-14T06:33:03.549 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.5931999999999999e-05 2024-09-14T06:33:03.549 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:33:03.549 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:33:03.549 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:33:03.549 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:03.550 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:03.550 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:19356 3.3 3:c34e5e51:::rbd_data.112d92027590.0000000000000024:f [read 1210368~1024 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:33:03.550 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:27.062908+0000", 2024-09-14T06:33:03.550 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 216.45203418599999, 2024-09-14T06:33:03.550 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046807498000000003, 2024-09-14T06:33:03.550 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:33:03.550 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:33:03.550 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:33:03.550 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:33:03.550 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:33:03.550 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:33:03.551 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 19356 2024-09-14T06:33:03.551 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:03.551 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:33:03.551 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:03.551 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:33:03.551 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062908+0000", 2024-09-14T06:33:03.551 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:33:03.551 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:03.551 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:03.551 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:33:03.551 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062908+0000", 2024-09-14T06:33:03.551 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:33:03.552 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:03.552 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:03.552 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:33:03.552 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062908+0000", 2024-09-14T06:33:03.552 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.5699999999999998e-07 2024-09-14T06:33:03.552 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:03.552 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:03.552 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:33:03.552 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062910+0000", 2024-09-14T06:33:03.552 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.7710000000000001e-06 2024-09-14T06:33:03.553 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:03.553 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:03.553 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:33:03.553 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062920+0000", 2024-09-14T06:33:03.553 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.0654e-05 2024-09-14T06:33:03.553 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:03.553 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:03.553 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:33:03.553 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062931+0000", 2024-09-14T06:33:03.553 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.075e-05 2024-09-14T06:33:03.553 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:03.553 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:03.553 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:33:03.554 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.109595+0000", 2024-09-14T06:33:03.554 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046663713000000002 2024-09-14T06:33:03.554 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:03.554 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:03.554 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:33:03.554 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.109632+0000", 2024-09-14T06:33:03.554 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.7505000000000001e-05 2024-09-14T06:33:03.554 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:03.554 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:03.554 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:33:03.554 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.109715+0000", 2024-09-14T06:33:03.554 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.2748000000000001e-05 2024-09-14T06:33:03.555 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:33:03.555 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:33:03.555 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:33:03.555 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:03.555 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:03.555 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:19357 3.3 3:c34e5e51:::rbd_data.112d92027590.0000000000000024:f [read 1220608~1024 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:33:03.555 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:27.062939+0000", 2024-09-14T06:33:03.555 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 216.452002738, 2024-09-14T06:33:03.555 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046920508, 2024-09-14T06:33:03.555 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:33:03.555 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:33:03.555 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:33:03.556 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:33:03.556 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:33:03.556 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:33:03.556 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 19357 2024-09-14T06:33:03.556 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:03.556 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:33:03.556 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:03.556 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:33:03.556 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062939+0000", 2024-09-14T06:33:03.556 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:33:03.556 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:03.556 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:03.557 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:33:03.557 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062939+0000", 2024-09-14T06:33:03.557 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:33:03.557 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:03.557 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:03.557 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:33:03.557 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062940+0000", 2024-09-14T06:33:03.557 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.32e-07 2024-09-14T06:33:03.557 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:03.557 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:03.557 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:33:03.557 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062942+0000", 2024-09-14T06:33:03.557 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.1919999999999999e-06 2024-09-14T06:33:03.558 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:03.558 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:03.558 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:33:03.558 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062948+0000", 2024-09-14T06:33:03.558 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.1149999999999999e-06 2024-09-14T06:33:03.558 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:03.558 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:03.558 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:33:03.558 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062960+0000", 2024-09-14T06:33:03.558 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.1895e-05 2024-09-14T06:33:03.558 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:03.559 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:03.559 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:33:03.559 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.109704+0000", 2024-09-14T06:33:03.559 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046744584999999998 2024-09-14T06:33:03.559 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:03.559 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:03.559 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:33:03.559 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.109777+0000", 2024-09-14T06:33:03.559 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.2282999999999999e-05 2024-09-14T06:33:03.559 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:03.559 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:03.559 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:33:03.559 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.109860+0000", 2024-09-14T06:33:03.560 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.3005999999999998e-05 2024-09-14T06:33:03.560 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:33:03.560 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:33:03.560 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:33:03.560 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:03.560 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:03.560 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:19358 3.3 3:c34e5e51:::rbd_data.112d92027590.0000000000000024:f [read 1230848~1024 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:33:03.560 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:27.062967+0000", 2024-09-14T06:33:03.560 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 216.451975108, 2024-09-14T06:33:03.560 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047144471, 2024-09-14T06:33:03.560 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:33:03.560 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:33:03.561 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:33:03.561 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:33:03.561 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:33:03.561 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:33:03.561 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 19358 2024-09-14T06:33:03.561 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:03.561 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:33:03.561 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:03.561 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:33:03.561 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062967+0000", 2024-09-14T06:33:03.561 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:33:03.561 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:03.562 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:03.562 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:33:03.562 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062967+0000", 2024-09-14T06:33:03.562 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:33:03.562 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:03.562 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:03.562 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:33:03.562 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062967+0000", 2024-09-14T06:33:03.562 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.89e-07 2024-09-14T06:33:03.562 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:03.562 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:03.562 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:33:03.563 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062969+0000", 2024-09-14T06:33:03.563 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.9929999999999998e-06 2024-09-14T06:33:03.563 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:03.563 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:03.563 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:33:03.563 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062975+0000", 2024-09-14T06:33:03.563 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.9030000000000001e-06 2024-09-14T06:33:03.563 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:03.563 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:03.563 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:33:03.563 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062979+0000", 2024-09-14T06:33:03.563 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.2250000000000002e-06 2024-09-14T06:33:03.564 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:03.564 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:03.564 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:33:03.564 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.109934+0000", 2024-09-14T06:33:03.564 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046954410000000002 2024-09-14T06:33:03.564 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:03.564 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:03.564 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:33:03.564 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.110007+0000", 2024-09-14T06:33:03.564 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.3829000000000001e-05 2024-09-14T06:33:03.564 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:03.565 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:03.565 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:33:03.565 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.110111+0000", 2024-09-14T06:33:03.565 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00010372199999999999 2024-09-14T06:33:03.565 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:33:03.565 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:33:03.565 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:33:03.565 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:03.565 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:03.565 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:19359 3.3 3:c34e5e51:::rbd_data.112d92027590.0000000000000024:f [read 1241088~1024 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:33:03.565 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:27.062987+0000", 2024-09-14T06:33:03.565 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 216.45195501800001, 2024-09-14T06:33:03.566 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047297182, 2024-09-14T06:33:03.566 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:33:03.566 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:33:03.566 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:33:03.566 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:33:03.566 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:33:03.566 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:33:03.566 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 19359 2024-09-14T06:33:03.566 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:03.566 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:33:03.566 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:03.566 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:33:03.567 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062987+0000", 2024-09-14T06:33:03.567 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:33:03.567 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:03.567 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:03.567 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:33:03.567 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062987+0000", 2024-09-14T06:33:03.567 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:33:03.567 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:03.567 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:03.567 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:33:03.567 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062987+0000", 2024-09-14T06:33:03.567 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.2500000000000001e-07 2024-09-14T06:33:03.568 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:03.568 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:03.568 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:33:03.568 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062989+0000", 2024-09-14T06:33:03.568 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.6080000000000001e-06 2024-09-14T06:33:03.568 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:03.568 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:03.568 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:33:03.568 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062994+0000", 2024-09-14T06:33:03.568 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.5520000000000004e-06 2024-09-14T06:33:03.568 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:03.569 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:03.569 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:33:03.569 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062999+0000", 2024-09-14T06:33:03.569 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.3930000000000001e-06 2024-09-14T06:33:03.569 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:03.569 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:03.569 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:33:03.569 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.110142+0000", 2024-09-14T06:33:03.569 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047143154 2024-09-14T06:33:03.569 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:03.569 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:03.570 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:33:03.570 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.110219+0000", 2024-09-14T06:33:03.570 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.6737999999999996e-05 2024-09-14T06:33:03.570 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:03.570 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:03.570 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:33:03.570 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.110284+0000", 2024-09-14T06:33:03.570 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.5412000000000003e-05 2024-09-14T06:33:03.570 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:33:03.570 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:33:03.570 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:33:03.570 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:03.571 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:03.571 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:19360 3.3 3:c34e5e51:::rbd_data.112d92027590.0000000000000024:f [read 1251328~1024 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:33:03.571 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:27.063017+0000", 2024-09-14T06:33:03.571 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 216.45192455700001, 2024-09-14T06:33:03.571 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047453823999999999, 2024-09-14T06:33:03.571 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:33:03.571 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:33:03.571 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:33:03.571 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:33:03.571 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:33:03.571 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:33:03.571 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 19360 2024-09-14T06:33:03.572 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:03.572 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:33:03.572 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:03.572 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:33:03.572 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.063017+0000", 2024-09-14T06:33:03.572 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:33:03.572 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:03.572 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:03.572 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:33:03.572 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.063017+0000", 2024-09-14T06:33:03.572 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:33:03.572 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:03.573 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:03.573 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:33:03.573 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.064977+0000", 2024-09-14T06:33:03.573 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.0019597080000000001 2024-09-14T06:33:03.573 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:03.573 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:03.573 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:33:03.573 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.064987+0000", 2024-09-14T06:33:03.573 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.9620000000000005e-06 2024-09-14T06:33:03.573 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:03.573 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:03.573 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:33:03.574 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065000+0000", 2024-09-14T06:33:03.574 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.3173000000000001e-05 2024-09-14T06:33:03.574 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:03.574 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:03.574 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:33:03.574 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065005+0000", 2024-09-14T06:33:03.574 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.9960000000000001e-06 2024-09-14T06:33:03.574 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:03.574 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:03.574 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:33:03.574 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.110314+0000", 2024-09-14T06:33:03.574 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.04530899 2024-09-14T06:33:03.575 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:03.575 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:03.575 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:33:03.575 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.110398+0000", 2024-09-14T06:33:03.575 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.3539000000000001e-05 2024-09-14T06:33:03.575 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:03.575 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:03.575 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:33:03.575 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.110471+0000", 2024-09-14T06:33:03.575 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.3455999999999996e-05 2024-09-14T06:33:03.575 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:33:03.575 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:33:03.576 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:33:03.576 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:03.576 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:03.576 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:19365 3.3 3:c34e5e51:::rbd_data.112d92027590.0000000000000024:f [read 1302528~1024 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:33:03.576 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:27.065058+0000", 2024-09-14T06:33:03.576 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 216.449883887, 2024-09-14T06:33:03.576 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046105526000000001, 2024-09-14T06:33:03.576 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:33:03.576 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:33:03.576 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:33:03.576 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:33:03.576 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:33:03.577 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:33:03.577 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 19365 2024-09-14T06:33:03.577 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:03.577 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:33:03.577 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:03.577 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:33:03.577 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065058+0000", 2024-09-14T06:33:03.577 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:33:03.577 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:03.577 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:03.577 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:33:03.577 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065058+0000", 2024-09-14T06:33:03.578 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:33:03.578 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:03.578 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:03.578 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:33:03.578 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065058+0000", 2024-09-14T06:33:03.578 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.1899999999999999e-07 2024-09-14T06:33:03.578 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:03.578 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:03.578 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:33:03.578 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065059+0000", 2024-09-14T06:33:03.578 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.1299999999999998e-07 2024-09-14T06:33:03.578 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:03.579 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:03.579 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:33:03.579 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065063+0000", 2024-09-14T06:33:03.579 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.1099999999999996e-06 2024-09-14T06:33:03.579 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:03.579 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:03.579 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:33:03.579 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065070+0000", 2024-09-14T06:33:03.579 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.973e-06 2024-09-14T06:33:03.579 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:03.579 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:03.579 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:33:03.580 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.111014+0000", 2024-09-14T06:33:03.580 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.045943373000000003 2024-09-14T06:33:03.580 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:03.580 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:03.580 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:33:03.580 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.111073+0000", 2024-09-14T06:33:03.580 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.8987999999999998e-05 2024-09-14T06:33:03.580 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:03.580 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:03.580 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:33:03.580 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.111163+0000", 2024-09-14T06:33:03.580 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.0950000000000002e-05 2024-09-14T06:33:03.581 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:33:03.581 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:33:03.581 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:33:03.581 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:03.581 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:03.581 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:19366 3.3 3:c34e5e51:::rbd_data.112d92027590.0000000000000024:f [read 1312768~1024 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:33:03.581 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:27.065074+0000", 2024-09-14T06:33:03.581 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 216.44986803699999, 2024-09-14T06:33:03.581 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046289858000000003, 2024-09-14T06:33:03.581 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:33:03.581 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:33:03.581 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:33:03.582 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:33:03.582 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:33:03.582 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:33:03.582 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 19366 2024-09-14T06:33:03.582 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:03.582 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:33:03.582 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:03.582 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:33:03.582 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065074+0000", 2024-09-14T06:33:03.582 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:33:03.582 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:03.582 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:03.583 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:33:03.583 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065074+0000", 2024-09-14T06:33:03.583 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:33:03.583 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:03.583 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:03.583 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:33:03.583 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065074+0000", 2024-09-14T06:33:03.583 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.3900000000000001e-07 2024-09-14T06:33:03.583 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:03.583 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:03.583 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:33:03.583 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065078+0000", 2024-09-14T06:33:03.583 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.3229999999999999e-06 2024-09-14T06:33:03.584 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:03.584 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:03.584 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:33:03.584 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065082+0000", 2024-09-14T06:33:03.584 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.8299999999999998e-06 2024-09-14T06:33:03.584 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:03.584 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:03.584 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:33:03.584 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065085+0000", 2024-09-14T06:33:03.584 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.8660000000000002e-06 2024-09-14T06:33:03.584 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:03.584 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:03.585 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:33:03.585 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.111270+0000", 2024-09-14T06:33:03.585 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046185035999999999 2024-09-14T06:33:03.585 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:03.585 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:03.585 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:33:03.585 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.111311+0000", 2024-09-14T06:33:03.585 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.0935e-05 2024-09-14T06:33:03.585 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:03.585 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:03.585 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:33:03.586 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.111364+0000", 2024-09-14T06:33:03.586 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.2629e-05 2024-09-14T06:33:03.586 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:33:03.586 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:33:03.586 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:33:03.586 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:03.586 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:03.586 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:19367 3.3 3:c34e5e51:::rbd_data.112d92027590.0000000000000024:f [read 1323008~1024 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:33:03.586 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:27.065115+0000", 2024-09-14T06:33:03.586 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 216.449826476, 2024-09-14T06:33:03.586 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046407003000000002, 2024-09-14T06:33:03.586 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:33:03.586 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:33:03.587 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:33:03.587 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:33:03.587 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:33:03.587 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:33:03.587 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 19367 2024-09-14T06:33:03.587 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:03.587 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:33:03.587 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:03.587 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:33:03.587 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065115+0000", 2024-09-14T06:33:03.587 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:33:03.587 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:03.588 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:03.588 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:33:03.588 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065115+0000", 2024-09-14T06:33:03.588 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:33:03.588 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:03.588 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:03.588 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:33:03.588 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065116+0000", 2024-09-14T06:33:03.588 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.4999999999999998e-07 2024-09-14T06:33:03.588 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:03.588 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:03.588 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:33:03.589 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065117+0000", 2024-09-14T06:33:03.589 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.189e-06 2024-09-14T06:33:03.589 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:03.589 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:03.589 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:33:03.589 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065122+0000", 2024-09-14T06:33:03.589 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.3709999999999999e-06 2024-09-14T06:33:03.589 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:03.589 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:03.589 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:33:03.589 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065125+0000", 2024-09-14T06:33:03.590 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.8940000000000002e-06 2024-09-14T06:33:03.590 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:03.590 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:03.590 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:33:03.590 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.111402+0000", 2024-09-14T06:33:03.590 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046276645999999998 2024-09-14T06:33:03.590 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:03.590 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:03.590 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:33:03.590 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.111445+0000", 2024-09-14T06:33:03.590 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.3288000000000003e-05 2024-09-14T06:33:03.590 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:03.591 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:03.591 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:33:03.591 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.111522+0000", 2024-09-14T06:33:03.591 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.7164999999999997e-05 2024-09-14T06:33:03.591 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:33:03.591 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:33:03.591 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:33:03.591 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:03.591 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:03.591 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:19368 3.3 3:c34e5e51:::rbd_data.112d92027590.0000000000000024:f [read 1333248~1024 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:33:03.591 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:27.065130+0000", 2024-09-14T06:33:03.591 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 216.449812186, 2024-09-14T06:33:03.592 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046507275000000001, 2024-09-14T06:33:03.592 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:33:03.592 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:33:03.592 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:33:03.592 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:33:03.592 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:33:03.592 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:33:03.592 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 19368 2024-09-14T06:33:03.592 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:03.592 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:33:03.592 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:03.592 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:33:03.593 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065130+0000", 2024-09-14T06:33:03.593 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:33:03.593 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:03.593 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:03.593 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:33:03.593 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065130+0000", 2024-09-14T06:33:03.593 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:33:03.593 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:03.593 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:03.593 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:33:03.593 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065130+0000", 2024-09-14T06:33:03.593 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.0800000000000001e-07 2024-09-14T06:33:03.594 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:03.594 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:03.594 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:33:03.594 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065131+0000", 2024-09-14T06:33:03.594 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.8700000000000004e-07 2024-09-14T06:33:03.594 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:03.594 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:03.594 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:33:03.594 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065134+0000", 2024-09-14T06:33:03.594 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.1039999999999998e-06 2024-09-14T06:33:03.594 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:03.594 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:03.595 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:33:03.595 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065136+0000", 2024-09-14T06:33:03.595 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.3949999999999999e-06 2024-09-14T06:33:03.595 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:03.595 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:03.595 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:33:03.595 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.111541+0000", 2024-09-14T06:33:03.595 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046404206000000003 2024-09-14T06:33:03.595 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:03.595 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:03.595 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:33:03.595 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.111582+0000", 2024-09-14T06:33:03.596 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.1051999999999999e-05 2024-09-14T06:33:03.596 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:03.596 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:03.596 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:33:03.596 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.111637+0000", 2024-09-14T06:33:03.596 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.5322999999999999e-05 2024-09-14T06:33:03.596 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:33:03.596 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:33:03.596 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:33:03.596 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:03.596 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:03.596 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:19369 3.3 3:c34e5e51:::rbd_data.112d92027590.0000000000000024:f [read 1343488~1024 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:33:03.597 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:27.065139+0000", 2024-09-14T06:33:03.597 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 216.449802369, 2024-09-14T06:33:03.597 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046682169000000003, 2024-09-14T06:33:03.597 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:33:03.597 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:33:03.597 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:33:03.597 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:33:03.597 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:33:03.597 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:33:03.597 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 19369 2024-09-14T06:33:03.597 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:03.597 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:33:03.598 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:03.598 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:33:03.598 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065139+0000", 2024-09-14T06:33:03.598 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:33:03.598 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:03.598 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:03.598 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:33:03.598 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065139+0000", 2024-09-14T06:33:03.598 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:33:03.598 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:03.598 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:03.598 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:33:03.599 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065140+0000", 2024-09-14T06:33:03.599 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.8299999999999998e-07 2024-09-14T06:33:03.599 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:03.599 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:03.599 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:33:03.599 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065141+0000", 2024-09-14T06:33:03.599 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.3900000000000003e-07 2024-09-14T06:33:03.599 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:03.599 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:03.599 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:33:03.599 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065144+0000", 2024-09-14T06:33:03.599 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.7409999999999998e-06 2024-09-14T06:33:03.600 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:03.600 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:03.600 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:33:03.600 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065147+0000", 2024-09-14T06:33:03.600 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.4150000000000002e-06 2024-09-14T06:33:03.600 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:03.600 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:03.600 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:33:03.600 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.111673+0000", 2024-09-14T06:33:03.600 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046526004000000003 2024-09-14T06:33:03.600 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:03.601 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:03.601 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:33:03.601 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.111749+0000", 2024-09-14T06:33:03.601 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.5741999999999994e-05 2024-09-14T06:33:03.601 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:03.601 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:03.601 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:33:03.601 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.111822+0000", 2024-09-14T06:33:03.601 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.3045000000000003e-05 2024-09-14T06:33:03.601 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:33:03.601 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:33:03.601 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:33:03.602 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:03.602 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:03.602 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:19370 3.3 3:c34e5e51:::rbd_data.112d92027590.0000000000000024:f [read 1353728~1024 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:33:03.602 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:27.065150+0000", 2024-09-14T06:33:03.602 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 216.44979164, 2024-09-14T06:33:03.602 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046897556999999999, 2024-09-14T06:33:03.602 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:33:03.602 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:33:03.602 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:33:03.602 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:33:03.602 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:33:03.602 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:33:03.602 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 19370 2024-09-14T06:33:03.603 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:03.603 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:33:03.603 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:03.603 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:33:03.603 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065150+0000", 2024-09-14T06:33:03.603 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:33:03.603 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:03.603 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:03.603 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:33:03.603 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065150+0000", 2024-09-14T06:33:03.603 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:33:03.603 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:03.604 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:03.604 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:33:03.604 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065150+0000", 2024-09-14T06:33:03.604 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.0200000000000001e-07 2024-09-14T06:33:03.604 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:03.604 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:03.604 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:33:03.604 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065151+0000", 2024-09-14T06:33:03.604 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.1100000000000004e-07 2024-09-14T06:33:03.604 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:03.604 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:03.604 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:33:03.605 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065154+0000", 2024-09-14T06:33:03.605 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.9639999999999999e-06 2024-09-14T06:33:03.605 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:03.605 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:03.605 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:33:03.605 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065157+0000", 2024-09-14T06:33:03.605 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.4260000000000002e-06 2024-09-14T06:33:03.605 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:03.605 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:03.605 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:33:03.605 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.111861+0000", 2024-09-14T06:33:03.605 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046704268 2024-09-14T06:33:03.606 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:03.606 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:03.606 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:33:03.606 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.111939+0000", 2024-09-14T06:33:03.606 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.8231000000000004e-05 2024-09-14T06:33:03.606 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:03.606 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:03.606 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:33:03.606 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.112048+0000", 2024-09-14T06:33:03.606 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00010855500000000001 2024-09-14T06:33:03.606 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:33:03.606 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:33:03.607 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:33:03.607 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:03.607 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:03.607 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:19371 3.3 3:c34e5e51:::rbd_data.112d92027590.0000000000000024:f [read 1363968~1024 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:33:03.607 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:27.065161+0000", 2024-09-14T06:33:03.607 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 216.44978098999999, 2024-09-14T06:33:03.607 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047021944000000003, 2024-09-14T06:33:03.607 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:33:03.607 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:33:03.607 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:33:03.607 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:33:03.608 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:33:03.608 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:33:03.608 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 19371 2024-09-14T06:33:03.608 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:03.608 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:33:03.608 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:03.608 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:33:03.608 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065161+0000", 2024-09-14T06:33:03.608 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:33:03.608 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:03.608 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:03.608 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:33:03.609 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065161+0000", 2024-09-14T06:33:03.609 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:33:03.609 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:03.609 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:03.609 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:33:03.609 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065161+0000", 2024-09-14T06:33:03.609 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.0200000000000001e-07 2024-09-14T06:33:03.609 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:03.609 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:03.609 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:33:03.609 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065166+0000", 2024-09-14T06:33:03.609 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.9869999999999999e-06 2024-09-14T06:33:03.610 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:03.610 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:03.610 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:33:03.610 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065169+0000", 2024-09-14T06:33:03.610 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.3799999999999998e-06 2024-09-14T06:33:03.610 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:03.610 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:03.610 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:33:03.610 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065172+0000", 2024-09-14T06:33:03.610 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.4830000000000002e-06 2024-09-14T06:33:03.610 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:03.610 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:03.611 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:33:03.611 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.112071+0000", 2024-09-14T06:33:03.611 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046899529000000002 2024-09-14T06:33:03.611 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:03.611 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:03.611 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:33:03.611 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.112128+0000", 2024-09-14T06:33:03.611 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.6341999999999997e-05 2024-09-14T06:33:03.611 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:03.611 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:03.611 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:33:03.612 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.112183+0000", 2024-09-14T06:33:03.612 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.5021000000000001e-05 2024-09-14T06:33:03.612 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:33:03.612 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:33:03.612 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:33:03.612 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:03.612 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:03.612 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:19372 3.3 3:c34e5e51:::rbd_data.112d92027590.0000000000000024:f [read 1374208~1024 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:33:03.612 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:27.065176+0000", 2024-09-14T06:33:03.612 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 216.44976584, 2024-09-14T06:33:03.612 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047188889999999997, 2024-09-14T06:33:03.612 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:33:03.613 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:33:03.613 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:33:03.613 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:33:03.613 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:33:03.613 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:33:03.613 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 19372 2024-09-14T06:33:03.613 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:03.613 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:33:03.613 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:03.613 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:33:03.613 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065176+0000", 2024-09-14T06:33:03.613 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:33:03.613 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:03.614 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:03.614 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:33:03.614 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065176+0000", 2024-09-14T06:33:03.614 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:33:03.614 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:03.614 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:03.614 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:33:03.614 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.071117+0000", 2024-09-14T06:33:03.614 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.0059404990000000001 2024-09-14T06:33:03.614 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:03.614 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:03.614 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:33:03.615 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.071120+0000", 2024-09-14T06:33:03.615 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.4340000000000001e-06 2024-09-14T06:33:03.615 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:03.615 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:03.615 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:33:03.615 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.071129+0000", 2024-09-14T06:33:03.615 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.8389999999999996e-06 2024-09-14T06:33:03.615 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:03.615 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:03.615 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:33:03.615 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.071134+0000", 2024-09-14T06:33:03.615 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.412e-06 2024-09-14T06:33:03.616 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:03.616 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:03.616 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:33:03.616 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.112221+0000", 2024-09-14T06:33:03.616 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.041086473999999998 2024-09-14T06:33:03.616 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:03.616 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:03.616 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:33:03.616 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.112299+0000", 2024-09-14T06:33:03.616 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.8746999999999997e-05 2024-09-14T06:33:03.616 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:03.616 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:03.617 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:33:03.617 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.112365+0000", 2024-09-14T06:33:03.617 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.5485000000000001e-05 2024-09-14T06:33:03.617 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:33:03.617 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:33:03.617 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:33:03.617 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:33:03.617 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:33:03.617 INFO:teuthology.orchestra.run.smithi138.stdout:} 2024-09-14T06:33:03.617 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-14T06:33:03.620 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-14T06:33:03.620 INFO:tasks.ceph.osd.6.smithi138.stderr:2024-09-14T06:33:03.618+0000 7f8cd8849640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33620) UID: 0 2024-09-14T06:33:03.624 INFO:teuthology.orchestra.run.smithi094.stderr:nodeep-scrub is set 2024-09-14T06:33:03.720 INFO:tasks.ceph.osd.4.smithi138.stderr:2024-09-14T06:33:03.719+0000 7f6c653c8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33615) UID: 0 2024-09-14T06:33:03.750 INFO:teuthology.orchestra.run.smithi094.stdout:osd.0: {} 2024-09-14T06:33:03.751 INFO:teuthology.orchestra.run.smithi094.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-14T06:33:03.767 INFO:teuthology.orchestra.run.smithi094.stdout:osd.1: {} 2024-09-14T06:33:03.767 INFO:teuthology.orchestra.run.smithi094.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-14T06:33:03.768 INFO:teuthology.orchestra.run.smithi138.stdout:{ 2024-09-14T06:33:03.768 INFO:teuthology.orchestra.run.smithi138.stdout: "ops": [], 2024-09-14T06:33:03.768 INFO:teuthology.orchestra.run.smithi138.stdout: "num_ops": 0 2024-09-14T06:33:03.768 INFO:teuthology.orchestra.run.smithi138.stdout:} 2024-09-14T06:33:03.777 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-14T06:33:03.784 INFO:teuthology.orchestra.run.smithi094.stdout:osd.2: {} 2024-09-14T06:33:03.784 INFO:teuthology.orchestra.run.smithi094.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-14T06:33:03.799 INFO:teuthology.orchestra.run.smithi094.stdout:osd.3: {} 2024-09-14T06:33:03.799 INFO:teuthology.orchestra.run.smithi094.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-14T06:33:03.816 INFO:teuthology.orchestra.run.smithi094.stdout:osd.4: {} 2024-09-14T06:33:03.816 INFO:teuthology.orchestra.run.smithi094.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-14T06:33:03.833 INFO:teuthology.orchestra.run.smithi094.stdout:osd.5: {} 2024-09-14T06:33:03.833 INFO:teuthology.orchestra.run.smithi094.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-14T06:33:03.849 INFO:teuthology.orchestra.run.smithi094.stdout:osd.6: {} 2024-09-14T06:33:03.849 INFO:teuthology.orchestra.run.smithi094.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-14T06:33:03.861 INFO:tasks.ceph.osd.4.smithi138.stderr:2024-09-14T06:33:03.860+0000 7f6c653c8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33615) UID: 0 2024-09-14T06:33:03.865 INFO:teuthology.orchestra.run.smithi094.stdout:osd.7: {} 2024-09-14T06:33:03.866 INFO:teuthology.orchestra.run.smithi094.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-14T06:33:03.920 INFO:teuthology.orchestra.run.smithi138.stdout:ERROR: (22) Invalid argument 2024-09-14T06:33:03.920 INFO:teuthology.orchestra.run.smithi138.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:33:03.921 INFO:tasks.ceph.osd.6.smithi138.stderr:2024-09-14T06:33:03.919+0000 7f8cd8849640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33620) UID: 0 2024-09-14T06:33:03.929 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-14T06:33:04.021 INFO:tasks.ceph.osd.5.smithi138.stderr:2024-09-14T06:33:04.020+0000 7fe13746f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33618) UID: 0 2024-09-14T06:33:04.072 INFO:teuthology.orchestra.run.smithi138.stdout:ERROR: (22) Invalid argument 2024-09-14T06:33:04.073 INFO:teuthology.orchestra.run.smithi138.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:33:04.081 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-14T06:33:04.123 INFO:tasks.ceph.osd.1.smithi094.stderr:2024-09-14T06:33:04.122+0000 7f328e47a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33641) UID: 0 2024-09-14T06:33:04.223 INFO:tasks.ceph.osd.1.smithi094.stderr:2024-09-14T06:33:04.222+0000 7f328e47a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33641) UID: 0 2024-09-14T06:33:04.224 INFO:teuthology.orchestra.run.smithi138.stdout:ERROR: (22) Invalid argument 2024-09-14T06:33:04.224 INFO:teuthology.orchestra.run.smithi138.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:33:04.232 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-14T06:33:04.324 INFO:tasks.ceph.osd.7.smithi138.stderr:2024-09-14T06:33:04.323+0000 7f3c6a21d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33623) UID: 0 2024-09-14T06:33:04.335 INFO:teuthology.orchestra.run.smithi138.stdout:ERROR: (22) Invalid argument 2024-09-14T06:33:04.335 INFO:teuthology.orchestra.run.smithi138.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:33:04.344 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-14T06:33:04.428 INFO:tasks.ceph.osd.5.smithi138.stderr:2024-09-14T06:33:04.427+0000 7fe13746f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33618) UID: 0 2024-09-14T06:33:04.487 INFO:teuthology.orchestra.run.smithi138.stdout:ERROR: (22) Invalid argument 2024-09-14T06:33:04.487 INFO:teuthology.orchestra.run.smithi138.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:33:04.495 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-14T06:33:04.536 INFO:tasks.ceph.osd.4.smithi138.stderr:2024-09-14T06:33:04.535+0000 7f6c653c8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33615) UID: 0 2024-09-14T06:33:04.627 INFO:tasks.ceph.osd.7.smithi138.stderr:2024-09-14T06:33:04.626+0000 7f3c6a21d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33623) UID: 0 2024-09-14T06:33:04.644 INFO:teuthology.orchestra.run.smithi138.stdout:ERROR: (22) Invalid argument 2024-09-14T06:33:04.645 INFO:teuthology.orchestra.run.smithi138.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:33:04.654 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-14T06:33:04.738 INFO:tasks.ceph.osd.5.smithi138.stderr:2024-09-14T06:33:04.737+0000 7fe13746f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33618) UID: 0 2024-09-14T06:33:04.796 INFO:teuthology.orchestra.run.smithi138.stdout:ERROR: (22) Invalid argument 2024-09-14T06:33:04.796 INFO:teuthology.orchestra.run.smithi138.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:33:04.804 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-14T06:33:04.845 INFO:tasks.ceph.osd.7.smithi138.stderr:2024-09-14T06:33:04.844+0000 7f3c6a21d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33623) UID: 0 2024-09-14T06:33:04.930 INFO:tasks.ceph.osd.5.smithi138.stderr:2024-09-14T06:33:04.929+0000 7fe13746f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33618) UID: 0 2024-09-14T06:33:04.948 INFO:teuthology.orchestra.run.smithi138.stdout:ERROR: (22) Invalid argument 2024-09-14T06:33:04.948 INFO:teuthology.orchestra.run.smithi138.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:33:04.957 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-14T06:33:05.031 INFO:tasks.ceph.osd.0.smithi094.stderr:2024-09-14T06:33:05.030+0000 7f07c9333640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33636) UID: 0 2024-09-14T06:33:05.064 INFO:teuthology.orchestra.run.smithi094.stdout:ERROR: (22) Invalid argument 2024-09-14T06:33:05.064 INFO:teuthology.orchestra.run.smithi094.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:33:05.073 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-14T06:33:05.158 INFO:tasks.ceph.osd.3.smithi094.stderr:2024-09-14T06:33:05.157+0000 7feeeafe8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33655) UID: 0 2024-09-14T06:33:05.217 INFO:teuthology.orchestra.run.smithi094.stdout:ERROR: (22) Invalid argument 2024-09-14T06:33:05.217 INFO:teuthology.orchestra.run.smithi094.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:33:05.225 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-14T06:33:05.266 INFO:tasks.ceph.osd.1.smithi094.stderr:2024-09-14T06:33:05.265+0000 7f328e47a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33641) UID: 0 2024-09-14T06:33:05.333 INFO:tasks.ceph.osd.1.smithi094.stderr:2024-09-14T06:33:05.332+0000 7f328e47a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33641) UID: 0 2024-09-14T06:33:05.369 INFO:teuthology.orchestra.run.smithi094.stdout:ERROR: (22) Invalid argument 2024-09-14T06:33:05.369 INFO:teuthology.orchestra.run.smithi094.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:33:05.377 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-14T06:33:05.462 INFO:tasks.ceph.osd.2.smithi094.stderr:2024-09-14T06:33:05.461+0000 7faf3269b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33649) UID: 0 2024-09-14T06:33:05.519 INFO:teuthology.orchestra.run.smithi094.stdout:ERROR: (22) Invalid argument 2024-09-14T06:33:05.519 INFO:teuthology.orchestra.run.smithi094.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:33:05.528 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-14T06:33:05.534 INFO:tasks.ceph.osd.4.smithi138.stderr:2024-09-14T06:33:05.533+0000 7f6c653c8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33615) UID: 0 2024-09-14T06:33:05.635 INFO:tasks.ceph.osd.1.smithi094.stderr:2024-09-14T06:33:05.634+0000 7f328e47a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33641) UID: 0 2024-09-14T06:33:05.639 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 4, 5, 7, 3] out_osds: [6, 2] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-14T06:33:05.639 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 0 1 2024-09-14T06:33:05.644 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-14T06:33:05.686 INFO:teuthology.orchestra.run.smithi094.stdout:ERROR: (22) Invalid argument 2024-09-14T06:33:05.686 INFO:teuthology.orchestra.run.smithi094.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:33:05.702 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-14T06:33:05.743 INFO:tasks.ceph.osd.0.smithi094.stderr:2024-09-14T06:33:05.742+0000 7f07c9333640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33636) UID: 0 2024-09-14T06:33:05.836 INFO:tasks.ceph.osd.2.smithi094.stderr:2024-09-14T06:33:05.836+0000 7faf3269b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33649) UID: 0 2024-09-14T06:33:05.851 INFO:teuthology.orchestra.run.smithi094.stdout:ERROR: (22) Invalid argument 2024-09-14T06:33:05.851 INFO:teuthology.orchestra.run.smithi094.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:33:05.860 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-14T06:33:05.877 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-14T06:33:05.945 INFO:tasks.ceph.osd.0.smithi094.stderr:2024-09-14T06:33:05.944+0000 7f07c9333640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33636) UID: 0 2024-09-14T06:33:06.008 INFO:teuthology.orchestra.run.smithi094.stdout:ERROR: (22) Invalid argument 2024-09-14T06:33:06.009 INFO:teuthology.orchestra.run.smithi094.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-14T06:33:06.017 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-14T06:33:06.023 INFO:teuthology.orchestra.run.smithi094.stdout:osd.0: {} 2024-09-14T06:33:06.024 INFO:teuthology.orchestra.run.smithi094.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-14T06:33:06.041 INFO:teuthology.orchestra.run.smithi094.stdout:osd.1: {} 2024-09-14T06:33:06.041 INFO:teuthology.orchestra.run.smithi094.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-14T06:33:06.041 INFO:tasks.ceph.osd.3.smithi094.stderr:2024-09-14T06:33:06.040+0000 7feeeafe8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33655) UID: 0 2024-09-14T06:33:06.057 INFO:teuthology.orchestra.run.smithi094.stdout:osd.2: {} 2024-09-14T06:33:06.057 INFO:teuthology.orchestra.run.smithi094.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-14T06:33:06.073 INFO:teuthology.orchestra.run.smithi094.stdout:osd.3: {} 2024-09-14T06:33:06.074 INFO:teuthology.orchestra.run.smithi094.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-14T06:33:06.091 INFO:teuthology.orchestra.run.smithi094.stdout:osd.4: {} 2024-09-14T06:33:06.091 INFO:teuthology.orchestra.run.smithi094.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-14T06:33:06.107 INFO:teuthology.orchestra.run.smithi094.stdout:osd.5: {} 2024-09-14T06:33:06.108 INFO:teuthology.orchestra.run.smithi094.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-14T06:33:06.126 INFO:teuthology.orchestra.run.smithi094.stdout:osd.6: {} 2024-09-14T06:33:06.126 INFO:teuthology.orchestra.run.smithi094.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-14T06:33:06.128 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-14T06:33:06.128 INFO:teuthology.orchestra.run.smithi094.stdout: "ops": [], 2024-09-14T06:33:06.129 INFO:teuthology.orchestra.run.smithi094.stdout: "complaint_time": 30, 2024-09-14T06:33:06.129 INFO:teuthology.orchestra.run.smithi094.stdout: "num_blocked_ops": 0 2024-09-14T06:33:06.129 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-14T06:33:06.137 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-14T06:33:06.140 INFO:tasks.ceph.osd.5.smithi138.stderr:2024-09-14T06:33:06.139+0000 7fe13746f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33618) UID: 0 2024-09-14T06:33:06.141 INFO:teuthology.orchestra.run.smithi094.stdout:osd.7: {} 2024-09-14T06:33:06.142 INFO:teuthology.orchestra.run.smithi094.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-14T06:33:06.241 INFO:tasks.ceph.osd.5.smithi138.stderr:2024-09-14T06:33:06.240+0000 7fe13746f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33618) UID: 0 2024-09-14T06:33:06.246 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-14T06:33:06.246 INFO:teuthology.orchestra.run.smithi094.stdout: "size": 20, 2024-09-14T06:33:06.246 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 600, 2024-09-14T06:33:06.246 INFO:teuthology.orchestra.run.smithi094.stdout: "ops": [ 2024-09-14T06:33:06.246 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:06.246 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4397.0:74241 3.0 e73/23 3:06e93fb9:::rbd_data.112d80ca42af.0000000000000000:head v 73'102, mlcod=71'100)", 2024-09-14T06:33:06.246 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:53.876730+0000", 2024-09-14T06:33:06.246 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 192.36739959100001, 2024-09-14T06:33:06.247 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0079470329999999992, 2024-09-14T06:33:06.247 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:33:06.247 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:33:06.247 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:33:06.247 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:33:06.247 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:06.247 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:33:06.247 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:53.876730+0000", 2024-09-14T06:33:06.247 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:33:06.247 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:06.247 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:06.247 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:33:06.247 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:53.876730+0000", 2024-09-14T06:33:06.247 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:33:06.248 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:06.248 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:06.248 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:33:06.248 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:53.876731+0000", 2024-09-14T06:33:06.248 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.0500000000000004e-07 2024-09-14T06:33:06.248 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:06.248 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:06.248 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:33:06.248 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:53.879616+0000", 2024-09-14T06:33:06.248 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0028856680000000001 2024-09-14T06:33:06.248 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:06.248 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:06.248 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:33:06.249 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:53.879643+0000", 2024-09-14T06:33:06.249 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.6120999999999999e-05 2024-09-14T06:33:06.249 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:06.249 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:06.249 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:33:06.249 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:53.879654+0000", 2024-09-14T06:33:06.249 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.1586999999999999e-05 2024-09-14T06:33:06.249 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:06.249 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:06.249 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:33:06.249 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:53.879756+0000", 2024-09-14T06:33:06.249 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00010223899999999999 2024-09-14T06:33:06.249 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:06.249 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:06.249 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:33:06.250 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:53.879783+0000", 2024-09-14T06:33:06.250 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.6913000000000001e-05 2024-09-14T06:33:06.250 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:06.250 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:06.250 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-14T06:33:06.250 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:53.884594+0000", 2024-09-14T06:33:06.250 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0048110729999999999 2024-09-14T06:33:06.250 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:06.250 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:06.250 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:33:06.250 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:53.884677+0000", 2024-09-14T06:33:06.250 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.2927000000000001e-05 2024-09-14T06:33:06.250 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:33:06.250 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:33:06.251 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:33:06.251 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:06.251 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:06.251 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4397.0:74243 3.9 e73/23 3:94977f81:::rbd_data.112d80ca42af.0000000000000002:head v 73'94, mlcod=71'92)", 2024-09-14T06:33:06.251 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:53.885156+0000", 2024-09-14T06:33:06.251 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 192.35897367300001, 2024-09-14T06:33:06.251 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0096009860000000006, 2024-09-14T06:33:06.251 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:33:06.251 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:33:06.251 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:33:06.251 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:33:06.251 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:06.251 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:33:06.252 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:53.885156+0000", 2024-09-14T06:33:06.252 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:33:06.252 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:06.252 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:06.252 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:33:06.252 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:53.885156+0000", 2024-09-14T06:33:06.252 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:33:06.252 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:06.252 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:06.252 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:33:06.252 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:53.885156+0000", 2024-09-14T06:33:06.252 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.23e-07 2024-09-14T06:33:06.252 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:06.253 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:06.253 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:33:06.253 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:53.889694+0000", 2024-09-14T06:33:06.253 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0045378709999999997 2024-09-14T06:33:06.253 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:06.253 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:06.253 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:33:06.253 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:53.889720+0000", 2024-09-14T06:33:06.253 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.5259e-05 2024-09-14T06:33:06.253 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:06.253 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:06.253 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:33:06.253 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:53.889731+0000", 2024-09-14T06:33:06.253 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.1739000000000001e-05 2024-09-14T06:33:06.253 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:06.254 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:06.254 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:33:06.254 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:53.889822+0000", 2024-09-14T06:33:06.254 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.1200999999999996e-05 2024-09-14T06:33:06.254 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:06.254 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:06.254 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:33:06.254 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:53.889850+0000", 2024-09-14T06:33:06.254 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.7084999999999999e-05 2024-09-14T06:33:06.254 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:06.254 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:06.254 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-14T06:33:06.254 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:53.894703+0000", 2024-09-14T06:33:06.254 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0048532690000000003 2024-09-14T06:33:06.254 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:06.255 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:06.255 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:33:06.255 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:53.894757+0000", 2024-09-14T06:33:06.255 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.4339000000000003e-05 2024-09-14T06:33:06.255 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:33:06.255 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:33:06.255 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:33:06.255 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:06.255 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:06.255 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4397.0:74249 3.0 e73/23 3:009b0225:::rbd_data.112d80ca42af.0000000000000009:head v 73'103, mlcod=71'101)", 2024-09-14T06:33:06.255 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:53.893219+0000", 2024-09-14T06:33:06.255 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 192.35091119500001, 2024-09-14T06:33:06.255 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0065381980000000003, 2024-09-14T06:33:06.255 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:33:06.255 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:33:06.256 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:33:06.256 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:33:06.256 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:06.256 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:33:06.256 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:53.893219+0000", 2024-09-14T06:33:06.256 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:33:06.256 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:06.256 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:06.256 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:33:06.256 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:53.893219+0000", 2024-09-14T06:33:06.256 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:33:06.256 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:06.256 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:06.256 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:33:06.256 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:53.893221+0000", 2024-09-14T06:33:06.257 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.4329999999999998e-06 2024-09-14T06:33:06.257 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:06.257 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:06.257 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:33:06.257 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:53.895968+0000", 2024-09-14T06:33:06.257 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.002746746 2024-09-14T06:33:06.257 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:06.257 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:06.257 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:33:06.257 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:53.895988+0000", 2024-09-14T06:33:06.257 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.0227000000000001e-05 2024-09-14T06:33:06.257 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:06.257 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:06.257 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:33:06.257 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:53.896000+0000", 2024-09-14T06:33:06.258 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.2359e-05 2024-09-14T06:33:06.258 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:06.258 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:06.258 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:33:06.258 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:53.896084+0000", 2024-09-14T06:33:06.258 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.3659999999999995e-05 2024-09-14T06:33:06.258 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:06.258 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:06.258 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:33:06.258 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:53.896109+0000", 2024-09-14T06:33:06.258 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.5259999999999999e-05 2024-09-14T06:33:06.258 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:06.258 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:06.258 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-14T06:33:06.258 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:53.899704+0000", 2024-09-14T06:33:06.258 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0035942209999999999 2024-09-14T06:33:06.259 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:06.259 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:06.259 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:33:06.259 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:53.899757+0000", 2024-09-14T06:33:06.259 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.3291999999999997e-05 2024-09-14T06:33:06.259 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:33:06.259 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:33:06.259 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:33:06.259 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:06.259 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:06.259 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4397.0:204589 3.4 e166/23 3:234552f0:::rbd_data.112d3bd84cb0.000000000000003f:head v 166'499, mlcod=165'497)", 2024-09-14T06:33:06.259 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:31:30.772894+0000", 2024-09-14T06:33:06.259 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 95.471235540999999, 2024-09-14T06:33:06.259 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0034564650000000001, 2024-09-14T06:33:06.259 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:33:06.260 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:33:06.260 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:33:06.260 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:33:06.260 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:06.260 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:33:06.260 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:30.772894+0000", 2024-09-14T06:33:06.260 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:33:06.260 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:06.260 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:06.260 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:33:06.260 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:30.772894+0000", 2024-09-14T06:33:06.260 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:33:06.260 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:06.260 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:06.261 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:33:06.261 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:30.772896+0000", 2024-09-14T06:33:06.261 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.5379999999999999e-06 2024-09-14T06:33:06.261 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:06.261 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:06.261 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:33:06.261 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:30.773042+0000", 2024-09-14T06:33:06.261 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000146107 2024-09-14T06:33:06.261 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:06.261 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:06.261 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:33:06.261 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:30.773061+0000", 2024-09-14T06:33:06.261 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.8862999999999999e-05 2024-09-14T06:33:06.261 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:06.262 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:06.262 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:33:06.262 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:30.773071+0000", 2024-09-14T06:33:06.262 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.0227000000000001e-05 2024-09-14T06:33:06.262 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:06.262 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:06.262 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:33:06.262 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:30.773149+0000", 2024-09-14T06:33:06.262 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.8071999999999994e-05 2024-09-14T06:33:06.262 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:06.262 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:06.262 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:33:06.262 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:30.773171+0000", 2024-09-14T06:33:06.262 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.1699999999999999e-05 2024-09-14T06:33:06.263 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:06.263 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:06.263 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-14T06:33:06.263 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:30.776285+0000", 2024-09-14T06:33:06.263 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0031142050000000001 2024-09-14T06:33:06.263 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:06.263 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:06.263 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:33:06.263 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:30.776351+0000", 2024-09-14T06:33:06.263 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.5753000000000006e-05 2024-09-14T06:33:06.263 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:33:06.263 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:33:06.263 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:33:06.263 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:06.264 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:06.264 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_repop(client.4397.0:204598 3.4 e166/23 3:245d069e:::rbd_data.112d3bd84cb0.0000000000000041:head v 166'500, mlcod=165'497)", 2024-09-14T06:33:06.264 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:31:30.775247+0000", 2024-09-14T06:33:06.264 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 95.468882418000007, 2024-09-14T06:33:06.264 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00318235, 2024-09-14T06:33:06.264 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:33:06.264 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:33:06.264 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:33:06.264 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:33:06.264 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:06.264 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:33:06.264 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:30.775247+0000", 2024-09-14T06:33:06.264 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:33:06.265 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:06.265 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:06.265 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:33:06.265 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:30.775247+0000", 2024-09-14T06:33:06.265 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:33:06.265 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:06.265 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:06.265 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:33:06.265 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:30.775249+0000", 2024-09-14T06:33:06.265 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.265e-06 2024-09-14T06:33:06.265 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:06.265 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:06.266 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:33:06.266 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:30.775340+0000", 2024-09-14T06:33:06.266 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.1131999999999994e-05 2024-09-14T06:33:06.266 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:06.266 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:06.266 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:33:06.266 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:30.775352+0000", 2024-09-14T06:33:06.266 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.2294e-05 2024-09-14T06:33:06.266 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:06.266 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:06.266 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:33:06.266 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:30.775361+0000", 2024-09-14T06:33:06.266 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.6950000000000006e-06 2024-09-14T06:33:06.266 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:06.267 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:06.267 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:33:06.267 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:30.775439+0000", 2024-09-14T06:33:06.267 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.7910000000000005e-05 2024-09-14T06:33:06.267 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:06.267 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:06.267 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:33:06.267 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:30.775460+0000", 2024-09-14T06:33:06.267 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.0900999999999999e-05 2024-09-14T06:33:06.267 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:06.267 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:06.267 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-14T06:33:06.267 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:30.778385+0000", 2024-09-14T06:33:06.268 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.002925539 2024-09-14T06:33:06.268 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:06.268 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:06.268 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:33:06.268 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:30.778430+0000", 2024-09-14T06:33:06.268 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.4613999999999997e-05 2024-09-14T06:33:06.268 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:33:06.268 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:33:06.268 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:33:06.268 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:06.268 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:06.268 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:236487 3.14 3:2d502bc2:::rbd_data.112d97471187.000000000000008e:head [delete] snapc 2f=[] ondisk+write+known_if_redirected+supports_pool_eio e234)", 2024-09-14T06:33:06.268 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:32:39.870309+0000", 2024-09-14T06:33:06.269 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 26.373820518999999, 2024-09-14T06:33:06.269 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0042368170000000004, 2024-09-14T06:33:06.269 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:33:06.269 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:33:06.269 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:33:06.269 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:33:06.269 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:33:06.269 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:33:06.269 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 236487 2024-09-14T06:33:06.269 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:06.269 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:33:06.269 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:06.269 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:33:06.269 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:39.870309+0000", 2024-09-14T06:33:06.270 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:33:06.270 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:06.270 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:06.270 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:33:06.270 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:39.870309+0000", 2024-09-14T06:33:06.270 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:33:06.270 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:06.270 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:06.270 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:33:06.270 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:39.870311+0000", 2024-09-14T06:33:06.270 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.0080000000000001e-06 2024-09-14T06:33:06.270 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:06.270 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:06.270 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:33:06.270 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:39.870316+0000", 2024-09-14T06:33:06.271 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.7160000000000002e-06 2024-09-14T06:33:06.271 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:06.271 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:06.271 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:33:06.271 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:39.870327+0000", 2024-09-14T06:33:06.271 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.0995999999999999e-05 2024-09-14T06:33:06.271 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:06.271 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:06.271 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:33:06.271 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:39.870335+0000", 2024-09-14T06:33:06.271 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.2139999999999996e-06 2024-09-14T06:33:06.271 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:06.271 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:06.271 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:33:06.271 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:39.870403+0000", 2024-09-14T06:33:06.272 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.7966999999999995e-05 2024-09-14T06:33:06.272 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:06.272 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:06.272 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:33:06.272 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:39.870543+0000", 2024-09-14T06:33:06.272 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00014018799999999999 2024-09-14T06:33:06.272 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:06.272 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:06.272 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "waiting for subops from 7", 2024-09-14T06:33:06.272 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:39.870669+0000", 2024-09-14T06:33:06.272 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000125778 2024-09-14T06:33:06.272 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:06.272 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:06.272 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "op_commit", 2024-09-14T06:33:06.272 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:39.871410+0000", 2024-09-14T06:33:06.273 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00074113599999999999 2024-09-14T06:33:06.273 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:06.273 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:06.273 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:33:06.273 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:39.874439+0000", 2024-09-14T06:33:06.273 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0030287259999999998 2024-09-14T06:33:06.273 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:06.273 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:06.273 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-14T06:33:06.273 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:39.874495+0000", 2024-09-14T06:33:06.273 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.5855e-05 2024-09-14T06:33:06.273 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:06.273 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:06.273 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:33:06.274 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:39.874546+0000", 2024-09-14T06:33:06.274 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.1233000000000002e-05 2024-09-14T06:33:06.274 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:33:06.274 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:33:06.274 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:33:06.274 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:06.274 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:06.274 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:236490 3.14 3:2e5aef87:::rbd_data.112d97471187.00000000000000ca:head [delete] snapc 2f=[] ondisk+write+known_if_redirected+supports_pool_eio e234)", 2024-09-14T06:33:06.274 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:32:39.872130+0000", 2024-09-14T06:33:06.274 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 26.371999868, 2024-09-14T06:33:06.274 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0048885589999999998, 2024-09-14T06:33:06.274 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:33:06.274 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:33:06.274 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:33:06.275 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:33:06.275 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:33:06.275 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:33:06.275 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 236490 2024-09-14T06:33:06.275 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:06.275 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:33:06.275 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:06.275 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:33:06.275 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:39.872130+0000", 2024-09-14T06:33:06.275 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:33:06.275 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:06.275 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:06.275 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:33:06.275 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:39.872130+0000", 2024-09-14T06:33:06.276 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:33:06.276 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:06.276 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:06.276 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:33:06.276 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:39.872131+0000", 2024-09-14T06:33:06.276 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.0720000000000001e-06 2024-09-14T06:33:06.276 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:06.276 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:06.276 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:33:06.276 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:39.872136+0000", 2024-09-14T06:33:06.276 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.8790000000000002e-06 2024-09-14T06:33:06.276 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:06.276 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:06.276 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:33:06.277 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:39.872150+0000", 2024-09-14T06:33:06.277 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.435e-05 2024-09-14T06:33:06.277 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:06.277 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:06.277 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:33:06.277 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:39.872158+0000", 2024-09-14T06:33:06.277 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.571e-06 2024-09-14T06:33:06.277 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:06.277 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:06.277 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:33:06.277 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:39.872234+0000", 2024-09-14T06:33:06.277 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.606e-05 2024-09-14T06:33:06.277 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:06.277 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:06.278 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:33:06.278 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:39.872356+0000", 2024-09-14T06:33:06.278 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00012250499999999999 2024-09-14T06:33:06.278 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:06.278 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:06.278 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "waiting for subops from 7", 2024-09-14T06:33:06.278 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:39.872480+0000", 2024-09-14T06:33:06.278 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00012316700000000001 2024-09-14T06:33:06.278 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:06.278 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:06.278 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "op_commit", 2024-09-14T06:33:06.278 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:39.873051+0000", 2024-09-14T06:33:06.278 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00057178100000000005 2024-09-14T06:33:06.278 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:06.278 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:06.278 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:33:06.279 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:39.876925+0000", 2024-09-14T06:33:06.279 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0038739239999999999 2024-09-14T06:33:06.279 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:06.279 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:06.279 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-14T06:33:06.279 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:39.876974+0000", 2024-09-14T06:33:06.279 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.8207000000000001e-05 2024-09-14T06:33:06.279 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:06.279 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:06.279 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:33:06.279 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:39.877019+0000", 2024-09-14T06:33:06.279 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.5043000000000002e-05 2024-09-14T06:33:06.279 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:33:06.279 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:33:06.280 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:33:06.280 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:06.280 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:06.280 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:237283 3.14 3:2f6164a9:::rbd_data.112d401cec05.000000000000000a:head [delete] snapc 25=[] ondisk+write+known_if_redirected+supports_pool_eio e239)", 2024-09-14T06:33:06.280 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:32:44.693535+0000", 2024-09-14T06:33:06.280 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 21.550594814, 2024-09-14T06:33:06.280 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0070517319999999998, 2024-09-14T06:33:06.280 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:33:06.280 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:33:06.280 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:33:06.280 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:33:06.280 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:33:06.280 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:33:06.280 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 237283 2024-09-14T06:33:06.281 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:06.281 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:33:06.281 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:06.281 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:33:06.281 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:44.693535+0000", 2024-09-14T06:33:06.281 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:33:06.281 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:06.281 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:06.281 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:33:06.281 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:44.693535+0000", 2024-09-14T06:33:06.281 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:33:06.281 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:06.281 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:06.281 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:33:06.282 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:44.693537+0000", 2024-09-14T06:33:06.282 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.4649999999999999e-06 2024-09-14T06:33:06.282 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:06.282 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:06.282 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:33:06.282 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:44.693541+0000", 2024-09-14T06:33:06.282 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.4950000000000002e-06 2024-09-14T06:33:06.282 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:06.282 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:06.282 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:33:06.282 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:44.693553+0000", 2024-09-14T06:33:06.282 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.2238e-05 2024-09-14T06:33:06.282 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:06.282 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:06.283 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:33:06.283 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:44.693561+0000", 2024-09-14T06:33:06.283 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.4379999999999998e-06 2024-09-14T06:33:06.283 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:06.283 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:06.283 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:33:06.283 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:44.693633+0000", 2024-09-14T06:33:06.283 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.2234e-05 2024-09-14T06:33:06.283 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:06.283 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:06.283 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "op must wait for map", 2024-09-14T06:33:06.283 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:44.693642+0000", 2024-09-14T06:33:06.283 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.6770000000000001e-06 2024-09-14T06:33:06.283 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:06.283 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:06.284 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:33:06.284 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:44.695664+0000", 2024-09-14T06:33:06.284 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.002022252 2024-09-14T06:33:06.284 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:06.284 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:06.284 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:33:06.284 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:44.695825+0000", 2024-09-14T06:33:06.284 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00016144099999999999 2024-09-14T06:33:06.284 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:06.284 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:06.284 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "waiting for subops from 7", 2024-09-14T06:33:06.284 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:44.695947+0000", 2024-09-14T06:33:06.284 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000121674 2024-09-14T06:33:06.284 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:06.284 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:06.285 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "op_commit", 2024-09-14T06:33:06.285 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:44.696853+0000", 2024-09-14T06:33:06.285 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00090605000000000004 2024-09-14T06:33:06.285 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:06.285 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:06.285 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:33:06.285 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:44.700467+0000", 2024-09-14T06:33:06.285 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0036137909999999999 2024-09-14T06:33:06.285 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:06.285 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:06.285 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-14T06:33:06.285 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:44.700533+0000", 2024-09-14T06:33:06.285 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.6583999999999998e-05 2024-09-14T06:33:06.285 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:06.285 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:06.286 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:33:06.286 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:44.700587+0000", 2024-09-14T06:33:06.286 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.3393000000000002e-05 2024-09-14T06:33:06.286 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:33:06.286 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:33:06.286 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:33:06.286 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:06.286 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:06.286 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:237364 3.14 3:2a8d6252:::rbd_data.112d401cec05.000000000000005a:head [delete] snapc 25=[] ondisk+write+known_if_redirected+supports_pool_eio e239)", 2024-09-14T06:33:06.286 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:32:44.724057+0000", 2024-09-14T06:33:06.286 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 21.520072672000001, 2024-09-14T06:33:06.286 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.005405923, 2024-09-14T06:33:06.286 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:33:06.286 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:33:06.286 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:33:06.286 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:33:06.287 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:33:06.287 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:33:06.287 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 237364 2024-09-14T06:33:06.287 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:06.287 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:33:06.287 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:06.287 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:33:06.287 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:44.724057+0000", 2024-09-14T06:33:06.287 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:33:06.287 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:06.287 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:06.287 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:33:06.287 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:44.724057+0000", 2024-09-14T06:33:06.288 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:33:06.288 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:06.288 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:06.288 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:33:06.288 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:44.724059+0000", 2024-09-14T06:33:06.288 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.854e-06 2024-09-14T06:33:06.288 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:06.288 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:06.288 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:33:06.288 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:44.724066+0000", 2024-09-14T06:33:06.288 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.4410000000000004e-06 2024-09-14T06:33:06.288 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:06.288 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:06.288 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:33:06.289 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:44.724086+0000", 2024-09-14T06:33:06.289 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.9128e-05 2024-09-14T06:33:06.289 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:06.289 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:06.289 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:33:06.289 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:44.724095+0000", 2024-09-14T06:33:06.289 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.4900000000000006e-06 2024-09-14T06:33:06.289 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:06.289 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:06.289 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:33:06.289 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:44.724172+0000", 2024-09-14T06:33:06.289 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.7226999999999998e-05 2024-09-14T06:33:06.289 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:06.289 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:06.289 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:33:06.290 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:44.724334+0000", 2024-09-14T06:33:06.290 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000161545 2024-09-14T06:33:06.290 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:06.290 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:06.290 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "waiting for subops from 7", 2024-09-14T06:33:06.290 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:44.724472+0000", 2024-09-14T06:33:06.290 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00013770300000000001 2024-09-14T06:33:06.290 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:06.290 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:06.290 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "op_commit", 2024-09-14T06:33:06.290 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:44.725367+0000", 2024-09-14T06:33:06.290 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00089564699999999996 2024-09-14T06:33:06.291 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:06.291 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:06.291 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:33:06.291 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:44.729349+0000", 2024-09-14T06:33:06.291 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0039818539999999999 2024-09-14T06:33:06.291 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:06.291 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:06.291 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-14T06:33:06.291 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:44.729410+0000", 2024-09-14T06:33:06.291 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.0523000000000003e-05 2024-09-14T06:33:06.291 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:06.291 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:06.291 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:33:06.291 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:44.729463+0000", 2024-09-14T06:33:06.292 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.3511000000000003e-05 2024-09-14T06:33:06.292 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:33:06.292 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:33:06.292 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:33:06.292 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:06.292 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:06.292 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:237438 3.14 3:2b4d012e:::rbd_data.112d401cec05.00000000000000ae:head [delete] snapc 25=[] ondisk+write+known_if_redirected+supports_pool_eio e239)", 2024-09-14T06:33:06.292 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:32:44.755579+0000", 2024-09-14T06:33:06.292 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 21.488550545999999, 2024-09-14T06:33:06.292 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0056954090000000002, 2024-09-14T06:33:06.292 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:33:06.292 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:33:06.292 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:33:06.293 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:33:06.293 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:33:06.293 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:33:06.293 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 237438 2024-09-14T06:33:06.293 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:06.293 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:33:06.293 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:06.293 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:33:06.293 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:44.755579+0000", 2024-09-14T06:33:06.293 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:33:06.293 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:06.293 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:06.293 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:33:06.293 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:44.755579+0000", 2024-09-14T06:33:06.294 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:33:06.294 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:06.294 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:06.294 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:33:06.294 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:44.755581+0000", 2024-09-14T06:33:06.294 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.9099999999999999e-06 2024-09-14T06:33:06.294 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:06.294 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:06.294 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:33:06.294 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:44.755588+0000", 2024-09-14T06:33:06.294 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.9580000000000001e-06 2024-09-14T06:33:06.294 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:06.294 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:06.294 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:33:06.294 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:44.755605+0000", 2024-09-14T06:33:06.294 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.6481e-05 2024-09-14T06:33:06.295 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:06.295 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:06.295 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:33:06.295 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:44.755613+0000", 2024-09-14T06:33:06.295 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.2490000000000001e-06 2024-09-14T06:33:06.295 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:06.295 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:06.295 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:33:06.295 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:44.755706+0000", 2024-09-14T06:33:06.295 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.3346999999999999e-05 2024-09-14T06:33:06.295 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:06.295 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:06.295 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:33:06.295 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:44.755858+0000", 2024-09-14T06:33:06.296 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00015165600000000001 2024-09-14T06:33:06.296 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:06.296 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:06.296 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "waiting for subops from 7", 2024-09-14T06:33:06.296 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:44.755984+0000", 2024-09-14T06:33:06.296 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000125907 2024-09-14T06:33:06.296 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:06.296 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:06.296 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "op_commit", 2024-09-14T06:33:06.296 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:44.756892+0000", 2024-09-14T06:33:06.296 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00090802199999999997 2024-09-14T06:33:06.296 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:06.296 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:06.296 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:33:06.297 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:44.761135+0000", 2024-09-14T06:33:06.297 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.004243251 2024-09-14T06:33:06.297 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:06.297 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:06.297 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-14T06:33:06.297 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:44.761211+0000", 2024-09-14T06:33:06.297 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.6027999999999998e-05 2024-09-14T06:33:06.297 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:06.297 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:06.297 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:33:06.297 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:44.761275+0000", 2024-09-14T06:33:06.297 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.3600000000000001e-05 2024-09-14T06:33:06.297 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:33:06.297 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:33:06.298 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:33:06.298 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:06.298 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:06.298 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:237445 3.14 3:2e12370a:::rbd_data.112d401cec05.00000000000000b5:head [delete] snapc 25=[] ondisk+write+known_if_redirected+supports_pool_eio e239)", 2024-09-14T06:33:06.298 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:32:44.757373+0000", 2024-09-14T06:33:06.298 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 21.486756722999999, 2024-09-14T06:33:06.298 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0063576639999999999, 2024-09-14T06:33:06.298 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:33:06.298 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:33:06.298 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:33:06.298 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:33:06.298 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:33:06.298 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:33:06.298 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 237445 2024-09-14T06:33:06.299 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:06.299 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:33:06.299 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:06.299 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:33:06.299 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:44.757373+0000", 2024-09-14T06:33:06.299 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:33:06.299 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:06.299 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:06.299 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:33:06.299 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:44.757373+0000", 2024-09-14T06:33:06.299 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:33:06.299 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:06.299 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:06.299 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:33:06.299 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:44.757374+0000", 2024-09-14T06:33:06.300 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.359e-06 2024-09-14T06:33:06.300 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:06.300 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:06.300 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:33:06.300 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:44.757381+0000", 2024-09-14T06:33:06.300 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.7769999999999997e-06 2024-09-14T06:33:06.300 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:06.300 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:06.300 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:33:06.300 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:44.757397+0000", 2024-09-14T06:33:06.301 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.5610000000000001e-05 2024-09-14T06:33:06.301 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:06.301 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:06.301 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:33:06.301 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:44.757406+0000", 2024-09-14T06:33:06.301 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.8300000000000002e-06 2024-09-14T06:33:06.301 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:06.301 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:06.301 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:33:06.301 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:44.757462+0000", 2024-09-14T06:33:06.301 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.6348000000000003e-05 2024-09-14T06:33:06.301 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:06.302 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:06.302 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:33:06.302 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:44.757579+0000", 2024-09-14T06:33:06.302 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000116747 2024-09-14T06:33:06.302 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:06.302 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:06.302 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "waiting for subops from 7", 2024-09-14T06:33:06.302 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:44.757704+0000", 2024-09-14T06:33:06.302 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00012546899999999999 2024-09-14T06:33:06.302 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:06.302 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:06.302 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "op_commit", 2024-09-14T06:33:06.302 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:44.758510+0000", 2024-09-14T06:33:06.302 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00080536099999999999 2024-09-14T06:33:06.303 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:06.303 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:06.303 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:33:06.303 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:44.763596+0000", 2024-09-14T06:33:06.303 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0050862049999999999 2024-09-14T06:33:06.303 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:06.303 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:06.303 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-14T06:33:06.303 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:44.763673+0000", 2024-09-14T06:33:06.303 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.7464000000000002e-05 2024-09-14T06:33:06.303 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:06.303 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:06.303 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:33:06.303 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:44.763731+0000", 2024-09-14T06:33:06.303 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.7494000000000002e-05 2024-09-14T06:33:06.304 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:33:06.304 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:33:06.304 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:33:06.304 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:06.304 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:06.304 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:238182 3.a 3.3eafbe6a (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e242)", 2024-09-14T06:33:06.304 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:32:47.709833+0000", 2024-09-14T06:33:06.304 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 18.534296596000001, 2024-09-14T06:33:06.304 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.90868078100000005, 2024-09-14T06:33:06.304 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:33:06.304 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:33:06.304 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "reached pg", 2024-09-14T06:33:06.304 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:33:06.304 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:33:06.304 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:33:06.305 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 238182 2024-09-14T06:33:06.305 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:06.305 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:33:06.305 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:06.305 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:33:06.305 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:47.709833+0000", 2024-09-14T06:33:06.305 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:33:06.305 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:06.305 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:06.305 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:33:06.305 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:47.709833+0000", 2024-09-14T06:33:06.305 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:33:06.306 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:06.306 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:06.306 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:33:06.306 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:47.709835+0000", 2024-09-14T06:33:06.306 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.0020000000000001e-06 2024-09-14T06:33:06.306 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:06.306 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:06.306 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:33:06.306 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:47.709843+0000", 2024-09-14T06:33:06.306 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.0779999999999996e-06 2024-09-14T06:33:06.306 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:06.306 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:06.306 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:33:06.306 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:47.709864+0000", 2024-09-14T06:33:06.306 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.0605999999999999e-05 2024-09-14T06:33:06.307 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:06.307 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:06.307 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:33:06.307 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:47.709874+0000", 2024-09-14T06:33:06.307 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.9340000000000001e-06 2024-09-14T06:33:06.307 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:06.307 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:06.307 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:33:06.307 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:47.709948+0000", 2024-09-14T06:33:06.307 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.3796999999999998e-05 2024-09-14T06:33:06.307 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:06.307 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:06.307 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "waiting for peered", 2024-09-14T06:33:06.307 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:47.709956+0000", 2024-09-14T06:33:06.307 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.7760000000000003e-06 2024-09-14T06:33:06.307 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:06.308 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:06.308 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:33:06.308 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:48.618503+0000", 2024-09-14T06:33:06.308 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.90854614899999997 2024-09-14T06:33:06.308 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:06.308 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:06.308 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:33:06.308 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:48.618514+0000", 2024-09-14T06:33:06.308 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.1439e-05 2024-09-14T06:33:06.308 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:33:06.308 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:33:06.308 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:33:06.308 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:06.308 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:06.308 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:238198 3.14 3:2d396db2:::rbd_data.112d80ca42af.0000000000000017:head [delete] snapc 1f=[] ondisk+write+known_if_redirected+supports_pool_eio e242)", 2024-09-14T06:33:06.309 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:32:47.719965+0000", 2024-09-14T06:33:06.309 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 18.524165084, 2024-09-14T06:33:06.309 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0039876809999999999, 2024-09-14T06:33:06.309 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:33:06.309 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:33:06.309 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:33:06.309 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:33:06.309 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:33:06.309 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:33:06.309 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 238198 2024-09-14T06:33:06.309 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:06.309 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:33:06.309 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:06.310 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:33:06.310 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:47.719965+0000", 2024-09-14T06:33:06.310 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:33:06.310 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:06.310 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:06.310 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:33:06.310 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:47.719965+0000", 2024-09-14T06:33:06.310 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:33:06.310 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:06.310 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:06.310 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:33:06.310 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:47.719966+0000", 2024-09-14T06:33:06.310 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.319e-06 2024-09-14T06:33:06.310 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:06.311 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:06.311 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:33:06.311 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:47.719972+0000", 2024-09-14T06:33:06.311 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.4709999999999998e-06 2024-09-14T06:33:06.311 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:06.311 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:06.311 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:33:06.311 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:47.719982+0000", 2024-09-14T06:33:06.311 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.029e-05 2024-09-14T06:33:06.311 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:06.311 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:06.311 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:33:06.311 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:47.719988+0000", 2024-09-14T06:33:06.311 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.6050000000000003e-06 2024-09-14T06:33:06.312 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:06.312 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:06.312 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:33:06.312 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:47.720064+0000", 2024-09-14T06:33:06.312 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.5893e-05 2024-09-14T06:33:06.312 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:06.312 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:06.312 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:33:06.312 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:47.720199+0000", 2024-09-14T06:33:06.312 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00013506100000000001 2024-09-14T06:33:06.312 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:06.312 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:06.312 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "waiting for subops from 7", 2024-09-14T06:33:06.312 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:47.720325+0000", 2024-09-14T06:33:06.312 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000125456 2024-09-14T06:33:06.313 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:06.313 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:06.313 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "op_commit", 2024-09-14T06:33:06.313 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:47.722213+0000", 2024-09-14T06:33:06.313 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0018876260000000001 2024-09-14T06:33:06.313 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:06.313 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:06.313 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:33:06.313 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:47.723839+0000", 2024-09-14T06:33:06.313 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0016260210000000001 2024-09-14T06:33:06.313 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:06.313 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:06.313 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-14T06:33:06.313 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:47.723898+0000", 2024-09-14T06:33:06.314 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.9728000000000001e-05 2024-09-14T06:33:06.314 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:06.314 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:06.314 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:33:06.314 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:47.723952+0000", 2024-09-14T06:33:06.314 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.4211e-05 2024-09-14T06:33:06.314 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:33:06.314 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:33:06.314 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:33:06.314 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:06.314 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:06.314 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:238917 3.a 3:52903c9c:::rbd_data.112d5a737e72.000000000000005c:head [delete] snapc 19=[] ondisk+write+known_if_redirected+supports_pool_eio e248)", 2024-09-14T06:33:06.314 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:32:53.735625+0000", 2024-09-14T06:33:06.314 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 12.508504523999999, 2024-09-14T06:33:06.314 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0035851860000000002, 2024-09-14T06:33:06.315 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:33:06.315 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:33:06.315 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:33:06.315 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:33:06.315 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:33:06.315 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:33:06.315 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 238917 2024-09-14T06:33:06.315 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:06.315 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:33:06.315 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:06.315 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:33:06.315 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:53.735625+0000", 2024-09-14T06:33:06.315 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:33:06.315 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:06.315 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:06.316 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:33:06.316 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:53.735625+0000", 2024-09-14T06:33:06.316 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:33:06.316 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:06.316 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:06.316 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:33:06.316 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:53.735627+0000", 2024-09-14T06:33:06.316 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.699e-06 2024-09-14T06:33:06.316 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:06.316 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:06.316 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:33:06.316 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:53.735633+0000", 2024-09-14T06:33:06.316 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.896e-06 2024-09-14T06:33:06.316 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:06.316 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:06.317 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:33:06.317 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:53.735649+0000", 2024-09-14T06:33:06.317 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.5573000000000002e-05 2024-09-14T06:33:06.317 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:06.317 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:06.317 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:33:06.317 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:53.735657+0000", 2024-09-14T06:33:06.317 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.0520000000000002e-06 2024-09-14T06:33:06.317 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:06.317 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:06.317 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:33:06.317 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:53.735724+0000", 2024-09-14T06:33:06.317 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.7594000000000004e-05 2024-09-14T06:33:06.318 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:06.318 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:06.318 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:33:06.318 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:53.735906+0000", 2024-09-14T06:33:06.318 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00018224800000000001 2024-09-14T06:33:06.318 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:06.318 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:06.318 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "waiting for subops from 3", 2024-09-14T06:33:06.318 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:53.736040+0000", 2024-09-14T06:33:06.318 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000133628 2024-09-14T06:33:06.318 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:06.318 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:06.318 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "op_commit", 2024-09-14T06:33:06.319 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:53.736863+0000", 2024-09-14T06:33:06.319 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00082262100000000005 2024-09-14T06:33:06.319 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:06.319 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:06.319 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:33:06.319 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:53.739079+0000", 2024-09-14T06:33:06.319 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.002216057 2024-09-14T06:33:06.319 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:06.319 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:06.319 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-14T06:33:06.319 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:53.739160+0000", 2024-09-14T06:33:06.319 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.1441999999999997e-05 2024-09-14T06:33:06.319 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:06.319 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:06.319 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:33:06.320 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:53.739211+0000", 2024-09-14T06:33:06.320 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.0376e-05 2024-09-14T06:33:06.320 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:33:06.320 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:33:06.320 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:33:06.320 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:06.320 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:06.320 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:239652 3.14 3:2fa4ee6b:::rbd_data.112d8f603570.0000000000000068:head [delete] snapc 11=[] ondisk+write+known_if_redirected+supports_pool_eio e252)", 2024-09-14T06:33:06.320 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:32:57.751430+0000", 2024-09-14T06:33:06.320 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 8.4926998749999996, 2024-09-14T06:33:06.320 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0032203599999999998, 2024-09-14T06:33:06.320 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:33:06.320 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:33:06.321 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:33:06.321 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:33:06.321 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:33:06.321 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:33:06.321 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 239652 2024-09-14T06:33:06.321 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:06.321 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:33:06.321 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:06.321 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:33:06.321 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:57.751430+0000", 2024-09-14T06:33:06.321 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:33:06.321 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:06.321 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:06.321 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:33:06.322 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:57.751430+0000", 2024-09-14T06:33:06.322 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:33:06.322 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:06.322 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:06.322 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:33:06.322 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:57.751431+0000", 2024-09-14T06:33:06.322 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.271e-06 2024-09-14T06:33:06.322 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:06.322 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:06.322 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:33:06.322 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:57.751436+0000", 2024-09-14T06:33:06.322 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.0520000000000001e-06 2024-09-14T06:33:06.322 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:06.322 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:06.322 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:33:06.323 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:57.751448+0000", 2024-09-14T06:33:06.323 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.1419e-05 2024-09-14T06:33:06.323 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:06.323 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:06.323 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:33:06.323 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:57.751455+0000", 2024-09-14T06:33:06.323 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.4420000000000001e-06 2024-09-14T06:33:06.323 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:06.323 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:06.323 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:33:06.323 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:57.751514+0000", 2024-09-14T06:33:06.323 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.9268999999999998e-05 2024-09-14T06:33:06.323 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:06.323 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:06.324 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:33:06.324 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:57.751650+0000", 2024-09-14T06:33:06.324 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000135332 2024-09-14T06:33:06.324 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:06.324 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:06.324 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "waiting for subops from 7", 2024-09-14T06:33:06.324 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:57.751765+0000", 2024-09-14T06:33:06.324 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000115109 2024-09-14T06:33:06.324 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:06.324 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:06.324 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "op_commit", 2024-09-14T06:33:06.324 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:57.753015+0000", 2024-09-14T06:33:06.324 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0012499919999999999 2024-09-14T06:33:06.324 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:06.325 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:06.325 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:33:06.325 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:57.754540+0000", 2024-09-14T06:33:06.325 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0015254400000000001 2024-09-14T06:33:06.325 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:06.325 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:06.325 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-14T06:33:06.325 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:57.754598+0000", 2024-09-14T06:33:06.325 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.7815999999999997e-05 2024-09-14T06:33:06.325 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:06.325 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:06.325 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:33:06.325 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:57.754650+0000", 2024-09-14T06:33:06.325 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.2218e-05 2024-09-14T06:33:06.326 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:33:06.326 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:33:06.326 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:33:06.326 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:06.326 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:06.326 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:239702 3.14 3:2ce24b3a:::rbd_data.112d8f603570.0000000000000130:head [delete] snapc 11=[] ondisk+write+known_if_redirected+supports_pool_eio e252)", 2024-09-14T06:33:06.326 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:32:57.764410+0000", 2024-09-14T06:33:06.326 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 8.4797197309999994, 2024-09-14T06:33:06.326 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0036256449999999998, 2024-09-14T06:33:06.326 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:33:06.326 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:33:06.326 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:33:06.326 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:33:06.326 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:33:06.327 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:33:06.327 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 239702 2024-09-14T06:33:06.327 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:06.327 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:33:06.327 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:06.327 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:33:06.327 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:57.764410+0000", 2024-09-14T06:33:06.327 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:33:06.327 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:06.327 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:06.327 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:33:06.327 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:57.764410+0000", 2024-09-14T06:33:06.327 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:33:06.327 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:06.328 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:06.328 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:33:06.328 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:57.764411+0000", 2024-09-14T06:33:06.328 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.062e-06 2024-09-14T06:33:06.328 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:06.328 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:06.328 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:33:06.328 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:57.764415+0000", 2024-09-14T06:33:06.328 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.0999999999999997e-06 2024-09-14T06:33:06.328 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:06.328 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:06.328 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:33:06.328 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:57.764429+0000", 2024-09-14T06:33:06.329 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.3365e-05 2024-09-14T06:33:06.329 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:06.329 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:06.329 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:33:06.329 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:57.764436+0000", 2024-09-14T06:33:06.329 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.9360000000000002e-06 2024-09-14T06:33:06.329 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:06.329 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:06.329 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:33:06.329 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:57.764483+0000", 2024-09-14T06:33:06.329 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.7747000000000002e-05 2024-09-14T06:33:06.329 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:06.329 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:06.329 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:33:06.330 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:57.764573+0000", 2024-09-14T06:33:06.330 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.9184999999999994e-05 2024-09-14T06:33:06.330 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:06.330 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:06.330 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "waiting for subops from 7", 2024-09-14T06:33:06.330 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:57.764657+0000", 2024-09-14T06:33:06.330 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.4374000000000002e-05 2024-09-14T06:33:06.330 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:06.330 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:06.330 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "op_commit", 2024-09-14T06:33:06.330 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:57.765242+0000", 2024-09-14T06:33:06.330 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00058536600000000003 2024-09-14T06:33:06.330 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:06.331 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:06.331 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:33:06.331 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:57.767916+0000", 2024-09-14T06:33:06.331 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.002673398 2024-09-14T06:33:06.331 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:06.331 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:06.331 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-14T06:33:06.331 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:57.767969+0000", 2024-09-14T06:33:06.331 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.3010000000000002e-05 2024-09-14T06:33:06.331 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:06.331 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:06.331 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:33:06.331 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:57.768036+0000", 2024-09-14T06:33:06.331 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.7101999999999996e-05 2024-09-14T06:33:06.331 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:33:06.332 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:33:06.332 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:33:06.332 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:06.332 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:06.332 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:239775 3.14 3:298708e5:::rbd_data.112d8f603570.0000000000000291:head [delete] snapc 11=[] ondisk+write+known_if_redirected+supports_pool_eio e252)", 2024-09-14T06:33:06.332 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:32:57.792966+0000", 2024-09-14T06:33:06.332 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 8.4511641839999996, 2024-09-14T06:33:06.332 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0054272879999999997, 2024-09-14T06:33:06.332 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:33:06.332 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:33:06.332 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:33:06.332 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:33:06.332 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:33:06.332 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:33:06.333 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 239775 2024-09-14T06:33:06.333 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:06.333 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:33:06.333 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:06.333 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:33:06.333 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:57.792966+0000", 2024-09-14T06:33:06.333 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:33:06.333 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:06.333 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:06.333 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:33:06.333 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:57.792966+0000", 2024-09-14T06:33:06.333 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:33:06.333 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:06.333 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:06.333 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:33:06.334 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:57.792967+0000", 2024-09-14T06:33:06.334 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.5600000000000001e-06 2024-09-14T06:33:06.334 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:06.334 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:06.334 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:33:06.334 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:57.792973+0000", 2024-09-14T06:33:06.334 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.7259999999999999e-06 2024-09-14T06:33:06.334 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:06.334 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:06.334 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:33:06.334 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:57.792990+0000", 2024-09-14T06:33:06.334 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.6651e-05 2024-09-14T06:33:06.334 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:06.334 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:06.334 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:33:06.335 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:57.792998+0000", 2024-09-14T06:33:06.335 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.3299999999999999e-06 2024-09-14T06:33:06.335 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:06.335 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:06.335 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:33:06.335 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:57.793066+0000", 2024-09-14T06:33:06.335 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.8182999999999994e-05 2024-09-14T06:33:06.335 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:06.335 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:06.335 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:33:06.335 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:57.793207+0000", 2024-09-14T06:33:06.335 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00014066600000000001 2024-09-14T06:33:06.335 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:06.335 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:06.335 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "waiting for subops from 7", 2024-09-14T06:33:06.336 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:57.793325+0000", 2024-09-14T06:33:06.336 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000118664 2024-09-14T06:33:06.336 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:06.336 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:06.336 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "op_commit", 2024-09-14T06:33:06.336 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:57.794251+0000", 2024-09-14T06:33:06.336 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00092560799999999996 2024-09-14T06:33:06.336 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:06.336 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:06.336 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:33:06.336 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:57.798268+0000", 2024-09-14T06:33:06.336 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0040170680000000004 2024-09-14T06:33:06.336 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:06.336 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:06.336 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-14T06:33:06.337 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:57.798337+0000", 2024-09-14T06:33:06.337 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.8813000000000007e-05 2024-09-14T06:33:06.337 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:06.337 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:06.337 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:33:06.337 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:57.798393+0000", 2024-09-14T06:33:06.337 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.6019e-05 2024-09-14T06:33:06.337 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:33:06.337 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:33:06.337 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:33:06.337 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:06.337 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:06.337 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:239927 3.a 3:51cf3b8a:::rbd_data.112d92027590.0000000000000006:head [delete] snapc f=[] ondisk+write+known_if_redirected+supports_pool_eio e253)", 2024-09-14T06:33:06.337 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:32:58.748865+0000", 2024-09-14T06:33:06.337 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 7.4952648679999996, 2024-09-14T06:33:06.338 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0074010889999999996, 2024-09-14T06:33:06.338 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:33:06.338 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:33:06.338 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:33:06.338 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:33:06.338 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:33:06.338 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:33:06.338 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 239927 2024-09-14T06:33:06.338 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:06.338 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:33:06.338 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:06.338 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:33:06.338 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:58.748865+0000", 2024-09-14T06:33:06.338 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:33:06.338 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:06.339 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:06.339 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:33:06.339 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:58.748865+0000", 2024-09-14T06:33:06.339 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:33:06.339 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:06.339 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:06.339 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:33:06.339 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:58.748867+0000", 2024-09-14T06:33:06.339 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.877e-06 2024-09-14T06:33:06.339 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:06.339 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:06.339 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:33:06.339 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:58.748874+0000", 2024-09-14T06:33:06.339 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.9110000000000004e-06 2024-09-14T06:33:06.339 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:06.340 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:06.340 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:33:06.340 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:58.748894+0000", 2024-09-14T06:33:06.340 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.0293e-05 2024-09-14T06:33:06.340 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:06.340 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:06.340 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:33:06.340 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:58.748904+0000", 2024-09-14T06:33:06.340 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.0395e-05 2024-09-14T06:33:06.340 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:06.340 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:06.340 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:33:06.340 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:58.748975+0000", 2024-09-14T06:33:06.341 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.0711999999999997e-05 2024-09-14T06:33:06.341 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:06.341 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:06.341 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "op must wait for map", 2024-09-14T06:33:06.341 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:58.748983+0000", 2024-09-14T06:33:06.341 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.7079999999999997e-06 2024-09-14T06:33:06.341 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:06.341 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:06.341 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:33:06.341 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:58.751692+0000", 2024-09-14T06:33:06.341 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0027086760000000001 2024-09-14T06:33:06.341 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:06.341 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:06.341 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:33:06.342 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:58.751828+0000", 2024-09-14T06:33:06.342 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00013688900000000001 2024-09-14T06:33:06.342 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:06.342 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:06.342 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:33:06.342 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:58.756266+0000", 2024-09-14T06:33:06.342 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0044376279999999999 2024-09-14T06:33:06.342 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:33:06.342 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:33:06.342 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:33:06.342 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:06.342 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:06.342 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:239960 3.a 3:518edc23:::rbd_data.112d92027590.0000000000000027:head [delete] snapc f=[] ondisk+write+known_if_redirected+supports_pool_eio e253)", 2024-09-14T06:33:06.342 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:32:58.759772+0000", 2024-09-14T06:33:06.342 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 7.4843580440000004, 2024-09-14T06:33:06.343 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0033111099999999999, 2024-09-14T06:33:06.343 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:33:06.343 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:33:06.343 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:33:06.343 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:33:06.343 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:33:06.343 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:33:06.343 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 239960 2024-09-14T06:33:06.343 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:06.343 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:33:06.343 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:06.343 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:33:06.343 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:58.759772+0000", 2024-09-14T06:33:06.343 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:33:06.343 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:06.344 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:06.344 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:33:06.344 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:58.759772+0000", 2024-09-14T06:33:06.344 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:33:06.344 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:06.344 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:06.344 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:33:06.344 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:58.759773+0000", 2024-09-14T06:33:06.344 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.85e-07 2024-09-14T06:33:06.344 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:06.344 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:06.344 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:33:06.344 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:58.759777+0000", 2024-09-14T06:33:06.344 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.0929999999999996e-06 2024-09-14T06:33:06.344 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:06.345 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:06.345 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:33:06.345 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:58.759785+0000", 2024-09-14T06:33:06.345 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.9209999999999992e-06 2024-09-14T06:33:06.345 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:06.345 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:06.345 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:33:06.345 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:58.759791+0000", 2024-09-14T06:33:06.345 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.2369999999999997e-06 2024-09-14T06:33:06.345 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:06.345 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:06.345 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:33:06.345 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:58.759987+0000", 2024-09-14T06:33:06.345 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00019611300000000001 2024-09-14T06:33:06.346 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:06.346 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:06.346 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:33:06.346 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:58.760119+0000", 2024-09-14T06:33:06.346 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00013234499999999999 2024-09-14T06:33:06.346 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:06.346 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:06.346 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:33:06.346 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:58.763083+0000", 2024-09-14T06:33:06.346 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0029635159999999998 2024-09-14T06:33:06.346 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:33:06.346 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:33:06.346 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:33:06.346 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:06.346 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:06.346 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:240324 3.a 3:535385f0:::rbd_children:head [call rbd.get_children in=15b] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e256)", 2024-09-14T06:33:06.347 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:33:01.618472+0000", 2024-09-14T06:33:06.347 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 4.6256577129999998, 2024-09-14T06:33:06.347 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.0267463640000001, 2024-09-14T06:33:06.347 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:33:06.347 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:33:06.347 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "reached pg", 2024-09-14T06:33:06.347 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:33:06.347 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:33:06.347 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:33:06.347 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 240324 2024-09-14T06:33:06.347 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:06.347 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:33:06.347 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:06.347 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:33:06.347 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:33:01.618472+0000", 2024-09-14T06:33:06.347 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:33:06.348 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:06.348 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:06.348 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:33:06.348 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:33:01.618472+0000", 2024-09-14T06:33:06.348 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:33:06.348 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:06.348 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:06.348 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:33:06.348 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:33:01.618474+0000", 2024-09-14T06:33:06.348 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.6750000000000001e-06 2024-09-14T06:33:06.348 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:06.348 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:06.348 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:33:06.348 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:33:01.618482+0000", 2024-09-14T06:33:06.349 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.8110000000000003e-06 2024-09-14T06:33:06.349 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:06.349 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:06.349 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:33:06.349 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:33:01.618506+0000", 2024-09-14T06:33:06.349 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.4037999999999999e-05 2024-09-14T06:33:06.349 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:06.349 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:06.349 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:33:06.349 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:33:01.618518+0000", 2024-09-14T06:33:06.349 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.2352e-05 2024-09-14T06:33:06.349 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:06.349 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:06.349 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:33:06.349 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:33:01.618610+0000", 2024-09-14T06:33:06.350 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.2021999999999994e-05 2024-09-14T06:33:06.350 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:06.350 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:06.350 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "op must wait for map", 2024-09-14T06:33:06.350 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:33:01.618621+0000", 2024-09-14T06:33:06.350 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.0753e-05 2024-09-14T06:33:06.350 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:06.350 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:06.350 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:33:06.350 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:33:01.621380+0000", 2024-09-14T06:33:06.350 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.002758768 2024-09-14T06:33:06.350 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:06.350 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:06.350 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "waiting for peered", 2024-09-14T06:33:06.351 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:33:01.621392+0000", 2024-09-14T06:33:06.351 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.2221e-05 2024-09-14T06:33:06.351 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:06.351 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:06.351 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:33:06.351 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:33:02.645082+0000", 2024-09-14T06:33:06.351 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.0236906210000001 2024-09-14T06:33:06.351 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:06.351 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:06.351 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:33:06.351 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:33:02.645219+0000", 2024-09-14T06:33:06.351 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00013610299999999999 2024-09-14T06:33:06.351 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:33:06.351 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:33:06.351 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:33:06.352 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:33:06.352 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:33:06.352 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-14T06:33:06.353 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-14T06:33:06.354 INFO:tasks.ceph.osd.5.smithi138.stderr:2024-09-14T06:33:06.352+0000 7fe13746f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33618) UID: 0 2024-09-14T06:33:06.454 INFO:tasks.ceph.osd.5.smithi138.stderr:2024-09-14T06:33:06.453+0000 7fe13746f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33618) UID: 0 2024-09-14T06:33:06.456 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-14T06:33:06.456 INFO:teuthology.orchestra.run.smithi094.stdout: "ops": [], 2024-09-14T06:33:06.456 INFO:teuthology.orchestra.run.smithi094.stdout: "num_ops": 0 2024-09-14T06:33:06.456 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-14T06:33:06.464 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-14T06:33:06.555 INFO:tasks.ceph.osd.2.smithi094.stderr:2024-09-14T06:33:06.554+0000 7faf3269b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33649) UID: 0 2024-09-14T06:33:06.566 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-14T06:33:06.567 INFO:teuthology.orchestra.run.smithi094.stdout: "ops": [], 2024-09-14T06:33:06.567 INFO:teuthology.orchestra.run.smithi094.stdout: "complaint_time": 30, 2024-09-14T06:33:06.567 INFO:teuthology.orchestra.run.smithi094.stdout: "num_blocked_ops": 0 2024-09-14T06:33:06.567 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-14T06:33:06.575 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-14T06:33:06.656 INFO:tasks.ceph.osd.7.smithi138.stderr:2024-09-14T06:33:06.655+0000 7f3c6a21d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33623) UID: 0 2024-09-14T06:33:06.690 INFO:teuthology.orchestra.run.smithi094.stderr:noscrub is unset 2024-09-14T06:33:06.694 INFO:teuthology.orchestra.run.smithi094.stderr:reweighted osd.0 to 1 (10000) 2024-09-14T06:33:06.713 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 1 1 2024-09-14T06:33:06.729 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-14T06:33:06.729 INFO:teuthology.orchestra.run.smithi094.stdout: "size": 20, 2024-09-14T06:33:06.729 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 600, 2024-09-14T06:33:06.729 INFO:teuthology.orchestra.run.smithi094.stdout: "ops": [ 2024-09-14T06:33:06.730 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:06.730 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:44426 3.2 3:4603ad7f:::rbd_data.112dca2f6ae1.0000000000000005:17 [read 892928~4096 [fadvise_nocache]] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:33:06.730 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:43.854977+0000", 2024-09-14T06:33:06.730 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 202.87236232500001, 2024-09-14T06:33:06.730 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.063369237999999994, 2024-09-14T06:33:06.730 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:33:06.730 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:33:06.730 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:33:06.730 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:33:06.730 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:33:06.730 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:33:06.730 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 44426 2024-09-14T06:33:06.730 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:06.730 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:33:06.731 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:06.731 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:33:06.731 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.854977+0000", 2024-09-14T06:33:06.731 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:33:06.731 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:06.731 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:06.731 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:33:06.731 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.854977+0000", 2024-09-14T06:33:06.731 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:33:06.732 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:06.732 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:06.732 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:33:06.732 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.854977+0000", 2024-09-14T06:33:06.732 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.1199999999999999e-07 2024-09-14T06:33:06.732 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:06.732 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:06.732 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:33:06.732 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.854980+0000", 2024-09-14T06:33:06.732 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.27e-06 2024-09-14T06:33:06.732 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:06.732 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:06.732 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:33:06.733 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.854983+0000", 2024-09-14T06:33:06.733 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.1650000000000002e-06 2024-09-14T06:33:06.733 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:06.733 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:06.733 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:33:06.733 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.854989+0000", 2024-09-14T06:33:06.733 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.6520000000000003e-06 2024-09-14T06:33:06.733 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:06.733 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:06.733 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:33:06.733 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.918143+0000", 2024-09-14T06:33:06.733 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.063154541999999994 2024-09-14T06:33:06.733 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:06.733 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:06.734 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:33:06.734 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.918209+0000", 2024-09-14T06:33:06.734 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.5529000000000002e-05 2024-09-14T06:33:06.734 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:06.734 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:06.734 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:33:06.734 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.918346+0000", 2024-09-14T06:33:06.734 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000136868 2024-09-14T06:33:06.734 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:33:06.734 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:33:06.734 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:33:06.734 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:06.734 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:06.734 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:44427 3.2 3:4603ad7f:::rbd_data.112dca2f6ae1.0000000000000005:17 [read 917504~4096 [fadvise_nocache]] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:33:06.735 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:43.854992+0000", 2024-09-14T06:33:06.735 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 202.872346679, 2024-09-14T06:33:06.735 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.063607792999999996, 2024-09-14T06:33:06.735 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:33:06.735 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:33:06.735 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:33:06.735 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:33:06.735 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:33:06.735 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:33:06.735 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 44427 2024-09-14T06:33:06.735 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:06.735 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:33:06.735 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:06.735 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:33:06.736 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.854992+0000", 2024-09-14T06:33:06.736 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:33:06.736 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:06.736 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:06.736 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:33:06.736 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.854992+0000", 2024-09-14T06:33:06.736 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:33:06.736 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:06.736 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:06.736 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:33:06.736 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.854993+0000", 2024-09-14T06:33:06.736 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.1400000000000001e-07 2024-09-14T06:33:06.736 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:06.737 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:06.737 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:33:06.737 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.854994+0000", 2024-09-14T06:33:06.737 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.091e-06 2024-09-14T06:33:06.737 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:06.737 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:06.737 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:33:06.737 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855001+0000", 2024-09-14T06:33:06.737 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.204e-06 2024-09-14T06:33:06.737 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:06.737 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:06.737 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:33:06.737 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855003+0000", 2024-09-14T06:33:06.737 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.514e-06 2024-09-14T06:33:06.738 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:06.738 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:06.738 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:33:06.738 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.918380+0000", 2024-09-14T06:33:06.738 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.063376738000000002 2024-09-14T06:33:06.738 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:06.738 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:06.738 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:33:06.738 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.918453+0000", 2024-09-14T06:33:06.738 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.3291e-05 2024-09-14T06:33:06.738 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:06.738 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:06.738 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:33:06.739 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.918600+0000", 2024-09-14T06:33:06.739 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00014674100000000001 2024-09-14T06:33:06.739 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:33:06.739 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:33:06.739 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:33:06.739 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:06.739 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:06.739 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:44428 3.2 3:4603ad7f:::rbd_data.112dca2f6ae1.0000000000000005:17 [read 942080~4096 [fadvise_nocache]] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:33:06.739 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:43.855006+0000", 2024-09-14T06:33:06.739 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 202.872332629, 2024-09-14T06:33:06.739 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.063846321999999997, 2024-09-14T06:33:06.739 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:33:06.739 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:33:06.739 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:33:06.740 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:33:06.740 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:33:06.740 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:33:06.740 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 44428 2024-09-14T06:33:06.740 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:06.740 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:33:06.740 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:06.740 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:33:06.740 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855006+0000", 2024-09-14T06:33:06.740 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:33:06.740 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:06.740 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:06.740 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:33:06.741 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855006+0000", 2024-09-14T06:33:06.741 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:33:06.741 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:06.741 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:06.741 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:33:06.741 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855007+0000", 2024-09-14T06:33:06.741 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.1199999999999999e-07 2024-09-14T06:33:06.741 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:06.741 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:06.741 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:33:06.741 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855011+0000", 2024-09-14T06:33:06.741 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.9539999999999998e-06 2024-09-14T06:33:06.741 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:06.741 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:06.742 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:33:06.742 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855014+0000", 2024-09-14T06:33:06.742 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.2930000000000001e-06 2024-09-14T06:33:06.742 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:06.742 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:06.742 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:33:06.742 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855016+0000", 2024-09-14T06:33:06.742 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.3039999999999999e-06 2024-09-14T06:33:06.742 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:06.742 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:06.742 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:33:06.742 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.918634+0000", 2024-09-14T06:33:06.742 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.063617928000000004 2024-09-14T06:33:06.742 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:06.743 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:06.743 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:33:06.743 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.918711+0000", 2024-09-14T06:33:06.743 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.6545000000000007e-05 2024-09-14T06:33:06.743 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:06.743 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:06.743 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:33:06.743 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.918853+0000", 2024-09-14T06:33:06.743 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00014208600000000001 2024-09-14T06:33:06.743 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:33:06.743 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:33:06.743 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:33:06.743 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:06.743 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:06.744 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:44429 3.2 3:4603ad7f:::rbd_data.112dca2f6ae1.0000000000000005:17 [read 966656~4096 [fadvise_nocache]] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:33:06.744 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:43.855019+0000", 2024-09-14T06:33:06.744 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 202.87231995600001, 2024-09-14T06:33:06.744 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.064059226999999996, 2024-09-14T06:33:06.744 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:33:06.744 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:33:06.744 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:33:06.744 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:33:06.744 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:33:06.744 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:33:06.744 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 44429 2024-09-14T06:33:06.744 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:06.744 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:33:06.745 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:06.745 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:33:06.745 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855019+0000", 2024-09-14T06:33:06.745 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:33:06.745 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:06.745 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:06.745 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:33:06.745 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855019+0000", 2024-09-14T06:33:06.745 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:33:06.745 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:06.745 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:06.745 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:33:06.745 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855019+0000", 2024-09-14T06:33:06.745 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.28e-07 2024-09-14T06:33:06.746 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:06.746 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:06.746 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:33:06.746 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855020+0000", 2024-09-14T06:33:06.746 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.0330000000000001e-06 2024-09-14T06:33:06.746 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:06.746 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:06.746 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:33:06.746 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855023+0000", 2024-09-14T06:33:06.746 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.79e-06 2024-09-14T06:33:06.746 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:06.746 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:06.746 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:33:06.747 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855025+0000", 2024-09-14T06:33:06.747 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.216e-06 2024-09-14T06:33:06.747 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:06.747 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:06.747 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:33:06.747 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.918886+0000", 2024-09-14T06:33:06.747 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.063861037999999995 2024-09-14T06:33:06.747 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:06.747 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:06.747 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:33:06.747 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.918952+0000", 2024-09-14T06:33:06.747 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.5993000000000003e-05 2024-09-14T06:33:06.747 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:06.747 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:06.747 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:33:06.748 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.919078+0000", 2024-09-14T06:33:06.748 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00012592899999999999 2024-09-14T06:33:06.748 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:33:06.748 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:33:06.748 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:33:06.748 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:06.748 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:06.748 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:44430 3.2 3:4603ad7f:::rbd_data.112dca2f6ae1.0000000000000005:17 [read 991232~4096 [fadvise_nocache]] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:33:06.748 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:43.855028+0000", 2024-09-14T06:33:06.748 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 202.872310946, 2024-09-14T06:33:06.748 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.064284346000000006, 2024-09-14T06:33:06.748 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:33:06.748 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:33:06.749 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:33:06.749 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:33:06.749 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:33:06.749 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:33:06.749 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 44430 2024-09-14T06:33:06.749 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:06.749 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:33:06.749 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:06.749 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:33:06.749 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855028+0000", 2024-09-14T06:33:06.749 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:33:06.749 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:06.749 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:06.749 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:33:06.750 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855028+0000", 2024-09-14T06:33:06.750 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:33:06.750 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:06.750 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:06.750 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:33:06.750 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855028+0000", 2024-09-14T06:33:06.750 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.1199999999999999e-07 2024-09-14T06:33:06.750 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:06.750 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:06.750 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:33:06.750 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855029+0000", 2024-09-14T06:33:06.750 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.1100000000000004e-07 2024-09-14T06:33:06.750 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:06.750 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:06.751 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:33:06.751 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855032+0000", 2024-09-14T06:33:06.751 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.7889999999999999e-06 2024-09-14T06:33:06.751 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:06.751 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:06.751 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:33:06.751 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855034+0000", 2024-09-14T06:33:06.751 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.125e-06 2024-09-14T06:33:06.751 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:06.751 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:06.751 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:33:06.751 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.919110+0000", 2024-09-14T06:33:06.751 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.064076413999999998 2024-09-14T06:33:06.751 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:06.751 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:06.752 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:33:06.752 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.919186+0000", 2024-09-14T06:33:06.752 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.5784999999999994e-05 2024-09-14T06:33:06.752 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:06.752 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:06.752 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:33:06.752 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.919312+0000", 2024-09-14T06:33:06.752 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00012611 2024-09-14T06:33:06.752 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:33:06.752 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:33:06.752 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:33:06.752 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:06.752 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:06.752 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:44431 3.2 3:4603ad7f:::rbd_data.112dca2f6ae1.0000000000000005:17 [read 1015808~4096 [fadvise_nocache]] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:33:06.753 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:43.855037+0000", 2024-09-14T06:33:06.753 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 202.87230207799999, 2024-09-14T06:33:06.753 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.064469755000000004, 2024-09-14T06:33:06.753 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:33:06.753 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:33:06.753 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:33:06.753 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:33:06.753 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:33:06.753 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:33:06.753 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 44431 2024-09-14T06:33:06.753 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:06.753 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:33:06.753 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:06.753 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:33:06.753 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855037+0000", 2024-09-14T06:33:06.754 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:33:06.754 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:06.754 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:06.754 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:33:06.754 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855037+0000", 2024-09-14T06:33:06.754 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:33:06.754 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:06.754 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:06.754 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:33:06.754 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855037+0000", 2024-09-14T06:33:06.754 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.1e-07 2024-09-14T06:33:06.754 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:06.754 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:06.754 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:33:06.755 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855038+0000", 2024-09-14T06:33:06.755 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.9400000000000004e-07 2024-09-14T06:33:06.755 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:06.755 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:06.755 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:33:06.755 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855041+0000", 2024-09-14T06:33:06.755 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.9639999999999999e-06 2024-09-14T06:33:06.755 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:06.755 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:06.755 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:33:06.755 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855043+0000", 2024-09-14T06:33:06.755 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.2259999999999999e-06 2024-09-14T06:33:06.755 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:06.755 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:06.755 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:33:06.756 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.919347+0000", 2024-09-14T06:33:06.756 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.064304091999999993 2024-09-14T06:33:06.756 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:06.756 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:06.756 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:33:06.756 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.919393+0000", 2024-09-14T06:33:06.756 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.5340999999999998e-05 2024-09-14T06:33:06.756 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:06.756 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:06.756 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:33:06.756 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.919507+0000", 2024-09-14T06:33:06.756 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000114028 2024-09-14T06:33:06.756 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:33:06.757 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:33:06.757 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:33:06.757 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:06.757 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:06.757 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:44432 3.2 3:4603ad7f:::rbd_data.112dca2f6ae1.0000000000000005:17 [read 1040384~4096 [fadvise_nocache]] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:33:06.757 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:43.855046+0000", 2024-09-14T06:33:06.757 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 202.87229300999999, 2024-09-14T06:33:06.757 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.064639253999999993, 2024-09-14T06:33:06.757 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:33:06.757 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:33:06.757 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:33:06.757 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:33:06.757 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:33:06.757 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:33:06.758 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 44432 2024-09-14T06:33:06.758 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:06.758 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:33:06.758 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:06.758 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:33:06.758 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855046+0000", 2024-09-14T06:33:06.758 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:33:06.758 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:06.758 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:06.758 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:33:06.758 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855046+0000", 2024-09-14T06:33:06.758 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:33:06.758 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:06.759 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:06.759 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:33:06.759 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855046+0000", 2024-09-14T06:33:06.759 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.05e-07 2024-09-14T06:33:06.759 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:06.759 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:06.759 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:33:06.759 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855047+0000", 2024-09-14T06:33:06.759 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.879999999999999e-07 2024-09-14T06:33:06.759 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:06.759 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:06.759 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:33:06.759 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855050+0000", 2024-09-14T06:33:06.759 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.8459999999999999e-06 2024-09-14T06:33:06.760 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:06.760 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:06.760 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:33:06.760 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855052+0000", 2024-09-14T06:33:06.760 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.136e-06 2024-09-14T06:33:06.760 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:06.760 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:06.760 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:33:06.760 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.919536+0000", 2024-09-14T06:33:06.760 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.064484132 2024-09-14T06:33:06.760 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:06.760 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:06.760 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:33:06.761 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.919611+0000", 2024-09-14T06:33:06.761 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.4407999999999996e-05 2024-09-14T06:33:06.761 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:06.761 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:06.761 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:33:06.761 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.919685+0000", 2024-09-14T06:33:06.761 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.4538999999999999e-05 2024-09-14T06:33:06.761 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:33:06.761 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:33:06.761 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:33:06.761 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:06.761 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:06.761 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:44518 3.2 3:4190d882:::rbd_data.112dca2f6ae1.0000000000000002:17 [read 12288~4096 [fadvise_nocache]] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:33:06.762 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:43.855055+0000", 2024-09-14T06:33:06.762 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 202.87228412600001, 2024-09-14T06:33:06.762 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.064894317000000007, 2024-09-14T06:33:06.762 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:33:06.762 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:33:06.762 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:33:06.762 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:33:06.762 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:33:06.762 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:33:06.762 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 44518 2024-09-14T06:33:06.762 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:06.762 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:33:06.762 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:06.762 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:33:06.763 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855055+0000", 2024-09-14T06:33:06.763 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:33:06.763 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:06.763 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:06.763 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:33:06.763 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855055+0000", 2024-09-14T06:33:06.763 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:33:06.763 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:06.763 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:06.763 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:33:06.763 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855055+0000", 2024-09-14T06:33:06.763 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.0699999999999999e-07 2024-09-14T06:33:06.763 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:06.764 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:06.764 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:33:06.764 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855056+0000", 2024-09-14T06:33:06.764 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.9400000000000004e-07 2024-09-14T06:33:06.764 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:06.764 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:06.764 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:33:06.764 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855059+0000", 2024-09-14T06:33:06.764 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.8870000000000001e-06 2024-09-14T06:33:06.764 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:06.764 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:06.764 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:33:06.764 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855061+0000", 2024-09-14T06:33:06.764 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.1919999999999999e-06 2024-09-14T06:33:06.765 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:06.765 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:06.765 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:33:06.765 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.919719+0000", 2024-09-14T06:33:06.765 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.064658013 2024-09-14T06:33:06.765 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:06.765 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:06.765 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:33:06.765 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.919788+0000", 2024-09-14T06:33:06.765 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.9282000000000005e-05 2024-09-14T06:33:06.765 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:06.765 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:06.766 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:33:06.766 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.919949+0000", 2024-09-14T06:33:06.766 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00016084199999999999 2024-09-14T06:33:06.766 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:33:06.766 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:33:06.766 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:33:06.766 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:06.766 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:06.766 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:44519 3.2 3:4190d882:::rbd_data.112dca2f6ae1.0000000000000002:17 [read 36864~4096 [fadvise_nocache]] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:33:06.766 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:43.855064+0000", 2024-09-14T06:33:06.766 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 202.87227514899999, 2024-09-14T06:33:06.766 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.065081865000000003, 2024-09-14T06:33:06.766 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:33:06.766 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:33:06.767 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:33:06.767 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:33:06.767 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:33:06.767 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:33:06.767 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 44519 2024-09-14T06:33:06.767 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:06.767 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:33:06.767 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:06.767 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:33:06.767 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855064+0000", 2024-09-14T06:33:06.767 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:33:06.767 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:06.767 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:06.767 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:33:06.768 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855064+0000", 2024-09-14T06:33:06.768 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:33:06.768 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:06.768 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:06.768 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:33:06.768 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855064+0000", 2024-09-14T06:33:06.768 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.1e-07 2024-09-14T06:33:06.768 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:06.768 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:06.768 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:33:06.768 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855065+0000", 2024-09-14T06:33:06.768 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.035e-06 2024-09-14T06:33:06.768 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:06.768 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:06.769 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:33:06.769 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855068+0000", 2024-09-14T06:33:06.769 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.8339999999999999e-06 2024-09-14T06:33:06.769 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:06.769 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:06.769 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:33:06.769 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855071+0000", 2024-09-14T06:33:06.769 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.3450000000000002e-06 2024-09-14T06:33:06.769 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:06.769 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:06.769 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:33:06.769 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.919979+0000", 2024-09-14T06:33:06.769 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.064907647999999998 2024-09-14T06:33:06.769 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:06.770 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:06.770 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:33:06.770 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.920026+0000", 2024-09-14T06:33:06.770 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.7595000000000001e-05 2024-09-14T06:33:06.770 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:06.770 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:06.770 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:33:06.770 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.920146+0000", 2024-09-14T06:33:06.770 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00011919800000000001 2024-09-14T06:33:06.770 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:33:06.770 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:33:06.770 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:33:06.770 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:06.771 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:06.771 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:44520 3.2 3:4190d882:::rbd_data.112dca2f6ae1.0000000000000002:17 [read 61440~4096 [fadvise_nocache]] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:33:06.771 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:43.855075+0000", 2024-09-14T06:33:06.771 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 202.87226409799999, 2024-09-14T06:33:06.771 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.065335078000000005, 2024-09-14T06:33:06.771 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:33:06.771 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:33:06.771 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:33:06.771 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:33:06.771 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:33:06.771 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:33:06.771 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 44520 2024-09-14T06:33:06.771 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:06.771 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:33:06.772 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:06.772 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:33:06.772 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855075+0000", 2024-09-14T06:33:06.772 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:33:06.772 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:06.772 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:06.772 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:33:06.772 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855075+0000", 2024-09-14T06:33:06.772 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:33:06.772 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:06.772 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:06.772 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:33:06.772 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855075+0000", 2024-09-14T06:33:06.773 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.1e-07 2024-09-14T06:33:06.773 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:06.773 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:06.773 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:33:06.773 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855076+0000", 2024-09-14T06:33:06.773 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.0839999999999999e-06 2024-09-14T06:33:06.773 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:06.773 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:06.773 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:33:06.773 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855079+0000", 2024-09-14T06:33:06.773 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.9359999999999999e-06 2024-09-14T06:33:06.773 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:06.773 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:06.774 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:33:06.774 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855084+0000", 2024-09-14T06:33:06.774 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.1730000000000003e-06 2024-09-14T06:33:06.774 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:06.774 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:06.774 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:33:06.774 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.920175+0000", 2024-09-14T06:33:06.774 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.065090234999999996 2024-09-14T06:33:06.774 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:06.774 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:06.774 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:33:06.774 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.920250+0000", 2024-09-14T06:33:06.774 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.5807000000000001e-05 2024-09-14T06:33:06.774 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:06.774 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:06.775 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:33:06.775 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.920410+0000", 2024-09-14T06:33:06.775 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00015963300000000001 2024-09-14T06:33:06.775 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:33:06.775 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:33:06.775 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:33:06.775 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:06.775 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:06.775 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:44521 3.2 3:4190d882:::rbd_data.112dca2f6ae1.0000000000000002:17 [read 86016~4096 [fadvise_nocache]] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:33:06.775 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:43.855087+0000", 2024-09-14T06:33:06.775 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 202.87225161699999, 2024-09-14T06:33:06.775 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.065563264999999996, 2024-09-14T06:33:06.775 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:33:06.775 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:33:06.776 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:33:06.776 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:33:06.776 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:33:06.776 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:33:06.776 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 44521 2024-09-14T06:33:06.776 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:06.776 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:33:06.776 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:06.776 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:33:06.776 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855087+0000", 2024-09-14T06:33:06.776 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:33:06.776 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:06.776 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:06.777 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:33:06.777 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855087+0000", 2024-09-14T06:33:06.777 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:33:06.777 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:06.777 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:06.777 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:33:06.777 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855088+0000", 2024-09-14T06:33:06.777 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.3699999999999999e-07 2024-09-14T06:33:06.777 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:06.777 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:06.777 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:33:06.777 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855089+0000", 2024-09-14T06:33:06.777 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 9.9900000000000009e-07 2024-09-14T06:33:06.778 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:06.778 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:06.778 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:33:06.778 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855091+0000", 2024-09-14T06:33:06.778 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.8940000000000002e-06 2024-09-14T06:33:06.778 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:06.778 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:06.778 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:33:06.778 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855094+0000", 2024-09-14T06:33:06.778 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.351e-06 2024-09-14T06:33:06.778 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:06.778 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:06.778 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:33:06.778 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.920437+0000", 2024-09-14T06:33:06.779 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.065343221000000007 2024-09-14T06:33:06.779 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:06.779 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:06.779 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:33:06.779 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.920481+0000", 2024-09-14T06:33:06.779 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.4431000000000002e-05 2024-09-14T06:33:06.779 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:06.779 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:06.779 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:33:06.779 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.920651+0000", 2024-09-14T06:33:06.779 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00016913200000000001 2024-09-14T06:33:06.779 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:33:06.779 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:33:06.779 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:33:06.780 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:06.780 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:06.780 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:44522 3.2 3:4190d882:::rbd_data.112dca2f6ae1.0000000000000002:17 [read 110592~4096 [fadvise_nocache]] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:33:06.780 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:43.855098+0000", 2024-09-14T06:33:06.780 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 202.87224139, 2024-09-14T06:33:06.780 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.065827628999999999, 2024-09-14T06:33:06.780 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:33:06.780 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:33:06.780 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:33:06.780 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:33:06.780 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:33:06.781 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:33:06.781 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 44522 2024-09-14T06:33:06.781 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:06.781 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:33:06.781 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:06.781 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:33:06.781 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855098+0000", 2024-09-14T06:33:06.781 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:33:06.781 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:06.781 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:06.781 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:33:06.781 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855098+0000", 2024-09-14T06:33:06.782 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:33:06.782 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:06.782 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:06.782 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:33:06.782 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855098+0000", 2024-09-14T06:33:06.782 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.1e-07 2024-09-14T06:33:06.782 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:06.782 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:06.782 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:33:06.782 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855099+0000", 2024-09-14T06:33:06.782 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.012e-06 2024-09-14T06:33:06.782 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:06.782 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:06.782 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:33:06.783 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855102+0000", 2024-09-14T06:33:06.783 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.9380000000000001e-06 2024-09-14T06:33:06.783 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:06.783 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:06.783 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:33:06.783 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855108+0000", 2024-09-14T06:33:06.783 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.4339999999999997e-06 2024-09-14T06:33:06.783 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:06.783 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:06.783 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:33:06.783 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.920681+0000", 2024-09-14T06:33:06.783 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.065572943999999994 2024-09-14T06:33:06.783 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:06.783 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:06.784 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:33:06.784 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.920724+0000", 2024-09-14T06:33:06.784 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.2877000000000003e-05 2024-09-14T06:33:06.784 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:06.784 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:06.784 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:33:06.784 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.920925+0000", 2024-09-14T06:33:06.784 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00020121399999999999 2024-09-14T06:33:06.784 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:33:06.784 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:33:06.784 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:33:06.784 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:06.784 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:06.784 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:44523 3.2 3:4190d882:::rbd_data.112dca2f6ae1.0000000000000002:17 [read 135168~4096 [fadvise_nocache]] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:33:06.785 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:43.855111+0000", 2024-09-14T06:33:06.785 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 202.87222766799999, 2024-09-14T06:33:06.785 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.066028198999999996, 2024-09-14T06:33:06.785 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:33:06.785 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:33:06.785 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:33:06.785 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:33:06.785 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:33:06.785 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:33:06.785 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 44523 2024-09-14T06:33:06.785 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:06.785 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:33:06.785 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:06.785 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:33:06.786 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855111+0000", 2024-09-14T06:33:06.786 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:33:06.786 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:06.786 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:06.786 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:33:06.786 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855111+0000", 2024-09-14T06:33:06.786 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:33:06.786 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:06.786 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:06.786 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:33:06.786 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855112+0000", 2024-09-14T06:33:06.786 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.1199999999999999e-07 2024-09-14T06:33:06.786 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:06.786 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:06.787 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:33:06.787 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855115+0000", 2024-09-14T06:33:06.787 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.3380000000000001e-06 2024-09-14T06:33:06.787 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:06.787 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:06.787 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:33:06.787 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855118+0000", 2024-09-14T06:33:06.787 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.366e-06 2024-09-14T06:33:06.787 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:06.787 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:06.787 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:33:06.787 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855121+0000", 2024-09-14T06:33:06.787 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.8329999999999998e-06 2024-09-14T06:33:06.788 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:06.788 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:06.788 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:33:06.788 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.920956+0000", 2024-09-14T06:33:06.788 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.065835251999999997 2024-09-14T06:33:06.788 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:06.788 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:06.788 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:33:06.788 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.921001+0000", 2024-09-14T06:33:06.788 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.4778999999999999e-05 2024-09-14T06:33:06.788 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:06.788 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:06.788 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:33:06.788 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.921139+0000", 2024-09-14T06:33:06.788 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000138419 2024-09-14T06:33:06.789 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:33:06.789 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:33:06.789 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:33:06.789 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:06.789 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:06.789 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:44524 3.2 3:4190d882:::rbd_data.112dca2f6ae1.0000000000000002:17 [read 159744~4096 [fadvise_nocache]] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:33:06.789 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:43.855124+0000", 2024-09-14T06:33:06.789 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 202.872214689, 2024-09-14T06:33:06.789 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.066249261000000004, 2024-09-14T06:33:06.789 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:33:06.789 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:33:06.789 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:33:06.789 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:33:06.789 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:33:06.790 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:33:06.790 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 44524 2024-09-14T06:33:06.790 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:06.790 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:33:06.790 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:06.790 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:33:06.790 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855124+0000", 2024-09-14T06:33:06.790 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:33:06.790 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:06.790 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:06.790 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:33:06.790 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.855124+0000", 2024-09-14T06:33:06.790 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:33:06.790 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:06.791 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:06.791 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:33:06.791 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861086+0000", 2024-09-14T06:33:06.791 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0059613829999999998 2024-09-14T06:33:06.791 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:06.791 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:06.791 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:33:06.791 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861100+0000", 2024-09-14T06:33:06.791 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.4644000000000001e-05 2024-09-14T06:33:06.791 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:06.791 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:06.791 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:33:06.791 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861109+0000", 2024-09-14T06:33:06.791 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.541e-06 2024-09-14T06:33:06.792 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:06.792 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:06.792 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:33:06.792 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861117+0000", 2024-09-14T06:33:06.792 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.1049999999999995e-06 2024-09-14T06:33:06.792 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:06.792 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:06.792 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:33:06.792 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.921169+0000", 2024-09-14T06:33:06.792 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.060052405000000003 2024-09-14T06:33:06.792 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:06.792 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:06.792 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:33:06.792 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.921213+0000", 2024-09-14T06:33:06.793 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.3565e-05 2024-09-14T06:33:06.793 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:06.793 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:06.793 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:33:06.793 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.921374+0000", 2024-09-14T06:33:06.793 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00016061800000000001 2024-09-14T06:33:06.793 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:33:06.793 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:33:06.793 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:33:06.793 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:06.793 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:06.793 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:44537 3.2 3:4190d882:::rbd_data.112dca2f6ae1.0000000000000002:17 [read 479232~4096 [fadvise_nocache]] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:33:06.793 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:43.861405+0000", 2024-09-14T06:33:06.793 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 202.86593408600001, 2024-09-14T06:33:06.794 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.063351795000000002, 2024-09-14T06:33:06.794 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:33:06.794 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:33:06.794 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:33:06.794 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:33:06.794 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:33:06.794 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:33:06.794 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 44537 2024-09-14T06:33:06.794 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:06.794 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:33:06.794 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:06.794 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:33:06.794 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861405+0000", 2024-09-14T06:33:06.794 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:33:06.795 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:06.795 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:06.795 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:33:06.795 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861405+0000", 2024-09-14T06:33:06.795 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:33:06.795 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:06.795 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:06.795 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:33:06.795 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861405+0000", 2024-09-14T06:33:06.795 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.9099999999999999e-07 2024-09-14T06:33:06.795 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:06.795 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:06.795 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:33:06.796 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861407+0000", 2024-09-14T06:33:06.796 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.075e-06 2024-09-14T06:33:06.796 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:06.796 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:06.796 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:33:06.796 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861414+0000", 2024-09-14T06:33:06.796 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.4659999999999996e-06 2024-09-14T06:33:06.796 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:06.796 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:06.796 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:33:06.796 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861418+0000", 2024-09-14T06:33:06.796 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.5519999999999998e-06 2024-09-14T06:33:06.796 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:06.796 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:06.797 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:33:06.797 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.924474+0000", 2024-09-14T06:33:06.797 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.063056012999999994 2024-09-14T06:33:06.797 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:06.797 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:06.797 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:33:06.797 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.924556+0000", 2024-09-14T06:33:06.797 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.1181999999999996e-05 2024-09-14T06:33:06.797 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:06.797 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:06.797 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:33:06.797 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.924757+0000", 2024-09-14T06:33:06.797 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00020111599999999999 2024-09-14T06:33:06.797 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:33:06.798 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:33:06.798 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:33:06.798 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:06.798 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:06.798 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:44538 3.2 3:4190d882:::rbd_data.112dca2f6ae1.0000000000000002:17 [read 503808~4096 [fadvise_nocache]] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:33:06.798 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:43.861425+0000", 2024-09-14T06:33:06.798 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 202.86591404500001, 2024-09-14T06:33:06.798 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.063627125000000007, 2024-09-14T06:33:06.798 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:33:06.798 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:33:06.798 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:33:06.798 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:33:06.798 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:33:06.799 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:33:06.799 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 44538 2024-09-14T06:33:06.799 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:06.799 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:33:06.799 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:06.799 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:33:06.799 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861425+0000", 2024-09-14T06:33:06.799 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:33:06.799 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:06.799 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:06.799 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:33:06.799 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861425+0000", 2024-09-14T06:33:06.799 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:33:06.799 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:06.800 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:06.800 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:33:06.800 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861425+0000", 2024-09-14T06:33:06.800 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.9999999999999998e-07 2024-09-14T06:33:06.800 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:06.800 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:06.800 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:33:06.800 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861427+0000", 2024-09-14T06:33:06.800 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.9829999999999999e-06 2024-09-14T06:33:06.800 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:06.800 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:06.800 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:33:06.800 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861433+0000", 2024-09-14T06:33:06.800 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.2889999999999997e-06 2024-09-14T06:33:06.801 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:06.801 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:06.801 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:33:06.801 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861454+0000", 2024-09-14T06:33:06.801 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.1844999999999998e-05 2024-09-14T06:33:06.801 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:06.801 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:06.801 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:33:06.801 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.924786+0000", 2024-09-14T06:33:06.801 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.063331497 2024-09-14T06:33:06.801 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:06.801 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:06.801 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:33:06.801 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.924864+0000", 2024-09-14T06:33:06.802 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.7744000000000006e-05 2024-09-14T06:33:06.802 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:06.802 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:06.802 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:33:06.802 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.925052+0000", 2024-09-14T06:33:06.802 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00018836699999999999 2024-09-14T06:33:06.802 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:33:06.802 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:33:06.802 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:33:06.802 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:06.802 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:06.802 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:44539 3.2 3:4190d882:::rbd_data.112dca2f6ae1.0000000000000002:17 [read 528384~4096 [fadvise_nocache]] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:33:06.802 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:43.861462+0000", 2024-09-14T06:33:06.802 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 202.86587671800001, 2024-09-14T06:33:06.803 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.063855705999999998, 2024-09-14T06:33:06.803 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:33:06.803 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:33:06.803 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:33:06.803 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:33:06.803 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:33:06.803 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:33:06.803 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 44539 2024-09-14T06:33:06.803 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:06.803 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:33:06.803 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:06.803 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:33:06.803 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861462+0000", 2024-09-14T06:33:06.803 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:33:06.804 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:06.804 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:06.804 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:33:06.804 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861462+0000", 2024-09-14T06:33:06.804 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:33:06.804 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:06.804 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:06.804 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:33:06.804 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861463+0000", 2024-09-14T06:33:06.804 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.5200000000000002e-07 2024-09-14T06:33:06.804 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:06.804 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:06.804 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:33:06.805 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861465+0000", 2024-09-14T06:33:06.805 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 2.5189999999999999e-06 2024-09-14T06:33:06.805 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:06.805 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:06.805 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:33:06.805 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861471+0000", 2024-09-14T06:33:06.805 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.6899999999999997e-06 2024-09-14T06:33:06.805 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:06.805 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:06.805 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:33:06.805 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861475+0000", 2024-09-14T06:33:06.805 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 4.5730000000000001e-06 2024-09-14T06:33:06.805 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:06.805 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:06.806 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:33:06.806 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.925085+0000", 2024-09-14T06:33:06.806 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.063609902999999995 2024-09-14T06:33:06.806 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:06.806 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:06.806 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:33:06.806 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.925145+0000", 2024-09-14T06:33:06.806 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.9383999999999999e-05 2024-09-14T06:33:06.806 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:06.806 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:06.806 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:33:06.806 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.925318+0000", 2024-09-14T06:33:06.806 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00017318499999999999 2024-09-14T06:33:06.806 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:33:06.807 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:33:06.807 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:33:06.807 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:06.807 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:06.807 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:44540 3.2 3:4190d882:::rbd_data.112dca2f6ae1.0000000000000002:17 [read 552960~4096 [fadvise_nocache]] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:33:06.807 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:29:43.861483+0000", 2024-09-14T06:33:06.807 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 202.86585628700001, 2024-09-14T06:33:06.807 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.064065681999999999, 2024-09-14T06:33:06.807 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:33:06.807 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:33:06.807 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:33:06.807 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:33:06.807 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:33:06.807 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:33:06.808 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 44540 2024-09-14T06:33:06.808 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:06.808 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:33:06.808 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:06.808 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:33:06.808 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861483+0000", 2024-09-14T06:33:06.808 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:33:06.808 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:06.808 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:06.808 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:33:06.808 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.861483+0000", 2024-09-14T06:33:06.808 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:33:06.808 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:06.808 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:06.809 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:33:06.809 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.867079+0000", 2024-09-14T06:33:06.809 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0055966480000000001 2024-09-14T06:33:06.809 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:06.809 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:06.809 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:33:06.809 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.867086+0000", 2024-09-14T06:33:06.809 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.387e-06 2024-09-14T06:33:06.809 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:06.809 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:06.809 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:33:06.809 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.867094+0000", 2024-09-14T06:33:06.809 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.9419999999999995e-06 2024-09-14T06:33:06.810 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:06.810 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:06.810 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:33:06.810 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.867109+0000", 2024-09-14T06:33:06.810 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.5818999999999999e-05 2024-09-14T06:33:06.810 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:06.810 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:06.810 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:33:06.810 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.925354+0000", 2024-09-14T06:33:06.810 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.058244047 2024-09-14T06:33:06.810 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:06.810 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:06.810 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:33:06.810 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.925406+0000", 2024-09-14T06:33:06.811 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.2661999999999998e-05 2024-09-14T06:33:06.811 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:06.811 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:06.811 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:33:06.811 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:29:43.925548+0000", 2024-09-14T06:33:06.811 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000142177 2024-09-14T06:33:06.811 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:33:06.811 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:33:06.811 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:33:06.811 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:06.811 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:06.811 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:225587 3.7 3:e112e2b3:::rbd_id.image_client.0-clone42:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e189)", 2024-09-14T06:33:06.811 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:31:53.874591+0000", 2024-09-14T06:33:06.811 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 72.852748070000004, 2024-09-14T06:33:06.812 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.875541507, 2024-09-14T06:33:06.812 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:33:06.812 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:33:06.812 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "started", 2024-09-14T06:33:06.812 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:33:06.812 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:33:06.812 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:33:06.812 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 225587 2024-09-14T06:33:06.812 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:06.812 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:33:06.812 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:06.812 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:33:06.812 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:53.874591+0000", 2024-09-14T06:33:06.812 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:33:06.812 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:06.813 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:06.813 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:33:06.813 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:53.874591+0000", 2024-09-14T06:33:06.813 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:33:06.813 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:06.813 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:06.813 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:33:06.813 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:53.874592+0000", 2024-09-14T06:33:06.813 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.5209999999999999e-06 2024-09-14T06:33:06.813 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:06.813 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:06.813 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:33:06.813 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:53.874599+0000", 2024-09-14T06:33:06.813 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.6290000000000004e-06 2024-09-14T06:33:06.814 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:06.814 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:06.814 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:33:06.814 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:53.874618+0000", 2024-09-14T06:33:06.814 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.8950999999999999e-05 2024-09-14T06:33:06.814 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:06.814 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:06.814 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:33:06.814 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:53.874626+0000", 2024-09-14T06:33:06.814 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 8.3960000000000006e-06 2024-09-14T06:33:06.814 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:06.814 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:06.814 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:33:06.814 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:53.874699+0000", 2024-09-14T06:33:06.815 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.2329999999999994e-05 2024-09-14T06:33:06.815 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:06.815 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:06.815 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "waiting for peered", 2024-09-14T06:33:06.815 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:53.874711+0000", 2024-09-14T06:33:06.815 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.2473999999999999e-05 2024-09-14T06:33:06.815 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:06.815 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:06.815 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:33:06.815 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:54.749578+0000", 2024-09-14T06:33:06.815 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.87486642100000001 2024-09-14T06:33:06.815 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:06.815 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:06.815 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:33:06.816 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:54.749706+0000", 2024-09-14T06:33:06.816 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00012856200000000001 2024-09-14T06:33:06.816 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:06.816 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:06.816 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:33:06.816 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:31:54.750132+0000", 2024-09-14T06:33:06.816 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00042622299999999999 2024-09-14T06:33:06.816 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:33:06.816 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:33:06.816 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:33:06.816 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:06.816 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:06.816 INFO:teuthology.orchestra.run.smithi094.stdout: "description": "osd_op(client.4397.0:233119 3.1e 3:7bae5948:::rbd_data.112df8d32353.000000000000000f:head [delete] snapc 53=[] ondisk+write+known_if_redirected+supports_pool_eio e211)", 2024-09-14T06:33:06.816 INFO:teuthology.orchestra.run.smithi094.stdout: "initiated_at": "2024-09-14T06:32:16.760588+0000", 2024-09-14T06:33:06.817 INFO:teuthology.orchestra.run.smithi094.stdout: "age": 49.966751082999998, 2024-09-14T06:33:06.817 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.92349815700000004, 2024-09-14T06:33:06.817 INFO:teuthology.orchestra.run.smithi094.stdout: "continuous": false, 2024-09-14T06:33:06.817 INFO:teuthology.orchestra.run.smithi094.stdout: "type_data": { 2024-09-14T06:33:06.817 INFO:teuthology.orchestra.run.smithi094.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:33:06.817 INFO:teuthology.orchestra.run.smithi094.stdout: "client_info": { 2024-09-14T06:33:06.817 INFO:teuthology.orchestra.run.smithi094.stdout: "client": "client.4397", 2024-09-14T06:33:06.817 INFO:teuthology.orchestra.run.smithi094.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:33:06.817 INFO:teuthology.orchestra.run.smithi094.stdout: "tid": 233119 2024-09-14T06:33:06.817 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:06.817 INFO:teuthology.orchestra.run.smithi094.stdout: "events": [ 2024-09-14T06:33:06.817 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:06.817 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "initiated", 2024-09-14T06:33:06.817 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:16.760588+0000", 2024-09-14T06:33:06.818 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:33:06.818 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:06.818 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:06.818 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "header_read", 2024-09-14T06:33:06.818 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:16.760588+0000", 2024-09-14T06:33:06.818 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0 2024-09-14T06:33:06.818 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:06.818 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:06.818 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "throttled", 2024-09-14T06:33:06.818 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:16.760588+0000", 2024-09-14T06:33:06.818 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.3900000000000005e-07 2024-09-14T06:33:06.818 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:06.818 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:06.818 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "all_read", 2024-09-14T06:33:06.818 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:16.760591+0000", 2024-09-14T06:33:06.819 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 3.0070000000000002e-06 2024-09-14T06:33:06.819 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:06.819 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:06.819 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "dispatched", 2024-09-14T06:33:06.819 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:16.760598+0000", 2024-09-14T06:33:06.819 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 7.0509999999999999e-06 2024-09-14T06:33:06.819 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:06.819 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:06.819 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "queued_for_pg", 2024-09-14T06:33:06.819 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:16.760605+0000", 2024-09-14T06:33:06.819 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.0310000000000004e-06 2024-09-14T06:33:06.819 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:06.819 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:06.819 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:33:06.820 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:16.760668+0000", 2024-09-14T06:33:06.820 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 6.3699000000000001e-05 2024-09-14T06:33:06.820 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:06.820 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:06.820 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "waiting for peered", 2024-09-14T06:33:06.820 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:16.760678+0000", 2024-09-14T06:33:06.820 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 1.0122999999999999e-05 2024-09-14T06:33:06.820 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:06.820 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:06.820 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "reached_pg", 2024-09-14T06:33:06.820 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:17.681459+0000", 2024-09-14T06:33:06.820 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.92078067399999997 2024-09-14T06:33:06.820 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:06.820 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:06.821 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "started", 2024-09-14T06:33:06.821 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:17.681700+0000", 2024-09-14T06:33:06.821 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00024128800000000001 2024-09-14T06:33:06.821 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:06.821 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:06.821 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "waiting for subops from 2", 2024-09-14T06:33:06.821 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:17.681860+0000", 2024-09-14T06:33:06.821 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00015995500000000001 2024-09-14T06:33:06.821 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:06.821 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:06.821 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:33:06.821 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:17.683399+0000", 2024-09-14T06:33:06.821 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.0015388209999999999 2024-09-14T06:33:06.821 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:06.822 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:06.822 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "op_commit", 2024-09-14T06:33:06.822 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:17.683687+0000", 2024-09-14T06:33:06.822 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.00028772299999999998 2024-09-14T06:33:06.822 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:06.822 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:06.822 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "commit_sent", 2024-09-14T06:33:06.822 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:17.684032+0000", 2024-09-14T06:33:06.822 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 0.000344836 2024-09-14T06:33:06.822 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-09-14T06:33:06.822 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-09-14T06:33:06.822 INFO:teuthology.orchestra.run.smithi094.stdout: "event": "done", 2024-09-14T06:33:06.822 INFO:teuthology.orchestra.run.smithi094.stdout: "time": "2024-09-14T06:32:17.684086+0000", 2024-09-14T06:33:06.822 INFO:teuthology.orchestra.run.smithi094.stdout: "duration": 5.4410000000000003e-05 2024-09-14T06:33:06.822 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:33:06.823 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:33:06.823 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:33:06.823 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-09-14T06:33:06.823 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-09-14T06:33:06.823 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-14T06:33:06.825 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-14T06:33:06.825 INFO:tasks.ceph.osd.6.smithi138.stderr:2024-09-14T06:33:06.823+0000 7f8cd8849640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33620) UID: 0 2024-09-14T06:33:06.925 INFO:tasks.ceph.osd.5.smithi138.stderr:2024-09-14T06:33:06.924+0000 7fe13746f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33618) UID: 0 2024-09-14T06:33:06.930 INFO:teuthology.orchestra.run.smithi138.stdout:{ 2024-09-14T06:33:06.930 INFO:teuthology.orchestra.run.smithi138.stdout: "ops": [], 2024-09-14T06:33:06.930 INFO:teuthology.orchestra.run.smithi138.stdout: "num_ops": 0 2024-09-14T06:33:06.930 INFO:teuthology.orchestra.run.smithi138.stdout:} 2024-09-14T06:33:06.939 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-14T06:33:07.027 INFO:tasks.ceph.osd.5.smithi138.stderr:2024-09-14T06:33:07.026+0000 7fe13746f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33618) UID: 0 2024-09-14T06:33:07.082 INFO:teuthology.orchestra.run.smithi138.stdout:{ 2024-09-14T06:33:07.082 INFO:teuthology.orchestra.run.smithi138.stdout: "ops": [], 2024-09-14T06:33:07.082 INFO:teuthology.orchestra.run.smithi138.stdout: "complaint_time": 30, 2024-09-14T06:33:07.082 INFO:teuthology.orchestra.run.smithi138.stdout: "num_blocked_ops": 0 2024-09-14T06:33:07.082 INFO:teuthology.orchestra.run.smithi138.stdout:} 2024-09-14T06:33:07.090 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-14T06:33:07.131 INFO:tasks.ceph.osd.6.smithi138.stderr:2024-09-14T06:33:07.130+0000 7f8cd8849640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33620) UID: 0 2024-09-14T06:33:07.228 INFO:tasks.ceph.osd.1.smithi094.stderr:2024-09-14T06:33:07.227+0000 7f328e47a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33641) UID: 0 2024-09-14T06:33:07.236 INFO:teuthology.orchestra.run.smithi138.stdout:{ 2024-09-14T06:33:07.236 INFO:teuthology.orchestra.run.smithi138.stdout: "size": 20, 2024-09-14T06:33:07.236 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 600, 2024-09-14T06:33:07.236 INFO:teuthology.orchestra.run.smithi138.stdout: "ops": [ 2024-09-14T06:33:07.236 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:07.236 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:19328 3.3 3:c34e5e51:::rbd_data.112d92027590.0000000000000024:f [read 1926144~1024 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:33:07.236 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:27.059768+0000", 2024-09-14T06:33:07.236 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 220.17458353999999, 2024-09-14T06:33:07.236 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046369147999999999, 2024-09-14T06:33:07.236 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:33:07.237 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:33:07.237 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:33:07.237 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:33:07.237 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:33:07.237 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:33:07.237 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 19328 2024-09-14T06:33:07.237 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:07.237 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:33:07.237 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:07.237 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:33:07.237 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.059768+0000", 2024-09-14T06:33:07.237 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:33:07.237 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:07.238 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:07.238 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:33:07.238 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.059768+0000", 2024-09-14T06:33:07.238 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:33:07.238 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:07.238 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:07.238 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:33:07.238 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.059874+0000", 2024-09-14T06:33:07.238 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00010551400000000001 2024-09-14T06:33:07.238 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:07.238 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:07.238 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:33:07.238 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.059877+0000", 2024-09-14T06:33:07.238 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.8279999999999999e-06 2024-09-14T06:33:07.239 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:07.239 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:07.239 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:33:07.239 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.059884+0000", 2024-09-14T06:33:07.239 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.6389999999999992e-06 2024-09-14T06:33:07.239 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:07.239 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:07.239 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:33:07.239 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.059890+0000", 2024-09-14T06:33:07.239 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.49e-06 2024-09-14T06:33:07.239 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:07.239 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:07.239 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:33:07.239 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.105804+0000", 2024-09-14T06:33:07.240 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.045914242000000001 2024-09-14T06:33:07.240 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:07.240 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:07.240 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:33:07.240 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.105886+0000", 2024-09-14T06:33:07.240 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.2305999999999994e-05 2024-09-14T06:33:07.240 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:07.240 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:07.240 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:33:07.240 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.106138+0000", 2024-09-14T06:33:07.240 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.000251129 2024-09-14T06:33:07.240 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:33:07.240 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:33:07.240 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:33:07.241 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:07.241 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:07.241 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:19329 3.3 3:c34e5e51:::rbd_data.112d92027590.0000000000000024:f [read 1936384~1024 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:33:07.241 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:27.059899+0000", 2024-09-14T06:33:07.241 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 220.17445301699999, 2024-09-14T06:33:07.241 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046572122000000001, 2024-09-14T06:33:07.241 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:33:07.241 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:33:07.241 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:33:07.241 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:33:07.241 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:33:07.241 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:33:07.241 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 19329 2024-09-14T06:33:07.242 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:07.242 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:33:07.242 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:07.242 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:33:07.242 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.059899+0000", 2024-09-14T06:33:07.242 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:33:07.242 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:07.242 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:07.242 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:33:07.242 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.059899+0000", 2024-09-14T06:33:07.242 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:33:07.242 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:07.242 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:07.243 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:33:07.243 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062501+0000", 2024-09-14T06:33:07.243 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.0026016419999999999 2024-09-14T06:33:07.243 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:07.243 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:07.243 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:33:07.243 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062514+0000", 2024-09-14T06:33:07.243 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.2947e-05 2024-09-14T06:33:07.243 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:07.243 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:07.243 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:33:07.243 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062523+0000", 2024-09-14T06:33:07.243 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.5519999999999993e-06 2024-09-14T06:33:07.243 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:07.244 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:07.244 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:33:07.244 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062534+0000", 2024-09-14T06:33:07.244 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.1416e-05 2024-09-14T06:33:07.244 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:07.244 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:07.244 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:33:07.244 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.106179+0000", 2024-09-14T06:33:07.244 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.043644049999999997 2024-09-14T06:33:07.244 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:07.244 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:07.244 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:33:07.244 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.106262+0000", 2024-09-14T06:33:07.244 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.3178999999999998e-05 2024-09-14T06:33:07.245 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:07.245 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:07.245 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:33:07.245 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.106471+0000", 2024-09-14T06:33:07.245 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00020933599999999999 2024-09-14T06:33:07.245 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:33:07.245 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:33:07.245 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:33:07.245 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:07.245 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:07.245 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:19351 3.3 3:c34e5e51:::rbd_data.112d92027590.0000000000000024:f [read 1159168~1024 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:33:07.245 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:27.062766+0000", 2024-09-14T06:33:07.245 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 220.171585792, 2024-09-14T06:33:07.245 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046099454999999998, 2024-09-14T06:33:07.246 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:33:07.246 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:33:07.246 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:33:07.246 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:33:07.246 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:33:07.246 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:33:07.246 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 19351 2024-09-14T06:33:07.246 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:07.246 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:33:07.246 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:07.246 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:33:07.246 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062766+0000", 2024-09-14T06:33:07.246 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:33:07.246 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:07.246 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:07.247 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:33:07.247 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062766+0000", 2024-09-14T06:33:07.247 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:33:07.247 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:07.247 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:07.247 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:33:07.247 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062766+0000", 2024-09-14T06:33:07.247 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.6e-07 2024-09-14T06:33:07.247 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:07.247 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:07.247 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:33:07.247 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062767+0000", 2024-09-14T06:33:07.247 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.0070000000000001e-06 2024-09-14T06:33:07.247 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:07.248 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:07.248 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:33:07.248 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062785+0000", 2024-09-14T06:33:07.248 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.7884999999999999e-05 2024-09-14T06:33:07.248 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:07.248 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:07.248 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:33:07.248 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062790+0000", 2024-09-14T06:33:07.248 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.0900000000000004e-06 2024-09-14T06:33:07.248 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:07.248 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:07.248 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:33:07.248 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.108735+0000", 2024-09-14T06:33:07.249 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.045944411999999997 2024-09-14T06:33:07.249 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:07.249 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:07.249 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:33:07.249 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.108809+0000", 2024-09-14T06:33:07.249 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.4157000000000002e-05 2024-09-14T06:33:07.249 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:07.249 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:07.249 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:33:07.249 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.108866+0000", 2024-09-14T06:33:07.249 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.6644000000000002e-05 2024-09-14T06:33:07.249 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:33:07.249 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:33:07.249 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:33:07.250 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:07.250 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:07.250 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:19352 3.3 3:c34e5e51:::rbd_data.112d92027590.0000000000000024:f [read 1169408~1024 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:33:07.250 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:27.062798+0000", 2024-09-14T06:33:07.250 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 220.17155438099999, 2024-09-14T06:33:07.250 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046255900000000003, 2024-09-14T06:33:07.250 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:33:07.250 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:33:07.250 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:33:07.250 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:33:07.250 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:33:07.250 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:33:07.250 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 19352 2024-09-14T06:33:07.251 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:07.251 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:33:07.251 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:07.251 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:33:07.251 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062798+0000", 2024-09-14T06:33:07.251 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:33:07.251 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:07.251 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:07.251 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:33:07.251 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062798+0000", 2024-09-14T06:33:07.251 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:33:07.251 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:07.251 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:07.251 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:33:07.252 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062798+0000", 2024-09-14T06:33:07.252 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.37e-07 2024-09-14T06:33:07.252 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:07.252 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:07.252 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:33:07.252 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062800+0000", 2024-09-14T06:33:07.252 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.0269999999999998e-06 2024-09-14T06:33:07.252 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:07.252 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:07.252 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:33:07.252 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062806+0000", 2024-09-14T06:33:07.252 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.6690000000000003e-06 2024-09-14T06:33:07.252 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:07.252 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:07.252 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:33:07.253 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062810+0000", 2024-09-14T06:33:07.253 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.3449999999999999e-06 2024-09-14T06:33:07.253 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:07.253 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:07.253 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:33:07.253 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.108892+0000", 2024-09-14T06:33:07.253 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046081715000000002 2024-09-14T06:33:07.253 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:07.253 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:07.253 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:33:07.253 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.108940+0000", 2024-09-14T06:33:07.253 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.7815999999999998e-05 2024-09-14T06:33:07.253 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:07.254 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:07.254 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:33:07.254 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.109053+0000", 2024-09-14T06:33:07.254 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.000113791 2024-09-14T06:33:07.254 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:33:07.254 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:33:07.254 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:33:07.254 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:07.254 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:07.254 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:19353 3.3 3:c34e5e51:::rbd_data.112d92027590.0000000000000024:f [read 1179648~1024 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:33:07.254 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:27.062816+0000", 2024-09-14T06:33:07.254 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 220.17153586200001, 2024-09-14T06:33:07.254 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046481256999999998, 2024-09-14T06:33:07.254 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:33:07.255 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:33:07.255 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:33:07.255 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:33:07.255 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:33:07.255 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:33:07.255 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 19353 2024-09-14T06:33:07.255 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:07.255 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:33:07.255 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:07.255 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:33:07.255 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062816+0000", 2024-09-14T06:33:07.255 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:33:07.255 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:07.255 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:07.256 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:33:07.256 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062816+0000", 2024-09-14T06:33:07.256 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:33:07.256 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:07.256 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:07.256 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:33:07.256 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062816+0000", 2024-09-14T06:33:07.256 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.2899999999999999e-07 2024-09-14T06:33:07.256 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:07.256 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:07.256 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:33:07.256 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062818+0000", 2024-09-14T06:33:07.256 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.699e-06 2024-09-14T06:33:07.256 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:07.256 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:07.257 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:33:07.257 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062825+0000", 2024-09-14T06:33:07.257 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.878e-06 2024-09-14T06:33:07.257 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:07.257 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:07.257 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:33:07.257 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062830+0000", 2024-09-14T06:33:07.257 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.5410000000000002e-06 2024-09-14T06:33:07.257 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:07.257 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:07.257 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:33:07.257 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.109096+0000", 2024-09-14T06:33:07.257 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046266517 2024-09-14T06:33:07.257 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:07.258 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:07.258 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:33:07.258 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.109190+0000", 2024-09-14T06:33:07.258 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.4000999999999997e-05 2024-09-14T06:33:07.258 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:07.258 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:07.258 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:33:07.258 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.109297+0000", 2024-09-14T06:33:07.258 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.000107292 2024-09-14T06:33:07.258 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:33:07.258 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:33:07.258 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:33:07.258 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:07.258 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:07.259 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:19354 3.3 3:c34e5e51:::rbd_data.112d92027590.0000000000000024:f [read 1189888~1024 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:33:07.259 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:27.062842+0000", 2024-09-14T06:33:07.259 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 220.171510407, 2024-09-14T06:33:07.259 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046594257, 2024-09-14T06:33:07.259 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:33:07.259 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:33:07.259 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:33:07.259 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:33:07.259 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:33:07.259 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:33:07.259 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 19354 2024-09-14T06:33:07.259 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:07.259 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:33:07.260 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:07.260 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:33:07.260 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062842+0000", 2024-09-14T06:33:07.260 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:33:07.260 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:07.260 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:07.260 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:33:07.260 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062842+0000", 2024-09-14T06:33:07.260 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:33:07.260 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:07.260 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:07.260 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:33:07.260 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062842+0000", 2024-09-14T06:33:07.260 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.7399999999999999e-07 2024-09-14T06:33:07.260 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:07.261 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:07.261 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:33:07.261 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062859+0000", 2024-09-14T06:33:07.261 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.7010000000000001e-05 2024-09-14T06:33:07.261 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:07.261 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:07.261 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:33:07.261 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062866+0000", 2024-09-14T06:33:07.261 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.9990000000000002e-06 2024-09-14T06:33:07.261 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:07.261 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:07.261 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:33:07.261 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062873+0000", 2024-09-14T06:33:07.262 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.8569999999999997e-06 2024-09-14T06:33:07.262 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:07.262 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:07.262 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:33:07.262 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.109335+0000", 2024-09-14T06:33:07.262 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046461943999999998 2024-09-14T06:33:07.262 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:07.262 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:07.262 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:33:07.262 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.109377+0000", 2024-09-14T06:33:07.262 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.2558999999999997e-05 2024-09-14T06:33:07.262 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:07.262 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:07.262 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:33:07.262 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.109436+0000", 2024-09-14T06:33:07.263 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.8514000000000003e-05 2024-09-14T06:33:07.263 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:33:07.263 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:33:07.263 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:33:07.263 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:07.263 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:07.263 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:19355 3.3 3:c34e5e51:::rbd_data.112d92027590.0000000000000024:f [read 1200128~1024 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:33:07.263 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:27.062884+0000", 2024-09-14T06:33:07.263 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 220.171467734, 2024-09-14T06:33:07.263 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046681650999999998, 2024-09-14T06:33:07.263 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:33:07.263 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:33:07.263 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:33:07.263 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:33:07.264 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:33:07.264 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:33:07.264 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 19355 2024-09-14T06:33:07.264 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:07.264 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:33:07.264 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:07.264 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:33:07.264 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062884+0000", 2024-09-14T06:33:07.264 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:33:07.264 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:07.264 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:07.264 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:33:07.264 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062884+0000", 2024-09-14T06:33:07.264 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:33:07.264 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:07.265 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:07.265 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:33:07.265 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062885+0000", 2024-09-14T06:33:07.265 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.8000000000000001e-07 2024-09-14T06:33:07.265 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:07.265 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:07.265 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:33:07.265 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062886+0000", 2024-09-14T06:33:07.265 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.8920000000000001e-06 2024-09-14T06:33:07.265 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:07.265 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:07.265 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:33:07.265 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062892+0000", 2024-09-14T06:33:07.265 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.84e-06 2024-09-14T06:33:07.266 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:07.266 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:07.266 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:33:07.266 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062899+0000", 2024-09-14T06:33:07.266 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.5339999999999996e-06 2024-09-14T06:33:07.266 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:07.266 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:07.266 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:33:07.266 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.109466+0000", 2024-09-14T06:33:07.266 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046566919999999998 2024-09-14T06:33:07.266 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:07.266 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:07.266 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:33:07.266 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.109510+0000", 2024-09-14T06:33:07.266 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.4153000000000002e-05 2024-09-14T06:33:07.267 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:07.267 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:07.267 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:33:07.267 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.109566+0000", 2024-09-14T06:33:07.267 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.5931999999999999e-05 2024-09-14T06:33:07.267 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:33:07.267 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:33:07.267 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:33:07.267 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:07.267 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:07.267 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:19356 3.3 3:c34e5e51:::rbd_data.112d92027590.0000000000000024:f [read 1210368~1024 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:33:07.267 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:27.062908+0000", 2024-09-14T06:33:07.267 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 220.17144429699999, 2024-09-14T06:33:07.267 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046807498000000003, 2024-09-14T06:33:07.268 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:33:07.268 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:33:07.268 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:33:07.268 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:33:07.268 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:33:07.268 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:33:07.268 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 19356 2024-09-14T06:33:07.268 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:07.268 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:33:07.269 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:07.269 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:33:07.269 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062908+0000", 2024-09-14T06:33:07.269 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:33:07.269 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:07.269 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:07.269 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:33:07.269 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062908+0000", 2024-09-14T06:33:07.269 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:33:07.269 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:07.269 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:07.269 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:33:07.269 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062908+0000", 2024-09-14T06:33:07.270 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.5699999999999998e-07 2024-09-14T06:33:07.270 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:07.270 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:07.270 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:33:07.270 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062910+0000", 2024-09-14T06:33:07.270 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.7710000000000001e-06 2024-09-14T06:33:07.270 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:07.270 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:07.270 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:33:07.270 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062920+0000", 2024-09-14T06:33:07.270 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.0654e-05 2024-09-14T06:33:07.270 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:07.270 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:07.271 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:33:07.271 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062931+0000", 2024-09-14T06:33:07.271 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.075e-05 2024-09-14T06:33:07.271 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:07.271 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:07.271 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:33:07.271 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.109595+0000", 2024-09-14T06:33:07.271 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046663713000000002 2024-09-14T06:33:07.271 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:07.271 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:07.271 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:33:07.271 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.109632+0000", 2024-09-14T06:33:07.271 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.7505000000000001e-05 2024-09-14T06:33:07.271 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:07.271 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:07.272 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:33:07.272 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.109715+0000", 2024-09-14T06:33:07.272 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.2748000000000001e-05 2024-09-14T06:33:07.272 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:33:07.272 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:33:07.272 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:33:07.272 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:07.272 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:07.272 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:19357 3.3 3:c34e5e51:::rbd_data.112d92027590.0000000000000024:f [read 1220608~1024 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:33:07.272 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:27.062939+0000", 2024-09-14T06:33:07.272 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 220.17141284900001, 2024-09-14T06:33:07.272 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046920508, 2024-09-14T06:33:07.272 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:33:07.273 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:33:07.273 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:33:07.273 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:33:07.273 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:33:07.273 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:33:07.273 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 19357 2024-09-14T06:33:07.273 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:07.273 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:33:07.273 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:07.273 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:33:07.273 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062939+0000", 2024-09-14T06:33:07.273 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:33:07.273 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:07.273 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:07.274 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:33:07.274 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062939+0000", 2024-09-14T06:33:07.274 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:33:07.274 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:07.274 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:07.274 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:33:07.274 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062940+0000", 2024-09-14T06:33:07.274 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.32e-07 2024-09-14T06:33:07.274 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:07.274 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:07.274 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:33:07.274 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062942+0000", 2024-09-14T06:33:07.274 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.1919999999999999e-06 2024-09-14T06:33:07.274 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:07.274 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:07.274 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:33:07.275 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062948+0000", 2024-09-14T06:33:07.275 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.1149999999999999e-06 2024-09-14T06:33:07.275 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:07.275 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:07.275 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:33:07.275 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062960+0000", 2024-09-14T06:33:07.275 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.1895e-05 2024-09-14T06:33:07.275 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:07.275 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:07.275 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:33:07.275 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.109704+0000", 2024-09-14T06:33:07.275 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046744584999999998 2024-09-14T06:33:07.275 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:07.275 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:07.276 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:33:07.276 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.109777+0000", 2024-09-14T06:33:07.276 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.2282999999999999e-05 2024-09-14T06:33:07.276 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:07.276 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:07.276 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:33:07.276 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.109860+0000", 2024-09-14T06:33:07.276 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.3005999999999998e-05 2024-09-14T06:33:07.276 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:33:07.276 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:33:07.276 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:33:07.276 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:07.276 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:07.276 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:19358 3.3 3:c34e5e51:::rbd_data.112d92027590.0000000000000024:f [read 1230848~1024 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:33:07.276 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:27.062967+0000", 2024-09-14T06:33:07.277 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 220.171385219, 2024-09-14T06:33:07.277 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047144471, 2024-09-14T06:33:07.277 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:33:07.277 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:33:07.277 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:33:07.277 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:33:07.277 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:33:07.277 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:33:07.277 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 19358 2024-09-14T06:33:07.277 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:07.277 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:33:07.277 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:07.277 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:33:07.277 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062967+0000", 2024-09-14T06:33:07.278 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:33:07.278 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:07.278 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:07.278 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:33:07.278 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062967+0000", 2024-09-14T06:33:07.278 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:33:07.278 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:07.278 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:07.278 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:33:07.278 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062967+0000", 2024-09-14T06:33:07.278 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.89e-07 2024-09-14T06:33:07.278 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:07.278 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:07.279 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:33:07.279 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062969+0000", 2024-09-14T06:33:07.279 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.9929999999999998e-06 2024-09-14T06:33:07.279 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:07.279 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:07.279 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:33:07.279 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062975+0000", 2024-09-14T06:33:07.279 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.9030000000000001e-06 2024-09-14T06:33:07.279 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:07.279 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:07.279 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:33:07.279 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062979+0000", 2024-09-14T06:33:07.279 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.2250000000000002e-06 2024-09-14T06:33:07.279 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:07.280 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:07.280 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:33:07.280 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.109934+0000", 2024-09-14T06:33:07.280 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046954410000000002 2024-09-14T06:33:07.280 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:07.280 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:07.280 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:33:07.280 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.110007+0000", 2024-09-14T06:33:07.280 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.3829000000000001e-05 2024-09-14T06:33:07.280 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:07.280 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:07.280 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:33:07.280 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.110111+0000", 2024-09-14T06:33:07.280 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00010372199999999999 2024-09-14T06:33:07.281 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:33:07.281 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:33:07.281 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:33:07.281 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:07.281 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:07.281 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:19359 3.3 3:c34e5e51:::rbd_data.112d92027590.0000000000000024:f [read 1241088~1024 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:33:07.281 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:27.062987+0000", 2024-09-14T06:33:07.281 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 220.17136512900001, 2024-09-14T06:33:07.281 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047297182, 2024-09-14T06:33:07.281 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:33:07.281 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:33:07.281 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:33:07.281 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:33:07.281 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:33:07.282 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:33:07.282 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 19359 2024-09-14T06:33:07.282 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:07.282 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:33:07.282 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:07.282 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:33:07.282 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062987+0000", 2024-09-14T06:33:07.282 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:33:07.282 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:07.282 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:07.282 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:33:07.282 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062987+0000", 2024-09-14T06:33:07.282 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:33:07.282 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:07.283 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:07.283 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:33:07.283 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062987+0000", 2024-09-14T06:33:07.283 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.2500000000000001e-07 2024-09-14T06:33:07.283 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:07.283 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:07.283 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:33:07.283 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062989+0000", 2024-09-14T06:33:07.283 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.6080000000000001e-06 2024-09-14T06:33:07.283 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:07.283 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:07.283 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:33:07.283 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062994+0000", 2024-09-14T06:33:07.283 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.5520000000000004e-06 2024-09-14T06:33:07.283 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:07.284 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:07.284 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:33:07.284 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.062999+0000", 2024-09-14T06:33:07.284 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.3930000000000001e-06 2024-09-14T06:33:07.284 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:07.284 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:07.284 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:33:07.284 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.110142+0000", 2024-09-14T06:33:07.284 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047143154 2024-09-14T06:33:07.284 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:07.284 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:07.284 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:33:07.284 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.110219+0000", 2024-09-14T06:33:07.284 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.6737999999999996e-05 2024-09-14T06:33:07.285 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:07.285 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:07.285 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:33:07.285 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.110284+0000", 2024-09-14T06:33:07.285 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.5412000000000003e-05 2024-09-14T06:33:07.285 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:33:07.285 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:33:07.285 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:33:07.285 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:07.285 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:07.285 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:19360 3.3 3:c34e5e51:::rbd_data.112d92027590.0000000000000024:f [read 1251328~1024 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:33:07.285 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:27.063017+0000", 2024-09-14T06:33:07.285 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 220.17133466799999, 2024-09-14T06:33:07.285 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047453823999999999, 2024-09-14T06:33:07.286 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:33:07.286 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:33:07.286 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:33:07.286 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:33:07.286 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:33:07.286 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:33:07.286 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 19360 2024-09-14T06:33:07.286 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:07.286 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:33:07.286 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:07.286 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:33:07.286 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.063017+0000", 2024-09-14T06:33:07.286 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:33:07.286 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:07.286 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:07.287 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:33:07.287 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.063017+0000", 2024-09-14T06:33:07.287 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:33:07.287 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:07.287 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:07.287 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:33:07.287 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.064977+0000", 2024-09-14T06:33:07.287 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.0019597080000000001 2024-09-14T06:33:07.287 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:07.287 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:07.287 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:33:07.287 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.064987+0000", 2024-09-14T06:33:07.287 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.9620000000000005e-06 2024-09-14T06:33:07.287 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:07.287 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:07.288 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:33:07.288 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065000+0000", 2024-09-14T06:33:07.288 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.3173000000000001e-05 2024-09-14T06:33:07.288 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:07.288 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:07.288 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:33:07.288 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065005+0000", 2024-09-14T06:33:07.288 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.9960000000000001e-06 2024-09-14T06:33:07.288 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:07.288 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:07.288 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:33:07.288 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.110314+0000", 2024-09-14T06:33:07.288 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.04530899 2024-09-14T06:33:07.288 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:07.289 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:07.289 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:33:07.289 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.110398+0000", 2024-09-14T06:33:07.289 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.3539000000000001e-05 2024-09-14T06:33:07.289 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:07.289 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:07.289 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:33:07.289 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.110471+0000", 2024-09-14T06:33:07.289 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.3455999999999996e-05 2024-09-14T06:33:07.289 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:33:07.289 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:33:07.289 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:33:07.289 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:07.289 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:07.289 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:19365 3.3 3:c34e5e51:::rbd_data.112d92027590.0000000000000024:f [read 1302528~1024 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:33:07.290 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:27.065058+0000", 2024-09-14T06:33:07.290 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 220.169293998, 2024-09-14T06:33:07.290 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046105526000000001, 2024-09-14T06:33:07.290 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:33:07.290 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:33:07.290 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:33:07.290 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:33:07.290 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:33:07.290 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:33:07.290 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 19365 2024-09-14T06:33:07.290 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:07.290 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:33:07.290 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:07.290 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:33:07.290 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065058+0000", 2024-09-14T06:33:07.291 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:33:07.291 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:07.291 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:07.291 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:33:07.291 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065058+0000", 2024-09-14T06:33:07.291 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:33:07.291 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:07.291 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:07.291 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:33:07.291 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065058+0000", 2024-09-14T06:33:07.291 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.1899999999999999e-07 2024-09-14T06:33:07.291 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:07.291 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:07.291 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:33:07.292 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065059+0000", 2024-09-14T06:33:07.292 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.1299999999999998e-07 2024-09-14T06:33:07.292 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:07.292 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:07.292 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:33:07.292 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065063+0000", 2024-09-14T06:33:07.292 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.1099999999999996e-06 2024-09-14T06:33:07.292 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:07.292 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:07.292 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:33:07.292 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065070+0000", 2024-09-14T06:33:07.292 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.973e-06 2024-09-14T06:33:07.292 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:07.292 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:07.292 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:33:07.293 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.111014+0000", 2024-09-14T06:33:07.293 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.045943373000000003 2024-09-14T06:33:07.293 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:07.293 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:07.293 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:33:07.293 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.111073+0000", 2024-09-14T06:33:07.293 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.8987999999999998e-05 2024-09-14T06:33:07.293 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:07.293 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:07.293 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:33:07.293 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.111163+0000", 2024-09-14T06:33:07.293 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.0950000000000002e-05 2024-09-14T06:33:07.293 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:33:07.293 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:33:07.293 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:33:07.294 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:07.294 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:07.294 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:19366 3.3 3:c34e5e51:::rbd_data.112d92027590.0000000000000024:f [read 1312768~1024 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:33:07.294 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:27.065074+0000", 2024-09-14T06:33:07.294 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 220.16927814799999, 2024-09-14T06:33:07.294 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046289858000000003, 2024-09-14T06:33:07.294 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:33:07.294 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:33:07.294 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:33:07.294 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:33:07.294 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:33:07.294 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:33:07.294 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 19366 2024-09-14T06:33:07.294 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:07.295 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:33:07.295 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:07.295 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:33:07.295 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065074+0000", 2024-09-14T06:33:07.295 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:33:07.295 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:07.295 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:07.295 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:33:07.295 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065074+0000", 2024-09-14T06:33:07.295 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:33:07.295 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:07.295 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:07.295 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:33:07.295 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065074+0000", 2024-09-14T06:33:07.296 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.3900000000000001e-07 2024-09-14T06:33:07.296 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:07.296 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:07.296 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:33:07.296 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065078+0000", 2024-09-14T06:33:07.296 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.3229999999999999e-06 2024-09-14T06:33:07.296 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:07.296 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:07.296 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:33:07.296 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065082+0000", 2024-09-14T06:33:07.296 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.8299999999999998e-06 2024-09-14T06:33:07.296 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:07.296 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:07.296 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:33:07.297 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065085+0000", 2024-09-14T06:33:07.297 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.8660000000000002e-06 2024-09-14T06:33:07.297 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:07.297 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:07.297 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:33:07.297 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.111270+0000", 2024-09-14T06:33:07.297 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046185035999999999 2024-09-14T06:33:07.297 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:07.297 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:07.297 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:33:07.297 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.111311+0000", 2024-09-14T06:33:07.297 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.0935e-05 2024-09-14T06:33:07.297 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:07.297 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:07.298 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:33:07.298 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.111364+0000", 2024-09-14T06:33:07.298 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.2629e-05 2024-09-14T06:33:07.298 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:33:07.298 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:33:07.298 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:33:07.298 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:07.298 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:07.298 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:19367 3.3 3:c34e5e51:::rbd_data.112d92027590.0000000000000024:f [read 1323008~1024 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:33:07.298 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:27.065115+0000", 2024-09-14T06:33:07.298 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 220.169236587, 2024-09-14T06:33:07.298 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046407003000000002, 2024-09-14T06:33:07.298 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:33:07.298 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:33:07.298 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:33:07.299 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:33:07.299 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:33:07.299 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:33:07.299 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 19367 2024-09-14T06:33:07.299 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:07.299 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:33:07.299 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:07.299 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:33:07.299 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065115+0000", 2024-09-14T06:33:07.299 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:33:07.299 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:07.299 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:07.299 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:33:07.299 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065115+0000", 2024-09-14T06:33:07.300 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:33:07.300 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:07.300 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:07.300 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:33:07.300 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065116+0000", 2024-09-14T06:33:07.300 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.4999999999999998e-07 2024-09-14T06:33:07.300 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:07.300 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:07.300 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:33:07.300 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065117+0000", 2024-09-14T06:33:07.300 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.189e-06 2024-09-14T06:33:07.300 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:07.300 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:07.300 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:33:07.301 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065122+0000", 2024-09-14T06:33:07.301 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.3709999999999999e-06 2024-09-14T06:33:07.301 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:07.301 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:07.301 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:33:07.301 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065125+0000", 2024-09-14T06:33:07.301 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.8940000000000002e-06 2024-09-14T06:33:07.301 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:07.301 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:07.301 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:33:07.301 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.111402+0000", 2024-09-14T06:33:07.301 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046276645999999998 2024-09-14T06:33:07.301 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:07.301 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:07.302 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:33:07.302 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.111445+0000", 2024-09-14T06:33:07.302 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.3288000000000003e-05 2024-09-14T06:33:07.302 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:07.302 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:07.302 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:33:07.302 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.111522+0000", 2024-09-14T06:33:07.302 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.7164999999999997e-05 2024-09-14T06:33:07.302 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:33:07.302 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:33:07.302 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:33:07.302 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:07.302 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:07.302 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:19368 3.3 3:c34e5e51:::rbd_data.112d92027590.0000000000000024:f [read 1333248~1024 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:33:07.302 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:27.065130+0000", 2024-09-14T06:33:07.303 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 220.169222297, 2024-09-14T06:33:07.303 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046507275000000001, 2024-09-14T06:33:07.303 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:33:07.303 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:33:07.303 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:33:07.303 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:33:07.303 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:33:07.303 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:33:07.303 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 19368 2024-09-14T06:33:07.303 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:07.303 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:33:07.303 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:07.303 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:33:07.303 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065130+0000", 2024-09-14T06:33:07.303 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:33:07.304 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:07.304 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:07.304 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:33:07.304 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065130+0000", 2024-09-14T06:33:07.304 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:33:07.304 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:07.304 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:07.304 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:33:07.304 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065130+0000", 2024-09-14T06:33:07.304 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.0800000000000001e-07 2024-09-14T06:33:07.304 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:07.304 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:07.304 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:33:07.304 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065131+0000", 2024-09-14T06:33:07.305 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.8700000000000004e-07 2024-09-14T06:33:07.305 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:07.305 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:07.305 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:33:07.305 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065134+0000", 2024-09-14T06:33:07.305 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.1039999999999998e-06 2024-09-14T06:33:07.305 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:07.305 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:07.305 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:33:07.305 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065136+0000", 2024-09-14T06:33:07.305 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.3949999999999999e-06 2024-09-14T06:33:07.305 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:07.305 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:07.305 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:33:07.306 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.111541+0000", 2024-09-14T06:33:07.306 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046404206000000003 2024-09-14T06:33:07.306 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:07.306 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:07.306 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:33:07.306 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.111582+0000", 2024-09-14T06:33:07.306 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.1051999999999999e-05 2024-09-14T06:33:07.306 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:07.306 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:07.306 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:33:07.306 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.111637+0000", 2024-09-14T06:33:07.306 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.5322999999999999e-05 2024-09-14T06:33:07.306 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:33:07.306 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:33:07.307 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:33:07.307 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:07.307 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:07.307 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:19369 3.3 3:c34e5e51:::rbd_data.112d92027590.0000000000000024:f [read 1343488~1024 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:33:07.307 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:27.065139+0000", 2024-09-14T06:33:07.307 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 220.16921248, 2024-09-14T06:33:07.307 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046682169000000003, 2024-09-14T06:33:07.307 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:33:07.307 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:33:07.307 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:33:07.307 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:33:07.307 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:33:07.307 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:33:07.307 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 19369 2024-09-14T06:33:07.308 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:07.308 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:33:07.308 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:07.308 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:33:07.308 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065139+0000", 2024-09-14T06:33:07.308 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:33:07.308 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:07.308 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:07.308 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:33:07.308 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065139+0000", 2024-09-14T06:33:07.308 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:33:07.308 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:07.308 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:07.308 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:33:07.308 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065140+0000", 2024-09-14T06:33:07.309 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.8299999999999998e-07 2024-09-14T06:33:07.309 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:07.309 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:07.309 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:33:07.309 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065141+0000", 2024-09-14T06:33:07.309 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.3900000000000003e-07 2024-09-14T06:33:07.309 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:07.309 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:07.309 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:33:07.309 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065144+0000", 2024-09-14T06:33:07.309 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.7409999999999998e-06 2024-09-14T06:33:07.309 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:07.309 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:07.309 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:33:07.310 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065147+0000", 2024-09-14T06:33:07.310 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.4150000000000002e-06 2024-09-14T06:33:07.310 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:07.310 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:07.310 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:33:07.310 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.111673+0000", 2024-09-14T06:33:07.310 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046526004000000003 2024-09-14T06:33:07.310 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:07.310 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:07.310 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:33:07.310 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.111749+0000", 2024-09-14T06:33:07.310 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.5741999999999994e-05 2024-09-14T06:33:07.310 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:07.310 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:07.311 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:33:07.311 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.111822+0000", 2024-09-14T06:33:07.311 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.3045000000000003e-05 2024-09-14T06:33:07.311 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:33:07.311 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:33:07.311 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:33:07.311 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:07.311 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:07.311 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:19370 3.3 3:c34e5e51:::rbd_data.112d92027590.0000000000000024:f [read 1353728~1024 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:33:07.311 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:27.065150+0000", 2024-09-14T06:33:07.311 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 220.169201751, 2024-09-14T06:33:07.311 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046897556999999999, 2024-09-14T06:33:07.311 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:33:07.312 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:33:07.312 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:33:07.312 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:33:07.312 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:33:07.312 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:33:07.312 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 19370 2024-09-14T06:33:07.312 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:07.312 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:33:07.312 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:07.312 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:33:07.312 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065150+0000", 2024-09-14T06:33:07.312 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:33:07.312 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:07.312 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:07.313 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:33:07.313 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065150+0000", 2024-09-14T06:33:07.313 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:33:07.313 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:07.313 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:07.313 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:33:07.313 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065150+0000", 2024-09-14T06:33:07.313 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.0200000000000001e-07 2024-09-14T06:33:07.313 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:07.313 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:07.313 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:33:07.313 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065151+0000", 2024-09-14T06:33:07.313 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.1100000000000004e-07 2024-09-14T06:33:07.313 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:07.314 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:07.314 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:33:07.314 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065154+0000", 2024-09-14T06:33:07.314 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.9639999999999999e-06 2024-09-14T06:33:07.314 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:07.314 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:07.314 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:33:07.314 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065157+0000", 2024-09-14T06:33:07.314 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.4260000000000002e-06 2024-09-14T06:33:07.314 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:07.314 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:07.314 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:33:07.314 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.111861+0000", 2024-09-14T06:33:07.314 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046704268 2024-09-14T06:33:07.314 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:07.315 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:07.315 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:33:07.315 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.111939+0000", 2024-09-14T06:33:07.315 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.8231000000000004e-05 2024-09-14T06:33:07.315 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:07.315 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:07.315 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:33:07.315 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.112048+0000", 2024-09-14T06:33:07.315 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00010855500000000001 2024-09-14T06:33:07.315 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:33:07.315 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:33:07.315 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:33:07.315 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:07.315 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:07.315 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:19371 3.3 3:c34e5e51:::rbd_data.112d92027590.0000000000000024:f [read 1363968~1024 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:33:07.316 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:27.065161+0000", 2024-09-14T06:33:07.316 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 220.169191101, 2024-09-14T06:33:07.316 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047021944000000003, 2024-09-14T06:33:07.316 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:33:07.316 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:33:07.316 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:33:07.316 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:33:07.316 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:33:07.316 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:33:07.316 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 19371 2024-09-14T06:33:07.316 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:07.316 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:33:07.316 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:07.316 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:33:07.317 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065161+0000", 2024-09-14T06:33:07.317 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:33:07.317 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:07.317 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:07.317 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:33:07.317 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065161+0000", 2024-09-14T06:33:07.317 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:33:07.317 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:07.317 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:07.317 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:33:07.317 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065161+0000", 2024-09-14T06:33:07.317 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.0200000000000001e-07 2024-09-14T06:33:07.317 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:07.317 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:07.318 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:33:07.318 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065166+0000", 2024-09-14T06:33:07.318 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.9869999999999999e-06 2024-09-14T06:33:07.318 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:07.318 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:07.318 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:33:07.318 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065169+0000", 2024-09-14T06:33:07.318 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.3799999999999998e-06 2024-09-14T06:33:07.318 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:07.318 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:07.318 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:33:07.318 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065172+0000", 2024-09-14T06:33:07.318 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.4830000000000002e-06 2024-09-14T06:33:07.318 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:07.318 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:07.319 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:33:07.319 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.112071+0000", 2024-09-14T06:33:07.319 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046899529000000002 2024-09-14T06:33:07.319 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:07.319 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:07.319 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:33:07.319 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.112128+0000", 2024-09-14T06:33:07.319 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.6341999999999997e-05 2024-09-14T06:33:07.319 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:07.319 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:07.319 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:33:07.319 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.112183+0000", 2024-09-14T06:33:07.319 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.5021000000000001e-05 2024-09-14T06:33:07.319 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:33:07.319 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:33:07.320 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:33:07.320 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:07.320 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:07.320 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:19372 3.3 3:c34e5e51:::rbd_data.112d92027590.0000000000000024:f [read 1374208~1024 [fadvise_nocache]] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:33:07.320 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:27.065176+0000", 2024-09-14T06:33:07.320 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 220.169175951, 2024-09-14T06:33:07.320 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047188889999999997, 2024-09-14T06:33:07.320 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:33:07.320 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:33:07.320 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:33:07.320 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:33:07.320 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:33:07.320 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:33:07.320 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 19372 2024-09-14T06:33:07.320 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:07.321 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:33:07.321 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:07.321 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:33:07.321 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065176+0000", 2024-09-14T06:33:07.321 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:33:07.321 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:07.321 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:07.321 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:33:07.321 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.065176+0000", 2024-09-14T06:33:07.321 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:33:07.321 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:07.321 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:07.321 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:33:07.321 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.071117+0000", 2024-09-14T06:33:07.322 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.0059404990000000001 2024-09-14T06:33:07.322 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:07.322 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:07.322 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:33:07.322 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.071120+0000", 2024-09-14T06:33:07.322 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.4340000000000001e-06 2024-09-14T06:33:07.322 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:07.322 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:07.322 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:33:07.322 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.071129+0000", 2024-09-14T06:33:07.322 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.8389999999999996e-06 2024-09-14T06:33:07.322 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:07.322 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:07.322 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:33:07.322 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.071134+0000", 2024-09-14T06:33:07.323 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.412e-06 2024-09-14T06:33:07.323 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:07.323 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:07.323 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:33:07.323 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.112221+0000", 2024-09-14T06:33:07.323 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.041086473999999998 2024-09-14T06:33:07.323 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:07.323 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:07.323 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:33:07.323 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.112299+0000", 2024-09-14T06:33:07.323 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.8746999999999997e-05 2024-09-14T06:33:07.323 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:07.323 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:07.324 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:33:07.324 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:27.112365+0000", 2024-09-14T06:33:07.324 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.5485000000000001e-05 2024-09-14T06:33:07.324 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:33:07.324 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:33:07.324 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:33:07.324 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:33:07.324 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:33:07.324 INFO:teuthology.orchestra.run.smithi138.stdout:} 2024-09-14T06:33:07.325 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-14T06:33:07.369 INFO:tasks.ceph.osd.4.smithi138.stderr:2024-09-14T06:33:07.368+0000 7f6c653c8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33615) UID: 0 2024-09-14T06:33:07.429 INFO:teuthology.orchestra.run.smithi138.stdout:{ 2024-09-14T06:33:07.429 INFO:teuthology.orchestra.run.smithi138.stdout: "ops": [], 2024-09-14T06:33:07.429 INFO:teuthology.orchestra.run.smithi138.stdout: "num_ops": 0 2024-09-14T06:33:07.429 INFO:teuthology.orchestra.run.smithi138.stdout:} 2024-09-14T06:33:07.430 INFO:tasks.ceph.osd.3.smithi094.stderr:2024-09-14T06:33:07.429+0000 7feeeafe8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33655) UID: 0 2024-09-14T06:33:07.438 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-14T06:33:07.530 INFO:tasks.ceph.osd.5.smithi138.stderr:2024-09-14T06:33:07.529+0000 7fe13746f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33618) UID: 0 2024-09-14T06:33:07.581 INFO:teuthology.orchestra.run.smithi138.stdout:{ 2024-09-14T06:33:07.581 INFO:teuthology.orchestra.run.smithi138.stdout: "ops": [], 2024-09-14T06:33:07.581 INFO:teuthology.orchestra.run.smithi138.stdout: "complaint_time": 30, 2024-09-14T06:33:07.582 INFO:teuthology.orchestra.run.smithi138.stdout: "num_blocked_ops": 0 2024-09-14T06:33:07.582 INFO:teuthology.orchestra.run.smithi138.stdout:} 2024-09-14T06:33:07.590 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-14T06:33:07.632 INFO:tasks.ceph.osd.2.smithi094.stderr:2024-09-14T06:33:07.631+0000 7faf3269b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33649) UID: 0 2024-09-14T06:33:07.706 INFO:teuthology.orchestra.run.smithi094.stderr:reweighted osd.1 to 1 (10000) 2024-09-14T06:33:07.721 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 4 1 2024-09-14T06:33:07.733 INFO:tasks.ceph.osd.4.smithi138.stderr:2024-09-14T06:33:07.732+0000 7f6c653c8640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33615) UID: 0 2024-09-14T06:33:07.739 INFO:teuthology.orchestra.run.smithi138.stdout:{ 2024-09-14T06:33:07.739 INFO:teuthology.orchestra.run.smithi138.stdout: "size": 20, 2024-09-14T06:33:07.739 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 600, 2024-09-14T06:33:07.739 INFO:teuthology.orchestra.run.smithi138.stdout: "ops": [ 2024-09-14T06:33:07.739 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:07.740 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:11453 3.1d 3:bcad7303:::rbd_data.112d92027590.0000000000000001:f [read 94208~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:33:07.740 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:25.654375+0000", 2024-09-14T06:33:07.740 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 222.08344415600001, 2024-09-14T06:33:07.740 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052449318000000002, 2024-09-14T06:33:07.740 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:33:07.740 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:33:07.740 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:33:07.740 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:33:07.740 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:33:07.740 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:33:07.740 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 11453 2024-09-14T06:33:07.740 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:07.740 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:33:07.741 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:07.741 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:33:07.741 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654375+0000", 2024-09-14T06:33:07.741 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:33:07.741 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:07.741 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:07.741 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:33:07.741 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654375+0000", 2024-09-14T06:33:07.741 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:33:07.741 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:07.741 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:07.741 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:33:07.741 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654496+0000", 2024-09-14T06:33:07.742 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00012081 2024-09-14T06:33:07.742 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:07.742 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:07.742 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:33:07.742 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654497+0000", 2024-09-14T06:33:07.742 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.8789999999999999e-06 2024-09-14T06:33:07.742 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:07.742 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:07.742 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:33:07.742 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654501+0000", 2024-09-14T06:33:07.742 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.8999999999999999e-06 2024-09-14T06:33:07.742 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:07.742 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:07.742 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:33:07.742 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654505+0000", 2024-09-14T06:33:07.743 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.2229999999999999e-06 2024-09-14T06:33:07.743 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:07.743 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:07.743 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:33:07.743 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.705917+0000", 2024-09-14T06:33:07.743 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.051412501999999999 2024-09-14T06:33:07.743 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:07.743 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:07.743 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:33:07.743 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.705999+0000", 2024-09-14T06:33:07.743 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.1551999999999994e-05 2024-09-14T06:33:07.743 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:07.743 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:07.744 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:33:07.744 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.706824+0000", 2024-09-14T06:33:07.744 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00082545199999999996 2024-09-14T06:33:07.744 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:33:07.744 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:33:07.744 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:33:07.744 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:07.744 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:07.744 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:11454 3.1d 3:bcad7303:::rbd_data.112d92027590.0000000000000001:f [read 104448~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:33:07.744 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:25.654510+0000", 2024-09-14T06:33:07.744 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 222.08330936600001, 2024-09-14T06:33:07.744 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052610713000000003, 2024-09-14T06:33:07.744 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:33:07.744 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:33:07.745 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:33:07.745 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:33:07.745 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:33:07.745 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:33:07.745 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 11454 2024-09-14T06:33:07.745 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:07.745 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:33:07.745 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:07.745 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:33:07.745 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654510+0000", 2024-09-14T06:33:07.745 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:33:07.745 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:07.745 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:07.745 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:33:07.746 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654510+0000", 2024-09-14T06:33:07.746 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:33:07.746 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:07.746 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:07.746 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:33:07.746 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654825+0000", 2024-09-14T06:33:07.746 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00031522400000000002 2024-09-14T06:33:07.746 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:07.746 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:07.746 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:33:07.746 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654827+0000", 2024-09-14T06:33:07.746 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.9709999999999998e-06 2024-09-14T06:33:07.746 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:07.746 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:07.747 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:33:07.747 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654832+0000", 2024-09-14T06:33:07.747 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.4099999999999999e-06 2024-09-14T06:33:07.747 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:07.747 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:07.747 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:33:07.747 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654835+0000", 2024-09-14T06:33:07.747 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.14e-06 2024-09-14T06:33:07.747 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:07.747 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:07.747 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:33:07.747 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.706851+0000", 2024-09-14T06:33:07.747 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052015761000000001 2024-09-14T06:33:07.748 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:07.748 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:07.748 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:33:07.748 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.706934+0000", 2024-09-14T06:33:07.748 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.2484999999999994e-05 2024-09-14T06:33:07.748 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:07.748 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:07.748 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:33:07.748 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.707120+0000", 2024-09-14T06:33:07.748 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00018672200000000001 2024-09-14T06:33:07.748 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:33:07.748 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:33:07.748 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:33:07.749 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:07.749 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:07.749 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:11455 3.1d 3:bcad7303:::rbd_data.112d92027590.0000000000000001:f [read 114688~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:33:07.749 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:25.654841+0000", 2024-09-14T06:33:07.749 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 222.082978037, 2024-09-14T06:33:07.749 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052579898, 2024-09-14T06:33:07.749 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:33:07.749 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:33:07.749 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:33:07.749 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:33:07.749 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:33:07.749 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:33:07.750 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 11455 2024-09-14T06:33:07.750 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:07.750 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:33:07.750 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:07.750 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:33:07.750 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654841+0000", 2024-09-14T06:33:07.750 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:33:07.750 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:07.750 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:07.750 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:33:07.750 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654841+0000", 2024-09-14T06:33:07.750 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:33:07.750 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:07.750 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:07.750 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:33:07.751 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654841+0000", 2024-09-14T06:33:07.751 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.7300000000000002e-07 2024-09-14T06:33:07.751 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:07.751 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:07.751 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:33:07.751 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654845+0000", 2024-09-14T06:33:07.751 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.0520000000000003e-06 2024-09-14T06:33:07.751 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:07.751 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:07.751 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:33:07.751 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654849+0000", 2024-09-14T06:33:07.751 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.0459999999999999e-06 2024-09-14T06:33:07.751 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:07.751 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:07.752 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:33:07.752 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654852+0000", 2024-09-14T06:33:07.752 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.9830000000000001e-06 2024-09-14T06:33:07.752 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:07.752 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:07.752 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:33:07.752 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.707159+0000", 2024-09-14T06:33:07.752 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052306826000000001 2024-09-14T06:33:07.752 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:07.752 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:07.752 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:33:07.752 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.707242+0000", 2024-09-14T06:33:07.752 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.2595000000000005e-05 2024-09-14T06:33:07.753 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:07.753 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:07.753 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:33:07.753 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.707421+0000", 2024-09-14T06:33:07.753 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.000179123 2024-09-14T06:33:07.753 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:33:07.753 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:33:07.753 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:33:07.753 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:07.753 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:07.753 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:11456 3.1d 3:bcad7303:::rbd_data.112d92027590.0000000000000001:f [read 124928~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:33:07.753 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:25.654856+0000", 2024-09-14T06:33:07.753 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 222.082963191, 2024-09-14T06:33:07.754 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052801580000000001, 2024-09-14T06:33:07.754 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:33:07.754 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:33:07.754 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:33:07.754 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:33:07.754 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:33:07.754 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:33:07.754 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 11456 2024-09-14T06:33:07.754 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:07.754 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:33:07.754 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:07.754 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:33:07.754 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654856+0000", 2024-09-14T06:33:07.755 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:33:07.755 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:07.755 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:07.755 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:33:07.755 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654856+0000", 2024-09-14T06:33:07.755 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:33:07.755 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:07.755 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:07.755 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:33:07.755 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654856+0000", 2024-09-14T06:33:07.755 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.29e-07 2024-09-14T06:33:07.755 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:07.755 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:07.756 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:33:07.756 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654857+0000", 2024-09-14T06:33:07.756 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.187e-06 2024-09-14T06:33:07.756 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:07.756 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:07.756 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:33:07.756 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654861+0000", 2024-09-14T06:33:07.756 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.3850000000000002e-06 2024-09-14T06:33:07.756 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:07.756 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:07.756 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:33:07.756 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654863+0000", 2024-09-14T06:33:07.756 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.9500000000000001e-06 2024-09-14T06:33:07.757 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:07.757 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:07.757 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:33:07.757 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.707456+0000", 2024-09-14T06:33:07.757 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052592434 2024-09-14T06:33:07.757 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:07.757 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:07.757 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:33:07.757 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.707516+0000", 2024-09-14T06:33:07.757 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.0167999999999997e-05 2024-09-14T06:33:07.757 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:07.757 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:07.757 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:33:07.758 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.707657+0000", 2024-09-14T06:33:07.758 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00014122700000000001 2024-09-14T06:33:07.758 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:33:07.758 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:33:07.758 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:33:07.758 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:07.758 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:07.758 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:11457 3.1d 3:bcad7303:::rbd_data.112d92027590.0000000000000001:f [read 135168~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:33:07.758 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:25.654867+0000", 2024-09-14T06:33:07.758 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 222.08295230600001, 2024-09-14T06:33:07.758 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052921902, 2024-09-14T06:33:07.758 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:33:07.758 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:33:07.758 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:33:07.759 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:33:07.759 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:33:07.759 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:33:07.759 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 11457 2024-09-14T06:33:07.759 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:07.759 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:33:07.759 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:07.759 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:33:07.759 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654867+0000", 2024-09-14T06:33:07.759 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:33:07.759 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:07.759 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:07.759 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:33:07.759 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654867+0000", 2024-09-14T06:33:07.760 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:33:07.760 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:07.760 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:07.760 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:33:07.760 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654972+0000", 2024-09-14T06:33:07.760 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00010584 2024-09-14T06:33:07.760 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:07.760 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:07.760 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:33:07.760 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654975+0000", 2024-09-14T06:33:07.760 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.1169999999999998e-06 2024-09-14T06:33:07.760 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:07.760 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:07.761 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:33:07.761 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654979+0000", 2024-09-14T06:33:07.761 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.1960000000000001e-06 2024-09-14T06:33:07.761 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:07.761 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:07.761 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:33:07.761 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654982+0000", 2024-09-14T06:33:07.761 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.2990000000000001e-06 2024-09-14T06:33:07.761 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:07.761 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:07.761 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:33:07.761 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.707684+0000", 2024-09-14T06:33:07.761 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052701979000000003 2024-09-14T06:33:07.761 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:07.762 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:07.762 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:33:07.762 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.707728+0000", 2024-09-14T06:33:07.762 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.3529999999999998e-05 2024-09-14T06:33:07.762 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:07.762 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:07.762 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:33:07.762 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.707789+0000", 2024-09-14T06:33:07.762 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.0940999999999998e-05 2024-09-14T06:33:07.762 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:33:07.762 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:33:07.762 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:33:07.762 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:07.762 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:07.763 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:11458 3.1d 3:bcad7303:::rbd_data.112d92027590.0000000000000001:f [read 145408~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:33:07.763 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:25.654987+0000", 2024-09-14T06:33:07.763 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 222.082831908, 2024-09-14T06:33:07.763 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.05291655, 2024-09-14T06:33:07.763 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:33:07.763 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:33:07.763 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:33:07.763 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:33:07.763 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:33:07.763 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:33:07.763 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 11458 2024-09-14T06:33:07.763 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:07.763 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:33:07.763 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:07.764 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:33:07.764 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654987+0000", 2024-09-14T06:33:07.764 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:33:07.764 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:07.764 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:07.764 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:33:07.764 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.654987+0000", 2024-09-14T06:33:07.764 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:33:07.764 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:07.764 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:07.764 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:33:07.764 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655077+0000", 2024-09-14T06:33:07.764 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.0253000000000005e-05 2024-09-14T06:33:07.765 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:07.765 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:07.765 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:33:07.765 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655079+0000", 2024-09-14T06:33:07.765 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.9989999999999998e-06 2024-09-14T06:33:07.765 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:07.765 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:07.765 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:33:07.765 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655083+0000", 2024-09-14T06:33:07.765 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.9280000000000003e-06 2024-09-14T06:33:07.765 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:07.765 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:07.765 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:33:07.765 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655086+0000", 2024-09-14T06:33:07.766 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.056e-06 2024-09-14T06:33:07.766 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:07.766 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:07.766 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:33:07.766 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.707805+0000", 2024-09-14T06:33:07.766 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052718627999999997 2024-09-14T06:33:07.766 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:07.766 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:07.766 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:33:07.766 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.707848+0000", 2024-09-14T06:33:07.766 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.2728999999999997e-05 2024-09-14T06:33:07.766 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:07.766 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:07.766 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:33:07.767 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.707904+0000", 2024-09-14T06:33:07.767 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.5957e-05 2024-09-14T06:33:07.767 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:33:07.767 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:33:07.767 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:33:07.767 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:07.767 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:07.767 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:11459 3.1d 3:bcad7303:::rbd_data.112d92027590.0000000000000001:f [read 155648~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:33:07.767 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:25.655091+0000", 2024-09-14T06:33:07.767 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 222.08272790000001, 2024-09-14T06:33:07.767 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052944242000000002, 2024-09-14T06:33:07.767 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:33:07.767 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:33:07.768 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:33:07.768 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:33:07.768 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:33:07.768 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:33:07.768 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 11459 2024-09-14T06:33:07.768 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:07.768 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:33:07.768 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:07.768 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:33:07.768 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655091+0000", 2024-09-14T06:33:07.768 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:33:07.768 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:07.768 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:07.769 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:33:07.769 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655091+0000", 2024-09-14T06:33:07.769 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:33:07.769 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:07.769 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:07.769 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:33:07.769 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655237+0000", 2024-09-14T06:33:07.769 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00014570100000000001 2024-09-14T06:33:07.769 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:07.769 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:07.769 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:33:07.769 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655239+0000", 2024-09-14T06:33:07.769 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.8670000000000001e-06 2024-09-14T06:33:07.769 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:07.770 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:07.770 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:33:07.770 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655243+0000", 2024-09-14T06:33:07.770 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.9829999999999998e-06 2024-09-14T06:33:07.770 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:07.770 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:07.770 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:33:07.770 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655246+0000", 2024-09-14T06:33:07.770 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.1540000000000002e-06 2024-09-14T06:33:07.770 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:07.770 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:07.770 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:33:07.770 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.707934+0000", 2024-09-14T06:33:07.770 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052688286000000001 2024-09-14T06:33:07.771 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:07.771 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:07.771 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:33:07.771 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.707977+0000", 2024-09-14T06:33:07.771 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.3291999999999998e-05 2024-09-14T06:33:07.771 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:07.771 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:07.771 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:33:07.771 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.708035+0000", 2024-09-14T06:33:07.771 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.7958999999999999e-05 2024-09-14T06:33:07.771 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:33:07.771 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:33:07.771 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:33:07.772 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:07.772 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:07.772 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:11460 3.1d 3:bcad7303:::rbd_data.112d92027590.0000000000000001:f [read 165888~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:33:07.772 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:25.655250+0000", 2024-09-14T06:33:07.772 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 222.082568603, 2024-09-14T06:33:07.772 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052910698999999999, 2024-09-14T06:33:07.772 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:33:07.772 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:33:07.772 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:33:07.772 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:33:07.772 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:33:07.772 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:33:07.772 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 11460 2024-09-14T06:33:07.772 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:07.773 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:33:07.773 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:07.773 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:33:07.773 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655250+0000", 2024-09-14T06:33:07.773 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:33:07.773 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:07.773 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:07.773 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:33:07.773 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655250+0000", 2024-09-14T06:33:07.773 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:33:07.773 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:07.773 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:07.773 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:33:07.774 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655356+0000", 2024-09-14T06:33:07.774 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00010535900000000001 2024-09-14T06:33:07.774 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:07.774 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:07.774 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:33:07.774 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655357+0000", 2024-09-14T06:33:07.774 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.8080000000000001e-06 2024-09-14T06:33:07.774 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:07.774 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:07.774 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:33:07.774 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655361+0000", 2024-09-14T06:33:07.774 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.861e-06 2024-09-14T06:33:07.774 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:07.774 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:07.774 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:33:07.775 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655364+0000", 2024-09-14T06:33:07.775 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.1089999999999998e-06 2024-09-14T06:33:07.775 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:07.775 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:07.775 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:33:07.775 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.708065+0000", 2024-09-14T06:33:07.775 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052700595000000003 2024-09-14T06:33:07.775 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:07.775 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:07.775 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:33:07.775 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.708108+0000", 2024-09-14T06:33:07.775 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.2484999999999997e-05 2024-09-14T06:33:07.775 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:07.775 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:07.776 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:33:07.776 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.708161+0000", 2024-09-14T06:33:07.776 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.3482000000000001e-05 2024-09-14T06:33:07.776 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:33:07.776 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:33:07.776 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:33:07.776 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:07.776 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:07.776 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:11461 3.1d 3:bcad7303:::rbd_data.112d92027590.0000000000000001:f [read 176128~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:33:07.776 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:25.655370+0000", 2024-09-14T06:33:07.776 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 222.08244932100001, 2024-09-14T06:33:07.776 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052915141999999998, 2024-09-14T06:33:07.776 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:33:07.776 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:33:07.776 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:33:07.777 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:33:07.777 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:33:07.777 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:33:07.777 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 11461 2024-09-14T06:33:07.777 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:07.777 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:33:07.777 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:07.777 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:33:07.777 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655370+0000", 2024-09-14T06:33:07.777 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:33:07.777 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:07.777 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:07.777 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:33:07.777 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655370+0000", 2024-09-14T06:33:07.777 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:33:07.778 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:07.778 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:07.778 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:33:07.778 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655394+0000", 2024-09-14T06:33:07.778 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.4553999999999999e-05 2024-09-14T06:33:07.778 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:07.778 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:07.778 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:33:07.778 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655396+0000", 2024-09-14T06:33:07.778 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.8950000000000001e-06 2024-09-14T06:33:07.778 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:07.778 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:07.778 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:33:07.778 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655401+0000", 2024-09-14T06:33:07.779 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.0089999999999998e-06 2024-09-14T06:33:07.779 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:07.779 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:07.779 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:33:07.779 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655404+0000", 2024-09-14T06:33:07.779 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.3400000000000002e-06 2024-09-14T06:33:07.779 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:07.779 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:07.779 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:33:07.779 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.708186+0000", 2024-09-14T06:33:07.779 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052781726000000001 2024-09-14T06:33:07.779 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:07.779 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:07.779 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:33:07.780 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.708229+0000", 2024-09-14T06:33:07.780 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.2670999999999999e-05 2024-09-14T06:33:07.780 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:07.780 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:07.780 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:33:07.780 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.708285+0000", 2024-09-14T06:33:07.780 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.5946999999999998e-05 2024-09-14T06:33:07.780 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:33:07.780 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:33:07.780 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:33:07.780 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:07.780 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:07.780 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:11462 3.1d 3:bcad7303:::rbd_data.112d92027590.0000000000000001:f [read 186368~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:33:07.780 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:25.655410+0000", 2024-09-14T06:33:07.781 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 222.082408843, 2024-09-14T06:33:07.781 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052995258000000003, 2024-09-14T06:33:07.781 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:33:07.781 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:33:07.781 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:33:07.781 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:33:07.781 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:33:07.781 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:33:07.781 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 11462 2024-09-14T06:33:07.781 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:07.781 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:33:07.781 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:07.781 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:33:07.781 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655410+0000", 2024-09-14T06:33:07.782 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:33:07.782 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:07.782 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:07.782 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:33:07.782 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655410+0000", 2024-09-14T06:33:07.782 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:33:07.782 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:07.782 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:07.782 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:33:07.782 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655625+0000", 2024-09-14T06:33:07.782 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00021442600000000001 2024-09-14T06:33:07.782 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:07.782 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:07.782 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:33:07.783 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655626+0000", 2024-09-14T06:33:07.783 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.815e-06 2024-09-14T06:33:07.783 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:07.783 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:07.783 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:33:07.783 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655631+0000", 2024-09-14T06:33:07.783 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.651e-06 2024-09-14T06:33:07.783 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:07.783 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:07.783 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:33:07.783 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655634+0000", 2024-09-14T06:33:07.783 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.2449999999999998e-06 2024-09-14T06:33:07.783 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:07.783 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:07.784 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:33:07.784 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.708312+0000", 2024-09-14T06:33:07.784 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052677412 2024-09-14T06:33:07.784 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:07.784 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:07.784 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:33:07.784 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.708352+0000", 2024-09-14T06:33:07.784 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.0729999999999998e-05 2024-09-14T06:33:07.784 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:07.784 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:07.784 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:33:07.784 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.708405+0000", 2024-09-14T06:33:07.784 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.2979000000000002e-05 2024-09-14T06:33:07.784 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:33:07.785 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:33:07.785 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:33:07.785 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:07.785 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:07.785 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:11463 3.1d 3:bcad7303:::rbd_data.112d92027590.0000000000000001:f [read 196608~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:33:07.785 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:25.655639+0000", 2024-09-14T06:33:07.785 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 222.08217969699999, 2024-09-14T06:33:07.785 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052888859000000003, 2024-09-14T06:33:07.785 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:33:07.785 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:33:07.785 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:33:07.785 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:33:07.785 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:33:07.785 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:33:07.785 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 11463 2024-09-14T06:33:07.786 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:07.786 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:33:07.786 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:07.786 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:33:07.786 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655639+0000", 2024-09-14T06:33:07.786 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:33:07.786 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:07.786 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:07.786 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:33:07.786 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655639+0000", 2024-09-14T06:33:07.786 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:33:07.786 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:07.786 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:07.786 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:33:07.787 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655640+0000", 2024-09-14T06:33:07.787 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.08e-07 2024-09-14T06:33:07.787 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:07.787 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:07.787 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:33:07.787 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655641+0000", 2024-09-14T06:33:07.787 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.156e-06 2024-09-14T06:33:07.787 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:07.787 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:07.787 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:33:07.787 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655648+0000", 2024-09-14T06:33:07.787 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.8460000000000002e-06 2024-09-14T06:33:07.787 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:07.787 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:07.788 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:33:07.788 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655651+0000", 2024-09-14T06:33:07.788 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.0079999999999998e-06 2024-09-14T06:33:07.788 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:07.788 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:07.788 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:33:07.788 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.708429+0000", 2024-09-14T06:33:07.788 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052778459 2024-09-14T06:33:07.788 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:07.788 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:07.788 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:33:07.788 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.708473+0000", 2024-09-14T06:33:07.788 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.3782999999999997e-05 2024-09-14T06:33:07.788 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:07.789 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:07.789 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:33:07.789 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.708528+0000", 2024-09-14T06:33:07.789 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.5198999999999998e-05 2024-09-14T06:33:07.789 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:33:07.789 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:33:07.789 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:33:07.789 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:07.789 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:07.789 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:11464 3.1d 3:bcad7303:::rbd_data.112d92027590.0000000000000001:f [read 206848~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:33:07.789 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:25.655654+0000", 2024-09-14T06:33:07.789 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 222.082164828, 2024-09-14T06:33:07.789 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052981845999999999, 2024-09-14T06:33:07.789 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:33:07.790 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:33:07.790 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:33:07.790 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:33:07.790 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:33:07.790 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:33:07.790 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 11464 2024-09-14T06:33:07.790 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:07.790 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:33:07.790 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:07.790 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:33:07.790 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655654+0000", 2024-09-14T06:33:07.790 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:33:07.790 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:07.790 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:07.790 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:33:07.791 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655654+0000", 2024-09-14T06:33:07.791 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:33:07.791 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:07.791 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:07.791 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:33:07.791 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655946+0000", 2024-09-14T06:33:07.791 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00029148200000000003 2024-09-14T06:33:07.791 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:07.791 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:07.791 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:33:07.791 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655948+0000", 2024-09-14T06:33:07.791 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.973e-06 2024-09-14T06:33:07.791 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:07.791 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:07.792 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:33:07.792 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655953+0000", 2024-09-14T06:33:07.792 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.1240000000000004e-06 2024-09-14T06:33:07.792 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:07.792 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:07.792 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:33:07.792 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655956+0000", 2024-09-14T06:33:07.792 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.315e-06 2024-09-14T06:33:07.792 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:07.792 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:07.792 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:33:07.792 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.708543+0000", 2024-09-14T06:33:07.792 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052587416999999997 2024-09-14T06:33:07.792 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:07.793 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:07.793 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:33:07.793 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.708584+0000", 2024-09-14T06:33:07.793 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.1004000000000002e-05 2024-09-14T06:33:07.793 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:07.793 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:07.793 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:33:07.793 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.708636+0000", 2024-09-14T06:33:07.793 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.1530999999999998e-05 2024-09-14T06:33:07.793 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:33:07.793 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:33:07.793 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:33:07.793 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:07.793 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:07.793 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:11465 3.1d 3:bcad7303:::rbd_data.112d92027590.0000000000000001:f [read 217088~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:33:07.794 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:25.655961+0000", 2024-09-14T06:33:07.794 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 222.081857676, 2024-09-14T06:33:07.794 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052798682999999999, 2024-09-14T06:33:07.794 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:33:07.794 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:33:07.794 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:33:07.794 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:33:07.794 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:33:07.794 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:33:07.794 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 11465 2024-09-14T06:33:07.794 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:07.794 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:33:07.794 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:07.794 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:33:07.794 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655961+0000", 2024-09-14T06:33:07.795 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:33:07.795 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:07.795 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:07.795 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:33:07.795 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.655961+0000", 2024-09-14T06:33:07.795 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:33:07.795 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:07.795 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:07.795 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:33:07.795 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.656114+0000", 2024-09-14T06:33:07.795 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.000152745 2024-09-14T06:33:07.795 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:07.795 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:07.796 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:33:07.796 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.656116+0000", 2024-09-14T06:33:07.796 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.804e-06 2024-09-14T06:33:07.796 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:07.796 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:07.796 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:33:07.796 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.656120+0000", 2024-09-14T06:33:07.796 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.4700000000000004e-06 2024-09-14T06:33:07.796 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:07.796 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:07.796 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:33:07.796 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.656123+0000", 2024-09-14T06:33:07.796 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.1269999999999999e-06 2024-09-14T06:33:07.797 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:07.797 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:07.797 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:33:07.797 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.708660+0000", 2024-09-14T06:33:07.797 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052536919000000001 2024-09-14T06:33:07.797 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:07.797 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:07.797 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:33:07.797 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.708703+0000", 2024-09-14T06:33:07.797 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.3077000000000001e-05 2024-09-14T06:33:07.797 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:07.797 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:07.797 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:33:07.797 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.708760+0000", 2024-09-14T06:33:07.798 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.6541e-05 2024-09-14T06:33:07.798 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:33:07.798 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:33:07.798 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:33:07.798 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:07.798 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:07.798 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:11466 3.1d 3:bcad7303:::rbd_data.112d92027590.0000000000000001:f [read 227328~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:33:07.798 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:25.656129+0000", 2024-09-14T06:33:07.798 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 222.081690288, 2024-09-14T06:33:07.798 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052750987999999999, 2024-09-14T06:33:07.798 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:33:07.798 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:33:07.798 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:33:07.798 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:33:07.798 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:33:07.799 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:33:07.799 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 11466 2024-09-14T06:33:07.799 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:07.799 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:33:07.799 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:07.799 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:33:07.799 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.656129+0000", 2024-09-14T06:33:07.799 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:33:07.799 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:07.799 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:07.799 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:33:07.799 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.656129+0000", 2024-09-14T06:33:07.799 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:33:07.799 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:07.800 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:07.800 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:33:07.800 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.656347+0000", 2024-09-14T06:33:07.800 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.000218054 2024-09-14T06:33:07.800 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:07.800 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:07.800 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:33:07.800 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.656349+0000", 2024-09-14T06:33:07.800 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.9199999999999998e-06 2024-09-14T06:33:07.800 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:07.800 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:07.800 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:33:07.800 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.656353+0000", 2024-09-14T06:33:07.800 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.9210000000000002e-06 2024-09-14T06:33:07.801 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:07.801 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:07.801 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:33:07.801 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.656356+0000", 2024-09-14T06:33:07.801 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.7670000000000001e-06 2024-09-14T06:33:07.801 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:07.801 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:07.801 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:33:07.801 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.708787+0000", 2024-09-14T06:33:07.801 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052430840999999999 2024-09-14T06:33:07.801 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:07.801 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:07.801 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:33:07.802 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.708827+0000", 2024-09-14T06:33:07.802 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.9538e-05 2024-09-14T06:33:07.802 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:07.802 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:07.802 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:33:07.802 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.708880+0000", 2024-09-14T06:33:07.802 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.2947e-05 2024-09-14T06:33:07.802 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:33:07.802 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:33:07.802 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:33:07.802 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:07.802 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:07.802 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:11467 3.1d 3:bcad7303:::rbd_data.112d92027590.0000000000000001:f [read 237568~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:33:07.802 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:25.656361+0000", 2024-09-14T06:33:07.803 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 222.08145764, 2024-09-14T06:33:07.803 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052643898000000001, 2024-09-14T06:33:07.803 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:33:07.803 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:33:07.803 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:33:07.803 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:33:07.803 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:33:07.803 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:33:07.803 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 11467 2024-09-14T06:33:07.803 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:07.803 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:33:07.803 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:07.803 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:33:07.804 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.656361+0000", 2024-09-14T06:33:07.804 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:33:07.804 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:07.804 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:07.804 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:33:07.804 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.656361+0000", 2024-09-14T06:33:07.804 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:33:07.804 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:07.804 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:07.804 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:33:07.804 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.656855+0000", 2024-09-14T06:33:07.804 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00049372000000000001 2024-09-14T06:33:07.804 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:07.804 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:07.805 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:33:07.805 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.656859+0000", 2024-09-14T06:33:07.805 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.1300000000000003e-06 2024-09-14T06:33:07.805 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:07.805 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:07.805 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:33:07.805 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.656863+0000", 2024-09-14T06:33:07.805 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.2629999999999997e-06 2024-09-14T06:33:07.805 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:07.805 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:07.805 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:33:07.805 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.656867+0000", 2024-09-14T06:33:07.805 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.3220000000000001e-06 2024-09-14T06:33:07.805 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:07.806 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:07.806 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:33:07.806 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.708903+0000", 2024-09-14T06:33:07.806 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052035855999999998 2024-09-14T06:33:07.806 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:07.806 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:07.806 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:33:07.806 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.708949+0000", 2024-09-14T06:33:07.806 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.6276999999999998e-05 2024-09-14T06:33:07.806 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:07.806 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:07.806 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:33:07.806 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.709005+0000", 2024-09-14T06:33:07.806 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.6329999999999998e-05 2024-09-14T06:33:07.807 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:33:07.807 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:33:07.807 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:33:07.807 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:07.807 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:07.807 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:11480 3.1d 3:bcad7303:::rbd_data.112d92027590.0000000000000001:f [read 370688~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:33:07.807 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:25.658563+0000", 2024-09-14T06:33:07.807 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 222.07925587599999, 2024-09-14T06:33:07.807 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052429164, 2024-09-14T06:33:07.807 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:33:07.807 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:33:07.807 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:33:07.807 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:33:07.807 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:33:07.807 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:33:07.808 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 11480 2024-09-14T06:33:07.808 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:07.808 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:33:07.808 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:07.808 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:33:07.808 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.658563+0000", 2024-09-14T06:33:07.808 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:33:07.808 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:07.808 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:07.808 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:33:07.808 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.658563+0000", 2024-09-14T06:33:07.808 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:33:07.808 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:07.808 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:07.808 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:33:07.809 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.658658+0000", 2024-09-14T06:33:07.809 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.5346999999999994e-05 2024-09-14T06:33:07.809 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:07.809 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:07.809 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:33:07.809 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.658662+0000", 2024-09-14T06:33:07.809 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.14e-06 2024-09-14T06:33:07.809 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:07.809 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:07.809 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:33:07.809 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.658669+0000", 2024-09-14T06:33:07.809 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.7379999999999994e-06 2024-09-14T06:33:07.809 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:07.809 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:07.810 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:33:07.810 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.658675+0000", 2024-09-14T06:33:07.810 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.5430000000000001e-06 2024-09-14T06:33:07.810 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:07.810 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:07.810 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:33:07.810 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.710824+0000", 2024-09-14T06:33:07.810 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052148847999999998 2024-09-14T06:33:07.810 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:07.810 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:07.810 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:33:07.810 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.710864+0000", 2024-09-14T06:33:07.810 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.9925999999999997e-05 2024-09-14T06:33:07.811 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:07.811 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:07.811 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:33:07.811 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.710992+0000", 2024-09-14T06:33:07.811 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.000128622 2024-09-14T06:33:07.811 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:33:07.811 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:33:07.811 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:33:07.811 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:07.811 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:07.811 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:11481 3.1d 3:bcad7303:::rbd_data.112d92027590.0000000000000001:f [read 380928~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:33:07.811 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:25.658684+0000", 2024-09-14T06:33:07.811 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 222.07913480600001, 2024-09-14T06:33:07.811 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052508565, 2024-09-14T06:33:07.812 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:33:07.812 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:33:07.812 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:33:07.812 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:33:07.812 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:33:07.812 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:33:07.812 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 11481 2024-09-14T06:33:07.812 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:07.812 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:33:07.812 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:07.812 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:33:07.812 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.658684+0000", 2024-09-14T06:33:07.812 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:33:07.813 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:07.813 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:07.813 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:33:07.813 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.658684+0000", 2024-09-14T06:33:07.813 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:33:07.813 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:07.813 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:07.813 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:33:07.813 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.658715+0000", 2024-09-14T06:33:07.813 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.0703000000000001e-05 2024-09-14T06:33:07.813 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:07.813 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:07.813 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:33:07.814 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.658718+0000", 2024-09-14T06:33:07.814 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.0709999999999999e-06 2024-09-14T06:33:07.814 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:07.814 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:07.814 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:33:07.814 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.658725+0000", 2024-09-14T06:33:07.814 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.2339999999999997e-06 2024-09-14T06:33:07.814 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:07.814 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:07.814 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:33:07.814 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.658731+0000", 2024-09-14T06:33:07.814 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.6450000000000002e-06 2024-09-14T06:33:07.814 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:07.814 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:07.814 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:33:07.815 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.711020+0000", 2024-09-14T06:33:07.815 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052288886 2024-09-14T06:33:07.815 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:07.815 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:07.815 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:33:07.815 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.711061+0000", 2024-09-14T06:33:07.815 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.1084000000000002e-05 2024-09-14T06:33:07.815 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:07.815 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:07.815 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:33:07.815 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.711193+0000", 2024-09-14T06:33:07.815 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00013194199999999999 2024-09-14T06:33:07.815 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:33:07.815 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:33:07.816 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:33:07.816 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:07.816 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:07.816 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:11482 3.1d 3:bcad7303:::rbd_data.112d92027590.0000000000000001:f [read 391168~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:33:07.816 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:25.658740+0000", 2024-09-14T06:33:07.816 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 222.07907934100001, 2024-09-14T06:33:07.816 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052637015000000002, 2024-09-14T06:33:07.816 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:33:07.816 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:33:07.816 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:33:07.816 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:33:07.816 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:33:07.816 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:33:07.816 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 11482 2024-09-14T06:33:07.817 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:07.817 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:33:07.817 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:07.817 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:33:07.817 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.658740+0000", 2024-09-14T06:33:07.817 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:33:07.817 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:07.817 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:07.817 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:33:07.817 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.658740+0000", 2024-09-14T06:33:07.817 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:33:07.817 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:07.817 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:07.817 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:33:07.817 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.658880+0000", 2024-09-14T06:33:07.818 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00014036100000000001 2024-09-14T06:33:07.818 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:07.818 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:07.818 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:33:07.818 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.658883+0000", 2024-09-14T06:33:07.818 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.3969999999999998e-06 2024-09-14T06:33:07.818 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:07.818 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:07.818 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:33:07.818 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.658890+0000", 2024-09-14T06:33:07.818 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.1500000000000002e-06 2024-09-14T06:33:07.818 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:07.818 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:07.818 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:33:07.818 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.658897+0000", 2024-09-14T06:33:07.819 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.1260000000000003e-06 2024-09-14T06:33:07.819 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:07.819 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:07.819 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:33:07.819 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.711207+0000", 2024-09-14T06:33:07.819 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052310777000000003 2024-09-14T06:33:07.819 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:07.819 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:07.819 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:33:07.819 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.711249+0000", 2024-09-14T06:33:07.819 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.1786999999999998e-05 2024-09-14T06:33:07.819 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:07.819 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:07.819 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:33:07.819 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.711377+0000", 2024-09-14T06:33:07.820 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.000127417 2024-09-14T06:33:07.820 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:33:07.820 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:33:07.820 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:33:07.820 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:07.820 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:07.820 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:11483 3.1d 3:bcad7303:::rbd_data.112d92027590.0000000000000001:f [read 401408~1024 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-14T06:33:07.820 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:25.658906+0000", 2024-09-14T06:33:07.820 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 222.078912557, 2024-09-14T06:33:07.820 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.052595109000000001, 2024-09-14T06:33:07.820 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:33:07.820 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:33:07.820 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:33:07.821 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:33:07.821 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:33:07.821 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:33:07.821 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 11483 2024-09-14T06:33:07.821 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:07.821 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:33:07.821 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:07.821 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:33:07.821 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.658906+0000", 2024-09-14T06:33:07.821 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:33:07.821 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:07.821 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:07.821 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:33:07.821 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.658906+0000", 2024-09-14T06:33:07.822 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:33:07.822 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:07.822 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:07.822 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:33:07.822 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.664985+0000", 2024-09-14T06:33:07.822 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.0060787250000000001 2024-09-14T06:33:07.822 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:07.822 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:07.822 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:33:07.822 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.664991+0000", 2024-09-14T06:33:07.822 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.7030000000000003e-06 2024-09-14T06:33:07.822 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:07.822 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:07.822 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:33:07.822 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.664999+0000", 2024-09-14T06:33:07.823 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.4430000000000003e-06 2024-09-14T06:33:07.823 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:07.823 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:07.823 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:33:07.823 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.665006+0000", 2024-09-14T06:33:07.823 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.0840000000000003e-06 2024-09-14T06:33:07.823 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:07.823 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:07.823 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:33:07.823 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.711402+0000", 2024-09-14T06:33:07.823 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046395714999999997 2024-09-14T06:33:07.823 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:07.823 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:07.824 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:33:07.824 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.711443+0000", 2024-09-14T06:33:07.824 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.0911999999999997e-05 2024-09-14T06:33:07.824 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:07.824 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:07.824 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:33:07.824 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:25.711501+0000", 2024-09-14T06:33:07.824 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.8526999999999997e-05 2024-09-14T06:33:07.824 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:33:07.824 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:33:07.824 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:33:07.824 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:07.824 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:07.824 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:214569 3.2 3:479633bd:::rbd_object_map.112d3bd84cb0:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e170)", 2024-09-14T06:33:07.824 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:31:34.933901+0000", 2024-09-14T06:33:07.825 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 92.803918173, 2024-09-14T06:33:07.825 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.68157511400000004, 2024-09-14T06:33:07.825 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:33:07.825 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:33:07.825 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:33:07.825 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:33:07.825 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:33:07.825 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:33:07.825 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 214569 2024-09-14T06:33:07.825 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:07.825 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:33:07.825 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:07.825 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:33:07.825 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:31:34.933901+0000", 2024-09-14T06:33:07.825 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:33:07.826 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:07.826 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:07.826 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:33:07.826 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:31:34.933901+0000", 2024-09-14T06:33:07.826 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:33:07.826 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:07.826 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:07.826 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:33:07.826 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:31:34.933902+0000", 2024-09-14T06:33:07.826 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.7439999999999999e-06 2024-09-14T06:33:07.826 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:07.826 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:07.826 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:33:07.826 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:31:34.933910+0000", 2024-09-14T06:33:07.827 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.8569999999999995e-06 2024-09-14T06:33:07.827 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:07.827 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:07.827 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:33:07.827 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:31:34.933932+0000", 2024-09-14T06:33:07.827 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.1997999999999998e-05 2024-09-14T06:33:07.827 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:07.827 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:07.827 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:33:07.827 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:31:34.933941+0000", 2024-09-14T06:33:07.827 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.7180000000000002e-06 2024-09-14T06:33:07.827 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:07.827 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:07.827 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:33:07.827 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:31:34.934013+0000", 2024-09-14T06:33:07.828 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.1761000000000006e-05 2024-09-14T06:33:07.828 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:07.828 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:07.828 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "waiting for peered", 2024-09-14T06:33:07.828 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:31:34.934022+0000", 2024-09-14T06:33:07.828 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.8740000000000001e-06 2024-09-14T06:33:07.828 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:07.828 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:07.828 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:33:07.828 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:31:35.608803+0000", 2024-09-14T06:33:07.828 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.67478164299999999 2024-09-14T06:33:07.828 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:07.828 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:07.828 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:33:07.829 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:31:35.608964+0000", 2024-09-14T06:33:07.829 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00016112900000000001 2024-09-14T06:33:07.829 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:07.829 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:07.829 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "waiting for subops from 6", 2024-09-14T06:33:07.829 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:31:35.609399+0000", 2024-09-14T06:33:07.829 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00043454000000000002 2024-09-14T06:33:07.829 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:07.829 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:07.829 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "op_commit", 2024-09-14T06:33:07.829 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:31:35.610495+0000", 2024-09-14T06:33:07.829 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.001096404 2024-09-14T06:33:07.829 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:07.829 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:07.829 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:33:07.829 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:31:35.615409+0000", 2024-09-14T06:33:07.830 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.0049133199999999997 2024-09-14T06:33:07.830 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:07.830 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:07.830 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "commit_sent", 2024-09-14T06:33:07.830 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:31:35.615444+0000", 2024-09-14T06:33:07.830 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.5457000000000003e-05 2024-09-14T06:33:07.830 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:07.830 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:07.830 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:33:07.830 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:31:35.615476+0000", 2024-09-14T06:33:07.830 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.1668999999999998e-05 2024-09-14T06:33:07.830 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:33:07.830 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:33:07.830 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:33:07.831 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:33:07.831 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:33:07.831 INFO:teuthology.orchestra.run.smithi138.stdout:} 2024-09-14T06:33:07.832 INFO:teuthology.orchestra.run.smithi094.stdout:All operations completed A-OK! 2024-09-14T06:33:07.833 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-14T06:33:07.833 DEBUG:teuthology.parallel:result is None 2024-09-14T06:33:07.834 DEBUG:teuthology.run_tasks:Unwinding manager rbd_fsx 2024-09-14T06:33:07.844 DEBUG:teuthology.run_tasks:Unwinding manager thrashosds 2024-09-14T06:33:07.853 INFO:tasks.thrashosds:joining thrashosds 2024-09-14T06:33:07.977 INFO:teuthology.orchestra.run.smithi138.stdout:{ 2024-09-14T06:33:07.978 INFO:teuthology.orchestra.run.smithi138.stdout: "ops": [], 2024-09-14T06:33:07.978 INFO:teuthology.orchestra.run.smithi138.stdout: "num_ops": 0 2024-09-14T06:33:07.978 INFO:teuthology.orchestra.run.smithi138.stdout:} 2024-09-14T06:33:07.986 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-14T06:33:08.088 INFO:teuthology.orchestra.run.smithi138.stdout:{ 2024-09-14T06:33:08.089 INFO:teuthology.orchestra.run.smithi138.stdout: "ops": [], 2024-09-14T06:33:08.089 INFO:teuthology.orchestra.run.smithi138.stdout: "complaint_time": 30, 2024-09-14T06:33:08.089 INFO:teuthology.orchestra.run.smithi138.stdout: "num_blocked_ops": 0 2024-09-14T06:33:08.089 INFO:teuthology.orchestra.run.smithi138.stdout:} 2024-09-14T06:33:08.098 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-14T06:33:08.201 INFO:teuthology.orchestra.run.smithi138.stdout:{ 2024-09-14T06:33:08.202 INFO:teuthology.orchestra.run.smithi138.stdout: "size": 20, 2024-09-14T06:33:08.202 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 600, 2024-09-14T06:33:08.202 INFO:teuthology.orchestra.run.smithi138.stdout: "ops": [ 2024-09-14T06:33:08.202 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:08.202 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:98505 3.10 3.1bf56f90 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T06:33:08.202 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:30:07.045453+0000", 2024-09-14T06:33:08.202 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 181.154516433, 2024-09-14T06:33:08.202 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032144440000000003, 2024-09-14T06:33:08.202 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:33:08.202 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:33:08.202 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "reached pg", 2024-09-14T06:33:08.202 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:33:08.202 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:33:08.203 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:33:08.203 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 98505 2024-09-14T06:33:08.203 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:08.203 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:33:08.203 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:08.203 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:33:08.203 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045453+0000", 2024-09-14T06:33:08.203 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:33:08.203 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:08.203 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:08.203 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:33:08.203 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045453+0000", 2024-09-14T06:33:08.203 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:33:08.203 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:08.204 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:08.204 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:33:08.204 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045453+0000", 2024-09-14T06:33:08.204 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.36e-07 2024-09-14T06:33:08.204 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:08.204 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:08.204 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:33:08.204 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045454+0000", 2024-09-14T06:33:08.204 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.1299999999999998e-07 2024-09-14T06:33:08.204 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:08.204 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:08.204 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:33:08.204 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045457+0000", 2024-09-14T06:33:08.204 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.8559999999999998e-06 2024-09-14T06:33:08.205 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:08.205 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:08.205 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:33:08.205 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045460+0000", 2024-09-14T06:33:08.205 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.8750000000000002e-06 2024-09-14T06:33:08.205 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:08.205 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:08.205 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:33:08.205 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.077592+0000", 2024-09-14T06:33:08.205 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032131727999999998 2024-09-14T06:33:08.205 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:08.205 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:08.205 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:33:08.205 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.077597+0000", 2024-09-14T06:33:08.205 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.8319999999999996e-06 2024-09-14T06:33:08.206 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:33:08.206 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:33:08.206 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:33:08.206 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:08.206 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:08.206 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:98506 3.10 3.1bf56f90 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T06:33:08.206 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:30:07.045481+0000", 2024-09-14T06:33:08.206 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 181.15448817800001, 2024-09-14T06:33:08.206 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032142759999999999, 2024-09-14T06:33:08.206 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:33:08.206 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:33:08.206 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "reached pg", 2024-09-14T06:33:08.206 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:33:08.206 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:33:08.207 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:33:08.207 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 98506 2024-09-14T06:33:08.207 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:08.207 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:33:08.207 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:08.207 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:33:08.207 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045481+0000", 2024-09-14T06:33:08.207 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:33:08.207 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:08.207 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:08.207 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:33:08.207 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045481+0000", 2024-09-14T06:33:08.207 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:33:08.207 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:08.208 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:08.208 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:33:08.208 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045481+0000", 2024-09-14T06:33:08.208 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.8500000000000001e-07 2024-09-14T06:33:08.208 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:08.208 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:08.208 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:33:08.208 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045483+0000", 2024-09-14T06:33:08.208 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.8250000000000001e-06 2024-09-14T06:33:08.208 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:08.208 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:08.208 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:33:08.208 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045489+0000", 2024-09-14T06:33:08.208 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.5740000000000003e-06 2024-09-14T06:33:08.209 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:08.209 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:08.209 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:33:08.209 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045495+0000", 2024-09-14T06:33:08.209 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.7690000000000002e-06 2024-09-14T06:33:08.209 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:08.209 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:08.209 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:33:08.209 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.077619+0000", 2024-09-14T06:33:08.209 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032124621999999999 2024-09-14T06:33:08.209 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:08.209 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:08.209 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:33:08.209 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.077624+0000", 2024-09-14T06:33:08.210 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.3850000000000004e-06 2024-09-14T06:33:08.210 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:33:08.210 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:33:08.210 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:33:08.210 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:08.210 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:08.210 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:98507 3.10 3.1bf56f90 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T06:33:08.210 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:30:07.045502+0000", 2024-09-14T06:33:08.210 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 181.15446675699999, 2024-09-14T06:33:08.210 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032147219999999997, 2024-09-14T06:33:08.210 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:33:08.210 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:33:08.210 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "reached pg", 2024-09-14T06:33:08.211 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:33:08.211 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:33:08.211 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:33:08.211 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 98507 2024-09-14T06:33:08.211 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:08.211 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:33:08.211 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:08.211 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:33:08.211 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045502+0000", 2024-09-14T06:33:08.211 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:33:08.211 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:08.211 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:08.211 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:33:08.211 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045502+0000", 2024-09-14T06:33:08.212 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:33:08.212 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:08.212 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:08.212 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:33:08.212 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045503+0000", 2024-09-14T06:33:08.212 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.4700000000000002e-07 2024-09-14T06:33:08.212 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:08.212 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:08.212 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:33:08.212 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045509+0000", 2024-09-14T06:33:08.212 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.7329999999999997e-06 2024-09-14T06:33:08.212 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:08.212 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:08.212 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:33:08.212 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045515+0000", 2024-09-14T06:33:08.213 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.8030000000000002e-06 2024-09-14T06:33:08.213 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:08.213 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:08.213 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:33:08.213 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045522+0000", 2024-09-14T06:33:08.213 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.0369999999999997e-06 2024-09-14T06:33:08.213 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:08.213 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:08.213 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:33:08.213 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.077641+0000", 2024-09-14T06:33:08.213 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032119091000000002 2024-09-14T06:33:08.213 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:08.213 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:08.213 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:33:08.214 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.077650+0000", 2024-09-14T06:33:08.214 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.2090000000000005e-06 2024-09-14T06:33:08.214 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:33:08.214 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:33:08.214 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:33:08.214 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:08.214 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:08.214 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:98508 3.10 3.1bf56f90 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T06:33:08.214 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:30:07.045530+0000", 2024-09-14T06:33:08.214 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 181.15443913999999, 2024-09-14T06:33:08.214 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032198773999999999, 2024-09-14T06:33:08.214 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:33:08.214 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:33:08.214 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "reached pg", 2024-09-14T06:33:08.214 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:33:08.215 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:33:08.215 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:33:08.215 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 98508 2024-09-14T06:33:08.215 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:08.215 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:33:08.215 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:08.215 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:33:08.215 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045530+0000", 2024-09-14T06:33:08.215 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:33:08.215 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:08.215 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:08.215 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:33:08.215 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045530+0000", 2024-09-14T06:33:08.215 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:33:08.215 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:08.216 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:08.216 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:33:08.216 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045530+0000", 2024-09-14T06:33:08.216 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.3099999999999998e-07 2024-09-14T06:33:08.216 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:08.216 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:08.216 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:33:08.216 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045539+0000", 2024-09-14T06:33:08.216 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.4780000000000008e-06 2024-09-14T06:33:08.216 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:08.216 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:08.216 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:33:08.216 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045546+0000", 2024-09-14T06:33:08.217 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.3259999999999998e-06 2024-09-14T06:33:08.217 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:08.217 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:08.217 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:33:08.217 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045551+0000", 2024-09-14T06:33:08.217 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.9960000000000001e-06 2024-09-14T06:33:08.217 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:08.217 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:08.217 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:33:08.217 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.077716+0000", 2024-09-14T06:33:08.217 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032165081999999998 2024-09-14T06:33:08.217 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:08.217 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:08.217 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:33:08.217 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.077729+0000", 2024-09-14T06:33:08.218 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.2461e-05 2024-09-14T06:33:08.218 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:33:08.218 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:33:08.218 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:33:08.218 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:08.218 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:08.218 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:98509 3.10 3.1bf56f90 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T06:33:08.218 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:30:07.045558+0000", 2024-09-14T06:33:08.218 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 181.154410939, 2024-09-14T06:33:08.218 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032242198, 2024-09-14T06:33:08.218 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:33:08.218 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:33:08.218 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "reached pg", 2024-09-14T06:33:08.218 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:33:08.219 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:33:08.219 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:33:08.219 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 98509 2024-09-14T06:33:08.219 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:08.219 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:33:08.219 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:08.219 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:33:08.219 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045558+0000", 2024-09-14T06:33:08.219 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:33:08.219 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:08.219 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:08.219 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:33:08.219 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045558+0000", 2024-09-14T06:33:08.219 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:33:08.219 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:08.220 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:08.220 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:33:08.220 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045559+0000", 2024-09-14T06:33:08.220 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.27e-07 2024-09-14T06:33:08.220 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:08.220 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:08.220 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:33:08.220 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045560+0000", 2024-09-14T06:33:08.220 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.6959999999999999e-06 2024-09-14T06:33:08.220 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:08.220 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:08.220 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:33:08.220 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045567+0000", 2024-09-14T06:33:08.220 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.6900000000000003e-06 2024-09-14T06:33:08.221 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:08.221 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:08.221 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:33:08.221 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045571+0000", 2024-09-14T06:33:08.221 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.5750000000000002e-06 2024-09-14T06:33:08.221 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:08.221 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:08.221 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:33:08.221 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.077789+0000", 2024-09-14T06:33:08.221 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032217186000000002 2024-09-14T06:33:08.221 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:08.221 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:08.221 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:33:08.221 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.077800+0000", 2024-09-14T06:33:08.222 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.1623999999999999e-05 2024-09-14T06:33:08.222 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:33:08.222 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:33:08.222 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:33:08.222 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:08.222 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:08.222 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:98510 3.10 3.1bf56f90 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T06:33:08.222 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:30:07.045580+0000", 2024-09-14T06:33:08.222 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 181.15438912499999, 2024-09-14T06:33:08.222 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032299845000000001, 2024-09-14T06:33:08.222 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:33:08.222 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:33:08.222 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "reached pg", 2024-09-14T06:33:08.222 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:33:08.222 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:33:08.223 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:33:08.223 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 98510 2024-09-14T06:33:08.223 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:08.223 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:33:08.223 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:08.223 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:33:08.223 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045580+0000", 2024-09-14T06:33:08.223 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:33:08.223 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:08.223 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:08.223 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:33:08.223 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045580+0000", 2024-09-14T06:33:08.223 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:33:08.223 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:08.223 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:08.224 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:33:08.224 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045580+0000", 2024-09-14T06:33:08.224 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.0400000000000002e-07 2024-09-14T06:33:08.224 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:08.224 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:08.224 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:33:08.224 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045582+0000", 2024-09-14T06:33:08.224 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.7600000000000001e-06 2024-09-14T06:33:08.224 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:08.224 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:08.224 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:33:08.224 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045588+0000", 2024-09-14T06:33:08.224 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.5650000000000001e-06 2024-09-14T06:33:08.224 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:08.225 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:08.225 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:33:08.225 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045597+0000", 2024-09-14T06:33:08.225 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.465e-06 2024-09-14T06:33:08.225 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:08.225 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:08.225 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:33:08.225 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.077871+0000", 2024-09-14T06:33:08.225 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032273562999999998 2024-09-14T06:33:08.225 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:08.225 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:08.225 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:33:08.225 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.077880+0000", 2024-09-14T06:33:08.225 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.0880000000000001e-06 2024-09-14T06:33:08.225 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:33:08.226 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:33:08.226 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:33:08.226 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:08.226 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:08.226 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:98511 3.10 3.1bf56f90 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T06:33:08.226 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:30:07.045610+0000", 2024-09-14T06:33:08.226 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 181.15435927799999, 2024-09-14T06:33:08.226 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032338894, 2024-09-14T06:33:08.226 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:33:08.226 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:33:08.226 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "reached pg", 2024-09-14T06:33:08.226 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:33:08.226 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:33:08.226 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:33:08.227 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 98511 2024-09-14T06:33:08.227 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:08.227 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:33:08.227 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:08.227 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:33:08.227 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045610+0000", 2024-09-14T06:33:08.227 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:33:08.227 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:08.227 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:08.227 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:33:08.227 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045610+0000", 2024-09-14T06:33:08.227 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:33:08.227 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:08.227 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:08.227 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:33:08.228 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045610+0000", 2024-09-14T06:33:08.228 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.1600000000000002e-07 2024-09-14T06:33:08.228 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:08.228 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:08.228 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:33:08.228 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045612+0000", 2024-09-14T06:33:08.228 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.1150000000000001e-06 2024-09-14T06:33:08.228 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:08.228 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:08.228 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:33:08.228 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045618+0000", 2024-09-14T06:33:08.228 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.0499999999999997e-06 2024-09-14T06:33:08.228 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:08.229 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:08.229 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:33:08.229 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045623+0000", 2024-09-14T06:33:08.229 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.6770000000000003e-06 2024-09-14T06:33:08.229 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:08.229 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:08.229 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:33:08.229 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.077928+0000", 2024-09-14T06:33:08.229 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032305111999999997 2024-09-14T06:33:08.229 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:08.229 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:08.229 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:33:08.229 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.077949+0000", 2024-09-14T06:33:08.229 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.0523999999999999e-05 2024-09-14T06:33:08.230 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:33:08.230 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:33:08.230 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:33:08.230 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:08.230 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:08.230 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:98512 3.10 3.1bf56f90 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T06:33:08.230 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:30:07.045630+0000", 2024-09-14T06:33:08.230 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 181.15433927300001, 2024-09-14T06:33:08.230 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032346632, 2024-09-14T06:33:08.230 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:33:08.230 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:33:08.230 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "reached pg", 2024-09-14T06:33:08.230 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:33:08.230 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:33:08.230 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:33:08.231 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 98512 2024-09-14T06:33:08.231 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:08.231 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:33:08.231 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:08.231 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:33:08.231 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045630+0000", 2024-09-14T06:33:08.231 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:33:08.231 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:08.231 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:08.231 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:33:08.231 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045630+0000", 2024-09-14T06:33:08.231 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:33:08.231 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:08.231 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:08.231 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:33:08.232 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045630+0000", 2024-09-14T06:33:08.232 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.8200000000000001e-07 2024-09-14T06:33:08.232 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:08.232 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:08.232 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:33:08.232 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045632+0000", 2024-09-14T06:33:08.232 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.7430000000000001e-06 2024-09-14T06:33:08.232 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:08.232 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:08.232 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:33:08.232 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045637+0000", 2024-09-14T06:33:08.232 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.4550000000000003e-06 2024-09-14T06:33:08.232 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:08.232 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:08.233 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:33:08.233 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045644+0000", 2024-09-14T06:33:08.233 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.1330000000000001e-06 2024-09-14T06:33:08.233 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:08.233 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:08.233 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:33:08.233 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.077968+0000", 2024-09-14T06:33:08.233 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032323401000000002 2024-09-14T06:33:08.233 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:08.233 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:08.233 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:33:08.233 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.077976+0000", 2024-09-14T06:33:08.233 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.5180000000000004e-06 2024-09-14T06:33:08.233 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:33:08.233 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:33:08.234 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:33:08.234 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:08.234 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:08.234 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:98513 3.10 3.1bf56f90 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T06:33:08.234 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:30:07.045652+0000", 2024-09-14T06:33:08.234 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 181.154316709, 2024-09-14T06:33:08.234 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032353436999999999, 2024-09-14T06:33:08.234 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:33:08.234 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:33:08.234 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "reached pg", 2024-09-14T06:33:08.234 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:33:08.234 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:33:08.234 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:33:08.234 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 98513 2024-09-14T06:33:08.235 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:08.235 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:33:08.235 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:08.235 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:33:08.235 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045652+0000", 2024-09-14T06:33:08.235 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:33:08.235 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:08.235 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:08.235 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:33:08.235 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045652+0000", 2024-09-14T06:33:08.235 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:33:08.235 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:08.236 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:08.236 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:33:08.236 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045653+0000", 2024-09-14T06:33:08.236 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.3700000000000001e-07 2024-09-14T06:33:08.236 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:08.236 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:08.236 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:33:08.236 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045655+0000", 2024-09-14T06:33:08.236 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.854e-06 2024-09-14T06:33:08.236 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:08.236 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:08.236 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:33:08.236 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045661+0000", 2024-09-14T06:33:08.236 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.5509999999999996e-06 2024-09-14T06:33:08.237 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:08.237 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:08.237 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:33:08.237 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045668+0000", 2024-09-14T06:33:08.237 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.1980000000000004e-06 2024-09-14T06:33:08.237 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:08.237 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:08.237 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:33:08.237 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.077997+0000", 2024-09-14T06:33:08.237 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032328575999999998 2024-09-14T06:33:08.237 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:08.237 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:08.237 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:33:08.237 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.078006+0000", 2024-09-14T06:33:08.238 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.8210000000000008e-06 2024-09-14T06:33:08.238 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:33:08.238 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:33:08.238 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:33:08.238 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:08.238 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:08.238 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:98514 3.10 3.1bf56f90 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T06:33:08.238 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:30:07.045699+0000", 2024-09-14T06:33:08.238 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 181.15427037399999, 2024-09-14T06:33:08.238 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032330223999999998, 2024-09-14T06:33:08.238 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:33:08.238 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:33:08.238 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "reached pg", 2024-09-14T06:33:08.238 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:33:08.238 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:33:08.239 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:33:08.239 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 98514 2024-09-14T06:33:08.239 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:08.239 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:33:08.239 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:08.239 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:33:08.239 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045699+0000", 2024-09-14T06:33:08.239 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:33:08.239 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:08.239 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:08.239 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:33:08.239 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045699+0000", 2024-09-14T06:33:08.239 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:33:08.239 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:08.240 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:08.240 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:33:08.240 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045699+0000", 2024-09-14T06:33:08.240 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.3200000000000001e-07 2024-09-14T06:33:08.240 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:08.240 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:08.240 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:33:08.240 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045701+0000", 2024-09-14T06:33:08.240 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.762e-06 2024-09-14T06:33:08.240 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:08.240 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:08.240 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:33:08.240 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045705+0000", 2024-09-14T06:33:08.240 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.2359999999999998e-06 2024-09-14T06:33:08.241 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:08.241 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:08.241 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:33:08.241 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045708+0000", 2024-09-14T06:33:08.241 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.796e-06 2024-09-14T06:33:08.241 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:08.241 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:08.241 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:33:08.241 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.078023+0000", 2024-09-14T06:33:08.241 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032315213000000002 2024-09-14T06:33:08.241 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:08.241 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:08.241 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:33:08.241 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.078029+0000", 2024-09-14T06:33:08.242 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.8850000000000004e-06 2024-09-14T06:33:08.242 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:33:08.242 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:33:08.242 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:33:08.242 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:08.242 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:08.242 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:98515 3.10 3.1bf56f90 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T06:33:08.242 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:30:07.045712+0000", 2024-09-14T06:33:08.242 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 181.15425706299999, 2024-09-14T06:33:08.242 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032346871999999999, 2024-09-14T06:33:08.242 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:33:08.242 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:33:08.242 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "reached pg", 2024-09-14T06:33:08.242 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:33:08.243 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:33:08.243 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:33:08.243 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 98515 2024-09-14T06:33:08.243 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:08.243 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:33:08.243 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:08.243 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:33:08.243 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045712+0000", 2024-09-14T06:33:08.243 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:33:08.243 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:08.243 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:08.243 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:33:08.243 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045712+0000", 2024-09-14T06:33:08.243 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:33:08.243 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:08.244 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:08.244 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:33:08.244 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045712+0000", 2024-09-14T06:33:08.244 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.2100000000000001e-07 2024-09-14T06:33:08.244 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:08.244 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:08.244 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:33:08.244 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045713+0000", 2024-09-14T06:33:08.244 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.068e-06 2024-09-14T06:33:08.244 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:08.244 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:08.244 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:33:08.244 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045716+0000", 2024-09-14T06:33:08.244 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.971e-06 2024-09-14T06:33:08.245 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:08.245 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:08.245 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:33:08.245 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045719+0000", 2024-09-14T06:33:08.245 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.3879999999999998e-06 2024-09-14T06:33:08.245 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:08.245 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:08.245 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:33:08.245 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.078049+0000", 2024-09-14T06:33:08.245 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032330553999999997 2024-09-14T06:33:08.245 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:08.245 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:08.245 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:33:08.245 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.078059+0000", 2024-09-14T06:33:08.246 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.6700000000000006e-06 2024-09-14T06:33:08.246 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:33:08.246 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:33:08.246 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:33:08.246 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:08.246 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:08.246 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:98516 3.10 3.1bf56f90 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T06:33:08.246 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:30:07.045722+0000", 2024-09-14T06:33:08.246 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 181.15424730999999, 2024-09-14T06:33:08.246 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032374334999999997, 2024-09-14T06:33:08.246 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:33:08.246 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:33:08.246 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "reached pg", 2024-09-14T06:33:08.246 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:33:08.246 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:33:08.247 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:33:08.247 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 98516 2024-09-14T06:33:08.247 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:08.247 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:33:08.247 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:08.247 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:33:08.247 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045722+0000", 2024-09-14T06:33:08.247 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:33:08.247 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:08.247 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:08.247 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:33:08.247 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045722+0000", 2024-09-14T06:33:08.247 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:33:08.247 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:08.248 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:08.248 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:33:08.248 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045722+0000", 2024-09-14T06:33:08.248 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.8999999999999998e-07 2024-09-14T06:33:08.248 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:08.248 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:08.248 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:33:08.248 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045723+0000", 2024-09-14T06:33:08.248 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.0049999999999999e-06 2024-09-14T06:33:08.248 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:08.248 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:08.248 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:33:08.248 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045726+0000", 2024-09-14T06:33:08.248 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.9019999999999999e-06 2024-09-14T06:33:08.249 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:08.249 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:08.249 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:33:08.249 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045728+0000", 2024-09-14T06:33:08.249 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.3290000000000001e-06 2024-09-14T06:33:08.249 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:08.249 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:08.249 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:33:08.249 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.078090+0000", 2024-09-14T06:33:08.249 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032362123 2024-09-14T06:33:08.249 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:08.249 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:08.249 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:33:08.249 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.078096+0000", 2024-09-14T06:33:08.250 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.6860000000000003e-06 2024-09-14T06:33:08.250 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:33:08.250 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:33:08.250 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:33:08.250 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:08.250 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:08.250 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:98517 3.10 3.1bf56f90 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T06:33:08.250 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:30:07.045731+0000", 2024-09-14T06:33:08.250 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 181.15423757900001, 2024-09-14T06:33:08.250 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.03241227, 2024-09-14T06:33:08.250 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:33:08.250 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:33:08.250 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "reached pg", 2024-09-14T06:33:08.251 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:33:08.251 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:33:08.251 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:33:08.251 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 98517 2024-09-14T06:33:08.251 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:08.251 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:33:08.251 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:08.251 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:33:08.251 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045731+0000", 2024-09-14T06:33:08.251 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:33:08.251 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:08.251 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:08.251 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:33:08.251 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045731+0000", 2024-09-14T06:33:08.252 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:33:08.252 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:08.252 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:08.252 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:33:08.252 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045732+0000", 2024-09-14T06:33:08.252 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.0699999999999999e-07 2024-09-14T06:33:08.252 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:08.252 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:08.252 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:33:08.252 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045733+0000", 2024-09-14T06:33:08.252 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.9199999999999999e-07 2024-09-14T06:33:08.252 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:08.252 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:08.252 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:33:08.253 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045735+0000", 2024-09-14T06:33:08.253 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.8150000000000002e-06 2024-09-14T06:33:08.253 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:08.253 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:08.253 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:33:08.253 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045738+0000", 2024-09-14T06:33:08.253 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.3640000000000002e-06 2024-09-14T06:33:08.253 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:08.253 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:08.253 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:33:08.253 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.078126+0000", 2024-09-14T06:33:08.253 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032388206000000003 2024-09-14T06:33:08.253 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:08.254 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:08.254 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:33:08.254 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.078144+0000", 2024-09-14T06:33:08.254 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.7686e-05 2024-09-14T06:33:08.254 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:33:08.254 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:33:08.254 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:33:08.254 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:08.254 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:08.254 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:98518 3.10 3.1bf56f90 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T06:33:08.254 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:30:07.045741+0000", 2024-09-14T06:33:08.254 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 181.154228184, 2024-09-14T06:33:08.254 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032451793999999999, 2024-09-14T06:33:08.254 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:33:08.254 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:33:08.255 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "reached pg", 2024-09-14T06:33:08.255 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:33:08.255 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:33:08.255 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:33:08.255 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 98518 2024-09-14T06:33:08.255 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:08.255 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:33:08.255 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:08.255 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:33:08.255 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045741+0000", 2024-09-14T06:33:08.255 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:33:08.255 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:08.255 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:08.255 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:33:08.256 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045741+0000", 2024-09-14T06:33:08.256 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:33:08.256 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:08.256 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:08.256 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:33:08.256 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045741+0000", 2024-09-14T06:33:08.256 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.05e-07 2024-09-14T06:33:08.256 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:08.256 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:08.256 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:33:08.256 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045746+0000", 2024-09-14T06:33:08.256 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.2259999999999996e-06 2024-09-14T06:33:08.256 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:08.257 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:08.257 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:33:08.257 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045750+0000", 2024-09-14T06:33:08.257 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.2490000000000001e-06 2024-09-14T06:33:08.257 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:08.257 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:08.257 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:33:08.257 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045752+0000", 2024-09-14T06:33:08.257 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.3980000000000002e-06 2024-09-14T06:33:08.257 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:08.257 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:08.257 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:33:08.257 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.078184+0000", 2024-09-14T06:33:08.257 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032431593000000002 2024-09-14T06:33:08.258 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:08.258 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:08.258 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:33:08.258 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.078193+0000", 2024-09-14T06:33:08.258 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.1230000000000006e-06 2024-09-14T06:33:08.258 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:33:08.258 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:33:08.258 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:33:08.258 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:08.258 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:08.258 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:98519 3.10 3.1bf56f90 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T06:33:08.258 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:30:07.045755+0000", 2024-09-14T06:33:08.258 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 181.15421375899999, 2024-09-14T06:33:08.258 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032489431999999999, 2024-09-14T06:33:08.259 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:33:08.259 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:33:08.259 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "reached pg", 2024-09-14T06:33:08.259 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:33:08.259 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:33:08.259 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:33:08.259 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 98519 2024-09-14T06:33:08.259 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:08.259 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:33:08.259 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:08.259 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:33:08.259 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045755+0000", 2024-09-14T06:33:08.259 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:33:08.260 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:08.260 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:08.260 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:33:08.260 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045755+0000", 2024-09-14T06:33:08.260 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:33:08.260 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:08.260 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:08.260 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:33:08.260 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045756+0000", 2024-09-14T06:33:08.260 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.23e-07 2024-09-14T06:33:08.260 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:08.260 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:08.260 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:33:08.260 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045757+0000", 2024-09-14T06:33:08.261 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.1060000000000001e-06 2024-09-14T06:33:08.261 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:08.261 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:08.261 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:33:08.261 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045760+0000", 2024-09-14T06:33:08.261 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.988e-06 2024-09-14T06:33:08.261 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:08.261 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:08.261 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:33:08.261 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045765+0000", 2024-09-14T06:33:08.261 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.2689999999999999e-06 2024-09-14T06:33:08.261 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:08.261 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:08.261 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:33:08.262 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.078234+0000", 2024-09-14T06:33:08.262 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032468915000000001 2024-09-14T06:33:08.262 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:08.262 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:08.262 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:33:08.262 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.078245+0000", 2024-09-14T06:33:08.262 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.0930999999999999e-05 2024-09-14T06:33:08.262 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:33:08.262 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:33:08.262 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:33:08.262 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:08.262 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:08.262 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:98520 3.10 3.1bf56f90 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T06:33:08.262 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:30:07.045768+0000", 2024-09-14T06:33:08.263 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 181.15420088100001, 2024-09-14T06:33:08.263 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032531379999999999, 2024-09-14T06:33:08.263 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:33:08.263 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:33:08.263 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "reached pg", 2024-09-14T06:33:08.263 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:33:08.263 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:33:08.263 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:33:08.263 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 98520 2024-09-14T06:33:08.263 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:08.263 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:33:08.263 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:08.263 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:33:08.264 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045768+0000", 2024-09-14T06:33:08.264 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:33:08.264 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:08.264 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:08.264 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:33:08.264 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045768+0000", 2024-09-14T06:33:08.264 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:33:08.264 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:08.264 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:08.264 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:33:08.264 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045768+0000", 2024-09-14T06:33:08.264 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.22e-07 2024-09-14T06:33:08.264 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:08.265 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:08.265 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:33:08.265 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045773+0000", 2024-09-14T06:33:08.265 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.3470000000000001e-06 2024-09-14T06:33:08.265 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:08.265 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:08.265 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:33:08.265 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045776+0000", 2024-09-14T06:33:08.265 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.1319999999999998e-06 2024-09-14T06:33:08.265 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:08.265 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:08.265 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:33:08.265 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045779+0000", 2024-09-14T06:33:08.265 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.9749999999999999e-06 2024-09-14T06:33:08.266 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:08.266 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:08.266 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:33:08.266 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.078289+0000", 2024-09-14T06:33:08.266 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032509693999999999 2024-09-14T06:33:08.266 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:08.266 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:08.266 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:33:08.266 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.078300+0000", 2024-09-14T06:33:08.266 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.101e-05 2024-09-14T06:33:08.266 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:33:08.266 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:33:08.266 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:33:08.266 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:08.267 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:08.267 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:98521 3.10 3.1bf56f90 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T06:33:08.267 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:30:07.045782+0000", 2024-09-14T06:33:08.267 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 181.15418670099999, 2024-09-14T06:33:08.267 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032566109000000003, 2024-09-14T06:33:08.267 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:33:08.267 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:33:08.267 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "reached pg", 2024-09-14T06:33:08.267 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:33:08.267 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:33:08.267 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:33:08.267 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 98521 2024-09-14T06:33:08.267 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:08.267 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:33:08.268 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:08.268 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:33:08.268 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045782+0000", 2024-09-14T06:33:08.268 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:33:08.268 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:08.268 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:08.268 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:33:08.268 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045782+0000", 2024-09-14T06:33:08.268 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:33:08.268 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:08.268 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:08.268 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:33:08.268 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045783+0000", 2024-09-14T06:33:08.268 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.2499999999999999e-07 2024-09-14T06:33:08.269 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:08.269 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:08.269 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:33:08.269 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045784+0000", 2024-09-14T06:33:08.269 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.0109999999999999e-06 2024-09-14T06:33:08.269 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:08.269 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:08.269 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:33:08.269 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045786+0000", 2024-09-14T06:33:08.269 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.8289999999999999e-06 2024-09-14T06:33:08.269 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:08.269 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:08.269 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:33:08.269 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045793+0000", 2024-09-14T06:33:08.269 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.579e-06 2024-09-14T06:33:08.270 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:08.270 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:08.270 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:33:08.270 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.078338+0000", 2024-09-14T06:33:08.270 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032545389000000001 2024-09-14T06:33:08.270 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:08.270 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:08.270 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:33:08.270 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.078348+0000", 2024-09-14T06:33:08.270 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.0076e-05 2024-09-14T06:33:08.270 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:33:08.270 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:33:08.270 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:33:08.270 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:08.271 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:08.271 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:98522 3.10 3.1bf56f90 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T06:33:08.271 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:30:07.045796+0000", 2024-09-14T06:33:08.271 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 181.15417279299999, 2024-09-14T06:33:08.271 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032577846000000001, 2024-09-14T06:33:08.271 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:33:08.271 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:33:08.271 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "reached pg", 2024-09-14T06:33:08.271 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:33:08.271 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:33:08.271 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:33:08.271 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 98522 2024-09-14T06:33:08.271 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:08.271 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:33:08.271 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:08.272 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:33:08.272 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045796+0000", 2024-09-14T06:33:08.272 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:33:08.272 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:08.272 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:08.272 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:33:08.272 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045796+0000", 2024-09-14T06:33:08.272 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:33:08.272 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:08.272 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:08.272 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:33:08.272 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045796+0000", 2024-09-14T06:33:08.272 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.2100000000000001e-07 2024-09-14T06:33:08.273 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:08.273 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:08.273 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:33:08.273 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045797+0000", 2024-09-14T06:33:08.273 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.6899999999999996e-07 2024-09-14T06:33:08.273 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:08.273 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:08.273 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:33:08.273 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045800+0000", 2024-09-14T06:33:08.273 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.9979999999999999e-06 2024-09-14T06:33:08.273 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:08.273 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:08.273 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:33:08.273 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045803+0000", 2024-09-14T06:33:08.274 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.2850000000000001e-06 2024-09-14T06:33:08.274 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:08.274 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:08.274 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:33:08.274 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.078366+0000", 2024-09-14T06:33:08.274 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032562932000000003 2024-09-14T06:33:08.274 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:08.274 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:08.274 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:33:08.274 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.078374+0000", 2024-09-14T06:33:08.274 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.4409999999999993e-06 2024-09-14T06:33:08.274 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:33:08.274 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:33:08.274 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:33:08.274 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:08.275 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:08.275 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:98523 3.10 3.1bf56f90 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T06:33:08.275 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:30:07.045806+0000", 2024-09-14T06:33:08.275 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 181.15416325000001, 2024-09-14T06:33:08.275 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032611167000000003, 2024-09-14T06:33:08.275 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:33:08.275 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:33:08.275 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "reached pg", 2024-09-14T06:33:08.275 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:33:08.275 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:33:08.275 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:33:08.275 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 98523 2024-09-14T06:33:08.275 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:08.275 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:33:08.276 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:08.276 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:33:08.276 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045806+0000", 2024-09-14T06:33:08.276 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:33:08.276 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:08.276 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:08.276 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:33:08.276 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045806+0000", 2024-09-14T06:33:08.276 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:33:08.276 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:08.276 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:08.276 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:33:08.276 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045806+0000", 2024-09-14T06:33:08.276 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.2000000000000001e-07 2024-09-14T06:33:08.277 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:08.277 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:08.277 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:33:08.277 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045807+0000", 2024-09-14T06:33:08.277 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.4e-07 2024-09-14T06:33:08.277 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:08.277 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:08.277 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:33:08.277 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045810+0000", 2024-09-14T06:33:08.277 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.9229999999999998e-06 2024-09-14T06:33:08.277 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:08.277 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:08.277 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:33:08.277 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045812+0000", 2024-09-14T06:33:08.277 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.2220000000000001e-06 2024-09-14T06:33:08.278 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:08.278 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:08.278 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:33:08.278 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.078407+0000", 2024-09-14T06:33:08.278 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032594513999999998 2024-09-14T06:33:08.278 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:08.278 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:08.278 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:33:08.278 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.078417+0000", 2024-09-14T06:33:08.278 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.0348e-05 2024-09-14T06:33:08.278 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:33:08.278 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:33:08.278 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:33:08.278 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:08.279 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:08.279 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:98524 3.10 3.1bf56f90 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-14T06:33:08.279 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:30:07.045815+0000", 2024-09-14T06:33:08.279 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 181.154153882, 2024-09-14T06:33:08.279 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032643208999999999, 2024-09-14T06:33:08.279 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:33:08.279 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:33:08.279 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "reached pg", 2024-09-14T06:33:08.279 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:33:08.279 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:33:08.279 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:33:08.279 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 98524 2024-09-14T06:33:08.279 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:08.279 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:33:08.280 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:08.280 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:33:08.280 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045815+0000", 2024-09-14T06:33:08.280 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:33:08.280 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:08.280 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:08.280 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:33:08.280 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045815+0000", 2024-09-14T06:33:08.280 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:33:08.280 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:08.280 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:08.281 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:33:08.281 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045815+0000", 2024-09-14T06:33:08.281 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.11e-07 2024-09-14T06:33:08.281 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:08.281 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:08.281 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:33:08.281 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045816+0000", 2024-09-14T06:33:08.281 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.9099999999999991e-07 2024-09-14T06:33:08.281 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:08.281 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:08.281 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:33:08.281 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045819+0000", 2024-09-14T06:33:08.281 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.9399999999999998e-06 2024-09-14T06:33:08.281 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:08.282 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:08.282 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:33:08.282 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.045822+0000", 2024-09-14T06:33:08.282 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.3470000000000001e-06 2024-09-14T06:33:08.282 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:08.282 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:08.282 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:33:08.282 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.078449+0000", 2024-09-14T06:33:08.282 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.032627529000000002 2024-09-14T06:33:08.282 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:08.282 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:08.282 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:33:08.282 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:30:07.078458+0000", 2024-09-14T06:33:08.282 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.1910000000000006e-06 2024-09-14T06:33:08.283 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:33:08.283 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:33:08.283 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:33:08.283 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:33:08.283 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:33:08.283 INFO:teuthology.orchestra.run.smithi138.stdout:} 2024-09-14T06:33:08.284 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-14T06:33:08.388 INFO:teuthology.orchestra.run.smithi138.stdout:{ 2024-09-14T06:33:08.388 INFO:teuthology.orchestra.run.smithi138.stdout: "ops": [], 2024-09-14T06:33:08.389 INFO:teuthology.orchestra.run.smithi138.stdout: "num_ops": 0 2024-09-14T06:33:08.389 INFO:teuthology.orchestra.run.smithi138.stdout:} 2024-09-14T06:33:08.398 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-14T06:33:08.500 INFO:teuthology.orchestra.run.smithi138.stdout:{ 2024-09-14T06:33:08.500 INFO:teuthology.orchestra.run.smithi138.stdout: "ops": [], 2024-09-14T06:33:08.501 INFO:teuthology.orchestra.run.smithi138.stdout: "complaint_time": 30, 2024-09-14T06:33:08.501 INFO:teuthology.orchestra.run.smithi138.stdout: "num_blocked_ops": 0 2024-09-14T06:33:08.501 INFO:teuthology.orchestra.run.smithi138.stdout:} 2024-09-14T06:33:08.508 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-14T06:33:08.613 INFO:teuthology.orchestra.run.smithi138.stdout:{ 2024-09-14T06:33:08.614 INFO:teuthology.orchestra.run.smithi138.stdout: "size": 20, 2024-09-14T06:33:08.614 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 600, 2024-09-14T06:33:08.614 INFO:teuthology.orchestra.run.smithi138.stdout: "ops": [ 2024-09-14T06:33:08.614 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:08.614 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:53244 3.f 3:f71f51a7:::rbd_data.112dca2f6ae1.0000000000000015:17 [read 1863680~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:33:08.614 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:44.369399+0000", 2024-09-14T06:33:08.614 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 204.242592237, 2024-09-14T06:33:08.614 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047291189999999997, 2024-09-14T06:33:08.614 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:33:08.614 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:33:08.614 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:33:08.614 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:33:08.615 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:33:08.615 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:33:08.615 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 53244 2024-09-14T06:33:08.615 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:08.615 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:33:08.615 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:08.615 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:33:08.615 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369399+0000", 2024-09-14T06:33:08.615 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:33:08.615 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:08.615 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:08.615 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:33:08.615 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369399+0000", 2024-09-14T06:33:08.615 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:33:08.616 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:08.616 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:08.616 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:33:08.616 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369400+0000", 2024-09-14T06:33:08.616 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.0699999999999999e-07 2024-09-14T06:33:08.616 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:08.616 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:08.616 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:33:08.616 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369401+0000", 2024-09-14T06:33:08.616 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.4300000000000001e-06 2024-09-14T06:33:08.616 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:08.616 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:08.616 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:33:08.616 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369404+0000", 2024-09-14T06:33:08.617 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.061e-06 2024-09-14T06:33:08.617 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:08.617 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:08.617 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:33:08.617 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369407+0000", 2024-09-14T06:33:08.617 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.6139999999999999e-06 2024-09-14T06:33:08.617 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:08.617 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:08.617 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:33:08.617 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.416517+0000", 2024-09-14T06:33:08.617 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047110338000000002 2024-09-14T06:33:08.617 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:08.617 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:08.617 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:33:08.618 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.416616+0000", 2024-09-14T06:33:08.618 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.8651000000000001e-05 2024-09-14T06:33:08.618 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:08.618 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:08.618 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:33:08.618 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.416691+0000", 2024-09-14T06:33:08.618 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.4888999999999994e-05 2024-09-14T06:33:08.618 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:33:08.618 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:33:08.618 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:33:08.618 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:08.618 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:08.618 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:53245 3.f 3:f71f51a7:::rbd_data.112dca2f6ae1.0000000000000015:17 [read 1888256~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:33:08.618 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:44.369410+0000", 2024-09-14T06:33:08.619 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 204.242581949, 2024-09-14T06:33:08.619 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047399275999999997, 2024-09-14T06:33:08.619 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:33:08.619 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:33:08.619 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:33:08.619 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:33:08.619 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:33:08.619 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:33:08.619 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 53245 2024-09-14T06:33:08.619 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:08.619 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:33:08.619 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:08.619 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:33:08.619 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369410+0000", 2024-09-14T06:33:08.620 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:33:08.620 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:08.620 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:08.620 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:33:08.620 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369410+0000", 2024-09-14T06:33:08.620 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:33:08.620 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:08.620 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:08.620 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:33:08.620 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369410+0000", 2024-09-14T06:33:08.620 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.1899999999999999e-07 2024-09-14T06:33:08.620 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:08.620 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:08.620 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:33:08.621 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369411+0000", 2024-09-14T06:33:08.621 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.0300000000000001e-06 2024-09-14T06:33:08.621 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:08.621 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:08.621 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:33:08.621 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369414+0000", 2024-09-14T06:33:08.621 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.1099999999999999e-06 2024-09-14T06:33:08.621 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:08.621 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:08.621 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:33:08.621 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369417+0000", 2024-09-14T06:33:08.621 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.6189999999999998e-06 2024-09-14T06:33:08.621 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:08.621 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:08.622 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:33:08.622 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.416718+0000", 2024-09-14T06:33:08.622 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047301322999999999 2024-09-14T06:33:08.622 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:08.622 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:08.622 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:33:08.622 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.416759+0000", 2024-09-14T06:33:08.622 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.1009e-05 2024-09-14T06:33:08.622 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:08.622 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:08.622 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:33:08.622 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.416809+0000", 2024-09-14T06:33:08.622 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.9966000000000002e-05 2024-09-14T06:33:08.622 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:33:08.622 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:33:08.623 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:33:08.623 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:08.623 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:08.623 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:53246 3.f 3:f71f51a7:::rbd_data.112dca2f6ae1.0000000000000015:17 [read 1912832~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:33:08.623 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:44.369420+0000", 2024-09-14T06:33:08.623 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 204.242572005, 2024-09-14T06:33:08.623 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047529463000000001, 2024-09-14T06:33:08.623 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:33:08.623 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:33:08.623 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:33:08.623 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:33:08.623 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:33:08.623 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:33:08.623 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 53246 2024-09-14T06:33:08.624 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:08.624 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:33:08.624 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:08.624 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:33:08.624 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369420+0000", 2024-09-14T06:33:08.624 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:33:08.624 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:08.624 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:08.624 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:33:08.624 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369420+0000", 2024-09-14T06:33:08.624 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:33:08.624 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:08.624 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:08.624 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:33:08.625 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369420+0000", 2024-09-14T06:33:08.625 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.1299999999999999e-07 2024-09-14T06:33:08.625 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:08.625 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:08.625 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:33:08.625 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369425+0000", 2024-09-14T06:33:08.625 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.0130000000000001e-06 2024-09-14T06:33:08.625 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:08.625 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:08.625 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:33:08.625 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369428+0000", 2024-09-14T06:33:08.625 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.349e-06 2024-09-14T06:33:08.625 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:08.625 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:08.625 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:33:08.626 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369431+0000", 2024-09-14T06:33:08.626 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.7389999999999999e-06 2024-09-14T06:33:08.626 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:08.626 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:08.626 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:33:08.626 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.416834+0000", 2024-09-14T06:33:08.626 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047402669000000001 2024-09-14T06:33:08.626 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:08.626 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:08.626 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:33:08.626 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.416882+0000", 2024-09-14T06:33:08.626 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.8068999999999997e-05 2024-09-14T06:33:08.626 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:08.626 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:08.627 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:33:08.627 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.416949+0000", 2024-09-14T06:33:08.627 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.7410999999999995e-05 2024-09-14T06:33:08.627 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:33:08.627 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:33:08.627 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:33:08.627 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:08.627 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:08.627 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:53247 3.f 3:f71f51a7:::rbd_data.112dca2f6ae1.0000000000000015:17 [read 1937408~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:33:08.627 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:44.369434+0000", 2024-09-14T06:33:08.627 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 204.24255724099999, 2024-09-14T06:33:08.627 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047652264999999999, 2024-09-14T06:33:08.627 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:33:08.627 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:33:08.628 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:33:08.628 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:33:08.628 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:33:08.628 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:33:08.628 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 53247 2024-09-14T06:33:08.628 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:08.628 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:33:08.628 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:08.628 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:33:08.628 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369434+0000", 2024-09-14T06:33:08.628 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:33:08.628 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:08.628 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:08.628 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:33:08.629 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369434+0000", 2024-09-14T06:33:08.629 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:33:08.629 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:08.629 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:08.629 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:33:08.629 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369435+0000", 2024-09-14T06:33:08.629 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.2100000000000001e-07 2024-09-14T06:33:08.629 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:08.629 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:08.629 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:33:08.629 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369436+0000", 2024-09-14T06:33:08.629 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.122e-06 2024-09-14T06:33:08.629 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:08.629 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:08.629 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:33:08.630 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369439+0000", 2024-09-14T06:33:08.630 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.0400000000000001e-06 2024-09-14T06:33:08.630 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:08.630 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:08.630 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:33:08.630 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369441+0000", 2024-09-14T06:33:08.630 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.728e-06 2024-09-14T06:33:08.630 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:08.630 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:08.630 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:33:08.630 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.416977+0000", 2024-09-14T06:33:08.630 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047535344 2024-09-14T06:33:08.630 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:08.630 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:08.631 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:33:08.631 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.417029+0000", 2024-09-14T06:33:08.631 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.2037999999999999e-05 2024-09-14T06:33:08.631 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:08.631 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:08.631 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:33:08.631 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.417087+0000", 2024-09-14T06:33:08.631 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.7772000000000002e-05 2024-09-14T06:33:08.631 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:33:08.631 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:33:08.631 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:33:08.631 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:08.631 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:08.631 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:53248 3.f 3:f71f51a7:::rbd_data.112dca2f6ae1.0000000000000015:17 [read 1961984~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:33:08.632 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:44.369445+0000", 2024-09-14T06:33:08.632 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 204.24254668899999, 2024-09-14T06:33:08.632 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047798489, 2024-09-14T06:33:08.632 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:33:08.632 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:33:08.632 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:33:08.632 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:33:08.632 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:33:08.632 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:33:08.632 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 53248 2024-09-14T06:33:08.632 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:08.632 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:33:08.632 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:08.633 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:33:08.633 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369445+0000", 2024-09-14T06:33:08.633 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:33:08.633 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:08.633 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:08.633 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:33:08.633 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369445+0000", 2024-09-14T06:33:08.633 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:33:08.633 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:08.633 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:08.633 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:33:08.633 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369445+0000", 2024-09-14T06:33:08.633 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.0699999999999999e-07 2024-09-14T06:33:08.633 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:08.634 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:08.634 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:33:08.634 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369446+0000", 2024-09-14T06:33:08.634 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.1000000000000001e-06 2024-09-14T06:33:08.634 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:08.634 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:08.634 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:33:08.634 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369449+0000", 2024-09-14T06:33:08.634 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.1039999999999998e-06 2024-09-14T06:33:08.634 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:08.634 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:08.634 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:33:08.634 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369452+0000", 2024-09-14T06:33:08.634 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.6199999999999999e-06 2024-09-14T06:33:08.635 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:08.635 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:08.635 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:33:08.635 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.417114+0000", 2024-09-14T06:33:08.635 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047661901999999999 2024-09-14T06:33:08.635 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:08.635 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:08.635 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:33:08.635 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.417181+0000", 2024-09-14T06:33:08.635 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.7385999999999995e-05 2024-09-14T06:33:08.635 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:08.635 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:08.635 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:33:08.635 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.417243+0000", 2024-09-14T06:33:08.635 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.2169999999999996e-05 2024-09-14T06:33:08.636 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:33:08.636 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:33:08.636 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:33:08.636 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:08.636 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:08.636 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:53249 3.f 3:f71f51a7:::rbd_data.112dca2f6ae1.0000000000000015:17 [read 1986560~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:33:08.636 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:44.369455+0000", 2024-09-14T06:33:08.636 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 204.24253637800001, 2024-09-14T06:33:08.636 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047890493999999999, 2024-09-14T06:33:08.636 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:33:08.636 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:33:08.636 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:33:08.636 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:33:08.637 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:33:08.637 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:33:08.637 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 53249 2024-09-14T06:33:08.637 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:08.637 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:33:08.637 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:08.637 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:33:08.637 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369455+0000", 2024-09-14T06:33:08.637 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:33:08.637 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:08.637 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:08.637 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:33:08.637 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369455+0000", 2024-09-14T06:33:08.637 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:33:08.637 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:08.638 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:08.638 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:33:08.638 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369455+0000", 2024-09-14T06:33:08.638 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.0800000000000001e-07 2024-09-14T06:33:08.638 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:08.638 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:08.638 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:33:08.638 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369456+0000", 2024-09-14T06:33:08.638 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.0729999999999999e-06 2024-09-14T06:33:08.638 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:08.638 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:08.638 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:33:08.638 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369460+0000", 2024-09-14T06:33:08.638 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.027e-06 2024-09-14T06:33:08.638 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:08.639 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:08.639 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:33:08.639 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369462+0000", 2024-09-14T06:33:08.639 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.5320000000000001e-06 2024-09-14T06:33:08.639 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:08.639 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:08.639 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:33:08.639 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.417257+0000", 2024-09-14T06:33:08.639 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.04779489 2024-09-14T06:33:08.639 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:08.639 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:08.639 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:33:08.639 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.417296+0000", 2024-09-14T06:33:08.639 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.8658000000000002e-05 2024-09-14T06:33:08.640 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:08.640 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:08.640 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:33:08.640 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.417346+0000", 2024-09-14T06:33:08.640 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.0105999999999998e-05 2024-09-14T06:33:08.640 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:33:08.640 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:33:08.640 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:33:08.640 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:08.640 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:08.640 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:53250 3.f 3:f71f51a7:::rbd_data.112dca2f6ae1.0000000000000015:17 [read 2011136~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:33:08.640 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:44.369465+0000", 2024-09-14T06:33:08.640 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 204.242526408, 2024-09-14T06:33:08.640 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.048032230000000002, 2024-09-14T06:33:08.641 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:33:08.641 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:33:08.641 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:33:08.641 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:33:08.641 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:33:08.641 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:33:08.641 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 53250 2024-09-14T06:33:08.641 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:08.641 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:33:08.641 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:08.641 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:33:08.641 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369465+0000", 2024-09-14T06:33:08.641 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:33:08.641 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:08.642 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:08.642 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:33:08.642 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369465+0000", 2024-09-14T06:33:08.642 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:33:08.642 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:08.642 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:08.642 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:33:08.642 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369465+0000", 2024-09-14T06:33:08.642 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.2100000000000001e-07 2024-09-14T06:33:08.642 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:08.642 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:08.642 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:33:08.642 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369466+0000", 2024-09-14T06:33:08.642 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.0640000000000001e-06 2024-09-14T06:33:08.643 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:08.643 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:08.643 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:33:08.643 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369469+0000", 2024-09-14T06:33:08.643 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.0170000000000001e-06 2024-09-14T06:33:08.643 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:08.643 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:08.643 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:33:08.643 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369472+0000", 2024-09-14T06:33:08.643 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.6189999999999998e-06 2024-09-14T06:33:08.643 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:08.643 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:08.643 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:33:08.643 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.417383+0000", 2024-09-14T06:33:08.644 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047911092000000002 2024-09-14T06:33:08.644 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:08.644 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:08.644 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:33:08.644 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.417430+0000", 2024-09-14T06:33:08.644 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.6898999999999999e-05 2024-09-14T06:33:08.644 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:08.644 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:08.644 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:33:08.644 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.417497+0000", 2024-09-14T06:33:08.644 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.7317999999999995e-05 2024-09-14T06:33:08.644 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:33:08.644 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:33:08.644 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:33:08.645 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:08.645 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:08.645 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:53251 3.f 3:f71f51a7:::rbd_data.112dca2f6ae1.0000000000000015:17 [read 2035712~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:33:08.645 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:44.369475+0000", 2024-09-14T06:33:08.645 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 204.24251641199999, 2024-09-14T06:33:08.645 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.048154542000000002, 2024-09-14T06:33:08.645 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:33:08.645 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:33:08.645 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:33:08.645 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:33:08.645 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:33:08.645 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:33:08.645 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 53251 2024-09-14T06:33:08.645 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:08.646 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:33:08.646 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:08.646 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:33:08.646 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369475+0000", 2024-09-14T06:33:08.646 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:33:08.646 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:08.646 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:08.646 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:33:08.646 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369475+0000", 2024-09-14T06:33:08.646 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:33:08.646 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:08.646 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:08.646 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:33:08.646 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369475+0000", 2024-09-14T06:33:08.647 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.0100000000000001e-07 2024-09-14T06:33:08.647 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:08.647 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:08.647 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:33:08.647 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369476+0000", 2024-09-14T06:33:08.647 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.0580000000000001e-06 2024-09-14T06:33:08.647 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:08.647 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:08.647 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:33:08.647 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369479+0000", 2024-09-14T06:33:08.647 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.9950000000000001e-06 2024-09-14T06:33:08.647 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:08.647 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:08.647 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:33:08.648 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369482+0000", 2024-09-14T06:33:08.648 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.6340000000000001e-06 2024-09-14T06:33:08.648 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:08.648 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:08.648 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:33:08.648 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.417530+0000", 2024-09-14T06:33:08.648 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.048047535000000002 2024-09-14T06:33:08.648 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:08.648 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:08.648 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:33:08.648 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.417574+0000", 2024-09-14T06:33:08.648 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.4255999999999998e-05 2024-09-14T06:33:08.648 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:08.648 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:08.648 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:33:08.649 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.417630+0000", 2024-09-14T06:33:08.649 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.5862999999999997e-05 2024-09-14T06:33:08.649 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:33:08.649 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:33:08.649 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:33:08.649 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:08.649 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:08.649 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:53252 3.f 3:f71f51a7:::rbd_data.112dca2f6ae1.0000000000000015:17 [read 2060288~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:33:08.649 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:44.369485+0000", 2024-09-14T06:33:08.649 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 204.242506564, 2024-09-14T06:33:08.649 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.048254116, 2024-09-14T06:33:08.649 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:33:08.649 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:33:08.649 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:33:08.650 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:33:08.650 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:33:08.650 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:33:08.650 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 53252 2024-09-14T06:33:08.650 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:08.650 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:33:08.650 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:08.650 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:33:08.650 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369485+0000", 2024-09-14T06:33:08.650 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:33:08.650 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:08.650 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:08.650 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:33:08.650 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369485+0000", 2024-09-14T06:33:08.651 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:33:08.651 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:08.651 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:08.651 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:33:08.651 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369485+0000", 2024-09-14T06:33:08.651 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.1e-07 2024-09-14T06:33:08.651 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:08.651 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:08.651 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:33:08.651 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369487+0000", 2024-09-14T06:33:08.651 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.42e-06 2024-09-14T06:33:08.651 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:08.651 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:08.651 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:33:08.652 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369490+0000", 2024-09-14T06:33:08.652 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.9639999999999999e-06 2024-09-14T06:33:08.652 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:08.652 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:08.652 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:33:08.652 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369492+0000", 2024-09-14T06:33:08.652 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.5679999999999998e-06 2024-09-14T06:33:08.652 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:08.652 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:08.652 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:33:08.652 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.417645+0000", 2024-09-14T06:33:08.652 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.048152753 2024-09-14T06:33:08.652 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:08.653 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:08.653 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:33:08.653 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.417687+0000", 2024-09-14T06:33:08.653 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.1900999999999997e-05 2024-09-14T06:33:08.653 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:08.653 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:08.653 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:33:08.653 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.417739+0000", 2024-09-14T06:33:08.653 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.2299999999999997e-05 2024-09-14T06:33:08.653 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:33:08.653 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:33:08.653 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:33:08.653 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:08.653 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:08.654 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:53253 3.f 3:f71f51a7:::rbd_data.112dca2f6ae1.0000000000000015:17 [read 2084864~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:33:08.654 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:44.369495+0000", 2024-09-14T06:33:08.654 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 204.24249644, 2024-09-14T06:33:08.654 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.048366785000000002, 2024-09-14T06:33:08.654 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:33:08.654 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:33:08.654 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:33:08.654 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:33:08.654 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:33:08.654 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:33:08.654 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 53253 2024-09-14T06:33:08.654 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:08.654 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:33:08.654 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:08.655 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:33:08.655 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369495+0000", 2024-09-14T06:33:08.655 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:33:08.655 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:08.655 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:08.655 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:33:08.655 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.369495+0000", 2024-09-14T06:33:08.655 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:33:08.655 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:08.655 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:08.655 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:33:08.655 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370388+0000", 2024-09-14T06:33:08.655 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00089287999999999998 2024-09-14T06:33:08.655 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:08.656 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:08.656 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:33:08.656 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370392+0000", 2024-09-14T06:33:08.656 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.2130000000000002e-06 2024-09-14T06:33:08.656 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:08.656 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:08.656 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:33:08.656 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370401+0000", 2024-09-14T06:33:08.656 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.7649999999999999e-06 2024-09-14T06:33:08.656 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:08.656 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:08.656 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:33:08.656 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370406+0000", 2024-09-14T06:33:08.656 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.0159999999999999e-06 2024-09-14T06:33:08.657 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:08.657 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:08.657 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:33:08.657 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.417764+0000", 2024-09-14T06:33:08.657 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047358348000000001 2024-09-14T06:33:08.657 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:08.657 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:08.657 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:33:08.657 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.417806+0000", 2024-09-14T06:33:08.657 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.1149999999999997e-05 2024-09-14T06:33:08.657 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:08.657 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:08.657 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:33:08.658 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.417862+0000", 2024-09-14T06:33:08.658 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.6412999999999997e-05 2024-09-14T06:33:08.658 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:33:08.658 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:33:08.658 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:33:08.658 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:08.658 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:08.658 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:53340 3.f 3:f71f51a7:::rbd_data.112dca2f6ae1.0000000000000015:17 [read 1056768~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:33:08.658 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:44.370421+0000", 2024-09-14T06:33:08.658 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 204.24157056000001, 2024-09-14T06:33:08.658 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047566313999999998, 2024-09-14T06:33:08.658 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:33:08.658 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:33:08.658 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:33:08.658 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:33:08.659 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:33:08.659 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:33:08.659 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 53340 2024-09-14T06:33:08.659 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:08.659 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:33:08.659 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:08.659 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:33:08.659 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370421+0000", 2024-09-14T06:33:08.659 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:33:08.659 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:08.659 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:08.659 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:33:08.659 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370421+0000", 2024-09-14T06:33:08.659 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:33:08.660 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:08.660 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:08.660 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:33:08.660 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370422+0000", 2024-09-14T06:33:08.660 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.3799999999999996e-07 2024-09-14T06:33:08.660 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:08.660 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:08.660 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:33:08.660 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370424+0000", 2024-09-14T06:33:08.660 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.6919999999999998e-06 2024-09-14T06:33:08.660 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:08.660 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:08.660 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:33:08.661 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370432+0000", 2024-09-14T06:33:08.661 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.2439999999999996e-06 2024-09-14T06:33:08.661 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:08.661 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:08.661 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:33:08.661 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370437+0000", 2024-09-14T06:33:08.661 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.5389999999999998e-06 2024-09-14T06:33:08.661 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:08.661 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:08.661 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:33:08.661 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.417888+0000", 2024-09-14T06:33:08.661 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047450432000000001 2024-09-14T06:33:08.661 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:08.661 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:08.661 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:33:08.662 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.417935+0000", 2024-09-14T06:33:08.662 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.7454999999999999e-05 2024-09-14T06:33:08.662 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:08.662 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:08.662 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:33:08.662 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.417987+0000", 2024-09-14T06:33:08.662 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.2213999999999998e-05 2024-09-14T06:33:08.662 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:33:08.662 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:33:08.662 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:33:08.662 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:08.662 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:08.662 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:53341 3.f 3:f71f51a7:::rbd_data.112dca2f6ae1.0000000000000015:17 [read 1081344~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:33:08.662 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:44.370446+0000", 2024-09-14T06:33:08.663 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 204.24154528700001, 2024-09-14T06:33:08.663 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047665723, 2024-09-14T06:33:08.663 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:33:08.663 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:33:08.663 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:33:08.663 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:33:08.663 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:33:08.663 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:33:08.663 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 53341 2024-09-14T06:33:08.663 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:08.663 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:33:08.663 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:08.663 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:33:08.663 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370446+0000", 2024-09-14T06:33:08.664 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:33:08.664 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:08.664 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:08.664 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:33:08.664 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370446+0000", 2024-09-14T06:33:08.664 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:33:08.664 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:08.664 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:08.664 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:33:08.664 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370447+0000", 2024-09-14T06:33:08.664 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.6100000000000002e-07 2024-09-14T06:33:08.664 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:08.665 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:08.665 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:33:08.665 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370448+0000", 2024-09-14T06:33:08.665 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.7370000000000001e-06 2024-09-14T06:33:08.665 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:08.665 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:08.665 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:33:08.665 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370462+0000", 2024-09-14T06:33:08.665 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.3227000000000001e-05 2024-09-14T06:33:08.665 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:08.665 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:08.665 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:33:08.665 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370467+0000", 2024-09-14T06:33:08.665 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.6540000000000004e-06 2024-09-14T06:33:08.666 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:08.666 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:08.666 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:33:08.666 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.418013+0000", 2024-09-14T06:33:08.666 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047545325999999999 2024-09-14T06:33:08.666 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:08.666 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:08.666 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:33:08.666 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.418054+0000", 2024-09-14T06:33:08.666 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.1696999999999997e-05 2024-09-14T06:33:08.666 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:08.666 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:08.666 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:33:08.666 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.418112+0000", 2024-09-14T06:33:08.666 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.7720999999999998e-05 2024-09-14T06:33:08.667 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:33:08.667 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:33:08.667 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:33:08.667 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:08.667 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:08.667 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:53342 3.f 3:f71f51a7:::rbd_data.112dca2f6ae1.0000000000000015:17 [read 1105920~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:33:08.667 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:44.370475+0000", 2024-09-14T06:33:08.667 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 204.24151622599999, 2024-09-14T06:33:08.667 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047748848000000003, 2024-09-14T06:33:08.667 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:33:08.667 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:33:08.667 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:33:08.667 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:33:08.667 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:33:08.668 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:33:08.668 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 53342 2024-09-14T06:33:08.668 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:08.668 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:33:08.668 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:08.668 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:33:08.668 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370475+0000", 2024-09-14T06:33:08.668 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:33:08.668 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:08.668 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:08.668 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:33:08.668 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370475+0000", 2024-09-14T06:33:08.668 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:33:08.668 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:08.669 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:08.669 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:33:08.669 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370476+0000", 2024-09-14T06:33:08.669 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.4700000000000002e-07 2024-09-14T06:33:08.669 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:08.669 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:08.669 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:33:08.669 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370478+0000", 2024-09-14T06:33:08.669 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.5890000000000001e-06 2024-09-14T06:33:08.669 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:08.669 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:08.669 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:33:08.669 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370486+0000", 2024-09-14T06:33:08.669 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.3470000000000001e-06 2024-09-14T06:33:08.670 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:08.670 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:08.670 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:33:08.670 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370491+0000", 2024-09-14T06:33:08.670 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.5720000000000002e-06 2024-09-14T06:33:08.670 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:08.670 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:08.670 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:33:08.670 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.418135+0000", 2024-09-14T06:33:08.670 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047644094999999997 2024-09-14T06:33:08.670 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:08.670 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:08.670 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:33:08.670 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.418174+0000", 2024-09-14T06:33:08.671 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.8204000000000002e-05 2024-09-14T06:33:08.671 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:08.671 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:08.671 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:33:08.671 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.418224+0000", 2024-09-14T06:33:08.671 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.0593999999999997e-05 2024-09-14T06:33:08.671 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:33:08.671 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:33:08.671 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:33:08.671 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:08.671 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:08.671 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:53343 3.f 3:f71f51a7:::rbd_data.112dca2f6ae1.0000000000000015:17 [read 1130496~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:33:08.671 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:44.370498+0000", 2024-09-14T06:33:08.671 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 204.24149335000001, 2024-09-14T06:33:08.672 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047841574999999997, 2024-09-14T06:33:08.672 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:33:08.672 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:33:08.672 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:33:08.672 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:33:08.672 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:33:08.672 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:33:08.672 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 53343 2024-09-14T06:33:08.672 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:08.672 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:33:08.672 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:08.672 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:33:08.672 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370498+0000", 2024-09-14T06:33:08.673 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:33:08.673 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:08.673 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:08.673 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:33:08.673 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370498+0000", 2024-09-14T06:33:08.673 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:33:08.673 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:08.673 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:08.673 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:33:08.673 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370499+0000", 2024-09-14T06:33:08.673 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.9999999999999998e-07 2024-09-14T06:33:08.673 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:08.673 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:08.673 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:33:08.674 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370501+0000", 2024-09-14T06:33:08.674 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.0310000000000001e-06 2024-09-14T06:33:08.674 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:08.674 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:08.674 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:33:08.674 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370507+0000", 2024-09-14T06:33:08.674 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.4169999999999997e-06 2024-09-14T06:33:08.674 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:08.674 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:08.674 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:33:08.674 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370516+0000", 2024-09-14T06:33:08.674 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.7190000000000007e-06 2024-09-14T06:33:08.674 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:08.674 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:08.674 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:33:08.675 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.418247+0000", 2024-09-14T06:33:08.675 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047731190999999999 2024-09-14T06:33:08.675 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:08.675 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:08.675 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:33:08.675 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.418286+0000", 2024-09-14T06:33:08.675 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.8988e-05 2024-09-14T06:33:08.675 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:08.675 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:08.675 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:33:08.675 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.418340+0000", 2024-09-14T06:33:08.675 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.3829000000000002e-05 2024-09-14T06:33:08.675 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:33:08.675 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:33:08.676 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:33:08.676 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:08.676 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:08.676 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:53344 3.f 3:f71f51a7:::rbd_data.112dca2f6ae1.0000000000000015:17 [read 1155072~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:33:08.676 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:44.370524+0000", 2024-09-14T06:33:08.676 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 204.24146758000001, 2024-09-14T06:33:08.676 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047972262000000002, 2024-09-14T06:33:08.676 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:33:08.676 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:33:08.676 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:33:08.676 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:33:08.676 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:33:08.676 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:33:08.676 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 53344 2024-09-14T06:33:08.677 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:08.677 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:33:08.677 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:08.677 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:33:08.677 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370524+0000", 2024-09-14T06:33:08.677 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:33:08.677 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:08.677 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:08.677 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:33:08.677 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370524+0000", 2024-09-14T06:33:08.677 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:33:08.677 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:08.677 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:08.678 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:33:08.678 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370618+0000", 2024-09-14T06:33:08.678 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.4425999999999993e-05 2024-09-14T06:33:08.678 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:08.678 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:08.678 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:33:08.678 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370622+0000", 2024-09-14T06:33:08.678 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.3519999999999998e-06 2024-09-14T06:33:08.678 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:08.678 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:08.678 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:33:08.678 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370631+0000", 2024-09-14T06:33:08.678 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.9299999999999992e-06 2024-09-14T06:33:08.678 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:08.679 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:08.679 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:33:08.679 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370637+0000", 2024-09-14T06:33:08.679 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.3269999999999997e-06 2024-09-14T06:33:08.679 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:08.679 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:08.679 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:33:08.679 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.418372+0000", 2024-09-14T06:33:08.679 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047734764999999998 2024-09-14T06:33:08.679 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:08.679 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:08.679 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:33:08.679 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.418431+0000", 2024-09-14T06:33:08.679 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.8696999999999997e-05 2024-09-14T06:33:08.680 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:08.680 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:08.680 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:33:08.680 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.418496+0000", 2024-09-14T06:33:08.680 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.5765000000000005e-05 2024-09-14T06:33:08.680 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:33:08.680 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:33:08.680 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:33:08.680 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:08.680 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:08.680 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:53345 3.f 3:f71f51a7:::rbd_data.112dca2f6ae1.0000000000000015:17 [read 1179648~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:33:08.680 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:44.370647+0000", 2024-09-14T06:33:08.680 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 204.241344113, 2024-09-14T06:33:08.680 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047953775999999997, 2024-09-14T06:33:08.681 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:33:08.681 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:33:08.681 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:33:08.681 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:33:08.681 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:33:08.681 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:33:08.681 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 53345 2024-09-14T06:33:08.681 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:08.681 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:33:08.681 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:08.681 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:33:08.681 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370647+0000", 2024-09-14T06:33:08.681 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:33:08.681 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:08.681 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:08.682 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:33:08.682 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.370647+0000", 2024-09-14T06:33:08.682 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:33:08.682 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:08.682 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:08.682 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:33:08.682 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.372346+0000", 2024-09-14T06:33:08.682 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.001698284 2024-09-14T06:33:08.682 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:08.682 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:08.682 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:33:08.682 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.372351+0000", 2024-09-14T06:33:08.682 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.8890000000000001e-06 2024-09-14T06:33:08.682 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:08.683 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:08.683 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:33:08.683 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.372362+0000", 2024-09-14T06:33:08.683 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.0941000000000001e-05 2024-09-14T06:33:08.683 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:08.683 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:08.683 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:33:08.683 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.372369+0000", 2024-09-14T06:33:08.683 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.362e-06 2024-09-14T06:33:08.683 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:08.683 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:08.683 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:33:08.683 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.418510+0000", 2024-09-14T06:33:08.683 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.046140914999999998 2024-09-14T06:33:08.684 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:08.684 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:08.684 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:33:08.684 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.418549+0000", 2024-09-14T06:33:08.684 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.8717999999999998e-05 2024-09-14T06:33:08.684 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:08.684 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:08.684 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:33:08.684 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.418601+0000", 2024-09-14T06:33:08.684 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.2667000000000002e-05 2024-09-14T06:33:08.684 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:33:08.684 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:33:08.684 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:33:08.684 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:08.685 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:08.685 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:53353 3.f 3:f71f51a7:::rbd_data.112dca2f6ae1.0000000000000015:17 [read 1376256~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:33:08.685 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:44.372545+0000", 2024-09-14T06:33:08.685 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 204.23944703800001, 2024-09-14T06:33:08.685 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047364232999999999, 2024-09-14T06:33:08.685 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:33:08.685 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:33:08.685 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:33:08.685 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:33:08.685 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:33:08.685 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:33:08.685 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 53353 2024-09-14T06:33:08.685 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:08.686 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:33:08.686 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:08.686 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:33:08.686 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.372545+0000", 2024-09-14T06:33:08.686 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:33:08.686 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:08.686 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:08.686 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:33:08.686 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.372545+0000", 2024-09-14T06:33:08.686 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:33:08.686 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:08.686 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:08.686 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:33:08.686 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.372545+0000", 2024-09-14T06:33:08.686 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.27e-07 2024-09-14T06:33:08.687 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:08.687 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:08.687 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:33:08.687 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.372547+0000", 2024-09-14T06:33:08.687 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.0600000000000002e-06 2024-09-14T06:33:08.687 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:08.687 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:08.687 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:33:08.687 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.372553+0000", 2024-09-14T06:33:08.687 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.7980000000000002e-06 2024-09-14T06:33:08.687 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:08.687 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:08.687 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:33:08.687 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.372557+0000", 2024-09-14T06:33:08.688 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.7389999999999999e-06 2024-09-14T06:33:08.688 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:08.688 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:08.688 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:33:08.688 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.419811+0000", 2024-09-14T06:33:08.688 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047253310999999999 2024-09-14T06:33:08.688 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:08.688 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:08.688 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:33:08.688 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.419854+0000", 2024-09-14T06:33:08.688 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.3448000000000001e-05 2024-09-14T06:33:08.688 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:08.688 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:08.688 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:33:08.689 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.419909+0000", 2024-09-14T06:33:08.689 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.4549999999999998e-05 2024-09-14T06:33:08.689 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:33:08.689 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:33:08.689 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:33:08.689 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:08.689 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:08.689 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:53354 3.f 3:f71f51a7:::rbd_data.112dca2f6ae1.0000000000000015:17 [read 1400832~4096 [fadvise_nocache]] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64)", 2024-09-14T06:33:08.689 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:29:44.372563+0000", 2024-09-14T06:33:08.689 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 204.23942860099999, 2024-09-14T06:33:08.689 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.047482233999999998, 2024-09-14T06:33:08.689 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:33:08.689 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:33:08.689 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:33:08.690 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:33:08.690 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:33:08.690 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:33:08.690 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 53354 2024-09-14T06:33:08.690 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:08.690 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:33:08.690 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:08.690 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:33:08.690 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.372563+0000", 2024-09-14T06:33:08.690 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:33:08.690 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:08.690 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:08.690 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:33:08.690 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.372563+0000", 2024-09-14T06:33:08.691 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:33:08.691 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:08.691 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:08.691 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:33:08.691 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.375009+0000", 2024-09-14T06:33:08.691 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.0024457680000000001 2024-09-14T06:33:08.691 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:08.691 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:08.691 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:33:08.691 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.375014+0000", 2024-09-14T06:33:08.691 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.0599999999999998e-06 2024-09-14T06:33:08.691 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:08.691 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:08.691 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:33:08.691 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.375026+0000", 2024-09-14T06:33:08.692 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.1783999999999999e-05 2024-09-14T06:33:08.692 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:08.692 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:08.692 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:33:08.692 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.375035+0000", 2024-09-14T06:33:08.692 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.7799999999999995e-06 2024-09-14T06:33:08.692 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:08.692 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:08.692 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:33:08.692 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.419942+0000", 2024-09-14T06:33:08.692 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.044906982999999998 2024-09-14T06:33:08.692 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:08.692 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:08.692 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:33:08.693 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.419989+0000", 2024-09-14T06:33:08.693 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.7104999999999997e-05 2024-09-14T06:33:08.693 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:08.693 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:08.693 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:33:08.693 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:29:44.420045+0000", 2024-09-14T06:33:08.693 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.5754000000000002e-05 2024-09-14T06:33:08.693 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:33:08.693 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:33:08.693 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:33:08.693 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:08.693 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:08.693 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:225577 3.15 3:ac159a70:::rbd_header.112daa9ec306:head [watch reconnect cookie 140616827151664 gen 4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e188)", 2024-09-14T06:33:08.693 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:31:52.733493+0000", 2024-09-14T06:33:08.694 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 75.878498136999994, 2024-09-14T06:33:08.694 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.0156593789999999, 2024-09-14T06:33:08.694 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:33:08.694 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:33:08.694 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "started", 2024-09-14T06:33:08.694 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:33:08.694 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:33:08.694 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:33:08.694 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 225577 2024-09-14T06:33:08.694 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:08.694 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:33:08.694 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:08.694 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:33:08.694 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:31:52.733493+0000", 2024-09-14T06:33:08.694 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:33:08.695 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:08.695 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:08.695 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:33:08.695 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:31:52.733493+0000", 2024-09-14T06:33:08.695 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:33:08.695 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:08.695 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:08.695 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:33:08.695 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:31:52.733495+0000", 2024-09-14T06:33:08.695 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 2.0059999999999999e-06 2024-09-14T06:33:08.695 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:08.695 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:08.695 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:33:08.695 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:31:52.733504+0000", 2024-09-14T06:33:08.696 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.5490000000000006e-06 2024-09-14T06:33:08.696 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:08.696 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:08.696 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:33:08.696 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:31:52.733522+0000", 2024-09-14T06:33:08.696 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.7660999999999999e-05 2024-09-14T06:33:08.696 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:08.696 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:08.696 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:33:08.696 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:31:52.733531+0000", 2024-09-14T06:33:08.696 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 9.7180000000000008e-06 2024-09-14T06:33:08.696 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:08.696 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:08.696 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:33:08.697 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:31:52.733598+0000", 2024-09-14T06:33:08.697 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 6.6490999999999997e-05 2024-09-14T06:33:08.697 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:08.697 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:08.697 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "op must wait for map", 2024-09-14T06:33:08.697 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:31:52.733605+0000", 2024-09-14T06:33:08.697 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 7.6109999999999996e-06 2024-09-14T06:33:08.697 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:08.697 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:08.697 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:33:08.697 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:31:52.737908+0000", 2024-09-14T06:33:08.697 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.0043023050000000002 2024-09-14T06:33:08.697 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:08.697 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:08.698 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "waiting for peered", 2024-09-14T06:33:08.698 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:31:52.737923+0000", 2024-09-14T06:33:08.698 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.488e-05 2024-09-14T06:33:08.698 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:08.698 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:08.698 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:33:08.698 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:31:53.746776+0000", 2024-09-14T06:33:08.698 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.0088535359999999 2024-09-14T06:33:08.698 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:08.698 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:08.698 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:33:08.698 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:31:53.746878+0000", 2024-09-14T06:33:08.698 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.000101788 2024-09-14T06:33:08.698 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:08.699 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:08.699 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:33:08.699 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:31:53.749153+0000", 2024-09-14T06:33:08.699 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.0022748339999999999 2024-09-14T06:33:08.699 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:33:08.699 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:33:08.699 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:33:08.699 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:08.699 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:08.699 INFO:teuthology.orchestra.run.smithi138.stdout: "description": "osd_op(client.4397.0:225581 3.15 3:ab407115:::rbd_object_map.112d838b9898:head [call lock.unlock in=10b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e188)", 2024-09-14T06:33:08.699 INFO:teuthology.orchestra.run.smithi138.stdout: "initiated_at": "2024-09-14T06:31:52.743472+0000", 2024-09-14T06:33:08.699 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 75.868519097999993, 2024-09-14T06:33:08.699 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.007270501, 2024-09-14T06:33:08.699 INFO:teuthology.orchestra.run.smithi138.stdout: "continuous": false, 2024-09-14T06:33:08.700 INFO:teuthology.orchestra.run.smithi138.stdout: "type_data": { 2024-09-14T06:33:08.700 INFO:teuthology.orchestra.run.smithi138.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-14T06:33:08.700 INFO:teuthology.orchestra.run.smithi138.stdout: "client_info": { 2024-09-14T06:33:08.700 INFO:teuthology.orchestra.run.smithi138.stdout: "client": "client.4397", 2024-09-14T06:33:08.700 INFO:teuthology.orchestra.run.smithi138.stdout: "client_addr": "172.21.15.94:0/2956055827", 2024-09-14T06:33:08.700 INFO:teuthology.orchestra.run.smithi138.stdout: "tid": 225581 2024-09-14T06:33:08.700 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:08.700 INFO:teuthology.orchestra.run.smithi138.stdout: "events": [ 2024-09-14T06:33:08.700 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:08.700 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "initiated", 2024-09-14T06:33:08.700 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:31:52.743472+0000", 2024-09-14T06:33:08.700 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:33:08.700 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:08.700 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:08.701 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "header_read", 2024-09-14T06:33:08.701 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:31:52.743472+0000", 2024-09-14T06:33:08.701 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0 2024-09-14T06:33:08.701 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:08.701 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:08.701 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "throttled", 2024-09-14T06:33:08.701 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:31:52.743473+0000", 2024-09-14T06:33:08.701 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 8.1999999999999998e-07 2024-09-14T06:33:08.701 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:08.701 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:08.701 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "all_read", 2024-09-14T06:33:08.701 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:31:52.743478+0000", 2024-09-14T06:33:08.701 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 4.515e-06 2024-09-14T06:33:08.701 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:08.701 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:08.702 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "dispatched", 2024-09-14T06:33:08.702 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:31:52.743489+0000", 2024-09-14T06:33:08.702 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.1411e-05 2024-09-14T06:33:08.702 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:08.702 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:08.702 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "queued_for_pg", 2024-09-14T06:33:08.702 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:31:52.743494+0000", 2024-09-14T06:33:08.702 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.1649999999999997e-06 2024-09-14T06:33:08.702 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:08.702 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:08.702 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:33:08.702 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:31:52.743548+0000", 2024-09-14T06:33:08.702 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 5.3758999999999998e-05 2024-09-14T06:33:08.702 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:08.703 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:08.703 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "waiting for peered", 2024-09-14T06:33:08.703 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:31:52.743558+0000", 2024-09-14T06:33:08.703 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.0336999999999999e-05 2024-09-14T06:33:08.703 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:08.703 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:08.703 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "reached_pg", 2024-09-14T06:33:08.703 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:31:53.747716+0000", 2024-09-14T06:33:08.703 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 1.004157035 2024-09-14T06:33:08.703 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:08.703 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:08.703 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "started", 2024-09-14T06:33:08.703 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:31:53.747831+0000", 2024-09-14T06:33:08.704 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.000115394 2024-09-14T06:33:08.704 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:08.704 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:08.704 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "waiting for subops from 3", 2024-09-14T06:33:08.704 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:31:53.748002+0000", 2024-09-14T06:33:08.704 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.00017085099999999999 2024-09-14T06:33:08.704 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:08.704 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:08.704 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "sub_op_commit_rec", 2024-09-14T06:33:08.704 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:31:53.749536+0000", 2024-09-14T06:33:08.704 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.0015339990000000001 2024-09-14T06:33:08.704 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:08.704 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:08.704 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "op_commit", 2024-09-14T06:33:08.705 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:31:53.750672+0000", 2024-09-14T06:33:08.705 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 0.001135973 2024-09-14T06:33:08.705 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:08.705 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:08.705 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "commit_sent", 2024-09-14T06:33:08.705 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:31:53.750704+0000", 2024-09-14T06:33:08.705 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.1791000000000001e-05 2024-09-14T06:33:08.705 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-09-14T06:33:08.705 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-09-14T06:33:08.705 INFO:teuthology.orchestra.run.smithi138.stdout: "event": "done", 2024-09-14T06:33:08.705 INFO:teuthology.orchestra.run.smithi138.stdout: "time": "2024-09-14T06:31:53.750743+0000", 2024-09-14T06:33:08.705 INFO:teuthology.orchestra.run.smithi138.stdout: "duration": 3.9450999999999999e-05 2024-09-14T06:33:08.705 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:33:08.705 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:33:08.706 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:33:08.706 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-09-14T06:33:08.706 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-09-14T06:33:08.706 INFO:teuthology.orchestra.run.smithi138.stdout:} 2024-09-14T06:33:08.706 INFO:teuthology.orchestra.run.smithi094.stderr:reweighted osd.4 to 1 (10000) 2024-09-14T06:33:08.710 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-14T06:33:08.720 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 5 1 2024-09-14T06:33:09.712 INFO:teuthology.orchestra.run.smithi094.stderr:noscrub is unset 2024-09-14T06:33:09.716 INFO:teuthology.orchestra.run.smithi094.stderr:reweighted osd.5 to 1 (10000) 2024-09-14T06:33:09.729 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-14T06:33:09.732 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 7 1 2024-09-14T06:33:10.619 INFO:teuthology.orchestra.run.smithi094.stderr:reweighted osd.7 to 1 (10000) 2024-09-14T06:33:10.622 INFO:teuthology.orchestra.run.smithi094.stderr:nodeep-scrub is unset 2024-09-14T06:33:10.635 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 3 1 2024-09-14T06:33:11.719 INFO:teuthology.orchestra.run.smithi094.stderr:reweighted osd.3 to 1 (10000) 2024-09-14T06:33:11.735 INFO:tasks.ceph.ceph_manager.ceph:waiting for recovery to complete 2024-09-14T06:33:11.736 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-14T06:33:12.020 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-14T06:33:12.021 INFO:teuthology.orchestra.run.smithi094.stderr:dumped all 2024-09-14T06:33:12.033 INFO:teuthology.orchestra.run.smithi094.stdout:{"pg_ready":true,"pg_map":{"version":395,"stamp":"2024-09-14T06:33:11.719567+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590406,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":240161,"num_read_kb":2184991,"num_write":38248,"num_write_kb":1480016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6299,"num_bytes_recovered":638497624,"num_keys_recovered":546,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":28723,"ondisk_log_size":28723,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":6,"num_per_pool_osds":6,"num_per_pool_omap_osds":6,"kb":566231040,"kb_used":456004,"kb_used_data":18752,"kb_used_omap":9,"kb_used_meta":436854,"kb_avail":565775036,"statfs":{"total":579820584960,"available":579353636864,"internally_reserved":0,"allocated":19202048,"data_stored":10594348,"data_compressed":65166,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":9563,"internal_metadata":447339173},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-27728395,"num_objects":-61,"num_object_clones":0,"num_object_copies":-122,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-61,"num_whiteouts":0,"num_read":661,"num_read_kb":571,"num_write":168,"num_write_kb":42,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.016974"},"pg_stats":[{"pgid":"3.1f","version":"262'855","reported_seq":8379,"reported_epoch":262,"state":"active+clean","last_fresh":"2024-09-14T06:33:07.706447+0000","last_change":"2024-09-14T06:33:07.706447+0000","last_active":"2024-09-14T06:33:07.706447+0000","last_peered":"2024-09-14T06:33:07.706447+0000","last_clean":"2024-09-14T06:33:07.706447+0000","last_became_active":"2024-09-14T06:29:00.579883+0000","last_became_peered":"2024-09-14T06:29:00.579883+0000","last_unstale":"2024-09-14T06:33:07.706447+0000","last_undegraded":"2024-09-14T06:33:07.706447+0000","last_fullsized":"2024-09-14T06:33:07.706447+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":855,"log_dups_size":0,"ondisk_log_size":855,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T14:40:25.634308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001666641,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":8468,"num_read_kb":86898,"num_write":1171,"num_write_kb":49543,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"3","length":"1"},{"start":"5","length":"1"},{"start":"7","length":"1"},{"start":"9","length":"1"},{"start":"b","length":"1"},{"start":"d","length":"1"}]},{"pgid":"3.1e","version":"262'783","reported_seq":10485,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-09-14T06:33:09.715079+0000","last_change":"2024-09-14T06:33:07.770279+0000","last_active":"2024-09-14T06:33:09.715079+0000","last_peered":"2024-09-14T06:33:09.715079+0000","last_clean":"2024-09-14T06:33:09.715079+0000","last_became_active":"2024-09-14T06:33:04.626456+0000","last_became_peered":"2024-09-14T06:33:04.626456+0000","last_unstale":"2024-09-14T06:33:09.715079+0000","last_undegraded":"2024-09-14T06:33:09.715079+0000","last_fullsized":"2024-09-14T06:33:09.715079+0000","mapping_epoch":258,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":259,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":783,"log_dups_size":0,"ondisk_log_size":783,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T16:31:58.414465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013639839999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":8315,"num_read_kb":67657,"num_write":1025,"num_write_kb":40407,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":869,"num_bytes_recovered":87221212,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"3","length":"1"},{"start":"5","length":"1"},{"start":"7","length":"1"},{"start":"9","length":"1"},{"start":"b","length":"1"},{"start":"d","length":"1"}]},{"pgid":"3.1d","version":"260'1077","reported_seq":7488,"reported_epoch":262,"state":"active+clean","last_fresh":"2024-09-14T06:33:07.704297+0000","last_change":"2024-09-14T06:33:07.704498+0000","last_active":"2024-09-14T06:33:07.704297+0000","last_peered":"2024-09-14T06:33:07.704297+0000","last_clean":"2024-09-14T06:33:07.704297+0000","last_became_active":"2024-09-14T06:32:24.716363+0000","last_became_peered":"2024-09-14T06:32:24.716363+0000","last_unstale":"2024-09-14T06:33:07.704297+0000","last_undegraded":"2024-09-14T06:33:07.704297+0000","last_fullsized":"2024-09-14T06:33:07.704297+0000","mapping_epoch":218,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":219,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":1077,"log_dups_size":0,"ondisk_log_size":1077,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T13:04:18.531851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00055792899999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":7873,"num_read_kb":64375,"num_write":1555,"num_write_kb":53292,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"3","length":"1"},{"start":"5","length":"1"},{"start":"7","length":"1"},{"start":"9","length":"1"},{"start":"b","length":"1"},{"start":"d","length":"1"}]},{"pgid":"3.1c","version":"262'805","reported_seq":6023,"reported_epoch":263,"state":"active+clean","last_fresh":"2024-09-14T06:33:08.705608+0000","last_change":"2024-09-14T06:33:07.704704+0000","last_active":"2024-09-14T06:33:08.705608+0000","last_peered":"2024-09-14T06:33:08.705608+0000","last_clean":"2024-09-14T06:33:08.705608+0000","last_became_active":"2024-09-14T06:32:39.775788+0000","last_became_peered":"2024-09-14T06:32:39.775788+0000","last_unstale":"2024-09-14T06:33:08.705608+0000","last_undegraded":"2024-09-14T06:33:08.705608+0000","last_fullsized":"2024-09-14T06:33:08.705608+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":805,"log_dups_size":0,"ondisk_log_size":805,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T13:58:06.384672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00068758600000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":4956,"num_read_kb":43812,"num_write":946,"num_write_kb":45551,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":505,"num_bytes_recovered":44145197,"num_keys_recovered":280,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"3","length":"1"},{"start":"5","length":"1"},{"start":"7","length":"1"},{"start":"9","length":"1"},{"start":"b","length":"1"},{"start":"d","length":"1"}]},{"pgid":"3.1b","version":"259'960","reported_seq":11470,"reported_epoch":262,"state":"active+clean","last_fresh":"2024-09-14T06:33:07.706551+0000","last_change":"2024-09-14T06:33:07.706745+0000","last_active":"2024-09-14T06:33:07.706551+0000","last_peered":"2024-09-14T06:33:07.706551+0000","last_clean":"2024-09-14T06:33:07.706551+0000","last_became_active":"2024-09-14T06:29:00.586183+0000","last_became_peered":"2024-09-14T06:29:00.586183+0000","last_unstale":"2024-09-14T06:33:07.706551+0000","last_undegraded":"2024-09-14T06:33:07.706551+0000","last_fullsized":"2024-09-14T06:33:07.706551+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":960,"log_dups_size":0,"ondisk_log_size":960,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T10:55:09.317291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001810606,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":11910,"num_read_kb":108094,"num_write":1463,"num_write_kb":55102,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"3","length":"1"},{"start":"5","length":"1"},{"start":"7","length":"1"},{"start":"9","length":"1"},{"start":"b","length":"1"},{"start":"d","length":"1"}]},{"pgid":"3.1a","version":"260'1678","reported_seq":10585,"reported_epoch":262,"state":"active+clean","last_fresh":"2024-09-14T06:33:07.738791+0000","last_change":"2024-09-14T06:33:07.705134+0000","last_active":"2024-09-14T06:33:07.738791+0000","last_peered":"2024-09-14T06:33:07.738791+0000","last_clean":"2024-09-14T06:33:07.738791+0000","last_became_active":"2024-09-14T06:29:00.576206+0000","last_became_peered":"2024-09-14T06:29:00.576206+0000","last_unstale":"2024-09-14T06:33:07.738791+0000","last_undegraded":"2024-09-14T06:33:07.738791+0000","last_fullsized":"2024-09-14T06:33:07.738791+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":1678,"log_dups_size":0,"ondisk_log_size":1678,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T10:47:38.190388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088286099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":13339,"num_read_kb":82904,"num_write":2451,"num_write_kb":55687,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"3","length":"1"},{"start":"5","length":"1"},{"start":"7","length":"1"},{"start":"9","length":"1"},{"start":"b","length":"1"},{"start":"d","length":"1"}]},{"pgid":"3.19","version":"261'561","reported_seq":6457,"reported_epoch":263,"state":"active+clean","last_fresh":"2024-09-14T06:33:08.705471+0000","last_change":"2024-09-14T06:33:07.704594+0000","last_active":"2024-09-14T06:33:08.705471+0000","last_peered":"2024-09-14T06:33:08.705471+0000","last_clean":"2024-09-14T06:33:08.705471+0000","last_became_active":"2024-09-14T06:31:53.744800+0000","last_became_peered":"2024-09-14T06:31:53.744800+0000","last_unstale":"2024-09-14T06:33:08.705471+0000","last_undegraded":"2024-09-14T06:33:08.705471+0000","last_fullsized":"2024-09-14T06:33:08.705471+0000","mapping_epoch":188,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":189,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":561,"log_dups_size":0,"ondisk_log_size":561,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T13:59:24.308993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062201200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":5123,"num_read_kb":67781,"num_write":764,"num_write_kb":43873,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":282,"num_bytes_recovered":41927489,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"3","length":"1"},{"start":"5","length":"1"},{"start":"7","length":"1"},{"start":"9","length":"1"},{"start":"b","length":"1"},{"start":"d","length":"1"}]},{"pgid":"3.18","version":"262'547","reported_seq":6241,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-09-14T06:33:09.714660+0000","last_change":"2024-09-14T06:33:07.769587+0000","last_active":"2024-09-14T06:33:09.714660+0000","last_peered":"2024-09-14T06:33:09.714660+0000","last_clean":"2024-09-14T06:33:09.714660+0000","last_became_active":"2024-09-14T06:31:53.745159+0000","last_became_peered":"2024-09-14T06:31:53.745159+0000","last_unstale":"2024-09-14T06:33:09.714660+0000","last_undegraded":"2024-09-14T06:33:09.714660+0000","last_fullsized":"2024-09-14T06:33:09.714660+0000","mapping_epoch":188,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":189,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":547,"log_dups_size":0,"ondisk_log_size":547,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T13:08:27.610323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00079631400000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4489,"num_read_kb":78917,"num_write":672,"num_write_kb":51581,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":310,"num_bytes_recovered":56082976,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"3","length":"1"},{"start":"5","length":"1"},{"start":"7","length":"1"},{"start":"9","length":"1"},{"start":"b","length":"1"},{"start":"d","length":"1"}]},{"pgid":"3.17","version":"261'1037","reported_seq":9677,"reported_epoch":262,"state":"active+clean","last_fresh":"2024-09-14T06:33:07.706630+0000","last_change":"2024-09-14T06:33:07.706863+0000","last_active":"2024-09-14T06:33:07.706630+0000","last_peered":"2024-09-14T06:33:07.706630+0000","last_clean":"2024-09-14T06:33:07.706630+0000","last_became_active":"2024-09-14T06:29:00.579714+0000","last_became_peered":"2024-09-14T06:29:00.579714+0000","last_unstale":"2024-09-14T06:33:07.706630+0000","last_undegraded":"2024-09-14T06:33:07.706630+0000","last_fullsized":"2024-09-14T06:33:07.706630+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":1037,"log_dups_size":0,"ondisk_log_size":1037,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T15:17:51.610057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0017747889999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":10688,"num_read_kb":68964,"num_write":1162,"num_write_kb":48482,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"3","length":"1"},{"start":"5","length":"1"},{"start":"7","length":"1"},{"start":"9","length":"1"},{"start":"b","length":"1"},{"start":"d","length":"1"}]},{"pgid":"3.16","version":"255'1156","reported_seq":4158,"reported_epoch":262,"state":"active+clean","last_fresh":"2024-09-14T06:33:07.769586+0000","last_change":"2024-09-14T06:33:07.769751+0000","last_active":"2024-09-14T06:33:07.769586+0000","last_peered":"2024-09-14T06:33:07.769586+0000","last_clean":"2024-09-14T06:33:07.769586+0000","last_became_active":"2024-09-14T06:32:24.718745+0000","last_became_peered":"2024-09-14T06:32:24.718745+0000","last_unstale":"2024-09-14T06:33:07.769586+0000","last_undegraded":"2024-09-14T06:33:07.769586+0000","last_fullsized":"2024-09-14T06:33:07.769586+0000","mapping_epoch":218,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":219,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":1156,"log_dups_size":0,"ondisk_log_size":1156,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T17:51:23.869206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059450099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":5658,"num_read_kb":38562,"num_write":1743,"num_write_kb":37248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"3","length":"1"},{"start":"5","length":"1"},{"start":"7","length":"1"},{"start":"9","length":"1"},{"start":"b","length":"1"},{"start":"d","length":"1"}]},{"pgid":"3.15","version":"261'650","reported_seq":6699,"reported_epoch":262,"state":"active+clean","last_fresh":"2024-09-14T06:33:07.769450+0000","last_change":"2024-09-14T06:33:07.769581+0000","last_active":"2024-09-14T06:33:07.769450+0000","last_peered":"2024-09-14T06:33:07.769450+0000","last_clean":"2024-09-14T06:33:07.769450+0000","last_became_active":"2024-09-14T06:31:53.746091+0000","last_became_peered":"2024-09-14T06:31:53.746091+0000","last_unstale":"2024-09-14T06:33:07.769450+0000","last_undegraded":"2024-09-14T06:33:07.769450+0000","last_fullsized":"2024-09-14T06:33:07.769450+0000","mapping_epoch":188,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":189,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":650,"log_dups_size":0,"ondisk_log_size":650,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T12:36:31.690559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000442557,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":5101,"num_read_kb":74899,"num_write":696,"num_write_kb":51196,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":271,"num_bytes_recovered":49981473,"num_keys_recovered":33,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"3","length":"1"},{"start":"5","length":"1"},{"start":"7","length":"1"},{"start":"9","length":"1"},{"start":"b","length":"1"},{"start":"d","length":"1"}]},{"pgid":"3.14","version":"261'1137","reported_seq":6288,"reported_epoch":262,"state":"active+clean","last_fresh":"2024-09-14T06:33:07.704499+0000","last_change":"2024-09-14T06:33:07.704760+0000","last_active":"2024-09-14T06:33:07.704499+0000","last_peered":"2024-09-14T06:33:07.704499+0000","last_clean":"2024-09-14T06:33:07.704499+0000","last_became_active":"2024-09-14T06:33:04.628663+0000","last_became_peered":"2024-09-14T06:33:04.628663+0000","last_unstale":"2024-09-14T06:33:07.704499+0000","last_undegraded":"2024-09-14T06:33:07.704499+0000","last_fullsized":"2024-09-14T06:33:07.704499+0000","mapping_epoch":258,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":259,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":1137,"log_dups_size":0,"ondisk_log_size":1137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T11:42:47.162926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00078945399999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":8948,"num_read_kb":46936,"num_write":1573,"num_write_kb":29829,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":296,"num_bytes_recovered":6118769,"num_keys_recovered":11,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"3","length":"1"},{"start":"5","length":"1"},{"start":"7","length":"1"},{"start":"9","length":"1"},{"start":"b","length":"1"},{"start":"d","length":"1"}]},{"pgid":"2.7","version":"0'0","reported_seq":148,"reported_epoch":262,"state":"active+clean","last_fresh":"2024-09-14T06:33:07.769239+0000","last_change":"2024-09-14T06:33:01.625180+0000","last_active":"2024-09-14T06:33:07.769239+0000","last_peered":"2024-09-14T06:33:07.769239+0000","last_clean":"2024-09-14T06:33:07.769239+0000","last_became_active":"2024-09-14T06:33:01.624918+0000","last_became_peered":"2024-09-14T06:33:01.624918+0000","last_unstale":"2024-09-14T06:33:07.769239+0000","last_undegraded":"2024-09-14T06:33:07.769239+0000","last_fullsized":"2024-09-14T06:33:07.769239+0000","mapping_epoch":255,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":256,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:28:50.205350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:28:50.205350+0000","last_clean_scrub_stamp":"2024-09-14T06:28:50.205350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T10:19:32.184994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.6","version":"262'893","reported_seq":8674,"reported_epoch":262,"state":"active+clean","last_fresh":"2024-09-14T06:33:07.705814+0000","last_change":"2024-09-14T06:33:07.705814+0000","last_active":"2024-09-14T06:33:07.705814+0000","last_peered":"2024-09-14T06:33:07.705814+0000","last_clean":"2024-09-14T06:33:07.705814+0000","last_became_active":"2024-09-14T06:29:00.578283+0000","last_became_peered":"2024-09-14T06:29:00.578283+0000","last_unstale":"2024-09-14T06:33:07.705814+0000","last_undegraded":"2024-09-14T06:33:07.705814+0000","last_fullsized":"2024-09-14T06:33:07.705814+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":893,"log_dups_size":0,"ondisk_log_size":893,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T15:09:37.855181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001076554,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":8608,"num_read_kb":78770,"num_write":923,"num_write_kb":48303,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"3","length":"1"},{"start":"5","length":"1"},{"start":"7","length":"1"},{"start":"9","length":"1"},{"start":"b","length":"1"},{"start":"d","length":"1"}]},{"pgid":"2.6","version":"0'0","reported_seq":509,"reported_epoch":263,"state":"active+clean","last_fresh":"2024-09-14T06:33:08.705755+0000","last_change":"2024-09-14T06:32:01.608456+0000","last_active":"2024-09-14T06:33:08.705755+0000","last_peered":"2024-09-14T06:33:08.705755+0000","last_clean":"2024-09-14T06:33:08.705755+0000","last_became_active":"2024-09-14T06:32:01.608178+0000","last_became_peered":"2024-09-14T06:32:01.608178+0000","last_unstale":"2024-09-14T06:33:08.705755+0000","last_undegraded":"2024-09-14T06:33:08.705755+0000","last_fullsized":"2024-09-14T06:33:08.705755+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:28:50.205350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:28:50.205350+0000","last_clean_scrub_stamp":"2024-09-14T06:28:50.205350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T10:16:40.480031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065018200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"261'840","reported_seq":4605,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-09-14T06:33:09.714940+0000","last_change":"2024-09-14T06:33:07.769882+0000","last_active":"2024-09-14T06:33:09.714940+0000","last_peered":"2024-09-14T06:33:09.714940+0000","last_clean":"2024-09-14T06:33:09.714940+0000","last_became_active":"2024-09-14T06:31:54.747923+0000","last_became_peered":"2024-09-14T06:31:54.747923+0000","last_unstale":"2024-09-14T06:33:09.714940+0000","last_undegraded":"2024-09-14T06:33:09.714940+0000","last_fullsized":"2024-09-14T06:33:09.714940+0000","mapping_epoch":189,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":190,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":840,"log_dups_size":0,"ondisk_log_size":840,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T09:01:49.581187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000927559,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4591,"num_read_kb":41129,"num_write":1178,"num_write_kb":36122,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":183,"num_bytes_recovered":13912513,"num_keys_recovered":22,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"3","length":"1"},{"start":"5","length":"1"},{"start":"7","length":"1"},{"start":"9","length":"1"},{"start":"b","length":"1"},{"start":"d","length":"1"}]},{"pgid":"2.5","version":"0'0","reported_seq":502,"reported_epoch":262,"state":"active+clean","last_fresh":"2024-09-14T06:33:07.769389+0000","last_change":"2024-09-14T06:28:53.640035+0000","last_active":"2024-09-14T06:33:07.769389+0000","last_peered":"2024-09-14T06:33:07.769389+0000","last_clean":"2024-09-14T06:33:07.769389+0000","last_became_active":"2024-09-14T06:28:51.308058+0000","last_became_peered":"2024-09-14T06:28:51.308058+0000","last_unstale":"2024-09-14T06:33:07.769389+0000","last_undegraded":"2024-09-14T06:33:07.769389+0000","last_fullsized":"2024-09-14T06:33:07.769389+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:28:50.205350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:28:50.205350+0000","last_clean_scrub_stamp":"2024-09-14T06:28:50.205350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T13:35:48.544057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00037824000000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"262'700","reported_seq":4458,"reported_epoch":263,"state":"active+clean","last_fresh":"2024-09-14T06:33:08.705657+0000","last_change":"2024-09-14T06:33:07.704937+0000","last_active":"2024-09-14T06:33:08.705657+0000","last_peered":"2024-09-14T06:33:08.705657+0000","last_clean":"2024-09-14T06:33:08.705657+0000","last_became_active":"2024-09-14T06:33:04.628919+0000","last_became_peered":"2024-09-14T06:33:04.628919+0000","last_unstale":"2024-09-14T06:33:08.705657+0000","last_undegraded":"2024-09-14T06:33:08.705657+0000","last_fullsized":"2024-09-14T06:33:08.705657+0000","mapping_epoch":258,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":259,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":700,"log_dups_size":0,"ondisk_log_size":700,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T09:33:23.339333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00074004799999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":2654,"num_read_kb":37435,"num_write":897,"num_write_kb":40860,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":509,"num_bytes_recovered":31411166,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"3","length":"1"},{"start":"5","length":"1"},{"start":"7","length":"1"},{"start":"9","length":"1"},{"start":"b","length":"1"},{"start":"d","length":"1"}]},{"pgid":"2.4","version":"0'0","reported_seq":504,"reported_epoch":263,"state":"active+clean","last_fresh":"2024-09-14T06:33:08.705759+0000","last_change":"2024-09-14T06:28:53.226072+0000","last_active":"2024-09-14T06:33:08.705759+0000","last_peered":"2024-09-14T06:33:08.705759+0000","last_clean":"2024-09-14T06:33:08.705759+0000","last_became_active":"2024-09-14T06:28:51.308072+0000","last_became_peered":"2024-09-14T06:28:51.308072+0000","last_unstale":"2024-09-14T06:33:08.705759+0000","last_undegraded":"2024-09-14T06:33:08.705759+0000","last_fullsized":"2024-09-14T06:33:08.705759+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:28:50.205350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:28:50.205350+0000","last_clean_scrub_stamp":"2024-09-14T06:28:50.205350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T13:15:09.052328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066197399999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"261'896","reported_seq":5621,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-09-14T06:33:09.715085+0000","last_change":"2024-09-14T06:33:07.770566+0000","last_active":"2024-09-14T06:33:09.715085+0000","last_peered":"2024-09-14T06:33:09.715085+0000","last_clean":"2024-09-14T06:33:09.715085+0000","last_became_active":"2024-09-14T06:32:24.716458+0000","last_became_peered":"2024-09-14T06:32:24.716458+0000","last_unstale":"2024-09-14T06:33:09.715085+0000","last_undegraded":"2024-09-14T06:33:09.715085+0000","last_fullsized":"2024-09-14T06:33:09.715085+0000","mapping_epoch":218,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":219,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":896,"log_dups_size":0,"ondisk_log_size":896,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T12:28:03.015849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013146799999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4516,"num_read_kb":42632,"num_write":1255,"num_write_kb":49494,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":330,"num_bytes_recovered":28798893,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"3","length":"1"},{"start":"5","length":"1"},{"start":"7","length":"1"},{"start":"9","length":"1"},{"start":"b","length":"1"},{"start":"d","length":"1"}]},{"pgid":"3.3","version":"262'711","reported_seq":10870,"reported_epoch":262,"state":"active+clean","last_fresh":"2024-09-14T06:33:07.790215+0000","last_change":"2024-09-14T06:33:07.704808+0000","last_active":"2024-09-14T06:33:07.790215+0000","last_peered":"2024-09-14T06:33:07.790215+0000","last_clean":"2024-09-14T06:33:07.790215+0000","last_became_active":"2024-09-14T06:29:00.584678+0000","last_became_peered":"2024-09-14T06:29:00.584678+0000","last_unstale":"2024-09-14T06:33:07.790215+0000","last_undegraded":"2024-09-14T06:33:07.790215+0000","last_fullsized":"2024-09-14T06:33:07.790215+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":711,"log_dups_size":0,"ondisk_log_size":711,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T07:39:08.849874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00074987800000000005,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":9560,"num_read_kb":89311,"num_write":777,"num_write_kb":51691,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"3","length":"1"},{"start":"5","length":"1"},{"start":"7","length":"1"},{"start":"9","length":"1"},{"start":"b","length":"1"},{"start":"d","length":"1"}]},{"pgid":"2.2","version":"22'2","reported_seq":118,"reported_epoch":263,"state":"active+clean","last_fresh":"2024-09-14T06:33:08.705546+0000","last_change":"2024-09-14T06:32:24.716937+0000","last_active":"2024-09-14T06:33:08.705546+0000","last_peered":"2024-09-14T06:33:08.705546+0000","last_clean":"2024-09-14T06:33:08.705546+0000","last_became_active":"2024-09-14T06:32:24.716539+0000","last_became_peered":"2024-09-14T06:32:24.716539+0000","last_unstale":"2024-09-14T06:33:08.705546+0000","last_undegraded":"2024-09-14T06:33:08.705546+0000","last_fullsized":"2024-09-14T06:33:08.705546+0000","mapping_epoch":218,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":219,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:28:50.205350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:28:50.205350+0000","last_clean_scrub_stamp":"2024-09-14T06:28:50.205350+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T07:16:29.639469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"260'1116","reported_seq":7457,"reported_epoch":263,"state":"active+clean","last_fresh":"2024-09-14T06:33:08.705427+0000","last_change":"2024-09-14T06:33:07.705043+0000","last_active":"2024-09-14T06:33:08.705427+0000","last_peered":"2024-09-14T06:33:08.705427+0000","last_clean":"2024-09-14T06:33:08.705427+0000","last_became_active":"2024-09-14T06:33:04.626601+0000","last_became_peered":"2024-09-14T06:33:04.626601+0000","last_unstale":"2024-09-14T06:33:08.705427+0000","last_undegraded":"2024-09-14T06:33:08.705427+0000","last_fullsized":"2024-09-14T06:33:08.705427+0000","mapping_epoch":258,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":259,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":1116,"log_dups_size":0,"ondisk_log_size":1116,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T07:23:09.155401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064233900000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":8259,"num_read_kb":85480,"num_write":1652,"num_write_kb":58700,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":179,"num_bytes_recovered":2852,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"3","length":"1"},{"start":"5","length":"1"},{"start":"7","length":"1"},{"start":"9","length":"1"},{"start":"b","length":"1"},{"start":"d","length":"1"}]},{"pgid":"2.1","version":"0'0","reported_seq":78,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-09-14T06:33:09.714596+0000","last_change":"2024-09-14T06:33:01.625894+0000","last_active":"2024-09-14T06:33:09.714596+0000","last_peered":"2024-09-14T06:33:09.714596+0000","last_clean":"2024-09-14T06:33:09.714596+0000","last_became_active":"2024-09-14T06:33:01.625561+0000","last_became_peered":"2024-09-14T06:33:01.625561+0000","last_unstale":"2024-09-14T06:33:09.714596+0000","last_undegraded":"2024-09-14T06:33:09.714596+0000","last_fullsized":"2024-09-14T06:33:09.714596+0000","mapping_epoch":255,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":256,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:28:50.205350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:28:50.205350+0000","last_clean_scrub_stamp":"2024-09-14T06:28:50.205350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T14:21:03.308190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053403100000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1","version":"261'822","reported_seq":11835,"reported_epoch":262,"state":"active+clean","last_fresh":"2024-09-14T06:33:07.705164+0000","last_change":"2024-09-14T06:33:07.705685+0000","last_active":"2024-09-14T06:33:07.705164+0000","last_peered":"2024-09-14T06:33:07.705164+0000","last_clean":"2024-09-14T06:33:07.705164+0000","last_became_active":"2024-09-14T06:30:57.768238+0000","last_became_peered":"2024-09-14T06:30:57.768238+0000","last_unstale":"2024-09-14T06:33:07.705164+0000","last_undegraded":"2024-09-14T06:33:07.705164+0000","last_fullsized":"2024-09-14T06:33:07.705164+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":822,"log_dups_size":0,"ondisk_log_size":822,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T15:07:18.804416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011933110000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":11284,"num_read_kb":112430,"num_write":991,"num_write_kb":47408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":183,"num_bytes_recovered":30431994,"num_keys_recovered":22,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"3","length":"1"},{"start":"5","length":"1"},{"start":"7","length":"1"},{"start":"9","length":"1"},{"start":"b","length":"1"},{"start":"d","length":"1"}]},{"pgid":"2.0","version":"0'0","reported_seq":502,"reported_epoch":262,"state":"active+clean","last_fresh":"2024-09-14T06:33:07.768921+0000","last_change":"2024-09-14T06:28:53.640096+0000","last_active":"2024-09-14T06:33:07.768921+0000","last_peered":"2024-09-14T06:33:07.768921+0000","last_clean":"2024-09-14T06:33:07.768921+0000","last_became_active":"2024-09-14T06:28:51.223069+0000","last_became_peered":"2024-09-14T06:28:51.223069+0000","last_unstale":"2024-09-14T06:33:07.768921+0000","last_undegraded":"2024-09-14T06:33:07.768921+0000","last_fullsized":"2024-09-14T06:33:07.768921+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:28:50.205350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:28:50.205350+0000","last_clean_scrub_stamp":"2024-09-14T06:28:50.205350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T15:13:00.471509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00043714600000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"262'1663","reported_seq":7806,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-09-14T06:33:09.714882+0000","last_change":"2024-09-14T06:33:07.770369+0000","last_active":"2024-09-14T06:33:09.714882+0000","last_peered":"2024-09-14T06:33:09.714882+0000","last_clean":"2024-09-14T06:33:09.714882+0000","last_became_active":"2024-09-14T06:31:54.744160+0000","last_became_peered":"2024-09-14T06:31:54.744160+0000","last_unstale":"2024-09-14T06:33:09.714882+0000","last_undegraded":"2024-09-14T06:33:09.714882+0000","last_fullsized":"2024-09-14T06:33:09.714882+0000","mapping_epoch":189,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":190,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":1663,"log_dups_size":0,"ondisk_log_size":1663,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T16:44:35.569843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001382608,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":10723,"num_read_kb":70589,"num_write":2596,"num_write_kb":53564,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":327,"num_bytes_recovered":50047063,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"3","length":"1"},{"start":"5","length":"1"},{"start":"7","length":"1"},{"start":"9","length":"1"},{"start":"b","length":"1"},{"start":"d","length":"1"}]},{"pgid":"2.3","version":"20'1","reported_seq":27,"reported_epoch":262,"state":"active+clean","last_fresh":"2024-09-14T06:33:07.769008+0000","last_change":"2024-09-14T06:33:01.625025+0000","last_active":"2024-09-14T06:33:07.769008+0000","last_peered":"2024-09-14T06:33:07.769008+0000","last_clean":"2024-09-14T06:33:07.769008+0000","last_became_active":"2024-09-14T06:33:01.622409+0000","last_became_peered":"2024-09-14T06:33:01.622409+0000","last_unstale":"2024-09-14T06:33:07.769008+0000","last_undegraded":"2024-09-14T06:33:07.769008+0000","last_fullsized":"2024-09-14T06:33:07.769008+0000","mapping_epoch":255,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":256,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:28:50.205350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:28:50.205350+0000","last_clean_scrub_stamp":"2024-09-14T06:28:50.205350+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T16:23:56.048067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"17'72","reported_seq":593,"reported_epoch":262,"state":"active+clean","last_fresh":"2024-09-14T06:33:07.768933+0000","last_change":"2024-09-14T06:28:44.839885+0000","last_active":"2024-09-14T06:33:07.768933+0000","last_peered":"2024-09-14T06:33:07.768933+0000","last_clean":"2024-09-14T06:33:07.768933+0000","last_became_active":"2024-09-14T06:28:44.839573+0000","last_became_peered":"2024-09-14T06:28:44.839573+0000","last_unstale":"2024-09-14T06:33:07.768933+0000","last_undegraded":"2024-09-14T06:33:07.768933+0000","last_fullsized":"2024-09-14T06:33:07.768933+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:28:43.828494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:28:43.828494+0000","last_clean_scrub_stamp":"2024-09-14T06:28:43.828494+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T14:41:25.787293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.8","version":"261'684","reported_seq":6613,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-09-14T06:33:09.714692+0000","last_change":"2024-09-14T06:33:07.769379+0000","last_active":"2024-09-14T06:33:09.714692+0000","last_peered":"2024-09-14T06:33:09.714692+0000","last_clean":"2024-09-14T06:33:09.714692+0000","last_became_active":"2024-09-14T06:31:55.754694+0000","last_became_peered":"2024-09-14T06:31:55.754694+0000","last_unstale":"2024-09-14T06:33:09.714692+0000","last_undegraded":"2024-09-14T06:33:09.714692+0000","last_fullsized":"2024-09-14T06:33:09.714692+0000","mapping_epoch":190,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":191,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":684,"log_dups_size":0,"ondisk_log_size":684,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T15:48:47.633183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067851599999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":5793,"num_read_kb":90852,"num_write":921,"num_write_kb":57691,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":319,"num_bytes_recovered":60352269,"num_keys_recovered":22,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"3","length":"1"},{"start":"5","length":"1"},{"start":"7","length":"1"},{"start":"9","length":"1"},{"start":"b","length":"1"},{"start":"d","length":"1"}]},{"pgid":"3.9","version":"261'597","reported_seq":4793,"reported_epoch":262,"state":"active+clean","last_fresh":"2024-09-14T06:33:07.704234+0000","last_change":"2024-09-14T06:33:07.704439+0000","last_active":"2024-09-14T06:33:07.704234+0000","last_peered":"2024-09-14T06:33:07.704234+0000","last_clean":"2024-09-14T06:33:07.704234+0000","last_became_active":"2024-09-14T06:33:04.628736+0000","last_became_peered":"2024-09-14T06:33:04.628736+0000","last_unstale":"2024-09-14T06:33:07.704234+0000","last_undegraded":"2024-09-14T06:33:07.704234+0000","last_fullsized":"2024-09-14T06:33:07.704234+0000","mapping_epoch":258,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":259,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":597,"log_dups_size":0,"ondisk_log_size":597,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T10:39:24.683086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057139500000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3678,"num_read_kb":64291,"num_write":778,"num_write_kb":44928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":158,"num_bytes_recovered":49117,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"3","length":"1"},{"start":"5","length":"1"},{"start":"7","length":"1"},{"start":"9","length":"1"},{"start":"b","length":"1"},{"start":"d","length":"1"}]},{"pgid":"3.a","version":"261'874","reported_seq":5139,"reported_epoch":262,"state":"active+clean","last_fresh":"2024-09-14T06:33:07.704826+0000","last_change":"2024-09-14T06:33:07.705019+0000","last_active":"2024-09-14T06:33:07.704826+0000","last_peered":"2024-09-14T06:33:07.704826+0000","last_clean":"2024-09-14T06:33:07.704826+0000","last_became_active":"2024-09-14T06:33:04.627306+0000","last_became_peered":"2024-09-14T06:33:04.627306+0000","last_unstale":"2024-09-14T06:33:07.704826+0000","last_undegraded":"2024-09-14T06:33:07.704826+0000","last_fullsized":"2024-09-14T06:33:07.704826+0000","mapping_epoch":258,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":259,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":874,"log_dups_size":0,"ondisk_log_size":874,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T11:52:51.272927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097383799999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":5331,"num_read_kb":58329,"num_write":1221,"num_write_kb":47535,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":491,"num_bytes_recovered":26968802,"num_keys_recovered":45,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"3","length":"1"},{"start":"5","length":"1"},{"start":"7","length":"1"},{"start":"9","length":"1"},{"start":"b","length":"1"},{"start":"d","length":"1"}]},{"pgid":"3.b","version":"260'1637","reported_seq":8941,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-09-14T06:33:09.714817+0000","last_change":"2024-09-14T06:33:07.769381+0000","last_active":"2024-09-14T06:33:09.714817+0000","last_peered":"2024-09-14T06:33:09.714817+0000","last_clean":"2024-09-14T06:33:09.714817+0000","last_became_active":"2024-09-14T06:31:55.756219+0000","last_became_peered":"2024-09-14T06:31:55.756219+0000","last_unstale":"2024-09-14T06:33:09.714817+0000","last_undegraded":"2024-09-14T06:33:09.714817+0000","last_fullsized":"2024-09-14T06:33:09.714817+0000","mapping_epoch":190,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":191,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":1637,"log_dups_size":0,"ondisk_log_size":1637,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T15:53:45.757678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065814200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":13043,"num_read_kb":64129,"num_write":2496,"num_write_kb":43679,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":291,"num_bytes_recovered":37370632,"num_keys_recovered":66,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"3","length":"1"},{"start":"5","length":"1"},{"start":"7","length":"1"},{"start":"9","length":"1"},{"start":"b","length":"1"},{"start":"d","length":"1"}]},{"pgid":"3.c","version":"255'622","reported_seq":6293,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-09-14T06:33:09.714912+0000","last_change":"2024-09-14T06:33:07.769705+0000","last_active":"2024-09-14T06:33:09.714912+0000","last_peered":"2024-09-14T06:33:09.714912+0000","last_clean":"2024-09-14T06:33:09.714912+0000","last_became_active":"2024-09-14T06:32:24.716421+0000","last_became_peered":"2024-09-14T06:32:24.716421+0000","last_unstale":"2024-09-14T06:33:09.714912+0000","last_undegraded":"2024-09-14T06:33:09.714912+0000","last_fullsized":"2024-09-14T06:33:09.714912+0000","mapping_epoch":218,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":219,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":622,"log_dups_size":0,"ondisk_log_size":622,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T08:41:04.782849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00083342900000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4842,"num_read_kb":50425,"num_write":795,"num_write_kb":40827,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":293,"num_bytes_recovered":36712421,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"3","length":"1"},{"start":"5","length":"1"},{"start":"7","length":"1"},{"start":"9","length":"1"},{"start":"b","length":"1"},{"start":"d","length":"1"}]},{"pgid":"3.d","version":"261'577","reported_seq":9275,"reported_epoch":262,"state":"active+clean","last_fresh":"2024-09-14T06:33:07.770026+0000","last_change":"2024-09-14T06:33:07.770219+0000","last_active":"2024-09-14T06:33:07.770026+0000","last_peered":"2024-09-14T06:33:07.770026+0000","last_clean":"2024-09-14T06:33:07.770026+0000","last_became_active":"2024-09-14T06:29:00.580362+0000","last_became_peered":"2024-09-14T06:29:00.580362+0000","last_unstale":"2024-09-14T06:33:07.770026+0000","last_undegraded":"2024-09-14T06:33:07.770026+0000","last_fullsized":"2024-09-14T06:33:07.770026+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":577,"log_dups_size":0,"ondisk_log_size":577,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T16:36:38.133044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010003270000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":8199,"num_read_kb":93694,"num_write":732,"num_write_kb":41032,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"3","length":"1"},{"start":"5","length":"1"},{"start":"7","length":"1"},{"start":"9","length":"1"},{"start":"b","length":"1"},{"start":"d","length":"1"}]},{"pgid":"3.e","version":"262'586","reported_seq":5473,"reported_epoch":262,"state":"active+clean","last_fresh":"2024-09-14T06:33:07.769825+0000","last_change":"2024-09-14T06:33:07.769825+0000","last_active":"2024-09-14T06:33:07.769825+0000","last_peered":"2024-09-14T06:33:07.769825+0000","last_clean":"2024-09-14T06:33:07.769825+0000","last_became_active":"2024-09-14T06:32:39.777524+0000","last_became_peered":"2024-09-14T06:32:39.777524+0000","last_unstale":"2024-09-14T06:33:07.769825+0000","last_undegraded":"2024-09-14T06:33:07.769825+0000","last_fullsized":"2024-09-14T06:33:07.769825+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":586,"log_dups_size":0,"ondisk_log_size":586,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T07:31:19.417437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065560200000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3986,"num_read_kb":61138,"num_write":753,"num_write_kb":43787,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":171,"num_bytes_recovered":10913227,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"3","length":"1"},{"start":"5","length":"1"},{"start":"7","length":"1"},{"start":"9","length":"1"},{"start":"b","length":"1"},{"start":"d","length":"1"}]},{"pgid":"3.f","version":"262'712","reported_seq":5895,"reported_epoch":262,"state":"active+clean","last_fresh":"2024-09-14T06:33:07.776689+0000","last_change":"2024-09-14T06:33:07.770138+0000","last_active":"2024-09-14T06:33:07.776689+0000","last_peered":"2024-09-14T06:33:07.776689+0000","last_clean":"2024-09-14T06:33:07.776689+0000","last_became_active":"2024-09-14T06:29:00.586491+0000","last_became_peered":"2024-09-14T06:29:00.586491+0000","last_unstale":"2024-09-14T06:33:07.776689+0000","last_undegraded":"2024-09-14T06:33:07.776689+0000","last_fullsized":"2024-09-14T06:33:07.776689+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":712,"log_dups_size":0,"ondisk_log_size":712,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T14:20:12.511497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000891002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":6081,"num_read_kb":45578,"num_write":836,"num_write_kb":36134,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"3","length":"1"},{"start":"5","length":"1"},{"start":"7","length":"1"},{"start":"9","length":"1"},{"start":"b","length":"1"},{"start":"d","length":"1"}]},{"pgid":"3.10","version":"261'952","reported_seq":6834,"reported_epoch":262,"state":"active+clean","last_fresh":"2024-09-14T06:33:07.706830+0000","last_change":"2024-09-14T06:33:07.707031+0000","last_active":"2024-09-14T06:33:07.706830+0000","last_peered":"2024-09-14T06:33:07.706830+0000","last_clean":"2024-09-14T06:33:07.706830+0000","last_became_active":"2024-09-14T06:32:04.621617+0000","last_became_peered":"2024-09-14T06:32:04.621617+0000","last_unstale":"2024-09-14T06:33:07.706830+0000","last_undegraded":"2024-09-14T06:33:07.706830+0000","last_fullsized":"2024-09-14T06:33:07.706830+0000","mapping_epoch":199,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":200,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":952,"log_dups_size":0,"ondisk_log_size":952,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T06:39:54.136142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012025289999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":7147,"num_read_kb":34474,"num_write":1082,"num_write_kb":36655,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":154,"num_bytes_recovered":9401812,"num_keys_recovered":45,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"3","length":"1"},{"start":"5","length":"1"},{"start":"7","length":"1"},{"start":"9","length":"1"},{"start":"b","length":"1"},{"start":"d","length":"1"}]},{"pgid":"3.11","version":"261'730","reported_seq":11694,"reported_epoch":262,"state":"active+clean","last_fresh":"2024-09-14T06:33:07.769799+0000","last_change":"2024-09-14T06:33:07.769975+0000","last_active":"2024-09-14T06:33:07.769799+0000","last_peered":"2024-09-14T06:33:07.769799+0000","last_clean":"2024-09-14T06:33:07.769799+0000","last_became_active":"2024-09-14T06:32:39.778010+0000","last_became_peered":"2024-09-14T06:32:39.778010+0000","last_unstale":"2024-09-14T06:33:07.769799+0000","last_undegraded":"2024-09-14T06:33:07.769799+0000","last_fullsized":"2024-09-14T06:33:07.769799+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":730,"log_dups_size":0,"ondisk_log_size":730,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T10:48:00.264177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00072597299999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":10271,"num_read_kb":109243,"num_write":859,"num_write_kb":48307,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":177,"num_bytes_recovered":16647747,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"3","length":"1"},{"start":"5","length":"1"},{"start":"7","length":"1"},{"start":"9","length":"1"},{"start":"b","length":"1"},{"start":"d","length":"1"}]},{"pgid":"3.12","version":"261'782","reported_seq":7454,"reported_epoch":262,"state":"active+clean","last_fresh":"2024-09-14T06:33:07.706309+0000","last_change":"2024-09-14T06:33:07.706504+0000","last_active":"2024-09-14T06:33:07.706309+0000","last_peered":"2024-09-14T06:33:07.706309+0000","last_clean":"2024-09-14T06:33:07.706309+0000","last_became_active":"2024-09-14T06:29:00.579826+0000","last_became_peered":"2024-09-14T06:29:00.579826+0000","last_unstale":"2024-09-14T06:33:07.706309+0000","last_undegraded":"2024-09-14T06:33:07.706309+0000","last_fullsized":"2024-09-14T06:33:07.706309+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":782,"log_dups_size":0,"ondisk_log_size":782,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T11:32:57.816012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001680792,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":7110,"num_read_kb":59089,"num_write":947,"num_write_kb":42441,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"3","length":"1"},{"start":"5","length":"1"},{"start":"7","length":"1"},{"start":"9","length":"1"},{"start":"b","length":"1"},{"start":"d","length":"1"}]},{"pgid":"3.13","version":"261'1008","reported_seq":8907,"reported_epoch":262,"state":"active+clean","last_fresh":"2024-09-14T06:33:07.769416+0000","last_change":"2024-09-14T06:33:07.769558+0000","last_active":"2024-09-14T06:33:07.769416+0000","last_peered":"2024-09-14T06:33:07.769416+0000","last_clean":"2024-09-14T06:33:07.769416+0000","last_became_active":"2024-09-14T06:29:00.586520+0000","last_became_peered":"2024-09-14T06:29:00.586520+0000","last_unstale":"2024-09-14T06:33:07.769416+0000","last_undegraded":"2024-09-14T06:33:07.769416+0000","last_fullsized":"2024-09-14T06:33:07.769416+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":1008,"log_dups_size":0,"ondisk_log_size":1008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T07:25:04.666000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041539399999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":9551,"num_read_kb":66119,"num_write":1235,"num_write_kb":47437,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"3","length":"1"},{"start":"5","length":"1"},{"start":"7","length":"1"},{"start":"9","length":"1"},{"start":"b","length":"1"},{"start":"d","length":"1"}]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":19,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":240095,"num_read_kb":2184936,"num_write":38145,"num_write_kb":1478386,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6298,"num_bytes_recovered":638497624,"num_keys_recovered":546,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":6868992,"data_stored":66096213,"data_compressed":4010572,"data_compressed_allocated":5459968,"data_compressed_original":65020628,"omap_allocated":0,"internal_metadata":0},"log_size":28648,"ondisk_log_size":28648,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":131072,"data_stored":1180736,"data_compressed":65166,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509496,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":70616,"kb_used_data":3160,"kb_used_omap":1,"kb_used_meta":67262,"kb_avail":94301224,"statfs":{"total":96636764160,"available":96564453376,"internally_reserved":0,"allocated":3235840,"data_stored":2156460,"data_compressed":32583,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":1590,"internal_metadata":68876746},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":0,"seq":64424509496,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509497,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":74556,"kb_used_data":3100,"kb_used_omap":1,"kb_used_meta":71422,"kb_avail":94297284,"statfs":{"total":96636764160,"available":96560418816,"internally_reserved":0,"allocated":3174400,"data_stored":1566111,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":73136581},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509496,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":88436,"kb_used_data":3100,"kb_used_omap":1,"kb_used_meta":85310,"kb_avail":94283404,"statfs":{"total":96636764160,"available":96546205696,"internally_reserved":0,"allocated":3174400,"data_stored":1566111,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":87357893},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509497,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":56580,"kb_used_data":3104,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94315260,"statfs":{"total":96636764160,"available":96578826240,"internally_reserved":0,"allocated":3178496,"data_stored":1566130,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":0,"seq":60129542200,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542199,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":82932,"kb_used_data":3168,"kb_used_omap":1,"kb_used_meta":79742,"kb_avail":94288908,"statfs":{"total":96636764160,"available":96551841792,"internally_reserved":0,"allocated":3244032,"data_stored":2161370,"data_compressed":32583,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":1595,"internal_metadata":81656261},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574905,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":82884,"kb_used_data":3120,"kb_used_omap":1,"kb_used_meta":79742,"kb_avail":94288956,"statfs":{"total":96636764160,"available":96551890944,"internally_reserved":0,"allocated":3194880,"data_stored":1578166,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":81656263},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32583,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32583,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":229376,"data_stored":726925,"data_compressed":74239,"data_compressed_allocated":98304,"data_compressed_original":626688,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":6631424,"data_stored":65369250,"data_compressed":3936333,"data_compressed_allocated":5361664,"data_compressed_original":64393940,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-14T06:33:12.035 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-14T06:33:12.322 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-14T06:33:12.323 INFO:teuthology.orchestra.run.smithi094.stderr:dumped all 2024-09-14T06:33:12.335 INFO:teuthology.orchestra.run.smithi094.stdout:{"pg_ready":true,"pg_map":{"version":395,"stamp":"2024-09-14T06:33:11.719567+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590406,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":240161,"num_read_kb":2184991,"num_write":38248,"num_write_kb":1480016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6299,"num_bytes_recovered":638497624,"num_keys_recovered":546,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":28723,"ondisk_log_size":28723,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":6,"num_per_pool_osds":6,"num_per_pool_omap_osds":6,"kb":566231040,"kb_used":456004,"kb_used_data":18752,"kb_used_omap":9,"kb_used_meta":436854,"kb_avail":565775036,"statfs":{"total":579820584960,"available":579353636864,"internally_reserved":0,"allocated":19202048,"data_stored":10594348,"data_compressed":65166,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":9563,"internal_metadata":447339173},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-27728395,"num_objects":-61,"num_object_clones":0,"num_object_copies":-122,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-61,"num_whiteouts":0,"num_read":661,"num_read_kb":571,"num_write":168,"num_write_kb":42,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.016974"},"pg_stats":[{"pgid":"3.1f","version":"262'855","reported_seq":8379,"reported_epoch":262,"state":"active+clean","last_fresh":"2024-09-14T06:33:07.706447+0000","last_change":"2024-09-14T06:33:07.706447+0000","last_active":"2024-09-14T06:33:07.706447+0000","last_peered":"2024-09-14T06:33:07.706447+0000","last_clean":"2024-09-14T06:33:07.706447+0000","last_became_active":"2024-09-14T06:29:00.579883+0000","last_became_peered":"2024-09-14T06:29:00.579883+0000","last_unstale":"2024-09-14T06:33:07.706447+0000","last_undegraded":"2024-09-14T06:33:07.706447+0000","last_fullsized":"2024-09-14T06:33:07.706447+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":855,"log_dups_size":0,"ondisk_log_size":855,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T14:40:25.634308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001666641,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":8468,"num_read_kb":86898,"num_write":1171,"num_write_kb":49543,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"3","length":"1"},{"start":"5","length":"1"},{"start":"7","length":"1"},{"start":"9","length":"1"},{"start":"b","length":"1"},{"start":"d","length":"1"}]},{"pgid":"3.1e","version":"262'783","reported_seq":10485,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-09-14T06:33:09.715079+0000","last_change":"2024-09-14T06:33:07.770279+0000","last_active":"2024-09-14T06:33:09.715079+0000","last_peered":"2024-09-14T06:33:09.715079+0000","last_clean":"2024-09-14T06:33:09.715079+0000","last_became_active":"2024-09-14T06:33:04.626456+0000","last_became_peered":"2024-09-14T06:33:04.626456+0000","last_unstale":"2024-09-14T06:33:09.715079+0000","last_undegraded":"2024-09-14T06:33:09.715079+0000","last_fullsized":"2024-09-14T06:33:09.715079+0000","mapping_epoch":258,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":259,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":783,"log_dups_size":0,"ondisk_log_size":783,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T16:31:58.414465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013639839999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":8315,"num_read_kb":67657,"num_write":1025,"num_write_kb":40407,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":869,"num_bytes_recovered":87221212,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"3","length":"1"},{"start":"5","length":"1"},{"start":"7","length":"1"},{"start":"9","length":"1"},{"start":"b","length":"1"},{"start":"d","length":"1"}]},{"pgid":"3.1d","version":"260'1077","reported_seq":7488,"reported_epoch":262,"state":"active+clean","last_fresh":"2024-09-14T06:33:07.704297+0000","last_change":"2024-09-14T06:33:07.704498+0000","last_active":"2024-09-14T06:33:07.704297+0000","last_peered":"2024-09-14T06:33:07.704297+0000","last_clean":"2024-09-14T06:33:07.704297+0000","last_became_active":"2024-09-14T06:32:24.716363+0000","last_became_peered":"2024-09-14T06:32:24.716363+0000","last_unstale":"2024-09-14T06:33:07.704297+0000","last_undegraded":"2024-09-14T06:33:07.704297+0000","last_fullsized":"2024-09-14T06:33:07.704297+0000","mapping_epoch":218,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":219,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":1077,"log_dups_size":0,"ondisk_log_size":1077,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T13:04:18.531851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00055792899999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":7873,"num_read_kb":64375,"num_write":1555,"num_write_kb":53292,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"3","length":"1"},{"start":"5","length":"1"},{"start":"7","length":"1"},{"start":"9","length":"1"},{"start":"b","length":"1"},{"start":"d","length":"1"}]},{"pgid":"3.1c","version":"262'805","reported_seq":6023,"reported_epoch":263,"state":"active+clean","last_fresh":"2024-09-14T06:33:08.705608+0000","last_change":"2024-09-14T06:33:07.704704+0000","last_active":"2024-09-14T06:33:08.705608+0000","last_peered":"2024-09-14T06:33:08.705608+0000","last_clean":"2024-09-14T06:33:08.705608+0000","last_became_active":"2024-09-14T06:32:39.775788+0000","last_became_peered":"2024-09-14T06:32:39.775788+0000","last_unstale":"2024-09-14T06:33:08.705608+0000","last_undegraded":"2024-09-14T06:33:08.705608+0000","last_fullsized":"2024-09-14T06:33:08.705608+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":805,"log_dups_size":0,"ondisk_log_size":805,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T13:58:06.384672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00068758600000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":4956,"num_read_kb":43812,"num_write":946,"num_write_kb":45551,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":505,"num_bytes_recovered":44145197,"num_keys_recovered":280,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"3","length":"1"},{"start":"5","length":"1"},{"start":"7","length":"1"},{"start":"9","length":"1"},{"start":"b","length":"1"},{"start":"d","length":"1"}]},{"pgid":"3.1b","version":"259'960","reported_seq":11470,"reported_epoch":262,"state":"active+clean","last_fresh":"2024-09-14T06:33:07.706551+0000","last_change":"2024-09-14T06:33:07.706745+0000","last_active":"2024-09-14T06:33:07.706551+0000","last_peered":"2024-09-14T06:33:07.706551+0000","last_clean":"2024-09-14T06:33:07.706551+0000","last_became_active":"2024-09-14T06:29:00.586183+0000","last_became_peered":"2024-09-14T06:29:00.586183+0000","last_unstale":"2024-09-14T06:33:07.706551+0000","last_undegraded":"2024-09-14T06:33:07.706551+0000","last_fullsized":"2024-09-14T06:33:07.706551+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":960,"log_dups_size":0,"ondisk_log_size":960,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T10:55:09.317291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001810606,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":11910,"num_read_kb":108094,"num_write":1463,"num_write_kb":55102,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"3","length":"1"},{"start":"5","length":"1"},{"start":"7","length":"1"},{"start":"9","length":"1"},{"start":"b","length":"1"},{"start":"d","length":"1"}]},{"pgid":"3.1a","version":"260'1678","reported_seq":10585,"reported_epoch":262,"state":"active+clean","last_fresh":"2024-09-14T06:33:07.738791+0000","last_change":"2024-09-14T06:33:07.705134+0000","last_active":"2024-09-14T06:33:07.738791+0000","last_peered":"2024-09-14T06:33:07.738791+0000","last_clean":"2024-09-14T06:33:07.738791+0000","last_became_active":"2024-09-14T06:29:00.576206+0000","last_became_peered":"2024-09-14T06:29:00.576206+0000","last_unstale":"2024-09-14T06:33:07.738791+0000","last_undegraded":"2024-09-14T06:33:07.738791+0000","last_fullsized":"2024-09-14T06:33:07.738791+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":1678,"log_dups_size":0,"ondisk_log_size":1678,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T10:47:38.190388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088286099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":13339,"num_read_kb":82904,"num_write":2451,"num_write_kb":55687,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"3","length":"1"},{"start":"5","length":"1"},{"start":"7","length":"1"},{"start":"9","length":"1"},{"start":"b","length":"1"},{"start":"d","length":"1"}]},{"pgid":"3.19","version":"261'561","reported_seq":6457,"reported_epoch":263,"state":"active+clean","last_fresh":"2024-09-14T06:33:08.705471+0000","last_change":"2024-09-14T06:33:07.704594+0000","last_active":"2024-09-14T06:33:08.705471+0000","last_peered":"2024-09-14T06:33:08.705471+0000","last_clean":"2024-09-14T06:33:08.705471+0000","last_became_active":"2024-09-14T06:31:53.744800+0000","last_became_peered":"2024-09-14T06:31:53.744800+0000","last_unstale":"2024-09-14T06:33:08.705471+0000","last_undegraded":"2024-09-14T06:33:08.705471+0000","last_fullsized":"2024-09-14T06:33:08.705471+0000","mapping_epoch":188,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":189,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":561,"log_dups_size":0,"ondisk_log_size":561,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T13:59:24.308993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062201200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":5123,"num_read_kb":67781,"num_write":764,"num_write_kb":43873,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":282,"num_bytes_recovered":41927489,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"3","length":"1"},{"start":"5","length":"1"},{"start":"7","length":"1"},{"start":"9","length":"1"},{"start":"b","length":"1"},{"start":"d","length":"1"}]},{"pgid":"3.18","version":"262'547","reported_seq":6241,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-09-14T06:33:09.714660+0000","last_change":"2024-09-14T06:33:07.769587+0000","last_active":"2024-09-14T06:33:09.714660+0000","last_peered":"2024-09-14T06:33:09.714660+0000","last_clean":"2024-09-14T06:33:09.714660+0000","last_became_active":"2024-09-14T06:31:53.745159+0000","last_became_peered":"2024-09-14T06:31:53.745159+0000","last_unstale":"2024-09-14T06:33:09.714660+0000","last_undegraded":"2024-09-14T06:33:09.714660+0000","last_fullsized":"2024-09-14T06:33:09.714660+0000","mapping_epoch":188,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":189,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":547,"log_dups_size":0,"ondisk_log_size":547,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T13:08:27.610323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00079631400000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4489,"num_read_kb":78917,"num_write":672,"num_write_kb":51581,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":310,"num_bytes_recovered":56082976,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"3","length":"1"},{"start":"5","length":"1"},{"start":"7","length":"1"},{"start":"9","length":"1"},{"start":"b","length":"1"},{"start":"d","length":"1"}]},{"pgid":"3.17","version":"261'1037","reported_seq":9677,"reported_epoch":262,"state":"active+clean","last_fresh":"2024-09-14T06:33:07.706630+0000","last_change":"2024-09-14T06:33:07.706863+0000","last_active":"2024-09-14T06:33:07.706630+0000","last_peered":"2024-09-14T06:33:07.706630+0000","last_clean":"2024-09-14T06:33:07.706630+0000","last_became_active":"2024-09-14T06:29:00.579714+0000","last_became_peered":"2024-09-14T06:29:00.579714+0000","last_unstale":"2024-09-14T06:33:07.706630+0000","last_undegraded":"2024-09-14T06:33:07.706630+0000","last_fullsized":"2024-09-14T06:33:07.706630+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":1037,"log_dups_size":0,"ondisk_log_size":1037,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T15:17:51.610057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0017747889999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":10688,"num_read_kb":68964,"num_write":1162,"num_write_kb":48482,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"3","length":"1"},{"start":"5","length":"1"},{"start":"7","length":"1"},{"start":"9","length":"1"},{"start":"b","length":"1"},{"start":"d","length":"1"}]},{"pgid":"3.16","version":"255'1156","reported_seq":4158,"reported_epoch":262,"state":"active+clean","last_fresh":"2024-09-14T06:33:07.769586+0000","last_change":"2024-09-14T06:33:07.769751+0000","last_active":"2024-09-14T06:33:07.769586+0000","last_peered":"2024-09-14T06:33:07.769586+0000","last_clean":"2024-09-14T06:33:07.769586+0000","last_became_active":"2024-09-14T06:32:24.718745+0000","last_became_peered":"2024-09-14T06:32:24.718745+0000","last_unstale":"2024-09-14T06:33:07.769586+0000","last_undegraded":"2024-09-14T06:33:07.769586+0000","last_fullsized":"2024-09-14T06:33:07.769586+0000","mapping_epoch":218,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":219,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":1156,"log_dups_size":0,"ondisk_log_size":1156,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T17:51:23.869206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059450099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":5658,"num_read_kb":38562,"num_write":1743,"num_write_kb":37248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"3","length":"1"},{"start":"5","length":"1"},{"start":"7","length":"1"},{"start":"9","length":"1"},{"start":"b","length":"1"},{"start":"d","length":"1"}]},{"pgid":"3.15","version":"261'650","reported_seq":6699,"reported_epoch":262,"state":"active+clean","last_fresh":"2024-09-14T06:33:07.769450+0000","last_change":"2024-09-14T06:33:07.769581+0000","last_active":"2024-09-14T06:33:07.769450+0000","last_peered":"2024-09-14T06:33:07.769450+0000","last_clean":"2024-09-14T06:33:07.769450+0000","last_became_active":"2024-09-14T06:31:53.746091+0000","last_became_peered":"2024-09-14T06:31:53.746091+0000","last_unstale":"2024-09-14T06:33:07.769450+0000","last_undegraded":"2024-09-14T06:33:07.769450+0000","last_fullsized":"2024-09-14T06:33:07.769450+0000","mapping_epoch":188,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":189,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":650,"log_dups_size":0,"ondisk_log_size":650,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T12:36:31.690559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000442557,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":5101,"num_read_kb":74899,"num_write":696,"num_write_kb":51196,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":271,"num_bytes_recovered":49981473,"num_keys_recovered":33,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"3","length":"1"},{"start":"5","length":"1"},{"start":"7","length":"1"},{"start":"9","length":"1"},{"start":"b","length":"1"},{"start":"d","length":"1"}]},{"pgid":"3.14","version":"261'1137","reported_seq":6288,"reported_epoch":262,"state":"active+clean","last_fresh":"2024-09-14T06:33:07.704499+0000","last_change":"2024-09-14T06:33:07.704760+0000","last_active":"2024-09-14T06:33:07.704499+0000","last_peered":"2024-09-14T06:33:07.704499+0000","last_clean":"2024-09-14T06:33:07.704499+0000","last_became_active":"2024-09-14T06:33:04.628663+0000","last_became_peered":"2024-09-14T06:33:04.628663+0000","last_unstale":"2024-09-14T06:33:07.704499+0000","last_undegraded":"2024-09-14T06:33:07.704499+0000","last_fullsized":"2024-09-14T06:33:07.704499+0000","mapping_epoch":258,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":259,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":1137,"log_dups_size":0,"ondisk_log_size":1137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T11:42:47.162926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00078945399999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":8948,"num_read_kb":46936,"num_write":1573,"num_write_kb":29829,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":296,"num_bytes_recovered":6118769,"num_keys_recovered":11,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"3","length":"1"},{"start":"5","length":"1"},{"start":"7","length":"1"},{"start":"9","length":"1"},{"start":"b","length":"1"},{"start":"d","length":"1"}]},{"pgid":"2.7","version":"0'0","reported_seq":148,"reported_epoch":262,"state":"active+clean","last_fresh":"2024-09-14T06:33:07.769239+0000","last_change":"2024-09-14T06:33:01.625180+0000","last_active":"2024-09-14T06:33:07.769239+0000","last_peered":"2024-09-14T06:33:07.769239+0000","last_clean":"2024-09-14T06:33:07.769239+0000","last_became_active":"2024-09-14T06:33:01.624918+0000","last_became_peered":"2024-09-14T06:33:01.624918+0000","last_unstale":"2024-09-14T06:33:07.769239+0000","last_undegraded":"2024-09-14T06:33:07.769239+0000","last_fullsized":"2024-09-14T06:33:07.769239+0000","mapping_epoch":255,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":256,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:28:50.205350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:28:50.205350+0000","last_clean_scrub_stamp":"2024-09-14T06:28:50.205350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T10:19:32.184994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.6","version":"262'893","reported_seq":8674,"reported_epoch":262,"state":"active+clean","last_fresh":"2024-09-14T06:33:07.705814+0000","last_change":"2024-09-14T06:33:07.705814+0000","last_active":"2024-09-14T06:33:07.705814+0000","last_peered":"2024-09-14T06:33:07.705814+0000","last_clean":"2024-09-14T06:33:07.705814+0000","last_became_active":"2024-09-14T06:29:00.578283+0000","last_became_peered":"2024-09-14T06:29:00.578283+0000","last_unstale":"2024-09-14T06:33:07.705814+0000","last_undegraded":"2024-09-14T06:33:07.705814+0000","last_fullsized":"2024-09-14T06:33:07.705814+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":893,"log_dups_size":0,"ondisk_log_size":893,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T15:09:37.855181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001076554,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":8608,"num_read_kb":78770,"num_write":923,"num_write_kb":48303,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"3","length":"1"},{"start":"5","length":"1"},{"start":"7","length":"1"},{"start":"9","length":"1"},{"start":"b","length":"1"},{"start":"d","length":"1"}]},{"pgid":"2.6","version":"0'0","reported_seq":509,"reported_epoch":263,"state":"active+clean","last_fresh":"2024-09-14T06:33:08.705755+0000","last_change":"2024-09-14T06:32:01.608456+0000","last_active":"2024-09-14T06:33:08.705755+0000","last_peered":"2024-09-14T06:33:08.705755+0000","last_clean":"2024-09-14T06:33:08.705755+0000","last_became_active":"2024-09-14T06:32:01.608178+0000","last_became_peered":"2024-09-14T06:32:01.608178+0000","last_unstale":"2024-09-14T06:33:08.705755+0000","last_undegraded":"2024-09-14T06:33:08.705755+0000","last_fullsized":"2024-09-14T06:33:08.705755+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:28:50.205350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:28:50.205350+0000","last_clean_scrub_stamp":"2024-09-14T06:28:50.205350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T10:16:40.480031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065018200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"261'840","reported_seq":4605,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-09-14T06:33:09.714940+0000","last_change":"2024-09-14T06:33:07.769882+0000","last_active":"2024-09-14T06:33:09.714940+0000","last_peered":"2024-09-14T06:33:09.714940+0000","last_clean":"2024-09-14T06:33:09.714940+0000","last_became_active":"2024-09-14T06:31:54.747923+0000","last_became_peered":"2024-09-14T06:31:54.747923+0000","last_unstale":"2024-09-14T06:33:09.714940+0000","last_undegraded":"2024-09-14T06:33:09.714940+0000","last_fullsized":"2024-09-14T06:33:09.714940+0000","mapping_epoch":189,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":190,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":840,"log_dups_size":0,"ondisk_log_size":840,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T09:01:49.581187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000927559,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4591,"num_read_kb":41129,"num_write":1178,"num_write_kb":36122,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":183,"num_bytes_recovered":13912513,"num_keys_recovered":22,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"3","length":"1"},{"start":"5","length":"1"},{"start":"7","length":"1"},{"start":"9","length":"1"},{"start":"b","length":"1"},{"start":"d","length":"1"}]},{"pgid":"2.5","version":"0'0","reported_seq":502,"reported_epoch":262,"state":"active+clean","last_fresh":"2024-09-14T06:33:07.769389+0000","last_change":"2024-09-14T06:28:53.640035+0000","last_active":"2024-09-14T06:33:07.769389+0000","last_peered":"2024-09-14T06:33:07.769389+0000","last_clean":"2024-09-14T06:33:07.769389+0000","last_became_active":"2024-09-14T06:28:51.308058+0000","last_became_peered":"2024-09-14T06:28:51.308058+0000","last_unstale":"2024-09-14T06:33:07.769389+0000","last_undegraded":"2024-09-14T06:33:07.769389+0000","last_fullsized":"2024-09-14T06:33:07.769389+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:28:50.205350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:28:50.205350+0000","last_clean_scrub_stamp":"2024-09-14T06:28:50.205350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T13:35:48.544057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00037824000000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"262'700","reported_seq":4458,"reported_epoch":263,"state":"active+clean","last_fresh":"2024-09-14T06:33:08.705657+0000","last_change":"2024-09-14T06:33:07.704937+0000","last_active":"2024-09-14T06:33:08.705657+0000","last_peered":"2024-09-14T06:33:08.705657+0000","last_clean":"2024-09-14T06:33:08.705657+0000","last_became_active":"2024-09-14T06:33:04.628919+0000","last_became_peered":"2024-09-14T06:33:04.628919+0000","last_unstale":"2024-09-14T06:33:08.705657+0000","last_undegraded":"2024-09-14T06:33:08.705657+0000","last_fullsized":"2024-09-14T06:33:08.705657+0000","mapping_epoch":258,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":259,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":700,"log_dups_size":0,"ondisk_log_size":700,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T09:33:23.339333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00074004799999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":2654,"num_read_kb":37435,"num_write":897,"num_write_kb":40860,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":509,"num_bytes_recovered":31411166,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"3","length":"1"},{"start":"5","length":"1"},{"start":"7","length":"1"},{"start":"9","length":"1"},{"start":"b","length":"1"},{"start":"d","length":"1"}]},{"pgid":"2.4","version":"0'0","reported_seq":504,"reported_epoch":263,"state":"active+clean","last_fresh":"2024-09-14T06:33:08.705759+0000","last_change":"2024-09-14T06:28:53.226072+0000","last_active":"2024-09-14T06:33:08.705759+0000","last_peered":"2024-09-14T06:33:08.705759+0000","last_clean":"2024-09-14T06:33:08.705759+0000","last_became_active":"2024-09-14T06:28:51.308072+0000","last_became_peered":"2024-09-14T06:28:51.308072+0000","last_unstale":"2024-09-14T06:33:08.705759+0000","last_undegraded":"2024-09-14T06:33:08.705759+0000","last_fullsized":"2024-09-14T06:33:08.705759+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:28:50.205350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:28:50.205350+0000","last_clean_scrub_stamp":"2024-09-14T06:28:50.205350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T13:15:09.052328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066197399999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"261'896","reported_seq":5621,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-09-14T06:33:09.715085+0000","last_change":"2024-09-14T06:33:07.770566+0000","last_active":"2024-09-14T06:33:09.715085+0000","last_peered":"2024-09-14T06:33:09.715085+0000","last_clean":"2024-09-14T06:33:09.715085+0000","last_became_active":"2024-09-14T06:32:24.716458+0000","last_became_peered":"2024-09-14T06:32:24.716458+0000","last_unstale":"2024-09-14T06:33:09.715085+0000","last_undegraded":"2024-09-14T06:33:09.715085+0000","last_fullsized":"2024-09-14T06:33:09.715085+0000","mapping_epoch":218,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":219,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":896,"log_dups_size":0,"ondisk_log_size":896,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T12:28:03.015849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013146799999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4516,"num_read_kb":42632,"num_write":1255,"num_write_kb":49494,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":330,"num_bytes_recovered":28798893,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"3","length":"1"},{"start":"5","length":"1"},{"start":"7","length":"1"},{"start":"9","length":"1"},{"start":"b","length":"1"},{"start":"d","length":"1"}]},{"pgid":"3.3","version":"262'711","reported_seq":10870,"reported_epoch":262,"state":"active+clean","last_fresh":"2024-09-14T06:33:07.790215+0000","last_change":"2024-09-14T06:33:07.704808+0000","last_active":"2024-09-14T06:33:07.790215+0000","last_peered":"2024-09-14T06:33:07.790215+0000","last_clean":"2024-09-14T06:33:07.790215+0000","last_became_active":"2024-09-14T06:29:00.584678+0000","last_became_peered":"2024-09-14T06:29:00.584678+0000","last_unstale":"2024-09-14T06:33:07.790215+0000","last_undegraded":"2024-09-14T06:33:07.790215+0000","last_fullsized":"2024-09-14T06:33:07.790215+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":711,"log_dups_size":0,"ondisk_log_size":711,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T07:39:08.849874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00074987800000000005,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":9560,"num_read_kb":89311,"num_write":777,"num_write_kb":51691,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"3","length":"1"},{"start":"5","length":"1"},{"start":"7","length":"1"},{"start":"9","length":"1"},{"start":"b","length":"1"},{"start":"d","length":"1"}]},{"pgid":"2.2","version":"22'2","reported_seq":118,"reported_epoch":263,"state":"active+clean","last_fresh":"2024-09-14T06:33:08.705546+0000","last_change":"2024-09-14T06:32:24.716937+0000","last_active":"2024-09-14T06:33:08.705546+0000","last_peered":"2024-09-14T06:33:08.705546+0000","last_clean":"2024-09-14T06:33:08.705546+0000","last_became_active":"2024-09-14T06:32:24.716539+0000","last_became_peered":"2024-09-14T06:32:24.716539+0000","last_unstale":"2024-09-14T06:33:08.705546+0000","last_undegraded":"2024-09-14T06:33:08.705546+0000","last_fullsized":"2024-09-14T06:33:08.705546+0000","mapping_epoch":218,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":219,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:28:50.205350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:28:50.205350+0000","last_clean_scrub_stamp":"2024-09-14T06:28:50.205350+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T07:16:29.639469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"260'1116","reported_seq":7457,"reported_epoch":263,"state":"active+clean","last_fresh":"2024-09-14T06:33:08.705427+0000","last_change":"2024-09-14T06:33:07.705043+0000","last_active":"2024-09-14T06:33:08.705427+0000","last_peered":"2024-09-14T06:33:08.705427+0000","last_clean":"2024-09-14T06:33:08.705427+0000","last_became_active":"2024-09-14T06:33:04.626601+0000","last_became_peered":"2024-09-14T06:33:04.626601+0000","last_unstale":"2024-09-14T06:33:08.705427+0000","last_undegraded":"2024-09-14T06:33:08.705427+0000","last_fullsized":"2024-09-14T06:33:08.705427+0000","mapping_epoch":258,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":259,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":1116,"log_dups_size":0,"ondisk_log_size":1116,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T07:23:09.155401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064233900000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":8259,"num_read_kb":85480,"num_write":1652,"num_write_kb":58700,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":179,"num_bytes_recovered":2852,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"3","length":"1"},{"start":"5","length":"1"},{"start":"7","length":"1"},{"start":"9","length":"1"},{"start":"b","length":"1"},{"start":"d","length":"1"}]},{"pgid":"2.1","version":"0'0","reported_seq":78,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-09-14T06:33:09.714596+0000","last_change":"2024-09-14T06:33:01.625894+0000","last_active":"2024-09-14T06:33:09.714596+0000","last_peered":"2024-09-14T06:33:09.714596+0000","last_clean":"2024-09-14T06:33:09.714596+0000","last_became_active":"2024-09-14T06:33:01.625561+0000","last_became_peered":"2024-09-14T06:33:01.625561+0000","last_unstale":"2024-09-14T06:33:09.714596+0000","last_undegraded":"2024-09-14T06:33:09.714596+0000","last_fullsized":"2024-09-14T06:33:09.714596+0000","mapping_epoch":255,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":256,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:28:50.205350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:28:50.205350+0000","last_clean_scrub_stamp":"2024-09-14T06:28:50.205350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T14:21:03.308190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053403100000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1","version":"261'822","reported_seq":11835,"reported_epoch":262,"state":"active+clean","last_fresh":"2024-09-14T06:33:07.705164+0000","last_change":"2024-09-14T06:33:07.705685+0000","last_active":"2024-09-14T06:33:07.705164+0000","last_peered":"2024-09-14T06:33:07.705164+0000","last_clean":"2024-09-14T06:33:07.705164+0000","last_became_active":"2024-09-14T06:30:57.768238+0000","last_became_peered":"2024-09-14T06:30:57.768238+0000","last_unstale":"2024-09-14T06:33:07.705164+0000","last_undegraded":"2024-09-14T06:33:07.705164+0000","last_fullsized":"2024-09-14T06:33:07.705164+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":822,"log_dups_size":0,"ondisk_log_size":822,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T15:07:18.804416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011933110000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":11284,"num_read_kb":112430,"num_write":991,"num_write_kb":47408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":183,"num_bytes_recovered":30431994,"num_keys_recovered":22,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"3","length":"1"},{"start":"5","length":"1"},{"start":"7","length":"1"},{"start":"9","length":"1"},{"start":"b","length":"1"},{"start":"d","length":"1"}]},{"pgid":"2.0","version":"0'0","reported_seq":502,"reported_epoch":262,"state":"active+clean","last_fresh":"2024-09-14T06:33:07.768921+0000","last_change":"2024-09-14T06:28:53.640096+0000","last_active":"2024-09-14T06:33:07.768921+0000","last_peered":"2024-09-14T06:33:07.768921+0000","last_clean":"2024-09-14T06:33:07.768921+0000","last_became_active":"2024-09-14T06:28:51.223069+0000","last_became_peered":"2024-09-14T06:28:51.223069+0000","last_unstale":"2024-09-14T06:33:07.768921+0000","last_undegraded":"2024-09-14T06:33:07.768921+0000","last_fullsized":"2024-09-14T06:33:07.768921+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:28:50.205350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:28:50.205350+0000","last_clean_scrub_stamp":"2024-09-14T06:28:50.205350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T15:13:00.471509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00043714600000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"262'1663","reported_seq":7806,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-09-14T06:33:09.714882+0000","last_change":"2024-09-14T06:33:07.770369+0000","last_active":"2024-09-14T06:33:09.714882+0000","last_peered":"2024-09-14T06:33:09.714882+0000","last_clean":"2024-09-14T06:33:09.714882+0000","last_became_active":"2024-09-14T06:31:54.744160+0000","last_became_peered":"2024-09-14T06:31:54.744160+0000","last_unstale":"2024-09-14T06:33:09.714882+0000","last_undegraded":"2024-09-14T06:33:09.714882+0000","last_fullsized":"2024-09-14T06:33:09.714882+0000","mapping_epoch":189,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":190,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":1663,"log_dups_size":0,"ondisk_log_size":1663,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T16:44:35.569843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001382608,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":10723,"num_read_kb":70589,"num_write":2596,"num_write_kb":53564,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":327,"num_bytes_recovered":50047063,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"3","length":"1"},{"start":"5","length":"1"},{"start":"7","length":"1"},{"start":"9","length":"1"},{"start":"b","length":"1"},{"start":"d","length":"1"}]},{"pgid":"2.3","version":"20'1","reported_seq":27,"reported_epoch":262,"state":"active+clean","last_fresh":"2024-09-14T06:33:07.769008+0000","last_change":"2024-09-14T06:33:01.625025+0000","last_active":"2024-09-14T06:33:07.769008+0000","last_peered":"2024-09-14T06:33:07.769008+0000","last_clean":"2024-09-14T06:33:07.769008+0000","last_became_active":"2024-09-14T06:33:01.622409+0000","last_became_peered":"2024-09-14T06:33:01.622409+0000","last_unstale":"2024-09-14T06:33:07.769008+0000","last_undegraded":"2024-09-14T06:33:07.769008+0000","last_fullsized":"2024-09-14T06:33:07.769008+0000","mapping_epoch":255,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":256,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:28:50.205350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:28:50.205350+0000","last_clean_scrub_stamp":"2024-09-14T06:28:50.205350+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T16:23:56.048067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"17'72","reported_seq":593,"reported_epoch":262,"state":"active+clean","last_fresh":"2024-09-14T06:33:07.768933+0000","last_change":"2024-09-14T06:28:44.839885+0000","last_active":"2024-09-14T06:33:07.768933+0000","last_peered":"2024-09-14T06:33:07.768933+0000","last_clean":"2024-09-14T06:33:07.768933+0000","last_became_active":"2024-09-14T06:28:44.839573+0000","last_became_peered":"2024-09-14T06:28:44.839573+0000","last_unstale":"2024-09-14T06:33:07.768933+0000","last_undegraded":"2024-09-14T06:33:07.768933+0000","last_fullsized":"2024-09-14T06:33:07.768933+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:28:43.828494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:28:43.828494+0000","last_clean_scrub_stamp":"2024-09-14T06:28:43.828494+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T14:41:25.787293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.8","version":"261'684","reported_seq":6613,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-09-14T06:33:09.714692+0000","last_change":"2024-09-14T06:33:07.769379+0000","last_active":"2024-09-14T06:33:09.714692+0000","last_peered":"2024-09-14T06:33:09.714692+0000","last_clean":"2024-09-14T06:33:09.714692+0000","last_became_active":"2024-09-14T06:31:55.754694+0000","last_became_peered":"2024-09-14T06:31:55.754694+0000","last_unstale":"2024-09-14T06:33:09.714692+0000","last_undegraded":"2024-09-14T06:33:09.714692+0000","last_fullsized":"2024-09-14T06:33:09.714692+0000","mapping_epoch":190,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":191,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":684,"log_dups_size":0,"ondisk_log_size":684,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T15:48:47.633183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067851599999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":5793,"num_read_kb":90852,"num_write":921,"num_write_kb":57691,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":319,"num_bytes_recovered":60352269,"num_keys_recovered":22,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"3","length":"1"},{"start":"5","length":"1"},{"start":"7","length":"1"},{"start":"9","length":"1"},{"start":"b","length":"1"},{"start":"d","length":"1"}]},{"pgid":"3.9","version":"261'597","reported_seq":4793,"reported_epoch":262,"state":"active+clean","last_fresh":"2024-09-14T06:33:07.704234+0000","last_change":"2024-09-14T06:33:07.704439+0000","last_active":"2024-09-14T06:33:07.704234+0000","last_peered":"2024-09-14T06:33:07.704234+0000","last_clean":"2024-09-14T06:33:07.704234+0000","last_became_active":"2024-09-14T06:33:04.628736+0000","last_became_peered":"2024-09-14T06:33:04.628736+0000","last_unstale":"2024-09-14T06:33:07.704234+0000","last_undegraded":"2024-09-14T06:33:07.704234+0000","last_fullsized":"2024-09-14T06:33:07.704234+0000","mapping_epoch":258,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":259,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":597,"log_dups_size":0,"ondisk_log_size":597,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T10:39:24.683086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057139500000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3678,"num_read_kb":64291,"num_write":778,"num_write_kb":44928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":158,"num_bytes_recovered":49117,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"3","length":"1"},{"start":"5","length":"1"},{"start":"7","length":"1"},{"start":"9","length":"1"},{"start":"b","length":"1"},{"start":"d","length":"1"}]},{"pgid":"3.a","version":"261'874","reported_seq":5139,"reported_epoch":262,"state":"active+clean","last_fresh":"2024-09-14T06:33:07.704826+0000","last_change":"2024-09-14T06:33:07.705019+0000","last_active":"2024-09-14T06:33:07.704826+0000","last_peered":"2024-09-14T06:33:07.704826+0000","last_clean":"2024-09-14T06:33:07.704826+0000","last_became_active":"2024-09-14T06:33:04.627306+0000","last_became_peered":"2024-09-14T06:33:04.627306+0000","last_unstale":"2024-09-14T06:33:07.704826+0000","last_undegraded":"2024-09-14T06:33:07.704826+0000","last_fullsized":"2024-09-14T06:33:07.704826+0000","mapping_epoch":258,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":259,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":874,"log_dups_size":0,"ondisk_log_size":874,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T11:52:51.272927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097383799999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":5331,"num_read_kb":58329,"num_write":1221,"num_write_kb":47535,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":491,"num_bytes_recovered":26968802,"num_keys_recovered":45,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"3","length":"1"},{"start":"5","length":"1"},{"start":"7","length":"1"},{"start":"9","length":"1"},{"start":"b","length":"1"},{"start":"d","length":"1"}]},{"pgid":"3.b","version":"260'1637","reported_seq":8941,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-09-14T06:33:09.714817+0000","last_change":"2024-09-14T06:33:07.769381+0000","last_active":"2024-09-14T06:33:09.714817+0000","last_peered":"2024-09-14T06:33:09.714817+0000","last_clean":"2024-09-14T06:33:09.714817+0000","last_became_active":"2024-09-14T06:31:55.756219+0000","last_became_peered":"2024-09-14T06:31:55.756219+0000","last_unstale":"2024-09-14T06:33:09.714817+0000","last_undegraded":"2024-09-14T06:33:09.714817+0000","last_fullsized":"2024-09-14T06:33:09.714817+0000","mapping_epoch":190,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":191,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":1637,"log_dups_size":0,"ondisk_log_size":1637,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T15:53:45.757678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065814200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":13043,"num_read_kb":64129,"num_write":2496,"num_write_kb":43679,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":291,"num_bytes_recovered":37370632,"num_keys_recovered":66,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"3","length":"1"},{"start":"5","length":"1"},{"start":"7","length":"1"},{"start":"9","length":"1"},{"start":"b","length":"1"},{"start":"d","length":"1"}]},{"pgid":"3.c","version":"255'622","reported_seq":6293,"reported_epoch":264,"state":"active+clean","last_fresh":"2024-09-14T06:33:09.714912+0000","last_change":"2024-09-14T06:33:07.769705+0000","last_active":"2024-09-14T06:33:09.714912+0000","last_peered":"2024-09-14T06:33:09.714912+0000","last_clean":"2024-09-14T06:33:09.714912+0000","last_became_active":"2024-09-14T06:32:24.716421+0000","last_became_peered":"2024-09-14T06:32:24.716421+0000","last_unstale":"2024-09-14T06:33:09.714912+0000","last_undegraded":"2024-09-14T06:33:09.714912+0000","last_fullsized":"2024-09-14T06:33:09.714912+0000","mapping_epoch":218,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":219,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":622,"log_dups_size":0,"ondisk_log_size":622,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T08:41:04.782849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00083342900000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4842,"num_read_kb":50425,"num_write":795,"num_write_kb":40827,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":293,"num_bytes_recovered":36712421,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"3","length":"1"},{"start":"5","length":"1"},{"start":"7","length":"1"},{"start":"9","length":"1"},{"start":"b","length":"1"},{"start":"d","length":"1"}]},{"pgid":"3.d","version":"261'577","reported_seq":9275,"reported_epoch":262,"state":"active+clean","last_fresh":"2024-09-14T06:33:07.770026+0000","last_change":"2024-09-14T06:33:07.770219+0000","last_active":"2024-09-14T06:33:07.770026+0000","last_peered":"2024-09-14T06:33:07.770026+0000","last_clean":"2024-09-14T06:33:07.770026+0000","last_became_active":"2024-09-14T06:29:00.580362+0000","last_became_peered":"2024-09-14T06:29:00.580362+0000","last_unstale":"2024-09-14T06:33:07.770026+0000","last_undegraded":"2024-09-14T06:33:07.770026+0000","last_fullsized":"2024-09-14T06:33:07.770026+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":577,"log_dups_size":0,"ondisk_log_size":577,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T16:36:38.133044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010003270000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":8199,"num_read_kb":93694,"num_write":732,"num_write_kb":41032,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"3","length":"1"},{"start":"5","length":"1"},{"start":"7","length":"1"},{"start":"9","length":"1"},{"start":"b","length":"1"},{"start":"d","length":"1"}]},{"pgid":"3.e","version":"262'586","reported_seq":5473,"reported_epoch":262,"state":"active+clean","last_fresh":"2024-09-14T06:33:07.769825+0000","last_change":"2024-09-14T06:33:07.769825+0000","last_active":"2024-09-14T06:33:07.769825+0000","last_peered":"2024-09-14T06:33:07.769825+0000","last_clean":"2024-09-14T06:33:07.769825+0000","last_became_active":"2024-09-14T06:32:39.777524+0000","last_became_peered":"2024-09-14T06:32:39.777524+0000","last_unstale":"2024-09-14T06:33:07.769825+0000","last_undegraded":"2024-09-14T06:33:07.769825+0000","last_fullsized":"2024-09-14T06:33:07.769825+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":586,"log_dups_size":0,"ondisk_log_size":586,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T07:31:19.417437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065560200000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3986,"num_read_kb":61138,"num_write":753,"num_write_kb":43787,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":171,"num_bytes_recovered":10913227,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"3","length":"1"},{"start":"5","length":"1"},{"start":"7","length":"1"},{"start":"9","length":"1"},{"start":"b","length":"1"},{"start":"d","length":"1"}]},{"pgid":"3.f","version":"262'712","reported_seq":5895,"reported_epoch":262,"state":"active+clean","last_fresh":"2024-09-14T06:33:07.776689+0000","last_change":"2024-09-14T06:33:07.770138+0000","last_active":"2024-09-14T06:33:07.776689+0000","last_peered":"2024-09-14T06:33:07.776689+0000","last_clean":"2024-09-14T06:33:07.776689+0000","last_became_active":"2024-09-14T06:29:00.586491+0000","last_became_peered":"2024-09-14T06:29:00.586491+0000","last_unstale":"2024-09-14T06:33:07.776689+0000","last_undegraded":"2024-09-14T06:33:07.776689+0000","last_fullsized":"2024-09-14T06:33:07.776689+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":712,"log_dups_size":0,"ondisk_log_size":712,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T14:20:12.511497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000891002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":6081,"num_read_kb":45578,"num_write":836,"num_write_kb":36134,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"3","length":"1"},{"start":"5","length":"1"},{"start":"7","length":"1"},{"start":"9","length":"1"},{"start":"b","length":"1"},{"start":"d","length":"1"}]},{"pgid":"3.10","version":"261'952","reported_seq":6834,"reported_epoch":262,"state":"active+clean","last_fresh":"2024-09-14T06:33:07.706830+0000","last_change":"2024-09-14T06:33:07.707031+0000","last_active":"2024-09-14T06:33:07.706830+0000","last_peered":"2024-09-14T06:33:07.706830+0000","last_clean":"2024-09-14T06:33:07.706830+0000","last_became_active":"2024-09-14T06:32:04.621617+0000","last_became_peered":"2024-09-14T06:32:04.621617+0000","last_unstale":"2024-09-14T06:33:07.706830+0000","last_undegraded":"2024-09-14T06:33:07.706830+0000","last_fullsized":"2024-09-14T06:33:07.706830+0000","mapping_epoch":199,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":200,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":952,"log_dups_size":0,"ondisk_log_size":952,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T06:39:54.136142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012025289999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":7147,"num_read_kb":34474,"num_write":1082,"num_write_kb":36655,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":154,"num_bytes_recovered":9401812,"num_keys_recovered":45,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"3","length":"1"},{"start":"5","length":"1"},{"start":"7","length":"1"},{"start":"9","length":"1"},{"start":"b","length":"1"},{"start":"d","length":"1"}]},{"pgid":"3.11","version":"261'730","reported_seq":11694,"reported_epoch":262,"state":"active+clean","last_fresh":"2024-09-14T06:33:07.769799+0000","last_change":"2024-09-14T06:33:07.769975+0000","last_active":"2024-09-14T06:33:07.769799+0000","last_peered":"2024-09-14T06:33:07.769799+0000","last_clean":"2024-09-14T06:33:07.769799+0000","last_became_active":"2024-09-14T06:32:39.778010+0000","last_became_peered":"2024-09-14T06:32:39.778010+0000","last_unstale":"2024-09-14T06:33:07.769799+0000","last_undegraded":"2024-09-14T06:33:07.769799+0000","last_fullsized":"2024-09-14T06:33:07.769799+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":730,"log_dups_size":0,"ondisk_log_size":730,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T10:48:00.264177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00072597299999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":10271,"num_read_kb":109243,"num_write":859,"num_write_kb":48307,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":177,"num_bytes_recovered":16647747,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"3","length":"1"},{"start":"5","length":"1"},{"start":"7","length":"1"},{"start":"9","length":"1"},{"start":"b","length":"1"},{"start":"d","length":"1"}]},{"pgid":"3.12","version":"261'782","reported_seq":7454,"reported_epoch":262,"state":"active+clean","last_fresh":"2024-09-14T06:33:07.706309+0000","last_change":"2024-09-14T06:33:07.706504+0000","last_active":"2024-09-14T06:33:07.706309+0000","last_peered":"2024-09-14T06:33:07.706309+0000","last_clean":"2024-09-14T06:33:07.706309+0000","last_became_active":"2024-09-14T06:29:00.579826+0000","last_became_peered":"2024-09-14T06:29:00.579826+0000","last_unstale":"2024-09-14T06:33:07.706309+0000","last_undegraded":"2024-09-14T06:33:07.706309+0000","last_fullsized":"2024-09-14T06:33:07.706309+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":782,"log_dups_size":0,"ondisk_log_size":782,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T11:32:57.816012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001680792,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":7110,"num_read_kb":59089,"num_write":947,"num_write_kb":42441,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"3","length":"1"},{"start":"5","length":"1"},{"start":"7","length":"1"},{"start":"9","length":"1"},{"start":"b","length":"1"},{"start":"d","length":"1"}]},{"pgid":"3.13","version":"261'1008","reported_seq":8907,"reported_epoch":262,"state":"active+clean","last_fresh":"2024-09-14T06:33:07.769416+0000","last_change":"2024-09-14T06:33:07.769558+0000","last_active":"2024-09-14T06:33:07.769416+0000","last_peered":"2024-09-14T06:33:07.769416+0000","last_clean":"2024-09-14T06:33:07.769416+0000","last_became_active":"2024-09-14T06:29:00.586520+0000","last_became_peered":"2024-09-14T06:29:00.586520+0000","last_unstale":"2024-09-14T06:33:07.769416+0000","last_undegraded":"2024-09-14T06:33:07.769416+0000","last_fullsized":"2024-09-14T06:33:07.769416+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":1008,"log_dups_size":0,"ondisk_log_size":1008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T07:25:04.666000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041539399999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":9551,"num_read_kb":66119,"num_write":1235,"num_write_kb":47437,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"3","length":"1"},{"start":"5","length":"1"},{"start":"7","length":"1"},{"start":"9","length":"1"},{"start":"b","length":"1"},{"start":"d","length":"1"}]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":19,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":240095,"num_read_kb":2184936,"num_write":38145,"num_write_kb":1478386,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6298,"num_bytes_recovered":638497624,"num_keys_recovered":546,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":6868992,"data_stored":66096213,"data_compressed":4010572,"data_compressed_allocated":5459968,"data_compressed_original":65020628,"omap_allocated":0,"internal_metadata":0},"log_size":28648,"ondisk_log_size":28648,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":131072,"data_stored":1180736,"data_compressed":65166,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509496,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":70616,"kb_used_data":3160,"kb_used_omap":1,"kb_used_meta":67262,"kb_avail":94301224,"statfs":{"total":96636764160,"available":96564453376,"internally_reserved":0,"allocated":3235840,"data_stored":2156460,"data_compressed":32583,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":1590,"internal_metadata":68876746},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":0,"seq":64424509496,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509497,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":74556,"kb_used_data":3100,"kb_used_omap":1,"kb_used_meta":71422,"kb_avail":94297284,"statfs":{"total":96636764160,"available":96560418816,"internally_reserved":0,"allocated":3174400,"data_stored":1566111,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":73136581},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509496,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":88436,"kb_used_data":3100,"kb_used_omap":1,"kb_used_meta":85310,"kb_avail":94283404,"statfs":{"total":96636764160,"available":96546205696,"internally_reserved":0,"allocated":3174400,"data_stored":1566111,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":87357893},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509497,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":56580,"kb_used_data":3104,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94315260,"statfs":{"total":96636764160,"available":96578826240,"internally_reserved":0,"allocated":3178496,"data_stored":1566130,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":0,"seq":60129542200,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542199,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":82932,"kb_used_data":3168,"kb_used_omap":1,"kb_used_meta":79742,"kb_avail":94288908,"statfs":{"total":96636764160,"available":96551841792,"internally_reserved":0,"allocated":3244032,"data_stored":2161370,"data_compressed":32583,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":1595,"internal_metadata":81656261},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574905,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":82884,"kb_used_data":3120,"kb_used_omap":1,"kb_used_meta":79742,"kb_avail":94288956,"statfs":{"total":96636764160,"available":96551890944,"internally_reserved":0,"allocated":3194880,"data_stored":1578166,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":81656263},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32583,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32583,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":229376,"data_stored":726925,"data_compressed":74239,"data_compressed_allocated":98304,"data_compressed_original":626688,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":6631424,"data_stored":65369250,"data_compressed":3936333,"data_compressed_allocated":5361664,"data_compressed_original":64393940,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-14T06:33:12.337 INFO:tasks.ceph.ceph_manager.ceph:recovered! 2024-09-14T06:33:12.337 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-14T06:33:12.337 INFO:tasks.thrashosds.thrasher:Adding osd 6 2024-09-14T06:33:12.337 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd in 6 2024-09-14T06:33:12.630 INFO:tasks.ceph.mon.a.smithi094.stderr:2024-09-14T06:33:12.628+0000 7f3f97784640 -1 mon.a@0(leader).osd e266 definitely_dead 0 2024-09-14T06:33:12.726 INFO:teuthology.orchestra.run.smithi094.stderr:marked in osd.6. 2024-09-14T06:33:12.741 INFO:tasks.thrashosds.thrasher:Added osd 6 2024-09-14T06:33:17.741 INFO:tasks.thrashosds.thrasher:inning osd 2024-09-14T06:33:17.742 INFO:tasks.thrashosds.thrasher:Adding osd 2 2024-09-14T06:33:17.742 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd in 2 2024-09-14T06:33:18.053 INFO:tasks.ceph.mon.a.smithi094.stderr:2024-09-14T06:33:18.052+0000 7f3f97784640 -1 mon.a@0(leader).osd e272 definitely_dead 0 2024-09-14T06:33:18.742 INFO:teuthology.orchestra.run.smithi094.stderr:marked in osd.2. 2024-09-14T06:33:18.761 INFO:tasks.thrashosds.thrasher:Added osd 2 2024-09-14T06:33:18.761 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'mon.*' injectargs --mon_osd_down_out_interval=600 2024-09-14T06:33:18.878 INFO:teuthology.orchestra.run.smithi094.stdout:mon.a: {} 2024-09-14T06:33:18.879 INFO:teuthology.orchestra.run.smithi094.stderr:mon.a: mon_osd_down_out_interval = '' (not observed, change may require restart) 2024-09-14T06:33:18.888 INFO:teuthology.orchestra.run.smithi094.stdout:mon.b: {} 2024-09-14T06:33:18.889 INFO:teuthology.orchestra.run.smithi094.stderr:mon.b: mon_osd_down_out_interval = '' (not observed, change may require restart) 2024-09-14T06:33:18.897 INFO:teuthology.orchestra.run.smithi094.stdout:mon.c: {} 2024-09-14T06:33:18.897 INFO:teuthology.orchestra.run.smithi094.stderr:mon.c: mon_osd_down_out_interval = '' (not observed, change may require restart) 2024-09-14T06:33:18.907 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 0 1 2024-09-14T06:33:19.748 INFO:teuthology.orchestra.run.smithi094.stderr:reweighted osd.0 to 1 (10000) 2024-09-14T06:33:19.763 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 0 1 2024-09-14T06:33:20.748 INFO:teuthology.orchestra.run.smithi094.stderr:set osd.0 primary-affinity to 1 (8655362) 2024-09-14T06:33:20.774 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 1 1 2024-09-14T06:33:21.757 INFO:teuthology.orchestra.run.smithi094.stderr:reweighted osd.1 to 1 (10000) 2024-09-14T06:33:21.773 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 1 1 2024-09-14T06:33:22.767 INFO:teuthology.orchestra.run.smithi094.stderr:set osd.1 primary-affinity to 1 (8655362) 2024-09-14T06:33:22.783 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 2 1 2024-09-14T06:33:23.766 INFO:teuthology.orchestra.run.smithi094.stderr:reweighted osd.2 to 1 (10000) 2024-09-14T06:33:23.783 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 2 1 2024-09-14T06:33:24.774 INFO:teuthology.orchestra.run.smithi094.stderr:set osd.2 primary-affinity to 1 (8655362) 2024-09-14T06:33:24.788 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 3 1 2024-09-14T06:33:25.776 INFO:teuthology.orchestra.run.smithi094.stderr:reweighted osd.3 to 1 (10000) 2024-09-14T06:33:25.793 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 3 1 2024-09-14T06:33:26.782 INFO:teuthology.orchestra.run.smithi094.stderr:set osd.3 primary-affinity to 1 (8655362) 2024-09-14T06:33:26.798 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 4 1 2024-09-14T06:33:27.791 INFO:teuthology.orchestra.run.smithi094.stderr:reweighted osd.4 to 1 (10000) 2024-09-14T06:33:27.808 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 4 1 2024-09-14T06:33:28.796 INFO:teuthology.orchestra.run.smithi094.stderr:set osd.4 primary-affinity to 1 (8655362) 2024-09-14T06:33:28.812 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 5 1 2024-09-14T06:33:29.804 INFO:teuthology.orchestra.run.smithi094.stderr:reweighted osd.5 to 1 (10000) 2024-09-14T06:33:29.818 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 5 1 2024-09-14T06:33:30.807 INFO:teuthology.orchestra.run.smithi094.stderr:set osd.5 primary-affinity to 1 (8655362) 2024-09-14T06:33:30.825 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 6 1 2024-09-14T06:33:31.816 INFO:teuthology.orchestra.run.smithi094.stderr:reweighted osd.6 to 1 (10000) 2024-09-14T06:33:31.833 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 6 1 2024-09-14T06:33:32.817 INFO:teuthology.orchestra.run.smithi094.stderr:set osd.6 primary-affinity to 1 (8655362) 2024-09-14T06:33:32.834 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 7 1 2024-09-14T06:33:33.828 INFO:teuthology.orchestra.run.smithi094.stderr:reweighted osd.7 to 1 (10000) 2024-09-14T06:33:33.843 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 7 1 2024-09-14T06:33:34.830 INFO:teuthology.orchestra.run.smithi094.stderr:set osd.7 primary-affinity to 1 (8655362) 2024-09-14T06:33:34.845 INFO:tasks.thrashosds.thrasher:joining the do_sighup greenlet 2024-09-14T06:33:34.845 INFO:tasks.thrashosds.thrasher:joining the do_optrack_toggle greenlet 2024-09-14T06:33:34.845 INFO:tasks.thrashosds.thrasher:joining the do_dump_ops greenlet 2024-09-14T06:33:34.845 INFO:tasks.thrashosds.thrasher:joining the do_noscrub_toggle greenlet 2024-09-14T06:33:34.845 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2024-09-14T06:33:34.846 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-14T06:33:35.133 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-14T06:33:35.133 INFO:teuthology.orchestra.run.smithi094.stdout:{"epoch":289,"fsid":"200ad3da-bd63-4e04-af74-efb5ca0af75e","created":"2024-09-14T06:28:30.782046+0000","modified":"2024-09-14T06:33:34.828528+0000","last_up_change":"2024-09-14T06:28:42.827239+0000","last_in_change":"2024-09-14T06:33:18.740365+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-14T06:28:43.295990+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2024-09-14T06:28:49.451714+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":2,"snap_epoch":22,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"pool_client.0","create_time":"2024-09-14T06:28:59.688535+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"262","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":92,"snap_epoch":262,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"8b22a249-b842-406a-a02c-c0f31f57b635","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":269,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6824","nonce":1809564923},{"type":"v1","addr":"172.21.15.94:6825","nonce":1809564923}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6826","nonce":1809564923},{"type":"v1","addr":"172.21.15.94:6827","nonce":1809564923}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6830","nonce":1809564923},{"type":"v1","addr":"172.21.15.94:6831","nonce":1809564923}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6828","nonce":1809564923},{"type":"v1","addr":"172.21.15.94:6829","nonce":1809564923}]},"public_addr":"172.21.15.94:6825/1809564923","cluster_addr":"172.21.15.94:6827/1809564923","heartbeat_back_addr":"172.21.15.94:6831/1809564923","heartbeat_front_addr":"172.21.15.94:6829/1809564923","state":["exists","up"]},{"osd":1,"uuid":"02251a03-84a0-414d-a980-fac4fd043cb6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":276,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6816","nonce":736402652},{"type":"v1","addr":"172.21.15.94:6817","nonce":736402652}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6818","nonce":736402652},{"type":"v1","addr":"172.21.15.94:6819","nonce":736402652}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6822","nonce":736402652},{"type":"v1","addr":"172.21.15.94:6823","nonce":736402652}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6820","nonce":736402652},{"type":"v1","addr":"172.21.15.94:6821","nonce":736402652}]},"public_addr":"172.21.15.94:6817/736402652","cluster_addr":"172.21.15.94:6819/736402652","heartbeat_back_addr":"172.21.15.94:6823/736402652","heartbeat_front_addr":"172.21.15.94:6821/736402652","state":["exists","up"]},{"osd":2,"uuid":"d45bf470-bb29-4739-8294-cd122b2b4102","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":273,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6808","nonce":1063204782},{"type":"v1","addr":"172.21.15.94:6809","nonce":1063204782}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6810","nonce":1063204782},{"type":"v1","addr":"172.21.15.94:6811","nonce":1063204782}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6814","nonce":1063204782},{"type":"v1","addr":"172.21.15.94:6815","nonce":1063204782}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6812","nonce":1063204782},{"type":"v1","addr":"172.21.15.94:6813","nonce":1063204782}]},"public_addr":"172.21.15.94:6809/1063204782","cluster_addr":"172.21.15.94:6811/1063204782","heartbeat_back_addr":"172.21.15.94:6815/1063204782","heartbeat_front_addr":"172.21.15.94:6813/1063204782","state":["exists","up"]},{"osd":3,"uuid":"456ec2d9-c438-4243-b931-335704f661eb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":270,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6800","nonce":3382046754},{"type":"v1","addr":"172.21.15.94:6801","nonce":3382046754}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6802","nonce":3382046754},{"type":"v1","addr":"172.21.15.94:6803","nonce":3382046754}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6806","nonce":3382046754},{"type":"v1","addr":"172.21.15.94:6807","nonce":3382046754}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6804","nonce":3382046754},{"type":"v1","addr":"172.21.15.94:6805","nonce":3382046754}]},"public_addr":"172.21.15.94:6801/3382046754","cluster_addr":"172.21.15.94:6803/3382046754","heartbeat_back_addr":"172.21.15.94:6807/3382046754","heartbeat_front_addr":"172.21.15.94:6805/3382046754","state":["exists","up"]},{"osd":4,"uuid":"2dab948e-c6e4-4437-bf17-017d1ac6821c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":276,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6810","nonce":501726729},{"type":"v1","addr":"172.21.15.138:6811","nonce":501726729}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6812","nonce":501726729},{"type":"v1","addr":"172.21.15.138:6813","nonce":501726729}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6816","nonce":501726729},{"type":"v1","addr":"172.21.15.138:6817","nonce":501726729}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6814","nonce":501726729},{"type":"v1","addr":"172.21.15.138:6815","nonce":501726729}]},"public_addr":"172.21.15.138:6811/501726729","cluster_addr":"172.21.15.138:6813/501726729","heartbeat_back_addr":"172.21.15.138:6817/501726729","heartbeat_front_addr":"172.21.15.138:6815/501726729","state":["exists","up"]},{"osd":5,"uuid":"8caf3aa3-ad87-4d33-b82e-4c15195176fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":285,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6826","nonce":1599292735},{"type":"v1","addr":"172.21.15.138:6827","nonce":1599292735}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6828","nonce":1599292735},{"type":"v1","addr":"172.21.15.138:6829","nonce":1599292735}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6832","nonce":1599292735},{"type":"v1","addr":"172.21.15.138:6833","nonce":1599292735}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6830","nonce":1599292735},{"type":"v1","addr":"172.21.15.138:6831","nonce":1599292735}]},"public_addr":"172.21.15.138:6827/1599292735","cluster_addr":"172.21.15.138:6829/1599292735","heartbeat_back_addr":"172.21.15.138:6833/1599292735","heartbeat_front_addr":"172.21.15.138:6831/1599292735","state":["exists","up"]},{"osd":6,"uuid":"e55d11fd-433d-4462-93a7-d5e84560221b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":271,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6818","nonce":163531479},{"type":"v1","addr":"172.21.15.138:6819","nonce":163531479}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6820","nonce":163531479},{"type":"v1","addr":"172.21.15.138:6821","nonce":163531479}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6824","nonce":163531479},{"type":"v1","addr":"172.21.15.138:6825","nonce":163531479}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6822","nonce":163531479},{"type":"v1","addr":"172.21.15.138:6823","nonce":163531479}]},"public_addr":"172.21.15.138:6819/163531479","cluster_addr":"172.21.15.138:6821/163531479","heartbeat_back_addr":"172.21.15.138:6825/163531479","heartbeat_front_addr":"172.21.15.138:6823/163531479","state":["exists","up"]},{"osd":7,"uuid":"ba14ba69-2c6c-4d09-81e2-2593dc3a7ea9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":255,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6800","nonce":218507235},{"type":"v1","addr":"172.21.15.138:6801","nonce":218507235}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6802","nonce":218507235},{"type":"v1","addr":"172.21.15.138:6803","nonce":218507235}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6806","nonce":218507235},{"type":"v1","addr":"172.21.15.138:6807","nonce":218507235}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6804","nonce":218507235},{"type":"v1","addr":"172.21.15.138:6805","nonce":218507235}]},"public_addr":"172.21.15.138:6801/218507235","cluster_addr":"172.21.15.138:6803/218507235","heartbeat_back_addr":"172.21.15.138:6807/218507235","heartbeat_front_addr":"172.21.15.138:6805/218507235","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-14T06:28:39.906639+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-14T06:28:40.541279+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-14T06:28:40.025350+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-14T06:28:39.217944+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-14T06:28:40.185898+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-14T06:28:40.783504+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-14T06:28:40.527305+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-14T06:28:40.349676+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.e","mappings":[{"from":4,"to":1}]},{"pgid":"3.11","mappings":[{"from":4,"to":1}]},{"pgid":"3.1c","mappings":[{"from":4,"to":1}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-14T06:33:35.146 INFO:tasks.ceph.ceph_manager.ceph:all up! 2024-09-14T06:33:35.146 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-14T06:33:35.430 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-14T06:33:35.430 INFO:teuthology.orchestra.run.smithi094.stdout:{"epoch":289,"fsid":"200ad3da-bd63-4e04-af74-efb5ca0af75e","created":"2024-09-14T06:28:30.782046+0000","modified":"2024-09-14T06:33:34.828528+0000","last_up_change":"2024-09-14T06:28:42.827239+0000","last_in_change":"2024-09-14T06:33:18.740365+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-14T06:28:43.295990+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2024-09-14T06:28:49.451714+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":2,"snap_epoch":22,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"pool_client.0","create_time":"2024-09-14T06:28:59.688535+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"262","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":92,"snap_epoch":262,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"8b22a249-b842-406a-a02c-c0f31f57b635","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":269,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6824","nonce":1809564923},{"type":"v1","addr":"172.21.15.94:6825","nonce":1809564923}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6826","nonce":1809564923},{"type":"v1","addr":"172.21.15.94:6827","nonce":1809564923}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6830","nonce":1809564923},{"type":"v1","addr":"172.21.15.94:6831","nonce":1809564923}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6828","nonce":1809564923},{"type":"v1","addr":"172.21.15.94:6829","nonce":1809564923}]},"public_addr":"172.21.15.94:6825/1809564923","cluster_addr":"172.21.15.94:6827/1809564923","heartbeat_back_addr":"172.21.15.94:6831/1809564923","heartbeat_front_addr":"172.21.15.94:6829/1809564923","state":["exists","up"]},{"osd":1,"uuid":"02251a03-84a0-414d-a980-fac4fd043cb6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":276,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6816","nonce":736402652},{"type":"v1","addr":"172.21.15.94:6817","nonce":736402652}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6818","nonce":736402652},{"type":"v1","addr":"172.21.15.94:6819","nonce":736402652}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6822","nonce":736402652},{"type":"v1","addr":"172.21.15.94:6823","nonce":736402652}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6820","nonce":736402652},{"type":"v1","addr":"172.21.15.94:6821","nonce":736402652}]},"public_addr":"172.21.15.94:6817/736402652","cluster_addr":"172.21.15.94:6819/736402652","heartbeat_back_addr":"172.21.15.94:6823/736402652","heartbeat_front_addr":"172.21.15.94:6821/736402652","state":["exists","up"]},{"osd":2,"uuid":"d45bf470-bb29-4739-8294-cd122b2b4102","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":273,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6808","nonce":1063204782},{"type":"v1","addr":"172.21.15.94:6809","nonce":1063204782}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6810","nonce":1063204782},{"type":"v1","addr":"172.21.15.94:6811","nonce":1063204782}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6814","nonce":1063204782},{"type":"v1","addr":"172.21.15.94:6815","nonce":1063204782}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6812","nonce":1063204782},{"type":"v1","addr":"172.21.15.94:6813","nonce":1063204782}]},"public_addr":"172.21.15.94:6809/1063204782","cluster_addr":"172.21.15.94:6811/1063204782","heartbeat_back_addr":"172.21.15.94:6815/1063204782","heartbeat_front_addr":"172.21.15.94:6813/1063204782","state":["exists","up"]},{"osd":3,"uuid":"456ec2d9-c438-4243-b931-335704f661eb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":270,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6800","nonce":3382046754},{"type":"v1","addr":"172.21.15.94:6801","nonce":3382046754}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6802","nonce":3382046754},{"type":"v1","addr":"172.21.15.94:6803","nonce":3382046754}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6806","nonce":3382046754},{"type":"v1","addr":"172.21.15.94:6807","nonce":3382046754}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6804","nonce":3382046754},{"type":"v1","addr":"172.21.15.94:6805","nonce":3382046754}]},"public_addr":"172.21.15.94:6801/3382046754","cluster_addr":"172.21.15.94:6803/3382046754","heartbeat_back_addr":"172.21.15.94:6807/3382046754","heartbeat_front_addr":"172.21.15.94:6805/3382046754","state":["exists","up"]},{"osd":4,"uuid":"2dab948e-c6e4-4437-bf17-017d1ac6821c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":276,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6810","nonce":501726729},{"type":"v1","addr":"172.21.15.138:6811","nonce":501726729}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6812","nonce":501726729},{"type":"v1","addr":"172.21.15.138:6813","nonce":501726729}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6816","nonce":501726729},{"type":"v1","addr":"172.21.15.138:6817","nonce":501726729}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6814","nonce":501726729},{"type":"v1","addr":"172.21.15.138:6815","nonce":501726729}]},"public_addr":"172.21.15.138:6811/501726729","cluster_addr":"172.21.15.138:6813/501726729","heartbeat_back_addr":"172.21.15.138:6817/501726729","heartbeat_front_addr":"172.21.15.138:6815/501726729","state":["exists","up"]},{"osd":5,"uuid":"8caf3aa3-ad87-4d33-b82e-4c15195176fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":285,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6826","nonce":1599292735},{"type":"v1","addr":"172.21.15.138:6827","nonce":1599292735}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6828","nonce":1599292735},{"type":"v1","addr":"172.21.15.138:6829","nonce":1599292735}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6832","nonce":1599292735},{"type":"v1","addr":"172.21.15.138:6833","nonce":1599292735}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6830","nonce":1599292735},{"type":"v1","addr":"172.21.15.138:6831","nonce":1599292735}]},"public_addr":"172.21.15.138:6827/1599292735","cluster_addr":"172.21.15.138:6829/1599292735","heartbeat_back_addr":"172.21.15.138:6833/1599292735","heartbeat_front_addr":"172.21.15.138:6831/1599292735","state":["exists","up"]},{"osd":6,"uuid":"e55d11fd-433d-4462-93a7-d5e84560221b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":271,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6818","nonce":163531479},{"type":"v1","addr":"172.21.15.138:6819","nonce":163531479}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6820","nonce":163531479},{"type":"v1","addr":"172.21.15.138:6821","nonce":163531479}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6824","nonce":163531479},{"type":"v1","addr":"172.21.15.138:6825","nonce":163531479}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6822","nonce":163531479},{"type":"v1","addr":"172.21.15.138:6823","nonce":163531479}]},"public_addr":"172.21.15.138:6819/163531479","cluster_addr":"172.21.15.138:6821/163531479","heartbeat_back_addr":"172.21.15.138:6825/163531479","heartbeat_front_addr":"172.21.15.138:6823/163531479","state":["exists","up"]},{"osd":7,"uuid":"ba14ba69-2c6c-4d09-81e2-2593dc3a7ea9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":255,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6800","nonce":218507235},{"type":"v1","addr":"172.21.15.138:6801","nonce":218507235}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6802","nonce":218507235},{"type":"v1","addr":"172.21.15.138:6803","nonce":218507235}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6806","nonce":218507235},{"type":"v1","addr":"172.21.15.138:6807","nonce":218507235}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6804","nonce":218507235},{"type":"v1","addr":"172.21.15.138:6805","nonce":218507235}]},"public_addr":"172.21.15.138:6801/218507235","cluster_addr":"172.21.15.138:6803/218507235","heartbeat_back_addr":"172.21.15.138:6807/218507235","heartbeat_front_addr":"172.21.15.138:6805/218507235","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-14T06:28:39.906639+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-14T06:28:40.541279+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-14T06:28:40.025350+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-14T06:28:39.217944+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-14T06:28:40.185898+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-14T06:28:40.783504+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-14T06:28:40.527305+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-14T06:28:40.349676+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.e","mappings":[{"from":4,"to":1}]},{"pgid":"3.11","mappings":[{"from":4,"to":1}]},{"pgid":"3.1c","mappings":[{"from":4,"to":1}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-14T06:33:35.443 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2024-09-14T06:33:35.443 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2024-09-14T06:33:35.444 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2024-09-14T06:33:35.444 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 flush_pg_stats 2024-09-14T06:33:35.444 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 flush_pg_stats 2024-09-14T06:33:35.445 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 flush_pg_stats 2024-09-14T06:33:35.445 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 flush_pg_stats 2024-09-14T06:33:35.446 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 flush_pg_stats 2024-09-14T06:33:35.681 INFO:teuthology.orchestra.run.smithi094.stdout:64424509503 2024-09-14T06:33:35.682 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-09-14T06:33:35.683 INFO:teuthology.orchestra.run.smithi094.stdout:60129542206 2024-09-14T06:33:35.684 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-09-14T06:33:35.685 INFO:teuthology.orchestra.run.smithi094.stdout:64424509502 2024-09-14T06:33:35.686 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-09-14T06:33:35.688 INFO:teuthology.orchestra.run.smithi094.stdout:64424509503 2024-09-14T06:33:35.688 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-09-14T06:33:35.693 INFO:teuthology.orchestra.run.smithi094.stdout:64424509502 2024-09-14T06:33:35.693 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2024-09-14T06:33:35.694 INFO:teuthology.orchestra.run.smithi094.stdout:60129542205 2024-09-14T06:33:35.695 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-09-14T06:33:35.696 INFO:teuthology.orchestra.run.smithi094.stdout:55834574910 2024-09-14T06:33:35.696 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-09-14T06:33:35.697 INFO:teuthology.orchestra.run.smithi094.stdout:64424509502 2024-09-14T06:33:35.697 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2024-09-14T06:33:36.240 INFO:teuthology.orchestra.run.smithi094.stdout:64424509502 2024-09-14T06:33:36.249 INFO:teuthology.orchestra.run.smithi094.stdout:60129542205 2024-09-14T06:33:36.259 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509503 got 64424509502 for osd.1 2024-09-14T06:33:36.269 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542206 got 60129542205 for osd.2 2024-09-14T06:33:36.287 INFO:teuthology.orchestra.run.smithi094.stdout:64424509501 2024-09-14T06:33:36.297 INFO:teuthology.orchestra.run.smithi094.stdout:64424509502 2024-09-14T06:33:36.301 INFO:teuthology.orchestra.run.smithi094.stdout:60129542204 2024-09-14T06:33:36.307 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509502 got 64424509501 for osd.4 2024-09-14T06:33:36.315 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509503 got 64424509502 for osd.5 2024-09-14T06:33:36.318 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542205 got 60129542204 for osd.0 2024-09-14T06:33:36.322 INFO:teuthology.orchestra.run.smithi094.stdout:55834574909 2024-09-14T06:33:36.323 INFO:teuthology.orchestra.run.smithi094.stdout:64424509501 2024-09-14T06:33:36.337 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509502 got 64424509501 for osd.6 2024-09-14T06:33:36.340 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574910 got 55834574909 for osd.3 2024-09-14T06:33:36.345 INFO:teuthology.orchestra.run.smithi094.stdout:64424509501 2024-09-14T06:33:36.358 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509502 got 64424509501 for osd.7 2024-09-14T06:33:37.260 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-09-14T06:33:37.270 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-09-14T06:33:37.308 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-09-14T06:33:37.316 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-09-14T06:33:37.319 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-09-14T06:33:37.338 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2024-09-14T06:33:37.341 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-09-14T06:33:37.359 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2024-09-14T06:33:37.807 INFO:teuthology.orchestra.run.smithi094.stdout:64424509502 2024-09-14T06:33:37.825 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509503 got 64424509502 for osd.1 2024-09-14T06:33:37.878 INFO:teuthology.orchestra.run.smithi094.stdout:60129542206 2024-09-14T06:33:37.891 INFO:teuthology.orchestra.run.smithi094.stdout:60129542205 2024-09-14T06:33:37.897 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542206 got 60129542206 for osd.2 2024-09-14T06:33:37.898 DEBUG:teuthology.parallel:result is None 2024-09-14T06:33:37.905 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542205 got 60129542205 for osd.0 2024-09-14T06:33:37.906 DEBUG:teuthology.parallel:result is None 2024-09-14T06:33:37.918 INFO:teuthology.orchestra.run.smithi094.stdout:64424509502 2024-09-14T06:33:37.930 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509502 got 64424509502 for osd.4 2024-09-14T06:33:37.930 DEBUG:teuthology.parallel:result is None 2024-09-14T06:33:37.940 INFO:teuthology.orchestra.run.smithi094.stdout:55834574911 2024-09-14T06:33:37.958 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574910 got 55834574911 for osd.3 2024-09-14T06:33:37.958 DEBUG:teuthology.parallel:result is None 2024-09-14T06:33:37.967 INFO:teuthology.orchestra.run.smithi094.stdout:64424509502 2024-09-14T06:33:37.969 INFO:teuthology.orchestra.run.smithi094.stdout:64424509503 2024-09-14T06:33:37.977 INFO:teuthology.orchestra.run.smithi094.stdout:64424509502 2024-09-14T06:33:37.982 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509502 got 64424509502 for osd.6 2024-09-14T06:33:37.982 DEBUG:teuthology.parallel:result is None 2024-09-14T06:33:37.983 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509503 got 64424509503 for osd.5 2024-09-14T06:33:37.983 DEBUG:teuthology.parallel:result is None 2024-09-14T06:33:37.991 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509502 got 64424509502 for osd.7 2024-09-14T06:33:37.991 DEBUG:teuthology.parallel:result is None 2024-09-14T06:33:38.826 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-09-14T06:33:39.115 INFO:teuthology.orchestra.run.smithi094.stdout:64424509503 2024-09-14T06:33:39.128 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509503 got 64424509503 for osd.1 2024-09-14T06:33:39.128 DEBUG:teuthology.parallel:result is None 2024-09-14T06:33:39.128 INFO:tasks.ceph.ceph_manager.ceph:waiting for recovery to complete 2024-09-14T06:33:39.128 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-14T06:33:39.411 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-14T06:33:39.412 INFO:teuthology.orchestra.run.smithi094.stderr:dumped all 2024-09-14T06:33:39.424 INFO:teuthology.orchestra.run.smithi094.stdout:{"pg_ready":true,"pg_map":{"version":432,"stamp":"2024-09-14T06:33:39.336800+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590406,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":240158,"num_read_kb":2184988,"num_write":38248,"num_write_kb":1480016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7477,"num_bytes_recovered":638497624,"num_keys_recovered":546,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":28723,"ondisk_log_size":28723,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":575440,"kb_used_data":27480,"kb_used_omap":12,"kb_used_meta":547763,"kb_avail":754399280,"statfs":{"total":773094113280,"available":772504862720,"internally_reserved":0,"allocated":28139520,"data_stored":14988124,"data_compressed":65166,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":12751,"internal_metadata":560909873},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":-3,"num_read_kb":-3,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":-1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.516956"},"pg_stats":[{"pgid":"3.1f","version":"262'855","reported_seq":8431,"reported_epoch":289,"state":"active+clean","last_fresh":"2024-09-14T06:33:34.979756+0000","last_change":"2024-09-14T06:33:07.706447+0000","last_active":"2024-09-14T06:33:34.979756+0000","last_peered":"2024-09-14T06:33:34.979756+0000","last_clean":"2024-09-14T06:33:34.979756+0000","last_became_active":"2024-09-14T06:29:00.579883+0000","last_became_peered":"2024-09-14T06:29:00.579883+0000","last_unstale":"2024-09-14T06:33:34.979756+0000","last_undegraded":"2024-09-14T06:33:34.979756+0000","last_fullsized":"2024-09-14T06:33:34.979756+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":855,"log_dups_size":0,"ondisk_log_size":855,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T14:40:25.634308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001666641,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":8468,"num_read_kb":86898,"num_write":1171,"num_write_kb":49543,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"262'783","reported_seq":10851,"reported_epoch":287,"state":"active+clean","last_fresh":"2024-09-14T06:33:32.819008+0000","last_change":"2024-09-14T06:33:16.628665+0000","last_active":"2024-09-14T06:33:32.819008+0000","last_peered":"2024-09-14T06:33:32.819008+0000","last_clean":"2024-09-14T06:33:32.819008+0000","last_became_active":"2024-09-14T06:33:16.626222+0000","last_became_peered":"2024-09-14T06:33:16.626222+0000","last_unstale":"2024-09-14T06:33:32.819008+0000","last_undegraded":"2024-09-14T06:33:32.819008+0000","last_fullsized":"2024-09-14T06:33:32.819008+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":783,"log_dups_size":0,"ondisk_log_size":783,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T15:32:44.526538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013639839999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":8315,"num_read_kb":67657,"num_write":1025,"num_write_kb":40407,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1025,"num_bytes_recovered":87221212,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1d","version":"260'1077","reported_seq":7485,"reported_epoch":289,"state":"active+clean","last_fresh":"2024-09-14T06:33:35.112385+0000","last_change":"2024-09-14T06:33:31.826989+0000","last_active":"2024-09-14T06:33:35.112385+0000","last_peered":"2024-09-14T06:33:35.112385+0000","last_clean":"2024-09-14T06:33:35.112385+0000","last_became_active":"2024-09-14T06:33:31.824454+0000","last_became_peered":"2024-09-14T06:33:31.824454+0000","last_unstale":"2024-09-14T06:33:35.112385+0000","last_undegraded":"2024-09-14T06:33:35.112385+0000","last_fullsized":"2024-09-14T06:33:35.112385+0000","mapping_epoch":285,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":286,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":1077,"log_dups_size":0,"ondisk_log_size":1077,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T07:07:12.782032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088638400000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":7873,"num_read_kb":64375,"num_write":1555,"num_write_kb":53292,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1c","version":"262'805","reported_seq":6034,"reported_epoch":289,"state":"active+clean","last_fresh":"2024-09-14T06:33:35.112317+0000","last_change":"2024-09-14T06:33:31.825738+0000","last_active":"2024-09-14T06:33:35.112317+0000","last_peered":"2024-09-14T06:33:35.112317+0000","last_clean":"2024-09-14T06:33:35.112317+0000","last_became_active":"2024-09-14T06:33:31.823955+0000","last_became_peered":"2024-09-14T06:33:31.823955+0000","last_unstale":"2024-09-14T06:33:35.112317+0000","last_undegraded":"2024-09-14T06:33:35.112317+0000","last_fullsized":"2024-09-14T06:33:35.112317+0000","mapping_epoch":285,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":286,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":805,"log_dups_size":0,"ondisk_log_size":805,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T06:54:41.083735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00068758600000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":4953,"num_read_kb":43809,"num_write":946,"num_write_kb":45551,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":505,"num_bytes_recovered":44145197,"num_keys_recovered":280,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"259'960","reported_seq":11521,"reported_epoch":289,"state":"active+clean","last_fresh":"2024-09-14T06:33:34.979721+0000","last_change":"2024-09-14T06:33:07.706745+0000","last_active":"2024-09-14T06:33:34.979721+0000","last_peered":"2024-09-14T06:33:34.979721+0000","last_clean":"2024-09-14T06:33:34.979721+0000","last_became_active":"2024-09-14T06:29:00.586183+0000","last_became_peered":"2024-09-14T06:29:00.586183+0000","last_unstale":"2024-09-14T06:33:34.979721+0000","last_undegraded":"2024-09-14T06:33:34.979721+0000","last_fullsized":"2024-09-14T06:33:34.979721+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":960,"log_dups_size":0,"ondisk_log_size":960,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T10:55:09.317291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001810606,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":11910,"num_read_kb":108094,"num_write":1463,"num_write_kb":55102,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1a","version":"260'1678","reported_seq":10638,"reported_epoch":289,"state":"active+clean","last_fresh":"2024-09-14T06:33:36.039825+0000","last_change":"2024-09-14T06:33:07.705134+0000","last_active":"2024-09-14T06:33:36.039825+0000","last_peered":"2024-09-14T06:33:36.039825+0000","last_clean":"2024-09-14T06:33:36.039825+0000","last_became_active":"2024-09-14T06:29:00.576206+0000","last_became_peered":"2024-09-14T06:29:00.576206+0000","last_unstale":"2024-09-14T06:33:36.039825+0000","last_undegraded":"2024-09-14T06:33:36.039825+0000","last_fullsized":"2024-09-14T06:33:36.039825+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":1678,"log_dups_size":0,"ondisk_log_size":1678,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T10:47:38.190388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088286099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":13339,"num_read_kb":82904,"num_write":2451,"num_write_kb":55687,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.19","version":"261'561","reported_seq":6508,"reported_epoch":289,"state":"active+clean","last_fresh":"2024-09-14T06:33:34.831894+0000","last_change":"2024-09-14T06:33:07.704594+0000","last_active":"2024-09-14T06:33:34.831894+0000","last_peered":"2024-09-14T06:33:34.831894+0000","last_clean":"2024-09-14T06:33:34.831894+0000","last_became_active":"2024-09-14T06:31:53.744800+0000","last_became_peered":"2024-09-14T06:31:53.744800+0000","last_unstale":"2024-09-14T06:33:34.831894+0000","last_undegraded":"2024-09-14T06:33:34.831894+0000","last_fullsized":"2024-09-14T06:33:34.831894+0000","mapping_epoch":188,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":189,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":561,"log_dups_size":0,"ondisk_log_size":561,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T13:59:24.308993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062201200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":5123,"num_read_kb":67781,"num_write":764,"num_write_kb":43873,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":282,"num_bytes_recovered":41927489,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"262'547","reported_seq":6285,"reported_epoch":287,"state":"active+clean","last_fresh":"2024-09-14T06:33:32.818617+0000","last_change":"2024-09-14T06:33:07.769587+0000","last_active":"2024-09-14T06:33:32.818617+0000","last_peered":"2024-09-14T06:33:32.818617+0000","last_clean":"2024-09-14T06:33:32.818617+0000","last_became_active":"2024-09-14T06:31:53.745159+0000","last_became_peered":"2024-09-14T06:31:53.745159+0000","last_unstale":"2024-09-14T06:33:32.818617+0000","last_undegraded":"2024-09-14T06:33:32.818617+0000","last_fullsized":"2024-09-14T06:33:32.818617+0000","mapping_epoch":188,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":189,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":547,"log_dups_size":0,"ondisk_log_size":547,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T13:08:27.610323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00079631400000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4489,"num_read_kb":78917,"num_write":672,"num_write_kb":51581,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":310,"num_bytes_recovered":56082976,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"261'1037","reported_seq":9728,"reported_epoch":289,"state":"active+clean","last_fresh":"2024-09-14T06:33:34.979868+0000","last_change":"2024-09-14T06:33:07.706863+0000","last_active":"2024-09-14T06:33:34.979868+0000","last_peered":"2024-09-14T06:33:34.979868+0000","last_clean":"2024-09-14T06:33:34.979868+0000","last_became_active":"2024-09-14T06:29:00.579714+0000","last_became_peered":"2024-09-14T06:29:00.579714+0000","last_unstale":"2024-09-14T06:33:34.979868+0000","last_undegraded":"2024-09-14T06:33:34.979868+0000","last_fullsized":"2024-09-14T06:33:34.979868+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":1037,"log_dups_size":0,"ondisk_log_size":1037,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T15:17:51.610057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0017747889999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":10688,"num_read_kb":68964,"num_write":1162,"num_write_kb":48482,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.16","version":"255'1156","reported_seq":4137,"reported_epoch":289,"state":"active+clean","last_fresh":"2024-09-14T06:33:35.112491+0000","last_change":"2024-09-14T06:33:31.824083+0000","last_active":"2024-09-14T06:33:35.112491+0000","last_peered":"2024-09-14T06:33:35.112491+0000","last_clean":"2024-09-14T06:33:35.112491+0000","last_became_active":"2024-09-14T06:33:31.821620+0000","last_became_peered":"2024-09-14T06:33:31.821620+0000","last_unstale":"2024-09-14T06:33:35.112491+0000","last_undegraded":"2024-09-14T06:33:35.112491+0000","last_fullsized":"2024-09-14T06:33:35.112491+0000","mapping_epoch":285,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":286,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":1156,"log_dups_size":0,"ondisk_log_size":1156,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T08:10:02.587758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00073433000000000005,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":5658,"num_read_kb":38562,"num_write":1743,"num_write_kb":37248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"261'650","reported_seq":6751,"reported_epoch":289,"state":"active+clean","last_fresh":"2024-09-14T06:33:34.834516+0000","last_change":"2024-09-14T06:33:07.769581+0000","last_active":"2024-09-14T06:33:34.834516+0000","last_peered":"2024-09-14T06:33:34.834516+0000","last_clean":"2024-09-14T06:33:34.834516+0000","last_became_active":"2024-09-14T06:31:53.746091+0000","last_became_peered":"2024-09-14T06:31:53.746091+0000","last_unstale":"2024-09-14T06:33:34.834516+0000","last_undegraded":"2024-09-14T06:33:34.834516+0000","last_fullsized":"2024-09-14T06:33:34.834516+0000","mapping_epoch":188,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":189,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":650,"log_dups_size":0,"ondisk_log_size":650,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T12:36:31.690559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000442557,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":5101,"num_read_kb":74899,"num_write":696,"num_write_kb":51196,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":271,"num_bytes_recovered":49981473,"num_keys_recovered":33,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.14","version":"261'1137","reported_seq":6341,"reported_epoch":289,"state":"active+clean","last_fresh":"2024-09-14T06:33:36.039887+0000","last_change":"2024-09-14T06:33:07.704760+0000","last_active":"2024-09-14T06:33:36.039887+0000","last_peered":"2024-09-14T06:33:36.039887+0000","last_clean":"2024-09-14T06:33:36.039887+0000","last_became_active":"2024-09-14T06:33:04.628663+0000","last_became_peered":"2024-09-14T06:33:04.628663+0000","last_unstale":"2024-09-14T06:33:36.039887+0000","last_undegraded":"2024-09-14T06:33:36.039887+0000","last_fullsized":"2024-09-14T06:33:36.039887+0000","mapping_epoch":258,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":259,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":1137,"log_dups_size":0,"ondisk_log_size":1137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T11:42:47.162926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00078945399999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":8948,"num_read_kb":46936,"num_write":1573,"num_write_kb":29829,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":296,"num_bytes_recovered":6118769,"num_keys_recovered":11,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":50,"reported_epoch":289,"state":"active+clean","last_fresh":"2024-09-14T06:33:36.039662+0000","last_change":"2024-09-14T06:33:13.859355+0000","last_active":"2024-09-14T06:33:36.039662+0000","last_peered":"2024-09-14T06:33:36.039662+0000","last_clean":"2024-09-14T06:33:36.039662+0000","last_became_active":"2024-09-14T06:33:13.859069+0000","last_became_peered":"2024-09-14T06:33:13.859069+0000","last_unstale":"2024-09-14T06:33:36.039662+0000","last_undegraded":"2024-09-14T06:33:36.039662+0000","last_fullsized":"2024-09-14T06:33:36.039662+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:28:50.205350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:28:50.205350+0000","last_clean_scrub_stamp":"2024-09-14T06:28:50.205350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T17:08:35.416194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"262'893","reported_seq":8726,"reported_epoch":289,"state":"active+clean","last_fresh":"2024-09-14T06:33:34.979753+0000","last_change":"2024-09-14T06:33:07.705814+0000","last_active":"2024-09-14T06:33:34.979753+0000","last_peered":"2024-09-14T06:33:34.979753+0000","last_clean":"2024-09-14T06:33:34.979753+0000","last_became_active":"2024-09-14T06:29:00.578283+0000","last_became_peered":"2024-09-14T06:29:00.578283+0000","last_unstale":"2024-09-14T06:33:34.979753+0000","last_undegraded":"2024-09-14T06:33:34.979753+0000","last_fullsized":"2024-09-14T06:33:34.979753+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":893,"log_dups_size":0,"ondisk_log_size":893,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T15:09:37.855181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001076554,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":8608,"num_read_kb":78770,"num_write":923,"num_write_kb":48303,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":567,"reported_epoch":289,"state":"active+clean","last_fresh":"2024-09-14T06:33:34.831888+0000","last_change":"2024-09-14T06:33:13.734687+0000","last_active":"2024-09-14T06:33:34.831888+0000","last_peered":"2024-09-14T06:33:34.831888+0000","last_clean":"2024-09-14T06:33:34.831888+0000","last_became_active":"2024-09-14T06:33:13.734442+0000","last_became_peered":"2024-09-14T06:33:13.734442+0000","last_unstale":"2024-09-14T06:33:34.831888+0000","last_undegraded":"2024-09-14T06:33:34.831888+0000","last_fullsized":"2024-09-14T06:33:34.831888+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:28:50.205350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:28:50.205350+0000","last_clean_scrub_stamp":"2024-09-14T06:28:50.205350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T12:50:57.166258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065018200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"261'840","reported_seq":4649,"reported_epoch":287,"state":"active+clean","last_fresh":"2024-09-14T06:33:32.819115+0000","last_change":"2024-09-14T06:33:07.769882+0000","last_active":"2024-09-14T06:33:32.819115+0000","last_peered":"2024-09-14T06:33:32.819115+0000","last_clean":"2024-09-14T06:33:32.819115+0000","last_became_active":"2024-09-14T06:31:54.747923+0000","last_became_peered":"2024-09-14T06:31:54.747923+0000","last_unstale":"2024-09-14T06:33:32.819115+0000","last_undegraded":"2024-09-14T06:33:32.819115+0000","last_fullsized":"2024-09-14T06:33:32.819115+0000","mapping_epoch":189,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":190,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":840,"log_dups_size":0,"ondisk_log_size":840,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T09:01:49.581187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000927559,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4591,"num_read_kb":41129,"num_write":1178,"num_write_kb":36122,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":183,"num_bytes_recovered":13912513,"num_keys_recovered":22,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":554,"reported_epoch":289,"state":"active+clean","last_fresh":"2024-09-14T06:33:34.834701+0000","last_change":"2024-09-14T06:28:53.640035+0000","last_active":"2024-09-14T06:33:34.834701+0000","last_peered":"2024-09-14T06:33:34.834701+0000","last_clean":"2024-09-14T06:33:34.834701+0000","last_became_active":"2024-09-14T06:28:51.308058+0000","last_became_peered":"2024-09-14T06:28:51.308058+0000","last_unstale":"2024-09-14T06:33:34.834701+0000","last_undegraded":"2024-09-14T06:33:34.834701+0000","last_fullsized":"2024-09-14T06:33:34.834701+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:28:50.205350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:28:50.205350+0000","last_clean_scrub_stamp":"2024-09-14T06:28:50.205350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T13:35:48.544057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00037824000000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"262'700","reported_seq":4876,"reported_epoch":289,"state":"active+clean","last_fresh":"2024-09-14T06:33:34.832050+0000","last_change":"2024-09-14T06:33:22.769657+0000","last_active":"2024-09-14T06:33:34.832050+0000","last_peered":"2024-09-14T06:33:34.832050+0000","last_clean":"2024-09-14T06:33:34.832050+0000","last_became_active":"2024-09-14T06:33:22.767544+0000","last_became_peered":"2024-09-14T06:33:22.767544+0000","last_unstale":"2024-09-14T06:33:34.832050+0000","last_undegraded":"2024-09-14T06:33:34.832050+0000","last_fullsized":"2024-09-14T06:33:34.832050+0000","mapping_epoch":276,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":277,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":700,"log_dups_size":0,"ondisk_log_size":700,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T13:09:29.858808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00074004799999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":2654,"num_read_kb":37435,"num_write":897,"num_write_kb":40860,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":687,"num_bytes_recovered":31411166,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":555,"reported_epoch":289,"state":"active+clean","last_fresh":"2024-09-14T06:33:34.832080+0000","last_change":"2024-09-14T06:28:53.226072+0000","last_active":"2024-09-14T06:33:34.832080+0000","last_peered":"2024-09-14T06:33:34.832080+0000","last_clean":"2024-09-14T06:33:34.832080+0000","last_became_active":"2024-09-14T06:28:51.308072+0000","last_became_peered":"2024-09-14T06:28:51.308072+0000","last_unstale":"2024-09-14T06:33:34.832080+0000","last_undegraded":"2024-09-14T06:33:34.832080+0000","last_fullsized":"2024-09-14T06:33:34.832080+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:28:50.205350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:28:50.205350+0000","last_clean_scrub_stamp":"2024-09-14T06:28:50.205350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T13:15:09.052328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066197399999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"261'896","reported_seq":5625,"reported_epoch":289,"state":"active+clean","last_fresh":"2024-09-14T06:33:35.112603+0000","last_change":"2024-09-14T06:33:31.823904+0000","last_active":"2024-09-14T06:33:35.112603+0000","last_peered":"2024-09-14T06:33:35.112603+0000","last_clean":"2024-09-14T06:33:35.112603+0000","last_became_active":"2024-09-14T06:33:31.821533+0000","last_became_peered":"2024-09-14T06:33:31.821533+0000","last_unstale":"2024-09-14T06:33:35.112603+0000","last_undegraded":"2024-09-14T06:33:35.112603+0000","last_fullsized":"2024-09-14T06:33:35.112603+0000","mapping_epoch":285,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":286,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":896,"log_dups_size":0,"ondisk_log_size":896,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T15:48:46.733188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015088129999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4516,"num_read_kb":42632,"num_write":1255,"num_write_kb":49494,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":330,"num_bytes_recovered":28798893,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"262'711","reported_seq":10924,"reported_epoch":289,"state":"active+clean","last_fresh":"2024-09-14T06:33:36.040034+0000","last_change":"2024-09-14T06:33:07.704808+0000","last_active":"2024-09-14T06:33:36.040034+0000","last_peered":"2024-09-14T06:33:36.040034+0000","last_clean":"2024-09-14T06:33:36.040034+0000","last_became_active":"2024-09-14T06:29:00.584678+0000","last_became_peered":"2024-09-14T06:29:00.584678+0000","last_unstale":"2024-09-14T06:33:36.040034+0000","last_undegraded":"2024-09-14T06:33:36.040034+0000","last_fullsized":"2024-09-14T06:33:36.040034+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":711,"log_dups_size":0,"ondisk_log_size":711,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T07:39:08.849874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00074987800000000005,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":9560,"num_read_kb":89311,"num_write":777,"num_write_kb":51691,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"22'2","reported_seq":428,"reported_epoch":289,"state":"active+clean","last_fresh":"2024-09-14T06:33:35.112361+0000","last_change":"2024-09-14T06:33:31.823823+0000","last_active":"2024-09-14T06:33:35.112361+0000","last_peered":"2024-09-14T06:33:35.112361+0000","last_clean":"2024-09-14T06:33:35.112361+0000","last_became_active":"2024-09-14T06:33:31.823429+0000","last_became_peered":"2024-09-14T06:33:31.823429+0000","last_unstale":"2024-09-14T06:33:35.112361+0000","last_undegraded":"2024-09-14T06:33:35.112361+0000","last_fullsized":"2024-09-14T06:33:35.112361+0000","mapping_epoch":285,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":286,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:28:50.205350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:28:50.205350+0000","last_clean_scrub_stamp":"2024-09-14T06:28:50.205350+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T18:01:36.855427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010072970000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.0","version":"260'1116","reported_seq":8244,"reported_epoch":289,"state":"active+clean","last_fresh":"2024-09-14T06:33:34.832141+0000","last_change":"2024-09-14T06:33:22.769183+0000","last_active":"2024-09-14T06:33:34.832141+0000","last_peered":"2024-09-14T06:33:34.832141+0000","last_clean":"2024-09-14T06:33:34.832141+0000","last_became_active":"2024-09-14T06:33:22.767520+0000","last_became_peered":"2024-09-14T06:33:22.767520+0000","last_unstale":"2024-09-14T06:33:34.832141+0000","last_undegraded":"2024-09-14T06:33:34.832141+0000","last_fullsized":"2024-09-14T06:33:34.832141+0000","mapping_epoch":276,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":277,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":1116,"log_dups_size":0,"ondisk_log_size":1116,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T17:26:46.362756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064233900000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":8259,"num_read_kb":85480,"num_write":1652,"num_write_kb":58700,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":537,"num_bytes_recovered":2852,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":40,"reported_epoch":289,"state":"active+clean","last_fresh":"2024-09-14T06:33:34.831819+0000","last_change":"2024-09-14T06:33:19.801852+0000","last_active":"2024-09-14T06:33:34.831819+0000","last_peered":"2024-09-14T06:33:34.831819+0000","last_clean":"2024-09-14T06:33:34.831819+0000","last_became_active":"2024-09-14T06:33:19.801557+0000","last_became_peered":"2024-09-14T06:33:19.801557+0000","last_unstale":"2024-09-14T06:33:34.831819+0000","last_undegraded":"2024-09-14T06:33:34.831819+0000","last_fullsized":"2024-09-14T06:33:34.831819+0000","mapping_epoch":273,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":274,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:28:50.205350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:28:50.205350+0000","last_clean_scrub_stamp":"2024-09-14T06:28:50.205350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T08:41:03.294249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053403100000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1","version":"261'822","reported_seq":11886,"reported_epoch":289,"state":"active+clean","last_fresh":"2024-09-14T06:33:34.979564+0000","last_change":"2024-09-14T06:33:07.705685+0000","last_active":"2024-09-14T06:33:34.979564+0000","last_peered":"2024-09-14T06:33:34.979564+0000","last_clean":"2024-09-14T06:33:34.979564+0000","last_became_active":"2024-09-14T06:30:57.768238+0000","last_became_peered":"2024-09-14T06:30:57.768238+0000","last_unstale":"2024-09-14T06:33:34.979564+0000","last_undegraded":"2024-09-14T06:33:34.979564+0000","last_fullsized":"2024-09-14T06:33:34.979564+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":822,"log_dups_size":0,"ondisk_log_size":822,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T15:07:18.804416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011933110000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":11284,"num_read_kb":112430,"num_write":991,"num_write_kb":47408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":183,"num_bytes_recovered":30431994,"num_keys_recovered":22,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":554,"reported_epoch":289,"state":"active+clean","last_fresh":"2024-09-14T06:33:34.834417+0000","last_change":"2024-09-14T06:28:53.640096+0000","last_active":"2024-09-14T06:33:34.834417+0000","last_peered":"2024-09-14T06:33:34.834417+0000","last_clean":"2024-09-14T06:33:34.834417+0000","last_became_active":"2024-09-14T06:28:51.223069+0000","last_became_peered":"2024-09-14T06:28:51.223069+0000","last_unstale":"2024-09-14T06:33:34.834417+0000","last_undegraded":"2024-09-14T06:33:34.834417+0000","last_fullsized":"2024-09-14T06:33:34.834417+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:28:50.205350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:28:50.205350+0000","last_clean_scrub_stamp":"2024-09-14T06:28:50.205350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T15:13:00.471509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00043714600000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"262'1663","reported_seq":7850,"reported_epoch":287,"state":"active+clean","last_fresh":"2024-09-14T06:33:32.818891+0000","last_change":"2024-09-14T06:33:07.770369+0000","last_active":"2024-09-14T06:33:32.818891+0000","last_peered":"2024-09-14T06:33:32.818891+0000","last_clean":"2024-09-14T06:33:32.818891+0000","last_became_active":"2024-09-14T06:31:54.744160+0000","last_became_peered":"2024-09-14T06:31:54.744160+0000","last_unstale":"2024-09-14T06:33:32.818891+0000","last_undegraded":"2024-09-14T06:33:32.818891+0000","last_fullsized":"2024-09-14T06:33:32.818891+0000","mapping_epoch":189,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":190,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":1663,"log_dups_size":0,"ondisk_log_size":1663,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T16:44:35.569843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001382608,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":10723,"num_read_kb":70589,"num_write":2596,"num_write_kb":53564,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":327,"num_bytes_recovered":50047063,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"20'1","reported_seq":427,"reported_epoch":289,"state":"active+clean","last_fresh":"2024-09-14T06:33:35.112503+0000","last_change":"2024-09-14T06:33:31.823426+0000","last_active":"2024-09-14T06:33:35.112503+0000","last_peered":"2024-09-14T06:33:35.112503+0000","last_clean":"2024-09-14T06:33:35.112503+0000","last_became_active":"2024-09-14T06:33:31.823032+0000","last_became_peered":"2024-09-14T06:33:31.823032+0000","last_unstale":"2024-09-14T06:33:35.112503+0000","last_undegraded":"2024-09-14T06:33:35.112503+0000","last_fullsized":"2024-09-14T06:33:35.112503+0000","mapping_epoch":285,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":286,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:28:50.205350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:28:50.205350+0000","last_clean_scrub_stamp":"2024-09-14T06:28:50.205350+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T08:31:58.917801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050808899999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'72","reported_seq":645,"reported_epoch":289,"state":"active+clean","last_fresh":"2024-09-14T06:33:34.834414+0000","last_change":"2024-09-14T06:28:44.839885+0000","last_active":"2024-09-14T06:33:34.834414+0000","last_peered":"2024-09-14T06:33:34.834414+0000","last_clean":"2024-09-14T06:33:34.834414+0000","last_became_active":"2024-09-14T06:28:44.839573+0000","last_became_peered":"2024-09-14T06:28:44.839573+0000","last_unstale":"2024-09-14T06:33:34.834414+0000","last_undegraded":"2024-09-14T06:33:34.834414+0000","last_fullsized":"2024-09-14T06:33:34.834414+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:28:43.828494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:28:43.828494+0000","last_clean_scrub_stamp":"2024-09-14T06:28:43.828494+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T14:41:25.787293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.8","version":"261'684","reported_seq":6657,"reported_epoch":287,"state":"active+clean","last_fresh":"2024-09-14T06:33:32.818645+0000","last_change":"2024-09-14T06:33:07.769379+0000","last_active":"2024-09-14T06:33:32.818645+0000","last_peered":"2024-09-14T06:33:32.818645+0000","last_clean":"2024-09-14T06:33:32.818645+0000","last_became_active":"2024-09-14T06:31:55.754694+0000","last_became_peered":"2024-09-14T06:31:55.754694+0000","last_unstale":"2024-09-14T06:33:32.818645+0000","last_undegraded":"2024-09-14T06:33:32.818645+0000","last_fullsized":"2024-09-14T06:33:32.818645+0000","mapping_epoch":190,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":191,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":684,"log_dups_size":0,"ondisk_log_size":684,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T15:48:47.633183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067851599999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":5793,"num_read_kb":90852,"num_write":921,"num_write_kb":57691,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":319,"num_bytes_recovered":60352269,"num_keys_recovered":22,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.9","version":"261'597","reported_seq":5173,"reported_epoch":289,"state":"active+clean","last_fresh":"2024-09-14T06:33:36.040020+0000","last_change":"2024-09-14T06:33:22.772981+0000","last_active":"2024-09-14T06:33:36.040020+0000","last_peered":"2024-09-14T06:33:36.040020+0000","last_clean":"2024-09-14T06:33:36.040020+0000","last_became_active":"2024-09-14T06:33:22.770956+0000","last_became_peered":"2024-09-14T06:33:22.770956+0000","last_unstale":"2024-09-14T06:33:36.040020+0000","last_undegraded":"2024-09-14T06:33:36.040020+0000","last_fullsized":"2024-09-14T06:33:36.040020+0000","mapping_epoch":276,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":277,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":597,"log_dups_size":0,"ondisk_log_size":597,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T17:14:06.174583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057139500000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3678,"num_read_kb":64291,"num_write":778,"num_write_kb":44928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":316,"num_bytes_recovered":49117,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"261'874","reported_seq":5176,"reported_epoch":289,"state":"active+clean","last_fresh":"2024-09-14T06:33:36.039739+0000","last_change":"2024-09-14T06:33:17.629900+0000","last_active":"2024-09-14T06:33:36.039739+0000","last_peered":"2024-09-14T06:33:36.039739+0000","last_clean":"2024-09-14T06:33:36.039739+0000","last_became_active":"2024-09-14T06:33:17.627910+0000","last_became_peered":"2024-09-14T06:33:17.627910+0000","last_unstale":"2024-09-14T06:33:36.039739+0000","last_undegraded":"2024-09-14T06:33:36.039739+0000","last_fullsized":"2024-09-14T06:33:36.039739+0000","mapping_epoch":271,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":272,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":874,"log_dups_size":0,"ondisk_log_size":874,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T14:22:28.835018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00074887399999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":5331,"num_read_kb":58329,"num_write":1221,"num_write_kb":47535,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":656,"num_bytes_recovered":26968802,"num_keys_recovered":45,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.b","version":"260'1637","reported_seq":8985,"reported_epoch":287,"state":"active+clean","last_fresh":"2024-09-14T06:33:32.818865+0000","last_change":"2024-09-14T06:33:07.769381+0000","last_active":"2024-09-14T06:33:32.818865+0000","last_peered":"2024-09-14T06:33:32.818865+0000","last_clean":"2024-09-14T06:33:32.818865+0000","last_became_active":"2024-09-14T06:31:55.756219+0000","last_became_peered":"2024-09-14T06:31:55.756219+0000","last_unstale":"2024-09-14T06:33:32.818865+0000","last_undegraded":"2024-09-14T06:33:32.818865+0000","last_fullsized":"2024-09-14T06:33:32.818865+0000","mapping_epoch":190,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":191,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":1637,"log_dups_size":0,"ondisk_log_size":1637,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T15:53:45.757678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065814200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":13043,"num_read_kb":64129,"num_write":2496,"num_write_kb":43679,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":291,"num_bytes_recovered":37370632,"num_keys_recovered":66,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.c","version":"255'622","reported_seq":6270,"reported_epoch":289,"state":"active+clean","last_fresh":"2024-09-14T06:33:35.112517+0000","last_change":"2024-09-14T06:33:31.824073+0000","last_active":"2024-09-14T06:33:35.112517+0000","last_peered":"2024-09-14T06:33:35.112517+0000","last_clean":"2024-09-14T06:33:35.112517+0000","last_became_active":"2024-09-14T06:33:31.821638+0000","last_became_peered":"2024-09-14T06:33:31.821638+0000","last_unstale":"2024-09-14T06:33:35.112517+0000","last_undegraded":"2024-09-14T06:33:35.112517+0000","last_fullsized":"2024-09-14T06:33:35.112517+0000","mapping_epoch":285,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":286,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":622,"log_dups_size":0,"ondisk_log_size":622,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T09:06:48.668360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013231740000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4842,"num_read_kb":50425,"num_write":795,"num_write_kb":40827,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":293,"num_bytes_recovered":36712421,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.d","version":"261'577","reported_seq":9328,"reported_epoch":289,"state":"active+clean","last_fresh":"2024-09-14T06:33:34.834667+0000","last_change":"2024-09-14T06:33:07.770219+0000","last_active":"2024-09-14T06:33:34.834667+0000","last_peered":"2024-09-14T06:33:34.834667+0000","last_clean":"2024-09-14T06:33:34.834667+0000","last_became_active":"2024-09-14T06:29:00.580362+0000","last_became_peered":"2024-09-14T06:29:00.580362+0000","last_unstale":"2024-09-14T06:33:34.834667+0000","last_undegraded":"2024-09-14T06:33:34.834667+0000","last_fullsized":"2024-09-14T06:33:34.834667+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":577,"log_dups_size":0,"ondisk_log_size":577,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T16:36:38.133044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010003270000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":8199,"num_read_kb":93694,"num_write":732,"num_write_kb":41032,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.e","version":"262'586","reported_seq":5526,"reported_epoch":289,"state":"active+clean","last_fresh":"2024-09-14T06:33:34.834589+0000","last_change":"2024-09-14T06:33:07.769825+0000","last_active":"2024-09-14T06:33:34.834589+0000","last_peered":"2024-09-14T06:33:34.834589+0000","last_clean":"2024-09-14T06:33:34.834589+0000","last_became_active":"2024-09-14T06:32:39.777524+0000","last_became_peered":"2024-09-14T06:32:39.777524+0000","last_unstale":"2024-09-14T06:33:34.834589+0000","last_undegraded":"2024-09-14T06:33:34.834589+0000","last_fullsized":"2024-09-14T06:33:34.834589+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":586,"log_dups_size":0,"ondisk_log_size":586,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T07:31:19.417437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065560200000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3986,"num_read_kb":61138,"num_write":753,"num_write_kb":43787,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":171,"num_bytes_recovered":10913227,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.f","version":"262'712","reported_seq":5948,"reported_epoch":289,"state":"active+clean","last_fresh":"2024-09-14T06:33:34.834579+0000","last_change":"2024-09-14T06:33:07.770138+0000","last_active":"2024-09-14T06:33:34.834579+0000","last_peered":"2024-09-14T06:33:34.834579+0000","last_clean":"2024-09-14T06:33:34.834579+0000","last_became_active":"2024-09-14T06:29:00.586491+0000","last_became_peered":"2024-09-14T06:29:00.586491+0000","last_unstale":"2024-09-14T06:33:34.834579+0000","last_undegraded":"2024-09-14T06:33:34.834579+0000","last_fullsized":"2024-09-14T06:33:34.834579+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":712,"log_dups_size":0,"ondisk_log_size":712,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T14:20:12.511497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000891002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":6081,"num_read_kb":45578,"num_write":836,"num_write_kb":36134,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.10","version":"261'952","reported_seq":6889,"reported_epoch":289,"state":"active+clean","last_fresh":"2024-09-14T06:33:36.039625+0000","last_change":"2024-09-14T06:33:16.630633+0000","last_active":"2024-09-14T06:33:36.039625+0000","last_peered":"2024-09-14T06:33:36.039625+0000","last_clean":"2024-09-14T06:33:36.039625+0000","last_became_active":"2024-09-14T06:33:16.628480+0000","last_became_peered":"2024-09-14T06:33:16.628480+0000","last_unstale":"2024-09-14T06:33:36.039625+0000","last_undegraded":"2024-09-14T06:33:36.039625+0000","last_fullsized":"2024-09-14T06:33:36.039625+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":952,"log_dups_size":0,"ondisk_log_size":952,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T17:09:51.353193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012025289999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":7147,"num_read_kb":34474,"num_write":1082,"num_write_kb":36655,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":318,"num_bytes_recovered":9401812,"num_keys_recovered":45,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.11","version":"261'730","reported_seq":11746,"reported_epoch":289,"state":"active+clean","last_fresh":"2024-09-14T06:33:34.834479+0000","last_change":"2024-09-14T06:33:07.769975+0000","last_active":"2024-09-14T06:33:34.834479+0000","last_peered":"2024-09-14T06:33:34.834479+0000","last_clean":"2024-09-14T06:33:34.834479+0000","last_became_active":"2024-09-14T06:32:39.778010+0000","last_became_peered":"2024-09-14T06:32:39.778010+0000","last_unstale":"2024-09-14T06:33:34.834479+0000","last_undegraded":"2024-09-14T06:33:34.834479+0000","last_fullsized":"2024-09-14T06:33:34.834479+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":730,"log_dups_size":0,"ondisk_log_size":730,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T10:48:00.264177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00072597299999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":10271,"num_read_kb":109243,"num_write":859,"num_write_kb":48307,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":177,"num_bytes_recovered":16647747,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.12","version":"261'782","reported_seq":7505,"reported_epoch":289,"state":"active+clean","last_fresh":"2024-09-14T06:33:34.979599+0000","last_change":"2024-09-14T06:33:07.706504+0000","last_active":"2024-09-14T06:33:34.979599+0000","last_peered":"2024-09-14T06:33:34.979599+0000","last_clean":"2024-09-14T06:33:34.979599+0000","last_became_active":"2024-09-14T06:29:00.579826+0000","last_became_peered":"2024-09-14T06:29:00.579826+0000","last_unstale":"2024-09-14T06:33:34.979599+0000","last_undegraded":"2024-09-14T06:33:34.979599+0000","last_fullsized":"2024-09-14T06:33:34.979599+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":782,"log_dups_size":0,"ondisk_log_size":782,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T11:32:57.816012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001680792,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":7110,"num_read_kb":59089,"num_write":947,"num_write_kb":42441,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.13","version":"261'1008","reported_seq":8959,"reported_epoch":289,"state":"active+clean","last_fresh":"2024-09-14T06:33:34.834443+0000","last_change":"2024-09-14T06:33:07.769558+0000","last_active":"2024-09-14T06:33:34.834443+0000","last_peered":"2024-09-14T06:33:34.834443+0000","last_clean":"2024-09-14T06:33:34.834443+0000","last_became_active":"2024-09-14T06:29:00.586520+0000","last_became_peered":"2024-09-14T06:29:00.586520+0000","last_unstale":"2024-09-14T06:33:34.834443+0000","last_undegraded":"2024-09-14T06:33:34.834443+0000","last_fullsized":"2024-09-14T06:33:34.834443+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":1008,"log_dups_size":0,"ondisk_log_size":1008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T07:25:04.666000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041539399999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":9551,"num_read_kb":66119,"num_write":1235,"num_write_kb":47437,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":19,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":240092,"num_read_kb":2184933,"num_write":38145,"num_write_kb":1478386,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7477,"num_bytes_recovered":638497624,"num_keys_recovered":546,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":28648,"ondisk_log_size":28648,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":131072,"data_stored":1180736,"data_compressed":65166,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509503,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":70776,"kb_used_data":3484,"kb_used_omap":1,"kb_used_meta":67262,"kb_avail":94301064,"statfs":{"total":96636764160,"available":96564289536,"internally_reserved":0,"allocated":3567616,"data_stored":2317757,"data_compressed":32583,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":1590,"internal_metadata":68876746},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509503,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":56816,"kb_used_data":3420,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94315024,"statfs":{"total":96636764160,"available":96578584576,"internally_reserved":0,"allocated":3502080,"data_stored":1727389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":54655430},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509504,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":74868,"kb_used_data":3424,"kb_used_omap":1,"kb_used_meta":71422,"kb_avail":94296972,"statfs":{"total":96636764160,"available":96560099328,"internally_reserved":0,"allocated":3506176,"data_stored":1727408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":73136581},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509503,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":88756,"kb_used_data":3424,"kb_used_omap":1,"kb_used_meta":85310,"kb_avail":94283084,"statfs":{"total":96636764160,"available":96545878016,"internally_reserved":0,"allocated":3506176,"data_stored":1727408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":87357893},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509504,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":56828,"kb_used_data":3428,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94315012,"statfs":{"total":96636764160,"available":96578572288,"internally_reserved":0,"allocated":3510272,"data_stored":1727427,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":14,"seq":60129542207,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":60976,"kb_used_data":3420,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94310864,"statfs":{"total":96636764160,"available":96574324736,"internally_reserved":0,"allocated":3502080,"data_stored":1727389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":58915270},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542206,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":83252,"kb_used_data":3484,"kb_used_omap":1,"kb_used_meta":79742,"kb_avail":94288588,"statfs":{"total":96636764160,"available":96551514112,"internally_reserved":0,"allocated":3567616,"data_stored":2317757,"data_compressed":32583,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":1595,"internal_metadata":81656261},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574911,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":83168,"kb_used_data":3396,"kb_used_omap":1,"kb_used_meta":79742,"kb_avail":94288672,"statfs":{"total":96636764160,"available":96551600128,"internally_reserved":0,"allocated":3477504,"data_stored":1715589,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":81656263},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32583,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32583,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-14T06:33:39.426 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-14T06:33:39.709 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-14T06:33:39.709 INFO:teuthology.orchestra.run.smithi094.stderr:dumped all 2024-09-14T06:33:39.721 INFO:teuthology.orchestra.run.smithi094.stdout:{"pg_ready":true,"pg_map":{"version":432,"stamp":"2024-09-14T06:33:39.336800+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590406,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":240158,"num_read_kb":2184988,"num_write":38248,"num_write_kb":1480016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7477,"num_bytes_recovered":638497624,"num_keys_recovered":546,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":28723,"ondisk_log_size":28723,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":575440,"kb_used_data":27480,"kb_used_omap":12,"kb_used_meta":547763,"kb_avail":754399280,"statfs":{"total":773094113280,"available":772504862720,"internally_reserved":0,"allocated":28139520,"data_stored":14988124,"data_compressed":65166,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":12751,"internal_metadata":560909873},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":-3,"num_read_kb":-3,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":-1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.516956"},"pg_stats":[{"pgid":"3.1f","version":"262'855","reported_seq":8431,"reported_epoch":289,"state":"active+clean","last_fresh":"2024-09-14T06:33:34.979756+0000","last_change":"2024-09-14T06:33:07.706447+0000","last_active":"2024-09-14T06:33:34.979756+0000","last_peered":"2024-09-14T06:33:34.979756+0000","last_clean":"2024-09-14T06:33:34.979756+0000","last_became_active":"2024-09-14T06:29:00.579883+0000","last_became_peered":"2024-09-14T06:29:00.579883+0000","last_unstale":"2024-09-14T06:33:34.979756+0000","last_undegraded":"2024-09-14T06:33:34.979756+0000","last_fullsized":"2024-09-14T06:33:34.979756+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":855,"log_dups_size":0,"ondisk_log_size":855,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T14:40:25.634308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001666641,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":8468,"num_read_kb":86898,"num_write":1171,"num_write_kb":49543,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"262'783","reported_seq":10851,"reported_epoch":287,"state":"active+clean","last_fresh":"2024-09-14T06:33:32.819008+0000","last_change":"2024-09-14T06:33:16.628665+0000","last_active":"2024-09-14T06:33:32.819008+0000","last_peered":"2024-09-14T06:33:32.819008+0000","last_clean":"2024-09-14T06:33:32.819008+0000","last_became_active":"2024-09-14T06:33:16.626222+0000","last_became_peered":"2024-09-14T06:33:16.626222+0000","last_unstale":"2024-09-14T06:33:32.819008+0000","last_undegraded":"2024-09-14T06:33:32.819008+0000","last_fullsized":"2024-09-14T06:33:32.819008+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":783,"log_dups_size":0,"ondisk_log_size":783,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T15:32:44.526538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013639839999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":8315,"num_read_kb":67657,"num_write":1025,"num_write_kb":40407,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1025,"num_bytes_recovered":87221212,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1d","version":"260'1077","reported_seq":7485,"reported_epoch":289,"state":"active+clean","last_fresh":"2024-09-14T06:33:35.112385+0000","last_change":"2024-09-14T06:33:31.826989+0000","last_active":"2024-09-14T06:33:35.112385+0000","last_peered":"2024-09-14T06:33:35.112385+0000","last_clean":"2024-09-14T06:33:35.112385+0000","last_became_active":"2024-09-14T06:33:31.824454+0000","last_became_peered":"2024-09-14T06:33:31.824454+0000","last_unstale":"2024-09-14T06:33:35.112385+0000","last_undegraded":"2024-09-14T06:33:35.112385+0000","last_fullsized":"2024-09-14T06:33:35.112385+0000","mapping_epoch":285,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":286,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":1077,"log_dups_size":0,"ondisk_log_size":1077,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T07:07:12.782032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088638400000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":7873,"num_read_kb":64375,"num_write":1555,"num_write_kb":53292,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1c","version":"262'805","reported_seq":6034,"reported_epoch":289,"state":"active+clean","last_fresh":"2024-09-14T06:33:35.112317+0000","last_change":"2024-09-14T06:33:31.825738+0000","last_active":"2024-09-14T06:33:35.112317+0000","last_peered":"2024-09-14T06:33:35.112317+0000","last_clean":"2024-09-14T06:33:35.112317+0000","last_became_active":"2024-09-14T06:33:31.823955+0000","last_became_peered":"2024-09-14T06:33:31.823955+0000","last_unstale":"2024-09-14T06:33:35.112317+0000","last_undegraded":"2024-09-14T06:33:35.112317+0000","last_fullsized":"2024-09-14T06:33:35.112317+0000","mapping_epoch":285,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":286,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":805,"log_dups_size":0,"ondisk_log_size":805,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T06:54:41.083735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00068758600000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":4953,"num_read_kb":43809,"num_write":946,"num_write_kb":45551,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":505,"num_bytes_recovered":44145197,"num_keys_recovered":280,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"259'960","reported_seq":11521,"reported_epoch":289,"state":"active+clean","last_fresh":"2024-09-14T06:33:34.979721+0000","last_change":"2024-09-14T06:33:07.706745+0000","last_active":"2024-09-14T06:33:34.979721+0000","last_peered":"2024-09-14T06:33:34.979721+0000","last_clean":"2024-09-14T06:33:34.979721+0000","last_became_active":"2024-09-14T06:29:00.586183+0000","last_became_peered":"2024-09-14T06:29:00.586183+0000","last_unstale":"2024-09-14T06:33:34.979721+0000","last_undegraded":"2024-09-14T06:33:34.979721+0000","last_fullsized":"2024-09-14T06:33:34.979721+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":960,"log_dups_size":0,"ondisk_log_size":960,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T10:55:09.317291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001810606,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":11910,"num_read_kb":108094,"num_write":1463,"num_write_kb":55102,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1a","version":"260'1678","reported_seq":10638,"reported_epoch":289,"state":"active+clean","last_fresh":"2024-09-14T06:33:36.039825+0000","last_change":"2024-09-14T06:33:07.705134+0000","last_active":"2024-09-14T06:33:36.039825+0000","last_peered":"2024-09-14T06:33:36.039825+0000","last_clean":"2024-09-14T06:33:36.039825+0000","last_became_active":"2024-09-14T06:29:00.576206+0000","last_became_peered":"2024-09-14T06:29:00.576206+0000","last_unstale":"2024-09-14T06:33:36.039825+0000","last_undegraded":"2024-09-14T06:33:36.039825+0000","last_fullsized":"2024-09-14T06:33:36.039825+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":1678,"log_dups_size":0,"ondisk_log_size":1678,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T10:47:38.190388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088286099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":13339,"num_read_kb":82904,"num_write":2451,"num_write_kb":55687,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.19","version":"261'561","reported_seq":6508,"reported_epoch":289,"state":"active+clean","last_fresh":"2024-09-14T06:33:34.831894+0000","last_change":"2024-09-14T06:33:07.704594+0000","last_active":"2024-09-14T06:33:34.831894+0000","last_peered":"2024-09-14T06:33:34.831894+0000","last_clean":"2024-09-14T06:33:34.831894+0000","last_became_active":"2024-09-14T06:31:53.744800+0000","last_became_peered":"2024-09-14T06:31:53.744800+0000","last_unstale":"2024-09-14T06:33:34.831894+0000","last_undegraded":"2024-09-14T06:33:34.831894+0000","last_fullsized":"2024-09-14T06:33:34.831894+0000","mapping_epoch":188,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":189,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":561,"log_dups_size":0,"ondisk_log_size":561,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T13:59:24.308993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062201200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":5123,"num_read_kb":67781,"num_write":764,"num_write_kb":43873,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":282,"num_bytes_recovered":41927489,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"262'547","reported_seq":6285,"reported_epoch":287,"state":"active+clean","last_fresh":"2024-09-14T06:33:32.818617+0000","last_change":"2024-09-14T06:33:07.769587+0000","last_active":"2024-09-14T06:33:32.818617+0000","last_peered":"2024-09-14T06:33:32.818617+0000","last_clean":"2024-09-14T06:33:32.818617+0000","last_became_active":"2024-09-14T06:31:53.745159+0000","last_became_peered":"2024-09-14T06:31:53.745159+0000","last_unstale":"2024-09-14T06:33:32.818617+0000","last_undegraded":"2024-09-14T06:33:32.818617+0000","last_fullsized":"2024-09-14T06:33:32.818617+0000","mapping_epoch":188,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":189,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":547,"log_dups_size":0,"ondisk_log_size":547,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T13:08:27.610323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00079631400000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4489,"num_read_kb":78917,"num_write":672,"num_write_kb":51581,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":310,"num_bytes_recovered":56082976,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"261'1037","reported_seq":9728,"reported_epoch":289,"state":"active+clean","last_fresh":"2024-09-14T06:33:34.979868+0000","last_change":"2024-09-14T06:33:07.706863+0000","last_active":"2024-09-14T06:33:34.979868+0000","last_peered":"2024-09-14T06:33:34.979868+0000","last_clean":"2024-09-14T06:33:34.979868+0000","last_became_active":"2024-09-14T06:29:00.579714+0000","last_became_peered":"2024-09-14T06:29:00.579714+0000","last_unstale":"2024-09-14T06:33:34.979868+0000","last_undegraded":"2024-09-14T06:33:34.979868+0000","last_fullsized":"2024-09-14T06:33:34.979868+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":1037,"log_dups_size":0,"ondisk_log_size":1037,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T15:17:51.610057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0017747889999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":10688,"num_read_kb":68964,"num_write":1162,"num_write_kb":48482,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.16","version":"255'1156","reported_seq":4137,"reported_epoch":289,"state":"active+clean","last_fresh":"2024-09-14T06:33:35.112491+0000","last_change":"2024-09-14T06:33:31.824083+0000","last_active":"2024-09-14T06:33:35.112491+0000","last_peered":"2024-09-14T06:33:35.112491+0000","last_clean":"2024-09-14T06:33:35.112491+0000","last_became_active":"2024-09-14T06:33:31.821620+0000","last_became_peered":"2024-09-14T06:33:31.821620+0000","last_unstale":"2024-09-14T06:33:35.112491+0000","last_undegraded":"2024-09-14T06:33:35.112491+0000","last_fullsized":"2024-09-14T06:33:35.112491+0000","mapping_epoch":285,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":286,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":1156,"log_dups_size":0,"ondisk_log_size":1156,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T08:10:02.587758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00073433000000000005,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":5658,"num_read_kb":38562,"num_write":1743,"num_write_kb":37248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"261'650","reported_seq":6751,"reported_epoch":289,"state":"active+clean","last_fresh":"2024-09-14T06:33:34.834516+0000","last_change":"2024-09-14T06:33:07.769581+0000","last_active":"2024-09-14T06:33:34.834516+0000","last_peered":"2024-09-14T06:33:34.834516+0000","last_clean":"2024-09-14T06:33:34.834516+0000","last_became_active":"2024-09-14T06:31:53.746091+0000","last_became_peered":"2024-09-14T06:31:53.746091+0000","last_unstale":"2024-09-14T06:33:34.834516+0000","last_undegraded":"2024-09-14T06:33:34.834516+0000","last_fullsized":"2024-09-14T06:33:34.834516+0000","mapping_epoch":188,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":189,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":650,"log_dups_size":0,"ondisk_log_size":650,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T12:36:31.690559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000442557,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":5101,"num_read_kb":74899,"num_write":696,"num_write_kb":51196,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":271,"num_bytes_recovered":49981473,"num_keys_recovered":33,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.14","version":"261'1137","reported_seq":6341,"reported_epoch":289,"state":"active+clean","last_fresh":"2024-09-14T06:33:36.039887+0000","last_change":"2024-09-14T06:33:07.704760+0000","last_active":"2024-09-14T06:33:36.039887+0000","last_peered":"2024-09-14T06:33:36.039887+0000","last_clean":"2024-09-14T06:33:36.039887+0000","last_became_active":"2024-09-14T06:33:04.628663+0000","last_became_peered":"2024-09-14T06:33:04.628663+0000","last_unstale":"2024-09-14T06:33:36.039887+0000","last_undegraded":"2024-09-14T06:33:36.039887+0000","last_fullsized":"2024-09-14T06:33:36.039887+0000","mapping_epoch":258,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":259,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":1137,"log_dups_size":0,"ondisk_log_size":1137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T11:42:47.162926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00078945399999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":8948,"num_read_kb":46936,"num_write":1573,"num_write_kb":29829,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":296,"num_bytes_recovered":6118769,"num_keys_recovered":11,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":50,"reported_epoch":289,"state":"active+clean","last_fresh":"2024-09-14T06:33:36.039662+0000","last_change":"2024-09-14T06:33:13.859355+0000","last_active":"2024-09-14T06:33:36.039662+0000","last_peered":"2024-09-14T06:33:36.039662+0000","last_clean":"2024-09-14T06:33:36.039662+0000","last_became_active":"2024-09-14T06:33:13.859069+0000","last_became_peered":"2024-09-14T06:33:13.859069+0000","last_unstale":"2024-09-14T06:33:36.039662+0000","last_undegraded":"2024-09-14T06:33:36.039662+0000","last_fullsized":"2024-09-14T06:33:36.039662+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:28:50.205350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:28:50.205350+0000","last_clean_scrub_stamp":"2024-09-14T06:28:50.205350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T17:08:35.416194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"262'893","reported_seq":8726,"reported_epoch":289,"state":"active+clean","last_fresh":"2024-09-14T06:33:34.979753+0000","last_change":"2024-09-14T06:33:07.705814+0000","last_active":"2024-09-14T06:33:34.979753+0000","last_peered":"2024-09-14T06:33:34.979753+0000","last_clean":"2024-09-14T06:33:34.979753+0000","last_became_active":"2024-09-14T06:29:00.578283+0000","last_became_peered":"2024-09-14T06:29:00.578283+0000","last_unstale":"2024-09-14T06:33:34.979753+0000","last_undegraded":"2024-09-14T06:33:34.979753+0000","last_fullsized":"2024-09-14T06:33:34.979753+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":893,"log_dups_size":0,"ondisk_log_size":893,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T15:09:37.855181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001076554,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":8608,"num_read_kb":78770,"num_write":923,"num_write_kb":48303,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":567,"reported_epoch":289,"state":"active+clean","last_fresh":"2024-09-14T06:33:34.831888+0000","last_change":"2024-09-14T06:33:13.734687+0000","last_active":"2024-09-14T06:33:34.831888+0000","last_peered":"2024-09-14T06:33:34.831888+0000","last_clean":"2024-09-14T06:33:34.831888+0000","last_became_active":"2024-09-14T06:33:13.734442+0000","last_became_peered":"2024-09-14T06:33:13.734442+0000","last_unstale":"2024-09-14T06:33:34.831888+0000","last_undegraded":"2024-09-14T06:33:34.831888+0000","last_fullsized":"2024-09-14T06:33:34.831888+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:28:50.205350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:28:50.205350+0000","last_clean_scrub_stamp":"2024-09-14T06:28:50.205350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T12:50:57.166258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065018200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"261'840","reported_seq":4649,"reported_epoch":287,"state":"active+clean","last_fresh":"2024-09-14T06:33:32.819115+0000","last_change":"2024-09-14T06:33:07.769882+0000","last_active":"2024-09-14T06:33:32.819115+0000","last_peered":"2024-09-14T06:33:32.819115+0000","last_clean":"2024-09-14T06:33:32.819115+0000","last_became_active":"2024-09-14T06:31:54.747923+0000","last_became_peered":"2024-09-14T06:31:54.747923+0000","last_unstale":"2024-09-14T06:33:32.819115+0000","last_undegraded":"2024-09-14T06:33:32.819115+0000","last_fullsized":"2024-09-14T06:33:32.819115+0000","mapping_epoch":189,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":190,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":840,"log_dups_size":0,"ondisk_log_size":840,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T09:01:49.581187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000927559,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4591,"num_read_kb":41129,"num_write":1178,"num_write_kb":36122,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":183,"num_bytes_recovered":13912513,"num_keys_recovered":22,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":554,"reported_epoch":289,"state":"active+clean","last_fresh":"2024-09-14T06:33:34.834701+0000","last_change":"2024-09-14T06:28:53.640035+0000","last_active":"2024-09-14T06:33:34.834701+0000","last_peered":"2024-09-14T06:33:34.834701+0000","last_clean":"2024-09-14T06:33:34.834701+0000","last_became_active":"2024-09-14T06:28:51.308058+0000","last_became_peered":"2024-09-14T06:28:51.308058+0000","last_unstale":"2024-09-14T06:33:34.834701+0000","last_undegraded":"2024-09-14T06:33:34.834701+0000","last_fullsized":"2024-09-14T06:33:34.834701+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:28:50.205350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:28:50.205350+0000","last_clean_scrub_stamp":"2024-09-14T06:28:50.205350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T13:35:48.544057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00037824000000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"262'700","reported_seq":4876,"reported_epoch":289,"state":"active+clean","last_fresh":"2024-09-14T06:33:34.832050+0000","last_change":"2024-09-14T06:33:22.769657+0000","last_active":"2024-09-14T06:33:34.832050+0000","last_peered":"2024-09-14T06:33:34.832050+0000","last_clean":"2024-09-14T06:33:34.832050+0000","last_became_active":"2024-09-14T06:33:22.767544+0000","last_became_peered":"2024-09-14T06:33:22.767544+0000","last_unstale":"2024-09-14T06:33:34.832050+0000","last_undegraded":"2024-09-14T06:33:34.832050+0000","last_fullsized":"2024-09-14T06:33:34.832050+0000","mapping_epoch":276,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":277,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":700,"log_dups_size":0,"ondisk_log_size":700,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T13:09:29.858808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00074004799999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":2654,"num_read_kb":37435,"num_write":897,"num_write_kb":40860,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":687,"num_bytes_recovered":31411166,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":555,"reported_epoch":289,"state":"active+clean","last_fresh":"2024-09-14T06:33:34.832080+0000","last_change":"2024-09-14T06:28:53.226072+0000","last_active":"2024-09-14T06:33:34.832080+0000","last_peered":"2024-09-14T06:33:34.832080+0000","last_clean":"2024-09-14T06:33:34.832080+0000","last_became_active":"2024-09-14T06:28:51.308072+0000","last_became_peered":"2024-09-14T06:28:51.308072+0000","last_unstale":"2024-09-14T06:33:34.832080+0000","last_undegraded":"2024-09-14T06:33:34.832080+0000","last_fullsized":"2024-09-14T06:33:34.832080+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:28:50.205350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:28:50.205350+0000","last_clean_scrub_stamp":"2024-09-14T06:28:50.205350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T13:15:09.052328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066197399999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"261'896","reported_seq":5625,"reported_epoch":289,"state":"active+clean","last_fresh":"2024-09-14T06:33:35.112603+0000","last_change":"2024-09-14T06:33:31.823904+0000","last_active":"2024-09-14T06:33:35.112603+0000","last_peered":"2024-09-14T06:33:35.112603+0000","last_clean":"2024-09-14T06:33:35.112603+0000","last_became_active":"2024-09-14T06:33:31.821533+0000","last_became_peered":"2024-09-14T06:33:31.821533+0000","last_unstale":"2024-09-14T06:33:35.112603+0000","last_undegraded":"2024-09-14T06:33:35.112603+0000","last_fullsized":"2024-09-14T06:33:35.112603+0000","mapping_epoch":285,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":286,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":896,"log_dups_size":0,"ondisk_log_size":896,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T15:48:46.733188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015088129999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4516,"num_read_kb":42632,"num_write":1255,"num_write_kb":49494,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":330,"num_bytes_recovered":28798893,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"262'711","reported_seq":10924,"reported_epoch":289,"state":"active+clean","last_fresh":"2024-09-14T06:33:36.040034+0000","last_change":"2024-09-14T06:33:07.704808+0000","last_active":"2024-09-14T06:33:36.040034+0000","last_peered":"2024-09-14T06:33:36.040034+0000","last_clean":"2024-09-14T06:33:36.040034+0000","last_became_active":"2024-09-14T06:29:00.584678+0000","last_became_peered":"2024-09-14T06:29:00.584678+0000","last_unstale":"2024-09-14T06:33:36.040034+0000","last_undegraded":"2024-09-14T06:33:36.040034+0000","last_fullsized":"2024-09-14T06:33:36.040034+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":711,"log_dups_size":0,"ondisk_log_size":711,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T07:39:08.849874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00074987800000000005,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":9560,"num_read_kb":89311,"num_write":777,"num_write_kb":51691,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"22'2","reported_seq":428,"reported_epoch":289,"state":"active+clean","last_fresh":"2024-09-14T06:33:35.112361+0000","last_change":"2024-09-14T06:33:31.823823+0000","last_active":"2024-09-14T06:33:35.112361+0000","last_peered":"2024-09-14T06:33:35.112361+0000","last_clean":"2024-09-14T06:33:35.112361+0000","last_became_active":"2024-09-14T06:33:31.823429+0000","last_became_peered":"2024-09-14T06:33:31.823429+0000","last_unstale":"2024-09-14T06:33:35.112361+0000","last_undegraded":"2024-09-14T06:33:35.112361+0000","last_fullsized":"2024-09-14T06:33:35.112361+0000","mapping_epoch":285,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":286,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:28:50.205350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:28:50.205350+0000","last_clean_scrub_stamp":"2024-09-14T06:28:50.205350+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T18:01:36.855427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010072970000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.0","version":"260'1116","reported_seq":8244,"reported_epoch":289,"state":"active+clean","last_fresh":"2024-09-14T06:33:34.832141+0000","last_change":"2024-09-14T06:33:22.769183+0000","last_active":"2024-09-14T06:33:34.832141+0000","last_peered":"2024-09-14T06:33:34.832141+0000","last_clean":"2024-09-14T06:33:34.832141+0000","last_became_active":"2024-09-14T06:33:22.767520+0000","last_became_peered":"2024-09-14T06:33:22.767520+0000","last_unstale":"2024-09-14T06:33:34.832141+0000","last_undegraded":"2024-09-14T06:33:34.832141+0000","last_fullsized":"2024-09-14T06:33:34.832141+0000","mapping_epoch":276,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":277,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":1116,"log_dups_size":0,"ondisk_log_size":1116,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T17:26:46.362756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064233900000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":8259,"num_read_kb":85480,"num_write":1652,"num_write_kb":58700,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":537,"num_bytes_recovered":2852,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":40,"reported_epoch":289,"state":"active+clean","last_fresh":"2024-09-14T06:33:34.831819+0000","last_change":"2024-09-14T06:33:19.801852+0000","last_active":"2024-09-14T06:33:34.831819+0000","last_peered":"2024-09-14T06:33:34.831819+0000","last_clean":"2024-09-14T06:33:34.831819+0000","last_became_active":"2024-09-14T06:33:19.801557+0000","last_became_peered":"2024-09-14T06:33:19.801557+0000","last_unstale":"2024-09-14T06:33:34.831819+0000","last_undegraded":"2024-09-14T06:33:34.831819+0000","last_fullsized":"2024-09-14T06:33:34.831819+0000","mapping_epoch":273,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":274,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:28:50.205350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:28:50.205350+0000","last_clean_scrub_stamp":"2024-09-14T06:28:50.205350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T08:41:03.294249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053403100000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1","version":"261'822","reported_seq":11886,"reported_epoch":289,"state":"active+clean","last_fresh":"2024-09-14T06:33:34.979564+0000","last_change":"2024-09-14T06:33:07.705685+0000","last_active":"2024-09-14T06:33:34.979564+0000","last_peered":"2024-09-14T06:33:34.979564+0000","last_clean":"2024-09-14T06:33:34.979564+0000","last_became_active":"2024-09-14T06:30:57.768238+0000","last_became_peered":"2024-09-14T06:30:57.768238+0000","last_unstale":"2024-09-14T06:33:34.979564+0000","last_undegraded":"2024-09-14T06:33:34.979564+0000","last_fullsized":"2024-09-14T06:33:34.979564+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":822,"log_dups_size":0,"ondisk_log_size":822,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T15:07:18.804416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011933110000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":11284,"num_read_kb":112430,"num_write":991,"num_write_kb":47408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":183,"num_bytes_recovered":30431994,"num_keys_recovered":22,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":554,"reported_epoch":289,"state":"active+clean","last_fresh":"2024-09-14T06:33:34.834417+0000","last_change":"2024-09-14T06:28:53.640096+0000","last_active":"2024-09-14T06:33:34.834417+0000","last_peered":"2024-09-14T06:33:34.834417+0000","last_clean":"2024-09-14T06:33:34.834417+0000","last_became_active":"2024-09-14T06:28:51.223069+0000","last_became_peered":"2024-09-14T06:28:51.223069+0000","last_unstale":"2024-09-14T06:33:34.834417+0000","last_undegraded":"2024-09-14T06:33:34.834417+0000","last_fullsized":"2024-09-14T06:33:34.834417+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:28:50.205350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:28:50.205350+0000","last_clean_scrub_stamp":"2024-09-14T06:28:50.205350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T15:13:00.471509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00043714600000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"262'1663","reported_seq":7850,"reported_epoch":287,"state":"active+clean","last_fresh":"2024-09-14T06:33:32.818891+0000","last_change":"2024-09-14T06:33:07.770369+0000","last_active":"2024-09-14T06:33:32.818891+0000","last_peered":"2024-09-14T06:33:32.818891+0000","last_clean":"2024-09-14T06:33:32.818891+0000","last_became_active":"2024-09-14T06:31:54.744160+0000","last_became_peered":"2024-09-14T06:31:54.744160+0000","last_unstale":"2024-09-14T06:33:32.818891+0000","last_undegraded":"2024-09-14T06:33:32.818891+0000","last_fullsized":"2024-09-14T06:33:32.818891+0000","mapping_epoch":189,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":190,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":1663,"log_dups_size":0,"ondisk_log_size":1663,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T16:44:35.569843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001382608,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":10723,"num_read_kb":70589,"num_write":2596,"num_write_kb":53564,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":327,"num_bytes_recovered":50047063,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"20'1","reported_seq":427,"reported_epoch":289,"state":"active+clean","last_fresh":"2024-09-14T06:33:35.112503+0000","last_change":"2024-09-14T06:33:31.823426+0000","last_active":"2024-09-14T06:33:35.112503+0000","last_peered":"2024-09-14T06:33:35.112503+0000","last_clean":"2024-09-14T06:33:35.112503+0000","last_became_active":"2024-09-14T06:33:31.823032+0000","last_became_peered":"2024-09-14T06:33:31.823032+0000","last_unstale":"2024-09-14T06:33:35.112503+0000","last_undegraded":"2024-09-14T06:33:35.112503+0000","last_fullsized":"2024-09-14T06:33:35.112503+0000","mapping_epoch":285,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":286,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:28:50.205350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:28:50.205350+0000","last_clean_scrub_stamp":"2024-09-14T06:28:50.205350+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T08:31:58.917801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050808899999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'72","reported_seq":645,"reported_epoch":289,"state":"active+clean","last_fresh":"2024-09-14T06:33:34.834414+0000","last_change":"2024-09-14T06:28:44.839885+0000","last_active":"2024-09-14T06:33:34.834414+0000","last_peered":"2024-09-14T06:33:34.834414+0000","last_clean":"2024-09-14T06:33:34.834414+0000","last_became_active":"2024-09-14T06:28:44.839573+0000","last_became_peered":"2024-09-14T06:28:44.839573+0000","last_unstale":"2024-09-14T06:33:34.834414+0000","last_undegraded":"2024-09-14T06:33:34.834414+0000","last_fullsized":"2024-09-14T06:33:34.834414+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:28:43.828494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:28:43.828494+0000","last_clean_scrub_stamp":"2024-09-14T06:28:43.828494+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T14:41:25.787293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.8","version":"261'684","reported_seq":6657,"reported_epoch":287,"state":"active+clean","last_fresh":"2024-09-14T06:33:32.818645+0000","last_change":"2024-09-14T06:33:07.769379+0000","last_active":"2024-09-14T06:33:32.818645+0000","last_peered":"2024-09-14T06:33:32.818645+0000","last_clean":"2024-09-14T06:33:32.818645+0000","last_became_active":"2024-09-14T06:31:55.754694+0000","last_became_peered":"2024-09-14T06:31:55.754694+0000","last_unstale":"2024-09-14T06:33:32.818645+0000","last_undegraded":"2024-09-14T06:33:32.818645+0000","last_fullsized":"2024-09-14T06:33:32.818645+0000","mapping_epoch":190,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":191,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":684,"log_dups_size":0,"ondisk_log_size":684,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T15:48:47.633183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067851599999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":5793,"num_read_kb":90852,"num_write":921,"num_write_kb":57691,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":319,"num_bytes_recovered":60352269,"num_keys_recovered":22,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.9","version":"261'597","reported_seq":5173,"reported_epoch":289,"state":"active+clean","last_fresh":"2024-09-14T06:33:36.040020+0000","last_change":"2024-09-14T06:33:22.772981+0000","last_active":"2024-09-14T06:33:36.040020+0000","last_peered":"2024-09-14T06:33:36.040020+0000","last_clean":"2024-09-14T06:33:36.040020+0000","last_became_active":"2024-09-14T06:33:22.770956+0000","last_became_peered":"2024-09-14T06:33:22.770956+0000","last_unstale":"2024-09-14T06:33:36.040020+0000","last_undegraded":"2024-09-14T06:33:36.040020+0000","last_fullsized":"2024-09-14T06:33:36.040020+0000","mapping_epoch":276,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":277,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":597,"log_dups_size":0,"ondisk_log_size":597,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T17:14:06.174583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057139500000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3678,"num_read_kb":64291,"num_write":778,"num_write_kb":44928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":316,"num_bytes_recovered":49117,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"261'874","reported_seq":5176,"reported_epoch":289,"state":"active+clean","last_fresh":"2024-09-14T06:33:36.039739+0000","last_change":"2024-09-14T06:33:17.629900+0000","last_active":"2024-09-14T06:33:36.039739+0000","last_peered":"2024-09-14T06:33:36.039739+0000","last_clean":"2024-09-14T06:33:36.039739+0000","last_became_active":"2024-09-14T06:33:17.627910+0000","last_became_peered":"2024-09-14T06:33:17.627910+0000","last_unstale":"2024-09-14T06:33:36.039739+0000","last_undegraded":"2024-09-14T06:33:36.039739+0000","last_fullsized":"2024-09-14T06:33:36.039739+0000","mapping_epoch":271,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":272,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":874,"log_dups_size":0,"ondisk_log_size":874,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T14:22:28.835018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00074887399999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":5331,"num_read_kb":58329,"num_write":1221,"num_write_kb":47535,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":656,"num_bytes_recovered":26968802,"num_keys_recovered":45,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.b","version":"260'1637","reported_seq":8985,"reported_epoch":287,"state":"active+clean","last_fresh":"2024-09-14T06:33:32.818865+0000","last_change":"2024-09-14T06:33:07.769381+0000","last_active":"2024-09-14T06:33:32.818865+0000","last_peered":"2024-09-14T06:33:32.818865+0000","last_clean":"2024-09-14T06:33:32.818865+0000","last_became_active":"2024-09-14T06:31:55.756219+0000","last_became_peered":"2024-09-14T06:31:55.756219+0000","last_unstale":"2024-09-14T06:33:32.818865+0000","last_undegraded":"2024-09-14T06:33:32.818865+0000","last_fullsized":"2024-09-14T06:33:32.818865+0000","mapping_epoch":190,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":191,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":1637,"log_dups_size":0,"ondisk_log_size":1637,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T15:53:45.757678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065814200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":13043,"num_read_kb":64129,"num_write":2496,"num_write_kb":43679,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":291,"num_bytes_recovered":37370632,"num_keys_recovered":66,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.c","version":"255'622","reported_seq":6270,"reported_epoch":289,"state":"active+clean","last_fresh":"2024-09-14T06:33:35.112517+0000","last_change":"2024-09-14T06:33:31.824073+0000","last_active":"2024-09-14T06:33:35.112517+0000","last_peered":"2024-09-14T06:33:35.112517+0000","last_clean":"2024-09-14T06:33:35.112517+0000","last_became_active":"2024-09-14T06:33:31.821638+0000","last_became_peered":"2024-09-14T06:33:31.821638+0000","last_unstale":"2024-09-14T06:33:35.112517+0000","last_undegraded":"2024-09-14T06:33:35.112517+0000","last_fullsized":"2024-09-14T06:33:35.112517+0000","mapping_epoch":285,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":286,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":622,"log_dups_size":0,"ondisk_log_size":622,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T09:06:48.668360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013231740000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4842,"num_read_kb":50425,"num_write":795,"num_write_kb":40827,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":293,"num_bytes_recovered":36712421,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.d","version":"261'577","reported_seq":9328,"reported_epoch":289,"state":"active+clean","last_fresh":"2024-09-14T06:33:34.834667+0000","last_change":"2024-09-14T06:33:07.770219+0000","last_active":"2024-09-14T06:33:34.834667+0000","last_peered":"2024-09-14T06:33:34.834667+0000","last_clean":"2024-09-14T06:33:34.834667+0000","last_became_active":"2024-09-14T06:29:00.580362+0000","last_became_peered":"2024-09-14T06:29:00.580362+0000","last_unstale":"2024-09-14T06:33:34.834667+0000","last_undegraded":"2024-09-14T06:33:34.834667+0000","last_fullsized":"2024-09-14T06:33:34.834667+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":577,"log_dups_size":0,"ondisk_log_size":577,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T16:36:38.133044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010003270000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":8199,"num_read_kb":93694,"num_write":732,"num_write_kb":41032,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.e","version":"262'586","reported_seq":5526,"reported_epoch":289,"state":"active+clean","last_fresh":"2024-09-14T06:33:34.834589+0000","last_change":"2024-09-14T06:33:07.769825+0000","last_active":"2024-09-14T06:33:34.834589+0000","last_peered":"2024-09-14T06:33:34.834589+0000","last_clean":"2024-09-14T06:33:34.834589+0000","last_became_active":"2024-09-14T06:32:39.777524+0000","last_became_peered":"2024-09-14T06:32:39.777524+0000","last_unstale":"2024-09-14T06:33:34.834589+0000","last_undegraded":"2024-09-14T06:33:34.834589+0000","last_fullsized":"2024-09-14T06:33:34.834589+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":586,"log_dups_size":0,"ondisk_log_size":586,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T07:31:19.417437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065560200000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3986,"num_read_kb":61138,"num_write":753,"num_write_kb":43787,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":171,"num_bytes_recovered":10913227,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.f","version":"262'712","reported_seq":5948,"reported_epoch":289,"state":"active+clean","last_fresh":"2024-09-14T06:33:34.834579+0000","last_change":"2024-09-14T06:33:07.770138+0000","last_active":"2024-09-14T06:33:34.834579+0000","last_peered":"2024-09-14T06:33:34.834579+0000","last_clean":"2024-09-14T06:33:34.834579+0000","last_became_active":"2024-09-14T06:29:00.586491+0000","last_became_peered":"2024-09-14T06:29:00.586491+0000","last_unstale":"2024-09-14T06:33:34.834579+0000","last_undegraded":"2024-09-14T06:33:34.834579+0000","last_fullsized":"2024-09-14T06:33:34.834579+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":712,"log_dups_size":0,"ondisk_log_size":712,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T14:20:12.511497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000891002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":6081,"num_read_kb":45578,"num_write":836,"num_write_kb":36134,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.10","version":"261'952","reported_seq":6889,"reported_epoch":289,"state":"active+clean","last_fresh":"2024-09-14T06:33:36.039625+0000","last_change":"2024-09-14T06:33:16.630633+0000","last_active":"2024-09-14T06:33:36.039625+0000","last_peered":"2024-09-14T06:33:36.039625+0000","last_clean":"2024-09-14T06:33:36.039625+0000","last_became_active":"2024-09-14T06:33:16.628480+0000","last_became_peered":"2024-09-14T06:33:16.628480+0000","last_unstale":"2024-09-14T06:33:36.039625+0000","last_undegraded":"2024-09-14T06:33:36.039625+0000","last_fullsized":"2024-09-14T06:33:36.039625+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":952,"log_dups_size":0,"ondisk_log_size":952,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T17:09:51.353193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012025289999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":7147,"num_read_kb":34474,"num_write":1082,"num_write_kb":36655,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":318,"num_bytes_recovered":9401812,"num_keys_recovered":45,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.11","version":"261'730","reported_seq":11746,"reported_epoch":289,"state":"active+clean","last_fresh":"2024-09-14T06:33:34.834479+0000","last_change":"2024-09-14T06:33:07.769975+0000","last_active":"2024-09-14T06:33:34.834479+0000","last_peered":"2024-09-14T06:33:34.834479+0000","last_clean":"2024-09-14T06:33:34.834479+0000","last_became_active":"2024-09-14T06:32:39.778010+0000","last_became_peered":"2024-09-14T06:32:39.778010+0000","last_unstale":"2024-09-14T06:33:34.834479+0000","last_undegraded":"2024-09-14T06:33:34.834479+0000","last_fullsized":"2024-09-14T06:33:34.834479+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":730,"log_dups_size":0,"ondisk_log_size":730,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T10:48:00.264177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00072597299999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":10271,"num_read_kb":109243,"num_write":859,"num_write_kb":48307,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":177,"num_bytes_recovered":16647747,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.12","version":"261'782","reported_seq":7505,"reported_epoch":289,"state":"active+clean","last_fresh":"2024-09-14T06:33:34.979599+0000","last_change":"2024-09-14T06:33:07.706504+0000","last_active":"2024-09-14T06:33:34.979599+0000","last_peered":"2024-09-14T06:33:34.979599+0000","last_clean":"2024-09-14T06:33:34.979599+0000","last_became_active":"2024-09-14T06:29:00.579826+0000","last_became_peered":"2024-09-14T06:29:00.579826+0000","last_unstale":"2024-09-14T06:33:34.979599+0000","last_undegraded":"2024-09-14T06:33:34.979599+0000","last_fullsized":"2024-09-14T06:33:34.979599+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":782,"log_dups_size":0,"ondisk_log_size":782,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T11:32:57.816012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001680792,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":7110,"num_read_kb":59089,"num_write":947,"num_write_kb":42441,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.13","version":"261'1008","reported_seq":8959,"reported_epoch":289,"state":"active+clean","last_fresh":"2024-09-14T06:33:34.834443+0000","last_change":"2024-09-14T06:33:07.769558+0000","last_active":"2024-09-14T06:33:34.834443+0000","last_peered":"2024-09-14T06:33:34.834443+0000","last_clean":"2024-09-14T06:33:34.834443+0000","last_became_active":"2024-09-14T06:29:00.586520+0000","last_became_peered":"2024-09-14T06:29:00.586520+0000","last_unstale":"2024-09-14T06:33:34.834443+0000","last_undegraded":"2024-09-14T06:33:34.834443+0000","last_fullsized":"2024-09-14T06:33:34.834443+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":1008,"log_dups_size":0,"ondisk_log_size":1008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T07:25:04.666000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041539399999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":9551,"num_read_kb":66119,"num_write":1235,"num_write_kb":47437,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":19,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":240092,"num_read_kb":2184933,"num_write":38145,"num_write_kb":1478386,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7477,"num_bytes_recovered":638497624,"num_keys_recovered":546,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":28648,"ondisk_log_size":28648,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":131072,"data_stored":1180736,"data_compressed":65166,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509503,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":70776,"kb_used_data":3484,"kb_used_omap":1,"kb_used_meta":67262,"kb_avail":94301064,"statfs":{"total":96636764160,"available":96564289536,"internally_reserved":0,"allocated":3567616,"data_stored":2317757,"data_compressed":32583,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":1590,"internal_metadata":68876746},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509503,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":56816,"kb_used_data":3420,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94315024,"statfs":{"total":96636764160,"available":96578584576,"internally_reserved":0,"allocated":3502080,"data_stored":1727389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":54655430},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509504,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":74868,"kb_used_data":3424,"kb_used_omap":1,"kb_used_meta":71422,"kb_avail":94296972,"statfs":{"total":96636764160,"available":96560099328,"internally_reserved":0,"allocated":3506176,"data_stored":1727408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":73136581},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509503,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":88756,"kb_used_data":3424,"kb_used_omap":1,"kb_used_meta":85310,"kb_avail":94283084,"statfs":{"total":96636764160,"available":96545878016,"internally_reserved":0,"allocated":3506176,"data_stored":1727408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":87357893},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509504,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":56828,"kb_used_data":3428,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94315012,"statfs":{"total":96636764160,"available":96578572288,"internally_reserved":0,"allocated":3510272,"data_stored":1727427,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":14,"seq":60129542207,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":60976,"kb_used_data":3420,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94310864,"statfs":{"total":96636764160,"available":96574324736,"internally_reserved":0,"allocated":3502080,"data_stored":1727389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":58915270},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542206,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":83252,"kb_used_data":3484,"kb_used_omap":1,"kb_used_meta":79742,"kb_avail":94288588,"statfs":{"total":96636764160,"available":96551514112,"internally_reserved":0,"allocated":3567616,"data_stored":2317757,"data_compressed":32583,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":1595,"internal_metadata":81656261},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574911,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":83168,"kb_used_data":3396,"kb_used_omap":1,"kb_used_meta":79742,"kb_avail":94288672,"statfs":{"total":96636764160,"available":96551600128,"internally_reserved":0,"allocated":3477504,"data_stored":1715589,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":81656263},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32583,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32583,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-14T06:33:39.723 INFO:tasks.ceph.ceph_manager.ceph:recovered! 2024-09-14T06:33:39.723 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mgr mgr_debug_aggressive_pg_num_changes 2024-09-14T06:33:40.028 DEBUG:teuthology.run_tasks:Unwinding manager ceph 2024-09-14T06:33:40.037 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-14T06:33:40.037 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-14T06:33:40.322 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-14T06:33:40.323 INFO:teuthology.orchestra.run.smithi094.stderr:dumped all 2024-09-14T06:33:40.335 INFO:teuthology.orchestra.run.smithi094.stdout:{"pg_ready":true,"pg_map":{"version":432,"stamp":"2024-09-14T06:33:39.336800+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590406,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":240158,"num_read_kb":2184988,"num_write":38248,"num_write_kb":1480016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7477,"num_bytes_recovered":638497624,"num_keys_recovered":546,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":28723,"ondisk_log_size":28723,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":575440,"kb_used_data":27480,"kb_used_omap":12,"kb_used_meta":547763,"kb_avail":754399280,"statfs":{"total":773094113280,"available":772504862720,"internally_reserved":0,"allocated":28139520,"data_stored":14988124,"data_compressed":65166,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":12751,"internal_metadata":560909873},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":-3,"num_read_kb":-3,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":-1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.516956"},"pg_stats":[{"pgid":"3.1f","version":"262'855","reported_seq":8431,"reported_epoch":289,"state":"active+clean","last_fresh":"2024-09-14T06:33:34.979756+0000","last_change":"2024-09-14T06:33:07.706447+0000","last_active":"2024-09-14T06:33:34.979756+0000","last_peered":"2024-09-14T06:33:34.979756+0000","last_clean":"2024-09-14T06:33:34.979756+0000","last_became_active":"2024-09-14T06:29:00.579883+0000","last_became_peered":"2024-09-14T06:29:00.579883+0000","last_unstale":"2024-09-14T06:33:34.979756+0000","last_undegraded":"2024-09-14T06:33:34.979756+0000","last_fullsized":"2024-09-14T06:33:34.979756+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":855,"log_dups_size":0,"ondisk_log_size":855,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T14:40:25.634308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001666641,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":8468,"num_read_kb":86898,"num_write":1171,"num_write_kb":49543,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"262'783","reported_seq":10851,"reported_epoch":287,"state":"active+clean","last_fresh":"2024-09-14T06:33:32.819008+0000","last_change":"2024-09-14T06:33:16.628665+0000","last_active":"2024-09-14T06:33:32.819008+0000","last_peered":"2024-09-14T06:33:32.819008+0000","last_clean":"2024-09-14T06:33:32.819008+0000","last_became_active":"2024-09-14T06:33:16.626222+0000","last_became_peered":"2024-09-14T06:33:16.626222+0000","last_unstale":"2024-09-14T06:33:32.819008+0000","last_undegraded":"2024-09-14T06:33:32.819008+0000","last_fullsized":"2024-09-14T06:33:32.819008+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":783,"log_dups_size":0,"ondisk_log_size":783,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T15:32:44.526538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013639839999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":8315,"num_read_kb":67657,"num_write":1025,"num_write_kb":40407,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1025,"num_bytes_recovered":87221212,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1d","version":"260'1077","reported_seq":7485,"reported_epoch":289,"state":"active+clean","last_fresh":"2024-09-14T06:33:35.112385+0000","last_change":"2024-09-14T06:33:31.826989+0000","last_active":"2024-09-14T06:33:35.112385+0000","last_peered":"2024-09-14T06:33:35.112385+0000","last_clean":"2024-09-14T06:33:35.112385+0000","last_became_active":"2024-09-14T06:33:31.824454+0000","last_became_peered":"2024-09-14T06:33:31.824454+0000","last_unstale":"2024-09-14T06:33:35.112385+0000","last_undegraded":"2024-09-14T06:33:35.112385+0000","last_fullsized":"2024-09-14T06:33:35.112385+0000","mapping_epoch":285,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":286,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":1077,"log_dups_size":0,"ondisk_log_size":1077,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T07:07:12.782032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088638400000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":7873,"num_read_kb":64375,"num_write":1555,"num_write_kb":53292,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1c","version":"262'805","reported_seq":6034,"reported_epoch":289,"state":"active+clean","last_fresh":"2024-09-14T06:33:35.112317+0000","last_change":"2024-09-14T06:33:31.825738+0000","last_active":"2024-09-14T06:33:35.112317+0000","last_peered":"2024-09-14T06:33:35.112317+0000","last_clean":"2024-09-14T06:33:35.112317+0000","last_became_active":"2024-09-14T06:33:31.823955+0000","last_became_peered":"2024-09-14T06:33:31.823955+0000","last_unstale":"2024-09-14T06:33:35.112317+0000","last_undegraded":"2024-09-14T06:33:35.112317+0000","last_fullsized":"2024-09-14T06:33:35.112317+0000","mapping_epoch":285,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":286,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":805,"log_dups_size":0,"ondisk_log_size":805,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T06:54:41.083735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00068758600000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":4953,"num_read_kb":43809,"num_write":946,"num_write_kb":45551,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":505,"num_bytes_recovered":44145197,"num_keys_recovered":280,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"259'960","reported_seq":11521,"reported_epoch":289,"state":"active+clean","last_fresh":"2024-09-14T06:33:34.979721+0000","last_change":"2024-09-14T06:33:07.706745+0000","last_active":"2024-09-14T06:33:34.979721+0000","last_peered":"2024-09-14T06:33:34.979721+0000","last_clean":"2024-09-14T06:33:34.979721+0000","last_became_active":"2024-09-14T06:29:00.586183+0000","last_became_peered":"2024-09-14T06:29:00.586183+0000","last_unstale":"2024-09-14T06:33:34.979721+0000","last_undegraded":"2024-09-14T06:33:34.979721+0000","last_fullsized":"2024-09-14T06:33:34.979721+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":960,"log_dups_size":0,"ondisk_log_size":960,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T10:55:09.317291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001810606,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":11910,"num_read_kb":108094,"num_write":1463,"num_write_kb":55102,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1a","version":"260'1678","reported_seq":10638,"reported_epoch":289,"state":"active+clean","last_fresh":"2024-09-14T06:33:36.039825+0000","last_change":"2024-09-14T06:33:07.705134+0000","last_active":"2024-09-14T06:33:36.039825+0000","last_peered":"2024-09-14T06:33:36.039825+0000","last_clean":"2024-09-14T06:33:36.039825+0000","last_became_active":"2024-09-14T06:29:00.576206+0000","last_became_peered":"2024-09-14T06:29:00.576206+0000","last_unstale":"2024-09-14T06:33:36.039825+0000","last_undegraded":"2024-09-14T06:33:36.039825+0000","last_fullsized":"2024-09-14T06:33:36.039825+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":1678,"log_dups_size":0,"ondisk_log_size":1678,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T10:47:38.190388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088286099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":13339,"num_read_kb":82904,"num_write":2451,"num_write_kb":55687,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.19","version":"261'561","reported_seq":6508,"reported_epoch":289,"state":"active+clean","last_fresh":"2024-09-14T06:33:34.831894+0000","last_change":"2024-09-14T06:33:07.704594+0000","last_active":"2024-09-14T06:33:34.831894+0000","last_peered":"2024-09-14T06:33:34.831894+0000","last_clean":"2024-09-14T06:33:34.831894+0000","last_became_active":"2024-09-14T06:31:53.744800+0000","last_became_peered":"2024-09-14T06:31:53.744800+0000","last_unstale":"2024-09-14T06:33:34.831894+0000","last_undegraded":"2024-09-14T06:33:34.831894+0000","last_fullsized":"2024-09-14T06:33:34.831894+0000","mapping_epoch":188,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":189,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":561,"log_dups_size":0,"ondisk_log_size":561,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T13:59:24.308993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062201200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":5123,"num_read_kb":67781,"num_write":764,"num_write_kb":43873,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":282,"num_bytes_recovered":41927489,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"262'547","reported_seq":6285,"reported_epoch":287,"state":"active+clean","last_fresh":"2024-09-14T06:33:32.818617+0000","last_change":"2024-09-14T06:33:07.769587+0000","last_active":"2024-09-14T06:33:32.818617+0000","last_peered":"2024-09-14T06:33:32.818617+0000","last_clean":"2024-09-14T06:33:32.818617+0000","last_became_active":"2024-09-14T06:31:53.745159+0000","last_became_peered":"2024-09-14T06:31:53.745159+0000","last_unstale":"2024-09-14T06:33:32.818617+0000","last_undegraded":"2024-09-14T06:33:32.818617+0000","last_fullsized":"2024-09-14T06:33:32.818617+0000","mapping_epoch":188,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":189,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":547,"log_dups_size":0,"ondisk_log_size":547,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T13:08:27.610323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00079631400000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4489,"num_read_kb":78917,"num_write":672,"num_write_kb":51581,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":310,"num_bytes_recovered":56082976,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"261'1037","reported_seq":9728,"reported_epoch":289,"state":"active+clean","last_fresh":"2024-09-14T06:33:34.979868+0000","last_change":"2024-09-14T06:33:07.706863+0000","last_active":"2024-09-14T06:33:34.979868+0000","last_peered":"2024-09-14T06:33:34.979868+0000","last_clean":"2024-09-14T06:33:34.979868+0000","last_became_active":"2024-09-14T06:29:00.579714+0000","last_became_peered":"2024-09-14T06:29:00.579714+0000","last_unstale":"2024-09-14T06:33:34.979868+0000","last_undegraded":"2024-09-14T06:33:34.979868+0000","last_fullsized":"2024-09-14T06:33:34.979868+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":1037,"log_dups_size":0,"ondisk_log_size":1037,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T15:17:51.610057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0017747889999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":10688,"num_read_kb":68964,"num_write":1162,"num_write_kb":48482,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.16","version":"255'1156","reported_seq":4137,"reported_epoch":289,"state":"active+clean","last_fresh":"2024-09-14T06:33:35.112491+0000","last_change":"2024-09-14T06:33:31.824083+0000","last_active":"2024-09-14T06:33:35.112491+0000","last_peered":"2024-09-14T06:33:35.112491+0000","last_clean":"2024-09-14T06:33:35.112491+0000","last_became_active":"2024-09-14T06:33:31.821620+0000","last_became_peered":"2024-09-14T06:33:31.821620+0000","last_unstale":"2024-09-14T06:33:35.112491+0000","last_undegraded":"2024-09-14T06:33:35.112491+0000","last_fullsized":"2024-09-14T06:33:35.112491+0000","mapping_epoch":285,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":286,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":1156,"log_dups_size":0,"ondisk_log_size":1156,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T08:10:02.587758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00073433000000000005,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":5658,"num_read_kb":38562,"num_write":1743,"num_write_kb":37248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"261'650","reported_seq":6751,"reported_epoch":289,"state":"active+clean","last_fresh":"2024-09-14T06:33:34.834516+0000","last_change":"2024-09-14T06:33:07.769581+0000","last_active":"2024-09-14T06:33:34.834516+0000","last_peered":"2024-09-14T06:33:34.834516+0000","last_clean":"2024-09-14T06:33:34.834516+0000","last_became_active":"2024-09-14T06:31:53.746091+0000","last_became_peered":"2024-09-14T06:31:53.746091+0000","last_unstale":"2024-09-14T06:33:34.834516+0000","last_undegraded":"2024-09-14T06:33:34.834516+0000","last_fullsized":"2024-09-14T06:33:34.834516+0000","mapping_epoch":188,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":189,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":650,"log_dups_size":0,"ondisk_log_size":650,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T12:36:31.690559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000442557,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":5101,"num_read_kb":74899,"num_write":696,"num_write_kb":51196,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":271,"num_bytes_recovered":49981473,"num_keys_recovered":33,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.14","version":"261'1137","reported_seq":6341,"reported_epoch":289,"state":"active+clean","last_fresh":"2024-09-14T06:33:36.039887+0000","last_change":"2024-09-14T06:33:07.704760+0000","last_active":"2024-09-14T06:33:36.039887+0000","last_peered":"2024-09-14T06:33:36.039887+0000","last_clean":"2024-09-14T06:33:36.039887+0000","last_became_active":"2024-09-14T06:33:04.628663+0000","last_became_peered":"2024-09-14T06:33:04.628663+0000","last_unstale":"2024-09-14T06:33:36.039887+0000","last_undegraded":"2024-09-14T06:33:36.039887+0000","last_fullsized":"2024-09-14T06:33:36.039887+0000","mapping_epoch":258,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":259,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":1137,"log_dups_size":0,"ondisk_log_size":1137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T11:42:47.162926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00078945399999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":8948,"num_read_kb":46936,"num_write":1573,"num_write_kb":29829,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":296,"num_bytes_recovered":6118769,"num_keys_recovered":11,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":50,"reported_epoch":289,"state":"active+clean","last_fresh":"2024-09-14T06:33:36.039662+0000","last_change":"2024-09-14T06:33:13.859355+0000","last_active":"2024-09-14T06:33:36.039662+0000","last_peered":"2024-09-14T06:33:36.039662+0000","last_clean":"2024-09-14T06:33:36.039662+0000","last_became_active":"2024-09-14T06:33:13.859069+0000","last_became_peered":"2024-09-14T06:33:13.859069+0000","last_unstale":"2024-09-14T06:33:36.039662+0000","last_undegraded":"2024-09-14T06:33:36.039662+0000","last_fullsized":"2024-09-14T06:33:36.039662+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:28:50.205350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:28:50.205350+0000","last_clean_scrub_stamp":"2024-09-14T06:28:50.205350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T17:08:35.416194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"262'893","reported_seq":8726,"reported_epoch":289,"state":"active+clean","last_fresh":"2024-09-14T06:33:34.979753+0000","last_change":"2024-09-14T06:33:07.705814+0000","last_active":"2024-09-14T06:33:34.979753+0000","last_peered":"2024-09-14T06:33:34.979753+0000","last_clean":"2024-09-14T06:33:34.979753+0000","last_became_active":"2024-09-14T06:29:00.578283+0000","last_became_peered":"2024-09-14T06:29:00.578283+0000","last_unstale":"2024-09-14T06:33:34.979753+0000","last_undegraded":"2024-09-14T06:33:34.979753+0000","last_fullsized":"2024-09-14T06:33:34.979753+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":893,"log_dups_size":0,"ondisk_log_size":893,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T15:09:37.855181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001076554,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":8608,"num_read_kb":78770,"num_write":923,"num_write_kb":48303,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":567,"reported_epoch":289,"state":"active+clean","last_fresh":"2024-09-14T06:33:34.831888+0000","last_change":"2024-09-14T06:33:13.734687+0000","last_active":"2024-09-14T06:33:34.831888+0000","last_peered":"2024-09-14T06:33:34.831888+0000","last_clean":"2024-09-14T06:33:34.831888+0000","last_became_active":"2024-09-14T06:33:13.734442+0000","last_became_peered":"2024-09-14T06:33:13.734442+0000","last_unstale":"2024-09-14T06:33:34.831888+0000","last_undegraded":"2024-09-14T06:33:34.831888+0000","last_fullsized":"2024-09-14T06:33:34.831888+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:28:50.205350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:28:50.205350+0000","last_clean_scrub_stamp":"2024-09-14T06:28:50.205350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T12:50:57.166258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065018200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"261'840","reported_seq":4649,"reported_epoch":287,"state":"active+clean","last_fresh":"2024-09-14T06:33:32.819115+0000","last_change":"2024-09-14T06:33:07.769882+0000","last_active":"2024-09-14T06:33:32.819115+0000","last_peered":"2024-09-14T06:33:32.819115+0000","last_clean":"2024-09-14T06:33:32.819115+0000","last_became_active":"2024-09-14T06:31:54.747923+0000","last_became_peered":"2024-09-14T06:31:54.747923+0000","last_unstale":"2024-09-14T06:33:32.819115+0000","last_undegraded":"2024-09-14T06:33:32.819115+0000","last_fullsized":"2024-09-14T06:33:32.819115+0000","mapping_epoch":189,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":190,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":840,"log_dups_size":0,"ondisk_log_size":840,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T09:01:49.581187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000927559,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4591,"num_read_kb":41129,"num_write":1178,"num_write_kb":36122,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":183,"num_bytes_recovered":13912513,"num_keys_recovered":22,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":554,"reported_epoch":289,"state":"active+clean","last_fresh":"2024-09-14T06:33:34.834701+0000","last_change":"2024-09-14T06:28:53.640035+0000","last_active":"2024-09-14T06:33:34.834701+0000","last_peered":"2024-09-14T06:33:34.834701+0000","last_clean":"2024-09-14T06:33:34.834701+0000","last_became_active":"2024-09-14T06:28:51.308058+0000","last_became_peered":"2024-09-14T06:28:51.308058+0000","last_unstale":"2024-09-14T06:33:34.834701+0000","last_undegraded":"2024-09-14T06:33:34.834701+0000","last_fullsized":"2024-09-14T06:33:34.834701+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:28:50.205350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:28:50.205350+0000","last_clean_scrub_stamp":"2024-09-14T06:28:50.205350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T13:35:48.544057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00037824000000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"262'700","reported_seq":4876,"reported_epoch":289,"state":"active+clean","last_fresh":"2024-09-14T06:33:34.832050+0000","last_change":"2024-09-14T06:33:22.769657+0000","last_active":"2024-09-14T06:33:34.832050+0000","last_peered":"2024-09-14T06:33:34.832050+0000","last_clean":"2024-09-14T06:33:34.832050+0000","last_became_active":"2024-09-14T06:33:22.767544+0000","last_became_peered":"2024-09-14T06:33:22.767544+0000","last_unstale":"2024-09-14T06:33:34.832050+0000","last_undegraded":"2024-09-14T06:33:34.832050+0000","last_fullsized":"2024-09-14T06:33:34.832050+0000","mapping_epoch":276,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":277,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":700,"log_dups_size":0,"ondisk_log_size":700,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T13:09:29.858808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00074004799999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":2654,"num_read_kb":37435,"num_write":897,"num_write_kb":40860,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":687,"num_bytes_recovered":31411166,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":555,"reported_epoch":289,"state":"active+clean","last_fresh":"2024-09-14T06:33:34.832080+0000","last_change":"2024-09-14T06:28:53.226072+0000","last_active":"2024-09-14T06:33:34.832080+0000","last_peered":"2024-09-14T06:33:34.832080+0000","last_clean":"2024-09-14T06:33:34.832080+0000","last_became_active":"2024-09-14T06:28:51.308072+0000","last_became_peered":"2024-09-14T06:28:51.308072+0000","last_unstale":"2024-09-14T06:33:34.832080+0000","last_undegraded":"2024-09-14T06:33:34.832080+0000","last_fullsized":"2024-09-14T06:33:34.832080+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:28:50.205350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:28:50.205350+0000","last_clean_scrub_stamp":"2024-09-14T06:28:50.205350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T13:15:09.052328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066197399999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"261'896","reported_seq":5625,"reported_epoch":289,"state":"active+clean","last_fresh":"2024-09-14T06:33:35.112603+0000","last_change":"2024-09-14T06:33:31.823904+0000","last_active":"2024-09-14T06:33:35.112603+0000","last_peered":"2024-09-14T06:33:35.112603+0000","last_clean":"2024-09-14T06:33:35.112603+0000","last_became_active":"2024-09-14T06:33:31.821533+0000","last_became_peered":"2024-09-14T06:33:31.821533+0000","last_unstale":"2024-09-14T06:33:35.112603+0000","last_undegraded":"2024-09-14T06:33:35.112603+0000","last_fullsized":"2024-09-14T06:33:35.112603+0000","mapping_epoch":285,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":286,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":896,"log_dups_size":0,"ondisk_log_size":896,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T15:48:46.733188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015088129999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4516,"num_read_kb":42632,"num_write":1255,"num_write_kb":49494,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":330,"num_bytes_recovered":28798893,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"262'711","reported_seq":10924,"reported_epoch":289,"state":"active+clean","last_fresh":"2024-09-14T06:33:36.040034+0000","last_change":"2024-09-14T06:33:07.704808+0000","last_active":"2024-09-14T06:33:36.040034+0000","last_peered":"2024-09-14T06:33:36.040034+0000","last_clean":"2024-09-14T06:33:36.040034+0000","last_became_active":"2024-09-14T06:29:00.584678+0000","last_became_peered":"2024-09-14T06:29:00.584678+0000","last_unstale":"2024-09-14T06:33:36.040034+0000","last_undegraded":"2024-09-14T06:33:36.040034+0000","last_fullsized":"2024-09-14T06:33:36.040034+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":711,"log_dups_size":0,"ondisk_log_size":711,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T07:39:08.849874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00074987800000000005,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":9560,"num_read_kb":89311,"num_write":777,"num_write_kb":51691,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"22'2","reported_seq":428,"reported_epoch":289,"state":"active+clean","last_fresh":"2024-09-14T06:33:35.112361+0000","last_change":"2024-09-14T06:33:31.823823+0000","last_active":"2024-09-14T06:33:35.112361+0000","last_peered":"2024-09-14T06:33:35.112361+0000","last_clean":"2024-09-14T06:33:35.112361+0000","last_became_active":"2024-09-14T06:33:31.823429+0000","last_became_peered":"2024-09-14T06:33:31.823429+0000","last_unstale":"2024-09-14T06:33:35.112361+0000","last_undegraded":"2024-09-14T06:33:35.112361+0000","last_fullsized":"2024-09-14T06:33:35.112361+0000","mapping_epoch":285,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":286,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:28:50.205350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:28:50.205350+0000","last_clean_scrub_stamp":"2024-09-14T06:28:50.205350+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T18:01:36.855427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010072970000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.0","version":"260'1116","reported_seq":8244,"reported_epoch":289,"state":"active+clean","last_fresh":"2024-09-14T06:33:34.832141+0000","last_change":"2024-09-14T06:33:22.769183+0000","last_active":"2024-09-14T06:33:34.832141+0000","last_peered":"2024-09-14T06:33:34.832141+0000","last_clean":"2024-09-14T06:33:34.832141+0000","last_became_active":"2024-09-14T06:33:22.767520+0000","last_became_peered":"2024-09-14T06:33:22.767520+0000","last_unstale":"2024-09-14T06:33:34.832141+0000","last_undegraded":"2024-09-14T06:33:34.832141+0000","last_fullsized":"2024-09-14T06:33:34.832141+0000","mapping_epoch":276,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":277,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":1116,"log_dups_size":0,"ondisk_log_size":1116,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T17:26:46.362756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064233900000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":8259,"num_read_kb":85480,"num_write":1652,"num_write_kb":58700,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":537,"num_bytes_recovered":2852,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":40,"reported_epoch":289,"state":"active+clean","last_fresh":"2024-09-14T06:33:34.831819+0000","last_change":"2024-09-14T06:33:19.801852+0000","last_active":"2024-09-14T06:33:34.831819+0000","last_peered":"2024-09-14T06:33:34.831819+0000","last_clean":"2024-09-14T06:33:34.831819+0000","last_became_active":"2024-09-14T06:33:19.801557+0000","last_became_peered":"2024-09-14T06:33:19.801557+0000","last_unstale":"2024-09-14T06:33:34.831819+0000","last_undegraded":"2024-09-14T06:33:34.831819+0000","last_fullsized":"2024-09-14T06:33:34.831819+0000","mapping_epoch":273,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":274,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:28:50.205350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:28:50.205350+0000","last_clean_scrub_stamp":"2024-09-14T06:28:50.205350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T08:41:03.294249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053403100000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1","version":"261'822","reported_seq":11886,"reported_epoch":289,"state":"active+clean","last_fresh":"2024-09-14T06:33:34.979564+0000","last_change":"2024-09-14T06:33:07.705685+0000","last_active":"2024-09-14T06:33:34.979564+0000","last_peered":"2024-09-14T06:33:34.979564+0000","last_clean":"2024-09-14T06:33:34.979564+0000","last_became_active":"2024-09-14T06:30:57.768238+0000","last_became_peered":"2024-09-14T06:30:57.768238+0000","last_unstale":"2024-09-14T06:33:34.979564+0000","last_undegraded":"2024-09-14T06:33:34.979564+0000","last_fullsized":"2024-09-14T06:33:34.979564+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":822,"log_dups_size":0,"ondisk_log_size":822,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T15:07:18.804416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011933110000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":11284,"num_read_kb":112430,"num_write":991,"num_write_kb":47408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":183,"num_bytes_recovered":30431994,"num_keys_recovered":22,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":554,"reported_epoch":289,"state":"active+clean","last_fresh":"2024-09-14T06:33:34.834417+0000","last_change":"2024-09-14T06:28:53.640096+0000","last_active":"2024-09-14T06:33:34.834417+0000","last_peered":"2024-09-14T06:33:34.834417+0000","last_clean":"2024-09-14T06:33:34.834417+0000","last_became_active":"2024-09-14T06:28:51.223069+0000","last_became_peered":"2024-09-14T06:28:51.223069+0000","last_unstale":"2024-09-14T06:33:34.834417+0000","last_undegraded":"2024-09-14T06:33:34.834417+0000","last_fullsized":"2024-09-14T06:33:34.834417+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:28:50.205350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:28:50.205350+0000","last_clean_scrub_stamp":"2024-09-14T06:28:50.205350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T15:13:00.471509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00043714600000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"262'1663","reported_seq":7850,"reported_epoch":287,"state":"active+clean","last_fresh":"2024-09-14T06:33:32.818891+0000","last_change":"2024-09-14T06:33:07.770369+0000","last_active":"2024-09-14T06:33:32.818891+0000","last_peered":"2024-09-14T06:33:32.818891+0000","last_clean":"2024-09-14T06:33:32.818891+0000","last_became_active":"2024-09-14T06:31:54.744160+0000","last_became_peered":"2024-09-14T06:31:54.744160+0000","last_unstale":"2024-09-14T06:33:32.818891+0000","last_undegraded":"2024-09-14T06:33:32.818891+0000","last_fullsized":"2024-09-14T06:33:32.818891+0000","mapping_epoch":189,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":190,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":1663,"log_dups_size":0,"ondisk_log_size":1663,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T16:44:35.569843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001382608,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":10723,"num_read_kb":70589,"num_write":2596,"num_write_kb":53564,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":327,"num_bytes_recovered":50047063,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"20'1","reported_seq":427,"reported_epoch":289,"state":"active+clean","last_fresh":"2024-09-14T06:33:35.112503+0000","last_change":"2024-09-14T06:33:31.823426+0000","last_active":"2024-09-14T06:33:35.112503+0000","last_peered":"2024-09-14T06:33:35.112503+0000","last_clean":"2024-09-14T06:33:35.112503+0000","last_became_active":"2024-09-14T06:33:31.823032+0000","last_became_peered":"2024-09-14T06:33:31.823032+0000","last_unstale":"2024-09-14T06:33:35.112503+0000","last_undegraded":"2024-09-14T06:33:35.112503+0000","last_fullsized":"2024-09-14T06:33:35.112503+0000","mapping_epoch":285,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":286,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:28:50.205350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:28:50.205350+0000","last_clean_scrub_stamp":"2024-09-14T06:28:50.205350+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T08:31:58.917801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050808899999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'72","reported_seq":645,"reported_epoch":289,"state":"active+clean","last_fresh":"2024-09-14T06:33:34.834414+0000","last_change":"2024-09-14T06:28:44.839885+0000","last_active":"2024-09-14T06:33:34.834414+0000","last_peered":"2024-09-14T06:33:34.834414+0000","last_clean":"2024-09-14T06:33:34.834414+0000","last_became_active":"2024-09-14T06:28:44.839573+0000","last_became_peered":"2024-09-14T06:28:44.839573+0000","last_unstale":"2024-09-14T06:33:34.834414+0000","last_undegraded":"2024-09-14T06:33:34.834414+0000","last_fullsized":"2024-09-14T06:33:34.834414+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:28:43.828494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:28:43.828494+0000","last_clean_scrub_stamp":"2024-09-14T06:28:43.828494+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T14:41:25.787293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.8","version":"261'684","reported_seq":6657,"reported_epoch":287,"state":"active+clean","last_fresh":"2024-09-14T06:33:32.818645+0000","last_change":"2024-09-14T06:33:07.769379+0000","last_active":"2024-09-14T06:33:32.818645+0000","last_peered":"2024-09-14T06:33:32.818645+0000","last_clean":"2024-09-14T06:33:32.818645+0000","last_became_active":"2024-09-14T06:31:55.754694+0000","last_became_peered":"2024-09-14T06:31:55.754694+0000","last_unstale":"2024-09-14T06:33:32.818645+0000","last_undegraded":"2024-09-14T06:33:32.818645+0000","last_fullsized":"2024-09-14T06:33:32.818645+0000","mapping_epoch":190,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":191,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":684,"log_dups_size":0,"ondisk_log_size":684,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T15:48:47.633183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067851599999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":5793,"num_read_kb":90852,"num_write":921,"num_write_kb":57691,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":319,"num_bytes_recovered":60352269,"num_keys_recovered":22,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.9","version":"261'597","reported_seq":5173,"reported_epoch":289,"state":"active+clean","last_fresh":"2024-09-14T06:33:36.040020+0000","last_change":"2024-09-14T06:33:22.772981+0000","last_active":"2024-09-14T06:33:36.040020+0000","last_peered":"2024-09-14T06:33:36.040020+0000","last_clean":"2024-09-14T06:33:36.040020+0000","last_became_active":"2024-09-14T06:33:22.770956+0000","last_became_peered":"2024-09-14T06:33:22.770956+0000","last_unstale":"2024-09-14T06:33:36.040020+0000","last_undegraded":"2024-09-14T06:33:36.040020+0000","last_fullsized":"2024-09-14T06:33:36.040020+0000","mapping_epoch":276,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":277,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":597,"log_dups_size":0,"ondisk_log_size":597,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T17:14:06.174583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057139500000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3678,"num_read_kb":64291,"num_write":778,"num_write_kb":44928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":316,"num_bytes_recovered":49117,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"261'874","reported_seq":5176,"reported_epoch":289,"state":"active+clean","last_fresh":"2024-09-14T06:33:36.039739+0000","last_change":"2024-09-14T06:33:17.629900+0000","last_active":"2024-09-14T06:33:36.039739+0000","last_peered":"2024-09-14T06:33:36.039739+0000","last_clean":"2024-09-14T06:33:36.039739+0000","last_became_active":"2024-09-14T06:33:17.627910+0000","last_became_peered":"2024-09-14T06:33:17.627910+0000","last_unstale":"2024-09-14T06:33:36.039739+0000","last_undegraded":"2024-09-14T06:33:36.039739+0000","last_fullsized":"2024-09-14T06:33:36.039739+0000","mapping_epoch":271,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":272,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":874,"log_dups_size":0,"ondisk_log_size":874,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T14:22:28.835018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00074887399999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":5331,"num_read_kb":58329,"num_write":1221,"num_write_kb":47535,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":656,"num_bytes_recovered":26968802,"num_keys_recovered":45,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.b","version":"260'1637","reported_seq":8985,"reported_epoch":287,"state":"active+clean","last_fresh":"2024-09-14T06:33:32.818865+0000","last_change":"2024-09-14T06:33:07.769381+0000","last_active":"2024-09-14T06:33:32.818865+0000","last_peered":"2024-09-14T06:33:32.818865+0000","last_clean":"2024-09-14T06:33:32.818865+0000","last_became_active":"2024-09-14T06:31:55.756219+0000","last_became_peered":"2024-09-14T06:31:55.756219+0000","last_unstale":"2024-09-14T06:33:32.818865+0000","last_undegraded":"2024-09-14T06:33:32.818865+0000","last_fullsized":"2024-09-14T06:33:32.818865+0000","mapping_epoch":190,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":191,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":1637,"log_dups_size":0,"ondisk_log_size":1637,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T15:53:45.757678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065814200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":13043,"num_read_kb":64129,"num_write":2496,"num_write_kb":43679,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":291,"num_bytes_recovered":37370632,"num_keys_recovered":66,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.c","version":"255'622","reported_seq":6270,"reported_epoch":289,"state":"active+clean","last_fresh":"2024-09-14T06:33:35.112517+0000","last_change":"2024-09-14T06:33:31.824073+0000","last_active":"2024-09-14T06:33:35.112517+0000","last_peered":"2024-09-14T06:33:35.112517+0000","last_clean":"2024-09-14T06:33:35.112517+0000","last_became_active":"2024-09-14T06:33:31.821638+0000","last_became_peered":"2024-09-14T06:33:31.821638+0000","last_unstale":"2024-09-14T06:33:35.112517+0000","last_undegraded":"2024-09-14T06:33:35.112517+0000","last_fullsized":"2024-09-14T06:33:35.112517+0000","mapping_epoch":285,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":286,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":622,"log_dups_size":0,"ondisk_log_size":622,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T09:06:48.668360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013231740000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4842,"num_read_kb":50425,"num_write":795,"num_write_kb":40827,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":293,"num_bytes_recovered":36712421,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.d","version":"261'577","reported_seq":9328,"reported_epoch":289,"state":"active+clean","last_fresh":"2024-09-14T06:33:34.834667+0000","last_change":"2024-09-14T06:33:07.770219+0000","last_active":"2024-09-14T06:33:34.834667+0000","last_peered":"2024-09-14T06:33:34.834667+0000","last_clean":"2024-09-14T06:33:34.834667+0000","last_became_active":"2024-09-14T06:29:00.580362+0000","last_became_peered":"2024-09-14T06:29:00.580362+0000","last_unstale":"2024-09-14T06:33:34.834667+0000","last_undegraded":"2024-09-14T06:33:34.834667+0000","last_fullsized":"2024-09-14T06:33:34.834667+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":577,"log_dups_size":0,"ondisk_log_size":577,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T16:36:38.133044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010003270000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":8199,"num_read_kb":93694,"num_write":732,"num_write_kb":41032,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.e","version":"262'586","reported_seq":5526,"reported_epoch":289,"state":"active+clean","last_fresh":"2024-09-14T06:33:34.834589+0000","last_change":"2024-09-14T06:33:07.769825+0000","last_active":"2024-09-14T06:33:34.834589+0000","last_peered":"2024-09-14T06:33:34.834589+0000","last_clean":"2024-09-14T06:33:34.834589+0000","last_became_active":"2024-09-14T06:32:39.777524+0000","last_became_peered":"2024-09-14T06:32:39.777524+0000","last_unstale":"2024-09-14T06:33:34.834589+0000","last_undegraded":"2024-09-14T06:33:34.834589+0000","last_fullsized":"2024-09-14T06:33:34.834589+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":586,"log_dups_size":0,"ondisk_log_size":586,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T07:31:19.417437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065560200000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3986,"num_read_kb":61138,"num_write":753,"num_write_kb":43787,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":171,"num_bytes_recovered":10913227,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.f","version":"262'712","reported_seq":5948,"reported_epoch":289,"state":"active+clean","last_fresh":"2024-09-14T06:33:34.834579+0000","last_change":"2024-09-14T06:33:07.770138+0000","last_active":"2024-09-14T06:33:34.834579+0000","last_peered":"2024-09-14T06:33:34.834579+0000","last_clean":"2024-09-14T06:33:34.834579+0000","last_became_active":"2024-09-14T06:29:00.586491+0000","last_became_peered":"2024-09-14T06:29:00.586491+0000","last_unstale":"2024-09-14T06:33:34.834579+0000","last_undegraded":"2024-09-14T06:33:34.834579+0000","last_fullsized":"2024-09-14T06:33:34.834579+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":712,"log_dups_size":0,"ondisk_log_size":712,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T14:20:12.511497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000891002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":6081,"num_read_kb":45578,"num_write":836,"num_write_kb":36134,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.10","version":"261'952","reported_seq":6889,"reported_epoch":289,"state":"active+clean","last_fresh":"2024-09-14T06:33:36.039625+0000","last_change":"2024-09-14T06:33:16.630633+0000","last_active":"2024-09-14T06:33:36.039625+0000","last_peered":"2024-09-14T06:33:36.039625+0000","last_clean":"2024-09-14T06:33:36.039625+0000","last_became_active":"2024-09-14T06:33:16.628480+0000","last_became_peered":"2024-09-14T06:33:16.628480+0000","last_unstale":"2024-09-14T06:33:36.039625+0000","last_undegraded":"2024-09-14T06:33:36.039625+0000","last_fullsized":"2024-09-14T06:33:36.039625+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":952,"log_dups_size":0,"ondisk_log_size":952,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T17:09:51.353193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012025289999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":7147,"num_read_kb":34474,"num_write":1082,"num_write_kb":36655,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":318,"num_bytes_recovered":9401812,"num_keys_recovered":45,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.11","version":"261'730","reported_seq":11746,"reported_epoch":289,"state":"active+clean","last_fresh":"2024-09-14T06:33:34.834479+0000","last_change":"2024-09-14T06:33:07.769975+0000","last_active":"2024-09-14T06:33:34.834479+0000","last_peered":"2024-09-14T06:33:34.834479+0000","last_clean":"2024-09-14T06:33:34.834479+0000","last_became_active":"2024-09-14T06:32:39.778010+0000","last_became_peered":"2024-09-14T06:32:39.778010+0000","last_unstale":"2024-09-14T06:33:34.834479+0000","last_undegraded":"2024-09-14T06:33:34.834479+0000","last_fullsized":"2024-09-14T06:33:34.834479+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":730,"log_dups_size":0,"ondisk_log_size":730,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T10:48:00.264177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00072597299999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":10271,"num_read_kb":109243,"num_write":859,"num_write_kb":48307,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":177,"num_bytes_recovered":16647747,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.12","version":"261'782","reported_seq":7505,"reported_epoch":289,"state":"active+clean","last_fresh":"2024-09-14T06:33:34.979599+0000","last_change":"2024-09-14T06:33:07.706504+0000","last_active":"2024-09-14T06:33:34.979599+0000","last_peered":"2024-09-14T06:33:34.979599+0000","last_clean":"2024-09-14T06:33:34.979599+0000","last_became_active":"2024-09-14T06:29:00.579826+0000","last_became_peered":"2024-09-14T06:29:00.579826+0000","last_unstale":"2024-09-14T06:33:34.979599+0000","last_undegraded":"2024-09-14T06:33:34.979599+0000","last_fullsized":"2024-09-14T06:33:34.979599+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":782,"log_dups_size":0,"ondisk_log_size":782,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T11:32:57.816012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001680792,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":7110,"num_read_kb":59089,"num_write":947,"num_write_kb":42441,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.13","version":"261'1008","reported_seq":8959,"reported_epoch":289,"state":"active+clean","last_fresh":"2024-09-14T06:33:34.834443+0000","last_change":"2024-09-14T06:33:07.769558+0000","last_active":"2024-09-14T06:33:34.834443+0000","last_peered":"2024-09-14T06:33:34.834443+0000","last_clean":"2024-09-14T06:33:34.834443+0000","last_became_active":"2024-09-14T06:29:00.586520+0000","last_became_peered":"2024-09-14T06:29:00.586520+0000","last_unstale":"2024-09-14T06:33:34.834443+0000","last_undegraded":"2024-09-14T06:33:34.834443+0000","last_fullsized":"2024-09-14T06:33:34.834443+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":1008,"log_dups_size":0,"ondisk_log_size":1008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T07:25:04.666000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041539399999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":9551,"num_read_kb":66119,"num_write":1235,"num_write_kb":47437,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":19,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":240092,"num_read_kb":2184933,"num_write":38145,"num_write_kb":1478386,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7477,"num_bytes_recovered":638497624,"num_keys_recovered":546,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":28648,"ondisk_log_size":28648,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":131072,"data_stored":1180736,"data_compressed":65166,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509503,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":70776,"kb_used_data":3484,"kb_used_omap":1,"kb_used_meta":67262,"kb_avail":94301064,"statfs":{"total":96636764160,"available":96564289536,"internally_reserved":0,"allocated":3567616,"data_stored":2317757,"data_compressed":32583,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":1590,"internal_metadata":68876746},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509503,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":56816,"kb_used_data":3420,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94315024,"statfs":{"total":96636764160,"available":96578584576,"internally_reserved":0,"allocated":3502080,"data_stored":1727389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":54655430},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509504,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":74868,"kb_used_data":3424,"kb_used_omap":1,"kb_used_meta":71422,"kb_avail":94296972,"statfs":{"total":96636764160,"available":96560099328,"internally_reserved":0,"allocated":3506176,"data_stored":1727408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":73136581},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509503,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":88756,"kb_used_data":3424,"kb_used_omap":1,"kb_used_meta":85310,"kb_avail":94283084,"statfs":{"total":96636764160,"available":96545878016,"internally_reserved":0,"allocated":3506176,"data_stored":1727408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":87357893},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509504,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":56828,"kb_used_data":3428,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94315012,"statfs":{"total":96636764160,"available":96578572288,"internally_reserved":0,"allocated":3510272,"data_stored":1727427,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":14,"seq":60129542207,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":60976,"kb_used_data":3420,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94310864,"statfs":{"total":96636764160,"available":96574324736,"internally_reserved":0,"allocated":3502080,"data_stored":1727389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":58915270},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542206,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":83252,"kb_used_data":3484,"kb_used_omap":1,"kb_used_meta":79742,"kb_avail":94288588,"statfs":{"total":96636764160,"available":96551514112,"internally_reserved":0,"allocated":3567616,"data_stored":2317757,"data_compressed":32583,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":1595,"internal_metadata":81656261},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574911,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":83168,"kb_used_data":3396,"kb_used_omap":1,"kb_used_meta":79742,"kb_avail":94288672,"statfs":{"total":96636764160,"available":96551600128,"internally_reserved":0,"allocated":3477504,"data_stored":1715589,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":81656263},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32583,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32583,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-14T06:33:40.337 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-14T06:33:40.618 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-14T06:33:40.619 INFO:teuthology.orchestra.run.smithi094.stderr:dumped all 2024-09-14T06:33:40.630 INFO:teuthology.orchestra.run.smithi094.stdout:{"pg_ready":true,"pg_map":{"version":432,"stamp":"2024-09-14T06:33:39.336800+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590406,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":240158,"num_read_kb":2184988,"num_write":38248,"num_write_kb":1480016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7477,"num_bytes_recovered":638497624,"num_keys_recovered":546,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":28723,"ondisk_log_size":28723,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":575440,"kb_used_data":27480,"kb_used_omap":12,"kb_used_meta":547763,"kb_avail":754399280,"statfs":{"total":773094113280,"available":772504862720,"internally_reserved":0,"allocated":28139520,"data_stored":14988124,"data_compressed":65166,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":12751,"internal_metadata":560909873},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":-3,"num_read_kb":-3,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":-1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.516956"},"pg_stats":[{"pgid":"3.1f","version":"262'855","reported_seq":8431,"reported_epoch":289,"state":"active+clean","last_fresh":"2024-09-14T06:33:34.979756+0000","last_change":"2024-09-14T06:33:07.706447+0000","last_active":"2024-09-14T06:33:34.979756+0000","last_peered":"2024-09-14T06:33:34.979756+0000","last_clean":"2024-09-14T06:33:34.979756+0000","last_became_active":"2024-09-14T06:29:00.579883+0000","last_became_peered":"2024-09-14T06:29:00.579883+0000","last_unstale":"2024-09-14T06:33:34.979756+0000","last_undegraded":"2024-09-14T06:33:34.979756+0000","last_fullsized":"2024-09-14T06:33:34.979756+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":855,"log_dups_size":0,"ondisk_log_size":855,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T14:40:25.634308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001666641,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":8468,"num_read_kb":86898,"num_write":1171,"num_write_kb":49543,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"262'783","reported_seq":10851,"reported_epoch":287,"state":"active+clean","last_fresh":"2024-09-14T06:33:32.819008+0000","last_change":"2024-09-14T06:33:16.628665+0000","last_active":"2024-09-14T06:33:32.819008+0000","last_peered":"2024-09-14T06:33:32.819008+0000","last_clean":"2024-09-14T06:33:32.819008+0000","last_became_active":"2024-09-14T06:33:16.626222+0000","last_became_peered":"2024-09-14T06:33:16.626222+0000","last_unstale":"2024-09-14T06:33:32.819008+0000","last_undegraded":"2024-09-14T06:33:32.819008+0000","last_fullsized":"2024-09-14T06:33:32.819008+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":783,"log_dups_size":0,"ondisk_log_size":783,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T15:32:44.526538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013639839999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":8315,"num_read_kb":67657,"num_write":1025,"num_write_kb":40407,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1025,"num_bytes_recovered":87221212,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1d","version":"260'1077","reported_seq":7485,"reported_epoch":289,"state":"active+clean","last_fresh":"2024-09-14T06:33:35.112385+0000","last_change":"2024-09-14T06:33:31.826989+0000","last_active":"2024-09-14T06:33:35.112385+0000","last_peered":"2024-09-14T06:33:35.112385+0000","last_clean":"2024-09-14T06:33:35.112385+0000","last_became_active":"2024-09-14T06:33:31.824454+0000","last_became_peered":"2024-09-14T06:33:31.824454+0000","last_unstale":"2024-09-14T06:33:35.112385+0000","last_undegraded":"2024-09-14T06:33:35.112385+0000","last_fullsized":"2024-09-14T06:33:35.112385+0000","mapping_epoch":285,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":286,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":1077,"log_dups_size":0,"ondisk_log_size":1077,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T07:07:12.782032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088638400000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":7873,"num_read_kb":64375,"num_write":1555,"num_write_kb":53292,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1c","version":"262'805","reported_seq":6034,"reported_epoch":289,"state":"active+clean","last_fresh":"2024-09-14T06:33:35.112317+0000","last_change":"2024-09-14T06:33:31.825738+0000","last_active":"2024-09-14T06:33:35.112317+0000","last_peered":"2024-09-14T06:33:35.112317+0000","last_clean":"2024-09-14T06:33:35.112317+0000","last_became_active":"2024-09-14T06:33:31.823955+0000","last_became_peered":"2024-09-14T06:33:31.823955+0000","last_unstale":"2024-09-14T06:33:35.112317+0000","last_undegraded":"2024-09-14T06:33:35.112317+0000","last_fullsized":"2024-09-14T06:33:35.112317+0000","mapping_epoch":285,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":286,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":805,"log_dups_size":0,"ondisk_log_size":805,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T06:54:41.083735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00068758600000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":4953,"num_read_kb":43809,"num_write":946,"num_write_kb":45551,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":505,"num_bytes_recovered":44145197,"num_keys_recovered":280,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"259'960","reported_seq":11521,"reported_epoch":289,"state":"active+clean","last_fresh":"2024-09-14T06:33:34.979721+0000","last_change":"2024-09-14T06:33:07.706745+0000","last_active":"2024-09-14T06:33:34.979721+0000","last_peered":"2024-09-14T06:33:34.979721+0000","last_clean":"2024-09-14T06:33:34.979721+0000","last_became_active":"2024-09-14T06:29:00.586183+0000","last_became_peered":"2024-09-14T06:29:00.586183+0000","last_unstale":"2024-09-14T06:33:34.979721+0000","last_undegraded":"2024-09-14T06:33:34.979721+0000","last_fullsized":"2024-09-14T06:33:34.979721+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":960,"log_dups_size":0,"ondisk_log_size":960,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T10:55:09.317291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001810606,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":11910,"num_read_kb":108094,"num_write":1463,"num_write_kb":55102,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1a","version":"260'1678","reported_seq":10638,"reported_epoch":289,"state":"active+clean","last_fresh":"2024-09-14T06:33:36.039825+0000","last_change":"2024-09-14T06:33:07.705134+0000","last_active":"2024-09-14T06:33:36.039825+0000","last_peered":"2024-09-14T06:33:36.039825+0000","last_clean":"2024-09-14T06:33:36.039825+0000","last_became_active":"2024-09-14T06:29:00.576206+0000","last_became_peered":"2024-09-14T06:29:00.576206+0000","last_unstale":"2024-09-14T06:33:36.039825+0000","last_undegraded":"2024-09-14T06:33:36.039825+0000","last_fullsized":"2024-09-14T06:33:36.039825+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":1678,"log_dups_size":0,"ondisk_log_size":1678,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T10:47:38.190388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088286099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":13339,"num_read_kb":82904,"num_write":2451,"num_write_kb":55687,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.19","version":"261'561","reported_seq":6508,"reported_epoch":289,"state":"active+clean","last_fresh":"2024-09-14T06:33:34.831894+0000","last_change":"2024-09-14T06:33:07.704594+0000","last_active":"2024-09-14T06:33:34.831894+0000","last_peered":"2024-09-14T06:33:34.831894+0000","last_clean":"2024-09-14T06:33:34.831894+0000","last_became_active":"2024-09-14T06:31:53.744800+0000","last_became_peered":"2024-09-14T06:31:53.744800+0000","last_unstale":"2024-09-14T06:33:34.831894+0000","last_undegraded":"2024-09-14T06:33:34.831894+0000","last_fullsized":"2024-09-14T06:33:34.831894+0000","mapping_epoch":188,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":189,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":561,"log_dups_size":0,"ondisk_log_size":561,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T13:59:24.308993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062201200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":5123,"num_read_kb":67781,"num_write":764,"num_write_kb":43873,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":282,"num_bytes_recovered":41927489,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"262'547","reported_seq":6285,"reported_epoch":287,"state":"active+clean","last_fresh":"2024-09-14T06:33:32.818617+0000","last_change":"2024-09-14T06:33:07.769587+0000","last_active":"2024-09-14T06:33:32.818617+0000","last_peered":"2024-09-14T06:33:32.818617+0000","last_clean":"2024-09-14T06:33:32.818617+0000","last_became_active":"2024-09-14T06:31:53.745159+0000","last_became_peered":"2024-09-14T06:31:53.745159+0000","last_unstale":"2024-09-14T06:33:32.818617+0000","last_undegraded":"2024-09-14T06:33:32.818617+0000","last_fullsized":"2024-09-14T06:33:32.818617+0000","mapping_epoch":188,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":189,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":547,"log_dups_size":0,"ondisk_log_size":547,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T13:08:27.610323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00079631400000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4489,"num_read_kb":78917,"num_write":672,"num_write_kb":51581,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":310,"num_bytes_recovered":56082976,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"261'1037","reported_seq":9728,"reported_epoch":289,"state":"active+clean","last_fresh":"2024-09-14T06:33:34.979868+0000","last_change":"2024-09-14T06:33:07.706863+0000","last_active":"2024-09-14T06:33:34.979868+0000","last_peered":"2024-09-14T06:33:34.979868+0000","last_clean":"2024-09-14T06:33:34.979868+0000","last_became_active":"2024-09-14T06:29:00.579714+0000","last_became_peered":"2024-09-14T06:29:00.579714+0000","last_unstale":"2024-09-14T06:33:34.979868+0000","last_undegraded":"2024-09-14T06:33:34.979868+0000","last_fullsized":"2024-09-14T06:33:34.979868+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":1037,"log_dups_size":0,"ondisk_log_size":1037,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T15:17:51.610057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0017747889999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":10688,"num_read_kb":68964,"num_write":1162,"num_write_kb":48482,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.16","version":"255'1156","reported_seq":4137,"reported_epoch":289,"state":"active+clean","last_fresh":"2024-09-14T06:33:35.112491+0000","last_change":"2024-09-14T06:33:31.824083+0000","last_active":"2024-09-14T06:33:35.112491+0000","last_peered":"2024-09-14T06:33:35.112491+0000","last_clean":"2024-09-14T06:33:35.112491+0000","last_became_active":"2024-09-14T06:33:31.821620+0000","last_became_peered":"2024-09-14T06:33:31.821620+0000","last_unstale":"2024-09-14T06:33:35.112491+0000","last_undegraded":"2024-09-14T06:33:35.112491+0000","last_fullsized":"2024-09-14T06:33:35.112491+0000","mapping_epoch":285,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":286,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":1156,"log_dups_size":0,"ondisk_log_size":1156,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T08:10:02.587758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00073433000000000005,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":5658,"num_read_kb":38562,"num_write":1743,"num_write_kb":37248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"261'650","reported_seq":6751,"reported_epoch":289,"state":"active+clean","last_fresh":"2024-09-14T06:33:34.834516+0000","last_change":"2024-09-14T06:33:07.769581+0000","last_active":"2024-09-14T06:33:34.834516+0000","last_peered":"2024-09-14T06:33:34.834516+0000","last_clean":"2024-09-14T06:33:34.834516+0000","last_became_active":"2024-09-14T06:31:53.746091+0000","last_became_peered":"2024-09-14T06:31:53.746091+0000","last_unstale":"2024-09-14T06:33:34.834516+0000","last_undegraded":"2024-09-14T06:33:34.834516+0000","last_fullsized":"2024-09-14T06:33:34.834516+0000","mapping_epoch":188,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":189,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":650,"log_dups_size":0,"ondisk_log_size":650,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T12:36:31.690559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000442557,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":5101,"num_read_kb":74899,"num_write":696,"num_write_kb":51196,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":271,"num_bytes_recovered":49981473,"num_keys_recovered":33,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.14","version":"261'1137","reported_seq":6341,"reported_epoch":289,"state":"active+clean","last_fresh":"2024-09-14T06:33:36.039887+0000","last_change":"2024-09-14T06:33:07.704760+0000","last_active":"2024-09-14T06:33:36.039887+0000","last_peered":"2024-09-14T06:33:36.039887+0000","last_clean":"2024-09-14T06:33:36.039887+0000","last_became_active":"2024-09-14T06:33:04.628663+0000","last_became_peered":"2024-09-14T06:33:04.628663+0000","last_unstale":"2024-09-14T06:33:36.039887+0000","last_undegraded":"2024-09-14T06:33:36.039887+0000","last_fullsized":"2024-09-14T06:33:36.039887+0000","mapping_epoch":258,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":259,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":1137,"log_dups_size":0,"ondisk_log_size":1137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T11:42:47.162926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00078945399999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":8948,"num_read_kb":46936,"num_write":1573,"num_write_kb":29829,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":296,"num_bytes_recovered":6118769,"num_keys_recovered":11,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":50,"reported_epoch":289,"state":"active+clean","last_fresh":"2024-09-14T06:33:36.039662+0000","last_change":"2024-09-14T06:33:13.859355+0000","last_active":"2024-09-14T06:33:36.039662+0000","last_peered":"2024-09-14T06:33:36.039662+0000","last_clean":"2024-09-14T06:33:36.039662+0000","last_became_active":"2024-09-14T06:33:13.859069+0000","last_became_peered":"2024-09-14T06:33:13.859069+0000","last_unstale":"2024-09-14T06:33:36.039662+0000","last_undegraded":"2024-09-14T06:33:36.039662+0000","last_fullsized":"2024-09-14T06:33:36.039662+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:28:50.205350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:28:50.205350+0000","last_clean_scrub_stamp":"2024-09-14T06:28:50.205350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T17:08:35.416194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"262'893","reported_seq":8726,"reported_epoch":289,"state":"active+clean","last_fresh":"2024-09-14T06:33:34.979753+0000","last_change":"2024-09-14T06:33:07.705814+0000","last_active":"2024-09-14T06:33:34.979753+0000","last_peered":"2024-09-14T06:33:34.979753+0000","last_clean":"2024-09-14T06:33:34.979753+0000","last_became_active":"2024-09-14T06:29:00.578283+0000","last_became_peered":"2024-09-14T06:29:00.578283+0000","last_unstale":"2024-09-14T06:33:34.979753+0000","last_undegraded":"2024-09-14T06:33:34.979753+0000","last_fullsized":"2024-09-14T06:33:34.979753+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":893,"log_dups_size":0,"ondisk_log_size":893,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T15:09:37.855181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001076554,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":8608,"num_read_kb":78770,"num_write":923,"num_write_kb":48303,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":567,"reported_epoch":289,"state":"active+clean","last_fresh":"2024-09-14T06:33:34.831888+0000","last_change":"2024-09-14T06:33:13.734687+0000","last_active":"2024-09-14T06:33:34.831888+0000","last_peered":"2024-09-14T06:33:34.831888+0000","last_clean":"2024-09-14T06:33:34.831888+0000","last_became_active":"2024-09-14T06:33:13.734442+0000","last_became_peered":"2024-09-14T06:33:13.734442+0000","last_unstale":"2024-09-14T06:33:34.831888+0000","last_undegraded":"2024-09-14T06:33:34.831888+0000","last_fullsized":"2024-09-14T06:33:34.831888+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:28:50.205350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:28:50.205350+0000","last_clean_scrub_stamp":"2024-09-14T06:28:50.205350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T12:50:57.166258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065018200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"261'840","reported_seq":4649,"reported_epoch":287,"state":"active+clean","last_fresh":"2024-09-14T06:33:32.819115+0000","last_change":"2024-09-14T06:33:07.769882+0000","last_active":"2024-09-14T06:33:32.819115+0000","last_peered":"2024-09-14T06:33:32.819115+0000","last_clean":"2024-09-14T06:33:32.819115+0000","last_became_active":"2024-09-14T06:31:54.747923+0000","last_became_peered":"2024-09-14T06:31:54.747923+0000","last_unstale":"2024-09-14T06:33:32.819115+0000","last_undegraded":"2024-09-14T06:33:32.819115+0000","last_fullsized":"2024-09-14T06:33:32.819115+0000","mapping_epoch":189,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":190,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":840,"log_dups_size":0,"ondisk_log_size":840,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T09:01:49.581187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000927559,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4591,"num_read_kb":41129,"num_write":1178,"num_write_kb":36122,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":183,"num_bytes_recovered":13912513,"num_keys_recovered":22,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":554,"reported_epoch":289,"state":"active+clean","last_fresh":"2024-09-14T06:33:34.834701+0000","last_change":"2024-09-14T06:28:53.640035+0000","last_active":"2024-09-14T06:33:34.834701+0000","last_peered":"2024-09-14T06:33:34.834701+0000","last_clean":"2024-09-14T06:33:34.834701+0000","last_became_active":"2024-09-14T06:28:51.308058+0000","last_became_peered":"2024-09-14T06:28:51.308058+0000","last_unstale":"2024-09-14T06:33:34.834701+0000","last_undegraded":"2024-09-14T06:33:34.834701+0000","last_fullsized":"2024-09-14T06:33:34.834701+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:28:50.205350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:28:50.205350+0000","last_clean_scrub_stamp":"2024-09-14T06:28:50.205350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T13:35:48.544057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00037824000000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"262'700","reported_seq":4876,"reported_epoch":289,"state":"active+clean","last_fresh":"2024-09-14T06:33:34.832050+0000","last_change":"2024-09-14T06:33:22.769657+0000","last_active":"2024-09-14T06:33:34.832050+0000","last_peered":"2024-09-14T06:33:34.832050+0000","last_clean":"2024-09-14T06:33:34.832050+0000","last_became_active":"2024-09-14T06:33:22.767544+0000","last_became_peered":"2024-09-14T06:33:22.767544+0000","last_unstale":"2024-09-14T06:33:34.832050+0000","last_undegraded":"2024-09-14T06:33:34.832050+0000","last_fullsized":"2024-09-14T06:33:34.832050+0000","mapping_epoch":276,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":277,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":700,"log_dups_size":0,"ondisk_log_size":700,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T13:09:29.858808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00074004799999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":2654,"num_read_kb":37435,"num_write":897,"num_write_kb":40860,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":687,"num_bytes_recovered":31411166,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":555,"reported_epoch":289,"state":"active+clean","last_fresh":"2024-09-14T06:33:34.832080+0000","last_change":"2024-09-14T06:28:53.226072+0000","last_active":"2024-09-14T06:33:34.832080+0000","last_peered":"2024-09-14T06:33:34.832080+0000","last_clean":"2024-09-14T06:33:34.832080+0000","last_became_active":"2024-09-14T06:28:51.308072+0000","last_became_peered":"2024-09-14T06:28:51.308072+0000","last_unstale":"2024-09-14T06:33:34.832080+0000","last_undegraded":"2024-09-14T06:33:34.832080+0000","last_fullsized":"2024-09-14T06:33:34.832080+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:28:50.205350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:28:50.205350+0000","last_clean_scrub_stamp":"2024-09-14T06:28:50.205350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T13:15:09.052328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066197399999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"261'896","reported_seq":5625,"reported_epoch":289,"state":"active+clean","last_fresh":"2024-09-14T06:33:35.112603+0000","last_change":"2024-09-14T06:33:31.823904+0000","last_active":"2024-09-14T06:33:35.112603+0000","last_peered":"2024-09-14T06:33:35.112603+0000","last_clean":"2024-09-14T06:33:35.112603+0000","last_became_active":"2024-09-14T06:33:31.821533+0000","last_became_peered":"2024-09-14T06:33:31.821533+0000","last_unstale":"2024-09-14T06:33:35.112603+0000","last_undegraded":"2024-09-14T06:33:35.112603+0000","last_fullsized":"2024-09-14T06:33:35.112603+0000","mapping_epoch":285,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":286,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":896,"log_dups_size":0,"ondisk_log_size":896,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T15:48:46.733188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015088129999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4516,"num_read_kb":42632,"num_write":1255,"num_write_kb":49494,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":330,"num_bytes_recovered":28798893,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"262'711","reported_seq":10924,"reported_epoch":289,"state":"active+clean","last_fresh":"2024-09-14T06:33:36.040034+0000","last_change":"2024-09-14T06:33:07.704808+0000","last_active":"2024-09-14T06:33:36.040034+0000","last_peered":"2024-09-14T06:33:36.040034+0000","last_clean":"2024-09-14T06:33:36.040034+0000","last_became_active":"2024-09-14T06:29:00.584678+0000","last_became_peered":"2024-09-14T06:29:00.584678+0000","last_unstale":"2024-09-14T06:33:36.040034+0000","last_undegraded":"2024-09-14T06:33:36.040034+0000","last_fullsized":"2024-09-14T06:33:36.040034+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":711,"log_dups_size":0,"ondisk_log_size":711,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T07:39:08.849874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00074987800000000005,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":9560,"num_read_kb":89311,"num_write":777,"num_write_kb":51691,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"22'2","reported_seq":428,"reported_epoch":289,"state":"active+clean","last_fresh":"2024-09-14T06:33:35.112361+0000","last_change":"2024-09-14T06:33:31.823823+0000","last_active":"2024-09-14T06:33:35.112361+0000","last_peered":"2024-09-14T06:33:35.112361+0000","last_clean":"2024-09-14T06:33:35.112361+0000","last_became_active":"2024-09-14T06:33:31.823429+0000","last_became_peered":"2024-09-14T06:33:31.823429+0000","last_unstale":"2024-09-14T06:33:35.112361+0000","last_undegraded":"2024-09-14T06:33:35.112361+0000","last_fullsized":"2024-09-14T06:33:35.112361+0000","mapping_epoch":285,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":286,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:28:50.205350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:28:50.205350+0000","last_clean_scrub_stamp":"2024-09-14T06:28:50.205350+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T18:01:36.855427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010072970000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.0","version":"260'1116","reported_seq":8244,"reported_epoch":289,"state":"active+clean","last_fresh":"2024-09-14T06:33:34.832141+0000","last_change":"2024-09-14T06:33:22.769183+0000","last_active":"2024-09-14T06:33:34.832141+0000","last_peered":"2024-09-14T06:33:34.832141+0000","last_clean":"2024-09-14T06:33:34.832141+0000","last_became_active":"2024-09-14T06:33:22.767520+0000","last_became_peered":"2024-09-14T06:33:22.767520+0000","last_unstale":"2024-09-14T06:33:34.832141+0000","last_undegraded":"2024-09-14T06:33:34.832141+0000","last_fullsized":"2024-09-14T06:33:34.832141+0000","mapping_epoch":276,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":277,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":1116,"log_dups_size":0,"ondisk_log_size":1116,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T17:26:46.362756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064233900000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":8259,"num_read_kb":85480,"num_write":1652,"num_write_kb":58700,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":537,"num_bytes_recovered":2852,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":40,"reported_epoch":289,"state":"active+clean","last_fresh":"2024-09-14T06:33:34.831819+0000","last_change":"2024-09-14T06:33:19.801852+0000","last_active":"2024-09-14T06:33:34.831819+0000","last_peered":"2024-09-14T06:33:34.831819+0000","last_clean":"2024-09-14T06:33:34.831819+0000","last_became_active":"2024-09-14T06:33:19.801557+0000","last_became_peered":"2024-09-14T06:33:19.801557+0000","last_unstale":"2024-09-14T06:33:34.831819+0000","last_undegraded":"2024-09-14T06:33:34.831819+0000","last_fullsized":"2024-09-14T06:33:34.831819+0000","mapping_epoch":273,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":274,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:28:50.205350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:28:50.205350+0000","last_clean_scrub_stamp":"2024-09-14T06:28:50.205350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T08:41:03.294249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053403100000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1","version":"261'822","reported_seq":11886,"reported_epoch":289,"state":"active+clean","last_fresh":"2024-09-14T06:33:34.979564+0000","last_change":"2024-09-14T06:33:07.705685+0000","last_active":"2024-09-14T06:33:34.979564+0000","last_peered":"2024-09-14T06:33:34.979564+0000","last_clean":"2024-09-14T06:33:34.979564+0000","last_became_active":"2024-09-14T06:30:57.768238+0000","last_became_peered":"2024-09-14T06:30:57.768238+0000","last_unstale":"2024-09-14T06:33:34.979564+0000","last_undegraded":"2024-09-14T06:33:34.979564+0000","last_fullsized":"2024-09-14T06:33:34.979564+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":822,"log_dups_size":0,"ondisk_log_size":822,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T15:07:18.804416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011933110000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":11284,"num_read_kb":112430,"num_write":991,"num_write_kb":47408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":183,"num_bytes_recovered":30431994,"num_keys_recovered":22,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":554,"reported_epoch":289,"state":"active+clean","last_fresh":"2024-09-14T06:33:34.834417+0000","last_change":"2024-09-14T06:28:53.640096+0000","last_active":"2024-09-14T06:33:34.834417+0000","last_peered":"2024-09-14T06:33:34.834417+0000","last_clean":"2024-09-14T06:33:34.834417+0000","last_became_active":"2024-09-14T06:28:51.223069+0000","last_became_peered":"2024-09-14T06:28:51.223069+0000","last_unstale":"2024-09-14T06:33:34.834417+0000","last_undegraded":"2024-09-14T06:33:34.834417+0000","last_fullsized":"2024-09-14T06:33:34.834417+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:28:50.205350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:28:50.205350+0000","last_clean_scrub_stamp":"2024-09-14T06:28:50.205350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T15:13:00.471509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00043714600000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"262'1663","reported_seq":7850,"reported_epoch":287,"state":"active+clean","last_fresh":"2024-09-14T06:33:32.818891+0000","last_change":"2024-09-14T06:33:07.770369+0000","last_active":"2024-09-14T06:33:32.818891+0000","last_peered":"2024-09-14T06:33:32.818891+0000","last_clean":"2024-09-14T06:33:32.818891+0000","last_became_active":"2024-09-14T06:31:54.744160+0000","last_became_peered":"2024-09-14T06:31:54.744160+0000","last_unstale":"2024-09-14T06:33:32.818891+0000","last_undegraded":"2024-09-14T06:33:32.818891+0000","last_fullsized":"2024-09-14T06:33:32.818891+0000","mapping_epoch":189,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":190,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":1663,"log_dups_size":0,"ondisk_log_size":1663,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T16:44:35.569843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001382608,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":10723,"num_read_kb":70589,"num_write":2596,"num_write_kb":53564,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":327,"num_bytes_recovered":50047063,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"20'1","reported_seq":427,"reported_epoch":289,"state":"active+clean","last_fresh":"2024-09-14T06:33:35.112503+0000","last_change":"2024-09-14T06:33:31.823426+0000","last_active":"2024-09-14T06:33:35.112503+0000","last_peered":"2024-09-14T06:33:35.112503+0000","last_clean":"2024-09-14T06:33:35.112503+0000","last_became_active":"2024-09-14T06:33:31.823032+0000","last_became_peered":"2024-09-14T06:33:31.823032+0000","last_unstale":"2024-09-14T06:33:35.112503+0000","last_undegraded":"2024-09-14T06:33:35.112503+0000","last_fullsized":"2024-09-14T06:33:35.112503+0000","mapping_epoch":285,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":286,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:28:50.205350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:28:50.205350+0000","last_clean_scrub_stamp":"2024-09-14T06:28:50.205350+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T08:31:58.917801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050808899999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'72","reported_seq":645,"reported_epoch":289,"state":"active+clean","last_fresh":"2024-09-14T06:33:34.834414+0000","last_change":"2024-09-14T06:28:44.839885+0000","last_active":"2024-09-14T06:33:34.834414+0000","last_peered":"2024-09-14T06:33:34.834414+0000","last_clean":"2024-09-14T06:33:34.834414+0000","last_became_active":"2024-09-14T06:28:44.839573+0000","last_became_peered":"2024-09-14T06:28:44.839573+0000","last_unstale":"2024-09-14T06:33:34.834414+0000","last_undegraded":"2024-09-14T06:33:34.834414+0000","last_fullsized":"2024-09-14T06:33:34.834414+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:28:43.828494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:28:43.828494+0000","last_clean_scrub_stamp":"2024-09-14T06:28:43.828494+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T14:41:25.787293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.8","version":"261'684","reported_seq":6657,"reported_epoch":287,"state":"active+clean","last_fresh":"2024-09-14T06:33:32.818645+0000","last_change":"2024-09-14T06:33:07.769379+0000","last_active":"2024-09-14T06:33:32.818645+0000","last_peered":"2024-09-14T06:33:32.818645+0000","last_clean":"2024-09-14T06:33:32.818645+0000","last_became_active":"2024-09-14T06:31:55.754694+0000","last_became_peered":"2024-09-14T06:31:55.754694+0000","last_unstale":"2024-09-14T06:33:32.818645+0000","last_undegraded":"2024-09-14T06:33:32.818645+0000","last_fullsized":"2024-09-14T06:33:32.818645+0000","mapping_epoch":190,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":191,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":684,"log_dups_size":0,"ondisk_log_size":684,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T15:48:47.633183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067851599999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":5793,"num_read_kb":90852,"num_write":921,"num_write_kb":57691,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":319,"num_bytes_recovered":60352269,"num_keys_recovered":22,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.9","version":"261'597","reported_seq":5173,"reported_epoch":289,"state":"active+clean","last_fresh":"2024-09-14T06:33:36.040020+0000","last_change":"2024-09-14T06:33:22.772981+0000","last_active":"2024-09-14T06:33:36.040020+0000","last_peered":"2024-09-14T06:33:36.040020+0000","last_clean":"2024-09-14T06:33:36.040020+0000","last_became_active":"2024-09-14T06:33:22.770956+0000","last_became_peered":"2024-09-14T06:33:22.770956+0000","last_unstale":"2024-09-14T06:33:36.040020+0000","last_undegraded":"2024-09-14T06:33:36.040020+0000","last_fullsized":"2024-09-14T06:33:36.040020+0000","mapping_epoch":276,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":277,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":597,"log_dups_size":0,"ondisk_log_size":597,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T17:14:06.174583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057139500000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3678,"num_read_kb":64291,"num_write":778,"num_write_kb":44928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":316,"num_bytes_recovered":49117,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"261'874","reported_seq":5176,"reported_epoch":289,"state":"active+clean","last_fresh":"2024-09-14T06:33:36.039739+0000","last_change":"2024-09-14T06:33:17.629900+0000","last_active":"2024-09-14T06:33:36.039739+0000","last_peered":"2024-09-14T06:33:36.039739+0000","last_clean":"2024-09-14T06:33:36.039739+0000","last_became_active":"2024-09-14T06:33:17.627910+0000","last_became_peered":"2024-09-14T06:33:17.627910+0000","last_unstale":"2024-09-14T06:33:36.039739+0000","last_undegraded":"2024-09-14T06:33:36.039739+0000","last_fullsized":"2024-09-14T06:33:36.039739+0000","mapping_epoch":271,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":272,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":874,"log_dups_size":0,"ondisk_log_size":874,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T14:22:28.835018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00074887399999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":5331,"num_read_kb":58329,"num_write":1221,"num_write_kb":47535,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":656,"num_bytes_recovered":26968802,"num_keys_recovered":45,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.b","version":"260'1637","reported_seq":8985,"reported_epoch":287,"state":"active+clean","last_fresh":"2024-09-14T06:33:32.818865+0000","last_change":"2024-09-14T06:33:07.769381+0000","last_active":"2024-09-14T06:33:32.818865+0000","last_peered":"2024-09-14T06:33:32.818865+0000","last_clean":"2024-09-14T06:33:32.818865+0000","last_became_active":"2024-09-14T06:31:55.756219+0000","last_became_peered":"2024-09-14T06:31:55.756219+0000","last_unstale":"2024-09-14T06:33:32.818865+0000","last_undegraded":"2024-09-14T06:33:32.818865+0000","last_fullsized":"2024-09-14T06:33:32.818865+0000","mapping_epoch":190,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":191,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":1637,"log_dups_size":0,"ondisk_log_size":1637,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T15:53:45.757678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065814200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":13043,"num_read_kb":64129,"num_write":2496,"num_write_kb":43679,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":291,"num_bytes_recovered":37370632,"num_keys_recovered":66,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.c","version":"255'622","reported_seq":6270,"reported_epoch":289,"state":"active+clean","last_fresh":"2024-09-14T06:33:35.112517+0000","last_change":"2024-09-14T06:33:31.824073+0000","last_active":"2024-09-14T06:33:35.112517+0000","last_peered":"2024-09-14T06:33:35.112517+0000","last_clean":"2024-09-14T06:33:35.112517+0000","last_became_active":"2024-09-14T06:33:31.821638+0000","last_became_peered":"2024-09-14T06:33:31.821638+0000","last_unstale":"2024-09-14T06:33:35.112517+0000","last_undegraded":"2024-09-14T06:33:35.112517+0000","last_fullsized":"2024-09-14T06:33:35.112517+0000","mapping_epoch":285,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":286,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":622,"log_dups_size":0,"ondisk_log_size":622,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T09:06:48.668360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013231740000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4842,"num_read_kb":50425,"num_write":795,"num_write_kb":40827,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":293,"num_bytes_recovered":36712421,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.d","version":"261'577","reported_seq":9328,"reported_epoch":289,"state":"active+clean","last_fresh":"2024-09-14T06:33:34.834667+0000","last_change":"2024-09-14T06:33:07.770219+0000","last_active":"2024-09-14T06:33:34.834667+0000","last_peered":"2024-09-14T06:33:34.834667+0000","last_clean":"2024-09-14T06:33:34.834667+0000","last_became_active":"2024-09-14T06:29:00.580362+0000","last_became_peered":"2024-09-14T06:29:00.580362+0000","last_unstale":"2024-09-14T06:33:34.834667+0000","last_undegraded":"2024-09-14T06:33:34.834667+0000","last_fullsized":"2024-09-14T06:33:34.834667+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":577,"log_dups_size":0,"ondisk_log_size":577,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T16:36:38.133044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010003270000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":8199,"num_read_kb":93694,"num_write":732,"num_write_kb":41032,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.e","version":"262'586","reported_seq":5526,"reported_epoch":289,"state":"active+clean","last_fresh":"2024-09-14T06:33:34.834589+0000","last_change":"2024-09-14T06:33:07.769825+0000","last_active":"2024-09-14T06:33:34.834589+0000","last_peered":"2024-09-14T06:33:34.834589+0000","last_clean":"2024-09-14T06:33:34.834589+0000","last_became_active":"2024-09-14T06:32:39.777524+0000","last_became_peered":"2024-09-14T06:32:39.777524+0000","last_unstale":"2024-09-14T06:33:34.834589+0000","last_undegraded":"2024-09-14T06:33:34.834589+0000","last_fullsized":"2024-09-14T06:33:34.834589+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":586,"log_dups_size":0,"ondisk_log_size":586,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T07:31:19.417437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065560200000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3986,"num_read_kb":61138,"num_write":753,"num_write_kb":43787,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":171,"num_bytes_recovered":10913227,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.f","version":"262'712","reported_seq":5948,"reported_epoch":289,"state":"active+clean","last_fresh":"2024-09-14T06:33:34.834579+0000","last_change":"2024-09-14T06:33:07.770138+0000","last_active":"2024-09-14T06:33:34.834579+0000","last_peered":"2024-09-14T06:33:34.834579+0000","last_clean":"2024-09-14T06:33:34.834579+0000","last_became_active":"2024-09-14T06:29:00.586491+0000","last_became_peered":"2024-09-14T06:29:00.586491+0000","last_unstale":"2024-09-14T06:33:34.834579+0000","last_undegraded":"2024-09-14T06:33:34.834579+0000","last_fullsized":"2024-09-14T06:33:34.834579+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":712,"log_dups_size":0,"ondisk_log_size":712,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T14:20:12.511497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000891002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":6081,"num_read_kb":45578,"num_write":836,"num_write_kb":36134,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.10","version":"261'952","reported_seq":6889,"reported_epoch":289,"state":"active+clean","last_fresh":"2024-09-14T06:33:36.039625+0000","last_change":"2024-09-14T06:33:16.630633+0000","last_active":"2024-09-14T06:33:36.039625+0000","last_peered":"2024-09-14T06:33:36.039625+0000","last_clean":"2024-09-14T06:33:36.039625+0000","last_became_active":"2024-09-14T06:33:16.628480+0000","last_became_peered":"2024-09-14T06:33:16.628480+0000","last_unstale":"2024-09-14T06:33:36.039625+0000","last_undegraded":"2024-09-14T06:33:36.039625+0000","last_fullsized":"2024-09-14T06:33:36.039625+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":952,"log_dups_size":0,"ondisk_log_size":952,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T17:09:51.353193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012025289999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":7147,"num_read_kb":34474,"num_write":1082,"num_write_kb":36655,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":318,"num_bytes_recovered":9401812,"num_keys_recovered":45,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.11","version":"261'730","reported_seq":11746,"reported_epoch":289,"state":"active+clean","last_fresh":"2024-09-14T06:33:34.834479+0000","last_change":"2024-09-14T06:33:07.769975+0000","last_active":"2024-09-14T06:33:34.834479+0000","last_peered":"2024-09-14T06:33:34.834479+0000","last_clean":"2024-09-14T06:33:34.834479+0000","last_became_active":"2024-09-14T06:32:39.778010+0000","last_became_peered":"2024-09-14T06:32:39.778010+0000","last_unstale":"2024-09-14T06:33:34.834479+0000","last_undegraded":"2024-09-14T06:33:34.834479+0000","last_fullsized":"2024-09-14T06:33:34.834479+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":730,"log_dups_size":0,"ondisk_log_size":730,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T10:48:00.264177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00072597299999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":10271,"num_read_kb":109243,"num_write":859,"num_write_kb":48307,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":177,"num_bytes_recovered":16647747,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.12","version":"261'782","reported_seq":7505,"reported_epoch":289,"state":"active+clean","last_fresh":"2024-09-14T06:33:34.979599+0000","last_change":"2024-09-14T06:33:07.706504+0000","last_active":"2024-09-14T06:33:34.979599+0000","last_peered":"2024-09-14T06:33:34.979599+0000","last_clean":"2024-09-14T06:33:34.979599+0000","last_became_active":"2024-09-14T06:29:00.579826+0000","last_became_peered":"2024-09-14T06:29:00.579826+0000","last_unstale":"2024-09-14T06:33:34.979599+0000","last_undegraded":"2024-09-14T06:33:34.979599+0000","last_fullsized":"2024-09-14T06:33:34.979599+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":782,"log_dups_size":0,"ondisk_log_size":782,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T11:32:57.816012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001680792,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":7110,"num_read_kb":59089,"num_write":947,"num_write_kb":42441,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.13","version":"261'1008","reported_seq":8959,"reported_epoch":289,"state":"active+clean","last_fresh":"2024-09-14T06:33:34.834443+0000","last_change":"2024-09-14T06:33:07.769558+0000","last_active":"2024-09-14T06:33:34.834443+0000","last_peered":"2024-09-14T06:33:34.834443+0000","last_clean":"2024-09-14T06:33:34.834443+0000","last_became_active":"2024-09-14T06:29:00.586520+0000","last_became_peered":"2024-09-14T06:29:00.586520+0000","last_unstale":"2024-09-14T06:33:34.834443+0000","last_undegraded":"2024-09-14T06:33:34.834443+0000","last_fullsized":"2024-09-14T06:33:34.834443+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":1008,"log_dups_size":0,"ondisk_log_size":1008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T07:25:04.666000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041539399999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":9551,"num_read_kb":66119,"num_write":1235,"num_write_kb":47437,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":19,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":240092,"num_read_kb":2184933,"num_write":38145,"num_write_kb":1478386,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7477,"num_bytes_recovered":638497624,"num_keys_recovered":546,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":28648,"ondisk_log_size":28648,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":131072,"data_stored":1180736,"data_compressed":65166,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509503,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":70776,"kb_used_data":3484,"kb_used_omap":1,"kb_used_meta":67262,"kb_avail":94301064,"statfs":{"total":96636764160,"available":96564289536,"internally_reserved":0,"allocated":3567616,"data_stored":2317757,"data_compressed":32583,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":1590,"internal_metadata":68876746},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509503,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":56816,"kb_used_data":3420,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94315024,"statfs":{"total":96636764160,"available":96578584576,"internally_reserved":0,"allocated":3502080,"data_stored":1727389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":54655430},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509504,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":74868,"kb_used_data":3424,"kb_used_omap":1,"kb_used_meta":71422,"kb_avail":94296972,"statfs":{"total":96636764160,"available":96560099328,"internally_reserved":0,"allocated":3506176,"data_stored":1727408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":73136581},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509503,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":88756,"kb_used_data":3424,"kb_used_omap":1,"kb_used_meta":85310,"kb_avail":94283084,"statfs":{"total":96636764160,"available":96545878016,"internally_reserved":0,"allocated":3506176,"data_stored":1727408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":87357893},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509504,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":56828,"kb_used_data":3428,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94315012,"statfs":{"total":96636764160,"available":96578572288,"internally_reserved":0,"allocated":3510272,"data_stored":1727427,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":14,"seq":60129542207,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":60976,"kb_used_data":3420,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94310864,"statfs":{"total":96636764160,"available":96574324736,"internally_reserved":0,"allocated":3502080,"data_stored":1727389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":58915270},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542206,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":83252,"kb_used_data":3484,"kb_used_omap":1,"kb_used_meta":79742,"kb_avail":94288588,"statfs":{"total":96636764160,"available":96551514112,"internally_reserved":0,"allocated":3567616,"data_stored":2317757,"data_compressed":32583,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":1595,"internal_metadata":81656261},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574911,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":83168,"kb_used_data":3396,"kb_used_omap":1,"kb_used_meta":79742,"kb_avail":94288672,"statfs":{"total":96636764160,"available":96551600128,"internally_reserved":0,"allocated":3477504,"data_stored":1715589,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":81656263},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32583,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32583,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-14T06:33:40.632 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-14T06:33:40.632 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-14T06:33:40.915 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-14T06:33:40.916 INFO:teuthology.orchestra.run.smithi094.stderr:dumped all 2024-09-14T06:33:40.928 INFO:teuthology.orchestra.run.smithi094.stdout:{"pg_ready":true,"pg_map":{"version":432,"stamp":"2024-09-14T06:33:39.336800+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590406,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":240158,"num_read_kb":2184988,"num_write":38248,"num_write_kb":1480016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7477,"num_bytes_recovered":638497624,"num_keys_recovered":546,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":28723,"ondisk_log_size":28723,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":575440,"kb_used_data":27480,"kb_used_omap":12,"kb_used_meta":547763,"kb_avail":754399280,"statfs":{"total":773094113280,"available":772504862720,"internally_reserved":0,"allocated":28139520,"data_stored":14988124,"data_compressed":65166,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":12751,"internal_metadata":560909873},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":-3,"num_read_kb":-3,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":-1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.516956"},"pg_stats":[{"pgid":"3.1f","version":"262'855","reported_seq":8431,"reported_epoch":289,"state":"active+clean","last_fresh":"2024-09-14T06:33:34.979756+0000","last_change":"2024-09-14T06:33:07.706447+0000","last_active":"2024-09-14T06:33:34.979756+0000","last_peered":"2024-09-14T06:33:34.979756+0000","last_clean":"2024-09-14T06:33:34.979756+0000","last_became_active":"2024-09-14T06:29:00.579883+0000","last_became_peered":"2024-09-14T06:29:00.579883+0000","last_unstale":"2024-09-14T06:33:34.979756+0000","last_undegraded":"2024-09-14T06:33:34.979756+0000","last_fullsized":"2024-09-14T06:33:34.979756+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":855,"log_dups_size":0,"ondisk_log_size":855,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T14:40:25.634308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001666641,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":8468,"num_read_kb":86898,"num_write":1171,"num_write_kb":49543,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"262'783","reported_seq":10851,"reported_epoch":287,"state":"active+clean","last_fresh":"2024-09-14T06:33:32.819008+0000","last_change":"2024-09-14T06:33:16.628665+0000","last_active":"2024-09-14T06:33:32.819008+0000","last_peered":"2024-09-14T06:33:32.819008+0000","last_clean":"2024-09-14T06:33:32.819008+0000","last_became_active":"2024-09-14T06:33:16.626222+0000","last_became_peered":"2024-09-14T06:33:16.626222+0000","last_unstale":"2024-09-14T06:33:32.819008+0000","last_undegraded":"2024-09-14T06:33:32.819008+0000","last_fullsized":"2024-09-14T06:33:32.819008+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":783,"log_dups_size":0,"ondisk_log_size":783,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T15:32:44.526538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013639839999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":8315,"num_read_kb":67657,"num_write":1025,"num_write_kb":40407,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1025,"num_bytes_recovered":87221212,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1d","version":"260'1077","reported_seq":7485,"reported_epoch":289,"state":"active+clean","last_fresh":"2024-09-14T06:33:35.112385+0000","last_change":"2024-09-14T06:33:31.826989+0000","last_active":"2024-09-14T06:33:35.112385+0000","last_peered":"2024-09-14T06:33:35.112385+0000","last_clean":"2024-09-14T06:33:35.112385+0000","last_became_active":"2024-09-14T06:33:31.824454+0000","last_became_peered":"2024-09-14T06:33:31.824454+0000","last_unstale":"2024-09-14T06:33:35.112385+0000","last_undegraded":"2024-09-14T06:33:35.112385+0000","last_fullsized":"2024-09-14T06:33:35.112385+0000","mapping_epoch":285,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":286,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":1077,"log_dups_size":0,"ondisk_log_size":1077,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T07:07:12.782032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088638400000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":7873,"num_read_kb":64375,"num_write":1555,"num_write_kb":53292,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1c","version":"262'805","reported_seq":6034,"reported_epoch":289,"state":"active+clean","last_fresh":"2024-09-14T06:33:35.112317+0000","last_change":"2024-09-14T06:33:31.825738+0000","last_active":"2024-09-14T06:33:35.112317+0000","last_peered":"2024-09-14T06:33:35.112317+0000","last_clean":"2024-09-14T06:33:35.112317+0000","last_became_active":"2024-09-14T06:33:31.823955+0000","last_became_peered":"2024-09-14T06:33:31.823955+0000","last_unstale":"2024-09-14T06:33:35.112317+0000","last_undegraded":"2024-09-14T06:33:35.112317+0000","last_fullsized":"2024-09-14T06:33:35.112317+0000","mapping_epoch":285,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":286,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":805,"log_dups_size":0,"ondisk_log_size":805,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T06:54:41.083735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00068758600000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":4953,"num_read_kb":43809,"num_write":946,"num_write_kb":45551,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":505,"num_bytes_recovered":44145197,"num_keys_recovered":280,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"259'960","reported_seq":11521,"reported_epoch":289,"state":"active+clean","last_fresh":"2024-09-14T06:33:34.979721+0000","last_change":"2024-09-14T06:33:07.706745+0000","last_active":"2024-09-14T06:33:34.979721+0000","last_peered":"2024-09-14T06:33:34.979721+0000","last_clean":"2024-09-14T06:33:34.979721+0000","last_became_active":"2024-09-14T06:29:00.586183+0000","last_became_peered":"2024-09-14T06:29:00.586183+0000","last_unstale":"2024-09-14T06:33:34.979721+0000","last_undegraded":"2024-09-14T06:33:34.979721+0000","last_fullsized":"2024-09-14T06:33:34.979721+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":960,"log_dups_size":0,"ondisk_log_size":960,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T10:55:09.317291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001810606,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":11910,"num_read_kb":108094,"num_write":1463,"num_write_kb":55102,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1a","version":"260'1678","reported_seq":10638,"reported_epoch":289,"state":"active+clean","last_fresh":"2024-09-14T06:33:36.039825+0000","last_change":"2024-09-14T06:33:07.705134+0000","last_active":"2024-09-14T06:33:36.039825+0000","last_peered":"2024-09-14T06:33:36.039825+0000","last_clean":"2024-09-14T06:33:36.039825+0000","last_became_active":"2024-09-14T06:29:00.576206+0000","last_became_peered":"2024-09-14T06:29:00.576206+0000","last_unstale":"2024-09-14T06:33:36.039825+0000","last_undegraded":"2024-09-14T06:33:36.039825+0000","last_fullsized":"2024-09-14T06:33:36.039825+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":1678,"log_dups_size":0,"ondisk_log_size":1678,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T10:47:38.190388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088286099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":13339,"num_read_kb":82904,"num_write":2451,"num_write_kb":55687,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.19","version":"261'561","reported_seq":6508,"reported_epoch":289,"state":"active+clean","last_fresh":"2024-09-14T06:33:34.831894+0000","last_change":"2024-09-14T06:33:07.704594+0000","last_active":"2024-09-14T06:33:34.831894+0000","last_peered":"2024-09-14T06:33:34.831894+0000","last_clean":"2024-09-14T06:33:34.831894+0000","last_became_active":"2024-09-14T06:31:53.744800+0000","last_became_peered":"2024-09-14T06:31:53.744800+0000","last_unstale":"2024-09-14T06:33:34.831894+0000","last_undegraded":"2024-09-14T06:33:34.831894+0000","last_fullsized":"2024-09-14T06:33:34.831894+0000","mapping_epoch":188,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":189,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":561,"log_dups_size":0,"ondisk_log_size":561,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T13:59:24.308993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062201200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":5123,"num_read_kb":67781,"num_write":764,"num_write_kb":43873,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":282,"num_bytes_recovered":41927489,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"262'547","reported_seq":6285,"reported_epoch":287,"state":"active+clean","last_fresh":"2024-09-14T06:33:32.818617+0000","last_change":"2024-09-14T06:33:07.769587+0000","last_active":"2024-09-14T06:33:32.818617+0000","last_peered":"2024-09-14T06:33:32.818617+0000","last_clean":"2024-09-14T06:33:32.818617+0000","last_became_active":"2024-09-14T06:31:53.745159+0000","last_became_peered":"2024-09-14T06:31:53.745159+0000","last_unstale":"2024-09-14T06:33:32.818617+0000","last_undegraded":"2024-09-14T06:33:32.818617+0000","last_fullsized":"2024-09-14T06:33:32.818617+0000","mapping_epoch":188,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":189,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":547,"log_dups_size":0,"ondisk_log_size":547,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T13:08:27.610323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00079631400000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4489,"num_read_kb":78917,"num_write":672,"num_write_kb":51581,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":310,"num_bytes_recovered":56082976,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"261'1037","reported_seq":9728,"reported_epoch":289,"state":"active+clean","last_fresh":"2024-09-14T06:33:34.979868+0000","last_change":"2024-09-14T06:33:07.706863+0000","last_active":"2024-09-14T06:33:34.979868+0000","last_peered":"2024-09-14T06:33:34.979868+0000","last_clean":"2024-09-14T06:33:34.979868+0000","last_became_active":"2024-09-14T06:29:00.579714+0000","last_became_peered":"2024-09-14T06:29:00.579714+0000","last_unstale":"2024-09-14T06:33:34.979868+0000","last_undegraded":"2024-09-14T06:33:34.979868+0000","last_fullsized":"2024-09-14T06:33:34.979868+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":1037,"log_dups_size":0,"ondisk_log_size":1037,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T15:17:51.610057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0017747889999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":10688,"num_read_kb":68964,"num_write":1162,"num_write_kb":48482,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.16","version":"255'1156","reported_seq":4137,"reported_epoch":289,"state":"active+clean","last_fresh":"2024-09-14T06:33:35.112491+0000","last_change":"2024-09-14T06:33:31.824083+0000","last_active":"2024-09-14T06:33:35.112491+0000","last_peered":"2024-09-14T06:33:35.112491+0000","last_clean":"2024-09-14T06:33:35.112491+0000","last_became_active":"2024-09-14T06:33:31.821620+0000","last_became_peered":"2024-09-14T06:33:31.821620+0000","last_unstale":"2024-09-14T06:33:35.112491+0000","last_undegraded":"2024-09-14T06:33:35.112491+0000","last_fullsized":"2024-09-14T06:33:35.112491+0000","mapping_epoch":285,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":286,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":1156,"log_dups_size":0,"ondisk_log_size":1156,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T08:10:02.587758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00073433000000000005,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":5658,"num_read_kb":38562,"num_write":1743,"num_write_kb":37248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"261'650","reported_seq":6751,"reported_epoch":289,"state":"active+clean","last_fresh":"2024-09-14T06:33:34.834516+0000","last_change":"2024-09-14T06:33:07.769581+0000","last_active":"2024-09-14T06:33:34.834516+0000","last_peered":"2024-09-14T06:33:34.834516+0000","last_clean":"2024-09-14T06:33:34.834516+0000","last_became_active":"2024-09-14T06:31:53.746091+0000","last_became_peered":"2024-09-14T06:31:53.746091+0000","last_unstale":"2024-09-14T06:33:34.834516+0000","last_undegraded":"2024-09-14T06:33:34.834516+0000","last_fullsized":"2024-09-14T06:33:34.834516+0000","mapping_epoch":188,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":189,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":650,"log_dups_size":0,"ondisk_log_size":650,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T12:36:31.690559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000442557,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":5101,"num_read_kb":74899,"num_write":696,"num_write_kb":51196,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":271,"num_bytes_recovered":49981473,"num_keys_recovered":33,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.14","version":"261'1137","reported_seq":6341,"reported_epoch":289,"state":"active+clean","last_fresh":"2024-09-14T06:33:36.039887+0000","last_change":"2024-09-14T06:33:07.704760+0000","last_active":"2024-09-14T06:33:36.039887+0000","last_peered":"2024-09-14T06:33:36.039887+0000","last_clean":"2024-09-14T06:33:36.039887+0000","last_became_active":"2024-09-14T06:33:04.628663+0000","last_became_peered":"2024-09-14T06:33:04.628663+0000","last_unstale":"2024-09-14T06:33:36.039887+0000","last_undegraded":"2024-09-14T06:33:36.039887+0000","last_fullsized":"2024-09-14T06:33:36.039887+0000","mapping_epoch":258,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":259,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":1137,"log_dups_size":0,"ondisk_log_size":1137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T11:42:47.162926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00078945399999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":8948,"num_read_kb":46936,"num_write":1573,"num_write_kb":29829,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":296,"num_bytes_recovered":6118769,"num_keys_recovered":11,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":50,"reported_epoch":289,"state":"active+clean","last_fresh":"2024-09-14T06:33:36.039662+0000","last_change":"2024-09-14T06:33:13.859355+0000","last_active":"2024-09-14T06:33:36.039662+0000","last_peered":"2024-09-14T06:33:36.039662+0000","last_clean":"2024-09-14T06:33:36.039662+0000","last_became_active":"2024-09-14T06:33:13.859069+0000","last_became_peered":"2024-09-14T06:33:13.859069+0000","last_unstale":"2024-09-14T06:33:36.039662+0000","last_undegraded":"2024-09-14T06:33:36.039662+0000","last_fullsized":"2024-09-14T06:33:36.039662+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:28:50.205350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:28:50.205350+0000","last_clean_scrub_stamp":"2024-09-14T06:28:50.205350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T17:08:35.416194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"262'893","reported_seq":8726,"reported_epoch":289,"state":"active+clean","last_fresh":"2024-09-14T06:33:34.979753+0000","last_change":"2024-09-14T06:33:07.705814+0000","last_active":"2024-09-14T06:33:34.979753+0000","last_peered":"2024-09-14T06:33:34.979753+0000","last_clean":"2024-09-14T06:33:34.979753+0000","last_became_active":"2024-09-14T06:29:00.578283+0000","last_became_peered":"2024-09-14T06:29:00.578283+0000","last_unstale":"2024-09-14T06:33:34.979753+0000","last_undegraded":"2024-09-14T06:33:34.979753+0000","last_fullsized":"2024-09-14T06:33:34.979753+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":893,"log_dups_size":0,"ondisk_log_size":893,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T15:09:37.855181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001076554,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":8608,"num_read_kb":78770,"num_write":923,"num_write_kb":48303,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":567,"reported_epoch":289,"state":"active+clean","last_fresh":"2024-09-14T06:33:34.831888+0000","last_change":"2024-09-14T06:33:13.734687+0000","last_active":"2024-09-14T06:33:34.831888+0000","last_peered":"2024-09-14T06:33:34.831888+0000","last_clean":"2024-09-14T06:33:34.831888+0000","last_became_active":"2024-09-14T06:33:13.734442+0000","last_became_peered":"2024-09-14T06:33:13.734442+0000","last_unstale":"2024-09-14T06:33:34.831888+0000","last_undegraded":"2024-09-14T06:33:34.831888+0000","last_fullsized":"2024-09-14T06:33:34.831888+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:28:50.205350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:28:50.205350+0000","last_clean_scrub_stamp":"2024-09-14T06:28:50.205350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T12:50:57.166258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065018200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"261'840","reported_seq":4649,"reported_epoch":287,"state":"active+clean","last_fresh":"2024-09-14T06:33:32.819115+0000","last_change":"2024-09-14T06:33:07.769882+0000","last_active":"2024-09-14T06:33:32.819115+0000","last_peered":"2024-09-14T06:33:32.819115+0000","last_clean":"2024-09-14T06:33:32.819115+0000","last_became_active":"2024-09-14T06:31:54.747923+0000","last_became_peered":"2024-09-14T06:31:54.747923+0000","last_unstale":"2024-09-14T06:33:32.819115+0000","last_undegraded":"2024-09-14T06:33:32.819115+0000","last_fullsized":"2024-09-14T06:33:32.819115+0000","mapping_epoch":189,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":190,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":840,"log_dups_size":0,"ondisk_log_size":840,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T09:01:49.581187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000927559,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4591,"num_read_kb":41129,"num_write":1178,"num_write_kb":36122,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":183,"num_bytes_recovered":13912513,"num_keys_recovered":22,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":554,"reported_epoch":289,"state":"active+clean","last_fresh":"2024-09-14T06:33:34.834701+0000","last_change":"2024-09-14T06:28:53.640035+0000","last_active":"2024-09-14T06:33:34.834701+0000","last_peered":"2024-09-14T06:33:34.834701+0000","last_clean":"2024-09-14T06:33:34.834701+0000","last_became_active":"2024-09-14T06:28:51.308058+0000","last_became_peered":"2024-09-14T06:28:51.308058+0000","last_unstale":"2024-09-14T06:33:34.834701+0000","last_undegraded":"2024-09-14T06:33:34.834701+0000","last_fullsized":"2024-09-14T06:33:34.834701+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:28:50.205350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:28:50.205350+0000","last_clean_scrub_stamp":"2024-09-14T06:28:50.205350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T13:35:48.544057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00037824000000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"262'700","reported_seq":4876,"reported_epoch":289,"state":"active+clean","last_fresh":"2024-09-14T06:33:34.832050+0000","last_change":"2024-09-14T06:33:22.769657+0000","last_active":"2024-09-14T06:33:34.832050+0000","last_peered":"2024-09-14T06:33:34.832050+0000","last_clean":"2024-09-14T06:33:34.832050+0000","last_became_active":"2024-09-14T06:33:22.767544+0000","last_became_peered":"2024-09-14T06:33:22.767544+0000","last_unstale":"2024-09-14T06:33:34.832050+0000","last_undegraded":"2024-09-14T06:33:34.832050+0000","last_fullsized":"2024-09-14T06:33:34.832050+0000","mapping_epoch":276,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":277,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":700,"log_dups_size":0,"ondisk_log_size":700,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T13:09:29.858808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00074004799999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":2654,"num_read_kb":37435,"num_write":897,"num_write_kb":40860,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":687,"num_bytes_recovered":31411166,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":555,"reported_epoch":289,"state":"active+clean","last_fresh":"2024-09-14T06:33:34.832080+0000","last_change":"2024-09-14T06:28:53.226072+0000","last_active":"2024-09-14T06:33:34.832080+0000","last_peered":"2024-09-14T06:33:34.832080+0000","last_clean":"2024-09-14T06:33:34.832080+0000","last_became_active":"2024-09-14T06:28:51.308072+0000","last_became_peered":"2024-09-14T06:28:51.308072+0000","last_unstale":"2024-09-14T06:33:34.832080+0000","last_undegraded":"2024-09-14T06:33:34.832080+0000","last_fullsized":"2024-09-14T06:33:34.832080+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:28:50.205350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:28:50.205350+0000","last_clean_scrub_stamp":"2024-09-14T06:28:50.205350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T13:15:09.052328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066197399999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"261'896","reported_seq":5625,"reported_epoch":289,"state":"active+clean","last_fresh":"2024-09-14T06:33:35.112603+0000","last_change":"2024-09-14T06:33:31.823904+0000","last_active":"2024-09-14T06:33:35.112603+0000","last_peered":"2024-09-14T06:33:35.112603+0000","last_clean":"2024-09-14T06:33:35.112603+0000","last_became_active":"2024-09-14T06:33:31.821533+0000","last_became_peered":"2024-09-14T06:33:31.821533+0000","last_unstale":"2024-09-14T06:33:35.112603+0000","last_undegraded":"2024-09-14T06:33:35.112603+0000","last_fullsized":"2024-09-14T06:33:35.112603+0000","mapping_epoch":285,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":286,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":896,"log_dups_size":0,"ondisk_log_size":896,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T15:48:46.733188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015088129999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4516,"num_read_kb":42632,"num_write":1255,"num_write_kb":49494,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":330,"num_bytes_recovered":28798893,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"262'711","reported_seq":10924,"reported_epoch":289,"state":"active+clean","last_fresh":"2024-09-14T06:33:36.040034+0000","last_change":"2024-09-14T06:33:07.704808+0000","last_active":"2024-09-14T06:33:36.040034+0000","last_peered":"2024-09-14T06:33:36.040034+0000","last_clean":"2024-09-14T06:33:36.040034+0000","last_became_active":"2024-09-14T06:29:00.584678+0000","last_became_peered":"2024-09-14T06:29:00.584678+0000","last_unstale":"2024-09-14T06:33:36.040034+0000","last_undegraded":"2024-09-14T06:33:36.040034+0000","last_fullsized":"2024-09-14T06:33:36.040034+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":711,"log_dups_size":0,"ondisk_log_size":711,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T07:39:08.849874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00074987800000000005,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":9560,"num_read_kb":89311,"num_write":777,"num_write_kb":51691,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"22'2","reported_seq":428,"reported_epoch":289,"state":"active+clean","last_fresh":"2024-09-14T06:33:35.112361+0000","last_change":"2024-09-14T06:33:31.823823+0000","last_active":"2024-09-14T06:33:35.112361+0000","last_peered":"2024-09-14T06:33:35.112361+0000","last_clean":"2024-09-14T06:33:35.112361+0000","last_became_active":"2024-09-14T06:33:31.823429+0000","last_became_peered":"2024-09-14T06:33:31.823429+0000","last_unstale":"2024-09-14T06:33:35.112361+0000","last_undegraded":"2024-09-14T06:33:35.112361+0000","last_fullsized":"2024-09-14T06:33:35.112361+0000","mapping_epoch":285,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":286,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:28:50.205350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:28:50.205350+0000","last_clean_scrub_stamp":"2024-09-14T06:28:50.205350+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T18:01:36.855427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010072970000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.0","version":"260'1116","reported_seq":8244,"reported_epoch":289,"state":"active+clean","last_fresh":"2024-09-14T06:33:34.832141+0000","last_change":"2024-09-14T06:33:22.769183+0000","last_active":"2024-09-14T06:33:34.832141+0000","last_peered":"2024-09-14T06:33:34.832141+0000","last_clean":"2024-09-14T06:33:34.832141+0000","last_became_active":"2024-09-14T06:33:22.767520+0000","last_became_peered":"2024-09-14T06:33:22.767520+0000","last_unstale":"2024-09-14T06:33:34.832141+0000","last_undegraded":"2024-09-14T06:33:34.832141+0000","last_fullsized":"2024-09-14T06:33:34.832141+0000","mapping_epoch":276,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":277,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":1116,"log_dups_size":0,"ondisk_log_size":1116,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T17:26:46.362756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064233900000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":8259,"num_read_kb":85480,"num_write":1652,"num_write_kb":58700,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":537,"num_bytes_recovered":2852,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":40,"reported_epoch":289,"state":"active+clean","last_fresh":"2024-09-14T06:33:34.831819+0000","last_change":"2024-09-14T06:33:19.801852+0000","last_active":"2024-09-14T06:33:34.831819+0000","last_peered":"2024-09-14T06:33:34.831819+0000","last_clean":"2024-09-14T06:33:34.831819+0000","last_became_active":"2024-09-14T06:33:19.801557+0000","last_became_peered":"2024-09-14T06:33:19.801557+0000","last_unstale":"2024-09-14T06:33:34.831819+0000","last_undegraded":"2024-09-14T06:33:34.831819+0000","last_fullsized":"2024-09-14T06:33:34.831819+0000","mapping_epoch":273,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":274,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:28:50.205350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:28:50.205350+0000","last_clean_scrub_stamp":"2024-09-14T06:28:50.205350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T08:41:03.294249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053403100000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1","version":"261'822","reported_seq":11886,"reported_epoch":289,"state":"active+clean","last_fresh":"2024-09-14T06:33:34.979564+0000","last_change":"2024-09-14T06:33:07.705685+0000","last_active":"2024-09-14T06:33:34.979564+0000","last_peered":"2024-09-14T06:33:34.979564+0000","last_clean":"2024-09-14T06:33:34.979564+0000","last_became_active":"2024-09-14T06:30:57.768238+0000","last_became_peered":"2024-09-14T06:30:57.768238+0000","last_unstale":"2024-09-14T06:33:34.979564+0000","last_undegraded":"2024-09-14T06:33:34.979564+0000","last_fullsized":"2024-09-14T06:33:34.979564+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":822,"log_dups_size":0,"ondisk_log_size":822,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T15:07:18.804416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011933110000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":11284,"num_read_kb":112430,"num_write":991,"num_write_kb":47408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":183,"num_bytes_recovered":30431994,"num_keys_recovered":22,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":554,"reported_epoch":289,"state":"active+clean","last_fresh":"2024-09-14T06:33:34.834417+0000","last_change":"2024-09-14T06:28:53.640096+0000","last_active":"2024-09-14T06:33:34.834417+0000","last_peered":"2024-09-14T06:33:34.834417+0000","last_clean":"2024-09-14T06:33:34.834417+0000","last_became_active":"2024-09-14T06:28:51.223069+0000","last_became_peered":"2024-09-14T06:28:51.223069+0000","last_unstale":"2024-09-14T06:33:34.834417+0000","last_undegraded":"2024-09-14T06:33:34.834417+0000","last_fullsized":"2024-09-14T06:33:34.834417+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:28:50.205350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:28:50.205350+0000","last_clean_scrub_stamp":"2024-09-14T06:28:50.205350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T15:13:00.471509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00043714600000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"262'1663","reported_seq":7850,"reported_epoch":287,"state":"active+clean","last_fresh":"2024-09-14T06:33:32.818891+0000","last_change":"2024-09-14T06:33:07.770369+0000","last_active":"2024-09-14T06:33:32.818891+0000","last_peered":"2024-09-14T06:33:32.818891+0000","last_clean":"2024-09-14T06:33:32.818891+0000","last_became_active":"2024-09-14T06:31:54.744160+0000","last_became_peered":"2024-09-14T06:31:54.744160+0000","last_unstale":"2024-09-14T06:33:32.818891+0000","last_undegraded":"2024-09-14T06:33:32.818891+0000","last_fullsized":"2024-09-14T06:33:32.818891+0000","mapping_epoch":189,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":190,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":1663,"log_dups_size":0,"ondisk_log_size":1663,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T16:44:35.569843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001382608,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":10723,"num_read_kb":70589,"num_write":2596,"num_write_kb":53564,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":327,"num_bytes_recovered":50047063,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"20'1","reported_seq":427,"reported_epoch":289,"state":"active+clean","last_fresh":"2024-09-14T06:33:35.112503+0000","last_change":"2024-09-14T06:33:31.823426+0000","last_active":"2024-09-14T06:33:35.112503+0000","last_peered":"2024-09-14T06:33:35.112503+0000","last_clean":"2024-09-14T06:33:35.112503+0000","last_became_active":"2024-09-14T06:33:31.823032+0000","last_became_peered":"2024-09-14T06:33:31.823032+0000","last_unstale":"2024-09-14T06:33:35.112503+0000","last_undegraded":"2024-09-14T06:33:35.112503+0000","last_fullsized":"2024-09-14T06:33:35.112503+0000","mapping_epoch":285,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":286,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:28:50.205350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:28:50.205350+0000","last_clean_scrub_stamp":"2024-09-14T06:28:50.205350+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T08:31:58.917801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050808899999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'72","reported_seq":645,"reported_epoch":289,"state":"active+clean","last_fresh":"2024-09-14T06:33:34.834414+0000","last_change":"2024-09-14T06:28:44.839885+0000","last_active":"2024-09-14T06:33:34.834414+0000","last_peered":"2024-09-14T06:33:34.834414+0000","last_clean":"2024-09-14T06:33:34.834414+0000","last_became_active":"2024-09-14T06:28:44.839573+0000","last_became_peered":"2024-09-14T06:28:44.839573+0000","last_unstale":"2024-09-14T06:33:34.834414+0000","last_undegraded":"2024-09-14T06:33:34.834414+0000","last_fullsized":"2024-09-14T06:33:34.834414+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:28:43.828494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:28:43.828494+0000","last_clean_scrub_stamp":"2024-09-14T06:28:43.828494+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T14:41:25.787293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.8","version":"261'684","reported_seq":6657,"reported_epoch":287,"state":"active+clean","last_fresh":"2024-09-14T06:33:32.818645+0000","last_change":"2024-09-14T06:33:07.769379+0000","last_active":"2024-09-14T06:33:32.818645+0000","last_peered":"2024-09-14T06:33:32.818645+0000","last_clean":"2024-09-14T06:33:32.818645+0000","last_became_active":"2024-09-14T06:31:55.754694+0000","last_became_peered":"2024-09-14T06:31:55.754694+0000","last_unstale":"2024-09-14T06:33:32.818645+0000","last_undegraded":"2024-09-14T06:33:32.818645+0000","last_fullsized":"2024-09-14T06:33:32.818645+0000","mapping_epoch":190,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":191,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":684,"log_dups_size":0,"ondisk_log_size":684,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T15:48:47.633183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067851599999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":5793,"num_read_kb":90852,"num_write":921,"num_write_kb":57691,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":319,"num_bytes_recovered":60352269,"num_keys_recovered":22,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.9","version":"261'597","reported_seq":5173,"reported_epoch":289,"state":"active+clean","last_fresh":"2024-09-14T06:33:36.040020+0000","last_change":"2024-09-14T06:33:22.772981+0000","last_active":"2024-09-14T06:33:36.040020+0000","last_peered":"2024-09-14T06:33:36.040020+0000","last_clean":"2024-09-14T06:33:36.040020+0000","last_became_active":"2024-09-14T06:33:22.770956+0000","last_became_peered":"2024-09-14T06:33:22.770956+0000","last_unstale":"2024-09-14T06:33:36.040020+0000","last_undegraded":"2024-09-14T06:33:36.040020+0000","last_fullsized":"2024-09-14T06:33:36.040020+0000","mapping_epoch":276,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":277,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":597,"log_dups_size":0,"ondisk_log_size":597,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T17:14:06.174583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057139500000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3678,"num_read_kb":64291,"num_write":778,"num_write_kb":44928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":316,"num_bytes_recovered":49117,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"261'874","reported_seq":5176,"reported_epoch":289,"state":"active+clean","last_fresh":"2024-09-14T06:33:36.039739+0000","last_change":"2024-09-14T06:33:17.629900+0000","last_active":"2024-09-14T06:33:36.039739+0000","last_peered":"2024-09-14T06:33:36.039739+0000","last_clean":"2024-09-14T06:33:36.039739+0000","last_became_active":"2024-09-14T06:33:17.627910+0000","last_became_peered":"2024-09-14T06:33:17.627910+0000","last_unstale":"2024-09-14T06:33:36.039739+0000","last_undegraded":"2024-09-14T06:33:36.039739+0000","last_fullsized":"2024-09-14T06:33:36.039739+0000","mapping_epoch":271,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":272,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":874,"log_dups_size":0,"ondisk_log_size":874,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T14:22:28.835018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00074887399999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":5331,"num_read_kb":58329,"num_write":1221,"num_write_kb":47535,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":656,"num_bytes_recovered":26968802,"num_keys_recovered":45,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.b","version":"260'1637","reported_seq":8985,"reported_epoch":287,"state":"active+clean","last_fresh":"2024-09-14T06:33:32.818865+0000","last_change":"2024-09-14T06:33:07.769381+0000","last_active":"2024-09-14T06:33:32.818865+0000","last_peered":"2024-09-14T06:33:32.818865+0000","last_clean":"2024-09-14T06:33:32.818865+0000","last_became_active":"2024-09-14T06:31:55.756219+0000","last_became_peered":"2024-09-14T06:31:55.756219+0000","last_unstale":"2024-09-14T06:33:32.818865+0000","last_undegraded":"2024-09-14T06:33:32.818865+0000","last_fullsized":"2024-09-14T06:33:32.818865+0000","mapping_epoch":190,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":191,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":1637,"log_dups_size":0,"ondisk_log_size":1637,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T15:53:45.757678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065814200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":13043,"num_read_kb":64129,"num_write":2496,"num_write_kb":43679,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":291,"num_bytes_recovered":37370632,"num_keys_recovered":66,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.c","version":"255'622","reported_seq":6270,"reported_epoch":289,"state":"active+clean","last_fresh":"2024-09-14T06:33:35.112517+0000","last_change":"2024-09-14T06:33:31.824073+0000","last_active":"2024-09-14T06:33:35.112517+0000","last_peered":"2024-09-14T06:33:35.112517+0000","last_clean":"2024-09-14T06:33:35.112517+0000","last_became_active":"2024-09-14T06:33:31.821638+0000","last_became_peered":"2024-09-14T06:33:31.821638+0000","last_unstale":"2024-09-14T06:33:35.112517+0000","last_undegraded":"2024-09-14T06:33:35.112517+0000","last_fullsized":"2024-09-14T06:33:35.112517+0000","mapping_epoch":285,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":286,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":622,"log_dups_size":0,"ondisk_log_size":622,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T09:06:48.668360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013231740000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4842,"num_read_kb":50425,"num_write":795,"num_write_kb":40827,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":293,"num_bytes_recovered":36712421,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.d","version":"261'577","reported_seq":9328,"reported_epoch":289,"state":"active+clean","last_fresh":"2024-09-14T06:33:34.834667+0000","last_change":"2024-09-14T06:33:07.770219+0000","last_active":"2024-09-14T06:33:34.834667+0000","last_peered":"2024-09-14T06:33:34.834667+0000","last_clean":"2024-09-14T06:33:34.834667+0000","last_became_active":"2024-09-14T06:29:00.580362+0000","last_became_peered":"2024-09-14T06:29:00.580362+0000","last_unstale":"2024-09-14T06:33:34.834667+0000","last_undegraded":"2024-09-14T06:33:34.834667+0000","last_fullsized":"2024-09-14T06:33:34.834667+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":577,"log_dups_size":0,"ondisk_log_size":577,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T16:36:38.133044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010003270000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":8199,"num_read_kb":93694,"num_write":732,"num_write_kb":41032,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.e","version":"262'586","reported_seq":5526,"reported_epoch":289,"state":"active+clean","last_fresh":"2024-09-14T06:33:34.834589+0000","last_change":"2024-09-14T06:33:07.769825+0000","last_active":"2024-09-14T06:33:34.834589+0000","last_peered":"2024-09-14T06:33:34.834589+0000","last_clean":"2024-09-14T06:33:34.834589+0000","last_became_active":"2024-09-14T06:32:39.777524+0000","last_became_peered":"2024-09-14T06:32:39.777524+0000","last_unstale":"2024-09-14T06:33:34.834589+0000","last_undegraded":"2024-09-14T06:33:34.834589+0000","last_fullsized":"2024-09-14T06:33:34.834589+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":586,"log_dups_size":0,"ondisk_log_size":586,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T07:31:19.417437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065560200000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3986,"num_read_kb":61138,"num_write":753,"num_write_kb":43787,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":171,"num_bytes_recovered":10913227,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.f","version":"262'712","reported_seq":5948,"reported_epoch":289,"state":"active+clean","last_fresh":"2024-09-14T06:33:34.834579+0000","last_change":"2024-09-14T06:33:07.770138+0000","last_active":"2024-09-14T06:33:34.834579+0000","last_peered":"2024-09-14T06:33:34.834579+0000","last_clean":"2024-09-14T06:33:34.834579+0000","last_became_active":"2024-09-14T06:29:00.586491+0000","last_became_peered":"2024-09-14T06:29:00.586491+0000","last_unstale":"2024-09-14T06:33:34.834579+0000","last_undegraded":"2024-09-14T06:33:34.834579+0000","last_fullsized":"2024-09-14T06:33:34.834579+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":712,"log_dups_size":0,"ondisk_log_size":712,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T14:20:12.511497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000891002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":6081,"num_read_kb":45578,"num_write":836,"num_write_kb":36134,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.10","version":"261'952","reported_seq":6889,"reported_epoch":289,"state":"active+clean","last_fresh":"2024-09-14T06:33:36.039625+0000","last_change":"2024-09-14T06:33:16.630633+0000","last_active":"2024-09-14T06:33:36.039625+0000","last_peered":"2024-09-14T06:33:36.039625+0000","last_clean":"2024-09-14T06:33:36.039625+0000","last_became_active":"2024-09-14T06:33:16.628480+0000","last_became_peered":"2024-09-14T06:33:16.628480+0000","last_unstale":"2024-09-14T06:33:36.039625+0000","last_undegraded":"2024-09-14T06:33:36.039625+0000","last_fullsized":"2024-09-14T06:33:36.039625+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":952,"log_dups_size":0,"ondisk_log_size":952,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T17:09:51.353193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012025289999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":7147,"num_read_kb":34474,"num_write":1082,"num_write_kb":36655,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":318,"num_bytes_recovered":9401812,"num_keys_recovered":45,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.11","version":"261'730","reported_seq":11746,"reported_epoch":289,"state":"active+clean","last_fresh":"2024-09-14T06:33:34.834479+0000","last_change":"2024-09-14T06:33:07.769975+0000","last_active":"2024-09-14T06:33:34.834479+0000","last_peered":"2024-09-14T06:33:34.834479+0000","last_clean":"2024-09-14T06:33:34.834479+0000","last_became_active":"2024-09-14T06:32:39.778010+0000","last_became_peered":"2024-09-14T06:32:39.778010+0000","last_unstale":"2024-09-14T06:33:34.834479+0000","last_undegraded":"2024-09-14T06:33:34.834479+0000","last_fullsized":"2024-09-14T06:33:34.834479+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":730,"log_dups_size":0,"ondisk_log_size":730,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T10:48:00.264177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00072597299999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":10271,"num_read_kb":109243,"num_write":859,"num_write_kb":48307,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":177,"num_bytes_recovered":16647747,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.12","version":"261'782","reported_seq":7505,"reported_epoch":289,"state":"active+clean","last_fresh":"2024-09-14T06:33:34.979599+0000","last_change":"2024-09-14T06:33:07.706504+0000","last_active":"2024-09-14T06:33:34.979599+0000","last_peered":"2024-09-14T06:33:34.979599+0000","last_clean":"2024-09-14T06:33:34.979599+0000","last_became_active":"2024-09-14T06:29:00.579826+0000","last_became_peered":"2024-09-14T06:29:00.579826+0000","last_unstale":"2024-09-14T06:33:34.979599+0000","last_undegraded":"2024-09-14T06:33:34.979599+0000","last_fullsized":"2024-09-14T06:33:34.979599+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":782,"log_dups_size":0,"ondisk_log_size":782,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T11:32:57.816012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001680792,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":7110,"num_read_kb":59089,"num_write":947,"num_write_kb":42441,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.13","version":"261'1008","reported_seq":8959,"reported_epoch":289,"state":"active+clean","last_fresh":"2024-09-14T06:33:34.834443+0000","last_change":"2024-09-14T06:33:07.769558+0000","last_active":"2024-09-14T06:33:34.834443+0000","last_peered":"2024-09-14T06:33:34.834443+0000","last_clean":"2024-09-14T06:33:34.834443+0000","last_became_active":"2024-09-14T06:29:00.586520+0000","last_became_peered":"2024-09-14T06:29:00.586520+0000","last_unstale":"2024-09-14T06:33:34.834443+0000","last_undegraded":"2024-09-14T06:33:34.834443+0000","last_fullsized":"2024-09-14T06:33:34.834443+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":1008,"log_dups_size":0,"ondisk_log_size":1008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T07:25:04.666000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041539399999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":9551,"num_read_kb":66119,"num_write":1235,"num_write_kb":47437,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":19,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":240092,"num_read_kb":2184933,"num_write":38145,"num_write_kb":1478386,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7477,"num_bytes_recovered":638497624,"num_keys_recovered":546,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":28648,"ondisk_log_size":28648,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":131072,"data_stored":1180736,"data_compressed":65166,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509503,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":70776,"kb_used_data":3484,"kb_used_omap":1,"kb_used_meta":67262,"kb_avail":94301064,"statfs":{"total":96636764160,"available":96564289536,"internally_reserved":0,"allocated":3567616,"data_stored":2317757,"data_compressed":32583,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":1590,"internal_metadata":68876746},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509503,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":56816,"kb_used_data":3420,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94315024,"statfs":{"total":96636764160,"available":96578584576,"internally_reserved":0,"allocated":3502080,"data_stored":1727389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":54655430},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509504,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":74868,"kb_used_data":3424,"kb_used_omap":1,"kb_used_meta":71422,"kb_avail":94296972,"statfs":{"total":96636764160,"available":96560099328,"internally_reserved":0,"allocated":3506176,"data_stored":1727408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":73136581},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509503,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":88756,"kb_used_data":3424,"kb_used_omap":1,"kb_used_meta":85310,"kb_avail":94283084,"statfs":{"total":96636764160,"available":96545878016,"internally_reserved":0,"allocated":3506176,"data_stored":1727408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":87357893},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509504,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":56828,"kb_used_data":3428,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94315012,"statfs":{"total":96636764160,"available":96578572288,"internally_reserved":0,"allocated":3510272,"data_stored":1727427,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":14,"seq":60129542207,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":60976,"kb_used_data":3420,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94310864,"statfs":{"total":96636764160,"available":96574324736,"internally_reserved":0,"allocated":3502080,"data_stored":1727389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":58915270},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542206,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":83252,"kb_used_data":3484,"kb_used_omap":1,"kb_used_meta":79742,"kb_avail":94288588,"statfs":{"total":96636764160,"available":96551514112,"internally_reserved":0,"allocated":3567616,"data_stored":2317757,"data_compressed":32583,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":1595,"internal_metadata":81656261},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574911,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":83168,"kb_used_data":3396,"kb_used_omap":1,"kb_used_meta":79742,"kb_avail":94288672,"statfs":{"total":96636764160,"available":96551600128,"internally_reserved":0,"allocated":3477504,"data_stored":1715589,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":81656263},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32583,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32583,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-14T06:33:40.930 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-14T06:33:41.212 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-14T06:33:41.212 INFO:teuthology.orchestra.run.smithi094.stdout:{"epoch":289,"fsid":"200ad3da-bd63-4e04-af74-efb5ca0af75e","created":"2024-09-14T06:28:30.782046+0000","modified":"2024-09-14T06:33:34.828528+0000","last_up_change":"2024-09-14T06:28:42.827239+0000","last_in_change":"2024-09-14T06:33:18.740365+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-14T06:28:43.295990+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2024-09-14T06:28:49.451714+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":2,"snap_epoch":22,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"pool_client.0","create_time":"2024-09-14T06:28:59.688535+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"262","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":92,"snap_epoch":262,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"8b22a249-b842-406a-a02c-c0f31f57b635","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":269,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6824","nonce":1809564923},{"type":"v1","addr":"172.21.15.94:6825","nonce":1809564923}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6826","nonce":1809564923},{"type":"v1","addr":"172.21.15.94:6827","nonce":1809564923}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6830","nonce":1809564923},{"type":"v1","addr":"172.21.15.94:6831","nonce":1809564923}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6828","nonce":1809564923},{"type":"v1","addr":"172.21.15.94:6829","nonce":1809564923}]},"public_addr":"172.21.15.94:6825/1809564923","cluster_addr":"172.21.15.94:6827/1809564923","heartbeat_back_addr":"172.21.15.94:6831/1809564923","heartbeat_front_addr":"172.21.15.94:6829/1809564923","state":["exists","up"]},{"osd":1,"uuid":"02251a03-84a0-414d-a980-fac4fd043cb6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":276,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6816","nonce":736402652},{"type":"v1","addr":"172.21.15.94:6817","nonce":736402652}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6818","nonce":736402652},{"type":"v1","addr":"172.21.15.94:6819","nonce":736402652}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6822","nonce":736402652},{"type":"v1","addr":"172.21.15.94:6823","nonce":736402652}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6820","nonce":736402652},{"type":"v1","addr":"172.21.15.94:6821","nonce":736402652}]},"public_addr":"172.21.15.94:6817/736402652","cluster_addr":"172.21.15.94:6819/736402652","heartbeat_back_addr":"172.21.15.94:6823/736402652","heartbeat_front_addr":"172.21.15.94:6821/736402652","state":["exists","up"]},{"osd":2,"uuid":"d45bf470-bb29-4739-8294-cd122b2b4102","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":273,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6808","nonce":1063204782},{"type":"v1","addr":"172.21.15.94:6809","nonce":1063204782}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6810","nonce":1063204782},{"type":"v1","addr":"172.21.15.94:6811","nonce":1063204782}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6814","nonce":1063204782},{"type":"v1","addr":"172.21.15.94:6815","nonce":1063204782}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6812","nonce":1063204782},{"type":"v1","addr":"172.21.15.94:6813","nonce":1063204782}]},"public_addr":"172.21.15.94:6809/1063204782","cluster_addr":"172.21.15.94:6811/1063204782","heartbeat_back_addr":"172.21.15.94:6815/1063204782","heartbeat_front_addr":"172.21.15.94:6813/1063204782","state":["exists","up"]},{"osd":3,"uuid":"456ec2d9-c438-4243-b931-335704f661eb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":270,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6800","nonce":3382046754},{"type":"v1","addr":"172.21.15.94:6801","nonce":3382046754}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6802","nonce":3382046754},{"type":"v1","addr":"172.21.15.94:6803","nonce":3382046754}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6806","nonce":3382046754},{"type":"v1","addr":"172.21.15.94:6807","nonce":3382046754}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6804","nonce":3382046754},{"type":"v1","addr":"172.21.15.94:6805","nonce":3382046754}]},"public_addr":"172.21.15.94:6801/3382046754","cluster_addr":"172.21.15.94:6803/3382046754","heartbeat_back_addr":"172.21.15.94:6807/3382046754","heartbeat_front_addr":"172.21.15.94:6805/3382046754","state":["exists","up"]},{"osd":4,"uuid":"2dab948e-c6e4-4437-bf17-017d1ac6821c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":276,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6810","nonce":501726729},{"type":"v1","addr":"172.21.15.138:6811","nonce":501726729}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6812","nonce":501726729},{"type":"v1","addr":"172.21.15.138:6813","nonce":501726729}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6816","nonce":501726729},{"type":"v1","addr":"172.21.15.138:6817","nonce":501726729}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6814","nonce":501726729},{"type":"v1","addr":"172.21.15.138:6815","nonce":501726729}]},"public_addr":"172.21.15.138:6811/501726729","cluster_addr":"172.21.15.138:6813/501726729","heartbeat_back_addr":"172.21.15.138:6817/501726729","heartbeat_front_addr":"172.21.15.138:6815/501726729","state":["exists","up"]},{"osd":5,"uuid":"8caf3aa3-ad87-4d33-b82e-4c15195176fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":285,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6826","nonce":1599292735},{"type":"v1","addr":"172.21.15.138:6827","nonce":1599292735}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6828","nonce":1599292735},{"type":"v1","addr":"172.21.15.138:6829","nonce":1599292735}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6832","nonce":1599292735},{"type":"v1","addr":"172.21.15.138:6833","nonce":1599292735}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6830","nonce":1599292735},{"type":"v1","addr":"172.21.15.138:6831","nonce":1599292735}]},"public_addr":"172.21.15.138:6827/1599292735","cluster_addr":"172.21.15.138:6829/1599292735","heartbeat_back_addr":"172.21.15.138:6833/1599292735","heartbeat_front_addr":"172.21.15.138:6831/1599292735","state":["exists","up"]},{"osd":6,"uuid":"e55d11fd-433d-4462-93a7-d5e84560221b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":271,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6818","nonce":163531479},{"type":"v1","addr":"172.21.15.138:6819","nonce":163531479}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6820","nonce":163531479},{"type":"v1","addr":"172.21.15.138:6821","nonce":163531479}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6824","nonce":163531479},{"type":"v1","addr":"172.21.15.138:6825","nonce":163531479}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6822","nonce":163531479},{"type":"v1","addr":"172.21.15.138:6823","nonce":163531479}]},"public_addr":"172.21.15.138:6819/163531479","cluster_addr":"172.21.15.138:6821/163531479","heartbeat_back_addr":"172.21.15.138:6825/163531479","heartbeat_front_addr":"172.21.15.138:6823/163531479","state":["exists","up"]},{"osd":7,"uuid":"ba14ba69-2c6c-4d09-81e2-2593dc3a7ea9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":255,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6800","nonce":218507235},{"type":"v1","addr":"172.21.15.138:6801","nonce":218507235}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6802","nonce":218507235},{"type":"v1","addr":"172.21.15.138:6803","nonce":218507235}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6806","nonce":218507235},{"type":"v1","addr":"172.21.15.138:6807","nonce":218507235}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6804","nonce":218507235},{"type":"v1","addr":"172.21.15.138:6805","nonce":218507235}]},"public_addr":"172.21.15.138:6801/218507235","cluster_addr":"172.21.15.138:6803/218507235","heartbeat_back_addr":"172.21.15.138:6807/218507235","heartbeat_front_addr":"172.21.15.138:6805/218507235","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-14T06:28:39.906639+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-14T06:28:40.541279+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-14T06:28:40.025350+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-14T06:28:39.217944+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-14T06:28:40.185898+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-14T06:28:40.783504+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-14T06:28:40.527305+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-14T06:28:40.349676+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.e","mappings":[{"from":4,"to":1}]},{"pgid":"3.11","mappings":[{"from":4,"to":1}]},{"pgid":"3.1c","mappings":[{"from":4,"to":1}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-14T06:33:42.226 INFO:tasks.ceph:Scrubbing osd.0 2024-09-14T06:33:42.226 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 config set osd_debug_deep_scrub_sleep 0 2024-09-14T06:33:42.361 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-14T06:33:42.361 INFO:teuthology.orchestra.run.smithi094.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-14T06:33:42.361 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-14T06:33:42.371 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 0 2024-09-14T06:33:42.656 INFO:teuthology.orchestra.run.smithi094.stderr:instructed osd(s) 0 to deep-scrub 2024-09-14T06:33:42.669 INFO:tasks.ceph:Scrubbing osd.1 2024-09-14T06:33:42.669 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 config set osd_debug_deep_scrub_sleep 0 2024-09-14T06:33:42.790 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-14T06:33:42.791 INFO:teuthology.orchestra.run.smithi094.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-14T06:33:42.791 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-14T06:33:42.801 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 1 2024-09-14T06:33:43.086 INFO:teuthology.orchestra.run.smithi094.stderr:instructed osd(s) 1 to deep-scrub 2024-09-14T06:33:43.099 INFO:tasks.ceph:Scrubbing osd.2 2024-09-14T06:33:43.099 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 config set osd_debug_deep_scrub_sleep 0 2024-09-14T06:33:43.221 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-14T06:33:43.222 INFO:teuthology.orchestra.run.smithi094.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-14T06:33:43.222 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-14T06:33:43.232 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 2 2024-09-14T06:33:43.518 INFO:teuthology.orchestra.run.smithi094.stderr:instructed osd(s) 2 to deep-scrub 2024-09-14T06:33:43.531 INFO:tasks.ceph:Scrubbing osd.3 2024-09-14T06:33:43.531 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 config set osd_debug_deep_scrub_sleep 0 2024-09-14T06:33:43.654 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-14T06:33:43.654 INFO:teuthology.orchestra.run.smithi094.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-14T06:33:43.654 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-14T06:33:43.665 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 3 2024-09-14T06:33:43.950 INFO:teuthology.orchestra.run.smithi094.stderr:instructed osd(s) 3 to deep-scrub 2024-09-14T06:33:43.964 INFO:tasks.ceph:Scrubbing osd.4 2024-09-14T06:33:43.964 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 config set osd_debug_deep_scrub_sleep 0 2024-09-14T06:33:44.087 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-14T06:33:44.087 INFO:teuthology.orchestra.run.smithi094.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-14T06:33:44.087 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-14T06:33:44.098 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 4 2024-09-14T06:33:44.383 INFO:teuthology.orchestra.run.smithi094.stderr:instructed osd(s) 4 to deep-scrub 2024-09-14T06:33:44.396 INFO:tasks.ceph:Scrubbing osd.5 2024-09-14T06:33:44.396 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 config set osd_debug_deep_scrub_sleep 0 2024-09-14T06:33:44.521 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-14T06:33:44.521 INFO:teuthology.orchestra.run.smithi094.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-14T06:33:44.521 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-14T06:33:44.531 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 5 2024-09-14T06:33:44.816 INFO:teuthology.orchestra.run.smithi094.stderr:instructed osd(s) 5 to deep-scrub 2024-09-14T06:33:44.829 INFO:tasks.ceph:Scrubbing osd.6 2024-09-14T06:33:44.829 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 config set osd_debug_deep_scrub_sleep 0 2024-09-14T06:33:44.954 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-14T06:33:44.954 INFO:teuthology.orchestra.run.smithi094.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-14T06:33:44.955 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-14T06:33:44.965 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 6 2024-09-14T06:33:45.249 INFO:teuthology.orchestra.run.smithi094.stderr:instructed osd(s) 6 to deep-scrub 2024-09-14T06:33:45.263 INFO:tasks.ceph:Scrubbing osd.7 2024-09-14T06:33:45.263 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 config set osd_debug_deep_scrub_sleep 0 2024-09-14T06:33:45.386 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-09-14T06:33:45.386 INFO:teuthology.orchestra.run.smithi094.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-14T06:33:45.387 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-09-14T06:33:45.397 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 7 2024-09-14T06:33:45.682 INFO:teuthology.orchestra.run.smithi094.stderr:instructed osd(s) 7 to deep-scrub 2024-09-14T06:33:45.695 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-14T06:33:45.981 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-14T06:33:45.982 INFO:teuthology.orchestra.run.smithi094.stderr:dumped all 2024-09-14T06:33:45.993 INFO:teuthology.orchestra.run.smithi094.stdout:{"pg_ready":true,"pg_map":{"version":435,"stamp":"2024-09-14T06:33:45.338102+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590406,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":240158,"num_read_kb":2184988,"num_write":38248,"num_write_kb":1480016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7477,"num_bytes_recovered":638497624,"num_keys_recovered":546,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":28723,"ondisk_log_size":28723,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":575456,"kb_used_data":27504,"kb_used_omap":12,"kb_used_meta":547763,"kb_avail":754399264,"statfs":{"total":773094113280,"available":772504846336,"internally_reserved":0,"allocated":28164096,"data_stored":14999924,"data_compressed":65166,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":12751,"internal_metadata":560909873},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.505032"},"pg_stats":[{"pgid":"3.1f","version":"262'855","reported_seq":8431,"reported_epoch":289,"state":"active+clean","last_fresh":"2024-09-14T06:33:34.979756+0000","last_change":"2024-09-14T06:33:07.706447+0000","last_active":"2024-09-14T06:33:34.979756+0000","last_peered":"2024-09-14T06:33:34.979756+0000","last_clean":"2024-09-14T06:33:34.979756+0000","last_became_active":"2024-09-14T06:29:00.579883+0000","last_became_peered":"2024-09-14T06:29:00.579883+0000","last_unstale":"2024-09-14T06:33:34.979756+0000","last_undegraded":"2024-09-14T06:33:34.979756+0000","last_fullsized":"2024-09-14T06:33:34.979756+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":855,"log_dups_size":0,"ondisk_log_size":855,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T14:40:25.634308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001666641,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":8468,"num_read_kb":86898,"num_write":1171,"num_write_kb":49543,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"262'783","reported_seq":10854,"reported_epoch":289,"state":"active+clean","last_fresh":"2024-09-14T06:33:36.040772+0000","last_change":"2024-09-14T06:33:16.628665+0000","last_active":"2024-09-14T06:33:36.040772+0000","last_peered":"2024-09-14T06:33:36.040772+0000","last_clean":"2024-09-14T06:33:36.040772+0000","last_became_active":"2024-09-14T06:33:16.626222+0000","last_became_peered":"2024-09-14T06:33:16.626222+0000","last_unstale":"2024-09-14T06:33:36.040772+0000","last_undegraded":"2024-09-14T06:33:36.040772+0000","last_fullsized":"2024-09-14T06:33:36.040772+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":783,"log_dups_size":0,"ondisk_log_size":783,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T15:32:44.526538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013639839999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":8315,"num_read_kb":67657,"num_write":1025,"num_write_kb":40407,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1025,"num_bytes_recovered":87221212,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1d","version":"260'1077","reported_seq":7485,"reported_epoch":289,"state":"active+clean","last_fresh":"2024-09-14T06:33:35.112385+0000","last_change":"2024-09-14T06:33:31.826989+0000","last_active":"2024-09-14T06:33:35.112385+0000","last_peered":"2024-09-14T06:33:35.112385+0000","last_clean":"2024-09-14T06:33:35.112385+0000","last_became_active":"2024-09-14T06:33:31.824454+0000","last_became_peered":"2024-09-14T06:33:31.824454+0000","last_unstale":"2024-09-14T06:33:35.112385+0000","last_undegraded":"2024-09-14T06:33:35.112385+0000","last_fullsized":"2024-09-14T06:33:35.112385+0000","mapping_epoch":285,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":286,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":1077,"log_dups_size":0,"ondisk_log_size":1077,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T07:07:12.782032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088638400000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":7873,"num_read_kb":64375,"num_write":1555,"num_write_kb":53292,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1c","version":"262'805","reported_seq":6034,"reported_epoch":289,"state":"active+clean","last_fresh":"2024-09-14T06:33:35.112317+0000","last_change":"2024-09-14T06:33:31.825738+0000","last_active":"2024-09-14T06:33:35.112317+0000","last_peered":"2024-09-14T06:33:35.112317+0000","last_clean":"2024-09-14T06:33:35.112317+0000","last_became_active":"2024-09-14T06:33:31.823955+0000","last_became_peered":"2024-09-14T06:33:31.823955+0000","last_unstale":"2024-09-14T06:33:35.112317+0000","last_undegraded":"2024-09-14T06:33:35.112317+0000","last_fullsized":"2024-09-14T06:33:35.112317+0000","mapping_epoch":285,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":286,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":805,"log_dups_size":0,"ondisk_log_size":805,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T06:54:41.083735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00068758600000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":4953,"num_read_kb":43809,"num_write":946,"num_write_kb":45551,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":505,"num_bytes_recovered":44145197,"num_keys_recovered":280,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"259'960","reported_seq":11521,"reported_epoch":289,"state":"active+clean","last_fresh":"2024-09-14T06:33:34.979721+0000","last_change":"2024-09-14T06:33:07.706745+0000","last_active":"2024-09-14T06:33:34.979721+0000","last_peered":"2024-09-14T06:33:34.979721+0000","last_clean":"2024-09-14T06:33:34.979721+0000","last_became_active":"2024-09-14T06:29:00.586183+0000","last_became_peered":"2024-09-14T06:29:00.586183+0000","last_unstale":"2024-09-14T06:33:34.979721+0000","last_undegraded":"2024-09-14T06:33:34.979721+0000","last_fullsized":"2024-09-14T06:33:34.979721+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":960,"log_dups_size":0,"ondisk_log_size":960,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T10:55:09.317291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001810606,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":11910,"num_read_kb":108094,"num_write":1463,"num_write_kb":55102,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1a","version":"260'1678","reported_seq":10638,"reported_epoch":289,"state":"active+clean","last_fresh":"2024-09-14T06:33:36.039825+0000","last_change":"2024-09-14T06:33:07.705134+0000","last_active":"2024-09-14T06:33:36.039825+0000","last_peered":"2024-09-14T06:33:36.039825+0000","last_clean":"2024-09-14T06:33:36.039825+0000","last_became_active":"2024-09-14T06:29:00.576206+0000","last_became_peered":"2024-09-14T06:29:00.576206+0000","last_unstale":"2024-09-14T06:33:36.039825+0000","last_undegraded":"2024-09-14T06:33:36.039825+0000","last_fullsized":"2024-09-14T06:33:36.039825+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":1678,"log_dups_size":0,"ondisk_log_size":1678,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T10:47:38.190388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088286099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":13339,"num_read_kb":82904,"num_write":2451,"num_write_kb":55687,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.19","version":"261'561","reported_seq":6508,"reported_epoch":289,"state":"active+clean","last_fresh":"2024-09-14T06:33:34.831894+0000","last_change":"2024-09-14T06:33:07.704594+0000","last_active":"2024-09-14T06:33:34.831894+0000","last_peered":"2024-09-14T06:33:34.831894+0000","last_clean":"2024-09-14T06:33:34.831894+0000","last_became_active":"2024-09-14T06:31:53.744800+0000","last_became_peered":"2024-09-14T06:31:53.744800+0000","last_unstale":"2024-09-14T06:33:34.831894+0000","last_undegraded":"2024-09-14T06:33:34.831894+0000","last_fullsized":"2024-09-14T06:33:34.831894+0000","mapping_epoch":188,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":189,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":561,"log_dups_size":0,"ondisk_log_size":561,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T13:59:24.308993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062201200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":5123,"num_read_kb":67781,"num_write":764,"num_write_kb":43873,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":282,"num_bytes_recovered":41927489,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"262'547","reported_seq":6288,"reported_epoch":289,"state":"active+clean","last_fresh":"2024-09-14T06:33:36.040340+0000","last_change":"2024-09-14T06:33:07.769587+0000","last_active":"2024-09-14T06:33:36.040340+0000","last_peered":"2024-09-14T06:33:36.040340+0000","last_clean":"2024-09-14T06:33:36.040340+0000","last_became_active":"2024-09-14T06:31:53.745159+0000","last_became_peered":"2024-09-14T06:31:53.745159+0000","last_unstale":"2024-09-14T06:33:36.040340+0000","last_undegraded":"2024-09-14T06:33:36.040340+0000","last_fullsized":"2024-09-14T06:33:36.040340+0000","mapping_epoch":188,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":189,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":547,"log_dups_size":0,"ondisk_log_size":547,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T13:08:27.610323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00079631400000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4489,"num_read_kb":78917,"num_write":672,"num_write_kb":51581,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":310,"num_bytes_recovered":56082976,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"261'1037","reported_seq":9728,"reported_epoch":289,"state":"active+clean","last_fresh":"2024-09-14T06:33:34.979868+0000","last_change":"2024-09-14T06:33:07.706863+0000","last_active":"2024-09-14T06:33:34.979868+0000","last_peered":"2024-09-14T06:33:34.979868+0000","last_clean":"2024-09-14T06:33:34.979868+0000","last_became_active":"2024-09-14T06:29:00.579714+0000","last_became_peered":"2024-09-14T06:29:00.579714+0000","last_unstale":"2024-09-14T06:33:34.979868+0000","last_undegraded":"2024-09-14T06:33:34.979868+0000","last_fullsized":"2024-09-14T06:33:34.979868+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":1037,"log_dups_size":0,"ondisk_log_size":1037,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T15:17:51.610057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0017747889999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":10688,"num_read_kb":68964,"num_write":1162,"num_write_kb":48482,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.16","version":"255'1156","reported_seq":4137,"reported_epoch":289,"state":"active+clean","last_fresh":"2024-09-14T06:33:35.112491+0000","last_change":"2024-09-14T06:33:31.824083+0000","last_active":"2024-09-14T06:33:35.112491+0000","last_peered":"2024-09-14T06:33:35.112491+0000","last_clean":"2024-09-14T06:33:35.112491+0000","last_became_active":"2024-09-14T06:33:31.821620+0000","last_became_peered":"2024-09-14T06:33:31.821620+0000","last_unstale":"2024-09-14T06:33:35.112491+0000","last_undegraded":"2024-09-14T06:33:35.112491+0000","last_fullsized":"2024-09-14T06:33:35.112491+0000","mapping_epoch":285,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":286,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":1156,"log_dups_size":0,"ondisk_log_size":1156,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T08:10:02.587758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00073433000000000005,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":5658,"num_read_kb":38562,"num_write":1743,"num_write_kb":37248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"261'650","reported_seq":6751,"reported_epoch":289,"state":"active+clean","last_fresh":"2024-09-14T06:33:34.834516+0000","last_change":"2024-09-14T06:33:07.769581+0000","last_active":"2024-09-14T06:33:34.834516+0000","last_peered":"2024-09-14T06:33:34.834516+0000","last_clean":"2024-09-14T06:33:34.834516+0000","last_became_active":"2024-09-14T06:31:53.746091+0000","last_became_peered":"2024-09-14T06:31:53.746091+0000","last_unstale":"2024-09-14T06:33:34.834516+0000","last_undegraded":"2024-09-14T06:33:34.834516+0000","last_fullsized":"2024-09-14T06:33:34.834516+0000","mapping_epoch":188,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":189,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":650,"log_dups_size":0,"ondisk_log_size":650,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T12:36:31.690559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000442557,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":5101,"num_read_kb":74899,"num_write":696,"num_write_kb":51196,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":271,"num_bytes_recovered":49981473,"num_keys_recovered":33,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.14","version":"261'1137","reported_seq":6341,"reported_epoch":289,"state":"active+clean","last_fresh":"2024-09-14T06:33:36.039887+0000","last_change":"2024-09-14T06:33:07.704760+0000","last_active":"2024-09-14T06:33:36.039887+0000","last_peered":"2024-09-14T06:33:36.039887+0000","last_clean":"2024-09-14T06:33:36.039887+0000","last_became_active":"2024-09-14T06:33:04.628663+0000","last_became_peered":"2024-09-14T06:33:04.628663+0000","last_unstale":"2024-09-14T06:33:36.039887+0000","last_undegraded":"2024-09-14T06:33:36.039887+0000","last_fullsized":"2024-09-14T06:33:36.039887+0000","mapping_epoch":258,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":259,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":1137,"log_dups_size":0,"ondisk_log_size":1137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T11:42:47.162926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00078945399999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":8948,"num_read_kb":46936,"num_write":1573,"num_write_kb":29829,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":296,"num_bytes_recovered":6118769,"num_keys_recovered":11,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":50,"reported_epoch":289,"state":"active+clean","last_fresh":"2024-09-14T06:33:36.039662+0000","last_change":"2024-09-14T06:33:13.859355+0000","last_active":"2024-09-14T06:33:36.039662+0000","last_peered":"2024-09-14T06:33:36.039662+0000","last_clean":"2024-09-14T06:33:36.039662+0000","last_became_active":"2024-09-14T06:33:13.859069+0000","last_became_peered":"2024-09-14T06:33:13.859069+0000","last_unstale":"2024-09-14T06:33:36.039662+0000","last_undegraded":"2024-09-14T06:33:36.039662+0000","last_fullsized":"2024-09-14T06:33:36.039662+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:28:50.205350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:28:50.205350+0000","last_clean_scrub_stamp":"2024-09-14T06:28:50.205350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T17:08:35.416194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"262'893","reported_seq":8726,"reported_epoch":289,"state":"active+clean","last_fresh":"2024-09-14T06:33:34.979753+0000","last_change":"2024-09-14T06:33:07.705814+0000","last_active":"2024-09-14T06:33:34.979753+0000","last_peered":"2024-09-14T06:33:34.979753+0000","last_clean":"2024-09-14T06:33:34.979753+0000","last_became_active":"2024-09-14T06:29:00.578283+0000","last_became_peered":"2024-09-14T06:29:00.578283+0000","last_unstale":"2024-09-14T06:33:34.979753+0000","last_undegraded":"2024-09-14T06:33:34.979753+0000","last_fullsized":"2024-09-14T06:33:34.979753+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":893,"log_dups_size":0,"ondisk_log_size":893,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T15:09:37.855181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001076554,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":8608,"num_read_kb":78770,"num_write":923,"num_write_kb":48303,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":575,"reported_epoch":289,"state":"active+clean","last_fresh":"2024-09-14T06:33:43.467676+0000","last_change":"2024-09-14T06:33:43.467676+0000","last_active":"2024-09-14T06:33:43.467676+0000","last_peered":"2024-09-14T06:33:43.467676+0000","last_clean":"2024-09-14T06:33:43.467676+0000","last_became_active":"2024-09-14T06:33:13.734442+0000","last_became_peered":"2024-09-14T06:33:13.734442+0000","last_unstale":"2024-09-14T06:33:43.467676+0000","last_undegraded":"2024-09-14T06:33:43.467676+0000","last_fullsized":"2024-09-14T06:33:43.467676+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:33:43.467614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:33:43.467614+0000","last_clean_scrub_stamp":"2024-09-14T06:33:43.467614+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T14:06:35.554181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065018200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"261'840","reported_seq":4652,"reported_epoch":289,"state":"active+clean","last_fresh":"2024-09-14T06:33:36.040918+0000","last_change":"2024-09-14T06:33:07.769882+0000","last_active":"2024-09-14T06:33:36.040918+0000","last_peered":"2024-09-14T06:33:36.040918+0000","last_clean":"2024-09-14T06:33:36.040918+0000","last_became_active":"2024-09-14T06:31:54.747923+0000","last_became_peered":"2024-09-14T06:31:54.747923+0000","last_unstale":"2024-09-14T06:33:36.040918+0000","last_undegraded":"2024-09-14T06:33:36.040918+0000","last_fullsized":"2024-09-14T06:33:36.040918+0000","mapping_epoch":189,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":190,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":840,"log_dups_size":0,"ondisk_log_size":840,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T09:01:49.581187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000927559,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4591,"num_read_kb":41129,"num_write":1178,"num_write_kb":36122,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":183,"num_bytes_recovered":13912513,"num_keys_recovered":22,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":554,"reported_epoch":289,"state":"active+clean","last_fresh":"2024-09-14T06:33:34.834701+0000","last_change":"2024-09-14T06:28:53.640035+0000","last_active":"2024-09-14T06:33:34.834701+0000","last_peered":"2024-09-14T06:33:34.834701+0000","last_clean":"2024-09-14T06:33:34.834701+0000","last_became_active":"2024-09-14T06:28:51.308058+0000","last_became_peered":"2024-09-14T06:28:51.308058+0000","last_unstale":"2024-09-14T06:33:34.834701+0000","last_undegraded":"2024-09-14T06:33:34.834701+0000","last_fullsized":"2024-09-14T06:33:34.834701+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:28:50.205350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:28:50.205350+0000","last_clean_scrub_stamp":"2024-09-14T06:28:50.205350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T13:35:48.544057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00037824000000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"262'700","reported_seq":4876,"reported_epoch":289,"state":"active+clean","last_fresh":"2024-09-14T06:33:34.832050+0000","last_change":"2024-09-14T06:33:22.769657+0000","last_active":"2024-09-14T06:33:34.832050+0000","last_peered":"2024-09-14T06:33:34.832050+0000","last_clean":"2024-09-14T06:33:34.832050+0000","last_became_active":"2024-09-14T06:33:22.767544+0000","last_became_peered":"2024-09-14T06:33:22.767544+0000","last_unstale":"2024-09-14T06:33:34.832050+0000","last_undegraded":"2024-09-14T06:33:34.832050+0000","last_fullsized":"2024-09-14T06:33:34.832050+0000","mapping_epoch":276,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":277,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":700,"log_dups_size":0,"ondisk_log_size":700,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T13:09:29.858808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00074004799999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":2654,"num_read_kb":37435,"num_write":897,"num_write_kb":40860,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":687,"num_bytes_recovered":31411166,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":555,"reported_epoch":289,"state":"active+clean","last_fresh":"2024-09-14T06:33:34.832080+0000","last_change":"2024-09-14T06:28:53.226072+0000","last_active":"2024-09-14T06:33:34.832080+0000","last_peered":"2024-09-14T06:33:34.832080+0000","last_clean":"2024-09-14T06:33:34.832080+0000","last_became_active":"2024-09-14T06:28:51.308072+0000","last_became_peered":"2024-09-14T06:28:51.308072+0000","last_unstale":"2024-09-14T06:33:34.832080+0000","last_undegraded":"2024-09-14T06:33:34.832080+0000","last_fullsized":"2024-09-14T06:33:34.832080+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:28:50.205350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:28:50.205350+0000","last_clean_scrub_stamp":"2024-09-14T06:28:50.205350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T13:15:09.052328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066197399999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"261'896","reported_seq":5625,"reported_epoch":289,"state":"active+clean","last_fresh":"2024-09-14T06:33:35.112603+0000","last_change":"2024-09-14T06:33:31.823904+0000","last_active":"2024-09-14T06:33:35.112603+0000","last_peered":"2024-09-14T06:33:35.112603+0000","last_clean":"2024-09-14T06:33:35.112603+0000","last_became_active":"2024-09-14T06:33:31.821533+0000","last_became_peered":"2024-09-14T06:33:31.821533+0000","last_unstale":"2024-09-14T06:33:35.112603+0000","last_undegraded":"2024-09-14T06:33:35.112603+0000","last_fullsized":"2024-09-14T06:33:35.112603+0000","mapping_epoch":285,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":286,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":896,"log_dups_size":0,"ondisk_log_size":896,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T15:48:46.733188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015088129999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4516,"num_read_kb":42632,"num_write":1255,"num_write_kb":49494,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":330,"num_bytes_recovered":28798893,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"262'711","reported_seq":10924,"reported_epoch":289,"state":"active+clean","last_fresh":"2024-09-14T06:33:36.040034+0000","last_change":"2024-09-14T06:33:07.704808+0000","last_active":"2024-09-14T06:33:36.040034+0000","last_peered":"2024-09-14T06:33:36.040034+0000","last_clean":"2024-09-14T06:33:36.040034+0000","last_became_active":"2024-09-14T06:29:00.584678+0000","last_became_peered":"2024-09-14T06:29:00.584678+0000","last_unstale":"2024-09-14T06:33:36.040034+0000","last_undegraded":"2024-09-14T06:33:36.040034+0000","last_fullsized":"2024-09-14T06:33:36.040034+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":711,"log_dups_size":0,"ondisk_log_size":711,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T07:39:08.849874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00074987800000000005,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":9560,"num_read_kb":89311,"num_write":777,"num_write_kb":51691,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"22'2","reported_seq":428,"reported_epoch":289,"state":"active+clean","last_fresh":"2024-09-14T06:33:35.112361+0000","last_change":"2024-09-14T06:33:31.823823+0000","last_active":"2024-09-14T06:33:35.112361+0000","last_peered":"2024-09-14T06:33:35.112361+0000","last_clean":"2024-09-14T06:33:35.112361+0000","last_became_active":"2024-09-14T06:33:31.823429+0000","last_became_peered":"2024-09-14T06:33:31.823429+0000","last_unstale":"2024-09-14T06:33:35.112361+0000","last_undegraded":"2024-09-14T06:33:35.112361+0000","last_fullsized":"2024-09-14T06:33:35.112361+0000","mapping_epoch":285,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":286,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:28:50.205350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:28:50.205350+0000","last_clean_scrub_stamp":"2024-09-14T06:28:50.205350+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T18:01:36.855427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010072970000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.0","version":"260'1116","reported_seq":8244,"reported_epoch":289,"state":"active+clean","last_fresh":"2024-09-14T06:33:34.832141+0000","last_change":"2024-09-14T06:33:22.769183+0000","last_active":"2024-09-14T06:33:34.832141+0000","last_peered":"2024-09-14T06:33:34.832141+0000","last_clean":"2024-09-14T06:33:34.832141+0000","last_became_active":"2024-09-14T06:33:22.767520+0000","last_became_peered":"2024-09-14T06:33:22.767520+0000","last_unstale":"2024-09-14T06:33:34.832141+0000","last_undegraded":"2024-09-14T06:33:34.832141+0000","last_fullsized":"2024-09-14T06:33:34.832141+0000","mapping_epoch":276,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":277,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":1116,"log_dups_size":0,"ondisk_log_size":1116,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T17:26:46.362756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064233900000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":8259,"num_read_kb":85480,"num_write":1652,"num_write_kb":58700,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":537,"num_bytes_recovered":2852,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":40,"reported_epoch":289,"state":"active+clean","last_fresh":"2024-09-14T06:33:34.831819+0000","last_change":"2024-09-14T06:33:19.801852+0000","last_active":"2024-09-14T06:33:34.831819+0000","last_peered":"2024-09-14T06:33:34.831819+0000","last_clean":"2024-09-14T06:33:34.831819+0000","last_became_active":"2024-09-14T06:33:19.801557+0000","last_became_peered":"2024-09-14T06:33:19.801557+0000","last_unstale":"2024-09-14T06:33:34.831819+0000","last_undegraded":"2024-09-14T06:33:34.831819+0000","last_fullsized":"2024-09-14T06:33:34.831819+0000","mapping_epoch":273,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":274,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:28:50.205350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:28:50.205350+0000","last_clean_scrub_stamp":"2024-09-14T06:28:50.205350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T08:41:03.294249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053403100000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1","version":"261'822","reported_seq":11886,"reported_epoch":289,"state":"active+clean","last_fresh":"2024-09-14T06:33:34.979564+0000","last_change":"2024-09-14T06:33:07.705685+0000","last_active":"2024-09-14T06:33:34.979564+0000","last_peered":"2024-09-14T06:33:34.979564+0000","last_clean":"2024-09-14T06:33:34.979564+0000","last_became_active":"2024-09-14T06:30:57.768238+0000","last_became_peered":"2024-09-14T06:30:57.768238+0000","last_unstale":"2024-09-14T06:33:34.979564+0000","last_undegraded":"2024-09-14T06:33:34.979564+0000","last_fullsized":"2024-09-14T06:33:34.979564+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":822,"log_dups_size":0,"ondisk_log_size":822,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T15:07:18.804416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011933110000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":11284,"num_read_kb":112430,"num_write":991,"num_write_kb":47408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":183,"num_bytes_recovered":30431994,"num_keys_recovered":22,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":554,"reported_epoch":289,"state":"active+clean","last_fresh":"2024-09-14T06:33:34.834417+0000","last_change":"2024-09-14T06:28:53.640096+0000","last_active":"2024-09-14T06:33:34.834417+0000","last_peered":"2024-09-14T06:33:34.834417+0000","last_clean":"2024-09-14T06:33:34.834417+0000","last_became_active":"2024-09-14T06:28:51.223069+0000","last_became_peered":"2024-09-14T06:28:51.223069+0000","last_unstale":"2024-09-14T06:33:34.834417+0000","last_undegraded":"2024-09-14T06:33:34.834417+0000","last_fullsized":"2024-09-14T06:33:34.834417+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:28:50.205350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:28:50.205350+0000","last_clean_scrub_stamp":"2024-09-14T06:28:50.205350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T15:13:00.471509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00043714600000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"262'1663","reported_seq":7853,"reported_epoch":289,"state":"active+clean","last_fresh":"2024-09-14T06:33:36.041388+0000","last_change":"2024-09-14T06:33:07.770369+0000","last_active":"2024-09-14T06:33:36.041388+0000","last_peered":"2024-09-14T06:33:36.041388+0000","last_clean":"2024-09-14T06:33:36.041388+0000","last_became_active":"2024-09-14T06:31:54.744160+0000","last_became_peered":"2024-09-14T06:31:54.744160+0000","last_unstale":"2024-09-14T06:33:36.041388+0000","last_undegraded":"2024-09-14T06:33:36.041388+0000","last_fullsized":"2024-09-14T06:33:36.041388+0000","mapping_epoch":189,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":190,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":1663,"log_dups_size":0,"ondisk_log_size":1663,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T16:44:35.569843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001382608,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":10723,"num_read_kb":70589,"num_write":2596,"num_write_kb":53564,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":327,"num_bytes_recovered":50047063,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"20'1","reported_seq":427,"reported_epoch":289,"state":"active+clean","last_fresh":"2024-09-14T06:33:35.112503+0000","last_change":"2024-09-14T06:33:31.823426+0000","last_active":"2024-09-14T06:33:35.112503+0000","last_peered":"2024-09-14T06:33:35.112503+0000","last_clean":"2024-09-14T06:33:35.112503+0000","last_became_active":"2024-09-14T06:33:31.823032+0000","last_became_peered":"2024-09-14T06:33:31.823032+0000","last_unstale":"2024-09-14T06:33:35.112503+0000","last_undegraded":"2024-09-14T06:33:35.112503+0000","last_fullsized":"2024-09-14T06:33:35.112503+0000","mapping_epoch":285,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":286,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:28:50.205350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:28:50.205350+0000","last_clean_scrub_stamp":"2024-09-14T06:28:50.205350+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T08:31:58.917801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050808899999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'72","reported_seq":645,"reported_epoch":289,"state":"active+clean","last_fresh":"2024-09-14T06:33:34.834414+0000","last_change":"2024-09-14T06:28:44.839885+0000","last_active":"2024-09-14T06:33:34.834414+0000","last_peered":"2024-09-14T06:33:34.834414+0000","last_clean":"2024-09-14T06:33:34.834414+0000","last_became_active":"2024-09-14T06:28:44.839573+0000","last_became_peered":"2024-09-14T06:28:44.839573+0000","last_unstale":"2024-09-14T06:33:34.834414+0000","last_undegraded":"2024-09-14T06:33:34.834414+0000","last_fullsized":"2024-09-14T06:33:34.834414+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:28:43.828494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:28:43.828494+0000","last_clean_scrub_stamp":"2024-09-14T06:28:43.828494+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T14:41:25.787293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.8","version":"261'684","reported_seq":6660,"reported_epoch":289,"state":"active+clean","last_fresh":"2024-09-14T06:33:36.040474+0000","last_change":"2024-09-14T06:33:07.769379+0000","last_active":"2024-09-14T06:33:36.040474+0000","last_peered":"2024-09-14T06:33:36.040474+0000","last_clean":"2024-09-14T06:33:36.040474+0000","last_became_active":"2024-09-14T06:31:55.754694+0000","last_became_peered":"2024-09-14T06:31:55.754694+0000","last_unstale":"2024-09-14T06:33:36.040474+0000","last_undegraded":"2024-09-14T06:33:36.040474+0000","last_fullsized":"2024-09-14T06:33:36.040474+0000","mapping_epoch":190,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":191,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":684,"log_dups_size":0,"ondisk_log_size":684,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T15:48:47.633183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067851599999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":5793,"num_read_kb":90852,"num_write":921,"num_write_kb":57691,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":319,"num_bytes_recovered":60352269,"num_keys_recovered":22,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.9","version":"261'597","reported_seq":5173,"reported_epoch":289,"state":"active+clean","last_fresh":"2024-09-14T06:33:36.040020+0000","last_change":"2024-09-14T06:33:22.772981+0000","last_active":"2024-09-14T06:33:36.040020+0000","last_peered":"2024-09-14T06:33:36.040020+0000","last_clean":"2024-09-14T06:33:36.040020+0000","last_became_active":"2024-09-14T06:33:22.770956+0000","last_became_peered":"2024-09-14T06:33:22.770956+0000","last_unstale":"2024-09-14T06:33:36.040020+0000","last_undegraded":"2024-09-14T06:33:36.040020+0000","last_fullsized":"2024-09-14T06:33:36.040020+0000","mapping_epoch":276,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":277,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":597,"log_dups_size":0,"ondisk_log_size":597,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T17:14:06.174583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057139500000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3678,"num_read_kb":64291,"num_write":778,"num_write_kb":44928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":316,"num_bytes_recovered":49117,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"261'874","reported_seq":5176,"reported_epoch":289,"state":"active+clean","last_fresh":"2024-09-14T06:33:36.039739+0000","last_change":"2024-09-14T06:33:17.629900+0000","last_active":"2024-09-14T06:33:36.039739+0000","last_peered":"2024-09-14T06:33:36.039739+0000","last_clean":"2024-09-14T06:33:36.039739+0000","last_became_active":"2024-09-14T06:33:17.627910+0000","last_became_peered":"2024-09-14T06:33:17.627910+0000","last_unstale":"2024-09-14T06:33:36.039739+0000","last_undegraded":"2024-09-14T06:33:36.039739+0000","last_fullsized":"2024-09-14T06:33:36.039739+0000","mapping_epoch":271,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":272,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":874,"log_dups_size":0,"ondisk_log_size":874,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T14:22:28.835018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00074887399999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":5331,"num_read_kb":58329,"num_write":1221,"num_write_kb":47535,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":656,"num_bytes_recovered":26968802,"num_keys_recovered":45,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.b","version":"260'1637","reported_seq":8988,"reported_epoch":289,"state":"active+clean","last_fresh":"2024-09-14T06:33:36.040615+0000","last_change":"2024-09-14T06:33:07.769381+0000","last_active":"2024-09-14T06:33:36.040615+0000","last_peered":"2024-09-14T06:33:36.040615+0000","last_clean":"2024-09-14T06:33:36.040615+0000","last_became_active":"2024-09-14T06:31:55.756219+0000","last_became_peered":"2024-09-14T06:31:55.756219+0000","last_unstale":"2024-09-14T06:33:36.040615+0000","last_undegraded":"2024-09-14T06:33:36.040615+0000","last_fullsized":"2024-09-14T06:33:36.040615+0000","mapping_epoch":190,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":191,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":1637,"log_dups_size":0,"ondisk_log_size":1637,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T15:53:45.757678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065814200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":13043,"num_read_kb":64129,"num_write":2496,"num_write_kb":43679,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":291,"num_bytes_recovered":37370632,"num_keys_recovered":66,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.c","version":"255'622","reported_seq":6270,"reported_epoch":289,"state":"active+clean","last_fresh":"2024-09-14T06:33:35.112517+0000","last_change":"2024-09-14T06:33:31.824073+0000","last_active":"2024-09-14T06:33:35.112517+0000","last_peered":"2024-09-14T06:33:35.112517+0000","last_clean":"2024-09-14T06:33:35.112517+0000","last_became_active":"2024-09-14T06:33:31.821638+0000","last_became_peered":"2024-09-14T06:33:31.821638+0000","last_unstale":"2024-09-14T06:33:35.112517+0000","last_undegraded":"2024-09-14T06:33:35.112517+0000","last_fullsized":"2024-09-14T06:33:35.112517+0000","mapping_epoch":285,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":286,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":622,"log_dups_size":0,"ondisk_log_size":622,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T09:06:48.668360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013231740000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4842,"num_read_kb":50425,"num_write":795,"num_write_kb":40827,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":293,"num_bytes_recovered":36712421,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.d","version":"261'577","reported_seq":9328,"reported_epoch":289,"state":"active+clean","last_fresh":"2024-09-14T06:33:34.834667+0000","last_change":"2024-09-14T06:33:07.770219+0000","last_active":"2024-09-14T06:33:34.834667+0000","last_peered":"2024-09-14T06:33:34.834667+0000","last_clean":"2024-09-14T06:33:34.834667+0000","last_became_active":"2024-09-14T06:29:00.580362+0000","last_became_peered":"2024-09-14T06:29:00.580362+0000","last_unstale":"2024-09-14T06:33:34.834667+0000","last_undegraded":"2024-09-14T06:33:34.834667+0000","last_fullsized":"2024-09-14T06:33:34.834667+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":577,"log_dups_size":0,"ondisk_log_size":577,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T16:36:38.133044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010003270000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":8199,"num_read_kb":93694,"num_write":732,"num_write_kb":41032,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.e","version":"262'586","reported_seq":5526,"reported_epoch":289,"state":"active+clean","last_fresh":"2024-09-14T06:33:34.834589+0000","last_change":"2024-09-14T06:33:07.769825+0000","last_active":"2024-09-14T06:33:34.834589+0000","last_peered":"2024-09-14T06:33:34.834589+0000","last_clean":"2024-09-14T06:33:34.834589+0000","last_became_active":"2024-09-14T06:32:39.777524+0000","last_became_peered":"2024-09-14T06:32:39.777524+0000","last_unstale":"2024-09-14T06:33:34.834589+0000","last_undegraded":"2024-09-14T06:33:34.834589+0000","last_fullsized":"2024-09-14T06:33:34.834589+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":586,"log_dups_size":0,"ondisk_log_size":586,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T07:31:19.417437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065560200000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3986,"num_read_kb":61138,"num_write":753,"num_write_kb":43787,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":171,"num_bytes_recovered":10913227,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.f","version":"262'712","reported_seq":5948,"reported_epoch":289,"state":"active+clean","last_fresh":"2024-09-14T06:33:34.834579+0000","last_change":"2024-09-14T06:33:07.770138+0000","last_active":"2024-09-14T06:33:34.834579+0000","last_peered":"2024-09-14T06:33:34.834579+0000","last_clean":"2024-09-14T06:33:34.834579+0000","last_became_active":"2024-09-14T06:29:00.586491+0000","last_became_peered":"2024-09-14T06:29:00.586491+0000","last_unstale":"2024-09-14T06:33:34.834579+0000","last_undegraded":"2024-09-14T06:33:34.834579+0000","last_fullsized":"2024-09-14T06:33:34.834579+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":712,"log_dups_size":0,"ondisk_log_size":712,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T14:20:12.511497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000891002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":6081,"num_read_kb":45578,"num_write":836,"num_write_kb":36134,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.10","version":"261'952","reported_seq":6889,"reported_epoch":289,"state":"active+clean","last_fresh":"2024-09-14T06:33:36.039625+0000","last_change":"2024-09-14T06:33:16.630633+0000","last_active":"2024-09-14T06:33:36.039625+0000","last_peered":"2024-09-14T06:33:36.039625+0000","last_clean":"2024-09-14T06:33:36.039625+0000","last_became_active":"2024-09-14T06:33:16.628480+0000","last_became_peered":"2024-09-14T06:33:16.628480+0000","last_unstale":"2024-09-14T06:33:36.039625+0000","last_undegraded":"2024-09-14T06:33:36.039625+0000","last_fullsized":"2024-09-14T06:33:36.039625+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":952,"log_dups_size":0,"ondisk_log_size":952,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T17:09:51.353193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012025289999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":7147,"num_read_kb":34474,"num_write":1082,"num_write_kb":36655,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":318,"num_bytes_recovered":9401812,"num_keys_recovered":45,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.11","version":"261'730","reported_seq":11747,"reported_epoch":289,"state":"active+clean","last_fresh":"2024-09-14T06:33:34.834687+0000","last_change":"2024-09-14T06:33:07.769975+0000","last_active":"2024-09-14T06:33:34.834687+0000","last_peered":"2024-09-14T06:33:34.834687+0000","last_clean":"2024-09-14T06:33:34.834687+0000","last_became_active":"2024-09-14T06:32:39.778010+0000","last_became_peered":"2024-09-14T06:32:39.778010+0000","last_unstale":"2024-09-14T06:33:34.834687+0000","last_undegraded":"2024-09-14T06:33:34.834687+0000","last_fullsized":"2024-09-14T06:33:34.834687+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":730,"log_dups_size":0,"ondisk_log_size":730,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T10:48:00.264177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00072597299999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":10271,"num_read_kb":109243,"num_write":859,"num_write_kb":48307,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":177,"num_bytes_recovered":16647747,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.12","version":"261'782","reported_seq":7506,"reported_epoch":289,"state":"active+clean","last_fresh":"2024-09-14T06:33:34.979641+0000","last_change":"2024-09-14T06:33:07.706504+0000","last_active":"2024-09-14T06:33:34.979641+0000","last_peered":"2024-09-14T06:33:34.979641+0000","last_clean":"2024-09-14T06:33:34.979641+0000","last_became_active":"2024-09-14T06:29:00.579826+0000","last_became_peered":"2024-09-14T06:29:00.579826+0000","last_unstale":"2024-09-14T06:33:34.979641+0000","last_undegraded":"2024-09-14T06:33:34.979641+0000","last_fullsized":"2024-09-14T06:33:34.979641+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":782,"log_dups_size":0,"ondisk_log_size":782,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001680792,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":7110,"num_read_kb":59089,"num_write":947,"num_write_kb":42441,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.13","version":"261'1008","reported_seq":8960,"reported_epoch":289,"state":"active+clean","last_fresh":"2024-09-14T06:33:34.834499+0000","last_change":"2024-09-14T06:33:07.769558+0000","last_active":"2024-09-14T06:33:34.834499+0000","last_peered":"2024-09-14T06:33:34.834499+0000","last_clean":"2024-09-14T06:33:34.834499+0000","last_became_active":"2024-09-14T06:29:00.586520+0000","last_became_peered":"2024-09-14T06:29:00.586520+0000","last_unstale":"2024-09-14T06:33:34.834499+0000","last_undegraded":"2024-09-14T06:33:34.834499+0000","last_fullsized":"2024-09-14T06:33:34.834499+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:29:00.276180+0000","last_clean_scrub_stamp":"2024-09-14T06:29:00.276180+0000","objects_scrubbed":0,"log_size":1008,"log_dups_size":0,"ondisk_log_size":1008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T07:25:04.666000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041539399999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":9551,"num_read_kb":66119,"num_write":1235,"num_write_kb":47437,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":19,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":240092,"num_read_kb":2184933,"num_write":38145,"num_write_kb":1478386,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7477,"num_bytes_recovered":638497624,"num_keys_recovered":546,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":28648,"ondisk_log_size":28648,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":131072,"data_stored":1180736,"data_compressed":65166,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509504,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":70776,"kb_used_data":3484,"kb_used_omap":1,"kb_used_meta":67262,"kb_avail":94301064,"statfs":{"total":96636764160,"available":96564289536,"internally_reserved":0,"allocated":3567616,"data_stored":2317757,"data_compressed":32583,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":1590,"internal_metadata":68876746},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509504,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":56816,"kb_used_data":3420,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94315024,"statfs":{"total":96636764160,"available":96578584576,"internally_reserved":0,"allocated":3502080,"data_stored":1727389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":54655430},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509505,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":74868,"kb_used_data":3424,"kb_used_omap":1,"kb_used_meta":71422,"kb_avail":94296972,"statfs":{"total":96636764160,"available":96560099328,"internally_reserved":0,"allocated":3506176,"data_stored":1727408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":73136581},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509504,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":88756,"kb_used_data":3424,"kb_used_omap":1,"kb_used_meta":85310,"kb_avail":94283084,"statfs":{"total":96636764160,"available":96545878016,"internally_reserved":0,"allocated":3506176,"data_stored":1727408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":87357893},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509505,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":56828,"kb_used_data":3428,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94315012,"statfs":{"total":96636764160,"available":96578572288,"internally_reserved":0,"allocated":3510272,"data_stored":1727427,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":14,"seq":60129542208,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":60976,"kb_used_data":3420,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94310864,"statfs":{"total":96636764160,"available":96574324736,"internally_reserved":0,"allocated":3502080,"data_stored":1727389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":58915270},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542207,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":83252,"kb_used_data":3484,"kb_used_omap":1,"kb_used_meta":79742,"kb_avail":94288588,"statfs":{"total":96636764160,"available":96551514112,"internally_reserved":0,"allocated":3567616,"data_stored":2317757,"data_compressed":32583,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":1595,"internal_metadata":81656261},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574912,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":83184,"kb_used_data":3420,"kb_used_omap":1,"kb_used_meta":79742,"kb_avail":94288656,"statfs":{"total":96636764160,"available":96551583744,"internally_reserved":0,"allocated":3502080,"data_stored":1727389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":81656263},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32583,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32583,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-14T06:33:45.998 INFO:tasks.ceph:pgid 3.1f last_scrub_stamp 2024-09-14T06:29:00.276180+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=14, tm_hour=6, tm_min=29, tm_sec=0, tm_wday=5, tm_yday=258, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=14, tm_hour=6, tm_min=33, tm_sec=41, tm_wday=5, tm_yday=258, tm_isdst=0) 2024-09-14T06:33:45.998 INFO:tasks.ceph:pgid 3.1e last_scrub_stamp 2024-09-14T06:29:00.276180+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=14, tm_hour=6, tm_min=29, tm_sec=0, tm_wday=5, tm_yday=258, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=14, tm_hour=6, tm_min=33, tm_sec=41, tm_wday=5, tm_yday=258, tm_isdst=0) 2024-09-14T06:33:45.998 INFO:tasks.ceph:pgid 3.1d last_scrub_stamp 2024-09-14T06:29:00.276180+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=14, tm_hour=6, tm_min=29, tm_sec=0, tm_wday=5, tm_yday=258, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=14, tm_hour=6, tm_min=33, tm_sec=41, tm_wday=5, tm_yday=258, tm_isdst=0) 2024-09-14T06:33:45.999 INFO:tasks.ceph:pgid 3.1c last_scrub_stamp 2024-09-14T06:29:00.276180+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=14, tm_hour=6, tm_min=29, tm_sec=0, tm_wday=5, tm_yday=258, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=14, tm_hour=6, tm_min=33, tm_sec=41, tm_wday=5, tm_yday=258, tm_isdst=0) 2024-09-14T06:33:45.999 INFO:tasks.ceph:pgid 3.1b last_scrub_stamp 2024-09-14T06:29:00.276180+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=14, tm_hour=6, tm_min=29, tm_sec=0, tm_wday=5, tm_yday=258, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=14, tm_hour=6, tm_min=33, tm_sec=41, tm_wday=5, tm_yday=258, tm_isdst=0) 2024-09-14T06:33:45.999 INFO:tasks.ceph:pgid 3.1a last_scrub_stamp 2024-09-14T06:29:00.276180+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=14, tm_hour=6, tm_min=29, tm_sec=0, tm_wday=5, tm_yday=258, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=14, tm_hour=6, tm_min=33, tm_sec=41, tm_wday=5, tm_yday=258, tm_isdst=0) 2024-09-14T06:33:45.999 INFO:tasks.ceph:pgid 3.19 last_scrub_stamp 2024-09-14T06:29:00.276180+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=14, tm_hour=6, tm_min=29, tm_sec=0, tm_wday=5, tm_yday=258, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=14, tm_hour=6, tm_min=33, tm_sec=41, tm_wday=5, tm_yday=258, tm_isdst=0) 2024-09-14T06:33:45.999 INFO:tasks.ceph:pgid 3.18 last_scrub_stamp 2024-09-14T06:29:00.276180+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=14, tm_hour=6, tm_min=29, tm_sec=0, tm_wday=5, tm_yday=258, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=14, tm_hour=6, tm_min=33, tm_sec=41, tm_wday=5, tm_yday=258, tm_isdst=0) 2024-09-14T06:33:45.999 INFO:tasks.ceph:pgid 3.17 last_scrub_stamp 2024-09-14T06:29:00.276180+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=14, tm_hour=6, tm_min=29, tm_sec=0, tm_wday=5, tm_yday=258, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=14, tm_hour=6, tm_min=33, tm_sec=41, tm_wday=5, tm_yday=258, tm_isdst=0) 2024-09-14T06:33:45.999 INFO:tasks.ceph:pgid 3.16 last_scrub_stamp 2024-09-14T06:29:00.276180+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=14, tm_hour=6, tm_min=29, tm_sec=0, tm_wday=5, tm_yday=258, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=14, tm_hour=6, tm_min=33, tm_sec=41, tm_wday=5, tm_yday=258, tm_isdst=0) 2024-09-14T06:33:45.999 INFO:tasks.ceph:pgid 3.15 last_scrub_stamp 2024-09-14T06:29:00.276180+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=14, tm_hour=6, tm_min=29, tm_sec=0, tm_wday=5, tm_yday=258, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=14, tm_hour=6, tm_min=33, tm_sec=41, tm_wday=5, tm_yday=258, tm_isdst=0) 2024-09-14T06:33:46.000 INFO:tasks.ceph:pgid 3.14 last_scrub_stamp 2024-09-14T06:29:00.276180+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=14, tm_hour=6, tm_min=29, tm_sec=0, tm_wday=5, tm_yday=258, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=14, tm_hour=6, tm_min=33, tm_sec=41, tm_wday=5, tm_yday=258, tm_isdst=0) 2024-09-14T06:33:46.000 INFO:tasks.ceph:pgid 2.7 last_scrub_stamp 2024-09-14T06:28:50.205350+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=14, tm_hour=6, tm_min=28, tm_sec=50, tm_wday=5, tm_yday=258, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=14, tm_hour=6, tm_min=33, tm_sec=41, tm_wday=5, tm_yday=258, tm_isdst=0) 2024-09-14T06:33:46.000 INFO:tasks.ceph:pgid 3.6 last_scrub_stamp 2024-09-14T06:29:00.276180+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=14, tm_hour=6, tm_min=29, tm_sec=0, tm_wday=5, tm_yday=258, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=14, tm_hour=6, tm_min=33, tm_sec=41, tm_wday=5, tm_yday=258, tm_isdst=0) 2024-09-14T06:33:46.000 INFO:tasks.ceph:pgid 3.7 last_scrub_stamp 2024-09-14T06:29:00.276180+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=14, tm_hour=6, tm_min=29, tm_sec=0, tm_wday=5, tm_yday=258, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=14, tm_hour=6, tm_min=33, tm_sec=41, tm_wday=5, tm_yday=258, tm_isdst=0) 2024-09-14T06:33:46.000 INFO:tasks.ceph:pgid 2.5 last_scrub_stamp 2024-09-14T06:28:50.205350+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=14, tm_hour=6, tm_min=28, tm_sec=50, tm_wday=5, tm_yday=258, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=14, tm_hour=6, tm_min=33, tm_sec=41, tm_wday=5, tm_yday=258, tm_isdst=0) 2024-09-14T06:33:46.000 INFO:tasks.ceph:pgid 3.4 last_scrub_stamp 2024-09-14T06:29:00.276180+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=14, tm_hour=6, tm_min=29, tm_sec=0, tm_wday=5, tm_yday=258, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=14, tm_hour=6, tm_min=33, tm_sec=41, tm_wday=5, tm_yday=258, tm_isdst=0) 2024-09-14T06:33:46.000 INFO:tasks.ceph:pgid 2.4 last_scrub_stamp 2024-09-14T06:28:50.205350+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=14, tm_hour=6, tm_min=28, tm_sec=50, tm_wday=5, tm_yday=258, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=14, tm_hour=6, tm_min=33, tm_sec=41, tm_wday=5, tm_yday=258, tm_isdst=0) 2024-09-14T06:33:46.000 INFO:tasks.ceph:pgid 3.5 last_scrub_stamp 2024-09-14T06:29:00.276180+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=14, tm_hour=6, tm_min=29, tm_sec=0, tm_wday=5, tm_yday=258, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=14, tm_hour=6, tm_min=33, tm_sec=41, tm_wday=5, tm_yday=258, tm_isdst=0) 2024-09-14T06:33:46.000 INFO:tasks.ceph:pgid 3.3 last_scrub_stamp 2024-09-14T06:29:00.276180+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=14, tm_hour=6, tm_min=29, tm_sec=0, tm_wday=5, tm_yday=258, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=14, tm_hour=6, tm_min=33, tm_sec=41, tm_wday=5, tm_yday=258, tm_isdst=0) 2024-09-14T06:33:46.001 INFO:tasks.ceph:pgid 2.2 last_scrub_stamp 2024-09-14T06:28:50.205350+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=14, tm_hour=6, tm_min=28, tm_sec=50, tm_wday=5, tm_yday=258, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=14, tm_hour=6, tm_min=33, tm_sec=41, tm_wday=5, tm_yday=258, tm_isdst=0) 2024-09-14T06:33:46.001 INFO:tasks.ceph:pgid 3.0 last_scrub_stamp 2024-09-14T06:29:00.276180+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=14, tm_hour=6, tm_min=29, tm_sec=0, tm_wday=5, tm_yday=258, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=14, tm_hour=6, tm_min=33, tm_sec=41, tm_wday=5, tm_yday=258, tm_isdst=0) 2024-09-14T06:33:46.001 INFO:tasks.ceph:pgid 2.1 last_scrub_stamp 2024-09-14T06:28:50.205350+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=14, tm_hour=6, tm_min=28, tm_sec=50, tm_wday=5, tm_yday=258, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=14, tm_hour=6, tm_min=33, tm_sec=41, tm_wday=5, tm_yday=258, tm_isdst=0) 2024-09-14T06:33:46.001 INFO:tasks.ceph:pgid 3.1 last_scrub_stamp 2024-09-14T06:29:00.276180+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=14, tm_hour=6, tm_min=29, tm_sec=0, tm_wday=5, tm_yday=258, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=14, tm_hour=6, tm_min=33, tm_sec=41, tm_wday=5, tm_yday=258, tm_isdst=0) 2024-09-14T06:33:46.001 INFO:tasks.ceph:pgid 2.0 last_scrub_stamp 2024-09-14T06:28:50.205350+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=14, tm_hour=6, tm_min=28, tm_sec=50, tm_wday=5, tm_yday=258, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=14, tm_hour=6, tm_min=33, tm_sec=41, tm_wday=5, tm_yday=258, tm_isdst=0) 2024-09-14T06:33:46.001 INFO:tasks.ceph:pgid 3.2 last_scrub_stamp 2024-09-14T06:29:00.276180+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=14, tm_hour=6, tm_min=29, tm_sec=0, tm_wday=5, tm_yday=258, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=14, tm_hour=6, tm_min=33, tm_sec=41, tm_wday=5, tm_yday=258, tm_isdst=0) 2024-09-14T06:33:46.002 INFO:tasks.ceph:pgid 2.3 last_scrub_stamp 2024-09-14T06:28:50.205350+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=14, tm_hour=6, tm_min=28, tm_sec=50, tm_wday=5, tm_yday=258, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=14, tm_hour=6, tm_min=33, tm_sec=41, tm_wday=5, tm_yday=258, tm_isdst=0) 2024-09-14T06:33:46.002 INFO:tasks.ceph:pgid 1.0 last_scrub_stamp 2024-09-14T06:28:43.828494+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=14, tm_hour=6, tm_min=28, tm_sec=43, tm_wday=5, tm_yday=258, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=14, tm_hour=6, tm_min=33, tm_sec=41, tm_wday=5, tm_yday=258, tm_isdst=0) 2024-09-14T06:33:46.002 INFO:tasks.ceph:pgid 3.8 last_scrub_stamp 2024-09-14T06:29:00.276180+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=14, tm_hour=6, tm_min=29, tm_sec=0, tm_wday=5, tm_yday=258, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=14, tm_hour=6, tm_min=33, tm_sec=41, tm_wday=5, tm_yday=258, tm_isdst=0) 2024-09-14T06:33:46.002 INFO:tasks.ceph:pgid 3.9 last_scrub_stamp 2024-09-14T06:29:00.276180+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=14, tm_hour=6, tm_min=29, tm_sec=0, tm_wday=5, tm_yday=258, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=14, tm_hour=6, tm_min=33, tm_sec=41, tm_wday=5, tm_yday=258, tm_isdst=0) 2024-09-14T06:33:46.002 INFO:tasks.ceph:pgid 3.a last_scrub_stamp 2024-09-14T06:29:00.276180+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=14, tm_hour=6, tm_min=29, tm_sec=0, tm_wday=5, tm_yday=258, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=14, tm_hour=6, tm_min=33, tm_sec=41, tm_wday=5, tm_yday=258, tm_isdst=0) 2024-09-14T06:33:46.002 INFO:tasks.ceph:pgid 3.b last_scrub_stamp 2024-09-14T06:29:00.276180+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=14, tm_hour=6, tm_min=29, tm_sec=0, tm_wday=5, tm_yday=258, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=14, tm_hour=6, tm_min=33, tm_sec=41, tm_wday=5, tm_yday=258, tm_isdst=0) 2024-09-14T06:33:46.002 INFO:tasks.ceph:pgid 3.c last_scrub_stamp 2024-09-14T06:29:00.276180+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=14, tm_hour=6, tm_min=29, tm_sec=0, tm_wday=5, tm_yday=258, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=14, tm_hour=6, tm_min=33, tm_sec=41, tm_wday=5, tm_yday=258, tm_isdst=0) 2024-09-14T06:33:46.002 INFO:tasks.ceph:pgid 3.d last_scrub_stamp 2024-09-14T06:29:00.276180+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=14, tm_hour=6, tm_min=29, tm_sec=0, tm_wday=5, tm_yday=258, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=14, tm_hour=6, tm_min=33, tm_sec=41, tm_wday=5, tm_yday=258, tm_isdst=0) 2024-09-14T06:33:46.002 INFO:tasks.ceph:pgid 3.e last_scrub_stamp 2024-09-14T06:29:00.276180+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=14, tm_hour=6, tm_min=29, tm_sec=0, tm_wday=5, tm_yday=258, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=14, tm_hour=6, tm_min=33, tm_sec=41, tm_wday=5, tm_yday=258, tm_isdst=0) 2024-09-14T06:33:46.003 INFO:tasks.ceph:pgid 3.f last_scrub_stamp 2024-09-14T06:29:00.276180+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=14, tm_hour=6, tm_min=29, tm_sec=0, tm_wday=5, tm_yday=258, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=14, tm_hour=6, tm_min=33, tm_sec=41, tm_wday=5, tm_yday=258, tm_isdst=0) 2024-09-14T06:33:46.003 INFO:tasks.ceph:pgid 3.10 last_scrub_stamp 2024-09-14T06:29:00.276180+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=14, tm_hour=6, tm_min=29, tm_sec=0, tm_wday=5, tm_yday=258, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=14, tm_hour=6, tm_min=33, tm_sec=41, tm_wday=5, tm_yday=258, tm_isdst=0) 2024-09-14T06:33:46.003 INFO:tasks.ceph:pgid 3.11 last_scrub_stamp 2024-09-14T06:29:00.276180+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=14, tm_hour=6, tm_min=29, tm_sec=0, tm_wday=5, tm_yday=258, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=14, tm_hour=6, tm_min=33, tm_sec=41, tm_wday=5, tm_yday=258, tm_isdst=0) 2024-09-14T06:33:46.003 INFO:tasks.ceph:pgid 3.12 last_scrub_stamp 2024-09-14T06:29:00.276180+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=14, tm_hour=6, tm_min=29, tm_sec=0, tm_wday=5, tm_yday=258, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=14, tm_hour=6, tm_min=33, tm_sec=41, tm_wday=5, tm_yday=258, tm_isdst=0) 2024-09-14T06:33:46.003 INFO:tasks.ceph:pgid 3.13 last_scrub_stamp 2024-09-14T06:29:00.276180+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=14, tm_hour=6, tm_min=29, tm_sec=0, tm_wday=5, tm_yday=258, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=14, tm_hour=6, tm_min=33, tm_sec=41, tm_wday=5, tm_yday=258, tm_isdst=0) 2024-09-14T06:33:46.003 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2024-09-14T06:34:06.004 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-14T06:34:06.301 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-09-14T06:34:06.302 INFO:teuthology.orchestra.run.smithi094.stderr:dumped all 2024-09-14T06:34:06.314 INFO:teuthology.orchestra.run.smithi094.stdout:{"pg_ready":true,"pg_map":{"version":445,"stamp":"2024-09-14T06:34:05.342525+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590406,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":240158,"num_read_kb":2184988,"num_write":38248,"num_write_kb":1480016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7477,"num_bytes_recovered":638497624,"num_keys_recovered":546,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":28730,"ondisk_log_size":28730,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":575408,"kb_used_data":27504,"kb_used_omap":12,"kb_used_meta":547763,"kb_avail":754399312,"statfs":{"total":773094113280,"available":772504895488,"internally_reserved":0,"allocated":28164096,"data_stored":14999924,"data_compressed":65166,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":12751,"internal_metadata":560909873},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002550"},"pg_stats":[{"pgid":"3.1f","version":"262'855","reported_seq":8439,"reported_epoch":289,"state":"active+clean","last_fresh":"2024-09-14T06:33:48.529011+0000","last_change":"2024-09-14T06:33:48.529011+0000","last_active":"2024-09-14T06:33:48.529011+0000","last_peered":"2024-09-14T06:33:48.529011+0000","last_clean":"2024-09-14T06:33:48.529011+0000","last_became_active":"2024-09-14T06:29:00.579883+0000","last_became_peered":"2024-09-14T06:29:00.579883+0000","last_unstale":"2024-09-14T06:33:48.529011+0000","last_undegraded":"2024-09-14T06:33:48.529011+0000","last_fullsized":"2024-09-14T06:33:48.529011+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"262'855","last_scrub_stamp":"2024-09-14T06:33:48.528919+0000","last_deep_scrub":"262'855","last_deep_scrub_stamp":"2024-09-14T06:33:48.528919+0000","last_clean_scrub_stamp":"2024-09-14T06:33:48.528919+0000","objects_scrubbed":0,"log_size":855,"log_dups_size":0,"ondisk_log_size":855,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T11:09:17.444040+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0.001666641,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":8468,"num_read_kb":86898,"num_write":1171,"num_write_kb":49543,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"262'783","reported_seq":10863,"reported_epoch":289,"state":"active+clean","last_fresh":"2024-09-14T06:33:49.198454+0000","last_change":"2024-09-14T06:33:49.198361+0000","last_active":"2024-09-14T06:33:49.198454+0000","last_peered":"2024-09-14T06:33:49.198454+0000","last_clean":"2024-09-14T06:33:49.198454+0000","last_became_active":"2024-09-14T06:33:16.626222+0000","last_became_peered":"2024-09-14T06:33:16.626222+0000","last_unstale":"2024-09-14T06:33:49.198454+0000","last_undegraded":"2024-09-14T06:33:49.198454+0000","last_fullsized":"2024-09-14T06:33:49.198454+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":0,"last_scrub":"262'783","last_scrub_stamp":"2024-09-14T06:33:49.198244+0000","last_deep_scrub":"262'783","last_deep_scrub_stamp":"2024-09-14T06:33:49.198244+0000","last_clean_scrub_stamp":"2024-09-14T06:33:49.198244+0000","objects_scrubbed":0,"log_size":783,"log_dups_size":0,"ondisk_log_size":783,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T13:22:17.877167+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0.0013639839999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":8315,"num_read_kb":67657,"num_write":1025,"num_write_kb":40407,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1025,"num_bytes_recovered":87221212,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1d","version":"260'1077","reported_seq":7494,"reported_epoch":289,"state":"active+clean","last_fresh":"2024-09-14T06:33:51.382859+0000","last_change":"2024-09-14T06:33:51.382770+0000","last_active":"2024-09-14T06:33:51.382859+0000","last_peered":"2024-09-14T06:33:51.382859+0000","last_clean":"2024-09-14T06:33:51.382859+0000","last_became_active":"2024-09-14T06:33:31.824454+0000","last_became_peered":"2024-09-14T06:33:31.824454+0000","last_unstale":"2024-09-14T06:33:51.382859+0000","last_undegraded":"2024-09-14T06:33:51.382859+0000","last_fullsized":"2024-09-14T06:33:51.382859+0000","mapping_epoch":285,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":286,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'1077","last_scrub_stamp":"2024-09-14T06:33:51.382665+0000","last_deep_scrub":"260'1077","last_deep_scrub_stamp":"2024-09-14T06:33:51.382665+0000","last_clean_scrub_stamp":"2024-09-14T06:33:51.382665+0000","objects_scrubbed":0,"log_size":1077,"log_dups_size":0,"ondisk_log_size":1077,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T14:25:42.128234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088638400000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":7873,"num_read_kb":64375,"num_write":1555,"num_write_kb":53292,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1c","version":"289'806","reported_seq":6044,"reported_epoch":289,"state":"active+clean","last_fresh":"2024-09-14T06:33:50.337635+0000","last_change":"2024-09-14T06:33:50.337564+0000","last_active":"2024-09-14T06:33:50.337635+0000","last_peered":"2024-09-14T06:33:50.337635+0000","last_clean":"2024-09-14T06:33:50.337635+0000","last_became_active":"2024-09-14T06:33:31.823955+0000","last_became_peered":"2024-09-14T06:33:31.823955+0000","last_unstale":"2024-09-14T06:33:50.337635+0000","last_undegraded":"2024-09-14T06:33:50.337635+0000","last_fullsized":"2024-09-14T06:33:50.337635+0000","mapping_epoch":285,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":286,"parent":"0.0","parent_split_bits":0,"last_scrub":"289'806","last_scrub_stamp":"2024-09-14T06:33:50.337476+0000","last_deep_scrub":"289'806","last_deep_scrub_stamp":"2024-09-14T06:33:50.337476+0000","last_clean_scrub_stamp":"2024-09-14T06:33:50.337476+0000","objects_scrubbed":1,"log_size":806,"log_dups_size":0,"ondisk_log_size":806,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T07:34:20.880630+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0.00068758600000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":4953,"num_read_kb":43809,"num_write":946,"num_write_kb":45551,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":505,"num_bytes_recovered":44145197,"num_keys_recovered":280,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"259'960","reported_seq":11530,"reported_epoch":289,"state":"active+clean","last_fresh":"2024-09-14T06:33:47.504836+0000","last_change":"2024-09-14T06:33:47.504741+0000","last_active":"2024-09-14T06:33:47.504836+0000","last_peered":"2024-09-14T06:33:47.504836+0000","last_clean":"2024-09-14T06:33:47.504836+0000","last_became_active":"2024-09-14T06:29:00.586183+0000","last_became_peered":"2024-09-14T06:29:00.586183+0000","last_unstale":"2024-09-14T06:33:47.504836+0000","last_undegraded":"2024-09-14T06:33:47.504836+0000","last_fullsized":"2024-09-14T06:33:47.504836+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"259'960","last_scrub_stamp":"2024-09-14T06:33:47.504658+0000","last_deep_scrub":"259'960","last_deep_scrub_stamp":"2024-09-14T06:33:47.504658+0000","last_clean_scrub_stamp":"2024-09-14T06:33:47.504658+0000","objects_scrubbed":0,"log_size":960,"log_dups_size":0,"ondisk_log_size":960,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T10:52:59.397257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001810606,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":11910,"num_read_kb":108094,"num_write":1463,"num_write_kb":55102,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1a","version":"289'1679","reported_seq":10648,"reported_epoch":289,"state":"active+clean","last_fresh":"2024-09-14T06:33:46.758274+0000","last_change":"2024-09-14T06:33:46.758219+0000","last_active":"2024-09-14T06:33:46.758274+0000","last_peered":"2024-09-14T06:33:46.758274+0000","last_clean":"2024-09-14T06:33:46.758274+0000","last_became_active":"2024-09-14T06:29:00.576206+0000","last_became_peered":"2024-09-14T06:29:00.576206+0000","last_unstale":"2024-09-14T06:33:46.758274+0000","last_undegraded":"2024-09-14T06:33:46.758274+0000","last_fullsized":"2024-09-14T06:33:46.758274+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"289'1679","last_scrub_stamp":"2024-09-14T06:33:46.758138+0000","last_deep_scrub":"289'1679","last_deep_scrub_stamp":"2024-09-14T06:33:46.758138+0000","last_clean_scrub_stamp":"2024-09-14T06:33:46.758138+0000","objects_scrubbed":1,"log_size":1679,"log_dups_size":0,"ondisk_log_size":1679,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T14:17:44.105188+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0.00088286099999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":13339,"num_read_kb":82904,"num_write":2451,"num_write_kb":55687,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.19","version":"261'561","reported_seq":6517,"reported_epoch":289,"state":"active+clean","last_fresh":"2024-09-14T06:33:45.444398+0000","last_change":"2024-09-14T06:33:45.444334+0000","last_active":"2024-09-14T06:33:45.444398+0000","last_peered":"2024-09-14T06:33:45.444398+0000","last_clean":"2024-09-14T06:33:45.444398+0000","last_became_active":"2024-09-14T06:31:53.744800+0000","last_became_peered":"2024-09-14T06:31:53.744800+0000","last_unstale":"2024-09-14T06:33:45.444398+0000","last_undegraded":"2024-09-14T06:33:45.444398+0000","last_fullsized":"2024-09-14T06:33:45.444398+0000","mapping_epoch":188,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":189,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'561","last_scrub_stamp":"2024-09-14T06:33:45.444262+0000","last_deep_scrub":"261'561","last_deep_scrub_stamp":"2024-09-14T06:33:45.444262+0000","last_clean_scrub_stamp":"2024-09-14T06:33:45.444262+0000","objects_scrubbed":0,"log_size":561,"log_dups_size":0,"ondisk_log_size":561,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T07:05:43.750153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062201200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":5123,"num_read_kb":67781,"num_write":764,"num_write_kb":43873,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":282,"num_bytes_recovered":41927489,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"262'547","reported_seq":6296,"reported_epoch":289,"state":"active+clean","last_fresh":"2024-09-14T06:33:48.231016+0000","last_change":"2024-09-14T06:33:48.231016+0000","last_active":"2024-09-14T06:33:48.231016+0000","last_peered":"2024-09-14T06:33:48.231016+0000","last_clean":"2024-09-14T06:33:48.231016+0000","last_became_active":"2024-09-14T06:31:53.745159+0000","last_became_peered":"2024-09-14T06:31:53.745159+0000","last_unstale":"2024-09-14T06:33:48.231016+0000","last_undegraded":"2024-09-14T06:33:48.231016+0000","last_fullsized":"2024-09-14T06:33:48.231016+0000","mapping_epoch":188,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":189,"parent":"0.0","parent_split_bits":0,"last_scrub":"262'547","last_scrub_stamp":"2024-09-14T06:33:48.230917+0000","last_deep_scrub":"262'547","last_deep_scrub_stamp":"2024-09-14T06:33:48.230917+0000","last_clean_scrub_stamp":"2024-09-14T06:33:48.230917+0000","objects_scrubbed":0,"log_size":547,"log_dups_size":0,"ondisk_log_size":547,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T15:57:34.816597+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0.00079631400000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4489,"num_read_kb":78917,"num_write":672,"num_write_kb":51581,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":310,"num_bytes_recovered":56082976,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"261'1037","reported_seq":9737,"reported_epoch":289,"state":"active+clean","last_fresh":"2024-09-14T06:33:46.514152+0000","last_change":"2024-09-14T06:33:46.514099+0000","last_active":"2024-09-14T06:33:46.514152+0000","last_peered":"2024-09-14T06:33:46.514152+0000","last_clean":"2024-09-14T06:33:46.514152+0000","last_became_active":"2024-09-14T06:29:00.579714+0000","last_became_peered":"2024-09-14T06:29:00.579714+0000","last_unstale":"2024-09-14T06:33:46.514152+0000","last_undegraded":"2024-09-14T06:33:46.514152+0000","last_fullsized":"2024-09-14T06:33:46.514152+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'1037","last_scrub_stamp":"2024-09-14T06:33:46.514018+0000","last_deep_scrub":"261'1037","last_deep_scrub_stamp":"2024-09-14T06:33:46.514018+0000","last_clean_scrub_stamp":"2024-09-14T06:33:46.514018+0000","objects_scrubbed":0,"log_size":1037,"log_dups_size":0,"ondisk_log_size":1037,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T17:38:25.628167+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0.0017747889999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":10688,"num_read_kb":68964,"num_write":1162,"num_write_kb":48482,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.16","version":"255'1156","reported_seq":4146,"reported_epoch":289,"state":"active+clean","last_fresh":"2024-09-14T06:33:49.294440+0000","last_change":"2024-09-14T06:33:49.294379+0000","last_active":"2024-09-14T06:33:49.294440+0000","last_peered":"2024-09-14T06:33:49.294440+0000","last_clean":"2024-09-14T06:33:49.294440+0000","last_became_active":"2024-09-14T06:33:31.821620+0000","last_became_peered":"2024-09-14T06:33:31.821620+0000","last_unstale":"2024-09-14T06:33:49.294440+0000","last_undegraded":"2024-09-14T06:33:49.294440+0000","last_fullsized":"2024-09-14T06:33:49.294440+0000","mapping_epoch":285,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":286,"parent":"0.0","parent_split_bits":0,"last_scrub":"255'1156","last_scrub_stamp":"2024-09-14T06:33:49.294290+0000","last_deep_scrub":"255'1156","last_deep_scrub_stamp":"2024-09-14T06:33:49.294290+0000","last_clean_scrub_stamp":"2024-09-14T06:33:49.294290+0000","objects_scrubbed":0,"log_size":1156,"log_dups_size":0,"ondisk_log_size":1156,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T13:12:08.557609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00073433000000000005,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":5658,"num_read_kb":38562,"num_write":1743,"num_write_kb":37248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"261'650","reported_seq":6760,"reported_epoch":289,"state":"active+clean","last_fresh":"2024-09-14T06:33:54.463511+0000","last_change":"2024-09-14T06:33:54.463511+0000","last_active":"2024-09-14T06:33:54.463511+0000","last_peered":"2024-09-14T06:33:54.463511+0000","last_clean":"2024-09-14T06:33:54.463511+0000","last_became_active":"2024-09-14T06:31:53.746091+0000","last_became_peered":"2024-09-14T06:31:53.746091+0000","last_unstale":"2024-09-14T06:33:54.463511+0000","last_undegraded":"2024-09-14T06:33:54.463511+0000","last_fullsized":"2024-09-14T06:33:54.463511+0000","mapping_epoch":188,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":189,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'650","last_scrub_stamp":"2024-09-14T06:33:54.463406+0000","last_deep_scrub":"261'650","last_deep_scrub_stamp":"2024-09-14T06:33:54.463406+0000","last_clean_scrub_stamp":"2024-09-14T06:33:54.463406+0000","objects_scrubbed":0,"log_size":650,"log_dups_size":0,"ondisk_log_size":650,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T09:06:39.743289+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0.000442557,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":5101,"num_read_kb":74899,"num_write":696,"num_write_kb":51196,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":271,"num_bytes_recovered":49981473,"num_keys_recovered":33,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.14","version":"261'1137","reported_seq":6349,"reported_epoch":289,"state":"active+clean","last_fresh":"2024-09-14T06:33:45.716483+0000","last_change":"2024-09-14T06:33:45.716483+0000","last_active":"2024-09-14T06:33:45.716483+0000","last_peered":"2024-09-14T06:33:45.716483+0000","last_clean":"2024-09-14T06:33:45.716483+0000","last_became_active":"2024-09-14T06:33:04.628663+0000","last_became_peered":"2024-09-14T06:33:04.628663+0000","last_unstale":"2024-09-14T06:33:45.716483+0000","last_undegraded":"2024-09-14T06:33:45.716483+0000","last_fullsized":"2024-09-14T06:33:45.716483+0000","mapping_epoch":258,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":259,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'1137","last_scrub_stamp":"2024-09-14T06:33:45.716366+0000","last_deep_scrub":"261'1137","last_deep_scrub_stamp":"2024-09-14T06:33:45.716366+0000","last_clean_scrub_stamp":"2024-09-14T06:33:45.716366+0000","objects_scrubbed":0,"log_size":1137,"log_dups_size":0,"ondisk_log_size":1137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T17:45:40.235617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00078945399999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":8948,"num_read_kb":46936,"num_write":1573,"num_write_kb":29829,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":296,"num_bytes_recovered":6118769,"num_keys_recovered":11,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":58,"reported_epoch":289,"state":"active+clean","last_fresh":"2024-09-14T06:33:45.827395+0000","last_change":"2024-09-14T06:33:45.827395+0000","last_active":"2024-09-14T06:33:45.827395+0000","last_peered":"2024-09-14T06:33:45.827395+0000","last_clean":"2024-09-14T06:33:45.827395+0000","last_became_active":"2024-09-14T06:33:13.859069+0000","last_became_peered":"2024-09-14T06:33:13.859069+0000","last_unstale":"2024-09-14T06:33:45.827395+0000","last_undegraded":"2024-09-14T06:33:45.827395+0000","last_fullsized":"2024-09-14T06:33:45.827395+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:33:45.827309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:33:45.827309+0000","last_clean_scrub_stamp":"2024-09-14T06:33:45.827309+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T17:00:57.573743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"262'893","reported_seq":8734,"reported_epoch":289,"state":"active+clean","last_fresh":"2024-09-14T06:33:45.465234+0000","last_change":"2024-09-14T06:33:45.465234+0000","last_active":"2024-09-14T06:33:45.465234+0000","last_peered":"2024-09-14T06:33:45.465234+0000","last_clean":"2024-09-14T06:33:45.465234+0000","last_became_active":"2024-09-14T06:29:00.578283+0000","last_became_peered":"2024-09-14T06:29:00.578283+0000","last_unstale":"2024-09-14T06:33:45.465234+0000","last_undegraded":"2024-09-14T06:33:45.465234+0000","last_fullsized":"2024-09-14T06:33:45.465234+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"262'893","last_scrub_stamp":"2024-09-14T06:33:45.465166+0000","last_deep_scrub":"262'893","last_deep_scrub_stamp":"2024-09-14T06:33:45.465166+0000","last_clean_scrub_stamp":"2024-09-14T06:33:45.465166+0000","objects_scrubbed":0,"log_size":893,"log_dups_size":0,"ondisk_log_size":893,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T16:03:43.522136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001076554,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":8608,"num_read_kb":78770,"num_write":923,"num_write_kb":48303,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":575,"reported_epoch":289,"state":"active+clean","last_fresh":"2024-09-14T06:33:43.467676+0000","last_change":"2024-09-14T06:33:43.467676+0000","last_active":"2024-09-14T06:33:43.467676+0000","last_peered":"2024-09-14T06:33:43.467676+0000","last_clean":"2024-09-14T06:33:43.467676+0000","last_became_active":"2024-09-14T06:33:13.734442+0000","last_became_peered":"2024-09-14T06:33:13.734442+0000","last_unstale":"2024-09-14T06:33:43.467676+0000","last_undegraded":"2024-09-14T06:33:43.467676+0000","last_fullsized":"2024-09-14T06:33:43.467676+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:33:43.467614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:33:43.467614+0000","last_clean_scrub_stamp":"2024-09-14T06:33:43.467614+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T14:06:35.554181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065018200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"261'840","reported_seq":4660,"reported_epoch":289,"state":"active+clean","last_fresh":"2024-09-14T06:33:45.321733+0000","last_change":"2024-09-14T06:33:45.321733+0000","last_active":"2024-09-14T06:33:45.321733+0000","last_peered":"2024-09-14T06:33:45.321733+0000","last_clean":"2024-09-14T06:33:45.321733+0000","last_became_active":"2024-09-14T06:31:54.747923+0000","last_became_peered":"2024-09-14T06:31:54.747923+0000","last_unstale":"2024-09-14T06:33:45.321733+0000","last_undegraded":"2024-09-14T06:33:45.321733+0000","last_fullsized":"2024-09-14T06:33:45.321733+0000","mapping_epoch":189,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":190,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'840","last_scrub_stamp":"2024-09-14T06:33:45.321655+0000","last_deep_scrub":"261'840","last_deep_scrub_stamp":"2024-09-14T06:33:45.321655+0000","last_clean_scrub_stamp":"2024-09-14T06:33:45.321655+0000","objects_scrubbed":0,"log_size":840,"log_dups_size":0,"ondisk_log_size":840,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T13:02:03.917140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000927559,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4591,"num_read_kb":41129,"num_write":1178,"num_write_kb":36122,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":183,"num_bytes_recovered":13912513,"num_keys_recovered":22,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":562,"reported_epoch":289,"state":"active+clean","last_fresh":"2024-09-14T06:33:53.437102+0000","last_change":"2024-09-14T06:33:53.437102+0000","last_active":"2024-09-14T06:33:53.437102+0000","last_peered":"2024-09-14T06:33:53.437102+0000","last_clean":"2024-09-14T06:33:53.437102+0000","last_became_active":"2024-09-14T06:28:51.308058+0000","last_became_peered":"2024-09-14T06:28:51.308058+0000","last_unstale":"2024-09-14T06:33:53.437102+0000","last_undegraded":"2024-09-14T06:33:53.437102+0000","last_fullsized":"2024-09-14T06:33:53.437102+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:33:53.437005+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:33:53.437005+0000","last_clean_scrub_stamp":"2024-09-14T06:33:53.437005+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T06:51:05.549774+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0.00037824000000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"262'700","reported_seq":4884,"reported_epoch":289,"state":"active+clean","last_fresh":"2024-09-14T06:33:44.425214+0000","last_change":"2024-09-14T06:33:44.425214+0000","last_active":"2024-09-14T06:33:44.425214+0000","last_peered":"2024-09-14T06:33:44.425214+0000","last_clean":"2024-09-14T06:33:44.425214+0000","last_became_active":"2024-09-14T06:33:22.767544+0000","last_became_peered":"2024-09-14T06:33:22.767544+0000","last_unstale":"2024-09-14T06:33:44.425214+0000","last_undegraded":"2024-09-14T06:33:44.425214+0000","last_fullsized":"2024-09-14T06:33:44.425214+0000","mapping_epoch":276,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":277,"parent":"0.0","parent_split_bits":0,"last_scrub":"262'700","last_scrub_stamp":"2024-09-14T06:33:44.425109+0000","last_deep_scrub":"262'700","last_deep_scrub_stamp":"2024-09-14T06:33:44.425109+0000","last_clean_scrub_stamp":"2024-09-14T06:33:44.425109+0000","objects_scrubbed":0,"log_size":700,"log_dups_size":0,"ondisk_log_size":700,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T16:54:57.512837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00074004799999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":2654,"num_read_kb":37435,"num_write":897,"num_write_kb":40860,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":687,"num_bytes_recovered":31411166,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":563,"reported_epoch":289,"state":"active+clean","last_fresh":"2024-09-14T06:33:47.503995+0000","last_change":"2024-09-14T06:33:47.503995+0000","last_active":"2024-09-14T06:33:47.503995+0000","last_peered":"2024-09-14T06:33:47.503995+0000","last_clean":"2024-09-14T06:33:47.503995+0000","last_became_active":"2024-09-14T06:28:51.308072+0000","last_became_peered":"2024-09-14T06:28:51.308072+0000","last_unstale":"2024-09-14T06:33:47.503995+0000","last_undegraded":"2024-09-14T06:33:47.503995+0000","last_fullsized":"2024-09-14T06:33:47.503995+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:33:47.503914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:33:47.503914+0000","last_clean_scrub_stamp":"2024-09-14T06:33:47.503914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T16:25:10.268823+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0.00066197399999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"261'896","reported_seq":5633,"reported_epoch":289,"state":"active+clean","last_fresh":"2024-09-14T06:33:47.281955+0000","last_change":"2024-09-14T06:33:47.281955+0000","last_active":"2024-09-14T06:33:47.281955+0000","last_peered":"2024-09-14T06:33:47.281955+0000","last_clean":"2024-09-14T06:33:47.281955+0000","last_became_active":"2024-09-14T06:33:31.821533+0000","last_became_peered":"2024-09-14T06:33:31.821533+0000","last_unstale":"2024-09-14T06:33:47.281955+0000","last_undegraded":"2024-09-14T06:33:47.281955+0000","last_fullsized":"2024-09-14T06:33:47.281955+0000","mapping_epoch":285,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":286,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'896","last_scrub_stamp":"2024-09-14T06:33:47.281851+0000","last_deep_scrub":"261'896","last_deep_scrub_stamp":"2024-09-14T06:33:47.281851+0000","last_clean_scrub_stamp":"2024-09-14T06:33:47.281851+0000","objects_scrubbed":0,"log_size":896,"log_dups_size":0,"ondisk_log_size":896,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T07:54:48.266129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015088129999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4516,"num_read_kb":42632,"num_write":1255,"num_write_kb":49494,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":330,"num_bytes_recovered":28798893,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"289'712","reported_seq":10933,"reported_epoch":289,"state":"active+clean","last_fresh":"2024-09-14T06:33:44.730847+0000","last_change":"2024-09-14T06:33:44.730847+0000","last_active":"2024-09-14T06:33:44.730847+0000","last_peered":"2024-09-14T06:33:44.730847+0000","last_clean":"2024-09-14T06:33:44.730847+0000","last_became_active":"2024-09-14T06:29:00.584678+0000","last_became_peered":"2024-09-14T06:29:00.584678+0000","last_unstale":"2024-09-14T06:33:44.730847+0000","last_undegraded":"2024-09-14T06:33:44.730847+0000","last_fullsized":"2024-09-14T06:33:44.730847+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"289'712","last_scrub_stamp":"2024-09-14T06:33:44.730758+0000","last_deep_scrub":"289'712","last_deep_scrub_stamp":"2024-09-14T06:33:44.730758+0000","last_clean_scrub_stamp":"2024-09-14T06:33:44.730758+0000","objects_scrubbed":1,"log_size":712,"log_dups_size":0,"ondisk_log_size":712,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T07:38:17.566422+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0.00074987800000000005,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":9560,"num_read_kb":89311,"num_write":777,"num_write_kb":51691,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"289'3","reported_seq":437,"reported_epoch":289,"state":"active+clean","last_fresh":"2024-09-14T06:33:46.286757+0000","last_change":"2024-09-14T06:33:46.286757+0000","last_active":"2024-09-14T06:33:46.286757+0000","last_peered":"2024-09-14T06:33:46.286757+0000","last_clean":"2024-09-14T06:33:46.286757+0000","last_became_active":"2024-09-14T06:33:31.823429+0000","last_became_peered":"2024-09-14T06:33:31.823429+0000","last_unstale":"2024-09-14T06:33:46.286757+0000","last_undegraded":"2024-09-14T06:33:46.286757+0000","last_fullsized":"2024-09-14T06:33:46.286757+0000","mapping_epoch":285,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":286,"parent":"0.0","parent_split_bits":0,"last_scrub":"289'3","last_scrub_stamp":"2024-09-14T06:33:46.286673+0000","last_deep_scrub":"289'3","last_deep_scrub_stamp":"2024-09-14T06:33:46.286673+0000","last_clean_scrub_stamp":"2024-09-14T06:33:46.286673+0000","objects_scrubbed":1,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T16:35:00.418139+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0.0010072970000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.0","version":"260'1116","reported_seq":8252,"reported_epoch":289,"state":"active+clean","last_fresh":"2024-09-14T06:33:46.492866+0000","last_change":"2024-09-14T06:33:46.492866+0000","last_active":"2024-09-14T06:33:46.492866+0000","last_peered":"2024-09-14T06:33:46.492866+0000","last_clean":"2024-09-14T06:33:46.492866+0000","last_became_active":"2024-09-14T06:33:22.767520+0000","last_became_peered":"2024-09-14T06:33:22.767520+0000","last_unstale":"2024-09-14T06:33:46.492866+0000","last_undegraded":"2024-09-14T06:33:46.492866+0000","last_fullsized":"2024-09-14T06:33:46.492866+0000","mapping_epoch":276,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":277,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'1116","last_scrub_stamp":"2024-09-14T06:33:46.492759+0000","last_deep_scrub":"260'1116","last_deep_scrub_stamp":"2024-09-14T06:33:46.492759+0000","last_clean_scrub_stamp":"2024-09-14T06:33:46.492759+0000","objects_scrubbed":0,"log_size":1116,"log_dups_size":0,"ondisk_log_size":1116,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T09:28:37.050427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064233900000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":8259,"num_read_kb":85480,"num_write":1652,"num_write_kb":58700,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":537,"num_bytes_recovered":2852,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":48,"reported_epoch":289,"state":"active+clean","last_fresh":"2024-09-14T06:33:44.243739+0000","last_change":"2024-09-14T06:33:44.243739+0000","last_active":"2024-09-14T06:33:44.243739+0000","last_peered":"2024-09-14T06:33:44.243739+0000","last_clean":"2024-09-14T06:33:44.243739+0000","last_became_active":"2024-09-14T06:33:19.801557+0000","last_became_peered":"2024-09-14T06:33:19.801557+0000","last_unstale":"2024-09-14T06:33:44.243739+0000","last_undegraded":"2024-09-14T06:33:44.243739+0000","last_fullsized":"2024-09-14T06:33:44.243739+0000","mapping_epoch":273,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":274,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:33:44.243667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:33:44.243667+0000","last_clean_scrub_stamp":"2024-09-14T06:33:44.243667+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T16:10:15.433709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053403100000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1","version":"261'822","reported_seq":11894,"reported_epoch":289,"state":"active+clean","last_fresh":"2024-09-14T06:33:43.452215+0000","last_change":"2024-09-14T06:33:43.452215+0000","last_active":"2024-09-14T06:33:43.452215+0000","last_peered":"2024-09-14T06:33:43.452215+0000","last_clean":"2024-09-14T06:33:43.452215+0000","last_became_active":"2024-09-14T06:30:57.768238+0000","last_became_peered":"2024-09-14T06:30:57.768238+0000","last_unstale":"2024-09-14T06:33:43.452215+0000","last_undegraded":"2024-09-14T06:33:43.452215+0000","last_fullsized":"2024-09-14T06:33:43.452215+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'822","last_scrub_stamp":"2024-09-14T06:33:43.452095+0000","last_deep_scrub":"261'822","last_deep_scrub_stamp":"2024-09-14T06:33:43.452095+0000","last_clean_scrub_stamp":"2024-09-14T06:33:43.452095+0000","objects_scrubbed":0,"log_size":822,"log_dups_size":0,"ondisk_log_size":822,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T07:05:06.764250+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0.0011933110000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":11284,"num_read_kb":112430,"num_write":991,"num_write_kb":47408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":183,"num_bytes_recovered":30431994,"num_keys_recovered":22,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":562,"reported_epoch":289,"state":"active+clean","last_fresh":"2024-09-14T06:33:47.497389+0000","last_change":"2024-09-14T06:33:47.497389+0000","last_active":"2024-09-14T06:33:47.497389+0000","last_peered":"2024-09-14T06:33:47.497389+0000","last_clean":"2024-09-14T06:33:47.497389+0000","last_became_active":"2024-09-14T06:28:51.223069+0000","last_became_peered":"2024-09-14T06:28:51.223069+0000","last_unstale":"2024-09-14T06:33:47.497389+0000","last_undegraded":"2024-09-14T06:33:47.497389+0000","last_fullsized":"2024-09-14T06:33:47.497389+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-14T06:33:47.497269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-14T06:33:47.497269+0000","last_clean_scrub_stamp":"2024-09-14T06:33:47.497269+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T08:22:29.421824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00043714600000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"262'1663","reported_seq":7862,"reported_epoch":289,"state":"active+clean","last_fresh":"2024-09-14T06:33:44.364632+0000","last_change":"2024-09-14T06:33:44.364575+0000","last_active":"2024-09-14T06:33:44.364632+0000","last_peered":"2024-09-14T06:33:44.364632+0000","last_clean":"2024-09-14T06:33:44.364632+0000","last_became_active":"2024-09-14T06:31:54.744160+0000","last_became_peered":"2024-09-14T06:31:54.744160+0000","last_unstale":"2024-09-14T06:33:44.364632+0000","last_undegraded":"2024-09-14T06:33:44.364632+0000","last_fullsized":"2024-09-14T06:33:44.364632+0000","mapping_epoch":189,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":190,"parent":"0.0","parent_split_bits":0,"last_scrub":"262'1663","last_scrub_stamp":"2024-09-14T06:33:44.364500+0000","last_deep_scrub":"262'1663","last_deep_scrub_stamp":"2024-09-14T06:33:44.364500+0000","last_clean_scrub_stamp":"2024-09-14T06:33:44.364500+0000","objects_scrubbed":0,"log_size":1663,"log_dups_size":0,"ondisk_log_size":1663,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T10:03:41.366205+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0.001382608,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":10723,"num_read_kb":70589,"num_write":2596,"num_write_kb":53564,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":327,"num_bytes_recovered":50047063,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"289'2","reported_seq":436,"reported_epoch":289,"state":"active+clean","last_fresh":"2024-09-14T06:33:45.328424+0000","last_change":"2024-09-14T06:33:45.328424+0000","last_active":"2024-09-14T06:33:45.328424+0000","last_peered":"2024-09-14T06:33:45.328424+0000","last_clean":"2024-09-14T06:33:45.328424+0000","last_became_active":"2024-09-14T06:33:31.823032+0000","last_became_peered":"2024-09-14T06:33:31.823032+0000","last_unstale":"2024-09-14T06:33:45.328424+0000","last_undegraded":"2024-09-14T06:33:45.328424+0000","last_fullsized":"2024-09-14T06:33:45.328424+0000","mapping_epoch":285,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":286,"parent":"0.0","parent_split_bits":0,"last_scrub":"289'2","last_scrub_stamp":"2024-09-14T06:33:45.328339+0000","last_deep_scrub":"289'2","last_deep_scrub_stamp":"2024-09-14T06:33:45.328339+0000","last_clean_scrub_stamp":"2024-09-14T06:33:45.328339+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T06:56:50.349996+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0.00050808899999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"289'74","reported_seq":655,"reported_epoch":289,"state":"active+clean","last_fresh":"2024-09-14T06:33:46.540585+0000","last_change":"2024-09-14T06:33:46.540585+0000","last_active":"2024-09-14T06:33:46.540585+0000","last_peered":"2024-09-14T06:33:46.540585+0000","last_clean":"2024-09-14T06:33:46.540585+0000","last_became_active":"2024-09-14T06:28:44.839573+0000","last_became_peered":"2024-09-14T06:28:44.839573+0000","last_unstale":"2024-09-14T06:33:46.540585+0000","last_undegraded":"2024-09-14T06:33:46.540585+0000","last_fullsized":"2024-09-14T06:33:46.540585+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"289'74","last_scrub_stamp":"2024-09-14T06:33:46.540487+0000","last_deep_scrub":"289'74","last_deep_scrub_stamp":"2024-09-14T06:33:46.540487+0000","last_clean_scrub_stamp":"2024-09-14T06:33:46.540487+0000","objects_scrubbed":2,"log_size":74,"log_dups_size":0,"ondisk_log_size":74,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T15:38:38.616649+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.8","version":"261'684","reported_seq":6668,"reported_epoch":289,"state":"active+clean","last_fresh":"2024-09-14T06:33:46.276073+0000","last_change":"2024-09-14T06:33:46.276073+0000","last_active":"2024-09-14T06:33:46.276073+0000","last_peered":"2024-09-14T06:33:46.276073+0000","last_clean":"2024-09-14T06:33:46.276073+0000","last_became_active":"2024-09-14T06:31:55.754694+0000","last_became_peered":"2024-09-14T06:31:55.754694+0000","last_unstale":"2024-09-14T06:33:46.276073+0000","last_undegraded":"2024-09-14T06:33:46.276073+0000","last_fullsized":"2024-09-14T06:33:46.276073+0000","mapping_epoch":190,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":191,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'684","last_scrub_stamp":"2024-09-14T06:33:46.275990+0000","last_deep_scrub":"261'684","last_deep_scrub_stamp":"2024-09-14T06:33:46.275990+0000","last_clean_scrub_stamp":"2024-09-14T06:33:46.275990+0000","objects_scrubbed":0,"log_size":684,"log_dups_size":0,"ondisk_log_size":684,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T17:18:46.313210+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0.00067851599999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":5793,"num_read_kb":90852,"num_write":921,"num_write_kb":57691,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":319,"num_bytes_recovered":60352269,"num_keys_recovered":22,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.9","version":"261'597","reported_seq":5181,"reported_epoch":289,"state":"active+clean","last_fresh":"2024-09-14T06:33:47.728967+0000","last_change":"2024-09-14T06:33:47.728967+0000","last_active":"2024-09-14T06:33:47.728967+0000","last_peered":"2024-09-14T06:33:47.728967+0000","last_clean":"2024-09-14T06:33:47.728967+0000","last_became_active":"2024-09-14T06:33:22.770956+0000","last_became_peered":"2024-09-14T06:33:22.770956+0000","last_unstale":"2024-09-14T06:33:47.728967+0000","last_undegraded":"2024-09-14T06:33:47.728967+0000","last_fullsized":"2024-09-14T06:33:47.728967+0000","mapping_epoch":276,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":277,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'597","last_scrub_stamp":"2024-09-14T06:33:47.728857+0000","last_deep_scrub":"261'597","last_deep_scrub_stamp":"2024-09-14T06:33:47.728857+0000","last_clean_scrub_stamp":"2024-09-14T06:33:47.728857+0000","objects_scrubbed":0,"log_size":597,"log_dups_size":0,"ondisk_log_size":597,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T11:19:39.471923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057139500000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3678,"num_read_kb":64291,"num_write":778,"num_write_kb":44928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":316,"num_bytes_recovered":49117,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"261'874","reported_seq":5184,"reported_epoch":289,"state":"active+clean","last_fresh":"2024-09-14T06:33:46.829770+0000","last_change":"2024-09-14T06:33:46.829770+0000","last_active":"2024-09-14T06:33:46.829770+0000","last_peered":"2024-09-14T06:33:46.829770+0000","last_clean":"2024-09-14T06:33:46.829770+0000","last_became_active":"2024-09-14T06:33:17.627910+0000","last_became_peered":"2024-09-14T06:33:17.627910+0000","last_unstale":"2024-09-14T06:33:46.829770+0000","last_undegraded":"2024-09-14T06:33:46.829770+0000","last_fullsized":"2024-09-14T06:33:46.829770+0000","mapping_epoch":271,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":272,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'874","last_scrub_stamp":"2024-09-14T06:33:46.829677+0000","last_deep_scrub":"261'874","last_deep_scrub_stamp":"2024-09-14T06:33:46.829677+0000","last_clean_scrub_stamp":"2024-09-14T06:33:46.829677+0000","objects_scrubbed":0,"log_size":874,"log_dups_size":0,"ondisk_log_size":874,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T15:46:26.032857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00074887399999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":5331,"num_read_kb":58329,"num_write":1221,"num_write_kb":47535,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":656,"num_bytes_recovered":26968802,"num_keys_recovered":45,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.b","version":"260'1637","reported_seq":8996,"reported_epoch":289,"state":"active+clean","last_fresh":"2024-09-14T06:33:47.255326+0000","last_change":"2024-09-14T06:33:47.255326+0000","last_active":"2024-09-14T06:33:47.255326+0000","last_peered":"2024-09-14T06:33:47.255326+0000","last_clean":"2024-09-14T06:33:47.255326+0000","last_became_active":"2024-09-14T06:31:55.756219+0000","last_became_peered":"2024-09-14T06:31:55.756219+0000","last_unstale":"2024-09-14T06:33:47.255326+0000","last_undegraded":"2024-09-14T06:33:47.255326+0000","last_fullsized":"2024-09-14T06:33:47.255326+0000","mapping_epoch":190,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":191,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'1637","last_scrub_stamp":"2024-09-14T06:33:47.255231+0000","last_deep_scrub":"260'1637","last_deep_scrub_stamp":"2024-09-14T06:33:47.255231+0000","last_clean_scrub_stamp":"2024-09-14T06:33:47.255231+0000","objects_scrubbed":0,"log_size":1637,"log_dups_size":0,"ondisk_log_size":1637,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T12:06:08.693084+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0.00065814200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":13043,"num_read_kb":64129,"num_write":2496,"num_write_kb":43679,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":291,"num_bytes_recovered":37370632,"num_keys_recovered":66,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.c","version":"255'622","reported_seq":6278,"reported_epoch":289,"state":"active+clean","last_fresh":"2024-09-14T06:33:48.331434+0000","last_change":"2024-09-14T06:33:48.331434+0000","last_active":"2024-09-14T06:33:48.331434+0000","last_peered":"2024-09-14T06:33:48.331434+0000","last_clean":"2024-09-14T06:33:48.331434+0000","last_became_active":"2024-09-14T06:33:31.821638+0000","last_became_peered":"2024-09-14T06:33:31.821638+0000","last_unstale":"2024-09-14T06:33:48.331434+0000","last_undegraded":"2024-09-14T06:33:48.331434+0000","last_fullsized":"2024-09-14T06:33:48.331434+0000","mapping_epoch":285,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":286,"parent":"0.0","parent_split_bits":0,"last_scrub":"255'622","last_scrub_stamp":"2024-09-14T06:33:48.331298+0000","last_deep_scrub":"255'622","last_deep_scrub_stamp":"2024-09-14T06:33:48.331298+0000","last_clean_scrub_stamp":"2024-09-14T06:33:48.331298+0000","objects_scrubbed":0,"log_size":622,"log_dups_size":0,"ondisk_log_size":622,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T08:22:23.368926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013231740000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4842,"num_read_kb":50425,"num_write":795,"num_write_kb":40827,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":293,"num_bytes_recovered":36712421,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.d","version":"261'577","reported_seq":9336,"reported_epoch":289,"state":"active+clean","last_fresh":"2024-09-14T06:33:52.463474+0000","last_change":"2024-09-14T06:33:52.463474+0000","last_active":"2024-09-14T06:33:52.463474+0000","last_peered":"2024-09-14T06:33:52.463474+0000","last_clean":"2024-09-14T06:33:52.463474+0000","last_became_active":"2024-09-14T06:29:00.580362+0000","last_became_peered":"2024-09-14T06:29:00.580362+0000","last_unstale":"2024-09-14T06:33:52.463474+0000","last_undegraded":"2024-09-14T06:33:52.463474+0000","last_fullsized":"2024-09-14T06:33:52.463474+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'577","last_scrub_stamp":"2024-09-14T06:33:52.463381+0000","last_deep_scrub":"261'577","last_deep_scrub_stamp":"2024-09-14T06:33:52.463381+0000","last_clean_scrub_stamp":"2024-09-14T06:33:52.463381+0000","objects_scrubbed":0,"log_size":577,"log_dups_size":0,"ondisk_log_size":577,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T11:48:33.664155+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0.0010003270000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":8199,"num_read_kb":93694,"num_write":732,"num_write_kb":41032,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.e","version":"262'586","reported_seq":5534,"reported_epoch":289,"state":"active+clean","last_fresh":"2024-09-14T06:33:48.478897+0000","last_change":"2024-09-14T06:33:48.478897+0000","last_active":"2024-09-14T06:33:48.478897+0000","last_peered":"2024-09-14T06:33:48.478897+0000","last_clean":"2024-09-14T06:33:48.478897+0000","last_became_active":"2024-09-14T06:32:39.777524+0000","last_became_peered":"2024-09-14T06:32:39.777524+0000","last_unstale":"2024-09-14T06:33:48.478897+0000","last_undegraded":"2024-09-14T06:33:48.478897+0000","last_fullsized":"2024-09-14T06:33:48.478897+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"262'586","last_scrub_stamp":"2024-09-14T06:33:48.478807+0000","last_deep_scrub":"262'586","last_deep_scrub_stamp":"2024-09-14T06:33:48.478807+0000","last_clean_scrub_stamp":"2024-09-14T06:33:48.478807+0000","objects_scrubbed":0,"log_size":586,"log_dups_size":0,"ondisk_log_size":586,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T11:48:07.733211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065560200000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3986,"num_read_kb":61138,"num_write":753,"num_write_kb":43787,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":171,"num_bytes_recovered":10913227,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.f","version":"262'712","reported_seq":5956,"reported_epoch":289,"state":"active+clean","last_fresh":"2024-09-14T06:33:51.508954+0000","last_change":"2024-09-14T06:33:51.508954+0000","last_active":"2024-09-14T06:33:51.508954+0000","last_peered":"2024-09-14T06:33:51.508954+0000","last_clean":"2024-09-14T06:33:51.508954+0000","last_became_active":"2024-09-14T06:29:00.586491+0000","last_became_peered":"2024-09-14T06:29:00.586491+0000","last_unstale":"2024-09-14T06:33:51.508954+0000","last_undegraded":"2024-09-14T06:33:51.508954+0000","last_fullsized":"2024-09-14T06:33:51.508954+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"262'712","last_scrub_stamp":"2024-09-14T06:33:51.508857+0000","last_deep_scrub":"262'712","last_deep_scrub_stamp":"2024-09-14T06:33:51.508857+0000","last_clean_scrub_stamp":"2024-09-14T06:33:51.508857+0000","objects_scrubbed":0,"log_size":712,"log_dups_size":0,"ondisk_log_size":712,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T15:23:10.040193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000891002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":6081,"num_read_kb":45578,"num_write":836,"num_write_kb":36134,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.10","version":"261'952","reported_seq":6897,"reported_epoch":289,"state":"active+clean","last_fresh":"2024-09-14T06:33:47.837567+0000","last_change":"2024-09-14T06:33:47.837567+0000","last_active":"2024-09-14T06:33:47.837567+0000","last_peered":"2024-09-14T06:33:47.837567+0000","last_clean":"2024-09-14T06:33:47.837567+0000","last_became_active":"2024-09-14T06:33:16.628480+0000","last_became_peered":"2024-09-14T06:33:16.628480+0000","last_unstale":"2024-09-14T06:33:47.837567+0000","last_undegraded":"2024-09-14T06:33:47.837567+0000","last_fullsized":"2024-09-14T06:33:47.837567+0000","mapping_epoch":270,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":271,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'952","last_scrub_stamp":"2024-09-14T06:33:47.837478+0000","last_deep_scrub":"261'952","last_deep_scrub_stamp":"2024-09-14T06:33:47.837478+0000","last_clean_scrub_stamp":"2024-09-14T06:33:47.837478+0000","objects_scrubbed":0,"log_size":952,"log_dups_size":0,"ondisk_log_size":952,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T13:36:28.779713+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0.0012025289999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":7147,"num_read_kb":34474,"num_write":1082,"num_write_kb":36655,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":318,"num_bytes_recovered":9401812,"num_keys_recovered":45,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.11","version":"261'730","reported_seq":11755,"reported_epoch":289,"state":"active+clean","last_fresh":"2024-09-14T06:33:49.512141+0000","last_change":"2024-09-14T06:33:49.512141+0000","last_active":"2024-09-14T06:33:49.512141+0000","last_peered":"2024-09-14T06:33:49.512141+0000","last_clean":"2024-09-14T06:33:49.512141+0000","last_became_active":"2024-09-14T06:32:39.778010+0000","last_became_peered":"2024-09-14T06:32:39.778010+0000","last_unstale":"2024-09-14T06:33:49.512141+0000","last_undegraded":"2024-09-14T06:33:49.512141+0000","last_fullsized":"2024-09-14T06:33:49.512141+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'730","last_scrub_stamp":"2024-09-14T06:33:49.512049+0000","last_deep_scrub":"261'730","last_deep_scrub_stamp":"2024-09-14T06:33:49.512049+0000","last_clean_scrub_stamp":"2024-09-14T06:33:49.512049+0000","objects_scrubbed":0,"log_size":730,"log_dups_size":0,"ondisk_log_size":730,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T14:15:12.913324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00072597299999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":10271,"num_read_kb":109243,"num_write":859,"num_write_kb":48307,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":177,"num_bytes_recovered":16647747,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.12","version":"261'782","reported_seq":7514,"reported_epoch":289,"state":"active+clean","last_fresh":"2024-09-14T06:33:44.438706+0000","last_change":"2024-09-14T06:33:44.438706+0000","last_active":"2024-09-14T06:33:44.438706+0000","last_peered":"2024-09-14T06:33:44.438706+0000","last_clean":"2024-09-14T06:33:44.438706+0000","last_became_active":"2024-09-14T06:29:00.579826+0000","last_became_peered":"2024-09-14T06:29:00.579826+0000","last_unstale":"2024-09-14T06:33:44.438706+0000","last_undegraded":"2024-09-14T06:33:44.438706+0000","last_fullsized":"2024-09-14T06:33:44.438706+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'782","last_scrub_stamp":"2024-09-14T06:33:44.438606+0000","last_deep_scrub":"261'782","last_deep_scrub_stamp":"2024-09-14T06:33:44.438606+0000","last_clean_scrub_stamp":"2024-09-14T06:33:44.438606+0000","objects_scrubbed":0,"log_size":782,"log_dups_size":0,"ondisk_log_size":782,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T17:20:39.783810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001680792,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":7110,"num_read_kb":59089,"num_write":947,"num_write_kb":42441,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.13","version":"261'1008","reported_seq":8968,"reported_epoch":289,"state":"active+clean","last_fresh":"2024-09-14T06:33:50.551771+0000","last_change":"2024-09-14T06:33:50.551771+0000","last_active":"2024-09-14T06:33:50.551771+0000","last_peered":"2024-09-14T06:33:50.551771+0000","last_clean":"2024-09-14T06:33:50.551771+0000","last_became_active":"2024-09-14T06:29:00.586520+0000","last_became_peered":"2024-09-14T06:29:00.586520+0000","last_unstale":"2024-09-14T06:33:50.551771+0000","last_undegraded":"2024-09-14T06:33:50.551771+0000","last_fullsized":"2024-09-14T06:33:50.551771+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"261'1008","last_scrub_stamp":"2024-09-14T06:33:50.551680+0000","last_deep_scrub":"261'1008","last_deep_scrub_stamp":"2024-09-14T06:33:50.551680+0000","last_clean_scrub_stamp":"2024-09-14T06:33:50.551680+0000","objects_scrubbed":0,"log_size":1008,"log_dups_size":0,"ondisk_log_size":1008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T08:34:30.267575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041539399999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":9551,"num_read_kb":66119,"num_write":1235,"num_write_kb":47437,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":19,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":240092,"num_read_kb":2184933,"num_write":38145,"num_write_kb":1478386,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7477,"num_bytes_recovered":638497624,"num_keys_recovered":546,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":28651,"ondisk_log_size":28651,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":5,"ondisk_log_size":5,"up":16,"acting":16,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":131072,"data_stored":1180736,"data_compressed":65166,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":74,"ondisk_log_size":74,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509508,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":70764,"kb_used_data":3484,"kb_used_omap":1,"kb_used_meta":67262,"kb_avail":94301076,"statfs":{"total":96636764160,"available":96564301824,"internally_reserved":0,"allocated":3567616,"data_stored":2317757,"data_compressed":32583,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":1590,"internal_metadata":68876746},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509508,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":56812,"kb_used_data":3420,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94315028,"statfs":{"total":96636764160,"available":96578588672,"internally_reserved":0,"allocated":3502080,"data_stored":1727389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":54655430},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509509,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":74864,"kb_used_data":3424,"kb_used_omap":1,"kb_used_meta":71422,"kb_avail":94296976,"statfs":{"total":96636764160,"available":96560103424,"internally_reserved":0,"allocated":3506176,"data_stored":1727408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":73136581},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509508,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":88752,"kb_used_data":3424,"kb_used_omap":1,"kb_used_meta":85310,"kb_avail":94283088,"statfs":{"total":96636764160,"available":96545882112,"internally_reserved":0,"allocated":3506176,"data_stored":1727408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":87357893},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509509,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":56820,"kb_used_data":3428,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94315020,"statfs":{"total":96636764160,"available":96578580480,"internally_reserved":0,"allocated":3510272,"data_stored":1727427,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":14,"seq":60129542212,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":60972,"kb_used_data":3420,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94310868,"statfs":{"total":96636764160,"available":96574328832,"internally_reserved":0,"allocated":3502080,"data_stored":1727389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":58915270},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542211,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":83244,"kb_used_data":3484,"kb_used_omap":1,"kb_used_meta":79742,"kb_avail":94288596,"statfs":{"total":96636764160,"available":96551522304,"internally_reserved":0,"allocated":3567616,"data_stored":2317757,"data_compressed":32583,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":1595,"internal_metadata":81656261},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574916,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":83180,"kb_used_data":3420,"kb_used_omap":1,"kb_used_meta":79742,"kb_avail":94288660,"statfs":{"total":96636764160,"available":96551587840,"internally_reserved":0,"allocated":3502080,"data_stored":1727389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":81656263},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32583,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32583,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-14T06:34:06.317 DEBUG:teuthology.orchestra.run.smithi094:> sudo ceph --cluster ceph config set global mon_health_to_clog false 2024-09-14T06:34:06.618 INFO:teuthology.misc:Shutting down mds daemons... 2024-09-14T06:34:06.618 INFO:teuthology.misc:Shutting down osd daemons... 2024-09-14T06:34:06.618 DEBUG:tasks.ceph.osd.0:waiting for process to exit 2024-09-14T06:34:06.619 INFO:teuthology.orchestra.run:waiting for 300 2024-09-14T06:34:06.665 INFO:tasks.ceph.osd.0:Stopped 2024-09-14T06:34:06.665 DEBUG:tasks.ceph.osd.1:waiting for process to exit 2024-09-14T06:34:06.666 INFO:teuthology.orchestra.run:waiting for 300 2024-09-14T06:34:06.707 INFO:tasks.ceph.osd.1:Stopped 2024-09-14T06:34:06.707 DEBUG:tasks.ceph.osd.2:waiting for process to exit 2024-09-14T06:34:06.708 INFO:teuthology.orchestra.run:waiting for 300 2024-09-14T06:34:06.748 INFO:tasks.ceph.osd.2:Stopped 2024-09-14T06:34:06.749 DEBUG:tasks.ceph.osd.3:waiting for process to exit 2024-09-14T06:34:06.749 INFO:teuthology.orchestra.run:waiting for 300 2024-09-14T06:34:06.787 INFO:tasks.ceph.osd.3:Stopped 2024-09-14T06:34:06.788 DEBUG:tasks.ceph.osd.4:waiting for process to exit 2024-09-14T06:34:06.788 INFO:teuthology.orchestra.run:waiting for 300 2024-09-14T06:34:06.833 INFO:tasks.ceph.osd.4:Stopped 2024-09-14T06:34:06.833 DEBUG:tasks.ceph.osd.5:waiting for process to exit 2024-09-14T06:34:06.834 INFO:teuthology.orchestra.run:waiting for 300 2024-09-14T06:34:06.876 INFO:tasks.ceph.osd.5:Stopped 2024-09-14T06:34:06.876 DEBUG:tasks.ceph.osd.6:waiting for process to exit 2024-09-14T06:34:06.877 INFO:teuthology.orchestra.run:waiting for 300 2024-09-14T06:34:06.920 INFO:tasks.ceph.osd.6:Stopped 2024-09-14T06:34:06.920 DEBUG:tasks.ceph.osd.7:waiting for process to exit 2024-09-14T06:34:06.920 INFO:teuthology.orchestra.run:waiting for 300 2024-09-14T06:34:06.958 INFO:tasks.ceph.osd.7:Stopped 2024-09-14T06:34:06.958 INFO:teuthology.misc:Shutting down mgr daemons... 2024-09-14T06:34:06.959 DEBUG:tasks.ceph.mgr.y:waiting for process to exit 2024-09-14T06:34:06.959 INFO:teuthology.orchestra.run:waiting for 300 2024-09-14T06:34:06.985 INFO:tasks.ceph.mgr.y:Stopped 2024-09-14T06:34:06.985 DEBUG:tasks.ceph.mgr.x:waiting for process to exit 2024-09-14T06:34:06.986 INFO:teuthology.orchestra.run:waiting for 300 2024-09-14T06:34:07.027 INFO:tasks.ceph.mgr.x:Stopped 2024-09-14T06:34:07.027 INFO:teuthology.misc:Shutting down mon daemons... 2024-09-14T06:34:07.028 DEBUG:tasks.ceph.mon.a:waiting for process to exit 2024-09-14T06:34:07.028 INFO:teuthology.orchestra.run:waiting for 300 2024-09-14T06:34:07.028 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mgr.x is failed for ~0s 2024-09-14T06:34:07.039 INFO:tasks.ceph.mon.a:Stopped 2024-09-14T06:34:07.039 DEBUG:tasks.ceph.mon.c:waiting for process to exit 2024-09-14T06:34:07.039 INFO:teuthology.orchestra.run:waiting for 300 2024-09-14T06:34:07.090 INFO:tasks.ceph.mon.c:Stopped 2024-09-14T06:34:07.091 DEBUG:tasks.ceph.mon.b:waiting for process to exit 2024-09-14T06:34:07.091 INFO:teuthology.orchestra.run:waiting for 300 2024-09-14T06:34:07.104 INFO:tasks.ceph.mon.b:Stopped 2024-09-14T06:34:07.104 INFO:tasks.ceph:Checking cluster log for badness... 2024-09-14T06:34:07.104 DEBUG:teuthology.orchestra.run.smithi094:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v '\(OSD_SLOW_PING_TIME' | egrep -v 'but it is still running' | egrep -v 'objects unfound and apparently lost' | egrep -v 'overall HEALTH_' | egrep -v '\(OSDMAP_FLAGS\)' | egrep -v '\(OSD_' | egrep -v '\(PG_' | egrep -v '\(POOL_' | egrep -v '\(CACHE_POOL_' | egrep -v '\(SMALLER_PGP_NUM\)' | egrep -v '\(OBJECT_' | egrep -v SLOW_OPS | egrep -v '\(REQUEST_SLOW\)' | egrep -v '\(TOO_FEW_PGS\)' | egrep -v 'slow request' | egrep -v 'timeout on replica' | egrep -v 'late reservation from' | egrep -v MON_DOWN | egrep -v OSDMAP_FLAGS | egrep -v OSD_DOWN | egrep -v PG_DEGRADED | egrep -v PG_AVAILABILITY | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'mons down' | egrep -v 'mon down' | egrep -v 'out of quorum' | egrep -v noscrub | egrep -v nodeep-scrub | head -n 1 2024-09-14T06:34:07.163 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-0 on ubuntu@smithi094.front.sepia.ceph.com 2024-09-14T06:34:07.163 DEBUG:teuthology.orchestra.run.smithi094:> sync && sudo umount -f /var/lib/ceph/osd/ceph-0 2024-09-14T06:34:07.404 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-1 on ubuntu@smithi094.front.sepia.ceph.com 2024-09-14T06:34:07.404 DEBUG:teuthology.orchestra.run.smithi094:> sync && sudo umount -f /var/lib/ceph/osd/ceph-1 2024-09-14T06:34:07.484 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-2 on ubuntu@smithi094.front.sepia.ceph.com 2024-09-14T06:34:07.485 DEBUG:teuthology.orchestra.run.smithi094:> sync && sudo umount -f /var/lib/ceph/osd/ceph-2 2024-09-14T06:34:07.540 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-3 on ubuntu@smithi094.front.sepia.ceph.com 2024-09-14T06:34:07.540 DEBUG:teuthology.orchestra.run.smithi094:> sync && sudo umount -f /var/lib/ceph/osd/ceph-3 2024-09-14T06:34:07.603 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-4 on ubuntu@smithi138.front.sepia.ceph.com 2024-09-14T06:34:07.603 DEBUG:teuthology.orchestra.run.smithi138:> sync && sudo umount -f /var/lib/ceph/osd/ceph-4 2024-09-14T06:34:07.724 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-5 on ubuntu@smithi138.front.sepia.ceph.com 2024-09-14T06:34:07.724 DEBUG:teuthology.orchestra.run.smithi138:> sync && sudo umount -f /var/lib/ceph/osd/ceph-5 2024-09-14T06:34:07.793 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-6 on ubuntu@smithi138.front.sepia.ceph.com 2024-09-14T06:34:07.793 DEBUG:teuthology.orchestra.run.smithi138:> sync && sudo umount -f /var/lib/ceph/osd/ceph-6 2024-09-14T06:34:07.840 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-7 on ubuntu@smithi138.front.sepia.ceph.com 2024-09-14T06:34:07.840 DEBUG:teuthology.orchestra.run.smithi138:> sync && sudo umount -f /var/lib/ceph/osd/ceph-7 2024-09-14T06:34:07.900 INFO:tasks.ceph:Archiving mon data... 2024-09-14T06:34:07.901 DEBUG:teuthology.misc:Transferring archived files from smithi094:/var/lib/ceph/mon/ceph-a to /home/teuthworker/archive/teuthology-2024-09-10_20:16:17-rbd-main-distro-default-smithi/7900381/data/mon.a.tgz 2024-09-14T06:34:07.901 DEBUG:teuthology.orchestra.run.smithi094:> mktemp 2024-09-14T06:34:07.917 INFO:teuthology.orchestra.run.smithi094.stdout:/tmp/tmp.itPyu22sZ0 2024-09-14T06:34:07.918 DEBUG:teuthology.orchestra.run.smithi094:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-a -- . > /tmp/tmp.itPyu22sZ0 2024-09-14T06:34:08.106 DEBUG:teuthology.orchestra.run.smithi094:> sudo chmod 0666 /tmp/tmp.itPyu22sZ0 2024-09-14T06:34:08.190 DEBUG:teuthology.orchestra.remote:smithi094:/tmp/tmp.itPyu22sZ0 is 542KB 2024-09-14T06:34:08.289 DEBUG:teuthology.orchestra.run.smithi094:> rm -fr /tmp/tmp.itPyu22sZ0 2024-09-14T06:34:08.304 DEBUG:teuthology.misc:Transferring archived files from smithi094:/var/lib/ceph/mon/ceph-c to /home/teuthworker/archive/teuthology-2024-09-10_20:16:17-rbd-main-distro-default-smithi/7900381/data/mon.c.tgz 2024-09-14T06:34:08.304 DEBUG:teuthology.orchestra.run.smithi094:> mktemp 2024-09-14T06:34:08.358 INFO:teuthology.orchestra.run.smithi094.stdout:/tmp/tmp.37lxW8LH4f 2024-09-14T06:34:08.358 DEBUG:teuthology.orchestra.run.smithi094:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-c -- . > /tmp/tmp.37lxW8LH4f 2024-09-14T06:34:08.576 DEBUG:teuthology.orchestra.run.smithi094:> sudo chmod 0666 /tmp/tmp.37lxW8LH4f 2024-09-14T06:34:08.658 DEBUG:teuthology.orchestra.remote:smithi094:/tmp/tmp.37lxW8LH4f is 718KB 2024-09-14T06:34:08.759 DEBUG:teuthology.orchestra.run.smithi094:> rm -fr /tmp/tmp.37lxW8LH4f 2024-09-14T06:34:08.775 DEBUG:teuthology.misc:Transferring archived files from smithi138:/var/lib/ceph/mon/ceph-b to /home/teuthworker/archive/teuthology-2024-09-10_20:16:17-rbd-main-distro-default-smithi/7900381/data/mon.b.tgz 2024-09-14T06:34:08.775 DEBUG:teuthology.orchestra.run.smithi138:> mktemp 2024-09-14T06:34:08.794 INFO:teuthology.orchestra.run.smithi138.stdout:/tmp/tmp.xK6XQi4MrO 2024-09-14T06:34:08.794 DEBUG:teuthology.orchestra.run.smithi138:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-b -- . > /tmp/tmp.xK6XQi4MrO 2024-09-14T06:34:09.028 DEBUG:teuthology.orchestra.run.smithi138:> sudo chmod 0666 /tmp/tmp.xK6XQi4MrO 2024-09-14T06:34:09.111 DEBUG:teuthology.orchestra.remote:smithi138:/tmp/tmp.xK6XQi4MrO is 719KB 2024-09-14T06:34:09.183 DEBUG:teuthology.orchestra.run.smithi138:> rm -fr /tmp/tmp.xK6XQi4MrO 2024-09-14T06:34:09.198 INFO:tasks.ceph:Cleaning ceph cluster... 2024-09-14T06:34:09.198 DEBUG:teuthology.orchestra.run.smithi094:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-09-14T06:34:09.200 DEBUG:teuthology.orchestra.run.smithi138:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-09-14T06:34:09.280 INFO:teuthology.util.scanner:summary_data or yaml_file is empty! 2024-09-14T06:34:09.318 INFO:teuthology.util.scanner:summary_data or yaml_file is empty! 2024-09-14T06:34:09.319 INFO:tasks.ceph:Archiving crash dumps... 2024-09-14T06:34:09.320 DEBUG:teuthology.misc:Transferring archived files from smithi094:/var/lib/ceph/crash to /home/teuthworker/archive/teuthology-2024-09-10_20:16:17-rbd-main-distro-default-smithi/7900381/remote/smithi094/crash 2024-09-14T06:34:09.321 DEBUG:teuthology.orchestra.run.smithi094:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-09-14T06:34:09.346 DEBUG:teuthology.misc:Transferring archived files from smithi138:/var/lib/ceph/crash to /home/teuthworker/archive/teuthology-2024-09-10_20:16:17-rbd-main-distro-default-smithi/7900381/remote/smithi138/crash 2024-09-14T06:34:09.347 DEBUG:teuthology.orchestra.run.smithi138:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-09-14T06:34:09.382 INFO:tasks.ceph:Compressing logs... 2024-09-14T06:34:09.383 DEBUG:teuthology.orchestra.run.smithi094:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-09-14T06:34:09.390 DEBUG:teuthology.orchestra.run.smithi138:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-09-14T06:34:09.463 INFO:teuthology.orchestra.run.smithi138.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33406.log 2024-09-14T06:34:09.463 INFO:teuthology.orchestra.run.smithi138.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.7.log 2024-09-14T06:34:09.463 INFO:teuthology.orchestra.run.smithi138.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33486.log 2024-09-14T06:34:09.463 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.33406.log: /var/log/ceph/ceph-osd.7.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33406.log.gz 2024-09-14T06:34:09.463 INFO:teuthology.orchestra.run.smithi138.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.4.log 2024-09-14T06:34:09.463 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.33486.log: gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2024-09-14T06:34:09.463 INFO:teuthology.orchestra.run.smithi138.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33486.log.gz 2024-09-14T06:34:09.463 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-osd.4.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33277.log 2024-09-14T06:34:09.463 INFO:teuthology.orchestra.run.smithi138.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33446.log 2024-09-14T06:34:09.463 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph.audit.log: /var/log/ceph/ceph-client.admin.33277.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33277.log.gz 2024-09-14T06:34:09.463 INFO:teuthology.orchestra.run.smithi138.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33526.log 2024-09-14T06:34:09.463 INFO:teuthology.orchestra.run.smithi138.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33366.log 2024-09-14T06:34:09.463 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.33446.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33446.log.gz 2024-09-14T06:34:09.464 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.33526.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33526.log.gzgzip 2024-09-14T06:34:09.464 INFO:teuthology.orchestra.run.smithi138.stderr: -5 --verbose -- /var/log/ceph/ceph-osd.6.log 2024-09-14T06:34:09.464 INFO:teuthology.orchestra.run.smithi138.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.5.log 2024-09-14T06:34:09.464 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.33366.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33366.log.gz 2024-09-14T06:34:09.464 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-osd.6.log: gzip -5 --verbose -- /var/log/ceph/ceph.log 2024-09-14T06:34:09.464 INFO:teuthology.orchestra.run.smithi138.stderr: 89.5% -- replaced with /var/log/ceph/ceph.audit.log.gz 2024-09-14T06:34:09.464 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-osd.5.log: gzip -5 --verbose -- /var/log/ceph/ceph-mgr.x.log 2024-09-14T06:34:09.464 INFO:teuthology.orchestra.run.smithi138.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33326.log/var/log/ceph/ceph.log: 2024-09-14T06:34:09.464 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-mgr.x.log: gzip -5 --verbose -- /var/log/ceph/ceph-mon.b.log 2024-09-14T06:34:09.464 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.33326.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33326.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33235.log 2024-09-14T06:34:09.465 INFO:teuthology.orchestra.run.smithi138.stderr: 2024-09-14T06:34:09.465 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-mon.b.log: 85.0% -- replaced with /var/log/ceph/ceph.log.gz 2024-09-14T06:34:09.475 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.33235.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33235.log.gz 2024-09-14T06:34:09.526 INFO:teuthology.orchestra.run.smithi138.stderr: 91.9% -- replaced with /var/log/ceph/ceph-mgr.x.log.gz 2024-09-14T06:34:09.610 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.0.log 2024-09-14T06:34:09.610 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39119.log 2024-09-14T06:34:09.611 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-osd.0.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48628.log 2024-09-14T06:34:09.611 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.39119.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51950.log 2024-09-14T06:34:09.611 INFO:teuthology.orchestra.run.smithi094.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39119.log.gz 2024-09-14T06:34:09.611 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.48628.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72969.log 2024-09-14T06:34:09.611 INFO:teuthology.orchestra.run.smithi094.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48628.log.gz 2024-09-14T06:34:09.611 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.51950.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71583.log 2024-09-14T06:34:09.611 INFO:teuthology.orchestra.run.smithi094.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.51950.log.gz 2024-09-14T06:34:09.611 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.72969.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.72969.log.gz 2024-09-14T06:34:09.611 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55950.log 2024-09-14T06:34:09.611 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.71583.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59606.log 2024-09-14T06:34:09.611 INFO:teuthology.orchestra.run.smithi094.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.71583.log.gz 2024-09-14T06:34:09.611 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.55950.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70402.log 2024-09-14T06:34:09.611 INFO:teuthology.orchestra.run.smithi094.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.55950.log.gz 2024-09-14T06:34:09.611 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.59606.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55869.log 2024-09-14T06:34:09.611 INFO:teuthology.orchestra.run.smithi094.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.59606.log.gz 2024-09-14T06:34:09.612 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.70402.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72675.log 2024-09-14T06:34:09.612 INFO:teuthology.orchestra.run.smithi094.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.70402.log.gz 2024-09-14T06:34:09.612 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.55869.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70590.log 2024-09-14T06:34:09.612 INFO:teuthology.orchestra.run.smithi094.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.55869.log.gz 2024-09-14T06:34:09.612 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.72675.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47452.log 2024-09-14T06:34:09.612 INFO:teuthology.orchestra.run.smithi094.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.72675.log.gz 2024-09-14T06:34:09.612 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.70590.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56830.log 2024-09-14T06:34:09.612 INFO:teuthology.orchestra.run.smithi094.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.70590.log.gz 2024-09-14T06:34:09.612 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.47452.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66206.log 2024-09-14T06:34:09.612 INFO:teuthology.orchestra.run.smithi094.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.47452.log.gz 2024-09-14T06:34:09.612 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.56830.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57069.log 2024-09-14T06:34:09.612 INFO:teuthology.orchestra.run.smithi094.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.56830.log.gz 2024-09-14T06:34:09.612 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.66206.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72468.log 2024-09-14T06:34:09.612 INFO:teuthology.orchestra.run.smithi094.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.66206.log.gz 2024-09-14T06:34:09.613 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.57069.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46338.log 2024-09-14T06:34:09.613 INFO:teuthology.orchestra.run.smithi094.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.57069.log.gz 2024-09-14T06:34:09.613 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.72468.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55812.log 2024-09-14T06:34:09.613 INFO:teuthology.orchestra.run.smithi094.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.72468.log.gz 2024-09-14T06:34:09.613 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.46338.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40755.log 2024-09-14T06:34:09.613 INFO:teuthology.orchestra.run.smithi094.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.46338.log.gz 2024-09-14T06:34:09.613 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.55812.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45778.log 2024-09-14T06:34:09.613 INFO:teuthology.orchestra.run.smithi094.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.55812.log.gz 2024-09-14T06:34:09.613 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.40755.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71163.log 2024-09-14T06:34:09.613 INFO:teuthology.orchestra.run.smithi094.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40755.log.gz 2024-09-14T06:34:09.613 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56630.log 2024-09-14T06:34:09.613 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.45778.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.45778.log.gz 2024-09-14T06:34:09.613 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.71163.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58928.log 2024-09-14T06:34:09.614 INFO:teuthology.orchestra.run.smithi094.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.71163.log.gz 2024-09-14T06:34:09.614 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.56630.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69165.log 2024-09-14T06:34:09.614 INFO:teuthology.orchestra.run.smithi094.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.56630.log.gz 2024-09-14T06:34:09.614 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.58928.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42364.log 2024-09-14T06:34:09.614 INFO:teuthology.orchestra.run.smithi094.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.58928.log.gz 2024-09-14T06:34:09.614 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.69165.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37971.log 2024-09-14T06:34:09.614 INFO:teuthology.orchestra.run.smithi094.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.69165.log.gz 2024-09-14T06:34:09.614 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.42364.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46117.log 2024-09-14T06:34:09.614 INFO:teuthology.orchestra.run.smithi094.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42364.log.gz 2024-09-14T06:34:09.614 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.37971.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50102.log 2024-09-14T06:34:09.614 INFO:teuthology.orchestra.run.smithi094.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37971.log.gz 2024-09-14T06:34:09.614 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.46117.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58607.log 2024-09-14T06:34:09.614 INFO:teuthology.orchestra.run.smithi094.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.46117.log.gz 2024-09-14T06:34:09.614 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.50102.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.50102.log.gz 2024-09-14T06:34:09.614 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46943.log 2024-09-14T06:34:09.615 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.58607.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.58607.log.gz 2024-09-14T06:34:09.615 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37457.log 2024-09-14T06:34:09.615 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.46943.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42643.log 2024-09-14T06:34:09.615 INFO:teuthology.orchestra.run.smithi094.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.46943.log.gz 2024-09-14T06:34:09.615 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.37457.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52237.log 2024-09-14T06:34:09.615 INFO:teuthology.orchestra.run.smithi094.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37457.log.gz 2024-09-14T06:34:09.615 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.38875.log 2024-09-14T06:34:09.615 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.42643.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42643.log.gz 2024-09-14T06:34:09.615 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.52237.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47004.log 2024-09-14T06:34:09.615 INFO:teuthology.orchestra.run.smithi094.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.52237.log.gz 2024-09-14T06:34:09.615 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.0.38875.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61000.log 2024-09-14T06:34:09.615 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.47004.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57166.log 2024-09-14T06:34:09.615 INFO:teuthology.orchestra.run.smithi094.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.47004.log.gz 2024-09-14T06:34:09.615 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.61000.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73515.log 2024-09-14T06:34:09.616 INFO:teuthology.orchestra.run.smithi094.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.61000.log.gz 2024-09-14T06:34:09.616 INFO:teuthology.orchestra.run.smithi094.stderr: 95.6% -- replaced with /var/log/ceph/ceph-client.0.38875.log.gz 2024-09-14T06:34:09.616 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.57166.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73473.log 2024-09-14T06:34:09.616 INFO:teuthology.orchestra.run.smithi094.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.57166.log.gz 2024-09-14T06:34:09.616 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.73515.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69541.log 2024-09-14T06:34:09.616 INFO:teuthology.orchestra.run.smithi094.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.73515.log.gz 2024-09-14T06:34:09.616 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.73473.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61875.log 2024-09-14T06:34:09.616 INFO:teuthology.orchestra.run.smithi094.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.73473.log.gz 2024-09-14T06:34:09.616 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.69541.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70953.log 2024-09-14T06:34:09.616 INFO:teuthology.orchestra.run.smithi094.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.69541.log.gz 2024-09-14T06:34:09.616 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.61875.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46683.log 2024-09-14T06:34:09.616 INFO:teuthology.orchestra.run.smithi094.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.61875.log.gz 2024-09-14T06:34:09.616 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.70953.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65964.log 2024-09-14T06:34:09.616 INFO:teuthology.orchestra.run.smithi094.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.70953.log.gz 2024-09-14T06:34:09.616 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.46683.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59426.log 2024-09-14T06:34:09.616 INFO:teuthology.orchestra.run.smithi094.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.46683.log.gz 2024-09-14T06:34:09.617 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.65964.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71803.log 2024-09-14T06:34:09.617 INFO:teuthology.orchestra.run.smithi094.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.65964.log.gz 2024-09-14T06:34:09.617 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.59426.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51846.log 2024-09-14T06:34:09.617 INFO:teuthology.orchestra.run.smithi094.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.59426.log.gz 2024-09-14T06:34:09.617 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.71803.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68284.log 2024-09-14T06:34:09.617 INFO:teuthology.orchestra.run.smithi094.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.71803.log.gz 2024-09-14T06:34:09.617 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.51846.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37965.log 2024-09-14T06:34:09.617 INFO:teuthology.orchestra.run.smithi094.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.51846.log.gz 2024-09-14T06:34:09.617 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.68284.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50706.log 2024-09-14T06:34:09.617 INFO:teuthology.orchestra.run.smithi094.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.68284.log.gz 2024-09-14T06:34:09.617 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.37965.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60600.log 2024-09-14T06:34:09.617 INFO:teuthology.orchestra.run.smithi094.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37965.log.gz 2024-09-14T06:34:09.617 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.50706.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42802.log 2024-09-14T06:34:09.618 INFO:teuthology.orchestra.run.smithi094.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.50706.log.gz 2024-09-14T06:34:09.618 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.60600.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65272.log 2024-09-14T06:34:09.618 INFO:teuthology.orchestra.run.smithi094.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.60600.log.gz 2024-09-14T06:34:09.618 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.42802.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43747.log 2024-09-14T06:34:09.618 INFO:teuthology.orchestra.run.smithi094.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42802.log.gz 2024-09-14T06:34:09.618 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.65272.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.1.log 2024-09-14T06:34:09.618 INFO:teuthology.orchestra.run.smithi094.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.65272.log.gz 2024-09-14T06:34:09.618 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54178.log 2024-09-14T06:34:09.618 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.43747.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43747.log.gz 2024-09-14T06:34:09.618 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61158.log 2024-09-14T06:34:09.618 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.54178.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58368.log 2024-09-14T06:34:09.618 INFO:teuthology.orchestra.run.smithi094.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.54178.log.gz 2024-09-14T06:34:09.618 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.61158.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60682.log 2024-09-14T06:34:09.618 INFO:teuthology.orchestra.run.smithi094.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.61158.log.gz 2024-09-14T06:34:09.618 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.58368.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.58368.log.gz 2024-09-14T06:34:09.618 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38352.log 2024-09-14T06:34:09.619 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.60682.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71121.log 2024-09-14T06:34:09.619 INFO:teuthology.orchestra.run.smithi094.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.60682.log.gz 2024-09-14T06:34:09.619 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.38352.log: gzip -5 --verbose -- 0.0% /var/log/ceph/ceph-client.admin.70995.log -- replaced with /var/log/ceph/ceph-client.admin.38352.log.gz 2024-09-14T06:34:09.619 INFO:teuthology.orchestra.run.smithi094.stderr: 2024-09-14T06:34:09.619 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.71121.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60443.log 2024-09-14T06:34:09.619 INFO:teuthology.orchestra.run.smithi094.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.71121.log.gz 2024-09-14T06:34:09.619 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.70995.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73431.log 2024-09-14T06:34:09.619 INFO:teuthology.orchestra.run.smithi094.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.70995.log.gz 2024-09-14T06:34:09.619 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.60443.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58748.log 2024-09-14T06:34:09.619 INFO:teuthology.orchestra.run.smithi094.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.60443.log.gz 2024-09-14T06:34:09.619 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.73431.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62740.log 2024-09-14T06:34:09.619 INFO:teuthology.orchestra.run.smithi094.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.73431.log.gz 2024-09-14T06:34:09.619 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.58748.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53190.log 2024-09-14T06:34:09.619 INFO:teuthology.orchestra.run.smithi094.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.58748.log.gz 2024-09-14T06:34:09.620 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.62740.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62930.log 2024-09-14T06:34:09.620 INFO:teuthology.orchestra.run.smithi094.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.62740.log.gz 2024-09-14T06:34:09.620 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.53190.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37855.log 2024-09-14T06:34:09.620 INFO:teuthology.orchestra.run.smithi094.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.53190.log.gz 2024-09-14T06:34:09.620 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.62930.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.62930.log.gz 2024-09-14T06:34:09.620 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73641.log 2024-09-14T06:34:09.620 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.37855.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56113.log 2024-09-14T06:34:09.620 INFO:teuthology.orchestra.run.smithi094.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37855.log.gz 2024-09-14T06:34:09.620 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.73641.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38290.log 2024-09-14T06:34:09.620 INFO:teuthology.orchestra.run.smithi094.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.73641.log.gz 2024-09-14T06:34:09.620 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.56113.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58410.log 2024-09-14T06:34:09.620 INFO:teuthology.orchestra.run.smithi094.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.56113.log.gz 2024-09-14T06:34:09.620 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.38290.log: 0.0%gzip -- replaced with /var/log/ceph/ceph-client.admin.38290.log.gz 2024-09-14T06:34:09.620 INFO:teuthology.orchestra.run.smithi094.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.38591.log 2024-09-14T06:34:09.620 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.58410.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59711.log 2024-09-14T06:34:09.620 INFO:teuthology.orchestra.run.smithi094.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.58410.log.gz 2024-09-14T06:34:09.621 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.38591.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44672.log 2024-09-14T06:34:09.621 INFO:teuthology.orchestra.run.smithi094.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38591.log.gz 2024-09-14T06:34:09.621 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.59711.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73389.log 2024-09-14T06:34:09.621 INFO:teuthology.orchestra.run.smithi094.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.59711.log.gz 2024-09-14T06:34:09.621 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.44672.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71802.log 2024-09-14T06:34:09.621 INFO:teuthology.orchestra.run.smithi094.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44672.log.gz 2024-09-14T06:34:09.621 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.73389.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65186.log 2024-09-14T06:34:09.621 INFO:teuthology.orchestra.run.smithi094.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.73389.log.gz 2024-09-14T06:34:09.621 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.71802.log: 0.0%gzip -- replaced with /var/log/ceph/ceph-client.admin.71802.log.gz 2024-09-14T06:34:09.621 INFO:teuthology.orchestra.run.smithi094.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.72359.log 2024-09-14T06:34:09.621 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.65186.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68722.log 2024-09-14T06:34:09.621 INFO:teuthology.orchestra.run.smithi094.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.65186.log.gz 2024-09-14T06:34:09.621 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.72359.log: 0.0%gzip -- replaced with /var/log/ceph/ceph-client.admin.72359.log.gz 2024-09-14T06:34:09.621 INFO:teuthology.orchestra.run.smithi094.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.72927.log 2024-09-14T06:34:09.621 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.68722.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70570.log 2024-09-14T06:34:09.622 INFO:teuthology.orchestra.run.smithi094.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.68722.log.gz 2024-09-14T06:34:09.622 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.72927.log: gzip 0.0% -- replaced with /var/log/ceph/ceph-client.admin.72927.log.gz -5 2024-09-14T06:34:09.622 INFO:teuthology.orchestra.run.smithi094.stderr: --verbose -- /var/log/ceph/ceph-client.admin.42405.log 2024-09-14T06:34:09.622 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.70570.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46725.log 2024-09-14T06:34:09.622 INFO:teuthology.orchestra.run.smithi094.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.70570.log.gz 2024-09-14T06:34:09.622 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.42405.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42405.log.gz 2024-09-14T06:34:09.622 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49093.log 2024-09-14T06:34:09.622 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.46725.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59244.log 2024-09-14T06:34:09.622 INFO:teuthology.orchestra.run.smithi094.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.46725.log.gz 2024-09-14T06:34:09.622 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.49093.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49093.log.gz 2024-09-14T06:34:09.622 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57976.log 2024-09-14T06:34:09.622 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.59244.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41513.log 2024-09-14T06:34:09.622 INFO:teuthology.orchestra.run.smithi094.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.59244.log.gz 2024-09-14T06:34:09.622 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.57976.log: gzip -5 --verbose 0.0% -- -- replaced with /var/log/ceph/ceph-client.admin.57976.log.gz 2024-09-14T06:34:09.622 INFO:teuthology.orchestra.run.smithi094.stderr: /var/log/ceph/ceph-client.admin.71541.log 2024-09-14T06:34:09.623 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72500.log 2024-09-14T06:34:09.623 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.41513.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41513.log.gz 2024-09-14T06:34:09.623 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.71541.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.71541.log.gz 2024-09-14T06:34:09.623 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64730.log 2024-09-14T06:34:09.623 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.72500.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49860.log 2024-09-14T06:34:09.623 INFO:teuthology.orchestra.run.smithi094.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.72500.log.gz 2024-09-14T06:34:09.623 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.64730.log: gzip -5 --verbose -- 0.0% /var/log/ceph/ceph-client.admin.41293.log 2024-09-14T06:34:09.623 INFO:teuthology.orchestra.run.smithi094.stderr: -- replaced with /var/log/ceph/ceph-client.admin.64730.log.gz 2024-09-14T06:34:09.623 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.49860.log: gzip -5 --verbose -- 0.0% /var/log/ceph/ceph-client.admin.72461.log 2024-09-14T06:34:09.623 INFO:teuthology.orchestra.run.smithi094.stderr: -- replaced with /var/log/ceph/ceph-client.admin.49860.log.gz 2024-09-14T06:34:09.623 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.41293.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70696.log 2024-09-14T06:34:09.623 INFO:teuthology.orchestra.run.smithi094.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41293.log.gz 2024-09-14T06:34:09.623 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.72461.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45860.log 2024-09-14T06:34:09.624 INFO:teuthology.orchestra.run.smithi094.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.72461.log.gz 2024-09-14T06:34:09.624 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.70696.log: gzip 0.0% -5 --verbose -- replaced with /var/log/ceph/ceph-client.admin.70696.log.gz 2024-09-14T06:34:09.624 INFO:teuthology.orchestra.run.smithi094.stderr: -- /var/log/ceph/ceph-client.admin.73683.log 2024-09-14T06:34:09.624 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.45860.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47189.log 2024-09-14T06:34:09.624 INFO:teuthology.orchestra.run.smithi094.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.45860.log.gz 2024-09-14T06:34:09.624 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.73683.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71796.log 2024-09-14T06:34:09.624 INFO:teuthology.orchestra.run.smithi094.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.73683.log.gz 2024-09-14T06:34:09.624 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.47189.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72038.log 2024-09-14T06:34:09.624 INFO:teuthology.orchestra.run.smithi094.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.47189.log.gz 2024-09-14T06:34:09.624 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.71796.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66871.log 2024-09-14T06:34:09.624 INFO:teuthology.orchestra.run.smithi094.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.71796.log.gz 2024-09-14T06:34:09.624 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.72038.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54603.log 2024-09-14T06:34:09.624 INFO:teuthology.orchestra.run.smithi094.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.72038.log.gz 2024-09-14T06:34:09.625 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.66871.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71247.log 2024-09-14T06:34:09.625 INFO:teuthology.orchestra.run.smithi094.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.66871.log.gz 2024-09-14T06:34:09.625 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.54603.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72043.log 2024-09-14T06:34:09.625 INFO:teuthology.orchestra.run.smithi094.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.54603.log.gz 2024-09-14T06:34:09.625 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.71247.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64689.log 2024-09-14T06:34:09.625 INFO:teuthology.orchestra.run.smithi094.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.71247.log.gz 2024-09-14T06:34:09.625 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.72043.log: gzip 0.0% -5 --verbose -- replaced with /var/log/ceph/ceph-client.admin.72043.log.gz 2024-09-14T06:34:09.625 INFO:teuthology.orchestra.run.smithi094.stderr: -- /var/log/ceph/ceph-client.admin.62234.log 2024-09-14T06:34:09.625 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.64689.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41390.log 2024-09-14T06:34:09.625 INFO:teuthology.orchestra.run.smithi094.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.64689.log.gz 2024-09-14T06:34:09.625 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.62234.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.62234.log.gz 2024-09-14T06:34:09.625 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53479.log 2024-09-14T06:34:09.625 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.41390.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41135.log 2024-09-14T06:34:09.625 INFO:teuthology.orchestra.run.smithi094.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41390.log.gz 2024-09-14T06:34:09.625 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.53479.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52175.log 2024-09-14T06:34:09.626 INFO:teuthology.orchestra.run.smithi094.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.53479.log.gz 2024-09-14T06:34:09.626 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.41135.log: 0.0%gzip -- replaced with /var/log/ceph/ceph-client.admin.41135.log.gz 2024-09-14T06:34:09.626 INFO:teuthology.orchestra.run.smithi094.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.63558.log 2024-09-14T06:34:09.626 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.52175.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53536.log 2024-09-14T06:34:09.626 INFO:teuthology.orchestra.run.smithi094.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.52175.log.gz 2024-09-14T06:34:09.626 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.63558.log: gzip 0.0% -5 -- replaced with /var/log/ceph/ceph-client.admin.63558.log.gz 2024-09-14T06:34:09.626 INFO:teuthology.orchestra.run.smithi094.stderr: --verbose -- /var/log/ceph/ceph-client.admin.42919.log 2024-09-14T06:34:09.626 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.53536.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63660.log 2024-09-14T06:34:09.626 INFO:teuthology.orchestra.run.smithi094.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.53536.log.gz 2024-09-14T06:34:09.626 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.42919.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42919.log.gz 2024-09-14T06:34:09.626 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52215.log 2024-09-14T06:34:09.626 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.63660.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61218.log 2024-09-14T06:34:09.626 INFO:teuthology.orchestra.run.smithi094.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.63660.log.gz 2024-09-14T06:34:09.626 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.52215.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53117.log 2024-09-14T06:34:09.627 INFO:teuthology.orchestra.run.smithi094.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.52215.log.gz 2024-09-14T06:34:09.627 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.61218.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53438.log 2024-09-14T06:34:09.627 INFO:teuthology.orchestra.run.smithi094.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.61218.log.gz 2024-09-14T06:34:09.627 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.53117.log: gzip -5 --verbose -- 0.0% /var/log/ceph/ceph-client.admin.37255.log -- replaced with /var/log/ceph/ceph-client.admin.53117.log.gz 2024-09-14T06:34:09.627 INFO:teuthology.orchestra.run.smithi094.stderr: 2024-09-14T06:34:09.627 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.53438.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71800.log 2024-09-14T06:34:09.627 INFO:teuthology.orchestra.run.smithi094.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.53438.log.gz 2024-09-14T06:34:09.627 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.37255.log: gzip 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37255.log.gz -5 2024-09-14T06:34:09.627 INFO:teuthology.orchestra.run.smithi094.stderr: --verbose -- /var/log/ceph/ceph-client.admin.37331.log 2024-09-14T06:34:09.627 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.71800.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65127.log 2024-09-14T06:34:09.627 INFO:teuthology.orchestra.run.smithi094.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.71800.log.gz 2024-09-14T06:34:09.627 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.37331.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37331.log.gzgzip 2024-09-14T06:34:09.627 INFO:teuthology.orchestra.run.smithi094.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.38190.log 2024-09-14T06:34:09.627 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.65127.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38821.log 2024-09-14T06:34:09.627 INFO:teuthology.orchestra.run.smithi094.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.65127.log.gz 2024-09-14T06:34:09.627 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.38190.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38190.log.gz 2024-09-14T06:34:09.627 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46439.log 2024-09-14T06:34:09.628 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.38821.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66244.log 2024-09-14T06:34:09.628 INFO:teuthology.orchestra.run.smithi094.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38821.log.gz 2024-09-14T06:34:09.628 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.46439.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37280.log 2024-09-14T06:34:09.628 INFO:teuthology.orchestra.run.smithi094.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.46439.log.gz 2024-09-14T06:34:09.628 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.66244.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48031.log 2024-09-14T06:34:09.628 INFO:teuthology.orchestra.run.smithi094.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.66244.log.gz 2024-09-14T06:34:09.628 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.37280.log: gzip 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37280.log.gz -5 2024-09-14T06:34:09.628 INFO:teuthology.orchestra.run.smithi094.stderr: --verbose -- /var/log/ceph/ceph-client.admin.46591.log 2024-09-14T06:34:09.628 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.48031.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72456.log 2024-09-14T06:34:09.628 INFO:teuthology.orchestra.run.smithi094.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48031.log.gz 2024-09-14T06:34:09.628 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.46591.log: 0.0%gzip -- replaced with /var/log/ceph/ceph-client.admin.46591.log.gz 2024-09-14T06:34:09.628 INFO:teuthology.orchestra.run.smithi094.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.68068.log 2024-09-14T06:34:09.628 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.72456.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67861.log 2024-09-14T06:34:09.628 INFO:teuthology.orchestra.run.smithi094.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.72456.log.gz 2024-09-14T06:34:09.629 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.68068.log: gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2024-09-14T06:34:09.629 INFO:teuthology.orchestra.run.smithi094.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.68068.log.gz 2024-09-14T06:34:09.629 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.67861.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41637.log 2024-09-14T06:34:09.629 INFO:teuthology.orchestra.run.smithi094.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.67861.log.gz 2024-09-14T06:34:09.629 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41575.log 2024-09-14T06:34:09.629 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.41637.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40897.log 2024-09-14T06:34:09.629 INFO:teuthology.orchestra.run.smithi094.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41637.log.gz 2024-09-14T06:34:09.629 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.41575.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52420.log 2024-09-14T06:34:09.629 INFO:teuthology.orchestra.run.smithi094.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41575.log.gz 2024-09-14T06:34:09.629 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.40897.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40897.log.gz 2024-09-14T06:34:09.629 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37415.log 2024-09-14T06:34:09.629 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38717.log 2024-09-14T06:34:09.629 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.52420.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.52420.log.gz 2024-09-14T06:34:09.629 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.37415.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37415.log.gz 2024-09-14T06:34:09.629 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66937.log 2024-09-14T06:34:09.630 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47348.log 2024-09-14T06:34:09.630 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.38717.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38717.log.gz 2024-09-14T06:34:09.630 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.66937.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.66937.log.gz 2024-09-14T06:34:09.630 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54543.log 2024-09-14T06:34:09.630 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49092.log 2024-09-14T06:34:09.630 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.47348.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.47348.log.gz 2024-09-14T06:34:09.630 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.54543.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.54543.log.gz 2024-09-14T06:34:09.630 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51908.log 2024-09-14T06:34:09.630 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65803.log 2024-09-14T06:34:09.631 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.49092.log: 93.9% -- replaced with /var/log/ceph/ceph.audit.log.gz 2024-09-14T06:34:09.631 INFO:teuthology.orchestra.run.smithi094.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49092.log.gz 2024-09-14T06:34:09.631 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.51908.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.51908.log.gz 2024-09-14T06:34:09.631 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49314.log 2024-09-14T06:34:09.631 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.65803.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40399.log 2024-09-14T06:34:09.631 INFO:teuthology.orchestra.run.smithi094.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.65803.log.gz 2024-09-14T06:34:09.631 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.49314.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72339.log 2024-09-14T06:34:09.631 INFO:teuthology.orchestra.run.smithi094.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49314.log.gz 2024-09-14T06:34:09.631 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.40399.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40399.log.gz 2024-09-14T06:34:09.631 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66365.log 2024-09-14T06:34:09.631 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.72339.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43705.log 2024-09-14T06:34:09.631 INFO:teuthology.orchestra.run.smithi094.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.72339.log.gz 2024-09-14T06:34:09.632 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.66365.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.66365.log.gz 2024-09-14T06:34:09.632 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59863.log 2024-09-14T06:34:09.632 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.43705.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65524.log 2024-09-14T06:34:09.632 INFO:teuthology.orchestra.run.smithi094.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43705.log.gz 2024-09-14T06:34:09.632 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.59863.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68764.log 2024-09-14T06:34:09.632 INFO:teuthology.orchestra.run.smithi094.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.59863.log.gz 2024-09-14T06:34:09.632 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5/var/log/ceph/ceph-client.admin.65524.log: --verbose -- /var/log/ceph/ceph-client.admin.50142.log 2024-09-14T06:34:09.632 INFO:teuthology.orchestra.run.smithi094.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.65524.log.gz 2024-09-14T06:34:09.632 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.68764.log: gzip -5 0.0% --verbose -- -- replaced with /var/log/ceph/ceph-client.admin.68764.log.gz /var/log/ceph/ceph-client.admin.52640.log 2024-09-14T06:34:09.632 INFO:teuthology.orchestra.run.smithi094.stderr: 2024-09-14T06:34:09.632 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.50142.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.50142.log.gz 2024-09-14T06:34:09.632 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52917.log 2024-09-14T06:34:09.632 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.52640.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72142.log 2024-09-14T06:34:09.632 INFO:teuthology.orchestra.run.smithi094.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.52640.log.gz 2024-09-14T06:34:09.632 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.52917.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67923.log 2024-09-14T06:34:09.633 INFO:teuthology.orchestra.run.smithi094.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.52917.log.gz 2024-09-14T06:34:09.633 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.72142.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.72142.log.gz 2024-09-14T06:34:09.633 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57591.log 2024-09-14T06:34:09.633 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.67923.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43789.log 2024-09-14T06:34:09.633 INFO:teuthology.orchestra.run.smithi094.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.67923.log.gz 2024-09-14T06:34:09.633 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.57591.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56569.log 0.0% 2024-09-14T06:34:09.633 INFO:teuthology.orchestra.run.smithi094.stderr: -- replaced with /var/log/ceph/ceph-client.admin.57591.log.gz 2024-09-14T06:34:09.633 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.43789.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68004.log 2024-09-14T06:34:09.633 INFO:teuthology.orchestra.run.smithi094.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43789.log.gz 2024-09-14T06:34:09.633 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.56569.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.56569.log.gz 2024-09-14T06:34:09.633 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44371.log 2024-09-14T06:34:09.633 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.68004.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37632.log 2024-09-14T06:34:09.634 INFO:teuthology.orchestra.run.smithi094.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.68004.log.gz 2024-09-14T06:34:09.634 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.44371.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48449.log 2024-09-14T06:34:09.634 INFO:teuthology.orchestra.run.smithi094.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44371.log.gz 2024-09-14T06:34:09.634 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.37632.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67191.log 2024-09-14T06:34:09.634 INFO:teuthology.orchestra.run.smithi094.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37632.log.gz 2024-09-14T06:34:09.634 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.48449.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43579.log 2024-09-14T06:34:09.634 INFO:teuthology.orchestra.run.smithi094.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48449.log.gz 2024-09-14T06:34:09.634 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.67191.log: gzip -5 0.0% --verbose -- replaced with /var/log/ceph/ceph-client.admin.67191.log.gz -- 2024-09-14T06:34:09.634 INFO:teuthology.orchestra.run.smithi094.stderr: /var/log/ceph/ceph-client.admin.72042.log 2024-09-14T06:34:09.634 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.43579.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67359.log 2024-09-14T06:34:09.634 INFO:teuthology.orchestra.run.smithi094.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43579.log.gz 2024-09-14T06:34:09.634 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.72042.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51005.log 2024-09-14T06:34:09.634 INFO:teuthology.orchestra.run.smithi094.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.72042.log.gz 2024-09-14T06:34:09.634 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.67359.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48551.log 2024-09-14T06:34:09.635 INFO:teuthology.orchestra.run.smithi094.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.67359.log.gz 2024-09-14T06:34:09.635 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.51005.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72717.log 2024-09-14T06:34:09.635 INFO:teuthology.orchestra.run.smithi094.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.51005.log.gz 2024-09-14T06:34:09.635 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.48551.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56250.log 2024-09-14T06:34:09.635 INFO:teuthology.orchestra.run.smithi094.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48551.log.gz 2024-09-14T06:34:09.635 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.72717.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50268.log 2024-09-14T06:34:09.635 INFO:teuthology.orchestra.run.smithi094.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.72717.log.gz 2024-09-14T06:34:09.635 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.56250.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58948.log 2024-09-14T06:34:09.635 INFO:teuthology.orchestra.run.smithi094.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.56250.log.gz 2024-09-14T06:34:09.635 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.50268.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70780.log 2024-09-14T06:34:09.635 INFO:teuthology.orchestra.run.smithi094.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.50268.log.gz 2024-09-14T06:34:09.635 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.58948.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60202.log 2024-09-14T06:34:09.635 INFO:teuthology.orchestra.run.smithi094.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.58948.log.gz 2024-09-14T06:34:09.635 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.70780.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37974.log 2024-09-14T06:34:09.636 INFO:teuthology.orchestra.run.smithi094.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.70780.log.gz 2024-09-14T06:34:09.636 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.60202.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43141.log 2024-09-14T06:34:09.636 INFO:teuthology.orchestra.run.smithi094.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.60202.log.gz 2024-09-14T06:34:09.636 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.37974.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67699.log 2024-09-14T06:34:09.636 INFO:teuthology.orchestra.run.smithi094.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37974.log.gz 2024-09-14T06:34:09.636 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.43141.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73347.log 2024-09-14T06:34:09.636 INFO:teuthology.orchestra.run.smithi094.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43141.log.gz 2024-09-14T06:34:09.636 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.67699.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71331.log 2024-09-14T06:34:09.636 INFO:teuthology.orchestra.run.smithi094.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.67699.log.gz 2024-09-14T06:34:09.636 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.73347.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55331.log 2024-09-14T06:34:09.636 INFO:teuthology.orchestra.run.smithi094.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.73347.log.gz 2024-09-14T06:34:09.636 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.71331.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37634.log 2024-09-14T06:34:09.636 INFO:teuthology.orchestra.run.smithi094.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.71331.log.gz 2024-09-14T06:34:09.636 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.55331.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57652.log 2024-09-14T06:34:09.637 INFO:teuthology.orchestra.run.smithi094.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.55331.log.gz 2024-09-14T06:34:09.637 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.37634.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67597.log 2024-09-14T06:34:09.637 INFO:teuthology.orchestra.run.smithi094.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37634.log.gz 2024-09-14T06:34:09.637 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.57652.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65344.log 2024-09-14T06:34:09.637 INFO:teuthology.orchestra.run.smithi094.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.57652.log.gz 2024-09-14T06:34:09.637 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.67597.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49476.log 2024-09-14T06:34:09.637 INFO:teuthology.orchestra.run.smithi094.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.67597.log.gz 2024-09-14T06:34:09.637 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.65344.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63137.log 2024-09-14T06:34:09.637 INFO:teuthology.orchestra.run.smithi094.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.65344.log.gz 2024-09-14T06:34:09.637 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.49476.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73095.log 2024-09-14T06:34:09.637 INFO:teuthology.orchestra.run.smithi094.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49476.log.gz 2024-09-14T06:34:09.637 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.63137.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52998.log 2024-09-14T06:34:09.638 INFO:teuthology.orchestra.run.smithi094.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.63137.log.gz 2024-09-14T06:34:09.638 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.73095.log: gzip -5 0.0% --verbose -- -- replaced with /var/log/ceph/ceph-client.admin.73095.log.gz 2024-09-14T06:34:09.638 INFO:teuthology.orchestra.run.smithi094.stderr: /var/log/ceph/ceph-client.admin.37638.log 2024-09-14T06:34:09.638 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.52998.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66184.log 2024-09-14T06:34:09.638 INFO:teuthology.orchestra.run.smithi094.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.52998.log.gz 2024-09-14T06:34:09.638 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.37638.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54401.log 2024-09-14T06:34:09.638 INFO:teuthology.orchestra.run.smithi094.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37638.log.gz 2024-09-14T06:34:09.638 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.66184.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41450.log 2024-09-14T06:34:09.638 INFO:teuthology.orchestra.run.smithi094.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.66184.log.gz 2024-09-14T06:34:09.638 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.54401.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63793.log 2024-09-14T06:34:09.638 INFO:teuthology.orchestra.run.smithi094.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.54401.log.gz 2024-09-14T06:34:09.638 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.41450.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53396.log 2024-09-14T06:34:09.638 INFO:teuthology.orchestra.run.smithi094.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41450.log.gz 2024-09-14T06:34:09.638 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.63793.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58804.log 2024-09-14T06:34:09.638 INFO:teuthology.orchestra.run.smithi094.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.63793.log.gz 2024-09-14T06:34:09.639 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.53396.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57348.log 2024-09-14T06:34:09.639 INFO:teuthology.orchestra.run.smithi094.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.53396.log.gz 2024-09-14T06:34:09.639 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.58804.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51389.log 2024-09-14T06:34:09.639 INFO:teuthology.orchestra.run.smithi094.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.58804.log.gz 2024-09-14T06:34:09.639 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.57348.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70738.log 2024-09-14T06:34:09.639 INFO:teuthology.orchestra.run.smithi094.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.57348.log.gz 2024-09-14T06:34:09.639 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.51389.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52054.log 2024-09-14T06:34:09.639 INFO:teuthology.orchestra.run.smithi094.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.51389.log.gz 2024-09-14T06:34:09.639 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.70738.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47752.log 2024-09-14T06:34:09.639 INFO:teuthology.orchestra.run.smithi094.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.70738.log.gz 2024-09-14T06:34:09.639 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.52054.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46158.log 2024-09-14T06:34:09.639 INFO:teuthology.orchestra.run.smithi094.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.52054.log.gz 2024-09-14T06:34:09.639 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.47752.log: gzip 0.0% -5 -- replaced with /var/log/ceph/ceph-client.admin.47752.log.gz 2024-09-14T06:34:09.639 INFO:teuthology.orchestra.run.smithi094.stderr: --verbose -- /var/log/ceph/ceph-client.admin.66726.log 2024-09-14T06:34:09.640 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.46158.log: gzip -5 --verbose -- /var/log/ceph/ceph-mon.c.log 2024-09-14T06:34:09.640 INFO:teuthology.orchestra.run.smithi094.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.46158.log.gz 2024-09-14T06:34:09.640 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.66726.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63459.log 2024-09-14T06:34:09.640 INFO:teuthology.orchestra.run.smithi094.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.66726.log.gz 2024-09-14T06:34:09.640 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-mon.c.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59626.log 2024-09-14T06:34:09.640 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.63459.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.63459.log.gz 2024-09-14T06:34:09.640 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57930.log 2024-09-14T06:34:09.640 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56947.log 2024-09-14T06:34:09.640 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.59626.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.59626.log.gz 2024-09-14T06:34:09.640 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.57930.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43621.log 2024-09-14T06:34:09.640 INFO:teuthology.orchestra.run.smithi094.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.57930.log.gz 2024-09-14T06:34:09.640 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.56947.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.56947.log.gz 2024-09-14T06:34:09.640 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38961.log 2024-09-14T06:34:09.640 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66790.log 2024-09-14T06:34:09.640 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.43621.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43621.log.gz 2024-09-14T06:34:09.641 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.38961.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38961.log.gz 2024-09-14T06:34:09.641 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62932.log 2024-09-14T06:34:09.641 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37199.log 2024-09-14T06:34:09.641 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.66790.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.66790.log.gz 2024-09-14T06:34:09.641 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.62932.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70911.log 2024-09-14T06:34:09.641 INFO:teuthology.orchestra.run.smithi094.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.62932.log.gz 2024-09-14T06:34:09.641 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.37199.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70486.log 2024-09-14T06:34:09.641 INFO:teuthology.orchestra.run.smithi094.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37199.log.gz 2024-09-14T06:34:09.641 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.70911.log: gzip -5 --verbose -- 0.0% /var/log/ceph/ceph-client.admin.44671.log 2024-09-14T06:34:09.641 INFO:teuthology.orchestra.run.smithi094.stderr: -- replaced with /var/log/ceph/ceph-client.admin.70911.log.gz 2024-09-14T06:34:09.641 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70141.log 2024-09-14T06:34:09.641 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.70486.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.70486.log.gz 2024-09-14T06:34:09.641 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.44671.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44671.log.gz 2024-09-14T06:34:09.641 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68603.log 2024-09-14T06:34:09.642 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55430.log 2024-09-14T06:34:09.642 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.70141.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.70141.log.gz 2024-09-14T06:34:09.642 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.68603.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72759.log 2024-09-14T06:34:09.642 INFO:teuthology.orchestra.run.smithi094.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.68603.log.gz 2024-09-14T06:34:09.642 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.55430.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.55430.log.gz 2024-09-14T06:34:09.642 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37110.log 2024-09-14T06:34:09.642 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54331.log 2024-09-14T06:34:09.642 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.72759.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.72759.log.gz 2024-09-14T06:34:09.642 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.37110.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44775.log 2024-09-14T06:34:09.642 INFO:teuthology.orchestra.run.smithi094.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37110.log.gz 2024-09-14T06:34:09.642 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.54331.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.54331.log.gz 2024-09-14T06:34:09.642 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44212.log 2024-09-14T06:34:09.642 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43913.log 2024-09-14T06:34:09.642 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.44775.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44775.log.gz 2024-09-14T06:34:09.643 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.44212.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44212.log.gz 2024-09-14T06:34:09.643 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64440.log 2024-09-14T06:34:09.643 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67358.log 2024-09-14T06:34:09.643 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.43913.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43913.log.gz 2024-09-14T06:34:09.643 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.64440.log: 0.0%gzip -- replaced with /var/log/ceph/ceph-client.admin.64440.log.gz 2024-09-14T06:34:09.643 INFO:teuthology.orchestra.run.smithi094.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.65544.log 2024-09-14T06:34:09.643 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47247.log 2024-09-14T06:34:09.643 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.67358.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.67358.log.gz 2024-09-14T06:34:09.643 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.65544.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43198.log 2024-09-14T06:34:09.643 INFO:teuthology.orchestra.run.smithi094.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.65544.log.gz 2024-09-14T06:34:09.643 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.47247.log: gzip 0.0% -5 -- replaced with /var/log/ceph/ceph-client.admin.47247.log.gz 2024-09-14T06:34:09.643 INFO:teuthology.orchestra.run.smithi094.stderr: --verbose -- /var/log/ceph/ceph-client.admin.53759.log 2024-09-14T06:34:09.643 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47148.log 2024-09-14T06:34:09.643 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.43198.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43198.log.gz 2024-09-14T06:34:09.643 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.53759.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.53759.log.gz 2024-09-14T06:34:09.644 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39761.log 2024-09-14T06:34:09.644 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43537.log 2024-09-14T06:34:09.644 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.47148.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.47148.log.gz 2024-09-14T06:34:09.644 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.39761.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39761.log.gz 2024-09-14T06:34:09.644 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38633.log 2024-09-14T06:34:09.644 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37627.log 2024-09-14T06:34:09.644 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.43537.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43537.log.gz 2024-09-14T06:34:09.644 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.38633.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38633.log.gz 2024-09-14T06:34:09.644 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72885.log 2024-09-14T06:34:09.644 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70822.log 2024-09-14T06:34:09.644 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.37627.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37627.log.gz 2024-09-14T06:34:09.644 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.72885.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46506.log 2024-09-14T06:34:09.644 INFO:teuthology.orchestra.run.smithi094.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.72885.log.gz 2024-09-14T06:34:09.644 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.70822.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.70822.log.gz 2024-09-14T06:34:09.645 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59783.log 2024-09-14T06:34:09.645 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45057.log 2024-09-14T06:34:09.645 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.46506.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.46506.log.gz 2024-09-14T06:34:09.645 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.59783.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.59783.log.gz 2024-09-14T06:34:09.645 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65020.log 2024-09-14T06:34:09.645 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64266.log 2024-09-14T06:34:09.645 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.45057.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.45057.log.gz 2024-09-14T06:34:09.645 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.65020.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.65020.log.gz 2024-09-14T06:34:09.645 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50527.log 2024-09-14T06:34:09.645 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5/var/log/ceph/ceph-client.admin.64266.log: --verbose -- /var/log/ceph/ceph-client.admin.45720.log 2024-09-14T06:34:09.645 INFO:teuthology.orchestra.run.smithi094.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.64266.log.gz 2024-09-14T06:34:09.645 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.50527.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.50527.log.gz 2024-09-14T06:34:09.645 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43873.log 2024-09-14T06:34:09.645 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38231.log 2024-09-14T06:34:09.646 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.45720.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.45720.log.gz 2024-09-14T06:34:09.646 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.43873.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57246.log 2024-09-14T06:34:09.646 INFO:teuthology.orchestra.run.smithi094.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43873.log.gz 2024-09-14T06:34:09.646 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.38231.log: gzip -5 --verbose -- 0.0% /var/log/ceph/ceph-client.admin.59546.log 2024-09-14T06:34:09.646 INFO:teuthology.orchestra.run.smithi094.stderr: -- replaced with /var/log/ceph/ceph-client.admin.38231.log.gz 2024-09-14T06:34:09.646 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49595.log 2024-09-14T06:34:09.646 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.57246.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.57246.log.gz 2024-09-14T06:34:09.646 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.59546.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.59546.log.gz 2024-09-14T06:34:09.646 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37630.log 2024-09-14T06:34:09.646 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38959.log 2024-09-14T06:34:09.646 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.49595.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49595.log.gz 2024-09-14T06:34:09.646 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.37630.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62673.log 2024-09-14T06:34:09.646 INFO:teuthology.orchestra.run.smithi094.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37630.log.gz 2024-09-14T06:34:09.646 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.38959.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38959.log.gz 2024-09-14T06:34:09.646 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40538.log 2024-09-14T06:34:09.647 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mon.a.log 2024-09-14T06:34:09.647 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.62673.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.62673.log.gz 2024-09-14T06:34:09.647 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.40538.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40538.log.gz 2024-09-14T06:34:09.647 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66547.log 2024-09-14T06:34:09.647 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49537.log 2024-09-14T06:34:09.647 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-mon.a.log: /var/log/ceph/ceph-client.admin.66547.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.66547.log.gz 2024-09-14T06:34:09.647 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50227.log 2024-09-14T06:34:09.647 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56292.log 2024-09-14T06:34:09.647 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.49537.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49537.log.gz 2024-09-14T06:34:09.647 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.50227.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62493.log 2024-09-14T06:34:09.647 INFO:teuthology.orchestra.run.smithi094.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.50227.log.gz 2024-09-14T06:34:09.647 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.56292.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.56292.log.gz 2024-09-14T06:34:09.647 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38960.log 2024-09-14T06:34:09.648 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5/var/log/ceph/ceph-client.admin.62493.log: --verbose -- /var/log/ceph/ceph-client.admin.58210.log 2024-09-14T06:34:09.648 INFO:teuthology.orchestra.run.smithi094.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.62493.log.gz 2024-09-14T06:34:09.648 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.38960.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37639.log 2024-09-14T06:34:09.648 INFO:teuthology.orchestra.run.smithi094.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38960.log.gz 2024-09-14T06:34:09.648 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.58210.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54259.log 2024-09-14T06:34:09.648 INFO:teuthology.orchestra.run.smithi094.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.58210.log.gz 2024-09-14T06:34:09.648 INFO:teuthology.orchestra.run.smithi094.stderr:gzip/var/log/ceph/ceph-client.admin.37639.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.59486.log 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37639.log.gz 2024-09-14T06:34:09.648 INFO:teuthology.orchestra.run.smithi094.stderr: 2024-09-14T06:34:09.648 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.54259.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.54259.log.gz 2024-09-14T06:34:09.648 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45256.log 2024-09-14T06:34:09.648 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.59486.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67131.log 2024-09-14T06:34:09.648 INFO:teuthology.orchestra.run.smithi094.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.59486.log.gz 2024-09-14T06:34:09.648 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.45256.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.45256.log.gz 2024-09-14T06:34:09.648 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38675.log 2024-09-14T06:34:09.649 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52129.log 2024-09-14T06:34:09.649 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.67131.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.67131.log.gz 2024-09-14T06:34:09.649 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.38675.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63195.log 2024-09-14T06:34:09.649 INFO:teuthology.orchestra.run.smithi094.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38675.log.gz 2024-09-14T06:34:09.649 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.52129.log: gzip -5 0.0% --verbose -- -- replaced with /var/log/ceph/ceph-client.admin.52129.log.gz 2024-09-14T06:34:09.649 INFO:teuthology.orchestra.run.smithi094.stderr: /var/log/ceph/ceph-client.admin.45334.log 2024-09-14T06:34:09.649 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.2.log 2024-09-14T06:34:09.649 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.63195.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.63195.log.gz 2024-09-14T06:34:09.649 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.45334.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50823.log 2024-09-14T06:34:09.649 INFO:teuthology.orchestra.run.smithi094.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.45334.log.gz 2024-09-14T06:34:09.649 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-osd.2.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68680.log 2024-09-14T06:34:09.649 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.50823.log: gzip -5 --verbose 0.0% -- replaced with /var/log/ceph/ceph-client.admin.50823.log.gz 2024-09-14T06:34:09.649 INFO:teuthology.orchestra.run.smithi094.stderr: -- /var/log/ceph/ceph-client.admin.61956.log 2024-09-14T06:34:09.649 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.68680.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.68680.log.gz 2024-09-14T06:34:09.650 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65404.log 2024-09-14T06:34:09.650 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.61956.log: gzip -5 0.0% -- replaced with /var/log/ceph/ceph-client.admin.61956.log.gz 2024-09-14T06:34:09.650 INFO:teuthology.orchestra.run.smithi094.stderr: --verbose -- /var/log/ceph/ceph-client.admin.72460.log 2024-09-14T06:34:09.650 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.65404.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.65404.log.gz 2024-09-14T06:34:09.650 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38549.log 2024-09-14T06:34:09.650 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.72460.log: gzip -5 --verbose 0.0% -- replaced with /var/log/ceph/ceph-client.admin.72460.log.gz 2024-09-14T06:34:09.650 INFO:teuthology.orchestra.run.smithi094.stderr: -- /var/log/ceph/ceph-client.admin.73599.log 2024-09-14T06:34:09.650 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.38549.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38549.log.gz 2024-09-14T06:34:09.650 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51568.log 2024-09-14T06:34:09.650 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.73599.log: gzip -5 0.0% -- replaced with /var/log/ceph/ceph-client.admin.73599.log.gz 2024-09-14T06:34:09.650 INFO:teuthology.orchestra.run.smithi094.stderr: --verbose -- /var/log/ceph/ceph-client.admin.73053.log 2024-09-14T06:34:09.650 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.51568.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.51568.log.gz 2024-09-14T06:34:09.650 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37972.log 2024-09-14T06:34:09.650 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.73053.log: gzip -5 --verbose -- 0.0% /var/log/ceph/ceph-client.admin.54866.log 2024-09-14T06:34:09.650 INFO:teuthology.orchestra.run.smithi094.stderr: -- replaced with /var/log/ceph/ceph-client.admin.73053.log.gz 2024-09-14T06:34:09.650 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.37972.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37972.log.gz 2024-09-14T06:34:09.651 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65746.log 2024-09-14T06:34:09.651 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose/var/log/ceph/ceph-client.admin.54866.log: -- /var/log/ceph/ceph-client.admin.69145.log 2024-09-14T06:34:09.651 INFO:teuthology.orchestra.run.smithi094.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.54866.log.gz 2024-09-14T06:34:09.651 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose/var/log/ceph/ceph-client.admin.65746.log: -- /var/log/ceph/ceph-client.admin.71799.log 2024-09-14T06:34:09.651 INFO:teuthology.orchestra.run.smithi094.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.65746.log.gz 2024-09-14T06:34:09.651 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.69145.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.69145.log.gz 2024-09-14T06:34:09.651 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51992.log 2024-09-14T06:34:09.651 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose/var/log/ceph/ceph-client.admin.71799.log: -- /var/log/ceph/ceph-client.admin.55147.log 2024-09-14T06:34:09.651 INFO:teuthology.orchestra.run.smithi094.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.71799.log.gz 2024-09-14T06:34:09.651 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.51992.log: 0.0%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67777.log 2024-09-14T06:34:09.651 INFO:teuthology.orchestra.run.smithi094.stderr: -- replaced with /var/log/ceph/ceph-client.admin.51992.log.gz 2024-09-14T06:34:09.651 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71205.log 2024-09-14T06:34:09.651 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.55147.log: /var/log/ceph/ceph-client.admin.67777.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73765.log 2024-09-14T06:34:09.651 INFO:teuthology.orchestra.run.smithi094.stderr: 0.0% 0.0% -- replaced with /var/log/ceph/ceph-client.admin.67777.log.gz 2024-09-14T06:34:09.651 INFO:teuthology.orchestra.run.smithi094.stderr: -- replaced with /var/log/ceph/ceph-client.admin.55147.log.gz 2024-09-14T06:34:09.652 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.71205.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.71205.log.gz 2024-09-14T06:34:09.652 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64209.log 2024-09-14T06:34:09.652 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.73765.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.73765.log.gz 2024-09-14T06:34:09.652 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mgr.y.log 2024-09-14T06:34:09.652 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.64209.log: gzip 0.0% -- replaced with /var/log/ceph/ceph-client.admin.64209.log.gz 2024-09-14T06:34:09.652 INFO:teuthology.orchestra.run.smithi094.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.71373.log 2024-09-14T06:34:09.652 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-mgr.y.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51208.log 2024-09-14T06:34:09.652 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.71373.log: 0.0%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61818.log 2024-09-14T06:34:09.652 INFO:teuthology.orchestra.run.smithi094.stderr: -- replaced with /var/log/ceph/ceph-client.admin.71373.log.gz 2024-09-14T06:34:09.652 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71805.log 2024-09-14T06:34:09.652 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.51208.log: /var/log/ceph/ceph-client.admin.61818.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.51208.log.gz 2024-09-14T06:34:09.652 INFO:teuthology.orchestra.run.smithi094.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.61818.log.gz 2024-09-14T06:34:09.652 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38191.log 2024-09-14T06:34:09.652 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.71805.log: 0.0%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73137.log 2024-09-14T06:34:09.652 INFO:teuthology.orchestra.run.smithi094.stderr: -- replaced with /var/log/ceph/ceph-client.admin.71805.log.gz 2024-09-14T06:34:09.653 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60301.log 2024-09-14T06:34:09.653 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.73137.log: /var/log/ceph/ceph-client.admin.38191.log: 0.0% 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38191.log.gz 2024-09-14T06:34:09.653 INFO:teuthology.orchestra.run.smithi094.stderr: -- replaced with /var/log/ceph/ceph-client.admin.73137.log.gz 2024-09-14T06:34:09.653 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40496.log 2024-09-14T06:34:09.653 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71415.log 2024-09-14T06:34:09.653 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.60301.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.60301.log.gz 2024-09-14T06:34:09.653 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.40496.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40496.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47532.log 2024-09-14T06:34:09.653 INFO:teuthology.orchestra.run.smithi094.stderr: 2024-09-14T06:34:09.653 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.log 2024-09-14T06:34:09.653 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.71415.log: /var/log/ceph/ceph-client.admin.47532.log: 0.0% 0.0% -- replaced with /var/log/ceph/ceph-client.admin.71415.log.gz 2024-09-14T06:34:09.653 INFO:teuthology.orchestra.run.smithi094.stderr: -- replaced with /var/log/ceph/ceph-client.admin.47532.log.gz 2024-09-14T06:34:09.654 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64047.log 2024-09-14T06:34:09.654 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54965.log 2024-09-14T06:34:09.654 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph.log: /var/log/ceph/ceph-client.admin.64047.log: 93.6% -- replaced with /var/log/ceph/ceph-mgr.y.log.gz 2024-09-14T06:34:09.654 INFO:teuthology.orchestra.run.smithi094.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.64047.log.gz 2024-09-14T06:34:09.654 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73305.log 2024-09-14T06:34:09.654 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.54965.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.54965.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38507.log 2024-09-14T06:34:09.654 INFO:teuthology.orchestra.run.smithi094.stderr: 2024-09-14T06:34:09.654 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73011.log 2024-09-14T06:34:09.654 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.73305.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.73305.log.gz 2024-09-14T06:34:09.654 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.38507.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38507.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49958.log 2024-09-14T06:34:09.654 INFO:teuthology.orchestra.run.smithi094.stderr: 2024-09-14T06:34:09.655 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.73011.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.73011.log.gz 2024-09-14T06:34:09.655 INFO:teuthology.orchestra.run.smithi094.stderr: 91.7% -- replaced with /var/log/ceph/ceph.log.gz 2024-09-14T06:34:09.655 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41968.log 2024-09-14T06:34:09.655 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48231.log 2024-09-14T06:34:09.655 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.49958.log: /var/log/ceph/ceph-client.admin.41968.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64527.log 2024-09-14T06:34:09.655 INFO:teuthology.orchestra.run.smithi094.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49958.log.gz 2024-09-14T06:34:09.655 INFO:teuthology.orchestra.run.smithi094.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41968.log.gz 2024-09-14T06:34:09.655 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.48231.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38759.log 2024-09-14T06:34:09.655 INFO:teuthology.orchestra.run.smithi094.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48231.log.gz 2024-09-14T06:34:09.655 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45558.log 2024-09-14T06:34:09.655 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.64527.log: /var/log/ceph/ceph-client.admin.38759.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63842.log 2024-09-14T06:34:09.655 INFO:teuthology.orchestra.run.smithi094.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.64527.log.gz 2024-09-14T06:34:09.655 INFO:teuthology.orchestra.run.smithi094.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38759.log.gz 2024-09-14T06:34:09.656 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.45558.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.45558.log.gz 2024-09-14T06:34:09.656 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40140.log 2024-09-14T06:34:09.656 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63096.log 2024-09-14T06:34:09.656 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.63842.log: /var/log/ceph/ceph-client.admin.40140.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48897.log 2024-09-14T06:34:09.656 INFO:teuthology.orchestra.run.smithi094.stderr: 0.0% 0.0% -- replaced with /var/log/ceph/ceph-client.admin.63842.log.gz 2024-09-14T06:34:09.656 INFO:teuthology.orchestra.run.smithi094.stderr: -- replaced with /var/log/ceph/ceph-client.admin.40140.log.gz 2024-09-14T06:34:09.656 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.63096.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.63096.log.gz 2024-09-14T06:34:09.656 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44269.log 2024-09-14T06:34:09.656 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57432.log 2024-09-14T06:34:09.656 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.48897.log: /var/log/ceph/ceph-client.admin.44269.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68866.log 2024-09-14T06:34:09.656 INFO:teuthology.orchestra.run.smithi094.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48897.log.gz 2024-09-14T06:34:09.656 INFO:teuthology.orchestra.run.smithi094.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44269.log.gz 2024-09-14T06:34:09.656 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.57432.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62590.log 2024-09-14T06:34:09.657 INFO:teuthology.orchestra.run.smithi094.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.57432.log.gz 2024-09-14T06:34:09.657 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66142.log 2024-09-14T06:34:09.657 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.68866.log: 0.0%/var/log/ceph/ceph-client.admin.62590.log: -- replaced with /var/log/ceph/ceph-client.admin.68866.log.gzgzip 2024-09-14T06:34:09.657 INFO:teuthology.orchestra.run.smithi094.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.49719.log 2024-09-14T06:34:09.657 INFO:teuthology.orchestra.run.smithi094.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.62590.log.gz 2024-09-14T06:34:09.657 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47732.log 2024-09-14T06:34:09.657 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.66142.log: gzip/var/log/ceph/ceph-client.admin.49719.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.52598.log 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49719.log.gz 2024-09-14T06:34:09.657 INFO:teuthology.orchestra.run.smithi094.stderr: 2024-09-14T06:34:09.657 INFO:teuthology.orchestra.run.smithi094.stderr: 0.0%/var/log/ceph/ceph-client.admin.47732.log: -- replaced with /var/log/ceph/ceph-client.admin.66142.log.gz 2024-09-14T06:34:09.657 INFO:teuthology.orchestra.run.smithi094.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.47732.log.gz 2024-09-14T06:34:09.657 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63944.log 2024-09-14T06:34:09.657 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.52598.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.52598.log.gz 2024-09-14T06:34:09.657 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42187.log 2024-09-14T06:34:09.657 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.63944.log: gzip -5 0.0% -- replaced with /var/log/ceph/ceph-client.admin.63944.log.gz 2024-09-14T06:34:09.658 INFO:teuthology.orchestra.run.smithi094.stderr: --verbose -- /var/log/ceph/ceph-client.admin.68341.log 2024-09-14T06:34:09.658 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.42187.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42187.log.gz 2024-09-14T06:34:09.658 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71457.log 2024-09-14T06:34:09.658 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.68341.log: gzip -5 0.0% --verbose -- /var/log/ceph/ceph-client.admin.40100.log 2024-09-14T06:34:09.658 INFO:teuthology.orchestra.run.smithi094.stderr: -- replaced with /var/log/ceph/ceph-client.admin.68341.log.gz 2024-09-14T06:34:09.658 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.71457.log: gzip -5 0.0% --verbose -- -- replaced with /var/log/ceph/ceph-client.admin.71457.log.gz /var/log/ceph/ceph-client.admin.73221.log 2024-09-14T06:34:09.658 INFO:teuthology.orchestra.run.smithi094.stderr: 2024-09-14T06:34:09.658 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41885.log/var/log/ceph/ceph-client.admin.40100.log: 2024-09-14T06:34:09.658 INFO:teuthology.orchestra.run.smithi094.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40100.log.gz/var/log/ceph/ceph-client.admin.73221.log: 2024-09-14T06:34:09.658 INFO:teuthology.orchestra.run.smithi094.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.73221.log.gz 2024-09-14T06:34:09.658 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52032.log 2024-09-14T06:34:09.658 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.41885.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41885.log.gz 2024-09-14T06:34:09.659 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59187.log 2024-09-14T06:34:09.659 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.52032.log: gzip -5 0.0% -- replaced with /var/log/ceph/ceph-client.admin.52032.log.gz 2024-09-14T06:34:09.659 INFO:teuthology.orchestra.run.smithi094.stderr: --verbose -- /var/log/ceph/ceph-client.admin.43339.log 2024-09-14T06:34:09.659 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.59187.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.59187.log.gz 2024-09-14T06:34:09.659 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54741.log 2024-09-14T06:34:09.659 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.43339.log: gzip -5 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43339.log.gz 2024-09-14T06:34:09.659 INFO:teuthology.orchestra.run.smithi094.stderr: --verbose -- /var/log/ceph/ceph-client.admin.63317.log 2024-09-14T06:34:09.659 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.54741.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.54741.log.gz 2024-09-14T06:34:09.659 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61437.log 2024-09-14T06:34:09.659 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.63317.log: gzip -5 0.0% -- replaced with /var/log/ceph/ceph-client.admin.63317.log.gz 2024-09-14T06:34:09.659 INFO:teuthology.orchestra.run.smithi094.stderr: --verbose -- /var/log/ceph/ceph-client.admin.48128.log 2024-09-14T06:34:09.659 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.61437.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.61437.log.gz 2024-09-14T06:34:09.659 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68804.log 2024-09-14T06:34:09.659 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.48128.log: gzip -5 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48128.log.gz 2024-09-14T06:34:09.660 INFO:teuthology.orchestra.run.smithi094.stderr: --verbose -- /var/log/ceph/ceph-client.admin.73557.log 2024-09-14T06:34:09.660 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.68804.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.68804.log.gz 2024-09-14T06:34:09.660 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38230.log 2024-09-14T06:34:09.660 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.73557.log: gzip -5 0.0% -- replaced with /var/log/ceph/ceph-client.admin.73557.log.gz 2024-09-14T06:34:09.660 INFO:teuthology.orchestra.run.smithi094.stderr: --verbose -- /var/log/ceph/ceph-client.admin.72143.log 2024-09-14T06:34:09.660 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.38230.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38230.log.gz 2024-09-14T06:34:09.660 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73725.log 2024-09-14T06:34:09.660 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.72143.log: gzip -5 0.0% -- replaced with /var/log/ceph/ceph-client.admin.72143.log.gz 2024-09-14T06:34:09.660 INFO:teuthology.orchestra.run.smithi094.stderr: --verbose -- /var/log/ceph/ceph-client.admin.59823.log 2024-09-14T06:34:09.660 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.73725.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.73725.log.gz 2024-09-14T06:34:09.660 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71037.log 2024-09-14T06:34:09.660 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.59823.log: gzip -5 0.0% --verbose -- /var/log/ceph/ceph-client.admin.51108.log 2024-09-14T06:34:09.660 INFO:teuthology.orchestra.run.smithi094.stderr: -- replaced with /var/log/ceph/ceph-client.admin.59823.log.gz 2024-09-14T06:34:09.661 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.71037.log: gzip -5 --verbose 0.0% -- -- replaced with /var/log/ceph/ceph-client.admin.71037.log.gz /var/log/ceph/ceph-client.admin.54907.log 2024-09-14T06:34:09.661 INFO:teuthology.orchestra.run.smithi094.stderr: 2024-09-14T06:34:09.661 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose/var/log/ceph/ceph-client.admin.51108.log: -- /var/log/ceph/ceph-client.admin.67296.log 2024-09-14T06:34:09.661 INFO:teuthology.orchestra.run.smithi094.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.51108.log.gz 2024-09-14T06:34:09.661 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.54907.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.54907.log.gz 2024-09-14T06:34:09.661 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50607.log 2024-09-14T06:34:09.661 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.67296.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.67296.log.gz 2024-09-14T06:34:09.661 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70241.log 2024-09-14T06:34:09.661 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.50607.log: 0.0%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55472.log 2024-09-14T06:34:09.661 INFO:teuthology.orchestra.run.smithi094.stderr: -- replaced with /var/log/ceph/ceph-client.admin.50607.log.gz 2024-09-14T06:34:09.661 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.70241.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41054.log 2024-09-14T06:34:09.661 INFO:teuthology.orchestra.run.smithi094.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.70241.log.gz 2024-09-14T06:34:09.662 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose/var/log/ceph/ceph-client.admin.55472.log: -- /var/log/ceph/ceph-client.admin.60242.log 2024-09-14T06:34:09.662 INFO:teuthology.orchestra.run.smithi094.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.55472.log.gz 2024-09-14T06:34:09.662 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.41054.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41054.log.gz 2024-09-14T06:34:09.662 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48708.log 2024-09-14T06:34:09.662 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54330.log 2024-09-14T06:34:09.662 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.60242.log: /var/log/ceph/ceph-client.admin.48708.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42025.log 2024-09-14T06:34:09.662 INFO:teuthology.orchestra.run.smithi094.stderr: 0.0% 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48708.log.gz 2024-09-14T06:34:09.662 INFO:teuthology.orchestra.run.smithi094.stderr: -- replaced with /var/log/ceph/ceph-client.admin.60242.log.gz 2024-09-14T06:34:09.662 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.54330.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.54330.log.gz 2024-09-14T06:34:09.662 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56670.log 2024-09-14T06:34:09.662 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.42025.log: 0.0%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51047.log 2024-09-14T06:34:09.662 INFO:teuthology.orchestra.run.smithi094.stderr: -- replaced with /var/log/ceph/ceph-client.admin.42025.log.gz 2024-09-14T06:34:09.662 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.56670.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70864.log 2024-09-14T06:34:09.662 INFO:teuthology.orchestra.run.smithi094.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.56670.log.gz 2024-09-14T06:34:09.663 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42663.log 2024-09-14T06:34:09.663 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.51047.log: gzip/var/log/ceph/ceph-client.admin.70864.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.69561.log 0.0% -- replaced with /var/log/ceph/ceph-client.admin.70864.log.gz 2024-09-14T06:34:09.663 INFO:teuthology.orchestra.run.smithi094.stderr: 2024-09-14T06:34:09.663 INFO:teuthology.orchestra.run.smithi094.stderr: 0.0%/var/log/ceph/ceph-client.admin.42663.log: -- replaced with /var/log/ceph/ceph-client.admin.51047.log.gz 2024-09-14T06:34:09.663 INFO:teuthology.orchestra.run.smithi094.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42663.log.gz 2024-09-14T06:34:09.663 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70444.log 2024-09-14T06:34:09.663 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.69561.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.69561.log.gz 2024-09-14T06:34:09.663 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72467.log 2024-09-14T06:34:09.663 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.70444.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.70444.log.gz 2024-09-14T06:34:09.663 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45475.log 2024-09-14T06:34:09.663 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.72467.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.72467.log.gz 2024-09-14T06:34:09.663 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41824.log 2024-09-14T06:34:09.663 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.45475.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.45475.log.gz 2024-09-14T06:34:09.663 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67253.log 2024-09-14T06:34:09.664 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.41824.log: gzip 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41824.log.gz 2024-09-14T06:34:09.664 INFO:teuthology.orchestra.run.smithi094.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.45353.log 2024-09-14T06:34:09.664 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.67253.log: 0.0%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63297.log 2024-09-14T06:34:09.664 INFO:teuthology.orchestra.run.smithi094.stderr: -- replaced with /var/log/ceph/ceph-client.admin.67253.log.gz 2024-09-14T06:34:09.664 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.45353.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61534.log 2024-09-14T06:34:09.664 INFO:teuthology.orchestra.run.smithi094.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.45353.log.gz 2024-09-14T06:34:09.664 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41762.log 2024-09-14T06:34:09.664 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.63297.log: /var/log/ceph/ceph-client.admin.61534.log: 0.0%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50185.log 2024-09-14T06:34:09.664 INFO:teuthology.orchestra.run.smithi094.stderr: -- replaced with /var/log/ceph/ceph-client.admin.61534.log.gz 2024-09-14T06:34:09.664 INFO:teuthology.orchestra.run.smithi094.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.63297.log.gz 2024-09-14T06:34:09.664 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.41762.log: 0.0%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44468.log 2024-09-14T06:34:09.664 INFO:teuthology.orchestra.run.smithi094.stderr: -- replaced with /var/log/ceph/ceph-client.admin.41762.log.gz 2024-09-14T06:34:09.664 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.50185.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60859.log 2024-09-14T06:34:09.665 INFO:teuthology.orchestra.run.smithi094.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.50185.log.gz 2024-09-14T06:34:09.665 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66705.log 2024-09-14T06:34:09.665 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.44468.log: /var/log/ceph/ceph-client.admin.60859.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38355.log 2024-09-14T06:34:09.665 INFO:teuthology.orchestra.run.smithi094.stderr: 0.0% 0.0% -- replaced with /var/log/ceph/ceph-client.admin.60859.log.gz 2024-09-14T06:34:09.665 INFO:teuthology.orchestra.run.smithi094.stderr: -- replaced with /var/log/ceph/ceph-client.admin.44468.log.gz 2024-09-14T06:34:09.665 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.66705.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.66705.log.gz 2024-09-14T06:34:09.665 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39883.log 2024-09-14T06:34:09.665 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.38355.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38355.log.gz 2024-09-14T06:34:09.665 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37975.log 2024-09-14T06:34:09.665 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.39883.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39883.log.gz 2024-09-14T06:34:09.665 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73263.log 2024-09-14T06:34:09.666 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.37975.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37975.log.gz 2024-09-14T06:34:09.666 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71807.log 2024-09-14T06:34:09.666 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.73263.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.73263.log.gz 2024-09-14T06:34:09.666 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37636.log 2024-09-14T06:34:09.666 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.71807.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.71807.log.gz 2024-09-14T06:34:09.666 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39399.log 2024-09-14T06:34:09.666 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.37636.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37636.log.gz 2024-09-14T06:34:09.666 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73179.log 2024-09-14T06:34:09.666 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.39399.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39399.log.gz 2024-09-14T06:34:09.666 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44860.log 2024-09-14T06:34:09.666 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.73179.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.73179.log.gz 2024-09-14T06:34:09.667 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71289.log 2024-09-14T06:34:09.667 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.44860.log: gzip -5 --verbose 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44860.log.gz 2024-09-14T06:34:09.667 INFO:teuthology.orchestra.run.smithi094.stderr: -- /var/log/ceph/ceph-client.admin.48168.log 2024-09-14T06:34:09.667 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.71289.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.71289.log.gz 2024-09-14T06:34:09.667 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53879.log 2024-09-14T06:34:09.667 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.48168.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48168.log.gz 2024-09-14T06:34:09.667 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54802.log 2024-09-14T06:34:09.667 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.53879.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.53879.log.gz 2024-09-14T06:34:09.667 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33358.log 2024-09-14T06:34:09.667 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.54802.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.54802.log.gz 2024-09-14T06:34:09.667 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48829.log 2024-09-14T06:34:09.667 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.33358.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33358.log.gz 2024-09-14T06:34:09.667 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55311.log 2024-09-14T06:34:09.667 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.48829.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48829.log.gz 2024-09-14T06:34:09.668 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50040.log 2024-09-14T06:34:09.668 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.55311.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.55311.log.gz 2024-09-14T06:34:09.668 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38354.log 2024-09-14T06:34:09.668 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.50040.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.50040.log.gz 2024-09-14T06:34:09.668 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.tmp-client.admin.27534.log 2024-09-14T06:34:09.668 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.38354.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38354.log.gz 2024-09-14T06:34:09.668 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51527.log 2024-09-14T06:34:09.668 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph.tmp-client.admin.27534.log: 0.0% -- replaced with /var/log/ceph/ceph.tmp-client.admin.27534.log.gz 2024-09-14T06:34:09.669 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72801.log 2024-09-14T06:34:09.669 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.51527.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.51527.log.gz 2024-09-14T06:34:09.669 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51109.log 2024-09-14T06:34:09.669 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.72801.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.72801.log.gz 2024-09-14T06:34:09.669 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66628.log 2024-09-14T06:34:09.669 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.51109.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.51109.log.gz 2024-09-14T06:34:09.669 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69404.log 2024-09-14T06:34:09.669 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.66628.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.66628.log.gz 2024-09-14T06:34:09.669 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63701.log 2024-09-14T06:34:09.669 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.69404.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.69404.log.gz 2024-09-14T06:34:09.669 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71625.log 2024-09-14T06:34:09.669 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.63701.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.63701.log.gz 2024-09-14T06:34:09.669 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64948.log 2024-09-14T06:34:09.670 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.71625.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.71625.log.gz 2024-09-14T06:34:09.670 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37373.log 2024-09-14T06:34:09.670 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.64948.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.64948.log.gz 2024-09-14T06:34:09.670 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69665.log 2024-09-14T06:34:09.670 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.37373.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37373.log.gz 2024-09-14T06:34:09.670 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55087.log 2024-09-14T06:34:09.671 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.69665.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.69665.log.gz 2024-09-14T06:34:09.671 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71079.log 2024-09-14T06:34:09.671 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.55087.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.55087.log.gz 2024-09-14T06:34:09.671 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59904.log 2024-09-14T06:34:09.671 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.71079.log: gzip 0.0% -5 --verbose -- /var/log/ceph/ceph-client.admin.39342.log 2024-09-14T06:34:09.672 INFO:teuthology.orchestra.run.smithi094.stderr: -- replaced with /var/log/ceph/ceph-client.admin.71079.log.gz 2024-09-14T06:34:09.672 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.59904.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43831.log 2024-09-14T06:34:09.672 INFO:teuthology.orchestra.run.smithi094.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.59904.log.gz 2024-09-14T06:34:09.672 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44817.log 2024-09-14T06:34:09.672 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.39342.log: /var/log/ceph/ceph-client.admin.43831.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49153.log 2024-09-14T06:34:09.672 INFO:teuthology.orchestra.run.smithi094.stderr: 0.0% 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43831.log.gz 2024-09-14T06:34:09.672 INFO:teuthology.orchestra.run.smithi094.stderr: -- replaced with /var/log/ceph/ceph-client.admin.39342.log.gz 2024-09-14T06:34:09.672 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.44817.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44817.log.gz 2024-09-14T06:34:09.673 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43663.log 2024-09-14T06:34:09.673 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.49153.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49153.log.gz 2024-09-14T06:34:09.673 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49615.log 2024-09-14T06:34:09.673 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.43663.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43663.log.gz 2024-09-14T06:34:09.673 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70506.log 2024-09-14T06:34:09.673 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.49615.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49615.log.gz 2024-09-14T06:34:09.673 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53960.log 2024-09-14T06:34:09.673 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.70506.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.70506.log.gz 2024-09-14T06:34:09.673 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33757.log 2024-09-14T06:34:09.673 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.53960.log: 0.0%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72140.log 2024-09-14T06:34:09.674 INFO:teuthology.orchestra.run.smithi094.stderr: -- replaced with /var/log/ceph/ceph-client.admin.53960.log.gz 2024-09-14T06:34:09.674 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.33757.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45136.log 2024-09-14T06:34:09.674 INFO:teuthology.orchestra.run.smithi094.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33757.log.gz 2024-09-14T06:34:09.674 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37637.log 2024-09-14T06:34:09.674 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.72140.log: gzip/var/log/ceph/ceph-client.admin.45136.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.72140.log.gz 2024-09-14T06:34:09.674 INFO:teuthology.orchestra.run.smithi094.stderr: -5 --verbose 0.0% -- replaced with /var/log/ceph/ceph-client.admin.45136.log.gz 2024-09-14T06:34:09.674 INFO:teuthology.orchestra.run.smithi094.stderr: -- /var/log/ceph/ceph-client.admin.37853.log 2024-09-14T06:34:09.674 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69964.log 2024-09-14T06:34:09.675 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.37637.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37637.log.gz 2024-09-14T06:34:09.675 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70654.log 2024-09-14T06:34:09.675 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.37853.log: /var/log/ceph/ceph-client.admin.69964.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70121.log 2024-09-14T06:34:09.675 INFO:teuthology.orchestra.run.smithi094.stderr: 0.0% 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37853.log.gz 2024-09-14T06:34:09.675 INFO:teuthology.orchestra.run.smithi094.stderr: -- replaced with /var/log/ceph/ceph-client.admin.69964.log.gz 2024-09-14T06:34:09.675 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.70654.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.70654.log.gz 2024-09-14T06:34:09.675 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62275.log 2024-09-14T06:34:09.675 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.3.log 2024-09-14T06:34:09.675 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.70121.log: /var/log/ceph/ceph-client.admin.62275.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.70121.log.gz 2024-09-14T06:34:09.675 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46805.log 2024-09-14T06:34:09.675 INFO:teuthology.orchestra.run.smithi094.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.62275.log.gz 2024-09-14T06:34:09.675 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-osd.3.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49031.log 2024-09-14T06:34:09.675 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57511.log 2024-09-14T06:34:09.675 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.46805.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.46805.log.gz 2024-09-14T06:34:09.676 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.49031.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49031.log.gz 2024-09-14T06:34:09.676 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72843.log 2024-09-14T06:34:09.676 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71499.log 2024-09-14T06:34:09.676 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.57511.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.57511.log.gz 2024-09-14T06:34:09.676 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.72843.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.72843.log.gz 2024-09-14T06:34:09.676 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65464.log 2024-09-14T06:34:09.676 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47349.log 2024-09-14T06:34:09.676 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.71499.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.71499.log.gz 2024-09-14T06:34:09.676 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.65464.log: 0.0%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58049.log 2024-09-14T06:34:09.676 INFO:teuthology.orchestra.run.smithi094.stderr: -- replaced with /var/log/ceph/ceph-client.admin.65464.log.gz 2024-09-14T06:34:09.676 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61494.log 2024-09-14T06:34:09.676 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.47349.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.47349.log.gz 2024-09-14T06:34:09.677 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.58049.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.58049.log.gz 2024-09-14T06:34:09.677 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41699.log 2024-09-14T06:34:09.677 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44570.log 2024-09-14T06:34:09.677 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.61494.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.61494.log.gz 2024-09-14T06:34:09.677 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.41699.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41699.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63721.log 2024-09-14T06:34:09.677 INFO:teuthology.orchestra.run.smithi094.stderr: 2024-09-14T06:34:09.677 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64368.log 2024-09-14T06:34:09.677 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.44570.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44570.log.gz 2024-09-14T06:34:09.677 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.63721.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.63721.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46216.log 2024-09-14T06:34:09.678 INFO:teuthology.orchestra.run.smithi094.stderr: 2024-09-14T06:34:09.678 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33660.log 2024-09-14T06:34:09.678 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.64368.log: /var/log/ceph/ceph-client.admin.46216.log: 0.0% 0.0% -- replaced with /var/log/ceph/ceph-client.admin.64368.log.gz 2024-09-14T06:34:09.678 INFO:teuthology.orchestra.run.smithi094.stderr: -- replaced with /var/log/ceph/ceph-client.admin.46216.log.gz 2024-09-14T06:34:09.678 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71806.log 2024-09-14T06:34:09.678 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65187.log 2024-09-14T06:34:09.678 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.33660.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33660.log.gz 2024-09-14T06:34:09.678 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.71806.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.71806.log.gz 2024-09-14T06:34:09.678 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67819.log 2024-09-14T06:34:09.678 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39643.log 2024-09-14T06:34:09.678 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.65187.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.65187.log.gz 2024-09-14T06:34:09.678 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.67819.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.67819.log.gz 2024-09-14T06:34:09.678 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69882.log 2024-09-14T06:34:09.678 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72133.log 2024-09-14T06:34:09.679 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.39643.log: /var/log/ceph/ceph-client.admin.69882.log: 0.0% 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39643.log.gz 2024-09-14T06:34:09.679 INFO:teuthology.orchestra.run.smithi094.stderr: -- replaced with /var/log/ceph/ceph-client.admin.69882.log.gz 2024-09-14T06:34:09.679 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55534.log 2024-09-14T06:34:09.679 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.72133.log: 0.0%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37973.log 2024-09-14T06:34:09.679 INFO:teuthology.orchestra.run.smithi094.stderr: -- replaced with /var/log/ceph/ceph-client.admin.72133.log.gz 2024-09-14T06:34:09.679 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72141.log 2024-09-14T06:34:09.679 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.55534.log: /var/log/ceph/ceph-client.admin.37973.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.55534.log.gz 2024-09-14T06:34:09.679 INFO:teuthology.orchestra.run.smithi094.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37973.log.gz 2024-09-14T06:34:09.679 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/ceph-client.admin.72141.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.72141.log.gz 2024-09-14T06:34:09.683 INFO:teuthology.orchestra.run.smithi094.stderr: 90.9% -- replaced with /var/log/ceph/ceph-mon.c.log.gz 2024-09-14T06:34:09.715 INFO:teuthology.orchestra.run.smithi138.stderr: 90.9% -- replaced with /var/log/ceph/ceph-mon.b.log.gz 2024-09-14T06:34:09.916 INFO:teuthology.orchestra.run.smithi094.stderr: 90.3% -- replaced with /var/log/ceph/ceph-mon.a.log.gz 2024-09-14T06:34:10.126 INFO:teuthology.orchestra.run.smithi094.stderr: 93.8% -- replaced with /var/log/ceph/ceph-osd.2.log.gz 2024-09-14T06:34:10.964 INFO:teuthology.orchestra.run.smithi138.stderr: 95.0% -- replaced with /var/log/ceph/ceph-osd.6.log.gz 2024-09-14T06:34:12.029 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mgr.x has been restored 2024-09-14T06:34:12.775 INFO:teuthology.orchestra.run.smithi094.stderr: 94.7% -- replaced with /var/log/ceph/ceph-osd.1.log.gz 2024-09-14T06:34:13.562 INFO:teuthology.orchestra.run.smithi138.stderr: 94.9% -- replaced with /var/log/ceph/ceph-osd.5.log.gz 2024-09-14T06:34:13.998 INFO:teuthology.orchestra.run.smithi138.stderr: 95.1% -- replaced with /var/log/ceph/ceph-osd.4.log.gz 2024-09-14T06:34:14.208 INFO:teuthology.orchestra.run.smithi094.stderr: 95.3% -- replaced with /var/log/ceph/ceph-osd.3.log.gz 2024-09-14T06:34:14.553 INFO:teuthology.orchestra.run.smithi138.stderr: 95.4% -- replaced with /var/log/ceph/ceph-osd.7.log.gz 2024-09-14T06:34:14.556 INFO:teuthology.orchestra.run.smithi138.stderr: 2024-09-14T06:34:14.556 INFO:teuthology.orchestra.run.smithi138.stderr:real 0m5.117s 2024-09-14T06:34:14.556 INFO:teuthology.orchestra.run.smithi138.stderr:user 0m14.992s 2024-09-14T06:34:14.556 INFO:teuthology.orchestra.run.smithi138.stderr:sys 0m0.622s 2024-09-14T06:34:16.058 INFO:teuthology.orchestra.run.smithi094.stderr: 95.6% -- replaced with /var/log/ceph/ceph-osd.0.log.gz 2024-09-14T06:34:16.060 INFO:teuthology.orchestra.run.smithi094.stderr: 2024-09-14T06:34:16.060 INFO:teuthology.orchestra.run.smithi094.stderr:real 0m6.658s 2024-09-14T06:34:16.060 INFO:teuthology.orchestra.run.smithi094.stderr:user 0m15.415s 2024-09-14T06:34:16.060 INFO:teuthology.orchestra.run.smithi094.stderr:sys 0m0.849s 2024-09-14T06:34:16.060 INFO:tasks.ceph:Archiving logs... 2024-09-14T06:34:16.061 DEBUG:teuthology.misc:Transferring archived files from smithi094:/var/log/ceph to /home/teuthworker/archive/teuthology-2024-09-10_20:16:17-rbd-main-distro-default-smithi/7900381/remote/smithi094/log 2024-09-14T06:34:16.062 DEBUG:teuthology.orchestra.run.smithi094:> sudo tar c -f - -C /var/log/ceph -- . 2024-09-14T06:34:18.370 DEBUG:teuthology.misc:Transferring archived files from smithi138:/var/log/ceph to /home/teuthworker/archive/teuthology-2024-09-10_20:16:17-rbd-main-distro-default-smithi/7900381/remote/smithi138/log 2024-09-14T06:34:18.371 DEBUG:teuthology.orchestra.run.smithi138:> sudo tar c -f - -C /var/log/ceph -- . 2024-09-14T06:34:20.349 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-09-14T06:34:20.357 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-09-14T06:34:20.357 DEBUG:teuthology.orchestra.run.smithi094:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-09-14T06:34:20.360 DEBUG:teuthology.orchestra.run.smithi138:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-09-14T06:34:20.453 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-09-14T06:34:20.462 INFO:teuthology.task.clock:Checking final clock skew... 2024-09-14T06:34:20.462 DEBUG:teuthology.orchestra.run.smithi094:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-14T06:34:20.465 DEBUG:teuthology.orchestra.run.smithi138:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-14T06:34:20.479 INFO:teuthology.orchestra.run.smithi094.stderr:bash: line 1: ntpq: command not found 2024-09-14T06:34:20.482 INFO:teuthology.orchestra.run.smithi138.stderr:bash: line 1: ntpq: command not found 2024-09-14T06:34:20.483 INFO:teuthology.orchestra.run.smithi094.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-14T06:34:20.483 INFO:teuthology.orchestra.run.smithi094.stdout:=============================================================================== 2024-09-14T06:34:20.484 INFO:teuthology.orchestra.run.smithi094.stdout:^* hv01.front.sepia.ceph.com 3 6 377 15 -543us[ -545us] +/- 52ms 2024-09-14T06:34:20.484 INFO:teuthology.orchestra.run.smithi094.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 14 +663us[ +663us] +/- 57ms 2024-09-14T06:34:20.484 INFO:teuthology.orchestra.run.smithi094.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 14 +30us[ +30us] +/- 73ms 2024-09-14T06:34:20.484 INFO:teuthology.orchestra.run.smithi094.stdout:^? hv04.front.sepia.ceph.com 0 8 0 - +0ns[ +0ns] +/- 0ns 2024-09-14T06:34:20.488 INFO:teuthology.orchestra.run.smithi138.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-14T06:34:20.488 INFO:teuthology.orchestra.run.smithi138.stdout:=============================================================================== 2024-09-14T06:34:20.488 INFO:teuthology.orchestra.run.smithi138.stdout:^* hv01.front.sepia.ceph.com 3 6 377 14 -535us[ -532us] +/- 52ms 2024-09-14T06:34:20.488 INFO:teuthology.orchestra.run.smithi138.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 13 +670us[ +670us] +/- 57ms 2024-09-14T06:34:20.488 INFO:teuthology.orchestra.run.smithi138.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 14 +28us[ +32us] +/- 73ms 2024-09-14T06:34:20.488 INFO:teuthology.orchestra.run.smithi138.stdout:^? hv04.front.sepia.ceph.com 0 8 0 - +0ns[ +0ns] +/- 0ns 2024-09-14T06:34:20.489 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-09-14T06:34:20.498 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-09-14T06:34:20.498 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-09-14T06:34:20.507 DEBUG:teuthology.orchestra.run.smithi094:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-09-14T06:34:20.616 DEBUG:teuthology.orchestra.run.smithi138:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-09-14T06:34:20.701 DEBUG:teuthology.orchestra.run.smithi094:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 2024-09-14T06:34:20.731 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-14T06:34:20.732 DEBUG:teuthology.orchestra.run.smithi138:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 2024-09-14T06:34:20.761 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-14T06:34:20.761 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-09-14T06:34:20.770 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-09-14T06:34:20.778 INFO:teuthology.task.internal:Duration was 762.008645 seconds 2024-09-14T06:34:20.779 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-09-14T06:34:20.787 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-09-14T06:34:20.787 DEBUG:teuthology.orchestra.run.smithi094:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-09-14T06:34:20.790 DEBUG:teuthology.orchestra.run.smithi138:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-09-14T06:34:20.826 INFO:teuthology.orchestra.run.smithi094.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-14T06:34:20.841 INFO:teuthology.orchestra.run.smithi138.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-14T06:34:21.217 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-09-14T06:34:21.217 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi094.front.sepia.ceph.com 2024-09-14T06:34:21.217 DEBUG:teuthology.orchestra.run.smithi094:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-09-14T06:34:21.277 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi138.front.sepia.ceph.com 2024-09-14T06:34:21.277 DEBUG:teuthology.orchestra.run.smithi138:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-09-14T06:34:21.303 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-09-14T06:34:21.303 DEBUG:teuthology.orchestra.run.smithi094:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-14T06:34:21.319 DEBUG:teuthology.orchestra.run.smithi138:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-14T06:34:21.370 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-09-14T06:34:21.370 DEBUG:teuthology.orchestra.run.smithi094:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-09-14T06:34:21.385 DEBUG:teuthology.orchestra.run.smithi138:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-09-14T06:34:21.539 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-09-14T06:34:21.549 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-09-14T06:34:21.549 DEBUG:teuthology.orchestra.run.smithi094:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-09-14T06:34:21.574 DEBUG:teuthology.orchestra.run.smithi138:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-09-14T06:34:21.598 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-09-14T06:34:21.609 DEBUG:teuthology.orchestra.run.smithi094:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-09-14T06:34:21.616 DEBUG:teuthology.orchestra.run.smithi138:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-09-14T06:34:21.637 INFO:teuthology.orchestra.run.smithi094.stdout:kernel.core_pattern = core 2024-09-14T06:34:21.662 INFO:teuthology.orchestra.run.smithi138.stdout:kernel.core_pattern = core 2024-09-14T06:34:21.711 DEBUG:teuthology.orchestra.run.smithi094:> test -e /home/ubuntu/cephtest/archive/coredump 2024-09-14T06:34:21.726 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-14T06:34:21.726 DEBUG:teuthology.orchestra.run.smithi138:> test -e /home/ubuntu/cephtest/archive/coredump 2024-09-14T06:34:21.766 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-14T06:34:21.766 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-09-14T06:34:21.777 INFO:teuthology.task.internal:Transferring archived files... 2024-09-14T06:34:21.777 DEBUG:teuthology.misc:Transferring archived files from smithi094:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/teuthology-2024-09-10_20:16:17-rbd-main-distro-default-smithi/7900381/remote/smithi094 2024-09-14T06:34:21.778 DEBUG:teuthology.orchestra.run.smithi094:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-09-14T06:34:21.818 DEBUG:teuthology.misc:Transferring archived files from smithi138:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/teuthology-2024-09-10_20:16:17-rbd-main-distro-default-smithi/7900381/remote/smithi138 2024-09-14T06:34:21.818 DEBUG:teuthology.orchestra.run.smithi138:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-09-14T06:34:21.853 INFO:teuthology.task.internal:Removing archive directory... 2024-09-14T06:34:21.853 DEBUG:teuthology.orchestra.run.smithi094:> rm -rf -- /home/ubuntu/cephtest/archive 2024-09-14T06:34:21.855 DEBUG:teuthology.orchestra.run.smithi138:> rm -rf -- /home/ubuntu/cephtest/archive 2024-09-14T06:34:21.902 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-09-14T06:34:21.914 INFO:teuthology.task.internal:Not uploading archives. 2024-09-14T06:34:21.914 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-09-14T06:34:21.924 INFO:teuthology.task.internal:Tidying up after the test... 2024-09-14T06:34:21.924 DEBUG:teuthology.orchestra.run.smithi094:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-09-14T06:34:21.926 DEBUG:teuthology.orchestra.run.smithi138:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-09-14T06:34:21.939 INFO:teuthology.orchestra.run.smithi094.stdout: 265324 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 14 06:34 /home/ubuntu/cephtest 2024-09-14T06:34:21.957 INFO:teuthology.orchestra.run.smithi138.stdout: 265324 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 14 06:34 /home/ubuntu/cephtest 2024-09-14T06:34:21.958 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-09-14T06:34:21.969 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-09-14T06:34:21.990 INFO:teuthology.run:Summary data: description: rbd/thrash/{base/install clusters/{fixed-2 openstack} conf/{disable-pool-app} msgr-failures/few objectstore/bluestore-comp-snappy supported-random-distro$/{centos_latest} thrashers/default thrashosds-health workloads/rbd_fsx_deep_copy} duration: 762.0086450576782 flavor: default owner: scheduled_teuthology@teuthology success: true 2024-09-14T06:34:21.990 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-09-14T06:34:22.073 INFO:teuthology.run:pass